2024-08-30T08:44:05.981 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-08-30T08:44:06.007 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-08-30T08:44:06.143 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/mchangir-2024-08-30_08:30:37-fs-wip-mchangir-testing-20240828.085847-main-debug-distro-default-smithi/7881205 branch: wip-mchangir-testing-20240828.085847-main-debug description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/secure wsync/no} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-quiesce/no 6-workunit/postgres}} email: mchangir@redhat.com first_in_suite: false job_id: '7881205' kernel: client: sha1: distro kdb: 1 sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: mchangir-2024-08-30_08:30:37-fs-wip-mchangir-testing-20240828.085847-main-debug-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-mchangir-testing-20240828.085847-main-debug ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd max_mds: 3 session_timeout: 300 standby_replay: true cluster-conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 2 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mon: mon op complaint time: 120 mon warn on pool no app: false osd: osd op complaint time: 180 conf: global: ms die on skipped message: false mgr: cephsqlite lock renewal timeout: 900000 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - fs.*is degraded - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - insufficient standby MDS daemons available - MDS_UP_LESS_THAN_MAX - online, but wants - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy - slow metadata IO - SLOW_OPS - slow request sha1: d19e6118f7015f0047264006cb0c873528eb3ae6 subvols: create: 2 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: mntopts: - context=system_u:object_r:postgresql_db_t:s0 mount_subvol_num: 0 mountpoint: /tmp/cephfs check-counter: counters: mds: - mds.exported - mds.imported install: ceph: flavor: default sha1: d19e6118f7015f0047264006cb0c873528eb3ae6 kclient: client.0: mntopts: - context=system_u:object_r:postgresql_db_t:s0 mount_subvol_num: 1 mountpoint: /tmp/cephfs mntopts: - ms_mode=secure - nowsync syntax: v2 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-mchangir-testing-20240828.085847-main-debug sha1: d19e6118f7015f0047264006cb0c873528eb3ae6 owner: scheduled_mchangir@teuthology priority: 150 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 5413 sha1: d19e6118f7015f0047264006cb0c873528eb3ae6 sleep_before_teardown: 0 subset: 347/512 suite: fs suite_branch: wip-mchangir-testing-20240828.085847-main-debug suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_d19e6118f7015f0047264006cb0c873528eb3ae6/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: d19e6118f7015f0047264006cb0c873528eb3ae6 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi042.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHz0JdoPZxNGJpHOO4Xjsc/B5gZGUbnLmbI9ZP8Yk/3MJcBEfM/dJ+Sac+AZYiStbtpQBc9q4Pwy+eMa0gKqk1Q= smithi167.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJYhTbAsjRMotN6o/OYPdF5EWup5KrADRWl6tK0V0rhrn+sOENJJQA/t0M0B1ciUX1GQw+43NVIqOwAFNxQnPeY= smithi190.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNl+kDD4dGw0/7+8qAMt09IuDzAe1l4B0GIk9chiBiyh/qSslBYc2Ki7tPWUPaIARPRD7po1UNjQ+0O7dVCMhPo= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate true - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - exec: client.0: - sudo ls -l /tmp/cephfs/ && sudo df -h /tmp/cephfs/ - sudo mkdir -m 755 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres && sudo chown postgres:postgres /tmp/cephfs/postgres - sudo -u postgres -- mkdir -m 700 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres/data - sudo semanage fcontext -a -t postgresql_db_t "/tmp/cephfs/postgres(/.*)?" - sudo grep -i postgresql /etc/selinux/targeted/contexts/files/file_contexts.local - sudo restorecon -R -v /tmp/cephfs/postgres - sudo ls -lZaR /tmp/cephfs/postgres/ - sudo mkdir -p /etc/systemd/system/postgresql.service.d/ && printf '[Service]\nEnvironment=PGDATA=/tmp/cephfs/postgres/data\nEnvironment=PGLOG=/tmp/cephfs/postgres/pgstartup.log\n' | sudo tee /etc/systemd/system/postgresql.service.d/env.conf - sudo -u postgres -- postgresql-setup --initdb - sudo ls -lZaR /tmp/cephfs/postgres/ - sudo systemctl start postgresql - sudo -u postgres -- pgbench -s 32 -i - sudo -u postgres -- pgbench -c 100 -j 4 --progress=5 --time=900 - sudo systemctl stop postgresql - sudo ls -lZaR /tmp/cephfs/postgres/ teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-08-30_08:30:37 tube: smithi user: mchangir verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-08-30T08:44:06.143 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_d19e6118f7015f0047264006cb0c873528eb3ae6/qa; will attempt to use it 2024-08-30T08:44:06.144 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_d19e6118f7015f0047264006cb0c873528eb3ae6/qa/tasks 2024-08-30T08:44:06.144 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-08-30T08:44:06.145 INFO:teuthology.task.internal:Checking packages... 2024-08-30T08:44:06.170 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'd19e6118f7015f0047264006cb0c873528eb3ae6' 2024-08-30T08:44:06.170 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-08-30T08:44:06.171 INFO:teuthology.packaging:ref: None 2024-08-30T08:44:06.171 INFO:teuthology.packaging:tag: None 2024-08-30T08:44:06.171 INFO:teuthology.packaging:branch: wip-mchangir-testing-20240828.085847-main-debug 2024-08-30T08:44:06.171 INFO:teuthology.packaging:sha1: d19e6118f7015f0047264006cb0c873528eb3ae6 2024-08-30T08:44:06.171 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-mchangir-testing-20240828.085847-main-debug 2024-08-30T08:44:06.322 INFO:teuthology.task.internal:Found packages for ceph version 19.3.0-4560.gd19e6118 2024-08-30T08:44:06.324 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-08-30T08:44:06.330 INFO:teuthology.task.internal:no buildpackages task found 2024-08-30T08:44:06.330 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-08-30T08:44:06.374 INFO:teuthology.task.internal:Saving configuration 2024-08-30T08:44:06.395 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-08-30T08:44:06.402 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-08-30T08:44:06.437 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi042.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/mchangir-2024-08-30_08:30:37-fs-wip-mchangir-testing-20240828.085847-main-debug-distro-default-smithi/7881205', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-08-30 08:39:05.378301', 'locked_by': 'scheduled_mchangir@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHz0JdoPZxNGJpHOO4Xjsc/B5gZGUbnLmbI9ZP8Yk/3MJcBEfM/dJ+Sac+AZYiStbtpQBc9q4Pwy+eMa0gKqk1Q='} 2024-08-30T08:44:06.469 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi167.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/mchangir-2024-08-30_08:30:37-fs-wip-mchangir-testing-20240828.085847-main-debug-distro-default-smithi/7881205', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-08-30 08:39:05.375254', 'locked_by': 'scheduled_mchangir@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJYhTbAsjRMotN6o/OYPdF5EWup5KrADRWl6tK0V0rhrn+sOENJJQA/t0M0B1ciUX1GQw+43NVIqOwAFNxQnPeY='} 2024-08-30T08:44:06.500 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi190.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/mchangir-2024-08-30_08:30:37-fs-wip-mchangir-testing-20240828.085847-main-debug-distro-default-smithi/7881205', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-08-30 08:39:05.377324', 'locked_by': 'scheduled_mchangir@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNl+kDD4dGw0/7+8qAMt09IuDzAe1l4B0GIk9chiBiyh/qSslBYc2Ki7tPWUPaIARPRD7po1UNjQ+0O7dVCMhPo='} 2024-08-30T08:44:06.501 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-08-30T08:44:06.507 INFO:teuthology.task.internal:roles: ubuntu@smithi042.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-08-30T08:44:06.507 INFO:teuthology.task.internal:roles: ubuntu@smithi167.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-08-30T08:44:06.508 INFO:teuthology.task.internal:roles: ubuntu@smithi190.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-08-30T08:44:06.508 INFO:teuthology.run_tasks:Running task console_log... 2024-08-30T08:44:06.618 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f6edf503ac0>, signals=[15]) 2024-08-30T08:44:06.618 INFO:teuthology.run_tasks:Running task internal.connect... 2024-08-30T08:44:06.624 INFO:teuthology.task.internal:Opening connections... 2024-08-30T08:44:06.625 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi042.front.sepia.ceph.com 2024-08-30T08:44:06.626 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi042.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-08-30T08:44:06.712 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi167.front.sepia.ceph.com 2024-08-30T08:44:06.713 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-08-30T08:44:06.796 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi190.front.sepia.ceph.com 2024-08-30T08:44:06.796 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi190.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-08-30T08:44:06.879 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-08-30T08:44:06.886 DEBUG:teuthology.orchestra.run.smithi042:> uname -m 2024-08-30T08:44:06.905 INFO:teuthology.orchestra.run.smithi042.stdout:x86_64 2024-08-30T08:44:06.905 DEBUG:teuthology.orchestra.run.smithi042:> cat /etc/os-release 2024-08-30T08:44:06.963 INFO:teuthology.orchestra.run.smithi042.stdout:NAME="CentOS Stream" 2024-08-30T08:44:06.963 INFO:teuthology.orchestra.run.smithi042.stdout:VERSION="9" 2024-08-30T08:44:06.963 INFO:teuthology.orchestra.run.smithi042.stdout:ID="centos" 2024-08-30T08:44:06.963 INFO:teuthology.orchestra.run.smithi042.stdout:ID_LIKE="rhel fedora" 2024-08-30T08:44:06.963 INFO:teuthology.orchestra.run.smithi042.stdout:VERSION_ID="9" 2024-08-30T08:44:06.963 INFO:teuthology.orchestra.run.smithi042.stdout:PLATFORM_ID="platform:el9" 2024-08-30T08:44:06.963 INFO:teuthology.orchestra.run.smithi042.stdout:PRETTY_NAME="CentOS Stream 9" 2024-08-30T08:44:06.964 INFO:teuthology.orchestra.run.smithi042.stdout:ANSI_COLOR="0;31" 2024-08-30T08:44:06.964 INFO:teuthology.orchestra.run.smithi042.stdout:LOGO="fedora-logo-icon" 2024-08-30T08:44:06.964 INFO:teuthology.orchestra.run.smithi042.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-08-30T08:44:06.964 INFO:teuthology.orchestra.run.smithi042.stdout:HOME_URL="https://centos.org/" 2024-08-30T08:44:06.964 INFO:teuthology.orchestra.run.smithi042.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-08-30T08:44:06.964 INFO:teuthology.orchestra.run.smithi042.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-08-30T08:44:06.964 INFO:teuthology.orchestra.run.smithi042.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-08-30T08:44:06.964 INFO:teuthology.lock.ops:Updating smithi042.front.sepia.ceph.com on lock server 2024-08-30T08:44:06.999 DEBUG:teuthology.orchestra.run.smithi167:> uname -m 2024-08-30T08:44:07.018 INFO:teuthology.orchestra.run.smithi167.stdout:x86_64 2024-08-30T08:44:07.018 DEBUG:teuthology.orchestra.run.smithi167:> cat /etc/os-release 2024-08-30T08:44:07.076 INFO:teuthology.orchestra.run.smithi167.stdout:NAME="CentOS Stream" 2024-08-30T08:44:07.076 INFO:teuthology.orchestra.run.smithi167.stdout:VERSION="9" 2024-08-30T08:44:07.076 INFO:teuthology.orchestra.run.smithi167.stdout:ID="centos" 2024-08-30T08:44:07.076 INFO:teuthology.orchestra.run.smithi167.stdout:ID_LIKE="rhel fedora" 2024-08-30T08:44:07.077 INFO:teuthology.orchestra.run.smithi167.stdout:VERSION_ID="9" 2024-08-30T08:44:07.077 INFO:teuthology.orchestra.run.smithi167.stdout:PLATFORM_ID="platform:el9" 2024-08-30T08:44:07.077 INFO:teuthology.orchestra.run.smithi167.stdout:PRETTY_NAME="CentOS Stream 9" 2024-08-30T08:44:07.077 INFO:teuthology.orchestra.run.smithi167.stdout:ANSI_COLOR="0;31" 2024-08-30T08:44:07.077 INFO:teuthology.orchestra.run.smithi167.stdout:LOGO="fedora-logo-icon" 2024-08-30T08:44:07.077 INFO:teuthology.orchestra.run.smithi167.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-08-30T08:44:07.077 INFO:teuthology.orchestra.run.smithi167.stdout:HOME_URL="https://centos.org/" 2024-08-30T08:44:07.077 INFO:teuthology.orchestra.run.smithi167.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-08-30T08:44:07.077 INFO:teuthology.orchestra.run.smithi167.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-08-30T08:44:07.077 INFO:teuthology.orchestra.run.smithi167.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-08-30T08:44:07.077 INFO:teuthology.lock.ops:Updating smithi167.front.sepia.ceph.com on lock server 2024-08-30T08:44:07.099 DEBUG:teuthology.orchestra.run.smithi190:> uname -m 2024-08-30T08:44:07.117 INFO:teuthology.orchestra.run.smithi190.stdout:x86_64 2024-08-30T08:44:07.117 DEBUG:teuthology.orchestra.run.smithi190:> cat /etc/os-release 2024-08-30T08:44:07.175 INFO:teuthology.orchestra.run.smithi190.stdout:NAME="CentOS Stream" 2024-08-30T08:44:07.176 INFO:teuthology.orchestra.run.smithi190.stdout:VERSION="9" 2024-08-30T08:44:07.176 INFO:teuthology.orchestra.run.smithi190.stdout:ID="centos" 2024-08-30T08:44:07.176 INFO:teuthology.orchestra.run.smithi190.stdout:ID_LIKE="rhel fedora" 2024-08-30T08:44:07.176 INFO:teuthology.orchestra.run.smithi190.stdout:VERSION_ID="9" 2024-08-30T08:44:07.176 INFO:teuthology.orchestra.run.smithi190.stdout:PLATFORM_ID="platform:el9" 2024-08-30T08:44:07.176 INFO:teuthology.orchestra.run.smithi190.stdout:PRETTY_NAME="CentOS Stream 9" 2024-08-30T08:44:07.176 INFO:teuthology.orchestra.run.smithi190.stdout:ANSI_COLOR="0;31" 2024-08-30T08:44:07.176 INFO:teuthology.orchestra.run.smithi190.stdout:LOGO="fedora-logo-icon" 2024-08-30T08:44:07.176 INFO:teuthology.orchestra.run.smithi190.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-08-30T08:44:07.176 INFO:teuthology.orchestra.run.smithi190.stdout:HOME_URL="https://centos.org/" 2024-08-30T08:44:07.176 INFO:teuthology.orchestra.run.smithi190.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-08-30T08:44:07.176 INFO:teuthology.orchestra.run.smithi190.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-08-30T08:44:07.176 INFO:teuthology.orchestra.run.smithi190.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-08-30T08:44:07.177 INFO:teuthology.lock.ops:Updating smithi190.front.sepia.ceph.com on lock server 2024-08-30T08:44:07.207 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-08-30T08:44:07.218 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-08-30T08:44:07.263 INFO:teuthology.task.internal:Checking for old test directory... 2024-08-30T08:44:07.263 DEBUG:teuthology.orchestra.run.smithi042:> test '!' -e /home/ubuntu/cephtest 2024-08-30T08:44:07.266 DEBUG:teuthology.orchestra.run.smithi167:> test '!' -e /home/ubuntu/cephtest 2024-08-30T08:44:07.268 DEBUG:teuthology.orchestra.run.smithi190:> test '!' -e /home/ubuntu/cephtest 2024-08-30T08:44:07.285 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-08-30T08:44:07.291 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-08-30T08:44:07.291 DEBUG:teuthology.orchestra.run.smithi042:> test -z $(ls -A /var/lib/ceph) 2024-08-30T08:44:07.323 DEBUG:teuthology.orchestra.run.smithi167:> test -z $(ls -A /var/lib/ceph) 2024-08-30T08:44:07.325 DEBUG:teuthology.orchestra.run.smithi190:> test -z $(ls -A /var/lib/ceph) 2024-08-30T08:44:07.436 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-08-30T08:44:07.539 INFO:teuthology.run_tasks:Running task kernel... 2024-08-30T08:44:07.553 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': 1, 'sha1': 'distro'} 2024-08-30T08:44:07.553 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2024-08-30T08:44:07.553 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2024-08-30T08:44:07.554 DEBUG:teuthology.orchestra.run.smithi042:> test -f /run/.containerenv -o -f /.dockerenv 2024-08-30T08:44:07.570 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-30T08:44:07.571 DEBUG:teuthology.orchestra.run.smithi042:> uname -r 2024-08-30T08:44:07.629 INFO:teuthology.orchestra.run.smithi042.stdout:5.14.0-503.el9.x86_64 2024-08-30T08:44:07.629 INFO:teuthology.task.kernel:Running kernel on smithi042: 5.14.0-503.el9.x86_64 2024-08-30T08:44:07.629 DEBUG:teuthology.orchestra.run.smithi042:> sudo yum install -y kernel 2024-08-30T08:44:11.148 INFO:teuthology.orchestra.run.smithi042.stdout:CentOS Stream 9 - BaseOS 4.7 MB/s | 8.2 MB 00:01 2024-08-30T08:44:16.790 INFO:teuthology.orchestra.run.smithi042.stdout:CentOS Stream 9 - AppStream 4.5 MB/s | 20 MB 00:04 2024-08-30T08:44:22.166 INFO:teuthology.orchestra.run.smithi042.stdout:CentOS Stream 9 - CRB 4.4 MB/s | 6.6 MB 00:01 2024-08-30T08:44:23.925 INFO:teuthology.orchestra.run.smithi042.stdout:CentOS Stream 9 - Extras packages 37 kB/s | 18 kB 00:00 2024-08-30T08:44:24.862 INFO:teuthology.orchestra.run.smithi042.stdout:Extra Packages for Enterprise Linux 28 MB/s | 22 MB 00:00 2024-08-30T08:44:30.477 INFO:teuthology.orchestra.run.smithi042.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-08-30T08:44:32.567 INFO:teuthology.orchestra.run.smithi042.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-08-30T08:44:32.567 INFO:teuthology.orchestra.run.smithi042.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-08-30T08:44:32.653 INFO:teuthology.orchestra.run.smithi042.stdout:Dependencies resolved. 2024-08-30T08:44:32.654 INFO:teuthology.orchestra.run.smithi042.stdout:Nothing to do. 2024-08-30T08:44:32.654 INFO:teuthology.orchestra.run.smithi042.stdout:Complete! 2024-08-30T08:44:32.763 DEBUG:teuthology.orchestra.run.smithi042:> echo no | sudo yum reinstall kernel || true 2024-08-30T08:44:33.313 INFO:teuthology.orchestra.run.smithi042.stdout:Last metadata expiration check: 0:00:03 ago on Fri 30 Aug 2024 08:44:30 AM UTC. 2024-08-30T08:44:33.454 INFO:teuthology.orchestra.run.smithi042.stdout:Dependencies resolved. 2024-08-30T08:44:33.466 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-08-30T08:44:33.466 INFO:teuthology.orchestra.run.smithi042.stdout: Package Architecture Version Repository Size 2024-08-30T08:44:33.466 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-08-30T08:44:33.466 INFO:teuthology.orchestra.run.smithi042.stdout:Reinstalling: 2024-08-30T08:44:33.466 INFO:teuthology.orchestra.run.smithi042.stdout: kernel x86_64 5.14.0-503.el9 baseos 1.9 M 2024-08-30T08:44:33.466 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-08-30T08:44:33.466 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction Summary 2024-08-30T08:44:33.466 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-08-30T08:44:33.466 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-08-30T08:44:33.467 INFO:teuthology.orchestra.run.smithi042.stdout:Total download size: 1.9 M 2024-08-30T08:44:33.467 INFO:teuthology.orchestra.run.smithi042.stdout:Installed size: 0 2024-08-30T08:44:33.467 INFO:teuthology.orchestra.run.smithi042.stderr:Operation aborted. 2024-08-30T08:44:33.529 INFO:teuthology.orchestra.run.smithi042.stdout:Is this ok [y/N]: 2024-08-30T08:44:33.529 DEBUG:teuthology.orchestra.run.smithi042:> sudo yum reinstall -y kernel || true 2024-08-30T08:44:34.074 INFO:teuthology.orchestra.run.smithi042.stdout:Last metadata expiration check: 0:00:04 ago on Fri 30 Aug 2024 08:44:30 AM UTC. 2024-08-30T08:44:34.207 INFO:teuthology.orchestra.run.smithi042.stdout:Dependencies resolved. 2024-08-30T08:44:34.207 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-08-30T08:44:34.207 INFO:teuthology.orchestra.run.smithi042.stdout: Package Architecture Version Repository Size 2024-08-30T08:44:34.207 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-08-30T08:44:34.207 INFO:teuthology.orchestra.run.smithi042.stdout:Reinstalling: 2024-08-30T08:44:34.207 INFO:teuthology.orchestra.run.smithi042.stdout: kernel x86_64 5.14.0-503.el9 baseos 1.9 M 2024-08-30T08:44:34.207 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-08-30T08:44:34.207 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction Summary 2024-08-30T08:44:34.208 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-08-30T08:44:34.208 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-08-30T08:44:34.208 INFO:teuthology.orchestra.run.smithi042.stdout:Total download size: 1.9 M 2024-08-30T08:44:34.208 INFO:teuthology.orchestra.run.smithi042.stdout:Installed size: 0 2024-08-30T08:44:34.208 INFO:teuthology.orchestra.run.smithi042.stdout:Downloading Packages: 2024-08-30T08:44:35.354 INFO:teuthology.orchestra.run.smithi042.stdout:kernel-5.14.0-503.el9.x86_64.rpm 1.7 MB/s | 1.9 MB 00:01 2024-08-30T08:44:35.354 INFO:teuthology.orchestra.run.smithi042.stdout:-------------------------------------------------------------------------------- 2024-08-30T08:44:35.354 INFO:teuthology.orchestra.run.smithi042.stdout:Total 1.6 MB/s | 1.9 MB 00:01 2024-08-30T08:44:35.386 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction check 2024-08-30T08:44:35.639 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction check succeeded. 2024-08-30T08:44:35.639 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction test 2024-08-30T08:44:35.670 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction test succeeded. 2024-08-30T08:44:35.671 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction 2024-08-30T08:44:35.831 INFO:teuthology.orchestra.run.smithi042.stdout: Preparing : 1/1 2024-08-30T08:44:35.931 INFO:teuthology.orchestra.run.smithi042.stdout: Reinstalling : kernel-5.14.0-503.el9.x86_64 1/2 2024-08-30T08:44:35.931 INFO:teuthology.orchestra.run.smithi042.stdout: Cleanup : kernel-5.14.0-503.el9.x86_64 2/2 2024-08-30T08:44:36.660 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: kernel-5.14.0-503.el9.x86_64 2/2 2024-08-30T08:44:36.660 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : kernel-5.14.0-503.el9.x86_64 1/2 2024-08-30T08:44:37.375 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : kernel-5.14.0-503.el9.x86_64 2/2 2024-08-30T08:44:37.375 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-08-30T08:44:37.375 INFO:teuthology.orchestra.run.smithi042.stdout:Reinstalled: 2024-08-30T08:44:37.375 INFO:teuthology.orchestra.run.smithi042.stdout: kernel-5.14.0-503.el9.x86_64 2024-08-30T08:44:37.375 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-08-30T08:44:37.375 INFO:teuthology.orchestra.run.smithi042.stdout:Complete! 2024-08-30T08:44:37.567 DEBUG:teuthology.orchestra.run.smithi042:> rpm -q kernel | sort -rV | head -n 1 2024-08-30T08:44:37.626 INFO:teuthology.orchestra.run.smithi042.stdout:kernel-5.14.0-503.el9.x86_64 2024-08-30T08:44:37.626 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-503.el9.x86_64 2024-08-30T08:44:37.626 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-08-30T08:44:37.627 DEBUG:teuthology.parallel:result is None 2024-08-30T08:44:37.627 INFO:teuthology.run_tasks:Running task internal.base... 2024-08-30T08:44:37.635 INFO:teuthology.task.internal:Creating test directory... 2024-08-30T08:44:37.635 DEBUG:teuthology.orchestra.run.smithi042:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-08-30T08:44:37.638 DEBUG:teuthology.orchestra.run.smithi167:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-08-30T08:44:37.640 DEBUG:teuthology.orchestra.run.smithi190:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-08-30T08:44:37.659 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-08-30T08:44:37.665 INFO:teuthology.run_tasks:Running task internal.archive... 2024-08-30T08:44:37.672 INFO:teuthology.task.internal:Creating archive directory... 2024-08-30T08:44:37.672 DEBUG:teuthology.orchestra.run.smithi042:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-08-30T08:44:37.693 DEBUG:teuthology.orchestra.run.smithi167:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-08-30T08:44:37.700 DEBUG:teuthology.orchestra.run.smithi190:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-08-30T08:44:37.805 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-08-30T08:44:37.814 INFO:teuthology.task.internal:Enabling coredump saving... 2024-08-30T08:44:37.814 DEBUG:teuthology.orchestra.run.smithi167:> test -f /run/.containerenv -o -f /.dockerenv 2024-08-30T08:44:37.831 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-30T08:44:37.832 DEBUG:teuthology.orchestra.run.smithi190:> test -f /run/.containerenv -o -f /.dockerenv 2024-08-30T08:44:37.849 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-30T08:44:37.850 DEBUG:teuthology.orchestra.run.smithi042:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-08-30T08:44:37.852 DEBUG:teuthology.orchestra.run.smithi167:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-08-30T08:44:37.875 DEBUG:teuthology.orchestra.run.smithi190:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-08-30T08:44:37.909 INFO:teuthology.orchestra.run.smithi042.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-08-30T08:44:37.925 INFO:teuthology.orchestra.run.smithi042.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-08-30T08:44:37.936 INFO:teuthology.orchestra.run.smithi167.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-08-30T08:44:37.940 INFO:teuthology.orchestra.run.smithi190.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-08-30T08:44:37.955 INFO:teuthology.orchestra.run.smithi167.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-08-30T08:44:37.956 INFO:teuthology.orchestra.run.smithi190.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-08-30T08:44:37.958 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-08-30T08:44:37.965 INFO:teuthology.task.internal:Configuring sudo... 2024-08-30T08:44:37.966 DEBUG:teuthology.orchestra.run.smithi042:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-08-30T08:44:37.968 DEBUG:teuthology.orchestra.run.smithi167:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-08-30T08:44:37.970 DEBUG:teuthology.orchestra.run.smithi190:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-08-30T08:44:37.998 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-08-30T08:44:38.007 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-08-30T08:44:38.007 DEBUG:teuthology.orchestra.run.smithi042:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-08-30T08:44:38.034 DEBUG:teuthology.orchestra.run.smithi167:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-08-30T08:44:38.038 DEBUG:teuthology.orchestra.run.smithi190:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-08-30T08:44:38.056 DEBUG:teuthology.orchestra.run.smithi042:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-08-30T08:44:38.135 DEBUG:teuthology.orchestra.run.smithi042:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-08-30T08:44:38.174 DEBUG:teuthology.orchestra.run.smithi042:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-08-30T08:44:38.262 DEBUG:teuthology.orchestra.run.smithi042:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-08-30T08:44:38.297 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-08-30T08:44:38.297 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-08-30T08:44:38.361 DEBUG:teuthology.orchestra.run.smithi167:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-08-30T08:44:38.414 DEBUG:teuthology.orchestra.run.smithi167:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-08-30T08:44:38.464 DEBUG:teuthology.orchestra.run.smithi167:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-08-30T08:44:38.556 DEBUG:teuthology.orchestra.run.smithi167:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-08-30T08:44:38.591 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-08-30T08:44:38.591 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-08-30T08:44:38.658 DEBUG:teuthology.orchestra.run.smithi190:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-08-30T08:44:38.711 DEBUG:teuthology.orchestra.run.smithi190:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-08-30T08:44:38.751 DEBUG:teuthology.orchestra.run.smithi190:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-08-30T08:44:38.841 DEBUG:teuthology.orchestra.run.smithi190:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-08-30T08:44:38.878 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-08-30T08:44:38.878 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-08-30T08:44:38.944 DEBUG:teuthology.orchestra.run.smithi042:> sudo service rsyslog restart 2024-08-30T08:44:38.947 DEBUG:teuthology.orchestra.run.smithi167:> sudo service rsyslog restart 2024-08-30T08:44:38.949 DEBUG:teuthology.orchestra.run.smithi190:> sudo service rsyslog restart 2024-08-30T08:44:38.983 INFO:teuthology.orchestra.run.smithi042.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-08-30T08:44:38.988 INFO:teuthology.orchestra.run.smithi167.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-08-30T08:44:39.022 INFO:teuthology.orchestra.run.smithi190.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-08-30T08:44:39.475 INFO:teuthology.run_tasks:Running task internal.timer... 2024-08-30T08:44:39.482 INFO:teuthology.task.internal:Starting timer... 2024-08-30T08:44:39.482 INFO:teuthology.run_tasks:Running task pcp... 2024-08-30T08:44:39.596 INFO:teuthology.run_tasks:Running task selinux... 2024-08-30T08:44:39.605 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0']} 2024-08-30T08:44:39.605 DEBUG:teuthology.orchestra.run.smithi042:> sudo service auditd rotate 2024-08-30T08:44:39.652 INFO:teuthology.orchestra.run.smithi042.stdout:Rotating logs: 2024-08-30T08:44:39.654 DEBUG:teuthology.orchestra.run.smithi167:> sudo service auditd rotate 2024-08-30T08:44:39.723 INFO:teuthology.orchestra.run.smithi167.stdout:Rotating logs: 2024-08-30T08:44:39.725 DEBUG:teuthology.orchestra.run.smithi190:> sudo service auditd rotate 2024-08-30T08:44:39.809 INFO:teuthology.orchestra.run.smithi190.stdout:Rotating logs: 2024-08-30T08:44:39.811 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-08-30T08:44:39.811 DEBUG:teuthology.orchestra.run.smithi042:> /usr/sbin/getenforce 2024-08-30T08:44:39.834 INFO:teuthology.orchestra.run.smithi042.stdout:Permissive 2024-08-30T08:44:39.834 DEBUG:teuthology.orchestra.run.smithi167:> /usr/sbin/getenforce 2024-08-30T08:44:39.858 INFO:teuthology.orchestra.run.smithi167.stdout:Permissive 2024-08-30T08:44:39.858 DEBUG:teuthology.orchestra.run.smithi190:> /usr/sbin/getenforce 2024-08-30T08:44:39.884 INFO:teuthology.orchestra.run.smithi190.stdout:Permissive 2024-08-30T08:44:39.884 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi042.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi167.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi190.front.sepia.ceph.com': 'permissive'} 2024-08-30T08:44:39.884 DEBUG:teuthology.orchestra.run.smithi042:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-08-30T08:44:39.909 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-30T08:44:39.909 DEBUG:teuthology.orchestra.run.smithi167:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-08-30T08:44:39.938 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-30T08:44:39.938 DEBUG:teuthology.orchestra.run.smithi190:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-08-30T08:44:39.967 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-30T08:44:39.967 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-08-30T08:44:39.967 DEBUG:teuthology.orchestra.run.smithi042:> sudo /usr/sbin/setenforce permissive 2024-08-30T08:44:39.999 DEBUG:teuthology.orchestra.run.smithi167:> sudo /usr/sbin/setenforce permissive 2024-08-30T08:44:40.037 DEBUG:teuthology.orchestra.run.smithi190:> sudo /usr/sbin/setenforce permissive 2024-08-30T08:44:40.076 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-08-30T08:44:40.085 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-08-30T08:44:40.088 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-08-30T08:44:40.203 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-08-30T08:44:40.213 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-08-30T08:44:40.214 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi042.front.sepia.ceph.com,smithi167.front.sepia.ceph.com,smithi190.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-08-30T08:46:43.799 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi042.front.sepia.ceph.com'), Remote(name='ubuntu@smithi167.front.sepia.ceph.com'), Remote(name='ubuntu@smithi190.front.sepia.ceph.com')] 2024-08-30T08:46:43.801 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi042.front.sepia.ceph.com' 2024-08-30T08:46:43.802 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi042.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-08-30T08:46:43.925 DEBUG:teuthology.orchestra.run.smithi042:> true 2024-08-30T08:46:44.004 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi042.front.sepia.ceph.com' 2024-08-30T08:46:44.005 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi167.front.sepia.ceph.com' 2024-08-30T08:46:44.005 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-08-30T08:46:44.105 DEBUG:teuthology.orchestra.run.smithi167:> true 2024-08-30T08:46:44.184 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi167.front.sepia.ceph.com' 2024-08-30T08:46:44.184 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi190.front.sepia.ceph.com' 2024-08-30T08:46:44.185 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi190.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-08-30T08:46:44.267 DEBUG:teuthology.orchestra.run.smithi190:> true 2024-08-30T08:46:44.344 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi190.front.sepia.ceph.com' 2024-08-30T08:46:44.345 INFO:teuthology.run_tasks:Running task clock... 2024-08-30T08:46:44.357 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-08-30T08:46:44.357 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-08-30T08:46:44.357 DEBUG:teuthology.orchestra.run.smithi042:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-08-30T08:46:44.360 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-08-30T08:46:44.360 DEBUG:teuthology.orchestra.run.smithi167:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-08-30T08:46:44.362 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-08-30T08:46:44.362 DEBUG:teuthology.orchestra.run.smithi190:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-08-30T08:46:44.390 INFO:teuthology.orchestra.run.smithi167.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-08-30T08:46:44.391 INFO:teuthology.orchestra.run.smithi042.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-08-30T08:46:44.405 INFO:teuthology.orchestra.run.smithi167.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-08-30T08:46:44.407 INFO:teuthology.orchestra.run.smithi042.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-08-30T08:46:44.416 INFO:teuthology.orchestra.run.smithi190.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-08-30T08:46:44.430 INFO:teuthology.orchestra.run.smithi167.stderr:sudo: ntpd: command not found 2024-08-30T08:46:44.431 INFO:teuthology.orchestra.run.smithi190.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-08-30T08:46:44.434 INFO:teuthology.orchestra.run.smithi042.stderr:sudo: ntpd: command not found 2024-08-30T08:46:44.441 INFO:teuthology.orchestra.run.smithi167.stdout:506 Cannot talk to daemon 2024-08-30T08:46:44.446 INFO:teuthology.orchestra.run.smithi042.stdout:506 Cannot talk to daemon 2024-08-30T08:46:44.455 INFO:teuthology.orchestra.run.smithi167.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-08-30T08:46:44.456 INFO:teuthology.orchestra.run.smithi190.stderr:sudo: ntpd: command not found 2024-08-30T08:46:44.461 INFO:teuthology.orchestra.run.smithi042.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-08-30T08:46:44.467 INFO:teuthology.orchestra.run.smithi190.stdout:506 Cannot talk to daemon 2024-08-30T08:46:44.470 INFO:teuthology.orchestra.run.smithi167.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-08-30T08:46:44.476 INFO:teuthology.orchestra.run.smithi042.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-08-30T08:46:44.481 INFO:teuthology.orchestra.run.smithi190.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-08-30T08:46:44.496 INFO:teuthology.orchestra.run.smithi190.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-08-30T08:46:44.525 INFO:teuthology.orchestra.run.smithi167.stderr:bash: line 1: ntpq: command not found 2024-08-30T08:46:44.529 INFO:teuthology.orchestra.run.smithi167.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-08-30T08:46:44.529 INFO:teuthology.orchestra.run.smithi167.stdout:=============================================================================== 2024-08-30T08:46:44.529 INFO:teuthology.orchestra.run.smithi167.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-08-30T08:46:44.529 INFO:teuthology.orchestra.run.smithi167.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-08-30T08:46:44.529 INFO:teuthology.orchestra.run.smithi167.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-08-30T08:46:44.529 INFO:teuthology.orchestra.run.smithi167.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-08-30T08:46:44.536 INFO:teuthology.orchestra.run.smithi042.stderr:bash: line 1: ntpq: command not found 2024-08-30T08:46:44.541 INFO:teuthology.orchestra.run.smithi042.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-08-30T08:46:44.541 INFO:teuthology.orchestra.run.smithi042.stdout:=============================================================================== 2024-08-30T08:46:44.541 INFO:teuthology.orchestra.run.smithi042.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-08-30T08:46:44.541 INFO:teuthology.orchestra.run.smithi042.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-08-30T08:46:44.541 INFO:teuthology.orchestra.run.smithi042.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-08-30T08:46:44.541 INFO:teuthology.orchestra.run.smithi042.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-08-30T08:46:44.563 INFO:teuthology.orchestra.run.smithi190.stderr:bash: line 1: ntpq: command not found 2024-08-30T08:46:44.568 INFO:teuthology.orchestra.run.smithi190.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-08-30T08:46:44.568 INFO:teuthology.orchestra.run.smithi190.stdout:=============================================================================== 2024-08-30T08:46:44.568 INFO:teuthology.orchestra.run.smithi190.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-08-30T08:46:44.568 INFO:teuthology.orchestra.run.smithi190.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-08-30T08:46:44.568 INFO:teuthology.orchestra.run.smithi190.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-08-30T08:46:44.568 INFO:teuthology.orchestra.run.smithi190.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-08-30T08:46:44.568 INFO:teuthology.run_tasks:Running task install... 2024-08-30T08:46:44.577 DEBUG:teuthology.task.install:project ceph 2024-08-30T08:46:44.577 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'd19e6118f7015f0047264006cb0c873528eb3ae6'}} 2024-08-30T08:46:44.577 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'd19e6118f7015f0047264006cb0c873528eb3ae6'} 2024-08-30T08:46:44.577 INFO:teuthology.task.install:Using flavor: default 2024-08-30T08:46:44.582 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-08-30T08:46:44.582 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-08-30T08:46:44.582 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd19e6118f7015f0047264006cb0c873528eb3ae6', 'tag': None, 'wait_for_package': False} 2024-08-30T08:46:44.583 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d19e6118f7015f0047264006cb0c873528eb3ae6 2024-08-30T08:46:44.585 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd19e6118f7015f0047264006cb0c873528eb3ae6', 'tag': None, 'wait_for_package': False} 2024-08-30T08:46:44.585 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d19e6118f7015f0047264006cb0c873528eb3ae6 2024-08-30T08:46:44.586 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd19e6118f7015f0047264006cb0c873528eb3ae6', 'tag': None, 'wait_for_package': False} 2024-08-30T08:46:44.586 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d19e6118f7015f0047264006cb0c873528eb3ae6 2024-08-30T08:46:44.731 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-mchangir-testing-20240828.085847-main-debug/d19e6118f7015f0047264006cb0c873528eb3ae6/centos/9/flavors/default/ 2024-08-30T08:46:44.732 INFO:teuthology.task.install.rpm:Package version is 19.3.0-4560.gd19e6118 2024-08-30T08:46:44.747 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-mchangir-testing-20240828.085847-main-debug/d19e6118f7015f0047264006cb0c873528eb3ae6/centos/9/flavors/default/ 2024-08-30T08:46:44.747 INFO:teuthology.task.install.rpm:Package version is 19.3.0-4560.gd19e6118 2024-08-30T08:46:44.748 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-mchangir-testing-20240828.085847-main-debug/d19e6118f7015f0047264006cb0c873528eb3ae6/centos/9/flavors/default/ 2024-08-30T08:46:44.748 INFO:teuthology.task.install.rpm:Package version is 19.3.0-4560.gd19e6118 2024-08-30T08:46:44.860 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-mchangir-testing-20240828.085847-main-debug/d19e6118f7015f0047264006cb0c873528eb3ae6/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-mchangir-testing-20240828.085847-main-debug/d19e6118f7015f0047264006cb0c873528eb3ae6/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-mchangir-testing-20240828.085847-main-debug/d19e6118f7015f0047264006cb0c873528eb3ae6/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-08-30T08:46:44.860 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-08-30T08:46:44.860 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-08-30T08:46:44.866 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-mchangir-testing-20240828.085847-main-debug/d19e6118f7015f0047264006cb0c873528eb3ae6/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-mchangir-testing-20240828.085847-main-debug/d19e6118f7015f0047264006cb0c873528eb3ae6/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-mchangir-testing-20240828.085847-main-debug/d19e6118f7015f0047264006cb0c873528eb3ae6/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-08-30T08:46:44.866 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-08-30T08:46:44.866 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-08-30T08:46:44.877 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-mchangir-testing-20240828.085847-main-debug/d19e6118f7015f0047264006cb0c873528eb3ae6/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-mchangir-testing-20240828.085847-main-debug/d19e6118f7015f0047264006cb0c873528eb3ae6/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-mchangir-testing-20240828.085847-main-debug/d19e6118f7015f0047264006cb0c873528eb3ae6/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-08-30T08:46:44.877 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-08-30T08:46:44.877 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-08-30T08:46:44.887 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-08-30T08:46:44.887 DEBUG:teuthology.orchestra.run.smithi042:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d19e6118f7015f0047264006cb0c873528eb3ae6/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-08-30T08:46:44.893 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-08-30T08:46:44.893 DEBUG:teuthology.orchestra.run.smithi190:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d19e6118f7015f0047264006cb0c873528eb3ae6/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-08-30T08:46:44.904 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-08-30T08:46:44.904 DEBUG:teuthology.orchestra.run.smithi167:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d19e6118f7015f0047264006cb0c873528eb3ae6/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-08-30T08:46:44.952 DEBUG:teuthology.orchestra.run.smithi042:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-08-30T08:46:44.958 DEBUG:teuthology.orchestra.run.smithi190:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-08-30T08:46:44.970 DEBUG:teuthology.orchestra.run.smithi167:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-08-30T08:46:45.027 DEBUG:teuthology.orchestra.run.smithi042:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-08-30T08:46:45.033 DEBUG:teuthology.orchestra.run.smithi190:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-08-30T08:46:45.046 DEBUG:teuthology.orchestra.run.smithi167:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-08-30T08:46:45.091 INFO:teuthology.orchestra.run.smithi042.stdout:check_obsoletes = 1 2024-08-30T08:46:45.101 INFO:teuthology.orchestra.run.smithi190.stdout:check_obsoletes = 1 2024-08-30T08:46:45.101 DEBUG:teuthology.orchestra.run.smithi042:> sudo yum clean all 2024-08-30T08:46:45.102 DEBUG:teuthology.orchestra.run.smithi190:> sudo yum clean all 2024-08-30T08:46:45.111 INFO:teuthology.orchestra.run.smithi167.stdout:check_obsoletes = 1 2024-08-30T08:46:45.112 DEBUG:teuthology.orchestra.run.smithi167:> sudo yum clean all 2024-08-30T08:46:45.326 INFO:teuthology.orchestra.run.smithi190.stdout:45 files removed 2024-08-30T08:46:45.331 INFO:teuthology.orchestra.run.smithi042.stdout:45 files removed 2024-08-30T08:46:45.339 INFO:teuthology.orchestra.run.smithi167.stdout:45 files removed 2024-08-30T08:46:45.352 DEBUG:teuthology.orchestra.run.smithi190:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-08-30T08:46:45.358 DEBUG:teuthology.orchestra.run.smithi042:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-08-30T08:46:45.365 DEBUG:teuthology.orchestra.run.smithi167:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-08-30T08:46:45.939 INFO:teuthology.orchestra.run.smithi042.stdout:ceph packages for x86_64 282 kB/s | 84 kB 00:00 2024-08-30T08:46:45.941 INFO:teuthology.orchestra.run.smithi167.stdout:ceph packages for x86_64 281 kB/s | 84 kB 00:00 2024-08-30T08:46:45.954 INFO:teuthology.orchestra.run.smithi190.stdout:ceph packages for x86_64 258 kB/s | 84 kB 00:00 2024-08-30T08:46:46.230 INFO:teuthology.orchestra.run.smithi190.stdout:ceph noarch packages 104 kB/s | 24 kB 00:00 2024-08-30T08:46:46.250 INFO:teuthology.orchestra.run.smithi167.stdout:ceph noarch packages 90 kB/s | 24 kB 00:00 2024-08-30T08:46:46.272 INFO:teuthology.orchestra.run.smithi042.stdout:ceph noarch packages 83 kB/s | 24 kB 00:00 2024-08-30T08:46:46.489 INFO:teuthology.orchestra.run.smithi190.stdout:ceph source packages 8.7 kB/s | 1.9 kB 00:00 2024-08-30T08:46:46.517 INFO:teuthology.orchestra.run.smithi167.stdout:ceph source packages 8.3 kB/s | 1.9 kB 00:00 2024-08-30T08:46:46.548 INFO:teuthology.orchestra.run.smithi042.stdout:ceph source packages 8.1 kB/s | 1.9 kB 00:00 2024-08-30T08:46:47.043 INFO:teuthology.orchestra.run.smithi167.stdout:CentOS Stream 9 - BaseOS 17 MB/s | 8.2 MB 00:00 2024-08-30T08:46:49.165 INFO:teuthology.orchestra.run.smithi042.stdout:CentOS Stream 9 - BaseOS 3.2 MB/s | 8.2 MB 00:02 2024-08-30T08:46:49.655 INFO:teuthology.orchestra.run.smithi190.stdout:CentOS Stream 9 - BaseOS 2.6 MB/s | 8.2 MB 00:03 2024-08-30T08:46:54.165 INFO:teuthology.orchestra.run.smithi190.stdout:CentOS Stream 9 - AppStream 6.1 MB/s | 20 MB 00:03 2024-08-30T08:46:56.119 INFO:teuthology.orchestra.run.smithi167.stdout:CentOS Stream 9 - AppStream 2.6 MB/s | 20 MB 00:07 2024-08-30T08:46:57.966 INFO:teuthology.orchestra.run.smithi042.stdout:CentOS Stream 9 - AppStream 2.7 MB/s | 20 MB 00:07 2024-08-30T08:46:59.381 INFO:teuthology.orchestra.run.smithi190.stdout:CentOS Stream 9 - CRB 4.6 MB/s | 6.6 MB 00:01 2024-08-30T08:47:00.403 INFO:teuthology.orchestra.run.smithi167.stdout:CentOS Stream 9 - CRB 14 MB/s | 6.6 MB 00:00 2024-08-30T08:47:01.149 INFO:teuthology.orchestra.run.smithi190.stdout:CentOS Stream 9 - Extras packages 33 kB/s | 18 kB 00:00 2024-08-30T08:47:01.945 INFO:teuthology.orchestra.run.smithi190.stdout:Extra Packages for Enterprise Linux 33 MB/s | 22 MB 00:00 2024-08-30T08:47:02.153 INFO:teuthology.orchestra.run.smithi167.stdout:CentOS Stream 9 - Extras packages 36 kB/s | 18 kB 00:00 2024-08-30T08:47:02.941 INFO:teuthology.orchestra.run.smithi167.stdout:Extra Packages for Enterprise Linux 34 MB/s | 22 MB 00:00 2024-08-30T08:47:03.583 INFO:teuthology.orchestra.run.smithi042.stdout:CentOS Stream 9 - CRB 3.8 MB/s | 6.6 MB 00:01 2024-08-30T08:47:05.134 INFO:teuthology.orchestra.run.smithi042.stdout:CentOS Stream 9 - Extras packages 64 kB/s | 18 kB 00:00 2024-08-30T08:47:06.180 INFO:teuthology.orchestra.run.smithi042.stdout:Extra Packages for Enterprise Linux 24 MB/s | 22 MB 00:00 2024-08-30T08:47:07.401 INFO:teuthology.orchestra.run.smithi190.stdout:lab-extras 46 kB/s | 1.7 kB 00:00 2024-08-30T08:47:08.413 INFO:teuthology.orchestra.run.smithi167.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-08-30T08:47:09.679 INFO:teuthology.orchestra.run.smithi190.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-08-30T08:47:09.680 INFO:teuthology.orchestra.run.smithi190.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-08-30T08:47:09.694 INFO:teuthology.orchestra.run.smithi190.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-08-30T08:47:09.695 INFO:teuthology.orchestra.run.smithi190.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-08-30T08:47:09.696 INFO:teuthology.orchestra.run.smithi190.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-08-30T08:47:09.697 INFO:teuthology.orchestra.run.smithi190.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-08-30T08:47:09.698 INFO:teuthology.orchestra.run.smithi190.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-08-30T08:47:09.701 INFO:teuthology.orchestra.run.smithi190.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-08-30T08:47:09.702 INFO:teuthology.orchestra.run.smithi190.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-08-30T08:47:09.764 INFO:teuthology.orchestra.run.smithi190.stdout:Dependencies resolved. 2024-08-30T08:47:09.774 INFO:teuthology.orchestra.run.smithi190.stdout:======================================================================================= 2024-08-30T08:47:09.774 INFO:teuthology.orchestra.run.smithi190.stdout: Package Arch Version Repository Size 2024-08-30T08:47:09.774 INFO:teuthology.orchestra.run.smithi190.stdout:======================================================================================= 2024-08-30T08:47:09.774 INFO:teuthology.orchestra.run.smithi190.stdout:Installing: 2024-08-30T08:47:09.774 INFO:teuthology.orchestra.run.smithi190.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-08-30T08:47:09.774 INFO:teuthology.orchestra.run.smithi190.stdout: ceph x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 6.6 k 2024-08-30T08:47:09.774 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-base x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 5.5 M 2024-08-30T08:47:09.775 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-fuse x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 874 k 2024-08-30T08:47:09.775 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-immutable-object-cache x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 148 k 2024-08-30T08:47:09.775 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 1.5 M 2024-08-30T08:47:09.775 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-cephadm noarch 2:19.3.0-4560.gd19e6118.el9 ceph-noarch 154 k 2024-08-30T08:47:09.775 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-dashboard noarch 2:19.3.0-4560.gd19e6118.el9 ceph-noarch 14 M 2024-08-30T08:47:09.775 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-diskprediction-local noarch 2:19.3.0-4560.gd19e6118.el9 ceph-noarch 7.4 M 2024-08-30T08:47:09.775 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-rook noarch 2:19.3.0-4560.gd19e6118.el9 ceph-noarch 49 k 2024-08-30T08:47:09.775 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-radosgw x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 10 M 2024-08-30T08:47:09.775 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-test x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 44 M 2024-08-30T08:47:09.775 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-volume noarch 2:19.3.0-4560.gd19e6118.el9 ceph-noarch 285 k 2024-08-30T08:47:09.775 INFO:teuthology.orchestra.run.smithi190.stdout: cephadm noarch 2:19.3.0-4560.gd19e6118.el9 ceph-noarch 781 k 2024-08-30T08:47:09.776 INFO:teuthology.orchestra.run.smithi190.stdout: cephfs-mirror x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 226 k 2024-08-30T08:47:09.776 INFO:teuthology.orchestra.run.smithi190.stdout: cephfs-shell x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 27 k 2024-08-30T08:47:09.776 INFO:teuthology.orchestra.run.smithi190.stdout: cephfs-top noarch 2:19.3.0-4560.gd19e6118.el9 ceph-noarch 18 k 2024-08-30T08:47:09.776 INFO:teuthology.orchestra.run.smithi190.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-08-30T08:47:09.776 INFO:teuthology.orchestra.run.smithi190.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-08-30T08:47:09.776 INFO:teuthology.orchestra.run.smithi190.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-08-30T08:47:09.776 INFO:teuthology.orchestra.run.smithi190.stdout: libcephfs-devel x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 32 k 2024-08-30T08:47:09.776 INFO:teuthology.orchestra.run.smithi190.stdout: libcephfs2 x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 766 k 2024-08-30T08:47:09.776 INFO:teuthology.orchestra.run.smithi190.stdout: librados-devel x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 127 k 2024-08-30T08:47:09.776 INFO:teuthology.orchestra.run.smithi190.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-08-30T08:47:09.777 INFO:teuthology.orchestra.run.smithi190.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-08-30T08:47:09.777 INFO:teuthology.orchestra.run.smithi190.stdout: openssl-devel x86_64 1:3.2.2-4.el9 appstream 4.4 M 2024-08-30T08:47:09.777 INFO:teuthology.orchestra.run.smithi190.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-08-30T08:47:09.777 INFO:teuthology.orchestra.run.smithi190.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-08-30T08:47:09.777 INFO:teuthology.orchestra.run.smithi190.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-08-30T08:47:09.777 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cephfs x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 161 k 2024-08-30T08:47:09.777 INFO:teuthology.orchestra.run.smithi190.stdout: python3-rados x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 321 k 2024-08-30T08:47:09.777 INFO:teuthology.orchestra.run.smithi190.stdout: python3-rbd x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 305 k 2024-08-30T08:47:09.777 INFO:teuthology.orchestra.run.smithi190.stdout: python3-rgw x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 99 k 2024-08-30T08:47:09.777 INFO:teuthology.orchestra.run.smithi190.stdout: rbd-fuse x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 86 k 2024-08-30T08:47:09.777 INFO:teuthology.orchestra.run.smithi190.stdout: rbd-mirror x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 3.1 M 2024-08-30T08:47:09.778 INFO:teuthology.orchestra.run.smithi190.stdout: rbd-nbd x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 170 k 2024-08-30T08:47:09.778 INFO:teuthology.orchestra.run.smithi190.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-08-30T08:47:09.778 INFO:teuthology.orchestra.run.smithi190.stdout:Upgrading: 2024-08-30T08:47:09.778 INFO:teuthology.orchestra.run.smithi190.stdout: librados2 x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 3.4 M 2024-08-30T08:47:09.778 INFO:teuthology.orchestra.run.smithi190.stdout: librbd1 x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 3.1 M 2024-08-30T08:47:09.778 INFO:teuthology.orchestra.run.smithi190.stdout:Installing dependencies: 2024-08-30T08:47:09.778 INFO:teuthology.orchestra.run.smithi190.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-08-30T08:47:09.778 INFO:teuthology.orchestra.run.smithi190.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-08-30T08:47:09.778 INFO:teuthology.orchestra.run.smithi190.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-08-30T08:47:09.778 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-common x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 20 M 2024-08-30T08:47:09.778 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-grafana-dashboards noarch 2:19.3.0-4560.gd19e6118.el9 ceph-noarch 34 k 2024-08-30T08:47:09.778 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mds x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 2.3 M 2024-08-30T08:47:09.779 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-modules-core noarch 2:19.3.0-4560.gd19e6118.el9 ceph-noarch 290 k 2024-08-30T08:47:09.779 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mon x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 4.9 M 2024-08-30T08:47:09.779 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-osd x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 18 M 2024-08-30T08:47:09.779 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-prometheus-alerts noarch 2:19.3.0-4560.gd19e6118.el9 ceph-noarch 16 k 2024-08-30T08:47:09.779 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-selinux x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 25 k 2024-08-30T08:47:09.779 INFO:teuthology.orchestra.run.smithi190.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-08-30T08:47:09.779 INFO:teuthology.orchestra.run.smithi190.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-08-30T08:47:09.779 INFO:teuthology.orchestra.run.smithi190.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-08-30T08:47:09.779 INFO:teuthology.orchestra.run.smithi190.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-08-30T08:47:09.779 INFO:teuthology.orchestra.run.smithi190.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-08-30T08:47:09.779 INFO:teuthology.orchestra.run.smithi190.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-08-30T08:47:09.779 INFO:teuthology.orchestra.run.smithi190.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-08-30T08:47:09.780 INFO:teuthology.orchestra.run.smithi190.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-08-30T08:47:09.780 INFO:teuthology.orchestra.run.smithi190.stdout: libcephsqlite x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 159 k 2024-08-30T08:47:09.780 INFO:teuthology.orchestra.run.smithi190.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-08-30T08:47:09.780 INFO:teuthology.orchestra.run.smithi190.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-08-30T08:47:09.780 INFO:teuthology.orchestra.run.smithi190.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-08-30T08:47:09.780 INFO:teuthology.orchestra.run.smithi190.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-08-30T08:47:09.780 INFO:teuthology.orchestra.run.smithi190.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-08-30T08:47:09.780 INFO:teuthology.orchestra.run.smithi190.stdout: libradosstriper1 x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 506 k 2024-08-30T08:47:09.780 INFO:teuthology.orchestra.run.smithi190.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-08-30T08:47:09.780 INFO:teuthology.orchestra.run.smithi190.stdout: librgw2 x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 5.3 M 2024-08-30T08:47:09.780 INFO:teuthology.orchestra.run.smithi190.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-08-30T08:47:09.780 INFO:teuthology.orchestra.run.smithi190.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-08-30T08:47:09.781 INFO:teuthology.orchestra.run.smithi190.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-08-30T08:47:09.781 INFO:teuthology.orchestra.run.smithi190.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-08-30T08:47:09.781 INFO:teuthology.orchestra.run.smithi190.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-08-30T08:47:09.781 INFO:teuthology.orchestra.run.smithi190.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-08-30T08:47:09.781 INFO:teuthology.orchestra.run.smithi190.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-08-30T08:47:09.781 INFO:teuthology.orchestra.run.smithi190.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-08-30T08:47:09.781 INFO:teuthology.orchestra.run.smithi190.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-08-30T08:47:09.781 INFO:teuthology.orchestra.run.smithi190.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-08-30T08:47:09.781 INFO:teuthology.orchestra.run.smithi190.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-08-30T08:47:09.781 INFO:teuthology.orchestra.run.smithi190.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-08-30T08:47:09.781 INFO:teuthology.orchestra.run.smithi190.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-08-30T08:47:09.781 INFO:teuthology.orchestra.run.smithi190.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-08-30T08:47:09.781 INFO:teuthology.orchestra.run.smithi190.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-08-30T08:47:09.782 INFO:teuthology.orchestra.run.smithi190.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-08-30T08:47:09.782 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-08-30T08:47:09.782 INFO:teuthology.orchestra.run.smithi190.stdout: python3-ceph-argparse x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 45 k 2024-08-30T08:47:09.782 INFO:teuthology.orchestra.run.smithi190.stdout: python3-ceph-common x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 141 k 2024-08-30T08:47:09.782 INFO:teuthology.orchestra.run.smithi190.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-08-30T08:47:09.782 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-08-30T08:47:09.782 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-08-30T08:47:09.782 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-08-30T08:47:09.782 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-08-30T08:47:09.782 INFO:teuthology.orchestra.run.smithi190.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-08-30T08:47:09.782 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-08-30T08:47:09.782 INFO:teuthology.orchestra.run.smithi190.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-08-30T08:47:09.782 INFO:teuthology.orchestra.run.smithi190.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-08-30T08:47:09.783 INFO:teuthology.orchestra.run.smithi190.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-08-30T08:47:09.783 INFO:teuthology.orchestra.run.smithi190.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-08-30T08:47:09.783 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-08-30T08:47:09.783 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-08-30T08:47:09.783 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-08-30T08:47:09.783 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-08-30T08:47:09.783 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-08-30T08:47:09.783 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-08-30T08:47:09.783 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-08-30T08:47:09.783 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-08-30T08:47:09.783 INFO:teuthology.orchestra.run.smithi190.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-08-30T08:47:09.783 INFO:teuthology.orchestra.run.smithi190.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-08-30T08:47:09.783 INFO:teuthology.orchestra.run.smithi190.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-08-30T08:47:09.784 INFO:teuthology.orchestra.run.smithi190.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-08-30T08:47:09.784 INFO:teuthology.orchestra.run.smithi190.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-08-30T08:47:09.784 INFO:teuthology.orchestra.run.smithi190.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-08-30T08:47:09.784 INFO:teuthology.orchestra.run.smithi190.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-08-30T08:47:09.784 INFO:teuthology.orchestra.run.smithi190.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-08-30T08:47:09.784 INFO:teuthology.orchestra.run.smithi190.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-08-30T08:47:09.784 INFO:teuthology.orchestra.run.smithi190.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-08-30T08:47:09.784 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-08-30T08:47:09.784 INFO:teuthology.orchestra.run.smithi190.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-08-30T08:47:09.784 INFO:teuthology.orchestra.run.smithi190.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-08-30T08:47:09.784 INFO:teuthology.orchestra.run.smithi190.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-08-30T08:47:09.784 INFO:teuthology.orchestra.run.smithi190.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-08-30T08:47:09.785 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-08-30T08:47:09.785 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-08-30T08:47:09.785 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-08-30T08:47:09.785 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-08-30T08:47:09.785 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-08-30T08:47:09.785 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-08-30T08:47:09.785 INFO:teuthology.orchestra.run.smithi190.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-08-30T08:47:09.785 INFO:teuthology.orchestra.run.smithi190.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-08-30T08:47:09.785 INFO:teuthology.orchestra.run.smithi190.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-08-30T08:47:09.785 INFO:teuthology.orchestra.run.smithi190.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-08-30T08:47:09.785 INFO:teuthology.orchestra.run.smithi190.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-08-30T08:47:09.785 INFO:teuthology.orchestra.run.smithi190.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-08-30T08:47:09.786 INFO:teuthology.orchestra.run.smithi190.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-08-30T08:47:09.786 INFO:teuthology.orchestra.run.smithi190.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-08-30T08:47:09.786 INFO:teuthology.orchestra.run.smithi190.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-08-30T08:47:09.786 INFO:teuthology.orchestra.run.smithi190.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-08-30T08:47:09.786 INFO:teuthology.orchestra.run.smithi190.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-08-30T08:47:09.786 INFO:teuthology.orchestra.run.smithi190.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-08-30T08:47:09.786 INFO:teuthology.orchestra.run.smithi190.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-08-30T08:47:09.786 INFO:teuthology.orchestra.run.smithi190.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-08-30T08:47:09.786 INFO:teuthology.orchestra.run.smithi190.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-08-30T08:47:09.786 INFO:teuthology.orchestra.run.smithi190.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-08-30T08:47:09.786 INFO:teuthology.orchestra.run.smithi190.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-08-30T08:47:09.786 INFO:teuthology.orchestra.run.smithi190.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-08-30T08:47:09.786 INFO:teuthology.orchestra.run.smithi190.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-08-30T08:47:09.787 INFO:teuthology.orchestra.run.smithi190.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-08-30T08:47:09.787 INFO:teuthology.orchestra.run.smithi190.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-08-30T08:47:09.787 INFO:teuthology.orchestra.run.smithi190.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-08-30T08:47:09.787 INFO:teuthology.orchestra.run.smithi190.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-08-30T08:47:09.787 INFO:teuthology.orchestra.run.smithi190.stdout:Installing weak dependencies: 2024-08-30T08:47:09.787 INFO:teuthology.orchestra.run.smithi190.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-08-30T08:47:09.787 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-08-30T08:47:09.787 INFO:teuthology.orchestra.run.smithi190.stdout:Transaction Summary 2024-08-30T08:47:09.787 INFO:teuthology.orchestra.run.smithi190.stdout:======================================================================================= 2024-08-30T08:47:09.787 INFO:teuthology.orchestra.run.smithi190.stdout:Install 148 Packages 2024-08-30T08:47:09.787 INFO:teuthology.orchestra.run.smithi190.stdout:Upgrade 2 Packages 2024-08-30T08:47:09.787 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-08-30T08:47:09.788 INFO:teuthology.orchestra.run.smithi190.stdout:Total download size: 224 M 2024-08-30T08:47:09.788 INFO:teuthology.orchestra.run.smithi190.stdout:Downloading Packages: 2024-08-30T08:47:10.194 INFO:teuthology.orchestra.run.smithi190.stdout:(1/150): ceph-19.3.0-4560.gd19e6118.el9.x86_64. 48 kB/s | 6.6 kB 00:00 2024-08-30T08:47:10.435 INFO:teuthology.orchestra.run.smithi190.stdout:(2/150): ceph-fuse-19.3.0-4560.gd19e6118.el9.x8 3.5 MB/s | 874 kB 00:00 2024-08-30T08:47:10.494 INFO:teuthology.orchestra.run.smithi190.stdout:(3/150): ceph-immutable-object-cache-19.3.0-456 2.5 MB/s | 148 kB 00:00 2024-08-30T08:47:10.628 INFO:teuthology.orchestra.run.smithi190.stdout:(4/150): ceph-base-19.3.0-4560.gd19e6118.el9.x8 9.7 MB/s | 5.5 MB 00:00 2024-08-30T08:47:10.707 INFO:teuthology.orchestra.run.smithi167.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-08-30T08:47:10.708 INFO:teuthology.orchestra.run.smithi167.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-08-30T08:47:10.722 INFO:teuthology.orchestra.run.smithi167.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-08-30T08:47:10.723 INFO:teuthology.orchestra.run.smithi167.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-08-30T08:47:10.723 INFO:teuthology.orchestra.run.smithi167.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-08-30T08:47:10.724 INFO:teuthology.orchestra.run.smithi167.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-08-30T08:47:10.726 INFO:teuthology.orchestra.run.smithi167.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-08-30T08:47:10.729 INFO:teuthology.orchestra.run.smithi167.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-08-30T08:47:10.730 INFO:teuthology.orchestra.run.smithi167.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-08-30T08:47:10.761 INFO:teuthology.orchestra.run.smithi190.stdout:(5/150): ceph-mds-19.3.0-4560.gd19e6118.el9.x86 8.5 MB/s | 2.3 MB 00:00 2024-08-30T08:47:10.792 INFO:teuthology.orchestra.run.smithi167.stdout:Dependencies resolved. 2024-08-30T08:47:10.808 INFO:teuthology.orchestra.run.smithi167.stdout:======================================================================================= 2024-08-30T08:47:10.808 INFO:teuthology.orchestra.run.smithi167.stdout: Package Arch Version Repository Size 2024-08-30T08:47:10.808 INFO:teuthology.orchestra.run.smithi167.stdout:======================================================================================= 2024-08-30T08:47:10.808 INFO:teuthology.orchestra.run.smithi167.stdout:Installing: 2024-08-30T08:47:10.808 INFO:teuthology.orchestra.run.smithi167.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-08-30T08:47:10.808 INFO:teuthology.orchestra.run.smithi167.stdout: ceph x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 6.6 k 2024-08-30T08:47:10.809 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-base x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 5.5 M 2024-08-30T08:47:10.809 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-fuse x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 874 k 2024-08-30T08:47:10.809 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-immutable-object-cache x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 148 k 2024-08-30T08:47:10.809 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 1.5 M 2024-08-30T08:47:10.809 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-cephadm noarch 2:19.3.0-4560.gd19e6118.el9 ceph-noarch 154 k 2024-08-30T08:47:10.809 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-dashboard noarch 2:19.3.0-4560.gd19e6118.el9 ceph-noarch 14 M 2024-08-30T08:47:10.809 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-diskprediction-local noarch 2:19.3.0-4560.gd19e6118.el9 ceph-noarch 7.4 M 2024-08-30T08:47:10.809 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-rook noarch 2:19.3.0-4560.gd19e6118.el9 ceph-noarch 49 k 2024-08-30T08:47:10.809 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-radosgw x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 10 M 2024-08-30T08:47:10.809 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-test x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 44 M 2024-08-30T08:47:10.809 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-volume noarch 2:19.3.0-4560.gd19e6118.el9 ceph-noarch 285 k 2024-08-30T08:47:10.810 INFO:teuthology.orchestra.run.smithi167.stdout: cephadm noarch 2:19.3.0-4560.gd19e6118.el9 ceph-noarch 781 k 2024-08-30T08:47:10.810 INFO:teuthology.orchestra.run.smithi167.stdout: cephfs-mirror x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 226 k 2024-08-30T08:47:10.810 INFO:teuthology.orchestra.run.smithi167.stdout: cephfs-shell x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 27 k 2024-08-30T08:47:10.810 INFO:teuthology.orchestra.run.smithi167.stdout: cephfs-top noarch 2:19.3.0-4560.gd19e6118.el9 ceph-noarch 18 k 2024-08-30T08:47:10.810 INFO:teuthology.orchestra.run.smithi167.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-08-30T08:47:10.810 INFO:teuthology.orchestra.run.smithi167.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-08-30T08:47:10.810 INFO:teuthology.orchestra.run.smithi167.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-08-30T08:47:10.810 INFO:teuthology.orchestra.run.smithi167.stdout: libcephfs-devel x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 32 k 2024-08-30T08:47:10.810 INFO:teuthology.orchestra.run.smithi167.stdout: libcephfs2 x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 766 k 2024-08-30T08:47:10.810 INFO:teuthology.orchestra.run.smithi167.stdout: librados-devel x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 127 k 2024-08-30T08:47:10.810 INFO:teuthology.orchestra.run.smithi167.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-08-30T08:47:10.810 INFO:teuthology.orchestra.run.smithi167.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-08-30T08:47:10.810 INFO:teuthology.orchestra.run.smithi167.stdout: openssl-devel x86_64 1:3.2.2-4.el9 appstream 4.4 M 2024-08-30T08:47:10.811 INFO:teuthology.orchestra.run.smithi167.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-08-30T08:47:10.811 INFO:teuthology.orchestra.run.smithi167.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-08-30T08:47:10.811 INFO:teuthology.orchestra.run.smithi167.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-08-30T08:47:10.811 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cephfs x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 161 k 2024-08-30T08:47:10.811 INFO:teuthology.orchestra.run.smithi167.stdout: python3-rados x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 321 k 2024-08-30T08:47:10.811 INFO:teuthology.orchestra.run.smithi167.stdout: python3-rbd x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 305 k 2024-08-30T08:47:10.811 INFO:teuthology.orchestra.run.smithi167.stdout: python3-rgw x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 99 k 2024-08-30T08:47:10.811 INFO:teuthology.orchestra.run.smithi167.stdout: rbd-fuse x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 86 k 2024-08-30T08:47:10.811 INFO:teuthology.orchestra.run.smithi167.stdout: rbd-mirror x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 3.1 M 2024-08-30T08:47:10.811 INFO:teuthology.orchestra.run.smithi167.stdout: rbd-nbd x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 170 k 2024-08-30T08:47:10.811 INFO:teuthology.orchestra.run.smithi167.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-08-30T08:47:10.811 INFO:teuthology.orchestra.run.smithi167.stdout:Upgrading: 2024-08-30T08:47:10.812 INFO:teuthology.orchestra.run.smithi167.stdout: librados2 x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 3.4 M 2024-08-30T08:47:10.812 INFO:teuthology.orchestra.run.smithi167.stdout: librbd1 x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 3.1 M 2024-08-30T08:47:10.812 INFO:teuthology.orchestra.run.smithi167.stdout:Installing dependencies: 2024-08-30T08:47:10.812 INFO:teuthology.orchestra.run.smithi167.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-08-30T08:47:10.812 INFO:teuthology.orchestra.run.smithi167.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-08-30T08:47:10.812 INFO:teuthology.orchestra.run.smithi167.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-08-30T08:47:10.812 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-common x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 20 M 2024-08-30T08:47:10.812 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-grafana-dashboards noarch 2:19.3.0-4560.gd19e6118.el9 ceph-noarch 34 k 2024-08-30T08:47:10.812 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mds x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 2.3 M 2024-08-30T08:47:10.812 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-modules-core noarch 2:19.3.0-4560.gd19e6118.el9 ceph-noarch 290 k 2024-08-30T08:47:10.812 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mon x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 4.9 M 2024-08-30T08:47:10.812 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-osd x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 18 M 2024-08-30T08:47:10.812 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-prometheus-alerts noarch 2:19.3.0-4560.gd19e6118.el9 ceph-noarch 16 k 2024-08-30T08:47:10.813 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-selinux x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 25 k 2024-08-30T08:47:10.813 INFO:teuthology.orchestra.run.smithi167.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-08-30T08:47:10.813 INFO:teuthology.orchestra.run.smithi167.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-08-30T08:47:10.813 INFO:teuthology.orchestra.run.smithi167.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-08-30T08:47:10.813 INFO:teuthology.orchestra.run.smithi167.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-08-30T08:47:10.813 INFO:teuthology.orchestra.run.smithi167.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-08-30T08:47:10.813 INFO:teuthology.orchestra.run.smithi167.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-08-30T08:47:10.813 INFO:teuthology.orchestra.run.smithi167.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-08-30T08:47:10.813 INFO:teuthology.orchestra.run.smithi167.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-08-30T08:47:10.813 INFO:teuthology.orchestra.run.smithi167.stdout: libcephsqlite x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 159 k 2024-08-30T08:47:10.813 INFO:teuthology.orchestra.run.smithi167.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-08-30T08:47:10.813 INFO:teuthology.orchestra.run.smithi167.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-08-30T08:47:10.813 INFO:teuthology.orchestra.run.smithi167.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-08-30T08:47:10.814 INFO:teuthology.orchestra.run.smithi167.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-08-30T08:47:10.814 INFO:teuthology.orchestra.run.smithi167.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-08-30T08:47:10.814 INFO:teuthology.orchestra.run.smithi167.stdout: libradosstriper1 x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 506 k 2024-08-30T08:47:10.814 INFO:teuthology.orchestra.run.smithi167.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-08-30T08:47:10.814 INFO:teuthology.orchestra.run.smithi167.stdout: librgw2 x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 5.3 M 2024-08-30T08:47:10.814 INFO:teuthology.orchestra.run.smithi167.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-08-30T08:47:10.814 INFO:teuthology.orchestra.run.smithi167.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-08-30T08:47:10.814 INFO:teuthology.orchestra.run.smithi167.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-08-30T08:47:10.814 INFO:teuthology.orchestra.run.smithi167.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-08-30T08:47:10.814 INFO:teuthology.orchestra.run.smithi167.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-08-30T08:47:10.814 INFO:teuthology.orchestra.run.smithi167.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-08-30T08:47:10.814 INFO:teuthology.orchestra.run.smithi167.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-08-30T08:47:10.815 INFO:teuthology.orchestra.run.smithi167.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-08-30T08:47:10.815 INFO:teuthology.orchestra.run.smithi167.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-08-30T08:47:10.815 INFO:teuthology.orchestra.run.smithi167.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-08-30T08:47:10.815 INFO:teuthology.orchestra.run.smithi167.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-08-30T08:47:10.815 INFO:teuthology.orchestra.run.smithi167.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-08-30T08:47:10.815 INFO:teuthology.orchestra.run.smithi167.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-08-30T08:47:10.815 INFO:teuthology.orchestra.run.smithi167.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-08-30T08:47:10.815 INFO:teuthology.orchestra.run.smithi167.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-08-30T08:47:10.815 INFO:teuthology.orchestra.run.smithi167.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-08-30T08:47:10.815 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-08-30T08:47:10.815 INFO:teuthology.orchestra.run.smithi167.stdout: python3-ceph-argparse x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 45 k 2024-08-30T08:47:10.815 INFO:teuthology.orchestra.run.smithi167.stdout: python3-ceph-common x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 141 k 2024-08-30T08:47:10.816 INFO:teuthology.orchestra.run.smithi167.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-08-30T08:47:10.816 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-08-30T08:47:10.816 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-08-30T08:47:10.816 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-08-30T08:47:10.816 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-08-30T08:47:10.816 INFO:teuthology.orchestra.run.smithi167.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-08-30T08:47:10.816 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-08-30T08:47:10.816 INFO:teuthology.orchestra.run.smithi167.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-08-30T08:47:10.816 INFO:teuthology.orchestra.run.smithi167.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-08-30T08:47:10.816 INFO:teuthology.orchestra.run.smithi167.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-08-30T08:47:10.816 INFO:teuthology.orchestra.run.smithi167.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-08-30T08:47:10.817 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-08-30T08:47:10.817 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-08-30T08:47:10.817 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-08-30T08:47:10.817 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-08-30T08:47:10.817 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-08-30T08:47:10.817 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-08-30T08:47:10.817 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-08-30T08:47:10.817 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-08-30T08:47:10.817 INFO:teuthology.orchestra.run.smithi167.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-08-30T08:47:10.817 INFO:teuthology.orchestra.run.smithi167.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-08-30T08:47:10.817 INFO:teuthology.orchestra.run.smithi167.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-08-30T08:47:10.817 INFO:teuthology.orchestra.run.smithi167.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-08-30T08:47:10.818 INFO:teuthology.orchestra.run.smithi167.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-08-30T08:47:10.818 INFO:teuthology.orchestra.run.smithi167.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-08-30T08:47:10.818 INFO:teuthology.orchestra.run.smithi167.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-08-30T08:47:10.818 INFO:teuthology.orchestra.run.smithi167.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-08-30T08:47:10.818 INFO:teuthology.orchestra.run.smithi167.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-08-30T08:47:10.818 INFO:teuthology.orchestra.run.smithi167.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-08-30T08:47:10.818 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-08-30T08:47:10.818 INFO:teuthology.orchestra.run.smithi167.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-08-30T08:47:10.818 INFO:teuthology.orchestra.run.smithi167.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-08-30T08:47:10.818 INFO:teuthology.orchestra.run.smithi167.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-08-30T08:47:10.818 INFO:teuthology.orchestra.run.smithi167.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-08-30T08:47:10.818 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-08-30T08:47:10.818 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-08-30T08:47:10.819 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-08-30T08:47:10.819 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-08-30T08:47:10.819 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-08-30T08:47:10.819 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-08-30T08:47:10.819 INFO:teuthology.orchestra.run.smithi167.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-08-30T08:47:10.819 INFO:teuthology.orchestra.run.smithi167.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-08-30T08:47:10.819 INFO:teuthology.orchestra.run.smithi167.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-08-30T08:47:10.819 INFO:teuthology.orchestra.run.smithi167.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-08-30T08:47:10.819 INFO:teuthology.orchestra.run.smithi167.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-08-30T08:47:10.819 INFO:teuthology.orchestra.run.smithi167.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-08-30T08:47:10.819 INFO:teuthology.orchestra.run.smithi167.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-08-30T08:47:10.819 INFO:teuthology.orchestra.run.smithi167.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-08-30T08:47:10.819 INFO:teuthology.orchestra.run.smithi167.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-08-30T08:47:10.819 INFO:teuthology.orchestra.run.smithi167.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-08-30T08:47:10.820 INFO:teuthology.orchestra.run.smithi167.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-08-30T08:47:10.820 INFO:teuthology.orchestra.run.smithi167.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-08-30T08:47:10.820 INFO:teuthology.orchestra.run.smithi167.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-08-30T08:47:10.820 INFO:teuthology.orchestra.run.smithi167.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-08-30T08:47:10.820 INFO:teuthology.orchestra.run.smithi167.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-08-30T08:47:10.820 INFO:teuthology.orchestra.run.smithi167.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-08-30T08:47:10.820 INFO:teuthology.orchestra.run.smithi167.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-08-30T08:47:10.820 INFO:teuthology.orchestra.run.smithi167.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-08-30T08:47:10.820 INFO:teuthology.orchestra.run.smithi167.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-08-30T08:47:10.820 INFO:teuthology.orchestra.run.smithi167.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-08-30T08:47:10.820 INFO:teuthology.orchestra.run.smithi167.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-08-30T08:47:10.821 INFO:teuthology.orchestra.run.smithi167.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-08-30T08:47:10.821 INFO:teuthology.orchestra.run.smithi167.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-08-30T08:47:10.821 INFO:teuthology.orchestra.run.smithi167.stdout:Installing weak dependencies: 2024-08-30T08:47:10.821 INFO:teuthology.orchestra.run.smithi167.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-08-30T08:47:10.821 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-08-30T08:47:10.821 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction Summary 2024-08-30T08:47:10.821 INFO:teuthology.orchestra.run.smithi167.stdout:======================================================================================= 2024-08-30T08:47:10.821 INFO:teuthology.orchestra.run.smithi167.stdout:Install 148 Packages 2024-08-30T08:47:10.821 INFO:teuthology.orchestra.run.smithi167.stdout:Upgrade 2 Packages 2024-08-30T08:47:10.821 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-08-30T08:47:10.822 INFO:teuthology.orchestra.run.smithi167.stdout:Total download size: 224 M 2024-08-30T08:47:10.822 INFO:teuthology.orchestra.run.smithi167.stdout:Downloading Packages: 2024-08-30T08:47:11.145 INFO:teuthology.orchestra.run.smithi190.stdout:(6/150): ceph-mgr-19.3.0-4560.gd19e6118.el9.x86 2.9 MB/s | 1.5 MB 00:00 2024-08-30T08:47:11.174 INFO:teuthology.orchestra.run.smithi167.stdout:(1/150): ceph-19.3.0-4560.gd19e6118.el9.x86_64. 50 kB/s | 6.6 kB 00:00 2024-08-30T08:47:11.357 INFO:teuthology.orchestra.run.smithi167.stdout:(2/150): ceph-fuse-19.3.0-4560.gd19e6118.el9.x8 4.7 MB/s | 874 kB 00:00 2024-08-30T08:47:11.407 INFO:teuthology.orchestra.run.smithi167.stdout:(3/150): ceph-immutable-object-cache-19.3.0-456 2.9 MB/s | 148 kB 00:00 2024-08-30T08:47:11.591 INFO:teuthology.orchestra.run.smithi167.stdout:(4/150): ceph-base-19.3.0-4560.gd19e6118.el9.x8 10 MB/s | 5.5 MB 00:00 2024-08-30T08:47:11.641 INFO:teuthology.orchestra.run.smithi167.stdout:(5/150): ceph-mds-19.3.0-4560.gd19e6118.el9.x86 9.8 MB/s | 2.3 MB 00:00 2024-08-30T08:47:11.725 INFO:teuthology.orchestra.run.smithi167.stdout:(6/150): ceph-mgr-19.3.0-4560.gd19e6118.el9.x86 11 MB/s | 1.5 MB 00:00 2024-08-30T08:47:11.745 INFO:teuthology.orchestra.run.smithi190.stdout:(7/150): ceph-common-19.3.0-4560.gd19e6118.el9. 12 MB/s | 20 MB 00:01 2024-08-30T08:47:11.769 INFO:teuthology.orchestra.run.smithi042.stdout:lab-extras 33 kB/s | 1.7 kB 00:00 2024-08-30T08:47:12.037 INFO:teuthology.orchestra.run.smithi190.stdout:(8/150): ceph-mon-19.3.0-4560.gd19e6118.el9.x86 3.8 MB/s | 4.9 MB 00:01 2024-08-30T08:47:12.109 INFO:teuthology.orchestra.run.smithi167.stdout:(7/150): ceph-mon-19.3.0-4560.gd19e6118.el9.x86 10 MB/s | 4.9 MB 00:00 2024-08-30T08:47:12.312 INFO:teuthology.orchestra.run.smithi190.stdout:(9/150): ceph-selinux-19.3.0-4560.gd19e6118.el9 91 kB/s | 25 kB 00:00 2024-08-30T08:47:12.437 INFO:teuthology.orchestra.run.smithi190.stdout:(10/150): ceph-radosgw-19.3.0-4560.gd19e6118.el 15 MB/s | 10 MB 00:00 2024-08-30T08:47:12.563 INFO:teuthology.orchestra.run.smithi190.stdout:(11/150): cephfs-mirror-19.3.0-4560.gd19e6118.e 1.8 MB/s | 226 kB 00:00 2024-08-30T08:47:12.663 INFO:teuthology.orchestra.run.smithi190.stdout:(12/150): cephfs-shell-19.3.0-4560.gd19e6118.el 270 kB/s | 27 kB 00:00 2024-08-30T08:47:12.830 INFO:teuthology.orchestra.run.smithi190.stdout:(13/150): libcephfs-devel-19.3.0-4560.gd19e6118 190 kB/s | 32 kB 00:00 2024-08-30T08:47:12.967 INFO:teuthology.orchestra.run.smithi167.stdout:(8/150): ceph-radosgw-19.3.0-4560.gd19e6118.el9 12 MB/s | 10 MB 00:00 2024-08-30T08:47:13.080 INFO:teuthology.orchestra.run.smithi190.stdout:(14/150): libcephfs2-19.3.0-4560.gd19e6118.el9. 3.0 MB/s | 766 kB 00:00 2024-08-30T08:47:13.142 INFO:teuthology.orchestra.run.smithi167.stdout:(9/150): ceph-osd-19.3.0-4560.gd19e6118.el9.x86 13 MB/s | 18 MB 00:01 2024-08-30T08:47:13.167 INFO:teuthology.orchestra.run.smithi167.stdout:(10/150): ceph-selinux-19.3.0-4560.gd19e6118.el 126 kB/s | 25 kB 00:00 2024-08-30T08:47:13.281 INFO:teuthology.orchestra.run.smithi190.stdout:(15/150): ceph-osd-19.3.0-4560.gd19e6118.el9.x8 8.3 MB/s | 18 MB 00:02 2024-08-30T08:47:13.284 INFO:teuthology.orchestra.run.smithi167.stdout:(11/150): cephfs-mirror-19.3.0-4560.gd19e6118.e 1.9 MB/s | 226 kB 00:00 2024-08-30T08:47:13.305 INFO:teuthology.orchestra.run.smithi190.stdout:(16/150): libcephsqlite-19.3.0-4560.gd19e6118.e 706 kB/s | 159 kB 00:00 2024-08-30T08:47:13.347 INFO:teuthology.orchestra.run.smithi190.stdout:(17/150): librados-devel-19.3.0-4560.gd19e6118. 1.9 MB/s | 127 kB 00:00 2024-08-30T08:47:13.390 INFO:teuthology.orchestra.run.smithi190.stdout:(18/150): libradosstriper1-19.3.0-4560.gd19e611 5.9 MB/s | 506 kB 00:00 2024-08-30T08:47:13.393 INFO:teuthology.orchestra.run.smithi167.stdout:(12/150): cephfs-shell-19.3.0-4560.gd19e6118.el 249 kB/s | 27 kB 00:00 2024-08-30T08:47:13.681 INFO:teuthology.orchestra.run.smithi190.stdout:(19/150): librgw2-19.3.0-4560.gd19e6118.el9.x86 16 MB/s | 5.3 MB 00:00 2024-08-30T08:47:13.706 INFO:teuthology.orchestra.run.smithi190.stdout:(20/150): python3-ceph-argparse-19.3.0-4560.gd1 142 kB/s | 45 kB 00:00 2024-08-30T08:47:13.757 INFO:teuthology.orchestra.run.smithi190.stdout:(21/150): python3-ceph-common-19.3.0-4560.gd19e 1.8 MB/s | 141 kB 00:00 2024-08-30T08:47:13.790 INFO:teuthology.orchestra.run.smithi190.stdout:(22/150): python3-cephfs-19.3.0-4560.gd19e6118. 1.9 MB/s | 161 kB 00:00 2024-08-30T08:47:13.866 INFO:teuthology.orchestra.run.smithi190.stdout:(23/150): python3-rados-19.3.0-4560.gd19e6118.e 2.9 MB/s | 321 kB 00:00 2024-08-30T08:47:13.900 INFO:teuthology.orchestra.run.smithi190.stdout:(24/150): python3-rbd-19.3.0-4560.gd19e6118.el9 2.7 MB/s | 305 kB 00:00 2024-08-30T08:47:13.933 INFO:teuthology.orchestra.run.smithi190.stdout:(25/150): python3-rgw-19.3.0-4560.gd19e6118.el9 1.4 MB/s | 99 kB 00:00 2024-08-30T08:47:13.975 INFO:teuthology.orchestra.run.smithi190.stdout:(26/150): rbd-fuse-19.3.0-4560.gd19e6118.el9.x8 1.1 MB/s | 86 kB 00:00 2024-08-30T08:47:14.148 INFO:teuthology.orchestra.run.smithi042.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-08-30T08:47:14.149 INFO:teuthology.orchestra.run.smithi042.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-08-30T08:47:14.163 INFO:teuthology.orchestra.run.smithi042.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-08-30T08:47:14.164 INFO:teuthology.orchestra.run.smithi042.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-08-30T08:47:14.165 INFO:teuthology.orchestra.run.smithi042.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-08-30T08:47:14.166 INFO:teuthology.orchestra.run.smithi042.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-08-30T08:47:14.168 INFO:teuthology.orchestra.run.smithi042.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-08-30T08:47:14.171 INFO:teuthology.orchestra.run.smithi042.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-08-30T08:47:14.172 INFO:teuthology.orchestra.run.smithi042.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-08-30T08:47:14.237 INFO:teuthology.orchestra.run.smithi042.stdout:Dependencies resolved. 2024-08-30T08:47:14.243 INFO:teuthology.orchestra.run.smithi190.stdout:(27/150): rbd-mirror-19.3.0-4560.gd19e6118.el9. 10 MB/s | 3.1 MB 00:00 2024-08-30T08:47:14.245 INFO:teuthology.orchestra.run.smithi042.stdout:======================================================================================= 2024-08-30T08:47:14.245 INFO:teuthology.orchestra.run.smithi042.stdout: Package Arch Version Repository Size 2024-08-30T08:47:14.246 INFO:teuthology.orchestra.run.smithi042.stdout:======================================================================================= 2024-08-30T08:47:14.246 INFO:teuthology.orchestra.run.smithi042.stdout:Installing: 2024-08-30T08:47:14.246 INFO:teuthology.orchestra.run.smithi042.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-08-30T08:47:14.246 INFO:teuthology.orchestra.run.smithi042.stdout: ceph x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 6.6 k 2024-08-30T08:47:14.246 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-base x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 5.5 M 2024-08-30T08:47:14.246 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-fuse x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 874 k 2024-08-30T08:47:14.246 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-immutable-object-cache x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 148 k 2024-08-30T08:47:14.246 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mgr x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 1.5 M 2024-08-30T08:47:14.246 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mgr-cephadm noarch 2:19.3.0-4560.gd19e6118.el9 ceph-noarch 154 k 2024-08-30T08:47:14.246 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mgr-dashboard noarch 2:19.3.0-4560.gd19e6118.el9 ceph-noarch 14 M 2024-08-30T08:47:14.246 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mgr-diskprediction-local noarch 2:19.3.0-4560.gd19e6118.el9 ceph-noarch 7.4 M 2024-08-30T08:47:14.246 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mgr-rook noarch 2:19.3.0-4560.gd19e6118.el9 ceph-noarch 49 k 2024-08-30T08:47:14.246 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-radosgw x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 10 M 2024-08-30T08:47:14.247 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-test x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 44 M 2024-08-30T08:47:14.247 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-volume noarch 2:19.3.0-4560.gd19e6118.el9 ceph-noarch 285 k 2024-08-30T08:47:14.247 INFO:teuthology.orchestra.run.smithi042.stdout: cephadm noarch 2:19.3.0-4560.gd19e6118.el9 ceph-noarch 781 k 2024-08-30T08:47:14.247 INFO:teuthology.orchestra.run.smithi042.stdout: cephfs-mirror x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 226 k 2024-08-30T08:47:14.247 INFO:teuthology.orchestra.run.smithi042.stdout: cephfs-shell x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 27 k 2024-08-30T08:47:14.247 INFO:teuthology.orchestra.run.smithi042.stdout: cephfs-top noarch 2:19.3.0-4560.gd19e6118.el9 ceph-noarch 18 k 2024-08-30T08:47:14.247 INFO:teuthology.orchestra.run.smithi042.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-08-30T08:47:14.247 INFO:teuthology.orchestra.run.smithi042.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-08-30T08:47:14.247 INFO:teuthology.orchestra.run.smithi042.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-08-30T08:47:14.247 INFO:teuthology.orchestra.run.smithi042.stdout: libcephfs-devel x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 32 k 2024-08-30T08:47:14.247 INFO:teuthology.orchestra.run.smithi042.stdout: libcephfs2 x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 766 k 2024-08-30T08:47:14.247 INFO:teuthology.orchestra.run.smithi042.stdout: librados-devel x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 127 k 2024-08-30T08:47:14.248 INFO:teuthology.orchestra.run.smithi042.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-08-30T08:47:14.248 INFO:teuthology.orchestra.run.smithi042.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-08-30T08:47:14.248 INFO:teuthology.orchestra.run.smithi042.stdout: openssl-devel x86_64 1:3.2.2-4.el9 appstream 4.4 M 2024-08-30T08:47:14.248 INFO:teuthology.orchestra.run.smithi042.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-08-30T08:47:14.248 INFO:teuthology.orchestra.run.smithi042.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-08-30T08:47:14.248 INFO:teuthology.orchestra.run.smithi042.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-08-30T08:47:14.248 INFO:teuthology.orchestra.run.smithi042.stdout: python3-cephfs x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 161 k 2024-08-30T08:47:14.248 INFO:teuthology.orchestra.run.smithi042.stdout: python3-rados x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 321 k 2024-08-30T08:47:14.248 INFO:teuthology.orchestra.run.smithi042.stdout: python3-rbd x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 305 k 2024-08-30T08:47:14.248 INFO:teuthology.orchestra.run.smithi042.stdout: python3-rgw x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 99 k 2024-08-30T08:47:14.248 INFO:teuthology.orchestra.run.smithi042.stdout: rbd-fuse x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 86 k 2024-08-30T08:47:14.248 INFO:teuthology.orchestra.run.smithi042.stdout: rbd-mirror x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 3.1 M 2024-08-30T08:47:14.248 INFO:teuthology.orchestra.run.smithi042.stdout: rbd-nbd x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 170 k 2024-08-30T08:47:14.249 INFO:teuthology.orchestra.run.smithi042.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-08-30T08:47:14.249 INFO:teuthology.orchestra.run.smithi042.stdout:Upgrading: 2024-08-30T08:47:14.249 INFO:teuthology.orchestra.run.smithi042.stdout: librados2 x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 3.4 M 2024-08-30T08:47:14.249 INFO:teuthology.orchestra.run.smithi042.stdout: librbd1 x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 3.1 M 2024-08-30T08:47:14.249 INFO:teuthology.orchestra.run.smithi042.stdout:Installing dependencies: 2024-08-30T08:47:14.249 INFO:teuthology.orchestra.run.smithi042.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-08-30T08:47:14.249 INFO:teuthology.orchestra.run.smithi042.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-08-30T08:47:14.249 INFO:teuthology.orchestra.run.smithi042.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-08-30T08:47:14.249 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-common x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 20 M 2024-08-30T08:47:14.249 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-grafana-dashboards noarch 2:19.3.0-4560.gd19e6118.el9 ceph-noarch 34 k 2024-08-30T08:47:14.249 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mds x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 2.3 M 2024-08-30T08:47:14.249 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mgr-modules-core noarch 2:19.3.0-4560.gd19e6118.el9 ceph-noarch 290 k 2024-08-30T08:47:14.250 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mon x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 4.9 M 2024-08-30T08:47:14.250 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-osd x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 18 M 2024-08-30T08:47:14.250 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-prometheus-alerts noarch 2:19.3.0-4560.gd19e6118.el9 ceph-noarch 16 k 2024-08-30T08:47:14.250 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-selinux x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 25 k 2024-08-30T08:47:14.250 INFO:teuthology.orchestra.run.smithi042.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-08-30T08:47:14.250 INFO:teuthology.orchestra.run.smithi042.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-08-30T08:47:14.250 INFO:teuthology.orchestra.run.smithi042.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-08-30T08:47:14.250 INFO:teuthology.orchestra.run.smithi042.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-08-30T08:47:14.250 INFO:teuthology.orchestra.run.smithi042.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-08-30T08:47:14.250 INFO:teuthology.orchestra.run.smithi042.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-08-30T08:47:14.250 INFO:teuthology.orchestra.run.smithi042.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-08-30T08:47:14.250 INFO:teuthology.orchestra.run.smithi042.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-08-30T08:47:14.250 INFO:teuthology.orchestra.run.smithi042.stdout: libcephsqlite x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 159 k 2024-08-30T08:47:14.251 INFO:teuthology.orchestra.run.smithi042.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-08-30T08:47:14.251 INFO:teuthology.orchestra.run.smithi042.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-08-30T08:47:14.251 INFO:teuthology.orchestra.run.smithi042.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-08-30T08:47:14.251 INFO:teuthology.orchestra.run.smithi042.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-08-30T08:47:14.251 INFO:teuthology.orchestra.run.smithi042.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-08-30T08:47:14.251 INFO:teuthology.orchestra.run.smithi042.stdout: libradosstriper1 x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 506 k 2024-08-30T08:47:14.251 INFO:teuthology.orchestra.run.smithi042.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-08-30T08:47:14.251 INFO:teuthology.orchestra.run.smithi042.stdout: librgw2 x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 5.3 M 2024-08-30T08:47:14.251 INFO:teuthology.orchestra.run.smithi042.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-08-30T08:47:14.251 INFO:teuthology.orchestra.run.smithi042.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-08-30T08:47:14.251 INFO:teuthology.orchestra.run.smithi042.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-08-30T08:47:14.251 INFO:teuthology.orchestra.run.smithi042.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-08-30T08:47:14.251 INFO:teuthology.orchestra.run.smithi042.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-08-30T08:47:14.252 INFO:teuthology.orchestra.run.smithi042.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-08-30T08:47:14.252 INFO:teuthology.orchestra.run.smithi042.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-08-30T08:47:14.252 INFO:teuthology.orchestra.run.smithi042.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-08-30T08:47:14.252 INFO:teuthology.orchestra.run.smithi042.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-08-30T08:47:14.252 INFO:teuthology.orchestra.run.smithi042.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-08-30T08:47:14.252 INFO:teuthology.orchestra.run.smithi042.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-08-30T08:47:14.252 INFO:teuthology.orchestra.run.smithi042.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-08-30T08:47:14.252 INFO:teuthology.orchestra.run.smithi042.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-08-30T08:47:14.252 INFO:teuthology.orchestra.run.smithi042.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-08-30T08:47:14.252 INFO:teuthology.orchestra.run.smithi042.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-08-30T08:47:14.252 INFO:teuthology.orchestra.run.smithi042.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-08-30T08:47:14.252 INFO:teuthology.orchestra.run.smithi042.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-08-30T08:47:14.252 INFO:teuthology.orchestra.run.smithi042.stdout: python3-ceph-argparse x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 45 k 2024-08-30T08:47:14.253 INFO:teuthology.orchestra.run.smithi042.stdout: python3-ceph-common x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 141 k 2024-08-30T08:47:14.253 INFO:teuthology.orchestra.run.smithi042.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-08-30T08:47:14.253 INFO:teuthology.orchestra.run.smithi042.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-08-30T08:47:14.253 INFO:teuthology.orchestra.run.smithi042.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-08-30T08:47:14.253 INFO:teuthology.orchestra.run.smithi042.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-08-30T08:47:14.253 INFO:teuthology.orchestra.run.smithi042.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-08-30T08:47:14.253 INFO:teuthology.orchestra.run.smithi042.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-08-30T08:47:14.253 INFO:teuthology.orchestra.run.smithi042.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-08-30T08:47:14.253 INFO:teuthology.orchestra.run.smithi042.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-08-30T08:47:14.253 INFO:teuthology.orchestra.run.smithi042.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-08-30T08:47:14.253 INFO:teuthology.orchestra.run.smithi042.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-08-30T08:47:14.253 INFO:teuthology.orchestra.run.smithi042.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-08-30T08:47:14.253 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-08-30T08:47:14.253 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-08-30T08:47:14.254 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-08-30T08:47:14.254 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-08-30T08:47:14.254 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-08-30T08:47:14.254 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-08-30T08:47:14.254 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-08-30T08:47:14.254 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-08-30T08:47:14.254 INFO:teuthology.orchestra.run.smithi042.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-08-30T08:47:14.254 INFO:teuthology.orchestra.run.smithi042.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-08-30T08:47:14.254 INFO:teuthology.orchestra.run.smithi042.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-08-30T08:47:14.254 INFO:teuthology.orchestra.run.smithi042.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-08-30T08:47:14.254 INFO:teuthology.orchestra.run.smithi042.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-08-30T08:47:14.254 INFO:teuthology.orchestra.run.smithi042.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-08-30T08:47:14.254 INFO:teuthology.orchestra.run.smithi042.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-08-30T08:47:14.254 INFO:teuthology.orchestra.run.smithi042.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-08-30T08:47:14.255 INFO:teuthology.orchestra.run.smithi042.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-08-30T08:47:14.255 INFO:teuthology.orchestra.run.smithi042.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-08-30T08:47:14.255 INFO:teuthology.orchestra.run.smithi042.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-08-30T08:47:14.255 INFO:teuthology.orchestra.run.smithi042.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-08-30T08:47:14.255 INFO:teuthology.orchestra.run.smithi042.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-08-30T08:47:14.255 INFO:teuthology.orchestra.run.smithi042.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-08-30T08:47:14.255 INFO:teuthology.orchestra.run.smithi042.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-08-30T08:47:14.255 INFO:teuthology.orchestra.run.smithi042.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-08-30T08:47:14.255 INFO:teuthology.orchestra.run.smithi042.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-08-30T08:47:14.256 INFO:teuthology.orchestra.run.smithi042.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-08-30T08:47:14.256 INFO:teuthology.orchestra.run.smithi042.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-08-30T08:47:14.256 INFO:teuthology.orchestra.run.smithi042.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-08-30T08:47:14.256 INFO:teuthology.orchestra.run.smithi042.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-08-30T08:47:14.256 INFO:teuthology.orchestra.run.smithi042.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-08-30T08:47:14.256 INFO:teuthology.orchestra.run.smithi042.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-08-30T08:47:14.256 INFO:teuthology.orchestra.run.smithi042.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-08-30T08:47:14.256 INFO:teuthology.orchestra.run.smithi042.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-08-30T08:47:14.256 INFO:teuthology.orchestra.run.smithi042.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-08-30T08:47:14.256 INFO:teuthology.orchestra.run.smithi042.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-08-30T08:47:14.256 INFO:teuthology.orchestra.run.smithi042.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-08-30T08:47:14.256 INFO:teuthology.orchestra.run.smithi042.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-08-30T08:47:14.256 INFO:teuthology.orchestra.run.smithi042.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-08-30T08:47:14.257 INFO:teuthology.orchestra.run.smithi042.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-08-30T08:47:14.257 INFO:teuthology.orchestra.run.smithi042.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-08-30T08:47:14.257 INFO:teuthology.orchestra.run.smithi042.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-08-30T08:47:14.257 INFO:teuthology.orchestra.run.smithi042.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-08-30T08:47:14.257 INFO:teuthology.orchestra.run.smithi042.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-08-30T08:47:14.257 INFO:teuthology.orchestra.run.smithi042.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-08-30T08:47:14.257 INFO:teuthology.orchestra.run.smithi042.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-08-30T08:47:14.257 INFO:teuthology.orchestra.run.smithi042.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-08-30T08:47:14.257 INFO:teuthology.orchestra.run.smithi042.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-08-30T08:47:14.257 INFO:teuthology.orchestra.run.smithi042.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-08-30T08:47:14.257 INFO:teuthology.orchestra.run.smithi042.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-08-30T08:47:14.258 INFO:teuthology.orchestra.run.smithi042.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-08-30T08:47:14.258 INFO:teuthology.orchestra.run.smithi042.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-08-30T08:47:14.258 INFO:teuthology.orchestra.run.smithi042.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-08-30T08:47:14.258 INFO:teuthology.orchestra.run.smithi042.stdout:Installing weak dependencies: 2024-08-30T08:47:14.258 INFO:teuthology.orchestra.run.smithi042.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-08-30T08:47:14.258 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-08-30T08:47:14.258 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction Summary 2024-08-30T08:47:14.258 INFO:teuthology.orchestra.run.smithi042.stdout:======================================================================================= 2024-08-30T08:47:14.258 INFO:teuthology.orchestra.run.smithi042.stdout:Install 148 Packages 2024-08-30T08:47:14.258 INFO:teuthology.orchestra.run.smithi042.stdout:Upgrade 2 Packages 2024-08-30T08:47:14.258 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-08-30T08:47:14.259 INFO:teuthology.orchestra.run.smithi042.stdout:Total download size: 224 M 2024-08-30T08:47:14.259 INFO:teuthology.orchestra.run.smithi042.stdout:Downloading Packages: 2024-08-30T08:47:14.276 INFO:teuthology.orchestra.run.smithi190.stdout:(28/150): rbd-nbd-19.3.0-4560.gd19e6118.el9.x86 566 kB/s | 170 kB 00:00 2024-08-30T08:47:14.301 INFO:teuthology.orchestra.run.smithi190.stdout:(29/150): ceph-grafana-dashboards-19.3.0-4560.g 589 kB/s | 34 kB 00:00 2024-08-30T08:47:14.368 INFO:teuthology.orchestra.run.smithi190.stdout:(30/150): ceph-mgr-cephadm-19.3.0-4560.gd19e611 1.6 MB/s | 154 kB 00:00 2024-08-30T08:47:14.895 INFO:teuthology.orchestra.run.smithi042.stdout:(1/150): ceph-19.3.0-4560.gd19e6118.el9.x86_64. 48 kB/s | 6.6 kB 00:00 2024-08-30T08:47:15.096 INFO:teuthology.orchestra.run.smithi042.stdout:(2/150): ceph-fuse-19.3.0-4560.gd19e6118.el9.x8 4.3 MB/s | 874 kB 00:00 2024-08-30T08:47:15.154 INFO:teuthology.orchestra.run.smithi042.stdout:(3/150): ceph-immutable-object-cache-19.3.0-456 2.5 MB/s | 148 kB 00:00 2024-08-30T08:47:15.388 INFO:teuthology.orchestra.run.smithi042.stdout:(4/150): ceph-base-19.3.0-4560.gd19e6118.el9.x8 8.8 MB/s | 5.5 MB 00:00 2024-08-30T08:47:15.438 INFO:teuthology.orchestra.run.smithi042.stdout:(5/150): ceph-mds-19.3.0-4560.gd19e6118.el9.x86 8.0 MB/s | 2.3 MB 00:00 2024-08-30T08:47:15.452 INFO:teuthology.orchestra.run.smithi190.stdout:(31/150): ceph-mgr-dashboard-19.3.0-4560.gd19e6 12 MB/s | 14 MB 00:01 2024-08-30T08:47:15.510 INFO:teuthology.orchestra.run.smithi167.stdout:(13/150): ceph-test-19.3.0-4560.gd19e6118.el9.x 19 MB/s | 44 MB 00:02 2024-08-30T08:47:15.535 INFO:teuthology.orchestra.run.smithi167.stdout:(14/150): libcephfs-devel-19.3.0-4560.gd19e6118 15 kB/s | 32 kB 00:02 2024-08-30T08:47:15.552 INFO:teuthology.orchestra.run.smithi190.stdout:(32/150): ceph-mgr-modules-core-19.3.0-4560.gd1 2.9 MB/s | 290 kB 00:00 2024-08-30T08:47:15.571 INFO:teuthology.orchestra.run.smithi042.stdout:(6/150): ceph-mgr-19.3.0-4560.gd19e6118.el9.x86 8.3 MB/s | 1.5 MB 00:00 2024-08-30T08:47:15.627 INFO:teuthology.orchestra.run.smithi190.stdout:(33/150): ceph-mgr-rook-19.3.0-4560.gd19e6118.e 657 kB/s | 49 kB 00:00 2024-08-30T08:47:15.694 INFO:teuthology.orchestra.run.smithi167.stdout:(15/150): libcephfs2-19.3.0-4560.gd19e6118.el9. 4.1 MB/s | 766 kB 00:00 2024-08-30T08:47:15.744 INFO:teuthology.orchestra.run.smithi190.stdout:(34/150): ceph-prometheus-alerts-19.3.0-4560.gd 141 kB/s | 16 kB 00:00 2024-08-30T08:47:15.794 INFO:teuthology.orchestra.run.smithi167.stdout:(16/150): libcephsqlite-19.3.0-4560.gd19e6118.e 614 kB/s | 159 kB 00:00 2024-08-30T08:47:15.863 INFO:teuthology.orchestra.run.smithi167.stdout:(17/150): librados-devel-19.3.0-4560.gd19e6118. 1.0 MB/s | 127 kB 00:00 2024-08-30T08:47:15.877 INFO:teuthology.orchestra.run.smithi190.stdout:(35/150): ceph-volume-19.3.0-4560.gd19e6118.el9 2.1 MB/s | 285 kB 00:00 2024-08-30T08:47:16.011 INFO:teuthology.orchestra.run.smithi190.stdout:(36/150): cephadm-19.3.0-4560.gd19e6118.el9.noa 5.7 MB/s | 781 kB 00:00 2024-08-30T08:47:16.072 INFO:teuthology.orchestra.run.smithi042.stdout:(7/150): ceph-mon-19.3.0-4560.gd19e6118.el9.x86 7.7 MB/s | 4.9 MB 00:00 2024-08-30T08:47:16.112 INFO:teuthology.orchestra.run.smithi190.stdout:(37/150): ceph-mgr-diskprediction-local-19.3.0- 4.2 MB/s | 7.4 MB 00:01 2024-08-30T08:47:16.137 INFO:teuthology.orchestra.run.smithi190.stdout:(38/150): cephfs-top-19.3.0-4560.gd19e6118.el9. 146 kB/s | 18 kB 00:00 2024-08-30T08:47:16.169 INFO:teuthology.orchestra.run.smithi167.stdout:(18/150): libradosstriper1-19.3.0-4560.gd19e611 1.3 MB/s | 506 kB 00:00 2024-08-30T08:47:16.254 INFO:teuthology.orchestra.run.smithi190.stdout:(39/150): fuse-2.9.9-16.el9.x86_64.rpm 563 kB/s | 80 kB 00:00 2024-08-30T08:47:16.261 INFO:teuthology.orchestra.run.smithi167.stdout:(19/150): python3-ceph-argparse-19.3.0-4560.gd1 493 kB/s | 45 kB 00:00 2024-08-30T08:47:16.321 INFO:teuthology.orchestra.run.smithi190.stdout:(40/150): libgfortran-11.5.0-2.el9.x86_64.rpm 4.3 MB/s | 801 kB 00:00 2024-08-30T08:47:16.346 INFO:teuthology.orchestra.run.smithi190.stdout:(41/150): libquadmath-11.5.0-2.el9.x86_64.rpm 2.0 MB/s | 191 kB 00:00 2024-08-30T08:47:16.371 INFO:teuthology.orchestra.run.smithi190.stdout:(42/150): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.9 MB/s | 253 kB 00:00 2024-08-30T08:47:16.413 INFO:teuthology.orchestra.run.smithi190.stdout:(43/150): python3-ply-3.11-14.el9.noarch.rpm 2.5 MB/s | 106 kB 00:00 2024-08-30T08:47:16.453 INFO:teuthology.orchestra.run.smithi167.stdout:(20/150): python3-ceph-common-19.3.0-4560.gd19e 736 kB/s | 141 kB 00:00 2024-08-30T08:47:16.455 INFO:teuthology.orchestra.run.smithi190.stdout:(44/150): python3-cryptography-36.0.1-4.el9.x86 11 MB/s | 1.2 MB 00:00 2024-08-30T08:47:16.480 INFO:teuthology.orchestra.run.smithi190.stdout:(45/150): python3-pycparser-2.20-6.el9.noarch.r 2.0 MB/s | 135 kB 00:00 2024-08-30T08:47:16.505 INFO:teuthology.orchestra.run.smithi190.stdout:(46/150): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-08-30T08:47:16.531 INFO:teuthology.orchestra.run.smithi190.stdout:(47/150): python3-urllib3-1.26.5-6.el9.noarch.r 4.2 MB/s | 215 kB 00:00 2024-08-30T08:47:16.564 INFO:teuthology.orchestra.run.smithi190.stdout:(48/150): automake-1.16.2-8.el9.noarch.rpm 12 MB/s | 689 kB 00:00 2024-08-30T08:47:16.598 INFO:teuthology.orchestra.run.smithi190.stdout:(49/150): bison-3.7.4-5.el9.x86_64.rpm 14 MB/s | 943 kB 00:00 2024-08-30T08:47:16.623 INFO:teuthology.orchestra.run.smithi190.stdout:(50/150): boost-program-options-1.75.0-8.el9.x8 1.8 MB/s | 107 kB 00:00 2024-08-30T08:47:16.645 INFO:teuthology.orchestra.run.smithi167.stdout:(21/150): ceph-common-19.3.0-4560.gd19e6118.el9 3.6 MB/s | 20 MB 00:05 2024-08-30T08:47:16.648 INFO:teuthology.orchestra.run.smithi190.stdout:(51/150): elfutils-libelf-devel-0.191-4.el9.x86 434 kB/s | 22 kB 00:00 2024-08-30T08:47:16.674 INFO:teuthology.orchestra.run.smithi190.stdout:(52/150): flex-2.6.4-9.el9.x86_64.rpm 6.1 MB/s | 313 kB 00:00 2024-08-30T08:47:16.679 INFO:teuthology.orchestra.run.smithi167.stdout:(22/150): python3-cephfs-19.3.0-4560.gd19e6118. 717 kB/s | 161 kB 00:00 2024-08-30T08:47:16.697 INFO:teuthology.orchestra.run.smithi042.stdout:(8/150): ceph-common-19.3.0-4560.gd19e6118.el9. 10 MB/s | 20 MB 00:01 2024-08-30T08:47:16.699 INFO:teuthology.orchestra.run.smithi190.stdout:(53/150): flexiblas-3.0.4-8.el9.x86_64.rpm 648 kB/s | 33 kB 00:00 2024-08-30T08:47:16.741 INFO:teuthology.orchestra.run.smithi190.stdout:(54/150): flexiblas-openblas-openmp-3.0.4-8.el9 438 kB/s | 18 kB 00:00 2024-08-30T08:47:16.754 INFO:teuthology.orchestra.run.smithi167.stdout:(23/150): librgw2-19.3.0-4560.gd19e6118.el9.x86 5.7 MB/s | 5.3 MB 00:00 2024-08-30T08:47:16.755 INFO:teuthology.orchestra.run.smithi042.stdout:(9/150): ceph-selinux-19.3.0-4560.gd19e6118.el9 433 kB/s | 25 kB 00:00 2024-08-30T08:47:16.780 INFO:teuthology.orchestra.run.smithi167.stdout:(24/150): python3-rados-19.3.0-4560.gd19e6118.e 2.3 MB/s | 321 kB 00:00 2024-08-30T08:47:16.792 INFO:teuthology.orchestra.run.smithi190.stdout:(55/150): flexiblas-netlib-3.0.4-8.el9.x86_64.r 25 MB/s | 3.0 MB 00:00 2024-08-30T08:47:16.805 INFO:teuthology.orchestra.run.smithi167.stdout:(25/150): python3-rgw-19.3.0-4560.gd19e6118.el9 1.9 MB/s | 99 kB 00:00 2024-08-30T08:47:16.817 INFO:teuthology.orchestra.run.smithi190.stdout:(56/150): libaio-devel-0.3.111-13.el9.x86_64.rp 148 kB/s | 11 kB 00:00 2024-08-30T08:47:16.830 INFO:teuthology.orchestra.run.smithi167.stdout:(26/150): python3-rbd-19.3.0-4560.gd19e6118.el9 2.0 MB/s | 305 kB 00:00 2024-08-30T08:47:16.842 INFO:teuthology.orchestra.run.smithi190.stdout:(57/150): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 160 kB 00:00 2024-08-30T08:47:16.856 INFO:teuthology.orchestra.run.smithi167.stdout:(27/150): rbd-fuse-19.3.0-4560.gd19e6118.el9.x8 1.1 MB/s | 86 kB 00:00 2024-08-30T08:47:16.868 INFO:teuthology.orchestra.run.smithi190.stdout:(58/150): librabbitmq-0.11.0-7.el9.x86_64.rpm 901 kB/s | 45 kB 00:00 2024-08-30T08:47:16.881 INFO:teuthology.orchestra.run.smithi042.stdout:(10/150): ceph-radosgw-19.3.0-4560.gd19e6118.el 13 MB/s | 10 MB 00:00 2024-08-30T08:47:16.901 INFO:teuthology.orchestra.run.smithi190.stdout:(59/150): librdkafka-1.6.1-102.el9.x86_64.rpm 11 MB/s | 662 kB 00:00 2024-08-30T08:47:16.906 INFO:teuthology.orchestra.run.smithi167.stdout:(28/150): ceph-grafana-dashboards-19.3.0-4560.g 686 kB/s | 34 kB 00:00 2024-08-30T08:47:16.931 INFO:teuthology.orchestra.run.smithi167.stdout:(29/150): rbd-nbd-19.3.0-4560.gd19e6118.el9.x86 2.2 MB/s | 170 kB 00:00 2024-08-30T08:47:16.935 INFO:teuthology.orchestra.run.smithi190.stdout:(60/150): libtool-2.4.6-46.el9.x86_64.rpm 8.5 MB/s | 581 kB 00:00 2024-08-30T08:47:16.956 INFO:teuthology.orchestra.run.smithi042.stdout:(11/150): cephfs-mirror-19.3.0-4560.gd19e6118.e 2.9 MB/s | 226 kB 00:00 2024-08-30T08:47:16.960 INFO:teuthology.orchestra.run.smithi190.stdout:(61/150): libuuid-devel-2.37.4-20.el9.x86_64.rp 418 kB/s | 24 kB 00:00 2024-08-30T08:47:16.984 INFO:teuthology.orchestra.run.smithi167.stdout:(30/150): ceph-mgr-cephadm-19.3.0-4560.gd19e611 2.0 MB/s | 154 kB 00:00 2024-08-30T08:47:16.985 INFO:teuthology.orchestra.run.smithi190.stdout:(62/150): libzstd-devel-1.5.1-2.el9.x86_64.rpm 945 kB/s | 47 kB 00:00 2024-08-30T08:47:17.010 INFO:teuthology.orchestra.run.smithi190.stdout:(63/150): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.7 MB/s | 292 kB 00:00 2024-08-30T08:47:17.014 INFO:teuthology.orchestra.run.smithi042.stdout:(12/150): cephfs-shell-19.3.0-4560.gd19e6118.el 465 kB/s | 27 kB 00:00 2024-08-30T08:47:17.036 INFO:teuthology.orchestra.run.smithi190.stdout:(64/150): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 188 kB 00:00 2024-08-30T08:47:17.053 INFO:teuthology.orchestra.run.smithi190.stdout:(65/150): openblas-0.3.26-2.el9.x86_64.rpm 921 kB/s | 38 kB 00:00 2024-08-30T08:47:17.099 INFO:teuthology.orchestra.run.smithi167.stdout:(31/150): rbd-mirror-19.3.0-4560.gd19e6118.el9. 11 MB/s | 3.1 MB 00:00 2024-08-30T08:47:17.149 INFO:teuthology.orchestra.run.smithi167.stdout:(32/150): ceph-mgr-modules-core-19.3.0-4560.gd1 5.7 MB/s | 290 kB 00:00 2024-08-30T08:47:17.161 INFO:teuthology.orchestra.run.smithi190.stdout:(66/150): openblas-openmp-0.3.26-2.el9.x86_64.r 40 MB/s | 4.9 MB 00:00 2024-08-30T08:47:17.199 INFO:teuthology.orchestra.run.smithi167.stdout:(33/150): ceph-mgr-rook-19.3.0-4560.gd19e6118.e 992 kB/s | 49 kB 00:00 2024-08-30T08:47:17.215 INFO:teuthology.orchestra.run.smithi042.stdout:(13/150): ceph-osd-19.3.0-4560.gd19e6118.el9.x8 11 MB/s | 18 MB 00:01 2024-08-30T08:47:17.228 INFO:teuthology.orchestra.run.smithi190.stdout:(67/150): openssl-devel-3.2.2-4.el9.x86_64.rpm 25 MB/s | 4.4 MB 00:00 2024-08-30T08:47:17.240 INFO:teuthology.orchestra.run.smithi042.stdout:(14/150): libcephfs-devel-19.3.0-4560.gd19e6118 141 kB/s | 32 kB 00:00 2024-08-30T08:47:17.241 INFO:teuthology.orchestra.run.smithi167.stdout:(34/150): ceph-prometheus-alerts-19.3.0-4560.gd 395 kB/s | 16 kB 00:00 2024-08-30T08:47:17.278 INFO:teuthology.orchestra.run.smithi190.stdout:(68/150): postgresql-13.16-1.el9.x86_64.rpm 14 MB/s | 1.6 MB 00:00 2024-08-30T08:47:17.307 INFO:teuthology.orchestra.run.smithi042.stdout:(15/150): libcephfs2-19.3.0-4560.gd19e6118.el9. 8.1 MB/s | 766 kB 00:00 2024-08-30T08:47:17.308 INFO:teuthology.orchestra.run.smithi167.stdout:(35/150): ceph-volume-19.3.0-4560.gd19e6118.el9 4.2 MB/s | 285 kB 00:00 2024-08-30T08:47:17.321 INFO:teuthology.orchestra.run.smithi190.stdout:(69/150): postgresql-contrib-13.16-1.el9.x86_64 9.7 MB/s | 910 kB 00:00 2024-08-30T08:47:17.333 INFO:teuthology.orchestra.run.smithi042.stdout:(16/150): libcephsqlite-19.3.0-4560.gd19e6118.e 1.7 MB/s | 159 kB 00:00 2024-08-30T08:47:17.346 INFO:teuthology.orchestra.run.smithi190.stdout:(70/150): postgresql-private-libs-13.16-1.el9.x 2.0 MB/s | 137 kB 00:00 2024-08-30T08:47:17.374 INFO:teuthology.orchestra.run.smithi042.stdout:(17/150): librados-devel-19.3.0-4560.gd19e6118. 1.9 MB/s | 127 kB 00:00 2024-08-30T08:47:17.396 INFO:teuthology.orchestra.run.smithi190.stdout:(71/150): protobuf-3.14.0-13.el9.x86_64.rpm 20 MB/s | 1.0 MB 00:00 2024-08-30T08:47:17.408 INFO:teuthology.orchestra.run.smithi042.stdout:(18/150): libradosstriper1-19.3.0-4560.gd19e611 6.6 MB/s | 506 kB 00:00 2024-08-30T08:47:17.425 INFO:teuthology.orchestra.run.smithi167.stdout:(36/150): cephadm-19.3.0-4560.gd19e6118.el9.noa 6.6 MB/s | 781 kB 00:00 2024-08-30T08:47:17.438 INFO:teuthology.orchestra.run.smithi190.stdout:(72/150): python3-attrs-20.3.0-7.el9.noarch.rpm 2.1 MB/s | 87 kB 00:00 2024-08-30T08:47:17.484 INFO:teuthology.orchestra.run.smithi042.stdout:(19/150): python3-ceph-argparse-19.3.0-4560.gd1 601 kB/s | 45 kB 00:00 2024-08-30T08:47:17.492 INFO:teuthology.orchestra.run.smithi167.stdout:(37/150): cephfs-top-19.3.0-4560.gd19e6118.el9. 275 kB/s | 18 kB 00:00 2024-08-30T08:47:17.505 INFO:teuthology.orchestra.run.smithi190.stdout:(73/150): postgresql-server-13.16-1.el9.x86_64. 31 MB/s | 5.8 MB 00:00 2024-08-30T08:47:17.547 INFO:teuthology.orchestra.run.smithi190.stdout:(74/150): python3-colorama-0.4.6-3.el9.noarch.r 1.4 MB/s | 61 kB 00:00 2024-08-30T08:47:17.551 INFO:teuthology.orchestra.run.smithi042.stdout:(20/150): python3-ceph-common-19.3.0-4560.gd19e 2.1 MB/s | 141 kB 00:00 2024-08-30T08:47:17.626 INFO:teuthology.orchestra.run.smithi042.stdout:(21/150): python3-cephfs-19.3.0-4560.gd19e6118. 2.1 MB/s | 161 kB 00:00 2024-08-30T08:47:17.630 INFO:teuthology.orchestra.run.smithi190.stdout:(75/150): python3-babel-2.9.1-2.el9.noarch.rpm 31 MB/s | 6.0 MB 00:00 2024-08-30T08:47:17.656 INFO:teuthology.orchestra.run.smithi190.stdout:(76/150): python3-devel-3.9.19-8.el9.x86_64.rpm 2.2 MB/s | 245 kB 00:00 2024-08-30T08:47:17.659 INFO:teuthology.orchestra.run.smithi167.stdout:(38/150): fuse-2.9.9-16.el9.x86_64.rpm 479 kB/s | 80 kB 00:00 2024-08-30T08:47:17.681 INFO:teuthology.orchestra.run.smithi190.stdout:(77/150): python3-jinja2-2.11.3-6.el9.noarch.rp 4.8 MB/s | 248 kB 00:00 2024-08-30T08:47:17.701 INFO:teuthology.orchestra.run.smithi042.stdout:(22/150): python3-rados-19.3.0-4560.gd19e6118.e 4.2 MB/s | 321 kB 00:00 2024-08-30T08:47:17.706 INFO:teuthology.orchestra.run.smithi190.stdout:(78/150): python3-jsonpatch-1.21-16.el9.noarch. 510 kB/s | 26 kB 00:00 2024-08-30T08:47:17.732 INFO:teuthology.orchestra.run.smithi190.stdout:(79/150): python3-jsonpointer-2.0-4.el9.noarch. 382 kB/s | 19 kB 00:00 2024-08-30T08:47:17.757 INFO:teuthology.orchestra.run.smithi190.stdout:(80/150): python3-mako-1.1.4-6.el9.noarch.rpm 3.3 MB/s | 172 kB 00:00 2024-08-30T08:47:17.768 INFO:teuthology.orchestra.run.smithi042.stdout:(23/150): python3-rbd-19.3.0-4560.gd19e6118.el9 4.5 MB/s | 305 kB 00:00 2024-08-30T08:47:17.782 INFO:teuthology.orchestra.run.smithi190.stdout:(81/150): python3-markupsafe-1.1.1-12.el9.x86_6 696 kB/s | 35 kB 00:00 2024-08-30T08:47:17.825 INFO:teuthology.orchestra.run.smithi190.stdout:(82/150): python3-numpy-f2py-1.20.1-5.el9.x86_6 5.8 MB/s | 250 kB 00:00 2024-08-30T08:47:17.843 INFO:teuthology.orchestra.run.smithi042.stdout:(24/150): librgw2-19.3.0-4560.gd19e6118.el9.x86 11 MB/s | 5.3 MB 00:00 2024-08-30T08:47:17.851 INFO:teuthology.orchestra.run.smithi167.stdout:(39/150): ceph-mgr-diskprediction-local-19.3.0- 8.5 MB/s | 7.4 MB 00:00 2024-08-30T08:47:17.869 INFO:teuthology.orchestra.run.smithi042.stdout:(25/150): python3-rgw-19.3.0-4560.gd19e6118.el9 993 kB/s | 99 kB 00:00 2024-08-30T08:47:17.884 INFO:teuthology.orchestra.run.smithi167.stdout:(40/150): libgfortran-11.5.0-2.el9.x86_64.rpm 3.5 MB/s | 801 kB 00:00 2024-08-30T08:47:17.894 INFO:teuthology.orchestra.run.smithi042.stdout:(26/150): rbd-fuse-19.3.0-4560.gd19e6118.el9.x8 1.7 MB/s | 86 kB 00:00 2024-08-30T08:47:17.900 INFO:teuthology.orchestra.run.smithi190.stdout:(83/150): python3-numpy-1.20.1-5.el9.x86_64.rpm 36 MB/s | 5.1 MB 00:00 2024-08-30T08:47:17.925 INFO:teuthology.orchestra.run.smithi190.stdout:(84/150): python3-oauthlib-3.1.1-5.el9.noarch.r 2.2 MB/s | 222 kB 00:00 2024-08-30T08:47:17.943 INFO:teuthology.orchestra.run.smithi167.stdout:(41/150): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.3 MB/s | 253 kB 00:00 2024-08-30T08:47:17.950 INFO:teuthology.orchestra.run.smithi190.stdout:(85/150): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-08-30T08:47:17.961 INFO:teuthology.orchestra.run.smithi042.stdout:(27/150): rbd-nbd-19.3.0-4560.gd19e6118.el9.x86 2.5 MB/s | 170 kB 00:00 2024-08-30T08:47:17.975 INFO:teuthology.orchestra.run.smithi190.stdout:(86/150): python3-prettytable-0.7.2-27.el9.noar 842 kB/s | 42 kB 00:00 2024-08-30T08:47:18.001 INFO:teuthology.orchestra.run.smithi190.stdout:(87/150): python3-protobuf-3.14.0-13.el9.noarch 5.2 MB/s | 269 kB 00:00 2024-08-30T08:47:18.035 INFO:teuthology.orchestra.run.smithi190.stdout:(88/150): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.6 MB/s | 159 kB 00:00 2024-08-30T08:47:18.052 INFO:teuthology.orchestra.run.smithi167.stdout:(42/150): libquadmath-11.5.0-2.el9.x86_64.rpm 954 kB/s | 191 kB 00:00 2024-08-30T08:47:18.053 INFO:teuthology.orchestra.run.smithi042.stdout:(28/150): rbd-mirror-19.3.0-4560.gd19e6118.el9. 17 MB/s | 3.1 MB 00:00 2024-08-30T08:47:18.060 INFO:teuthology.orchestra.run.smithi190.stdout:(89/150): python3-pyasn1-modules-0.4.8-6.el9.no 4.7 MB/s | 279 kB 00:00 2024-08-30T08:47:18.078 INFO:teuthology.orchestra.run.smithi042.stdout:(29/150): ceph-grafana-dashboards-19.3.0-4560.g 293 kB/s | 34 kB 00:00 2024-08-30T08:47:18.085 INFO:teuthology.orchestra.run.smithi190.stdout:(90/150): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-08-30T08:47:18.103 INFO:teuthology.orchestra.run.smithi042.stdout:(30/150): ceph-mgr-cephadm-19.3.0-4560.gd19e611 3.0 MB/s | 154 kB 00:00 2024-08-30T08:47:18.110 INFO:teuthology.orchestra.run.smithi190.stdout:(91/150): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-08-30T08:47:18.111 INFO:teuthology.orchestra.run.smithi167.stdout:(43/150): python3-cryptography-36.0.1-4.el9.x86 7.3 MB/s | 1.2 MB 00:00 2024-08-30T08:47:18.136 INFO:teuthology.orchestra.run.smithi167.stdout:(44/150): python3-ply-3.11-14.el9.noarch.rpm 1.2 MB/s | 106 kB 00:00 2024-08-30T08:47:18.152 INFO:teuthology.orchestra.run.smithi190.stdout:(92/150): python3-toml-0.10.2-6.el9.noarch.rpm 1.0 MB/s | 42 kB 00:00 2024-08-30T08:47:18.169 INFO:teuthology.orchestra.run.smithi167.stdout:(45/150): python3-pycparser-2.20-6.el9.noarch.r 2.3 MB/s | 135 kB 00:00 2024-08-30T08:47:18.194 INFO:teuthology.orchestra.run.smithi190.stdout:(93/150): python3-wcwidth-0.2.5-8.el9.noarch.rp 1.0 MB/s | 44 kB 00:00 2024-08-30T08:47:18.195 INFO:teuthology.orchestra.run.smithi167.stdout:(46/150): python3-requests-2.25.1-8.el9.noarch. 2.1 MB/s | 125 kB 00:00 2024-08-30T08:47:18.236 INFO:teuthology.orchestra.run.smithi167.stdout:(47/150): python3-urllib3-1.26.5-6.el9.noarch.r 3.1 MB/s | 215 kB 00:00 2024-08-30T08:47:18.328 INFO:teuthology.orchestra.run.smithi167.stdout:(48/150): automake-1.16.2-8.el9.noarch.rpm 5.0 MB/s | 689 kB 00:00 2024-08-30T08:47:18.362 INFO:teuthology.orchestra.run.smithi167.stdout:(49/150): bison-3.7.4-5.el9.x86_64.rpm 7.4 MB/s | 943 kB 00:00 2024-08-30T08:47:18.387 INFO:teuthology.orchestra.run.smithi167.stdout:(50/150): boost-program-options-1.75.0-8.el9.x8 1.8 MB/s | 107 kB 00:00 2024-08-30T08:47:18.413 INFO:teuthology.orchestra.run.smithi167.stdout:(51/150): elfutils-libelf-devel-0.191-4.el9.x86 432 kB/s | 22 kB 00:00 2024-08-30T08:47:18.446 INFO:teuthology.orchestra.run.smithi167.stdout:(52/150): flex-2.6.4-9.el9.x86_64.rpm 5.2 MB/s | 313 kB 00:00 2024-08-30T08:47:18.471 INFO:teuthology.orchestra.run.smithi167.stdout:(53/150): flexiblas-3.0.4-8.el9.x86_64.rpm 558 kB/s | 33 kB 00:00 2024-08-30T08:47:18.479 INFO:teuthology.orchestra.run.smithi042.stdout:(31/150): ceph-mgr-diskprediction-local-19.3.0- 20 MB/s | 7.4 MB 00:00 2024-08-30T08:47:18.522 INFO:teuthology.orchestra.run.smithi167.stdout:(54/150): flexiblas-openblas-openmp-3.0.4-8.el9 364 kB/s | 18 kB 00:00 2024-08-30T08:47:18.546 INFO:teuthology.orchestra.run.smithi042.stdout:(32/150): ceph-mgr-modules-core-19.3.0-4560.gd1 4.3 MB/s | 290 kB 00:00 2024-08-30T08:47:18.578 INFO:teuthology.orchestra.run.smithi190.stdout:(94/150): ceph-test-19.3.0-4560.gd19e6118.el9.x 7.0 MB/s | 44 MB 00:06 2024-08-30T08:47:18.589 INFO:teuthology.orchestra.run.smithi167.stdout:(55/150): libaio-devel-0.3.111-13.el9.x86_64.rp 166 kB/s | 11 kB 00:00 2024-08-30T08:47:18.603 INFO:teuthology.orchestra.run.smithi190.stdout:(95/150): qatlib-24.02.0-1.el9.x86_64.rpm 540 kB/s | 221 kB 00:00 2024-08-30T08:47:18.604 INFO:teuthology.orchestra.run.smithi042.stdout:(33/150): ceph-mgr-rook-19.3.0-4560.gd19e6118.e 848 kB/s | 49 kB 00:00 2024-08-30T08:47:18.645 INFO:teuthology.orchestra.run.smithi190.stdout:(96/150): qatzip-libs-1.2.0-1.el9.x86_64.rpm 1.1 MB/s | 47 kB 00:00 2024-08-30T08:47:18.648 INFO:teuthology.orchestra.run.smithi167.stdout:(56/150): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.7 MB/s | 160 kB 00:00 2024-08-30T08:47:18.663 INFO:teuthology.orchestra.run.smithi042.stdout:(34/150): ceph-prometheus-alerts-19.3.0-4560.gd 282 kB/s | 16 kB 00:00 2024-08-30T08:47:18.670 INFO:teuthology.orchestra.run.smithi190.stdout:(97/150): qatlib-service-24.02.0-1.el9.x86_64.r 391 kB/s | 36 kB 00:00 2024-08-30T08:47:18.698 INFO:teuthology.orchestra.run.smithi167.stdout:(57/150): librabbitmq-0.11.0-7.el9.x86_64.rpm 907 kB/s | 45 kB 00:00 2024-08-30T08:47:18.704 INFO:teuthology.orchestra.run.smithi190.stdout:(98/150): socat-1.7.4.1-6.el9.x86_64.rpm 5.1 MB/s | 304 kB 00:00 2024-08-30T08:47:18.730 INFO:teuthology.orchestra.run.smithi042.stdout:(35/150): ceph-volume-19.3.0-4560.gd19e6118.el9 4.2 MB/s | 285 kB 00:00 2024-08-30T08:47:18.737 INFO:teuthology.orchestra.run.smithi190.stdout:(99/150): uuid-1.6.2-55.el9.x86_64.rpm 867 kB/s | 58 kB 00:00 2024-08-30T08:47:18.763 INFO:teuthology.orchestra.run.smithi190.stdout:(100/150): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.6 MB/s | 96 kB 00:00 2024-08-30T08:47:18.788 INFO:teuthology.orchestra.run.smithi190.stdout:(101/150): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-08-30T08:47:18.813 INFO:teuthology.orchestra.run.smithi042.stdout:(36/150): cephadm-19.3.0-4560.gd19e6118.el9.noa 9.2 MB/s | 781 kB 00:00 2024-08-30T08:47:18.814 INFO:teuthology.orchestra.run.smithi190.stdout:(102/150): zlib-devel-1.2.11-41.el9.x86_64.rpm 893 kB/s | 45 kB 00:00 2024-08-30T08:47:18.815 INFO:teuthology.orchestra.run.smithi167.stdout:(58/150): librdkafka-1.6.1-102.el9.x86_64.rpm 5.6 MB/s | 662 kB 00:00 2024-08-30T08:47:18.881 INFO:teuthology.orchestra.run.smithi042.stdout:(37/150): cephfs-top-19.3.0-4560.gd19e6118.el9. 273 kB/s | 18 kB 00:00 2024-08-30T08:47:18.972 INFO:teuthology.orchestra.run.smithi190.stdout:(103/150): python3-scipy-1.6.2-8.el9.x86_64.rpm 18 MB/s | 16 MB 00:00 2024-08-30T08:47:18.990 INFO:teuthology.orchestra.run.smithi167.stdout:(59/150): libtool-2.4.6-46.el9.x86_64.rpm 3.2 MB/s | 581 kB 00:00 2024-08-30T08:47:19.030 INFO:teuthology.orchestra.run.smithi190.stdout:(104/150): abseil-cpp-20211102.0-3.el9.x86_64.r 9.3 MB/s | 548 kB 00:00 2024-08-30T08:47:19.031 INFO:teuthology.orchestra.run.smithi042.stdout:(38/150): ceph-mgr-dashboard-19.3.0-4560.gd19e6 15 MB/s | 14 MB 00:00 2024-08-30T08:47:19.041 INFO:teuthology.orchestra.run.smithi167.stdout:(60/150): libuuid-devel-2.37.4-20.el9.x86_64.rp 490 kB/s | 24 kB 00:00 2024-08-30T08:47:19.069 INFO:teuthology.orchestra.run.smithi190.stdout:(105/150): gperftools-libs-2.9.1-2.el9.x86_64.r 7.9 MB/s | 309 kB 00:00 2024-08-30T08:47:19.086 INFO:teuthology.orchestra.run.smithi190.stdout:(106/150): grpc-data-1.46.7-10.el9.noarch.rpm 1.1 MB/s | 19 kB 00:00 2024-08-30T08:47:19.091 INFO:teuthology.orchestra.run.smithi167.stdout:(61/150): libzstd-devel-1.5.1-2.el9.x86_64.rpm 948 kB/s | 47 kB 00:00 2024-08-30T08:47:19.136 INFO:teuthology.orchestra.run.smithi190.stdout:(107/150): lua-devel-5.4.4-4.el9.x86_64.rpm 64 kB/s | 22 kB 00:00 2024-08-30T08:47:19.175 INFO:teuthology.orchestra.run.smithi167.stdout:(62/150): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.4 MB/s | 292 kB 00:00 2024-08-30T08:47:19.212 INFO:teuthology.orchestra.run.smithi190.stdout:(108/150): libarrow-9.0.0-11.el9.x86_64.rpm 35 MB/s | 4.4 MB 00:00 2024-08-30T08:47:19.229 INFO:teuthology.orchestra.run.smithi190.stdout:(109/150): liboath-2.6.7-2.el9.x86_64.rpm 3.2 MB/s | 49 kB 00:00 2024-08-30T08:47:19.233 INFO:teuthology.orchestra.run.smithi167.stdout:(63/150): lua-5.4.4-4.el9.x86_64.rpm 3.2 MB/s | 188 kB 00:00 2024-08-30T08:47:19.246 INFO:teuthology.orchestra.run.smithi190.stdout:(110/150): libarrow-doc-9.0.0-11.el9.noarch.rpm 238 kB/s | 26 kB 00:00 2024-08-30T08:47:19.271 INFO:teuthology.orchestra.run.smithi190.stdout:(111/150): libunwind-1.6.2-1.el9.x86_64.rpm 1.6 MB/s | 67 kB 00:00 2024-08-30T08:47:19.283 INFO:teuthology.orchestra.run.smithi167.stdout:(64/150): openblas-0.3.26-2.el9.x86_64.rpm 772 kB/s | 38 kB 00:00 2024-08-30T08:47:19.304 INFO:teuthology.orchestra.run.smithi190.stdout:(112/150): luarocks-3.9.2-5.el9.noarch.rpm 2.5 MB/s | 151 kB 00:00 2024-08-30T08:47:19.338 INFO:teuthology.orchestra.run.smithi190.stdout:(113/150): protobuf-compiler-3.14.0-13.el9.x86_ 1.6 MB/s | 863 kB 00:00 2024-08-30T08:47:19.388 INFO:teuthology.orchestra.run.smithi190.stdout:(114/150): parquet-libs-9.0.0-11.el9.x86_64.rpm 7.0 MB/s | 839 kB 00:00 2024-08-30T08:47:19.413 INFO:teuthology.orchestra.run.smithi190.stdout:(115/150): python3-asyncssh-2.13.2-5.el9.noarch 4.9 MB/s | 548 kB 00:00 2024-08-30T08:47:19.439 INFO:teuthology.orchestra.run.smithi190.stdout:(116/150): python3-cachetools-4.2.4-1.el9.noarc 666 kB/s | 32 kB 00:00 2024-08-30T08:47:19.456 INFO:teuthology.orchestra.run.smithi042.stdout:(39/150): ceph-test-19.3.0-4560.gd19e6118.el9.x 16 MB/s | 44 MB 00:02 2024-08-30T08:47:19.464 INFO:teuthology.orchestra.run.smithi190.stdout:(117/150): python3-bcrypt-3.2.2-1.el9.x86_64.rp 346 kB/s | 43 kB 00:00 2024-08-30T08:47:19.481 INFO:teuthology.orchestra.run.smithi042.stdout:(40/150): fuse-2.9.9-16.el9.x86_64.rpm 133 kB/s | 80 kB 00:00 2024-08-30T08:47:19.489 INFO:teuthology.orchestra.run.smithi190.stdout:(118/150): python3-certifi-2023.05.07-4.el9.noa 187 kB/s | 14 kB 00:00 2024-08-30T08:47:19.514 INFO:teuthology.orchestra.run.smithi190.stdout:(119/150): python3-cheroot-8.6.0-4.el9.noarch.r 2.2 MB/s | 172 kB 00:00 2024-08-30T08:47:19.548 INFO:teuthology.orchestra.run.smithi190.stdout:(120/150): python3-cherrypy-18.6.1-2.el9.noarch 4.2 MB/s | 358 kB 00:00 2024-08-30T08:47:19.573 INFO:teuthology.orchestra.run.smithi190.stdout:(121/150): python3-google-auth-2.34.0-1.el9.noa 3.7 MB/s | 220 kB 00:00 2024-08-30T08:47:19.598 INFO:teuthology.orchestra.run.smithi190.stdout:(122/150): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.7 MB/s | 303 kB 00:00 2024-08-30T08:47:19.624 INFO:teuthology.orchestra.run.smithi190.stdout:(123/150): python3-grpcio-tools-1.46.7-10.el9.x 2.8 MB/s | 144 kB 00:00 2024-08-30T08:47:19.649 INFO:teuthology.orchestra.run.smithi190.stdout:(124/150): python3-jaraco-8.2.1-3.el9.noarch.rp 213 kB/s | 11 kB 00:00 2024-08-30T08:47:19.651 INFO:teuthology.orchestra.run.smithi167.stdout:(65/150): flexiblas-netlib-3.0.4-8.el9.x86_64.r 2.5 MB/s | 3.0 MB 00:01 2024-08-30T08:47:19.657 INFO:teuthology.orchestra.run.smithi042.stdout:(41/150): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.4 MB/s | 253 kB 00:00 2024-08-30T08:47:19.674 INFO:teuthology.orchestra.run.smithi190.stdout:(125/150): python3-jaraco-classes-3.2.1-5.el9.n 352 kB/s | 18 kB 00:00 2024-08-30T08:47:19.682 INFO:teuthology.orchestra.run.smithi042.stdout:(42/150): libquadmath-11.5.0-2.el9.x86_64.rpm 849 kB/s | 191 kB 00:00 2024-08-30T08:47:19.700 INFO:teuthology.orchestra.run.smithi190.stdout:(126/150): python3-jaraco-collections-3.0.0-8.e 460 kB/s | 23 kB 00:00 2024-08-30T08:47:19.725 INFO:teuthology.orchestra.run.smithi190.stdout:(127/150): python3-jaraco-functools-3.5.0-2.el9 386 kB/s | 19 kB 00:00 2024-08-30T08:47:19.732 INFO:teuthology.orchestra.run.smithi042.stdout:(43/150): python3-ply-3.11-14.el9.noarch.rpm 2.1 MB/s | 106 kB 00:00 2024-08-30T08:47:19.750 INFO:teuthology.orchestra.run.smithi190.stdout:(128/150): python3-jaraco-text-3.2.0-6.el9.noar 392 kB/s | 20 kB 00:00 2024-08-30T08:47:19.757 INFO:teuthology.orchestra.run.smithi042.stdout:(44/150): libgfortran-11.5.0-2.el9.x86_64.rpm 1.1 MB/s | 801 kB 00:00 2024-08-30T08:47:19.775 INFO:teuthology.orchestra.run.smithi190.stdout:(129/150): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-08-30T08:47:19.782 INFO:teuthology.orchestra.run.smithi042.stdout:(45/150): python3-pycparser-2.20-6.el9.noarch.r 2.6 MB/s | 135 kB 00:00 2024-08-30T08:47:19.800 INFO:teuthology.orchestra.run.smithi042.stdout:(46/150): python3-requests-2.25.1-8.el9.noarch. 2.9 MB/s | 125 kB 00:00 2024-08-30T08:47:19.801 INFO:teuthology.orchestra.run.smithi190.stdout:(130/150): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-08-30T08:47:19.826 INFO:teuthology.orchestra.run.smithi190.stdout:(131/150): python3-natsort-7.1.1-5.el9.noarch.r 2.3 MB/s | 58 kB 00:00 2024-08-30T08:47:19.842 INFO:teuthology.orchestra.run.smithi042.stdout:(47/150): python3-cryptography-36.0.1-4.el9.x86 6.6 MB/s | 1.2 MB 00:00 2024-08-30T08:47:19.860 INFO:teuthology.orchestra.run.smithi190.stdout:(132/150): python3-kubernetes-26.1.0-2.el9.noar 7.6 MB/s | 1.0 MB 00:00 2024-08-30T08:47:19.867 INFO:teuthology.orchestra.run.smithi042.stdout:(48/150): python3-urllib3-1.26.5-6.el9.noarch.r 2.5 MB/s | 215 kB 00:00 2024-08-30T08:47:19.885 INFO:teuthology.orchestra.run.smithi190.stdout:(133/150): python3-portend-3.1.0-2.el9.noarch.r 659 kB/s | 16 kB 00:00 2024-08-30T08:47:19.910 INFO:teuthology.orchestra.run.smithi190.stdout:(134/150): python3-pecan-1.4.2-3.el9.noarch.rpm 3.2 MB/s | 272 kB 00:00 2024-08-30T08:47:19.935 INFO:teuthology.orchestra.run.smithi190.stdout:(135/150): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.8 MB/s | 90 kB 00:00 2024-08-30T08:47:19.961 INFO:teuthology.orchestra.run.smithi190.stdout:(136/150): python3-pyperclip-1.8.2-2.el9.noarch 563 kB/s | 28 kB 00:00 2024-08-30T08:47:19.986 INFO:teuthology.orchestra.run.smithi190.stdout:(137/150): python3-repoze-lru-0.7-16.el9.noarch 613 kB/s | 31 kB 00:00 2024-08-30T08:47:20.012 INFO:teuthology.orchestra.run.smithi190.stdout:(138/150): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-08-30T08:47:20.037 INFO:teuthology.orchestra.run.smithi190.stdout:(139/150): python3-routes-2.5.1-5.el9.noarch.rp 2.4 MB/s | 188 kB 00:00 2024-08-30T08:47:20.062 INFO:teuthology.orchestra.run.smithi190.stdout:(140/150): python3-tempora-5.0.0-2.el9.noarch.r 711 kB/s | 36 kB 00:00 2024-08-30T08:47:20.087 INFO:teuthology.orchestra.run.smithi190.stdout:(141/150): python3-typing-extensions-4.12.2-2.e 1.5 MB/s | 74 kB 00:00 2024-08-30T08:47:20.109 INFO:teuthology.orchestra.run.smithi167.stdout:(66/150): ceph-mgr-dashboard-19.3.0-4560.gd19e6 4.4 MB/s | 14 MB 00:03 2024-08-30T08:47:20.138 INFO:teuthology.orchestra.run.smithi190.stdout:(142/150): python3-grpcio-1.46.7-10.el9.x86_64. 3.5 MB/s | 2.0 MB 00:00 2024-08-30T08:47:20.163 INFO:teuthology.orchestra.run.smithi190.stdout:(143/150): python3-websocket-client-1.2.3-2.el9 1.2 MB/s | 90 kB 00:00 2024-08-30T08:47:20.197 INFO:teuthology.orchestra.run.smithi190.stdout:(144/150): python3-webob-1.8.8-2.el9.noarch.rpm 1.7 MB/s | 230 kB 00:00 2024-08-30T08:47:20.222 INFO:teuthology.orchestra.run.smithi190.stdout:(145/150): python3-zc-lockfile-2.0-10.el9.noarc 341 kB/s | 20 kB 00:00 2024-08-30T08:47:20.247 INFO:teuthology.orchestra.run.smithi190.stdout:(146/150): re2-20211101-20.el9.x86_64.rpm 3.7 MB/s | 191 kB 00:00 2024-08-30T08:47:20.272 INFO:teuthology.orchestra.run.smithi190.stdout:(147/150): python3-werkzeug-2.0.3-3.el9.1.noarc 3.1 MB/s | 427 kB 00:00 2024-08-30T08:47:20.285 INFO:teuthology.orchestra.run.smithi167.stdout:(67/150): openblas-openmp-0.3.26-2.el9.x86_64.r 4.9 MB/s | 4.9 MB 00:01 2024-08-30T08:47:20.322 INFO:teuthology.orchestra.run.smithi190.stdout:(148/150): thrift-0.15.0-2.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-08-30T08:47:20.375 INFO:teuthology.orchestra.run.smithi042.stdout:(49/150): boost-program-options-1.75.0-8.el9.x8 210 kB/s | 107 kB 00:00 2024-08-30T08:47:20.484 INFO:teuthology.orchestra.run.smithi042.stdout:(50/150): elfutils-libelf-devel-0.191-4.el9.x86 200 kB/s | 22 kB 00:00 2024-08-30T08:47:20.493 INFO:teuthology.orchestra.run.smithi167.stdout:(68/150): postgresql-contrib-13.16-1.el9.x86_64 4.3 MB/s | 910 kB 00:00 2024-08-30T08:47:20.544 INFO:teuthology.orchestra.run.smithi167.stdout:(69/150): postgresql-private-libs-13.16-1.el9.x 2.7 MB/s | 137 kB 00:00 2024-08-30T08:47:20.559 INFO:teuthology.orchestra.run.smithi042.stdout:(51/150): automake-1.16.2-8.el9.noarch.rpm 907 kB/s | 689 kB 00:00 2024-08-30T08:47:20.593 INFO:teuthology.orchestra.run.smithi042.stdout:(52/150): bison-3.7.4-5.el9.x86_64.rpm 1.2 MB/s | 943 kB 00:00 2024-08-30T08:47:20.606 INFO:teuthology.orchestra.run.smithi190.stdout:(149/150): librados2-19.3.0-4560.gd19e6118.el9. 9.4 MB/s | 3.4 MB 00:00 2024-08-30T08:47:20.643 INFO:teuthology.orchestra.run.smithi167.stdout:(70/150): postgresql-13.16-1.el9.x86_64.rpm 3.0 MB/s | 1.6 MB 00:00 2024-08-30T08:47:20.660 INFO:teuthology.orchestra.run.smithi042.stdout:(53/150): flexiblas-3.0.4-8.el9.x86_64.rpm 326 kB/s | 33 kB 00:00 2024-08-30T08:47:20.665 INFO:teuthology.orchestra.run.smithi190.stdout:(150/150): librbd1-19.3.0-4560.gd19e6118.el9.x8 8.0 MB/s | 3.1 MB 00:00 2024-08-30T08:47:20.671 INFO:teuthology.orchestra.run.smithi190.stdout:-------------------------------------------------------------------------------- 2024-08-30T08:47:20.671 INFO:teuthology.orchestra.run.smithi190.stdout:Total 21 MB/s | 224 MB 00:10 2024-08-30T08:47:20.760 INFO:teuthology.orchestra.run.smithi042.stdout:(54/150): flexiblas-openblas-openmp-3.0.4-8.el9 182 kB/s | 18 kB 00:00 2024-08-30T08:47:20.827 INFO:teuthology.orchestra.run.smithi042.stdout:(55/150): flex-2.6.4-9.el9.x86_64.rpm 915 kB/s | 313 kB 00:00 2024-08-30T08:47:20.861 INFO:teuthology.orchestra.run.smithi042.stdout:(56/150): libaio-devel-0.3.111-13.el9.x86_64.rp 111 kB/s | 11 kB 00:00 2024-08-30T08:47:20.936 INFO:teuthology.orchestra.run.smithi042.stdout:(57/150): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.4 MB/s | 160 kB 00:00 2024-08-30T08:47:20.970 INFO:teuthology.orchestra.run.smithi042.stdout:(58/150): librabbitmq-0.11.0-7.el9.x86_64.rpm 417 kB/s | 45 kB 00:00 2024-08-30T08:47:21.169 INFO:teuthology.orchestra.run.smithi167.stdout:(71/150): protobuf-3.14.0-13.el9.x86_64.rpm 1.9 MB/s | 1.0 MB 00:00 2024-08-30T08:47:21.212 INFO:teuthology.orchestra.run.smithi042.stdout:(59/150): librdkafka-1.6.1-102.el9.x86_64.rpm 2.3 MB/s | 662 kB 00:00 2024-08-30T08:47:21.245 INFO:teuthology.orchestra.run.smithi042.stdout:(60/150): libtool-2.4.6-46.el9.x86_64.rpm 2.1 MB/s | 581 kB 00:00 2024-08-30T08:47:21.253 INFO:teuthology.orchestra.run.smithi167.stdout:(72/150): python3-attrs-20.3.0-7.el9.noarch.rpm 1.0 MB/s | 87 kB 00:00 2024-08-30T08:47:21.304 INFO:teuthology.orchestra.run.smithi042.stdout:(61/150): flexiblas-netlib-3.0.4-8.el9.x86_64.r 4.2 MB/s | 3.0 MB 00:00 2024-08-30T08:47:21.329 INFO:teuthology.orchestra.run.smithi042.stdout:(62/150): libuuid-devel-2.37.4-20.el9.x86_64.rp 209 kB/s | 24 kB 00:00 2024-08-30T08:47:21.354 INFO:teuthology.orchestra.run.smithi042.stdout:(63/150): libzstd-devel-1.5.1-2.el9.x86_64.rpm 435 kB/s | 47 kB 00:00 2024-08-30T08:47:21.438 INFO:teuthology.orchestra.run.smithi042.stdout:(64/150): lua-5.4.4-4.el9.x86_64.rpm 1.7 MB/s | 188 kB 00:00 2024-08-30T08:47:21.547 INFO:teuthology.orchestra.run.smithi042.stdout:(65/150): openblas-0.3.26-2.el9.x86_64.rpm 200 kB/s | 38 kB 00:00 2024-08-30T08:47:21.553 INFO:teuthology.orchestra.run.smithi167.stdout:(73/150): postgresql-server-13.16-1.el9.x86_64. 5.7 MB/s | 5.8 MB 00:01 2024-08-30T08:47:21.603 INFO:teuthology.orchestra.run.smithi167.stdout:(74/150): python3-colorama-0.4.6-3.el9.noarch.r 1.2 MB/s | 61 kB 00:00 2024-08-30T08:47:21.662 INFO:teuthology.orchestra.run.smithi167.stdout:(75/150): python3-devel-3.9.19-8.el9.x86_64.rpm 4.1 MB/s | 245 kB 00:00 2024-08-30T08:47:21.720 INFO:teuthology.orchestra.run.smithi167.stdout:(76/150): python3-jinja2-2.11.3-6.el9.noarch.rp 4.2 MB/s | 248 kB 00:00 2024-08-30T08:47:21.726 INFO:teuthology.orchestra.run.smithi190.stdout:Running transaction check 2024-08-30T08:47:21.730 INFO:teuthology.orchestra.run.smithi042.stdout:(66/150): lttng-ust-2.12.0-6.el9.x86_64.rpm 686 kB/s | 292 kB 00:00 2024-08-30T08:47:21.787 INFO:teuthology.orchestra.run.smithi167.stdout:(77/150): python3-jsonpatch-1.21-16.el9.noarch. 384 kB/s | 26 kB 00:00 2024-08-30T08:47:21.838 INFO:teuthology.orchestra.run.smithi167.stdout:(78/150): python3-jsonpointer-2.0-4.el9.noarch. 383 kB/s | 19 kB 00:00 2024-08-30T08:47:21.852 INFO:teuthology.orchestra.run.smithi190.stdout:Transaction check succeeded. 2024-08-30T08:47:21.852 INFO:teuthology.orchestra.run.smithi190.stdout:Running transaction test 2024-08-30T08:47:21.897 INFO:teuthology.orchestra.run.smithi167.stdout:(79/150): python3-mako-1.1.4-6.el9.noarch.rpm 2.9 MB/s | 172 kB 00:00 2024-08-30T08:47:21.947 INFO:teuthology.orchestra.run.smithi167.stdout:(80/150): python3-markupsafe-1.1.1-12.el9.x86_6 697 kB/s | 35 kB 00:00 2024-08-30T08:47:22.106 INFO:teuthology.orchestra.run.smithi167.stdout:(81/150): openssl-devel-3.2.2-4.el9.x86_64.rpm 1.8 MB/s | 4.4 MB 00:02 2024-08-30T08:47:22.256 INFO:teuthology.orchestra.run.smithi167.stdout:(82/150): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.6 MB/s | 250 kB 00:00 2024-08-30T08:47:22.398 INFO:teuthology.orchestra.run.smithi167.stdout:(83/150): python3-oauthlib-3.1.1-5.el9.noarch.r 1.5 MB/s | 222 kB 00:00 2024-08-30T08:47:22.498 INFO:teuthology.orchestra.run.smithi167.stdout:(84/150): python3-packaging-20.9-5.el9.noarch.r 771 kB/s | 77 kB 00:00 2024-08-30T08:47:22.557 INFO:teuthology.orchestra.run.smithi167.stdout:(85/150): python3-prettytable-0.7.2-27.el9.noar 727 kB/s | 42 kB 00:00 2024-08-30T08:47:22.724 INFO:teuthology.orchestra.run.smithi167.stdout:(86/150): python3-numpy-1.20.1-5.el9.x86_64.rpm 6.6 MB/s | 5.1 MB 00:00 2024-08-30T08:47:22.774 INFO:teuthology.orchestra.run.smithi167.stdout:(87/150): python3-protobuf-3.14.0-13.el9.noarch 1.2 MB/s | 269 kB 00:00 2024-08-30T08:47:22.799 INFO:teuthology.orchestra.run.smithi167.stdout:(88/150): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.1 MB/s | 159 kB 00:00 2024-08-30T08:47:22.850 INFO:teuthology.orchestra.run.smithi167.stdout:(89/150): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-08-30T08:47:22.900 INFO:teuthology.orchestra.run.smithi167.stdout:(90/150): python3-requests-oauthlib-1.3.0-12.el 1.1 MB/s | 54 kB 00:00 2024-08-30T08:47:22.967 INFO:teuthology.orchestra.run.smithi167.stdout:(91/150): python3-pyasn1-modules-0.4.8-6.el9.no 1.4 MB/s | 279 kB 00:00 2024-08-30T08:47:23.026 INFO:teuthology.orchestra.run.smithi167.stdout:(92/150): python3-toml-0.10.2-6.el9.noarch.rpm 713 kB/s | 42 kB 00:00 2024-08-30T08:47:23.084 INFO:teuthology.orchestra.run.smithi167.stdout:(93/150): python3-wcwidth-0.2.5-8.el9.noarch.rp 760 kB/s | 44 kB 00:00 2024-08-30T08:47:23.243 INFO:teuthology.orchestra.run.smithi167.stdout:(94/150): qatlib-24.02.0-1.el9.x86_64.rpm 1.4 MB/s | 221 kB 00:00 2024-08-30T08:47:23.289 INFO:teuthology.orchestra.run.smithi042.stdout:(67/150): openblas-openmp-0.3.26-2.el9.x86_64.r 2.7 MB/s | 4.9 MB 00:01 2024-08-30T08:47:23.310 INFO:teuthology.orchestra.run.smithi167.stdout:(95/150): qatlib-service-24.02.0-1.el9.x86_64.r 540 kB/s | 36 kB 00:00 2024-08-30T08:47:23.385 INFO:teuthology.orchestra.run.smithi167.stdout:(96/150): qatzip-libs-1.2.0-1.el9.x86_64.rpm 632 kB/s | 47 kB 00:00 2024-08-30T08:47:23.560 INFO:teuthology.orchestra.run.smithi167.stdout:(97/150): socat-1.7.4.1-6.el9.x86_64.rpm 1.7 MB/s | 304 kB 00:00 2024-08-30T08:47:23.597 INFO:teuthology.orchestra.run.smithi190.stdout:Transaction test succeeded. 2024-08-30T08:47:23.597 INFO:teuthology.orchestra.run.smithi190.stdout:Running transaction 2024-08-30T08:47:23.619 INFO:teuthology.orchestra.run.smithi167.stdout:(98/150): uuid-1.6.2-55.el9.x86_64.rpm 994 kB/s | 58 kB 00:00 2024-08-30T08:47:23.711 INFO:teuthology.orchestra.run.smithi167.stdout:(99/150): xfsprogs-devel-6.4.0-4.el9.x86_64.rpm 1.0 MB/s | 96 kB 00:00 2024-08-30T08:47:23.796 INFO:teuthology.orchestra.run.smithi167.stdout:(100/150): xmlstarlet-1.6.1-20.el9.x86_64.rpm 750 kB/s | 64 kB 00:00 2024-08-30T08:47:23.846 INFO:teuthology.orchestra.run.smithi167.stdout:(101/150): zlib-devel-1.2.11-41.el9.x86_64.rpm 897 kB/s | 45 kB 00:00 2024-08-30T08:47:23.864 INFO:teuthology.orchestra.run.smithi042.stdout:(68/150): postgresql-13.16-1.el9.x86_64.rpm 761 kB/s | 1.6 MB 00:02 2024-08-30T08:47:23.930 INFO:teuthology.orchestra.run.smithi167.stdout:(102/150): lua-devel-5.4.4-4.el9.x86_64.rpm 269 kB/s | 22 kB 00:00 2024-08-30T08:47:24.055 INFO:teuthology.orchestra.run.smithi167.stdout:(103/150): protobuf-compiler-3.14.0-13.el9.x86_ 6.7 MB/s | 863 kB 00:00 2024-08-30T08:47:24.106 INFO:teuthology.orchestra.run.smithi167.stdout:(104/150): abseil-cpp-20211102.0-3.el9.x86_64.r 11 MB/s | 548 kB 00:00 2024-08-30T08:47:24.131 INFO:teuthology.orchestra.run.smithi042.stdout:(69/150): postgresql-private-libs-13.16-1.el9.x 515 kB/s | 137 kB 00:00 2024-08-30T08:47:24.139 INFO:teuthology.orchestra.run.smithi167.stdout:(105/150): gperftools-libs-2.9.1-2.el9.x86_64.r 9.1 MB/s | 309 kB 00:00 2024-08-30T08:47:24.190 INFO:teuthology.orchestra.run.smithi167.stdout:(106/150): grpc-data-1.46.7-10.el9.noarch.rpm 388 kB/s | 19 kB 00:00 2024-08-30T08:47:24.298 INFO:teuthology.orchestra.run.smithi167.stdout:(107/150): libarrow-9.0.0-11.el9.x86_64.rpm 41 MB/s | 4.4 MB 00:00 2024-08-30T08:47:24.381 INFO:teuthology.orchestra.run.smithi167.stdout:(108/150): python3-babel-2.9.1-2.el9.noarch.rpm 1.9 MB/s | 6.0 MB 00:03 2024-08-30T08:47:24.407 INFO:teuthology.orchestra.run.smithi167.stdout:(109/150): libarrow-doc-9.0.0-11.el9.noarch.rpm 239 kB/s | 26 kB 00:00 2024-08-30T08:47:24.440 INFO:teuthology.orchestra.run.smithi167.stdout:(110/150): libunwind-1.6.2-1.el9.x86_64.rpm 2.5 MB/s | 67 kB 00:00 2024-08-30T08:47:24.466 INFO:teuthology.orchestra.run.smithi167.stdout:(111/150): luarocks-3.9.2-5.el9.noarch.rpm 6.4 MB/s | 151 kB 00:00 2024-08-30T08:47:24.491 INFO:teuthology.orchestra.run.smithi167.stdout:(112/150): liboath-2.6.7-2.el9.x86_64.rpm 444 kB/s | 49 kB 00:00 2024-08-30T08:47:24.533 INFO:teuthology.orchestra.run.smithi167.stdout:(113/150): parquet-libs-9.0.0-11.el9.x86_64.rpm 12 MB/s | 839 kB 00:00 2024-08-30T08:47:24.539 INFO:teuthology.orchestra.run.smithi042.stdout:(70/150): postgresql-contrib-13.16-1.el9.x86_64 728 kB/s | 910 kB 00:01 2024-08-30T08:47:24.558 INFO:teuthology.orchestra.run.smithi167.stdout:(114/150): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-08-30T08:47:24.584 INFO:teuthology.orchestra.run.smithi167.stdout:(115/150): python3-asyncssh-2.13.2-5.el9.noarch 5.8 MB/s | 548 kB 00:00 2024-08-30T08:47:24.609 INFO:teuthology.orchestra.run.smithi167.stdout:(116/150): python3-cachetools-4.2.4-1.el9.noarc 642 kB/s | 32 kB 00:00 2024-08-30T08:47:24.634 INFO:teuthology.orchestra.run.smithi167.stdout:(117/150): python3-certifi-2023.05.07-4.el9.noa 280 kB/s | 14 kB 00:00 2024-08-30T08:47:24.659 INFO:teuthology.orchestra.run.smithi167.stdout:(118/150): python3-cheroot-8.6.0-4.el9.noarch.r 3.3 MB/s | 172 kB 00:00 2024-08-30T08:47:24.693 INFO:teuthology.orchestra.run.smithi167.stdout:(119/150): python3-cherrypy-18.6.1-2.el9.noarch 6.0 MB/s | 358 kB 00:00 2024-08-30T08:47:24.715 INFO:teuthology.orchestra.run.smithi042.stdout:(71/150): openssl-devel-3.2.2-4.el9.x86_64.rpm 1.4 MB/s | 4.4 MB 00:03 2024-08-30T08:47:24.719 INFO:teuthology.orchestra.run.smithi167.stdout:(120/150): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.0 MB/s | 303 kB 00:00 2024-08-30T08:47:24.744 INFO:teuthology.orchestra.run.smithi167.stdout:(121/150): python3-google-auth-2.34.0-1.el9.noa 4.3 MB/s | 220 kB 00:00 2024-08-30T08:47:24.778 INFO:teuthology.orchestra.run.smithi167.stdout:(122/150): python3-grpcio-tools-1.46.7-10.el9.x 4.2 MB/s | 144 kB 00:00 2024-08-30T08:47:24.803 INFO:teuthology.orchestra.run.smithi167.stdout:(123/150): python3-jaraco-8.2.1-3.el9.noarch.rp 424 kB/s | 11 kB 00:00 2024-08-30T08:47:24.828 INFO:teuthology.orchestra.run.smithi167.stdout:(124/150): python3-jaraco-classes-3.2.1-5.el9.n 706 kB/s | 18 kB 00:00 2024-08-30T08:47:24.854 INFO:teuthology.orchestra.run.smithi167.stdout:(125/150): python3-jaraco-collections-3.0.0-8.e 925 kB/s | 23 kB 00:00 2024-08-30T08:47:24.879 INFO:teuthology.orchestra.run.smithi167.stdout:(126/150): python3-jaraco-functools-3.5.0-2.el9 775 kB/s | 19 kB 00:00 2024-08-30T08:47:24.905 INFO:teuthology.orchestra.run.smithi167.stdout:(127/150): python3-jaraco-text-3.2.0-6.el9.noar 782 kB/s | 20 kB 00:00 2024-08-30T08:47:24.906 INFO:teuthology.orchestra.run.smithi042.stdout:(72/150): python3-attrs-20.3.0-7.el9.noarch.rpm 457 kB/s | 87 kB 00:00 2024-08-30T08:47:24.963 INFO:teuthology.orchestra.run.smithi167.stdout:(128/150): python3-grpcio-1.46.7-10.el9.x86_64. 8.3 MB/s | 2.0 MB 00:00 2024-08-30T08:47:25.122 INFO:teuthology.orchestra.run.smithi167.stdout:(129/150): python3-scipy-1.6.2-8.el9.x86_64.rpm 7.3 MB/s | 16 MB 00:02 2024-08-30T08:47:25.147 INFO:teuthology.orchestra.run.smithi167.stdout:(130/150): python3-logutils-0.3.5-21.el9.noarch 252 kB/s | 46 kB 00:00 2024-08-30T08:47:25.181 INFO:teuthology.orchestra.run.smithi167.stdout:(131/150): python3-natsort-7.1.1-5.el9.noarch.r 2.2 MB/s | 58 kB 00:00 2024-08-30T08:47:25.215 INFO:teuthology.orchestra.run.smithi167.stdout:(132/150): python3-kubernetes-26.1.0-2.el9.noar 3.3 MB/s | 1.0 MB 00:00 2024-08-30T08:47:25.240 INFO:teuthology.orchestra.run.smithi167.stdout:(133/150): python3-portend-3.1.0-2.el9.noarch.r 658 kB/s | 16 kB 00:00 2024-08-30T08:47:25.265 INFO:teuthology.orchestra.run.smithi167.stdout:(134/150): python3-more-itertools-8.12.0-2.el9. 553 kB/s | 79 kB 00:00 2024-08-30T08:47:25.291 INFO:teuthology.orchestra.run.smithi167.stdout:(135/150): python3-pyperclip-1.8.2-2.el9.noarch 1.1 MB/s | 28 kB 00:00 2024-08-30T08:47:25.316 INFO:teuthology.orchestra.run.smithi167.stdout:(136/150): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.2 MB/s | 90 kB 00:00 2024-08-30T08:47:25.341 INFO:teuthology.orchestra.run.smithi167.stdout:(137/150): python3-repoze-lru-0.7-16.el9.noarch 613 kB/s | 31 kB 00:00 2024-08-30T08:47:25.367 INFO:teuthology.orchestra.run.smithi167.stdout:(138/150): python3-pecan-1.4.2-3.el9.noarch.rpm 1.5 MB/s | 272 kB 00:00 2024-08-30T08:47:25.392 INFO:teuthology.orchestra.run.smithi167.stdout:(139/150): python3-rsa-4.9-2.el9.noarch.rpm 1.1 MB/s | 59 kB 00:00 2024-08-30T08:47:25.417 INFO:teuthology.orchestra.run.smithi167.stdout:(140/150): python3-tempora-5.0.0-2.el9.noarch.r 718 kB/s | 36 kB 00:00 2024-08-30T08:47:25.442 INFO:teuthology.orchestra.run.smithi167.stdout:(141/150): python3-routes-2.5.1-5.el9.noarch.rp 1.5 MB/s | 188 kB 00:00 2024-08-30T08:47:25.468 INFO:teuthology.orchestra.run.smithi167.stdout:(142/150): python3-typing-extensions-4.12.2-2.e 986 kB/s | 74 kB 00:00 2024-08-30T08:47:25.493 INFO:teuthology.orchestra.run.smithi167.stdout:(143/150): python3-websocket-client-1.2.3-2.el9 1.8 MB/s | 90 kB 00:00 2024-08-30T08:47:25.518 INFO:teuthology.orchestra.run.smithi167.stdout:(144/150): python3-zc-lockfile-2.0-10.el9.noarc 801 kB/s | 20 kB 00:00 2024-08-30T08:47:25.543 INFO:teuthology.orchestra.run.smithi167.stdout:(145/150): python3-webob-1.8.8-2.el9.noarch.rpm 1.8 MB/s | 230 kB 00:00 2024-08-30T08:47:25.569 INFO:teuthology.orchestra.run.smithi167.stdout:(146/150): re2-20211101-20.el9.x86_64.rpm 3.7 MB/s | 191 kB 00:00 2024-08-30T08:47:25.594 INFO:teuthology.orchestra.run.smithi167.stdout:(147/150): python3-werkzeug-2.0.3-3.el9.1.noarc 3.3 MB/s | 427 kB 00:00 2024-08-30T08:47:25.644 INFO:teuthology.orchestra.run.smithi167.stdout:(148/150): thrift-0.15.0-2.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-08-30T08:47:25.904 INFO:teuthology.orchestra.run.smithi167.stdout:(149/150): librbd1-19.3.0-4560.gd19e6118.el9.x8 10 MB/s | 3.1 MB 00:00 2024-08-30T08:47:25.973 INFO:teuthology.orchestra.run.smithi042.stdout:(73/150): protobuf-3.14.0-13.el9.x86_64.rpm 721 kB/s | 1.0 MB 00:01 2024-08-30T08:47:26.082 INFO:teuthology.orchestra.run.smithi042.stdout:(74/150): python3-colorama-0.4.6-3.el9.noarch.r 566 kB/s | 61 kB 00:00 2024-08-30T08:47:26.103 INFO:teuthology.orchestra.run.smithi167.stdout:(150/150): librados2-19.3.0-4560.gd19e6118.el9. 6.3 MB/s | 3.4 MB 00:00 2024-08-30T08:47:26.110 INFO:teuthology.orchestra.run.smithi167.stdout:-------------------------------------------------------------------------------- 2024-08-30T08:47:26.110 INFO:teuthology.orchestra.run.smithi167.stdout:Total 15 MB/s | 224 MB 00:15 2024-08-30T08:47:26.424 INFO:teuthology.orchestra.run.smithi042.stdout:(75/150): python3-devel-3.9.19-8.el9.x86_64.rpm 716 kB/s | 245 kB 00:00 2024-08-30T08:47:26.783 INFO:teuthology.orchestra.run.smithi042.stdout:(76/150): python3-jinja2-2.11.3-6.el9.noarch.rp 692 kB/s | 248 kB 00:00 2024-08-30T08:47:26.883 INFO:teuthology.orchestra.run.smithi042.stdout:(77/150): python3-jsonpatch-1.21-16.el9.noarch. 256 kB/s | 26 kB 00:00 2024-08-30T08:47:26.983 INFO:teuthology.orchestra.run.smithi042.stdout:(78/150): python3-jsonpointer-2.0-4.el9.noarch. 191 kB/s | 19 kB 00:00 2024-08-30T08:47:27.171 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction check 2024-08-30T08:47:27.251 INFO:teuthology.orchestra.run.smithi042.stdout:(79/150): python3-mako-1.1.4-6.el9.noarch.rpm 646 kB/s | 172 kB 00:00 2024-08-30T08:47:27.300 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction check succeeded. 2024-08-30T08:47:27.300 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction test 2024-08-30T08:47:27.359 INFO:teuthology.orchestra.run.smithi042.stdout:(80/150): python3-markupsafe-1.1.1-12.el9.x86_6 321 kB/s | 35 kB 00:00 2024-08-30T08:47:29.062 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction test succeeded. 2024-08-30T08:47:29.063 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction 2024-08-30T08:47:29.506 INFO:teuthology.orchestra.run.smithi190.stdout: Preparing : 1/1 2024-08-30T08:47:29.575 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : thrift-0.15.0-2.el9.x86_64 1/152 2024-08-30T08:47:29.676 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 2/152 2024-08-30T08:47:29.853 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 3/152 2024-08-30T08:47:30.193 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 4/152 2024-08-30T08:47:30.318 INFO:teuthology.orchestra.run.smithi190.stdout: Upgrading : librados2-2:19.3.0-4560.gd19e6118.el9.x86_64 5/152 2024-08-30T08:47:30.445 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: librados2-2:19.3.0-4560.gd19e6118.el9.x86_64 5/152 2024-08-30T08:47:30.469 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libcephfs2-2:19.3.0-4560.gd19e6118.el9.x86_64 6/152 2024-08-30T08:47:30.561 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: libcephfs2-2:19.3.0-4560.gd19e6118.el9.x86_64 6/152 2024-08-30T08:47:30.603 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-rados-2:19.3.0-4560.gd19e6118.el9.x86_64 7/152 2024-08-30T08:47:30.632 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 8/152 2024-08-30T08:47:30.658 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 9/152 2024-08-30T08:47:30.811 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 10/152 2024-08-30T08:47:30.863 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : postgresql-13.16-1.el9.x86_64 11/152 2024-08-30T08:47:30.891 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libcephsqlite-2:19.3.0-4560.gd19e6118.el9.x86_64 12/152 2024-08-30T08:47:30.980 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: libcephsqlite-2:19.3.0-4560.gd19e6118.el9.x86_64 12/152 2024-08-30T08:47:31.009 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libradosstriper1-2:19.3.0-4560.gd19e6118.el9.x86 13/152 2024-08-30T08:47:31.055 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: libradosstriper1-2:19.3.0-4560.gd19e6118.el9.x86 13/152 2024-08-30T08:47:31.084 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 14/152 2024-08-30T08:47:31.106 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 14/152 2024-08-30T08:47:31.395 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 15/152 2024-08-30T08:47:31.437 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 15/152 2024-08-30T08:47:31.552 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : re2-1:20211101-20.el9.x86_64 16/152 2024-08-30T08:47:31.585 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 17/152 2024-08-30T08:47:31.619 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : liboath-2.6.7-2.el9.x86_64 18/152 2024-08-30T08:47:31.693 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 19/152 2024-08-30T08:47:31.746 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 20/152 2024-08-30T08:47:31.786 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-packaging-20.9-5.el9.noarch 21/152 2024-08-30T08:47:31.830 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 22/152 2024-08-30T08:47:31.862 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 23/152 2024-08-30T08:47:31.891 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : lua-5.4.4-4.el9.x86_64 24/152 2024-08-30T08:47:31.900 INFO:teuthology.orchestra.run.smithi042.stdout:(81/150): postgresql-server-13.16-1.el9.x86_64. 761 kB/s | 5.8 MB 00:07 2024-08-30T08:47:31.970 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 25/152 2024-08-30T08:47:32.022 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 26/152 2024-08-30T08:47:32.052 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 27/152 2024-08-30T08:47:32.096 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 28/152 2024-08-30T08:47:32.183 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 29/152 2024-08-30T08:47:32.224 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-ceph-common-2:19.3.0-4560.gd19e6118.el9. 30/152 2024-08-30T08:47:32.242 INFO:teuthology.orchestra.run.smithi042.stdout:(82/150): python3-numpy-f2py-1.20.1-5.el9.x86_6 734 kB/s | 250 kB 00:00 2024-08-30T08:47:32.267 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-ceph-argparse-2:19.3.0-4560.gd19e6118.el 31/152 2024-08-30T08:47:32.296 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-cephfs-2:19.3.0-4560.gd19e6118.el9.x86_6 32/152 2024-08-30T08:47:32.368 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 33/152 2024-08-30T08:47:32.410 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : luarocks-3.9.2-5.el9.noarch 34/152 2024-08-30T08:47:32.486 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 35/152 2024-08-30T08:47:32.584 INFO:teuthology.orchestra.run.smithi042.stdout:(83/150): python3-oauthlib-3.1.1-5.el9.noarch.r 650 kB/s | 222 kB 00:00 2024-08-30T08:47:32.619 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 36/152 2024-08-30T08:47:32.679 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 37/152 2024-08-30T08:47:32.776 INFO:teuthology.orchestra.run.smithi042.stdout:(84/150): python3-packaging-20.9-5.el9.noarch.r 403 kB/s | 77 kB 00:00 2024-08-30T08:47:32.860 INFO:teuthology.orchestra.run.smithi042.stdout:(85/150): python3-babel-2.9.1-2.el9.noarch.rpm 768 kB/s | 6.0 MB 00:07 2024-08-30T08:47:32.885 INFO:teuthology.orchestra.run.smithi042.stdout:(86/150): python3-prettytable-0.7.2-27.el9.noar 389 kB/s | 42 kB 00:00 2024-08-30T08:47:33.160 INFO:teuthology.orchestra.run.smithi042.stdout:(87/150): python3-pyasn1-0.4.8-6.el9.noarch.rpm 578 kB/s | 159 kB 00:00 2024-08-30T08:47:33.231 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-rsa-4.9-2.el9.noarch 38/152 2024-08-30T08:47:33.269 INFO:teuthology.orchestra.run.smithi042.stdout:(88/150): python3-pyasn1-modules-0.4.8-6.el9.no 2.5 MB/s | 279 kB 00:00 2024-08-30T08:47:33.302 INFO:teuthology.orchestra.run.smithi042.stdout:(89/150): python3-protobuf-3.14.0-13.el9.noarch 609 kB/s | 269 kB 00:00 2024-08-30T08:47:33.350 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 39/152 2024-08-30T08:47:33.369 INFO:teuthology.orchestra.run.smithi042.stdout:(90/150): python3-pytz-2021.1-5.el9.noarch.rpm 508 kB/s | 51 kB 00:00 2024-08-30T08:47:33.379 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 40/152 2024-08-30T08:47:33.403 INFO:teuthology.orchestra.run.smithi042.stdout:(91/150): python3-requests-oauthlib-1.3.0-12.el 535 kB/s | 54 kB 00:00 2024-08-30T08:47:33.430 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 41/152 2024-08-30T08:47:33.468 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : librados-devel-2:19.3.0-4560.gd19e6118.el9.x86_6 42/152 2024-08-30T08:47:33.501 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 43/152 2024-08-30T08:47:33.512 INFO:teuthology.orchestra.run.smithi042.stdout:(92/150): python3-toml-0.10.2-6.el9.noarch.rpm 385 kB/s | 42 kB 00:00 2024-08-30T08:47:33.594 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 44/152 2024-08-30T08:47:33.629 INFO:teuthology.orchestra.run.smithi042.stdout:(93/150): python3-wcwidth-0.2.5-8.el9.noarch.rp 376 kB/s | 44 kB 00:00 2024-08-30T08:47:33.658 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 45/152 2024-08-30T08:47:33.731 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 46/152 2024-08-30T08:47:33.762 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 47/152 2024-08-30T08:47:33.798 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 48/152 2024-08-30T08:47:33.845 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 49/152 2024-08-30T08:47:33.889 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 50/152 2024-08-30T08:47:33.932 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 51/152 2024-08-30T08:47:33.974 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 52/152 2024-08-30T08:47:34.013 INFO:teuthology.orchestra.run.smithi042.stdout:(94/150): qatlib-24.02.0-1.el9.x86_64.rpm 576 kB/s | 221 kB 00:00 2024-08-30T08:47:34.112 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 53/152 2024-08-30T08:47:34.130 INFO:teuthology.orchestra.run.smithi042.stdout:(95/150): qatlib-service-24.02.0-1.el9.x86_64.r 307 kB/s | 36 kB 00:00 2024-08-30T08:47:34.151 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 54/152 2024-08-30T08:47:34.193 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 55/152 2024-08-30T08:47:34.230 INFO:teuthology.orchestra.run.smithi042.stdout:(96/150): qatzip-libs-1.2.0-1.el9.x86_64.rpm 474 kB/s | 47 kB 00:00 2024-08-30T08:47:34.295 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 56/152 2024-08-30T08:47:34.326 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 57/152 2024-08-30T08:47:34.374 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 58/152 2024-08-30T08:47:34.381 INFO:teuthology.orchestra.run.smithi042.stdout:(97/150): python3-numpy-1.20.1-5.el9.x86_64.rpm 743 kB/s | 5.1 MB 00:07 2024-08-30T08:47:34.408 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 59/152 2024-08-30T08:47:34.433 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 60/152 2024-08-30T08:47:34.506 INFO:teuthology.orchestra.run.smithi042.stdout:(98/150): uuid-1.6.2-55.el9.x86_64.rpm 462 kB/s | 58 kB 00:00 2024-08-30T08:47:34.509 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 61/152 2024-08-30T08:47:34.644 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 62/152 2024-08-30T08:47:34.665 INFO:teuthology.orchestra.run.smithi042.stdout:(99/150): socat-1.7.4.1-6.el9.x86_64.rpm 701 kB/s | 304 kB 00:00 2024-08-30T08:47:34.682 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 63/152 2024-08-30T08:47:34.708 INFO:teuthology.orchestra.run.smithi042.stdout:(100/150): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 474 kB/s | 96 kB 00:00 2024-08-30T08:47:34.710 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 64/152 2024-08-30T08:47:34.745 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 65/152 2024-08-30T08:47:34.789 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : uuid-1.6.2-55.el9.x86_64 66/152 2024-08-30T08:47:34.817 INFO:teuthology.orchestra.run.smithi042.stdout:(101/150): zlib-devel-1.2.11-41.el9.x86_64.rpm 413 kB/s | 45 kB 00:00 2024-08-30T08:47:34.826 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 67/152 2024-08-30T08:47:34.851 INFO:teuthology.orchestra.run.smithi042.stdout:(102/150): xmlstarlet-1.6.1-20.el9.x86_64.rpm 343 kB/s | 64 kB 00:00 2024-08-30T08:47:34.866 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 68/152 2024-08-30T08:47:34.905 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 69/152 2024-08-30T08:47:34.926 INFO:teuthology.orchestra.run.smithi042.stdout:(103/150): lua-devel-5.4.4-4.el9.x86_64.rpm 205 kB/s | 22 kB 00:00 2024-08-30T08:47:34.939 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 70/152 2024-08-30T08:47:34.972 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 71/152 2024-08-30T08:47:34.976 INFO:teuthology.orchestra.run.smithi042.stdout:(104/150): abseil-cpp-20211102.0-3.el9.x86_64.r 11 MB/s | 548 kB 00:00 2024-08-30T08:47:35.010 INFO:teuthology.orchestra.run.smithi042.stdout:(105/150): gperftools-libs-2.9.1-2.el9.x86_64.r 9.0 MB/s | 309 kB 00:00 2024-08-30T08:47:35.014 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 72/152 2024-08-30T08:47:35.035 INFO:teuthology.orchestra.run.smithi042.stdout:(106/150): grpc-data-1.46.7-10.el9.noarch.rpm 781 kB/s | 19 kB 00:00 2024-08-30T08:47:35.048 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 73/152 2024-08-30T08:47:35.136 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 74/152 2024-08-30T08:47:35.156 INFO:teuthology.orchestra.run.smithi167.stdout: Preparing : 1/1 2024-08-30T08:47:35.168 INFO:teuthology.orchestra.run.smithi042.stdout:(107/150): libarrow-9.0.0-11.el9.x86_64.rpm 33 MB/s | 4.4 MB 00:00 2024-08-30T08:47:35.179 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 75/152 2024-08-30T08:47:35.194 INFO:teuthology.orchestra.run.smithi042.stdout:(108/150): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.0 MB/s | 26 kB 00:00 2024-08-30T08:47:35.210 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 76/152 2024-08-30T08:47:35.219 INFO:teuthology.orchestra.run.smithi042.stdout:(109/150): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-08-30T08:47:35.231 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : thrift-0.15.0-2.el9.x86_64 1/152 2024-08-30T08:47:35.244 INFO:teuthology.orchestra.run.smithi042.stdout:(110/150): libunwind-1.6.2-1.el9.x86_64.rpm 2.6 MB/s | 67 kB 00:00 2024-08-30T08:47:35.264 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 2/152 2024-08-30T08:47:35.270 INFO:teuthology.orchestra.run.smithi042.stdout:(111/150): luarocks-3.9.2-5.el9.noarch.rpm 5.9 MB/s | 151 kB 00:00 2024-08-30T08:47:35.312 INFO:teuthology.orchestra.run.smithi042.stdout:(112/150): parquet-libs-9.0.0-11.el9.x86_64.rpm 20 MB/s | 839 kB 00:00 2024-08-30T08:47:35.316 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 3/152 2024-08-30T08:47:35.354 INFO:teuthology.orchestra.run.smithi042.stdout:(113/150): python3-asyncssh-2.13.2-5.el9.noarch 13 MB/s | 548 kB 00:00 2024-08-30T08:47:35.379 INFO:teuthology.orchestra.run.smithi042.stdout:(114/150): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-08-30T08:47:35.381 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 77/152 2024-08-30T08:47:35.404 INFO:teuthology.orchestra.run.smithi042.stdout:(115/150): python3-cachetools-4.2.4-1.el9.noarc 1.3 MB/s | 32 kB 00:00 2024-08-30T08:47:35.429 INFO:teuthology.orchestra.run.smithi042.stdout:(116/150): python3-certifi-2023.05.07-4.el9.noa 562 kB/s | 14 kB 00:00 2024-08-30T08:47:35.455 INFO:teuthology.orchestra.run.smithi042.stdout:(117/150): python3-cheroot-8.6.0-4.el9.noarch.r 6.7 MB/s | 172 kB 00:00 2024-08-30T08:47:35.489 INFO:teuthology.orchestra.run.smithi042.stdout:(118/150): python3-cherrypy-18.6.1-2.el9.noarch 11 MB/s | 358 kB 00:00 2024-08-30T08:47:35.522 INFO:teuthology.orchestra.run.smithi042.stdout:(119/150): python3-cmd2-2.4.2-2.el9.noarch.rpm 8.9 MB/s | 303 kB 00:00 2024-08-30T08:47:35.546 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 78/152 2024-08-30T08:47:35.547 INFO:teuthology.orchestra.run.smithi042.stdout:(120/150): python3-google-auth-2.34.0-1.el9.noa 8.6 MB/s | 220 kB 00:00 2024-08-30T08:47:35.631 INFO:teuthology.orchestra.run.smithi042.stdout:(121/150): python3-grpcio-1.46.7-10.el9.x86_64. 25 MB/s | 2.0 MB 00:00 2024-08-30T08:47:35.656 INFO:teuthology.orchestra.run.smithi042.stdout:(122/150): python3-grpcio-tools-1.46.7-10.el9.x 5.7 MB/s | 144 kB 00:00 2024-08-30T08:47:35.682 INFO:teuthology.orchestra.run.smithi042.stdout:(123/150): python3-jaraco-8.2.1-3.el9.noarch.rp 426 kB/s | 11 kB 00:00 2024-08-30T08:47:35.683 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 4/152 2024-08-30T08:47:35.707 INFO:teuthology.orchestra.run.smithi042.stdout:(124/150): python3-jaraco-classes-3.2.1-5.el9.n 710 kB/s | 18 kB 00:00 2024-08-30T08:47:35.732 INFO:teuthology.orchestra.run.smithi042.stdout:(125/150): python3-jaraco-collections-3.0.0-8.e 921 kB/s | 23 kB 00:00 2024-08-30T08:47:35.757 INFO:teuthology.orchestra.run.smithi042.stdout:(126/150): python3-jaraco-functools-3.5.0-2.el9 783 kB/s | 19 kB 00:00 2024-08-30T08:47:35.783 INFO:teuthology.orchestra.run.smithi042.stdout:(127/150): python3-jaraco-text-3.2.0-6.el9.noar 788 kB/s | 20 kB 00:00 2024-08-30T08:47:35.791 INFO:teuthology.orchestra.run.smithi167.stdout: Upgrading : librados2-2:19.3.0-4560.gd19e6118.el9.x86_64 5/152 2024-08-30T08:47:35.833 INFO:teuthology.orchestra.run.smithi042.stdout:(128/150): python3-kubernetes-26.1.0-2.el9.noar 20 MB/s | 1.0 MB 00:00 2024-08-30T08:47:35.858 INFO:teuthology.orchestra.run.smithi042.stdout:(129/150): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-08-30T08:47:35.884 INFO:teuthology.orchestra.run.smithi042.stdout:(130/150): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-08-30T08:47:35.909 INFO:teuthology.orchestra.run.smithi042.stdout:(131/150): python3-natsort-7.1.1-5.el9.noarch.r 2.2 MB/s | 58 kB 00:00 2024-08-30T08:47:35.916 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: librados2-2:19.3.0-4560.gd19e6118.el9.x86_64 5/152 2024-08-30T08:47:35.942 INFO:teuthology.orchestra.run.smithi042.stdout:(132/150): python3-pecan-1.4.2-3.el9.noarch.rpm 8.0 MB/s | 272 kB 00:00 2024-08-30T08:47:35.950 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libcephfs2-2:19.3.0-4560.gd19e6118.el9.x86_64 6/152 2024-08-30T08:47:35.968 INFO:teuthology.orchestra.run.smithi042.stdout:(133/150): python3-portend-3.1.0-2.el9.noarch.r 660 kB/s | 16 kB 00:00 2024-08-30T08:47:35.993 INFO:teuthology.orchestra.run.smithi042.stdout:(134/150): python3-pyOpenSSL-21.0.0-1.el9.noarc 3.5 MB/s | 90 kB 00:00 2024-08-30T08:47:36.018 INFO:teuthology.orchestra.run.smithi042.stdout:(135/150): python3-pyperclip-1.8.2-2.el9.noarch 1.1 MB/s | 28 kB 00:00 2024-08-30T08:47:36.034 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: libcephfs2-2:19.3.0-4560.gd19e6118.el9.x86_64 6/152 2024-08-30T08:47:36.044 INFO:teuthology.orchestra.run.smithi042.stdout:(136/150): python3-repoze-lru-0.7-16.el9.noarch 1.2 MB/s | 31 kB 00:00 2024-08-30T08:47:36.069 INFO:teuthology.orchestra.run.smithi042.stdout:(137/150): python3-routes-2.5.1-5.el9.noarch.rp 7.4 MB/s | 188 kB 00:00 2024-08-30T08:47:36.076 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-rados-2:19.3.0-4560.gd19e6118.el9.x86_64 7/152 2024-08-30T08:47:36.094 INFO:teuthology.orchestra.run.smithi042.stdout:(138/150): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-08-30T08:47:36.105 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 8/152 2024-08-30T08:47:36.122 INFO:teuthology.orchestra.run.smithi042.stdout:(139/150): python3-tempora-5.0.0-2.el9.noarch.r 1.3 MB/s | 36 kB 00:00 2024-08-30T08:47:36.132 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 9/152 2024-08-30T08:47:36.138 INFO:teuthology.orchestra.run.smithi042.stdout:(140/150): python3-typing-extensions-4.12.2-2.e 4.4 MB/s | 74 kB 00:00 2024-08-30T08:47:36.164 INFO:teuthology.orchestra.run.smithi042.stdout:(141/150): python3-webob-1.8.8-2.el9.noarch.rpm 9.0 MB/s | 230 kB 00:00 2024-08-30T08:47:36.189 INFO:teuthology.orchestra.run.smithi042.stdout:(142/150): python3-websocket-client-1.2.3-2.el9 3.5 MB/s | 90 kB 00:00 2024-08-30T08:47:36.214 INFO:teuthology.orchestra.run.smithi042.stdout:(143/150): protobuf-compiler-3.14.0-13.el9.x86_ 633 kB/s | 863 kB 00:01 2024-08-30T08:47:36.256 INFO:teuthology.orchestra.run.smithi042.stdout:(144/150): python3-werkzeug-2.0.3-3.el9.1.noarc 6.3 MB/s | 427 kB 00:00 2024-08-30T08:47:36.277 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 79/152 2024-08-30T08:47:36.282 INFO:teuthology.orchestra.run.smithi042.stdout:(145/150): python3-zc-lockfile-2.0-10.el9.noarc 299 kB/s | 20 kB 00:00 2024-08-30T08:47:36.301 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 10/152 2024-08-30T08:47:36.307 INFO:teuthology.orchestra.run.smithi042.stdout:(146/150): re2-20211101-20.el9.x86_64.rpm 3.9 MB/s | 191 kB 00:00 2024-08-30T08:47:36.317 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 80/152 2024-08-30T08:47:36.345 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : postgresql-13.16-1.el9.x86_64 11/152 2024-08-30T08:47:36.350 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 81/152 2024-08-30T08:47:36.373 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libcephsqlite-2:19.3.0-4560.gd19e6118.el9.x86_64 12/152 2024-08-30T08:47:36.374 INFO:teuthology.orchestra.run.smithi042.stdout:(147/150): thrift-0.15.0-2.el9.x86_64.rpm 17 MB/s | 1.6 MB 00:00 2024-08-30T08:47:36.452 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 82/152 2024-08-30T08:47:36.479 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: libcephsqlite-2:19.3.0-4560.gd19e6118.el9.x86_64 12/152 2024-08-30T08:47:36.506 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 83/152 2024-08-30T08:47:36.507 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libradosstriper1-2:19.3.0-4560.gd19e6118.el9.x86 13/152 2024-08-30T08:47:36.551 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 84/152 2024-08-30T08:47:36.570 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: libradosstriper1-2:19.3.0-4560.gd19e6118.el9.x86 13/152 2024-08-30T08:47:36.591 INFO:teuthology.orchestra.run.smithi042.stdout:(148/150): librados2-19.3.0-4560.gd19e6118.el9. 12 MB/s | 3.4 MB 00:00 2024-08-30T08:47:36.592 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 14/152 2024-08-30T08:47:36.617 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 14/152 2024-08-30T08:47:36.639 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 85/152 2024-08-30T08:47:36.674 INFO:teuthology.orchestra.run.smithi042.stdout:(149/150): librbd1-19.3.0-4560.gd19e6118.el9.x8 10 MB/s | 3.1 MB 00:00 2024-08-30T08:47:36.681 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 86/152 2024-08-30T08:47:36.865 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : openblas-0.3.26-2.el9.x86_64 87/152 2024-08-30T08:47:36.904 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 88/152 2024-08-30T08:47:36.922 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 15/152 2024-08-30T08:47:36.958 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 15/152 2024-08-30T08:47:36.981 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 89/152 2024-08-30T08:47:37.067 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : re2-1:20211101-20.el9.x86_64 16/152 2024-08-30T08:47:37.099 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 17/152 2024-08-30T08:47:37.134 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : liboath-2.6.7-2.el9.x86_64 18/152 2024-08-30T08:47:37.218 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 19/152 2024-08-30T08:47:37.278 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 20/152 2024-08-30T08:47:37.318 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-packaging-20.9-5.el9.noarch 21/152 2024-08-30T08:47:37.362 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 22/152 2024-08-30T08:47:37.402 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 23/152 2024-08-30T08:47:37.438 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : lua-5.4.4-4.el9.x86_64 24/152 2024-08-30T08:47:37.511 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 25/152 2024-08-30T08:47:37.570 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 26/152 2024-08-30T08:47:37.600 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 27/152 2024-08-30T08:47:37.638 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 90/152 2024-08-30T08:47:37.653 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 28/152 2024-08-30T08:47:37.740 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 29/152 2024-08-30T08:47:37.753 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 91/152 2024-08-30T08:47:37.771 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-ceph-common-2:19.3.0-4560.gd19e6118.el9. 30/152 2024-08-30T08:47:37.814 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-ceph-argparse-2:19.3.0-4560.gd19e6118.el 31/152 2024-08-30T08:47:37.843 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-cephfs-2:19.3.0-4560.gd19e6118.el9.x86_6 32/152 2024-08-30T08:47:37.914 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 33/152 2024-08-30T08:47:37.958 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : luarocks-3.9.2-5.el9.noarch 34/152 2024-08-30T08:47:38.034 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 35/152 2024-08-30T08:47:38.169 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 36/152 2024-08-30T08:47:38.237 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 37/152 2024-08-30T08:47:38.787 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-rsa-4.9-2.el9.noarch 38/152 2024-08-30T08:47:38.866 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 39/152 2024-08-30T08:47:38.894 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 40/152 2024-08-30T08:47:38.945 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 41/152 2024-08-30T08:47:38.984 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : librados-devel-2:19.3.0-4560.gd19e6118.el9.x86_6 42/152 2024-08-30T08:47:39.001 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 92/152 2024-08-30T08:47:39.016 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 43/152 2024-08-30T08:47:39.083 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 93/152 2024-08-30T08:47:39.110 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 44/152 2024-08-30T08:47:39.111 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 94/152 2024-08-30T08:47:39.153 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 95/152 2024-08-30T08:47:39.173 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 45/152 2024-08-30T08:47:39.246 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 46/152 2024-08-30T08:47:39.286 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 47/152 2024-08-30T08:47:39.322 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 48/152 2024-08-30T08:47:39.369 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 49/152 2024-08-30T08:47:39.405 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 50/152 2024-08-30T08:47:39.457 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 51/152 2024-08-30T08:47:39.493 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 96/152 2024-08-30T08:47:39.507 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 52/152 2024-08-30T08:47:39.541 INFO:teuthology.orchestra.run.smithi190.stdout: Upgrading : librbd1-2:19.3.0-4560.gd19e6118.el9.x86_64 97/152 2024-08-30T08:47:39.631 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: librbd1-2:19.3.0-4560.gd19e6118.el9.x86_64 97/152 2024-08-30T08:47:39.646 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 53/152 2024-08-30T08:47:39.662 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-rbd-2:19.3.0-4560.gd19e6118.el9.x86_64 98/152 2024-08-30T08:47:39.692 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 54/152 2024-08-30T08:47:39.698 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 99/152 2024-08-30T08:47:39.755 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 55/152 2024-08-30T08:47:39.863 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 56/152 2024-08-30T08:47:39.909 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 57/152 2024-08-30T08:47:39.957 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 58/152 2024-08-30T08:47:39.983 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 59/152 2024-08-30T08:47:40.008 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 60/152 2024-08-30T08:47:40.083 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 61/152 2024-08-30T08:47:40.194 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 100/152 2024-08-30T08:47:40.211 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 62/152 2024-08-30T08:47:40.227 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : librgw2-2:19.3.0-4560.gd19e6118.el9.x86_64 101/152 2024-08-30T08:47:40.249 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 63/152 2024-08-30T08:47:40.277 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 64/152 2024-08-30T08:47:40.298 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: librgw2-2:19.3.0-4560.gd19e6118.el9.x86_64 101/152 2024-08-30T08:47:40.311 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 65/152 2024-08-30T08:47:40.337 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-rgw-2:19.3.0-4560.gd19e6118.el9.x86_64 102/152 2024-08-30T08:47:40.355 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : uuid-1.6.2-55.el9.x86_64 66/152 2024-08-30T08:47:40.663 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 67/152 2024-08-30T08:47:40.710 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 68/152 2024-08-30T08:47:40.739 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 69/152 2024-08-30T08:47:40.774 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 70/152 2024-08-30T08:47:40.807 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 71/152 2024-08-30T08:47:40.858 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 72/152 2024-08-30T08:47:40.892 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 73/152 2024-08-30T08:47:40.986 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 74/152 2024-08-30T08:47:41.031 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 75/152 2024-08-30T08:47:41.070 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 76/152 2024-08-30T08:47:41.185 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 77/152 2024-08-30T08:47:41.245 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 78/152 2024-08-30T08:47:41.979 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 79/152 2024-08-30T08:47:42.034 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 80/152 2024-08-30T08:47:42.068 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 81/152 2024-08-30T08:47:42.169 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 82/152 2024-08-30T08:47:42.229 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 83/152 2024-08-30T08:47:42.274 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 84/152 2024-08-30T08:47:42.361 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 85/152 2024-08-30T08:47:42.402 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 86/152 2024-08-30T08:47:42.586 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : openblas-0.3.26-2.el9.x86_64 87/152 2024-08-30T08:47:42.616 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 88/152 2024-08-30T08:47:42.642 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-common-2:19.3.0-4560.gd19e6118.el9.x86_64 103/152 2024-08-30T08:47:42.683 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-common-2:19.3.0-4560.gd19e6118.el9.x86_64 103/152 2024-08-30T08:47:42.693 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 89/152 2024-08-30T08:47:43.260 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-common-2:19.3.0-4560.gd19e6118.el9.x86_64 103/152 2024-08-30T08:47:43.295 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-base-2:19.3.0-4560.gd19e6118.el9.x86_64 104/152 2024-08-30T08:47:43.349 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 90/152 2024-08-30T08:47:43.388 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-base-2:19.3.0-4560.gd19e6118.el9.x86_64 104/152 2024-08-30T08:47:43.388 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-08-30T08:47:43.388 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-08-30T08:47:43.388 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-08-30T08:47:43.421 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-selinux-2:19.3.0-4560.gd19e6118.el9.x86_64 105/152 2024-08-30T08:47:43.468 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 91/152 2024-08-30T08:47:44.743 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 92/152 2024-08-30T08:47:44.839 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 93/152 2024-08-30T08:47:44.866 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 94/152 2024-08-30T08:47:44.900 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 95/152 2024-08-30T08:47:45.227 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 96/152 2024-08-30T08:47:45.279 INFO:teuthology.orchestra.run.smithi167.stdout: Upgrading : librbd1-2:19.3.0-4560.gd19e6118.el9.x86_64 97/152 2024-08-30T08:47:45.371 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: librbd1-2:19.3.0-4560.gd19e6118.el9.x86_64 97/152 2024-08-30T08:47:45.401 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-rbd-2:19.3.0-4560.gd19e6118.el9.x86_64 98/152 2024-08-30T08:47:45.437 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 99/152 2024-08-30T08:47:45.934 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 100/152 2024-08-30T08:47:45.974 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : librgw2-2:19.3.0-4560.gd19e6118.el9.x86_64 101/152 2024-08-30T08:47:46.054 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: librgw2-2:19.3.0-4560.gd19e6118.el9.x86_64 101/152 2024-08-30T08:47:46.093 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-rgw-2:19.3.0-4560.gd19e6118.el9.x86_64 102/152 2024-08-30T08:47:48.333 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-common-2:19.3.0-4560.gd19e6118.el9.x86_64 103/152 2024-08-30T08:47:48.365 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-common-2:19.3.0-4560.gd19e6118.el9.x86_64 103/152 2024-08-30T08:47:48.936 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-common-2:19.3.0-4560.gd19e6118.el9.x86_64 103/152 2024-08-30T08:47:49.002 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-base-2:19.3.0-4560.gd19e6118.el9.x86_64 104/152 2024-08-30T08:47:49.094 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-base-2:19.3.0-4560.gd19e6118.el9.x86_64 104/152 2024-08-30T08:47:49.094 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-08-30T08:47:49.094 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-08-30T08:47:49.094 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-08-30T08:47:49.136 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-selinux-2:19.3.0-4560.gd19e6118.el9.x86_64 105/152 2024-08-30T08:47:50.733 INFO:teuthology.orchestra.run.smithi042.stdout:(150/150): python3-scipy-1.6.2-8.el9.x86_64.rpm 958 kB/s | 16 MB 00:17 2024-08-30T08:47:50.738 INFO:teuthology.orchestra.run.smithi042.stdout:-------------------------------------------------------------------------------- 2024-08-30T08:47:50.738 INFO:teuthology.orchestra.run.smithi042.stdout:Total 6.1 MB/s | 224 MB 00:36 2024-08-30T08:47:51.798 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction check 2024-08-30T08:47:51.936 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction check succeeded. 2024-08-30T08:47:51.936 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction test 2024-08-30T08:47:53.793 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction test succeeded. 2024-08-30T08:47:53.793 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction 2024-08-30T08:47:55.163 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-selinux-2:19.3.0-4560.gd19e6118.el9.x86_64 105/152 2024-08-30T08:47:55.163 INFO:teuthology.orchestra.run.smithi190.stdout:skipping the directory /sys 2024-08-30T08:47:55.163 INFO:teuthology.orchestra.run.smithi190.stdout:skipping the directory /proc 2024-08-30T08:47:55.163 INFO:teuthology.orchestra.run.smithi190.stdout:skipping the directory /mnt 2024-08-30T08:47:55.163 INFO:teuthology.orchestra.run.smithi190.stdout:skipping the directory /var/tmp 2024-08-30T08:47:55.163 INFO:teuthology.orchestra.run.smithi190.stdout:skipping the directory /home 2024-08-30T08:47:55.163 INFO:teuthology.orchestra.run.smithi190.stdout:skipping the directory /root 2024-08-30T08:47:55.163 INFO:teuthology.orchestra.run.smithi190.stdout:skipping the directory /tmp 2024-08-30T08:47:55.163 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-08-30T08:47:55.336 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-volume-2:19.3.0-4560.gd19e6118.el9.noarch 106/152 2024-08-30T08:47:55.352 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-volume-2:19.3.0-4560.gd19e6118.el9.noarch 106/152 2024-08-30T08:47:55.352 INFO:teuthology.orchestra.run.smithi190.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-30T08:47:55.352 INFO:teuthology.orchestra.run.smithi190.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-08-30T08:47:55.352 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-08-30T08:47:56.932 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-osd-2:19.3.0-4560.gd19e6118.el9.x86_64 107/152 2024-08-30T08:47:56.959 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-osd-2:19.3.0-4560.gd19e6118.el9.x86_64 107/152 2024-08-30T08:47:56.959 INFO:teuthology.orchestra.run.smithi190.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-30T08:47:56.959 INFO:teuthology.orchestra.run.smithi190.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-08-30T08:47:56.959 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-08-30T08:47:56.959 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-08-30T08:47:56.959 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-08-30T08:47:57.183 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-mds-2:19.3.0-4560.gd19e6118.el9.x86_64 108/152 2024-08-30T08:47:57.206 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-mds-2:19.3.0-4560.gd19e6118.el9.x86_64 108/152 2024-08-30T08:47:57.206 INFO:teuthology.orchestra.run.smithi190.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-30T08:47:57.206 INFO:teuthology.orchestra.run.smithi190.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-08-30T08:47:57.206 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-08-30T08:47:57.206 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-08-30T08:47:57.207 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-08-30T08:47:57.667 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-mon-2:19.3.0-4560.gd19e6118.el9.x86_64 109/152 2024-08-30T08:47:57.690 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-mon-2:19.3.0-4560.gd19e6118.el9.x86_64 109/152 2024-08-30T08:47:57.690 INFO:teuthology.orchestra.run.smithi190.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-30T08:47:57.690 INFO:teuthology.orchestra.run.smithi190.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-08-30T08:47:57.690 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-08-30T08:47:57.690 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-08-30T08:47:57.690 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-08-30T08:47:57.797 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : automake-1.16.2-8.el9.noarch 110/152 2024-08-30T08:47:57.845 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-ply-3.11-14.el9.noarch 111/152 2024-08-30T08:47:57.903 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 112/152 2024-08-30T08:47:58.087 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 113/152 2024-08-30T08:47:58.136 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 114/152 2024-08-30T08:47:58.205 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 115/152 2024-08-30T08:47:58.293 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 116/152 2024-08-30T08:47:58.425 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 117/152 2024-08-30T08:47:58.470 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 118/152 2024-08-30T08:47:58.502 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 119/152 2024-08-30T08:47:58.530 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : fuse-2.9.9-16.el9.x86_64 120/152 2024-08-30T08:47:58.967 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: cephadm-2:19.3.0-4560.gd19e6118.el9.noarch 121/152 2024-08-30T08:47:58.986 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : cephadm-2:19.3.0-4560.gd19e6118.el9.noarch 121/152 2024-08-30T08:47:59.019 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-prometheus-alerts-2:19.3.0-4560.gd19e6118.e 122/152 2024-08-30T08:47:59.084 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-grafana-dashboards-2:19.3.0-4560.gd19e6118. 123/152 2024-08-30T08:47:59.110 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-mgr-cephadm-2:19.3.0-4560.gd19e6118.el9.noa 124/152 2024-08-30T08:47:59.868 INFO:teuthology.orchestra.run.smithi042.stdout: Preparing : 1/1 2024-08-30T08:47:59.927 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : thrift-0.15.0-2.el9.x86_64 1/152 2024-08-30T08:47:59.951 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 2/152 2024-08-30T08:47:59.994 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 3/152 2024-08-30T08:48:00.433 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 4/152 2024-08-30T08:48:00.477 INFO:teuthology.orchestra.run.smithi042.stdout: Upgrading : librados2-2:19.3.0-4560.gd19e6118.el9.x86_64 5/152 2024-08-30T08:48:00.606 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: librados2-2:19.3.0-4560.gd19e6118.el9.x86_64 5/152 2024-08-30T08:48:00.636 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : libcephfs2-2:19.3.0-4560.gd19e6118.el9.x86_64 6/152 2024-08-30T08:48:00.729 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: libcephfs2-2:19.3.0-4560.gd19e6118.el9.x86_64 6/152 2024-08-30T08:48:00.770 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-rados-2:19.3.0-4560.gd19e6118.el9.x86_64 7/152 2024-08-30T08:48:00.784 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-selinux-2:19.3.0-4560.gd19e6118.el9.x86_64 105/152 2024-08-30T08:48:00.784 INFO:teuthology.orchestra.run.smithi167.stdout:skipping the directory /sys 2024-08-30T08:48:00.784 INFO:teuthology.orchestra.run.smithi167.stdout:skipping the directory /proc 2024-08-30T08:48:00.785 INFO:teuthology.orchestra.run.smithi167.stdout:skipping the directory /mnt 2024-08-30T08:48:00.785 INFO:teuthology.orchestra.run.smithi167.stdout:skipping the directory /var/tmp 2024-08-30T08:48:00.785 INFO:teuthology.orchestra.run.smithi167.stdout:skipping the directory /home 2024-08-30T08:48:00.785 INFO:teuthology.orchestra.run.smithi167.stdout:skipping the directory /root 2024-08-30T08:48:00.785 INFO:teuthology.orchestra.run.smithi167.stdout:skipping the directory /tmp 2024-08-30T08:48:00.785 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-08-30T08:48:00.799 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 8/152 2024-08-30T08:48:00.825 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 9/152 2024-08-30T08:48:00.973 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-volume-2:19.3.0-4560.gd19e6118.el9.noarch 106/152 2024-08-30T08:48:00.986 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 10/152 2024-08-30T08:48:00.989 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-volume-2:19.3.0-4560.gd19e6118.el9.noarch 106/152 2024-08-30T08:48:00.990 INFO:teuthology.orchestra.run.smithi167.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-30T08:48:00.990 INFO:teuthology.orchestra.run.smithi167.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-08-30T08:48:00.990 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-08-30T08:48:01.047 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : postgresql-13.16-1.el9.x86_64 11/152 2024-08-30T08:48:01.075 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : libcephsqlite-2:19.3.0-4560.gd19e6118.el9.x86_64 12/152 2024-08-30T08:48:01.182 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: libcephsqlite-2:19.3.0-4560.gd19e6118.el9.x86_64 12/152 2024-08-30T08:48:01.209 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : libradosstriper1-2:19.3.0-4560.gd19e6118.el9.x86 13/152 2024-08-30T08:48:01.271 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: libradosstriper1-2:19.3.0-4560.gd19e6118.el9.x86 13/152 2024-08-30T08:48:01.293 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 14/152 2024-08-30T08:48:01.314 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 14/152 2024-08-30T08:48:01.608 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 15/152 2024-08-30T08:48:01.643 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 15/152 2024-08-30T08:48:01.754 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : re2-1:20211101-20.el9.x86_64 16/152 2024-08-30T08:48:01.783 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 17/152 2024-08-30T08:48:01.818 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : liboath-2.6.7-2.el9.x86_64 18/152 2024-08-30T08:48:01.894 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 19/152 2024-08-30T08:48:01.944 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 20/152 2024-08-30T08:48:01.984 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-packaging-20.9-5.el9.noarch 21/152 2024-08-30T08:48:02.028 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 22/152 2024-08-30T08:48:02.060 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 23/152 2024-08-30T08:48:02.088 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : lua-5.4.4-4.el9.x86_64 24/152 2024-08-30T08:48:02.159 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 25/152 2024-08-30T08:48:02.212 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 26/152 2024-08-30T08:48:02.241 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 27/152 2024-08-30T08:48:02.247 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.3.0-4560.gd19e6118.el9.noa 124/152 2024-08-30T08:48:02.276 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 28/152 2024-08-30T08:48:02.299 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-mgr-dashboard-2:19.3.0-4560.gd19e6118.el9.n 125/152 2024-08-30T08:48:02.356 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 29/152 2024-08-30T08:48:02.395 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-ceph-common-2:19.3.0-4560.gd19e6118.el9. 30/152 2024-08-30T08:48:02.464 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-ceph-argparse-2:19.3.0-4560.gd19e6118.el 31/152 2024-08-30T08:48:02.492 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-cephfs-2:19.3.0-4560.gd19e6118.el9.x86_6 32/152 2024-08-30T08:48:02.568 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 33/152 2024-08-30T08:48:02.577 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-osd-2:19.3.0-4560.gd19e6118.el9.x86_64 107/152 2024-08-30T08:48:02.603 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-osd-2:19.3.0-4560.gd19e6118.el9.x86_64 107/152 2024-08-30T08:48:02.603 INFO:teuthology.orchestra.run.smithi167.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-30T08:48:02.603 INFO:teuthology.orchestra.run.smithi167.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-08-30T08:48:02.603 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-08-30T08:48:02.604 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-08-30T08:48:02.604 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-08-30T08:48:02.616 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : luarocks-3.9.2-5.el9.noarch 34/152 2024-08-30T08:48:02.695 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 35/152 2024-08-30T08:48:02.836 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-mds-2:19.3.0-4560.gd19e6118.el9.x86_64 108/152 2024-08-30T08:48:02.841 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 36/152 2024-08-30T08:48:02.859 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-mds-2:19.3.0-4560.gd19e6118.el9.x86_64 108/152 2024-08-30T08:48:02.859 INFO:teuthology.orchestra.run.smithi167.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-30T08:48:02.859 INFO:teuthology.orchestra.run.smithi167.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-08-30T08:48:02.860 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-08-30T08:48:02.860 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-08-30T08:48:02.860 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-08-30T08:48:02.895 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 37/152 2024-08-30T08:48:03.263 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.3.0-4560.gd19e6118.el9.n 125/152 2024-08-30T08:48:03.291 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-mgr-diskprediction-local-2:19.3.0-4560.gd19 126/152 2024-08-30T08:48:03.328 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-mon-2:19.3.0-4560.gd19e6118.el9.x86_64 109/152 2024-08-30T08:48:03.351 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-mon-2:19.3.0-4560.gd19e6118.el9.x86_64 109/152 2024-08-30T08:48:03.351 INFO:teuthology.orchestra.run.smithi167.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-30T08:48:03.351 INFO:teuthology.orchestra.run.smithi167.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-08-30T08:48:03.351 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-08-30T08:48:03.352 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-08-30T08:48:03.352 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-08-30T08:48:03.405 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.3.0-4560.gd19 126/152 2024-08-30T08:48:03.455 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : automake-1.16.2-8.el9.noarch 110/152 2024-08-30T08:48:03.461 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-rsa-4.9-2.el9.noarch 38/152 2024-08-30T08:48:03.512 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-ply-3.11-14.el9.noarch 111/152 2024-08-30T08:48:03.567 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-mgr-modules-core-2:19.3.0-4560.gd19e6118.el 127/152 2024-08-30T08:48:03.567 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 39/152 2024-08-30T08:48:03.577 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 112/152 2024-08-30T08:48:03.593 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-mgr-2:19.3.0-4560.gd19e6118.el9.x86_64 128/152 2024-08-30T08:48:03.601 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 40/152 2024-08-30T08:48:03.616 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-mgr-2:19.3.0-4560.gd19e6118.el9.x86_64 128/152 2024-08-30T08:48:03.616 INFO:teuthology.orchestra.run.smithi190.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-30T08:48:03.616 INFO:teuthology.orchestra.run.smithi190.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-08-30T08:48:03.616 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-08-30T08:48:03.617 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-08-30T08:48:03.617 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-08-30T08:48:03.644 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 41/152 2024-08-30T08:48:03.660 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-mgr-rook-2:19.3.0-4560.gd19e6118.el9.noarch 129/152 2024-08-30T08:48:03.672 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-mgr-rook-2:19.3.0-4560.gd19e6118.el9.noarch 129/152 2024-08-30T08:48:03.682 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : librados-devel-2:19.3.0-4560.gd19e6118.el9.x86_6 42/152 2024-08-30T08:48:03.715 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 43/152 2024-08-30T08:48:03.761 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 113/152 2024-08-30T08:48:03.770 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-2:19.3.0-4560.gd19e6118.el9.x86_64 130/152 2024-08-30T08:48:03.810 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 114/152 2024-08-30T08:48:03.810 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 44/152 2024-08-30T08:48:03.816 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-fuse-2:19.3.0-4560.gd19e6118.el9.x86_64 131/152 2024-08-30T08:48:03.872 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 45/152 2024-08-30T08:48:03.886 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 115/152 2024-08-30T08:48:03.947 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 46/152 2024-08-30T08:48:03.984 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 47/152 2024-08-30T08:48:03.985 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 116/152 2024-08-30T08:48:04.020 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 48/152 2024-08-30T08:48:04.067 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 49/152 2024-08-30T08:48:04.103 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 50/152 2024-08-30T08:48:04.125 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 117/152 2024-08-30T08:48:04.146 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 51/152 2024-08-30T08:48:04.168 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 118/152 2024-08-30T08:48:04.188 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 52/152 2024-08-30T08:48:04.201 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 119/152 2024-08-30T08:48:04.228 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : fuse-2.9.9-16.el9.x86_64 120/152 2024-08-30T08:48:04.331 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 53/152 2024-08-30T08:48:04.381 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 54/152 2024-08-30T08:48:04.423 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 55/152 2024-08-30T08:48:04.531 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 56/152 2024-08-30T08:48:04.564 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 57/152 2024-08-30T08:48:04.604 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 58/152 2024-08-30T08:48:04.629 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 59/152 2024-08-30T08:48:04.642 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: cephadm-2:19.3.0-4560.gd19e6118.el9.noarch 121/152 2024-08-30T08:48:04.654 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 60/152 2024-08-30T08:48:04.668 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : cephadm-2:19.3.0-4560.gd19e6118.el9.noarch 121/152 2024-08-30T08:48:04.701 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-prometheus-alerts-2:19.3.0-4560.gd19e6118.e 122/152 2024-08-30T08:48:04.732 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 61/152 2024-08-30T08:48:04.767 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-grafana-dashboards-2:19.3.0-4560.gd19e6118. 123/152 2024-08-30T08:48:04.779 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libtool-2.4.6-46.el9.x86_64 132/152 2024-08-30T08:48:04.813 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-radosgw-2:19.3.0-4560.gd19e6118.el9.x86_64 133/152 2024-08-30T08:48:04.817 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-mgr-cephadm-2:19.3.0-4560.gd19e6118.el9.noa 124/152 2024-08-30T08:48:04.836 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-radosgw-2:19.3.0-4560.gd19e6118.el9.x86_64 133/152 2024-08-30T08:48:04.836 INFO:teuthology.orchestra.run.smithi190.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-30T08:48:04.836 INFO:teuthology.orchestra.run.smithi190.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-08-30T08:48:04.836 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-08-30T08:48:04.836 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-08-30T08:48:04.837 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-08-30T08:48:04.871 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 62/152 2024-08-30T08:48:04.880 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-immutable-object-cache-2:19.3.0-4560.gd19e6 134/152 2024-08-30T08:48:04.903 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.3.0-4560.gd19e6 134/152 2024-08-30T08:48:04.903 INFO:teuthology.orchestra.run.smithi190.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-30T08:48:04.903 INFO:teuthology.orchestra.run.smithi190.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-08-30T08:48:04.903 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-08-30T08:48:04.912 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 63/152 2024-08-30T08:48:04.940 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 64/152 2024-08-30T08:48:04.948 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : cephfs-mirror-2:19.3.0-4560.gd19e6118.el9.x86_64 135/152 2024-08-30T08:48:04.971 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: cephfs-mirror-2:19.3.0-4560.gd19e6118.el9.x86_64 135/152 2024-08-30T08:48:04.971 INFO:teuthology.orchestra.run.smithi190.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-30T08:48:04.971 INFO:teuthology.orchestra.run.smithi190.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-08-30T08:48:04.971 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-08-30T08:48:04.971 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-08-30T08:48:04.971 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-08-30T08:48:04.974 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 65/152 2024-08-30T08:48:05.019 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : uuid-1.6.2-55.el9.x86_64 66/152 2024-08-30T08:48:05.056 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 67/152 2024-08-30T08:48:05.095 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 68/152 2024-08-30T08:48:05.125 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 69/152 2024-08-30T08:48:05.160 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 70/152 2024-08-30T08:48:05.193 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 71/152 2024-08-30T08:48:05.235 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 72/152 2024-08-30T08:48:05.261 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 73/152 2024-08-30T08:48:05.292 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : rbd-mirror-2:19.3.0-4560.gd19e6118.el9.x86_64 136/152 2024-08-30T08:48:05.315 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: rbd-mirror-2:19.3.0-4560.gd19e6118.el9.x86_64 136/152 2024-08-30T08:48:05.316 INFO:teuthology.orchestra.run.smithi190.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-30T08:48:05.316 INFO:teuthology.orchestra.run.smithi190.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-08-30T08:48:05.316 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-08-30T08:48:05.316 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-08-30T08:48:05.316 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-08-30T08:48:05.352 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 74/152 2024-08-30T08:48:05.400 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 75/152 2024-08-30T08:48:05.431 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 76/152 2024-08-30T08:48:05.540 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 77/152 2024-08-30T08:48:05.600 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 78/152 2024-08-30T08:48:06.373 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 79/152 2024-08-30T08:48:06.421 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 80/152 2024-08-30T08:48:06.455 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 81/152 2024-08-30T08:48:06.560 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 82/152 2024-08-30T08:48:06.836 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 83/152 2024-08-30T08:48:06.890 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 84/152 2024-08-30T08:48:06.981 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 85/152 2024-08-30T08:48:07.024 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 86/152 2024-08-30T08:48:07.216 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : openblas-0.3.26-2.el9.x86_64 87/152 2024-08-30T08:48:07.246 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 88/152 2024-08-30T08:48:07.327 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 89/152 2024-08-30T08:48:07.981 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.3.0-4560.gd19e6118.el9.noa 124/152 2024-08-30T08:48:08.026 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 90/152 2024-08-30T08:48:08.049 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-mgr-dashboard-2:19.3.0-4560.gd19e6118.el9.n 125/152 2024-08-30T08:48:08.167 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 91/152 2024-08-30T08:48:09.008 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.3.0-4560.gd19e6118.el9.n 125/152 2024-08-30T08:48:09.032 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-mgr-diskprediction-local-2:19.3.0-4560.gd19 126/152 2024-08-30T08:48:09.147 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.3.0-4560.gd19 126/152 2024-08-30T08:48:09.317 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-mgr-modules-core-2:19.3.0-4560.gd19e6118.el 127/152 2024-08-30T08:48:09.351 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-mgr-2:19.3.0-4560.gd19e6118.el9.x86_64 128/152 2024-08-30T08:48:09.375 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-mgr-2:19.3.0-4560.gd19e6118.el9.x86_64 128/152 2024-08-30T08:48:09.375 INFO:teuthology.orchestra.run.smithi167.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-30T08:48:09.375 INFO:teuthology.orchestra.run.smithi167.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-08-30T08:48:09.376 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-08-30T08:48:09.376 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-08-30T08:48:09.376 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-08-30T08:48:09.427 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-mgr-rook-2:19.3.0-4560.gd19e6118.el9.noarch 129/152 2024-08-30T08:48:09.438 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-mgr-rook-2:19.3.0-4560.gd19e6118.el9.noarch 129/152 2024-08-30T08:48:09.491 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 92/152 2024-08-30T08:48:09.537 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-2:19.3.0-4560.gd19e6118.el9.x86_64 130/152 2024-08-30T08:48:09.576 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 93/152 2024-08-30T08:48:09.592 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-fuse-2:19.3.0-4560.gd19e6118.el9.x86_64 131/152 2024-08-30T08:48:09.605 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 94/152 2024-08-30T08:48:09.637 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 95/152 2024-08-30T08:48:09.637 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-test-2:19.3.0-4560.gd19e6118.el9.x86_64 137/152 2024-08-30T08:48:09.679 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : rbd-fuse-2:19.3.0-4560.gd19e6118.el9.x86_64 138/152 2024-08-30T08:48:09.719 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : rbd-nbd-2:19.3.0-4560.gd19e6118.el9.x86_64 139/152 2024-08-30T08:48:09.751 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 140/152 2024-08-30T08:48:09.779 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 141/152 2024-08-30T08:48:09.957 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : cephfs-shell-2:19.3.0-4560.gd19e6118.el9.x86_64 142/152 2024-08-30T08:48:09.981 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 96/152 2024-08-30T08:48:10.000 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 143/152 2024-08-30T08:48:10.030 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libcephfs-devel-2:19.3.0-4560.gd19e6118.el9.x86_ 144/152 2024-08-30T08:48:10.032 INFO:teuthology.orchestra.run.smithi042.stdout: Upgrading : librbd1-2:19.3.0-4560.gd19e6118.el9.x86_64 97/152 2024-08-30T08:48:10.125 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: librbd1-2:19.3.0-4560.gd19e6118.el9.x86_64 97/152 2024-08-30T08:48:10.154 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-rbd-2:19.3.0-4560.gd19e6118.el9.x86_64 98/152 2024-08-30T08:48:10.191 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 99/152 2024-08-30T08:48:10.548 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libtool-2.4.6-46.el9.x86_64 132/152 2024-08-30T08:48:10.580 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-radosgw-2:19.3.0-4560.gd19e6118.el9.x86_64 133/152 2024-08-30T08:48:10.603 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-radosgw-2:19.3.0-4560.gd19e6118.el9.x86_64 133/152 2024-08-30T08:48:10.603 INFO:teuthology.orchestra.run.smithi167.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-30T08:48:10.603 INFO:teuthology.orchestra.run.smithi167.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-08-30T08:48:10.603 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-08-30T08:48:10.603 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-08-30T08:48:10.604 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-08-30T08:48:10.623 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 145/152 2024-08-30T08:48:10.664 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-immutable-object-cache-2:19.3.0-4560.gd19e6 134/152 2024-08-30T08:48:10.669 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 145/152 2024-08-30T08:48:10.686 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.3.0-4560.gd19e6 134/152 2024-08-30T08:48:10.686 INFO:teuthology.orchestra.run.smithi167.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-30T08:48:10.686 INFO:teuthology.orchestra.run.smithi167.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-08-30T08:48:10.686 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-08-30T08:48:10.689 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 145/152 2024-08-30T08:48:10.715 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 100/152 2024-08-30T08:48:10.740 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : cephfs-mirror-2:19.3.0-4560.gd19e6118.el9.x86_64 135/152 2024-08-30T08:48:10.744 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : librgw2-2:19.3.0-4560.gd19e6118.el9.x86_64 101/152 2024-08-30T08:48:10.762 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: cephfs-mirror-2:19.3.0-4560.gd19e6118.el9.x86_64 135/152 2024-08-30T08:48:10.763 INFO:teuthology.orchestra.run.smithi167.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-30T08:48:10.763 INFO:teuthology.orchestra.run.smithi167.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-08-30T08:48:10.763 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-08-30T08:48:10.763 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-08-30T08:48:10.763 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-08-30T08:48:10.815 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: librgw2-2:19.3.0-4560.gd19e6118.el9.x86_64 101/152 2024-08-30T08:48:10.845 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-rgw-2:19.3.0-4560.gd19e6118.el9.x86_64 102/152 2024-08-30T08:48:11.082 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : rbd-mirror-2:19.3.0-4560.gd19e6118.el9.x86_64 136/152 2024-08-30T08:48:11.105 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: rbd-mirror-2:19.3.0-4560.gd19e6118.el9.x86_64 136/152 2024-08-30T08:48:11.106 INFO:teuthology.orchestra.run.smithi167.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-30T08:48:11.106 INFO:teuthology.orchestra.run.smithi167.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-08-30T08:48:11.106 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-08-30T08:48:11.106 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-08-30T08:48:11.106 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-08-30T08:48:11.516 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : cephfs-top-2:19.3.0-4560.gd19e6118.el9.noarch 146/152 2024-08-30T08:48:11.600 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : openssl-devel-1:3.2.2-4.el9.x86_64 147/152 2024-08-30T08:48:11.640 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 148/152 2024-08-30T08:48:11.720 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : flex-2.6.4-9.el9.x86_64 149/152 2024-08-30T08:48:11.754 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : bison-3.7.4-5.el9.x86_64 150/152 2024-08-30T08:48:11.754 INFO:teuthology.orchestra.run.smithi190.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 151/152 2024-08-30T08:48:11.821 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 151/152 2024-08-30T08:48:11.821 INFO:teuthology.orchestra.run.smithi190.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 152/152 2024-08-30T08:48:13.255 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-common-2:19.3.0-4560.gd19e6118.el9.x86_64 103/152 2024-08-30T08:48:13.292 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-common-2:19.3.0-4560.gd19e6118.el9.x86_64 103/152 2024-08-30T08:48:13.887 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-common-2:19.3.0-4560.gd19e6118.el9.x86_64 103/152 2024-08-30T08:48:13.928 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-base-2:19.3.0-4560.gd19e6118.el9.x86_64 104/152 2024-08-30T08:48:14.020 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-base-2:19.3.0-4560.gd19e6118.el9.x86_64 104/152 2024-08-30T08:48:14.020 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-08-30T08:48:14.020 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-08-30T08:48:14.020 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-08-30T08:48:14.054 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-selinux-2:19.3.0-4560.gd19e6118.el9.x86_64 105/152 2024-08-30T08:48:15.453 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-test-2:19.3.0-4560.gd19e6118.el9.x86_64 137/152 2024-08-30T08:48:15.494 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : rbd-fuse-2:19.3.0-4560.gd19e6118.el9.x86_64 138/152 2024-08-30T08:48:15.535 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : rbd-nbd-2:19.3.0-4560.gd19e6118.el9.x86_64 139/152 2024-08-30T08:48:15.567 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 140/152 2024-08-30T08:48:15.594 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 141/152 2024-08-30T08:48:15.777 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : cephfs-shell-2:19.3.0-4560.gd19e6118.el9.x86_64 142/152 2024-08-30T08:48:15.825 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 143/152 2024-08-30T08:48:15.854 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libcephfs-devel-2:19.3.0-4560.gd19e6118.el9.x86_ 144/152 2024-08-30T08:48:16.486 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 145/152 2024-08-30T08:48:16.528 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 145/152 2024-08-30T08:48:16.548 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 145/152 2024-08-30T08:48:17.401 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : cephfs-top-2:19.3.0-4560.gd19e6118.el9.noarch 146/152 2024-08-30T08:48:17.492 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : openssl-devel-1:3.2.2-4.el9.x86_64 147/152 2024-08-30T08:48:17.533 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 148/152 2024-08-30T08:48:17.614 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : flex-2.6.4-9.el9.x86_64 149/152 2024-08-30T08:48:17.655 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : bison-3.7.4-5.el9.x86_64 150/152 2024-08-30T08:48:17.655 INFO:teuthology.orchestra.run.smithi167.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 151/152 2024-08-30T08:48:17.747 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 151/152 2024-08-30T08:48:17.748 INFO:teuthology.orchestra.run.smithi167.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 152/152 2024-08-30T08:48:18.548 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 152/152 2024-08-30T08:48:18.548 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-2:19.3.0-4560.gd19e6118.el9.x86_64 1/152 2024-08-30T08:48:18.549 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-base-2:19.3.0-4560.gd19e6118.el9.x86_64 2/152 2024-08-30T08:48:18.549 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-common-2:19.3.0-4560.gd19e6118.el9.x86_64 3/152 2024-08-30T08:48:18.549 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-fuse-2:19.3.0-4560.gd19e6118.el9.x86_64 4/152 2024-08-30T08:48:18.549 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-immutable-object-cache-2:19.3.0-4560.gd19e6 5/152 2024-08-30T08:48:18.549 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-mds-2:19.3.0-4560.gd19e6118.el9.x86_64 6/152 2024-08-30T08:48:18.549 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-mgr-2:19.3.0-4560.gd19e6118.el9.x86_64 7/152 2024-08-30T08:48:18.549 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-mon-2:19.3.0-4560.gd19e6118.el9.x86_64 8/152 2024-08-30T08:48:18.549 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-osd-2:19.3.0-4560.gd19e6118.el9.x86_64 9/152 2024-08-30T08:48:18.549 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-radosgw-2:19.3.0-4560.gd19e6118.el9.x86_64 10/152 2024-08-30T08:48:18.549 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-selinux-2:19.3.0-4560.gd19e6118.el9.x86_64 11/152 2024-08-30T08:48:18.549 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-test-2:19.3.0-4560.gd19e6118.el9.x86_64 12/152 2024-08-30T08:48:18.549 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : cephfs-mirror-2:19.3.0-4560.gd19e6118.el9.x86_64 13/152 2024-08-30T08:48:18.550 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : cephfs-shell-2:19.3.0-4560.gd19e6118.el9.x86_64 14/152 2024-08-30T08:48:18.550 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libcephfs-devel-2:19.3.0-4560.gd19e6118.el9.x86_ 15/152 2024-08-30T08:48:18.550 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libcephfs2-2:19.3.0-4560.gd19e6118.el9.x86_64 16/152 2024-08-30T08:48:18.550 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libcephsqlite-2:19.3.0-4560.gd19e6118.el9.x86_64 17/152 2024-08-30T08:48:18.550 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : librados-devel-2:19.3.0-4560.gd19e6118.el9.x86_6 18/152 2024-08-30T08:48:18.550 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libradosstriper1-2:19.3.0-4560.gd19e6118.el9.x86 19/152 2024-08-30T08:48:18.550 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : librgw2-2:19.3.0-4560.gd19e6118.el9.x86_64 20/152 2024-08-30T08:48:18.550 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-ceph-argparse-2:19.3.0-4560.gd19e6118.el 21/152 2024-08-30T08:48:18.550 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-ceph-common-2:19.3.0-4560.gd19e6118.el9. 22/152 2024-08-30T08:48:18.550 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-cephfs-2:19.3.0-4560.gd19e6118.el9.x86_6 23/152 2024-08-30T08:48:18.550 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-rados-2:19.3.0-4560.gd19e6118.el9.x86_64 24/152 2024-08-30T08:48:18.550 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-rbd-2:19.3.0-4560.gd19e6118.el9.x86_64 25/152 2024-08-30T08:48:18.551 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-rgw-2:19.3.0-4560.gd19e6118.el9.x86_64 26/152 2024-08-30T08:48:18.551 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : rbd-fuse-2:19.3.0-4560.gd19e6118.el9.x86_64 27/152 2024-08-30T08:48:18.551 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : rbd-mirror-2:19.3.0-4560.gd19e6118.el9.x86_64 28/152 2024-08-30T08:48:18.551 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : rbd-nbd-2:19.3.0-4560.gd19e6118.el9.x86_64 29/152 2024-08-30T08:48:18.551 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-grafana-dashboards-2:19.3.0-4560.gd19e6118. 30/152 2024-08-30T08:48:18.561 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-mgr-cephadm-2:19.3.0-4560.gd19e6118.el9.noa 31/152 2024-08-30T08:48:18.561 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-mgr-dashboard-2:19.3.0-4560.gd19e6118.el9.n 32/152 2024-08-30T08:48:18.561 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.3.0-4560.gd19 33/152 2024-08-30T08:48:18.561 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-mgr-modules-core-2:19.3.0-4560.gd19e6118.el 34/152 2024-08-30T08:48:18.562 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-mgr-rook-2:19.3.0-4560.gd19e6118.el9.noarch 35/152 2024-08-30T08:48:18.562 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-prometheus-alerts-2:19.3.0-4560.gd19e6118.e 36/152 2024-08-30T08:48:18.562 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-volume-2:19.3.0-4560.gd19e6118.el9.noarch 37/152 2024-08-30T08:48:18.562 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : cephadm-2:19.3.0-4560.gd19e6118.el9.noarch 38/152 2024-08-30T08:48:18.562 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : cephfs-top-2:19.3.0-4560.gd19e6118.el9.noarch 39/152 2024-08-30T08:48:18.562 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/152 2024-08-30T08:48:18.562 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/152 2024-08-30T08:48:18.562 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/152 2024-08-30T08:48:18.562 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/152 2024-08-30T08:48:18.562 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/152 2024-08-30T08:48:18.562 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/152 2024-08-30T08:48:18.562 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/152 2024-08-30T08:48:18.562 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/152 2024-08-30T08:48:18.563 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/152 2024-08-30T08:48:18.563 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/152 2024-08-30T08:48:18.563 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/152 2024-08-30T08:48:18.563 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/152 2024-08-30T08:48:18.563 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/152 2024-08-30T08:48:18.563 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/152 2024-08-30T08:48:18.563 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/152 2024-08-30T08:48:18.563 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/152 2024-08-30T08:48:18.563 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/152 2024-08-30T08:48:18.563 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/152 2024-08-30T08:48:18.563 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/152 2024-08-30T08:48:18.563 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/152 2024-08-30T08:48:18.563 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/152 2024-08-30T08:48:18.564 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/152 2024-08-30T08:48:18.564 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/152 2024-08-30T08:48:18.564 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/152 2024-08-30T08:48:18.564 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/152 2024-08-30T08:48:18.564 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/152 2024-08-30T08:48:18.564 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/152 2024-08-30T08:48:18.564 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/152 2024-08-30T08:48:18.564 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : openssl-devel-1:3.2.2-4.el9.x86_64 68/152 2024-08-30T08:48:18.564 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/152 2024-08-30T08:48:18.565 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/152 2024-08-30T08:48:18.565 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/152 2024-08-30T08:48:18.565 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/152 2024-08-30T08:48:18.565 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/152 2024-08-30T08:48:18.565 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/152 2024-08-30T08:48:18.565 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/152 2024-08-30T08:48:18.565 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/152 2024-08-30T08:48:18.565 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/152 2024-08-30T08:48:18.565 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/152 2024-08-30T08:48:18.565 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/152 2024-08-30T08:48:18.565 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/152 2024-08-30T08:48:18.565 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/152 2024-08-30T08:48:18.566 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/152 2024-08-30T08:48:18.566 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/152 2024-08-30T08:48:18.566 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/152 2024-08-30T08:48:18.566 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/152 2024-08-30T08:48:18.566 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/152 2024-08-30T08:48:18.566 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/152 2024-08-30T08:48:18.566 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/152 2024-08-30T08:48:18.566 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/152 2024-08-30T08:48:18.566 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/152 2024-08-30T08:48:18.566 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/152 2024-08-30T08:48:18.566 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/152 2024-08-30T08:48:18.566 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/152 2024-08-30T08:48:18.567 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/152 2024-08-30T08:48:18.567 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/152 2024-08-30T08:48:18.567 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/152 2024-08-30T08:48:18.567 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/152 2024-08-30T08:48:18.567 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/152 2024-08-30T08:48:18.567 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/152 2024-08-30T08:48:18.567 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/152 2024-08-30T08:48:18.567 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/152 2024-08-30T08:48:18.567 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/152 2024-08-30T08:48:18.567 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/152 2024-08-30T08:48:18.567 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/152 2024-08-30T08:48:18.567 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/152 2024-08-30T08:48:18.567 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/152 2024-08-30T08:48:18.567 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/152 2024-08-30T08:48:18.568 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/152 2024-08-30T08:48:18.568 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/152 2024-08-30T08:48:18.568 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/152 2024-08-30T08:48:18.568 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/152 2024-08-30T08:48:18.568 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/152 2024-08-30T08:48:18.568 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/152 2024-08-30T08:48:18.568 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/152 2024-08-30T08:48:18.568 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/152 2024-08-30T08:48:18.568 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/152 2024-08-30T08:48:18.568 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/152 2024-08-30T08:48:18.568 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/152 2024-08-30T08:48:18.568 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/152 2024-08-30T08:48:18.568 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/152 2024-08-30T08:48:18.569 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/152 2024-08-30T08:48:18.569 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/152 2024-08-30T08:48:18.569 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/152 2024-08-30T08:48:18.569 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/152 2024-08-30T08:48:18.569 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/152 2024-08-30T08:48:18.569 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/152 2024-08-30T08:48:18.569 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/152 2024-08-30T08:48:18.569 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/152 2024-08-30T08:48:18.569 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/152 2024-08-30T08:48:18.569 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/152 2024-08-30T08:48:18.569 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/152 2024-08-30T08:48:18.569 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/152 2024-08-30T08:48:18.569 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/152 2024-08-30T08:48:18.570 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/152 2024-08-30T08:48:18.570 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/152 2024-08-30T08:48:18.570 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/152 2024-08-30T08:48:18.570 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/152 2024-08-30T08:48:18.570 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/152 2024-08-30T08:48:18.570 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/152 2024-08-30T08:48:18.570 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/152 2024-08-30T08:48:18.570 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/152 2024-08-30T08:48:18.570 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/152 2024-08-30T08:48:18.570 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/152 2024-08-30T08:48:18.570 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/152 2024-08-30T08:48:18.570 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/152 2024-08-30T08:48:18.571 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/152 2024-08-30T08:48:18.571 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/152 2024-08-30T08:48:18.571 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/152 2024-08-30T08:48:18.571 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : librados2-2:19.3.0-4560.gd19e6118.el9.x86_64 149/152 2024-08-30T08:48:18.571 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/152 2024-08-30T08:48:18.571 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : librbd1-2:19.3.0-4560.gd19e6118.el9.x86_64 151/152 2024-08-30T08:48:20.712 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/152 2024-08-30T08:48:20.712 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-08-30T08:48:20.712 INFO:teuthology.orchestra.run.smithi190.stdout:Upgraded: 2024-08-30T08:48:20.712 INFO:teuthology.orchestra.run.smithi190.stdout: librados2-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-30T08:48:20.713 INFO:teuthology.orchestra.run.smithi190.stdout: librbd1-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-30T08:48:20.713 INFO:teuthology.orchestra.run.smithi190.stdout:Installed: 2024-08-30T08:48:20.713 INFO:teuthology.orchestra.run.smithi190.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-08-30T08:48:20.713 INFO:teuthology.orchestra.run.smithi190.stdout: automake-1.16.2-8.el9.noarch 2024-08-30T08:48:20.713 INFO:teuthology.orchestra.run.smithi190.stdout: bison-3.7.4-5.el9.x86_64 2024-08-30T08:48:20.713 INFO:teuthology.orchestra.run.smithi190.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-08-30T08:48:20.713 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-30T08:48:20.713 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-base-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-30T08:48:20.713 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-common-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-30T08:48:20.713 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-fuse-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-30T08:48:20.713 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-grafana-dashboards-2:19.3.0-4560.gd19e6118.el9.noarch 2024-08-30T08:48:20.713 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-immutable-object-cache-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-30T08:48:20.713 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mds-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-30T08:48:20.714 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-30T08:48:20.714 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-cephadm-2:19.3.0-4560.gd19e6118.el9.noarch 2024-08-30T08:48:20.714 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-dashboard-2:19.3.0-4560.gd19e6118.el9.noarch 2024-08-30T08:48:20.714 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-diskprediction-local-2:19.3.0-4560.gd19e6118.el9.noarch 2024-08-30T08:48:20.714 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-modules-core-2:19.3.0-4560.gd19e6118.el9.noarch 2024-08-30T08:48:20.714 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-rook-2:19.3.0-4560.gd19e6118.el9.noarch 2024-08-30T08:48:20.714 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mon-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-30T08:48:20.714 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-osd-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-30T08:48:20.714 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-prometheus-alerts-2:19.3.0-4560.gd19e6118.el9.noarch 2024-08-30T08:48:20.714 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-radosgw-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-30T08:48:20.714 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-selinux-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-30T08:48:20.715 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-test-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-30T08:48:20.715 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-volume-2:19.3.0-4560.gd19e6118.el9.noarch 2024-08-30T08:48:20.715 INFO:teuthology.orchestra.run.smithi190.stdout: cephadm-2:19.3.0-4560.gd19e6118.el9.noarch 2024-08-30T08:48:20.715 INFO:teuthology.orchestra.run.smithi190.stdout: cephfs-mirror-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-30T08:48:20.715 INFO:teuthology.orchestra.run.smithi190.stdout: cephfs-shell-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-30T08:48:20.715 INFO:teuthology.orchestra.run.smithi190.stdout: cephfs-top-2:19.3.0-4560.gd19e6118.el9.noarch 2024-08-30T08:48:20.715 INFO:teuthology.orchestra.run.smithi190.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-08-30T08:48:20.715 INFO:teuthology.orchestra.run.smithi190.stdout: flex-2.6.4-9.el9.x86_64 2024-08-30T08:48:20.715 INFO:teuthology.orchestra.run.smithi190.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-08-30T08:48:20.715 INFO:teuthology.orchestra.run.smithi190.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-08-30T08:48:20.715 INFO:teuthology.orchestra.run.smithi190.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-08-30T08:48:20.715 INFO:teuthology.orchestra.run.smithi190.stdout: fuse-2.9.9-16.el9.x86_64 2024-08-30T08:48:20.715 INFO:teuthology.orchestra.run.smithi190.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-08-30T08:48:20.715 INFO:teuthology.orchestra.run.smithi190.stdout: grpc-data-1.46.7-10.el9.noarch 2024-08-30T08:48:20.715 INFO:teuthology.orchestra.run.smithi190.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-08-30T08:48:20.716 INFO:teuthology.orchestra.run.smithi190.stdout: libarrow-9.0.0-11.el9.x86_64 2024-08-30T08:48:20.716 INFO:teuthology.orchestra.run.smithi190.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-08-30T08:48:20.716 INFO:teuthology.orchestra.run.smithi190.stdout: libcephfs-devel-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-30T08:48:20.716 INFO:teuthology.orchestra.run.smithi190.stdout: libcephfs2-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-30T08:48:20.716 INFO:teuthology.orchestra.run.smithi190.stdout: libcephsqlite-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-30T08:48:20.716 INFO:teuthology.orchestra.run.smithi190.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-08-30T08:48:20.716 INFO:teuthology.orchestra.run.smithi190.stdout: liboath-2.6.7-2.el9.x86_64 2024-08-30T08:48:20.716 INFO:teuthology.orchestra.run.smithi190.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-08-30T08:48:20.716 INFO:teuthology.orchestra.run.smithi190.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-08-30T08:48:20.716 INFO:teuthology.orchestra.run.smithi190.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-08-30T08:48:20.716 INFO:teuthology.orchestra.run.smithi190.stdout: librados-devel-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-30T08:48:20.716 INFO:teuthology.orchestra.run.smithi190.stdout: libradosstriper1-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-30T08:48:20.716 INFO:teuthology.orchestra.run.smithi190.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-08-30T08:48:20.717 INFO:teuthology.orchestra.run.smithi190.stdout: librgw2-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-30T08:48:20.717 INFO:teuthology.orchestra.run.smithi190.stdout: libtool-2.4.6-46.el9.x86_64 2024-08-30T08:48:20.717 INFO:teuthology.orchestra.run.smithi190.stdout: libunwind-1.6.2-1.el9.x86_64 2024-08-30T08:48:20.717 INFO:teuthology.orchestra.run.smithi190.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-08-30T08:48:20.717 INFO:teuthology.orchestra.run.smithi190.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-08-30T08:48:20.717 INFO:teuthology.orchestra.run.smithi190.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-08-30T08:48:20.717 INFO:teuthology.orchestra.run.smithi190.stdout: lua-5.4.4-4.el9.x86_64 2024-08-30T08:48:20.717 INFO:teuthology.orchestra.run.smithi190.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-08-30T08:48:20.717 INFO:teuthology.orchestra.run.smithi190.stdout: luarocks-3.9.2-5.el9.noarch 2024-08-30T08:48:20.717 INFO:teuthology.orchestra.run.smithi190.stdout: openblas-0.3.26-2.el9.x86_64 2024-08-30T08:48:20.717 INFO:teuthology.orchestra.run.smithi190.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-08-30T08:48:20.717 INFO:teuthology.orchestra.run.smithi190.stdout: openssl-devel-1:3.2.2-4.el9.x86_64 2024-08-30T08:48:20.717 INFO:teuthology.orchestra.run.smithi190.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-08-30T08:48:20.718 INFO:teuthology.orchestra.run.smithi190.stdout: postgresql-13.16-1.el9.x86_64 2024-08-30T08:48:20.718 INFO:teuthology.orchestra.run.smithi190.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-08-30T08:48:20.718 INFO:teuthology.orchestra.run.smithi190.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-08-30T08:48:20.718 INFO:teuthology.orchestra.run.smithi190.stdout: postgresql-server-13.16-1.el9.x86_64 2024-08-30T08:48:20.718 INFO:teuthology.orchestra.run.smithi190.stdout: protobuf-3.14.0-13.el9.x86_64 2024-08-30T08:48:20.718 INFO:teuthology.orchestra.run.smithi190.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-08-30T08:48:20.718 INFO:teuthology.orchestra.run.smithi190.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-08-30T08:48:20.718 INFO:teuthology.orchestra.run.smithi190.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-08-30T08:48:20.718 INFO:teuthology.orchestra.run.smithi190.stdout: python3-babel-2.9.1-2.el9.noarch 2024-08-30T08:48:20.718 INFO:teuthology.orchestra.run.smithi190.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-08-30T08:48:20.718 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-08-30T08:48:20.718 INFO:teuthology.orchestra.run.smithi190.stdout: python3-ceph-argparse-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-30T08:48:20.718 INFO:teuthology.orchestra.run.smithi190.stdout: python3-ceph-common-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-30T08:48:20.718 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cephfs-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-30T08:48:20.718 INFO:teuthology.orchestra.run.smithi190.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-08-30T08:48:20.719 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-08-30T08:48:20.719 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-08-30T08:48:20.719 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-08-30T08:48:20.719 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-08-30T08:48:20.719 INFO:teuthology.orchestra.run.smithi190.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-08-30T08:48:20.719 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-08-30T08:48:20.719 INFO:teuthology.orchestra.run.smithi190.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-08-30T08:48:20.719 INFO:teuthology.orchestra.run.smithi190.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-08-30T08:48:20.719 INFO:teuthology.orchestra.run.smithi190.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-08-30T08:48:20.719 INFO:teuthology.orchestra.run.smithi190.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-08-30T08:48:20.719 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-08-30T08:48:20.719 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-08-30T08:48:20.719 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-08-30T08:48:20.720 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-08-30T08:48:20.720 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-08-30T08:48:20.720 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-08-30T08:48:20.720 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-08-30T08:48:20.720 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-08-30T08:48:20.720 INFO:teuthology.orchestra.run.smithi190.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-08-30T08:48:20.720 INFO:teuthology.orchestra.run.smithi190.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-08-30T08:48:20.720 INFO:teuthology.orchestra.run.smithi190.stdout: python3-mako-1.1.4-6.el9.noarch 2024-08-30T08:48:20.720 INFO:teuthology.orchestra.run.smithi190.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-08-30T08:48:20.720 INFO:teuthology.orchestra.run.smithi190.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-08-30T08:48:20.720 INFO:teuthology.orchestra.run.smithi190.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-08-30T08:48:20.720 INFO:teuthology.orchestra.run.smithi190.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-08-30T08:48:20.720 INFO:teuthology.orchestra.run.smithi190.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-08-30T08:48:20.721 INFO:teuthology.orchestra.run.smithi190.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-08-30T08:48:20.721 INFO:teuthology.orchestra.run.smithi190.stdout: python3-packaging-20.9-5.el9.noarch 2024-08-30T08:48:20.721 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-08-30T08:48:20.721 INFO:teuthology.orchestra.run.smithi190.stdout: python3-ply-3.11-14.el9.noarch 2024-08-30T08:48:20.721 INFO:teuthology.orchestra.run.smithi190.stdout: python3-portend-3.1.0-2.el9.noarch 2024-08-30T08:48:20.721 INFO:teuthology.orchestra.run.smithi190.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-08-30T08:48:20.721 INFO:teuthology.orchestra.run.smithi190.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-08-30T08:48:20.721 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-08-30T08:48:20.721 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-08-30T08:48:20.721 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-08-30T08:48:20.721 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pycparser-2.20-6.el9.noarch 2024-08-30T08:48:20.721 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-08-30T08:48:20.721 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pytz-2021.1-5.el9.noarch 2024-08-30T08:48:20.722 INFO:teuthology.orchestra.run.smithi190.stdout: python3-rados-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-30T08:48:20.722 INFO:teuthology.orchestra.run.smithi190.stdout: python3-rbd-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-30T08:48:20.722 INFO:teuthology.orchestra.run.smithi190.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-08-30T08:48:20.722 INFO:teuthology.orchestra.run.smithi190.stdout: python3-requests-2.25.1-8.el9.noarch 2024-08-30T08:48:20.722 INFO:teuthology.orchestra.run.smithi190.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-08-30T08:48:20.722 INFO:teuthology.orchestra.run.smithi190.stdout: python3-rgw-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-30T08:48:20.722 INFO:teuthology.orchestra.run.smithi190.stdout: python3-routes-2.5.1-5.el9.noarch 2024-08-30T08:48:20.722 INFO:teuthology.orchestra.run.smithi190.stdout: python3-rsa-4.9-2.el9.noarch 2024-08-30T08:48:20.722 INFO:teuthology.orchestra.run.smithi190.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-08-30T08:48:20.722 INFO:teuthology.orchestra.run.smithi190.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-08-30T08:48:20.722 INFO:teuthology.orchestra.run.smithi190.stdout: python3-toml-0.10.2-6.el9.noarch 2024-08-30T08:48:20.722 INFO:teuthology.orchestra.run.smithi190.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-08-30T08:48:20.722 INFO:teuthology.orchestra.run.smithi190.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-08-30T08:48:20.723 INFO:teuthology.orchestra.run.smithi190.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-08-30T08:48:20.723 INFO:teuthology.orchestra.run.smithi190.stdout: python3-webob-1.8.8-2.el9.noarch 2024-08-30T08:48:20.723 INFO:teuthology.orchestra.run.smithi190.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-08-30T08:48:20.723 INFO:teuthology.orchestra.run.smithi190.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-08-30T08:48:20.723 INFO:teuthology.orchestra.run.smithi190.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-08-30T08:48:20.723 INFO:teuthology.orchestra.run.smithi190.stdout: qatlib-24.02.0-1.el9.x86_64 2024-08-30T08:48:20.723 INFO:teuthology.orchestra.run.smithi190.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-08-30T08:48:20.723 INFO:teuthology.orchestra.run.smithi190.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-08-30T08:48:20.723 INFO:teuthology.orchestra.run.smithi190.stdout: rbd-fuse-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-30T08:48:20.723 INFO:teuthology.orchestra.run.smithi190.stdout: rbd-mirror-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-30T08:48:20.723 INFO:teuthology.orchestra.run.smithi190.stdout: rbd-nbd-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-30T08:48:20.723 INFO:teuthology.orchestra.run.smithi190.stdout: re2-1:20211101-20.el9.x86_64 2024-08-30T08:48:20.723 INFO:teuthology.orchestra.run.smithi190.stdout: socat-1.7.4.1-6.el9.x86_64 2024-08-30T08:48:20.724 INFO:teuthology.orchestra.run.smithi190.stdout: thrift-0.15.0-2.el9.x86_64 2024-08-30T08:48:20.724 INFO:teuthology.orchestra.run.smithi190.stdout: uuid-1.6.2-55.el9.x86_64 2024-08-30T08:48:20.724 INFO:teuthology.orchestra.run.smithi190.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-08-30T08:48:20.724 INFO:teuthology.orchestra.run.smithi190.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-08-30T08:48:20.724 INFO:teuthology.orchestra.run.smithi190.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-08-30T08:48:20.724 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-08-30T08:48:20.724 INFO:teuthology.orchestra.run.smithi190.stdout:Complete! 2024-08-30T08:48:21.060 DEBUG:teuthology.parallel:result is None 2024-08-30T08:48:24.729 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 152/152 2024-08-30T08:48:24.729 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-2:19.3.0-4560.gd19e6118.el9.x86_64 1/152 2024-08-30T08:48:24.730 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-base-2:19.3.0-4560.gd19e6118.el9.x86_64 2/152 2024-08-30T08:48:24.730 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-common-2:19.3.0-4560.gd19e6118.el9.x86_64 3/152 2024-08-30T08:48:24.730 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-fuse-2:19.3.0-4560.gd19e6118.el9.x86_64 4/152 2024-08-30T08:48:24.730 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-immutable-object-cache-2:19.3.0-4560.gd19e6 5/152 2024-08-30T08:48:24.730 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-mds-2:19.3.0-4560.gd19e6118.el9.x86_64 6/152 2024-08-30T08:48:24.730 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-mgr-2:19.3.0-4560.gd19e6118.el9.x86_64 7/152 2024-08-30T08:48:24.730 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-mon-2:19.3.0-4560.gd19e6118.el9.x86_64 8/152 2024-08-30T08:48:24.730 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-osd-2:19.3.0-4560.gd19e6118.el9.x86_64 9/152 2024-08-30T08:48:24.730 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-radosgw-2:19.3.0-4560.gd19e6118.el9.x86_64 10/152 2024-08-30T08:48:24.730 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-selinux-2:19.3.0-4560.gd19e6118.el9.x86_64 11/152 2024-08-30T08:48:24.731 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-test-2:19.3.0-4560.gd19e6118.el9.x86_64 12/152 2024-08-30T08:48:24.731 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : cephfs-mirror-2:19.3.0-4560.gd19e6118.el9.x86_64 13/152 2024-08-30T08:48:24.731 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : cephfs-shell-2:19.3.0-4560.gd19e6118.el9.x86_64 14/152 2024-08-30T08:48:24.731 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libcephfs-devel-2:19.3.0-4560.gd19e6118.el9.x86_ 15/152 2024-08-30T08:48:24.731 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libcephfs2-2:19.3.0-4560.gd19e6118.el9.x86_64 16/152 2024-08-30T08:48:24.731 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libcephsqlite-2:19.3.0-4560.gd19e6118.el9.x86_64 17/152 2024-08-30T08:48:24.731 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : librados-devel-2:19.3.0-4560.gd19e6118.el9.x86_6 18/152 2024-08-30T08:48:24.731 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libradosstriper1-2:19.3.0-4560.gd19e6118.el9.x86 19/152 2024-08-30T08:48:24.731 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : librgw2-2:19.3.0-4560.gd19e6118.el9.x86_64 20/152 2024-08-30T08:48:24.731 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-ceph-argparse-2:19.3.0-4560.gd19e6118.el 21/152 2024-08-30T08:48:24.731 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-ceph-common-2:19.3.0-4560.gd19e6118.el9. 22/152 2024-08-30T08:48:24.731 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-cephfs-2:19.3.0-4560.gd19e6118.el9.x86_6 23/152 2024-08-30T08:48:24.731 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-rados-2:19.3.0-4560.gd19e6118.el9.x86_64 24/152 2024-08-30T08:48:24.732 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-rbd-2:19.3.0-4560.gd19e6118.el9.x86_64 25/152 2024-08-30T08:48:24.732 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-rgw-2:19.3.0-4560.gd19e6118.el9.x86_64 26/152 2024-08-30T08:48:24.732 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : rbd-fuse-2:19.3.0-4560.gd19e6118.el9.x86_64 27/152 2024-08-30T08:48:24.732 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : rbd-mirror-2:19.3.0-4560.gd19e6118.el9.x86_64 28/152 2024-08-30T08:48:24.732 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : rbd-nbd-2:19.3.0-4560.gd19e6118.el9.x86_64 29/152 2024-08-30T08:48:24.732 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-grafana-dashboards-2:19.3.0-4560.gd19e6118. 30/152 2024-08-30T08:48:24.743 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-mgr-cephadm-2:19.3.0-4560.gd19e6118.el9.noa 31/152 2024-08-30T08:48:24.743 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-mgr-dashboard-2:19.3.0-4560.gd19e6118.el9.n 32/152 2024-08-30T08:48:24.743 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.3.0-4560.gd19 33/152 2024-08-30T08:48:24.743 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-mgr-modules-core-2:19.3.0-4560.gd19e6118.el 34/152 2024-08-30T08:48:24.743 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-mgr-rook-2:19.3.0-4560.gd19e6118.el9.noarch 35/152 2024-08-30T08:48:24.743 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-prometheus-alerts-2:19.3.0-4560.gd19e6118.e 36/152 2024-08-30T08:48:24.744 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-volume-2:19.3.0-4560.gd19e6118.el9.noarch 37/152 2024-08-30T08:48:24.744 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : cephadm-2:19.3.0-4560.gd19e6118.el9.noarch 38/152 2024-08-30T08:48:24.744 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : cephfs-top-2:19.3.0-4560.gd19e6118.el9.noarch 39/152 2024-08-30T08:48:24.744 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/152 2024-08-30T08:48:24.744 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/152 2024-08-30T08:48:24.744 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/152 2024-08-30T08:48:24.744 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/152 2024-08-30T08:48:24.744 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/152 2024-08-30T08:48:24.744 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/152 2024-08-30T08:48:24.744 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/152 2024-08-30T08:48:24.744 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/152 2024-08-30T08:48:24.744 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/152 2024-08-30T08:48:24.744 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/152 2024-08-30T08:48:24.745 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/152 2024-08-30T08:48:24.745 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/152 2024-08-30T08:48:24.745 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/152 2024-08-30T08:48:24.745 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/152 2024-08-30T08:48:24.745 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/152 2024-08-30T08:48:24.745 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/152 2024-08-30T08:48:24.745 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/152 2024-08-30T08:48:24.745 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/152 2024-08-30T08:48:24.745 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/152 2024-08-30T08:48:24.745 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/152 2024-08-30T08:48:24.745 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/152 2024-08-30T08:48:24.745 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/152 2024-08-30T08:48:24.745 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/152 2024-08-30T08:48:24.745 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/152 2024-08-30T08:48:24.746 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/152 2024-08-30T08:48:24.746 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/152 2024-08-30T08:48:24.746 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/152 2024-08-30T08:48:24.746 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/152 2024-08-30T08:48:24.746 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : openssl-devel-1:3.2.2-4.el9.x86_64 68/152 2024-08-30T08:48:24.746 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/152 2024-08-30T08:48:24.746 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/152 2024-08-30T08:48:24.746 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/152 2024-08-30T08:48:24.746 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/152 2024-08-30T08:48:24.747 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/152 2024-08-30T08:48:24.747 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/152 2024-08-30T08:48:24.747 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/152 2024-08-30T08:48:24.747 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/152 2024-08-30T08:48:24.747 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/152 2024-08-30T08:48:24.747 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/152 2024-08-30T08:48:24.747 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/152 2024-08-30T08:48:24.747 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/152 2024-08-30T08:48:24.747 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/152 2024-08-30T08:48:24.747 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/152 2024-08-30T08:48:24.747 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/152 2024-08-30T08:48:24.747 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/152 2024-08-30T08:48:24.748 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/152 2024-08-30T08:48:24.748 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/152 2024-08-30T08:48:24.748 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/152 2024-08-30T08:48:24.748 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/152 2024-08-30T08:48:24.748 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/152 2024-08-30T08:48:24.748 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/152 2024-08-30T08:48:24.748 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/152 2024-08-30T08:48:24.748 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/152 2024-08-30T08:48:24.748 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/152 2024-08-30T08:48:24.748 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/152 2024-08-30T08:48:24.748 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/152 2024-08-30T08:48:24.748 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/152 2024-08-30T08:48:24.749 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/152 2024-08-30T08:48:24.749 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/152 2024-08-30T08:48:24.749 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/152 2024-08-30T08:48:24.749 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/152 2024-08-30T08:48:24.749 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/152 2024-08-30T08:48:24.749 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/152 2024-08-30T08:48:24.749 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/152 2024-08-30T08:48:24.749 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/152 2024-08-30T08:48:24.749 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/152 2024-08-30T08:48:24.749 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/152 2024-08-30T08:48:24.749 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/152 2024-08-30T08:48:24.749 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/152 2024-08-30T08:48:24.749 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/152 2024-08-30T08:48:24.750 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/152 2024-08-30T08:48:24.750 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/152 2024-08-30T08:48:24.750 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/152 2024-08-30T08:48:24.750 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/152 2024-08-30T08:48:24.750 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/152 2024-08-30T08:48:24.750 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/152 2024-08-30T08:48:24.750 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/152 2024-08-30T08:48:24.750 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/152 2024-08-30T08:48:24.750 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/152 2024-08-30T08:48:24.750 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/152 2024-08-30T08:48:24.750 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/152 2024-08-30T08:48:24.750 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/152 2024-08-30T08:48:24.751 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/152 2024-08-30T08:48:24.751 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/152 2024-08-30T08:48:24.751 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/152 2024-08-30T08:48:24.751 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/152 2024-08-30T08:48:24.751 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/152 2024-08-30T08:48:24.751 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/152 2024-08-30T08:48:24.751 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/152 2024-08-30T08:48:24.751 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/152 2024-08-30T08:48:24.751 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/152 2024-08-30T08:48:24.751 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/152 2024-08-30T08:48:24.751 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/152 2024-08-30T08:48:24.751 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/152 2024-08-30T08:48:24.751 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/152 2024-08-30T08:48:24.752 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/152 2024-08-30T08:48:24.752 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/152 2024-08-30T08:48:24.752 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/152 2024-08-30T08:48:24.752 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/152 2024-08-30T08:48:24.752 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/152 2024-08-30T08:48:24.752 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/152 2024-08-30T08:48:24.752 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/152 2024-08-30T08:48:24.752 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/152 2024-08-30T08:48:24.752 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/152 2024-08-30T08:48:24.752 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/152 2024-08-30T08:48:24.752 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/152 2024-08-30T08:48:24.752 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/152 2024-08-30T08:48:24.752 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/152 2024-08-30T08:48:24.752 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/152 2024-08-30T08:48:24.753 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : librados2-2:19.3.0-4560.gd19e6118.el9.x86_64 149/152 2024-08-30T08:48:24.753 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/152 2024-08-30T08:48:24.753 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : librbd1-2:19.3.0-4560.gd19e6118.el9.x86_64 151/152 2024-08-30T08:48:26.676 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-selinux-2:19.3.0-4560.gd19e6118.el9.x86_64 105/152 2024-08-30T08:48:26.676 INFO:teuthology.orchestra.run.smithi042.stdout:skipping the directory /sys 2024-08-30T08:48:26.676 INFO:teuthology.orchestra.run.smithi042.stdout:skipping the directory /proc 2024-08-30T08:48:26.676 INFO:teuthology.orchestra.run.smithi042.stdout:skipping the directory /mnt 2024-08-30T08:48:26.676 INFO:teuthology.orchestra.run.smithi042.stdout:skipping the directory /var/tmp 2024-08-30T08:48:26.676 INFO:teuthology.orchestra.run.smithi042.stdout:skipping the directory /home 2024-08-30T08:48:26.676 INFO:teuthology.orchestra.run.smithi042.stdout:skipping the directory /root 2024-08-30T08:48:26.676 INFO:teuthology.orchestra.run.smithi042.stdout:skipping the directory /tmp 2024-08-30T08:48:26.676 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-08-30T08:48:26.863 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-volume-2:19.3.0-4560.gd19e6118.el9.noarch 106/152 2024-08-30T08:48:26.879 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-volume-2:19.3.0-4560.gd19e6118.el9.noarch 106/152 2024-08-30T08:48:26.879 INFO:teuthology.orchestra.run.smithi042.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-30T08:48:26.879 INFO:teuthology.orchestra.run.smithi042.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-08-30T08:48:26.879 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-08-30T08:48:27.029 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/152 2024-08-30T08:48:27.029 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-08-30T08:48:27.029 INFO:teuthology.orchestra.run.smithi167.stdout:Upgraded: 2024-08-30T08:48:27.029 INFO:teuthology.orchestra.run.smithi167.stdout: librados2-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-30T08:48:27.029 INFO:teuthology.orchestra.run.smithi167.stdout: librbd1-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-30T08:48:27.029 INFO:teuthology.orchestra.run.smithi167.stdout:Installed: 2024-08-30T08:48:27.029 INFO:teuthology.orchestra.run.smithi167.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-08-30T08:48:27.029 INFO:teuthology.orchestra.run.smithi167.stdout: automake-1.16.2-8.el9.noarch 2024-08-30T08:48:27.029 INFO:teuthology.orchestra.run.smithi167.stdout: bison-3.7.4-5.el9.x86_64 2024-08-30T08:48:27.029 INFO:teuthology.orchestra.run.smithi167.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-08-30T08:48:27.029 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-30T08:48:27.030 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-base-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-30T08:48:27.030 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-common-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-30T08:48:27.030 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-fuse-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-30T08:48:27.030 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-grafana-dashboards-2:19.3.0-4560.gd19e6118.el9.noarch 2024-08-30T08:48:27.030 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-immutable-object-cache-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-30T08:48:27.030 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mds-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-30T08:48:27.030 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-30T08:48:27.030 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-cephadm-2:19.3.0-4560.gd19e6118.el9.noarch 2024-08-30T08:48:27.030 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-dashboard-2:19.3.0-4560.gd19e6118.el9.noarch 2024-08-30T08:48:27.030 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-diskprediction-local-2:19.3.0-4560.gd19e6118.el9.noarch 2024-08-30T08:48:27.030 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-modules-core-2:19.3.0-4560.gd19e6118.el9.noarch 2024-08-30T08:48:27.030 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-rook-2:19.3.0-4560.gd19e6118.el9.noarch 2024-08-30T08:48:27.030 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mon-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-30T08:48:27.031 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-osd-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-30T08:48:27.031 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-prometheus-alerts-2:19.3.0-4560.gd19e6118.el9.noarch 2024-08-30T08:48:27.031 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-radosgw-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-30T08:48:27.031 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-selinux-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-30T08:48:27.031 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-test-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-30T08:48:27.031 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-volume-2:19.3.0-4560.gd19e6118.el9.noarch 2024-08-30T08:48:27.031 INFO:teuthology.orchestra.run.smithi167.stdout: cephadm-2:19.3.0-4560.gd19e6118.el9.noarch 2024-08-30T08:48:27.031 INFO:teuthology.orchestra.run.smithi167.stdout: cephfs-mirror-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-30T08:48:27.031 INFO:teuthology.orchestra.run.smithi167.stdout: cephfs-shell-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-30T08:48:27.031 INFO:teuthology.orchestra.run.smithi167.stdout: cephfs-top-2:19.3.0-4560.gd19e6118.el9.noarch 2024-08-30T08:48:27.031 INFO:teuthology.orchestra.run.smithi167.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-08-30T08:48:27.031 INFO:teuthology.orchestra.run.smithi167.stdout: flex-2.6.4-9.el9.x86_64 2024-08-30T08:48:27.031 INFO:teuthology.orchestra.run.smithi167.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-08-30T08:48:27.032 INFO:teuthology.orchestra.run.smithi167.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-08-30T08:48:27.032 INFO:teuthology.orchestra.run.smithi167.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-08-30T08:48:27.032 INFO:teuthology.orchestra.run.smithi167.stdout: fuse-2.9.9-16.el9.x86_64 2024-08-30T08:48:27.032 INFO:teuthology.orchestra.run.smithi167.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-08-30T08:48:27.032 INFO:teuthology.orchestra.run.smithi167.stdout: grpc-data-1.46.7-10.el9.noarch 2024-08-30T08:48:27.032 INFO:teuthology.orchestra.run.smithi167.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-08-30T08:48:27.032 INFO:teuthology.orchestra.run.smithi167.stdout: libarrow-9.0.0-11.el9.x86_64 2024-08-30T08:48:27.032 INFO:teuthology.orchestra.run.smithi167.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-08-30T08:48:27.032 INFO:teuthology.orchestra.run.smithi167.stdout: libcephfs-devel-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-30T08:48:27.032 INFO:teuthology.orchestra.run.smithi167.stdout: libcephfs2-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-30T08:48:27.032 INFO:teuthology.orchestra.run.smithi167.stdout: libcephsqlite-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-30T08:48:27.032 INFO:teuthology.orchestra.run.smithi167.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-08-30T08:48:27.032 INFO:teuthology.orchestra.run.smithi167.stdout: liboath-2.6.7-2.el9.x86_64 2024-08-30T08:48:27.033 INFO:teuthology.orchestra.run.smithi167.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-08-30T08:48:27.033 INFO:teuthology.orchestra.run.smithi167.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-08-30T08:48:27.033 INFO:teuthology.orchestra.run.smithi167.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-08-30T08:48:27.033 INFO:teuthology.orchestra.run.smithi167.stdout: librados-devel-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-30T08:48:27.033 INFO:teuthology.orchestra.run.smithi167.stdout: libradosstriper1-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-30T08:48:27.033 INFO:teuthology.orchestra.run.smithi167.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-08-30T08:48:27.033 INFO:teuthology.orchestra.run.smithi167.stdout: librgw2-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-30T08:48:27.033 INFO:teuthology.orchestra.run.smithi167.stdout: libtool-2.4.6-46.el9.x86_64 2024-08-30T08:48:27.033 INFO:teuthology.orchestra.run.smithi167.stdout: libunwind-1.6.2-1.el9.x86_64 2024-08-30T08:48:27.033 INFO:teuthology.orchestra.run.smithi167.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-08-30T08:48:27.033 INFO:teuthology.orchestra.run.smithi167.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-08-30T08:48:27.033 INFO:teuthology.orchestra.run.smithi167.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-08-30T08:48:27.033 INFO:teuthology.orchestra.run.smithi167.stdout: lua-5.4.4-4.el9.x86_64 2024-08-30T08:48:27.033 INFO:teuthology.orchestra.run.smithi167.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-08-30T08:48:27.034 INFO:teuthology.orchestra.run.smithi167.stdout: luarocks-3.9.2-5.el9.noarch 2024-08-30T08:48:27.034 INFO:teuthology.orchestra.run.smithi167.stdout: openblas-0.3.26-2.el9.x86_64 2024-08-30T08:48:27.034 INFO:teuthology.orchestra.run.smithi167.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-08-30T08:48:27.034 INFO:teuthology.orchestra.run.smithi167.stdout: openssl-devel-1:3.2.2-4.el9.x86_64 2024-08-30T08:48:27.034 INFO:teuthology.orchestra.run.smithi167.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-08-30T08:48:27.034 INFO:teuthology.orchestra.run.smithi167.stdout: postgresql-13.16-1.el9.x86_64 2024-08-30T08:48:27.034 INFO:teuthology.orchestra.run.smithi167.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-08-30T08:48:27.034 INFO:teuthology.orchestra.run.smithi167.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-08-30T08:48:27.034 INFO:teuthology.orchestra.run.smithi167.stdout: postgresql-server-13.16-1.el9.x86_64 2024-08-30T08:48:27.034 INFO:teuthology.orchestra.run.smithi167.stdout: protobuf-3.14.0-13.el9.x86_64 2024-08-30T08:48:27.034 INFO:teuthology.orchestra.run.smithi167.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-08-30T08:48:27.034 INFO:teuthology.orchestra.run.smithi167.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-08-30T08:48:27.035 INFO:teuthology.orchestra.run.smithi167.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-08-30T08:48:27.035 INFO:teuthology.orchestra.run.smithi167.stdout: python3-babel-2.9.1-2.el9.noarch 2024-08-30T08:48:27.035 INFO:teuthology.orchestra.run.smithi167.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-08-30T08:48:27.035 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-08-30T08:48:27.035 INFO:teuthology.orchestra.run.smithi167.stdout: python3-ceph-argparse-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-30T08:48:27.035 INFO:teuthology.orchestra.run.smithi167.stdout: python3-ceph-common-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-30T08:48:27.035 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cephfs-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-30T08:48:27.035 INFO:teuthology.orchestra.run.smithi167.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-08-30T08:48:27.035 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-08-30T08:48:27.035 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-08-30T08:48:27.035 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-08-30T08:48:27.035 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-08-30T08:48:27.035 INFO:teuthology.orchestra.run.smithi167.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-08-30T08:48:27.035 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-08-30T08:48:27.036 INFO:teuthology.orchestra.run.smithi167.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-08-30T08:48:27.036 INFO:teuthology.orchestra.run.smithi167.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-08-30T08:48:27.036 INFO:teuthology.orchestra.run.smithi167.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-08-30T08:48:27.036 INFO:teuthology.orchestra.run.smithi167.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-08-30T08:48:27.036 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-08-30T08:48:27.036 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-08-30T08:48:27.036 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-08-30T08:48:27.036 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-08-30T08:48:27.036 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-08-30T08:48:27.036 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-08-30T08:48:27.036 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-08-30T08:48:27.036 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-08-30T08:48:27.036 INFO:teuthology.orchestra.run.smithi167.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-08-30T08:48:27.037 INFO:teuthology.orchestra.run.smithi167.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-08-30T08:48:27.037 INFO:teuthology.orchestra.run.smithi167.stdout: python3-mako-1.1.4-6.el9.noarch 2024-08-30T08:48:27.037 INFO:teuthology.orchestra.run.smithi167.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-08-30T08:48:27.037 INFO:teuthology.orchestra.run.smithi167.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-08-30T08:48:27.037 INFO:teuthology.orchestra.run.smithi167.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-08-30T08:48:27.037 INFO:teuthology.orchestra.run.smithi167.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-08-30T08:48:27.037 INFO:teuthology.orchestra.run.smithi167.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-08-30T08:48:27.037 INFO:teuthology.orchestra.run.smithi167.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-08-30T08:48:27.037 INFO:teuthology.orchestra.run.smithi167.stdout: python3-packaging-20.9-5.el9.noarch 2024-08-30T08:48:27.037 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-08-30T08:48:27.037 INFO:teuthology.orchestra.run.smithi167.stdout: python3-ply-3.11-14.el9.noarch 2024-08-30T08:48:27.037 INFO:teuthology.orchestra.run.smithi167.stdout: python3-portend-3.1.0-2.el9.noarch 2024-08-30T08:48:27.038 INFO:teuthology.orchestra.run.smithi167.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-08-30T08:48:27.038 INFO:teuthology.orchestra.run.smithi167.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-08-30T08:48:27.038 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-08-30T08:48:27.038 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-08-30T08:48:27.038 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-08-30T08:48:27.038 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pycparser-2.20-6.el9.noarch 2024-08-30T08:48:27.038 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-08-30T08:48:27.038 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pytz-2021.1-5.el9.noarch 2024-08-30T08:48:27.038 INFO:teuthology.orchestra.run.smithi167.stdout: python3-rados-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-30T08:48:27.038 INFO:teuthology.orchestra.run.smithi167.stdout: python3-rbd-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-30T08:48:27.038 INFO:teuthology.orchestra.run.smithi167.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-08-30T08:48:27.038 INFO:teuthology.orchestra.run.smithi167.stdout: python3-requests-2.25.1-8.el9.noarch 2024-08-30T08:48:27.038 INFO:teuthology.orchestra.run.smithi167.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-08-30T08:48:27.038 INFO:teuthology.orchestra.run.smithi167.stdout: python3-rgw-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-30T08:48:27.039 INFO:teuthology.orchestra.run.smithi167.stdout: python3-routes-2.5.1-5.el9.noarch 2024-08-30T08:48:27.039 INFO:teuthology.orchestra.run.smithi167.stdout: python3-rsa-4.9-2.el9.noarch 2024-08-30T08:48:27.039 INFO:teuthology.orchestra.run.smithi167.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-08-30T08:48:27.039 INFO:teuthology.orchestra.run.smithi167.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-08-30T08:48:27.039 INFO:teuthology.orchestra.run.smithi167.stdout: python3-toml-0.10.2-6.el9.noarch 2024-08-30T08:48:27.039 INFO:teuthology.orchestra.run.smithi167.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-08-30T08:48:27.039 INFO:teuthology.orchestra.run.smithi167.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-08-30T08:48:27.039 INFO:teuthology.orchestra.run.smithi167.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-08-30T08:48:27.039 INFO:teuthology.orchestra.run.smithi167.stdout: python3-webob-1.8.8-2.el9.noarch 2024-08-30T08:48:27.039 INFO:teuthology.orchestra.run.smithi167.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-08-30T08:48:27.039 INFO:teuthology.orchestra.run.smithi167.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-08-30T08:48:27.039 INFO:teuthology.orchestra.run.smithi167.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-08-30T08:48:27.039 INFO:teuthology.orchestra.run.smithi167.stdout: qatlib-24.02.0-1.el9.x86_64 2024-08-30T08:48:27.040 INFO:teuthology.orchestra.run.smithi167.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-08-30T08:48:27.040 INFO:teuthology.orchestra.run.smithi167.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-08-30T08:48:27.040 INFO:teuthology.orchestra.run.smithi167.stdout: rbd-fuse-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-30T08:48:27.040 INFO:teuthology.orchestra.run.smithi167.stdout: rbd-mirror-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-30T08:48:27.040 INFO:teuthology.orchestra.run.smithi167.stdout: rbd-nbd-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-30T08:48:27.040 INFO:teuthology.orchestra.run.smithi167.stdout: re2-1:20211101-20.el9.x86_64 2024-08-30T08:48:27.040 INFO:teuthology.orchestra.run.smithi167.stdout: socat-1.7.4.1-6.el9.x86_64 2024-08-30T08:48:27.040 INFO:teuthology.orchestra.run.smithi167.stdout: thrift-0.15.0-2.el9.x86_64 2024-08-30T08:48:27.040 INFO:teuthology.orchestra.run.smithi167.stdout: uuid-1.6.2-55.el9.x86_64 2024-08-30T08:48:27.040 INFO:teuthology.orchestra.run.smithi167.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-08-30T08:48:27.040 INFO:teuthology.orchestra.run.smithi167.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-08-30T08:48:27.041 INFO:teuthology.orchestra.run.smithi167.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-08-30T08:48:27.041 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-08-30T08:48:27.041 INFO:teuthology.orchestra.run.smithi167.stdout:Complete! 2024-08-30T08:48:27.405 DEBUG:teuthology.parallel:result is None 2024-08-30T08:48:28.533 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-osd-2:19.3.0-4560.gd19e6118.el9.x86_64 107/152 2024-08-30T08:48:28.560 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-osd-2:19.3.0-4560.gd19e6118.el9.x86_64 107/152 2024-08-30T08:48:28.560 INFO:teuthology.orchestra.run.smithi042.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-30T08:48:28.560 INFO:teuthology.orchestra.run.smithi042.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-08-30T08:48:28.560 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-08-30T08:48:28.560 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-08-30T08:48:28.560 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-08-30T08:48:28.793 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-mds-2:19.3.0-4560.gd19e6118.el9.x86_64 108/152 2024-08-30T08:48:28.815 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-mds-2:19.3.0-4560.gd19e6118.el9.x86_64 108/152 2024-08-30T08:48:28.815 INFO:teuthology.orchestra.run.smithi042.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-30T08:48:28.815 INFO:teuthology.orchestra.run.smithi042.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-08-30T08:48:28.815 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-08-30T08:48:28.816 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-08-30T08:48:28.816 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-08-30T08:48:29.301 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-mon-2:19.3.0-4560.gd19e6118.el9.x86_64 109/152 2024-08-30T08:48:29.324 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-mon-2:19.3.0-4560.gd19e6118.el9.x86_64 109/152 2024-08-30T08:48:29.324 INFO:teuthology.orchestra.run.smithi042.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-30T08:48:29.324 INFO:teuthology.orchestra.run.smithi042.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-08-30T08:48:29.324 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-08-30T08:48:29.325 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-08-30T08:48:29.325 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-08-30T08:48:29.432 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : automake-1.16.2-8.el9.noarch 110/152 2024-08-30T08:48:29.490 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-ply-3.11-14.el9.noarch 111/152 2024-08-30T08:48:29.554 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 112/152 2024-08-30T08:48:29.746 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 113/152 2024-08-30T08:48:29.803 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 114/152 2024-08-30T08:48:29.883 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 115/152 2024-08-30T08:48:29.980 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 116/152 2024-08-30T08:48:30.122 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 117/152 2024-08-30T08:48:30.161 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 118/152 2024-08-30T08:48:30.192 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 119/152 2024-08-30T08:48:30.229 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : fuse-2.9.9-16.el9.x86_64 120/152 2024-08-30T08:48:30.626 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: cephadm-2:19.3.0-4560.gd19e6118.el9.noarch 121/152 2024-08-30T08:48:30.652 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : cephadm-2:19.3.0-4560.gd19e6118.el9.noarch 121/152 2024-08-30T08:48:30.685 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-prometheus-alerts-2:19.3.0-4560.gd19e6118.e 122/152 2024-08-30T08:48:30.752 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-grafana-dashboards-2:19.3.0-4560.gd19e6118. 123/152 2024-08-30T08:48:30.783 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-mgr-cephadm-2:19.3.0-4560.gd19e6118.el9.noa 124/152 2024-08-30T08:48:34.136 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.3.0-4560.gd19e6118.el9.noa 124/152 2024-08-30T08:48:34.191 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-mgr-dashboard-2:19.3.0-4560.gd19e6118.el9.n 125/152 2024-08-30T08:48:35.206 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.3.0-4560.gd19e6118.el9.n 125/152 2024-08-30T08:48:35.233 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-mgr-diskprediction-local-2:19.3.0-4560.gd19 126/152 2024-08-30T08:48:35.354 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.3.0-4560.gd19 126/152 2024-08-30T08:48:35.533 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-mgr-modules-core-2:19.3.0-4560.gd19e6118.el 127/152 2024-08-30T08:48:35.560 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-mgr-2:19.3.0-4560.gd19e6118.el9.x86_64 128/152 2024-08-30T08:48:35.583 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-mgr-2:19.3.0-4560.gd19e6118.el9.x86_64 128/152 2024-08-30T08:48:35.583 INFO:teuthology.orchestra.run.smithi042.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-30T08:48:35.583 INFO:teuthology.orchestra.run.smithi042.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-08-30T08:48:35.584 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-08-30T08:48:35.584 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-08-30T08:48:35.584 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-08-30T08:48:35.635 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-mgr-rook-2:19.3.0-4560.gd19e6118.el9.noarch 129/152 2024-08-30T08:48:35.647 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-mgr-rook-2:19.3.0-4560.gd19e6118.el9.noarch 129/152 2024-08-30T08:48:35.750 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-2:19.3.0-4560.gd19e6118.el9.x86_64 130/152 2024-08-30T08:48:35.801 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-fuse-2:19.3.0-4560.gd19e6118.el9.x86_64 131/152 2024-08-30T08:48:36.813 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : libtool-2.4.6-46.el9.x86_64 132/152 2024-08-30T08:48:36.838 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-radosgw-2:19.3.0-4560.gd19e6118.el9.x86_64 133/152 2024-08-30T08:48:36.861 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-radosgw-2:19.3.0-4560.gd19e6118.el9.x86_64 133/152 2024-08-30T08:48:36.861 INFO:teuthology.orchestra.run.smithi042.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-30T08:48:36.861 INFO:teuthology.orchestra.run.smithi042.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-08-30T08:48:36.861 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-08-30T08:48:36.861 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-08-30T08:48:36.861 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-08-30T08:48:36.913 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-immutable-object-cache-2:19.3.0-4560.gd19e6 134/152 2024-08-30T08:48:36.935 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.3.0-4560.gd19e6 134/152 2024-08-30T08:48:36.936 INFO:teuthology.orchestra.run.smithi042.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-30T08:48:36.936 INFO:teuthology.orchestra.run.smithi042.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-08-30T08:48:36.936 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-08-30T08:48:36.997 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : cephfs-mirror-2:19.3.0-4560.gd19e6118.el9.x86_64 135/152 2024-08-30T08:48:37.020 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: cephfs-mirror-2:19.3.0-4560.gd19e6118.el9.x86_64 135/152 2024-08-30T08:48:37.020 INFO:teuthology.orchestra.run.smithi042.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-30T08:48:37.020 INFO:teuthology.orchestra.run.smithi042.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-08-30T08:48:37.020 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-08-30T08:48:37.021 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-08-30T08:48:37.021 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-08-30T08:48:37.356 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : rbd-mirror-2:19.3.0-4560.gd19e6118.el9.x86_64 136/152 2024-08-30T08:48:37.382 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: rbd-mirror-2:19.3.0-4560.gd19e6118.el9.x86_64 136/152 2024-08-30T08:48:37.382 INFO:teuthology.orchestra.run.smithi042.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-30T08:48:37.382 INFO:teuthology.orchestra.run.smithi042.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-08-30T08:48:37.382 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-08-30T08:48:37.382 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-08-30T08:48:37.382 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-08-30T08:48:41.977 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-test-2:19.3.0-4560.gd19e6118.el9.x86_64 137/152 2024-08-30T08:48:42.019 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : rbd-fuse-2:19.3.0-4560.gd19e6118.el9.x86_64 138/152 2024-08-30T08:48:42.060 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : rbd-nbd-2:19.3.0-4560.gd19e6118.el9.x86_64 139/152 2024-08-30T08:48:42.091 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 140/152 2024-08-30T08:48:42.118 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 141/152 2024-08-30T08:48:42.315 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : cephfs-shell-2:19.3.0-4560.gd19e6118.el9.x86_64 142/152 2024-08-30T08:48:42.365 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 143/152 2024-08-30T08:48:42.394 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : libcephfs-devel-2:19.3.0-4560.gd19e6118.el9.x86_ 144/152 2024-08-30T08:48:43.020 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 145/152 2024-08-30T08:48:43.058 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 145/152 2024-08-30T08:48:43.078 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 145/152 2024-08-30T08:48:43.967 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : cephfs-top-2:19.3.0-4560.gd19e6118.el9.noarch 146/152 2024-08-30T08:48:44.055 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : openssl-devel-1:3.2.2-4.el9.x86_64 147/152 2024-08-30T08:48:44.097 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 148/152 2024-08-30T08:48:44.181 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : flex-2.6.4-9.el9.x86_64 149/152 2024-08-30T08:48:44.209 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : bison-3.7.4-5.el9.x86_64 150/152 2024-08-30T08:48:44.209 INFO:teuthology.orchestra.run.smithi042.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 151/152 2024-08-30T08:48:44.293 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 151/152 2024-08-30T08:48:44.293 INFO:teuthology.orchestra.run.smithi042.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 152/152 2024-08-30T08:48:51.030 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 152/152 2024-08-30T08:48:51.030 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-2:19.3.0-4560.gd19e6118.el9.x86_64 1/152 2024-08-30T08:48:51.031 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-base-2:19.3.0-4560.gd19e6118.el9.x86_64 2/152 2024-08-30T08:48:51.031 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-common-2:19.3.0-4560.gd19e6118.el9.x86_64 3/152 2024-08-30T08:48:51.031 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-fuse-2:19.3.0-4560.gd19e6118.el9.x86_64 4/152 2024-08-30T08:48:51.031 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-immutable-object-cache-2:19.3.0-4560.gd19e6 5/152 2024-08-30T08:48:51.031 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-mds-2:19.3.0-4560.gd19e6118.el9.x86_64 6/152 2024-08-30T08:48:51.031 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-mgr-2:19.3.0-4560.gd19e6118.el9.x86_64 7/152 2024-08-30T08:48:51.031 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-mon-2:19.3.0-4560.gd19e6118.el9.x86_64 8/152 2024-08-30T08:48:51.031 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-osd-2:19.3.0-4560.gd19e6118.el9.x86_64 9/152 2024-08-30T08:48:51.031 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-radosgw-2:19.3.0-4560.gd19e6118.el9.x86_64 10/152 2024-08-30T08:48:51.031 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-selinux-2:19.3.0-4560.gd19e6118.el9.x86_64 11/152 2024-08-30T08:48:51.031 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-test-2:19.3.0-4560.gd19e6118.el9.x86_64 12/152 2024-08-30T08:48:51.031 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : cephfs-mirror-2:19.3.0-4560.gd19e6118.el9.x86_64 13/152 2024-08-30T08:48:51.032 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : cephfs-shell-2:19.3.0-4560.gd19e6118.el9.x86_64 14/152 2024-08-30T08:48:51.032 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : libcephfs-devel-2:19.3.0-4560.gd19e6118.el9.x86_ 15/152 2024-08-30T08:48:51.032 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : libcephfs2-2:19.3.0-4560.gd19e6118.el9.x86_64 16/152 2024-08-30T08:48:51.032 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : libcephsqlite-2:19.3.0-4560.gd19e6118.el9.x86_64 17/152 2024-08-30T08:48:51.032 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : librados-devel-2:19.3.0-4560.gd19e6118.el9.x86_6 18/152 2024-08-30T08:48:51.032 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : libradosstriper1-2:19.3.0-4560.gd19e6118.el9.x86 19/152 2024-08-30T08:48:51.032 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : librgw2-2:19.3.0-4560.gd19e6118.el9.x86_64 20/152 2024-08-30T08:48:51.032 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-ceph-argparse-2:19.3.0-4560.gd19e6118.el 21/152 2024-08-30T08:48:51.032 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-ceph-common-2:19.3.0-4560.gd19e6118.el9. 22/152 2024-08-30T08:48:51.032 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-cephfs-2:19.3.0-4560.gd19e6118.el9.x86_6 23/152 2024-08-30T08:48:51.032 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-rados-2:19.3.0-4560.gd19e6118.el9.x86_64 24/152 2024-08-30T08:48:51.032 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-rbd-2:19.3.0-4560.gd19e6118.el9.x86_64 25/152 2024-08-30T08:48:51.033 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-rgw-2:19.3.0-4560.gd19e6118.el9.x86_64 26/152 2024-08-30T08:48:51.033 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : rbd-fuse-2:19.3.0-4560.gd19e6118.el9.x86_64 27/152 2024-08-30T08:48:51.033 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : rbd-mirror-2:19.3.0-4560.gd19e6118.el9.x86_64 28/152 2024-08-30T08:48:51.033 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : rbd-nbd-2:19.3.0-4560.gd19e6118.el9.x86_64 29/152 2024-08-30T08:48:51.033 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-grafana-dashboards-2:19.3.0-4560.gd19e6118. 30/152 2024-08-30T08:48:51.033 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-mgr-cephadm-2:19.3.0-4560.gd19e6118.el9.noa 31/152 2024-08-30T08:48:51.033 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-mgr-dashboard-2:19.3.0-4560.gd19e6118.el9.n 32/152 2024-08-30T08:48:51.033 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.3.0-4560.gd19 33/152 2024-08-30T08:48:51.033 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-mgr-modules-core-2:19.3.0-4560.gd19e6118.el 34/152 2024-08-30T08:48:51.033 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-mgr-rook-2:19.3.0-4560.gd19e6118.el9.noarch 35/152 2024-08-30T08:48:51.033 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-prometheus-alerts-2:19.3.0-4560.gd19e6118.e 36/152 2024-08-30T08:48:51.033 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-volume-2:19.3.0-4560.gd19e6118.el9.noarch 37/152 2024-08-30T08:48:51.033 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : cephadm-2:19.3.0-4560.gd19e6118.el9.noarch 38/152 2024-08-30T08:48:51.033 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : cephfs-top-2:19.3.0-4560.gd19e6118.el9.noarch 39/152 2024-08-30T08:48:51.034 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/152 2024-08-30T08:48:51.034 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/152 2024-08-30T08:48:51.034 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/152 2024-08-30T08:48:51.034 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/152 2024-08-30T08:48:51.034 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/152 2024-08-30T08:48:51.034 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/152 2024-08-30T08:48:51.034 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/152 2024-08-30T08:48:51.034 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/152 2024-08-30T08:48:51.034 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/152 2024-08-30T08:48:51.034 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/152 2024-08-30T08:48:51.035 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/152 2024-08-30T08:48:51.035 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/152 2024-08-30T08:48:51.035 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/152 2024-08-30T08:48:51.035 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/152 2024-08-30T08:48:51.035 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/152 2024-08-30T08:48:51.035 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/152 2024-08-30T08:48:51.035 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/152 2024-08-30T08:48:51.035 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/152 2024-08-30T08:48:51.035 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/152 2024-08-30T08:48:51.035 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/152 2024-08-30T08:48:51.035 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/152 2024-08-30T08:48:51.035 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/152 2024-08-30T08:48:51.036 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/152 2024-08-30T08:48:51.036 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/152 2024-08-30T08:48:51.036 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/152 2024-08-30T08:48:51.036 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/152 2024-08-30T08:48:51.036 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/152 2024-08-30T08:48:51.036 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/152 2024-08-30T08:48:51.036 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : openssl-devel-1:3.2.2-4.el9.x86_64 68/152 2024-08-30T08:48:51.036 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/152 2024-08-30T08:48:51.036 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/152 2024-08-30T08:48:51.036 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/152 2024-08-30T08:48:51.036 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/152 2024-08-30T08:48:51.036 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/152 2024-08-30T08:48:51.036 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/152 2024-08-30T08:48:51.037 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/152 2024-08-30T08:48:51.037 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/152 2024-08-30T08:48:51.037 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/152 2024-08-30T08:48:51.037 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/152 2024-08-30T08:48:51.037 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/152 2024-08-30T08:48:51.037 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/152 2024-08-30T08:48:51.037 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/152 2024-08-30T08:48:51.037 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/152 2024-08-30T08:48:51.037 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/152 2024-08-30T08:48:51.037 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/152 2024-08-30T08:48:51.037 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/152 2024-08-30T08:48:51.037 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/152 2024-08-30T08:48:51.037 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/152 2024-08-30T08:48:51.038 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/152 2024-08-30T08:48:51.038 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/152 2024-08-30T08:48:51.038 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/152 2024-08-30T08:48:51.038 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/152 2024-08-30T08:48:51.038 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/152 2024-08-30T08:48:51.038 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/152 2024-08-30T08:48:51.038 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/152 2024-08-30T08:48:51.038 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/152 2024-08-30T08:48:51.038 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/152 2024-08-30T08:48:51.038 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/152 2024-08-30T08:48:51.038 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/152 2024-08-30T08:48:51.038 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/152 2024-08-30T08:48:51.038 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/152 2024-08-30T08:48:51.038 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/152 2024-08-30T08:48:51.039 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/152 2024-08-30T08:48:51.039 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/152 2024-08-30T08:48:51.039 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/152 2024-08-30T08:48:51.039 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/152 2024-08-30T08:48:51.039 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/152 2024-08-30T08:48:51.039 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/152 2024-08-30T08:48:51.039 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/152 2024-08-30T08:48:51.039 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/152 2024-08-30T08:48:51.039 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/152 2024-08-30T08:48:51.039 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/152 2024-08-30T08:48:51.039 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/152 2024-08-30T08:48:51.040 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/152 2024-08-30T08:48:51.040 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/152 2024-08-30T08:48:51.040 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/152 2024-08-30T08:48:51.040 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/152 2024-08-30T08:48:51.040 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/152 2024-08-30T08:48:51.040 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/152 2024-08-30T08:48:51.040 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/152 2024-08-30T08:48:51.040 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/152 2024-08-30T08:48:51.040 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/152 2024-08-30T08:48:51.040 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/152 2024-08-30T08:48:51.040 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/152 2024-08-30T08:48:51.041 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/152 2024-08-30T08:48:51.041 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/152 2024-08-30T08:48:51.041 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/152 2024-08-30T08:48:51.041 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/152 2024-08-30T08:48:51.041 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/152 2024-08-30T08:48:51.041 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/152 2024-08-30T08:48:51.041 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/152 2024-08-30T08:48:51.041 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/152 2024-08-30T08:48:51.041 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/152 2024-08-30T08:48:51.041 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/152 2024-08-30T08:48:51.041 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/152 2024-08-30T08:48:51.041 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/152 2024-08-30T08:48:51.042 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/152 2024-08-30T08:48:51.042 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/152 2024-08-30T08:48:51.042 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/152 2024-08-30T08:48:51.042 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/152 2024-08-30T08:48:51.042 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/152 2024-08-30T08:48:51.042 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/152 2024-08-30T08:48:51.042 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/152 2024-08-30T08:48:51.042 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/152 2024-08-30T08:48:51.042 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/152 2024-08-30T08:48:51.042 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/152 2024-08-30T08:48:51.042 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/152 2024-08-30T08:48:51.042 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/152 2024-08-30T08:48:51.042 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/152 2024-08-30T08:48:51.042 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : librados2-2:19.3.0-4560.gd19e6118.el9.x86_64 149/152 2024-08-30T08:48:51.042 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/152 2024-08-30T08:48:51.043 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : librbd1-2:19.3.0-4560.gd19e6118.el9.x86_64 151/152 2024-08-30T08:48:53.476 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/152 2024-08-30T08:48:53.477 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-08-30T08:48:53.477 INFO:teuthology.orchestra.run.smithi042.stdout:Upgraded: 2024-08-30T08:48:53.477 INFO:teuthology.orchestra.run.smithi042.stdout: librados2-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-30T08:48:53.477 INFO:teuthology.orchestra.run.smithi042.stdout: librbd1-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-30T08:48:53.477 INFO:teuthology.orchestra.run.smithi042.stdout:Installed: 2024-08-30T08:48:53.477 INFO:teuthology.orchestra.run.smithi042.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-08-30T08:48:53.477 INFO:teuthology.orchestra.run.smithi042.stdout: automake-1.16.2-8.el9.noarch 2024-08-30T08:48:53.477 INFO:teuthology.orchestra.run.smithi042.stdout: bison-3.7.4-5.el9.x86_64 2024-08-30T08:48:53.477 INFO:teuthology.orchestra.run.smithi042.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-08-30T08:48:53.477 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-30T08:48:53.477 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-base-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-30T08:48:53.477 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-common-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-30T08:48:53.478 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-fuse-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-30T08:48:53.478 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-grafana-dashboards-2:19.3.0-4560.gd19e6118.el9.noarch 2024-08-30T08:48:53.478 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-immutable-object-cache-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-30T08:48:53.478 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mds-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-30T08:48:53.478 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mgr-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-30T08:48:53.478 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mgr-cephadm-2:19.3.0-4560.gd19e6118.el9.noarch 2024-08-30T08:48:53.478 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mgr-dashboard-2:19.3.0-4560.gd19e6118.el9.noarch 2024-08-30T08:48:53.478 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mgr-diskprediction-local-2:19.3.0-4560.gd19e6118.el9.noarch 2024-08-30T08:48:53.478 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mgr-modules-core-2:19.3.0-4560.gd19e6118.el9.noarch 2024-08-30T08:48:53.478 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mgr-rook-2:19.3.0-4560.gd19e6118.el9.noarch 2024-08-30T08:48:53.478 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mon-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-30T08:48:53.478 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-osd-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-30T08:48:53.478 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-prometheus-alerts-2:19.3.0-4560.gd19e6118.el9.noarch 2024-08-30T08:48:53.479 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-radosgw-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-30T08:48:53.479 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-selinux-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-30T08:48:53.479 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-test-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-30T08:48:53.479 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-volume-2:19.3.0-4560.gd19e6118.el9.noarch 2024-08-30T08:48:53.479 INFO:teuthology.orchestra.run.smithi042.stdout: cephadm-2:19.3.0-4560.gd19e6118.el9.noarch 2024-08-30T08:48:53.479 INFO:teuthology.orchestra.run.smithi042.stdout: cephfs-mirror-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-30T08:48:53.479 INFO:teuthology.orchestra.run.smithi042.stdout: cephfs-shell-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-30T08:48:53.479 INFO:teuthology.orchestra.run.smithi042.stdout: cephfs-top-2:19.3.0-4560.gd19e6118.el9.noarch 2024-08-30T08:48:53.479 INFO:teuthology.orchestra.run.smithi042.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-08-30T08:48:53.479 INFO:teuthology.orchestra.run.smithi042.stdout: flex-2.6.4-9.el9.x86_64 2024-08-30T08:48:53.479 INFO:teuthology.orchestra.run.smithi042.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-08-30T08:48:53.479 INFO:teuthology.orchestra.run.smithi042.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-08-30T08:48:53.479 INFO:teuthology.orchestra.run.smithi042.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-08-30T08:48:53.480 INFO:teuthology.orchestra.run.smithi042.stdout: fuse-2.9.9-16.el9.x86_64 2024-08-30T08:48:53.480 INFO:teuthology.orchestra.run.smithi042.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-08-30T08:48:53.480 INFO:teuthology.orchestra.run.smithi042.stdout: grpc-data-1.46.7-10.el9.noarch 2024-08-30T08:48:53.480 INFO:teuthology.orchestra.run.smithi042.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-08-30T08:48:53.480 INFO:teuthology.orchestra.run.smithi042.stdout: libarrow-9.0.0-11.el9.x86_64 2024-08-30T08:48:53.480 INFO:teuthology.orchestra.run.smithi042.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-08-30T08:48:53.480 INFO:teuthology.orchestra.run.smithi042.stdout: libcephfs-devel-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-30T08:48:53.480 INFO:teuthology.orchestra.run.smithi042.stdout: libcephfs2-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-30T08:48:53.480 INFO:teuthology.orchestra.run.smithi042.stdout: libcephsqlite-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-30T08:48:53.480 INFO:teuthology.orchestra.run.smithi042.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-08-30T08:48:53.480 INFO:teuthology.orchestra.run.smithi042.stdout: liboath-2.6.7-2.el9.x86_64 2024-08-30T08:48:53.480 INFO:teuthology.orchestra.run.smithi042.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-08-30T08:48:53.480 INFO:teuthology.orchestra.run.smithi042.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-08-30T08:48:53.481 INFO:teuthology.orchestra.run.smithi042.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-08-30T08:48:53.481 INFO:teuthology.orchestra.run.smithi042.stdout: librados-devel-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-30T08:48:53.481 INFO:teuthology.orchestra.run.smithi042.stdout: libradosstriper1-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-30T08:48:53.481 INFO:teuthology.orchestra.run.smithi042.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-08-30T08:48:53.481 INFO:teuthology.orchestra.run.smithi042.stdout: librgw2-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-30T08:48:53.481 INFO:teuthology.orchestra.run.smithi042.stdout: libtool-2.4.6-46.el9.x86_64 2024-08-30T08:48:53.481 INFO:teuthology.orchestra.run.smithi042.stdout: libunwind-1.6.2-1.el9.x86_64 2024-08-30T08:48:53.481 INFO:teuthology.orchestra.run.smithi042.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-08-30T08:48:53.481 INFO:teuthology.orchestra.run.smithi042.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-08-30T08:48:53.481 INFO:teuthology.orchestra.run.smithi042.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-08-30T08:48:53.481 INFO:teuthology.orchestra.run.smithi042.stdout: lua-5.4.4-4.el9.x86_64 2024-08-30T08:48:53.481 INFO:teuthology.orchestra.run.smithi042.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-08-30T08:48:53.481 INFO:teuthology.orchestra.run.smithi042.stdout: luarocks-3.9.2-5.el9.noarch 2024-08-30T08:48:53.482 INFO:teuthology.orchestra.run.smithi042.stdout: openblas-0.3.26-2.el9.x86_64 2024-08-30T08:48:53.482 INFO:teuthology.orchestra.run.smithi042.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-08-30T08:48:53.482 INFO:teuthology.orchestra.run.smithi042.stdout: openssl-devel-1:3.2.2-4.el9.x86_64 2024-08-30T08:48:53.482 INFO:teuthology.orchestra.run.smithi042.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-08-30T08:48:53.482 INFO:teuthology.orchestra.run.smithi042.stdout: postgresql-13.16-1.el9.x86_64 2024-08-30T08:48:53.482 INFO:teuthology.orchestra.run.smithi042.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-08-30T08:48:53.482 INFO:teuthology.orchestra.run.smithi042.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-08-30T08:48:53.482 INFO:teuthology.orchestra.run.smithi042.stdout: postgresql-server-13.16-1.el9.x86_64 2024-08-30T08:48:53.482 INFO:teuthology.orchestra.run.smithi042.stdout: protobuf-3.14.0-13.el9.x86_64 2024-08-30T08:48:53.482 INFO:teuthology.orchestra.run.smithi042.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-08-30T08:48:53.482 INFO:teuthology.orchestra.run.smithi042.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-08-30T08:48:53.482 INFO:teuthology.orchestra.run.smithi042.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-08-30T08:48:53.482 INFO:teuthology.orchestra.run.smithi042.stdout: python3-babel-2.9.1-2.el9.noarch 2024-08-30T08:48:53.483 INFO:teuthology.orchestra.run.smithi042.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-08-30T08:48:53.483 INFO:teuthology.orchestra.run.smithi042.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-08-30T08:48:53.483 INFO:teuthology.orchestra.run.smithi042.stdout: python3-ceph-argparse-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-30T08:48:53.483 INFO:teuthology.orchestra.run.smithi042.stdout: python3-ceph-common-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-30T08:48:53.483 INFO:teuthology.orchestra.run.smithi042.stdout: python3-cephfs-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-30T08:48:53.483 INFO:teuthology.orchestra.run.smithi042.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-08-30T08:48:53.483 INFO:teuthology.orchestra.run.smithi042.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-08-30T08:48:53.483 INFO:teuthology.orchestra.run.smithi042.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-08-30T08:48:53.483 INFO:teuthology.orchestra.run.smithi042.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-08-30T08:48:53.483 INFO:teuthology.orchestra.run.smithi042.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-08-30T08:48:53.483 INFO:teuthology.orchestra.run.smithi042.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-08-30T08:48:53.483 INFO:teuthology.orchestra.run.smithi042.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-08-30T08:48:53.483 INFO:teuthology.orchestra.run.smithi042.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-08-30T08:48:53.483 INFO:teuthology.orchestra.run.smithi042.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-08-30T08:48:53.484 INFO:teuthology.orchestra.run.smithi042.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-08-30T08:48:53.484 INFO:teuthology.orchestra.run.smithi042.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-08-30T08:48:53.484 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-08-30T08:48:53.484 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-08-30T08:48:53.484 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-08-30T08:48:53.484 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-08-30T08:48:53.484 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-08-30T08:48:53.484 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-08-30T08:48:53.484 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-08-30T08:48:53.484 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-08-30T08:48:53.484 INFO:teuthology.orchestra.run.smithi042.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-08-30T08:48:53.484 INFO:teuthology.orchestra.run.smithi042.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-08-30T08:48:53.484 INFO:teuthology.orchestra.run.smithi042.stdout: python3-mako-1.1.4-6.el9.noarch 2024-08-30T08:48:53.485 INFO:teuthology.orchestra.run.smithi042.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-08-30T08:48:53.485 INFO:teuthology.orchestra.run.smithi042.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-08-30T08:48:53.485 INFO:teuthology.orchestra.run.smithi042.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-08-30T08:48:53.485 INFO:teuthology.orchestra.run.smithi042.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-08-30T08:48:53.485 INFO:teuthology.orchestra.run.smithi042.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-08-30T08:48:53.485 INFO:teuthology.orchestra.run.smithi042.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-08-30T08:48:53.485 INFO:teuthology.orchestra.run.smithi042.stdout: python3-packaging-20.9-5.el9.noarch 2024-08-30T08:48:53.485 INFO:teuthology.orchestra.run.smithi042.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-08-30T08:48:53.485 INFO:teuthology.orchestra.run.smithi042.stdout: python3-ply-3.11-14.el9.noarch 2024-08-30T08:48:53.485 INFO:teuthology.orchestra.run.smithi042.stdout: python3-portend-3.1.0-2.el9.noarch 2024-08-30T08:48:53.485 INFO:teuthology.orchestra.run.smithi042.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-08-30T08:48:53.485 INFO:teuthology.orchestra.run.smithi042.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-08-30T08:48:53.485 INFO:teuthology.orchestra.run.smithi042.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-08-30T08:48:53.486 INFO:teuthology.orchestra.run.smithi042.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-08-30T08:48:53.486 INFO:teuthology.orchestra.run.smithi042.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-08-30T08:48:53.486 INFO:teuthology.orchestra.run.smithi042.stdout: python3-pycparser-2.20-6.el9.noarch 2024-08-30T08:48:53.486 INFO:teuthology.orchestra.run.smithi042.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-08-30T08:48:53.486 INFO:teuthology.orchestra.run.smithi042.stdout: python3-pytz-2021.1-5.el9.noarch 2024-08-30T08:48:53.486 INFO:teuthology.orchestra.run.smithi042.stdout: python3-rados-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-30T08:48:53.486 INFO:teuthology.orchestra.run.smithi042.stdout: python3-rbd-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-30T08:48:53.486 INFO:teuthology.orchestra.run.smithi042.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-08-30T08:48:53.486 INFO:teuthology.orchestra.run.smithi042.stdout: python3-requests-2.25.1-8.el9.noarch 2024-08-30T08:48:53.486 INFO:teuthology.orchestra.run.smithi042.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-08-30T08:48:53.486 INFO:teuthology.orchestra.run.smithi042.stdout: python3-rgw-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-30T08:48:53.486 INFO:teuthology.orchestra.run.smithi042.stdout: python3-routes-2.5.1-5.el9.noarch 2024-08-30T08:48:53.486 INFO:teuthology.orchestra.run.smithi042.stdout: python3-rsa-4.9-2.el9.noarch 2024-08-30T08:48:53.487 INFO:teuthology.orchestra.run.smithi042.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-08-30T08:48:53.487 INFO:teuthology.orchestra.run.smithi042.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-08-30T08:48:53.487 INFO:teuthology.orchestra.run.smithi042.stdout: python3-toml-0.10.2-6.el9.noarch 2024-08-30T08:48:53.487 INFO:teuthology.orchestra.run.smithi042.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-08-30T08:48:53.487 INFO:teuthology.orchestra.run.smithi042.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-08-30T08:48:53.487 INFO:teuthology.orchestra.run.smithi042.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-08-30T08:48:53.487 INFO:teuthology.orchestra.run.smithi042.stdout: python3-webob-1.8.8-2.el9.noarch 2024-08-30T08:48:53.487 INFO:teuthology.orchestra.run.smithi042.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-08-30T08:48:53.487 INFO:teuthology.orchestra.run.smithi042.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-08-30T08:48:53.487 INFO:teuthology.orchestra.run.smithi042.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-08-30T08:48:53.487 INFO:teuthology.orchestra.run.smithi042.stdout: qatlib-24.02.0-1.el9.x86_64 2024-08-30T08:48:53.487 INFO:teuthology.orchestra.run.smithi042.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-08-30T08:48:53.487 INFO:teuthology.orchestra.run.smithi042.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-08-30T08:48:53.488 INFO:teuthology.orchestra.run.smithi042.stdout: rbd-fuse-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-30T08:48:53.488 INFO:teuthology.orchestra.run.smithi042.stdout: rbd-mirror-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-30T08:48:53.488 INFO:teuthology.orchestra.run.smithi042.stdout: rbd-nbd-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-30T08:48:53.488 INFO:teuthology.orchestra.run.smithi042.stdout: re2-1:20211101-20.el9.x86_64 2024-08-30T08:48:53.488 INFO:teuthology.orchestra.run.smithi042.stdout: socat-1.7.4.1-6.el9.x86_64 2024-08-30T08:48:53.488 INFO:teuthology.orchestra.run.smithi042.stdout: thrift-0.15.0-2.el9.x86_64 2024-08-30T08:48:53.488 INFO:teuthology.orchestra.run.smithi042.stdout: uuid-1.6.2-55.el9.x86_64 2024-08-30T08:48:53.488 INFO:teuthology.orchestra.run.smithi042.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-08-30T08:48:53.488 INFO:teuthology.orchestra.run.smithi042.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-08-30T08:48:53.488 INFO:teuthology.orchestra.run.smithi042.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-08-30T08:48:53.489 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-08-30T08:48:53.489 INFO:teuthology.orchestra.run.smithi042.stdout:Complete! 2024-08-30T08:48:53.820 DEBUG:teuthology.parallel:result is None 2024-08-30T08:48:53.821 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d19e6118f7015f0047264006cb0c873528eb3ae6 2024-08-30T08:48:53.977 DEBUG:teuthology.orchestra.run.smithi042:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-08-30T08:48:53.999 INFO:teuthology.orchestra.run.smithi042.stdout:19.3.0-4560.gd19e6118.el9 2024-08-30T08:48:54.000 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-4560.gd19e6118.el9 2024-08-30T08:48:54.000 INFO:teuthology.task.install:The correct ceph version 19.3.0-4560.gd19e6118 is installed. 2024-08-30T08:48:54.001 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d19e6118f7015f0047264006cb0c873528eb3ae6 2024-08-30T08:48:54.149 DEBUG:teuthology.orchestra.run.smithi167:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-08-30T08:48:54.176 INFO:teuthology.orchestra.run.smithi167.stdout:19.3.0-4560.gd19e6118.el9 2024-08-30T08:48:54.176 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-4560.gd19e6118.el9 2024-08-30T08:48:54.176 INFO:teuthology.task.install:The correct ceph version 19.3.0-4560.gd19e6118 is installed. 2024-08-30T08:48:54.178 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d19e6118f7015f0047264006cb0c873528eb3ae6 2024-08-30T08:48:54.330 DEBUG:teuthology.orchestra.run.smithi190:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-08-30T08:48:54.357 INFO:teuthology.orchestra.run.smithi190.stdout:19.3.0-4560.gd19e6118.el9 2024-08-30T08:48:54.357 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-4560.gd19e6118.el9 2024-08-30T08:48:54.357 INFO:teuthology.task.install:The correct ceph version 19.3.0-4560.gd19e6118 is installed. 2024-08-30T08:48:54.359 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-08-30T08:48:54.359 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-08-30T08:48:54.360 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-08-30T08:48:54.386 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-08-30T08:48:54.387 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-08-30T08:48:54.420 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-08-30T08:48:54.420 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-08-30T08:48:54.452 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-08-30T08:48:54.453 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-08-30T08:48:54.453 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd of=/usr/bin/daemon-helper 2024-08-30T08:48:54.477 DEBUG:teuthology.orchestra.run.smithi042:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-08-30T08:48:54.541 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-08-30T08:48:54.541 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd of=/usr/bin/daemon-helper 2024-08-30T08:48:54.570 DEBUG:teuthology.orchestra.run.smithi167:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-08-30T08:48:54.638 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-08-30T08:48:54.639 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd of=/usr/bin/daemon-helper 2024-08-30T08:48:54.668 DEBUG:teuthology.orchestra.run.smithi190:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-08-30T08:48:54.739 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-08-30T08:48:54.739 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-08-30T08:48:54.739 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd of=/usr/bin/adjust-ulimits 2024-08-30T08:48:54.765 DEBUG:teuthology.orchestra.run.smithi042:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-08-30T08:48:54.831 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-08-30T08:48:54.831 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd of=/usr/bin/adjust-ulimits 2024-08-30T08:48:54.862 DEBUG:teuthology.orchestra.run.smithi167:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-08-30T08:48:54.928 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-08-30T08:48:54.928 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd of=/usr/bin/adjust-ulimits 2024-08-30T08:48:54.954 DEBUG:teuthology.orchestra.run.smithi190:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-08-30T08:48:55.020 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-08-30T08:48:55.020 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-08-30T08:48:55.020 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd of=/usr/bin/stdin-killer 2024-08-30T08:48:55.045 DEBUG:teuthology.orchestra.run.smithi042:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-08-30T08:48:55.110 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-08-30T08:48:55.110 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd of=/usr/bin/stdin-killer 2024-08-30T08:48:55.135 DEBUG:teuthology.orchestra.run.smithi167:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-08-30T08:48:55.200 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-08-30T08:48:55.200 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd of=/usr/bin/stdin-killer 2024-08-30T08:48:55.229 DEBUG:teuthology.orchestra.run.smithi190:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-08-30T08:48:55.294 INFO:teuthology.run_tasks:Running task cephadm... 2024-08-30T08:48:55.396 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 3, 'session_timeout': 300, 'standby_replay': True}, 'cluster-conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 2, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon': {'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'osd op complaint time': 180}}, 'conf': {'global': {'ms die on skipped message': False}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'fs.*is degraded', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'insufficient standby MDS daemons available', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': 'd19e6118f7015f0047264006cb0c873528eb3ae6', 'subvols': {'create': 2}} 2024-08-30T08:48:55.396 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 2024-08-30T08:48:55.397 INFO:tasks.cephadm:Cluster fsid is b0895c1a-66ac-11ef-bcd4-c7b262605968 2024-08-30T08:48:55.397 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-08-30T08:48:55.397 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.42', 'mon.b': '172.21.15.167', 'mon.c': '172.21.15.190'} 2024-08-30T08:48:55.398 INFO:tasks.cephadm:First mon is mon.a on smithi042 2024-08-30T08:48:55.398 INFO:tasks.cephadm:First mgr is x 2024-08-30T08:48:55.398 INFO:tasks.cephadm:Normalizing hostnames... 2024-08-30T08:48:55.398 DEBUG:teuthology.orchestra.run.smithi042:> sudo hostname $(hostname -s) 2024-08-30T08:48:55.424 DEBUG:teuthology.orchestra.run.smithi167:> sudo hostname $(hostname -s) 2024-08-30T08:48:55.449 DEBUG:teuthology.orchestra.run.smithi190:> sudo hostname $(hostname -s) 2024-08-30T08:48:55.474 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-08-30T08:48:55.475 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d19e6118f7015f0047264006cb0c873528eb3ae6 2024-08-30T08:48:55.635 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': 'd19e6118f7015f0047264006cb0c873528eb3ae6', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/82586/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.3.0-4560-gd19e6118', 'node_name': '172.21.5.31+adami01', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.3.0-4560.gd19e6118'}, 'url': 'https://4.chacra.ceph.com/r/ceph/wip-mchangir-testing-20240828.085847-main-debug/d19e6118f7015f0047264006cb0c873528eb3ae6/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-08-28 11:15:32.801765', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-mchangir-testing-20240828.085847-main-debug', 'chacra_url': 'https://4.chacra.ceph.com/repos/ceph/wip-mchangir-testing-20240828.085847-main-debug/d19e6118f7015f0047264006cb0c873528eb3ae6/centos/9/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}] 2024-08-30T08:48:55.785 INFO:tasks.util.chacra:got chacra host 4.chacra.ceph.com, ref wip-mchangir-testing-20240828.085847-main-debug, sha1 d19e6118f7015f0047264006cb0c873528eb3ae6 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=d19e6118f7015f0047264006cb0c873528eb3ae6 2024-08-30T08:48:55.787 INFO:tasks.cephadm:Discovered cachra url: https://4.chacra.ceph.com/binaries/ceph/wip-mchangir-testing-20240828.085847-main-debug/d19e6118f7015f0047264006cb0c873528eb3ae6/centos/9/x86_64/flavors/default/cephadm 2024-08-30T08:48:55.787 INFO:tasks.cephadm:Downloading cephadm from url: https://4.chacra.ceph.com/binaries/ceph/wip-mchangir-testing-20240828.085847-main-debug/d19e6118f7015f0047264006cb0c873528eb3ae6/centos/9/x86_64/flavors/default/cephadm 2024-08-30T08:48:55.787 DEBUG:teuthology.orchestra.run.smithi042:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-mchangir-testing-20240828.085847-main-debug/d19e6118f7015f0047264006cb0c873528eb3ae6/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-08-30T08:48:56.389 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r--. 1 ubuntu ubuntu 801010 Aug 30 08:48 /home/ubuntu/cephtest/cephadm 2024-08-30T08:48:56.390 DEBUG:teuthology.orchestra.run.smithi167:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-mchangir-testing-20240828.085847-main-debug/d19e6118f7015f0047264006cb0c873528eb3ae6/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-08-30T08:48:56.966 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-r--r--. 1 ubuntu ubuntu 801010 Aug 30 08:48 /home/ubuntu/cephtest/cephadm 2024-08-30T08:48:56.966 DEBUG:teuthology.orchestra.run.smithi190:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-mchangir-testing-20240828.085847-main-debug/d19e6118f7015f0047264006cb0c873528eb3ae6/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-08-30T08:48:57.566 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r--. 1 ubuntu ubuntu 801010 Aug 30 08:48 /home/ubuntu/cephtest/cephadm 2024-08-30T08:48:57.567 DEBUG:teuthology.orchestra.run.smithi042:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-08-30T08:48:57.585 DEBUG:teuthology.orchestra.run.smithi167:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-08-30T08:48:57.605 DEBUG:teuthology.orchestra.run.smithi190:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-08-30T08:48:57.639 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 on all hosts... 2024-08-30T08:48:57.639 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 pull 2024-08-30T08:48:57.641 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 pull 2024-08-30T08:48:57.648 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 pull 2024-08-30T08:48:59.241 INFO:teuthology.orchestra.run.smithi167.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6... 2024-08-30T08:48:59.242 INFO:teuthology.orchestra.run.smithi190.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6... 2024-08-30T08:48:59.358 INFO:teuthology.orchestra.run.smithi042.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6... 2024-08-30T08:49:28.740 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-08-30T08:49:28.740 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph_version": "ceph version 19.3.0-4560-gd19e6118 (d19e6118f7015f0047264006cb0c873528eb3ae6) squid (dev)", 2024-08-30T08:49:28.740 INFO:teuthology.orchestra.run.smithi042.stdout: "image_id": "79f4cde5e36911a5dca995eeb98098c2c8aae81dd52a088ef2c867bc3363bad0", 2024-08-30T08:49:28.740 INFO:teuthology.orchestra.run.smithi042.stdout: "repo_digests": [ 2024-08-30T08:49:28.740 INFO:teuthology.orchestra.run.smithi042.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61" 2024-08-30T08:49:28.740 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-08-30T08:49:28.741 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-08-30T08:49:31.924 INFO:teuthology.orchestra.run.smithi167.stdout:{ 2024-08-30T08:49:31.924 INFO:teuthology.orchestra.run.smithi167.stdout: "ceph_version": "ceph version 19.3.0-4560-gd19e6118 (d19e6118f7015f0047264006cb0c873528eb3ae6) squid (dev)", 2024-08-30T08:49:31.925 INFO:teuthology.orchestra.run.smithi167.stdout: "image_id": "79f4cde5e36911a5dca995eeb98098c2c8aae81dd52a088ef2c867bc3363bad0", 2024-08-30T08:49:31.925 INFO:teuthology.orchestra.run.smithi167.stdout: "repo_digests": [ 2024-08-30T08:49:31.925 INFO:teuthology.orchestra.run.smithi167.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61" 2024-08-30T08:49:31.925 INFO:teuthology.orchestra.run.smithi167.stdout: ] 2024-08-30T08:49:31.925 INFO:teuthology.orchestra.run.smithi167.stdout:} 2024-08-30T08:49:35.622 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-08-30T08:49:35.622 INFO:teuthology.orchestra.run.smithi190.stdout: "ceph_version": "ceph version 19.3.0-4560-gd19e6118 (d19e6118f7015f0047264006cb0c873528eb3ae6) squid (dev)", 2024-08-30T08:49:35.622 INFO:teuthology.orchestra.run.smithi190.stdout: "image_id": "79f4cde5e36911a5dca995eeb98098c2c8aae81dd52a088ef2c867bc3363bad0", 2024-08-30T08:49:35.622 INFO:teuthology.orchestra.run.smithi190.stdout: "repo_digests": [ 2024-08-30T08:49:35.623 INFO:teuthology.orchestra.run.smithi190.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61" 2024-08-30T08:49:35.623 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-08-30T08:49:35.623 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-08-30T08:49:35.648 DEBUG:teuthology.orchestra.run.smithi042:> sudo mkdir -p /etc/ceph 2024-08-30T08:49:35.682 DEBUG:teuthology.orchestra.run.smithi167:> sudo mkdir -p /etc/ceph 2024-08-30T08:49:35.715 DEBUG:teuthology.orchestra.run.smithi190:> sudo mkdir -p /etc/ceph 2024-08-30T08:49:35.742 DEBUG:teuthology.orchestra.run.smithi042:> sudo chmod 777 /etc/ceph 2024-08-30T08:49:35.771 DEBUG:teuthology.orchestra.run.smithi167:> sudo chmod 777 /etc/ceph 2024-08-30T08:49:35.800 DEBUG:teuthology.orchestra.run.smithi190:> sudo chmod 777 /etc/ceph 2024-08-30T08:49:35.824 INFO:tasks.cephadm:Writing seed config... 2024-08-30T08:49:35.825 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-08-30T08:49:35.826 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-08-30T08:49:35.826 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-08-30T08:49:35.826 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-08-30T08:49:35.826 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-08-30T08:49:35.826 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-08-30T08:49:35.826 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-08-30T08:49:35.826 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-08-30T08:49:35.826 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-08-30T08:49:35.826 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-08-30T08:49:35.826 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-08-30T08:49:35.826 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-08-30T08:49:35.826 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-08-30T08:49:35.826 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-08-30T08:49:35.826 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-08-30T08:49:35.826 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-08-30T08:49:35.827 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-08-30T08:49:35.827 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-08-30T08:49:35.827 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-08-30T08:49:35.827 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-08-30T08:49:35.827 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-08-30T08:49:35.827 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-08-30T08:49:35.827 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2024-08-30T08:49:35.828 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-08-30T08:49:35.828 DEBUG:teuthology.orchestra.run.smithi042:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-08-30T08:49:35.846 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = b0895c1a-66ac-11ef-bcd4-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true 2024-08-30T08:49:35.846 DEBUG:teuthology.orchestra.run.smithi042:mon.a> sudo journalctl -f -n 0 -u ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968@mon.a.service 2024-08-30T08:49:35.889 DEBUG:teuthology.orchestra.run.smithi042:mgr.x> sudo journalctl -f -n 0 -u ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968@mgr.x.service 2024-08-30T08:49:35.931 INFO:tasks.cephadm:Bootstrapping... 2024-08-30T08:49:35.931 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 -v bootstrap --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.42 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-08-30T08:49:36.146 INFO:teuthology.orchestra.run.smithi042.stdout:-------------------------------------------------------------------------------- 2024-08-30T08:49:36.146 INFO:teuthology.orchestra.run.smithi042.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6', '-v', 'bootstrap', '--fsid', 'b0895c1a-66ac-11ef-bcd4-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.42', '--skip-admin-label'] 2024-08-30T08:49:36.146 INFO:teuthology.orchestra.run.smithi042.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-08-30T08:49:36.146 INFO:teuthology.orchestra.run.smithi042.stdout:Verifying podman|docker is present... 2024-08-30T08:49:36.189 INFO:teuthology.orchestra.run.smithi042.stdout:/bin/podman: stdout 5.2.2 2024-08-30T08:49:36.189 INFO:teuthology.orchestra.run.smithi042.stdout:Verifying lvm2 is present... 2024-08-30T08:49:36.189 INFO:teuthology.orchestra.run.smithi042.stdout:Verifying time synchronization is in place... 2024-08-30T08:49:36.197 INFO:teuthology.orchestra.run.smithi042.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-08-30T08:49:36.197 INFO:teuthology.orchestra.run.smithi042.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-08-30T08:49:36.203 INFO:teuthology.orchestra.run.smithi042.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-08-30T08:49:36.203 INFO:teuthology.orchestra.run.smithi042.stdout:systemctl: stdout inactive 2024-08-30T08:49:36.210 INFO:teuthology.orchestra.run.smithi042.stdout:systemctl: stdout enabled 2024-08-30T08:49:36.216 INFO:teuthology.orchestra.run.smithi042.stdout:systemctl: stdout active 2024-08-30T08:49:36.216 INFO:teuthology.orchestra.run.smithi042.stdout:Unit chronyd.service is enabled and running 2024-08-30T08:49:36.216 INFO:teuthology.orchestra.run.smithi042.stdout:Repeating the final host check... 2024-08-30T08:49:36.239 INFO:teuthology.orchestra.run.smithi042.stdout:/bin/podman: stdout 5.2.2 2024-08-30T08:49:36.239 INFO:teuthology.orchestra.run.smithi042.stdout:podman (/bin/podman) version 5.2.2 is present 2024-08-30T08:49:36.240 INFO:teuthology.orchestra.run.smithi042.stdout:systemctl is present 2024-08-30T08:49:36.240 INFO:teuthology.orchestra.run.smithi042.stdout:lvcreate is present 2024-08-30T08:49:36.246 INFO:teuthology.orchestra.run.smithi042.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-08-30T08:49:36.246 INFO:teuthology.orchestra.run.smithi042.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-08-30T08:49:36.252 INFO:teuthology.orchestra.run.smithi042.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-08-30T08:49:36.252 INFO:teuthology.orchestra.run.smithi042.stdout:systemctl: stdout inactive 2024-08-30T08:49:36.258 INFO:teuthology.orchestra.run.smithi042.stdout:systemctl: stdout enabled 2024-08-30T08:49:36.265 INFO:teuthology.orchestra.run.smithi042.stdout:systemctl: stdout active 2024-08-30T08:49:36.265 INFO:teuthology.orchestra.run.smithi042.stdout:Unit chronyd.service is enabled and running 2024-08-30T08:49:36.265 INFO:teuthology.orchestra.run.smithi042.stdout:Host looks OK 2024-08-30T08:49:36.265 INFO:teuthology.orchestra.run.smithi042.stdout:Cluster fsid: b0895c1a-66ac-11ef-bcd4-c7b262605968 2024-08-30T08:49:36.265 INFO:teuthology.orchestra.run.smithi042.stdout:Acquiring lock 140549607576624 on /run/cephadm/b0895c1a-66ac-11ef-bcd4-c7b262605968.lock 2024-08-30T08:49:36.265 INFO:teuthology.orchestra.run.smithi042.stdout:Lock 140549607576624 acquired on /run/cephadm/b0895c1a-66ac-11ef-bcd4-c7b262605968.lock 2024-08-30T08:49:36.266 INFO:teuthology.orchestra.run.smithi042.stdout:Verifying IP 172.21.15.42 port 3300 ... 2024-08-30T08:49:36.266 INFO:teuthology.orchestra.run.smithi042.stdout:Verifying IP 172.21.15.42 port 6789 ... 2024-08-30T08:49:36.266 INFO:teuthology.orchestra.run.smithi042.stdout:Base mon IP(s) is [172.21.15.42:3300, 172.21.15.42:6789], mon addrv is [v2:172.21.15.42:3300,v1:172.21.15.42:6789] 2024-08-30T08:49:36.269 INFO:teuthology.orchestra.run.smithi042.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.42 metric 100 2024-08-30T08:49:36.269 INFO:teuthology.orchestra.run.smithi042.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.42 metric 100 2024-08-30T08:49:36.271 INFO:teuthology.orchestra.run.smithi042.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-08-30T08:49:36.272 INFO:teuthology.orchestra.run.smithi042.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 1024 pref medium 2024-08-30T08:49:36.272 INFO:teuthology.orchestra.run.smithi042.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 100 pref medium 2024-08-30T08:49:36.274 INFO:teuthology.orchestra.run.smithi042.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-08-30T08:49:36.274 INFO:teuthology.orchestra.run.smithi042.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-08-30T08:49:36.274 INFO:teuthology.orchestra.run.smithi042.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-08-30T08:49:36.274 INFO:teuthology.orchestra.run.smithi042.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-08-30T08:49:36.274 INFO:teuthology.orchestra.run.smithi042.stdout:/sbin/ip: stdout inet6 fe80::666b:514f:12c9:e93f/64 scope link noprefixroute 2024-08-30T08:49:36.274 INFO:teuthology.orchestra.run.smithi042.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-08-30T08:49:36.274 INFO:teuthology.orchestra.run.smithi042.stdout:Mon IP `172.21.15.42` is in CIDR network `172.21.0.0/20` 2024-08-30T08:49:36.278 INFO:teuthology.orchestra.run.smithi042.stdout:Mon IP `172.21.15.42` is in CIDR network `172.21.0.0/20` 2024-08-30T08:49:36.278 INFO:teuthology.orchestra.run.smithi042.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-08-30T08:49:36.278 INFO:teuthology.orchestra.run.smithi042.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-08-30T08:49:36.278 INFO:teuthology.orchestra.run.smithi042.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6... 2024-08-30T08:49:36.784 INFO:teuthology.orchestra.run.smithi042.stdout:/bin/podman: stdout 79f4cde5e36911a5dca995eeb98098c2c8aae81dd52a088ef2c867bc3363bad0 2024-08-30T08:49:36.784 INFO:teuthology.orchestra.run.smithi042.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6... 2024-08-30T08:49:36.784 INFO:teuthology.orchestra.run.smithi042.stdout:/bin/podman: stderr Getting image source signatures 2024-08-30T08:49:36.784 INFO:teuthology.orchestra.run.smithi042.stdout:/bin/podman: stderr Copying blob sha256:1c7f48e91f1eb9e77df623f13bab8a42aaf12d95954655c07fb62a1a85ca19b2 2024-08-30T08:49:36.784 INFO:teuthology.orchestra.run.smithi042.stdout:/bin/podman: stderr Copying blob sha256:e8b54c863393b7a8216a71737771b73b16a8e43ec7acf927c7faa175c255e551 2024-08-30T08:49:36.784 INFO:teuthology.orchestra.run.smithi042.stdout:/bin/podman: stderr Copying config sha256:79f4cde5e36911a5dca995eeb98098c2c8aae81dd52a088ef2c867bc3363bad0 2024-08-30T08:49:36.785 INFO:teuthology.orchestra.run.smithi042.stdout:/bin/podman: stderr Writing manifest to image destination 2024-08-30T08:49:37.597 INFO:teuthology.orchestra.run.smithi042.stdout:ceph: stdout ceph version 19.3.0-4560-gd19e6118 (d19e6118f7015f0047264006cb0c873528eb3ae6) squid (dev) 2024-08-30T08:49:37.597 INFO:teuthology.orchestra.run.smithi042.stdout:Ceph version: ceph version 19.3.0-4560-gd19e6118 (d19e6118f7015f0047264006cb0c873528eb3ae6) squid (dev) 2024-08-30T08:49:37.597 INFO:teuthology.orchestra.run.smithi042.stdout:Extracting ceph user uid/gid from container image... 2024-08-30T08:49:38.628 INFO:teuthology.orchestra.run.smithi042.stdout:stat: stdout 167 167 2024-08-30T08:49:38.628 INFO:teuthology.orchestra.run.smithi042.stdout:Creating initial keys... 2024-08-30T08:49:39.558 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph-authtool: stdout AQCih9FmeS34NxAAd7JTCMYEE8va2I1QAR2ojQ== 2024-08-30T08:49:40.422 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph-authtool: stdout AQCjh9Fm7/44NBAAUQozNO+L7z0AUwUF/CF3YA== 2024-08-30T08:49:41.347 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph-authtool: stdout AQCkh9Fmi6fMLRAAPxS6ADWu2yVZj+vlHN0Q8Q== 2024-08-30T08:49:41.348 INFO:teuthology.orchestra.run.smithi042.stdout:Creating initial monmap... 2024-08-30T08:49:42.326 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-08-30T08:49:42.326 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-08-30T08:49:42.326 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to b0895c1a-66ac-11ef-bcd4-c7b262605968 2024-08-30T08:49:42.326 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-08-30T08:49:42.327 INFO:teuthology.orchestra.run.smithi042.stdout:monmaptool for a [v2:172.21.15.42:3300,v1:172.21.15.42:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-08-30T08:49:42.327 INFO:teuthology.orchestra.run.smithi042.stdout:setting min_mon_release = quincy 2024-08-30T08:49:42.327 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/monmaptool: set fsid to b0895c1a-66ac-11ef-bcd4-c7b262605968 2024-08-30T08:49:42.327 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-08-30T08:49:42.327 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-08-30T08:49:42.327 INFO:teuthology.orchestra.run.smithi042.stdout:Creating mon... 2024-08-30T08:49:43.257 INFO:teuthology.orchestra.run.smithi042.stdout:create mon.a on 2024-08-30T08:49:43.590 INFO:teuthology.orchestra.run.smithi042.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-08-30T08:49:43.754 INFO:teuthology.orchestra.run.smithi042.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-08-30T08:49:43.956 INFO:teuthology.orchestra.run.smithi042.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968.target → /etc/systemd/system/ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968.target. 2024-08-30T08:49:43.956 INFO:teuthology.orchestra.run.smithi042.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968.target → /etc/systemd/system/ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968.target. 2024-08-30T08:49:44.200 INFO:teuthology.orchestra.run.smithi042.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968@mon.a 2024-08-30T08:49:44.201 INFO:teuthology.orchestra.run.smithi042.stdout:systemctl: stderr Failed to reset failed state of unit ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968@mon.a.service: Unit ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968@mon.a.service not loaded. 2024-08-30T08:49:44.373 INFO:teuthology.orchestra.run.smithi042.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968.target.wants/ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968@mon.a.service → /etc/systemd/system/ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968@.service. 2024-08-30T08:49:44.906 INFO:teuthology.orchestra.run.smithi042.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-08-30T08:49:44.907 INFO:teuthology.orchestra.run.smithi042.stdout:systemctl: stdout disabled 2024-08-30T08:49:44.913 INFO:teuthology.orchestra.run.smithi042.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-08-30T08:49:44.913 INFO:teuthology.orchestra.run.smithi042.stdout:systemctl: stdout inactive 2024-08-30T08:49:44.914 INFO:teuthology.orchestra.run.smithi042.stdout:firewalld.service is not enabled 2024-08-30T08:49:44.914 INFO:teuthology.orchestra.run.smithi042.stdout:Not possible to enable service . firewalld.service is not available 2024-08-30T08:49:44.914 INFO:teuthology.orchestra.run.smithi042.stdout:Waiting for mon to start... 2024-08-30T08:49:44.914 INFO:teuthology.orchestra.run.smithi042.stdout:Waiting for mon... 2024-08-30T08:49:44.984 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:44 smithi042 systemd[1]: Started Ceph mon.a for b0895c1a-66ac-11ef-bcd4-c7b262605968. 2024-08-30T08:49:46.033 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:45 smithi042 ceph-mon[18284]: mon.a is new leader, mons a in quorum (ranks 0) 2024-08-30T08:49:46.033 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:45 smithi042 ceph-mon[18284]: monmap epoch 1 2024-08-30T08:49:46.033 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:45 smithi042 ceph-mon[18284]: fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 2024-08-30T08:49:46.033 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:45 smithi042 ceph-mon[18284]: last_changed 2024-08-30T08:49:41.668361+0000 2024-08-30T08:49:46.033 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:45 smithi042 ceph-mon[18284]: created 2024-08-30T08:49:41.668361+0000 2024-08-30T08:49:46.033 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:45 smithi042 ceph-mon[18284]: min_mon_release 19 (squid) 2024-08-30T08:49:46.033 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:45 smithi042 ceph-mon[18284]: election_strategy: 1 2024-08-30T08:49:46.033 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:45 smithi042 ceph-mon[18284]: 0: [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] mon.a 2024-08-30T08:49:46.033 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:45 smithi042 ceph-mon[18284]: fsmap 2024-08-30T08:49:46.033 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:45 smithi042 ceph-mon[18284]: osdmap e1: 0 total, 0 up, 0 in 2024-08-30T08:49:46.034 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:45 smithi042 ceph-mon[18284]: mgrmap e1: no daemons active 2024-08-30T08:49:46.034 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:45 smithi042 ceph-mon[18284]: from='client.? 172.21.15.42:0/2661685429' entity='client.admin' cmd={"prefix": "status"} : dispatch 2024-08-30T08:49:46.035 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout cluster: 2024-08-30T08:49:46.037 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout id: b0895c1a-66ac-11ef-bcd4-c7b262605968 2024-08-30T08:49:46.037 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-08-30T08:49:46.037 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout 2024-08-30T08:49:46.037 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout services: 2024-08-30T08:49:46.037 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.59902s) 2024-08-30T08:49:46.037 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-08-30T08:49:46.037 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-08-30T08:49:46.037 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout 2024-08-30T08:49:46.037 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout data: 2024-08-30T08:49:46.038 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-08-30T08:49:46.038 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-08-30T08:49:46.038 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-08-30T08:49:46.038 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout pgs: 2024-08-30T08:49:46.038 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout 2024-08-30T08:49:46.038 INFO:teuthology.orchestra.run.smithi042.stdout:mon is available 2024-08-30T08:49:46.038 INFO:teuthology.orchestra.run.smithi042.stdout:Assimilating anything we can from ceph.conf... 2024-08-30T08:49:47.060 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout 2024-08-30T08:49:47.060 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout [global] 2024-08-30T08:49:47.060 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout fsid = b0895c1a-66ac-11ef-bcd4-c7b262605968 2024-08-30T08:49:47.060 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-08-30T08:49:47.060 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.42:3300,v1:172.21.15.42:6789] 2024-08-30T08:49:47.060 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-08-30T08:49:47.060 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-08-30T08:49:47.060 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-08-30T08:49:47.060 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-08-30T08:49:47.061 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout 2024-08-30T08:49:47.061 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout [mgr] 2024-08-30T08:49:47.061 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-08-30T08:49:47.061 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout 2024-08-30T08:49:47.061 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout [osd] 2024-08-30T08:49:47.061 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-08-30T08:49:47.061 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-08-30T08:49:47.061 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-08-30T08:49:47.061 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-08-30T08:49:47.061 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-08-30T08:49:47.061 INFO:teuthology.orchestra.run.smithi042.stdout:Generating new minimal ceph.conf... 2024-08-30T08:49:47.132 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:46 smithi042 ceph-mon[18284]: from='client.? 172.21.15.42:0/319823928' entity='client.admin' cmd={"prefix": "config assimilate-conf"} : dispatch 2024-08-30T08:49:47.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:46 smithi042 ceph-mon[18284]: from='client.? 172.21.15.42:0/319823928' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-08-30T08:49:48.086 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:47 smithi042 ceph-mon[18284]: from='client.? 172.21.15.42:0/754269708' entity='client.admin' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:49:48.089 INFO:teuthology.orchestra.run.smithi042.stdout:Restarting the monitor... 2024-08-30T08:49:48.365 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:48 smithi042 systemd[1]: Stopping Ceph mon.a for b0895c1a-66ac-11ef-bcd4-c7b262605968... 2024-08-30T08:49:48.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:48 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mon-a[18280]: 2024-08-30T08:49:48.362+0000 7f6676d04640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-08-30T08:49:48.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:48 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mon-a[18280]: 2024-08-30T08:49:48.362+0000 7f6676d04640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-08-30T08:49:48.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:48 smithi042 podman[18526]: 2024-08-30 08:49:48.547126062 +0000 UTC m=+0.296371362 container died eb28b996166aa1486c0d41c5d343664980ed4efe4bb8197a6617f4d5eaddd6c9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6, name=ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mon-a, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.37.1, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux ) 2024-08-30T08:49:49.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:48 smithi042 podman[18526]: 2024-08-30 08:49:48.846483061 +0000 UTC m=+0.595728360 container cleanup eb28b996166aa1486c0d41c5d343664980ed4efe4bb8197a6617f4d5eaddd6c9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6, name=ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.1, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True) 2024-08-30T08:49:49.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:48 smithi042 bash[18526]: ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mon-a 2024-08-30T08:49:51.611 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:51 smithi042 podman[18538]: 2024-08-30 08:49:51.165912942 +0000 UTC m=+2.613127844 container remove eb28b996166aa1486c0d41c5d343664980ed4efe4bb8197a6617f4d5eaddd6c9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6, name=ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mon-a, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, io.buildah.version=1.37.1, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-08-30T08:49:51.611 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:51 smithi042 systemd[1]: ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968@mon.a.service: Deactivated successfully. 2024-08-30T08:49:51.611 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:51 smithi042 systemd[1]: Stopped Ceph mon.a for b0895c1a-66ac-11ef-bcd4-c7b262605968. 2024-08-30T08:49:51.611 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:51 smithi042 systemd[1]: Starting Ceph mon.a for b0895c1a-66ac-11ef-bcd4-c7b262605968... 2024-08-30T08:49:51.882 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:51 smithi042 podman[18633]: 2024-08-30 08:49:51.51240864 +0000 UTC m=+0.122687338 image pull 79f4cde5e36911a5dca995eeb98098c2c8aae81dd52a088ef2c867bc3363bad0 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 2024-08-30T08:49:53.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:52 smithi042 podman[18633]: 2024-08-30 08:49:52.662317405 +0000 UTC m=+1.272596131 container create e677a01ba2f722b1e6e32fc4353612a39c416728efa70ee5abac8539d253b076 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6, name=ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, ceph=True, io.buildah.version=1.37.1, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-08-30T08:49:54.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 podman[18633]: 2024-08-30 08:49:54.439811936 +0000 UTC m=+3.050090637 container init e677a01ba2f722b1e6e32fc4353612a39c416728efa70ee5abac8539d253b076 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6, name=ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mon-a, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.37.1, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, ceph=True) 2024-08-30T08:49:54.884 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 podman[18633]: 2024-08-30 08:49:54.444081989 +0000 UTC m=+3.054360687 container start e677a01ba2f722b1e6e32fc4353612a39c416728efa70ee5abac8539d253b076 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6, name=ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mon-a, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, GIT_CLEAN=True, io.buildah.version=1.37.1) 2024-08-30T08:49:54.884 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: set uid:gid to 167:167 (ceph:ceph) 2024-08-30T08:49:54.884 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: ceph version 19.3.0-4560-gd19e6118 (d19e6118f7015f0047264006cb0c873528eb3ae6) squid (dev), process ceph-mon, pid 2 2024-08-30T08:49:54.884 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: pidfile_write: ignore empty --pid-file 2024-08-30T08:49:54.884 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: load: jerasure load: lrc 2024-08-30T08:49:54.884 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: RocksDB version: 7.9.2 2024-08-30T08:49:54.884 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: Git sha 0 2024-08-30T08:49:54.884 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: Compile date 2024-08-28 10:19:06 2024-08-30T08:49:54.884 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: DB SUMMARY 2024-08-30T08:49:54.884 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: DB Session ID: 09O5AQKWXNX6WIHZ2I2Y 2024-08-30T08:49:54.885 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: CURRENT file: CURRENT 2024-08-30T08:49:54.885 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: IDENTITY file: IDENTITY 2024-08-30T08:49:54.885 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-08-30T08:49:54.885 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-08-30T08:49:54.885 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 93855 ; 2024-08-30T08:49:54.885 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: Options.error_if_exists: 0 2024-08-30T08:49:54.885 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: Options.create_if_missing: 0 2024-08-30T08:49:54.885 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: Options.paranoid_checks: 1 2024-08-30T08:49:54.885 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: Options.flush_verify_memtable_count: 1 2024-08-30T08:49:54.885 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-08-30T08:49:54.885 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-08-30T08:49:54.885 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: Options.env: 0x55b65bb405a0 2024-08-30T08:49:54.886 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: Options.fs: PosixFileSystem 2024-08-30T08:49:54.886 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: Options.info_log: 0x55b6611794e0 2024-08-30T08:49:54.886 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: Options.max_file_opening_threads: 16 2024-08-30T08:49:54.887 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: Options.statistics: (nil) 2024-08-30T08:49:54.887 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: Options.use_fsync: 0 2024-08-30T08:49:54.887 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: Options.max_log_file_size: 0 2024-08-30T08:49:54.887 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-08-30T08:49:54.887 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: Options.log_file_time_to_roll: 0 2024-08-30T08:49:54.887 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: Options.keep_log_file_num: 1000 2024-08-30T08:49:54.887 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: Options.recycle_log_file_num: 0 2024-08-30T08:49:54.887 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: Options.allow_fallocate: 1 2024-08-30T08:49:54.888 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: Options.allow_mmap_reads: 0 2024-08-30T08:49:54.888 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: Options.allow_mmap_writes: 0 2024-08-30T08:49:54.888 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: Options.use_direct_reads: 0 2024-08-30T08:49:54.888 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-08-30T08:49:54.888 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: Options.create_missing_column_families: 0 2024-08-30T08:49:54.888 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: Options.db_log_dir: 2024-08-30T08:49:54.888 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: Options.wal_dir: 2024-08-30T08:49:54.888 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: Options.table_cache_numshardbits: 6 2024-08-30T08:49:54.888 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: Options.WAL_ttl_seconds: 0 2024-08-30T08:49:54.888 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: Options.WAL_size_limit_MB: 0 2024-08-30T08:49:54.888 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-08-30T08:49:54.889 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-08-30T08:49:54.889 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: Options.is_fd_close_on_exec: 1 2024-08-30T08:49:54.889 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: Options.advise_random_on_open: 1 2024-08-30T08:49:54.889 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: Options.db_write_buffer_size: 0 2024-08-30T08:49:54.889 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: Options.write_buffer_manager: 0x55b66117d5e0 2024-08-30T08:49:54.889 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-08-30T08:49:54.889 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-08-30T08:49:54.889 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: Options.use_adaptive_mutex: 0 2024-08-30T08:49:54.889 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: Options.rate_limiter: (nil) 2024-08-30T08:49:54.889 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-08-30T08:49:54.889 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: Options.wal_recovery_mode: 2 2024-08-30T08:49:54.889 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: Options.enable_thread_tracking: 0 2024-08-30T08:49:54.890 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: Options.enable_pipelined_write: 0 2024-08-30T08:49:54.890 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: Options.unordered_write: 0 2024-08-30T08:49:54.890 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-08-30T08:49:54.890 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-08-30T08:49:54.890 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-08-30T08:49:54.890 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-08-30T08:49:54.890 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: Options.row_cache: None 2024-08-30T08:49:54.890 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: Options.wal_filter: None 2024-08-30T08:49:54.890 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-08-30T08:49:54.890 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: Options.allow_ingest_behind: 0 2024-08-30T08:49:54.890 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: Options.two_write_queues: 0 2024-08-30T08:49:54.890 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: Options.manual_wal_flush: 0 2024-08-30T08:49:54.890 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: Options.wal_compression: 0 2024-08-30T08:49:54.891 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: Options.atomic_flush: 0 2024-08-30T08:49:54.891 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-08-30T08:49:54.891 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: Options.persist_stats_to_disk: 0 2024-08-30T08:49:54.891 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: Options.write_dbid_to_manifest: 0 2024-08-30T08:49:54.891 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: Options.log_readahead_size: 0 2024-08-30T08:49:54.891 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-08-30T08:49:54.891 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: Options.best_efforts_recovery: 0 2024-08-30T08:49:54.891 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-08-30T08:49:54.891 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-08-30T08:49:54.891 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: Options.allow_data_in_errors: 0 2024-08-30T08:49:54.891 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: Options.db_host_id: __hostname__ 2024-08-30T08:49:54.891 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: Options.enforce_single_del_contracts: true 2024-08-30T08:49:54.891 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: Options.max_background_jobs: 2 2024-08-30T08:49:54.891 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: Options.max_background_compactions: -1 2024-08-30T08:49:54.892 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: Options.max_subcompactions: 1 2024-08-30T08:49:54.892 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-08-30T08:49:54.892 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-08-30T08:49:54.892 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: Options.delayed_write_rate : 16777216 2024-08-30T08:49:54.892 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: Options.max_total_wal_size: 0 2024-08-30T08:49:54.892 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-08-30T08:49:54.892 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: Options.stats_dump_period_sec: 600 2024-08-30T08:49:54.892 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: Options.stats_persist_period_sec: 600 2024-08-30T08:49:54.892 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-08-30T08:49:54.892 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: Options.max_open_files: -1 2024-08-30T08:49:54.892 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: Options.bytes_per_sync: 0 2024-08-30T08:49:54.892 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: Options.wal_bytes_per_sync: 0 2024-08-30T08:49:54.892 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: Options.strict_bytes_per_sync: 0 2024-08-30T08:49:54.892 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: Options.compaction_readahead_size: 0 2024-08-30T08:49:54.893 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: Options.max_background_flushes: -1 2024-08-30T08:49:54.893 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: Compression algorithms supported: 2024-08-30T08:49:54.893 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: kZSTD supported: 0 2024-08-30T08:49:54.893 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: kXpressCompression supported: 0 2024-08-30T08:49:54.893 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: kBZip2Compression supported: 0 2024-08-30T08:49:54.893 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-08-30T08:49:54.893 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: kLZ4Compression supported: 1 2024-08-30T08:49:54.893 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: kZlibCompression supported: 1 2024-08-30T08:49:54.893 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: kLZ4HCCompression supported: 1 2024-08-30T08:49:54.893 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: kSnappyCompression supported: 1 2024-08-30T08:49:54.893 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: Fast CRC32 supported: Supported on x86 2024-08-30T08:49:54.893 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: DMutex implementation: pthread_mutex_t 2024-08-30T08:49:54.893 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-08-30T08:49:54.893 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-08-30T08:49:54.894 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-08-30T08:49:54.894 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: Options.merge_operator: 2024-08-30T08:49:54.894 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: Options.compaction_filter: None 2024-08-30T08:49:54.894 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: Options.compaction_filter_factory: None 2024-08-30T08:49:54.894 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: Options.sst_partitioner_factory: None 2024-08-30T08:49:54.894 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: Options.memtable_factory: SkipListFactory 2024-08-30T08:49:54.894 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: Options.table_factory: BlockBasedTable 2024-08-30T08:49:54.894 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55b661179100) 2024-08-30T08:49:54.894 INFO:journalctl@ceph.mon.a.smithi042.stdout: cache_index_and_filter_blocks: 1 2024-08-30T08:49:54.894 INFO:journalctl@ceph.mon.a.smithi042.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-08-30T08:49:54.894 INFO:journalctl@ceph.mon.a.smithi042.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-08-30T08:49:54.894 INFO:journalctl@ceph.mon.a.smithi042.stdout: pin_top_level_index_and_filter: 1 2024-08-30T08:49:54.894 INFO:journalctl@ceph.mon.a.smithi042.stdout: index_type: 0 2024-08-30T08:49:54.895 INFO:journalctl@ceph.mon.a.smithi042.stdout: data_block_index_type: 0 2024-08-30T08:49:54.895 INFO:journalctl@ceph.mon.a.smithi042.stdout: index_shortening: 1 2024-08-30T08:49:54.895 INFO:journalctl@ceph.mon.a.smithi042.stdout: data_block_hash_table_util_ratio: 0.750000 2024-08-30T08:49:54.895 INFO:journalctl@ceph.mon.a.smithi042.stdout: checksum: 4 2024-08-30T08:49:54.895 INFO:journalctl@ceph.mon.a.smithi042.stdout: no_block_cache: 0 2024-08-30T08:49:54.895 INFO:journalctl@ceph.mon.a.smithi042.stdout: block_cache: 0x55b65d13c850 2024-08-30T08:49:54.895 INFO:journalctl@ceph.mon.a.smithi042.stdout: block_cache_name: BinnedLRUCache 2024-08-30T08:49:54.895 INFO:journalctl@ceph.mon.a.smithi042.stdout: block_cache_options: 2024-08-30T08:49:54.895 INFO:journalctl@ceph.mon.a.smithi042.stdout: capacity : 536870912 2024-08-30T08:49:54.895 INFO:journalctl@ceph.mon.a.smithi042.stdout: num_shard_bits : 4 2024-08-30T08:49:54.895 INFO:journalctl@ceph.mon.a.smithi042.stdout: strict_capacity_limit : 0 2024-08-30T08:49:54.895 INFO:journalctl@ceph.mon.a.smithi042.stdout: high_pri_pool_ratio: 0.000 2024-08-30T08:49:54.895 INFO:journalctl@ceph.mon.a.smithi042.stdout: block_cache_compressed: (nil) 2024-08-30T08:49:54.896 INFO:journalctl@ceph.mon.a.smithi042.stdout: persistent_cache: (nil) 2024-08-30T08:49:54.896 INFO:journalctl@ceph.mon.a.smithi042.stdout: block_size: 4096 2024-08-30T08:49:54.896 INFO:journalctl@ceph.mon.a.smithi042.stdout: block_size_deviation: 10 2024-08-30T08:49:54.896 INFO:journalctl@ceph.mon.a.smithi042.stdout: block_restart_interval: 16 2024-08-30T08:49:54.896 INFO:journalctl@ceph.mon.a.smithi042.stdout: index_block_restart_interval: 1 2024-08-30T08:49:54.896 INFO:journalctl@ceph.mon.a.smithi042.stdout: metadata_block_size: 4096 2024-08-30T08:49:54.896 INFO:journalctl@ceph.mon.a.smithi042.stdout: partition_filters: 0 2024-08-30T08:49:54.896 INFO:journalctl@ceph.mon.a.smithi042.stdout: use_delta_encoding: 1 2024-08-30T08:49:54.896 INFO:journalctl@ceph.mon.a.smithi042.stdout: filter_policy: bloomfilter 2024-08-30T08:49:54.896 INFO:journalctl@ceph.mon.a.smithi042.stdout: whole_key_filtering: 1 2024-08-30T08:49:54.896 INFO:journalctl@ceph.mon.a.smithi042.stdout: verify_compression: 0 2024-08-30T08:49:54.896 INFO:journalctl@ceph.mon.a.smithi042.stdout: read_amp_bytes_per_bit: 0 2024-08-30T08:49:54.897 INFO:journalctl@ceph.mon.a.smithi042.stdout: format_version: 5 2024-08-30T08:49:54.897 INFO:journalctl@ceph.mon.a.smithi042.stdout: enable_index_compression: 1 2024-08-30T08:49:54.897 INFO:journalctl@ceph.mon.a.smithi042.stdout: block_align: 0 2024-08-30T08:49:54.897 INFO:journalctl@ceph.mon.a.smithi042.stdout: max_auto_readahead_size: 262144 2024-08-30T08:49:54.897 INFO:journalctl@ceph.mon.a.smithi042.stdout: prepopulate_block_cache: 0 2024-08-30T08:49:54.897 INFO:journalctl@ceph.mon.a.smithi042.stdout: initial_auto_readahead_size: 8192 2024-08-30T08:49:54.897 INFO:journalctl@ceph.mon.a.smithi042.stdout: num_file_reads_for_auto_readahead: 2 2024-08-30T08:49:54.897 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: Options.write_buffer_size: 33554432 2024-08-30T08:49:54.897 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: Options.max_write_buffer_number: 2 2024-08-30T08:49:54.897 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: Options.compression: NoCompression 2024-08-30T08:49:54.897 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: Options.bottommost_compression: Disabled 2024-08-30T08:49:54.897 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: Options.prefix_extractor: nullptr 2024-08-30T08:49:54.897 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-08-30T08:49:54.897 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: Options.num_levels: 7 2024-08-30T08:49:54.898 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-08-30T08:49:54.898 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-08-30T08:49:54.898 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-08-30T08:49:54.898 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-08-30T08:49:54.898 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-08-30T08:49:54.898 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-08-30T08:49:54.898 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-08-30T08:49:54.898 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-08-30T08:49:54.898 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-08-30T08:49:54.898 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-08-30T08:49:54.898 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-08-30T08:49:54.898 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-08-30T08:49:54.898 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: Options.compression_opts.window_bits: -14 2024-08-30T08:49:54.898 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: Options.compression_opts.level: 32767 2024-08-30T08:49:54.898 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: Options.compression_opts.strategy: 0 2024-08-30T08:49:54.899 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-08-30T08:49:54.899 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-08-30T08:49:54.899 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-08-30T08:49:54.899 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-08-30T08:49:54.899 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: Options.compression_opts.enabled: false 2024-08-30T08:49:54.899 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-08-30T08:49:54.899 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-08-30T08:49:54.899 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-08-30T08:49:54.899 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-08-30T08:49:54.899 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: Options.target_file_size_base: 67108864 2024-08-30T08:49:54.899 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: Options.target_file_size_multiplier: 1 2024-08-30T08:49:54.899 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-08-30T08:49:54.899 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-08-30T08:49:54.899 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-08-30T08:49:54.899 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-08-30T08:49:54.899 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-08-30T08:49:54.900 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-08-30T08:49:54.900 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-08-30T08:49:54.900 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-08-30T08:49:54.900 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-08-30T08:49:54.900 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-08-30T08:49:54.900 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-08-30T08:49:54.900 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-08-30T08:49:54.900 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-08-30T08:49:54.900 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: Options.arena_block_size: 1048576 2024-08-30T08:49:54.900 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-08-30T08:49:54.900 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-08-30T08:49:54.900 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: Options.disable_auto_compactions: 0 2024-08-30T08:49:54.900 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-08-30T08:49:54.901 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-08-30T08:49:54.901 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-08-30T08:49:54.901 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-08-30T08:49:54.901 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-08-30T08:49:54.901 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-08-30T08:49:54.901 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-08-30T08:49:54.901 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-08-30T08:49:54.901 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-08-30T08:49:54.901 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-08-30T08:49:54.901 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-08-30T08:49:54.901 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: Options.inplace_update_support: 0 2024-08-30T08:49:54.901 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: Options.inplace_update_num_locks: 10000 2024-08-30T08:49:54.901 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-08-30T08:49:54.901 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-08-30T08:49:54.902 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: Options.memtable_huge_page_size: 0 2024-08-30T08:49:54.902 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: Options.bloom_locality: 0 2024-08-30T08:49:54.902 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: Options.max_successive_merges: 0 2024-08-30T08:49:54.902 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: Options.optimize_filters_for_hits: 0 2024-08-30T08:49:54.902 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: Options.paranoid_file_checks: 0 2024-08-30T08:49:54.902 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: Options.force_consistency_checks: 1 2024-08-30T08:49:54.902 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: Options.report_bg_io_stats: 0 2024-08-30T08:49:54.902 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: Options.ttl: 2592000 2024-08-30T08:49:54.902 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: Options.periodic_compaction_seconds: 0 2024-08-30T08:49:54.902 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-08-30T08:49:54.902 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-08-30T08:49:54.902 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: Options.enable_blob_files: false 2024-08-30T08:49:54.902 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: Options.min_blob_size: 0 2024-08-30T08:49:54.902 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: Options.blob_file_size: 268435456 2024-08-30T08:49:54.902 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: Options.blob_compression_type: NoCompression 2024-08-30T08:49:54.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: Options.enable_blob_garbage_collection: false 2024-08-30T08:49:54.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-08-30T08:49:54.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-08-30T08:49:54.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-08-30T08:49:54.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: Options.blob_file_starting_level: 0 2024-08-30T08:49:54.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-08-30T08:49:54.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-08-30T08:49:54.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-08-30T08:49:54.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: d56c09b8-7d15-46a8-8dbf-791299e087d5 2024-08-30T08:49:54.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: EVENT_LOG_v1 {"time_micros": 1725007794487788, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-08-30T08:49:54.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-08-30T08:49:54.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: EVENT_LOG_v1 {"time_micros": 1725007794488935, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 90055, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 278, "table_properties": {"data_size": 88125, "index_size": 250, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 645, "raw_key_size": 11355, "raw_average_key_size": 47, "raw_value_size": 81772, "raw_average_value_size": 345, "num_data_blocks": 11, "num_entries": 237, "num_filter_entries": 237, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1725007794, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d56c09b8-7d15-46a8-8dbf-791299e087d5", "db_session_id": "09O5AQKWXNX6WIHZ2I2Y", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-08-30T08:49:54.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: EVENT_LOG_v1 {"time_micros": 1725007794489049, "job": 1, "event": "recovery_finished"} 2024-08-30T08:49:54.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-08-30T08:49:54.904 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-08-30T08:49:54.904 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55b66119ca80 2024-08-30T08:49:54.904 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: DB pointer 0x55b661f3e000 2024-08-30T08:49:54.904 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-08-30T08:49:54.904 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-08-30T08:49:54.904 INFO:journalctl@ceph.mon.a.smithi042.stdout: ** DB Stats ** 2024-08-30T08:49:54.904 INFO:journalctl@ceph.mon.a.smithi042.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-08-30T08:49:54.904 INFO:journalctl@ceph.mon.a.smithi042.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-08-30T08:49:54.904 INFO:journalctl@ceph.mon.a.smithi042.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-08-30T08:49:54.904 INFO:journalctl@ceph.mon.a.smithi042.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-08-30T08:49:54.904 INFO:journalctl@ceph.mon.a.smithi042.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-08-30T08:49:54.904 INFO:journalctl@ceph.mon.a.smithi042.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-08-30T08:49:54.904 INFO:journalctl@ceph.mon.a.smithi042.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-08-30T08:49:54.904 INFO:journalctl@ceph.mon.a.smithi042.stdout: 2024-08-30T08:49:54.905 INFO:journalctl@ceph.mon.a.smithi042.stdout: ** Compaction Stats [default] ** 2024-08-30T08:49:54.905 INFO:journalctl@ceph.mon.a.smithi042.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-08-30T08:49:54.905 INFO:journalctl@ceph.mon.a.smithi042.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-08-30T08:49:54.905 INFO:journalctl@ceph.mon.a.smithi042.stdout: L0 2/0 89.80 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 119.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-08-30T08:49:54.905 INFO:journalctl@ceph.mon.a.smithi042.stdout: Sum 2/0 89.80 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 119.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-08-30T08:49:54.905 INFO:journalctl@ceph.mon.a.smithi042.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 119.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-08-30T08:49:54.905 INFO:journalctl@ceph.mon.a.smithi042.stdout: 2024-08-30T08:49:54.905 INFO:journalctl@ceph.mon.a.smithi042.stdout: ** Compaction Stats [default] ** 2024-08-30T08:49:54.905 INFO:journalctl@ceph.mon.a.smithi042.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-08-30T08:49:54.905 INFO:journalctl@ceph.mon.a.smithi042.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-08-30T08:49:54.905 INFO:journalctl@ceph.mon.a.smithi042.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 119.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-08-30T08:49:54.905 INFO:journalctl@ceph.mon.a.smithi042.stdout: 2024-08-30T08:49:54.906 INFO:journalctl@ceph.mon.a.smithi042.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-08-30T08:49:54.906 INFO:journalctl@ceph.mon.a.smithi042.stdout: 2024-08-30T08:49:54.906 INFO:journalctl@ceph.mon.a.smithi042.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-08-30T08:49:54.906 INFO:journalctl@ceph.mon.a.smithi042.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-08-30T08:49:54.906 INFO:journalctl@ceph.mon.a.smithi042.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-08-30T08:49:54.906 INFO:journalctl@ceph.mon.a.smithi042.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-08-30T08:49:54.906 INFO:journalctl@ceph.mon.a.smithi042.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-08-30T08:49:54.906 INFO:journalctl@ceph.mon.a.smithi042.stdout: AddFile(Keys): cumulative 0, interval 0 2024-08-30T08:49:54.906 INFO:journalctl@ceph.mon.a.smithi042.stdout: Cumulative compaction: 0.00 GB write, 24.16 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-08-30T08:49:54.906 INFO:journalctl@ceph.mon.a.smithi042.stdout: Interval compaction: 0.00 GB write, 24.16 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-08-30T08:49:54.906 INFO:journalctl@ceph.mon.a.smithi042.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-08-30T08:49:54.906 INFO:journalctl@ceph.mon.a.smithi042.stdout: Block cache BinnedLRUCache@0x55b65d13c850#2 capacity: 512.00 MB usage: 1.28 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.1e-05 secs_since: 0 2024-08-30T08:49:54.906 INFO:journalctl@ceph.mon.a.smithi042.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.84 KB,0.000160933%) IndexBlock(2,0.44 KB,8.34465e-05%) Misc(1,0.00 KB,0%) 2024-08-30T08:49:54.906 INFO:journalctl@ceph.mon.a.smithi042.stdout: 2024-08-30T08:49:54.907 INFO:journalctl@ceph.mon.a.smithi042.stdout: ** File Read Latency Histogram By Level [default] ** 2024-08-30T08:49:54.907 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: starting mon.a rank 0 at public addrs [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] at bind addrs [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 2024-08-30T08:49:54.907 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: mon.a@-1(???) e1 preinit fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 2024-08-30T08:49:54.907 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: mon.a@-1(???).mds e1 new map 2024-08-30T08:49:54.907 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: mon.a@-1(???).mds e1 print_map 2024-08-30T08:49:54.907 INFO:journalctl@ceph.mon.a.smithi042.stdout: e1 2024-08-30T08:49:54.907 INFO:journalctl@ceph.mon.a.smithi042.stdout: btime 2024-08-30T08:49:44:858072+0000 2024-08-30T08:49:54.907 INFO:journalctl@ceph.mon.a.smithi042.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-08-30T08:49:54.907 INFO:journalctl@ceph.mon.a.smithi042.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-08-30T08:49:54.907 INFO:journalctl@ceph.mon.a.smithi042.stdout: legacy client fscid: -1 2024-08-30T08:49:54.907 INFO:journalctl@ceph.mon.a.smithi042.stdout: 2024-08-30T08:49:54.907 INFO:journalctl@ceph.mon.a.smithi042.stdout: No filesystems configured 2024-08-30T08:49:54.907 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-08-30T08:49:54.907 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-08-30T08:49:54.907 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-08-30T08:49:54.908 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-08-30T08:49:54.908 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-08-30T08:49:54.908 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: mon.a is new leader, mons a in quorum (ranks 0) 2024-08-30T08:49:54.908 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: monmap epoch 1 2024-08-30T08:49:54.908 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 2024-08-30T08:49:54.908 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: last_changed 2024-08-30T08:49:41.668361+0000 2024-08-30T08:49:54.908 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: created 2024-08-30T08:49:41.668361+0000 2024-08-30T08:49:54.908 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: min_mon_release 19 (squid) 2024-08-30T08:49:54.908 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: election_strategy: 1 2024-08-30T08:49:54.908 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: 0: [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] mon.a 2024-08-30T08:49:54.908 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: fsmap 2024-08-30T08:49:54.908 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: osdmap e1: 0 total, 0 up, 0 in 2024-08-30T08:49:54.908 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:54 smithi042 ceph-mon[18652]: mgrmap e1: no daemons active 2024-08-30T08:49:55.338 INFO:teuthology.orchestra.run.smithi042.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-08-30T08:49:55.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:55 smithi042 bash[18633]: e677a01ba2f722b1e6e32fc4353612a39c416728efa70ee5abac8539d253b076 2024-08-30T08:49:55.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:55 smithi042 systemd[1]: Started Ceph mon.a for b0895c1a-66ac-11ef-bcd4-c7b262605968. 2024-08-30T08:49:59.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:49:59 smithi042 ceph-mon[18652]: from='client.? 172.21.15.42:0/2651488521' entity='client.admin' 2024-08-30T08:50:02.187 INFO:teuthology.orchestra.run.smithi042.stdout:Wrote config to /etc/ceph/ceph.conf 2024-08-30T08:50:02.207 INFO:teuthology.orchestra.run.smithi042.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-08-30T08:50:02.207 INFO:teuthology.orchestra.run.smithi042.stdout:Creating mgr... 2024-08-30T08:50:02.207 INFO:teuthology.orchestra.run.smithi042.stdout:Verifying port 0.0.0.0:9283 ... 2024-08-30T08:50:02.208 INFO:teuthology.orchestra.run.smithi042.stdout:Verifying port 0.0.0.0:8765 ... 2024-08-30T08:50:02.468 INFO:teuthology.orchestra.run.smithi042.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968@mgr.x 2024-08-30T08:50:02.469 INFO:teuthology.orchestra.run.smithi042.stdout:systemctl: stderr Failed to reset failed state of unit ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968@mgr.x.service: Unit ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968@mgr.x.service not loaded. 2024-08-30T08:50:02.614 INFO:teuthology.orchestra.run.smithi042.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968.target.wants/ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968@.service. 2024-08-30T08:50:02.883 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Aug 30 08:50:02 smithi042 podman[18880]: 2024-08-30 08:50:02.841527889 +0000 UTC m=+0.113913511 container create c24e99374a5dc1ba7b14271d729edf828792ec9b703b90e1f5e4cc17b0a95294 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6, name=ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mgr-x, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.37.1, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.build-date=20240731, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True) 2024-08-30T08:50:03.151 INFO:teuthology.orchestra.run.smithi042.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-08-30T08:50:03.151 INFO:teuthology.orchestra.run.smithi042.stdout:systemctl: stdout disabled 2024-08-30T08:50:03.159 INFO:teuthology.orchestra.run.smithi042.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-08-30T08:50:03.159 INFO:teuthology.orchestra.run.smithi042.stdout:systemctl: stdout inactive 2024-08-30T08:50:03.159 INFO:teuthology.orchestra.run.smithi042.stdout:firewalld.service is not enabled 2024-08-30T08:50:03.159 INFO:teuthology.orchestra.run.smithi042.stdout:Not possible to enable service . firewalld.service is not available 2024-08-30T08:50:03.168 INFO:teuthology.orchestra.run.smithi042.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-08-30T08:50:03.168 INFO:teuthology.orchestra.run.smithi042.stdout:systemctl: stdout disabled 2024-08-30T08:50:03.176 INFO:teuthology.orchestra.run.smithi042.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-08-30T08:50:03.176 INFO:teuthology.orchestra.run.smithi042.stdout:systemctl: stdout inactive 2024-08-30T08:50:03.176 INFO:teuthology.orchestra.run.smithi042.stdout:firewalld.service is not enabled 2024-08-30T08:50:03.176 INFO:teuthology.orchestra.run.smithi042.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-08-30T08:50:03.176 INFO:teuthology.orchestra.run.smithi042.stdout:Waiting for mgr to start... 2024-08-30T08:50:03.176 INFO:teuthology.orchestra.run.smithi042.stdout:Waiting for mgr... 2024-08-30T08:50:03.307 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Aug 30 08:50:03 smithi042 podman[18880]: 2024-08-30 08:50:03.052525751 +0000 UTC m=+0.324911374 container init c24e99374a5dc1ba7b14271d729edf828792ec9b703b90e1f5e4cc17b0a95294 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6, name=ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mgr-x, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.37.1, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-08-30T08:50:03.307 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Aug 30 08:50:03 smithi042 podman[18880]: 2024-08-30 08:50:03.055099583 +0000 UTC m=+0.327485206 container start c24e99374a5dc1ba7b14271d729edf828792ec9b703b90e1f5e4cc17b0a95294 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6, name=ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mgr-x, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.37.1, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, ceph=True, GIT_CLEAN=True) 2024-08-30T08:50:03.307 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Aug 30 08:50:03 smithi042 bash[18880]: c24e99374a5dc1ba7b14271d729edf828792ec9b703b90e1f5e4cc17b0a95294 2024-08-30T08:50:03.307 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Aug 30 08:50:03 smithi042 systemd[1]: Started Ceph mgr.x for b0895c1a-66ac-11ef-bcd4-c7b262605968. 2024-08-30T08:50:03.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:50:03 smithi042 ceph-mon[18652]: from='client.? 172.21.15.42:0/205339670' entity='client.admin' cmd={"prefix": "status", "format": "json-pretty"} : dispatch 2024-08-30T08:50:04.301 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout 2024-08-30T08:50:04.301 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout { 2024-08-30T08:50:04.302 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "fsid": "b0895c1a-66ac-11ef-bcd4-c7b262605968", 2024-08-30T08:50:04.302 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "health": { 2024-08-30T08:50:04.302 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-08-30T08:50:04.302 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-08-30T08:50:04.302 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-08-30T08:50:04.302 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout }, 2024-08-30T08:50:04.302 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-08-30T08:50:04.302 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-08-30T08:50:04.302 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout 0 2024-08-30T08:50:04.302 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout ], 2024-08-30T08:50:04.302 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-08-30T08:50:04.302 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "a" 2024-08-30T08:50:04.302 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout ], 2024-08-30T08:50:04.302 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "quorum_age": 9, 2024-08-30T08:50:04.302 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "monmap": { 2024-08-30T08:50:04.303 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-08-30T08:50:04.303 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-08-30T08:50:04.303 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-08-30T08:50:04.303 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout }, 2024-08-30T08:50:04.303 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-08-30T08:50:04.303 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-08-30T08:50:04.303 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-08-30T08:50:04.303 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-08-30T08:50:04.303 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-08-30T08:50:04.303 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-08-30T08:50:04.303 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-08-30T08:50:04.303 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-08-30T08:50:04.303 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout }, 2024-08-30T08:50:04.303 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-08-30T08:50:04.304 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-08-30T08:50:04.304 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-08-30T08:50:04.304 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-08-30T08:50:04.304 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-08-30T08:50:04.304 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-08-30T08:50:04.304 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-08-30T08:50:04.304 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-08-30T08:50:04.304 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-08-30T08:50:04.304 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout }, 2024-08-30T08:50:04.304 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-08-30T08:50:04.304 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-08-30T08:50:04.304 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "btime": "2024-08-30T08:49:44:858072+0000", 2024-08-30T08:50:04.304 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-08-30T08:50:04.305 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-08-30T08:50:04.305 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout }, 2024-08-30T08:50:04.305 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-08-30T08:50:04.305 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "available": false, 2024-08-30T08:50:04.305 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-08-30T08:50:04.305 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "modules": [ 2024-08-30T08:50:04.305 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "iostat", 2024-08-30T08:50:04.305 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "nfs", 2024-08-30T08:50:04.305 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "restful" 2024-08-30T08:50:04.305 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout ], 2024-08-30T08:50:04.305 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "services": {} 2024-08-30T08:50:04.305 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout }, 2024-08-30T08:50:04.305 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-08-30T08:50:04.305 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-08-30T08:50:04.306 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "modified": "2024-08-30T08:49:44.858917+0000", 2024-08-30T08:50:04.306 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "services": {} 2024-08-30T08:50:04.306 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout }, 2024-08-30T08:50:04.306 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-08-30T08:50:04.306 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout } 2024-08-30T08:50:04.306 INFO:teuthology.orchestra.run.smithi042.stdout:mgr not available, waiting (1/15)... 2024-08-30T08:50:07.132 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:50:06 smithi042 ceph-mon[18652]: from='client.? 172.21.15.42:0/1876014280' entity='client.admin' cmd={"prefix": "status", "format": "json-pretty"} : dispatch 2024-08-30T08:50:07.607 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout 2024-08-30T08:50:07.608 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout { 2024-08-30T08:50:07.608 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "fsid": "b0895c1a-66ac-11ef-bcd4-c7b262605968", 2024-08-30T08:50:07.608 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "health": { 2024-08-30T08:50:07.608 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-08-30T08:50:07.608 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-08-30T08:50:07.608 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-08-30T08:50:07.608 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout }, 2024-08-30T08:50:07.608 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-08-30T08:50:07.608 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-08-30T08:50:07.608 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout 0 2024-08-30T08:50:07.608 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout ], 2024-08-30T08:50:07.608 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-08-30T08:50:07.609 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "a" 2024-08-30T08:50:07.609 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout ], 2024-08-30T08:50:07.609 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "quorum_age": 12, 2024-08-30T08:50:07.609 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "monmap": { 2024-08-30T08:50:07.609 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-08-30T08:50:07.609 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-08-30T08:50:07.609 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-08-30T08:50:07.609 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout }, 2024-08-30T08:50:07.609 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-08-30T08:50:07.609 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-08-30T08:50:07.609 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-08-30T08:50:07.609 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-08-30T08:50:07.609 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-08-30T08:50:07.610 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-08-30T08:50:07.610 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-08-30T08:50:07.610 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-08-30T08:50:07.610 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout }, 2024-08-30T08:50:07.610 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-08-30T08:50:07.610 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-08-30T08:50:07.610 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-08-30T08:50:07.610 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-08-30T08:50:07.610 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-08-30T08:50:07.610 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-08-30T08:50:07.610 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-08-30T08:50:07.610 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-08-30T08:50:07.610 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-08-30T08:50:07.611 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout }, 2024-08-30T08:50:07.611 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-08-30T08:50:07.611 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-08-30T08:50:07.611 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "btime": "2024-08-30T08:49:44:858072+0000", 2024-08-30T08:50:07.611 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-08-30T08:50:07.611 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-08-30T08:50:07.611 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout }, 2024-08-30T08:50:07.611 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-08-30T08:50:07.611 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "available": false, 2024-08-30T08:50:07.611 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-08-30T08:50:07.611 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "modules": [ 2024-08-30T08:50:07.611 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "iostat", 2024-08-30T08:50:07.611 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "nfs", 2024-08-30T08:50:07.612 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "restful" 2024-08-30T08:50:07.612 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout ], 2024-08-30T08:50:07.612 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "services": {} 2024-08-30T08:50:07.612 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout }, 2024-08-30T08:50:07.612 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-08-30T08:50:07.612 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-08-30T08:50:07.612 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "modified": "2024-08-30T08:49:44.858917+0000", 2024-08-30T08:50:07.612 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "services": {} 2024-08-30T08:50:07.612 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout }, 2024-08-30T08:50:07.612 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-08-30T08:50:07.612 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout } 2024-08-30T08:50:07.612 INFO:teuthology.orchestra.run.smithi042.stdout:mgr not available, waiting (2/15)... 2024-08-30T08:50:08.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:50:08 smithi042 ceph-mon[18652]: Activating manager daemon x 2024-08-30T08:50:08.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:50:08 smithi042 ceph-mon[18652]: mgrmap e2: x(active, starting, since 0.0036163s) 2024-08-30T08:50:08.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:50:08 smithi042 ceph-mon[18652]: from='mgr.14100 172.21.15.42:0/1069863380' entity='mgr.x' cmd={"prefix": "mds metadata"} : dispatch 2024-08-30T08:50:08.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:50:08 smithi042 ceph-mon[18652]: from='mgr.14100 172.21.15.42:0/1069863380' entity='mgr.x' cmd={"prefix": "osd metadata"} : dispatch 2024-08-30T08:50:08.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:50:08 smithi042 ceph-mon[18652]: from='mgr.14100 172.21.15.42:0/1069863380' entity='mgr.x' cmd={"prefix": "mon metadata"} : dispatch 2024-08-30T08:50:08.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:50:08 smithi042 ceph-mon[18652]: from='mgr.14100 172.21.15.42:0/1069863380' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-08-30T08:50:08.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:50:08 smithi042 ceph-mon[18652]: from='mgr.14100 172.21.15.42:0/1069863380' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-08-30T08:50:08.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:50:08 smithi042 ceph-mon[18652]: Manager daemon x is now available 2024-08-30T08:50:08.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:50:08 smithi042 ceph-mon[18652]: from='mgr.14100 172.21.15.42:0/1069863380' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"} : dispatch 2024-08-30T08:50:08.634 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:50:08 smithi042 ceph-mon[18652]: from='mgr.14100 172.21.15.42:0/1069863380' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"} : dispatch 2024-08-30T08:50:08.634 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:50:08 smithi042 ceph-mon[18652]: from='mgr.14100 172.21.15.42:0/1069863380' entity='mgr.x' 2024-08-30T08:50:08.634 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:50:08 smithi042 ceph-mon[18652]: from='mgr.14100 172.21.15.42:0/1069863380' entity='mgr.x' 2024-08-30T08:50:08.634 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:50:08 smithi042 ceph-mon[18652]: from='mgr.14100 172.21.15.42:0/1069863380' entity='mgr.x' 2024-08-30T08:50:10.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:50:10 smithi042 ceph-mon[18652]: mgrmap e3: x(active, since 1.00706s) 2024-08-30T08:50:10.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:50:10 smithi042 ceph-mon[18652]: from='client.? 172.21.15.42:0/3054587884' entity='client.admin' cmd={"prefix": "status", "format": "json-pretty"} : dispatch 2024-08-30T08:50:10.839 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout 2024-08-30T08:50:10.840 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout { 2024-08-30T08:50:10.840 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "fsid": "b0895c1a-66ac-11ef-bcd4-c7b262605968", 2024-08-30T08:50:10.840 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "health": { 2024-08-30T08:50:10.840 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-08-30T08:50:10.840 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-08-30T08:50:10.840 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-08-30T08:50:10.840 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout }, 2024-08-30T08:50:10.840 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-08-30T08:50:10.840 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-08-30T08:50:10.840 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout 0 2024-08-30T08:50:10.840 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout ], 2024-08-30T08:50:10.840 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-08-30T08:50:10.840 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "a" 2024-08-30T08:50:10.841 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout ], 2024-08-30T08:50:10.841 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "quorum_age": 15, 2024-08-30T08:50:10.841 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "monmap": { 2024-08-30T08:50:10.841 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-08-30T08:50:10.841 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-08-30T08:50:10.841 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-08-30T08:50:10.841 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout }, 2024-08-30T08:50:10.841 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-08-30T08:50:10.841 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-08-30T08:50:10.841 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-08-30T08:50:10.841 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-08-30T08:50:10.841 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-08-30T08:50:10.841 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-08-30T08:50:10.841 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-08-30T08:50:10.842 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-08-30T08:50:10.842 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout }, 2024-08-30T08:50:10.842 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-08-30T08:50:10.842 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-08-30T08:50:10.842 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-08-30T08:50:10.842 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-08-30T08:50:10.842 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-08-30T08:50:10.842 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-08-30T08:50:10.842 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-08-30T08:50:10.842 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-08-30T08:50:10.842 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-08-30T08:50:10.842 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout }, 2024-08-30T08:50:10.842 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-08-30T08:50:10.843 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-08-30T08:50:10.843 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "btime": "2024-08-30T08:49:44:858072+0000", 2024-08-30T08:50:10.843 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-08-30T08:50:10.843 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-08-30T08:50:10.843 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout }, 2024-08-30T08:50:10.843 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-08-30T08:50:10.843 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "available": true, 2024-08-30T08:50:10.843 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-08-30T08:50:10.843 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "modules": [ 2024-08-30T08:50:10.843 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "iostat", 2024-08-30T08:50:10.843 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "nfs", 2024-08-30T08:50:10.843 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "restful" 2024-08-30T08:50:10.843 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout ], 2024-08-30T08:50:10.843 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "services": {} 2024-08-30T08:50:10.843 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout }, 2024-08-30T08:50:10.844 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-08-30T08:50:10.844 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-08-30T08:50:10.844 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "modified": "2024-08-30T08:49:44.858917+0000", 2024-08-30T08:50:10.844 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "services": {} 2024-08-30T08:50:10.844 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout }, 2024-08-30T08:50:10.844 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-08-30T08:50:10.844 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout } 2024-08-30T08:50:10.844 INFO:teuthology.orchestra.run.smithi042.stdout:mgr is available 2024-08-30T08:50:11.382 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:50:11 smithi042 ceph-mon[18652]: mgrmap e4: x(active, since 2s) 2024-08-30T08:50:12.042 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout 2024-08-30T08:50:12.043 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout [global] 2024-08-30T08:50:12.043 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout fsid = b0895c1a-66ac-11ef-bcd4-c7b262605968 2024-08-30T08:50:12.043 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-08-30T08:50:12.043 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.42:3300,v1:172.21.15.42:6789] 2024-08-30T08:50:12.043 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-08-30T08:50:12.043 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-08-30T08:50:12.043 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-08-30T08:50:12.043 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-08-30T08:50:12.043 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout 2024-08-30T08:50:12.043 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout [mgr] 2024-08-30T08:50:12.044 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-08-30T08:50:12.044 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout 2024-08-30T08:50:12.044 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout [osd] 2024-08-30T08:50:12.044 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-08-30T08:50:12.044 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-08-30T08:50:12.044 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-08-30T08:50:12.044 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-08-30T08:50:12.044 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-08-30T08:50:12.044 INFO:teuthology.orchestra.run.smithi042.stdout:Enabling cephadm module... 2024-08-30T08:50:12.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:50:12 smithi042 ceph-mon[18652]: from='client.? 172.21.15.42:0/3371398612' entity='client.admin' cmd={"prefix": "config assimilate-conf"} : dispatch 2024-08-30T08:50:13.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:50:13 smithi042 ceph-mon[18652]: from='client.? 172.21.15.42:0/1411275985' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "cephadm"} : dispatch 2024-08-30T08:50:13.627 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Aug 30 08:50:13 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mgr-x[18896]: ignoring --setuser ceph since I am not root 2024-08-30T08:50:13.627 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Aug 30 08:50:13 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mgr-x[18896]: ignoring --setgroup ceph since I am not root 2024-08-30T08:50:14.632 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:50:14 smithi042 ceph-mon[18652]: from='client.? 172.21.15.42:0/1411275985' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-08-30T08:50:14.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:50:14 smithi042 ceph-mon[18652]: mgrmap e5: x(active, since 5s) 2024-08-30T08:50:15.087 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout { 2024-08-30T08:50:15.087 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-08-30T08:50:15.087 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "available": true, 2024-08-30T08:50:15.087 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-08-30T08:50:15.088 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-08-30T08:50:15.088 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout } 2024-08-30T08:50:15.088 INFO:teuthology.orchestra.run.smithi042.stdout:Waiting for the mgr to restart... 2024-08-30T08:50:15.088 INFO:teuthology.orchestra.run.smithi042.stdout:Waiting for mgr epoch 5... 2024-08-30T08:50:15.382 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:50:15 smithi042 ceph-mon[18652]: from='client.? 172.21.15.42:0/1719386238' entity='client.admin' cmd={"prefix": "mgr stat"} : dispatch 2024-08-30T08:50:18.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:50:18 smithi042 ceph-mon[18652]: Active manager daemon x restarted 2024-08-30T08:50:18.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:50:18 smithi042 ceph-mon[18652]: Activating manager daemon x 2024-08-30T08:50:18.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:50:18 smithi042 ceph-mon[18652]: osdmap e2: 0 total, 0 up, 0 in 2024-08-30T08:50:18.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:50:18 smithi042 ceph-mon[18652]: mgrmap e6: x(active, starting, since 0.00372898s) 2024-08-30T08:50:18.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:50:18 smithi042 ceph-mon[18652]: from='mgr.14118 172.21.15.42:0/231404219' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-08-30T08:50:18.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:50:18 smithi042 ceph-mon[18652]: from='mgr.14118 172.21.15.42:0/231404219' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-08-30T08:50:18.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:50:18 smithi042 ceph-mon[18652]: from='mgr.14118 172.21.15.42:0/231404219' entity='mgr.x' cmd={"prefix": "mds metadata"} : dispatch 2024-08-30T08:50:18.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:50:18 smithi042 ceph-mon[18652]: from='mgr.14118 172.21.15.42:0/231404219' entity='mgr.x' cmd={"prefix": "osd metadata"} : dispatch 2024-08-30T08:50:18.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:50:18 smithi042 ceph-mon[18652]: from='mgr.14118 172.21.15.42:0/231404219' entity='mgr.x' cmd={"prefix": "mon metadata"} : dispatch 2024-08-30T08:50:18.884 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:50:18 smithi042 ceph-mon[18652]: Manager daemon x is now available 2024-08-30T08:50:20.201 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout { 2024-08-30T08:50:20.201 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-08-30T08:50:20.201 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "initialized": true 2024-08-30T08:50:20.201 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout } 2024-08-30T08:50:20.201 INFO:teuthology.orchestra.run.smithi042.stdout:mgr epoch 5 is available 2024-08-30T08:50:20.202 INFO:teuthology.orchestra.run.smithi042.stdout:Setting orchestrator backend to cephadm... 2024-08-30T08:50:20.860 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:50:20 smithi042 ceph-mon[18652]: mgrmap e7: x(active, since 1.00559s) 2024-08-30T08:50:20.860 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:50:20 smithi042 ceph-mon[18652]: from='mgr.14118 172.21.15.42:0/231404219' entity='mgr.x' 2024-08-30T08:50:20.860 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:50:20 smithi042 ceph-mon[18652]: from='mgr.14118 172.21.15.42:0/231404219' entity='mgr.x' 2024-08-30T08:50:20.860 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:50:20 smithi042 ceph-mon[18652]: Found migration_current of "None". Setting to last migration. 2024-08-30T08:50:20.861 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:50:20 smithi042 ceph-mon[18652]: from='mgr.14118 172.21.15.42:0/231404219' entity='mgr.x' 2024-08-30T08:50:20.861 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:50:20 smithi042 ceph-mon[18652]: from='mgr.14118 172.21.15.42:0/231404219' entity='mgr.x' 2024-08-30T08:50:20.861 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:50:20 smithi042 ceph-mon[18652]: from='mgr.14118 172.21.15.42:0/231404219' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-30T08:50:20.861 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:50:20 smithi042 ceph-mon[18652]: from='mgr.14118 172.21.15.42:0/231404219' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-30T08:50:20.861 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:50:20 smithi042 ceph-mon[18652]: from='mgr.14118 172.21.15.42:0/231404219' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"} : dispatch 2024-08-30T08:50:20.861 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:50:20 smithi042 ceph-mon[18652]: from='mgr.14118 172.21.15.42:0/231404219' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"} : dispatch 2024-08-30T08:50:21.679 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Aug 30 08:50:21 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mgr-x[18896]: 2024-08-30T08:50:21.410+0000 7f9bb1a0e640 -1 log_channel(cephadm) log [ERR] : [30/Aug/2024:08:50:21] ENGINE Error in HTTPServer.serve 2024-08-30T08:50:21.679 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Aug 30 08:50:21 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mgr-x[18896]: Traceback (most recent call last): 2024-08-30T08:50:21.679 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Aug 30 08:50:21 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mgr-x[18896]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-08-30T08:50:21.679 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Aug 30 08:50:21 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mgr-x[18896]: self._connections.run(self.expiration_interval) 2024-08-30T08:50:21.679 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Aug 30 08:50:21 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mgr-x[18896]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-08-30T08:50:21.679 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Aug 30 08:50:21 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mgr-x[18896]: self._run(expiration_interval) 2024-08-30T08:50:21.679 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Aug 30 08:50:21 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mgr-x[18896]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-08-30T08:50:21.679 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Aug 30 08:50:21 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mgr-x[18896]: new_conn = self._from_server_socket(self.server.socket) 2024-08-30T08:50:21.679 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Aug 30 08:50:21 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mgr-x[18896]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-08-30T08:50:21.679 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Aug 30 08:50:21 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mgr-x[18896]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-08-30T08:50:21.680 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Aug 30 08:50:21 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mgr-x[18896]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-08-30T08:50:21.680 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Aug 30 08:50:21 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mgr-x[18896]: s = self.context.wrap_socket( 2024-08-30T08:50:21.680 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Aug 30 08:50:21 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mgr-x[18896]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-08-30T08:50:21.680 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Aug 30 08:50:21 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mgr-x[18896]: return self.sslsocket_class._create( 2024-08-30T08:50:21.680 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Aug 30 08:50:21 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mgr-x[18896]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-08-30T08:50:21.680 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Aug 30 08:50:21 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mgr-x[18896]: self.do_handshake() 2024-08-30T08:50:21.680 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Aug 30 08:50:21 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mgr-x[18896]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-08-30T08:50:21.680 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Aug 30 08:50:21 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mgr-x[18896]: self._sslobj.do_handshake() 2024-08-30T08:50:21.680 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Aug 30 08:50:21 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mgr-x[18896]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-08-30T08:50:21.680 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Aug 30 08:50:21 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mgr-x[18896]: 2024-08-30T08:50:21.941 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:50:21 smithi042 ceph-mon[18652]: mgrmap e8: x(active, since 2s) 2024-08-30T08:50:21.941 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:50:21 smithi042 ceph-mon[18652]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-08-30T08:50:21.941 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:50:21 smithi042 ceph-mon[18652]: from='mgr.14118 172.21.15.42:0/231404219' entity='mgr.x' 2024-08-30T08:50:21.942 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:50:21 smithi042 ceph-mon[18652]: from='mgr.14118 172.21.15.42:0/231404219' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-30T08:50:21.942 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:50:21 smithi042 ceph-mon[18652]: [30/Aug/2024:08:50:21] ENGINE Bus STARTING 2024-08-30T08:50:21.942 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:50:21 smithi042 ceph-mon[18652]: [30/Aug/2024:08:50:21] ENGINE Serving on http://172.21.15.42:8765 2024-08-30T08:50:21.942 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:50:21 smithi042 ceph-mon[18652]: from='mgr.14118 172.21.15.42:0/231404219' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-30T08:50:22.572 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout value unchanged 2024-08-30T08:50:22.572 INFO:teuthology.orchestra.run.smithi042.stdout:Generating ssh key... 2024-08-30T08:50:22.882 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:50:22 smithi042 ceph-mon[18652]: [30/Aug/2024:08:50:21] ENGINE Serving on https://172.21.15.42:7150 2024-08-30T08:50:22.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:50:22 smithi042 ceph-mon[18652]: [30/Aug/2024:08:50:21] ENGINE Bus STARTED 2024-08-30T08:50:22.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:50:22 smithi042 ceph-mon[18652]: [30/Aug/2024:08:50:21] ENGINE Error in HTTPServer.serve 2024-08-30T08:50:22.883 INFO:journalctl@ceph.mon.a.smithi042.stdout: Traceback (most recent call last): 2024-08-30T08:50:22.883 INFO:journalctl@ceph.mon.a.smithi042.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-08-30T08:50:22.883 INFO:journalctl@ceph.mon.a.smithi042.stdout: self._connections.run(self.expiration_interval) 2024-08-30T08:50:22.883 INFO:journalctl@ceph.mon.a.smithi042.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-08-30T08:50:22.883 INFO:journalctl@ceph.mon.a.smithi042.stdout: self._run(expiration_interval) 2024-08-30T08:50:22.883 INFO:journalctl@ceph.mon.a.smithi042.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-08-30T08:50:22.883 INFO:journalctl@ceph.mon.a.smithi042.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-08-30T08:50:22.883 INFO:journalctl@ceph.mon.a.smithi042.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-08-30T08:50:22.884 INFO:journalctl@ceph.mon.a.smithi042.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-08-30T08:50:22.884 INFO:journalctl@ceph.mon.a.smithi042.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-08-30T08:50:22.884 INFO:journalctl@ceph.mon.a.smithi042.stdout: s = self.context.wrap_socket( 2024-08-30T08:50:22.884 INFO:journalctl@ceph.mon.a.smithi042.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-08-30T08:50:22.884 INFO:journalctl@ceph.mon.a.smithi042.stdout: return self.sslsocket_class._create( 2024-08-30T08:50:22.884 INFO:journalctl@ceph.mon.a.smithi042.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-08-30T08:50:22.884 INFO:journalctl@ceph.mon.a.smithi042.stdout: self.do_handshake() 2024-08-30T08:50:22.884 INFO:journalctl@ceph.mon.a.smithi042.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-08-30T08:50:22.884 INFO:journalctl@ceph.mon.a.smithi042.stdout: self._sslobj.do_handshake() 2024-08-30T08:50:22.884 INFO:journalctl@ceph.mon.a.smithi042.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-08-30T08:50:22.884 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:50:22 smithi042 ceph-mon[18652]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-08-30T08:50:24.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:50:23 smithi042 ceph-mon[18652]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-08-30T08:50:24.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:50:23 smithi042 ceph-mon[18652]: Generating ssh key... 2024-08-30T08:50:24.133 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Aug 30 08:50:23 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mgr-x[18896]: Generating public/private rsa key pair. 2024-08-30T08:50:24.133 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Aug 30 08:50:23 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mgr-x[18896]: Your identification has been saved in /tmp/tmpzzgpqedj/key 2024-08-30T08:50:24.133 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Aug 30 08:50:23 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mgr-x[18896]: Your public key has been saved in /tmp/tmpzzgpqedj/key.pub 2024-08-30T08:50:24.133 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Aug 30 08:50:23 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mgr-x[18896]: The key fingerprint is: 2024-08-30T08:50:24.134 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Aug 30 08:50:23 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mgr-x[18896]: SHA256:jE2B24Ab4/0jrCMKEAA1CjM1sAhfB/z/JhVq2lo5Vnk ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968 2024-08-30T08:50:24.134 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Aug 30 08:50:23 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mgr-x[18896]: The key's randomart image is: 2024-08-30T08:50:24.134 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Aug 30 08:50:23 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mgr-x[18896]: +---[RSA 3072]----+ 2024-08-30T08:50:24.134 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Aug 30 08:50:23 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mgr-x[18896]: |X+=.oo... | 2024-08-30T08:50:24.134 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Aug 30 08:50:23 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mgr-x[18896]: |=* +=.o . | 2024-08-30T08:50:24.134 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Aug 30 08:50:23 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mgr-x[18896]: |= .. * +. | 2024-08-30T08:50:24.134 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Aug 30 08:50:23 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mgr-x[18896]: | . o +=. .. | 2024-08-30T08:50:24.134 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Aug 30 08:50:23 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mgr-x[18896]: |. ..oS.o.E | 2024-08-30T08:50:24.134 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Aug 30 08:50:23 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mgr-x[18896]: |. o *o.. | 2024-08-30T08:50:24.134 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Aug 30 08:50:23 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mgr-x[18896]: |. . ==+ | 2024-08-30T08:50:24.135 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Aug 30 08:50:23 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mgr-x[18896]: |. . o .oo.o | 2024-08-30T08:50:24.135 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Aug 30 08:50:23 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mgr-x[18896]: |.. . ... o | 2024-08-30T08:50:24.135 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Aug 30 08:50:23 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mgr-x[18896]: +----[SHA256]-----+ 2024-08-30T08:50:25.132 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:50:24 smithi042 ceph-mon[18652]: from='mgr.14118 172.21.15.42:0/231404219' entity='mgr.x' 2024-08-30T08:50:25.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:50:24 smithi042 ceph-mon[18652]: from='mgr.14118 172.21.15.42:0/231404219' entity='mgr.x' 2024-08-30T08:50:25.735 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCwFGca4P/Kl1WAiOKsCR2aXY0VXBnEMh41bZBwTE4rQRFkb5+cISrqz1pDpQEVyQLM7ndRmhSy82Mu6eg9chBZpfudkgdmSqTrPsHKkK0uOVYUaHOmDT8fYycofSI7yEVP+4Lo0+ChNrnp4+avwV+eq/lHewznjUAEC7V6qZ6fS9IUO10IF5FkgoDAHafwT2rLml/mWqMuhB5pHTwlFytjWwtQQsUZoz6sIpk5R5exRWfiMCl+DSXelChr48vYVfdTwj08J65AR3MzrLMRhq3ouLHBfMxboT4CFBoGkBeBVdtxow2YamdlYcOz7WNHi2AwZiMUplkfEgm7VWHgzuAV/H/YpG65TDiWqbSd3yNxDrxTK4guhQf+NUCV2yazO0uCtPcmjg0JMiic+5TEyMu417vJJuS2xX8SC+VOJF3snj7vB8+tznbPrhgT1E+jUWuzjnleglf3x3UhJM4u7p2atZlLv52TBm5SXTQua3blYI11sLsNirlPoNcB7kIjRbs= ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968 2024-08-30T08:50:25.735 INFO:teuthology.orchestra.run.smithi042.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-08-30T08:50:25.735 INFO:teuthology.orchestra.run.smithi042.stdout:Adding key to root@localhost authorized_keys... 2024-08-30T08:50:25.735 INFO:teuthology.orchestra.run.smithi042.stdout:Adding host smithi042... 2024-08-30T08:50:26.051 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:50:25 smithi042 ceph-mon[18652]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-08-30T08:50:27.075 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:50:26 smithi042 ceph-mon[18652]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi042", "addr": "172.21.15.42", "target": ["mon-mgr", ""]}]: dispatch 2024-08-30T08:50:28.615 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:50:28 smithi042 ceph-mon[18652]: Deploying cephadm binary to smithi042 2024-08-30T08:50:31.070 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout Added host 'smithi042' with addr '172.21.15.42' 2024-08-30T08:50:31.070 INFO:teuthology.orchestra.run.smithi042.stdout:Deploying unmanaged mon service... 2024-08-30T08:50:31.328 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:50:31 smithi042 ceph-mon[18652]: from='mgr.14118 172.21.15.42:0/231404219' entity='mgr.x' 2024-08-30T08:50:31.329 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:50:31 smithi042 ceph-mon[18652]: Added host smithi042 2024-08-30T08:50:31.329 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:50:31 smithi042 ceph-mon[18652]: from='mgr.14118 172.21.15.42:0/231404219' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-30T08:50:32.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:50:32 smithi042 ceph-mon[18652]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-08-30T08:50:32.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:50:32 smithi042 ceph-mon[18652]: Saving service mon spec with placement count:5 2024-08-30T08:50:32.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:50:32 smithi042 ceph-mon[18652]: from='mgr.14118 172.21.15.42:0/231404219' entity='mgr.x' 2024-08-30T08:50:32.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:50:32 smithi042 ceph-mon[18652]: from='mgr.14118 172.21.15.42:0/231404219' entity='mgr.x' 2024-08-30T08:50:33.191 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-08-30T08:50:33.191 INFO:teuthology.orchestra.run.smithi042.stdout:Deploying unmanaged mgr service... 2024-08-30T08:50:34.407 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-08-30T08:50:34.814 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:50:34 smithi042 ceph-mon[18652]: from='mgr.14118 172.21.15.42:0/231404219' entity='mgr.x' 2024-08-30T08:50:34.815 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:50:34 smithi042 ceph-mon[18652]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-08-30T08:50:34.815 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:50:34 smithi042 ceph-mon[18652]: Saving service mgr spec with placement count:2 2024-08-30T08:50:34.815 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:50:34 smithi042 ceph-mon[18652]: from='mgr.14118 172.21.15.42:0/231404219' entity='mgr.x' 2024-08-30T08:50:36.263 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:50:36 smithi042 ceph-mon[18652]: from='mgr.14118 172.21.15.42:0/231404219' entity='mgr.x' 2024-08-30T08:50:36.264 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:50:36 smithi042 ceph-mon[18652]: from='client.? 172.21.15.42:0/1613840969' entity='client.admin' 2024-08-30T08:50:36.264 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:50:36 smithi042 ceph-mon[18652]: from='mgr.14118 172.21.15.42:0/231404219' entity='mgr.x' 2024-08-30T08:50:38.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:50:37 smithi042 ceph-mon[18652]: from='client.? 172.21.15.42:0/1393912' entity='client.admin' 2024-08-30T08:50:38.240 INFO:teuthology.orchestra.run.smithi042.stdout:Enabling the dashboard module... 2024-08-30T08:50:39.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:50:39 smithi042 ceph-mon[18652]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-30T08:50:39.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:50:39 smithi042 ceph-mon[18652]: from='client.? 172.21.15.42:0/2211775198' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "dashboard"} : dispatch 2024-08-30T08:50:39.883 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Aug 30 08:50:39 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mgr-x[18896]: ignoring --setuser ceph since I am not root 2024-08-30T08:50:39.883 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Aug 30 08:50:39 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mgr-x[18896]: ignoring --setgroup ceph since I am not root 2024-08-30T08:50:40.857 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:50:40 smithi042 ceph-mon[18652]: from='client.? 172.21.15.42:0/2211775198' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-08-30T08:50:40.858 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:50:40 smithi042 ceph-mon[18652]: mgrmap e9: x(active, since 21s) 2024-08-30T08:50:41.388 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout { 2024-08-30T08:50:41.388 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-08-30T08:50:41.388 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "available": true, 2024-08-30T08:50:41.388 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-08-30T08:50:41.388 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-08-30T08:50:41.388 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout } 2024-08-30T08:50:41.388 INFO:teuthology.orchestra.run.smithi042.stdout:Waiting for the mgr to restart... 2024-08-30T08:50:41.388 INFO:teuthology.orchestra.run.smithi042.stdout:Waiting for mgr epoch 9... 2024-08-30T08:50:41.804 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:50:41 smithi042 ceph-mon[18652]: from='client.? 172.21.15.42:0/3399830898' entity='client.admin' cmd={"prefix": "mgr stat"} : dispatch 2024-08-30T08:50:45.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:50:44 smithi042 ceph-mon[18652]: Active manager daemon x restarted 2024-08-30T08:50:45.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:50:44 smithi042 ceph-mon[18652]: Activating manager daemon x 2024-08-30T08:50:45.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:50:44 smithi042 ceph-mon[18652]: osdmap e3: 0 total, 0 up, 0 in 2024-08-30T08:50:45.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:50:44 smithi042 ceph-mon[18652]: mgrmap e10: x(active, starting, since 0.00394266s) 2024-08-30T08:50:45.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:50:44 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-08-30T08:50:45.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:50:44 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-08-30T08:50:45.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:50:44 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "mds metadata"} : dispatch 2024-08-30T08:50:45.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:50:44 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata"} : dispatch 2024-08-30T08:50:45.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:50:44 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "mon metadata"} : dispatch 2024-08-30T08:50:45.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:50:44 smithi042 ceph-mon[18652]: Manager daemon x is now available 2024-08-30T08:50:46.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:50:45 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-30T08:50:46.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:50:45 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"} : dispatch 2024-08-30T08:50:46.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:50:45 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"} : dispatch 2024-08-30T08:50:46.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:50:45 smithi042 ceph-mon[18652]: mgrmap e11: x(active, since 1.00768s) 2024-08-30T08:50:46.879 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Aug 30 08:50:46 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mgr-x[18896]: 2024-08-30T08:50:46.591+0000 7f1adac29640 -1 log_channel(cephadm) log [ERR] : [30/Aug/2024:08:50:46] ENGINE Error in HTTPServer.serve 2024-08-30T08:50:46.879 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Aug 30 08:50:46 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mgr-x[18896]: Traceback (most recent call last): 2024-08-30T08:50:46.879 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Aug 30 08:50:46 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mgr-x[18896]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-08-30T08:50:46.879 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Aug 30 08:50:46 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mgr-x[18896]: self._connections.run(self.expiration_interval) 2024-08-30T08:50:46.879 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Aug 30 08:50:46 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mgr-x[18896]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-08-30T08:50:46.879 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Aug 30 08:50:46 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mgr-x[18896]: self._run(expiration_interval) 2024-08-30T08:50:46.880 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Aug 30 08:50:46 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mgr-x[18896]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-08-30T08:50:46.880 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Aug 30 08:50:46 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mgr-x[18896]: new_conn = self._from_server_socket(self.server.socket) 2024-08-30T08:50:46.880 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Aug 30 08:50:46 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mgr-x[18896]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-08-30T08:50:46.880 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Aug 30 08:50:46 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mgr-x[18896]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-08-30T08:50:46.880 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Aug 30 08:50:46 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mgr-x[18896]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-08-30T08:50:46.880 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Aug 30 08:50:46 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mgr-x[18896]: s = self.context.wrap_socket( 2024-08-30T08:50:46.880 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Aug 30 08:50:46 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mgr-x[18896]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-08-30T08:50:46.880 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Aug 30 08:50:46 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mgr-x[18896]: return self.sslsocket_class._create( 2024-08-30T08:50:46.880 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Aug 30 08:50:46 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mgr-x[18896]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-08-30T08:50:46.880 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Aug 30 08:50:46 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mgr-x[18896]: self.do_handshake() 2024-08-30T08:50:46.880 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Aug 30 08:50:46 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mgr-x[18896]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-08-30T08:50:46.880 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Aug 30 08:50:46 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mgr-x[18896]: self._sslobj.do_handshake() 2024-08-30T08:50:46.880 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Aug 30 08:50:46 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mgr-x[18896]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-08-30T08:50:46.881 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Aug 30 08:50:46 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mgr-x[18896]: 2024-08-30T08:50:46.998 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout { 2024-08-30T08:50:46.999 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-08-30T08:50:46.999 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "initialized": true 2024-08-30T08:50:46.999 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout } 2024-08-30T08:50:46.999 INFO:teuthology.orchestra.run.smithi042.stdout:mgr epoch 9 is available 2024-08-30T08:50:46.999 INFO:teuthology.orchestra.run.smithi042.stdout:Using certmgr to generate dashboard self-signed certificate... 2024-08-30T08:50:47.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:50:46 smithi042 ceph-mon[18652]: [30/Aug/2024:08:50:46] ENGINE Bus STARTING 2024-08-30T08:50:47.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:50:46 smithi042 ceph-mon[18652]: [30/Aug/2024:08:50:46] ENGINE Serving on http://172.21.15.42:8765 2024-08-30T08:50:47.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:50:46 smithi042 ceph-mon[18652]: [30/Aug/2024:08:50:46] ENGINE Serving on https://172.21.15.42:7150 2024-08-30T08:50:47.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:50:46 smithi042 ceph-mon[18652]: [30/Aug/2024:08:50:46] ENGINE Bus STARTED 2024-08-30T08:50:47.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:50:46 smithi042 ceph-mon[18652]: [30/Aug/2024:08:50:46] ENGINE Error in HTTPServer.serve 2024-08-30T08:50:47.133 INFO:journalctl@ceph.mon.a.smithi042.stdout: Traceback (most recent call last): 2024-08-30T08:50:47.133 INFO:journalctl@ceph.mon.a.smithi042.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-08-30T08:50:47.133 INFO:journalctl@ceph.mon.a.smithi042.stdout: self._connections.run(self.expiration_interval) 2024-08-30T08:50:47.133 INFO:journalctl@ceph.mon.a.smithi042.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-08-30T08:50:47.134 INFO:journalctl@ceph.mon.a.smithi042.stdout: self._run(expiration_interval) 2024-08-30T08:50:47.134 INFO:journalctl@ceph.mon.a.smithi042.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-08-30T08:50:47.134 INFO:journalctl@ceph.mon.a.smithi042.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-08-30T08:50:47.134 INFO:journalctl@ceph.mon.a.smithi042.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-08-30T08:50:47.134 INFO:journalctl@ceph.mon.a.smithi042.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-08-30T08:50:47.134 INFO:journalctl@ceph.mon.a.smithi042.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-08-30T08:50:47.134 INFO:journalctl@ceph.mon.a.smithi042.stdout: s = self.context.wrap_socket( 2024-08-30T08:50:47.134 INFO:journalctl@ceph.mon.a.smithi042.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-08-30T08:50:47.134 INFO:journalctl@ceph.mon.a.smithi042.stdout: return self.sslsocket_class._create( 2024-08-30T08:50:47.134 INFO:journalctl@ceph.mon.a.smithi042.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-08-30T08:50:47.134 INFO:journalctl@ceph.mon.a.smithi042.stdout: self.do_handshake() 2024-08-30T08:50:47.134 INFO:journalctl@ceph.mon.a.smithi042.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-08-30T08:50:47.134 INFO:journalctl@ceph.mon.a.smithi042.stdout: self._sslobj.do_handshake() 2024-08-30T08:50:47.134 INFO:journalctl@ceph.mon.a.smithi042.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-08-30T08:50:48.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:50:47 smithi042 ceph-mon[18652]: mgrmap e12: x(active, since 2s) 2024-08-30T08:50:48.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:50:47 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:50:48.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:50:47 smithi042 ceph-mon[18652]: from='client.14162 -' entity='client.admin' cmd=[{"prefix": "orch certmgr generate-certificates", "module_name": "dashboard", "target": ["mon-mgr", ""]}]: dispatch 2024-08-30T08:50:49.196 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:50:49 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:50:49.199 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout {"cert": "-----BEGIN CERTIFICATE-----\nMIIE+zCCAuOgAwIBAgIUEpJqib0Xs5vFmMt+wGaNFIAhlB4wDQYJKoZIhvcNAQEL\nBQAwFzEVMBMGA1UEAwwMY2VwaGFkbS1yb290MB4XDTI0MDgzMDA4NTA0OFoXDTM0\nMDgzMTA4NTA0OFowFzEVMBMGA1UEAwwMMTcyLjIxLjE1LjQyMIICIjANBgkqhkiG\n9w0BAQEFAAOCAg8AMIICCgKCAgEAr7wvn+txR1U74aD56Zywk8cTUORwkK11g8aS\npDUSZcX1+WHRehopJPewCpvv4O5E3Espxpp9O1+G1vrPDnu/wNglbU9PPiavk5yW\nLkLti8dvGz9kVZjLQjHHVmw+S62YeW2MxGQi+hOwPEDBbYJE3vXskRXQ6wjUGLn5\nwUjclgNmFAllMC9dKP0qd0+r0unRVmCQgMFhxdVB130DgRpgU6nh7bldT9sskfc4\nLHUoxG6ZoIwKfJLmfrTCVwrftmNurf1RYmsjmqbl0Q8sQb7jZARybQCXjRjwzaNq\nr73edlKfqIihOIEeq06rEtU2fLaVn8cjN2kGqvjRsukufCTL9UaZSHQFNTfhiiUB\noYrudb9Wt3PM00GcIUMBSWIQlE1OzmuM6vUxZYYkmkPXomHt3nZ9Dps3YiMi3Lst\nvXROdmKjzU46wFe7tKLBdhap9Ow4FsQ4hJlcRAORilchw29UyjOPEd4vmEaMlipl\nGlFD/2TDUDU3N4N/e4EL+TOHxY/pH/+596net8W8Ac8GS1eHYYRJ0rKNZRBEHItp\nzu5+8wN9CHb/J0tHFBTHIDIMdPNYJdsRnRcSIeN2bc4goIcBAjITFNfhMibEbckN\ntSb+2v951qrywIky1pOetb/scJJHCjVjj50Yk8pzh9pCy6lVEJQzhM5DGq0Uvk7v\njELAY3ECAwEAAaM/MD0wLQYDVR0RBCYwJIIJc21pdGhpMDQyghFkYXNoYm9hcmRf\nc2VydmVyc4cErBUPKjAMBgNVHRMBAf8EAjAAMA0GCSqGSIb3DQEBCwUAA4ICAQBu\nPcHDzAXgx4S9AY7jYGI8T5tZ429H1nGT5lXTgotwCQRVyD0VoNHOTlEt39bHuO7N\nTnfpigWvEaqkGR9WVTfF7Iw85RTilXZElnpCFoOEn+JM4X06kOecAWEiJPyq77Xq\nlcbj8fuXj+F3ZnH9ZwS457XkIv7PwrJhgqyZmhSIhPyDteqztUJVu+ka1ajGo7fP\nS7MCDFh1Wmdixi2ellh0g2A5c562ZEGrZr5vMaSSP3kcGScABLi6R/ziCUg8KMp5\nWddRM5vt5cHq9zke64X75xsdP6c37hvriGD5xzftamV3b9PCccBPNvvcew1xyeQu\nV9hhb+UbUmlfg/v84DoApGD9rITlYO6lglQ87Pw2nAvTCl6QRbQvs/CBD2Z8Q2oz\ntf3JWACaIn3/gBzG5Cy8Ni8PMyO2v4DIXpQqck7lo/Yv74nOuDZZYFDzcngcbP8M\nI2r+iP7akBgMkD0BpqU7jEL22bPqwYfwIjbYO88ZGEOi3iOxsv42OLDK5hh7wMHw\nZxcEsn+QEIQVaWvh/0W5XB8qmXYgdeyqmVkkAN7TdZdHbWY+sfoF2o9xXfPErhRc\na4jHNm5Xejg834IF6YytB0OWScERCz+a5DaDdyjMDBezqvts729C6yxO2psjUgNV\nI5smTH2UY0AMRM6I1K/HKzjgMV45LN8xImRK/+mV/g==\n-----END CERTIFICATE-----\n", "key": "-----BEGIN RSA PRIVATE KEY-----\nMIIJKAIBAAKCAgEAr7wvn+txR1U74aD56Zywk8cTUORwkK11g8aSpDUSZcX1+WHR\nehopJPewCpvv4O5E3Espxpp9O1+G1vrPDnu/wNglbU9PPiavk5yWLkLti8dvGz9k\nVZjLQjHHVmw+S62YeW2MxGQi+hOwPEDBbYJE3vXskRXQ6wjUGLn5wUjclgNmFAll\nMC9dKP0qd0+r0unRVmCQgMFhxdVB130DgRpgU6nh7bldT9sskfc4LHUoxG6ZoIwK\nfJLmfrTCVwrftmNurf1RYmsjmqbl0Q8sQb7jZARybQCXjRjwzaNqr73edlKfqIih\nOIEeq06rEtU2fLaVn8cjN2kGqvjRsukufCTL9UaZSHQFNTfhiiUBoYrudb9Wt3PM\n00GcIUMBSWIQlE1OzmuM6vUxZYYkmkPXomHt3nZ9Dps3YiMi3LstvXROdmKjzU46\nwFe7tKLBdhap9Ow4FsQ4hJlcRAORilchw29UyjOPEd4vmEaMliplGlFD/2TDUDU3\nN4N/e4EL+TOHxY/pH/+596net8W8Ac8GS1eHYYRJ0rKNZRBEHItpzu5+8wN9CHb/\nJ0tHFBTHIDIMdPNYJdsRnRcSIeN2bc4goIcBAjITFNfhMibEbckNtSb+2v951qry\nwIky1pOetb/scJJHCjVjj50Yk8pzh9pCy6lVEJQzhM5DGq0Uvk7vjELAY3ECAwEA\nAQKCAgALBGKddVybjtzDUUzdoBLDXveHapziexdrEJ3EoOWjfwDD1sWz0JRMAqG1\nFo/YM4PaZjVrI6gOeveIfn6Q30H3IG3fP5m9kR4QhGB3a7959AGzjPq4sYyrnJSN\nRtm0c6CDm1I4BTmaoTWbH2mo+IHiEvuW4G3hsELgvncJjqxylKm3f5vKBKwj29RH\n2NUyMrGqp0Dj64u6Q/t925Lmyv0nPZciCM0QarKNf5jUU69/uhnUQndNVkXQQ3uM\nCqzaAbuQeZk6sPIRWd1+SOPaxtF5ZZx5WDBIGpxNHg28EQyTcSxce91UniaqsXLA\n6mBBOw6gDtNWv9U4zoLSAzOkeYeWQTNxfIY8kEDEs7nIcmHMz3qzmcrQ17EdsRZe\ny7RHVzPLK3D5yUYhCMIXj3UywTDYeyx/OqorbiYLMwRlwE62DlwHSqANekrkLKhd\nhlcGj8mY2tDGreT8Boppe6Jc2fgSGx0vLAQaZ9zZLwJ048mVU4MmgxBCMMhOBibF\nzWSQCjbTwMgXIOVoskOURibQPA0HqKq0cM5JBehET+9hQm/QIHIw1aYI4XIv0OR3\n7AOAFvRl4EACvAHBIMgmzb1T6hgSdanZCTLReUyhllCN9/yZVVXQLqCIH3Oq2rYW\nvJ5cnX4j1TI0e3Zz60I9pkQ0gYkJdcRkVtanSCH2CAJd3UtCyQKCAQEA1d8tJ+/w\ndwbavzAGznwfwd9+ctbFoJis454VN0lFerfPLwXQYoLqPWLGEbxPdDvEmnNFsONE\ngSAZV5lHbqpi0TtM/D5qnNu5CbJPNuyUyNsgtdpO+2w1yqM9XnVImww2VYJYPVOy\nJnoVdr0Z/1bZe3sZ2uvgxZqwlOJy2Qse7scamueOXPJsuprnvsrkXOblQ576D250\nNdh7sCF67SjU8CaZPhsU8gNY7/k3T4cqcuTrUJUcCg/SfJfMMeX0oOXSQrRb5zRr\nEDxDcOjyD3NL347WKUA7+Xhv0njj3ITRZ0FJ6l4eOJGQqT7IiLlKuq1KbF3B3lEa\nKXZLl44dSIcyaQKCAQEA0lnoNOn7gNHnDAC2zajN7lr0B3w9lwv7gYFKTB/ORlOG\nhgTJwnQkYyEAAL4jDz4GstijP3aO74QF1C4QhnMKcXpX03qmabAj4H4mXtpE6dXQ\na3aoaKF/fSd3fGMFBe6loRQxKVXPLrjgrjIQ/N6xf5v34Ctf5gM2BFuvg1yPB1OC\n3hM9laJR7uzym6v5+M+vpvZPX0LfPT0EH7k/yjhyeJlz2c8wW+SUCGXGBlBKqwnS\nbEYoLtnJOCsB+e6hb5zmuV5p/jpqkadVkOe0vDaLFihb0JrVQFJp6JmZXLBYC3iQ\nfV0dXCapDOY2wS28+20CmUaeL0IpVtPn6WGc23N3yQKCAQEAx6MOrKPta73LPQHg\nqKf2oOs1Ro8yZMl/QO1UUoojv/yJ+hPMQ9h+PJTR3MZRyGbuC1okqIxMatgCr+FV\na3jCyxo2r5K3w/naDLIT/WnzLFZXf0jrxwdoYuHjrCOAhbJtmX4VThGtTvtQw8ia\n1t1EH152cQMTioDt6dxDZlRaKHtUW1pgamSkl793KYKMjie9xFRGiVcKRSTxuCRY\ncb5WnmD5VJ7E+2+iekbAmYzLU/pZqZu2U7IXQdQUeFN1+SbuEPP4gaZNdhc6wM/A\n/Ckcnu7e/ByIn0ML5oSP9k1IhsDL9WirfxRp24HOgMmtdvnBacxO5m4AJMnWn/65\nWPVSWQKCAQBXWp7j2odnZ8PbaDCCF6ftZDgTI7UwFPnNPjPsiQjQi79ByJtA9Zpm\n0D2AEXXdzpRZkT26DTIupVUGWFrkesHLflMar8SL/2nmwHyOx3X25aQ3/H7TdL3M\n66bjMPvVCtYdnpUL5Wa0sHId5QzI9nIf3FWD8n9QteSoyqqIkteS40tljff1PANq\nhVv66eBVQ2pem8s6dDrg44Xi5rtm7yoaix9hRHmAr9wJ4LnrxUw820dXT4tAaZD2\nkmWvtTm1cCMgt+SsfvXipusR0idsQfpgqUH3z1Tf8VmqDp9xronhuZ/Hb/PRYPyz\nLqduceAZSZvaq0jlt0QTmsykTrUlLT6BAoIBAD6pv8axRShXbyz8QdoPoAfgK+pV\nOTx8crqcyM8e1fRHopUFt4hd/gcscJBFOdH2dCPwv5JvXEhkZP7XpvIpPuBuml75\n8hmcTZKuC/0p+dPBYiG13tIqursGinUI4CCzTugHW0ISMpQgt0jmcNFzpD15VKVi\naWZy2voxI4MhWAuC65twt44Z+w7Nnh+z86oQQiURTl+WRhvzK5W7ceH/sx0Gvesp\nwJUWoq/irt7AKM/alKVeLs9jYVXuC+K/LIH/Du3GhRkGNin9p61dOHmgbHcZxcsk\nNowaiUSJ70OJxB9/DXRR9erbGfoYNqP/T3TUduYLA/BgnlGXyVA9pQNpjak=\n-----END RSA PRIVATE KEY-----\n"} 2024-08-30T08:50:51.125 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:50:50 smithi042 ceph-mon[18652]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard set-ssl-certificate", "target": ["mon-mgr", ""]}]: dispatch 2024-08-30T08:50:51.125 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:50:50 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:50:51.128 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout SSL certificate updated 2024-08-30T08:50:52.360 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout SSL certificate key updated 2024-08-30T08:50:52.360 INFO:teuthology.orchestra.run.smithi042.stdout:Creating initial admin user... 2024-08-30T08:50:53.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:50:52 smithi042 ceph-mon[18652]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard set-ssl-certificate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-08-30T08:50:53.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:50:52 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:50:53.841 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$qU2Wwdn81/wQmm9RM9M9L.Nxbq.Ohd.eYLwkO/lL12Z8nBsjd.wve", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1725007853, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-08-30T08:50:53.841 INFO:teuthology.orchestra.run.smithi042.stdout:Fetching dashboard port number... 2024-08-30T08:50:54.447 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:50:54 smithi042 ceph-mon[18652]: from='client.14168 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-08-30T08:50:54.447 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:50:54 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:50:55.015 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout 8443 2024-08-30T08:50:55.026 INFO:teuthology.orchestra.run.smithi042.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-08-30T08:50:55.026 INFO:teuthology.orchestra.run.smithi042.stdout:systemctl: stdout disabled 2024-08-30T08:50:55.034 INFO:teuthology.orchestra.run.smithi042.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-08-30T08:50:55.034 INFO:teuthology.orchestra.run.smithi042.stdout:systemctl: stdout inactive 2024-08-30T08:50:55.034 INFO:teuthology.orchestra.run.smithi042.stdout:firewalld.service is not enabled 2024-08-30T08:50:55.034 INFO:teuthology.orchestra.run.smithi042.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-08-30T08:50:55.035 INFO:teuthology.orchestra.run.smithi042.stdout:Ceph Dashboard is now available at: 2024-08-30T08:50:55.036 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-08-30T08:50:55.036 INFO:teuthology.orchestra.run.smithi042.stdout: URL: https://smithi042:8443/ 2024-08-30T08:50:55.036 INFO:teuthology.orchestra.run.smithi042.stdout: User: admin 2024-08-30T08:50:55.036 INFO:teuthology.orchestra.run.smithi042.stdout: Password: naziren2n7 2024-08-30T08:50:55.036 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-08-30T08:50:55.036 INFO:teuthology.orchestra.run.smithi042.stdout:Saving cluster configuration to /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/config directory 2024-08-30T08:50:55.632 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:50:55 smithi042 ceph-mon[18652]: from='client.? 172.21.15.42:0/570680088' entity='client.admin' cmd={"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} : dispatch 2024-08-30T08:50:56.347 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-08-30T08:50:56.347 INFO:teuthology.orchestra.run.smithi042.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-08-30T08:50:56.347 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-08-30T08:50:56.348 INFO:teuthology.orchestra.run.smithi042.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-08-30T08:50:56.348 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-08-30T08:50:56.348 INFO:teuthology.orchestra.run.smithi042.stdout:Or, if you are only running a single cluster on this host: 2024-08-30T08:50:56.348 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-08-30T08:50:56.348 INFO:teuthology.orchestra.run.smithi042.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-08-30T08:50:56.348 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-08-30T08:50:56.348 INFO:teuthology.orchestra.run.smithi042.stdout:Please consider enabling telemetry to help improve Ceph: 2024-08-30T08:50:56.348 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-08-30T08:50:56.348 INFO:teuthology.orchestra.run.smithi042.stdout: ceph telemetry on 2024-08-30T08:50:56.348 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-08-30T08:50:56.349 INFO:teuthology.orchestra.run.smithi042.stdout:For more information see: 2024-08-30T08:50:56.349 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-08-30T08:50:56.349 INFO:teuthology.orchestra.run.smithi042.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-08-30T08:50:56.349 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-08-30T08:50:56.349 INFO:teuthology.orchestra.run.smithi042.stdout:Bootstrap complete. 2024-08-30T08:50:56.388 INFO:tasks.cephadm:Fetching config... 2024-08-30T08:50:56.389 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-08-30T08:50:56.389 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-08-30T08:50:56.405 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-08-30T08:50:56.406 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-08-30T08:50:56.406 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-08-30T08:50:56.461 INFO:tasks.cephadm:Fetching mon keyring... 2024-08-30T08:50:56.461 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-08-30T08:50:56.462 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd if=/var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/keyring of=/dev/stdout 2024-08-30T08:50:56.526 INFO:tasks.cephadm:Fetching pub ssh key... 2024-08-30T08:50:56.527 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-08-30T08:50:56.527 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-08-30T08:50:56.581 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-08-30T08:50:56.582 DEBUG:teuthology.orchestra.run.smithi042:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCwFGca4P/Kl1WAiOKsCR2aXY0VXBnEMh41bZBwTE4rQRFkb5+cISrqz1pDpQEVyQLM7ndRmhSy82Mu6eg9chBZpfudkgdmSqTrPsHKkK0uOVYUaHOmDT8fYycofSI7yEVP+4Lo0+ChNrnp4+avwV+eq/lHewznjUAEC7V6qZ6fS9IUO10IF5FkgoDAHafwT2rLml/mWqMuhB5pHTwlFytjWwtQQsUZoz6sIpk5R5exRWfiMCl+DSXelChr48vYVfdTwj08J65AR3MzrLMRhq3ouLHBfMxboT4CFBoGkBeBVdtxow2YamdlYcOz7WNHi2AwZiMUplkfEgm7VWHgzuAV/H/YpG65TDiWqbSd3yNxDrxTK4guhQf+NUCV2yazO0uCtPcmjg0JMiic+5TEyMu417vJJuS2xX8SC+VOJF3snj7vB8+tznbPrhgT1E+jUWuzjnleglf3x3UhJM4u7p2atZlLv52TBm5SXTQua3blYI11sLsNirlPoNcB7kIjRbs= ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-08-30T08:50:56.654 INFO:teuthology.orchestra.run.smithi042.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCwFGca4P/Kl1WAiOKsCR2aXY0VXBnEMh41bZBwTE4rQRFkb5+cISrqz1pDpQEVyQLM7ndRmhSy82Mu6eg9chBZpfudkgdmSqTrPsHKkK0uOVYUaHOmDT8fYycofSI7yEVP+4Lo0+ChNrnp4+avwV+eq/lHewznjUAEC7V6qZ6fS9IUO10IF5FkgoDAHafwT2rLml/mWqMuhB5pHTwlFytjWwtQQsUZoz6sIpk5R5exRWfiMCl+DSXelChr48vYVfdTwj08J65AR3MzrLMRhq3ouLHBfMxboT4CFBoGkBeBVdtxow2YamdlYcOz7WNHi2AwZiMUplkfEgm7VWHgzuAV/H/YpG65TDiWqbSd3yNxDrxTK4guhQf+NUCV2yazO0uCtPcmjg0JMiic+5TEyMu417vJJuS2xX8SC+VOJF3snj7vB8+tznbPrhgT1E+jUWuzjnleglf3x3UhJM4u7p2atZlLv52TBm5SXTQua3blYI11sLsNirlPoNcB7kIjRbs= ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968 2024-08-30T08:50:56.666 DEBUG:teuthology.orchestra.run.smithi167:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCwFGca4P/Kl1WAiOKsCR2aXY0VXBnEMh41bZBwTE4rQRFkb5+cISrqz1pDpQEVyQLM7ndRmhSy82Mu6eg9chBZpfudkgdmSqTrPsHKkK0uOVYUaHOmDT8fYycofSI7yEVP+4Lo0+ChNrnp4+avwV+eq/lHewznjUAEC7V6qZ6fS9IUO10IF5FkgoDAHafwT2rLml/mWqMuhB5pHTwlFytjWwtQQsUZoz6sIpk5R5exRWfiMCl+DSXelChr48vYVfdTwj08J65AR3MzrLMRhq3ouLHBfMxboT4CFBoGkBeBVdtxow2YamdlYcOz7WNHi2AwZiMUplkfEgm7VWHgzuAV/H/YpG65TDiWqbSd3yNxDrxTK4guhQf+NUCV2yazO0uCtPcmjg0JMiic+5TEyMu417vJJuS2xX8SC+VOJF3snj7vB8+tznbPrhgT1E+jUWuzjnleglf3x3UhJM4u7p2atZlLv52TBm5SXTQua3blYI11sLsNirlPoNcB7kIjRbs= ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-08-30T08:50:56.709 INFO:teuthology.orchestra.run.smithi167.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCwFGca4P/Kl1WAiOKsCR2aXY0VXBnEMh41bZBwTE4rQRFkb5+cISrqz1pDpQEVyQLM7ndRmhSy82Mu6eg9chBZpfudkgdmSqTrPsHKkK0uOVYUaHOmDT8fYycofSI7yEVP+4Lo0+ChNrnp4+avwV+eq/lHewznjUAEC7V6qZ6fS9IUO10IF5FkgoDAHafwT2rLml/mWqMuhB5pHTwlFytjWwtQQsUZoz6sIpk5R5exRWfiMCl+DSXelChr48vYVfdTwj08J65AR3MzrLMRhq3ouLHBfMxboT4CFBoGkBeBVdtxow2YamdlYcOz7WNHi2AwZiMUplkfEgm7VWHgzuAV/H/YpG65TDiWqbSd3yNxDrxTK4guhQf+NUCV2yazO0uCtPcmjg0JMiic+5TEyMu417vJJuS2xX8SC+VOJF3snj7vB8+tznbPrhgT1E+jUWuzjnleglf3x3UhJM4u7p2atZlLv52TBm5SXTQua3blYI11sLsNirlPoNcB7kIjRbs= ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968 2024-08-30T08:50:56.722 DEBUG:teuthology.orchestra.run.smithi190:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCwFGca4P/Kl1WAiOKsCR2aXY0VXBnEMh41bZBwTE4rQRFkb5+cISrqz1pDpQEVyQLM7ndRmhSy82Mu6eg9chBZpfudkgdmSqTrPsHKkK0uOVYUaHOmDT8fYycofSI7yEVP+4Lo0+ChNrnp4+avwV+eq/lHewznjUAEC7V6qZ6fS9IUO10IF5FkgoDAHafwT2rLml/mWqMuhB5pHTwlFytjWwtQQsUZoz6sIpk5R5exRWfiMCl+DSXelChr48vYVfdTwj08J65AR3MzrLMRhq3ouLHBfMxboT4CFBoGkBeBVdtxow2YamdlYcOz7WNHi2AwZiMUplkfEgm7VWHgzuAV/H/YpG65TDiWqbSd3yNxDrxTK4guhQf+NUCV2yazO0uCtPcmjg0JMiic+5TEyMu417vJJuS2xX8SC+VOJF3snj7vB8+tznbPrhgT1E+jUWuzjnleglf3x3UhJM4u7p2atZlLv52TBm5SXTQua3blYI11sLsNirlPoNcB7kIjRbs= ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-08-30T08:50:56.765 INFO:teuthology.orchestra.run.smithi190.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCwFGca4P/Kl1WAiOKsCR2aXY0VXBnEMh41bZBwTE4rQRFkb5+cISrqz1pDpQEVyQLM7ndRmhSy82Mu6eg9chBZpfudkgdmSqTrPsHKkK0uOVYUaHOmDT8fYycofSI7yEVP+4Lo0+ChNrnp4+avwV+eq/lHewznjUAEC7V6qZ6fS9IUO10IF5FkgoDAHafwT2rLml/mWqMuhB5pHTwlFytjWwtQQsUZoz6sIpk5R5exRWfiMCl+DSXelChr48vYVfdTwj08J65AR3MzrLMRhq3ouLHBfMxboT4CFBoGkBeBVdtxow2YamdlYcOz7WNHi2AwZiMUplkfEgm7VWHgzuAV/H/YpG65TDiWqbSd3yNxDrxTK4guhQf+NUCV2yazO0uCtPcmjg0JMiic+5TEyMu417vJJuS2xX8SC+VOJF3snj7vB8+tznbPrhgT1E+jUWuzjnleglf3x3UhJM4u7p2atZlLv52TBm5SXTQua3blYI11sLsNirlPoNcB7kIjRbs= ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968 2024-08-30T08:50:56.778 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-08-30T08:50:56.898 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:50:56 smithi042 ceph-mon[18652]: from='client.? 172.21.15.42:0/3718472343' entity='client.admin' 2024-08-30T08:50:57.030 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T08:50:59.012 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:50:58 smithi042 ceph-mon[18652]: from='client.? 172.21.15.42:0/4054284668' entity='client.admin' 2024-08-30T08:50:59.012 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:50:58 smithi042 ceph-mon[18652]: mgrmap e13: x(active, since 12s) 2024-08-30T08:50:59.013 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:50:58 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:50:59.013 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:50:58 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:50:59.013 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:50:58 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi042", "name": "osd_memory_target"} : dispatch 2024-08-30T08:50:59.013 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:50:58 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:50:59.013 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:50:58 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-30T08:50:59.013 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:50:58 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:50:59.042 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-08-30T08:50:59.042 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-08-30T08:50:59.326 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T08:51:01.237 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:01 smithi042 ceph-mon[18652]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-08-30T08:51:01.237 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:01 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:01.237 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:01 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-30T08:51:01.237 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:01 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:51:01.237 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:01 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-30T08:51:01.237 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:01 smithi042 ceph-mon[18652]: Updating smithi042:/etc/ceph/ceph.conf 2024-08-30T08:51:01.237 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:01 smithi042 ceph-mon[18652]: Updating smithi042:/var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/config/ceph.conf 2024-08-30T08:51:01.238 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:01 smithi042 ceph-mon[18652]: Updating smithi042:/etc/ceph/ceph.client.admin.keyring 2024-08-30T08:51:01.238 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:01 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:01.238 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:01 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:01.238 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:01 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:01.266 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi167 2024-08-30T08:51:01.266 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-08-30T08:51:01.266 DEBUG:teuthology.orchestra.run.smithi167:> dd of=/etc/ceph/ceph.conf 2024-08-30T08:51:01.285 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-08-30T08:51:01.285 DEBUG:teuthology.orchestra.run.smithi167:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-08-30T08:51:01.343 INFO:tasks.cephadm:Adding host smithi167 to orchestrator... 2024-08-30T08:51:01.343 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph orch host add smithi167 2024-08-30T08:51:01.590 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T08:51:02.382 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:02 smithi042 ceph-mon[18652]: Updating smithi042:/var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/config/ceph.client.admin.keyring 2024-08-30T08:51:03.382 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:03 smithi042 ceph-mon[18652]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi167", "target": ["mon-mgr", ""]}]: dispatch 2024-08-30T08:51:04.382 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:04 smithi042 ceph-mon[18652]: Deploying cephadm binary to smithi167 2024-08-30T08:51:06.114 INFO:teuthology.orchestra.run.smithi042.stdout:Added host 'smithi167' with addr '172.21.15.167' 2024-08-30T08:51:06.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:06 smithi042 ceph-mon[18652]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-30T08:51:06.706 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph orch host ls --format=json 2024-08-30T08:51:06.952 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T08:51:07.326 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:07 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:07.326 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:07 smithi042 ceph-mon[18652]: Added host smithi167 2024-08-30T08:51:07.326 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:07 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-30T08:51:07.326 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:07 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:07.326 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:07 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:07.612 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-08-30T08:51:07.612 INFO:teuthology.orchestra.run.smithi042.stdout:[{"addr": "172.21.15.42", "hostname": "smithi042", "labels": [], "status": ""}, {"addr": "172.21.15.167", "hostname": "smithi167", "labels": [], "status": ""}] 2024-08-30T08:51:08.310 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi190 2024-08-30T08:51:08.310 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-08-30T08:51:08.310 DEBUG:teuthology.orchestra.run.smithi190:> dd of=/etc/ceph/ceph.conf 2024-08-30T08:51:08.329 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-08-30T08:51:08.330 DEBUG:teuthology.orchestra.run.smithi190:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-08-30T08:51:08.388 INFO:tasks.cephadm:Adding host smithi190 to orchestrator... 2024-08-30T08:51:08.388 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph orch host add smithi190 2024-08-30T08:51:08.632 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:08 smithi042 ceph-mon[18652]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-30T08:51:08.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:08 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:08.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:08 smithi042 ceph-mon[18652]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-30T08:51:08.636 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T08:51:10.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:10 smithi042 ceph-mon[18652]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-30T08:51:10.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:10 smithi042 ceph-mon[18652]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi190", "target": ["mon-mgr", ""]}]: dispatch 2024-08-30T08:51:10.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:10 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:10.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:10 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:10.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:10 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:10.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:10 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:10.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:10 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi167", "name": "osd_memory_target"} : dispatch 2024-08-30T08:51:10.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:10 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:51:10.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:10 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-30T08:51:11.882 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:11 smithi042 ceph-mon[18652]: Updating smithi167:/etc/ceph/ceph.conf 2024-08-30T08:51:11.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:11 smithi042 ceph-mon[18652]: Deploying cephadm binary to smithi190 2024-08-30T08:51:11.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:11 smithi042 ceph-mon[18652]: Updating smithi167:/var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/config/ceph.conf 2024-08-30T08:51:11.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:11 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:11.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:11 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:11.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:11 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:12.882 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:12 smithi042 ceph-mon[18652]: Updating smithi167:/etc/ceph/ceph.client.admin.keyring 2024-08-30T08:51:12.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:12 smithi042 ceph-mon[18652]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-30T08:51:12.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:12 smithi042 ceph-mon[18652]: Updating smithi167:/var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/config/ceph.client.admin.keyring 2024-08-30T08:51:13.229 INFO:teuthology.orchestra.run.smithi042.stdout:Added host 'smithi190' with addr '172.21.15.190' 2024-08-30T08:51:13.931 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph orch host ls --format=json 2024-08-30T08:51:14.177 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T08:51:14.481 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:14 smithi042 ceph-mon[18652]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-30T08:51:14.481 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:14 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:14.481 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:14 smithi042 ceph-mon[18652]: Added host smithi190 2024-08-30T08:51:14.482 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:14 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-30T08:51:14.482 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:14 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:14.482 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:14 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:14.901 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-08-30T08:51:14.902 INFO:teuthology.orchestra.run.smithi042.stdout:[{"addr": "172.21.15.42", "hostname": "smithi042", "labels": [], "status": ""}, {"addr": "172.21.15.167", "hostname": "smithi167", "labels": [], "status": ""}, {"addr": "172.21.15.190", "hostname": "smithi190", "labels": [], "status": ""}] 2024-08-30T08:51:15.467 INFO:tasks.cephadm:Setting crush tunables to default 2024-08-30T08:51:15.468 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph osd crush tunables default 2024-08-30T08:51:15.714 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T08:51:15.823 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:15 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:16.575 INFO:teuthology.orchestra.run.smithi042.stderr:adjusted tunables profile to default 2024-08-30T08:51:16.878 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:16 smithi042 ceph-mon[18652]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-30T08:51:16.879 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:16 smithi042 ceph-mon[18652]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-30T08:51:16.879 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:16 smithi042 ceph-mon[18652]: from='client.? 172.21.15.42:0/256350112' entity='client.admin' cmd={"prefix": "osd crush tunables", "profile": "default"} : dispatch 2024-08-30T08:51:17.157 INFO:tasks.cephadm:Adding mon.a on smithi042 2024-08-30T08:51:17.158 INFO:tasks.cephadm:Adding mon.b on smithi167 2024-08-30T08:51:17.158 INFO:tasks.cephadm:Adding mon.c on smithi190 2024-08-30T08:51:17.158 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph orch apply mon '3;smithi042:172.21.15.42=a;smithi167:172.21.15.167=b;smithi190:172.21.15.190=c' 2024-08-30T08:51:17.401 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /etc/ceph/ceph.conf 2024-08-30T08:51:17.444 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /etc/ceph/ceph.conf 2024-08-30T08:51:17.882 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:17 smithi042 ceph-mon[18652]: from='client.? 172.21.15.42:0/256350112' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-08-30T08:51:17.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:17 smithi042 ceph-mon[18652]: osdmap e4: 0 total, 0 up, 0 in 2024-08-30T08:51:17.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:17 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:17.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:17 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:17.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:17 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:17.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:17 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:17.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:17 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi190", "name": "osd_memory_target"} : dispatch 2024-08-30T08:51:17.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:17 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:51:17.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:17 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-30T08:51:18.131 INFO:teuthology.orchestra.run.smithi190.stdout:Scheduled mon update... 2024-08-30T08:51:18.697 DEBUG:teuthology.orchestra.run.smithi167:mon.b> sudo journalctl -f -n 0 -u ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968@mon.b.service 2024-08-30T08:51:18.700 DEBUG:teuthology.orchestra.run.smithi190:mon.c> sudo journalctl -f -n 0 -u ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968@mon.c.service 2024-08-30T08:51:18.702 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-08-30T08:51:18.702 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph mon dump -f json 2024-08-30T08:51:18.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:18 smithi042 ceph-mon[18652]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-30T08:51:18.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:18 smithi042 ceph-mon[18652]: Updating smithi190:/etc/ceph/ceph.conf 2024-08-30T08:51:18.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:18 smithi042 ceph-mon[18652]: Updating smithi190:/var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/config/ceph.conf 2024-08-30T08:51:18.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:18 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:18.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:18 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:18.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:18 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:18.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:18 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:18.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:18 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-08-30T08:51:18.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:18 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:51:19.459 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/config/ceph.conf 2024-08-30T08:51:19.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:19 smithi042 ceph-mon[18652]: Updating smithi190:/etc/ceph/ceph.client.admin.keyring 2024-08-30T08:51:19.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:19 smithi042 ceph-mon[18652]: Updating smithi190:/var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/config/ceph.client.admin.keyring 2024-08-30T08:51:19.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:19 smithi042 ceph-mon[18652]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi042:172.21.15.42=a;smithi167:172.21.15.167=b;smithi190:172.21.15.190=c", "target": ["mon-mgr", ""]}]: dispatch 2024-08-30T08:51:19.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:19 smithi042 ceph-mon[18652]: Saving service mon spec with placement smithi042:172.21.15.42=a;smithi167:172.21.15.167=b;smithi190:172.21.15.190=c;count:3 2024-08-30T08:51:19.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:19 smithi042 ceph-mon[18652]: Deploying daemon mon.c on smithi190 2024-08-30T08:51:20.502 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-08-30T08:51:20.502 INFO:teuthology.orchestra.run.smithi190.stdout:{"epoch":1,"fsid":"b0895c1a-66ac-11ef-bcd4-c7b262605968","modified":"2024-08-30T08:49:41.668361Z","created":"2024-08-30T08:49:41.668361Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:3300","nonce":0},{"type":"v1","addr":"172.21.15.42:6789","nonce":0}]},"addr":"172.21.15.42:6789/0","public_addr":"172.21.15.42:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-08-30T08:51:20.502 INFO:teuthology.orchestra.run.smithi190.stderr:dumped monmap epoch 1 2024-08-30T08:51:20.884 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:20 smithi042 ceph-mon[18652]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-30T08:51:20.884 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:20 smithi042 ceph-mon[18652]: from='client.? 172.21.15.190:0/3954226789' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-08-30T08:51:22.394 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-08-30T08:51:22.395 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph mon dump -f json 2024-08-30T08:51:22.782 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.c/config 2024-08-30T08:51:22.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:22 smithi042 ceph-mon[18652]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-30T08:51:23.630 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-08-30T08:51:23.630 INFO:teuthology.orchestra.run.smithi190.stdout:{"epoch":1,"fsid":"b0895c1a-66ac-11ef-bcd4-c7b262605968","modified":"2024-08-30T08:49:41.668361Z","created":"2024-08-30T08:49:41.668361Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:3300","nonce":0},{"type":"v1","addr":"172.21.15.42:6789","nonce":0}]},"addr":"172.21.15.42:6789/0","public_addr":"172.21.15.42:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-08-30T08:51:23.630 INFO:teuthology.orchestra.run.smithi190.stderr:dumped monmap epoch 1 2024-08-30T08:51:23.773 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 podman[21968]: 2024-08-30 08:51:23.738271851 +0000 UTC m=+0.331180353 container start a523197617ba300e0d1dc684b83cabc78c4c056ba02591ffda49b8e1e7bd9cf9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mon-c, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.1, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2) 2024-08-30T08:51:23.773 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: set uid:gid to 167:167 (ceph:ceph) 2024-08-30T08:51:23.773 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: ceph version 19.3.0-4560-gd19e6118 (d19e6118f7015f0047264006cb0c873528eb3ae6) squid (dev), process ceph-mon, pid 2 2024-08-30T08:51:23.773 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: pidfile_write: ignore empty --pid-file 2024-08-30T08:51:24.089 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: load: jerasure load: lrc 2024-08-30T08:51:24.089 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: RocksDB version: 7.9.2 2024-08-30T08:51:24.089 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: Git sha 0 2024-08-30T08:51:24.089 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: Compile date 2024-08-28 10:19:06 2024-08-30T08:51:24.089 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: DB SUMMARY 2024-08-30T08:51:24.089 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: DB Session ID: SPQZH4S336YZZACDIB4Q 2024-08-30T08:51:24.090 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: CURRENT file: CURRENT 2024-08-30T08:51:24.090 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: IDENTITY file: IDENTITY 2024-08-30T08:51:24.090 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-08-30T08:51:24.090 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-08-30T08:51:24.090 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2024-08-30T08:51:24.090 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: Options.error_if_exists: 0 2024-08-30T08:51:24.090 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: Options.create_if_missing: 0 2024-08-30T08:51:24.090 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: Options.paranoid_checks: 1 2024-08-30T08:51:24.090 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: Options.flush_verify_memtable_count: 1 2024-08-30T08:51:24.090 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-08-30T08:51:24.090 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-08-30T08:51:24.090 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: Options.env: 0x55fcbfe165a0 2024-08-30T08:51:24.090 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: Options.fs: PosixFileSystem 2024-08-30T08:51:24.091 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: Options.info_log: 0x55fcc48b54e0 2024-08-30T08:51:24.091 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: Options.max_file_opening_threads: 16 2024-08-30T08:51:24.091 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: Options.statistics: (nil) 2024-08-30T08:51:24.091 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: Options.use_fsync: 0 2024-08-30T08:51:24.091 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: Options.max_log_file_size: 0 2024-08-30T08:51:24.091 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-08-30T08:51:24.091 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: Options.log_file_time_to_roll: 0 2024-08-30T08:51:24.091 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: Options.keep_log_file_num: 1000 2024-08-30T08:51:24.091 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: Options.recycle_log_file_num: 0 2024-08-30T08:51:24.091 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: Options.allow_fallocate: 1 2024-08-30T08:51:24.091 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: Options.allow_mmap_reads: 0 2024-08-30T08:51:24.091 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: Options.allow_mmap_writes: 0 2024-08-30T08:51:24.091 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: Options.use_direct_reads: 0 2024-08-30T08:51:24.091 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-08-30T08:51:24.092 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: Options.create_missing_column_families: 0 2024-08-30T08:51:24.092 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: Options.db_log_dir: 2024-08-30T08:51:24.092 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: Options.wal_dir: 2024-08-30T08:51:24.092 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: Options.table_cache_numshardbits: 6 2024-08-30T08:51:24.092 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: Options.WAL_ttl_seconds: 0 2024-08-30T08:51:24.092 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: Options.WAL_size_limit_MB: 0 2024-08-30T08:51:24.092 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-08-30T08:51:24.094 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-08-30T08:51:24.094 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: Options.is_fd_close_on_exec: 1 2024-08-30T08:51:24.094 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: Options.advise_random_on_open: 1 2024-08-30T08:51:24.094 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: Options.db_write_buffer_size: 0 2024-08-30T08:51:24.094 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: Options.write_buffer_manager: 0x55fcc48b95e0 2024-08-30T08:51:24.094 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-08-30T08:51:24.094 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-08-30T08:51:24.094 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: Options.use_adaptive_mutex: 0 2024-08-30T08:51:24.094 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: Options.rate_limiter: (nil) 2024-08-30T08:51:24.094 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-08-30T08:51:24.094 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: Options.wal_recovery_mode: 2 2024-08-30T08:51:24.094 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: Options.enable_thread_tracking: 0 2024-08-30T08:51:24.095 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: Options.enable_pipelined_write: 0 2024-08-30T08:51:24.095 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: Options.unordered_write: 0 2024-08-30T08:51:24.095 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-08-30T08:51:24.095 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-08-30T08:51:24.095 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-08-30T08:51:24.095 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-08-30T08:51:24.095 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: Options.row_cache: None 2024-08-30T08:51:24.095 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: Options.wal_filter: None 2024-08-30T08:51:24.095 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-08-30T08:51:24.095 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: Options.allow_ingest_behind: 0 2024-08-30T08:51:24.095 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: Options.two_write_queues: 0 2024-08-30T08:51:24.095 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: Options.manual_wal_flush: 0 2024-08-30T08:51:24.095 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: Options.wal_compression: 0 2024-08-30T08:51:24.096 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: Options.atomic_flush: 0 2024-08-30T08:51:24.096 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-08-30T08:51:24.096 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: Options.persist_stats_to_disk: 0 2024-08-30T08:51:24.096 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: Options.write_dbid_to_manifest: 0 2024-08-30T08:51:24.096 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: Options.log_readahead_size: 0 2024-08-30T08:51:24.096 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-08-30T08:51:24.096 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: Options.best_efforts_recovery: 0 2024-08-30T08:51:24.096 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-08-30T08:51:24.096 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-08-30T08:51:24.096 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: Options.allow_data_in_errors: 0 2024-08-30T08:51:24.096 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: Options.db_host_id: __hostname__ 2024-08-30T08:51:24.096 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: Options.enforce_single_del_contracts: true 2024-08-30T08:51:24.096 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: Options.max_background_jobs: 2 2024-08-30T08:51:24.097 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: Options.max_background_compactions: -1 2024-08-30T08:51:24.097 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: Options.max_subcompactions: 1 2024-08-30T08:51:24.097 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-08-30T08:51:24.097 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-08-30T08:51:24.097 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: Options.delayed_write_rate : 16777216 2024-08-30T08:51:24.097 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: Options.max_total_wal_size: 0 2024-08-30T08:51:24.097 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-08-30T08:51:24.097 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: Options.stats_dump_period_sec: 600 2024-08-30T08:51:24.097 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: Options.stats_persist_period_sec: 600 2024-08-30T08:51:24.097 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-08-30T08:51:24.097 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: Options.max_open_files: -1 2024-08-30T08:51:24.097 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: Options.bytes_per_sync: 0 2024-08-30T08:51:24.097 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: Options.wal_bytes_per_sync: 0 2024-08-30T08:51:24.097 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: Options.strict_bytes_per_sync: 0 2024-08-30T08:51:24.098 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: Options.compaction_readahead_size: 0 2024-08-30T08:51:24.098 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: Options.max_background_flushes: -1 2024-08-30T08:51:24.098 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: Compression algorithms supported: 2024-08-30T08:51:24.098 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: kZSTD supported: 0 2024-08-30T08:51:24.098 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: kXpressCompression supported: 0 2024-08-30T08:51:24.098 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: kBZip2Compression supported: 0 2024-08-30T08:51:24.098 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-08-30T08:51:24.098 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: kLZ4Compression supported: 1 2024-08-30T08:51:24.098 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: kZlibCompression supported: 1 2024-08-30T08:51:24.098 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: kLZ4HCCompression supported: 1 2024-08-30T08:51:24.098 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: kSnappyCompression supported: 1 2024-08-30T08:51:24.098 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: Fast CRC32 supported: Supported on x86 2024-08-30T08:51:24.098 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: DMutex implementation: pthread_mutex_t 2024-08-30T08:51:24.098 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-08-30T08:51:24.098 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-08-30T08:51:24.099 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-08-30T08:51:24.099 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: Options.merge_operator: 2024-08-30T08:51:24.099 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: Options.compaction_filter: None 2024-08-30T08:51:24.099 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: Options.compaction_filter_factory: None 2024-08-30T08:51:24.099 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: Options.sst_partitioner_factory: None 2024-08-30T08:51:24.099 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: Options.memtable_factory: SkipListFactory 2024-08-30T08:51:24.099 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: Options.table_factory: BlockBasedTable 2024-08-30T08:51:24.099 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55fcc48b5100) 2024-08-30T08:51:24.099 INFO:journalctl@ceph.mon.c.smithi190.stdout: cache_index_and_filter_blocks: 1 2024-08-30T08:51:24.099 INFO:journalctl@ceph.mon.c.smithi190.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-08-30T08:51:24.099 INFO:journalctl@ceph.mon.c.smithi190.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-08-30T08:51:24.099 INFO:journalctl@ceph.mon.c.smithi190.stdout: pin_top_level_index_and_filter: 1 2024-08-30T08:51:24.099 INFO:journalctl@ceph.mon.c.smithi190.stdout: index_type: 0 2024-08-30T08:51:24.099 INFO:journalctl@ceph.mon.c.smithi190.stdout: data_block_index_type: 0 2024-08-30T08:51:24.099 INFO:journalctl@ceph.mon.c.smithi190.stdout: index_shortening: 1 2024-08-30T08:51:24.100 INFO:journalctl@ceph.mon.c.smithi190.stdout: data_block_hash_table_util_ratio: 0.750000 2024-08-30T08:51:24.100 INFO:journalctl@ceph.mon.c.smithi190.stdout: checksum: 4 2024-08-30T08:51:24.100 INFO:journalctl@ceph.mon.c.smithi190.stdout: no_block_cache: 0 2024-08-30T08:51:24.100 INFO:journalctl@ceph.mon.c.smithi190.stdout: block_cache: 0x55fcc0878850 2024-08-30T08:51:24.100 INFO:journalctl@ceph.mon.c.smithi190.stdout: block_cache_name: BinnedLRUCache 2024-08-30T08:51:24.100 INFO:journalctl@ceph.mon.c.smithi190.stdout: block_cache_options: 2024-08-30T08:51:24.100 INFO:journalctl@ceph.mon.c.smithi190.stdout: capacity : 536870912 2024-08-30T08:51:24.100 INFO:journalctl@ceph.mon.c.smithi190.stdout: num_shard_bits : 4 2024-08-30T08:51:24.100 INFO:journalctl@ceph.mon.c.smithi190.stdout: strict_capacity_limit : 0 2024-08-30T08:51:24.100 INFO:journalctl@ceph.mon.c.smithi190.stdout: high_pri_pool_ratio: 0.000 2024-08-30T08:51:24.100 INFO:journalctl@ceph.mon.c.smithi190.stdout: block_cache_compressed: (nil) 2024-08-30T08:51:24.100 INFO:journalctl@ceph.mon.c.smithi190.stdout: persistent_cache: (nil) 2024-08-30T08:51:24.100 INFO:journalctl@ceph.mon.c.smithi190.stdout: block_size: 4096 2024-08-30T08:51:24.100 INFO:journalctl@ceph.mon.c.smithi190.stdout: block_size_deviation: 10 2024-08-30T08:51:24.101 INFO:journalctl@ceph.mon.c.smithi190.stdout: block_restart_interval: 16 2024-08-30T08:51:24.101 INFO:journalctl@ceph.mon.c.smithi190.stdout: index_block_restart_interval: 1 2024-08-30T08:51:24.101 INFO:journalctl@ceph.mon.c.smithi190.stdout: metadata_block_size: 4096 2024-08-30T08:51:24.101 INFO:journalctl@ceph.mon.c.smithi190.stdout: partition_filters: 0 2024-08-30T08:51:24.101 INFO:journalctl@ceph.mon.c.smithi190.stdout: use_delta_encoding: 1 2024-08-30T08:51:24.101 INFO:journalctl@ceph.mon.c.smithi190.stdout: filter_policy: bloomfilter 2024-08-30T08:51:24.101 INFO:journalctl@ceph.mon.c.smithi190.stdout: whole_key_filtering: 1 2024-08-30T08:51:24.101 INFO:journalctl@ceph.mon.c.smithi190.stdout: verify_compression: 0 2024-08-30T08:51:24.101 INFO:journalctl@ceph.mon.c.smithi190.stdout: read_amp_bytes_per_bit: 0 2024-08-30T08:51:24.101 INFO:journalctl@ceph.mon.c.smithi190.stdout: format_version: 5 2024-08-30T08:51:24.101 INFO:journalctl@ceph.mon.c.smithi190.stdout: enable_index_compression: 1 2024-08-30T08:51:24.101 INFO:journalctl@ceph.mon.c.smithi190.stdout: block_align: 0 2024-08-30T08:51:24.101 INFO:journalctl@ceph.mon.c.smithi190.stdout: max_auto_readahead_size: 262144 2024-08-30T08:51:24.102 INFO:journalctl@ceph.mon.c.smithi190.stdout: prepopulate_block_cache: 0 2024-08-30T08:51:24.102 INFO:journalctl@ceph.mon.c.smithi190.stdout: initial_auto_readahead_size: 8192 2024-08-30T08:51:24.102 INFO:journalctl@ceph.mon.c.smithi190.stdout: num_file_reads_for_auto_readahead: 2 2024-08-30T08:51:24.102 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: Options.write_buffer_size: 33554432 2024-08-30T08:51:24.102 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: Options.max_write_buffer_number: 2 2024-08-30T08:51:24.102 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: Options.compression: NoCompression 2024-08-30T08:51:24.102 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: Options.bottommost_compression: Disabled 2024-08-30T08:51:24.102 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: Options.prefix_extractor: nullptr 2024-08-30T08:51:24.102 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-08-30T08:51:24.102 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: Options.num_levels: 7 2024-08-30T08:51:24.102 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-08-30T08:51:24.102 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-08-30T08:51:24.102 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-08-30T08:51:24.102 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-08-30T08:51:24.103 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-08-30T08:51:24.103 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-08-30T08:51:24.103 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-08-30T08:51:24.103 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-08-30T08:51:24.103 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-08-30T08:51:24.103 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-08-30T08:51:24.103 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-08-30T08:51:24.103 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-08-30T08:51:24.103 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: Options.compression_opts.window_bits: -14 2024-08-30T08:51:24.103 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: Options.compression_opts.level: 32767 2024-08-30T08:51:24.103 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: Options.compression_opts.strategy: 0 2024-08-30T08:51:24.103 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-08-30T08:51:24.103 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-08-30T08:51:24.103 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-08-30T08:51:24.103 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-08-30T08:51:24.104 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: Options.compression_opts.enabled: false 2024-08-30T08:51:24.104 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-08-30T08:51:24.104 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-08-30T08:51:24.104 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-08-30T08:51:24.104 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-08-30T08:51:24.104 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: Options.target_file_size_base: 67108864 2024-08-30T08:51:24.104 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: Options.target_file_size_multiplier: 1 2024-08-30T08:51:24.104 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-08-30T08:51:24.104 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-08-30T08:51:24.104 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-08-30T08:51:24.104 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-08-30T08:51:24.104 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-08-30T08:51:24.104 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-08-30T08:51:24.104 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-08-30T08:51:24.104 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-08-30T08:51:24.105 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-08-30T08:51:24.105 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-08-30T08:51:24.105 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-08-30T08:51:24.105 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-08-30T08:51:24.105 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-08-30T08:51:24.105 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: Options.arena_block_size: 1048576 2024-08-30T08:51:24.105 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-08-30T08:51:24.105 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-08-30T08:51:24.105 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: Options.disable_auto_compactions: 0 2024-08-30T08:51:24.105 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-08-30T08:51:24.105 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-08-30T08:51:24.105 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-08-30T08:51:24.105 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-08-30T08:51:24.105 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-08-30T08:51:24.106 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-08-30T08:51:24.106 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-08-30T08:51:24.106 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-08-30T08:51:24.106 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-08-30T08:51:24.106 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-08-30T08:51:24.106 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-08-30T08:51:24.106 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: Options.inplace_update_support: 0 2024-08-30T08:51:24.106 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: Options.inplace_update_num_locks: 10000 2024-08-30T08:51:24.106 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-08-30T08:51:24.106 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-08-30T08:51:24.106 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: Options.memtable_huge_page_size: 0 2024-08-30T08:51:24.106 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: Options.bloom_locality: 0 2024-08-30T08:51:24.106 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: Options.max_successive_merges: 0 2024-08-30T08:51:24.106 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: Options.optimize_filters_for_hits: 0 2024-08-30T08:51:24.107 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: Options.paranoid_file_checks: 0 2024-08-30T08:51:24.107 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: Options.force_consistency_checks: 1 2024-08-30T08:51:24.107 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: Options.report_bg_io_stats: 0 2024-08-30T08:51:24.107 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: Options.ttl: 2592000 2024-08-30T08:51:24.107 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: Options.periodic_compaction_seconds: 0 2024-08-30T08:51:24.107 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-08-30T08:51:24.107 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-08-30T08:51:24.107 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: Options.enable_blob_files: false 2024-08-30T08:51:24.107 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: Options.min_blob_size: 0 2024-08-30T08:51:24.107 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: Options.blob_file_size: 268435456 2024-08-30T08:51:24.107 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: Options.blob_compression_type: NoCompression 2024-08-30T08:51:24.107 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: Options.enable_blob_garbage_collection: false 2024-08-30T08:51:24.107 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-08-30T08:51:24.107 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-08-30T08:51:24.107 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-08-30T08:51:24.108 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: Options.blob_file_starting_level: 0 2024-08-30T08:51:24.108 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-08-30T08:51:24.108 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-08-30T08:51:24.108 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-08-30T08:51:24.108 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 57012c68-9bbe-4834-86b7-fb51ea16964c 2024-08-30T08:51:24.108 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: EVENT_LOG_v1 {"time_micros": 1725007883777777, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-08-30T08:51:24.108 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-08-30T08:51:24.108 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: EVENT_LOG_v1 {"time_micros": 1725007883778234, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1648, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1725007883, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "57012c68-9bbe-4834-86b7-fb51ea16964c", "db_session_id": "SPQZH4S336YZZACDIB4Q", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-08-30T08:51:24.108 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: EVENT_LOG_v1 {"time_micros": 1725007883778335, "job": 1, "event": "recovery_finished"} 2024-08-30T08:51:24.108 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-08-30T08:51:24.108 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-08-30T08:51:24.108 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55fcc48d8a80 2024-08-30T08:51:24.108 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: DB pointer 0x55fcc567a000 2024-08-30T08:51:24.109 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-08-30T08:51:24.109 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-08-30T08:51:24.109 INFO:journalctl@ceph.mon.c.smithi190.stdout: ** DB Stats ** 2024-08-30T08:51:24.109 INFO:journalctl@ceph.mon.c.smithi190.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-08-30T08:51:24.109 INFO:journalctl@ceph.mon.c.smithi190.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-08-30T08:51:24.109 INFO:journalctl@ceph.mon.c.smithi190.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-08-30T08:51:24.109 INFO:journalctl@ceph.mon.c.smithi190.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-08-30T08:51:24.109 INFO:journalctl@ceph.mon.c.smithi190.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-08-30T08:51:24.109 INFO:journalctl@ceph.mon.c.smithi190.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-08-30T08:51:24.109 INFO:journalctl@ceph.mon.c.smithi190.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-08-30T08:51:24.109 INFO:journalctl@ceph.mon.c.smithi190.stdout: 2024-08-30T08:51:24.109 INFO:journalctl@ceph.mon.c.smithi190.stdout: ** Compaction Stats [default] ** 2024-08-30T08:51:24.109 INFO:journalctl@ceph.mon.c.smithi190.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-08-30T08:51:24.109 INFO:journalctl@ceph.mon.c.smithi190.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-08-30T08:51:24.109 INFO:journalctl@ceph.mon.c.smithi190.stdout: L0 1/0 1.61 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.6 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-08-30T08:51:24.110 INFO:journalctl@ceph.mon.c.smithi190.stdout: Sum 1/0 1.61 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.6 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-08-30T08:51:24.110 INFO:journalctl@ceph.mon.c.smithi190.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.6 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-08-30T08:51:24.110 INFO:journalctl@ceph.mon.c.smithi190.stdout: 2024-08-30T08:51:24.110 INFO:journalctl@ceph.mon.c.smithi190.stdout: ** Compaction Stats [default] ** 2024-08-30T08:51:24.110 INFO:journalctl@ceph.mon.c.smithi190.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-08-30T08:51:24.110 INFO:journalctl@ceph.mon.c.smithi190.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-08-30T08:51:24.110 INFO:journalctl@ceph.mon.c.smithi190.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.6 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-08-30T08:51:24.110 INFO:journalctl@ceph.mon.c.smithi190.stdout: 2024-08-30T08:51:24.110 INFO:journalctl@ceph.mon.c.smithi190.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-08-30T08:51:24.110 INFO:journalctl@ceph.mon.c.smithi190.stdout: 2024-08-30T08:51:24.110 INFO:journalctl@ceph.mon.c.smithi190.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-08-30T08:51:24.110 INFO:journalctl@ceph.mon.c.smithi190.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-08-30T08:51:24.110 INFO:journalctl@ceph.mon.c.smithi190.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-08-30T08:51:24.110 INFO:journalctl@ceph.mon.c.smithi190.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-08-30T08:51:24.111 INFO:journalctl@ceph.mon.c.smithi190.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-08-30T08:51:24.111 INFO:journalctl@ceph.mon.c.smithi190.stdout: AddFile(Keys): cumulative 0, interval 0 2024-08-30T08:51:24.111 INFO:journalctl@ceph.mon.c.smithi190.stdout: Cumulative compaction: 0.00 GB write, 0.57 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-08-30T08:51:24.111 INFO:journalctl@ceph.mon.c.smithi190.stdout: Interval compaction: 0.00 GB write, 0.57 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-08-30T08:51:24.111 INFO:journalctl@ceph.mon.c.smithi190.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-08-30T08:51:24.111 INFO:journalctl@ceph.mon.c.smithi190.stdout: Block cache BinnedLRUCache@0x55fcc0878850#2 capacity: 512.00 MB usage: 0.23 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.1e-05 secs_since: 0 2024-08-30T08:51:24.111 INFO:journalctl@ceph.mon.c.smithi190.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.12 KB,2.23517e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-08-30T08:51:24.111 INFO:journalctl@ceph.mon.c.smithi190.stdout: 2024-08-30T08:51:24.111 INFO:journalctl@ceph.mon.c.smithi190.stdout: ** File Read Latency Histogram By Level [default] ** 2024-08-30T08:51:24.111 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-08-30T08:51:24.111 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: using public_addr v2:172.21.15.190:0/0 -> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-30T08:51:24.111 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: starting mon.c rank -1 at public addrs [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] at bind addrs [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 2024-08-30T08:51:24.112 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: mon.c@-1(???) e0 preinit fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 2024-08-30T08:51:24.112 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: mon.c@-1(synchronizing).mds e1 new map 2024-08-30T08:51:24.112 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: mon.c@-1(synchronizing).mds e1 print_map 2024-08-30T08:51:24.112 INFO:journalctl@ceph.mon.c.smithi190.stdout: e1 2024-08-30T08:51:24.112 INFO:journalctl@ceph.mon.c.smithi190.stdout: btime 2024-08-30T08:49:44:858072+0000 2024-08-30T08:51:24.112 INFO:journalctl@ceph.mon.c.smithi190.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-08-30T08:51:24.112 INFO:journalctl@ceph.mon.c.smithi190.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-08-30T08:51:24.112 INFO:journalctl@ceph.mon.c.smithi190.stdout: legacy client fscid: -1 2024-08-30T08:51:24.112 INFO:journalctl@ceph.mon.c.smithi190.stdout: 2024-08-30T08:51:24.112 INFO:journalctl@ceph.mon.c.smithi190.stdout: No filesystems configured 2024-08-30T08:51:24.112 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-08-30T08:51:24.112 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-08-30T08:51:24.112 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-08-30T08:51:24.113 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-08-30T08:51:24.113 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-08-30T08:51:24.113 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-08-30T08:51:24.113 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 bash[21968]: a523197617ba300e0d1dc684b83cabc78c4c056ba02591ffda49b8e1e7bd9cf9 2024-08-30T08:51:24.113 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-08-30T08:51:24.113 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-08-30T08:51:24.113 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-08-30T08:51:24.113 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-08-30T08:51:24.113 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-30T08:51:24.113 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi190", "target": ["mon-mgr", ""]}]: dispatch 2024-08-30T08:51:24.113 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:24.113 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:24.113 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:24.113 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:24.114 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi167", "name": "osd_memory_target"} : dispatch 2024-08-30T08:51:24.114 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:51:24.114 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-30T08:51:24.114 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: Updating smithi167:/etc/ceph/ceph.conf 2024-08-30T08:51:24.114 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: Deploying cephadm binary to smithi190 2024-08-30T08:51:24.114 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: Updating smithi167:/var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/config/ceph.conf 2024-08-30T08:51:24.114 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:24.114 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:24.114 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:24.114 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: Updating smithi167:/etc/ceph/ceph.client.admin.keyring 2024-08-30T08:51:24.114 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-30T08:51:24.114 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: Updating smithi167:/var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/config/ceph.client.admin.keyring 2024-08-30T08:51:24.114 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-30T08:51:24.114 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:24.114 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: Added host smithi190 2024-08-30T08:51:24.115 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-30T08:51:24.115 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:24.115 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:24.115 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:24.115 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-30T08:51:24.115 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-30T08:51:24.115 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: from='client.? 172.21.15.42:0/256350112' entity='client.admin' cmd={"prefix": "osd crush tunables", "profile": "default"} : dispatch 2024-08-30T08:51:24.115 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: from='client.? 172.21.15.42:0/256350112' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-08-30T08:51:24.115 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: osdmap e4: 0 total, 0 up, 0 in 2024-08-30T08:51:24.115 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:24.115 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:24.115 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:24.115 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:24.115 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi190", "name": "osd_memory_target"} : dispatch 2024-08-30T08:51:24.116 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:51:24.116 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-30T08:51:24.116 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-30T08:51:24.116 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: Updating smithi190:/etc/ceph/ceph.conf 2024-08-30T08:51:24.116 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: Updating smithi190:/var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/config/ceph.conf 2024-08-30T08:51:24.116 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:24.116 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:24.116 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:24.116 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:24.116 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-08-30T08:51:24.116 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:51:24.116 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: Updating smithi190:/etc/ceph/ceph.client.admin.keyring 2024-08-30T08:51:24.116 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: Updating smithi190:/var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/config/ceph.client.admin.keyring 2024-08-30T08:51:24.116 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi042:172.21.15.42=a;smithi167:172.21.15.167=b;smithi190:172.21.15.190=c", "target": ["mon-mgr", ""]}]: dispatch 2024-08-30T08:51:24.116 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: Saving service mon spec with placement smithi042:172.21.15.42=a;smithi167:172.21.15.167=b;smithi190:172.21.15.190=c;count:3 2024-08-30T08:51:24.117 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: Deploying daemon mon.c on smithi190 2024-08-30T08:51:24.117 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-30T08:51:24.117 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: from='client.? 172.21.15.190:0/3954226789' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-08-30T08:51:24.117 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-30T08:51:24.117 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 ceph-mon[22001]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-08-30T08:51:24.117 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:23 smithi190 systemd[1]: Started Ceph mon.c for b0895c1a-66ac-11ef-bcd4-c7b262605968. 2024-08-30T08:51:25.466 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-08-30T08:51:25.466 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph mon dump -f json 2024-08-30T08:51:25.729 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.c/config 2024-08-30T08:51:29.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:28 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-08-30T08:51:29.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:28 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-30T08:51:29.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:28 smithi042 ceph-mon[18652]: mon.a calling monitor election 2024-08-30T08:51:29.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:28 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-30T08:51:29.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:28 smithi042 ceph-mon[18652]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-30T08:51:29.134 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:28 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-30T08:51:29.134 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:28 smithi042 ceph-mon[18652]: mon.c calling monitor election 2024-08-30T08:51:29.134 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:28 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-30T08:51:29.134 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:28 smithi042 ceph-mon[18652]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-30T08:51:29.134 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:28 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-30T08:51:29.134 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:28 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-30T08:51:29.134 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:28 smithi042 ceph-mon[18652]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-08-30T08:51:29.134 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:28 smithi042 ceph-mon[18652]: monmap epoch 2 2024-08-30T08:51:29.134 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:28 smithi042 ceph-mon[18652]: fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 2024-08-30T08:51:29.134 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:28 smithi042 ceph-mon[18652]: last_changed 2024-08-30T08:51:23.808828+0000 2024-08-30T08:51:29.134 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:28 smithi042 ceph-mon[18652]: created 2024-08-30T08:49:41.668361+0000 2024-08-30T08:51:29.134 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:28 smithi042 ceph-mon[18652]: min_mon_release 19 (squid) 2024-08-30T08:51:29.134 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:28 smithi042 ceph-mon[18652]: election_strategy: 1 2024-08-30T08:51:29.134 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:28 smithi042 ceph-mon[18652]: 0: [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] mon.a 2024-08-30T08:51:29.135 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:28 smithi042 ceph-mon[18652]: 1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.c 2024-08-30T08:51:29.135 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:28 smithi042 ceph-mon[18652]: fsmap 2024-08-30T08:51:29.135 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:28 smithi042 ceph-mon[18652]: osdmap e4: 0 total, 0 up, 0 in 2024-08-30T08:51:29.135 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:28 smithi042 ceph-mon[18652]: mgrmap e13: x(active, since 44s) 2024-08-30T08:51:29.135 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:28 smithi042 ceph-mon[18652]: overall HEALTH_OK 2024-08-30T08:51:29.135 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:28 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:29.135 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:28 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:29.135 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:28 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:29.135 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:28 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-08-30T08:51:29.135 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:28 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:51:29.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:28 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-08-30T08:51:29.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:28 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-30T08:51:29.187 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:28 smithi190 ceph-mon[22001]: mon.a calling monitor election 2024-08-30T08:51:29.187 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:28 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-30T08:51:29.187 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:28 smithi190 ceph-mon[22001]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-30T08:51:29.187 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:28 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-30T08:51:29.187 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:28 smithi190 ceph-mon[22001]: mon.c calling monitor election 2024-08-30T08:51:29.187 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:28 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-30T08:51:29.187 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:28 smithi190 ceph-mon[22001]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-30T08:51:29.187 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:28 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-30T08:51:29.187 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:28 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-30T08:51:29.187 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:28 smithi190 ceph-mon[22001]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-08-30T08:51:29.187 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:28 smithi190 ceph-mon[22001]: monmap epoch 2 2024-08-30T08:51:29.187 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:28 smithi190 ceph-mon[22001]: fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 2024-08-30T08:51:29.187 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:28 smithi190 ceph-mon[22001]: last_changed 2024-08-30T08:51:23.808828+0000 2024-08-30T08:51:29.187 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:28 smithi190 ceph-mon[22001]: created 2024-08-30T08:49:41.668361+0000 2024-08-30T08:51:29.187 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:28 smithi190 ceph-mon[22001]: min_mon_release 19 (squid) 2024-08-30T08:51:29.188 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:28 smithi190 ceph-mon[22001]: election_strategy: 1 2024-08-30T08:51:29.188 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:28 smithi190 ceph-mon[22001]: 0: [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] mon.a 2024-08-30T08:51:29.188 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:28 smithi190 ceph-mon[22001]: 1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.c 2024-08-30T08:51:29.188 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:28 smithi190 ceph-mon[22001]: fsmap 2024-08-30T08:51:29.188 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:28 smithi190 ceph-mon[22001]: osdmap e4: 0 total, 0 up, 0 in 2024-08-30T08:51:29.188 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:28 smithi190 ceph-mon[22001]: mgrmap e13: x(active, since 44s) 2024-08-30T08:51:29.188 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:28 smithi190 ceph-mon[22001]: overall HEALTH_OK 2024-08-30T08:51:29.188 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:28 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:29.188 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:28 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:29.188 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:28 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:29.188 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:28 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-08-30T08:51:29.188 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:28 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:51:29.409 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-08-30T08:51:29.409 INFO:teuthology.orchestra.run.smithi190.stdout:{"epoch":2,"fsid":"b0895c1a-66ac-11ef-bcd4-c7b262605968","modified":"2024-08-30T08:51:23.808828Z","created":"2024-08-30T08:49:41.668361Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:3300","nonce":0},{"type":"v1","addr":"172.21.15.42:6789","nonce":0}]},"addr":"172.21.15.42:6789/0","public_addr":"172.21.15.42:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3300","nonce":0},{"type":"v1","addr":"172.21.15.190:6789","nonce":0}]},"addr":"172.21.15.190:6789/0","public_addr":"172.21.15.190:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-08-30T08:51:29.409 INFO:teuthology.orchestra.run.smithi190.stderr:dumped monmap epoch 2 2024-08-30T08:51:29.962 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:29 smithi190 ceph-mon[22001]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-30T08:51:29.962 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:29 smithi190 ceph-mon[22001]: Deploying daemon mon.b on smithi167 2024-08-30T08:51:29.962 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:29 smithi190 ceph-mon[22001]: from='client.? 172.21.15.190:0/4164405556' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-08-30T08:51:29.962 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:29 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-30T08:51:30.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:29 smithi042 ceph-mon[18652]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-30T08:51:30.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:29 smithi042 ceph-mon[18652]: Deploying daemon mon.b on smithi167 2024-08-30T08:51:30.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:29 smithi042 ceph-mon[18652]: from='client.? 172.21.15.190:0/4164405556' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-08-30T08:51:30.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:29 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-30T08:51:30.990 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-08-30T08:51:30.990 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph mon dump -f json 2024-08-30T08:51:31.132 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Aug 30 08:51:30 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mgr-x[18896]: 2024-08-30T08:51:30.806+0000 7f1b13e69640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-08-30T08:51:31.240 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.c/config 2024-08-30T08:51:31.976 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-08-30T08:51:31.976 INFO:teuthology.orchestra.run.smithi190.stdout:{"epoch":2,"fsid":"b0895c1a-66ac-11ef-bcd4-c7b262605968","modified":"2024-08-30T08:51:23.808828Z","created":"2024-08-30T08:49:41.668361Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:3300","nonce":0},{"type":"v1","addr":"172.21.15.42:6789","nonce":0}]},"addr":"172.21.15.42:6789/0","public_addr":"172.21.15.42:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3300","nonce":0},{"type":"v1","addr":"172.21.15.190:6789","nonce":0}]},"addr":"172.21.15.190:6789/0","public_addr":"172.21.15.190:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-08-30T08:51:31.977 INFO:teuthology.orchestra.run.smithi190.stderr:dumped monmap epoch 2 2024-08-30T08:51:32.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:31 smithi042 ceph-mon[18652]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-30T08:51:32.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:31 smithi190 ceph-mon[22001]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-30T08:51:32.665 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: IDENTITY file: IDENTITY 2024-08-30T08:51:32.665 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-08-30T08:51:32.665 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2024-08-30T08:51:32.666 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 636 ; 2024-08-30T08:51:32.666 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: Options.error_if_exists: 0 2024-08-30T08:51:32.666 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: Options.create_if_missing: 0 2024-08-30T08:51:32.666 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: Options.paranoid_checks: 1 2024-08-30T08:51:32.666 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: Options.flush_verify_memtable_count: 1 2024-08-30T08:51:32.666 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-08-30T08:51:32.666 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-08-30T08:51:32.666 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: Options.env: 0x55e2253935a0 2024-08-30T08:51:32.666 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: Options.fs: PosixFileSystem 2024-08-30T08:51:32.666 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: Options.info_log: 0x55e229e9d4e0 2024-08-30T08:51:32.667 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: Options.max_file_opening_threads: 16 2024-08-30T08:51:32.667 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: Options.statistics: (nil) 2024-08-30T08:51:32.667 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: Options.use_fsync: 0 2024-08-30T08:51:32.667 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: Options.max_log_file_size: 0 2024-08-30T08:51:32.667 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-08-30T08:51:32.667 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: Options.log_file_time_to_roll: 0 2024-08-30T08:51:32.667 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: Options.keep_log_file_num: 1000 2024-08-30T08:51:32.667 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: Options.recycle_log_file_num: 0 2024-08-30T08:51:32.667 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: Options.allow_fallocate: 1 2024-08-30T08:51:32.667 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: Options.allow_mmap_reads: 0 2024-08-30T08:51:32.667 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: Options.allow_mmap_writes: 0 2024-08-30T08:51:32.667 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: Options.use_direct_reads: 0 2024-08-30T08:51:32.667 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-08-30T08:51:33.078 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: Options.create_missing_column_families: 0 2024-08-30T08:51:33.078 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: Options.db_log_dir: 2024-08-30T08:51:33.078 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: Options.wal_dir: 2024-08-30T08:51:33.078 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: Options.table_cache_numshardbits: 6 2024-08-30T08:51:33.078 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: Options.WAL_ttl_seconds: 0 2024-08-30T08:51:33.078 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: Options.WAL_size_limit_MB: 0 2024-08-30T08:51:33.078 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-08-30T08:51:33.079 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-08-30T08:51:33.079 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: Options.is_fd_close_on_exec: 1 2024-08-30T08:51:33.079 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: Options.advise_random_on_open: 1 2024-08-30T08:51:33.079 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: Options.db_write_buffer_size: 0 2024-08-30T08:51:33.079 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: Options.write_buffer_manager: 0x55e229ea15e0 2024-08-30T08:51:33.079 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-08-30T08:51:33.079 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-08-30T08:51:33.079 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: Options.use_adaptive_mutex: 0 2024-08-30T08:51:33.079 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: Options.rate_limiter: (nil) 2024-08-30T08:51:33.079 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-08-30T08:51:33.079 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: Options.wal_recovery_mode: 2 2024-08-30T08:51:33.079 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: Options.enable_thread_tracking: 0 2024-08-30T08:51:33.079 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: Options.enable_pipelined_write: 0 2024-08-30T08:51:33.079 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: Options.unordered_write: 0 2024-08-30T08:51:33.080 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-08-30T08:51:33.080 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-08-30T08:51:33.080 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-08-30T08:51:33.080 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-08-30T08:51:33.080 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: Options.row_cache: None 2024-08-30T08:51:33.080 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: Options.wal_filter: None 2024-08-30T08:51:33.080 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-08-30T08:51:33.080 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: Options.allow_ingest_behind: 0 2024-08-30T08:51:33.080 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: Options.two_write_queues: 0 2024-08-30T08:51:33.080 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: Options.manual_wal_flush: 0 2024-08-30T08:51:33.080 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: Options.wal_compression: 0 2024-08-30T08:51:33.080 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: Options.atomic_flush: 0 2024-08-30T08:51:33.080 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-08-30T08:51:33.080 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: Options.persist_stats_to_disk: 0 2024-08-30T08:51:33.081 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: Options.write_dbid_to_manifest: 0 2024-08-30T08:51:33.081 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: Options.log_readahead_size: 0 2024-08-30T08:51:33.081 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-08-30T08:51:33.081 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: Options.best_efforts_recovery: 0 2024-08-30T08:51:33.081 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-08-30T08:51:33.081 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-08-30T08:51:33.081 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: Options.allow_data_in_errors: 0 2024-08-30T08:51:33.081 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: Options.db_host_id: __hostname__ 2024-08-30T08:51:33.081 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: Options.enforce_single_del_contracts: true 2024-08-30T08:51:33.081 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: Options.max_background_jobs: 2 2024-08-30T08:51:33.081 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: Options.max_background_compactions: -1 2024-08-30T08:51:33.081 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: Options.max_subcompactions: 1 2024-08-30T08:51:33.081 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-08-30T08:51:33.082 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-08-30T08:51:33.082 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: Options.delayed_write_rate : 16777216 2024-08-30T08:51:33.082 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: Options.max_total_wal_size: 0 2024-08-30T08:51:33.082 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-08-30T08:51:33.082 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: Options.stats_dump_period_sec: 600 2024-08-30T08:51:33.082 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: Options.stats_persist_period_sec: 600 2024-08-30T08:51:33.082 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-08-30T08:51:33.082 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: Options.max_open_files: -1 2024-08-30T08:51:33.082 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: Options.bytes_per_sync: 0 2024-08-30T08:51:33.082 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: Options.wal_bytes_per_sync: 0 2024-08-30T08:51:33.082 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: Options.strict_bytes_per_sync: 0 2024-08-30T08:51:33.082 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: Options.compaction_readahead_size: 0 2024-08-30T08:51:33.082 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: Options.max_background_flushes: -1 2024-08-30T08:51:33.082 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: Compression algorithms supported: 2024-08-30T08:51:33.083 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: kZSTD supported: 0 2024-08-30T08:51:33.083 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: kXpressCompression supported: 0 2024-08-30T08:51:33.083 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: kBZip2Compression supported: 0 2024-08-30T08:51:33.083 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-08-30T08:51:33.083 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: kLZ4Compression supported: 1 2024-08-30T08:51:33.083 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: kZlibCompression supported: 1 2024-08-30T08:51:33.083 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: kLZ4HCCompression supported: 1 2024-08-30T08:51:33.083 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: kSnappyCompression supported: 1 2024-08-30T08:51:33.084 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: Fast CRC32 supported: Supported on x86 2024-08-30T08:51:33.084 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: DMutex implementation: pthread_mutex_t 2024-08-30T08:51:33.084 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2024-08-30T08:51:33.084 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-08-30T08:51:33.084 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-08-30T08:51:33.084 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: Options.merge_operator: 2024-08-30T08:51:33.084 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: Options.compaction_filter: None 2024-08-30T08:51:33.084 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: Options.compaction_filter_factory: None 2024-08-30T08:51:33.084 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: Options.sst_partitioner_factory: None 2024-08-30T08:51:33.084 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: Options.memtable_factory: SkipListFactory 2024-08-30T08:51:33.084 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: Options.table_factory: BlockBasedTable 2024-08-30T08:51:33.084 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55e229e9d100) 2024-08-30T08:51:33.084 INFO:journalctl@ceph.mon.b.smithi167.stdout: cache_index_and_filter_blocks: 1 2024-08-30T08:51:33.085 INFO:journalctl@ceph.mon.b.smithi167.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-08-30T08:51:33.085 INFO:journalctl@ceph.mon.b.smithi167.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-08-30T08:51:33.085 INFO:journalctl@ceph.mon.b.smithi167.stdout: pin_top_level_index_and_filter: 1 2024-08-30T08:51:33.085 INFO:journalctl@ceph.mon.b.smithi167.stdout: index_type: 0 2024-08-30T08:51:33.085 INFO:journalctl@ceph.mon.b.smithi167.stdout: data_block_index_type: 0 2024-08-30T08:51:33.085 INFO:journalctl@ceph.mon.b.smithi167.stdout: index_shortening: 1 2024-08-30T08:51:33.085 INFO:journalctl@ceph.mon.b.smithi167.stdout: data_block_hash_table_util_ratio: 0.750000 2024-08-30T08:51:33.085 INFO:journalctl@ceph.mon.b.smithi167.stdout: checksum: 4 2024-08-30T08:51:33.085 INFO:journalctl@ceph.mon.b.smithi167.stdout: no_block_cache: 0 2024-08-30T08:51:33.085 INFO:journalctl@ceph.mon.b.smithi167.stdout: block_cache: 0x55e225e60850 2024-08-30T08:51:33.085 INFO:journalctl@ceph.mon.b.smithi167.stdout: block_cache_name: BinnedLRUCache 2024-08-30T08:51:33.085 INFO:journalctl@ceph.mon.b.smithi167.stdout: block_cache_options: 2024-08-30T08:51:33.085 INFO:journalctl@ceph.mon.b.smithi167.stdout: capacity : 536870912 2024-08-30T08:51:33.085 INFO:journalctl@ceph.mon.b.smithi167.stdout: num_shard_bits : 4 2024-08-30T08:51:33.086 INFO:journalctl@ceph.mon.b.smithi167.stdout: strict_capacity_limit : 0 2024-08-30T08:51:33.086 INFO:journalctl@ceph.mon.b.smithi167.stdout: high_pri_pool_ratio: 0.000 2024-08-30T08:51:33.086 INFO:journalctl@ceph.mon.b.smithi167.stdout: block_cache_compressed: (nil) 2024-08-30T08:51:33.086 INFO:journalctl@ceph.mon.b.smithi167.stdout: persistent_cache: (nil) 2024-08-30T08:51:33.086 INFO:journalctl@ceph.mon.b.smithi167.stdout: block_size: 4096 2024-08-30T08:51:33.086 INFO:journalctl@ceph.mon.b.smithi167.stdout: block_size_deviation: 10 2024-08-30T08:51:33.086 INFO:journalctl@ceph.mon.b.smithi167.stdout: block_restart_interval: 16 2024-08-30T08:51:33.086 INFO:journalctl@ceph.mon.b.smithi167.stdout: index_block_restart_interval: 1 2024-08-30T08:51:33.086 INFO:journalctl@ceph.mon.b.smithi167.stdout: metadata_block_size: 4096 2024-08-30T08:51:33.086 INFO:journalctl@ceph.mon.b.smithi167.stdout: partition_filters: 0 2024-08-30T08:51:33.086 INFO:journalctl@ceph.mon.b.smithi167.stdout: use_delta_encoding: 1 2024-08-30T08:51:33.086 INFO:journalctl@ceph.mon.b.smithi167.stdout: filter_policy: bloomfilter 2024-08-30T08:51:33.086 INFO:journalctl@ceph.mon.b.smithi167.stdout: whole_key_filtering: 1 2024-08-30T08:51:33.086 INFO:journalctl@ceph.mon.b.smithi167.stdout: verify_compression: 0 2024-08-30T08:51:33.086 INFO:journalctl@ceph.mon.b.smithi167.stdout: read_amp_bytes_per_bit: 0 2024-08-30T08:51:33.087 INFO:journalctl@ceph.mon.b.smithi167.stdout: format_version: 5 2024-08-30T08:51:33.087 INFO:journalctl@ceph.mon.b.smithi167.stdout: enable_index_compression: 1 2024-08-30T08:51:33.087 INFO:journalctl@ceph.mon.b.smithi167.stdout: block_align: 0 2024-08-30T08:51:33.087 INFO:journalctl@ceph.mon.b.smithi167.stdout: max_auto_readahead_size: 262144 2024-08-30T08:51:33.087 INFO:journalctl@ceph.mon.b.smithi167.stdout: prepopulate_block_cache: 0 2024-08-30T08:51:33.087 INFO:journalctl@ceph.mon.b.smithi167.stdout: initial_auto_readahead_size: 8192 2024-08-30T08:51:33.087 INFO:journalctl@ceph.mon.b.smithi167.stdout: num_file_reads_for_auto_readahead: 2 2024-08-30T08:51:33.087 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: Options.write_buffer_size: 33554432 2024-08-30T08:51:33.087 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: Options.max_write_buffer_number: 2 2024-08-30T08:51:33.087 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: Options.compression: NoCompression 2024-08-30T08:51:33.087 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: Options.bottommost_compression: Disabled 2024-08-30T08:51:33.087 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: Options.prefix_extractor: nullptr 2024-08-30T08:51:33.087 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-08-30T08:51:33.087 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: Options.num_levels: 7 2024-08-30T08:51:33.088 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-08-30T08:51:33.088 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-08-30T08:51:33.088 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-08-30T08:51:33.088 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-08-30T08:51:33.089 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-08-30T08:51:33.089 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-08-30T08:51:33.089 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-08-30T08:51:33.089 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-08-30T08:51:33.089 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-08-30T08:51:33.089 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-08-30T08:51:33.089 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-08-30T08:51:33.089 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-08-30T08:51:33.090 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: Options.compression_opts.window_bits: -14 2024-08-30T08:51:33.090 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: Options.compression_opts.level: 32767 2024-08-30T08:51:33.090 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: Options.compression_opts.strategy: 0 2024-08-30T08:51:33.090 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-08-30T08:51:33.090 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-08-30T08:51:33.090 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-08-30T08:51:33.090 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-08-30T08:51:33.090 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: Options.compression_opts.enabled: false 2024-08-30T08:51:33.090 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-08-30T08:51:33.090 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-08-30T08:51:33.090 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-08-30T08:51:33.090 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-08-30T08:51:33.090 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: Options.target_file_size_base: 67108864 2024-08-30T08:51:33.090 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: Options.target_file_size_multiplier: 1 2024-08-30T08:51:33.091 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-08-30T08:51:33.091 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-08-30T08:51:33.091 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-08-30T08:51:33.091 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-08-30T08:51:33.091 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-08-30T08:51:33.091 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-08-30T08:51:33.091 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-08-30T08:51:33.091 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-08-30T08:51:33.091 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-08-30T08:51:33.091 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-08-30T08:51:33.091 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-08-30T08:51:33.091 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-08-30T08:51:33.091 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-08-30T08:51:33.091 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: Options.arena_block_size: 1048576 2024-08-30T08:51:33.092 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-08-30T08:51:33.092 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-08-30T08:51:33.092 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: Options.disable_auto_compactions: 0 2024-08-30T08:51:33.092 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-08-30T08:51:33.092 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-08-30T08:51:33.092 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-08-30T08:51:33.092 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-08-30T08:51:33.092 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-08-30T08:51:33.092 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-08-30T08:51:33.092 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-08-30T08:51:33.092 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-08-30T08:51:33.092 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-08-30T08:51:33.092 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-08-30T08:51:33.092 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-08-30T08:51:33.093 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: Options.inplace_update_support: 0 2024-08-30T08:51:33.093 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: Options.inplace_update_num_locks: 10000 2024-08-30T08:51:33.093 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-08-30T08:51:33.093 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-08-30T08:51:33.093 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: Options.memtable_huge_page_size: 0 2024-08-30T08:51:33.093 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: Options.bloom_locality: 0 2024-08-30T08:51:33.093 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: Options.max_successive_merges: 0 2024-08-30T08:51:33.093 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: Options.optimize_filters_for_hits: 0 2024-08-30T08:51:33.093 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: Options.paranoid_file_checks: 0 2024-08-30T08:51:33.093 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: Options.force_consistency_checks: 1 2024-08-30T08:51:33.093 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: Options.report_bg_io_stats: 0 2024-08-30T08:51:33.093 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: Options.ttl: 2592000 2024-08-30T08:51:33.093 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: Options.periodic_compaction_seconds: 0 2024-08-30T08:51:33.093 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-08-30T08:51:33.093 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-08-30T08:51:33.094 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: Options.enable_blob_files: false 2024-08-30T08:51:33.094 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: Options.min_blob_size: 0 2024-08-30T08:51:33.094 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: Options.blob_file_size: 268435456 2024-08-30T08:51:33.094 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: Options.blob_compression_type: NoCompression 2024-08-30T08:51:33.094 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: Options.enable_blob_garbage_collection: false 2024-08-30T08:51:33.094 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-08-30T08:51:33.094 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-08-30T08:51:33.094 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-08-30T08:51:33.094 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: Options.blob_file_starting_level: 0 2024-08-30T08:51:33.094 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-08-30T08:51:33.094 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-08-30T08:51:33.094 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-08-30T08:51:33.094 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: bf0acf96-72cd-4435-a1b4-719a785f6c2b 2024-08-30T08:51:33.094 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: EVENT_LOG_v1 {"time_micros": 1725007892664588, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-08-30T08:51:33.095 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-08-30T08:51:33.095 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: EVENT_LOG_v1 {"time_micros": 1725007892665098, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1773, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1725007892, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "bf0acf96-72cd-4435-a1b4-719a785f6c2b", "db_session_id": "L8C7UI35PV46YBAQ5KIX", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-08-30T08:51:33.095 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: EVENT_LOG_v1 {"time_micros": 1725007892665213, "job": 1, "event": "recovery_finished"} 2024-08-30T08:51:33.095 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-08-30T08:51:33.095 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-08-30T08:51:33.095 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55e229ec0a80 2024-08-30T08:51:33.095 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: DB pointer 0x55e22ac62000 2024-08-30T08:51:33.095 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-08-30T08:51:33.095 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-08-30T08:51:33.095 INFO:journalctl@ceph.mon.b.smithi167.stdout: ** DB Stats ** 2024-08-30T08:51:33.095 INFO:journalctl@ceph.mon.b.smithi167.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-08-30T08:51:33.095 INFO:journalctl@ceph.mon.b.smithi167.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-08-30T08:51:33.095 INFO:journalctl@ceph.mon.b.smithi167.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-08-30T08:51:33.095 INFO:journalctl@ceph.mon.b.smithi167.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-08-30T08:51:33.095 INFO:journalctl@ceph.mon.b.smithi167.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-08-30T08:51:33.096 INFO:journalctl@ceph.mon.b.smithi167.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-08-30T08:51:33.096 INFO:journalctl@ceph.mon.b.smithi167.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-08-30T08:51:33.096 INFO:journalctl@ceph.mon.b.smithi167.stdout: 2024-08-30T08:51:33.096 INFO:journalctl@ceph.mon.b.smithi167.stdout: ** Compaction Stats [default] ** 2024-08-30T08:51:33.096 INFO:journalctl@ceph.mon.b.smithi167.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-08-30T08:51:33.096 INFO:journalctl@ceph.mon.b.smithi167.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-08-30T08:51:33.096 INFO:journalctl@ceph.mon.b.smithi167.stdout: L0 1/0 1.73 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.5 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-08-30T08:51:33.096 INFO:journalctl@ceph.mon.b.smithi167.stdout: Sum 1/0 1.73 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.5 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-08-30T08:51:33.096 INFO:journalctl@ceph.mon.b.smithi167.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.5 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-08-30T08:51:33.096 INFO:journalctl@ceph.mon.b.smithi167.stdout: 2024-08-30T08:51:33.096 INFO:journalctl@ceph.mon.b.smithi167.stdout: ** Compaction Stats [default] ** 2024-08-30T08:51:33.096 INFO:journalctl@ceph.mon.b.smithi167.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-08-30T08:51:33.096 INFO:journalctl@ceph.mon.b.smithi167.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-08-30T08:51:33.096 INFO:journalctl@ceph.mon.b.smithi167.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.5 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-08-30T08:51:33.097 INFO:journalctl@ceph.mon.b.smithi167.stdout: 2024-08-30T08:51:33.097 INFO:journalctl@ceph.mon.b.smithi167.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-08-30T08:51:33.097 INFO:journalctl@ceph.mon.b.smithi167.stdout: 2024-08-30T08:51:33.097 INFO:journalctl@ceph.mon.b.smithi167.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-08-30T08:51:33.097 INFO:journalctl@ceph.mon.b.smithi167.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-08-30T08:51:33.097 INFO:journalctl@ceph.mon.b.smithi167.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-08-30T08:51:33.097 INFO:journalctl@ceph.mon.b.smithi167.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-08-30T08:51:33.097 INFO:journalctl@ceph.mon.b.smithi167.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-08-30T08:51:33.097 INFO:journalctl@ceph.mon.b.smithi167.stdout: AddFile(Keys): cumulative 0, interval 0 2024-08-30T08:51:33.097 INFO:journalctl@ceph.mon.b.smithi167.stdout: Cumulative compaction: 0.00 GB write, 0.58 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-08-30T08:51:33.097 INFO:journalctl@ceph.mon.b.smithi167.stdout: Interval compaction: 0.00 GB write, 0.58 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-08-30T08:51:33.097 INFO:journalctl@ceph.mon.b.smithi167.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-08-30T08:51:33.097 INFO:journalctl@ceph.mon.b.smithi167.stdout: Block cache BinnedLRUCache@0x55e225e60850#2 capacity: 512.00 MB usage: 0.23 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-08-30T08:51:33.097 INFO:journalctl@ceph.mon.b.smithi167.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.12 KB,2.23517e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-08-30T08:51:33.098 INFO:journalctl@ceph.mon.b.smithi167.stdout: 2024-08-30T08:51:33.098 INFO:journalctl@ceph.mon.b.smithi167.stdout: ** File Read Latency Histogram By Level [default] ** 2024-08-30T08:51:33.098 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: mon.b does not exist in monmap, will attempt to join an existing cluster 2024-08-30T08:51:33.098 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: using public_addr v2:172.21.15.167:0/0 -> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] 2024-08-30T08:51:33.098 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: starting mon.b rank -1 at public addrs [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] at bind addrs [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 2024-08-30T08:51:33.098 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: mon.b@-1(???) e0 preinit fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 2024-08-30T08:51:33.098 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: mon.b@-1(synchronizing).mds e1 new map 2024-08-30T08:51:33.098 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: mon.b@-1(synchronizing).mds e1 print_map 2024-08-30T08:51:33.098 INFO:journalctl@ceph.mon.b.smithi167.stdout: e1 2024-08-30T08:51:33.098 INFO:journalctl@ceph.mon.b.smithi167.stdout: btime 2024-08-30T08:49:44:858072+0000 2024-08-30T08:51:33.098 INFO:journalctl@ceph.mon.b.smithi167.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-08-30T08:51:33.098 INFO:journalctl@ceph.mon.b.smithi167.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-08-30T08:51:33.098 INFO:journalctl@ceph.mon.b.smithi167.stdout: legacy client fscid: -1 2024-08-30T08:51:33.098 INFO:journalctl@ceph.mon.b.smithi167.stdout: 2024-08-30T08:51:33.099 INFO:journalctl@ceph.mon.b.smithi167.stdout: No filesystems configured 2024-08-30T08:51:33.099 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-08-30T08:51:33.099 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-08-30T08:51:33.099 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-08-30T08:51:33.099 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-08-30T08:51:33.099 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-08-30T08:51:33.099 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-08-30T08:51:33.099 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-08-30T08:51:33.099 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-08-30T08:51:33.099 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-08-30T08:51:33.099 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-08-30T08:51:33.099 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-30T08:51:33.099 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi190", "target": ["mon-mgr", ""]}]: dispatch 2024-08-30T08:51:33.100 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:33.100 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:33.100 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:33.100 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:33.100 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi167", "name": "osd_memory_target"} : dispatch 2024-08-30T08:51:33.100 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:51:33.100 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-30T08:51:33.100 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: Updating smithi167:/etc/ceph/ceph.conf 2024-08-30T08:51:33.100 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: Deploying cephadm binary to smithi190 2024-08-30T08:51:33.100 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: Updating smithi167:/var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/config/ceph.conf 2024-08-30T08:51:33.100 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:33.100 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:33.100 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:33.100 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: Updating smithi167:/etc/ceph/ceph.client.admin.keyring 2024-08-30T08:51:33.101 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-30T08:51:33.101 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: Updating smithi167:/var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/config/ceph.client.admin.keyring 2024-08-30T08:51:33.101 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-30T08:51:33.101 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:33.101 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: Added host smithi190 2024-08-30T08:51:33.101 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-30T08:51:33.101 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:33.101 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:33.101 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:33.101 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-30T08:51:33.101 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-30T08:51:33.101 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: from='client.? 172.21.15.42:0/256350112' entity='client.admin' cmd={"prefix": "osd crush tunables", "profile": "default"} : dispatch 2024-08-30T08:51:33.101 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: from='client.? 172.21.15.42:0/256350112' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-08-30T08:51:33.101 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: osdmap e4: 0 total, 0 up, 0 in 2024-08-30T08:51:33.102 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:33.102 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:33.102 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:33.102 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:33.102 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi190", "name": "osd_memory_target"} : dispatch 2024-08-30T08:51:33.102 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:51:33.102 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-30T08:51:33.102 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-30T08:51:33.102 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: Updating smithi190:/etc/ceph/ceph.conf 2024-08-30T08:51:33.102 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: Updating smithi190:/var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/config/ceph.conf 2024-08-30T08:51:33.102 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:33.102 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:33.102 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:33.102 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:33.102 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-08-30T08:51:33.103 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:51:33.103 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: Updating smithi190:/etc/ceph/ceph.client.admin.keyring 2024-08-30T08:51:33.103 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: Updating smithi190:/var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/config/ceph.client.admin.keyring 2024-08-30T08:51:33.103 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi042:172.21.15.42=a;smithi167:172.21.15.167=b;smithi190:172.21.15.190=c", "target": ["mon-mgr", ""]}]: dispatch 2024-08-30T08:51:33.103 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: Saving service mon spec with placement smithi042:172.21.15.42=a;smithi167:172.21.15.167=b;smithi190:172.21.15.190=c;count:3 2024-08-30T08:51:33.103 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: Deploying daemon mon.c on smithi190 2024-08-30T08:51:33.103 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-30T08:51:33.103 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: from='client.? 172.21.15.190:0/3954226789' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-08-30T08:51:33.103 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-30T08:51:33.103 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-08-30T08:51:33.103 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-30T08:51:33.103 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: mon.a calling monitor election 2024-08-30T08:51:33.103 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-30T08:51:33.103 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-30T08:51:33.104 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-30T08:51:33.104 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: mon.c calling monitor election 2024-08-30T08:51:33.104 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-30T08:51:33.104 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-30T08:51:33.104 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-30T08:51:33.104 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-30T08:51:33.104 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-08-30T08:51:33.104 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: monmap epoch 2 2024-08-30T08:51:33.104 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 2024-08-30T08:51:33.104 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: last_changed 2024-08-30T08:51:23.808828+0000 2024-08-30T08:51:33.104 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: created 2024-08-30T08:49:41.668361+0000 2024-08-30T08:51:33.104 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: min_mon_release 19 (squid) 2024-08-30T08:51:33.104 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: election_strategy: 1 2024-08-30T08:51:33.104 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: 0: [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] mon.a 2024-08-30T08:51:33.104 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: 1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.c 2024-08-30T08:51:33.105 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: fsmap 2024-08-30T08:51:33.105 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: osdmap e4: 0 total, 0 up, 0 in 2024-08-30T08:51:33.105 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: mgrmap e13: x(active, since 44s) 2024-08-30T08:51:33.105 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: overall HEALTH_OK 2024-08-30T08:51:33.105 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:33.105 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:33.105 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:33.105 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-08-30T08:51:33.105 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:51:33.105 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-30T08:51:33.105 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: Deploying daemon mon.b on smithi167 2024-08-30T08:51:33.105 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: from='client.? 172.21.15.190:0/4164405556' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-08-30T08:51:33.105 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-30T08:51:33.106 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-30T08:51:33.106 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 ceph-mon[22063]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-08-30T08:51:33.106 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 bash[22044]: 124e20337385547f2cef1d71179df9116b9312accfbded0e24f9a41b248792ab 2024-08-30T08:51:33.106 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:32 smithi167 systemd[1]: Started Ceph mon.b for b0895c1a-66ac-11ef-bcd4-c7b262605968. 2024-08-30T08:51:33.545 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-08-30T08:51:33.545 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph mon dump -f json 2024-08-30T08:51:33.787 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.c/config 2024-08-30T08:51:38.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:37 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-08-30T08:51:38.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:37 smithi167 ceph-mon[22063]: mon.a calling monitor election 2024-08-30T08:51:38.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:37 smithi167 ceph-mon[22063]: mon.c calling monitor election 2024-08-30T08:51:38.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:37 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-30T08:51:38.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:37 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-30T08:51:38.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:37 smithi167 ceph-mon[22063]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-30T08:51:38.078 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:37 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-30T08:51:38.078 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:37 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-30T08:51:38.078 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:37 smithi167 ceph-mon[22063]: mon.b calling monitor election 2024-08-30T08:51:38.078 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:37 smithi167 ceph-mon[22063]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-30T08:51:38.078 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:37 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-30T08:51:38.078 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:37 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-30T08:51:38.078 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:37 smithi167 ceph-mon[22063]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-30T08:51:38.078 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:37 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-30T08:51:38.078 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:37 smithi167 ceph-mon[22063]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-08-30T08:51:38.078 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:37 smithi167 ceph-mon[22063]: monmap epoch 3 2024-08-30T08:51:38.078 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:37 smithi167 ceph-mon[22063]: fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 2024-08-30T08:51:38.078 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:37 smithi167 ceph-mon[22063]: last_changed 2024-08-30T08:51:32.691750+0000 2024-08-30T08:51:38.078 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:37 smithi167 ceph-mon[22063]: created 2024-08-30T08:49:41.668361+0000 2024-08-30T08:51:38.078 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:37 smithi167 ceph-mon[22063]: min_mon_release 19 (squid) 2024-08-30T08:51:38.078 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:37 smithi167 ceph-mon[22063]: election_strategy: 1 2024-08-30T08:51:38.078 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:37 smithi167 ceph-mon[22063]: 0: [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] mon.a 2024-08-30T08:51:38.079 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:37 smithi167 ceph-mon[22063]: 1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.c 2024-08-30T08:51:38.079 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:37 smithi167 ceph-mon[22063]: 2: [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] mon.b 2024-08-30T08:51:38.079 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:37 smithi167 ceph-mon[22063]: fsmap 2024-08-30T08:51:38.079 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:37 smithi167 ceph-mon[22063]: osdmap e4: 0 total, 0 up, 0 in 2024-08-30T08:51:38.079 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:37 smithi167 ceph-mon[22063]: mgrmap e13: x(active, since 52s) 2024-08-30T08:51:38.079 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:37 smithi167 ceph-mon[22063]: overall HEALTH_OK 2024-08-30T08:51:38.079 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:37 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:38.079 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:37 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:38.079 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:37 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:38.079 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:37 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:38.114 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-08-30T08:51:38.114 INFO:teuthology.orchestra.run.smithi190.stdout:{"epoch":3,"fsid":"b0895c1a-66ac-11ef-bcd4-c7b262605968","modified":"2024-08-30T08:51:32.691750Z","created":"2024-08-30T08:49:41.668361Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:3300","nonce":0},{"type":"v1","addr":"172.21.15.42:6789","nonce":0}]},"addr":"172.21.15.42:6789/0","public_addr":"172.21.15.42:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3300","nonce":0},{"type":"v1","addr":"172.21.15.190:6789","nonce":0}]},"addr":"172.21.15.190:6789/0","public_addr":"172.21.15.190:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:3300","nonce":0},{"type":"v1","addr":"172.21.15.167:6789","nonce":0}]},"addr":"172.21.15.167:6789/0","public_addr":"172.21.15.167:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2024-08-30T08:51:38.114 INFO:teuthology.orchestra.run.smithi190.stderr:dumped monmap epoch 3 2024-08-30T08:51:38.115 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:37 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-08-30T08:51:38.115 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:37 smithi190 ceph-mon[22001]: mon.a calling monitor election 2024-08-30T08:51:38.115 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:37 smithi190 ceph-mon[22001]: mon.c calling monitor election 2024-08-30T08:51:38.115 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:37 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-30T08:51:38.115 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:37 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-30T08:51:38.115 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:37 smithi190 ceph-mon[22001]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-30T08:51:38.115 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:37 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-30T08:51:38.115 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:37 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-30T08:51:38.115 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:37 smithi190 ceph-mon[22001]: mon.b calling monitor election 2024-08-30T08:51:38.115 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:37 smithi190 ceph-mon[22001]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-30T08:51:38.115 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:37 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-30T08:51:38.115 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:37 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-30T08:51:38.115 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:37 smithi190 ceph-mon[22001]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-30T08:51:38.116 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:37 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-30T08:51:38.116 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:37 smithi190 ceph-mon[22001]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-08-30T08:51:38.116 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:37 smithi190 ceph-mon[22001]: monmap epoch 3 2024-08-30T08:51:38.116 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:37 smithi190 ceph-mon[22001]: fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 2024-08-30T08:51:38.116 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:37 smithi190 ceph-mon[22001]: last_changed 2024-08-30T08:51:32.691750+0000 2024-08-30T08:51:38.116 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:37 smithi190 ceph-mon[22001]: created 2024-08-30T08:49:41.668361+0000 2024-08-30T08:51:38.116 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:37 smithi190 ceph-mon[22001]: min_mon_release 19 (squid) 2024-08-30T08:51:38.116 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:37 smithi190 ceph-mon[22001]: election_strategy: 1 2024-08-30T08:51:38.116 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:37 smithi190 ceph-mon[22001]: 0: [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] mon.a 2024-08-30T08:51:38.116 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:37 smithi190 ceph-mon[22001]: 1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.c 2024-08-30T08:51:38.116 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:37 smithi190 ceph-mon[22001]: 2: [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] mon.b 2024-08-30T08:51:38.116 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:37 smithi190 ceph-mon[22001]: fsmap 2024-08-30T08:51:38.116 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:37 smithi190 ceph-mon[22001]: osdmap e4: 0 total, 0 up, 0 in 2024-08-30T08:51:38.116 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:37 smithi190 ceph-mon[22001]: mgrmap e13: x(active, since 52s) 2024-08-30T08:51:38.116 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:37 smithi190 ceph-mon[22001]: overall HEALTH_OK 2024-08-30T08:51:38.117 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:37 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:38.117 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:37 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:38.117 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:37 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:38.117 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:37 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:38.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:37 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-08-30T08:51:38.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:37 smithi042 ceph-mon[18652]: mon.a calling monitor election 2024-08-30T08:51:38.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:37 smithi042 ceph-mon[18652]: mon.c calling monitor election 2024-08-30T08:51:38.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:37 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-30T08:51:38.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:37 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-30T08:51:38.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:37 smithi042 ceph-mon[18652]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-30T08:51:38.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:37 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-30T08:51:38.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:37 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-30T08:51:38.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:37 smithi042 ceph-mon[18652]: mon.b calling monitor election 2024-08-30T08:51:38.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:37 smithi042 ceph-mon[18652]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-30T08:51:38.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:37 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-30T08:51:38.134 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:37 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-30T08:51:38.134 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:37 smithi042 ceph-mon[18652]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-30T08:51:38.134 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:37 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-30T08:51:38.134 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:37 smithi042 ceph-mon[18652]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-08-30T08:51:38.134 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:37 smithi042 ceph-mon[18652]: monmap epoch 3 2024-08-30T08:51:38.134 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:37 smithi042 ceph-mon[18652]: fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 2024-08-30T08:51:38.134 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:37 smithi042 ceph-mon[18652]: last_changed 2024-08-30T08:51:32.691750+0000 2024-08-30T08:51:38.134 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:37 smithi042 ceph-mon[18652]: created 2024-08-30T08:49:41.668361+0000 2024-08-30T08:51:38.134 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:37 smithi042 ceph-mon[18652]: min_mon_release 19 (squid) 2024-08-30T08:51:38.134 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:37 smithi042 ceph-mon[18652]: election_strategy: 1 2024-08-30T08:51:38.134 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:37 smithi042 ceph-mon[18652]: 0: [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] mon.a 2024-08-30T08:51:38.134 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:37 smithi042 ceph-mon[18652]: 1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.c 2024-08-30T08:51:38.134 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:37 smithi042 ceph-mon[18652]: 2: [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] mon.b 2024-08-30T08:51:38.134 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:37 smithi042 ceph-mon[18652]: fsmap 2024-08-30T08:51:38.134 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:37 smithi042 ceph-mon[18652]: osdmap e4: 0 total, 0 up, 0 in 2024-08-30T08:51:38.135 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:37 smithi042 ceph-mon[18652]: mgrmap e13: x(active, since 52s) 2024-08-30T08:51:38.135 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:37 smithi042 ceph-mon[18652]: overall HEALTH_OK 2024-08-30T08:51:38.135 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:37 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:38.135 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:37 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:38.135 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:37 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:38.135 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:37 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:38.817 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-08-30T08:51:38.817 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph config generate-minimal-conf 2024-08-30T08:51:38.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:38 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-30T08:51:38.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:38 smithi167 ceph-mon[22063]: from='client.? 172.21.15.190:0/1981626615' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-08-30T08:51:38.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:38 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-30T08:51:39.079 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T08:51:39.132 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:38 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-30T08:51:39.132 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:38 smithi042 ceph-mon[18652]: from='client.? 172.21.15.190:0/1981626615' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-08-30T08:51:39.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:38 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-30T08:51:39.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:38 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-30T08:51:39.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:38 smithi190 ceph-mon[22001]: from='client.? 172.21.15.190:0/1981626615' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-08-30T08:51:39.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:38 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-30T08:51:39.818 INFO:teuthology.orchestra.run.smithi042.stdout:# minimal ceph.conf for b0895c1a-66ac-11ef-bcd4-c7b262605968 2024-08-30T08:51:39.818 INFO:teuthology.orchestra.run.smithi042.stdout:[global] 2024-08-30T08:51:39.818 INFO:teuthology.orchestra.run.smithi042.stdout: fsid = b0895c1a-66ac-11ef-bcd4-c7b262605968 2024-08-30T08:51:39.818 INFO:teuthology.orchestra.run.smithi042.stdout: mon_host = [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-30T08:51:40.132 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:40 smithi042 ceph-mon[18652]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-30T08:51:40.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:40 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:40.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:40 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:40.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:40 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:40.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:40 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:40.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:40 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:51:40.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:40 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-30T08:51:40.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:40 smithi042 ceph-mon[18652]: Updating smithi042:/etc/ceph/ceph.conf 2024-08-30T08:51:40.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:40 smithi042 ceph-mon[18652]: Updating smithi167:/etc/ceph/ceph.conf 2024-08-30T08:51:40.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:40 smithi042 ceph-mon[18652]: Updating smithi190:/etc/ceph/ceph.conf 2024-08-30T08:51:40.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:40 smithi042 ceph-mon[18652]: from='client.? 172.21.15.42:0/286489989' entity='client.admin' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:51:40.282 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:40 smithi190 ceph-mon[22001]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-30T08:51:40.282 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:40 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:40.282 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:40 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:40.282 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:40 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:40.282 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:40 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:40.282 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:40 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:51:40.283 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:40 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-30T08:51:40.283 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:40 smithi190 ceph-mon[22001]: Updating smithi042:/etc/ceph/ceph.conf 2024-08-30T08:51:40.283 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:40 smithi190 ceph-mon[22001]: Updating smithi167:/etc/ceph/ceph.conf 2024-08-30T08:51:40.283 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:40 smithi190 ceph-mon[22001]: Updating smithi190:/etc/ceph/ceph.conf 2024-08-30T08:51:40.283 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:40 smithi190 ceph-mon[22001]: from='client.? 172.21.15.42:0/286489989' entity='client.admin' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:51:40.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:40 smithi167 ceph-mon[22063]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-30T08:51:40.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:40 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:40.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:40 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:40.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:40 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:40.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:40 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:40.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:40 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:51:40.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:40 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-30T08:51:40.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:40 smithi167 ceph-mon[22063]: Updating smithi042:/etc/ceph/ceph.conf 2024-08-30T08:51:40.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:40 smithi167 ceph-mon[22063]: Updating smithi167:/etc/ceph/ceph.conf 2024-08-30T08:51:40.328 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:40 smithi167 ceph-mon[22063]: Updating smithi190:/etc/ceph/ceph.conf 2024-08-30T08:51:40.328 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:40 smithi167 ceph-mon[22063]: from='client.? 172.21.15.42:0/286489989' entity='client.admin' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:51:40.555 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-08-30T08:51:40.555 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-08-30T08:51:40.555 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd of=/etc/ceph/ceph.conf 2024-08-30T08:51:40.582 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-08-30T08:51:40.582 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-08-30T08:51:40.646 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-08-30T08:51:40.647 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd of=/etc/ceph/ceph.conf 2024-08-30T08:51:40.675 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-08-30T08:51:40.676 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-08-30T08:51:40.742 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-08-30T08:51:40.742 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd of=/etc/ceph/ceph.conf 2024-08-30T08:51:40.769 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-08-30T08:51:40.770 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-08-30T08:51:40.834 INFO:tasks.cephadm:Adding mgr.x on smithi042 2024-08-30T08:51:40.835 INFO:tasks.cephadm:Adding mgr.y on smithi167 2024-08-30T08:51:40.835 INFO:tasks.cephadm:Adding mgr.z on smithi190 2024-08-30T08:51:40.835 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph orch apply mgr '3;smithi042=x;smithi167=y;smithi190=z' 2024-08-30T08:51:41.115 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.c/config 2024-08-30T08:51:41.357 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:41 smithi190 ceph-mon[22001]: Updating smithi042:/var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/config/ceph.conf 2024-08-30T08:51:41.357 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:41 smithi190 ceph-mon[22001]: Updating smithi190:/var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/config/ceph.conf 2024-08-30T08:51:41.357 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:41 smithi190 ceph-mon[22001]: Updating smithi167:/var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/config/ceph.conf 2024-08-30T08:51:41.357 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:41 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:41.358 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:41 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:41.358 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:41 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:41.358 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:41 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:41.358 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:41 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:41.358 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:41 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:41.358 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:41 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:41.358 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:41 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:41.358 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:41 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:41.358 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:41 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:41.358 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:41 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:41.358 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:41 smithi190 ceph-mon[22001]: Reconfiguring mon.a (unknown last config time)... 2024-08-30T08:51:41.358 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:41 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-08-30T08:51:41.358 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:41 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-08-30T08:51:41.359 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:41 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:51:41.359 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:41 smithi190 ceph-mon[22001]: Reconfiguring daemon mon.a on smithi042 2024-08-30T08:51:41.359 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:41 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:41.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:41 smithi042 ceph-mon[18652]: Updating smithi042:/var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/config/ceph.conf 2024-08-30T08:51:41.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:41 smithi042 ceph-mon[18652]: Updating smithi190:/var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/config/ceph.conf 2024-08-30T08:51:41.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:41 smithi042 ceph-mon[18652]: Updating smithi167:/var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/config/ceph.conf 2024-08-30T08:51:41.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:41 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:41.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:41 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:41.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:41 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:41.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:41 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:41.384 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:41 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:41.384 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:41 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:41.384 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:41 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:41.384 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:41 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:41.384 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:41 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:41.384 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:41 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:41.384 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:41 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:41.384 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:41 smithi042 ceph-mon[18652]: Reconfiguring mon.a (unknown last config time)... 2024-08-30T08:51:41.384 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:41 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-08-30T08:51:41.384 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:41 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-08-30T08:51:41.384 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:41 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:51:41.384 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:41 smithi042 ceph-mon[18652]: Reconfiguring daemon mon.a on smithi042 2024-08-30T08:51:41.384 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:41 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:41.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:41 smithi167 ceph-mon[22063]: Updating smithi042:/var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/config/ceph.conf 2024-08-30T08:51:41.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:41 smithi167 ceph-mon[22063]: Updating smithi190:/var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/config/ceph.conf 2024-08-30T08:51:41.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:41 smithi167 ceph-mon[22063]: Updating smithi167:/var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/config/ceph.conf 2024-08-30T08:51:41.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:41 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:41.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:41 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:41.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:41 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:41.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:41 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:41.578 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:41 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:41.578 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:41 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:41.578 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:41 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:41.578 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:41 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:41.578 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:41 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:41.578 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:41 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:41.578 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:41 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:41.578 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:41 smithi167 ceph-mon[22063]: Reconfiguring mon.a (unknown last config time)... 2024-08-30T08:51:41.578 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:41 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-08-30T08:51:41.578 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:41 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-08-30T08:51:41.578 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:41 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:51:41.578 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:41 smithi167 ceph-mon[22063]: Reconfiguring daemon mon.a on smithi042 2024-08-30T08:51:41.578 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:41 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:41.760 INFO:teuthology.orchestra.run.smithi190.stdout:Scheduled mgr update... 2024-08-30T08:51:42.336 DEBUG:teuthology.orchestra.run.smithi167:mgr.y> sudo journalctl -f -n 0 -u ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968@mgr.y.service 2024-08-30T08:51:42.339 DEBUG:teuthology.orchestra.run.smithi190:mgr.z> sudo journalctl -f -n 0 -u ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968@mgr.z.service 2024-08-30T08:51:42.341 INFO:tasks.cephadm:setting config [client] client_mount_timeout = 600 2024-08-30T08:51:42.341 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph config set client client_mount_timeout 600 2024-08-30T08:51:42.344 INFO:tasks.cephadm:setting config [client] debug_client = 20 2024-08-30T08:51:42.344 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph config set client debug_client 20 2024-08-30T08:51:42.387 INFO:tasks.cephadm:setting config [client] debug_ms = 1 2024-08-30T08:51:42.387 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph config set client debug_ms 1 2024-08-30T08:51:42.430 INFO:tasks.cephadm:setting config [client] rados_mon_op_timeout = 900 2024-08-30T08:51:42.430 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph config set client rados_mon_op_timeout 900 2024-08-30T08:51:42.474 INFO:tasks.cephadm:setting config [client] rados_osd_op_timeout = 900 2024-08-30T08:51:42.475 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph config set client rados_osd_op_timeout 900 2024-08-30T08:51:42.517 INFO:tasks.cephadm:setting config [mds] debug_mds = 20 2024-08-30T08:51:42.518 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph config set mds debug_mds 20 2024-08-30T08:51:42.560 INFO:tasks.cephadm:setting config [mds] debug_mds_balancer = 20 2024-08-30T08:51:42.561 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph config set mds debug_mds_balancer 20 2024-08-30T08:51:42.603 INFO:tasks.cephadm:setting config [mds] debug_ms = 2 2024-08-30T08:51:42.604 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph config set mds debug_ms 2 2024-08-30T08:51:42.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:42 smithi042 ceph-mon[18652]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-30T08:51:42.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:42 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:42.634 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:42 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:42.634 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:42 smithi042 ceph-mon[18652]: Reconfiguring mon.b (monmap changed)... 2024-08-30T08:51:42.634 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:42 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-08-30T08:51:42.634 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:42 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-08-30T08:51:42.634 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:42 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:51:42.634 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:42 smithi042 ceph-mon[18652]: Reconfiguring daemon mon.b on smithi167 2024-08-30T08:51:42.634 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:42 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:42.635 INFO:tasks.cephadm:setting config [mds] mds_bal_fragment_size_max = 10000 2024-08-30T08:51:42.635 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph config set mds mds_bal_fragment_size_max 10000 2024-08-30T08:51:42.668 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T08:51:42.677 INFO:tasks.cephadm:setting config [mds] mds_bal_merge_size = 5 2024-08-30T08:51:42.677 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph config set mds mds_bal_merge_size 5 2024-08-30T08:51:42.723 INFO:tasks.cephadm:setting config [mds] mds_bal_split_bits = 3 2024-08-30T08:51:42.723 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph config set mds mds_bal_split_bits 3 2024-08-30T08:51:42.724 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T08:51:42.770 INFO:tasks.cephadm:setting config [mds] mds_bal_split_size = 100 2024-08-30T08:51:42.770 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph config set mds mds_bal_split_size 100 2024-08-30T08:51:42.814 INFO:tasks.cephadm:setting config [mds] mds_debug_frag = True 2024-08-30T08:51:42.814 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph config set mds mds_debug_frag True 2024-08-30T08:51:42.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:42 smithi167 ceph-mon[22063]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-30T08:51:42.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:42 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:42.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:42 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:42.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:42 smithi167 ceph-mon[22063]: Reconfiguring mon.b (monmap changed)... 2024-08-30T08:51:42.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:42 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-08-30T08:51:42.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:42 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-08-30T08:51:42.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:42 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:51:42.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:42 smithi167 ceph-mon[22063]: Reconfiguring daemon mon.b on smithi167 2024-08-30T08:51:42.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:42 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:42.857 INFO:tasks.cephadm:setting config [mds] mds_debug_scatterstat = True 2024-08-30T08:51:42.857 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph config set mds mds_debug_scatterstat True 2024-08-30T08:51:42.900 INFO:tasks.cephadm:setting config [mds] mds_op_complaint_time = 180 2024-08-30T08:51:42.901 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph config set mds mds_op_complaint_time 180 2024-08-30T08:51:42.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:42 smithi190 ceph-mon[22001]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-30T08:51:42.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:42 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:42.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:42 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:42.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:42 smithi190 ceph-mon[22001]: Reconfiguring mon.b (monmap changed)... 2024-08-30T08:51:42.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:42 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-08-30T08:51:42.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:42 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-08-30T08:51:42.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:42 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:51:42.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:42 smithi190 ceph-mon[22001]: Reconfiguring daemon mon.b on smithi167 2024-08-30T08:51:42.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:42 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:42.954 INFO:tasks.cephadm:setting config [mds] mds_verify_scatter = True 2024-08-30T08:51:42.955 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph config set mds mds_verify_scatter True 2024-08-30T08:51:43.013 INFO:tasks.cephadm:setting config [mds] osd_op_complaint_time = 180 2024-08-30T08:51:43.013 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph config set mds osd_op_complaint_time 180 2024-08-30T08:51:43.058 INFO:tasks.cephadm:setting config [mds] rados_mon_op_timeout = 900 2024-08-30T08:51:43.058 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph config set mds rados_mon_op_timeout 900 2024-08-30T08:51:43.102 INFO:tasks.cephadm:setting config [mds] rados_osd_op_timeout = 900 2024-08-30T08:51:43.102 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph config set mds rados_osd_op_timeout 900 2024-08-30T08:51:43.146 INFO:tasks.cephadm:setting config [mon] mon_op_complaint_time = 120 2024-08-30T08:51:43.146 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph config set mon mon_op_complaint_time 120 2024-08-30T08:51:43.190 INFO:tasks.cephadm:setting config [mon] mon_warn_on_pool_no_app = False 2024-08-30T08:51:43.190 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph config set mon mon_warn_on_pool_no_app False 2024-08-30T08:51:43.235 INFO:tasks.cephadm:setting config [osd] osd_op_complaint_time = 180 2024-08-30T08:51:43.236 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph config set osd osd_op_complaint_time 180 2024-08-30T08:51:43.279 DEBUG:tasks.cephadm:set 22 configs 2024-08-30T08:51:43.279 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'b0895c1a-66ac-11ef-bcd4-c7b262605968', '--', 'ceph', 'config', 'set', 'client', 'client_mount_timeout', '600'], check_status=True, hostname='smithi042') 2024-08-30T08:51:43.615 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T08:51:43.811 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:43 smithi042 ceph-mon[18652]: from='client.24100 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi042=x;smithi167=y;smithi190=z", "target": ["mon-mgr", ""]}]: dispatch 2024-08-30T08:51:43.811 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:43 smithi042 ceph-mon[18652]: Saving service mgr spec with placement smithi042=x;smithi167=y;smithi190=z;count:3 2024-08-30T08:51:43.811 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:43 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:43.811 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:43 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:43.811 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:43 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-08-30T08:51:43.811 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:43 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-08-30T08:51:43.811 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:43 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:51:43.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:43 smithi167 ceph-mon[22063]: from='client.24100 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi042=x;smithi167=y;smithi190=z", "target": ["mon-mgr", ""]}]: dispatch 2024-08-30T08:51:43.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:43 smithi167 ceph-mon[22063]: Saving service mgr spec with placement smithi042=x;smithi167=y;smithi190=z;count:3 2024-08-30T08:51:43.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:43 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:43.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:43 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:43.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:43 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-08-30T08:51:43.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:43 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-08-30T08:51:43.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:43 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:51:43.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:43 smithi190 ceph-mon[22001]: from='client.24100 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi042=x;smithi167=y;smithi190=z", "target": ["mon-mgr", ""]}]: dispatch 2024-08-30T08:51:43.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:43 smithi190 ceph-mon[22001]: Saving service mgr spec with placement smithi042=x;smithi167=y;smithi190=z;count:3 2024-08-30T08:51:43.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:43 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:43.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:43 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:43.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:43 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-08-30T08:51:43.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:43 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-08-30T08:51:43.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:43 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:51:44.699 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:44 smithi042 ceph-mon[18652]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-30T08:51:44.699 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:44 smithi042 ceph-mon[18652]: Reconfiguring mon.c (monmap changed)... 2024-08-30T08:51:44.699 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:44 smithi042 ceph-mon[18652]: Reconfiguring daemon mon.c on smithi190 2024-08-30T08:51:44.700 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:44 smithi042 ceph-mon[18652]: from='client.? 172.21.15.42:0/1835584177' entity='client.admin' 2024-08-30T08:51:44.700 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:44 smithi042 ceph-mon[18652]: from='client.? 172.21.15.42:0/2425835822' entity='client.admin' 2024-08-30T08:51:44.700 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:44 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:44.700 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:44 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:44.700 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:44 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-30T08:51:44.700 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:44 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:51:44.700 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:44 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-30T08:51:44.700 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:44 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:44.700 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:44 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-08-30T08:51:44.700 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:44 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-08-30T08:51:44.700 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:44 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-08-30T08:51:44.700 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:44 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:51:44.728 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'b0895c1a-66ac-11ef-bcd4-c7b262605968', '--', 'ceph', 'config', 'set', 'client', 'debug_client', '20'], check_status=True, hostname='smithi042') 2024-08-30T08:51:44.818 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:44 smithi190 ceph-mon[22001]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-30T08:51:44.818 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:44 smithi190 ceph-mon[22001]: Reconfiguring mon.c (monmap changed)... 2024-08-30T08:51:44.818 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:44 smithi190 ceph-mon[22001]: Reconfiguring daemon mon.c on smithi190 2024-08-30T08:51:44.818 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:44 smithi190 ceph-mon[22001]: from='client.? 172.21.15.42:0/1835584177' entity='client.admin' 2024-08-30T08:51:44.818 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:44 smithi190 ceph-mon[22001]: from='client.? 172.21.15.42:0/2425835822' entity='client.admin' 2024-08-30T08:51:44.818 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:44 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:44.819 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:44 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:44.819 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:44 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-30T08:51:44.819 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:44 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:51:44.819 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:44 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-30T08:51:44.819 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:44 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:44.819 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:44 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-08-30T08:51:44.819 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:44 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-08-30T08:51:44.819 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:44 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-08-30T08:51:44.819 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:44 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:51:44.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:44 smithi167 ceph-mon[22063]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-30T08:51:44.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:44 smithi167 ceph-mon[22063]: Reconfiguring mon.c (monmap changed)... 2024-08-30T08:51:44.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:44 smithi167 ceph-mon[22063]: Reconfiguring daemon mon.c on smithi190 2024-08-30T08:51:44.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:44 smithi167 ceph-mon[22063]: from='client.? 172.21.15.42:0/1835584177' entity='client.admin' 2024-08-30T08:51:44.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:44 smithi167 ceph-mon[22063]: from='client.? 172.21.15.42:0/2425835822' entity='client.admin' 2024-08-30T08:51:44.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:44 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:44.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:44 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:44.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:44 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-30T08:51:44.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:44 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:51:44.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:44 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-30T08:51:44.828 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:44 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:44.828 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:44 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-08-30T08:51:44.828 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:44 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-08-30T08:51:44.828 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:44 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-08-30T08:51:44.828 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:44 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:51:45.161 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'b0895c1a-66ac-11ef-bcd4-c7b262605968', '--', 'ceph', 'config', 'set', 'client', 'debug_ms', '1'], check_status=True, hostname='smithi042') 2024-08-30T08:51:45.754 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:45 smithi042 ceph-mon[18652]: Deploying daemon mgr.z on smithi190 2024-08-30T08:51:45.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:45 smithi167 ceph-mon[22063]: Deploying daemon mgr.z on smithi190 2024-08-30T08:51:45.853 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T08:51:45.854 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T08:51:45.854 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T08:51:45.854 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T08:51:45.862 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T08:51:45.868 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T08:51:45.868 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T08:51:45.919 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:45 smithi190 ceph-mon[22001]: Deploying daemon mgr.z on smithi190 2024-08-30T08:51:45.973 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T08:51:45.975 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T08:51:45.975 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T08:51:45.978 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T08:51:45.979 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T08:51:45.980 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T08:51:45.980 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T08:51:45.985 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T08:51:45.998 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T08:51:46.001 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T08:51:46.003 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T08:51:46.006 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T08:51:46.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:46.037+0000 7fc9c7b07640 1 -- 172.21.15.42:0/396565395 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fc9ac03f280 msgr2=0x7fc9ac041790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:51:46.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:46.037+0000 7fc9c7b07640 1 --2- 172.21.15.42:0/396565395 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fc9ac03f280 0x7fc9ac041790 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fc9bc009bd0 tx=0x7fc9bc005f90 comp rx=0 tx=0).stop 2024-08-30T08:51:46.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:46.037+0000 7fc9c7b07640 1 -- 172.21.15.42:0/396565395 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc9c0103e60 msgr2=0x7fc9c019ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:51:46.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:46.037+0000 7fc9c7b07640 1 --2- 172.21.15.42:0/396565395 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc9c0103e60 0x7fc9c019ad20 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fc9b800eaa0 tx=0x7fc9b800ef60 comp rx=0 tx=0).stop 2024-08-30T08:51:46.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:46.038+0000 7fc9c7b07640 1 -- 172.21.15.42:0/396565395 shutdown_connections 2024-08-30T08:51:46.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:46.038+0000 7fc9c7b07640 1 --2- 172.21.15.42:0/396565395 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fc9ac03f280 0x7fc9ac041790 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:51:46.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:46.038+0000 7fc9c7b07640 1 --2- 172.21.15.42:0/396565395 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc9c0103e60 0x7fc9c019ad20 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:51:46.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:46.038+0000 7fc9c7b07640 1 --2- 172.21.15.42:0/396565395 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc9c01034c0 0x7fc9c019a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:51:46.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:46.038+0000 7fc9c7b07640 1 --2- 172.21.15.42:0/396565395 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc9c0102430 0x7fc9c0199b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:51:46.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:46.038+0000 7fc9c7b07640 1 -- 172.21.15.42:0/396565395 >> 172.21.15.42:0/396565395 conn(0x7fc9c00fdda0 msgr2=0x7fc9c00ff730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:51:46.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:46.038+0000 7fc9c7b07640 1 -- 172.21.15.42:0/396565395 shutdown_connections 2024-08-30T08:51:46.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:46.041+0000 7fc9c7b07640 1 -- 172.21.15.42:0/396565395 wait complete. 2024-08-30T08:51:46.186 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Aug 30 08:51:46 smithi190 podman[23577]: 2024-08-30 08:51:46.138187014 +0000 UTC m=+0.124562698 container create 4d9568cd493dbda9f9982cb47ed695eb535f5617f393d6f0f99b31bb7a1a3458 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mgr-z, ceph=True, io.buildah.version=1.37.1, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image) 2024-08-30T08:51:46.565 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Aug 30 08:51:46 smithi190 podman[23577]: 2024-08-30 08:51:46.382132191 +0000 UTC m=+0.368507830 container init 4d9568cd493dbda9f9982cb47ed695eb535f5617f393d6f0f99b31bb7a1a3458 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mgr-z, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.37.1, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611) 2024-08-30T08:51:46.566 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Aug 30 08:51:46 smithi190 podman[23577]: 2024-08-30 08:51:46.384935477 +0000 UTC m=+0.371311122 container start 4d9568cd493dbda9f9982cb47ed695eb535f5617f393d6f0f99b31bb7a1a3458 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mgr-z, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.1, GIT_CLEAN=True, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-08-30T08:51:46.566 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Aug 30 08:51:46 smithi190 ceph-mgr[23596]: -- 172.21.15.190:0/2091760645 <== mon.1 v2:172.21.15.190:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5650a92c50e0 con 0x5650a92e3a80 2024-08-30T08:51:46.566 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Aug 30 08:51:46 smithi190 bash[23577]: 4d9568cd493dbda9f9982cb47ed695eb535f5617f393d6f0f99b31bb7a1a3458 2024-08-30T08:51:46.566 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Aug 30 08:51:46 smithi190 systemd[1]: Started Ceph mgr.z for b0895c1a-66ac-11ef-bcd4-c7b262605968. 2024-08-30T08:51:46.825 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:46 smithi042 ceph-mon[18652]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-30T08:51:46.825 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:46 smithi042 ceph-mon[18652]: from='client.? 172.21.15.42:0/396565395' entity='client.admin' 2024-08-30T08:51:46.825 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:46 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:46.825 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:46 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:46.825 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:46 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:46.825 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:46 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-08-30T08:51:46.825 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:46 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-08-30T08:51:46.826 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:46 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-08-30T08:51:46.826 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:46 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:51:46.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:46 smithi167 ceph-mon[22063]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-30T08:51:46.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:46 smithi167 ceph-mon[22063]: from='client.? 172.21.15.42:0/396565395' entity='client.admin' 2024-08-30T08:51:46.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:46 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:46.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:46 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:46.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:46 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:46.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:46 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-08-30T08:51:46.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:46 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-08-30T08:51:46.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:46 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-08-30T08:51:46.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:46 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:51:46.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:46.880+0000 7f71a75b2640 1 -- 172.21.15.42:0/1206111928 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f71a0103470 msgr2=0x7f71a0103880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:51:46.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:46.880+0000 7f71a75b2640 1 --2- 172.21.15.42:0/1206111928 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f71a0103470 0x7f71a0103880 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f7190009900 tx=0x7f719002a480 comp rx=0 tx=0).stop 2024-08-30T08:51:46.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:46.880+0000 7f71a75b2640 1 -- 172.21.15.42:0/1206111928 shutdown_connections 2024-08-30T08:51:46.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:46.880+0000 7f71a75b2640 1 --2- 172.21.15.42:0/1206111928 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f71a0103e10 0x7f71a010a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:51:46.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:46.880+0000 7f71a75b2640 1 --2- 172.21.15.42:0/1206111928 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f71a0103470 0x7f71a0103880 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:51:46.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:46.880+0000 7f71a75b2640 1 --2- 172.21.15.42:0/1206111928 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f71a01023e0 0x7f71a0102850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:51:46.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:46.880+0000 7f71a75b2640 1 -- 172.21.15.42:0/1206111928 >> 172.21.15.42:0/1206111928 conn(0x7f71a00fdd50 msgr2=0x7f71a0100200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:51:46.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:46.880+0000 7f71a75b2640 1 -- 172.21.15.42:0/1206111928 shutdown_connections 2024-08-30T08:51:46.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:46.881+0000 7f71a75b2640 1 -- 172.21.15.42:0/1206111928 wait complete. 2024-08-30T08:51:46.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:46.881+0000 7f71a75b2640 1 Processor -- start 2024-08-30T08:51:46.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:46.881+0000 7f71a75b2640 1 -- start start 2024-08-30T08:51:46.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:46.881+0000 7f71a75b2640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f71a01023e0 0x7f71a0105fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:51:46.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:46.881+0000 7f71a75b2640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f71a0103470 0x7f71a01048a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:51:46.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:46.881+0000 7f71a75b2640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f71a0103e10 0x7f71a0104de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:51:46.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:46.881+0000 7f71a75b2640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f71a010de00 con 0x7f71a0103470 2024-08-30T08:51:46.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:46.881+0000 7f71a75b2640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f71a010dc80 con 0x7f71a01023e0 2024-08-30T08:51:46.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:46.881+0000 7f71a75b2640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f71a010df80 con 0x7f71a0103e10 2024-08-30T08:51:46.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:46.882+0000 7f71a4b4d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f71a0103470 0x7f71a01048a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:51:46.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:46.882+0000 7f71a534e640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f71a01023e0 0x7f71a0105fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:51:46.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:46.882+0000 7f71a4b4d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f71a0103470 0x7f71a01048a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:50188/0 (socket says 172.21.15.42:50188) 2024-08-30T08:51:46.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:46.882+0000 7f71a4b4d640 1 -- 172.21.15.42:0/205127670 learned_addr learned my addr 172.21.15.42:0/205127670 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T08:51:46.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:46.882+0000 7f71a5b4f640 1 --2- 172.21.15.42:0/205127670 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f71a0103e10 0x7f71a0104de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:51:46.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:46.882+0000 7f71a534e640 1 -- 172.21.15.42:0/205127670 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f71a0103e10 msgr2=0x7f71a0104de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:51:46.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:46.882+0000 7f71a534e640 1 --2- 172.21.15.42:0/205127670 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f71a0103e10 0x7f71a0104de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:51:46.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:46.882+0000 7f71a534e640 1 -- 172.21.15.42:0/205127670 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f71a0103470 msgr2=0x7f71a01048a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:51:46.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:46.882+0000 7f71a534e640 1 --2- 172.21.15.42:0/205127670 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f71a0103470 0x7f71a01048a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:51:46.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:46.882+0000 7f71a534e640 1 -- 172.21.15.42:0/205127670 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71a0105450 con 0x7f71a01023e0 2024-08-30T08:51:46.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:46.882+0000 7f71a5b4f640 1 --2- 172.21.15.42:0/205127670 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f71a0103e10 0x7f71a0104de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-30T08:51:46.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:46.882+0000 7f71a4b4d640 1 --2- 172.21.15.42:0/205127670 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f71a0103470 0x7f71a01048a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-30T08:51:46.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:46.882+0000 7f71a534e640 1 --2- 172.21.15.42:0/205127670 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f71a01023e0 0x7f71a0105fd0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f719c00c970 tx=0x7f719c00ce30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:51:46.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:46.882+0000 7f71967fc640 1 -- 172.21.15.42:0/205127670 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f719c007bb0 con 0x7f71a01023e0 2024-08-30T08:51:46.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:46.882+0000 7f71967fc640 1 -- 172.21.15.42:0/205127670 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(24 keys) ==== 1039+0+0 (secure 0 0 0) 0x7f719c007d50 con 0x7f71a01023e0 2024-08-30T08:51:46.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:46.883+0000 7f71a75b2640 1 -- 172.21.15.42:0/205127670 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71a01a3c20 con 0x7f71a01023e0 2024-08-30T08:51:46.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:46.883+0000 7f71967fc640 1 -- 172.21.15.42:0/205127670 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f719c018660 con 0x7f71a01023e0 2024-08-30T08:51:46.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:46.883+0000 7f71a75b2640 1 -- 172.21.15.42:0/205127670 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71a01a4070 con 0x7f71a01023e0 2024-08-30T08:51:46.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:46.884+0000 7f71a75b2640 1 -- 172.21.15.42:0/205127670 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71a0102850 con 0x7f71a01023e0 2024-08-30T08:51:46.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:46.884+0000 7f71967fc640 1 -- 172.21.15.42:0/205127670 <== mon.2 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 13) ==== 51253+0+0 (secure 0 0 0) 0x7f719c005020 con 0x7f71a01023e0 2024-08-30T08:51:46.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:46.884+0000 7f71967fc640 1 --2- 172.21.15.42:0/205127670 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f718c03f280 0x7f718c041790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:51:46.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:46.884+0000 7f71967fc640 1 -- 172.21.15.42:0/205127670 <== mon.2 v2:172.21.15.167:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f719c053880 con 0x7f71a01023e0 2024-08-30T08:51:46.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:46.884+0000 7f71a4b4d640 1 --2- 172.21.15.42:0/205127670 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f718c03f280 0x7f718c041790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:51:46.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:46.885+0000 7f71a4b4d640 1 --2- 172.21.15.42:0/205127670 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f718c03f280 0x7f718c041790 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f71900096c0 tx=0x7f7190005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:51:46.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:46.889+0000 7f71967fc640 1 -- 172.21.15.42:0/205127670 <== mon.2 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f719c014030 con 0x7f71a01023e0 2024-08-30T08:51:46.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:46 smithi190 ceph-mon[22001]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-30T08:51:46.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:46 smithi190 ceph-mon[22001]: from='client.? 172.21.15.42:0/396565395' entity='client.admin' 2024-08-30T08:51:46.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:46 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:46.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:46 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:46.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:46 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:46.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:46 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-08-30T08:51:46.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:46 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-08-30T08:51:46.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:46 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-08-30T08:51:46.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:46 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:51:47.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:47.078+0000 7f71a75b2640 1 -- 172.21.15.42:0/205127670 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command([{prefix=config set, name=mds_bal_merge_size}] v 0) -- 0x7f71a01a4240 con 0x7f71a01023e0 2024-08-30T08:51:47.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:47.085+0000 7f71967fc640 1 -- 172.21.15.42:0/205127670 <== mon.2 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_merge_size}]=0 v13)=0 v13) ==== 116+0+0 (secure 0 0 0) 0x7f719c008230 con 0x7f71a01023e0 2024-08-30T08:51:47.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:47.088+0000 7f71a75b2640 1 -- 172.21.15.42:0/205127670 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f718c03f280 msgr2=0x7f718c041790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:51:47.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:47.088+0000 7f71a75b2640 1 --2- 172.21.15.42:0/205127670 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f718c03f280 0x7f718c041790 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f71900096c0 tx=0x7f7190005a20 comp rx=0 tx=0).stop 2024-08-30T08:51:47.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:47.088+0000 7f71a75b2640 1 -- 172.21.15.42:0/205127670 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f71a01023e0 msgr2=0x7f71a0105fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:51:47.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:47.088+0000 7f71a75b2640 1 --2- 172.21.15.42:0/205127670 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f71a01023e0 0x7f71a0105fd0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f719c00c970 tx=0x7f719c00ce30 comp rx=0 tx=0).stop 2024-08-30T08:51:47.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:47.088+0000 7f71a75b2640 1 -- 172.21.15.42:0/205127670 shutdown_connections 2024-08-30T08:51:47.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:47.088+0000 7f71a75b2640 1 --2- 172.21.15.42:0/205127670 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f718c03f280 0x7f718c041790 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:51:47.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:47.088+0000 7f71a75b2640 1 --2- 172.21.15.42:0/205127670 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f71a0103e10 0x7f71a0104de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:51:47.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:47.088+0000 7f71a75b2640 1 --2- 172.21.15.42:0/205127670 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f71a0103470 0x7f71a01048a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:51:47.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:47.088+0000 7f71a75b2640 1 --2- 172.21.15.42:0/205127670 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f71a01023e0 0x7f71a0105fd0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:51:47.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:47.088+0000 7f71a75b2640 1 -- 172.21.15.42:0/205127670 >> 172.21.15.42:0/205127670 conn(0x7f71a00fdd50 msgr2=0x7f71a00ff6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:51:47.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:47.088+0000 7f71a75b2640 1 -- 172.21.15.42:0/205127670 shutdown_connections 2024-08-30T08:51:47.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:47.088+0000 7f71a75b2640 1 -- 172.21.15.42:0/205127670 wait complete. 2024-08-30T08:51:47.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:47.517+0000 7f561d4d1640 1 -- 172.21.15.42:0/1929947526 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f56181034a0 msgr2=0x7f56181038b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:51:47.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:47.517+0000 7f561d4d1640 1 --2- 172.21.15.42:0/1929947526 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f56181034a0 0x7f56181038b0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f5600009900 tx=0x7f560002a480 comp rx=0 tx=0).stop 2024-08-30T08:51:47.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:47.517+0000 7f561d4d1640 1 -- 172.21.15.42:0/1929947526 shutdown_connections 2024-08-30T08:51:47.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:47.517+0000 7f561d4d1640 1 --2- 172.21.15.42:0/1929947526 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5618103e40 0x7f561810a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:51:47.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:47.517+0000 7f561d4d1640 1 --2- 172.21.15.42:0/1929947526 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f56181034a0 0x7f56181038b0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:51:47.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:47.517+0000 7f561d4d1640 1 --2- 172.21.15.42:0/1929947526 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5618102410 0x7f5618102880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:51:47.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:47.517+0000 7f561d4d1640 1 -- 172.21.15.42:0/1929947526 >> 172.21.15.42:0/1929947526 conn(0x7f56180fdd80 msgr2=0x7f5618100230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:51:47.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:47.517+0000 7f561d4d1640 1 -- 172.21.15.42:0/1929947526 shutdown_connections 2024-08-30T08:51:47.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:47.517+0000 7f561d4d1640 1 -- 172.21.15.42:0/1929947526 wait complete. 2024-08-30T08:51:47.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:47.518+0000 7f561d4d1640 1 Processor -- start 2024-08-30T08:51:47.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:47.518+0000 7f561d4d1640 1 -- start start 2024-08-30T08:51:47.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:47.518+0000 7f561d4d1640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5618102410 0x7f5618199c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:51:47.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:47.518+0000 7f561d4d1640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f56181034a0 0x7f561819a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:51:47.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:47.518+0000 7f561d4d1640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5618103e40 0x7f561819ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:51:47.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:47.518+0000 7f561d4d1640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f561810de30 con 0x7f56181034a0 2024-08-30T08:51:47.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:47.518+0000 7f561d4d1640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f561810dcb0 con 0x7f5618103e40 2024-08-30T08:51:47.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:47.518+0000 7f561d4d1640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f561810dfb0 con 0x7f5618102410 2024-08-30T08:51:47.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:47.518+0000 7f56167fc640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f56181034a0 0x7f561819a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:51:47.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:47.518+0000 7f5616ffd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5618102410 0x7f5618199c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:51:47.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:47.518+0000 7f56167fc640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f56181034a0 0x7f561819a2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:50202/0 (socket says 172.21.15.42:50202) 2024-08-30T08:51:47.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:47.518+0000 7f56167fc640 1 -- 172.21.15.42:0/4109209989 learned_addr learned my addr 172.21.15.42:0/4109209989 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T08:51:47.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:47.518+0000 7f56177fe640 1 --2- 172.21.15.42:0/4109209989 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5618103e40 0x7f561819ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:51:47.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:47.519+0000 7f56167fc640 1 -- 172.21.15.42:0/4109209989 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5618102410 msgr2=0x7f5618199c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:51:47.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:47.519+0000 7f56167fc640 1 --2- 172.21.15.42:0/4109209989 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5618102410 0x7f5618199c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:51:47.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:47.519+0000 7f56167fc640 1 -- 172.21.15.42:0/4109209989 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5618103e40 msgr2=0x7f561819ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:51:47.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:47.519+0000 7f56167fc640 1 --2- 172.21.15.42:0/4109209989 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5618103e40 0x7f561819ae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:51:47.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:47.519+0000 7f56167fc640 1 -- 172.21.15.42:0/4109209989 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f561819b5d0 con 0x7f56181034a0 2024-08-30T08:51:47.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:47.519+0000 7f5616ffd640 1 --2- 172.21.15.42:0/4109209989 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5618102410 0x7f5618199c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-30T08:51:47.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:47.519+0000 7f56167fc640 1 --2- 172.21.15.42:0/4109209989 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f56181034a0 0x7f561819a2a0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f5600006010 tx=0x7f5600003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:51:47.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:47.519+0000 7f560ffff640 1 -- 172.21.15.42:0/4109209989 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f560003f070 con 0x7f56181034a0 2024-08-30T08:51:47.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:47.519+0000 7f561d4d1640 1 -- 172.21.15.42:0/4109209989 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f561819b860 con 0x7f56181034a0 2024-08-30T08:51:47.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:47.519+0000 7f561d4d1640 1 -- 172.21.15.42:0/4109209989 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56181a1e80 con 0x7f56181034a0 2024-08-30T08:51:47.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:47.519+0000 7f560ffff640 1 -- 172.21.15.42:0/4109209989 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(24 keys) ==== 1039+0+0 (secure 0 0 0) 0x7f5600031d20 con 0x7f56181034a0 2024-08-30T08:51:47.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:47.519+0000 7f560ffff640 1 -- 172.21.15.42:0/4109209989 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5600035030 con 0x7f56181034a0 2024-08-30T08:51:47.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:47.520+0000 7f561d4d1640 1 -- 172.21.15.42:0/4109209989 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55f80050b0 con 0x7f56181034a0 2024-08-30T08:51:47.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:47.520+0000 7f560ffff640 1 -- 172.21.15.42:0/4109209989 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 13) ==== 51253+0+0 (secure 0 0 0) 0x7f5600042070 con 0x7f56181034a0 2024-08-30T08:51:47.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:47.521+0000 7f560ffff640 1 --2- 172.21.15.42:0/4109209989 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f560403f280 0x7f5604041790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:51:47.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:47.521+0000 7f560ffff640 1 -- 172.21.15.42:0/4109209989 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f56000741c0 con 0x7f56181034a0 2024-08-30T08:51:47.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:47.521+0000 7f5616ffd640 1 --2- 172.21.15.42:0/4109209989 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f560403f280 0x7f5604041790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:51:47.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:47.521+0000 7f5616ffd640 1 --2- 172.21.15.42:0/4109209989 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f560403f280 0x7f5604041790 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f5608009fc0 tx=0x7f5608005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:51:47.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:47.526+0000 7f560ffff640 1 -- 172.21.15.42:0/4109209989 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f5600042360 con 0x7f56181034a0 2024-08-30T08:51:47.702 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:47 smithi042 ceph-mon[18652]: Deploying daemon mgr.y on smithi167 2024-08-30T08:51:47.702 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:47 smithi042 ceph-mon[18652]: from='client.? ' entity='client.admin' 2024-08-30T08:51:47.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:47.723+0000 7f561d4d1640 1 -- 172.21.15.42:0/4109209989 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command([{prefix=config set, name=mds_debug_frag}] v 0) -- 0x7f55f80053a0 con 0x7f56181034a0 2024-08-30T08:51:47.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:47.728+0000 7f560ffff640 1 -- 172.21.15.42:0/4109209989 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_debug_frag}]=0 v14)=0 v14) ==== 115+0+0 (secure 0 0 0) 0x7f560004a260 con 0x7f56181034a0 2024-08-30T08:51:47.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:47.730+0000 7f561d4d1640 1 -- 172.21.15.42:0/4109209989 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f560403f280 msgr2=0x7f5604041790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:51:47.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:47.730+0000 7f561d4d1640 1 --2- 172.21.15.42:0/4109209989 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f560403f280 0x7f5604041790 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f5608009fc0 tx=0x7f5608005f90 comp rx=0 tx=0).stop 2024-08-30T08:51:47.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:47.730+0000 7f561d4d1640 1 -- 172.21.15.42:0/4109209989 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f56181034a0 msgr2=0x7f561819a2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:51:47.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:47.730+0000 7f561d4d1640 1 --2- 172.21.15.42:0/4109209989 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f56181034a0 0x7f561819a2a0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f5600006010 tx=0x7f5600003c10 comp rx=0 tx=0).stop 2024-08-30T08:51:47.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:47.731+0000 7f561d4d1640 1 -- 172.21.15.42:0/4109209989 shutdown_connections 2024-08-30T08:51:47.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:47.731+0000 7f561d4d1640 1 --2- 172.21.15.42:0/4109209989 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f560403f280 0x7f5604041790 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:51:47.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:47.731+0000 7f561d4d1640 1 --2- 172.21.15.42:0/4109209989 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5618103e40 0x7f561819ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:51:47.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:47.731+0000 7f561d4d1640 1 --2- 172.21.15.42:0/4109209989 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f56181034a0 0x7f561819a2a0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:51:47.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:47.731+0000 7f561d4d1640 1 --2- 172.21.15.42:0/4109209989 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5618102410 0x7f5618199c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:51:47.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:47.731+0000 7f561d4d1640 1 -- 172.21.15.42:0/4109209989 >> 172.21.15.42:0/4109209989 conn(0x7f56180fdd80 msgr2=0x7f56180ff7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:51:47.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:47.731+0000 7f561d4d1640 1 -- 172.21.15.42:0/4109209989 shutdown_connections 2024-08-30T08:51:47.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:47.731+0000 7f561d4d1640 1 -- 172.21.15.42:0/4109209989 wait complete. 2024-08-30T08:51:47.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:47 smithi167 ceph-mon[22063]: Deploying daemon mgr.y on smithi167 2024-08-30T08:51:47.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:47 smithi167 ceph-mon[22063]: from='client.? ' entity='client.admin' 2024-08-30T08:51:47.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:47 smithi190 ceph-mon[22001]: Deploying daemon mgr.y on smithi167 2024-08-30T08:51:47.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:47 smithi190 ceph-mon[22001]: from='client.? ' entity='client.admin' 2024-08-30T08:51:48.572 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:48 smithi167 ceph-mon[22063]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-30T08:51:48.572 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:48 smithi167 ceph-mon[22063]: from='client.? 172.21.15.42:0/4109209989' entity='client.admin' 2024-08-30T08:51:48.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:48 smithi167 ceph-mon[22063]: from='client.? 172.21.15.42:0/3761656594' entity='client.admin' 2024-08-30T08:51:48.827 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Aug 30 08:51:48 smithi167 podman[23186]: 2024-08-30 08:51:48.815725013 +0000 UTC m=+0.341396573 container start f497860332d64d70cbaf577853b0328a857a63b912d97ba81cd114c3767d6165 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mgr-y, org.label-schema.build-date=20240731, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.1, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image) 2024-08-30T08:51:48.830 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:48 smithi042 ceph-mon[18652]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-30T08:51:48.830 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:48 smithi042 ceph-mon[18652]: from='client.? 172.21.15.42:0/4109209989' entity='client.admin' 2024-08-30T08:51:48.830 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:48 smithi042 ceph-mon[18652]: from='client.? 172.21.15.42:0/3761656594' entity='client.admin' 2024-08-30T08:51:48.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:48 smithi190 ceph-mon[22001]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-30T08:51:48.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:48 smithi190 ceph-mon[22001]: from='client.? 172.21.15.42:0/4109209989' entity='client.admin' 2024-08-30T08:51:48.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:48 smithi190 ceph-mon[22001]: from='client.? 172.21.15.42:0/3761656594' entity='client.admin' 2024-08-30T08:51:48.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:48.922+0000 7f0dbb59e640 1 -- 172.21.15.42:0/3928318772 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0dbc06c0e0 msgr2=0x7f0dbc105fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:51:48.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:48.922+0000 7f0dbb59e640 1 --2- 172.21.15.42:0/3928318772 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0dbc06c0e0 0x7f0dbc105fb0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f0dac009900 tx=0x7f0dac02a410 comp rx=0 tx=0).stop 2024-08-30T08:51:48.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:48.922+0000 7f0dbb59e640 1 -- 172.21.15.42:0/3928318772 shutdown_connections 2024-08-30T08:51:48.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:48.922+0000 7f0dbb59e640 1 --2- 172.21.15.42:0/3928318772 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0dbc109090 0x7f0dbc10b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:51:48.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:48.922+0000 7f0dbb59e640 1 --2- 172.21.15.42:0/3928318772 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0dbc1064f0 0x7f0dbc108950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:51:48.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:48.922+0000 7f0dbb59e640 1 --2- 172.21.15.42:0/3928318772 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0dbc06c0e0 0x7f0dbc105fb0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:51:48.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:48.922+0000 7f0dbb59e640 1 -- 172.21.15.42:0/3928318772 >> 172.21.15.42:0/3928318772 conn(0x7f0dbc0fdbf0 msgr2=0x7f0dbc100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:51:48.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:48.922+0000 7f0dbb59e640 1 -- 172.21.15.42:0/3928318772 shutdown_connections 2024-08-30T08:51:48.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:48.922+0000 7f0dbb59e640 1 -- 172.21.15.42:0/3928318772 wait complete. 2024-08-30T08:51:48.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:48.923+0000 7f0dbb59e640 1 Processor -- start 2024-08-30T08:51:48.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:48.923+0000 7f0dbb59e640 1 -- start start 2024-08-30T08:51:48.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:48.923+0000 7f0dbb59e640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0dbc06c0e0 0x7f0dbc199a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:51:48.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:48.923+0000 7f0dbb59e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0dbc1064f0 0x7f0dbc19a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:51:48.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:48.923+0000 7f0dbb59e640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0dbc109090 0x7f0dbc19ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:51:48.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:48.924+0000 7f0dbb59e640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f0dbc10ed70 con 0x7f0dbc06c0e0 2024-08-30T08:51:48.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:48.924+0000 7f0dbb59e640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f0dbc10ebf0 con 0x7f0dbc109090 2024-08-30T08:51:48.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:48.924+0000 7f0dbb59e640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f0dbc10eef0 con 0x7f0dbc1064f0 2024-08-30T08:51:48.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:48.924+0000 7f0dba59c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0dbc06c0e0 0x7f0dbc199a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:51:48.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:48.924+0000 7f0dba59c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0dbc06c0e0 0x7f0dbc199a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:39272/0 (socket says 172.21.15.42:39272) 2024-08-30T08:51:48.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:48.924+0000 7f0dba59c640 1 -- 172.21.15.42:0/3654450464 learned_addr learned my addr 172.21.15.42:0/3654450464 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T08:51:48.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:48.924+0000 7f0db9d9b640 1 --2- 172.21.15.42:0/3654450464 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0dbc1064f0 0x7f0dbc19a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:51:48.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:48.924+0000 7f0dbad9d640 1 --2- 172.21.15.42:0/3654450464 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0dbc109090 0x7f0dbc19ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:51:48.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:48.924+0000 7f0dba59c640 1 -- 172.21.15.42:0/3654450464 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0dbc1064f0 msgr2=0x7f0dbc19a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:51:48.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:48.924+0000 7f0dba59c640 1 --2- 172.21.15.42:0/3654450464 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0dbc1064f0 0x7f0dbc19a0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:51:48.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:48.924+0000 7f0dba59c640 1 -- 172.21.15.42:0/3654450464 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0dbc109090 msgr2=0x7f0dbc19ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:51:48.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:48.924+0000 7f0dba59c640 1 --2- 172.21.15.42:0/3654450464 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0dbc109090 0x7f0dbc19ac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:51:48.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:48.924+0000 7f0dba59c640 1 -- 172.21.15.42:0/3654450464 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0dbc19b460 con 0x7f0dbc06c0e0 2024-08-30T08:51:48.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:48.924+0000 7f0dba59c640 1 --2- 172.21.15.42:0/3654450464 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0dbc06c0e0 0x7f0dbc199a30 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f0dac006010 tx=0x7f0dac003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:51:48.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:48.924+0000 7f0dab7fe640 1 -- 172.21.15.42:0/3654450464 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dac03f070 con 0x7f0dbc06c0e0 2024-08-30T08:51:48.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:48.924+0000 7f0dab7fe640 1 -- 172.21.15.42:0/3654450464 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(24 keys) ==== 1039+0+0 (secure 0 0 0) 0x7f0dac031d20 con 0x7f0dbc06c0e0 2024-08-30T08:51:48.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:48.924+0000 7f0dbb59e640 1 -- 172.21.15.42:0/3654450464 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0dbc19b6f0 con 0x7f0dbc06c0e0 2024-08-30T08:51:48.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:48.924+0000 7f0dab7fe640 1 -- 172.21.15.42:0/3654450464 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dac035030 con 0x7f0dbc06c0e0 2024-08-30T08:51:48.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:48.924+0000 7f0dbb59e640 1 -- 172.21.15.42:0/3654450464 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0dbc1a1e50 con 0x7f0dbc06c0e0 2024-08-30T08:51:48.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:48.925+0000 7f0dbb59e640 1 -- 172.21.15.42:0/3654450464 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0dbc10f4d0 con 0x7f0dbc06c0e0 2024-08-30T08:51:48.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:48.925+0000 7f0dab7fe640 1 -- 172.21.15.42:0/3654450464 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 13) ==== 51253+0+0 (secure 0 0 0) 0x7f0dac042070 con 0x7f0dbc06c0e0 2024-08-30T08:51:48.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:48.926+0000 7f0dab7fe640 1 --2- 172.21.15.42:0/3654450464 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f0da403f280 0x7f0da4041790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:51:48.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:48.926+0000 7f0dab7fe640 1 -- 172.21.15.42:0/3654450464 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f0dac0742a0 con 0x7f0dbc06c0e0 2024-08-30T08:51:48.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:48.926+0000 7f0db9d9b640 1 --2- 172.21.15.42:0/3654450464 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f0da403f280 0x7f0da4041790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:51:48.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:48.926+0000 7f0db9d9b640 1 --2- 172.21.15.42:0/3654450464 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f0da403f280 0x7f0da4041790 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f0db0009bd0 tx=0x7f0db0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:51:48.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:48.931+0000 7f0dab7fe640 1 -- 172.21.15.42:0/3654450464 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f0dac04a260 con 0x7f0dbc06c0e0 2024-08-30T08:51:49.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:49.124+0000 7f859516c640 1 -- 172.21.15.42:0/384934353 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8590106680 msgr2=0x7f8590108ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:51:49.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:49.124+0000 7f859516c640 1 --2- 172.21.15.42:0/384934353 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8590106680 0x7f8590108ae0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f8578009900 tx=0x7f857802a410 comp rx=0 tx=0).stop 2024-08-30T08:51:49.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:49.124+0000 7f859516c640 1 -- 172.21.15.42:0/384934353 shutdown_connections 2024-08-30T08:51:49.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:49.124+0000 7f859516c640 1 --2- 172.21.15.42:0/384934353 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8590109220 0x7f859010b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:51:49.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:49.124+0000 7f859516c640 1 --2- 172.21.15.42:0/384934353 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8590106680 0x7f8590108ae0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:51:49.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:49.124+0000 7f859516c640 1 --2- 172.21.15.42:0/384934353 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f859006c400 0x7f8590106140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:51:49.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:49.124+0000 7f859516c640 1 -- 172.21.15.42:0/384934353 >> 172.21.15.42:0/384934353 conn(0x7f85900fdd80 msgr2=0x7f8590100210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:51:49.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:49.124+0000 7f859516c640 1 -- 172.21.15.42:0/384934353 shutdown_connections 2024-08-30T08:51:49.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:49.124+0000 7f859516c640 1 -- 172.21.15.42:0/384934353 wait complete. 2024-08-30T08:51:49.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:49.125+0000 7f859516c640 1 Processor -- start 2024-08-30T08:51:49.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:49.125+0000 7f859516c640 1 -- start start 2024-08-30T08:51:49.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:49.125+0000 7f859516c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f859006c400 0x7f8590199bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:51:49.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:49.125+0000 7f859516c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8590106680 0x7f859019a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:51:49.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:49.125+0000 7f859516c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8590109220 0x7f859019adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:51:49.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:49.125+0000 7f859516c640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f859010eea0 con 0x7f8590106680 2024-08-30T08:51:49.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:49.125+0000 7f859516c640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f859010ed20 con 0x7f859006c400 2024-08-30T08:51:49.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:49.125+0000 7f859516c640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f859010f020 con 0x7f8590109220 2024-08-30T08:51:49.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:49.125+0000 7f8587fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8590106680 0x7f859019a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:51:49.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:49.125+0000 7f8587fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8590106680 0x7f859019a220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:39300/0 (socket says 172.21.15.42:39300) 2024-08-30T08:51:49.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:49.125+0000 7f8587fff640 1 -- 172.21.15.42:0/2767826657 learned_addr learned my addr 172.21.15.42:0/2767826657 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T08:51:49.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:49.125+0000 7f858f59e640 1 --2- 172.21.15.42:0/2767826657 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8590109220 0x7f859019adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:51:49.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:49.125+0000 7f858ed9d640 1 --2- 172.21.15.42:0/2767826657 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f859006c400 0x7f8590199bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:51:49.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:49.125+0000 7f8587fff640 1 -- 172.21.15.42:0/2767826657 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8590109220 msgr2=0x7f859019adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:51:49.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:49.125+0000 7f8587fff640 1 --2- 172.21.15.42:0/2767826657 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8590109220 0x7f859019adc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:51:49.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:49.125+0000 7f8587fff640 1 -- 172.21.15.42:0/2767826657 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f859006c400 msgr2=0x7f8590199bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:51:49.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:49.125+0000 7f8587fff640 1 --2- 172.21.15.42:0/2767826657 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f859006c400 0x7f8590199bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:51:49.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:49.125+0000 7f8587fff640 1 -- 172.21.15.42:0/2767826657 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f859019b5e0 con 0x7f8590106680 2024-08-30T08:51:49.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:49.126+0000 7f8587fff640 1 --2- 172.21.15.42:0/2767826657 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8590106680 0x7f859019a220 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f8578006010 tx=0x7f8578003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:51:49.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:49.126+0000 7f858cd99640 1 -- 172.21.15.42:0/2767826657 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f857803f070 con 0x7f8590106680 2024-08-30T08:51:49.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:49.126+0000 7f858cd99640 1 -- 172.21.15.42:0/2767826657 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(24 keys) ==== 1039+0+0 (secure 0 0 0) 0x7f8578031b20 con 0x7f8590106680 2024-08-30T08:51:49.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:49.126+0000 7f859516c640 1 -- 172.21.15.42:0/2767826657 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f859019b870 con 0x7f8590106680 2024-08-30T08:51:49.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:49.126+0000 7f859516c640 1 -- 172.21.15.42:0/2767826657 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85901a1fd0 con 0x7f8590106680 2024-08-30T08:51:49.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:49.126+0000 7f858cd99640 1 -- 172.21.15.42:0/2767826657 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8578035030 con 0x7f8590106680 2024-08-30T08:51:49.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:49.127+0000 7f858cd99640 1 -- 172.21.15.42:0/2767826657 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 13) ==== 51253+0+0 (secure 0 0 0) 0x7f8578042070 con 0x7f8590106680 2024-08-30T08:51:49.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:49.127+0000 7f859516c640 1 -- 172.21.15.42:0/2767826657 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8590106ad0 con 0x7f8590106680 2024-08-30T08:51:49.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:49.127+0000 7f858cd99640 1 --2- 172.21.15.42:0/2767826657 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f857c03f280 0x7f857c041790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:51:49.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:49.127+0000 7f858cd99640 1 -- 172.21.15.42:0/2767826657 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f85780741f0 con 0x7f8590106680 2024-08-30T08:51:49.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:49.128+0000 7f858ed9d640 1 --2- 172.21.15.42:0/2767826657 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f857c03f280 0x7f857c041790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:51:49.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:49.128+0000 7f858ed9d640 1 --2- 172.21.15.42:0/2767826657 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f857c03f280 0x7f857c041790 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f8580009bd0 tx=0x7f8580005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:51:49.135 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:49.133+0000 7f858cd99640 1 -- 172.21.15.42:0/2767826657 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f8578042360 con 0x7f8590106680 2024-08-30T08:51:49.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:49.141+0000 7f0dbb59e640 1 -- 172.21.15.42:0/3654450464 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command([{prefix=config set, name=debug_mds}] v 0) -- 0x7f0dbc0ff710 con 0x7f0dbc06c0e0 2024-08-30T08:51:49.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:49.146+0000 7f0dab7fe640 1 -- 172.21.15.42:0/3654450464 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_mds}]=0 v16)=0 v16) ==== 108+0+0 (secure 0 0 0) 0x7f0dac03b460 con 0x7f0dbc06c0e0 2024-08-30T08:51:49.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:49.148+0000 7f0dbb59e640 1 -- 172.21.15.42:0/3654450464 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f0da403f280 msgr2=0x7f0da4041790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:51:49.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:49.148+0000 7f0dbb59e640 1 --2- 172.21.15.42:0/3654450464 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f0da403f280 0x7f0da4041790 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f0db0009bd0 tx=0x7f0db0005f90 comp rx=0 tx=0).stop 2024-08-30T08:51:49.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:49.148+0000 7f0dbb59e640 1 -- 172.21.15.42:0/3654450464 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0dbc06c0e0 msgr2=0x7f0dbc199a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:51:49.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:49.148+0000 7f0dbb59e640 1 --2- 172.21.15.42:0/3654450464 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0dbc06c0e0 0x7f0dbc199a30 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f0dac006010 tx=0x7f0dac003c10 comp rx=0 tx=0).stop 2024-08-30T08:51:49.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:49.148+0000 7f0dbb59e640 1 -- 172.21.15.42:0/3654450464 shutdown_connections 2024-08-30T08:51:49.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:49.148+0000 7f0dbb59e640 1 --2- 172.21.15.42:0/3654450464 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f0da403f280 0x7f0da4041790 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:51:49.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:49.149+0000 7f0dbb59e640 1 --2- 172.21.15.42:0/3654450464 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0dbc1064f0 0x7f0dbc19a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:51:49.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:49.149+0000 7f0dbb59e640 1 --2- 172.21.15.42:0/3654450464 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0dbc109090 0x7f0dbc19ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:51:49.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:49.149+0000 7f0dbb59e640 1 --2- 172.21.15.42:0/3654450464 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0dbc06c0e0 0x7f0dbc199a30 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:51:49.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:49.149+0000 7f0dbb59e640 1 -- 172.21.15.42:0/3654450464 >> 172.21.15.42:0/3654450464 conn(0x7f0dbc0fdbf0 msgr2=0x7f0dbc104a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:51:49.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:49.149+0000 7f0dbb59e640 1 -- 172.21.15.42:0/3654450464 shutdown_connections 2024-08-30T08:51:49.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:49.149+0000 7f0dbb59e640 1 -- 172.21.15.42:0/3654450464 wait complete. 2024-08-30T08:51:49.327 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Aug 30 08:51:48 smithi167 ceph-mgr[23205]: -- 172.21.15.167:0/2810735861 <== mon.2 v2:172.21.15.167:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55698b2f90e0 con 0x556990098480 2024-08-30T08:51:49.327 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Aug 30 08:51:48 smithi167 bash[23186]: f497860332d64d70cbaf577853b0328a857a63b912d97ba81cd114c3767d6165 2024-08-30T08:51:49.327 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Aug 30 08:51:48 smithi167 systemd[1]: Started Ceph mgr.y for b0895c1a-66ac-11ef-bcd4-c7b262605968. 2024-08-30T08:51:49.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:49.330+0000 7f859516c640 1 -- 172.21.15.42:0/2767826657 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command([{prefix=config set, name=rados_osd_op_timeout}] v 0) -- 0x7f85901a21a0 con 0x7f8590106680 2024-08-30T08:51:49.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:49.335+0000 7f858cd99640 1 -- 172.21.15.42:0/2767826657 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{prefix=config set, name=rados_osd_op_timeout}]=0 v17)=0 v17) ==== 123+0+0 (secure 0 0 0) 0x7f8578039df0 con 0x7f8590106680 2024-08-30T08:51:49.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:49.336+0000 7f858cd99640 1 -- 172.21.15.42:0/2767826657 <== mon.0 v2:172.21.15.42:3300/0 8 ==== config(25 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f857803b460 con 0x7f8590106680 2024-08-30T08:51:49.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:49.337+0000 7f859516c640 1 -- 172.21.15.42:0/2767826657 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f857c03f280 msgr2=0x7f857c041790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:51:49.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:49.337+0000 7f859516c640 1 --2- 172.21.15.42:0/2767826657 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f857c03f280 0x7f857c041790 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f8580009bd0 tx=0x7f8580005f90 comp rx=0 tx=0).stop 2024-08-30T08:51:49.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:49.337+0000 7f859516c640 1 -- 172.21.15.42:0/2767826657 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8590106680 msgr2=0x7f859019a220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:51:49.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:49.337+0000 7f859516c640 1 --2- 172.21.15.42:0/2767826657 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8590106680 0x7f859019a220 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f8578006010 tx=0x7f8578003c10 comp rx=0 tx=0).stop 2024-08-30T08:51:49.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:49.338+0000 7f859516c640 1 -- 172.21.15.42:0/2767826657 shutdown_connections 2024-08-30T08:51:49.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:49.338+0000 7f859516c640 1 --2- 172.21.15.42:0/2767826657 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f857c03f280 0x7f857c041790 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:51:49.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:49.338+0000 7f859516c640 1 --2- 172.21.15.42:0/2767826657 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8590109220 0x7f859019adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:51:49.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:49.338+0000 7f859516c640 1 --2- 172.21.15.42:0/2767826657 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8590106680 0x7f859019a220 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:51:49.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:49.338+0000 7f859516c640 1 --2- 172.21.15.42:0/2767826657 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f859006c400 0x7f8590199bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:51:49.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:49.338+0000 7f859516c640 1 -- 172.21.15.42:0/2767826657 >> 172.21.15.42:0/2767826657 conn(0x7f85900fdd80 msgr2=0x7f859010b640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:51:49.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:49.338+0000 7f859516c640 1 -- 172.21.15.42:0/2767826657 shutdown_connections 2024-08-30T08:51:49.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:49.338+0000 7f859516c640 1 -- 172.21.15.42:0/2767826657 wait complete. 2024-08-30T08:51:50.302 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:49 smithi042 ceph-mon[18652]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-30T08:51:50.302 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:49 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:50.302 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:49 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:50.302 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:49 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:50.302 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:49 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:50.302 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:49 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-30T08:51:50.302 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:49 smithi042 ceph-mon[18652]: from='client.? 172.21.15.42:0/3654450464' entity='client.admin' 2024-08-30T08:51:50.302 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:49 smithi042 ceph-mon[18652]: from='client.? 172.21.15.42:0/2767826657' entity='client.admin' 2024-08-30T08:51:50.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:49 smithi167 ceph-mon[22063]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-30T08:51:50.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:49 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:50.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:49 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:50.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:49 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:50.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:49 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:50.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:49 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-30T08:51:50.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:49 smithi167 ceph-mon[22063]: from='client.? 172.21.15.42:0/3654450464' entity='client.admin' 2024-08-30T08:51:50.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:49 smithi167 ceph-mon[22063]: from='client.? 172.21.15.42:0/2767826657' entity='client.admin' 2024-08-30T08:51:50.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:49 smithi190 ceph-mon[22001]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-30T08:51:50.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:49 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:50.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:49 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:50.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:49 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:50.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:49 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:50.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:49 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-30T08:51:50.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:49 smithi190 ceph-mon[22001]: from='client.? 172.21.15.42:0/3654450464' entity='client.admin' 2024-08-30T08:51:50.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:49 smithi190 ceph-mon[22001]: from='client.? 172.21.15.42:0/2767826657' entity='client.admin' 2024-08-30T08:51:51.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:47.920+0000 7f2ecd8e3640 1 -- 172.21.15.42:0/2506890344 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2ec81023e0 msgr2=0x7f2ec8102850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:51:51.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:47.920+0000 7f2ecd8e3640 1 --2- 172.21.15.42:0/2506890344 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2ec81023e0 0x7f2ec8102850 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f2eb8009900 tx=0x7f2eb802a480 comp rx=0 tx=0).stop 2024-08-30T08:51:51.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:47.920+0000 7f2ecd8e3640 1 -- 172.21.15.42:0/2506890344 shutdown_connections 2024-08-30T08:51:51.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:47.920+0000 7f2ecd8e3640 1 --2- 172.21.15.42:0/2506890344 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2ec8103470 0x7f2ec8103880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:51:51.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:47.920+0000 7f2ecd8e3640 1 --2- 172.21.15.42:0/2506890344 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2ec8103e10 0x7f2ec810a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:51:51.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:47.920+0000 7f2ecd8e3640 1 --2- 172.21.15.42:0/2506890344 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2ec81023e0 0x7f2ec8102850 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:51:51.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:47.920+0000 7f2ecd8e3640 1 -- 172.21.15.42:0/2506890344 >> 172.21.15.42:0/2506890344 conn(0x7f2ec80fdd50 msgr2=0x7f2ec8100200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:51:51.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:47.920+0000 7f2ecd8e3640 1 -- 172.21.15.42:0/2506890344 shutdown_connections 2024-08-30T08:51:51.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:47.920+0000 7f2ecd8e3640 1 -- 172.21.15.42:0/2506890344 wait complete. 2024-08-30T08:51:51.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:47.921+0000 7f2ecd8e3640 1 Processor -- start 2024-08-30T08:51:51.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:47.921+0000 7f2ecd8e3640 1 -- start start 2024-08-30T08:51:51.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:47.921+0000 7f2ecd8e3640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2ec81023e0 0x7f2ec8199be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:51:51.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:47.921+0000 7f2ecd8e3640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2ec8103470 0x7f2ec819a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:51:51.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:47.921+0000 7f2ecd8e3640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2ec8103e10 0x7f2ec819adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:51:51.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:47.921+0000 7f2ecd8e3640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f2ec810de50 con 0x7f2ec8103e10 2024-08-30T08:51:51.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:47.921+0000 7f2ecd8e3640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f2ec810dcd0 con 0x7f2ec81023e0 2024-08-30T08:51:51.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:47.921+0000 7f2ecd8e3640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f2ec810dfd0 con 0x7f2ec8103470 2024-08-30T08:51:51.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:47.921+0000 7f2ec77fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2ec8103e10 0x7f2ec819adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:51:51.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:47.921+0000 7f2ec77fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2ec8103e10 0x7f2ec819adf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:50226/0 (socket says 172.21.15.42:50226) 2024-08-30T08:51:51.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:47.921+0000 7f2ec77fe640 1 -- 172.21.15.42:0/3761656594 learned_addr learned my addr 172.21.15.42:0/3761656594 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T08:51:51.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:47.921+0000 7f2ebe7fc640 1 --2- 172.21.15.42:0/3761656594 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2ec8103470 0x7f2ec819a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:51:51.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:47.921+0000 7f2ec6ffd640 1 --2- 172.21.15.42:0/3761656594 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2ec81023e0 0x7f2ec8199be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:51:51.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:47.922+0000 7f2ec77fe640 1 -- 172.21.15.42:0/3761656594 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2ec8103470 msgr2=0x7f2ec819a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:51:51.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:47.922+0000 7f2ec77fe640 1 --2- 172.21.15.42:0/3761656594 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2ec8103470 0x7f2ec819a250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:51:51.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:47.922+0000 7f2ec77fe640 1 -- 172.21.15.42:0/3761656594 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2ec81023e0 msgr2=0x7f2ec8199be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:51:51.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:47.922+0000 7f2ec77fe640 1 --2- 172.21.15.42:0/3761656594 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2ec81023e0 0x7f2ec8199be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:51:51.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:47.922+0000 7f2ec77fe640 1 -- 172.21.15.42:0/3761656594 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ec819b610 con 0x7f2ec8103e10 2024-08-30T08:51:51.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:47.922+0000 7f2ec77fe640 1 --2- 172.21.15.42:0/3761656594 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2ec8103e10 0x7f2ec819adf0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f2ec000cad0 tx=0x7f2ec000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:51:51.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:47.922+0000 7f2ecc8e1640 1 -- 172.21.15.42:0/3761656594 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ec000ae20 con 0x7f2ec8103e10 2024-08-30T08:51:51.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:47.922+0000 7f2ecc8e1640 1 -- 172.21.15.42:0/3761656594 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(24 keys) ==== 1039+0+0 (secure 0 0 0) 0x7f2ec0005020 con 0x7f2ec8103e10 2024-08-30T08:51:51.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:47.922+0000 7f2ecc8e1640 1 -- 172.21.15.42:0/3761656594 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ec000f6b0 con 0x7f2ec8103e10 2024-08-30T08:51:51.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:47.922+0000 7f2ecd8e3640 1 -- 172.21.15.42:0/3761656594 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ec819b900 con 0x7f2ec8103e10 2024-08-30T08:51:51.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:47.922+0000 7f2ecd8e3640 1 -- 172.21.15.42:0/3761656594 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ec81a2000 con 0x7f2ec8103e10 2024-08-30T08:51:51.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:47.923+0000 7f2ecd8e3640 1 -- 172.21.15.42:0/3761656594 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ec8102850 con 0x7f2ec8103e10 2024-08-30T08:51:51.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:47.923+0000 7f2ecc8e1640 1 -- 172.21.15.42:0/3761656594 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 13) ==== 51253+0+0 (secure 0 0 0) 0x7f2ec000f850 con 0x7f2ec8103e10 2024-08-30T08:51:51.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:47.924+0000 7f2ecc8e1640 1 --2- 172.21.15.42:0/3761656594 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f2eb403f280 0x7f2eb4041790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:51:51.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:47.924+0000 7f2ecc8e1640 1 -- 172.21.15.42:0/3761656594 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f2ec00539c0 con 0x7f2ec8103e10 2024-08-30T08:51:51.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:47.924+0000 7f2ec6ffd640 1 --2- 172.21.15.42:0/3761656594 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f2eb403f280 0x7f2eb4041790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:51:51.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:47.924+0000 7f2ec6ffd640 1 --2- 172.21.15.42:0/3761656594 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f2eb403f280 0x7f2eb4041790 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f2eb8000c00 tx=0x7f2eb8005ef0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:51:51.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:47.929+0000 7f2ecc8e1640 1 -- 172.21.15.42:0/3761656594 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f2ec0014030 con 0x7f2ec8103e10 2024-08-30T08:51:51.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:48.124+0000 7f2ecd8e3640 1 -- 172.21.15.42:0/3761656594 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command([{prefix=config set, name=rados_mon_op_timeout}] v 0) -- 0x7f2ec81a22f0 con 0x7f2ec8103e10 2024-08-30T08:51:51.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:48.127+0000 7f2ecc8e1640 1 -- 172.21.15.42:0/3761656594 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{prefix=config set, name=rados_mon_op_timeout}]=0 v15)=0 v15) ==== 120+0+0 (secure 0 0 0) 0x7f2ec0020350 con 0x7f2ec8103e10 2024-08-30T08:51:51.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:48.130+0000 7f2ecd8e3640 1 -- 172.21.15.42:0/3761656594 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f2eb403f280 msgr2=0x7f2eb4041790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:51:51.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:48.130+0000 7f2ecd8e3640 1 --2- 172.21.15.42:0/3761656594 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f2eb403f280 0x7f2eb4041790 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f2eb8000c00 tx=0x7f2eb8005ef0 comp rx=0 tx=0).stop 2024-08-30T08:51:51.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:48.130+0000 7f2ecd8e3640 1 -- 172.21.15.42:0/3761656594 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2ec8103e10 msgr2=0x7f2ec819adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:51:51.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:48.130+0000 7f2ecd8e3640 1 --2- 172.21.15.42:0/3761656594 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2ec8103e10 0x7f2ec819adf0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f2ec000cad0 tx=0x7f2ec000cf90 comp rx=0 tx=0).stop 2024-08-30T08:51:51.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:48.130+0000 7f2ecd8e3640 1 -- 172.21.15.42:0/3761656594 shutdown_connections 2024-08-30T08:51:51.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:48.130+0000 7f2ecd8e3640 1 --2- 172.21.15.42:0/3761656594 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f2eb403f280 0x7f2eb4041790 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:51:51.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:48.130+0000 7f2ecd8e3640 1 --2- 172.21.15.42:0/3761656594 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2ec8103470 0x7f2ec819a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:51:51.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:48.130+0000 7f2ecd8e3640 1 --2- 172.21.15.42:0/3761656594 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2ec8103e10 0x7f2ec819adf0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:51:51.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:48.130+0000 7f2ecd8e3640 1 --2- 172.21.15.42:0/3761656594 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2ec81023e0 0x7f2ec8199be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:51:51.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:48.130+0000 7f2ecd8e3640 1 -- 172.21.15.42:0/3761656594 >> 172.21.15.42:0/3761656594 conn(0x7f2ec80fdd50 msgr2=0x7f2ec806ac00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:51:51.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:48.130+0000 7f2ecd8e3640 1 -- 172.21.15.42:0/3761656594 shutdown_connections 2024-08-30T08:51:51.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:48.130+0000 7f2ecd8e3640 1 -- 172.21.15.42:0/3761656594 wait complete. 2024-08-30T08:51:51.327 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:51 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:51.327 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:51 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:51.327 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:51 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:51.327 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:51 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:51.327 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:51 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:51:51.328 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:51 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-30T08:51:51.328 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:51 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:51.328 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:51 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:51.328 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:51 smithi190 ceph-mon[22001]: Reconfiguring mgr.x (unknown last config time)... 2024-08-30T08:51:51.328 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:51 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-08-30T08:51:51.328 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:51 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-08-30T08:51:51.328 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:51 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:51:51.328 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:51 smithi190 ceph-mon[22001]: Reconfiguring daemon mgr.x on smithi042 2024-08-30T08:51:51.329 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:51 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:51.332 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:51 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:51.332 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:51 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:51.332 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:51 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:51.332 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:51 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:51:51.332 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:51 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-30T08:51:51.332 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:51 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:51.332 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:51 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:51.332 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:51 smithi167 ceph-mon[22063]: Reconfiguring mgr.x (unknown last config time)... 2024-08-30T08:51:51.332 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:51 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-08-30T08:51:51.332 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:51 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-08-30T08:51:51.333 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:51 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:51:51.333 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:51 smithi167 ceph-mon[22063]: Reconfiguring daemon mgr.x on smithi042 2024-08-30T08:51:51.382 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:51 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:51.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:51 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:51.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:51 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:51.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:51 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:51.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:51 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:51:51.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:51 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-30T08:51:51.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:51 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:51.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:51 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:51:51.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:51 smithi042 ceph-mon[18652]: Reconfiguring mgr.x (unknown last config time)... 2024-08-30T08:51:51.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:51 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-08-30T08:51:51.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:51 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-08-30T08:51:51.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:51 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:51:51.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:51 smithi042 ceph-mon[18652]: Reconfiguring daemon mgr.x on smithi042 2024-08-30T08:51:51.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:51.574+0000 7f8fb49f4640 1 -- 172.21.15.42:0/3648627184 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8fb0105530 msgr2=0x7f8fb0107a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:51:51.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:51.574+0000 7f8fb49f4640 1 --2- 172.21.15.42:0/3648627184 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8fb0105530 0x7f8fb0107a10 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f8fa4009900 tx=0x7f8fa402a410 comp rx=0 tx=0).stop 2024-08-30T08:51:51.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:51.574+0000 7f8fb49f4640 1 -- 172.21.15.42:0/3648627184 shutdown_connections 2024-08-30T08:51:51.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:51.574+0000 7f8fb49f4640 1 --2- 172.21.15.42:0/3648627184 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8fb0108150 0x7f8fb010a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:51:51.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:51.574+0000 7f8fb49f4640 1 --2- 172.21.15.42:0/3648627184 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8fb0105530 0x7f8fb0107a10 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:51:51.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:51.574+0000 7f8fb49f4640 1 --2- 172.21.15.42:0/3648627184 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8fb0102a70 0x7f8fb0104ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:51:51.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:51.574+0000 7f8fb49f4640 1 -- 172.21.15.42:0/3648627184 >> 172.21.15.42:0/3648627184 conn(0x7f8fb00fd510 msgr2=0x7f8fb00ff9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:51:51.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:51.575+0000 7f8fb49f4640 1 -- 172.21.15.42:0/3648627184 shutdown_connections 2024-08-30T08:51:51.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:51.575+0000 7f8fb49f4640 1 -- 172.21.15.42:0/3648627184 wait complete. 2024-08-30T08:51:51.580 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:51.575+0000 7f8fb49f4640 1 Processor -- start 2024-08-30T08:51:51.580 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:51.575+0000 7f8fb49f4640 1 -- start start 2024-08-30T08:51:51.580 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:51.576+0000 7f8fb49f4640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8fb0102a70 0x7f8fb0104730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:51:51.580 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:51.576+0000 7f8fb49f4640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8fb0105530 0x7f8fb0103000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:51:51.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:51.576+0000 7f8fb49f4640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8fb0108150 0x7f8fb0103540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:51:51.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:51.576+0000 7f8fb49f4640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f8fb010df50 con 0x7f8fb0108150 2024-08-30T08:51:51.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:51.576+0000 7f8fb49f4640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f8fb010ddd0 con 0x7f8fb0105530 2024-08-30T08:51:51.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:51.576+0000 7f8fb49f4640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f8fb010e0d0 con 0x7f8fb0102a70 2024-08-30T08:51:51.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:51.576+0000 7f8faed9d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8fb0108150 0x7f8fb0103540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:51:51.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:51.576+0000 7f8fae59c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8fb0102a70 0x7f8fb0104730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:51:51.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:51.576+0000 7f8faed9d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8fb0108150 0x7f8fb0103540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:39320/0 (socket says 172.21.15.42:39320) 2024-08-30T08:51:51.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:51.576+0000 7f8faed9d640 1 -- 172.21.15.42:0/4060965035 learned_addr learned my addr 172.21.15.42:0/4060965035 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T08:51:51.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:51.576+0000 7f8fadd9b640 1 --2- 172.21.15.42:0/4060965035 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8fb0105530 0x7f8fb0103000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:51:51.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:51.576+0000 7f8fae59c640 1 -- 172.21.15.42:0/4060965035 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8fb0105530 msgr2=0x7f8fb0103000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:51:51.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:51.576+0000 7f8fae59c640 1 --2- 172.21.15.42:0/4060965035 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8fb0105530 0x7f8fb0103000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:51:51.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:51.576+0000 7f8fae59c640 1 -- 172.21.15.42:0/4060965035 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8fb0108150 msgr2=0x7f8fb0103540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:51:51.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:51.576+0000 7f8fae59c640 1 --2- 172.21.15.42:0/4060965035 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8fb0108150 0x7f8fb0103540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:51:51.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:51.576+0000 7f8fae59c640 1 -- 172.21.15.42:0/4060965035 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fb0103c40 con 0x7f8fb0102a70 2024-08-30T08:51:51.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:51.576+0000 7f8faed9d640 1 --2- 172.21.15.42:0/4060965035 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8fb0108150 0x7f8fb0103540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-30T08:51:51.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:51.576+0000 7f8fae59c640 1 --2- 172.21.15.42:0/4060965035 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8fb0102a70 0x7f8fb0104730 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f8fa000e9f0 tx=0x7f8fa000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:51:51.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:51.577+0000 7f8f9f7fe640 1 -- 172.21.15.42:0/4060965035 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fa000ccc0 con 0x7f8fb0102a70 2024-08-30T08:51:51.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:51.577+0000 7f8f9f7fe640 1 -- 172.21.15.42:0/4060965035 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(25 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f8fa00108c0 con 0x7f8fb0102a70 2024-08-30T08:51:51.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:51.577+0000 7f8fb49f4640 1 -- 172.21.15.42:0/4060965035 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fb01a3d50 con 0x7f8fb0102a70 2024-08-30T08:51:51.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:51.577+0000 7f8f9f7fe640 1 -- 172.21.15.42:0/4060965035 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fa00187c0 con 0x7f8fb0102a70 2024-08-30T08:51:51.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:51.577+0000 7f8fb49f4640 1 -- 172.21.15.42:0/4060965035 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fb01a41b0 con 0x7f8fb0102a70 2024-08-30T08:51:51.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:51.578+0000 7f8f9f7fe640 1 -- 172.21.15.42:0/4060965035 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 13) ==== 51253+0+0 (secure 0 0 0) 0x7f8fa0018960 con 0x7f8fb0102a70 2024-08-30T08:51:51.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:51.578+0000 7f8fb49f4640 1 -- 172.21.15.42:0/4060965035 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8fb0105a00 con 0x7f8fb0102a70 2024-08-30T08:51:51.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:51.578+0000 7f8f9f7fe640 1 --2- 172.21.15.42:0/4060965035 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f8f9803f280 0x7f8f98041790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:51:51.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:51.578+0000 7f8f9f7fe640 1 -- 172.21.15.42:0/4060965035 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f8fa0053dd0 con 0x7f8fb0102a70 2024-08-30T08:51:51.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:51.578+0000 7f8fadd9b640 1 --2- 172.21.15.42:0/4060965035 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f8f9803f280 0x7f8f98041790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:51:51.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:51.579+0000 7f8fadd9b640 1 --2- 172.21.15.42:0/4060965035 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f8f9803f280 0x7f8f98041790 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f8fa4000c00 tx=0x7f8fa4031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:51:51.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:51.583+0000 7f8f9f7fe640 1 -- 172.21.15.42:0/4060965035 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f8fa0020360 con 0x7f8fb0102a70 2024-08-30T08:51:51.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:51.744+0000 7f5409b23640 1 -- 172.21.15.42:0/1990159619 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f54041034c0 msgr2=0x7f54041038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:51:51.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:51.744+0000 7f5409b23640 1 --2- 172.21.15.42:0/1990159619 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f54041034c0 0x7f54041038d0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f53f8009900 tx=0x7f53f802a480 comp rx=0 tx=0).stop 2024-08-30T08:51:51.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:51.744+0000 7f5409b23640 1 -- 172.21.15.42:0/1990159619 shutdown_connections 2024-08-30T08:51:51.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:51.744+0000 7f5409b23640 1 --2- 172.21.15.42:0/1990159619 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5404103e60 0x7f540410a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:51:51.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:51.744+0000 7f5409b23640 1 --2- 172.21.15.42:0/1990159619 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f54041034c0 0x7f54041038d0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:51:51.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:51.744+0000 7f5409b23640 1 --2- 172.21.15.42:0/1990159619 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5404102430 0x7f54041028a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:51:51.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:51.744+0000 7f5409b23640 1 -- 172.21.15.42:0/1990159619 >> 172.21.15.42:0/1990159619 conn(0x7f54040fdda0 msgr2=0x7f5404100250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:51:51.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:51.744+0000 7f5409b23640 1 -- 172.21.15.42:0/1990159619 shutdown_connections 2024-08-30T08:51:51.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:51.744+0000 7f5409b23640 1 -- 172.21.15.42:0/1990159619 wait complete. 2024-08-30T08:51:51.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:51.745+0000 7f5409b23640 1 Processor -- start 2024-08-30T08:51:51.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:51.745+0000 7f5409b23640 1 -- start start 2024-08-30T08:51:51.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:51.745+0000 7f5409b23640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5404102430 0x7f5404106000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:51:51.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:51.745+0000 7f5409b23640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f54041034c0 0x7f54041048d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:51:51.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:51.745+0000 7f5409b23640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5404103e60 0x7f5404104e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:51:51.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:51.745+0000 7f5409b23640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f540410df70 con 0x7f54041034c0 2024-08-30T08:51:51.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:51.745+0000 7f5409b23640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f540410ddf0 con 0x7f5404102430 2024-08-30T08:51:51.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:51.745+0000 7f5409b23640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f540410e0f0 con 0x7f5404103e60 2024-08-30T08:51:51.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:51.746+0000 7f5402ffd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f54041034c0 0x7f54041048d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:51:51.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:51.746+0000 7f54037fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5404102430 0x7f5404106000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:51:51.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:51.746+0000 7f54037fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5404102430 0x7f5404106000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:33842/0 (socket says 172.21.15.42:33842) 2024-08-30T08:51:51.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:51.746+0000 7f5402ffd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f54041034c0 0x7f54041048d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:39344/0 (socket says 172.21.15.42:39344) 2024-08-30T08:51:51.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:51.746+0000 7f54037fe640 1 -- 172.21.15.42:0/3388699895 learned_addr learned my addr 172.21.15.42:0/3388699895 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T08:51:51.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:51.746+0000 7f5403fff640 1 --2- 172.21.15.42:0/3388699895 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5404103e60 0x7f5404104e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:51:51.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:51.746+0000 7f54037fe640 1 -- 172.21.15.42:0/3388699895 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5404103e60 msgr2=0x7f5404104e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:51:51.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:51.746+0000 7f54037fe640 1 --2- 172.21.15.42:0/3388699895 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5404103e60 0x7f5404104e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:51:51.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:51.746+0000 7f54037fe640 1 -- 172.21.15.42:0/3388699895 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f54041034c0 msgr2=0x7f54041048d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:51:51.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:51.746+0000 7f54037fe640 1 --2- 172.21.15.42:0/3388699895 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f54041034c0 0x7f54041048d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:51:51.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:51.746+0000 7f54037fe640 1 -- 172.21.15.42:0/3388699895 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5404105510 con 0x7f5404102430 2024-08-30T08:51:51.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:51.746+0000 7f54037fe640 1 --2- 172.21.15.42:0/3388699895 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5404102430 0x7f5404106000 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f53f400e9f0 tx=0x7f53f400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:51:51.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:51.747+0000 7f5400ff9640 1 -- 172.21.15.42:0/3388699895 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53f400cd80 con 0x7f5404102430 2024-08-30T08:51:51.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:51.747+0000 7f5400ff9640 1 -- 172.21.15.42:0/3388699895 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(25 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f53f40108c0 con 0x7f5404102430 2024-08-30T08:51:51.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:51.747+0000 7f5400ff9640 1 -- 172.21.15.42:0/3388699895 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53f40187d0 con 0x7f5404102430 2024-08-30T08:51:51.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:51.747+0000 7f5409b23640 1 -- 172.21.15.42:0/3388699895 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54041a3d70 con 0x7f5404102430 2024-08-30T08:51:51.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:51.747+0000 7f5409b23640 1 -- 172.21.15.42:0/3388699895 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54041a4110 con 0x7f5404102430 2024-08-30T08:51:51.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:51.748+0000 7f5400ff9640 1 -- 172.21.15.42:0/3388699895 <== mon.2 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 13) ==== 51253+0+0 (secure 0 0 0) 0x7f53f40189b0 con 0x7f5404102430 2024-08-30T08:51:51.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:51.748+0000 7f5400ff9640 1 --2- 172.21.15.42:0/3388699895 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f53ec03f280 0x7f53ec041790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:51:51.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:51.748+0000 7f5409b23640 1 -- 172.21.15.42:0/3388699895 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54041028a0 con 0x7f5404102430 2024-08-30T08:51:51.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:51.748+0000 7f5400ff9640 1 -- 172.21.15.42:0/3388699895 <== mon.2 v2:172.21.15.167:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f53f4053f50 con 0x7f5404102430 2024-08-30T08:51:51.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:51.748+0000 7f5402ffd640 1 --2- 172.21.15.42:0/3388699895 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f53ec03f280 0x7f53ec041790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:51:51.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:51.749+0000 7f5402ffd640 1 --2- 172.21.15.42:0/3388699895 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f53ec03f280 0x7f53ec041790 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f53f8000c00 tx=0x7f53f8031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:51:51.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:51.754+0000 7f5400ff9640 1 -- 172.21.15.42:0/3388699895 <== mon.2 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f53f4010d50 con 0x7f5404102430 2024-08-30T08:51:51.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:51.785+0000 7f8fb49f4640 1 -- 172.21.15.42:0/4060965035 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command([{prefix=config set, name=mds_debug_scatterstat}] v 0) -- 0x7f8fb01a4380 con 0x7f8fb0102a70 2024-08-30T08:51:51.793 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:51.790+0000 7f8f9f7fe640 1 -- 172.21.15.42:0/4060965035 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_debug_scatterstat}]=0 v18)=0 v18) ==== 122+0+0 (secure 0 0 0) 0x7f8fa001fe00 con 0x7f8fb0102a70 2024-08-30T08:51:51.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:51.792+0000 7f8fb49f4640 1 -- 172.21.15.42:0/4060965035 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f8f9803f280 msgr2=0x7f8f98041790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:51:51.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:51.792+0000 7f8fb49f4640 1 --2- 172.21.15.42:0/4060965035 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f8f9803f280 0x7f8f98041790 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f8fa4000c00 tx=0x7f8fa4031400 comp rx=0 tx=0).stop 2024-08-30T08:51:51.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:51.792+0000 7f8fb49f4640 1 -- 172.21.15.42:0/4060965035 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8fb0102a70 msgr2=0x7f8fb0104730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:51:51.796 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:51.792+0000 7f8fb49f4640 1 --2- 172.21.15.42:0/4060965035 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8fb0102a70 0x7f8fb0104730 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f8fa000e9f0 tx=0x7f8fa000eeb0 comp rx=0 tx=0).stop 2024-08-30T08:51:51.796 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:51.792+0000 7f8fb49f4640 1 -- 172.21.15.42:0/4060965035 shutdown_connections 2024-08-30T08:51:51.796 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:51.792+0000 7f8fb49f4640 1 --2- 172.21.15.42:0/4060965035 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f8f9803f280 0x7f8f98041790 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:51:51.796 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:51.792+0000 7f8fb49f4640 1 --2- 172.21.15.42:0/4060965035 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8fb0108150 0x7f8fb0103540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:51:51.796 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:51.792+0000 7f8fb49f4640 1 --2- 172.21.15.42:0/4060965035 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8fb0105530 0x7f8fb0103000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:51:51.796 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:51.792+0000 7f8fb49f4640 1 --2- 172.21.15.42:0/4060965035 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8fb0102a70 0x7f8fb0104730 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:51:51.796 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:51.792+0000 7f8fb49f4640 1 -- 172.21.15.42:0/4060965035 >> 172.21.15.42:0/4060965035 conn(0x7f8fb00fd510 msgr2=0x7f8fb0107800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:51:51.796 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:51.793+0000 7f8fb49f4640 1 -- 172.21.15.42:0/4060965035 shutdown_connections 2024-08-30T08:51:51.796 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:51.793+0000 7f8fb49f4640 1 -- 172.21.15.42:0/4060965035 wait complete. 2024-08-30T08:51:51.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:51.954+0000 7f5409b23640 1 -- 172.21.15.42:0/3388699895 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command([{prefix=config set, name=debug_mds_balancer}] v 0) -- 0x7f54041a48b0 con 0x7f5404102430 2024-08-30T08:51:51.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:51.960+0000 7f5400ff9640 1 -- 172.21.15.42:0/3388699895 <== mon.2 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_mds_balancer}]=0 v19)=0 v19) ==== 117+0+0 (secure 0 0 0) 0x7f53f4042020 con 0x7f5404102430 2024-08-30T08:51:51.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:51.962+0000 7f5409b23640 1 -- 172.21.15.42:0/3388699895 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f53ec03f280 msgr2=0x7f53ec041790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:51:51.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:51.962+0000 7f5409b23640 1 --2- 172.21.15.42:0/3388699895 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f53ec03f280 0x7f53ec041790 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f53f8000c00 tx=0x7f53f8031400 comp rx=0 tx=0).stop 2024-08-30T08:51:51.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:51.962+0000 7f5409b23640 1 -- 172.21.15.42:0/3388699895 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5404102430 msgr2=0x7f5404106000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:51:51.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:51.962+0000 7f5409b23640 1 --2- 172.21.15.42:0/3388699895 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5404102430 0x7f5404106000 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f53f400e9f0 tx=0x7f53f400eeb0 comp rx=0 tx=0).stop 2024-08-30T08:51:51.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:51.962+0000 7f5409b23640 1 -- 172.21.15.42:0/3388699895 shutdown_connections 2024-08-30T08:51:51.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:51.962+0000 7f5409b23640 1 --2- 172.21.15.42:0/3388699895 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f53ec03f280 0x7f53ec041790 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:51:51.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:51.962+0000 7f5409b23640 1 --2- 172.21.15.42:0/3388699895 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5404103e60 0x7f5404104e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:51:51.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:51.962+0000 7f5409b23640 1 --2- 172.21.15.42:0/3388699895 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f54041034c0 0x7f54041048d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:51:51.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:51.962+0000 7f5409b23640 1 --2- 172.21.15.42:0/3388699895 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5404102430 0x7f5404106000 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:51:51.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:51.962+0000 7f5409b23640 1 -- 172.21.15.42:0/3388699895 >> 172.21.15.42:0/3388699895 conn(0x7f54040fdda0 msgr2=0x7f54040ff730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:51:51.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:51.963+0000 7f5409b23640 1 -- 172.21.15.42:0/3388699895 shutdown_connections 2024-08-30T08:51:51.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:51.963+0000 7f5409b23640 1 -- 172.21.15.42:0/3388699895 wait complete. 2024-08-30T08:51:52.354 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:52 smithi042 ceph-mon[18652]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-30T08:51:52.354 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:52 smithi042 ceph-mon[18652]: Standby manager daemon z started 2024-08-30T08:51:52.355 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:52 smithi042 ceph-mon[18652]: from='mgr.? 172.21.15.190:0/3883631783' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/crt"} : dispatch 2024-08-30T08:51:52.355 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:52 smithi042 ceph-mon[18652]: from='mgr.? 172.21.15.190:0/3883631783' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-08-30T08:51:52.355 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:52 smithi042 ceph-mon[18652]: from='mgr.? 172.21.15.190:0/3883631783' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/key"} : dispatch 2024-08-30T08:51:52.355 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:52 smithi042 ceph-mon[18652]: from='mgr.? 172.21.15.190:0/3883631783' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-08-30T08:51:52.355 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:52 smithi042 ceph-mon[18652]: from='client.? ' entity='client.admin' 2024-08-30T08:51:52.355 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:52 smithi042 ceph-mon[18652]: from='client.? ' entity='client.admin' 2024-08-30T08:51:52.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:52.443+0000 7fe4cf23a640 1 -- 172.21.15.42:0/1158806485 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe4c8104580 msgr2=0x7fe4c8104990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:51:52.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:52.443+0000 7fe4cf23a640 1 --2- 172.21.15.42:0/1158806485 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe4c8104580 0x7fe4c8104990 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fe4b4009900 tx=0x7fe4b402a480 comp rx=0 tx=0).stop 2024-08-30T08:51:52.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:52.443+0000 7fe4cf23a640 1 -- 172.21.15.42:0/1158806485 shutdown_connections 2024-08-30T08:51:52.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:52.443+0000 7fe4cf23a640 1 --2- 172.21.15.42:0/1158806485 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe4c8104f20 0x7fe4c810b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:51:52.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:52.443+0000 7fe4cf23a640 1 --2- 172.21.15.42:0/1158806485 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe4c8104580 0x7fe4c8104990 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:51:52.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:52.443+0000 7fe4cf23a640 1 --2- 172.21.15.42:0/1158806485 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe4c81034f0 0x7fe4c8103960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:51:52.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:52.443+0000 7fe4cf23a640 1 -- 172.21.15.42:0/1158806485 >> 172.21.15.42:0/1158806485 conn(0x7fe4c80fee60 msgr2=0x7fe4c8101310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:51:52.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:52.443+0000 7fe4cf23a640 1 -- 172.21.15.42:0/1158806485 shutdown_connections 2024-08-30T08:51:52.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:52.443+0000 7fe4cf23a640 1 -- 172.21.15.42:0/1158806485 wait complete. 2024-08-30T08:51:52.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:52.443+0000 7fe4cf23a640 1 Processor -- start 2024-08-30T08:51:52.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:52.443+0000 7fe4cf23a640 1 -- start start 2024-08-30T08:51:52.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:52.444+0000 7fe4cf23a640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe4c81034f0 0x7fe4c8076be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:51:52.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:52.444+0000 7fe4cf23a640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe4c8104580 0x7fe4c80754b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:51:52.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:52.444+0000 7fe4cf23a640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe4c8104f20 0x7fe4c80759f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:51:52.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:52.444+0000 7fe4cf23a640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fe4c810eff0 con 0x7fe4c8104580 2024-08-30T08:51:52.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:52.444+0000 7fe4cf23a640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe4c810ee70 con 0x7fe4c8104f20 2024-08-30T08:51:52.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:52.444+0000 7fe4cf23a640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fe4c810f170 con 0x7fe4c81034f0 2024-08-30T08:51:52.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:52.444+0000 7fe4c7fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe4c8104580 0x7fe4c80754b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:51:52.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:52.444+0000 7fe4c7fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe4c8104580 0x7fe4c80754b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:39352/0 (socket says 172.21.15.42:39352) 2024-08-30T08:51:52.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:52.444+0000 7fe4c7fff640 1 -- 172.21.15.42:0/4137099351 learned_addr learned my addr 172.21.15.42:0/4137099351 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T08:51:52.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:52.444+0000 7fe4cd7d7640 1 --2- 172.21.15.42:0/4137099351 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe4c8104f20 0x7fe4c80759f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:51:52.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:52.444+0000 7fe4ccfd6640 1 --2- 172.21.15.42:0/4137099351 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe4c81034f0 0x7fe4c8076be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:51:52.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:52.444+0000 7fe4c7fff640 1 -- 172.21.15.42:0/4137099351 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe4c81034f0 msgr2=0x7fe4c8076be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:51:52.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:52.444+0000 7fe4c7fff640 1 --2- 172.21.15.42:0/4137099351 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe4c81034f0 0x7fe4c8076be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:51:52.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:52.444+0000 7fe4c7fff640 1 -- 172.21.15.42:0/4137099351 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe4c8104f20 msgr2=0x7fe4c80759f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:51:52.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:52.444+0000 7fe4c7fff640 1 --2- 172.21.15.42:0/4137099351 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe4c8104f20 0x7fe4c80759f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:51:52.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:52.444+0000 7fe4c7fff640 1 -- 172.21.15.42:0/4137099351 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4c80760f0 con 0x7fe4c8104580 2024-08-30T08:51:52.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:52.444+0000 7fe4cd7d7640 1 --2- 172.21.15.42:0/4137099351 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe4c8104f20 0x7fe4c80759f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-30T08:51:52.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:52.444+0000 7fe4c7fff640 1 --2- 172.21.15.42:0/4137099351 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe4c8104580 0x7fe4c80754b0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fe4b4004770 tx=0x7fe4b40040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:51:52.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:52.445+0000 7fe4c5ffb640 1 -- 172.21.15.42:0/4137099351 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4b403f030 con 0x7fe4c8104580 2024-08-30T08:51:52.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:52.445+0000 7fe4c5ffb640 1 -- 172.21.15.42:0/4137099351 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fe4b40044c0 con 0x7fe4c8104580 2024-08-30T08:51:52.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:52.445+0000 7fe4cf23a640 1 -- 172.21.15.42:0/4137099351 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4c81a3d10 con 0x7fe4c8104580 2024-08-30T08:51:52.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:52.445+0000 7fe4c5ffb640 1 -- 172.21.15.42:0/4137099351 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4b4035070 con 0x7fe4c8104580 2024-08-30T08:51:52.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:52.445+0000 7fe4cf23a640 1 -- 172.21.15.42:0/4137099351 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4c81a4100 con 0x7fe4c8104580 2024-08-30T08:51:52.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:52.446+0000 7fe4cf23a640 1 -- 172.21.15.42:0/4137099351 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4c810f7b0 con 0x7fe4c8104580 2024-08-30T08:51:52.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:52.446+0000 7fe4c5ffb640 1 -- 172.21.15.42:0/4137099351 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 14) ==== 101887+0+0 (secure 0 0 0) 0x7fe4b4042030 con 0x7fe4c8104580 2024-08-30T08:51:52.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:52.447+0000 7fe4c5ffb640 1 --2- 172.21.15.42:0/4137099351 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fe4b8079e90 0x7fe4b807c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:51:52.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:52.447+0000 7fe4ccfd6640 1 --2- 172.21.15.42:0/4137099351 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fe4b8079e90 0x7fe4b807c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:51:52.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:52.447+0000 7fe4c5ffb640 1 -- 172.21.15.42:0/4137099351 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fe4b40bbb40 con 0x7fe4c8104580 2024-08-30T08:51:52.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:52.448+0000 7fe4ccfd6640 1 --2- 172.21.15.42:0/4137099351 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fe4b8079e90 0x7fe4b807c3a0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fe4bc009bd0 tx=0x7fe4bc005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:51:52.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:52.452+0000 7fe4c5ffb640 1 -- 172.21.15.42:0/4137099351 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7fe4b4084490 con 0x7fe4c8104580 2024-08-30T08:51:52.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:52 smithi167 ceph-mon[22063]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-30T08:51:52.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:52 smithi167 ceph-mon[22063]: Standby manager daemon z started 2024-08-30T08:51:52.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:52 smithi167 ceph-mon[22063]: from='mgr.? 172.21.15.190:0/3883631783' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/crt"} : dispatch 2024-08-30T08:51:52.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:52 smithi167 ceph-mon[22063]: from='mgr.? 172.21.15.190:0/3883631783' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-08-30T08:51:52.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:52 smithi167 ceph-mon[22063]: from='mgr.? 172.21.15.190:0/3883631783' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/key"} : dispatch 2024-08-30T08:51:52.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:52 smithi167 ceph-mon[22063]: from='mgr.? 172.21.15.190:0/3883631783' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-08-30T08:51:52.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:52 smithi167 ceph-mon[22063]: from='client.? ' entity='client.admin' 2024-08-30T08:51:52.578 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:52 smithi167 ceph-mon[22063]: from='client.? ' entity='client.admin' 2024-08-30T08:51:52.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:52.641+0000 7fe4cf23a640 1 -- 172.21.15.42:0/4137099351 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command([{prefix=config set, name=mds_bal_fragment_size_max}] v 0) -- 0x7fe4c802e6f0 con 0x7fe4c8104580 2024-08-30T08:51:52.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:52.645+0000 7fe4c5ffb640 1 -- 172.21.15.42:0/4137099351 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_fragment_size_max}]=0 v20)=0 v20) ==== 127+0+0 (secure 0 0 0) 0x7fe4b4088c50 con 0x7fe4c8104580 2024-08-30T08:51:52.650 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:52.647+0000 7fe4cf23a640 1 -- 172.21.15.42:0/4137099351 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fe4b8079e90 msgr2=0x7fe4b807c3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:51:52.650 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:52.647+0000 7fe4cf23a640 1 --2- 172.21.15.42:0/4137099351 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fe4b8079e90 0x7fe4b807c3a0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fe4bc009bd0 tx=0x7fe4bc005f90 comp rx=0 tx=0).stop 2024-08-30T08:51:52.650 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:52.647+0000 7fe4cf23a640 1 -- 172.21.15.42:0/4137099351 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe4c8104580 msgr2=0x7fe4c80754b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:51:52.650 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:52.647+0000 7fe4cf23a640 1 --2- 172.21.15.42:0/4137099351 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe4c8104580 0x7fe4c80754b0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fe4b4004770 tx=0x7fe4b40040c0 comp rx=0 tx=0).stop 2024-08-30T08:51:52.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:52.647+0000 7fe4cf23a640 1 -- 172.21.15.42:0/4137099351 shutdown_connections 2024-08-30T08:51:52.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:52.647+0000 7fe4cf23a640 1 --2- 172.21.15.42:0/4137099351 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fe4b8079e90 0x7fe4b807c3a0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:51:52.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:52.647+0000 7fe4cf23a640 1 --2- 172.21.15.42:0/4137099351 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe4c8104f20 0x7fe4c80759f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:51:52.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:52.647+0000 7fe4cf23a640 1 --2- 172.21.15.42:0/4137099351 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe4c8104580 0x7fe4c80754b0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:51:52.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:52.647+0000 7fe4cf23a640 1 --2- 172.21.15.42:0/4137099351 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe4c81034f0 0x7fe4c8076be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:51:52.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:52.647+0000 7fe4cf23a640 1 -- 172.21.15.42:0/4137099351 >> 172.21.15.42:0/4137099351 conn(0x7fe4c80fee60 msgr2=0x7fe4c81078a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:51:52.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:52.647+0000 7fe4cf23a640 1 -- 172.21.15.42:0/4137099351 shutdown_connections 2024-08-30T08:51:52.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:52.648+0000 7fe4cf23a640 1 -- 172.21.15.42:0/4137099351 wait complete. 2024-08-30T08:51:52.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:52 smithi190 ceph-mon[22001]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-30T08:51:52.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:52 smithi190 ceph-mon[22001]: Standby manager daemon z started 2024-08-30T08:51:52.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:52 smithi190 ceph-mon[22001]: from='mgr.? 172.21.15.190:0/3883631783' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/crt"} : dispatch 2024-08-30T08:51:52.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:52 smithi190 ceph-mon[22001]: from='mgr.? 172.21.15.190:0/3883631783' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-08-30T08:51:52.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:52 smithi190 ceph-mon[22001]: from='mgr.? 172.21.15.190:0/3883631783' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/key"} : dispatch 2024-08-30T08:51:52.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:52 smithi190 ceph-mon[22001]: from='mgr.? 172.21.15.190:0/3883631783' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-08-30T08:51:52.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:52 smithi190 ceph-mon[22001]: from='client.? ' entity='client.admin' 2024-08-30T08:51:52.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:52 smithi190 ceph-mon[22001]: from='client.? ' entity='client.admin' 2024-08-30T08:51:53.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:53 smithi042 ceph-mon[18652]: mgrmap e14: x(active, since 67s), standbys: z 2024-08-30T08:51:53.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:53 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "z", "id": "z"} : dispatch 2024-08-30T08:51:53.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:53 smithi042 ceph-mon[18652]: from='client.? 172.21.15.42:0/4137099351' entity='client.admin' 2024-08-30T08:51:53.461 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'b0895c1a-66ac-11ef-bcd4-c7b262605968', '--', 'ceph', 'config', 'set', 'client', 'rados_mon_op_timeout', '900'], check_status=True, hostname='smithi042') 2024-08-30T08:51:53.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:53 smithi167 ceph-mon[22063]: mgrmap e14: x(active, since 67s), standbys: z 2024-08-30T08:51:53.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:53 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "z", "id": "z"} : dispatch 2024-08-30T08:51:53.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:53 smithi167 ceph-mon[22063]: from='client.? 172.21.15.42:0/4137099351' entity='client.admin' 2024-08-30T08:51:53.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:53 smithi190 ceph-mon[22001]: mgrmap e14: x(active, since 67s), standbys: z 2024-08-30T08:51:53.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:53 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "z", "id": "z"} : dispatch 2024-08-30T08:51:53.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:53 smithi190 ceph-mon[22001]: from='client.? 172.21.15.42:0/4137099351' entity='client.admin' 2024-08-30T08:51:53.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:53.894+0000 7fa94981f640 1 -- 172.21.15.42:0/2210012854 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa944103e10 msgr2=0x7fa94410a7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:51:53.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:53.894+0000 7fa94981f640 1 --2- 172.21.15.42:0/2210012854 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa944103e10 0x7fa94410a7b0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fa93c009900 tx=0x7fa93c02a480 comp rx=0 tx=0).stop 2024-08-30T08:51:53.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:53.894+0000 7fa94981f640 1 -- 172.21.15.42:0/2210012854 shutdown_connections 2024-08-30T08:51:53.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:53.894+0000 7fa94981f640 1 --2- 172.21.15.42:0/2210012854 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa944103470 0x7fa944103880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:51:53.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:53.894+0000 7fa94981f640 1 --2- 172.21.15.42:0/2210012854 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa944103e10 0x7fa94410a7b0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:51:53.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:53.894+0000 7fa94981f640 1 --2- 172.21.15.42:0/2210012854 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa9441023e0 0x7fa944102850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:51:53.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:53.894+0000 7fa94981f640 1 -- 172.21.15.42:0/2210012854 >> 172.21.15.42:0/2210012854 conn(0x7fa9440fdd50 msgr2=0x7fa944100200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:51:53.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:53.894+0000 7fa94981f640 1 -- 172.21.15.42:0/2210012854 shutdown_connections 2024-08-30T08:51:53.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:53.894+0000 7fa94981f640 1 -- 172.21.15.42:0/2210012854 wait complete. 2024-08-30T08:51:53.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:53.895+0000 7fa94981f640 1 Processor -- start 2024-08-30T08:51:53.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:53.895+0000 7fa94981f640 1 -- start start 2024-08-30T08:51:53.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:53.895+0000 7fa94981f640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa9441023e0 0x7fa94406b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:51:53.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:53.895+0000 7fa94981f640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa944103470 0x7fa94419e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:51:53.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:53.895+0000 7fa94981f640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa944103e10 0x7fa9441a0f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:51:53.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:53.895+0000 7fa94981f640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fa94410df20 con 0x7fa9441023e0 2024-08-30T08:51:53.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:53.895+0000 7fa94981f640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa94410dda0 con 0x7fa944103e10 2024-08-30T08:51:53.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:53.895+0000 7fa94981f640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fa94410e0a0 con 0x7fa944103470 2024-08-30T08:51:53.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:53.895+0000 7fa942ffd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa9441023e0 0x7fa94406b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:51:53.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:53.895+0000 7fa942ffd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa9441023e0 0x7fa94406b440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:39368/0 (socket says 172.21.15.42:39368) 2024-08-30T08:51:53.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:53.895+0000 7fa942ffd640 1 -- 172.21.15.42:0/98315464 learned_addr learned my addr 172.21.15.42:0/98315464 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T08:51:53.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:53.895+0000 7fa9427fc640 1 --2- 172.21.15.42:0/98315464 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa944103470 0x7fa94419e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:51:53.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:53.895+0000 7fa9437fe640 1 --2- 172.21.15.42:0/98315464 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa944103e10 0x7fa9441a0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:51:53.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:53.896+0000 7fa942ffd640 1 -- 172.21.15.42:0/98315464 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa944103470 msgr2=0x7fa94419e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:51:53.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:53.896+0000 7fa942ffd640 1 --2- 172.21.15.42:0/98315464 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa944103470 0x7fa94419e9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:51:53.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:53.896+0000 7fa942ffd640 1 -- 172.21.15.42:0/98315464 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa944103e10 msgr2=0x7fa9441a0f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:51:53.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:53.896+0000 7fa942ffd640 1 --2- 172.21.15.42:0/98315464 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa944103e10 0x7fa9441a0f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:51:53.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:53.896+0000 7fa942ffd640 1 -- 172.21.15.42:0/98315464 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9441a1700 con 0x7fa9441023e0 2024-08-30T08:51:53.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:53.896+0000 7fa942ffd640 1 --2- 172.21.15.42:0/98315464 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa9441023e0 0x7fa94406b440 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fa93400ebc0 tx=0x7fa93400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:51:53.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:53.896+0000 7fa94881d640 1 -- 172.21.15.42:0/98315464 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa934010630 con 0x7fa9441023e0 2024-08-30T08:51:53.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:53.896+0000 7fa94881d640 1 -- 172.21.15.42:0/98315464 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fa934010c60 con 0x7fa9441023e0 2024-08-30T08:51:53.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:53.896+0000 7fa94981f640 1 -- 172.21.15.42:0/98315464 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9441a19f0 con 0x7fa9441023e0 2024-08-30T08:51:53.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:53.896+0000 7fa94881d640 1 -- 172.21.15.42:0/98315464 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa934018b10 con 0x7fa9441023e0 2024-08-30T08:51:53.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:53.896+0000 7fa94981f640 1 -- 172.21.15.42:0/98315464 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9441a1f80 con 0x7fa9441023e0 2024-08-30T08:51:53.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:53.897+0000 7fa94981f640 1 -- 172.21.15.42:0/98315464 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa944102850 con 0x7fa9441023e0 2024-08-30T08:51:53.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:53.898+0000 7fa94881d640 1 -- 172.21.15.42:0/98315464 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 14) ==== 101887+0+0 (secure 0 0 0) 0x7fa934018cb0 con 0x7fa9441023e0 2024-08-30T08:51:53.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:53.898+0000 7fa94881d640 1 --2- 172.21.15.42:0/98315464 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fa930079e90 0x7fa93007c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:51:53.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:53.898+0000 7fa94881d640 1 -- 172.21.15.42:0/98315464 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fa93409bcd0 con 0x7fa9441023e0 2024-08-30T08:51:53.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:53.898+0000 7fa9427fc640 1 --2- 172.21.15.42:0/98315464 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fa930079e90 0x7fa93007c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:51:53.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:53.899+0000 7fa9427fc640 1 --2- 172.21.15.42:0/98315464 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fa930079e90 0x7fa93007c3a0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fa92c005fd0 tx=0x7fa92c00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:51:53.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:53.903+0000 7fa94881d640 1 -- 172.21.15.42:0/98315464 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7fa934064620 con 0x7fa9441023e0 2024-08-30T08:51:54.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:54.092+0000 7fa94981f640 1 -- 172.21.15.42:0/98315464 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command([{prefix=config set, name=mds_op_complaint_time}] v 0) -- 0x7fa944063d60 con 0x7fa9441023e0 2024-08-30T08:51:54.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:54.096+0000 7fa94881d640 1 -- 172.21.15.42:0/98315464 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_op_complaint_time}]=0 v21)=0 v21) ==== 121+0+0 (secure 0 0 0) 0x7fa93406d120 con 0x7fa9441023e0 2024-08-30T08:51:54.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:54.098+0000 7fa94981f640 1 -- 172.21.15.42:0/98315464 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fa930079e90 msgr2=0x7fa93007c3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:51:54.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:54.098+0000 7fa94981f640 1 --2- 172.21.15.42:0/98315464 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fa930079e90 0x7fa93007c3a0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fa92c005fd0 tx=0x7fa92c00b390 comp rx=0 tx=0).stop 2024-08-30T08:51:54.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:54.098+0000 7fa94981f640 1 -- 172.21.15.42:0/98315464 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa9441023e0 msgr2=0x7fa94406b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:51:54.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:54.098+0000 7fa94981f640 1 --2- 172.21.15.42:0/98315464 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa9441023e0 0x7fa94406b440 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fa93400ebc0 tx=0x7fa93400c6a0 comp rx=0 tx=0).stop 2024-08-30T08:51:54.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:54.098+0000 7fa94981f640 1 -- 172.21.15.42:0/98315464 shutdown_connections 2024-08-30T08:51:54.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:54.099+0000 7fa94981f640 1 --2- 172.21.15.42:0/98315464 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fa930079e90 0x7fa93007c3a0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:51:54.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:54.099+0000 7fa94981f640 1 --2- 172.21.15.42:0/98315464 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa944103470 0x7fa94419e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:51:54.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:54.099+0000 7fa94981f640 1 --2- 172.21.15.42:0/98315464 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa944103e10 0x7fa9441a0f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:51:54.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:54.099+0000 7fa94981f640 1 --2- 172.21.15.42:0/98315464 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa9441023e0 0x7fa94406b440 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:51:54.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:54.099+0000 7fa94981f640 1 -- 172.21.15.42:0/98315464 >> 172.21.15.42:0/98315464 conn(0x7fa9440fdd50 msgr2=0x7fa944106770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:51:54.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:54.099+0000 7fa94981f640 1 -- 172.21.15.42:0/98315464 shutdown_connections 2024-08-30T08:51:54.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:54.099+0000 7fa94981f640 1 -- 172.21.15.42:0/98315464 wait complete. 2024-08-30T08:51:54.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:54 smithi190 ceph-mon[22001]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-30T08:51:54.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:54 smithi190 ceph-mon[22001]: Standby manager daemon y started 2024-08-30T08:51:54.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:54 smithi190 ceph-mon[22001]: from='mgr.? 172.21.15.167:0/2795043480' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/crt"} : dispatch 2024-08-30T08:51:54.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:54 smithi190 ceph-mon[22001]: from='mgr.? 172.21.15.167:0/2795043480' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-08-30T08:51:54.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:54 smithi190 ceph-mon[22001]: from='mgr.? 172.21.15.167:0/2795043480' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/key"} : dispatch 2024-08-30T08:51:54.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:54 smithi190 ceph-mon[22001]: from='mgr.? 172.21.15.167:0/2795043480' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-08-30T08:51:54.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:54 smithi190 ceph-mon[22001]: from='client.? 172.21.15.42:0/98315464' entity='client.admin' 2024-08-30T08:51:54.961 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:54 smithi042 ceph-mon[18652]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-30T08:51:54.961 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:54 smithi042 ceph-mon[18652]: Standby manager daemon y started 2024-08-30T08:51:54.961 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:54 smithi042 ceph-mon[18652]: from='mgr.? 172.21.15.167:0/2795043480' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/crt"} : dispatch 2024-08-30T08:51:54.961 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:54 smithi042 ceph-mon[18652]: from='mgr.? 172.21.15.167:0/2795043480' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-08-30T08:51:54.961 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:54 smithi042 ceph-mon[18652]: from='mgr.? 172.21.15.167:0/2795043480' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/key"} : dispatch 2024-08-30T08:51:54.961 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:54 smithi042 ceph-mon[18652]: from='mgr.? 172.21.15.167:0/2795043480' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-08-30T08:51:54.961 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:54 smithi042 ceph-mon[18652]: from='client.? 172.21.15.42:0/98315464' entity='client.admin' 2024-08-30T08:51:55.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:54 smithi167 ceph-mon[22063]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-30T08:51:55.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:54 smithi167 ceph-mon[22063]: Standby manager daemon y started 2024-08-30T08:51:55.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:54 smithi167 ceph-mon[22063]: from='mgr.? 172.21.15.167:0/2795043480' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/crt"} : dispatch 2024-08-30T08:51:55.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:54 smithi167 ceph-mon[22063]: from='mgr.? 172.21.15.167:0/2795043480' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-08-30T08:51:55.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:54 smithi167 ceph-mon[22063]: from='mgr.? 172.21.15.167:0/2795043480' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/key"} : dispatch 2024-08-30T08:51:55.078 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:54 smithi167 ceph-mon[22063]: from='mgr.? 172.21.15.167:0/2795043480' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-08-30T08:51:55.078 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:54 smithi167 ceph-mon[22063]: from='client.? 172.21.15.42:0/98315464' entity='client.admin' 2024-08-30T08:51:55.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:54.958+0000 7f2c74365640 1 -- 172.21.15.42:0/1737264420 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2c6c104f20 msgr2=0x7f2c6c10b8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:51:55.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:54.958+0000 7f2c74365640 1 --2- 172.21.15.42:0/1737264420 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2c6c104f20 0x7f2c6c10b8c0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f2c64009ca0 tx=0x7f2c6402a950 comp rx=0 tx=0).stop 2024-08-30T08:51:55.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:54.958+0000 7f2c74365640 1 -- 172.21.15.42:0/1737264420 shutdown_connections 2024-08-30T08:51:55.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:54.958+0000 7f2c74365640 1 --2- 172.21.15.42:0/1737264420 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2c6c104f20 0x7f2c6c10b8c0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:51:55.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:54.958+0000 7f2c74365640 1 --2- 172.21.15.42:0/1737264420 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2c6c104580 0x7f2c6c104990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:51:55.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:54.958+0000 7f2c74365640 1 --2- 172.21.15.42:0/1737264420 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2c6c1034f0 0x7f2c6c103960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:51:55.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:54.958+0000 7f2c74365640 1 -- 172.21.15.42:0/1737264420 >> 172.21.15.42:0/1737264420 conn(0x7f2c6c0fee60 msgr2=0x7f2c6c101310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:51:55.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:54.958+0000 7f2c74365640 1 -- 172.21.15.42:0/1737264420 shutdown_connections 2024-08-30T08:51:55.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:54.958+0000 7f2c74365640 1 -- 172.21.15.42:0/1737264420 wait complete. 2024-08-30T08:51:55.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:54.959+0000 7f2c74365640 1 Processor -- start 2024-08-30T08:51:55.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:54.959+0000 7f2c74365640 1 -- start start 2024-08-30T08:51:55.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:54.959+0000 7f2c74365640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2c6c1034f0 0x7f2c6c199c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:51:55.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:54.959+0000 7f2c74365640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2c6c104580 0x7f2c6c19a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:51:55.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:54.959+0000 7f2c74365640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2c6c104f20 0x7f2c6c19ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:51:55.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:54.959+0000 7f2c74365640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f2c6c10f030 con 0x7f2c6c104580 2024-08-30T08:51:55.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:54.959+0000 7f2c74365640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f2c6c10eeb0 con 0x7f2c6c104f20 2024-08-30T08:51:55.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:54.959+0000 7f2c74365640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f2c6c10f1b0 con 0x7f2c6c1034f0 2024-08-30T08:51:55.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:54.959+0000 7f2c71900640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2c6c104580 0x7f2c6c19a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:51:55.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:54.959+0000 7f2c71900640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2c6c104580 0x7f2c6c19a290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:39394/0 (socket says 172.21.15.42:39394) 2024-08-30T08:51:55.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:54.959+0000 7f2c71900640 1 -- 172.21.15.42:0/2660975208 learned_addr learned my addr 172.21.15.42:0/2660975208 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T08:51:55.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:54.960+0000 7f2c72902640 1 --2- 172.21.15.42:0/2660975208 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2c6c104f20 0x7f2c6c19ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:51:55.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:54.960+0000 7f2c72101640 1 --2- 172.21.15.42:0/2660975208 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2c6c1034f0 0x7f2c6c199c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:51:55.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:54.960+0000 7f2c71900640 1 -- 172.21.15.42:0/2660975208 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2c6c1034f0 msgr2=0x7f2c6c199c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:51:55.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:54.960+0000 7f2c71900640 1 --2- 172.21.15.42:0/2660975208 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2c6c1034f0 0x7f2c6c199c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:51:55.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:54.960+0000 7f2c71900640 1 -- 172.21.15.42:0/2660975208 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2c6c104f20 msgr2=0x7f2c6c19ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:51:55.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:54.960+0000 7f2c71900640 1 --2- 172.21.15.42:0/2660975208 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2c6c104f20 0x7f2c6c19ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:51:55.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:54.960+0000 7f2c71900640 1 -- 172.21.15.42:0/2660975208 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c6c19b650 con 0x7f2c6c104580 2024-08-30T08:51:55.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:54.960+0000 7f2c71900640 1 --2- 172.21.15.42:0/2660975208 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2c6c104580 0x7f2c6c19a290 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f2c5c00c890 tx=0x7f2c5c00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:51:55.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:54.960+0000 7f2c637fe640 1 -- 172.21.15.42:0/2660975208 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c5c004220 con 0x7f2c6c104580 2024-08-30T08:51:55.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:54.960+0000 7f2c637fe640 1 -- 172.21.15.42:0/2660975208 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f2c5c007e40 con 0x7f2c6c104580 2024-08-30T08:51:55.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:54.960+0000 7f2c74365640 1 -- 172.21.15.42:0/2660975208 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c6c19b940 con 0x7f2c6c104580 2024-08-30T08:51:55.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:54.960+0000 7f2c637fe640 1 -- 172.21.15.42:0/2660975208 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c5c004a00 con 0x7f2c6c104580 2024-08-30T08:51:55.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:54.960+0000 7f2c74365640 1 -- 172.21.15.42:0/2660975208 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c6c1a2040 con 0x7f2c6c104580 2024-08-30T08:51:55.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:54.961+0000 7f2c74365640 1 -- 172.21.15.42:0/2660975208 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c6c103960 con 0x7f2c6c104580 2024-08-30T08:51:55.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:54.963+0000 7f2c637fe640 1 -- 172.21.15.42:0/2660975208 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 15) ==== 152521+0+0 (secure 0 0 0) 0x7f2c5c0079b0 con 0x7f2c6c104580 2024-08-30T08:51:55.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:54.964+0000 7f2c637fe640 1 --2- 172.21.15.42:0/2660975208 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f2c580b4b20 0x7f2c580b7030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:51:55.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:54.964+0000 7f2c72101640 1 --2- 172.21.15.42:0/2660975208 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f2c580b4b20 0x7f2c580b7030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:51:55.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:54.964+0000 7f2c637fe640 1 -- 172.21.15.42:0/2660975208 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f2c5c0e3430 con 0x7f2c6c104580 2024-08-30T08:51:55.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:54.964+0000 7f2c72101640 1 --2- 172.21.15.42:0/2660975208 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f2c580b4b20 0x7f2c580b7030 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f2c680058c0 tx=0x7f2c6800a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:51:55.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:54.967+0000 7f2c637fe640 1 -- 172.21.15.42:0/2660975208 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f2c5c014030 con 0x7f2c6c104580 2024-08-30T08:51:55.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:55.157+0000 7f2c74365640 1 -- 172.21.15.42:0/2660975208 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command([{prefix=config set, name=osd_op_complaint_time}] v 0) -- 0x7f2c6c10a6d0 con 0x7f2c6c104580 2024-08-30T08:51:55.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:55.161+0000 7f2c637fe640 1 -- 172.21.15.42:0/2660975208 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_op_complaint_time}]=0 v22)=0 v22) ==== 121+0+0 (secure 0 0 0) 0x7f2c6c10a6d0 con 0x7f2c6c104580 2024-08-30T08:51:55.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:55.163+0000 7f2c74365640 1 -- 172.21.15.42:0/2660975208 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f2c580b4b20 msgr2=0x7f2c580b7030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:51:55.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:55.163+0000 7f2c74365640 1 --2- 172.21.15.42:0/2660975208 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f2c580b4b20 0x7f2c580b7030 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f2c680058c0 tx=0x7f2c6800a2d0 comp rx=0 tx=0).stop 2024-08-30T08:51:55.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:55.163+0000 7f2c74365640 1 -- 172.21.15.42:0/2660975208 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2c6c104580 msgr2=0x7f2c6c19a290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:51:55.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:55.163+0000 7f2c74365640 1 --2- 172.21.15.42:0/2660975208 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2c6c104580 0x7f2c6c19a290 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f2c5c00c890 tx=0x7f2c5c00cd50 comp rx=0 tx=0).stop 2024-08-30T08:51:55.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:55.163+0000 7f2c74365640 1 -- 172.21.15.42:0/2660975208 shutdown_connections 2024-08-30T08:51:55.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:55.163+0000 7f2c74365640 1 --2- 172.21.15.42:0/2660975208 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f2c580b4b20 0x7f2c580b7030 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:51:55.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:55.163+0000 7f2c74365640 1 --2- 172.21.15.42:0/2660975208 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2c6c104f20 0x7f2c6c19ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:51:55.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:55.163+0000 7f2c74365640 1 --2- 172.21.15.42:0/2660975208 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2c6c104580 0x7f2c6c19a290 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:51:55.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:55.163+0000 7f2c74365640 1 --2- 172.21.15.42:0/2660975208 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2c6c1034f0 0x7f2c6c199c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:51:55.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:55.163+0000 7f2c74365640 1 -- 172.21.15.42:0/2660975208 >> 172.21.15.42:0/2660975208 conn(0x7f2c6c0fee60 msgr2=0x7f2c6c1078a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:51:55.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:55.163+0000 7f2c74365640 1 -- 172.21.15.42:0/2660975208 shutdown_connections 2024-08-30T08:51:55.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:55.163+0000 7f2c74365640 1 -- 172.21.15.42:0/2660975208 wait complete. 2024-08-30T08:51:55.767 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:55 smithi042 ceph-mon[18652]: mgrmap e15: x(active, since 69s), standbys: z, y 2024-08-30T08:51:55.767 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:55 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "y", "id": "y"} : dispatch 2024-08-30T08:51:55.767 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:55 smithi042 ceph-mon[18652]: from='client.? 172.21.15.42:0/2660975208' entity='client.admin' 2024-08-30T08:51:55.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:55.764+0000 7f4641fc9640 1 -- 172.21.15.42:0/300933165 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f463c1035d0 msgr2=0x7f463c109f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:51:55.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:55.764+0000 7f4641fc9640 1 --2- 172.21.15.42:0/300933165 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f463c1035d0 0x7f463c109f70 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f4634009ca0 tx=0x7f463402a950 comp rx=0 tx=0).stop 2024-08-30T08:51:55.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:55.764+0000 7f4641fc9640 1 -- 172.21.15.42:0/300933165 shutdown_connections 2024-08-30T08:51:55.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:55.764+0000 7f4641fc9640 1 --2- 172.21.15.42:0/300933165 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f463c102c30 0x7f463c103040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:51:55.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:55.764+0000 7f4641fc9640 1 --2- 172.21.15.42:0/300933165 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f463c1035d0 0x7f463c109f70 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:51:55.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:55.764+0000 7f4641fc9640 1 --2- 172.21.15.42:0/300933165 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f463c101ba0 0x7f463c102010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:51:55.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:55.764+0000 7f4641fc9640 1 -- 172.21.15.42:0/300933165 >> 172.21.15.42:0/300933165 conn(0x7f463c0fd510 msgr2=0x7f463c0ff9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:51:55.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:55.764+0000 7f4641fc9640 1 -- 172.21.15.42:0/300933165 shutdown_connections 2024-08-30T08:51:55.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:55.764+0000 7f4641fc9640 1 -- 172.21.15.42:0/300933165 wait complete. 2024-08-30T08:51:55.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:55.765+0000 7f4641fc9640 1 Processor -- start 2024-08-30T08:51:55.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:55.765+0000 7f4641fc9640 1 -- start start 2024-08-30T08:51:55.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:55.766+0000 7f4641fc9640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f463c101ba0 0x7f463c199c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:51:55.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:55.766+0000 7f4641fc9640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f463c102c30 0x7f463c19a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:51:55.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:55.766+0000 7f4641fc9640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f463c1035d0 0x7f463c19ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:51:55.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:55.766+0000 7f4641fc9640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f463c10f060 con 0x7f463c102c30 2024-08-30T08:51:55.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:55.766+0000 7f4641fc9640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f463c10eee0 con 0x7f463c1035d0 2024-08-30T08:51:55.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:55.766+0000 7f4641fc9640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f463c10f1e0 con 0x7f463c101ba0 2024-08-30T08:51:55.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:55.766+0000 7f463affd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f463c102c30 0x7f463c19a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:51:55.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:55.766+0000 7f463affd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f463c102c30 0x7f463c19a2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:39420/0 (socket says 172.21.15.42:39420) 2024-08-30T08:51:55.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:55.766+0000 7f463affd640 1 -- 172.21.15.42:0/1262259656 learned_addr learned my addr 172.21.15.42:0/1262259656 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T08:51:55.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:55.766+0000 7f463b7fe640 1 --2- 172.21.15.42:0/1262259656 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f463c101ba0 0x7f463c199c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:51:55.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:55.766+0000 7f463bfff640 1 --2- 172.21.15.42:0/1262259656 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f463c1035d0 0x7f463c19ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:51:55.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:55.766+0000 7f463affd640 1 -- 172.21.15.42:0/1262259656 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f463c101ba0 msgr2=0x7f463c199c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:51:55.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:55.766+0000 7f463affd640 1 --2- 172.21.15.42:0/1262259656 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f463c101ba0 0x7f463c199c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:51:55.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:55.766+0000 7f463affd640 1 -- 172.21.15.42:0/1262259656 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f463c1035d0 msgr2=0x7f463c19ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:51:55.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:55.766+0000 7f463affd640 1 --2- 172.21.15.42:0/1262259656 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f463c1035d0 0x7f463c19ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:51:55.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:55.766+0000 7f463affd640 1 -- 172.21.15.42:0/1262259656 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f463c19b5f0 con 0x7f463c102c30 2024-08-30T08:51:55.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:55.766+0000 7f463bfff640 1 --2- 172.21.15.42:0/1262259656 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f463c1035d0 0x7f463c19ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-30T08:51:55.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:55.766+0000 7f463affd640 1 --2- 172.21.15.42:0/1262259656 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f463c102c30 0x7f463c19a2c0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f463000c920 tx=0x7f463000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:51:55.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:55.766+0000 7f4638ff9640 1 -- 172.21.15.42:0/1262259656 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4630007c70 con 0x7f463c102c30 2024-08-30T08:51:55.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:55.767+0000 7f4638ff9640 1 -- 172.21.15.42:0/1262259656 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f4630007e10 con 0x7f463c102c30 2024-08-30T08:51:55.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:55.767+0000 7f4638ff9640 1 -- 172.21.15.42:0/1262259656 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4630018720 con 0x7f463c102c30 2024-08-30T08:51:55.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:55.767+0000 7f4641fc9640 1 -- 172.21.15.42:0/1262259656 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f463c19b8e0 con 0x7f463c102c30 2024-08-30T08:51:55.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:55.767+0000 7f4641fc9640 1 -- 172.21.15.42:0/1262259656 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f463c1a2070 con 0x7f463c102c30 2024-08-30T08:51:55.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:55.768+0000 7f4641fc9640 1 -- 172.21.15.42:0/1262259656 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f463c10f7c0 con 0x7f463c102c30 2024-08-30T08:51:55.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:55.770+0000 7f4638ff9640 1 -- 172.21.15.42:0/1262259656 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 15) ==== 152521+0+0 (secure 0 0 0) 0x7f463000f450 con 0x7f463c102c30 2024-08-30T08:51:55.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:55.771+0000 7f4638ff9640 1 --2- 172.21.15.42:0/1262259656 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f46240b4b20 0x7f46240b7030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:51:55.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:55.771+0000 7f463b7fe640 1 --2- 172.21.15.42:0/1262259656 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f46240b4b20 0x7f46240b7030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:51:55.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:55.771+0000 7f4638ff9640 1 -- 172.21.15.42:0/1262259656 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f46300e3800 con 0x7f463c102c30 2024-08-30T08:51:55.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:55.772+0000 7f463b7fe640 1 --2- 172.21.15.42:0/1262259656 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f46240b4b20 0x7f46240b7030 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f462c0058c0 tx=0x7f462c00a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:51:55.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:55.774+0000 7f4638ff9640 1 -- 172.21.15.42:0/1262259656 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f4630014030 con 0x7f463c102c30 2024-08-30T08:51:55.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:55 smithi190 ceph-mon[22001]: mgrmap e15: x(active, since 69s), standbys: z, y 2024-08-30T08:51:55.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:55 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "y", "id": "y"} : dispatch 2024-08-30T08:51:55.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:55 smithi190 ceph-mon[22001]: from='client.? 172.21.15.42:0/2660975208' entity='client.admin' 2024-08-30T08:51:55.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:55.961+0000 7f4641fc9640 1 -- 172.21.15.42:0/1262259656 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command([{prefix=config set, name=osd_op_complaint_time}] v 0) -- 0x7f463c102010 con 0x7f463c102c30 2024-08-30T08:51:55.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:55.965+0000 7f4638ff9640 1 -- 172.21.15.42:0/1262259656 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_op_complaint_time}]=0 v23)=0 v23) ==== 121+0+0 (secure 0 0 0) 0x7f463c102010 con 0x7f463c102c30 2024-08-30T08:51:55.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:55.967+0000 7f4641fc9640 1 -- 172.21.15.42:0/1262259656 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f46240b4b20 msgr2=0x7f46240b7030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:51:55.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:55.967+0000 7f4641fc9640 1 --2- 172.21.15.42:0/1262259656 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f46240b4b20 0x7f46240b7030 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f462c0058c0 tx=0x7f462c00a2d0 comp rx=0 tx=0).stop 2024-08-30T08:51:55.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:55.967+0000 7f4641fc9640 1 -- 172.21.15.42:0/1262259656 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f463c102c30 msgr2=0x7f463c19a2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:51:55.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:55.967+0000 7f4641fc9640 1 --2- 172.21.15.42:0/1262259656 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f463c102c30 0x7f463c19a2c0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f463000c920 tx=0x7f463000cde0 comp rx=0 tx=0).stop 2024-08-30T08:51:55.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:55.967+0000 7f4641fc9640 1 -- 172.21.15.42:0/1262259656 shutdown_connections 2024-08-30T08:51:55.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:55.967+0000 7f4641fc9640 1 --2- 172.21.15.42:0/1262259656 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f46240b4b20 0x7f46240b7030 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:51:55.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:55.967+0000 7f4641fc9640 1 --2- 172.21.15.42:0/1262259656 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f463c1035d0 0x7f463c19ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:51:55.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:55.967+0000 7f4641fc9640 1 --2- 172.21.15.42:0/1262259656 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f463c102c30 0x7f463c19a2c0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:51:55.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:55.967+0000 7f4641fc9640 1 --2- 172.21.15.42:0/1262259656 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f463c101ba0 0x7f463c199c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:51:55.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:55.967+0000 7f4641fc9640 1 -- 172.21.15.42:0/1262259656 >> 172.21.15.42:0/1262259656 conn(0x7f463c0fd510 msgr2=0x7f463c105f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:51:55.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:55.967+0000 7f4641fc9640 1 -- 172.21.15.42:0/1262259656 shutdown_connections 2024-08-30T08:51:55.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:55.968+0000 7f4641fc9640 1 -- 172.21.15.42:0/1262259656 wait complete. 2024-08-30T08:51:56.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:55 smithi167 ceph-mon[22063]: mgrmap e15: x(active, since 69s), standbys: z, y 2024-08-30T08:51:56.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:55 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "y", "id": "y"} : dispatch 2024-08-30T08:51:56.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:55 smithi167 ceph-mon[22063]: from='client.? 172.21.15.42:0/2660975208' entity='client.admin' 2024-08-30T08:51:56.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:56.400+0000 7f6d40546640 1 -- 172.21.15.42:0/3304625719 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6d38102410 msgr2=0x7f6d38102880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:51:56.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:56.400+0000 7f6d40546640 1 --2- 172.21.15.42:0/3304625719 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6d38102410 0x7f6d38102880 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f6d34009900 tx=0x7f6d3402a480 comp rx=0 tx=0).stop 2024-08-30T08:51:56.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:56.400+0000 7f6d40546640 1 -- 172.21.15.42:0/3304625719 shutdown_connections 2024-08-30T08:51:56.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:56.400+0000 7f6d40546640 1 --2- 172.21.15.42:0/3304625719 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6d381034a0 0x7f6d381038b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:51:56.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:56.400+0000 7f6d40546640 1 --2- 172.21.15.42:0/3304625719 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6d38103e40 0x7f6d3810a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:51:56.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:56.400+0000 7f6d40546640 1 --2- 172.21.15.42:0/3304625719 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6d38102410 0x7f6d38102880 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:51:56.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:56.400+0000 7f6d40546640 1 -- 172.21.15.42:0/3304625719 >> 172.21.15.42:0/3304625719 conn(0x7f6d380fdd80 msgr2=0x7f6d38100230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:51:56.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:56.400+0000 7f6d40546640 1 -- 172.21.15.42:0/3304625719 shutdown_connections 2024-08-30T08:51:56.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:56.400+0000 7f6d40546640 1 -- 172.21.15.42:0/3304625719 wait complete. 2024-08-30T08:51:56.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:56.401+0000 7f6d40546640 1 Processor -- start 2024-08-30T08:51:56.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:56.401+0000 7f6d40546640 1 -- start start 2024-08-30T08:51:56.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:56.401+0000 7f6d40546640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6d38102410 0x7f6d38198bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:51:56.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:56.401+0000 7f6d40546640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6d381034a0 0x7f6d38199260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:51:56.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:56.401+0000 7f6d40546640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6d38103e40 0x7f6d38199e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:51:56.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:56.401+0000 7f6d40546640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f6d3810df50 con 0x7f6d38103e40 2024-08-30T08:51:56.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:56.401+0000 7f6d40546640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f6d3810ddd0 con 0x7f6d38102410 2024-08-30T08:51:56.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:56.401+0000 7f6d40546640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f6d3810e0d0 con 0x7f6d381034a0 2024-08-30T08:51:56.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:56.402+0000 7f6d3eae3640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6d38103e40 0x7f6d38199e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:51:56.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:56.402+0000 7f6d3eae3640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6d38103e40 0x7f6d38199e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:39436/0 (socket says 172.21.15.42:39436) 2024-08-30T08:51:56.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:56.402+0000 7f6d3eae3640 1 -- 172.21.15.42:0/1006219813 learned_addr learned my addr 172.21.15.42:0/1006219813 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T08:51:56.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:56.402+0000 7f6d3dae1640 1 --2- 172.21.15.42:0/1006219813 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6d381034a0 0x7f6d38199260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:51:56.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:56.402+0000 7f6d3e2e2640 1 --2- 172.21.15.42:0/1006219813 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6d38102410 0x7f6d38198bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:51:56.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:56.402+0000 7f6d3eae3640 1 -- 172.21.15.42:0/1006219813 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6d381034a0 msgr2=0x7f6d38199260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:51:56.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:56.402+0000 7f6d3eae3640 1 --2- 172.21.15.42:0/1006219813 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6d381034a0 0x7f6d38199260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:51:56.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:56.402+0000 7f6d3eae3640 1 -- 172.21.15.42:0/1006219813 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6d38102410 msgr2=0x7f6d38198bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:51:56.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:56.402+0000 7f6d3eae3640 1 --2- 172.21.15.42:0/1006219813 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6d38102410 0x7f6d38198bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:51:56.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:56.402+0000 7f6d3eae3640 1 -- 172.21.15.42:0/1006219813 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d3819a590 con 0x7f6d38103e40 2024-08-30T08:51:56.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:56.402+0000 7f6d3eae3640 1 --2- 172.21.15.42:0/1006219813 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6d38103e40 0x7f6d38199e00 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f6d3000cce0 tx=0x7f6d3000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:51:56.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:56.402+0000 7f6d2f7fe640 1 -- 172.21.15.42:0/1006219813 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d3000f450 con 0x7f6d38103e40 2024-08-30T08:51:56.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:56.402+0000 7f6d2f7fe640 1 -- 172.21.15.42:0/1006219813 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f6d3000fa80 con 0x7f6d38103e40 2024-08-30T08:51:56.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:56.402+0000 7f6d40546640 1 -- 172.21.15.42:0/1006219813 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d3819a880 con 0x7f6d38103e40 2024-08-30T08:51:56.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:56.402+0000 7f6d2f7fe640 1 -- 172.21.15.42:0/1006219813 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d30018ae0 con 0x7f6d38103e40 2024-08-30T08:51:56.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:56.403+0000 7f6d40546640 1 -- 172.21.15.42:0/1006219813 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d381a0e40 con 0x7f6d38103e40 2024-08-30T08:51:56.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:56.404+0000 7f6d40546640 1 -- 172.21.15.42:0/1006219813 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d38102880 con 0x7f6d38103e40 2024-08-30T08:51:56.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:56.405+0000 7f6d2f7fe640 1 -- 172.21.15.42:0/1006219813 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 15) ==== 152521+0+0 (secure 0 0 0) 0x7f6d3000f5f0 con 0x7f6d38103e40 2024-08-30T08:51:56.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:56.406+0000 7f6d2f7fe640 1 --2- 172.21.15.42:0/1006219813 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f6d240b4aa0 0x7f6d240b6fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:51:56.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:56.406+0000 7f6d3e2e2640 1 --2- 172.21.15.42:0/1006219813 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f6d240b4aa0 0x7f6d240b6fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:51:56.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:56.406+0000 7f6d2f7fe640 1 -- 172.21.15.42:0/1006219813 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f6d300e3c10 con 0x7f6d38103e40 2024-08-30T08:51:56.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:56.406+0000 7f6d3e2e2640 1 --2- 172.21.15.42:0/1006219813 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f6d240b4aa0 0x7f6d240b6fb0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f6d34000c00 tx=0x7f6d34031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:51:56.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:56.409+0000 7f6d2f7fe640 1 -- 172.21.15.42:0/1006219813 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f6d30014030 con 0x7f6d38103e40 2024-08-30T08:51:56.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:56.597+0000 7f6d40546640 1 -- 172.21.15.42:0/1006219813 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command([{prefix=config set, name=mds_bal_split_size}] v 0) -- 0x7f6d381095f0 con 0x7f6d38103e40 2024-08-30T08:51:56.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:56.601+0000 7f6d2f7fe640 1 -- 172.21.15.42:0/1006219813 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_split_size}]=0 v24)=0 v24) ==== 118+0+0 (secure 0 0 0) 0x7f6d300acaa0 con 0x7f6d38103e40 2024-08-30T08:51:56.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:56.603+0000 7f6d40546640 1 -- 172.21.15.42:0/1006219813 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f6d240b4aa0 msgr2=0x7f6d240b6fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:51:56.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:56.603+0000 7f6d40546640 1 --2- 172.21.15.42:0/1006219813 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f6d240b4aa0 0x7f6d240b6fb0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f6d34000c00 tx=0x7f6d34031400 comp rx=0 tx=0).stop 2024-08-30T08:51:56.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:56.603+0000 7f6d40546640 1 -- 172.21.15.42:0/1006219813 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6d38103e40 msgr2=0x7f6d38199e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:51:56.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:56.603+0000 7f6d40546640 1 --2- 172.21.15.42:0/1006219813 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6d38103e40 0x7f6d38199e00 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f6d3000cce0 tx=0x7f6d3000a620 comp rx=0 tx=0).stop 2024-08-30T08:51:56.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:56.603+0000 7f6d40546640 1 -- 172.21.15.42:0/1006219813 shutdown_connections 2024-08-30T08:51:56.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:56.603+0000 7f6d40546640 1 --2- 172.21.15.42:0/1006219813 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f6d240b4aa0 0x7f6d240b6fb0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:51:56.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:56.603+0000 7f6d40546640 1 --2- 172.21.15.42:0/1006219813 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6d381034a0 0x7f6d38199260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:51:56.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:56.603+0000 7f6d40546640 1 --2- 172.21.15.42:0/1006219813 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6d38103e40 0x7f6d38199e00 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:51:56.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:56.603+0000 7f6d40546640 1 --2- 172.21.15.42:0/1006219813 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6d38102410 0x7f6d38198bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:51:56.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:56.603+0000 7f6d40546640 1 -- 172.21.15.42:0/1006219813 >> 172.21.15.42:0/1006219813 conn(0x7f6d380fdd80 msgr2=0x7f6d381067c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:51:56.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:56.604+0000 7f6d40546640 1 -- 172.21.15.42:0/1006219813 shutdown_connections 2024-08-30T08:51:56.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:56.604+0000 7f6d40546640 1 -- 172.21.15.42:0/1006219813 wait complete. 2024-08-30T08:51:56.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:56 smithi042 ceph-mon[18652]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-30T08:51:56.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:56 smithi042 ceph-mon[18652]: from='client.? 172.21.15.42:0/1262259656' entity='client.admin' 2024-08-30T08:51:56.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:56 smithi042 ceph-mon[18652]: from='client.? 172.21.15.42:0/1006219813' entity='client.admin' 2024-08-30T08:51:56.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:56 smithi190 ceph-mon[22001]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-30T08:51:56.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:56 smithi190 ceph-mon[22001]: from='client.? 172.21.15.42:0/1262259656' entity='client.admin' 2024-08-30T08:51:56.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:56 smithi190 ceph-mon[22001]: from='client.? 172.21.15.42:0/1006219813' entity='client.admin' 2024-08-30T08:51:57.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:56 smithi167 ceph-mon[22063]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-30T08:51:57.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:56 smithi167 ceph-mon[22063]: from='client.? 172.21.15.42:0/1262259656' entity='client.admin' 2024-08-30T08:51:57.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:56 smithi167 ceph-mon[22063]: from='client.? 172.21.15.42:0/1006219813' entity='client.admin' 2024-08-30T08:51:57.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:57.125+0000 7f385162b640 1 -- 172.21.15.42:0/462792807 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f384c109210 msgr2=0x7f384c10b720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:51:57.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:57.125+0000 7f385162b640 1 --2- 172.21.15.42:0/462792807 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f384c109210 0x7f384c10b720 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f3844009ca0 tx=0x7f384402a8c0 comp rx=0 tx=0).stop 2024-08-30T08:51:57.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:57.125+0000 7f385162b640 1 -- 172.21.15.42:0/462792807 shutdown_connections 2024-08-30T08:51:57.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:57.125+0000 7f385162b640 1 --2- 172.21.15.42:0/462792807 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f384c109210 0x7f384c10b720 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:51:57.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:57.125+0000 7f385162b640 1 --2- 172.21.15.42:0/462792807 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f384c1022d0 0x7f384c108ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:51:57.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:57.125+0000 7f385162b640 1 --2- 172.21.15.42:0/462792807 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f384c1018a0 0x7f384c101cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:51:57.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:57.125+0000 7f385162b640 1 -- 172.21.15.42:0/462792807 >> 172.21.15.42:0/462792807 conn(0x7f384c0fd510 msgr2=0x7f384c0ff980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:51:57.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:57.125+0000 7f385162b640 1 -- 172.21.15.42:0/462792807 shutdown_connections 2024-08-30T08:51:57.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:57.125+0000 7f385162b640 1 -- 172.21.15.42:0/462792807 wait complete. 2024-08-30T08:51:57.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:57.126+0000 7f385162b640 1 Processor -- start 2024-08-30T08:51:57.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:57.126+0000 7f385162b640 1 -- start start 2024-08-30T08:51:57.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:57.126+0000 7f385162b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f384c1018a0 0x7f384c104030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:51:57.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:57.126+0000 7f385162b640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f384c1022d0 0x7f384c102900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:51:57.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:57.126+0000 7f385162b640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f384c109210 0x7f384c102e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:51:57.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:57.126+0000 7f385162b640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f384c10efb0 con 0x7f384c1022d0 2024-08-30T08:51:57.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:57.126+0000 7f385162b640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f384c10ee30 con 0x7f384c109210 2024-08-30T08:51:57.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:57.126+0000 7f385162b640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f384c10f130 con 0x7f384c1018a0 2024-08-30T08:51:57.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:57.126+0000 7f384a7fc640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f384c1022d0 0x7f384c102900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:51:57.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:57.127+0000 7f384a7fc640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f384c1022d0 0x7f384c102900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:39450/0 (socket says 172.21.15.42:39450) 2024-08-30T08:51:57.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:57.127+0000 7f384a7fc640 1 -- 172.21.15.42:0/2764184071 learned_addr learned my addr 172.21.15.42:0/2764184071 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T08:51:57.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:57.127+0000 7f384affd640 1 --2- 172.21.15.42:0/2764184071 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f384c1018a0 0x7f384c104030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:51:57.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:57.127+0000 7f384b7fe640 1 --2- 172.21.15.42:0/2764184071 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f384c109210 0x7f384c102e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:51:57.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:57.127+0000 7f384a7fc640 1 -- 172.21.15.42:0/2764184071 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f384c1018a0 msgr2=0x7f384c104030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:51:57.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:57.127+0000 7f384a7fc640 1 --2- 172.21.15.42:0/2764184071 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f384c1018a0 0x7f384c104030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:51:57.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:57.127+0000 7f384a7fc640 1 -- 172.21.15.42:0/2764184071 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f384c109210 msgr2=0x7f384c102e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:51:57.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:57.127+0000 7f384a7fc640 1 --2- 172.21.15.42:0/2764184071 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f384c109210 0x7f384c102e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:51:57.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:57.127+0000 7f384a7fc640 1 -- 172.21.15.42:0/2764184071 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f384c103540 con 0x7f384c1022d0 2024-08-30T08:51:57.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:57.127+0000 7f384a7fc640 1 --2- 172.21.15.42:0/2764184071 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f384c1022d0 0x7f384c102900 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f383400d950 tx=0x7f383400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:51:57.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:57.127+0000 7f3843fff640 1 -- 172.21.15.42:0/2764184071 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3834017030 con 0x7f384c1022d0 2024-08-30T08:51:57.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:57.127+0000 7f3843fff640 1 -- 172.21.15.42:0/2764184071 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f3834004020 con 0x7f384c1022d0 2024-08-30T08:51:57.134 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:57.127+0000 7f385162b640 1 -- 172.21.15.42:0/2764184071 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f384c1a3c40 con 0x7f384c1022d0 2024-08-30T08:51:57.134 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:57.127+0000 7f3843fff640 1 -- 172.21.15.42:0/2764184071 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38340055e0 con 0x7f384c1022d0 2024-08-30T08:51:57.134 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:57.127+0000 7f385162b640 1 -- 172.21.15.42:0/2764184071 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f384c1a40a0 con 0x7f384c1022d0 2024-08-30T08:51:57.134 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:57.128+0000 7f385162b640 1 -- 172.21.15.42:0/2764184071 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f384c101cb0 con 0x7f384c1022d0 2024-08-30T08:51:57.135 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:57.130+0000 7f3843fff640 1 -- 172.21.15.42:0/2764184071 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 15) ==== 152521+0+0 (secure 0 0 0) 0x7f3834009a30 con 0x7f384c1022d0 2024-08-30T08:51:57.135 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:57.130+0000 7f3843fff640 1 --2- 172.21.15.42:0/2764184071 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f38380b4aa0 0x7f38380b6fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:51:57.135 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:57.131+0000 7f384affd640 1 --2- 172.21.15.42:0/2764184071 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f38380b4aa0 0x7f38380b6fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:51:57.135 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:57.131+0000 7f3843fff640 1 -- 172.21.15.42:0/2764184071 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f38340e3430 con 0x7f384c1022d0 2024-08-30T08:51:57.135 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:57.131+0000 7f384affd640 1 --2- 172.21.15.42:0/2764184071 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f38380b4aa0 0x7f38380b6fb0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f383c009bd0 tx=0x7f383c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:51:57.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:57.134+0000 7f3843fff640 1 -- 172.21.15.42:0/2764184071 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f38340ac230 con 0x7f384c1022d0 2024-08-30T08:51:57.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:57.321+0000 7f385162b640 1 -- 172.21.15.42:0/2764184071 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command([{prefix=config set, name=debug_ms}] v 0) -- 0x7f384c108110 con 0x7f384c1022d0 2024-08-30T08:51:57.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:57.325+0000 7f3843fff640 1 -- 172.21.15.42:0/2764184071 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_ms}]=0 v25)=0 v25) ==== 106+0+0 (secure 0 0 0) 0x7f384c108110 con 0x7f384c1022d0 2024-08-30T08:51:57.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:57.327+0000 7f385162b640 1 -- 172.21.15.42:0/2764184071 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f38380b4aa0 msgr2=0x7f38380b6fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:51:57.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:57.327+0000 7f385162b640 1 --2- 172.21.15.42:0/2764184071 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f38380b4aa0 0x7f38380b6fb0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f383c009bd0 tx=0x7f383c005f90 comp rx=0 tx=0).stop 2024-08-30T08:51:57.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:57.327+0000 7f385162b640 1 -- 172.21.15.42:0/2764184071 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f384c1022d0 msgr2=0x7f384c102900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:51:57.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:57.327+0000 7f385162b640 1 --2- 172.21.15.42:0/2764184071 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f384c1022d0 0x7f384c102900 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f383400d950 tx=0x7f383400de10 comp rx=0 tx=0).stop 2024-08-30T08:51:57.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:57.328+0000 7f385162b640 1 -- 172.21.15.42:0/2764184071 shutdown_connections 2024-08-30T08:51:57.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:57.328+0000 7f385162b640 1 --2- 172.21.15.42:0/2764184071 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f38380b4aa0 0x7f38380b6fb0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:51:57.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:57.328+0000 7f385162b640 1 --2- 172.21.15.42:0/2764184071 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f384c109210 0x7f384c102e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:51:57.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:57.328+0000 7f385162b640 1 --2- 172.21.15.42:0/2764184071 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f384c1022d0 0x7f384c102900 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:51:57.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:57.328+0000 7f385162b640 1 --2- 172.21.15.42:0/2764184071 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f384c1018a0 0x7f384c104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:51:57.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:57.328+0000 7f385162b640 1 -- 172.21.15.42:0/2764184071 >> 172.21.15.42:0/2764184071 conn(0x7f384c0fd510 msgr2=0x7f384c0fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:51:57.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:57.328+0000 7f385162b640 1 -- 172.21.15.42:0/2764184071 shutdown_connections 2024-08-30T08:51:57.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:57.328+0000 7f385162b640 1 -- 172.21.15.42:0/2764184071 wait complete. 2024-08-30T08:51:58.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:58.431+0000 7f271835c640 1 -- 172.21.15.42:0/2095590270 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2710103e20 msgr2=0x7f271010a7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:51:58.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:58.431+0000 7f271835c640 1 --2- 172.21.15.42:0/2095590270 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2710103e20 0x7f271010a7c0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f2708009ca0 tx=0x7f270802a950 comp rx=0 tx=0).stop 2024-08-30T08:51:58.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:58.431+0000 7f271835c640 1 -- 172.21.15.42:0/2095590270 shutdown_connections 2024-08-30T08:51:58.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:58.431+0000 7f271835c640 1 --2- 172.21.15.42:0/2095590270 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2710103e20 0x7f271010a7c0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:51:58.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:58.431+0000 7f271835c640 1 --2- 172.21.15.42:0/2095590270 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2710103480 0x7f2710103890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:51:58.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:58.431+0000 7f271835c640 1 --2- 172.21.15.42:0/2095590270 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f27101023f0 0x7f2710102860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:51:58.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:58.431+0000 7f271835c640 1 -- 172.21.15.42:0/2095590270 >> 172.21.15.42:0/2095590270 conn(0x7f27100fdd80 msgr2=0x7f2710100210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:51:58.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:58.431+0000 7f271835c640 1 -- 172.21.15.42:0/2095590270 shutdown_connections 2024-08-30T08:51:58.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:58.431+0000 7f271835c640 1 -- 172.21.15.42:0/2095590270 wait complete. 2024-08-30T08:51:58.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:58.432+0000 7f271835c640 1 Processor -- start 2024-08-30T08:51:58.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:58.432+0000 7f271835c640 1 -- start start 2024-08-30T08:51:58.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:58.432+0000 7f271835c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f27101023f0 0x7f2710199c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:51:58.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:58.432+0000 7f271835c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2710103480 0x7f271019a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:51:58.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:58.432+0000 7f271835c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2710103e20 0x7f271019ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:51:58.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:58.432+0000 7f271835c640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f271010f040 con 0x7f2710103480 2024-08-30T08:51:58.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:58.432+0000 7f271835c640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f271010eec0 con 0x7f27101023f0 2024-08-30T08:51:58.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:58.432+0000 7f271835c640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f271010f1c0 con 0x7f2710103e20 2024-08-30T08:51:58.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:58.432+0000 7f27158f7640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2710103480 0x7f271019a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:51:58.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:58.432+0000 7f27158f7640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2710103480 0x7f271019a2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37694/0 (socket says 172.21.15.42:37694) 2024-08-30T08:51:58.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:58.432+0000 7f27158f7640 1 -- 172.21.15.42:0/3952375214 learned_addr learned my addr 172.21.15.42:0/3952375214 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T08:51:58.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:58.432+0000 7f27160f8640 1 --2- 172.21.15.42:0/3952375214 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f27101023f0 0x7f2710199c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:51:58.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:58.432+0000 7f27168f9640 1 --2- 172.21.15.42:0/3952375214 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2710103e20 0x7f271019ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:51:58.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:58.433+0000 7f27158f7640 1 -- 172.21.15.42:0/3952375214 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2710103e20 msgr2=0x7f271019ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:51:58.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:58.433+0000 7f27158f7640 1 --2- 172.21.15.42:0/3952375214 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2710103e20 0x7f271019ae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:51:58.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:58.433+0000 7f27158f7640 1 -- 172.21.15.42:0/3952375214 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f27101023f0 msgr2=0x7f2710199c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:51:58.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:58.433+0000 7f27158f7640 1 --2- 172.21.15.42:0/3952375214 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f27101023f0 0x7f2710199c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:51:58.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:58.433+0000 7f27158f7640 1 -- 172.21.15.42:0/3952375214 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f271019b660 con 0x7f2710103480 2024-08-30T08:51:58.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:58.433+0000 7f27160f8640 1 --2- 172.21.15.42:0/3952375214 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f27101023f0 0x7f2710199c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-30T08:51:58.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:58.433+0000 7f27158f7640 1 --2- 172.21.15.42:0/3952375214 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2710103480 0x7f271019a2a0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f270000c890 tx=0x7f270000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:51:58.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:58.433+0000 7f27077fe640 1 -- 172.21.15.42:0/3952375214 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2700004220 con 0x7f2710103480 2024-08-30T08:51:58.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:58.433+0000 7f27077fe640 1 -- 172.21.15.42:0/3952375214 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(25 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f2700007e40 con 0x7f2710103480 2024-08-30T08:51:58.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:58.433+0000 7f271835c640 1 -- 172.21.15.42:0/3952375214 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f271019b950 con 0x7f2710103480 2024-08-30T08:51:58.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:58.433+0000 7f27077fe640 1 -- 172.21.15.42:0/3952375214 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2700004a00 con 0x7f2710103480 2024-08-30T08:51:58.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:58.433+0000 7f271835c640 1 -- 172.21.15.42:0/3952375214 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27101a2050 con 0x7f2710103480 2024-08-30T08:51:58.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:58.434+0000 7f271835c640 1 -- 172.21.15.42:0/3952375214 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2710102860 con 0x7f2710103480 2024-08-30T08:51:58.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:58.436+0000 7f27077fe640 1 -- 172.21.15.42:0/3952375214 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 15) ==== 152521+0+0 (secure 0 0 0) 0x7f27000079b0 con 0x7f2710103480 2024-08-30T08:51:58.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:58.436+0000 7f27077fe640 1 --2- 172.21.15.42:0/3952375214 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f26fc0b4ba0 0x7f26fc0b70b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:51:58.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:58.436+0000 7f27160f8640 1 --2- 172.21.15.42:0/3952375214 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f26fc0b4ba0 0x7f26fc0b70b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:51:58.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:58.436+0000 7f27077fe640 1 -- 172.21.15.42:0/3952375214 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f27000e3630 con 0x7f2710103480 2024-08-30T08:51:58.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:58.437+0000 7f27160f8640 1 --2- 172.21.15.42:0/3952375214 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f26fc0b4ba0 0x7f26fc0b70b0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f270c005fd0 tx=0x7f270c005ea0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:51:58.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:58.439+0000 7f27077fe640 1 -- 172.21.15.42:0/3952375214 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f2700014030 con 0x7f2710103480 2024-08-30T08:51:58.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:58.628+0000 7f271835c640 1 -- 172.21.15.42:0/3952375214 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command([{prefix=config set, name=rados_mon_op_timeout}] v 0) -- 0x7f27101095d0 con 0x7f2710103480 2024-08-30T08:51:58.631 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:58 smithi042 ceph-mon[18652]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-30T08:51:58.631 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:58 smithi042 ceph-mon[18652]: from='client.? 172.21.15.42:0/2764184071' entity='client.admin' 2024-08-30T08:51:58.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:58.633+0000 7f27077fe640 1 -- 172.21.15.42:0/3952375214 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{prefix=config set, name=rados_mon_op_timeout}]=0 v26)=0 v26) ==== 123+0+0 (secure 0 0 0) 0x7f27101095d0 con 0x7f2710103480 2024-08-30T08:51:58.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:58.633+0000 7f27077fe640 1 -- 172.21.15.42:0/3952375214 <== mon.0 v2:172.21.15.42:3300/0 8 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f27000056b0 con 0x7f2710103480 2024-08-30T08:51:58.637 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:58.634+0000 7f271835c640 1 -- 172.21.15.42:0/3952375214 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f26fc0b4ba0 msgr2=0x7f26fc0b70b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:51:58.637 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:58.634+0000 7f271835c640 1 --2- 172.21.15.42:0/3952375214 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f26fc0b4ba0 0x7f26fc0b70b0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f270c005fd0 tx=0x7f270c005ea0 comp rx=0 tx=0).stop 2024-08-30T08:51:58.637 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:58.634+0000 7f271835c640 1 -- 172.21.15.42:0/3952375214 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2710103480 msgr2=0x7f271019a2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:51:58.637 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:58.634+0000 7f271835c640 1 --2- 172.21.15.42:0/3952375214 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2710103480 0x7f271019a2a0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f270000c890 tx=0x7f270000cd50 comp rx=0 tx=0).stop 2024-08-30T08:51:58.637 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:58.634+0000 7f271835c640 1 -- 172.21.15.42:0/3952375214 shutdown_connections 2024-08-30T08:51:58.638 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:58.634+0000 7f271835c640 1 --2- 172.21.15.42:0/3952375214 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f26fc0b4ba0 0x7f26fc0b70b0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:51:58.638 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:58.634+0000 7f271835c640 1 --2- 172.21.15.42:0/3952375214 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2710103e20 0x7f271019ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:51:58.638 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:58.634+0000 7f271835c640 1 --2- 172.21.15.42:0/3952375214 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2710103480 0x7f271019a2a0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:51:58.638 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:58.634+0000 7f271835c640 1 --2- 172.21.15.42:0/3952375214 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f27101023f0 0x7f2710199c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:51:58.638 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:58.634+0000 7f271835c640 1 -- 172.21.15.42:0/3952375214 >> 172.21.15.42:0/3952375214 conn(0x7f27100fdd80 msgr2=0x7f27101067a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:51:58.638 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:58.634+0000 7f271835c640 1 -- 172.21.15.42:0/3952375214 shutdown_connections 2024-08-30T08:51:58.638 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:58.634+0000 7f271835c640 1 -- 172.21.15.42:0/3952375214 wait complete. 2024-08-30T08:51:58.685 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:58 smithi190 ceph-mon[22001]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-30T08:51:58.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:58 smithi190 ceph-mon[22001]: from='client.? 172.21.15.42:0/2764184071' entity='client.admin' 2024-08-30T08:51:58.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:58 smithi167 ceph-mon[22063]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-30T08:51:58.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:58 smithi167 ceph-mon[22063]: from='client.? 172.21.15.42:0/2764184071' entity='client.admin' 2024-08-30T08:51:59.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:59.206+0000 7f4f395ce640 1 -- 172.21.15.42:0/3338884883 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4f3406c0e0 msgr2=0x7f4f34106180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:51:59.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:59.206+0000 7f4f395ce640 1 --2- 172.21.15.42:0/3338884883 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4f3406c0e0 0x7f4f34106180 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f4f24009900 tx=0x7f4f2402a410 comp rx=0 tx=0).stop 2024-08-30T08:51:59.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:59.206+0000 7f4f395ce640 1 -- 172.21.15.42:0/3338884883 shutdown_connections 2024-08-30T08:51:59.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:59.206+0000 7f4f395ce640 1 --2- 172.21.15.42:0/3338884883 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4f341066c0 0x7f4f34108b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:51:59.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:59.206+0000 7f4f395ce640 1 --2- 172.21.15.42:0/3338884883 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4f34109260 0x7f4f3410b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:51:59.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:59.206+0000 7f4f395ce640 1 --2- 172.21.15.42:0/3338884883 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4f3406c0e0 0x7f4f34106180 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:51:59.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:59.206+0000 7f4f395ce640 1 -- 172.21.15.42:0/3338884883 >> 172.21.15.42:0/3338884883 conn(0x7f4f340fdda0 msgr2=0x7f4f34100250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:51:59.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:59.206+0000 7f4f395ce640 1 -- 172.21.15.42:0/3338884883 shutdown_connections 2024-08-30T08:51:59.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:59.206+0000 7f4f395ce640 1 -- 172.21.15.42:0/3338884883 wait complete. 2024-08-30T08:51:59.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:59.206+0000 7f4f395ce640 1 Processor -- start 2024-08-30T08:51:59.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:59.207+0000 7f4f395ce640 1 -- start start 2024-08-30T08:51:59.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:59.207+0000 7f4f395ce640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4f3406c0e0 0x7f4f34199d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:51:59.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:59.207+0000 7f4f395ce640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4f341066c0 0x7f4f3419a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:51:59.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:59.207+0000 7f4f395ce640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4f34109260 0x7f4f3419af90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:51:59.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:59.207+0000 7f4f395ce640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f4f3410f1d0 con 0x7f4f34109260 2024-08-30T08:51:59.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:59.207+0000 7f4f395ce640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4f3410f050 con 0x7f4f3406c0e0 2024-08-30T08:51:59.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:59.207+0000 7f4f395ce640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f4f3410f350 con 0x7f4f341066c0 2024-08-30T08:51:59.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:59.207+0000 7f4f337fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4f34109260 0x7f4f3419af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:51:59.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:59.207+0000 7f4f337fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4f34109260 0x7f4f3419af90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37704/0 (socket says 172.21.15.42:37704) 2024-08-30T08:51:59.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:59.207+0000 7f4f337fe640 1 -- 172.21.15.42:0/3798946941 learned_addr learned my addr 172.21.15.42:0/3798946941 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T08:51:59.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:59.207+0000 7f4f2bfff640 1 --2- 172.21.15.42:0/3798946941 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4f341066c0 0x7f4f3419a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:51:59.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:59.207+0000 7f4f32ffd640 1 --2- 172.21.15.42:0/3798946941 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4f3406c0e0 0x7f4f34199d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:51:59.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:59.207+0000 7f4f337fe640 1 -- 172.21.15.42:0/3798946941 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4f341066c0 msgr2=0x7f4f3419a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:51:59.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:59.207+0000 7f4f337fe640 1 --2- 172.21.15.42:0/3798946941 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4f341066c0 0x7f4f3419a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:51:59.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:59.207+0000 7f4f337fe640 1 -- 172.21.15.42:0/3798946941 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4f3406c0e0 msgr2=0x7f4f34199d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:51:59.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:59.207+0000 7f4f337fe640 1 --2- 172.21.15.42:0/3798946941 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4f3406c0e0 0x7f4f34199d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:51:59.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:59.207+0000 7f4f337fe640 1 -- 172.21.15.42:0/3798946941 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f3419b7b0 con 0x7f4f34109260 2024-08-30T08:51:59.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:59.207+0000 7f4f2bfff640 1 --2- 172.21.15.42:0/3798946941 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4f341066c0 0x7f4f3419a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-30T08:51:59.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:59.207+0000 7f4f337fe640 1 --2- 172.21.15.42:0/3798946941 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4f34109260 0x7f4f3419af90 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f4f2c009fd0 tx=0x7f4f2c00b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:51:59.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:59.208+0000 7f4f30ff9640 1 -- 172.21.15.42:0/3798946941 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f2c017070 con 0x7f4f34109260 2024-08-30T08:51:59.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:59.208+0000 7f4f30ff9640 1 -- 172.21.15.42:0/3798946941 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4f2c003e20 con 0x7f4f34109260 2024-08-30T08:51:59.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:59.208+0000 7f4f395ce640 1 -- 172.21.15.42:0/3798946941 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f3419baa0 con 0x7f4f34109260 2024-08-30T08:51:59.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:59.208+0000 7f4f30ff9640 1 -- 172.21.15.42:0/3798946941 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f2c010460 con 0x7f4f34109260 2024-08-30T08:51:59.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:59.208+0000 7f4f395ce640 1 -- 172.21.15.42:0/3798946941 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f341a21a0 con 0x7f4f34109260 2024-08-30T08:51:59.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:59.209+0000 7f4f395ce640 1 -- 172.21.15.42:0/3798946941 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f34106b10 con 0x7f4f34109260 2024-08-30T08:51:59.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:59.211+0000 7f4f30ff9640 1 -- 172.21.15.42:0/3798946941 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 15) ==== 152521+0+0 (secure 0 0 0) 0x7f4f2c003970 con 0x7f4f34109260 2024-08-30T08:51:59.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:59.212+0000 7f4f30ff9640 1 --2- 172.21.15.42:0/3798946941 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f4f200b4b20 0x7f4f200b7030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:51:59.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:59.212+0000 7f4f32ffd640 1 --2- 172.21.15.42:0/3798946941 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f4f200b4b20 0x7f4f200b7030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:51:59.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:59.212+0000 7f4f30ff9640 1 -- 172.21.15.42:0/3798946941 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f4f2c0e31e0 con 0x7f4f34109260 2024-08-30T08:51:59.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:59.213+0000 7f4f32ffd640 1 --2- 172.21.15.42:0/3798946941 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f4f200b4b20 0x7f4f200b7030 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f4f24000c00 tx=0x7f4f24031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:51:59.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:59.215+0000 7f4f30ff9640 1 -- 172.21.15.42:0/3798946941 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f4f2c0ac060 con 0x7f4f34109260 2024-08-30T08:51:59.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:59.408+0000 7f4f395ce640 1 -- 172.21.15.42:0/3798946941 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command([{prefix=config set, name=mds_bal_split_bits}] v 0) -- 0x7f4f340ff8c0 con 0x7f4f34109260 2024-08-30T08:51:59.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:59.412+0000 7f4f30ff9640 1 -- 172.21.15.42:0/3798946941 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_split_bits}]=0 v27)=0 v27) ==== 116+0+0 (secure 0 0 0) 0x7f4f340ff8c0 con 0x7f4f34109260 2024-08-30T08:51:59.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:59.414+0000 7f4f395ce640 1 -- 172.21.15.42:0/3798946941 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f4f200b4b20 msgr2=0x7f4f200b7030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:51:59.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:59.414+0000 7f4f395ce640 1 --2- 172.21.15.42:0/3798946941 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f4f200b4b20 0x7f4f200b7030 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f4f24000c00 tx=0x7f4f24031400 comp rx=0 tx=0).stop 2024-08-30T08:51:59.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:59.414+0000 7f4f395ce640 1 -- 172.21.15.42:0/3798946941 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4f34109260 msgr2=0x7f4f3419af90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:51:59.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:59.414+0000 7f4f395ce640 1 --2- 172.21.15.42:0/3798946941 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4f34109260 0x7f4f3419af90 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f4f2c009fd0 tx=0x7f4f2c00b260 comp rx=0 tx=0).stop 2024-08-30T08:51:59.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:59.414+0000 7f4f395ce640 1 -- 172.21.15.42:0/3798946941 shutdown_connections 2024-08-30T08:51:59.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:59.414+0000 7f4f395ce640 1 --2- 172.21.15.42:0/3798946941 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f4f200b4b20 0x7f4f200b7030 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:51:59.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:59.414+0000 7f4f395ce640 1 --2- 172.21.15.42:0/3798946941 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4f341066c0 0x7f4f3419a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:51:59.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:59.414+0000 7f4f395ce640 1 --2- 172.21.15.42:0/3798946941 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4f34109260 0x7f4f3419af90 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:51:59.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:59.414+0000 7f4f395ce640 1 --2- 172.21.15.42:0/3798946941 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4f3406c0e0 0x7f4f34199d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:51:59.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:59.414+0000 7f4f395ce640 1 -- 172.21.15.42:0/3798946941 >> 172.21.15.42:0/3798946941 conn(0x7f4f340fdda0 msgr2=0x7f4f34104b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:51:59.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:59.414+0000 7f4f395ce640 1 -- 172.21.15.42:0/3798946941 shutdown_connections 2024-08-30T08:51:59.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:59.414+0000 7f4f395ce640 1 -- 172.21.15.42:0/3798946941 wait complete. 2024-08-30T08:51:59.919 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:59 smithi042 ceph-mon[18652]: from='client.? 172.21.15.42:0/3952375214' entity='client.admin' 2024-08-30T08:51:59.919 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:51:59 smithi042 ceph-mon[18652]: from='client.? 172.21.15.42:0/3798946941' entity='client.admin' 2024-08-30T08:51:59.935 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:59 smithi190 ceph-mon[22001]: from='client.? 172.21.15.42:0/3952375214' entity='client.admin' 2024-08-30T08:51:59.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:51:59 smithi190 ceph-mon[22001]: from='client.? 172.21.15.42:0/3798946941' entity='client.admin' 2024-08-30T08:52:00.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:59.930+0000 7fc17c8c3640 1 -- 172.21.15.42:0/842999133 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc1781054f0 msgr2=0x7fc1781079d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:52:00.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:59.930+0000 7fc17c8c3640 1 --2- 172.21.15.42:0/842999133 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc1781054f0 0x7fc1781079d0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fc160009900 tx=0x7fc16002a410 comp rx=0 tx=0).stop 2024-08-30T08:52:00.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:59.930+0000 7fc17c8c3640 1 -- 172.21.15.42:0/842999133 shutdown_connections 2024-08-30T08:52:00.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:59.930+0000 7fc17c8c3640 1 --2- 172.21.15.42:0/842999133 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc178108110 0x7fc17810a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:52:00.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:59.930+0000 7fc17c8c3640 1 --2- 172.21.15.42:0/842999133 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc1781054f0 0x7fc1781079d0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:52:00.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:59.930+0000 7fc17c8c3640 1 --2- 172.21.15.42:0/842999133 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc178102a70 0x7fc178104eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:52:00.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:59.930+0000 7fc17c8c3640 1 -- 172.21.15.42:0/842999133 >> 172.21.15.42:0/842999133 conn(0x7fc1780fd510 msgr2=0x7fc1780ff980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:52:00.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:59.930+0000 7fc17c8c3640 1 -- 172.21.15.42:0/842999133 shutdown_connections 2024-08-30T08:52:00.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:59.930+0000 7fc17c8c3640 1 -- 172.21.15.42:0/842999133 wait complete. 2024-08-30T08:52:00.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:59.931+0000 7fc17c8c3640 1 Processor -- start 2024-08-30T08:52:00.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:59.931+0000 7fc17c8c3640 1 -- start start 2024-08-30T08:52:00.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:59.931+0000 7fc17c8c3640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc178102a70 0x7fc1781046f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:52:00.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:59.931+0000 7fc17c8c3640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc1781054f0 0x7fc178102fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:52:00.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:59.931+0000 7fc17c8c3640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc178108110 0x7fc178103500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:52:00.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:59.931+0000 7fc17c8c3640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fc17810e030 con 0x7fc178108110 2024-08-30T08:52:00.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:59.931+0000 7fc17c8c3640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc17810deb0 con 0x7fc178102a70 2024-08-30T08:52:00.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:59.931+0000 7fc17c8c3640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fc17810e1b0 con 0x7fc1781054f0 2024-08-30T08:52:00.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:59.931+0000 7fc176d9d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc178108110 0x7fc178103500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:52:00.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:59.931+0000 7fc176d9d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc178108110 0x7fc178103500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37730/0 (socket says 172.21.15.42:37730) 2024-08-30T08:52:00.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:59.931+0000 7fc176d9d640 1 -- 172.21.15.42:0/1980246558 learned_addr learned my addr 172.21.15.42:0/1980246558 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T08:52:00.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:59.932+0000 7fc175d9b640 1 --2- 172.21.15.42:0/1980246558 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc1781054f0 0x7fc178102fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:52:00.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:59.932+0000 7fc17659c640 1 --2- 172.21.15.42:0/1980246558 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc178102a70 0x7fc1781046f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:52:00.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:59.932+0000 7fc176d9d640 1 -- 172.21.15.42:0/1980246558 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc1781054f0 msgr2=0x7fc178102fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:52:00.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:59.932+0000 7fc176d9d640 1 --2- 172.21.15.42:0/1980246558 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc1781054f0 0x7fc178102fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:52:00.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:59.932+0000 7fc176d9d640 1 -- 172.21.15.42:0/1980246558 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc178102a70 msgr2=0x7fc1781046f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:52:00.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:59.932+0000 7fc176d9d640 1 --2- 172.21.15.42:0/1980246558 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc178102a70 0x7fc1781046f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:52:00.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:59.932+0000 7fc176d9d640 1 -- 172.21.15.42:0/1980246558 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc178103c00 con 0x7fc178108110 2024-08-30T08:52:00.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:59.932+0000 7fc176d9d640 1 --2- 172.21.15.42:0/1980246558 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc178108110 0x7fc178103500 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fc17000eaa0 tx=0x7fc17000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:52:00.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:59.932+0000 7fc16f7fe640 1 -- 172.21.15.42:0/1980246558 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc17000cdb0 con 0x7fc178108110 2024-08-30T08:52:00.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:59.932+0000 7fc17c8c3640 1 -- 172.21.15.42:0/1980246558 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1781a3da0 con 0x7fc178108110 2024-08-30T08:52:00.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:59.932+0000 7fc16f7fe640 1 -- 172.21.15.42:0/1980246558 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc170005020 con 0x7fc178108110 2024-08-30T08:52:00.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:59.932+0000 7fc16f7fe640 1 -- 172.21.15.42:0/1980246558 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1700116d0 con 0x7fc178108110 2024-08-30T08:52:00.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:59.932+0000 7fc17c8c3640 1 -- 172.21.15.42:0/1980246558 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1781a41a0 con 0x7fc178108110 2024-08-30T08:52:00.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:59.933+0000 7fc17c8c3640 1 -- 172.21.15.42:0/1980246558 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1781a43d0 con 0x7fc178108110 2024-08-30T08:52:00.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:59.935+0000 7fc16f7fe640 1 -- 172.21.15.42:0/1980246558 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 15) ==== 152521+0+0 (secure 0 0 0) 0x7fc1700051d0 con 0x7fc178108110 2024-08-30T08:52:00.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:59.936+0000 7fc16f7fe640 1 --2- 172.21.15.42:0/1980246558 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fc1640b4aa0 0x7fc1640b6fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:52:00.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:59.936+0000 7fc17659c640 1 --2- 172.21.15.42:0/1980246558 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fc1640b4aa0 0x7fc1640b6fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:52:00.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:59.936+0000 7fc16f7fe640 1 -- 172.21.15.42:0/1980246558 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fc1700e3950 con 0x7fc178108110 2024-08-30T08:52:00.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:59.937+0000 7fc17659c640 1 --2- 172.21.15.42:0/1980246558 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fc1640b4aa0 0x7fc1640b6fb0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fc168005fd0 tx=0x7fc16800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:52:00.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:51:59.943+0000 7fc16f7fe640 1 -- 172.21.15.42:0/1980246558 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7fc170016030 con 0x7fc178108110 2024-08-30T08:52:00.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:59 smithi167 ceph-mon[22063]: from='client.? 172.21.15.42:0/3952375214' entity='client.admin' 2024-08-30T08:52:00.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:51:59 smithi167 ceph-mon[22063]: from='client.? 172.21.15.42:0/3798946941' entity='client.admin' 2024-08-30T08:52:00.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:00.130+0000 7fc17c8c3640 1 -- 172.21.15.42:0/1980246558 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command([{prefix=config set, name=mds_verify_scatter}] v 0) -- 0x7fc1781a45a0 con 0x7fc178108110 2024-08-30T08:52:00.135 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:00.134+0000 7fc16f7fe640 1 -- 172.21.15.42:0/1980246558 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_verify_scatter}]=0 v28)=0 v28) ==== 119+0+0 (secure 0 0 0) 0x7fc1700ac100 con 0x7fc178108110 2024-08-30T08:52:00.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:00.136+0000 7fc17c8c3640 1 -- 172.21.15.42:0/1980246558 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fc1640b4aa0 msgr2=0x7fc1640b6fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:52:00.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:00.136+0000 7fc17c8c3640 1 --2- 172.21.15.42:0/1980246558 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fc1640b4aa0 0x7fc1640b6fb0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fc168005fd0 tx=0x7fc16800b390 comp rx=0 tx=0).stop 2024-08-30T08:52:00.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:00.136+0000 7fc17c8c3640 1 -- 172.21.15.42:0/1980246558 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc178108110 msgr2=0x7fc178103500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:52:00.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:00.136+0000 7fc17c8c3640 1 --2- 172.21.15.42:0/1980246558 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc178108110 0x7fc178103500 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fc17000eaa0 tx=0x7fc17000ef60 comp rx=0 tx=0).stop 2024-08-30T08:52:00.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:00.136+0000 7fc17c8c3640 1 -- 172.21.15.42:0/1980246558 shutdown_connections 2024-08-30T08:52:00.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:00.136+0000 7fc17c8c3640 1 --2- 172.21.15.42:0/1980246558 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fc1640b4aa0 0x7fc1640b6fb0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:52:00.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:00.136+0000 7fc17c8c3640 1 --2- 172.21.15.42:0/1980246558 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc1781054f0 0x7fc178102fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:52:00.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:00.136+0000 7fc17c8c3640 1 --2- 172.21.15.42:0/1980246558 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc178108110 0x7fc178103500 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:52:00.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:00.136+0000 7fc17c8c3640 1 --2- 172.21.15.42:0/1980246558 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc178102a70 0x7fc1781046f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:52:00.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:00.136+0000 7fc17c8c3640 1 -- 172.21.15.42:0/1980246558 >> 172.21.15.42:0/1980246558 conn(0x7fc1780fd510 msgr2=0x7fc1781076a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:52:00.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:00.136+0000 7fc17c8c3640 1 -- 172.21.15.42:0/1980246558 shutdown_connections 2024-08-30T08:52:00.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:00.136+0000 7fc17c8c3640 1 -- 172.21.15.42:0/1980246558 wait complete. 2024-08-30T08:52:00.756 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:52:00 smithi042 ceph-mon[18652]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-30T08:52:00.757 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:52:00 smithi042 ceph-mon[18652]: from='client.? 172.21.15.42:0/1980246558' entity='client.admin' 2024-08-30T08:52:00.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:52:00 smithi190 ceph-mon[22001]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-30T08:52:00.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:52:00 smithi190 ceph-mon[22001]: from='client.? 172.21.15.42:0/1980246558' entity='client.admin' 2024-08-30T08:52:01.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:52:00 smithi167 ceph-mon[22063]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-30T08:52:01.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:52:00 smithi167 ceph-mon[22063]: from='client.? 172.21.15.42:0/1980246558' entity='client.admin' 2024-08-30T08:52:01.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:01.177+0000 7f52f8483640 1 -- 172.21.15.42:0/2585145743 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f52f01071c0 msgr2=0x7f52f010db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:52:01.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:01.177+0000 7f52f8483640 1 --2- 172.21.15.42:0/2585145743 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f52f01071c0 0x7f52f010db60 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f52e8009ca0 tx=0x7f52e802a950 comp rx=0 tx=0).stop 2024-08-30T08:52:01.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:01.178+0000 7f52f8483640 1 -- 172.21.15.42:0/2585145743 shutdown_connections 2024-08-30T08:52:01.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:01.178+0000 7f52f8483640 1 --2- 172.21.15.42:0/2585145743 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f52f0106820 0x7f52f0106c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:52:01.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:01.178+0000 7f52f8483640 1 --2- 172.21.15.42:0/2585145743 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f52f01071c0 0x7f52f010db60 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:52:01.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:01.178+0000 7f52f8483640 1 --2- 172.21.15.42:0/2585145743 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f52f0105790 0x7f52f0105c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:52:01.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:01.178+0000 7f52f8483640 1 -- 172.21.15.42:0/2585145743 >> 172.21.15.42:0/2585145743 conn(0x7f52f0101140 msgr2=0x7f52f01035b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:52:01.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:01.178+0000 7f52f8483640 1 -- 172.21.15.42:0/2585145743 shutdown_connections 2024-08-30T08:52:01.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:01.178+0000 7f52f8483640 1 -- 172.21.15.42:0/2585145743 wait complete. 2024-08-30T08:52:01.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:01.178+0000 7f52f8483640 1 Processor -- start 2024-08-30T08:52:01.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:01.178+0000 7f52f8483640 1 -- start start 2024-08-30T08:52:01.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:01.179+0000 7f52f8483640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f52f0105790 0x7f52f019bf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:52:01.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:01.179+0000 7f52f8483640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f52f0106820 0x7f52f019c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:52:01.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:01.179+0000 7f52f8483640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f52f01071c0 0x7f52f019d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:52:01.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:01.179+0000 7f52f8483640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f52f01113e0 con 0x7f52f0105790 2024-08-30T08:52:01.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:01.179+0000 7f52f8483640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f52f0111260 con 0x7f52f01071c0 2024-08-30T08:52:01.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:01.179+0000 7f52f8483640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f52f0111560 con 0x7f52f0106820 2024-08-30T08:52:01.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:01.179+0000 7f52f621f640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f52f0105790 0x7f52f019bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:52:01.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:01.179+0000 7f52f621f640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f52f0105790 0x7f52f019bf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37750/0 (socket says 172.21.15.42:37750) 2024-08-30T08:52:01.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:01.179+0000 7f52f621f640 1 -- 172.21.15.42:0/1656560958 learned_addr learned my addr 172.21.15.42:0/1656560958 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T08:52:01.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:01.179+0000 7f52f5a1e640 1 --2- 172.21.15.42:0/1656560958 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f52f0106820 0x7f52f019c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:52:01.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:01.179+0000 7f52f6a20640 1 --2- 172.21.15.42:0/1656560958 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f52f01071c0 0x7f52f019d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:52:01.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:01.179+0000 7f52f621f640 1 -- 172.21.15.42:0/1656560958 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f52f0106820 msgr2=0x7f52f019c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:52:01.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:01.179+0000 7f52f621f640 1 --2- 172.21.15.42:0/1656560958 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f52f0106820 0x7f52f019c5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:52:01.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:01.179+0000 7f52f621f640 1 -- 172.21.15.42:0/1656560958 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f52f01071c0 msgr2=0x7f52f019d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:52:01.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:01.179+0000 7f52f621f640 1 --2- 172.21.15.42:0/1656560958 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f52f01071c0 0x7f52f019d180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:52:01.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:01.179+0000 7f52f621f640 1 -- 172.21.15.42:0/1656560958 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52f019d9a0 con 0x7f52f0105790 2024-08-30T08:52:01.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:01.179+0000 7f52f621f640 1 --2- 172.21.15.42:0/1656560958 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f52f0105790 0x7f52f019bf70 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f52ec00b4a0 tx=0x7f52ec00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:52:01.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:01.180+0000 7f52e77fe640 1 -- 172.21.15.42:0/1656560958 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52ec014070 con 0x7f52f0105790 2024-08-30T08:52:01.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:01.180+0000 7f52e77fe640 1 -- 172.21.15.42:0/1656560958 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f52ec010ac0 con 0x7f52f0105790 2024-08-30T08:52:01.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:01.180+0000 7f52f8483640 1 -- 172.21.15.42:0/1656560958 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52f019dc90 con 0x7f52f0105790 2024-08-30T08:52:01.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:01.180+0000 7f52e77fe640 1 -- 172.21.15.42:0/1656560958 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52ec017c30 con 0x7f52f0105790 2024-08-30T08:52:01.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:01.180+0000 7f52f8483640 1 -- 172.21.15.42:0/1656560958 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52f01a4390 con 0x7f52f0105790 2024-08-30T08:52:01.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:01.181+0000 7f52f8483640 1 -- 172.21.15.42:0/1656560958 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52f010c970 con 0x7f52f0105790 2024-08-30T08:52:01.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:01.182+0000 7f52e77fe640 1 -- 172.21.15.42:0/1656560958 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 15) ==== 152521+0+0 (secure 0 0 0) 0x7f52ec017070 con 0x7f52f0105790 2024-08-30T08:52:01.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:01.183+0000 7f52e77fe640 1 --2- 172.21.15.42:0/1656560958 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f52dc0b4aa0 0x7f52dc0b6fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:52:01.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:01.183+0000 7f52f5a1e640 1 --2- 172.21.15.42:0/1656560958 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f52dc0b4aa0 0x7f52dc0b6fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:52:01.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:01.184+0000 7f52e77fe640 1 -- 172.21.15.42:0/1656560958 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f52ec0e2af0 con 0x7f52f0105790 2024-08-30T08:52:01.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:01.184+0000 7f52f5a1e640 1 --2- 172.21.15.42:0/1656560958 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f52dc0b4aa0 0x7f52dc0b6fb0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f52e0005fd0 tx=0x7f52e000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:52:01.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:01.187+0000 7f52e77fe640 1 -- 172.21.15.42:0/1656560958 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f52ec0ab2a0 con 0x7f52f0105790 2024-08-30T08:52:01.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:01.374+0000 7f52f8483640 1 -- 172.21.15.42:0/1656560958 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command([{prefix=config set, name=rados_osd_op_timeout}] v 0) -- 0x7f52f002e430 con 0x7f52f0105790 2024-08-30T08:52:01.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:01.378+0000 7f52e77fe640 1 -- 172.21.15.42:0/1656560958 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{prefix=config set, name=rados_osd_op_timeout}]=0 v29)=0 v29) ==== 120+0+0 (secure 0 0 0) 0x7f52ec0afa60 con 0x7f52f0105790 2024-08-30T08:52:01.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:01.380+0000 7f52f8483640 1 -- 172.21.15.42:0/1656560958 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f52dc0b4aa0 msgr2=0x7f52dc0b6fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:52:01.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:01.380+0000 7f52f8483640 1 --2- 172.21.15.42:0/1656560958 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f52dc0b4aa0 0x7f52dc0b6fb0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f52e0005fd0 tx=0x7f52e000b390 comp rx=0 tx=0).stop 2024-08-30T08:52:01.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:01.380+0000 7f52f8483640 1 -- 172.21.15.42:0/1656560958 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f52f0105790 msgr2=0x7f52f019bf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:52:01.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:01.380+0000 7f52f8483640 1 --2- 172.21.15.42:0/1656560958 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f52f0105790 0x7f52f019bf70 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f52ec00b4a0 tx=0x7f52ec00b960 comp rx=0 tx=0).stop 2024-08-30T08:52:01.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:01.380+0000 7f52f8483640 1 -- 172.21.15.42:0/1656560958 shutdown_connections 2024-08-30T08:52:01.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:01.380+0000 7f52f8483640 1 --2- 172.21.15.42:0/1656560958 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f52dc0b4aa0 0x7f52dc0b6fb0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:52:01.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:01.380+0000 7f52f8483640 1 --2- 172.21.15.42:0/1656560958 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f52f0106820 0x7f52f019c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:52:01.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:01.380+0000 7f52f8483640 1 --2- 172.21.15.42:0/1656560958 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f52f01071c0 0x7f52f019d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:52:01.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:01.380+0000 7f52f8483640 1 --2- 172.21.15.42:0/1656560958 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f52f0105790 0x7f52f019bf70 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:52:01.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:01.380+0000 7f52f8483640 1 -- 172.21.15.42:0/1656560958 >> 172.21.15.42:0/1656560958 conn(0x7f52f0101140 msgr2=0x7f52f0109b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:52:01.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:01.380+0000 7f52f8483640 1 -- 172.21.15.42:0/1656560958 shutdown_connections 2024-08-30T08:52:01.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:01.380+0000 7f52f8483640 1 -- 172.21.15.42:0/1656560958 wait complete. 2024-08-30T08:52:02.580 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:52:02 smithi042 ceph-mon[18652]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-30T08:52:02.580 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:52:02 smithi042 ceph-mon[18652]: from='client.? 172.21.15.42:0/1656560958' entity='client.admin' 2024-08-30T08:52:02.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:52:02 smithi190 ceph-mon[22001]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-30T08:52:02.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:52:02 smithi190 ceph-mon[22001]: from='client.? 172.21.15.42:0/1656560958' entity='client.admin' 2024-08-30T08:52:02.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:52:02 smithi167 ceph-mon[22063]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-30T08:52:02.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:52:02 smithi167 ceph-mon[22063]: from='client.? 172.21.15.42:0/1656560958' entity='client.admin' 2024-08-30T08:52:03.808 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'b0895c1a-66ac-11ef-bcd4-c7b262605968', '--', 'ceph', 'config', 'set', 'client', 'rados_osd_op_timeout', '900'], check_status=True, hostname='smithi042') 2024-08-30T08:52:03.809 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'b0895c1a-66ac-11ef-bcd4-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'debug_mds', '20'], check_status=True, hostname='smithi042') 2024-08-30T08:52:03.809 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'b0895c1a-66ac-11ef-bcd4-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'debug_mds_balancer', '20'], check_status=True, hostname='smithi042') 2024-08-30T08:52:03.809 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'b0895c1a-66ac-11ef-bcd4-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'debug_ms', '2'], check_status=True, hostname='smithi042') 2024-08-30T08:52:03.809 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'b0895c1a-66ac-11ef-bcd4-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'mds_bal_fragment_size_max', '10000'], check_status=True, hostname='smithi042') 2024-08-30T08:52:03.809 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'b0895c1a-66ac-11ef-bcd4-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'mds_bal_merge_size', '5'], check_status=True, hostname='smithi042') 2024-08-30T08:52:03.809 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'b0895c1a-66ac-11ef-bcd4-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'mds_bal_split_bits', '3'], check_status=True, hostname='smithi042') 2024-08-30T08:52:04.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:04.531+0000 7fa4ef943640 1 -- 172.21.15.42:0/3795500303 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa4e810b490 msgr2=0x7fa4e810d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:52:04.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:04.531+0000 7fa4ef943640 1 --2- 172.21.15.42:0/3795500303 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa4e810b490 0x7fa4e810d930 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fa4e0009ca0 tx=0x7fa4e002a8c0 comp rx=0 tx=0).stop 2024-08-30T08:52:04.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:04.531+0000 7fa4ef943640 1 -- 172.21.15.42:0/3795500303 shutdown_connections 2024-08-30T08:52:04.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:04.531+0000 7fa4ef943640 1 --2- 172.21.15.42:0/3795500303 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa4e8075a90 0x7fa4e8075ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:52:04.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:04.531+0000 7fa4ef943640 1 --2- 172.21.15.42:0/3795500303 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa4e810b490 0x7fa4e810d930 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:52:04.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:04.531+0000 7fa4ef943640 1 --2- 172.21.15.42:0/3795500303 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa4e8076e80 0x7fa4e8075550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:52:04.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:04.531+0000 7fa4ef943640 1 -- 172.21.15.42:0/3795500303 >> 172.21.15.42:0/3795500303 conn(0x7fa4e80fee60 msgr2=0x7fa4e8101310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:52:04.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:04.531+0000 7fa4ef943640 1 -- 172.21.15.42:0/3795500303 shutdown_connections 2024-08-30T08:52:04.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:04.531+0000 7fa4ef943640 1 -- 172.21.15.42:0/3795500303 wait complete. 2024-08-30T08:52:04.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:04.532+0000 7fa4ef943640 1 Processor -- start 2024-08-30T08:52:04.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:04.532+0000 7fa4ef943640 1 -- start start 2024-08-30T08:52:04.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:04.532+0000 7fa4ef943640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa4e8075a90 0x7fa4e819bf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:52:04.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:04.532+0000 7fa4ef943640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa4e8076e80 0x7fa4e819c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:52:04.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:04.532+0000 7fa4ef943640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa4e810b490 0x7fa4e819d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:52:04.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:04.532+0000 7fa4ef943640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fa4e8111330 con 0x7fa4e8076e80 2024-08-30T08:52:04.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:04.532+0000 7fa4ef943640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa4e81111b0 con 0x7fa4e8075a90 2024-08-30T08:52:04.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:04.532+0000 7fa4ef943640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fa4e81114b0 con 0x7fa4e810b490 2024-08-30T08:52:04.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:04.532+0000 7fa4ecede640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa4e8076e80 0x7fa4e819c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:52:04.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:04.532+0000 7fa4ecede640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa4e8076e80 0x7fa4e819c5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37768/0 (socket says 172.21.15.42:37768) 2024-08-30T08:52:04.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:04.532+0000 7fa4ecede640 1 -- 172.21.15.42:0/2780601908 learned_addr learned my addr 172.21.15.42:0/2780601908 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T08:52:04.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:04.532+0000 7fa4ed6df640 1 --2- 172.21.15.42:0/2780601908 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa4e8075a90 0x7fa4e819bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:52:04.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:04.533+0000 7fa4edee0640 1 --2- 172.21.15.42:0/2780601908 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa4e810b490 0x7fa4e819d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:52:04.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:04.533+0000 7fa4ecede640 1 -- 172.21.15.42:0/2780601908 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa4e810b490 msgr2=0x7fa4e819d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:52:04.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:04.533+0000 7fa4ecede640 1 --2- 172.21.15.42:0/2780601908 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa4e810b490 0x7fa4e819d140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:52:04.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:04.533+0000 7fa4ecede640 1 -- 172.21.15.42:0/2780601908 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa4e8075a90 msgr2=0x7fa4e819bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:52:04.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:04.533+0000 7fa4ecede640 1 --2- 172.21.15.42:0/2780601908 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa4e8075a90 0x7fa4e819bf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:52:04.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:04.533+0000 7fa4ecede640 1 -- 172.21.15.42:0/2780601908 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4e819d960 con 0x7fa4e8076e80 2024-08-30T08:52:04.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:04.533+0000 7fa4ed6df640 1 --2- 172.21.15.42:0/2780601908 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa4e8075a90 0x7fa4e819bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-30T08:52:04.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:04.533+0000 7fa4ecede640 1 --2- 172.21.15.42:0/2780601908 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa4e8076e80 0x7fa4e819c5a0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fa4d800d770 tx=0x7fa4d800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:52:04.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:04.533+0000 7fa4de7fc640 1 -- 172.21.15.42:0/2780601908 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4d8017070 con 0x7fa4e8076e80 2024-08-30T08:52:04.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:04.533+0000 7fa4de7fc640 1 -- 172.21.15.42:0/2780601908 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa4d80054d0 con 0x7fa4e8076e80 2024-08-30T08:52:04.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:04.533+0000 7fa4de7fc640 1 -- 172.21.15.42:0/2780601908 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4d8004c30 con 0x7fa4e8076e80 2024-08-30T08:52:04.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:04.533+0000 7fa4ef943640 1 -- 172.21.15.42:0/2780601908 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4e819dc50 con 0x7fa4e8076e80 2024-08-30T08:52:04.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:04.533+0000 7fa4ef943640 1 -- 172.21.15.42:0/2780601908 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4e81a4350 con 0x7fa4e8076e80 2024-08-30T08:52:04.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:04.534+0000 7fa4ef943640 1 -- 172.21.15.42:0/2780601908 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4e8109b90 con 0x7fa4e8076e80 2024-08-30T08:52:04.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:04.536+0000 7fa4de7fc640 1 -- 172.21.15.42:0/2780601908 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 15) ==== 152521+0+0 (secure 0 0 0) 0x7fa4d8004dd0 con 0x7fa4e8076e80 2024-08-30T08:52:04.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:04.537+0000 7fa4de7fc640 1 --2- 172.21.15.42:0/2780601908 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fa4d40b4b20 0x7fa4d40b7030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:52:04.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:04.537+0000 7fa4ed6df640 1 --2- 172.21.15.42:0/2780601908 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fa4d40b4b20 0x7fa4d40b7030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:52:04.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:04.537+0000 7fa4de7fc640 1 -- 172.21.15.42:0/2780601908 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fa4d80e3350 con 0x7fa4e8076e80 2024-08-30T08:52:04.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:04.537+0000 7fa4ed6df640 1 --2- 172.21.15.42:0/2780601908 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fa4d40b4b20 0x7fa4d40b7030 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fa4e4009fc0 tx=0x7fa4e4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:52:04.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:04.540+0000 7fa4de7fc640 1 -- 172.21.15.42:0/2780601908 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7fa4d80ac130 con 0x7fa4e8076e80 2024-08-30T08:52:04.624 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:52:04 smithi042 ceph-mon[18652]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-30T08:52:04.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:52:04 smithi190 ceph-mon[22001]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-30T08:52:04.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:04.729+0000 7fa4ef943640 1 -- 172.21.15.42:0/2780601908 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command([{prefix=config set, name=mon_op_complaint_time}] v 0) -- 0x7fa4e802e430 con 0x7fa4e8076e80 2024-08-30T08:52:04.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:04.713+0000 7f35591bc640 1 -- 172.21.15.42:0/3822015023 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3554109240 msgr2=0x7f355410b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:52:04.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:04.714+0000 7f35591bc640 1 --2- 172.21.15.42:0/3822015023 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3554109240 0x7f355410b750 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f354c009ca0 tx=0x7f354c02a8c0 comp rx=0 tx=0).stop 2024-08-30T08:52:04.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:04.714+0000 7f35591bc640 1 -- 172.21.15.42:0/3822015023 shutdown_connections 2024-08-30T08:52:04.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:04.714+0000 7f35591bc640 1 --2- 172.21.15.42:0/3822015023 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f35541066a0 0x7f3554108b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:52:04.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:04.714+0000 7f35591bc640 1 --2- 172.21.15.42:0/3822015023 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3554109240 0x7f355410b750 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:52:04.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:04.714+0000 7f35591bc640 1 --2- 172.21.15.42:0/3822015023 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f355406c0e0 0x7f3554106160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:52:04.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:04.714+0000 7f35591bc640 1 -- 172.21.15.42:0/3822015023 >> 172.21.15.42:0/3822015023 conn(0x7f35540fdd80 msgr2=0x7f3554100230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:52:04.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:04.714+0000 7f35591bc640 1 -- 172.21.15.42:0/3822015023 shutdown_connections 2024-08-30T08:52:04.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:04.714+0000 7f35591bc640 1 -- 172.21.15.42:0/3822015023 wait complete. 2024-08-30T08:52:04.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:04.714+0000 7f35591bc640 1 Processor -- start 2024-08-30T08:52:04.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:04.715+0000 7f35591bc640 1 -- start start 2024-08-30T08:52:04.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:04.715+0000 7f35591bc640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f355406c0e0 0x7f3554103790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:52:04.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:04.715+0000 7f35591bc640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f35541066a0 0x7f3554102060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:52:04.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:04.715+0000 7f35591bc640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3554109240 0x7f35541025a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:52:04.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:04.715+0000 7f35591bc640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f355410f130 con 0x7f355406c0e0 2024-08-30T08:52:04.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:04.715+0000 7f35591bc640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f355410efb0 con 0x7f3554109240 2024-08-30T08:52:04.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:04.715+0000 7f35591bc640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f355410f2b0 con 0x7f35541066a0 2024-08-30T08:52:04.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:04.715+0000 7f3552d9d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f355406c0e0 0x7f3554103790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:52:04.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:04.715+0000 7f3552d9d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f355406c0e0 0x7f3554103790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37786/0 (socket says 172.21.15.42:37786) 2024-08-30T08:52:04.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:04.715+0000 7f3552d9d640 1 -- 172.21.15.42:0/1334352639 learned_addr learned my addr 172.21.15.42:0/1334352639 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T08:52:04.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:04.715+0000 7f355359e640 1 --2- 172.21.15.42:0/1334352639 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3554109240 0x7f35541025a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:52:04.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:04.715+0000 7f355259c640 1 --2- 172.21.15.42:0/1334352639 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f35541066a0 0x7f3554102060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:52:04.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:04.715+0000 7f3552d9d640 1 -- 172.21.15.42:0/1334352639 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f35541066a0 msgr2=0x7f3554102060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:52:04.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:04.715+0000 7f3552d9d640 1 --2- 172.21.15.42:0/1334352639 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f35541066a0 0x7f3554102060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:52:04.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:04.715+0000 7f3552d9d640 1 -- 172.21.15.42:0/1334352639 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3554109240 msgr2=0x7f35541025a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:52:04.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:04.715+0000 7f3552d9d640 1 --2- 172.21.15.42:0/1334352639 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3554109240 0x7f35541025a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:52:04.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:04.715+0000 7f3552d9d640 1 -- 172.21.15.42:0/1334352639 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3554102c10 con 0x7f355406c0e0 2024-08-30T08:52:04.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:04.716+0000 7f3552d9d640 1 --2- 172.21.15.42:0/1334352639 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f355406c0e0 0x7f3554103790 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f354400d950 tx=0x7f354400dd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:52:04.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:04.716+0000 7f353bfff640 1 -- 172.21.15.42:0/1334352639 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3544017070 con 0x7f355406c0e0 2024-08-30T08:52:04.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:04.716+0000 7f353bfff640 1 -- 172.21.15.42:0/1334352639 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f354400de80 con 0x7f355406c0e0 2024-08-30T08:52:04.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:04.716+0000 7f353bfff640 1 -- 172.21.15.42:0/1334352639 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f354400f620 con 0x7f355406c0e0 2024-08-30T08:52:04.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:04.716+0000 7f35591bc640 1 -- 172.21.15.42:0/1334352639 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35541a3d10 con 0x7f355406c0e0 2024-08-30T08:52:04.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:04.716+0000 7f35591bc640 1 -- 172.21.15.42:0/1334352639 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35541a40b0 con 0x7f355406c0e0 2024-08-30T08:52:04.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:04.717+0000 7f35591bc640 1 -- 172.21.15.42:0/1334352639 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35540ff8a0 con 0x7f355406c0e0 2024-08-30T08:52:04.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:04.720+0000 7f353bfff640 1 -- 172.21.15.42:0/1334352639 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 15) ==== 152521+0+0 (secure 0 0 0) 0x7f3544009c30 con 0x7f355406c0e0 2024-08-30T08:52:04.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:04.720+0000 7f353bfff640 1 --2- 172.21.15.42:0/1334352639 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f353c0b4aa0 0x7f353c0b6fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:52:04.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:04.720+0000 7f355259c640 1 --2- 172.21.15.42:0/1334352639 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f353c0b4aa0 0x7f353c0b6fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:52:04.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:04.720+0000 7f353bfff640 1 -- 172.21.15.42:0/1334352639 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f35440e2f80 con 0x7f355406c0e0 2024-08-30T08:52:04.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:04.721+0000 7f355259c640 1 --2- 172.21.15.42:0/1334352639 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f353c0b4aa0 0x7f353c0b6fb0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f354800abe0 tx=0x7f354800a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:52:04.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:04.723+0000 7f353bfff640 1 -- 172.21.15.42:0/1334352639 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f35440abd80 con 0x7f355406c0e0 2024-08-30T08:52:04.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:04.733+0000 7fa4de7fc640 1 -- 172.21.15.42:0/2780601908 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_op_complaint_time}]=0 v30)=0 v30) ==== 121+0+0 (secure 0 0 0) 0x7fa4d80aba30 con 0x7fa4e8076e80 2024-08-30T08:52:04.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:04.735+0000 7fa4ef943640 1 -- 172.21.15.42:0/2780601908 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fa4d40b4b20 msgr2=0x7fa4d40b7030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:52:04.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:04.735+0000 7fa4ef943640 1 --2- 172.21.15.42:0/2780601908 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fa4d40b4b20 0x7fa4d40b7030 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fa4e4009fc0 tx=0x7fa4e4005f90 comp rx=0 tx=0).stop 2024-08-30T08:52:04.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:04.735+0000 7fa4ef943640 1 -- 172.21.15.42:0/2780601908 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa4e8076e80 msgr2=0x7fa4e819c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:52:04.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:04.735+0000 7fa4ef943640 1 --2- 172.21.15.42:0/2780601908 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa4e8076e80 0x7fa4e819c5a0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fa4d800d770 tx=0x7fa4d800dc30 comp rx=0 tx=0).stop 2024-08-30T08:52:04.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:04.736+0000 7fa4ef943640 1 -- 172.21.15.42:0/2780601908 shutdown_connections 2024-08-30T08:52:04.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:04.736+0000 7fa4ef943640 1 --2- 172.21.15.42:0/2780601908 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fa4d40b4b20 0x7fa4d40b7030 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:52:04.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:04.736+0000 7fa4ef943640 1 --2- 172.21.15.42:0/2780601908 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa4e810b490 0x7fa4e819d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:52:04.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:04.736+0000 7fa4ef943640 1 --2- 172.21.15.42:0/2780601908 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa4e8076e80 0x7fa4e819c5a0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:52:04.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:04.736+0000 7fa4ef943640 1 --2- 172.21.15.42:0/2780601908 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa4e8075a90 0x7fa4e819bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:52:04.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:04.736+0000 7fa4ef943640 1 -- 172.21.15.42:0/2780601908 >> 172.21.15.42:0/2780601908 conn(0x7fa4e80fee60 msgr2=0x7fa4e81012f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:52:04.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:04.736+0000 7fa4ef943640 1 -- 172.21.15.42:0/2780601908 shutdown_connections 2024-08-30T08:52:04.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:04.736+0000 7fa4ef943640 1 -- 172.21.15.42:0/2780601908 wait complete. 2024-08-30T08:52:04.828 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:52:04 smithi167 ceph-mon[22063]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-30T08:52:04.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:04.921+0000 7f35591bc640 1 -- 172.21.15.42:0/1334352639 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_pool_no_app}] v 0) -- 0x7f3554063dd0 con 0x7f355406c0e0 2024-08-30T08:52:04.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:04.926+0000 7f353bfff640 1 -- 172.21.15.42:0/1334352639 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_pool_no_app}]=0 v31)=0 v31) ==== 125+0+0 (secure 0 0 0) 0x7f35440ab680 con 0x7f355406c0e0 2024-08-30T08:52:04.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:04.928+0000 7f35591bc640 1 -- 172.21.15.42:0/1334352639 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f353c0b4aa0 msgr2=0x7f353c0b6fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:52:04.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:04.928+0000 7f35591bc640 1 --2- 172.21.15.42:0/1334352639 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f353c0b4aa0 0x7f353c0b6fb0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f354800abe0 tx=0x7f354800a2d0 comp rx=0 tx=0).stop 2024-08-30T08:52:04.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:04.928+0000 7f35591bc640 1 -- 172.21.15.42:0/1334352639 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f355406c0e0 msgr2=0x7f3554103790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:52:04.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:04.928+0000 7f35591bc640 1 --2- 172.21.15.42:0/1334352639 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f355406c0e0 0x7f3554103790 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f354400d950 tx=0x7f354400dd50 comp rx=0 tx=0).stop 2024-08-30T08:52:04.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:04.928+0000 7f35591bc640 1 -- 172.21.15.42:0/1334352639 shutdown_connections 2024-08-30T08:52:04.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:04.928+0000 7f35591bc640 1 --2- 172.21.15.42:0/1334352639 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f353c0b4aa0 0x7f353c0b6fb0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:52:04.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:04.928+0000 7f35591bc640 1 --2- 172.21.15.42:0/1334352639 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f35541066a0 0x7f3554102060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:52:04.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:04.928+0000 7f35591bc640 1 --2- 172.21.15.42:0/1334352639 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3554109240 0x7f35541025a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:52:04.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:04.928+0000 7f35591bc640 1 --2- 172.21.15.42:0/1334352639 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f355406c0e0 0x7f3554103790 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:52:04.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:04.928+0000 7f35591bc640 1 -- 172.21.15.42:0/1334352639 >> 172.21.15.42:0/1334352639 conn(0x7f35540fdd80 msgr2=0x7f3554104b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:52:04.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:04.928+0000 7f35591bc640 1 -- 172.21.15.42:0/1334352639 shutdown_connections 2024-08-30T08:52:04.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:04.928+0000 7f35591bc640 1 -- 172.21.15.42:0/1334352639 wait complete. 2024-08-30T08:52:05.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:52:05 smithi042 ceph-mon[18652]: from='client.? 172.21.15.42:0/2780601908' entity='client.admin' 2024-08-30T08:52:05.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:52:05 smithi042 ceph-mon[18652]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-30T08:52:05.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:52:05 smithi042 ceph-mon[18652]: from='client.? 172.21.15.42:0/1334352639' entity='client.admin' 2024-08-30T08:52:06.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:52:05 smithi167 ceph-mon[22063]: from='client.? 172.21.15.42:0/2780601908' entity='client.admin' 2024-08-30T08:52:06.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:52:05 smithi167 ceph-mon[22063]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-30T08:52:06.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:52:05 smithi167 ceph-mon[22063]: from='client.? 172.21.15.42:0/1334352639' entity='client.admin' 2024-08-30T08:52:06.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:52:05 smithi190 ceph-mon[22001]: from='client.? 172.21.15.42:0/2780601908' entity='client.admin' 2024-08-30T08:52:06.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:52:05 smithi190 ceph-mon[22001]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-30T08:52:06.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:52:05 smithi190 ceph-mon[22001]: from='client.? 172.21.15.42:0/1334352639' entity='client.admin' 2024-08-30T08:52:06.762 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'b0895c1a-66ac-11ef-bcd4-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'mds_bal_split_size', '100'], check_status=True, hostname='smithi042') 2024-08-30T08:52:06.762 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'b0895c1a-66ac-11ef-bcd4-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'mds_debug_frag', 'True'], check_status=True, hostname='smithi042') 2024-08-30T08:52:06.763 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'b0895c1a-66ac-11ef-bcd4-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'mds_debug_scatterstat', 'True'], check_status=True, hostname='smithi042') 2024-08-30T08:52:06.763 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'b0895c1a-66ac-11ef-bcd4-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'mds_op_complaint_time', '180'], check_status=True, hostname='smithi042') 2024-08-30T08:52:06.763 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'b0895c1a-66ac-11ef-bcd4-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'mds_verify_scatter', 'True'], check_status=True, hostname='smithi042') 2024-08-30T08:52:06.763 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'b0895c1a-66ac-11ef-bcd4-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'osd_op_complaint_time', '180'], check_status=True, hostname='smithi042') 2024-08-30T08:52:06.763 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'b0895c1a-66ac-11ef-bcd4-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'rados_mon_op_timeout', '900'], check_status=True, hostname='smithi042') 2024-08-30T08:52:06.763 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'b0895c1a-66ac-11ef-bcd4-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'rados_osd_op_timeout', '900'], check_status=True, hostname='smithi042') 2024-08-30T08:52:06.763 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'b0895c1a-66ac-11ef-bcd4-c7b262605968', '--', 'ceph', 'config', 'set', 'mon', 'mon_op_complaint_time', '120'], check_status=True, hostname='smithi042') 2024-08-30T08:52:07.664 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'b0895c1a-66ac-11ef-bcd4-c7b262605968', '--', 'ceph', 'config', 'set', 'mon', 'mon_warn_on_pool_no_app', 'False'], check_status=True, hostname='smithi042') 2024-08-30T08:52:07.664 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'b0895c1a-66ac-11ef-bcd4-c7b262605968', '--', 'ceph', 'config', 'set', 'osd', 'osd_op_complaint_time', '180'], check_status=True, hostname='smithi042') 2024-08-30T08:52:07.665 INFO:tasks.cephadm:Deploying OSDs... 2024-08-30T08:52:07.665 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-08-30T08:52:07.665 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/scratch_devs of=/dev/stdout 2024-08-30T08:52:07.682 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-08-30T08:52:07.683 DEBUG:teuthology.orchestra.run.smithi042:> stat /dev/vg_nvme/lv_1 2024-08-30T08:52:07.739 INFO:teuthology.orchestra.run.smithi042.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-08-30T08:52:07.739 INFO:teuthology.orchestra.run.smithi042.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-30T08:52:07.739 INFO:teuthology.orchestra.run.smithi042.stdout:Device: 5h/5d Inode: 664 Links: 1 2024-08-30T08:52:07.739 INFO:teuthology.orchestra.run.smithi042.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-30T08:52:07.739 INFO:teuthology.orchestra.run.smithi042.stdout:Context: system_u:object_r:device_t:s0 2024-08-30T08:52:07.740 INFO:teuthology.orchestra.run.smithi042.stdout:Access: 2024-08-30 08:50:59.618246521 +0000 2024-08-30T08:52:07.740 INFO:teuthology.orchestra.run.smithi042.stdout:Modify: 2024-08-30 08:46:07.049929420 +0000 2024-08-30T08:52:07.740 INFO:teuthology.orchestra.run.smithi042.stdout:Change: 2024-08-30 08:46:07.049929420 +0000 2024-08-30T08:52:07.740 INFO:teuthology.orchestra.run.smithi042.stdout: Birth: 2024-08-30 08:46:07.049929420 +0000 2024-08-30T08:52:07.740 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-08-30T08:52:07.807 INFO:teuthology.orchestra.run.smithi042.stderr:1+0 records in 2024-08-30T08:52:07.807 INFO:teuthology.orchestra.run.smithi042.stderr:1+0 records out 2024-08-30T08:52:07.807 INFO:teuthology.orchestra.run.smithi042.stderr:512 bytes copied, 0.000115472 s, 4.4 MB/s 2024-08-30T08:52:07.809 DEBUG:teuthology.orchestra.run.smithi042:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-08-30T08:52:07.866 DEBUG:teuthology.orchestra.run.smithi042:> stat /dev/vg_nvme/lv_2 2024-08-30T08:52:07.923 INFO:teuthology.orchestra.run.smithi042.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-08-30T08:52:07.923 INFO:teuthology.orchestra.run.smithi042.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-30T08:52:07.923 INFO:teuthology.orchestra.run.smithi042.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-08-30T08:52:07.923 INFO:teuthology.orchestra.run.smithi042.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-30T08:52:07.923 INFO:teuthology.orchestra.run.smithi042.stdout:Context: system_u:object_r:device_t:s0 2024-08-30T08:52:07.924 INFO:teuthology.orchestra.run.smithi042.stdout:Access: 2024-08-30 08:50:59.651246234 +0000 2024-08-30T08:52:07.924 INFO:teuthology.orchestra.run.smithi042.stdout:Modify: 2024-08-30 08:46:07.371929122 +0000 2024-08-30T08:52:07.924 INFO:teuthology.orchestra.run.smithi042.stdout:Change: 2024-08-30 08:46:07.371929122 +0000 2024-08-30T08:52:07.924 INFO:teuthology.orchestra.run.smithi042.stdout: Birth: 2024-08-30 08:46:07.371929122 +0000 2024-08-30T08:52:07.924 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-08-30T08:52:07.988 INFO:teuthology.orchestra.run.smithi042.stderr:1+0 records in 2024-08-30T08:52:07.988 INFO:teuthology.orchestra.run.smithi042.stderr:1+0 records out 2024-08-30T08:52:07.988 INFO:teuthology.orchestra.run.smithi042.stderr:512 bytes copied, 0.000113279 s, 4.5 MB/s 2024-08-30T08:52:07.989 DEBUG:teuthology.orchestra.run.smithi042:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-08-30T08:52:08.046 DEBUG:teuthology.orchestra.run.smithi042:> stat /dev/vg_nvme/lv_3 2024-08-30T08:52:08.102 INFO:teuthology.orchestra.run.smithi042.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-08-30T08:52:08.102 INFO:teuthology.orchestra.run.smithi042.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-30T08:52:08.102 INFO:teuthology.orchestra.run.smithi042.stdout:Device: 5h/5d Inode: 702 Links: 1 2024-08-30T08:52:08.102 INFO:teuthology.orchestra.run.smithi042.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-30T08:52:08.102 INFO:teuthology.orchestra.run.smithi042.stdout:Context: system_u:object_r:device_t:s0 2024-08-30T08:52:08.102 INFO:teuthology.orchestra.run.smithi042.stdout:Access: 2024-08-30 08:50:59.684245947 +0000 2024-08-30T08:52:08.102 INFO:teuthology.orchestra.run.smithi042.stdout:Modify: 2024-08-30 08:46:07.691928826 +0000 2024-08-30T08:52:08.102 INFO:teuthology.orchestra.run.smithi042.stdout:Change: 2024-08-30 08:46:07.691928826 +0000 2024-08-30T08:52:08.102 INFO:teuthology.orchestra.run.smithi042.stdout: Birth: 2024-08-30 08:46:07.691928826 +0000 2024-08-30T08:52:08.103 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-08-30T08:52:08.163 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:52:07 smithi042 ceph-mon[18652]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-30T08:52:08.163 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:52:07 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:52:08.163 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:52:07 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:52:08.164 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:52:07 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-30T08:52:08.164 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:52:07 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:52:08.164 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:52:07 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-30T08:52:08.164 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:52:07 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:52:08.165 INFO:teuthology.orchestra.run.smithi042.stderr:1+0 records in 2024-08-30T08:52:08.166 INFO:teuthology.orchestra.run.smithi042.stderr:1+0 records out 2024-08-30T08:52:08.166 INFO:teuthology.orchestra.run.smithi042.stderr:512 bytes copied, 0.000102971 s, 5.0 MB/s 2024-08-30T08:52:08.167 DEBUG:teuthology.orchestra.run.smithi042:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-08-30T08:52:08.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:52:07 smithi190 ceph-mon[22001]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-30T08:52:08.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:52:07 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:52:08.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:52:07 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:52:08.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:52:07 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-30T08:52:08.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:52:07 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:52:08.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:52:07 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-30T08:52:08.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:52:07 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:52:08.223 DEBUG:teuthology.orchestra.run.smithi042:> stat /dev/vg_nvme/lv_4 2024-08-30T08:52:08.278 INFO:teuthology.orchestra.run.smithi042.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-08-30T08:52:08.278 INFO:teuthology.orchestra.run.smithi042.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-30T08:52:08.279 INFO:teuthology.orchestra.run.smithi042.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-08-30T08:52:08.279 INFO:teuthology.orchestra.run.smithi042.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-30T08:52:08.279 INFO:teuthology.orchestra.run.smithi042.stdout:Context: system_u:object_r:device_t:s0 2024-08-30T08:52:08.279 INFO:teuthology.orchestra.run.smithi042.stdout:Access: 2024-08-30 08:50:59.716245669 +0000 2024-08-30T08:52:08.279 INFO:teuthology.orchestra.run.smithi042.stdout:Modify: 2024-08-30 08:46:08.025928517 +0000 2024-08-30T08:52:08.279 INFO:teuthology.orchestra.run.smithi042.stdout:Change: 2024-08-30 08:46:08.025928517 +0000 2024-08-30T08:52:08.279 INFO:teuthology.orchestra.run.smithi042.stdout: Birth: 2024-08-30 08:46:08.025928517 +0000 2024-08-30T08:52:08.279 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-08-30T08:52:08.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:52:07 smithi167 ceph-mon[22063]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-30T08:52:08.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:52:07 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:52:08.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:52:07 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:52:08.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:52:07 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-30T08:52:08.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:52:07 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:52:08.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:52:07 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-30T08:52:08.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:52:07 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:52:08.341 INFO:teuthology.orchestra.run.smithi042.stderr:1+0 records in 2024-08-30T08:52:08.342 INFO:teuthology.orchestra.run.smithi042.stderr:1+0 records out 2024-08-30T08:52:08.342 INFO:teuthology.orchestra.run.smithi042.stderr:512 bytes copied, 0.000108234 s, 4.7 MB/s 2024-08-30T08:52:08.343 DEBUG:teuthology.orchestra.run.smithi042:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-08-30T08:52:08.399 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-08-30T08:52:08.399 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/scratch_devs of=/dev/stdout 2024-08-30T08:52:08.419 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-08-30T08:52:08.420 DEBUG:teuthology.orchestra.run.smithi167:> stat /dev/vg_nvme/lv_1 2024-08-30T08:52:08.478 INFO:teuthology.orchestra.run.smithi167.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-08-30T08:52:08.478 INFO:teuthology.orchestra.run.smithi167.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-30T08:52:08.478 INFO:teuthology.orchestra.run.smithi167.stdout:Device: 5h/5d Inode: 663 Links: 1 2024-08-30T08:52:08.478 INFO:teuthology.orchestra.run.smithi167.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-30T08:52:08.478 INFO:teuthology.orchestra.run.smithi167.stdout:Context: system_u:object_r:device_t:s0 2024-08-30T08:52:08.478 INFO:teuthology.orchestra.run.smithi167.stdout:Access: 2024-08-30 08:51:48.316596136 +0000 2024-08-30T08:52:08.479 INFO:teuthology.orchestra.run.smithi167.stdout:Modify: 2024-08-30 08:46:26.042780276 +0000 2024-08-30T08:52:08.479 INFO:teuthology.orchestra.run.smithi167.stdout:Change: 2024-08-30 08:46:26.042780276 +0000 2024-08-30T08:52:08.479 INFO:teuthology.orchestra.run.smithi167.stdout: Birth: 2024-08-30 08:46:26.042780276 +0000 2024-08-30T08:52:08.479 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-08-30T08:52:08.549 INFO:teuthology.orchestra.run.smithi167.stderr:1+0 records in 2024-08-30T08:52:08.550 INFO:teuthology.orchestra.run.smithi167.stderr:1+0 records out 2024-08-30T08:52:08.550 INFO:teuthology.orchestra.run.smithi167.stderr:512 bytes copied, 0.000289988 s, 1.8 MB/s 2024-08-30T08:52:08.551 DEBUG:teuthology.orchestra.run.smithi167:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-08-30T08:52:08.610 DEBUG:teuthology.orchestra.run.smithi167:> stat /dev/vg_nvme/lv_2 2024-08-30T08:52:08.668 INFO:teuthology.orchestra.run.smithi167.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-08-30T08:52:08.668 INFO:teuthology.orchestra.run.smithi167.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-30T08:52:08.668 INFO:teuthology.orchestra.run.smithi167.stdout:Device: 5h/5d Inode: 683 Links: 1 2024-08-30T08:52:08.668 INFO:teuthology.orchestra.run.smithi167.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-30T08:52:08.668 INFO:teuthology.orchestra.run.smithi167.stdout:Context: system_u:object_r:device_t:s0 2024-08-30T08:52:08.668 INFO:teuthology.orchestra.run.smithi167.stdout:Access: 2024-08-30 08:51:48.316596136 +0000 2024-08-30T08:52:08.669 INFO:teuthology.orchestra.run.smithi167.stdout:Modify: 2024-08-30 08:46:26.368774602 +0000 2024-08-30T08:52:08.669 INFO:teuthology.orchestra.run.smithi167.stdout:Change: 2024-08-30 08:46:26.368774602 +0000 2024-08-30T08:52:08.669 INFO:teuthology.orchestra.run.smithi167.stdout: Birth: 2024-08-30 08:46:26.368774602 +0000 2024-08-30T08:52:08.669 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-08-30T08:52:08.735 INFO:teuthology.orchestra.run.smithi167.stderr:1+0 records in 2024-08-30T08:52:08.735 INFO:teuthology.orchestra.run.smithi167.stderr:1+0 records out 2024-08-30T08:52:08.735 INFO:teuthology.orchestra.run.smithi167.stderr:512 bytes copied, 0.000232142 s, 2.2 MB/s 2024-08-30T08:52:08.736 DEBUG:teuthology.orchestra.run.smithi167:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-08-30T08:52:08.794 DEBUG:teuthology.orchestra.run.smithi167:> stat /dev/vg_nvme/lv_3 2024-08-30T08:52:08.852 INFO:teuthology.orchestra.run.smithi167.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-08-30T08:52:08.852 INFO:teuthology.orchestra.run.smithi167.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-30T08:52:08.852 INFO:teuthology.orchestra.run.smithi167.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-08-30T08:52:08.852 INFO:teuthology.orchestra.run.smithi167.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-30T08:52:08.852 INFO:teuthology.orchestra.run.smithi167.stdout:Context: system_u:object_r:device_t:s0 2024-08-30T08:52:08.852 INFO:teuthology.orchestra.run.smithi167.stdout:Access: 2024-08-30 08:51:48.317596119 +0000 2024-08-30T08:52:08.852 INFO:teuthology.orchestra.run.smithi167.stdout:Modify: 2024-08-30 08:46:26.729768317 +0000 2024-08-30T08:52:08.852 INFO:teuthology.orchestra.run.smithi167.stdout:Change: 2024-08-30 08:46:26.729768317 +0000 2024-08-30T08:52:08.852 INFO:teuthology.orchestra.run.smithi167.stdout: Birth: 2024-08-30 08:46:26.729768317 +0000 2024-08-30T08:52:08.853 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-08-30T08:52:08.918 INFO:teuthology.orchestra.run.smithi167.stderr:1+0 records in 2024-08-30T08:52:08.918 INFO:teuthology.orchestra.run.smithi167.stderr:1+0 records out 2024-08-30T08:52:08.918 INFO:teuthology.orchestra.run.smithi167.stderr:512 bytes copied, 0.000226239 s, 2.3 MB/s 2024-08-30T08:52:08.919 DEBUG:teuthology.orchestra.run.smithi167:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-08-30T08:52:08.976 DEBUG:teuthology.orchestra.run.smithi167:> stat /dev/vg_nvme/lv_4 2024-08-30T08:52:09.033 INFO:teuthology.orchestra.run.smithi167.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-08-30T08:52:09.033 INFO:teuthology.orchestra.run.smithi167.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-30T08:52:09.033 INFO:teuthology.orchestra.run.smithi167.stdout:Device: 5h/5d Inode: 721 Links: 1 2024-08-30T08:52:09.033 INFO:teuthology.orchestra.run.smithi167.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-30T08:52:09.033 INFO:teuthology.orchestra.run.smithi167.stdout:Context: system_u:object_r:device_t:s0 2024-08-30T08:52:09.033 INFO:teuthology.orchestra.run.smithi167.stdout:Access: 2024-08-30 08:51:48.317596119 +0000 2024-08-30T08:52:09.033 INFO:teuthology.orchestra.run.smithi167.stdout:Modify: 2024-08-30 08:46:27.048762764 +0000 2024-08-30T08:52:09.033 INFO:teuthology.orchestra.run.smithi167.stdout:Change: 2024-08-30 08:46:27.048762764 +0000 2024-08-30T08:52:09.034 INFO:teuthology.orchestra.run.smithi167.stdout: Birth: 2024-08-30 08:46:27.048762764 +0000 2024-08-30T08:52:09.034 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-08-30T08:52:09.100 INFO:teuthology.orchestra.run.smithi167.stderr:1+0 records in 2024-08-30T08:52:09.100 INFO:teuthology.orchestra.run.smithi167.stderr:1+0 records out 2024-08-30T08:52:09.100 INFO:teuthology.orchestra.run.smithi167.stderr:512 bytes copied, 0.000217221 s, 2.4 MB/s 2024-08-30T08:52:09.101 DEBUG:teuthology.orchestra.run.smithi167:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-08-30T08:52:09.159 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-08-30T08:52:09.159 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/scratch_devs of=/dev/stdout 2024-08-30T08:52:09.179 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-08-30T08:52:09.180 DEBUG:teuthology.orchestra.run.smithi190:> stat /dev/vg_nvme/lv_1 2024-08-30T08:52:09.238 INFO:teuthology.orchestra.run.smithi190.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-08-30T08:52:09.238 INFO:teuthology.orchestra.run.smithi190.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-30T08:52:09.238 INFO:teuthology.orchestra.run.smithi190.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-08-30T08:52:09.239 INFO:teuthology.orchestra.run.smithi190.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-30T08:52:09.239 INFO:teuthology.orchestra.run.smithi190.stdout:Context: system_u:object_r:device_t:s0 2024-08-30T08:52:09.239 INFO:teuthology.orchestra.run.smithi190.stdout:Access: 2024-08-30 08:51:45.855854773 +0000 2024-08-30T08:52:09.239 INFO:teuthology.orchestra.run.smithi190.stdout:Modify: 2024-08-30 08:46:22.219779023 +0000 2024-08-30T08:52:09.239 INFO:teuthology.orchestra.run.smithi190.stdout:Change: 2024-08-30 08:46:22.219779023 +0000 2024-08-30T08:52:09.239 INFO:teuthology.orchestra.run.smithi190.stdout: Birth: 2024-08-30 08:46:22.219779023 +0000 2024-08-30T08:52:09.239 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-08-30T08:52:09.308 INFO:teuthology.orchestra.run.smithi190.stderr:1+0 records in 2024-08-30T08:52:09.308 INFO:teuthology.orchestra.run.smithi190.stderr:1+0 records out 2024-08-30T08:52:09.308 INFO:teuthology.orchestra.run.smithi190.stderr:512 bytes copied, 0.000290002 s, 1.8 MB/s 2024-08-30T08:52:09.309 DEBUG:teuthology.orchestra.run.smithi190:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-08-30T08:52:09.368 DEBUG:teuthology.orchestra.run.smithi190:> stat /dev/vg_nvme/lv_2 2024-08-30T08:52:09.425 INFO:teuthology.orchestra.run.smithi190.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-08-30T08:52:09.425 INFO:teuthology.orchestra.run.smithi190.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-30T08:52:09.425 INFO:teuthology.orchestra.run.smithi190.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-08-30T08:52:09.425 INFO:teuthology.orchestra.run.smithi190.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-30T08:52:09.425 INFO:teuthology.orchestra.run.smithi190.stdout:Context: system_u:object_r:device_t:s0 2024-08-30T08:52:09.425 INFO:teuthology.orchestra.run.smithi190.stdout:Access: 2024-08-30 08:51:45.855854773 +0000 2024-08-30T08:52:09.426 INFO:teuthology.orchestra.run.smithi190.stdout:Modify: 2024-08-30 08:46:22.551766848 +0000 2024-08-30T08:52:09.426 INFO:teuthology.orchestra.run.smithi190.stdout:Change: 2024-08-30 08:46:22.551766848 +0000 2024-08-30T08:52:09.426 INFO:teuthology.orchestra.run.smithi190.stdout: Birth: 2024-08-30 08:46:22.551766848 +0000 2024-08-30T08:52:09.426 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-08-30T08:52:09.491 INFO:teuthology.orchestra.run.smithi190.stderr:1+0 records in 2024-08-30T08:52:09.491 INFO:teuthology.orchestra.run.smithi190.stderr:1+0 records out 2024-08-30T08:52:09.491 INFO:teuthology.orchestra.run.smithi190.stderr:512 bytes copied, 0.000230535 s, 2.2 MB/s 2024-08-30T08:52:09.492 DEBUG:teuthology.orchestra.run.smithi190:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-08-30T08:52:09.550 DEBUG:teuthology.orchestra.run.smithi190:> stat /dev/vg_nvme/lv_3 2024-08-30T08:52:09.606 INFO:teuthology.orchestra.run.smithi190.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-08-30T08:52:09.606 INFO:teuthology.orchestra.run.smithi190.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-30T08:52:09.606 INFO:teuthology.orchestra.run.smithi190.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-08-30T08:52:09.606 INFO:teuthology.orchestra.run.smithi190.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-30T08:52:09.606 INFO:teuthology.orchestra.run.smithi190.stdout:Context: system_u:object_r:device_t:s0 2024-08-30T08:52:09.606 INFO:teuthology.orchestra.run.smithi190.stdout:Access: 2024-08-30 08:51:45.855854773 +0000 2024-08-30T08:52:09.606 INFO:teuthology.orchestra.run.smithi190.stdout:Modify: 2024-08-30 08:46:22.867755261 +0000 2024-08-30T08:52:09.606 INFO:teuthology.orchestra.run.smithi190.stdout:Change: 2024-08-30 08:46:22.867755261 +0000 2024-08-30T08:52:09.606 INFO:teuthology.orchestra.run.smithi190.stdout: Birth: 2024-08-30 08:46:22.867755261 +0000 2024-08-30T08:52:09.606 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-08-30T08:52:09.670 INFO:teuthology.orchestra.run.smithi190.stderr:1+0 records in 2024-08-30T08:52:09.670 INFO:teuthology.orchestra.run.smithi190.stderr:1+0 records out 2024-08-30T08:52:09.671 INFO:teuthology.orchestra.run.smithi190.stderr:512 bytes copied, 0.000219663 s, 2.3 MB/s 2024-08-30T08:52:09.672 DEBUG:teuthology.orchestra.run.smithi190:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-08-30T08:52:09.729 DEBUG:teuthology.orchestra.run.smithi190:> stat /dev/vg_nvme/lv_4 2024-08-30T08:52:09.785 INFO:teuthology.orchestra.run.smithi190.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-08-30T08:52:09.785 INFO:teuthology.orchestra.run.smithi190.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-30T08:52:09.786 INFO:teuthology.orchestra.run.smithi190.stdout:Device: 5h/5d Inode: 721 Links: 1 2024-08-30T08:52:09.786 INFO:teuthology.orchestra.run.smithi190.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-30T08:52:09.786 INFO:teuthology.orchestra.run.smithi190.stdout:Context: system_u:object_r:device_t:s0 2024-08-30T08:52:09.786 INFO:teuthology.orchestra.run.smithi190.stdout:Access: 2024-08-30 08:51:45.856854744 +0000 2024-08-30T08:52:09.786 INFO:teuthology.orchestra.run.smithi190.stdout:Modify: 2024-08-30 08:46:23.179743819 +0000 2024-08-30T08:52:09.786 INFO:teuthology.orchestra.run.smithi190.stdout:Change: 2024-08-30 08:46:23.179743819 +0000 2024-08-30T08:52:09.786 INFO:teuthology.orchestra.run.smithi190.stdout: Birth: 2024-08-30 08:46:23.179743819 +0000 2024-08-30T08:52:09.786 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-08-30T08:52:09.850 INFO:teuthology.orchestra.run.smithi190.stderr:1+0 records in 2024-08-30T08:52:09.850 INFO:teuthology.orchestra.run.smithi190.stderr:1+0 records out 2024-08-30T08:52:09.850 INFO:teuthology.orchestra.run.smithi190.stderr:512 bytes copied, 0.000241668 s, 2.1 MB/s 2024-08-30T08:52:09.851 DEBUG:teuthology.orchestra.run.smithi190:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-08-30T08:52:09.908 INFO:tasks.cephadm:Deploying osd.0 on smithi042 with /dev/vg_nvme/lv_4... 2024-08-30T08:52:09.908 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-08-30T08:52:09.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:52:09 smithi190 ceph-mon[22001]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-30T08:52:10.167 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T08:52:10.287 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:52:09 smithi042 ceph-mon[18652]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-30T08:52:10.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:52:09 smithi167 ceph-mon[22063]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-30T08:52:12.163 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:52:11 smithi042 ceph-mon[18652]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-30T08:52:12.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:52:11 smithi190 ceph-mon[22001]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-30T08:52:12.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:52:11 smithi167 ceph-mon[22063]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-30T08:52:12.726 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-08-30T08:52:12.753 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph orch daemon add osd smithi042:vg_nvme/lv_4 2024-08-30T08:52:13.059 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T08:52:13.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:13.502+0000 7f79f0dea640 1 -- 172.21.15.42:0/3024292092 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f79ec1034a0 msgr2=0x7f79ec1038b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:52:13.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:13.502+0000 7f79f0dea640 1 --2- 172.21.15.42:0/3024292092 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f79ec1034a0 0x7f79ec1038b0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f79e0009900 tx=0x7f79e002a480 comp rx=0 tx=0).stop 2024-08-30T08:52:13.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:13.502+0000 7f79f0dea640 1 -- 172.21.15.42:0/3024292092 shutdown_connections 2024-08-30T08:52:13.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:13.502+0000 7f79f0dea640 1 --2- 172.21.15.42:0/3024292092 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f79ec103e40 0x7f79ec10a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:52:13.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:13.502+0000 7f79f0dea640 1 --2- 172.21.15.42:0/3024292092 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f79ec1034a0 0x7f79ec1038b0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:52:13.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:13.502+0000 7f79f0dea640 1 --2- 172.21.15.42:0/3024292092 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f79ec102410 0x7f79ec102880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:52:13.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:13.502+0000 7f79f0dea640 1 -- 172.21.15.42:0/3024292092 >> 172.21.15.42:0/3024292092 conn(0x7f79ec0fdd80 msgr2=0x7f79ec100230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:52:13.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:13.502+0000 7f79f0dea640 1 -- 172.21.15.42:0/3024292092 shutdown_connections 2024-08-30T08:52:13.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:13.502+0000 7f79f0dea640 1 -- 172.21.15.42:0/3024292092 wait complete. 2024-08-30T08:52:13.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:13.503+0000 7f79f0dea640 1 Processor -- start 2024-08-30T08:52:13.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:13.503+0000 7f79f0dea640 1 -- start start 2024-08-30T08:52:13.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:13.503+0000 7f79f0dea640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f79ec102410 0x7f79ec199e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:52:13.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:13.503+0000 7f79f0dea640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f79ec1034a0 0x7f79ec19a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:52:13.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:13.503+0000 7f79f0dea640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f79ec103e40 0x7f79ec19b040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:52:13.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:13.503+0000 7f79f0dea640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f79ec10e0b0 con 0x7f79ec1034a0 2024-08-30T08:52:13.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:13.503+0000 7f79f0dea640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f79ec10df30 con 0x7f79ec103e40 2024-08-30T08:52:13.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:13.503+0000 7f79f0dea640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f79ec10e230 con 0x7f79ec102410 2024-08-30T08:52:13.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:13.503+0000 7f79e9d9b640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f79ec1034a0 0x7f79ec19a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:52:13.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:13.503+0000 7f79e9d9b640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f79ec1034a0 0x7f79ec19a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:40244/0 (socket says 172.21.15.42:40244) 2024-08-30T08:52:13.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:13.503+0000 7f79e9d9b640 1 -- 172.21.15.42:0/640624045 learned_addr learned my addr 172.21.15.42:0/640624045 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T08:52:13.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:13.503+0000 7f79ead9d640 1 --2- 172.21.15.42:0/640624045 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f79ec103e40 0x7f79ec19b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:52:13.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:13.503+0000 7f79ea59c640 1 --2- 172.21.15.42:0/640624045 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f79ec102410 0x7f79ec199e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:52:13.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:13.504+0000 7f79e9d9b640 1 -- 172.21.15.42:0/640624045 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f79ec102410 msgr2=0x7f79ec199e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:52:13.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:13.504+0000 7f79e9d9b640 1 --2- 172.21.15.42:0/640624045 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f79ec102410 0x7f79ec199e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:52:13.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:13.504+0000 7f79e9d9b640 1 -- 172.21.15.42:0/640624045 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f79ec103e40 msgr2=0x7f79ec19b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:52:13.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:13.504+0000 7f79e9d9b640 1 --2- 172.21.15.42:0/640624045 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f79ec103e40 0x7f79ec19b040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:52:13.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:13.504+0000 7f79e9d9b640 1 -- 172.21.15.42:0/640624045 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79ec19b860 con 0x7f79ec1034a0 2024-08-30T08:52:13.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:13.504+0000 7f79e9d9b640 1 --2- 172.21.15.42:0/640624045 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f79ec1034a0 0x7f79ec19a4a0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f79e0006010 tx=0x7f79e00043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:52:13.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:13.504+0000 7f79db7fe640 1 -- 172.21.15.42:0/640624045 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79e003f030 con 0x7f79ec1034a0 2024-08-30T08:52:13.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:13.504+0000 7f79db7fe640 1 -- 172.21.15.42:0/640624045 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f79e0003cb0 con 0x7f79ec1034a0 2024-08-30T08:52:13.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:13.504+0000 7f79db7fe640 1 -- 172.21.15.42:0/640624045 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79e0035070 con 0x7f79ec1034a0 2024-08-30T08:52:13.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:13.504+0000 7f79f0dea640 1 -- 172.21.15.42:0/640624045 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79ec19baf0 con 0x7f79ec1034a0 2024-08-30T08:52:13.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:13.504+0000 7f79f0dea640 1 -- 172.21.15.42:0/640624045 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79ec1a2250 con 0x7f79ec1034a0 2024-08-30T08:52:13.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:13.505+0000 7f79f0dea640 1 -- 172.21.15.42:0/640624045 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79ec1a2420 con 0x7f79ec1034a0 2024-08-30T08:52:13.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:13.507+0000 7f79db7fe640 1 -- 172.21.15.42:0/640624045 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 15) ==== 152521+0+0 (secure 0 0 0) 0x7f79e0042030 con 0x7f79ec1034a0 2024-08-30T08:52:13.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:13.507+0000 7f79db7fe640 1 --2- 172.21.15.42:0/640624045 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f79d40b4b20 0x7f79d40b7030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:52:13.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:13.507+0000 7f79ea59c640 1 --2- 172.21.15.42:0/640624045 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f79d40b4b20 0x7f79d40b7030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:52:13.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:13.508+0000 7f79db7fe640 1 -- 172.21.15.42:0/640624045 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f79e01040c0 con 0x7f79ec1034a0 2024-08-30T08:52:13.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:13.508+0000 7f79ea59c640 1 --2- 172.21.15.42:0/640624045 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f79d40b4b20 0x7f79d40b7030 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f79dc009990 tx=0x7f79dc009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:52:13.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:13.511+0000 7f79db7fe640 1 -- 172.21.15.42:0/640624045 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f79e00ccec0 con 0x7f79ec1034a0 2024-08-30T08:52:13.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:13.707+0000 7f79f0dea640 1 -- 172.21.15.42:0/640624045 --> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi042:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f79ec0ffd80 con 0x7f79d40b4b20 2024-08-30T08:52:14.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:52:13 smithi190 ceph-mon[22001]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-30T08:52:14.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:52:13 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-30T08:52:14.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:52:13 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-30T08:52:14.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:52:13 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:52:14.195 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:52:13 smithi042 ceph-mon[18652]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-30T08:52:14.195 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:52:13 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-30T08:52:14.195 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:52:13 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-30T08:52:14.195 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:52:13 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:52:14.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:52:13 smithi167 ceph-mon[22063]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-30T08:52:14.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:52:13 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-30T08:52:14.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:52:13 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-30T08:52:14.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:52:13 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:52:15.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:52:14 smithi190 ceph-mon[22001]: from='client.14364 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi042:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-08-30T08:52:15.243 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:52:14 smithi042 ceph-mon[18652]: from='client.14364 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi042:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-08-30T08:52:15.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:52:14 smithi167 ceph-mon[22063]: from='client.14364 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi042:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-08-30T08:52:16.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:52:15 smithi190 ceph-mon[22001]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-30T08:52:16.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:52:15 smithi190 ceph-mon[22001]: from='client.? 172.21.15.42:0/708326293' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "029d62d7-a6c6-4905-8c76-6dffa9d57b92"} : dispatch 2024-08-30T08:52:16.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:52:15 smithi190 ceph-mon[22001]: from='client.? 172.21.15.42:0/708326293' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "029d62d7-a6c6-4905-8c76-6dffa9d57b92"}]': finished 2024-08-30T08:52:16.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:52:15 smithi190 ceph-mon[22001]: osdmap e5: 1 total, 0 up, 1 in 2024-08-30T08:52:16.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:52:15 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-08-30T08:52:16.186 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:52:15 smithi042 ceph-mon[18652]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-30T08:52:16.187 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:52:15 smithi042 ceph-mon[18652]: from='client.? 172.21.15.42:0/708326293' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "029d62d7-a6c6-4905-8c76-6dffa9d57b92"} : dispatch 2024-08-30T08:52:16.187 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:52:15 smithi042 ceph-mon[18652]: from='client.? 172.21.15.42:0/708326293' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "029d62d7-a6c6-4905-8c76-6dffa9d57b92"}]': finished 2024-08-30T08:52:16.187 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:52:15 smithi042 ceph-mon[18652]: osdmap e5: 1 total, 0 up, 1 in 2024-08-30T08:52:16.187 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:52:15 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-08-30T08:52:16.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:52:15 smithi167 ceph-mon[22063]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-30T08:52:16.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:52:15 smithi167 ceph-mon[22063]: from='client.? 172.21.15.42:0/708326293' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "029d62d7-a6c6-4905-8c76-6dffa9d57b92"} : dispatch 2024-08-30T08:52:16.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:52:15 smithi167 ceph-mon[22063]: from='client.? 172.21.15.42:0/708326293' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "029d62d7-a6c6-4905-8c76-6dffa9d57b92"}]': finished 2024-08-30T08:52:16.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:52:15 smithi167 ceph-mon[22063]: osdmap e5: 1 total, 0 up, 1 in 2024-08-30T08:52:16.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:52:15 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-08-30T08:52:17.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:52:16 smithi190 ceph-mon[22001]: from='client.? 172.21.15.42:0/1098365502' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-30T08:52:17.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:52:16 smithi167 ceph-mon[22063]: from='client.? 172.21.15.42:0/1098365502' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-30T08:52:17.382 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:52:16 smithi042 ceph-mon[18652]: from='client.? 172.21.15.42:0/1098365502' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-30T08:52:18.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:52:17 smithi190 ceph-mon[22001]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-30T08:52:18.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:52:17 smithi167 ceph-mon[22063]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-30T08:52:18.382 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:52:17 smithi042 ceph-mon[18652]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-30T08:52:20.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:52:19 smithi190 ceph-mon[22001]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-30T08:52:20.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:52:19 smithi167 ceph-mon[22063]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-30T08:52:20.382 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:52:19 smithi042 ceph-mon[18652]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-30T08:52:22.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:52:21 smithi190 ceph-mon[22001]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-30T08:52:22.196 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:52:21 smithi042 ceph-mon[18652]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-30T08:52:22.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:52:21 smithi167 ceph-mon[22063]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-30T08:52:24.070 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:52:23 smithi042 ceph-mon[18652]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-30T08:52:24.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:52:23 smithi190 ceph-mon[22001]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-30T08:52:24.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:52:23 smithi167 ceph-mon[22063]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-30T08:52:25.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:52:24 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.0"} : dispatch 2024-08-30T08:52:25.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:52:24 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:52:25.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:52:24 smithi042 ceph-mon[18652]: Deploying daemon osd.0 on smithi042 2024-08-30T08:52:25.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:52:24 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.0"} : dispatch 2024-08-30T08:52:25.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:52:24 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:52:25.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:52:24 smithi190 ceph-mon[22001]: Deploying daemon osd.0 on smithi042 2024-08-30T08:52:25.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:52:24 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.0"} : dispatch 2024-08-30T08:52:25.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:52:24 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:52:25.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:52:24 smithi167 ceph-mon[22063]: Deploying daemon osd.0 on smithi042 2024-08-30T08:52:25.993 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:52:25 smithi042 ceph-mon[18652]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-30T08:52:26.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:52:25 smithi190 ceph-mon[22001]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-30T08:52:26.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:52:25 smithi167 ceph-mon[22063]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-30T08:52:28.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:52:27 smithi190 ceph-mon[22001]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-30T08:52:28.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:52:27 smithi167 ceph-mon[22063]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-30T08:52:28.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:52:27 smithi042 ceph-mon[18652]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-30T08:52:30.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:52:29 smithi190 ceph-mon[22001]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-30T08:52:30.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:52:29 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-30T08:52:30.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:52:29 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:52:30.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:52:29 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:52:30.293 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:52:29 smithi042 ceph-mon[18652]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-30T08:52:30.293 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:52:29 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-30T08:52:30.294 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:52:29 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:52:30.294 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:52:29 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:52:30.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:52:29 smithi167 ceph-mon[22063]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-30T08:52:30.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:52:29 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-30T08:52:30.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:52:29 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:52:30.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:52:29 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:52:32.278 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:52:31 smithi042 ceph-mon[18652]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-30T08:52:32.278 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:52:31 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:52:32.278 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:52:31 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:52:32.278 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:52:31 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:52:32.278 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:52:31 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-30T08:52:32.278 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:52:31 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:52:32.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:52:31 smithi167 ceph-mon[22063]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-30T08:52:32.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:52:31 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:52:32.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:52:31 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:52:32.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:52:31 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:52:32.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:52:31 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-30T08:52:32.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:52:31 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:52:32.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:52:31 smithi190 ceph-mon[22001]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-30T08:52:32.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:52:31 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:52:32.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:52:31 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:52:32.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:52:31 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:52:32.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:52:31 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-30T08:52:32.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:52:31 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:52:33.370 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:33.368+0000 7f79db7fe640 1 -- 172.21.15.42:0/640624045 <== mgr.14150 v2:172.21.15.42:6800/3005140902 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f79ec0ffd80 con 0x7f79d40b4b20 2024-08-30T08:52:33.371 INFO:teuthology.orchestra.run.smithi042.stdout:Created osd(s) 0 on host 'smithi042' 2024-08-30T08:52:33.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:33.370+0000 7f79f0dea640 1 -- 172.21.15.42:0/640624045 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f79d40b4b20 msgr2=0x7f79d40b7030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:52:33.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:33.370+0000 7f79f0dea640 1 --2- 172.21.15.42:0/640624045 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f79d40b4b20 0x7f79d40b7030 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f79dc009990 tx=0x7f79dc009110 comp rx=0 tx=0).stop 2024-08-30T08:52:33.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:33.370+0000 7f79f0dea640 1 -- 172.21.15.42:0/640624045 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f79ec1034a0 msgr2=0x7f79ec19a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:52:33.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:33.370+0000 7f79f0dea640 1 --2- 172.21.15.42:0/640624045 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f79ec1034a0 0x7f79ec19a4a0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f79e0006010 tx=0x7f79e00043b0 comp rx=0 tx=0).stop 2024-08-30T08:52:33.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:33.371+0000 7f79f0dea640 1 -- 172.21.15.42:0/640624045 shutdown_connections 2024-08-30T08:52:33.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:33.371+0000 7f79f0dea640 1 --2- 172.21.15.42:0/640624045 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f79d40b4b20 0x7f79d40b7030 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:52:33.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:33.371+0000 7f79f0dea640 1 --2- 172.21.15.42:0/640624045 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f79ec103e40 0x7f79ec19b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:52:33.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:33.371+0000 7f79f0dea640 1 --2- 172.21.15.42:0/640624045 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f79ec1034a0 0x7f79ec19a4a0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:52:33.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:33.371+0000 7f79f0dea640 1 --2- 172.21.15.42:0/640624045 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f79ec102410 0x7f79ec199e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:52:33.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:33.371+0000 7f79f0dea640 1 -- 172.21.15.42:0/640624045 >> 172.21.15.42:0/640624045 conn(0x7f79ec0fdd80 msgr2=0x7f79ec0ff670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:52:33.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:33.371+0000 7f79f0dea640 1 -- 172.21.15.42:0/640624045 shutdown_connections 2024-08-30T08:52:33.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:52:33.371+0000 7f79f0dea640 1 -- 172.21.15.42:0/640624045 wait complete. 2024-08-30T08:52:33.964 DEBUG:teuthology.orchestra.run.smithi042:osd.0> sudo journalctl -f -n 0 -u ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968@osd.0.service 2024-08-30T08:52:33.966 INFO:tasks.cephadm:Deploying osd.1 on smithi167 with /dev/vg_nvme/lv_4... 2024-08-30T08:52:33.967 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-08-30T08:52:34.208 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:52:33 smithi042 ceph-mon[18652]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-30T08:52:34.208 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:52:33 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-30T08:52:34.208 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:52:33 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:52:34.208 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:52:33 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:52:34.218 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.b/config 2024-08-30T08:52:34.324 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:52:33 smithi167 ceph-mon[22063]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-30T08:52:34.324 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:52:33 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-30T08:52:34.324 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:52:33 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:52:34.324 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:52:33 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:52:34.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:52:33 smithi190 ceph-mon[22001]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-30T08:52:34.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:52:33 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-30T08:52:34.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:52:33 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:52:34.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:52:33 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:52:35.883 INFO:journalctl@ceph.osd.0.smithi042.stdout:Aug 30 08:52:35 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-0[35040]: 2024-08-30T08:52:35.535+0000 7f46fbf87640 -1 osd.0 0 log_to_monitors true 2024-08-30T08:52:36.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:52:35 smithi167 ceph-mon[22063]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-30T08:52:36.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:52:35 smithi167 ceph-mon[22063]: from='osd.0 [v2:172.21.15.42:6802/2617890537,v1:172.21.15.42:6803/2617890537]' entity='osd.0' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]} : dispatch 2024-08-30T08:52:36.197 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:52:35 smithi042 ceph-mon[18652]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-30T08:52:36.197 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:52:35 smithi042 ceph-mon[18652]: from='osd.0 [v2:172.21.15.42:6802/2617890537,v1:172.21.15.42:6803/2617890537]' entity='osd.0' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]} : dispatch 2024-08-30T08:52:36.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:52:35 smithi190 ceph-mon[22001]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-30T08:52:36.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:52:35 smithi190 ceph-mon[22001]: from='osd.0 [v2:172.21.15.42:6802/2617890537,v1:172.21.15.42:6803/2617890537]' entity='osd.0' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]} : dispatch 2024-08-30T08:52:36.524 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-08-30T08:52:36.554 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph orch daemon add osd smithi167:vg_nvme/lv_4 2024-08-30T08:52:36.807 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.b/config 2024-08-30T08:52:37.065 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:52:36 smithi042 ceph-mon[18652]: from='osd.0 [v2:172.21.15.42:6802/2617890537,v1:172.21.15.42:6803/2617890537]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-08-30T08:52:37.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:52:36 smithi042 ceph-mon[18652]: osdmap e6: 1 total, 0 up, 1 in 2024-08-30T08:52:37.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:52:36 smithi042 ceph-mon[18652]: from='osd.0 [v2:172.21.15.42:6802/2617890537,v1:172.21.15.42:6803/2617890537]' entity='osd.0' cmd={"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi042", "root=default"]} : dispatch 2024-08-30T08:52:37.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:52:36 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-08-30T08:52:37.066 INFO:journalctl@ceph.osd.0.smithi042.stdout:Aug 30 08:52:36 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-0[35040]: 2024-08-30T08:52:36.956+0000 7f46f7f09640 -1 osd.0 0 waiting for initial osdmap 2024-08-30T08:52:37.066 INFO:journalctl@ceph.osd.0.smithi042.stdout:Aug 30 08:52:36 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-0[35040]: 2024-08-30T08:52:36.961+0000 7f46f3d07640 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-30T08:52:37.157 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:52:36 smithi167 ceph-mon[22063]: from='osd.0 [v2:172.21.15.42:6802/2617890537,v1:172.21.15.42:6803/2617890537]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-08-30T08:52:37.157 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:52:36 smithi167 ceph-mon[22063]: osdmap e6: 1 total, 0 up, 1 in 2024-08-30T08:52:37.157 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:52:36 smithi167 ceph-mon[22063]: from='osd.0 [v2:172.21.15.42:6802/2617890537,v1:172.21.15.42:6803/2617890537]' entity='osd.0' cmd={"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi042", "root=default"]} : dispatch 2024-08-30T08:52:37.157 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:52:36 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-08-30T08:52:37.256 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:52:37.250+0000 7fa8b6c5d640 1 -- 172.21.15.167:0/3852054898 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa8b0103520 msgr2=0x7fa8b0103990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:52:37.256 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:52:37.250+0000 7fa8b6c5d640 1 --2- 172.21.15.167:0/3852054898 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa8b0103520 0x7fa8b0103990 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fa8a4009900 tx=0x7fa8a402a480 comp rx=0 tx=0).stop 2024-08-30T08:52:37.256 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:52:37.250+0000 7fa8b6c5d640 1 -- 172.21.15.167:0/3852054898 shutdown_connections 2024-08-30T08:52:37.256 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:52:37.250+0000 7fa8b6c5d640 1 --2- 172.21.15.167:0/3852054898 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa8b01045b0 0x7fa8b01049c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:52:37.256 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:52:37.250+0000 7fa8b6c5d640 1 --2- 172.21.15.167:0/3852054898 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa8b0104f50 0x7fa8b010b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:52:37.256 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:52:37.250+0000 7fa8b6c5d640 1 --2- 172.21.15.167:0/3852054898 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa8b0103520 0x7fa8b0103990 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:52:37.257 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:52:37.250+0000 7fa8b6c5d640 1 -- 172.21.15.167:0/3852054898 >> 172.21.15.167:0/3852054898 conn(0x7fa8b00fee90 msgr2=0x7fa8b0101340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:52:37.257 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:52:37.250+0000 7fa8b6c5d640 1 -- 172.21.15.167:0/3852054898 shutdown_connections 2024-08-30T08:52:37.257 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:52:37.250+0000 7fa8b6c5d640 1 -- 172.21.15.167:0/3852054898 wait complete. 2024-08-30T08:52:37.257 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:52:37.250+0000 7fa8b6c5d640 1 Processor -- start 2024-08-30T08:52:37.257 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:52:37.251+0000 7fa8b6c5d640 1 -- start start 2024-08-30T08:52:37.257 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:52:37.251+0000 7fa8b6c5d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa8b0103520 0x7fa8b0199db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:52:37.257 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:52:37.251+0000 7fa8b6c5d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa8b01045b0 0x7fa8b019a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:52:37.257 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:52:37.251+0000 7fa8b6c5d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa8b0104f50 0x7fa8b019afc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:52:37.257 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:52:37.251+0000 7fa8b6c5d640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fa8b010f170 con 0x7fa8b01045b0 2024-08-30T08:52:37.257 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:52:37.251+0000 7fa8b6c5d640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa8b010eff0 con 0x7fa8b0103520 2024-08-30T08:52:37.257 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:52:37.251+0000 7fa8b6c5d640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fa8b010f2f0 con 0x7fa8b0104f50 2024-08-30T08:52:37.257 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:52:37.251+0000 7fa8b49f9640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa8b0103520 0x7fa8b0199db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:52:37.258 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:52:37.251+0000 7fa8b49f9640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa8b0103520 0x7fa8b0199db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:48494/0 (socket says 172.21.15.167:48494) 2024-08-30T08:52:37.258 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:52:37.251+0000 7fa8b49f9640 1 -- 172.21.15.167:0/630983451 learned_addr learned my addr 172.21.15.167:0/630983451 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-08-30T08:52:37.258 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:52:37.251+0000 7fa8affff640 1 --2- 172.21.15.167:0/630983451 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa8b01045b0 0x7fa8b019a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:52:37.258 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:52:37.251+0000 7fa8b51fa640 1 --2- 172.21.15.167:0/630983451 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa8b0104f50 0x7fa8b019afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:52:37.258 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:52:37.251+0000 7fa8b49f9640 1 -- 172.21.15.167:0/630983451 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa8b0104f50 msgr2=0x7fa8b019afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:52:37.258 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:52:37.251+0000 7fa8b49f9640 1 --2- 172.21.15.167:0/630983451 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa8b0104f50 0x7fa8b019afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:52:37.258 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:52:37.251+0000 7fa8b49f9640 1 -- 172.21.15.167:0/630983451 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa8b01045b0 msgr2=0x7fa8b019a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:52:37.258 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:52:37.251+0000 7fa8b49f9640 1 --2- 172.21.15.167:0/630983451 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa8b01045b0 0x7fa8b019a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:52:37.258 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:52:37.251+0000 7fa8b49f9640 1 -- 172.21.15.167:0/630983451 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8b019b7e0 con 0x7fa8b0103520 2024-08-30T08:52:37.258 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:52:37.251+0000 7fa8b49f9640 1 --2- 172.21.15.167:0/630983451 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa8b0103520 0x7fa8b0199db0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fa8a4006010 tx=0x7fa8a4004340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:52:37.258 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:52:37.252+0000 7fa8adffb640 1 -- 172.21.15.167:0/630983451 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8a403f030 con 0x7fa8b0103520 2024-08-30T08:52:37.258 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:52:37.252+0000 7fa8b6c5d640 1 -- 172.21.15.167:0/630983451 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8b019ba70 con 0x7fa8b0103520 2024-08-30T08:52:37.258 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:52:37.252+0000 7fa8adffb640 1 -- 172.21.15.167:0/630983451 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa8a4003cb0 con 0x7fa8b0103520 2024-08-30T08:52:37.258 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:52:37.252+0000 7fa8adffb640 1 -- 172.21.15.167:0/630983451 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8a4035070 con 0x7fa8b0103520 2024-08-30T08:52:37.259 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:52:37.252+0000 7fa8b6c5d640 1 -- 172.21.15.167:0/630983451 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8b01a2000 con 0x7fa8b0103520 2024-08-30T08:52:37.259 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:52:37.253+0000 7fa8b6c5d640 1 -- 172.21.15.167:0/630983451 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8b010a700 con 0x7fa8b0103520 2024-08-30T08:52:37.259 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:52:37.255+0000 7fa8adffb640 1 -- 172.21.15.167:0/630983451 <== mon.2 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 15) ==== 152521+0+0 (secure 0 0 0) 0x7fa8a4042030 con 0x7fa8b0103520 2024-08-30T08:52:37.259 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:52:37.256+0000 7fa8adffb640 1 --2- 172.21.15.167:0/630983451 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fa8a00b4aa0 0x7fa8a00b6fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:52:37.259 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:52:37.256+0000 7fa8adffb640 1 -- 172.21.15.167:0/630983451 <== mon.2 v2:172.21.15.167:3300/0 5 ==== osd_map(7..7 src has 1..7) ==== 1487+0+0 (secure 0 0 0) 0x7fa8a4103c20 con 0x7fa8b0103520 2024-08-30T08:52:37.260 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:52:37.256+0000 7fa8affff640 1 --2- 172.21.15.167:0/630983451 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fa8a00b4aa0 0x7fa8a00b6fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:52:37.260 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:52:37.257+0000 7fa8affff640 1 --2- 172.21.15.167:0/630983451 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fa8a00b4aa0 0x7fa8a00b6fb0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fa89c005fd0 tx=0x7fa89c00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:52:37.263 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:52:37.261+0000 7fa8adffb640 1 -- 172.21.15.167:0/630983451 <== mon.2 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7fa8a4003e60 con 0x7fa8b0103520 2024-08-30T08:52:37.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:52:36 smithi190 ceph-mon[22001]: from='osd.0 [v2:172.21.15.42:6802/2617890537,v1:172.21.15.42:6803/2617890537]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-08-30T08:52:37.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:52:36 smithi190 ceph-mon[22001]: osdmap e6: 1 total, 0 up, 1 in 2024-08-30T08:52:37.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:52:36 smithi190 ceph-mon[22001]: from='osd.0 [v2:172.21.15.42:6802/2617890537,v1:172.21.15.42:6803/2617890537]' entity='osd.0' cmd={"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi042", "root=default"]} : dispatch 2024-08-30T08:52:37.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:52:36 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-08-30T08:52:37.454 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:52:37.451+0000 7fa8b6c5d640 1 -- 172.21.15.167:0/630983451 --> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fa8b00635a0 con 0x7fa8a00b4aa0 2024-08-30T08:52:38.091 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:52:37 smithi167 ceph-mon[22063]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-30T08:52:38.091 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:52:37 smithi167 ceph-mon[22063]: from='osd.0 [v2:172.21.15.42:6802/2617890537,v1:172.21.15.42:6803/2617890537]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi042", "root=default"]}]': finished 2024-08-30T08:52:38.091 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:52:37 smithi167 ceph-mon[22063]: osdmap e7: 1 total, 0 up, 1 in 2024-08-30T08:52:38.091 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:52:37 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-08-30T08:52:38.091 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:52:37 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-08-30T08:52:38.091 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:52:37 smithi167 ceph-mon[22063]: Detected new or changed devices on smithi042 2024-08-30T08:52:38.091 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:52:37 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:52:38.091 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:52:37 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:52:38.091 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:52:37 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-08-30T08:52:38.091 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:52:37 smithi167 ceph-mon[22063]: Adjusting osd_memory_target on smithi042 to 17092M 2024-08-30T08:52:38.091 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:52:37 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:52:38.092 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:52:37 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:52:38.092 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:52:37 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-30T08:52:38.092 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:52:37 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:52:38.092 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:52:37 smithi167 ceph-mon[22063]: from='client.24242 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-08-30T08:52:38.092 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:52:37 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-30T08:52:38.092 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:52:37 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-30T08:52:38.092 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:52:37 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:52:38.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:52:37 smithi042 ceph-mon[18652]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-30T08:52:38.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:52:37 smithi042 ceph-mon[18652]: from='osd.0 [v2:172.21.15.42:6802/2617890537,v1:172.21.15.42:6803/2617890537]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi042", "root=default"]}]': finished 2024-08-30T08:52:38.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:52:37 smithi042 ceph-mon[18652]: osdmap e7: 1 total, 0 up, 1 in 2024-08-30T08:52:38.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:52:37 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-08-30T08:52:38.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:52:37 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-08-30T08:52:38.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:52:37 smithi042 ceph-mon[18652]: Detected new or changed devices on smithi042 2024-08-30T08:52:38.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:52:37 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:52:38.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:52:37 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:52:38.384 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:52:37 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-08-30T08:52:38.384 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:52:37 smithi042 ceph-mon[18652]: Adjusting osd_memory_target on smithi042 to 17092M 2024-08-30T08:52:38.384 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:52:37 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:52:38.384 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:52:37 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:52:38.384 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:52:37 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-30T08:52:38.384 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:52:37 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:52:38.384 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:52:37 smithi042 ceph-mon[18652]: from='client.24242 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-08-30T08:52:38.384 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:52:37 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-30T08:52:38.384 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:52:37 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-30T08:52:38.384 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:52:37 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:52:38.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:52:37 smithi190 ceph-mon[22001]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-30T08:52:38.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:52:37 smithi190 ceph-mon[22001]: from='osd.0 [v2:172.21.15.42:6802/2617890537,v1:172.21.15.42:6803/2617890537]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi042", "root=default"]}]': finished 2024-08-30T08:52:38.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:52:37 smithi190 ceph-mon[22001]: osdmap e7: 1 total, 0 up, 1 in 2024-08-30T08:52:38.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:52:37 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-08-30T08:52:38.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:52:37 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-08-30T08:52:38.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:52:37 smithi190 ceph-mon[22001]: Detected new or changed devices on smithi042 2024-08-30T08:52:38.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:52:37 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:52:38.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:52:37 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:52:38.437 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:52:37 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-08-30T08:52:38.437 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:52:37 smithi190 ceph-mon[22001]: Adjusting osd_memory_target on smithi042 to 17092M 2024-08-30T08:52:38.437 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:52:37 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:52:38.437 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:52:37 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:52:38.437 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:52:37 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-30T08:52:38.437 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:52:37 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:52:38.437 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:52:37 smithi190 ceph-mon[22001]: from='client.24242 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-08-30T08:52:38.437 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:52:37 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-30T08:52:38.437 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:52:37 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-30T08:52:38.437 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:52:37 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:52:39.294 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:52:38 smithi167 ceph-mon[22063]: purged_snaps scrub starts 2024-08-30T08:52:39.294 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:52:38 smithi167 ceph-mon[22063]: purged_snaps scrub ok 2024-08-30T08:52:39.294 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:52:38 smithi167 ceph-mon[22063]: osd.0 [v2:172.21.15.42:6802/2617890537,v1:172.21.15.42:6803/2617890537] boot 2024-08-30T08:52:39.294 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:52:38 smithi167 ceph-mon[22063]: osdmap e8: 1 total, 1 up, 1 in 2024-08-30T08:52:39.294 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:52:38 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-08-30T08:52:39.337 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:52:38 smithi190 ceph-mon[22001]: purged_snaps scrub starts 2024-08-30T08:52:39.337 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:52:38 smithi190 ceph-mon[22001]: purged_snaps scrub ok 2024-08-30T08:52:39.337 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:52:38 smithi190 ceph-mon[22001]: osd.0 [v2:172.21.15.42:6802/2617890537,v1:172.21.15.42:6803/2617890537] boot 2024-08-30T08:52:39.337 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:52:38 smithi190 ceph-mon[22001]: osdmap e8: 1 total, 1 up, 1 in 2024-08-30T08:52:39.337 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:52:38 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-08-30T08:52:39.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:52:38 smithi042 ceph-mon[18652]: purged_snaps scrub starts 2024-08-30T08:52:39.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:52:38 smithi042 ceph-mon[18652]: purged_snaps scrub ok 2024-08-30T08:52:39.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:52:38 smithi042 ceph-mon[18652]: osd.0 [v2:172.21.15.42:6802/2617890537,v1:172.21.15.42:6803/2617890537] boot 2024-08-30T08:52:39.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:52:38 smithi042 ceph-mon[18652]: osdmap e8: 1 total, 1 up, 1 in 2024-08-30T08:52:39.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:52:38 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-08-30T08:52:40.245 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:52:39 smithi167 ceph-mon[22063]: pgmap v55: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-30T08:52:40.245 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:52:39 smithi167 ceph-mon[22063]: osdmap e9: 1 total, 1 up, 1 in 2024-08-30T08:52:40.246 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:52:39 smithi167 ceph-mon[22063]: from='client.? 172.21.15.167:0/507211456' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "3a78384b-0207-41df-9384-22665007d5b3"} : dispatch 2024-08-30T08:52:40.246 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:52:39 smithi167 ceph-mon[22063]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "3a78384b-0207-41df-9384-22665007d5b3"} : dispatch 2024-08-30T08:52:40.246 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:52:39 smithi167 ceph-mon[22063]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3a78384b-0207-41df-9384-22665007d5b3"}]': finished 2024-08-30T08:52:40.246 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:52:39 smithi167 ceph-mon[22063]: osdmap e10: 2 total, 1 up, 2 in 2024-08-30T08:52:40.246 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:52:39 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-08-30T08:52:40.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:52:39 smithi042 ceph-mon[18652]: pgmap v55: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-30T08:52:40.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:52:39 smithi042 ceph-mon[18652]: osdmap e9: 1 total, 1 up, 1 in 2024-08-30T08:52:40.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:52:39 smithi042 ceph-mon[18652]: from='client.? 172.21.15.167:0/507211456' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "3a78384b-0207-41df-9384-22665007d5b3"} : dispatch 2024-08-30T08:52:40.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:52:39 smithi042 ceph-mon[18652]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "3a78384b-0207-41df-9384-22665007d5b3"} : dispatch 2024-08-30T08:52:40.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:52:39 smithi042 ceph-mon[18652]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3a78384b-0207-41df-9384-22665007d5b3"}]': finished 2024-08-30T08:52:40.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:52:39 smithi042 ceph-mon[18652]: osdmap e10: 2 total, 1 up, 2 in 2024-08-30T08:52:40.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:52:39 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-08-30T08:52:40.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:52:39 smithi190 ceph-mon[22001]: pgmap v55: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-30T08:52:40.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:52:39 smithi190 ceph-mon[22001]: osdmap e9: 1 total, 1 up, 1 in 2024-08-30T08:52:40.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:52:39 smithi190 ceph-mon[22001]: from='client.? 172.21.15.167:0/507211456' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "3a78384b-0207-41df-9384-22665007d5b3"} : dispatch 2024-08-30T08:52:40.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:52:39 smithi190 ceph-mon[22001]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "3a78384b-0207-41df-9384-22665007d5b3"} : dispatch 2024-08-30T08:52:40.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:52:39 smithi190 ceph-mon[22001]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3a78384b-0207-41df-9384-22665007d5b3"}]': finished 2024-08-30T08:52:40.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:52:39 smithi190 ceph-mon[22001]: osdmap e10: 2 total, 1 up, 2 in 2024-08-30T08:52:40.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:52:39 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-08-30T08:52:41.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:52:40 smithi167 ceph-mon[22063]: from='client.? 172.21.15.167:0/2890591041' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-30T08:52:41.382 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:52:40 smithi042 ceph-mon[18652]: from='client.? 172.21.15.167:0/2890591041' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-30T08:52:41.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:52:40 smithi190 ceph-mon[22001]: from='client.? 172.21.15.167:0/2890591041' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-30T08:52:42.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:52:41 smithi042 ceph-mon[18652]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-30T08:52:42.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:52:41 smithi167 ceph-mon[22063]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-30T08:52:42.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:52:41 smithi190 ceph-mon[22001]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-30T08:52:44.242 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:52:43 smithi167 ceph-mon[22063]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-30T08:52:44.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:52:43 smithi042 ceph-mon[18652]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-30T08:52:44.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:52:43 smithi190 ceph-mon[22001]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-30T08:52:46.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:52:45 smithi167 ceph-mon[22063]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-30T08:52:46.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:52:45 smithi042 ceph-mon[18652]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-30T08:52:46.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:52:45 smithi190 ceph-mon[22001]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-30T08:52:48.324 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:52:47 smithi167 ceph-mon[22063]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-30T08:52:48.324 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:52:47 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.1"} : dispatch 2024-08-30T08:52:48.324 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:52:47 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:52:48.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:52:47 smithi042 ceph-mon[18652]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-30T08:52:48.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:52:47 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.1"} : dispatch 2024-08-30T08:52:48.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:52:47 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:52:48.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:52:47 smithi190 ceph-mon[22001]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-30T08:52:48.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:52:47 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.1"} : dispatch 2024-08-30T08:52:48.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:52:47 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:52:49.266 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:52:48 smithi167 ceph-mon[22063]: Deploying daemon osd.1 on smithi167 2024-08-30T08:52:49.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:52:48 smithi042 ceph-mon[18652]: Deploying daemon osd.1 on smithi167 2024-08-30T08:52:49.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:52:48 smithi190 ceph-mon[22001]: Deploying daemon osd.1 on smithi167 2024-08-30T08:52:50.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:52:49 smithi167 ceph-mon[22063]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-30T08:52:50.382 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:52:49 smithi042 ceph-mon[18652]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-30T08:52:50.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:52:49 smithi190 ceph-mon[22001]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-30T08:52:52.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:52:51 smithi167 ceph-mon[22063]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-30T08:52:52.382 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:52:51 smithi042 ceph-mon[18652]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-30T08:52:52.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:52:51 smithi190 ceph-mon[22001]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-30T08:52:53.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:52:53 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-30T08:52:53.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:52:53 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:52:53.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:52:53 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:52:53.382 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:52:52 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-30T08:52:53.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:52:52 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:52:53.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:52:52 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:52:53.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:52:52 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-30T08:52:53.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:52:52 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:52:53.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:52:52 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:52:54.326 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:52:54 smithi167 ceph-mon[22063]: pgmap v64: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-30T08:52:54.382 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:52:53 smithi042 ceph-mon[18652]: pgmap v64: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-30T08:52:54.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:52:54 smithi190 ceph-mon[22001]: pgmap v64: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-30T08:52:56.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:52:56 smithi167 ceph-mon[22063]: pgmap v65: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-30T08:52:56.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:52:56 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:52:56.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:52:56 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:52:56.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:52:56 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:52:56.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:52:56 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-30T08:52:56.828 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:52:56 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:52:56.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:52:56 smithi042 ceph-mon[18652]: pgmap v65: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-30T08:52:56.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:52:56 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:52:56.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:52:56 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:52:56.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:52:56 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:52:56.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:52:56 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-30T08:52:56.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:52:56 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:52:56.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:52:56 smithi190 ceph-mon[22001]: pgmap v65: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-30T08:52:56.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:52:56 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:52:56.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:52:56 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:52:56.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:52:56 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:52:56.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:52:56 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-30T08:52:56.937 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:52:56 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:52:57.126 INFO:teuthology.orchestra.run.smithi167.stdout:Created osd(s) 1 on host 'smithi167' 2024-08-30T08:52:57.127 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:52:57.125+0000 7fa8adffb640 1 -- 172.21.15.167:0/630983451 <== mgr.14150 v2:172.21.15.42:6800/3005140902 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fa8b00635a0 con 0x7fa8a00b4aa0 2024-08-30T08:52:57.129 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:52:57.127+0000 7fa8b6c5d640 1 -- 172.21.15.167:0/630983451 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fa8a00b4aa0 msgr2=0x7fa8a00b6fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:52:57.129 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:52:57.127+0000 7fa8b6c5d640 1 --2- 172.21.15.167:0/630983451 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fa8a00b4aa0 0x7fa8a00b6fb0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fa89c005fd0 tx=0x7fa89c00b390 comp rx=0 tx=0).stop 2024-08-30T08:52:57.129 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:52:57.127+0000 7fa8b6c5d640 1 -- 172.21.15.167:0/630983451 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa8b0103520 msgr2=0x7fa8b0199db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:52:57.129 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:52:57.127+0000 7fa8b6c5d640 1 --2- 172.21.15.167:0/630983451 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa8b0103520 0x7fa8b0199db0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fa8a4006010 tx=0x7fa8a4004340 comp rx=0 tx=0).stop 2024-08-30T08:52:57.129 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:52:57.127+0000 7fa8b6c5d640 1 -- 172.21.15.167:0/630983451 shutdown_connections 2024-08-30T08:52:57.129 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:52:57.127+0000 7fa8b6c5d640 1 --2- 172.21.15.167:0/630983451 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fa8a00b4aa0 0x7fa8a00b6fb0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:52:57.130 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:52:57.127+0000 7fa8b6c5d640 1 --2- 172.21.15.167:0/630983451 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa8b0104f50 0x7fa8b019afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:52:57.130 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:52:57.127+0000 7fa8b6c5d640 1 --2- 172.21.15.167:0/630983451 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa8b01045b0 0x7fa8b019a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:52:57.130 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:52:57.127+0000 7fa8b6c5d640 1 --2- 172.21.15.167:0/630983451 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa8b0103520 0x7fa8b0199db0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:52:57.130 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:52:57.127+0000 7fa8b6c5d640 1 -- 172.21.15.167:0/630983451 >> 172.21.15.167:0/630983451 conn(0x7fa8b00fee90 msgr2=0x7fa8b01078d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:52:57.130 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:52:57.128+0000 7fa8b6c5d640 1 -- 172.21.15.167:0/630983451 shutdown_connections 2024-08-30T08:52:57.130 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:52:57.128+0000 7fa8b6c5d640 1 -- 172.21.15.167:0/630983451 wait complete. 2024-08-30T08:52:57.782 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:52:57 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-30T08:52:57.782 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:52:57 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:52:57.782 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:52:57 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:52:57.808 DEBUG:teuthology.orchestra.run.smithi167:osd.1> sudo journalctl -f -n 0 -u ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968@osd.1.service 2024-08-30T08:52:57.811 INFO:tasks.cephadm:Deploying osd.2 on smithi190 with /dev/vg_nvme/lv_4... 2024-08-30T08:52:57.811 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-08-30T08:52:57.838 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:52:57 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-30T08:52:57.838 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:52:57 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:52:57.838 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:52:57 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:52:57.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:52:57 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-30T08:52:57.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:52:57 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:52:57.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:52:57 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:52:58.063 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.c/config 2024-08-30T08:52:58.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:52:58 smithi167 ceph-mon[22063]: pgmap v66: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-30T08:52:58.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:52:58 smithi042 ceph-mon[18652]: pgmap v66: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-30T08:52:58.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:52:58 smithi190 ceph-mon[22001]: pgmap v66: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-30T08:52:59.233 INFO:journalctl@ceph.osd.1.smithi167.stdout:Aug 30 08:52:59 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-1[26506]: 2024-08-30T08:52:59.031+0000 7f9a72779640 -1 osd.1 0 log_to_monitors true 2024-08-30T08:52:59.784 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:52:59 smithi042 ceph-mon[18652]: from='osd.1 [v2:172.21.15.167:6800/2791960082,v1:172.21.15.167:6801/2791960082]' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-08-30T08:52:59.784 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:52:59 smithi042 ceph-mon[18652]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-08-30T08:52:59.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:52:59 smithi167 ceph-mon[22063]: from='osd.1 [v2:172.21.15.167:6800/2791960082,v1:172.21.15.167:6801/2791960082]' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-08-30T08:52:59.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:52:59 smithi167 ceph-mon[22063]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-08-30T08:52:59.877 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:52:59 smithi190 ceph-mon[22001]: from='osd.1 [v2:172.21.15.167:6800/2791960082,v1:172.21.15.167:6801/2791960082]' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-08-30T08:52:59.877 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:52:59 smithi190 ceph-mon[22001]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-08-30T08:53:00.503 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-08-30T08:53:00.531 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph orch daemon add osd smithi190:vg_nvme/lv_4 2024-08-30T08:53:00.577 INFO:journalctl@ceph.osd.1.smithi167.stdout:Aug 30 08:53:00 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-1[26506]: 2024-08-30T08:53:00.490+0000 7f9a6e6fb640 -1 osd.1 0 waiting for initial osdmap 2024-08-30T08:53:00.577 INFO:journalctl@ceph.osd.1.smithi167.stdout:Aug 30 08:53:00 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-1[26506]: 2024-08-30T08:53:00.494+0000 7f9a6a4f9640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-30T08:53:00.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:00 smithi167 ceph-mon[22063]: pgmap v67: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-30T08:53:00.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:00 smithi167 ceph-mon[22063]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-08-30T08:53:00.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:00 smithi167 ceph-mon[22063]: osdmap e11: 2 total, 1 up, 2 in 2024-08-30T08:53:00.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:00 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-08-30T08:53:00.578 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:00 smithi167 ceph-mon[22063]: from='osd.1 [v2:172.21.15.167:6800/2791960082,v1:172.21.15.167:6801/2791960082]' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi167", "root=default"]} : dispatch 2024-08-30T08:53:00.578 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:00 smithi167 ceph-mon[22063]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi167", "root=default"]} : dispatch 2024-08-30T08:53:00.595 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:00 smithi190 ceph-mon[22001]: pgmap v67: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-30T08:53:00.595 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:00 smithi190 ceph-mon[22001]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-08-30T08:53:00.595 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:00 smithi190 ceph-mon[22001]: osdmap e11: 2 total, 1 up, 2 in 2024-08-30T08:53:00.595 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:00 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-08-30T08:53:00.595 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:00 smithi190 ceph-mon[22001]: from='osd.1 [v2:172.21.15.167:6800/2791960082,v1:172.21.15.167:6801/2791960082]' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi167", "root=default"]} : dispatch 2024-08-30T08:53:00.595 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:00 smithi190 ceph-mon[22001]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi167", "root=default"]} : dispatch 2024-08-30T08:53:00.820 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.c/config 2024-08-30T08:53:00.882 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:00 smithi042 ceph-mon[18652]: pgmap v67: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-30T08:53:00.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:00 smithi042 ceph-mon[18652]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-08-30T08:53:00.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:00 smithi042 ceph-mon[18652]: osdmap e11: 2 total, 1 up, 2 in 2024-08-30T08:53:00.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:00 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-08-30T08:53:00.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:00 smithi042 ceph-mon[18652]: from='osd.1 [v2:172.21.15.167:6800/2791960082,v1:172.21.15.167:6801/2791960082]' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi167", "root=default"]} : dispatch 2024-08-30T08:53:00.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:00 smithi042 ceph-mon[18652]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi167", "root=default"]} : dispatch 2024-08-30T08:53:01.257 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:53:01.253+0000 7fbd21d58640 1 -- 172.21.15.190:0/2121617751 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbd1c1023e0 msgr2=0x7fbd1c102850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:53:01.257 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:53:01.253+0000 7fbd21d58640 1 --2- 172.21.15.190:0/2121617751 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbd1c1023e0 0x7fbd1c102850 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fbd10009900 tx=0x7fbd1002a480 comp rx=0 tx=0).stop 2024-08-30T08:53:01.257 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:53:01.253+0000 7fbd21d58640 1 -- 172.21.15.190:0/2121617751 shutdown_connections 2024-08-30T08:53:01.257 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:53:01.253+0000 7fbd21d58640 1 --2- 172.21.15.190:0/2121617751 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbd1c103e10 0x7fbd1c10a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:53:01.258 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:53:01.253+0000 7fbd21d58640 1 --2- 172.21.15.190:0/2121617751 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbd1c103470 0x7fbd1c103880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:53:01.258 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:53:01.253+0000 7fbd21d58640 1 --2- 172.21.15.190:0/2121617751 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbd1c1023e0 0x7fbd1c102850 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:53:01.258 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:53:01.253+0000 7fbd21d58640 1 -- 172.21.15.190:0/2121617751 >> 172.21.15.190:0/2121617751 conn(0x7fbd1c0fdd50 msgr2=0x7fbd1c100200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:53:01.258 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:53:01.253+0000 7fbd21d58640 1 -- 172.21.15.190:0/2121617751 shutdown_connections 2024-08-30T08:53:01.258 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:53:01.253+0000 7fbd21d58640 1 -- 172.21.15.190:0/2121617751 wait complete. 2024-08-30T08:53:01.258 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:53:01.254+0000 7fbd21d58640 1 Processor -- start 2024-08-30T08:53:01.258 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:53:01.254+0000 7fbd21d58640 1 -- start start 2024-08-30T08:53:01.258 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:53:01.254+0000 7fbd21d58640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbd1c1023e0 0x7fbd1c06b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:53:01.258 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:53:01.254+0000 7fbd21d58640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbd1c103470 0x7fbd1c1a0e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:53:01.258 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:53:01.254+0000 7fbd21d58640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbd1c103e10 0x7fbd1c1a33e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:53:01.258 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:53:01.254+0000 7fbd21d58640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fbd1c10e090 con 0x7fbd1c103470 2024-08-30T08:53:01.258 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:53:01.254+0000 7fbd21d58640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fbd1c10df10 con 0x7fbd1c1023e0 2024-08-30T08:53:01.259 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:53:01.254+0000 7fbd21d58640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fbd1c10e210 con 0x7fbd1c103e10 2024-08-30T08:53:01.259 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:53:01.254+0000 7fbd1bfff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbd1c103e10 0x7fbd1c1a33e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:53:01.259 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:53:01.254+0000 7fbd1b7fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbd1c1023e0 0x7fbd1c06b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:53:01.259 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:53:01.254+0000 7fbd1bfff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbd1c103e10 0x7fbd1c1a33e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:49494/0 (socket says 172.21.15.190:49494) 2024-08-30T08:53:01.259 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:53:01.254+0000 7fbd1b7fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbd1c1023e0 0x7fbd1c06b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.190:45054/0 (socket says 172.21.15.190:45054) 2024-08-30T08:53:01.259 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:53:01.254+0000 7fbd1bfff640 1 -- 172.21.15.190:0/3834170667 learned_addr learned my addr 172.21.15.190:0/3834170667 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-30T08:53:01.259 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:53:01.254+0000 7fbd1affd640 1 --2- 172.21.15.190:0/3834170667 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbd1c103470 0x7fbd1c1a0e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:53:01.259 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:53:01.255+0000 7fbd1bfff640 1 -- 172.21.15.190:0/3834170667 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbd1c1023e0 msgr2=0x7fbd1c06b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:53:01.259 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:53:01.255+0000 7fbd1bfff640 1 --2- 172.21.15.190:0/3834170667 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbd1c1023e0 0x7fbd1c06b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:53:01.259 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:53:01.255+0000 7fbd1bfff640 1 -- 172.21.15.190:0/3834170667 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbd1c103470 msgr2=0x7fbd1c1a0e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:53:01.259 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:53:01.255+0000 7fbd1bfff640 1 --2- 172.21.15.190:0/3834170667 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbd1c103470 0x7fbd1c1a0e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:53:01.259 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:53:01.255+0000 7fbd1bfff640 1 -- 172.21.15.190:0/3834170667 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd1c1a3950 con 0x7fbd1c103e10 2024-08-30T08:53:01.260 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:53:01.255+0000 7fbd1b7fe640 1 --2- 172.21.15.190:0/3834170667 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbd1c1023e0 0x7fbd1c06b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-30T08:53:01.260 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:53:01.255+0000 7fbd1bfff640 1 --2- 172.21.15.190:0/3834170667 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbd1c103e10 0x7fbd1c1a33e0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fbd1400eb10 tx=0x7fbd1400ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:53:01.260 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:53:01.255+0000 7fbd18ff9640 1 -- 172.21.15.190:0/3834170667 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd1400ce20 con 0x7fbd1c103e10 2024-08-30T08:53:01.260 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:53:01.255+0000 7fbd21d58640 1 -- 172.21.15.190:0/3834170667 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd1c1a3c40 con 0x7fbd1c103e10 2024-08-30T08:53:01.260 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:53:01.255+0000 7fbd18ff9640 1 -- 172.21.15.190:0/3834170667 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbd14005020 con 0x7fbd1c103e10 2024-08-30T08:53:01.260 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:53:01.255+0000 7fbd18ff9640 1 -- 172.21.15.190:0/3834170667 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd14010690 con 0x7fbd1c103e10 2024-08-30T08:53:01.260 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:53:01.255+0000 7fbd21d58640 1 -- 172.21.15.190:0/3834170667 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd1c1a4180 con 0x7fbd1c103e10 2024-08-30T08:53:01.261 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:53:01.256+0000 7fbd21d58640 1 -- 172.21.15.190:0/3834170667 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd1c1095c0 con 0x7fbd1c103e10 2024-08-30T08:53:01.261 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:53:01.258+0000 7fbd18ff9640 1 -- 172.21.15.190:0/3834170667 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 15) ==== 152521+0+0 (secure 0 0 0) 0x7fbd14004b20 con 0x7fbd1c103e10 2024-08-30T08:53:01.261 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:53:01.258+0000 7fbd18ff9640 1 --2- 172.21.15.190:0/3834170667 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fbd040b4aa0 0x7fbd040b6fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:53:01.261 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:53:01.258+0000 7fbd18ff9640 1 -- 172.21.15.190:0/3834170667 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(12..12 src has 1..12) ==== 2029+0+0 (secure 0 0 0) 0x7fbd140e3ea0 con 0x7fbd1c103e10 2024-08-30T08:53:01.261 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:53:01.259+0000 7fbd1b7fe640 1 --2- 172.21.15.190:0/3834170667 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fbd040b4aa0 0x7fbd040b6fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:53:01.261 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:53:01.259+0000 7fbd1b7fe640 1 --2- 172.21.15.190:0/3834170667 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fbd040b4aa0 0x7fbd040b6fb0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fbd10004770 tx=0x7fbd100314a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:53:01.265 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:53:01.263+0000 7fbd18ff9640 1 -- 172.21.15.190:0/3834170667 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7fbd140ac940 con 0x7fbd1c103e10 2024-08-30T08:53:01.453 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:53:01.451+0000 7fbd21d58640 1 -- 172.21.15.190:0/3834170667 --> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi190:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fbd1c0635a0 con 0x7fbd040b4aa0 2024-08-30T08:53:01.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:01 smithi190 ceph-mon[22001]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]': finished 2024-08-30T08:53:01.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:01 smithi190 ceph-mon[22001]: osdmap e12: 2 total, 1 up, 2 in 2024-08-30T08:53:01.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:01 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-08-30T08:53:01.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:01 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-08-30T08:53:01.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:01 smithi190 ceph-mon[22001]: Detected new or changed devices on smithi167 2024-08-30T08:53:01.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:01 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:53:01.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:01 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:53:01.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:01 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-08-30T08:53:01.687 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:01 smithi190 ceph-mon[22001]: Adjusting osd_memory_target on smithi167 to 17088M 2024-08-30T08:53:01.687 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:01 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:53:01.687 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:01 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:53:01.687 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:01 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-30T08:53:01.687 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:01 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:53:01.687 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:01 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-30T08:53:01.687 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:01 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-30T08:53:01.687 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:01 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:53:01.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:01 smithi167 ceph-mon[22063]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]': finished 2024-08-30T08:53:01.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:01 smithi167 ceph-mon[22063]: osdmap e12: 2 total, 1 up, 2 in 2024-08-30T08:53:01.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:01 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-08-30T08:53:01.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:01 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-08-30T08:53:01.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:01 smithi167 ceph-mon[22063]: Detected new or changed devices on smithi167 2024-08-30T08:53:01.828 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:01 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:53:01.828 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:01 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:53:01.828 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:01 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-08-30T08:53:01.828 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:01 smithi167 ceph-mon[22063]: Adjusting osd_memory_target on smithi167 to 17088M 2024-08-30T08:53:01.828 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:01 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:53:01.828 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:01 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:53:01.828 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:01 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-30T08:53:01.828 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:01 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:53:01.828 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:01 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-30T08:53:01.828 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:01 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-30T08:53:01.828 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:01 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:53:01.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:01 smithi042 ceph-mon[18652]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]': finished 2024-08-30T08:53:01.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:01 smithi042 ceph-mon[18652]: osdmap e12: 2 total, 1 up, 2 in 2024-08-30T08:53:01.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:01 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-08-30T08:53:01.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:01 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-08-30T08:53:01.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:01 smithi042 ceph-mon[18652]: Detected new or changed devices on smithi167 2024-08-30T08:53:01.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:01 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:53:01.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:01 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:53:01.884 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:01 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-08-30T08:53:01.884 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:01 smithi042 ceph-mon[18652]: Adjusting osd_memory_target on smithi167 to 17088M 2024-08-30T08:53:01.884 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:01 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:53:01.884 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:01 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:53:01.884 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:01 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-30T08:53:01.884 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:01 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:53:01.884 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:01 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-30T08:53:01.884 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:01 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-30T08:53:01.884 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:01 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:53:02.747 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:02 smithi167 ceph-mon[22063]: purged_snaps scrub starts 2024-08-30T08:53:02.747 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:02 smithi167 ceph-mon[22063]: purged_snaps scrub ok 2024-08-30T08:53:02.747 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:02 smithi167 ceph-mon[22063]: pgmap v70: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-30T08:53:02.747 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:02 smithi167 ceph-mon[22063]: from='client.24277 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi190:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-08-30T08:53:02.747 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:02 smithi167 ceph-mon[22063]: osd.1 [v2:172.21.15.167:6800/2791960082,v1:172.21.15.167:6801/2791960082] boot 2024-08-30T08:53:02.747 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:02 smithi167 ceph-mon[22063]: osdmap e13: 2 total, 2 up, 2 in 2024-08-30T08:53:02.747 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:02 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-08-30T08:53:02.759 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:02 smithi190 ceph-mon[22001]: purged_snaps scrub starts 2024-08-30T08:53:02.759 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:02 smithi190 ceph-mon[22001]: purged_snaps scrub ok 2024-08-30T08:53:02.759 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:02 smithi190 ceph-mon[22001]: pgmap v70: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-30T08:53:02.759 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:02 smithi190 ceph-mon[22001]: from='client.24277 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi190:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-08-30T08:53:02.759 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:02 smithi190 ceph-mon[22001]: osd.1 [v2:172.21.15.167:6800/2791960082,v1:172.21.15.167:6801/2791960082] boot 2024-08-30T08:53:02.759 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:02 smithi190 ceph-mon[22001]: osdmap e13: 2 total, 2 up, 2 in 2024-08-30T08:53:02.760 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:02 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-08-30T08:53:02.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:02 smithi042 ceph-mon[18652]: purged_snaps scrub starts 2024-08-30T08:53:02.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:02 smithi042 ceph-mon[18652]: purged_snaps scrub ok 2024-08-30T08:53:02.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:02 smithi042 ceph-mon[18652]: pgmap v70: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-30T08:53:02.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:02 smithi042 ceph-mon[18652]: from='client.24277 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi190:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-08-30T08:53:02.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:02 smithi042 ceph-mon[18652]: osd.1 [v2:172.21.15.167:6800/2791960082,v1:172.21.15.167:6801/2791960082] boot 2024-08-30T08:53:02.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:02 smithi042 ceph-mon[18652]: osdmap e13: 2 total, 2 up, 2 in 2024-08-30T08:53:02.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:02 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-08-30T08:53:03.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:03 smithi190 ceph-mon[22001]: osdmap e14: 2 total, 2 up, 2 in 2024-08-30T08:53:03.961 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:03 smithi167 ceph-mon[22063]: osdmap e14: 2 total, 2 up, 2 in 2024-08-30T08:53:04.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:03 smithi042 ceph-mon[18652]: osdmap e14: 2 total, 2 up, 2 in 2024-08-30T08:53:04.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:04 smithi190 ceph-mon[22001]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-08-30T08:53:04.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:04 smithi190 ceph-mon[22001]: from='client.? 172.21.15.190:0/2768238559' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "16001809-c27c-4f52-948a-9998261acb4b"} : dispatch 2024-08-30T08:53:04.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:04 smithi190 ceph-mon[22001]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "16001809-c27c-4f52-948a-9998261acb4b"} : dispatch 2024-08-30T08:53:04.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:04 smithi190 ceph-mon[22001]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "16001809-c27c-4f52-948a-9998261acb4b"}]': finished 2024-08-30T08:53:04.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:04 smithi190 ceph-mon[22001]: osdmap e15: 3 total, 2 up, 3 in 2024-08-30T08:53:04.937 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:04 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-08-30T08:53:04.937 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:04 smithi190 ceph-mon[22001]: from='client.? 172.21.15.190:0/3599900574' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-30T08:53:05.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:04 smithi167 ceph-mon[22063]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-08-30T08:53:05.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:04 smithi167 ceph-mon[22063]: from='client.? 172.21.15.190:0/2768238559' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "16001809-c27c-4f52-948a-9998261acb4b"} : dispatch 2024-08-30T08:53:05.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:04 smithi167 ceph-mon[22063]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "16001809-c27c-4f52-948a-9998261acb4b"} : dispatch 2024-08-30T08:53:05.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:04 smithi167 ceph-mon[22063]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "16001809-c27c-4f52-948a-9998261acb4b"}]': finished 2024-08-30T08:53:05.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:04 smithi167 ceph-mon[22063]: osdmap e15: 3 total, 2 up, 3 in 2024-08-30T08:53:05.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:04 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-08-30T08:53:05.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:04 smithi167 ceph-mon[22063]: from='client.? 172.21.15.190:0/3599900574' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-30T08:53:05.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:04 smithi042 ceph-mon[18652]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-08-30T08:53:05.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:04 smithi042 ceph-mon[18652]: from='client.? 172.21.15.190:0/2768238559' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "16001809-c27c-4f52-948a-9998261acb4b"} : dispatch 2024-08-30T08:53:05.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:04 smithi042 ceph-mon[18652]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "16001809-c27c-4f52-948a-9998261acb4b"} : dispatch 2024-08-30T08:53:05.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:04 smithi042 ceph-mon[18652]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "16001809-c27c-4f52-948a-9998261acb4b"}]': finished 2024-08-30T08:53:05.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:04 smithi042 ceph-mon[18652]: osdmap e15: 3 total, 2 up, 3 in 2024-08-30T08:53:05.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:04 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-08-30T08:53:05.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:04 smithi042 ceph-mon[18652]: from='client.? 172.21.15.190:0/3599900574' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-30T08:53:06.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:06 smithi190 ceph-mon[22001]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-08-30T08:53:07.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:06 smithi167 ceph-mon[22063]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-08-30T08:53:07.132 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:06 smithi042 ceph-mon[18652]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-08-30T08:53:08.923 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:08 smithi190 ceph-mon[22001]: pgmap v76: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-08-30T08:53:09.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:08 smithi167 ceph-mon[22063]: pgmap v76: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-08-30T08:53:09.132 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:08 smithi042 ceph-mon[18652]: pgmap v76: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-08-30T08:53:10.881 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:10 smithi190 ceph-mon[22001]: pgmap v77: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-08-30T08:53:10.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:10 smithi042 ceph-mon[18652]: pgmap v77: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-08-30T08:53:11.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:10 smithi167 ceph-mon[22063]: pgmap v77: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-08-30T08:53:12.789 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:12 smithi190 ceph-mon[22001]: pgmap v78: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-08-30T08:53:12.790 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:12 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.2"} : dispatch 2024-08-30T08:53:12.790 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:12 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:53:13.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:12 smithi167 ceph-mon[22063]: pgmap v78: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-08-30T08:53:13.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:12 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.2"} : dispatch 2024-08-30T08:53:13.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:12 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:53:13.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:12 smithi042 ceph-mon[18652]: pgmap v78: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-08-30T08:53:13.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:12 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.2"} : dispatch 2024-08-30T08:53:13.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:12 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:53:13.768 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:13 smithi190 ceph-mon[22001]: Deploying daemon osd.2 on smithi190 2024-08-30T08:53:13.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:13 smithi167 ceph-mon[22063]: Deploying daemon osd.2 on smithi190 2024-08-30T08:53:14.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:13 smithi042 ceph-mon[18652]: Deploying daemon osd.2 on smithi190 2024-08-30T08:53:15.067 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:14 smithi190 ceph-mon[22001]: pgmap v79: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-08-30T08:53:15.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:14 smithi167 ceph-mon[22063]: pgmap v79: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-08-30T08:53:15.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:14 smithi042 ceph-mon[18652]: pgmap v79: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-08-30T08:53:16.782 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:16 smithi190 ceph-mon[22001]: pgmap v80: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-08-30T08:53:17.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:16 smithi167 ceph-mon[22063]: pgmap v80: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-08-30T08:53:17.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:16 smithi042 ceph-mon[18652]: pgmap v80: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-08-30T08:53:18.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:17 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-30T08:53:18.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:17 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:53:18.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:17 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:53:18.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:17 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-30T08:53:18.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:17 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:53:18.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:17 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:53:18.140 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:17 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-30T08:53:18.140 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:17 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:53:18.140 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:17 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:53:19.024 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:18 smithi190 ceph-mon[22001]: pgmap v81: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-08-30T08:53:19.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:18 smithi167 ceph-mon[22063]: pgmap v81: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-08-30T08:53:19.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:18 smithi042 ceph-mon[18652]: pgmap v81: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-08-30T08:53:20.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:20 smithi167 ceph-mon[22063]: pgmap v82: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-08-30T08:53:20.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:20 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:53:20.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:20 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:53:20.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:20 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:53:20.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:20 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-30T08:53:20.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:20 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:53:20.882 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:20 smithi042 ceph-mon[18652]: pgmap v82: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-08-30T08:53:20.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:20 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:53:20.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:20 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:53:20.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:20 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:53:20.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:20 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-30T08:53:20.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:20 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:53:20.923 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:20 smithi190 ceph-mon[22001]: pgmap v82: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-08-30T08:53:20.923 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:20 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:53:20.924 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:20 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:53:20.924 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:20 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:53:20.924 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:20 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-30T08:53:20.924 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:20 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:53:21.250 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:53:21.248+0000 7fbd18ff9640 1 -- 172.21.15.190:0/3834170667 <== mgr.14150 v2:172.21.15.42:6800/3005140902 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fbd1c0635a0 con 0x7fbd040b4aa0 2024-08-30T08:53:21.250 INFO:teuthology.orchestra.run.smithi190.stdout:Created osd(s) 2 on host 'smithi190' 2024-08-30T08:53:21.253 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:53:21.251+0000 7fbd21d58640 1 -- 172.21.15.190:0/3834170667 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fbd040b4aa0 msgr2=0x7fbd040b6fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:53:21.253 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:53:21.251+0000 7fbd21d58640 1 --2- 172.21.15.190:0/3834170667 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fbd040b4aa0 0x7fbd040b6fb0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fbd10004770 tx=0x7fbd100314a0 comp rx=0 tx=0).stop 2024-08-30T08:53:21.253 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:53:21.251+0000 7fbd21d58640 1 -- 172.21.15.190:0/3834170667 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbd1c103e10 msgr2=0x7fbd1c1a33e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:53:21.253 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:53:21.251+0000 7fbd21d58640 1 --2- 172.21.15.190:0/3834170667 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbd1c103e10 0x7fbd1c1a33e0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fbd1400eb10 tx=0x7fbd1400ef10 comp rx=0 tx=0).stop 2024-08-30T08:53:21.253 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:53:21.251+0000 7fbd21d58640 1 -- 172.21.15.190:0/3834170667 shutdown_connections 2024-08-30T08:53:21.253 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:53:21.251+0000 7fbd21d58640 1 --2- 172.21.15.190:0/3834170667 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fbd040b4aa0 0x7fbd040b6fb0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:53:21.253 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:53:21.251+0000 7fbd21d58640 1 --2- 172.21.15.190:0/3834170667 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbd1c103e10 0x7fbd1c1a33e0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:53:21.253 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:53:21.251+0000 7fbd21d58640 1 --2- 172.21.15.190:0/3834170667 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbd1c103470 0x7fbd1c1a0e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:53:21.253 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:53:21.251+0000 7fbd21d58640 1 --2- 172.21.15.190:0/3834170667 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbd1c1023e0 0x7fbd1c06b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:53:21.253 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:53:21.251+0000 7fbd21d58640 1 -- 172.21.15.190:0/3834170667 >> 172.21.15.190:0/3834170667 conn(0x7fbd1c0fdd50 msgr2=0x7fbd1c106790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:53:21.254 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:53:21.251+0000 7fbd21d58640 1 -- 172.21.15.190:0/3834170667 shutdown_connections 2024-08-30T08:53:21.254 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:53:21.251+0000 7fbd21d58640 1 -- 172.21.15.190:0/3834170667 wait complete. 2024-08-30T08:53:21.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:21 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-30T08:53:21.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:21 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:53:21.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:21 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:53:21.861 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:21 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-30T08:53:21.861 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:21 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:53:21.861 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:21 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:53:21.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:21 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-30T08:53:21.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:21 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:53:21.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:21 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:53:21.887 DEBUG:teuthology.orchestra.run.smithi190:osd.2> sudo journalctl -f -n 0 -u ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968@osd.2.service 2024-08-30T08:53:21.929 INFO:tasks.cephadm:Deploying osd.3 on smithi042 with /dev/vg_nvme/lv_3... 2024-08-30T08:53:21.929 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-08-30T08:53:22.192 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T08:53:22.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:22 smithi042 ceph-mon[18652]: pgmap v83: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-08-30T08:53:22.813 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:22 smithi190 ceph-mon[22001]: pgmap v83: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-08-30T08:53:22.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:22 smithi167 ceph-mon[22063]: pgmap v83: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-08-30T08:53:23.089 INFO:journalctl@ceph.osd.2.smithi190.stdout:Aug 30 08:53:22 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-2[26986]: 2024-08-30T08:53:22.810+0000 7f4be8c14640 -1 osd.2 0 log_to_monitors true 2024-08-30T08:53:23.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:23 smithi190 ceph-mon[22001]: from='osd.2 [v2:172.21.15.190:6800/1101326404,v1:172.21.15.190:6801/1101326404]' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-08-30T08:53:23.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:23 smithi190 ceph-mon[22001]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-08-30T08:53:23.763 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:23 smithi042 ceph-mon[18652]: from='osd.2 [v2:172.21.15.190:6800/1101326404,v1:172.21.15.190:6801/1101326404]' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-08-30T08:53:23.763 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:23 smithi042 ceph-mon[18652]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-08-30T08:53:23.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:23 smithi167 ceph-mon[22063]: from='osd.2 [v2:172.21.15.190:6800/1101326404,v1:172.21.15.190:6801/1101326404]' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-08-30T08:53:23.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:23 smithi167 ceph-mon[22063]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-08-30T08:53:24.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:24 smithi190 ceph-mon[22001]: pgmap v84: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-08-30T08:53:24.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:24 smithi190 ceph-mon[22001]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-08-30T08:53:24.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:24 smithi190 ceph-mon[22001]: from='osd.2 [v2:172.21.15.190:6800/1101326404,v1:172.21.15.190:6801/1101326404]' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi190", "root=default"]} : dispatch 2024-08-30T08:53:24.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:24 smithi190 ceph-mon[22001]: osdmap e16: 3 total, 2 up, 3 in 2024-08-30T08:53:24.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:24 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-08-30T08:53:24.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:24 smithi190 ceph-mon[22001]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi190", "root=default"]} : dispatch 2024-08-30T08:53:24.704 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:24 smithi042 ceph-mon[18652]: pgmap v84: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-08-30T08:53:24.704 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:24 smithi042 ceph-mon[18652]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-08-30T08:53:24.704 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:24 smithi042 ceph-mon[18652]: from='osd.2 [v2:172.21.15.190:6800/1101326404,v1:172.21.15.190:6801/1101326404]' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi190", "root=default"]} : dispatch 2024-08-30T08:53:24.704 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:24 smithi042 ceph-mon[18652]: osdmap e16: 3 total, 2 up, 3 in 2024-08-30T08:53:24.704 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:24 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-08-30T08:53:24.704 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:24 smithi042 ceph-mon[18652]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi190", "root=default"]} : dispatch 2024-08-30T08:53:24.706 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-08-30T08:53:24.733 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph orch daemon add osd smithi042:vg_nvme/lv_3 2024-08-30T08:53:24.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:24 smithi167 ceph-mon[22063]: pgmap v84: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-08-30T08:53:24.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:24 smithi167 ceph-mon[22063]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-08-30T08:53:24.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:24 smithi167 ceph-mon[22063]: from='osd.2 [v2:172.21.15.190:6800/1101326404,v1:172.21.15.190:6801/1101326404]' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi190", "root=default"]} : dispatch 2024-08-30T08:53:24.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:24 smithi167 ceph-mon[22063]: osdmap e16: 3 total, 2 up, 3 in 2024-08-30T08:53:24.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:24 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-08-30T08:53:24.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:24 smithi167 ceph-mon[22063]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi190", "root=default"]} : dispatch 2024-08-30T08:53:25.018 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T08:53:25.566 INFO:journalctl@ceph.osd.2.smithi190.stdout:Aug 30 08:53:25 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-2[26986]: 2024-08-30T08:53:25.285+0000 7f4be4b96640 -1 osd.2 0 waiting for initial osdmap 2024-08-30T08:53:25.566 INFO:journalctl@ceph.osd.2.smithi190.stdout:Aug 30 08:53:25 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-2[26986]: 2024-08-30T08:53:25.290+0000 7f4bdc98c640 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-30T08:53:25.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:53:25.584+0000 7f0140ef1640 1 -- 172.21.15.42:0/2131457934 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f013c103e20 msgr2=0x7f013c10a7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:53:25.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:53:25.584+0000 7f0140ef1640 1 --2- 172.21.15.42:0/2131457934 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f013c103e20 0x7f013c10a7c0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f0134009900 tx=0x7f013402a480 comp rx=0 tx=0).stop 2024-08-30T08:53:25.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:53:25.585+0000 7f0140ef1640 1 -- 172.21.15.42:0/2131457934 shutdown_connections 2024-08-30T08:53:25.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:53:25.585+0000 7f0140ef1640 1 --2- 172.21.15.42:0/2131457934 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f013c103e20 0x7f013c10a7c0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:53:25.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:53:25.585+0000 7f0140ef1640 1 --2- 172.21.15.42:0/2131457934 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f013c103480 0x7f013c103890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:53:25.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:53:25.585+0000 7f0140ef1640 1 --2- 172.21.15.42:0/2131457934 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f013c1023f0 0x7f013c102860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:53:25.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:53:25.585+0000 7f0140ef1640 1 -- 172.21.15.42:0/2131457934 >> 172.21.15.42:0/2131457934 conn(0x7f013c0fdd80 msgr2=0x7f013c100210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:53:25.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:53:25.585+0000 7f0140ef1640 1 -- 172.21.15.42:0/2131457934 shutdown_connections 2024-08-30T08:53:25.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:53:25.585+0000 7f0140ef1640 1 -- 172.21.15.42:0/2131457934 wait complete. 2024-08-30T08:53:25.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:53:25.585+0000 7f0140ef1640 1 Processor -- start 2024-08-30T08:53:25.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:53:25.585+0000 7f0140ef1640 1 -- start start 2024-08-30T08:53:25.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:53:25.586+0000 7f0140ef1640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f013c1023f0 0x7f013c06c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:53:25.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:53:25.586+0000 7f0140ef1640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f013c103480 0x7f013c19ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:53:25.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:53:25.586+0000 7f0140ef1640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f013c103e20 0x7f013c1a0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:53:25.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:53:25.586+0000 7f0140ef1640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f013c10e050 con 0x7f013c1023f0 2024-08-30T08:53:25.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:53:25.586+0000 7f0140ef1640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f013c10ded0 con 0x7f013c103e20 2024-08-30T08:53:25.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:53:25.586+0000 7f0140ef1640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f013c10e1d0 con 0x7f013c103480 2024-08-30T08:53:25.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:53:25.586+0000 7f013a59c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f013c1023f0 0x7f013c06c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:53:25.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:53:25.586+0000 7f013a59c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f013c1023f0 0x7f013c06c1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60032/0 (socket says 172.21.15.42:60032) 2024-08-30T08:53:25.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:53:25.586+0000 7f013a59c640 1 -- 172.21.15.42:0/1355384392 learned_addr learned my addr 172.21.15.42:0/1355384392 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T08:53:25.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:53:25.586+0000 7f0139d9b640 1 --2- 172.21.15.42:0/1355384392 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f013c103480 0x7f013c19ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:53:25.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:53:25.586+0000 7f013ad9d640 1 --2- 172.21.15.42:0/1355384392 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f013c103e20 0x7f013c1a0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:53:25.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:53:25.586+0000 7f013a59c640 1 -- 172.21.15.42:0/1355384392 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f013c103480 msgr2=0x7f013c19ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:53:25.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:53:25.586+0000 7f013a59c640 1 --2- 172.21.15.42:0/1355384392 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f013c103480 0x7f013c19ea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:53:25.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:53:25.586+0000 7f013a59c640 1 -- 172.21.15.42:0/1355384392 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f013c103e20 msgr2=0x7f013c1a0fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:53:25.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:53:25.586+0000 7f013a59c640 1 --2- 172.21.15.42:0/1355384392 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f013c103e20 0x7f013c1a0fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:53:25.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:53:25.586+0000 7f013a59c640 1 -- 172.21.15.42:0/1355384392 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f013c1a17c0 con 0x7f013c1023f0 2024-08-30T08:53:25.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:53:25.586+0000 7f013ad9d640 1 --2- 172.21.15.42:0/1355384392 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f013c103e20 0x7f013c1a0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-30T08:53:25.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:53:25.586+0000 7f013a59c640 1 --2- 172.21.15.42:0/1355384392 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f013c1023f0 0x7f013c06c1a0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f012c00cce0 tx=0x7f012c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:53:25.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:53:25.587+0000 7f012b7fe640 1 -- 172.21.15.42:0/1355384392 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f012c004b20 con 0x7f013c1023f0 2024-08-30T08:53:25.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:53:25.587+0000 7f012b7fe640 1 -- 172.21.15.42:0/1355384392 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f012c004cc0 con 0x7f013c1023f0 2024-08-30T08:53:25.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:53:25.587+0000 7f0140ef1640 1 -- 172.21.15.42:0/1355384392 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f013c1a1ab0 con 0x7f013c1023f0 2024-08-30T08:53:25.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:53:25.587+0000 7f012b7fe640 1 -- 172.21.15.42:0/1355384392 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f012c00f7c0 con 0x7f013c1023f0 2024-08-30T08:53:25.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:53:25.587+0000 7f0140ef1640 1 -- 172.21.15.42:0/1355384392 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f013c1a1ff0 con 0x7f013c1023f0 2024-08-30T08:53:25.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:53:25.588+0000 7f0140ef1640 1 -- 172.21.15.42:0/1355384392 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f013c1095d0 con 0x7f013c1023f0 2024-08-30T08:53:25.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:53:25.589+0000 7f012b7fe640 1 -- 172.21.15.42:0/1355384392 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 15) ==== 152521+0+0 (secure 0 0 0) 0x7f012c005560 con 0x7f013c1023f0 2024-08-30T08:53:25.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:53:25.590+0000 7f012b7fe640 1 --2- 172.21.15.42:0/1355384392 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f01240b4b20 0x7f01240b7030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:53:25.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:53:25.590+0000 7f0139d9b640 1 --2- 172.21.15.42:0/1355384392 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f01240b4b20 0x7f01240b7030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:53:25.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:53:25.590+0000 7f012b7fe640 1 -- 172.21.15.42:0/1355384392 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 2571+0+0 (secure 0 0 0) 0x7f012c0e3a00 con 0x7f013c1023f0 2024-08-30T08:53:25.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:53:25.590+0000 7f0139d9b640 1 --2- 172.21.15.42:0/1355384392 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f01240b4b20 0x7f01240b7030 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f01300058c0 tx=0x7f013000a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:53:25.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:53:25.594+0000 7f012b7fe640 1 -- 172.21.15.42:0/1355384392 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f012c0e5e20 con 0x7f013c1023f0 2024-08-30T08:53:25.792 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:53:25.790+0000 7f0140ef1640 1 -- 172.21.15.42:0/1355384392 --> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi042:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f013c0635a0 con 0x7f01240b4b20 2024-08-30T08:53:25.793 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:25 smithi042 ceph-mon[18652]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]': finished 2024-08-30T08:53:25.793 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:25 smithi042 ceph-mon[18652]: osdmap e17: 3 total, 2 up, 3 in 2024-08-30T08:53:25.793 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:25 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-08-30T08:53:25.793 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:25 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-08-30T08:53:25.793 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:25 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:53:25.793 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:25 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:53:25.793 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:25 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-08-30T08:53:25.793 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:25 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:53:25.793 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:25 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:53:25.793 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:25 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-30T08:53:25.794 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:25 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:53:25.794 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:25 smithi042 ceph-mon[18652]: from='osd.2 ' entity='osd.2' 2024-08-30T08:53:25.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:25 smithi167 ceph-mon[22063]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]': finished 2024-08-30T08:53:25.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:25 smithi167 ceph-mon[22063]: osdmap e17: 3 total, 2 up, 3 in 2024-08-30T08:53:25.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:25 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-08-30T08:53:25.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:25 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-08-30T08:53:25.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:25 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:53:25.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:25 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:53:25.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:25 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-08-30T08:53:25.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:25 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:53:25.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:25 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:53:25.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:25 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-30T08:53:25.828 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:25 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:53:25.828 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:25 smithi167 ceph-mon[22063]: from='osd.2 ' entity='osd.2' 2024-08-30T08:53:25.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:25 smithi190 ceph-mon[22001]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]': finished 2024-08-30T08:53:25.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:25 smithi190 ceph-mon[22001]: osdmap e17: 3 total, 2 up, 3 in 2024-08-30T08:53:25.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:25 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-08-30T08:53:25.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:25 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-08-30T08:53:25.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:25 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:53:25.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:25 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:53:25.937 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:25 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-08-30T08:53:25.937 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:25 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:53:25.937 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:25 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:53:25.937 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:25 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-30T08:53:25.937 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:25 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:53:25.937 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:25 smithi190 ceph-mon[22001]: from='osd.2 ' entity='osd.2' 2024-08-30T08:53:26.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:26 smithi167 ceph-mon[22063]: purged_snaps scrub starts 2024-08-30T08:53:26.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:26 smithi167 ceph-mon[22063]: purged_snaps scrub ok 2024-08-30T08:53:26.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:26 smithi167 ceph-mon[22063]: pgmap v87: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-08-30T08:53:26.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:26 smithi167 ceph-mon[22063]: Detected new or changed devices on smithi190 2024-08-30T08:53:26.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:26 smithi167 ceph-mon[22063]: Adjusting osd_memory_target on smithi190 to 17088M 2024-08-30T08:53:26.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:26 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-08-30T08:53:26.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:26 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-30T08:53:26.828 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:26 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-30T08:53:26.828 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:26 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:53:26.828 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:26 smithi167 ceph-mon[22063]: osd.2 [v2:172.21.15.190:6800/1101326404,v1:172.21.15.190:6801/1101326404] boot 2024-08-30T08:53:26.828 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:26 smithi167 ceph-mon[22063]: osdmap e18: 3 total, 3 up, 3 in 2024-08-30T08:53:26.828 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:26 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-08-30T08:53:26.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:26 smithi042 ceph-mon[18652]: purged_snaps scrub starts 2024-08-30T08:53:26.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:26 smithi042 ceph-mon[18652]: purged_snaps scrub ok 2024-08-30T08:53:26.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:26 smithi042 ceph-mon[18652]: pgmap v87: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-08-30T08:53:26.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:26 smithi042 ceph-mon[18652]: Detected new or changed devices on smithi190 2024-08-30T08:53:26.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:26 smithi042 ceph-mon[18652]: Adjusting osd_memory_target on smithi190 to 17088M 2024-08-30T08:53:26.884 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:26 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-08-30T08:53:26.884 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:26 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-30T08:53:26.884 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:26 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-30T08:53:26.884 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:26 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:53:26.884 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:26 smithi042 ceph-mon[18652]: osd.2 [v2:172.21.15.190:6800/1101326404,v1:172.21.15.190:6801/1101326404] boot 2024-08-30T08:53:26.884 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:26 smithi042 ceph-mon[18652]: osdmap e18: 3 total, 3 up, 3 in 2024-08-30T08:53:26.884 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:26 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-08-30T08:53:26.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:26 smithi190 ceph-mon[22001]: purged_snaps scrub starts 2024-08-30T08:53:26.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:26 smithi190 ceph-mon[22001]: purged_snaps scrub ok 2024-08-30T08:53:26.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:26 smithi190 ceph-mon[22001]: pgmap v87: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-08-30T08:53:26.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:26 smithi190 ceph-mon[22001]: Detected new or changed devices on smithi190 2024-08-30T08:53:26.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:26 smithi190 ceph-mon[22001]: Adjusting osd_memory_target on smithi190 to 17088M 2024-08-30T08:53:26.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:26 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-08-30T08:53:26.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:26 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-30T08:53:26.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:26 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-30T08:53:26.937 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:26 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:53:26.937 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:26 smithi190 ceph-mon[22001]: osd.2 [v2:172.21.15.190:6800/1101326404,v1:172.21.15.190:6801/1101326404] boot 2024-08-30T08:53:26.937 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:26 smithi190 ceph-mon[22001]: osdmap e18: 3 total, 3 up, 3 in 2024-08-30T08:53:26.937 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:26 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-08-30T08:53:27.810 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:27 smithi042 ceph-mon[18652]: from='client.14433 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi042:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-08-30T08:53:27.810 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:27 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true} : dispatch 2024-08-30T08:53:27.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:27 smithi167 ceph-mon[22063]: from='client.14433 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi042:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-08-30T08:53:27.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:27 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true} : dispatch 2024-08-30T08:53:27.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:27 smithi190 ceph-mon[22001]: from='client.14433 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi042:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-08-30T08:53:27.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:27 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true} : dispatch 2024-08-30T08:53:28.750 INFO:journalctl@ceph.osd.2.smithi190.stdout:Aug 30 08:53:28 smithi190 sudo[31056]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-08-30T08:53:28.750 INFO:journalctl@ceph.osd.2.smithi190.stdout:Aug 30 08:53:28 smithi190 sudo[31056]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-08-30T08:53:28.750 INFO:journalctl@ceph.osd.2.smithi190.stdout:Aug 30 08:53:28 smithi190 sudo[31056]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-08-30T08:53:28.750 INFO:journalctl@ceph.osd.2.smithi190.stdout:Aug 30 08:53:28 smithi190 sudo[31056]: pam_unix(sudo:session): session closed for user root 2024-08-30T08:53:28.750 INFO:journalctl@ceph.osd.2.smithi190.stdout:Aug 30 08:53:28 smithi190 sudo[31059]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-08-30T08:53:28.750 INFO:journalctl@ceph.osd.2.smithi190.stdout:Aug 30 08:53:28 smithi190 sudo[31059]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-08-30T08:53:28.750 INFO:journalctl@ceph.osd.2.smithi190.stdout:Aug 30 08:53:28 smithi190 sudo[31059]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-08-30T08:53:28.751 INFO:journalctl@ceph.osd.2.smithi190.stdout:Aug 30 08:53:28 smithi190 sudo[31059]: pam_unix(sudo:session): session closed for user root 2024-08-30T08:53:28.751 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:28 smithi190 ceph-mon[22001]: pgmap v89: 0 pgs: ; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-08-30T08:53:28.751 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:28 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-08-30T08:53:28.751 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:28 smithi190 ceph-mon[22001]: osdmap e19: 3 total, 3 up, 3 in 2024-08-30T08:53:28.751 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:28 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true} : dispatch 2024-08-30T08:53:28.751 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:28 smithi190 ceph-mon[22001]: from='client.? 172.21.15.42:0/1227457398' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "87bb3e36-9e9e-4b19-9d8c-0cdf6e089d3e"} : dispatch 2024-08-30T08:53:28.751 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:28 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-08-30T08:53:28.751 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:28 smithi190 ceph-mon[22001]: from='client.? 172.21.15.42:0/1227457398' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "87bb3e36-9e9e-4b19-9d8c-0cdf6e089d3e"}]': finished 2024-08-30T08:53:28.751 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:28 smithi190 ceph-mon[22001]: osdmap e20: 4 total, 3 up, 4 in 2024-08-30T08:53:28.751 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:28 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-08-30T08:53:28.751 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:28 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:53:28.751 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:28 smithi190 sudo[31062]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-08-30T08:53:28.780 INFO:journalctl@ceph.osd.0.smithi042.stdout:Aug 30 08:53:28 smithi042 sudo[39899]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-08-30T08:53:28.780 INFO:journalctl@ceph.osd.0.smithi042.stdout:Aug 30 08:53:28 smithi042 sudo[39899]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-08-30T08:53:28.780 INFO:journalctl@ceph.osd.0.smithi042.stdout:Aug 30 08:53:28 smithi042 sudo[39899]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-08-30T08:53:28.781 INFO:journalctl@ceph.osd.0.smithi042.stdout:Aug 30 08:53:28 smithi042 sudo[39899]: pam_unix(sudo:session): session closed for user root 2024-08-30T08:53:28.781 INFO:journalctl@ceph.osd.0.smithi042.stdout:Aug 30 08:53:28 smithi042 sudo[39902]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-08-30T08:53:28.781 INFO:journalctl@ceph.osd.0.smithi042.stdout:Aug 30 08:53:28 smithi042 sudo[39902]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-08-30T08:53:28.781 INFO:journalctl@ceph.osd.0.smithi042.stdout:Aug 30 08:53:28 smithi042 sudo[39902]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-08-30T08:53:28.781 INFO:journalctl@ceph.osd.0.smithi042.stdout:Aug 30 08:53:28 smithi042 sudo[39902]: pam_unix(sudo:session): session closed for user root 2024-08-30T08:53:28.781 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:28 smithi042 ceph-mon[18652]: pgmap v89: 0 pgs: ; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-08-30T08:53:28.781 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:28 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-08-30T08:53:28.781 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:28 smithi042 ceph-mon[18652]: osdmap e19: 3 total, 3 up, 3 in 2024-08-30T08:53:28.781 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:28 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true} : dispatch 2024-08-30T08:53:28.781 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:28 smithi042 ceph-mon[18652]: from='client.? 172.21.15.42:0/1227457398' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "87bb3e36-9e9e-4b19-9d8c-0cdf6e089d3e"} : dispatch 2024-08-30T08:53:28.781 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:28 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-08-30T08:53:28.781 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:28 smithi042 ceph-mon[18652]: from='client.? 172.21.15.42:0/1227457398' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "87bb3e36-9e9e-4b19-9d8c-0cdf6e089d3e"}]': finished 2024-08-30T08:53:28.782 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:28 smithi042 ceph-mon[18652]: osdmap e20: 4 total, 3 up, 4 in 2024-08-30T08:53:28.782 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:28 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-08-30T08:53:28.782 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:28 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:53:28.782 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:28 smithi042 sudo[39911]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-08-30T08:53:28.782 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:28 smithi042 sudo[39911]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-08-30T08:53:28.782 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:28 smithi042 sudo[39911]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-08-30T08:53:28.782 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:28 smithi042 sudo[39911]: pam_unix(sudo:session): session closed for user root 2024-08-30T08:53:28.782 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:28 smithi042 sudo[39914]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-08-30T08:53:28.782 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:28 smithi042 sudo[39914]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-08-30T08:53:28.782 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:28 smithi042 sudo[39914]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-08-30T08:53:28.782 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:28 smithi042 sudo[39914]: pam_unix(sudo:session): session closed for user root 2024-08-30T08:53:28.827 INFO:journalctl@ceph.osd.1.smithi167.stdout:Aug 30 08:53:28 smithi167 sudo[30636]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-08-30T08:53:28.827 INFO:journalctl@ceph.osd.1.smithi167.stdout:Aug 30 08:53:28 smithi167 sudo[30636]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-08-30T08:53:28.827 INFO:journalctl@ceph.osd.1.smithi167.stdout:Aug 30 08:53:28 smithi167 sudo[30636]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-08-30T08:53:28.827 INFO:journalctl@ceph.osd.1.smithi167.stdout:Aug 30 08:53:28 smithi167 sudo[30636]: pam_unix(sudo:session): session closed for user root 2024-08-30T08:53:28.827 INFO:journalctl@ceph.osd.1.smithi167.stdout:Aug 30 08:53:28 smithi167 sudo[30639]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-08-30T08:53:28.827 INFO:journalctl@ceph.osd.1.smithi167.stdout:Aug 30 08:53:28 smithi167 sudo[30639]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-08-30T08:53:28.827 INFO:journalctl@ceph.osd.1.smithi167.stdout:Aug 30 08:53:28 smithi167 sudo[30639]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-08-30T08:53:28.827 INFO:journalctl@ceph.osd.1.smithi167.stdout:Aug 30 08:53:28 smithi167 sudo[30639]: pam_unix(sudo:session): session closed for user root 2024-08-30T08:53:28.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:28 smithi167 ceph-mon[22063]: pgmap v89: 0 pgs: ; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-08-30T08:53:28.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:28 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-08-30T08:53:28.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:28 smithi167 ceph-mon[22063]: osdmap e19: 3 total, 3 up, 3 in 2024-08-30T08:53:28.828 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:28 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true} : dispatch 2024-08-30T08:53:28.828 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:28 smithi167 ceph-mon[22063]: from='client.? 172.21.15.42:0/1227457398' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "87bb3e36-9e9e-4b19-9d8c-0cdf6e089d3e"} : dispatch 2024-08-30T08:53:28.828 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:28 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-08-30T08:53:28.828 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:28 smithi167 ceph-mon[22063]: from='client.? 172.21.15.42:0/1227457398' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "87bb3e36-9e9e-4b19-9d8c-0cdf6e089d3e"}]': finished 2024-08-30T08:53:28.828 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:28 smithi167 ceph-mon[22063]: osdmap e20: 4 total, 3 up, 4 in 2024-08-30T08:53:28.828 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:28 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-08-30T08:53:28.828 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:28 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:53:28.828 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:28 smithi167 sudo[30642]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-08-30T08:53:28.828 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:28 smithi167 sudo[30642]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-08-30T08:53:28.828 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:28 smithi167 sudo[30642]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-08-30T08:53:28.828 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:28 smithi167 sudo[30642]: pam_unix(sudo:session): session closed for user root 2024-08-30T08:53:28.828 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:28 smithi167 sudo[30645]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-08-30T08:53:28.828 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:28 smithi167 sudo[30645]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-08-30T08:53:28.828 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:28 smithi167 sudo[30645]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-08-30T08:53:28.829 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:28 smithi167 sudo[30645]: pam_unix(sudo:session): session closed for user root 2024-08-30T08:53:29.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:28 smithi190 sudo[31062]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-08-30T08:53:29.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:28 smithi190 sudo[31062]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-08-30T08:53:29.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:28 smithi190 sudo[31062]: pam_unix(sudo:session): session closed for user root 2024-08-30T08:53:29.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:28 smithi190 sudo[31065]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-08-30T08:53:29.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:28 smithi190 sudo[31065]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-08-30T08:53:29.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:28 smithi190 sudo[31065]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-08-30T08:53:29.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:28 smithi190 sudo[31065]: pam_unix(sudo:session): session closed for user root 2024-08-30T08:53:29.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:53:29.707+0000 7f012b7fe640 1 -- 172.21.15.42:0/1355384392 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mgrmap(e 16) ==== 152577+0+0 (secure 0 0 0) 0x7f012c0a7e10 con 0x7f013c1023f0 2024-08-30T08:53:29.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:29 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:53:29.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:29 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:53:29.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:29 smithi042 ceph-mon[18652]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-08-30T08:53:29.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:29 smithi042 ceph-mon[18652]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-08-30T08:53:29.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:29 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-08-30T08:53:29.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:29 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-30T08:53:29.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:29 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-30T08:53:29.884 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:29 smithi042 ceph-mon[18652]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-08-30T08:53:29.884 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:29 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-08-30T08:53:29.884 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:29 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-30T08:53:29.884 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:29 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-30T08:53:29.884 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:29 smithi042 ceph-mon[18652]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-08-30T08:53:29.884 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:29 smithi042 ceph-mon[18652]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-08-30T08:53:29.884 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:29 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-08-30T08:53:29.884 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:29 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-30T08:53:29.884 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:29 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-30T08:53:29.884 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:29 smithi042 ceph-mon[18652]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-08-30T08:53:29.884 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:29 smithi042 ceph-mon[18652]: from='client.? 172.21.15.42:0/1230862342' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-30T08:53:29.884 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:29 smithi042 ceph-mon[18652]: osdmap e21: 4 total, 3 up, 4 in 2024-08-30T08:53:29.884 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:29 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-08-30T08:53:29.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:29 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:53:29.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:29 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:53:29.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:29 smithi190 ceph-mon[22001]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-08-30T08:53:29.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:29 smithi190 ceph-mon[22001]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-08-30T08:53:29.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:29 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-08-30T08:53:29.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:29 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-30T08:53:29.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:29 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-30T08:53:29.937 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:29 smithi190 ceph-mon[22001]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-08-30T08:53:29.937 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:29 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-08-30T08:53:29.937 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:29 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-30T08:53:29.937 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:29 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-30T08:53:29.937 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:29 smithi190 ceph-mon[22001]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-08-30T08:53:29.937 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:29 smithi190 ceph-mon[22001]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-08-30T08:53:29.937 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:29 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-08-30T08:53:29.937 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:29 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-30T08:53:29.937 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:29 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-30T08:53:29.937 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:29 smithi190 ceph-mon[22001]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-08-30T08:53:29.937 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:29 smithi190 ceph-mon[22001]: from='client.? 172.21.15.42:0/1230862342' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-30T08:53:29.937 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:29 smithi190 ceph-mon[22001]: osdmap e21: 4 total, 3 up, 4 in 2024-08-30T08:53:29.937 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:29 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-08-30T08:53:30.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:29 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:53:30.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:29 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:53:30.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:29 smithi167 ceph-mon[22063]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-08-30T08:53:30.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:29 smithi167 ceph-mon[22063]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-08-30T08:53:30.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:29 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-08-30T08:53:30.078 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:29 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-30T08:53:30.078 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:29 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-30T08:53:30.078 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:29 smithi167 ceph-mon[22063]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-08-30T08:53:30.078 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:29 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-08-30T08:53:30.078 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:29 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-30T08:53:30.078 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:29 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-30T08:53:30.078 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:29 smithi167 ceph-mon[22063]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-08-30T08:53:30.078 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:29 smithi167 ceph-mon[22063]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-08-30T08:53:30.078 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:29 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-08-30T08:53:30.078 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:29 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-30T08:53:30.078 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:29 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-30T08:53:30.078 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:29 smithi167 ceph-mon[22063]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-08-30T08:53:30.078 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:29 smithi167 ceph-mon[22063]: from='client.? 172.21.15.42:0/1230862342' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-30T08:53:30.078 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:29 smithi167 ceph-mon[22063]: osdmap e21: 4 total, 3 up, 4 in 2024-08-30T08:53:30.078 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:29 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-08-30T08:53:30.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:30 smithi042 ceph-mon[18652]: pgmap v92: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-08-30T08:53:30.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:30 smithi042 ceph-mon[18652]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-08-30T08:53:30.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:30 smithi190 ceph-mon[22001]: pgmap v92: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-08-30T08:53:30.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:30 smithi190 ceph-mon[22001]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-08-30T08:53:31.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:30 smithi167 ceph-mon[22063]: pgmap v92: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-08-30T08:53:31.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:30 smithi167 ceph-mon[22063]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-08-30T08:53:32.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:32 smithi167 ceph-mon[22063]: pgmap v94: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-08-30T08:53:32.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:32 smithi190 ceph-mon[22001]: pgmap v94: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-08-30T08:53:33.132 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:32 smithi042 ceph-mon[18652]: pgmap v94: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-08-30T08:53:34.919 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:34 smithi042 ceph-mon[18652]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 89 MiB used, 268 GiB / 268 GiB avail 2024-08-30T08:53:34.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:34 smithi190 ceph-mon[22001]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 89 MiB used, 268 GiB / 268 GiB avail 2024-08-30T08:53:35.051 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:34 smithi167 ceph-mon[22063]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 89 MiB used, 268 GiB / 268 GiB avail 2024-08-30T08:53:36.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:36 smithi042 ceph-mon[18652]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-08-30T08:53:36.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:36 smithi190 ceph-mon[22001]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-08-30T08:53:37.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:36 smithi167 ceph-mon[22063]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-08-30T08:53:37.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:37 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.3"} : dispatch 2024-08-30T08:53:37.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:37 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:53:37.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:37 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.3"} : dispatch 2024-08-30T08:53:37.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:37 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:53:38.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:37 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.3"} : dispatch 2024-08-30T08:53:38.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:37 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:53:38.805 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:38 smithi042 ceph-mon[18652]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-08-30T08:53:38.805 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:38 smithi042 ceph-mon[18652]: Deploying daemon osd.3 on smithi042 2024-08-30T08:53:38.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:38 smithi190 ceph-mon[22001]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-08-30T08:53:38.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:38 smithi190 ceph-mon[22001]: Deploying daemon osd.3 on smithi042 2024-08-30T08:53:39.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:38 smithi167 ceph-mon[22063]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-08-30T08:53:39.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:38 smithi167 ceph-mon[22063]: Deploying daemon osd.3 on smithi042 2024-08-30T08:53:40.933 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:40 smithi042 ceph-mon[18652]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-08-30T08:53:40.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:40 smithi190 ceph-mon[22001]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-08-30T08:53:41.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:40 smithi167 ceph-mon[22063]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-08-30T08:53:42.906 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:42 smithi042 ceph-mon[18652]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-08-30T08:53:42.906 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:42 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-30T08:53:42.906 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:42 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:53:42.906 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:42 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:53:42.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:42 smithi190 ceph-mon[22001]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-08-30T08:53:42.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:42 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-30T08:53:42.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:42 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:53:42.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:42 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:53:43.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:42 smithi167 ceph-mon[22063]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-08-30T08:53:43.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:42 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-30T08:53:43.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:42 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:53:43.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:42 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:53:44.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:44 smithi167 ceph-mon[22063]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-08-30T08:53:44.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:44 smithi190 ceph-mon[22001]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-08-30T08:53:44.983 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:44 smithi042 ceph-mon[18652]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-08-30T08:53:46.177 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:45 smithi042 ceph-mon[18652]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-08-30T08:53:46.177 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:45 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:53:46.178 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:45 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:53:46.178 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:45 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:53:46.178 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:45 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-30T08:53:46.178 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:45 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:53:46.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:45 smithi190 ceph-mon[22001]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-08-30T08:53:46.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:45 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:53:46.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:45 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:53:46.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:45 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:53:46.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:45 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-30T08:53:46.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:45 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:53:46.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:45 smithi167 ceph-mon[22063]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-08-30T08:53:46.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:45 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:53:46.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:45 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:53:46.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:45 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:53:46.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:45 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-30T08:53:46.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:45 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:53:46.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:53:46.539+0000 7f012b7fe640 1 -- 172.21.15.42:0/1355384392 <== mgr.14150 v2:172.21.15.42:6800/3005140902 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f013c0635a0 con 0x7f01240b4b20 2024-08-30T08:53:46.542 INFO:teuthology.orchestra.run.smithi042.stdout:Created osd(s) 3 on host 'smithi042' 2024-08-30T08:53:46.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:53:46.542+0000 7f0140ef1640 1 -- 172.21.15.42:0/1355384392 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f01240b4b20 msgr2=0x7f01240b7030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:53:46.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:53:46.542+0000 7f0140ef1640 1 --2- 172.21.15.42:0/1355384392 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f01240b4b20 0x7f01240b7030 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f01300058c0 tx=0x7f013000a2d0 comp rx=0 tx=0).stop 2024-08-30T08:53:46.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:53:46.542+0000 7f0140ef1640 1 -- 172.21.15.42:0/1355384392 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f013c1023f0 msgr2=0x7f013c06c1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:53:46.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:53:46.542+0000 7f0140ef1640 1 --2- 172.21.15.42:0/1355384392 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f013c1023f0 0x7f013c06c1a0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f012c00cce0 tx=0x7f012c00a620 comp rx=0 tx=0).stop 2024-08-30T08:53:46.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:53:46.542+0000 7f0140ef1640 1 -- 172.21.15.42:0/1355384392 shutdown_connections 2024-08-30T08:53:46.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:53:46.542+0000 7f0140ef1640 1 --2- 172.21.15.42:0/1355384392 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f01240b4b20 0x7f01240b7030 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:53:46.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:53:46.542+0000 7f0140ef1640 1 --2- 172.21.15.42:0/1355384392 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f013c103480 0x7f013c19ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:53:46.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:53:46.542+0000 7f0140ef1640 1 --2- 172.21.15.42:0/1355384392 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f013c103e20 0x7f013c1a0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:53:46.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:53:46.542+0000 7f0140ef1640 1 --2- 172.21.15.42:0/1355384392 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f013c1023f0 0x7f013c06c1a0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:53:46.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:53:46.542+0000 7f0140ef1640 1 -- 172.21.15.42:0/1355384392 >> 172.21.15.42:0/1355384392 conn(0x7f013c0fdd80 msgr2=0x7f013c1067a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:53:46.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:53:46.542+0000 7f0140ef1640 1 -- 172.21.15.42:0/1355384392 shutdown_connections 2024-08-30T08:53:46.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:53:46.542+0000 7f0140ef1640 1 -- 172.21.15.42:0/1355384392 wait complete. 2024-08-30T08:53:47.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:46 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-30T08:53:47.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:46 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:53:47.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:46 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:53:47.185 DEBUG:teuthology.orchestra.run.smithi042:osd.3> sudo journalctl -f -n 0 -u ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968@osd.3.service 2024-08-30T08:53:47.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:46 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-30T08:53:47.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:46 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:53:47.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:46 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:53:47.188 INFO:tasks.cephadm:Deploying osd.4 on smithi167 with /dev/vg_nvme/lv_3... 2024-08-30T08:53:47.188 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-08-30T08:53:47.216 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:46 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-30T08:53:47.216 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:46 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:53:47.216 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:46 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:53:47.445 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.b/config 2024-08-30T08:53:48.357 INFO:journalctl@ceph.osd.3.smithi042.stdout:Aug 30 08:53:48 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-3[41913]: 2024-08-30T08:53:48.283+0000 7f8019430640 -1 osd.3 0 log_to_monitors true 2024-08-30T08:53:48.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:48 smithi042 ceph-mon[18652]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-08-30T08:53:48.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:48 smithi042 ceph-mon[18652]: from='osd.3 [v2:172.21.15.42:6810/1135599916,v1:172.21.15.42:6811/1135599916]' entity='osd.3' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]} : dispatch 2024-08-30T08:53:48.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:48 smithi167 ceph-mon[22063]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-08-30T08:53:48.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:48 smithi167 ceph-mon[22063]: from='osd.3 [v2:172.21.15.42:6810/1135599916,v1:172.21.15.42:6811/1135599916]' entity='osd.3' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]} : dispatch 2024-08-30T08:53:48.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:48 smithi190 ceph-mon[22001]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-08-30T08:53:48.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:48 smithi190 ceph-mon[22001]: from='osd.3 [v2:172.21.15.42:6810/1135599916,v1:172.21.15.42:6811/1135599916]' entity='osd.3' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]} : dispatch 2024-08-30T08:53:49.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:49 smithi042 ceph-mon[18652]: from='osd.3 [v2:172.21.15.42:6810/1135599916,v1:172.21.15.42:6811/1135599916]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-08-30T08:53:49.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:49 smithi042 ceph-mon[18652]: osdmap e22: 4 total, 3 up, 4 in 2024-08-30T08:53:49.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:49 smithi042 ceph-mon[18652]: from='osd.3 [v2:172.21.15.42:6810/1135599916,v1:172.21.15.42:6811/1135599916]' entity='osd.3' cmd={"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi042", "root=default"]} : dispatch 2024-08-30T08:53:49.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:49 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-08-30T08:53:49.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:49 smithi042 ceph-mon[18652]: from='osd.3 [v2:172.21.15.42:6810/1135599916,v1:172.21.15.42:6811/1135599916]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi042", "root=default"]}]': finished 2024-08-30T08:53:49.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:49 smithi042 ceph-mon[18652]: osdmap e23: 4 total, 3 up, 4 in 2024-08-30T08:53:49.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:49 smithi167 ceph-mon[22063]: from='osd.3 [v2:172.21.15.42:6810/1135599916,v1:172.21.15.42:6811/1135599916]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-08-30T08:53:49.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:49 smithi167 ceph-mon[22063]: osdmap e22: 4 total, 3 up, 4 in 2024-08-30T08:53:49.828 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:49 smithi167 ceph-mon[22063]: from='osd.3 [v2:172.21.15.42:6810/1135599916,v1:172.21.15.42:6811/1135599916]' entity='osd.3' cmd={"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi042", "root=default"]} : dispatch 2024-08-30T08:53:49.828 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:49 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-08-30T08:53:49.828 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:49 smithi167 ceph-mon[22063]: from='osd.3 [v2:172.21.15.42:6810/1135599916,v1:172.21.15.42:6811/1135599916]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi042", "root=default"]}]': finished 2024-08-30T08:53:49.828 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:49 smithi167 ceph-mon[22063]: osdmap e23: 4 total, 3 up, 4 in 2024-08-30T08:53:49.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:49 smithi190 ceph-mon[22001]: from='osd.3 [v2:172.21.15.42:6810/1135599916,v1:172.21.15.42:6811/1135599916]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-08-30T08:53:49.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:49 smithi190 ceph-mon[22001]: osdmap e22: 4 total, 3 up, 4 in 2024-08-30T08:53:49.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:49 smithi190 ceph-mon[22001]: from='osd.3 [v2:172.21.15.42:6810/1135599916,v1:172.21.15.42:6811/1135599916]' entity='osd.3' cmd={"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi042", "root=default"]} : dispatch 2024-08-30T08:53:49.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:49 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-08-30T08:53:49.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:49 smithi190 ceph-mon[22001]: from='osd.3 [v2:172.21.15.42:6810/1135599916,v1:172.21.15.42:6811/1135599916]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi042", "root=default"]}]': finished 2024-08-30T08:53:49.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:49 smithi190 ceph-mon[22001]: osdmap e23: 4 total, 3 up, 4 in 2024-08-30T08:53:49.997 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-08-30T08:53:50.023 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph orch daemon add osd smithi167:vg_nvme/lv_3 2024-08-30T08:53:50.317 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.b/config 2024-08-30T08:53:50.383 INFO:journalctl@ceph.osd.3.smithi042.stdout:Aug 30 08:53:50 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-3[41913]: 2024-08-30T08:53:50.131+0000 7f8015bc7640 -1 osd.3 0 waiting for initial osdmap 2024-08-30T08:53:50.383 INFO:journalctl@ceph.osd.3.smithi042.stdout:Aug 30 08:53:50 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-3[41913]: 2024-08-30T08:53:50.137+0000 7f800d1a8640 -1 osd.3 23 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-30T08:53:50.722 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:50 smithi167 ceph-mon[22063]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-08-30T08:53:50.722 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:50 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-08-30T08:53:50.722 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:50 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-08-30T08:53:50.722 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:50 smithi167 ceph-mon[22063]: from='osd.3 [v2:172.21.15.42:6810/1135599916,v1:172.21.15.42:6811/1135599916]' entity='osd.3' 2024-08-30T08:53:50.722 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:50 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:53:50.722 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:50 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:53:50.723 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:50 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-08-30T08:53:50.723 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:50 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-08-30T08:53:50.723 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:50 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:53:50.723 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:50 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:53:50.723 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:50 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-30T08:53:50.723 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:50 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:53:50.723 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:50 smithi167 ceph-mon[22063]: osd.3 [v2:172.21.15.42:6810/1135599916,v1:172.21.15.42:6811/1135599916] boot 2024-08-30T08:53:50.723 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:50 smithi167 ceph-mon[22063]: osdmap e24: 4 total, 4 up, 4 in 2024-08-30T08:53:50.723 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:50 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-08-30T08:53:50.831 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:53:50.814+0000 7fa1869a6640 1 -- 172.21.15.167:0/2980995143 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa180108140 msgr2=0x7fa18010a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:53:50.831 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:53:50.814+0000 7fa1869a6640 1 --2- 172.21.15.167:0/2980995143 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa180108140 0x7fa18010a650 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fa178009ca0 tx=0x7fa17802a8c0 comp rx=0 tx=0).stop 2024-08-30T08:53:50.831 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:53:50.814+0000 7fa1869a6640 1 -- 172.21.15.167:0/2980995143 shutdown_connections 2024-08-30T08:53:50.831 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:53:50.814+0000 7fa1869a6640 1 --2- 172.21.15.167:0/2980995143 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa180108140 0x7fa18010a650 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:53:50.831 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:53:50.814+0000 7fa1869a6640 1 --2- 172.21.15.167:0/2980995143 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa180105520 0x7fa180107a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:53:50.831 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:53:50.814+0000 7fa1869a6640 1 --2- 172.21.15.167:0/2980995143 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa180102ac0 0x7fa180104ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:53:50.831 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:53:50.814+0000 7fa1869a6640 1 -- 172.21.15.167:0/2980995143 >> 172.21.15.167:0/2980995143 conn(0x7fa1800fd500 msgr2=0x7fa1800ff9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:53:50.831 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:53:50.814+0000 7fa1869a6640 1 -- 172.21.15.167:0/2980995143 shutdown_connections 2024-08-30T08:53:50.831 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:53:50.814+0000 7fa1869a6640 1 -- 172.21.15.167:0/2980995143 wait complete. 2024-08-30T08:53:50.832 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:53:50.815+0000 7fa1869a6640 1 Processor -- start 2024-08-30T08:53:50.832 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:53:50.815+0000 7fa1869a6640 1 -- start start 2024-08-30T08:53:50.832 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:53:50.815+0000 7fa1869a6640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa180102ac0 0x7fa180101f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:53:50.832 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:53:50.815+0000 7fa1869a6640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa180105520 0x7fa18019ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:53:50.832 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:53:50.815+0000 7fa1869a6640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa180108140 0x7fa1801a0fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:53:50.832 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:53:50.815+0000 7fa1869a6640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fa18010e060 con 0x7fa180105520 2024-08-30T08:53:50.832 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:53:50.815+0000 7fa1869a6640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa18010dee0 con 0x7fa180108140 2024-08-30T08:53:50.832 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:53:50.815+0000 7fa1869a6640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fa18010e1e0 con 0x7fa180102ac0 2024-08-30T08:53:50.832 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:53:50.815+0000 7fa184f43640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa180108140 0x7fa1801a0fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:53:50.832 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:53:50.815+0000 7fa184f43640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa180108140 0x7fa1801a0fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:56854/0 (socket says 172.21.15.167:56854) 2024-08-30T08:53:50.832 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:53:50.815+0000 7fa184f43640 1 -- 172.21.15.167:0/4260687628 learned_addr learned my addr 172.21.15.167:0/4260687628 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-08-30T08:53:50.833 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:53:50.815+0000 7fa17f7fe640 1 --2- 172.21.15.167:0/4260687628 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa180105520 0x7fa18019ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:53:50.833 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:53:50.816+0000 7fa17ffff640 1 --2- 172.21.15.167:0/4260687628 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa180102ac0 0x7fa180101f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:53:50.833 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:53:50.816+0000 7fa184f43640 1 -- 172.21.15.167:0/4260687628 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa180102ac0 msgr2=0x7fa180101f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:53:50.833 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:53:50.816+0000 7fa184f43640 1 --2- 172.21.15.167:0/4260687628 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa180102ac0 0x7fa180101f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:53:50.833 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:53:50.816+0000 7fa184f43640 1 -- 172.21.15.167:0/4260687628 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa180105520 msgr2=0x7fa18019ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:53:50.833 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:53:50.816+0000 7fa184f43640 1 --2- 172.21.15.167:0/4260687628 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa180105520 0x7fa18019ea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:53:50.833 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:53:50.816+0000 7fa184f43640 1 -- 172.21.15.167:0/4260687628 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1801a17d0 con 0x7fa180108140 2024-08-30T08:53:50.833 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:53:50.816+0000 7fa17ffff640 1 --2- 172.21.15.167:0/4260687628 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa180102ac0 0x7fa180101f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-30T08:53:50.833 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:53:50.816+0000 7fa184f43640 1 --2- 172.21.15.167:0/4260687628 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa180108140 0x7fa1801a0fb0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fa1780097b0 tx=0x7fa178031840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:53:50.833 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:53:50.816+0000 7fa17d7fa640 1 -- 172.21.15.167:0/4260687628 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa178003bf0 con 0x7fa180108140 2024-08-30T08:53:50.833 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:53:50.816+0000 7fa1869a6640 1 -- 172.21.15.167:0/4260687628 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1801a1a60 con 0x7fa180108140 2024-08-30T08:53:50.833 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:53:50.816+0000 7fa17d7fa640 1 -- 172.21.15.167:0/4260687628 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa178003d90 con 0x7fa180108140 2024-08-30T08:53:50.834 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:53:50.816+0000 7fa17d7fa640 1 -- 172.21.15.167:0/4260687628 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa17802fc30 con 0x7fa180108140 2024-08-30T08:53:50.834 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:53:50.816+0000 7fa1869a6640 1 -- 172.21.15.167:0/4260687628 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1801a1ff0 con 0x7fa180108140 2024-08-30T08:53:50.834 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:53:50.817+0000 7fa1869a6640 1 -- 172.21.15.167:0/4260687628 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1800ff020 con 0x7fa180108140 2024-08-30T08:53:50.834 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:53:50.819+0000 7fa17d7fa640 1 -- 172.21.15.167:0/4260687628 <== mon.2 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 16) ==== 152577+0+0 (secure 0 0 0) 0x7fa178004040 con 0x7fa180108140 2024-08-30T08:53:50.834 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:53:50.820+0000 7fa17d7fa640 1 --2- 172.21.15.167:0/4260687628 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fa16c0b4b30 0x7fa16c0b7040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:53:50.834 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:53:50.820+0000 7fa17d7fa640 1 -- 172.21.15.167:0/4260687628 <== mon.2 v2:172.21.15.167:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 3702+0+0 (secure 0 0 0) 0x7fa1781055b0 con 0x7fa180108140 2024-08-30T08:53:50.834 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:53:50.820+0000 7fa17ffff640 1 --2- 172.21.15.167:0/4260687628 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fa16c0b4b30 0x7fa16c0b7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:53:50.834 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:53:50.821+0000 7fa17ffff640 1 --2- 172.21.15.167:0/4260687628 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fa16c0b4b30 0x7fa16c0b7040 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fa1700058c0 tx=0x7fa170005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:53:50.834 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:53:50.823+0000 7fa17d7fa640 1 -- 172.21.15.167:0/4260687628 <== mon.2 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7fa1780cd8e0 con 0x7fa180108140 2024-08-30T08:53:50.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:50 smithi042 ceph-mon[18652]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-08-30T08:53:50.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:50 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-08-30T08:53:50.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:50 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-08-30T08:53:50.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:50 smithi042 ceph-mon[18652]: from='osd.3 [v2:172.21.15.42:6810/1135599916,v1:172.21.15.42:6811/1135599916]' entity='osd.3' 2024-08-30T08:53:50.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:50 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:53:50.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:50 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:53:50.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:50 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-08-30T08:53:50.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:50 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-08-30T08:53:50.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:50 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:53:50.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:50 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:53:50.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:50 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-30T08:53:50.884 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:50 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:53:50.884 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:50 smithi042 ceph-mon[18652]: osd.3 [v2:172.21.15.42:6810/1135599916,v1:172.21.15.42:6811/1135599916] boot 2024-08-30T08:53:50.884 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:50 smithi042 ceph-mon[18652]: osdmap e24: 4 total, 4 up, 4 in 2024-08-30T08:53:50.884 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:50 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-08-30T08:53:50.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:50 smithi190 ceph-mon[22001]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-08-30T08:53:50.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:50 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-08-30T08:53:50.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:50 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-08-30T08:53:50.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:50 smithi190 ceph-mon[22001]: from='osd.3 [v2:172.21.15.42:6810/1135599916,v1:172.21.15.42:6811/1135599916]' entity='osd.3' 2024-08-30T08:53:50.937 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:50 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:53:50.937 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:50 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:53:50.937 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:50 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-08-30T08:53:50.937 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:50 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-08-30T08:53:50.937 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:50 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:53:50.937 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:50 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:53:50.937 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:50 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-30T08:53:50.937 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:50 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:53:50.937 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:50 smithi190 ceph-mon[22001]: osd.3 [v2:172.21.15.42:6810/1135599916,v1:172.21.15.42:6811/1135599916] boot 2024-08-30T08:53:50.937 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:50 smithi190 ceph-mon[22001]: osdmap e24: 4 total, 4 up, 4 in 2024-08-30T08:53:50.937 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:50 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-08-30T08:53:51.012 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:53:51.010+0000 7fa1869a6640 1 -- 172.21.15.167:0/4260687628 --> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fa1800635a0 con 0x7fa16c0b4b30 2024-08-30T08:53:51.725 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:51 smithi167 ceph-mon[22063]: purged_snaps scrub starts 2024-08-30T08:53:51.725 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:51 smithi167 ceph-mon[22063]: purged_snaps scrub ok 2024-08-30T08:53:51.725 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:51 smithi167 ceph-mon[22063]: Detected new or changed devices on smithi042 2024-08-30T08:53:51.725 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:51 smithi167 ceph-mon[22063]: Adjusting osd_memory_target on smithi042 to 8546M 2024-08-30T08:53:51.725 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:51 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-30T08:53:51.726 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:51 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-30T08:53:51.726 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:51 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:53:51.726 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:51 smithi167 ceph-mon[22063]: osdmap e25: 4 total, 4 up, 4 in 2024-08-30T08:53:51.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:51 smithi042 ceph-mon[18652]: purged_snaps scrub starts 2024-08-30T08:53:51.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:51 smithi042 ceph-mon[18652]: purged_snaps scrub ok 2024-08-30T08:53:51.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:51 smithi042 ceph-mon[18652]: Detected new or changed devices on smithi042 2024-08-30T08:53:51.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:51 smithi042 ceph-mon[18652]: Adjusting osd_memory_target on smithi042 to 8546M 2024-08-30T08:53:51.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:51 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-30T08:53:51.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:51 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-30T08:53:51.884 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:51 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:53:51.884 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:51 smithi042 ceph-mon[18652]: osdmap e25: 4 total, 4 up, 4 in 2024-08-30T08:53:51.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:51 smithi190 ceph-mon[22001]: purged_snaps scrub starts 2024-08-30T08:53:51.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:51 smithi190 ceph-mon[22001]: purged_snaps scrub ok 2024-08-30T08:53:51.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:51 smithi190 ceph-mon[22001]: Detected new or changed devices on smithi042 2024-08-30T08:53:51.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:51 smithi190 ceph-mon[22001]: Adjusting osd_memory_target on smithi042 to 8546M 2024-08-30T08:53:51.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:51 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-30T08:53:51.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:51 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-30T08:53:51.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:51 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:53:51.937 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:51 smithi190 ceph-mon[22001]: osdmap e25: 4 total, 4 up, 4 in 2024-08-30T08:53:52.686 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:52 smithi167 ceph-mon[22063]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail 2024-08-30T08:53:52.686 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:52 smithi167 ceph-mon[22063]: from='client.24320 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-08-30T08:53:52.686 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:52 smithi167 ceph-mon[22063]: osdmap e26: 4 total, 4 up, 4 in 2024-08-30T08:53:52.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:52 smithi042 ceph-mon[18652]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail 2024-08-30T08:53:52.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:52 smithi042 ceph-mon[18652]: from='client.24320 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-08-30T08:53:52.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:52 smithi042 ceph-mon[18652]: osdmap e26: 4 total, 4 up, 4 in 2024-08-30T08:53:52.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:52 smithi190 ceph-mon[22001]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail 2024-08-30T08:53:52.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:52 smithi190 ceph-mon[22001]: from='client.24320 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-08-30T08:53:52.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:52 smithi190 ceph-mon[22001]: osdmap e26: 4 total, 4 up, 4 in 2024-08-30T08:53:53.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:53 smithi167 ceph-mon[22063]: from='client.? 172.21.15.167:0/596430346' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "3ca1aae3-c972-4fb9-ac86-236279c30d64"} : dispatch 2024-08-30T08:53:53.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:53 smithi167 ceph-mon[22063]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "3ca1aae3-c972-4fb9-ac86-236279c30d64"} : dispatch 2024-08-30T08:53:53.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:53 smithi167 ceph-mon[22063]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3ca1aae3-c972-4fb9-ac86-236279c30d64"}]': finished 2024-08-30T08:53:53.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:53 smithi167 ceph-mon[22063]: osdmap e27: 5 total, 4 up, 5 in 2024-08-30T08:53:53.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:53 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-08-30T08:53:53.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:53 smithi042 ceph-mon[18652]: from='client.? 172.21.15.167:0/596430346' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "3ca1aae3-c972-4fb9-ac86-236279c30d64"} : dispatch 2024-08-30T08:53:53.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:53 smithi042 ceph-mon[18652]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "3ca1aae3-c972-4fb9-ac86-236279c30d64"} : dispatch 2024-08-30T08:53:53.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:53 smithi042 ceph-mon[18652]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3ca1aae3-c972-4fb9-ac86-236279c30d64"}]': finished 2024-08-30T08:53:53.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:53 smithi042 ceph-mon[18652]: osdmap e27: 5 total, 4 up, 5 in 2024-08-30T08:53:53.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:53 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-08-30T08:53:53.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:53 smithi190 ceph-mon[22001]: from='client.? 172.21.15.167:0/596430346' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "3ca1aae3-c972-4fb9-ac86-236279c30d64"} : dispatch 2024-08-30T08:53:53.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:53 smithi190 ceph-mon[22001]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "3ca1aae3-c972-4fb9-ac86-236279c30d64"} : dispatch 2024-08-30T08:53:53.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:53 smithi190 ceph-mon[22001]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3ca1aae3-c972-4fb9-ac86-236279c30d64"}]': finished 2024-08-30T08:53:53.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:53 smithi190 ceph-mon[22001]: osdmap e27: 5 total, 4 up, 5 in 2024-08-30T08:53:53.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:53 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-08-30T08:53:54.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:54 smithi167 ceph-mon[22063]: pgmap v110: 1 pgs: 1 peering; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-08-30T08:53:54.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:54 smithi167 ceph-mon[22063]: from='client.? 172.21.15.167:0/3895888863' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-30T08:53:54.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:54 smithi042 ceph-mon[18652]: pgmap v110: 1 pgs: 1 peering; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-08-30T08:53:54.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:54 smithi042 ceph-mon[18652]: from='client.? 172.21.15.167:0/3895888863' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-30T08:53:54.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:54 smithi190 ceph-mon[22001]: pgmap v110: 1 pgs: 1 peering; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-08-30T08:53:54.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:54 smithi190 ceph-mon[22001]: from='client.? 172.21.15.167:0/3895888863' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-30T08:53:56.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:56 smithi167 ceph-mon[22063]: pgmap v112: 1 pgs: 1 peering; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-08-30T08:53:56.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:56 smithi042 ceph-mon[18652]: pgmap v112: 1 pgs: 1 peering; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-08-30T08:53:56.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:56 smithi190 ceph-mon[22001]: pgmap v112: 1 pgs: 1 peering; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-08-30T08:53:58.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:53:58 smithi167 ceph-mon[22063]: pgmap v113: 1 pgs: 1 peering; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-08-30T08:53:58.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:53:58 smithi042 ceph-mon[18652]: pgmap v113: 1 pgs: 1 peering; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-08-30T08:53:58.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:53:58 smithi190 ceph-mon[22001]: pgmap v113: 1 pgs: 1 peering; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-08-30T08:54:00.746 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:54:00 smithi167 ceph-mon[22063]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-08-30T08:54:00.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:54:00 smithi042 ceph-mon[18652]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-08-30T08:54:00.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:54:00 smithi190 ceph-mon[22001]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-08-30T08:54:02.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:54:02 smithi042 ceph-mon[18652]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-08-30T08:54:02.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:54:02 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.4"} : dispatch 2024-08-30T08:54:02.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:54:02 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:54:02.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:54:02 smithi190 ceph-mon[22001]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-08-30T08:54:02.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:54:02 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.4"} : dispatch 2024-08-30T08:54:02.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:54:02 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:54:03.034 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:54:02 smithi167 ceph-mon[22063]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-08-30T08:54:03.034 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:54:02 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.4"} : dispatch 2024-08-30T08:54:03.034 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:54:02 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:54:03.867 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:54:03 smithi167 ceph-mon[22063]: Deploying daemon osd.4 on smithi167 2024-08-30T08:54:03.884 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:54:03 smithi042 ceph-mon[18652]: Deploying daemon osd.4 on smithi167 2024-08-30T08:54:03.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:54:03 smithi190 ceph-mon[22001]: Deploying daemon osd.4 on smithi167 2024-08-30T08:54:04.797 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:54:04 smithi167 ceph-mon[22063]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-08-30T08:54:04.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:54:04 smithi042 ceph-mon[18652]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-08-30T08:54:04.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:54:04 smithi190 ceph-mon[22001]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-08-30T08:54:06.660 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:54:06 smithi167 ceph-mon[22063]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-08-30T08:54:06.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:54:06 smithi042 ceph-mon[18652]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-08-30T08:54:06.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:54:06 smithi190 ceph-mon[22001]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-08-30T08:54:07.882 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:54:07 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-30T08:54:07.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:54:07 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:54:07.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:54:07 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:54:07.891 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:54:07 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-30T08:54:07.891 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:54:07 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:54:07.891 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:54:07 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:54:07.935 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:54:07 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-30T08:54:07.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:54:07 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:54:07.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:54:07 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:54:08.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:54:08 smithi042 ceph-mon[18652]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-08-30T08:54:08.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:54:08 smithi190 ceph-mon[22001]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-08-30T08:54:08.953 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:54:08 smithi167 ceph-mon[22063]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-08-30T08:54:10.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:54:10 smithi042 ceph-mon[18652]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-08-30T08:54:10.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:54:10 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:54:10.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:54:10 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:54:10.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:54:10 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:54:10.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:54:10 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-30T08:54:10.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:54:10 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:54:10.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:54:10 smithi190 ceph-mon[22001]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-08-30T08:54:10.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:54:10 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:54:10.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:54:10 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:54:10.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:54:10 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:54:10.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:54:10 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-30T08:54:10.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:54:10 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:54:11.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:54:10 smithi167 ceph-mon[22063]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-08-30T08:54:11.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:54:10 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:54:11.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:54:10 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:54:11.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:54:10 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:54:11.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:54:10 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-30T08:54:11.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:54:10 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:54:11.450 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:54:11.448+0000 7fa17d7fa640 1 -- 172.21.15.167:0/4260687628 <== mgr.14150 v2:172.21.15.42:6800/3005140902 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fa1800635a0 con 0x7fa16c0b4b30 2024-08-30T08:54:11.451 INFO:teuthology.orchestra.run.smithi167.stdout:Created osd(s) 4 on host 'smithi167' 2024-08-30T08:54:11.453 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:54:11.451+0000 7fa1869a6640 1 -- 172.21.15.167:0/4260687628 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fa16c0b4b30 msgr2=0x7fa16c0b7040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:54:11.453 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:54:11.451+0000 7fa1869a6640 1 --2- 172.21.15.167:0/4260687628 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fa16c0b4b30 0x7fa16c0b7040 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fa1700058c0 tx=0x7fa170005850 comp rx=0 tx=0).stop 2024-08-30T08:54:11.453 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:54:11.451+0000 7fa1869a6640 1 -- 172.21.15.167:0/4260687628 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa180108140 msgr2=0x7fa1801a0fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:54:11.453 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:54:11.451+0000 7fa1869a6640 1 --2- 172.21.15.167:0/4260687628 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa180108140 0x7fa1801a0fb0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fa1780097b0 tx=0x7fa178031840 comp rx=0 tx=0).stop 2024-08-30T08:54:11.453 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:54:11.451+0000 7fa1869a6640 1 -- 172.21.15.167:0/4260687628 shutdown_connections 2024-08-30T08:54:11.453 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:54:11.451+0000 7fa1869a6640 1 --2- 172.21.15.167:0/4260687628 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fa16c0b4b30 0x7fa16c0b7040 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:54:11.453 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:54:11.451+0000 7fa1869a6640 1 --2- 172.21.15.167:0/4260687628 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa180108140 0x7fa1801a0fb0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:54:11.453 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:54:11.451+0000 7fa1869a6640 1 --2- 172.21.15.167:0/4260687628 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa180105520 0x7fa18019ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:54:11.454 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:54:11.451+0000 7fa1869a6640 1 --2- 172.21.15.167:0/4260687628 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa180102ac0 0x7fa180101f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:54:11.454 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:54:11.451+0000 7fa1869a6640 1 -- 172.21.15.167:0/4260687628 >> 172.21.15.167:0/4260687628 conn(0x7fa1800fd500 msgr2=0x7fa1801038d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:54:11.454 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:54:11.451+0000 7fa1869a6640 1 -- 172.21.15.167:0/4260687628 shutdown_connections 2024-08-30T08:54:11.454 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:54:11.451+0000 7fa1869a6640 1 -- 172.21.15.167:0/4260687628 wait complete. 2024-08-30T08:54:11.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:54:11 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-30T08:54:11.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:54:11 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:54:11.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:54:11 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:54:11.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:54:11 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-30T08:54:11.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:54:11 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:54:11.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:54:11 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:54:11.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:54:11 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-30T08:54:11.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:54:11 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:54:11.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:54:11 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:54:12.146 DEBUG:teuthology.orchestra.run.smithi167:osd.4> sudo journalctl -f -n 0 -u ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968@osd.4.service 2024-08-30T08:54:12.148 INFO:tasks.cephadm:Deploying osd.5 on smithi190 with /dev/vg_nvme/lv_3... 2024-08-30T08:54:12.148 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-08-30T08:54:12.400 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.c/config 2024-08-30T08:54:12.792 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:54:12 smithi190 ceph-mon[22001]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-08-30T08:54:12.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:54:12 smithi042 ceph-mon[18652]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-08-30T08:54:13.018 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:54:12 smithi167 ceph-mon[22063]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-08-30T08:54:13.577 INFO:journalctl@ceph.osd.4.smithi167.stdout:Aug 30 08:54:13 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-4[33404]: 2024-08-30T08:54:13.311+0000 7fd0a3d35640 -1 osd.4 0 log_to_monitors true 2024-08-30T08:54:13.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:54:13 smithi042 ceph-mon[18652]: from='osd.4 [v2:172.21.15.167:6808/2927511419,v1:172.21.15.167:6809/2927511419]' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-08-30T08:54:13.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:54:13 smithi042 ceph-mon[18652]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-08-30T08:54:13.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:54:13 smithi190 ceph-mon[22001]: from='osd.4 [v2:172.21.15.167:6808/2927511419,v1:172.21.15.167:6809/2927511419]' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-08-30T08:54:13.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:54:13 smithi190 ceph-mon[22001]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-08-30T08:54:14.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:54:13 smithi167 ceph-mon[22063]: from='osd.4 [v2:172.21.15.167:6808/2927511419,v1:172.21.15.167:6809/2927511419]' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-08-30T08:54:14.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:54:13 smithi167 ceph-mon[22063]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-08-30T08:54:14.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:54:14 smithi042 ceph-mon[18652]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-08-30T08:54:14.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:54:14 smithi042 ceph-mon[18652]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-08-30T08:54:14.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:54:14 smithi042 ceph-mon[18652]: from='osd.4 [v2:172.21.15.167:6808/2927511419,v1:172.21.15.167:6809/2927511419]' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi167", "root=default"]} : dispatch 2024-08-30T08:54:14.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:54:14 smithi042 ceph-mon[18652]: osdmap e28: 5 total, 4 up, 5 in 2024-08-30T08:54:14.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:54:14 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-08-30T08:54:14.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:54:14 smithi042 ceph-mon[18652]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi167", "root=default"]} : dispatch 2024-08-30T08:54:14.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:54:14 smithi190 ceph-mon[22001]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-08-30T08:54:14.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:54:14 smithi190 ceph-mon[22001]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-08-30T08:54:14.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:54:14 smithi190 ceph-mon[22001]: from='osd.4 [v2:172.21.15.167:6808/2927511419,v1:172.21.15.167:6809/2927511419]' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi167", "root=default"]} : dispatch 2024-08-30T08:54:14.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:54:14 smithi190 ceph-mon[22001]: osdmap e28: 5 total, 4 up, 5 in 2024-08-30T08:54:14.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:54:14 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-08-30T08:54:14.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:54:14 smithi190 ceph-mon[22001]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi167", "root=default"]} : dispatch 2024-08-30T08:54:14.938 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:54:14 smithi167 ceph-mon[22063]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-08-30T08:54:14.938 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:54:14 smithi167 ceph-mon[22063]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-08-30T08:54:14.938 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:54:14 smithi167 ceph-mon[22063]: from='osd.4 [v2:172.21.15.167:6808/2927511419,v1:172.21.15.167:6809/2927511419]' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi167", "root=default"]} : dispatch 2024-08-30T08:54:14.938 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:54:14 smithi167 ceph-mon[22063]: osdmap e28: 5 total, 4 up, 5 in 2024-08-30T08:54:14.938 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:54:14 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-08-30T08:54:14.938 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:54:14 smithi167 ceph-mon[22063]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi167", "root=default"]} : dispatch 2024-08-30T08:54:15.089 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-08-30T08:54:15.114 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph orch daemon add osd smithi190:vg_nvme/lv_3 2024-08-30T08:54:15.360 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.c/config 2024-08-30T08:54:15.628 INFO:journalctl@ceph.osd.4.smithi167.stdout:Aug 30 08:54:15 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-4[33404]: 2024-08-30T08:54:15.350+0000 7fd09fcb7640 -1 osd.4 0 waiting for initial osdmap 2024-08-30T08:54:15.629 INFO:journalctl@ceph.osd.4.smithi167.stdout:Aug 30 08:54:15 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-4[33404]: 2024-08-30T08:54:15.356+0000 7fd097aad640 -1 osd.4 29 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-30T08:54:15.790 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:54:15 smithi190 ceph-mon[22001]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]': finished 2024-08-30T08:54:15.790 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:54:15 smithi190 ceph-mon[22001]: osdmap e29: 5 total, 4 up, 5 in 2024-08-30T08:54:15.790 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:54:15 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-08-30T08:54:15.790 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:54:15 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-08-30T08:54:15.790 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:54:15 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:54:15.790 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:54:15 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:54:15.790 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:54:15 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-08-30T08:54:15.790 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:54:15 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-08-30T08:54:15.790 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:54:15 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:54:15.790 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:54:15 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:54:15.790 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:54:15 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-30T08:54:15.791 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:54:15 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:54:15.791 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:54:15 smithi190 ceph-mon[22001]: from='osd.4 ' entity='osd.4' 2024-08-30T08:54:15.816 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:54:15.814+0000 7fce50c8f640 1 -- 172.21.15.190:0/861273798 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fce4c102410 msgr2=0x7fce4c102880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:54:15.816 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:54:15.814+0000 7fce50c8f640 1 --2- 172.21.15.190:0/861273798 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fce4c102410 0x7fce4c102880 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fce3c009900 tx=0x7fce3c02a480 comp rx=0 tx=0).stop 2024-08-30T08:54:15.817 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:54:15.814+0000 7fce50c8f640 1 -- 172.21.15.190:0/861273798 shutdown_connections 2024-08-30T08:54:15.817 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:54:15.814+0000 7fce50c8f640 1 --2- 172.21.15.190:0/861273798 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fce4c103e40 0x7fce4c10a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:54:15.817 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:54:15.814+0000 7fce50c8f640 1 --2- 172.21.15.190:0/861273798 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fce4c1034a0 0x7fce4c1038b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:54:15.817 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:54:15.814+0000 7fce50c8f640 1 --2- 172.21.15.190:0/861273798 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fce4c102410 0x7fce4c102880 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:54:15.817 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:54:15.814+0000 7fce50c8f640 1 -- 172.21.15.190:0/861273798 >> 172.21.15.190:0/861273798 conn(0x7fce4c0fdd80 msgr2=0x7fce4c100230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:54:15.817 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:54:15.814+0000 7fce50c8f640 1 -- 172.21.15.190:0/861273798 shutdown_connections 2024-08-30T08:54:15.817 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:54:15.814+0000 7fce50c8f640 1 -- 172.21.15.190:0/861273798 wait complete. 2024-08-30T08:54:15.819 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:54:15.815+0000 7fce50c8f640 1 Processor -- start 2024-08-30T08:54:15.819 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:54:15.815+0000 7fce50c8f640 1 -- start start 2024-08-30T08:54:15.819 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:54:15.815+0000 7fce50c8f640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fce4c102410 0x7fce4c106000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:54:15.819 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:54:15.815+0000 7fce50c8f640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fce4c1034a0 0x7fce4c1048d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:54:15.819 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:54:15.815+0000 7fce50c8f640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fce4c103e40 0x7fce4c104e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:54:15.819 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:54:15.815+0000 7fce50c8f640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fce4c10e070 con 0x7fce4c103e40 2024-08-30T08:54:15.819 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:54:15.815+0000 7fce50c8f640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fce4c10def0 con 0x7fce4c1034a0 2024-08-30T08:54:15.819 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:54:15.815+0000 7fce50c8f640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fce4c10e1f0 con 0x7fce4c102410 2024-08-30T08:54:15.820 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:54:15.816+0000 7fce4a59c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fce4c102410 0x7fce4c106000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:54:15.820 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:54:15.816+0000 7fce4a59c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fce4c102410 0x7fce4c106000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:51430/0 (socket says 172.21.15.190:51430) 2024-08-30T08:54:15.820 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:54:15.816+0000 7fce4a59c640 1 -- 172.21.15.190:0/2415316349 learned_addr learned my addr 172.21.15.190:0/2415316349 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-30T08:54:15.820 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:54:15.816+0000 7fce4ad9d640 1 --2- 172.21.15.190:0/2415316349 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fce4c103e40 0x7fce4c104e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:54:15.820 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:54:15.816+0000 7fce49d9b640 1 --2- 172.21.15.190:0/2415316349 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fce4c1034a0 0x7fce4c1048d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:54:15.820 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:54:15.816+0000 7fce4a59c640 1 -- 172.21.15.190:0/2415316349 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fce4c1034a0 msgr2=0x7fce4c1048d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:54:15.820 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:54:15.816+0000 7fce4a59c640 1 --2- 172.21.15.190:0/2415316349 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fce4c1034a0 0x7fce4c1048d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:54:15.820 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:54:15.816+0000 7fce4a59c640 1 -- 172.21.15.190:0/2415316349 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fce4c103e40 msgr2=0x7fce4c104e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:54:15.820 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:54:15.816+0000 7fce4a59c640 1 --2- 172.21.15.190:0/2415316349 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fce4c103e40 0x7fce4c104e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:54:15.820 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:54:15.816+0000 7fce4a59c640 1 -- 172.21.15.190:0/2415316349 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce4c105510 con 0x7fce4c102410 2024-08-30T08:54:15.820 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:54:15.816+0000 7fce4a59c640 1 --2- 172.21.15.190:0/2415316349 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fce4c102410 0x7fce4c106000 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fce3c006010 tx=0x7fce3c031a50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:54:15.820 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:54:15.816+0000 7fce437fe640 1 -- 172.21.15.190:0/2415316349 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce3c03f030 con 0x7fce4c102410 2024-08-30T08:54:15.821 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:54:15.816+0000 7fce437fe640 1 -- 172.21.15.190:0/2415316349 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce3c003aa0 con 0x7fce4c102410 2024-08-30T08:54:15.821 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:54:15.816+0000 7fce50c8f640 1 -- 172.21.15.190:0/2415316349 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce4c1a3e00 con 0x7fce4c102410 2024-08-30T08:54:15.821 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:54:15.816+0000 7fce437fe640 1 -- 172.21.15.190:0/2415316349 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce3c035070 con 0x7fce4c102410 2024-08-30T08:54:15.821 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:54:15.816+0000 7fce50c8f640 1 -- 172.21.15.190:0/2415316349 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce4c1a41a0 con 0x7fce4c102410 2024-08-30T08:54:15.821 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:54:15.817+0000 7fce50c8f640 1 -- 172.21.15.190:0/2415316349 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce4c1a4370 con 0x7fce4c102410 2024-08-30T08:54:15.822 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:54:15.819+0000 7fce437fe640 1 -- 172.21.15.190:0/2415316349 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 16) ==== 152577+0+0 (secure 0 0 0) 0x7fce3c042030 con 0x7fce4c102410 2024-08-30T08:54:15.822 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:54:15.819+0000 7fce437fe640 1 --2- 172.21.15.190:0/2415316349 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fce380b4b30 0x7fce380b7040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:54:15.822 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:54:15.820+0000 7fce437fe640 1 -- 172.21.15.190:0/2415316349 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 3821+0+0 (secure 0 0 0) 0x7fce3c104bf0 con 0x7fce4c102410 2024-08-30T08:54:15.822 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:54:15.820+0000 7fce49d9b640 1 --2- 172.21.15.190:0/2415316349 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fce380b4b30 0x7fce380b7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:54:15.822 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:54:15.820+0000 7fce49d9b640 1 --2- 172.21.15.190:0/2415316349 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fce380b4b30 0x7fce380b7040 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fce340058c0 tx=0x7fce3400a4c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:54:15.825 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:54:15.823+0000 7fce437fe640 1 -- 172.21.15.190:0/2415316349 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7fce3c0cceb0 con 0x7fce4c102410 2024-08-30T08:54:15.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:54:15 smithi042 ceph-mon[18652]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]': finished 2024-08-30T08:54:15.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:54:15 smithi042 ceph-mon[18652]: osdmap e29: 5 total, 4 up, 5 in 2024-08-30T08:54:15.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:54:15 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-08-30T08:54:15.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:54:15 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-08-30T08:54:15.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:54:15 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:54:15.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:54:15 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:54:15.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:54:15 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-08-30T08:54:15.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:54:15 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-08-30T08:54:15.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:54:15 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:54:15.884 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:54:15 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:54:15.884 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:54:15 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-30T08:54:15.884 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:54:15 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:54:15.884 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:54:15 smithi042 ceph-mon[18652]: from='osd.4 ' entity='osd.4' 2024-08-30T08:54:16.013 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:54:16.011+0000 7fce50c8f640 1 -- 172.21.15.190:0/2415316349 --> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi190:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fce4c0ffd40 con 0x7fce380b4b30 2024-08-30T08:54:16.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:54:15 smithi167 ceph-mon[22063]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]': finished 2024-08-30T08:54:16.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:54:15 smithi167 ceph-mon[22063]: osdmap e29: 5 total, 4 up, 5 in 2024-08-30T08:54:16.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:54:15 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-08-30T08:54:16.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:54:15 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-08-30T08:54:16.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:54:15 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:54:16.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:54:15 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:54:16.078 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:54:15 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-08-30T08:54:16.078 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:54:15 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-08-30T08:54:16.078 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:54:15 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:54:16.078 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:54:15 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:54:16.078 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:54:15 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-30T08:54:16.078 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:54:15 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:54:16.078 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:54:15 smithi167 ceph-mon[22063]: from='osd.4 ' entity='osd.4' 2024-08-30T08:54:16.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:54:16 smithi190 ceph-mon[22001]: purged_snaps scrub starts 2024-08-30T08:54:16.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:54:16 smithi190 ceph-mon[22001]: purged_snaps scrub ok 2024-08-30T08:54:16.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:54:16 smithi190 ceph-mon[22001]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-08-30T08:54:16.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:54:16 smithi190 ceph-mon[22001]: Detected new or changed devices on smithi167 2024-08-30T08:54:16.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:54:16 smithi190 ceph-mon[22001]: Adjusting osd_memory_target on smithi167 to 8544M 2024-08-30T08:54:16.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:54:16 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-08-30T08:54:16.687 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:54:16 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-30T08:54:16.687 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:54:16 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-30T08:54:16.687 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:54:16 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:54:16.687 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:54:16 smithi190 ceph-mon[22001]: osd.4 [v2:172.21.15.167:6808/2927511419,v1:172.21.15.167:6809/2927511419] boot 2024-08-30T08:54:16.687 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:54:16 smithi190 ceph-mon[22001]: osdmap e30: 5 total, 5 up, 5 in 2024-08-30T08:54:16.687 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:54:16 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-08-30T08:54:16.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:54:16 smithi042 ceph-mon[18652]: purged_snaps scrub starts 2024-08-30T08:54:16.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:54:16 smithi042 ceph-mon[18652]: purged_snaps scrub ok 2024-08-30T08:54:16.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:54:16 smithi042 ceph-mon[18652]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-08-30T08:54:16.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:54:16 smithi042 ceph-mon[18652]: Detected new or changed devices on smithi167 2024-08-30T08:54:16.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:54:16 smithi042 ceph-mon[18652]: Adjusting osd_memory_target on smithi167 to 8544M 2024-08-30T08:54:16.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:54:16 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-08-30T08:54:16.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:54:16 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-30T08:54:16.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:54:16 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-30T08:54:16.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:54:16 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:54:16.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:54:16 smithi042 ceph-mon[18652]: osd.4 [v2:172.21.15.167:6808/2927511419,v1:172.21.15.167:6809/2927511419] boot 2024-08-30T08:54:16.884 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:54:16 smithi042 ceph-mon[18652]: osdmap e30: 5 total, 5 up, 5 in 2024-08-30T08:54:16.884 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:54:16 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-08-30T08:54:17.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:54:16 smithi167 ceph-mon[22063]: purged_snaps scrub starts 2024-08-30T08:54:17.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:54:16 smithi167 ceph-mon[22063]: purged_snaps scrub ok 2024-08-30T08:54:17.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:54:16 smithi167 ceph-mon[22063]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-08-30T08:54:17.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:54:16 smithi167 ceph-mon[22063]: Detected new or changed devices on smithi167 2024-08-30T08:54:17.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:54:16 smithi167 ceph-mon[22063]: Adjusting osd_memory_target on smithi167 to 8544M 2024-08-30T08:54:17.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:54:16 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-08-30T08:54:17.078 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:54:16 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-30T08:54:17.078 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:54:16 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-30T08:54:17.078 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:54:16 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:54:17.078 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:54:16 smithi167 ceph-mon[22063]: osd.4 [v2:172.21.15.167:6808/2927511419,v1:172.21.15.167:6809/2927511419] boot 2024-08-30T08:54:17.078 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:54:16 smithi167 ceph-mon[22063]: osdmap e30: 5 total, 5 up, 5 in 2024-08-30T08:54:17.078 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:54:16 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-08-30T08:54:17.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:54:17 smithi190 ceph-mon[22001]: from='client.24352 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi190:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-08-30T08:54:18.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:54:17 smithi167 ceph-mon[22063]: from='client.24352 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi190:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-08-30T08:54:18.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:54:17 smithi042 ceph-mon[18652]: from='client.24352 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi190:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-08-30T08:54:18.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:54:18 smithi190 ceph-mon[22001]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-08-30T08:54:18.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:54:18 smithi190 ceph-mon[22001]: osdmap e31: 5 total, 5 up, 5 in 2024-08-30T08:54:18.937 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:54:18 smithi190 ceph-mon[22001]: from='client.? 172.21.15.190:0/3422274035' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "5d07f54b-0d07-4efe-9bcf-390755ce0258"} : dispatch 2024-08-30T08:54:18.937 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:54:18 smithi190 ceph-mon[22001]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "5d07f54b-0d07-4efe-9bcf-390755ce0258"} : dispatch 2024-08-30T08:54:18.937 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:54:18 smithi190 ceph-mon[22001]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5d07f54b-0d07-4efe-9bcf-390755ce0258"}]': finished 2024-08-30T08:54:18.937 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:54:18 smithi190 ceph-mon[22001]: osdmap e32: 6 total, 5 up, 6 in 2024-08-30T08:54:18.937 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:54:18 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-08-30T08:54:19.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:54:18 smithi167 ceph-mon[22063]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-08-30T08:54:19.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:54:18 smithi167 ceph-mon[22063]: osdmap e31: 5 total, 5 up, 5 in 2024-08-30T08:54:19.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:54:18 smithi167 ceph-mon[22063]: from='client.? 172.21.15.190:0/3422274035' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "5d07f54b-0d07-4efe-9bcf-390755ce0258"} : dispatch 2024-08-30T08:54:19.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:54:18 smithi167 ceph-mon[22063]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "5d07f54b-0d07-4efe-9bcf-390755ce0258"} : dispatch 2024-08-30T08:54:19.078 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:54:18 smithi167 ceph-mon[22063]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5d07f54b-0d07-4efe-9bcf-390755ce0258"}]': finished 2024-08-30T08:54:19.078 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:54:18 smithi167 ceph-mon[22063]: osdmap e32: 6 total, 5 up, 6 in 2024-08-30T08:54:19.078 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:54:18 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-08-30T08:54:19.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:54:18 smithi042 ceph-mon[18652]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-08-30T08:54:19.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:54:18 smithi042 ceph-mon[18652]: osdmap e31: 5 total, 5 up, 5 in 2024-08-30T08:54:19.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:54:18 smithi042 ceph-mon[18652]: from='client.? 172.21.15.190:0/3422274035' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "5d07f54b-0d07-4efe-9bcf-390755ce0258"} : dispatch 2024-08-30T08:54:19.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:54:18 smithi042 ceph-mon[18652]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "5d07f54b-0d07-4efe-9bcf-390755ce0258"} : dispatch 2024-08-30T08:54:19.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:54:18 smithi042 ceph-mon[18652]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5d07f54b-0d07-4efe-9bcf-390755ce0258"}]': finished 2024-08-30T08:54:19.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:54:18 smithi042 ceph-mon[18652]: osdmap e32: 6 total, 5 up, 6 in 2024-08-30T08:54:19.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:54:18 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-08-30T08:54:19.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:54:19 smithi190 ceph-mon[22001]: from='client.? 172.21.15.190:0/1834316904' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-30T08:54:20.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:54:19 smithi167 ceph-mon[22063]: from='client.? 172.21.15.190:0/1834316904' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-30T08:54:20.132 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:54:19 smithi042 ceph-mon[18652]: from='client.? 172.21.15.190:0/1834316904' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-30T08:54:20.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:54:20 smithi190 ceph-mon[22001]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-08-30T08:54:21.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:54:20 smithi167 ceph-mon[22063]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-08-30T08:54:21.132 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:54:20 smithi042 ceph-mon[18652]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-08-30T08:54:22.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:54:22 smithi190 ceph-mon[22001]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-08-30T08:54:23.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:54:22 smithi167 ceph-mon[22063]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-08-30T08:54:23.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:54:22 smithi042 ceph-mon[18652]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-08-30T08:54:24.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:54:24 smithi190 ceph-mon[22001]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-08-30T08:54:25.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:54:24 smithi167 ceph-mon[22063]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-08-30T08:54:25.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:54:24 smithi042 ceph-mon[18652]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-08-30T08:54:26.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:54:26 smithi190 ceph-mon[22001]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-08-30T08:54:27.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:54:26 smithi167 ceph-mon[22063]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-08-30T08:54:27.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:54:26 smithi042 ceph-mon[18652]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-08-30T08:54:27.714 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:54:27 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.5"} : dispatch 2024-08-30T08:54:27.715 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:54:27 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:54:28.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:54:27 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.5"} : dispatch 2024-08-30T08:54:28.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:54:27 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:54:28.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:54:27 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.5"} : dispatch 2024-08-30T08:54:28.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:54:27 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:54:28.815 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:54:28 smithi190 ceph-mon[22001]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-08-30T08:54:28.815 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:54:28 smithi190 ceph-mon[22001]: Deploying daemon osd.5 on smithi190 2024-08-30T08:54:29.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:54:28 smithi167 ceph-mon[22063]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-08-30T08:54:29.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:54:28 smithi167 ceph-mon[22063]: Deploying daemon osd.5 on smithi190 2024-08-30T08:54:29.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:54:28 smithi042 ceph-mon[18652]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-08-30T08:54:29.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:54:28 smithi042 ceph-mon[18652]: Deploying daemon osd.5 on smithi190 2024-08-30T08:54:30.758 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:54:30 smithi190 ceph-mon[22001]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-08-30T08:54:30.935 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:54:30 smithi042 ceph-mon[18652]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-08-30T08:54:31.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:54:30 smithi167 ceph-mon[22063]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-08-30T08:54:32.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:54:32 smithi190 ceph-mon[22001]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-08-30T08:54:32.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:54:32 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-30T08:54:32.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:54:32 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:54:32.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:54:32 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:54:33.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:54:32 smithi167 ceph-mon[22063]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-08-30T08:54:33.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:54:32 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-30T08:54:33.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:54:32 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:54:33.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:54:32 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:54:33.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:54:32 smithi042 ceph-mon[18652]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-08-30T08:54:33.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:54:32 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-30T08:54:33.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:54:32 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:54:33.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:54:32 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:54:34.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:54:34 smithi190 ceph-mon[22001]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-08-30T08:54:35.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:54:34 smithi167 ceph-mon[22063]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-08-30T08:54:35.132 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:54:34 smithi042 ceph-mon[18652]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-08-30T08:54:36.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:54:35 smithi190 ceph-mon[22001]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-08-30T08:54:36.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:54:35 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:54:36.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:54:35 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:54:36.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:54:35 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:54:36.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:54:35 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-30T08:54:36.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:54:35 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:54:36.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:54:35 smithi167 ceph-mon[22063]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-08-30T08:54:36.328 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:54:35 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:54:36.328 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:54:35 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:54:36.328 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:54:35 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:54:36.328 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:54:35 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-30T08:54:36.328 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:54:35 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:54:36.328 INFO:teuthology.orchestra.run.smithi190.stdout:Created osd(s) 5 on host 'smithi190' 2024-08-30T08:54:36.329 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:54:36.326+0000 7fce437fe640 1 -- 172.21.15.190:0/2415316349 <== mgr.14150 v2:172.21.15.42:6800/3005140902 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fce4c0ffd40 con 0x7fce380b4b30 2024-08-30T08:54:36.331 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:54:36.329+0000 7fce50c8f640 1 -- 172.21.15.190:0/2415316349 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fce380b4b30 msgr2=0x7fce380b7040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:54:36.331 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:54:36.329+0000 7fce50c8f640 1 --2- 172.21.15.190:0/2415316349 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fce380b4b30 0x7fce380b7040 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fce340058c0 tx=0x7fce3400a4c0 comp rx=0 tx=0).stop 2024-08-30T08:54:36.332 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:54:36.329+0000 7fce50c8f640 1 -- 172.21.15.190:0/2415316349 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fce4c102410 msgr2=0x7fce4c106000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:54:36.332 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:54:36.329+0000 7fce50c8f640 1 --2- 172.21.15.190:0/2415316349 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fce4c102410 0x7fce4c106000 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fce3c006010 tx=0x7fce3c031a50 comp rx=0 tx=0).stop 2024-08-30T08:54:36.332 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:54:36.329+0000 7fce50c8f640 1 -- 172.21.15.190:0/2415316349 shutdown_connections 2024-08-30T08:54:36.332 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:54:36.329+0000 7fce50c8f640 1 --2- 172.21.15.190:0/2415316349 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fce380b4b30 0x7fce380b7040 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:54:36.332 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:54:36.329+0000 7fce50c8f640 1 --2- 172.21.15.190:0/2415316349 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fce4c103e40 0x7fce4c104e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:54:36.332 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:54:36.329+0000 7fce50c8f640 1 --2- 172.21.15.190:0/2415316349 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fce4c1034a0 0x7fce4c1048d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:54:36.332 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:54:36.329+0000 7fce50c8f640 1 --2- 172.21.15.190:0/2415316349 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fce4c102410 0x7fce4c106000 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:54:36.332 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:54:36.329+0000 7fce50c8f640 1 -- 172.21.15.190:0/2415316349 >> 172.21.15.190:0/2415316349 conn(0x7fce4c0fdd80 msgr2=0x7fce4c0ff630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:54:36.332 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:54:36.329+0000 7fce50c8f640 1 -- 172.21.15.190:0/2415316349 shutdown_connections 2024-08-30T08:54:36.332 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:54:36.329+0000 7fce50c8f640 1 -- 172.21.15.190:0/2415316349 wait complete. 2024-08-30T08:54:36.382 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:54:35 smithi042 ceph-mon[18652]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-08-30T08:54:36.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:54:35 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:54:36.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:54:35 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:54:36.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:54:35 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:54:36.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:54:35 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-30T08:54:36.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:54:35 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:54:37.004 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:54:36 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-30T08:54:37.004 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:54:36 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:54:37.004 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:54:36 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:54:37.032 DEBUG:teuthology.orchestra.run.smithi190:osd.5> sudo journalctl -f -n 0 -u ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968@osd.5.service 2024-08-30T08:54:37.034 INFO:tasks.cephadm:Deploying osd.6 on smithi042 with /dev/vg_nvme/lv_2... 2024-08-30T08:54:37.035 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-08-30T08:54:37.297 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T08:54:37.326 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:54:36 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-30T08:54:37.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:54:36 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:54:37.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:54:36 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:54:37.382 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:54:36 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-30T08:54:37.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:54:36 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:54:37.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:54:36 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:54:38.377 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:54:38 smithi190 ceph-mon[22001]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-08-30T08:54:38.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:54:38 smithi167 ceph-mon[22063]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-08-30T08:54:38.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:54:38 smithi042 ceph-mon[18652]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-08-30T08:54:38.686 INFO:journalctl@ceph.osd.5.smithi190.stdout:Aug 30 08:54:38 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-5[33866]: 2024-08-30T08:54:38.479+0000 7ffa6f751640 -1 osd.5 0 log_to_monitors true 2024-08-30T08:54:39.629 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:54:39 smithi042 ceph-mon[18652]: from='osd.5 [v2:172.21.15.190:6808/2557145333,v1:172.21.15.190:6809/2557145333]' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-08-30T08:54:39.630 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:54:39 smithi042 ceph-mon[18652]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-08-30T08:54:39.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:54:39 smithi190 ceph-mon[22001]: from='osd.5 [v2:172.21.15.190:6808/2557145333,v1:172.21.15.190:6809/2557145333]' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-08-30T08:54:39.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:54:39 smithi190 ceph-mon[22001]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-08-30T08:54:39.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:54:39 smithi167 ceph-mon[22063]: from='osd.5 [v2:172.21.15.190:6808/2557145333,v1:172.21.15.190:6809/2557145333]' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-08-30T08:54:39.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:54:39 smithi167 ceph-mon[22063]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-08-30T08:54:40.134 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-08-30T08:54:40.159 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph orch daemon add osd smithi042:vg_nvme/lv_2 2024-08-30T08:54:40.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:54:40 smithi042 ceph-mon[18652]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-08-30T08:54:40.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:54:40 smithi042 ceph-mon[18652]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-08-30T08:54:40.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:54:40 smithi042 ceph-mon[18652]: from='osd.5 [v2:172.21.15.190:6808/2557145333,v1:172.21.15.190:6809/2557145333]' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi190", "root=default"]} : dispatch 2024-08-30T08:54:40.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:54:40 smithi042 ceph-mon[18652]: osdmap e33: 6 total, 5 up, 6 in 2024-08-30T08:54:40.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:54:40 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-08-30T08:54:40.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:54:40 smithi042 ceph-mon[18652]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi190", "root=default"]} : dispatch 2024-08-30T08:54:40.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:54:40 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:54:40.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:54:40 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:54:40.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:54:40 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-08-30T08:54:40.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:54:40 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-08-30T08:54:40.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:54:40 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:54:40.384 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:54:40 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:54:40.384 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:54:40 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-30T08:54:40.384 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:54:40 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:54:40.418 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T08:54:40.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:54:40 smithi190 ceph-mon[22001]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-08-30T08:54:40.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:54:40 smithi190 ceph-mon[22001]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-08-30T08:54:40.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:54:40 smithi190 ceph-mon[22001]: from='osd.5 [v2:172.21.15.190:6808/2557145333,v1:172.21.15.190:6809/2557145333]' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi190", "root=default"]} : dispatch 2024-08-30T08:54:40.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:54:40 smithi190 ceph-mon[22001]: osdmap e33: 6 total, 5 up, 6 in 2024-08-30T08:54:40.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:54:40 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-08-30T08:54:40.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:54:40 smithi190 ceph-mon[22001]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi190", "root=default"]} : dispatch 2024-08-30T08:54:40.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:54:40 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:54:40.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:54:40 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:54:40.687 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:54:40 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-08-30T08:54:40.687 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:54:40 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-08-30T08:54:40.687 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:54:40 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:54:40.687 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:54:40 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:54:40.687 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:54:40 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-30T08:54:40.687 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:54:40 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:54:40.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:54:40 smithi167 ceph-mon[22063]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-08-30T08:54:40.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:54:40 smithi167 ceph-mon[22063]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-08-30T08:54:40.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:54:40 smithi167 ceph-mon[22063]: from='osd.5 [v2:172.21.15.190:6808/2557145333,v1:172.21.15.190:6809/2557145333]' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi190", "root=default"]} : dispatch 2024-08-30T08:54:40.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:54:40 smithi167 ceph-mon[22063]: osdmap e33: 6 total, 5 up, 6 in 2024-08-30T08:54:40.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:54:40 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-08-30T08:54:40.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:54:40 smithi167 ceph-mon[22063]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi190", "root=default"]} : dispatch 2024-08-30T08:54:40.828 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:54:40 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:54:40.828 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:54:40 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:54:40.828 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:54:40 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-08-30T08:54:40.828 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:54:40 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-08-30T08:54:40.828 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:54:40 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:54:40.828 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:54:40 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:54:40.828 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:54:40 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-30T08:54:40.828 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:54:40 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:54:40.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:54:40.911+0000 7f2479e63640 1 -- 172.21.15.42:0/4016681887 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2474103590 msgr2=0x7f2474103a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:54:40.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:54:40.911+0000 7f2479e63640 1 --2- 172.21.15.42:0/4016681887 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2474103590 0x7f2474103a00 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f2464009ca0 tx=0x7f246402a950 comp rx=0 tx=0).stop 2024-08-30T08:54:40.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:54:40.911+0000 7f2479e63640 1 -- 172.21.15.42:0/4016681887 shutdown_connections 2024-08-30T08:54:40.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:54:40.911+0000 7f2479e63640 1 --2- 172.21.15.42:0/4016681887 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2474104620 0x7f2474104a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:54:40.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:54:40.911+0000 7f2479e63640 1 --2- 172.21.15.42:0/4016681887 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2474104fc0 0x7f247410b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:54:40.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:54:40.911+0000 7f2479e63640 1 --2- 172.21.15.42:0/4016681887 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2474103590 0x7f2474103a00 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:54:40.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:54:40.911+0000 7f2479e63640 1 -- 172.21.15.42:0/4016681887 >> 172.21.15.42:0/4016681887 conn(0x7f24740fef20 msgr2=0x7f24741013b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:54:40.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:54:40.911+0000 7f2479e63640 1 -- 172.21.15.42:0/4016681887 shutdown_connections 2024-08-30T08:54:40.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:54:40.911+0000 7f2479e63640 1 -- 172.21.15.42:0/4016681887 wait complete. 2024-08-30T08:54:40.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:54:40.912+0000 7f2479e63640 1 Processor -- start 2024-08-30T08:54:40.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:54:40.912+0000 7f2479e63640 1 -- start start 2024-08-30T08:54:40.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:54:40.912+0000 7f2479e63640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2474103590 0x7f2474076be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:54:40.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:54:40.912+0000 7f2479e63640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2474104620 0x7f24740754b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:54:40.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:54:40.912+0000 7f2479e63640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2474104fc0 0x7f24740759f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:54:40.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:54:40.912+0000 7f2479e63640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f247410f170 con 0x7f2474104fc0 2024-08-30T08:54:40.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:54:40.912+0000 7f2479e63640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f247410eff0 con 0x7f2474103590 2024-08-30T08:54:40.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:54:40.912+0000 7f2479e63640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f247410f2f0 con 0x7f2474104620 2024-08-30T08:54:40.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:54:40.912+0000 7f2473fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2474104fc0 0x7f24740759f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:54:40.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:54:40.913+0000 7f2472ffd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2474104620 0x7f24740754b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:54:40.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:54:40.913+0000 7f2473fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2474104fc0 0x7f24740759f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:48080/0 (socket says 172.21.15.42:48080) 2024-08-30T08:54:40.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:54:40.913+0000 7f2473fff640 1 -- 172.21.15.42:0/198323960 learned_addr learned my addr 172.21.15.42:0/198323960 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T08:54:40.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:54:40.913+0000 7f24737fe640 1 --2- 172.21.15.42:0/198323960 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2474103590 0x7f2474076be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:54:40.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:54:40.913+0000 7f2472ffd640 1 -- 172.21.15.42:0/198323960 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2474103590 msgr2=0x7f2474076be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:54:40.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:54:40.913+0000 7f2472ffd640 1 --2- 172.21.15.42:0/198323960 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2474103590 0x7f2474076be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:54:40.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:54:40.913+0000 7f2472ffd640 1 -- 172.21.15.42:0/198323960 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2474104fc0 msgr2=0x7f24740759f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:54:40.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:54:40.913+0000 7f2472ffd640 1 --2- 172.21.15.42:0/198323960 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2474104fc0 0x7f24740759f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:54:40.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:54:40.913+0000 7f2472ffd640 1 -- 172.21.15.42:0/198323960 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24740760f0 con 0x7f2474104620 2024-08-30T08:54:40.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:54:40.913+0000 7f2473fff640 1 --2- 172.21.15.42:0/198323960 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2474104fc0 0x7f24740759f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-30T08:54:40.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:54:40.913+0000 7f2472ffd640 1 --2- 172.21.15.42:0/198323960 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2474104620 0x7f24740754b0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f245c00e9f0 tx=0x7f245c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:54:40.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:54:40.913+0000 7f2470ff9640 1 -- 172.21.15.42:0/198323960 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f245c00ccc0 con 0x7f2474104620 2024-08-30T08:54:40.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:54:40.913+0000 7f2479e63640 1 -- 172.21.15.42:0/198323960 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24741a3e30 con 0x7f2474104620 2024-08-30T08:54:40.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:54:40.913+0000 7f2470ff9640 1 -- 172.21.15.42:0/198323960 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f245c005020 con 0x7f2474104620 2024-08-30T08:54:40.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:54:40.913+0000 7f2470ff9640 1 -- 172.21.15.42:0/198323960 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f245c004a00 con 0x7f2474104620 2024-08-30T08:54:40.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:54:40.913+0000 7f2479e63640 1 -- 172.21.15.42:0/198323960 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24741a41d0 con 0x7f2474104620 2024-08-30T08:54:40.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:54:40.914+0000 7f2479e63640 1 -- 172.21.15.42:0/198323960 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2474103a00 con 0x7f2474104620 2024-08-30T08:54:40.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:54:40.916+0000 7f2470ff9640 1 -- 172.21.15.42:0/198323960 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 16) ==== 152577+0+0 (secure 0 0 0) 0x7f245c004ba0 con 0x7f2474104620 2024-08-30T08:54:40.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:54:40.917+0000 7f2470ff9640 1 --2- 172.21.15.42:0/198323960 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f24600b4b30 0x7f24600b7040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:54:40.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:54:40.917+0000 7f24737fe640 1 --2- 172.21.15.42:0/198323960 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f24600b4b30 0x7f24600b7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:54:40.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:54:40.917+0000 7f2470ff9640 1 -- 172.21.15.42:0/198323960 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 4253+0+0 (secure 0 0 0) 0x7f245c0e4150 con 0x7f2474104620 2024-08-30T08:54:40.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:54:40.918+0000 7f24737fe640 1 --2- 172.21.15.42:0/198323960 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f24600b4b30 0x7f24600b7040 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f246402f8d0 tx=0x7f2464009360 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:54:40.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:54:40.924+0000 7f2470ff9640 1 -- 172.21.15.42:0/198323960 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f245c0ac2f0 con 0x7f2474104620 2024-08-30T08:54:41.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:54:41.120+0000 7f2479e63640 1 -- 172.21.15.42:0/198323960 --> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi042:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f24740635a0 con 0x7f24600b4b30 2024-08-30T08:54:41.354 INFO:journalctl@ceph.osd.5.smithi190.stdout:Aug 30 08:54:41 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-5[33866]: 2024-08-30T08:54:41.020+0000 7ffa6b6d3640 -1 osd.5 0 waiting for initial osdmap 2024-08-30T08:54:41.354 INFO:journalctl@ceph.osd.5.smithi190.stdout:Aug 30 08:54:41 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-5[33866]: 2024-08-30T08:54:41.026+0000 7ffa634c9640 -1 osd.5 34 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-30T08:54:41.382 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:54:41 smithi042 ceph-mon[18652]: Detected new or changed devices on smithi190 2024-08-30T08:54:41.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:54:41 smithi042 ceph-mon[18652]: Adjusting osd_memory_target on smithi190 to 8544M 2024-08-30T08:54:41.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:54:41 smithi042 ceph-mon[18652]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]': finished 2024-08-30T08:54:41.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:54:41 smithi042 ceph-mon[18652]: osdmap e34: 6 total, 5 up, 6 in 2024-08-30T08:54:41.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:54:41 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-08-30T08:54:41.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:54:41 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-08-30T08:54:41.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:54:41 smithi042 ceph-mon[18652]: from='osd.5 ' entity='osd.5' 2024-08-30T08:54:41.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:54:41 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-30T08:54:41.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:54:41 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-30T08:54:41.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:54:41 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:54:41.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:54:41 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-08-30T08:54:41.649 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:54:41 smithi190 ceph-mon[22001]: Detected new or changed devices on smithi190 2024-08-30T08:54:41.649 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:54:41 smithi190 ceph-mon[22001]: Adjusting osd_memory_target on smithi190 to 8544M 2024-08-30T08:54:41.649 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:54:41 smithi190 ceph-mon[22001]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]': finished 2024-08-30T08:54:41.650 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:54:41 smithi190 ceph-mon[22001]: osdmap e34: 6 total, 5 up, 6 in 2024-08-30T08:54:41.650 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:54:41 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-08-30T08:54:41.650 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:54:41 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-08-30T08:54:41.650 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:54:41 smithi190 ceph-mon[22001]: from='osd.5 ' entity='osd.5' 2024-08-30T08:54:41.650 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:54:41 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-30T08:54:41.650 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:54:41 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-30T08:54:41.650 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:54:41 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:54:41.650 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:54:41 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-08-30T08:54:41.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:54:41 smithi167 ceph-mon[22063]: Detected new or changed devices on smithi190 2024-08-30T08:54:41.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:54:41 smithi167 ceph-mon[22063]: Adjusting osd_memory_target on smithi190 to 8544M 2024-08-30T08:54:41.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:54:41 smithi167 ceph-mon[22063]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]': finished 2024-08-30T08:54:41.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:54:41 smithi167 ceph-mon[22063]: osdmap e34: 6 total, 5 up, 6 in 2024-08-30T08:54:41.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:54:41 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-08-30T08:54:41.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:54:41 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-08-30T08:54:41.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:54:41 smithi167 ceph-mon[22063]: from='osd.5 ' entity='osd.5' 2024-08-30T08:54:41.828 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:54:41 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-30T08:54:41.828 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:54:41 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-30T08:54:41.828 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:54:41 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:54:41.828 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:54:41 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-08-30T08:54:42.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:54:42 smithi042 ceph-mon[18652]: purged_snaps scrub starts 2024-08-30T08:54:42.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:54:42 smithi042 ceph-mon[18652]: purged_snaps scrub ok 2024-08-30T08:54:42.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:54:42 smithi042 ceph-mon[18652]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-08-30T08:54:42.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:54:42 smithi042 ceph-mon[18652]: from='client.24379 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi042:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-08-30T08:54:42.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:54:42 smithi042 ceph-mon[18652]: osd.5 [v2:172.21.15.190:6808/2557145333,v1:172.21.15.190:6809/2557145333] boot 2024-08-30T08:54:42.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:54:42 smithi042 ceph-mon[18652]: osdmap e35: 6 total, 6 up, 6 in 2024-08-30T08:54:42.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:54:42 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-08-30T08:54:42.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:54:42 smithi190 ceph-mon[22001]: purged_snaps scrub starts 2024-08-30T08:54:42.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:54:42 smithi190 ceph-mon[22001]: purged_snaps scrub ok 2024-08-30T08:54:42.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:54:42 smithi190 ceph-mon[22001]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-08-30T08:54:42.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:54:42 smithi190 ceph-mon[22001]: from='client.24379 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi042:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-08-30T08:54:42.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:54:42 smithi190 ceph-mon[22001]: osd.5 [v2:172.21.15.190:6808/2557145333,v1:172.21.15.190:6809/2557145333] boot 2024-08-30T08:54:42.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:54:42 smithi190 ceph-mon[22001]: osdmap e35: 6 total, 6 up, 6 in 2024-08-30T08:54:42.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:54:42 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-08-30T08:54:42.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:54:42 smithi167 ceph-mon[22063]: purged_snaps scrub starts 2024-08-30T08:54:42.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:54:42 smithi167 ceph-mon[22063]: purged_snaps scrub ok 2024-08-30T08:54:42.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:54:42 smithi167 ceph-mon[22063]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-08-30T08:54:42.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:54:42 smithi167 ceph-mon[22063]: from='client.24379 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi042:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-08-30T08:54:42.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:54:42 smithi167 ceph-mon[22063]: osd.5 [v2:172.21.15.190:6808/2557145333,v1:172.21.15.190:6809/2557145333] boot 2024-08-30T08:54:42.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:54:42 smithi167 ceph-mon[22063]: osdmap e35: 6 total, 6 up, 6 in 2024-08-30T08:54:42.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:54:42 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-08-30T08:54:43.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:54:43 smithi042 ceph-mon[18652]: osdmap e36: 6 total, 6 up, 6 in 2024-08-30T08:54:43.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:54:43 smithi190 ceph-mon[22001]: osdmap e36: 6 total, 6 up, 6 in 2024-08-30T08:54:43.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:54:43 smithi167 ceph-mon[22063]: osdmap e36: 6 total, 6 up, 6 in 2024-08-30T08:54:44.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:54:44 smithi042 ceph-mon[18652]: pgmap v145: 1 pgs: 1 peering; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-08-30T08:54:44.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:54:44 smithi042 ceph-mon[18652]: osdmap e37: 6 total, 6 up, 6 in 2024-08-30T08:54:44.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:54:44 smithi042 ceph-mon[18652]: from='client.? 172.21.15.42:0/1528186432' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "96ad64af-44ff-4197-97d4-156f0cb88e81"} : dispatch 2024-08-30T08:54:44.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:54:44 smithi042 ceph-mon[18652]: from='client.? 172.21.15.42:0/1528186432' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "96ad64af-44ff-4197-97d4-156f0cb88e81"}]': finished 2024-08-30T08:54:44.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:54:44 smithi042 ceph-mon[18652]: osdmap e38: 7 total, 6 up, 7 in 2024-08-30T08:54:44.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:54:44 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-08-30T08:54:44.634 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:54:44 smithi042 ceph-mon[18652]: from='client.? 172.21.15.42:0/2863227952' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-30T08:54:44.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:54:44 smithi190 ceph-mon[22001]: pgmap v145: 1 pgs: 1 peering; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-08-30T08:54:44.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:54:44 smithi190 ceph-mon[22001]: osdmap e37: 6 total, 6 up, 6 in 2024-08-30T08:54:44.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:54:44 smithi190 ceph-mon[22001]: from='client.? 172.21.15.42:0/1528186432' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "96ad64af-44ff-4197-97d4-156f0cb88e81"} : dispatch 2024-08-30T08:54:44.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:54:44 smithi190 ceph-mon[22001]: from='client.? 172.21.15.42:0/1528186432' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "96ad64af-44ff-4197-97d4-156f0cb88e81"}]': finished 2024-08-30T08:54:44.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:54:44 smithi190 ceph-mon[22001]: osdmap e38: 7 total, 6 up, 7 in 2024-08-30T08:54:44.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:54:44 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-08-30T08:54:44.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:54:44 smithi190 ceph-mon[22001]: from='client.? 172.21.15.42:0/2863227952' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-30T08:54:44.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:54:44 smithi167 ceph-mon[22063]: pgmap v145: 1 pgs: 1 peering; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-08-30T08:54:44.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:54:44 smithi167 ceph-mon[22063]: osdmap e37: 6 total, 6 up, 6 in 2024-08-30T08:54:44.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:54:44 smithi167 ceph-mon[22063]: from='client.? 172.21.15.42:0/1528186432' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "96ad64af-44ff-4197-97d4-156f0cb88e81"} : dispatch 2024-08-30T08:54:44.828 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:54:44 smithi167 ceph-mon[22063]: from='client.? 172.21.15.42:0/1528186432' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "96ad64af-44ff-4197-97d4-156f0cb88e81"}]': finished 2024-08-30T08:54:44.828 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:54:44 smithi167 ceph-mon[22063]: osdmap e38: 7 total, 6 up, 7 in 2024-08-30T08:54:44.828 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:54:44 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-08-30T08:54:44.828 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:54:44 smithi167 ceph-mon[22063]: from='client.? 172.21.15.42:0/2863227952' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-30T08:54:46.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:54:46 smithi042 ceph-mon[18652]: pgmap v148: 1 pgs: 1 peering; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-08-30T08:54:46.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:54:46 smithi190 ceph-mon[22001]: pgmap v148: 1 pgs: 1 peering; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-08-30T08:54:46.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:54:46 smithi167 ceph-mon[22063]: pgmap v148: 1 pgs: 1 peering; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-08-30T08:54:48.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:54:48 smithi042 ceph-mon[18652]: pgmap v149: 1 pgs: 1 peering; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-08-30T08:54:48.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:54:48 smithi190 ceph-mon[22001]: pgmap v149: 1 pgs: 1 peering; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-08-30T08:54:48.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:54:48 smithi167 ceph-mon[22063]: pgmap v149: 1 pgs: 1 peering; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-08-30T08:54:50.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:54:50 smithi042 ceph-mon[18652]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail; 89 KiB/s, 0 objects/s recovering 2024-08-30T08:54:50.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:54:50 smithi190 ceph-mon[22001]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail; 89 KiB/s, 0 objects/s recovering 2024-08-30T08:54:50.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:54:50 smithi167 ceph-mon[22063]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail; 89 KiB/s, 0 objects/s recovering 2024-08-30T08:54:52.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:54:52 smithi042 ceph-mon[18652]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-08-30T08:54:52.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:54:52 smithi190 ceph-mon[22001]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-08-30T08:54:52.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:54:52 smithi167 ceph-mon[22063]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-08-30T08:54:53.482 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:54:53 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.6"} : dispatch 2024-08-30T08:54:53.482 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:54:53 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:54:53.483 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:54:53 smithi042 ceph-mon[18652]: Deploying daemon osd.6 on smithi042 2024-08-30T08:54:53.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:54:53 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.6"} : dispatch 2024-08-30T08:54:53.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:54:53 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:54:53.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:54:53 smithi190 ceph-mon[22001]: Deploying daemon osd.6 on smithi042 2024-08-30T08:54:53.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:54:53 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.6"} : dispatch 2024-08-30T08:54:53.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:54:53 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:54:53.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:54:53 smithi167 ceph-mon[22063]: Deploying daemon osd.6 on smithi042 2024-08-30T08:54:54.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:54:54 smithi042 ceph-mon[18652]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-08-30T08:54:54.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:54:54 smithi190 ceph-mon[22001]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-08-30T08:54:54.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:54:54 smithi167 ceph-mon[22063]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-08-30T08:54:56.551 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:54:56 smithi042 ceph-mon[18652]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-08-30T08:54:56.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:54:56 smithi190 ceph-mon[22001]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-08-30T08:54:56.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:54:56 smithi167 ceph-mon[22063]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-08-30T08:54:58.515 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:54:58 smithi042 ceph-mon[18652]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-08-30T08:54:58.515 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:54:58 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-30T08:54:58.515 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:54:58 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:54:58.515 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:54:58 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:54:58.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:54:58 smithi190 ceph-mon[22001]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-08-30T08:54:58.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:54:58 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-30T08:54:58.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:54:58 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:54:58.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:54:58 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:54:58.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:54:58 smithi167 ceph-mon[22063]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-08-30T08:54:58.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:54:58 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-30T08:54:58.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:54:58 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:54:58.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:54:58 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:55:00.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:55:00 smithi190 ceph-mon[22001]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-08-30T08:55:00.696 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:55:00 smithi042 ceph-mon[18652]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-08-30T08:55:00.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:55:00 smithi167 ceph-mon[22063]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-08-30T08:55:01.815 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:55:01 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:55:01.815 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:55:01 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:55:01.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:55:01 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:55:01.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:55:01 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-30T08:55:01.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:55:01 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:55:01.853 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:55:01.851+0000 7f2470ff9640 1 -- 172.21.15.42:0/198323960 <== mgr.14150 v2:172.21.15.42:6800/3005140902 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f24740635a0 con 0x7f24600b4b30 2024-08-30T08:55:01.853 INFO:teuthology.orchestra.run.smithi042.stdout:Created osd(s) 6 on host 'smithi042' 2024-08-30T08:55:01.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:55:01.853+0000 7f2479e63640 1 -- 172.21.15.42:0/198323960 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f24600b4b30 msgr2=0x7f24600b7040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:55:01.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:55:01.853+0000 7f2479e63640 1 --2- 172.21.15.42:0/198323960 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f24600b4b30 0x7f24600b7040 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f246402f8d0 tx=0x7f2464009360 comp rx=0 tx=0).stop 2024-08-30T08:55:01.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:55:01.853+0000 7f2479e63640 1 -- 172.21.15.42:0/198323960 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2474104620 msgr2=0x7f24740754b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:55:01.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:55:01.853+0000 7f2479e63640 1 --2- 172.21.15.42:0/198323960 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2474104620 0x7f24740754b0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f245c00e9f0 tx=0x7f245c00eeb0 comp rx=0 tx=0).stop 2024-08-30T08:55:01.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:55:01.853+0000 7f2479e63640 1 -- 172.21.15.42:0/198323960 shutdown_connections 2024-08-30T08:55:01.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:55:01.853+0000 7f2479e63640 1 --2- 172.21.15.42:0/198323960 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f24600b4b30 0x7f24600b7040 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:55:01.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:55:01.853+0000 7f2479e63640 1 --2- 172.21.15.42:0/198323960 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2474104620 0x7f24740754b0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:55:01.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:55:01.853+0000 7f2479e63640 1 --2- 172.21.15.42:0/198323960 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2474104fc0 0x7f24740759f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:55:01.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:55:01.853+0000 7f2479e63640 1 --2- 172.21.15.42:0/198323960 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2474103590 0x7f2474076be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:55:01.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:55:01.853+0000 7f2479e63640 1 -- 172.21.15.42:0/198323960 >> 172.21.15.42:0/198323960 conn(0x7f24740fef20 msgr2=0x7f2474107940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:55:01.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:55:01.853+0000 7f2479e63640 1 -- 172.21.15.42:0/198323960 shutdown_connections 2024-08-30T08:55:01.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:55:01.853+0000 7f2479e63640 1 -- 172.21.15.42:0/198323960 wait complete. 2024-08-30T08:55:01.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:55:01 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:55:01.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:55:01 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:55:01.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:55:01 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:55:01.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:55:01 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-30T08:55:01.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:55:01 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:55:02.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:55:01 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:55:02.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:55:01 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:55:02.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:55:01 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:55:02.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:55:01 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-30T08:55:02.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:55:01 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:55:02.481 DEBUG:teuthology.orchestra.run.smithi042:osd.6> sudo journalctl -f -n 0 -u ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968@osd.6.service 2024-08-30T08:55:02.483 INFO:tasks.cephadm:Deploying osd.7 on smithi167 with /dev/vg_nvme/lv_2... 2024-08-30T08:55:02.483 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-08-30T08:55:02.703 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:55:02 smithi042 ceph-mon[18652]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-08-30T08:55:02.704 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:55:02 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-30T08:55:02.704 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:55:02 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:55:02.704 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:55:02 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:55:02.738 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.b/config 2024-08-30T08:55:02.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:55:02 smithi167 ceph-mon[22063]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-08-30T08:55:02.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:55:02 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-30T08:55:02.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:55:02 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:55:02.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:55:02 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:55:02.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:55:02 smithi190 ceph-mon[22001]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-08-30T08:55:02.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:55:02 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-30T08:55:02.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:55:02 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:55:02.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:55:02 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:55:04.031 INFO:journalctl@ceph.osd.6.smithi042.stdout:Aug 30 08:55:03 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-6[48371]: 2024-08-30T08:55:03.893+0000 7fcbcb245640 -1 osd.6 0 log_to_monitors true 2024-08-30T08:55:04.767 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:55:04 smithi167 ceph-mon[22063]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-08-30T08:55:04.767 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:55:04 smithi167 ceph-mon[22063]: from='osd.6 [v2:172.21.15.42:6818/3213526100,v1:172.21.15.42:6819/3213526100]' entity='osd.6' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]} : dispatch 2024-08-30T08:55:04.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:55:04 smithi190 ceph-mon[22001]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-08-30T08:55:04.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:55:04 smithi190 ceph-mon[22001]: from='osd.6 [v2:172.21.15.42:6818/3213526100,v1:172.21.15.42:6819/3213526100]' entity='osd.6' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]} : dispatch 2024-08-30T08:55:05.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:55:04 smithi042 ceph-mon[18652]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-08-30T08:55:05.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:55:04 smithi042 ceph-mon[18652]: from='osd.6 [v2:172.21.15.42:6818/3213526100,v1:172.21.15.42:6819/3213526100]' entity='osd.6' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]} : dispatch 2024-08-30T08:55:05.428 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-08-30T08:55:05.456 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph orch daemon add osd smithi167:vg_nvme/lv_2 2024-08-30T08:55:05.710 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.b/config 2024-08-30T08:55:05.883 INFO:journalctl@ceph.osd.6.smithi042.stdout:Aug 30 08:55:05 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-6[48371]: 2024-08-30T08:55:05.673+0000 7fcbc71c7640 -1 osd.6 0 waiting for initial osdmap 2024-08-30T08:55:05.883 INFO:journalctl@ceph.osd.6.smithi042.stdout:Aug 30 08:55:05 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-6[48371]: 2024-08-30T08:55:05.681+0000 7fcbc2fc5640 -1 osd.6 40 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-30T08:55:05.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:55:05 smithi042 ceph-mon[18652]: from='osd.6 [v2:172.21.15.42:6818/3213526100,v1:172.21.15.42:6819/3213526100]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-08-30T08:55:05.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:55:05 smithi042 ceph-mon[18652]: osdmap e39: 7 total, 6 up, 7 in 2024-08-30T08:55:05.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:55:05 smithi042 ceph-mon[18652]: from='osd.6 [v2:172.21.15.42:6818/3213526100,v1:172.21.15.42:6819/3213526100]' entity='osd.6' cmd={"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi042", "root=default"]} : dispatch 2024-08-30T08:55:05.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:55:05 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-08-30T08:55:05.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:55:05 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:55:05.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:55:05 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:55:05.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:55:05 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-08-30T08:55:05.884 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:55:05 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-08-30T08:55:05.884 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:55:05 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-08-30T08:55:05.884 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:55:05 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:55:05.884 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:55:05 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:55:05.884 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:55:05 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-30T08:55:05.884 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:55:05 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:55:05.884 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:55:05 smithi042 ceph-mon[18652]: from='osd.6 [v2:172.21.15.42:6818/3213526100,v1:172.21.15.42:6819/3213526100]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi042", "root=default"]}]': finished 2024-08-30T08:55:05.884 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:55:05 smithi042 ceph-mon[18652]: osdmap e40: 7 total, 6 up, 7 in 2024-08-30T08:55:05.884 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:55:05 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-08-30T08:55:05.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:55:05 smithi190 ceph-mon[22001]: from='osd.6 [v2:172.21.15.42:6818/3213526100,v1:172.21.15.42:6819/3213526100]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-08-30T08:55:05.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:55:05 smithi190 ceph-mon[22001]: osdmap e39: 7 total, 6 up, 7 in 2024-08-30T08:55:05.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:55:05 smithi190 ceph-mon[22001]: from='osd.6 [v2:172.21.15.42:6818/3213526100,v1:172.21.15.42:6819/3213526100]' entity='osd.6' cmd={"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi042", "root=default"]} : dispatch 2024-08-30T08:55:05.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:55:05 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-08-30T08:55:05.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:55:05 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:55:05.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:55:05 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:55:05.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:55:05 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-08-30T08:55:05.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:55:05 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-08-30T08:55:05.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:55:05 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-08-30T08:55:05.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:55:05 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:55:05.937 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:55:05 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:55:05.937 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:55:05 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-30T08:55:05.937 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:55:05 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:55:05.937 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:55:05 smithi190 ceph-mon[22001]: from='osd.6 [v2:172.21.15.42:6818/3213526100,v1:172.21.15.42:6819/3213526100]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi042", "root=default"]}]': finished 2024-08-30T08:55:05.937 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:55:05 smithi190 ceph-mon[22001]: osdmap e40: 7 total, 6 up, 7 in 2024-08-30T08:55:05.937 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:55:05 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-08-30T08:55:05.979 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:55:05 smithi167 ceph-mon[22063]: from='osd.6 [v2:172.21.15.42:6818/3213526100,v1:172.21.15.42:6819/3213526100]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-08-30T08:55:05.980 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:55:05 smithi167 ceph-mon[22063]: osdmap e39: 7 total, 6 up, 7 in 2024-08-30T08:55:05.980 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:55:05 smithi167 ceph-mon[22063]: from='osd.6 [v2:172.21.15.42:6818/3213526100,v1:172.21.15.42:6819/3213526100]' entity='osd.6' cmd={"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi042", "root=default"]} : dispatch 2024-08-30T08:55:05.980 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:55:05 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-08-30T08:55:05.980 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:55:05 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:55:05.980 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:55:05 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:55:05.980 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:55:05 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-08-30T08:55:05.980 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:55:05 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-08-30T08:55:05.980 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:55:05 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-08-30T08:55:05.980 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:55:05 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:55:05.980 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:55:05 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:55:05.980 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:55:05 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-30T08:55:05.980 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:55:05 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:55:05.981 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:55:05 smithi167 ceph-mon[22063]: from='osd.6 [v2:172.21.15.42:6818/3213526100,v1:172.21.15.42:6819/3213526100]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi042", "root=default"]}]': finished 2024-08-30T08:55:05.981 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:55:05 smithi167 ceph-mon[22063]: osdmap e40: 7 total, 6 up, 7 in 2024-08-30T08:55:05.981 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:55:05 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-08-30T08:55:06.203 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:55:06.195+0000 7f66302d3640 1 -- 172.21.15.167:0/2126743485 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6628104f30 msgr2=0x7f662810b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:55:06.203 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:55:06.195+0000 7f66302d3640 1 --2- 172.21.15.167:0/2126743485 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6628104f30 0x7f662810b8d0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f6620009ca0 tx=0x7f662002a950 comp rx=0 tx=0).stop 2024-08-30T08:55:06.204 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:55:06.195+0000 7f66302d3640 1 -- 172.21.15.167:0/2126743485 shutdown_connections 2024-08-30T08:55:06.204 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:55:06.195+0000 7f66302d3640 1 --2- 172.21.15.167:0/2126743485 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6628104590 0x7f66281049a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:55:06.204 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:55:06.195+0000 7f66302d3640 1 --2- 172.21.15.167:0/2126743485 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6628104f30 0x7f662810b8d0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:55:06.204 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:55:06.195+0000 7f66302d3640 1 --2- 172.21.15.167:0/2126743485 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6628103500 0x7f6628103970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:55:06.204 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:55:06.195+0000 7f66302d3640 1 -- 172.21.15.167:0/2126743485 >> 172.21.15.167:0/2126743485 conn(0x7f66280fee50 msgr2=0x7f6628101320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:55:06.204 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:55:06.195+0000 7f66302d3640 1 -- 172.21.15.167:0/2126743485 shutdown_connections 2024-08-30T08:55:06.204 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:55:06.195+0000 7f66302d3640 1 -- 172.21.15.167:0/2126743485 wait complete. 2024-08-30T08:55:06.204 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:55:06.196+0000 7f66302d3640 1 Processor -- start 2024-08-30T08:55:06.204 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:55:06.196+0000 7f66302d3640 1 -- start start 2024-08-30T08:55:06.204 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:55:06.196+0000 7f66302d3640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6628103500 0x7f6628199d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:55:06.204 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:55:06.196+0000 7f66302d3640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6628104590 0x7f662819a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:55:06.205 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:55:06.196+0000 7f66302d3640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6628104f30 0x7f662819afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:55:06.205 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:55:06.196+0000 7f66302d3640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f662810f140 con 0x7f6628103500 2024-08-30T08:55:06.205 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:55:06.196+0000 7f66302d3640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f662810efc0 con 0x7f6628104590 2024-08-30T08:55:06.205 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:55:06.196+0000 7f66302d3640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f662810f2c0 con 0x7f6628104f30 2024-08-30T08:55:06.205 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:55:06.196+0000 7f662d86e640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6628104590 0x7f662819a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:55:06.205 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:55:06.196+0000 7f662d86e640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6628104590 0x7f662819a400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:55470/0 (socket says 172.21.15.167:55470) 2024-08-30T08:55:06.205 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:55:06.196+0000 7f662d86e640 1 -- 172.21.15.167:0/3599141545 learned_addr learned my addr 172.21.15.167:0/3599141545 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-08-30T08:55:06.205 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:55:06.197+0000 7f662e870640 1 --2- 172.21.15.167:0/3599141545 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6628104f30 0x7f662819afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:55:06.205 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:55:06.197+0000 7f662e06f640 1 --2- 172.21.15.167:0/3599141545 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6628103500 0x7f6628199d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:55:06.205 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:55:06.197+0000 7f662d86e640 1 -- 172.21.15.167:0/3599141545 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6628104f30 msgr2=0x7f662819afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:55:06.205 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:55:06.197+0000 7f662d86e640 1 --2- 172.21.15.167:0/3599141545 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6628104f30 0x7f662819afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:55:06.205 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:55:06.197+0000 7f662d86e640 1 -- 172.21.15.167:0/3599141545 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6628103500 msgr2=0x7f6628199d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:55:06.206 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:55:06.197+0000 7f662d86e640 1 --2- 172.21.15.167:0/3599141545 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6628103500 0x7f6628199d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:55:06.206 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:55:06.197+0000 7f662d86e640 1 -- 172.21.15.167:0/3599141545 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f662819b7c0 con 0x7f6628104590 2024-08-30T08:55:06.206 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:55:06.197+0000 7f662e870640 1 --2- 172.21.15.167:0/3599141545 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6628104f30 0x7f662819afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-30T08:55:06.206 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:55:06.197+0000 7f662d86e640 1 --2- 172.21.15.167:0/3599141545 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6628104590 0x7f662819a400 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f661800c970 tx=0x7f661800ce30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:55:06.206 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:55:06.197+0000 7f661f7fe640 1 -- 172.21.15.167:0/3599141545 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6618007bb0 con 0x7f6628104590 2024-08-30T08:55:06.206 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:55:06.197+0000 7f661f7fe640 1 -- 172.21.15.167:0/3599141545 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6618007d50 con 0x7f6628104590 2024-08-30T08:55:06.206 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:55:06.197+0000 7f66302d3640 1 -- 172.21.15.167:0/3599141545 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f662819bab0 con 0x7f6628104590 2024-08-30T08:55:06.206 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:55:06.197+0000 7f661f7fe640 1 -- 172.21.15.167:0/3599141545 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f661800f450 con 0x7f6628104590 2024-08-30T08:55:06.206 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:55:06.197+0000 7f66302d3640 1 -- 172.21.15.167:0/3599141545 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66281a21b0 con 0x7f6628104590 2024-08-30T08:55:06.206 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:55:06.198+0000 7f66302d3640 1 -- 172.21.15.167:0/3599141545 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f662810a6c0 con 0x7f6628104590 2024-08-30T08:55:06.206 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:55:06.201+0000 7f661f7fe640 1 -- 172.21.15.167:0/3599141545 <== mon.2 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 16) ==== 152577+0+0 (secure 0 0 0) 0x7f6618004f00 con 0x7f6628104590 2024-08-30T08:55:06.206 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:55:06.201+0000 7f661f7fe640 1 --2- 172.21.15.167:0/3599141545 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f66140b4b30 0x7f66140b7040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:55:06.206 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:55:06.201+0000 7f661f7fe640 1 -- 172.21.15.167:0/3599141545 <== mon.2 v2:172.21.15.167:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 4685+0+0 (secure 0 0 0) 0x7f66180e44c0 con 0x7f6628104590 2024-08-30T08:55:06.207 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:55:06.202+0000 7f662e06f640 1 --2- 172.21.15.167:0/3599141545 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f66140b4b30 0x7f66140b7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:55:06.207 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:55:06.202+0000 7f662e06f640 1 --2- 172.21.15.167:0/3599141545 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f66140b4b30 0x7f66140b7040 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f6624005fd0 tx=0x7f6624005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:55:06.207 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:55:06.204+0000 7f661f7fe640 1 -- 172.21.15.167:0/3599141545 <== mon.2 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f6618014030 con 0x7f6628104590 2024-08-30T08:55:06.396 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:55:06.394+0000 7f66302d3640 1 -- 172.21.15.167:0/3599141545 --> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f662802e810 con 0x7f66140b4b30 2024-08-30T08:55:06.932 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:55:06 smithi167 ceph-mon[22063]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-08-30T08:55:06.933 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:55:06 smithi167 ceph-mon[22063]: Detected new or changed devices on smithi042 2024-08-30T08:55:06.933 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:55:06 smithi167 ceph-mon[22063]: Adjusting osd_memory_target on smithi042 to 5697M 2024-08-30T08:55:06.933 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:55:06 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-08-30T08:55:06.933 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:55:06 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-30T08:55:06.933 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:55:06 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-30T08:55:06.933 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:55:06 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:55:06.933 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:55:06 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-08-30T08:55:06.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:55:06 smithi190 ceph-mon[22001]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-08-30T08:55:06.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:55:06 smithi190 ceph-mon[22001]: Detected new or changed devices on smithi042 2024-08-30T08:55:06.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:55:06 smithi190 ceph-mon[22001]: Adjusting osd_memory_target on smithi042 to 5697M 2024-08-30T08:55:06.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:55:06 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-08-30T08:55:06.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:55:06 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-30T08:55:06.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:55:06 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-30T08:55:06.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:55:06 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:55:06.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:55:06 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-08-30T08:55:07.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:55:06 smithi042 ceph-mon[18652]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-08-30T08:55:07.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:55:06 smithi042 ceph-mon[18652]: Detected new or changed devices on smithi042 2024-08-30T08:55:07.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:55:06 smithi042 ceph-mon[18652]: Adjusting osd_memory_target on smithi042 to 5697M 2024-08-30T08:55:07.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:55:06 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-08-30T08:55:07.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:55:06 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-30T08:55:07.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:55:06 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-30T08:55:07.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:55:06 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:55:07.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:55:06 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-08-30T08:55:07.795 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:55:07 smithi167 ceph-mon[22063]: purged_snaps scrub starts 2024-08-30T08:55:07.795 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:55:07 smithi167 ceph-mon[22063]: purged_snaps scrub ok 2024-08-30T08:55:07.795 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:55:07 smithi167 ceph-mon[22063]: from='client.24395 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-08-30T08:55:07.795 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:55:07 smithi167 ceph-mon[22063]: osd.6 [v2:172.21.15.42:6818/3213526100,v1:172.21.15.42:6819/3213526100] boot 2024-08-30T08:55:07.795 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:55:07 smithi167 ceph-mon[22063]: osdmap e41: 7 total, 7 up, 7 in 2024-08-30T08:55:07.795 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:55:07 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-08-30T08:55:08.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:55:07 smithi042 ceph-mon[18652]: purged_snaps scrub starts 2024-08-30T08:55:08.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:55:07 smithi042 ceph-mon[18652]: purged_snaps scrub ok 2024-08-30T08:55:08.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:55:07 smithi042 ceph-mon[18652]: from='client.24395 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-08-30T08:55:08.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:55:07 smithi042 ceph-mon[18652]: osd.6 [v2:172.21.15.42:6818/3213526100,v1:172.21.15.42:6819/3213526100] boot 2024-08-30T08:55:08.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:55:07 smithi042 ceph-mon[18652]: osdmap e41: 7 total, 7 up, 7 in 2024-08-30T08:55:08.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:55:07 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-08-30T08:55:08.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:55:07 smithi190 ceph-mon[22001]: purged_snaps scrub starts 2024-08-30T08:55:08.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:55:07 smithi190 ceph-mon[22001]: purged_snaps scrub ok 2024-08-30T08:55:08.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:55:07 smithi190 ceph-mon[22001]: from='client.24395 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-08-30T08:55:08.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:55:07 smithi190 ceph-mon[22001]: osd.6 [v2:172.21.15.42:6818/3213526100,v1:172.21.15.42:6819/3213526100] boot 2024-08-30T08:55:08.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:55:07 smithi190 ceph-mon[22001]: osdmap e41: 7 total, 7 up, 7 in 2024-08-30T08:55:08.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:55:07 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-08-30T08:55:08.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:55:08 smithi167 ceph-mon[22063]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-08-30T08:55:08.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:55:08 smithi167 ceph-mon[22063]: osdmap e42: 7 total, 7 up, 7 in 2024-08-30T08:55:09.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:55:08 smithi042 ceph-mon[18652]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-08-30T08:55:09.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:55:08 smithi042 ceph-mon[18652]: osdmap e42: 7 total, 7 up, 7 in 2024-08-30T08:55:09.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:55:08 smithi190 ceph-mon[22001]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-08-30T08:55:09.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:55:08 smithi190 ceph-mon[22001]: osdmap e42: 7 total, 7 up, 7 in 2024-08-30T08:55:09.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:55:09 smithi167 ceph-mon[22063]: from='client.? 172.21.15.167:0/1472217257' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "337a2112-eaf7-4e6b-86be-4e40d6ddf835"} : dispatch 2024-08-30T08:55:09.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:55:09 smithi167 ceph-mon[22063]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "337a2112-eaf7-4e6b-86be-4e40d6ddf835"} : dispatch 2024-08-30T08:55:09.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:55:09 smithi167 ceph-mon[22063]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "337a2112-eaf7-4e6b-86be-4e40d6ddf835"}]': finished 2024-08-30T08:55:09.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:55:09 smithi167 ceph-mon[22063]: osdmap e43: 8 total, 7 up, 8 in 2024-08-30T08:55:09.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:55:09 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-08-30T08:55:09.828 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:55:09 smithi167 ceph-mon[22063]: from='client.? 172.21.15.167:0/459148147' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-30T08:55:10.087 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:55:09 smithi190 ceph-mon[22001]: from='client.? 172.21.15.167:0/1472217257' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "337a2112-eaf7-4e6b-86be-4e40d6ddf835"} : dispatch 2024-08-30T08:55:10.087 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:55:09 smithi190 ceph-mon[22001]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "337a2112-eaf7-4e6b-86be-4e40d6ddf835"} : dispatch 2024-08-30T08:55:10.088 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:55:09 smithi190 ceph-mon[22001]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "337a2112-eaf7-4e6b-86be-4e40d6ddf835"}]': finished 2024-08-30T08:55:10.088 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:55:09 smithi190 ceph-mon[22001]: osdmap e43: 8 total, 7 up, 8 in 2024-08-30T08:55:10.088 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:55:09 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-08-30T08:55:10.088 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:55:09 smithi190 ceph-mon[22001]: from='client.? 172.21.15.167:0/459148147' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-30T08:55:10.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:55:09 smithi042 ceph-mon[18652]: from='client.? 172.21.15.167:0/1472217257' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "337a2112-eaf7-4e6b-86be-4e40d6ddf835"} : dispatch 2024-08-30T08:55:10.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:55:09 smithi042 ceph-mon[18652]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "337a2112-eaf7-4e6b-86be-4e40d6ddf835"} : dispatch 2024-08-30T08:55:10.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:55:09 smithi042 ceph-mon[18652]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "337a2112-eaf7-4e6b-86be-4e40d6ddf835"}]': finished 2024-08-30T08:55:10.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:55:09 smithi042 ceph-mon[18652]: osdmap e43: 8 total, 7 up, 8 in 2024-08-30T08:55:10.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:55:09 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-08-30T08:55:10.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:55:09 smithi042 ceph-mon[18652]: from='client.? 172.21.15.167:0/459148147' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-30T08:55:11.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:55:10 smithi167 ceph-mon[22063]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 213 MiB used, 626 GiB / 626 GiB avail 2024-08-30T08:55:11.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:55:10 smithi042 ceph-mon[18652]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 213 MiB used, 626 GiB / 626 GiB avail 2024-08-30T08:55:11.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:55:10 smithi190 ceph-mon[22001]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 213 MiB used, 626 GiB / 626 GiB avail 2024-08-30T08:55:13.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:55:12 smithi167 ceph-mon[22063]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 213 MiB used, 626 GiB / 626 GiB avail 2024-08-30T08:55:13.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:55:12 smithi042 ceph-mon[18652]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 213 MiB used, 626 GiB / 626 GiB avail 2024-08-30T08:55:13.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:55:12 smithi190 ceph-mon[22001]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 213 MiB used, 626 GiB / 626 GiB avail 2024-08-30T08:55:15.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:55:14 smithi167 ceph-mon[22063]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-08-30T08:55:15.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:55:14 smithi042 ceph-mon[18652]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-08-30T08:55:15.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:55:14 smithi190 ceph-mon[22001]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-08-30T08:55:16.822 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:55:16 smithi167 ceph-mon[22063]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-08-30T08:55:17.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:55:16 smithi042 ceph-mon[18652]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-08-30T08:55:17.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:55:16 smithi190 ceph-mon[22001]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-08-30T08:55:17.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:55:17 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.7"} : dispatch 2024-08-30T08:55:17.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:55:17 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:55:18.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:55:17 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.7"} : dispatch 2024-08-30T08:55:18.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:55:17 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:55:18.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:55:17 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.7"} : dispatch 2024-08-30T08:55:18.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:55:17 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:55:18.936 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:55:18 smithi167 ceph-mon[22063]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-08-30T08:55:18.936 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:55:18 smithi167 ceph-mon[22063]: Deploying daemon osd.7 on smithi167 2024-08-30T08:55:19.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:55:18 smithi042 ceph-mon[18652]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-08-30T08:55:19.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:55:18 smithi042 ceph-mon[18652]: Deploying daemon osd.7 on smithi167 2024-08-30T08:55:19.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:55:18 smithi190 ceph-mon[22001]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-08-30T08:55:19.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:55:18 smithi190 ceph-mon[22001]: Deploying daemon osd.7 on smithi167 2024-08-30T08:55:19.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:55:19 smithi167 ceph-mon[22063]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-08-30T08:55:20.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:55:19 smithi042 ceph-mon[18652]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-08-30T08:55:20.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:55:19 smithi190 ceph-mon[22001]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-08-30T08:55:22.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:55:21 smithi167 ceph-mon[22063]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-08-30T08:55:22.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:55:21 smithi190 ceph-mon[22001]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-08-30T08:55:22.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:55:21 smithi042 ceph-mon[18652]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-08-30T08:55:24.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:55:23 smithi167 ceph-mon[22063]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-08-30T08:55:24.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:55:23 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-30T08:55:24.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:55:23 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:55:24.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:55:23 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:55:24.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:55:23 smithi042 ceph-mon[18652]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-08-30T08:55:24.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:55:23 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-30T08:55:24.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:55:23 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:55:24.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:55:23 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:55:24.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:55:23 smithi190 ceph-mon[22001]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-08-30T08:55:24.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:55:23 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-30T08:55:24.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:55:23 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:55:24.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:55:23 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:55:26.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:55:25 smithi167 ceph-mon[22063]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-08-30T08:55:26.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:55:25 smithi042 ceph-mon[18652]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-08-30T08:55:26.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:55:25 smithi190 ceph-mon[22001]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-08-30T08:55:27.053 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:55:27.050+0000 7f661f7fe640 1 -- 172.21.15.167:0/3599141545 <== mgr.14150 v2:172.21.15.42:6800/3005140902 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f662802e810 con 0x7f66140b4b30 2024-08-30T08:55:27.053 INFO:teuthology.orchestra.run.smithi167.stdout:Created osd(s) 7 on host 'smithi167' 2024-08-30T08:55:27.055 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:55:27.052+0000 7f66302d3640 1 -- 172.21.15.167:0/3599141545 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f66140b4b30 msgr2=0x7f66140b7040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:55:27.055 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:55:27.052+0000 7f66302d3640 1 --2- 172.21.15.167:0/3599141545 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f66140b4b30 0x7f66140b7040 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f6624005fd0 tx=0x7f6624005850 comp rx=0 tx=0).stop 2024-08-30T08:55:27.055 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:55:27.052+0000 7f66302d3640 1 -- 172.21.15.167:0/3599141545 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6628104590 msgr2=0x7f662819a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:55:27.055 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:55:27.052+0000 7f66302d3640 1 --2- 172.21.15.167:0/3599141545 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6628104590 0x7f662819a400 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f661800c970 tx=0x7f661800ce30 comp rx=0 tx=0).stop 2024-08-30T08:55:27.055 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:55:27.052+0000 7f66302d3640 1 -- 172.21.15.167:0/3599141545 shutdown_connections 2024-08-30T08:55:27.055 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:55:27.052+0000 7f66302d3640 1 --2- 172.21.15.167:0/3599141545 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f66140b4b30 0x7f66140b7040 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:55:27.055 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:55:27.052+0000 7f66302d3640 1 --2- 172.21.15.167:0/3599141545 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6628104f30 0x7f662819afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:55:27.055 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:55:27.052+0000 7f66302d3640 1 --2- 172.21.15.167:0/3599141545 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6628104590 0x7f662819a400 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:55:27.056 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:55:27.052+0000 7f66302d3640 1 --2- 172.21.15.167:0/3599141545 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6628103500 0x7f6628199d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:55:27.056 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:55:27.052+0000 7f66302d3640 1 -- 172.21.15.167:0/3599141545 >> 172.21.15.167:0/3599141545 conn(0x7f66280fee50 msgr2=0x7f6628107890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:55:27.056 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:55:27.053+0000 7f66302d3640 1 -- 172.21.15.167:0/3599141545 shutdown_connections 2024-08-30T08:55:27.056 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:55:27.053+0000 7f66302d3640 1 -- 172.21.15.167:0/3599141545 wait complete. 2024-08-30T08:55:27.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:55:27 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:55:27.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:55:27 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:55:27.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:55:27 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:55:27.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:55:27 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-30T08:55:27.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:55:27 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:55:27.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:55:27 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:55:27.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:55:27 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:55:27.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:55:27 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:55:27.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:55:27 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-30T08:55:27.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:55:27 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:55:27.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:55:27 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:55:27.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:55:27 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:55:27.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:55:27 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:55:27.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:55:27 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-30T08:55:27.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:55:27 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:55:27.761 DEBUG:teuthology.orchestra.run.smithi167:osd.7> sudo journalctl -f -n 0 -u ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968@osd.7.service 2024-08-30T08:55:27.763 INFO:tasks.cephadm:Deploying osd.8 on smithi190 with /dev/vg_nvme/lv_2... 2024-08-30T08:55:27.763 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-08-30T08:55:28.016 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.c/config 2024-08-30T08:55:28.137 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:55:28 smithi190 ceph-mon[22001]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-08-30T08:55:28.137 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:55:28 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-30T08:55:28.137 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:55:28 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:55:28.137 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:55:28 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:55:28.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:55:28 smithi167 ceph-mon[22063]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-08-30T08:55:28.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:55:28 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-30T08:55:28.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:55:28 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:55:28.328 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:55:28 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:55:28.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:55:28 smithi042 ceph-mon[18652]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-08-30T08:55:28.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:55:28 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-30T08:55:28.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:55:28 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:55:28.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:55:28 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:55:29.577 INFO:journalctl@ceph.osd.7.smithi167.stdout:Aug 30 08:55:29 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-7[40306]: 2024-08-30T08:55:29.348+0000 7f33ebfc8640 -1 osd.7 0 log_to_monitors true 2024-08-30T08:55:30.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:55:30 smithi190 ceph-mon[22001]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-08-30T08:55:30.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:55:30 smithi190 ceph-mon[22001]: from='osd.7 [v2:172.21.15.167:6816/2962799814,v1:172.21.15.167:6817/2962799814]' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-08-30T08:55:30.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:55:30 smithi190 ceph-mon[22001]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-08-30T08:55:30.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:55:30 smithi167 ceph-mon[22063]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-08-30T08:55:30.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:55:30 smithi167 ceph-mon[22063]: from='osd.7 [v2:172.21.15.167:6816/2962799814,v1:172.21.15.167:6817/2962799814]' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-08-30T08:55:30.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:55:30 smithi167 ceph-mon[22063]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-08-30T08:55:30.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:55:30 smithi042 ceph-mon[18652]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-08-30T08:55:30.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:55:30 smithi042 ceph-mon[18652]: from='osd.7 [v2:172.21.15.167:6816/2962799814,v1:172.21.15.167:6817/2962799814]' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-08-30T08:55:30.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:55:30 smithi042 ceph-mon[18652]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-08-30T08:55:30.657 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-08-30T08:55:30.684 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph orch daemon add osd smithi190:vg_nvme/lv_2 2024-08-30T08:55:30.930 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.c/config 2024-08-30T08:55:31.317 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:55:31 smithi190 ceph-mon[22001]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-08-30T08:55:31.317 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:55:31 smithi190 ceph-mon[22001]: from='osd.7 [v2:172.21.15.167:6816/2962799814,v1:172.21.15.167:6817/2962799814]' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi167", "root=default"]} : dispatch 2024-08-30T08:55:31.317 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:55:31 smithi190 ceph-mon[22001]: osdmap e44: 8 total, 7 up, 8 in 2024-08-30T08:55:31.317 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:55:31 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-08-30T08:55:31.317 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:55:31 smithi190 ceph-mon[22001]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi167", "root=default"]} : dispatch 2024-08-30T08:55:31.317 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:55:31 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:55:31.317 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:55:31 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:55:31.317 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:55:31 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-08-30T08:55:31.317 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:55:31 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-08-30T08:55:31.317 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:55:31 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-08-30T08:55:31.317 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:55:31 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:55:31.318 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:55:31 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:55:31.318 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:55:31 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-30T08:55:31.318 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:55:31 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:55:31.327 INFO:journalctl@ceph.osd.7.smithi167.stdout:Aug 30 08:55:31 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-7[40306]: 2024-08-30T08:55:31.040+0000 7f33e7f4a640 -1 osd.7 0 waiting for initial osdmap 2024-08-30T08:55:31.327 INFO:journalctl@ceph.osd.7.smithi167.stdout:Aug 30 08:55:31 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-7[40306]: 2024-08-30T08:55:31.048+0000 7f33e3d48640 -1 osd.7 45 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-30T08:55:31.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:55:31 smithi167 ceph-mon[22063]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-08-30T08:55:31.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:55:31 smithi167 ceph-mon[22063]: from='osd.7 [v2:172.21.15.167:6816/2962799814,v1:172.21.15.167:6817/2962799814]' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi167", "root=default"]} : dispatch 2024-08-30T08:55:31.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:55:31 smithi167 ceph-mon[22063]: osdmap e44: 8 total, 7 up, 8 in 2024-08-30T08:55:31.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:55:31 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-08-30T08:55:31.328 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:55:31 smithi167 ceph-mon[22063]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi167", "root=default"]} : dispatch 2024-08-30T08:55:31.328 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:55:31 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:55:31.328 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:55:31 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:55:31.328 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:55:31 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-08-30T08:55:31.328 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:55:31 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-08-30T08:55:31.328 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:55:31 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-08-30T08:55:31.328 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:55:31 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:55:31.328 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:55:31 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:55:31.328 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:55:31 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-30T08:55:31.328 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:55:31 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:55:31.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:55:31 smithi042 ceph-mon[18652]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-08-30T08:55:31.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:55:31 smithi042 ceph-mon[18652]: from='osd.7 [v2:172.21.15.167:6816/2962799814,v1:172.21.15.167:6817/2962799814]' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi167", "root=default"]} : dispatch 2024-08-30T08:55:31.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:55:31 smithi042 ceph-mon[18652]: osdmap e44: 8 total, 7 up, 8 in 2024-08-30T08:55:31.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:55:31 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-08-30T08:55:31.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:55:31 smithi042 ceph-mon[18652]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi167", "root=default"]} : dispatch 2024-08-30T08:55:31.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:55:31 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:55:31.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:55:31 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:55:31.384 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:55:31 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-08-30T08:55:31.384 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:55:31 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-08-30T08:55:31.384 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:55:31 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-08-30T08:55:31.384 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:55:31 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:55:31.384 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:55:31 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:55:31.384 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:55:31 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-30T08:55:31.384 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:55:31 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:55:31.433 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:55:31.407+0000 7ff681269640 1 -- 172.21.15.190:0/1919433300 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff67c103480 msgr2=0x7ff67c103890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:55:31.433 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:55:31.407+0000 7ff681269640 1 --2- 172.21.15.190:0/1919433300 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff67c103480 0x7ff67c103890 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7ff670009900 tx=0x7ff67002a480 comp rx=0 tx=0).stop 2024-08-30T08:55:31.433 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:55:31.407+0000 7ff681269640 1 -- 172.21.15.190:0/1919433300 shutdown_connections 2024-08-30T08:55:31.433 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:55:31.407+0000 7ff681269640 1 --2- 172.21.15.190:0/1919433300 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff67c103480 0x7ff67c103890 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:55:31.433 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:55:31.407+0000 7ff681269640 1 --2- 172.21.15.190:0/1919433300 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff67c103e20 0x7ff67c10a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:55:31.433 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:55:31.407+0000 7ff681269640 1 --2- 172.21.15.190:0/1919433300 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff67c1023f0 0x7ff67c102860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:55:31.433 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:55:31.407+0000 7ff681269640 1 -- 172.21.15.190:0/1919433300 >> 172.21.15.190:0/1919433300 conn(0x7ff67c0fdd80 msgr2=0x7ff67c100210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:55:31.433 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:55:31.407+0000 7ff681269640 1 -- 172.21.15.190:0/1919433300 shutdown_connections 2024-08-30T08:55:31.433 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:55:31.407+0000 7ff681269640 1 -- 172.21.15.190:0/1919433300 wait complete. 2024-08-30T08:55:31.433 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:55:31.408+0000 7ff681269640 1 Processor -- start 2024-08-30T08:55:31.434 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:55:31.408+0000 7ff681269640 1 -- start start 2024-08-30T08:55:31.434 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:55:31.408+0000 7ff681269640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff67c1023f0 0x7ff67c105fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:55:31.434 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:55:31.408+0000 7ff681269640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff67c103480 0x7ff67c1048b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:55:31.434 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:55:31.408+0000 7ff681269640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff67c103e20 0x7ff67c104df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:55:31.434 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:55:31.408+0000 7ff681269640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7ff67c10e050 con 0x7ff67c103e20 2024-08-30T08:55:31.434 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:55:31.408+0000 7ff681269640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff67c10ded0 con 0x7ff67c103480 2024-08-30T08:55:31.434 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:55:31.408+0000 7ff681269640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7ff67c10e1d0 con 0x7ff67c1023f0 2024-08-30T08:55:31.434 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:55:31.408+0000 7ff67affd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff67c1023f0 0x7ff67c105fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:55:31.434 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:55:31.408+0000 7ff67affd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff67c1023f0 0x7ff67c105fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:48498/0 (socket says 172.21.15.190:48498) 2024-08-30T08:55:31.434 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:55:31.408+0000 7ff67affd640 1 -- 172.21.15.190:0/3365461700 learned_addr learned my addr 172.21.15.190:0/3365461700 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-30T08:55:31.434 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:55:31.409+0000 7ff67b7fe640 1 --2- 172.21.15.190:0/3365461700 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff67c103e20 0x7ff67c104df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:55:31.434 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:55:31.409+0000 7ff67a7fc640 1 --2- 172.21.15.190:0/3365461700 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff67c103480 0x7ff67c1048b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:55:31.435 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:55:31.409+0000 7ff67affd640 1 -- 172.21.15.190:0/3365461700 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff67c103480 msgr2=0x7ff67c1048b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:55:31.435 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:55:31.409+0000 7ff67affd640 1 --2- 172.21.15.190:0/3365461700 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff67c103480 0x7ff67c1048b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:55:31.435 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:55:31.409+0000 7ff67affd640 1 -- 172.21.15.190:0/3365461700 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff67c103e20 msgr2=0x7ff67c104df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:55:31.435 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:55:31.409+0000 7ff67affd640 1 --2- 172.21.15.190:0/3365461700 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff67c103e20 0x7ff67c104df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:55:31.435 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:55:31.409+0000 7ff67affd640 1 -- 172.21.15.190:0/3365461700 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff67c1054f0 con 0x7ff67c1023f0 2024-08-30T08:55:31.435 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:55:31.409+0000 7ff67a7fc640 1 --2- 172.21.15.190:0/3365461700 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff67c103480 0x7ff67c1048b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-30T08:55:31.435 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:55:31.409+0000 7ff67affd640 1 --2- 172.21.15.190:0/3365461700 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff67c1023f0 0x7ff67c105fe0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7ff66c00d950 tx=0x7ff66c00de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:55:31.435 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:55:31.409+0000 7ff663fff640 1 -- 172.21.15.190:0/3365461700 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff66c017030 con 0x7ff67c1023f0 2024-08-30T08:55:31.435 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:55:31.409+0000 7ff681269640 1 -- 172.21.15.190:0/3365461700 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff67c1a3df0 con 0x7ff67c1023f0 2024-08-30T08:55:31.435 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:55:31.409+0000 7ff663fff640 1 -- 172.21.15.190:0/3365461700 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff66c004020 con 0x7ff67c1023f0 2024-08-30T08:55:31.435 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:55:31.409+0000 7ff663fff640 1 -- 172.21.15.190:0/3365461700 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff66c0047a0 con 0x7ff67c1023f0 2024-08-30T08:55:31.435 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:55:31.409+0000 7ff681269640 1 -- 172.21.15.190:0/3365461700 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff67c1a41e0 con 0x7ff67c1023f0 2024-08-30T08:55:31.435 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:55:31.410+0000 7ff681269640 1 -- 172.21.15.190:0/3365461700 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff67c102860 con 0x7ff67c1023f0 2024-08-30T08:55:31.436 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:55:31.412+0000 7ff663fff640 1 -- 172.21.15.190:0/3365461700 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 16) ==== 152577+0+0 (secure 0 0 0) 0x7ff66c009a30 con 0x7ff67c1023f0 2024-08-30T08:55:31.436 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:55:31.412+0000 7ff663fff640 1 --2- 172.21.15.190:0/3365461700 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7ff6640b4b30 0x7ff6640b7040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:55:31.436 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:55:31.413+0000 7ff663fff640 1 -- 172.21.15.190:0/3365461700 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5117+0+0 (secure 0 0 0) 0x7ff66c0e4620 con 0x7ff67c1023f0 2024-08-30T08:55:31.436 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:55:31.413+0000 7ff67a7fc640 1 --2- 172.21.15.190:0/3365461700 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7ff6640b4b30 0x7ff6640b7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:55:31.436 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:55:31.413+0000 7ff67a7fc640 1 --2- 172.21.15.190:0/3365461700 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7ff6640b4b30 0x7ff6640b7040 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7ff670031440 tx=0x7ff670005d50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:55:31.436 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:55:31.416+0000 7ff663fff640 1 -- 172.21.15.190:0/3365461700 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7ff66c0ac3d0 con 0x7ff67c1023f0 2024-08-30T08:55:31.607 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:55:31.605+0000 7ff681269640 1 -- 172.21.15.190:0/3365461700 --> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi190:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7ff67c1a44a0 con 0x7ff6640b4b30 2024-08-30T08:55:32.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:55:32 smithi167 ceph-mon[22063]: Detected new or changed devices on smithi167 2024-08-30T08:55:32.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:55:32 smithi167 ceph-mon[22063]: Adjusting osd_memory_target on smithi167 to 5696M 2024-08-30T08:55:32.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:55:32 smithi167 ceph-mon[22063]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-08-30T08:55:32.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:55:32 smithi167 ceph-mon[22063]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]': finished 2024-08-30T08:55:32.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:55:32 smithi167 ceph-mon[22063]: osdmap e45: 8 total, 7 up, 8 in 2024-08-30T08:55:32.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:55:32 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-08-30T08:55:32.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:55:32 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-08-30T08:55:32.328 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:55:32 smithi167 ceph-mon[22063]: from='client.24421 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi190:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-08-30T08:55:32.328 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:55:32 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-30T08:55:32.328 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:55:32 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-30T08:55:32.328 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:55:32 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:55:32.344 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:55:32 smithi190 ceph-mon[22001]: Detected new or changed devices on smithi167 2024-08-30T08:55:32.345 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:55:32 smithi190 ceph-mon[22001]: Adjusting osd_memory_target on smithi167 to 5696M 2024-08-30T08:55:32.345 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:55:32 smithi190 ceph-mon[22001]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-08-30T08:55:32.345 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:55:32 smithi190 ceph-mon[22001]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]': finished 2024-08-30T08:55:32.345 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:55:32 smithi190 ceph-mon[22001]: osdmap e45: 8 total, 7 up, 8 in 2024-08-30T08:55:32.345 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:55:32 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-08-30T08:55:32.345 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:55:32 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-08-30T08:55:32.345 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:55:32 smithi190 ceph-mon[22001]: from='client.24421 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi190:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-08-30T08:55:32.346 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:55:32 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-30T08:55:32.346 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:55:32 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-30T08:55:32.346 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:55:32 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:55:32.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:55:32 smithi042 ceph-mon[18652]: Detected new or changed devices on smithi167 2024-08-30T08:55:32.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:55:32 smithi042 ceph-mon[18652]: Adjusting osd_memory_target on smithi167 to 5696M 2024-08-30T08:55:32.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:55:32 smithi042 ceph-mon[18652]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-08-30T08:55:32.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:55:32 smithi042 ceph-mon[18652]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]': finished 2024-08-30T08:55:32.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:55:32 smithi042 ceph-mon[18652]: osdmap e45: 8 total, 7 up, 8 in 2024-08-30T08:55:32.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:55:32 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-08-30T08:55:32.384 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:55:32 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-08-30T08:55:32.384 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:55:32 smithi042 ceph-mon[18652]: from='client.24421 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi190:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-08-30T08:55:32.384 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:55:32 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-30T08:55:32.384 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:55:32 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-30T08:55:32.384 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:55:32 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:55:33.144 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:55:33 smithi190 ceph-mon[22001]: purged_snaps scrub starts 2024-08-30T08:55:33.144 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:55:33 smithi190 ceph-mon[22001]: purged_snaps scrub ok 2024-08-30T08:55:33.144 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:55:33 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-08-30T08:55:33.144 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:55:33 smithi190 ceph-mon[22001]: osd.7 [v2:172.21.15.167:6816/2962799814,v1:172.21.15.167:6817/2962799814] boot 2024-08-30T08:55:33.144 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:55:33 smithi190 ceph-mon[22001]: osdmap e46: 8 total, 8 up, 8 in 2024-08-30T08:55:33.145 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:55:33 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-08-30T08:55:33.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:55:33 smithi167 ceph-mon[22063]: purged_snaps scrub starts 2024-08-30T08:55:33.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:55:33 smithi167 ceph-mon[22063]: purged_snaps scrub ok 2024-08-30T08:55:33.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:55:33 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-08-30T08:55:33.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:55:33 smithi167 ceph-mon[22063]: osd.7 [v2:172.21.15.167:6816/2962799814,v1:172.21.15.167:6817/2962799814] boot 2024-08-30T08:55:33.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:55:33 smithi167 ceph-mon[22063]: osdmap e46: 8 total, 8 up, 8 in 2024-08-30T08:55:33.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:55:33 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-08-30T08:55:33.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:55:33 smithi042 ceph-mon[18652]: purged_snaps scrub starts 2024-08-30T08:55:33.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:55:33 smithi042 ceph-mon[18652]: purged_snaps scrub ok 2024-08-30T08:55:33.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:55:33 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-08-30T08:55:33.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:55:33 smithi042 ceph-mon[18652]: osd.7 [v2:172.21.15.167:6816/2962799814,v1:172.21.15.167:6817/2962799814] boot 2024-08-30T08:55:33.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:55:33 smithi042 ceph-mon[18652]: osdmap e46: 8 total, 8 up, 8 in 2024-08-30T08:55:33.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:55:33 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-08-30T08:55:34.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:55:34 smithi167 ceph-mon[22063]: pgmap v180: 1 pgs: 1 remapped; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-08-30T08:55:34.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:55:34 smithi167 ceph-mon[22063]: osdmap e47: 8 total, 8 up, 8 in 2024-08-30T08:55:34.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:55:34 smithi167 ceph-mon[22063]: from='client.? 172.21.15.190:0/1951478216' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "219a7224-4331-464b-ac86-c51054d51d91"} : dispatch 2024-08-30T08:55:34.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:55:34 smithi167 ceph-mon[22063]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "219a7224-4331-464b-ac86-c51054d51d91"} : dispatch 2024-08-30T08:55:34.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:55:34 smithi167 ceph-mon[22063]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "219a7224-4331-464b-ac86-c51054d51d91"}]': finished 2024-08-30T08:55:34.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:55:34 smithi167 ceph-mon[22063]: osdmap e48: 9 total, 8 up, 9 in 2024-08-30T08:55:34.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:55:34 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-08-30T08:55:34.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:55:34 smithi042 ceph-mon[18652]: pgmap v180: 1 pgs: 1 remapped; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-08-30T08:55:34.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:55:34 smithi042 ceph-mon[18652]: osdmap e47: 8 total, 8 up, 8 in 2024-08-30T08:55:34.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:55:34 smithi042 ceph-mon[18652]: from='client.? 172.21.15.190:0/1951478216' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "219a7224-4331-464b-ac86-c51054d51d91"} : dispatch 2024-08-30T08:55:34.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:55:34 smithi042 ceph-mon[18652]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "219a7224-4331-464b-ac86-c51054d51d91"} : dispatch 2024-08-30T08:55:34.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:55:34 smithi042 ceph-mon[18652]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "219a7224-4331-464b-ac86-c51054d51d91"}]': finished 2024-08-30T08:55:34.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:55:34 smithi042 ceph-mon[18652]: osdmap e48: 9 total, 8 up, 9 in 2024-08-30T08:55:34.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:55:34 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-08-30T08:55:34.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:55:34 smithi190 ceph-mon[22001]: pgmap v180: 1 pgs: 1 remapped; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-08-30T08:55:34.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:55:34 smithi190 ceph-mon[22001]: osdmap e47: 8 total, 8 up, 8 in 2024-08-30T08:55:34.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:55:34 smithi190 ceph-mon[22001]: from='client.? 172.21.15.190:0/1951478216' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "219a7224-4331-464b-ac86-c51054d51d91"} : dispatch 2024-08-30T08:55:34.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:55:34 smithi190 ceph-mon[22001]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "219a7224-4331-464b-ac86-c51054d51d91"} : dispatch 2024-08-30T08:55:34.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:55:34 smithi190 ceph-mon[22001]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "219a7224-4331-464b-ac86-c51054d51d91"}]': finished 2024-08-30T08:55:34.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:55:34 smithi190 ceph-mon[22001]: osdmap e48: 9 total, 8 up, 9 in 2024-08-30T08:55:34.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:55:34 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-08-30T08:55:35.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:55:35 smithi167 ceph-mon[22063]: from='client.? 172.21.15.190:0/2926896310' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-30T08:55:35.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:55:35 smithi042 ceph-mon[18652]: from='client.? 172.21.15.190:0/2926896310' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-30T08:55:35.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:55:35 smithi190 ceph-mon[22001]: from='client.? 172.21.15.190:0/2926896310' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-30T08:55:36.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:55:36 smithi167 ceph-mon[22063]: pgmap v183: 1 pgs: 1 remapped; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-08-30T08:55:36.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:55:36 smithi042 ceph-mon[18652]: pgmap v183: 1 pgs: 1 remapped; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-08-30T08:55:36.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:55:36 smithi190 ceph-mon[22001]: pgmap v183: 1 pgs: 1 remapped; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-08-30T08:55:38.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:55:38 smithi167 ceph-mon[22063]: pgmap v184: 1 pgs: 1 remapped; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-08-30T08:55:38.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:55:38 smithi042 ceph-mon[18652]: pgmap v184: 1 pgs: 1 remapped; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-08-30T08:55:38.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:55:38 smithi190 ceph-mon[22001]: pgmap v184: 1 pgs: 1 remapped; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-08-30T08:55:40.135 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:55:40 smithi190 ceph-mon[22001]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail; 84 KiB/s, 0 objects/s recovering 2024-08-30T08:55:40.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:55:40 smithi167 ceph-mon[22063]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail; 84 KiB/s, 0 objects/s recovering 2024-08-30T08:55:40.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:55:40 smithi042 ceph-mon[18652]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail; 84 KiB/s, 0 objects/s recovering 2024-08-30T08:55:42.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:55:42 smithi167 ceph-mon[22063]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-08-30T08:55:42.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:55:42 smithi042 ceph-mon[18652]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-08-30T08:55:42.386 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:55:42 smithi190 ceph-mon[22001]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-08-30T08:55:43.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:55:43 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.8"} : dispatch 2024-08-30T08:55:43.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:55:43 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:55:43.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:55:43 smithi167 ceph-mon[22063]: Deploying daemon osd.8 on smithi190 2024-08-30T08:55:43.367 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:55:43 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.8"} : dispatch 2024-08-30T08:55:43.367 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:55:43 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:55:43.367 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:55:43 smithi190 ceph-mon[22001]: Deploying daemon osd.8 on smithi190 2024-08-30T08:55:43.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:55:43 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.8"} : dispatch 2024-08-30T08:55:43.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:55:43 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:55:43.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:55:43 smithi042 ceph-mon[18652]: Deploying daemon osd.8 on smithi190 2024-08-30T08:55:44.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:55:44 smithi167 ceph-mon[22063]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail; 117 KiB/s, 0 objects/s recovering 2024-08-30T08:55:44.360 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:55:44 smithi190 ceph-mon[22001]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail; 117 KiB/s, 0 objects/s recovering 2024-08-30T08:55:44.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:55:44 smithi042 ceph-mon[18652]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail; 117 KiB/s, 0 objects/s recovering 2024-08-30T08:55:46.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:55:46 smithi167 ceph-mon[22063]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail; 105 KiB/s, 0 objects/s recovering 2024-08-30T08:55:46.382 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:55:46 smithi042 ceph-mon[18652]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail; 105 KiB/s, 0 objects/s recovering 2024-08-30T08:55:46.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:55:46 smithi190 ceph-mon[22001]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail; 105 KiB/s, 0 objects/s recovering 2024-08-30T08:55:48.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:55:48 smithi190 ceph-mon[22001]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-08-30T08:55:48.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:55:48 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-30T08:55:48.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:55:48 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:55:48.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:55:48 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:55:48.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:55:48 smithi042 ceph-mon[18652]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-08-30T08:55:48.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:55:48 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-30T08:55:48.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:55:48 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:55:48.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:55:48 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:55:48.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:55:48 smithi167 ceph-mon[22063]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-08-30T08:55:48.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:55:48 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-30T08:55:48.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:55:48 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:55:48.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:55:48 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:55:51.024 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:55:50 smithi190 ceph-mon[22001]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-08-30T08:55:51.024 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:55:50 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:55:51.024 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:55:50 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:55:51.024 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:55:50 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:55:51.024 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:55:50 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-30T08:55:51.025 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:55:50 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:55:51.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:55:50 smithi167 ceph-mon[22063]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-08-30T08:55:51.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:55:50 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:55:51.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:55:50 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:55:51.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:55:50 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:55:51.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:55:50 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-30T08:55:51.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:55:50 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:55:51.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:55:50 smithi042 ceph-mon[18652]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-08-30T08:55:51.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:55:50 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:55:51.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:55:50 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:55:51.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:55:50 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:55:51.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:55:50 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-30T08:55:51.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:55:50 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:55:51.351 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:55:51.349+0000 7ff663fff640 1 -- 172.21.15.190:0/3365461700 <== mgr.14150 v2:172.21.15.42:6800/3005140902 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7ff67c1a44a0 con 0x7ff6640b4b30 2024-08-30T08:55:51.351 INFO:teuthology.orchestra.run.smithi190.stdout:Created osd(s) 8 on host 'smithi190' 2024-08-30T08:55:51.354 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:55:51.351+0000 7ff681269640 1 -- 172.21.15.190:0/3365461700 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7ff6640b4b30 msgr2=0x7ff6640b7040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:55:51.354 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:55:51.351+0000 7ff681269640 1 --2- 172.21.15.190:0/3365461700 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7ff6640b4b30 0x7ff6640b7040 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7ff670031440 tx=0x7ff670005d50 comp rx=0 tx=0).stop 2024-08-30T08:55:51.354 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:55:51.352+0000 7ff681269640 1 -- 172.21.15.190:0/3365461700 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff67c1023f0 msgr2=0x7ff67c105fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:55:51.354 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:55:51.352+0000 7ff681269640 1 --2- 172.21.15.190:0/3365461700 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff67c1023f0 0x7ff67c105fe0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7ff66c00d950 tx=0x7ff66c00de10 comp rx=0 tx=0).stop 2024-08-30T08:55:51.354 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:55:51.352+0000 7ff681269640 1 -- 172.21.15.190:0/3365461700 shutdown_connections 2024-08-30T08:55:51.354 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:55:51.352+0000 7ff681269640 1 --2- 172.21.15.190:0/3365461700 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7ff6640b4b30 0x7ff6640b7040 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:55:51.354 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:55:51.352+0000 7ff681269640 1 --2- 172.21.15.190:0/3365461700 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff67c103e20 0x7ff67c104df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:55:51.354 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:55:51.352+0000 7ff681269640 1 --2- 172.21.15.190:0/3365461700 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff67c103480 0x7ff67c1048b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:55:51.354 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:55:51.352+0000 7ff681269640 1 --2- 172.21.15.190:0/3365461700 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff67c1023f0 0x7ff67c105fe0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:55:51.354 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:55:51.352+0000 7ff681269640 1 -- 172.21.15.190:0/3365461700 >> 172.21.15.190:0/3365461700 conn(0x7ff67c0fdd80 msgr2=0x7ff67c0ff650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:55:51.354 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:55:51.352+0000 7ff681269640 1 -- 172.21.15.190:0/3365461700 shutdown_connections 2024-08-30T08:55:51.354 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:55:51.352+0000 7ff681269640 1 -- 172.21.15.190:0/3365461700 wait complete. 2024-08-30T08:55:52.028 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:55:51 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-30T08:55:52.028 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:55:51 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:55:52.029 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:55:51 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:55:52.054 DEBUG:teuthology.orchestra.run.smithi190:osd.8> sudo journalctl -f -n 0 -u ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968@osd.8.service 2024-08-30T08:55:52.057 INFO:tasks.cephadm:Deploying osd.9 on smithi042 with /dev/vg_nvme/lv_1... 2024-08-30T08:55:52.057 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-08-30T08:55:52.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:55:51 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-30T08:55:52.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:55:51 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:55:52.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:55:51 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:55:52.084 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:55:51 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-30T08:55:52.084 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:55:51 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:55:52.084 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:55:51 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:55:52.322 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T08:55:53.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:55:52 smithi167 ceph-mon[22063]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-08-30T08:55:53.132 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:55:52 smithi042 ceph-mon[18652]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-08-30T08:55:53.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:55:52 smithi190 ceph-mon[22001]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-08-30T08:55:53.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:55:53 smithi042 ceph-mon[18652]: from='osd.8 [v2:172.21.15.190:6816/3242198843,v1:172.21.15.190:6817/3242198843]' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-08-30T08:55:53.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:55:53 smithi042 ceph-mon[18652]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-08-30T08:55:53.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:55:53 smithi190 ceph-mon[22001]: from='osd.8 [v2:172.21.15.190:6816/3242198843,v1:172.21.15.190:6817/3242198843]' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-08-30T08:55:53.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:55:53 smithi190 ceph-mon[22001]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-08-30T08:55:53.936 INFO:journalctl@ceph.osd.8.smithi190.stdout:Aug 30 08:55:53 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-8[40364]: 2024-08-30T08:55:53.511+0000 7fa49959d640 -1 osd.8 0 log_to_monitors true 2024-08-30T08:55:54.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:55:53 smithi167 ceph-mon[22063]: from='osd.8 [v2:172.21.15.190:6816/3242198843,v1:172.21.15.190:6817/3242198843]' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-08-30T08:55:54.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:55:53 smithi167 ceph-mon[22063]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-08-30T08:55:54.869 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:55:54 smithi190 ceph-mon[22001]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-08-30T08:55:54.869 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:55:54 smithi190 ceph-mon[22001]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-08-30T08:55:54.869 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:55:54 smithi190 ceph-mon[22001]: from='osd.8 [v2:172.21.15.190:6816/3242198843,v1:172.21.15.190:6817/3242198843]' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi190", "root=default"]} : dispatch 2024-08-30T08:55:54.869 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:55:54 smithi190 ceph-mon[22001]: osdmap e49: 9 total, 8 up, 9 in 2024-08-30T08:55:54.869 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:55:54 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-08-30T08:55:54.869 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:55:54 smithi190 ceph-mon[22001]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi190", "root=default"]} : dispatch 2024-08-30T08:55:54.870 INFO:journalctl@ceph.osd.8.smithi190.stdout:Aug 30 08:55:54 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-8[40364]: 2024-08-30T08:55:54.719+0000 7fa49551f640 -1 osd.8 0 waiting for initial osdmap 2024-08-30T08:55:54.870 INFO:journalctl@ceph.osd.8.smithi190.stdout:Aug 30 08:55:54 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-8[40364]: 2024-08-30T08:55:54.728+0000 7fa49131d640 -1 osd.8 50 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-30T08:55:55.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:55:54 smithi167 ceph-mon[22063]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-08-30T08:55:55.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:55:54 smithi167 ceph-mon[22063]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-08-30T08:55:55.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:55:54 smithi167 ceph-mon[22063]: from='osd.8 [v2:172.21.15.190:6816/3242198843,v1:172.21.15.190:6817/3242198843]' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi190", "root=default"]} : dispatch 2024-08-30T08:55:55.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:55:54 smithi167 ceph-mon[22063]: osdmap e49: 9 total, 8 up, 9 in 2024-08-30T08:55:55.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:55:54 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-08-30T08:55:55.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:55:54 smithi167 ceph-mon[22063]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi190", "root=default"]} : dispatch 2024-08-30T08:55:55.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:55:54 smithi042 ceph-mon[18652]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-08-30T08:55:55.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:55:54 smithi042 ceph-mon[18652]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-08-30T08:55:55.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:55:54 smithi042 ceph-mon[18652]: from='osd.8 [v2:172.21.15.190:6816/3242198843,v1:172.21.15.190:6817/3242198843]' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi190", "root=default"]} : dispatch 2024-08-30T08:55:55.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:55:54 smithi042 ceph-mon[18652]: osdmap e49: 9 total, 8 up, 9 in 2024-08-30T08:55:55.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:55:54 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-08-30T08:55:55.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:55:54 smithi042 ceph-mon[18652]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi190", "root=default"]} : dispatch 2024-08-30T08:55:55.331 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-08-30T08:55:55.356 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph orch daemon add osd smithi042:vg_nvme/lv_1 2024-08-30T08:55:55.649 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T08:55:56.040 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:55:55 smithi042 ceph-mon[18652]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]': finished 2024-08-30T08:55:56.041 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:55:55 smithi042 ceph-mon[18652]: osdmap e50: 9 total, 8 up, 9 in 2024-08-30T08:55:56.041 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:55:55 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-08-30T08:55:56.041 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:55:55 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-08-30T08:55:56.041 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:55:55 smithi042 ceph-mon[18652]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-08-30T08:55:56.041 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:55:55 smithi042 ceph-mon[18652]: Detected new or changed devices on smithi190 2024-08-30T08:55:56.041 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:55:55 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:55:56.041 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:55:55 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:55:56.041 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:55:55 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-08-30T08:55:56.041 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:55:55 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-08-30T08:55:56.041 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:55:55 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-08-30T08:55:56.041 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:55:55 smithi042 ceph-mon[18652]: Adjusting osd_memory_target on smithi190 to 5696M 2024-08-30T08:55:56.041 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:55:55 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:55:56.041 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:55:55 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:55:56.042 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:55:55 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-30T08:55:56.042 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:55:55 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:55:56.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:55:55 smithi167 ceph-mon[22063]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]': finished 2024-08-30T08:55:56.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:55:55 smithi167 ceph-mon[22063]: osdmap e50: 9 total, 8 up, 9 in 2024-08-30T08:55:56.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:55:55 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-08-30T08:55:56.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:55:55 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-08-30T08:55:56.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:55:55 smithi167 ceph-mon[22063]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-08-30T08:55:56.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:55:55 smithi167 ceph-mon[22063]: Detected new or changed devices on smithi190 2024-08-30T08:55:56.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:55:55 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:55:56.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:55:55 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:55:56.078 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:55:55 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-08-30T08:55:56.078 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:55:55 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-08-30T08:55:56.078 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:55:55 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-08-30T08:55:56.078 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:55:55 smithi167 ceph-mon[22063]: Adjusting osd_memory_target on smithi190 to 5696M 2024-08-30T08:55:56.078 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:55:55 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:55:56.078 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:55:55 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:55:56.078 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:55:55 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-30T08:55:56.078 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:55:55 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:55:56.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:55:56.132+0000 7f9ab467d640 1 -- 172.21.15.42:0/3542750824 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9aac104580 msgr2=0x7f9aac104990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:55:56.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:55:56.132+0000 7f9ab467d640 1 --2- 172.21.15.42:0/3542750824 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9aac104580 0x7f9aac104990 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f9a9c009900 tx=0x7f9a9c02a480 comp rx=0 tx=0).stop 2024-08-30T08:55:56.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:55:56.133+0000 7f9ab467d640 1 -- 172.21.15.42:0/3542750824 shutdown_connections 2024-08-30T08:55:56.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:55:56.133+0000 7f9ab467d640 1 --2- 172.21.15.42:0/3542750824 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9aac104f20 0x7f9aac10b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:55:56.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:55:56.133+0000 7f9ab467d640 1 --2- 172.21.15.42:0/3542750824 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9aac104580 0x7f9aac104990 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:55:56.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:55:56.133+0000 7f9ab467d640 1 --2- 172.21.15.42:0/3542750824 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9aac1034f0 0x7f9aac103960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:55:56.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:55:56.133+0000 7f9ab467d640 1 -- 172.21.15.42:0/3542750824 >> 172.21.15.42:0/3542750824 conn(0x7f9aac0fee60 msgr2=0x7f9aac101310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:55:56.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:55:56.133+0000 7f9ab467d640 1 -- 172.21.15.42:0/3542750824 shutdown_connections 2024-08-30T08:55:56.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:55:56.133+0000 7f9ab467d640 1 -- 172.21.15.42:0/3542750824 wait complete. 2024-08-30T08:55:56.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:55:56.133+0000 7f9ab467d640 1 Processor -- start 2024-08-30T08:55:56.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:55:56.133+0000 7f9ab467d640 1 -- start start 2024-08-30T08:55:56.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:55:56.134+0000 7f9ab467d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9aac1034f0 0x7f9aac199d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:55:56.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:55:56.134+0000 7f9ab467d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9aac104580 0x7f9aac19a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:55:56.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:55:56.134+0000 7f9ab467d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9aac104f20 0x7f9aac19af60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:55:56.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:55:56.134+0000 7f9ab467d640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f9aac10f110 con 0x7f9aac1034f0 2024-08-30T08:55:56.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:55:56.134+0000 7f9ab467d640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f9aac10ef90 con 0x7f9aac104f20 2024-08-30T08:55:56.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:55:56.134+0000 7f9ab467d640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f9aac10f290 con 0x7f9aac104580 2024-08-30T08:55:56.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:55:56.134+0000 7f9ab2419640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9aac1034f0 0x7f9aac199d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:55:56.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:55:56.134+0000 7f9ab2419640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9aac1034f0 0x7f9aac199d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:44470/0 (socket says 172.21.15.42:44470) 2024-08-30T08:55:56.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:55:56.134+0000 7f9ab2419640 1 -- 172.21.15.42:0/3871522536 learned_addr learned my addr 172.21.15.42:0/3871522536 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T08:55:56.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:55:56.134+0000 7f9ab1c18640 1 --2- 172.21.15.42:0/3871522536 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9aac104580 0x7f9aac19a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:55:56.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:55:56.134+0000 7f9ab2c1a640 1 --2- 172.21.15.42:0/3871522536 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9aac104f20 0x7f9aac19af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:55:56.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:55:56.134+0000 7f9ab2419640 1 -- 172.21.15.42:0/3871522536 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9aac104580 msgr2=0x7f9aac19a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:55:56.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:55:56.134+0000 7f9ab2419640 1 --2- 172.21.15.42:0/3871522536 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9aac104580 0x7f9aac19a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:55:56.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:55:56.134+0000 7f9ab2419640 1 -- 172.21.15.42:0/3871522536 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9aac104f20 msgr2=0x7f9aac19af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:55:56.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:55:56.134+0000 7f9ab2419640 1 --2- 172.21.15.42:0/3871522536 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9aac104f20 0x7f9aac19af60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:55:56.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:55:56.134+0000 7f9ab2419640 1 -- 172.21.15.42:0/3871522536 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9aac19b6f0 con 0x7f9aac1034f0 2024-08-30T08:55:56.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:55:56.134+0000 7f9ab2419640 1 --2- 172.21.15.42:0/3871522536 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9aac1034f0 0x7f9aac199d50 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f9aa800c920 tx=0x7f9aa800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:55:56.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:55:56.134+0000 7f9aa37fe640 1 -- 172.21.15.42:0/3871522536 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aa8007c70 con 0x7f9aac1034f0 2024-08-30T08:55:56.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:55:56.135+0000 7f9ab467d640 1 -- 172.21.15.42:0/3871522536 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9aac19b9e0 con 0x7f9aac1034f0 2024-08-30T08:55:56.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:55:56.135+0000 7f9aa37fe640 1 -- 172.21.15.42:0/3871522536 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9aa8007e10 con 0x7f9aac1034f0 2024-08-30T08:55:56.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:55:56.135+0000 7f9aa37fe640 1 -- 172.21.15.42:0/3871522536 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aa8018820 con 0x7f9aac1034f0 2024-08-30T08:55:56.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:55:56.135+0000 7f9ab467d640 1 -- 172.21.15.42:0/3871522536 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9aac1a2170 con 0x7f9aac1034f0 2024-08-30T08:55:56.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:55:56.136+0000 7f9ab467d640 1 -- 172.21.15.42:0/3871522536 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9aac103960 con 0x7f9aac1034f0 2024-08-30T08:55:56.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:55:56.137+0000 7f9aa37fe640 1 -- 172.21.15.42:0/3871522536 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) ==== 152577+0+0 (secure 0 0 0) 0x7f9aa800f450 con 0x7f9aac1034f0 2024-08-30T08:55:56.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:55:56.138+0000 7f9aa37fe640 1 --2- 172.21.15.42:0/3871522536 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f9a980b4b30 0x7f9a980b7040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:55:56.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:55:56.138+0000 7f9ab1c18640 1 --2- 172.21.15.42:0/3871522536 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f9a980b4b30 0x7f9a980b7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:55:56.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:55:56.138+0000 7f9aa37fe640 1 -- 172.21.15.42:0/3871522536 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 5862+0+0 (secure 0 0 0) 0x7f9aa80e4b80 con 0x7f9aac1034f0 2024-08-30T08:55:56.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:55:56.138+0000 7f9ab1c18640 1 --2- 172.21.15.42:0/3871522536 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f9a980b4b30 0x7f9a980b7040 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f9a9c000c00 tx=0x7f9a9c031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:55:56.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:55:56.141+0000 7f9aa37fe640 1 -- 172.21.15.42:0/3871522536 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f9aa8014030 con 0x7f9aac1034f0 2024-08-30T08:55:56.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:55:55 smithi190 ceph-mon[22001]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]': finished 2024-08-30T08:55:56.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:55:55 smithi190 ceph-mon[22001]: osdmap e50: 9 total, 8 up, 9 in 2024-08-30T08:55:56.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:55:55 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-08-30T08:55:56.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:55:55 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-08-30T08:55:56.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:55:55 smithi190 ceph-mon[22001]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-08-30T08:55:56.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:55:55 smithi190 ceph-mon[22001]: Detected new or changed devices on smithi190 2024-08-30T08:55:56.187 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:55:55 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:55:56.187 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:55:55 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:55:56.187 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:55:55 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-08-30T08:55:56.187 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:55:55 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-08-30T08:55:56.187 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:55:55 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-08-30T08:55:56.187 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:55:55 smithi190 ceph-mon[22001]: Adjusting osd_memory_target on smithi190 to 5696M 2024-08-30T08:55:56.187 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:55:55 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:55:56.187 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:55:55 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:55:56.187 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:55:55 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-30T08:55:56.187 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:55:55 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:55:56.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:55:56.336+0000 7f9ab467d640 1 -- 172.21.15.42:0/3871522536 --> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi042:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f9aac02e810 con 0x7f9a980b4b30 2024-08-30T08:55:57.002 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:55:56 smithi042 ceph-mon[18652]: purged_snaps scrub starts 2024-08-30T08:55:57.002 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:55:56 smithi042 ceph-mon[18652]: purged_snaps scrub ok 2024-08-30T08:55:57.002 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:55:56 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-08-30T08:55:57.003 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:55:56 smithi042 ceph-mon[18652]: osd.8 [v2:172.21.15.190:6816/3242198843,v1:172.21.15.190:6817/3242198843] boot 2024-08-30T08:55:57.003 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:55:56 smithi042 ceph-mon[18652]: osdmap e51: 9 total, 9 up, 9 in 2024-08-30T08:55:57.003 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:55:56 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-08-30T08:55:57.003 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:55:56 smithi042 ceph-mon[18652]: from='client.14580 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi042:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-08-30T08:55:57.003 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:55:56 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-30T08:55:57.003 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:55:56 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-30T08:55:57.003 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:55:56 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:55:57.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:55:56 smithi167 ceph-mon[22063]: purged_snaps scrub starts 2024-08-30T08:55:57.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:55:56 smithi167 ceph-mon[22063]: purged_snaps scrub ok 2024-08-30T08:55:57.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:55:56 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-08-30T08:55:57.078 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:55:56 smithi167 ceph-mon[22063]: osd.8 [v2:172.21.15.190:6816/3242198843,v1:172.21.15.190:6817/3242198843] boot 2024-08-30T08:55:57.078 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:55:56 smithi167 ceph-mon[22063]: osdmap e51: 9 total, 9 up, 9 in 2024-08-30T08:55:57.078 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:55:56 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-08-30T08:55:57.078 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:55:56 smithi167 ceph-mon[22063]: from='client.14580 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi042:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-08-30T08:55:57.078 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:55:56 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-30T08:55:57.078 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:55:56 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-30T08:55:57.078 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:55:56 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:55:57.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:55:56 smithi190 ceph-mon[22001]: purged_snaps scrub starts 2024-08-30T08:55:57.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:55:56 smithi190 ceph-mon[22001]: purged_snaps scrub ok 2024-08-30T08:55:57.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:55:56 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-08-30T08:55:57.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:55:56 smithi190 ceph-mon[22001]: osd.8 [v2:172.21.15.190:6816/3242198843,v1:172.21.15.190:6817/3242198843] boot 2024-08-30T08:55:57.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:55:56 smithi190 ceph-mon[22001]: osdmap e51: 9 total, 9 up, 9 in 2024-08-30T08:55:57.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:55:56 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-08-30T08:55:57.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:55:56 smithi190 ceph-mon[22001]: from='client.14580 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi042:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-08-30T08:55:57.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:55:56 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-30T08:55:57.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:55:56 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-30T08:55:57.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:55:56 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:55:58.060 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:55:57 smithi042 ceph-mon[18652]: osdmap e52: 9 total, 9 up, 9 in 2024-08-30T08:55:58.060 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:55:57 smithi042 ceph-mon[18652]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-08-30T08:55:58.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:55:57 smithi167 ceph-mon[22063]: osdmap e52: 9 total, 9 up, 9 in 2024-08-30T08:55:58.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:55:57 smithi167 ceph-mon[22063]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-08-30T08:55:58.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:55:57 smithi190 ceph-mon[22001]: osdmap e52: 9 total, 9 up, 9 in 2024-08-30T08:55:58.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:55:57 smithi190 ceph-mon[22001]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-08-30T08:55:58.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:55:58 smithi042 ceph-mon[18652]: osdmap e53: 9 total, 9 up, 9 in 2024-08-30T08:55:59.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:55:58 smithi167 ceph-mon[22063]: osdmap e53: 9 total, 9 up, 9 in 2024-08-30T08:55:59.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:55:58 smithi190 ceph-mon[22001]: osdmap e53: 9 total, 9 up, 9 in 2024-08-30T08:55:59.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:55:59 smithi042 ceph-mon[18652]: pgmap v200: 1 pgs: 1 activating+degraded; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 2/6 objects degraded (33.333%) 2024-08-30T08:55:59.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:55:59 smithi042 ceph-mon[18652]: from='client.? 172.21.15.42:0/2101040027' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "2f2f5177-a009-40d9-991c-76aec8a5cba6"} : dispatch 2024-08-30T08:55:59.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:55:59 smithi042 ceph-mon[18652]: from='client.? 172.21.15.42:0/2101040027' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2f2f5177-a009-40d9-991c-76aec8a5cba6"}]': finished 2024-08-30T08:55:59.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:55:59 smithi042 ceph-mon[18652]: osdmap e54: 10 total, 9 up, 10 in 2024-08-30T08:55:59.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:55:59 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-08-30T08:55:59.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:55:59 smithi042 ceph-mon[18652]: from='client.? 172.21.15.42:0/2254160395' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-30T08:56:00.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:55:59 smithi167 ceph-mon[22063]: pgmap v200: 1 pgs: 1 activating+degraded; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 2/6 objects degraded (33.333%) 2024-08-30T08:56:00.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:55:59 smithi167 ceph-mon[22063]: from='client.? 172.21.15.42:0/2101040027' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "2f2f5177-a009-40d9-991c-76aec8a5cba6"} : dispatch 2024-08-30T08:56:00.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:55:59 smithi167 ceph-mon[22063]: from='client.? 172.21.15.42:0/2101040027' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2f2f5177-a009-40d9-991c-76aec8a5cba6"}]': finished 2024-08-30T08:56:00.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:55:59 smithi167 ceph-mon[22063]: osdmap e54: 10 total, 9 up, 10 in 2024-08-30T08:56:00.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:55:59 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-08-30T08:56:00.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:55:59 smithi167 ceph-mon[22063]: from='client.? 172.21.15.42:0/2254160395' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-30T08:56:00.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:55:59 smithi190 ceph-mon[22001]: pgmap v200: 1 pgs: 1 activating+degraded; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 2/6 objects degraded (33.333%) 2024-08-30T08:56:00.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:55:59 smithi190 ceph-mon[22001]: from='client.? 172.21.15.42:0/2101040027' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "2f2f5177-a009-40d9-991c-76aec8a5cba6"} : dispatch 2024-08-30T08:56:00.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:55:59 smithi190 ceph-mon[22001]: from='client.? 172.21.15.42:0/2101040027' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2f2f5177-a009-40d9-991c-76aec8a5cba6"}]': finished 2024-08-30T08:56:00.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:55:59 smithi190 ceph-mon[22001]: osdmap e54: 10 total, 9 up, 10 in 2024-08-30T08:56:00.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:55:59 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-08-30T08:56:00.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:55:59 smithi190 ceph-mon[22001]: from='client.? 172.21.15.42:0/2254160395' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-30T08:56:01.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:56:00 smithi167 ceph-mon[22063]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-08-30T08:56:01.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:56:00 smithi042 ceph-mon[18652]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-08-30T08:56:01.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:56:00 smithi190 ceph-mon[22001]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-08-30T08:56:02.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:56:01 smithi167 ceph-mon[22063]: pgmap v202: 1 pgs: 1 activating+degraded; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 2/6 objects degraded (33.333%) 2024-08-30T08:56:02.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:56:01 smithi042 ceph-mon[18652]: pgmap v202: 1 pgs: 1 activating+degraded; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 2/6 objects degraded (33.333%) 2024-08-30T08:56:02.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:56:01 smithi190 ceph-mon[22001]: pgmap v202: 1 pgs: 1 activating+degraded; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 2/6 objects degraded (33.333%) 2024-08-30T08:56:04.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:56:03 smithi042 ceph-mon[18652]: pgmap v203: 1 pgs: 1 activating+degraded; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 2/6 objects degraded (33.333%) 2024-08-30T08:56:04.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:56:03 smithi167 ceph-mon[22063]: pgmap v203: 1 pgs: 1 activating+degraded; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 2/6 objects degraded (33.333%) 2024-08-30T08:56:04.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:56:03 smithi190 ceph-mon[22001]: pgmap v203: 1 pgs: 1 activating+degraded; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 2/6 objects degraded (33.333%) 2024-08-30T08:56:05.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:56:04 smithi167 ceph-mon[22063]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-08-30T08:56:05.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:56:04 smithi167 ceph-mon[22063]: Cluster is now healthy 2024-08-30T08:56:05.375 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:56:04 smithi042 ceph-mon[18652]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-08-30T08:56:05.375 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:56:04 smithi042 ceph-mon[18652]: Cluster is now healthy 2024-08-30T08:56:05.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:56:04 smithi190 ceph-mon[22001]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-08-30T08:56:05.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:56:04 smithi190 ceph-mon[22001]: Cluster is now healthy 2024-08-30T08:56:06.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:56:05 smithi167 ceph-mon[22063]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-08-30T08:56:06.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:56:05 smithi042 ceph-mon[18652]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-08-30T08:56:06.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:56:05 smithi190 ceph-mon[22001]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-08-30T08:56:08.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:56:07 smithi167 ceph-mon[22063]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-08-30T08:56:08.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:56:07 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.9"} : dispatch 2024-08-30T08:56:08.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:56:07 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:56:08.349 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:56:07 smithi042 ceph-mon[18652]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-08-30T08:56:08.349 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:56:07 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.9"} : dispatch 2024-08-30T08:56:08.349 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:56:07 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:56:08.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:56:07 smithi190 ceph-mon[22001]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-08-30T08:56:08.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:56:07 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.9"} : dispatch 2024-08-30T08:56:08.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:56:07 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:56:09.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:56:09 smithi167 ceph-mon[22063]: Deploying daemon osd.9 on smithi042 2024-08-30T08:56:09.369 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:56:09 smithi042 ceph-mon[18652]: Deploying daemon osd.9 on smithi042 2024-08-30T08:56:09.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:56:09 smithi190 ceph-mon[22001]: Deploying daemon osd.9 on smithi042 2024-08-30T08:56:10.301 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:56:10 smithi042 ceph-mon[18652]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-08-30T08:56:10.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:56:10 smithi167 ceph-mon[22063]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-08-30T08:56:10.337 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:56:10 smithi190 ceph-mon[22001]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-08-30T08:56:12.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:56:12 smithi167 ceph-mon[22063]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-08-30T08:56:12.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:56:12 smithi042 ceph-mon[18652]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-08-30T08:56:12.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:56:12 smithi190 ceph-mon[22001]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-08-30T08:56:14.154 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:56:14 smithi042 ceph-mon[18652]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-08-30T08:56:14.154 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:56:14 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-30T08:56:14.154 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:56:14 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:56:14.154 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:56:14 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:56:14.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:56:14 smithi167 ceph-mon[22063]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-08-30T08:56:14.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:56:14 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-30T08:56:14.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:56:14 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:56:14.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:56:14 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:56:14.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:56:14 smithi190 ceph-mon[22001]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-08-30T08:56:14.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:56:14 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-30T08:56:14.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:56:14 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:56:14.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:56:14 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:56:16.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:56:16 smithi042 ceph-mon[18652]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-08-30T08:56:16.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:56:16 smithi167 ceph-mon[22063]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-08-30T08:56:16.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:56:16 smithi190 ceph-mon[22001]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-08-30T08:56:17.308 INFO:teuthology.orchestra.run.smithi042.stdout:Created osd(s) 9 on host 'smithi042' 2024-08-30T08:56:17.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:56:17.304+0000 7f9aa37fe640 1 -- 172.21.15.42:0/3871522536 <== mgr.14150 v2:172.21.15.42:6800/3005140902 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f9aac02e810 con 0x7f9a980b4b30 2024-08-30T08:56:17.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:56:17.306+0000 7f9ab467d640 1 -- 172.21.15.42:0/3871522536 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f9a980b4b30 msgr2=0x7f9a980b7040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:56:17.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:56:17.306+0000 7f9ab467d640 1 --2- 172.21.15.42:0/3871522536 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f9a980b4b30 0x7f9a980b7040 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f9a9c000c00 tx=0x7f9a9c031400 comp rx=0 tx=0).stop 2024-08-30T08:56:17.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:56:17.307+0000 7f9ab467d640 1 -- 172.21.15.42:0/3871522536 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9aac1034f0 msgr2=0x7f9aac199d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:56:17.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:56:17.307+0000 7f9ab467d640 1 --2- 172.21.15.42:0/3871522536 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9aac1034f0 0x7f9aac199d50 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f9aa800c920 tx=0x7f9aa800cde0 comp rx=0 tx=0).stop 2024-08-30T08:56:17.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:56:17.307+0000 7f9ab467d640 1 -- 172.21.15.42:0/3871522536 shutdown_connections 2024-08-30T08:56:17.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:56:17.307+0000 7f9ab467d640 1 --2- 172.21.15.42:0/3871522536 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f9a980b4b30 0x7f9a980b7040 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:56:17.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:56:17.307+0000 7f9ab467d640 1 --2- 172.21.15.42:0/3871522536 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9aac104580 0x7f9aac19a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:56:17.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:56:17.307+0000 7f9ab467d640 1 --2- 172.21.15.42:0/3871522536 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9aac104f20 0x7f9aac19af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:56:17.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:56:17.307+0000 7f9ab467d640 1 --2- 172.21.15.42:0/3871522536 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9aac1034f0 0x7f9aac199d50 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:56:17.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:56:17.307+0000 7f9ab467d640 1 -- 172.21.15.42:0/3871522536 >> 172.21.15.42:0/3871522536 conn(0x7f9aac0fee60 msgr2=0x7f9aac1078a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:56:17.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:56:17.307+0000 7f9ab467d640 1 -- 172.21.15.42:0/3871522536 shutdown_connections 2024-08-30T08:56:17.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:56:17.307+0000 7f9ab467d640 1 -- 172.21.15.42:0/3871522536 wait complete. 2024-08-30T08:56:17.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:56:17 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:56:17.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:56:17 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:56:17.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:56:17 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:56:17.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:56:17 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-30T08:56:17.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:56:17 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:56:17.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:56:17 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-30T08:56:17.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:56:17 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:56:17.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:56:17 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:56:17.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:56:17 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:56:17.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:56:17 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:56:17.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:56:17 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:56:17.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:56:17 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-30T08:56:17.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:56:17 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:56:17.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:56:17 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-30T08:56:17.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:56:17 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:56:17.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:56:17 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:56:17.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:56:17 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:56:17.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:56:17 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:56:17.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:56:17 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:56:17.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:56:17 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-30T08:56:17.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:56:17 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:56:17.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:56:17 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-30T08:56:17.937 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:56:17 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:56:17.937 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:56:17 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:56:18.006 DEBUG:teuthology.orchestra.run.smithi042:osd.9> sudo journalctl -f -n 0 -u ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968@osd.9.service 2024-08-30T08:56:18.008 INFO:tasks.cephadm:Deploying osd.10 on smithi167 with /dev/vg_nvme/lv_1... 2024-08-30T08:56:18.009 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-08-30T08:56:18.266 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.b/config 2024-08-30T08:56:18.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:56:18 smithi042 ceph-mon[18652]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-08-30T08:56:18.638 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:56:18 smithi167 ceph-mon[22063]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-08-30T08:56:18.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:56:18 smithi190 ceph-mon[22001]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-08-30T08:56:19.883 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 08:56:19 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9[55148]: 2024-08-30T08:56:19.553+0000 7f75929c2640 -1 osd.9 0 log_to_monitors true 2024-08-30T08:56:20.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:56:20 smithi167 ceph-mon[22063]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-08-30T08:56:20.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:56:20 smithi167 ceph-mon[22063]: from='osd.9 [v2:172.21.15.42:6826/1123536325,v1:172.21.15.42:6827/1123536325]' entity='osd.9' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]} : dispatch 2024-08-30T08:56:20.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:56:20 smithi042 ceph-mon[18652]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-08-30T08:56:20.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:56:20 smithi042 ceph-mon[18652]: from='osd.9 [v2:172.21.15.42:6826/1123536325,v1:172.21.15.42:6827/1123536325]' entity='osd.9' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]} : dispatch 2024-08-30T08:56:20.911 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-08-30T08:56:20.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:56:20 smithi190 ceph-mon[22001]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-08-30T08:56:20.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:56:20 smithi190 ceph-mon[22001]: from='osd.9 [v2:172.21.15.42:6826/1123536325,v1:172.21.15.42:6827/1123536325]' entity='osd.9' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]} : dispatch 2024-08-30T08:56:20.942 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph orch daemon add osd smithi167:vg_nvme/lv_1 2024-08-30T08:56:21.199 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.b/config 2024-08-30T08:56:21.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:56:21 smithi167 ceph-mon[22063]: from='osd.9 [v2:172.21.15.42:6826/1123536325,v1:172.21.15.42:6827/1123536325]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-08-30T08:56:21.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:56:21 smithi167 ceph-mon[22063]: osdmap e55: 10 total, 9 up, 10 in 2024-08-30T08:56:21.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:56:21 smithi167 ceph-mon[22063]: from='osd.9 [v2:172.21.15.42:6826/1123536325,v1:172.21.15.42:6827/1123536325]' entity='osd.9' cmd={"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi042", "root=default"]} : dispatch 2024-08-30T08:56:21.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:56:21 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-08-30T08:56:21.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:56:21 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:56:21.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:56:21 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:56:21.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:56:21 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-08-30T08:56:21.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:56:21 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-08-30T08:56:21.578 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:56:21 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-08-30T08:56:21.578 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:56:21 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-08-30T08:56:21.578 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:56:21 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:56:21.578 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:56:21 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:56:21.578 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:56:21 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-30T08:56:21.578 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:56:21 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:56:21.708 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:56:21.704+0000 7fb5e0eae640 1 -- 172.21.15.167:0/1868750477 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb5dc104580 msgr2=0x7fb5dc104990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:56:21.709 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:56:21.704+0000 7fb5e0eae640 1 --2- 172.21.15.167:0/1868750477 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb5dc104580 0x7fb5dc104990 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fb5d0009900 tx=0x7fb5d002a480 comp rx=0 tx=0).stop 2024-08-30T08:56:21.709 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:56:21.704+0000 7fb5e0eae640 1 -- 172.21.15.167:0/1868750477 shutdown_connections 2024-08-30T08:56:21.709 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:56:21.704+0000 7fb5e0eae640 1 --2- 172.21.15.167:0/1868750477 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb5dc104f20 0x7fb5dc10b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:56:21.709 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:56:21.704+0000 7fb5e0eae640 1 --2- 172.21.15.167:0/1868750477 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb5dc104580 0x7fb5dc104990 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:56:21.709 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:56:21.704+0000 7fb5e0eae640 1 --2- 172.21.15.167:0/1868750477 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb5dc1034f0 0x7fb5dc103960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:56:21.709 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:56:21.704+0000 7fb5e0eae640 1 -- 172.21.15.167:0/1868750477 >> 172.21.15.167:0/1868750477 conn(0x7fb5dc0fee60 msgr2=0x7fb5dc101310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:56:21.709 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:56:21.704+0000 7fb5e0eae640 1 -- 172.21.15.167:0/1868750477 shutdown_connections 2024-08-30T08:56:21.709 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:56:21.704+0000 7fb5e0eae640 1 -- 172.21.15.167:0/1868750477 wait complete. 2024-08-30T08:56:21.709 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:56:21.705+0000 7fb5e0eae640 1 Processor -- start 2024-08-30T08:56:21.709 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:56:21.705+0000 7fb5e0eae640 1 -- start start 2024-08-30T08:56:21.709 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:56:21.705+0000 7fb5e0eae640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb5dc1034f0 0x7fb5dc199db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:56:21.710 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:56:21.705+0000 7fb5e0eae640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb5dc104580 0x7fb5dc19a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:56:21.710 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:56:21.705+0000 7fb5e0eae640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb5dc104f20 0x7fb5dc19afc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:56:21.710 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:56:21.705+0000 7fb5e0eae640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fb5dc10f190 con 0x7fb5dc104f20 2024-08-30T08:56:21.710 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:56:21.705+0000 7fb5e0eae640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb5dc10f010 con 0x7fb5dc104580 2024-08-30T08:56:21.710 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:56:21.705+0000 7fb5e0eae640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fb5dc10f310 con 0x7fb5dc1034f0 2024-08-30T08:56:21.710 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:56:21.705+0000 7fb5d9d9b640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb5dc104580 0x7fb5dc19a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:56:21.710 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:56:21.705+0000 7fb5d9d9b640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb5dc104580 0x7fb5dc19a420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:45318/0 (socket says 172.21.15.167:45318) 2024-08-30T08:56:21.710 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:56:21.705+0000 7fb5d9d9b640 1 -- 172.21.15.167:0/2022129010 learned_addr learned my addr 172.21.15.167:0/2022129010 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-08-30T08:56:21.710 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:56:21.705+0000 7fb5dad9d640 1 --2- 172.21.15.167:0/2022129010 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb5dc104f20 0x7fb5dc19afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:56:21.710 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:56:21.705+0000 7fb5da59c640 1 --2- 172.21.15.167:0/2022129010 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb5dc1034f0 0x7fb5dc199db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:56:21.710 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:56:21.705+0000 7fb5d9d9b640 1 -- 172.21.15.167:0/2022129010 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb5dc1034f0 msgr2=0x7fb5dc199db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:56:21.710 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:56:21.705+0000 7fb5d9d9b640 1 --2- 172.21.15.167:0/2022129010 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb5dc1034f0 0x7fb5dc199db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:56:21.710 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:56:21.705+0000 7fb5d9d9b640 1 -- 172.21.15.167:0/2022129010 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb5dc104f20 msgr2=0x7fb5dc19afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:56:21.711 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:56:21.706+0000 7fb5d9d9b640 1 --2- 172.21.15.167:0/2022129010 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb5dc104f20 0x7fb5dc19afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:56:21.711 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:56:21.706+0000 7fb5d9d9b640 1 -- 172.21.15.167:0/2022129010 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5dc19b7e0 con 0x7fb5dc104580 2024-08-30T08:56:21.711 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:56:21.706+0000 7fb5da59c640 1 --2- 172.21.15.167:0/2022129010 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb5dc1034f0 0x7fb5dc199db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-30T08:56:21.711 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:56:21.706+0000 7fb5d9d9b640 1 --2- 172.21.15.167:0/2022129010 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb5dc104580 0x7fb5dc19a420 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fb5d0006010 tx=0x7fb5d0004340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:56:21.711 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:56:21.706+0000 7fb5cb7fe640 1 -- 172.21.15.167:0/2022129010 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5d003f030 con 0x7fb5dc104580 2024-08-30T08:56:21.711 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:56:21.706+0000 7fb5e0eae640 1 -- 172.21.15.167:0/2022129010 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5dc19ba70 con 0x7fb5dc104580 2024-08-30T08:56:21.711 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:56:21.706+0000 7fb5cb7fe640 1 -- 172.21.15.167:0/2022129010 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb5d0003cb0 con 0x7fb5dc104580 2024-08-30T08:56:21.711 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:56:21.706+0000 7fb5cb7fe640 1 -- 172.21.15.167:0/2022129010 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5d0035070 con 0x7fb5dc104580 2024-08-30T08:56:21.711 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:56:21.706+0000 7fb5e0eae640 1 -- 172.21.15.167:0/2022129010 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5dc1a2080 con 0x7fb5dc104580 2024-08-30T08:56:21.712 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:56:21.707+0000 7fb5e0eae640 1 -- 172.21.15.167:0/2022129010 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5dc10a6b0 con 0x7fb5dc104580 2024-08-30T08:56:21.712 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:56:21.710+0000 7fb5cb7fe640 1 -- 172.21.15.167:0/2022129010 <== mon.2 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 16) ==== 152577+0+0 (secure 0 0 0) 0x7fb5d0042030 con 0x7fb5dc104580 2024-08-30T08:56:21.712 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:56:21.710+0000 7fb5cb7fe640 1 --2- 172.21.15.167:0/2022129010 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fb5c40b4b30 0x7fb5c40b7040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:56:21.713 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:56:21.710+0000 7fb5cb7fe640 1 -- 172.21.15.167:0/2022129010 <== mon.2 v2:172.21.15.167:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5981+0+0 (secure 0 0 0) 0x7fb5d0105540 con 0x7fb5dc104580 2024-08-30T08:56:21.713 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:56:21.710+0000 7fb5da59c640 1 --2- 172.21.15.167:0/2022129010 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fb5c40b4b30 0x7fb5c40b7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:56:21.713 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:56:21.711+0000 7fb5da59c640 1 --2- 172.21.15.167:0/2022129010 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fb5c40b4b30 0x7fb5c40b7040 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fb5cc0058c0 tx=0x7fb5cc005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:56:21.716 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:56:21.714+0000 7fb5cb7fe640 1 -- 172.21.15.167:0/2022129010 <== mon.2 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7fb5d00ccf90 con 0x7fb5dc104580 2024-08-30T08:56:21.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:56:21 smithi042 ceph-mon[18652]: from='osd.9 [v2:172.21.15.42:6826/1123536325,v1:172.21.15.42:6827/1123536325]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-08-30T08:56:21.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:56:21 smithi042 ceph-mon[18652]: osdmap e55: 10 total, 9 up, 10 in 2024-08-30T08:56:21.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:56:21 smithi042 ceph-mon[18652]: from='osd.9 [v2:172.21.15.42:6826/1123536325,v1:172.21.15.42:6827/1123536325]' entity='osd.9' cmd={"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi042", "root=default"]} : dispatch 2024-08-30T08:56:21.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:56:21 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-08-30T08:56:21.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:56:21 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:56:21.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:56:21 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:56:21.884 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:56:21 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-08-30T08:56:21.884 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:56:21 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-08-30T08:56:21.884 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:56:21 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-08-30T08:56:21.884 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:56:21 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-08-30T08:56:21.884 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:56:21 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:56:21.884 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:56:21 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:56:21.884 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:56:21 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-30T08:56:21.884 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:56:21 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:56:21.903 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:56:21.901+0000 7fb5e0eae640 1 -- 172.21.15.167:0/2022129010 --> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fb5dc0635a0 con 0x7fb5c40b4b30 2024-08-30T08:56:21.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:56:21 smithi190 ceph-mon[22001]: from='osd.9 [v2:172.21.15.42:6826/1123536325,v1:172.21.15.42:6827/1123536325]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-08-30T08:56:21.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:56:21 smithi190 ceph-mon[22001]: osdmap e55: 10 total, 9 up, 10 in 2024-08-30T08:56:21.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:56:21 smithi190 ceph-mon[22001]: from='osd.9 [v2:172.21.15.42:6826/1123536325,v1:172.21.15.42:6827/1123536325]' entity='osd.9' cmd={"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi042", "root=default"]} : dispatch 2024-08-30T08:56:21.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:56:21 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-08-30T08:56:21.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:56:21 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:56:21.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:56:21 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:56:21.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:56:21 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-08-30T08:56:21.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:56:21 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-08-30T08:56:21.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:56:21 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-08-30T08:56:21.937 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:56:21 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-08-30T08:56:21.937 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:56:21 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:56:21.937 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:56:21 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:56:21.937 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:56:21 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-30T08:56:21.937 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:56:21 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:56:22.382 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 08:56:22 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9[55148]: 2024-08-30T08:56:22.032+0000 7f758e944640 -1 osd.9 0 waiting for initial osdmap 2024-08-30T08:56:22.383 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 08:56:22 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9[55148]: 2024-08-30T08:56:22.044+0000 7f758673a640 -1 osd.9 56 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-30T08:56:22.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:56:22 smithi167 ceph-mon[22063]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-08-30T08:56:22.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:56:22 smithi167 ceph-mon[22063]: Detected new or changed devices on smithi042 2024-08-30T08:56:22.578 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:56:22 smithi167 ceph-mon[22063]: Adjusting osd_memory_target on smithi042 to 4273M 2024-08-30T08:56:22.578 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:56:22 smithi167 ceph-mon[22063]: from='osd.9 [v2:172.21.15.42:6826/1123536325,v1:172.21.15.42:6827/1123536325]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi042", "root=default"]}]': finished 2024-08-30T08:56:22.578 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:56:22 smithi167 ceph-mon[22063]: osdmap e56: 10 total, 9 up, 10 in 2024-08-30T08:56:22.578 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:56:22 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-08-30T08:56:22.578 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:56:22 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-08-30T08:56:22.578 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:56:22 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-30T08:56:22.578 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:56:22 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-30T08:56:22.578 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:56:22 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:56:22.578 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:56:22 smithi167 ceph-mon[22063]: from='osd.9 [v2:172.21.15.42:6826/1123536325,v1:172.21.15.42:6827/1123536325]' entity='osd.9' 2024-08-30T08:56:22.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:56:22 smithi042 ceph-mon[18652]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-08-30T08:56:22.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:56:22 smithi042 ceph-mon[18652]: Detected new or changed devices on smithi042 2024-08-30T08:56:22.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:56:22 smithi042 ceph-mon[18652]: Adjusting osd_memory_target on smithi042 to 4273M 2024-08-30T08:56:22.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:56:22 smithi042 ceph-mon[18652]: from='osd.9 [v2:172.21.15.42:6826/1123536325,v1:172.21.15.42:6827/1123536325]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi042", "root=default"]}]': finished 2024-08-30T08:56:22.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:56:22 smithi042 ceph-mon[18652]: osdmap e56: 10 total, 9 up, 10 in 2024-08-30T08:56:22.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:56:22 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-08-30T08:56:22.884 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:56:22 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-08-30T08:56:22.884 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:56:22 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-30T08:56:22.884 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:56:22 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-30T08:56:22.884 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:56:22 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:56:22.884 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:56:22 smithi042 ceph-mon[18652]: from='osd.9 [v2:172.21.15.42:6826/1123536325,v1:172.21.15.42:6827/1123536325]' entity='osd.9' 2024-08-30T08:56:22.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:56:22 smithi190 ceph-mon[22001]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-08-30T08:56:22.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:56:22 smithi190 ceph-mon[22001]: Detected new or changed devices on smithi042 2024-08-30T08:56:22.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:56:22 smithi190 ceph-mon[22001]: Adjusting osd_memory_target on smithi042 to 4273M 2024-08-30T08:56:22.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:56:22 smithi190 ceph-mon[22001]: from='osd.9 [v2:172.21.15.42:6826/1123536325,v1:172.21.15.42:6827/1123536325]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi042", "root=default"]}]': finished 2024-08-30T08:56:22.937 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:56:22 smithi190 ceph-mon[22001]: osdmap e56: 10 total, 9 up, 10 in 2024-08-30T08:56:22.937 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:56:22 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-08-30T08:56:22.937 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:56:22 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-08-30T08:56:22.937 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:56:22 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-30T08:56:22.937 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:56:22 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-30T08:56:22.937 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:56:22 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:56:22.937 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:56:22 smithi190 ceph-mon[22001]: from='osd.9 [v2:172.21.15.42:6826/1123536325,v1:172.21.15.42:6827/1123536325]' entity='osd.9' 2024-08-30T08:56:23.668 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:56:23 smithi167 ceph-mon[22063]: purged_snaps scrub starts 2024-08-30T08:56:23.669 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:56:23 smithi167 ceph-mon[22063]: purged_snaps scrub ok 2024-08-30T08:56:23.669 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:56:23 smithi167 ceph-mon[22063]: from='client.24470 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-08-30T08:56:23.669 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:56:23 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-08-30T08:56:23.669 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:56:23 smithi167 ceph-mon[22063]: osd.9 [v2:172.21.15.42:6826/1123536325,v1:172.21.15.42:6827/1123536325] boot 2024-08-30T08:56:23.669 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:56:23 smithi167 ceph-mon[22063]: osdmap e57: 10 total, 10 up, 10 in 2024-08-30T08:56:23.669 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:56:23 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-08-30T08:56:23.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:56:23 smithi042 ceph-mon[18652]: purged_snaps scrub starts 2024-08-30T08:56:23.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:56:23 smithi042 ceph-mon[18652]: purged_snaps scrub ok 2024-08-30T08:56:23.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:56:23 smithi042 ceph-mon[18652]: from='client.24470 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-08-30T08:56:23.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:56:23 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-08-30T08:56:23.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:56:23 smithi042 ceph-mon[18652]: osd.9 [v2:172.21.15.42:6826/1123536325,v1:172.21.15.42:6827/1123536325] boot 2024-08-30T08:56:23.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:56:23 smithi042 ceph-mon[18652]: osdmap e57: 10 total, 10 up, 10 in 2024-08-30T08:56:23.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:56:23 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-08-30T08:56:23.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:56:23 smithi190 ceph-mon[22001]: purged_snaps scrub starts 2024-08-30T08:56:23.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:56:23 smithi190 ceph-mon[22001]: purged_snaps scrub ok 2024-08-30T08:56:23.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:56:23 smithi190 ceph-mon[22001]: from='client.24470 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-08-30T08:56:23.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:56:23 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-08-30T08:56:23.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:56:23 smithi190 ceph-mon[22001]: osd.9 [v2:172.21.15.42:6826/1123536325,v1:172.21.15.42:6827/1123536325] boot 2024-08-30T08:56:23.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:56:23 smithi190 ceph-mon[22001]: osdmap e57: 10 total, 10 up, 10 in 2024-08-30T08:56:23.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:56:23 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-08-30T08:56:24.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:56:24 smithi167 ceph-mon[22063]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-08-30T08:56:24.778 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:56:24 smithi167 ceph-mon[22063]: from='client.? 172.21.15.167:0/960502576' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "96ef5058-2518-4862-a165-17666b33a3e6"} : dispatch 2024-08-30T08:56:24.778 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:56:24 smithi167 ceph-mon[22063]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "96ef5058-2518-4862-a165-17666b33a3e6"} : dispatch 2024-08-30T08:56:24.778 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:56:24 smithi167 ceph-mon[22063]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "96ef5058-2518-4862-a165-17666b33a3e6"}]': finished 2024-08-30T08:56:24.778 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:56:24 smithi167 ceph-mon[22063]: osdmap e58: 11 total, 10 up, 11 in 2024-08-30T08:56:24.778 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:56:24 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-08-30T08:56:24.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:56:24 smithi042 ceph-mon[18652]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-08-30T08:56:24.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:56:24 smithi042 ceph-mon[18652]: from='client.? 172.21.15.167:0/960502576' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "96ef5058-2518-4862-a165-17666b33a3e6"} : dispatch 2024-08-30T08:56:24.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:56:24 smithi042 ceph-mon[18652]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "96ef5058-2518-4862-a165-17666b33a3e6"} : dispatch 2024-08-30T08:56:24.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:56:24 smithi042 ceph-mon[18652]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "96ef5058-2518-4862-a165-17666b33a3e6"}]': finished 2024-08-30T08:56:24.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:56:24 smithi042 ceph-mon[18652]: osdmap e58: 11 total, 10 up, 11 in 2024-08-30T08:56:24.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:56:24 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-08-30T08:56:24.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:56:24 smithi190 ceph-mon[22001]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-08-30T08:56:24.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:56:24 smithi190 ceph-mon[22001]: from='client.? 172.21.15.167:0/960502576' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "96ef5058-2518-4862-a165-17666b33a3e6"} : dispatch 2024-08-30T08:56:24.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:56:24 smithi190 ceph-mon[22001]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "96ef5058-2518-4862-a165-17666b33a3e6"} : dispatch 2024-08-30T08:56:24.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:56:24 smithi190 ceph-mon[22001]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "96ef5058-2518-4862-a165-17666b33a3e6"}]': finished 2024-08-30T08:56:24.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:56:24 smithi190 ceph-mon[22001]: osdmap e58: 11 total, 10 up, 11 in 2024-08-30T08:56:24.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:56:24 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-08-30T08:56:25.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:56:25 smithi167 ceph-mon[22063]: from='client.? 172.21.15.167:0/3301575854' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-30T08:56:25.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:56:25 smithi042 ceph-mon[18652]: from='client.? 172.21.15.167:0/3301575854' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-30T08:56:25.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:56:25 smithi190 ceph-mon[22001]: from='client.? 172.21.15.167:0/3301575854' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-30T08:56:26.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:56:26 smithi167 ceph-mon[22063]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-08-30T08:56:26.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:56:26 smithi042 ceph-mon[18652]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-08-30T08:56:26.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:56:26 smithi190 ceph-mon[22001]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-08-30T08:56:28.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:56:28 smithi167 ceph-mon[22063]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-08-30T08:56:28.882 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:56:28 smithi042 ceph-mon[18652]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-08-30T08:56:28.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:56:28 smithi190 ceph-mon[22001]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-08-30T08:56:30.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:56:30 smithi167 ceph-mon[22063]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-08-30T08:56:30.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:56:30 smithi042 ceph-mon[18652]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-08-30T08:56:30.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:56:30 smithi190 ceph-mon[22001]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-08-30T08:56:32.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:56:32 smithi167 ceph-mon[22063]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-08-30T08:56:32.882 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:56:32 smithi042 ceph-mon[18652]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-08-30T08:56:32.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:56:32 smithi190 ceph-mon[22001]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-08-30T08:56:33.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:56:33 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.10"} : dispatch 2024-08-30T08:56:33.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:56:33 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:56:33.882 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:56:33 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.10"} : dispatch 2024-08-30T08:56:33.927 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:56:33 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:56:33.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:56:33 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.10"} : dispatch 2024-08-30T08:56:33.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:56:33 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:56:34.798 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:56:34 smithi167 ceph-mon[22063]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-08-30T08:56:34.798 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:56:34 smithi167 ceph-mon[22063]: Deploying daemon osd.10 on smithi167 2024-08-30T08:56:34.882 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:56:34 smithi042 ceph-mon[18652]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-08-30T08:56:34.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:56:34 smithi042 ceph-mon[18652]: Deploying daemon osd.10 on smithi167 2024-08-30T08:56:34.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:56:34 smithi190 ceph-mon[22001]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-08-30T08:56:34.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:56:34 smithi190 ceph-mon[22001]: Deploying daemon osd.10 on smithi167 2024-08-30T08:56:36.689 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:56:36 smithi167 ceph-mon[22063]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-08-30T08:56:36.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:56:36 smithi042 ceph-mon[18652]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-08-30T08:56:36.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:56:36 smithi190 ceph-mon[22001]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-08-30T08:56:38.762 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:56:38 smithi167 ceph-mon[22063]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-08-30T08:56:38.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:56:38 smithi042 ceph-mon[18652]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-08-30T08:56:38.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:56:38 smithi190 ceph-mon[22001]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-08-30T08:56:39.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:56:39 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-30T08:56:39.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:56:39 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:56:39.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:56:39 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:56:39.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:56:39 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-30T08:56:39.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:56:39 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:56:39.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:56:39 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:56:39.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:56:39 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-30T08:56:39.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:56:39 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:56:39.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:56:39 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:56:40.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:56:40 smithi167 ceph-mon[22063]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-08-30T08:56:40.837 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:56:40 smithi190 ceph-mon[22001]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-08-30T08:56:40.882 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:56:40 smithi042 ceph-mon[18652]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-08-30T08:56:42.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:56:42 smithi167 ceph-mon[22063]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-08-30T08:56:42.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:56:42 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:56:42.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:56:42 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:56:42.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:56:42 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:56:42.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:56:42 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-30T08:56:42.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:56:42 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:56:42.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:56:42 smithi042 ceph-mon[18652]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-08-30T08:56:42.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:56:42 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:56:42.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:56:42 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:56:42.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:56:42 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:56:42.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:56:42 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-30T08:56:42.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:56:42 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:56:42.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:56:42 smithi190 ceph-mon[22001]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-08-30T08:56:42.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:56:42 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:56:42.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:56:42 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:56:42.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:56:42 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:56:42.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:56:42 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-30T08:56:42.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:56:42 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:56:43.021 INFO:teuthology.orchestra.run.smithi167.stdout:Created osd(s) 10 on host 'smithi167' 2024-08-30T08:56:43.021 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:56:43.019+0000 7fb5cb7fe640 1 -- 172.21.15.167:0/2022129010 <== mgr.14150 v2:172.21.15.42:6800/3005140902 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fb5dc0635a0 con 0x7fb5c40b4b30 2024-08-30T08:56:43.023 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:56:43.022+0000 7fb5e0eae640 1 -- 172.21.15.167:0/2022129010 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fb5c40b4b30 msgr2=0x7fb5c40b7040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:56:43.023 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:56:43.022+0000 7fb5e0eae640 1 --2- 172.21.15.167:0/2022129010 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fb5c40b4b30 0x7fb5c40b7040 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fb5cc0058c0 tx=0x7fb5cc005850 comp rx=0 tx=0).stop 2024-08-30T08:56:43.024 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:56:43.022+0000 7fb5e0eae640 1 -- 172.21.15.167:0/2022129010 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb5dc104580 msgr2=0x7fb5dc19a420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:56:43.024 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:56:43.022+0000 7fb5e0eae640 1 --2- 172.21.15.167:0/2022129010 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb5dc104580 0x7fb5dc19a420 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fb5d0006010 tx=0x7fb5d0004340 comp rx=0 tx=0).stop 2024-08-30T08:56:43.024 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:56:43.022+0000 7fb5e0eae640 1 -- 172.21.15.167:0/2022129010 shutdown_connections 2024-08-30T08:56:43.024 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:56:43.022+0000 7fb5e0eae640 1 --2- 172.21.15.167:0/2022129010 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fb5c40b4b30 0x7fb5c40b7040 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:56:43.024 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:56:43.022+0000 7fb5e0eae640 1 --2- 172.21.15.167:0/2022129010 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb5dc104f20 0x7fb5dc19afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:56:43.024 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:56:43.022+0000 7fb5e0eae640 1 --2- 172.21.15.167:0/2022129010 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb5dc104580 0x7fb5dc19a420 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:56:43.024 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:56:43.022+0000 7fb5e0eae640 1 --2- 172.21.15.167:0/2022129010 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb5dc1034f0 0x7fb5dc199db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:56:43.024 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:56:43.022+0000 7fb5e0eae640 1 -- 172.21.15.167:0/2022129010 >> 172.21.15.167:0/2022129010 conn(0x7fb5dc0fee60 msgr2=0x7fb5dc107880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:56:43.024 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:56:43.022+0000 7fb5e0eae640 1 -- 172.21.15.167:0/2022129010 shutdown_connections 2024-08-30T08:56:43.024 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-30T08:56:43.022+0000 7fb5e0eae640 1 -- 172.21.15.167:0/2022129010 wait complete. 2024-08-30T08:56:43.716 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:56:43 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-30T08:56:43.716 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:56:43 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:56:43.716 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:56:43 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:56:43.751 DEBUG:teuthology.orchestra.run.smithi167:osd.10> sudo journalctl -f -n 0 -u ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968@osd.10.service 2024-08-30T08:56:43.754 INFO:tasks.cephadm:Deploying osd.11 on smithi190 with /dev/vg_nvme/lv_1... 2024-08-30T08:56:43.754 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-08-30T08:56:43.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:56:43 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-30T08:56:43.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:56:43 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:56:43.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:56:43 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:56:43.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:56:43 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-30T08:56:43.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:56:43 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:56:43.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:56:43 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:56:44.011 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.c/config 2024-08-30T08:56:44.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:56:44 smithi190 ceph-mon[22001]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-08-30T08:56:44.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:56:44 smithi167 ceph-mon[22063]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-08-30T08:56:44.827 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 08:56:44 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T08:56:44.628+0000 7f2d076a6640 -1 osd.10 0 log_to_monitors true 2024-08-30T08:56:44.882 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:56:44 smithi042 ceph-mon[18652]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-08-30T08:56:45.685 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:56:45 smithi190 ceph-mon[22001]: from='osd.10 [v2:172.21.15.167:6824/3451198319,v1:172.21.15.167:6825/3451198319]' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-08-30T08:56:45.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:56:45 smithi190 ceph-mon[22001]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-08-30T08:56:45.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:56:45 smithi167 ceph-mon[22063]: from='osd.10 [v2:172.21.15.167:6824/3451198319,v1:172.21.15.167:6825/3451198319]' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-08-30T08:56:45.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:56:45 smithi167 ceph-mon[22063]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-08-30T08:56:45.882 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:56:45 smithi042 ceph-mon[18652]: from='osd.10 [v2:172.21.15.167:6824/3451198319,v1:172.21.15.167:6825/3451198319]' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-08-30T08:56:45.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:56:45 smithi042 ceph-mon[18652]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-08-30T08:56:46.784 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:56:46 smithi167 ceph-mon[22063]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-08-30T08:56:46.784 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:56:46 smithi167 ceph-mon[22063]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-08-30T08:56:46.784 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:56:46 smithi167 ceph-mon[22063]: osdmap e59: 11 total, 10 up, 11 in 2024-08-30T08:56:46.785 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:56:46 smithi167 ceph-mon[22063]: from='osd.10 [v2:172.21.15.167:6824/3451198319,v1:172.21.15.167:6825/3451198319]' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi167", "root=default"]} : dispatch 2024-08-30T08:56:46.785 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:56:46 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-08-30T08:56:46.785 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:56:46 smithi167 ceph-mon[22063]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi167", "root=default"]} : dispatch 2024-08-30T08:56:46.785 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 08:56:46 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T08:56:46.579+0000 7f2d03628640 -1 osd.10 0 waiting for initial osdmap 2024-08-30T08:56:46.785 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 08:56:46 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T08:56:46.590+0000 7f2cff426640 -1 osd.10 60 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-30T08:56:46.802 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-08-30T08:56:46.828 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph orch daemon add osd smithi190:vg_nvme/lv_1 2024-08-30T08:56:46.852 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:56:46 smithi190 ceph-mon[22001]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-08-30T08:56:46.852 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:56:46 smithi190 ceph-mon[22001]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-08-30T08:56:46.852 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:56:46 smithi190 ceph-mon[22001]: osdmap e59: 11 total, 10 up, 11 in 2024-08-30T08:56:46.852 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:56:46 smithi190 ceph-mon[22001]: from='osd.10 [v2:172.21.15.167:6824/3451198319,v1:172.21.15.167:6825/3451198319]' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi167", "root=default"]} : dispatch 2024-08-30T08:56:46.852 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:56:46 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-08-30T08:56:46.852 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:56:46 smithi190 ceph-mon[22001]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi167", "root=default"]} : dispatch 2024-08-30T08:56:46.882 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:56:46 smithi042 ceph-mon[18652]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-08-30T08:56:46.882 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:56:46 smithi042 ceph-mon[18652]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-08-30T08:56:46.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:56:46 smithi042 ceph-mon[18652]: osdmap e59: 11 total, 10 up, 11 in 2024-08-30T08:56:46.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:56:46 smithi042 ceph-mon[18652]: from='osd.10 [v2:172.21.15.167:6824/3451198319,v1:172.21.15.167:6825/3451198319]' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi167", "root=default"]} : dispatch 2024-08-30T08:56:46.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:56:46 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-08-30T08:56:46.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:56:46 smithi042 ceph-mon[18652]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi167", "root=default"]} : dispatch 2024-08-30T08:56:47.083 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.c/config 2024-08-30T08:56:47.602 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:56:47.595+0000 7fbcb077e640 1 -- 172.21.15.190:0/923795005 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbca8104f20 msgr2=0x7fbca810b8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:56:47.602 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:56:47.595+0000 7fbcb077e640 1 --2- 172.21.15.190:0/923795005 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbca8104f20 0x7fbca810b8c0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fbca0009ca0 tx=0x7fbca002a950 comp rx=0 tx=0).stop 2024-08-30T08:56:47.603 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:56:47.596+0000 7fbcb077e640 1 -- 172.21.15.190:0/923795005 shutdown_connections 2024-08-30T08:56:47.603 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:56:47.596+0000 7fbcb077e640 1 --2- 172.21.15.190:0/923795005 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbca8104f20 0x7fbca810b8c0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:56:47.603 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:56:47.596+0000 7fbcb077e640 1 --2- 172.21.15.190:0/923795005 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbca8104580 0x7fbca8104990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:56:47.603 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:56:47.596+0000 7fbcb077e640 1 --2- 172.21.15.190:0/923795005 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbca81034f0 0x7fbca8103960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:56:47.603 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:56:47.596+0000 7fbcb077e640 1 -- 172.21.15.190:0/923795005 >> 172.21.15.190:0/923795005 conn(0x7fbca80fee60 msgr2=0x7fbca8101310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:56:47.603 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:56:47.596+0000 7fbcb077e640 1 -- 172.21.15.190:0/923795005 shutdown_connections 2024-08-30T08:56:47.604 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:56:47.596+0000 7fbcb077e640 1 -- 172.21.15.190:0/923795005 wait complete. 2024-08-30T08:56:47.604 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:56:47.596+0000 7fbcb077e640 1 Processor -- start 2024-08-30T08:56:47.604 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:56:47.596+0000 7fbcb077e640 1 -- start start 2024-08-30T08:56:47.604 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:56:47.596+0000 7fbcb077e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbca81034f0 0x7fbca8199cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:56:47.604 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:56:47.596+0000 7fbcb077e640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbca8104580 0x7fbca819a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:56:47.604 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:56:47.596+0000 7fbcb077e640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbca8104f20 0x7fbca819af00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:56:47.604 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:56:47.596+0000 7fbcb077e640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fbca810f130 con 0x7fbca8104f20 2024-08-30T08:56:47.604 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:56:47.596+0000 7fbcb077e640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fbca810efb0 con 0x7fbca8104580 2024-08-30T08:56:47.604 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:56:47.596+0000 7fbcb077e640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fbca810f2b0 con 0x7fbca81034f0 2024-08-30T08:56:47.604 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:56:47.596+0000 7fbcae51a640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbca81034f0 0x7fbca8199cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:56:47.604 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:56:47.597+0000 7fbcae51a640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbca81034f0 0x7fbca8199cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:34890/0 (socket says 172.21.15.190:34890) 2024-08-30T08:56:47.604 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:56:47.597+0000 7fbcae51a640 1 -- 172.21.15.190:0/3997659790 learned_addr learned my addr 172.21.15.190:0/3997659790 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-30T08:56:47.605 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:56:47.597+0000 7fbcaed1b640 1 --2- 172.21.15.190:0/3997659790 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbca8104f20 0x7fbca819af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:56:47.605 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:56:47.597+0000 7fbcadd19640 1 --2- 172.21.15.190:0/3997659790 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbca8104580 0x7fbca819a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:56:47.605 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:56:47.597+0000 7fbcae51a640 1 -- 172.21.15.190:0/3997659790 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbca8104580 msgr2=0x7fbca819a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:56:47.605 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:56:47.597+0000 7fbcae51a640 1 --2- 172.21.15.190:0/3997659790 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbca8104580 0x7fbca819a360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:56:47.605 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:56:47.597+0000 7fbcae51a640 1 -- 172.21.15.190:0/3997659790 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbca8104f20 msgr2=0x7fbca819af00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:56:47.605 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:56:47.597+0000 7fbcae51a640 1 --2- 172.21.15.190:0/3997659790 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbca8104f20 0x7fbca819af00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:56:47.605 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:56:47.597+0000 7fbcae51a640 1 -- 172.21.15.190:0/3997659790 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbca819b720 con 0x7fbca81034f0 2024-08-30T08:56:47.605 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:56:47.597+0000 7fbcae51a640 1 --2- 172.21.15.190:0/3997659790 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbca81034f0 0x7fbca8199cf0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fbca400e9f0 tx=0x7fbca400edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:56:47.605 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:56:47.597+0000 7fbc9b7fe640 1 -- 172.21.15.190:0/3997659790 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbca400cd80 con 0x7fbca81034f0 2024-08-30T08:56:47.605 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:56:47.597+0000 7fbc9b7fe640 1 -- 172.21.15.190:0/3997659790 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbca40054d0 con 0x7fbca81034f0 2024-08-30T08:56:47.605 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:56:47.597+0000 7fbcb077e640 1 -- 172.21.15.190:0/3997659790 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbca819ba10 con 0x7fbca81034f0 2024-08-30T08:56:47.605 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:56:47.597+0000 7fbc9b7fe640 1 -- 172.21.15.190:0/3997659790 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbca4010690 con 0x7fbca81034f0 2024-08-30T08:56:47.605 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:56:47.597+0000 7fbcb077e640 1 -- 172.21.15.190:0/3997659790 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbca81a2110 con 0x7fbca81034f0 2024-08-30T08:56:47.605 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:56:47.598+0000 7fbcb077e640 1 -- 172.21.15.190:0/3997659790 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbca8103960 con 0x7fbca81034f0 2024-08-30T08:56:47.606 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:56:47.601+0000 7fbc9b7fe640 1 -- 172.21.15.190:0/3997659790 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 16) ==== 152577+0+0 (secure 0 0 0) 0x7fbca4004e30 con 0x7fbca81034f0 2024-08-30T08:56:47.606 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:56:47.602+0000 7fbc9b7fe640 1 --2- 172.21.15.190:0/3997659790 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fbc940b4b30 0x7fbc940b7040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:56:47.606 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:56:47.602+0000 7fbc9b7fe640 1 -- 172.21.15.190:0/3997659790 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 6727+0+0 (secure 0 0 0) 0x7fbca40e51f0 con 0x7fbca81034f0 2024-08-30T08:56:47.606 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:56:47.602+0000 7fbcadd19640 1 --2- 172.21.15.190:0/3997659790 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fbc940b4b30 0x7fbc940b7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:56:47.606 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:56:47.602+0000 7fbcadd19640 1 --2- 172.21.15.190:0/3997659790 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fbc940b4b30 0x7fbc940b7040 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fbc9c009aa0 tx=0x7fbc9c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:56:47.607 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:56:47.605+0000 7fbc9b7fe640 1 -- 172.21.15.190:0/3997659790 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7fbca4056130 con 0x7fbca81034f0 2024-08-30T08:56:47.795 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:56:47.791+0000 7fbcb077e640 1 -- 172.21.15.190:0/3997659790 --> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi190:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fbca80635a0 con 0x7fbc940b4b30 2024-08-30T08:56:47.795 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:56:47 smithi190 ceph-mon[22001]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]': finished 2024-08-30T08:56:47.795 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:56:47 smithi190 ceph-mon[22001]: osdmap e60: 11 total, 10 up, 11 in 2024-08-30T08:56:47.795 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:56:47 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-08-30T08:56:47.795 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:56:47 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-08-30T08:56:47.795 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:56:47 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:56:47.795 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:56:47 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:56:47.795 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:56:47 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-08-30T08:56:47.795 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:56:47 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-08-30T08:56:47.795 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:56:47 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-08-30T08:56:47.796 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:56:47 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-08-30T08:56:47.796 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:56:47 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:56:47.796 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:56:47 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:56:47.796 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:56:47 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-30T08:56:47.796 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:56:47 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:56:47.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:56:47 smithi042 ceph-mon[18652]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]': finished 2024-08-30T08:56:47.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:56:47 smithi042 ceph-mon[18652]: osdmap e60: 11 total, 10 up, 11 in 2024-08-30T08:56:47.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:56:47 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-08-30T08:56:47.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:56:47 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-08-30T08:56:47.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:56:47 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:56:47.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:56:47 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:56:47.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:56:47 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-08-30T08:56:47.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:56:47 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-08-30T08:56:47.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:56:47 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-08-30T08:56:47.884 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:56:47 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-08-30T08:56:47.884 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:56:47 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:56:47.884 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:56:47 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:56:47.884 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:56:47 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-30T08:56:47.884 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:56:47 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:56:48.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:56:47 smithi167 ceph-mon[22063]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]': finished 2024-08-30T08:56:48.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:56:47 smithi167 ceph-mon[22063]: osdmap e60: 11 total, 10 up, 11 in 2024-08-30T08:56:48.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:56:47 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-08-30T08:56:48.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:56:47 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-08-30T08:56:48.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:56:47 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:56:48.078 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:56:47 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:56:48.078 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:56:47 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-08-30T08:56:48.078 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:56:47 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-08-30T08:56:48.078 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:56:47 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-08-30T08:56:48.078 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:56:47 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-08-30T08:56:48.078 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:56:47 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:56:48.078 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:56:47 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:56:48.078 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:56:47 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-30T08:56:48.078 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:56:47 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:56:48.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:56:48 smithi042 ceph-mon[18652]: purged_snaps scrub starts 2024-08-30T08:56:48.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:56:48 smithi042 ceph-mon[18652]: purged_snaps scrub ok 2024-08-30T08:56:48.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:56:48 smithi042 ceph-mon[18652]: Detected new or changed devices on smithi167 2024-08-30T08:56:48.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:56:48 smithi042 ceph-mon[18652]: Adjusting osd_memory_target on smithi167 to 4272M 2024-08-30T08:56:48.884 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:56:48 smithi042 ceph-mon[18652]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-08-30T08:56:48.884 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:56:48 smithi042 ceph-mon[18652]: osd.10 [v2:172.21.15.167:6824/3451198319,v1:172.21.15.167:6825/3451198319] boot 2024-08-30T08:56:48.884 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:56:48 smithi042 ceph-mon[18652]: osdmap e61: 11 total, 11 up, 11 in 2024-08-30T08:56:48.884 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:56:48 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-08-30T08:56:48.884 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:56:48 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-30T08:56:48.884 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:56:48 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-30T08:56:48.884 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:56:48 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:56:48.902 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:56:48 smithi190 ceph-mon[22001]: purged_snaps scrub starts 2024-08-30T08:56:48.902 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:56:48 smithi190 ceph-mon[22001]: purged_snaps scrub ok 2024-08-30T08:56:48.902 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:56:48 smithi190 ceph-mon[22001]: Detected new or changed devices on smithi167 2024-08-30T08:56:48.902 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:56:48 smithi190 ceph-mon[22001]: Adjusting osd_memory_target on smithi167 to 4272M 2024-08-30T08:56:48.902 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:56:48 smithi190 ceph-mon[22001]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-08-30T08:56:48.902 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:56:48 smithi190 ceph-mon[22001]: osd.10 [v2:172.21.15.167:6824/3451198319,v1:172.21.15.167:6825/3451198319] boot 2024-08-30T08:56:48.902 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:56:48 smithi190 ceph-mon[22001]: osdmap e61: 11 total, 11 up, 11 in 2024-08-30T08:56:48.902 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:56:48 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-08-30T08:56:48.902 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:56:48 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-30T08:56:48.903 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:56:48 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-30T08:56:48.903 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:56:48 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:56:49.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:56:48 smithi167 ceph-mon[22063]: purged_snaps scrub starts 2024-08-30T08:56:49.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:56:48 smithi167 ceph-mon[22063]: purged_snaps scrub ok 2024-08-30T08:56:49.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:56:48 smithi167 ceph-mon[22063]: Detected new or changed devices on smithi167 2024-08-30T08:56:49.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:56:48 smithi167 ceph-mon[22063]: Adjusting osd_memory_target on smithi167 to 4272M 2024-08-30T08:56:49.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:56:48 smithi167 ceph-mon[22063]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-08-30T08:56:49.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:56:48 smithi167 ceph-mon[22063]: osd.10 [v2:172.21.15.167:6824/3451198319,v1:172.21.15.167:6825/3451198319] boot 2024-08-30T08:56:49.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:56:48 smithi167 ceph-mon[22063]: osdmap e61: 11 total, 11 up, 11 in 2024-08-30T08:56:49.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:56:48 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-08-30T08:56:49.078 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:56:48 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-30T08:56:49.078 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:56:48 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-30T08:56:49.078 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:56:48 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:56:49.775 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:56:49 smithi190 ceph-mon[22001]: from='client.24493 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi190:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-08-30T08:56:49.775 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:56:49 smithi190 ceph-mon[22001]: osdmap e62: 11 total, 11 up, 11 in 2024-08-30T08:56:49.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:56:49 smithi042 ceph-mon[18652]: from='client.24493 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi190:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-08-30T08:56:49.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:56:49 smithi042 ceph-mon[18652]: osdmap e62: 11 total, 11 up, 11 in 2024-08-30T08:56:50.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:56:49 smithi167 ceph-mon[22063]: from='client.24493 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi190:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-08-30T08:56:50.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:56:49 smithi167 ceph-mon[22063]: osdmap e62: 11 total, 11 up, 11 in 2024-08-30T08:56:50.876 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:56:50 smithi190 ceph-mon[22001]: pgmap v234: 1 pgs: 1 remapped+peering; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-08-30T08:56:50.876 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:56:50 smithi190 ceph-mon[22001]: osdmap e63: 11 total, 11 up, 11 in 2024-08-30T08:56:50.877 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:56:50 smithi190 ceph-mon[22001]: from='client.? 172.21.15.190:0/332877823' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "a4eb9bd3-3faa-4c81-ba6d-c2084b72c9ee"} : dispatch 2024-08-30T08:56:50.877 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:56:50 smithi190 ceph-mon[22001]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "a4eb9bd3-3faa-4c81-ba6d-c2084b72c9ee"} : dispatch 2024-08-30T08:56:50.877 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:56:50 smithi190 ceph-mon[22001]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a4eb9bd3-3faa-4c81-ba6d-c2084b72c9ee"}]': finished 2024-08-30T08:56:50.877 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:56:50 smithi190 ceph-mon[22001]: osdmap e64: 12 total, 11 up, 12 in 2024-08-30T08:56:50.877 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:56:50 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-08-30T08:56:50.882 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:56:50 smithi042 ceph-mon[18652]: pgmap v234: 1 pgs: 1 remapped+peering; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-08-30T08:56:50.882 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:56:50 smithi042 ceph-mon[18652]: osdmap e63: 11 total, 11 up, 11 in 2024-08-30T08:56:50.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:56:50 smithi042 ceph-mon[18652]: from='client.? 172.21.15.190:0/332877823' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "a4eb9bd3-3faa-4c81-ba6d-c2084b72c9ee"} : dispatch 2024-08-30T08:56:50.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:56:50 smithi042 ceph-mon[18652]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "a4eb9bd3-3faa-4c81-ba6d-c2084b72c9ee"} : dispatch 2024-08-30T08:56:50.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:56:50 smithi042 ceph-mon[18652]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a4eb9bd3-3faa-4c81-ba6d-c2084b72c9ee"}]': finished 2024-08-30T08:56:50.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:56:50 smithi042 ceph-mon[18652]: osdmap e64: 12 total, 11 up, 12 in 2024-08-30T08:56:50.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:56:50 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-08-30T08:56:51.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:56:50 smithi167 ceph-mon[22063]: pgmap v234: 1 pgs: 1 remapped+peering; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-08-30T08:56:51.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:56:50 smithi167 ceph-mon[22063]: osdmap e63: 11 total, 11 up, 11 in 2024-08-30T08:56:51.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:56:50 smithi167 ceph-mon[22063]: from='client.? 172.21.15.190:0/332877823' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "a4eb9bd3-3faa-4c81-ba6d-c2084b72c9ee"} : dispatch 2024-08-30T08:56:51.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:56:50 smithi167 ceph-mon[22063]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "a4eb9bd3-3faa-4c81-ba6d-c2084b72c9ee"} : dispatch 2024-08-30T08:56:51.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:56:50 smithi167 ceph-mon[22063]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a4eb9bd3-3faa-4c81-ba6d-c2084b72c9ee"}]': finished 2024-08-30T08:56:51.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:56:50 smithi167 ceph-mon[22063]: osdmap e64: 12 total, 11 up, 12 in 2024-08-30T08:56:51.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:56:50 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-08-30T08:56:51.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:56:51 smithi042 ceph-mon[18652]: from='client.? 172.21.15.190:0/2010247970' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-30T08:56:51.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:56:51 smithi190 ceph-mon[22001]: from='client.? 172.21.15.190:0/2010247970' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-30T08:56:52.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:56:51 smithi167 ceph-mon[22063]: from='client.? 172.21.15.190:0/2010247970' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-30T08:56:52.882 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:56:52 smithi042 ceph-mon[18652]: pgmap v237: 1 pgs: 1 remapped+peering; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-08-30T08:56:52.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:56:52 smithi190 ceph-mon[22001]: pgmap v237: 1 pgs: 1 remapped+peering; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-08-30T08:56:53.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:56:52 smithi167 ceph-mon[22063]: pgmap v237: 1 pgs: 1 remapped+peering; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-08-30T08:56:54.871 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:56:54 smithi190 ceph-mon[22001]: pgmap v238: 1 pgs: 1 remapped+peering; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-08-30T08:56:54.882 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:56:54 smithi042 ceph-mon[18652]: pgmap v238: 1 pgs: 1 remapped+peering; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-08-30T08:56:55.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:56:54 smithi167 ceph-mon[22063]: pgmap v238: 1 pgs: 1 remapped+peering; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-08-30T08:56:56.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:56:56 smithi042 ceph-mon[18652]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail; 91 KiB/s, 0 objects/s recovering 2024-08-30T08:56:56.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:56:56 smithi190 ceph-mon[22001]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail; 91 KiB/s, 0 objects/s recovering 2024-08-30T08:56:57.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:56:56 smithi167 ceph-mon[22063]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail; 91 KiB/s, 0 objects/s recovering 2024-08-30T08:56:58.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:56:58 smithi042 ceph-mon[18652]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-08-30T08:56:58.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:56:58 smithi190 ceph-mon[22001]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-08-30T08:56:59.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:56:58 smithi167 ceph-mon[22063]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-08-30T08:56:59.853 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:56:59 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.11"} : dispatch 2024-08-30T08:56:59.853 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:56:59 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:56:59.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:56:59 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.11"} : dispatch 2024-08-30T08:56:59.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:56:59 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:57:00.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:56:59 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.11"} : dispatch 2024-08-30T08:57:00.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:56:59 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:57:00.808 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:00 smithi190 ceph-mon[22001]: Deploying daemon osd.11 on smithi190 2024-08-30T08:57:00.809 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:00 smithi190 ceph-mon[22001]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-08-30T08:57:00.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:00 smithi042 ceph-mon[18652]: Deploying daemon osd.11 on smithi190 2024-08-30T08:57:00.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:00 smithi042 ceph-mon[18652]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-08-30T08:57:01.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:00 smithi167 ceph-mon[22063]: Deploying daemon osd.11 on smithi190 2024-08-30T08:57:01.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:00 smithi167 ceph-mon[22063]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-08-30T08:57:02.815 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:02 smithi190 ceph-mon[22001]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-08-30T08:57:02.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:02 smithi042 ceph-mon[18652]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-08-30T08:57:03.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:02 smithi167 ceph-mon[22063]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-08-30T08:57:04.879 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:04 smithi167 ceph-mon[22063]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-08-30T08:57:04.879 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:04 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-30T08:57:04.879 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:04 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:04.879 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:04 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:04.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:04 smithi042 ceph-mon[18652]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-08-30T08:57:04.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:04 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-30T08:57:04.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:04 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:04.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:04 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:04.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:04 smithi190 ceph-mon[22001]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-08-30T08:57:04.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:04 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-30T08:57:04.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:04 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:04.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:04 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:06.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:06 smithi042 ceph-mon[18652]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-08-30T08:57:06.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:06 smithi190 ceph-mon[22001]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-08-30T08:57:07.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:06 smithi167 ceph-mon[22063]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-08-30T08:57:08.273 INFO:teuthology.orchestra.run.smithi190.stdout:Created osd(s) 11 on host 'smithi190' 2024-08-30T08:57:08.273 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:57:08.270+0000 7fbc9b7fe640 1 -- 172.21.15.190:0/3997659790 <== mgr.14150 v2:172.21.15.42:6800/3005140902 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fbca80635a0 con 0x7fbc940b4b30 2024-08-30T08:57:08.276 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:57:08.273+0000 7fbcb077e640 1 -- 172.21.15.190:0/3997659790 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fbc940b4b30 msgr2=0x7fbc940b7040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:57:08.276 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:57:08.273+0000 7fbcb077e640 1 --2- 172.21.15.190:0/3997659790 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fbc940b4b30 0x7fbc940b7040 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fbc9c009aa0 tx=0x7fbc9c005f90 comp rx=0 tx=0).stop 2024-08-30T08:57:08.276 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:57:08.273+0000 7fbcb077e640 1 -- 172.21.15.190:0/3997659790 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbca81034f0 msgr2=0x7fbca8199cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:57:08.276 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:57:08.273+0000 7fbcb077e640 1 --2- 172.21.15.190:0/3997659790 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbca81034f0 0x7fbca8199cf0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fbca400e9f0 tx=0x7fbca400edf0 comp rx=0 tx=0).stop 2024-08-30T08:57:08.276 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:57:08.273+0000 7fbcb077e640 1 -- 172.21.15.190:0/3997659790 shutdown_connections 2024-08-30T08:57:08.276 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:57:08.273+0000 7fbcb077e640 1 --2- 172.21.15.190:0/3997659790 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fbc940b4b30 0x7fbc940b7040 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:57:08.276 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:57:08.273+0000 7fbcb077e640 1 --2- 172.21.15.190:0/3997659790 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbca8104f20 0x7fbca819af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:57:08.276 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:57:08.273+0000 7fbcb077e640 1 --2- 172.21.15.190:0/3997659790 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbca8104580 0x7fbca819a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:57:08.277 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:57:08.273+0000 7fbcb077e640 1 --2- 172.21.15.190:0/3997659790 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbca81034f0 0x7fbca8199cf0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:57:08.277 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:57:08.273+0000 7fbcb077e640 1 -- 172.21.15.190:0/3997659790 >> 172.21.15.190:0/3997659790 conn(0x7fbca80fee60 msgr2=0x7fbca81078a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:57:08.277 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:57:08.274+0000 7fbcb077e640 1 -- 172.21.15.190:0/3997659790 shutdown_connections 2024-08-30T08:57:08.277 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:57:08.274+0000 7fbcb077e640 1 -- 172.21.15.190:0/3997659790 wait complete. 2024-08-30T08:57:08.581 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:08 smithi190 ceph-mon[22001]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-08-30T08:57:08.581 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:08 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:08.581 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:08 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:08.581 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:08 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:57:08.581 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:08 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-30T08:57:08.581 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:08 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:08.581 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:08 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-30T08:57:08.582 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:08 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:08.582 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:08 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:08.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:08 smithi042 ceph-mon[18652]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-08-30T08:57:08.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:08 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:08.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:08 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:08.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:08 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:57:08.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:08 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-30T08:57:08.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:08 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:08.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:08 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-30T08:57:08.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:08 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:08.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:08 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:08.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:08 smithi167 ceph-mon[22063]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-08-30T08:57:08.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:08 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:08.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:08 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:08.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:08 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:57:08.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:08 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-30T08:57:08.828 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:08 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:08.828 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:08 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-30T08:57:08.828 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:08 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:08.828 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:08 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:08.953 DEBUG:teuthology.orchestra.run.smithi190:osd.11> sudo journalctl -f -n 0 -u ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968@osd.11.service 2024-08-30T08:57:08.995 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-08-30T08:57:08.995 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph osd stat -f json 2024-08-30T08:57:09.259 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T08:57:09.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:09.729+0000 7fe95376d640 1 -- 172.21.15.42:0/967698789 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe94c1054f0 msgr2=0x7fe94c1079d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:57:09.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:09.729+0000 7fe95376d640 1 --2- 172.21.15.42:0/967698789 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe94c1054f0 0x7fe94c1079d0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fe940009900 tx=0x7fe94002a410 comp rx=0 tx=0).stop 2024-08-30T08:57:09.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:09.729+0000 7fe95376d640 1 -- 172.21.15.42:0/967698789 shutdown_connections 2024-08-30T08:57:09.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:09.729+0000 7fe95376d640 1 --2- 172.21.15.42:0/967698789 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe94c108110 0x7fe94c10a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:57:09.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:09.729+0000 7fe95376d640 1 --2- 172.21.15.42:0/967698789 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe94c1054f0 0x7fe94c1079d0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:57:09.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:09.729+0000 7fe95376d640 1 --2- 172.21.15.42:0/967698789 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe94c102a30 0x7fe94c104eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:57:09.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:09.729+0000 7fe95376d640 1 -- 172.21.15.42:0/967698789 >> 172.21.15.42:0/967698789 conn(0x7fe94c0fd510 msgr2=0x7fe94c0ff980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:57:09.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:09.729+0000 7fe95376d640 1 -- 172.21.15.42:0/967698789 shutdown_connections 2024-08-30T08:57:09.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:09.729+0000 7fe95376d640 1 -- 172.21.15.42:0/967698789 wait complete. 2024-08-30T08:57:09.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:09.729+0000 7fe95376d640 1 Processor -- start 2024-08-30T08:57:09.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:09.729+0000 7fe95376d640 1 -- start start 2024-08-30T08:57:09.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:09.730+0000 7fe95376d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe94c102a30 0x7fe94c199d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:57:09.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:09.730+0000 7fe95376d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe94c1054f0 0x7fe94c19a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:57:09.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:09.730+0000 7fe95376d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe94c108110 0x7fe94c19af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:57:09.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:09.730+0000 7fe95376d640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fe94c10e030 con 0x7fe94c1054f0 2024-08-30T08:57:09.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:09.730+0000 7fe95376d640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe94c10deb0 con 0x7fe94c102a30 2024-08-30T08:57:09.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:09.730+0000 7fe95376d640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fe94c10e1b0 con 0x7fe94c108110 2024-08-30T08:57:09.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:09.730+0000 7fe950d08640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe94c1054f0 0x7fe94c19a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:57:09.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:09.730+0000 7fe950d08640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe94c1054f0 0x7fe94c19a3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36132/0 (socket says 172.21.15.42:36132) 2024-08-30T08:57:09.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:09.730+0000 7fe950d08640 1 -- 172.21.15.42:0/1490429364 learned_addr learned my addr 172.21.15.42:0/1490429364 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T08:57:09.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:09.730+0000 7fe951d0a640 1 --2- 172.21.15.42:0/1490429364 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe94c108110 0x7fe94c19af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:57:09.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:09.730+0000 7fe951509640 1 --2- 172.21.15.42:0/1490429364 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe94c102a30 0x7fe94c199d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:57:09.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:09.730+0000 7fe950d08640 1 -- 172.21.15.42:0/1490429364 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe94c108110 msgr2=0x7fe94c19af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:57:09.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:09.730+0000 7fe950d08640 1 --2- 172.21.15.42:0/1490429364 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe94c108110 0x7fe94c19af40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:57:09.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:09.730+0000 7fe950d08640 1 -- 172.21.15.42:0/1490429364 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe94c102a30 msgr2=0x7fe94c199d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:57:09.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:09.730+0000 7fe950d08640 1 --2- 172.21.15.42:0/1490429364 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe94c102a30 0x7fe94c199d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:57:09.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:09.730+0000 7fe950d08640 1 -- 172.21.15.42:0/1490429364 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe94c19b760 con 0x7fe94c1054f0 2024-08-30T08:57:09.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:09.730+0000 7fe950d08640 1 --2- 172.21.15.42:0/1490429364 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe94c1054f0 0x7fe94c19a3a0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fe940006010 tx=0x7fe940031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:57:09.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:09.731+0000 7fe93e7fc640 1 -- 172.21.15.42:0/1490429364 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe94003f030 con 0x7fe94c1054f0 2024-08-30T08:57:09.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:09.731+0000 7fe95376d640 1 -- 172.21.15.42:0/1490429364 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe94c19b9f0 con 0x7fe94c1054f0 2024-08-30T08:57:09.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:09.731+0000 7fe93e7fc640 1 -- 172.21.15.42:0/1490429364 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe940003aa0 con 0x7fe94c1054f0 2024-08-30T08:57:09.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:09.731+0000 7fe93e7fc640 1 -- 172.21.15.42:0/1490429364 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe940035070 con 0x7fe94c1054f0 2024-08-30T08:57:09.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:09.731+0000 7fe95376d640 1 -- 172.21.15.42:0/1490429364 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe94c1a2150 con 0x7fe94c1054f0 2024-08-30T08:57:09.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:09.732+0000 7fe95376d640 1 -- 172.21.15.42:0/1490429364 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe94c1059c0 con 0x7fe94c1054f0 2024-08-30T08:57:09.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:09.733+0000 7fe93e7fc640 1 -- 172.21.15.42:0/1490429364 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) ==== 152577+0+0 (secure 0 0 0) 0x7fe940042030 con 0x7fe94c1054f0 2024-08-30T08:57:09.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:09.734+0000 7fe93e7fc640 1 --2- 172.21.15.42:0/1490429364 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fe9380b4b30 0x7fe9380b7040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:57:09.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:09.734+0000 7fe951509640 1 --2- 172.21.15.42:0/1490429364 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fe9380b4b30 0x7fe9380b7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:57:09.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:09.734+0000 7fe93e7fc640 1 -- 172.21.15.42:0/1490429364 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 6809+0+0 (secure 0 0 0) 0x7fe940105120 con 0x7fe94c1054f0 2024-08-30T08:57:09.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:09.735+0000 7fe951509640 1 --2- 172.21.15.42:0/1490429364 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fe9380b4b30 0x7fe9380b7040 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fe948005d00 tx=0x7fe94800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:57:09.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:09.739+0000 7fe93e7fc640 1 -- 172.21.15.42:0/1490429364 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7fe9400cc140 con 0x7fe94c1054f0 2024-08-30T08:57:09.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:09.926+0000 7fe95376d640 1 -- 172.21.15.42:0/1490429364 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7fe94c1a27e0 con 0x7fe94c1054f0 2024-08-30T08:57:09.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:09.926+0000 7fe93e7fc640 1 -- 172.21.15.42:0/1490429364 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v64) ==== 74+0+133 (secure 0 0 0) 0x7fe9400d0900 con 0x7fe94c1054f0 2024-08-30T08:57:09.928 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-08-30T08:57:09.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:09.928+0000 7fe95376d640 1 -- 172.21.15.42:0/1490429364 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fe9380b4b30 msgr2=0x7fe9380b7040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:57:09.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:09.928+0000 7fe95376d640 1 --2- 172.21.15.42:0/1490429364 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fe9380b4b30 0x7fe9380b7040 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fe948005d00 tx=0x7fe94800b410 comp rx=0 tx=0).stop 2024-08-30T08:57:09.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:09.928+0000 7fe95376d640 1 -- 172.21.15.42:0/1490429364 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe94c1054f0 msgr2=0x7fe94c19a3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:57:09.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:09.928+0000 7fe95376d640 1 --2- 172.21.15.42:0/1490429364 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe94c1054f0 0x7fe94c19a3a0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fe940006010 tx=0x7fe940031a50 comp rx=0 tx=0).stop 2024-08-30T08:57:09.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:09.929+0000 7fe95376d640 1 -- 172.21.15.42:0/1490429364 shutdown_connections 2024-08-30T08:57:09.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:09.929+0000 7fe95376d640 1 --2- 172.21.15.42:0/1490429364 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fe9380b4b30 0x7fe9380b7040 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:57:09.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:09.929+0000 7fe95376d640 1 --2- 172.21.15.42:0/1490429364 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe94c108110 0x7fe94c19af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:57:09.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:09.929+0000 7fe95376d640 1 --2- 172.21.15.42:0/1490429364 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe94c1054f0 0x7fe94c19a3a0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:57:09.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:09.929+0000 7fe95376d640 1 --2- 172.21.15.42:0/1490429364 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe94c102a30 0x7fe94c199d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:57:09.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:09.929+0000 7fe95376d640 1 -- 172.21.15.42:0/1490429364 >> 172.21.15.42:0/1490429364 conn(0x7fe94c0fd510 msgr2=0x7fe94c1076b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:57:09.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:09.929+0000 7fe95376d640 1 -- 172.21.15.42:0/1490429364 shutdown_connections 2024-08-30T08:57:09.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:09.929+0000 7fe95376d640 1 -- 172.21.15.42:0/1490429364 wait complete. 2024-08-30T08:57:09.936 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 08:57:09 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T08:57:09.666+0000 7f0276f64640 -1 osd.11 0 log_to_monitors true 2024-08-30T08:57:10.424 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:10 smithi190 ceph-mon[22001]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-08-30T08:57:10.424 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:10 smithi190 ceph-mon[22001]: from='osd.11 [v2:172.21.15.190:6824/380064457,v1:172.21.15.190:6825/380064457]' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-08-30T08:57:10.424 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:10 smithi190 ceph-mon[22001]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-08-30T08:57:10.424 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:10 smithi190 ceph-mon[22001]: from='client.? 172.21.15.42:0/1490429364' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-08-30T08:57:10.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:10 smithi042 ceph-mon[18652]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-08-30T08:57:10.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:10 smithi042 ceph-mon[18652]: from='osd.11 [v2:172.21.15.190:6824/380064457,v1:172.21.15.190:6825/380064457]' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-08-30T08:57:10.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:10 smithi042 ceph-mon[18652]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-08-30T08:57:10.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:10 smithi042 ceph-mon[18652]: from='client.? 172.21.15.42:0/1490429364' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-08-30T08:57:10.721 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":64,"num_osds":12,"num_up_osds":11,"osd_up_since":1725008207,"num_in_osds":12,"osd_in_since":1725008210,"num_remapped_pgs":0} 2024-08-30T08:57:10.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:10 smithi167 ceph-mon[22063]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-08-30T08:57:10.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:10 smithi167 ceph-mon[22063]: from='osd.11 [v2:172.21.15.190:6824/380064457,v1:172.21.15.190:6825/380064457]' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-08-30T08:57:10.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:10 smithi167 ceph-mon[22063]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-08-30T08:57:10.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:10 smithi167 ceph-mon[22063]: from='client.? 172.21.15.42:0/1490429364' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-08-30T08:57:11.436 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 08:57:11 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T08:57:11.379+0000 7f0272ee6640 -1 osd.11 0 waiting for initial osdmap 2024-08-30T08:57:11.436 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 08:57:11 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T08:57:11.392+0000 7f026ece4640 -1 osd.11 66 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-30T08:57:11.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:11 smithi190 ceph-mon[22001]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-08-30T08:57:11.437 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:11 smithi190 ceph-mon[22001]: from='osd.11 [v2:172.21.15.190:6824/380064457,v1:172.21.15.190:6825/380064457]' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi190", "root=default"]} : dispatch 2024-08-30T08:57:11.437 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:11 smithi190 ceph-mon[22001]: osdmap e65: 12 total, 11 up, 12 in 2024-08-30T08:57:11.437 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:11 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-08-30T08:57:11.437 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:11 smithi190 ceph-mon[22001]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi190", "root=default"]} : dispatch 2024-08-30T08:57:11.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:11 smithi042 ceph-mon[18652]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-08-30T08:57:11.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:11 smithi042 ceph-mon[18652]: from='osd.11 [v2:172.21.15.190:6824/380064457,v1:172.21.15.190:6825/380064457]' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi190", "root=default"]} : dispatch 2024-08-30T08:57:11.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:11 smithi042 ceph-mon[18652]: osdmap e65: 12 total, 11 up, 12 in 2024-08-30T08:57:11.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:11 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-08-30T08:57:11.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:11 smithi042 ceph-mon[18652]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi190", "root=default"]} : dispatch 2024-08-30T08:57:11.723 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph osd stat -f json 2024-08-30T08:57:11.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:11 smithi167 ceph-mon[22063]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-08-30T08:57:11.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:11 smithi167 ceph-mon[22063]: from='osd.11 [v2:172.21.15.190:6824/380064457,v1:172.21.15.190:6825/380064457]' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi190", "root=default"]} : dispatch 2024-08-30T08:57:11.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:11 smithi167 ceph-mon[22063]: osdmap e65: 12 total, 11 up, 12 in 2024-08-30T08:57:11.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:11 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-08-30T08:57:11.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:11 smithi167 ceph-mon[22063]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi190", "root=default"]} : dispatch 2024-08-30T08:57:11.996 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T08:57:12.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:12.505+0000 7f6fcf067640 1 -- 172.21.15.42:0/2384052828 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6fc8104f20 msgr2=0x7f6fc810b8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:57:12.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:12.505+0000 7f6fcf067640 1 --2- 172.21.15.42:0/2384052828 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6fc8104f20 0x7f6fc810b8c0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f6fc0009ca0 tx=0x7f6fc002a950 comp rx=0 tx=0).stop 2024-08-30T08:57:12.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:12.505+0000 7f6fcf067640 1 -- 172.21.15.42:0/2384052828 shutdown_connections 2024-08-30T08:57:12.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:12.505+0000 7f6fcf067640 1 --2- 172.21.15.42:0/2384052828 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6fc8104f20 0x7f6fc810b8c0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:57:12.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:12.505+0000 7f6fcf067640 1 --2- 172.21.15.42:0/2384052828 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6fc8104580 0x7f6fc8104990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:57:12.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:12.505+0000 7f6fcf067640 1 --2- 172.21.15.42:0/2384052828 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6fc81034f0 0x7f6fc8103960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:57:12.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:12.505+0000 7f6fcf067640 1 -- 172.21.15.42:0/2384052828 >> 172.21.15.42:0/2384052828 conn(0x7f6fc80fee60 msgr2=0x7f6fc8101310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:57:12.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:12.505+0000 7f6fcf067640 1 -- 172.21.15.42:0/2384052828 shutdown_connections 2024-08-30T08:57:12.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:12.505+0000 7f6fcf067640 1 -- 172.21.15.42:0/2384052828 wait complete. 2024-08-30T08:57:12.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:12.506+0000 7f6fcf067640 1 Processor -- start 2024-08-30T08:57:12.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:12.506+0000 7f6fcf067640 1 -- start start 2024-08-30T08:57:12.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:12.506+0000 7f6fcf067640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6fc81034f0 0x7f6fc8076be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:57:12.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:12.506+0000 7f6fcf067640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6fc8104580 0x7f6fc80754b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:57:12.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:12.506+0000 7f6fcf067640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6fc8104f20 0x7f6fc80759f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:57:12.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:12.506+0000 7f6fcf067640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f6fc810f110 con 0x7f6fc8104580 2024-08-30T08:57:12.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:12.506+0000 7f6fcf067640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f6fc810ef90 con 0x7f6fc81034f0 2024-08-30T08:57:12.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:12.506+0000 7f6fcf067640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f6fc810f290 con 0x7f6fc8104f20 2024-08-30T08:57:12.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:12.506+0000 7f6fc7fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6fc8104580 0x7f6fc80754b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:57:12.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:12.506+0000 7f6fc7fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6fc8104580 0x7f6fc80754b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36150/0 (socket says 172.21.15.42:36150) 2024-08-30T08:57:12.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:12.506+0000 7f6fc7fff640 1 -- 172.21.15.42:0/4169732229 learned_addr learned my addr 172.21.15.42:0/4169732229 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T08:57:12.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:12.506+0000 7f6fcce03640 1 --2- 172.21.15.42:0/4169732229 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6fc81034f0 0x7f6fc8076be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:57:12.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:12.506+0000 7f6fcd604640 1 --2- 172.21.15.42:0/4169732229 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6fc8104f20 0x7f6fc80759f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:57:12.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:12.507+0000 7f6fc7fff640 1 -- 172.21.15.42:0/4169732229 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6fc8104f20 msgr2=0x7f6fc80759f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:57:12.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:12.507+0000 7f6fc7fff640 1 --2- 172.21.15.42:0/4169732229 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6fc8104f20 0x7f6fc80759f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:57:12.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:12.507+0000 7f6fc7fff640 1 -- 172.21.15.42:0/4169732229 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6fc81034f0 msgr2=0x7f6fc8076be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:57:12.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:12.507+0000 7f6fc7fff640 1 --2- 172.21.15.42:0/4169732229 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6fc81034f0 0x7f6fc8076be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:57:12.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:12.507+0000 7f6fc7fff640 1 -- 172.21.15.42:0/4169732229 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fc80760f0 con 0x7f6fc8104580 2024-08-30T08:57:12.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:12.507+0000 7f6fcd604640 1 --2- 172.21.15.42:0/4169732229 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6fc8104f20 0x7f6fc80759f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-30T08:57:12.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:12.507+0000 7f6fc7fff640 1 --2- 172.21.15.42:0/4169732229 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6fc8104580 0x7f6fc80754b0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f6fb000d950 tx=0x7f6fb000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:57:12.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:12.507+0000 7f6fc5ffb640 1 -- 172.21.15.42:0/4169732229 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fb0017070 con 0x7f6fc8104580 2024-08-30T08:57:12.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:12.507+0000 7f6fc5ffb640 1 -- 172.21.15.42:0/4169732229 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6fb0009c30 con 0x7f6fc8104580 2024-08-30T08:57:12.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:12.507+0000 7f6fcf067640 1 -- 172.21.15.42:0/4169732229 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fc81a3d60 con 0x7f6fc8104580 2024-08-30T08:57:12.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:12.507+0000 7f6fc5ffb640 1 -- 172.21.15.42:0/4169732229 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fb000f620 con 0x7f6fc8104580 2024-08-30T08:57:12.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:12.507+0000 7f6fcf067640 1 -- 172.21.15.42:0/4169732229 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fc81a41c0 con 0x7f6fc8104580 2024-08-30T08:57:12.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:12.508+0000 7f6fcf067640 1 -- 172.21.15.42:0/4169732229 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6fc810a6d0 con 0x7f6fc8104580 2024-08-30T08:57:12.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:12.510+0000 7f6fc5ffb640 1 -- 172.21.15.42:0/4169732229 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) ==== 152577+0+0 (secure 0 0 0) 0x7f6fb0010460 con 0x7f6fc8104580 2024-08-30T08:57:12.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:12.510+0000 7f6fc5ffb640 1 --2- 172.21.15.42:0/4169732229 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f6fb40b4b30 0x7f6fb40b7040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:57:12.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:12.510+0000 7f6fcce03640 1 --2- 172.21.15.42:0/4169732229 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f6fb40b4b30 0x7f6fb40b7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:57:12.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:12.510+0000 7f6fc5ffb640 1 -- 172.21.15.42:0/4169732229 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 7127+0+0 (secure 0 0 0) 0x7f6fb00e4860 con 0x7f6fc8104580 2024-08-30T08:57:12.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:12.511+0000 7f6fcce03640 1 --2- 172.21.15.42:0/4169732229 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f6fb40b4b30 0x7f6fb40b7040 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f6fb8005d00 tx=0x7f6fb800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:57:12.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:12.513+0000 7f6fc5ffb640 1 -- 172.21.15.42:0/4169732229 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f6fb00abe30 con 0x7f6fc8104580 2024-08-30T08:57:12.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:12 smithi042 ceph-mon[18652]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-08-30T08:57:12.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:12 smithi042 ceph-mon[18652]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]': finished 2024-08-30T08:57:12.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:12 smithi042 ceph-mon[18652]: osdmap e66: 12 total, 11 up, 12 in 2024-08-30T08:57:12.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:12 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-08-30T08:57:12.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:12 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-08-30T08:57:12.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:12 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:12.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:12 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:12.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:12 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-08-30T08:57:12.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:12 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-08-30T08:57:12.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:12 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-08-30T08:57:12.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:12 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-08-30T08:57:12.634 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:12 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:12.634 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:12 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:57:12.634 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:12 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-30T08:57:12.634 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:12 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:12.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:12 smithi190 ceph-mon[22001]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-08-30T08:57:12.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:12 smithi190 ceph-mon[22001]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]': finished 2024-08-30T08:57:12.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:12 smithi190 ceph-mon[22001]: osdmap e66: 12 total, 11 up, 12 in 2024-08-30T08:57:12.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:12 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-08-30T08:57:12.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:12 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-08-30T08:57:12.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:12 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:12.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:12 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:12.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:12 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-08-30T08:57:12.687 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:12 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-08-30T08:57:12.687 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:12 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-08-30T08:57:12.687 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:12 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-08-30T08:57:12.687 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:12 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:12.687 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:12 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:57:12.687 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:12 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-30T08:57:12.687 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:12 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:12.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:12.699+0000 7f6fcf067640 1 -- 172.21.15.42:0/4169732229 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f6fc802e6a0 con 0x7f6fc8104580 2024-08-30T08:57:12.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:12.700+0000 7f6fc5ffb640 1 -- 172.21.15.42:0/4169732229 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v67) ==== 74+0+133 (secure 0 0 0) 0x7f6fb0009de0 con 0x7f6fc8104580 2024-08-30T08:57:12.702 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-08-30T08:57:12.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:12.702+0000 7f6fcf067640 1 -- 172.21.15.42:0/4169732229 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f6fb40b4b30 msgr2=0x7f6fb40b7040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:57:12.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:12.702+0000 7f6fcf067640 1 --2- 172.21.15.42:0/4169732229 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f6fb40b4b30 0x7f6fb40b7040 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f6fb8005d00 tx=0x7f6fb800b410 comp rx=0 tx=0).stop 2024-08-30T08:57:12.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:12.702+0000 7f6fcf067640 1 -- 172.21.15.42:0/4169732229 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6fc8104580 msgr2=0x7f6fc80754b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:57:12.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:12.702+0000 7f6fcf067640 1 --2- 172.21.15.42:0/4169732229 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6fc8104580 0x7f6fc80754b0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f6fb000d950 tx=0x7f6fb000de10 comp rx=0 tx=0).stop 2024-08-30T08:57:12.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:12.702+0000 7f6fcf067640 1 -- 172.21.15.42:0/4169732229 shutdown_connections 2024-08-30T08:57:12.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:12.702+0000 7f6fcf067640 1 --2- 172.21.15.42:0/4169732229 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f6fb40b4b30 0x7f6fb40b7040 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:57:12.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:12.702+0000 7f6fcf067640 1 --2- 172.21.15.42:0/4169732229 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6fc8104f20 0x7f6fc80759f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:57:12.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:12.702+0000 7f6fcf067640 1 --2- 172.21.15.42:0/4169732229 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6fc8104580 0x7f6fc80754b0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:57:12.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:12.702+0000 7f6fcf067640 1 --2- 172.21.15.42:0/4169732229 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6fc81034f0 0x7f6fc8076be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:57:12.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:12.702+0000 7f6fcf067640 1 -- 172.21.15.42:0/4169732229 >> 172.21.15.42:0/4169732229 conn(0x7f6fc80fee60 msgr2=0x7f6fc81078a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:57:12.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:12.702+0000 7f6fcf067640 1 -- 172.21.15.42:0/4169732229 shutdown_connections 2024-08-30T08:57:12.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:12.702+0000 7f6fcf067640 1 -- 172.21.15.42:0/4169732229 wait complete. 2024-08-30T08:57:12.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:12 smithi167 ceph-mon[22063]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-08-30T08:57:12.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:12 smithi167 ceph-mon[22063]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]': finished 2024-08-30T08:57:12.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:12 smithi167 ceph-mon[22063]: osdmap e66: 12 total, 11 up, 12 in 2024-08-30T08:57:12.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:12 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-08-30T08:57:12.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:12 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-08-30T08:57:12.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:12 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:12.828 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:12 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:12.828 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:12 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-08-30T08:57:12.828 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:12 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-08-30T08:57:12.828 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:12 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-08-30T08:57:12.828 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:12 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-08-30T08:57:12.828 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:12 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:12.828 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:12 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:57:12.828 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:12 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-30T08:57:12.828 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:12 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:13.418 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:13 smithi042 ceph-mon[18652]: purged_snaps scrub starts 2024-08-30T08:57:13.418 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:13 smithi042 ceph-mon[18652]: purged_snaps scrub ok 2024-08-30T08:57:13.418 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:13 smithi042 ceph-mon[18652]: Detected new or changed devices on smithi190 2024-08-30T08:57:13.418 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:13 smithi042 ceph-mon[18652]: Adjusting osd_memory_target on smithi190 to 4272M 2024-08-30T08:57:13.419 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:13 smithi042 ceph-mon[18652]: osd.11 [v2:172.21.15.190:6824/380064457,v1:172.21.15.190:6825/380064457] boot 2024-08-30T08:57:13.419 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:13 smithi042 ceph-mon[18652]: osdmap e67: 12 total, 12 up, 12 in 2024-08-30T08:57:13.419 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:13 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-08-30T08:57:13.419 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:13 smithi042 ceph-mon[18652]: from='client.? 172.21.15.42:0/4169732229' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-08-30T08:57:13.420 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":67,"num_osds":12,"num_up_osds":12,"osd_up_since":1725008232,"num_in_osds":12,"osd_in_since":1725008210,"num_remapped_pgs":0} 2024-08-30T08:57:13.420 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph osd dump --format=json 2024-08-30T08:57:13.675 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T08:57:13.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:13 smithi190 ceph-mon[22001]: purged_snaps scrub starts 2024-08-30T08:57:13.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:13 smithi190 ceph-mon[22001]: purged_snaps scrub ok 2024-08-30T08:57:13.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:13 smithi190 ceph-mon[22001]: Detected new or changed devices on smithi190 2024-08-30T08:57:13.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:13 smithi190 ceph-mon[22001]: Adjusting osd_memory_target on smithi190 to 4272M 2024-08-30T08:57:13.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:13 smithi190 ceph-mon[22001]: osd.11 [v2:172.21.15.190:6824/380064457,v1:172.21.15.190:6825/380064457] boot 2024-08-30T08:57:13.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:13 smithi190 ceph-mon[22001]: osdmap e67: 12 total, 12 up, 12 in 2024-08-30T08:57:13.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:13 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-08-30T08:57:13.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:13 smithi190 ceph-mon[22001]: from='client.? 172.21.15.42:0/4169732229' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-08-30T08:57:13.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:13 smithi167 ceph-mon[22063]: purged_snaps scrub starts 2024-08-30T08:57:13.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:13 smithi167 ceph-mon[22063]: purged_snaps scrub ok 2024-08-30T08:57:13.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:13 smithi167 ceph-mon[22063]: Detected new or changed devices on smithi190 2024-08-30T08:57:13.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:13 smithi167 ceph-mon[22063]: Adjusting osd_memory_target on smithi190 to 4272M 2024-08-30T08:57:13.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:13 smithi167 ceph-mon[22063]: osd.11 [v2:172.21.15.190:6824/380064457,v1:172.21.15.190:6825/380064457] boot 2024-08-30T08:57:13.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:13 smithi167 ceph-mon[22063]: osdmap e67: 12 total, 12 up, 12 in 2024-08-30T08:57:13.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:13 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-08-30T08:57:13.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:13 smithi167 ceph-mon[22063]: from='client.? 172.21.15.42:0/4169732229' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-08-30T08:57:14.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:14.144+0000 7f7ac5756640 1 -- 172.21.15.42:0/2382303472 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7ac0108ab0 msgr2=0x7f7ac010f2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:57:14.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:14.144+0000 7f7ac5756640 1 --2- 172.21.15.42:0/2382303472 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7ac0108ab0 0x7f7ac010f2e0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f7ab8009900 tx=0x7f7ab802a410 comp rx=0 tx=0).stop 2024-08-30T08:57:14.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:14.144+0000 7f7ac5756640 1 -- 172.21.15.42:0/2382303472 shutdown_connections 2024-08-30T08:57:14.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:14.144+0000 7f7ac5756640 1 --2- 172.21.15.42:0/2382303472 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7ac0108ab0 0x7f7ac010f2e0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:57:14.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:14.144+0000 7f7ac5756640 1 --2- 172.21.15.42:0/2382303472 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7ac0108160 0x7f7ac0108570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:57:14.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:14.144+0000 7f7ac5756640 1 --2- 172.21.15.42:0/2382303472 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7ac0102820 0x7f7ac0102c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:57:14.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:14.144+0000 7f7ac5756640 1 -- 172.21.15.42:0/2382303472 >> 172.21.15.42:0/2382303472 conn(0x7f7ac00794e0 msgr2=0x7f7ac00798f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:57:14.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:14.144+0000 7f7ac5756640 1 -- 172.21.15.42:0/2382303472 shutdown_connections 2024-08-30T08:57:14.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:14.144+0000 7f7ac5756640 1 -- 172.21.15.42:0/2382303472 wait complete. 2024-08-30T08:57:14.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:14.145+0000 7f7ac5756640 1 Processor -- start 2024-08-30T08:57:14.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:14.145+0000 7f7ac5756640 1 -- start start 2024-08-30T08:57:14.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:14.145+0000 7f7ac5756640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7ac0102820 0x7f7ac0199160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:57:14.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:14.145+0000 7f7ac5756640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7ac0108160 0x7f7ac0199700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:57:14.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:14.145+0000 7f7ac5756640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7ac0108ab0 0x7f7ac019e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:57:14.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:14.145+0000 7f7ac5756640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f7ac0112730 con 0x7f7ac0102820 2024-08-30T08:57:14.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:14.145+0000 7f7ac5756640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f7ac01125b0 con 0x7f7ac0108160 2024-08-30T08:57:14.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:14.145+0000 7f7ac5756640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f7ac01128b0 con 0x7f7ac0108ab0 2024-08-30T08:57:14.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:14.145+0000 7f7abeffd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7ac0102820 0x7f7ac0199160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:57:14.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:14.145+0000 7f7abeffd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7ac0102820 0x7f7ac0199160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36168/0 (socket says 172.21.15.42:36168) 2024-08-30T08:57:14.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:14.145+0000 7f7abeffd640 1 -- 172.21.15.42:0/4184225460 learned_addr learned my addr 172.21.15.42:0/4184225460 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T08:57:14.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:14.146+0000 7f7ab67fc640 1 --2- 172.21.15.42:0/4184225460 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7ac0108160 0x7f7ac0199700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:57:14.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:14.146+0000 7f7abf7fe640 1 --2- 172.21.15.42:0/4184225460 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7ac0108ab0 0x7f7ac019e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:57:14.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:14.146+0000 7f7abeffd640 1 -- 172.21.15.42:0/4184225460 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7ac0108ab0 msgr2=0x7f7ac019e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:57:14.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:14.146+0000 7f7abeffd640 1 --2- 172.21.15.42:0/4184225460 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7ac0108ab0 0x7f7ac019e9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:57:14.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:14.146+0000 7f7abeffd640 1 -- 172.21.15.42:0/4184225460 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7ac0108160 msgr2=0x7f7ac0199700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:57:14.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:14.146+0000 7f7abeffd640 1 --2- 172.21.15.42:0/4184225460 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7ac0108160 0x7f7ac0199700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:57:14.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:14.146+0000 7f7abeffd640 1 -- 172.21.15.42:0/4184225460 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ac019f0d0 con 0x7f7ac0102820 2024-08-30T08:57:14.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:14.146+0000 7f7ab67fc640 1 --2- 172.21.15.42:0/4184225460 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7ac0108160 0x7f7ac0199700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-30T08:57:14.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:14.146+0000 7f7abeffd640 1 --2- 172.21.15.42:0/4184225460 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7ac0102820 0x7f7ac0199160 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f7ab000d770 tx=0x7f7ab000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:57:14.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:14.146+0000 7f7abcff9640 1 -- 172.21.15.42:0/4184225460 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ab0017070 con 0x7f7ac0102820 2024-08-30T08:57:14.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:14.146+0000 7f7ac5756640 1 -- 172.21.15.42:0/4184225460 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ac019f3c0 con 0x7f7ac0102820 2024-08-30T08:57:14.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:14.146+0000 7f7abcff9640 1 -- 172.21.15.42:0/4184225460 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7ab00054d0 con 0x7f7ac0102820 2024-08-30T08:57:14.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:14.146+0000 7f7abcff9640 1 -- 172.21.15.42:0/4184225460 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ab0004c30 con 0x7f7ac0102820 2024-08-30T08:57:14.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:14.146+0000 7f7ac5756640 1 -- 172.21.15.42:0/4184225460 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ac01a5b20 con 0x7f7ac0102820 2024-08-30T08:57:14.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:14.148+0000 7f7ac5756640 1 -- 172.21.15.42:0/4184225460 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ac0103aa0 con 0x7f7ac0102820 2024-08-30T08:57:14.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:14.149+0000 7f7abcff9640 1 -- 172.21.15.42:0/4184225460 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) ==== 152577+0+0 (secure 0 0 0) 0x7f7ab0004dd0 con 0x7f7ac0102820 2024-08-30T08:57:14.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:14.150+0000 7f7abcff9640 1 --2- 172.21.15.42:0/4184225460 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f7aac0b4b30 0x7f7aac0b7040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:57:14.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:14.151+0000 7f7ab67fc640 1 --2- 172.21.15.42:0/4184225460 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f7aac0b4b30 0x7f7aac0b7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:57:14.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:14.151+0000 7f7abcff9640 1 -- 172.21.15.42:0/4184225460 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 7127+0+0 (secure 0 0 0) 0x7f7ab00e4b60 con 0x7f7ac0102820 2024-08-30T08:57:14.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:14.151+0000 7f7ab67fc640 1 --2- 172.21.15.42:0/4184225460 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f7aac0b4b30 0x7f7aac0b7040 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f7aa8009fd0 tx=0x7f7aa8009680 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:57:14.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:14.154+0000 7f7abcff9640 1 -- 172.21.15.42:0/4184225460 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f7ab00ac110 con 0x7f7ac0102820 2024-08-30T08:57:14.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:14.340+0000 7f7ac5756640 1 -- 172.21.15.42:0/4184225460 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7ac0102c30 con 0x7f7ac0102820 2024-08-30T08:57:14.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:14.341+0000 7f7abcff9640 1 -- 172.21.15.42:0/4184225460 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v68) ==== 74+0+18504 (secure 0 0 0) 0x7f7ab00aba10 con 0x7f7ac0102820 2024-08-30T08:57:14.344 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-08-30T08:57:14.344 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":68,"fsid":"b0895c1a-66ac-11ef-bcd4-c7b262605968","created":"2024-08-30T08:49:44.858265+0000","modified":"2024-08-30T08:57:13.378917+0000","last_up_change":"2024-08-30T08:57:12.367791+0000","last_in_change":"2024-08-30T08:56:50.274678+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-30T08:53:27.297022+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"029d62d7-a6c6-4905-8c76-6dffa9d57b92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":2617890537},{"type":"v1","addr":"172.21.15.42:6803","nonce":2617890537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":2617890537},{"type":"v1","addr":"172.21.15.42:6805","nonce":2617890537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2617890537},{"type":"v1","addr":"172.21.15.42:6809","nonce":2617890537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":2617890537},{"type":"v1","addr":"172.21.15.42:6807","nonce":2617890537}]},"public_addr":"172.21.15.42:6803/2617890537","cluster_addr":"172.21.15.42:6805/2617890537","heartbeat_back_addr":"172.21.15.42:6809/2617890537","heartbeat_front_addr":"172.21.15.42:6807/2617890537","state":["exists","up"]},{"osd":1,"uuid":"3a78384b-0207-41df-9384-22665007d5b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":2791960082},{"type":"v1","addr":"172.21.15.167:6801","nonce":2791960082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":2791960082},{"type":"v1","addr":"172.21.15.167:6803","nonce":2791960082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":2791960082},{"type":"v1","addr":"172.21.15.167:6807","nonce":2791960082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":2791960082},{"type":"v1","addr":"172.21.15.167:6805","nonce":2791960082}]},"public_addr":"172.21.15.167:6801/2791960082","cluster_addr":"172.21.15.167:6803/2791960082","heartbeat_back_addr":"172.21.15.167:6807/2791960082","heartbeat_front_addr":"172.21.15.167:6805/2791960082","state":["exists","up"]},{"osd":2,"uuid":"16001809-c27c-4f52-948a-9998261acb4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":1101326404},{"type":"v1","addr":"172.21.15.190:6801","nonce":1101326404}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":1101326404},{"type":"v1","addr":"172.21.15.190:6803","nonce":1101326404}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":1101326404},{"type":"v1","addr":"172.21.15.190:6807","nonce":1101326404}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":1101326404},{"type":"v1","addr":"172.21.15.190:6805","nonce":1101326404}]},"public_addr":"172.21.15.190:6801/1101326404","cluster_addr":"172.21.15.190:6803/1101326404","heartbeat_back_addr":"172.21.15.190:6807/1101326404","heartbeat_front_addr":"172.21.15.190:6805/1101326404","state":["exists","up"]},{"osd":3,"uuid":"87bb3e36-9e9e-4b19-9d8c-0cdf6e089d3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":1135599916},{"type":"v1","addr":"172.21.15.42:6811","nonce":1135599916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":1135599916},{"type":"v1","addr":"172.21.15.42:6813","nonce":1135599916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":1135599916},{"type":"v1","addr":"172.21.15.42:6817","nonce":1135599916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":1135599916},{"type":"v1","addr":"172.21.15.42:6815","nonce":1135599916}]},"public_addr":"172.21.15.42:6811/1135599916","cluster_addr":"172.21.15.42:6813/1135599916","heartbeat_back_addr":"172.21.15.42:6817/1135599916","heartbeat_front_addr":"172.21.15.42:6815/1135599916","state":["exists","up"]},{"osd":4,"uuid":"3ca1aae3-c972-4fb9-ac86-236279c30d64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":2927511419},{"type":"v1","addr":"172.21.15.167:6809","nonce":2927511419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":2927511419},{"type":"v1","addr":"172.21.15.167:6811","nonce":2927511419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":2927511419},{"type":"v1","addr":"172.21.15.167:6815","nonce":2927511419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":2927511419},{"type":"v1","addr":"172.21.15.167:6813","nonce":2927511419}]},"public_addr":"172.21.15.167:6809/2927511419","cluster_addr":"172.21.15.167:6811/2927511419","heartbeat_back_addr":"172.21.15.167:6815/2927511419","heartbeat_front_addr":"172.21.15.167:6813/2927511419","state":["exists","up"]},{"osd":5,"uuid":"5d07f54b-0d07-4efe-9bcf-390755ce0258","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":2557145333},{"type":"v1","addr":"172.21.15.190:6809","nonce":2557145333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":2557145333},{"type":"v1","addr":"172.21.15.190:6811","nonce":2557145333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":2557145333},{"type":"v1","addr":"172.21.15.190:6815","nonce":2557145333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":2557145333},{"type":"v1","addr":"172.21.15.190:6813","nonce":2557145333}]},"public_addr":"172.21.15.190:6809/2557145333","cluster_addr":"172.21.15.190:6811/2557145333","heartbeat_back_addr":"172.21.15.190:6815/2557145333","heartbeat_front_addr":"172.21.15.190:6813/2557145333","state":["exists","up"]},{"osd":6,"uuid":"96ad64af-44ff-4197-97d4-156f0cb88e81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":3213526100},{"type":"v1","addr":"172.21.15.42:6819","nonce":3213526100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":3213526100},{"type":"v1","addr":"172.21.15.42:6821","nonce":3213526100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":3213526100},{"type":"v1","addr":"172.21.15.42:6825","nonce":3213526100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":3213526100},{"type":"v1","addr":"172.21.15.42:6823","nonce":3213526100}]},"public_addr":"172.21.15.42:6819/3213526100","cluster_addr":"172.21.15.42:6821/3213526100","heartbeat_back_addr":"172.21.15.42:6825/3213526100","heartbeat_front_addr":"172.21.15.42:6823/3213526100","state":["exists","up"]},{"osd":7,"uuid":"337a2112-eaf7-4e6b-86be-4e40d6ddf835","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":2962799814},{"type":"v1","addr":"172.21.15.167:6817","nonce":2962799814}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":2962799814},{"type":"v1","addr":"172.21.15.167:6819","nonce":2962799814}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":2962799814},{"type":"v1","addr":"172.21.15.167:6823","nonce":2962799814}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":2962799814},{"type":"v1","addr":"172.21.15.167:6821","nonce":2962799814}]},"public_addr":"172.21.15.167:6817/2962799814","cluster_addr":"172.21.15.167:6819/2962799814","heartbeat_back_addr":"172.21.15.167:6823/2962799814","heartbeat_front_addr":"172.21.15.167:6821/2962799814","state":["exists","up"]},{"osd":8,"uuid":"219a7224-4331-464b-ac86-c51054d51d91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":3242198843},{"type":"v1","addr":"172.21.15.190:6817","nonce":3242198843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":3242198843},{"type":"v1","addr":"172.21.15.190:6819","nonce":3242198843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":3242198843},{"type":"v1","addr":"172.21.15.190:6823","nonce":3242198843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":3242198843},{"type":"v1","addr":"172.21.15.190:6821","nonce":3242198843}]},"public_addr":"172.21.15.190:6817/3242198843","cluster_addr":"172.21.15.190:6819/3242198843","heartbeat_back_addr":"172.21.15.190:6823/3242198843","heartbeat_front_addr":"172.21.15.190:6821/3242198843","state":["exists","up"]},{"osd":9,"uuid":"2f2f5177-a009-40d9-991c-76aec8a5cba6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":1123536325},{"type":"v1","addr":"172.21.15.42:6827","nonce":1123536325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":1123536325},{"type":"v1","addr":"172.21.15.42:6829","nonce":1123536325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":1123536325},{"type":"v1","addr":"172.21.15.42:6833","nonce":1123536325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":1123536325},{"type":"v1","addr":"172.21.15.42:6831","nonce":1123536325}]},"public_addr":"172.21.15.42:6827/1123536325","cluster_addr":"172.21.15.42:6829/1123536325","heartbeat_back_addr":"172.21.15.42:6833/1123536325","heartbeat_front_addr":"172.21.15.42:6831/1123536325","state":["exists","up"]},{"osd":10,"uuid":"96ef5058-2518-4862-a165-17666b33a3e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":3451198319},{"type":"v1","addr":"172.21.15.167:6825","nonce":3451198319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":3451198319},{"type":"v1","addr":"172.21.15.167:6827","nonce":3451198319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":3451198319},{"type":"v1","addr":"172.21.15.167:6831","nonce":3451198319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":3451198319},{"type":"v1","addr":"172.21.15.167:6829","nonce":3451198319}]},"public_addr":"172.21.15.167:6825/3451198319","cluster_addr":"172.21.15.167:6827/3451198319","heartbeat_back_addr":"172.21.15.167:6831/3451198319","heartbeat_front_addr":"172.21.15.167:6829/3451198319","state":["exists","up"]},{"osd":11,"uuid":"a4eb9bd3-3faa-4c81-ba6d-c2084b72c9ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":380064457},{"type":"v1","addr":"172.21.15.190:6825","nonce":380064457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":380064457},{"type":"v1","addr":"172.21.15.190:6827","nonce":380064457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":380064457},{"type":"v1","addr":"172.21.15.190:6831","nonce":380064457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":380064457},{"type":"v1","addr":"172.21.15.190:6829","nonce":380064457}]},"public_addr":"172.21.15.190:6825/380064457","cluster_addr":"172.21.15.190:6827/380064457","heartbeat_back_addr":"172.21.15.190:6831/380064457","heartbeat_front_addr":"172.21.15.190:6829/380064457","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-30T08:52:36.511448+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-30T08:53:00.027277+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-30T08:53:23.770245+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-30T08:53:49.322226+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-30T08:54:14.350234+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-30T08:54:39.491793+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-30T08:55:04.870596+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-30T08:55:30.314150+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-30T08:55:54.483843+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-30T08:56:20.586115+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-30T08:56:45.666783+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-30T08:57:10.718387+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.42:0/2691233260":"2024-08-31T08:50:44.821070+0000","172.21.15.42:0/1518053351":"2024-08-31T08:50:44.821070+0000","172.21.15.42:0/352001020":"2024-08-31T08:50:44.821070+0000","172.21.15.42:6801/1733768634":"2024-08-31T08:50:44.821070+0000","172.21.15.42:0/1000597888":"2024-08-31T08:50:18.561851+0000","172.21.15.42:6800/1733768634":"2024-08-31T08:50:44.821070+0000","172.21.15.42:6801/3538882081":"2024-08-31T08:50:18.561851+0000","172.21.15.42:6800/3538882081":"2024-08-31T08:50:18.561851+0000","172.21.15.42:0/145366508":"2024-08-31T08:50:18.561851+0000","172.21.15.42:0/3346355213":"2024-08-31T08:50:18.561851+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-30T08:57:14.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:14.343+0000 7f7ac5756640 1 -- 172.21.15.42:0/4184225460 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f7aac0b4b30 msgr2=0x7f7aac0b7040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:57:14.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:14.343+0000 7f7ac5756640 1 --2- 172.21.15.42:0/4184225460 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f7aac0b4b30 0x7f7aac0b7040 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f7aa8009fd0 tx=0x7f7aa8009680 comp rx=0 tx=0).stop 2024-08-30T08:57:14.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:14.343+0000 7f7ac5756640 1 -- 172.21.15.42:0/4184225460 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7ac0102820 msgr2=0x7f7ac0199160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:57:14.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:14.343+0000 7f7ac5756640 1 --2- 172.21.15.42:0/4184225460 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7ac0102820 0x7f7ac0199160 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f7ab000d770 tx=0x7f7ab000dc30 comp rx=0 tx=0).stop 2024-08-30T08:57:14.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:14.343+0000 7f7ac5756640 1 -- 172.21.15.42:0/4184225460 shutdown_connections 2024-08-30T08:57:14.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:14.343+0000 7f7ac5756640 1 --2- 172.21.15.42:0/4184225460 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f7aac0b4b30 0x7f7aac0b7040 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:57:14.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:14.343+0000 7f7ac5756640 1 --2- 172.21.15.42:0/4184225460 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7ac0108ab0 0x7f7ac019e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:57:14.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:14.343+0000 7f7ac5756640 1 --2- 172.21.15.42:0/4184225460 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7ac0108160 0x7f7ac0199700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:57:14.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:14.343+0000 7f7ac5756640 1 --2- 172.21.15.42:0/4184225460 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7ac0102820 0x7f7ac0199160 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:57:14.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:14.343+0000 7f7ac5756640 1 -- 172.21.15.42:0/4184225460 >> 172.21.15.42:0/4184225460 conn(0x7f7ac00794e0 msgr2=0x7f7ac01041d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:57:14.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:14.344+0000 7f7ac5756640 1 -- 172.21.15.42:0/4184225460 shutdown_connections 2024-08-30T08:57:14.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:14.344+0000 7f7ac5756640 1 -- 172.21.15.42:0/4184225460 wait complete. 2024-08-30T08:57:14.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:14 smithi042 ceph-mon[18652]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-30T08:57:14.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:14 smithi042 ceph-mon[18652]: osdmap e68: 12 total, 12 up, 12 in 2024-08-30T08:57:14.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:14 smithi042 ceph-mon[18652]: from='client.? 172.21.15.42:0/4184225460' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-30T08:57:14.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:14 smithi190 ceph-mon[22001]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-30T08:57:14.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:14 smithi190 ceph-mon[22001]: osdmap e68: 12 total, 12 up, 12 in 2024-08-30T08:57:14.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:14 smithi190 ceph-mon[22001]: from='client.? 172.21.15.42:0/4184225460' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-30T08:57:14.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:14 smithi167 ceph-mon[22063]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-30T08:57:14.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:14 smithi167 ceph-mon[22063]: osdmap e68: 12 total, 12 up, 12 in 2024-08-30T08:57:14.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:14 smithi167 ceph-mon[22063]: from='client.? 172.21.15.42:0/4184225460' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-30T08:57:15.121 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-08-30T08:53:27.297022+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-08-30T08:57:15.121 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-08-30T08:57:15.375 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T08:57:15.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:15.816+0000 7fa1e7a69640 1 -- 172.21.15.42:0/3516411334 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa1e01081a0 msgr2=0x7fa1e01085b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:57:15.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:15.816+0000 7fa1e7a69640 1 --2- 172.21.15.42:0/3516411334 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa1e01081a0 0x7fa1e01085b0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fa1d0009900 tx=0x7fa1d002a410 comp rx=0 tx=0).stop 2024-08-30T08:57:15.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:15.816+0000 7fa1e7a69640 1 -- 172.21.15.42:0/3516411334 shutdown_connections 2024-08-30T08:57:15.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:15.816+0000 7fa1e7a69640 1 --2- 172.21.15.42:0/3516411334 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa1e0108af0 0x7fa1e010f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:57:15.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:15.816+0000 7fa1e7a69640 1 --2- 172.21.15.42:0/3516411334 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa1e01081a0 0x7fa1e01085b0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:57:15.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:15.816+0000 7fa1e7a69640 1 --2- 172.21.15.42:0/3516411334 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa1e0102f30 0x7fa1e0103340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:57:15.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:15.816+0000 7fa1e7a69640 1 -- 172.21.15.42:0/3516411334 >> 172.21.15.42:0/3516411334 conn(0x7fa1e0079d30 msgr2=0x7fa1e007a140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:57:15.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:15.816+0000 7fa1e7a69640 1 -- 172.21.15.42:0/3516411334 shutdown_connections 2024-08-30T08:57:15.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:15.816+0000 7fa1e7a69640 1 -- 172.21.15.42:0/3516411334 wait complete. 2024-08-30T08:57:15.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:15.816+0000 7fa1e7a69640 1 Processor -- start 2024-08-30T08:57:15.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:15.816+0000 7fa1e7a69640 1 -- start start 2024-08-30T08:57:15.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:15.817+0000 7fa1e7a69640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa1e0102f30 0x7fa1e019c570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:57:15.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:15.817+0000 7fa1e7a69640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa1e01081a0 0x7fa1e019cb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:57:15.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:15.817+0000 7fa1e7a69640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa1e0108af0 0x7fa1e01a1db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:57:15.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:15.817+0000 7fa1e7a69640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fa1e0112780 con 0x7fa1e01081a0 2024-08-30T08:57:15.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:15.817+0000 7fa1e7a69640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa1e0112600 con 0x7fa1e0102f30 2024-08-30T08:57:15.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:15.817+0000 7fa1e7a69640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fa1e0112900 con 0x7fa1e0108af0 2024-08-30T08:57:15.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:15.817+0000 7fa1e5004640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa1e01081a0 0x7fa1e019cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:57:15.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:15.817+0000 7fa1e5004640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa1e01081a0 0x7fa1e019cb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36192/0 (socket says 172.21.15.42:36192) 2024-08-30T08:57:15.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:15.817+0000 7fa1e5004640 1 -- 172.21.15.42:0/995798188 learned_addr learned my addr 172.21.15.42:0/995798188 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T08:57:15.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:15.817+0000 7fa1e6006640 1 --2- 172.21.15.42:0/995798188 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa1e0108af0 0x7fa1e01a1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:57:15.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:15.817+0000 7fa1e5805640 1 --2- 172.21.15.42:0/995798188 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa1e0102f30 0x7fa1e019c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:57:15.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:15.817+0000 7fa1e5004640 1 -- 172.21.15.42:0/995798188 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa1e0108af0 msgr2=0x7fa1e01a1db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:57:15.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:15.817+0000 7fa1e5004640 1 --2- 172.21.15.42:0/995798188 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa1e0108af0 0x7fa1e01a1db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:57:15.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:15.817+0000 7fa1e5004640 1 -- 172.21.15.42:0/995798188 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa1e0102f30 msgr2=0x7fa1e019c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:57:15.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:15.817+0000 7fa1e5004640 1 --2- 172.21.15.42:0/995798188 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa1e0102f30 0x7fa1e019c570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:57:15.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:15.817+0000 7fa1e5004640 1 -- 172.21.15.42:0/995798188 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1e01a24e0 con 0x7fa1e01081a0 2024-08-30T08:57:15.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:15.817+0000 7fa1e5004640 1 --2- 172.21.15.42:0/995798188 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa1e01081a0 0x7fa1e019cb10 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fa1d0006010 tx=0x7fa1d0004060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:57:15.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:15.818+0000 7fa1d6ffd640 1 -- 172.21.15.42:0/995798188 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1d003f030 con 0x7fa1e01081a0 2024-08-30T08:57:15.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:15.818+0000 7fa1d6ffd640 1 -- 172.21.15.42:0/995798188 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa1d0003aa0 con 0x7fa1e01081a0 2024-08-30T08:57:15.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:15.818+0000 7fa1e7a69640 1 -- 172.21.15.42:0/995798188 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1e01a2770 con 0x7fa1e01081a0 2024-08-30T08:57:15.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:15.818+0000 7fa1d6ffd640 1 -- 172.21.15.42:0/995798188 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1d0035070 con 0x7fa1e01081a0 2024-08-30T08:57:15.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:15.818+0000 7fa1e7a69640 1 -- 172.21.15.42:0/995798188 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1e01a8f30 con 0x7fa1e01081a0 2024-08-30T08:57:15.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:15.819+0000 7fa1e7a69640 1 -- 172.21.15.42:0/995798188 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1e0103cb0 con 0x7fa1e01081a0 2024-08-30T08:57:15.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:15.820+0000 7fa1d6ffd640 1 -- 172.21.15.42:0/995798188 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) ==== 152577+0+0 (secure 0 0 0) 0x7fa1d0042030 con 0x7fa1e01081a0 2024-08-30T08:57:15.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:15.821+0000 7fa1d6ffd640 1 --2- 172.21.15.42:0/995798188 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fa1cc0b4b30 0x7fa1cc0b7040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:57:15.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:15.821+0000 7fa1e5805640 1 --2- 172.21.15.42:0/995798188 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fa1cc0b4b30 0x7fa1cc0b7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:57:15.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:15.821+0000 7fa1d6ffd640 1 -- 172.21.15.42:0/995798188 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 7127+0+0 (secure 0 0 0) 0x7fa1d0105190 con 0x7fa1e01081a0 2024-08-30T08:57:15.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:15.822+0000 7fa1e5805640 1 --2- 172.21.15.42:0/995798188 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fa1cc0b4b30 0x7fa1cc0b7040 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fa1dc005fd0 tx=0x7fa1dc00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:57:15.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:15.826+0000 7fa1d6ffd640 1 -- 172.21.15.42:0/995798188 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7fa1d00cc070 con 0x7fa1e01081a0 2024-08-30T08:57:16.012 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:16.009+0000 7fa1e7a69640 1 -- 172.21.15.42:0/995798188 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa1e0101190 con 0x7fa1e01081a0 2024-08-30T08:57:16.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:16.010+0000 7fa1d6ffd640 1 -- 172.21.15.42:0/995798188 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v68) ==== 93+0+10 (secure 0 0 0) 0x7fa1d00d0830 con 0x7fa1e01081a0 2024-08-30T08:57:16.013 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 1 2024-08-30T08:57:16.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:16.012+0000 7fa1e7a69640 1 -- 172.21.15.42:0/995798188 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fa1cc0b4b30 msgr2=0x7fa1cc0b7040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:57:16.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:16.012+0000 7fa1e7a69640 1 --2- 172.21.15.42:0/995798188 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fa1cc0b4b30 0x7fa1cc0b7040 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fa1dc005fd0 tx=0x7fa1dc00b390 comp rx=0 tx=0).stop 2024-08-30T08:57:16.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:16.012+0000 7fa1e7a69640 1 -- 172.21.15.42:0/995798188 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa1e01081a0 msgr2=0x7fa1e019cb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:57:16.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:16.012+0000 7fa1e7a69640 1 --2- 172.21.15.42:0/995798188 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa1e01081a0 0x7fa1e019cb10 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fa1d0006010 tx=0x7fa1d0004060 comp rx=0 tx=0).stop 2024-08-30T08:57:16.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:16.012+0000 7fa1e7a69640 1 -- 172.21.15.42:0/995798188 shutdown_connections 2024-08-30T08:57:16.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:16.012+0000 7fa1e7a69640 1 --2- 172.21.15.42:0/995798188 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fa1cc0b4b30 0x7fa1cc0b7040 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:57:16.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:16.012+0000 7fa1e7a69640 1 --2- 172.21.15.42:0/995798188 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa1e0108af0 0x7fa1e01a1db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:57:16.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:16.012+0000 7fa1e7a69640 1 --2- 172.21.15.42:0/995798188 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa1e01081a0 0x7fa1e019cb10 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:57:16.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:16.012+0000 7fa1e7a69640 1 --2- 172.21.15.42:0/995798188 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa1e0102f30 0x7fa1e019c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:57:16.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:16.012+0000 7fa1e7a69640 1 -- 172.21.15.42:0/995798188 >> 172.21.15.42:0/995798188 conn(0x7fa1e0079d30 msgr2=0x7fa1e01061b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:57:16.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:16.012+0000 7fa1e7a69640 1 -- 172.21.15.42:0/995798188 shutdown_connections 2024-08-30T08:57:16.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:16.012+0000 7fa1e7a69640 1 -- 172.21.15.42:0/995798188 wait complete. 2024-08-30T08:57:16.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:16 smithi167 ceph-mon[22063]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-30T08:57:16.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:16 smithi167 ceph-mon[22063]: from='client.? 172.21.15.42:0/995798188' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-08-30T08:57:16.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:16 smithi042 ceph-mon[18652]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-30T08:57:16.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:16 smithi042 ceph-mon[18652]: from='client.? 172.21.15.42:0/995798188' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-08-30T08:57:16.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:16 smithi190 ceph-mon[22001]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-30T08:57:16.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:16 smithi190 ceph-mon[22001]: from='client.? 172.21.15.42:0/995798188' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-08-30T08:57:16.721 INFO:tasks.cephadm:Adding mds.a on smithi042 2024-08-30T08:57:16.721 INFO:tasks.cephadm:Adding mds.d on smithi042 2024-08-30T08:57:16.721 INFO:tasks.cephadm:Adding mds.g on smithi042 2024-08-30T08:57:16.721 INFO:tasks.cephadm:Adding mds.j on smithi042 2024-08-30T08:57:16.721 INFO:tasks.cephadm:Adding mds.b on smithi167 2024-08-30T08:57:16.721 INFO:tasks.cephadm:Adding mds.e on smithi167 2024-08-30T08:57:16.721 INFO:tasks.cephadm:Adding mds.h on smithi167 2024-08-30T08:57:16.721 INFO:tasks.cephadm:Adding mds.k on smithi167 2024-08-30T08:57:16.721 INFO:tasks.cephadm:Adding mds.c on smithi190 2024-08-30T08:57:16.721 INFO:tasks.cephadm:Adding mds.f on smithi190 2024-08-30T08:57:16.722 INFO:tasks.cephadm:Adding mds.i on smithi190 2024-08-30T08:57:16.722 INFO:tasks.cephadm:Adding mds.l on smithi190 2024-08-30T08:57:16.722 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph orch apply mds all '12;smithi042=a;smithi042=d;smithi042=g;smithi042=j;smithi167=b;smithi167=e;smithi167=h;smithi167=k;smithi190=c;smithi190=f;smithi190=i;smithi190=l' 2024-08-30T08:57:16.980 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.c/config 2024-08-30T08:57:17.527 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:57:17.506+0000 7f0a6cc70640 1 -- 172.21.15.190:0/792494569 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0a68106750 msgr2=0x7f0a68108b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:57:17.528 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:57:17.506+0000 7f0a6cc70640 1 --2- 172.21.15.190:0/792494569 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0a68106750 0x7f0a68108b70 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f0a5c009900 tx=0x7f0a5c02a410 comp rx=0 tx=0).stop 2024-08-30T08:57:17.528 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:57:17.506+0000 7f0a6cc70640 1 -- 172.21.15.190:0/792494569 shutdown_connections 2024-08-30T08:57:17.528 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:57:17.506+0000 7f0a6cc70640 1 --2- 172.21.15.190:0/792494569 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0a68106750 0x7f0a68108b70 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:57:17.528 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:57:17.506+0000 7f0a6cc70640 1 --2- 172.21.15.190:0/792494569 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0a68109230 0x7f0a6810b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:57:17.528 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:57:17.506+0000 7f0a6cc70640 1 --2- 172.21.15.190:0/792494569 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0a6806c380 0x7f0a68106210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:57:17.528 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:57:17.506+0000 7f0a6cc70640 1 -- 172.21.15.190:0/792494569 >> 172.21.15.190:0/792494569 conn(0x7f0a680fdd70 msgr2=0x7f0a68100220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:57:17.528 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:57:17.506+0000 7f0a6cc70640 1 -- 172.21.15.190:0/792494569 shutdown_connections 2024-08-30T08:57:17.529 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:57:17.506+0000 7f0a6cc70640 1 -- 172.21.15.190:0/792494569 wait complete. 2024-08-30T08:57:17.529 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:57:17.507+0000 7f0a6cc70640 1 Processor -- start 2024-08-30T08:57:17.529 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:57:17.507+0000 7f0a6cc70640 1 -- start start 2024-08-30T08:57:17.529 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:57:17.507+0000 7f0a6cc70640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0a6806c380 0x7f0a68199dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:57:17.529 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:57:17.507+0000 7f0a6cc70640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0a68106750 0x7f0a6819a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:57:17.529 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:57:17.507+0000 7f0a6cc70640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0a68109230 0x7f0a6819afd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:57:17.529 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:57:17.507+0000 7f0a6cc70640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f0a6810f110 con 0x7f0a68109230 2024-08-30T08:57:17.529 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:57:17.507+0000 7f0a6cc70640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f0a6810ef90 con 0x7f0a68106750 2024-08-30T08:57:17.529 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:57:17.507+0000 7f0a6cc70640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f0a6810f290 con 0x7f0a6806c380 2024-08-30T08:57:17.529 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:57:17.507+0000 7f0a6659c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0a6806c380 0x7f0a68199dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:57:17.529 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:57:17.507+0000 7f0a6659c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0a6806c380 0x7f0a68199dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:39480/0 (socket says 172.21.15.190:39480) 2024-08-30T08:57:17.529 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:57:17.507+0000 7f0a6659c640 1 -- 172.21.15.190:0/696192314 learned_addr learned my addr 172.21.15.190:0/696192314 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-30T08:57:17.530 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:57:17.508+0000 7f0a65d9b640 1 --2- 172.21.15.190:0/696192314 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0a68106750 0x7f0a6819a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:57:17.530 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:57:17.508+0000 7f0a66d9d640 1 --2- 172.21.15.190:0/696192314 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0a68109230 0x7f0a6819afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:57:17.530 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:57:17.508+0000 7f0a6659c640 1 -- 172.21.15.190:0/696192314 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0a68106750 msgr2=0x7f0a6819a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:57:17.530 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:57:17.508+0000 7f0a6659c640 1 --2- 172.21.15.190:0/696192314 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0a68106750 0x7f0a6819a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:57:17.530 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:57:17.508+0000 7f0a6659c640 1 -- 172.21.15.190:0/696192314 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0a68109230 msgr2=0x7f0a6819afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:57:17.530 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:57:17.508+0000 7f0a6659c640 1 --2- 172.21.15.190:0/696192314 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0a68109230 0x7f0a6819afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:57:17.530 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:57:17.508+0000 7f0a6659c640 1 -- 172.21.15.190:0/696192314 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a6819b760 con 0x7f0a6806c380 2024-08-30T08:57:17.530 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:57:17.508+0000 7f0a65d9b640 1 --2- 172.21.15.190:0/696192314 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0a68106750 0x7f0a6819a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-30T08:57:17.530 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:57:17.508+0000 7f0a66d9d640 1 --2- 172.21.15.190:0/696192314 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0a68109230 0x7f0a6819afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-30T08:57:17.530 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:57:17.508+0000 7f0a6659c640 1 --2- 172.21.15.190:0/696192314 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0a6806c380 0x7f0a68199dc0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f0a5800c920 tx=0x7f0a5800cde0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:57:17.530 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:57:17.508+0000 7f0a577fe640 1 -- 172.21.15.190:0/696192314 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a58007c70 con 0x7f0a6806c380 2024-08-30T08:57:17.530 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:57:17.508+0000 7f0a6cc70640 1 -- 172.21.15.190:0/696192314 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a6819ba50 con 0x7f0a6806c380 2024-08-30T08:57:17.530 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:57:17.508+0000 7f0a577fe640 1 -- 172.21.15.190:0/696192314 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0a58007e10 con 0x7f0a6806c380 2024-08-30T08:57:17.531 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:57:17.508+0000 7f0a577fe640 1 -- 172.21.15.190:0/696192314 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a58018820 con 0x7f0a6806c380 2024-08-30T08:57:17.531 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:57:17.508+0000 7f0a6cc70640 1 -- 172.21.15.190:0/696192314 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a681a21e0 con 0x7f0a6806c380 2024-08-30T08:57:17.531 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:57:17.509+0000 7f0a6cc70640 1 -- 172.21.15.190:0/696192314 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a681a23b0 con 0x7f0a6806c380 2024-08-30T08:57:17.531 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:57:17.511+0000 7f0a577fe640 1 -- 172.21.15.190:0/696192314 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 16) ==== 152577+0+0 (secure 0 0 0) 0x7f0a5800f450 con 0x7f0a6806c380 2024-08-30T08:57:17.531 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:57:17.511+0000 7f0a577fe640 1 --2- 172.21.15.190:0/696192314 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f0a500b4b30 0x7f0a500b7040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:57:17.531 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:57:17.512+0000 7f0a577fe640 1 -- 172.21.15.190:0/696192314 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 7127+0+0 (secure 0 0 0) 0x7f0a580e5160 con 0x7f0a6806c380 2024-08-30T08:57:17.531 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:57:17.512+0000 7f0a65d9b640 1 --2- 172.21.15.190:0/696192314 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f0a500b4b30 0x7f0a500b7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:57:17.531 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:57:17.512+0000 7f0a65d9b640 1 --2- 172.21.15.190:0/696192314 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f0a500b4b30 0x7f0a500b7040 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f0a5c031440 tx=0x7f0a5c005d50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:57:17.531 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:57:17.515+0000 7f0a577fe640 1 -- 172.21.15.190:0/696192314 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f0a58014030 con 0x7f0a6806c380 2024-08-30T08:57:17.702 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:57:17.700+0000 7f0a6cc70640 1 -- 172.21.15.190:0/696192314 --> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi042=a;smithi042=d;smithi042=g;smithi042=j;smithi167=b;smithi167=e;smithi167=h;smithi167=k;smithi190=c;smithi190=f;smithi190=i;smithi190=l", "target": ["mon-mgr", ""]}) -- 0x7f0a68105e70 con 0x7f0a500b4b30 2024-08-30T08:57:17.711 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:57:17.709+0000 7f0a577fe640 1 -- 172.21.15.190:0/696192314 <== mgr.14150 v2:172.21.15.42:6800/3005140902 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7f0a68105e70 con 0x7f0a500b4b30 2024-08-30T08:57:17.711 INFO:teuthology.orchestra.run.smithi190.stdout:Scheduled mds.all update... 2024-08-30T08:57:17.713 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:57:17.711+0000 7f0a6cc70640 1 -- 172.21.15.190:0/696192314 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f0a500b4b30 msgr2=0x7f0a500b7040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:57:17.714 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:57:17.711+0000 7f0a6cc70640 1 --2- 172.21.15.190:0/696192314 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f0a500b4b30 0x7f0a500b7040 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f0a5c031440 tx=0x7f0a5c005d50 comp rx=0 tx=0).stop 2024-08-30T08:57:17.714 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:57:17.711+0000 7f0a6cc70640 1 -- 172.21.15.190:0/696192314 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0a6806c380 msgr2=0x7f0a68199dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:57:17.714 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:57:17.711+0000 7f0a6cc70640 1 --2- 172.21.15.190:0/696192314 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0a6806c380 0x7f0a68199dc0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f0a5800c920 tx=0x7f0a5800cde0 comp rx=0 tx=0).stop 2024-08-30T08:57:17.714 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:57:17.712+0000 7f0a6cc70640 1 -- 172.21.15.190:0/696192314 shutdown_connections 2024-08-30T08:57:17.714 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:57:17.712+0000 7f0a6cc70640 1 --2- 172.21.15.190:0/696192314 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f0a500b4b30 0x7f0a500b7040 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:57:17.714 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:57:17.712+0000 7f0a6cc70640 1 --2- 172.21.15.190:0/696192314 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0a68109230 0x7f0a6819afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:57:17.714 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:57:17.712+0000 7f0a6cc70640 1 --2- 172.21.15.190:0/696192314 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0a68106750 0x7f0a6819a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:57:17.714 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:57:17.712+0000 7f0a6cc70640 1 --2- 172.21.15.190:0/696192314 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0a6806c380 0x7f0a68199dc0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:57:17.714 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:57:17.712+0000 7f0a6cc70640 1 -- 172.21.15.190:0/696192314 >> 172.21.15.190:0/696192314 conn(0x7f0a680fdd70 msgr2=0x7f0a6810b650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:57:17.714 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:57:17.712+0000 7f0a6cc70640 1 -- 172.21.15.190:0/696192314 shutdown_connections 2024-08-30T08:57:17.714 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-30T08:57:17.712+0000 7f0a6cc70640 1 -- 172.21.15.190:0/696192314 wait complete. 2024-08-30T08:57:18.365 DEBUG:teuthology.orchestra.run.smithi042:mds.a> sudo journalctl -f -n 0 -u ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968@mds.a.service 2024-08-30T08:57:18.369 DEBUG:teuthology.orchestra.run.smithi042:mds.d> sudo journalctl -f -n 0 -u ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968@mds.d.service 2024-08-30T08:57:18.412 DEBUG:teuthology.orchestra.run.smithi042:mds.g> sudo journalctl -f -n 0 -u ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968@mds.g.service 2024-08-30T08:57:18.454 DEBUG:teuthology.orchestra.run.smithi042:mds.j> sudo journalctl -f -n 0 -u ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968@mds.j.service 2024-08-30T08:57:18.498 DEBUG:teuthology.orchestra.run.smithi167:mds.b> sudo journalctl -f -n 0 -u ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968@mds.b.service 2024-08-30T08:57:18.501 DEBUG:teuthology.orchestra.run.smithi167:mds.e> sudo journalctl -f -n 0 -u ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968@mds.e.service 2024-08-30T08:57:18.544 DEBUG:teuthology.orchestra.run.smithi167:mds.h> sudo journalctl -f -n 0 -u ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968@mds.h.service 2024-08-30T08:57:18.587 DEBUG:teuthology.orchestra.run.smithi167:mds.k> sudo journalctl -f -n 0 -u ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968@mds.k.service 2024-08-30T08:57:18.630 DEBUG:teuthology.orchestra.run.smithi190:mds.c> sudo journalctl -f -n 0 -u ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968@mds.c.service 2024-08-30T08:57:18.633 DEBUG:teuthology.orchestra.run.smithi190:mds.f> sudo journalctl -f -n 0 -u ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968@mds.f.service 2024-08-30T08:57:18.676 DEBUG:teuthology.orchestra.run.smithi190:mds.i> sudo journalctl -f -n 0 -u ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968@mds.i.service 2024-08-30T08:57:18.718 DEBUG:teuthology.orchestra.run.smithi190:mds.l> sudo journalctl -f -n 0 -u ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968@mds.l.service 2024-08-30T08:57:18.761 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-08-30T08:57:18.762 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph fs dump --format=json 2024-08-30T08:57:18.802 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:18 smithi190 ceph-mon[22001]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-30T08:57:18.803 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:18 smithi190 ceph-mon[22001]: from='client.24544 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi042=a;smithi042=d;smithi042=g;smithi042=j;smithi167=b;smithi167=e;smithi167=h;smithi167=k;smithi190=c;smithi190=f;smithi190=i;smithi190=l", "target": ["mon-mgr", ""]}]: dispatch 2024-08-30T08:57:18.803 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:18 smithi190 ceph-mon[22001]: Saving service mds.all spec with placement smithi042=a;smithi042=d;smithi042=g;smithi042=j;smithi167=b;smithi167=e;smithi167=h;smithi167=k;smithi190=c;smithi190=f;smithi190=i;smithi190=l;count:12 2024-08-30T08:57:18.803 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:18 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:18.803 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:18 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-30T08:57:18.803 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:18 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:57:18.803 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:18 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-30T08:57:18.803 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:18 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:18.803 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:18 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-30T08:57:18.803 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:18 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-30T08:57:18.803 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:18 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:57:18.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:18 smithi167 ceph-mon[22063]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-30T08:57:18.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:18 smithi167 ceph-mon[22063]: from='client.24544 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi042=a;smithi042=d;smithi042=g;smithi042=j;smithi167=b;smithi167=e;smithi167=h;smithi167=k;smithi190=c;smithi190=f;smithi190=i;smithi190=l", "target": ["mon-mgr", ""]}]: dispatch 2024-08-30T08:57:18.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:18 smithi167 ceph-mon[22063]: Saving service mds.all spec with placement smithi042=a;smithi042=d;smithi042=g;smithi042=j;smithi167=b;smithi167=e;smithi167=h;smithi167=k;smithi190=c;smithi190=f;smithi190=i;smithi190=l;count:12 2024-08-30T08:57:18.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:18 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:18.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:18 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-30T08:57:18.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:18 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:57:18.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:18 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-30T08:57:18.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:18 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:18.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:18 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-30T08:57:18.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:18 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-30T08:57:18.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:18 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:57:19.013 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T08:57:19.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:18 smithi042 ceph-mon[18652]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-30T08:57:19.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:18 smithi042 ceph-mon[18652]: from='client.24544 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi042=a;smithi042=d;smithi042=g;smithi042=j;smithi167=b;smithi167=e;smithi167=h;smithi167=k;smithi190=c;smithi190=f;smithi190=i;smithi190=l", "target": ["mon-mgr", ""]}]: dispatch 2024-08-30T08:57:19.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:18 smithi042 ceph-mon[18652]: Saving service mds.all spec with placement smithi042=a;smithi042=d;smithi042=g;smithi042=j;smithi167=b;smithi167=e;smithi167=h;smithi167=k;smithi190=c;smithi190=f;smithi190=i;smithi190=l;count:12 2024-08-30T08:57:19.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:18 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:19.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:18 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-30T08:57:19.134 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:18 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:57:19.134 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:18 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-30T08:57:19.134 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:18 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:19.134 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:18 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-30T08:57:19.134 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:18 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-30T08:57:19.134 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:18 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:57:19.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:19.471+0000 7fb6f6063640 1 -- 172.21.15.42:0/3833406813 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb6f0102800 msgr2=0x7fb6f0102c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:57:19.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:19.471+0000 7fb6f6063640 1 --2- 172.21.15.42:0/3833406813 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb6f0102800 0x7fb6f0102c10 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fb6e0009900 tx=0x7fb6e002a410 comp rx=0 tx=0).stop 2024-08-30T08:57:19.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:19.472+0000 7fb6f6063640 1 -- 172.21.15.42:0/3833406813 shutdown_connections 2024-08-30T08:57:19.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:19.472+0000 7fb6f6063640 1 --2- 172.21.15.42:0/3833406813 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb6f0108140 0x7fb6f0108550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:57:19.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:19.472+0000 7fb6f6063640 1 --2- 172.21.15.42:0/3833406813 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb6f0108a90 0x7fb6f010f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:57:19.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:19.472+0000 7fb6f6063640 1 --2- 172.21.15.42:0/3833406813 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb6f0102800 0x7fb6f0102c10 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:57:19.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:19.472+0000 7fb6f6063640 1 -- 172.21.15.42:0/3833406813 >> 172.21.15.42:0/3833406813 conn(0x7fb6f00794d0 msgr2=0x7fb6f00798e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:57:19.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:19.472+0000 7fb6f6063640 1 -- 172.21.15.42:0/3833406813 shutdown_connections 2024-08-30T08:57:19.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:19.472+0000 7fb6f6063640 1 -- 172.21.15.42:0/3833406813 wait complete. 2024-08-30T08:57:19.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:19.472+0000 7fb6f6063640 1 Processor -- start 2024-08-30T08:57:19.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:19.472+0000 7fb6f6063640 1 -- start start 2024-08-30T08:57:19.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:19.473+0000 7fb6f6063640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb6f0102800 0x7fb6f019a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:57:19.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:19.473+0000 7fb6f6063640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb6f0108140 0x7fb6f019a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:57:19.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:19.473+0000 7fb6f6063640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb6f0108a90 0x7fb6f019fa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:57:19.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:19.473+0000 7fb6f6063640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fb6f0112710 con 0x7fb6f0102800 2024-08-30T08:57:19.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:19.473+0000 7fb6f6063640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb6f0112590 con 0x7fb6f0108140 2024-08-30T08:57:19.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:19.473+0000 7fb6f6063640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fb6f0112890 con 0x7fb6f0108a90 2024-08-30T08:57:19.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:19.473+0000 7fb6ef7fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb6f0102800 0x7fb6f019a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:57:19.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:19.473+0000 7fb6ef7fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb6f0102800 0x7fb6f019a230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:55884/0 (socket says 172.21.15.42:55884) 2024-08-30T08:57:19.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:19.473+0000 7fb6ef7fe640 1 -- 172.21.15.42:0/40972281 learned_addr learned my addr 172.21.15.42:0/40972281 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T08:57:19.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:19.473+0000 7fb6effff640 1 --2- 172.21.15.42:0/40972281 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb6f0108a90 0x7fb6f019fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:57:19.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:19.473+0000 7fb6eeffd640 1 --2- 172.21.15.42:0/40972281 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb6f0108140 0x7fb6f019a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:57:19.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:19.473+0000 7fb6ef7fe640 1 -- 172.21.15.42:0/40972281 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb6f0108a90 msgr2=0x7fb6f019fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:57:19.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:19.473+0000 7fb6ef7fe640 1 --2- 172.21.15.42:0/40972281 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb6f0108a90 0x7fb6f019fa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:57:19.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:19.473+0000 7fb6ef7fe640 1 -- 172.21.15.42:0/40972281 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb6f0108140 msgr2=0x7fb6f019a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:57:19.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:19.473+0000 7fb6ef7fe640 1 --2- 172.21.15.42:0/40972281 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb6f0108140 0x7fb6f019a7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:57:19.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:19.473+0000 7fb6ef7fe640 1 -- 172.21.15.42:0/40972281 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6f01a01a0 con 0x7fb6f0102800 2024-08-30T08:57:19.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:19.473+0000 7fb6ef7fe640 1 --2- 172.21.15.42:0/40972281 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb6f0102800 0x7fb6f019a230 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fb6e0006010 tx=0x7fb6e00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:57:19.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:19.474+0000 7fb6ecff9640 1 -- 172.21.15.42:0/40972281 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6e003f030 con 0x7fb6f0102800 2024-08-30T08:57:19.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:19.474+0000 7fb6f6063640 1 -- 172.21.15.42:0/40972281 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6f01a0430 con 0x7fb6f0102800 2024-08-30T08:57:19.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:19.474+0000 7fb6ecff9640 1 -- 172.21.15.42:0/40972281 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb6e0003800 con 0x7fb6f0102800 2024-08-30T08:57:19.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:19.474+0000 7fb6ecff9640 1 -- 172.21.15.42:0/40972281 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6e0035070 con 0x7fb6f0102800 2024-08-30T08:57:19.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:19.474+0000 7fb6f6063640 1 -- 172.21.15.42:0/40972281 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6f01a6bf0 con 0x7fb6f0102800 2024-08-30T08:57:19.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:19.475+0000 7fb6f6063640 1 -- 172.21.15.42:0/40972281 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6f0102c10 con 0x7fb6f0102800 2024-08-30T08:57:19.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:19.476+0000 7fb6ecff9640 1 -- 172.21.15.42:0/40972281 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) ==== 152577+0+0 (secure 0 0 0) 0x7fb6e0003cb0 con 0x7fb6f0102800 2024-08-30T08:57:19.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:19.477+0000 7fb6ecff9640 1 --2- 172.21.15.42:0/40972281 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fb6d80b4b30 0x7fb6d80b7040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:57:19.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:19.477+0000 7fb6eeffd640 1 --2- 172.21.15.42:0/40972281 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fb6d80b4b30 0x7fb6d80b7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:57:19.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:19.477+0000 7fb6ecff9640 1 -- 172.21.15.42:0/40972281 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 7127+0+0 (secure 0 0 0) 0x7fb6e0105820 con 0x7fb6f0102800 2024-08-30T08:57:19.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:19.478+0000 7fb6eeffd640 1 --2- 172.21.15.42:0/40972281 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fb6d80b4b30 0x7fb6d80b7040 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fb6f0100910 tx=0x7fb6e400a250 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:57:19.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:19.480+0000 7fb6ecff9640 1 -- 172.21.15.42:0/40972281 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7fb6e00cc700 con 0x7fb6f0102800 2024-08-30T08:57:19.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:19.700+0000 7fb6f6063640 1 -- 172.21.15.42:0/40972281 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb6f01a6e50 con 0x7fb6f0102800 2024-08-30T08:57:19.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:19.701+0000 7fb6ecff9640 1 -- 172.21.15.42:0/40972281 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+611 (secure 0 0 0) 0x7fb6e00c6e20 con 0x7fb6f0102800 2024-08-30T08:57:19.704 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-08-30T08:57:19.704 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":1,"btime":"2024-08-30T08:49:44:858072+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-08-30T08:57:19.704 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 1 2024-08-30T08:57:19.706 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:19.703+0000 7fb6f6063640 1 -- 172.21.15.42:0/40972281 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fb6d80b4b30 msgr2=0x7fb6d80b7040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:57:19.706 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:19.703+0000 7fb6f6063640 1 --2- 172.21.15.42:0/40972281 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fb6d80b4b30 0x7fb6d80b7040 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fb6f0100910 tx=0x7fb6e400a250 comp rx=0 tx=0).stop 2024-08-30T08:57:19.706 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:19.703+0000 7fb6f6063640 1 -- 172.21.15.42:0/40972281 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb6f0102800 msgr2=0x7fb6f019a230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:57:19.706 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:19.703+0000 7fb6f6063640 1 --2- 172.21.15.42:0/40972281 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb6f0102800 0x7fb6f019a230 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fb6e0006010 tx=0x7fb6e00042f0 comp rx=0 tx=0).stop 2024-08-30T08:57:19.706 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:19.703+0000 7fb6f6063640 1 -- 172.21.15.42:0/40972281 shutdown_connections 2024-08-30T08:57:19.706 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:19.703+0000 7fb6f6063640 1 --2- 172.21.15.42:0/40972281 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fb6d80b4b30 0x7fb6d80b7040 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:57:19.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:19.703+0000 7fb6f6063640 1 --2- 172.21.15.42:0/40972281 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb6f0108a90 0x7fb6f019fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:57:19.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:19.703+0000 7fb6f6063640 1 --2- 172.21.15.42:0/40972281 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb6f0108140 0x7fb6f019a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:57:19.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:19.703+0000 7fb6f6063640 1 --2- 172.21.15.42:0/40972281 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb6f0102800 0x7fb6f019a230 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:57:19.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:19.703+0000 7fb6f6063640 1 -- 172.21.15.42:0/40972281 >> 172.21.15.42:0/40972281 conn(0x7fb6f00794d0 msgr2=0x7fb6f0101870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:57:19.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:19.703+0000 7fb6f6063640 1 -- 172.21.15.42:0/40972281 shutdown_connections 2024-08-30T08:57:19.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:19.703+0000 7fb6f6063640 1 -- 172.21.15.42:0/40972281 wait complete. 2024-08-30T08:57:19.857 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:19 smithi190 ceph-mon[22001]: Deploying daemon mds.l on smithi190 2024-08-30T08:57:19.858 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:19 smithi190 ceph-mon[22001]: from='client.? 172.21.15.42:0/40972281' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-30T08:57:20.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:19 smithi167 ceph-mon[22063]: Deploying daemon mds.l on smithi190 2024-08-30T08:57:20.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:19 smithi167 ceph-mon[22063]: from='client.? 172.21.15.42:0/40972281' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-30T08:57:20.132 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:19 smithi042 ceph-mon[18652]: Deploying daemon mds.l on smithi190 2024-08-30T08:57:20.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:19 smithi042 ceph-mon[18652]: from='client.? 172.21.15.42:0/40972281' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-30T08:57:20.186 INFO:journalctl@ceph.mds.l.smithi190.stdout:Aug 30 08:57:20 smithi190 systemd[1]: Starting Ceph mds.l for b0895c1a-66ac-11ef-bcd4-c7b262605968... 2024-08-30T08:57:20.520 INFO:journalctl@ceph.mds.l.smithi190.stdout:Aug 30 08:57:20 smithi190 podman[51796]: 2024-08-30 08:57:20.158131986 +0000 UTC m=+0.015428180 image pull 79f4cde5e36911a5dca995eeb98098c2c8aae81dd52a088ef2c867bc3363bad0 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61 2024-08-30T08:57:20.520 INFO:journalctl@ceph.mds.l.smithi190.stdout:Aug 30 08:57:20 smithi190 podman[51796]: 2024-08-30 08:57:20.275136285 +0000 UTC m=+0.132432505 container create b0102237447b71169a91a36b5e2376e684526e91f1661337e627ef20df84d0c2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mds-l, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.37.1, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS) 2024-08-30T08:57:20.520 INFO:journalctl@ceph.mds.l.smithi190.stdout:Aug 30 08:57:20 smithi190 podman[51796]: 2024-08-30 08:57:20.519241827 +0000 UTC m=+0.376538004 container init b0102237447b71169a91a36b5e2376e684526e91f1661337e627ef20df84d0c2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mds-l, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, io.buildah.version=1.37.1, GIT_BRANCH=HEAD) 2024-08-30T08:57:20.936 INFO:journalctl@ceph.mds.l.smithi190.stdout:Aug 30 08:57:20 smithi190 podman[51796]: 2024-08-30 08:57:20.522172943 +0000 UTC m=+0.379469137 container start b0102237447b71169a91a36b5e2376e684526e91f1661337e627ef20df84d0c2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mds-l, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.1, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-08-30T08:57:20.936 INFO:journalctl@ceph.mds.l.smithi190.stdout:Aug 30 08:57:20 smithi190 ceph-mds[51815]: -- 172.21.15.190:0/352690259 <== mon.1 v2:172.21.15.190:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x562fe6e95860 con 0x562feb03b600 2024-08-30T08:57:20.936 INFO:journalctl@ceph.mds.l.smithi190.stdout:Aug 30 08:57:20 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mds-l[51811]: starting mds.l at 2024-08-30T08:57:20.936 INFO:journalctl@ceph.mds.l.smithi190.stdout:Aug 30 08:57:20 smithi190 bash[51796]: b0102237447b71169a91a36b5e2376e684526e91f1661337e627ef20df84d0c2 2024-08-30T08:57:20.936 INFO:journalctl@ceph.mds.l.smithi190.stdout:Aug 30 08:57:20 smithi190 systemd[1]: Started Ceph mds.l for b0895c1a-66ac-11ef-bcd4-c7b262605968. 2024-08-30T08:57:20.937 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:20 smithi190 ceph-mon[22001]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-30T08:57:20.937 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:20 smithi190 ceph-mon[22001]: mds.? [v2:172.21.15.190:6832/543555570,v1:172.21.15.190:6833/543555570] up:boot 2024-08-30T08:57:20.937 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:20 smithi190 ceph-mon[22001]: fsmap 1 up:standby 2024-08-30T08:57:20.937 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:20 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "l"} : dispatch 2024-08-30T08:57:20.937 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:20 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:20.937 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:20 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:20.937 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:20 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:20.937 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:20 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-30T08:57:20.937 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:20 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-30T08:57:20.937 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:20 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:57:20.985 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:20 smithi042 ceph-mon[18652]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-30T08:57:20.985 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:20 smithi042 ceph-mon[18652]: mds.? [v2:172.21.15.190:6832/543555570,v1:172.21.15.190:6833/543555570] up:boot 2024-08-30T08:57:20.986 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:20 smithi042 ceph-mon[18652]: fsmap 1 up:standby 2024-08-30T08:57:20.986 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:20 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "l"} : dispatch 2024-08-30T08:57:20.986 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:20 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:20.986 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:20 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:20.986 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:20 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:20.986 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:20 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-30T08:57:20.986 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:20 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-30T08:57:20.986 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:20 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:57:21.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:20 smithi167 ceph-mon[22063]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-30T08:57:21.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:20 smithi167 ceph-mon[22063]: mds.? [v2:172.21.15.190:6832/543555570,v1:172.21.15.190:6833/543555570] up:boot 2024-08-30T08:57:21.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:20 smithi167 ceph-mon[22063]: fsmap 1 up:standby 2024-08-30T08:57:21.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:20 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "l"} : dispatch 2024-08-30T08:57:21.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:20 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:21.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:20 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:21.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:20 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:21.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:20 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-30T08:57:21.078 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:20 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-30T08:57:21.078 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:20 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:57:22.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:21 smithi167 ceph-mon[22063]: Deploying daemon mds.k on smithi167 2024-08-30T08:57:22.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:21 smithi042 ceph-mon[18652]: Deploying daemon mds.k on smithi167 2024-08-30T08:57:22.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:21 smithi190 ceph-mon[22001]: Deploying daemon mds.k on smithi167 2024-08-30T08:57:22.393 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph fs dump --format=json 2024-08-30T08:57:22.656 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T08:57:22.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:22 smithi167 ceph-mon[22063]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-30T08:57:23.002 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:22 smithi042 ceph-mon[18652]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-30T08:57:23.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:23.090+0000 7f132120e640 1 -- 172.21.15.42:0/565342119 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f131c108180 msgr2=0x7f131c108590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:57:23.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:23.090+0000 7f132120e640 1 --2- 172.21.15.42:0/565342119 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f131c108180 0x7f131c108590 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f1310009900 tx=0x7f131002a410 comp rx=0 tx=0).stop 2024-08-30T08:57:23.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:23.090+0000 7f132120e640 1 -- 172.21.15.42:0/565342119 shutdown_connections 2024-08-30T08:57:23.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:23.090+0000 7f132120e640 1 --2- 172.21.15.42:0/565342119 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f131c108ad0 0x7f131c10f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:57:23.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:23.090+0000 7f132120e640 1 --2- 172.21.15.42:0/565342119 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f131c108180 0x7f131c108590 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:57:23.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:23.090+0000 7f132120e640 1 --2- 172.21.15.42:0/565342119 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f131c102840 0x7f131c102c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:57:23.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:23.090+0000 7f132120e640 1 -- 172.21.15.42:0/565342119 >> 172.21.15.42:0/565342119 conn(0x7f131c0794f0 msgr2=0x7f131c079900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:57:23.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:23.090+0000 7f132120e640 1 -- 172.21.15.42:0/565342119 shutdown_connections 2024-08-30T08:57:23.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:23.090+0000 7f132120e640 1 -- 172.21.15.42:0/565342119 wait complete. 2024-08-30T08:57:23.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:23.090+0000 7f132120e640 1 Processor -- start 2024-08-30T08:57:23.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:23.091+0000 7f132120e640 1 -- start start 2024-08-30T08:57:23.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:23.091+0000 7f132120e640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f131c102840 0x7f131c199280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:57:23.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:23.091+0000 7f132120e640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f131c108180 0x7f131c199820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:57:23.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:23.091+0000 7f132120e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f131c108ad0 0x7f131c19eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:57:23.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:23.091+0000 7f132120e640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f131c1127b0 con 0x7f131c108180 2024-08-30T08:57:23.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:23.091+0000 7f132120e640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f131c112630 con 0x7f131c102840 2024-08-30T08:57:23.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:23.091+0000 7f132120e640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f131c112930 con 0x7f131c108ad0 2024-08-30T08:57:23.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:23.091+0000 7f131a59c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f131c108180 0x7f131c199820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:57:23.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:23.091+0000 7f131ad9d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f131c102840 0x7f131c199280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:57:23.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:23.091+0000 7f131a59c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f131c108180 0x7f131c199820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:55902/0 (socket says 172.21.15.42:55902) 2024-08-30T08:57:23.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:23.091+0000 7f131a59c640 1 -- 172.21.15.42:0/3100506588 learned_addr learned my addr 172.21.15.42:0/3100506588 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T08:57:23.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:23.091+0000 7f131b59e640 1 --2- 172.21.15.42:0/3100506588 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f131c108ad0 0x7f131c19eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:57:23.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:23.091+0000 7f131a59c640 1 -- 172.21.15.42:0/3100506588 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f131c108ad0 msgr2=0x7f131c19eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:57:23.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:23.091+0000 7f131a59c640 1 --2- 172.21.15.42:0/3100506588 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f131c108ad0 0x7f131c19eac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:57:23.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:23.091+0000 7f131a59c640 1 -- 172.21.15.42:0/3100506588 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f131c102840 msgr2=0x7f131c199280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:57:23.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:23.091+0000 7f131a59c640 1 --2- 172.21.15.42:0/3100506588 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f131c102840 0x7f131c199280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:57:23.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:23.091+0000 7f131a59c640 1 -- 172.21.15.42:0/3100506588 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f131c19f1f0 con 0x7f131c108180 2024-08-30T08:57:23.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:23.092+0000 7f131a59c640 1 --2- 172.21.15.42:0/3100506588 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f131c108180 0x7f131c199820 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f131002a930 tx=0x7f1310004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:57:23.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:23.092+0000 7f1303fff640 1 -- 172.21.15.42:0/3100506588 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f131003f030 con 0x7f131c108180 2024-08-30T08:57:23.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:23.092+0000 7f1303fff640 1 -- 172.21.15.42:0/3100506588 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1310003aa0 con 0x7f131c108180 2024-08-30T08:57:23.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:23.092+0000 7f132120e640 1 -- 172.21.15.42:0/3100506588 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f131c19f480 con 0x7f131c108180 2024-08-30T08:57:23.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:23.092+0000 7f1303fff640 1 -- 172.21.15.42:0/3100506588 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1310035070 con 0x7f131c108180 2024-08-30T08:57:23.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:23.092+0000 7f132120e640 1 -- 172.21.15.42:0/3100506588 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f131c1a5c40 con 0x7f131c108180 2024-08-30T08:57:23.114 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:23.093+0000 7f132120e640 1 -- 172.21.15.42:0/3100506588 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f131c103ac0 con 0x7f131c108180 2024-08-30T08:57:23.114 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:23.095+0000 7f1303fff640 1 -- 172.21.15.42:0/3100506588 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) ==== 152577+0+0 (secure 0 0 0) 0x7f1310042030 con 0x7f131c108180 2024-08-30T08:57:23.114 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:23.095+0000 7f1303fff640 1 --2- 172.21.15.42:0/3100506588 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f13040b4b30 0x7f13040b7040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:57:23.114 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:23.095+0000 7f131ad9d640 1 --2- 172.21.15.42:0/3100506588 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f13040b4b30 0x7f13040b7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:57:23.114 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:23.095+0000 7f1303fff640 1 -- 172.21.15.42:0/3100506588 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 7127+0+0 (secure 0 0 0) 0x7f13101056a0 con 0x7f131c108180 2024-08-30T08:57:23.114 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:23.096+0000 7f131ad9d640 1 --2- 172.21.15.42:0/3100506588 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f13040b4b30 0x7f13040b7040 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f130c00bbe0 tx=0x7f130c00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:57:23.114 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:23.098+0000 7f1303fff640 1 -- 172.21.15.42:0/3100506588 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f13100cc620 con 0x7f131c108180 2024-08-30T08:57:23.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:22 smithi190 ceph-mon[22001]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-30T08:57:23.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:23.327+0000 7f132120e640 1 -- 172.21.15.42:0/3100506588 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f131c10a8c0 con 0x7f131c108180 2024-08-30T08:57:23.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:23.327+0000 7f1303fff640 1 -- 172.21.15.42:0/3100506588 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1435 (secure 0 0 0) 0x7f13100d0de0 con 0x7f131c108180 2024-08-30T08:57:23.330 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-08-30T08:57:23.330 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":2,"btime":"2024-08-30T08:57:20:560979+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24553,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/543555570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":543555570},{"type":"v1","addr":"172.21.15.190:6833","nonce":543555570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-08-30T08:57:23.330 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 2 2024-08-30T08:57:23.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:23.330+0000 7f132120e640 1 -- 172.21.15.42:0/3100506588 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f13040b4b30 msgr2=0x7f13040b7040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:57:23.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:23.330+0000 7f132120e640 1 --2- 172.21.15.42:0/3100506588 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f13040b4b30 0x7f13040b7040 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f130c00bbe0 tx=0x7f130c00b390 comp rx=0 tx=0).stop 2024-08-30T08:57:23.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:23.330+0000 7f132120e640 1 -- 172.21.15.42:0/3100506588 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f131c108180 msgr2=0x7f131c199820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:57:23.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:23.330+0000 7f132120e640 1 --2- 172.21.15.42:0/3100506588 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f131c108180 0x7f131c199820 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f131002a930 tx=0x7f1310004270 comp rx=0 tx=0).stop 2024-08-30T08:57:23.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:23.330+0000 7f132120e640 1 -- 172.21.15.42:0/3100506588 shutdown_connections 2024-08-30T08:57:23.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:23.330+0000 7f132120e640 1 --2- 172.21.15.42:0/3100506588 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f13040b4b30 0x7f13040b7040 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:57:23.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:23.330+0000 7f132120e640 1 --2- 172.21.15.42:0/3100506588 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f131c108ad0 0x7f131c19eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:57:23.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:23.330+0000 7f132120e640 1 --2- 172.21.15.42:0/3100506588 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f131c108180 0x7f131c199820 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:57:23.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:23.330+0000 7f132120e640 1 --2- 172.21.15.42:0/3100506588 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f131c102840 0x7f131c199280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:57:23.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:23.330+0000 7f132120e640 1 -- 172.21.15.42:0/3100506588 >> 172.21.15.42:0/3100506588 conn(0x7f131c0794f0 msgr2=0x7f131c106290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:57:23.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:23.330+0000 7f132120e640 1 -- 172.21.15.42:0/3100506588 shutdown_connections 2024-08-30T08:57:23.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:23.330+0000 7f132120e640 1 -- 172.21.15.42:0/3100506588 wait complete. 2024-08-30T08:57:23.577 INFO:journalctl@ceph.mds.k.smithi167.stdout:Aug 30 08:57:23 smithi167 systemd[1]: Started Ceph mds.k for b0895c1a-66ac-11ef-bcd4-c7b262605968. 2024-08-30T08:57:24.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:23 smithi167 ceph-mon[22063]: from='client.? 172.21.15.42:0/3100506588' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-30T08:57:24.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:23 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:24.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:23 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:24.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:23 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:24.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:23 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-30T08:57:24.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:23 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-30T08:57:24.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:23 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:57:24.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:23 smithi042 ceph-mon[18652]: from='client.? 172.21.15.42:0/3100506588' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-30T08:57:24.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:23 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:24.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:23 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:24.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:23 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:24.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:23 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-30T08:57:24.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:23 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-30T08:57:24.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:23 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:57:24.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:23 smithi190 ceph-mon[22001]: from='client.? 172.21.15.42:0/3100506588' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-30T08:57:24.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:23 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:24.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:23 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:24.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:23 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:24.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:23 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-30T08:57:24.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:23 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-30T08:57:24.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:23 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:57:25.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:24 smithi167 ceph-mon[22063]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-30T08:57:25.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:24 smithi167 ceph-mon[22063]: Deploying daemon mds.j on smithi042 2024-08-30T08:57:25.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:24 smithi167 ceph-mon[22063]: mds.? [v2:172.21.15.167:6832/3907818941,v1:172.21.15.167:6833/3907818941] up:boot 2024-08-30T08:57:25.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:24 smithi167 ceph-mon[22063]: fsmap 2 up:standby 2024-08-30T08:57:25.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:24 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "k"} : dispatch 2024-08-30T08:57:25.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:24 smithi042 ceph-mon[18652]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-30T08:57:25.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:24 smithi042 ceph-mon[18652]: Deploying daemon mds.j on smithi042 2024-08-30T08:57:25.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:24 smithi042 ceph-mon[18652]: mds.? [v2:172.21.15.167:6832/3907818941,v1:172.21.15.167:6833/3907818941] up:boot 2024-08-30T08:57:25.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:24 smithi042 ceph-mon[18652]: fsmap 2 up:standby 2024-08-30T08:57:25.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:24 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "k"} : dispatch 2024-08-30T08:57:25.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:24 smithi190 ceph-mon[22001]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-30T08:57:25.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:24 smithi190 ceph-mon[22001]: Deploying daemon mds.j on smithi042 2024-08-30T08:57:25.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:24 smithi190 ceph-mon[22001]: mds.? [v2:172.21.15.167:6832/3907818941,v1:172.21.15.167:6833/3907818941] up:boot 2024-08-30T08:57:25.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:24 smithi190 ceph-mon[22001]: fsmap 2 up:standby 2024-08-30T08:57:25.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:24 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "k"} : dispatch 2024-08-30T08:57:25.822 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:25 smithi042 ceph-mon[18652]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-30T08:57:26.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:25 smithi167 ceph-mon[22063]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-30T08:57:26.133 INFO:journalctl@ceph.mds.j.smithi042.stdout:Aug 30 08:57:25 smithi042 systemd[1]: Starting Ceph mds.j for b0895c1a-66ac-11ef-bcd4-c7b262605968... 2024-08-30T08:57:26.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:25 smithi190 ceph-mon[22001]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-30T08:57:26.228 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph fs dump --format=json 2024-08-30T08:57:26.472 INFO:journalctl@ceph.mds.j.smithi042.stdout:Aug 30 08:57:26 smithi042 podman[60611]: 2024-08-30 08:57:26.113082009 +0000 UTC m=+0.016089632 image pull 79f4cde5e36911a5dca995eeb98098c2c8aae81dd52a088ef2c867bc3363bad0 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61 2024-08-30T08:57:26.472 INFO:journalctl@ceph.mds.j.smithi042.stdout:Aug 30 08:57:26 smithi042 podman[60611]: 2024-08-30 08:57:26.234714067 +0000 UTC m=+0.137721690 container create f4e7191e96589b434ceb97793e9c3562cd8c70c439b0ac6d141aac6a78631e0e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mds-j, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.1, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-08-30T08:57:26.473 INFO:journalctl@ceph.mds.j.smithi042.stdout:Aug 30 08:57:26 smithi042 podman[60611]: 2024-08-30 08:57:26.471195861 +0000 UTC m=+0.374203484 container init f4e7191e96589b434ceb97793e9c3562cd8c70c439b0ac6d141aac6a78631e0e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mds-j, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_CLEAN=True, ceph=True, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.37.1, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-08-30T08:57:26.673 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T08:57:26.794 INFO:journalctl@ceph.mds.j.smithi042.stdout:Aug 30 08:57:26 smithi042 podman[60611]: 2024-08-30 08:57:26.474486281 +0000 UTC m=+0.377493915 container start f4e7191e96589b434ceb97793e9c3562cd8c70c439b0ac6d141aac6a78631e0e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mds-j, GIT_CLEAN=True, ceph=True, io.buildah.version=1.37.1, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-08-30T08:57:26.794 INFO:journalctl@ceph.mds.j.smithi042.stdout:Aug 30 08:57:26 smithi042 ceph-mds[60660]: -- 172.21.15.42:0/2886960052 <== mon.0 v2:172.21.15.42:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x560e0d081c20 con 0x560e11227600 2024-08-30T08:57:26.795 INFO:journalctl@ceph.mds.j.smithi042.stdout:Aug 30 08:57:26 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mds-j[60644]: starting mds.j at 2024-08-30T08:57:26.795 INFO:journalctl@ceph.mds.j.smithi042.stdout:Aug 30 08:57:26 smithi042 bash[60611]: f4e7191e96589b434ceb97793e9c3562cd8c70c439b0ac6d141aac6a78631e0e 2024-08-30T08:57:26.795 INFO:journalctl@ceph.mds.j.smithi042.stdout:Aug 30 08:57:26 smithi042 systemd[1]: Started Ceph mds.j for b0895c1a-66ac-11ef-bcd4-c7b262605968. 2024-08-30T08:57:27.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:27.176+0000 7fd674904640 1 -- 172.21.15.42:0/4284452964 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd67010dc60 msgr2=0x7fd670110150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:57:27.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:27.176+0000 7fd674904640 1 --2- 172.21.15.42:0/4284452964 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd67010dc60 0x7fd670110150 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fd668009ca0 tx=0x7fd66802a8e0 comp rx=0 tx=0).stop 2024-08-30T08:57:27.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:27.176+0000 7fd674904640 1 -- 172.21.15.42:0/4284452964 shutdown_connections 2024-08-30T08:57:27.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:27.176+0000 7fd674904640 1 --2- 172.21.15.42:0/4284452964 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd670105150 0x7fd67010d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:57:27.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:27.176+0000 7fd674904640 1 --2- 172.21.15.42:0/4284452964 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd67010dc60 0x7fd670110150 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:57:27.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:27.176+0000 7fd674904640 1 --2- 172.21.15.42:0/4284452964 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd670104800 0x7fd670104c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:57:27.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:27.176+0000 7fd674904640 1 -- 172.21.15.42:0/4284452964 >> 172.21.15.42:0/4284452964 conn(0x7fd670079470 msgr2=0x7fd670079880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:57:27.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:27.176+0000 7fd674904640 1 -- 172.21.15.42:0/4284452964 shutdown_connections 2024-08-30T08:57:27.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:27.176+0000 7fd674904640 1 -- 172.21.15.42:0/4284452964 wait complete. 2024-08-30T08:57:27.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:27.177+0000 7fd674904640 1 Processor -- start 2024-08-30T08:57:27.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:27.177+0000 7fd674904640 1 -- start start 2024-08-30T08:57:27.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:27.177+0000 7fd674904640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd670104800 0x7fd670102540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:57:27.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:27.177+0000 7fd674904640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd670105150 0x7fd670102a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:57:27.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:27.177+0000 7fd674904640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd67010dc60 0x7fd670102fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:57:27.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:27.177+0000 7fd674904640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fd670113920 con 0x7fd67010dc60 2024-08-30T08:57:27.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:27.177+0000 7fd674904640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd6701137a0 con 0x7fd670105150 2024-08-30T08:57:27.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:27.177+0000 7fd674904640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fd670113aa0 con 0x7fd670104800 2024-08-30T08:57:27.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:27.177+0000 7fd66ed9d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd67010dc60 0x7fd670102fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:57:27.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:27.177+0000 7fd66ed9d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd67010dc60 0x7fd670102fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:55958/0 (socket says 172.21.15.42:55958) 2024-08-30T08:57:27.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:27.177+0000 7fd66ed9d640 1 -- 172.21.15.42:0/4254222140 learned_addr learned my addr 172.21.15.42:0/4254222140 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T08:57:27.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:27.177+0000 7fd66dd9b640 1 --2- 172.21.15.42:0/4254222140 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd670105150 0x7fd670102a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:57:27.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:27.177+0000 7fd66e59c640 1 --2- 172.21.15.42:0/4254222140 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd670104800 0x7fd670102540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:57:27.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:27.177+0000 7fd66ed9d640 1 -- 172.21.15.42:0/4254222140 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd670104800 msgr2=0x7fd670102540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:57:27.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:27.177+0000 7fd66ed9d640 1 --2- 172.21.15.42:0/4254222140 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd670104800 0x7fd670102540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:57:27.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:27.177+0000 7fd66ed9d640 1 -- 172.21.15.42:0/4254222140 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd670105150 msgr2=0x7fd670102a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:57:27.221 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:27.177+0000 7fd66ed9d640 1 --2- 172.21.15.42:0/4254222140 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd670105150 0x7fd670102a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:57:27.221 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:27.177+0000 7fd66ed9d640 1 -- 172.21.15.42:0/4254222140 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6701a84e0 con 0x7fd67010dc60 2024-08-30T08:57:27.221 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:27.178+0000 7fd66ed9d640 1 --2- 172.21.15.42:0/4254222140 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd67010dc60 0x7fd670102fc0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fd668005cc0 tx=0x7fd668003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:57:27.221 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:27.178+0000 7fd65f7fe640 1 -- 172.21.15.42:0/4254222140 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd668004040 con 0x7fd67010dc60 2024-08-30T08:57:27.221 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:27.178+0000 7fd674904640 1 -- 172.21.15.42:0/4254222140 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6701a8770 con 0x7fd67010dc60 2024-08-30T08:57:27.221 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:27.178+0000 7fd65f7fe640 1 -- 172.21.15.42:0/4254222140 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd668004690 con 0x7fd67010dc60 2024-08-30T08:57:27.221 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:27.178+0000 7fd65f7fe640 1 -- 172.21.15.42:0/4254222140 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd668030d90 con 0x7fd67010dc60 2024-08-30T08:57:27.221 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:27.178+0000 7fd674904640 1 -- 172.21.15.42:0/4254222140 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6701a8cb0 con 0x7fd67010dc60 2024-08-30T08:57:27.221 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:27.179+0000 7fd674904640 1 -- 172.21.15.42:0/4254222140 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd670104c10 con 0x7fd67010dc60 2024-08-30T08:57:27.221 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:27.180+0000 7fd65f7fe640 1 -- 172.21.15.42:0/4254222140 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) ==== 152577+0+0 (secure 0 0 0) 0x7fd6680041e0 con 0x7fd67010dc60 2024-08-30T08:57:27.221 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:27.181+0000 7fd65f7fe640 1 --2- 172.21.15.42:0/4254222140 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fd6580b4b30 0x7fd6580b7040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:57:27.221 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:27.181+0000 7fd66e59c640 1 --2- 172.21.15.42:0/4254222140 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fd6580b4b30 0x7fd6580b7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:57:27.221 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:27.181+0000 7fd65f7fe640 1 -- 172.21.15.42:0/4254222140 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 7127+0+0 (secure 0 0 0) 0x7fd668106150 con 0x7fd67010dc60 2024-08-30T08:57:27.221 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:27.182+0000 7fd66e59c640 1 --2- 172.21.15.42:0/4254222140 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fd6580b4b30 0x7fd6580b7040 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fd660005e10 tx=0x7fd660005da0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:57:27.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:27.186+0000 7fd65f7fe640 1 -- 172.21.15.42:0/4254222140 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7fd6680cd680 con 0x7fd67010dc60 2024-08-30T08:57:27.421 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:27.418+0000 7fd674904640 1 -- 172.21.15.42:0/4254222140 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd6701a9030 con 0x7fd67010dc60 2024-08-30T08:57:27.421 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:27.419+0000 7fd65f7fe640 1 -- 172.21.15.42:0/4254222140 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3088 (secure 0 0 0) 0x7fd6680ccf80 con 0x7fd67010dc60 2024-08-30T08:57:27.422 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-08-30T08:57:27.422 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":4,"btime":"2024-08-30T08:57:26:779709+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14703,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/3188400000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":3188400000},{"type":"v1","addr":"172.21.15.42:6835","nonce":3188400000}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24553,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/543555570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":543555570},{"type":"v1","addr":"172.21.15.190:6833","nonce":543555570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24554,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/3907818941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":3907818941},{"type":"v1","addr":"172.21.15.167:6833","nonce":3907818941}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-08-30T08:57:27.422 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 4 2024-08-30T08:57:27.424 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:27.421+0000 7fd674904640 1 -- 172.21.15.42:0/4254222140 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fd6580b4b30 msgr2=0x7fd6580b7040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:57:27.424 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:27.421+0000 7fd674904640 1 --2- 172.21.15.42:0/4254222140 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fd6580b4b30 0x7fd6580b7040 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fd660005e10 tx=0x7fd660005da0 comp rx=0 tx=0).stop 2024-08-30T08:57:27.424 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:27.421+0000 7fd674904640 1 -- 172.21.15.42:0/4254222140 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd67010dc60 msgr2=0x7fd670102fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:57:27.424 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:27.421+0000 7fd674904640 1 --2- 172.21.15.42:0/4254222140 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd67010dc60 0x7fd670102fc0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fd668005cc0 tx=0x7fd668003ba0 comp rx=0 tx=0).stop 2024-08-30T08:57:27.424 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:27.421+0000 7fd674904640 1 -- 172.21.15.42:0/4254222140 shutdown_connections 2024-08-30T08:57:27.424 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:27.421+0000 7fd674904640 1 --2- 172.21.15.42:0/4254222140 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fd6580b4b30 0x7fd6580b7040 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:57:27.425 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:27.421+0000 7fd674904640 1 --2- 172.21.15.42:0/4254222140 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd67010dc60 0x7fd670102fc0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:57:27.425 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:27.421+0000 7fd674904640 1 --2- 172.21.15.42:0/4254222140 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd670105150 0x7fd670102a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:57:27.425 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:27.421+0000 7fd674904640 1 --2- 172.21.15.42:0/4254222140 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd670104800 0x7fd670102540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:57:27.425 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:27.421+0000 7fd674904640 1 -- 172.21.15.42:0/4254222140 >> 172.21.15.42:0/4254222140 conn(0x7fd670079470 msgr2=0x7fd670107710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:57:27.425 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:27.422+0000 7fd674904640 1 -- 172.21.15.42:0/4254222140 shutdown_connections 2024-08-30T08:57:27.425 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:27.422+0000 7fd674904640 1 -- 172.21.15.42:0/4254222140 wait complete. 2024-08-30T08:57:27.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:27 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:27.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:27 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:27.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:27 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:27.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:27 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-30T08:57:27.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:27 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-30T08:57:27.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:27 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:57:27.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:27 smithi042 ceph-mon[18652]: Deploying daemon mds.d on smithi042 2024-08-30T08:57:27.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:27 smithi042 ceph-mon[18652]: mds.? [v2:172.21.15.42:6834/3188400000,v1:172.21.15.42:6835/3188400000] up:boot 2024-08-30T08:57:27.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:27 smithi042 ceph-mon[18652]: fsmap 3 up:standby 2024-08-30T08:57:27.634 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:27 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "j"} : dispatch 2024-08-30T08:57:27.634 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:27 smithi042 ceph-mon[18652]: from='client.? 172.21.15.42:0/4254222140' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-30T08:57:27.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:27 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:27.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:27 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:27.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:27 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:27.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:27 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-30T08:57:27.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:27 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-30T08:57:27.937 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:27 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:57:27.937 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:27 smithi190 ceph-mon[22001]: Deploying daemon mds.d on smithi042 2024-08-30T08:57:27.937 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:27 smithi190 ceph-mon[22001]: mds.? [v2:172.21.15.42:6834/3188400000,v1:172.21.15.42:6835/3188400000] up:boot 2024-08-30T08:57:27.937 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:27 smithi190 ceph-mon[22001]: fsmap 3 up:standby 2024-08-30T08:57:27.937 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:27 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "j"} : dispatch 2024-08-30T08:57:27.937 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:27 smithi190 ceph-mon[22001]: from='client.? 172.21.15.42:0/4254222140' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-30T08:57:28.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:27 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:28.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:27 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:28.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:27 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:28.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:27 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-30T08:57:28.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:27 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-30T08:57:28.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:27 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:57:28.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:27 smithi167 ceph-mon[22063]: Deploying daemon mds.d on smithi042 2024-08-30T08:57:28.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:27 smithi167 ceph-mon[22063]: mds.? [v2:172.21.15.42:6834/3188400000,v1:172.21.15.42:6835/3188400000] up:boot 2024-08-30T08:57:28.078 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:27 smithi167 ceph-mon[22063]: fsmap 3 up:standby 2024-08-30T08:57:28.078 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:27 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "j"} : dispatch 2024-08-30T08:57:28.078 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:27 smithi167 ceph-mon[22063]: from='client.? 172.21.15.42:0/4254222140' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-30T08:57:28.788 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:28 smithi042 ceph-mon[18652]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-30T08:57:28.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:28 smithi190 ceph-mon[22001]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-30T08:57:29.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:28 smithi167 ceph-mon[22063]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-30T08:57:29.874 INFO:journalctl@ceph.mds.d.smithi042.stdout:Aug 30 08:57:29 smithi042 systemd[1]: Starting Ceph mds.d for b0895c1a-66ac-11ef-bcd4-c7b262605968... 2024-08-30T08:57:30.124 INFO:journalctl@ceph.mds.d.smithi042.stdout:Aug 30 08:57:29 smithi042 podman[61022]: 2024-08-30 08:57:29.774920267 +0000 UTC m=+0.015860624 image pull 79f4cde5e36911a5dca995eeb98098c2c8aae81dd52a088ef2c867bc3363bad0 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61 2024-08-30T08:57:30.125 INFO:journalctl@ceph.mds.d.smithi042.stdout:Aug 30 08:57:29 smithi042 podman[61022]: 2024-08-30 08:57:29.984176323 +0000 UTC m=+0.225116663 container create e53c84315db0acd7dedbe9f8e1880bdc7c4793da98a260cd440334b9b46f4d78 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mds-d, CEPH_POINT_RELEASE=, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.37.1, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True) 2024-08-30T08:57:30.383 INFO:journalctl@ceph.mds.d.smithi042.stdout:Aug 30 08:57:30 smithi042 podman[61022]: 2024-08-30 08:57:30.228846765 +0000 UTC m=+0.469787115 container init e53c84315db0acd7dedbe9f8e1880bdc7c4793da98a260cd440334b9b46f4d78 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mds-d, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.37.1, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611) 2024-08-30T08:57:30.383 INFO:journalctl@ceph.mds.d.smithi042.stdout:Aug 30 08:57:30 smithi042 podman[61022]: 2024-08-30 08:57:30.231605287 +0000 UTC m=+0.472545628 container start e53c84315db0acd7dedbe9f8e1880bdc7c4793da98a260cd440334b9b46f4d78 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mds-d, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.1, org.label-schema.build-date=20240731, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-08-30T08:57:30.383 INFO:journalctl@ceph.mds.d.smithi042.stdout:Aug 30 08:57:30 smithi042 ceph-mds[61041]: -- 172.21.15.42:0/1195736186 <== mon.0 v2:172.21.15.42:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5586b9851860 con 0x5586bd9f7600 2024-08-30T08:57:30.383 INFO:journalctl@ceph.mds.d.smithi042.stdout:Aug 30 08:57:30 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mds-d[61037]: starting mds.d at 2024-08-30T08:57:30.383 INFO:journalctl@ceph.mds.d.smithi042.stdout:Aug 30 08:57:30 smithi042 bash[61022]: e53c84315db0acd7dedbe9f8e1880bdc7c4793da98a260cd440334b9b46f4d78 2024-08-30T08:57:30.383 INFO:journalctl@ceph.mds.d.smithi042.stdout:Aug 30 08:57:30 smithi042 systemd[1]: Started Ceph mds.d for b0895c1a-66ac-11ef-bcd4-c7b262605968. 2024-08-30T08:57:30.823 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph fs dump --format=json 2024-08-30T08:57:30.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:30 smithi042 ceph-mon[18652]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-30T08:57:30.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:30 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:30.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:30 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:30.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:30 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:30.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:30 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-30T08:57:30.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:30 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-30T08:57:30.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:30 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:57:30.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:30 smithi190 ceph-mon[22001]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-30T08:57:30.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:30 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:30.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:30 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:30.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:30 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:30.937 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:30 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-30T08:57:30.937 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:30 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-30T08:57:30.937 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:30 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:57:30.950 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:30 smithi167 ceph-mon[22063]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-30T08:57:30.950 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:30 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:30.950 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:30 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:30.950 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:30 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:30.951 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:30 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-30T08:57:30.951 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:30 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-30T08:57:30.951 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:30 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:57:31.079 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T08:57:31.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:31.599+0000 7fa1ddd7c640 1 -- 172.21.15.42:0/1027358442 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa1d8108180 msgr2=0x7fa1d8108590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:57:31.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:31.599+0000 7fa1ddd7c640 1 --2- 172.21.15.42:0/1027358442 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa1d8108180 0x7fa1d8108590 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fa1c0009900 tx=0x7fa1c002a410 comp rx=0 tx=0).stop 2024-08-30T08:57:31.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:31.599+0000 7fa1ddd7c640 1 -- 172.21.15.42:0/1027358442 shutdown_connections 2024-08-30T08:57:31.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:31.599+0000 7fa1ddd7c640 1 --2- 172.21.15.42:0/1027358442 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa1d8108ad0 0x7fa1d810f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:57:31.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:31.599+0000 7fa1ddd7c640 1 --2- 172.21.15.42:0/1027358442 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa1d8108180 0x7fa1d8108590 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:57:31.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:31.599+0000 7fa1ddd7c640 1 --2- 172.21.15.42:0/1027358442 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa1d8102840 0x7fa1d8102c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:57:31.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:31.599+0000 7fa1ddd7c640 1 -- 172.21.15.42:0/1027358442 >> 172.21.15.42:0/1027358442 conn(0x7fa1d8079580 msgr2=0x7fa1d8079990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:57:31.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:31.599+0000 7fa1ddd7c640 1 -- 172.21.15.42:0/1027358442 shutdown_connections 2024-08-30T08:57:31.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:31.599+0000 7fa1ddd7c640 1 -- 172.21.15.42:0/1027358442 wait complete. 2024-08-30T08:57:31.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:31.600+0000 7fa1ddd7c640 1 Processor -- start 2024-08-30T08:57:31.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:31.600+0000 7fa1ddd7c640 1 -- start start 2024-08-30T08:57:31.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:31.600+0000 7fa1ddd7c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa1d8102840 0x7fa1d819b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:57:31.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:31.600+0000 7fa1ddd7c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa1d8108180 0x7fa1d819b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:57:31.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:31.600+0000 7fa1ddd7c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa1d8108ad0 0x7fa1d819bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:57:31.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:31.600+0000 7fa1ddd7c640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fa1d8112720 con 0x7fa1d8108180 2024-08-30T08:57:31.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:31.600+0000 7fa1ddd7c640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa1d81125a0 con 0x7fa1d8102840 2024-08-30T08:57:31.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:31.600+0000 7fa1ddd7c640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fa1d81128a0 con 0x7fa1d8108ad0 2024-08-30T08:57:31.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:31.600+0000 7fa1ceffd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa1d8108180 0x7fa1d819b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:57:31.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:31.600+0000 7fa1ceffd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa1d8108180 0x7fa1d819b5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:45362/0 (socket says 172.21.15.42:45362) 2024-08-30T08:57:31.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:31.600+0000 7fa1ceffd640 1 -- 172.21.15.42:0/1716206117 learned_addr learned my addr 172.21.15.42:0/1716206117 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T08:57:31.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:31.600+0000 7fa1d77fe640 1 --2- 172.21.15.42:0/1716206117 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa1d8102840 0x7fa1d819b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:57:31.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:31.600+0000 7fa1d7fff640 1 --2- 172.21.15.42:0/1716206117 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa1d8108ad0 0x7fa1d819bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:57:31.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:31.601+0000 7fa1ceffd640 1 -- 172.21.15.42:0/1716206117 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa1d8108ad0 msgr2=0x7fa1d819bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:57:31.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:31.601+0000 7fa1ceffd640 1 --2- 172.21.15.42:0/1716206117 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa1d8108ad0 0x7fa1d819bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:57:31.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:31.601+0000 7fa1ceffd640 1 -- 172.21.15.42:0/1716206117 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa1d8102840 msgr2=0x7fa1d819b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:57:31.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:31.601+0000 7fa1ceffd640 1 --2- 172.21.15.42:0/1716206117 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa1d8102840 0x7fa1d819b0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:57:31.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:31.601+0000 7fa1ceffd640 1 -- 172.21.15.42:0/1716206117 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1d81a6340 con 0x7fa1d8108180 2024-08-30T08:57:31.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:31.601+0000 7fa1ceffd640 1 --2- 172.21.15.42:0/1716206117 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa1d8108180 0x7fa1d819b5f0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fa1c0032fd0 tx=0x7fa1c0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:57:31.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:31.601+0000 7fa1d57fa640 1 -- 172.21.15.42:0/1716206117 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1c003f030 con 0x7fa1d8108180 2024-08-30T08:57:31.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:31.601+0000 7fa1ddd7c640 1 -- 172.21.15.42:0/1716206117 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1d81a65d0 con 0x7fa1d8108180 2024-08-30T08:57:31.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:31.601+0000 7fa1d57fa640 1 -- 172.21.15.42:0/1716206117 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa1c0004490 con 0x7fa1d8108180 2024-08-30T08:57:31.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:31.601+0000 7fa1d57fa640 1 -- 172.21.15.42:0/1716206117 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1c0035070 con 0x7fa1d8108180 2024-08-30T08:57:31.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:31.602+0000 7fa1ddd7c640 1 -- 172.21.15.42:0/1716206117 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1d81a6ab0 con 0x7fa1d8108180 2024-08-30T08:57:31.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:31.603+0000 7fa1ddd7c640 1 -- 172.21.15.42:0/1716206117 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1d8102c50 con 0x7fa1d8108180 2024-08-30T08:57:31.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:31.604+0000 7fa1d57fa640 1 -- 172.21.15.42:0/1716206117 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) ==== 152577+0+0 (secure 0 0 0) 0x7fa1c0042030 con 0x7fa1d8108180 2024-08-30T08:57:31.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:31.605+0000 7fa1d57fa640 1 --2- 172.21.15.42:0/1716206117 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fa1c40b4b60 0x7fa1c40b7070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:57:31.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:31.605+0000 7fa1d77fe640 1 --2- 172.21.15.42:0/1716206117 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fa1c40b4b60 0x7fa1c40b7070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:57:31.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:31.605+0000 7fa1d57fa640 1 -- 172.21.15.42:0/1716206117 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 7127+0+0 (secure 0 0 0) 0x7fa1c0105150 con 0x7fa1d8108180 2024-08-30T08:57:31.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:31.605+0000 7fa1d77fe640 1 --2- 172.21.15.42:0/1716206117 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fa1c40b4b60 0x7fa1c40b7070 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fa1d806c120 tx=0x7fa1c800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:57:31.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:31.610+0000 7fa1d57fa640 1 -- 172.21.15.42:0/1716206117 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7fa1c00cc030 con 0x7fa1d8108180 2024-08-30T08:57:31.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:31.837+0000 7fa1ddd7c640 1 -- 172.21.15.42:0/1716206117 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa1d810a920 con 0x7fa1d8108180 2024-08-30T08:57:31.840 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:31 smithi042 ceph-mon[18652]: Deploying daemon mds.e on smithi167 2024-08-30T08:57:31.840 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:31 smithi042 ceph-mon[18652]: mds.? [v2:172.21.15.42:6836/3356537674,v1:172.21.15.42:6837/3356537674] up:boot 2024-08-30T08:57:31.840 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:31 smithi042 ceph-mon[18652]: fsmap 4 up:standby 2024-08-30T08:57:31.840 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:31 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "d"} : dispatch 2024-08-30T08:57:31.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:31.837+0000 7fa1d57fa640 1 -- 172.21.15.42:0/1716206117 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3913 (secure 0 0 0) 0x7fa1c00d07f0 con 0x7fa1d8108180 2024-08-30T08:57:31.841 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-08-30T08:57:31.841 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":5,"btime":"2024-08-30T08:57:30:607119+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14703,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/3188400000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":3188400000},{"type":"v1","addr":"172.21.15.42:6835","nonce":3188400000}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14715,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/3356537674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":3356537674},{"type":"v1","addr":"172.21.15.42:6837","nonce":3356537674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24553,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/543555570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":543555570},{"type":"v1","addr":"172.21.15.190:6833","nonce":543555570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24554,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/3907818941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":3907818941},{"type":"v1","addr":"172.21.15.167:6833","nonce":3907818941}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-08-30T08:57:31.841 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 5 2024-08-30T08:57:31.843 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:31.841+0000 7fa1ddd7c640 1 -- 172.21.15.42:0/1716206117 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fa1c40b4b60 msgr2=0x7fa1c40b7070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:57:31.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:31.841+0000 7fa1ddd7c640 1 --2- 172.21.15.42:0/1716206117 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fa1c40b4b60 0x7fa1c40b7070 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fa1d806c120 tx=0x7fa1c800b390 comp rx=0 tx=0).stop 2024-08-30T08:57:31.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:31.841+0000 7fa1ddd7c640 1 -- 172.21.15.42:0/1716206117 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa1d8108180 msgr2=0x7fa1d819b5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:57:31.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:31.841+0000 7fa1ddd7c640 1 --2- 172.21.15.42:0/1716206117 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa1d8108180 0x7fa1d819b5f0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fa1c0032fd0 tx=0x7fa1c0004270 comp rx=0 tx=0).stop 2024-08-30T08:57:31.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:31.841+0000 7fa1ddd7c640 1 -- 172.21.15.42:0/1716206117 shutdown_connections 2024-08-30T08:57:31.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:31.841+0000 7fa1ddd7c640 1 --2- 172.21.15.42:0/1716206117 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fa1c40b4b60 0x7fa1c40b7070 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:57:31.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:31.841+0000 7fa1ddd7c640 1 --2- 172.21.15.42:0/1716206117 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa1d8108ad0 0x7fa1d819bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:57:31.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:31.841+0000 7fa1ddd7c640 1 --2- 172.21.15.42:0/1716206117 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa1d8108180 0x7fa1d819b5f0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:57:31.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:31.841+0000 7fa1ddd7c640 1 --2- 172.21.15.42:0/1716206117 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa1d8102840 0x7fa1d819b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:57:31.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:31.841+0000 7fa1ddd7c640 1 -- 172.21.15.42:0/1716206117 >> 172.21.15.42:0/1716206117 conn(0x7fa1d8079580 msgr2=0x7fa1d8104240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:57:31.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:31.841+0000 7fa1ddd7c640 1 -- 172.21.15.42:0/1716206117 shutdown_connections 2024-08-30T08:57:31.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:31.841+0000 7fa1ddd7c640 1 -- 172.21.15.42:0/1716206117 wait complete. 2024-08-30T08:57:31.924 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:31 smithi167 ceph-mon[22063]: Deploying daemon mds.e on smithi167 2024-08-30T08:57:31.924 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:31 smithi167 ceph-mon[22063]: mds.? [v2:172.21.15.42:6836/3356537674,v1:172.21.15.42:6837/3356537674] up:boot 2024-08-30T08:57:31.924 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:31 smithi167 ceph-mon[22063]: fsmap 4 up:standby 2024-08-30T08:57:31.924 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:31 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "d"} : dispatch 2024-08-30T08:57:31.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:31 smithi190 ceph-mon[22001]: Deploying daemon mds.e on smithi167 2024-08-30T08:57:31.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:31 smithi190 ceph-mon[22001]: mds.? [v2:172.21.15.42:6836/3356537674,v1:172.21.15.42:6837/3356537674] up:boot 2024-08-30T08:57:31.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:31 smithi190 ceph-mon[22001]: fsmap 4 up:standby 2024-08-30T08:57:31.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:31 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "d"} : dispatch 2024-08-30T08:57:32.320 INFO:journalctl@ceph.mds.e.smithi167.stdout:Aug 30 08:57:32 smithi167 systemd[1]: Starting Ceph mds.e for b0895c1a-66ac-11ef-bcd4-c7b262605968... 2024-08-30T08:57:32.577 INFO:journalctl@ceph.mds.e.smithi167.stdout:Aug 30 08:57:32 smithi167 podman[51847]: 2024-08-30 08:57:32.430887914 +0000 UTC m=+0.016002817 image pull 79f4cde5e36911a5dca995eeb98098c2c8aae81dd52a088ef2c867bc3363bad0 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61 2024-08-30T08:57:32.577 INFO:journalctl@ceph.mds.e.smithi167.stdout:Aug 30 08:57:32 smithi167 podman[51847]: 2024-08-30 08:57:32.566940835 +0000 UTC m=+0.152055739 container create 7bbee5d4b3ee459b66da5d0b33556e2b04008534654bb68a95be91e272a1c26e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mds-e, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.1, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.build-date=20240731, GIT_CLEAN=True) 2024-08-30T08:57:32.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:32 smithi042 ceph-mon[18652]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-30T08:57:32.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:32 smithi042 ceph-mon[18652]: from='client.? 172.21.15.42:0/1716206117' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-30T08:57:32.921 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:32 smithi167 ceph-mon[22063]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-30T08:57:32.921 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:32 smithi167 ceph-mon[22063]: from='client.? 172.21.15.42:0/1716206117' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-30T08:57:32.922 INFO:journalctl@ceph.mds.e.smithi167.stdout:Aug 30 08:57:32 smithi167 podman[51847]: 2024-08-30 08:57:32.819976079 +0000 UTC m=+0.405090982 container init 7bbee5d4b3ee459b66da5d0b33556e2b04008534654bb68a95be91e272a1c26e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mds-e, CEPH_POINT_RELEASE=, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, io.buildah.version=1.37.1) 2024-08-30T08:57:32.922 INFO:journalctl@ceph.mds.e.smithi167.stdout:Aug 30 08:57:32 smithi167 podman[51847]: 2024-08-30 08:57:32.823060122 +0000 UTC m=+0.408175026 container start 7bbee5d4b3ee459b66da5d0b33556e2b04008534654bb68a95be91e272a1c26e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mds-e, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.1, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-08-30T08:57:32.922 INFO:journalctl@ceph.mds.e.smithi167.stdout:Aug 30 08:57:32 smithi167 ceph-mds[51866]: -- 172.21.15.167:0/3630250672 <== mon.2 v2:172.21.15.167:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5633e0913860 con 0x5633e4ab9600 2024-08-30T08:57:32.922 INFO:journalctl@ceph.mds.e.smithi167.stdout:Aug 30 08:57:32 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mds-e[51862]: starting mds.e at 2024-08-30T08:57:32.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:32 smithi190 ceph-mon[22001]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-30T08:57:32.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:32 smithi190 ceph-mon[22001]: from='client.? 172.21.15.42:0/1716206117' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-30T08:57:33.327 INFO:journalctl@ceph.mds.e.smithi167.stdout:Aug 30 08:57:32 smithi167 bash[51847]: 7bbee5d4b3ee459b66da5d0b33556e2b04008534654bb68a95be91e272a1c26e 2024-08-30T08:57:33.327 INFO:journalctl@ceph.mds.e.smithi167.stdout:Aug 30 08:57:32 smithi167 systemd[1]: Started Ceph mds.e for b0895c1a-66ac-11ef-bcd4-c7b262605968. 2024-08-30T08:57:34.228 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:33 smithi167 ceph-mon[22063]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-30T08:57:34.229 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:33 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:34.229 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:33 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:34.229 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:33 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:34.229 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:33 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-30T08:57:34.229 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:33 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-30T08:57:34.229 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:33 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:57:34.229 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:33 smithi167 ceph-mon[22063]: Deploying daemon mds.f on smithi190 2024-08-30T08:57:34.229 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:33 smithi167 ceph-mon[22063]: mds.? [v2:172.21.15.167:6834/234596845,v1:172.21.15.167:6835/234596845] up:boot 2024-08-30T08:57:34.229 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:33 smithi167 ceph-mon[22063]: fsmap 5 up:standby 2024-08-30T08:57:34.229 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:33 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "e"} : dispatch 2024-08-30T08:57:34.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:33 smithi042 ceph-mon[18652]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-30T08:57:34.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:33 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:34.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:33 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:34.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:33 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:34.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:33 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-30T08:57:34.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:33 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-30T08:57:34.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:33 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:57:34.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:33 smithi042 ceph-mon[18652]: Deploying daemon mds.f on smithi190 2024-08-30T08:57:34.384 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:33 smithi042 ceph-mon[18652]: mds.? [v2:172.21.15.167:6834/234596845,v1:172.21.15.167:6835/234596845] up:boot 2024-08-30T08:57:34.384 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:33 smithi042 ceph-mon[18652]: fsmap 5 up:standby 2024-08-30T08:57:34.384 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:33 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "e"} : dispatch 2024-08-30T08:57:34.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:33 smithi190 ceph-mon[22001]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-30T08:57:34.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:33 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:34.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:33 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:34.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:33 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:34.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:33 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-30T08:57:34.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:33 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-30T08:57:34.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:33 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:57:34.437 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:33 smithi190 ceph-mon[22001]: Deploying daemon mds.f on smithi190 2024-08-30T08:57:34.437 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:33 smithi190 ceph-mon[22001]: mds.? [v2:172.21.15.167:6834/234596845,v1:172.21.15.167:6835/234596845] up:boot 2024-08-30T08:57:34.437 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:33 smithi190 ceph-mon[22001]: fsmap 5 up:standby 2024-08-30T08:57:34.437 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:33 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "e"} : dispatch 2024-08-30T08:57:34.647 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph fs dump --format=json 2024-08-30T08:57:34.917 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T08:57:35.096 INFO:journalctl@ceph.mds.f.smithi190.stdout:Aug 30 08:57:34 smithi190 systemd[1]: Starting Ceph mds.f for b0895c1a-66ac-11ef-bcd4-c7b262605968... 2024-08-30T08:57:35.391 INFO:journalctl@ceph.mds.f.smithi190.stdout:Aug 30 08:57:35 smithi190 podman[52099]: 2024-08-30 08:57:34.998026606 +0000 UTC m=+0.015927274 image pull 79f4cde5e36911a5dca995eeb98098c2c8aae81dd52a088ef2c867bc3363bad0 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61 2024-08-30T08:57:35.444 INFO:journalctl@ceph.mds.f.smithi190.stdout:Aug 30 08:57:35 smithi190 podman[52099]: 2024-08-30 08:57:35.120842223 +0000 UTC m=+0.138742935 container create 5f43b96cf342144dd1d4da00172a5619c632f8d5e68b671b9d2cc22c21645ff3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mds-f, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.37.1) 2024-08-30T08:57:35.445 INFO:journalctl@ceph.mds.f.smithi190.stdout:Aug 30 08:57:35 smithi190 podman[52099]: 2024-08-30 08:57:35.389509622 +0000 UTC m=+0.407410302 container init 5f43b96cf342144dd1d4da00172a5619c632f8d5e68b671b9d2cc22c21645ff3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mds-f, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_CLEAN=True, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.schema-version=1.0, io.buildah.version=1.37.1, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-08-30T08:57:35.445 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:35.398+0000 7f7d0c93b640 1 -- 172.21.15.42:0/2265030377 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7d08102880 msgr2=0x7f7d08102c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:57:35.446 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:35.398+0000 7f7d0c93b640 1 --2- 172.21.15.42:0/2265030377 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7d08102880 0x7f7d08102c90 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f7cf8009900 tx=0x7f7cf802a410 comp rx=0 tx=0).stop 2024-08-30T08:57:35.446 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:35.398+0000 7f7d0c93b640 1 -- 172.21.15.42:0/2265030377 shutdown_connections 2024-08-30T08:57:35.446 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:35.398+0000 7f7d0c93b640 1 --2- 172.21.15.42:0/2265030377 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7d081081c0 0x7f7d081085d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:57:35.446 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:35.398+0000 7f7d0c93b640 1 --2- 172.21.15.42:0/2265030377 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7d08108b10 0x7f7d0810f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:57:35.446 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:35.398+0000 7f7d0c93b640 1 --2- 172.21.15.42:0/2265030377 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7d08102880 0x7f7d08102c90 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:57:35.446 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:35.398+0000 7f7d0c93b640 1 -- 172.21.15.42:0/2265030377 >> 172.21.15.42:0/2265030377 conn(0x7f7d08079510 msgr2=0x7f7d08079920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:57:35.446 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:35.398+0000 7f7d0c93b640 1 -- 172.21.15.42:0/2265030377 shutdown_connections 2024-08-30T08:57:35.446 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:35.398+0000 7f7d0c93b640 1 -- 172.21.15.42:0/2265030377 wait complete. 2024-08-30T08:57:35.446 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:35.399+0000 7f7d0c93b640 1 Processor -- start 2024-08-30T08:57:35.446 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:35.399+0000 7f7d0c93b640 1 -- start start 2024-08-30T08:57:35.447 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:35.399+0000 7f7d0c93b640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7d08102880 0x7f7d08199230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:57:35.447 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:35.399+0000 7f7d0c93b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7d081081c0 0x7f7d081997d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:57:35.447 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:35.399+0000 7f7d0c93b640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7d08108b10 0x7f7d0819ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:57:35.447 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:35.399+0000 7f7d0c93b640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f7d081127e0 con 0x7f7d08102880 2024-08-30T08:57:35.447 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:35.399+0000 7f7d0c93b640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f7d08112660 con 0x7f7d08108b10 2024-08-30T08:57:35.447 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:35.399+0000 7f7d0c93b640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f7d08112960 con 0x7f7d081081c0 2024-08-30T08:57:35.447 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:35.399+0000 7f7d0659c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7d08102880 0x7f7d08199230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:57:35.447 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:35.399+0000 7f7d0659c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7d08102880 0x7f7d08199230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:45366/0 (socket says 172.21.15.42:45366) 2024-08-30T08:57:35.447 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:35.399+0000 7f7d0659c640 1 -- 172.21.15.42:0/3232790366 learned_addr learned my addr 172.21.15.42:0/3232790366 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T08:57:35.447 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:35.399+0000 7f7d05d9b640 1 --2- 172.21.15.42:0/3232790366 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7d081081c0 0x7f7d081997d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:57:35.447 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:35.399+0000 7f7d06d9d640 1 --2- 172.21.15.42:0/3232790366 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7d08108b10 0x7f7d0819ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:57:35.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:35.400+0000 7f7d0659c640 1 -- 172.21.15.42:0/3232790366 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7d081081c0 msgr2=0x7f7d081997d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:57:35.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:35.400+0000 7f7d0659c640 1 --2- 172.21.15.42:0/3232790366 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7d081081c0 0x7f7d081997d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:57:35.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:35.400+0000 7f7d0659c640 1 -- 172.21.15.42:0/3232790366 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7d08108b10 msgr2=0x7f7d0819ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:57:35.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:35.400+0000 7f7d0659c640 1 --2- 172.21.15.42:0/3232790366 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7d08108b10 0x7f7d0819ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:57:35.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:35.400+0000 7f7d0659c640 1 -- 172.21.15.42:0/3232790366 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d0819f1a0 con 0x7f7d08102880 2024-08-30T08:57:35.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:35.400+0000 7f7d0659c640 1 --2- 172.21.15.42:0/3232790366 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7d08102880 0x7f7d08199230 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f7cf8005940 tx=0x7f7cf8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:57:35.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:35.400+0000 7f7cf77fe640 1 -- 172.21.15.42:0/3232790366 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cf803f030 con 0x7f7d08102880 2024-08-30T08:57:35.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:35.400+0000 7f7cf77fe640 1 -- 172.21.15.42:0/3232790366 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7cf8003aa0 con 0x7f7d08102880 2024-08-30T08:57:35.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:35.400+0000 7f7d0c93b640 1 -- 172.21.15.42:0/3232790366 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d0819f430 con 0x7f7d08102880 2024-08-30T08:57:35.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:35.400+0000 7f7cf77fe640 1 -- 172.21.15.42:0/3232790366 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cf8035070 con 0x7f7d08102880 2024-08-30T08:57:35.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:35.400+0000 7f7d0c93b640 1 -- 172.21.15.42:0/3232790366 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d081a5bf0 con 0x7f7d08102880 2024-08-30T08:57:35.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:35.401+0000 7f7d0c93b640 1 -- 172.21.15.42:0/3232790366 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d08103b00 con 0x7f7d08102880 2024-08-30T08:57:35.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:35.403+0000 7f7cf77fe640 1 -- 172.21.15.42:0/3232790366 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) ==== 152577+0+0 (secure 0 0 0) 0x7f7cf8042030 con 0x7f7d08102880 2024-08-30T08:57:35.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:35.403+0000 7f7cf77fe640 1 --2- 172.21.15.42:0/3232790366 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f7cf00b4bb0 0x7f7cf00b70c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:57:35.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:35.404+0000 7f7d05d9b640 1 --2- 172.21.15.42:0/3232790366 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f7cf00b4bb0 0x7f7cf00b70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:57:35.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:35.404+0000 7f7cf77fe640 1 -- 172.21.15.42:0/3232790366 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 7127+0+0 (secure 0 0 0) 0x7f7cf81055e0 con 0x7f7d08102880 2024-08-30T08:57:35.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:35.404+0000 7f7d05d9b640 1 --2- 172.21.15.42:0/3232790366 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f7cf00b4bb0 0x7f7cf00b70c0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f7d08105110 tx=0x7f7cfc005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:57:35.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:35.407+0000 7f7cf77fe640 1 -- 172.21.15.42:0/3232790366 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f7cf80cc4c0 con 0x7f7d08102880 2024-08-30T08:57:35.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:35.633+0000 7f7d0c93b640 1 -- 172.21.15.42:0/3232790366 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7d0810a890 con 0x7f7d08102880 2024-08-30T08:57:35.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:35.634+0000 7f7cf77fe640 1 -- 172.21.15.42:0/3232790366 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4738 (secure 0 0 0) 0x7f7cf80d0c80 con 0x7f7d08102880 2024-08-30T08:57:35.637 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-08-30T08:57:35.637 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":6,"btime":"2024-08-30T08:57:33:621176+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14703,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/3188400000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":3188400000},{"type":"v1","addr":"172.21.15.42:6835","nonce":3188400000}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14715,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/3356537674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":3356537674},{"type":"v1","addr":"172.21.15.42:6837","nonce":3356537674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24553,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/543555570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":543555570},{"type":"v1","addr":"172.21.15.190:6833","nonce":543555570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24554,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/3907818941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":3907818941},{"type":"v1","addr":"172.21.15.167:6833","nonce":3907818941}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24572,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/234596845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":234596845},{"type":"v1","addr":"172.21.15.167:6835","nonce":234596845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-08-30T08:57:35.637 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 6 2024-08-30T08:57:35.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:35.636+0000 7f7d0c93b640 1 -- 172.21.15.42:0/3232790366 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f7cf00b4bb0 msgr2=0x7f7cf00b70c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:57:35.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:35.636+0000 7f7d0c93b640 1 --2- 172.21.15.42:0/3232790366 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f7cf00b4bb0 0x7f7cf00b70c0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f7d08105110 tx=0x7f7cfc005f90 comp rx=0 tx=0).stop 2024-08-30T08:57:35.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:35.636+0000 7f7d0c93b640 1 -- 172.21.15.42:0/3232790366 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7d08102880 msgr2=0x7f7d08199230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:57:35.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:35.636+0000 7f7d0c93b640 1 --2- 172.21.15.42:0/3232790366 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7d08102880 0x7f7d08199230 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f7cf8005940 tx=0x7f7cf8004270 comp rx=0 tx=0).stop 2024-08-30T08:57:35.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:35.636+0000 7f7d0c93b640 1 -- 172.21.15.42:0/3232790366 shutdown_connections 2024-08-30T08:57:35.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:35.636+0000 7f7d0c93b640 1 --2- 172.21.15.42:0/3232790366 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f7cf00b4bb0 0x7f7cf00b70c0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:57:35.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:35.636+0000 7f7d0c93b640 1 --2- 172.21.15.42:0/3232790366 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7d081081c0 0x7f7d081997d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:57:35.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:35.636+0000 7f7d0c93b640 1 --2- 172.21.15.42:0/3232790366 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7d08108b10 0x7f7d0819ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:57:35.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:35.636+0000 7f7d0c93b640 1 --2- 172.21.15.42:0/3232790366 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7d08102880 0x7f7d08199230 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:57:35.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:35.636+0000 7f7d0c93b640 1 -- 172.21.15.42:0/3232790366 >> 172.21.15.42:0/3232790366 conn(0x7f7d08079510 msgr2=0x7f7d08106310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:57:35.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:35.636+0000 7f7d0c93b640 1 -- 172.21.15.42:0/3232790366 shutdown_connections 2024-08-30T08:57:35.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:35.636+0000 7f7d0c93b640 1 -- 172.21.15.42:0/3232790366 wait complete. 2024-08-30T08:57:35.686 INFO:journalctl@ceph.mds.f.smithi190.stdout:Aug 30 08:57:35 smithi190 podman[52099]: 2024-08-30 08:57:35.392387662 +0000 UTC m=+0.410288337 container start 5f43b96cf342144dd1d4da00172a5619c632f8d5e68b671b9d2cc22c21645ff3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mds-f, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, io.buildah.version=1.37.1, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-08-30T08:57:35.686 INFO:journalctl@ceph.mds.f.smithi190.stdout:Aug 30 08:57:35 smithi190 ceph-mds[52118]: -- 172.21.15.190:0/2051005017 <== mon.1 v2:172.21.15.190:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557a65671860 con 0x557a69817180 2024-08-30T08:57:35.686 INFO:journalctl@ceph.mds.f.smithi190.stdout:Aug 30 08:57:35 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mds-f[52114]: starting mds.f at 2024-08-30T08:57:35.686 INFO:journalctl@ceph.mds.f.smithi190.stdout:Aug 30 08:57:35 smithi190 bash[52099]: 5f43b96cf342144dd1d4da00172a5619c632f8d5e68b671b9d2cc22c21645ff3 2024-08-30T08:57:35.686 INFO:journalctl@ceph.mds.f.smithi190.stdout:Aug 30 08:57:35 smithi190 systemd[1]: Started Ceph mds.f for b0895c1a-66ac-11ef-bcd4-c7b262605968. 2024-08-30T08:57:36.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:36 smithi167 ceph-mon[22063]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-30T08:57:36.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:36 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:36.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:36 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:36.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:36 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:36.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:36 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-30T08:57:36.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:36 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-30T08:57:36.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:36 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:57:36.828 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:36 smithi167 ceph-mon[22063]: Deploying daemon mds.c on smithi190 2024-08-30T08:57:36.828 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:36 smithi167 ceph-mon[22063]: from='client.? 172.21.15.42:0/3232790366' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-30T08:57:36.828 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:36 smithi167 ceph-mon[22063]: mds.? [v2:172.21.15.190:6834/247905709,v1:172.21.15.190:6835/247905709] up:boot 2024-08-30T08:57:36.828 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:36 smithi167 ceph-mon[22063]: fsmap 6 up:standby 2024-08-30T08:57:36.828 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:36 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "f"} : dispatch 2024-08-30T08:57:36.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:36 smithi042 ceph-mon[18652]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-30T08:57:36.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:36 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:36.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:36 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:36.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:36 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:36.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:36 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-30T08:57:36.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:36 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-30T08:57:36.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:36 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:57:36.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:36 smithi042 ceph-mon[18652]: Deploying daemon mds.c on smithi190 2024-08-30T08:57:36.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:36 smithi042 ceph-mon[18652]: from='client.? 172.21.15.42:0/3232790366' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-30T08:57:36.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:36 smithi042 ceph-mon[18652]: mds.? [v2:172.21.15.190:6834/247905709,v1:172.21.15.190:6835/247905709] up:boot 2024-08-30T08:57:36.884 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:36 smithi042 ceph-mon[18652]: fsmap 6 up:standby 2024-08-30T08:57:36.884 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:36 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "f"} : dispatch 2024-08-30T08:57:36.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:36 smithi190 ceph-mon[22001]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-30T08:57:36.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:36 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:36.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:36 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:36.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:36 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:36.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:36 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-30T08:57:36.937 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:36 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-30T08:57:36.937 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:36 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:57:36.937 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:36 smithi190 ceph-mon[22001]: Deploying daemon mds.c on smithi190 2024-08-30T08:57:36.937 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:36 smithi190 ceph-mon[22001]: from='client.? 172.21.15.42:0/3232790366' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-30T08:57:36.937 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:36 smithi190 ceph-mon[22001]: mds.? [v2:172.21.15.190:6834/247905709,v1:172.21.15.190:6835/247905709] up:boot 2024-08-30T08:57:36.937 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:36 smithi190 ceph-mon[22001]: fsmap 6 up:standby 2024-08-30T08:57:36.937 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:36 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "f"} : dispatch 2024-08-30T08:57:37.651 INFO:journalctl@ceph.mds.c.smithi190.stdout:Aug 30 08:57:37 smithi190 systemd[1]: Starting Ceph mds.c for b0895c1a-66ac-11ef-bcd4-c7b262605968... 2024-08-30T08:57:37.934 INFO:journalctl@ceph.mds.c.smithi190.stdout:Aug 30 08:57:37 smithi190 podman[52399]: 2024-08-30 08:57:37.552275944 +0000 UTC m=+0.016400684 image pull 79f4cde5e36911a5dca995eeb98098c2c8aae81dd52a088ef2c867bc3363bad0 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61 2024-08-30T08:57:37.934 INFO:journalctl@ceph.mds.c.smithi190.stdout:Aug 30 08:57:37 smithi190 podman[52399]: 2024-08-30 08:57:37.650807843 +0000 UTC m=+0.114932573 container create 0526a2005b1e060a9394f473a6a6b82e598280c98eee9e48acbaaf1d75b28120 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mds-c, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.37.1, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-08-30T08:57:37.934 INFO:journalctl@ceph.mds.c.smithi190.stdout:Aug 30 08:57:37 smithi190 podman[52399]: 2024-08-30 08:57:37.896638643 +0000 UTC m=+0.360763373 container init 0526a2005b1e060a9394f473a6a6b82e598280c98eee9e48acbaaf1d75b28120 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mds-c, maintainer=Guillaume Abrioux , RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.build-date=20240731, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.37.1, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=) 2024-08-30T08:57:37.935 INFO:journalctl@ceph.mds.c.smithi190.stdout:Aug 30 08:57:37 smithi190 podman[52399]: 2024-08-30 08:57:37.899539415 +0000 UTC m=+0.363664153 container start 0526a2005b1e060a9394f473a6a6b82e598280c98eee9e48acbaaf1d75b28120 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mds-c, maintainer=Guillaume Abrioux , io.buildah.version=1.37.1, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611) 2024-08-30T08:57:38.186 INFO:journalctl@ceph.mds.c.smithi190.stdout:Aug 30 08:57:37 smithi190 ceph-mds[52417]: -- 172.21.15.190:0/2732621321 <== mon.1 v2:172.21.15.190:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f4ea50b860 con 0x55f4ee6b1a80 2024-08-30T08:57:38.186 INFO:journalctl@ceph.mds.c.smithi190.stdout:Aug 30 08:57:37 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mds-c[52413]: starting mds.c at 2024-08-30T08:57:38.186 INFO:journalctl@ceph.mds.c.smithi190.stdout:Aug 30 08:57:38 smithi190 bash[52399]: 0526a2005b1e060a9394f473a6a6b82e598280c98eee9e48acbaaf1d75b28120 2024-08-30T08:57:38.186 INFO:journalctl@ceph.mds.c.smithi190.stdout:Aug 30 08:57:38 smithi190 systemd[1]: Started Ceph mds.c for b0895c1a-66ac-11ef-bcd4-c7b262605968. 2024-08-30T08:57:38.535 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph fs dump --format=json 2024-08-30T08:57:38.766 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:38 smithi167 ceph-mon[22063]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-30T08:57:38.767 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:38 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:38.767 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:38 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:38.767 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:38 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:38.767 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:38 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-30T08:57:38.767 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:38 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-30T08:57:38.767 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:38 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:57:38.803 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T08:57:38.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:38 smithi042 ceph-mon[18652]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-30T08:57:38.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:38 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:38.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:38 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:38.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:38 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:38.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:38 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-30T08:57:38.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:38 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-30T08:57:38.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:38 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:57:38.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:38 smithi190 ceph-mon[22001]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-30T08:57:38.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:38 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:38.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:38 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:38.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:38 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:38.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:38 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-30T08:57:38.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:38 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-30T08:57:38.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:38 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:57:39.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:39.300+0000 7f3dce54e640 1 -- 172.21.15.42:0/3054295002 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3dc8103900 msgr2=0x7f3dc8103d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:57:39.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:39.300+0000 7f3dce54e640 1 --2- 172.21.15.42:0/3054295002 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3dc8103900 0x7f3dc8103d10 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f3db8009900 tx=0x7f3db802a410 comp rx=0 tx=0).stop 2024-08-30T08:57:39.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:39.300+0000 7f3dce54e640 1 -- 172.21.15.42:0/3054295002 shutdown_connections 2024-08-30T08:57:39.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:39.300+0000 7f3dce54e640 1 --2- 172.21.15.42:0/3054295002 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3dc8109240 0x7f3dc8109650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:57:39.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:39.300+0000 7f3dce54e640 1 --2- 172.21.15.42:0/3054295002 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3dc8109b90 0x7f3dc81103c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:57:39.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:39.300+0000 7f3dce54e640 1 --2- 172.21.15.42:0/3054295002 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3dc8103900 0x7f3dc8103d10 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:57:39.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:39.300+0000 7f3dce54e640 1 -- 172.21.15.42:0/3054295002 >> 172.21.15.42:0/3054295002 conn(0x7f3dc807b5f0 msgr2=0x7f3dc807ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:57:39.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:39.300+0000 7f3dce54e640 1 -- 172.21.15.42:0/3054295002 shutdown_connections 2024-08-30T08:57:39.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:39.300+0000 7f3dce54e640 1 -- 172.21.15.42:0/3054295002 wait complete. 2024-08-30T08:57:39.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:39.301+0000 7f3dce54e640 1 Processor -- start 2024-08-30T08:57:39.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:39.301+0000 7f3dce54e640 1 -- start start 2024-08-30T08:57:39.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:39.301+0000 7f3dce54e640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3dc8103900 0x7f3dc819a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:57:39.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:39.301+0000 7f3dce54e640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3dc8109240 0x7f3dc819a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:57:39.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:39.301+0000 7f3dce54e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3dc8109b90 0x7f3dc819fae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:57:39.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:39.301+0000 7f3dce54e640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f3dc8113850 con 0x7f3dc8103900 2024-08-30T08:57:39.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:39.301+0000 7f3dce54e640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f3dc81136d0 con 0x7f3dc8109240 2024-08-30T08:57:39.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:39.301+0000 7f3dce54e640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f3dc81139d0 con 0x7f3dc8109b90 2024-08-30T08:57:39.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:39.301+0000 7f3dc7fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3dc8103900 0x7f3dc819a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:57:39.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:39.301+0000 7f3dc7fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3dc8103900 0x7f3dc819a2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:44526/0 (socket says 172.21.15.42:44526) 2024-08-30T08:57:39.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:39.301+0000 7f3dc7fff640 1 -- 172.21.15.42:0/2144465453 learned_addr learned my addr 172.21.15.42:0/2144465453 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T08:57:39.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:39.301+0000 7f3dbffff640 1 --2- 172.21.15.42:0/2144465453 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3dc8109240 0x7f3dc819a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:57:39.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:39.302+0000 7f3dccaeb640 1 --2- 172.21.15.42:0/2144465453 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3dc8109b90 0x7f3dc819fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:57:39.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:39.302+0000 7f3dc7fff640 1 -- 172.21.15.42:0/2144465453 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3dc8109b90 msgr2=0x7f3dc819fae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:57:39.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:39.302+0000 7f3dc7fff640 1 --2- 172.21.15.42:0/2144465453 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3dc8109b90 0x7f3dc819fae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:57:39.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:39.302+0000 7f3dc7fff640 1 -- 172.21.15.42:0/2144465453 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3dc8109240 msgr2=0x7f3dc819a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:57:39.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:39.302+0000 7f3dc7fff640 1 --2- 172.21.15.42:0/2144465453 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3dc8109240 0x7f3dc819a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:57:39.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:39.302+0000 7f3dc7fff640 1 -- 172.21.15.42:0/2144465453 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3dc81a0210 con 0x7f3dc8103900 2024-08-30T08:57:39.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:39.302+0000 7f3dccaeb640 1 --2- 172.21.15.42:0/2144465453 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3dc8109b90 0x7f3dc819fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-30T08:57:39.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:39.302+0000 7f3dc7fff640 1 --2- 172.21.15.42:0/2144465453 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3dc8103900 0x7f3dc819a2a0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f3db8005940 tx=0x7f3db80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:57:39.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:39.302+0000 7f3dc5ffb640 1 -- 172.21.15.42:0/2144465453 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3db803f030 con 0x7f3dc8103900 2024-08-30T08:57:39.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:39.302+0000 7f3dce54e640 1 -- 172.21.15.42:0/2144465453 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3dc81a04a0 con 0x7f3dc8103900 2024-08-30T08:57:39.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:39.302+0000 7f3dc5ffb640 1 -- 172.21.15.42:0/2144465453 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3db8003cb0 con 0x7f3dc8103900 2024-08-30T08:57:39.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:39.302+0000 7f3dc5ffb640 1 -- 172.21.15.42:0/2144465453 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3db8035070 con 0x7f3dc8103900 2024-08-30T08:57:39.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:39.302+0000 7f3dce54e640 1 -- 172.21.15.42:0/2144465453 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3dc81a6c60 con 0x7f3dc8103900 2024-08-30T08:57:39.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:39.303+0000 7f3dce54e640 1 -- 172.21.15.42:0/2144465453 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3dc8103d10 con 0x7f3dc8103900 2024-08-30T08:57:39.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:39.305+0000 7f3dc5ffb640 1 -- 172.21.15.42:0/2144465453 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) ==== 152577+0+0 (secure 0 0 0) 0x7f3db8042030 con 0x7f3dc8103900 2024-08-30T08:57:39.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:39.305+0000 7f3dc5ffb640 1 --2- 172.21.15.42:0/2144465453 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f3db40b4b30 0x7f3db40b7040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:57:39.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:39.306+0000 7f3dbffff640 1 --2- 172.21.15.42:0/2144465453 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f3db40b4b30 0x7f3db40b7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:57:39.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:39.306+0000 7f3dc5ffb640 1 -- 172.21.15.42:0/2144465453 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 7127+0+0 (secure 0 0 0) 0x7f3db8105770 con 0x7f3dc8103900 2024-08-30T08:57:39.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:39.306+0000 7f3dbffff640 1 --2- 172.21.15.42:0/2144465453 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f3db40b4b30 0x7f3db40b7040 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f3dc8106260 tx=0x7f3db000a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:57:39.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:39.309+0000 7f3dc5ffb640 1 -- 172.21.15.42:0/2144465453 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f3db80ccd40 con 0x7f3dc8103900 2024-08-30T08:57:39.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:39.533+0000 7f3dce54e640 1 -- 172.21.15.42:0/2144465453 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3dc819b3d0 con 0x7f3dc8103900 2024-08-30T08:57:39.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:39.534+0000 7f3dc5ffb640 1 -- 172.21.15.42:0/2144465453 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6388 (secure 0 0 0) 0x7f3db80cc640 con 0x7f3dc8103900 2024-08-30T08:57:39.537 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 8 2024-08-30T08:57:39.537 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-08-30T08:57:39.537 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":8,"btime":"2024-08-30T08:57:38:549600+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14703,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/3188400000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":3188400000},{"type":"v1","addr":"172.21.15.42:6835","nonce":3188400000}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14715,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/3356537674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":3356537674},{"type":"v1","addr":"172.21.15.42:6837","nonce":3356537674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24553,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/543555570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":543555570},{"type":"v1","addr":"172.21.15.190:6833","nonce":543555570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24554,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/3907818941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":3907818941},{"type":"v1","addr":"172.21.15.167:6833","nonce":3907818941}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24572,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/234596845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":234596845},{"type":"v1","addr":"172.21.15.167:6835","nonce":234596845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24592,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/247905709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":247905709},{"type":"v1","addr":"172.21.15.190:6835","nonce":247905709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24598,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6837/863256349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6836","nonce":863256349},{"type":"v1","addr":"172.21.15.190:6837","nonce":863256349}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-08-30T08:57:39.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:39.536+0000 7f3dce54e640 1 -- 172.21.15.42:0/2144465453 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f3db40b4b30 msgr2=0x7f3db40b7040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:57:39.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:39.536+0000 7f3dce54e640 1 --2- 172.21.15.42:0/2144465453 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f3db40b4b30 0x7f3db40b7040 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f3dc8106260 tx=0x7f3db000a2d0 comp rx=0 tx=0).stop 2024-08-30T08:57:39.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:39.536+0000 7f3dce54e640 1 -- 172.21.15.42:0/2144465453 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3dc8103900 msgr2=0x7f3dc819a2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:57:39.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:39.536+0000 7f3dce54e640 1 --2- 172.21.15.42:0/2144465453 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3dc8103900 0x7f3dc819a2a0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f3db8005940 tx=0x7f3db80042f0 comp rx=0 tx=0).stop 2024-08-30T08:57:39.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:39.536+0000 7f3dce54e640 1 -- 172.21.15.42:0/2144465453 shutdown_connections 2024-08-30T08:57:39.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:39.536+0000 7f3dce54e640 1 --2- 172.21.15.42:0/2144465453 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f3db40b4b30 0x7f3db40b7040 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:57:39.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:39.536+0000 7f3dce54e640 1 --2- 172.21.15.42:0/2144465453 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3dc8109b90 0x7f3dc819fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:57:39.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:39.536+0000 7f3dce54e640 1 --2- 172.21.15.42:0/2144465453 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3dc8109240 0x7f3dc819a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:57:39.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:39.536+0000 7f3dce54e640 1 --2- 172.21.15.42:0/2144465453 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3dc8103900 0x7f3dc819a2a0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:57:39.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:39.536+0000 7f3dce54e640 1 -- 172.21.15.42:0/2144465453 >> 172.21.15.42:0/2144465453 conn(0x7f3dc807b5f0 msgr2=0x7f3dc8107380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:57:39.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:39.537+0000 7f3dce54e640 1 -- 172.21.15.42:0/2144465453 shutdown_connections 2024-08-30T08:57:39.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:39.537+0000 7f3dce54e640 1 -- 172.21.15.42:0/2144465453 wait complete. 2024-08-30T08:57:39.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:39 smithi167 ceph-mon[22063]: Deploying daemon mds.h on smithi167 2024-08-30T08:57:39.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:39 smithi167 ceph-mon[22063]: mds.? [v2:172.21.15.190:6836/863256349,v1:172.21.15.190:6837/863256349] up:boot 2024-08-30T08:57:39.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:39 smithi167 ceph-mon[22063]: fsmap 7 up:standby 2024-08-30T08:57:39.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:39 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-08-30T08:57:39.828 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:39 smithi167 ceph-mon[22063]: from='client.? 172.21.15.42:0/2144465453' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-30T08:57:39.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:39 smithi042 ceph-mon[18652]: Deploying daemon mds.h on smithi167 2024-08-30T08:57:39.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:39 smithi042 ceph-mon[18652]: mds.? [v2:172.21.15.190:6836/863256349,v1:172.21.15.190:6837/863256349] up:boot 2024-08-30T08:57:39.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:39 smithi042 ceph-mon[18652]: fsmap 7 up:standby 2024-08-30T08:57:39.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:39 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-08-30T08:57:39.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:39 smithi042 ceph-mon[18652]: from='client.? 172.21.15.42:0/2144465453' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-30T08:57:39.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:39 smithi190 ceph-mon[22001]: Deploying daemon mds.h on smithi167 2024-08-30T08:57:39.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:39 smithi190 ceph-mon[22001]: mds.? [v2:172.21.15.190:6836/863256349,v1:172.21.15.190:6837/863256349] up:boot 2024-08-30T08:57:39.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:39 smithi190 ceph-mon[22001]: fsmap 7 up:standby 2024-08-30T08:57:39.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:39 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-08-30T08:57:39.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:39 smithi190 ceph-mon[22001]: from='client.? 172.21.15.42:0/2144465453' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-30T08:57:40.327 INFO:journalctl@ceph.mds.h.smithi167.stdout:Aug 30 08:57:40 smithi167 systemd[1]: Starting Ceph mds.h for b0895c1a-66ac-11ef-bcd4-c7b262605968... 2024-08-30T08:57:40.619 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:40 smithi167 ceph-mon[22063]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-30T08:57:40.620 INFO:journalctl@ceph.mds.h.smithi167.stdout:Aug 30 08:57:40 smithi167 podman[52151]: 2024-08-30 08:57:40.235335011 +0000 UTC m=+0.015674352 image pull 79f4cde5e36911a5dca995eeb98098c2c8aae81dd52a088ef2c867bc3363bad0 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61 2024-08-30T08:57:40.620 INFO:journalctl@ceph.mds.h.smithi167.stdout:Aug 30 08:57:40 smithi167 podman[52151]: 2024-08-30 08:57:40.348692367 +0000 UTC m=+0.129031698 container create ed5d7ebb56704ae437e9c0753b464521c7c13d7634a7b62b914162a51742a356 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mds-h, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.37.1, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS) 2024-08-30T08:57:40.620 INFO:journalctl@ceph.mds.h.smithi167.stdout:Aug 30 08:57:40 smithi167 podman[52151]: 2024-08-30 08:57:40.618808268 +0000 UTC m=+0.399147598 container init ed5d7ebb56704ae437e9c0753b464521c7c13d7634a7b62b914162a51742a356 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mds-h, GIT_CLEAN=True, io.buildah.version=1.37.1, maintainer=Guillaume Abrioux , RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, ceph=True) 2024-08-30T08:57:40.632 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:40 smithi042 ceph-mon[18652]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-30T08:57:40.837 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:40 smithi190 ceph-mon[22001]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-30T08:57:41.077 INFO:journalctl@ceph.mds.h.smithi167.stdout:Aug 30 08:57:40 smithi167 podman[52151]: 2024-08-30 08:57:40.621736731 +0000 UTC m=+0.402076062 container start ed5d7ebb56704ae437e9c0753b464521c7c13d7634a7b62b914162a51742a356 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mds-h, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.37.1, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-08-30T08:57:41.077 INFO:journalctl@ceph.mds.h.smithi167.stdout:Aug 30 08:57:40 smithi167 ceph-mds[52171]: -- 172.21.15.167:0/3274835064 <== mon.2 v2:172.21.15.167:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x564dd3253860 con 0x564dd73f9180 2024-08-30T08:57:41.077 INFO:journalctl@ceph.mds.h.smithi167.stdout:Aug 30 08:57:40 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mds-h[52167]: starting mds.h at 2024-08-30T08:57:41.077 INFO:journalctl@ceph.mds.h.smithi167.stdout:Aug 30 08:57:40 smithi167 bash[52151]: ed5d7ebb56704ae437e9c0753b464521c7c13d7634a7b62b914162a51742a356 2024-08-30T08:57:41.077 INFO:journalctl@ceph.mds.h.smithi167.stdout:Aug 30 08:57:40 smithi167 systemd[1]: Started Ceph mds.h for b0895c1a-66ac-11ef-bcd4-c7b262605968. 2024-08-30T08:57:42.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:41 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:42.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:41 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:42.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:41 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:42.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:41 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-30T08:57:42.078 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:41 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-30T08:57:42.078 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:41 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:57:42.078 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:41 smithi167 ceph-mon[22063]: Deploying daemon mds.i on smithi190 2024-08-30T08:57:42.078 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:41 smithi167 ceph-mon[22063]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-30T08:57:42.078 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:41 smithi167 ceph-mon[22063]: mds.? [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] up:boot 2024-08-30T08:57:42.078 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:41 smithi167 ceph-mon[22063]: fsmap 8 up:standby 2024-08-30T08:57:42.078 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:41 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-08-30T08:57:42.086 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:41 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:42.086 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:41 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:42.086 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:41 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:42.086 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:41 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-30T08:57:42.086 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:41 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-30T08:57:42.087 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:41 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:57:42.087 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:41 smithi190 ceph-mon[22001]: Deploying daemon mds.i on smithi190 2024-08-30T08:57:42.087 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:41 smithi190 ceph-mon[22001]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-30T08:57:42.087 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:41 smithi190 ceph-mon[22001]: mds.? [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] up:boot 2024-08-30T08:57:42.087 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:41 smithi190 ceph-mon[22001]: fsmap 8 up:standby 2024-08-30T08:57:42.087 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:41 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-08-30T08:57:42.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:41 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:42.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:41 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:42.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:41 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:42.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:41 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-30T08:57:42.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:41 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-30T08:57:42.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:41 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:57:42.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:41 smithi042 ceph-mon[18652]: Deploying daemon mds.i on smithi190 2024-08-30T08:57:42.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:41 smithi042 ceph-mon[18652]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-30T08:57:42.134 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:41 smithi042 ceph-mon[18652]: mds.? [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] up:boot 2024-08-30T08:57:42.134 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:41 smithi042 ceph-mon[18652]: fsmap 8 up:standby 2024-08-30T08:57:42.134 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:41 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-08-30T08:57:42.350 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph fs dump --format=json 2024-08-30T08:57:42.615 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T08:57:43.032 INFO:journalctl@ceph.mds.i.smithi190.stdout:Aug 30 08:57:42 smithi190 podman[52713]: 2024-08-30 08:57:42.862645837 +0000 UTC m=+0.015878106 image pull 79f4cde5e36911a5dca995eeb98098c2c8aae81dd52a088ef2c867bc3363bad0 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61 2024-08-30T08:57:43.032 INFO:journalctl@ceph.mds.i.smithi190.stdout:Aug 30 08:57:43 smithi190 podman[52713]: 2024-08-30 08:57:43.031174544 +0000 UTC m=+0.184406847 container create 0987ddbec8abf03313f1a21b63efee1dc3876668467712f47854ea96231ded68 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mds-i, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, io.buildah.version=1.37.1, GIT_CLEAN=True) 2024-08-30T08:57:43.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:43.118+0000 7f35c2461640 1 -- 172.21.15.42:0/1579799730 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f35bc104f70 msgr2=0x7f35bc105380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:57:43.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:43.118+0000 7f35c2461640 1 --2- 172.21.15.42:0/1579799730 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f35bc104f70 0x7f35bc105380 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f35a4009900 tx=0x7f35a402a410 comp rx=0 tx=0).stop 2024-08-30T08:57:43.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:43.118+0000 7f35c2461640 1 -- 172.21.15.42:0/1579799730 shutdown_connections 2024-08-30T08:57:43.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:43.118+0000 7f35c2461640 1 --2- 172.21.15.42:0/1579799730 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f35bc1058c0 0x7f35bc10f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:57:43.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:43.118+0000 7f35c2461640 1 --2- 172.21.15.42:0/1579799730 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f35bc104f70 0x7f35bc105380 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:57:43.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:43.118+0000 7f35c2461640 1 --2- 172.21.15.42:0/1579799730 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f35bc102ef0 0x7f35bc103300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:57:43.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:43.118+0000 7f35c2461640 1 -- 172.21.15.42:0/1579799730 >> 172.21.15.42:0/1579799730 conn(0x7f35bc079d30 msgr2=0x7f35bc07a140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:57:43.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:43.118+0000 7f35c2461640 1 -- 172.21.15.42:0/1579799730 shutdown_connections 2024-08-30T08:57:43.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:43.118+0000 7f35c2461640 1 -- 172.21.15.42:0/1579799730 wait complete. 2024-08-30T08:57:43.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:43.118+0000 7f35c2461640 1 Processor -- start 2024-08-30T08:57:43.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:43.118+0000 7f35c2461640 1 -- start start 2024-08-30T08:57:43.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:43.119+0000 7f35c2461640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f35bc102ef0 0x7f35bc19b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:57:43.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:43.119+0000 7f35c2461640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f35bc104f70 0x7f35bc19b630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:57:43.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:43.119+0000 7f35c2461640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f35bc1058c0 0x7f35bc19bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:57:43.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:43.119+0000 7f35c2461640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f35bc112760 con 0x7f35bc102ef0 2024-08-30T08:57:43.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:43.119+0000 7f35c2461640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f35bc1125e0 con 0x7f35bc104f70 2024-08-30T08:57:43.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:43.119+0000 7f35c2461640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f35bc1128e0 con 0x7f35bc1058c0 2024-08-30T08:57:43.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:43.119+0000 7f35bbfff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f35bc102ef0 0x7f35bc19b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:57:43.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:43.119+0000 7f35bbfff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f35bc102ef0 0x7f35bc19b0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:44546/0 (socket says 172.21.15.42:44546) 2024-08-30T08:57:43.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:43.119+0000 7f35bbfff640 1 -- 172.21.15.42:0/980636749 learned_addr learned my addr 172.21.15.42:0/980636749 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T08:57:43.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:43.119+0000 7f35c09fe640 1 --2- 172.21.15.42:0/980636749 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f35bc1058c0 0x7f35bc19bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:57:43.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:43.119+0000 7f35b3fff640 1 --2- 172.21.15.42:0/980636749 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f35bc104f70 0x7f35bc19b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:57:43.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:43.119+0000 7f35bbfff640 1 -- 172.21.15.42:0/980636749 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f35bc1058c0 msgr2=0x7f35bc19bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:57:43.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:43.119+0000 7f35bbfff640 1 --2- 172.21.15.42:0/980636749 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f35bc1058c0 0x7f35bc19bb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:57:43.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:43.119+0000 7f35bbfff640 1 -- 172.21.15.42:0/980636749 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f35bc104f70 msgr2=0x7f35bc19b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:57:43.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:43.119+0000 7f35bbfff640 1 --2- 172.21.15.42:0/980636749 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f35bc104f70 0x7f35bc19b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:57:43.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:43.119+0000 7f35bbfff640 1 -- 172.21.15.42:0/980636749 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35bc1a6380 con 0x7f35bc102ef0 2024-08-30T08:57:43.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:43.119+0000 7f35bbfff640 1 --2- 172.21.15.42:0/980636749 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f35bc102ef0 0x7f35bc19b0f0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f35ac00c970 tx=0x7f35ac00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:57:43.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:43.119+0000 7f35b9ffb640 1 -- 172.21.15.42:0/980636749 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35ac004220 con 0x7f35bc102ef0 2024-08-30T08:57:43.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:43.120+0000 7f35b9ffb640 1 -- 172.21.15.42:0/980636749 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f35ac0043c0 con 0x7f35bc102ef0 2024-08-30T08:57:43.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:43.120+0000 7f35c2461640 1 -- 172.21.15.42:0/980636749 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35bc1a6670 con 0x7f35bc102ef0 2024-08-30T08:57:43.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:43.120+0000 7f35b9ffb640 1 -- 172.21.15.42:0/980636749 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35ac00f450 con 0x7f35bc102ef0 2024-08-30T08:57:43.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:43.120+0000 7f35c2461640 1 -- 172.21.15.42:0/980636749 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35bc1a6bd0 con 0x7f35bc102ef0 2024-08-30T08:57:43.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:43.121+0000 7f35c2461640 1 -- 172.21.15.42:0/980636749 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35bc103300 con 0x7f35bc102ef0 2024-08-30T08:57:43.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:43.122+0000 7f35b9ffb640 1 -- 172.21.15.42:0/980636749 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) ==== 152577+0+0 (secure 0 0 0) 0x7f35ac003610 con 0x7f35bc102ef0 2024-08-30T08:57:43.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:43.123+0000 7f35b9ffb640 1 --2- 172.21.15.42:0/980636749 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f35a80b4be0 0x7f35a80b70f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:57:43.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:43.123+0000 7f35b3fff640 1 --2- 172.21.15.42:0/980636749 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f35a80b4be0 0x7f35a80b70f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:57:43.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:43.123+0000 7f35b9ffb640 1 -- 172.21.15.42:0/980636749 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 7127+0+0 (secure 0 0 0) 0x7f35ac0e4cf0 con 0x7f35bc102ef0 2024-08-30T08:57:43.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:43.123+0000 7f35b3fff640 1 --2- 172.21.15.42:0/980636749 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f35a80b4be0 0x7f35a80b70f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f35a4005940 tx=0x7f35a40058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:57:43.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:43.126+0000 7f35b9ffb640 1 -- 172.21.15.42:0/980636749 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f35ac014030 con 0x7f35bc102ef0 2024-08-30T08:57:43.305 INFO:journalctl@ceph.mds.i.smithi190.stdout:Aug 30 08:57:43 smithi190 podman[52713]: 2024-08-30 08:57:43.267370228 +0000 UTC m=+0.420602485 container init 0987ddbec8abf03313f1a21b63efee1dc3876668467712f47854ea96231ded68 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mds-i, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.1, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731) 2024-08-30T08:57:43.305 INFO:journalctl@ceph.mds.i.smithi190.stdout:Aug 30 08:57:43 smithi190 podman[52713]: 2024-08-30 08:57:43.27021595 +0000 UTC m=+0.423448216 container start 0987ddbec8abf03313f1a21b63efee1dc3876668467712f47854ea96231ded68 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mds-i, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.37.1, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=) 2024-08-30T08:57:43.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:43.348+0000 7f35c2461640 1 -- 172.21.15.42:0/980636749 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f35bc101f60 con 0x7f35bc102ef0 2024-08-30T08:57:43.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:43.349+0000 7f35b9ffb640 1 -- 172.21.15.42:0/980636749 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7216 (secure 0 0 0) 0x7f35ac0abcf0 con 0x7f35bc102ef0 2024-08-30T08:57:43.352 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-08-30T08:57:43.352 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":9,"btime":"2024-08-30T08:57:41:564488+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14703,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/3188400000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":3188400000},{"type":"v1","addr":"172.21.15.42:6835","nonce":3188400000}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14715,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/3356537674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":3356537674},{"type":"v1","addr":"172.21.15.42:6837","nonce":3356537674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24553,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/543555570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":543555570},{"type":"v1","addr":"172.21.15.190:6833","nonce":543555570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24554,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/3907818941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":3907818941},{"type":"v1","addr":"172.21.15.167:6833","nonce":3907818941}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24572,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/234596845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":234596845},{"type":"v1","addr":"172.21.15.167:6835","nonce":234596845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24592,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/247905709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":247905709},{"type":"v1","addr":"172.21.15.190:6835","nonce":247905709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24598,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6837/863256349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6836","nonce":863256349},{"type":"v1","addr":"172.21.15.190:6837","nonce":863256349}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24599,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/3011837856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":3011837856},{"type":"v1","addr":"172.21.15.167:6837","nonce":3011837856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-08-30T08:57:43.352 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 9 2024-08-30T08:57:43.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:43.351+0000 7f35c2461640 1 -- 172.21.15.42:0/980636749 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f35a80b4be0 msgr2=0x7f35a80b70f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:57:43.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:43.351+0000 7f35c2461640 1 --2- 172.21.15.42:0/980636749 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f35a80b4be0 0x7f35a80b70f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f35a4005940 tx=0x7f35a40058d0 comp rx=0 tx=0).stop 2024-08-30T08:57:43.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:43.351+0000 7f35c2461640 1 -- 172.21.15.42:0/980636749 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f35bc102ef0 msgr2=0x7f35bc19b0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:57:43.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:43.351+0000 7f35c2461640 1 --2- 172.21.15.42:0/980636749 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f35bc102ef0 0x7f35bc19b0f0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f35ac00c970 tx=0x7f35ac00ce30 comp rx=0 tx=0).stop 2024-08-30T08:57:43.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:43.351+0000 7f35c2461640 1 -- 172.21.15.42:0/980636749 shutdown_connections 2024-08-30T08:57:43.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:43.351+0000 7f35c2461640 1 --2- 172.21.15.42:0/980636749 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f35a80b4be0 0x7f35a80b70f0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:57:43.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:43.351+0000 7f35c2461640 1 --2- 172.21.15.42:0/980636749 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f35bc1058c0 0x7f35bc19bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:57:43.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:43.351+0000 7f35c2461640 1 --2- 172.21.15.42:0/980636749 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f35bc104f70 0x7f35bc19b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:57:43.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:43.351+0000 7f35c2461640 1 --2- 172.21.15.42:0/980636749 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f35bc102ef0 0x7f35bc19b0f0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:57:43.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:43.351+0000 7f35c2461640 1 -- 172.21.15.42:0/980636749 >> 172.21.15.42:0/980636749 conn(0x7f35bc079d30 msgr2=0x7f35bc106240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:57:43.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:43.351+0000 7f35c2461640 1 -- 172.21.15.42:0/980636749 shutdown_connections 2024-08-30T08:57:43.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:43.351+0000 7f35c2461640 1 -- 172.21.15.42:0/980636749 wait complete. 2024-08-30T08:57:43.686 INFO:journalctl@ceph.mds.i.smithi190.stdout:Aug 30 08:57:43 smithi190 ceph-mds[52732]: -- 172.21.15.190:0/1053057103 <== mon.1 v2:172.21.15.190:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f3c906f860 con 0x55f3cd215180 2024-08-30T08:57:43.686 INFO:journalctl@ceph.mds.i.smithi190.stdout:Aug 30 08:57:43 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mds-i[52728]: starting mds.i at 2024-08-30T08:57:43.686 INFO:journalctl@ceph.mds.i.smithi190.stdout:Aug 30 08:57:43 smithi190 bash[52713]: 0987ddbec8abf03313f1a21b63efee1dc3876668467712f47854ea96231ded68 2024-08-30T08:57:43.686 INFO:journalctl@ceph.mds.i.smithi190.stdout:Aug 30 08:57:43 smithi190 systemd[1]: Started Ceph mds.i for b0895c1a-66ac-11ef-bcd4-c7b262605968. 2024-08-30T08:57:44.050 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:43 smithi042 ceph-mon[18652]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-30T08:57:44.051 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:43 smithi042 ceph-mon[18652]: from='client.? 172.21.15.42:0/980636749' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-30T08:57:44.051 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:43 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:44.051 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:43 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:44.051 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:43 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:44.051 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:43 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-30T08:57:44.051 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:43 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-30T08:57:44.051 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:43 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:57:44.051 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:43 smithi042 ceph-mon[18652]: Deploying daemon mds.g on smithi042 2024-08-30T08:57:44.280 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:43 smithi167 ceph-mon[22063]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-30T08:57:44.280 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:43 smithi167 ceph-mon[22063]: from='client.? 172.21.15.42:0/980636749' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-30T08:57:44.281 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:43 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:44.281 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:43 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:44.281 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:43 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:44.281 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:43 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-30T08:57:44.281 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:43 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-30T08:57:44.281 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:43 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:57:44.281 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:43 smithi167 ceph-mon[22063]: Deploying daemon mds.g on smithi042 2024-08-30T08:57:44.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:43 smithi190 ceph-mon[22001]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-30T08:57:44.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:43 smithi190 ceph-mon[22001]: from='client.? 172.21.15.42:0/980636749' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-30T08:57:44.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:43 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:44.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:43 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:44.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:43 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:44.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:43 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-30T08:57:44.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:43 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-30T08:57:44.437 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:43 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:57:44.437 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:43 smithi190 ceph-mon[22001]: Deploying daemon mds.g on smithi042 2024-08-30T08:57:45.309 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:44 smithi042 ceph-mon[18652]: mds.? [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] up:boot 2024-08-30T08:57:45.309 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:44 smithi042 ceph-mon[18652]: fsmap 9 up:standby 2024-08-30T08:57:45.309 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:44 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2024-08-30T08:57:45.328 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:44 smithi167 ceph-mon[22063]: mds.? [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] up:boot 2024-08-30T08:57:45.329 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:44 smithi167 ceph-mon[22063]: fsmap 9 up:standby 2024-08-30T08:57:45.329 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:44 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2024-08-30T08:57:45.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:44 smithi190 ceph-mon[22001]: mds.? [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] up:boot 2024-08-30T08:57:45.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:44 smithi190 ceph-mon[22001]: fsmap 9 up:standby 2024-08-30T08:57:45.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:44 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2024-08-30T08:57:45.883 INFO:journalctl@ceph.mds.g.smithi042.stdout:Aug 30 08:57:45 smithi042 systemd[1]: Starting Ceph mds.g for b0895c1a-66ac-11ef-bcd4-c7b262605968... 2024-08-30T08:57:46.080 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph fs dump --format=json 2024-08-30T08:57:46.264 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:45 smithi042 ceph-mon[18652]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-30T08:57:46.265 INFO:journalctl@ceph.mds.g.smithi042.stdout:Aug 30 08:57:45 smithi042 podman[61794]: 2024-08-30 08:57:45.873695915 +0000 UTC m=+0.015739701 image pull 79f4cde5e36911a5dca995eeb98098c2c8aae81dd52a088ef2c867bc3363bad0 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61 2024-08-30T08:57:46.265 INFO:journalctl@ceph.mds.g.smithi042.stdout:Aug 30 08:57:45 smithi042 podman[61794]: 2024-08-30 08:57:45.985102828 +0000 UTC m=+0.127146610 container create 83975ccb3d1e3282f93ea03c7651343123e1348dcbd40668c9e49fb7cafa336f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mds-g, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.1, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-08-30T08:57:46.265 INFO:journalctl@ceph.mds.g.smithi042.stdout:Aug 30 08:57:46 smithi042 podman[61794]: 2024-08-30 08:57:46.263428391 +0000 UTC m=+0.405472168 container init 83975ccb3d1e3282f93ea03c7651343123e1348dcbd40668c9e49fb7cafa336f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mds-g, io.buildah.version=1.37.1, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_CLEAN=True, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-08-30T08:57:46.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:45 smithi167 ceph-mon[22063]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-30T08:57:46.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:45 smithi190 ceph-mon[22001]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-30T08:57:46.445 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T08:57:46.569 INFO:journalctl@ceph.mds.g.smithi042.stdout:Aug 30 08:57:46 smithi042 podman[61794]: 2024-08-30 08:57:46.267696 +0000 UTC m=+0.409739783 container start 83975ccb3d1e3282f93ea03c7651343123e1348dcbd40668c9e49fb7cafa336f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mds-g, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.1, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731) 2024-08-30T08:57:46.569 INFO:journalctl@ceph.mds.g.smithi042.stdout:Aug 30 08:57:46 smithi042 ceph-mds[61840]: -- 172.21.15.42:0/1191408197 <== mon.0 v2:172.21.15.42:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x564b5f6b9c20 con 0x564b6385f600 2024-08-30T08:57:46.569 INFO:journalctl@ceph.mds.g.smithi042.stdout:Aug 30 08:57:46 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mds-g[61825]: starting mds.g at 2024-08-30T08:57:46.569 INFO:journalctl@ceph.mds.g.smithi042.stdout:Aug 30 08:57:46 smithi042 bash[61794]: 83975ccb3d1e3282f93ea03c7651343123e1348dcbd40668c9e49fb7cafa336f 2024-08-30T08:57:46.569 INFO:journalctl@ceph.mds.g.smithi042.stdout:Aug 30 08:57:46 smithi042 systemd[1]: Started Ceph mds.g for b0895c1a-66ac-11ef-bcd4-c7b262605968. 2024-08-30T08:57:46.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:46.917+0000 7f287e2b3640 1 -- 172.21.15.42:0/1675082551 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2878105070 msgr2=0x7f287810d6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:57:46.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:46.917+0000 7f287e2b3640 1 --2- 172.21.15.42:0/1675082551 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2878105070 0x7f287810d6b0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f286c009900 tx=0x7f286c02a410 comp rx=0 tx=0).stop 2024-08-30T08:57:46.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:46.917+0000 7f287e2b3640 1 -- 172.21.15.42:0/1675082551 shutdown_connections 2024-08-30T08:57:46.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:46.917+0000 7f287e2b3640 1 --2- 172.21.15.42:0/1675082551 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f287810dbf0 0x7f28781100d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:57:46.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:46.917+0000 7f287e2b3640 1 --2- 172.21.15.42:0/1675082551 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2878105070 0x7f287810d6b0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:57:46.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:46.917+0000 7f287e2b3640 1 --2- 172.21.15.42:0/1675082551 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2878104640 0x7f2878104a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:57:46.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:46.917+0000 7f287e2b3640 1 -- 172.21.15.42:0/1675082551 >> 172.21.15.42:0/1675082551 conn(0x7f2878079500 msgr2=0x7f2878079910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:57:46.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:46.917+0000 7f287e2b3640 1 -- 172.21.15.42:0/1675082551 shutdown_connections 2024-08-30T08:57:46.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:46.917+0000 7f287e2b3640 1 -- 172.21.15.42:0/1675082551 wait complete. 2024-08-30T08:57:46.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:46.918+0000 7f287e2b3640 1 Processor -- start 2024-08-30T08:57:46.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:46.918+0000 7f287e2b3640 1 -- start start 2024-08-30T08:57:46.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:46.918+0000 7f287e2b3640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2878104640 0x7f287819a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:57:46.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:46.918+0000 7f287e2b3640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2878105070 0x7f287819a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:57:46.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:46.918+0000 7f287e2b3640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f287810dbf0 0x7f287819fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:57:46.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:46.918+0000 7f287e2b3640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f2878113880 con 0x7f2878104640 2024-08-30T08:57:46.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:46.918+0000 7f287e2b3640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f2878113700 con 0x7f2878105070 2024-08-30T08:57:46.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:46.918+0000 7f287e2b3640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f2878113a00 con 0x7f287810dbf0 2024-08-30T08:57:46.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:46.918+0000 7f2877fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2878104640 0x7f287819a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:57:46.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:46.918+0000 7f2877fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2878104640 0x7f287819a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:44582/0 (socket says 172.21.15.42:44582) 2024-08-30T08:57:46.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:46.918+0000 7f2877fff640 1 -- 172.21.15.42:0/681940621 learned_addr learned my addr 172.21.15.42:0/681940621 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T08:57:46.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:46.919+0000 7f28777fe640 1 --2- 172.21.15.42:0/681940621 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2878105070 0x7f287819a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:57:46.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:46.919+0000 7f287c850640 1 --2- 172.21.15.42:0/681940621 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f287810dbf0 0x7f287819fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:57:46.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:46.919+0000 7f2877fff640 1 -- 172.21.15.42:0/681940621 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f287810dbf0 msgr2=0x7f287819fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:57:46.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:46.919+0000 7f2877fff640 1 --2- 172.21.15.42:0/681940621 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f287810dbf0 0x7f287819fb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:57:46.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:46.919+0000 7f2877fff640 1 -- 172.21.15.42:0/681940621 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2878105070 msgr2=0x7f287819a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:57:46.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:46.919+0000 7f2877fff640 1 --2- 172.21.15.42:0/681940621 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2878105070 0x7f287819a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:57:46.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:46.919+0000 7f2877fff640 1 -- 172.21.15.42:0/681940621 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28781a0260 con 0x7f2878104640 2024-08-30T08:57:46.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:46.919+0000 7f2877fff640 1 --2- 172.21.15.42:0/681940621 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2878104640 0x7f287819a2f0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f286800c970 tx=0x7f286800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:57:46.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:46.919+0000 7f28757fa640 1 -- 172.21.15.42:0/681940621 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2868004220 con 0x7f2878104640 2024-08-30T08:57:46.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:46.919+0000 7f28757fa640 1 -- 172.21.15.42:0/681940621 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f28680043c0 con 0x7f2878104640 2024-08-30T08:57:46.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:46.919+0000 7f287e2b3640 1 -- 172.21.15.42:0/681940621 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28781a0550 con 0x7f2878104640 2024-08-30T08:57:46.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:46.919+0000 7f28757fa640 1 -- 172.21.15.42:0/681940621 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f286800f450 con 0x7f2878104640 2024-08-30T08:57:46.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:46.919+0000 7f287e2b3640 1 -- 172.21.15.42:0/681940621 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28781a6cb0 con 0x7f2878104640 2024-08-30T08:57:46.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:46.920+0000 7f287e2b3640 1 -- 172.21.15.42:0/681940621 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f287806bf50 con 0x7f2878104640 2024-08-30T08:57:46.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:46.922+0000 7f28757fa640 1 -- 172.21.15.42:0/681940621 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) ==== 152577+0+0 (secure 0 0 0) 0x7f2868003610 con 0x7f2878104640 2024-08-30T08:57:46.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:46.922+0000 7f28757fa640 1 --2- 172.21.15.42:0/681940621 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f28600b4b30 0x7f28600b7040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:57:46.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:46.923+0000 7f28777fe640 1 --2- 172.21.15.42:0/681940621 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f28600b4b30 0x7f28600b7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:57:46.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:46.923+0000 7f28757fa640 1 -- 172.21.15.42:0/681940621 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 7127+0+0 (secure 0 0 0) 0x7f28680e4fc0 con 0x7f2878104640 2024-08-30T08:57:46.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:46.923+0000 7f28777fe640 1 --2- 172.21.15.42:0/681940621 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f28600b4b30 0x7f28600b7040 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f286c009230 tx=0x7f286c031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:57:46.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:46.926+0000 7f28757fa640 1 -- 172.21.15.42:0/681940621 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f2868014030 con 0x7f2878104640 2024-08-30T08:57:47.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:47.169+0000 7f287e2b3640 1 -- 172.21.15.42:0/681940621 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2878104a50 con 0x7f2878104640 2024-08-30T08:57:47.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:47.170+0000 7f28757fa640 1 -- 172.21.15.42:0/681940621 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8869 (secure 0 0 0) 0x7f28680ac4f0 con 0x7f2878104640 2024-08-30T08:57:47.172 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-08-30T08:57:47.172 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":11,"btime":"2024-08-30T08:57:46:981489+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14703,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/3188400000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":3188400000},{"type":"v1","addr":"172.21.15.42:6835","nonce":3188400000}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14715,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/3356537674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":3356537674},{"type":"v1","addr":"172.21.15.42:6837","nonce":3356537674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14766,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6839/2107785513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6838","nonce":2107785513},{"type":"v1","addr":"172.21.15.42:6839","nonce":2107785513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24553,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/543555570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":543555570},{"type":"v1","addr":"172.21.15.190:6833","nonce":543555570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24554,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/3907818941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":3907818941},{"type":"v1","addr":"172.21.15.167:6833","nonce":3907818941}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24572,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/234596845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":234596845},{"type":"v1","addr":"172.21.15.167:6835","nonce":234596845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24592,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/247905709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":247905709},{"type":"v1","addr":"172.21.15.190:6835","nonce":247905709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24598,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6837/863256349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6836","nonce":863256349},{"type":"v1","addr":"172.21.15.190:6837","nonce":863256349}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24599,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/3011837856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":3011837856},{"type":"v1","addr":"172.21.15.167:6837","nonce":3011837856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24610,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6839/719981110","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6838","nonce":719981110},{"type":"v1","addr":"172.21.15.190:6839","nonce":719981110}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-08-30T08:57:47.172 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 11 2024-08-30T08:57:47.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:47.172+0000 7f287e2b3640 1 -- 172.21.15.42:0/681940621 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f28600b4b30 msgr2=0x7f28600b7040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:57:47.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:47.172+0000 7f287e2b3640 1 --2- 172.21.15.42:0/681940621 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f28600b4b30 0x7f28600b7040 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f286c009230 tx=0x7f286c031400 comp rx=0 tx=0).stop 2024-08-30T08:57:47.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:47.172+0000 7f287e2b3640 1 -- 172.21.15.42:0/681940621 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2878104640 msgr2=0x7f287819a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:57:47.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:47.172+0000 7f287e2b3640 1 --2- 172.21.15.42:0/681940621 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2878104640 0x7f287819a2f0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f286800c970 tx=0x7f286800ce30 comp rx=0 tx=0).stop 2024-08-30T08:57:47.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:47.172+0000 7f287e2b3640 1 -- 172.21.15.42:0/681940621 shutdown_connections 2024-08-30T08:57:47.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:47.172+0000 7f287e2b3640 1 --2- 172.21.15.42:0/681940621 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f28600b4b30 0x7f28600b7040 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:57:47.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:47.172+0000 7f287e2b3640 1 --2- 172.21.15.42:0/681940621 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f287810dbf0 0x7f287819fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:57:47.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:47.172+0000 7f287e2b3640 1 --2- 172.21.15.42:0/681940621 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2878105070 0x7f287819a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:57:47.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:47.172+0000 7f287e2b3640 1 --2- 172.21.15.42:0/681940621 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2878104640 0x7f287819a2f0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:57:47.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:47.172+0000 7f287e2b3640 1 -- 172.21.15.42:0/681940621 >> 172.21.15.42:0/681940621 conn(0x7f2878079500 msgr2=0x7f2878100690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:57:47.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:47.172+0000 7f287e2b3640 1 -- 172.21.15.42:0/681940621 shutdown_connections 2024-08-30T08:57:47.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:47.172+0000 7f287e2b3640 1 -- 172.21.15.42:0/681940621 wait complete. 2024-08-30T08:57:47.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:47 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:47.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:47 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:47.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:47 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:47.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:47 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-30T08:57:47.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:47 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-30T08:57:47.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:47 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:57:47.687 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:47 smithi190 ceph-mon[22001]: Deploying daemon mds.a on smithi042 2024-08-30T08:57:47.687 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:47 smithi190 ceph-mon[22001]: mds.? [v2:172.21.15.42:6838/2107785513,v1:172.21.15.42:6839/2107785513] up:boot 2024-08-30T08:57:47.687 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:47 smithi190 ceph-mon[22001]: fsmap 10 up:standby 2024-08-30T08:57:47.687 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:47 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "g"} : dispatch 2024-08-30T08:57:47.687 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:47 smithi190 ceph-mon[22001]: from='client.? 172.21.15.42:0/681940621' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-30T08:57:47.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:47 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:47.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:47 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:47.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:47 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:47.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:47 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-30T08:57:47.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:47 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-30T08:57:47.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:47 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:57:47.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:47 smithi167 ceph-mon[22063]: Deploying daemon mds.a on smithi042 2024-08-30T08:57:47.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:47 smithi167 ceph-mon[22063]: mds.? [v2:172.21.15.42:6838/2107785513,v1:172.21.15.42:6839/2107785513] up:boot 2024-08-30T08:57:47.828 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:47 smithi167 ceph-mon[22063]: fsmap 10 up:standby 2024-08-30T08:57:47.828 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:47 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "g"} : dispatch 2024-08-30T08:57:47.828 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:47 smithi167 ceph-mon[22063]: from='client.? 172.21.15.42:0/681940621' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-30T08:57:47.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:47 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:47.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:47 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:47.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:47 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:47.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:47 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-30T08:57:47.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:47 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-30T08:57:47.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:47 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:57:47.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:47 smithi042 ceph-mon[18652]: Deploying daemon mds.a on smithi042 2024-08-30T08:57:47.884 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:47 smithi042 ceph-mon[18652]: mds.? [v2:172.21.15.42:6838/2107785513,v1:172.21.15.42:6839/2107785513] up:boot 2024-08-30T08:57:47.884 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:47 smithi042 ceph-mon[18652]: fsmap 10 up:standby 2024-08-30T08:57:47.884 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:47 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "g"} : dispatch 2024-08-30T08:57:47.884 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:47 smithi042 ceph-mon[18652]: from='client.? 172.21.15.42:0/681940621' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-30T08:57:48.583 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:48 smithi042 ceph-mon[18652]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-30T08:57:48.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:48 smithi167 ceph-mon[22063]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-30T08:57:48.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:48 smithi190 ceph-mon[22001]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-30T08:57:49.602 INFO:journalctl@ceph.mds.a.smithi042.stdout:Aug 30 08:57:49 smithi042 systemd[1]: Starting Ceph mds.a for b0895c1a-66ac-11ef-bcd4-c7b262605968... 2024-08-30T08:57:49.872 INFO:journalctl@ceph.mds.a.smithi042.stdout:Aug 30 08:57:49 smithi042 podman[62208]: 2024-08-30 08:57:49.503469549 +0000 UTC m=+0.016132951 image pull 79f4cde5e36911a5dca995eeb98098c2c8aae81dd52a088ef2c867bc3363bad0 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61 2024-08-30T08:57:49.872 INFO:journalctl@ceph.mds.a.smithi042.stdout:Aug 30 08:57:49 smithi042 podman[62208]: 2024-08-30 08:57:49.634459695 +0000 UTC m=+0.147123095 container create f14af3600daf0bd4031f8f92306a9a6dcc2aafa7bd51531d585bfe8d7a5e708e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mds-a, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.1, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-08-30T08:57:49.872 INFO:journalctl@ceph.mds.a.smithi042.stdout:Aug 30 08:57:49 smithi042 podman[62208]: 2024-08-30 08:57:49.870837646 +0000 UTC m=+0.383501035 container init f14af3600daf0bd4031f8f92306a9a6dcc2aafa7bd51531d585bfe8d7a5e708e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mds-a, org.label-schema.schema-version=1.0, io.buildah.version=1.37.1, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240731) 2024-08-30T08:57:49.979 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph fs dump --format=json 2024-08-30T08:57:50.133 INFO:journalctl@ceph.mds.a.smithi042.stdout:Aug 30 08:57:49 smithi042 podman[62208]: 2024-08-30 08:57:49.873652969 +0000 UTC m=+0.386316363 container start f14af3600daf0bd4031f8f92306a9a6dcc2aafa7bd51531d585bfe8d7a5e708e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mds-a, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.37.1, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, GIT_CLEAN=True, ceph=True) 2024-08-30T08:57:50.133 INFO:journalctl@ceph.mds.a.smithi042.stdout:Aug 30 08:57:49 smithi042 ceph-mds[62227]: -- 172.21.15.42:0/3224805149 <== mon.0 v2:172.21.15.42:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d7d1b87860 con 0x55d7d5d2da80 2024-08-30T08:57:50.133 INFO:journalctl@ceph.mds.a.smithi042.stdout:Aug 30 08:57:49 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mds-a[62223]: starting mds.a at 2024-08-30T08:57:50.133 INFO:journalctl@ceph.mds.a.smithi042.stdout:Aug 30 08:57:49 smithi042 bash[62208]: f14af3600daf0bd4031f8f92306a9a6dcc2aafa7bd51531d585bfe8d7a5e708e 2024-08-30T08:57:50.133 INFO:journalctl@ceph.mds.a.smithi042.stdout:Aug 30 08:57:49 smithi042 systemd[1]: Started Ceph mds.a for b0895c1a-66ac-11ef-bcd4-c7b262605968. 2024-08-30T08:57:50.241 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T08:57:50.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:50 smithi042 ceph-mon[18652]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-30T08:57:50.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:50 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:50.634 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:50 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:50.634 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:50 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:50.634 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:50 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-30T08:57:50.634 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:50 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-30T08:57:50.634 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:50 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:57:50.719 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:50 smithi167 ceph-mon[22063]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-30T08:57:50.719 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:50 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:50.719 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:50 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:50.719 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:50 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:50.720 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:50 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-30T08:57:50.720 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:50 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-30T08:57:50.720 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:50 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:57:50.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:50.745+0000 7ff463823640 1 -- 172.21.15.42:0/3017277860 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff45c102800 msgr2=0x7ff45c102c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:57:50.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:50.745+0000 7ff463823640 1 --2- 172.21.15.42:0/3017277860 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff45c102800 0x7ff45c102c10 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ff454009900 tx=0x7ff45402a410 comp rx=0 tx=0).stop 2024-08-30T08:57:50.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:50.745+0000 7ff463823640 1 -- 172.21.15.42:0/3017277860 shutdown_connections 2024-08-30T08:57:50.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:50.745+0000 7ff463823640 1 --2- 172.21.15.42:0/3017277860 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff45c108140 0x7ff45c108550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:57:50.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:50.745+0000 7ff463823640 1 --2- 172.21.15.42:0/3017277860 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff45c108a90 0x7ff45c10f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:57:50.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:50.745+0000 7ff463823640 1 --2- 172.21.15.42:0/3017277860 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff45c102800 0x7ff45c102c10 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:57:50.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:50.745+0000 7ff463823640 1 -- 172.21.15.42:0/3017277860 >> 172.21.15.42:0/3017277860 conn(0x7ff45c0794d0 msgr2=0x7ff45c0798e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:57:50.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:50.745+0000 7ff463823640 1 -- 172.21.15.42:0/3017277860 shutdown_connections 2024-08-30T08:57:50.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:50.745+0000 7ff463823640 1 -- 172.21.15.42:0/3017277860 wait complete. 2024-08-30T08:57:50.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:50.746+0000 7ff463823640 1 Processor -- start 2024-08-30T08:57:50.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:50.746+0000 7ff463823640 1 -- start start 2024-08-30T08:57:50.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:50.746+0000 7ff463823640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff45c102800 0x7ff45c19afe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:57:50.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:50.746+0000 7ff463823640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff45c108140 0x7ff45c19b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:57:50.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:50.746+0000 7ff463823640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff45c108a90 0x7ff45c19ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:57:50.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:50.746+0000 7ff463823640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7ff45c112710 con 0x7ff45c108a90 2024-08-30T08:57:50.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:50.746+0000 7ff463823640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff45c112590 con 0x7ff45c102800 2024-08-30T08:57:50.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:50.746+0000 7ff463823640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7ff45c112890 con 0x7ff45c108140 2024-08-30T08:57:50.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:50.746+0000 7ff461dc0640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff45c108a90 0x7ff45c19ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:57:50.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:50.746+0000 7ff4615bf640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff45c102800 0x7ff45c19afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:57:50.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:50.746+0000 7ff4615bf640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff45c102800 0x7ff45c19afe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:51252/0 (socket says 172.21.15.42:51252) 2024-08-30T08:57:50.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:50.746+0000 7ff4615bf640 1 -- 172.21.15.42:0/899098205 learned_addr learned my addr 172.21.15.42:0/899098205 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T08:57:50.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:50.747+0000 7ff460dbe640 1 --2- 172.21.15.42:0/899098205 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff45c108140 0x7ff45c19b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:57:50.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:50.747+0000 7ff4615bf640 1 -- 172.21.15.42:0/899098205 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff45c108140 msgr2=0x7ff45c19b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:57:50.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:50.747+0000 7ff4615bf640 1 --2- 172.21.15.42:0/899098205 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff45c108140 0x7ff45c19b520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:57:50.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:50.747+0000 7ff4615bf640 1 -- 172.21.15.42:0/899098205 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff45c108a90 msgr2=0x7ff45c19ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:57:50.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:50.747+0000 7ff4615bf640 1 --2- 172.21.15.42:0/899098205 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff45c108a90 0x7ff45c19ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:57:50.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:50.747+0000 7ff4615bf640 1 -- 172.21.15.42:0/899098205 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff45c1a62f0 con 0x7ff45c102800 2024-08-30T08:57:50.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:50.747+0000 7ff460dbe640 1 --2- 172.21.15.42:0/899098205 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff45c108140 0x7ff45c19b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-30T08:57:50.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:50.747+0000 7ff461dc0640 1 --2- 172.21.15.42:0/899098205 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff45c108a90 0x7ff45c19ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-30T08:57:50.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:50.747+0000 7ff4615bf640 1 --2- 172.21.15.42:0/899098205 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff45c102800 0x7ff45c19afe0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff454005940 tx=0x7ff4540040c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:57:50.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:50.747+0000 7ff45a7fc640 1 -- 172.21.15.42:0/899098205 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff45403f030 con 0x7ff45c102800 2024-08-30T08:57:50.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:50.747+0000 7ff463823640 1 -- 172.21.15.42:0/899098205 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff45c1a6580 con 0x7ff45c102800 2024-08-30T08:57:50.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:50.747+0000 7ff45a7fc640 1 -- 172.21.15.42:0/899098205 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff4540044c0 con 0x7ff45c102800 2024-08-30T08:57:50.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:50.747+0000 7ff45a7fc640 1 -- 172.21.15.42:0/899098205 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff454035070 con 0x7ff45c102800 2024-08-30T08:57:50.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:50.747+0000 7ff463823640 1 -- 172.21.15.42:0/899098205 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff45c1a6a60 con 0x7ff45c102800 2024-08-30T08:57:50.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:50.748+0000 7ff463823640 1 -- 172.21.15.42:0/899098205 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff45c102c10 con 0x7ff45c102800 2024-08-30T08:57:50.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:50.750+0000 7ff45a7fc640 1 -- 172.21.15.42:0/899098205 <== mon.2 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 16) ==== 152577+0+0 (secure 0 0 0) 0x7ff454042030 con 0x7ff45c102800 2024-08-30T08:57:50.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:50.750+0000 7ff45a7fc640 1 --2- 172.21.15.42:0/899098205 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7ff4440b4b30 0x7ff4440b7040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:57:50.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:50.750+0000 7ff460dbe640 1 --2- 172.21.15.42:0/899098205 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7ff4440b4b30 0x7ff4440b7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:57:50.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:50.751+0000 7ff45a7fc640 1 -- 172.21.15.42:0/899098205 <== mon.2 v2:172.21.15.167:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 7127+0+0 (secure 0 0 0) 0x7ff4541053b0 con 0x7ff45c102800 2024-08-30T08:57:50.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:50.751+0000 7ff460dbe640 1 --2- 172.21.15.42:0/899098205 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7ff4440b4b30 0x7ff4440b7040 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7ff45c100a60 tx=0x7ff450005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:57:50.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:50.753+0000 7ff45a7fc640 1 -- 172.21.15.42:0/899098205 <== mon.2 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7ff4540cc8e0 con 0x7ff45c102800 2024-08-30T08:57:50.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:50 smithi190 ceph-mon[22001]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-30T08:57:50.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:50 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:50.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:50 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:50.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:50 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:50.937 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:50 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-30T08:57:50.937 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:50 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-30T08:57:50.937 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:50 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:57:50.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:50.976+0000 7ff463823640 1 -- 172.21.15.42:0/899098205 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff45c02e430 con 0x7ff45c102800 2024-08-30T08:57:50.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:50.977+0000 7ff45a7fc640 1 -- 172.21.15.42:0/899098205 <== mon.2 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9695 (secure 0 0 0) 0x7ff4540cc1e0 con 0x7ff45c102800 2024-08-30T08:57:50.979 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-08-30T08:57:50.979 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":12,"btime":"2024-08-30T08:57:50:586444+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14703,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/3188400000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":3188400000},{"type":"v1","addr":"172.21.15.42:6835","nonce":3188400000}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14715,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/3356537674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":3356537674},{"type":"v1","addr":"172.21.15.42:6837","nonce":3356537674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14766,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6839/2107785513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6838","nonce":2107785513},{"type":"v1","addr":"172.21.15.42:6839","nonce":2107785513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14778,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6841/2298438674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6840","nonce":2298438674},{"type":"v1","addr":"172.21.15.42:6841","nonce":2298438674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24553,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/543555570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":543555570},{"type":"v1","addr":"172.21.15.190:6833","nonce":543555570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24554,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/3907818941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":3907818941},{"type":"v1","addr":"172.21.15.167:6833","nonce":3907818941}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24572,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/234596845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":234596845},{"type":"v1","addr":"172.21.15.167:6835","nonce":234596845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24592,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/247905709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":247905709},{"type":"v1","addr":"172.21.15.190:6835","nonce":247905709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24598,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6837/863256349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6836","nonce":863256349},{"type":"v1","addr":"172.21.15.190:6837","nonce":863256349}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24599,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/3011837856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":3011837856},{"type":"v1","addr":"172.21.15.167:6837","nonce":3011837856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24610,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6839/719981110","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6838","nonce":719981110},{"type":"v1","addr":"172.21.15.190:6839","nonce":719981110}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-08-30T08:57:50.979 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 12 2024-08-30T08:57:50.981 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:50.979+0000 7ff463823640 1 -- 172.21.15.42:0/899098205 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7ff4440b4b30 msgr2=0x7ff4440b7040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:57:50.981 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:50.979+0000 7ff463823640 1 --2- 172.21.15.42:0/899098205 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7ff4440b4b30 0x7ff4440b7040 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7ff45c100a60 tx=0x7ff450005f90 comp rx=0 tx=0).stop 2024-08-30T08:57:50.981 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:50.979+0000 7ff463823640 1 -- 172.21.15.42:0/899098205 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff45c102800 msgr2=0x7ff45c19afe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:57:50.981 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:50.979+0000 7ff463823640 1 --2- 172.21.15.42:0/899098205 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff45c102800 0x7ff45c19afe0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff454005940 tx=0x7ff4540040c0 comp rx=0 tx=0).stop 2024-08-30T08:57:50.981 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:50.979+0000 7ff463823640 1 -- 172.21.15.42:0/899098205 shutdown_connections 2024-08-30T08:57:50.981 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:50.979+0000 7ff463823640 1 --2- 172.21.15.42:0/899098205 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7ff4440b4b30 0x7ff4440b7040 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:57:50.981 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:50.979+0000 7ff463823640 1 --2- 172.21.15.42:0/899098205 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff45c108140 0x7ff45c19b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:57:50.981 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:50.979+0000 7ff463823640 1 --2- 172.21.15.42:0/899098205 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff45c108a90 0x7ff45c19ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:57:50.981 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:50.979+0000 7ff463823640 1 --2- 172.21.15.42:0/899098205 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff45c102800 0x7ff45c19afe0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:57:50.981 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:50.979+0000 7ff463823640 1 -- 172.21.15.42:0/899098205 >> 172.21.15.42:0/899098205 conn(0x7ff45c0794d0 msgr2=0x7ff45c101170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:57:50.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:50.979+0000 7ff463823640 1 -- 172.21.15.42:0/899098205 shutdown_connections 2024-08-30T08:57:50.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:50.979+0000 7ff463823640 1 -- 172.21.15.42:0/899098205 wait complete. 2024-08-30T08:57:51.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:51 smithi167 ceph-mon[22063]: Deploying daemon mds.b on smithi167 2024-08-30T08:57:51.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:51 smithi167 ceph-mon[22063]: mds.? [v2:172.21.15.42:6840/2298438674,v1:172.21.15.42:6841/2298438674] up:boot 2024-08-30T08:57:51.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:51 smithi167 ceph-mon[22063]: fsmap 11 up:standby 2024-08-30T08:57:51.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:51 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "a"} : dispatch 2024-08-30T08:57:51.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:51 smithi167 ceph-mon[22063]: from='client.? 172.21.15.42:0/899098205' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-30T08:57:51.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:51 smithi042 ceph-mon[18652]: Deploying daemon mds.b on smithi167 2024-08-30T08:57:51.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:51 smithi042 ceph-mon[18652]: mds.? [v2:172.21.15.42:6840/2298438674,v1:172.21.15.42:6841/2298438674] up:boot 2024-08-30T08:57:51.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:51 smithi042 ceph-mon[18652]: fsmap 11 up:standby 2024-08-30T08:57:51.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:51 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "a"} : dispatch 2024-08-30T08:57:51.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:51 smithi042 ceph-mon[18652]: from='client.? 172.21.15.42:0/899098205' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-30T08:57:51.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:51 smithi190 ceph-mon[22001]: Deploying daemon mds.b on smithi167 2024-08-30T08:57:51.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:51 smithi190 ceph-mon[22001]: mds.? [v2:172.21.15.42:6840/2298438674,v1:172.21.15.42:6841/2298438674] up:boot 2024-08-30T08:57:51.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:51 smithi190 ceph-mon[22001]: fsmap 11 up:standby 2024-08-30T08:57:51.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:51 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "a"} : dispatch 2024-08-30T08:57:51.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:51 smithi190 ceph-mon[22001]: from='client.? 172.21.15.42:0/899098205' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-30T08:57:52.321 INFO:journalctl@ceph.mds.b.smithi167.stdout:Aug 30 08:57:52 smithi167 systemd[1]: Starting Ceph mds.b for b0895c1a-66ac-11ef-bcd4-c7b262605968... 2024-08-30T08:57:52.577 INFO:journalctl@ceph.mds.b.smithi167.stdout:Aug 30 08:57:52 smithi167 podman[52458]: 2024-08-30 08:57:52.221951973 +0000 UTC m=+0.015564585 image pull 79f4cde5e36911a5dca995eeb98098c2c8aae81dd52a088ef2c867bc3363bad0 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61 2024-08-30T08:57:52.577 INFO:journalctl@ceph.mds.b.smithi167.stdout:Aug 30 08:57:52 smithi167 podman[52458]: 2024-08-30 08:57:52.356689361 +0000 UTC m=+0.150301958 container create 7ec7569892e37e2a5c54a19a462030d6d5928b79ad6180962113e3ef08db023c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mds-b, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.37.1, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611) 2024-08-30T08:57:52.882 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:52 smithi042 ceph-mon[18652]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-30T08:57:52.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:52 smithi190 ceph-mon[22001]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-30T08:57:53.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:52 smithi167 ceph-mon[22063]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-30T08:57:53.077 INFO:journalctl@ceph.mds.b.smithi167.stdout:Aug 30 08:57:52 smithi167 podman[52458]: 2024-08-30 08:57:52.592465515 +0000 UTC m=+0.386078117 container init 7ec7569892e37e2a5c54a19a462030d6d5928b79ad6180962113e3ef08db023c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mds-b, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.1, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-08-30T08:57:53.077 INFO:journalctl@ceph.mds.b.smithi167.stdout:Aug 30 08:57:52 smithi167 podman[52458]: 2024-08-30 08:57:52.595466585 +0000 UTC m=+0.389079181 container start 7ec7569892e37e2a5c54a19a462030d6d5928b79ad6180962113e3ef08db023c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mds-b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, io.buildah.version=1.37.1, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-08-30T08:57:53.077 INFO:journalctl@ceph.mds.b.smithi167.stdout:Aug 30 08:57:52 smithi167 ceph-mds[52477]: -- 172.21.15.167:0/4261485205 <== mon.2 v2:172.21.15.167:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x555a2dce9860 con 0x555a31e8f180 2024-08-30T08:57:53.077 INFO:journalctl@ceph.mds.b.smithi167.stdout:Aug 30 08:57:52 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mds-b[52473]: starting mds.b at 2024-08-30T08:57:53.077 INFO:journalctl@ceph.mds.b.smithi167.stdout:Aug 30 08:57:52 smithi167 bash[52458]: 7ec7569892e37e2a5c54a19a462030d6d5928b79ad6180962113e3ef08db023c 2024-08-30T08:57:53.077 INFO:journalctl@ceph.mds.b.smithi167.stdout:Aug 30 08:57:52 smithi167 systemd[1]: Started Ceph mds.b for b0895c1a-66ac-11ef-bcd4-c7b262605968. 2024-08-30T08:57:53.701 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph fs dump --format=json 2024-08-30T08:57:53.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:53 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:53.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:53 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:53.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:53 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:53.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:53 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:53.937 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:53 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:53.937 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:53 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-30T08:57:53.937 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:53 smithi190 ceph-mon[22001]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-30T08:57:53.937 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:53 smithi190 ceph-mon[22001]: mds.? [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] up:boot 2024-08-30T08:57:53.937 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:53 smithi190 ceph-mon[22001]: fsmap 12 up:standby 2024-08-30T08:57:53.937 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:53 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2024-08-30T08:57:54.005 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T08:57:54.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:53 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:54.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:53 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:54.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:53 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:54.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:53 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:54.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:53 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:54.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:53 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-30T08:57:54.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:53 smithi167 ceph-mon[22063]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-30T08:57:54.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:53 smithi167 ceph-mon[22063]: mds.? [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] up:boot 2024-08-30T08:57:54.078 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:53 smithi167 ceph-mon[22063]: fsmap 12 up:standby 2024-08-30T08:57:54.078 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:53 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2024-08-30T08:57:54.128 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:53 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:54.128 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:53 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:54.128 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:53 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:54.128 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:53 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:54.128 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:53 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:54.128 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:53 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-30T08:57:54.128 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:53 smithi042 ceph-mon[18652]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-30T08:57:54.128 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:53 smithi042 ceph-mon[18652]: mds.? [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] up:boot 2024-08-30T08:57:54.128 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:53 smithi042 ceph-mon[18652]: fsmap 12 up:standby 2024-08-30T08:57:54.128 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:53 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2024-08-30T08:57:54.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:54.556+0000 7fef845b0640 1 -- 172.21.15.42:0/3879470836 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fef7c102800 msgr2=0x7fef7c102c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:57:54.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:54.556+0000 7fef845b0640 1 --2- 172.21.15.42:0/3879470836 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fef7c102800 0x7fef7c102c10 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fef78009900 tx=0x7fef7802a410 comp rx=0 tx=0).stop 2024-08-30T08:57:54.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:54.556+0000 7fef845b0640 1 -- 172.21.15.42:0/3879470836 shutdown_connections 2024-08-30T08:57:54.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:54.556+0000 7fef845b0640 1 --2- 172.21.15.42:0/3879470836 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fef7c108140 0x7fef7c108550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:57:54.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:54.556+0000 7fef845b0640 1 --2- 172.21.15.42:0/3879470836 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fef7c108a90 0x7fef7c10f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:57:54.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:54.556+0000 7fef845b0640 1 --2- 172.21.15.42:0/3879470836 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fef7c102800 0x7fef7c102c10 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:57:54.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:54.556+0000 7fef845b0640 1 -- 172.21.15.42:0/3879470836 >> 172.21.15.42:0/3879470836 conn(0x7fef7c0794d0 msgr2=0x7fef7c0798e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:57:54.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:54.556+0000 7fef845b0640 1 -- 172.21.15.42:0/3879470836 shutdown_connections 2024-08-30T08:57:54.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:54.556+0000 7fef845b0640 1 -- 172.21.15.42:0/3879470836 wait complete. 2024-08-30T08:57:54.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:54.557+0000 7fef845b0640 1 Processor -- start 2024-08-30T08:57:54.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:54.557+0000 7fef845b0640 1 -- start start 2024-08-30T08:57:54.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:54.558+0000 7fef845b0640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fef7c102800 0x7fef7c19a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:57:54.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:54.558+0000 7fef845b0640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fef7c108140 0x7fef7c19a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:57:54.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:54.558+0000 7fef845b0640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fef7c108a90 0x7fef7c19faf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:57:54.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:54.558+0000 7fef845b0640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fef7c112710 con 0x7fef7c102800 2024-08-30T08:57:54.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:54.558+0000 7fef845b0640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fef7c112590 con 0x7fef7c108a90 2024-08-30T08:57:54.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:54.558+0000 7fef845b0640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fef7c112890 con 0x7fef7c108140 2024-08-30T08:57:54.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:54.558+0000 7fef8234c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fef7c102800 0x7fef7c19a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:57:54.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:54.558+0000 7fef8234c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fef7c102800 0x7fef7c19a2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56594/0 (socket says 172.21.15.42:56594) 2024-08-30T08:57:54.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:54.558+0000 7fef8234c640 1 -- 172.21.15.42:0/3110378757 learned_addr learned my addr 172.21.15.42:0/3110378757 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T08:57:54.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:54.558+0000 7fef81b4b640 1 --2- 172.21.15.42:0/3110378757 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fef7c108140 0x7fef7c19a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:57:54.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:54.558+0000 7fef82b4d640 1 --2- 172.21.15.42:0/3110378757 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fef7c108a90 0x7fef7c19faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:57:54.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:54.558+0000 7fef8234c640 1 -- 172.21.15.42:0/3110378757 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fef7c108140 msgr2=0x7fef7c19a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:57:54.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:54.558+0000 7fef8234c640 1 --2- 172.21.15.42:0/3110378757 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fef7c108140 0x7fef7c19a850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:57:54.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:54.558+0000 7fef8234c640 1 -- 172.21.15.42:0/3110378757 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fef7c108a90 msgr2=0x7fef7c19faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:57:54.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:54.558+0000 7fef8234c640 1 --2- 172.21.15.42:0/3110378757 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fef7c108a90 0x7fef7c19faf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:57:54.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:54.558+0000 7fef8234c640 1 -- 172.21.15.42:0/3110378757 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef7c1a0160 con 0x7fef7c102800 2024-08-30T08:57:54.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:54.558+0000 7fef8234c640 1 --2- 172.21.15.42:0/3110378757 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fef7c102800 0x7fef7c19a2b0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fef78006010 tx=0x7fef780042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:57:54.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:54.559+0000 7fef737fe640 1 -- 172.21.15.42:0/3110378757 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef7803f030 con 0x7fef7c102800 2024-08-30T08:57:54.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:54.559+0000 7fef845b0640 1 -- 172.21.15.42:0/3110378757 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef7c1a03f0 con 0x7fef7c102800 2024-08-30T08:57:54.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:54.559+0000 7fef737fe640 1 -- 172.21.15.42:0/3110378757 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef78003800 con 0x7fef7c102800 2024-08-30T08:57:54.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:54.559+0000 7fef845b0640 1 -- 172.21.15.42:0/3110378757 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef7c1a0920 con 0x7fef7c102800 2024-08-30T08:57:54.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:54.559+0000 7fef737fe640 1 -- 172.21.15.42:0/3110378757 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef78035070 con 0x7fef7c102800 2024-08-30T08:57:54.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:54.560+0000 7fef845b0640 1 -- 172.21.15.42:0/3110378757 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef7c102c10 con 0x7fef7c102800 2024-08-30T08:57:54.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:54.561+0000 7fef737fe640 1 -- 172.21.15.42:0/3110378757 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) ==== 152577+0+0 (secure 0 0 0) 0x7fef78003cb0 con 0x7fef7c102800 2024-08-30T08:57:54.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:54.562+0000 7fef737fe640 1 --2- 172.21.15.42:0/3110378757 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fef680b4b60 0x7fef680b7070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:57:54.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:54.562+0000 7fef81b4b640 1 --2- 172.21.15.42:0/3110378757 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fef680b4b60 0x7fef680b7070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:57:54.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:54.562+0000 7fef737fe640 1 -- 172.21.15.42:0/3110378757 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 7127+0+0 (secure 0 0 0) 0x7fef781057f0 con 0x7fef7c102800 2024-08-30T08:57:54.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:54.563+0000 7fef81b4b640 1 --2- 172.21.15.42:0/3110378757 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fef680b4b60 0x7fef680b7070 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fef6c005fd0 tx=0x7fef6c005e20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:57:54.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:54.566+0000 7fef737fe640 1 -- 172.21.15.42:0/3110378757 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7fef7c1a0920 con 0x7fef7c102800 2024-08-30T08:57:54.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:54.802+0000 7fef845b0640 1 -- 172.21.15.42:0/3110378757 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fef7c19b3e0 con 0x7fef7c102800 2024-08-30T08:57:54.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:54.803+0000 7fef737fe640 1 -- 172.21.15.42:0/3110378757 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10524 (secure 0 0 0) 0x7fef780ccdc0 con 0x7fef7c102800 2024-08-30T08:57:54.805 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-08-30T08:57:54.805 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":13,"btime":"2024-08-30T08:57:53:599541+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14703,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/3188400000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":3188400000},{"type":"v1","addr":"172.21.15.42:6835","nonce":3188400000}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14715,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/3356537674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":3356537674},{"type":"v1","addr":"172.21.15.42:6837","nonce":3356537674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14766,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6839/2107785513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6838","nonce":2107785513},{"type":"v1","addr":"172.21.15.42:6839","nonce":2107785513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14778,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6841/2298438674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6840","nonce":2298438674},{"type":"v1","addr":"172.21.15.42:6841","nonce":2298438674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24553,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/543555570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":543555570},{"type":"v1","addr":"172.21.15.190:6833","nonce":543555570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24554,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/3907818941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":3907818941},{"type":"v1","addr":"172.21.15.167:6833","nonce":3907818941}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24572,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/234596845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":234596845},{"type":"v1","addr":"172.21.15.167:6835","nonce":234596845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24592,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/247905709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":247905709},{"type":"v1","addr":"172.21.15.190:6835","nonce":247905709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24598,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6837/863256349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6836","nonce":863256349},{"type":"v1","addr":"172.21.15.190:6837","nonce":863256349}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24599,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/3011837856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":3011837856},{"type":"v1","addr":"172.21.15.167:6837","nonce":3011837856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24610,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6839/719981110","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6838","nonce":719981110},{"type":"v1","addr":"172.21.15.190:6839","nonce":719981110}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24629,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6839/3604645207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":3604645207},{"type":"v1","addr":"172.21.15.167:6839","nonce":3604645207}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-08-30T08:57:54.805 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 13 2024-08-30T08:57:54.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:54.805+0000 7fef845b0640 1 -- 172.21.15.42:0/3110378757 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fef680b4b60 msgr2=0x7fef680b7070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:57:54.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:54.805+0000 7fef845b0640 1 --2- 172.21.15.42:0/3110378757 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fef680b4b60 0x7fef680b7070 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fef6c005fd0 tx=0x7fef6c005e20 comp rx=0 tx=0).stop 2024-08-30T08:57:54.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:54.805+0000 7fef845b0640 1 -- 172.21.15.42:0/3110378757 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fef7c102800 msgr2=0x7fef7c19a2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:57:54.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:54.805+0000 7fef845b0640 1 --2- 172.21.15.42:0/3110378757 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fef7c102800 0x7fef7c19a2b0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fef78006010 tx=0x7fef780042f0 comp rx=0 tx=0).stop 2024-08-30T08:57:54.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:54.805+0000 7fef845b0640 1 -- 172.21.15.42:0/3110378757 shutdown_connections 2024-08-30T08:57:54.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:54.805+0000 7fef845b0640 1 --2- 172.21.15.42:0/3110378757 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fef680b4b60 0x7fef680b7070 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:57:54.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:54.805+0000 7fef845b0640 1 --2- 172.21.15.42:0/3110378757 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fef7c108140 0x7fef7c19a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:57:54.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:54.805+0000 7fef845b0640 1 --2- 172.21.15.42:0/3110378757 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fef7c108a90 0x7fef7c19faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:57:54.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:54.805+0000 7fef845b0640 1 --2- 172.21.15.42:0/3110378757 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fef7c102800 0x7fef7c19a2b0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:57:54.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:54.805+0000 7fef845b0640 1 -- 172.21.15.42:0/3110378757 >> 172.21.15.42:0/3110378757 conn(0x7fef7c0794d0 msgr2=0x7fef7c1018e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:57:54.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:54.805+0000 7fef845b0640 1 -- 172.21.15.42:0/3110378757 shutdown_connections 2024-08-30T08:57:54.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:54.806+0000 7fef845b0640 1 -- 172.21.15.42:0/3110378757 wait complete. 2024-08-30T08:57:55.439 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:55 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:55.439 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:55 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:55.440 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:55 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:55.440 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:55 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:55.440 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:55 smithi042 ceph-mon[18652]: from='client.? 172.21.15.42:0/3110378757' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-30T08:57:55.440 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:55 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:55.440 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:55 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:55.440 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:55 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:55.472 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph --format=json-pretty osd lspools 2024-08-30T08:57:55.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:55 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:55.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:55 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:55.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:55 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:55.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:55 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:55.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:55 smithi190 ceph-mon[22001]: from='client.? 172.21.15.42:0/3110378757' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-30T08:57:55.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:55 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:55.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:55 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:55.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:55 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:55.743 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T08:57:55.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:55 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:55.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:55 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:55.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:55 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:55.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:55 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:55.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:55 smithi167 ceph-mon[22063]: from='client.? 172.21.15.42:0/3110378757' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-30T08:57:55.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:55 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:55.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:55 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:55.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:55 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:56.285 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:56.275+0000 7f84bfb6a640 1 -- 172.21.15.42:0/3410364626 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f84b810ffa0 msgr2=0x7f84b81123c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:57:56.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:56.275+0000 7f84bfb6a640 1 --2- 172.21.15.42:0/3410364626 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f84b810ffa0 0x7f84b81123c0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f84b0009ca0 tx=0x7f84b002a950 comp rx=0 tx=0).stop 2024-08-30T08:57:56.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:56.275+0000 7f84bfb6a640 1 -- 172.21.15.42:0/3410364626 shutdown_connections 2024-08-30T08:57:56.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:56.275+0000 7f84bfb6a640 1 --2- 172.21.15.42:0/3410364626 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f84b8102d40 0x7f84b810fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:57:56.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:56.275+0000 7f84bfb6a640 1 --2- 172.21.15.42:0/3410364626 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f84b810ffa0 0x7f84b81123c0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:57:56.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:56.275+0000 7f84bfb6a640 1 --2- 172.21.15.42:0/3410364626 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f84b81023f0 0x7f84b8102800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:57:56.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:56.275+0000 7f84bfb6a640 1 -- 172.21.15.42:0/3410364626 >> 172.21.15.42:0/3410364626 conn(0x7f84b8079500 msgr2=0x7f84b8079910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:57:56.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:56.275+0000 7f84bfb6a640 1 -- 172.21.15.42:0/3410364626 shutdown_connections 2024-08-30T08:57:56.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:56.275+0000 7f84bfb6a640 1 -- 172.21.15.42:0/3410364626 wait complete. 2024-08-30T08:57:56.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:56.276+0000 7f84bfb6a640 1 Processor -- start 2024-08-30T08:57:56.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:56.276+0000 7f84bfb6a640 1 -- start start 2024-08-30T08:57:56.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:56.276+0000 7f84bfb6a640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f84b81023f0 0x7f84b819c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:57:56.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:56.276+0000 7f84bfb6a640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f84b8102d40 0x7f84b819c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:57:56.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:56.276+0000 7f84bfb6a640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f84b810ffa0 0x7f84b81a1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:57:56.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:56.276+0000 7f84bfb6a640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f84b8115a40 con 0x7f84b810ffa0 2024-08-30T08:57:56.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:56.276+0000 7f84bfb6a640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f84b81158c0 con 0x7f84b81023f0 2024-08-30T08:57:56.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:56.276+0000 7f84bfb6a640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f84b8115bc0 con 0x7f84b8102d40 2024-08-30T08:57:56.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:56.276+0000 7f84be107640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f84b810ffa0 0x7f84b81a1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:57:56.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:56.276+0000 7f84be107640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f84b810ffa0 0x7f84b81a1bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56618/0 (socket says 172.21.15.42:56618) 2024-08-30T08:57:56.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:56.276+0000 7f84be107640 1 -- 172.21.15.42:0/3119863138 learned_addr learned my addr 172.21.15.42:0/3119863138 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T08:57:56.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:56.276+0000 7f84bd906640 1 --2- 172.21.15.42:0/3119863138 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f84b81023f0 0x7f84b819c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:57:56.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:56.276+0000 7f84bd105640 1 --2- 172.21.15.42:0/3119863138 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f84b8102d40 0x7f84b819c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:57:56.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:56.277+0000 7f84be107640 1 -- 172.21.15.42:0/3119863138 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f84b8102d40 msgr2=0x7f84b819c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:57:56.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:56.277+0000 7f84be107640 1 --2- 172.21.15.42:0/3119863138 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f84b8102d40 0x7f84b819c930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:57:56.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:56.277+0000 7f84be107640 1 -- 172.21.15.42:0/3119863138 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f84b81023f0 msgr2=0x7f84b819c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:57:56.288 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:56.277+0000 7f84be107640 1 --2- 172.21.15.42:0/3119863138 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f84b81023f0 0x7f84b819c390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:57:56.288 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:56.277+0000 7f84be107640 1 -- 172.21.15.42:0/3119863138 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84b81a2300 con 0x7f84b810ffa0 2024-08-30T08:57:56.288 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:56.277+0000 7f84be107640 1 --2- 172.21.15.42:0/3119863138 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f84b810ffa0 0x7f84b81a1bd0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f84b002ae70 tx=0x7f84b002f5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:57:56.288 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:56.277+0000 7f84aeffd640 1 -- 172.21.15.42:0/3119863138 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84b0034040 con 0x7f84b810ffa0 2024-08-30T08:57:56.288 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:56.277+0000 7f84aeffd640 1 -- 172.21.15.42:0/3119863138 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f84b00308f0 con 0x7f84b810ffa0 2024-08-30T08:57:56.288 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:56.277+0000 7f84bfb6a640 1 -- 172.21.15.42:0/3119863138 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84b81a2590 con 0x7f84b810ffa0 2024-08-30T08:57:56.288 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:56.277+0000 7f84aeffd640 1 -- 172.21.15.42:0/3119863138 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84b00047f0 con 0x7f84b810ffa0 2024-08-30T08:57:56.288 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:56.277+0000 7f84bfb6a640 1 -- 172.21.15.42:0/3119863138 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84b81a8d50 con 0x7f84b810ffa0 2024-08-30T08:57:56.288 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:56.278+0000 7f84bfb6a640 1 -- 172.21.15.42:0/3119863138 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84b806bf90 con 0x7f84b810ffa0 2024-08-30T08:57:56.288 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:56.280+0000 7f84aeffd640 1 -- 172.21.15.42:0/3119863138 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) ==== 152577+0+0 (secure 0 0 0) 0x7f84b0037020 con 0x7f84b810ffa0 2024-08-30T08:57:56.288 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:56.280+0000 7f84aeffd640 1 --2- 172.21.15.42:0/3119863138 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f84a40b4bb0 0x7f84a40b70c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:57:56.288 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:56.280+0000 7f84bd906640 1 --2- 172.21.15.42:0/3119863138 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f84a40b4bb0 0x7f84a40b70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:57:56.288 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:56.280+0000 7f84aeffd640 1 -- 172.21.15.42:0/3119863138 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 7127+0+0 (secure 0 0 0) 0x7f84b0105c10 con 0x7f84b810ffa0 2024-08-30T08:57:56.288 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:56.281+0000 7f84bd906640 1 --2- 172.21.15.42:0/3119863138 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f84a40b4bb0 0x7f84a40b70c0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f84b4005d00 tx=0x7f84b400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:57:56.289 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:56.285+0000 7f84aeffd640 1 -- 172.21.15.42:0/3119863138 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f84b00cd140 con 0x7f84b810ffa0 2024-08-30T08:57:56.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:56.492+0000 7f84bfb6a640 1 -- 172.21.15.42:0/3119863138 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f84b81a8f20 con 0x7f84b810ffa0 2024-08-30T08:57:56.494 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:56 smithi042 ceph-mon[18652]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-30T08:57:56.494 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:56 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:57:56.494 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:56 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-30T08:57:56.494 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:56 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:56.494 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:56 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-30T08:57:56.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:56.492+0000 7f84aeffd640 1 -- 172.21.15.42:0/3119863138 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v68) ==== 84+0+65 (secure 0 0 0) 0x7f84b00ef060 con 0x7f84b810ffa0 2024-08-30T08:57:56.495 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-08-30T08:57:56.495 INFO:teuthology.orchestra.run.smithi042.stdout:[ 2024-08-30T08:57:56.495 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-08-30T08:57:56.495 INFO:teuthology.orchestra.run.smithi042.stdout: "poolnum": 1, 2024-08-30T08:57:56.495 INFO:teuthology.orchestra.run.smithi042.stdout: "poolname": ".mgr" 2024-08-30T08:57:56.495 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T08:57:56.495 INFO:teuthology.orchestra.run.smithi042.stdout:] 2024-08-30T08:57:56.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:56.494+0000 7f84bfb6a640 1 -- 172.21.15.42:0/3119863138 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f84a40b4bb0 msgr2=0x7f84a40b70c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:57:56.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:56.494+0000 7f84bfb6a640 1 --2- 172.21.15.42:0/3119863138 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f84a40b4bb0 0x7f84a40b70c0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f84b4005d00 tx=0x7f84b400b410 comp rx=0 tx=0).stop 2024-08-30T08:57:56.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:56.495+0000 7f84bfb6a640 1 -- 172.21.15.42:0/3119863138 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f84b810ffa0 msgr2=0x7f84b81a1bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:57:56.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:56.495+0000 7f84bfb6a640 1 --2- 172.21.15.42:0/3119863138 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f84b810ffa0 0x7f84b81a1bd0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f84b002ae70 tx=0x7f84b002f5d0 comp rx=0 tx=0).stop 2024-08-30T08:57:56.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:56.495+0000 7f84bfb6a640 1 -- 172.21.15.42:0/3119863138 shutdown_connections 2024-08-30T08:57:56.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:56.495+0000 7f84bfb6a640 1 --2- 172.21.15.42:0/3119863138 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f84a40b4bb0 0x7f84a40b70c0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:57:56.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:56.495+0000 7f84bfb6a640 1 --2- 172.21.15.42:0/3119863138 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f84b8102d40 0x7f84b819c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:57:56.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:56.495+0000 7f84bfb6a640 1 --2- 172.21.15.42:0/3119863138 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f84b810ffa0 0x7f84b81a1bd0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:57:56.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:56.495+0000 7f84bfb6a640 1 --2- 172.21.15.42:0/3119863138 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f84b81023f0 0x7f84b819c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:57:56.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:56.495+0000 7f84bfb6a640 1 -- 172.21.15.42:0/3119863138 >> 172.21.15.42:0/3119863138 conn(0x7f84b8079500 msgr2=0x7f84b8109ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:57:56.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:56.495+0000 7f84bfb6a640 1 -- 172.21.15.42:0/3119863138 shutdown_connections 2024-08-30T08:57:56.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:56.495+0000 7f84bfb6a640 1 -- 172.21.15.42:0/3119863138 wait complete. 2024-08-30T08:57:56.611 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:56 smithi167 ceph-mon[22063]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-30T08:57:56.611 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:56 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:57:56.611 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:56 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-30T08:57:56.611 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:56 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:56.611 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:56 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-30T08:57:56.626 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:56 smithi190 ceph-mon[22001]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-30T08:57:56.626 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:56 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:57:56.626 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:56 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-30T08:57:56.626 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:56 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:56.626 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:56 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-30T08:57:57.386 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-08-30T08:57:57.386 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph osd pool create cephfs_metadata --pg_num_min 64 2024-08-30T08:57:57.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:57 smithi042 ceph-mon[18652]: from='client.? 172.21.15.42:0/3119863138' entity='client.admin' cmd={"prefix": "osd lspools", "format": "json-pretty"} : dispatch 2024-08-30T08:57:57.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:57 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:57.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:57 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:57.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:57 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:57.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:57 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:57.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:57 smithi190 ceph-mon[22001]: from='client.? 172.21.15.42:0/3119863138' entity='client.admin' cmd={"prefix": "osd lspools", "format": "json-pretty"} : dispatch 2024-08-30T08:57:57.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:57 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:57.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:57 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:57.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:57 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:57.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:57 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:57.755 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T08:57:57.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:57 smithi167 ceph-mon[22063]: from='client.? 172.21.15.42:0/3119863138' entity='client.admin' cmd={"prefix": "osd lspools", "format": "json-pretty"} : dispatch 2024-08-30T08:57:57.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:57 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:57.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:57 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:57.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:57 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:57.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:57 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:58.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:58.284+0000 7f782cc9d640 1 -- 172.21.15.42:0/2434935527 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7828108a90 msgr2=0x7f782810f2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:57:58.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:58.284+0000 7f782cc9d640 1 --2- 172.21.15.42:0/2434935527 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7828108a90 0x7f782810f2c0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f7820009ca0 tx=0x7f782002a8c0 comp rx=0 tx=0).stop 2024-08-30T08:57:58.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:58.284+0000 7f782cc9d640 1 -- 172.21.15.42:0/2434935527 shutdown_connections 2024-08-30T08:57:58.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:58.284+0000 7f782cc9d640 1 --2- 172.21.15.42:0/2434935527 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7828108a90 0x7f782810f2c0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:57:58.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:58.284+0000 7f782cc9d640 1 --2- 172.21.15.42:0/2434935527 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7828108140 0x7f7828108550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:57:58.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:58.284+0000 7f782cc9d640 1 --2- 172.21.15.42:0/2434935527 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7828102800 0x7f7828102c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:57:58.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:58.284+0000 7f782cc9d640 1 -- 172.21.15.42:0/2434935527 >> 172.21.15.42:0/2434935527 conn(0x7f78280794d0 msgr2=0x7f78280798e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:57:58.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:58.284+0000 7f782cc9d640 1 -- 172.21.15.42:0/2434935527 shutdown_connections 2024-08-30T08:57:58.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:58.284+0000 7f782cc9d640 1 -- 172.21.15.42:0/2434935527 wait complete. 2024-08-30T08:57:58.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:58.285+0000 7f782cc9d640 1 Processor -- start 2024-08-30T08:57:58.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:58.285+0000 7f782cc9d640 1 -- start start 2024-08-30T08:57:58.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:58.285+0000 7f782cc9d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7828102800 0x7f782819a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:57:58.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:58.285+0000 7f782cc9d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7828108140 0x7f782819a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:57:58.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:58.285+0000 7f782cc9d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7828108a90 0x7f782819fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:57:58.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:58.285+0000 7f782cc9d640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f7828112770 con 0x7f7828108a90 2024-08-30T08:57:58.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:58.285+0000 7f782cc9d640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f78281125f0 con 0x7f7828108140 2024-08-30T08:57:58.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:58.285+0000 7f782cc9d640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f78281128f0 con 0x7f7828102800 2024-08-30T08:57:58.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:58.285+0000 7f7826d9d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7828108a90 0x7f782819fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:57:58.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:58.285+0000 7f7826d9d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7828108a90 0x7f782819fb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42036/0 (socket says 172.21.15.42:42036) 2024-08-30T08:57:58.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:58.285+0000 7f7826d9d640 1 -- 172.21.15.42:0/1154901797 learned_addr learned my addr 172.21.15.42:0/1154901797 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T08:57:58.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:58.285+0000 7f781ffff640 1 --2- 172.21.15.42:0/1154901797 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7828108140 0x7f782819a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:57:58.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:58.286+0000 7f782659c640 1 --2- 172.21.15.42:0/1154901797 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7828102800 0x7f782819a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:57:58.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:58.286+0000 7f7826d9d640 1 -- 172.21.15.42:0/1154901797 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7828102800 msgr2=0x7f782819a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:57:58.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:58.286+0000 7f7826d9d640 1 --2- 172.21.15.42:0/1154901797 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7828102800 0x7f782819a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:57:58.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:58.286+0000 7f7826d9d640 1 -- 172.21.15.42:0/1154901797 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7828108140 msgr2=0x7f782819a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:57:58.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:58.286+0000 7f7826d9d640 1 --2- 172.21.15.42:0/1154901797 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7828108140 0x7f782819a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:57:58.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:58.286+0000 7f7826d9d640 1 -- 172.21.15.42:0/1154901797 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78281a0260 con 0x7f7828108a90 2024-08-30T08:57:58.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:58.286+0000 7f781ffff640 1 --2- 172.21.15.42:0/1154901797 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7828108140 0x7f782819a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-30T08:57:58.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:58.286+0000 7f7826d9d640 1 --2- 172.21.15.42:0/1154901797 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7828108a90 0x7f782819fb30 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f78200098d0 tx=0x7f78200317b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:57:58.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:58.286+0000 7f781f7fe640 1 -- 172.21.15.42:0/1154901797 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7820003b70 con 0x7f7828108a90 2024-08-30T08:57:58.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:58.286+0000 7f781f7fe640 1 -- 172.21.15.42:0/1154901797 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7820003d10 con 0x7f7828108a90 2024-08-30T08:57:58.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:58.286+0000 7f782cc9d640 1 -- 172.21.15.42:0/1154901797 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78281a04f0 con 0x7f7828108a90 2024-08-30T08:57:58.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:58.286+0000 7f781f7fe640 1 -- 172.21.15.42:0/1154901797 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f782002f9d0 con 0x7f7828108a90 2024-08-30T08:57:58.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:58.286+0000 7f782cc9d640 1 -- 172.21.15.42:0/1154901797 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78281a6cb0 con 0x7f7828108a90 2024-08-30T08:57:58.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:58.288+0000 7f782cc9d640 1 -- 172.21.15.42:0/1154901797 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7828102c10 con 0x7f7828108a90 2024-08-30T08:57:58.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:58.289+0000 7f781f7fe640 1 -- 172.21.15.42:0/1154901797 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) ==== 152577+0+0 (secure 0 0 0) 0x7f782002fb70 con 0x7f7828108a90 2024-08-30T08:57:58.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:58.289+0000 7f781f7fe640 1 --2- 172.21.15.42:0/1154901797 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f78140b4bb0 0x7f78140b70c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:57:58.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:58.290+0000 7f782659c640 1 --2- 172.21.15.42:0/1154901797 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f78140b4bb0 0x7f78140b70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:57:58.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:58.290+0000 7f781f7fe640 1 -- 172.21.15.42:0/1154901797 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 7127+0+0 (secure 0 0 0) 0x7f7820105fb0 con 0x7f7828108a90 2024-08-30T08:57:58.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:58.290+0000 7f782659c640 1 --2- 172.21.15.42:0/1154901797 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f78140b4bb0 0x7f78140b70c0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f7818009990 tx=0x7f7818009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:57:58.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:58.294+0000 7f781f7fe640 1 -- 172.21.15.42:0/1154901797 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f78200cd580 con 0x7f7828108a90 2024-08-30T08:57:58.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:58.484+0000 7f782cc9d640 1 -- 172.21.15.42:0/1154901797 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f78281a6f10 con 0x7f7828108a90 2024-08-30T08:57:58.486 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:58 smithi042 ceph-mon[18652]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-30T08:57:58.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:58 smithi190 ceph-mon[22001]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-30T08:57:58.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:58 smithi167 ceph-mon[22063]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-30T08:57:59.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:59.368+0000 7f781f7fe640 1 -- 172.21.15.42:0/1154901797 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v69) ==== 138+0+0 (secure 0 0 0) 0x7f78200cce80 con 0x7f7828108a90 2024-08-30T08:57:59.371 INFO:teuthology.orchestra.run.smithi042.stderr:pool 'cephfs_metadata' created 2024-08-30T08:57:59.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:59.372+0000 7f782cc9d640 1 -- 172.21.15.42:0/1154901797 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f78140b4bb0 msgr2=0x7f78140b70c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:57:59.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:59.372+0000 7f782cc9d640 1 --2- 172.21.15.42:0/1154901797 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f78140b4bb0 0x7f78140b70c0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f7818009990 tx=0x7f7818009110 comp rx=0 tx=0).stop 2024-08-30T08:57:59.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:59.372+0000 7f782cc9d640 1 -- 172.21.15.42:0/1154901797 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7828108a90 msgr2=0x7f782819fb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:57:59.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:59.372+0000 7f782cc9d640 1 --2- 172.21.15.42:0/1154901797 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7828108a90 0x7f782819fb30 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f78200098d0 tx=0x7f78200317b0 comp rx=0 tx=0).stop 2024-08-30T08:57:59.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:59.372+0000 7f782cc9d640 1 -- 172.21.15.42:0/1154901797 shutdown_connections 2024-08-30T08:57:59.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:59.372+0000 7f782cc9d640 1 --2- 172.21.15.42:0/1154901797 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f78140b4bb0 0x7f78140b70c0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:57:59.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:59.372+0000 7f782cc9d640 1 --2- 172.21.15.42:0/1154901797 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7828108a90 0x7f782819fb30 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:57:59.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:59.372+0000 7f782cc9d640 1 --2- 172.21.15.42:0/1154901797 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7828108140 0x7f782819a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:57:59.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:59.372+0000 7f782cc9d640 1 --2- 172.21.15.42:0/1154901797 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7828102800 0x7f782819a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:57:59.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:59.372+0000 7f782cc9d640 1 -- 172.21.15.42:0/1154901797 >> 172.21.15.42:0/1154901797 conn(0x7f78280794d0 msgr2=0x7f78280ff9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:57:59.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:59.372+0000 7f782cc9d640 1 -- 172.21.15.42:0/1154901797 shutdown_connections 2024-08-30T08:57:59.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:57:59.372+0000 7f782cc9d640 1 -- 172.21.15.42:0/1154901797 wait complete. 2024-08-30T08:57:59.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:59 smithi042 ceph-mon[18652]: from='client.? 172.21.15.42:0/1154901797' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} : dispatch 2024-08-30T08:57:59.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:59 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:59.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:59 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:59.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:59 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:57:59.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:59 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-30T08:57:59.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:57:59 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:59.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:59 smithi190 ceph-mon[22001]: from='client.? 172.21.15.42:0/1154901797' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} : dispatch 2024-08-30T08:57:59.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:59 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:59.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:59 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:59.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:59 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:57:59.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:59 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-30T08:57:59.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:57:59 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:59.828 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:59 smithi167 ceph-mon[22063]: from='client.? 172.21.15.42:0/1154901797' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} : dispatch 2024-08-30T08:57:59.828 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:59 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:59.829 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:59 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:57:59.829 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:59 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:57:59.829 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:59 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-30T08:57:59.829 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:57:59 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:58:00.048 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-08-30T08:58:00.309 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T08:58:00.621 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:58:00 smithi042 ceph-mon[18652]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-30T08:58:00.622 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:58:00 smithi042 ceph-mon[18652]: from='client.? 172.21.15.42:0/1154901797' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-08-30T08:58:00.622 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:58:00 smithi042 ceph-mon[18652]: osdmap e69: 12 total, 12 up, 12 in 2024-08-30T08:58:00.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:58:00 smithi190 ceph-mon[22001]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-30T08:58:00.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:58:00 smithi190 ceph-mon[22001]: from='client.? 172.21.15.42:0/1154901797' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-08-30T08:58:00.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:58:00 smithi190 ceph-mon[22001]: osdmap e69: 12 total, 12 up, 12 in 2024-08-30T08:58:00.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:58:00 smithi167 ceph-mon[22063]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-30T08:58:00.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:58:00 smithi167 ceph-mon[22063]: from='client.? 172.21.15.42:0/1154901797' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-08-30T08:58:00.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:58:00 smithi167 ceph-mon[22063]: osdmap e69: 12 total, 12 up, 12 in 2024-08-30T08:58:00.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:00.863+0000 7fb9a8125640 1 -- 172.21.15.42:0/2082910512 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb9a0109b90 msgr2=0x7fb9a01103c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:58:00.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:00.863+0000 7fb9a8125640 1 --2- 172.21.15.42:0/2082910512 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb9a0109b90 0x7fb9a01103c0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fb998009ca0 tx=0x7fb99802a8c0 comp rx=0 tx=0).stop 2024-08-30T08:58:00.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:00.863+0000 7fb9a8125640 1 -- 172.21.15.42:0/2082910512 shutdown_connections 2024-08-30T08:58:00.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:00.863+0000 7fb9a8125640 1 --2- 172.21.15.42:0/2082910512 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb9a0109240 0x7fb9a0109650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:00.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:00.863+0000 7fb9a8125640 1 --2- 172.21.15.42:0/2082910512 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb9a0109b90 0x7fb9a01103c0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:00.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:00.863+0000 7fb9a8125640 1 --2- 172.21.15.42:0/2082910512 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb9a0103900 0x7fb9a0103d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:00.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:00.863+0000 7fb9a8125640 1 -- 172.21.15.42:0/2082910512 >> 172.21.15.42:0/2082910512 conn(0x7fb9a007b5f0 msgr2=0x7fb9a007ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:58:00.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:00.863+0000 7fb9a8125640 1 -- 172.21.15.42:0/2082910512 shutdown_connections 2024-08-30T08:58:00.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:00.863+0000 7fb9a8125640 1 -- 172.21.15.42:0/2082910512 wait complete. 2024-08-30T08:58:00.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:00.864+0000 7fb9a8125640 1 Processor -- start 2024-08-30T08:58:00.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:00.864+0000 7fb9a8125640 1 -- start start 2024-08-30T08:58:00.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:00.864+0000 7fb9a8125640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb9a0103900 0x7fb9a00754b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:58:00.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:00.864+0000 7fb9a8125640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb9a0109240 0x7fb9a00759f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:58:00.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:00.864+0000 7fb9a8125640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb9a0109b90 0x7fb9a0075f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:58:00.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:00.864+0000 7fb9a8125640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fb9a01138d0 con 0x7fb9a0109240 2024-08-30T08:58:00.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:00.864+0000 7fb9a8125640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb9a0113750 con 0x7fb9a0103900 2024-08-30T08:58:00.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:00.864+0000 7fb9a8125640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fb9a0113a50 con 0x7fb9a0109b90 2024-08-30T08:58:00.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:00.864+0000 7fb9a56c0640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb9a0109240 0x7fb9a00759f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:58:00.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:00.864+0000 7fb9a56c0640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb9a0109240 0x7fb9a00759f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42066/0 (socket says 172.21.15.42:42066) 2024-08-30T08:58:00.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:00.864+0000 7fb9a56c0640 1 -- 172.21.15.42:0/2076892428 learned_addr learned my addr 172.21.15.42:0/2076892428 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T08:58:00.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:00.864+0000 7fb9a5ec1640 1 --2- 172.21.15.42:0/2076892428 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb9a0103900 0x7fb9a00754b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:58:00.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:00.864+0000 7fb9a56c0640 1 -- 172.21.15.42:0/2076892428 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb9a0109b90 msgr2=0x7fb9a0075f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:58:00.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:00.864+0000 7fb9a56c0640 1 --2- 172.21.15.42:0/2076892428 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb9a0109b90 0x7fb9a0075f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:00.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:00.864+0000 7fb9a56c0640 1 -- 172.21.15.42:0/2076892428 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb9a0103900 msgr2=0x7fb9a00754b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:58:00.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:00.864+0000 7fb9a56c0640 1 --2- 172.21.15.42:0/2076892428 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb9a0103900 0x7fb9a00754b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:00.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:00.864+0000 7fb9a56c0640 1 -- 172.21.15.42:0/2076892428 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9a01a8500 con 0x7fb9a0109240 2024-08-30T08:58:00.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:00.865+0000 7fb9a56c0640 1 --2- 172.21.15.42:0/2076892428 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb9a0109240 0x7fb9a00759f0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fb99000d750 tx=0x7fb99000db50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:58:00.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:00.865+0000 7fb996ffd640 1 -- 172.21.15.42:0/2076892428 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb990017070 con 0x7fb9a0109240 2024-08-30T08:58:00.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:00.865+0000 7fb996ffd640 1 -- 172.21.15.42:0/2076892428 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb990009820 con 0x7fb9a0109240 2024-08-30T08:58:00.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:00.865+0000 7fb9a8125640 1 -- 172.21.15.42:0/2076892428 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9a01a87f0 con 0x7fb9a0109240 2024-08-30T08:58:00.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:00.865+0000 7fb996ffd640 1 -- 172.21.15.42:0/2076892428 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb990004c40 con 0x7fb9a0109240 2024-08-30T08:58:00.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:00.865+0000 7fb9a8125640 1 -- 172.21.15.42:0/2076892428 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9a01a8db0 con 0x7fb9a0109240 2024-08-30T08:58:00.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:00.866+0000 7fb9a8125640 1 -- 172.21.15.42:0/2076892428 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9a0103d10 con 0x7fb9a0109240 2024-08-30T08:58:00.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:00.867+0000 7fb996ffd640 1 -- 172.21.15.42:0/2076892428 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) ==== 152577+0+0 (secure 0 0 0) 0x7fb9900099d0 con 0x7fb9a0109240 2024-08-30T08:58:00.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:00.868+0000 7fb996ffd640 1 --2- 172.21.15.42:0/2076892428 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fb98c0b4bb0 0x7fb98c0b70c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:58:00.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:00.868+0000 7fb9a5ec1640 1 --2- 172.21.15.42:0/2076892428 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fb98c0b4bb0 0x7fb98c0b70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:58:00.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:00.868+0000 7fb996ffd640 1 -- 172.21.15.42:0/2076892428 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7497+0+0 (secure 0 0 0) 0x7fb9900e4c10 con 0x7fb9a0109240 2024-08-30T08:58:00.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:00.869+0000 7fb9a5ec1640 1 --2- 172.21.15.42:0/2076892428 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fb98c0b4bb0 0x7fb98c0b70c0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fb9a0076f00 tx=0x7fb99c009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:58:00.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:00.872+0000 7fb996ffd640 1 -- 172.21.15.42:0/2076892428 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7fb9900ac050 con 0x7fb9a0109240 2024-08-30T08:58:01.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:01.060+0000 7fb9a8125640 1 -- 172.21.15.42:0/2076892428 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) -- 0x7fb9a002e430 con 0x7fb9a0109240 2024-08-30T08:58:01.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:01.381+0000 7fb996ffd640 1 -- 172.21.15.42:0/2076892428 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v71) ==== 171+0+0 (secure 0 0 0) 0x7fb9900ab950 con 0x7fb9a0109240 2024-08-30T08:58:01.384 INFO:teuthology.orchestra.run.smithi042.stderr:pool 'cephfs_data' created 2024-08-30T08:58:01.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:01.384+0000 7fb9a8125640 1 -- 172.21.15.42:0/2076892428 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fb98c0b4bb0 msgr2=0x7fb98c0b70c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:58:01.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:01.384+0000 7fb9a8125640 1 --2- 172.21.15.42:0/2076892428 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fb98c0b4bb0 0x7fb98c0b70c0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fb9a0076f00 tx=0x7fb99c009110 comp rx=0 tx=0).stop 2024-08-30T08:58:01.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:01.384+0000 7fb9a8125640 1 -- 172.21.15.42:0/2076892428 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb9a0109240 msgr2=0x7fb9a00759f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:58:01.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:01.384+0000 7fb9a8125640 1 --2- 172.21.15.42:0/2076892428 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb9a0109240 0x7fb9a00759f0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fb99000d750 tx=0x7fb99000db50 comp rx=0 tx=0).stop 2024-08-30T08:58:01.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:01.384+0000 7fb9a8125640 1 -- 172.21.15.42:0/2076892428 shutdown_connections 2024-08-30T08:58:01.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:01.384+0000 7fb9a8125640 1 --2- 172.21.15.42:0/2076892428 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fb98c0b4bb0 0x7fb98c0b70c0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:01.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:01.384+0000 7fb9a8125640 1 --2- 172.21.15.42:0/2076892428 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb9a0109b90 0x7fb9a0075f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:01.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:01.384+0000 7fb9a8125640 1 --2- 172.21.15.42:0/2076892428 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb9a0109240 0x7fb9a00759f0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:01.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:01.384+0000 7fb9a8125640 1 --2- 172.21.15.42:0/2076892428 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb9a0103900 0x7fb9a00754b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:01.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:01.384+0000 7fb9a8125640 1 -- 172.21.15.42:0/2076892428 >> 172.21.15.42:0/2076892428 conn(0x7fb9a007b5f0 msgr2=0x7fb9a0107250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:58:01.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:01.385+0000 7fb9a8125640 1 -- 172.21.15.42:0/2076892428 shutdown_connections 2024-08-30T08:58:01.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:01.385+0000 7fb9a8125640 1 -- 172.21.15.42:0/2076892428 wait complete. 2024-08-30T08:58:01.681 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:58:01 smithi042 ceph-mon[18652]: osdmap e70: 12 total, 12 up, 12 in 2024-08-30T08:58:01.681 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:58:01 smithi042 ceph-mon[18652]: from='client.? 172.21.15.42:0/2076892428' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} : dispatch 2024-08-30T08:58:01.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:58:01 smithi190 ceph-mon[22001]: osdmap e70: 12 total, 12 up, 12 in 2024-08-30T08:58:01.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:58:01 smithi190 ceph-mon[22001]: from='client.? 172.21.15.42:0/2076892428' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} : dispatch 2024-08-30T08:58:01.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:58:01 smithi167 ceph-mon[22063]: osdmap e70: 12 total, 12 up, 12 in 2024-08-30T08:58:01.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:58:01 smithi167 ceph-mon[22063]: from='client.? 172.21.15.42:0/2076892428' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} : dispatch 2024-08-30T08:58:02.017 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph fs new cephfs cephfs_metadata cephfs_data 2024-08-30T08:58:02.281 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T08:58:02.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:58:02 smithi190 ceph-mon[22001]: pgmap v278: 33 pgs: 30 unknown, 3 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-30T08:58:02.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:58:02 smithi190 ceph-mon[22001]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-08-30T08:58:02.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:58:02 smithi190 ceph-mon[22001]: from='client.? 172.21.15.42:0/2076892428' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-08-30T08:58:02.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:58:02 smithi190 ceph-mon[22001]: osdmap e71: 12 total, 12 up, 12 in 2024-08-30T08:58:02.696 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:58:02 smithi042 ceph-mon[18652]: pgmap v278: 33 pgs: 30 unknown, 3 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-30T08:58:02.696 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:58:02 smithi042 ceph-mon[18652]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-08-30T08:58:02.696 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:58:02 smithi042 ceph-mon[18652]: from='client.? 172.21.15.42:0/2076892428' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-08-30T08:58:02.696 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:58:02 smithi042 ceph-mon[18652]: osdmap e71: 12 total, 12 up, 12 in 2024-08-30T08:58:02.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:58:02 smithi167 ceph-mon[22063]: pgmap v278: 33 pgs: 30 unknown, 3 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-30T08:58:02.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:58:02 smithi167 ceph-mon[22063]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-08-30T08:58:02.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:58:02 smithi167 ceph-mon[22063]: from='client.? 172.21.15.42:0/2076892428' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-08-30T08:58:02.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:58:02 smithi167 ceph-mon[22063]: osdmap e71: 12 total, 12 up, 12 in 2024-08-30T08:58:02.843 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:02.820+0000 7f3669bd5640 1 -- 172.21.15.42:0/248721034 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3664108a90 msgr2=0x7f366410f2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:58:02.843 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:02.820+0000 7f3669bd5640 1 --2- 172.21.15.42:0/248721034 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3664108a90 0x7f366410f2c0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f365c009ca0 tx=0x7f365c02a8c0 comp rx=0 tx=0).stop 2024-08-30T08:58:02.843 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:02.820+0000 7f3669bd5640 1 -- 172.21.15.42:0/248721034 shutdown_connections 2024-08-30T08:58:02.843 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:02.820+0000 7f3669bd5640 1 --2- 172.21.15.42:0/248721034 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3664108a90 0x7f366410f2c0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:02.843 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:02.820+0000 7f3669bd5640 1 --2- 172.21.15.42:0/248721034 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3664108140 0x7f3664108550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:02.843 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:02.820+0000 7f3669bd5640 1 --2- 172.21.15.42:0/248721034 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3664102800 0x7f3664102c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:02.843 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:02.820+0000 7f3669bd5640 1 -- 172.21.15.42:0/248721034 >> 172.21.15.42:0/248721034 conn(0x7f36640794c0 msgr2=0x7f36640798d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:58:02.843 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:02.820+0000 7f3669bd5640 1 -- 172.21.15.42:0/248721034 shutdown_connections 2024-08-30T08:58:02.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:02.820+0000 7f3669bd5640 1 -- 172.21.15.42:0/248721034 wait complete. 2024-08-30T08:58:02.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:02.821+0000 7f3669bd5640 1 Processor -- start 2024-08-30T08:58:02.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:02.821+0000 7f3669bd5640 1 -- start start 2024-08-30T08:58:02.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:02.822+0000 7f3669bd5640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3664102800 0x7f366419b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:58:02.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:02.822+0000 7f3669bd5640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3664108140 0x7f366419b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:58:02.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:02.822+0000 7f3669bd5640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3664108a90 0x7f366419bb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:58:02.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:02.822+0000 7f3669bd5640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f36641126f0 con 0x7f3664108140 2024-08-30T08:58:02.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:02.822+0000 7f3669bd5640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f3664112570 con 0x7f3664108a90 2024-08-30T08:58:02.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:02.822+0000 7f3669bd5640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f3664112870 con 0x7f3664102800 2024-08-30T08:58:02.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:02.822+0000 7f3662ffd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3664108140 0x7f366419b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:58:02.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:02.822+0000 7f3662ffd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3664108140 0x7f366419b600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42074/0 (socket says 172.21.15.42:42074) 2024-08-30T08:58:02.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:02.822+0000 7f3662ffd640 1 -- 172.21.15.42:0/1070778053 learned_addr learned my addr 172.21.15.42:0/1070778053 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T08:58:02.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:02.822+0000 7f36637fe640 1 --2- 172.21.15.42:0/1070778053 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3664102800 0x7f366419b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:58:02.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:02.822+0000 7f3663fff640 1 --2- 172.21.15.42:0/1070778053 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3664108a90 0x7f366419bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:58:02.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:02.822+0000 7f3662ffd640 1 -- 172.21.15.42:0/1070778053 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3664102800 msgr2=0x7f366419b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:58:02.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:02.822+0000 7f3662ffd640 1 --2- 172.21.15.42:0/1070778053 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3664102800 0x7f366419b0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:02.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:02.822+0000 7f3662ffd640 1 -- 172.21.15.42:0/1070778053 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3664108a90 msgr2=0x7f366419bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:58:02.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:02.822+0000 7f3662ffd640 1 --2- 172.21.15.42:0/1070778053 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3664108a90 0x7f366419bb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:02.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:02.822+0000 7f3662ffd640 1 -- 172.21.15.42:0/1070778053 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36641a6350 con 0x7f3664108140 2024-08-30T08:58:02.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:02.822+0000 7f36637fe640 1 --2- 172.21.15.42:0/1070778053 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3664102800 0x7f366419b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-30T08:58:02.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:02.822+0000 7f3662ffd640 1 --2- 172.21.15.42:0/1070778053 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3664108140 0x7f366419b600 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f365800d770 tx=0x7f365800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:58:02.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:02.822+0000 7f3660ff9640 1 -- 172.21.15.42:0/1070778053 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3658017070 con 0x7f3664108140 2024-08-30T08:58:02.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:02.822+0000 7f3669bd5640 1 -- 172.21.15.42:0/1070778053 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36641a6640 con 0x7f3664108140 2024-08-30T08:58:02.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:02.822+0000 7f3660ff9640 1 -- 172.21.15.42:0/1070778053 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3658009c30 con 0x7f3664108140 2024-08-30T08:58:02.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:02.822+0000 7f3660ff9640 1 -- 172.21.15.42:0/1070778053 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f365800f620 con 0x7f3664108140 2024-08-30T08:58:02.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:02.823+0000 7f3669bd5640 1 -- 172.21.15.42:0/1070778053 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36641a6bd0 con 0x7f3664108140 2024-08-30T08:58:02.846 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:02.823+0000 7f3669bd5640 1 -- 172.21.15.42:0/1070778053 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3664103a80 con 0x7f3664108140 2024-08-30T08:58:02.846 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:02.825+0000 7f3660ff9640 1 -- 172.21.15.42:0/1070778053 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) ==== 152577+0+0 (secure 0 0 0) 0x7f365801a070 con 0x7f3664108140 2024-08-30T08:58:02.846 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:02.826+0000 7f3660ff9640 1 --2- 172.21.15.42:0/1070778053 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f364c0b4b60 0x7f364c0b7070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:58:02.846 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:02.826+0000 7f36637fe640 1 --2- 172.21.15.42:0/1070778053 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f364c0b4b60 0x7f364c0b7070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:58:02.846 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:02.826+0000 7f3660ff9640 1 -- 172.21.15.42:0/1070778053 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7879+0+0 (secure 0 0 0) 0x7f36580e4a90 con 0x7f3664108140 2024-08-30T08:58:02.846 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:02.826+0000 7f36637fe640 1 --2- 172.21.15.42:0/1070778053 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f364c0b4b60 0x7f364c0b7070 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f36540058c0 tx=0x7f3654005d50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:58:02.846 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:02.829+0000 7f3660ff9640 1 -- 172.21.15.42:0/1070778053 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f36580abd70 con 0x7f3664108140 2024-08-30T08:58:03.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:03.053+0000 7f3669bd5640 1 -- 172.21.15.42:0/1070778053 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f366406bd30 con 0x7f3664108140 2024-08-30T08:58:03.400 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:03.398+0000 7f3660ff9640 1 -- 172.21.15.42:0/1070778053 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7f366406bd30 con 0x7f3664108140 2024-08-30T08:58:03.400 INFO:teuthology.orchestra.run.smithi042.stderr:new fs with metadata pool 2 and data pool 3 2024-08-30T08:58:03.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:03.400+0000 7f3669bd5640 1 -- 172.21.15.42:0/1070778053 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f364c0b4b60 msgr2=0x7f364c0b7070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:58:03.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:03.400+0000 7f3669bd5640 1 --2- 172.21.15.42:0/1070778053 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f364c0b4b60 0x7f364c0b7070 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f36540058c0 tx=0x7f3654005d50 comp rx=0 tx=0).stop 2024-08-30T08:58:03.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:03.400+0000 7f3669bd5640 1 -- 172.21.15.42:0/1070778053 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3664108140 msgr2=0x7f366419b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:58:03.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:03.400+0000 7f3669bd5640 1 --2- 172.21.15.42:0/1070778053 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3664108140 0x7f366419b600 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f365800d770 tx=0x7f365800dc30 comp rx=0 tx=0).stop 2024-08-30T08:58:03.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:03.400+0000 7f3669bd5640 1 -- 172.21.15.42:0/1070778053 shutdown_connections 2024-08-30T08:58:03.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:03.400+0000 7f3669bd5640 1 --2- 172.21.15.42:0/1070778053 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f364c0b4b60 0x7f364c0b7070 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:03.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:03.400+0000 7f3669bd5640 1 --2- 172.21.15.42:0/1070778053 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3664108a90 0x7f366419bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:03.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:03.400+0000 7f3669bd5640 1 --2- 172.21.15.42:0/1070778053 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3664108140 0x7f366419b600 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:03.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:03.400+0000 7f3669bd5640 1 --2- 172.21.15.42:0/1070778053 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3664102800 0x7f366419b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:03.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:03.400+0000 7f3669bd5640 1 -- 172.21.15.42:0/1070778053 >> 172.21.15.42:0/1070778053 conn(0x7f36640794c0 msgr2=0x7f36641041b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:58:03.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:03.400+0000 7f3669bd5640 1 -- 172.21.15.42:0/1070778053 shutdown_connections 2024-08-30T08:58:03.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:03.400+0000 7f3669bd5640 1 -- 172.21.15.42:0/1070778053 wait complete. 2024-08-30T08:58:03.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:58:03 smithi190 ceph-mon[22001]: osdmap e72: 12 total, 12 up, 12 in 2024-08-30T08:58:03.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:58:03 smithi190 ceph-mon[22001]: from='client.? 172.21.15.42:0/1070778053' entity='client.admin' cmd={"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} : dispatch 2024-08-30T08:58:03.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:58:03 smithi190 ceph-mon[22001]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-08-30T08:58:03.789 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:58:03 smithi042 ceph-mon[18652]: osdmap e72: 12 total, 12 up, 12 in 2024-08-30T08:58:03.789 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:58:03 smithi042 ceph-mon[18652]: from='client.? 172.21.15.42:0/1070778053' entity='client.admin' cmd={"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} : dispatch 2024-08-30T08:58:03.789 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:58:03 smithi042 ceph-mon[18652]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-08-30T08:58:03.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:58:03 smithi167 ceph-mon[22063]: osdmap e72: 12 total, 12 up, 12 in 2024-08-30T08:58:03.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:58:03 smithi167 ceph-mon[22063]: from='client.? 172.21.15.42:0/1070778053' entity='client.admin' cmd={"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} : dispatch 2024-08-30T08:58:03.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:58:03 smithi167 ceph-mon[22063]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-08-30T08:58:04.116 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-08-30T08:58:04.117 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-08-30T08:58:04.375 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T08:58:04.648 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:58:04 smithi042 ceph-mon[18652]: pgmap v281: 97 pgs: 22 creating+peering, 32 unknown, 43 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-30T08:58:04.648 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:58:04 smithi042 ceph-mon[18652]: osdmap e73: 12 total, 12 up, 12 in 2024-08-30T08:58:04.648 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:58:04 smithi042 ceph-mon[18652]: from='client.? 172.21.15.42:0/1070778053' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-08-30T08:58:04.648 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:58:04 smithi042 ceph-mon[18652]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-08-30T08:58:04.648 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:58:04 smithi042 ceph-mon[18652]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-08-30T08:58:04.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:58:04 smithi190 ceph-mon[22001]: pgmap v281: 97 pgs: 22 creating+peering, 32 unknown, 43 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-30T08:58:04.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:58:04 smithi190 ceph-mon[22001]: osdmap e73: 12 total, 12 up, 12 in 2024-08-30T08:58:04.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:58:04 smithi190 ceph-mon[22001]: from='client.? 172.21.15.42:0/1070778053' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-08-30T08:58:04.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:58:04 smithi190 ceph-mon[22001]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-08-30T08:58:04.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:58:04 smithi190 ceph-mon[22001]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-08-30T08:58:04.728 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:58:04 smithi167 ceph-mon[22063]: pgmap v281: 97 pgs: 22 creating+peering, 32 unknown, 43 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-30T08:58:04.728 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:58:04 smithi167 ceph-mon[22063]: osdmap e73: 12 total, 12 up, 12 in 2024-08-30T08:58:04.728 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:58:04 smithi167 ceph-mon[22063]: from='client.? 172.21.15.42:0/1070778053' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-08-30T08:58:04.729 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:58:04 smithi167 ceph-mon[22063]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-08-30T08:58:04.729 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:58:04 smithi167 ceph-mon[22063]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-08-30T08:58:04.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:04.888+0000 7efcac5d8640 1 -- 172.21.15.42:0/824928903 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efca4105100 msgr2=0x7efca410d650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:58:04.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:04.888+0000 7efcac5d8640 1 --2- 172.21.15.42:0/824928903 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efca4105100 0x7efca410d650 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7efc94009900 tx=0x7efc9402a410 comp rx=0 tx=0).stop 2024-08-30T08:58:04.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:04.888+0000 7efcac5d8640 1 -- 172.21.15.42:0/824928903 shutdown_connections 2024-08-30T08:58:04.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:04.888+0000 7efcac5d8640 1 --2- 172.21.15.42:0/824928903 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efca410dbe0 0x7efca41100b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:04.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:04.888+0000 7efcac5d8640 1 --2- 172.21.15.42:0/824928903 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efca4105100 0x7efca410d650 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:04.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:04.888+0000 7efcac5d8640 1 --2- 172.21.15.42:0/824928903 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efca41047b0 0x7efca4104bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:04.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:04.888+0000 7efcac5d8640 1 -- 172.21.15.42:0/824928903 >> 172.21.15.42:0/824928903 conn(0x7efca4079530 msgr2=0x7efca4079940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:58:04.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:04.888+0000 7efcac5d8640 1 -- 172.21.15.42:0/824928903 shutdown_connections 2024-08-30T08:58:04.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:04.888+0000 7efcac5d8640 1 -- 172.21.15.42:0/824928903 wait complete. 2024-08-30T08:58:04.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:04.889+0000 7efcac5d8640 1 Processor -- start 2024-08-30T08:58:04.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:04.889+0000 7efcac5d8640 1 -- start start 2024-08-30T08:58:04.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:04.889+0000 7efcac5d8640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efca41047b0 0x7efca419a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:58:04.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:04.889+0000 7efcac5d8640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efca4105100 0x7efca419a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:58:04.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:04.889+0000 7efcac5d8640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efca410dbe0 0x7efca419fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:58:04.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:04.889+0000 7efcac5d8640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7efca41138e0 con 0x7efca410dbe0 2024-08-30T08:58:04.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:04.889+0000 7efcac5d8640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7efca4113760 con 0x7efca4105100 2024-08-30T08:58:04.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:04.889+0000 7efcac5d8640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7efca4113a60 con 0x7efca41047b0 2024-08-30T08:58:04.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:04.889+0000 7efcaab75640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efca410dbe0 0x7efca419fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:58:04.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:04.890+0000 7efcaab75640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efca410dbe0 0x7efca419fbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42100/0 (socket says 172.21.15.42:42100) 2024-08-30T08:58:04.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:04.890+0000 7efcaab75640 1 -- 172.21.15.42:0/1830346928 learned_addr learned my addr 172.21.15.42:0/1830346928 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T08:58:04.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:04.890+0000 7efcaa374640 1 --2- 172.21.15.42:0/1830346928 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efca41047b0 0x7efca419a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:58:04.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:04.890+0000 7efca9b73640 1 --2- 172.21.15.42:0/1830346928 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efca4105100 0x7efca419a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:58:04.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:04.890+0000 7efcaab75640 1 -- 172.21.15.42:0/1830346928 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efca41047b0 msgr2=0x7efca419a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:58:04.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:04.890+0000 7efcaab75640 1 --2- 172.21.15.42:0/1830346928 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efca41047b0 0x7efca419a3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:04.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:04.890+0000 7efcaab75640 1 -- 172.21.15.42:0/1830346928 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efca4105100 msgr2=0x7efca419a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:58:04.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:04.890+0000 7efcaab75640 1 --2- 172.21.15.42:0/1830346928 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efca4105100 0x7efca419a950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:04.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:04.890+0000 7efcaab75640 1 -- 172.21.15.42:0/1830346928 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efca41a0320 con 0x7efca410dbe0 2024-08-30T08:58:04.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:04.890+0000 7efcaab75640 1 --2- 172.21.15.42:0/1830346928 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efca410dbe0 0x7efca419fbf0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7efc9c00b970 tx=0x7efc9c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:58:04.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:04.890+0000 7efc9b7fe640 1 -- 172.21.15.42:0/1830346928 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc9c019070 con 0x7efca410dbe0 2024-08-30T08:58:04.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:04.890+0000 7efc9b7fe640 1 -- 172.21.15.42:0/1830346928 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efc9c014440 con 0x7efca410dbe0 2024-08-30T08:58:04.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:04.890+0000 7efc9b7fe640 1 -- 172.21.15.42:0/1830346928 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc9c0075d0 con 0x7efca410dbe0 2024-08-30T08:58:04.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:04.890+0000 7efcac5d8640 1 -- 172.21.15.42:0/1830346928 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efca41a0610 con 0x7efca410dbe0 2024-08-30T08:58:04.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:04.890+0000 7efcac5d8640 1 -- 172.21.15.42:0/1830346928 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efca41a6d70 con 0x7efca410dbe0 2024-08-30T08:58:04.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:04.891+0000 7efcac5d8640 1 -- 172.21.15.42:0/1830346928 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efca4104c40 con 0x7efca410dbe0 2024-08-30T08:58:04.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:04.893+0000 7efc9b7fe640 1 -- 172.21.15.42:0/1830346928 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) ==== 152577+0+0 (secure 0 0 0) 0x7efc9c01c070 con 0x7efca410dbe0 2024-08-30T08:58:04.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:04.893+0000 7efc9b7fe640 1 --2- 172.21.15.42:0/1830346928 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7efc900b4bb0 0x7efc900b70c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:58:04.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:04.894+0000 7efcaa374640 1 --2- 172.21.15.42:0/1830346928 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7efc900b4bb0 0x7efc900b70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:58:04.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:04.894+0000 7efc9b7fe640 1 -- 172.21.15.42:0/1830346928 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7979+0+0 (secure 0 0 0) 0x7efc9c0e4a00 con 0x7efca410dbe0 2024-08-30T08:58:04.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:04.894+0000 7efcaa374640 1 --2- 172.21.15.42:0/1830346928 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7efc900b4bb0 0x7efc900b70c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7efca0009890 tx=0x7efca0008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:58:04.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:04.897+0000 7efc9b7fe640 1 -- 172.21.15.42:0/1830346928 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7efc9c0abbe0 con 0x7efca410dbe0 2024-08-30T08:58:05.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:05.092+0000 7efcac5d8640 1 -- 172.21.15.42:0/1830346928 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) -- 0x7efca419b4e0 con 0x7efca410dbe0 2024-08-30T08:58:05.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:05.414+0000 7efc9b7fe640 1 -- 172.21.15.42:0/1830346928 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v74) ==== 157+0+0 (secure 0 0 0) 0x7efca419b4e0 con 0x7efca410dbe0 2024-08-30T08:58:05.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:05.417+0000 7efcac5d8640 1 -- 172.21.15.42:0/1830346928 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7efc900b4bb0 msgr2=0x7efc900b70c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:58:05.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:05.417+0000 7efcac5d8640 1 --2- 172.21.15.42:0/1830346928 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7efc900b4bb0 0x7efc900b70c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7efca0009890 tx=0x7efca0008040 comp rx=0 tx=0).stop 2024-08-30T08:58:05.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:05.417+0000 7efcac5d8640 1 -- 172.21.15.42:0/1830346928 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efca410dbe0 msgr2=0x7efca419fbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:58:05.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:05.417+0000 7efcac5d8640 1 --2- 172.21.15.42:0/1830346928 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efca410dbe0 0x7efca419fbf0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7efc9c00b970 tx=0x7efc9c00be30 comp rx=0 tx=0).stop 2024-08-30T08:58:05.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:05.417+0000 7efcac5d8640 1 -- 172.21.15.42:0/1830346928 shutdown_connections 2024-08-30T08:58:05.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:05.417+0000 7efcac5d8640 1 --2- 172.21.15.42:0/1830346928 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7efc900b4bb0 0x7efc900b70c0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:05.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:05.417+0000 7efcac5d8640 1 --2- 172.21.15.42:0/1830346928 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efca410dbe0 0x7efca419fbf0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:05.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:05.417+0000 7efcac5d8640 1 --2- 172.21.15.42:0/1830346928 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efca4105100 0x7efca419a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:05.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:05.417+0000 7efcac5d8640 1 --2- 172.21.15.42:0/1830346928 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efca41047b0 0x7efca419a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:05.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:05.417+0000 7efcac5d8640 1 -- 172.21.15.42:0/1830346928 >> 172.21.15.42:0/1830346928 conn(0x7efca4079530 msgr2=0x7efca4100740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:58:05.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:05.417+0000 7efcac5d8640 1 -- 172.21.15.42:0/1830346928 shutdown_connections 2024-08-30T08:58:05.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:05.417+0000 7efcac5d8640 1 -- 172.21.15.42:0/1830346928 wait complete. 2024-08-30T08:58:05.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:58:05 smithi190 ceph-mon[22001]: mds.? [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] up:active 2024-08-30T08:58:05.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:58:05 smithi190 ceph-mon[22001]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-08-30T08:58:05.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:58:05 smithi190 ceph-mon[22001]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-08-30T08:58:05.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:58:05 smithi190 ceph-mon[22001]: from='client.? 172.21.15.42:0/1830346928' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} : dispatch 2024-08-30T08:58:05.744 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:58:05 smithi167 ceph-mon[22063]: mds.? [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] up:active 2024-08-30T08:58:05.745 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:58:05 smithi167 ceph-mon[22063]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-08-30T08:58:05.745 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:58:05 smithi167 ceph-mon[22063]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-08-30T08:58:05.745 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:58:05 smithi167 ceph-mon[22063]: from='client.? 172.21.15.42:0/1830346928' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} : dispatch 2024-08-30T08:58:05.789 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:58:05 smithi042 ceph-mon[18652]: mds.? [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] up:active 2024-08-30T08:58:05.789 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:58:05 smithi042 ceph-mon[18652]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-08-30T08:58:05.790 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:58:05 smithi042 ceph-mon[18652]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-08-30T08:58:05.790 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:58:05 smithi042 ceph-mon[18652]: from='client.? 172.21.15.42:0/1830346928' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} : dispatch 2024-08-30T08:58:06.139 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-08-30T08:58:06.447 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T08:58:06.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:58:06 smithi190 ceph-mon[22001]: pgmap v283: 97 pgs: 22 creating+peering, 8 unknown, 67 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 225 B/s wr, 0 op/s 2024-08-30T08:58:06.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:58:06 smithi190 ceph-mon[22001]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-08-30T08:58:06.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:58:06 smithi190 ceph-mon[22001]: Cluster is now healthy 2024-08-30T08:58:06.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:58:06 smithi190 ceph-mon[22001]: from='client.? 172.21.15.42:0/1830346928' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-08-30T08:58:06.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:58:06 smithi190 ceph-mon[22001]: osdmap e74: 12 total, 12 up, 12 in 2024-08-30T08:58:06.702 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:58:06 smithi042 ceph-mon[18652]: pgmap v283: 97 pgs: 22 creating+peering, 8 unknown, 67 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 225 B/s wr, 0 op/s 2024-08-30T08:58:06.702 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:58:06 smithi042 ceph-mon[18652]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-08-30T08:58:06.703 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:58:06 smithi042 ceph-mon[18652]: Cluster is now healthy 2024-08-30T08:58:06.703 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:58:06 smithi042 ceph-mon[18652]: from='client.? 172.21.15.42:0/1830346928' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-08-30T08:58:06.703 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:58:06 smithi042 ceph-mon[18652]: osdmap e74: 12 total, 12 up, 12 in 2024-08-30T08:58:06.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:58:06 smithi167 ceph-mon[22063]: pgmap v283: 97 pgs: 22 creating+peering, 8 unknown, 67 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 225 B/s wr, 0 op/s 2024-08-30T08:58:06.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:58:06 smithi167 ceph-mon[22063]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-08-30T08:58:06.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:58:06 smithi167 ceph-mon[22063]: Cluster is now healthy 2024-08-30T08:58:06.828 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:58:06 smithi167 ceph-mon[22063]: from='client.? 172.21.15.42:0/1830346928' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-08-30T08:58:06.828 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:58:06 smithi167 ceph-mon[22063]: osdmap e74: 12 total, 12 up, 12 in 2024-08-30T08:58:06.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:06.941+0000 7f116600b640 1 -- 172.21.15.42:0/756688769 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f116010dcb0 msgr2=0x7f11601101c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:58:06.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:06.941+0000 7f116600b640 1 --2- 172.21.15.42:0/756688769 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f116010dcb0 0x7f11601101c0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f1158009ca0 tx=0x7f115802a8c0 comp rx=0 tx=0).stop 2024-08-30T08:58:06.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:06.941+0000 7f116600b640 1 -- 172.21.15.42:0/756688769 shutdown_connections 2024-08-30T08:58:06.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:06.941+0000 7f116600b640 1 --2- 172.21.15.42:0/756688769 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f116010dcb0 0x7f11601101c0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:06.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:06.941+0000 7f116600b640 1 --2- 172.21.15.42:0/756688769 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f11601051a0 0x7f116010d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:06.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:06.941+0000 7f116600b640 1 --2- 172.21.15.42:0/756688769 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1160104770 0x7f1160104b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:06.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:06.941+0000 7f116600b640 1 -- 172.21.15.42:0/756688769 >> 172.21.15.42:0/756688769 conn(0x7f11600794c0 msgr2=0x7f11600798d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:58:06.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:06.941+0000 7f116600b640 1 -- 172.21.15.42:0/756688769 shutdown_connections 2024-08-30T08:58:06.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:06.941+0000 7f116600b640 1 -- 172.21.15.42:0/756688769 wait complete. 2024-08-30T08:58:06.946 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:06.942+0000 7f116600b640 1 Processor -- start 2024-08-30T08:58:06.946 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:06.942+0000 7f116600b640 1 -- start start 2024-08-30T08:58:06.946 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:06.943+0000 7f116600b640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1160104770 0x7f116019a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:58:06.946 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:06.943+0000 7f116600b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f11601051a0 0x7f116019a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:58:06.946 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:06.943+0000 7f116600b640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f116010dcb0 0x7f116019fb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:58:06.946 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:06.943+0000 7f116600b640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f1160113930 con 0x7f116010dcb0 2024-08-30T08:58:06.947 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:06.943+0000 7f116600b640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f11601137b0 con 0x7f1160104770 2024-08-30T08:58:06.947 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:06.943+0000 7f116600b640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f1160113ab0 con 0x7f11601051a0 2024-08-30T08:58:06.947 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:06.943+0000 7f115ffff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f116010dcb0 0x7f116019fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:58:06.947 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:06.943+0000 7f115f7fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1160104770 0x7f116019a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:58:06.947 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:06.943+0000 7f115ffff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f116010dcb0 0x7f116019fb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42128/0 (socket says 172.21.15.42:42128) 2024-08-30T08:58:06.947 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:06.943+0000 7f115ffff640 1 -- 172.21.15.42:0/3367203121 learned_addr learned my addr 172.21.15.42:0/3367203121 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T08:58:06.947 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:06.943+0000 7f115effd640 1 --2- 172.21.15.42:0/3367203121 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f11601051a0 0x7f116019a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:58:06.947 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:06.943+0000 7f115f7fe640 1 -- 172.21.15.42:0/3367203121 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f11601051a0 msgr2=0x7f116019a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:58:06.947 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:06.943+0000 7f115f7fe640 1 --2- 172.21.15.42:0/3367203121 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f11601051a0 0x7f116019a860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:06.947 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:06.943+0000 7f115f7fe640 1 -- 172.21.15.42:0/3367203121 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f116010dcb0 msgr2=0x7f116019fb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:58:06.947 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:06.943+0000 7f115f7fe640 1 --2- 172.21.15.42:0/3367203121 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f116010dcb0 0x7f116019fb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:06.947 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:06.943+0000 7f115f7fe640 1 -- 172.21.15.42:0/3367203121 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11601a0230 con 0x7f1160104770 2024-08-30T08:58:06.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:06.943+0000 7f115ffff640 1 --2- 172.21.15.42:0/3367203121 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f116010dcb0 0x7f116019fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-30T08:58:06.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:06.943+0000 7f115f7fe640 1 --2- 172.21.15.42:0/3367203121 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1160104770 0x7f116019a2c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f115000d720 tx=0x7f115000dbe0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:58:06.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:06.944+0000 7f115cff9640 1 -- 172.21.15.42:0/3367203121 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1150017070 con 0x7f1160104770 2024-08-30T08:58:06.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:06.944+0000 7f116600b640 1 -- 172.21.15.42:0/3367203121 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11601a0520 con 0x7f1160104770 2024-08-30T08:58:06.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:06.944+0000 7f115cff9640 1 -- 172.21.15.42:0/3367203121 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f115000de30 con 0x7f1160104770 2024-08-30T08:58:06.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:06.944+0000 7f115cff9640 1 -- 172.21.15.42:0/3367203121 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11500047a0 con 0x7f1160104770 2024-08-30T08:58:06.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:06.944+0000 7f116600b640 1 -- 172.21.15.42:0/3367203121 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11601a6c80 con 0x7f1160104770 2024-08-30T08:58:06.949 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:06.945+0000 7f116600b640 1 -- 172.21.15.42:0/3367203121 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f116006bf60 con 0x7f1160104770 2024-08-30T08:58:06.949 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:06.946+0000 7f115cff9640 1 -- 172.21.15.42:0/3367203121 <== mon.2 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 16) ==== 152577+0+0 (secure 0 0 0) 0x7f11500099d0 con 0x7f1160104770 2024-08-30T08:58:06.949 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:06.947+0000 7f115cff9640 1 --2- 172.21.15.42:0/3367203121 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f114c0b4b30 0x7f114c0b7040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:58:06.949 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:06.947+0000 7f115effd640 1 --2- 172.21.15.42:0/3367203121 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f114c0b4b30 0x7f114c0b7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:58:06.949 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:06.947+0000 7f115cff9640 1 -- 172.21.15.42:0/3367203121 <== mon.2 v2:172.21.15.167:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 8277+0+0 (secure 0 0 0) 0x7f11500e5180 con 0x7f1160104770 2024-08-30T08:58:06.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:06.948+0000 7f115effd640 1 --2- 172.21.15.42:0/3367203121 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f114c0b4b30 0x7f114c0b7040 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f116010f970 tx=0x7f1148005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:58:06.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:06.950+0000 7f115cff9640 1 -- 172.21.15.42:0/3367203121 <== mon.2 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f1150020020 con 0x7f1160104770 2024-08-30T08:58:07.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:07.138+0000 7f116600b640 1 -- 172.21.15.42:0/3367203121 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f1160104b80 con 0x7f1160104770 2024-08-30T08:58:07.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:58:07 smithi190 ceph-mon[22001]: from='client.? 172.21.15.42:0/3367203121' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} : dispatch 2024-08-30T08:58:07.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:58:07 smithi190 ceph-mon[22001]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} : dispatch 2024-08-30T08:58:07.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:58:07 smithi167 ceph-mon[22063]: from='client.? 172.21.15.42:0/3367203121' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} : dispatch 2024-08-30T08:58:07.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:58:07 smithi167 ceph-mon[22063]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} : dispatch 2024-08-30T08:58:07.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:58:07 smithi042 ceph-mon[18652]: from='client.? 172.21.15.42:0/3367203121' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} : dispatch 2024-08-30T08:58:07.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:58:07 smithi042 ceph-mon[18652]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} : dispatch 2024-08-30T08:58:08.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:08.415+0000 7f115cff9640 1 -- 172.21.15.42:0/3367203121 <== mon.2 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v76) ==== 228+0+0 (secure 0 0 0) 0x7f1160104b80 con 0x7f1160104770 2024-08-30T08:58:08.418 INFO:teuthology.orchestra.run.smithi042.stderr:pool 'cephfs_data_ec' created 2024-08-30T08:58:08.424 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:08.420+0000 7f116600b640 1 -- 172.21.15.42:0/3367203121 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f114c0b4b30 msgr2=0x7f114c0b7040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:58:08.425 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:08.420+0000 7f116600b640 1 --2- 172.21.15.42:0/3367203121 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f114c0b4b30 0x7f114c0b7040 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f116010f970 tx=0x7f1148005f90 comp rx=0 tx=0).stop 2024-08-30T08:58:08.425 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:08.420+0000 7f116600b640 1 -- 172.21.15.42:0/3367203121 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1160104770 msgr2=0x7f116019a2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:58:08.425 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:08.420+0000 7f116600b640 1 --2- 172.21.15.42:0/3367203121 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1160104770 0x7f116019a2c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f115000d720 tx=0x7f115000dbe0 comp rx=0 tx=0).stop 2024-08-30T08:58:08.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:08.425+0000 7f116600b640 1 -- 172.21.15.42:0/3367203121 shutdown_connections 2024-08-30T08:58:08.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:08.425+0000 7f116600b640 1 --2- 172.21.15.42:0/3367203121 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f114c0b4b30 0x7f114c0b7040 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:08.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:08.425+0000 7f116600b640 1 --2- 172.21.15.42:0/3367203121 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f11601051a0 0x7f116019a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:08.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:08.425+0000 7f116600b640 1 --2- 172.21.15.42:0/3367203121 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f116010dcb0 0x7f116019fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:08.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:08.425+0000 7f116600b640 1 --2- 172.21.15.42:0/3367203121 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1160104770 0x7f116019a2c0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:08.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:08.425+0000 7f116600b640 1 -- 172.21.15.42:0/3367203121 >> 172.21.15.42:0/3367203121 conn(0x7f11600794c0 msgr2=0x7f11600ff310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:58:08.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:08.425+0000 7f116600b640 1 -- 172.21.15.42:0/3367203121 shutdown_connections 2024-08-30T08:58:08.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:08.425+0000 7f116600b640 1 -- 172.21.15.42:0/3367203121 wait complete. 2024-08-30T08:58:08.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:58:08 smithi190 ceph-mon[22001]: pgmap v285: 97 pgs: 12 creating+peering, 85 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s wr, 5 op/s 2024-08-30T08:58:08.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:58:08 smithi190 ceph-mon[22001]: osdmap e75: 12 total, 12 up, 12 in 2024-08-30T08:58:08.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:58:08 smithi042 ceph-mon[18652]: pgmap v285: 97 pgs: 12 creating+peering, 85 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s wr, 5 op/s 2024-08-30T08:58:08.817 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:58:08 smithi042 ceph-mon[18652]: osdmap e75: 12 total, 12 up, 12 in 2024-08-30T08:58:08.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:58:08 smithi167 ceph-mon[22063]: pgmap v285: 97 pgs: 12 creating+peering, 85 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s wr, 5 op/s 2024-08-30T08:58:08.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:58:08 smithi167 ceph-mon[22063]: osdmap e75: 12 total, 12 up, 12 in 2024-08-30T08:58:08.827 INFO:journalctl@ceph.mds.b.smithi167.stdout:Aug 30 08:58:08 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mds-b[52473]: 2024-08-30T08:58:08.608+0000 7f9d3d6df640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-08-30T08:58:09.133 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-08-30T08:58:09.390 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T08:58:09.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:58:09 smithi190 ceph-mon[22001]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-08-30T08:58:09.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:58:09 smithi190 ceph-mon[22001]: osdmap e76: 12 total, 12 up, 12 in 2024-08-30T08:58:09.773 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:58:09 smithi042 ceph-mon[18652]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-08-30T08:58:09.773 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:58:09 smithi042 ceph-mon[18652]: osdmap e76: 12 total, 12 up, 12 in 2024-08-30T08:58:09.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:58:09 smithi167 ceph-mon[22063]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-08-30T08:58:09.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:58:09 smithi167 ceph-mon[22063]: osdmap e76: 12 total, 12 up, 12 in 2024-08-30T08:58:09.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:09.866+0000 7f656a865640 1 -- 172.21.15.42:0/3943196383 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6564108b10 msgr2=0x7f656410f340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:58:09.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:09.866+0000 7f656a865640 1 --2- 172.21.15.42:0/3943196383 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6564108b10 0x7f656410f340 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f655c009ca0 tx=0x7f655c02a8c0 comp rx=0 tx=0).stop 2024-08-30T08:58:09.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:09.866+0000 7f656a865640 1 -- 172.21.15.42:0/3943196383 shutdown_connections 2024-08-30T08:58:09.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:09.866+0000 7f656a865640 1 --2- 172.21.15.42:0/3943196383 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f65641081c0 0x7f65641085d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:09.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:09.866+0000 7f656a865640 1 --2- 172.21.15.42:0/3943196383 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6564108b10 0x7f656410f340 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:09.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:09.866+0000 7f656a865640 1 --2- 172.21.15.42:0/3943196383 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6564102880 0x7f6564102c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:09.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:09.866+0000 7f656a865640 1 -- 172.21.15.42:0/3943196383 >> 172.21.15.42:0/3943196383 conn(0x7f6564079510 msgr2=0x7f6564079920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:58:09.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:09.866+0000 7f656a865640 1 -- 172.21.15.42:0/3943196383 shutdown_connections 2024-08-30T08:58:09.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:09.866+0000 7f656a865640 1 -- 172.21.15.42:0/3943196383 wait complete. 2024-08-30T08:58:09.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:09.867+0000 7f656a865640 1 Processor -- start 2024-08-30T08:58:09.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:09.867+0000 7f656a865640 1 -- start start 2024-08-30T08:58:09.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:09.867+0000 7f656a865640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6564102880 0x7f65640ff410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:58:09.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:09.867+0000 7f656a865640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f65641081c0 0x7f65640ff950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:58:09.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:09.867+0000 7f656a865640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6564108b10 0x7f65640ffe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:58:09.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:09.867+0000 7f656a865640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f65641127e0 con 0x7f65641081c0 2024-08-30T08:58:09.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:09.867+0000 7f656a865640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f6564112660 con 0x7f6564102880 2024-08-30T08:58:09.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:09.867+0000 7f656a865640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f6564112960 con 0x7f6564108b10 2024-08-30T08:58:09.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:09.868+0000 7f65637fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f65641081c0 0x7f65640ff950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:58:09.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:09.868+0000 7f65637fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f65641081c0 0x7f65640ff950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:49040/0 (socket says 172.21.15.42:49040) 2024-08-30T08:58:09.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:09.868+0000 7f65637fe640 1 -- 172.21.15.42:0/3076449328 learned_addr learned my addr 172.21.15.42:0/3076449328 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T08:58:09.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:09.868+0000 7f6563fff640 1 --2- 172.21.15.42:0/3076449328 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6564102880 0x7f65640ff410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:58:09.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:09.868+0000 7f65637fe640 1 -- 172.21.15.42:0/3076449328 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6564108b10 msgr2=0x7f65640ffe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:58:09.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:09.868+0000 7f65637fe640 1 --2- 172.21.15.42:0/3076449328 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6564108b10 0x7f65640ffe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:09.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:09.868+0000 7f65637fe640 1 -- 172.21.15.42:0/3076449328 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6564102880 msgr2=0x7f65640ff410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:58:09.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:09.868+0000 7f65637fe640 1 --2- 172.21.15.42:0/3076449328 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6564102880 0x7f65640ff410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:09.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:09.868+0000 7f65637fe640 1 -- 172.21.15.42:0/3076449328 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65641a8520 con 0x7f65641081c0 2024-08-30T08:58:09.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:09.868+0000 7f6563fff640 1 --2- 172.21.15.42:0/3076449328 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6564102880 0x7f65640ff410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-30T08:58:09.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:09.868+0000 7f65637fe640 1 --2- 172.21.15.42:0/3076449328 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f65641081c0 0x7f65640ff950 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f655800d750 tx=0x7f655800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:58:09.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:09.868+0000 7f65617fa640 1 -- 172.21.15.42:0/3076449328 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6558017070 con 0x7f65641081c0 2024-08-30T08:58:09.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:09.868+0000 7f656a865640 1 -- 172.21.15.42:0/3076449328 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65641a8810 con 0x7f65641081c0 2024-08-30T08:58:09.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:09.868+0000 7f65617fa640 1 -- 172.21.15.42:0/3076449328 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f65580054d0 con 0x7f65641081c0 2024-08-30T08:58:09.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:09.868+0000 7f65617fa640 1 -- 172.21.15.42:0/3076449328 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6558004c30 con 0x7f65641081c0 2024-08-30T08:58:09.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:09.868+0000 7f656a865640 1 -- 172.21.15.42:0/3076449328 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65641a8d00 con 0x7f65641081c0 2024-08-30T08:58:09.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:09.869+0000 7f656a865640 1 -- 172.21.15.42:0/3076449328 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6564103b00 con 0x7f65641081c0 2024-08-30T08:58:09.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:09.871+0000 7f65617fa640 1 -- 172.21.15.42:0/3076449328 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) ==== 152577+0+0 (secure 0 0 0) 0x7f6558004dd0 con 0x7f65641081c0 2024-08-30T08:58:09.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:09.872+0000 7f65617fa640 1 --2- 172.21.15.42:0/3076449328 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f654c0b4b30 0x7f654c0b7040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:58:09.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:09.872+0000 7f6563fff640 1 --2- 172.21.15.42:0/3076449328 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f654c0b4b30 0x7f654c0b7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:58:09.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:09.872+0000 7f65617fa640 1 -- 172.21.15.42:0/3076449328 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8770+0+0 (secure 0 0 0) 0x7f65580e51e0 con 0x7f65641081c0 2024-08-30T08:58:09.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:09.872+0000 7f6563fff640 1 --2- 172.21.15.42:0/3076449328 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f654c0b4b30 0x7f654c0b7040 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f6554005d00 tx=0x7f6554005ef0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:58:09.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:09.875+0000 7f65617fa640 1 -- 172.21.15.42:0/3076449328 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f65580ac0f0 con 0x7f65641081c0 2024-08-30T08:58:10.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:10.061+0000 7f656a865640 1 -- 172.21.15.42:0/3076449328 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) -- 0x7f6564100710 con 0x7f65641081c0 2024-08-30T08:58:10.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:10.434+0000 7f65617fa640 1 -- 172.21.15.42:0/3076449328 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v78) ==== 169+0+0 (secure 0 0 0) 0x7f65580ab9f0 con 0x7f65641081c0 2024-08-30T08:58:10.436 INFO:teuthology.orchestra.run.smithi042.stderr:set pool 4 allow_ec_overwrites to true 2024-08-30T08:58:10.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:10.437+0000 7f656a865640 1 -- 172.21.15.42:0/3076449328 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f654c0b4b30 msgr2=0x7f654c0b7040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:58:10.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:10.437+0000 7f656a865640 1 --2- 172.21.15.42:0/3076449328 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f654c0b4b30 0x7f654c0b7040 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f6554005d00 tx=0x7f6554005ef0 comp rx=0 tx=0).stop 2024-08-30T08:58:10.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:10.437+0000 7f656a865640 1 -- 172.21.15.42:0/3076449328 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f65641081c0 msgr2=0x7f65640ff950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:58:10.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:10.437+0000 7f656a865640 1 --2- 172.21.15.42:0/3076449328 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f65641081c0 0x7f65640ff950 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f655800d750 tx=0x7f655800dc10 comp rx=0 tx=0).stop 2024-08-30T08:58:10.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:10.437+0000 7f656a865640 1 -- 172.21.15.42:0/3076449328 shutdown_connections 2024-08-30T08:58:10.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:10.437+0000 7f656a865640 1 --2- 172.21.15.42:0/3076449328 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f654c0b4b30 0x7f654c0b7040 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:10.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:10.437+0000 7f656a865640 1 --2- 172.21.15.42:0/3076449328 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6564108b10 0x7f65640ffe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:10.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:10.437+0000 7f656a865640 1 --2- 172.21.15.42:0/3076449328 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f65641081c0 0x7f65640ff950 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:10.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:10.437+0000 7f656a865640 1 --2- 172.21.15.42:0/3076449328 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6564102880 0x7f65640ff410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:10.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:10.437+0000 7f656a865640 1 -- 172.21.15.42:0/3076449328 >> 172.21.15.42:0/3076449328 conn(0x7f6564079510 msgr2=0x7f65641017e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:58:10.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:10.437+0000 7f656a865640 1 -- 172.21.15.42:0/3076449328 shutdown_connections 2024-08-30T08:58:10.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:10.437+0000 7f656a865640 1 -- 172.21.15.42:0/3076449328 wait complete. 2024-08-30T08:58:10.687 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:58:10 smithi190 ceph-mon[22001]: pgmap v288: 129 pgs: 4 creating+peering, 28 unknown, 97 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s wr, 8 op/s 2024-08-30T08:58:10.687 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:58:10 smithi190 ceph-mon[22001]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-08-30T08:58:10.687 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:58:10 smithi190 ceph-mon[22001]: osdmap e77: 12 total, 12 up, 12 in 2024-08-30T08:58:10.687 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:58:10 smithi190 ceph-mon[22001]: from='client.? 172.21.15.42:0/3076449328' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} : dispatch 2024-08-30T08:58:10.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:58:10 smithi167 ceph-mon[22063]: pgmap v288: 129 pgs: 4 creating+peering, 28 unknown, 97 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s wr, 8 op/s 2024-08-30T08:58:10.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:58:10 smithi167 ceph-mon[22063]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-08-30T08:58:10.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:58:10 smithi167 ceph-mon[22063]: osdmap e77: 12 total, 12 up, 12 in 2024-08-30T08:58:10.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:58:10 smithi167 ceph-mon[22063]: from='client.? 172.21.15.42:0/3076449328' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} : dispatch 2024-08-30T08:58:10.861 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:58:10 smithi042 ceph-mon[18652]: pgmap v288: 129 pgs: 4 creating+peering, 28 unknown, 97 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s wr, 8 op/s 2024-08-30T08:58:10.861 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:58:10 smithi042 ceph-mon[18652]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-08-30T08:58:10.861 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:58:10 smithi042 ceph-mon[18652]: osdmap e77: 12 total, 12 up, 12 in 2024-08-30T08:58:10.861 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:58:10 smithi042 ceph-mon[18652]: from='client.? 172.21.15.42:0/3076449328' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} : dispatch 2024-08-30T08:58:11.193 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph fs add_data_pool cephfs cephfs_data_ec 2024-08-30T08:58:11.451 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T08:58:11.750 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:58:11 smithi042 ceph-mon[18652]: from='client.? 172.21.15.42:0/3076449328' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-08-30T08:58:11.750 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:58:11 smithi042 ceph-mon[18652]: osdmap e78: 12 total, 12 up, 12 in 2024-08-30T08:58:11.819 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:58:11 smithi190 ceph-mon[22001]: from='client.? 172.21.15.42:0/3076449328' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-08-30T08:58:11.819 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:58:11 smithi190 ceph-mon[22001]: osdmap e78: 12 total, 12 up, 12 in 2024-08-30T08:58:11.826 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:58:11 smithi167 ceph-mon[22063]: from='client.? 172.21.15.42:0/3076449328' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-08-30T08:58:11.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:58:11 smithi167 ceph-mon[22063]: osdmap e78: 12 total, 12 up, 12 in 2024-08-30T08:58:11.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:11.989+0000 7f261940f640 1 -- 172.21.15.42:0/2234999135 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2614108170 msgr2=0x7f2614108580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:58:11.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:11.989+0000 7f261940f640 1 --2- 172.21.15.42:0/2234999135 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2614108170 0x7f2614108580 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f2608009900 tx=0x7f260802a410 comp rx=0 tx=0).stop 2024-08-30T08:58:11.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:11.989+0000 7f261940f640 1 -- 172.21.15.42:0/2234999135 shutdown_connections 2024-08-30T08:58:11.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:11.989+0000 7f261940f640 1 --2- 172.21.15.42:0/2234999135 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2614108ac0 0x7f261410f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:11.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:11.989+0000 7f261940f640 1 --2- 172.21.15.42:0/2234999135 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2614108170 0x7f2614108580 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:11.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:11.989+0000 7f261940f640 1 --2- 172.21.15.42:0/2234999135 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2614102830 0x7f2614102c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:11.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:11.989+0000 7f261940f640 1 -- 172.21.15.42:0/2234999135 >> 172.21.15.42:0/2234999135 conn(0x7f26140794d0 msgr2=0x7f26140798e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:58:11.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:11.989+0000 7f261940f640 1 -- 172.21.15.42:0/2234999135 shutdown_connections 2024-08-30T08:58:11.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:11.989+0000 7f261940f640 1 -- 172.21.15.42:0/2234999135 wait complete. 2024-08-30T08:58:11.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:11.989+0000 7f261940f640 1 Processor -- start 2024-08-30T08:58:11.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:11.989+0000 7f261940f640 1 -- start start 2024-08-30T08:58:11.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:11.990+0000 7f261940f640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2614102830 0x7f261419b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:58:11.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:11.990+0000 7f261940f640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2614108170 0x7f261419b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:58:11.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:11.990+0000 7f261940f640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2614108ac0 0x7f261419bb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:58:11.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:11.990+0000 7f261940f640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f2614112730 con 0x7f2614108ac0 2024-08-30T08:58:11.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:11.990+0000 7f261940f640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f26141125b0 con 0x7f2614108170 2024-08-30T08:58:11.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:11.990+0000 7f261940f640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f26141128b0 con 0x7f2614102830 2024-08-30T08:58:11.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:11.990+0000 7f26137fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2614108ac0 0x7f261419bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:58:11.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:11.990+0000 7f26137fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2614108ac0 0x7f261419bb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:49048/0 (socket says 172.21.15.42:49048) 2024-08-30T08:58:11.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:11.990+0000 7f26137fe640 1 -- 172.21.15.42:0/2519461611 learned_addr learned my addr 172.21.15.42:0/2519461611 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T08:58:11.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:11.990+0000 7f26127fc640 1 --2- 172.21.15.42:0/2519461611 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2614108170 0x7f261419b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:58:11.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:11.990+0000 7f2612ffd640 1 --2- 172.21.15.42:0/2519461611 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2614102830 0x7f261419b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:58:11.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:11.990+0000 7f26137fe640 1 -- 172.21.15.42:0/2519461611 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2614102830 msgr2=0x7f261419b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:58:11.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:11.990+0000 7f26137fe640 1 --2- 172.21.15.42:0/2519461611 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2614102830 0x7f261419b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:11.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:11.990+0000 7f26137fe640 1 -- 172.21.15.42:0/2519461611 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2614108170 msgr2=0x7f261419b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:58:11.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:11.990+0000 7f26137fe640 1 --2- 172.21.15.42:0/2519461611 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2614108170 0x7f261419b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:11.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:11.990+0000 7f26137fe640 1 -- 172.21.15.42:0/2519461611 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26141a6310 con 0x7f2614108ac0 2024-08-30T08:58:11.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:11.990+0000 7f26127fc640 1 --2- 172.21.15.42:0/2519461611 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2614108170 0x7f261419b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-30T08:58:11.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:11.990+0000 7f2612ffd640 1 --2- 172.21.15.42:0/2519461611 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2614102830 0x7f261419b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-30T08:58:11.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:11.990+0000 7f26137fe640 1 --2- 172.21.15.42:0/2519461611 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2614108ac0 0x7f261419bb00 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f260c00eb10 tx=0x7f260c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:58:11.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:11.991+0000 7f25fbfff640 1 -- 172.21.15.42:0/2519461611 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f260c004b20 con 0x7f2614108ac0 2024-08-30T08:58:11.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:11.991+0000 7f261940f640 1 -- 172.21.15.42:0/2519461611 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26141a6600 con 0x7f2614108ac0 2024-08-30T08:58:11.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:11.991+0000 7f25fbfff640 1 -- 172.21.15.42:0/2519461611 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f260c004cc0 con 0x7f2614108ac0 2024-08-30T08:58:11.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:11.991+0000 7f25fbfff640 1 -- 172.21.15.42:0/2519461611 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f260c010670 con 0x7f2614108ac0 2024-08-30T08:58:11.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:11.991+0000 7f261940f640 1 -- 172.21.15.42:0/2519461611 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26141a6b10 con 0x7f2614108ac0 2024-08-30T08:58:11.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:11.992+0000 7f261940f640 1 -- 172.21.15.42:0/2519461611 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2614103ab0 con 0x7f2614108ac0 2024-08-30T08:58:11.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:11.993+0000 7f25fbfff640 1 -- 172.21.15.42:0/2519461611 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) ==== 152577+0+0 (secure 0 0 0) 0x7f260c010810 con 0x7f2614108ac0 2024-08-30T08:58:11.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:11.994+0000 7f25fbfff640 1 --2- 172.21.15.42:0/2519461611 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f25fc0b4b60 0x7f25fc0b7070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:58:11.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:11.994+0000 7f2612ffd640 1 --2- 172.21.15.42:0/2519461611 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f25fc0b4b60 0x7f25fc0b7070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:58:11.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:11.994+0000 7f25fbfff640 1 -- 172.21.15.42:0/2519461611 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8770+0+0 (secure 0 0 0) 0x7f260c0e5be0 con 0x7f2614108ac0 2024-08-30T08:58:11.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:11.995+0000 7f2612ffd640 1 --2- 172.21.15.42:0/2519461611 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f25fc0b4b60 0x7f25fc0b7070 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f2604005fd0 tx=0x7f2604005ec0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:58:11.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:11.997+0000 7f25fbfff640 1 -- 172.21.15.42:0/2519461611 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f260c056130 con 0x7f2614108ac0 2024-08-30T08:58:12.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:12.221+0000 7f261940f640 1 -- 172.21.15.42:0/2519461611 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) -- 0x7f2614102c40 con 0x7f2614108ac0 2024-08-30T08:58:12.450 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:12.448+0000 7f25fbfff640 1 -- 172.21.15.42:0/2519461611 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v17) ==== 137+0+0 (secure 0 0 0) 0x7f260c0acaf0 con 0x7f2614108ac0 2024-08-30T08:58:12.450 INFO:teuthology.orchestra.run.smithi042.stderr:added data pool 4 to fsmap 2024-08-30T08:58:12.452 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:12.450+0000 7f261940f640 1 -- 172.21.15.42:0/2519461611 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f25fc0b4b60 msgr2=0x7f25fc0b7070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:58:12.452 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:12.450+0000 7f261940f640 1 --2- 172.21.15.42:0/2519461611 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f25fc0b4b60 0x7f25fc0b7070 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f2604005fd0 tx=0x7f2604005ec0 comp rx=0 tx=0).stop 2024-08-30T08:58:12.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:12.450+0000 7f261940f640 1 -- 172.21.15.42:0/2519461611 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2614108ac0 msgr2=0x7f261419bb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:58:12.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:12.450+0000 7f261940f640 1 --2- 172.21.15.42:0/2519461611 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2614108ac0 0x7f261419bb00 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f260c00eb10 tx=0x7f260c00c2d0 comp rx=0 tx=0).stop 2024-08-30T08:58:12.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:12.450+0000 7f261940f640 1 -- 172.21.15.42:0/2519461611 shutdown_connections 2024-08-30T08:58:12.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:12.450+0000 7f261940f640 1 --2- 172.21.15.42:0/2519461611 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f25fc0b4b60 0x7f25fc0b7070 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:12.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:12.450+0000 7f261940f640 1 --2- 172.21.15.42:0/2519461611 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2614108ac0 0x7f261419bb00 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:12.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:12.450+0000 7f261940f640 1 --2- 172.21.15.42:0/2519461611 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2614108170 0x7f261419b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:12.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:12.450+0000 7f261940f640 1 --2- 172.21.15.42:0/2519461611 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2614102830 0x7f261419b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:12.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:12.450+0000 7f261940f640 1 -- 172.21.15.42:0/2519461611 >> 172.21.15.42:0/2519461611 conn(0x7f26140794d0 msgr2=0x7f2614106220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:58:12.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:12.450+0000 7f261940f640 1 -- 172.21.15.42:0/2519461611 shutdown_connections 2024-08-30T08:58:12.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:12.451+0000 7f261940f640 1 -- 172.21.15.42:0/2519461611 wait complete. 2024-08-30T08:58:12.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:58:12 smithi042 ceph-mon[18652]: pgmap v291: 129 pgs: 6 creating+peering, 24 unknown, 99 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 3 op/s 2024-08-30T08:58:12.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:58:12 smithi042 ceph-mon[18652]: from='client.? 172.21.15.42:0/2519461611' entity='client.admin' cmd={"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} : dispatch 2024-08-30T08:58:12.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:58:12 smithi167 ceph-mon[22063]: pgmap v291: 129 pgs: 6 creating+peering, 24 unknown, 99 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 3 op/s 2024-08-30T08:58:12.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:58:12 smithi167 ceph-mon[22063]: from='client.? 172.21.15.42:0/2519461611' entity='client.admin' cmd={"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} : dispatch 2024-08-30T08:58:12.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:58:12 smithi190 ceph-mon[22001]: pgmap v291: 129 pgs: 6 creating+peering, 24 unknown, 99 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 3 op/s 2024-08-30T08:58:12.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:58:12 smithi190 ceph-mon[22001]: from='client.? 172.21.15.42:0/2519461611' entity='client.admin' cmd={"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} : dispatch 2024-08-30T08:58:13.151 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph fs dump --format=json 2024-08-30T08:58:13.408 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T08:58:13.795 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:58:13 smithi042 ceph-mon[18652]: osdmap e79: 12 total, 12 up, 12 in 2024-08-30T08:58:13.795 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:58:13 smithi042 ceph-mon[18652]: from='client.? 172.21.15.42:0/2519461611' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-08-30T08:58:13.795 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:58:13 smithi042 ceph-mon[18652]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-08-30T08:58:13.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:58:13 smithi167 ceph-mon[22063]: osdmap e79: 12 total, 12 up, 12 in 2024-08-30T08:58:13.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:58:13 smithi167 ceph-mon[22063]: from='client.? 172.21.15.42:0/2519461611' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-08-30T08:58:13.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:58:13 smithi167 ceph-mon[22063]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-08-30T08:58:13.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:13.883+0000 7f0e5bbea640 1 -- 172.21.15.42:0/652516084 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0e54101e20 msgr2=0x7f0e54102230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:58:13.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:13.883+0000 7f0e5bbea640 1 --2- 172.21.15.42:0/652516084 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0e54101e20 0x7f0e54102230 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f0e50009900 tx=0x7f0e5002a410 comp rx=0 tx=0).stop 2024-08-30T08:58:13.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:13.883+0000 7f0e5bbea640 1 -- 172.21.15.42:0/652516084 shutdown_connections 2024-08-30T08:58:13.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:13.883+0000 7f0e5bbea640 1 --2- 172.21.15.42:0/652516084 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0e54106070 0x7f0e54106480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:13.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:13.883+0000 7f0e5bbea640 1 --2- 172.21.15.42:0/652516084 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0e541069c0 0x7f0e54110450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:13.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:13.883+0000 7f0e5bbea640 1 --2- 172.21.15.42:0/652516084 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0e54101e20 0x7f0e54102230 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:13.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:13.883+0000 7f0e5bbea640 1 -- 172.21.15.42:0/652516084 >> 172.21.15.42:0/652516084 conn(0x7f0e54079d30 msgr2=0x7f0e5407a140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:58:13.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:13.884+0000 7f0e5bbea640 1 -- 172.21.15.42:0/652516084 shutdown_connections 2024-08-30T08:58:13.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:13.884+0000 7f0e5bbea640 1 -- 172.21.15.42:0/652516084 wait complete. 2024-08-30T08:58:13.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:13.884+0000 7f0e5bbea640 1 Processor -- start 2024-08-30T08:58:13.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:13.884+0000 7f0e5bbea640 1 -- start start 2024-08-30T08:58:13.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:13.885+0000 7f0e5bbea640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0e54101e20 0x7f0e54102dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:58:13.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:13.885+0000 7f0e5bbea640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0e54106070 0x7f0e54103300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:58:13.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:13.885+0000 7f0e5bbea640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0e541069c0 0x7f0e54103840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:58:13.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:13.885+0000 7f0e5bbea640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f0e54113930 con 0x7f0e54106070 2024-08-30T08:58:13.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:13.885+0000 7f0e5bbea640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f0e541137b0 con 0x7f0e54101e20 2024-08-30T08:58:13.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:13.885+0000 7f0e5bbea640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f0e54113ab0 con 0x7f0e541069c0 2024-08-30T08:58:13.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:13.885+0000 7f0e59185640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0e54106070 0x7f0e54103300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:58:13.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:13.885+0000 7f0e59185640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0e54106070 0x7f0e54103300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:49068/0 (socket says 172.21.15.42:49068) 2024-08-30T08:58:13.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:13.885+0000 7f0e59185640 1 -- 172.21.15.42:0/2432360338 learned_addr learned my addr 172.21.15.42:0/2432360338 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T08:58:13.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:13.885+0000 7f0e5a187640 1 --2- 172.21.15.42:0/2432360338 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0e541069c0 0x7f0e54103840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:58:13.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:13.885+0000 7f0e59986640 1 --2- 172.21.15.42:0/2432360338 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0e54101e20 0x7f0e54102dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:58:13.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:13.885+0000 7f0e59185640 1 -- 172.21.15.42:0/2432360338 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0e541069c0 msgr2=0x7f0e54103840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:58:13.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:13.885+0000 7f0e59185640 1 --2- 172.21.15.42:0/2432360338 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0e541069c0 0x7f0e54103840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:13.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:13.885+0000 7f0e59185640 1 -- 172.21.15.42:0/2432360338 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0e54101e20 msgr2=0x7f0e54102dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:58:13.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:13.885+0000 7f0e59185640 1 --2- 172.21.15.42:0/2432360338 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0e54101e20 0x7f0e54102dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:13.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:13.885+0000 7f0e59185640 1 -- 172.21.15.42:0/2432360338 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e541a8600 con 0x7f0e54106070 2024-08-30T08:58:13.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:13.885+0000 7f0e59185640 1 --2- 172.21.15.42:0/2432360338 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0e54106070 0x7f0e54103300 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f0e4400d750 tx=0x7f0e4400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:58:13.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:13.885+0000 7f0e4affd640 1 -- 172.21.15.42:0/2432360338 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e44017070 con 0x7f0e54106070 2024-08-30T08:58:13.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:13.885+0000 7f0e5bbea640 1 -- 172.21.15.42:0/2432360338 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e541a88f0 con 0x7f0e54106070 2024-08-30T08:58:13.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:13.885+0000 7f0e4affd640 1 -- 172.21.15.42:0/2432360338 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e440054d0 con 0x7f0e54106070 2024-08-30T08:58:13.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:13.886+0000 7f0e4affd640 1 -- 172.21.15.42:0/2432360338 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e44004c30 con 0x7f0e54106070 2024-08-30T08:58:13.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:13.886+0000 7f0e5bbea640 1 -- 172.21.15.42:0/2432360338 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e541a8e30 con 0x7f0e54106070 2024-08-30T08:58:13.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:13.886+0000 7f0e5bbea640 1 -- 172.21.15.42:0/2432360338 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e54102230 con 0x7f0e54106070 2024-08-30T08:58:13.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:13.888+0000 7f0e4affd640 1 -- 172.21.15.42:0/2432360338 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) ==== 152577+0+0 (secure 0 0 0) 0x7f0e44004dd0 con 0x7f0e54106070 2024-08-30T08:58:13.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:13.889+0000 7f0e4affd640 1 --2- 172.21.15.42:0/2432360338 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f0e400b4b30 0x7f0e400b7040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:58:13.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:13.889+0000 7f0e59986640 1 --2- 172.21.15.42:0/2432360338 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f0e400b4b30 0x7f0e400b7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:58:13.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:13.889+0000 7f0e4affd640 1 -- 172.21.15.42:0/2432360338 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8802+0+0 (secure 0 0 0) 0x7f0e440e51f0 con 0x7f0e54106070 2024-08-30T08:58:13.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:13.889+0000 7f0e59986640 1 --2- 172.21.15.42:0/2432360338 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f0e400b4b30 0x7f0e400b7040 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f0e50009230 tx=0x7f0e50009680 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:58:13.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:13.892+0000 7f0e4affd640 1 -- 172.21.15.42:0/2432360338 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f0e440ac0e0 con 0x7f0e54106070 2024-08-30T08:58:13.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:58:13 smithi190 ceph-mon[22001]: osdmap e79: 12 total, 12 up, 12 in 2024-08-30T08:58:13.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:58:13 smithi190 ceph-mon[22001]: from='client.? 172.21.15.42:0/2519461611' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-08-30T08:58:13.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:58:13 smithi190 ceph-mon[22001]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-08-30T08:58:14.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:14.116+0000 7f0e5bbea640 1 -- 172.21.15.42:0/2432360338 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0e541040c0 con 0x7f0e54106070 2024-08-30T08:58:14.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:14.117+0000 7f0e4affd640 1 -- 172.21.15.42:0/2432360338 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+11842 (secure 0 0 0) 0x7f0e541040c0 con 0x7f0e54106070 2024-08-30T08:58:14.119 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-08-30T08:58:14.119 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":17,"btime":"2024-08-30T08:58:12:441452+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14703,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/3188400000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":3188400000},{"type":"v1","addr":"172.21.15.42:6835","nonce":3188400000}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14715,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/3356537674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":3356537674},{"type":"v1","addr":"172.21.15.42:6837","nonce":3356537674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14766,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6839/2107785513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6838","nonce":2107785513},{"type":"v1","addr":"172.21.15.42:6839","nonce":2107785513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14778,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6841/2298438674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6840","nonce":2298438674},{"type":"v1","addr":"172.21.15.42:6841","nonce":2298438674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24553,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/543555570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":543555570},{"type":"v1","addr":"172.21.15.190:6833","nonce":543555570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24554,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/3907818941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":3907818941},{"type":"v1","addr":"172.21.15.167:6833","nonce":3907818941}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24572,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/234596845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":234596845},{"type":"v1","addr":"172.21.15.167:6835","nonce":234596845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24592,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/247905709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":247905709},{"type":"v1","addr":"172.21.15.190:6835","nonce":247905709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24598,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6837/863256349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6836","nonce":863256349},{"type":"v1","addr":"172.21.15.190:6837","nonce":863256349}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24599,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/3011837856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":3011837856},{"type":"v1","addr":"172.21.15.167:6837","nonce":3011837856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24610,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6839/719981110","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6838","nonce":719981110},{"type":"v1","addr":"172.21.15.190:6839","nonce":719981110}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-30T08:58:03.055084+0000","modified":"2024-08-30T08:58:12.223290+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24629},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24629":{"gid":24629,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.167:6839/3604645207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":3604645207},{"type":"v1","addr":"172.21.15.167:6839","nonce":3604645207}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24629,"qdb_cluster":[24629]},"id":1}]} 2024-08-30T08:58:14.119 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 17 2024-08-30T08:58:14.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:14.119+0000 7f0e5bbea640 1 -- 172.21.15.42:0/2432360338 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f0e400b4b30 msgr2=0x7f0e400b7040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:58:14.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:14.119+0000 7f0e5bbea640 1 --2- 172.21.15.42:0/2432360338 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f0e400b4b30 0x7f0e400b7040 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f0e50009230 tx=0x7f0e50009680 comp rx=0 tx=0).stop 2024-08-30T08:58:14.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:14.119+0000 7f0e5bbea640 1 -- 172.21.15.42:0/2432360338 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0e54106070 msgr2=0x7f0e54103300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:58:14.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:14.119+0000 7f0e5bbea640 1 --2- 172.21.15.42:0/2432360338 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0e54106070 0x7f0e54103300 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f0e4400d750 tx=0x7f0e4400dc10 comp rx=0 tx=0).stop 2024-08-30T08:58:14.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:14.119+0000 7f0e5bbea640 1 -- 172.21.15.42:0/2432360338 shutdown_connections 2024-08-30T08:58:14.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:14.119+0000 7f0e5bbea640 1 --2- 172.21.15.42:0/2432360338 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f0e400b4b30 0x7f0e400b7040 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:14.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:14.119+0000 7f0e5bbea640 1 --2- 172.21.15.42:0/2432360338 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0e541069c0 0x7f0e54103840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:14.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:14.119+0000 7f0e5bbea640 1 --2- 172.21.15.42:0/2432360338 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0e54106070 0x7f0e54103300 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:14.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:14.119+0000 7f0e5bbea640 1 --2- 172.21.15.42:0/2432360338 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0e54101e20 0x7f0e54102dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:14.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:14.119+0000 7f0e5bbea640 1 -- 172.21.15.42:0/2432360338 >> 172.21.15.42:0/2432360338 conn(0x7f0e54079d30 msgr2=0x7f0e54107340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:58:14.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:14.119+0000 7f0e5bbea640 1 -- 172.21.15.42:0/2432360338 shutdown_connections 2024-08-30T08:58:14.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:14.119+0000 7f0e5bbea640 1 -- 172.21.15.42:0/2432360338 wait complete. 2024-08-30T08:58:14.750 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:58:14 smithi042 ceph-mon[18652]: pgmap v293: 129 pgs: 6 creating+peering, 1 unknown, 122 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-08-30T08:58:14.750 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:58:14 smithi042 ceph-mon[18652]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-08-30T08:58:14.750 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:58:14 smithi042 ceph-mon[18652]: Cluster is now healthy 2024-08-30T08:58:14.751 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:58:14 smithi042 ceph-mon[18652]: from='client.? 172.21.15.42:0/2432360338' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-30T08:58:14.781 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph osd dump --format=json 2024-08-30T08:58:14.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:58:14 smithi167 ceph-mon[22063]: pgmap v293: 129 pgs: 6 creating+peering, 1 unknown, 122 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-08-30T08:58:14.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:58:14 smithi167 ceph-mon[22063]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-08-30T08:58:14.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:58:14 smithi167 ceph-mon[22063]: Cluster is now healthy 2024-08-30T08:58:14.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:58:14 smithi167 ceph-mon[22063]: from='client.? 172.21.15.42:0/2432360338' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-30T08:58:14.937 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:58:14 smithi190 ceph-mon[22001]: pgmap v293: 129 pgs: 6 creating+peering, 1 unknown, 122 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-08-30T08:58:14.937 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:58:14 smithi190 ceph-mon[22001]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-08-30T08:58:14.937 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:58:14 smithi190 ceph-mon[22001]: Cluster is now healthy 2024-08-30T08:58:14.937 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:58:14 smithi190 ceph-mon[22001]: from='client.? 172.21.15.42:0/2432360338' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-30T08:58:15.038 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T08:58:15.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:15.527+0000 7fa65f0cf640 1 -- 172.21.15.42:0/3400191794 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa658102880 msgr2=0x7fa658102c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:58:15.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:15.527+0000 7fa65f0cf640 1 --2- 172.21.15.42:0/3400191794 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa658102880 0x7fa658102c90 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fa64c009900 tx=0x7fa64c02a410 comp rx=0 tx=0).stop 2024-08-30T08:58:15.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:15.527+0000 7fa65f0cf640 1 -- 172.21.15.42:0/3400191794 shutdown_connections 2024-08-30T08:58:15.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:15.527+0000 7fa65f0cf640 1 --2- 172.21.15.42:0/3400191794 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa6581081c0 0x7fa6581085d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:15.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:15.527+0000 7fa65f0cf640 1 --2- 172.21.15.42:0/3400191794 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa658108b10 0x7fa65810f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:15.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:15.527+0000 7fa65f0cf640 1 --2- 172.21.15.42:0/3400191794 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa658102880 0x7fa658102c90 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:15.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:15.527+0000 7fa65f0cf640 1 -- 172.21.15.42:0/3400191794 >> 172.21.15.42:0/3400191794 conn(0x7fa658079510 msgr2=0x7fa658079920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:58:15.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:15.528+0000 7fa65f0cf640 1 -- 172.21.15.42:0/3400191794 shutdown_connections 2024-08-30T08:58:15.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:15.528+0000 7fa65f0cf640 1 -- 172.21.15.42:0/3400191794 wait complete. 2024-08-30T08:58:15.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:15.528+0000 7fa65f0cf640 1 Processor -- start 2024-08-30T08:58:15.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:15.528+0000 7fa65f0cf640 1 -- start start 2024-08-30T08:58:15.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:15.528+0000 7fa65f0cf640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa658102880 0x7fa6580ff410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:58:15.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:15.529+0000 7fa65f0cf640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa6581081c0 0x7fa6580ff950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:58:15.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:15.529+0000 7fa65f0cf640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa658108b10 0x7fa6580ffe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:58:15.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:15.529+0000 7fa65f0cf640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fa6581127e0 con 0x7fa6581081c0 2024-08-30T08:58:15.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:15.529+0000 7fa65f0cf640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa658112660 con 0x7fa658102880 2024-08-30T08:58:15.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:15.529+0000 7fa65f0cf640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fa658112960 con 0x7fa658108b10 2024-08-30T08:58:15.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:15.529+0000 7fa657fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa6581081c0 0x7fa6580ff950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:58:15.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:15.529+0000 7fa657fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa6581081c0 0x7fa6580ff950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:49086/0 (socket says 172.21.15.42:49086) 2024-08-30T08:58:15.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:15.529+0000 7fa657fff640 1 -- 172.21.15.42:0/1543223640 learned_addr learned my addr 172.21.15.42:0/1543223640 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T08:58:15.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:15.529+0000 7fa65ce6b640 1 --2- 172.21.15.42:0/1543223640 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa658102880 0x7fa6580ff410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:58:15.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:15.529+0000 7fa65d66c640 1 --2- 172.21.15.42:0/1543223640 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa658108b10 0x7fa6580ffe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:58:15.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:15.529+0000 7fa657fff640 1 -- 172.21.15.42:0/1543223640 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa658108b10 msgr2=0x7fa6580ffe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:58:15.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:15.529+0000 7fa657fff640 1 --2- 172.21.15.42:0/1543223640 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa658108b10 0x7fa6580ffe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:15.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:15.529+0000 7fa657fff640 1 -- 172.21.15.42:0/1543223640 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa658102880 msgr2=0x7fa6580ff410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:58:15.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:15.529+0000 7fa657fff640 1 --2- 172.21.15.42:0/1543223640 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa658102880 0x7fa6580ff410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:15.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:15.529+0000 7fa657fff640 1 -- 172.21.15.42:0/1543223640 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6581a85a0 con 0x7fa6581081c0 2024-08-30T08:58:15.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:15.529+0000 7fa657fff640 1 --2- 172.21.15.42:0/1543223640 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa6581081c0 0x7fa6580ff950 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fa64400d750 tx=0x7fa64400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:58:15.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:15.529+0000 7fa655ffb640 1 -- 172.21.15.42:0/1543223640 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa644017070 con 0x7fa6581081c0 2024-08-30T08:58:15.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:15.529+0000 7fa65f0cf640 1 -- 172.21.15.42:0/1543223640 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6581a8890 con 0x7fa6581081c0 2024-08-30T08:58:15.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:15.529+0000 7fa655ffb640 1 -- 172.21.15.42:0/1543223640 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa6440054d0 con 0x7fa6581081c0 2024-08-30T08:58:15.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:15.529+0000 7fa655ffb640 1 -- 172.21.15.42:0/1543223640 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa644004c30 con 0x7fa6581081c0 2024-08-30T08:58:15.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:15.530+0000 7fa65f0cf640 1 -- 172.21.15.42:0/1543223640 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6581a8d80 con 0x7fa6581081c0 2024-08-30T08:58:15.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:15.531+0000 7fa65f0cf640 1 -- 172.21.15.42:0/1543223640 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa658103b00 con 0x7fa6581081c0 2024-08-30T08:58:15.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:15.532+0000 7fa655ffb640 1 -- 172.21.15.42:0/1543223640 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) ==== 152577+0+0 (secure 0 0 0) 0x7fa644004dd0 con 0x7fa6581081c0 2024-08-30T08:58:15.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:15.533+0000 7fa655ffb640 1 --2- 172.21.15.42:0/1543223640 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fa6480b4b30 0x7fa6480b7040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:58:15.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:15.533+0000 7fa65ce6b640 1 --2- 172.21.15.42:0/1543223640 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fa6480b4b30 0x7fa6480b7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:58:15.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:15.533+0000 7fa655ffb640 1 -- 172.21.15.42:0/1543223640 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8802+0+0 (secure 0 0 0) 0x7fa6440e4e80 con 0x7fa6581081c0 2024-08-30T08:58:15.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:15.533+0000 7fa65ce6b640 1 --2- 172.21.15.42:0/1543223640 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fa6480b4b30 0x7fa6480b7040 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fa64c000c00 tx=0x7fa64c031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:58:15.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:15.537+0000 7fa655ffb640 1 -- 172.21.15.42:0/1543223640 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7fa6440abd70 con 0x7fa6581081c0 2024-08-30T08:58:15.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:15.725+0000 7fa65f0cf640 1 -- 172.21.15.42:0/1543223640 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa65810b310 con 0x7fa6581081c0 2024-08-30T08:58:15.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:15.726+0000 7fa655ffb640 1 -- 172.21.15.42:0/1543223640 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) ==== 74+0+24248 (secure 0 0 0) 0x7fa6440ab670 con 0x7fa6581081c0 2024-08-30T08:58:15.728 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-08-30T08:58:15.729 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":79,"fsid":"b0895c1a-66ac-11ef-bcd4-c7b262605968","created":"2024-08-30T08:49:44.858265+0000","modified":"2024-08-30T08:58:12.223108+0000","last_up_change":"2024-08-30T08:57:12.367791+0000","last_in_change":"2024-08-30T08:56:50.274678+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-30T08:53:27.297022+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-30T08:57:58.489964+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-30T08:58:01.064783+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-08-30T08:58:07.416534+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"029d62d7-a6c6-4905-8c76-6dffa9d57b92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":2617890537},{"type":"v1","addr":"172.21.15.42:6803","nonce":2617890537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":2617890537},{"type":"v1","addr":"172.21.15.42:6805","nonce":2617890537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2617890537},{"type":"v1","addr":"172.21.15.42:6809","nonce":2617890537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":2617890537},{"type":"v1","addr":"172.21.15.42:6807","nonce":2617890537}]},"public_addr":"172.21.15.42:6803/2617890537","cluster_addr":"172.21.15.42:6805/2617890537","heartbeat_back_addr":"172.21.15.42:6809/2617890537","heartbeat_front_addr":"172.21.15.42:6807/2617890537","state":["exists","up"]},{"osd":1,"uuid":"3a78384b-0207-41df-9384-22665007d5b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":2791960082},{"type":"v1","addr":"172.21.15.167:6801","nonce":2791960082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":2791960082},{"type":"v1","addr":"172.21.15.167:6803","nonce":2791960082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":2791960082},{"type":"v1","addr":"172.21.15.167:6807","nonce":2791960082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":2791960082},{"type":"v1","addr":"172.21.15.167:6805","nonce":2791960082}]},"public_addr":"172.21.15.167:6801/2791960082","cluster_addr":"172.21.15.167:6803/2791960082","heartbeat_back_addr":"172.21.15.167:6807/2791960082","heartbeat_front_addr":"172.21.15.167:6805/2791960082","state":["exists","up"]},{"osd":2,"uuid":"16001809-c27c-4f52-948a-9998261acb4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":1101326404},{"type":"v1","addr":"172.21.15.190:6801","nonce":1101326404}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":1101326404},{"type":"v1","addr":"172.21.15.190:6803","nonce":1101326404}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":1101326404},{"type":"v1","addr":"172.21.15.190:6807","nonce":1101326404}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":1101326404},{"type":"v1","addr":"172.21.15.190:6805","nonce":1101326404}]},"public_addr":"172.21.15.190:6801/1101326404","cluster_addr":"172.21.15.190:6803/1101326404","heartbeat_back_addr":"172.21.15.190:6807/1101326404","heartbeat_front_addr":"172.21.15.190:6805/1101326404","state":["exists","up"]},{"osd":3,"uuid":"87bb3e36-9e9e-4b19-9d8c-0cdf6e089d3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":1135599916},{"type":"v1","addr":"172.21.15.42:6811","nonce":1135599916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":1135599916},{"type":"v1","addr":"172.21.15.42:6813","nonce":1135599916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":1135599916},{"type":"v1","addr":"172.21.15.42:6817","nonce":1135599916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":1135599916},{"type":"v1","addr":"172.21.15.42:6815","nonce":1135599916}]},"public_addr":"172.21.15.42:6811/1135599916","cluster_addr":"172.21.15.42:6813/1135599916","heartbeat_back_addr":"172.21.15.42:6817/1135599916","heartbeat_front_addr":"172.21.15.42:6815/1135599916","state":["exists","up"]},{"osd":4,"uuid":"3ca1aae3-c972-4fb9-ac86-236279c30d64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":2927511419},{"type":"v1","addr":"172.21.15.167:6809","nonce":2927511419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":2927511419},{"type":"v1","addr":"172.21.15.167:6811","nonce":2927511419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":2927511419},{"type":"v1","addr":"172.21.15.167:6815","nonce":2927511419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":2927511419},{"type":"v1","addr":"172.21.15.167:6813","nonce":2927511419}]},"public_addr":"172.21.15.167:6809/2927511419","cluster_addr":"172.21.15.167:6811/2927511419","heartbeat_back_addr":"172.21.15.167:6815/2927511419","heartbeat_front_addr":"172.21.15.167:6813/2927511419","state":["exists","up"]},{"osd":5,"uuid":"5d07f54b-0d07-4efe-9bcf-390755ce0258","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":2557145333},{"type":"v1","addr":"172.21.15.190:6809","nonce":2557145333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":2557145333},{"type":"v1","addr":"172.21.15.190:6811","nonce":2557145333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":2557145333},{"type":"v1","addr":"172.21.15.190:6815","nonce":2557145333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":2557145333},{"type":"v1","addr":"172.21.15.190:6813","nonce":2557145333}]},"public_addr":"172.21.15.190:6809/2557145333","cluster_addr":"172.21.15.190:6811/2557145333","heartbeat_back_addr":"172.21.15.190:6815/2557145333","heartbeat_front_addr":"172.21.15.190:6813/2557145333","state":["exists","up"]},{"osd":6,"uuid":"96ad64af-44ff-4197-97d4-156f0cb88e81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":3213526100},{"type":"v1","addr":"172.21.15.42:6819","nonce":3213526100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":3213526100},{"type":"v1","addr":"172.21.15.42:6821","nonce":3213526100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":3213526100},{"type":"v1","addr":"172.21.15.42:6825","nonce":3213526100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":3213526100},{"type":"v1","addr":"172.21.15.42:6823","nonce":3213526100}]},"public_addr":"172.21.15.42:6819/3213526100","cluster_addr":"172.21.15.42:6821/3213526100","heartbeat_back_addr":"172.21.15.42:6825/3213526100","heartbeat_front_addr":"172.21.15.42:6823/3213526100","state":["exists","up"]},{"osd":7,"uuid":"337a2112-eaf7-4e6b-86be-4e40d6ddf835","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":2962799814},{"type":"v1","addr":"172.21.15.167:6817","nonce":2962799814}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":2962799814},{"type":"v1","addr":"172.21.15.167:6819","nonce":2962799814}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":2962799814},{"type":"v1","addr":"172.21.15.167:6823","nonce":2962799814}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":2962799814},{"type":"v1","addr":"172.21.15.167:6821","nonce":2962799814}]},"public_addr":"172.21.15.167:6817/2962799814","cluster_addr":"172.21.15.167:6819/2962799814","heartbeat_back_addr":"172.21.15.167:6823/2962799814","heartbeat_front_addr":"172.21.15.167:6821/2962799814","state":["exists","up"]},{"osd":8,"uuid":"219a7224-4331-464b-ac86-c51054d51d91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":3242198843},{"type":"v1","addr":"172.21.15.190:6817","nonce":3242198843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":3242198843},{"type":"v1","addr":"172.21.15.190:6819","nonce":3242198843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":3242198843},{"type":"v1","addr":"172.21.15.190:6823","nonce":3242198843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":3242198843},{"type":"v1","addr":"172.21.15.190:6821","nonce":3242198843}]},"public_addr":"172.21.15.190:6817/3242198843","cluster_addr":"172.21.15.190:6819/3242198843","heartbeat_back_addr":"172.21.15.190:6823/3242198843","heartbeat_front_addr":"172.21.15.190:6821/3242198843","state":["exists","up"]},{"osd":9,"uuid":"2f2f5177-a009-40d9-991c-76aec8a5cba6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":1123536325},{"type":"v1","addr":"172.21.15.42:6827","nonce":1123536325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":1123536325},{"type":"v1","addr":"172.21.15.42:6829","nonce":1123536325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":1123536325},{"type":"v1","addr":"172.21.15.42:6833","nonce":1123536325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":1123536325},{"type":"v1","addr":"172.21.15.42:6831","nonce":1123536325}]},"public_addr":"172.21.15.42:6827/1123536325","cluster_addr":"172.21.15.42:6829/1123536325","heartbeat_back_addr":"172.21.15.42:6833/1123536325","heartbeat_front_addr":"172.21.15.42:6831/1123536325","state":["exists","up"]},{"osd":10,"uuid":"96ef5058-2518-4862-a165-17666b33a3e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":3451198319},{"type":"v1","addr":"172.21.15.167:6825","nonce":3451198319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":3451198319},{"type":"v1","addr":"172.21.15.167:6827","nonce":3451198319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":3451198319},{"type":"v1","addr":"172.21.15.167:6831","nonce":3451198319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":3451198319},{"type":"v1","addr":"172.21.15.167:6829","nonce":3451198319}]},"public_addr":"172.21.15.167:6825/3451198319","cluster_addr":"172.21.15.167:6827/3451198319","heartbeat_back_addr":"172.21.15.167:6831/3451198319","heartbeat_front_addr":"172.21.15.167:6829/3451198319","state":["exists","up"]},{"osd":11,"uuid":"a4eb9bd3-3faa-4c81-ba6d-c2084b72c9ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":380064457},{"type":"v1","addr":"172.21.15.190:6825","nonce":380064457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":380064457},{"type":"v1","addr":"172.21.15.190:6827","nonce":380064457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":380064457},{"type":"v1","addr":"172.21.15.190:6831","nonce":380064457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":380064457},{"type":"v1","addr":"172.21.15.190:6829","nonce":380064457}]},"public_addr":"172.21.15.190:6825/380064457","cluster_addr":"172.21.15.190:6827/380064457","heartbeat_back_addr":"172.21.15.190:6831/380064457","heartbeat_front_addr":"172.21.15.190:6829/380064457","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-30T08:52:36.511448+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-30T08:53:00.027277+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-30T08:53:23.770245+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-30T08:53:49.322226+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-30T08:54:14.350234+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-30T08:54:39.491793+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-30T08:55:04.870596+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-30T08:55:30.314150+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-30T08:55:54.483843+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-30T08:56:20.586115+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-30T08:56:45.666783+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-30T08:57:10.718387+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.42:0/2691233260":"2024-08-31T08:50:44.821070+0000","172.21.15.42:0/1518053351":"2024-08-31T08:50:44.821070+0000","172.21.15.42:0/352001020":"2024-08-31T08:50:44.821070+0000","172.21.15.42:6801/1733768634":"2024-08-31T08:50:44.821070+0000","172.21.15.42:0/1000597888":"2024-08-31T08:50:18.561851+0000","172.21.15.42:6800/1733768634":"2024-08-31T08:50:44.821070+0000","172.21.15.42:6801/3538882081":"2024-08-31T08:50:18.561851+0000","172.21.15.42:6800/3538882081":"2024-08-31T08:50:18.561851+0000","172.21.15.42:0/145366508":"2024-08-31T08:50:18.561851+0000","172.21.15.42:0/3346355213":"2024-08-31T08:50:18.561851+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-30T08:58:15.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:15.729+0000 7fa65f0cf640 1 -- 172.21.15.42:0/1543223640 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fa6480b4b30 msgr2=0x7fa6480b7040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:58:15.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:15.729+0000 7fa65f0cf640 1 --2- 172.21.15.42:0/1543223640 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fa6480b4b30 0x7fa6480b7040 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fa64c000c00 tx=0x7fa64c031400 comp rx=0 tx=0).stop 2024-08-30T08:58:15.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:15.729+0000 7fa65f0cf640 1 -- 172.21.15.42:0/1543223640 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa6581081c0 msgr2=0x7fa6580ff950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:58:15.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:15.729+0000 7fa65f0cf640 1 --2- 172.21.15.42:0/1543223640 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa6581081c0 0x7fa6580ff950 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fa64400d750 tx=0x7fa64400dc10 comp rx=0 tx=0).stop 2024-08-30T08:58:15.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:15.729+0000 7fa65f0cf640 1 -- 172.21.15.42:0/1543223640 shutdown_connections 2024-08-30T08:58:15.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:15.729+0000 7fa65f0cf640 1 --2- 172.21.15.42:0/1543223640 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fa6480b4b30 0x7fa6480b7040 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:15.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:15.729+0000 7fa65f0cf640 1 --2- 172.21.15.42:0/1543223640 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa658108b10 0x7fa6580ffe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:15.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:15.729+0000 7fa65f0cf640 1 --2- 172.21.15.42:0/1543223640 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa6581081c0 0x7fa6580ff950 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:15.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:15.729+0000 7fa65f0cf640 1 --2- 172.21.15.42:0/1543223640 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa658102880 0x7fa6580ff410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:15.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:15.729+0000 7fa65f0cf640 1 -- 172.21.15.42:0/1543223640 >> 172.21.15.42:0/1543223640 conn(0x7fa658079510 msgr2=0x7fa6581018f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:58:15.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:15.730+0000 7fa65f0cf640 1 -- 172.21.15.42:0/1543223640 shutdown_connections 2024-08-30T08:58:15.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:15.730+0000 7fa65f0cf640 1 -- 172.21.15.42:0/1543223640 wait complete. 2024-08-30T08:58:16.461 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph osd dump --format=json 2024-08-30T08:58:16.714 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T08:58:16.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:58:16 smithi167 ceph-mon[22063]: pgmap v294: 129 pgs: 129 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-30T08:58:16.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:58:16 smithi167 ceph-mon[22063]: from='client.? 172.21.15.42:0/1543223640' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-30T08:58:16.834 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:58:16 smithi042 ceph-mon[18652]: pgmap v294: 129 pgs: 129 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-30T08:58:16.834 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:58:16 smithi042 ceph-mon[18652]: from='client.? 172.21.15.42:0/1543223640' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-30T08:58:16.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:58:16 smithi190 ceph-mon[22001]: pgmap v294: 129 pgs: 129 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-30T08:58:16.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:58:16 smithi190 ceph-mon[22001]: from='client.? 172.21.15.42:0/1543223640' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-30T08:58:17.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:17.194+0000 7efd38111640 1 -- 172.21.15.42:0/2836595348 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efd30109240 msgr2=0x7efd30109650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:58:17.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:17.194+0000 7efd38111640 1 --2- 172.21.15.42:0/2836595348 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efd30109240 0x7efd30109650 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7efd20009900 tx=0x7efd2002a410 comp rx=0 tx=0).stop 2024-08-30T08:58:17.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:17.194+0000 7efd38111640 1 -- 172.21.15.42:0/2836595348 shutdown_connections 2024-08-30T08:58:17.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:17.195+0000 7efd38111640 1 --2- 172.21.15.42:0/2836595348 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efd30109b90 0x7efd301103c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:17.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:17.195+0000 7efd38111640 1 --2- 172.21.15.42:0/2836595348 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efd30109240 0x7efd30109650 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:17.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:17.195+0000 7efd38111640 1 --2- 172.21.15.42:0/2836595348 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efd30103900 0x7efd30103d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:17.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:17.195+0000 7efd38111640 1 -- 172.21.15.42:0/2836595348 >> 172.21.15.42:0/2836595348 conn(0x7efd3007b5f0 msgr2=0x7efd3007ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:58:17.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:17.195+0000 7efd38111640 1 -- 172.21.15.42:0/2836595348 shutdown_connections 2024-08-30T08:58:17.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:17.195+0000 7efd38111640 1 -- 172.21.15.42:0/2836595348 wait complete. 2024-08-30T08:58:17.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:17.195+0000 7efd38111640 1 Processor -- start 2024-08-30T08:58:17.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:17.195+0000 7efd38111640 1 -- start start 2024-08-30T08:58:17.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:17.196+0000 7efd38111640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efd30103900 0x7efd3019a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:58:17.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:17.196+0000 7efd38111640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efd30109240 0x7efd3019a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:58:17.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:17.196+0000 7efd38111640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efd30109b90 0x7efd3019fb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:58:17.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:17.196+0000 7efd38111640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7efd30113850 con 0x7efd30109b90 2024-08-30T08:58:17.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:17.196+0000 7efd38111640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7efd301136d0 con 0x7efd30103900 2024-08-30T08:58:17.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:17.196+0000 7efd38111640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7efd301139d0 con 0x7efd30109240 2024-08-30T08:58:17.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:17.196+0000 7efd366ae640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efd30109b90 0x7efd3019fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:58:17.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:17.196+0000 7efd366ae640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efd30109b90 0x7efd3019fb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:49104/0 (socket says 172.21.15.42:49104) 2024-08-30T08:58:17.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:17.196+0000 7efd366ae640 1 -- 172.21.15.42:0/1755846689 learned_addr learned my addr 172.21.15.42:0/1755846689 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T08:58:17.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:17.196+0000 7efd356ac640 1 --2- 172.21.15.42:0/1755846689 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efd30109240 0x7efd3019a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:58:17.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:17.196+0000 7efd35ead640 1 --2- 172.21.15.42:0/1755846689 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efd30103900 0x7efd3019a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:58:17.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:17.196+0000 7efd366ae640 1 -- 172.21.15.42:0/1755846689 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efd30109240 msgr2=0x7efd3019a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:58:17.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:17.196+0000 7efd366ae640 1 --2- 172.21.15.42:0/1755846689 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efd30109240 0x7efd3019a8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:17.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:17.196+0000 7efd366ae640 1 -- 172.21.15.42:0/1755846689 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efd30103900 msgr2=0x7efd3019a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:58:17.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:17.196+0000 7efd366ae640 1 --2- 172.21.15.42:0/1755846689 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efd30103900 0x7efd3019a320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:17.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:17.196+0000 7efd366ae640 1 -- 172.21.15.42:0/1755846689 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd301a0290 con 0x7efd30109b90 2024-08-30T08:58:17.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:17.196+0000 7efd366ae640 1 --2- 172.21.15.42:0/1755846689 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efd30109b90 0x7efd3019fb60 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7efd2800cad0 tx=0x7efd2800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:58:17.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:17.196+0000 7efd26ffd640 1 -- 172.21.15.42:0/1755846689 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd2800ae20 con 0x7efd30109b90 2024-08-30T08:58:17.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:17.196+0000 7efd26ffd640 1 -- 172.21.15.42:0/1755846689 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd28005020 con 0x7efd30109b90 2024-08-30T08:58:17.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:17.197+0000 7efd38111640 1 -- 172.21.15.42:0/1755846689 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd301a0580 con 0x7efd30109b90 2024-08-30T08:58:17.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:17.197+0000 7efd26ffd640 1 -- 172.21.15.42:0/1755846689 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd2800f6b0 con 0x7efd30109b90 2024-08-30T08:58:17.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:17.197+0000 7efd38111640 1 -- 172.21.15.42:0/1755846689 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd301a6ce0 con 0x7efd30109b90 2024-08-30T08:58:17.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:17.197+0000 7efd38111640 1 -- 172.21.15.42:0/1755846689 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd30103d10 con 0x7efd30109b90 2024-08-30T08:58:17.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:17.199+0000 7efd26ffd640 1 -- 172.21.15.42:0/1755846689 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) ==== 152577+0+0 (secure 0 0 0) 0x7efd280051d0 con 0x7efd30109b90 2024-08-30T08:58:17.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:17.200+0000 7efd26ffd640 1 --2- 172.21.15.42:0/1755846689 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7efd1c0b4b30 0x7efd1c0b7040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:58:17.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:17.200+0000 7efd35ead640 1 --2- 172.21.15.42:0/1755846689 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7efd1c0b4b30 0x7efd1c0b7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:58:17.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:17.200+0000 7efd26ffd640 1 -- 172.21.15.42:0/1755846689 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8802+0+0 (secure 0 0 0) 0x7efd280e56d0 con 0x7efd30109b90 2024-08-30T08:58:17.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:17.200+0000 7efd35ead640 1 --2- 172.21.15.42:0/1755846689 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7efd1c0b4b30 0x7efd1c0b7040 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7efd2c005fd0 tx=0x7efd2c005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:58:17.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:17.203+0000 7efd26ffd640 1 -- 172.21.15.42:0/1755846689 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7efd28014030 con 0x7efd30109b90 2024-08-30T08:58:17.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:17.389+0000 7efd38111640 1 -- 172.21.15.42:0/1755846689 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7efd3019b450 con 0x7efd30109b90 2024-08-30T08:58:17.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:17.391+0000 7efd26ffd640 1 -- 172.21.15.42:0/1755846689 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) ==== 74+0+24248 (secure 0 0 0) 0x7efd3019b450 con 0x7efd30109b90 2024-08-30T08:58:17.393 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-08-30T08:58:17.393 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":79,"fsid":"b0895c1a-66ac-11ef-bcd4-c7b262605968","created":"2024-08-30T08:49:44.858265+0000","modified":"2024-08-30T08:58:12.223108+0000","last_up_change":"2024-08-30T08:57:12.367791+0000","last_in_change":"2024-08-30T08:56:50.274678+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-30T08:53:27.297022+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-30T08:57:58.489964+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-30T08:58:01.064783+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-08-30T08:58:07.416534+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"029d62d7-a6c6-4905-8c76-6dffa9d57b92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":2617890537},{"type":"v1","addr":"172.21.15.42:6803","nonce":2617890537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":2617890537},{"type":"v1","addr":"172.21.15.42:6805","nonce":2617890537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2617890537},{"type":"v1","addr":"172.21.15.42:6809","nonce":2617890537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":2617890537},{"type":"v1","addr":"172.21.15.42:6807","nonce":2617890537}]},"public_addr":"172.21.15.42:6803/2617890537","cluster_addr":"172.21.15.42:6805/2617890537","heartbeat_back_addr":"172.21.15.42:6809/2617890537","heartbeat_front_addr":"172.21.15.42:6807/2617890537","state":["exists","up"]},{"osd":1,"uuid":"3a78384b-0207-41df-9384-22665007d5b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":2791960082},{"type":"v1","addr":"172.21.15.167:6801","nonce":2791960082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":2791960082},{"type":"v1","addr":"172.21.15.167:6803","nonce":2791960082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":2791960082},{"type":"v1","addr":"172.21.15.167:6807","nonce":2791960082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":2791960082},{"type":"v1","addr":"172.21.15.167:6805","nonce":2791960082}]},"public_addr":"172.21.15.167:6801/2791960082","cluster_addr":"172.21.15.167:6803/2791960082","heartbeat_back_addr":"172.21.15.167:6807/2791960082","heartbeat_front_addr":"172.21.15.167:6805/2791960082","state":["exists","up"]},{"osd":2,"uuid":"16001809-c27c-4f52-948a-9998261acb4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":1101326404},{"type":"v1","addr":"172.21.15.190:6801","nonce":1101326404}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":1101326404},{"type":"v1","addr":"172.21.15.190:6803","nonce":1101326404}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":1101326404},{"type":"v1","addr":"172.21.15.190:6807","nonce":1101326404}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":1101326404},{"type":"v1","addr":"172.21.15.190:6805","nonce":1101326404}]},"public_addr":"172.21.15.190:6801/1101326404","cluster_addr":"172.21.15.190:6803/1101326404","heartbeat_back_addr":"172.21.15.190:6807/1101326404","heartbeat_front_addr":"172.21.15.190:6805/1101326404","state":["exists","up"]},{"osd":3,"uuid":"87bb3e36-9e9e-4b19-9d8c-0cdf6e089d3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":1135599916},{"type":"v1","addr":"172.21.15.42:6811","nonce":1135599916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":1135599916},{"type":"v1","addr":"172.21.15.42:6813","nonce":1135599916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":1135599916},{"type":"v1","addr":"172.21.15.42:6817","nonce":1135599916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":1135599916},{"type":"v1","addr":"172.21.15.42:6815","nonce":1135599916}]},"public_addr":"172.21.15.42:6811/1135599916","cluster_addr":"172.21.15.42:6813/1135599916","heartbeat_back_addr":"172.21.15.42:6817/1135599916","heartbeat_front_addr":"172.21.15.42:6815/1135599916","state":["exists","up"]},{"osd":4,"uuid":"3ca1aae3-c972-4fb9-ac86-236279c30d64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":2927511419},{"type":"v1","addr":"172.21.15.167:6809","nonce":2927511419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":2927511419},{"type":"v1","addr":"172.21.15.167:6811","nonce":2927511419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":2927511419},{"type":"v1","addr":"172.21.15.167:6815","nonce":2927511419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":2927511419},{"type":"v1","addr":"172.21.15.167:6813","nonce":2927511419}]},"public_addr":"172.21.15.167:6809/2927511419","cluster_addr":"172.21.15.167:6811/2927511419","heartbeat_back_addr":"172.21.15.167:6815/2927511419","heartbeat_front_addr":"172.21.15.167:6813/2927511419","state":["exists","up"]},{"osd":5,"uuid":"5d07f54b-0d07-4efe-9bcf-390755ce0258","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":2557145333},{"type":"v1","addr":"172.21.15.190:6809","nonce":2557145333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":2557145333},{"type":"v1","addr":"172.21.15.190:6811","nonce":2557145333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":2557145333},{"type":"v1","addr":"172.21.15.190:6815","nonce":2557145333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":2557145333},{"type":"v1","addr":"172.21.15.190:6813","nonce":2557145333}]},"public_addr":"172.21.15.190:6809/2557145333","cluster_addr":"172.21.15.190:6811/2557145333","heartbeat_back_addr":"172.21.15.190:6815/2557145333","heartbeat_front_addr":"172.21.15.190:6813/2557145333","state":["exists","up"]},{"osd":6,"uuid":"96ad64af-44ff-4197-97d4-156f0cb88e81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":3213526100},{"type":"v1","addr":"172.21.15.42:6819","nonce":3213526100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":3213526100},{"type":"v1","addr":"172.21.15.42:6821","nonce":3213526100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":3213526100},{"type":"v1","addr":"172.21.15.42:6825","nonce":3213526100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":3213526100},{"type":"v1","addr":"172.21.15.42:6823","nonce":3213526100}]},"public_addr":"172.21.15.42:6819/3213526100","cluster_addr":"172.21.15.42:6821/3213526100","heartbeat_back_addr":"172.21.15.42:6825/3213526100","heartbeat_front_addr":"172.21.15.42:6823/3213526100","state":["exists","up"]},{"osd":7,"uuid":"337a2112-eaf7-4e6b-86be-4e40d6ddf835","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":2962799814},{"type":"v1","addr":"172.21.15.167:6817","nonce":2962799814}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":2962799814},{"type":"v1","addr":"172.21.15.167:6819","nonce":2962799814}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":2962799814},{"type":"v1","addr":"172.21.15.167:6823","nonce":2962799814}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":2962799814},{"type":"v1","addr":"172.21.15.167:6821","nonce":2962799814}]},"public_addr":"172.21.15.167:6817/2962799814","cluster_addr":"172.21.15.167:6819/2962799814","heartbeat_back_addr":"172.21.15.167:6823/2962799814","heartbeat_front_addr":"172.21.15.167:6821/2962799814","state":["exists","up"]},{"osd":8,"uuid":"219a7224-4331-464b-ac86-c51054d51d91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":3242198843},{"type":"v1","addr":"172.21.15.190:6817","nonce":3242198843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":3242198843},{"type":"v1","addr":"172.21.15.190:6819","nonce":3242198843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":3242198843},{"type":"v1","addr":"172.21.15.190:6823","nonce":3242198843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":3242198843},{"type":"v1","addr":"172.21.15.190:6821","nonce":3242198843}]},"public_addr":"172.21.15.190:6817/3242198843","cluster_addr":"172.21.15.190:6819/3242198843","heartbeat_back_addr":"172.21.15.190:6823/3242198843","heartbeat_front_addr":"172.21.15.190:6821/3242198843","state":["exists","up"]},{"osd":9,"uuid":"2f2f5177-a009-40d9-991c-76aec8a5cba6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":1123536325},{"type":"v1","addr":"172.21.15.42:6827","nonce":1123536325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":1123536325},{"type":"v1","addr":"172.21.15.42:6829","nonce":1123536325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":1123536325},{"type":"v1","addr":"172.21.15.42:6833","nonce":1123536325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":1123536325},{"type":"v1","addr":"172.21.15.42:6831","nonce":1123536325}]},"public_addr":"172.21.15.42:6827/1123536325","cluster_addr":"172.21.15.42:6829/1123536325","heartbeat_back_addr":"172.21.15.42:6833/1123536325","heartbeat_front_addr":"172.21.15.42:6831/1123536325","state":["exists","up"]},{"osd":10,"uuid":"96ef5058-2518-4862-a165-17666b33a3e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":3451198319},{"type":"v1","addr":"172.21.15.167:6825","nonce":3451198319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":3451198319},{"type":"v1","addr":"172.21.15.167:6827","nonce":3451198319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":3451198319},{"type":"v1","addr":"172.21.15.167:6831","nonce":3451198319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":3451198319},{"type":"v1","addr":"172.21.15.167:6829","nonce":3451198319}]},"public_addr":"172.21.15.167:6825/3451198319","cluster_addr":"172.21.15.167:6827/3451198319","heartbeat_back_addr":"172.21.15.167:6831/3451198319","heartbeat_front_addr":"172.21.15.167:6829/3451198319","state":["exists","up"]},{"osd":11,"uuid":"a4eb9bd3-3faa-4c81-ba6d-c2084b72c9ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":380064457},{"type":"v1","addr":"172.21.15.190:6825","nonce":380064457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":380064457},{"type":"v1","addr":"172.21.15.190:6827","nonce":380064457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":380064457},{"type":"v1","addr":"172.21.15.190:6831","nonce":380064457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":380064457},{"type":"v1","addr":"172.21.15.190:6829","nonce":380064457}]},"public_addr":"172.21.15.190:6825/380064457","cluster_addr":"172.21.15.190:6827/380064457","heartbeat_back_addr":"172.21.15.190:6831/380064457","heartbeat_front_addr":"172.21.15.190:6829/380064457","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-30T08:52:36.511448+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-30T08:53:00.027277+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-30T08:53:23.770245+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-30T08:53:49.322226+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-30T08:54:14.350234+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-30T08:54:39.491793+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-30T08:55:04.870596+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-30T08:55:30.314150+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-30T08:55:54.483843+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-30T08:56:20.586115+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-30T08:56:45.666783+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-30T08:57:10.718387+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.42:0/2691233260":"2024-08-31T08:50:44.821070+0000","172.21.15.42:0/1518053351":"2024-08-31T08:50:44.821070+0000","172.21.15.42:0/352001020":"2024-08-31T08:50:44.821070+0000","172.21.15.42:6801/1733768634":"2024-08-31T08:50:44.821070+0000","172.21.15.42:0/1000597888":"2024-08-31T08:50:18.561851+0000","172.21.15.42:6800/1733768634":"2024-08-31T08:50:44.821070+0000","172.21.15.42:6801/3538882081":"2024-08-31T08:50:18.561851+0000","172.21.15.42:6800/3538882081":"2024-08-31T08:50:18.561851+0000","172.21.15.42:0/145366508":"2024-08-31T08:50:18.561851+0000","172.21.15.42:0/3346355213":"2024-08-31T08:50:18.561851+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-30T08:58:17.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:17.393+0000 7efd38111640 1 -- 172.21.15.42:0/1755846689 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7efd1c0b4b30 msgr2=0x7efd1c0b7040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:58:17.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:17.393+0000 7efd38111640 1 --2- 172.21.15.42:0/1755846689 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7efd1c0b4b30 0x7efd1c0b7040 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7efd2c005fd0 tx=0x7efd2c005850 comp rx=0 tx=0).stop 2024-08-30T08:58:17.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:17.393+0000 7efd38111640 1 -- 172.21.15.42:0/1755846689 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efd30109b90 msgr2=0x7efd3019fb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:58:17.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:17.393+0000 7efd38111640 1 --2- 172.21.15.42:0/1755846689 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efd30109b90 0x7efd3019fb60 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7efd2800cad0 tx=0x7efd2800cf90 comp rx=0 tx=0).stop 2024-08-30T08:58:17.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:17.393+0000 7efd38111640 1 -- 172.21.15.42:0/1755846689 shutdown_connections 2024-08-30T08:58:17.396 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:17.393+0000 7efd38111640 1 --2- 172.21.15.42:0/1755846689 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7efd1c0b4b30 0x7efd1c0b7040 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:17.396 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:17.393+0000 7efd38111640 1 --2- 172.21.15.42:0/1755846689 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efd30109240 0x7efd3019a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:17.396 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:17.393+0000 7efd38111640 1 --2- 172.21.15.42:0/1755846689 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efd30109b90 0x7efd3019fb60 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:17.396 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:17.393+0000 7efd38111640 1 --2- 172.21.15.42:0/1755846689 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efd30103900 0x7efd3019a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:17.396 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:17.393+0000 7efd38111640 1 -- 172.21.15.42:0/1755846689 >> 172.21.15.42:0/1755846689 conn(0x7efd3007b5f0 msgr2=0x7efd30105370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:58:17.396 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:17.393+0000 7efd38111640 1 -- 172.21.15.42:0/1755846689 shutdown_connections 2024-08-30T08:58:17.396 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:17.393+0000 7efd38111640 1 -- 172.21.15.42:0/1755846689 wait complete. 2024-08-30T08:58:17.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:58:17 smithi167 ceph-mon[22063]: from='client.? 172.21.15.42:0/1755846689' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-30T08:58:17.873 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:58:17 smithi042 ceph-mon[18652]: from='client.? 172.21.15.42:0/1755846689' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-30T08:58:17.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:58:17 smithi190 ceph-mon[22001]: from='client.? 172.21.15.42:0/1755846689' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-30T08:58:18.110 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph fs dump --format=json 2024-08-30T08:58:18.364 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T08:58:18.759 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:58:18 smithi042 ceph-mon[18652]: pgmap v295: 129 pgs: 129 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-30T08:58:18.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:58:18 smithi167 ceph-mon[22063]: pgmap v295: 129 pgs: 129 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-30T08:58:18.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:18.848+0000 7fcfffe79640 1 -- 172.21.15.42:0/3037918654 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcff8102880 msgr2=0x7fcff8102c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:58:18.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:18.848+0000 7fcfffe79640 1 --2- 172.21.15.42:0/3037918654 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcff8102880 0x7fcff8102c90 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fcff4009900 tx=0x7fcff402a410 comp rx=0 tx=0).stop 2024-08-30T08:58:18.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:18.848+0000 7fcfffe79640 1 -- 172.21.15.42:0/3037918654 shutdown_connections 2024-08-30T08:58:18.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:18.848+0000 7fcfffe79640 1 --2- 172.21.15.42:0/3037918654 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcff81081c0 0x7fcff81085d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:18.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:18.848+0000 7fcfffe79640 1 --2- 172.21.15.42:0/3037918654 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcff8108b10 0x7fcff810f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:18.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:18.848+0000 7fcfffe79640 1 --2- 172.21.15.42:0/3037918654 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcff8102880 0x7fcff8102c90 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:18.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:18.848+0000 7fcfffe79640 1 -- 172.21.15.42:0/3037918654 >> 172.21.15.42:0/3037918654 conn(0x7fcff8079510 msgr2=0x7fcff8079920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:58:18.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:18.848+0000 7fcfffe79640 1 -- 172.21.15.42:0/3037918654 shutdown_connections 2024-08-30T08:58:18.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:18.848+0000 7fcfffe79640 1 -- 172.21.15.42:0/3037918654 wait complete. 2024-08-30T08:58:18.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:18.848+0000 7fcfffe79640 1 Processor -- start 2024-08-30T08:58:18.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:18.848+0000 7fcfffe79640 1 -- start start 2024-08-30T08:58:18.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:18.849+0000 7fcfffe79640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcff8102880 0x7fcff819b110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:58:18.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:18.849+0000 7fcfffe79640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcff81081c0 0x7fcff819b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:58:18.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:18.849+0000 7fcfffe79640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcff8108b10 0x7fcff819bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:58:18.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:18.849+0000 7fcfffe79640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fcff81127e0 con 0x7fcff8108b10 2024-08-30T08:58:18.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:18.849+0000 7fcfffe79640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fcff8112660 con 0x7fcff81081c0 2024-08-30T08:58:18.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:18.849+0000 7fcfffe79640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fcff8112960 con 0x7fcff8102880 2024-08-30T08:58:18.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:18.849+0000 7fcffe416640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcff8108b10 0x7fcff819bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:58:18.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:18.849+0000 7fcffe416640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcff8108b10 0x7fcff819bb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36442/0 (socket says 172.21.15.42:36442) 2024-08-30T08:58:18.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:18.849+0000 7fcffe416640 1 -- 172.21.15.42:0/3828892481 learned_addr learned my addr 172.21.15.42:0/3828892481 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T08:58:18.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:18.849+0000 7fcffdc15640 1 --2- 172.21.15.42:0/3828892481 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcff8102880 0x7fcff819b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:58:18.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:18.849+0000 7fcffd414640 1 --2- 172.21.15.42:0/3828892481 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcff81081c0 0x7fcff819b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:58:18.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:18.849+0000 7fcffe416640 1 -- 172.21.15.42:0/3828892481 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcff8102880 msgr2=0x7fcff819b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:58:18.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:18.849+0000 7fcffe416640 1 --2- 172.21.15.42:0/3828892481 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcff8102880 0x7fcff819b110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:18.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:18.849+0000 7fcffe416640 1 -- 172.21.15.42:0/3828892481 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcff81081c0 msgr2=0x7fcff819b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:58:18.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:18.849+0000 7fcffe416640 1 --2- 172.21.15.42:0/3828892481 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcff81081c0 0x7fcff819b650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:18.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:18.849+0000 7fcffe416640 1 -- 172.21.15.42:0/3828892481 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcff81a63a0 con 0x7fcff8108b10 2024-08-30T08:58:18.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:18.849+0000 7fcffe416640 1 --2- 172.21.15.42:0/3828892481 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcff8108b10 0x7fcff819bb90 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fcff0009fd0 tx=0x7fcff000b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:58:18.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:18.850+0000 7fcfeeffd640 1 -- 172.21.15.42:0/3828892481 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcff0017070 con 0x7fcff8108b10 2024-08-30T08:58:18.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:18.850+0000 7fcfeeffd640 1 -- 172.21.15.42:0/3828892481 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcff0003e20 con 0x7fcff8108b10 2024-08-30T08:58:18.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:18.850+0000 7fcfeeffd640 1 -- 172.21.15.42:0/3828892481 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcff0010460 con 0x7fcff8108b10 2024-08-30T08:58:18.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:18.850+0000 7fcfffe79640 1 -- 172.21.15.42:0/3828892481 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcff81a6690 con 0x7fcff8108b10 2024-08-30T08:58:18.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:18.850+0000 7fcfffe79640 1 -- 172.21.15.42:0/3828892481 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcff81a6b80 con 0x7fcff8108b10 2024-08-30T08:58:18.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:18.851+0000 7fcfffe79640 1 -- 172.21.15.42:0/3828892481 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcff8103b00 con 0x7fcff8108b10 2024-08-30T08:58:18.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:18.852+0000 7fcfeeffd640 1 -- 172.21.15.42:0/3828892481 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) ==== 152577+0+0 (secure 0 0 0) 0x7fcff0003970 con 0x7fcff8108b10 2024-08-30T08:58:18.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:18.853+0000 7fcfeeffd640 1 --2- 172.21.15.42:0/3828892481 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fcfe40b4be0 0x7fcfe40b70f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:58:18.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:18.853+0000 7fcffdc15640 1 --2- 172.21.15.42:0/3828892481 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fcfe40b4be0 0x7fcfe40b70f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:58:18.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:18.853+0000 7fcfeeffd640 1 -- 172.21.15.42:0/3828892481 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8802+0+0 (secure 0 0 0) 0x7fcff00e4f30 con 0x7fcff8108b10 2024-08-30T08:58:18.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:18.853+0000 7fcffdc15640 1 --2- 172.21.15.42:0/3828892481 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fcfe40b4be0 0x7fcfe40b70f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fcff4000c00 tx=0x7fcff4031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:58:18.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:18.857+0000 7fcfeeffd640 1 -- 172.21.15.42:0/3828892481 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7fcff00ab7d0 con 0x7fcff8108b10 2024-08-30T08:58:18.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:58:18 smithi190 ceph-mon[22001]: pgmap v295: 129 pgs: 129 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-30T08:58:19.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:19.078+0000 7fcfffe79640 1 -- 172.21.15.42:0/3828892481 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcff8102c90 con 0x7fcff8108b10 2024-08-30T08:58:19.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:19.079+0000 7fcfeeffd640 1 -- 172.21.15.42:0/3828892481 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+11842 (secure 0 0 0) 0x7fcff00b4360 con 0x7fcff8108b10 2024-08-30T08:58:19.081 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-08-30T08:58:19.081 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":17,"btime":"2024-08-30T08:58:12:441452+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14703,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/3188400000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":3188400000},{"type":"v1","addr":"172.21.15.42:6835","nonce":3188400000}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14715,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/3356537674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":3356537674},{"type":"v1","addr":"172.21.15.42:6837","nonce":3356537674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14766,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6839/2107785513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6838","nonce":2107785513},{"type":"v1","addr":"172.21.15.42:6839","nonce":2107785513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14778,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6841/2298438674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6840","nonce":2298438674},{"type":"v1","addr":"172.21.15.42:6841","nonce":2298438674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24553,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/543555570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":543555570},{"type":"v1","addr":"172.21.15.190:6833","nonce":543555570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24554,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/3907818941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":3907818941},{"type":"v1","addr":"172.21.15.167:6833","nonce":3907818941}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24572,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/234596845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":234596845},{"type":"v1","addr":"172.21.15.167:6835","nonce":234596845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24592,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/247905709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":247905709},{"type":"v1","addr":"172.21.15.190:6835","nonce":247905709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24598,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6837/863256349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6836","nonce":863256349},{"type":"v1","addr":"172.21.15.190:6837","nonce":863256349}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24599,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/3011837856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":3011837856},{"type":"v1","addr":"172.21.15.167:6837","nonce":3011837856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24610,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6839/719981110","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6838","nonce":719981110},{"type":"v1","addr":"172.21.15.190:6839","nonce":719981110}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-30T08:58:03.055084+0000","modified":"2024-08-30T08:58:12.223290+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24629},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24629":{"gid":24629,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.167:6839/3604645207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":3604645207},{"type":"v1","addr":"172.21.15.167:6839","nonce":3604645207}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24629,"qdb_cluster":[24629]},"id":1}]} 2024-08-30T08:58:19.081 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 17 2024-08-30T08:58:19.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:19.081+0000 7fcfffe79640 1 -- 172.21.15.42:0/3828892481 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fcfe40b4be0 msgr2=0x7fcfe40b70f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:58:19.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:19.081+0000 7fcfffe79640 1 --2- 172.21.15.42:0/3828892481 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fcfe40b4be0 0x7fcfe40b70f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fcff4000c00 tx=0x7fcff4031400 comp rx=0 tx=0).stop 2024-08-30T08:58:19.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:19.081+0000 7fcfffe79640 1 -- 172.21.15.42:0/3828892481 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcff8108b10 msgr2=0x7fcff819bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:58:19.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:19.081+0000 7fcfffe79640 1 --2- 172.21.15.42:0/3828892481 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcff8108b10 0x7fcff819bb90 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fcff0009fd0 tx=0x7fcff000b260 comp rx=0 tx=0).stop 2024-08-30T08:58:19.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:19.081+0000 7fcfffe79640 1 -- 172.21.15.42:0/3828892481 shutdown_connections 2024-08-30T08:58:19.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:19.081+0000 7fcfffe79640 1 --2- 172.21.15.42:0/3828892481 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fcfe40b4be0 0x7fcfe40b70f0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:19.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:19.081+0000 7fcfffe79640 1 --2- 172.21.15.42:0/3828892481 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcff8108b10 0x7fcff819bb90 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:19.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:19.081+0000 7fcfffe79640 1 --2- 172.21.15.42:0/3828892481 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcff81081c0 0x7fcff819b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:19.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:19.081+0000 7fcfffe79640 1 --2- 172.21.15.42:0/3828892481 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcff8102880 0x7fcff819b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:19.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:19.081+0000 7fcfffe79640 1 -- 172.21.15.42:0/3828892481 >> 172.21.15.42:0/3828892481 conn(0x7fcff8079510 msgr2=0x7fcff81062c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:58:19.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:19.082+0000 7fcfffe79640 1 -- 172.21.15.42:0/3828892481 shutdown_connections 2024-08-30T08:58:19.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:19.082+0000 7fcfffe79640 1 -- 172.21.15.42:0/3828892481 wait complete. 2024-08-30T08:58:19.714 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:58:19 smithi042 ceph-mon[18652]: from='client.? 172.21.15.42:0/3828892481' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-30T08:58:19.744 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph osd dump --format=json 2024-08-30T08:58:19.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:58:19 smithi167 ceph-mon[22063]: from='client.? 172.21.15.42:0/3828892481' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-30T08:58:19.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:58:19 smithi190 ceph-mon[22001]: from='client.? 172.21.15.42:0/3828892481' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-30T08:58:19.998 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T08:58:20.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:20.510+0000 7f8d33fc7640 1 -- 172.21.15.42:0/2605004109 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8d2c109310 msgr2=0x7f8d2c109720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:58:20.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:20.510+0000 7f8d33fc7640 1 --2- 172.21.15.42:0/2605004109 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8d2c109310 0x7f8d2c109720 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c009900 tx=0x7f8d1c02a410 comp rx=0 tx=0).stop 2024-08-30T08:58:20.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:20.510+0000 7f8d33fc7640 1 -- 172.21.15.42:0/2605004109 shutdown_connections 2024-08-30T08:58:20.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:20.510+0000 7f8d33fc7640 1 --2- 172.21.15.42:0/2605004109 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8d2c109c60 0x7f8d2c110490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:20.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:20.510+0000 7f8d33fc7640 1 --2- 172.21.15.42:0/2605004109 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8d2c109310 0x7f8d2c109720 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:20.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:20.510+0000 7f8d33fc7640 1 --2- 172.21.15.42:0/2605004109 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8d2c1039d0 0x7f8d2c103de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:20.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:20.510+0000 7f8d33fc7640 1 -- 172.21.15.42:0/2605004109 >> 172.21.15.42:0/2605004109 conn(0x7f8d2c07b670 msgr2=0x7f8d2c07ba80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:58:20.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:20.511+0000 7f8d33fc7640 1 -- 172.21.15.42:0/2605004109 shutdown_connections 2024-08-30T08:58:20.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:20.511+0000 7f8d33fc7640 1 -- 172.21.15.42:0/2605004109 wait complete. 2024-08-30T08:58:20.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:20.511+0000 7f8d33fc7640 1 Processor -- start 2024-08-30T08:58:20.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:20.511+0000 7f8d33fc7640 1 -- start start 2024-08-30T08:58:20.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:20.512+0000 7f8d33fc7640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8d2c1039d0 0x7f8d2c0754b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:58:20.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:20.512+0000 7f8d33fc7640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8d2c109310 0x7f8d2c0759f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:58:20.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:20.512+0000 7f8d33fc7640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8d2c109c60 0x7f8d2c075f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:58:20.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:20.512+0000 7f8d33fc7640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f8d2c113880 con 0x7f8d2c109310 2024-08-30T08:58:20.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:20.512+0000 7f8d33fc7640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f8d2c113700 con 0x7f8d2c109c60 2024-08-30T08:58:20.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:20.512+0000 7f8d33fc7640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f8d2c113a00 con 0x7f8d2c1039d0 2024-08-30T08:58:20.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:20.512+0000 7f8d31562640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8d2c109310 0x7f8d2c0759f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:58:20.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:20.512+0000 7f8d31562640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8d2c109310 0x7f8d2c0759f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36472/0 (socket says 172.21.15.42:36472) 2024-08-30T08:58:20.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:20.512+0000 7f8d31562640 1 -- 172.21.15.42:0/1120677115 learned_addr learned my addr 172.21.15.42:0/1120677115 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T08:58:20.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:20.512+0000 7f8d32564640 1 --2- 172.21.15.42:0/1120677115 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8d2c109c60 0x7f8d2c075f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:58:20.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:20.512+0000 7f8d31d63640 1 --2- 172.21.15.42:0/1120677115 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8d2c1039d0 0x7f8d2c0754b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:58:20.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:20.512+0000 7f8d31562640 1 -- 172.21.15.42:0/1120677115 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8d2c1039d0 msgr2=0x7f8d2c0754b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:58:20.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:20.512+0000 7f8d31562640 1 --2- 172.21.15.42:0/1120677115 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8d2c1039d0 0x7f8d2c0754b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:20.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:20.512+0000 7f8d31562640 1 -- 172.21.15.42:0/1120677115 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8d2c109c60 msgr2=0x7f8d2c075f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:58:20.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:20.512+0000 7f8d31562640 1 --2- 172.21.15.42:0/1120677115 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8d2c109c60 0x7f8d2c075f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:20.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:20.512+0000 7f8d31562640 1 -- 172.21.15.42:0/1120677115 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d2c1a8530 con 0x7f8d2c109310 2024-08-30T08:58:20.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:20.512+0000 7f8d31d63640 1 --2- 172.21.15.42:0/1120677115 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8d2c1039d0 0x7f8d2c0754b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-30T08:58:20.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:20.512+0000 7f8d31562640 1 --2- 172.21.15.42:0/1120677115 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8d2c109310 0x7f8d2c0759f0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c02a930 tx=0x7f8d1c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:58:20.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:20.513+0000 7f8d22ffd640 1 -- 172.21.15.42:0/1120677115 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d1c03f030 con 0x7f8d2c109310 2024-08-30T08:58:20.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:20.513+0000 7f8d33fc7640 1 -- 172.21.15.42:0/1120677115 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d2c1a87c0 con 0x7f8d2c109310 2024-08-30T08:58:20.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:20.513+0000 7f8d22ffd640 1 -- 172.21.15.42:0/1120677115 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8d1c003aa0 con 0x7f8d2c109310 2024-08-30T08:58:20.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:20.513+0000 7f8d22ffd640 1 -- 172.21.15.42:0/1120677115 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d1c035070 con 0x7f8d2c109310 2024-08-30T08:58:20.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:20.513+0000 7f8d33fc7640 1 -- 172.21.15.42:0/1120677115 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d2c1a8ca0 con 0x7f8d2c109310 2024-08-30T08:58:20.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:20.514+0000 7f8d33fc7640 1 -- 172.21.15.42:0/1120677115 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d2c103de0 con 0x7f8d2c109310 2024-08-30T08:58:20.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:20.515+0000 7f8d22ffd640 1 -- 172.21.15.42:0/1120677115 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) ==== 152577+0+0 (secure 0 0 0) 0x7f8d1c042030 con 0x7f8d2c109310 2024-08-30T08:58:20.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:20.516+0000 7f8d22ffd640 1 --2- 172.21.15.42:0/1120677115 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f8d180b4b30 0x7f8d180b7040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:58:20.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:20.516+0000 7f8d31d63640 1 --2- 172.21.15.42:0/1120677115 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f8d180b4b30 0x7f8d180b7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:58:20.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:20.516+0000 7f8d22ffd640 1 -- 172.21.15.42:0/1120677115 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8802+0+0 (secure 0 0 0) 0x7f8d1c105ab0 con 0x7f8d2c109310 2024-08-30T08:58:20.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:20.517+0000 7f8d31d63640 1 --2- 172.21.15.42:0/1120677115 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f8d180b4b30 0x7f8d180b7040 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f8d28005d00 tx=0x7f8d28005c90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:58:20.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:20.521+0000 7f8d22ffd640 1 -- 172.21.15.42:0/1120677115 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f8d1c0cc9a0 con 0x7f8d2c109310 2024-08-30T08:58:20.710 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:20.708+0000 7f8d33fc7640 1 -- 172.21.15.42:0/1120677115 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8d2c02e430 con 0x7f8d2c109310 2024-08-30T08:58:20.711 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:58:20 smithi042 ceph-mon[18652]: pgmap v296: 129 pgs: 129 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-30T08:58:20.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:20.710+0000 7f8d22ffd640 1 -- 172.21.15.42:0/1120677115 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) ==== 74+0+24248 (secure 0 0 0) 0x7f8d1c0cc2a0 con 0x7f8d2c109310 2024-08-30T08:58:20.712 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-08-30T08:58:20.712 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":79,"fsid":"b0895c1a-66ac-11ef-bcd4-c7b262605968","created":"2024-08-30T08:49:44.858265+0000","modified":"2024-08-30T08:58:12.223108+0000","last_up_change":"2024-08-30T08:57:12.367791+0000","last_in_change":"2024-08-30T08:56:50.274678+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-30T08:53:27.297022+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-30T08:57:58.489964+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-30T08:58:01.064783+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-08-30T08:58:07.416534+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"029d62d7-a6c6-4905-8c76-6dffa9d57b92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":2617890537},{"type":"v1","addr":"172.21.15.42:6803","nonce":2617890537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":2617890537},{"type":"v1","addr":"172.21.15.42:6805","nonce":2617890537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2617890537},{"type":"v1","addr":"172.21.15.42:6809","nonce":2617890537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":2617890537},{"type":"v1","addr":"172.21.15.42:6807","nonce":2617890537}]},"public_addr":"172.21.15.42:6803/2617890537","cluster_addr":"172.21.15.42:6805/2617890537","heartbeat_back_addr":"172.21.15.42:6809/2617890537","heartbeat_front_addr":"172.21.15.42:6807/2617890537","state":["exists","up"]},{"osd":1,"uuid":"3a78384b-0207-41df-9384-22665007d5b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":2791960082},{"type":"v1","addr":"172.21.15.167:6801","nonce":2791960082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":2791960082},{"type":"v1","addr":"172.21.15.167:6803","nonce":2791960082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":2791960082},{"type":"v1","addr":"172.21.15.167:6807","nonce":2791960082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":2791960082},{"type":"v1","addr":"172.21.15.167:6805","nonce":2791960082}]},"public_addr":"172.21.15.167:6801/2791960082","cluster_addr":"172.21.15.167:6803/2791960082","heartbeat_back_addr":"172.21.15.167:6807/2791960082","heartbeat_front_addr":"172.21.15.167:6805/2791960082","state":["exists","up"]},{"osd":2,"uuid":"16001809-c27c-4f52-948a-9998261acb4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":1101326404},{"type":"v1","addr":"172.21.15.190:6801","nonce":1101326404}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":1101326404},{"type":"v1","addr":"172.21.15.190:6803","nonce":1101326404}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":1101326404},{"type":"v1","addr":"172.21.15.190:6807","nonce":1101326404}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":1101326404},{"type":"v1","addr":"172.21.15.190:6805","nonce":1101326404}]},"public_addr":"172.21.15.190:6801/1101326404","cluster_addr":"172.21.15.190:6803/1101326404","heartbeat_back_addr":"172.21.15.190:6807/1101326404","heartbeat_front_addr":"172.21.15.190:6805/1101326404","state":["exists","up"]},{"osd":3,"uuid":"87bb3e36-9e9e-4b19-9d8c-0cdf6e089d3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":1135599916},{"type":"v1","addr":"172.21.15.42:6811","nonce":1135599916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":1135599916},{"type":"v1","addr":"172.21.15.42:6813","nonce":1135599916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":1135599916},{"type":"v1","addr":"172.21.15.42:6817","nonce":1135599916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":1135599916},{"type":"v1","addr":"172.21.15.42:6815","nonce":1135599916}]},"public_addr":"172.21.15.42:6811/1135599916","cluster_addr":"172.21.15.42:6813/1135599916","heartbeat_back_addr":"172.21.15.42:6817/1135599916","heartbeat_front_addr":"172.21.15.42:6815/1135599916","state":["exists","up"]},{"osd":4,"uuid":"3ca1aae3-c972-4fb9-ac86-236279c30d64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":2927511419},{"type":"v1","addr":"172.21.15.167:6809","nonce":2927511419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":2927511419},{"type":"v1","addr":"172.21.15.167:6811","nonce":2927511419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":2927511419},{"type":"v1","addr":"172.21.15.167:6815","nonce":2927511419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":2927511419},{"type":"v1","addr":"172.21.15.167:6813","nonce":2927511419}]},"public_addr":"172.21.15.167:6809/2927511419","cluster_addr":"172.21.15.167:6811/2927511419","heartbeat_back_addr":"172.21.15.167:6815/2927511419","heartbeat_front_addr":"172.21.15.167:6813/2927511419","state":["exists","up"]},{"osd":5,"uuid":"5d07f54b-0d07-4efe-9bcf-390755ce0258","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":2557145333},{"type":"v1","addr":"172.21.15.190:6809","nonce":2557145333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":2557145333},{"type":"v1","addr":"172.21.15.190:6811","nonce":2557145333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":2557145333},{"type":"v1","addr":"172.21.15.190:6815","nonce":2557145333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":2557145333},{"type":"v1","addr":"172.21.15.190:6813","nonce":2557145333}]},"public_addr":"172.21.15.190:6809/2557145333","cluster_addr":"172.21.15.190:6811/2557145333","heartbeat_back_addr":"172.21.15.190:6815/2557145333","heartbeat_front_addr":"172.21.15.190:6813/2557145333","state":["exists","up"]},{"osd":6,"uuid":"96ad64af-44ff-4197-97d4-156f0cb88e81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":3213526100},{"type":"v1","addr":"172.21.15.42:6819","nonce":3213526100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":3213526100},{"type":"v1","addr":"172.21.15.42:6821","nonce":3213526100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":3213526100},{"type":"v1","addr":"172.21.15.42:6825","nonce":3213526100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":3213526100},{"type":"v1","addr":"172.21.15.42:6823","nonce":3213526100}]},"public_addr":"172.21.15.42:6819/3213526100","cluster_addr":"172.21.15.42:6821/3213526100","heartbeat_back_addr":"172.21.15.42:6825/3213526100","heartbeat_front_addr":"172.21.15.42:6823/3213526100","state":["exists","up"]},{"osd":7,"uuid":"337a2112-eaf7-4e6b-86be-4e40d6ddf835","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":2962799814},{"type":"v1","addr":"172.21.15.167:6817","nonce":2962799814}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":2962799814},{"type":"v1","addr":"172.21.15.167:6819","nonce":2962799814}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":2962799814},{"type":"v1","addr":"172.21.15.167:6823","nonce":2962799814}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":2962799814},{"type":"v1","addr":"172.21.15.167:6821","nonce":2962799814}]},"public_addr":"172.21.15.167:6817/2962799814","cluster_addr":"172.21.15.167:6819/2962799814","heartbeat_back_addr":"172.21.15.167:6823/2962799814","heartbeat_front_addr":"172.21.15.167:6821/2962799814","state":["exists","up"]},{"osd":8,"uuid":"219a7224-4331-464b-ac86-c51054d51d91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":3242198843},{"type":"v1","addr":"172.21.15.190:6817","nonce":3242198843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":3242198843},{"type":"v1","addr":"172.21.15.190:6819","nonce":3242198843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":3242198843},{"type":"v1","addr":"172.21.15.190:6823","nonce":3242198843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":3242198843},{"type":"v1","addr":"172.21.15.190:6821","nonce":3242198843}]},"public_addr":"172.21.15.190:6817/3242198843","cluster_addr":"172.21.15.190:6819/3242198843","heartbeat_back_addr":"172.21.15.190:6823/3242198843","heartbeat_front_addr":"172.21.15.190:6821/3242198843","state":["exists","up"]},{"osd":9,"uuid":"2f2f5177-a009-40d9-991c-76aec8a5cba6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":1123536325},{"type":"v1","addr":"172.21.15.42:6827","nonce":1123536325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":1123536325},{"type":"v1","addr":"172.21.15.42:6829","nonce":1123536325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":1123536325},{"type":"v1","addr":"172.21.15.42:6833","nonce":1123536325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":1123536325},{"type":"v1","addr":"172.21.15.42:6831","nonce":1123536325}]},"public_addr":"172.21.15.42:6827/1123536325","cluster_addr":"172.21.15.42:6829/1123536325","heartbeat_back_addr":"172.21.15.42:6833/1123536325","heartbeat_front_addr":"172.21.15.42:6831/1123536325","state":["exists","up"]},{"osd":10,"uuid":"96ef5058-2518-4862-a165-17666b33a3e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":3451198319},{"type":"v1","addr":"172.21.15.167:6825","nonce":3451198319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":3451198319},{"type":"v1","addr":"172.21.15.167:6827","nonce":3451198319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":3451198319},{"type":"v1","addr":"172.21.15.167:6831","nonce":3451198319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":3451198319},{"type":"v1","addr":"172.21.15.167:6829","nonce":3451198319}]},"public_addr":"172.21.15.167:6825/3451198319","cluster_addr":"172.21.15.167:6827/3451198319","heartbeat_back_addr":"172.21.15.167:6831/3451198319","heartbeat_front_addr":"172.21.15.167:6829/3451198319","state":["exists","up"]},{"osd":11,"uuid":"a4eb9bd3-3faa-4c81-ba6d-c2084b72c9ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":380064457},{"type":"v1","addr":"172.21.15.190:6825","nonce":380064457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":380064457},{"type":"v1","addr":"172.21.15.190:6827","nonce":380064457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":380064457},{"type":"v1","addr":"172.21.15.190:6831","nonce":380064457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":380064457},{"type":"v1","addr":"172.21.15.190:6829","nonce":380064457}]},"public_addr":"172.21.15.190:6825/380064457","cluster_addr":"172.21.15.190:6827/380064457","heartbeat_back_addr":"172.21.15.190:6831/380064457","heartbeat_front_addr":"172.21.15.190:6829/380064457","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-30T08:52:36.511448+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-30T08:53:00.027277+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-30T08:53:23.770245+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-30T08:53:49.322226+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-30T08:54:14.350234+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-30T08:54:39.491793+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-30T08:55:04.870596+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-30T08:55:30.314150+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-30T08:55:54.483843+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-30T08:56:20.586115+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-30T08:56:45.666783+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-30T08:57:10.718387+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.42:0/2691233260":"2024-08-31T08:50:44.821070+0000","172.21.15.42:0/1518053351":"2024-08-31T08:50:44.821070+0000","172.21.15.42:0/352001020":"2024-08-31T08:50:44.821070+0000","172.21.15.42:6801/1733768634":"2024-08-31T08:50:44.821070+0000","172.21.15.42:0/1000597888":"2024-08-31T08:50:18.561851+0000","172.21.15.42:6800/1733768634":"2024-08-31T08:50:44.821070+0000","172.21.15.42:6801/3538882081":"2024-08-31T08:50:18.561851+0000","172.21.15.42:6800/3538882081":"2024-08-31T08:50:18.561851+0000","172.21.15.42:0/145366508":"2024-08-31T08:50:18.561851+0000","172.21.15.42:0/3346355213":"2024-08-31T08:50:18.561851+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-30T08:58:20.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:20.712+0000 7f8d33fc7640 1 -- 172.21.15.42:0/1120677115 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f8d180b4b30 msgr2=0x7f8d180b7040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:58:20.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:20.712+0000 7f8d33fc7640 1 --2- 172.21.15.42:0/1120677115 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f8d180b4b30 0x7f8d180b7040 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f8d28005d00 tx=0x7f8d28005c90 comp rx=0 tx=0).stop 2024-08-30T08:58:20.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:20.712+0000 7f8d33fc7640 1 -- 172.21.15.42:0/1120677115 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8d2c109310 msgr2=0x7f8d2c0759f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:58:20.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:20.712+0000 7f8d33fc7640 1 --2- 172.21.15.42:0/1120677115 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8d2c109310 0x7f8d2c0759f0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c02a930 tx=0x7f8d1c004270 comp rx=0 tx=0).stop 2024-08-30T08:58:20.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:20.712+0000 7f8d33fc7640 1 -- 172.21.15.42:0/1120677115 shutdown_connections 2024-08-30T08:58:20.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:20.712+0000 7f8d33fc7640 1 --2- 172.21.15.42:0/1120677115 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f8d180b4b30 0x7f8d180b7040 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:20.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:20.712+0000 7f8d33fc7640 1 --2- 172.21.15.42:0/1120677115 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8d2c109c60 0x7f8d2c075f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:20.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:20.712+0000 7f8d33fc7640 1 --2- 172.21.15.42:0/1120677115 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8d2c109310 0x7f8d2c0759f0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:20.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:20.712+0000 7f8d33fc7640 1 --2- 172.21.15.42:0/1120677115 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8d2c1039d0 0x7f8d2c0754b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:20.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:20.712+0000 7f8d33fc7640 1 -- 172.21.15.42:0/1120677115 >> 172.21.15.42:0/1120677115 conn(0x7f8d2c07b670 msgr2=0x7f8d2c107400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:58:20.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:20.712+0000 7f8d33fc7640 1 -- 172.21.15.42:0/1120677115 shutdown_connections 2024-08-30T08:58:20.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:20.712+0000 7f8d33fc7640 1 -- 172.21.15.42:0/1120677115 wait complete. 2024-08-30T08:58:20.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:58:20 smithi167 ceph-mon[22063]: pgmap v296: 129 pgs: 129 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-30T08:58:20.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:58:20 smithi190 ceph-mon[22001]: pgmap v296: 129 pgs: 129 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-30T08:58:21.362 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-08-30T08:58:03.055084+0000', 'modified': '2024-08-30T08:58:12.223290+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24629}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24629': {'gid': 24629, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.167:6839/3604645207', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.167:6838', 'nonce': 3604645207}, {'type': 'v1', 'addr': '172.21.15.167:6839', 'nonce': 3604645207}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1, 'qdb_leader': 24629, 'qdb_cluster': [24629]} 2024-08-30T08:58:21.362 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-08-30T08:58:21.363 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph tell mds.b status 2024-08-30T08:58:21.617 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T08:58:21.632 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:58:21 smithi042 ceph-mon[18652]: from='client.? 172.21.15.42:0/1120677115' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-30T08:58:21.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:58:21 smithi190 ceph-mon[22001]: from='client.? 172.21.15.42:0/1120677115' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-30T08:58:21.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:58:21 smithi167 ceph-mon[22063]: from='client.? 172.21.15.42:0/1120677115' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-30T08:58:22.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.126+0000 7f100f69f640 1 -- 172.21.15.42:0/3854889437 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1008108aa0 msgr2=0x7f100810f2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:58:22.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.126+0000 7f100f69f640 1 --2- 172.21.15.42:0/3854889437 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1008108aa0 0x7f100810f2d0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f1000009900 tx=0x7f100002a410 comp rx=0 tx=0).stop 2024-08-30T08:58:22.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.126+0000 7f100f69f640 1 -- 172.21.15.42:0/3854889437 shutdown_connections 2024-08-30T08:58:22.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.126+0000 7f100f69f640 1 --2- 172.21.15.42:0/3854889437 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1008108aa0 0x7f100810f2d0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:22.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.126+0000 7f100f69f640 1 --2- 172.21.15.42:0/3854889437 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1008108150 0x7f1008108560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:22.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.126+0000 7f100f69f640 1 --2- 172.21.15.42:0/3854889437 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1008102810 0x7f1008102c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:22.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.126+0000 7f100f69f640 1 -- 172.21.15.42:0/3854889437 >> 172.21.15.42:0/3854889437 conn(0x7f10080794b0 msgr2=0x7f10080798c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:58:22.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.126+0000 7f100f69f640 1 -- 172.21.15.42:0/3854889437 shutdown_connections 2024-08-30T08:58:22.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.126+0000 7f100f69f640 1 -- 172.21.15.42:0/3854889437 wait complete. 2024-08-30T08:58:22.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.127+0000 7f100f69f640 1 Processor -- start 2024-08-30T08:58:22.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.127+0000 7f100f69f640 1 -- start start 2024-08-30T08:58:22.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.127+0000 7f100f69f640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1008102810 0x7f100819c410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:58:22.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.127+0000 7f100f69f640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1008108150 0x7f100819c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:58:22.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.127+0000 7f100f69f640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1008108aa0 0x7f10081a1c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:58:22.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.127+0000 7f100f69f640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f1008111660 con 0x7f1008102810 2024-08-30T08:58:22.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.127+0000 7f100f69f640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f10081114e0 con 0x7f1008108150 2024-08-30T08:58:22.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.127+0000 7f100f69f640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f10081117e0 con 0x7f1008108aa0 2024-08-30T08:58:22.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.127+0000 7f100d43b640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1008102810 0x7f100819c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:58:22.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.127+0000 7f100d43b640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1008102810 0x7f100819c410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36498/0 (socket says 172.21.15.42:36498) 2024-08-30T08:58:22.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.127+0000 7f100d43b640 1 -- 172.21.15.42:0/7197037 learned_addr learned my addr 172.21.15.42:0/7197037 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T08:58:22.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.127+0000 7f100cc3a640 1 --2- 172.21.15.42:0/7197037 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1008108150 0x7f100819c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:58:22.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.127+0000 7f100dc3c640 1 --2- 172.21.15.42:0/7197037 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1008108aa0 0x7f10081a1c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:58:22.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.127+0000 7f100d43b640 1 -- 172.21.15.42:0/7197037 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1008108aa0 msgr2=0x7f10081a1c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:58:22.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.127+0000 7f100d43b640 1 --2- 172.21.15.42:0/7197037 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1008108aa0 0x7f10081a1c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:22.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.127+0000 7f100d43b640 1 -- 172.21.15.42:0/7197037 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1008108150 msgr2=0x7f100819c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:58:22.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.127+0000 7f100d43b640 1 --2- 172.21.15.42:0/7197037 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1008108150 0x7f100819c9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:22.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.127+0000 7f100d43b640 1 -- 172.21.15.42:0/7197037 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10081a2380 con 0x7f1008102810 2024-08-30T08:58:22.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.128+0000 7f100d43b640 1 --2- 172.21.15.42:0/7197037 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1008102810 0x7f100819c410 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f0ff800db10 tx=0x7f0ff800b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:58:22.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.128+0000 7f10067fc640 1 -- 172.21.15.42:0/7197037 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ff801b020 con 0x7f1008102810 2024-08-30T08:58:22.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.128+0000 7f100f69f640 1 -- 172.21.15.42:0/7197037 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10081a2670 con 0x7f1008102810 2024-08-30T08:58:22.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.128+0000 7f10067fc640 1 -- 172.21.15.42:0/7197037 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ff800bc70 con 0x7f1008102810 2024-08-30T08:58:22.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.128+0000 7f10067fc640 1 -- 172.21.15.42:0/7197037 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ff8005990 con 0x7f1008102810 2024-08-30T08:58:22.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.128+0000 7f100f69f640 1 -- 172.21.15.42:0/7197037 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10081a8dd0 con 0x7f1008102810 2024-08-30T08:58:22.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.131+0000 7f1015c14740 1 Processor -- start 2024-08-30T08:58:22.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.131+0000 7f1015c14740 1 -- start start 2024-08-30T08:58:22.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.131+0000 7f1015c14740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5617052b3f60 0x5617056597b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:58:22.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.131+0000 7f1015c14740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56170528b0a0 0x561705659cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:58:22.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.131+0000 7f1015c14740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561705293280 0x56170565a230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:58:22.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.131+0000 7f1015c14740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x5617055d03f0 con 0x561705293280 2024-08-30T08:58:22.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.131+0000 7f1015c14740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5617055bf980 con 0x56170528b0a0 2024-08-30T08:58:22.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.131+0000 7f1015c14740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5617055c3930 con 0x5617052b3f60 2024-08-30T08:58:22.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.131+0000 7f10067fc640 1 -- 172.21.15.42:0/7197037 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) ==== 152577+0+0 (secure 0 0 0) 0x7f0ff8004a40 con 0x7f1008102810 2024-08-30T08:58:22.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.132+0000 7f100cc3a640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561705293280 0x56170565a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:58:22.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.132+0000 7f100dc3c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5617052b3f60 0x5617056597b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:58:22.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.132+0000 7f100cc3a640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561705293280 0x56170565a230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36514/0 (socket says 172.21.15.42:36514) 2024-08-30T08:58:22.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.132+0000 7f100cc3a640 1 -- 172.21.15.42:0/636969486 learned_addr learned my addr 172.21.15.42:0/636969486 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T08:58:22.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.132+0000 7f100d43b640 1 --2- 172.21.15.42:0/636969486 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56170528b0a0 0x561705659cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:58:22.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.132+0000 7f100cc3a640 1 -- 172.21.15.42:0/636969486 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5617052b3f60 msgr2=0x5617056597b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T08:58:22.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.132+0000 7f100cc3a640 1 --2- 172.21.15.42:0/636969486 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5617052b3f60 0x5617056597b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:22.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.132+0000 7f100cc3a640 1 -- 172.21.15.42:0/636969486 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56170528b0a0 msgr2=0x561705659cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T08:58:22.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.132+0000 7f100cc3a640 1 --2- 172.21.15.42:0/636969486 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56170528b0a0 0x561705659cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:22.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.132+0000 7f100cc3a640 1 -- 172.21.15.42:0/636969486 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56170531f540 con 0x561705293280 2024-08-30T08:58:22.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.132+0000 7f100dc3c640 1 --2- 172.21.15.42:0/636969486 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5617052b3f60 0x5617056597b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-30T08:58:22.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.132+0000 7f100d43b640 1 --2- 172.21.15.42:0/636969486 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56170528b0a0 0x561705659cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-30T08:58:22.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.132+0000 7f100cc3a640 1 --2- 172.21.15.42:0/636969486 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561705293280 0x56170565a230 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f0ffc00bc30 tx=0x7f0ffc0107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b0de14b6c4f2cea server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:58:22.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.132+0000 7f10067fc640 1 --2- 172.21.15.42:0/7197037 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f0ff00b4b30 0x7f0ff00b7040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:58:22.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.132+0000 7f0fef7fe640 1 -- 172.21.15.42:0/636969486 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ffc01c480 con 0x561705293280 2024-08-30T08:58:22.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.132+0000 7f100dc3c640 1 --2- 172.21.15.42:0/7197037 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f0ff00b4b30 0x7f0ff00b7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:58:22.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.132+0000 7f0fef7fe640 1 -- 172.21.15.42:0/636969486 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ffc01cad0 con 0x561705293280 2024-08-30T08:58:22.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.132+0000 7f0fef7fe640 1 -- 172.21.15.42:0/636969486 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ffc011b50 con 0x561705293280 2024-08-30T08:58:22.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.133+0000 7f10067fc640 1 -- 172.21.15.42:0/7197037 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8802+0+0 (secure 0 0 0) 0x7f0ff80e5320 con 0x7f1008102810 2024-08-30T08:58:22.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.133+0000 7f1015c14740 1 -- 172.21.15.42:0/636969486 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561705293280 msgr2=0x56170565a230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:58:22.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.133+0000 7f100dc3c640 1 --2- 172.21.15.42:0/7197037 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f0ff00b4b30 0x7f0ff00b7040 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f1000005ab0 tx=0x7f1000005a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:58:22.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.133+0000 7f1015c14740 1 --2- 172.21.15.42:0/636969486 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561705293280 0x56170565a230 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f0ffc00bc30 tx=0x7f0ffc0107a0 comp rx=0 tx=0).stop 2024-08-30T08:58:22.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.133+0000 7f1015c14740 1 -- 172.21.15.42:0/636969486 shutdown_connections 2024-08-30T08:58:22.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.133+0000 7f1015c14740 1 --2- 172.21.15.42:0/636969486 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561705293280 0x56170565a230 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:22.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.133+0000 7f1015c14740 1 --2- 172.21.15.42:0/636969486 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56170528b0a0 0x561705659cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:22.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.133+0000 7f1015c14740 1 --2- 172.21.15.42:0/636969486 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5617052b3f60 0x5617056597b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:22.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.133+0000 7f1015c14740 1 -- 172.21.15.42:0/636969486 >> 172.21.15.42:0/636969486 conn(0x5617052a8350 msgr2=0x5617056576a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:58:22.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.134+0000 7f1015c14740 1 -- 172.21.15.42:0/636969486 shutdown_connections 2024-08-30T08:58:22.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.134+0000 7f1015c14740 1 -- 172.21.15.42:0/636969486 wait complete. 2024-08-30T08:58:22.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.134+0000 7f1015c14740 1 -- start start 2024-08-30T08:58:22.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.134+0000 7f1015c14740 1 Processor -- start 2024-08-30T08:58:22.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.134+0000 7f1015c14740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56170528b0a0 0x561705664dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:58:22.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.134+0000 7f1015c14740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561705293280 0x561705665300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:58:22.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.134+0000 7f1015c14740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5617052b3f60 0x561705665860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:58:22.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.134+0000 7f1015c14740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x5617054225f0 con 0x56170528b0a0 2024-08-30T08:58:22.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.134+0000 7f1015c14740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x561705546600 con 0x5617052b3f60 2024-08-30T08:58:22.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.134+0000 7f1015c14740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x561705667f90 con 0x561705293280 2024-08-30T08:58:22.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.135+0000 7f1015c14740 10 client.0 _ll_register_callbacks cb 0x56170556caf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T08:58:22.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.135+0000 7f100d43b640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56170528b0a0 0x561705664dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:58:22.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.135+0000 7f100d43b640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56170528b0a0 0x561705664dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36526/0 (socket says 172.21.15.42:36526) 2024-08-30T08:58:22.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.135+0000 7f100d43b640 1 -- 172.21.15.42:0/303614432 learned_addr learned my addr 172.21.15.42:0/303614432 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T08:58:22.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.135+0000 7f100cc3a640 1 --2- 172.21.15.42:0/303614432 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561705293280 0x561705665300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:58:22.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.135+0000 7f100dc3c640 1 --2- 172.21.15.42:0/303614432 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5617052b3f60 0x561705665860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:58:22.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.135+0000 7f100d43b640 1 -- 172.21.15.42:0/303614432 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561705293280 msgr2=0x561705665300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T08:58:22.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.135+0000 7f100d43b640 1 --2- 172.21.15.42:0/303614432 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561705293280 0x561705665300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:22.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.135+0000 7f100d43b640 1 -- 172.21.15.42:0/303614432 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5617052b3f60 msgr2=0x561705665860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T08:58:22.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.135+0000 7f100d43b640 1 --2- 172.21.15.42:0/303614432 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5617052b3f60 0x561705665860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:22.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.135+0000 7f100d43b640 1 -- 172.21.15.42:0/303614432 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56170557f7b0 con 0x56170528b0a0 2024-08-30T08:58:22.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.135+0000 7f100d43b640 1 --2- 172.21.15.42:0/303614432 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56170528b0a0 0x561705664dc0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f0ff800dfd0 tx=0x7f0ff80ab5e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7178383151855cb server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:58:22.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.135+0000 7f0fecff9640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-08-30T08:58:22.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.135+0000 7f0fecff9640 1 -- 172.21.15.42:0/303614432 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ff80ab6b0 con 0x56170528b0a0 2024-08-30T08:58:22.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.135+0000 7f0fecff9640 1 -- 172.21.15.42:0/303614432 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8802+0+0 (secure 0 0 0) 0x7f0ff80aae00 con 0x56170528b0a0 2024-08-30T08:58:22.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.135+0000 7f1015c14740 1 -- 172.21.15.42:0/303614432 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ffc01cad0 con 0x56170528b0a0 2024-08-30T08:58:22.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.136+0000 7f0fecff9640 1 -- 172.21.15.42:0/303614432 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f0ff80ac2a0 con 0x56170528b0a0 2024-08-30T08:58:22.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.136+0000 7f1015c14740 10 client.14883 fetch_fsmap learned FSMap version 17 2024-08-30T08:58:22.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.136+0000 7f1015c14740 1 -- 172.21.15.42:0/303614432 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) -- 0x561705673c10 con 0x56170528b0a0 2024-08-30T08:58:22.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.136+0000 7f0fecff9640 1 -- 172.21.15.42:0/303614432 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ffc01cad0 con 0x56170528b0a0 2024-08-30T08:58:22.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.136+0000 7f0fecff9640 1 -- 172.21.15.42:0/303614432 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 17) ==== 8350+0+0 (secure 0 0 0) 0x7f0ff8016070 con 0x56170528b0a0 2024-08-30T08:58:22.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.136+0000 7f1015c14740 10 client.14883 fetch_fsmap finished waiting for FSMap version 17 2024-08-30T08:58:22.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.136+0000 7f1015c14740 10 client.14883 resolve_mds: resolved name 'b' to daemon mds.b 2024-08-30T08:58:22.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.136+0000 7f1015c14740 20 client.14883 populate_metadata read hostname 'smithi042' 2024-08-30T08:58:22.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.136+0000 7f1015c14740 1 --2- 172.21.15.42:0/303614432 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x5617052728e0 0x561705677670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:58:22.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.136+0000 7f1015c14740 4 client.14883 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-30T08:58:22.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.136+0000 7f1015c14740 1 -- 172.21.15.42:0/303614432 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5617055d2340 con 0x5617052728e0 2024-08-30T08:58:22.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.136+0000 7f100cc3a640 1 --2- 172.21.15.42:0/303614432 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x5617052728e0 0x561705677670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:58:22.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.137+0000 7f100cc3a640 1 --2- 172.21.15.42:0/303614432 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x5617052728e0 0x561705677670 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=539f19b116245c89 server_cookie=2933c9b96932e9af in_seq=0 out_seq=0 2024-08-30T08:58:22.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.137+0000 7f0fecff9640 10 client.14883 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T08:58:22.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.138+0000 7f0fecff9640 1 -- 172.21.15.42:0/303614432 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5617055d2340 con 0x5617052728e0 2024-08-30T08:58:22.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.138+0000 7f0fecff9640 10 client.14883 handle_command_reply: tid=0 2024-08-30T08:58:22.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.138+0000 7f1015c14740 1 client.14883 shutdown 2024-08-30T08:58:22.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.138+0000 7f1015c14740 1 -- 172.21.15.42:0/303614432 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56170528b0a0 msgr2=0x561705664dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:58:22.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.138+0000 7f1015c14740 1 --2- 172.21.15.42:0/303614432 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56170528b0a0 0x561705664dc0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f0ff800dfd0 tx=0x7f0ff80ab5e0 comp rx=0 tx=0).stop 2024-08-30T08:58:22.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.138+0000 7f1015c14740 1 -- 172.21.15.42:0/303614432 shutdown_connections 2024-08-30T08:58:22.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.138+0000 7f1015c14740 1 --2- 172.21.15.42:0/303614432 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x5617052728e0 0x561705677670 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:22.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.138+0000 7f1015c14740 1 --2- 172.21.15.42:0/303614432 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561705293280 0x561705665300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:22.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.138+0000 7f1015c14740 1 --2- 172.21.15.42:0/303614432 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5617052b3f60 0x561705665860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:22.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.138+0000 7f1015c14740 1 --2- 172.21.15.42:0/303614432 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56170528b0a0 0x561705664dc0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:22.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.138+0000 7f1015c14740 1 -- 172.21.15.42:0/303614432 >> 172.21.15.42:0/303614432 conn(0x5617052a8350 msgr2=0x56170565c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:58:22.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.138+0000 7f0fecff9640 0 client.14883 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T08:58:22.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.138+0000 7f1015c14740 1 -- 172.21.15.42:0/303614432 shutdown_connections 2024-08-30T08:58:22.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.138+0000 7f1015c14740 1 -- 172.21.15.42:0/303614432 wait complete. 2024-08-30T08:58:22.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.138+0000 7f1015c14740 20 client.14883 trim_cache size 0 max 16384 2024-08-30T08:58:22.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.157+0000 7f1015c14740 1 Processor -- start 2024-08-30T08:58:22.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.157+0000 7f1015c14740 1 -- start start 2024-08-30T08:58:22.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.157+0000 7f1015c14740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56170528b0a0 0x561705676200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:58:22.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.157+0000 7f1015c14740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561705293280 0x5617056750d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:58:22.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.158+0000 7f1015c14740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5617052b3f60 0x561705663640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:58:22.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.158+0000 7f1015c14740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x5617056759b0 con 0x561705293280 2024-08-30T08:58:22.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.158+0000 7f1015c14740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x561705676080 con 0x5617052b3f60 2024-08-30T08:58:22.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.158+0000 7f1015c14740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x561705676750 con 0x56170528b0a0 2024-08-30T08:58:22.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.158+0000 7f100cc3a640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561705293280 0x5617056750d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:58:22.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.158+0000 7f100cc3a640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561705293280 0x5617056750d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36528/0 (socket says 172.21.15.42:36528) 2024-08-30T08:58:22.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.158+0000 7f100cc3a640 1 -- 172.21.15.42:0/388243735 learned_addr learned my addr 172.21.15.42:0/388243735 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T08:58:22.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.158+0000 7f100d43b640 1 --2- 172.21.15.42:0/388243735 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56170528b0a0 0x561705676200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:58:22.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.158+0000 7f100cc3a640 1 -- 172.21.15.42:0/388243735 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56170528b0a0 msgr2=0x561705676200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T08:58:22.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.158+0000 7f100cc3a640 1 --2- 172.21.15.42:0/388243735 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56170528b0a0 0x561705676200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:22.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.158+0000 7f100cc3a640 1 -- 172.21.15.42:0/388243735 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5617052b3f60 msgr2=0x561705663640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T08:58:22.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.158+0000 7f100cc3a640 1 --2- 172.21.15.42:0/388243735 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5617052b3f60 0x561705663640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:22.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.158+0000 7f100cc3a640 1 -- 172.21.15.42:0/388243735 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561705660930 con 0x561705293280 2024-08-30T08:58:22.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.158+0000 7f100d43b640 1 --2- 172.21.15.42:0/388243735 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56170528b0a0 0x561705676200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-30T08:58:22.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.158+0000 7f100cc3a640 1 --2- 172.21.15.42:0/388243735 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561705293280 0x5617056750d0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f0ffc011620 tx=0x7f0ffc01e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27a65108918b71ed server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:58:22.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.158+0000 7f0feeffd640 1 -- 172.21.15.42:0/388243735 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ffc033070 con 0x561705293280 2024-08-30T08:58:22.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.158+0000 7f0feeffd640 1 -- 172.21.15.42:0/388243735 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ffc00b830 con 0x561705293280 2024-08-30T08:58:22.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.158+0000 7f0feeffd640 1 -- 172.21.15.42:0/388243735 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ffc0035f0 con 0x561705293280 2024-08-30T08:58:22.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.159+0000 7f1015c14740 1 -- 172.21.15.42:0/388243735 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561705293280 msgr2=0x5617056750d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:58:22.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.159+0000 7f1015c14740 1 --2- 172.21.15.42:0/388243735 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561705293280 0x5617056750d0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f0ffc011620 tx=0x7f0ffc01e040 comp rx=0 tx=0).stop 2024-08-30T08:58:22.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.159+0000 7f1015c14740 1 -- 172.21.15.42:0/388243735 shutdown_connections 2024-08-30T08:58:22.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.159+0000 7f1015c14740 1 --2- 172.21.15.42:0/388243735 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5617052b3f60 0x561705663640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:22.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.159+0000 7f1015c14740 1 --2- 172.21.15.42:0/388243735 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561705293280 0x5617056750d0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:22.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.159+0000 7f1015c14740 1 --2- 172.21.15.42:0/388243735 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56170528b0a0 0x561705676200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:22.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.159+0000 7f1015c14740 1 -- 172.21.15.42:0/388243735 >> 172.21.15.42:0/388243735 conn(0x5617052a8350 msgr2=0x5617056713b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:58:22.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.159+0000 7f1015c14740 1 -- 172.21.15.42:0/388243735 shutdown_connections 2024-08-30T08:58:22.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.159+0000 7f1015c14740 1 -- 172.21.15.42:0/388243735 wait complete. 2024-08-30T08:58:22.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.160+0000 7f1015c14740 1 -- start start 2024-08-30T08:58:22.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.160+0000 7f1015c14740 1 Processor -- start 2024-08-30T08:58:22.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.160+0000 7f1015c14740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56170528b0a0 0x56170567fc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:58:22.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.160+0000 7f1015c14740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561705293280 0x561705680200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:58:22.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.160+0000 7f1015c14740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5617052b3f60 0x561705679130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:58:22.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.160+0000 7f1015c14740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x5617056774f0 con 0x56170528b0a0 2024-08-30T08:58:22.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.160+0000 7f1015c14740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x561705676e20 con 0x5617052b3f60 2024-08-30T08:58:22.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.160+0000 7f1015c14740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x561705677bc0 con 0x561705293280 2024-08-30T08:58:22.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.160+0000 7f1015c14740 10 client.0 _ll_register_callbacks cb 0x7f0ff80aba60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T08:58:22.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.160+0000 7f100d43b640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56170528b0a0 0x56170567fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:58:22.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.160+0000 7f100d43b640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56170528b0a0 0x56170567fc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36530/0 (socket says 172.21.15.42:36530) 2024-08-30T08:58:22.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.160+0000 7f100d43b640 1 -- 172.21.15.42:0/3595811473 learned_addr learned my addr 172.21.15.42:0/3595811473 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T08:58:22.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.161+0000 7f100dc3c640 1 --2- 172.21.15.42:0/3595811473 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5617052b3f60 0x561705679130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:58:22.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.161+0000 7f100cc3a640 1 --2- 172.21.15.42:0/3595811473 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561705293280 0x561705680200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:58:22.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.161+0000 7f100d43b640 1 -- 172.21.15.42:0/3595811473 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561705293280 msgr2=0x561705680200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T08:58:22.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.161+0000 7f100d43b640 1 --2- 172.21.15.42:0/3595811473 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561705293280 0x561705680200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:22.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.161+0000 7f100d43b640 1 -- 172.21.15.42:0/3595811473 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5617052b3f60 msgr2=0x561705679130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T08:58:22.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.161+0000 7f100d43b640 1 --2- 172.21.15.42:0/3595811473 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5617052b3f60 0x561705679130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:22.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.161+0000 7f100d43b640 1 -- 172.21.15.42:0/3595811473 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5617056798c0 con 0x56170528b0a0 2024-08-30T08:58:22.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.161+0000 7f100d43b640 1 --2- 172.21.15.42:0/3595811473 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56170528b0a0 0x56170567fc20 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f0ff80a8e10 tx=0x7f0ff8093160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ff3fb9b764a300 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:58:22.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.161+0000 7f0fecff9640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-08-30T08:58:22.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.161+0000 7f0fecff9640 1 -- 172.21.15.42:0/3595811473 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ff80ab6b0 con 0x56170528b0a0 2024-08-30T08:58:22.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.161+0000 7f0fecff9640 1 -- 172.21.15.42:0/3595811473 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8802+0+0 (secure 0 0 0) 0x7f0ff80aaa80 con 0x56170528b0a0 2024-08-30T08:58:22.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.161+0000 7f1015c14740 1 -- 172.21.15.42:0/3595811473 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5617055cdde0 con 0x56170528b0a0 2024-08-30T08:58:22.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.161+0000 7f0fecff9640 1 -- 172.21.15.42:0/3595811473 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f0ff80b0410 con 0x56170528b0a0 2024-08-30T08:58:22.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.161+0000 7f1015c14740 10 client.14889 fetch_fsmap learned FSMap version 17 2024-08-30T08:58:22.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.161+0000 7f1015c14740 1 -- 172.21.15.42:0/3595811473 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) -- 0x56170567f2c0 con 0x56170528b0a0 2024-08-30T08:58:22.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.161+0000 7f0fecff9640 1 -- 172.21.15.42:0/3595811473 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5617055cdde0 con 0x56170528b0a0 2024-08-30T08:58:22.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.162+0000 7f0fecff9640 1 -- 172.21.15.42:0/3595811473 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 17) ==== 8350+0+0 (secure 0 0 0) 0x7f0ff8016070 con 0x56170528b0a0 2024-08-30T08:58:22.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.162+0000 7f1015c14740 10 client.14889 fetch_fsmap finished waiting for FSMap version 17 2024-08-30T08:58:22.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.162+0000 7f1015c14740 10 client.14889 resolve_mds: resolved name 'b' to daemon mds.b 2024-08-30T08:58:22.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.162+0000 7f1015c14740 20 client.14889 populate_metadata read hostname 'smithi042' 2024-08-30T08:58:22.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.162+0000 7f1015c14740 1 --2- 172.21.15.42:0/3595811473 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x561705247c90 0x5617056e5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:58:22.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.162+0000 7f1015c14740 4 client.14889 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "status"}] 2024-08-30T08:58:22.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.162+0000 7f1015c14740 1 -- 172.21.15.42:0/3595811473 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "status"}) -- 0x5617055cb520 con 0x561705247c90 2024-08-30T08:58:22.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.162+0000 7f100cc3a640 1 --2- 172.21.15.42:0/3595811473 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x561705247c90 0x5617056e5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:58:22.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.163+0000 7f0fecff9640 10 client.14889 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T08:58:22.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.163+0000 7f100cc3a640 1 --2- 172.21.15.42:0/3595811473 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x561705247c90 0x5617056e5eb0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b62d3521cb4489e3 server_cookie=7be2c40cd9e11a14 in_seq=0 out_seq=0 2024-08-30T08:58:22.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.163+0000 7f0fecff9640 1 -- 172.21.15.42:0/3595811473 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x5617055cb520 con 0x561705247c90 2024-08-30T08:58:22.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.163+0000 7f0fecff9640 10 client.14889 handle_command_reply: tid=0 2024-08-30T08:58:22.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.163+0000 7f1015c14740 1 client.14889 shutdown 2024-08-30T08:58:22.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.163+0000 7f1015c14740 1 -- 172.21.15.42:0/3595811473 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56170528b0a0 msgr2=0x56170567fc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:58:22.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.163+0000 7f1015c14740 1 --2- 172.21.15.42:0/3595811473 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56170528b0a0 0x56170567fc20 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f0ff80a8e10 tx=0x7f0ff8093160 comp rx=0 tx=0).stop 2024-08-30T08:58:22.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.163+0000 7f1015c14740 1 -- 172.21.15.42:0/3595811473 shutdown_connections 2024-08-30T08:58:22.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.163+0000 7f1015c14740 1 --2- 172.21.15.42:0/3595811473 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x561705247c90 0x5617056e5eb0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:22.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.163+0000 7f1015c14740 1 --2- 172.21.15.42:0/3595811473 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561705293280 0x561705680200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:22.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.163+0000 7f1015c14740 1 --2- 172.21.15.42:0/3595811473 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5617052b3f60 0x561705679130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:22.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.163+0000 7f1015c14740 1 --2- 172.21.15.42:0/3595811473 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56170528b0a0 0x56170567fc20 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:22.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.163+0000 7f1015c14740 1 -- 172.21.15.42:0/3595811473 >> 172.21.15.42:0/3595811473 conn(0x5617052a8350 msgr2=0x5617056750d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:58:22.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.163+0000 7f0fecff9640 0 client.14889 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T08:58:22.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.163+0000 7f1015c14740 1 -- 172.21.15.42:0/3595811473 shutdown_connections 2024-08-30T08:58:22.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.163+0000 7f1015c14740 1 -- 172.21.15.42:0/3595811473 wait complete. 2024-08-30T08:58:22.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.163+0000 7f1015c14740 20 client.14889 trim_cache size 0 max 16384 2024-08-30T08:58:22.170 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-08-30T08:58:22.170 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_fsid": "b0895c1a-66ac-11ef-bcd4-c7b262605968", 2024-08-30T08:58:22.170 INFO:teuthology.orchestra.run.smithi042.stdout: "whoami": 0, 2024-08-30T08:58:22.170 INFO:teuthology.orchestra.run.smithi042.stdout: "id": 24629, 2024-08-30T08:58:22.170 INFO:teuthology.orchestra.run.smithi042.stdout: "want_state": "up:active", 2024-08-30T08:58:22.170 INFO:teuthology.orchestra.run.smithi042.stdout: "state": "up:active", 2024-08-30T08:58:22.171 INFO:teuthology.orchestra.run.smithi042.stdout: "fs_name": "cephfs", 2024-08-30T08:58:22.171 INFO:teuthology.orchestra.run.smithi042.stdout: "rank_uptime": 18.763734927000002, 2024-08-30T08:58:22.171 INFO:teuthology.orchestra.run.smithi042.stdout: "mdsmap_epoch": 17, 2024-08-30T08:58:22.171 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_epoch": 79, 2024-08-30T08:58:22.171 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_epoch_barrier": 79, 2024-08-30T08:58:22.171 INFO:teuthology.orchestra.run.smithi042.stdout: "uptime": 29.532066978 2024-08-30T08:58:22.171 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-08-30T08:58:22.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.164+0000 7f100f69f640 1 -- 172.21.15.42:0/7197037 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f0ff00b4b30 msgr2=0x7f0ff00b7040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:58:22.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.164+0000 7f100f69f640 1 --2- 172.21.15.42:0/7197037 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f0ff00b4b30 0x7f0ff00b7040 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f1000005ab0 tx=0x7f1000005a40 comp rx=0 tx=0).stop 2024-08-30T08:58:22.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.164+0000 7f100f69f640 1 -- 172.21.15.42:0/7197037 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1008102810 msgr2=0x7f100819c410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:58:22.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.164+0000 7f100f69f640 1 --2- 172.21.15.42:0/7197037 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1008102810 0x7f100819c410 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f0ff800db10 tx=0x7f0ff800b260 comp rx=0 tx=0).stop 2024-08-30T08:58:22.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.164+0000 7f100f69f640 1 -- 172.21.15.42:0/7197037 shutdown_connections 2024-08-30T08:58:22.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.164+0000 7f100f69f640 1 --2- 172.21.15.42:0/7197037 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f0ff00b4b30 0x7f0ff00b7040 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:22.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.164+0000 7f100f69f640 1 --2- 172.21.15.42:0/7197037 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1008108aa0 0x7f10081a1c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:22.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.164+0000 7f100f69f640 1 --2- 172.21.15.42:0/7197037 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1008108150 0x7f100819c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:22.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.164+0000 7f100f69f640 1 --2- 172.21.15.42:0/7197037 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1008102810 0x7f100819c410 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:22.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.164+0000 7f100f69f640 1 -- 172.21.15.42:0/7197037 >> 172.21.15.42:0/7197037 conn(0x7f10080794b0 msgr2=0x7f1008103f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:58:22.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.164+0000 7f100f69f640 1 -- 172.21.15.42:0/7197037 shutdown_connections 2024-08-30T08:58:22.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:22.164+0000 7f100f69f640 1 -- 172.21.15.42:0/7197037 wait complete. 2024-08-30T08:58:22.768 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:58:22 smithi042 ceph-mon[18652]: pgmap v297: 129 pgs: 129 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-30T08:58:22.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:58:22 smithi167 ceph-mon[22063]: pgmap v297: 129 pgs: 129 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-30T08:58:22.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:58:22 smithi190 ceph-mon[22001]: pgmap v297: 129 pgs: 129 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-30T08:58:22.953 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-08-30T08:58:22.953 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-08-30T08:58:22.953 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph fs ls 2024-08-30T08:58:23.203 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T08:58:23.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:23.769+0000 7f2749d60640 1 -- 172.21.15.42:0/674676413 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f27441036f0 msgr2=0x7f274410f2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:58:23.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:23.769+0000 7f2749d60640 1 --2- 172.21.15.42:0/674676413 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f27441036f0 0x7f274410f2a0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f273c009ca0 tx=0x7f273c02a8c0 comp rx=0 tx=0).stop 2024-08-30T08:58:23.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:23.769+0000 7f2749d60640 1 -- 172.21.15.42:0/674676413 shutdown_connections 2024-08-30T08:58:23.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:23.769+0000 7f2749d60640 1 --2- 172.21.15.42:0/674676413 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2744102da0 0x7f27441031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:23.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:23.769+0000 7f2749d60640 1 --2- 172.21.15.42:0/674676413 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f27441036f0 0x7f274410f2a0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:23.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:23.769+0000 7f2749d60640 1 --2- 172.21.15.42:0/674676413 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2744107430 0x7f2744107840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:23.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:23.769+0000 7f2749d60640 1 -- 172.21.15.42:0/674676413 >> 172.21.15.42:0/674676413 conn(0x7f2744079c40 msgr2=0x7f274407a050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:58:23.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:23.769+0000 7f2749d60640 1 -- 172.21.15.42:0/674676413 shutdown_connections 2024-08-30T08:58:23.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:23.769+0000 7f2749d60640 1 -- 172.21.15.42:0/674676413 wait complete. 2024-08-30T08:58:23.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:23.770+0000 7f2749d60640 1 Processor -- start 2024-08-30T08:58:23.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:23.770+0000 7f2749d60640 1 -- start start 2024-08-30T08:58:23.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:23.770+0000 7f2749d60640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2744102da0 0x7f2744199200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:58:23.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:23.771+0000 7f2749d60640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f27441036f0 0x7f27441997a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:58:23.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:23.771+0000 7f2749d60640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2744107430 0x7f274419ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:58:23.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:23.771+0000 7f2749d60640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f2744112850 con 0x7f27441036f0 2024-08-30T08:58:23.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:23.771+0000 7f2749d60640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f27441126d0 con 0x7f2744107430 2024-08-30T08:58:23.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:23.771+0000 7f2749d60640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f27441129d0 con 0x7f2744102da0 2024-08-30T08:58:23.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:23.771+0000 7f2742ffd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f27441036f0 0x7f27441997a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:58:23.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:23.771+0000 7f2742ffd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f27441036f0 0x7f27441997a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36550/0 (socket says 172.21.15.42:36550) 2024-08-30T08:58:23.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:23.771+0000 7f2742ffd640 1 -- 172.21.15.42:0/1337419516 learned_addr learned my addr 172.21.15.42:0/1337419516 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T08:58:23.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:23.771+0000 7f27437fe640 1 --2- 172.21.15.42:0/1337419516 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2744102da0 0x7f2744199200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:58:23.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:23.771+0000 7f2743fff640 1 --2- 172.21.15.42:0/1337419516 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2744107430 0x7f274419ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:58:23.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:23.771+0000 7f2742ffd640 1 -- 172.21.15.42:0/1337419516 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2744102da0 msgr2=0x7f2744199200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:58:23.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:23.771+0000 7f2742ffd640 1 --2- 172.21.15.42:0/1337419516 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2744102da0 0x7f2744199200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:23.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:23.771+0000 7f2742ffd640 1 -- 172.21.15.42:0/1337419516 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2744107430 msgr2=0x7f274419ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:58:23.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:23.771+0000 7f2742ffd640 1 --2- 172.21.15.42:0/1337419516 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2744107430 0x7f274419ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:23.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:23.771+0000 7f2742ffd640 1 -- 172.21.15.42:0/1337419516 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f274419f0b0 con 0x7f27441036f0 2024-08-30T08:58:23.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:23.771+0000 7f2743fff640 1 --2- 172.21.15.42:0/1337419516 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2744107430 0x7f274419ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-30T08:58:23.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:23.771+0000 7f2742ffd640 1 --2- 172.21.15.42:0/1337419516 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f27441036f0 0x7f27441997a0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f272c00c970 tx=0x7f272c00cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:58:23.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:23.771+0000 7f2740ff9640 1 -- 172.21.15.42:0/1337419516 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f272c007bb0 con 0x7f27441036f0 2024-08-30T08:58:23.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:23.771+0000 7f2749d60640 1 -- 172.21.15.42:0/1337419516 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f274419f3a0 con 0x7f27441036f0 2024-08-30T08:58:23.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:23.771+0000 7f2740ff9640 1 -- 172.21.15.42:0/1337419516 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f272c007d50 con 0x7f27441036f0 2024-08-30T08:58:23.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:23.771+0000 7f2740ff9640 1 -- 172.21.15.42:0/1337419516 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f272c018660 con 0x7f27441036f0 2024-08-30T08:58:23.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:23.771+0000 7f2749d60640 1 -- 172.21.15.42:0/1337419516 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27441a5bc0 con 0x7f27441036f0 2024-08-30T08:58:23.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:23.773+0000 7f2749d60640 1 -- 172.21.15.42:0/1337419516 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27441031b0 con 0x7f27441036f0 2024-08-30T08:58:23.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:23.774+0000 7f2740ff9640 1 -- 172.21.15.42:0/1337419516 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) ==== 152577+0+0 (secure 0 0 0) 0x7f272c00f990 con 0x7f27441036f0 2024-08-30T08:58:23.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:23.775+0000 7f2740ff9640 1 --2- 172.21.15.42:0/1337419516 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f27300b4b60 0x7f27300b7070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:58:23.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:23.775+0000 7f27437fe640 1 --2- 172.21.15.42:0/1337419516 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f27300b4b60 0x7f27300b7070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:58:23.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:23.775+0000 7f2740ff9640 1 -- 172.21.15.42:0/1337419516 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8802+0+0 (secure 0 0 0) 0x7f272c0e5570 con 0x7f27441036f0 2024-08-30T08:58:23.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:23.776+0000 7f27437fe640 1 --2- 172.21.15.42:0/1337419516 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f27300b4b60 0x7f27300b7070 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f2734009990 tx=0x7f2734009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:58:23.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:23.778+0000 7f2740ff9640 1 -- 172.21.15.42:0/1337419516 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f272c056130 con 0x7f27441036f0 2024-08-30T08:58:24.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:24.001+0000 7f2749d60640 1 -- 172.21.15.42:0/1337419516 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f274419a330 con 0x7f27441036f0 2024-08-30T08:58:24.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:24.002+0000 7f2740ff9640 1 -- 172.21.15.42:0/1337419516 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) ==== 53+0+88 (secure 0 0 0) 0x7f272c014030 con 0x7f27441036f0 2024-08-30T08:58:24.004 INFO:teuthology.orchestra.run.smithi042.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-08-30T08:58:24.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:24.004+0000 7f2749d60640 1 -- 172.21.15.42:0/1337419516 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f27300b4b60 msgr2=0x7f27300b7070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:58:24.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:24.004+0000 7f2749d60640 1 --2- 172.21.15.42:0/1337419516 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f27300b4b60 0x7f27300b7070 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f2734009990 tx=0x7f2734009110 comp rx=0 tx=0).stop 2024-08-30T08:58:24.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:24.004+0000 7f2749d60640 1 -- 172.21.15.42:0/1337419516 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f27441036f0 msgr2=0x7f27441997a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:58:24.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:24.004+0000 7f2749d60640 1 --2- 172.21.15.42:0/1337419516 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f27441036f0 0x7f27441997a0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f272c00c970 tx=0x7f272c00cd70 comp rx=0 tx=0).stop 2024-08-30T08:58:24.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:24.004+0000 7f2749d60640 1 -- 172.21.15.42:0/1337419516 shutdown_connections 2024-08-30T08:58:24.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:24.004+0000 7f2749d60640 1 --2- 172.21.15.42:0/1337419516 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f27300b4b60 0x7f27300b7070 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:24.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:24.004+0000 7f2749d60640 1 --2- 172.21.15.42:0/1337419516 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2744107430 0x7f274419ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:24.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:24.004+0000 7f2749d60640 1 --2- 172.21.15.42:0/1337419516 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f27441036f0 0x7f27441997a0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:24.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:24.004+0000 7f2749d60640 1 --2- 172.21.15.42:0/1337419516 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2744102da0 0x7f2744199200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:24.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:24.004+0000 7f2749d60640 1 -- 172.21.15.42:0/1337419516 >> 172.21.15.42:0/1337419516 conn(0x7f2744079c40 msgr2=0x7f2744103fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:58:24.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:24.005+0000 7f2749d60640 1 -- 172.21.15.42:0/1337419516 shutdown_connections 2024-08-30T08:58:24.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:24.005+0000 7f2749d60640 1 -- 172.21.15.42:0/1337419516 wait complete. 2024-08-30T08:58:24.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:58:24 smithi042 ceph-mon[18652]: pgmap v298: 129 pgs: 129 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-30T08:58:24.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:58:24 smithi042 ceph-mon[18652]: from='client.? 172.21.15.42:0/1337419516' entity='client.admin' cmd={"prefix": "fs ls"} : dispatch 2024-08-30T08:58:24.707 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-08-30T08:58:24.707 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-08-30T08:58:24.707 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi042.front.sepia.ceph.com 2024-08-30T08:58:24.707 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-08-30T08:58:24.707 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-08-30T08:58:24.707 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-08-30T08:58:24.707 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-08-30T08:58:24.707 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-08-30T08:58:24.707 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-08-30T08:58:24.708 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-30T08:58:24.708 DEBUG:teuthology.orchestra.run.smithi042:> ip addr 2024-08-30T08:58:24.724 INFO:teuthology.orchestra.run.smithi042.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-08-30T08:58:24.724 INFO:teuthology.orchestra.run.smithi042.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-08-30T08:58:24.724 INFO:teuthology.orchestra.run.smithi042.stdout: inet 127.0.0.1/8 scope host lo 2024-08-30T08:58:24.724 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-08-30T08:58:24.724 INFO:teuthology.orchestra.run.smithi042.stdout: inet6 ::1/128 scope host 2024-08-30T08:58:24.724 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-08-30T08:58:24.725 INFO:teuthology.orchestra.run.smithi042.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-08-30T08:58:24.725 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether 0c:c4:7a:6b:fd:0e brd ff:ff:ff:ff:ff:ff 2024-08-30T08:58:24.725 INFO:teuthology.orchestra.run.smithi042.stdout: altname enp7s0f0 2024-08-30T08:58:24.725 INFO:teuthology.orchestra.run.smithi042.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-08-30T08:58:24.725 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether 0c:c4:7a:6b:fd:0f brd ff:ff:ff:ff:ff:ff 2024-08-30T08:58:24.725 INFO:teuthology.orchestra.run.smithi042.stdout: altname enp7s0f1 2024-08-30T08:58:24.725 INFO:teuthology.orchestra.run.smithi042.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-08-30T08:58:24.725 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether 0c:c4:7a:bd:14:40 brd ff:ff:ff:ff:ff:ff 2024-08-30T08:58:24.725 INFO:teuthology.orchestra.run.smithi042.stdout: altname enp1s0f0 2024-08-30T08:58:24.725 INFO:teuthology.orchestra.run.smithi042.stdout: inet 172.21.15.42/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-08-30T08:58:24.725 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft 42295sec preferred_lft 42295sec 2024-08-30T08:58:24.725 INFO:teuthology.orchestra.run.smithi042.stdout: inet6 fe80::666b:514f:12c9:e93f/64 scope link noprefixroute 2024-08-30T08:58:24.725 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-08-30T08:58:24.725 INFO:teuthology.orchestra.run.smithi042.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-08-30T08:58:24.726 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether 0c:c4:7a:bd:14:41 brd ff:ff:ff:ff:ff:ff 2024-08-30T08:58:24.726 INFO:teuthology.orchestra.run.smithi042.stdout: altname enp1s0f1 2024-08-30T08:58:24.726 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-08-30T08:58:24.726 DEBUG:teuthology.orchestra.run.smithi042:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-08-30T08:58:24.726 DEBUG:teuthology.orchestra.run.smithi042:> set -e 2024-08-30T08:58:24.727 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip link add name ceph-brx type bridge 2024-08-30T08:58:24.727 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip addr flush dev ceph-brx 2024-08-30T08:58:24.727 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip link set ceph-brx up 2024-08-30T08:58:24.727 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-08-30T08:58:24.727 DEBUG:teuthology.orchestra.run.smithi042:> ') 2024-08-30T08:58:24.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:24 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-08-30T08:58:24.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:58:24 smithi167 ceph-mon[22063]: pgmap v298: 129 pgs: 129 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-30T08:58:24.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:58:24 smithi167 ceph-mon[22063]: from='client.? 172.21.15.42:0/1337419516' entity='client.admin' cmd={"prefix": "fs ls"} : dispatch 2024-08-30T08:58:24.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-08-30T08:58:24.922 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-30T08:58:24.922 DEBUG:teuthology.orchestra.run.smithi042:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-08-30T08:58:24.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:58:24 smithi190 ceph-mon[22001]: pgmap v298: 129 pgs: 129 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-30T08:58:24.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:58:24 smithi190 ceph-mon[22001]: from='client.? 172.21.15.42:0/1337419516' entity='client.admin' cmd={"prefix": "fs ls"} : dispatch 2024-08-30T08:58:24.947 INFO:teuthology.orchestra.run.smithi042.stdout:1 2024-08-30T08:58:24.949 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-30T08:58:24.949 DEBUG:teuthology.orchestra.run.smithi042:> route 2024-08-30T08:58:25.023 INFO:teuthology.orchestra.run.smithi042.stdout:Kernel IP routing table 2024-08-30T08:58:25.023 INFO:teuthology.orchestra.run.smithi042.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-08-30T08:58:25.023 INFO:teuthology.orchestra.run.smithi042.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-08-30T08:58:25.023 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-08-30T08:58:25.023 INFO:teuthology.orchestra.run.smithi042.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-08-30T08:58:25.023 DEBUG:teuthology.orchestra.run.smithi042:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-08-30T08:58:25.023 DEBUG:teuthology.orchestra.run.smithi042:> set -e 2024-08-30T08:58:25.023 DEBUG:teuthology.orchestra.run.smithi042:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-08-30T08:58:25.024 DEBUG:teuthology.orchestra.run.smithi042:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-08-30T08:58:25.024 DEBUG:teuthology.orchestra.run.smithi042:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-08-30T08:58:25.024 DEBUG:teuthology.orchestra.run.smithi042:> ') 2024-08-30T08:58:25.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:25 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-08-30T08:58:25.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-08-30T08:58:25.293 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-30T08:58:25.293 DEBUG:teuthology.orchestra.run.smithi042:> ip netns list 2024-08-30T08:58:25.309 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-30T08:58:25.309 DEBUG:teuthology.orchestra.run.smithi042:> ip netns list-id 2024-08-30T08:58:25.364 DEBUG:teuthology.orchestra.run.smithi042:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-08-30T08:58:25.364 DEBUG:teuthology.orchestra.run.smithi042:> set -e 2024-08-30T08:58:25.365 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-08-30T08:58:25.365 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-08-30T08:58:25.365 DEBUG:teuthology.orchestra.run.smithi042:> ') 2024-08-30T08:58:25.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:25 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-08-30T08:58:25.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-08-30T08:58:25.484 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-08-30T08:58:25.484 DEBUG:teuthology.orchestra.run.smithi042:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-08-30T08:58:25.484 DEBUG:teuthology.orchestra.run.smithi042:> set -e 2024-08-30T08:58:25.484 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-08-30T08:58:25.484 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-08-30T08:58:25.484 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-08-30T08:58:25.485 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-08-30T08:58:25.485 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-08-30T08:58:25.485 DEBUG:teuthology.orchestra.run.smithi042:> ') 2024-08-30T08:58:25.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:25 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-08-30T08:58:25.605 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-08-30T08:58:25.611 DEBUG:teuthology.orchestra.run.smithi042:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-08-30T08:58:25.611 DEBUG:teuthology.orchestra.run.smithi042:> set -e 2024-08-30T08:58:25.611 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip link set brx.0 up 2024-08-30T08:58:25.611 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip link set dev brx.0 master ceph-brx 2024-08-30T08:58:25.611 DEBUG:teuthology.orchestra.run.smithi042:> ') 2024-08-30T08:58:25.659 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:25 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-08-30T08:58:25.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-08-30T08:58:25.688 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-08-30T08:58:25.688 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-08-30T08:58:25.688 DEBUG:teuthology.orchestra.run.smithi042:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-08-30T08:58:25.704 INFO:teuthology.orchestra.run.smithi042.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-08-30T08:58:25.704 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-08-30T08:58:25.704 DEBUG:teuthology.orchestra.run.smithi042:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-08-30T08:58:25.758 DEBUG:teuthology.orchestra.run.smithi042:> sudo modprobe fuse 2024-08-30T08:58:25.824 DEBUG:teuthology.orchestra.run.smithi042:> cat /proc/self/mounts | awk '{print $2}' 2024-08-30T08:58:25.879 INFO:teuthology.orchestra.run.smithi042.stdout:/proc 2024-08-30T08:58:25.879 INFO:teuthology.orchestra.run.smithi042.stdout:/sys 2024-08-30T08:58:25.879 INFO:teuthology.orchestra.run.smithi042.stdout:/dev 2024-08-30T08:58:25.880 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/security 2024-08-30T08:58:25.880 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/shm 2024-08-30T08:58:25.880 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/pts 2024-08-30T08:58:25.880 INFO:teuthology.orchestra.run.smithi042.stdout:/run 2024-08-30T08:58:25.880 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/cgroup 2024-08-30T08:58:25.880 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/pstore 2024-08-30T08:58:25.880 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/bpf 2024-08-30T08:58:25.880 INFO:teuthology.orchestra.run.smithi042.stdout:/ 2024-08-30T08:58:25.880 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/selinux 2024-08-30T08:58:25.880 INFO:teuthology.orchestra.run.smithi042.stdout:/proc/sys/fs/binfmt_misc 2024-08-30T08:58:25.880 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/hugepages 2024-08-30T08:58:25.880 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/mqueue 2024-08-30T08:58:25.880 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/debug 2024-08-30T08:58:25.881 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/tracing 2024-08-30T08:58:25.881 INFO:teuthology.orchestra.run.smithi042.stdout:/run/credentials/systemd-sysusers.service 2024-08-30T08:58:25.881 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/config 2024-08-30T08:58:25.881 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/fuse/connections 2024-08-30T08:58:25.881 INFO:teuthology.orchestra.run.smithi042.stdout:/run/credentials/systemd-sysctl.service 2024-08-30T08:58:25.881 INFO:teuthology.orchestra.run.smithi042.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-08-30T08:58:25.881 INFO:teuthology.orchestra.run.smithi042.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-08-30T08:58:25.881 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/nfs/rpc_pipefs 2024-08-30T08:58:25.881 INFO:teuthology.orchestra.run.smithi042.stdout:/run/user/1000 2024-08-30T08:58:25.881 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph 2024-08-30T08:58:25.881 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/containers/storage/overlay 2024-08-30T08:58:25.881 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/containers/storage/overlay/c6279c773e026ef694553d70ec33f8afb14820a6066f64c45d9c39456b4638d3/merged 2024-08-30T08:58:25.881 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/containers/storage/overlay/a545b1d52a8a4e1285f2a454de6d5a01be8e551166f542bf0e42f63b6cd8a1f3/merged 2024-08-30T08:58:25.881 INFO:teuthology.orchestra.run.smithi042.stdout:/run/user/0 2024-08-30T08:58:25.881 INFO:teuthology.orchestra.run.smithi042.stdout:/proc/sys/fs/binfmt_misc 2024-08-30T08:58:25.882 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/containers/storage/overlay/abac7c8709f5c6f621a0878b36318c9d3e418c80bd35b345c13cf7cd6b532b63/merged 2024-08-30T08:58:25.882 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/containers/storage/overlay/11046875c4b1ed90e01bbe06efa420c6acd1412703f22d935b75dd44fa368e19/merged 2024-08-30T08:58:25.882 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/containers/storage/overlay/29f92005d7722229c25f3149df32caf2353e595e368e2eca936c20720b3d851a/merged 2024-08-30T08:58:25.882 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/containers/storage/overlay/4c4f3461b3186a60e4c5a9fb3491c63c9754ba8969d9bc02f3a26d4a02d90923/merged 2024-08-30T08:58:25.882 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/containers/storage/overlay/c37b258053fca2bdb45d7a3e7b901912e1721fab159be6b7b8699e1e1fe8ffc0/merged 2024-08-30T08:58:25.882 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/containers/storage/overlay/f0339e5857f16fc143cd14af325e3f494a782092e539ac209aa7c2ed21d1c54b/merged 2024-08-30T08:58:25.882 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/containers/storage/overlay/bcc217cb69c2705702bb806ab7651e748bf424c775ffd75c93df3a3cc9d19e7e/merged 2024-08-30T08:58:25.882 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/containers/storage/overlay/0f0194329d9a51b9310cfed8ca8b4c07a7ef5baf70f274ac7e8e02dfd59a4a01/merged 2024-08-30T08:58:25.882 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns 2024-08-30T08:58:25.882 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-08-30T08:58:25.882 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-08-30T08:58:25.882 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-30T08:58:25.882 DEBUG:teuthology.orchestra.run.smithi042:> ls /sys/fs/fuse/connections 2024-08-30T08:58:25.934 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-08-30T08:58:25.934 DEBUG:teuthology.orchestra.run.smithi042:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-08-30T08:58:25.977 DEBUG:teuthology.orchestra.run.smithi042:> sudo modprobe fuse 2024-08-30T08:58:26.001 DEBUG:teuthology.orchestra.run.smithi042:> cat /proc/self/mounts | awk '{print $2}' 2024-08-30T08:58:26.057 INFO:teuthology.orchestra.run.smithi042.stdout:/proc 2024-08-30T08:58:26.058 INFO:teuthology.orchestra.run.smithi042.stdout:/sys 2024-08-30T08:58:26.058 INFO:teuthology.orchestra.run.smithi042.stdout:/dev 2024-08-30T08:58:26.058 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/security 2024-08-30T08:58:26.058 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/shm 2024-08-30T08:58:26.058 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/pts 2024-08-30T08:58:26.058 INFO:teuthology.orchestra.run.smithi042.stdout:/run 2024-08-30T08:58:26.058 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/cgroup 2024-08-30T08:58:26.058 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/pstore 2024-08-30T08:58:26.058 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/bpf 2024-08-30T08:58:26.058 INFO:teuthology.orchestra.run.smithi042.stdout:/ 2024-08-30T08:58:26.058 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/selinux 2024-08-30T08:58:26.058 INFO:teuthology.orchestra.run.smithi042.stdout:/proc/sys/fs/binfmt_misc 2024-08-30T08:58:26.059 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/hugepages 2024-08-30T08:58:26.059 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/mqueue 2024-08-30T08:58:26.059 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/debug 2024-08-30T08:58:26.059 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/tracing 2024-08-30T08:58:26.059 INFO:teuthology.orchestra.run.smithi042.stdout:/run/credentials/systemd-sysusers.service 2024-08-30T08:58:26.059 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/config 2024-08-30T08:58:26.059 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/fuse/connections 2024-08-30T08:58:26.059 INFO:teuthology.orchestra.run.smithi042.stdout:/run/credentials/systemd-sysctl.service 2024-08-30T08:58:26.059 INFO:teuthology.orchestra.run.smithi042.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-08-30T08:58:26.059 INFO:teuthology.orchestra.run.smithi042.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-08-30T08:58:26.059 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/nfs/rpc_pipefs 2024-08-30T08:58:26.059 INFO:teuthology.orchestra.run.smithi042.stdout:/run/user/1000 2024-08-30T08:58:26.059 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph 2024-08-30T08:58:26.059 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/containers/storage/overlay 2024-08-30T08:58:26.059 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/containers/storage/overlay/c6279c773e026ef694553d70ec33f8afb14820a6066f64c45d9c39456b4638d3/merged 2024-08-30T08:58:26.059 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/containers/storage/overlay/a545b1d52a8a4e1285f2a454de6d5a01be8e551166f542bf0e42f63b6cd8a1f3/merged 2024-08-30T08:58:26.060 INFO:teuthology.orchestra.run.smithi042.stdout:/run/user/0 2024-08-30T08:58:26.060 INFO:teuthology.orchestra.run.smithi042.stdout:/proc/sys/fs/binfmt_misc 2024-08-30T08:58:26.060 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/containers/storage/overlay/abac7c8709f5c6f621a0878b36318c9d3e418c80bd35b345c13cf7cd6b532b63/merged 2024-08-30T08:58:26.060 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/containers/storage/overlay/11046875c4b1ed90e01bbe06efa420c6acd1412703f22d935b75dd44fa368e19/merged 2024-08-30T08:58:26.060 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/containers/storage/overlay/29f92005d7722229c25f3149df32caf2353e595e368e2eca936c20720b3d851a/merged 2024-08-30T08:58:26.060 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/containers/storage/overlay/4c4f3461b3186a60e4c5a9fb3491c63c9754ba8969d9bc02f3a26d4a02d90923/merged 2024-08-30T08:58:26.060 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/containers/storage/overlay/c37b258053fca2bdb45d7a3e7b901912e1721fab159be6b7b8699e1e1fe8ffc0/merged 2024-08-30T08:58:26.060 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/containers/storage/overlay/f0339e5857f16fc143cd14af325e3f494a782092e539ac209aa7c2ed21d1c54b/merged 2024-08-30T08:58:26.060 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/containers/storage/overlay/bcc217cb69c2705702bb806ab7651e748bf424c775ffd75c93df3a3cc9d19e7e/merged 2024-08-30T08:58:26.060 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/containers/storage/overlay/0f0194329d9a51b9310cfed8ca8b4c07a7ef5baf70f274ac7e8e02dfd59a4a01/merged 2024-08-30T08:58:26.060 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns 2024-08-30T08:58:26.060 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-08-30T08:58:26.060 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-08-30T08:58:26.061 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-30T08:58:26.061 DEBUG:teuthology.orchestra.run.smithi042:> ls /sys/fs/fuse/connections 2024-08-30T08:58:26.236 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi042.stderr:2024-08-30T08:58:26.234+0000 7f13b7cf2480 -1 init, newargv = 0x7f139c054370 newargc=15 2024-08-30T08:58:26.236 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi042.stderr:ceph-fuse[65508]: starting ceph client 2024-08-30T08:58:26.243 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi042.stderr:ceph-fuse[65508]: starting fuse 2024-08-30T08:58:26.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:58:26 smithi167 ceph-mon[22063]: pgmap v299: 129 pgs: 129 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-30T08:58:26.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:58:26 smithi042 ceph-mon[18652]: pgmap v299: 129 pgs: 129 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-30T08:58:26.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:58:26 smithi190 ceph-mon[22001]: pgmap v299: 129 pgs: 129 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-30T08:58:27.214 DEBUG:teuthology.orchestra.run.smithi042:> sudo modprobe fuse 2024-08-30T08:58:27.246 DEBUG:teuthology.orchestra.run.smithi042:> cat /proc/self/mounts | awk '{print $2}' 2024-08-30T08:58:27.306 INFO:teuthology.orchestra.run.smithi042.stdout:/proc 2024-08-30T08:58:27.306 INFO:teuthology.orchestra.run.smithi042.stdout:/sys 2024-08-30T08:58:27.306 INFO:teuthology.orchestra.run.smithi042.stdout:/dev 2024-08-30T08:58:27.306 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/security 2024-08-30T08:58:27.306 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/shm 2024-08-30T08:58:27.306 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/pts 2024-08-30T08:58:27.306 INFO:teuthology.orchestra.run.smithi042.stdout:/run 2024-08-30T08:58:27.306 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/cgroup 2024-08-30T08:58:27.306 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/pstore 2024-08-30T08:58:27.306 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/bpf 2024-08-30T08:58:27.306 INFO:teuthology.orchestra.run.smithi042.stdout:/ 2024-08-30T08:58:27.306 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/selinux 2024-08-30T08:58:27.306 INFO:teuthology.orchestra.run.smithi042.stdout:/proc/sys/fs/binfmt_misc 2024-08-30T08:58:27.307 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/hugepages 2024-08-30T08:58:27.307 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/mqueue 2024-08-30T08:58:27.307 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/debug 2024-08-30T08:58:27.307 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/tracing 2024-08-30T08:58:27.307 INFO:teuthology.orchestra.run.smithi042.stdout:/run/credentials/systemd-sysusers.service 2024-08-30T08:58:27.307 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/config 2024-08-30T08:58:27.307 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/fuse/connections 2024-08-30T08:58:27.307 INFO:teuthology.orchestra.run.smithi042.stdout:/run/credentials/systemd-sysctl.service 2024-08-30T08:58:27.307 INFO:teuthology.orchestra.run.smithi042.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-08-30T08:58:27.307 INFO:teuthology.orchestra.run.smithi042.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-08-30T08:58:27.307 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/nfs/rpc_pipefs 2024-08-30T08:58:27.307 INFO:teuthology.orchestra.run.smithi042.stdout:/run/user/1000 2024-08-30T08:58:27.307 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph 2024-08-30T08:58:27.307 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/containers/storage/overlay 2024-08-30T08:58:27.307 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/containers/storage/overlay/c6279c773e026ef694553d70ec33f8afb14820a6066f64c45d9c39456b4638d3/merged 2024-08-30T08:58:27.308 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/containers/storage/overlay/a545b1d52a8a4e1285f2a454de6d5a01be8e551166f542bf0e42f63b6cd8a1f3/merged 2024-08-30T08:58:27.308 INFO:teuthology.orchestra.run.smithi042.stdout:/run/user/0 2024-08-30T08:58:27.308 INFO:teuthology.orchestra.run.smithi042.stdout:/proc/sys/fs/binfmt_misc 2024-08-30T08:58:27.308 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/containers/storage/overlay/abac7c8709f5c6f621a0878b36318c9d3e418c80bd35b345c13cf7cd6b532b63/merged 2024-08-30T08:58:27.308 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/containers/storage/overlay/11046875c4b1ed90e01bbe06efa420c6acd1412703f22d935b75dd44fa368e19/merged 2024-08-30T08:58:27.308 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/containers/storage/overlay/29f92005d7722229c25f3149df32caf2353e595e368e2eca936c20720b3d851a/merged 2024-08-30T08:58:27.308 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/containers/storage/overlay/4c4f3461b3186a60e4c5a9fb3491c63c9754ba8969d9bc02f3a26d4a02d90923/merged 2024-08-30T08:58:27.308 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/containers/storage/overlay/c37b258053fca2bdb45d7a3e7b901912e1721fab159be6b7b8699e1e1fe8ffc0/merged 2024-08-30T08:58:27.308 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/containers/storage/overlay/f0339e5857f16fc143cd14af325e3f494a782092e539ac209aa7c2ed21d1c54b/merged 2024-08-30T08:58:27.308 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/containers/storage/overlay/bcc217cb69c2705702bb806ab7651e748bf424c775ffd75c93df3a3cc9d19e7e/merged 2024-08-30T08:58:27.309 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/containers/storage/overlay/0f0194329d9a51b9310cfed8ca8b4c07a7ef5baf70f274ac7e8e02dfd59a4a01/merged 2024-08-30T08:58:27.309 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns 2024-08-30T08:58:27.309 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-08-30T08:58:27.309 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-08-30T08:58:27.309 INFO:teuthology.orchestra.run.smithi042.stdout:/home/ubuntu/cephtest/mnt.admin 2024-08-30T08:58:27.309 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-30T08:58:27.309 DEBUG:teuthology.orchestra.run.smithi042:> ls /sys/fs/fuse/connections 2024-08-30T08:58:27.363 INFO:teuthology.orchestra.run.smithi042.stdout:121 2024-08-30T08:58:27.363 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [121] 2024-08-30T08:58:27.364 DEBUG:teuthology.orchestra.run.smithi042:> sudo stdin-killer -- python3 -c ' 2024-08-30T08:58:27.364 DEBUG:teuthology.orchestra.run.smithi042:> import glob 2024-08-30T08:58:27.364 DEBUG:teuthology.orchestra.run.smithi042:> import re 2024-08-30T08:58:27.364 DEBUG:teuthology.orchestra.run.smithi042:> import os 2024-08-30T08:58:27.364 DEBUG:teuthology.orchestra.run.smithi042:> import subprocess 2024-08-30T08:58:27.364 DEBUG:teuthology.orchestra.run.smithi042:> 2024-08-30T08:58:27.364 DEBUG:teuthology.orchestra.run.smithi042:> def _find_admin_socket(client_name): 2024-08-30T08:58:27.364 DEBUG:teuthology.orchestra.run.smithi042:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-08-30T08:58:27.364 DEBUG:teuthology.orchestra.run.smithi042:> files = glob.glob(asok_path) 2024-08-30T08:58:27.364 DEBUG:teuthology.orchestra.run.smithi042:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-08-30T08:58:27.364 DEBUG:teuthology.orchestra.run.smithi042:> 2024-08-30T08:58:27.364 DEBUG:teuthology.orchestra.run.smithi042:> # Given a non-glob path, it better be there 2024-08-30T08:58:27.364 DEBUG:teuthology.orchestra.run.smithi042:> if "*" not in asok_path: 2024-08-30T08:58:27.364 DEBUG:teuthology.orchestra.run.smithi042:> assert(len(files) == 1) 2024-08-30T08:58:27.365 DEBUG:teuthology.orchestra.run.smithi042:> return files[0] 2024-08-30T08:58:27.365 DEBUG:teuthology.orchestra.run.smithi042:> 2024-08-30T08:58:27.365 DEBUG:teuthology.orchestra.run.smithi042:> for f in files: 2024-08-30T08:58:27.365 DEBUG:teuthology.orchestra.run.smithi042:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-08-30T08:58:27.365 DEBUG:teuthology.orchestra.run.smithi042:> if os.path.exists("/proc/{0}".format(pid)): 2024-08-30T08:58:27.365 DEBUG:teuthology.orchestra.run.smithi042:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-08-30T08:58:27.365 DEBUG:teuthology.orchestra.run.smithi042:> contents = proc_f.read() 2024-08-30T08:58:27.365 DEBUG:teuthology.orchestra.run.smithi042:> if mountpoint in contents: 2024-08-30T08:58:27.365 DEBUG:teuthology.orchestra.run.smithi042:> return f 2024-08-30T08:58:27.365 DEBUG:teuthology.orchestra.run.smithi042:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-08-30T08:58:27.365 DEBUG:teuthology.orchestra.run.smithi042:> 2024-08-30T08:58:27.365 DEBUG:teuthology.orchestra.run.smithi042:> print(_find_admin_socket("client.admin")) 2024-08-30T08:58:27.365 DEBUG:teuthology.orchestra.run.smithi042:> ' 2024-08-30T08:58:27.487 INFO:teuthology.orchestra.run.smithi042.stdout:/var/run/ceph/ceph-client.admin.65508.asok 2024-08-30T08:58:27.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-08-30T08:58:27.499 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.65508.asok 2024-08-30T08:58:27.499 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-30T08:58:27.499 DEBUG:teuthology.orchestra.run.smithi042:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.65508.asok status 2024-08-30T08:58:27.646 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-08-30T08:58:27.646 INFO:teuthology.orchestra.run.smithi042.stdout: "metadata": { 2024-08-30T08:58:27.646 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph_sha1": "d19e6118f7015f0047264006cb0c873528eb3ae6", 2024-08-30T08:58:27.646 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph_version": "ceph version 19.3.0-4560-gd19e6118 (d19e6118f7015f0047264006cb0c873528eb3ae6) squid (dev)", 2024-08-30T08:58:27.646 INFO:teuthology.orchestra.run.smithi042.stdout: "entity_id": "admin", 2024-08-30T08:58:27.646 INFO:teuthology.orchestra.run.smithi042.stdout: "hostname": "smithi042", 2024-08-30T08:58:27.646 INFO:teuthology.orchestra.run.smithi042.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-08-30T08:58:27.646 INFO:teuthology.orchestra.run.smithi042.stdout: "pid": "65508", 2024-08-30T08:58:27.646 INFO:teuthology.orchestra.run.smithi042.stdout: "root": "/" 2024-08-30T08:58:27.646 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T08:58:27.646 INFO:teuthology.orchestra.run.smithi042.stdout: "dentry_count": 0, 2024-08-30T08:58:27.646 INFO:teuthology.orchestra.run.smithi042.stdout: "dentry_pinned_count": 0, 2024-08-30T08:58:27.647 INFO:teuthology.orchestra.run.smithi042.stdout: "id": 14901, 2024-08-30T08:58:27.647 INFO:teuthology.orchestra.run.smithi042.stdout: "inst": { 2024-08-30T08:58:27.647 INFO:teuthology.orchestra.run.smithi042.stdout: "name": { 2024-08-30T08:58:27.647 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "client", 2024-08-30T08:58:27.647 INFO:teuthology.orchestra.run.smithi042.stdout: "num": 14901 2024-08-30T08:58:27.647 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T08:58:27.647 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": { 2024-08-30T08:58:27.647 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-08-30T08:58:27.647 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "192.168.0.1:0", 2024-08-30T08:58:27.647 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 4055183681 2024-08-30T08:58:27.647 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T08:58:27.647 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T08:58:27.647 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": { 2024-08-30T08:58:27.647 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-08-30T08:58:27.648 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "192.168.0.1:0", 2024-08-30T08:58:27.648 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 4055183681 2024-08-30T08:58:27.648 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T08:58:27.648 INFO:teuthology.orchestra.run.smithi042.stdout: "inst_str": "client.14901 192.168.0.1:0/4055183681", 2024-08-30T08:58:27.648 INFO:teuthology.orchestra.run.smithi042.stdout: "addr_str": "192.168.0.1:0/4055183681", 2024-08-30T08:58:27.648 INFO:teuthology.orchestra.run.smithi042.stdout: "inode_count": 1, 2024-08-30T08:58:27.648 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_epoch": 17, 2024-08-30T08:58:27.648 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_epoch": 79, 2024-08-30T08:58:27.648 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_epoch_barrier": 0, 2024-08-30T08:58:27.648 INFO:teuthology.orchestra.run.smithi042.stdout: "blocklisted": false, 2024-08-30T08:58:27.648 INFO:teuthology.orchestra.run.smithi042.stdout: "fs_name": "cephfs" 2024-08-30T08:58:27.648 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-08-30T08:58:27.654 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-30T08:58:27.654 DEBUG:teuthology.orchestra.run.smithi042:> stat --file-system '--printf=%T 2024-08-30T08:58:27.655 DEBUG:teuthology.orchestra.run.smithi042:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-08-30T08:58:27.670 INFO:teuthology.orchestra.run.smithi042.stdout:fuseblk 2024-08-30T08:58:27.671 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-08-30T08:58:27.671 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-30T08:58:27.671 DEBUG:teuthology.orchestra.run.smithi042:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-08-30T08:58:27.736 DEBUG:teuthology.orchestra.run.smithi042:> (cd /home/ubuntu/cephtest/mnt.admin && exec stdin-killer --timeout=900 -- bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-08-30T08:58:27.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:27 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-08-30T08:58:27.853 INFO:teuthology.orchestra.run.smithi042.stdout:# file: . 2024-08-30T08:58:27.853 INFO:teuthology.orchestra.run.smithi042.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-08-30T08:58:27.853 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-08-30T08:58:27.853 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-08-30T08:58:27.859 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-08-30T08:58:27.859 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/proc/self/mounts of=/dev/stdout 2024-08-30T08:58:27.876 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph osd blocklist ls 2024-08-30T08:58:28.168 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T08:58:28.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:58:28 smithi042 ceph-mon[18652]: pgmap v300: 129 pgs: 129 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-30T08:58:28.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:28.752+0000 7f2dbf3d8640 1 -- 172.21.15.42:0/1036414018 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2db810ff50 msgr2=0x7f2db8112370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:58:28.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:28.752+0000 7f2dbf3d8640 1 --2- 172.21.15.42:0/1036414018 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2db810ff50 0x7f2db8112370 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f2db0009ca0 tx=0x7f2db002a950 comp rx=0 tx=0).stop 2024-08-30T08:58:28.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:28.752+0000 7f2dbf3d8640 1 -- 172.21.15.42:0/1036414018 shutdown_connections 2024-08-30T08:58:28.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:28.752+0000 7f2dbf3d8640 1 --2- 172.21.15.42:0/1036414018 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2db810ff50 0x7f2db8112370 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:28.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:28.752+0000 7f2dbf3d8640 1 --2- 172.21.15.42:0/1036414018 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2db8102cf0 0x7f2db810fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:28.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:28.752+0000 7f2dbf3d8640 1 --2- 172.21.15.42:0/1036414018 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2db81023a0 0x7f2db81027b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:28.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:28.752+0000 7f2dbf3d8640 1 -- 172.21.15.42:0/1036414018 >> 172.21.15.42:0/1036414018 conn(0x7f2db80794f0 msgr2=0x7f2db8079900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:58:28.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:28.752+0000 7f2dbf3d8640 1 -- 172.21.15.42:0/1036414018 shutdown_connections 2024-08-30T08:58:28.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:28.752+0000 7f2dbf3d8640 1 -- 172.21.15.42:0/1036414018 wait complete. 2024-08-30T08:58:28.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:28.753+0000 7f2dbf3d8640 1 Processor -- start 2024-08-30T08:58:28.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:28.753+0000 7f2dbf3d8640 1 -- start start 2024-08-30T08:58:28.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:28.753+0000 7f2dbf3d8640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2db81023a0 0x7f2db819c410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:58:28.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:28.753+0000 7f2dbf3d8640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2db8102cf0 0x7f2db819c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:58:28.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:28.753+0000 7f2dbf3d8640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2db810ff50 0x7f2db81a1c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:58:28.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:28.753+0000 7f2dbf3d8640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f2db8115a40 con 0x7f2db8102cf0 2024-08-30T08:58:28.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:28.753+0000 7f2dbf3d8640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f2db81158c0 con 0x7f2db810ff50 2024-08-30T08:58:28.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:28.753+0000 7f2dbf3d8640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f2db8115bc0 con 0x7f2db81023a0 2024-08-30T08:58:28.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:28.753+0000 7f2dbc973640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2db8102cf0 0x7f2db819c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:58:28.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:28.753+0000 7f2dbc973640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2db8102cf0 0x7f2db819c9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52932/0 (socket says 172.21.15.42:52932) 2024-08-30T08:58:28.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:28.753+0000 7f2dbc973640 1 -- 172.21.15.42:0/3145434416 learned_addr learned my addr 172.21.15.42:0/3145434416 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T08:58:28.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:28.753+0000 7f2dbd174640 1 --2- 172.21.15.42:0/3145434416 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2db81023a0 0x7f2db819c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:58:28.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:28.753+0000 7f2dbd975640 1 --2- 172.21.15.42:0/3145434416 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2db810ff50 0x7f2db81a1c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:58:28.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:28.754+0000 7f2dbc973640 1 -- 172.21.15.42:0/3145434416 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2db81023a0 msgr2=0x7f2db819c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:58:28.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:28.754+0000 7f2dbc973640 1 --2- 172.21.15.42:0/3145434416 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2db81023a0 0x7f2db819c410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:28.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:28.754+0000 7f2dbc973640 1 -- 172.21.15.42:0/3145434416 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2db810ff50 msgr2=0x7f2db81a1c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:58:28.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:28.754+0000 7f2dbc973640 1 --2- 172.21.15.42:0/3145434416 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2db810ff50 0x7f2db81a1c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:28.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:28.754+0000 7f2dbc973640 1 -- 172.21.15.42:0/3145434416 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2db81a2380 con 0x7f2db8102cf0 2024-08-30T08:58:28.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:28.754+0000 7f2dbc973640 1 --2- 172.21.15.42:0/3145434416 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2db8102cf0 0x7f2db819c9b0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f2da800d950 tx=0x7f2da800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:58:28.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:28.754+0000 7f2dae7fc640 1 -- 172.21.15.42:0/3145434416 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2da8017070 con 0x7f2db8102cf0 2024-08-30T08:58:28.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:28.754+0000 7f2dbf3d8640 1 -- 172.21.15.42:0/3145434416 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2db81a2670 con 0x7f2db8102cf0 2024-08-30T08:58:28.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:28.754+0000 7f2dae7fc640 1 -- 172.21.15.42:0/3145434416 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2da8009c30 con 0x7f2db8102cf0 2024-08-30T08:58:28.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:28.754+0000 7f2dae7fc640 1 -- 172.21.15.42:0/3145434416 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2da800f620 con 0x7f2db8102cf0 2024-08-30T08:58:28.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:28.754+0000 7f2dbf3d8640 1 -- 172.21.15.42:0/3145434416 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2db81a8dd0 con 0x7f2db8102cf0 2024-08-30T08:58:28.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:28.755+0000 7f2dbf3d8640 1 -- 172.21.15.42:0/3145434416 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2db806c020 con 0x7f2db8102cf0 2024-08-30T08:58:28.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:28.757+0000 7f2dae7fc640 1 -- 172.21.15.42:0/3145434416 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) ==== 152577+0+0 (secure 0 0 0) 0x7f2da8005030 con 0x7f2db8102cf0 2024-08-30T08:58:28.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:28.757+0000 7f2dae7fc640 1 --2- 172.21.15.42:0/3145434416 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f2da40b4b30 0x7f2da40b7040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:58:28.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:28.758+0000 7f2dbd174640 1 --2- 172.21.15.42:0/3145434416 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f2da40b4b30 0x7f2da40b7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:58:28.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:28.758+0000 7f2dae7fc640 1 -- 172.21.15.42:0/3145434416 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8802+0+0 (secure 0 0 0) 0x7f2da80e4e50 con 0x7f2db8102cf0 2024-08-30T08:58:28.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:28.758+0000 7f2dbd174640 1 --2- 172.21.15.42:0/3145434416 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f2da40b4b30 0x7f2da40b7040 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f2db400bbe0 tx=0x7f2db400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:58:28.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:28.760+0000 7f2dae7fc640 1 -- 172.21.15.42:0/3145434416 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f2da8009de0 con 0x7f2db8102cf0 2024-08-30T08:58:28.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:58:28 smithi167 ceph-mon[22063]: pgmap v300: 129 pgs: 129 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-30T08:58:28.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:58:28 smithi190 ceph-mon[22001]: pgmap v300: 129 pgs: 129 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-30T08:58:28.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:28.959+0000 7f2dbf3d8640 1 -- 172.21.15.42:0/3145434416 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f2db81027b0 con 0x7f2db8102cf0 2024-08-30T08:58:28.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:28.960+0000 7f2dae7fc640 1 -- 172.21.15.42:0/3145434416 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v79) ==== 81+0+590 (secure 0 0 0) 0x7f2da80ab6f0 con 0x7f2db8102cf0 2024-08-30T08:58:28.962 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/2691233260 2024-08-31T08:50:44.821070+0000 2024-08-30T08:58:28.962 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/1518053351 2024-08-31T08:50:44.821070+0000 2024-08-30T08:58:28.962 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/352001020 2024-08-31T08:50:44.821070+0000 2024-08-30T08:58:28.962 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:6801/1733768634 2024-08-31T08:50:44.821070+0000 2024-08-30T08:58:28.962 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/1000597888 2024-08-31T08:50:18.561851+0000 2024-08-30T08:58:28.962 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:6800/1733768634 2024-08-31T08:50:44.821070+0000 2024-08-30T08:58:28.962 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:6801/3538882081 2024-08-31T08:50:18.561851+0000 2024-08-30T08:58:28.962 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:6800/3538882081 2024-08-31T08:50:18.561851+0000 2024-08-30T08:58:28.962 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/145366508 2024-08-31T08:50:18.561851+0000 2024-08-30T08:58:28.962 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/3346355213 2024-08-31T08:50:18.561851+0000 2024-08-30T08:58:28.962 INFO:teuthology.orchestra.run.smithi042.stderr:listed 10 entries 2024-08-30T08:58:28.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:28.962+0000 7f2dbf3d8640 1 -- 172.21.15.42:0/3145434416 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f2da40b4b30 msgr2=0x7f2da40b7040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:58:28.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:28.962+0000 7f2dbf3d8640 1 --2- 172.21.15.42:0/3145434416 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f2da40b4b30 0x7f2da40b7040 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f2db400bbe0 tx=0x7f2db400b390 comp rx=0 tx=0).stop 2024-08-30T08:58:28.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:28.962+0000 7f2dbf3d8640 1 -- 172.21.15.42:0/3145434416 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2db8102cf0 msgr2=0x7f2db819c9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:58:28.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:28.962+0000 7f2dbf3d8640 1 --2- 172.21.15.42:0/3145434416 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2db8102cf0 0x7f2db819c9b0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f2da800d950 tx=0x7f2da800de10 comp rx=0 tx=0).stop 2024-08-30T08:58:28.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:28.962+0000 7f2dbf3d8640 1 -- 172.21.15.42:0/3145434416 shutdown_connections 2024-08-30T08:58:28.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:28.962+0000 7f2dbf3d8640 1 --2- 172.21.15.42:0/3145434416 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f2da40b4b30 0x7f2da40b7040 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:28.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:28.962+0000 7f2dbf3d8640 1 --2- 172.21.15.42:0/3145434416 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2db810ff50 0x7f2db81a1c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:28.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:28.962+0000 7f2dbf3d8640 1 --2- 172.21.15.42:0/3145434416 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2db8102cf0 0x7f2db819c9b0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:28.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:28.962+0000 7f2dbf3d8640 1 --2- 172.21.15.42:0/3145434416 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2db81023a0 0x7f2db819c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:28.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:28.962+0000 7f2dbf3d8640 1 -- 172.21.15.42:0/3145434416 >> 172.21.15.42:0/3145434416 conn(0x7f2db80794f0 msgr2=0x7f2db8109b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:58:28.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:28.962+0000 7f2dbf3d8640 1 -- 172.21.15.42:0/3145434416 shutdown_connections 2024-08-30T08:58:28.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:28.962+0000 7f2dbf3d8640 1 -- 172.21.15.42:0/3145434416 wait complete. 2024-08-30T08:58:29.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:58:29 smithi042 ceph-mon[18652]: from='client.? 172.21.15.42:0/3145434416' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-08-30T08:58:29.689 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-08-30T08:58:29.689 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/proc/self/mounts of=/dev/stdout 2024-08-30T08:58:29.707 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph osd blocklist ls 2024-08-30T08:58:29.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:58:29 smithi167 ceph-mon[22063]: from='client.? 172.21.15.42:0/3145434416' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-08-30T08:58:29.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:58:29 smithi190 ceph-mon[22001]: from='client.? 172.21.15.42:0/3145434416' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-08-30T08:58:30.003 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T08:58:30.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:30.511+0000 7f94953e6640 1 -- 172.21.15.42:0/29845238 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9490109bc0 msgr2=0x7f94901103f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:58:30.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:30.511+0000 7f94953e6640 1 --2- 172.21.15.42:0/29845238 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9490109bc0 0x7f94901103f0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f9488009ca0 tx=0x7f948802a8c0 comp rx=0 tx=0).stop 2024-08-30T08:58:30.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:30.511+0000 7f94953e6640 1 -- 172.21.15.42:0/29845238 shutdown_connections 2024-08-30T08:58:30.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:30.511+0000 7f94953e6640 1 --2- 172.21.15.42:0/29845238 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9490109bc0 0x7f94901103f0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:30.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:30.511+0000 7f94953e6640 1 --2- 172.21.15.42:0/29845238 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9490109270 0x7f9490109680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:30.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:30.511+0000 7f94953e6640 1 --2- 172.21.15.42:0/29845238 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9490103930 0x7f9490103d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:30.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:30.511+0000 7f94953e6640 1 -- 172.21.15.42:0/29845238 >> 172.21.15.42:0/29845238 conn(0x7f94900ff3a0 msgr2=0x7f94901017f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:58:30.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:30.512+0000 7f94953e6640 1 -- 172.21.15.42:0/29845238 shutdown_connections 2024-08-30T08:58:30.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:30.512+0000 7f94953e6640 1 -- 172.21.15.42:0/29845238 wait complete. 2024-08-30T08:58:30.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:30.512+0000 7f94953e6640 1 Processor -- start 2024-08-30T08:58:30.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:30.512+0000 7f94953e6640 1 -- start start 2024-08-30T08:58:30.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:30.513+0000 7f94953e6640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9490103930 0x7f949019a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:58:30.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:30.513+0000 7f94953e6640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9490109270 0x7f949019a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:58:30.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:30.513+0000 7f94953e6640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9490109bc0 0x7f949019f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:58:30.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:30.513+0000 7f94953e6640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f9490113880 con 0x7f9490103930 2024-08-30T08:58:30.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:30.513+0000 7f94953e6640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f9490113700 con 0x7f9490109270 2024-08-30T08:58:30.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:30.513+0000 7f94953e6640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f9490113a00 con 0x7f9490109bc0 2024-08-30T08:58:30.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:30.513+0000 7f948effd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9490103930 0x7f949019a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:58:30.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:30.513+0000 7f948effd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9490103930 0x7f949019a1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52942/0 (socket says 172.21.15.42:52942) 2024-08-30T08:58:30.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:30.513+0000 7f948effd640 1 -- 172.21.15.42:0/1878037921 learned_addr learned my addr 172.21.15.42:0/1878037921 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T08:58:30.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:30.513+0000 7f9487fff640 1 --2- 172.21.15.42:0/1878037921 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9490109270 0x7f949019a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:58:30.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:30.513+0000 7f948f7fe640 1 --2- 172.21.15.42:0/1878037921 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9490109bc0 0x7f949019f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:58:30.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:30.513+0000 7f948effd640 1 -- 172.21.15.42:0/1878037921 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9490109bc0 msgr2=0x7f949019f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:58:30.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:30.513+0000 7f948effd640 1 --2- 172.21.15.42:0/1878037921 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9490109bc0 0x7f949019f9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:30.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:30.513+0000 7f948effd640 1 -- 172.21.15.42:0/1878037921 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9490109270 msgr2=0x7f949019a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:58:30.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:30.513+0000 7f948effd640 1 --2- 172.21.15.42:0/1878037921 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9490109270 0x7f949019a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:30.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:30.513+0000 7f948effd640 1 -- 172.21.15.42:0/1878037921 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94901a0060 con 0x7f9490103930 2024-08-30T08:58:30.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:30.513+0000 7f948f7fe640 1 --2- 172.21.15.42:0/1878037921 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9490109bc0 0x7f949019f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-30T08:58:30.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:30.513+0000 7f948effd640 1 --2- 172.21.15.42:0/1878037921 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9490103930 0x7f949019a1b0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f948000c920 tx=0x7f948000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:58:30.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:30.514+0000 7f948cff9640 1 -- 172.21.15.42:0/1878037921 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9480007c70 con 0x7f9490103930 2024-08-30T08:58:30.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:30.514+0000 7f94953e6640 1 -- 172.21.15.42:0/1878037921 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94901a0350 con 0x7f9490103930 2024-08-30T08:58:30.519 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:30.514+0000 7f94953e6640 1 -- 172.21.15.42:0/1878037921 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94901a6b70 con 0x7f9490103930 2024-08-30T08:58:30.519 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:30.514+0000 7f948cff9640 1 -- 172.21.15.42:0/1878037921 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9480007e10 con 0x7f9490103930 2024-08-30T08:58:30.519 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:30.514+0000 7f948cff9640 1 -- 172.21.15.42:0/1878037921 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9480018820 con 0x7f9490103930 2024-08-30T08:58:30.519 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:30.515+0000 7f94953e6640 1 -- 172.21.15.42:0/1878037921 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94700052c0 con 0x7f9490103930 2024-08-30T08:58:30.519 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:30.517+0000 7f948cff9640 1 -- 172.21.15.42:0/1878037921 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) ==== 152577+0+0 (secure 0 0 0) 0x7f94800189c0 con 0x7f9490103930 2024-08-30T08:58:30.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:30.518+0000 7f948cff9640 1 --2- 172.21.15.42:0/1878037921 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f947c0b4be0 0x7f947c0b70f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:58:30.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:30.518+0000 7f9487fff640 1 --2- 172.21.15.42:0/1878037921 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f947c0b4be0 0x7f947c0b70f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:58:30.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:30.518+0000 7f948cff9640 1 -- 172.21.15.42:0/1878037921 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8802+0+0 (secure 0 0 0) 0x7f94800e5450 con 0x7f9490103930 2024-08-30T08:58:30.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:30.519+0000 7f9487fff640 1 --2- 172.21.15.42:0/1878037921 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f947c0b4be0 0x7f947c0b70f0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f9490102aa0 tx=0x7f9478005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:58:30.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:30.522+0000 7f948cff9640 1 -- 172.21.15.42:0/1878037921 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f9480014030 con 0x7f9490103930 2024-08-30T08:58:30.723 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:58:30 smithi042 ceph-mon[18652]: pgmap v301: 129 pgs: 129 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-08-30T08:58:30.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:30.721+0000 7f94953e6640 1 -- 172.21.15.42:0/1878037921 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f94700055b0 con 0x7f9490103930 2024-08-30T08:58:30.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:30.722+0000 7f948cff9640 1 -- 172.21.15.42:0/1878037921 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v79) ==== 81+0+590 (secure 0 0 0) 0x7f94800ac450 con 0x7f9490103930 2024-08-30T08:58:30.724 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/2691233260 2024-08-31T08:50:44.821070+0000 2024-08-30T08:58:30.724 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/1518053351 2024-08-31T08:50:44.821070+0000 2024-08-30T08:58:30.724 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/352001020 2024-08-31T08:50:44.821070+0000 2024-08-30T08:58:30.724 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:6801/1733768634 2024-08-31T08:50:44.821070+0000 2024-08-30T08:58:30.724 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/1000597888 2024-08-31T08:50:18.561851+0000 2024-08-30T08:58:30.724 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:6800/1733768634 2024-08-31T08:50:44.821070+0000 2024-08-30T08:58:30.724 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:6801/3538882081 2024-08-31T08:50:18.561851+0000 2024-08-30T08:58:30.724 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:6800/3538882081 2024-08-31T08:50:18.561851+0000 2024-08-30T08:58:30.724 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/145366508 2024-08-31T08:50:18.561851+0000 2024-08-30T08:58:30.725 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/3346355213 2024-08-31T08:50:18.561851+0000 2024-08-30T08:58:30.725 INFO:teuthology.orchestra.run.smithi042.stderr:listed 10 entries 2024-08-30T08:58:30.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:30.724+0000 7f94953e6640 1 -- 172.21.15.42:0/1878037921 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f947c0b4be0 msgr2=0x7f947c0b70f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:58:30.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:30.724+0000 7f94953e6640 1 --2- 172.21.15.42:0/1878037921 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f947c0b4be0 0x7f947c0b70f0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f9490102aa0 tx=0x7f9478005f90 comp rx=0 tx=0).stop 2024-08-30T08:58:30.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:30.724+0000 7f94953e6640 1 -- 172.21.15.42:0/1878037921 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9490103930 msgr2=0x7f949019a1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:58:30.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:30.724+0000 7f94953e6640 1 --2- 172.21.15.42:0/1878037921 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9490103930 0x7f949019a1b0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f948000c920 tx=0x7f948000cde0 comp rx=0 tx=0).stop 2024-08-30T08:58:30.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:30.724+0000 7f94953e6640 1 -- 172.21.15.42:0/1878037921 shutdown_connections 2024-08-30T08:58:30.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:30.724+0000 7f94953e6640 1 --2- 172.21.15.42:0/1878037921 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f947c0b4be0 0x7f947c0b70f0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:30.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:30.724+0000 7f94953e6640 1 --2- 172.21.15.42:0/1878037921 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9490109bc0 0x7f949019f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:30.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:30.724+0000 7f94953e6640 1 --2- 172.21.15.42:0/1878037921 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9490109270 0x7f949019a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:30.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:30.724+0000 7f94953e6640 1 --2- 172.21.15.42:0/1878037921 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9490103930 0x7f949019a1b0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:30.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:30.724+0000 7f94953e6640 1 -- 172.21.15.42:0/1878037921 >> 172.21.15.42:0/1878037921 conn(0x7f94900ff3a0 msgr2=0x7f94901088a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:58:30.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:30.724+0000 7f94953e6640 1 -- 172.21.15.42:0/1878037921 shutdown_connections 2024-08-30T08:58:30.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:30.724+0000 7f94953e6640 1 -- 172.21.15.42:0/1878037921 wait complete. 2024-08-30T08:58:30.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:58:30 smithi167 ceph-mon[22063]: pgmap v301: 129 pgs: 129 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-08-30T08:58:30.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:58:30 smithi190 ceph-mon[22001]: pgmap v301: 129 pgs: 129 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-08-30T08:58:31.450 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi042.front.sepia.ceph.com... 2024-08-30T08:58:31.450 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-30T08:58:31.450 DEBUG:teuthology.orchestra.run.smithi042:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-08-30T08:58:31.477 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi042.stderr:ceph-fuse[65508]: fuse finished with error 0 and tester_r 0 2024-08-30T08:58:31.478 INFO:teuthology.orchestra.run:waiting for 300 2024-08-30T08:58:31.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:58:31 smithi167 ceph-mon[22063]: from='client.? 172.21.15.42:0/1878037921' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-08-30T08:58:31.882 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:58:31 smithi042 ceph-mon[18652]: from='client.? 172.21.15.42:0/1878037921' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-08-30T08:58:31.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:58:31 smithi190 ceph-mon[22001]: from='client.? 172.21.15.42:0/1878037921' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-08-30T08:58:32.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:58:32 smithi167 ceph-mon[22063]: pgmap v302: 129 pgs: 129 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-08-30T08:58:32.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:58:32 smithi042 ceph-mon[18652]: pgmap v302: 129 pgs: 129 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-08-30T08:58:32.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:58:32 smithi190 ceph-mon[22001]: pgmap v302: 129 pgs: 129 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-08-30T08:58:34.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:58:34 smithi167 ceph-mon[22063]: pgmap v303: 129 pgs: 129 active+clean; 581 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-08-30T08:58:34.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:58:34 smithi042 ceph-mon[18652]: pgmap v303: 129 pgs: 129 active+clean; 581 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-08-30T08:58:34.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:58:34 smithi190 ceph-mon[22001]: pgmap v303: 129 pgs: 129 active+clean; 581 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-08-30T08:58:36.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:58:36 smithi167 ceph-mon[22063]: pgmap v304: 129 pgs: 129 active+clean; 581 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-08-30T08:58:36.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:58:36 smithi042 ceph-mon[18652]: pgmap v304: 129 pgs: 129 active+clean; 581 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-08-30T08:58:36.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:58:36 smithi190 ceph-mon[22001]: pgmap v304: 129 pgs: 129 active+clean; 581 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-08-30T08:58:37.579 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi042.front.sepia.ceph.com 2024-08-30T08:58:37.579 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-30T08:58:37.580 DEBUG:teuthology.orchestra.run.smithi042:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-08-30T08:58:37.614 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph osd dump --format=json 2024-08-30T08:58:37.922 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T08:58:38.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:38.410+0000 7ff46e644640 1 -- 172.21.15.42:0/293417247 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff468109bb0 msgr2=0x7ff4681103e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:58:38.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:38.410+0000 7ff46e644640 1 --2- 172.21.15.42:0/293417247 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff468109bb0 0x7ff4681103e0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7ff460009ca0 tx=0x7ff46002a8c0 comp rx=0 tx=0).stop 2024-08-30T08:58:38.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:38.410+0000 7ff46e644640 1 -- 172.21.15.42:0/293417247 shutdown_connections 2024-08-30T08:58:38.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:38.410+0000 7ff46e644640 1 --2- 172.21.15.42:0/293417247 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff468109260 0x7ff468109670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:38.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:38.410+0000 7ff46e644640 1 --2- 172.21.15.42:0/293417247 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff468109bb0 0x7ff4681103e0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:38.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:38.410+0000 7ff46e644640 1 --2- 172.21.15.42:0/293417247 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff468103920 0x7ff468103d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:38.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:38.410+0000 7ff46e644640 1 -- 172.21.15.42:0/293417247 >> 172.21.15.42:0/293417247 conn(0x7ff4680ff370 msgr2=0x7ff4681017c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:58:38.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:38.410+0000 7ff46e644640 1 -- 172.21.15.42:0/293417247 shutdown_connections 2024-08-30T08:58:38.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:38.410+0000 7ff46e644640 1 -- 172.21.15.42:0/293417247 wait complete. 2024-08-30T08:58:38.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:38.411+0000 7ff46e644640 1 Processor -- start 2024-08-30T08:58:38.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:38.411+0000 7ff46e644640 1 -- start start 2024-08-30T08:58:38.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:38.411+0000 7ff46e644640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff468103920 0x7ff4680754b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:58:38.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:38.411+0000 7ff46e644640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff468109260 0x7ff4680759f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:58:38.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:38.411+0000 7ff46e644640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff468109bb0 0x7ff468075f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:58:38.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:38.411+0000 7ff46e644640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7ff468113870 con 0x7ff468109260 2024-08-30T08:58:38.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:38.411+0000 7ff46e644640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff4681136f0 con 0x7ff468109bb0 2024-08-30T08:58:38.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:38.411+0000 7ff46e644640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7ff4681139f0 con 0x7ff468103920 2024-08-30T08:58:38.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:38.411+0000 7ff45ffff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff468109260 0x7ff4680759f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:58:38.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:38.411+0000 7ff45ffff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff468109260 0x7ff4680759f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:54512/0 (socket says 172.21.15.42:54512) 2024-08-30T08:58:38.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:38.411+0000 7ff45ffff640 1 -- 172.21.15.42:0/3605599357 learned_addr learned my addr 172.21.15.42:0/3605599357 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T08:58:38.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:38.412+0000 7ff46cbe1640 1 --2- 172.21.15.42:0/3605599357 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff468109bb0 0x7ff468075f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:58:38.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:38.412+0000 7ff467fff640 1 --2- 172.21.15.42:0/3605599357 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff468103920 0x7ff4680754b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:58:38.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:38.412+0000 7ff45ffff640 1 -- 172.21.15.42:0/3605599357 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff468103920 msgr2=0x7ff4680754b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:58:38.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:38.412+0000 7ff45ffff640 1 --2- 172.21.15.42:0/3605599357 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff468103920 0x7ff4680754b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:38.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:38.412+0000 7ff45ffff640 1 -- 172.21.15.42:0/3605599357 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff468109bb0 msgr2=0x7ff468075f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:58:38.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:38.412+0000 7ff45ffff640 1 --2- 172.21.15.42:0/3605599357 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff468109bb0 0x7ff468075f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:38.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:38.412+0000 7ff45ffff640 1 -- 172.21.15.42:0/3605599357 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4681a84a0 con 0x7ff468109260 2024-08-30T08:58:38.421 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:38.412+0000 7ff45ffff640 1 --2- 172.21.15.42:0/3605599357 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff468109260 0x7ff4680759f0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7ff45000d740 tx=0x7ff45000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:58:38.421 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:38.412+0000 7ff465ffb640 1 -- 172.21.15.42:0/3605599357 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff450017070 con 0x7ff468109260 2024-08-30T08:58:38.421 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:38.412+0000 7ff46e644640 1 -- 172.21.15.42:0/3605599357 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4681a8790 con 0x7ff468109260 2024-08-30T08:58:38.421 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:38.412+0000 7ff465ffb640 1 -- 172.21.15.42:0/3605599357 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff45000de50 con 0x7ff468109260 2024-08-30T08:58:38.421 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:38.412+0000 7ff465ffb640 1 -- 172.21.15.42:0/3605599357 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff45000f620 con 0x7ff468109260 2024-08-30T08:58:38.421 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:38.412+0000 7ff46e644640 1 -- 172.21.15.42:0/3605599357 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4681a8d20 con 0x7ff468109260 2024-08-30T08:58:38.421 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:38.413+0000 7ff46e644640 1 -- 172.21.15.42:0/3605599357 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff468103d30 con 0x7ff468109260 2024-08-30T08:58:38.421 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:38.415+0000 7ff465ffb640 1 -- 172.21.15.42:0/3605599357 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) ==== 152577+0+0 (secure 0 0 0) 0x7ff45001a070 con 0x7ff468109260 2024-08-30T08:58:38.421 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:38.415+0000 7ff465ffb640 1 --2- 172.21.15.42:0/3605599357 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7ff4540b4b30 0x7ff4540b7040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:58:38.421 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:38.415+0000 7ff465ffb640 1 -- 172.21.15.42:0/3605599357 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8802+0+0 (secure 0 0 0) 0x7ff4500e4db0 con 0x7ff468109260 2024-08-30T08:58:38.421 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:38.415+0000 7ff467fff640 1 --2- 172.21.15.42:0/3605599357 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7ff4540b4b30 0x7ff4540b7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:58:38.421 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:38.416+0000 7ff467fff640 1 --2- 172.21.15.42:0/3605599357 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7ff4540b4b30 0x7ff4540b7040 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ff458005fd0 tx=0x7ff458005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:58:38.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:38.419+0000 7ff465ffb640 1 -- 172.21.15.42:0/3605599357 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7ff4500abca0 con 0x7ff468109260 2024-08-30T08:58:38.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:38.606+0000 7ff46e644640 1 -- 172.21.15.42:0/3605599357 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff4680767b0 con 0x7ff468109260 2024-08-30T08:58:38.608 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:58:38 smithi042 ceph-mon[18652]: pgmap v305: 129 pgs: 129 active+clean; 581 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-08-30T08:58:38.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:38.608+0000 7ff465ffb640 1 -- 172.21.15.42:0/3605599357 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) ==== 74+0+24248 (secure 0 0 0) 0x7ff4680767b0 con 0x7ff468109260 2024-08-30T08:58:38.610 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-08-30T08:58:38.610 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":79,"fsid":"b0895c1a-66ac-11ef-bcd4-c7b262605968","created":"2024-08-30T08:49:44.858265+0000","modified":"2024-08-30T08:58:12.223108+0000","last_up_change":"2024-08-30T08:57:12.367791+0000","last_in_change":"2024-08-30T08:56:50.274678+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-30T08:53:27.297022+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-30T08:57:58.489964+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-30T08:58:01.064783+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-08-30T08:58:07.416534+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"029d62d7-a6c6-4905-8c76-6dffa9d57b92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":2617890537},{"type":"v1","addr":"172.21.15.42:6803","nonce":2617890537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":2617890537},{"type":"v1","addr":"172.21.15.42:6805","nonce":2617890537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2617890537},{"type":"v1","addr":"172.21.15.42:6809","nonce":2617890537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":2617890537},{"type":"v1","addr":"172.21.15.42:6807","nonce":2617890537}]},"public_addr":"172.21.15.42:6803/2617890537","cluster_addr":"172.21.15.42:6805/2617890537","heartbeat_back_addr":"172.21.15.42:6809/2617890537","heartbeat_front_addr":"172.21.15.42:6807/2617890537","state":["exists","up"]},{"osd":1,"uuid":"3a78384b-0207-41df-9384-22665007d5b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":2791960082},{"type":"v1","addr":"172.21.15.167:6801","nonce":2791960082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":2791960082},{"type":"v1","addr":"172.21.15.167:6803","nonce":2791960082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":2791960082},{"type":"v1","addr":"172.21.15.167:6807","nonce":2791960082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":2791960082},{"type":"v1","addr":"172.21.15.167:6805","nonce":2791960082}]},"public_addr":"172.21.15.167:6801/2791960082","cluster_addr":"172.21.15.167:6803/2791960082","heartbeat_back_addr":"172.21.15.167:6807/2791960082","heartbeat_front_addr":"172.21.15.167:6805/2791960082","state":["exists","up"]},{"osd":2,"uuid":"16001809-c27c-4f52-948a-9998261acb4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":1101326404},{"type":"v1","addr":"172.21.15.190:6801","nonce":1101326404}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":1101326404},{"type":"v1","addr":"172.21.15.190:6803","nonce":1101326404}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":1101326404},{"type":"v1","addr":"172.21.15.190:6807","nonce":1101326404}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":1101326404},{"type":"v1","addr":"172.21.15.190:6805","nonce":1101326404}]},"public_addr":"172.21.15.190:6801/1101326404","cluster_addr":"172.21.15.190:6803/1101326404","heartbeat_back_addr":"172.21.15.190:6807/1101326404","heartbeat_front_addr":"172.21.15.190:6805/1101326404","state":["exists","up"]},{"osd":3,"uuid":"87bb3e36-9e9e-4b19-9d8c-0cdf6e089d3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":1135599916},{"type":"v1","addr":"172.21.15.42:6811","nonce":1135599916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":1135599916},{"type":"v1","addr":"172.21.15.42:6813","nonce":1135599916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":1135599916},{"type":"v1","addr":"172.21.15.42:6817","nonce":1135599916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":1135599916},{"type":"v1","addr":"172.21.15.42:6815","nonce":1135599916}]},"public_addr":"172.21.15.42:6811/1135599916","cluster_addr":"172.21.15.42:6813/1135599916","heartbeat_back_addr":"172.21.15.42:6817/1135599916","heartbeat_front_addr":"172.21.15.42:6815/1135599916","state":["exists","up"]},{"osd":4,"uuid":"3ca1aae3-c972-4fb9-ac86-236279c30d64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":2927511419},{"type":"v1","addr":"172.21.15.167:6809","nonce":2927511419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":2927511419},{"type":"v1","addr":"172.21.15.167:6811","nonce":2927511419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":2927511419},{"type":"v1","addr":"172.21.15.167:6815","nonce":2927511419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":2927511419},{"type":"v1","addr":"172.21.15.167:6813","nonce":2927511419}]},"public_addr":"172.21.15.167:6809/2927511419","cluster_addr":"172.21.15.167:6811/2927511419","heartbeat_back_addr":"172.21.15.167:6815/2927511419","heartbeat_front_addr":"172.21.15.167:6813/2927511419","state":["exists","up"]},{"osd":5,"uuid":"5d07f54b-0d07-4efe-9bcf-390755ce0258","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":2557145333},{"type":"v1","addr":"172.21.15.190:6809","nonce":2557145333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":2557145333},{"type":"v1","addr":"172.21.15.190:6811","nonce":2557145333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":2557145333},{"type":"v1","addr":"172.21.15.190:6815","nonce":2557145333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":2557145333},{"type":"v1","addr":"172.21.15.190:6813","nonce":2557145333}]},"public_addr":"172.21.15.190:6809/2557145333","cluster_addr":"172.21.15.190:6811/2557145333","heartbeat_back_addr":"172.21.15.190:6815/2557145333","heartbeat_front_addr":"172.21.15.190:6813/2557145333","state":["exists","up"]},{"osd":6,"uuid":"96ad64af-44ff-4197-97d4-156f0cb88e81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":3213526100},{"type":"v1","addr":"172.21.15.42:6819","nonce":3213526100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":3213526100},{"type":"v1","addr":"172.21.15.42:6821","nonce":3213526100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":3213526100},{"type":"v1","addr":"172.21.15.42:6825","nonce":3213526100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":3213526100},{"type":"v1","addr":"172.21.15.42:6823","nonce":3213526100}]},"public_addr":"172.21.15.42:6819/3213526100","cluster_addr":"172.21.15.42:6821/3213526100","heartbeat_back_addr":"172.21.15.42:6825/3213526100","heartbeat_front_addr":"172.21.15.42:6823/3213526100","state":["exists","up"]},{"osd":7,"uuid":"337a2112-eaf7-4e6b-86be-4e40d6ddf835","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":2962799814},{"type":"v1","addr":"172.21.15.167:6817","nonce":2962799814}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":2962799814},{"type":"v1","addr":"172.21.15.167:6819","nonce":2962799814}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":2962799814},{"type":"v1","addr":"172.21.15.167:6823","nonce":2962799814}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":2962799814},{"type":"v1","addr":"172.21.15.167:6821","nonce":2962799814}]},"public_addr":"172.21.15.167:6817/2962799814","cluster_addr":"172.21.15.167:6819/2962799814","heartbeat_back_addr":"172.21.15.167:6823/2962799814","heartbeat_front_addr":"172.21.15.167:6821/2962799814","state":["exists","up"]},{"osd":8,"uuid":"219a7224-4331-464b-ac86-c51054d51d91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":3242198843},{"type":"v1","addr":"172.21.15.190:6817","nonce":3242198843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":3242198843},{"type":"v1","addr":"172.21.15.190:6819","nonce":3242198843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":3242198843},{"type":"v1","addr":"172.21.15.190:6823","nonce":3242198843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":3242198843},{"type":"v1","addr":"172.21.15.190:6821","nonce":3242198843}]},"public_addr":"172.21.15.190:6817/3242198843","cluster_addr":"172.21.15.190:6819/3242198843","heartbeat_back_addr":"172.21.15.190:6823/3242198843","heartbeat_front_addr":"172.21.15.190:6821/3242198843","state":["exists","up"]},{"osd":9,"uuid":"2f2f5177-a009-40d9-991c-76aec8a5cba6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":1123536325},{"type":"v1","addr":"172.21.15.42:6827","nonce":1123536325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":1123536325},{"type":"v1","addr":"172.21.15.42:6829","nonce":1123536325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":1123536325},{"type":"v1","addr":"172.21.15.42:6833","nonce":1123536325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":1123536325},{"type":"v1","addr":"172.21.15.42:6831","nonce":1123536325}]},"public_addr":"172.21.15.42:6827/1123536325","cluster_addr":"172.21.15.42:6829/1123536325","heartbeat_back_addr":"172.21.15.42:6833/1123536325","heartbeat_front_addr":"172.21.15.42:6831/1123536325","state":["exists","up"]},{"osd":10,"uuid":"96ef5058-2518-4862-a165-17666b33a3e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":3451198319},{"type":"v1","addr":"172.21.15.167:6825","nonce":3451198319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":3451198319},{"type":"v1","addr":"172.21.15.167:6827","nonce":3451198319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":3451198319},{"type":"v1","addr":"172.21.15.167:6831","nonce":3451198319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":3451198319},{"type":"v1","addr":"172.21.15.167:6829","nonce":3451198319}]},"public_addr":"172.21.15.167:6825/3451198319","cluster_addr":"172.21.15.167:6827/3451198319","heartbeat_back_addr":"172.21.15.167:6831/3451198319","heartbeat_front_addr":"172.21.15.167:6829/3451198319","state":["exists","up"]},{"osd":11,"uuid":"a4eb9bd3-3faa-4c81-ba6d-c2084b72c9ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":380064457},{"type":"v1","addr":"172.21.15.190:6825","nonce":380064457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":380064457},{"type":"v1","addr":"172.21.15.190:6827","nonce":380064457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":380064457},{"type":"v1","addr":"172.21.15.190:6831","nonce":380064457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":380064457},{"type":"v1","addr":"172.21.15.190:6829","nonce":380064457}]},"public_addr":"172.21.15.190:6825/380064457","cluster_addr":"172.21.15.190:6827/380064457","heartbeat_back_addr":"172.21.15.190:6831/380064457","heartbeat_front_addr":"172.21.15.190:6829/380064457","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-30T08:52:36.511448+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-30T08:53:00.027277+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-30T08:53:23.770245+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-30T08:53:49.322226+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-30T08:54:14.350234+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-30T08:54:39.491793+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-30T08:55:04.870596+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-30T08:55:30.314150+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-30T08:55:54.483843+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-30T08:56:20.586115+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-30T08:56:45.666783+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-30T08:57:10.718387+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.42:0/2691233260":"2024-08-31T08:50:44.821070+0000","172.21.15.42:0/1518053351":"2024-08-31T08:50:44.821070+0000","172.21.15.42:0/352001020":"2024-08-31T08:50:44.821070+0000","172.21.15.42:6801/1733768634":"2024-08-31T08:50:44.821070+0000","172.21.15.42:0/1000597888":"2024-08-31T08:50:18.561851+0000","172.21.15.42:6800/1733768634":"2024-08-31T08:50:44.821070+0000","172.21.15.42:6801/3538882081":"2024-08-31T08:50:18.561851+0000","172.21.15.42:6800/3538882081":"2024-08-31T08:50:18.561851+0000","172.21.15.42:0/145366508":"2024-08-31T08:50:18.561851+0000","172.21.15.42:0/3346355213":"2024-08-31T08:50:18.561851+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-30T08:58:38.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:38.610+0000 7ff46e644640 1 -- 172.21.15.42:0/3605599357 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7ff4540b4b30 msgr2=0x7ff4540b7040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:58:38.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:38.610+0000 7ff46e644640 1 --2- 172.21.15.42:0/3605599357 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7ff4540b4b30 0x7ff4540b7040 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ff458005fd0 tx=0x7ff458005850 comp rx=0 tx=0).stop 2024-08-30T08:58:38.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:38.610+0000 7ff46e644640 1 -- 172.21.15.42:0/3605599357 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff468109260 msgr2=0x7ff4680759f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:58:38.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:38.610+0000 7ff46e644640 1 --2- 172.21.15.42:0/3605599357 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff468109260 0x7ff4680759f0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7ff45000d740 tx=0x7ff45000dc00 comp rx=0 tx=0).stop 2024-08-30T08:58:38.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:38.610+0000 7ff46e644640 1 -- 172.21.15.42:0/3605599357 shutdown_connections 2024-08-30T08:58:38.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:38.610+0000 7ff46e644640 1 --2- 172.21.15.42:0/3605599357 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7ff4540b4b30 0x7ff4540b7040 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:38.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:38.610+0000 7ff46e644640 1 --2- 172.21.15.42:0/3605599357 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff468109bb0 0x7ff468075f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:38.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:38.610+0000 7ff46e644640 1 --2- 172.21.15.42:0/3605599357 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff468109260 0x7ff4680759f0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:38.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:38.610+0000 7ff46e644640 1 --2- 172.21.15.42:0/3605599357 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff468103920 0x7ff4680754b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:38.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:38.610+0000 7ff46e644640 1 -- 172.21.15.42:0/3605599357 >> 172.21.15.42:0/3605599357 conn(0x7ff4680ff370 msgr2=0x7ff468101790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:58:38.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:38.610+0000 7ff46e644640 1 -- 172.21.15.42:0/3605599357 shutdown_connections 2024-08-30T08:58:38.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:38.610+0000 7ff46e644640 1 -- 172.21.15.42:0/3605599357 wait complete. 2024-08-30T08:58:38.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:58:38 smithi167 ceph-mon[22063]: pgmap v305: 129 pgs: 129 active+clean; 581 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-08-30T08:58:38.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:58:38 smithi190 ceph-mon[22001]: pgmap v305: 129 pgs: 129 active+clean; 581 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-08-30T08:58:39.339 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph osd dump --format=json 2024-08-30T08:58:39.596 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T08:58:39.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:58:39 smithi042 ceph-mon[18652]: from='client.? 172.21.15.42:0/3605599357' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-30T08:58:39.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:58:39 smithi167 ceph-mon[22063]: from='client.? 172.21.15.42:0/3605599357' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-30T08:58:39.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:58:39 smithi190 ceph-mon[22001]: from='client.? 172.21.15.42:0/3605599357' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-30T08:58:40.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:40.083+0000 7f7ec1b41640 1 -- 172.21.15.42:0/454698997 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7ebc10dc60 msgr2=0x7f7ebc110150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:58:40.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:40.083+0000 7f7ec1b41640 1 --2- 172.21.15.42:0/454698997 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7ebc10dc60 0x7f7ebc110150 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f7eb4009ca0 tx=0x7f7eb402a8c0 comp rx=0 tx=0).stop 2024-08-30T08:58:40.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:40.083+0000 7f7ec1b41640 1 -- 172.21.15.42:0/454698997 shutdown_connections 2024-08-30T08:58:40.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:40.083+0000 7f7ec1b41640 1 --2- 172.21.15.42:0/454698997 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7ebc10dc60 0x7f7ebc110150 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:40.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:40.083+0000 7f7ec1b41640 1 --2- 172.21.15.42:0/454698997 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7ebc105150 0x7f7ebc10d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:40.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:40.083+0000 7f7ec1b41640 1 --2- 172.21.15.42:0/454698997 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7ebc104800 0x7f7ebc104c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:40.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:40.083+0000 7f7ec1b41640 1 -- 172.21.15.42:0/454698997 >> 172.21.15.42:0/454698997 conn(0x7f7ebc079470 msgr2=0x7f7ebc079880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:58:40.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:40.083+0000 7f7ec1b41640 1 -- 172.21.15.42:0/454698997 shutdown_connections 2024-08-30T08:58:40.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:40.083+0000 7f7ec1b41640 1 -- 172.21.15.42:0/454698997 wait complete. 2024-08-30T08:58:40.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:40.083+0000 7f7ec1b41640 1 Processor -- start 2024-08-30T08:58:40.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:40.084+0000 7f7ec1b41640 1 -- start start 2024-08-30T08:58:40.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:40.084+0000 7f7ec1b41640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7ebc104800 0x7f7ebc19a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:58:40.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:40.084+0000 7f7ec1b41640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7ebc105150 0x7f7ebc19a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:58:40.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:40.084+0000 7f7ec1b41640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7ebc10dc60 0x7f7ebc19fad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:58:40.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:40.084+0000 7f7ec1b41640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f7ebc1138e0 con 0x7f7ebc104800 2024-08-30T08:58:40.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:40.084+0000 7f7ec1b41640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f7ebc113760 con 0x7f7ebc105150 2024-08-30T08:58:40.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:40.084+0000 7f7ec1b41640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f7ebc113a60 con 0x7f7ebc10dc60 2024-08-30T08:58:40.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:40.084+0000 7f7ebb7fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7ebc104800 0x7f7ebc19a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:58:40.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:40.084+0000 7f7ebb7fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7ebc104800 0x7f7ebc19a290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:54528/0 (socket says 172.21.15.42:54528) 2024-08-30T08:58:40.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:40.084+0000 7f7ebb7fe640 1 -- 172.21.15.42:0/4001035811 learned_addr learned my addr 172.21.15.42:0/4001035811 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T08:58:40.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:40.084+0000 7f7ebbfff640 1 --2- 172.21.15.42:0/4001035811 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7ebc10dc60 0x7f7ebc19fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:58:40.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:40.084+0000 7f7ebaffd640 1 --2- 172.21.15.42:0/4001035811 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7ebc105150 0x7f7ebc19a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:58:40.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:40.084+0000 7f7ebb7fe640 1 -- 172.21.15.42:0/4001035811 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7ebc10dc60 msgr2=0x7f7ebc19fad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:58:40.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:40.084+0000 7f7ebb7fe640 1 --2- 172.21.15.42:0/4001035811 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7ebc10dc60 0x7f7ebc19fad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:40.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:40.084+0000 7f7ebb7fe640 1 -- 172.21.15.42:0/4001035811 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7ebc105150 msgr2=0x7f7ebc19a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:58:40.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:40.084+0000 7f7ebb7fe640 1 --2- 172.21.15.42:0/4001035811 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7ebc105150 0x7f7ebc19a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:40.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:40.084+0000 7f7ebb7fe640 1 -- 172.21.15.42:0/4001035811 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ebc1a0200 con 0x7f7ebc104800 2024-08-30T08:58:40.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:40.084+0000 7f7ebaffd640 1 --2- 172.21.15.42:0/4001035811 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7ebc105150 0x7f7ebc19a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-30T08:58:40.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:40.085+0000 7f7ebb7fe640 1 --2- 172.21.15.42:0/4001035811 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7ebc104800 0x7f7ebc19a290 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f7eac00d770 tx=0x7f7eac00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:58:40.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:40.085+0000 7f7eb8ff9640 1 -- 172.21.15.42:0/4001035811 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eac017070 con 0x7f7ebc104800 2024-08-30T08:58:40.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:40.085+0000 7f7ec1b41640 1 -- 172.21.15.42:0/4001035811 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ebc1a04f0 con 0x7f7ebc104800 2024-08-30T08:58:40.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:40.085+0000 7f7eb8ff9640 1 -- 172.21.15.42:0/4001035811 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7eac0054d0 con 0x7f7ebc104800 2024-08-30T08:58:40.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:40.085+0000 7f7eb8ff9640 1 -- 172.21.15.42:0/4001035811 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eac004c30 con 0x7f7ebc104800 2024-08-30T08:58:40.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:40.085+0000 7f7ec1b41640 1 -- 172.21.15.42:0/4001035811 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ebc1a6c50 con 0x7f7ebc104800 2024-08-30T08:58:40.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:40.086+0000 7f7ec1b41640 1 -- 172.21.15.42:0/4001035811 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ebc104c10 con 0x7f7ebc104800 2024-08-30T08:58:40.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:40.088+0000 7f7eb8ff9640 1 -- 172.21.15.42:0/4001035811 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) ==== 152577+0+0 (secure 0 0 0) 0x7f7eac004dd0 con 0x7f7ebc104800 2024-08-30T08:58:40.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:40.088+0000 7f7eb8ff9640 1 --2- 172.21.15.42:0/4001035811 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f7ea40b4b30 0x7f7ea40b7040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:58:40.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:40.088+0000 7f7eb8ff9640 1 -- 172.21.15.42:0/4001035811 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8802+0+0 (secure 0 0 0) 0x7f7eac0e5220 con 0x7f7ebc104800 2024-08-30T08:58:40.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:40.088+0000 7f7ebaffd640 1 --2- 172.21.15.42:0/4001035811 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f7ea40b4b30 0x7f7ea40b7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:58:40.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:40.089+0000 7f7ebaffd640 1 --2- 172.21.15.42:0/4001035811 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f7ea40b4b30 0x7f7ea40b7040 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f7ebc10f960 tx=0x7f7eb0005ec0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:58:40.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:40.091+0000 7f7eb8ff9640 1 -- 172.21.15.42:0/4001035811 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f7eac0ac110 con 0x7f7ebc104800 2024-08-30T08:58:40.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:40.277+0000 7f7ec1b41640 1 -- 172.21.15.42:0/4001035811 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7ebc1a6e80 con 0x7f7ebc104800 2024-08-30T08:58:40.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:40.278+0000 7f7eb8ff9640 1 -- 172.21.15.42:0/4001035811 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) ==== 74+0+24248 (secure 0 0 0) 0x7f7eac0aba10 con 0x7f7ebc104800 2024-08-30T08:58:40.280 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-08-30T08:58:40.280 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":79,"fsid":"b0895c1a-66ac-11ef-bcd4-c7b262605968","created":"2024-08-30T08:49:44.858265+0000","modified":"2024-08-30T08:58:12.223108+0000","last_up_change":"2024-08-30T08:57:12.367791+0000","last_in_change":"2024-08-30T08:56:50.274678+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-30T08:53:27.297022+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-30T08:57:58.489964+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-30T08:58:01.064783+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-08-30T08:58:07.416534+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"029d62d7-a6c6-4905-8c76-6dffa9d57b92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":2617890537},{"type":"v1","addr":"172.21.15.42:6803","nonce":2617890537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":2617890537},{"type":"v1","addr":"172.21.15.42:6805","nonce":2617890537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2617890537},{"type":"v1","addr":"172.21.15.42:6809","nonce":2617890537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":2617890537},{"type":"v1","addr":"172.21.15.42:6807","nonce":2617890537}]},"public_addr":"172.21.15.42:6803/2617890537","cluster_addr":"172.21.15.42:6805/2617890537","heartbeat_back_addr":"172.21.15.42:6809/2617890537","heartbeat_front_addr":"172.21.15.42:6807/2617890537","state":["exists","up"]},{"osd":1,"uuid":"3a78384b-0207-41df-9384-22665007d5b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":2791960082},{"type":"v1","addr":"172.21.15.167:6801","nonce":2791960082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":2791960082},{"type":"v1","addr":"172.21.15.167:6803","nonce":2791960082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":2791960082},{"type":"v1","addr":"172.21.15.167:6807","nonce":2791960082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":2791960082},{"type":"v1","addr":"172.21.15.167:6805","nonce":2791960082}]},"public_addr":"172.21.15.167:6801/2791960082","cluster_addr":"172.21.15.167:6803/2791960082","heartbeat_back_addr":"172.21.15.167:6807/2791960082","heartbeat_front_addr":"172.21.15.167:6805/2791960082","state":["exists","up"]},{"osd":2,"uuid":"16001809-c27c-4f52-948a-9998261acb4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":1101326404},{"type":"v1","addr":"172.21.15.190:6801","nonce":1101326404}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":1101326404},{"type":"v1","addr":"172.21.15.190:6803","nonce":1101326404}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":1101326404},{"type":"v1","addr":"172.21.15.190:6807","nonce":1101326404}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":1101326404},{"type":"v1","addr":"172.21.15.190:6805","nonce":1101326404}]},"public_addr":"172.21.15.190:6801/1101326404","cluster_addr":"172.21.15.190:6803/1101326404","heartbeat_back_addr":"172.21.15.190:6807/1101326404","heartbeat_front_addr":"172.21.15.190:6805/1101326404","state":["exists","up"]},{"osd":3,"uuid":"87bb3e36-9e9e-4b19-9d8c-0cdf6e089d3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":1135599916},{"type":"v1","addr":"172.21.15.42:6811","nonce":1135599916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":1135599916},{"type":"v1","addr":"172.21.15.42:6813","nonce":1135599916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":1135599916},{"type":"v1","addr":"172.21.15.42:6817","nonce":1135599916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":1135599916},{"type":"v1","addr":"172.21.15.42:6815","nonce":1135599916}]},"public_addr":"172.21.15.42:6811/1135599916","cluster_addr":"172.21.15.42:6813/1135599916","heartbeat_back_addr":"172.21.15.42:6817/1135599916","heartbeat_front_addr":"172.21.15.42:6815/1135599916","state":["exists","up"]},{"osd":4,"uuid":"3ca1aae3-c972-4fb9-ac86-236279c30d64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":2927511419},{"type":"v1","addr":"172.21.15.167:6809","nonce":2927511419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":2927511419},{"type":"v1","addr":"172.21.15.167:6811","nonce":2927511419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":2927511419},{"type":"v1","addr":"172.21.15.167:6815","nonce":2927511419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":2927511419},{"type":"v1","addr":"172.21.15.167:6813","nonce":2927511419}]},"public_addr":"172.21.15.167:6809/2927511419","cluster_addr":"172.21.15.167:6811/2927511419","heartbeat_back_addr":"172.21.15.167:6815/2927511419","heartbeat_front_addr":"172.21.15.167:6813/2927511419","state":["exists","up"]},{"osd":5,"uuid":"5d07f54b-0d07-4efe-9bcf-390755ce0258","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":2557145333},{"type":"v1","addr":"172.21.15.190:6809","nonce":2557145333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":2557145333},{"type":"v1","addr":"172.21.15.190:6811","nonce":2557145333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":2557145333},{"type":"v1","addr":"172.21.15.190:6815","nonce":2557145333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":2557145333},{"type":"v1","addr":"172.21.15.190:6813","nonce":2557145333}]},"public_addr":"172.21.15.190:6809/2557145333","cluster_addr":"172.21.15.190:6811/2557145333","heartbeat_back_addr":"172.21.15.190:6815/2557145333","heartbeat_front_addr":"172.21.15.190:6813/2557145333","state":["exists","up"]},{"osd":6,"uuid":"96ad64af-44ff-4197-97d4-156f0cb88e81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":3213526100},{"type":"v1","addr":"172.21.15.42:6819","nonce":3213526100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":3213526100},{"type":"v1","addr":"172.21.15.42:6821","nonce":3213526100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":3213526100},{"type":"v1","addr":"172.21.15.42:6825","nonce":3213526100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":3213526100},{"type":"v1","addr":"172.21.15.42:6823","nonce":3213526100}]},"public_addr":"172.21.15.42:6819/3213526100","cluster_addr":"172.21.15.42:6821/3213526100","heartbeat_back_addr":"172.21.15.42:6825/3213526100","heartbeat_front_addr":"172.21.15.42:6823/3213526100","state":["exists","up"]},{"osd":7,"uuid":"337a2112-eaf7-4e6b-86be-4e40d6ddf835","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":2962799814},{"type":"v1","addr":"172.21.15.167:6817","nonce":2962799814}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":2962799814},{"type":"v1","addr":"172.21.15.167:6819","nonce":2962799814}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":2962799814},{"type":"v1","addr":"172.21.15.167:6823","nonce":2962799814}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":2962799814},{"type":"v1","addr":"172.21.15.167:6821","nonce":2962799814}]},"public_addr":"172.21.15.167:6817/2962799814","cluster_addr":"172.21.15.167:6819/2962799814","heartbeat_back_addr":"172.21.15.167:6823/2962799814","heartbeat_front_addr":"172.21.15.167:6821/2962799814","state":["exists","up"]},{"osd":8,"uuid":"219a7224-4331-464b-ac86-c51054d51d91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":3242198843},{"type":"v1","addr":"172.21.15.190:6817","nonce":3242198843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":3242198843},{"type":"v1","addr":"172.21.15.190:6819","nonce":3242198843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":3242198843},{"type":"v1","addr":"172.21.15.190:6823","nonce":3242198843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":3242198843},{"type":"v1","addr":"172.21.15.190:6821","nonce":3242198843}]},"public_addr":"172.21.15.190:6817/3242198843","cluster_addr":"172.21.15.190:6819/3242198843","heartbeat_back_addr":"172.21.15.190:6823/3242198843","heartbeat_front_addr":"172.21.15.190:6821/3242198843","state":["exists","up"]},{"osd":9,"uuid":"2f2f5177-a009-40d9-991c-76aec8a5cba6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":1123536325},{"type":"v1","addr":"172.21.15.42:6827","nonce":1123536325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":1123536325},{"type":"v1","addr":"172.21.15.42:6829","nonce":1123536325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":1123536325},{"type":"v1","addr":"172.21.15.42:6833","nonce":1123536325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":1123536325},{"type":"v1","addr":"172.21.15.42:6831","nonce":1123536325}]},"public_addr":"172.21.15.42:6827/1123536325","cluster_addr":"172.21.15.42:6829/1123536325","heartbeat_back_addr":"172.21.15.42:6833/1123536325","heartbeat_front_addr":"172.21.15.42:6831/1123536325","state":["exists","up"]},{"osd":10,"uuid":"96ef5058-2518-4862-a165-17666b33a3e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":3451198319},{"type":"v1","addr":"172.21.15.167:6825","nonce":3451198319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":3451198319},{"type":"v1","addr":"172.21.15.167:6827","nonce":3451198319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":3451198319},{"type":"v1","addr":"172.21.15.167:6831","nonce":3451198319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":3451198319},{"type":"v1","addr":"172.21.15.167:6829","nonce":3451198319}]},"public_addr":"172.21.15.167:6825/3451198319","cluster_addr":"172.21.15.167:6827/3451198319","heartbeat_back_addr":"172.21.15.167:6831/3451198319","heartbeat_front_addr":"172.21.15.167:6829/3451198319","state":["exists","up"]},{"osd":11,"uuid":"a4eb9bd3-3faa-4c81-ba6d-c2084b72c9ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":380064457},{"type":"v1","addr":"172.21.15.190:6825","nonce":380064457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":380064457},{"type":"v1","addr":"172.21.15.190:6827","nonce":380064457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":380064457},{"type":"v1","addr":"172.21.15.190:6831","nonce":380064457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":380064457},{"type":"v1","addr":"172.21.15.190:6829","nonce":380064457}]},"public_addr":"172.21.15.190:6825/380064457","cluster_addr":"172.21.15.190:6827/380064457","heartbeat_back_addr":"172.21.15.190:6831/380064457","heartbeat_front_addr":"172.21.15.190:6829/380064457","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-30T08:52:36.511448+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-30T08:53:00.027277+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-30T08:53:23.770245+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-30T08:53:49.322226+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-30T08:54:14.350234+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-30T08:54:39.491793+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-30T08:55:04.870596+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-30T08:55:30.314150+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-30T08:55:54.483843+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-30T08:56:20.586115+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-30T08:56:45.666783+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-30T08:57:10.718387+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.42:0/2691233260":"2024-08-31T08:50:44.821070+0000","172.21.15.42:0/1518053351":"2024-08-31T08:50:44.821070+0000","172.21.15.42:0/352001020":"2024-08-31T08:50:44.821070+0000","172.21.15.42:6801/1733768634":"2024-08-31T08:50:44.821070+0000","172.21.15.42:0/1000597888":"2024-08-31T08:50:18.561851+0000","172.21.15.42:6800/1733768634":"2024-08-31T08:50:44.821070+0000","172.21.15.42:6801/3538882081":"2024-08-31T08:50:18.561851+0000","172.21.15.42:6800/3538882081":"2024-08-31T08:50:18.561851+0000","172.21.15.42:0/145366508":"2024-08-31T08:50:18.561851+0000","172.21.15.42:0/3346355213":"2024-08-31T08:50:18.561851+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-30T08:58:40.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:40.281+0000 7f7ec1b41640 1 -- 172.21.15.42:0/4001035811 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f7ea40b4b30 msgr2=0x7f7ea40b7040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:58:40.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:40.281+0000 7f7ec1b41640 1 --2- 172.21.15.42:0/4001035811 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f7ea40b4b30 0x7f7ea40b7040 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f7ebc10f960 tx=0x7f7eb0005ec0 comp rx=0 tx=0).stop 2024-08-30T08:58:40.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:40.281+0000 7f7ec1b41640 1 -- 172.21.15.42:0/4001035811 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7ebc104800 msgr2=0x7f7ebc19a290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:58:40.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:40.281+0000 7f7ec1b41640 1 --2- 172.21.15.42:0/4001035811 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7ebc104800 0x7f7ebc19a290 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f7eac00d770 tx=0x7f7eac00dc30 comp rx=0 tx=0).stop 2024-08-30T08:58:40.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:40.281+0000 7f7ec1b41640 1 -- 172.21.15.42:0/4001035811 shutdown_connections 2024-08-30T08:58:40.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:40.281+0000 7f7ec1b41640 1 --2- 172.21.15.42:0/4001035811 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f7ea40b4b30 0x7f7ea40b7040 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:40.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:40.281+0000 7f7ec1b41640 1 --2- 172.21.15.42:0/4001035811 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7ebc10dc60 0x7f7ebc19fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:40.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:40.281+0000 7f7ec1b41640 1 --2- 172.21.15.42:0/4001035811 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7ebc105150 0x7f7ebc19a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:40.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:40.281+0000 7f7ec1b41640 1 --2- 172.21.15.42:0/4001035811 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7ebc104800 0x7f7ebc19a290 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:40.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:40.281+0000 7f7ec1b41640 1 -- 172.21.15.42:0/4001035811 >> 172.21.15.42:0/4001035811 conn(0x7f7ebc079470 msgr2=0x7f7ebc0ff310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:58:40.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:40.281+0000 7f7ec1b41640 1 -- 172.21.15.42:0/4001035811 shutdown_connections 2024-08-30T08:58:40.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:40.281+0000 7f7ec1b41640 1 -- 172.21.15.42:0/4001035811 wait complete. 2024-08-30T08:58:40.590 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:58:40 smithi042 ceph-mon[18652]: pgmap v306: 129 pgs: 129 active+clean; 581 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-08-30T08:58:40.590 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:58:40 smithi042 ceph-mon[18652]: from='client.? 172.21.15.42:0/4001035811' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-30T08:58:40.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:58:40 smithi167 ceph-mon[22063]: pgmap v306: 129 pgs: 129 active+clean; 581 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-08-30T08:58:40.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:58:40 smithi167 ceph-mon[22063]: from='client.? 172.21.15.42:0/4001035811' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-30T08:58:40.837 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:58:40 smithi190 ceph-mon[22001]: pgmap v306: 129 pgs: 129 active+clean; 581 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-08-30T08:58:40.837 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:58:40 smithi190 ceph-mon[22001]: from='client.? 172.21.15.42:0/4001035811' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-30T08:58:40.912 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph fs set cephfs standby_count_wanted 0 2024-08-30T08:58:41.166 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T08:58:41.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:41.655+0000 7faab727d640 1 -- 172.21.15.42:0/238917874 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faab01036f0 msgr2=0x7faab010f2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:58:41.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:41.655+0000 7faab727d640 1 --2- 172.21.15.42:0/238917874 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faab01036f0 0x7faab010f2a0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7faaa8009ca0 tx=0x7faaa802a8c0 comp rx=0 tx=0).stop 2024-08-30T08:58:41.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:41.655+0000 7faab727d640 1 -- 172.21.15.42:0/238917874 shutdown_connections 2024-08-30T08:58:41.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:41.655+0000 7faab727d640 1 --2- 172.21.15.42:0/238917874 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faab01036f0 0x7faab010f2a0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:41.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:41.655+0000 7faab727d640 1 --2- 172.21.15.42:0/238917874 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faab0102da0 0x7faab01031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:41.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:41.655+0000 7faab727d640 1 --2- 172.21.15.42:0/238917874 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faab0107430 0x7faab0107840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:41.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:41.655+0000 7faab727d640 1 -- 172.21.15.42:0/238917874 >> 172.21.15.42:0/238917874 conn(0x7faab0079c40 msgr2=0x7faab007a050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:58:41.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:41.655+0000 7faab727d640 1 -- 172.21.15.42:0/238917874 shutdown_connections 2024-08-30T08:58:41.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:41.655+0000 7faab727d640 1 -- 172.21.15.42:0/238917874 wait complete. 2024-08-30T08:58:41.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:41.655+0000 7faab727d640 1 Processor -- start 2024-08-30T08:58:41.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:41.655+0000 7faab727d640 1 -- start start 2024-08-30T08:58:41.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:41.656+0000 7faab727d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faab0102da0 0x7faab019a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:58:41.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:41.656+0000 7faab727d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faab01036f0 0x7faab019a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:58:41.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:41.656+0000 7faab727d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faab0107430 0x7faab019fa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:58:41.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:41.656+0000 7faab727d640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7faab0113900 con 0x7faab0107430 2024-08-30T08:58:41.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:41.656+0000 7faab727d640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7faab0113780 con 0x7faab0102da0 2024-08-30T08:58:41.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:41.656+0000 7faab727d640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7faab0113a80 con 0x7faab01036f0 2024-08-30T08:58:41.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:41.656+0000 7faab581a640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faab0107430 0x7faab019fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:58:41.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:41.656+0000 7faab581a640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faab0107430 0x7faab019fa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:54540/0 (socket says 172.21.15.42:54540) 2024-08-30T08:58:41.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:41.656+0000 7faab581a640 1 -- 172.21.15.42:0/872559190 learned_addr learned my addr 172.21.15.42:0/872559190 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T08:58:41.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:41.656+0000 7faab5019640 1 --2- 172.21.15.42:0/872559190 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faab0102da0 0x7faab019a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:58:41.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:41.656+0000 7faab4818640 1 --2- 172.21.15.42:0/872559190 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faab01036f0 0x7faab019a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:58:41.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:41.656+0000 7faab581a640 1 -- 172.21.15.42:0/872559190 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faab01036f0 msgr2=0x7faab019a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:58:41.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:41.656+0000 7faab581a640 1 --2- 172.21.15.42:0/872559190 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faab01036f0 0x7faab019a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:41.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:41.656+0000 7faab581a640 1 -- 172.21.15.42:0/872559190 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faab0102da0 msgr2=0x7faab019a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:58:41.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:41.656+0000 7faab581a640 1 --2- 172.21.15.42:0/872559190 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faab0102da0 0x7faab019a1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:41.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:41.656+0000 7faab581a640 1 -- 172.21.15.42:0/872559190 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faab01a0080 con 0x7faab0107430 2024-08-30T08:58:41.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:41.656+0000 7faab4818640 1 --2- 172.21.15.42:0/872559190 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faab01036f0 0x7faab019a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-30T08:58:41.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:41.656+0000 7faab581a640 1 --2- 172.21.15.42:0/872559190 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faab0107430 0x7faab019fa10 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7faaa8005cc0 tx=0x7faaa802f470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:58:41.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:41.656+0000 7faaa67fc640 1 -- 172.21.15.42:0/872559190 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaa8003cf0 con 0x7faab0107430 2024-08-30T08:58:41.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:41.657+0000 7faab727d640 1 -- 172.21.15.42:0/872559190 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faab01a0310 con 0x7faab0107430 2024-08-30T08:58:41.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:41.657+0000 7faaa67fc640 1 -- 172.21.15.42:0/872559190 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faaa80044f0 con 0x7faab0107430 2024-08-30T08:58:41.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:41.657+0000 7faaa67fc640 1 -- 172.21.15.42:0/872559190 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaa802fc30 con 0x7faab0107430 2024-08-30T08:58:41.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:41.657+0000 7faab727d640 1 -- 172.21.15.42:0/872559190 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faab01a6b90 con 0x7faab0107430 2024-08-30T08:58:41.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:41.658+0000 7faab727d640 1 -- 172.21.15.42:0/872559190 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faab01031b0 con 0x7faab0107430 2024-08-30T08:58:41.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:41.659+0000 7faaa67fc640 1 -- 172.21.15.42:0/872559190 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) ==== 152577+0+0 (secure 0 0 0) 0x7faaa80046a0 con 0x7faab0107430 2024-08-30T08:58:41.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:41.660+0000 7faaa67fc640 1 --2- 172.21.15.42:0/872559190 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7faa9c0b4b60 0x7faa9c0b7070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:58:41.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:41.660+0000 7faab5019640 1 --2- 172.21.15.42:0/872559190 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7faa9c0b4b60 0x7faa9c0b7070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:58:41.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:41.660+0000 7faaa67fc640 1 -- 172.21.15.42:0/872559190 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8802+0+0 (secure 0 0 0) 0x7faaa8106a90 con 0x7faab0107430 2024-08-30T08:58:41.666 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:41.661+0000 7faab5019640 1 --2- 172.21.15.42:0/872559190 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7faa9c0b4b60 0x7faa9c0b7070 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7faaac00abe0 tx=0x7faaac00a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:58:41.666 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:41.663+0000 7faaa67fc640 1 -- 172.21.15.42:0/872559190 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7faaa80cd9b0 con 0x7faab0107430 2024-08-30T08:58:41.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:41.890+0000 7faab727d640 1 -- 172.21.15.42:0/872559190 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7faab019b300 con 0x7faab0107430 2024-08-30T08:58:42.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:42.519+0000 7faaa67fc640 1 -- 172.21.15.42:0/872559190 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v18) ==== 118+0+0 (secure 0 0 0) 0x7faaa80cd2b0 con 0x7faab0107430 2024-08-30T08:58:42.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:42.521+0000 7faab727d640 1 -- 172.21.15.42:0/872559190 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7faa9c0b4b60 msgr2=0x7faa9c0b7070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:58:42.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:42.521+0000 7faab727d640 1 --2- 172.21.15.42:0/872559190 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7faa9c0b4b60 0x7faa9c0b7070 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7faaac00abe0 tx=0x7faaac00a2d0 comp rx=0 tx=0).stop 2024-08-30T08:58:42.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:42.521+0000 7faab727d640 1 -- 172.21.15.42:0/872559190 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faab0107430 msgr2=0x7faab019fa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:58:42.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:42.521+0000 7faab727d640 1 --2- 172.21.15.42:0/872559190 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faab0107430 0x7faab019fa10 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7faaa8005cc0 tx=0x7faaa802f470 comp rx=0 tx=0).stop 2024-08-30T08:58:42.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:42.522+0000 7faab727d640 1 -- 172.21.15.42:0/872559190 shutdown_connections 2024-08-30T08:58:42.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:42.522+0000 7faab727d640 1 --2- 172.21.15.42:0/872559190 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7faa9c0b4b60 0x7faa9c0b7070 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:42.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:42.522+0000 7faab727d640 1 --2- 172.21.15.42:0/872559190 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faab01036f0 0x7faab019a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:42.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:42.522+0000 7faab727d640 1 --2- 172.21.15.42:0/872559190 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faab0107430 0x7faab019fa10 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:42.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:42.522+0000 7faab727d640 1 --2- 172.21.15.42:0/872559190 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faab0102da0 0x7faab019a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:42.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:42.522+0000 7faab727d640 1 -- 172.21.15.42:0/872559190 >> 172.21.15.42:0/872559190 conn(0x7faab0079c40 msgr2=0x7faab00fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:58:42.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:42.522+0000 7faab727d640 1 -- 172.21.15.42:0/872559190 shutdown_connections 2024-08-30T08:58:42.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:42.522+0000 7faab727d640 1 -- 172.21.15.42:0/872559190 wait complete. 2024-08-30T08:58:42.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:58:42 smithi167 ceph-mon[22063]: pgmap v307: 129 pgs: 129 active+clean; 581 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-08-30T08:58:42.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:58:42 smithi167 ceph-mon[22063]: from='client.? 172.21.15.42:0/872559190' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} : dispatch 2024-08-30T08:58:42.832 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:58:42 smithi042 ceph-mon[18652]: pgmap v307: 129 pgs: 129 active+clean; 581 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-08-30T08:58:42.833 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:58:42 smithi042 ceph-mon[18652]: from='client.? 172.21.15.42:0/872559190' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} : dispatch 2024-08-30T08:58:42.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:58:42 smithi190 ceph-mon[22001]: pgmap v307: 129 pgs: 129 active+clean; 581 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-08-30T08:58:42.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:58:42 smithi190 ceph-mon[22001]: from='client.? 172.21.15.42:0/872559190' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} : dispatch 2024-08-30T08:58:43.270 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 3, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2}} 2024-08-30T08:58:43.270 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph fs set cephfs max_mds 3 2024-08-30T08:58:43.533 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T08:58:43.784 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:58:43 smithi042 ceph-mon[18652]: from='client.? 172.21.15.42:0/872559190' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-08-30T08:58:43.784 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:58:43 smithi042 ceph-mon[18652]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-08-30T08:58:43.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:58:43 smithi167 ceph-mon[22063]: from='client.? 172.21.15.42:0/872559190' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-08-30T08:58:43.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:58:43 smithi167 ceph-mon[22063]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-08-30T08:58:43.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:58:43 smithi190 ceph-mon[22001]: from='client.? 172.21.15.42:0/872559190' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-08-30T08:58:43.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:58:43 smithi190 ceph-mon[22001]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-08-30T08:58:43.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:43.962+0000 7fc40c8d0640 1 -- 172.21.15.42:0/2914715114 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc408109240 msgr2=0x7fc408109650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:58:43.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:43.962+0000 7fc40c8d0640 1 --2- 172.21.15.42:0/2914715114 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc408109240 0x7fc408109650 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fc3fc009900 tx=0x7fc3fc02a410 comp rx=0 tx=0).stop 2024-08-30T08:58:43.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:43.962+0000 7fc40c8d0640 1 -- 172.21.15.42:0/2914715114 shutdown_connections 2024-08-30T08:58:43.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:43.962+0000 7fc40c8d0640 1 --2- 172.21.15.42:0/2914715114 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc408109b90 0x7fc4081103c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:43.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:43.962+0000 7fc40c8d0640 1 --2- 172.21.15.42:0/2914715114 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc408109240 0x7fc408109650 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:43.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:43.962+0000 7fc40c8d0640 1 --2- 172.21.15.42:0/2914715114 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc408103900 0x7fc408103d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:43.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:43.962+0000 7fc40c8d0640 1 -- 172.21.15.42:0/2914715114 >> 172.21.15.42:0/2914715114 conn(0x7fc40807b5f0 msgr2=0x7fc40807ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:58:43.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:43.962+0000 7fc40c8d0640 1 -- 172.21.15.42:0/2914715114 shutdown_connections 2024-08-30T08:58:43.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:43.962+0000 7fc40c8d0640 1 -- 172.21.15.42:0/2914715114 wait complete. 2024-08-30T08:58:43.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:43.963+0000 7fc40c8d0640 1 Processor -- start 2024-08-30T08:58:43.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:43.963+0000 7fc40c8d0640 1 -- start start 2024-08-30T08:58:43.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:43.964+0000 7fc40c8d0640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc408103900 0x7fc40819a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:58:43.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:43.964+0000 7fc40c8d0640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc408109240 0x7fc40819a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:58:43.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:43.964+0000 7fc40c8d0640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc408109b90 0x7fc40819fb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:58:43.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:43.964+0000 7fc40c8d0640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fc408113850 con 0x7fc408109b90 2024-08-30T08:58:43.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:43.964+0000 7fc40c8d0640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc4081136d0 con 0x7fc408109240 2024-08-30T08:58:43.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:43.964+0000 7fc40c8d0640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fc4081139d0 con 0x7fc408103900 2024-08-30T08:58:43.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:43.964+0000 7fc406d9d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc408109b90 0x7fc40819fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:58:43.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:43.964+0000 7fc406d9d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc408109b90 0x7fc40819fb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:54560/0 (socket says 172.21.15.42:54560) 2024-08-30T08:58:43.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:43.964+0000 7fc406d9d640 1 -- 172.21.15.42:0/1093164666 learned_addr learned my addr 172.21.15.42:0/1093164666 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T08:58:43.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:43.964+0000 7fc40659c640 1 --2- 172.21.15.42:0/1093164666 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc408103900 0x7fc40819a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:58:43.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:43.964+0000 7fc405d9b640 1 --2- 172.21.15.42:0/1093164666 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc408109240 0x7fc40819a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:58:43.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:43.964+0000 7fc406d9d640 1 -- 172.21.15.42:0/1093164666 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc408103900 msgr2=0x7fc40819a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:58:43.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:43.964+0000 7fc406d9d640 1 --2- 172.21.15.42:0/1093164666 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc408103900 0x7fc40819a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:43.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:43.964+0000 7fc406d9d640 1 -- 172.21.15.42:0/1093164666 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc408109240 msgr2=0x7fc40819a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:58:43.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:43.964+0000 7fc406d9d640 1 --2- 172.21.15.42:0/1093164666 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc408109240 0x7fc40819a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:43.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:43.964+0000 7fc406d9d640 1 -- 172.21.15.42:0/1093164666 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4081a01b0 con 0x7fc408109b90 2024-08-30T08:58:43.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:43.964+0000 7fc406d9d640 1 --2- 172.21.15.42:0/1093164666 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc408109b90 0x7fc40819fb40 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fc40000eaa0 tx=0x7fc40000eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:58:43.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:43.965+0000 7fc3f77fe640 1 -- 172.21.15.42:0/1093164666 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc400004d20 con 0x7fc408109b90 2024-08-30T08:58:43.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:43.965+0000 7fc3f77fe640 1 -- 172.21.15.42:0/1093164666 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc400005020 con 0x7fc408109b90 2024-08-30T08:58:43.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:43.965+0000 7fc40c8d0640 1 -- 172.21.15.42:0/1093164666 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4081a04a0 con 0x7fc408109b90 2024-08-30T08:58:43.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:43.965+0000 7fc3f77fe640 1 -- 172.21.15.42:0/1093164666 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4000116b0 con 0x7fc408109b90 2024-08-30T08:58:43.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:43.965+0000 7fc40c8d0640 1 -- 172.21.15.42:0/1093164666 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4081a0960 con 0x7fc408109b90 2024-08-30T08:58:43.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:43.966+0000 7fc40c8d0640 1 -- 172.21.15.42:0/1093164666 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc408103d10 con 0x7fc408109b90 2024-08-30T08:58:43.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:43.967+0000 7fc3f77fe640 1 -- 172.21.15.42:0/1093164666 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) ==== 152577+0+0 (secure 0 0 0) 0x7fc4000051d0 con 0x7fc408109b90 2024-08-30T08:58:43.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:43.968+0000 7fc3f77fe640 1 --2- 172.21.15.42:0/1093164666 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fc3f00b4b60 0x7fc3f00b7070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:58:43.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:43.969+0000 7fc40659c640 1 --2- 172.21.15.42:0/1093164666 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fc3f00b4b60 0x7fc3f00b7070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:58:43.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:43.969+0000 7fc3f77fe640 1 -- 172.21.15.42:0/1093164666 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8802+0+0 (secure 0 0 0) 0x7fc4000e5880 con 0x7fc408109b90 2024-08-30T08:58:43.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:43.969+0000 7fc40659c640 1 --2- 172.21.15.42:0/1093164666 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fc3f00b4b60 0x7fc3f00b7070 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fc3f800abe0 tx=0x7fc3f800a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:58:43.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:43.972+0000 7fc3f77fe640 1 -- 172.21.15.42:0/1093164666 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7fc400016030 con 0x7fc408109b90 2024-08-30T08:58:44.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:44.200+0000 7fc40c8d0640 1 -- 172.21.15.42:0/1093164666 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) -- 0x7fc40802e430 con 0x7fc408109b90 2024-08-30T08:58:44.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:44.536+0000 7fc3f77fe640 1 -- 172.21.15.42:0/1093164666 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v19) ==== 105+0+0 (secure 0 0 0) 0x7fc4000ac710 con 0x7fc408109b90 2024-08-30T08:58:44.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:44.538+0000 7fc40c8d0640 1 -- 172.21.15.42:0/1093164666 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fc3f00b4b60 msgr2=0x7fc3f00b7070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:58:44.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:44.538+0000 7fc40c8d0640 1 --2- 172.21.15.42:0/1093164666 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fc3f00b4b60 0x7fc3f00b7070 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fc3f800abe0 tx=0x7fc3f800a2d0 comp rx=0 tx=0).stop 2024-08-30T08:58:44.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:44.538+0000 7fc40c8d0640 1 -- 172.21.15.42:0/1093164666 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc408109b90 msgr2=0x7fc40819fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:58:44.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:44.538+0000 7fc40c8d0640 1 --2- 172.21.15.42:0/1093164666 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc408109b90 0x7fc40819fb40 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fc40000eaa0 tx=0x7fc40000eea0 comp rx=0 tx=0).stop 2024-08-30T08:58:44.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:44.538+0000 7fc40c8d0640 1 -- 172.21.15.42:0/1093164666 shutdown_connections 2024-08-30T08:58:44.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:44.539+0000 7fc40c8d0640 1 --2- 172.21.15.42:0/1093164666 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fc3f00b4b60 0x7fc3f00b7070 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:44.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:44.539+0000 7fc40c8d0640 1 --2- 172.21.15.42:0/1093164666 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc408109b90 0x7fc40819fb40 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:44.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:44.539+0000 7fc40c8d0640 1 --2- 172.21.15.42:0/1093164666 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc408109240 0x7fc40819a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:44.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:44.539+0000 7fc40c8d0640 1 --2- 172.21.15.42:0/1093164666 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc408103900 0x7fc40819a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:44.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:44.539+0000 7fc40c8d0640 1 -- 172.21.15.42:0/1093164666 >> 172.21.15.42:0/1093164666 conn(0x7fc40807b5f0 msgr2=0x7fc408107390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:58:44.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:44.539+0000 7fc40c8d0640 1 -- 172.21.15.42:0/1093164666 shutdown_connections 2024-08-30T08:58:44.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:44.539+0000 7fc40c8d0640 1 -- 172.21.15.42:0/1093164666 wait complete. 2024-08-30T08:58:44.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:58:44 smithi167 ceph-mon[22063]: pgmap v308: 129 pgs: 129 active+clean; 581 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-08-30T08:58:44.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:58:44 smithi167 ceph-mon[22063]: from='client.? 172.21.15.42:0/1093164666' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} : dispatch 2024-08-30T08:58:44.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:58:44 smithi042 ceph-mon[18652]: pgmap v308: 129 pgs: 129 active+clean; 581 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-08-30T08:58:44.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:58:44 smithi042 ceph-mon[18652]: from='client.? 172.21.15.42:0/1093164666' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} : dispatch 2024-08-30T08:58:44.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:58:44 smithi190 ceph-mon[22001]: pgmap v308: 129 pgs: 129 active+clean; 581 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-08-30T08:58:44.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:58:44 smithi190 ceph-mon[22001]: from='client.? 172.21.15.42:0/1093164666' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} : dispatch 2024-08-30T08:58:45.375 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph fs set cephfs allow_standby_replay true 2024-08-30T08:58:45.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:58:45 smithi042 ceph-mon[18652]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-08-30T08:58:45.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:58:45 smithi042 ceph-mon[18652]: from='client.? 172.21.15.42:0/1093164666' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-08-30T08:58:45.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:58:45 smithi042 ceph-mon[18652]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-08-30T08:58:45.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:58:45 smithi042 ceph-mon[18652]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-08-30T08:58:45.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:58:45 smithi042 ceph-mon[18652]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-08-30T08:58:45.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:58:45 smithi042 ceph-mon[18652]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-08-30T08:58:45.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:58:45 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]} : dispatch 2024-08-30T08:58:45.634 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:58:45 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]} : dispatch 2024-08-30T08:58:45.634 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:58:45 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]} : dispatch 2024-08-30T08:58:45.634 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:58:45 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 6]} : dispatch 2024-08-30T08:58:45.634 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:58:45 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]} : dispatch 2024-08-30T08:58:45.634 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:58:45 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]} : dispatch 2024-08-30T08:58:45.634 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:58:45 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.9", "id": [5, 8]} : dispatch 2024-08-30T08:58:45.634 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:58:45 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 8]} : dispatch 2024-08-30T08:58:45.634 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:58:45 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 8]} : dispatch 2024-08-30T08:58:45.634 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:58:45 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [9, 10]} : dispatch 2024-08-30T08:58:45.641 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T08:58:45.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:58:45 smithi167 ceph-mon[22063]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-08-30T08:58:45.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:58:45 smithi167 ceph-mon[22063]: from='client.? 172.21.15.42:0/1093164666' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-08-30T08:58:45.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:58:45 smithi167 ceph-mon[22063]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-08-30T08:58:45.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:58:45 smithi167 ceph-mon[22063]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-08-30T08:58:45.828 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:58:45 smithi167 ceph-mon[22063]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-08-30T08:58:45.828 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:58:45 smithi167 ceph-mon[22063]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-08-30T08:58:45.828 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:58:45 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]} : dispatch 2024-08-30T08:58:45.828 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:58:45 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]} : dispatch 2024-08-30T08:58:45.828 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:58:45 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]} : dispatch 2024-08-30T08:58:45.828 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:58:45 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 6]} : dispatch 2024-08-30T08:58:45.828 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:58:45 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]} : dispatch 2024-08-30T08:58:45.828 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:58:45 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]} : dispatch 2024-08-30T08:58:45.828 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:58:45 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.9", "id": [5, 8]} : dispatch 2024-08-30T08:58:45.828 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:58:45 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 8]} : dispatch 2024-08-30T08:58:45.828 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:58:45 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 8]} : dispatch 2024-08-30T08:58:45.828 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:58:45 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [9, 10]} : dispatch 2024-08-30T08:58:45.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:58:45 smithi190 ceph-mon[22001]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-08-30T08:58:45.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:58:45 smithi190 ceph-mon[22001]: from='client.? 172.21.15.42:0/1093164666' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-08-30T08:58:45.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:58:45 smithi190 ceph-mon[22001]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-08-30T08:58:45.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:58:45 smithi190 ceph-mon[22001]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-08-30T08:58:45.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:58:45 smithi190 ceph-mon[22001]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-08-30T08:58:45.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:58:45 smithi190 ceph-mon[22001]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-08-30T08:58:45.937 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:58:45 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]} : dispatch 2024-08-30T08:58:45.937 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:58:45 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]} : dispatch 2024-08-30T08:58:45.937 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:58:45 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]} : dispatch 2024-08-30T08:58:45.937 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:58:45 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 6]} : dispatch 2024-08-30T08:58:45.937 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:58:45 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]} : dispatch 2024-08-30T08:58:45.937 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:58:45 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]} : dispatch 2024-08-30T08:58:45.937 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:58:45 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.9", "id": [5, 8]} : dispatch 2024-08-30T08:58:45.937 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:58:45 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 8]} : dispatch 2024-08-30T08:58:45.937 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:58:45 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 8]} : dispatch 2024-08-30T08:58:45.937 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:58:45 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [9, 10]} : dispatch 2024-08-30T08:58:46.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:46.147+0000 7f9198c53640 1 -- 172.21.15.42:0/4151238879 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9194102fd0 msgr2=0x7f91941033e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:58:46.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:46.147+0000 7f9198c53640 1 --2- 172.21.15.42:0/4151238879 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9194102fd0 0x7f91941033e0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f9184009900 tx=0x7f918402a410 comp rx=0 tx=0).stop 2024-08-30T08:58:46.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:46.148+0000 7f9198c53640 1 -- 172.21.15.42:0/4151238879 shutdown_connections 2024-08-30T08:58:46.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:46.148+0000 7f9198c53640 1 --2- 172.21.15.42:0/4151238879 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9194108d60 0x7f919410f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:46.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:46.148+0000 7f9198c53640 1 --2- 172.21.15.42:0/4151238879 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9194108410 0x7f9194108820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:46.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:46.148+0000 7f9198c53640 1 --2- 172.21.15.42:0/4151238879 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9194102fd0 0x7f91941033e0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:46.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:46.148+0000 7f9198c53640 1 -- 172.21.15.42:0/4151238879 >> 172.21.15.42:0/4151238879 conn(0x7f9194079c40 msgr2=0x7f919407a050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:58:46.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:46.148+0000 7f9198c53640 1 -- 172.21.15.42:0/4151238879 shutdown_connections 2024-08-30T08:58:46.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:46.148+0000 7f9198c53640 1 -- 172.21.15.42:0/4151238879 wait complete. 2024-08-30T08:58:46.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:46.148+0000 7f9198c53640 1 Processor -- start 2024-08-30T08:58:46.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:46.149+0000 7f9198c53640 1 -- start start 2024-08-30T08:58:46.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:46.149+0000 7f9198c53640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9194102fd0 0x7f919419b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:58:46.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:46.149+0000 7f9198c53640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9194108410 0x7f919419b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:58:46.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:46.149+0000 7f9198c53640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9194108d60 0x7f919419bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:58:46.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:46.149+0000 7f9198c53640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f9194112a20 con 0x7f9194108d60 2024-08-30T08:58:46.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:46.149+0000 7f9198c53640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f91941128a0 con 0x7f9194108410 2024-08-30T08:58:46.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:46.149+0000 7f9198c53640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f9194112ba0 con 0x7f9194102fd0 2024-08-30T08:58:46.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:46.149+0000 7f9192d9d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9194108d60 0x7f919419bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:58:46.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:46.149+0000 7f9192d9d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9194108d60 0x7f919419bd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:54588/0 (socket says 172.21.15.42:54588) 2024-08-30T08:58:46.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:46.149+0000 7f9192d9d640 1 -- 172.21.15.42:0/1257691844 learned_addr learned my addr 172.21.15.42:0/1257691844 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T08:58:46.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:46.149+0000 7f919259c640 1 --2- 172.21.15.42:0/1257691844 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9194102fd0 0x7f919419b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:58:46.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:46.149+0000 7f9191d9b640 1 --2- 172.21.15.42:0/1257691844 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9194108410 0x7f919419b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:58:46.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:46.149+0000 7f9192d9d640 1 -- 172.21.15.42:0/1257691844 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9194102fd0 msgr2=0x7f919419b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:58:46.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:46.149+0000 7f9192d9d640 1 --2- 172.21.15.42:0/1257691844 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9194102fd0 0x7f919419b2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:46.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:46.149+0000 7f9192d9d640 1 -- 172.21.15.42:0/1257691844 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9194108410 msgr2=0x7f919419b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:58:46.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:46.149+0000 7f9192d9d640 1 --2- 172.21.15.42:0/1257691844 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9194108410 0x7f919419b810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:46.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:46.149+0000 7f9192d9d640 1 -- 172.21.15.42:0/1257691844 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91941a6560 con 0x7f9194108d60 2024-08-30T08:58:46.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:46.149+0000 7f9192d9d640 1 --2- 172.21.15.42:0/1257691844 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9194108d60 0x7f919419bd50 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f918c00eaa0 tx=0x7f918c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:58:46.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:46.150+0000 7f918b7fe640 1 -- 172.21.15.42:0/1257691844 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f918c00cdb0 con 0x7f9194108d60 2024-08-30T08:58:46.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:46.150+0000 7f9198c53640 1 -- 172.21.15.42:0/1257691844 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91941a6850 con 0x7f9194108d60 2024-08-30T08:58:46.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:46.150+0000 7f918b7fe640 1 -- 172.21.15.42:0/1257691844 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f918c005020 con 0x7f9194108d60 2024-08-30T08:58:46.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:46.150+0000 7f918b7fe640 1 -- 172.21.15.42:0/1257691844 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f918c0116d0 con 0x7f9194108d60 2024-08-30T08:58:46.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:46.150+0000 7f9198c53640 1 -- 172.21.15.42:0/1257691844 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91941a6d90 con 0x7f9194108d60 2024-08-30T08:58:46.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:46.151+0000 7f9198c53640 1 -- 172.21.15.42:0/1257691844 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91941033e0 con 0x7f9194108d60 2024-08-30T08:58:46.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:46.152+0000 7f918b7fe640 1 -- 172.21.15.42:0/1257691844 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) ==== 152577+0+0 (secure 0 0 0) 0x7f918c0051d0 con 0x7f9194108d60 2024-08-30T08:58:46.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:46.154+0000 7f918b7fe640 1 --2- 172.21.15.42:0/1257691844 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f91800b4b60 0x7f91800b7070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:58:46.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:46.154+0000 7f919259c640 1 --2- 172.21.15.42:0/1257691844 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f91800b4b60 0x7f91800b7070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:58:46.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:46.154+0000 7f918b7fe640 1 -- 172.21.15.42:0/1257691844 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 9092+0+0 (secure 0 0 0) 0x7f918c0e5aa0 con 0x7f9194108d60 2024-08-30T08:58:46.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:46.154+0000 7f919259c640 1 --2- 172.21.15.42:0/1257691844 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f91800b4b60 0x7f91800b7070 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f9184000c00 tx=0x7f9184031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:58:46.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:46.157+0000 7f918b7fe640 1 -- 172.21.15.42:0/1257691844 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f918c016030 con 0x7f9194108d60 2024-08-30T08:58:46.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:46.383+0000 7f9198c53640 1 -- 172.21.15.42:0/1257691844 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7f919410ab30 con 0x7f9194108d60 2024-08-30T08:58:46.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:46.565+0000 7f918b7fe640 1 -- 172.21.15.42:0/1257691844 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v23) ==== 121+0+0 (secure 0 0 0) 0x7f918c0ac810 con 0x7f9194108d60 2024-08-30T08:58:46.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:46.567+0000 7f9198c53640 1 -- 172.21.15.42:0/1257691844 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f91800b4b60 msgr2=0x7f91800b7070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:58:46.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:46.567+0000 7f9198c53640 1 --2- 172.21.15.42:0/1257691844 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f91800b4b60 0x7f91800b7070 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f9184000c00 tx=0x7f9184031400 comp rx=0 tx=0).stop 2024-08-30T08:58:46.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:46.567+0000 7f9198c53640 1 -- 172.21.15.42:0/1257691844 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9194108d60 msgr2=0x7f919419bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:58:46.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:46.567+0000 7f9198c53640 1 --2- 172.21.15.42:0/1257691844 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9194108d60 0x7f919419bd50 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f918c00eaa0 tx=0x7f918c00ef60 comp rx=0 tx=0).stop 2024-08-30T08:58:46.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:46.567+0000 7f9198c53640 1 -- 172.21.15.42:0/1257691844 shutdown_connections 2024-08-30T08:58:46.570 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:46.567+0000 7f9198c53640 1 --2- 172.21.15.42:0/1257691844 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f91800b4b60 0x7f91800b7070 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:46.570 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:46.567+0000 7f9198c53640 1 --2- 172.21.15.42:0/1257691844 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9194108d60 0x7f919419bd50 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:46.570 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:46.567+0000 7f9198c53640 1 --2- 172.21.15.42:0/1257691844 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9194108410 0x7f919419b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:46.570 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:46.567+0000 7f9198c53640 1 --2- 172.21.15.42:0/1257691844 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9194102fd0 0x7f919419b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:46.570 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:46.567+0000 7f9198c53640 1 -- 172.21.15.42:0/1257691844 >> 172.21.15.42:0/1257691844 conn(0x7f9194079c40 msgr2=0x7f91941064a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:58:46.570 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:46.567+0000 7f9198c53640 1 -- 172.21.15.42:0/1257691844 shutdown_connections 2024-08-30T08:58:46.570 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:46.567+0000 7f9198c53640 1 -- 172.21.15.42:0/1257691844 wait complete. 2024-08-30T08:58:46.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:58:46 smithi167 ceph-mon[22063]: pgmap v309: 129 pgs: 129 active+clean; 581 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-30T08:58:46.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:58:46 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]': finished 2024-08-30T08:58:46.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:58:46 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]': finished 2024-08-30T08:58:46.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:58:46 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]': finished 2024-08-30T08:58:46.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:58:46 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 6]}]': finished 2024-08-30T08:58:46.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:58:46 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]': finished 2024-08-30T08:58:46.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:58:46 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]': finished 2024-08-30T08:58:46.828 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:58:46 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.9", "id": [5, 8]}]': finished 2024-08-30T08:58:46.828 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:58:46 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 8]}]': finished 2024-08-30T08:58:46.828 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:58:46 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 8]}]': finished 2024-08-30T08:58:46.828 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:58:46 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [9, 10]}]': finished 2024-08-30T08:58:46.828 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:58:46 smithi167 ceph-mon[22063]: osdmap e80: 12 total, 12 up, 12 in 2024-08-30T08:58:46.828 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:58:46 smithi167 ceph-mon[22063]: mds.? [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] up:active 2024-08-30T08:58:46.828 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:58:46 smithi167 ceph-mon[22063]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-08-30T08:58:46.828 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:58:46 smithi167 ceph-mon[22063]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-08-30T08:58:46.828 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:58:46 smithi167 ceph-mon[22063]: Cluster is now healthy 2024-08-30T08:58:46.828 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:58:46 smithi167 ceph-mon[22063]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-08-30T08:58:46.828 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:58:46 smithi167 ceph-mon[22063]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-08-30T08:58:46.828 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:58:46 smithi167 ceph-mon[22063]: from='client.? 172.21.15.42:0/1257691844' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} : dispatch 2024-08-30T08:58:46.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:58:46 smithi042 ceph-mon[18652]: pgmap v309: 129 pgs: 129 active+clean; 581 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-30T08:58:46.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:58:46 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]': finished 2024-08-30T08:58:46.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:58:46 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]': finished 2024-08-30T08:58:46.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:58:46 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]': finished 2024-08-30T08:58:46.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:58:46 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 6]}]': finished 2024-08-30T08:58:46.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:58:46 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]': finished 2024-08-30T08:58:46.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:58:46 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]': finished 2024-08-30T08:58:46.884 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:58:46 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.9", "id": [5, 8]}]': finished 2024-08-30T08:58:46.884 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:58:46 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 8]}]': finished 2024-08-30T08:58:46.884 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:58:46 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 8]}]': finished 2024-08-30T08:58:46.884 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:58:46 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [9, 10]}]': finished 2024-08-30T08:58:46.884 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:58:46 smithi042 ceph-mon[18652]: osdmap e80: 12 total, 12 up, 12 in 2024-08-30T08:58:46.884 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:58:46 smithi042 ceph-mon[18652]: mds.? [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] up:active 2024-08-30T08:58:46.884 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:58:46 smithi042 ceph-mon[18652]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-08-30T08:58:46.884 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:58:46 smithi042 ceph-mon[18652]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-08-30T08:58:46.884 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:58:46 smithi042 ceph-mon[18652]: Cluster is now healthy 2024-08-30T08:58:46.884 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:58:46 smithi042 ceph-mon[18652]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-08-30T08:58:46.884 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:58:46 smithi042 ceph-mon[18652]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-08-30T08:58:46.884 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:58:46 smithi042 ceph-mon[18652]: from='client.? 172.21.15.42:0/1257691844' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} : dispatch 2024-08-30T08:58:46.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:58:46 smithi190 ceph-mon[22001]: pgmap v309: 129 pgs: 129 active+clean; 581 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-30T08:58:46.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:58:46 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]': finished 2024-08-30T08:58:46.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:58:46 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]': finished 2024-08-30T08:58:46.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:58:46 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]': finished 2024-08-30T08:58:46.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:58:46 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 6]}]': finished 2024-08-30T08:58:46.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:58:46 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]': finished 2024-08-30T08:58:46.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:58:46 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]': finished 2024-08-30T08:58:46.937 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:58:46 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.9", "id": [5, 8]}]': finished 2024-08-30T08:58:46.937 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:58:46 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 8]}]': finished 2024-08-30T08:58:46.937 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:58:46 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 8]}]': finished 2024-08-30T08:58:46.937 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:58:46 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [9, 10]}]': finished 2024-08-30T08:58:46.937 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:58:46 smithi190 ceph-mon[22001]: osdmap e80: 12 total, 12 up, 12 in 2024-08-30T08:58:46.937 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:58:46 smithi190 ceph-mon[22001]: mds.? [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] up:active 2024-08-30T08:58:46.937 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:58:46 smithi190 ceph-mon[22001]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-08-30T08:58:46.937 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:58:46 smithi190 ceph-mon[22001]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-08-30T08:58:46.937 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:58:46 smithi190 ceph-mon[22001]: Cluster is now healthy 2024-08-30T08:58:46.937 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:58:46 smithi190 ceph-mon[22001]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-08-30T08:58:46.937 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:58:46 smithi190 ceph-mon[22001]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-08-30T08:58:46.937 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:58:46 smithi190 ceph-mon[22001]: from='client.? 172.21.15.42:0/1257691844' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} : dispatch 2024-08-30T08:58:47.518 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph fs set cephfs session_timeout 300 2024-08-30T08:58:47.772 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T08:58:47.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:58:47 smithi167 ceph-mon[22063]: osdmap e81: 12 total, 12 up, 12 in 2024-08-30T08:58:47.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:58:47 smithi167 ceph-mon[22063]: from='client.? 172.21.15.42:0/1257691844' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-08-30T08:58:47.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:58:47 smithi167 ceph-mon[22063]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-08-30T08:58:47.882 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:58:47 smithi042 ceph-mon[18652]: osdmap e81: 12 total, 12 up, 12 in 2024-08-30T08:58:47.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:58:47 smithi042 ceph-mon[18652]: from='client.? 172.21.15.42:0/1257691844' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-08-30T08:58:47.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:58:47 smithi042 ceph-mon[18652]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-08-30T08:58:47.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:58:47 smithi190 ceph-mon[22001]: osdmap e81: 12 total, 12 up, 12 in 2024-08-30T08:58:47.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:58:47 smithi190 ceph-mon[22001]: from='client.? 172.21.15.42:0/1257691844' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-08-30T08:58:47.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:58:47 smithi190 ceph-mon[22001]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-08-30T08:58:48.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:48.241+0000 7f5f1cdec640 1 -- 172.21.15.42:0/2000982234 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5f18107980 msgr2=0x7f5f18107d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:58:48.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:48.241+0000 7f5f1cdec640 1 --2- 172.21.15.42:0/2000982234 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5f18107980 0x7f5f18107d90 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f5f00009900 tx=0x7f5f0002a410 comp rx=0 tx=0).stop 2024-08-30T08:58:48.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:48.241+0000 7f5f1cdec640 1 -- 172.21.15.42:0/2000982234 shutdown_connections 2024-08-30T08:58:48.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:48.241+0000 7f5f1cdec640 1 --2- 172.21.15.42:0/2000982234 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5f181082d0 0x7f5f1810eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:48.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:48.241+0000 7f5f1cdec640 1 --2- 172.21.15.42:0/2000982234 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5f18107980 0x7f5f18107d90 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:48.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:48.241+0000 7f5f1cdec640 1 --2- 172.21.15.42:0/2000982234 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5f18102040 0x7f5f18102450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:48.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:48.241+0000 7f5f1cdec640 1 -- 172.21.15.42:0/2000982234 >> 172.21.15.42:0/2000982234 conn(0x7f5f18079c60 msgr2=0x7f5f1807a070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:58:48.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:48.241+0000 7f5f1cdec640 1 -- 172.21.15.42:0/2000982234 shutdown_connections 2024-08-30T08:58:48.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:48.241+0000 7f5f1cdec640 1 -- 172.21.15.42:0/2000982234 wait complete. 2024-08-30T08:58:48.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:48.242+0000 7f5f1cdec640 1 Processor -- start 2024-08-30T08:58:48.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:48.242+0000 7f5f1cdec640 1 -- start start 2024-08-30T08:58:48.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:48.242+0000 7f5f1cdec640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5f18102040 0x7f5f1819a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:58:48.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:48.242+0000 7f5f1cdec640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5f18107980 0x7f5f1819a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:58:48.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:48.242+0000 7f5f1cdec640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5f181082d0 0x7f5f1819fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:58:48.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:48.242+0000 7f5f1cdec640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f5f18111710 con 0x7f5f18107980 2024-08-30T08:58:48.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:48.242+0000 7f5f1cdec640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f5f18111590 con 0x7f5f18102040 2024-08-30T08:58:48.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:48.242+0000 7f5f1cdec640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f5f18111890 con 0x7f5f181082d0 2024-08-30T08:58:48.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:48.242+0000 7f5f15d9b640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5f18107980 0x7f5f1819a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:58:48.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:48.242+0000 7f5f15d9b640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5f18107980 0x7f5f1819a880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43218/0 (socket says 172.21.15.42:43218) 2024-08-30T08:58:48.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:48.242+0000 7f5f15d9b640 1 -- 172.21.15.42:0/1551357923 learned_addr learned my addr 172.21.15.42:0/1551357923 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T08:58:48.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:48.242+0000 7f5f1659c640 1 --2- 172.21.15.42:0/1551357923 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5f18102040 0x7f5f1819a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:58:48.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:48.242+0000 7f5f16d9d640 1 --2- 172.21.15.42:0/1551357923 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5f181082d0 0x7f5f1819fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:58:48.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:48.243+0000 7f5f15d9b640 1 -- 172.21.15.42:0/1551357923 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5f181082d0 msgr2=0x7f5f1819fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:58:48.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:48.243+0000 7f5f15d9b640 1 --2- 172.21.15.42:0/1551357923 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5f181082d0 0x7f5f1819fb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:48.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:48.243+0000 7f5f15d9b640 1 -- 172.21.15.42:0/1551357923 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5f18102040 msgr2=0x7f5f1819a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:58:48.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:48.243+0000 7f5f15d9b640 1 --2- 172.21.15.42:0/1551357923 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5f18102040 0x7f5f1819a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:48.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:48.243+0000 7f5f15d9b640 1 -- 172.21.15.42:0/1551357923 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f181a0190 con 0x7f5f18107980 2024-08-30T08:58:48.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:48.243+0000 7f5f16d9d640 1 --2- 172.21.15.42:0/1551357923 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5f181082d0 0x7f5f1819fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-30T08:58:48.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:48.243+0000 7f5f15d9b640 1 --2- 172.21.15.42:0/1551357923 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5f18107980 0x7f5f1819a880 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f5f0002a930 tx=0x7f5f00004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:58:48.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:48.243+0000 7f5f0f7fe640 1 -- 172.21.15.42:0/1551357923 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f0003f030 con 0x7f5f18107980 2024-08-30T08:58:48.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:48.243+0000 7f5f0f7fe640 1 -- 172.21.15.42:0/1551357923 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f00003aa0 con 0x7f5f18107980 2024-08-30T08:58:48.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:48.243+0000 7f5f1cdec640 1 -- 172.21.15.42:0/1551357923 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f181a0420 con 0x7f5f18107980 2024-08-30T08:58:48.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:48.243+0000 7f5f0f7fe640 1 -- 172.21.15.42:0/1551357923 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f00035070 con 0x7f5f18107980 2024-08-30T08:58:48.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:48.243+0000 7f5f1cdec640 1 -- 172.21.15.42:0/1551357923 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f181a0900 con 0x7f5f18107980 2024-08-30T08:58:48.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:48.244+0000 7f5f1cdec640 1 -- 172.21.15.42:0/1551357923 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f18102450 con 0x7f5f18107980 2024-08-30T08:58:48.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:48.245+0000 7f5f0f7fe640 1 -- 172.21.15.42:0/1551357923 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) ==== 152577+0+0 (secure 0 0 0) 0x7f5f00042030 con 0x7f5f18107980 2024-08-30T08:58:48.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:48.246+0000 7f5f0f7fe640 1 --2- 172.21.15.42:0/1551357923 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f5f040b4b60 0x7f5f040b7070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:58:48.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:48.246+0000 7f5f0f7fe640 1 -- 172.21.15.42:0/1551357923 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 9092+0+0 (secure 0 0 0) 0x7f5f00105f80 con 0x7f5f18107980 2024-08-30T08:58:48.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:48.246+0000 7f5f1659c640 1 --2- 172.21.15.42:0/1551357923 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f5f040b4b60 0x7f5f040b7070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:58:48.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:48.247+0000 7f5f1659c640 1 --2- 172.21.15.42:0/1551357923 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f5f040b4b60 0x7f5f040b7070 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f5f0800abe0 tx=0x7f5f0800a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:58:48.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:48.250+0000 7f5f0f7fe640 1 -- 172.21.15.42:0/1551357923 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f5f000ccd50 con 0x7f5f18107980 2024-08-30T08:58:48.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:48.477+0000 7f5f1cdec640 1 -- 172.21.15.42:0/1551357923 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7f5f1810c0d0 con 0x7f5f18107980 2024-08-30T08:58:48.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:48.567+0000 7f5f0f7fe640 1 -- 172.21.15.42:0/1551357923 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v24) ==== 115+0+0 (secure 0 0 0) 0x7f5f000cc650 con 0x7f5f18107980 2024-08-30T08:58:48.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:48.569+0000 7f5f1cdec640 1 -- 172.21.15.42:0/1551357923 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f5f040b4b60 msgr2=0x7f5f040b7070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:58:48.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:48.569+0000 7f5f1cdec640 1 --2- 172.21.15.42:0/1551357923 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f5f040b4b60 0x7f5f040b7070 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f5f0800abe0 tx=0x7f5f0800a2d0 comp rx=0 tx=0).stop 2024-08-30T08:58:48.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:48.569+0000 7f5f1cdec640 1 -- 172.21.15.42:0/1551357923 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5f18107980 msgr2=0x7f5f1819a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:58:48.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:48.569+0000 7f5f1cdec640 1 --2- 172.21.15.42:0/1551357923 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5f18107980 0x7f5f1819a880 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f5f0002a930 tx=0x7f5f00004270 comp rx=0 tx=0).stop 2024-08-30T08:58:48.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:48.569+0000 7f5f1cdec640 1 -- 172.21.15.42:0/1551357923 shutdown_connections 2024-08-30T08:58:48.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:48.569+0000 7f5f1cdec640 1 --2- 172.21.15.42:0/1551357923 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f5f040b4b60 0x7f5f040b7070 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:48.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:48.569+0000 7f5f1cdec640 1 --2- 172.21.15.42:0/1551357923 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5f181082d0 0x7f5f1819fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:48.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:48.569+0000 7f5f1cdec640 1 --2- 172.21.15.42:0/1551357923 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5f18107980 0x7f5f1819a880 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:48.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:48.569+0000 7f5f1cdec640 1 --2- 172.21.15.42:0/1551357923 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5f18102040 0x7f5f1819a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:48.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:48.569+0000 7f5f1cdec640 1 -- 172.21.15.42:0/1551357923 >> 172.21.15.42:0/1551357923 conn(0x7f5f18079c60 msgr2=0x7f5f18105c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:58:48.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:48.569+0000 7f5f1cdec640 1 -- 172.21.15.42:0/1551357923 shutdown_connections 2024-08-30T08:58:48.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:48.569+0000 7f5f1cdec640 1 -- 172.21.15.42:0/1551357923 wait complete. 2024-08-30T08:58:48.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:58:48 smithi167 ceph-mon[22063]: pgmap v312: 129 pgs: 2 activating, 127 active+clean; 582 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s wr, 3 op/s 2024-08-30T08:58:48.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:58:48 smithi167 ceph-mon[22063]: from='client.? 172.21.15.42:0/1551357923' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} : dispatch 2024-08-30T08:58:48.827 INFO:journalctl@ceph.mds.b.smithi167.stdout:Aug 30 08:58:48 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mds-b[52473]: 2024-08-30T08:58:48.612+0000 7f9d3d6df640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-08-30T08:58:48.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:58:48 smithi042 ceph-mon[18652]: pgmap v312: 129 pgs: 2 activating, 127 active+clean; 582 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s wr, 3 op/s 2024-08-30T08:58:48.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:58:48 smithi042 ceph-mon[18652]: from='client.? 172.21.15.42:0/1551357923' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} : dispatch 2024-08-30T08:58:48.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:58:48 smithi190 ceph-mon[22001]: pgmap v312: 129 pgs: 2 activating, 127 active+clean; 582 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s wr, 3 op/s 2024-08-30T08:58:48.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:58:48 smithi190 ceph-mon[22001]: from='client.? 172.21.15.42:0/1551357923' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} : dispatch 2024-08-30T08:58:49.385 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2} subvols for filesystem 'cephfs' 2024-08-30T08:58:49.386 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph fs subvolumegroup create cephfs qa 2024-08-30T08:58:49.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:58:49 smithi042 ceph-mon[18652]: from='client.? 172.21.15.42:0/1551357923' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-08-30T08:58:49.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:58:49 smithi042 ceph-mon[18652]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-08-30T08:58:49.644 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T08:58:49.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:58:49 smithi167 ceph-mon[22063]: from='client.? 172.21.15.42:0/1551357923' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-08-30T08:58:49.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:58:49 smithi167 ceph-mon[22063]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-08-30T08:58:49.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:58:49 smithi190 ceph-mon[22001]: from='client.? 172.21.15.42:0/1551357923' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-08-30T08:58:49.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:58:49 smithi190 ceph-mon[22001]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-08-30T08:58:50.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:50.134+0000 7f8392a73640 1 -- 172.21.15.42:0/588033329 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f838c10dbb0 msgr2=0x7f838c110090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:58:50.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:50.134+0000 7f8392a73640 1 --2- 172.21.15.42:0/588033329 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f838c10dbb0 0x7f838c110090 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f8384009900 tx=0x7f838402a410 comp rx=0 tx=0).stop 2024-08-30T08:58:50.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:50.134+0000 7f8392a73640 1 -- 172.21.15.42:0/588033329 shutdown_connections 2024-08-30T08:58:50.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:50.134+0000 7f8392a73640 1 --2- 172.21.15.42:0/588033329 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f838c105030 0x7f838c10d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:50.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:50.134+0000 7f8392a73640 1 --2- 172.21.15.42:0/588033329 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f838c10dbb0 0x7f838c110090 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:50.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:50.134+0000 7f8392a73640 1 --2- 172.21.15.42:0/588033329 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f838c104600 0x7f838c104a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:50.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:50.134+0000 7f8392a73640 1 -- 172.21.15.42:0/588033329 >> 172.21.15.42:0/588033329 conn(0x7f838c0794d0 msgr2=0x7f838c0798e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:58:50.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:50.134+0000 7f8392a73640 1 -- 172.21.15.42:0/588033329 shutdown_connections 2024-08-30T08:58:50.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:50.134+0000 7f8392a73640 1 -- 172.21.15.42:0/588033329 wait complete. 2024-08-30T08:58:50.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:50.134+0000 7f8392a73640 1 Processor -- start 2024-08-30T08:58:50.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:50.134+0000 7f8392a73640 1 -- start start 2024-08-30T08:58:50.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:50.135+0000 7f8392a73640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f838c104600 0x7f838c19a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:58:50.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:50.135+0000 7f8392a73640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f838c105030 0x7f838c19a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:58:50.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:50.135+0000 7f8392a73640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f838c10dbb0 0x7f838c19fa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:58:50.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:50.135+0000 7f8392a73640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f838c113880 con 0x7f838c105030 2024-08-30T08:58:50.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:50.135+0000 7f8392a73640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f838c113700 con 0x7f838c104600 2024-08-30T08:58:50.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:50.135+0000 7f8392a73640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f838c113a00 con 0x7f838c10dbb0 2024-08-30T08:58:50.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:50.135+0000 7f838bfff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f838c105030 0x7f838c19a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:58:50.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:50.135+0000 7f838bfff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f838c105030 0x7f838c19a7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43238/0 (socket says 172.21.15.42:43238) 2024-08-30T08:58:50.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:50.135+0000 7f838bfff640 1 -- 172.21.15.42:0/2431116290 learned_addr learned my addr 172.21.15.42:0/2431116290 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T08:58:50.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:50.135+0000 7f8391010640 1 --2- 172.21.15.42:0/2431116290 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f838c10dbb0 0x7f838c19fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:58:50.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:50.135+0000 7f839080f640 1 --2- 172.21.15.42:0/2431116290 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f838c104600 0x7f838c19a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:58:50.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:50.135+0000 7f838bfff640 1 -- 172.21.15.42:0/2431116290 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f838c10dbb0 msgr2=0x7f838c19fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:58:50.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:50.135+0000 7f838bfff640 1 --2- 172.21.15.42:0/2431116290 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f838c10dbb0 0x7f838c19fa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:50.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:50.135+0000 7f838bfff640 1 -- 172.21.15.42:0/2431116290 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f838c104600 msgr2=0x7f838c19a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:58:50.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:50.135+0000 7f838bfff640 1 --2- 172.21.15.42:0/2431116290 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f838c104600 0x7f838c19a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:50.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:50.135+0000 7f838bfff640 1 -- 172.21.15.42:0/2431116290 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f838c1a00e0 con 0x7f838c105030 2024-08-30T08:58:50.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:50.136+0000 7f838bfff640 1 --2- 172.21.15.42:0/2431116290 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f838c105030 0x7f838c19a7d0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f837400c8c0 tx=0x7f837400cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:58:50.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:50.136+0000 7f8389ffb640 1 -- 172.21.15.42:0/2431116290 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8374007bb0 con 0x7f838c105030 2024-08-30T08:58:50.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:50.136+0000 7f8389ffb640 1 -- 172.21.15.42:0/2431116290 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8374007d50 con 0x7f838c105030 2024-08-30T08:58:50.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:50.136+0000 7f8392a73640 1 -- 172.21.15.42:0/2431116290 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f838c1a03d0 con 0x7f838c105030 2024-08-30T08:58:50.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:50.136+0000 7f8389ffb640 1 -- 172.21.15.42:0/2431116290 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83740187c0 con 0x7f838c105030 2024-08-30T08:58:50.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:50.136+0000 7f8392a73640 1 -- 172.21.15.42:0/2431116290 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f838c1a6bf0 con 0x7f838c105030 2024-08-30T08:58:50.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:50.137+0000 7f8392a73640 1 -- 172.21.15.42:0/2431116290 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f838c104a10 con 0x7f838c105030 2024-08-30T08:58:50.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:50.138+0000 7f8389ffb640 1 -- 172.21.15.42:0/2431116290 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) ==== 152577+0+0 (secure 0 0 0) 0x7f837400fa20 con 0x7f838c105030 2024-08-30T08:58:50.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:50.139+0000 7f8389ffb640 1 --2- 172.21.15.42:0/2431116290 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f83780b4be0 0x7f83780b70f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:58:50.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:50.139+0000 7f839080f640 1 --2- 172.21.15.42:0/2431116290 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f83780b4be0 0x7f83780b70f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:58:50.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:50.140+0000 7f8389ffb640 1 -- 172.21.15.42:0/2431116290 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 9092+0+0 (secure 0 0 0) 0x7f83740e5440 con 0x7f838c105030 2024-08-30T08:58:50.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:50.140+0000 7f839080f640 1 --2- 172.21.15.42:0/2431116290 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f83780b4be0 0x7f83780b70f0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f837c00a400 tx=0x7f837c00d040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:58:50.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:50.144+0000 7f8389ffb640 1 -- 172.21.15.42:0/2431116290 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f8374014030 con 0x7f838c105030 2024-08-30T08:58:50.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:50.356+0000 7f8392a73640 1 -- 172.21.15.42:0/2431116290 --> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f838c101af0 con 0x7f83780b4be0 2024-08-30T08:58:50.590 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Aug 30 08:58:50 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mgr-x[18896]: 2024-08-30T08:58:50.362+0000 7f1ae0cf5640 -1 client.0 error registering admin socket command: (17) File exists 2024-08-30T08:58:50.590 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Aug 30 08:58:50 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mgr-x[18896]: 2024-08-30T08:58:50.362+0000 7f1ae0cf5640 -1 client.0 error registering admin socket command: (17) File exists 2024-08-30T08:58:50.591 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Aug 30 08:58:50 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mgr-x[18896]: 2024-08-30T08:58:50.362+0000 7f1ae0cf5640 -1 client.0 error registering admin socket command: (17) File exists 2024-08-30T08:58:50.591 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Aug 30 08:58:50 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mgr-x[18896]: 2024-08-30T08:58:50.362+0000 7f1ae0cf5640 -1 client.0 error registering admin socket command: (17) File exists 2024-08-30T08:58:50.591 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Aug 30 08:58:50 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mgr-x[18896]: 2024-08-30T08:58:50.362+0000 7f1ae0cf5640 -1 client.0 error registering admin socket command: (17) File exists 2024-08-30T08:58:50.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:50.600+0000 7f8389ffb640 1 -- 172.21.15.42:0/2431116290 <== mgr.14150 v2:172.21.15.42:6800/3005140902 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f838c101af0 con 0x7f83780b4be0 2024-08-30T08:58:50.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:50.602+0000 7f8392a73640 1 -- 172.21.15.42:0/2431116290 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f83780b4be0 msgr2=0x7f83780b70f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:58:50.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:50.602+0000 7f8392a73640 1 --2- 172.21.15.42:0/2431116290 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f83780b4be0 0x7f83780b70f0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f837c00a400 tx=0x7f837c00d040 comp rx=0 tx=0).stop 2024-08-30T08:58:50.605 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:50.602+0000 7f8392a73640 1 -- 172.21.15.42:0/2431116290 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f838c105030 msgr2=0x7f838c19a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:58:50.605 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:50.602+0000 7f8392a73640 1 --2- 172.21.15.42:0/2431116290 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f838c105030 0x7f838c19a7d0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f837400c8c0 tx=0x7f837400cd80 comp rx=0 tx=0).stop 2024-08-30T08:58:50.605 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:50.603+0000 7f8392a73640 1 -- 172.21.15.42:0/2431116290 shutdown_connections 2024-08-30T08:58:50.605 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:50.603+0000 7f8392a73640 1 --2- 172.21.15.42:0/2431116290 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f83780b4be0 0x7f83780b70f0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:50.605 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:50.603+0000 7f8392a73640 1 --2- 172.21.15.42:0/2431116290 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f838c10dbb0 0x7f838c19fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:50.605 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:50.603+0000 7f8392a73640 1 --2- 172.21.15.42:0/2431116290 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f838c105030 0x7f838c19a7d0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:50.605 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:50.603+0000 7f8392a73640 1 --2- 172.21.15.42:0/2431116290 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f838c104600 0x7f838c19a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:50.605 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:50.603+0000 7f8392a73640 1 -- 172.21.15.42:0/2431116290 >> 172.21.15.42:0/2431116290 conn(0x7f838c0794d0 msgr2=0x7f838c1003d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:58:50.605 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:50.603+0000 7f8392a73640 1 -- 172.21.15.42:0/2431116290 shutdown_connections 2024-08-30T08:58:50.605 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:50.603+0000 7f8392a73640 1 -- 172.21.15.42:0/2431116290 wait complete. 2024-08-30T08:58:50.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:58:50 smithi042 ceph-mon[18652]: pgmap v313: 129 pgs: 2 activating, 127 active+clean; 584 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s wr, 9 op/s; 0 B/s, 0 objects/s recovering 2024-08-30T08:58:50.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:58:50 smithi042 ceph-mon[18652]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-08-30T08:58:50.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:58:50 smithi190 ceph-mon[22001]: pgmap v313: 129 pgs: 2 activating, 127 active+clean; 584 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s wr, 9 op/s; 0 B/s, 0 objects/s recovering 2024-08-30T08:58:50.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:58:50 smithi190 ceph-mon[22001]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-08-30T08:58:51.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:58:50 smithi167 ceph-mon[22063]: pgmap v313: 129 pgs: 2 activating, 127 active+clean; 584 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s wr, 9 op/s; 0 B/s, 0 objects/s recovering 2024-08-30T08:58:51.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:58:50 smithi167 ceph-mon[22063]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-08-30T08:58:51.317 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph fs subvolume create cephfs sv_0 --group_name qa 2024-08-30T08:58:51.569 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T08:58:51.851 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:58:51 smithi042 ceph-mon[18652]: from='client.14955 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-08-30T08:58:51.851 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:58:51 smithi042 ceph-mon[18652]: mds.? [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] up:active 2024-08-30T08:58:51.851 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:58:51 smithi042 ceph-mon[18652]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 9 up:standby 2024-08-30T08:58:51.851 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:58:51 smithi042 ceph-mon[18652]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-30T08:58:51.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:58:51 smithi190 ceph-mon[22001]: from='client.14955 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-08-30T08:58:51.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:58:51 smithi190 ceph-mon[22001]: mds.? [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] up:active 2024-08-30T08:58:51.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:58:51 smithi190 ceph-mon[22001]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 9 up:standby 2024-08-30T08:58:51.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:58:51 smithi190 ceph-mon[22001]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-30T08:58:52.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:58:51 smithi167 ceph-mon[22063]: from='client.14955 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-08-30T08:58:52.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:58:51 smithi167 ceph-mon[22063]: mds.? [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] up:active 2024-08-30T08:58:52.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:58:51 smithi167 ceph-mon[22063]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 9 up:standby 2024-08-30T08:58:52.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:58:51 smithi167 ceph-mon[22063]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-30T08:58:52.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:52.080+0000 7f9766179640 1 -- 172.21.15.42:0/2841562370 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9760108a90 msgr2=0x7f976010f2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:58:52.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:52.080+0000 7f9766179640 1 --2- 172.21.15.42:0/2841562370 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9760108a90 0x7f976010f2c0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f9758009ca0 tx=0x7f975802a8c0 comp rx=0 tx=0).stop 2024-08-30T08:58:52.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:52.080+0000 7f9766179640 1 -- 172.21.15.42:0/2841562370 shutdown_connections 2024-08-30T08:58:52.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:52.080+0000 7f9766179640 1 --2- 172.21.15.42:0/2841562370 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9760108140 0x7f9760108550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:52.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:52.080+0000 7f9766179640 1 --2- 172.21.15.42:0/2841562370 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9760108a90 0x7f976010f2c0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:52.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:52.080+0000 7f9766179640 1 --2- 172.21.15.42:0/2841562370 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9760102800 0x7f9760102c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:52.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:52.080+0000 7f9766179640 1 -- 172.21.15.42:0/2841562370 >> 172.21.15.42:0/2841562370 conn(0x7f97600794d0 msgr2=0x7f97600798e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:58:52.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:52.080+0000 7f9766179640 1 -- 172.21.15.42:0/2841562370 shutdown_connections 2024-08-30T08:58:52.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:52.080+0000 7f9766179640 1 -- 172.21.15.42:0/2841562370 wait complete. 2024-08-30T08:58:52.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:52.080+0000 7f9766179640 1 Processor -- start 2024-08-30T08:58:52.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:52.081+0000 7f9766179640 1 -- start start 2024-08-30T08:58:52.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:52.081+0000 7f9766179640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9760102800 0x7f9760104e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:58:52.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:52.081+0000 7f9766179640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9760108140 0x7f97601053c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:58:52.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:52.081+0000 7f9766179640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9760108a90 0x7f9760105900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:58:52.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:52.081+0000 7f9766179640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f9760112710 con 0x7f9760102800 2024-08-30T08:58:52.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:52.081+0000 7f9766179640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f9760112590 con 0x7f9760108a90 2024-08-30T08:58:52.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:52.081+0000 7f9766179640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f9760112890 con 0x7f9760108140 2024-08-30T08:58:52.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:52.081+0000 7f975f7fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9760102800 0x7f9760104e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:58:52.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:52.081+0000 7f975f7fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9760102800 0x7f9760104e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43258/0 (socket says 172.21.15.42:43258) 2024-08-30T08:58:52.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:52.081+0000 7f975f7fe640 1 -- 172.21.15.42:0/1734667882 learned_addr learned my addr 172.21.15.42:0/1734667882 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T08:58:52.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:52.081+0000 7f975effd640 1 --2- 172.21.15.42:0/1734667882 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9760108140 0x7f97601053c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:58:52.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:52.081+0000 7f975f7fe640 1 -- 172.21.15.42:0/1734667882 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9760108140 msgr2=0x7f97601053c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:58:52.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:52.081+0000 7f975f7fe640 1 --2- 172.21.15.42:0/1734667882 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9760108140 0x7f97601053c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:52.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:52.081+0000 7f975f7fe640 1 -- 172.21.15.42:0/1734667882 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9760108a90 msgr2=0x7f9760105900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:58:52.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:52.081+0000 7f975f7fe640 1 --2- 172.21.15.42:0/1734667882 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9760108a90 0x7f9760105900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:52.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:52.081+0000 7f975f7fe640 1 -- 172.21.15.42:0/1734667882 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97601a8480 con 0x7f9760102800 2024-08-30T08:58:52.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:52.082+0000 7f975f7fe640 1 --2- 172.21.15.42:0/1734667882 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9760102800 0x7f9760104e80 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f975400e9f0 tx=0x7f975400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:58:52.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:52.082+0000 7f975cff9640 1 -- 172.21.15.42:0/1734667882 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f975400cd80 con 0x7f9760102800 2024-08-30T08:58:52.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:52.082+0000 7f9766179640 1 -- 172.21.15.42:0/1734667882 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97601a8770 con 0x7f9760102800 2024-08-30T08:58:52.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:52.082+0000 7f975cff9640 1 -- 172.21.15.42:0/1734667882 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f97540108e0 con 0x7f9760102800 2024-08-30T08:58:52.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:52.082+0000 7f975cff9640 1 -- 172.21.15.42:0/1734667882 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9754018980 con 0x7f9760102800 2024-08-30T08:58:52.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:52.082+0000 7f9766179640 1 -- 172.21.15.42:0/1734667882 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97601a8cb0 con 0x7f9760102800 2024-08-30T08:58:52.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:52.083+0000 7f9766179640 1 -- 172.21.15.42:0/1734667882 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9760102c10 con 0x7f9760102800 2024-08-30T08:58:52.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:52.085+0000 7f975cff9640 1 -- 172.21.15.42:0/1734667882 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) ==== 152628+0+0 (secure 0 0 0) 0x7f9754010430 con 0x7f9760102800 2024-08-30T08:58:52.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:52.085+0000 7f975cff9640 1 --2- 172.21.15.42:0/1734667882 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f97480b4bc0 0x7f97480b70d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:58:52.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:52.086+0000 7f975effd640 1 --2- 172.21.15.42:0/1734667882 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f97480b4bc0 0x7f97480b70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:58:52.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:52.086+0000 7f975cff9640 1 -- 172.21.15.42:0/1734667882 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 9092+0+0 (secure 0 0 0) 0x7f97540e5ec0 con 0x7f9760102800 2024-08-30T08:58:52.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:52.086+0000 7f975effd640 1 --2- 172.21.15.42:0/1734667882 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f97480b4bc0 0x7f97480b70d0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f9760107800 tx=0x7f9750005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:58:52.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:52.089+0000 7f975cff9640 1 -- 172.21.15.42:0/1734667882 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f97540acb30 con 0x7f9760102800 2024-08-30T08:58:52.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:52.300+0000 7f9766179640 1 -- 172.21.15.42:0/1734667882 --> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f97600635a0 con 0x7f97480b4bc0 2024-08-30T08:58:52.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:52.327+0000 7f975cff9640 1 -- 172.21.15.42:0/1734667882 <== mgr.14150 v2:172.21.15.42:6800/3005140902 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f97600635a0 con 0x7f97480b4bc0 2024-08-30T08:58:52.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:52.330+0000 7f9766179640 1 -- 172.21.15.42:0/1734667882 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f97480b4bc0 msgr2=0x7f97480b70d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:58:52.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:52.330+0000 7f9766179640 1 --2- 172.21.15.42:0/1734667882 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f97480b4bc0 0x7f97480b70d0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f9760107800 tx=0x7f9750005f90 comp rx=0 tx=0).stop 2024-08-30T08:58:52.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:52.330+0000 7f9766179640 1 -- 172.21.15.42:0/1734667882 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9760102800 msgr2=0x7f9760104e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:58:52.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:52.330+0000 7f9766179640 1 --2- 172.21.15.42:0/1734667882 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9760102800 0x7f9760104e80 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f975400e9f0 tx=0x7f975400eeb0 comp rx=0 tx=0).stop 2024-08-30T08:58:52.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:52.330+0000 7f9766179640 1 -- 172.21.15.42:0/1734667882 shutdown_connections 2024-08-30T08:58:52.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:52.330+0000 7f9766179640 1 --2- 172.21.15.42:0/1734667882 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f97480b4bc0 0x7f97480b70d0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:52.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:52.330+0000 7f9766179640 1 --2- 172.21.15.42:0/1734667882 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9760108140 0x7f97601053c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:52.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:52.330+0000 7f9766179640 1 --2- 172.21.15.42:0/1734667882 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9760108a90 0x7f9760105900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:52.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:52.330+0000 7f9766179640 1 --2- 172.21.15.42:0/1734667882 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9760102800 0x7f9760104e80 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:52.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:52.330+0000 7f9766179640 1 -- 172.21.15.42:0/1734667882 >> 172.21.15.42:0/1734667882 conn(0x7f97600794d0 msgr2=0x7f976006af50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:58:52.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:52.330+0000 7f9766179640 1 -- 172.21.15.42:0/1734667882 shutdown_connections 2024-08-30T08:58:52.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:52.330+0000 7f9766179640 1 -- 172.21.15.42:0/1734667882 wait complete. 2024-08-30T08:58:52.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:58:52 smithi042 ceph-mon[18652]: pgmap v314: 129 pgs: 2 activating, 127 active+clean; 584 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s wr, 9 op/s; 0 B/s, 0 objects/s recovering 2024-08-30T08:58:52.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:58:52 smithi042 ceph-mon[18652]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-08-30T08:58:52.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:58:52 smithi190 ceph-mon[22001]: pgmap v314: 129 pgs: 2 activating, 127 active+clean; 584 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s wr, 9 op/s; 0 B/s, 0 objects/s recovering 2024-08-30T08:58:52.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:58:52 smithi190 ceph-mon[22001]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-08-30T08:58:53.069 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-08-30T08:58:53.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:58:52 smithi167 ceph-mon[22063]: pgmap v314: 129 pgs: 2 activating, 127 active+clean; 584 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s wr, 9 op/s; 0 B/s, 0 objects/s recovering 2024-08-30T08:58:53.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:58:52 smithi167 ceph-mon[22063]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-08-30T08:58:53.321 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T08:58:53.719 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:58:53 smithi042 ceph-mon[18652]: from='client.14967 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-08-30T08:58:53.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:53.813+0000 7f6667fa1640 1 -- 172.21.15.42:0/1037465800 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6660109b90 msgr2=0x7f66601103c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:58:53.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:53.813+0000 7f6667fa1640 1 --2- 172.21.15.42:0/1037465800 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6660109b90 0x7f66601103c0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f6658009ca0 tx=0x7f665802a8c0 comp rx=0 tx=0).stop 2024-08-30T08:58:53.829 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:53.813+0000 7f6667fa1640 1 -- 172.21.15.42:0/1037465800 shutdown_connections 2024-08-30T08:58:53.829 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:53.813+0000 7f6667fa1640 1 --2- 172.21.15.42:0/1037465800 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6660109b90 0x7f66601103c0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:53.829 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:53.813+0000 7f6667fa1640 1 --2- 172.21.15.42:0/1037465800 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6660109240 0x7f6660109650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:53.829 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:53.813+0000 7f6667fa1640 1 --2- 172.21.15.42:0/1037465800 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6660103900 0x7f6660103d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:53.829 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:53.813+0000 7f6667fa1640 1 -- 172.21.15.42:0/1037465800 >> 172.21.15.42:0/1037465800 conn(0x7f666007b5f0 msgr2=0x7f666007ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:58:53.829 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:53.813+0000 7f6667fa1640 1 -- 172.21.15.42:0/1037465800 shutdown_connections 2024-08-30T08:58:53.829 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:53.813+0000 7f6667fa1640 1 -- 172.21.15.42:0/1037465800 wait complete. 2024-08-30T08:58:53.829 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:53.814+0000 7f6667fa1640 1 Processor -- start 2024-08-30T08:58:53.829 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:53.814+0000 7f6667fa1640 1 -- start start 2024-08-30T08:58:53.829 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:53.814+0000 7f6667fa1640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6660103900 0x7f666019a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:58:53.829 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:53.814+0000 7f6667fa1640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6660109240 0x7f666019a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:58:53.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:53.814+0000 7f6667fa1640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6660109b90 0x7f666019fa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:58:53.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:53.814+0000 7f6667fa1640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f6660113850 con 0x7f6660109b90 2024-08-30T08:58:53.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:53.814+0000 7f6667fa1640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f66601136d0 con 0x7f6660103900 2024-08-30T08:58:53.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:53.814+0000 7f6667fa1640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f66601139d0 con 0x7f6660109240 2024-08-30T08:58:53.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:53.814+0000 7f666653e640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6660109b90 0x7f666019fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:58:53.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:53.815+0000 7f666653e640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6660109b90 0x7f666019fa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43278/0 (socket says 172.21.15.42:43278) 2024-08-30T08:58:53.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:53.815+0000 7f666653e640 1 -- 172.21.15.42:0/686599024 learned_addr learned my addr 172.21.15.42:0/686599024 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T08:58:53.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:53.815+0000 7f6665d3d640 1 --2- 172.21.15.42:0/686599024 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6660103900 0x7f666019a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:58:53.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:53.815+0000 7f666553c640 1 --2- 172.21.15.42:0/686599024 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6660109240 0x7f666019a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:58:53.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:53.815+0000 7f666653e640 1 -- 172.21.15.42:0/686599024 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6660109240 msgr2=0x7f666019a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:58:53.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:53.815+0000 7f666653e640 1 --2- 172.21.15.42:0/686599024 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6660109240 0x7f666019a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:53.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:53.815+0000 7f666653e640 1 -- 172.21.15.42:0/686599024 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6660103900 msgr2=0x7f666019a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:58:53.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:53.815+0000 7f666653e640 1 --2- 172.21.15.42:0/686599024 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6660103900 0x7f666019a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:53.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:53.815+0000 7f666653e640 1 -- 172.21.15.42:0/686599024 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66601a0090 con 0x7f6660109b90 2024-08-30T08:58:53.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:53.815+0000 7f6665d3d640 1 --2- 172.21.15.42:0/686599024 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6660103900 0x7f666019a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-30T08:58:53.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:53.815+0000 7f666653e640 1 --2- 172.21.15.42:0/686599024 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6660109b90 0x7f666019fa20 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f6658005cc0 tx=0x7f665802f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:58:53.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:53.815+0000 7f6656ffd640 1 -- 172.21.15.42:0/686599024 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6658003bf0 con 0x7f6660109b90 2024-08-30T08:58:53.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:53.815+0000 7f6667fa1640 1 -- 172.21.15.42:0/686599024 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66601a0320 con 0x7f6660109b90 2024-08-30T08:58:53.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:53.815+0000 7f6656ffd640 1 -- 172.21.15.42:0/686599024 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6658003d90 con 0x7f6660109b90 2024-08-30T08:58:53.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:53.815+0000 7f6656ffd640 1 -- 172.21.15.42:0/686599024 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f665802fbc0 con 0x7f6660109b90 2024-08-30T08:58:53.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:53.815+0000 7f6667fa1640 1 -- 172.21.15.42:0/686599024 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66601a0860 con 0x7f6660109b90 2024-08-30T08:58:53.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:53.817+0000 7f6667fa1640 1 -- 172.21.15.42:0/686599024 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6660103d10 con 0x7f6660109b90 2024-08-30T08:58:53.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:53.819+0000 7f6656ffd640 1 -- 172.21.15.42:0/686599024 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) ==== 152628+0+0 (secure 0 0 0) 0x7f665802fd60 con 0x7f6660109b90 2024-08-30T08:58:53.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:53.820+0000 7f6656ffd640 1 --2- 172.21.15.42:0/686599024 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f664c0b4bf0 0x7f664c0b7100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:58:53.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:53.820+0000 7f6665d3d640 1 --2- 172.21.15.42:0/686599024 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f664c0b4bf0 0x7f664c0b7100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:58:53.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:53.820+0000 7f6656ffd640 1 -- 172.21.15.42:0/686599024 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 9092+0+0 (secure 0 0 0) 0x7f6658106b80 con 0x7f6660109b90 2024-08-30T08:58:53.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:53.820+0000 7f6665d3d640 1 --2- 172.21.15.42:0/686599024 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f664c0b4bf0 0x7f664c0b7100 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f665c00abe0 tx=0x7f665c00a5c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:58:53.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:53.823+0000 7f6656ffd640 1 -- 172.21.15.42:0/686599024 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f66580cd790 con 0x7f6660109b90 2024-08-30T08:58:53.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:58:53 smithi190 ceph-mon[22001]: from='client.14967 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-08-30T08:58:54.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:54.040+0000 7f6667fa1640 1 -- 172.21.15.42:0/686599024 --> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7f66600635a0 con 0x7f664c0b4bf0 2024-08-30T08:58:54.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:54.045+0000 7f6656ffd640 1 -- 172.21.15.42:0/686599024 <== mgr.14150 v2:172.21.15.42:6800/3005140902 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f66600635a0 con 0x7f664c0b4bf0 2024-08-30T08:58:54.047 INFO:teuthology.orchestra.run.smithi042.stdout:/volumes/qa/sv_0/ce39f6f2-ddb7-4adc-9748-b8a85396827e 2024-08-30T08:58:54.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:54.047+0000 7f6667fa1640 1 -- 172.21.15.42:0/686599024 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f664c0b4bf0 msgr2=0x7f664c0b7100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:58:54.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:54.047+0000 7f6667fa1640 1 --2- 172.21.15.42:0/686599024 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f664c0b4bf0 0x7f664c0b7100 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f665c00abe0 tx=0x7f665c00a5c0 comp rx=0 tx=0).stop 2024-08-30T08:58:54.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:54.047+0000 7f6667fa1640 1 -- 172.21.15.42:0/686599024 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6660109b90 msgr2=0x7f666019fa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:58:54.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:54.047+0000 7f6667fa1640 1 --2- 172.21.15.42:0/686599024 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6660109b90 0x7f666019fa20 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f6658005cc0 tx=0x7f665802f380 comp rx=0 tx=0).stop 2024-08-30T08:58:54.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:54.047+0000 7f6667fa1640 1 -- 172.21.15.42:0/686599024 shutdown_connections 2024-08-30T08:58:54.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:54.047+0000 7f6667fa1640 1 --2- 172.21.15.42:0/686599024 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f664c0b4bf0 0x7f664c0b7100 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:54.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:54.047+0000 7f6667fa1640 1 --2- 172.21.15.42:0/686599024 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6660109240 0x7f666019a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:54.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:54.047+0000 7f6667fa1640 1 --2- 172.21.15.42:0/686599024 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6660109b90 0x7f666019fa20 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:54.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:54.047+0000 7f6667fa1640 1 --2- 172.21.15.42:0/686599024 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6660103900 0x7f666019a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:54.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:54.047+0000 7f6667fa1640 1 -- 172.21.15.42:0/686599024 >> 172.21.15.42:0/686599024 conn(0x7f666007b5f0 msgr2=0x7f6660107250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:58:54.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:54.048+0000 7f6667fa1640 1 -- 172.21.15.42:0/686599024 shutdown_connections 2024-08-30T08:58:54.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:54.048+0000 7f6667fa1640 1 -- 172.21.15.42:0/686599024 wait complete. 2024-08-30T08:58:54.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:58:53 smithi167 ceph-mon[22063]: from='client.14967 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-08-30T08:58:54.077 INFO:journalctl@ceph.mds.b.smithi167.stdout:Aug 30 08:58:53 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mds-b[52473]: 2024-08-30T08:58:53.612+0000 7f9d3d6df640 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-08-30T08:58:54.800 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:58:54 smithi042 ceph-mon[18652]: pgmap v315: 129 pgs: 129 active+clean; 605 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 6.2 KiB/s wr, 15 op/s; 0 B/s, 0 objects/s recovering 2024-08-30T08:58:54.829 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph fs subvolume create cephfs sv_1 --group_name qa 2024-08-30T08:58:54.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:58:54 smithi190 ceph-mon[22001]: pgmap v315: 129 pgs: 129 active+clean; 605 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 6.2 KiB/s wr, 15 op/s; 0 B/s, 0 objects/s recovering 2024-08-30T08:58:55.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:58:54 smithi167 ceph-mon[22063]: pgmap v315: 129 pgs: 129 active+clean; 605 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 6.2 KiB/s wr, 15 op/s; 0 B/s, 0 objects/s recovering 2024-08-30T08:58:55.080 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T08:58:55.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:55.548+0000 7facfa14e640 1 -- 172.21.15.42:0/657969671 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7facf4102f50 msgr2=0x7facf4103360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:58:55.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:55.548+0000 7facfa14e640 1 --2- 172.21.15.42:0/657969671 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7facf4102f50 0x7facf4103360 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7face4009900 tx=0x7face402a3f0 comp rx=0 tx=0).stop 2024-08-30T08:58:55.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:55.548+0000 7facfa14e640 1 -- 172.21.15.42:0/657969671 shutdown_connections 2024-08-30T08:58:55.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:55.548+0000 7facfa14e640 1 --2- 172.21.15.42:0/657969671 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7facf4104f60 0x7facf4105370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:55.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:55.548+0000 7facfa14e640 1 --2- 172.21.15.42:0/657969671 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7facf41058b0 0x7facf410f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:55.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:55.548+0000 7facfa14e640 1 --2- 172.21.15.42:0/657969671 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7facf4102f50 0x7facf4103360 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:55.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:55.548+0000 7facfa14e640 1 -- 172.21.15.42:0/657969671 >> 172.21.15.42:0/657969671 conn(0x7facf4079be0 msgr2=0x7facf4079ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:58:55.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:55.548+0000 7facfa14e640 1 -- 172.21.15.42:0/657969671 shutdown_connections 2024-08-30T08:58:55.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:55.548+0000 7facfa14e640 1 -- 172.21.15.42:0/657969671 wait complete. 2024-08-30T08:58:55.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:55.548+0000 7facfa14e640 1 Processor -- start 2024-08-30T08:58:55.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:55.548+0000 7facfa14e640 1 -- start start 2024-08-30T08:58:55.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:55.549+0000 7facfa14e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7facf4102f50 0x7facf419a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:58:55.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:55.549+0000 7facfa14e640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7facf4104f60 0x7facf419a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:58:55.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:55.549+0000 7facfa14e640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7facf41058b0 0x7facf419fa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:58:55.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:55.549+0000 7facfa14e640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7facf4112790 con 0x7facf41058b0 2024-08-30T08:58:55.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:55.549+0000 7facfa14e640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7facf4112610 con 0x7facf4104f60 2024-08-30T08:58:55.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:55.549+0000 7facfa14e640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7facf4112910 con 0x7facf4102f50 2024-08-30T08:58:55.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:55.549+0000 7facf3fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7facf41058b0 0x7facf419fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:58:55.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:55.549+0000 7facf3fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7facf41058b0 0x7facf419fa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43294/0 (socket says 172.21.15.42:43294) 2024-08-30T08:58:55.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:55.549+0000 7facf3fff640 1 -- 172.21.15.42:0/3686978348 learned_addr learned my addr 172.21.15.42:0/3686978348 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T08:58:55.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:55.549+0000 7facf37fe640 1 --2- 172.21.15.42:0/3686978348 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7facf4102f50 0x7facf419a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:58:55.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:55.549+0000 7faceaffd640 1 --2- 172.21.15.42:0/3686978348 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7facf4104f60 0x7facf419a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:58:55.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:55.549+0000 7facf3fff640 1 -- 172.21.15.42:0/3686978348 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7facf4102f50 msgr2=0x7facf419a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:58:55.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:55.549+0000 7facf3fff640 1 --2- 172.21.15.42:0/3686978348 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7facf4102f50 0x7facf419a210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:55.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:55.549+0000 7facf3fff640 1 -- 172.21.15.42:0/3686978348 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7facf4104f60 msgr2=0x7facf419a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:58:55.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:55.549+0000 7facf3fff640 1 --2- 172.21.15.42:0/3686978348 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7facf4104f60 0x7facf419a7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:55.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:55.549+0000 7facf3fff640 1 -- 172.21.15.42:0/3686978348 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facf41a00c0 con 0x7facf41058b0 2024-08-30T08:58:55.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:55.549+0000 7faceaffd640 1 --2- 172.21.15.42:0/3686978348 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7facf4104f60 0x7facf419a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-30T08:58:55.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:55.550+0000 7facf3fff640 1 --2- 172.21.15.42:0/3686978348 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7facf41058b0 0x7facf419fa50 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7facec00cce0 tx=0x7facec00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:58:55.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:55.550+0000 7facf17fa640 1 -- 172.21.15.42:0/3686978348 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facec00f450 con 0x7facf41058b0 2024-08-30T08:58:55.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:55.550+0000 7facfa14e640 1 -- 172.21.15.42:0/3686978348 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7facf41a03b0 con 0x7facf41058b0 2024-08-30T08:58:55.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:55.550+0000 7facf17fa640 1 -- 172.21.15.42:0/3686978348 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7facec00faa0 con 0x7facf41058b0 2024-08-30T08:58:55.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:55.550+0000 7facf17fa640 1 -- 172.21.15.42:0/3686978348 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facec018b70 con 0x7facf41058b0 2024-08-30T08:58:55.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:55.550+0000 7facfa14e640 1 -- 172.21.15.42:0/3686978348 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7facf41a6bd0 con 0x7facf41058b0 2024-08-30T08:58:55.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:55.551+0000 7facfa14e640 1 -- 172.21.15.42:0/3686978348 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7facf4103360 con 0x7facf41058b0 2024-08-30T08:58:55.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:55.553+0000 7facf17fa640 1 -- 172.21.15.42:0/3686978348 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) ==== 152628+0+0 (secure 0 0 0) 0x7facec018d10 con 0x7facf41058b0 2024-08-30T08:58:55.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:55.554+0000 7facf17fa640 1 --2- 172.21.15.42:0/3686978348 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7face00b4bf0 0x7face00b7100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:58:55.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:55.554+0000 7facf37fe640 1 --2- 172.21.15.42:0/3686978348 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7face00b4bf0 0x7face00b7100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:58:55.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:55.554+0000 7facf17fa640 1 -- 172.21.15.42:0/3686978348 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 9092+0+0 (secure 0 0 0) 0x7facec0e5bc0 con 0x7facf41058b0 2024-08-30T08:58:55.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:55.555+0000 7facf37fe640 1 --2- 172.21.15.42:0/3686978348 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7face00b4bf0 0x7face00b7100 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7face4005ec0 tx=0x7face40058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:58:55.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:55.557+0000 7facf17fa640 1 -- 172.21.15.42:0/3686978348 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7facec014030 con 0x7facf41058b0 2024-08-30T08:58:55.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:55.771+0000 7facfa14e640 1 -- 172.21.15.42:0/3686978348 --> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7facf4101bb0 con 0x7face00b4bf0 2024-08-30T08:58:55.773 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:58:55 smithi042 ceph-mon[18652]: from='client.14973 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-08-30T08:58:55.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:55.796+0000 7facf17fa640 1 -- 172.21.15.42:0/3686978348 <== mgr.14150 v2:172.21.15.42:6800/3005140902 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7facf4101bb0 con 0x7face00b4bf0 2024-08-30T08:58:55.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:55.798+0000 7facfa14e640 1 -- 172.21.15.42:0/3686978348 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7face00b4bf0 msgr2=0x7face00b7100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:58:55.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:55.798+0000 7facfa14e640 1 --2- 172.21.15.42:0/3686978348 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7face00b4bf0 0x7face00b7100 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7face4005ec0 tx=0x7face40058d0 comp rx=0 tx=0).stop 2024-08-30T08:58:55.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:55.798+0000 7facfa14e640 1 -- 172.21.15.42:0/3686978348 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7facf41058b0 msgr2=0x7facf419fa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:58:55.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:55.798+0000 7facfa14e640 1 --2- 172.21.15.42:0/3686978348 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7facf41058b0 0x7facf419fa50 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7facec00cce0 tx=0x7facec00a620 comp rx=0 tx=0).stop 2024-08-30T08:58:55.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:55.798+0000 7facfa14e640 1 -- 172.21.15.42:0/3686978348 shutdown_connections 2024-08-30T08:58:55.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:55.798+0000 7facfa14e640 1 --2- 172.21.15.42:0/3686978348 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7face00b4bf0 0x7face00b7100 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:55.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:55.798+0000 7facfa14e640 1 --2- 172.21.15.42:0/3686978348 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7facf41058b0 0x7facf419fa50 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:55.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:55.798+0000 7facfa14e640 1 --2- 172.21.15.42:0/3686978348 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7facf4104f60 0x7facf419a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:55.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:55.798+0000 7facfa14e640 1 --2- 172.21.15.42:0/3686978348 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7facf4102f50 0x7facf419a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:55.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:55.798+0000 7facfa14e640 1 -- 172.21.15.42:0/3686978348 >> 172.21.15.42:0/3686978348 conn(0x7facf4079be0 msgr2=0x7facf4100e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:58:55.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:55.798+0000 7facfa14e640 1 -- 172.21.15.42:0/3686978348 shutdown_connections 2024-08-30T08:58:55.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:55.798+0000 7facfa14e640 1 -- 172.21.15.42:0/3686978348 wait complete. 2024-08-30T08:58:55.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:58:55 smithi190 ceph-mon[22001]: from='client.14973 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-08-30T08:58:56.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:58:55 smithi167 ceph-mon[22063]: from='client.14973 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-08-30T08:58:56.512 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-08-30T08:58:56.765 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T08:58:56.882 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:58:56 smithi042 ceph-mon[18652]: pgmap v316: 129 pgs: 129 active+clean; 605 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 5.8 KiB/s wr, 14 op/s; 0 B/s, 0 objects/s recovering 2024-08-30T08:58:56.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:58:56 smithi190 ceph-mon[22001]: pgmap v316: 129 pgs: 129 active+clean; 605 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 5.8 KiB/s wr, 14 op/s; 0 B/s, 0 objects/s recovering 2024-08-30T08:58:57.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:58:56 smithi167 ceph-mon[22063]: pgmap v316: 129 pgs: 129 active+clean; 605 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 5.8 KiB/s wr, 14 op/s; 0 B/s, 0 objects/s recovering 2024-08-30T08:58:57.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:57.215+0000 7f5d4af9a640 1 -- 172.21.15.42:0/1602429335 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5d44109b90 msgr2=0x7f5d441103c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:58:57.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:57.215+0000 7f5d4af9a640 1 --2- 172.21.15.42:0/1602429335 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5d44109b90 0x7f5d441103c0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f5d3c009ca0 tx=0x7f5d3c02a8c0 comp rx=0 tx=0).stop 2024-08-30T08:58:57.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:57.215+0000 7f5d4af9a640 1 -- 172.21.15.42:0/1602429335 shutdown_connections 2024-08-30T08:58:57.294 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:57.215+0000 7f5d4af9a640 1 --2- 172.21.15.42:0/1602429335 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5d44109240 0x7f5d44109650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:57.294 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:57.215+0000 7f5d4af9a640 1 --2- 172.21.15.42:0/1602429335 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5d44109b90 0x7f5d441103c0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:57.294 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:57.215+0000 7f5d4af9a640 1 --2- 172.21.15.42:0/1602429335 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5d44103900 0x7f5d44103d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:57.294 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:57.215+0000 7f5d4af9a640 1 -- 172.21.15.42:0/1602429335 >> 172.21.15.42:0/1602429335 conn(0x7f5d4407b5f0 msgr2=0x7f5d4407ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:58:57.294 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:57.215+0000 7f5d4af9a640 1 -- 172.21.15.42:0/1602429335 shutdown_connections 2024-08-30T08:58:57.294 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:57.215+0000 7f5d4af9a640 1 -- 172.21.15.42:0/1602429335 wait complete. 2024-08-30T08:58:57.294 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:57.216+0000 7f5d4af9a640 1 Processor -- start 2024-08-30T08:58:57.294 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:57.216+0000 7f5d4af9a640 1 -- start start 2024-08-30T08:58:57.294 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:57.216+0000 7f5d4af9a640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5d44103900 0x7f5d4419a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:58:57.294 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:57.216+0000 7f5d4af9a640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5d44109240 0x7f5d4419a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:58:57.294 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:57.216+0000 7f5d4af9a640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5d44109b90 0x7f5d4419fa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:58:57.294 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:57.216+0000 7f5d4af9a640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f5d44113850 con 0x7f5d44109b90 2024-08-30T08:58:57.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:57.216+0000 7f5d4af9a640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f5d441136d0 con 0x7f5d44109240 2024-08-30T08:58:57.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:57.216+0000 7f5d4af9a640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f5d441139d0 con 0x7f5d44103900 2024-08-30T08:58:57.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:57.217+0000 7f5d49537640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5d44109b90 0x7f5d4419fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:58:57.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:57.217+0000 7f5d43fff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5d44109240 0x7f5d4419a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:58:57.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:57.217+0000 7f5d49537640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5d44109b90 0x7f5d4419fa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43308/0 (socket says 172.21.15.42:43308) 2024-08-30T08:58:57.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:57.217+0000 7f5d49537640 1 -- 172.21.15.42:0/3079096212 learned_addr learned my addr 172.21.15.42:0/3079096212 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T08:58:57.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:57.217+0000 7f5d48d36640 1 --2- 172.21.15.42:0/3079096212 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5d44103900 0x7f5d4419a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:58:57.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:57.217+0000 7f5d49537640 1 -- 172.21.15.42:0/3079096212 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5d44103900 msgr2=0x7f5d4419a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:58:57.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:57.217+0000 7f5d49537640 1 --2- 172.21.15.42:0/3079096212 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5d44103900 0x7f5d4419a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:57.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:57.217+0000 7f5d49537640 1 -- 172.21.15.42:0/3079096212 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5d44109240 msgr2=0x7f5d4419a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:58:57.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:57.217+0000 7f5d49537640 1 --2- 172.21.15.42:0/3079096212 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5d44109240 0x7f5d4419a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:57.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:57.217+0000 7f5d49537640 1 -- 172.21.15.42:0/3079096212 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d441a0150 con 0x7f5d44109b90 2024-08-30T08:58:57.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:57.217+0000 7f5d49537640 1 --2- 172.21.15.42:0/3079096212 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5d44109b90 0x7f5d4419fa20 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f5d3c005cc0 tx=0x7f5d3c02f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:58:57.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:57.217+0000 7f5d41ffb640 1 -- 172.21.15.42:0/3079096212 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d3c003bf0 con 0x7f5d44109b90 2024-08-30T08:58:57.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:57.217+0000 7f5d41ffb640 1 -- 172.21.15.42:0/3079096212 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d3c003d90 con 0x7f5d44109b90 2024-08-30T08:58:57.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:57.217+0000 7f5d4af9a640 1 -- 172.21.15.42:0/3079096212 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d441a03e0 con 0x7f5d44109b90 2024-08-30T08:58:57.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:57.217+0000 7f5d41ffb640 1 -- 172.21.15.42:0/3079096212 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d3c02fbc0 con 0x7f5d44109b90 2024-08-30T08:58:57.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:57.218+0000 7f5d4af9a640 1 -- 172.21.15.42:0/3079096212 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d441a6ba0 con 0x7f5d44109b90 2024-08-30T08:58:57.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:57.218+0000 7f5d4af9a640 1 -- 172.21.15.42:0/3079096212 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d44103d10 con 0x7f5d44109b90 2024-08-30T08:58:57.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:57.220+0000 7f5d41ffb640 1 -- 172.21.15.42:0/3079096212 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) ==== 152628+0+0 (secure 0 0 0) 0x7f5d3c02fd60 con 0x7f5d44109b90 2024-08-30T08:58:57.297 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:57.221+0000 7f5d41ffb640 1 --2- 172.21.15.42:0/3079096212 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f5d340b4c40 0x7f5d340b7150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:58:57.297 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:57.221+0000 7f5d48d36640 1 --2- 172.21.15.42:0/3079096212 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f5d340b4c40 0x7f5d340b7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:58:57.297 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:57.222+0000 7f5d41ffb640 1 -- 172.21.15.42:0/3079096212 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 9092+0+0 (secure 0 0 0) 0x7f5d3c106b90 con 0x7f5d44109b90 2024-08-30T08:58:57.297 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:57.222+0000 7f5d48d36640 1 --2- 172.21.15.42:0/3079096212 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f5d340b4c40 0x7f5d340b7150 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f5d3800bbe0 tx=0x7f5d3800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:58:57.297 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:57.224+0000 7f5d41ffb640 1 -- 172.21.15.42:0/3079096212 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f5d3c0cd800 con 0x7f5d44109b90 2024-08-30T08:58:57.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:57.438+0000 7f5d4af9a640 1 -- 172.21.15.42:0/3079096212 --> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7f5d440635a0 con 0x7f5d340b4c40 2024-08-30T08:58:57.444 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:57.443+0000 7f5d41ffb640 1 -- 172.21.15.42:0/3079096212 <== mgr.14150 v2:172.21.15.42:6800/3005140902 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f5d440635a0 con 0x7f5d340b4c40 2024-08-30T08:58:57.445 INFO:teuthology.orchestra.run.smithi042.stdout:/volumes/qa/sv_1/d3168875-d2e5-410d-8754-72d29955777b 2024-08-30T08:58:57.447 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:57.445+0000 7f5d4af9a640 1 -- 172.21.15.42:0/3079096212 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f5d340b4c40 msgr2=0x7f5d340b7150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:58:57.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:57.445+0000 7f5d4af9a640 1 --2- 172.21.15.42:0/3079096212 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f5d340b4c40 0x7f5d340b7150 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f5d3800bbe0 tx=0x7f5d3800b390 comp rx=0 tx=0).stop 2024-08-30T08:58:57.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:57.445+0000 7f5d4af9a640 1 -- 172.21.15.42:0/3079096212 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5d44109b90 msgr2=0x7f5d4419fa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:58:57.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:57.445+0000 7f5d4af9a640 1 --2- 172.21.15.42:0/3079096212 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5d44109b90 0x7f5d4419fa20 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f5d3c005cc0 tx=0x7f5d3c02f380 comp rx=0 tx=0).stop 2024-08-30T08:58:57.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:57.446+0000 7f5d4af9a640 1 -- 172.21.15.42:0/3079096212 shutdown_connections 2024-08-30T08:58:57.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:57.446+0000 7f5d4af9a640 1 --2- 172.21.15.42:0/3079096212 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f5d340b4c40 0x7f5d340b7150 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:57.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:57.446+0000 7f5d4af9a640 1 --2- 172.21.15.42:0/3079096212 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5d44109b90 0x7f5d4419fa20 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:57.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:57.446+0000 7f5d4af9a640 1 --2- 172.21.15.42:0/3079096212 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5d44109240 0x7f5d4419a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:57.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:57.446+0000 7f5d4af9a640 1 --2- 172.21.15.42:0/3079096212 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5d44103900 0x7f5d4419a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:57.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:57.446+0000 7f5d4af9a640 1 -- 172.21.15.42:0/3079096212 >> 172.21.15.42:0/3079096212 conn(0x7f5d4407b5f0 msgr2=0x7f5d44105210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:58:57.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:57.446+0000 7f5d4af9a640 1 -- 172.21.15.42:0/3079096212 shutdown_connections 2024-08-30T08:58:57.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:57.446+0000 7f5d4af9a640 1 -- 172.21.15.42:0/3079096212 wait complete. 2024-08-30T08:58:57.882 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:58:57 smithi042 ceph-mon[18652]: from='client.14979 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-08-30T08:58:57.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:58:57 smithi190 ceph-mon[22001]: from='client.14979 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-08-30T08:58:58.047 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph fs dump --format=json 2024-08-30T08:58:58.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:58:57 smithi167 ceph-mon[22063]: from='client.14979 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-08-30T08:58:58.299 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T08:58:58.689 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:58:58 smithi042 ceph-mon[18652]: pgmap v317: 129 pgs: 129 active+clean; 605 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 5.2 KiB/s wr, 12 op/s; 0 B/s, 0 objects/s recovering 2024-08-30T08:58:58.689 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:58:58 smithi042 ceph-mon[18652]: from='client.14985 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-08-30T08:58:58.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:58.777+0000 7fd727d5c640 1 -- 172.21.15.42:0/1003377656 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd720102880 msgr2=0x7fd720102c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:58:58.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:58.777+0000 7fd727d5c640 1 --2- 172.21.15.42:0/1003377656 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd720102880 0x7fd720102c90 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fd71c009900 tx=0x7fd71c02a3f0 comp rx=0 tx=0).stop 2024-08-30T08:58:58.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:58.777+0000 7fd727d5c640 1 -- 172.21.15.42:0/1003377656 shutdown_connections 2024-08-30T08:58:58.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:58.777+0000 7fd727d5c640 1 --2- 172.21.15.42:0/1003377656 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd720108170 0x7fd720108580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:58.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:58.777+0000 7fd727d5c640 1 --2- 172.21.15.42:0/1003377656 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd720108ac0 0x7fd72010f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:58.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:58.777+0000 7fd727d5c640 1 --2- 172.21.15.42:0/1003377656 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd720102880 0x7fd720102c90 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:58.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:58.777+0000 7fd727d5c640 1 -- 172.21.15.42:0/1003377656 >> 172.21.15.42:0/1003377656 conn(0x7fd720079500 msgr2=0x7fd720079910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:58:58.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:58.777+0000 7fd727d5c640 1 -- 172.21.15.42:0/1003377656 shutdown_connections 2024-08-30T08:58:58.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:58.777+0000 7fd727d5c640 1 -- 172.21.15.42:0/1003377656 wait complete. 2024-08-30T08:58:58.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:58.778+0000 7fd727d5c640 1 Processor -- start 2024-08-30T08:58:58.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:58.778+0000 7fd727d5c640 1 -- start start 2024-08-30T08:58:58.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:58.778+0000 7fd727d5c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd720102880 0x7fd72019afd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:58:58.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:58.779+0000 7fd727d5c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd720108170 0x7fd72019b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:58:58.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:58.779+0000 7fd727d5c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd720108ac0 0x7fd72019ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:58:58.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:58.779+0000 7fd727d5c640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fd720112720 con 0x7fd720108170 2024-08-30T08:58:58.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:58.779+0000 7fd727d5c640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd7201125a0 con 0x7fd720108ac0 2024-08-30T08:58:58.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:58.779+0000 7fd727d5c640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fd7201128a0 con 0x7fd720102880 2024-08-30T08:58:58.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:58.779+0000 7fd7252f7640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd720108170 0x7fd72019b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:58:58.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:58.779+0000 7fd7252f7640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd720108170 0x7fd72019b510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60988/0 (socket says 172.21.15.42:60988) 2024-08-30T08:58:58.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:58.779+0000 7fd7252f7640 1 -- 172.21.15.42:0/4030202542 learned_addr learned my addr 172.21.15.42:0/4030202542 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T08:58:58.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:58.779+0000 7fd725af8640 1 --2- 172.21.15.42:0/4030202542 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd720102880 0x7fd72019afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:58:58.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:58.779+0000 7fd7262f9640 1 --2- 172.21.15.42:0/4030202542 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd720108ac0 0x7fd72019ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:58:58.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:58.779+0000 7fd7252f7640 1 -- 172.21.15.42:0/4030202542 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd720102880 msgr2=0x7fd72019afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:58:58.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:58.779+0000 7fd7252f7640 1 --2- 172.21.15.42:0/4030202542 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd720102880 0x7fd72019afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:58.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:58.779+0000 7fd7252f7640 1 -- 172.21.15.42:0/4030202542 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd720108ac0 msgr2=0x7fd72019ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:58:58.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:58.779+0000 7fd7252f7640 1 --2- 172.21.15.42:0/4030202542 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd720108ac0 0x7fd72019ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:58.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:58.779+0000 7fd7252f7640 1 -- 172.21.15.42:0/4030202542 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7201a61d0 con 0x7fd720108170 2024-08-30T08:58:58.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:58.779+0000 7fd7252f7640 1 --2- 172.21.15.42:0/4030202542 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd720108170 0x7fd72019b510 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fd71400c970 tx=0x7fd71400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:58:58.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:58.780+0000 7fd712ffd640 1 -- 172.21.15.42:0/4030202542 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd714014070 con 0x7fd720108170 2024-08-30T08:58:58.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:58.780+0000 7fd727d5c640 1 -- 172.21.15.42:0/4030202542 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7201a64c0 con 0x7fd720108170 2024-08-30T08:58:58.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:58.780+0000 7fd712ffd640 1 -- 172.21.15.42:0/4030202542 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd714004440 con 0x7fd720108170 2024-08-30T08:58:58.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:58.780+0000 7fd712ffd640 1 -- 172.21.15.42:0/4030202542 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd714003e50 con 0x7fd720108170 2024-08-30T08:58:58.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:58.780+0000 7fd727d5c640 1 -- 172.21.15.42:0/4030202542 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7201a6a00 con 0x7fd720108170 2024-08-30T08:58:58.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:58.781+0000 7fd727d5c640 1 -- 172.21.15.42:0/4030202542 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd720103ab0 con 0x7fd720108170 2024-08-30T08:58:58.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:58.782+0000 7fd712ffd640 1 -- 172.21.15.42:0/4030202542 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) ==== 152628+0+0 (secure 0 0 0) 0x7fd714017070 con 0x7fd720108170 2024-08-30T08:58:58.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:58.783+0000 7fd712ffd640 1 --2- 172.21.15.42:0/4030202542 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fd70c0b4d90 0x7fd70c0b72a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:58:58.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:58.783+0000 7fd725af8640 1 --2- 172.21.15.42:0/4030202542 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fd70c0b4d90 0x7fd70c0b72a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:58:58.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:58.783+0000 7fd712ffd640 1 -- 172.21.15.42:0/4030202542 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 9092+0+0 (secure 0 0 0) 0x7fd7140e5870 con 0x7fd720108170 2024-08-30T08:58:58.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:58.784+0000 7fd725af8640 1 --2- 172.21.15.42:0/4030202542 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fd70c0b4d90 0x7fd70c0b72a0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fd71c000c00 tx=0x7fd71c031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:58:58.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:58.790+0000 7fd712ffd640 1 -- 172.21.15.42:0/4030202542 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7fd71401e030 con 0x7fd720108170 2024-08-30T08:58:58.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:58:58 smithi190 ceph-mon[22001]: pgmap v317: 129 pgs: 129 active+clean; 605 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 5.2 KiB/s wr, 12 op/s; 0 B/s, 0 objects/s recovering 2024-08-30T08:58:58.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:58:58 smithi190 ceph-mon[22001]: from='client.14985 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-08-30T08:58:59.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:59.035+0000 7fd727d5c640 1 -- 172.21.15.42:0/4030202542 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd720102c90 con 0x7fd720108170 2024-08-30T08:58:59.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:59.037+0000 7fd712ffd640 1 -- 172.21.15.42:0/4030202542 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11915 (secure 0 0 0) 0x7fd7140ac4e0 con 0x7fd720108170 2024-08-30T08:58:59.039 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-08-30T08:58:59.039 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":26,"btime":"2024-08-30T08:58:50:586137+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14703,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/3188400000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":3188400000},{"type":"v1","addr":"172.21.15.42:6835","nonce":3188400000}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14715,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/3356537674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":3356537674},{"type":"v1","addr":"172.21.15.42:6837","nonce":3356537674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14766,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6839/2107785513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6838","nonce":2107785513},{"type":"v1","addr":"172.21.15.42:6839","nonce":2107785513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14778,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6841/2298438674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6840","nonce":2298438674},{"type":"v1","addr":"172.21.15.42:6841","nonce":2298438674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24553,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/543555570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":543555570},{"type":"v1","addr":"172.21.15.190:6833","nonce":543555570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24554,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/3907818941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":3907818941},{"type":"v1","addr":"172.21.15.167:6833","nonce":3907818941}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-30T08:58:03.055084+0000","modified":"2024-08-30T08:58:50.586122+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24629,"mds_1":24610,"mds_2":24599},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24572":{"gid":24572,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6835/234596845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":234596845},{"type":"v1","addr":"172.21.15.167:6835","nonce":234596845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24592":{"gid":24592,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6835/247905709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":247905709},{"type":"v1","addr":"172.21.15.190:6835","nonce":247905709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24598":{"gid":24598,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6837/863256349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6836","nonce":863256349},{"type":"v1","addr":"172.21.15.190:6837","nonce":863256349}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24599":{"gid":24599,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.167:6837/3011837856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":3011837856},{"type":"v1","addr":"172.21.15.167:6837","nonce":3011837856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24610":{"gid":24610,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.190:6839/719981110","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6838","nonce":719981110},{"type":"v1","addr":"172.21.15.190:6839","nonce":719981110}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24629":{"gid":24629,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.167:6839/3604645207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":3604645207},{"type":"v1","addr":"172.21.15.167:6839","nonce":3604645207}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24629,"qdb_cluster":[24629,24610,24599]},"id":1}]} 2024-08-30T08:58:59.039 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 26 2024-08-30T08:58:59.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:59.040+0000 7fd727d5c640 1 -- 172.21.15.42:0/4030202542 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fd70c0b4d90 msgr2=0x7fd70c0b72a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:58:59.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:59.040+0000 7fd727d5c640 1 --2- 172.21.15.42:0/4030202542 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fd70c0b4d90 0x7fd70c0b72a0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fd71c000c00 tx=0x7fd71c031400 comp rx=0 tx=0).stop 2024-08-30T08:58:59.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:59.040+0000 7fd727d5c640 1 -- 172.21.15.42:0/4030202542 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd720108170 msgr2=0x7fd72019b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:58:59.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:59.040+0000 7fd727d5c640 1 --2- 172.21.15.42:0/4030202542 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd720108170 0x7fd72019b510 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fd71400c970 tx=0x7fd71400ce30 comp rx=0 tx=0).stop 2024-08-30T08:58:59.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:59.040+0000 7fd727d5c640 1 -- 172.21.15.42:0/4030202542 shutdown_connections 2024-08-30T08:58:59.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:59.040+0000 7fd727d5c640 1 --2- 172.21.15.42:0/4030202542 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fd70c0b4d90 0x7fd70c0b72a0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:59.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:59.040+0000 7fd727d5c640 1 --2- 172.21.15.42:0/4030202542 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd720108ac0 0x7fd72019ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:59.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:59.040+0000 7fd727d5c640 1 --2- 172.21.15.42:0/4030202542 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd720108170 0x7fd72019b510 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:59.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:59.040+0000 7fd727d5c640 1 --2- 172.21.15.42:0/4030202542 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd720102880 0x7fd72019afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:58:59.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:59.040+0000 7fd727d5c640 1 -- 172.21.15.42:0/4030202542 >> 172.21.15.42:0/4030202542 conn(0x7fd720079500 msgr2=0x7fd7201011c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:58:59.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:59.040+0000 7fd727d5c640 1 -- 172.21.15.42:0/4030202542 shutdown_connections 2024-08-30T08:58:59.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:58:59.040+0000 7fd727d5c640 1 -- 172.21.15.42:0/4030202542 wait complete. 2024-08-30T08:58:59.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:58:58 smithi167 ceph-mon[22063]: pgmap v317: 129 pgs: 129 active+clean; 605 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 5.2 KiB/s wr, 12 op/s; 0 B/s, 0 objects/s recovering 2024-08-30T08:58:59.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:58:58 smithi167 ceph-mon[22063]: from='client.14985 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-08-30T08:58:59.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:58:59 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-30T08:58:59.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:58:59 smithi190 ceph-mon[22001]: from='client.? 172.21.15.42:0/4030202542' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-30T08:58:59.694 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:58:59 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-30T08:58:59.694 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:58:59 smithi042 ceph-mon[18652]: from='client.? 172.21.15.42:0/4030202542' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-30T08:58:59.727 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph osd dump --format=json 2024-08-30T08:58:59.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:58:59 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-30T08:58:59.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:58:59 smithi167 ceph-mon[22063]: from='client.? 172.21.15.42:0/4030202542' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-30T08:59:00.654 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T08:59:00.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:59:00 smithi042 ceph-mon[18652]: pgmap v318: 129 pgs: 129 active+clean; 623 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 5.0 KiB/s wr, 12 op/s; 0 B/s, 0 objects/s recovering 2024-08-30T08:59:00.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:59:00 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:59:00.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:59:00 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:59:00.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:59:00 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:59:00.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:59:00 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:59:00.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:59:00 smithi190 ceph-mon[22001]: pgmap v318: 129 pgs: 129 active+clean; 623 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 5.0 KiB/s wr, 12 op/s; 0 B/s, 0 objects/s recovering 2024-08-30T08:59:00.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:59:00 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:59:00.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:59:00 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:59:00.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:59:00 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:59:00.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:59:00 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:59:01.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:59:00 smithi167 ceph-mon[22063]: pgmap v318: 129 pgs: 129 active+clean; 623 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 5.0 KiB/s wr, 12 op/s; 0 B/s, 0 objects/s recovering 2024-08-30T08:59:01.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:59:00 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:59:01.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:59:00 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:59:01.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:59:00 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:59:01.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:59:00 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:59:01.195 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:01.179+0000 7f43e6549640 1 -- 172.21.15.42:0/3226207909 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f43e0105110 msgr2=0x7f43e010d750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:01.195 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:01.179+0000 7f43e6549640 1 --2- 172.21.15.42:0/3226207909 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f43e0105110 0x7f43e010d750 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f43d4009900 tx=0x7f43d402a410 comp rx=0 tx=0).stop 2024-08-30T08:59:01.195 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:01.179+0000 7f43e6549640 1 -- 172.21.15.42:0/3226207909 shutdown_connections 2024-08-30T08:59:01.195 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:01.179+0000 7f43e6549640 1 --2- 172.21.15.42:0/3226207909 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f43e010dc90 0x7f43e0110170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:01.195 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:01.180+0000 7f43e6549640 1 --2- 172.21.15.42:0/3226207909 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f43e0105110 0x7f43e010d750 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:01.195 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:01.180+0000 7f43e6549640 1 --2- 172.21.15.42:0/3226207909 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f43e01046e0 0x7f43e0104af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:01.195 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:01.180+0000 7f43e6549640 1 -- 172.21.15.42:0/3226207909 >> 172.21.15.42:0/3226207909 conn(0x7f43e00794f0 msgr2=0x7f43e0079900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:59:01.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:01.180+0000 7f43e6549640 1 -- 172.21.15.42:0/3226207909 shutdown_connections 2024-08-30T08:59:01.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:01.180+0000 7f43e6549640 1 -- 172.21.15.42:0/3226207909 wait complete. 2024-08-30T08:59:01.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:01.180+0000 7f43e6549640 1 Processor -- start 2024-08-30T08:59:01.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:01.180+0000 7f43e6549640 1 -- start start 2024-08-30T08:59:01.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:01.181+0000 7f43e6549640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f43e01046e0 0x7f43e019a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:01.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:01.181+0000 7f43e6549640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f43e0105110 0x7f43e019a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:01.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:01.181+0000 7f43e6549640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f43e010dc90 0x7f43e019fab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:01.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:01.181+0000 7f43e6549640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f43e0113800 con 0x7f43e010dc90 2024-08-30T08:59:01.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:01.181+0000 7f43e6549640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f43e0113680 con 0x7f43e01046e0 2024-08-30T08:59:01.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:01.181+0000 7f43e6549640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f43e0113980 con 0x7f43e0105110 2024-08-30T08:59:01.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:01.181+0000 7f43e4ae6640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f43e010dc90 0x7f43e019fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:01.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:01.181+0000 7f43e4ae6640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f43e010dc90 0x7f43e019fab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:32770/0 (socket says 172.21.15.42:32770) 2024-08-30T08:59:01.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:01.181+0000 7f43e4ae6640 1 -- 172.21.15.42:0/3180091651 learned_addr learned my addr 172.21.15.42:0/3180091651 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T08:59:01.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:01.181+0000 7f43dffff640 1 --2- 172.21.15.42:0/3180091651 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f43e01046e0 0x7f43e019a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:01.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:01.181+0000 7f43df7fe640 1 --2- 172.21.15.42:0/3180091651 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f43e0105110 0x7f43e019a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:01.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:01.181+0000 7f43e4ae6640 1 -- 172.21.15.42:0/3180091651 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f43e0105110 msgr2=0x7f43e019a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:01.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:01.181+0000 7f43e4ae6640 1 --2- 172.21.15.42:0/3180091651 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f43e0105110 0x7f43e019a810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:01.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:01.181+0000 7f43e4ae6640 1 -- 172.21.15.42:0/3180091651 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f43e01046e0 msgr2=0x7f43e019a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:01.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:01.181+0000 7f43e4ae6640 1 --2- 172.21.15.42:0/3180091651 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f43e01046e0 0x7f43e019a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:01.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:01.181+0000 7f43e4ae6640 1 -- 172.21.15.42:0/3180091651 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43e01a0120 con 0x7f43e010dc90 2024-08-30T08:59:01.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:01.181+0000 7f43df7fe640 1 --2- 172.21.15.42:0/3180091651 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f43e0105110 0x7f43e019a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-30T08:59:01.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:01.182+0000 7f43e4ae6640 1 --2- 172.21.15.42:0/3180091651 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f43e010dc90 0x7f43e019fab0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f43d800cad0 tx=0x7f43d800ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:59:01.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:01.182+0000 7f43dd7fa640 1 -- 172.21.15.42:0/3180091651 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43d800ad60 con 0x7f43e010dc90 2024-08-30T08:59:01.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:01.182+0000 7f43dd7fa640 1 -- 172.21.15.42:0/3180091651 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f43d8005020 con 0x7f43e010dc90 2024-08-30T08:59:01.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:01.182+0000 7f43e6549640 1 -- 172.21.15.42:0/3180091651 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43e01a0410 con 0x7f43e010dc90 2024-08-30T08:59:01.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:01.182+0000 7f43dd7fa640 1 -- 172.21.15.42:0/3180091651 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43d800f6b0 con 0x7f43e010dc90 2024-08-30T08:59:01.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:01.182+0000 7f43e6549640 1 -- 172.21.15.42:0/3180091651 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43e01a08d0 con 0x7f43e010dc90 2024-08-30T08:59:01.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:01.183+0000 7f43e6549640 1 -- 172.21.15.42:0/3180091651 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43e006bf60 con 0x7f43e010dc90 2024-08-30T08:59:01.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:01.184+0000 7f43dd7fa640 1 -- 172.21.15.42:0/3180091651 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) ==== 152628+0+0 (secure 0 0 0) 0x7f43d80051d0 con 0x7f43e010dc90 2024-08-30T08:59:01.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:01.185+0000 7f43dd7fa640 1 --2- 172.21.15.42:0/3180091651 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f43c80b4bf0 0x7f43c80b7100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:01.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:01.185+0000 7f43dd7fa640 1 -- 172.21.15.42:0/3180091651 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 9092+0+0 (secure 0 0 0) 0x7f43d80e5a40 con 0x7f43e010dc90 2024-08-30T08:59:01.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:01.185+0000 7f43dffff640 1 --2- 172.21.15.42:0/3180091651 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f43c80b4bf0 0x7f43c80b7100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:01.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:01.186+0000 7f43dffff640 1 --2- 172.21.15.42:0/3180091651 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f43c80b4bf0 0x7f43c80b7100 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f43d000abe0 tx=0x7f43d000a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:59:01.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:01.188+0000 7f43dd7fa640 1 -- 172.21.15.42:0/3180091651 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f43d8014030 con 0x7f43e010dc90 2024-08-30T08:59:01.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:01.381+0000 7f43e6549640 1 -- 172.21.15.42:0/3180091651 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f43e0104af0 con 0x7f43e010dc90 2024-08-30T08:59:01.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:01.384+0000 7f43dd7fa640 1 -- 172.21.15.42:0/3180091651 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+24772 (secure 0 0 0) 0x7f43e0104af0 con 0x7f43e010dc90 2024-08-30T08:59:01.386 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-08-30T08:59:01.386 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":81,"fsid":"b0895c1a-66ac-11ef-bcd4-c7b262605968","created":"2024-08-30T08:49:44.858265+0000","modified":"2024-08-30T08:58:46.548905+0000","last_up_change":"2024-08-30T08:57:12.367791+0000","last_in_change":"2024-08-30T08:56:50.274678+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-30T08:53:27.297022+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-30T08:57:58.489964+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-30T08:58:01.064783+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-08-30T08:58:07.416534+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"029d62d7-a6c6-4905-8c76-6dffa9d57b92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":2617890537},{"type":"v1","addr":"172.21.15.42:6803","nonce":2617890537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":2617890537},{"type":"v1","addr":"172.21.15.42:6805","nonce":2617890537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2617890537},{"type":"v1","addr":"172.21.15.42:6809","nonce":2617890537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":2617890537},{"type":"v1","addr":"172.21.15.42:6807","nonce":2617890537}]},"public_addr":"172.21.15.42:6803/2617890537","cluster_addr":"172.21.15.42:6805/2617890537","heartbeat_back_addr":"172.21.15.42:6809/2617890537","heartbeat_front_addr":"172.21.15.42:6807/2617890537","state":["exists","up"]},{"osd":1,"uuid":"3a78384b-0207-41df-9384-22665007d5b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":2791960082},{"type":"v1","addr":"172.21.15.167:6801","nonce":2791960082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":2791960082},{"type":"v1","addr":"172.21.15.167:6803","nonce":2791960082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":2791960082},{"type":"v1","addr":"172.21.15.167:6807","nonce":2791960082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":2791960082},{"type":"v1","addr":"172.21.15.167:6805","nonce":2791960082}]},"public_addr":"172.21.15.167:6801/2791960082","cluster_addr":"172.21.15.167:6803/2791960082","heartbeat_back_addr":"172.21.15.167:6807/2791960082","heartbeat_front_addr":"172.21.15.167:6805/2791960082","state":["exists","up"]},{"osd":2,"uuid":"16001809-c27c-4f52-948a-9998261acb4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":1101326404},{"type":"v1","addr":"172.21.15.190:6801","nonce":1101326404}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":1101326404},{"type":"v1","addr":"172.21.15.190:6803","nonce":1101326404}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":1101326404},{"type":"v1","addr":"172.21.15.190:6807","nonce":1101326404}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":1101326404},{"type":"v1","addr":"172.21.15.190:6805","nonce":1101326404}]},"public_addr":"172.21.15.190:6801/1101326404","cluster_addr":"172.21.15.190:6803/1101326404","heartbeat_back_addr":"172.21.15.190:6807/1101326404","heartbeat_front_addr":"172.21.15.190:6805/1101326404","state":["exists","up"]},{"osd":3,"uuid":"87bb3e36-9e9e-4b19-9d8c-0cdf6e089d3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":1135599916},{"type":"v1","addr":"172.21.15.42:6811","nonce":1135599916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":1135599916},{"type":"v1","addr":"172.21.15.42:6813","nonce":1135599916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":1135599916},{"type":"v1","addr":"172.21.15.42:6817","nonce":1135599916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":1135599916},{"type":"v1","addr":"172.21.15.42:6815","nonce":1135599916}]},"public_addr":"172.21.15.42:6811/1135599916","cluster_addr":"172.21.15.42:6813/1135599916","heartbeat_back_addr":"172.21.15.42:6817/1135599916","heartbeat_front_addr":"172.21.15.42:6815/1135599916","state":["exists","up"]},{"osd":4,"uuid":"3ca1aae3-c972-4fb9-ac86-236279c30d64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":2927511419},{"type":"v1","addr":"172.21.15.167:6809","nonce":2927511419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":2927511419},{"type":"v1","addr":"172.21.15.167:6811","nonce":2927511419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":2927511419},{"type":"v1","addr":"172.21.15.167:6815","nonce":2927511419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":2927511419},{"type":"v1","addr":"172.21.15.167:6813","nonce":2927511419}]},"public_addr":"172.21.15.167:6809/2927511419","cluster_addr":"172.21.15.167:6811/2927511419","heartbeat_back_addr":"172.21.15.167:6815/2927511419","heartbeat_front_addr":"172.21.15.167:6813/2927511419","state":["exists","up"]},{"osd":5,"uuid":"5d07f54b-0d07-4efe-9bcf-390755ce0258","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":2557145333},{"type":"v1","addr":"172.21.15.190:6809","nonce":2557145333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":2557145333},{"type":"v1","addr":"172.21.15.190:6811","nonce":2557145333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":2557145333},{"type":"v1","addr":"172.21.15.190:6815","nonce":2557145333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":2557145333},{"type":"v1","addr":"172.21.15.190:6813","nonce":2557145333}]},"public_addr":"172.21.15.190:6809/2557145333","cluster_addr":"172.21.15.190:6811/2557145333","heartbeat_back_addr":"172.21.15.190:6815/2557145333","heartbeat_front_addr":"172.21.15.190:6813/2557145333","state":["exists","up"]},{"osd":6,"uuid":"96ad64af-44ff-4197-97d4-156f0cb88e81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":3213526100},{"type":"v1","addr":"172.21.15.42:6819","nonce":3213526100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":3213526100},{"type":"v1","addr":"172.21.15.42:6821","nonce":3213526100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":3213526100},{"type":"v1","addr":"172.21.15.42:6825","nonce":3213526100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":3213526100},{"type":"v1","addr":"172.21.15.42:6823","nonce":3213526100}]},"public_addr":"172.21.15.42:6819/3213526100","cluster_addr":"172.21.15.42:6821/3213526100","heartbeat_back_addr":"172.21.15.42:6825/3213526100","heartbeat_front_addr":"172.21.15.42:6823/3213526100","state":["exists","up"]},{"osd":7,"uuid":"337a2112-eaf7-4e6b-86be-4e40d6ddf835","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":2962799814},{"type":"v1","addr":"172.21.15.167:6817","nonce":2962799814}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":2962799814},{"type":"v1","addr":"172.21.15.167:6819","nonce":2962799814}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":2962799814},{"type":"v1","addr":"172.21.15.167:6823","nonce":2962799814}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":2962799814},{"type":"v1","addr":"172.21.15.167:6821","nonce":2962799814}]},"public_addr":"172.21.15.167:6817/2962799814","cluster_addr":"172.21.15.167:6819/2962799814","heartbeat_back_addr":"172.21.15.167:6823/2962799814","heartbeat_front_addr":"172.21.15.167:6821/2962799814","state":["exists","up"]},{"osd":8,"uuid":"219a7224-4331-464b-ac86-c51054d51d91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":3242198843},{"type":"v1","addr":"172.21.15.190:6817","nonce":3242198843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":3242198843},{"type":"v1","addr":"172.21.15.190:6819","nonce":3242198843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":3242198843},{"type":"v1","addr":"172.21.15.190:6823","nonce":3242198843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":3242198843},{"type":"v1","addr":"172.21.15.190:6821","nonce":3242198843}]},"public_addr":"172.21.15.190:6817/3242198843","cluster_addr":"172.21.15.190:6819/3242198843","heartbeat_back_addr":"172.21.15.190:6823/3242198843","heartbeat_front_addr":"172.21.15.190:6821/3242198843","state":["exists","up"]},{"osd":9,"uuid":"2f2f5177-a009-40d9-991c-76aec8a5cba6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":1123536325},{"type":"v1","addr":"172.21.15.42:6827","nonce":1123536325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":1123536325},{"type":"v1","addr":"172.21.15.42:6829","nonce":1123536325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":1123536325},{"type":"v1","addr":"172.21.15.42:6833","nonce":1123536325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":1123536325},{"type":"v1","addr":"172.21.15.42:6831","nonce":1123536325}]},"public_addr":"172.21.15.42:6827/1123536325","cluster_addr":"172.21.15.42:6829/1123536325","heartbeat_back_addr":"172.21.15.42:6833/1123536325","heartbeat_front_addr":"172.21.15.42:6831/1123536325","state":["exists","up"]},{"osd":10,"uuid":"96ef5058-2518-4862-a165-17666b33a3e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":3451198319},{"type":"v1","addr":"172.21.15.167:6825","nonce":3451198319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":3451198319},{"type":"v1","addr":"172.21.15.167:6827","nonce":3451198319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":3451198319},{"type":"v1","addr":"172.21.15.167:6831","nonce":3451198319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":3451198319},{"type":"v1","addr":"172.21.15.167:6829","nonce":3451198319}]},"public_addr":"172.21.15.167:6825/3451198319","cluster_addr":"172.21.15.167:6827/3451198319","heartbeat_back_addr":"172.21.15.167:6831/3451198319","heartbeat_front_addr":"172.21.15.167:6829/3451198319","state":["exists","up"]},{"osd":11,"uuid":"a4eb9bd3-3faa-4c81-ba6d-c2084b72c9ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":380064457},{"type":"v1","addr":"172.21.15.190:6825","nonce":380064457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":380064457},{"type":"v1","addr":"172.21.15.190:6827","nonce":380064457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":380064457},{"type":"v1","addr":"172.21.15.190:6831","nonce":380064457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":380064457},{"type":"v1","addr":"172.21.15.190:6829","nonce":380064457}]},"public_addr":"172.21.15.190:6825/380064457","cluster_addr":"172.21.15.190:6827/380064457","heartbeat_back_addr":"172.21.15.190:6831/380064457","heartbeat_front_addr":"172.21.15.190:6829/380064457","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-30T08:52:36.511448+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-30T08:53:00.027277+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-30T08:53:23.770245+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-30T08:53:49.322226+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-30T08:54:14.350234+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-30T08:54:39.491793+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-30T08:55:04.870596+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-30T08:55:30.314150+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-30T08:55:54.483843+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-30T08:56:20.586115+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-30T08:56:45.666783+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-30T08:57:10.718387+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"4.9","mappings":[{"from":5,"to":8}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.42:0/2691233260":"2024-08-31T08:50:44.821070+0000","172.21.15.42:0/1518053351":"2024-08-31T08:50:44.821070+0000","172.21.15.42:0/352001020":"2024-08-31T08:50:44.821070+0000","172.21.15.42:6801/1733768634":"2024-08-31T08:50:44.821070+0000","172.21.15.42:0/1000597888":"2024-08-31T08:50:18.561851+0000","172.21.15.42:6800/1733768634":"2024-08-31T08:50:44.821070+0000","172.21.15.42:6801/3538882081":"2024-08-31T08:50:18.561851+0000","172.21.15.42:6800/3538882081":"2024-08-31T08:50:18.561851+0000","172.21.15.42:0/145366508":"2024-08-31T08:50:18.561851+0000","172.21.15.42:0/3346355213":"2024-08-31T08:50:18.561851+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-30T08:59:01.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:01.386+0000 7f43e6549640 1 -- 172.21.15.42:0/3180091651 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f43c80b4bf0 msgr2=0x7f43c80b7100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:01.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:01.386+0000 7f43e6549640 1 --2- 172.21.15.42:0/3180091651 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f43c80b4bf0 0x7f43c80b7100 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f43d000abe0 tx=0x7f43d000a2d0 comp rx=0 tx=0).stop 2024-08-30T08:59:01.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:01.386+0000 7f43e6549640 1 -- 172.21.15.42:0/3180091651 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f43e010dc90 msgr2=0x7f43e019fab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:01.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:01.386+0000 7f43e6549640 1 --2- 172.21.15.42:0/3180091651 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f43e010dc90 0x7f43e019fab0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f43d800cad0 tx=0x7f43d800ced0 comp rx=0 tx=0).stop 2024-08-30T08:59:01.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:01.386+0000 7f43e6549640 1 -- 172.21.15.42:0/3180091651 shutdown_connections 2024-08-30T08:59:01.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:01.386+0000 7f43e6549640 1 --2- 172.21.15.42:0/3180091651 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f43c80b4bf0 0x7f43c80b7100 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:01.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:01.386+0000 7f43e6549640 1 --2- 172.21.15.42:0/3180091651 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f43e0105110 0x7f43e019a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:01.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:01.386+0000 7f43e6549640 1 --2- 172.21.15.42:0/3180091651 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f43e010dc90 0x7f43e019fab0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:01.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:01.386+0000 7f43e6549640 1 --2- 172.21.15.42:0/3180091651 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f43e01046e0 0x7f43e019a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:01.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:01.386+0000 7f43e6549640 1 -- 172.21.15.42:0/3180091651 >> 172.21.15.42:0/3180091651 conn(0x7f43e00794f0 msgr2=0x7f43e0100730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:59:01.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:01.386+0000 7f43e6549640 1 -- 172.21.15.42:0/3180091651 shutdown_connections 2024-08-30T08:59:01.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:01.386+0000 7f43e6549640 1 -- 172.21.15.42:0/3180091651 wait complete. 2024-08-30T08:59:02.061 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-08-30T08:59:02.062 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph pg dump --format=json 2024-08-30T08:59:02.350 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T08:59:02.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:59:02 smithi042 ceph-mon[18652]: pgmap v319: 129 pgs: 129 active+clean; 623 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 4.4 KiB/s wr, 8 op/s 2024-08-30T08:59:02.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:59:02 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:59:02.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:59:02 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:59:02.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:59:02 smithi042 ceph-mon[18652]: from='client.? 172.21.15.42:0/3180091651' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-30T08:59:02.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:59:02 smithi167 ceph-mon[22063]: pgmap v319: 129 pgs: 129 active+clean; 623 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 4.4 KiB/s wr, 8 op/s 2024-08-30T08:59:02.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:59:02 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:59:02.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:59:02 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:59:02.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:59:02 smithi167 ceph-mon[22063]: from='client.? 172.21.15.42:0/3180091651' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-30T08:59:02.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:59:02 smithi190 ceph-mon[22001]: pgmap v319: 129 pgs: 129 active+clean; 623 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 4.4 KiB/s wr, 8 op/s 2024-08-30T08:59:02.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:59:02 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:59:02.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:59:02 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:59:02.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:59:02 smithi190 ceph-mon[22001]: from='client.? 172.21.15.42:0/3180091651' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-30T08:59:02.847 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:02.828+0000 7f1c0d088640 1 -- 172.21.15.42:0/1327256942 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1c08109b90 msgr2=0x7f1c081103c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:02.847 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:02.828+0000 7f1c0d088640 1 --2- 172.21.15.42:0/1327256942 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1c08109b90 0x7f1c081103c0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f1c00009ca0 tx=0x7f1c0002a8c0 comp rx=0 tx=0).stop 2024-08-30T08:59:02.847 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:02.828+0000 7f1c0d088640 1 -- 172.21.15.42:0/1327256942 shutdown_connections 2024-08-30T08:59:02.847 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:02.828+0000 7f1c0d088640 1 --2- 172.21.15.42:0/1327256942 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1c08109240 0x7f1c08109650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:02.847 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:02.828+0000 7f1c0d088640 1 --2- 172.21.15.42:0/1327256942 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1c08109b90 0x7f1c081103c0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:02.847 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:02.828+0000 7f1c0d088640 1 --2- 172.21.15.42:0/1327256942 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1c08103900 0x7f1c08103d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:02.847 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:02.828+0000 7f1c0d088640 1 -- 172.21.15.42:0/1327256942 >> 172.21.15.42:0/1327256942 conn(0x7f1c0807b5f0 msgr2=0x7f1c0807ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:59:02.847 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:02.828+0000 7f1c0d088640 1 -- 172.21.15.42:0/1327256942 shutdown_connections 2024-08-30T08:59:02.847 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:02.828+0000 7f1c0d088640 1 -- 172.21.15.42:0/1327256942 wait complete. 2024-08-30T08:59:02.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:02.829+0000 7f1c0d088640 1 Processor -- start 2024-08-30T08:59:02.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:02.829+0000 7f1c0d088640 1 -- start start 2024-08-30T08:59:02.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:02.830+0000 7f1c0d088640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1c08103900 0x7f1c0819a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:02.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:02.830+0000 7f1c0d088640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1c08109240 0x7f1c0819a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:02.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:02.830+0000 7f1c0d088640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1c08109b90 0x7f1c0819f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:02.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:02.830+0000 7f1c0d088640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f1c081138a0 con 0x7f1c08109b90 2024-08-30T08:59:02.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:02.830+0000 7f1c0d088640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f1c08113720 con 0x7f1c08109240 2024-08-30T08:59:02.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:02.830+0000 7f1c0d088640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f1c08113a20 con 0x7f1c08103900 2024-08-30T08:59:02.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:02.830+0000 7f1c0759e640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1c08109b90 0x7f1c0819f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:02.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:02.830+0000 7f1c0759e640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1c08109b90 0x7f1c0819f9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:32792/0 (socket says 172.21.15.42:32792) 2024-08-30T08:59:02.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:02.830+0000 7f1c0759e640 1 -- 172.21.15.42:0/2424209024 learned_addr learned my addr 172.21.15.42:0/2424209024 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T08:59:02.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:02.830+0000 7f1c0659c640 1 --2- 172.21.15.42:0/2424209024 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1c08109240 0x7f1c0819a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:02.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:02.830+0000 7f1c06d9d640 1 --2- 172.21.15.42:0/2424209024 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1c08103900 0x7f1c0819a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:02.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:02.830+0000 7f1c0759e640 1 -- 172.21.15.42:0/2424209024 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1c08103900 msgr2=0x7f1c0819a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:02.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:02.830+0000 7f1c0759e640 1 --2- 172.21.15.42:0/2424209024 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1c08103900 0x7f1c0819a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:02.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:02.830+0000 7f1c0759e640 1 -- 172.21.15.42:0/2424209024 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1c08109240 msgr2=0x7f1c0819a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:02.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:02.830+0000 7f1c0759e640 1 --2- 172.21.15.42:0/2424209024 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1c08109240 0x7f1c0819a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:02.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:02.830+0000 7f1c0759e640 1 -- 172.21.15.42:0/2424209024 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c081a0060 con 0x7f1c08109b90 2024-08-30T08:59:02.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:02.830+0000 7f1c06d9d640 1 --2- 172.21.15.42:0/2424209024 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1c08103900 0x7f1c0819a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-30T08:59:02.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:02.831+0000 7f1c0759e640 1 --2- 172.21.15.42:0/2424209024 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1c08109b90 0x7f1c0819f9f0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f1c00031a20 tx=0x7f1c00031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:59:02.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:02.831+0000 7f1beffff640 1 -- 172.21.15.42:0/2424209024 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c00003d30 con 0x7f1c08109b90 2024-08-30T08:59:02.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:02.831+0000 7f1beffff640 1 -- 172.21.15.42:0/2424209024 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c000044f0 con 0x7f1c08109b90 2024-08-30T08:59:02.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:02.831+0000 7f1c0d088640 1 -- 172.21.15.42:0/2424209024 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c081a02f0 con 0x7f1c08109b90 2024-08-30T08:59:02.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:02.831+0000 7f1beffff640 1 -- 172.21.15.42:0/2424209024 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c0002fd90 con 0x7f1c08109b90 2024-08-30T08:59:02.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:02.831+0000 7f1c0d088640 1 -- 172.21.15.42:0/2424209024 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c081a0830 con 0x7f1c08109b90 2024-08-30T08:59:02.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:02.832+0000 7f1c0d088640 1 -- 172.21.15.42:0/2424209024 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c08103d10 con 0x7f1c08109b90 2024-08-30T08:59:02.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:02.834+0000 7f1beffff640 1 -- 172.21.15.42:0/2424209024 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) ==== 152628+0+0 (secure 0 0 0) 0x7f1c00004040 con 0x7f1c08109b90 2024-08-30T08:59:02.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:02.835+0000 7f1beffff640 1 --2- 172.21.15.42:0/2424209024 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f1bf00b4bf0 0x7f1bf00b7100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:02.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:02.835+0000 7f1c06d9d640 1 --2- 172.21.15.42:0/2424209024 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f1bf00b4bf0 0x7f1bf00b7100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:02.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:02.835+0000 7f1beffff640 1 -- 172.21.15.42:0/2424209024 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 9092+0+0 (secure 0 0 0) 0x7f1c00106c10 con 0x7f1c08109b90 2024-08-30T08:59:02.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:02.835+0000 7f1c06d9d640 1 --2- 172.21.15.42:0/2424209024 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f1bf00b4bf0 0x7f1bf00b7100 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f1bf800abe0 tx=0x7f1bf800a5c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:59:02.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:02.838+0000 7f1beffff640 1 -- 172.21.15.42:0/2424209024 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f1c000cd910 con 0x7f1c08109b90 2024-08-30T08:59:03.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:03.021+0000 7f1c0d088640 1 -- 172.21.15.42:0/2424209024 --> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f1c0810b9e0 con 0x7f1bf00b4bf0 2024-08-30T08:59:03.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:03.028+0000 7f1beffff640 1 -- 172.21.15.42:0/2424209024 <== mgr.14150 v2:172.21.15.42:6800/3005140902 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340325 (secure 0 0 0) 0x7f1c0810b9e0 con 0x7f1bf00b4bf0 2024-08-30T08:59:03.033 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-08-30T08:59:03.036 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-08-30T08:59:03.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:03.033+0000 7f1c0d088640 1 -- 172.21.15.42:0/2424209024 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f1bf00b4bf0 msgr2=0x7f1bf00b7100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:03.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:03.033+0000 7f1c0d088640 1 --2- 172.21.15.42:0/2424209024 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f1bf00b4bf0 0x7f1bf00b7100 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f1bf800abe0 tx=0x7f1bf800a5c0 comp rx=0 tx=0).stop 2024-08-30T08:59:03.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:03.033+0000 7f1c0d088640 1 -- 172.21.15.42:0/2424209024 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1c08109b90 msgr2=0x7f1c0819f9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:03.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:03.033+0000 7f1c0d088640 1 --2- 172.21.15.42:0/2424209024 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1c08109b90 0x7f1c0819f9f0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f1c00031a20 tx=0x7f1c00031a50 comp rx=0 tx=0).stop 2024-08-30T08:59:03.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:03.033+0000 7f1c0d088640 1 -- 172.21.15.42:0/2424209024 shutdown_connections 2024-08-30T08:59:03.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:03.033+0000 7f1c0d088640 1 --2- 172.21.15.42:0/2424209024 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f1bf00b4bf0 0x7f1bf00b7100 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:03.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:03.033+0000 7f1c0d088640 1 --2- 172.21.15.42:0/2424209024 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1c08109b90 0x7f1c0819f9f0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:03.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:03.033+0000 7f1c0d088640 1 --2- 172.21.15.42:0/2424209024 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1c08109240 0x7f1c0819a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:03.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:03.033+0000 7f1c0d088640 1 --2- 172.21.15.42:0/2424209024 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1c08103900 0x7f1c0819a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:03.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:03.033+0000 7f1c0d088640 1 -- 172.21.15.42:0/2424209024 >> 172.21.15.42:0/2424209024 conn(0x7f1c0807b5f0 msgr2=0x7f1c08107230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:59:03.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:03.033+0000 7f1c0d088640 1 -- 172.21.15.42:0/2424209024 shutdown_connections 2024-08-30T08:59:03.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:03.033+0000 7f1c0d088640 1 -- 172.21.15.42:0/2424209024 wait complete. 2024-08-30T08:59:03.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:59:03 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:59:03.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:59:03 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-30T08:59:03.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:59:03 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:59:03.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:59:03 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:59:03.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:59:03 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-30T08:59:03.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:59:03 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:59:03.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:59:03 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T08:59:03.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:59:03 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-30T08:59:03.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:59:03 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T08:59:03.825 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":320,"stamp":"2024-08-30T08:59:02.938778+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":649386,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":174,"num_read_kb":175,"num_write":286,"num_write_kb":2259,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":233,"ondisk_log_size":233,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":374556,"kb_used_data":12824,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124567268,"statfs":{"total":1151940427776,"available":1151556882432,"internally_reserved":0,"allocated":13131776,"data_stored":7388576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":370193721},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":51566,"num_objects":3,"num_object_clones":0,"num_object_copies":11,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":98,"num_read_kb":111,"num_write":35,"num_write_kb":66,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003731"},"pg_stats":[{"pgid":"2.1f","version":"80'9","reported_seq":45,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.559175+0000","last_change":"2024-08-30T08:58:00.388595+0000","last_active":"2024-08-30T08:58:46.559175+0000","last_peered":"2024-08-30T08:58:46.559175+0000","last_clean":"2024-08-30T08:58:46.559175+0000","last_became_active":"2024-08-30T08:58:00.388215+0000","last_became_peered":"2024-08-30T08:58:00.388215+0000","last_unstale":"2024-08-30T08:58:46.559175+0000","last_undegraded":"2024-08-30T08:58:46.559175+0000","last_fullsized":"2024-08-30T08:58:46.559175+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T13:00:06.109935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.555091+0000","last_change":"2024-08-30T08:58:02.416042+0000","last_active":"2024-08-30T08:58:46.555091+0000","last_peered":"2024-08-30T08:58:46.555091+0000","last_clean":"2024-08-30T08:58:46.555091+0000","last_became_active":"2024-08-30T08:58:02.415507+0000","last_became_peered":"2024-08-30T08:58:02.415507+0000","last_unstale":"2024-08-30T08:58:46.555091+0000","last_undegraded":"2024-08-30T08:58:46.555091+0000","last_fullsized":"2024-08-30T08:58:46.555091+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T14:33:57.192004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.554665+0000","last_change":"2024-08-30T08:58:09.448785+0000","last_active":"2024-08-30T08:58:46.554665+0000","last_peered":"2024-08-30T08:58:46.554665+0000","last_clean":"2024-08-30T08:58:46.554665+0000","last_became_active":"2024-08-30T08:58:09.448191+0000","last_became_peered":"2024-08-30T08:58:09.448191+0000","last_unstale":"2024-08-30T08:58:46.554665+0000","last_undegraded":"2024-08-30T08:58:46.554665+0000","last_fullsized":"2024-08-30T08:58:46.554665+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T13:37:19.148201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"80'2","reported_seq":37,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.555518+0000","last_change":"2024-08-30T08:58:00.480406+0000","last_active":"2024-08-30T08:58:46.555518+0000","last_peered":"2024-08-30T08:58:46.555518+0000","last_clean":"2024-08-30T08:58:46.555518+0000","last_became_active":"2024-08-30T08:58:00.479896+0000","last_became_peered":"2024-08-30T08:58:00.479896+0000","last_unstale":"2024-08-30T08:58:46.555518+0000","last_undegraded":"2024-08-30T08:58:46.555518+0000","last_fullsized":"2024-08-30T08:58:46.555518+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T15:16:15.316343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.557006+0000","last_change":"2024-08-30T08:58:02.406915+0000","last_active":"2024-08-30T08:58:46.557006+0000","last_peered":"2024-08-30T08:58:46.557006+0000","last_clean":"2024-08-30T08:58:46.557006+0000","last_became_active":"2024-08-30T08:58:02.406557+0000","last_became_peered":"2024-08-30T08:58:02.406557+0000","last_unstale":"2024-08-30T08:58:46.557006+0000","last_undegraded":"2024-08-30T08:58:46.557006+0000","last_fullsized":"2024-08-30T08:58:46.557006+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T19:15:30.847212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:47.089607+0000","last_change":"2024-08-30T08:58:09.446974+0000","last_active":"2024-08-30T08:58:47.089607+0000","last_peered":"2024-08-30T08:58:47.089607+0000","last_clean":"2024-08-30T08:58:47.089607+0000","last_became_active":"2024-08-30T08:58:09.446173+0000","last_became_peered":"2024-08-30T08:58:09.446173+0000","last_unstale":"2024-08-30T08:58:47.089607+0000","last_undegraded":"2024-08-30T08:58:47.089607+0000","last_fullsized":"2024-08-30T08:58:47.089607+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T12:13:41.276825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"80'4","reported_seq":39,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.557615+0000","last_change":"2024-08-30T08:58:00.390609+0000","last_active":"2024-08-30T08:58:46.557615+0000","last_peered":"2024-08-30T08:58:46.557615+0000","last_clean":"2024-08-30T08:58:46.557615+0000","last_became_active":"2024-08-30T08:58:00.390229+0000","last_became_peered":"2024-08-30T08:58:00.390229+0000","last_unstale":"2024-08-30T08:58:46.557615+0000","last_undegraded":"2024-08-30T08:58:46.557615+0000","last_fullsized":"2024-08-30T08:58:46.557615+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T11:58:25.037431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.822738+0000","last_change":"2024-08-30T08:58:02.415594+0000","last_active":"2024-08-30T08:58:46.822738+0000","last_peered":"2024-08-30T08:58:46.822738+0000","last_clean":"2024-08-30T08:58:46.822738+0000","last_became_active":"2024-08-30T08:58:02.415066+0000","last_became_peered":"2024-08-30T08:58:02.415066+0000","last_unstale":"2024-08-30T08:58:46.822738+0000","last_undegraded":"2024-08-30T08:58:46.822738+0000","last_fullsized":"2024-08-30T08:58:46.822738+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T12:07:27.668442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:47.089672+0000","last_change":"2024-08-30T08:58:09.444403+0000","last_active":"2024-08-30T08:58:47.089672+0000","last_peered":"2024-08-30T08:58:47.089672+0000","last_clean":"2024-08-30T08:58:47.089672+0000","last_became_active":"2024-08-30T08:58:09.443914+0000","last_became_peered":"2024-08-30T08:58:09.443914+0000","last_unstale":"2024-08-30T08:58:47.089672+0000","last_undegraded":"2024-08-30T08:58:47.089672+0000","last_fullsized":"2024-08-30T08:58:47.089672+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T14:17:43.866352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"73'2","reported_seq":37,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.557996+0000","last_change":"2024-08-30T08:58:00.392108+0000","last_active":"2024-08-30T08:58:46.557996+0000","last_peered":"2024-08-30T08:58:46.557996+0000","last_clean":"2024-08-30T08:58:46.557996+0000","last_became_active":"2024-08-30T08:58:00.391157+0000","last_became_peered":"2024-08-30T08:58:00.391157+0000","last_unstale":"2024-08-30T08:58:46.557996+0000","last_undegraded":"2024-08-30T08:58:46.557996+0000","last_fullsized":"2024-08-30T08:58:46.557996+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T17:21:03.369776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.774221+0000","last_change":"2024-08-30T08:58:02.405550+0000","last_active":"2024-08-30T08:58:46.774221+0000","last_peered":"2024-08-30T08:58:46.774221+0000","last_clean":"2024-08-30T08:58:46.774221+0000","last_became_active":"2024-08-30T08:58:02.405129+0000","last_became_peered":"2024-08-30T08:58:02.405129+0000","last_unstale":"2024-08-30T08:58:46.774221+0000","last_undegraded":"2024-08-30T08:58:46.774221+0000","last_fullsized":"2024-08-30T08:58:46.774221+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T11:36:47.624026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:47.089364+0000","last_change":"2024-08-30T08:58:09.445746+0000","last_active":"2024-08-30T08:58:47.089364+0000","last_peered":"2024-08-30T08:58:47.089364+0000","last_clean":"2024-08-30T08:58:47.089364+0000","last_became_active":"2024-08-30T08:58:09.445257+0000","last_became_peered":"2024-08-30T08:58:09.445257+0000","last_unstale":"2024-08-30T08:58:47.089364+0000","last_undegraded":"2024-08-30T08:58:47.089364+0000","last_fullsized":"2024-08-30T08:58:47.089364+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T18:02:52.007131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:47.090324+0000","last_change":"2024-08-30T08:58:02.408213+0000","last_active":"2024-08-30T08:58:47.090324+0000","last_peered":"2024-08-30T08:58:47.090324+0000","last_clean":"2024-08-30T08:58:47.090324+0000","last_became_active":"2024-08-30T08:58:02.407856+0000","last_became_peered":"2024-08-30T08:58:02.407856+0000","last_unstale":"2024-08-30T08:58:47.090324+0000","last_undegraded":"2024-08-30T08:58:47.090324+0000","last_fullsized":"2024-08-30T08:58:47.090324+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T12:00:22.226653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"79'4","reported_seq":40,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.557487+0000","last_change":"2024-08-30T08:58:00.391714+0000","last_active":"2024-08-30T08:58:46.557487+0000","last_peered":"2024-08-30T08:58:46.557487+0000","last_clean":"2024-08-30T08:58:46.557487+0000","last_became_active":"2024-08-30T08:58:00.391211+0000","last_became_peered":"2024-08-30T08:58:00.391211+0000","last_unstale":"2024-08-30T08:58:46.557487+0000","last_undegraded":"2024-08-30T08:58:46.557487+0000","last_fullsized":"2024-08-30T08:58:46.557487+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T11:55:37.469983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:47.089359+0000","last_change":"2024-08-30T08:58:09.728177+0000","last_active":"2024-08-30T08:58:47.089359+0000","last_peered":"2024-08-30T08:58:47.089359+0000","last_clean":"2024-08-30T08:58:47.089359+0000","last_became_active":"2024-08-30T08:58:09.727630+0000","last_became_peered":"2024-08-30T08:58:09.727630+0000","last_unstale":"2024-08-30T08:58:47.089359+0000","last_undegraded":"2024-08-30T08:58:47.089359+0000","last_fullsized":"2024-08-30T08:58:47.089359+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T12:07:52.965014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.554094+0000","last_change":"2024-08-30T08:58:02.403378+0000","last_active":"2024-08-30T08:58:46.554094+0000","last_peered":"2024-08-30T08:58:46.554094+0000","last_clean":"2024-08-30T08:58:46.554094+0000","last_became_active":"2024-08-30T08:58:02.403036+0000","last_became_peered":"2024-08-30T08:58:02.403036+0000","last_unstale":"2024-08-30T08:58:46.554094+0000","last_undegraded":"2024-08-30T08:58:46.554094+0000","last_fullsized":"2024-08-30T08:58:46.554094+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T16:37:14.876384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"80'1","reported_seq":36,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.555381+0000","last_change":"2024-08-30T08:58:00.393860+0000","last_active":"2024-08-30T08:58:46.555381+0000","last_peered":"2024-08-30T08:58:46.555381+0000","last_clean":"2024-08-30T08:58:46.555381+0000","last_became_active":"2024-08-30T08:58:00.392486+0000","last_became_peered":"2024-08-30T08:58:00.392486+0000","last_unstale":"2024-08-30T08:58:46.555381+0000","last_undegraded":"2024-08-30T08:58:46.555381+0000","last_fullsized":"2024-08-30T08:58:46.555381+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T12:50:27.337817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"81'1","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:47.089678+0000","last_change":"2024-08-30T08:58:09.441476+0000","last_active":"2024-08-30T08:58:47.089678+0000","last_peered":"2024-08-30T08:58:47.089678+0000","last_clean":"2024-08-30T08:58:47.089678+0000","last_became_active":"2024-08-30T08:58:09.440344+0000","last_became_peered":"2024-08-30T08:58:09.440344+0000","last_unstale":"2024-08-30T08:58:47.089678+0000","last_undegraded":"2024-08-30T08:58:47.089678+0000","last_fullsized":"2024-08-30T08:58:47.089678+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T12:06:47.113611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.553171+0000","last_change":"2024-08-30T08:58:02.407098+0000","last_active":"2024-08-30T08:58:46.553171+0000","last_peered":"2024-08-30T08:58:46.553171+0000","last_clean":"2024-08-30T08:58:46.553171+0000","last_became_active":"2024-08-30T08:58:02.406690+0000","last_became_peered":"2024-08-30T08:58:02.406690+0000","last_unstale":"2024-08-30T08:58:46.553171+0000","last_undegraded":"2024-08-30T08:58:46.553171+0000","last_fullsized":"2024-08-30T08:58:46.553171+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T18:27:39.534923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"79'3","reported_seq":38,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.554200+0000","last_change":"2024-08-30T08:58:00.386434+0000","last_active":"2024-08-30T08:58:46.554200+0000","last_peered":"2024-08-30T08:58:46.554200+0000","last_clean":"2024-08-30T08:58:46.554200+0000","last_became_active":"2024-08-30T08:58:00.385930+0000","last_became_peered":"2024-08-30T08:58:00.385930+0000","last_unstale":"2024-08-30T08:58:46.554200+0000","last_undegraded":"2024-08-30T08:58:46.554200+0000","last_fullsized":"2024-08-30T08:58:46.554200+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T10:39:22.441568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.775800+0000","last_change":"2024-08-30T08:58:46.775674+0000","last_active":"2024-08-30T08:58:46.775800+0000","last_peered":"2024-08-30T08:58:46.775800+0000","last_clean":"2024-08-30T08:58:46.775800+0000","last_became_active":"2024-08-30T08:58:46.775157+0000","last_became_peered":"2024-08-30T08:58:46.775157+0000","last_unstale":"2024-08-30T08:58:46.775800+0000","last_undegraded":"2024-08-30T08:58:46.775800+0000","last_fullsized":"2024-08-30T08:58:46.775800+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T17:13:47.066475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,2,5],"acting":[8,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.559287+0000","last_change":"2024-08-30T08:58:02.410500+0000","last_active":"2024-08-30T08:58:46.559287+0000","last_peered":"2024-08-30T08:58:46.559287+0000","last_clean":"2024-08-30T08:58:46.559287+0000","last_became_active":"2024-08-30T08:58:02.409260+0000","last_became_peered":"2024-08-30T08:58:02.409260+0000","last_unstale":"2024-08-30T08:58:46.559287+0000","last_undegraded":"2024-08-30T08:58:46.559287+0000","last_fullsized":"2024-08-30T08:58:46.559287+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T16:47:59.701956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"79'2","reported_seq":37,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.822713+0000","last_change":"2024-08-30T08:58:00.482296+0000","last_active":"2024-08-30T08:58:46.822713+0000","last_peered":"2024-08-30T08:58:46.822713+0000","last_clean":"2024-08-30T08:58:46.822713+0000","last_became_active":"2024-08-30T08:58:00.481920+0000","last_became_peered":"2024-08-30T08:58:00.481920+0000","last_unstale":"2024-08-30T08:58:46.822713+0000","last_undegraded":"2024-08-30T08:58:46.822713+0000","last_fullsized":"2024-08-30T08:58:46.822713+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T09:29:30.724410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:47.090717+0000","last_change":"2024-08-30T08:58:09.727879+0000","last_active":"2024-08-30T08:58:47.090717+0000","last_peered":"2024-08-30T08:58:47.090717+0000","last_clean":"2024-08-30T08:58:47.090717+0000","last_became_active":"2024-08-30T08:58:09.727339+0000","last_became_peered":"2024-08-30T08:58:09.727339+0000","last_unstale":"2024-08-30T08:58:47.090717+0000","last_undegraded":"2024-08-30T08:58:47.090717+0000","last_fullsized":"2024-08-30T08:58:47.090717+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T13:19:18.582668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":37,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.560099+0000","last_change":"2024-08-30T08:58:46.559988+0000","last_active":"2024-08-30T08:58:46.560099+0000","last_peered":"2024-08-30T08:58:46.560099+0000","last_clean":"2024-08-30T08:58:46.560099+0000","last_became_active":"2024-08-30T08:58:46.559442+0000","last_became_peered":"2024-08-30T08:58:46.559442+0000","last_unstale":"2024-08-30T08:58:46.560099+0000","last_undegraded":"2024-08-30T08:58:46.560099+0000","last_fullsized":"2024-08-30T08:58:46.560099+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T09:29:09.747756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"79'4","reported_seq":39,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:47.091026+0000","last_change":"2024-08-30T08:58:00.380338+0000","last_active":"2024-08-30T08:58:47.091026+0000","last_peered":"2024-08-30T08:58:47.091026+0000","last_clean":"2024-08-30T08:58:47.091026+0000","last_became_active":"2024-08-30T08:58:00.379860+0000","last_became_peered":"2024-08-30T08:58:00.379860+0000","last_unstale":"2024-08-30T08:58:47.091026+0000","last_undegraded":"2024-08-30T08:58:47.091026+0000","last_fullsized":"2024-08-30T08:58:47.091026+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T19:57:15.349906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.555200+0000","last_change":"2024-08-30T08:58:09.443631+0000","last_active":"2024-08-30T08:58:46.555200+0000","last_peered":"2024-08-30T08:58:46.555200+0000","last_clean":"2024-08-30T08:58:46.555200+0000","last_became_active":"2024-08-30T08:58:09.443103+0000","last_became_peered":"2024-08-30T08:58:09.443103+0000","last_unstale":"2024-08-30T08:58:46.555200+0000","last_undegraded":"2024-08-30T08:58:46.555200+0000","last_fullsized":"2024-08-30T08:58:46.555200+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T19:58:11.031826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.555619+0000","last_change":"2024-08-30T08:58:02.406216+0000","last_active":"2024-08-30T08:58:46.555619+0000","last_peered":"2024-08-30T08:58:46.555619+0000","last_clean":"2024-08-30T08:58:46.555619+0000","last_became_active":"2024-08-30T08:58:02.405040+0000","last_became_peered":"2024-08-30T08:58:02.405040+0000","last_unstale":"2024-08-30T08:58:46.555619+0000","last_undegraded":"2024-08-30T08:58:46.555619+0000","last_fullsized":"2024-08-30T08:58:46.555619+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T14:28:28.721386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"81'9","reported_seq":44,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:57.412986+0000","last_change":"2024-08-30T08:58:00.387350+0000","last_active":"2024-08-30T08:58:57.412986+0000","last_peered":"2024-08-30T08:58:57.412986+0000","last_clean":"2024-08-30T08:58:57.412986+0000","last_became_active":"2024-08-30T08:58:00.387004+0000","last_became_peered":"2024-08-30T08:58:00.387004+0000","last_unstale":"2024-08-30T08:58:57.412986+0000","last_undegraded":"2024-08-30T08:58:57.412986+0000","last_fullsized":"2024-08-30T08:58:57.412986+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T19:24:25.604544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.553684+0000","last_change":"2024-08-30T08:58:09.727388+0000","last_active":"2024-08-30T08:58:46.553684+0000","last_peered":"2024-08-30T08:58:46.553684+0000","last_clean":"2024-08-30T08:58:46.553684+0000","last_became_active":"2024-08-30T08:58:09.726935+0000","last_became_peered":"2024-08-30T08:58:09.726935+0000","last_unstale":"2024-08-30T08:58:46.553684+0000","last_undegraded":"2024-08-30T08:58:46.553684+0000","last_fullsized":"2024-08-30T08:58:46.553684+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T17:53:16.066602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.552226+0000","last_change":"2024-08-30T08:58:02.402661+0000","last_active":"2024-08-30T08:58:46.552226+0000","last_peered":"2024-08-30T08:58:46.552226+0000","last_clean":"2024-08-30T08:58:46.552226+0000","last_became_active":"2024-08-30T08:58:02.402314+0000","last_became_peered":"2024-08-30T08:58:02.402314+0000","last_unstale":"2024-08-30T08:58:46.552226+0000","last_undegraded":"2024-08-30T08:58:46.552226+0000","last_fullsized":"2024-08-30T08:58:46.552226+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T10:56:41.336312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"79'1","reported_seq":37,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.557749+0000","last_change":"2024-08-30T08:58:00.482720+0000","last_active":"2024-08-30T08:58:46.557749+0000","last_peered":"2024-08-30T08:58:46.557749+0000","last_clean":"2024-08-30T08:58:46.557749+0000","last_became_active":"2024-08-30T08:58:00.482193+0000","last_became_peered":"2024-08-30T08:58:00.482193+0000","last_unstale":"2024-08-30T08:58:46.557749+0000","last_undegraded":"2024-08-30T08:58:46.557749+0000","last_fullsized":"2024-08-30T08:58:46.557749+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T18:42:06.654806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.553347+0000","last_change":"2024-08-30T08:58:09.449970+0000","last_active":"2024-08-30T08:58:46.553347+0000","last_peered":"2024-08-30T08:58:46.553347+0000","last_clean":"2024-08-30T08:58:46.553347+0000","last_became_active":"2024-08-30T08:58:09.447891+0000","last_became_peered":"2024-08-30T08:58:09.447891+0000","last_unstale":"2024-08-30T08:58:46.553347+0000","last_undegraded":"2024-08-30T08:58:46.553347+0000","last_fullsized":"2024-08-30T08:58:46.553347+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T15:06:45.414470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.553365+0000","last_change":"2024-08-30T08:58:02.409756+0000","last_active":"2024-08-30T08:58:46.553365+0000","last_peered":"2024-08-30T08:58:46.553365+0000","last_clean":"2024-08-30T08:58:46.553365+0000","last_became_active":"2024-08-30T08:58:02.409357+0000","last_became_peered":"2024-08-30T08:58:02.409357+0000","last_unstale":"2024-08-30T08:58:46.553365+0000","last_undegraded":"2024-08-30T08:58:46.553365+0000","last_fullsized":"2024-08-30T08:58:46.553365+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T13:27:41.696527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":35,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.558745+0000","last_change":"2024-08-30T08:58:00.482342+0000","last_active":"2024-08-30T08:58:46.558745+0000","last_peered":"2024-08-30T08:58:46.558745+0000","last_clean":"2024-08-30T08:58:46.558745+0000","last_became_active":"2024-08-30T08:58:00.481902+0000","last_became_peered":"2024-08-30T08:58:00.481902+0000","last_unstale":"2024-08-30T08:58:46.558745+0000","last_undegraded":"2024-08-30T08:58:46.558745+0000","last_fullsized":"2024-08-30T08:58:46.558745+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T14:12:36.789356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.556901+0000","last_change":"2024-08-30T08:58:09.728008+0000","last_active":"2024-08-30T08:58:46.556901+0000","last_peered":"2024-08-30T08:58:46.556901+0000","last_clean":"2024-08-30T08:58:46.556901+0000","last_became_active":"2024-08-30T08:58:09.727513+0000","last_became_peered":"2024-08-30T08:58:09.727513+0000","last_unstale":"2024-08-30T08:58:46.556901+0000","last_undegraded":"2024-08-30T08:58:46.556901+0000","last_fullsized":"2024-08-30T08:58:46.556901+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T14:34:04.165300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.553329+0000","last_change":"2024-08-30T08:58:02.409383+0000","last_active":"2024-08-30T08:58:46.553329+0000","last_peered":"2024-08-30T08:58:46.553329+0000","last_clean":"2024-08-30T08:58:46.553329+0000","last_became_active":"2024-08-30T08:58:02.409027+0000","last_became_peered":"2024-08-30T08:58:02.409027+0000","last_unstale":"2024-08-30T08:58:46.553329+0000","last_undegraded":"2024-08-30T08:58:46.553329+0000","last_fullsized":"2024-08-30T08:58:46.553329+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T10:44:23.467130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"80'1","reported_seq":36,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:47.090595+0000","last_change":"2024-08-30T08:58:00.389083+0000","last_active":"2024-08-30T08:58:47.090595+0000","last_peered":"2024-08-30T08:58:47.090595+0000","last_clean":"2024-08-30T08:58:47.090595+0000","last_became_active":"2024-08-30T08:58:00.387552+0000","last_became_peered":"2024-08-30T08:58:00.387552+0000","last_unstale":"2024-08-30T08:58:47.090595+0000","last_undegraded":"2024-08-30T08:58:47.090595+0000","last_fullsized":"2024-08-30T08:58:47.090595+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T11:56:38.827406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.552061+0000","last_change":"2024-08-30T08:58:09.447758+0000","last_active":"2024-08-30T08:58:46.552061+0000","last_peered":"2024-08-30T08:58:46.552061+0000","last_clean":"2024-08-30T08:58:46.552061+0000","last_became_active":"2024-08-30T08:58:09.447333+0000","last_became_peered":"2024-08-30T08:58:09.447333+0000","last_unstale":"2024-08-30T08:58:46.552061+0000","last_undegraded":"2024-08-30T08:58:46.552061+0000","last_fullsized":"2024-08-30T08:58:46.552061+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T12:43:35.289967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:47.091640+0000","last_change":"2024-08-30T08:58:47.091191+0000","last_active":"2024-08-30T08:58:47.091640+0000","last_peered":"2024-08-30T08:58:47.091640+0000","last_clean":"2024-08-30T08:58:47.091640+0000","last_became_active":"2024-08-30T08:58:47.090826+0000","last_became_peered":"2024-08-30T08:58:47.090826+0000","last_unstale":"2024-08-30T08:58:47.091640+0000","last_undegraded":"2024-08-30T08:58:47.091640+0000","last_fullsized":"2024-08-30T08:58:47.091640+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T17:47:22.862514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"80'4","reported_seq":52,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:59:01.632921+0000","last_change":"2024-08-30T08:58:00.480651+0000","last_active":"2024-08-30T08:59:01.632921+0000","last_peered":"2024-08-30T08:59:01.632921+0000","last_clean":"2024-08-30T08:59:01.632921+0000","last_became_active":"2024-08-30T08:58:00.480205+0000","last_became_peered":"2024-08-30T08:58:00.480205+0000","last_unstale":"2024-08-30T08:59:01.632921+0000","last_undegraded":"2024-08-30T08:59:01.632921+0000","last_fullsized":"2024-08-30T08:59:01.632921+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T16:28:30.687076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":13,"num_read_kb":13,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.552066+0000","last_change":"2024-08-30T08:58:09.727530+0000","last_active":"2024-08-30T08:58:46.552066+0000","last_peered":"2024-08-30T08:58:46.552066+0000","last_clean":"2024-08-30T08:58:46.552066+0000","last_became_active":"2024-08-30T08:58:09.726965+0000","last_became_peered":"2024-08-30T08:58:09.726965+0000","last_unstale":"2024-08-30T08:58:46.552066+0000","last_undegraded":"2024-08-30T08:58:46.552066+0000","last_fullsized":"2024-08-30T08:58:46.552066+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T12:56:42.885615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.551578+0000","last_change":"2024-08-30T08:58:02.401673+0000","last_active":"2024-08-30T08:58:46.551578+0000","last_peered":"2024-08-30T08:58:46.551578+0000","last_clean":"2024-08-30T08:58:46.551578+0000","last_became_active":"2024-08-30T08:58:02.398876+0000","last_became_peered":"2024-08-30T08:58:02.398876+0000","last_unstale":"2024-08-30T08:58:46.551578+0000","last_undegraded":"2024-08-30T08:58:46.551578+0000","last_fullsized":"2024-08-30T08:58:46.551578+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T16:13:20.893737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"80'3","reported_seq":38,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:47.091220+0000","last_change":"2024-08-30T08:58:00.391722+0000","last_active":"2024-08-30T08:58:47.091220+0000","last_peered":"2024-08-30T08:58:47.091220+0000","last_clean":"2024-08-30T08:58:47.091220+0000","last_became_active":"2024-08-30T08:58:00.391325+0000","last_became_peered":"2024-08-30T08:58:00.391325+0000","last_unstale":"2024-08-30T08:58:47.091220+0000","last_undegraded":"2024-08-30T08:58:47.091220+0000","last_fullsized":"2024-08-30T08:58:47.091220+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T11:58:08.189202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.774243+0000","last_change":"2024-08-30T08:58:09.728324+0000","last_active":"2024-08-30T08:58:46.774243+0000","last_peered":"2024-08-30T08:58:46.774243+0000","last_clean":"2024-08-30T08:58:46.774243+0000","last_became_active":"2024-08-30T08:58:09.727520+0000","last_became_peered":"2024-08-30T08:58:09.727520+0000","last_unstale":"2024-08-30T08:58:46.774243+0000","last_undegraded":"2024-08-30T08:58:46.774243+0000","last_fullsized":"2024-08-30T08:58:46.774243+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T10:09:39.046820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.554063+0000","last_change":"2024-08-30T08:58:02.410826+0000","last_active":"2024-08-30T08:58:46.554063+0000","last_peered":"2024-08-30T08:58:46.554063+0000","last_clean":"2024-08-30T08:58:46.554063+0000","last_became_active":"2024-08-30T08:58:02.410464+0000","last_became_peered":"2024-08-30T08:58:02.410464+0000","last_unstale":"2024-08-30T08:58:46.554063+0000","last_undegraded":"2024-08-30T08:58:46.554063+0000","last_fullsized":"2024-08-30T08:58:46.554063+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T12:14:59.666614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"80'3","reported_seq":40,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:50.596022+0000","last_change":"2024-08-30T08:58:00.385464+0000","last_active":"2024-08-30T08:58:50.596022+0000","last_peered":"2024-08-30T08:58:50.596022+0000","last_clean":"2024-08-30T08:58:50.596022+0000","last_became_active":"2024-08-30T08:58:00.385083+0000","last_became_peered":"2024-08-30T08:58:00.385083+0000","last_unstale":"2024-08-30T08:58:50.596022+0000","last_undegraded":"2024-08-30T08:58:50.596022+0000","last_fullsized":"2024-08-30T08:58:50.596022+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T16:13:52.304755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":107,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:47.091434+0000","last_change":"2024-08-30T08:56:49.645080+0000","last_active":"2024-08-30T08:58:47.091434+0000","last_peered":"2024-08-30T08:58:47.091434+0000","last_clean":"2024-08-30T08:58:47.091434+0000","last_became_active":"2024-08-30T08:56:49.639066+0000","last_became_peered":"2024-08-30T08:56:49.639066+0000","last_unstale":"2024-08-30T08:58:47.091434+0000","last_undegraded":"2024-08-30T08:58:47.091434+0000","last_fullsized":"2024-08-30T08:58:47.091434+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:53:27.566846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:53:27.566846+0000","last_clean_scrub_stamp":"2024-08-30T08:53:27.566846+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T20:23:45.723824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"81'1","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:47.090916+0000","last_change":"2024-08-30T08:58:09.444007+0000","last_active":"2024-08-30T08:58:47.090916+0000","last_peered":"2024-08-30T08:58:47.090916+0000","last_clean":"2024-08-30T08:58:47.090916+0000","last_became_active":"2024-08-30T08:58:09.443580+0000","last_became_peered":"2024-08-30T08:58:09.443580+0000","last_unstale":"2024-08-30T08:58:47.090916+0000","last_undegraded":"2024-08-30T08:58:47.090916+0000","last_fullsized":"2024-08-30T08:58:47.090916+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T17:00:58.654026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:47.091022+0000","last_change":"2024-08-30T08:58:02.406973+0000","last_active":"2024-08-30T08:58:47.091022+0000","last_peered":"2024-08-30T08:58:47.091022+0000","last_clean":"2024-08-30T08:58:47.091022+0000","last_became_active":"2024-08-30T08:58:02.406576+0000","last_became_peered":"2024-08-30T08:58:02.406576+0000","last_unstale":"2024-08-30T08:58:47.091022+0000","last_undegraded":"2024-08-30T08:58:47.091022+0000","last_fullsized":"2024-08-30T08:58:47.091022+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T16:04:48.785310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"80'3","reported_seq":39,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.556017+0000","last_change":"2024-08-30T08:58:00.481019+0000","last_active":"2024-08-30T08:58:46.556017+0000","last_peered":"2024-08-30T08:58:46.556017+0000","last_clean":"2024-08-30T08:58:46.556017+0000","last_became_active":"2024-08-30T08:58:00.480590+0000","last_became_peered":"2024-08-30T08:58:00.480590+0000","last_unstale":"2024-08-30T08:58:46.556017+0000","last_undegraded":"2024-08-30T08:58:46.556017+0000","last_fullsized":"2024-08-30T08:58:46.556017+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T13:20:46.731591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.822612+0000","last_change":"2024-08-30T08:58:09.445723+0000","last_active":"2024-08-30T08:58:46.822612+0000","last_peered":"2024-08-30T08:58:46.822612+0000","last_clean":"2024-08-30T08:58:46.822612+0000","last_became_active":"2024-08-30T08:58:09.445252+0000","last_became_peered":"2024-08-30T08:58:09.445252+0000","last_unstale":"2024-08-30T08:58:46.822612+0000","last_undegraded":"2024-08-30T08:58:46.822612+0000","last_fullsized":"2024-08-30T08:58:46.822612+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T12:16:12.255990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:47.090625+0000","last_change":"2024-08-30T08:58:02.414250+0000","last_active":"2024-08-30T08:58:47.090625+0000","last_peered":"2024-08-30T08:58:47.090625+0000","last_clean":"2024-08-30T08:58:47.090625+0000","last_became_active":"2024-08-30T08:58:02.413684+0000","last_became_peered":"2024-08-30T08:58:02.413684+0000","last_unstale":"2024-08-30T08:58:47.090625+0000","last_undegraded":"2024-08-30T08:58:47.090625+0000","last_fullsized":"2024-08-30T08:58:47.090625+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T16:55:12.878806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"73'1","reported_seq":37,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.774624+0000","last_change":"2024-08-30T08:58:00.388869+0000","last_active":"2024-08-30T08:58:46.774624+0000","last_peered":"2024-08-30T08:58:46.774624+0000","last_clean":"2024-08-30T08:58:46.774624+0000","last_became_active":"2024-08-30T08:58:00.388116+0000","last_became_peered":"2024-08-30T08:58:00.388116+0000","last_unstale":"2024-08-30T08:58:46.774624+0000","last_undegraded":"2024-08-30T08:58:46.774624+0000","last_fullsized":"2024-08-30T08:58:46.774624+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T09:50:48.449205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:47.092091+0000","last_change":"2024-08-30T08:58:47.092004+0000","last_active":"2024-08-30T08:58:47.092091+0000","last_peered":"2024-08-30T08:58:47.092091+0000","last_clean":"2024-08-30T08:58:47.092091+0000","last_became_active":"2024-08-30T08:58:47.091661+0000","last_became_peered":"2024-08-30T08:58:47.091661+0000","last_unstale":"2024-08-30T08:58:47.092091+0000","last_undegraded":"2024-08-30T08:58:47.092091+0000","last_fullsized":"2024-08-30T08:58:47.092091+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T09:14:55.353465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,1,4],"acting":[0,8,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:47.090626+0000","last_change":"2024-08-30T08:58:02.410891+0000","last_active":"2024-08-30T08:58:47.090626+0000","last_peered":"2024-08-30T08:58:47.090626+0000","last_clean":"2024-08-30T08:58:47.090626+0000","last_became_active":"2024-08-30T08:58:02.410522+0000","last_became_peered":"2024-08-30T08:58:02.410522+0000","last_unstale":"2024-08-30T08:58:47.090626+0000","last_undegraded":"2024-08-30T08:58:47.090626+0000","last_fullsized":"2024-08-30T08:58:47.090626+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T09:32:21.540298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"73'2","reported_seq":48,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:47.095913+0000","last_change":"2024-08-30T08:58:47.095801+0000","last_active":"2024-08-30T08:58:47.095913+0000","last_peered":"2024-08-30T08:58:47.095913+0000","last_clean":"2024-08-30T08:58:47.095913+0000","last_became_active":"2024-08-30T08:58:47.092756+0000","last_became_peered":"2024-08-30T08:58:47.092756+0000","last_unstale":"2024-08-30T08:58:47.095913+0000","last_undegraded":"2024-08-30T08:58:47.095913+0000","last_fullsized":"2024-08-30T08:58:47.095913+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T19:10:02.064649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.773596+0000","last_change":"2024-08-30T08:58:09.449210+0000","last_active":"2024-08-30T08:58:46.773596+0000","last_peered":"2024-08-30T08:58:46.773596+0000","last_clean":"2024-08-30T08:58:46.773596+0000","last_became_active":"2024-08-30T08:58:09.448769+0000","last_became_peered":"2024-08-30T08:58:09.448769+0000","last_unstale":"2024-08-30T08:58:46.773596+0000","last_undegraded":"2024-08-30T08:58:46.773596+0000","last_fullsized":"2024-08-30T08:58:46.773596+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T09:26:13.845520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.553484+0000","last_change":"2024-08-30T08:58:02.414152+0000","last_active":"2024-08-30T08:58:46.553484+0000","last_peered":"2024-08-30T08:58:46.553484+0000","last_clean":"2024-08-30T08:58:46.553484+0000","last_became_active":"2024-08-30T08:58:02.413768+0000","last_became_peered":"2024-08-30T08:58:02.413768+0000","last_unstale":"2024-08-30T08:58:46.553484+0000","last_undegraded":"2024-08-30T08:58:46.553484+0000","last_fullsized":"2024-08-30T08:58:46.553484+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T18:34:09.447071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"81'2","reported_seq":38,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:50.596425+0000","last_change":"2024-08-30T08:58:00.390839+0000","last_active":"2024-08-30T08:58:50.596425+0000","last_peered":"2024-08-30T08:58:50.596425+0000","last_clean":"2024-08-30T08:58:50.596425+0000","last_became_active":"2024-08-30T08:58:00.390422+0000","last_became_peered":"2024-08-30T08:58:00.390422+0000","last_unstale":"2024-08-30T08:58:50.596425+0000","last_undegraded":"2024-08-30T08:58:50.596425+0000","last_fullsized":"2024-08-30T08:58:50.596425+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T15:05:23.005075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.824657+0000","last_change":"2024-08-30T08:58:46.824563+0000","last_active":"2024-08-30T08:58:46.824657+0000","last_peered":"2024-08-30T08:58:46.824657+0000","last_clean":"2024-08-30T08:58:46.824657+0000","last_became_active":"2024-08-30T08:58:46.824163+0000","last_became_peered":"2024-08-30T08:58:46.824163+0000","last_unstale":"2024-08-30T08:58:46.824657+0000","last_undegraded":"2024-08-30T08:58:46.824657+0000","last_fullsized":"2024-08-30T08:58:46.824657+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T19:25:13.202762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,3,1],"acting":[8,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.773839+0000","last_change":"2024-08-30T08:58:02.405122+0000","last_active":"2024-08-30T08:58:46.773839+0000","last_peered":"2024-08-30T08:58:46.773839+0000","last_clean":"2024-08-30T08:58:46.773839+0000","last_became_active":"2024-08-30T08:58:02.404695+0000","last_became_peered":"2024-08-30T08:58:02.404695+0000","last_unstale":"2024-08-30T08:58:46.773839+0000","last_undegraded":"2024-08-30T08:58:46.773839+0000","last_fullsized":"2024-08-30T08:58:46.773839+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T15:50:01.447032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"81'15","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:47.395221+0000","last_change":"2024-08-30T08:58:46.573221+0000","last_active":"2024-08-30T08:58:47.395221+0000","last_peered":"2024-08-30T08:58:47.395221+0000","last_clean":"2024-08-30T08:58:47.395221+0000","last_became_active":"2024-08-30T08:58:46.559562+0000","last_became_peered":"2024-08-30T08:58:46.559562+0000","last_unstale":"2024-08-30T08:58:47.395221+0000","last_undegraded":"2024-08-30T08:58:47.395221+0000","last_fullsized":"2024-08-30T08:58:47.395221+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T16:21:23.182676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.552177+0000","last_change":"2024-08-30T08:58:09.446063+0000","last_active":"2024-08-30T08:58:46.552177+0000","last_peered":"2024-08-30T08:58:46.552177+0000","last_clean":"2024-08-30T08:58:46.552177+0000","last_became_active":"2024-08-30T08:58:09.445615+0000","last_became_peered":"2024-08-30T08:58:09.445615+0000","last_unstale":"2024-08-30T08:58:46.552177+0000","last_undegraded":"2024-08-30T08:58:46.552177+0000","last_fullsized":"2024-08-30T08:58:46.552177+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T10:25:10.904892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:47.090684+0000","last_change":"2024-08-30T08:58:02.412178+0000","last_active":"2024-08-30T08:58:47.090684+0000","last_peered":"2024-08-30T08:58:47.090684+0000","last_clean":"2024-08-30T08:58:47.090684+0000","last_became_active":"2024-08-30T08:58:02.411814+0000","last_became_peered":"2024-08-30T08:58:02.411814+0000","last_unstale":"2024-08-30T08:58:47.090684+0000","last_undegraded":"2024-08-30T08:58:47.090684+0000","last_fullsized":"2024-08-30T08:58:47.090684+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T10:02:59.070204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"81'4","reported_seq":53,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:59:00.632290+0000","last_change":"2024-08-30T08:58:00.390231+0000","last_active":"2024-08-30T08:59:00.632290+0000","last_peered":"2024-08-30T08:59:00.632290+0000","last_clean":"2024-08-30T08:59:00.632290+0000","last_became_active":"2024-08-30T08:58:00.389852+0000","last_became_peered":"2024-08-30T08:58:00.389852+0000","last_unstale":"2024-08-30T08:59:00.632290+0000","last_undegraded":"2024-08-30T08:59:00.632290+0000","last_fullsized":"2024-08-30T08:59:00.632290+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T12:09:12.105416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":15,"num_read_kb":3,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.558813+0000","last_change":"2024-08-30T08:58:09.441652+0000","last_active":"2024-08-30T08:58:46.558813+0000","last_peered":"2024-08-30T08:58:46.558813+0000","last_clean":"2024-08-30T08:58:46.558813+0000","last_became_active":"2024-08-30T08:58:09.441264+0000","last_became_peered":"2024-08-30T08:58:09.441264+0000","last_unstale":"2024-08-30T08:58:46.558813+0000","last_undegraded":"2024-08-30T08:58:46.558813+0000","last_fullsized":"2024-08-30T08:58:46.558813+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T09:42:25.636669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:47.090297+0000","last_change":"2024-08-30T08:58:02.416347+0000","last_active":"2024-08-30T08:58:47.090297+0000","last_peered":"2024-08-30T08:58:47.090297+0000","last_clean":"2024-08-30T08:58:47.090297+0000","last_became_active":"2024-08-30T08:58:02.415898+0000","last_became_peered":"2024-08-30T08:58:02.415898+0000","last_unstale":"2024-08-30T08:58:47.090297+0000","last_undegraded":"2024-08-30T08:58:47.090297+0000","last_fullsized":"2024-08-30T08:58:47.090297+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T10:18:15.507076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"79'3","reported_seq":38,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.774168+0000","last_change":"2024-08-30T08:58:00.393616+0000","last_active":"2024-08-30T08:58:46.774168+0000","last_peered":"2024-08-30T08:58:46.774168+0000","last_clean":"2024-08-30T08:58:46.774168+0000","last_became_active":"2024-08-30T08:58:00.393218+0000","last_became_peered":"2024-08-30T08:58:00.393218+0000","last_unstale":"2024-08-30T08:58:46.774168+0000","last_undegraded":"2024-08-30T08:58:46.774168+0000","last_fullsized":"2024-08-30T08:58:46.774168+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T10:56:37.175968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:47.093131+0000","last_change":"2024-08-30T08:58:47.092990+0000","last_active":"2024-08-30T08:58:47.093131+0000","last_peered":"2024-08-30T08:58:47.093131+0000","last_clean":"2024-08-30T08:58:47.093131+0000","last_became_active":"2024-08-30T08:58:47.092448+0000","last_became_peered":"2024-08-30T08:58:47.092448+0000","last_unstale":"2024-08-30T08:58:47.093131+0000","last_undegraded":"2024-08-30T08:58:47.093131+0000","last_fullsized":"2024-08-30T08:58:47.093131+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T20:23:06.527336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,7,6],"acting":[11,10,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.551703+0000","last_change":"2024-08-30T08:58:02.405783+0000","last_active":"2024-08-30T08:58:46.551703+0000","last_peered":"2024-08-30T08:58:46.551703+0000","last_clean":"2024-08-30T08:58:46.551703+0000","last_became_active":"2024-08-30T08:58:02.405430+0000","last_became_peered":"2024-08-30T08:58:02.405430+0000","last_unstale":"2024-08-30T08:58:46.551703+0000","last_undegraded":"2024-08-30T08:58:46.551703+0000","last_fullsized":"2024-08-30T08:58:46.551703+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T19:40:08.377259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"80'6","reported_seq":42,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.555590+0000","last_change":"2024-08-30T08:58:00.389617+0000","last_active":"2024-08-30T08:58:46.555590+0000","last_peered":"2024-08-30T08:58:46.555590+0000","last_clean":"2024-08-30T08:58:46.555590+0000","last_became_active":"2024-08-30T08:58:00.389031+0000","last_became_peered":"2024-08-30T08:58:00.389031+0000","last_unstale":"2024-08-30T08:58:46.555590+0000","last_undegraded":"2024-08-30T08:58:46.555590+0000","last_fullsized":"2024-08-30T08:58:46.555590+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T15:42:38.050768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.774094+0000","last_change":"2024-08-30T08:58:09.451925+0000","last_active":"2024-08-30T08:58:46.774094+0000","last_peered":"2024-08-30T08:58:46.774094+0000","last_clean":"2024-08-30T08:58:46.774094+0000","last_became_active":"2024-08-30T08:58:09.446893+0000","last_became_peered":"2024-08-30T08:58:09.446893+0000","last_unstale":"2024-08-30T08:58:46.774094+0000","last_undegraded":"2024-08-30T08:58:46.774094+0000","last_fullsized":"2024-08-30T08:58:46.774094+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T14:34:25.134006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.557378+0000","last_change":"2024-08-30T08:58:02.411194+0000","last_active":"2024-08-30T08:58:46.557378+0000","last_peered":"2024-08-30T08:58:46.557378+0000","last_clean":"2024-08-30T08:58:46.557378+0000","last_became_active":"2024-08-30T08:58:02.410183+0000","last_became_peered":"2024-08-30T08:58:02.410183+0000","last_unstale":"2024-08-30T08:58:46.557378+0000","last_undegraded":"2024-08-30T08:58:46.557378+0000","last_fullsized":"2024-08-30T08:58:46.557378+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T10:56:19.016115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"81'30","reported_seq":96,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:59:01.624908+0000","last_change":"2024-08-30T08:58:00.389808+0000","last_active":"2024-08-30T08:59:01.624908+0000","last_peered":"2024-08-30T08:59:01.624908+0000","last_clean":"2024-08-30T08:59:01.624908+0000","last_became_active":"2024-08-30T08:58:00.389411+0000","last_became_peered":"2024-08-30T08:58:00.389411+0000","last_unstale":"2024-08-30T08:59:01.624908+0000","last_undegraded":"2024-08-30T08:59:01.624908+0000","last_fullsized":"2024-08-30T08:59:01.624908+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T18:59:45.468456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":54392,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":31,"num_read_kb":68,"num_write":31,"num_write_kb":65,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.557593+0000","last_change":"2024-08-30T08:58:09.447360+0000","last_active":"2024-08-30T08:58:46.557593+0000","last_peered":"2024-08-30T08:58:46.557593+0000","last_clean":"2024-08-30T08:58:46.557593+0000","last_became_active":"2024-08-30T08:58:09.446913+0000","last_became_peered":"2024-08-30T08:58:09.446913+0000","last_unstale":"2024-08-30T08:58:46.557593+0000","last_undegraded":"2024-08-30T08:58:46.557593+0000","last_fullsized":"2024-08-30T08:58:46.557593+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T13:59:43.147370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.822816+0000","last_change":"2024-08-30T08:58:02.408920+0000","last_active":"2024-08-30T08:58:46.822816+0000","last_peered":"2024-08-30T08:58:46.822816+0000","last_clean":"2024-08-30T08:58:46.822816+0000","last_became_active":"2024-08-30T08:58:02.408204+0000","last_became_peered":"2024-08-30T08:58:02.408204+0000","last_unstale":"2024-08-30T08:58:46.822816+0000","last_undegraded":"2024-08-30T08:58:46.822816+0000","last_fullsized":"2024-08-30T08:58:46.822816+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T11:44:00.765672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"79'4","reported_seq":40,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.774530+0000","last_change":"2024-08-30T08:58:00.482348+0000","last_active":"2024-08-30T08:58:46.774530+0000","last_peered":"2024-08-30T08:58:46.774530+0000","last_clean":"2024-08-30T08:58:46.774530+0000","last_became_active":"2024-08-30T08:58:00.481929+0000","last_became_peered":"2024-08-30T08:58:00.481929+0000","last_unstale":"2024-08-30T08:58:46.774530+0000","last_undegraded":"2024-08-30T08:58:46.774530+0000","last_fullsized":"2024-08-30T08:58:46.774530+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T11:13:55.332639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:47.089847+0000","last_change":"2024-08-30T08:58:09.443259+0000","last_active":"2024-08-30T08:58:47.089847+0000","last_peered":"2024-08-30T08:58:47.089847+0000","last_clean":"2024-08-30T08:58:47.089847+0000","last_became_active":"2024-08-30T08:58:09.442209+0000","last_became_peered":"2024-08-30T08:58:09.442209+0000","last_unstale":"2024-08-30T08:58:47.089847+0000","last_undegraded":"2024-08-30T08:58:47.089847+0000","last_fullsized":"2024-08-30T08:58:47.089847+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T09:25:56.233525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.557115+0000","last_change":"2024-08-30T08:58:02.407183+0000","last_active":"2024-08-30T08:58:46.557115+0000","last_peered":"2024-08-30T08:58:46.557115+0000","last_clean":"2024-08-30T08:58:46.557115+0000","last_became_active":"2024-08-30T08:58:02.406838+0000","last_became_peered":"2024-08-30T08:58:02.406838+0000","last_unstale":"2024-08-30T08:58:46.557115+0000","last_undegraded":"2024-08-30T08:58:46.557115+0000","last_fullsized":"2024-08-30T08:58:46.557115+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T13:40:32.937626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"79'2","reported_seq":37,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:47.090297+0000","last_change":"2024-08-30T08:58:00.392322+0000","last_active":"2024-08-30T08:58:47.090297+0000","last_peered":"2024-08-30T08:58:47.090297+0000","last_clean":"2024-08-30T08:58:47.090297+0000","last_became_active":"2024-08-30T08:58:00.391935+0000","last_became_peered":"2024-08-30T08:58:00.391935+0000","last_unstale":"2024-08-30T08:58:47.090297+0000","last_undegraded":"2024-08-30T08:58:47.090297+0000","last_fullsized":"2024-08-30T08:58:47.090297+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T12:20:52.368285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.557659+0000","last_change":"2024-08-30T08:58:09.441824+0000","last_active":"2024-08-30T08:58:46.557659+0000","last_peered":"2024-08-30T08:58:46.557659+0000","last_clean":"2024-08-30T08:58:46.557659+0000","last_became_active":"2024-08-30T08:58:09.441398+0000","last_became_peered":"2024-08-30T08:58:09.441398+0000","last_unstale":"2024-08-30T08:58:46.557659+0000","last_undegraded":"2024-08-30T08:58:46.557659+0000","last_fullsized":"2024-08-30T08:58:46.557659+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T14:05:45.710280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.823379+0000","last_change":"2024-08-30T08:58:02.403431+0000","last_active":"2024-08-30T08:58:46.823379+0000","last_peered":"2024-08-30T08:58:46.823379+0000","last_clean":"2024-08-30T08:58:46.823379+0000","last_became_active":"2024-08-30T08:58:02.403029+0000","last_became_peered":"2024-08-30T08:58:02.403029+0000","last_unstale":"2024-08-30T08:58:46.823379+0000","last_undegraded":"2024-08-30T08:58:46.823379+0000","last_fullsized":"2024-08-30T08:58:46.823379+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T09:34:56.452675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"79'3","reported_seq":50,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:59:00.615276+0000","last_change":"2024-08-30T08:58:00.389856+0000","last_active":"2024-08-30T08:59:00.615276+0000","last_peered":"2024-08-30T08:59:00.615276+0000","last_clean":"2024-08-30T08:59:00.615276+0000","last_became_active":"2024-08-30T08:58:00.389459+0000","last_became_peered":"2024-08-30T08:58:00.389459+0000","last_unstale":"2024-08-30T08:59:00.615276+0000","last_undegraded":"2024-08-30T08:59:00.615276+0000","last_fullsized":"2024-08-30T08:59:00.615276+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T18:32:06.214570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":12,"num_read_kb":12,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.553852+0000","last_change":"2024-08-30T08:58:09.445317+0000","last_active":"2024-08-30T08:58:46.553852+0000","last_peered":"2024-08-30T08:58:46.553852+0000","last_clean":"2024-08-30T08:58:46.553852+0000","last_became_active":"2024-08-30T08:58:09.444921+0000","last_became_peered":"2024-08-30T08:58:09.444921+0000","last_unstale":"2024-08-30T08:58:46.553852+0000","last_undegraded":"2024-08-30T08:58:46.553852+0000","last_fullsized":"2024-08-30T08:58:46.553852+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T09:22:13.038437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.556314+0000","last_change":"2024-08-30T08:58:09.447469+0000","last_active":"2024-08-30T08:58:46.556314+0000","last_peered":"2024-08-30T08:58:46.556314+0000","last_clean":"2024-08-30T08:58:46.556314+0000","last_became_active":"2024-08-30T08:58:09.447015+0000","last_became_peered":"2024-08-30T08:58:09.447015+0000","last_unstale":"2024-08-30T08:58:46.556314+0000","last_undegraded":"2024-08-30T08:58:46.556314+0000","last_fullsized":"2024-08-30T08:58:46.556314+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T09:18:36.734380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.557549+0000","last_change":"2024-08-30T08:58:02.403371+0000","last_active":"2024-08-30T08:58:46.557549+0000","last_peered":"2024-08-30T08:58:46.557549+0000","last_clean":"2024-08-30T08:58:46.557549+0000","last_became_active":"2024-08-30T08:58:02.403012+0000","last_became_peered":"2024-08-30T08:58:02.403012+0000","last_unstale":"2024-08-30T08:58:46.557549+0000","last_undegraded":"2024-08-30T08:58:46.557549+0000","last_fullsized":"2024-08-30T08:58:46.557549+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T10:29:29.683081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"79'1","reported_seq":36,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.774090+0000","last_change":"2024-08-30T08:58:00.394450+0000","last_active":"2024-08-30T08:58:46.774090+0000","last_peered":"2024-08-30T08:58:46.774090+0000","last_clean":"2024-08-30T08:58:46.774090+0000","last_became_active":"2024-08-30T08:58:00.391843+0000","last_became_peered":"2024-08-30T08:58:00.391843+0000","last_unstale":"2024-08-30T08:58:46.774090+0000","last_undegraded":"2024-08-30T08:58:46.774090+0000","last_fullsized":"2024-08-30T08:58:46.774090+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T10:29:18.947898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.553483+0000","last_change":"2024-08-30T08:58:09.449172+0000","last_active":"2024-08-30T08:58:46.553483+0000","last_peered":"2024-08-30T08:58:46.553483+0000","last_clean":"2024-08-30T08:58:46.553483+0000","last_became_active":"2024-08-30T08:58:09.448687+0000","last_became_peered":"2024-08-30T08:58:09.448687+0000","last_unstale":"2024-08-30T08:58:46.553483+0000","last_undegraded":"2024-08-30T08:58:46.553483+0000","last_fullsized":"2024-08-30T08:58:46.553483+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T13:23:16.090251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.553825+0000","last_change":"2024-08-30T08:58:02.400834+0000","last_active":"2024-08-30T08:58:46.553825+0000","last_peered":"2024-08-30T08:58:46.553825+0000","last_clean":"2024-08-30T08:58:46.553825+0000","last_became_active":"2024-08-30T08:58:02.400477+0000","last_became_peered":"2024-08-30T08:58:02.400477+0000","last_unstale":"2024-08-30T08:58:46.553825+0000","last_undegraded":"2024-08-30T08:58:46.553825+0000","last_fullsized":"2024-08-30T08:58:46.553825+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T13:40:45.239732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"80'2","reported_seq":37,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.551581+0000","last_change":"2024-08-30T08:58:00.381699+0000","last_active":"2024-08-30T08:58:46.551581+0000","last_peered":"2024-08-30T08:58:46.551581+0000","last_clean":"2024-08-30T08:58:46.551581+0000","last_became_active":"2024-08-30T08:58:00.381189+0000","last_became_peered":"2024-08-30T08:58:00.381189+0000","last_unstale":"2024-08-30T08:58:46.551581+0000","last_undegraded":"2024-08-30T08:58:46.551581+0000","last_fullsized":"2024-08-30T08:58:46.551581+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T14:09:16.939062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"80'3","reported_seq":38,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.553179+0000","last_change":"2024-08-30T08:58:00.387430+0000","last_active":"2024-08-30T08:58:46.553179+0000","last_peered":"2024-08-30T08:58:46.553179+0000","last_clean":"2024-08-30T08:58:46.553179+0000","last_became_active":"2024-08-30T08:58:00.387043+0000","last_became_peered":"2024-08-30T08:58:00.387043+0000","last_unstale":"2024-08-30T08:58:46.553179+0000","last_undegraded":"2024-08-30T08:58:46.553179+0000","last_fullsized":"2024-08-30T08:58:46.553179+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T13:32:25.418105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.774175+0000","last_change":"2024-08-30T08:58:02.404798+0000","last_active":"2024-08-30T08:58:46.774175+0000","last_peered":"2024-08-30T08:58:46.774175+0000","last_clean":"2024-08-30T08:58:46.774175+0000","last_became_active":"2024-08-30T08:58:02.404435+0000","last_became_peered":"2024-08-30T08:58:02.404435+0000","last_unstale":"2024-08-30T08:58:46.774175+0000","last_undegraded":"2024-08-30T08:58:46.774175+0000","last_fullsized":"2024-08-30T08:58:46.774175+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T20:31:49.472832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.555057+0000","last_change":"2024-08-30T08:58:09.448297+0000","last_active":"2024-08-30T08:58:46.555057+0000","last_peered":"2024-08-30T08:58:46.555057+0000","last_clean":"2024-08-30T08:58:46.555057+0000","last_became_active":"2024-08-30T08:58:09.447444+0000","last_became_peered":"2024-08-30T08:58:09.447444+0000","last_unstale":"2024-08-30T08:58:46.555057+0000","last_undegraded":"2024-08-30T08:58:46.555057+0000","last_fullsized":"2024-08-30T08:58:46.555057+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T11:40:03.507824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"81'6","reported_seq":55,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:59:00.615979+0000","last_change":"2024-08-30T08:58:00.392418+0000","last_active":"2024-08-30T08:59:00.615979+0000","last_peered":"2024-08-30T08:59:00.615979+0000","last_clean":"2024-08-30T08:59:00.615979+0000","last_became_active":"2024-08-30T08:58:00.391985+0000","last_became_peered":"2024-08-30T08:58:00.391985+0000","last_unstale":"2024-08-30T08:59:00.615979+0000","last_undegraded":"2024-08-30T08:59:00.615979+0000","last_fullsized":"2024-08-30T08:59:00.615979+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T17:42:18.147754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":14,"num_read_kb":2,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.557148+0000","last_change":"2024-08-30T08:58:02.408754+0000","last_active":"2024-08-30T08:58:46.557148+0000","last_peered":"2024-08-30T08:58:46.557148+0000","last_clean":"2024-08-30T08:58:46.557148+0000","last_became_active":"2024-08-30T08:58:02.407127+0000","last_became_peered":"2024-08-30T08:58:02.407127+0000","last_unstale":"2024-08-30T08:58:46.557148+0000","last_undegraded":"2024-08-30T08:58:46.557148+0000","last_fullsized":"2024-08-30T08:58:46.557148+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T16:15:41.796091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.555654+0000","last_change":"2024-08-30T08:58:09.445389+0000","last_active":"2024-08-30T08:58:46.555654+0000","last_peered":"2024-08-30T08:58:46.555654+0000","last_clean":"2024-08-30T08:58:46.555654+0000","last_became_active":"2024-08-30T08:58:09.444831+0000","last_became_peered":"2024-08-30T08:58:09.444831+0000","last_unstale":"2024-08-30T08:58:46.555654+0000","last_undegraded":"2024-08-30T08:58:46.555654+0000","last_fullsized":"2024-08-30T08:58:46.555654+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T09:55:58.162397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.553020+0000","last_change":"2024-08-30T08:58:02.415184+0000","last_active":"2024-08-30T08:58:46.553020+0000","last_peered":"2024-08-30T08:58:46.553020+0000","last_clean":"2024-08-30T08:58:46.553020+0000","last_became_active":"2024-08-30T08:58:02.414625+0000","last_became_peered":"2024-08-30T08:58:02.414625+0000","last_unstale":"2024-08-30T08:58:46.553020+0000","last_undegraded":"2024-08-30T08:58:46.553020+0000","last_fullsized":"2024-08-30T08:58:46.553020+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T11:10:31.130524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.555567+0000","last_change":"2024-08-30T08:58:02.405392+0000","last_active":"2024-08-30T08:58:46.555567+0000","last_peered":"2024-08-30T08:58:46.555567+0000","last_clean":"2024-08-30T08:58:46.555567+0000","last_became_active":"2024-08-30T08:58:02.405009+0000","last_became_peered":"2024-08-30T08:58:02.405009+0000","last_unstale":"2024-08-30T08:58:46.555567+0000","last_undegraded":"2024-08-30T08:58:46.555567+0000","last_fullsized":"2024-08-30T08:58:46.555567+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T13:22:46.375598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.555680+0000","last_change":"2024-08-30T08:58:02.402360+0000","last_active":"2024-08-30T08:58:46.555680+0000","last_peered":"2024-08-30T08:58:46.555680+0000","last_clean":"2024-08-30T08:58:46.555680+0000","last_became_active":"2024-08-30T08:58:02.401856+0000","last_became_peered":"2024-08-30T08:58:02.401856+0000","last_unstale":"2024-08-30T08:58:46.555680+0000","last_undegraded":"2024-08-30T08:58:46.555680+0000","last_fullsized":"2024-08-30T08:58:46.555680+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T10:25:14.491345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.557920+0000","last_change":"2024-08-30T08:58:02.415940+0000","last_active":"2024-08-30T08:58:46.557920+0000","last_peered":"2024-08-30T08:58:46.557920+0000","last_clean":"2024-08-30T08:58:46.557920+0000","last_became_active":"2024-08-30T08:58:02.415567+0000","last_became_peered":"2024-08-30T08:58:02.415567+0000","last_unstale":"2024-08-30T08:58:46.557920+0000","last_undegraded":"2024-08-30T08:58:46.557920+0000","last_fullsized":"2024-08-30T08:58:46.557920+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T15:02:26.270343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.552160+0000","last_change":"2024-08-30T08:58:02.406766+0000","last_active":"2024-08-30T08:58:46.552160+0000","last_peered":"2024-08-30T08:58:46.552160+0000","last_clean":"2024-08-30T08:58:46.552160+0000","last_became_active":"2024-08-30T08:58:02.406345+0000","last_became_peered":"2024-08-30T08:58:02.406345+0000","last_unstale":"2024-08-30T08:58:46.552160+0000","last_undegraded":"2024-08-30T08:58:46.552160+0000","last_fullsized":"2024-08-30T08:58:46.552160+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T13:10:32.986817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.553714+0000","last_change":"2024-08-30T08:58:02.412891+0000","last_active":"2024-08-30T08:58:46.553714+0000","last_peered":"2024-08-30T08:58:46.553714+0000","last_clean":"2024-08-30T08:58:46.553714+0000","last_became_active":"2024-08-30T08:58:02.412532+0000","last_became_peered":"2024-08-30T08:58:02.412532+0000","last_unstale":"2024-08-30T08:58:46.553714+0000","last_undegraded":"2024-08-30T08:58:46.553714+0000","last_fullsized":"2024-08-30T08:58:46.553714+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T14:40:30.663678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.556273+0000","last_change":"2024-08-30T08:58:02.405862+0000","last_active":"2024-08-30T08:58:46.556273+0000","last_peered":"2024-08-30T08:58:46.556273+0000","last_clean":"2024-08-30T08:58:46.556273+0000","last_became_active":"2024-08-30T08:58:02.405319+0000","last_became_peered":"2024-08-30T08:58:02.405319+0000","last_unstale":"2024-08-30T08:58:46.556273+0000","last_undegraded":"2024-08-30T08:58:46.556273+0000","last_fullsized":"2024-08-30T08:58:46.556273+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T20:36:10.315814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.555535+0000","last_change":"2024-08-30T08:58:02.401410+0000","last_active":"2024-08-30T08:58:46.555535+0000","last_peered":"2024-08-30T08:58:46.555535+0000","last_clean":"2024-08-30T08:58:46.555535+0000","last_became_active":"2024-08-30T08:58:02.401060+0000","last_became_peered":"2024-08-30T08:58:02.401060+0000","last_unstale":"2024-08-30T08:58:46.555535+0000","last_undegraded":"2024-08-30T08:58:46.555535+0000","last_fullsized":"2024-08-30T08:58:46.555535+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T16:00:41.275367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.773667+0000","last_change":"2024-08-30T08:58:02.401703+0000","last_active":"2024-08-30T08:58:46.773667+0000","last_peered":"2024-08-30T08:58:46.773667+0000","last_clean":"2024-08-30T08:58:46.773667+0000","last_became_active":"2024-08-30T08:58:02.401218+0000","last_became_peered":"2024-08-30T08:58:02.401218+0000","last_unstale":"2024-08-30T08:58:46.773667+0000","last_undegraded":"2024-08-30T08:58:46.773667+0000","last_fullsized":"2024-08-30T08:58:46.773667+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T18:36:25.249814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.554007+0000","last_change":"2024-08-30T08:58:02.403609+0000","last_active":"2024-08-30T08:58:46.554007+0000","last_peered":"2024-08-30T08:58:46.554007+0000","last_clean":"2024-08-30T08:58:46.554007+0000","last_became_active":"2024-08-30T08:58:02.402195+0000","last_became_peered":"2024-08-30T08:58:02.402195+0000","last_unstale":"2024-08-30T08:58:46.554007+0000","last_undegraded":"2024-08-30T08:58:46.554007+0000","last_fullsized":"2024-08-30T08:58:46.554007+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T09:59:05.877523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.823042+0000","last_change":"2024-08-30T08:58:02.404389+0000","last_active":"2024-08-30T08:58:46.823042+0000","last_peered":"2024-08-30T08:58:46.823042+0000","last_clean":"2024-08-30T08:58:46.823042+0000","last_became_active":"2024-08-30T08:58:02.404039+0000","last_became_peered":"2024-08-30T08:58:02.404039+0000","last_unstale":"2024-08-30T08:58:46.823042+0000","last_undegraded":"2024-08-30T08:58:46.823042+0000","last_fullsized":"2024-08-30T08:58:46.823042+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T20:55:51.739542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.552060+0000","last_change":"2024-08-30T08:58:02.411807+0000","last_active":"2024-08-30T08:58:46.552060+0000","last_peered":"2024-08-30T08:58:46.552060+0000","last_clean":"2024-08-30T08:58:46.552060+0000","last_became_active":"2024-08-30T08:58:02.410977+0000","last_became_peered":"2024-08-30T08:58:02.410977+0000","last_unstale":"2024-08-30T08:58:46.552060+0000","last_undegraded":"2024-08-30T08:58:46.552060+0000","last_fullsized":"2024-08-30T08:58:46.552060+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T13:04:29.931373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":37,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.560319+0000","last_change":"2024-08-30T08:58:46.559661+0000","last_active":"2024-08-30T08:58:46.560319+0000","last_peered":"2024-08-30T08:58:46.560319+0000","last_clean":"2024-08-30T08:58:46.560319+0000","last_became_active":"2024-08-30T08:58:46.558824+0000","last_became_peered":"2024-08-30T08:58:46.558824+0000","last_unstale":"2024-08-30T08:58:46.560319+0000","last_undegraded":"2024-08-30T08:58:46.560319+0000","last_fullsized":"2024-08-30T08:58:46.560319+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T09:53:07.650921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.556117+0000","last_change":"2024-08-30T08:58:02.396186+0000","last_active":"2024-08-30T08:58:46.556117+0000","last_peered":"2024-08-30T08:58:46.556117+0000","last_clean":"2024-08-30T08:58:46.556117+0000","last_became_active":"2024-08-30T08:58:02.395776+0000","last_became_peered":"2024-08-30T08:58:02.395776+0000","last_unstale":"2024-08-30T08:58:46.556117+0000","last_undegraded":"2024-08-30T08:58:46.556117+0000","last_fullsized":"2024-08-30T08:58:46.556117+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T09:33:30.068673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.559487+0000","last_change":"2024-08-30T08:58:02.408378+0000","last_active":"2024-08-30T08:58:46.559487+0000","last_peered":"2024-08-30T08:58:46.559487+0000","last_clean":"2024-08-30T08:58:46.559487+0000","last_became_active":"2024-08-30T08:58:02.407991+0000","last_became_peered":"2024-08-30T08:58:02.407991+0000","last_unstale":"2024-08-30T08:58:46.559487+0000","last_undegraded":"2024-08-30T08:58:46.559487+0000","last_fullsized":"2024-08-30T08:58:46.559487+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T20:57:02.411176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.553823+0000","last_change":"2024-08-30T08:58:02.411829+0000","last_active":"2024-08-30T08:58:46.553823+0000","last_peered":"2024-08-30T08:58:46.553823+0000","last_clean":"2024-08-30T08:58:46.553823+0000","last_became_active":"2024-08-30T08:58:02.411416+0000","last_became_peered":"2024-08-30T08:58:02.411416+0000","last_unstale":"2024-08-30T08:58:46.553823+0000","last_undegraded":"2024-08-30T08:58:46.553823+0000","last_fullsized":"2024-08-30T08:58:46.553823+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T17:12:08.855150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.822489+0000","last_change":"2024-08-30T08:58:02.405122+0000","last_active":"2024-08-30T08:58:46.822489+0000","last_peered":"2024-08-30T08:58:46.822489+0000","last_clean":"2024-08-30T08:58:46.822489+0000","last_became_active":"2024-08-30T08:58:02.403312+0000","last_became_peered":"2024-08-30T08:58:02.403312+0000","last_unstale":"2024-08-30T08:58:46.822489+0000","last_undegraded":"2024-08-30T08:58:46.822489+0000","last_fullsized":"2024-08-30T08:58:46.822489+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T18:11:53.077433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.556239+0000","last_change":"2024-08-30T08:58:02.404263+0000","last_active":"2024-08-30T08:58:46.556239+0000","last_peered":"2024-08-30T08:58:46.556239+0000","last_clean":"2024-08-30T08:58:46.556239+0000","last_became_active":"2024-08-30T08:58:02.403897+0000","last_became_peered":"2024-08-30T08:58:02.403897+0000","last_unstale":"2024-08-30T08:58:46.556239+0000","last_undegraded":"2024-08-30T08:58:46.556239+0000","last_fullsized":"2024-08-30T08:58:46.556239+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T18:38:31.439434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.556196+0000","last_change":"2024-08-30T08:58:02.411578+0000","last_active":"2024-08-30T08:58:46.556196+0000","last_peered":"2024-08-30T08:58:46.556196+0000","last_clean":"2024-08-30T08:58:46.556196+0000","last_became_active":"2024-08-30T08:58:02.411271+0000","last_became_peered":"2024-08-30T08:58:02.411271+0000","last_unstale":"2024-08-30T08:58:46.556196+0000","last_undegraded":"2024-08-30T08:58:46.556196+0000","last_fullsized":"2024-08-30T08:58:46.556196+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T10:02:46.382906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.552115+0000","last_change":"2024-08-30T08:58:02.412168+0000","last_active":"2024-08-30T08:58:46.552115+0000","last_peered":"2024-08-30T08:58:46.552115+0000","last_clean":"2024-08-30T08:58:46.552115+0000","last_became_active":"2024-08-30T08:58:02.411420+0000","last_became_peered":"2024-08-30T08:58:02.411420+0000","last_unstale":"2024-08-30T08:58:46.552115+0000","last_undegraded":"2024-08-30T08:58:46.552115+0000","last_fullsized":"2024-08-30T08:58:46.552115+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T10:35:53.654344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:47.089927+0000","last_change":"2024-08-30T08:58:02.409735+0000","last_active":"2024-08-30T08:58:47.089927+0000","last_peered":"2024-08-30T08:58:47.089927+0000","last_clean":"2024-08-30T08:58:47.089927+0000","last_became_active":"2024-08-30T08:58:02.409349+0000","last_became_peered":"2024-08-30T08:58:02.409349+0000","last_unstale":"2024-08-30T08:58:47.089927+0000","last_undegraded":"2024-08-30T08:58:47.089927+0000","last_fullsized":"2024-08-30T08:58:47.089927+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T20:10:22.956554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:47.090762+0000","last_change":"2024-08-30T08:58:02.416261+0000","last_active":"2024-08-30T08:58:47.090762+0000","last_peered":"2024-08-30T08:58:47.090762+0000","last_clean":"2024-08-30T08:58:47.090762+0000","last_became_active":"2024-08-30T08:58:02.415194+0000","last_became_peered":"2024-08-30T08:58:02.415194+0000","last_unstale":"2024-08-30T08:58:47.090762+0000","last_undegraded":"2024-08-30T08:58:47.090762+0000","last_fullsized":"2024-08-30T08:58:47.090762+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T17:59:04.339699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":37,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.560326+0000","last_change":"2024-08-30T08:58:46.560190+0000","last_active":"2024-08-30T08:58:46.560326+0000","last_peered":"2024-08-30T08:58:46.560326+0000","last_clean":"2024-08-30T08:58:46.560326+0000","last_became_active":"2024-08-30T08:58:46.559674+0000","last_became_peered":"2024-08-30T08:58:46.559674+0000","last_unstale":"2024-08-30T08:58:46.560326+0000","last_undegraded":"2024-08-30T08:58:46.560326+0000","last_fullsized":"2024-08-30T08:58:46.560326+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T10:08:02.835142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.823261+0000","last_change":"2024-08-30T08:58:02.406763+0000","last_active":"2024-08-30T08:58:46.823261+0000","last_peered":"2024-08-30T08:58:46.823261+0000","last_clean":"2024-08-30T08:58:46.823261+0000","last_became_active":"2024-08-30T08:58:02.406377+0000","last_became_peered":"2024-08-30T08:58:02.406377+0000","last_unstale":"2024-08-30T08:58:46.823261+0000","last_undegraded":"2024-08-30T08:58:46.823261+0000","last_fullsized":"2024-08-30T08:58:46.823261+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T13:32:50.492921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.553816+0000","last_change":"2024-08-30T08:58:02.416638+0000","last_active":"2024-08-30T08:58:46.553816+0000","last_peered":"2024-08-30T08:58:46.553816+0000","last_clean":"2024-08-30T08:58:46.553816+0000","last_became_active":"2024-08-30T08:58:02.415518+0000","last_became_peered":"2024-08-30T08:58:02.415518+0000","last_unstale":"2024-08-30T08:58:46.553816+0000","last_undegraded":"2024-08-30T08:58:46.553816+0000","last_fullsized":"2024-08-30T08:58:46.553816+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T17:05:07.531585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:47.089992+0000","last_change":"2024-08-30T08:58:02.412973+0000","last_active":"2024-08-30T08:58:47.089992+0000","last_peered":"2024-08-30T08:58:47.089992+0000","last_clean":"2024-08-30T08:58:47.089992+0000","last_became_active":"2024-08-30T08:58:02.412609+0000","last_became_peered":"2024-08-30T08:58:02.412609+0000","last_unstale":"2024-08-30T08:58:47.089992+0000","last_undegraded":"2024-08-30T08:58:47.089992+0000","last_fullsized":"2024-08-30T08:58:47.089992+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T14:22:37.704098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.553161+0000","last_change":"2024-08-30T08:58:02.402779+0000","last_active":"2024-08-30T08:58:46.553161+0000","last_peered":"2024-08-30T08:58:46.553161+0000","last_clean":"2024-08-30T08:58:46.553161+0000","last_became_active":"2024-08-30T08:58:02.401514+0000","last_became_peered":"2024-08-30T08:58:02.401514+0000","last_unstale":"2024-08-30T08:58:46.553161+0000","last_undegraded":"2024-08-30T08:58:46.553161+0000","last_fullsized":"2024-08-30T08:58:46.553161+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T11:37:51.450334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.557162+0000","last_change":"2024-08-30T08:58:02.406607+0000","last_active":"2024-08-30T08:58:46.557162+0000","last_peered":"2024-08-30T08:58:46.557162+0000","last_clean":"2024-08-30T08:58:46.557162+0000","last_became_active":"2024-08-30T08:58:02.406237+0000","last_became_peered":"2024-08-30T08:58:02.406237+0000","last_unstale":"2024-08-30T08:58:46.557162+0000","last_undegraded":"2024-08-30T08:58:46.557162+0000","last_fullsized":"2024-08-30T08:58:46.557162+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T12:09:03.247424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:47.089986+0000","last_change":"2024-08-30T08:58:02.410749+0000","last_active":"2024-08-30T08:58:47.089986+0000","last_peered":"2024-08-30T08:58:47.089986+0000","last_clean":"2024-08-30T08:58:47.089986+0000","last_became_active":"2024-08-30T08:58:02.410350+0000","last_became_peered":"2024-08-30T08:58:02.410350+0000","last_unstale":"2024-08-30T08:58:47.089986+0000","last_undegraded":"2024-08-30T08:58:47.089986+0000","last_fullsized":"2024-08-30T08:58:47.089986+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T15:22:51.451789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.557090+0000","last_change":"2024-08-30T08:58:02.405946+0000","last_active":"2024-08-30T08:58:46.557090+0000","last_peered":"2024-08-30T08:58:46.557090+0000","last_clean":"2024-08-30T08:58:46.557090+0000","last_became_active":"2024-08-30T08:58:02.405590+0000","last_became_peered":"2024-08-30T08:58:02.405590+0000","last_unstale":"2024-08-30T08:58:46.557090+0000","last_undegraded":"2024-08-30T08:58:46.557090+0000","last_fullsized":"2024-08-30T08:58:46.557090+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T11:42:14.966065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.556440+0000","last_change":"2024-08-30T08:58:02.409649+0000","last_active":"2024-08-30T08:58:46.556440+0000","last_peered":"2024-08-30T08:58:46.556440+0000","last_clean":"2024-08-30T08:58:46.556440+0000","last_became_active":"2024-08-30T08:58:02.409282+0000","last_became_peered":"2024-08-30T08:58:02.409282+0000","last_unstale":"2024-08-30T08:58:46.556440+0000","last_undegraded":"2024-08-30T08:58:46.556440+0000","last_fullsized":"2024-08-30T08:58:46.556440+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T15:33:52.166290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.553800+0000","last_change":"2024-08-30T08:58:02.410711+0000","last_active":"2024-08-30T08:58:46.553800+0000","last_peered":"2024-08-30T08:58:46.553800+0000","last_clean":"2024-08-30T08:58:46.553800+0000","last_became_active":"2024-08-30T08:58:02.410367+0000","last_became_peered":"2024-08-30T08:58:02.410367+0000","last_unstale":"2024-08-30T08:58:46.553800+0000","last_undegraded":"2024-08-30T08:58:46.553800+0000","last_fullsized":"2024-08-30T08:58:46.553800+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T15:27:40.392588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":58784,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":98,"num_read_kb":111,"num_write":161,"num_write_kb":107,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":405504,"data_stored":176352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":139,"ondisk_log_size":139,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":67,"seq":287762808855,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":925696,"data_stored":455486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":61,"seq":261993005085,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28588,"kb_used_data":1564,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716564,"statfs":{"total":95995035648,"available":95965761536,"internally_reserved":0,"allocated":1601536,"data_stored":1097425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":57,"seq":244813135906,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36260,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708892,"statfs":{"total":95995035648,"available":95957905408,"internally_reserved":0,"allocated":937984,"data_stored":459706,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":51,"seq":219043332136,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":925696,"data_stored":455418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":46,"seq":197568495660,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28516,"kb_used_data":1492,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716636,"statfs":{"total":95995035648,"available":95965835264,"internally_reserved":0,"allocated":1527808,"data_stored":1051830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":41,"seq":176093659186,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":913408,"data_stored":450786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":35,"seq":150323855415,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36844,"kb_used_data":1500,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708308,"statfs":{"total":95995035648,"available":95957307392,"internally_reserved":0,"allocated":1536000,"data_stored":1047027,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":30,"seq":128849018940,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32088,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713064,"statfs":{"total":95995035648,"available":95962177536,"internally_reserved":0,"allocated":925696,"data_stored":455827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215168,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36248,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708904,"statfs":{"total":95995035648,"available":95957917696,"internally_reserved":0,"allocated":925696,"data_stored":450980,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411398,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36320,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708832,"statfs":{"total":95995035648,"available":95957843968,"internally_reserved":0,"allocated":999424,"data_stored":506829,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574923,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":974848,"data_stored":505088,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738447,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":937984,"data_stored":452174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":54392,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":56133,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":56361,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-30T08:59:03.830 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph pg dump --format=json 2024-08-30T08:59:04.129 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T08:59:04.541 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:59:04 smithi042 ceph-mon[18652]: pgmap v320: 129 pgs: 129 active+clean; 634 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 5.5 KiB/s wr, 11 op/s 2024-08-30T08:59:04.541 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:59:04 smithi042 ceph-mon[18652]: from='client.15003 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-30T08:59:04.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:59:04 smithi167 ceph-mon[22063]: pgmap v320: 129 pgs: 129 active+clean; 634 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 5.5 KiB/s wr, 11 op/s 2024-08-30T08:59:04.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:59:04 smithi167 ceph-mon[22063]: from='client.15003 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-30T08:59:04.641 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:04.630+0000 7f31d1ea2640 1 -- 172.21.15.42:0/1231911797 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f31cc10dbb0 msgr2=0x7f31cc1100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:04.641 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:04.630+0000 7f31d1ea2640 1 --2- 172.21.15.42:0/1231911797 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f31cc10dbb0 0x7f31cc1100a0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f31c4009900 tx=0x7f31c402a410 comp rx=0 tx=0).stop 2024-08-30T08:59:04.641 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:04.630+0000 7f31d1ea2640 1 -- 172.21.15.42:0/1231911797 shutdown_connections 2024-08-30T08:59:04.641 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:04.630+0000 7f31d1ea2640 1 --2- 172.21.15.42:0/1231911797 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f31cc10dbb0 0x7f31cc1100a0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:04.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:04.630+0000 7f31d1ea2640 1 --2- 172.21.15.42:0/1231911797 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f31cc1050a0 0x7f31cc10d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:04.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:04.630+0000 7f31d1ea2640 1 --2- 172.21.15.42:0/1231911797 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f31cc104750 0x7f31cc104b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:04.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:04.630+0000 7f31d1ea2640 1 -- 172.21.15.42:0/1231911797 >> 172.21.15.42:0/1231911797 conn(0x7f31cc0794d0 msgr2=0x7f31cc0798e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:59:04.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:04.630+0000 7f31d1ea2640 1 -- 172.21.15.42:0/1231911797 shutdown_connections 2024-08-30T08:59:04.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:04.630+0000 7f31d1ea2640 1 -- 172.21.15.42:0/1231911797 wait complete. 2024-08-30T08:59:04.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:04.630+0000 7f31d1ea2640 1 Processor -- start 2024-08-30T08:59:04.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:04.631+0000 7f31d1ea2640 1 -- start start 2024-08-30T08:59:04.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:04.631+0000 7f31d1ea2640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f31cc104750 0x7f31cc102490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:04.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:04.631+0000 7f31d1ea2640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f31cc1050a0 0x7f31cc1029d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:04.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:04.631+0000 7f31d1ea2640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f31cc10dbb0 0x7f31cc102f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:04.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:04.631+0000 7f31d1ea2640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f31cc113890 con 0x7f31cc1050a0 2024-08-30T08:59:04.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:04.631+0000 7f31d1ea2640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f31cc113710 con 0x7f31cc104750 2024-08-30T08:59:04.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:04.631+0000 7f31d1ea2640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f31cc113a10 con 0x7f31cc10dbb0 2024-08-30T08:59:04.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:04.631+0000 7f31caffd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f31cc1050a0 0x7f31cc1029d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:04.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:04.631+0000 7f31caffd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f31cc1050a0 0x7f31cc1029d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:32802/0 (socket says 172.21.15.42:32802) 2024-08-30T08:59:04.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:04.631+0000 7f31caffd640 1 -- 172.21.15.42:0/1668244165 learned_addr learned my addr 172.21.15.42:0/1668244165 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T08:59:04.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:04.631+0000 7f31cb7fe640 1 --2- 172.21.15.42:0/1668244165 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f31cc104750 0x7f31cc102490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:04.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:04.631+0000 7f31cbfff640 1 --2- 172.21.15.42:0/1668244165 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f31cc10dbb0 0x7f31cc102f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:04.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:04.631+0000 7f31caffd640 1 -- 172.21.15.42:0/1668244165 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f31cc10dbb0 msgr2=0x7f31cc102f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:04.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:04.631+0000 7f31caffd640 1 --2- 172.21.15.42:0/1668244165 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f31cc10dbb0 0x7f31cc102f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:04.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:04.631+0000 7f31caffd640 1 -- 172.21.15.42:0/1668244165 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f31cc104750 msgr2=0x7f31cc102490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:04.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:04.631+0000 7f31caffd640 1 --2- 172.21.15.42:0/1668244165 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f31cc104750 0x7f31cc102490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:04.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:04.631+0000 7f31caffd640 1 -- 172.21.15.42:0/1668244165 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31cc1a8520 con 0x7f31cc1050a0 2024-08-30T08:59:04.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:04.631+0000 7f31cbfff640 1 --2- 172.21.15.42:0/1668244165 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f31cc10dbb0 0x7f31cc102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-30T08:59:04.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:04.632+0000 7f31caffd640 1 --2- 172.21.15.42:0/1668244165 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f31cc1050a0 0x7f31cc1029d0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f31b4006fd0 tx=0x7f31b400b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:59:04.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:04.632+0000 7f31c8ff9640 1 -- 172.21.15.42:0/1668244165 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31b4011830 con 0x7f31cc1050a0 2024-08-30T08:59:04.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:04.632+0000 7f31d1ea2640 1 -- 172.21.15.42:0/1668244165 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31cc1a8810 con 0x7f31cc1050a0 2024-08-30T08:59:04.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:04.632+0000 7f31c8ff9640 1 -- 172.21.15.42:0/1668244165 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f31b4016070 con 0x7f31cc1050a0 2024-08-30T08:59:04.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:04.632+0000 7f31c8ff9640 1 -- 172.21.15.42:0/1668244165 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31b4010e00 con 0x7f31cc1050a0 2024-08-30T08:59:04.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:04.632+0000 7f31d1ea2640 1 -- 172.21.15.42:0/1668244165 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31cc1a8e20 con 0x7f31cc1050a0 2024-08-30T08:59:04.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:04.633+0000 7f31d1ea2640 1 -- 172.21.15.42:0/1668244165 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31cc104b60 con 0x7f31cc1050a0 2024-08-30T08:59:04.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:04.634+0000 7f31c8ff9640 1 -- 172.21.15.42:0/1668244165 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) ==== 152628+0+0 (secure 0 0 0) 0x7f31b401b460 con 0x7f31cc1050a0 2024-08-30T08:59:04.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:04.635+0000 7f31c8ff9640 1 --2- 172.21.15.42:0/1668244165 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f31b80b4bc0 0x7f31b80b70d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:04.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:04.635+0000 7f31c8ff9640 1 -- 172.21.15.42:0/1668244165 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 9092+0+0 (secure 0 0 0) 0x7f31b40e51f0 con 0x7f31cc1050a0 2024-08-30T08:59:04.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:04.635+0000 7f31cb7fe640 1 --2- 172.21.15.42:0/1668244165 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f31b80b4bc0 0x7f31b80b70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:04.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:04.636+0000 7f31cb7fe640 1 --2- 172.21.15.42:0/1668244165 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f31b80b4bc0 0x7f31b80b70d0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f31cc103ee0 tx=0x7f31bc009680 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:59:04.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:04.638+0000 7f31c8ff9640 1 -- 172.21.15.42:0/1668244165 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f31b40abe60 con 0x7f31cc1050a0 2024-08-30T08:59:04.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:59:04 smithi190 ceph-mon[22001]: pgmap v320: 129 pgs: 129 active+clean; 634 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 5.5 KiB/s wr, 11 op/s 2024-08-30T08:59:04.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:59:04 smithi190 ceph-mon[22001]: from='client.15003 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-30T08:59:04.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:04.820+0000 7f31d1ea2640 1 -- 172.21.15.42:0/1668244165 --> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f31cc106660 con 0x7f31b80b4bc0 2024-08-30T08:59:04.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:04.826+0000 7f31c8ff9640 1 -- 172.21.15.42:0/1668244165 <== mgr.14150 v2:172.21.15.42:6800/3005140902 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340325 (secure 0 0 0) 0x7f31cc106660 con 0x7f31b80b4bc0 2024-08-30T08:59:04.831 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-08-30T08:59:04.832 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-08-30T08:59:04.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:04.830+0000 7f31d1ea2640 1 -- 172.21.15.42:0/1668244165 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f31b80b4bc0 msgr2=0x7f31b80b70d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:04.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:04.830+0000 7f31d1ea2640 1 --2- 172.21.15.42:0/1668244165 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f31b80b4bc0 0x7f31b80b70d0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f31cc103ee0 tx=0x7f31bc009680 comp rx=0 tx=0).stop 2024-08-30T08:59:04.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:04.830+0000 7f31d1ea2640 1 -- 172.21.15.42:0/1668244165 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f31cc1050a0 msgr2=0x7f31cc1029d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:04.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:04.830+0000 7f31d1ea2640 1 --2- 172.21.15.42:0/1668244165 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f31cc1050a0 0x7f31cc1029d0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f31b4006fd0 tx=0x7f31b400b8e0 comp rx=0 tx=0).stop 2024-08-30T08:59:04.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:04.830+0000 7f31d1ea2640 1 -- 172.21.15.42:0/1668244165 shutdown_connections 2024-08-30T08:59:04.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:04.830+0000 7f31d1ea2640 1 --2- 172.21.15.42:0/1668244165 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f31b80b4bc0 0x7f31b80b70d0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:04.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:04.830+0000 7f31d1ea2640 1 --2- 172.21.15.42:0/1668244165 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f31cc10dbb0 0x7f31cc102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:04.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:04.830+0000 7f31d1ea2640 1 --2- 172.21.15.42:0/1668244165 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f31cc1050a0 0x7f31cc1029d0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:04.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:04.830+0000 7f31d1ea2640 1 --2- 172.21.15.42:0/1668244165 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f31cc104750 0x7f31cc102490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:04.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:04.830+0000 7f31d1ea2640 1 -- 172.21.15.42:0/1668244165 >> 172.21.15.42:0/1668244165 conn(0x7f31cc0794d0 msgr2=0x7f31cc105f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:59:04.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:04.830+0000 7f31d1ea2640 1 -- 172.21.15.42:0/1668244165 shutdown_connections 2024-08-30T08:59:04.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:04.830+0000 7f31d1ea2640 1 -- 172.21.15.42:0/1668244165 wait complete. 2024-08-30T08:59:05.543 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":320,"stamp":"2024-08-30T08:59:02.938778+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":649386,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":174,"num_read_kb":175,"num_write":286,"num_write_kb":2259,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":233,"ondisk_log_size":233,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":374556,"kb_used_data":12824,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124567268,"statfs":{"total":1151940427776,"available":1151556882432,"internally_reserved":0,"allocated":13131776,"data_stored":7388576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":370193721},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":51566,"num_objects":3,"num_object_clones":0,"num_object_copies":11,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":98,"num_read_kb":111,"num_write":35,"num_write_kb":66,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003731"},"pg_stats":[{"pgid":"2.1f","version":"80'9","reported_seq":45,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.559175+0000","last_change":"2024-08-30T08:58:00.388595+0000","last_active":"2024-08-30T08:58:46.559175+0000","last_peered":"2024-08-30T08:58:46.559175+0000","last_clean":"2024-08-30T08:58:46.559175+0000","last_became_active":"2024-08-30T08:58:00.388215+0000","last_became_peered":"2024-08-30T08:58:00.388215+0000","last_unstale":"2024-08-30T08:58:46.559175+0000","last_undegraded":"2024-08-30T08:58:46.559175+0000","last_fullsized":"2024-08-30T08:58:46.559175+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T13:00:06.109935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.555091+0000","last_change":"2024-08-30T08:58:02.416042+0000","last_active":"2024-08-30T08:58:46.555091+0000","last_peered":"2024-08-30T08:58:46.555091+0000","last_clean":"2024-08-30T08:58:46.555091+0000","last_became_active":"2024-08-30T08:58:02.415507+0000","last_became_peered":"2024-08-30T08:58:02.415507+0000","last_unstale":"2024-08-30T08:58:46.555091+0000","last_undegraded":"2024-08-30T08:58:46.555091+0000","last_fullsized":"2024-08-30T08:58:46.555091+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T14:33:57.192004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.554665+0000","last_change":"2024-08-30T08:58:09.448785+0000","last_active":"2024-08-30T08:58:46.554665+0000","last_peered":"2024-08-30T08:58:46.554665+0000","last_clean":"2024-08-30T08:58:46.554665+0000","last_became_active":"2024-08-30T08:58:09.448191+0000","last_became_peered":"2024-08-30T08:58:09.448191+0000","last_unstale":"2024-08-30T08:58:46.554665+0000","last_undegraded":"2024-08-30T08:58:46.554665+0000","last_fullsized":"2024-08-30T08:58:46.554665+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T13:37:19.148201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"80'2","reported_seq":37,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.555518+0000","last_change":"2024-08-30T08:58:00.480406+0000","last_active":"2024-08-30T08:58:46.555518+0000","last_peered":"2024-08-30T08:58:46.555518+0000","last_clean":"2024-08-30T08:58:46.555518+0000","last_became_active":"2024-08-30T08:58:00.479896+0000","last_became_peered":"2024-08-30T08:58:00.479896+0000","last_unstale":"2024-08-30T08:58:46.555518+0000","last_undegraded":"2024-08-30T08:58:46.555518+0000","last_fullsized":"2024-08-30T08:58:46.555518+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T15:16:15.316343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.557006+0000","last_change":"2024-08-30T08:58:02.406915+0000","last_active":"2024-08-30T08:58:46.557006+0000","last_peered":"2024-08-30T08:58:46.557006+0000","last_clean":"2024-08-30T08:58:46.557006+0000","last_became_active":"2024-08-30T08:58:02.406557+0000","last_became_peered":"2024-08-30T08:58:02.406557+0000","last_unstale":"2024-08-30T08:58:46.557006+0000","last_undegraded":"2024-08-30T08:58:46.557006+0000","last_fullsized":"2024-08-30T08:58:46.557006+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T19:15:30.847212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:47.089607+0000","last_change":"2024-08-30T08:58:09.446974+0000","last_active":"2024-08-30T08:58:47.089607+0000","last_peered":"2024-08-30T08:58:47.089607+0000","last_clean":"2024-08-30T08:58:47.089607+0000","last_became_active":"2024-08-30T08:58:09.446173+0000","last_became_peered":"2024-08-30T08:58:09.446173+0000","last_unstale":"2024-08-30T08:58:47.089607+0000","last_undegraded":"2024-08-30T08:58:47.089607+0000","last_fullsized":"2024-08-30T08:58:47.089607+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T12:13:41.276825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"80'4","reported_seq":39,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.557615+0000","last_change":"2024-08-30T08:58:00.390609+0000","last_active":"2024-08-30T08:58:46.557615+0000","last_peered":"2024-08-30T08:58:46.557615+0000","last_clean":"2024-08-30T08:58:46.557615+0000","last_became_active":"2024-08-30T08:58:00.390229+0000","last_became_peered":"2024-08-30T08:58:00.390229+0000","last_unstale":"2024-08-30T08:58:46.557615+0000","last_undegraded":"2024-08-30T08:58:46.557615+0000","last_fullsized":"2024-08-30T08:58:46.557615+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T11:58:25.037431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.822738+0000","last_change":"2024-08-30T08:58:02.415594+0000","last_active":"2024-08-30T08:58:46.822738+0000","last_peered":"2024-08-30T08:58:46.822738+0000","last_clean":"2024-08-30T08:58:46.822738+0000","last_became_active":"2024-08-30T08:58:02.415066+0000","last_became_peered":"2024-08-30T08:58:02.415066+0000","last_unstale":"2024-08-30T08:58:46.822738+0000","last_undegraded":"2024-08-30T08:58:46.822738+0000","last_fullsized":"2024-08-30T08:58:46.822738+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T12:07:27.668442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:47.089672+0000","last_change":"2024-08-30T08:58:09.444403+0000","last_active":"2024-08-30T08:58:47.089672+0000","last_peered":"2024-08-30T08:58:47.089672+0000","last_clean":"2024-08-30T08:58:47.089672+0000","last_became_active":"2024-08-30T08:58:09.443914+0000","last_became_peered":"2024-08-30T08:58:09.443914+0000","last_unstale":"2024-08-30T08:58:47.089672+0000","last_undegraded":"2024-08-30T08:58:47.089672+0000","last_fullsized":"2024-08-30T08:58:47.089672+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T14:17:43.866352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"73'2","reported_seq":37,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.557996+0000","last_change":"2024-08-30T08:58:00.392108+0000","last_active":"2024-08-30T08:58:46.557996+0000","last_peered":"2024-08-30T08:58:46.557996+0000","last_clean":"2024-08-30T08:58:46.557996+0000","last_became_active":"2024-08-30T08:58:00.391157+0000","last_became_peered":"2024-08-30T08:58:00.391157+0000","last_unstale":"2024-08-30T08:58:46.557996+0000","last_undegraded":"2024-08-30T08:58:46.557996+0000","last_fullsized":"2024-08-30T08:58:46.557996+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T17:21:03.369776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.774221+0000","last_change":"2024-08-30T08:58:02.405550+0000","last_active":"2024-08-30T08:58:46.774221+0000","last_peered":"2024-08-30T08:58:46.774221+0000","last_clean":"2024-08-30T08:58:46.774221+0000","last_became_active":"2024-08-30T08:58:02.405129+0000","last_became_peered":"2024-08-30T08:58:02.405129+0000","last_unstale":"2024-08-30T08:58:46.774221+0000","last_undegraded":"2024-08-30T08:58:46.774221+0000","last_fullsized":"2024-08-30T08:58:46.774221+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T11:36:47.624026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:47.089364+0000","last_change":"2024-08-30T08:58:09.445746+0000","last_active":"2024-08-30T08:58:47.089364+0000","last_peered":"2024-08-30T08:58:47.089364+0000","last_clean":"2024-08-30T08:58:47.089364+0000","last_became_active":"2024-08-30T08:58:09.445257+0000","last_became_peered":"2024-08-30T08:58:09.445257+0000","last_unstale":"2024-08-30T08:58:47.089364+0000","last_undegraded":"2024-08-30T08:58:47.089364+0000","last_fullsized":"2024-08-30T08:58:47.089364+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T18:02:52.007131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:47.090324+0000","last_change":"2024-08-30T08:58:02.408213+0000","last_active":"2024-08-30T08:58:47.090324+0000","last_peered":"2024-08-30T08:58:47.090324+0000","last_clean":"2024-08-30T08:58:47.090324+0000","last_became_active":"2024-08-30T08:58:02.407856+0000","last_became_peered":"2024-08-30T08:58:02.407856+0000","last_unstale":"2024-08-30T08:58:47.090324+0000","last_undegraded":"2024-08-30T08:58:47.090324+0000","last_fullsized":"2024-08-30T08:58:47.090324+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T12:00:22.226653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"79'4","reported_seq":40,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.557487+0000","last_change":"2024-08-30T08:58:00.391714+0000","last_active":"2024-08-30T08:58:46.557487+0000","last_peered":"2024-08-30T08:58:46.557487+0000","last_clean":"2024-08-30T08:58:46.557487+0000","last_became_active":"2024-08-30T08:58:00.391211+0000","last_became_peered":"2024-08-30T08:58:00.391211+0000","last_unstale":"2024-08-30T08:58:46.557487+0000","last_undegraded":"2024-08-30T08:58:46.557487+0000","last_fullsized":"2024-08-30T08:58:46.557487+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T11:55:37.469983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:47.089359+0000","last_change":"2024-08-30T08:58:09.728177+0000","last_active":"2024-08-30T08:58:47.089359+0000","last_peered":"2024-08-30T08:58:47.089359+0000","last_clean":"2024-08-30T08:58:47.089359+0000","last_became_active":"2024-08-30T08:58:09.727630+0000","last_became_peered":"2024-08-30T08:58:09.727630+0000","last_unstale":"2024-08-30T08:58:47.089359+0000","last_undegraded":"2024-08-30T08:58:47.089359+0000","last_fullsized":"2024-08-30T08:58:47.089359+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T12:07:52.965014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.554094+0000","last_change":"2024-08-30T08:58:02.403378+0000","last_active":"2024-08-30T08:58:46.554094+0000","last_peered":"2024-08-30T08:58:46.554094+0000","last_clean":"2024-08-30T08:58:46.554094+0000","last_became_active":"2024-08-30T08:58:02.403036+0000","last_became_peered":"2024-08-30T08:58:02.403036+0000","last_unstale":"2024-08-30T08:58:46.554094+0000","last_undegraded":"2024-08-30T08:58:46.554094+0000","last_fullsized":"2024-08-30T08:58:46.554094+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T16:37:14.876384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"80'1","reported_seq":36,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.555381+0000","last_change":"2024-08-30T08:58:00.393860+0000","last_active":"2024-08-30T08:58:46.555381+0000","last_peered":"2024-08-30T08:58:46.555381+0000","last_clean":"2024-08-30T08:58:46.555381+0000","last_became_active":"2024-08-30T08:58:00.392486+0000","last_became_peered":"2024-08-30T08:58:00.392486+0000","last_unstale":"2024-08-30T08:58:46.555381+0000","last_undegraded":"2024-08-30T08:58:46.555381+0000","last_fullsized":"2024-08-30T08:58:46.555381+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T12:50:27.337817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"81'1","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:47.089678+0000","last_change":"2024-08-30T08:58:09.441476+0000","last_active":"2024-08-30T08:58:47.089678+0000","last_peered":"2024-08-30T08:58:47.089678+0000","last_clean":"2024-08-30T08:58:47.089678+0000","last_became_active":"2024-08-30T08:58:09.440344+0000","last_became_peered":"2024-08-30T08:58:09.440344+0000","last_unstale":"2024-08-30T08:58:47.089678+0000","last_undegraded":"2024-08-30T08:58:47.089678+0000","last_fullsized":"2024-08-30T08:58:47.089678+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T12:06:47.113611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.553171+0000","last_change":"2024-08-30T08:58:02.407098+0000","last_active":"2024-08-30T08:58:46.553171+0000","last_peered":"2024-08-30T08:58:46.553171+0000","last_clean":"2024-08-30T08:58:46.553171+0000","last_became_active":"2024-08-30T08:58:02.406690+0000","last_became_peered":"2024-08-30T08:58:02.406690+0000","last_unstale":"2024-08-30T08:58:46.553171+0000","last_undegraded":"2024-08-30T08:58:46.553171+0000","last_fullsized":"2024-08-30T08:58:46.553171+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T18:27:39.534923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"79'3","reported_seq":38,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.554200+0000","last_change":"2024-08-30T08:58:00.386434+0000","last_active":"2024-08-30T08:58:46.554200+0000","last_peered":"2024-08-30T08:58:46.554200+0000","last_clean":"2024-08-30T08:58:46.554200+0000","last_became_active":"2024-08-30T08:58:00.385930+0000","last_became_peered":"2024-08-30T08:58:00.385930+0000","last_unstale":"2024-08-30T08:58:46.554200+0000","last_undegraded":"2024-08-30T08:58:46.554200+0000","last_fullsized":"2024-08-30T08:58:46.554200+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T10:39:22.441568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.775800+0000","last_change":"2024-08-30T08:58:46.775674+0000","last_active":"2024-08-30T08:58:46.775800+0000","last_peered":"2024-08-30T08:58:46.775800+0000","last_clean":"2024-08-30T08:58:46.775800+0000","last_became_active":"2024-08-30T08:58:46.775157+0000","last_became_peered":"2024-08-30T08:58:46.775157+0000","last_unstale":"2024-08-30T08:58:46.775800+0000","last_undegraded":"2024-08-30T08:58:46.775800+0000","last_fullsized":"2024-08-30T08:58:46.775800+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T17:13:47.066475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,2,5],"acting":[8,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.559287+0000","last_change":"2024-08-30T08:58:02.410500+0000","last_active":"2024-08-30T08:58:46.559287+0000","last_peered":"2024-08-30T08:58:46.559287+0000","last_clean":"2024-08-30T08:58:46.559287+0000","last_became_active":"2024-08-30T08:58:02.409260+0000","last_became_peered":"2024-08-30T08:58:02.409260+0000","last_unstale":"2024-08-30T08:58:46.559287+0000","last_undegraded":"2024-08-30T08:58:46.559287+0000","last_fullsized":"2024-08-30T08:58:46.559287+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T16:47:59.701956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"79'2","reported_seq":37,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.822713+0000","last_change":"2024-08-30T08:58:00.482296+0000","last_active":"2024-08-30T08:58:46.822713+0000","last_peered":"2024-08-30T08:58:46.822713+0000","last_clean":"2024-08-30T08:58:46.822713+0000","last_became_active":"2024-08-30T08:58:00.481920+0000","last_became_peered":"2024-08-30T08:58:00.481920+0000","last_unstale":"2024-08-30T08:58:46.822713+0000","last_undegraded":"2024-08-30T08:58:46.822713+0000","last_fullsized":"2024-08-30T08:58:46.822713+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T09:29:30.724410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:47.090717+0000","last_change":"2024-08-30T08:58:09.727879+0000","last_active":"2024-08-30T08:58:47.090717+0000","last_peered":"2024-08-30T08:58:47.090717+0000","last_clean":"2024-08-30T08:58:47.090717+0000","last_became_active":"2024-08-30T08:58:09.727339+0000","last_became_peered":"2024-08-30T08:58:09.727339+0000","last_unstale":"2024-08-30T08:58:47.090717+0000","last_undegraded":"2024-08-30T08:58:47.090717+0000","last_fullsized":"2024-08-30T08:58:47.090717+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T13:19:18.582668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":37,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.560099+0000","last_change":"2024-08-30T08:58:46.559988+0000","last_active":"2024-08-30T08:58:46.560099+0000","last_peered":"2024-08-30T08:58:46.560099+0000","last_clean":"2024-08-30T08:58:46.560099+0000","last_became_active":"2024-08-30T08:58:46.559442+0000","last_became_peered":"2024-08-30T08:58:46.559442+0000","last_unstale":"2024-08-30T08:58:46.560099+0000","last_undegraded":"2024-08-30T08:58:46.560099+0000","last_fullsized":"2024-08-30T08:58:46.560099+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T09:29:09.747756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"79'4","reported_seq":39,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:47.091026+0000","last_change":"2024-08-30T08:58:00.380338+0000","last_active":"2024-08-30T08:58:47.091026+0000","last_peered":"2024-08-30T08:58:47.091026+0000","last_clean":"2024-08-30T08:58:47.091026+0000","last_became_active":"2024-08-30T08:58:00.379860+0000","last_became_peered":"2024-08-30T08:58:00.379860+0000","last_unstale":"2024-08-30T08:58:47.091026+0000","last_undegraded":"2024-08-30T08:58:47.091026+0000","last_fullsized":"2024-08-30T08:58:47.091026+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T19:57:15.349906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.555200+0000","last_change":"2024-08-30T08:58:09.443631+0000","last_active":"2024-08-30T08:58:46.555200+0000","last_peered":"2024-08-30T08:58:46.555200+0000","last_clean":"2024-08-30T08:58:46.555200+0000","last_became_active":"2024-08-30T08:58:09.443103+0000","last_became_peered":"2024-08-30T08:58:09.443103+0000","last_unstale":"2024-08-30T08:58:46.555200+0000","last_undegraded":"2024-08-30T08:58:46.555200+0000","last_fullsized":"2024-08-30T08:58:46.555200+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T19:58:11.031826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.555619+0000","last_change":"2024-08-30T08:58:02.406216+0000","last_active":"2024-08-30T08:58:46.555619+0000","last_peered":"2024-08-30T08:58:46.555619+0000","last_clean":"2024-08-30T08:58:46.555619+0000","last_became_active":"2024-08-30T08:58:02.405040+0000","last_became_peered":"2024-08-30T08:58:02.405040+0000","last_unstale":"2024-08-30T08:58:46.555619+0000","last_undegraded":"2024-08-30T08:58:46.555619+0000","last_fullsized":"2024-08-30T08:58:46.555619+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T14:28:28.721386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"81'9","reported_seq":44,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:57.412986+0000","last_change":"2024-08-30T08:58:00.387350+0000","last_active":"2024-08-30T08:58:57.412986+0000","last_peered":"2024-08-30T08:58:57.412986+0000","last_clean":"2024-08-30T08:58:57.412986+0000","last_became_active":"2024-08-30T08:58:00.387004+0000","last_became_peered":"2024-08-30T08:58:00.387004+0000","last_unstale":"2024-08-30T08:58:57.412986+0000","last_undegraded":"2024-08-30T08:58:57.412986+0000","last_fullsized":"2024-08-30T08:58:57.412986+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T19:24:25.604544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.553684+0000","last_change":"2024-08-30T08:58:09.727388+0000","last_active":"2024-08-30T08:58:46.553684+0000","last_peered":"2024-08-30T08:58:46.553684+0000","last_clean":"2024-08-30T08:58:46.553684+0000","last_became_active":"2024-08-30T08:58:09.726935+0000","last_became_peered":"2024-08-30T08:58:09.726935+0000","last_unstale":"2024-08-30T08:58:46.553684+0000","last_undegraded":"2024-08-30T08:58:46.553684+0000","last_fullsized":"2024-08-30T08:58:46.553684+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T17:53:16.066602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.552226+0000","last_change":"2024-08-30T08:58:02.402661+0000","last_active":"2024-08-30T08:58:46.552226+0000","last_peered":"2024-08-30T08:58:46.552226+0000","last_clean":"2024-08-30T08:58:46.552226+0000","last_became_active":"2024-08-30T08:58:02.402314+0000","last_became_peered":"2024-08-30T08:58:02.402314+0000","last_unstale":"2024-08-30T08:58:46.552226+0000","last_undegraded":"2024-08-30T08:58:46.552226+0000","last_fullsized":"2024-08-30T08:58:46.552226+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T10:56:41.336312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"79'1","reported_seq":37,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.557749+0000","last_change":"2024-08-30T08:58:00.482720+0000","last_active":"2024-08-30T08:58:46.557749+0000","last_peered":"2024-08-30T08:58:46.557749+0000","last_clean":"2024-08-30T08:58:46.557749+0000","last_became_active":"2024-08-30T08:58:00.482193+0000","last_became_peered":"2024-08-30T08:58:00.482193+0000","last_unstale":"2024-08-30T08:58:46.557749+0000","last_undegraded":"2024-08-30T08:58:46.557749+0000","last_fullsized":"2024-08-30T08:58:46.557749+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T18:42:06.654806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.553347+0000","last_change":"2024-08-30T08:58:09.449970+0000","last_active":"2024-08-30T08:58:46.553347+0000","last_peered":"2024-08-30T08:58:46.553347+0000","last_clean":"2024-08-30T08:58:46.553347+0000","last_became_active":"2024-08-30T08:58:09.447891+0000","last_became_peered":"2024-08-30T08:58:09.447891+0000","last_unstale":"2024-08-30T08:58:46.553347+0000","last_undegraded":"2024-08-30T08:58:46.553347+0000","last_fullsized":"2024-08-30T08:58:46.553347+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T15:06:45.414470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.553365+0000","last_change":"2024-08-30T08:58:02.409756+0000","last_active":"2024-08-30T08:58:46.553365+0000","last_peered":"2024-08-30T08:58:46.553365+0000","last_clean":"2024-08-30T08:58:46.553365+0000","last_became_active":"2024-08-30T08:58:02.409357+0000","last_became_peered":"2024-08-30T08:58:02.409357+0000","last_unstale":"2024-08-30T08:58:46.553365+0000","last_undegraded":"2024-08-30T08:58:46.553365+0000","last_fullsized":"2024-08-30T08:58:46.553365+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T13:27:41.696527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":35,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.558745+0000","last_change":"2024-08-30T08:58:00.482342+0000","last_active":"2024-08-30T08:58:46.558745+0000","last_peered":"2024-08-30T08:58:46.558745+0000","last_clean":"2024-08-30T08:58:46.558745+0000","last_became_active":"2024-08-30T08:58:00.481902+0000","last_became_peered":"2024-08-30T08:58:00.481902+0000","last_unstale":"2024-08-30T08:58:46.558745+0000","last_undegraded":"2024-08-30T08:58:46.558745+0000","last_fullsized":"2024-08-30T08:58:46.558745+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T14:12:36.789356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.556901+0000","last_change":"2024-08-30T08:58:09.728008+0000","last_active":"2024-08-30T08:58:46.556901+0000","last_peered":"2024-08-30T08:58:46.556901+0000","last_clean":"2024-08-30T08:58:46.556901+0000","last_became_active":"2024-08-30T08:58:09.727513+0000","last_became_peered":"2024-08-30T08:58:09.727513+0000","last_unstale":"2024-08-30T08:58:46.556901+0000","last_undegraded":"2024-08-30T08:58:46.556901+0000","last_fullsized":"2024-08-30T08:58:46.556901+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T14:34:04.165300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.553329+0000","last_change":"2024-08-30T08:58:02.409383+0000","last_active":"2024-08-30T08:58:46.553329+0000","last_peered":"2024-08-30T08:58:46.553329+0000","last_clean":"2024-08-30T08:58:46.553329+0000","last_became_active":"2024-08-30T08:58:02.409027+0000","last_became_peered":"2024-08-30T08:58:02.409027+0000","last_unstale":"2024-08-30T08:58:46.553329+0000","last_undegraded":"2024-08-30T08:58:46.553329+0000","last_fullsized":"2024-08-30T08:58:46.553329+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T10:44:23.467130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"80'1","reported_seq":36,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:47.090595+0000","last_change":"2024-08-30T08:58:00.389083+0000","last_active":"2024-08-30T08:58:47.090595+0000","last_peered":"2024-08-30T08:58:47.090595+0000","last_clean":"2024-08-30T08:58:47.090595+0000","last_became_active":"2024-08-30T08:58:00.387552+0000","last_became_peered":"2024-08-30T08:58:00.387552+0000","last_unstale":"2024-08-30T08:58:47.090595+0000","last_undegraded":"2024-08-30T08:58:47.090595+0000","last_fullsized":"2024-08-30T08:58:47.090595+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T11:56:38.827406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.552061+0000","last_change":"2024-08-30T08:58:09.447758+0000","last_active":"2024-08-30T08:58:46.552061+0000","last_peered":"2024-08-30T08:58:46.552061+0000","last_clean":"2024-08-30T08:58:46.552061+0000","last_became_active":"2024-08-30T08:58:09.447333+0000","last_became_peered":"2024-08-30T08:58:09.447333+0000","last_unstale":"2024-08-30T08:58:46.552061+0000","last_undegraded":"2024-08-30T08:58:46.552061+0000","last_fullsized":"2024-08-30T08:58:46.552061+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T12:43:35.289967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:47.091640+0000","last_change":"2024-08-30T08:58:47.091191+0000","last_active":"2024-08-30T08:58:47.091640+0000","last_peered":"2024-08-30T08:58:47.091640+0000","last_clean":"2024-08-30T08:58:47.091640+0000","last_became_active":"2024-08-30T08:58:47.090826+0000","last_became_peered":"2024-08-30T08:58:47.090826+0000","last_unstale":"2024-08-30T08:58:47.091640+0000","last_undegraded":"2024-08-30T08:58:47.091640+0000","last_fullsized":"2024-08-30T08:58:47.091640+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T17:47:22.862514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"80'4","reported_seq":52,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:59:01.632921+0000","last_change":"2024-08-30T08:58:00.480651+0000","last_active":"2024-08-30T08:59:01.632921+0000","last_peered":"2024-08-30T08:59:01.632921+0000","last_clean":"2024-08-30T08:59:01.632921+0000","last_became_active":"2024-08-30T08:58:00.480205+0000","last_became_peered":"2024-08-30T08:58:00.480205+0000","last_unstale":"2024-08-30T08:59:01.632921+0000","last_undegraded":"2024-08-30T08:59:01.632921+0000","last_fullsized":"2024-08-30T08:59:01.632921+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T16:28:30.687076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":13,"num_read_kb":13,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.552066+0000","last_change":"2024-08-30T08:58:09.727530+0000","last_active":"2024-08-30T08:58:46.552066+0000","last_peered":"2024-08-30T08:58:46.552066+0000","last_clean":"2024-08-30T08:58:46.552066+0000","last_became_active":"2024-08-30T08:58:09.726965+0000","last_became_peered":"2024-08-30T08:58:09.726965+0000","last_unstale":"2024-08-30T08:58:46.552066+0000","last_undegraded":"2024-08-30T08:58:46.552066+0000","last_fullsized":"2024-08-30T08:58:46.552066+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T12:56:42.885615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.551578+0000","last_change":"2024-08-30T08:58:02.401673+0000","last_active":"2024-08-30T08:58:46.551578+0000","last_peered":"2024-08-30T08:58:46.551578+0000","last_clean":"2024-08-30T08:58:46.551578+0000","last_became_active":"2024-08-30T08:58:02.398876+0000","last_became_peered":"2024-08-30T08:58:02.398876+0000","last_unstale":"2024-08-30T08:58:46.551578+0000","last_undegraded":"2024-08-30T08:58:46.551578+0000","last_fullsized":"2024-08-30T08:58:46.551578+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T16:13:20.893737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"80'3","reported_seq":38,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:47.091220+0000","last_change":"2024-08-30T08:58:00.391722+0000","last_active":"2024-08-30T08:58:47.091220+0000","last_peered":"2024-08-30T08:58:47.091220+0000","last_clean":"2024-08-30T08:58:47.091220+0000","last_became_active":"2024-08-30T08:58:00.391325+0000","last_became_peered":"2024-08-30T08:58:00.391325+0000","last_unstale":"2024-08-30T08:58:47.091220+0000","last_undegraded":"2024-08-30T08:58:47.091220+0000","last_fullsized":"2024-08-30T08:58:47.091220+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T11:58:08.189202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.774243+0000","last_change":"2024-08-30T08:58:09.728324+0000","last_active":"2024-08-30T08:58:46.774243+0000","last_peered":"2024-08-30T08:58:46.774243+0000","last_clean":"2024-08-30T08:58:46.774243+0000","last_became_active":"2024-08-30T08:58:09.727520+0000","last_became_peered":"2024-08-30T08:58:09.727520+0000","last_unstale":"2024-08-30T08:58:46.774243+0000","last_undegraded":"2024-08-30T08:58:46.774243+0000","last_fullsized":"2024-08-30T08:58:46.774243+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T10:09:39.046820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.554063+0000","last_change":"2024-08-30T08:58:02.410826+0000","last_active":"2024-08-30T08:58:46.554063+0000","last_peered":"2024-08-30T08:58:46.554063+0000","last_clean":"2024-08-30T08:58:46.554063+0000","last_became_active":"2024-08-30T08:58:02.410464+0000","last_became_peered":"2024-08-30T08:58:02.410464+0000","last_unstale":"2024-08-30T08:58:46.554063+0000","last_undegraded":"2024-08-30T08:58:46.554063+0000","last_fullsized":"2024-08-30T08:58:46.554063+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T12:14:59.666614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"80'3","reported_seq":40,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:50.596022+0000","last_change":"2024-08-30T08:58:00.385464+0000","last_active":"2024-08-30T08:58:50.596022+0000","last_peered":"2024-08-30T08:58:50.596022+0000","last_clean":"2024-08-30T08:58:50.596022+0000","last_became_active":"2024-08-30T08:58:00.385083+0000","last_became_peered":"2024-08-30T08:58:00.385083+0000","last_unstale":"2024-08-30T08:58:50.596022+0000","last_undegraded":"2024-08-30T08:58:50.596022+0000","last_fullsized":"2024-08-30T08:58:50.596022+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T16:13:52.304755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":107,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:47.091434+0000","last_change":"2024-08-30T08:56:49.645080+0000","last_active":"2024-08-30T08:58:47.091434+0000","last_peered":"2024-08-30T08:58:47.091434+0000","last_clean":"2024-08-30T08:58:47.091434+0000","last_became_active":"2024-08-30T08:56:49.639066+0000","last_became_peered":"2024-08-30T08:56:49.639066+0000","last_unstale":"2024-08-30T08:58:47.091434+0000","last_undegraded":"2024-08-30T08:58:47.091434+0000","last_fullsized":"2024-08-30T08:58:47.091434+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:53:27.566846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:53:27.566846+0000","last_clean_scrub_stamp":"2024-08-30T08:53:27.566846+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T20:23:45.723824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"81'1","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:47.090916+0000","last_change":"2024-08-30T08:58:09.444007+0000","last_active":"2024-08-30T08:58:47.090916+0000","last_peered":"2024-08-30T08:58:47.090916+0000","last_clean":"2024-08-30T08:58:47.090916+0000","last_became_active":"2024-08-30T08:58:09.443580+0000","last_became_peered":"2024-08-30T08:58:09.443580+0000","last_unstale":"2024-08-30T08:58:47.090916+0000","last_undegraded":"2024-08-30T08:58:47.090916+0000","last_fullsized":"2024-08-30T08:58:47.090916+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T17:00:58.654026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:47.091022+0000","last_change":"2024-08-30T08:58:02.406973+0000","last_active":"2024-08-30T08:58:47.091022+0000","last_peered":"2024-08-30T08:58:47.091022+0000","last_clean":"2024-08-30T08:58:47.091022+0000","last_became_active":"2024-08-30T08:58:02.406576+0000","last_became_peered":"2024-08-30T08:58:02.406576+0000","last_unstale":"2024-08-30T08:58:47.091022+0000","last_undegraded":"2024-08-30T08:58:47.091022+0000","last_fullsized":"2024-08-30T08:58:47.091022+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T16:04:48.785310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"80'3","reported_seq":39,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.556017+0000","last_change":"2024-08-30T08:58:00.481019+0000","last_active":"2024-08-30T08:58:46.556017+0000","last_peered":"2024-08-30T08:58:46.556017+0000","last_clean":"2024-08-30T08:58:46.556017+0000","last_became_active":"2024-08-30T08:58:00.480590+0000","last_became_peered":"2024-08-30T08:58:00.480590+0000","last_unstale":"2024-08-30T08:58:46.556017+0000","last_undegraded":"2024-08-30T08:58:46.556017+0000","last_fullsized":"2024-08-30T08:58:46.556017+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T13:20:46.731591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.822612+0000","last_change":"2024-08-30T08:58:09.445723+0000","last_active":"2024-08-30T08:58:46.822612+0000","last_peered":"2024-08-30T08:58:46.822612+0000","last_clean":"2024-08-30T08:58:46.822612+0000","last_became_active":"2024-08-30T08:58:09.445252+0000","last_became_peered":"2024-08-30T08:58:09.445252+0000","last_unstale":"2024-08-30T08:58:46.822612+0000","last_undegraded":"2024-08-30T08:58:46.822612+0000","last_fullsized":"2024-08-30T08:58:46.822612+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T12:16:12.255990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:47.090625+0000","last_change":"2024-08-30T08:58:02.414250+0000","last_active":"2024-08-30T08:58:47.090625+0000","last_peered":"2024-08-30T08:58:47.090625+0000","last_clean":"2024-08-30T08:58:47.090625+0000","last_became_active":"2024-08-30T08:58:02.413684+0000","last_became_peered":"2024-08-30T08:58:02.413684+0000","last_unstale":"2024-08-30T08:58:47.090625+0000","last_undegraded":"2024-08-30T08:58:47.090625+0000","last_fullsized":"2024-08-30T08:58:47.090625+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T16:55:12.878806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"73'1","reported_seq":37,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.774624+0000","last_change":"2024-08-30T08:58:00.388869+0000","last_active":"2024-08-30T08:58:46.774624+0000","last_peered":"2024-08-30T08:58:46.774624+0000","last_clean":"2024-08-30T08:58:46.774624+0000","last_became_active":"2024-08-30T08:58:00.388116+0000","last_became_peered":"2024-08-30T08:58:00.388116+0000","last_unstale":"2024-08-30T08:58:46.774624+0000","last_undegraded":"2024-08-30T08:58:46.774624+0000","last_fullsized":"2024-08-30T08:58:46.774624+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T09:50:48.449205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:47.092091+0000","last_change":"2024-08-30T08:58:47.092004+0000","last_active":"2024-08-30T08:58:47.092091+0000","last_peered":"2024-08-30T08:58:47.092091+0000","last_clean":"2024-08-30T08:58:47.092091+0000","last_became_active":"2024-08-30T08:58:47.091661+0000","last_became_peered":"2024-08-30T08:58:47.091661+0000","last_unstale":"2024-08-30T08:58:47.092091+0000","last_undegraded":"2024-08-30T08:58:47.092091+0000","last_fullsized":"2024-08-30T08:58:47.092091+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T09:14:55.353465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,1,4],"acting":[0,8,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:47.090626+0000","last_change":"2024-08-30T08:58:02.410891+0000","last_active":"2024-08-30T08:58:47.090626+0000","last_peered":"2024-08-30T08:58:47.090626+0000","last_clean":"2024-08-30T08:58:47.090626+0000","last_became_active":"2024-08-30T08:58:02.410522+0000","last_became_peered":"2024-08-30T08:58:02.410522+0000","last_unstale":"2024-08-30T08:58:47.090626+0000","last_undegraded":"2024-08-30T08:58:47.090626+0000","last_fullsized":"2024-08-30T08:58:47.090626+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T09:32:21.540298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"73'2","reported_seq":48,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:47.095913+0000","last_change":"2024-08-30T08:58:47.095801+0000","last_active":"2024-08-30T08:58:47.095913+0000","last_peered":"2024-08-30T08:58:47.095913+0000","last_clean":"2024-08-30T08:58:47.095913+0000","last_became_active":"2024-08-30T08:58:47.092756+0000","last_became_peered":"2024-08-30T08:58:47.092756+0000","last_unstale":"2024-08-30T08:58:47.095913+0000","last_undegraded":"2024-08-30T08:58:47.095913+0000","last_fullsized":"2024-08-30T08:58:47.095913+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T19:10:02.064649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.773596+0000","last_change":"2024-08-30T08:58:09.449210+0000","last_active":"2024-08-30T08:58:46.773596+0000","last_peered":"2024-08-30T08:58:46.773596+0000","last_clean":"2024-08-30T08:58:46.773596+0000","last_became_active":"2024-08-30T08:58:09.448769+0000","last_became_peered":"2024-08-30T08:58:09.448769+0000","last_unstale":"2024-08-30T08:58:46.773596+0000","last_undegraded":"2024-08-30T08:58:46.773596+0000","last_fullsized":"2024-08-30T08:58:46.773596+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T09:26:13.845520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.553484+0000","last_change":"2024-08-30T08:58:02.414152+0000","last_active":"2024-08-30T08:58:46.553484+0000","last_peered":"2024-08-30T08:58:46.553484+0000","last_clean":"2024-08-30T08:58:46.553484+0000","last_became_active":"2024-08-30T08:58:02.413768+0000","last_became_peered":"2024-08-30T08:58:02.413768+0000","last_unstale":"2024-08-30T08:58:46.553484+0000","last_undegraded":"2024-08-30T08:58:46.553484+0000","last_fullsized":"2024-08-30T08:58:46.553484+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T18:34:09.447071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"81'2","reported_seq":38,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:50.596425+0000","last_change":"2024-08-30T08:58:00.390839+0000","last_active":"2024-08-30T08:58:50.596425+0000","last_peered":"2024-08-30T08:58:50.596425+0000","last_clean":"2024-08-30T08:58:50.596425+0000","last_became_active":"2024-08-30T08:58:00.390422+0000","last_became_peered":"2024-08-30T08:58:00.390422+0000","last_unstale":"2024-08-30T08:58:50.596425+0000","last_undegraded":"2024-08-30T08:58:50.596425+0000","last_fullsized":"2024-08-30T08:58:50.596425+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T15:05:23.005075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.824657+0000","last_change":"2024-08-30T08:58:46.824563+0000","last_active":"2024-08-30T08:58:46.824657+0000","last_peered":"2024-08-30T08:58:46.824657+0000","last_clean":"2024-08-30T08:58:46.824657+0000","last_became_active":"2024-08-30T08:58:46.824163+0000","last_became_peered":"2024-08-30T08:58:46.824163+0000","last_unstale":"2024-08-30T08:58:46.824657+0000","last_undegraded":"2024-08-30T08:58:46.824657+0000","last_fullsized":"2024-08-30T08:58:46.824657+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T19:25:13.202762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,3,1],"acting":[8,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.773839+0000","last_change":"2024-08-30T08:58:02.405122+0000","last_active":"2024-08-30T08:58:46.773839+0000","last_peered":"2024-08-30T08:58:46.773839+0000","last_clean":"2024-08-30T08:58:46.773839+0000","last_became_active":"2024-08-30T08:58:02.404695+0000","last_became_peered":"2024-08-30T08:58:02.404695+0000","last_unstale":"2024-08-30T08:58:46.773839+0000","last_undegraded":"2024-08-30T08:58:46.773839+0000","last_fullsized":"2024-08-30T08:58:46.773839+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T15:50:01.447032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"81'15","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:47.395221+0000","last_change":"2024-08-30T08:58:46.573221+0000","last_active":"2024-08-30T08:58:47.395221+0000","last_peered":"2024-08-30T08:58:47.395221+0000","last_clean":"2024-08-30T08:58:47.395221+0000","last_became_active":"2024-08-30T08:58:46.559562+0000","last_became_peered":"2024-08-30T08:58:46.559562+0000","last_unstale":"2024-08-30T08:58:47.395221+0000","last_undegraded":"2024-08-30T08:58:47.395221+0000","last_fullsized":"2024-08-30T08:58:47.395221+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T16:21:23.182676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.552177+0000","last_change":"2024-08-30T08:58:09.446063+0000","last_active":"2024-08-30T08:58:46.552177+0000","last_peered":"2024-08-30T08:58:46.552177+0000","last_clean":"2024-08-30T08:58:46.552177+0000","last_became_active":"2024-08-30T08:58:09.445615+0000","last_became_peered":"2024-08-30T08:58:09.445615+0000","last_unstale":"2024-08-30T08:58:46.552177+0000","last_undegraded":"2024-08-30T08:58:46.552177+0000","last_fullsized":"2024-08-30T08:58:46.552177+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T10:25:10.904892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:47.090684+0000","last_change":"2024-08-30T08:58:02.412178+0000","last_active":"2024-08-30T08:58:47.090684+0000","last_peered":"2024-08-30T08:58:47.090684+0000","last_clean":"2024-08-30T08:58:47.090684+0000","last_became_active":"2024-08-30T08:58:02.411814+0000","last_became_peered":"2024-08-30T08:58:02.411814+0000","last_unstale":"2024-08-30T08:58:47.090684+0000","last_undegraded":"2024-08-30T08:58:47.090684+0000","last_fullsized":"2024-08-30T08:58:47.090684+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T10:02:59.070204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"81'4","reported_seq":53,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:59:00.632290+0000","last_change":"2024-08-30T08:58:00.390231+0000","last_active":"2024-08-30T08:59:00.632290+0000","last_peered":"2024-08-30T08:59:00.632290+0000","last_clean":"2024-08-30T08:59:00.632290+0000","last_became_active":"2024-08-30T08:58:00.389852+0000","last_became_peered":"2024-08-30T08:58:00.389852+0000","last_unstale":"2024-08-30T08:59:00.632290+0000","last_undegraded":"2024-08-30T08:59:00.632290+0000","last_fullsized":"2024-08-30T08:59:00.632290+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T12:09:12.105416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":15,"num_read_kb":3,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.558813+0000","last_change":"2024-08-30T08:58:09.441652+0000","last_active":"2024-08-30T08:58:46.558813+0000","last_peered":"2024-08-30T08:58:46.558813+0000","last_clean":"2024-08-30T08:58:46.558813+0000","last_became_active":"2024-08-30T08:58:09.441264+0000","last_became_peered":"2024-08-30T08:58:09.441264+0000","last_unstale":"2024-08-30T08:58:46.558813+0000","last_undegraded":"2024-08-30T08:58:46.558813+0000","last_fullsized":"2024-08-30T08:58:46.558813+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T09:42:25.636669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:47.090297+0000","last_change":"2024-08-30T08:58:02.416347+0000","last_active":"2024-08-30T08:58:47.090297+0000","last_peered":"2024-08-30T08:58:47.090297+0000","last_clean":"2024-08-30T08:58:47.090297+0000","last_became_active":"2024-08-30T08:58:02.415898+0000","last_became_peered":"2024-08-30T08:58:02.415898+0000","last_unstale":"2024-08-30T08:58:47.090297+0000","last_undegraded":"2024-08-30T08:58:47.090297+0000","last_fullsized":"2024-08-30T08:58:47.090297+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T10:18:15.507076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"79'3","reported_seq":38,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.774168+0000","last_change":"2024-08-30T08:58:00.393616+0000","last_active":"2024-08-30T08:58:46.774168+0000","last_peered":"2024-08-30T08:58:46.774168+0000","last_clean":"2024-08-30T08:58:46.774168+0000","last_became_active":"2024-08-30T08:58:00.393218+0000","last_became_peered":"2024-08-30T08:58:00.393218+0000","last_unstale":"2024-08-30T08:58:46.774168+0000","last_undegraded":"2024-08-30T08:58:46.774168+0000","last_fullsized":"2024-08-30T08:58:46.774168+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T10:56:37.175968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:47.093131+0000","last_change":"2024-08-30T08:58:47.092990+0000","last_active":"2024-08-30T08:58:47.093131+0000","last_peered":"2024-08-30T08:58:47.093131+0000","last_clean":"2024-08-30T08:58:47.093131+0000","last_became_active":"2024-08-30T08:58:47.092448+0000","last_became_peered":"2024-08-30T08:58:47.092448+0000","last_unstale":"2024-08-30T08:58:47.093131+0000","last_undegraded":"2024-08-30T08:58:47.093131+0000","last_fullsized":"2024-08-30T08:58:47.093131+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T20:23:06.527336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,7,6],"acting":[11,10,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.551703+0000","last_change":"2024-08-30T08:58:02.405783+0000","last_active":"2024-08-30T08:58:46.551703+0000","last_peered":"2024-08-30T08:58:46.551703+0000","last_clean":"2024-08-30T08:58:46.551703+0000","last_became_active":"2024-08-30T08:58:02.405430+0000","last_became_peered":"2024-08-30T08:58:02.405430+0000","last_unstale":"2024-08-30T08:58:46.551703+0000","last_undegraded":"2024-08-30T08:58:46.551703+0000","last_fullsized":"2024-08-30T08:58:46.551703+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T19:40:08.377259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"80'6","reported_seq":42,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.555590+0000","last_change":"2024-08-30T08:58:00.389617+0000","last_active":"2024-08-30T08:58:46.555590+0000","last_peered":"2024-08-30T08:58:46.555590+0000","last_clean":"2024-08-30T08:58:46.555590+0000","last_became_active":"2024-08-30T08:58:00.389031+0000","last_became_peered":"2024-08-30T08:58:00.389031+0000","last_unstale":"2024-08-30T08:58:46.555590+0000","last_undegraded":"2024-08-30T08:58:46.555590+0000","last_fullsized":"2024-08-30T08:58:46.555590+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T15:42:38.050768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.774094+0000","last_change":"2024-08-30T08:58:09.451925+0000","last_active":"2024-08-30T08:58:46.774094+0000","last_peered":"2024-08-30T08:58:46.774094+0000","last_clean":"2024-08-30T08:58:46.774094+0000","last_became_active":"2024-08-30T08:58:09.446893+0000","last_became_peered":"2024-08-30T08:58:09.446893+0000","last_unstale":"2024-08-30T08:58:46.774094+0000","last_undegraded":"2024-08-30T08:58:46.774094+0000","last_fullsized":"2024-08-30T08:58:46.774094+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T14:34:25.134006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.557378+0000","last_change":"2024-08-30T08:58:02.411194+0000","last_active":"2024-08-30T08:58:46.557378+0000","last_peered":"2024-08-30T08:58:46.557378+0000","last_clean":"2024-08-30T08:58:46.557378+0000","last_became_active":"2024-08-30T08:58:02.410183+0000","last_became_peered":"2024-08-30T08:58:02.410183+0000","last_unstale":"2024-08-30T08:58:46.557378+0000","last_undegraded":"2024-08-30T08:58:46.557378+0000","last_fullsized":"2024-08-30T08:58:46.557378+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T10:56:19.016115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"81'30","reported_seq":96,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:59:01.624908+0000","last_change":"2024-08-30T08:58:00.389808+0000","last_active":"2024-08-30T08:59:01.624908+0000","last_peered":"2024-08-30T08:59:01.624908+0000","last_clean":"2024-08-30T08:59:01.624908+0000","last_became_active":"2024-08-30T08:58:00.389411+0000","last_became_peered":"2024-08-30T08:58:00.389411+0000","last_unstale":"2024-08-30T08:59:01.624908+0000","last_undegraded":"2024-08-30T08:59:01.624908+0000","last_fullsized":"2024-08-30T08:59:01.624908+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T18:59:45.468456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":54392,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":31,"num_read_kb":68,"num_write":31,"num_write_kb":65,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.557593+0000","last_change":"2024-08-30T08:58:09.447360+0000","last_active":"2024-08-30T08:58:46.557593+0000","last_peered":"2024-08-30T08:58:46.557593+0000","last_clean":"2024-08-30T08:58:46.557593+0000","last_became_active":"2024-08-30T08:58:09.446913+0000","last_became_peered":"2024-08-30T08:58:09.446913+0000","last_unstale":"2024-08-30T08:58:46.557593+0000","last_undegraded":"2024-08-30T08:58:46.557593+0000","last_fullsized":"2024-08-30T08:58:46.557593+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T13:59:43.147370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.822816+0000","last_change":"2024-08-30T08:58:02.408920+0000","last_active":"2024-08-30T08:58:46.822816+0000","last_peered":"2024-08-30T08:58:46.822816+0000","last_clean":"2024-08-30T08:58:46.822816+0000","last_became_active":"2024-08-30T08:58:02.408204+0000","last_became_peered":"2024-08-30T08:58:02.408204+0000","last_unstale":"2024-08-30T08:58:46.822816+0000","last_undegraded":"2024-08-30T08:58:46.822816+0000","last_fullsized":"2024-08-30T08:58:46.822816+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T11:44:00.765672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"79'4","reported_seq":40,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.774530+0000","last_change":"2024-08-30T08:58:00.482348+0000","last_active":"2024-08-30T08:58:46.774530+0000","last_peered":"2024-08-30T08:58:46.774530+0000","last_clean":"2024-08-30T08:58:46.774530+0000","last_became_active":"2024-08-30T08:58:00.481929+0000","last_became_peered":"2024-08-30T08:58:00.481929+0000","last_unstale":"2024-08-30T08:58:46.774530+0000","last_undegraded":"2024-08-30T08:58:46.774530+0000","last_fullsized":"2024-08-30T08:58:46.774530+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T11:13:55.332639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:47.089847+0000","last_change":"2024-08-30T08:58:09.443259+0000","last_active":"2024-08-30T08:58:47.089847+0000","last_peered":"2024-08-30T08:58:47.089847+0000","last_clean":"2024-08-30T08:58:47.089847+0000","last_became_active":"2024-08-30T08:58:09.442209+0000","last_became_peered":"2024-08-30T08:58:09.442209+0000","last_unstale":"2024-08-30T08:58:47.089847+0000","last_undegraded":"2024-08-30T08:58:47.089847+0000","last_fullsized":"2024-08-30T08:58:47.089847+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T09:25:56.233525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.557115+0000","last_change":"2024-08-30T08:58:02.407183+0000","last_active":"2024-08-30T08:58:46.557115+0000","last_peered":"2024-08-30T08:58:46.557115+0000","last_clean":"2024-08-30T08:58:46.557115+0000","last_became_active":"2024-08-30T08:58:02.406838+0000","last_became_peered":"2024-08-30T08:58:02.406838+0000","last_unstale":"2024-08-30T08:58:46.557115+0000","last_undegraded":"2024-08-30T08:58:46.557115+0000","last_fullsized":"2024-08-30T08:58:46.557115+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T13:40:32.937626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"79'2","reported_seq":37,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:47.090297+0000","last_change":"2024-08-30T08:58:00.392322+0000","last_active":"2024-08-30T08:58:47.090297+0000","last_peered":"2024-08-30T08:58:47.090297+0000","last_clean":"2024-08-30T08:58:47.090297+0000","last_became_active":"2024-08-30T08:58:00.391935+0000","last_became_peered":"2024-08-30T08:58:00.391935+0000","last_unstale":"2024-08-30T08:58:47.090297+0000","last_undegraded":"2024-08-30T08:58:47.090297+0000","last_fullsized":"2024-08-30T08:58:47.090297+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T12:20:52.368285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.557659+0000","last_change":"2024-08-30T08:58:09.441824+0000","last_active":"2024-08-30T08:58:46.557659+0000","last_peered":"2024-08-30T08:58:46.557659+0000","last_clean":"2024-08-30T08:58:46.557659+0000","last_became_active":"2024-08-30T08:58:09.441398+0000","last_became_peered":"2024-08-30T08:58:09.441398+0000","last_unstale":"2024-08-30T08:58:46.557659+0000","last_undegraded":"2024-08-30T08:58:46.557659+0000","last_fullsized":"2024-08-30T08:58:46.557659+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T14:05:45.710280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.823379+0000","last_change":"2024-08-30T08:58:02.403431+0000","last_active":"2024-08-30T08:58:46.823379+0000","last_peered":"2024-08-30T08:58:46.823379+0000","last_clean":"2024-08-30T08:58:46.823379+0000","last_became_active":"2024-08-30T08:58:02.403029+0000","last_became_peered":"2024-08-30T08:58:02.403029+0000","last_unstale":"2024-08-30T08:58:46.823379+0000","last_undegraded":"2024-08-30T08:58:46.823379+0000","last_fullsized":"2024-08-30T08:58:46.823379+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T09:34:56.452675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"79'3","reported_seq":50,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:59:00.615276+0000","last_change":"2024-08-30T08:58:00.389856+0000","last_active":"2024-08-30T08:59:00.615276+0000","last_peered":"2024-08-30T08:59:00.615276+0000","last_clean":"2024-08-30T08:59:00.615276+0000","last_became_active":"2024-08-30T08:58:00.389459+0000","last_became_peered":"2024-08-30T08:58:00.389459+0000","last_unstale":"2024-08-30T08:59:00.615276+0000","last_undegraded":"2024-08-30T08:59:00.615276+0000","last_fullsized":"2024-08-30T08:59:00.615276+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T18:32:06.214570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":12,"num_read_kb":12,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.553852+0000","last_change":"2024-08-30T08:58:09.445317+0000","last_active":"2024-08-30T08:58:46.553852+0000","last_peered":"2024-08-30T08:58:46.553852+0000","last_clean":"2024-08-30T08:58:46.553852+0000","last_became_active":"2024-08-30T08:58:09.444921+0000","last_became_peered":"2024-08-30T08:58:09.444921+0000","last_unstale":"2024-08-30T08:58:46.553852+0000","last_undegraded":"2024-08-30T08:58:46.553852+0000","last_fullsized":"2024-08-30T08:58:46.553852+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T09:22:13.038437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.556314+0000","last_change":"2024-08-30T08:58:09.447469+0000","last_active":"2024-08-30T08:58:46.556314+0000","last_peered":"2024-08-30T08:58:46.556314+0000","last_clean":"2024-08-30T08:58:46.556314+0000","last_became_active":"2024-08-30T08:58:09.447015+0000","last_became_peered":"2024-08-30T08:58:09.447015+0000","last_unstale":"2024-08-30T08:58:46.556314+0000","last_undegraded":"2024-08-30T08:58:46.556314+0000","last_fullsized":"2024-08-30T08:58:46.556314+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T09:18:36.734380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.557549+0000","last_change":"2024-08-30T08:58:02.403371+0000","last_active":"2024-08-30T08:58:46.557549+0000","last_peered":"2024-08-30T08:58:46.557549+0000","last_clean":"2024-08-30T08:58:46.557549+0000","last_became_active":"2024-08-30T08:58:02.403012+0000","last_became_peered":"2024-08-30T08:58:02.403012+0000","last_unstale":"2024-08-30T08:58:46.557549+0000","last_undegraded":"2024-08-30T08:58:46.557549+0000","last_fullsized":"2024-08-30T08:58:46.557549+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T10:29:29.683081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"79'1","reported_seq":36,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.774090+0000","last_change":"2024-08-30T08:58:00.394450+0000","last_active":"2024-08-30T08:58:46.774090+0000","last_peered":"2024-08-30T08:58:46.774090+0000","last_clean":"2024-08-30T08:58:46.774090+0000","last_became_active":"2024-08-30T08:58:00.391843+0000","last_became_peered":"2024-08-30T08:58:00.391843+0000","last_unstale":"2024-08-30T08:58:46.774090+0000","last_undegraded":"2024-08-30T08:58:46.774090+0000","last_fullsized":"2024-08-30T08:58:46.774090+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T10:29:18.947898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.553483+0000","last_change":"2024-08-30T08:58:09.449172+0000","last_active":"2024-08-30T08:58:46.553483+0000","last_peered":"2024-08-30T08:58:46.553483+0000","last_clean":"2024-08-30T08:58:46.553483+0000","last_became_active":"2024-08-30T08:58:09.448687+0000","last_became_peered":"2024-08-30T08:58:09.448687+0000","last_unstale":"2024-08-30T08:58:46.553483+0000","last_undegraded":"2024-08-30T08:58:46.553483+0000","last_fullsized":"2024-08-30T08:58:46.553483+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T13:23:16.090251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.553825+0000","last_change":"2024-08-30T08:58:02.400834+0000","last_active":"2024-08-30T08:58:46.553825+0000","last_peered":"2024-08-30T08:58:46.553825+0000","last_clean":"2024-08-30T08:58:46.553825+0000","last_became_active":"2024-08-30T08:58:02.400477+0000","last_became_peered":"2024-08-30T08:58:02.400477+0000","last_unstale":"2024-08-30T08:58:46.553825+0000","last_undegraded":"2024-08-30T08:58:46.553825+0000","last_fullsized":"2024-08-30T08:58:46.553825+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T13:40:45.239732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"80'2","reported_seq":37,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.551581+0000","last_change":"2024-08-30T08:58:00.381699+0000","last_active":"2024-08-30T08:58:46.551581+0000","last_peered":"2024-08-30T08:58:46.551581+0000","last_clean":"2024-08-30T08:58:46.551581+0000","last_became_active":"2024-08-30T08:58:00.381189+0000","last_became_peered":"2024-08-30T08:58:00.381189+0000","last_unstale":"2024-08-30T08:58:46.551581+0000","last_undegraded":"2024-08-30T08:58:46.551581+0000","last_fullsized":"2024-08-30T08:58:46.551581+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T14:09:16.939062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"80'3","reported_seq":38,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.553179+0000","last_change":"2024-08-30T08:58:00.387430+0000","last_active":"2024-08-30T08:58:46.553179+0000","last_peered":"2024-08-30T08:58:46.553179+0000","last_clean":"2024-08-30T08:58:46.553179+0000","last_became_active":"2024-08-30T08:58:00.387043+0000","last_became_peered":"2024-08-30T08:58:00.387043+0000","last_unstale":"2024-08-30T08:58:46.553179+0000","last_undegraded":"2024-08-30T08:58:46.553179+0000","last_fullsized":"2024-08-30T08:58:46.553179+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T13:32:25.418105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.774175+0000","last_change":"2024-08-30T08:58:02.404798+0000","last_active":"2024-08-30T08:58:46.774175+0000","last_peered":"2024-08-30T08:58:46.774175+0000","last_clean":"2024-08-30T08:58:46.774175+0000","last_became_active":"2024-08-30T08:58:02.404435+0000","last_became_peered":"2024-08-30T08:58:02.404435+0000","last_unstale":"2024-08-30T08:58:46.774175+0000","last_undegraded":"2024-08-30T08:58:46.774175+0000","last_fullsized":"2024-08-30T08:58:46.774175+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T20:31:49.472832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.555057+0000","last_change":"2024-08-30T08:58:09.448297+0000","last_active":"2024-08-30T08:58:46.555057+0000","last_peered":"2024-08-30T08:58:46.555057+0000","last_clean":"2024-08-30T08:58:46.555057+0000","last_became_active":"2024-08-30T08:58:09.447444+0000","last_became_peered":"2024-08-30T08:58:09.447444+0000","last_unstale":"2024-08-30T08:58:46.555057+0000","last_undegraded":"2024-08-30T08:58:46.555057+0000","last_fullsized":"2024-08-30T08:58:46.555057+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T11:40:03.507824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"81'6","reported_seq":55,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:59:00.615979+0000","last_change":"2024-08-30T08:58:00.392418+0000","last_active":"2024-08-30T08:59:00.615979+0000","last_peered":"2024-08-30T08:59:00.615979+0000","last_clean":"2024-08-30T08:59:00.615979+0000","last_became_active":"2024-08-30T08:58:00.391985+0000","last_became_peered":"2024-08-30T08:58:00.391985+0000","last_unstale":"2024-08-30T08:59:00.615979+0000","last_undegraded":"2024-08-30T08:59:00.615979+0000","last_fullsized":"2024-08-30T08:59:00.615979+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T17:42:18.147754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":14,"num_read_kb":2,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.557148+0000","last_change":"2024-08-30T08:58:02.408754+0000","last_active":"2024-08-30T08:58:46.557148+0000","last_peered":"2024-08-30T08:58:46.557148+0000","last_clean":"2024-08-30T08:58:46.557148+0000","last_became_active":"2024-08-30T08:58:02.407127+0000","last_became_peered":"2024-08-30T08:58:02.407127+0000","last_unstale":"2024-08-30T08:58:46.557148+0000","last_undegraded":"2024-08-30T08:58:46.557148+0000","last_fullsized":"2024-08-30T08:58:46.557148+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T16:15:41.796091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.555654+0000","last_change":"2024-08-30T08:58:09.445389+0000","last_active":"2024-08-30T08:58:46.555654+0000","last_peered":"2024-08-30T08:58:46.555654+0000","last_clean":"2024-08-30T08:58:46.555654+0000","last_became_active":"2024-08-30T08:58:09.444831+0000","last_became_peered":"2024-08-30T08:58:09.444831+0000","last_unstale":"2024-08-30T08:58:46.555654+0000","last_undegraded":"2024-08-30T08:58:46.555654+0000","last_fullsized":"2024-08-30T08:58:46.555654+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T09:55:58.162397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.553020+0000","last_change":"2024-08-30T08:58:02.415184+0000","last_active":"2024-08-30T08:58:46.553020+0000","last_peered":"2024-08-30T08:58:46.553020+0000","last_clean":"2024-08-30T08:58:46.553020+0000","last_became_active":"2024-08-30T08:58:02.414625+0000","last_became_peered":"2024-08-30T08:58:02.414625+0000","last_unstale":"2024-08-30T08:58:46.553020+0000","last_undegraded":"2024-08-30T08:58:46.553020+0000","last_fullsized":"2024-08-30T08:58:46.553020+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T11:10:31.130524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.555567+0000","last_change":"2024-08-30T08:58:02.405392+0000","last_active":"2024-08-30T08:58:46.555567+0000","last_peered":"2024-08-30T08:58:46.555567+0000","last_clean":"2024-08-30T08:58:46.555567+0000","last_became_active":"2024-08-30T08:58:02.405009+0000","last_became_peered":"2024-08-30T08:58:02.405009+0000","last_unstale":"2024-08-30T08:58:46.555567+0000","last_undegraded":"2024-08-30T08:58:46.555567+0000","last_fullsized":"2024-08-30T08:58:46.555567+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T13:22:46.375598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.555680+0000","last_change":"2024-08-30T08:58:02.402360+0000","last_active":"2024-08-30T08:58:46.555680+0000","last_peered":"2024-08-30T08:58:46.555680+0000","last_clean":"2024-08-30T08:58:46.555680+0000","last_became_active":"2024-08-30T08:58:02.401856+0000","last_became_peered":"2024-08-30T08:58:02.401856+0000","last_unstale":"2024-08-30T08:58:46.555680+0000","last_undegraded":"2024-08-30T08:58:46.555680+0000","last_fullsized":"2024-08-30T08:58:46.555680+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T10:25:14.491345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.557920+0000","last_change":"2024-08-30T08:58:02.415940+0000","last_active":"2024-08-30T08:58:46.557920+0000","last_peered":"2024-08-30T08:58:46.557920+0000","last_clean":"2024-08-30T08:58:46.557920+0000","last_became_active":"2024-08-30T08:58:02.415567+0000","last_became_peered":"2024-08-30T08:58:02.415567+0000","last_unstale":"2024-08-30T08:58:46.557920+0000","last_undegraded":"2024-08-30T08:58:46.557920+0000","last_fullsized":"2024-08-30T08:58:46.557920+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T15:02:26.270343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.552160+0000","last_change":"2024-08-30T08:58:02.406766+0000","last_active":"2024-08-30T08:58:46.552160+0000","last_peered":"2024-08-30T08:58:46.552160+0000","last_clean":"2024-08-30T08:58:46.552160+0000","last_became_active":"2024-08-30T08:58:02.406345+0000","last_became_peered":"2024-08-30T08:58:02.406345+0000","last_unstale":"2024-08-30T08:58:46.552160+0000","last_undegraded":"2024-08-30T08:58:46.552160+0000","last_fullsized":"2024-08-30T08:58:46.552160+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T13:10:32.986817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.553714+0000","last_change":"2024-08-30T08:58:02.412891+0000","last_active":"2024-08-30T08:58:46.553714+0000","last_peered":"2024-08-30T08:58:46.553714+0000","last_clean":"2024-08-30T08:58:46.553714+0000","last_became_active":"2024-08-30T08:58:02.412532+0000","last_became_peered":"2024-08-30T08:58:02.412532+0000","last_unstale":"2024-08-30T08:58:46.553714+0000","last_undegraded":"2024-08-30T08:58:46.553714+0000","last_fullsized":"2024-08-30T08:58:46.553714+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T14:40:30.663678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.556273+0000","last_change":"2024-08-30T08:58:02.405862+0000","last_active":"2024-08-30T08:58:46.556273+0000","last_peered":"2024-08-30T08:58:46.556273+0000","last_clean":"2024-08-30T08:58:46.556273+0000","last_became_active":"2024-08-30T08:58:02.405319+0000","last_became_peered":"2024-08-30T08:58:02.405319+0000","last_unstale":"2024-08-30T08:58:46.556273+0000","last_undegraded":"2024-08-30T08:58:46.556273+0000","last_fullsized":"2024-08-30T08:58:46.556273+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T20:36:10.315814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.555535+0000","last_change":"2024-08-30T08:58:02.401410+0000","last_active":"2024-08-30T08:58:46.555535+0000","last_peered":"2024-08-30T08:58:46.555535+0000","last_clean":"2024-08-30T08:58:46.555535+0000","last_became_active":"2024-08-30T08:58:02.401060+0000","last_became_peered":"2024-08-30T08:58:02.401060+0000","last_unstale":"2024-08-30T08:58:46.555535+0000","last_undegraded":"2024-08-30T08:58:46.555535+0000","last_fullsized":"2024-08-30T08:58:46.555535+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T16:00:41.275367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.773667+0000","last_change":"2024-08-30T08:58:02.401703+0000","last_active":"2024-08-30T08:58:46.773667+0000","last_peered":"2024-08-30T08:58:46.773667+0000","last_clean":"2024-08-30T08:58:46.773667+0000","last_became_active":"2024-08-30T08:58:02.401218+0000","last_became_peered":"2024-08-30T08:58:02.401218+0000","last_unstale":"2024-08-30T08:58:46.773667+0000","last_undegraded":"2024-08-30T08:58:46.773667+0000","last_fullsized":"2024-08-30T08:58:46.773667+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T18:36:25.249814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.554007+0000","last_change":"2024-08-30T08:58:02.403609+0000","last_active":"2024-08-30T08:58:46.554007+0000","last_peered":"2024-08-30T08:58:46.554007+0000","last_clean":"2024-08-30T08:58:46.554007+0000","last_became_active":"2024-08-30T08:58:02.402195+0000","last_became_peered":"2024-08-30T08:58:02.402195+0000","last_unstale":"2024-08-30T08:58:46.554007+0000","last_undegraded":"2024-08-30T08:58:46.554007+0000","last_fullsized":"2024-08-30T08:58:46.554007+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T09:59:05.877523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.823042+0000","last_change":"2024-08-30T08:58:02.404389+0000","last_active":"2024-08-30T08:58:46.823042+0000","last_peered":"2024-08-30T08:58:46.823042+0000","last_clean":"2024-08-30T08:58:46.823042+0000","last_became_active":"2024-08-30T08:58:02.404039+0000","last_became_peered":"2024-08-30T08:58:02.404039+0000","last_unstale":"2024-08-30T08:58:46.823042+0000","last_undegraded":"2024-08-30T08:58:46.823042+0000","last_fullsized":"2024-08-30T08:58:46.823042+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T20:55:51.739542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.552060+0000","last_change":"2024-08-30T08:58:02.411807+0000","last_active":"2024-08-30T08:58:46.552060+0000","last_peered":"2024-08-30T08:58:46.552060+0000","last_clean":"2024-08-30T08:58:46.552060+0000","last_became_active":"2024-08-30T08:58:02.410977+0000","last_became_peered":"2024-08-30T08:58:02.410977+0000","last_unstale":"2024-08-30T08:58:46.552060+0000","last_undegraded":"2024-08-30T08:58:46.552060+0000","last_fullsized":"2024-08-30T08:58:46.552060+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T13:04:29.931373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":37,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.560319+0000","last_change":"2024-08-30T08:58:46.559661+0000","last_active":"2024-08-30T08:58:46.560319+0000","last_peered":"2024-08-30T08:58:46.560319+0000","last_clean":"2024-08-30T08:58:46.560319+0000","last_became_active":"2024-08-30T08:58:46.558824+0000","last_became_peered":"2024-08-30T08:58:46.558824+0000","last_unstale":"2024-08-30T08:58:46.560319+0000","last_undegraded":"2024-08-30T08:58:46.560319+0000","last_fullsized":"2024-08-30T08:58:46.560319+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T09:53:07.650921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.556117+0000","last_change":"2024-08-30T08:58:02.396186+0000","last_active":"2024-08-30T08:58:46.556117+0000","last_peered":"2024-08-30T08:58:46.556117+0000","last_clean":"2024-08-30T08:58:46.556117+0000","last_became_active":"2024-08-30T08:58:02.395776+0000","last_became_peered":"2024-08-30T08:58:02.395776+0000","last_unstale":"2024-08-30T08:58:46.556117+0000","last_undegraded":"2024-08-30T08:58:46.556117+0000","last_fullsized":"2024-08-30T08:58:46.556117+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T09:33:30.068673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.559487+0000","last_change":"2024-08-30T08:58:02.408378+0000","last_active":"2024-08-30T08:58:46.559487+0000","last_peered":"2024-08-30T08:58:46.559487+0000","last_clean":"2024-08-30T08:58:46.559487+0000","last_became_active":"2024-08-30T08:58:02.407991+0000","last_became_peered":"2024-08-30T08:58:02.407991+0000","last_unstale":"2024-08-30T08:58:46.559487+0000","last_undegraded":"2024-08-30T08:58:46.559487+0000","last_fullsized":"2024-08-30T08:58:46.559487+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T20:57:02.411176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.553823+0000","last_change":"2024-08-30T08:58:02.411829+0000","last_active":"2024-08-30T08:58:46.553823+0000","last_peered":"2024-08-30T08:58:46.553823+0000","last_clean":"2024-08-30T08:58:46.553823+0000","last_became_active":"2024-08-30T08:58:02.411416+0000","last_became_peered":"2024-08-30T08:58:02.411416+0000","last_unstale":"2024-08-30T08:58:46.553823+0000","last_undegraded":"2024-08-30T08:58:46.553823+0000","last_fullsized":"2024-08-30T08:58:46.553823+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T17:12:08.855150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.822489+0000","last_change":"2024-08-30T08:58:02.405122+0000","last_active":"2024-08-30T08:58:46.822489+0000","last_peered":"2024-08-30T08:58:46.822489+0000","last_clean":"2024-08-30T08:58:46.822489+0000","last_became_active":"2024-08-30T08:58:02.403312+0000","last_became_peered":"2024-08-30T08:58:02.403312+0000","last_unstale":"2024-08-30T08:58:46.822489+0000","last_undegraded":"2024-08-30T08:58:46.822489+0000","last_fullsized":"2024-08-30T08:58:46.822489+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T18:11:53.077433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.556239+0000","last_change":"2024-08-30T08:58:02.404263+0000","last_active":"2024-08-30T08:58:46.556239+0000","last_peered":"2024-08-30T08:58:46.556239+0000","last_clean":"2024-08-30T08:58:46.556239+0000","last_became_active":"2024-08-30T08:58:02.403897+0000","last_became_peered":"2024-08-30T08:58:02.403897+0000","last_unstale":"2024-08-30T08:58:46.556239+0000","last_undegraded":"2024-08-30T08:58:46.556239+0000","last_fullsized":"2024-08-30T08:58:46.556239+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T18:38:31.439434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.556196+0000","last_change":"2024-08-30T08:58:02.411578+0000","last_active":"2024-08-30T08:58:46.556196+0000","last_peered":"2024-08-30T08:58:46.556196+0000","last_clean":"2024-08-30T08:58:46.556196+0000","last_became_active":"2024-08-30T08:58:02.411271+0000","last_became_peered":"2024-08-30T08:58:02.411271+0000","last_unstale":"2024-08-30T08:58:46.556196+0000","last_undegraded":"2024-08-30T08:58:46.556196+0000","last_fullsized":"2024-08-30T08:58:46.556196+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T10:02:46.382906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.552115+0000","last_change":"2024-08-30T08:58:02.412168+0000","last_active":"2024-08-30T08:58:46.552115+0000","last_peered":"2024-08-30T08:58:46.552115+0000","last_clean":"2024-08-30T08:58:46.552115+0000","last_became_active":"2024-08-30T08:58:02.411420+0000","last_became_peered":"2024-08-30T08:58:02.411420+0000","last_unstale":"2024-08-30T08:58:46.552115+0000","last_undegraded":"2024-08-30T08:58:46.552115+0000","last_fullsized":"2024-08-30T08:58:46.552115+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T10:35:53.654344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:47.089927+0000","last_change":"2024-08-30T08:58:02.409735+0000","last_active":"2024-08-30T08:58:47.089927+0000","last_peered":"2024-08-30T08:58:47.089927+0000","last_clean":"2024-08-30T08:58:47.089927+0000","last_became_active":"2024-08-30T08:58:02.409349+0000","last_became_peered":"2024-08-30T08:58:02.409349+0000","last_unstale":"2024-08-30T08:58:47.089927+0000","last_undegraded":"2024-08-30T08:58:47.089927+0000","last_fullsized":"2024-08-30T08:58:47.089927+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T20:10:22.956554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:47.090762+0000","last_change":"2024-08-30T08:58:02.416261+0000","last_active":"2024-08-30T08:58:47.090762+0000","last_peered":"2024-08-30T08:58:47.090762+0000","last_clean":"2024-08-30T08:58:47.090762+0000","last_became_active":"2024-08-30T08:58:02.415194+0000","last_became_peered":"2024-08-30T08:58:02.415194+0000","last_unstale":"2024-08-30T08:58:47.090762+0000","last_undegraded":"2024-08-30T08:58:47.090762+0000","last_fullsized":"2024-08-30T08:58:47.090762+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T17:59:04.339699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":37,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.560326+0000","last_change":"2024-08-30T08:58:46.560190+0000","last_active":"2024-08-30T08:58:46.560326+0000","last_peered":"2024-08-30T08:58:46.560326+0000","last_clean":"2024-08-30T08:58:46.560326+0000","last_became_active":"2024-08-30T08:58:46.559674+0000","last_became_peered":"2024-08-30T08:58:46.559674+0000","last_unstale":"2024-08-30T08:58:46.560326+0000","last_undegraded":"2024-08-30T08:58:46.560326+0000","last_fullsized":"2024-08-30T08:58:46.560326+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T10:08:02.835142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.823261+0000","last_change":"2024-08-30T08:58:02.406763+0000","last_active":"2024-08-30T08:58:46.823261+0000","last_peered":"2024-08-30T08:58:46.823261+0000","last_clean":"2024-08-30T08:58:46.823261+0000","last_became_active":"2024-08-30T08:58:02.406377+0000","last_became_peered":"2024-08-30T08:58:02.406377+0000","last_unstale":"2024-08-30T08:58:46.823261+0000","last_undegraded":"2024-08-30T08:58:46.823261+0000","last_fullsized":"2024-08-30T08:58:46.823261+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T13:32:50.492921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.553816+0000","last_change":"2024-08-30T08:58:02.416638+0000","last_active":"2024-08-30T08:58:46.553816+0000","last_peered":"2024-08-30T08:58:46.553816+0000","last_clean":"2024-08-30T08:58:46.553816+0000","last_became_active":"2024-08-30T08:58:02.415518+0000","last_became_peered":"2024-08-30T08:58:02.415518+0000","last_unstale":"2024-08-30T08:58:46.553816+0000","last_undegraded":"2024-08-30T08:58:46.553816+0000","last_fullsized":"2024-08-30T08:58:46.553816+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T17:05:07.531585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:47.089992+0000","last_change":"2024-08-30T08:58:02.412973+0000","last_active":"2024-08-30T08:58:47.089992+0000","last_peered":"2024-08-30T08:58:47.089992+0000","last_clean":"2024-08-30T08:58:47.089992+0000","last_became_active":"2024-08-30T08:58:02.412609+0000","last_became_peered":"2024-08-30T08:58:02.412609+0000","last_unstale":"2024-08-30T08:58:47.089992+0000","last_undegraded":"2024-08-30T08:58:47.089992+0000","last_fullsized":"2024-08-30T08:58:47.089992+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T14:22:37.704098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.553161+0000","last_change":"2024-08-30T08:58:02.402779+0000","last_active":"2024-08-30T08:58:46.553161+0000","last_peered":"2024-08-30T08:58:46.553161+0000","last_clean":"2024-08-30T08:58:46.553161+0000","last_became_active":"2024-08-30T08:58:02.401514+0000","last_became_peered":"2024-08-30T08:58:02.401514+0000","last_unstale":"2024-08-30T08:58:46.553161+0000","last_undegraded":"2024-08-30T08:58:46.553161+0000","last_fullsized":"2024-08-30T08:58:46.553161+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T11:37:51.450334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.557162+0000","last_change":"2024-08-30T08:58:02.406607+0000","last_active":"2024-08-30T08:58:46.557162+0000","last_peered":"2024-08-30T08:58:46.557162+0000","last_clean":"2024-08-30T08:58:46.557162+0000","last_became_active":"2024-08-30T08:58:02.406237+0000","last_became_peered":"2024-08-30T08:58:02.406237+0000","last_unstale":"2024-08-30T08:58:46.557162+0000","last_undegraded":"2024-08-30T08:58:46.557162+0000","last_fullsized":"2024-08-30T08:58:46.557162+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T12:09:03.247424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:47.089986+0000","last_change":"2024-08-30T08:58:02.410749+0000","last_active":"2024-08-30T08:58:47.089986+0000","last_peered":"2024-08-30T08:58:47.089986+0000","last_clean":"2024-08-30T08:58:47.089986+0000","last_became_active":"2024-08-30T08:58:02.410350+0000","last_became_peered":"2024-08-30T08:58:02.410350+0000","last_unstale":"2024-08-30T08:58:47.089986+0000","last_undegraded":"2024-08-30T08:58:47.089986+0000","last_fullsized":"2024-08-30T08:58:47.089986+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T15:22:51.451789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.557090+0000","last_change":"2024-08-30T08:58:02.405946+0000","last_active":"2024-08-30T08:58:46.557090+0000","last_peered":"2024-08-30T08:58:46.557090+0000","last_clean":"2024-08-30T08:58:46.557090+0000","last_became_active":"2024-08-30T08:58:02.405590+0000","last_became_peered":"2024-08-30T08:58:02.405590+0000","last_unstale":"2024-08-30T08:58:46.557090+0000","last_undegraded":"2024-08-30T08:58:46.557090+0000","last_fullsized":"2024-08-30T08:58:46.557090+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T11:42:14.966065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.556440+0000","last_change":"2024-08-30T08:58:02.409649+0000","last_active":"2024-08-30T08:58:46.556440+0000","last_peered":"2024-08-30T08:58:46.556440+0000","last_clean":"2024-08-30T08:58:46.556440+0000","last_became_active":"2024-08-30T08:58:02.409282+0000","last_became_peered":"2024-08-30T08:58:02.409282+0000","last_unstale":"2024-08-30T08:58:46.556440+0000","last_undegraded":"2024-08-30T08:58:46.556440+0000","last_fullsized":"2024-08-30T08:58:46.556440+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T15:33:52.166290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.553800+0000","last_change":"2024-08-30T08:58:02.410711+0000","last_active":"2024-08-30T08:58:46.553800+0000","last_peered":"2024-08-30T08:58:46.553800+0000","last_clean":"2024-08-30T08:58:46.553800+0000","last_became_active":"2024-08-30T08:58:02.410367+0000","last_became_peered":"2024-08-30T08:58:02.410367+0000","last_unstale":"2024-08-30T08:58:46.553800+0000","last_undegraded":"2024-08-30T08:58:46.553800+0000","last_fullsized":"2024-08-30T08:58:46.553800+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T15:27:40.392588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":58784,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":98,"num_read_kb":111,"num_write":161,"num_write_kb":107,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":405504,"data_stored":176352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":139,"ondisk_log_size":139,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":67,"seq":287762808855,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":925696,"data_stored":455486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":61,"seq":261993005085,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28588,"kb_used_data":1564,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716564,"statfs":{"total":95995035648,"available":95965761536,"internally_reserved":0,"allocated":1601536,"data_stored":1097425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":57,"seq":244813135906,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36260,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708892,"statfs":{"total":95995035648,"available":95957905408,"internally_reserved":0,"allocated":937984,"data_stored":459706,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":51,"seq":219043332136,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":925696,"data_stored":455418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":46,"seq":197568495660,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28516,"kb_used_data":1492,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716636,"statfs":{"total":95995035648,"available":95965835264,"internally_reserved":0,"allocated":1527808,"data_stored":1051830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":41,"seq":176093659186,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":913408,"data_stored":450786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":35,"seq":150323855415,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36844,"kb_used_data":1500,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708308,"statfs":{"total":95995035648,"available":95957307392,"internally_reserved":0,"allocated":1536000,"data_stored":1047027,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":30,"seq":128849018940,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32088,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713064,"statfs":{"total":95995035648,"available":95962177536,"internally_reserved":0,"allocated":925696,"data_stored":455827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215168,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36248,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708904,"statfs":{"total":95995035648,"available":95957917696,"internally_reserved":0,"allocated":925696,"data_stored":450980,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411398,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36320,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708832,"statfs":{"total":95995035648,"available":95957843968,"internally_reserved":0,"allocated":999424,"data_stored":506829,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574923,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":974848,"data_stored":505088,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738447,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":937984,"data_stored":452174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":54392,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":56133,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":56361,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-30T08:59:05.547 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-08-30T08:59:05.548 INFO:tasks.cephadm:Setting up client nodes... 2024-08-30T08:59:05.548 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-08-30T08:59:05.804 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T08:59:06.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:06.283+0000 7f67c3354640 1 -- 172.21.15.42:0/1148044187 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f67bc10b470 msgr2=0x7f67bc10d980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:06.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:06.283+0000 7f67c3354640 1 --2- 172.21.15.42:0/1148044187 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f67bc10b470 0x7f67bc10d980 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f67b4009ca0 tx=0x7f67b402a8c0 comp rx=0 tx=0).stop 2024-08-30T08:59:06.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:06.283+0000 7f67c3354640 1 -- 172.21.15.42:0/1148044187 shutdown_connections 2024-08-30T08:59:06.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:06.283+0000 7f67c3354640 1 --2- 172.21.15.42:0/1148044187 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f67bc108ac0 0x7f67bc10aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:06.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:06.283+0000 7f67c3354640 1 --2- 172.21.15.42:0/1148044187 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f67bc10b470 0x7f67bc10d980 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:06.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:06.283+0000 7f67c3354640 1 --2- 172.21.15.42:0/1148044187 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f67bc06c1b0 0x7f67bc108580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:06.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:06.283+0000 7f67c3354640 1 -- 172.21.15.42:0/1148044187 >> 172.21.15.42:0/1148044187 conn(0x7f67bc0fddc0 msgr2=0x7f67bc100250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:59:06.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:06.283+0000 7f67c3354640 1 -- 172.21.15.42:0/1148044187 shutdown_connections 2024-08-30T08:59:06.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:06.283+0000 7f67c3354640 1 -- 172.21.15.42:0/1148044187 wait complete. 2024-08-30T08:59:06.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:06.283+0000 7f67c3354640 1 Processor -- start 2024-08-30T08:59:06.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:06.283+0000 7f67c3354640 1 -- start start 2024-08-30T08:59:06.294 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:06.284+0000 7f67c3354640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f67bc06c1b0 0x7f67bc19bed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:06.294 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:06.284+0000 7f67c3354640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f67bc108ac0 0x7f67bc19c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:06.294 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:06.284+0000 7f67c3354640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f67bc10b470 0x7f67bc19d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:06.294 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:06.284+0000 7f67c3354640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f67bc1113d0 con 0x7f67bc06c1b0 2024-08-30T08:59:06.294 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:06.284+0000 7f67c3354640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f67bc111250 con 0x7f67bc108ac0 2024-08-30T08:59:06.294 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:06.284+0000 7f67c3354640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f67bc111550 con 0x7f67bc10b470 2024-08-30T08:59:06.294 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:06.284+0000 7f67c10f0640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f67bc06c1b0 0x7f67bc19bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:06.294 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:06.284+0000 7f67c10f0640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f67bc06c1b0 0x7f67bc19bed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:32824/0 (socket says 172.21.15.42:32824) 2024-08-30T08:59:06.294 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:06.284+0000 7f67c10f0640 1 -- 172.21.15.42:0/3222538518 learned_addr learned my addr 172.21.15.42:0/3222538518 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T08:59:06.294 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:06.284+0000 7f67c08ef640 1 --2- 172.21.15.42:0/3222538518 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f67bc108ac0 0x7f67bc19c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:06.294 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:06.284+0000 7f67c18f1640 1 --2- 172.21.15.42:0/3222538518 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f67bc10b470 0x7f67bc19d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:06.294 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:06.284+0000 7f67c10f0640 1 -- 172.21.15.42:0/3222538518 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f67bc10b470 msgr2=0x7f67bc19d0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:06.294 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:06.284+0000 7f67c10f0640 1 --2- 172.21.15.42:0/3222538518 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f67bc10b470 0x7f67bc19d0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:06.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:06.284+0000 7f67c10f0640 1 -- 172.21.15.42:0/3222538518 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f67bc108ac0 msgr2=0x7f67bc19c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:06.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:06.284+0000 7f67c10f0640 1 --2- 172.21.15.42:0/3222538518 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f67bc108ac0 0x7f67bc19c540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:06.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:06.284+0000 7f67c10f0640 1 -- 172.21.15.42:0/3222538518 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67bc19d900 con 0x7f67bc06c1b0 2024-08-30T08:59:06.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:06.284+0000 7f67c10f0640 1 --2- 172.21.15.42:0/3222538518 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f67bc06c1b0 0x7f67bc19bed0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f67b800e9f0 tx=0x7f67b800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:59:06.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:06.284+0000 7f67b27fc640 1 -- 172.21.15.42:0/3222538518 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67b800cd80 con 0x7f67bc06c1b0 2024-08-30T08:59:06.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:06.284+0000 7f67c3354640 1 -- 172.21.15.42:0/3222538518 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67bc19dbf0 con 0x7f67bc06c1b0 2024-08-30T08:59:06.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:06.284+0000 7f67b27fc640 1 -- 172.21.15.42:0/3222538518 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f67b80108e0 con 0x7f67bc06c1b0 2024-08-30T08:59:06.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:06.285+0000 7f67b27fc640 1 -- 172.21.15.42:0/3222538518 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67b8018980 con 0x7f67bc06c1b0 2024-08-30T08:59:06.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:06.285+0000 7f67c3354640 1 -- 172.21.15.42:0/3222538518 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67bc1a42f0 con 0x7f67bc06c1b0 2024-08-30T08:59:06.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:06.285+0000 7f67c3354640 1 -- 172.21.15.42:0/3222538518 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67bc108580 con 0x7f67bc06c1b0 2024-08-30T08:59:06.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:06.287+0000 7f67b27fc640 1 -- 172.21.15.42:0/3222538518 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) ==== 152628+0+0 (secure 0 0 0) 0x7f67b8010430 con 0x7f67bc06c1b0 2024-08-30T08:59:06.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:06.288+0000 7f67b27fc640 1 --2- 172.21.15.42:0/3222538518 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f67a80b4bc0 0x7f67a80b70d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:06.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:06.288+0000 7f67b27fc640 1 -- 172.21.15.42:0/3222538518 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 9092+0+0 (secure 0 0 0) 0x7f67b80e5ec0 con 0x7f67bc06c1b0 2024-08-30T08:59:06.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:06.288+0000 7f67c08ef640 1 --2- 172.21.15.42:0/3222538518 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f67a80b4bc0 0x7f67a80b70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:06.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:06.288+0000 7f67c08ef640 1 --2- 172.21.15.42:0/3222538518 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f67a80b4bc0 0x7f67a80b70d0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f67ac005fd0 tx=0x7f67ac00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:59:06.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:06.291+0000 7f67b27fc640 1 -- 172.21.15.42:0/3222538518 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f67b80acb30 con 0x7f67bc06c1b0 2024-08-30T08:59:06.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:06.553+0000 7f67c3354640 1 -- 172.21.15.42:0/3222538518 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7f67bc0ff8e0 con 0x7f67bc06c1b0 2024-08-30T08:59:06.556 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:59:06 smithi042 ceph-mon[18652]: from='client.15009 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-30T08:59:06.556 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:59:06 smithi042 ceph-mon[18652]: pgmap v321: 129 pgs: 129 active+clean; 634 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 3.3 KiB/s wr, 7 op/s 2024-08-30T08:59:06.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:06.556+0000 7f67b27fc640 1 -- 172.21.15.42:0/3222538518 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7f67b80b5070 con 0x7f67bc06c1b0 2024-08-30T08:59:06.558 INFO:teuthology.orchestra.run.smithi042.stdout:[client.0] 2024-08-30T08:59:06.558 INFO:teuthology.orchestra.run.smithi042.stdout: key = AQDaidFmLf8XIRAAoGwfW0c6M+u79n1oNfvauA== 2024-08-30T08:59:06.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:06.559+0000 7f67c3354640 1 -- 172.21.15.42:0/3222538518 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f67a80b4bc0 msgr2=0x7f67a80b70d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:06.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:06.559+0000 7f67c3354640 1 --2- 172.21.15.42:0/3222538518 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f67a80b4bc0 0x7f67a80b70d0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f67ac005fd0 tx=0x7f67ac00b390 comp rx=0 tx=0).stop 2024-08-30T08:59:06.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:06.559+0000 7f67c3354640 1 -- 172.21.15.42:0/3222538518 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f67bc06c1b0 msgr2=0x7f67bc19bed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:06.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:06.559+0000 7f67c3354640 1 --2- 172.21.15.42:0/3222538518 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f67bc06c1b0 0x7f67bc19bed0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f67b800e9f0 tx=0x7f67b800eeb0 comp rx=0 tx=0).stop 2024-08-30T08:59:06.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:06.559+0000 7f67c3354640 1 -- 172.21.15.42:0/3222538518 shutdown_connections 2024-08-30T08:59:06.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:06.560+0000 7f67c3354640 1 --2- 172.21.15.42:0/3222538518 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f67a80b4bc0 0x7f67a80b70d0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:06.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:06.560+0000 7f67c3354640 1 --2- 172.21.15.42:0/3222538518 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f67bc10b470 0x7f67bc19d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:06.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:06.560+0000 7f67c3354640 1 --2- 172.21.15.42:0/3222538518 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f67bc108ac0 0x7f67bc19c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:06.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:06.560+0000 7f67c3354640 1 --2- 172.21.15.42:0/3222538518 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f67bc06c1b0 0x7f67bc19bed0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:06.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:06.560+0000 7f67c3354640 1 -- 172.21.15.42:0/3222538518 >> 172.21.15.42:0/3222538518 conn(0x7f67bc0fddc0 msgr2=0x7f67bc109890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:59:06.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:06.560+0000 7f67c3354640 1 -- 172.21.15.42:0/3222538518 shutdown_connections 2024-08-30T08:59:06.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:06.560+0000 7f67c3354640 1 -- 172.21.15.42:0/3222538518 wait complete. 2024-08-30T08:59:06.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:59:06 smithi167 ceph-mon[22063]: from='client.15009 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-30T08:59:06.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:59:06 smithi167 ceph-mon[22063]: pgmap v321: 129 pgs: 129 active+clean; 634 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 3.3 KiB/s wr, 7 op/s 2024-08-30T08:59:06.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:59:06 smithi190 ceph-mon[22001]: from='client.15009 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-30T08:59:06.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:59:06 smithi190 ceph-mon[22001]: pgmap v321: 129 pgs: 129 active+clean; 634 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 3.3 KiB/s wr, 7 op/s 2024-08-30T08:59:07.310 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-08-30T08:59:07.310 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-08-30T08:59:07.310 DEBUG:teuthology.orchestra.run.smithi042:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-08-30T08:59:07.348 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph config log 1 --format=json 2024-08-30T08:59:07.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:59:07 smithi167 ceph-mon[22063]: from='client.? 172.21.15.42:0/3222538518' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-08-30T08:59:07.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:59:07 smithi167 ceph-mon[22063]: from='client.? 172.21.15.42:0/3222538518' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-08-30T08:59:07.632 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:59:07 smithi042 ceph-mon[18652]: from='client.? 172.21.15.42:0/3222538518' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-08-30T08:59:07.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:59:07 smithi042 ceph-mon[18652]: from='client.? 172.21.15.42:0/3222538518' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-08-30T08:59:07.640 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T08:59:07.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:59:07 smithi190 ceph-mon[22001]: from='client.? 172.21.15.42:0/3222538518' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-08-30T08:59:07.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:59:07 smithi190 ceph-mon[22001]: from='client.? 172.21.15.42:0/3222538518' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-08-30T08:59:08.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:08.084+0000 7f7980cda640 1 -- 172.21.15.42:0/2561087251 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f797c108ac0 msgr2=0x7f797c10f2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:08.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:08.084+0000 7f7980cda640 1 --2- 172.21.15.42:0/2561087251 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f797c108ac0 0x7f797c10f2f0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f7974009ca0 tx=0x7f797402a8c0 comp rx=0 tx=0).stop 2024-08-30T08:59:08.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:08.084+0000 7f7980cda640 1 -- 172.21.15.42:0/2561087251 shutdown_connections 2024-08-30T08:59:08.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:08.084+0000 7f7980cda640 1 --2- 172.21.15.42:0/2561087251 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f797c108170 0x7f797c108580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:08.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:08.084+0000 7f7980cda640 1 --2- 172.21.15.42:0/2561087251 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f797c108ac0 0x7f797c10f2f0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:08.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:08.084+0000 7f7980cda640 1 --2- 172.21.15.42:0/2561087251 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f797c102880 0x7f797c102c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:08.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:08.084+0000 7f7980cda640 1 -- 172.21.15.42:0/2561087251 >> 172.21.15.42:0/2561087251 conn(0x7f797c079500 msgr2=0x7f797c079910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:59:08.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:08.084+0000 7f7980cda640 1 -- 172.21.15.42:0/2561087251 shutdown_connections 2024-08-30T08:59:08.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:08.084+0000 7f7980cda640 1 -- 172.21.15.42:0/2561087251 wait complete. 2024-08-30T08:59:08.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:08.085+0000 7f7980cda640 1 Processor -- start 2024-08-30T08:59:08.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:08.085+0000 7f7980cda640 1 -- start start 2024-08-30T08:59:08.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:08.085+0000 7f7980cda640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f797c102880 0x7f797c19aff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:08.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:08.085+0000 7f7980cda640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f797c108170 0x7f797c19b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:08.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:08.086+0000 7f7980cda640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f797c108ac0 0x7f797c19ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:08.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:08.086+0000 7f7980cda640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f797c112740 con 0x7f797c108ac0 2024-08-30T08:59:08.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:08.086+0000 7f7980cda640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f797c1125c0 con 0x7f797c108170 2024-08-30T08:59:08.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:08.086+0000 7f7980cda640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f797c1128c0 con 0x7f797c102880 2024-08-30T08:59:08.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:08.086+0000 7f797ad9d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f797c108ac0 0x7f797c19ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:08.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:08.086+0000 7f797ad9d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f797c108ac0 0x7f797c19ba70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52590/0 (socket says 172.21.15.42:52590) 2024-08-30T08:59:08.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:08.086+0000 7f797ad9d640 1 -- 172.21.15.42:0/4091801849 learned_addr learned my addr 172.21.15.42:0/4091801849 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T08:59:08.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:08.086+0000 7f797a59c640 1 --2- 172.21.15.42:0/4091801849 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f797c102880 0x7f797c19aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:08.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:08.086+0000 7f7973fff640 1 --2- 172.21.15.42:0/4091801849 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f797c108170 0x7f797c19b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:08.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:08.086+0000 7f797ad9d640 1 -- 172.21.15.42:0/4091801849 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f797c102880 msgr2=0x7f797c19aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:08.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:08.086+0000 7f797ad9d640 1 --2- 172.21.15.42:0/4091801849 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f797c102880 0x7f797c19aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:08.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:08.086+0000 7f797ad9d640 1 -- 172.21.15.42:0/4091801849 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f797c108170 msgr2=0x7f797c19b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:08.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:08.086+0000 7f797ad9d640 1 --2- 172.21.15.42:0/4091801849 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f797c108170 0x7f797c19b530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:08.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:08.086+0000 7f797ad9d640 1 -- 172.21.15.42:0/4091801849 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f797c1a6280 con 0x7f797c108ac0 2024-08-30T08:59:08.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:08.086+0000 7f7973fff640 1 --2- 172.21.15.42:0/4091801849 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f797c108170 0x7f797c19b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-30T08:59:08.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:08.086+0000 7f797ad9d640 1 --2- 172.21.15.42:0/4091801849 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f797c108ac0 0x7f797c19ba70 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f7974005cc0 tx=0x7f797402f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:59:08.114 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:08.086+0000 7f79737fe640 1 -- 172.21.15.42:0/4091801849 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7974003bf0 con 0x7f797c108ac0 2024-08-30T08:59:08.114 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:08.086+0000 7f79737fe640 1 -- 172.21.15.42:0/4091801849 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7974003d90 con 0x7f797c108ac0 2024-08-30T08:59:08.114 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:08.086+0000 7f7980cda640 1 -- 172.21.15.42:0/4091801849 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f797c1a6510 con 0x7f797c108ac0 2024-08-30T08:59:08.114 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:08.086+0000 7f79737fe640 1 -- 172.21.15.42:0/4091801849 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f797402fbc0 con 0x7f797c108ac0 2024-08-30T08:59:08.114 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:08.087+0000 7f7980cda640 1 -- 172.21.15.42:0/4091801849 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f797c1a6a50 con 0x7f797c108ac0 2024-08-30T08:59:08.114 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:08.088+0000 7f7980cda640 1 -- 172.21.15.42:0/4091801849 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f797c103ab0 con 0x7f797c108ac0 2024-08-30T08:59:08.114 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:08.089+0000 7f79737fe640 1 -- 172.21.15.42:0/4091801849 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) ==== 152628+0+0 (secure 0 0 0) 0x7f797402fd60 con 0x7f797c108ac0 2024-08-30T08:59:08.114 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:08.090+0000 7f79737fe640 1 --2- 172.21.15.42:0/4091801849 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f79680b4bf0 0x7f79680b7100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:08.114 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:08.090+0000 7f797a59c640 1 --2- 172.21.15.42:0/4091801849 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f79680b4bf0 0x7f79680b7100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:08.114 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:08.090+0000 7f79737fe640 1 -- 172.21.15.42:0/4091801849 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 9092+0+0 (secure 0 0 0) 0x7f7974106bf0 con 0x7f797c108ac0 2024-08-30T08:59:08.114 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:08.090+0000 7f797a59c640 1 --2- 172.21.15.42:0/4091801849 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f79680b4bf0 0x7f79680b7100 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f797c06c440 tx=0x7f796c00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:59:08.114 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:08.093+0000 7f79737fe640 1 -- 172.21.15.42:0/4091801849 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f79740cd890 con 0x7f797c108ac0 2024-08-30T08:59:08.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:08.281+0000 7f7980cda640 1 -- 172.21.15.42:0/4091801849 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "config log", "num": 1, "format": "json"} v 0) -- 0x7f797c10a910 con 0x7f797c108ac0 2024-08-30T08:59:08.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:08.282+0000 7f79737fe640 1 -- 172.21.15.42:0/4091801849 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "config log", "num": 1, "format": "json"}]=0 v49) ==== 86+0+133 (secure 0 0 0) 0x7f79740098b0 con 0x7f797c108ac0 2024-08-30T08:59:08.284 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-08-30T08:59:08.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:08.284+0000 7f7980cda640 1 -- 172.21.15.42:0/4091801849 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f79680b4bf0 msgr2=0x7f79680b7100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:08.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:08.284+0000 7f7980cda640 1 --2- 172.21.15.42:0/4091801849 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f79680b4bf0 0x7f79680b7100 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f797c06c440 tx=0x7f796c00b390 comp rx=0 tx=0).stop 2024-08-30T08:59:08.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:08.284+0000 7f7980cda640 1 -- 172.21.15.42:0/4091801849 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f797c108ac0 msgr2=0x7f797c19ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:08.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:08.284+0000 7f7980cda640 1 --2- 172.21.15.42:0/4091801849 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f797c108ac0 0x7f797c19ba70 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f7974005cc0 tx=0x7f797402f380 comp rx=0 tx=0).stop 2024-08-30T08:59:08.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:08.284+0000 7f7980cda640 1 -- 172.21.15.42:0/4091801849 shutdown_connections 2024-08-30T08:59:08.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:08.284+0000 7f7980cda640 1 --2- 172.21.15.42:0/4091801849 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f79680b4bf0 0x7f79680b7100 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:08.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:08.284+0000 7f7980cda640 1 --2- 172.21.15.42:0/4091801849 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f797c108ac0 0x7f797c19ba70 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:08.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:08.284+0000 7f7980cda640 1 --2- 172.21.15.42:0/4091801849 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f797c108170 0x7f797c19b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:08.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:08.284+0000 7f7980cda640 1 --2- 172.21.15.42:0/4091801849 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f797c102880 0x7f797c19aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:08.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:08.284+0000 7f7980cda640 1 -- 172.21.15.42:0/4091801849 >> 172.21.15.42:0/4091801849 conn(0x7f797c079500 msgr2=0x7f797c104fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:59:08.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:08.284+0000 7f7980cda640 1 -- 172.21.15.42:0/4091801849 shutdown_connections 2024-08-30T08:59:08.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:08.284+0000 7f7980cda640 1 -- 172.21.15.42:0/4091801849 wait complete. 2024-08-30T08:59:08.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:59:08 smithi167 ceph-mon[22063]: pgmap v322: 129 pgs: 129 active+clean; 634 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 3.0 KiB/s wr, 7 op/s 2024-08-30T08:59:08.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:59:08 smithi167 ceph-mon[22063]: from='client.? 172.21.15.42:0/4091801849' entity='client.admin' cmd={"prefix": "config log", "num": 1, "format": "json"} : dispatch 2024-08-30T08:59:08.882 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:59:08 smithi042 ceph-mon[18652]: pgmap v322: 129 pgs: 129 active+clean; 634 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 3.0 KiB/s wr, 7 op/s 2024-08-30T08:59:08.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:59:08 smithi042 ceph-mon[18652]: from='client.? 172.21.15.42:0/4091801849' entity='client.admin' cmd={"prefix": "config log", "num": 1, "format": "json"} : dispatch 2024-08-30T08:59:08.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:59:08 smithi190 ceph-mon[22001]: pgmap v322: 129 pgs: 129 active+clean; 634 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 3.0 KiB/s wr, 7 op/s 2024-08-30T08:59:08.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:59:08 smithi190 ceph-mon[22001]: from='client.? 172.21.15.42:0/4091801849' entity='client.admin' cmd={"prefix": "config log", "num": 1, "format": "json"} : dispatch 2024-08-30T08:59:08.995 INFO:teuthology.orchestra.run.smithi042.stdout:[{"version":49,"timestamp":"2024-08-30T08:57:52.758627+0000","name":"","changes":[{"name":"mds.all/mds_join_fs","new_value":"all"}]}] 2024-08-30T08:59:08.995 INFO:tasks.ceph_manager:config epoch is 49 2024-08-30T08:59:08.995 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-08-30T08:59:08.995 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-08-30T08:59:08.995 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph mgr dump --format=json 2024-08-30T08:59:09.249 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T08:59:09.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:09.749+0000 7fc0c98c4640 1 -- 172.21.15.42:0/2909859120 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc0c4102800 msgr2=0x7fc0c4102c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:09.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:09.749+0000 7fc0c98c4640 1 --2- 172.21.15.42:0/2909859120 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc0c4102800 0x7fc0c4102c10 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fc0ac009900 tx=0x7fc0ac02a410 comp rx=0 tx=0).stop 2024-08-30T08:59:09.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:09.749+0000 7fc0c98c4640 1 -- 172.21.15.42:0/2909859120 shutdown_connections 2024-08-30T08:59:09.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:09.749+0000 7fc0c98c4640 1 --2- 172.21.15.42:0/2909859120 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc0c4108a90 0x7fc0c410f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:09.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:09.749+0000 7fc0c98c4640 1 --2- 172.21.15.42:0/2909859120 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc0c4108140 0x7fc0c4108550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:09.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:09.749+0000 7fc0c98c4640 1 --2- 172.21.15.42:0/2909859120 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc0c4102800 0x7fc0c4102c10 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:09.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:09.749+0000 7fc0c98c4640 1 -- 172.21.15.42:0/2909859120 >> 172.21.15.42:0/2909859120 conn(0x7fc0c4079500 msgr2=0x7fc0c4079910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:59:09.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:09.749+0000 7fc0c98c4640 1 -- 172.21.15.42:0/2909859120 shutdown_connections 2024-08-30T08:59:09.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:09.749+0000 7fc0c98c4640 1 -- 172.21.15.42:0/2909859120 wait complete. 2024-08-30T08:59:09.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:09.750+0000 7fc0c98c4640 1 Processor -- start 2024-08-30T08:59:09.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:09.750+0000 7fc0c98c4640 1 -- start start 2024-08-30T08:59:09.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:09.750+0000 7fc0c98c4640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc0c4102800 0x7fc0c419b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:09.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:09.750+0000 7fc0c98c4640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc0c4108140 0x7fc0c419b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:09.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:09.750+0000 7fc0c98c4640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc0c4108a90 0x7fc0c419bb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:09.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:09.750+0000 7fc0c98c4640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fc0c41127b0 con 0x7fc0c4108140 2024-08-30T08:59:09.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:09.750+0000 7fc0c98c4640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc0c4112630 con 0x7fc0c4108a90 2024-08-30T08:59:09.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:09.750+0000 7fc0c98c4640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fc0c4112930 con 0x7fc0c4102800 2024-08-30T08:59:09.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:09.750+0000 7fc0c27fc640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc0c4108140 0x7fc0c419b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:09.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:09.750+0000 7fc0c27fc640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc0c4108140 0x7fc0c419b5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52604/0 (socket says 172.21.15.42:52604) 2024-08-30T08:59:09.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:09.750+0000 7fc0c27fc640 1 -- 172.21.15.42:0/3908572068 learned_addr learned my addr 172.21.15.42:0/3908572068 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T08:59:09.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:09.750+0000 7fc0c2ffd640 1 --2- 172.21.15.42:0/3908572068 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc0c4102800 0x7fc0c419b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:09.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:09.750+0000 7fc0c37fe640 1 --2- 172.21.15.42:0/3908572068 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc0c4108a90 0x7fc0c419bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:09.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:09.751+0000 7fc0c27fc640 1 -- 172.21.15.42:0/3908572068 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc0c4102800 msgr2=0x7fc0c419b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:09.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:09.751+0000 7fc0c27fc640 1 --2- 172.21.15.42:0/3908572068 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc0c4102800 0x7fc0c419b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:09.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:09.751+0000 7fc0c27fc640 1 -- 172.21.15.42:0/3908572068 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc0c4108a90 msgr2=0x7fc0c419bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:09.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:09.751+0000 7fc0c27fc640 1 --2- 172.21.15.42:0/3908572068 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc0c4108a90 0x7fc0c419bb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:09.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:09.751+0000 7fc0c27fc640 1 -- 172.21.15.42:0/3908572068 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0c41a62b0 con 0x7fc0c4108140 2024-08-30T08:59:09.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:09.751+0000 7fc0c27fc640 1 --2- 172.21.15.42:0/3908572068 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc0c4108140 0x7fc0c419b5c0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fc0b400c8c0 tx=0x7fc0b400cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:59:09.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:09.751+0000 7fc0c88c2640 1 -- 172.21.15.42:0/3908572068 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0b4007bb0 con 0x7fc0c4108140 2024-08-30T08:59:09.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:09.751+0000 7fc0c88c2640 1 -- 172.21.15.42:0/3908572068 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc0b4007d50 con 0x7fc0c4108140 2024-08-30T08:59:09.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:09.751+0000 7fc0c98c4640 1 -- 172.21.15.42:0/3908572068 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0c41a65a0 con 0x7fc0c4108140 2024-08-30T08:59:09.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:09.751+0000 7fc0c88c2640 1 -- 172.21.15.42:0/3908572068 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0b40187c0 con 0x7fc0c4108140 2024-08-30T08:59:09.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:09.751+0000 7fc0c98c4640 1 -- 172.21.15.42:0/3908572068 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0c41a6b60 con 0x7fc0c4108140 2024-08-30T08:59:09.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:09.752+0000 7fc0c98c4640 1 -- 172.21.15.42:0/3908572068 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0c4103a80 con 0x7fc0c4108140 2024-08-30T08:59:09.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:09.754+0000 7fc0c88c2640 1 -- 172.21.15.42:0/3908572068 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) ==== 152628+0+0 (secure 0 0 0) 0x7fc0b400fa20 con 0x7fc0c4108140 2024-08-30T08:59:09.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:09.754+0000 7fc0c88c2640 1 --2- 172.21.15.42:0/3908572068 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fc0b00b4bc0 0x7fc0b00b70d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:09.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:09.755+0000 7fc0c2ffd640 1 --2- 172.21.15.42:0/3908572068 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fc0b00b4bc0 0x7fc0b00b70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:09.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:09.755+0000 7fc0c88c2640 1 -- 172.21.15.42:0/3908572068 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 9092+0+0 (secure 0 0 0) 0x7fc0b40e5aa0 con 0x7fc0c4108140 2024-08-30T08:59:09.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:09.755+0000 7fc0c2ffd640 1 --2- 172.21.15.42:0/3908572068 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fc0b00b4bc0 0x7fc0b00b70d0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fc0ac000c00 tx=0x7fc0ac031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:59:09.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:09.757+0000 7fc0c88c2640 1 -- 172.21.15.42:0/3908572068 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7fc0b4014030 con 0x7fc0c4108140 2024-08-30T08:59:09.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:09.991+0000 7fc0c98c4640 1 -- 172.21.15.42:0/3908572068 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7fc0c406bd70 con 0x7fc0c4108140 2024-08-30T08:59:09.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:09.996+0000 7fc0c88c2640 1 -- 172.21.15.42:0/3908572068 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+293191 (secure 0 0 0) 0x7fc0c406bd70 con 0x7fc0c4108140 2024-08-30T08:59:10.002 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-08-30T08:59:10.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:09.999+0000 7fc0c98c4640 1 -- 172.21.15.42:0/3908572068 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fc0b00b4bc0 msgr2=0x7fc0b00b70d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:10.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:09.999+0000 7fc0c98c4640 1 --2- 172.21.15.42:0/3908572068 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fc0b00b4bc0 0x7fc0b00b70d0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fc0ac000c00 tx=0x7fc0ac031400 comp rx=0 tx=0).stop 2024-08-30T08:59:10.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:09.999+0000 7fc0c98c4640 1 -- 172.21.15.42:0/3908572068 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc0c4108140 msgr2=0x7fc0c419b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:10.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:09.999+0000 7fc0c98c4640 1 --2- 172.21.15.42:0/3908572068 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc0c4108140 0x7fc0c419b5c0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fc0b400c8c0 tx=0x7fc0b400cd80 comp rx=0 tx=0).stop 2024-08-30T08:59:10.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:09.999+0000 7fc0c98c4640 1 -- 172.21.15.42:0/3908572068 shutdown_connections 2024-08-30T08:59:10.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:09.999+0000 7fc0c98c4640 1 --2- 172.21.15.42:0/3908572068 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fc0b00b4bc0 0x7fc0b00b70d0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:10.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:09.999+0000 7fc0c98c4640 1 --2- 172.21.15.42:0/3908572068 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc0c4108a90 0x7fc0c419bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:10.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:09.999+0000 7fc0c98c4640 1 --2- 172.21.15.42:0/3908572068 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc0c4108140 0x7fc0c419b5c0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:10.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:09.999+0000 7fc0c98c4640 1 --2- 172.21.15.42:0/3908572068 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc0c4102800 0x7fc0c419b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:10.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:09.999+0000 7fc0c98c4640 1 -- 172.21.15.42:0/3908572068 >> 172.21.15.42:0/3908572068 conn(0x7fc0c4079500 msgr2=0x7fc0c41061f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:59:10.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:09.999+0000 7fc0c98c4640 1 -- 172.21.15.42:0/3908572068 shutdown_connections 2024-08-30T08:59:10.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:09.999+0000 7fc0c98c4640 1 -- 172.21.15.42:0/3908572068 wait complete. 2024-08-30T08:59:10.668 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:59:10 smithi042 ceph-mon[18652]: pgmap v323: 129 pgs: 129 active+clean; 638 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 3.3 KiB/s wr, 9 op/s 2024-08-30T08:59:10.668 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:59:10 smithi042 ceph-mon[18652]: from='client.? 172.21.15.42:0/3908572068' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json"} : dispatch 2024-08-30T08:59:10.696 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":17,"flags":0,"active_gid":14150,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":3005140902},{"type":"v1","addr":"172.21.15.42:6801","nonce":3005140902}]},"active_addr":"172.21.15.42:6801/3005140902","active_change":"2024-08-30T08:50:44.821228+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24121,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:1.26.1","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24134,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:1.26.1","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:1.26.1","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.42:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.42:0","nonce":3049932087}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.42:0","nonce":569652528}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.42:0","nonce":1784618714}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.42:0","nonce":1749217175}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.42:0","nonce":3938864986}]}]} 2024-08-30T08:59:10.703 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-08-30T08:59:10.703 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-08-30T08:59:10.703 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph osd dump --format=json 2024-08-30T08:59:10.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:59:10 smithi167 ceph-mon[22063]: pgmap v323: 129 pgs: 129 active+clean; 638 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 3.3 KiB/s wr, 9 op/s 2024-08-30T08:59:10.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:59:10 smithi167 ceph-mon[22063]: from='client.? 172.21.15.42:0/3908572068' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json"} : dispatch 2024-08-30T08:59:10.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:59:10 smithi190 ceph-mon[22001]: pgmap v323: 129 pgs: 129 active+clean; 638 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 3.3 KiB/s wr, 9 op/s 2024-08-30T08:59:10.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:59:10 smithi190 ceph-mon[22001]: from='client.? 172.21.15.42:0/3908572068' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json"} : dispatch 2024-08-30T08:59:10.959 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T08:59:11.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:11.480+0000 7f5315f50640 1 -- 172.21.15.42:0/1908776506 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5310108140 msgr2=0x7f5310108550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:11.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:11.480+0000 7f5315f50640 1 --2- 172.21.15.42:0/1908776506 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5310108140 0x7f5310108550 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f52f8009900 tx=0x7f52f802a410 comp rx=0 tx=0).stop 2024-08-30T08:59:11.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:11.480+0000 7f5315f50640 1 -- 172.21.15.42:0/1908776506 shutdown_connections 2024-08-30T08:59:11.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:11.480+0000 7f5315f50640 1 --2- 172.21.15.42:0/1908776506 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5310108a90 0x7f531010f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:11.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:11.480+0000 7f5315f50640 1 --2- 172.21.15.42:0/1908776506 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5310108140 0x7f5310108550 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:11.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:11.480+0000 7f5315f50640 1 --2- 172.21.15.42:0/1908776506 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5310102800 0x7f5310102c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:11.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:11.480+0000 7f5315f50640 1 -- 172.21.15.42:0/1908776506 >> 172.21.15.42:0/1908776506 conn(0x7f53100794d0 msgr2=0x7f53100798e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:59:11.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:11.480+0000 7f5315f50640 1 -- 172.21.15.42:0/1908776506 shutdown_connections 2024-08-30T08:59:11.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:11.480+0000 7f5315f50640 1 -- 172.21.15.42:0/1908776506 wait complete. 2024-08-30T08:59:11.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:11.481+0000 7f5315f50640 1 Processor -- start 2024-08-30T08:59:11.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:11.481+0000 7f5315f50640 1 -- start start 2024-08-30T08:59:11.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:11.481+0000 7f5315f50640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5310102800 0x7f5310198ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:11.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:11.481+0000 7f5315f50640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5310108140 0x7f53101993e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:11.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:11.481+0000 7f5315f50640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5310108a90 0x7f5310199920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:11.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:11.481+0000 7f5315f50640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f5310112740 con 0x7f5310102800 2024-08-30T08:59:11.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:11.481+0000 7f5315f50640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f53101125c0 con 0x7f5310108a90 2024-08-30T08:59:11.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:11.481+0000 7f5315f50640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f53101128c0 con 0x7f5310108140 2024-08-30T08:59:11.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:11.481+0000 7f530f7fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5310102800 0x7f5310198ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:11.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:11.481+0000 7f530f7fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5310102800 0x7f5310198ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52608/0 (socket says 172.21.15.42:52608) 2024-08-30T08:59:11.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:11.481+0000 7f530f7fe640 1 -- 172.21.15.42:0/651179611 learned_addr learned my addr 172.21.15.42:0/651179611 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T08:59:11.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:11.482+0000 7f5306ffd640 1 --2- 172.21.15.42:0/651179611 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5310108140 0x7f53101993e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:11.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:11.482+0000 7f530ffff640 1 --2- 172.21.15.42:0/651179611 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5310108a90 0x7f5310199920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:11.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:11.482+0000 7f530f7fe640 1 -- 172.21.15.42:0/651179611 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5310108140 msgr2=0x7f53101993e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:11.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:11.482+0000 7f530f7fe640 1 --2- 172.21.15.42:0/651179611 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5310108140 0x7f53101993e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:11.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:11.482+0000 7f530f7fe640 1 -- 172.21.15.42:0/651179611 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5310108a90 msgr2=0x7f5310199920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:11.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:11.482+0000 7f530f7fe640 1 --2- 172.21.15.42:0/651179611 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5310108a90 0x7f5310199920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:11.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:11.482+0000 7f530f7fe640 1 -- 172.21.15.42:0/651179611 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53101a8550 con 0x7f5310102800 2024-08-30T08:59:11.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:11.482+0000 7f530ffff640 1 --2- 172.21.15.42:0/651179611 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5310108a90 0x7f5310199920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-30T08:59:11.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:11.482+0000 7f530f7fe640 1 --2- 172.21.15.42:0/651179611 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5310102800 0x7f5310198ea0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f530000b4a0 tx=0x7f530000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:59:11.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:11.482+0000 7f530d7fa640 1 -- 172.21.15.42:0/651179611 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5300014070 con 0x7f5310102800 2024-08-30T08:59:11.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:11.482+0000 7f5315f50640 1 -- 172.21.15.42:0/651179611 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53101a8840 con 0x7f5310102800 2024-08-30T08:59:11.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:11.482+0000 7f530d7fa640 1 -- 172.21.15.42:0/651179611 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5300010ac0 con 0x7f5310102800 2024-08-30T08:59:11.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:11.482+0000 7f530d7fa640 1 -- 172.21.15.42:0/651179611 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5300017c30 con 0x7f5310102800 2024-08-30T08:59:11.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:11.482+0000 7f5315f50640 1 -- 172.21.15.42:0/651179611 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53101a8d50 con 0x7f5310102800 2024-08-30T08:59:11.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:11.483+0000 7f5315f50640 1 -- 172.21.15.42:0/651179611 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5310102c10 con 0x7f5310102800 2024-08-30T08:59:11.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:11.486+0000 7f530d7fa640 1 -- 172.21.15.42:0/651179611 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) ==== 152628+0+0 (secure 0 0 0) 0x7f5300017070 con 0x7f5310102800 2024-08-30T08:59:11.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:11.487+0000 7f530d7fa640 1 --2- 172.21.15.42:0/651179611 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f52fc0b4bc0 0x7f52fc0b70d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:11.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:11.487+0000 7f5306ffd640 1 --2- 172.21.15.42:0/651179611 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f52fc0b4bc0 0x7f52fc0b70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:11.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:11.487+0000 7f530d7fa640 1 -- 172.21.15.42:0/651179611 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 9092+0+0 (secure 0 0 0) 0x7f53000e4cb0 con 0x7f5310102800 2024-08-30T08:59:11.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:11.487+0000 7f5306ffd640 1 --2- 172.21.15.42:0/651179611 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f52fc0b4bc0 0x7f52fc0b70d0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f52f8000c00 tx=0x7f52f8031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:59:11.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:11.490+0000 7f530d7fa640 1 -- 172.21.15.42:0/651179611 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f53000ab8c0 con 0x7f5310102800 2024-08-30T08:59:11.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:11.677+0000 7f5315f50640 1 -- 172.21.15.42:0/651179611 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f531010a8e0 con 0x7f5310102800 2024-08-30T08:59:11.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:11.678+0000 7f530d7fa640 1 -- 172.21.15.42:0/651179611 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+24772 (secure 0 0 0) 0x7f53000ab1c0 con 0x7f5310102800 2024-08-30T08:59:11.682 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-08-30T08:59:11.682 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":81,"fsid":"b0895c1a-66ac-11ef-bcd4-c7b262605968","created":"2024-08-30T08:49:44.858265+0000","modified":"2024-08-30T08:58:46.548905+0000","last_up_change":"2024-08-30T08:57:12.367791+0000","last_in_change":"2024-08-30T08:56:50.274678+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-30T08:53:27.297022+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-30T08:57:58.489964+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-30T08:58:01.064783+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-08-30T08:58:07.416534+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"029d62d7-a6c6-4905-8c76-6dffa9d57b92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":2617890537},{"type":"v1","addr":"172.21.15.42:6803","nonce":2617890537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":2617890537},{"type":"v1","addr":"172.21.15.42:6805","nonce":2617890537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2617890537},{"type":"v1","addr":"172.21.15.42:6809","nonce":2617890537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":2617890537},{"type":"v1","addr":"172.21.15.42:6807","nonce":2617890537}]},"public_addr":"172.21.15.42:6803/2617890537","cluster_addr":"172.21.15.42:6805/2617890537","heartbeat_back_addr":"172.21.15.42:6809/2617890537","heartbeat_front_addr":"172.21.15.42:6807/2617890537","state":["exists","up"]},{"osd":1,"uuid":"3a78384b-0207-41df-9384-22665007d5b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":2791960082},{"type":"v1","addr":"172.21.15.167:6801","nonce":2791960082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":2791960082},{"type":"v1","addr":"172.21.15.167:6803","nonce":2791960082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":2791960082},{"type":"v1","addr":"172.21.15.167:6807","nonce":2791960082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":2791960082},{"type":"v1","addr":"172.21.15.167:6805","nonce":2791960082}]},"public_addr":"172.21.15.167:6801/2791960082","cluster_addr":"172.21.15.167:6803/2791960082","heartbeat_back_addr":"172.21.15.167:6807/2791960082","heartbeat_front_addr":"172.21.15.167:6805/2791960082","state":["exists","up"]},{"osd":2,"uuid":"16001809-c27c-4f52-948a-9998261acb4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":1101326404},{"type":"v1","addr":"172.21.15.190:6801","nonce":1101326404}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":1101326404},{"type":"v1","addr":"172.21.15.190:6803","nonce":1101326404}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":1101326404},{"type":"v1","addr":"172.21.15.190:6807","nonce":1101326404}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":1101326404},{"type":"v1","addr":"172.21.15.190:6805","nonce":1101326404}]},"public_addr":"172.21.15.190:6801/1101326404","cluster_addr":"172.21.15.190:6803/1101326404","heartbeat_back_addr":"172.21.15.190:6807/1101326404","heartbeat_front_addr":"172.21.15.190:6805/1101326404","state":["exists","up"]},{"osd":3,"uuid":"87bb3e36-9e9e-4b19-9d8c-0cdf6e089d3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":1135599916},{"type":"v1","addr":"172.21.15.42:6811","nonce":1135599916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":1135599916},{"type":"v1","addr":"172.21.15.42:6813","nonce":1135599916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":1135599916},{"type":"v1","addr":"172.21.15.42:6817","nonce":1135599916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":1135599916},{"type":"v1","addr":"172.21.15.42:6815","nonce":1135599916}]},"public_addr":"172.21.15.42:6811/1135599916","cluster_addr":"172.21.15.42:6813/1135599916","heartbeat_back_addr":"172.21.15.42:6817/1135599916","heartbeat_front_addr":"172.21.15.42:6815/1135599916","state":["exists","up"]},{"osd":4,"uuid":"3ca1aae3-c972-4fb9-ac86-236279c30d64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":2927511419},{"type":"v1","addr":"172.21.15.167:6809","nonce":2927511419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":2927511419},{"type":"v1","addr":"172.21.15.167:6811","nonce":2927511419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":2927511419},{"type":"v1","addr":"172.21.15.167:6815","nonce":2927511419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":2927511419},{"type":"v1","addr":"172.21.15.167:6813","nonce":2927511419}]},"public_addr":"172.21.15.167:6809/2927511419","cluster_addr":"172.21.15.167:6811/2927511419","heartbeat_back_addr":"172.21.15.167:6815/2927511419","heartbeat_front_addr":"172.21.15.167:6813/2927511419","state":["exists","up"]},{"osd":5,"uuid":"5d07f54b-0d07-4efe-9bcf-390755ce0258","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":2557145333},{"type":"v1","addr":"172.21.15.190:6809","nonce":2557145333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":2557145333},{"type":"v1","addr":"172.21.15.190:6811","nonce":2557145333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":2557145333},{"type":"v1","addr":"172.21.15.190:6815","nonce":2557145333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":2557145333},{"type":"v1","addr":"172.21.15.190:6813","nonce":2557145333}]},"public_addr":"172.21.15.190:6809/2557145333","cluster_addr":"172.21.15.190:6811/2557145333","heartbeat_back_addr":"172.21.15.190:6815/2557145333","heartbeat_front_addr":"172.21.15.190:6813/2557145333","state":["exists","up"]},{"osd":6,"uuid":"96ad64af-44ff-4197-97d4-156f0cb88e81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":3213526100},{"type":"v1","addr":"172.21.15.42:6819","nonce":3213526100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":3213526100},{"type":"v1","addr":"172.21.15.42:6821","nonce":3213526100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":3213526100},{"type":"v1","addr":"172.21.15.42:6825","nonce":3213526100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":3213526100},{"type":"v1","addr":"172.21.15.42:6823","nonce":3213526100}]},"public_addr":"172.21.15.42:6819/3213526100","cluster_addr":"172.21.15.42:6821/3213526100","heartbeat_back_addr":"172.21.15.42:6825/3213526100","heartbeat_front_addr":"172.21.15.42:6823/3213526100","state":["exists","up"]},{"osd":7,"uuid":"337a2112-eaf7-4e6b-86be-4e40d6ddf835","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":2962799814},{"type":"v1","addr":"172.21.15.167:6817","nonce":2962799814}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":2962799814},{"type":"v1","addr":"172.21.15.167:6819","nonce":2962799814}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":2962799814},{"type":"v1","addr":"172.21.15.167:6823","nonce":2962799814}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":2962799814},{"type":"v1","addr":"172.21.15.167:6821","nonce":2962799814}]},"public_addr":"172.21.15.167:6817/2962799814","cluster_addr":"172.21.15.167:6819/2962799814","heartbeat_back_addr":"172.21.15.167:6823/2962799814","heartbeat_front_addr":"172.21.15.167:6821/2962799814","state":["exists","up"]},{"osd":8,"uuid":"219a7224-4331-464b-ac86-c51054d51d91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":3242198843},{"type":"v1","addr":"172.21.15.190:6817","nonce":3242198843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":3242198843},{"type":"v1","addr":"172.21.15.190:6819","nonce":3242198843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":3242198843},{"type":"v1","addr":"172.21.15.190:6823","nonce":3242198843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":3242198843},{"type":"v1","addr":"172.21.15.190:6821","nonce":3242198843}]},"public_addr":"172.21.15.190:6817/3242198843","cluster_addr":"172.21.15.190:6819/3242198843","heartbeat_back_addr":"172.21.15.190:6823/3242198843","heartbeat_front_addr":"172.21.15.190:6821/3242198843","state":["exists","up"]},{"osd":9,"uuid":"2f2f5177-a009-40d9-991c-76aec8a5cba6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":1123536325},{"type":"v1","addr":"172.21.15.42:6827","nonce":1123536325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":1123536325},{"type":"v1","addr":"172.21.15.42:6829","nonce":1123536325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":1123536325},{"type":"v1","addr":"172.21.15.42:6833","nonce":1123536325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":1123536325},{"type":"v1","addr":"172.21.15.42:6831","nonce":1123536325}]},"public_addr":"172.21.15.42:6827/1123536325","cluster_addr":"172.21.15.42:6829/1123536325","heartbeat_back_addr":"172.21.15.42:6833/1123536325","heartbeat_front_addr":"172.21.15.42:6831/1123536325","state":["exists","up"]},{"osd":10,"uuid":"96ef5058-2518-4862-a165-17666b33a3e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":3451198319},{"type":"v1","addr":"172.21.15.167:6825","nonce":3451198319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":3451198319},{"type":"v1","addr":"172.21.15.167:6827","nonce":3451198319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":3451198319},{"type":"v1","addr":"172.21.15.167:6831","nonce":3451198319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":3451198319},{"type":"v1","addr":"172.21.15.167:6829","nonce":3451198319}]},"public_addr":"172.21.15.167:6825/3451198319","cluster_addr":"172.21.15.167:6827/3451198319","heartbeat_back_addr":"172.21.15.167:6831/3451198319","heartbeat_front_addr":"172.21.15.167:6829/3451198319","state":["exists","up"]},{"osd":11,"uuid":"a4eb9bd3-3faa-4c81-ba6d-c2084b72c9ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":380064457},{"type":"v1","addr":"172.21.15.190:6825","nonce":380064457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":380064457},{"type":"v1","addr":"172.21.15.190:6827","nonce":380064457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":380064457},{"type":"v1","addr":"172.21.15.190:6831","nonce":380064457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":380064457},{"type":"v1","addr":"172.21.15.190:6829","nonce":380064457}]},"public_addr":"172.21.15.190:6825/380064457","cluster_addr":"172.21.15.190:6827/380064457","heartbeat_back_addr":"172.21.15.190:6831/380064457","heartbeat_front_addr":"172.21.15.190:6829/380064457","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-30T08:52:36.511448+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-30T08:53:00.027277+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-30T08:53:23.770245+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-30T08:53:49.322226+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-30T08:54:14.350234+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-30T08:54:39.491793+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-30T08:55:04.870596+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-30T08:55:30.314150+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-30T08:55:54.483843+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-30T08:56:20.586115+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-30T08:56:45.666783+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-30T08:57:10.718387+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"4.9","mappings":[{"from":5,"to":8}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.42:0/2691233260":"2024-08-31T08:50:44.821070+0000","172.21.15.42:0/1518053351":"2024-08-31T08:50:44.821070+0000","172.21.15.42:0/352001020":"2024-08-31T08:50:44.821070+0000","172.21.15.42:6801/1733768634":"2024-08-31T08:50:44.821070+0000","172.21.15.42:0/1000597888":"2024-08-31T08:50:18.561851+0000","172.21.15.42:6800/1733768634":"2024-08-31T08:50:44.821070+0000","172.21.15.42:6801/3538882081":"2024-08-31T08:50:18.561851+0000","172.21.15.42:6800/3538882081":"2024-08-31T08:50:18.561851+0000","172.21.15.42:0/145366508":"2024-08-31T08:50:18.561851+0000","172.21.15.42:0/3346355213":"2024-08-31T08:50:18.561851+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-30T08:59:11.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:11.681+0000 7f5315f50640 1 -- 172.21.15.42:0/651179611 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f52fc0b4bc0 msgr2=0x7f52fc0b70d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:11.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:11.681+0000 7f5315f50640 1 --2- 172.21.15.42:0/651179611 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f52fc0b4bc0 0x7f52fc0b70d0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f52f8000c00 tx=0x7f52f8031400 comp rx=0 tx=0).stop 2024-08-30T08:59:11.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:11.681+0000 7f5315f50640 1 -- 172.21.15.42:0/651179611 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5310102800 msgr2=0x7f5310198ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:11.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:11.681+0000 7f5315f50640 1 --2- 172.21.15.42:0/651179611 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5310102800 0x7f5310198ea0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f530000b4a0 tx=0x7f530000b960 comp rx=0 tx=0).stop 2024-08-30T08:59:11.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:11.681+0000 7f5315f50640 1 -- 172.21.15.42:0/651179611 shutdown_connections 2024-08-30T08:59:11.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:11.681+0000 7f5315f50640 1 --2- 172.21.15.42:0/651179611 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f52fc0b4bc0 0x7f52fc0b70d0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:11.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:11.681+0000 7f5315f50640 1 --2- 172.21.15.42:0/651179611 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5310108140 0x7f53101993e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:11.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:11.681+0000 7f5315f50640 1 --2- 172.21.15.42:0/651179611 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5310108a90 0x7f5310199920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:11.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:11.681+0000 7f5315f50640 1 --2- 172.21.15.42:0/651179611 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5310102800 0x7f5310198ea0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:11.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:11.681+0000 7f5315f50640 1 -- 172.21.15.42:0/651179611 >> 172.21.15.42:0/651179611 conn(0x7f53100794d0 msgr2=0x7f5310105050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:59:11.685 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:11.681+0000 7f5315f50640 1 -- 172.21.15.42:0/651179611 shutdown_connections 2024-08-30T08:59:11.685 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:11.681+0000 7f5315f50640 1 -- 172.21.15.42:0/651179611 wait complete. 2024-08-30T08:59:12.487 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-08-30T08:59:12.488 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph osd dump --format=json 2024-08-30T08:59:12.747 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T08:59:12.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:59:12 smithi167 ceph-mon[22063]: pgmap v324: 129 pgs: 129 active+clean; 638 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.5 KiB/s wr, 6 op/s 2024-08-30T08:59:12.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:59:12 smithi167 ceph-mon[22063]: from='client.? 172.21.15.42:0/651179611' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-30T08:59:12.866 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:59:12 smithi042 ceph-mon[18652]: pgmap v324: 129 pgs: 129 active+clean; 638 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.5 KiB/s wr, 6 op/s 2024-08-30T08:59:12.866 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:59:12 smithi042 ceph-mon[18652]: from='client.? 172.21.15.42:0/651179611' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-30T08:59:12.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:59:12 smithi190 ceph-mon[22001]: pgmap v324: 129 pgs: 129 active+clean; 638 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.5 KiB/s wr, 6 op/s 2024-08-30T08:59:12.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:59:12 smithi190 ceph-mon[22001]: from='client.? 172.21.15.42:0/651179611' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-30T08:59:13.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:13.221+0000 7fb4afea0640 1 -- 172.21.15.42:0/1511645634 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb4a8109260 msgr2=0x7fb4a8109670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:13.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:13.221+0000 7fb4afea0640 1 --2- 172.21.15.42:0/1511645634 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb4a8109260 0x7fb4a8109670 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fb498009900 tx=0x7fb49802a410 comp rx=0 tx=0).stop 2024-08-30T08:59:13.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:13.221+0000 7fb4afea0640 1 -- 172.21.15.42:0/1511645634 shutdown_connections 2024-08-30T08:59:13.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:13.221+0000 7fb4afea0640 1 --2- 172.21.15.42:0/1511645634 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb4a8109bb0 0x7fb4a81103e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:13.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:13.221+0000 7fb4afea0640 1 --2- 172.21.15.42:0/1511645634 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb4a8109260 0x7fb4a8109670 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:13.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:13.221+0000 7fb4afea0640 1 --2- 172.21.15.42:0/1511645634 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb4a8103920 0x7fb4a8103d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:13.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:13.221+0000 7fb4afea0640 1 -- 172.21.15.42:0/1511645634 >> 172.21.15.42:0/1511645634 conn(0x7fb4a80ff3b0 msgr2=0x7fb4a8101800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:59:13.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:13.221+0000 7fb4afea0640 1 -- 172.21.15.42:0/1511645634 shutdown_connections 2024-08-30T08:59:13.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:13.221+0000 7fb4afea0640 1 -- 172.21.15.42:0/1511645634 wait complete. 2024-08-30T08:59:13.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:13.222+0000 7fb4afea0640 1 Processor -- start 2024-08-30T08:59:13.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:13.222+0000 7fb4afea0640 1 -- start start 2024-08-30T08:59:13.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:13.222+0000 7fb4afea0640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb4a8103920 0x7fb4a819a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:13.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:13.222+0000 7fb4afea0640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb4a8109260 0x7fb4a819a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:13.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:13.222+0000 7fb4afea0640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb4a8109bb0 0x7fb4a819fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:13.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:13.222+0000 7fb4afea0640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fb4a8113870 con 0x7fb4a8103920 2024-08-30T08:59:13.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:13.222+0000 7fb4afea0640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb4a81136f0 con 0x7fb4a8109260 2024-08-30T08:59:13.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:13.222+0000 7fb4afea0640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fb4a81139f0 con 0x7fb4a8109bb0 2024-08-30T08:59:13.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:13.222+0000 7fb4adc3c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb4a8103920 0x7fb4a819a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:13.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:13.222+0000 7fb4adc3c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb4a8103920 0x7fb4a819a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52614/0 (socket says 172.21.15.42:52614) 2024-08-30T08:59:13.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:13.222+0000 7fb4adc3c640 1 -- 172.21.15.42:0/3843251288 learned_addr learned my addr 172.21.15.42:0/3843251288 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T08:59:13.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:13.222+0000 7fb4ad43b640 1 --2- 172.21.15.42:0/3843251288 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb4a8109260 0x7fb4a819a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:13.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:13.223+0000 7fb4ae43d640 1 --2- 172.21.15.42:0/3843251288 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb4a8109bb0 0x7fb4a819fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:13.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:13.223+0000 7fb4adc3c640 1 -- 172.21.15.42:0/3843251288 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb4a8109bb0 msgr2=0x7fb4a819fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:13.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:13.223+0000 7fb4adc3c640 1 --2- 172.21.15.42:0/3843251288 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb4a8109bb0 0x7fb4a819fb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:13.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:13.223+0000 7fb4adc3c640 1 -- 172.21.15.42:0/3843251288 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb4a8109260 msgr2=0x7fb4a819a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:13.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:13.223+0000 7fb4adc3c640 1 --2- 172.21.15.42:0/3843251288 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb4a8109260 0x7fb4a819a880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:13.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:13.223+0000 7fb4adc3c640 1 -- 172.21.15.42:0/3843251288 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4a81a0190 con 0x7fb4a8103920 2024-08-30T08:59:13.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:13.223+0000 7fb4adc3c640 1 --2- 172.21.15.42:0/3843251288 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb4a8103920 0x7fb4a819a2e0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fb4a400d950 tx=0x7fb4a400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:59:13.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:13.223+0000 7fb49effd640 1 -- 172.21.15.42:0/3843251288 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4a4017070 con 0x7fb4a8103920 2024-08-30T08:59:13.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:13.223+0000 7fb49effd640 1 -- 172.21.15.42:0/3843251288 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb4a4009c30 con 0x7fb4a8103920 2024-08-30T08:59:13.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:13.223+0000 7fb4afea0640 1 -- 172.21.15.42:0/3843251288 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4a81a0480 con 0x7fb4a8103920 2024-08-30T08:59:13.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:13.223+0000 7fb49effd640 1 -- 172.21.15.42:0/3843251288 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4a400f620 con 0x7fb4a8103920 2024-08-30T08:59:13.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:13.223+0000 7fb4afea0640 1 -- 172.21.15.42:0/3843251288 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4a81a6ca0 con 0x7fb4a8103920 2024-08-30T08:59:13.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:13.224+0000 7fb4afea0640 1 -- 172.21.15.42:0/3843251288 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4a8103d30 con 0x7fb4a8103920 2024-08-30T08:59:13.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:13.226+0000 7fb49effd640 1 -- 172.21.15.42:0/3843251288 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) ==== 152628+0+0 (secure 0 0 0) 0x7fb4a4005030 con 0x7fb4a8103920 2024-08-30T08:59:13.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:13.226+0000 7fb49effd640 1 --2- 172.21.15.42:0/3843251288 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fb4940b4c70 0x7fb4940b7180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:13.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:13.227+0000 7fb4ad43b640 1 --2- 172.21.15.42:0/3843251288 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fb4940b4c70 0x7fb4940b7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:13.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:13.227+0000 7fb49effd640 1 -- 172.21.15.42:0/3843251288 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 9092+0+0 (secure 0 0 0) 0x7fb4a40e51e0 con 0x7fb4a8103920 2024-08-30T08:59:13.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:13.227+0000 7fb4ad43b640 1 --2- 172.21.15.42:0/3843251288 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fb4940b4c70 0x7fb4940b7180 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fb498009230 tx=0x7fb498031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:59:13.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:13.230+0000 7fb49effd640 1 -- 172.21.15.42:0/3843251288 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7fb4a40abe70 con 0x7fb4a8103920 2024-08-30T08:59:13.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:13.416+0000 7fb4afea0640 1 -- 172.21.15.42:0/3843251288 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb4a810b3f0 con 0x7fb4a8103920 2024-08-30T08:59:13.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:13.417+0000 7fb49effd640 1 -- 172.21.15.42:0/3843251288 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+24772 (secure 0 0 0) 0x7fb4a40ab770 con 0x7fb4a8103920 2024-08-30T08:59:13.421 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-08-30T08:59:13.421 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":81,"fsid":"b0895c1a-66ac-11ef-bcd4-c7b262605968","created":"2024-08-30T08:49:44.858265+0000","modified":"2024-08-30T08:58:46.548905+0000","last_up_change":"2024-08-30T08:57:12.367791+0000","last_in_change":"2024-08-30T08:56:50.274678+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-30T08:53:27.297022+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-30T08:57:58.489964+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-30T08:58:01.064783+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-08-30T08:58:07.416534+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"029d62d7-a6c6-4905-8c76-6dffa9d57b92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":2617890537},{"type":"v1","addr":"172.21.15.42:6803","nonce":2617890537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":2617890537},{"type":"v1","addr":"172.21.15.42:6805","nonce":2617890537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2617890537},{"type":"v1","addr":"172.21.15.42:6809","nonce":2617890537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":2617890537},{"type":"v1","addr":"172.21.15.42:6807","nonce":2617890537}]},"public_addr":"172.21.15.42:6803/2617890537","cluster_addr":"172.21.15.42:6805/2617890537","heartbeat_back_addr":"172.21.15.42:6809/2617890537","heartbeat_front_addr":"172.21.15.42:6807/2617890537","state":["exists","up"]},{"osd":1,"uuid":"3a78384b-0207-41df-9384-22665007d5b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":2791960082},{"type":"v1","addr":"172.21.15.167:6801","nonce":2791960082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":2791960082},{"type":"v1","addr":"172.21.15.167:6803","nonce":2791960082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":2791960082},{"type":"v1","addr":"172.21.15.167:6807","nonce":2791960082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":2791960082},{"type":"v1","addr":"172.21.15.167:6805","nonce":2791960082}]},"public_addr":"172.21.15.167:6801/2791960082","cluster_addr":"172.21.15.167:6803/2791960082","heartbeat_back_addr":"172.21.15.167:6807/2791960082","heartbeat_front_addr":"172.21.15.167:6805/2791960082","state":["exists","up"]},{"osd":2,"uuid":"16001809-c27c-4f52-948a-9998261acb4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":1101326404},{"type":"v1","addr":"172.21.15.190:6801","nonce":1101326404}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":1101326404},{"type":"v1","addr":"172.21.15.190:6803","nonce":1101326404}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":1101326404},{"type":"v1","addr":"172.21.15.190:6807","nonce":1101326404}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":1101326404},{"type":"v1","addr":"172.21.15.190:6805","nonce":1101326404}]},"public_addr":"172.21.15.190:6801/1101326404","cluster_addr":"172.21.15.190:6803/1101326404","heartbeat_back_addr":"172.21.15.190:6807/1101326404","heartbeat_front_addr":"172.21.15.190:6805/1101326404","state":["exists","up"]},{"osd":3,"uuid":"87bb3e36-9e9e-4b19-9d8c-0cdf6e089d3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":1135599916},{"type":"v1","addr":"172.21.15.42:6811","nonce":1135599916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":1135599916},{"type":"v1","addr":"172.21.15.42:6813","nonce":1135599916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":1135599916},{"type":"v1","addr":"172.21.15.42:6817","nonce":1135599916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":1135599916},{"type":"v1","addr":"172.21.15.42:6815","nonce":1135599916}]},"public_addr":"172.21.15.42:6811/1135599916","cluster_addr":"172.21.15.42:6813/1135599916","heartbeat_back_addr":"172.21.15.42:6817/1135599916","heartbeat_front_addr":"172.21.15.42:6815/1135599916","state":["exists","up"]},{"osd":4,"uuid":"3ca1aae3-c972-4fb9-ac86-236279c30d64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":2927511419},{"type":"v1","addr":"172.21.15.167:6809","nonce":2927511419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":2927511419},{"type":"v1","addr":"172.21.15.167:6811","nonce":2927511419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":2927511419},{"type":"v1","addr":"172.21.15.167:6815","nonce":2927511419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":2927511419},{"type":"v1","addr":"172.21.15.167:6813","nonce":2927511419}]},"public_addr":"172.21.15.167:6809/2927511419","cluster_addr":"172.21.15.167:6811/2927511419","heartbeat_back_addr":"172.21.15.167:6815/2927511419","heartbeat_front_addr":"172.21.15.167:6813/2927511419","state":["exists","up"]},{"osd":5,"uuid":"5d07f54b-0d07-4efe-9bcf-390755ce0258","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":2557145333},{"type":"v1","addr":"172.21.15.190:6809","nonce":2557145333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":2557145333},{"type":"v1","addr":"172.21.15.190:6811","nonce":2557145333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":2557145333},{"type":"v1","addr":"172.21.15.190:6815","nonce":2557145333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":2557145333},{"type":"v1","addr":"172.21.15.190:6813","nonce":2557145333}]},"public_addr":"172.21.15.190:6809/2557145333","cluster_addr":"172.21.15.190:6811/2557145333","heartbeat_back_addr":"172.21.15.190:6815/2557145333","heartbeat_front_addr":"172.21.15.190:6813/2557145333","state":["exists","up"]},{"osd":6,"uuid":"96ad64af-44ff-4197-97d4-156f0cb88e81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":3213526100},{"type":"v1","addr":"172.21.15.42:6819","nonce":3213526100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":3213526100},{"type":"v1","addr":"172.21.15.42:6821","nonce":3213526100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":3213526100},{"type":"v1","addr":"172.21.15.42:6825","nonce":3213526100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":3213526100},{"type":"v1","addr":"172.21.15.42:6823","nonce":3213526100}]},"public_addr":"172.21.15.42:6819/3213526100","cluster_addr":"172.21.15.42:6821/3213526100","heartbeat_back_addr":"172.21.15.42:6825/3213526100","heartbeat_front_addr":"172.21.15.42:6823/3213526100","state":["exists","up"]},{"osd":7,"uuid":"337a2112-eaf7-4e6b-86be-4e40d6ddf835","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":2962799814},{"type":"v1","addr":"172.21.15.167:6817","nonce":2962799814}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":2962799814},{"type":"v1","addr":"172.21.15.167:6819","nonce":2962799814}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":2962799814},{"type":"v1","addr":"172.21.15.167:6823","nonce":2962799814}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":2962799814},{"type":"v1","addr":"172.21.15.167:6821","nonce":2962799814}]},"public_addr":"172.21.15.167:6817/2962799814","cluster_addr":"172.21.15.167:6819/2962799814","heartbeat_back_addr":"172.21.15.167:6823/2962799814","heartbeat_front_addr":"172.21.15.167:6821/2962799814","state":["exists","up"]},{"osd":8,"uuid":"219a7224-4331-464b-ac86-c51054d51d91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":3242198843},{"type":"v1","addr":"172.21.15.190:6817","nonce":3242198843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":3242198843},{"type":"v1","addr":"172.21.15.190:6819","nonce":3242198843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":3242198843},{"type":"v1","addr":"172.21.15.190:6823","nonce":3242198843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":3242198843},{"type":"v1","addr":"172.21.15.190:6821","nonce":3242198843}]},"public_addr":"172.21.15.190:6817/3242198843","cluster_addr":"172.21.15.190:6819/3242198843","heartbeat_back_addr":"172.21.15.190:6823/3242198843","heartbeat_front_addr":"172.21.15.190:6821/3242198843","state":["exists","up"]},{"osd":9,"uuid":"2f2f5177-a009-40d9-991c-76aec8a5cba6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":1123536325},{"type":"v1","addr":"172.21.15.42:6827","nonce":1123536325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":1123536325},{"type":"v1","addr":"172.21.15.42:6829","nonce":1123536325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":1123536325},{"type":"v1","addr":"172.21.15.42:6833","nonce":1123536325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":1123536325},{"type":"v1","addr":"172.21.15.42:6831","nonce":1123536325}]},"public_addr":"172.21.15.42:6827/1123536325","cluster_addr":"172.21.15.42:6829/1123536325","heartbeat_back_addr":"172.21.15.42:6833/1123536325","heartbeat_front_addr":"172.21.15.42:6831/1123536325","state":["exists","up"]},{"osd":10,"uuid":"96ef5058-2518-4862-a165-17666b33a3e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":3451198319},{"type":"v1","addr":"172.21.15.167:6825","nonce":3451198319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":3451198319},{"type":"v1","addr":"172.21.15.167:6827","nonce":3451198319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":3451198319},{"type":"v1","addr":"172.21.15.167:6831","nonce":3451198319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":3451198319},{"type":"v1","addr":"172.21.15.167:6829","nonce":3451198319}]},"public_addr":"172.21.15.167:6825/3451198319","cluster_addr":"172.21.15.167:6827/3451198319","heartbeat_back_addr":"172.21.15.167:6831/3451198319","heartbeat_front_addr":"172.21.15.167:6829/3451198319","state":["exists","up"]},{"osd":11,"uuid":"a4eb9bd3-3faa-4c81-ba6d-c2084b72c9ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":380064457},{"type":"v1","addr":"172.21.15.190:6825","nonce":380064457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":380064457},{"type":"v1","addr":"172.21.15.190:6827","nonce":380064457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":380064457},{"type":"v1","addr":"172.21.15.190:6831","nonce":380064457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":380064457},{"type":"v1","addr":"172.21.15.190:6829","nonce":380064457}]},"public_addr":"172.21.15.190:6825/380064457","cluster_addr":"172.21.15.190:6827/380064457","heartbeat_back_addr":"172.21.15.190:6831/380064457","heartbeat_front_addr":"172.21.15.190:6829/380064457","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-30T08:52:36.511448+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-30T08:53:00.027277+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-30T08:53:23.770245+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-30T08:53:49.322226+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-30T08:54:14.350234+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-30T08:54:39.491793+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-30T08:55:04.870596+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-30T08:55:30.314150+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-30T08:55:54.483843+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-30T08:56:20.586115+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-30T08:56:45.666783+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-30T08:57:10.718387+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"4.9","mappings":[{"from":5,"to":8}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.42:0/2691233260":"2024-08-31T08:50:44.821070+0000","172.21.15.42:0/1518053351":"2024-08-31T08:50:44.821070+0000","172.21.15.42:0/352001020":"2024-08-31T08:50:44.821070+0000","172.21.15.42:6801/1733768634":"2024-08-31T08:50:44.821070+0000","172.21.15.42:0/1000597888":"2024-08-31T08:50:18.561851+0000","172.21.15.42:6800/1733768634":"2024-08-31T08:50:44.821070+0000","172.21.15.42:6801/3538882081":"2024-08-31T08:50:18.561851+0000","172.21.15.42:6800/3538882081":"2024-08-31T08:50:18.561851+0000","172.21.15.42:0/145366508":"2024-08-31T08:50:18.561851+0000","172.21.15.42:0/3346355213":"2024-08-31T08:50:18.561851+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-30T08:59:13.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:13.420+0000 7fb4afea0640 1 -- 172.21.15.42:0/3843251288 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fb4940b4c70 msgr2=0x7fb4940b7180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:13.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:13.420+0000 7fb4afea0640 1 --2- 172.21.15.42:0/3843251288 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fb4940b4c70 0x7fb4940b7180 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fb498009230 tx=0x7fb498031400 comp rx=0 tx=0).stop 2024-08-30T08:59:13.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:13.420+0000 7fb4afea0640 1 -- 172.21.15.42:0/3843251288 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb4a8103920 msgr2=0x7fb4a819a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:13.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:13.420+0000 7fb4afea0640 1 --2- 172.21.15.42:0/3843251288 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb4a8103920 0x7fb4a819a2e0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fb4a400d950 tx=0x7fb4a400de10 comp rx=0 tx=0).stop 2024-08-30T08:59:13.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:13.420+0000 7fb4afea0640 1 -- 172.21.15.42:0/3843251288 shutdown_connections 2024-08-30T08:59:13.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:13.420+0000 7fb4afea0640 1 --2- 172.21.15.42:0/3843251288 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fb4940b4c70 0x7fb4940b7180 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:13.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:13.420+0000 7fb4afea0640 1 --2- 172.21.15.42:0/3843251288 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb4a8109bb0 0x7fb4a819fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:13.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:13.420+0000 7fb4afea0640 1 --2- 172.21.15.42:0/3843251288 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb4a8109260 0x7fb4a819a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:13.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:13.420+0000 7fb4afea0640 1 --2- 172.21.15.42:0/3843251288 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb4a8103920 0x7fb4a819a2e0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:13.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:13.420+0000 7fb4afea0640 1 -- 172.21.15.42:0/3843251288 >> 172.21.15.42:0/3843251288 conn(0x7fb4a80ff3b0 msgr2=0x7fb4a81017d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:59:13.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:13.420+0000 7fb4afea0640 1 -- 172.21.15.42:0/3843251288 shutdown_connections 2024-08-30T08:59:13.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:13.420+0000 7fb4afea0640 1 -- 172.21.15.42:0/3843251288 wait complete. 2024-08-30T08:59:13.796 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:59:13 smithi042 ceph-mon[18652]: from='client.? 172.21.15.42:0/3843251288' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-30T08:59:13.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:59:13 smithi167 ceph-mon[22063]: from='client.? 172.21.15.42:0/3843251288' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-30T08:59:13.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:59:13 smithi190 ceph-mon[22001]: from='client.? 172.21.15.42:0/3843251288' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-30T08:59:14.109 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-08-30T08:59:14.110 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-08-30T08:59:14.110 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-08-30T08:59:14.110 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-08-30T08:59:14.111 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-08-30T08:59:14.111 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-08-30T08:59:14.112 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-08-30T08:59:14.112 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-08-30T08:59:14.112 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-08-30T08:59:14.113 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-08-30T08:59:14.113 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-08-30T08:59:14.114 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-08-30T08:59:14.818 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T08:59:14.818 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T08:59:14.819 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T08:59:14.820 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T08:59:14.823 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T08:59:14.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:59:14 smithi167 ceph-mon[22063]: pgmap v325: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 2.2 KiB/s wr, 8 op/s 2024-08-30T08:59:14.865 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T08:59:14.866 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T08:59:14.866 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T08:59:14.881 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T08:59:14.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:59:14 smithi042 ceph-mon[18652]: pgmap v325: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 2.2 KiB/s wr, 8 op/s 2024-08-30T08:59:14.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:59:14 smithi190 ceph-mon[22001]: pgmap v325: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 2.2 KiB/s wr, 8 op/s 2024-08-30T08:59:15.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:15.738+0000 7fafe51bc640 1 -- 172.21.15.42:0/2344012008 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fafe0109b90 msgr2=0x7fafe01103c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:15.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:15.738+0000 7fafe51bc640 1 --2- 172.21.15.42:0/2344012008 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fafe0109b90 0x7fafe01103c0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fafd8009ca0 tx=0x7fafd802a8c0 comp rx=0 tx=0).stop 2024-08-30T08:59:15.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:15.738+0000 7fafe51bc640 1 -- 172.21.15.42:0/2344012008 shutdown_connections 2024-08-30T08:59:15.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:15.738+0000 7fafe51bc640 1 --2- 172.21.15.42:0/2344012008 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fafe0109240 0x7fafe0109650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:15.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:15.738+0000 7fafe51bc640 1 --2- 172.21.15.42:0/2344012008 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fafe0109b90 0x7fafe01103c0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:15.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:15.738+0000 7fafe51bc640 1 --2- 172.21.15.42:0/2344012008 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fafe0103900 0x7fafe0103d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:15.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:15.738+0000 7fafe51bc640 1 -- 172.21.15.42:0/2344012008 >> 172.21.15.42:0/2344012008 conn(0x7fafe007b5f0 msgr2=0x7fafe007ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:59:15.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:15.738+0000 7fafe51bc640 1 -- 172.21.15.42:0/2344012008 shutdown_connections 2024-08-30T08:59:15.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:15.738+0000 7fafe51bc640 1 -- 172.21.15.42:0/2344012008 wait complete. 2024-08-30T08:59:15.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:15.738+0000 7fafe51bc640 1 Processor -- start 2024-08-30T08:59:15.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:15.739+0000 7fafe51bc640 1 -- start start 2024-08-30T08:59:15.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:15.739+0000 7fafe51bc640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fafe0103900 0x7fafe00754b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:15.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:15.739+0000 7fafe51bc640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fafe0109240 0x7fafe00759f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:15.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:15.739+0000 7fafe51bc640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fafe0109b90 0x7fafe0075f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:15.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:15.739+0000 7fafe51bc640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fafe0113850 con 0x7fafe0109240 2024-08-30T08:59:15.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:15.739+0000 7fafe51bc640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fafe01136d0 con 0x7fafe0109b90 2024-08-30T08:59:15.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:15.739+0000 7fafe51bc640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fafe01139d0 con 0x7fafe0103900 2024-08-30T08:59:15.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:15.739+0000 7fafde59c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fafe0109240 0x7fafe00759f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:15.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:15.739+0000 7fafde59c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fafe0109240 0x7fafe00759f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52646/0 (socket says 172.21.15.42:52646) 2024-08-30T08:59:15.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:15.739+0000 7fafde59c640 1 -- 172.21.15.42:0/4075015642 learned_addr learned my addr 172.21.15.42:0/4075015642 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T08:59:15.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:15.739+0000 7fafdf59e640 1 --2- 172.21.15.42:0/4075015642 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fafe0109b90 0x7fafe0075f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:15.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:15.739+0000 7fafded9d640 1 --2- 172.21.15.42:0/4075015642 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fafe0103900 0x7fafe00754b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:15.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:15.740+0000 7fafde59c640 1 -- 172.21.15.42:0/4075015642 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fafe0103900 msgr2=0x7fafe00754b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:15.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:15.740+0000 7fafde59c640 1 --2- 172.21.15.42:0/4075015642 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fafe0103900 0x7fafe00754b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:15.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:15.740+0000 7fafde59c640 1 -- 172.21.15.42:0/4075015642 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fafe0109b90 msgr2=0x7fafe0075f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:15.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:15.740+0000 7fafde59c640 1 --2- 172.21.15.42:0/4075015642 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fafe0109b90 0x7fafe0075f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:15.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:15.740+0000 7fafde59c640 1 -- 172.21.15.42:0/4075015642 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafe01a8430 con 0x7fafe0109240 2024-08-30T08:59:15.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:15.740+0000 7fafded9d640 1 --2- 172.21.15.42:0/4075015642 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fafe0103900 0x7fafe00754b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-30T08:59:15.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:15.740+0000 7fafde59c640 1 --2- 172.21.15.42:0/4075015642 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fafe0109240 0x7fafe00759f0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fafd400c8c0 tx=0x7fafd400cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:59:15.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:15.740+0000 7fafc7fff640 1 -- 172.21.15.42:0/4075015642 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafd4007bb0 con 0x7fafe0109240 2024-08-30T08:59:15.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:15.740+0000 7fafe51bc640 1 -- 172.21.15.42:0/4075015642 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafe01a8720 con 0x7fafe0109240 2024-08-30T08:59:15.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:15.740+0000 7fafc7fff640 1 -- 172.21.15.42:0/4075015642 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fafd4007d50 con 0x7fafe0109240 2024-08-30T08:59:15.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:15.740+0000 7fafc7fff640 1 -- 172.21.15.42:0/4075015642 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafd40187c0 con 0x7fafe0109240 2024-08-30T08:59:15.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:15.740+0000 7fafe51bc640 1 -- 172.21.15.42:0/4075015642 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafe01a8cb0 con 0x7fafe0109240 2024-08-30T08:59:15.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:15.741+0000 7fafe51bc640 1 -- 172.21.15.42:0/4075015642 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fafe00638f0 con 0x7fafe0109240 2024-08-30T08:59:15.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:15.743+0000 7fafc7fff640 1 -- 172.21.15.42:0/4075015642 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) ==== 152628+0+0 (secure 0 0 0) 0x7fafd400fa20 con 0x7fafe0109240 2024-08-30T08:59:15.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:15.743+0000 7fafc7fff640 1 --2- 172.21.15.42:0/4075015642 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fafc80b4bc0 0x7fafc80b70d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:15.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:15.744+0000 7fafded9d640 1 --2- 172.21.15.42:0/4075015642 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fafc80b4bc0 0x7fafc80b70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:15.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:15.744+0000 7fafc7fff640 1 -- 172.21.15.42:0/4075015642 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 9092+0+0 (secure 0 0 0) 0x7fafd40e5d50 con 0x7fafe0109240 2024-08-30T08:59:15.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:15.744+0000 7fafc7fff640 1 --2- 172.21.15.42:0/4075015642 >> [v2:172.21.15.42:6802/2617890537,v1:172.21.15.42:6803/2617890537] conn(0x7fafc80bfcd0 0x7fafc80c21c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:15.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:15.744+0000 7fafc7fff640 1 -- 172.21.15.42:0/4075015642 --> [v2:172.21.15.42:6802/2617890537,v1:172.21.15.42:6803/2617890537] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fafd401ee50 con 0x7fafc80bfcd0 2024-08-30T08:59:15.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:15.744+0000 7fafc7fff640 1 -- 172.21.15.42:0/4075015642 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7fafd40e6100 con 0x7fafe0109240 2024-08-30T08:59:15.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:15.744+0000 7fafdf59e640 1 --2- 172.21.15.42:0/4075015642 >> [v2:172.21.15.42:6802/2617890537,v1:172.21.15.42:6803/2617890537] conn(0x7fafc80bfcd0 0x7fafc80c21c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:15.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:15.744+0000 7fafded9d640 1 --2- 172.21.15.42:0/4075015642 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fafc80b4bc0 0x7fafc80b70d0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fafe0076f00 tx=0x7fafd0005e70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:59:15.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:15.744+0000 7fafdf59e640 1 --2- 172.21.15.42:0/4075015642 >> [v2:172.21.15.42:6802/2617890537,v1:172.21.15.42:6803/2617890537] conn(0x7fafc80bfcd0 0x7fafc80c21c0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:59:15.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:15.745+0000 7fafc7fff640 1 -- 172.21.15.42:0/4075015642 <== osd.0 v2:172.21.15.42:6802/2617890537 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fafd401ee50 con 0x7fafc80bfcd0 2024-08-30T08:59:15.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:15.759+0000 7fafe51bc640 1 -- 172.21.15.42:0/4075015642 --> [v2:172.21.15.42:6802/2617890537,v1:172.21.15.42:6803/2617890537] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fafe0063cf0 con 0x7fafc80bfcd0 2024-08-30T08:59:15.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:15.759+0000 7fafc7fff640 1 -- 172.21.15.42:0/4075015642 <== osd.0 v2:172.21.15.42:6802/2617890537 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fafe0063cf0 con 0x7fafc80bfcd0 2024-08-30T08:59:15.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:15.760+0000 7fafe51bc640 1 -- 172.21.15.42:0/4075015642 >> [v2:172.21.15.42:6802/2617890537,v1:172.21.15.42:6803/2617890537] conn(0x7fafc80bfcd0 msgr2=0x7fafc80c21c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:15.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:15.760+0000 7fafe51bc640 1 --2- 172.21.15.42:0/4075015642 >> [v2:172.21.15.42:6802/2617890537,v1:172.21.15.42:6803/2617890537] conn(0x7fafc80bfcd0 0x7fafc80c21c0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:15.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:15.760+0000 7fafe51bc640 1 -- 172.21.15.42:0/4075015642 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fafc80b4bc0 msgr2=0x7fafc80b70d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:15.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:15.760+0000 7fafe51bc640 1 --2- 172.21.15.42:0/4075015642 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fafc80b4bc0 0x7fafc80b70d0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fafe0076f00 tx=0x7fafd0005e70 comp rx=0 tx=0).stop 2024-08-30T08:59:15.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:15.760+0000 7fafe51bc640 1 -- 172.21.15.42:0/4075015642 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fafe0109240 msgr2=0x7fafe00759f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:15.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:15.760+0000 7fafe51bc640 1 --2- 172.21.15.42:0/4075015642 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fafe0109240 0x7fafe00759f0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fafd400c8c0 tx=0x7fafd400cd80 comp rx=0 tx=0).stop 2024-08-30T08:59:15.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:15.760+0000 7fafdf59e640 1 -- 172.21.15.42:0/4075015642 reap_dead start 2024-08-30T08:59:15.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:15.760+0000 7fafe51bc640 1 -- 172.21.15.42:0/4075015642 shutdown_connections 2024-08-30T08:59:15.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:15.760+0000 7fafe51bc640 1 -- 172.21.15.42:0/4075015642 >> 172.21.15.42:0/4075015642 conn(0x7fafe007b5f0 msgr2=0x7fafe0105250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:59:15.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:15.760+0000 7fafe51bc640 1 -- 172.21.15.42:0/4075015642 shutdown_connections 2024-08-30T08:59:15.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:15.760+0000 7fafe51bc640 1 -- 172.21.15.42:0/4075015642 wait complete. 2024-08-30T08:59:16.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:16.354+0000 7fad1ff5f640 1 -- 172.21.15.42:0/1742585848 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fad18109240 msgr2=0x7fad18109650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:16.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:16.354+0000 7fad1ff5f640 1 --2- 172.21.15.42:0/1742585848 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fad18109240 0x7fad18109650 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fad08009900 tx=0x7fad0802a410 comp rx=0 tx=0).stop 2024-08-30T08:59:16.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:16.354+0000 7fad1ff5f640 1 -- 172.21.15.42:0/1742585848 shutdown_connections 2024-08-30T08:59:16.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:16.354+0000 7fad1ff5f640 1 --2- 172.21.15.42:0/1742585848 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fad18109b90 0x7fad181103c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:16.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:16.354+0000 7fad1ff5f640 1 --2- 172.21.15.42:0/1742585848 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fad18109240 0x7fad18109650 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:16.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:16.354+0000 7fad1ff5f640 1 --2- 172.21.15.42:0/1742585848 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fad18103900 0x7fad18103d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:16.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:16.354+0000 7fad1ff5f640 1 -- 172.21.15.42:0/1742585848 >> 172.21.15.42:0/1742585848 conn(0x7fad1807b5f0 msgr2=0x7fad1807ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:59:16.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:16.354+0000 7fad1ff5f640 1 -- 172.21.15.42:0/1742585848 shutdown_connections 2024-08-30T08:59:16.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:16.354+0000 7fad1ff5f640 1 -- 172.21.15.42:0/1742585848 wait complete. 2024-08-30T08:59:16.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:16.355+0000 7fad1ff5f640 1 Processor -- start 2024-08-30T08:59:16.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:16.355+0000 7fad1ff5f640 1 -- start start 2024-08-30T08:59:16.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:16.355+0000 7fad1ff5f640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fad18103900 0x7fad180754b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:16.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:16.355+0000 7fad1ff5f640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fad18109240 0x7fad180759f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:16.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:16.355+0000 7fad1ff5f640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fad18109b90 0x7fad18075f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:16.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:16.355+0000 7fad1ff5f640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fad18113850 con 0x7fad18103900 2024-08-30T08:59:16.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:16.355+0000 7fad1ff5f640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fad181136d0 con 0x7fad18109b90 2024-08-30T08:59:16.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:16.355+0000 7fad1ff5f640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fad181139d0 con 0x7fad18109240 2024-08-30T08:59:16.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:16.355+0000 7fad1dcfb640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fad18103900 0x7fad180754b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:16.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:16.355+0000 7fad1dcfb640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fad18103900 0x7fad180754b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52656/0 (socket says 172.21.15.42:52656) 2024-08-30T08:59:16.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:16.355+0000 7fad1dcfb640 1 -- 172.21.15.42:0/4002328053 learned_addr learned my addr 172.21.15.42:0/4002328053 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T08:59:16.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:16.355+0000 7fad1e4fc640 1 --2- 172.21.15.42:0/4002328053 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fad18109b90 0x7fad18075f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:16.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:16.355+0000 7fad1d4fa640 1 --2- 172.21.15.42:0/4002328053 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fad18109240 0x7fad180759f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:16.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:16.355+0000 7fad1dcfb640 1 -- 172.21.15.42:0/4002328053 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fad18109240 msgr2=0x7fad180759f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:16.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:16.355+0000 7fad1dcfb640 1 --2- 172.21.15.42:0/4002328053 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fad18109240 0x7fad180759f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:16.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:16.355+0000 7fad1dcfb640 1 -- 172.21.15.42:0/4002328053 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fad18109b90 msgr2=0x7fad18075f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:16.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:16.355+0000 7fad1dcfb640 1 --2- 172.21.15.42:0/4002328053 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fad18109b90 0x7fad18075f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:16.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:16.356+0000 7fad1dcfb640 1 -- 172.21.15.42:0/4002328053 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad181a8490 con 0x7fad18103900 2024-08-30T08:59:16.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:16.356+0000 7fad1dcfb640 1 --2- 172.21.15.42:0/4002328053 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fad18103900 0x7fad180754b0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fad1400d770 tx=0x7fad1400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:59:16.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:16.356+0000 7fad0effd640 1 -- 172.21.15.42:0/4002328053 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad14017070 con 0x7fad18103900 2024-08-30T08:59:16.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:16.356+0000 7fad0effd640 1 -- 172.21.15.42:0/4002328053 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fad140054d0 con 0x7fad18103900 2024-08-30T08:59:16.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:16.356+0000 7fad1ff5f640 1 -- 172.21.15.42:0/4002328053 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad181a8780 con 0x7fad18103900 2024-08-30T08:59:16.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:16.356+0000 7fad0effd640 1 -- 172.21.15.42:0/4002328053 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad14004c30 con 0x7fad18103900 2024-08-30T08:59:16.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:16.356+0000 7fad1ff5f640 1 -- 172.21.15.42:0/4002328053 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad181a8c90 con 0x7fad18103900 2024-08-30T08:59:16.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:16.357+0000 7fad1ff5f640 1 -- 172.21.15.42:0/4002328053 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fad180638f0 con 0x7fad18103900 2024-08-30T08:59:16.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:16.358+0000 7fad0effd640 1 -- 172.21.15.42:0/4002328053 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) ==== 152628+0+0 (secure 0 0 0) 0x7fad14004dd0 con 0x7fad18103900 2024-08-30T08:59:16.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:16.359+0000 7fad0effd640 1 --2- 172.21.15.42:0/4002328053 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fad040b4c40 0x7fad040b7150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:16.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:16.359+0000 7fad1d4fa640 1 --2- 172.21.15.42:0/4002328053 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fad040b4c40 0x7fad040b7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:16.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:16.359+0000 7fad0effd640 1 -- 172.21.15.42:0/4002328053 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 9092+0+0 (secure 0 0 0) 0x7fad140e5540 con 0x7fad18103900 2024-08-30T08:59:16.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:16.359+0000 7fad0effd640 1 --2- 172.21.15.42:0/4002328053 >> [v2:172.21.15.42:6818/3213526100,v1:172.21.15.42:6819/3213526100] conn(0x7fad040bfd50 0x7fad040c2240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:16.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:16.359+0000 7fad0effd640 1 -- 172.21.15.42:0/4002328053 --> [v2:172.21.15.42:6818/3213526100,v1:172.21.15.42:6819/3213526100] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fad14010e50 con 0x7fad040bfd50 2024-08-30T08:59:16.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:16.359+0000 7fad0effd640 1 -- 172.21.15.42:0/4002328053 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7fad140e5990 con 0x7fad18103900 2024-08-30T08:59:16.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:16.360+0000 7fad1e4fc640 1 --2- 172.21.15.42:0/4002328053 >> [v2:172.21.15.42:6818/3213526100,v1:172.21.15.42:6819/3213526100] conn(0x7fad040bfd50 0x7fad040c2240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:16.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:16.360+0000 7fad1d4fa640 1 --2- 172.21.15.42:0/4002328053 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fad040b4c40 0x7fad040b7150 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fad08009230 tx=0x7fad08031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:59:16.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:16.360+0000 7fad1e4fc640 1 --2- 172.21.15.42:0/4002328053 >> [v2:172.21.15.42:6818/3213526100,v1:172.21.15.42:6819/3213526100] conn(0x7fad040bfd50 0x7fad040c2240 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:59:16.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:16.361+0000 7fad0effd640 1 -- 172.21.15.42:0/4002328053 <== osd.6 v2:172.21.15.42:6818/3213526100 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fad14010e50 con 0x7fad040bfd50 2024-08-30T08:59:16.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:16.376+0000 7fad1ff5f640 1 -- 172.21.15.42:0/4002328053 --> [v2:172.21.15.42:6818/3213526100,v1:172.21.15.42:6819/3213526100] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fad18063d70 con 0x7fad040bfd50 2024-08-30T08:59:16.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:16.377+0000 7fad0effd640 1 -- 172.21.15.42:0/4002328053 <== osd.6 v2:172.21.15.42:6818/3213526100 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fad18063d70 con 0x7fad040bfd50 2024-08-30T08:59:16.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:16.377+0000 7fad1ff5f640 1 -- 172.21.15.42:0/4002328053 >> [v2:172.21.15.42:6818/3213526100,v1:172.21.15.42:6819/3213526100] conn(0x7fad040bfd50 msgr2=0x7fad040c2240 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:16.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:16.377+0000 7fad1ff5f640 1 --2- 172.21.15.42:0/4002328053 >> [v2:172.21.15.42:6818/3213526100,v1:172.21.15.42:6819/3213526100] conn(0x7fad040bfd50 0x7fad040c2240 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:16.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:16.377+0000 7fad1ff5f640 1 -- 172.21.15.42:0/4002328053 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fad040b4c40 msgr2=0x7fad040b7150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:16.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:16.377+0000 7fad1ff5f640 1 --2- 172.21.15.42:0/4002328053 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fad040b4c40 0x7fad040b7150 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fad08009230 tx=0x7fad08031400 comp rx=0 tx=0).stop 2024-08-30T08:59:16.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:16.377+0000 7fad1ff5f640 1 -- 172.21.15.42:0/4002328053 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fad18103900 msgr2=0x7fad180754b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:16.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:16.377+0000 7fad1ff5f640 1 --2- 172.21.15.42:0/4002328053 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fad18103900 0x7fad180754b0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fad1400d770 tx=0x7fad1400dc30 comp rx=0 tx=0).stop 2024-08-30T08:59:16.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:16.377+0000 7fad1e4fc640 1 -- 172.21.15.42:0/4002328053 reap_dead start 2024-08-30T08:59:16.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:16.377+0000 7fad1ff5f640 1 -- 172.21.15.42:0/4002328053 shutdown_connections 2024-08-30T08:59:16.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:16.377+0000 7fad1ff5f640 1 -- 172.21.15.42:0/4002328053 >> 172.21.15.42:0/4002328053 conn(0x7fad1807b5f0 msgr2=0x7fad18105300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:59:16.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:16.377+0000 7fad1ff5f640 1 -- 172.21.15.42:0/4002328053 shutdown_connections 2024-08-30T08:59:16.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:16.377+0000 7fad1ff5f640 1 -- 172.21.15.42:0/4002328053 wait complete. 2024-08-30T08:59:16.742 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:59:16 smithi042 ceph-mon[18652]: pgmap v326: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.1 KiB/s wr, 5 op/s 2024-08-30T08:59:16.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:59:16 smithi167 ceph-mon[22063]: pgmap v326: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.1 KiB/s wr, 5 op/s 2024-08-30T08:59:16.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:59:16 smithi190 ceph-mon[22001]: pgmap v326: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.1 KiB/s wr, 5 op/s 2024-08-30T08:59:17.419 INFO:teuthology.orchestra.run.smithi042.stdout:176093659189 2024-08-30T08:59:17.419 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-08-30T08:59:17.623 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T08:59:17.623 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T08:59:18.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:16.472+0000 7fbba6da2640 1 -- 172.21.15.42:0/2677490572 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbba0108160 msgr2=0x7fbba0108570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:18.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:16.472+0000 7fbba6da2640 1 --2- 172.21.15.42:0/2677490572 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbba0108160 0x7fbba0108570 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fbb88009900 tx=0x7fbb8802a410 comp rx=0 tx=0).stop 2024-08-30T08:59:18.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:16.472+0000 7fbba6da2640 1 -- 172.21.15.42:0/2677490572 shutdown_connections 2024-08-30T08:59:18.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:16.472+0000 7fbba6da2640 1 --2- 172.21.15.42:0/2677490572 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbba0108ab0 0x7fbba010f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:18.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:16.472+0000 7fbba6da2640 1 --2- 172.21.15.42:0/2677490572 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbba0108160 0x7fbba0108570 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:18.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:16.472+0000 7fbba6da2640 1 --2- 172.21.15.42:0/2677490572 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbba0102820 0x7fbba0102c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:18.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:16.472+0000 7fbba6da2640 1 -- 172.21.15.42:0/2677490572 >> 172.21.15.42:0/2677490572 conn(0x7fbba00794e0 msgr2=0x7fbba00798f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:59:18.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:16.472+0000 7fbba6da2640 1 -- 172.21.15.42:0/2677490572 shutdown_connections 2024-08-30T08:59:18.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:16.473+0000 7fbba6da2640 1 -- 172.21.15.42:0/2677490572 wait complete. 2024-08-30T08:59:18.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:16.473+0000 7fbba6da2640 1 Processor -- start 2024-08-30T08:59:18.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:16.473+0000 7fbba6da2640 1 -- start start 2024-08-30T08:59:18.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:16.473+0000 7fbba6da2640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbba0102820 0x7fbba019b110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:18.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:16.473+0000 7fbba6da2640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbba0108160 0x7fbba019b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:18.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:16.473+0000 7fbba6da2640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbba0108ab0 0x7fbba019bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:18.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:16.473+0000 7fbba6da2640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fbba0112700 con 0x7fbba0102820 2024-08-30T08:59:18.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:16.473+0000 7fbba6da2640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fbba0112580 con 0x7fbba0108ab0 2024-08-30T08:59:18.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:16.473+0000 7fbba6da2640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fbba0112880 con 0x7fbba0108160 2024-08-30T08:59:18.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:16.474+0000 7fbba4b3e640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbba0102820 0x7fbba019b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:18.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:16.474+0000 7fbba4b3e640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbba0102820 0x7fbba019b110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52672/0 (socket says 172.21.15.42:52672) 2024-08-30T08:59:18.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:16.474+0000 7fbba4b3e640 1 -- 172.21.15.42:0/1614369335 learned_addr learned my addr 172.21.15.42:0/1614369335 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T08:59:18.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:16.474+0000 7fbba533f640 1 --2- 172.21.15.42:0/1614369335 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbba0108ab0 0x7fbba019bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:18.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:16.474+0000 7fbb9ffff640 1 --2- 172.21.15.42:0/1614369335 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbba0108160 0x7fbba019b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:18.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:16.474+0000 7fbba4b3e640 1 -- 172.21.15.42:0/1614369335 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbba0108160 msgr2=0x7fbba019b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:18.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:16.474+0000 7fbba4b3e640 1 --2- 172.21.15.42:0/1614369335 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbba0108160 0x7fbba019b650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:18.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:16.474+0000 7fbba4b3e640 1 -- 172.21.15.42:0/1614369335 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbba0108ab0 msgr2=0x7fbba019bb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:18.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:16.474+0000 7fbba4b3e640 1 --2- 172.21.15.42:0/1614369335 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbba0108ab0 0x7fbba019bb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:18.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:16.474+0000 7fbba4b3e640 1 -- 172.21.15.42:0/1614369335 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbba01a6310 con 0x7fbba0102820 2024-08-30T08:59:18.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:16.474+0000 7fbba4b3e640 1 --2- 172.21.15.42:0/1614369335 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbba0102820 0x7fbba019b110 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fbb9000cce0 tx=0x7fbb9000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:59:18.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:16.474+0000 7fbb9dffb640 1 -- 172.21.15.42:0/1614369335 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb9000f650 con 0x7fbba0102820 2024-08-30T08:59:18.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:16.474+0000 7fbb9dffb640 1 -- 172.21.15.42:0/1614369335 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb9000fca0 con 0x7fbba0102820 2024-08-30T08:59:18.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:16.474+0000 7fbb9dffb640 1 -- 172.21.15.42:0/1614369335 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb90018b40 con 0x7fbba0102820 2024-08-30T08:59:18.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:16.474+0000 7fbba6da2640 1 -- 172.21.15.42:0/1614369335 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbba01a6600 con 0x7fbba0102820 2024-08-30T08:59:18.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:16.475+0000 7fbba6da2640 1 -- 172.21.15.42:0/1614369335 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbba01a6b10 con 0x7fbba0102820 2024-08-30T08:59:18.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:16.475+0000 7fbba6da2640 1 -- 172.21.15.42:0/1614369335 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fbba00638f0 con 0x7fbba0102820 2024-08-30T08:59:18.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:16.477+0000 7fbb9dffb640 1 -- 172.21.15.42:0/1614369335 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) ==== 152628+0+0 (secure 0 0 0) 0x7fbb90018ce0 con 0x7fbba0102820 2024-08-30T08:59:18.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:16.478+0000 7fbb9dffb640 1 --2- 172.21.15.42:0/1614369335 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fbb8c0b4bf0 0x7fbb8c0b7100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:18.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:16.478+0000 7fbb9ffff640 1 --2- 172.21.15.42:0/1614369335 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fbb8c0b4bf0 0x7fbb8c0b7100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:18.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:16.478+0000 7fbb9dffb640 1 -- 172.21.15.42:0/1614369335 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 9092+0+0 (secure 0 0 0) 0x7fbb900e5fc0 con 0x7fbba0102820 2024-08-30T08:59:18.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:16.478+0000 7fbb9dffb640 1 --2- 172.21.15.42:0/1614369335 >> [v2:172.21.15.190:6800/1101326404,v1:172.21.15.190:6801/1101326404] conn(0x7fbb8c0bfd00 0x7fbb8c0c21f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:18.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:16.478+0000 7fbb9dffb640 1 -- 172.21.15.42:0/1614369335 --> [v2:172.21.15.190:6800/1101326404,v1:172.21.15.190:6801/1101326404] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fbb90011ec0 con 0x7fbb8c0bfd00 2024-08-30T08:59:18.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:16.478+0000 7fbb9dffb640 1 -- 172.21.15.42:0/1614369335 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7fbb900e6410 con 0x7fbba0102820 2024-08-30T08:59:18.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:16.479+0000 7fbb9ffff640 1 --2- 172.21.15.42:0/1614369335 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fbb8c0b4bf0 0x7fbb8c0b7100 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fbb88000c00 tx=0x7fbb88031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:59:18.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:16.479+0000 7fbba533f640 1 --2- 172.21.15.42:0/1614369335 >> [v2:172.21.15.190:6800/1101326404,v1:172.21.15.190:6801/1101326404] conn(0x7fbb8c0bfd00 0x7fbb8c0c21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:18.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:16.479+0000 7fbba533f640 1 --2- 172.21.15.42:0/1614369335 >> [v2:172.21.15.190:6800/1101326404,v1:172.21.15.190:6801/1101326404] conn(0x7fbb8c0bfd00 0x7fbb8c0c21f0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:59:18.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:16.480+0000 7fbb9dffb640 1 -- 172.21.15.42:0/1614369335 <== osd.2 v2:172.21.15.190:6800/1101326404 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fbb90011ec0 con 0x7fbb8c0bfd00 2024-08-30T08:59:18.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:16.494+0000 7fbba6da2640 1 -- 172.21.15.42:0/1614369335 --> [v2:172.21.15.190:6800/1101326404,v1:172.21.15.190:6801/1101326404] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fbba0063d70 con 0x7fbb8c0bfd00 2024-08-30T08:59:18.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:16.495+0000 7fbb9dffb640 1 -- 172.21.15.42:0/1614369335 <== osd.2 v2:172.21.15.190:6800/1101326404 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fbba0063d70 con 0x7fbb8c0bfd00 2024-08-30T08:59:18.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:16.495+0000 7fbba6da2640 1 -- 172.21.15.42:0/1614369335 >> [v2:172.21.15.190:6800/1101326404,v1:172.21.15.190:6801/1101326404] conn(0x7fbb8c0bfd00 msgr2=0x7fbb8c0c21f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:18.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:16.495+0000 7fbba6da2640 1 --2- 172.21.15.42:0/1614369335 >> [v2:172.21.15.190:6800/1101326404,v1:172.21.15.190:6801/1101326404] conn(0x7fbb8c0bfd00 0x7fbb8c0c21f0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:18.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:16.495+0000 7fbba6da2640 1 -- 172.21.15.42:0/1614369335 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fbb8c0b4bf0 msgr2=0x7fbb8c0b7100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:18.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:16.495+0000 7fbba6da2640 1 --2- 172.21.15.42:0/1614369335 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fbb8c0b4bf0 0x7fbb8c0b7100 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fbb88000c00 tx=0x7fbb88031400 comp rx=0 tx=0).stop 2024-08-30T08:59:18.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:16.495+0000 7fbba6da2640 1 -- 172.21.15.42:0/1614369335 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbba0102820 msgr2=0x7fbba019b110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:18.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:16.495+0000 7fbba6da2640 1 --2- 172.21.15.42:0/1614369335 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbba0102820 0x7fbba019b110 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fbb9000cce0 tx=0x7fbb9000a620 comp rx=0 tx=0).stop 2024-08-30T08:59:18.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:16.495+0000 7fbba533f640 1 -- 172.21.15.42:0/1614369335 reap_dead start 2024-08-30T08:59:18.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:16.495+0000 7fbba6da2640 1 -- 172.21.15.42:0/1614369335 shutdown_connections 2024-08-30T08:59:18.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:16.495+0000 7fbba6da2640 1 -- 172.21.15.42:0/1614369335 >> 172.21.15.42:0/1614369335 conn(0x7fbba00794e0 msgr2=0x7fbba0105110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:59:18.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:16.495+0000 7fbba6da2640 1 -- 172.21.15.42:0/1614369335 shutdown_connections 2024-08-30T08:59:18.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:16.495+0000 7fbba6da2640 1 -- 172.21.15.42:0/1614369335 wait complete. 2024-08-30T08:59:18.661 INFO:teuthology.orchestra.run.smithi042.stdout:77309411401 2024-08-30T08:59:18.662 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-08-30T08:59:18.801 INFO:teuthology.orchestra.run.smithi042.stdout:34359738451 2024-08-30T08:59:18.801 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-08-30T08:59:18.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:59:18 smithi042 ceph-mon[18652]: pgmap v327: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-08-30T08:59:18.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:59:18 smithi190 ceph-mon[22001]: pgmap v327: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-08-30T08:59:19.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:59:18 smithi167 ceph-mon[22063]: pgmap v327: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-08-30T08:59:20.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:20.582+0000 7f264744f640 1 -- 172.21.15.42:0/4182118956 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2640102800 msgr2=0x7f2640102c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:20.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:20.582+0000 7f264744f640 1 --2- 172.21.15.42:0/4182118956 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2640102800 0x7f2640102c10 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f263c009900 tx=0x7f263c02a410 comp rx=0 tx=0).stop 2024-08-30T08:59:20.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:20.583+0000 7f264744f640 1 -- 172.21.15.42:0/4182118956 shutdown_connections 2024-08-30T08:59:20.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:20.583+0000 7f264744f640 1 --2- 172.21.15.42:0/4182118956 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2640108140 0x7f2640108550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:20.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:20.583+0000 7f264744f640 1 --2- 172.21.15.42:0/4182118956 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2640108a90 0x7f264010f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:20.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:20.583+0000 7f264744f640 1 --2- 172.21.15.42:0/4182118956 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2640102800 0x7f2640102c10 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:20.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:20.583+0000 7f264744f640 1 -- 172.21.15.42:0/4182118956 >> 172.21.15.42:0/4182118956 conn(0x7f26400794d0 msgr2=0x7f26400798e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:59:20.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:20.583+0000 7f264744f640 1 -- 172.21.15.42:0/4182118956 shutdown_connections 2024-08-30T08:59:20.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:20.583+0000 7f264744f640 1 -- 172.21.15.42:0/4182118956 wait complete. 2024-08-30T08:59:20.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:20.583+0000 7f264744f640 1 Processor -- start 2024-08-30T08:59:20.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:20.583+0000 7f264744f640 1 -- start start 2024-08-30T08:59:20.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:20.584+0000 7f264744f640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2640102800 0x7f264019b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:20.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:20.584+0000 7f264744f640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2640108140 0x7f264019b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:20.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:20.584+0000 7f264744f640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2640108a90 0x7f264019bb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:20.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:20.584+0000 7f264744f640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f26401126f0 con 0x7f2640102800 2024-08-30T08:59:20.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:20.584+0000 7f264744f640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f2640112570 con 0x7f2640108a90 2024-08-30T08:59:20.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:20.584+0000 7f264744f640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f2640112870 con 0x7f2640108140 2024-08-30T08:59:20.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:20.584+0000 7f26451eb640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2640102800 0x7f264019b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:20.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:20.584+0000 7f26451eb640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2640102800 0x7f264019b0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38748/0 (socket says 172.21.15.42:38748) 2024-08-30T08:59:20.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:20.584+0000 7f26451eb640 1 -- 172.21.15.42:0/2017603885 learned_addr learned my addr 172.21.15.42:0/2017603885 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T08:59:20.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:20.584+0000 7f26449ea640 1 --2- 172.21.15.42:0/2017603885 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2640108140 0x7f264019b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:20.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:20.584+0000 7f26459ec640 1 --2- 172.21.15.42:0/2017603885 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2640108a90 0x7f264019bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:20.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:20.584+0000 7f26451eb640 1 -- 172.21.15.42:0/2017603885 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2640108140 msgr2=0x7f264019b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:20.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:20.584+0000 7f26451eb640 1 --2- 172.21.15.42:0/2017603885 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2640108140 0x7f264019b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:20.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:20.584+0000 7f26451eb640 1 -- 172.21.15.42:0/2017603885 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2640108a90 msgr2=0x7f264019bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:20.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:20.584+0000 7f26451eb640 1 --2- 172.21.15.42:0/2017603885 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2640108a90 0x7f264019bb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:20.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:20.584+0000 7f26451eb640 1 -- 172.21.15.42:0/2017603885 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26401a6350 con 0x7f2640102800 2024-08-30T08:59:20.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:20.585+0000 7f26451eb640 1 --2- 172.21.15.42:0/2017603885 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2640102800 0x7f264019b0c0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f263c005940 tx=0x7f263c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:59:20.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:20.585+0000 7f26367fc640 1 -- 172.21.15.42:0/2017603885 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f263c03f030 con 0x7f2640102800 2024-08-30T08:59:20.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:20.585+0000 7f264744f640 1 -- 172.21.15.42:0/2017603885 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26401a65e0 con 0x7f2640102800 2024-08-30T08:59:20.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:20.585+0000 7f26367fc640 1 -- 172.21.15.42:0/2017603885 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f263c003aa0 con 0x7f2640102800 2024-08-30T08:59:20.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:20.585+0000 7f26367fc640 1 -- 172.21.15.42:0/2017603885 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f263c035070 con 0x7f2640102800 2024-08-30T08:59:20.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:20.585+0000 7f264744f640 1 -- 172.21.15.42:0/2017603885 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26401a6b10 con 0x7f2640102800 2024-08-30T08:59:20.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:20.586+0000 7f264744f640 1 -- 172.21.15.42:0/2017603885 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f26400638f0 con 0x7f2640102800 2024-08-30T08:59:20.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:20.588+0000 7f26367fc640 1 -- 172.21.15.42:0/2017603885 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) ==== 152628+0+0 (secure 0 0 0) 0x7f263c042030 con 0x7f2640102800 2024-08-30T08:59:20.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:20.588+0000 7f26367fc640 1 --2- 172.21.15.42:0/2017603885 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f262c0b4bf0 0x7f262c0b7100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:20.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:20.588+0000 7f26449ea640 1 --2- 172.21.15.42:0/2017603885 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f262c0b4bf0 0x7f262c0b7100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:20.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:20.588+0000 7f26367fc640 1 -- 172.21.15.42:0/2017603885 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 9092+0+0 (secure 0 0 0) 0x7f263c106100 con 0x7f2640102800 2024-08-30T08:59:20.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:20.589+0000 7f26367fc640 1 --2- 172.21.15.42:0/2017603885 >> [v2:172.21.15.42:6826/1123536325,v1:172.21.15.42:6827/1123536325] conn(0x7f262c0bfc80 0x7f262c0c2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:20.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:20.589+0000 7f26367fc640 1 -- 172.21.15.42:0/2017603885 --> [v2:172.21.15.42:6826/1123536325,v1:172.21.15.42:6827/1123536325] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f262c0c2840 con 0x7f262c0bfc80 2024-08-30T08:59:20.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:20.589+0000 7f26367fc640 1 -- 172.21.15.42:0/2017603885 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7f263c106550 con 0x7f2640102800 2024-08-30T08:59:20.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:20.589+0000 7f26459ec640 1 --2- 172.21.15.42:0/2017603885 >> [v2:172.21.15.42:6826/1123536325,v1:172.21.15.42:6827/1123536325] conn(0x7f262c0bfc80 0x7f262c0c2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:20.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:20.589+0000 7f26449ea640 1 --2- 172.21.15.42:0/2017603885 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f262c0b4bf0 0x7f262c0b7100 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f2640100910 tx=0x7f263000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:59:20.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:20.589+0000 7f26459ec640 1 --2- 172.21.15.42:0/2017603885 >> [v2:172.21.15.42:6826/1123536325,v1:172.21.15.42:6827/1123536325] conn(0x7f262c0bfc80 0x7f262c0c2170 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:59:20.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:20.590+0000 7f26367fc640 1 -- 172.21.15.42:0/2017603885 <== osd.9 v2:172.21.15.42:6826/1123536325 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f262c0c2840 con 0x7f262c0bfc80 2024-08-30T08:59:20.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:20.604+0000 7f264744f640 1 -- 172.21.15.42:0/2017603885 --> [v2:172.21.15.42:6826/1123536325,v1:172.21.15.42:6827/1123536325] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f2640063d20 con 0x7f262c0bfc80 2024-08-30T08:59:20.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:20.604+0000 7f26367fc640 1 -- 172.21.15.42:0/2017603885 <== osd.9 v2:172.21.15.42:6826/1123536325 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f2640063d20 con 0x7f262c0bfc80 2024-08-30T08:59:20.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:20.604+0000 7f264744f640 1 -- 172.21.15.42:0/2017603885 >> [v2:172.21.15.42:6826/1123536325,v1:172.21.15.42:6827/1123536325] conn(0x7f262c0bfc80 msgr2=0x7f262c0c2170 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:20.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:20.604+0000 7f264744f640 1 --2- 172.21.15.42:0/2017603885 >> [v2:172.21.15.42:6826/1123536325,v1:172.21.15.42:6827/1123536325] conn(0x7f262c0bfc80 0x7f262c0c2170 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:20.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:20.604+0000 7f264744f640 1 -- 172.21.15.42:0/2017603885 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f262c0b4bf0 msgr2=0x7f262c0b7100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:20.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:20.604+0000 7f264744f640 1 --2- 172.21.15.42:0/2017603885 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f262c0b4bf0 0x7f262c0b7100 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f2640100910 tx=0x7f263000b410 comp rx=0 tx=0).stop 2024-08-30T08:59:20.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:20.604+0000 7f264744f640 1 -- 172.21.15.42:0/2017603885 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2640102800 msgr2=0x7f264019b0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:20.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:20.604+0000 7f264744f640 1 --2- 172.21.15.42:0/2017603885 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2640102800 0x7f264019b0c0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f263c005940 tx=0x7f263c004270 comp rx=0 tx=0).stop 2024-08-30T08:59:20.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:20.604+0000 7f26459ec640 1 -- 172.21.15.42:0/2017603885 reap_dead start 2024-08-30T08:59:20.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:20.604+0000 7f264744f640 1 -- 172.21.15.42:0/2017603885 shutdown_connections 2024-08-30T08:59:20.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:20.604+0000 7f264744f640 1 -- 172.21.15.42:0/2017603885 >> 172.21.15.42:0/2017603885 conn(0x7f26400794d0 msgr2=0x7f26401012a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:59:20.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:20.605+0000 7f264744f640 1 -- 172.21.15.42:0/2017603885 shutdown_connections 2024-08-30T08:59:20.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:20.605+0000 7f264744f640 1 -- 172.21.15.42:0/2017603885 wait complete. 2024-08-30T08:59:20.632 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:59:20 smithi042 ceph-mon[18652]: pgmap v328: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-08-30T08:59:20.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:19.850+0000 7f4a264fc640 1 -- 172.21.15.42:0/4144343313 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4a20108a90 msgr2=0x7f4a2010f2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:20.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:19.850+0000 7f4a264fc640 1 --2- 172.21.15.42:0/4144343313 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4a20108a90 0x7f4a2010f2c0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f4a18009ca0 tx=0x7f4a1802a8c0 comp rx=0 tx=0).stop 2024-08-30T08:59:20.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:19.850+0000 7f4a264fc640 1 -- 172.21.15.42:0/4144343313 shutdown_connections 2024-08-30T08:59:20.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:19.850+0000 7f4a264fc640 1 --2- 172.21.15.42:0/4144343313 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4a20108a90 0x7f4a2010f2c0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:20.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:19.850+0000 7f4a264fc640 1 --2- 172.21.15.42:0/4144343313 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4a20108140 0x7f4a20108550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:20.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:19.850+0000 7f4a264fc640 1 --2- 172.21.15.42:0/4144343313 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4a20102800 0x7f4a20102c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:20.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:19.850+0000 7f4a264fc640 1 -- 172.21.15.42:0/4144343313 >> 172.21.15.42:0/4144343313 conn(0x7f4a200794d0 msgr2=0x7f4a200798e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:59:20.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:19.850+0000 7f4a264fc640 1 -- 172.21.15.42:0/4144343313 shutdown_connections 2024-08-30T08:59:20.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:19.851+0000 7f4a264fc640 1 -- 172.21.15.42:0/4144343313 wait complete. 2024-08-30T08:59:20.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:19.851+0000 7f4a264fc640 1 Processor -- start 2024-08-30T08:59:20.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:19.851+0000 7f4a264fc640 1 -- start start 2024-08-30T08:59:20.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:19.851+0000 7f4a264fc640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4a20102800 0x7f4a2019a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:20.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:19.851+0000 7f4a264fc640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4a20108140 0x7f4a2019a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:20.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:19.851+0000 7f4a264fc640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4a20108a90 0x7f4a2019f970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:20.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:19.851+0000 7f4a264fc640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f4a20113820 con 0x7f4a20108a90 2024-08-30T08:59:20.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:19.851+0000 7f4a264fc640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4a201136a0 con 0x7f4a20102800 2024-08-30T08:59:20.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:19.851+0000 7f4a264fc640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f4a201139a0 con 0x7f4a20108140 2024-08-30T08:59:20.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:19.852+0000 7f4a24a99640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4a20108a90 0x7f4a2019f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:20.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:19.852+0000 7f4a24a99640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4a20108a90 0x7f4a2019f970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38696/0 (socket says 172.21.15.42:38696) 2024-08-30T08:59:20.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:19.852+0000 7f4a24a99640 1 -- 172.21.15.42:0/2135753871 learned_addr learned my addr 172.21.15.42:0/2135753871 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T08:59:20.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:19.852+0000 7f4a17fff640 1 --2- 172.21.15.42:0/2135753871 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4a20108140 0x7f4a2019a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:20.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:19.852+0000 7f4a1ffff640 1 --2- 172.21.15.42:0/2135753871 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4a20102800 0x7f4a2019a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:20.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:19.852+0000 7f4a24a99640 1 -- 172.21.15.42:0/2135753871 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4a20108140 msgr2=0x7f4a2019a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:20.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:19.852+0000 7f4a24a99640 1 --2- 172.21.15.42:0/2135753871 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4a20108140 0x7f4a2019a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:20.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:19.852+0000 7f4a24a99640 1 -- 172.21.15.42:0/2135753871 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4a20102800 msgr2=0x7f4a2019a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:20.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:19.852+0000 7f4a24a99640 1 --2- 172.21.15.42:0/2135753871 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4a20102800 0x7f4a2019a130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:20.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:19.852+0000 7f4a24a99640 1 -- 172.21.15.42:0/2135753871 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a2019ffe0 con 0x7f4a20108a90 2024-08-30T08:59:20.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:19.852+0000 7f4a24a99640 1 --2- 172.21.15.42:0/2135753871 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4a20108a90 0x7f4a2019f970 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f4a18005cc0 tx=0x7f4a1802f470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:59:20.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:19.852+0000 7f4a1dffb640 1 -- 172.21.15.42:0/2135753871 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a18003cf0 con 0x7f4a20108a90 2024-08-30T08:59:20.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:19.852+0000 7f4a1dffb640 1 -- 172.21.15.42:0/2135753871 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a180044f0 con 0x7f4a20108a90 2024-08-30T08:59:20.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:19.852+0000 7f4a264fc640 1 -- 172.21.15.42:0/2135753871 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a201a0270 con 0x7f4a20108a90 2024-08-30T08:59:20.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:19.852+0000 7f4a1dffb640 1 -- 172.21.15.42:0/2135753871 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a1802fd10 con 0x7f4a20108a90 2024-08-30T08:59:20.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:19.852+0000 7f4a264fc640 1 -- 172.21.15.42:0/2135753871 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a201a6af0 con 0x7f4a20108a90 2024-08-30T08:59:20.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:19.853+0000 7f4a264fc640 1 -- 172.21.15.42:0/2135753871 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4a200638f0 con 0x7f4a20108a90 2024-08-30T08:59:20.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:19.855+0000 7f4a1dffb640 1 -- 172.21.15.42:0/2135753871 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) ==== 152628+0+0 (secure 0 0 0) 0x7f4a18004040 con 0x7f4a20108a90 2024-08-30T08:59:20.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:19.856+0000 7f4a1dffb640 1 --2- 172.21.15.42:0/2135753871 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f4a0c0b4c70 0x7f4a0c0b7180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:20.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:19.856+0000 7f4a1ffff640 1 --2- 172.21.15.42:0/2135753871 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f4a0c0b4c70 0x7f4a0c0b7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:20.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:19.856+0000 7f4a1dffb640 1 -- 172.21.15.42:0/2135753871 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 9092+0+0 (secure 0 0 0) 0x7f4a18106e00 con 0x7f4a20108a90 2024-08-30T08:59:20.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:19.856+0000 7f4a1dffb640 1 --2- 172.21.15.42:0/2135753871 >> [v2:172.21.15.167:6808/2927511419,v1:172.21.15.167:6809/2927511419] conn(0x7f4a0c0bfd80 0x7f4a0c0c2270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:20.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:19.856+0000 7f4a1dffb640 1 -- 172.21.15.42:0/2135753871 --> [v2:172.21.15.167:6808/2927511419,v1:172.21.15.167:6809/2927511419] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4a1803fec0 con 0x7f4a0c0bfd80 2024-08-30T08:59:20.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:19.856+0000 7f4a1dffb640 1 -- 172.21.15.42:0/2135753871 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7f4a181071b0 con 0x7f4a20108a90 2024-08-30T08:59:20.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:19.856+0000 7f4a1ffff640 1 --2- 172.21.15.42:0/2135753871 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f4a0c0b4c70 0x7f4a0c0b7180 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f4a10005e10 tx=0x7f4a10005da0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:59:20.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:19.856+0000 7f4a17fff640 1 --2- 172.21.15.42:0/2135753871 >> [v2:172.21.15.167:6808/2927511419,v1:172.21.15.167:6809/2927511419] conn(0x7f4a0c0bfd80 0x7f4a0c0c2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:20.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:19.857+0000 7f4a17fff640 1 --2- 172.21.15.42:0/2135753871 >> [v2:172.21.15.167:6808/2927511419,v1:172.21.15.167:6809/2927511419] conn(0x7f4a0c0bfd80 0x7f4a0c0c2270 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:59:20.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:19.858+0000 7f4a1dffb640 1 -- 172.21.15.42:0/2135753871 <== osd.4 v2:172.21.15.167:6808/2927511419 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4a1803fec0 con 0x7f4a0c0bfd80 2024-08-30T08:59:20.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:19.872+0000 7f4a264fc640 1 -- 172.21.15.42:0/2135753871 --> [v2:172.21.15.167:6808/2927511419,v1:172.21.15.167:6809/2927511419] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f4a20063cf0 con 0x7f4a0c0bfd80 2024-08-30T08:59:20.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:19.872+0000 7f4a1dffb640 1 -- 172.21.15.42:0/2135753871 <== osd.4 v2:172.21.15.167:6808/2927511419 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f4a20063cf0 con 0x7f4a0c0bfd80 2024-08-30T08:59:20.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:19.872+0000 7f4a264fc640 1 -- 172.21.15.42:0/2135753871 >> [v2:172.21.15.167:6808/2927511419,v1:172.21.15.167:6809/2927511419] conn(0x7f4a0c0bfd80 msgr2=0x7f4a0c0c2270 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:20.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:19.872+0000 7f4a264fc640 1 --2- 172.21.15.42:0/2135753871 >> [v2:172.21.15.167:6808/2927511419,v1:172.21.15.167:6809/2927511419] conn(0x7f4a0c0bfd80 0x7f4a0c0c2270 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:20.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:19.872+0000 7f4a264fc640 1 -- 172.21.15.42:0/2135753871 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f4a0c0b4c70 msgr2=0x7f4a0c0b7180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:20.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:19.872+0000 7f4a264fc640 1 --2- 172.21.15.42:0/2135753871 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f4a0c0b4c70 0x7f4a0c0b7180 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f4a10005e10 tx=0x7f4a10005da0 comp rx=0 tx=0).stop 2024-08-30T08:59:20.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:19.872+0000 7f4a264fc640 1 -- 172.21.15.42:0/2135753871 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4a20108a90 msgr2=0x7f4a2019f970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:20.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:19.872+0000 7f4a264fc640 1 --2- 172.21.15.42:0/2135753871 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4a20108a90 0x7f4a2019f970 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f4a18005cc0 tx=0x7f4a1802f470 comp rx=0 tx=0).stop 2024-08-30T08:59:20.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:19.872+0000 7f4a24a99640 1 -- 172.21.15.42:0/2135753871 reap_dead start 2024-08-30T08:59:20.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:19.873+0000 7f4a264fc640 1 -- 172.21.15.42:0/2135753871 shutdown_connections 2024-08-30T08:59:20.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:19.873+0000 7f4a264fc640 1 -- 172.21.15.42:0/2135753871 >> 172.21.15.42:0/2135753871 conn(0x7f4a200794d0 msgr2=0x7f4a20104fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:59:20.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:19.873+0000 7f4a264fc640 1 -- 172.21.15.42:0/2135753871 shutdown_connections 2024-08-30T08:59:20.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:19.873+0000 7f4a264fc640 1 -- 172.21.15.42:0/2135753871 wait complete. 2024-08-30T08:59:20.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:59:20 smithi190 ceph-mon[22001]: pgmap v328: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-08-30T08:59:21.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:19.493+0000 7f30dc5d7640 1 -- 172.21.15.42:0/2030037874 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f30d4108ac0 msgr2=0x7f30d410f2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:21.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:19.493+0000 7f30dc5d7640 1 --2- 172.21.15.42:0/2030037874 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f30d4108ac0 0x7f30d410f2f0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f30cc009900 tx=0x7f30cc02a410 comp rx=0 tx=0).stop 2024-08-30T08:59:21.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:19.493+0000 7f30dc5d7640 1 -- 172.21.15.42:0/2030037874 shutdown_connections 2024-08-30T08:59:21.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:19.493+0000 7f30dc5d7640 1 --2- 172.21.15.42:0/2030037874 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f30d4108ac0 0x7f30d410f2f0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:21.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:19.493+0000 7f30dc5d7640 1 --2- 172.21.15.42:0/2030037874 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f30d4108170 0x7f30d4108580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:21.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:19.493+0000 7f30dc5d7640 1 --2- 172.21.15.42:0/2030037874 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f30d4102880 0x7f30d4102c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:21.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:19.493+0000 7f30dc5d7640 1 -- 172.21.15.42:0/2030037874 >> 172.21.15.42:0/2030037874 conn(0x7f30d4079530 msgr2=0x7f30d4079940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:59:21.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:19.493+0000 7f30dc5d7640 1 -- 172.21.15.42:0/2030037874 shutdown_connections 2024-08-30T08:59:21.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:19.493+0000 7f30dc5d7640 1 -- 172.21.15.42:0/2030037874 wait complete. 2024-08-30T08:59:21.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:19.494+0000 7f30dc5d7640 1 Processor -- start 2024-08-30T08:59:21.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:19.494+0000 7f30dc5d7640 1 -- start start 2024-08-30T08:59:21.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:19.494+0000 7f30dc5d7640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f30d4102880 0x7f30d4199130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:21.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:19.494+0000 7f30dc5d7640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f30d4108170 0x7f30d41996d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:21.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:19.494+0000 7f30dc5d7640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f30d4108ac0 0x7f30d419e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:21.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:19.494+0000 7f30dc5d7640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f30d41127e0 con 0x7f30d4102880 2024-08-30T08:59:21.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:19.494+0000 7f30dc5d7640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f30d4112660 con 0x7f30d4108170 2024-08-30T08:59:21.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:19.494+0000 7f30dc5d7640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f30d4112960 con 0x7f30d4108ac0 2024-08-30T08:59:21.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:19.494+0000 7f30da373640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f30d4102880 0x7f30d4199130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:21.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:19.494+0000 7f30da373640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f30d4102880 0x7f30d4199130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38666/0 (socket says 172.21.15.42:38666) 2024-08-30T08:59:21.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:19.494+0000 7f30da373640 1 -- 172.21.15.42:0/2830938219 learned_addr learned my addr 172.21.15.42:0/2830938219 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T08:59:21.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:19.495+0000 7f30d9b72640 1 --2- 172.21.15.42:0/2830938219 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f30d4108170 0x7f30d41996d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:21.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:19.495+0000 7f30dab74640 1 --2- 172.21.15.42:0/2830938219 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f30d4108ac0 0x7f30d419e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:21.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:19.495+0000 7f30da373640 1 -- 172.21.15.42:0/2830938219 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f30d4108ac0 msgr2=0x7f30d419e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:21.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:19.495+0000 7f30da373640 1 --2- 172.21.15.42:0/2830938219 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f30d4108ac0 0x7f30d419e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:21.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:19.495+0000 7f30da373640 1 -- 172.21.15.42:0/2830938219 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f30d4108170 msgr2=0x7f30d41996d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:21.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:19.495+0000 7f30da373640 1 --2- 172.21.15.42:0/2830938219 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f30d4108170 0x7f30d41996d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:21.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:19.495+0000 7f30da373640 1 -- 172.21.15.42:0/2830938219 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30d419efe0 con 0x7f30d4102880 2024-08-30T08:59:21.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:19.495+0000 7f30dab74640 1 --2- 172.21.15.42:0/2830938219 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f30d4108ac0 0x7f30d419e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-30T08:59:21.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:19.495+0000 7f30da373640 1 --2- 172.21.15.42:0/2830938219 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f30d4102880 0x7f30d4199130 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f30c400ba50 tx=0x7f30c400be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:59:21.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:19.495+0000 7f30d37fe640 1 -- 172.21.15.42:0/2830938219 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30c4019070 con 0x7f30d4102880 2024-08-30T08:59:21.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:19.495+0000 7f30d37fe640 1 -- 172.21.15.42:0/2830938219 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f30c4014440 con 0x7f30d4102880 2024-08-30T08:59:21.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:19.495+0000 7f30dc5d7640 1 -- 172.21.15.42:0/2830938219 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30d419f2d0 con 0x7f30d4102880 2024-08-30T08:59:21.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:19.495+0000 7f30d37fe640 1 -- 172.21.15.42:0/2830938219 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30c40075b0 con 0x7f30d4102880 2024-08-30T08:59:21.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:19.495+0000 7f30dc5d7640 1 -- 172.21.15.42:0/2830938219 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30d41a5af0 con 0x7f30d4102880 2024-08-30T08:59:21.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:19.496+0000 7f30dc5d7640 1 -- 172.21.15.42:0/2830938219 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f30d40638f0 con 0x7f30d4102880 2024-08-30T08:59:21.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:19.498+0000 7f30d37fe640 1 -- 172.21.15.42:0/2830938219 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) ==== 152628+0+0 (secure 0 0 0) 0x7f30c401c070 con 0x7f30d4102880 2024-08-30T08:59:21.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:19.498+0000 7f30d37fe640 1 --2- 172.21.15.42:0/2830938219 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f30bc0b4bf0 0x7f30bc0b7100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:21.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:19.499+0000 7f30d9b72640 1 --2- 172.21.15.42:0/2830938219 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f30bc0b4bf0 0x7f30bc0b7100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:21.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:19.499+0000 7f30d37fe640 1 -- 172.21.15.42:0/2830938219 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 9092+0+0 (secure 0 0 0) 0x7f30c40e52e0 con 0x7f30d4102880 2024-08-30T08:59:21.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:19.499+0000 7f30d37fe640 1 --2- 172.21.15.42:0/2830938219 >> [v2:172.21.15.190:6816/3242198843,v1:172.21.15.190:6817/3242198843] conn(0x7f30bc0bfd00 0x7f30bc0c21f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:21.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:19.499+0000 7f30d37fe640 1 -- 172.21.15.42:0/2830938219 --> [v2:172.21.15.190:6816/3242198843,v1:172.21.15.190:6817/3242198843] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f30c4020e50 con 0x7f30bc0bfd00 2024-08-30T08:59:21.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:19.499+0000 7f30d37fe640 1 -- 172.21.15.42:0/2830938219 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7f30c40e5730 con 0x7f30d4102880 2024-08-30T08:59:21.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:19.499+0000 7f30d9b72640 1 --2- 172.21.15.42:0/2830938219 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f30bc0b4bf0 0x7f30bc0b7100 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f30d4079b30 tx=0x7f30c800a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:59:21.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:19.499+0000 7f30dab74640 1 --2- 172.21.15.42:0/2830938219 >> [v2:172.21.15.190:6816/3242198843,v1:172.21.15.190:6817/3242198843] conn(0x7f30bc0bfd00 0x7f30bc0c21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:21.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:19.500+0000 7f30dab74640 1 --2- 172.21.15.42:0/2830938219 >> [v2:172.21.15.190:6816/3242198843,v1:172.21.15.190:6817/3242198843] conn(0x7f30bc0bfd00 0x7f30bc0c21f0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:59:21.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:19.501+0000 7f30d37fe640 1 -- 172.21.15.42:0/2830938219 <== osd.8 v2:172.21.15.190:6816/3242198843 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f30c4020e50 con 0x7f30bc0bfd00 2024-08-30T08:59:21.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:19.514+0000 7f30dc5d7640 1 -- 172.21.15.42:0/2830938219 --> [v2:172.21.15.190:6816/3242198843,v1:172.21.15.190:6817/3242198843] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f30d4063d70 con 0x7f30bc0bfd00 2024-08-30T08:59:21.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:19.515+0000 7f30d37fe640 1 -- 172.21.15.42:0/2830938219 <== osd.8 v2:172.21.15.190:6816/3242198843 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f30d4063d70 con 0x7f30bc0bfd00 2024-08-30T08:59:21.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:19.515+0000 7f30dc5d7640 1 -- 172.21.15.42:0/2830938219 >> [v2:172.21.15.190:6816/3242198843,v1:172.21.15.190:6817/3242198843] conn(0x7f30bc0bfd00 msgr2=0x7f30bc0c21f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:21.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:19.515+0000 7f30dc5d7640 1 --2- 172.21.15.42:0/2830938219 >> [v2:172.21.15.190:6816/3242198843,v1:172.21.15.190:6817/3242198843] conn(0x7f30bc0bfd00 0x7f30bc0c21f0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:21.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:19.516+0000 7f30dc5d7640 1 -- 172.21.15.42:0/2830938219 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f30bc0b4bf0 msgr2=0x7f30bc0b7100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:21.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:19.516+0000 7f30dc5d7640 1 --2- 172.21.15.42:0/2830938219 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f30bc0b4bf0 0x7f30bc0b7100 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f30d4079b30 tx=0x7f30c800a2d0 comp rx=0 tx=0).stop 2024-08-30T08:59:21.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:19.516+0000 7f30dc5d7640 1 -- 172.21.15.42:0/2830938219 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f30d4102880 msgr2=0x7f30d4199130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:21.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:19.516+0000 7f30dc5d7640 1 --2- 172.21.15.42:0/2830938219 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f30d4102880 0x7f30d4199130 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f30c400ba50 tx=0x7f30c400be50 comp rx=0 tx=0).stop 2024-08-30T08:59:21.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:19.516+0000 7f30dab74640 1 -- 172.21.15.42:0/2830938219 reap_dead start 2024-08-30T08:59:21.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:19.516+0000 7f30dc5d7640 1 -- 172.21.15.42:0/2830938219 shutdown_connections 2024-08-30T08:59:21.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:19.516+0000 7f30dc5d7640 1 -- 172.21.15.42:0/2830938219 >> 172.21.15.42:0/2830938219 conn(0x7f30d4079530 msgr2=0x7f30d4106120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:59:21.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:19.516+0000 7f30dc5d7640 1 -- 172.21.15.42:0/2830938219 shutdown_connections 2024-08-30T08:59:21.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:19.516+0000 7f30dc5d7640 1 -- 172.21.15.42:0/2830938219 wait complete. 2024-08-30T08:59:21.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:59:20 smithi167 ceph-mon[22063]: pgmap v328: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-08-30T08:59:21.418 INFO:teuthology.orchestra.run.smithi042.stdout:128849018944 2024-08-30T08:59:21.418 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-08-30T08:59:21.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:20.467+0000 7fb11c4ca640 1 -- 172.21.15.42:0/2271834910 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb114108b50 msgr2=0x7fb11410f380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:21.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:20.467+0000 7fb11c4ca640 1 --2- 172.21.15.42:0/2271834910 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb114108b50 0x7fb11410f380 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fb10c009ca0 tx=0x7fb10c02a8c0 comp rx=0 tx=0).stop 2024-08-30T08:59:21.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:20.468+0000 7fb11c4ca640 1 -- 172.21.15.42:0/2271834910 shutdown_connections 2024-08-30T08:59:21.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:20.468+0000 7fb11c4ca640 1 --2- 172.21.15.42:0/2271834910 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb114108b50 0x7fb11410f380 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:21.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:20.468+0000 7fb11c4ca640 1 --2- 172.21.15.42:0/2271834910 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb114108200 0x7fb114108610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:21.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:20.468+0000 7fb11c4ca640 1 --2- 172.21.15.42:0/2271834910 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb114102f90 0x7fb1141033a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:21.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:20.468+0000 7fb11c4ca640 1 -- 172.21.15.42:0/2271834910 >> 172.21.15.42:0/2271834910 conn(0x7fb114079c60 msgr2=0x7fb11407a070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:59:21.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:20.468+0000 7fb11c4ca640 1 -- 172.21.15.42:0/2271834910 shutdown_connections 2024-08-30T08:59:21.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:20.468+0000 7fb11c4ca640 1 -- 172.21.15.42:0/2271834910 wait complete. 2024-08-30T08:59:21.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:20.468+0000 7fb11c4ca640 1 Processor -- start 2024-08-30T08:59:21.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:20.468+0000 7fb11c4ca640 1 -- start start 2024-08-30T08:59:21.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:20.469+0000 7fb11c4ca640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb114102f90 0x7fb114199210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:21.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:20.469+0000 7fb11c4ca640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb114108200 0x7fb1141997b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:21.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:20.469+0000 7fb11c4ca640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb114108b50 0x7fb11419ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:21.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:20.469+0000 7fb11c4ca640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fb114112820 con 0x7fb114108b50 2024-08-30T08:59:21.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:20.469+0000 7fb11c4ca640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb1141126a0 con 0x7fb114102f90 2024-08-30T08:59:21.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:20.469+0000 7fb11c4ca640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fb1141129a0 con 0x7fb114108200 2024-08-30T08:59:21.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:20.469+0000 7fb11aa67640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb114108b50 0x7fb11419ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:21.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:20.469+0000 7fb11aa67640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb114108b50 0x7fb11419ea50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38716/0 (socket says 172.21.15.42:38716) 2024-08-30T08:59:21.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:20.469+0000 7fb11aa67640 1 -- 172.21.15.42:0/1788458262 learned_addr learned my addr 172.21.15.42:0/1788458262 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T08:59:21.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:20.469+0000 7fb119a65640 1 --2- 172.21.15.42:0/1788458262 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb114108200 0x7fb1141997b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:21.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:20.469+0000 7fb11a266640 1 --2- 172.21.15.42:0/1788458262 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb114102f90 0x7fb114199210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:21.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:20.469+0000 7fb11aa67640 1 -- 172.21.15.42:0/1788458262 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb114108200 msgr2=0x7fb1141997b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:21.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:20.469+0000 7fb11aa67640 1 --2- 172.21.15.42:0/1788458262 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb114108200 0x7fb1141997b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:21.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:20.469+0000 7fb11aa67640 1 -- 172.21.15.42:0/1788458262 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb114102f90 msgr2=0x7fb114199210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:21.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:20.469+0000 7fb11aa67640 1 --2- 172.21.15.42:0/1788458262 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb114102f90 0x7fb114199210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:21.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:20.469+0000 7fb11aa67640 1 -- 172.21.15.42:0/1788458262 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb11419f180 con 0x7fb114108b50 2024-08-30T08:59:21.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:20.469+0000 7fb11aa67640 1 --2- 172.21.15.42:0/1788458262 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb114108b50 0x7fb11419ea50 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fb10c005cc0 tx=0x7fb10c02f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:59:21.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:20.470+0000 7fb10b7fe640 1 -- 172.21.15.42:0/1788458262 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb10c003bf0 con 0x7fb114108b50 2024-08-30T08:59:21.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:20.470+0000 7fb11c4ca640 1 -- 172.21.15.42:0/1788458262 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb11419f410 con 0x7fb114108b50 2024-08-30T08:59:21.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:20.470+0000 7fb10b7fe640 1 -- 172.21.15.42:0/1788458262 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb10c003d90 con 0x7fb114108b50 2024-08-30T08:59:21.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:20.470+0000 7fb10b7fe640 1 -- 172.21.15.42:0/1788458262 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb10c02fbc0 con 0x7fb114108b50 2024-08-30T08:59:21.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:20.470+0000 7fb11c4ca640 1 -- 172.21.15.42:0/1788458262 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1141a5bd0 con 0x7fb114108b50 2024-08-30T08:59:21.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:20.471+0000 7fb11c4ca640 1 -- 172.21.15.42:0/1788458262 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb114063870 con 0x7fb114108b50 2024-08-30T08:59:21.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:20.472+0000 7fb10b7fe640 1 -- 172.21.15.42:0/1788458262 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) ==== 152628+0+0 (secure 0 0 0) 0x7fb10c02fd60 con 0x7fb114108b50 2024-08-30T08:59:21.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:20.473+0000 7fb10b7fe640 1 --2- 172.21.15.42:0/1788458262 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fb1000b4bc0 0x7fb1000b70d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:21.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:20.473+0000 7fb11a266640 1 --2- 172.21.15.42:0/1788458262 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fb1000b4bc0 0x7fb1000b70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:21.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:20.473+0000 7fb10b7fe640 1 -- 172.21.15.42:0/1788458262 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 9092+0+0 (secure 0 0 0) 0x7fb10c106d90 con 0x7fb114108b50 2024-08-30T08:59:21.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:20.473+0000 7fb10b7fe640 1 --2- 172.21.15.42:0/1788458262 >> [v2:172.21.15.190:6808/2557145333,v1:172.21.15.190:6809/2557145333] conn(0x7fb1000bfcd0 0x7fb1000c21c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:21.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:20.473+0000 7fb10b7fe640 1 -- 172.21.15.42:0/1788458262 --> [v2:172.21.15.190:6808/2557145333,v1:172.21.15.190:6809/2557145333] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb1000c2890 con 0x7fb1000bfcd0 2024-08-30T08:59:21.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:20.473+0000 7fb10b7fe640 1 -- 172.21.15.42:0/1788458262 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7fb10c1071e0 con 0x7fb114108b50 2024-08-30T08:59:21.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:20.474+0000 7fb11a266640 1 --2- 172.21.15.42:0/1788458262 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fb1000b4bc0 0x7fb1000b70d0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fb110005fd0 tx=0x7fb110005e20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:59:21.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:20.474+0000 7fb119a65640 1 --2- 172.21.15.42:0/1788458262 >> [v2:172.21.15.190:6808/2557145333,v1:172.21.15.190:6809/2557145333] conn(0x7fb1000bfcd0 0x7fb1000c21c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:21.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:20.474+0000 7fb119a65640 1 --2- 172.21.15.42:0/1788458262 >> [v2:172.21.15.190:6808/2557145333,v1:172.21.15.190:6809/2557145333] conn(0x7fb1000bfcd0 0x7fb1000c21c0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:59:21.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:20.475+0000 7fb10b7fe640 1 -- 172.21.15.42:0/1788458262 <== osd.5 v2:172.21.15.190:6808/2557145333 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb1000c2890 con 0x7fb1000bfcd0 2024-08-30T08:59:21.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:20.490+0000 7fb11c4ca640 1 -- 172.21.15.42:0/1788458262 --> [v2:172.21.15.190:6808/2557145333,v1:172.21.15.190:6809/2557145333] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fb114063cf0 con 0x7fb1000bfcd0 2024-08-30T08:59:21.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:20.491+0000 7fb10b7fe640 1 -- 172.21.15.42:0/1788458262 <== osd.5 v2:172.21.15.190:6808/2557145333 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fb114063cf0 con 0x7fb1000bfcd0 2024-08-30T08:59:21.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:20.491+0000 7fb11c4ca640 1 -- 172.21.15.42:0/1788458262 >> [v2:172.21.15.190:6808/2557145333,v1:172.21.15.190:6809/2557145333] conn(0x7fb1000bfcd0 msgr2=0x7fb1000c21c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:21.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:20.491+0000 7fb11c4ca640 1 --2- 172.21.15.42:0/1788458262 >> [v2:172.21.15.190:6808/2557145333,v1:172.21.15.190:6809/2557145333] conn(0x7fb1000bfcd0 0x7fb1000c21c0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:21.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:20.491+0000 7fb11c4ca640 1 -- 172.21.15.42:0/1788458262 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fb1000b4bc0 msgr2=0x7fb1000b70d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:21.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:20.491+0000 7fb11c4ca640 1 --2- 172.21.15.42:0/1788458262 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fb1000b4bc0 0x7fb1000b70d0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fb110005fd0 tx=0x7fb110005e20 comp rx=0 tx=0).stop 2024-08-30T08:59:21.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:20.491+0000 7fb11c4ca640 1 -- 172.21.15.42:0/1788458262 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb114108b50 msgr2=0x7fb11419ea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:21.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:20.491+0000 7fb11c4ca640 1 --2- 172.21.15.42:0/1788458262 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb114108b50 0x7fb11419ea50 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fb10c005cc0 tx=0x7fb10c02f380 comp rx=0 tx=0).stop 2024-08-30T08:59:21.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:20.491+0000 7fb11aa67640 1 -- 172.21.15.42:0/1788458262 reap_dead start 2024-08-30T08:59:21.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:20.491+0000 7fb11c4ca640 1 -- 172.21.15.42:0/1788458262 shutdown_connections 2024-08-30T08:59:21.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:20.491+0000 7fb11c4ca640 1 -- 172.21.15.42:0/1788458262 >> 172.21.15.42:0/1788458262 conn(0x7fb114079c60 msgr2=0x7fb114104170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:59:21.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:20.491+0000 7fb11c4ca640 1 -- 172.21.15.42:0/1788458262 shutdown_connections 2024-08-30T08:59:21.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:20.492+0000 7fb11c4ca640 1 -- 172.21.15.42:0/1788458262 wait complete. 2024-08-30T08:59:22.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:22.022+0000 7f8498437640 1 -- 172.21.15.42:0/1724773205 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f84901050a0 msgr2=0x7f849010d5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:22.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:22.022+0000 7f8498437640 1 --2- 172.21.15.42:0/1724773205 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f84901050a0 0x7f849010d5f0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f8480009900 tx=0x7f848002a410 comp rx=0 tx=0).stop 2024-08-30T08:59:22.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:22.022+0000 7f8498437640 1 -- 172.21.15.42:0/1724773205 shutdown_connections 2024-08-30T08:59:22.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:22.022+0000 7f8498437640 1 --2- 172.21.15.42:0/1724773205 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f849010dbb0 0x7f84901100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:22.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:22.022+0000 7f8498437640 1 --2- 172.21.15.42:0/1724773205 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f84901050a0 0x7f849010d5f0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:22.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:22.022+0000 7f8498437640 1 --2- 172.21.15.42:0/1724773205 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8490104750 0x7f8490104b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:22.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:22.022+0000 7f8498437640 1 -- 172.21.15.42:0/1724773205 >> 172.21.15.42:0/1724773205 conn(0x7f84900794c0 msgr2=0x7f84900798d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:59:22.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:22.022+0000 7f8498437640 1 -- 172.21.15.42:0/1724773205 shutdown_connections 2024-08-30T08:59:22.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:22.022+0000 7f8498437640 1 -- 172.21.15.42:0/1724773205 wait complete. 2024-08-30T08:59:22.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:22.023+0000 7f8498437640 1 Processor -- start 2024-08-30T08:59:22.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:22.023+0000 7f8498437640 1 -- start start 2024-08-30T08:59:22.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:22.023+0000 7f8498437640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8490104750 0x7f8490102490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:22.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:22.023+0000 7f8498437640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f84901050a0 0x7f84901029d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:22.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:22.023+0000 7f8498437640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f849010dbb0 0x7f8490102f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:22.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:22.023+0000 7f8498437640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f84901137d0 con 0x7f84901050a0 2024-08-30T08:59:22.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:22.023+0000 7f8498437640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f8490113650 con 0x7f8490104750 2024-08-30T08:59:22.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:22.023+0000 7f8498437640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f8490113950 con 0x7f849010dbb0 2024-08-30T08:59:22.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:22.023+0000 7f84959d2640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f84901050a0 0x7f84901029d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:22.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:22.023+0000 7f84959d2640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f84901050a0 0x7f84901029d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38762/0 (socket says 172.21.15.42:38762) 2024-08-30T08:59:22.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:22.023+0000 7f84959d2640 1 -- 172.21.15.42:0/1500899055 learned_addr learned my addr 172.21.15.42:0/1500899055 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T08:59:22.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:22.023+0000 7f84961d3640 1 --2- 172.21.15.42:0/1500899055 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8490104750 0x7f8490102490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:22.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:22.024+0000 7f84969d4640 1 --2- 172.21.15.42:0/1500899055 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f849010dbb0 0x7f8490102f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:22.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:22.024+0000 7f84959d2640 1 -- 172.21.15.42:0/1500899055 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f849010dbb0 msgr2=0x7f8490102f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:22.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:22.024+0000 7f84959d2640 1 --2- 172.21.15.42:0/1500899055 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f849010dbb0 0x7f8490102f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:22.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:22.024+0000 7f84959d2640 1 -- 172.21.15.42:0/1500899055 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8490104750 msgr2=0x7f8490102490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:22.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:22.024+0000 7f84959d2640 1 --2- 172.21.15.42:0/1500899055 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8490104750 0x7f8490102490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:22.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:22.024+0000 7f84959d2640 1 -- 172.21.15.42:0/1500899055 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84901a8500 con 0x7f84901050a0 2024-08-30T08:59:22.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:22.024+0000 7f84961d3640 1 --2- 172.21.15.42:0/1500899055 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8490104750 0x7f8490102490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-30T08:59:22.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:22.024+0000 7f84959d2640 1 --2- 172.21.15.42:0/1500899055 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f84901050a0 0x7f84901029d0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f848002a930 tx=0x7f8480004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:59:22.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:22.024+0000 7f84877fe640 1 -- 172.21.15.42:0/1500899055 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f848003f030 con 0x7f84901050a0 2024-08-30T08:59:22.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:22.024+0000 7f84877fe640 1 -- 172.21.15.42:0/1500899055 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8480003aa0 con 0x7f84901050a0 2024-08-30T08:59:22.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:22.024+0000 7f8498437640 1 -- 172.21.15.42:0/1500899055 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84901a8790 con 0x7f84901050a0 2024-08-30T08:59:22.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:22.025+0000 7f84877fe640 1 -- 172.21.15.42:0/1500899055 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8480035070 con 0x7f84901050a0 2024-08-30T08:59:22.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:22.025+0000 7f8498437640 1 -- 172.21.15.42:0/1500899055 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84901a8c70 con 0x7f84901050a0 2024-08-30T08:59:22.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:22.026+0000 7f8498437640 1 -- 172.21.15.42:0/1500899055 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f84900638f0 con 0x7f84901050a0 2024-08-30T08:59:22.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:22.028+0000 7f84877fe640 1 -- 172.21.15.42:0/1500899055 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) ==== 152628+0+0 (secure 0 0 0) 0x7f8480042030 con 0x7f84901050a0 2024-08-30T08:59:22.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:22.030+0000 7f84877fe640 1 --2- 172.21.15.42:0/1500899055 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f847c0b4c40 0x7f847c0b7150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:22.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:22.030+0000 7f84961d3640 1 --2- 172.21.15.42:0/1500899055 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f847c0b4c40 0x7f847c0b7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:22.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:22.030+0000 7f84877fe640 1 -- 172.21.15.42:0/1500899055 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 9092+0+0 (secure 0 0 0) 0x7f84801060e0 con 0x7f84901050a0 2024-08-30T08:59:22.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:22.030+0000 7f84877fe640 1 --2- 172.21.15.42:0/1500899055 >> [v2:172.21.15.167:6800/2791960082,v1:172.21.15.167:6801/2791960082] conn(0x7f847c0bfd50 0x7f847c0c2240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:22.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:22.030+0000 7f84877fe640 1 -- 172.21.15.42:0/1500899055 --> [v2:172.21.15.167:6800/2791960082,v1:172.21.15.167:6801/2791960082] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f847c0c2910 con 0x7f847c0bfd50 2024-08-30T08:59:22.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:22.030+0000 7f84877fe640 1 -- 172.21.15.42:0/1500899055 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7f8480106530 con 0x7f84901050a0 2024-08-30T08:59:22.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:22.030+0000 7f84969d4640 1 --2- 172.21.15.42:0/1500899055 >> [v2:172.21.15.167:6800/2791960082,v1:172.21.15.167:6801/2791960082] conn(0x7f847c0bfd50 0x7f847c0c2240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:22.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:22.031+0000 7f84961d3640 1 --2- 172.21.15.42:0/1500899055 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f847c0b4c40 0x7f847c0b7150 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f8490103ee0 tx=0x7f848c00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:59:22.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:22.031+0000 7f84969d4640 1 --2- 172.21.15.42:0/1500899055 >> [v2:172.21.15.167:6800/2791960082,v1:172.21.15.167:6801/2791960082] conn(0x7f847c0bfd50 0x7f847c0c2240 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:59:22.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:22.032+0000 7f84877fe640 1 -- 172.21.15.42:0/1500899055 <== osd.1 v2:172.21.15.167:6800/2791960082 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f847c0c2910 con 0x7f847c0bfd50 2024-08-30T08:59:22.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:22.055+0000 7f8498437640 1 -- 172.21.15.42:0/1500899055 --> [v2:172.21.15.167:6800/2791960082,v1:172.21.15.167:6801/2791960082] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f8490063b00 con 0x7f847c0bfd50 2024-08-30T08:59:22.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:22.056+0000 7f84877fe640 1 -- 172.21.15.42:0/1500899055 <== osd.1 v2:172.21.15.167:6800/2791960082 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f8490063b00 con 0x7f847c0bfd50 2024-08-30T08:59:22.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:22.056+0000 7f8498437640 1 -- 172.21.15.42:0/1500899055 >> [v2:172.21.15.167:6800/2791960082,v1:172.21.15.167:6801/2791960082] conn(0x7f847c0bfd50 msgr2=0x7f847c0c2240 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:22.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:22.056+0000 7f8498437640 1 --2- 172.21.15.42:0/1500899055 >> [v2:172.21.15.167:6800/2791960082,v1:172.21.15.167:6801/2791960082] conn(0x7f847c0bfd50 0x7f847c0c2240 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:22.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:22.056+0000 7f8498437640 1 -- 172.21.15.42:0/1500899055 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f847c0b4c40 msgr2=0x7f847c0b7150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:22.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:22.056+0000 7f8498437640 1 --2- 172.21.15.42:0/1500899055 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f847c0b4c40 0x7f847c0b7150 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f8490103ee0 tx=0x7f848c00b410 comp rx=0 tx=0).stop 2024-08-30T08:59:22.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:22.056+0000 7f8498437640 1 -- 172.21.15.42:0/1500899055 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f84901050a0 msgr2=0x7f84901029d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:22.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:22.056+0000 7f8498437640 1 --2- 172.21.15.42:0/1500899055 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f84901050a0 0x7f84901029d0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f848002a930 tx=0x7f8480004270 comp rx=0 tx=0).stop 2024-08-30T08:59:22.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:22.056+0000 7f84969d4640 1 -- 172.21.15.42:0/1500899055 reap_dead start 2024-08-30T08:59:22.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:22.057+0000 7f8498437640 1 -- 172.21.15.42:0/1500899055 shutdown_connections 2024-08-30T08:59:22.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:22.057+0000 7f8498437640 1 -- 172.21.15.42:0/1500899055 >> 172.21.15.42:0/1500899055 conn(0x7f84900794c0 msgr2=0x7f8490100610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:59:22.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:22.057+0000 7f8498437640 1 -- 172.21.15.42:0/1500899055 shutdown_connections 2024-08-30T08:59:22.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:22.057+0000 7f8498437640 1 -- 172.21.15.42:0/1500899055 wait complete. 2024-08-30T08:59:22.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:59:22 smithi042 ceph-mon[18652]: pgmap v329: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 682 B/s wr, 5 op/s 2024-08-30T08:59:22.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:59:22 smithi190 ceph-mon[22001]: pgmap v329: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 682 B/s wr, 5 op/s 2024-08-30T08:59:23.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:59:22 smithi167 ceph-mon[22063]: pgmap v329: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 682 B/s wr, 5 op/s 2024-08-30T08:59:23.206 INFO:teuthology.orchestra.run.smithi042.stdout:219043332140 2024-08-30T08:59:23.206 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-08-30T08:59:23.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:22.662+0000 7fcd749a6640 1 -- 172.21.15.42:0/4033276650 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcd70102800 msgr2=0x7fcd70102c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:23.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:22.662+0000 7fcd749a6640 1 --2- 172.21.15.42:0/4033276650 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcd70102800 0x7fcd70102c10 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fcd60009900 tx=0x7fcd6002a410 comp rx=0 tx=0).stop 2024-08-30T08:59:23.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:22.663+0000 7fcd749a6640 1 -- 172.21.15.42:0/4033276650 shutdown_connections 2024-08-30T08:59:23.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:22.663+0000 7fcd749a6640 1 --2- 172.21.15.42:0/4033276650 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcd70108140 0x7fcd70108550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:23.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:22.663+0000 7fcd749a6640 1 --2- 172.21.15.42:0/4033276650 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcd70108a90 0x7fcd7010f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:23.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:22.663+0000 7fcd749a6640 1 --2- 172.21.15.42:0/4033276650 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcd70102800 0x7fcd70102c10 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:23.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:22.663+0000 7fcd749a6640 1 -- 172.21.15.42:0/4033276650 >> 172.21.15.42:0/4033276650 conn(0x7fcd70079500 msgr2=0x7fcd70079910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:59:23.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:22.663+0000 7fcd749a6640 1 -- 172.21.15.42:0/4033276650 shutdown_connections 2024-08-30T08:59:23.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:22.663+0000 7fcd749a6640 1 -- 172.21.15.42:0/4033276650 wait complete. 2024-08-30T08:59:23.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:22.663+0000 7fcd749a6640 1 Processor -- start 2024-08-30T08:59:23.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:22.664+0000 7fcd749a6640 1 -- start start 2024-08-30T08:59:23.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:22.664+0000 7fcd749a6640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcd70102800 0x7fcd701991d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:23.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:22.664+0000 7fcd749a6640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcd70108140 0x7fcd70199770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:23.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:22.664+0000 7fcd749a6640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcd70108a90 0x7fcd7019ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:23.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:22.664+0000 7fcd749a6640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fcd70112760 con 0x7fcd70108140 2024-08-30T08:59:23.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:22.664+0000 7fcd749a6640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fcd701125e0 con 0x7fcd70102800 2024-08-30T08:59:23.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:22.664+0000 7fcd749a6640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fcd701128e0 con 0x7fcd70108a90 2024-08-30T08:59:23.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:22.664+0000 7fcd67fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcd70108140 0x7fcd70199770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:23.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:22.664+0000 7fcd67fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcd70108140 0x7fcd70199770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38782/0 (socket says 172.21.15.42:38782) 2024-08-30T08:59:23.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:22.664+0000 7fcd67fff640 1 -- 172.21.15.42:0/2772131736 learned_addr learned my addr 172.21.15.42:0/2772131736 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T08:59:23.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:22.664+0000 7fcd6e59c640 1 --2- 172.21.15.42:0/2772131736 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcd70102800 0x7fcd701991d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:23.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:22.664+0000 7fcd6ed9d640 1 --2- 172.21.15.42:0/2772131736 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcd70108a90 0x7fcd7019ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:23.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:22.664+0000 7fcd67fff640 1 -- 172.21.15.42:0/2772131736 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcd70108a90 msgr2=0x7fcd7019ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:23.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:22.664+0000 7fcd67fff640 1 --2- 172.21.15.42:0/2772131736 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcd70108a90 0x7fcd7019ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:23.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:22.664+0000 7fcd67fff640 1 -- 172.21.15.42:0/2772131736 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcd70102800 msgr2=0x7fcd701991d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:23.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:22.664+0000 7fcd67fff640 1 --2- 172.21.15.42:0/2772131736 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcd70102800 0x7fcd701991d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:23.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:22.664+0000 7fcd67fff640 1 -- 172.21.15.42:0/2772131736 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd7019f140 con 0x7fcd70108140 2024-08-30T08:59:23.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:22.664+0000 7fcd6ed9d640 1 --2- 172.21.15.42:0/2772131736 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcd70108a90 0x7fcd7019ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-30T08:59:23.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:22.665+0000 7fcd67fff640 1 --2- 172.21.15.42:0/2772131736 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcd70108140 0x7fcd70199770 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fcd5800e9c0 tx=0x7fcd5800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:59:23.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:22.665+0000 7fcd677fe640 1 -- 172.21.15.42:0/2772131736 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd5800ccc0 con 0x7fcd70108140 2024-08-30T08:59:23.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:22.665+0000 7fcd677fe640 1 -- 172.21.15.42:0/2772131736 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcd580054d0 con 0x7fcd70108140 2024-08-30T08:59:23.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:22.665+0000 7fcd749a6640 1 -- 172.21.15.42:0/2772131736 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd7019f430 con 0x7fcd70108140 2024-08-30T08:59:23.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:22.665+0000 7fcd677fe640 1 -- 172.21.15.42:0/2772131736 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd58010430 con 0x7fcd70108140 2024-08-30T08:59:23.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:22.665+0000 7fcd749a6640 1 -- 172.21.15.42:0/2772131736 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd701a5b90 con 0x7fcd70108140 2024-08-30T08:59:23.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:22.666+0000 7fcd749a6640 1 -- 172.21.15.42:0/2772131736 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fcd700638b0 con 0x7fcd70108140 2024-08-30T08:59:23.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:22.668+0000 7fcd677fe640 1 -- 172.21.15.42:0/2772131736 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) ==== 152628+0+0 (secure 0 0 0) 0x7fcd58005000 con 0x7fcd70108140 2024-08-30T08:59:23.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:22.668+0000 7fcd677fe640 1 --2- 172.21.15.42:0/2772131736 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fcd5c0b4c40 0x7fcd5c0b7150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:23.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:22.669+0000 7fcd677fe640 1 -- 172.21.15.42:0/2772131736 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 9092+0+0 (secure 0 0 0) 0x7fcd580e5d90 con 0x7fcd70108140 2024-08-30T08:59:23.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:22.669+0000 7fcd6e59c640 1 --2- 172.21.15.42:0/2772131736 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fcd5c0b4c40 0x7fcd5c0b7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:23.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:22.669+0000 7fcd677fe640 1 --2- 172.21.15.42:0/2772131736 >> [v2:172.21.15.167:6824/3451198319,v1:172.21.15.167:6825/3451198319] conn(0x7fcd5c0bfd50 0x7fcd5c0c2240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:23.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:22.669+0000 7fcd677fe640 1 -- 172.21.15.42:0/2772131736 --> [v2:172.21.15.167:6824/3451198319,v1:172.21.15.167:6825/3451198319] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fcd58011e50 con 0x7fcd5c0bfd50 2024-08-30T08:59:23.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:22.669+0000 7fcd677fe640 1 -- 172.21.15.42:0/2772131736 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7fcd580e61e0 con 0x7fcd70108140 2024-08-30T08:59:23.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:22.669+0000 7fcd6e59c640 1 --2- 172.21.15.42:0/2772131736 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fcd5c0b4c40 0x7fcd5c0b7150 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fcd60009230 tx=0x7fcd60000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:59:23.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:22.669+0000 7fcd6ed9d640 1 --2- 172.21.15.42:0/2772131736 >> [v2:172.21.15.167:6824/3451198319,v1:172.21.15.167:6825/3451198319] conn(0x7fcd5c0bfd50 0x7fcd5c0c2240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:23.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:22.670+0000 7fcd6ed9d640 1 --2- 172.21.15.42:0/2772131736 >> [v2:172.21.15.167:6824/3451198319,v1:172.21.15.167:6825/3451198319] conn(0x7fcd5c0bfd50 0x7fcd5c0c2240 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:59:23.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:22.670+0000 7fcd677fe640 1 -- 172.21.15.42:0/2772131736 <== osd.10 v2:172.21.15.167:6824/3451198319 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fcd58011e50 con 0x7fcd5c0bfd50 2024-08-30T08:59:23.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:22.684+0000 7fcd749a6640 1 -- 172.21.15.42:0/2772131736 --> [v2:172.21.15.167:6824/3451198319,v1:172.21.15.167:6825/3451198319] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fcd70063d30 con 0x7fcd5c0bfd50 2024-08-30T08:59:23.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:22.684+0000 7fcd677fe640 1 -- 172.21.15.42:0/2772131736 <== osd.10 v2:172.21.15.167:6824/3451198319 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fcd70063d30 con 0x7fcd5c0bfd50 2024-08-30T08:59:23.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:22.685+0000 7fcd749a6640 1 -- 172.21.15.42:0/2772131736 >> [v2:172.21.15.167:6824/3451198319,v1:172.21.15.167:6825/3451198319] conn(0x7fcd5c0bfd50 msgr2=0x7fcd5c0c2240 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:23.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:22.685+0000 7fcd749a6640 1 --2- 172.21.15.42:0/2772131736 >> [v2:172.21.15.167:6824/3451198319,v1:172.21.15.167:6825/3451198319] conn(0x7fcd5c0bfd50 0x7fcd5c0c2240 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:23.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:22.685+0000 7fcd749a6640 1 -- 172.21.15.42:0/2772131736 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fcd5c0b4c40 msgr2=0x7fcd5c0b7150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:23.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:22.685+0000 7fcd749a6640 1 --2- 172.21.15.42:0/2772131736 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fcd5c0b4c40 0x7fcd5c0b7150 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fcd60009230 tx=0x7fcd60000bc0 comp rx=0 tx=0).stop 2024-08-30T08:59:23.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:22.685+0000 7fcd749a6640 1 -- 172.21.15.42:0/2772131736 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcd70108140 msgr2=0x7fcd70199770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:23.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:22.685+0000 7fcd749a6640 1 --2- 172.21.15.42:0/2772131736 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcd70108140 0x7fcd70199770 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fcd5800e9c0 tx=0x7fcd5800ee80 comp rx=0 tx=0).stop 2024-08-30T08:59:23.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:22.685+0000 7fcd6ed9d640 1 -- 172.21.15.42:0/2772131736 reap_dead start 2024-08-30T08:59:23.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:22.685+0000 7fcd749a6640 1 -- 172.21.15.42:0/2772131736 shutdown_connections 2024-08-30T08:59:23.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:22.685+0000 7fcd749a6640 1 -- 172.21.15.42:0/2772131736 >> 172.21.15.42:0/2772131736 conn(0x7fcd70079500 msgr2=0x7fcd701062f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:59:23.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:22.685+0000 7fcd749a6640 1 -- 172.21.15.42:0/2772131736 shutdown_connections 2024-08-30T08:59:23.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:22.685+0000 7fcd749a6640 1 -- 172.21.15.42:0/2772131736 wait complete. 2024-08-30T08:59:24.161 INFO:teuthology.orchestra.run.smithi042.stdout:244813135911 2024-08-30T08:59:24.162 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-08-30T08:59:24.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:23.497+0000 7ff211926640 1 -- 172.21.15.42:0/2299073835 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff20c109240 msgr2=0x7ff20c109650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:24.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:23.497+0000 7ff211926640 1 --2- 172.21.15.42:0/2299073835 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff20c109240 0x7ff20c109650 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7ff200009900 tx=0x7ff20002a410 comp rx=0 tx=0).stop 2024-08-30T08:59:24.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:23.498+0000 7ff211926640 1 -- 172.21.15.42:0/2299073835 shutdown_connections 2024-08-30T08:59:24.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:23.498+0000 7ff211926640 1 --2- 172.21.15.42:0/2299073835 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff20c109b90 0x7ff20c1103c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:24.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:23.498+0000 7ff211926640 1 --2- 172.21.15.42:0/2299073835 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff20c109240 0x7ff20c109650 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:24.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:23.498+0000 7ff211926640 1 --2- 172.21.15.42:0/2299073835 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff20c103900 0x7ff20c103d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:24.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:23.498+0000 7ff211926640 1 -- 172.21.15.42:0/2299073835 >> 172.21.15.42:0/2299073835 conn(0x7ff20c07b5f0 msgr2=0x7ff20c07ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:59:24.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:23.498+0000 7ff211926640 1 -- 172.21.15.42:0/2299073835 shutdown_connections 2024-08-30T08:59:24.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:23.498+0000 7ff211926640 1 -- 172.21.15.42:0/2299073835 wait complete. 2024-08-30T08:59:24.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:23.498+0000 7ff211926640 1 Processor -- start 2024-08-30T08:59:24.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:23.498+0000 7ff211926640 1 -- start start 2024-08-30T08:59:24.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:23.499+0000 7ff211926640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff20c103900 0x7ff20c19a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:24.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:23.499+0000 7ff211926640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff20c109240 0x7ff20c19a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:24.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:23.499+0000 7ff211926640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff20c109b90 0x7ff20c19fac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:24.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:23.499+0000 7ff211926640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7ff20c113850 con 0x7ff20c109b90 2024-08-30T08:59:24.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:23.499+0000 7ff211926640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff20c1136d0 con 0x7ff20c103900 2024-08-30T08:59:24.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:23.499+0000 7ff211926640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7ff20c1139d0 con 0x7ff20c109240 2024-08-30T08:59:24.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:23.499+0000 7ff20b7fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff20c109b90 0x7ff20c19fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:24.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:23.499+0000 7ff20b7fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff20c109b90 0x7ff20c19fac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38834/0 (socket says 172.21.15.42:38834) 2024-08-30T08:59:24.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:23.499+0000 7ff20b7fe640 1 -- 172.21.15.42:0/3293568003 learned_addr learned my addr 172.21.15.42:0/3293568003 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T08:59:24.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:23.499+0000 7ff20affd640 1 --2- 172.21.15.42:0/3293568003 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff20c103900 0x7ff20c19a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:24.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:23.499+0000 7ff20a7fc640 1 --2- 172.21.15.42:0/3293568003 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff20c109240 0x7ff20c19a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:24.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:23.499+0000 7ff20b7fe640 1 -- 172.21.15.42:0/3293568003 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff20c109240 msgr2=0x7ff20c19a820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:24.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:23.499+0000 7ff20b7fe640 1 --2- 172.21.15.42:0/3293568003 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff20c109240 0x7ff20c19a820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:24.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:23.499+0000 7ff20b7fe640 1 -- 172.21.15.42:0/3293568003 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff20c103900 msgr2=0x7ff20c19a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:24.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:23.499+0000 7ff20b7fe640 1 --2- 172.21.15.42:0/3293568003 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff20c103900 0x7ff20c19a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:24.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:23.499+0000 7ff20b7fe640 1 -- 172.21.15.42:0/3293568003 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff20c1a01f0 con 0x7ff20c109b90 2024-08-30T08:59:24.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:23.499+0000 7ff20a7fc640 1 --2- 172.21.15.42:0/3293568003 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff20c109240 0x7ff20c19a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-30T08:59:24.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:23.499+0000 7ff20b7fe640 1 --2- 172.21.15.42:0/3293568003 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff20c109b90 0x7ff20c19fac0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7ff20400eaa0 tx=0x7ff20400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:59:24.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:23.500+0000 7ff210924640 1 -- 172.21.15.42:0/3293568003 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff20400cdb0 con 0x7ff20c109b90 2024-08-30T08:59:24.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:23.500+0000 7ff211926640 1 -- 172.21.15.42:0/3293568003 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff20c1a04e0 con 0x7ff20c109b90 2024-08-30T08:59:24.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:23.500+0000 7ff210924640 1 -- 172.21.15.42:0/3293568003 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff204005020 con 0x7ff20c109b90 2024-08-30T08:59:24.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:23.500+0000 7ff210924640 1 -- 172.21.15.42:0/3293568003 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2040116d0 con 0x7ff20c109b90 2024-08-30T08:59:24.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:23.500+0000 7ff211926640 1 -- 172.21.15.42:0/3293568003 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff20c1a6c40 con 0x7ff20c109b90 2024-08-30T08:59:24.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:23.501+0000 7ff211926640 1 -- 172.21.15.42:0/3293568003 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff20c0638f0 con 0x7ff20c109b90 2024-08-30T08:59:24.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:23.502+0000 7ff210924640 1 -- 172.21.15.42:0/3293568003 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) ==== 152628+0+0 (secure 0 0 0) 0x7ff2040054a0 con 0x7ff20c109b90 2024-08-30T08:59:24.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:23.503+0000 7ff210924640 1 --2- 172.21.15.42:0/3293568003 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7ff1f40b4bc0 0x7ff1f40b70d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:24.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:23.503+0000 7ff20affd640 1 --2- 172.21.15.42:0/3293568003 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7ff1f40b4bc0 0x7ff1f40b70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:24.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:23.503+0000 7ff210924640 1 -- 172.21.15.42:0/3293568003 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 9092+0+0 (secure 0 0 0) 0x7ff2040e5c30 con 0x7ff20c109b90 2024-08-30T08:59:24.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:23.503+0000 7ff210924640 1 --2- 172.21.15.42:0/3293568003 >> [v2:172.21.15.42:6810/1135599916,v1:172.21.15.42:6811/1135599916] conn(0x7ff1f40bfcd0 0x7ff1f40c21c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:24.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:23.503+0000 7ff210924640 1 -- 172.21.15.42:0/3293568003 --> [v2:172.21.15.42:6810/1135599916,v1:172.21.15.42:6811/1135599916] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff204014ec0 con 0x7ff1f40bfcd0 2024-08-30T08:59:24.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:23.503+0000 7ff210924640 1 -- 172.21.15.42:0/3293568003 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7ff2040e5fe0 con 0x7ff20c109b90 2024-08-30T08:59:24.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:23.503+0000 7ff20a7fc640 1 --2- 172.21.15.42:0/3293568003 >> [v2:172.21.15.42:6810/1135599916,v1:172.21.15.42:6811/1135599916] conn(0x7ff1f40bfcd0 0x7ff1f40c21c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:24.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:23.503+0000 7ff20affd640 1 --2- 172.21.15.42:0/3293568003 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7ff1f40b4bc0 0x7ff1f40b70d0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7ff1fc00bbe0 tx=0x7ff1fc00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:59:24.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:23.504+0000 7ff20a7fc640 1 --2- 172.21.15.42:0/3293568003 >> [v2:172.21.15.42:6810/1135599916,v1:172.21.15.42:6811/1135599916] conn(0x7ff1f40bfcd0 0x7ff1f40c21c0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:59:24.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:23.504+0000 7ff210924640 1 -- 172.21.15.42:0/3293568003 <== osd.3 v2:172.21.15.42:6810/1135599916 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff204014ec0 con 0x7ff1f40bfcd0 2024-08-30T08:59:24.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:23.518+0000 7ff211926640 1 -- 172.21.15.42:0/3293568003 --> [v2:172.21.15.42:6810/1135599916,v1:172.21.15.42:6811/1135599916] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ff20c063da0 con 0x7ff1f40bfcd0 2024-08-30T08:59:24.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:23.518+0000 7ff210924640 1 -- 172.21.15.42:0/3293568003 <== osd.3 v2:172.21.15.42:6810/1135599916 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7ff20c063da0 con 0x7ff1f40bfcd0 2024-08-30T08:59:24.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:23.519+0000 7ff211926640 1 -- 172.21.15.42:0/3293568003 >> [v2:172.21.15.42:6810/1135599916,v1:172.21.15.42:6811/1135599916] conn(0x7ff1f40bfcd0 msgr2=0x7ff1f40c21c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:24.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:23.519+0000 7ff211926640 1 --2- 172.21.15.42:0/3293568003 >> [v2:172.21.15.42:6810/1135599916,v1:172.21.15.42:6811/1135599916] conn(0x7ff1f40bfcd0 0x7ff1f40c21c0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:24.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:23.519+0000 7ff211926640 1 -- 172.21.15.42:0/3293568003 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7ff1f40b4bc0 msgr2=0x7ff1f40b70d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:24.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:23.519+0000 7ff211926640 1 --2- 172.21.15.42:0/3293568003 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7ff1f40b4bc0 0x7ff1f40b70d0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7ff1fc00bbe0 tx=0x7ff1fc00b390 comp rx=0 tx=0).stop 2024-08-30T08:59:24.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:23.519+0000 7ff211926640 1 -- 172.21.15.42:0/3293568003 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff20c109b90 msgr2=0x7ff20c19fac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:24.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:23.519+0000 7ff211926640 1 --2- 172.21.15.42:0/3293568003 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff20c109b90 0x7ff20c19fac0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7ff20400eaa0 tx=0x7ff20400ef60 comp rx=0 tx=0).stop 2024-08-30T08:59:24.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:23.519+0000 7ff20b7fe640 1 -- 172.21.15.42:0/3293568003 reap_dead start 2024-08-30T08:59:24.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:23.519+0000 7ff211926640 1 -- 172.21.15.42:0/3293568003 shutdown_connections 2024-08-30T08:59:24.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:23.519+0000 7ff211926640 1 -- 172.21.15.42:0/3293568003 >> 172.21.15.42:0/3293568003 conn(0x7ff20c07b5f0 msgr2=0x7ff20c107340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:59:24.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:23.519+0000 7ff211926640 1 -- 172.21.15.42:0/3293568003 shutdown_connections 2024-08-30T08:59:24.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:23.519+0000 7ff211926640 1 -- 172.21.15.42:0/3293568003 wait complete. 2024-08-30T08:59:24.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:23.414+0000 7f1e1c877640 1 -- 172.21.15.42:0/3889577029 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1e18103940 msgr2=0x7f1e18103d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:24.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:23.414+0000 7f1e1c877640 1 --2- 172.21.15.42:0/3889577029 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1e18103940 0x7f1e18103d50 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f1e00009900 tx=0x7f1e0002a410 comp rx=0 tx=0).stop 2024-08-30T08:59:24.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:23.415+0000 7f1e1c877640 1 -- 172.21.15.42:0/3889577029 shutdown_connections 2024-08-30T08:59:24.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:23.415+0000 7f1e1c877640 1 --2- 172.21.15.42:0/3889577029 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1e18109bd0 0x7f1e18110400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:24.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:23.415+0000 7f1e1c877640 1 --2- 172.21.15.42:0/3889577029 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1e18109280 0x7f1e18109690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:24.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:23.415+0000 7f1e1c877640 1 --2- 172.21.15.42:0/3889577029 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1e18103940 0x7f1e18103d50 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:24.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:23.415+0000 7f1e1c877640 1 -- 172.21.15.42:0/3889577029 >> 172.21.15.42:0/3889577029 conn(0x7f1e180ff3d0 msgr2=0x7f1e18101820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:59:24.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:23.415+0000 7f1e1c877640 1 -- 172.21.15.42:0/3889577029 shutdown_connections 2024-08-30T08:59:24.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:23.415+0000 7f1e1c877640 1 -- 172.21.15.42:0/3889577029 wait complete. 2024-08-30T08:59:24.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:23.416+0000 7f1e1c877640 1 Processor -- start 2024-08-30T08:59:24.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:23.416+0000 7f1e1c877640 1 -- start start 2024-08-30T08:59:24.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:23.416+0000 7f1e1c877640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1e18103940 0x7f1e1819a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:24.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:23.416+0000 7f1e1c877640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1e18109280 0x7f1e1819a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:24.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:23.416+0000 7f1e1c877640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1e18109bd0 0x7f1e1819faa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:24.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:23.416+0000 7f1e1c877640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f1e18113890 con 0x7f1e18109bd0 2024-08-30T08:59:24.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:23.416+0000 7f1e1c877640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f1e18113710 con 0x7f1e18109280 2024-08-30T08:59:24.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:23.416+0000 7f1e1c877640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f1e18113a10 con 0x7f1e18103940 2024-08-30T08:59:24.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:23.416+0000 7f1e16d9d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1e18109bd0 0x7f1e1819faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:24.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:23.416+0000 7f1e16d9d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1e18109bd0 0x7f1e1819faa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38810/0 (socket says 172.21.15.42:38810) 2024-08-30T08:59:24.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:23.416+0000 7f1e16d9d640 1 -- 172.21.15.42:0/4006915161 learned_addr learned my addr 172.21.15.42:0/4006915161 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T08:59:24.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:23.416+0000 7f1e1659c640 1 --2- 172.21.15.42:0/4006915161 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1e18103940 0x7f1e1819a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:24.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:23.416+0000 7f1e15d9b640 1 --2- 172.21.15.42:0/4006915161 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1e18109280 0x7f1e1819a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:24.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:23.416+0000 7f1e16d9d640 1 -- 172.21.15.42:0/4006915161 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1e18103940 msgr2=0x7f1e1819a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:24.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:23.417+0000 7f1e16d9d640 1 --2- 172.21.15.42:0/4006915161 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1e18103940 0x7f1e1819a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:24.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:23.417+0000 7f1e16d9d640 1 -- 172.21.15.42:0/4006915161 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1e18109280 msgr2=0x7f1e1819a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:24.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:23.417+0000 7f1e16d9d640 1 --2- 172.21.15.42:0/4006915161 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1e18109280 0x7f1e1819a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:24.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:23.417+0000 7f1e16d9d640 1 -- 172.21.15.42:0/4006915161 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e181a01d0 con 0x7f1e18109bd0 2024-08-30T08:59:24.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:23.417+0000 7f1e16d9d640 1 --2- 172.21.15.42:0/4006915161 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1e18109bd0 0x7f1e1819faa0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f1e1000eaa0 tx=0x7f1e1000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:59:24.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:23.417+0000 7f1e0f7fe640 1 -- 172.21.15.42:0/4006915161 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e1000cdb0 con 0x7f1e18109bd0 2024-08-30T08:59:24.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:23.417+0000 7f1e0f7fe640 1 -- 172.21.15.42:0/4006915161 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1e10005020 con 0x7f1e18109bd0 2024-08-30T08:59:24.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:23.417+0000 7f1e1c877640 1 -- 172.21.15.42:0/4006915161 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e181a04c0 con 0x7f1e18109bd0 2024-08-30T08:59:24.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:23.417+0000 7f1e0f7fe640 1 -- 172.21.15.42:0/4006915161 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e100116d0 con 0x7f1e18109bd0 2024-08-30T08:59:24.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:23.417+0000 7f1e1c877640 1 -- 172.21.15.42:0/4006915161 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e181a6c20 con 0x7f1e18109bd0 2024-08-30T08:59:24.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:23.418+0000 7f1e1c877640 1 -- 172.21.15.42:0/4006915161 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1e180638b0 con 0x7f1e18109bd0 2024-08-30T08:59:24.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:23.420+0000 7f1e0f7fe640 1 -- 172.21.15.42:0/4006915161 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) ==== 152628+0+0 (secure 0 0 0) 0x7f1e100051d0 con 0x7f1e18109bd0 2024-08-30T08:59:24.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:23.420+0000 7f1e0f7fe640 1 --2- 172.21.15.42:0/4006915161 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f1e040b4bc0 0x7f1e040b70d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:24.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:23.421+0000 7f1e0f7fe640 1 -- 172.21.15.42:0/4006915161 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 9092+0+0 (secure 0 0 0) 0x7f1e100e5c80 con 0x7f1e18109bd0 2024-08-30T08:59:24.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:23.421+0000 7f1e1659c640 1 --2- 172.21.15.42:0/4006915161 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f1e040b4bc0 0x7f1e040b70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:24.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:23.421+0000 7f1e0f7fe640 1 --2- 172.21.15.42:0/4006915161 >> [v2:172.21.15.167:6816/2962799814,v1:172.21.15.167:6817/2962799814] conn(0x7f1e040bfcd0 0x7f1e040c21c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:24.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:23.421+0000 7f1e0f7fe640 1 -- 172.21.15.42:0/4006915161 --> [v2:172.21.15.167:6816/2962799814,v1:172.21.15.167:6817/2962799814] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1e10014ec0 con 0x7f1e040bfcd0 2024-08-30T08:59:24.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:23.421+0000 7f1e0f7fe640 1 -- 172.21.15.42:0/4006915161 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7f1e100e6030 con 0x7f1e18109bd0 2024-08-30T08:59:24.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:23.421+0000 7f1e1659c640 1 --2- 172.21.15.42:0/4006915161 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f1e040b4bc0 0x7f1e040b70d0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f1e00031440 tx=0x7f1e00009560 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:59:24.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:23.421+0000 7f1e15d9b640 1 --2- 172.21.15.42:0/4006915161 >> [v2:172.21.15.167:6816/2962799814,v1:172.21.15.167:6817/2962799814] conn(0x7f1e040bfcd0 0x7f1e040c21c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:24.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:23.422+0000 7f1e15d9b640 1 --2- 172.21.15.42:0/4006915161 >> [v2:172.21.15.167:6816/2962799814,v1:172.21.15.167:6817/2962799814] conn(0x7f1e040bfcd0 0x7f1e040c21c0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:59:24.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:23.423+0000 7f1e0f7fe640 1 -- 172.21.15.42:0/4006915161 <== osd.7 v2:172.21.15.167:6816/2962799814 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1e10014ec0 con 0x7f1e040bfcd0 2024-08-30T08:59:24.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:23.436+0000 7f1e1c877640 1 -- 172.21.15.42:0/4006915161 --> [v2:172.21.15.167:6816/2962799814,v1:172.21.15.167:6817/2962799814] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f1e18063d30 con 0x7f1e040bfcd0 2024-08-30T08:59:24.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:23.437+0000 7f1e0f7fe640 1 -- 172.21.15.42:0/4006915161 <== osd.7 v2:172.21.15.167:6816/2962799814 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f1e18063d30 con 0x7f1e040bfcd0 2024-08-30T08:59:24.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:23.437+0000 7f1e1c877640 1 -- 172.21.15.42:0/4006915161 >> [v2:172.21.15.167:6816/2962799814,v1:172.21.15.167:6817/2962799814] conn(0x7f1e040bfcd0 msgr2=0x7f1e040c21c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:24.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:23.437+0000 7f1e1c877640 1 --2- 172.21.15.42:0/4006915161 >> [v2:172.21.15.167:6816/2962799814,v1:172.21.15.167:6817/2962799814] conn(0x7f1e040bfcd0 0x7f1e040c21c0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:24.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:23.438+0000 7f1e1c877640 1 -- 172.21.15.42:0/4006915161 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f1e040b4bc0 msgr2=0x7f1e040b70d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:24.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:23.438+0000 7f1e1c877640 1 --2- 172.21.15.42:0/4006915161 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f1e040b4bc0 0x7f1e040b70d0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f1e00031440 tx=0x7f1e00009560 comp rx=0 tx=0).stop 2024-08-30T08:59:24.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:23.438+0000 7f1e1c877640 1 -- 172.21.15.42:0/4006915161 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1e18109bd0 msgr2=0x7f1e1819faa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:24.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:23.438+0000 7f1e1c877640 1 --2- 172.21.15.42:0/4006915161 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1e18109bd0 0x7f1e1819faa0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f1e1000eaa0 tx=0x7f1e1000ef60 comp rx=0 tx=0).stop 2024-08-30T08:59:24.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:23.438+0000 7f1e16d9d640 1 -- 172.21.15.42:0/4006915161 reap_dead start 2024-08-30T08:59:24.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:23.438+0000 7f1e1c877640 1 -- 172.21.15.42:0/4006915161 shutdown_connections 2024-08-30T08:59:24.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:23.438+0000 7f1e1c877640 1 -- 172.21.15.42:0/4006915161 >> 172.21.15.42:0/4006915161 conn(0x7f1e180ff3d0 msgr2=0x7f1e181017f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:59:24.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:23.438+0000 7f1e1c877640 1 -- 172.21.15.42:0/4006915161 shutdown_connections 2024-08-30T08:59:24.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:23.438+0000 7f1e1c877640 1 -- 172.21.15.42:0/4006915161 wait complete. 2024-08-30T08:59:24.826 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:59:24 smithi042 ceph-mon[18652]: pgmap v330: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 682 B/s wr, 7 op/s 2024-08-30T08:59:24.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:59:24 smithi190 ceph-mon[22001]: pgmap v330: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 682 B/s wr, 7 op/s 2024-08-30T08:59:25.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:59:24 smithi167 ceph-mon[22063]: pgmap v330: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 682 B/s wr, 7 op/s 2024-08-30T08:59:25.194 INFO:teuthology.orchestra.run.smithi042.stdout:261993005090 2024-08-30T08:59:25.195 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-08-30T08:59:25.834 INFO:teuthology.orchestra.run.smithi042.stdout:55834574927 2024-08-30T08:59:25.834 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-08-30T08:59:26.264 INFO:teuthology.orchestra.run.smithi042.stdout:103079215173 2024-08-30T08:59:26.265 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-08-30T08:59:26.293 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T08:59:26.741 INFO:teuthology.orchestra.run.smithi042.stdout:150323855419 2024-08-30T08:59:26.742 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-08-30T08:59:26.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:59:26 smithi042 ceph-mon[18652]: pgmap v331: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-08-30T08:59:26.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:59:26 smithi190 ceph-mon[22001]: pgmap v331: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-08-30T08:59:27.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:59:26 smithi167 ceph-mon[22063]: pgmap v331: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-08-30T08:59:27.179 INFO:teuthology.orchestra.run.smithi042.stdout:197568495666 2024-08-30T08:59:27.180 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-08-30T08:59:27.433 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T08:59:27.433 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T08:59:27.438 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T08:59:27.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:27.666+0000 7fa466153640 1 -- 172.21.15.42:0/3418175436 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa460108a90 msgr2=0x7fa46010f2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:27.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:27.666+0000 7fa466153640 1 --2- 172.21.15.42:0/3418175436 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa460108a90 0x7fa46010f2c0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fa458009900 tx=0x7fa45802a410 comp rx=0 tx=0).stop 2024-08-30T08:59:27.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:27.666+0000 7fa466153640 1 -- 172.21.15.42:0/3418175436 shutdown_connections 2024-08-30T08:59:27.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:27.666+0000 7fa466153640 1 --2- 172.21.15.42:0/3418175436 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa460108140 0x7fa460108550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:27.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:27.666+0000 7fa466153640 1 --2- 172.21.15.42:0/3418175436 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa460108a90 0x7fa46010f2c0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:27.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:27.666+0000 7fa466153640 1 --2- 172.21.15.42:0/3418175436 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa460102800 0x7fa460102c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:27.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:27.666+0000 7fa466153640 1 -- 172.21.15.42:0/3418175436 >> 172.21.15.42:0/3418175436 conn(0x7fa4600794d0 msgr2=0x7fa4600798e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:59:27.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:27.666+0000 7fa466153640 1 -- 172.21.15.42:0/3418175436 shutdown_connections 2024-08-30T08:59:27.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:27.666+0000 7fa466153640 1 -- 172.21.15.42:0/3418175436 wait complete. 2024-08-30T08:59:27.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:27.667+0000 7fa466153640 1 Processor -- start 2024-08-30T08:59:27.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:27.667+0000 7fa466153640 1 -- start start 2024-08-30T08:59:27.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:27.667+0000 7fa466153640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa460102800 0x7fa46019a190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:27.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:27.667+0000 7fa466153640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa460108140 0x7fa46019a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:27.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:27.667+0000 7fa466153640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa460108a90 0x7fa46019f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:27.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:27.667+0000 7fa466153640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fa460112710 con 0x7fa460102800 2024-08-30T08:59:27.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:27.667+0000 7fa466153640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa460112590 con 0x7fa460108140 2024-08-30T08:59:27.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:27.667+0000 7fa466153640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fa460112890 con 0x7fa460108a90 2024-08-30T08:59:27.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:27.667+0000 7fa45f7fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa460102800 0x7fa46019a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:27.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:27.667+0000 7fa45f7fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa460102800 0x7fa46019a190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38850/0 (socket says 172.21.15.42:38850) 2024-08-30T08:59:27.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:27.667+0000 7fa45f7fe640 1 -- 172.21.15.42:0/2419607491 learned_addr learned my addr 172.21.15.42:0/2419607491 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T08:59:27.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:27.667+0000 7fa45effd640 1 --2- 172.21.15.42:0/2419607491 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa460108140 0x7fa46019a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:27.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:27.667+0000 7fa45ffff640 1 --2- 172.21.15.42:0/2419607491 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa460108a90 0x7fa46019f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:27.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:27.667+0000 7fa45f7fe640 1 -- 172.21.15.42:0/2419607491 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa460108a90 msgr2=0x7fa46019f9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:27.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:27.667+0000 7fa45f7fe640 1 --2- 172.21.15.42:0/2419607491 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa460108a90 0x7fa46019f9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:27.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:27.668+0000 7fa45f7fe640 1 -- 172.21.15.42:0/2419607491 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa460108140 msgr2=0x7fa46019a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:27.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:27.668+0000 7fa45f7fe640 1 --2- 172.21.15.42:0/2419607491 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa460108140 0x7fa46019a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:27.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:27.668+0000 7fa45f7fe640 1 -- 172.21.15.42:0/2419607491 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4601a0100 con 0x7fa460102800 2024-08-30T08:59:27.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:27.668+0000 7fa45ffff640 1 --2- 172.21.15.42:0/2419607491 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa460108a90 0x7fa46019f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-30T08:59:27.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:27.668+0000 7fa45effd640 1 --2- 172.21.15.42:0/2419607491 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa460108140 0x7fa46019a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-30T08:59:27.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:27.668+0000 7fa45f7fe640 1 --2- 172.21.15.42:0/2419607491 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa460102800 0x7fa46019a190 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fa45000cce0 tx=0x7fa45000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:59:27.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:27.668+0000 7fa45cff9640 1 -- 172.21.15.42:0/2419607491 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa450004b20 con 0x7fa460102800 2024-08-30T08:59:27.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:27.668+0000 7fa45cff9640 1 -- 172.21.15.42:0/2419607491 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa450004cc0 con 0x7fa460102800 2024-08-30T08:59:27.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:27.668+0000 7fa45cff9640 1 -- 172.21.15.42:0/2419607491 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa45000f6b0 con 0x7fa460102800 2024-08-30T08:59:27.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:27.668+0000 7fa466153640 1 -- 172.21.15.42:0/2419607491 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4601a03f0 con 0x7fa460102800 2024-08-30T08:59:27.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:27.668+0000 7fa466153640 1 -- 172.21.15.42:0/2419607491 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4601a6b50 con 0x7fa460102800 2024-08-30T08:59:27.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:27.669+0000 7fa466153640 1 -- 172.21.15.42:0/2419607491 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa460104930 con 0x7fa460102800 2024-08-30T08:59:27.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:27.671+0000 7fa45cff9640 1 -- 172.21.15.42:0/2419607491 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) ==== 152628+0+0 (secure 0 0 0) 0x7fa45000f850 con 0x7fa460102800 2024-08-30T08:59:27.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:27.671+0000 7fa45cff9640 1 --2- 172.21.15.42:0/2419607491 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fa44c0b4bc0 0x7fa44c0b70d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:27.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:27.672+0000 7fa45cff9640 1 -- 172.21.15.42:0/2419607491 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 9092+0+0 (secure 0 0 0) 0x7fa4500e5b50 con 0x7fa460102800 2024-08-30T08:59:27.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:27.672+0000 7fa45effd640 1 --2- 172.21.15.42:0/2419607491 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fa44c0b4bc0 0x7fa44c0b70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:27.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:27.672+0000 7fa45cff9640 1 --2- 172.21.15.42:0/2419607491 >> [v2:172.21.15.190:6824/380064457,v1:172.21.15.190:6825/380064457] conn(0x7fa44c0bfcd0 0x7fa44c0c21c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:27.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:27.672+0000 7fa45cff9640 1 -- 172.21.15.42:0/2419607491 --> [v2:172.21.15.190:6824/380064457,v1:172.21.15.190:6825/380064457] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa44c0c2890 con 0x7fa44c0bfcd0 2024-08-30T08:59:27.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:27.672+0000 7fa45cff9640 1 -- 172.21.15.42:0/2419607491 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7fa4500e5fa0 con 0x7fa460102800 2024-08-30T08:59:27.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:27.672+0000 7fa45effd640 1 --2- 172.21.15.42:0/2419607491 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fa44c0b4bc0 0x7fa44c0b70d0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fa460079ad0 tx=0x7fa448005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:59:27.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:27.672+0000 7fa45ffff640 1 --2- 172.21.15.42:0/2419607491 >> [v2:172.21.15.190:6824/380064457,v1:172.21.15.190:6825/380064457] conn(0x7fa44c0bfcd0 0x7fa44c0c21c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:27.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:27.673+0000 7fa45ffff640 1 --2- 172.21.15.42:0/2419607491 >> [v2:172.21.15.190:6824/380064457,v1:172.21.15.190:6825/380064457] conn(0x7fa44c0bfcd0 0x7fa44c0c21c0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:59:27.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:27.674+0000 7fa45cff9640 1 -- 172.21.15.42:0/2419607491 <== osd.11 v2:172.21.15.190:6824/380064457 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa44c0c2890 con 0x7fa44c0bfcd0 2024-08-30T08:59:27.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:27.692+0000 7fa466153640 1 -- 172.21.15.42:0/2419607491 --> [v2:172.21.15.190:6824/380064457,v1:172.21.15.190:6825/380064457] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa460103a80 con 0x7fa44c0bfcd0 2024-08-30T08:59:27.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:27.693+0000 7fa45cff9640 1 -- 172.21.15.42:0/2419607491 <== osd.11 v2:172.21.15.190:6824/380064457 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fa460103a80 con 0x7fa44c0bfcd0 2024-08-30T08:59:27.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:27.694+0000 7fa466153640 1 -- 172.21.15.42:0/2419607491 >> [v2:172.21.15.190:6824/380064457,v1:172.21.15.190:6825/380064457] conn(0x7fa44c0bfcd0 msgr2=0x7fa44c0c21c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:27.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:27.694+0000 7fa466153640 1 --2- 172.21.15.42:0/2419607491 >> [v2:172.21.15.190:6824/380064457,v1:172.21.15.190:6825/380064457] conn(0x7fa44c0bfcd0 0x7fa44c0c21c0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:27.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:27.694+0000 7fa466153640 1 -- 172.21.15.42:0/2419607491 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fa44c0b4bc0 msgr2=0x7fa44c0b70d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:27.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:27.694+0000 7fa466153640 1 --2- 172.21.15.42:0/2419607491 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fa44c0b4bc0 0x7fa44c0b70d0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fa460079ad0 tx=0x7fa448005f90 comp rx=0 tx=0).stop 2024-08-30T08:59:27.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:27.694+0000 7fa466153640 1 -- 172.21.15.42:0/2419607491 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa460102800 msgr2=0x7fa46019a190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:27.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:27.694+0000 7fa466153640 1 --2- 172.21.15.42:0/2419607491 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa460102800 0x7fa46019a190 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fa45000cce0 tx=0x7fa45000a620 comp rx=0 tx=0).stop 2024-08-30T08:59:27.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:27.694+0000 7fa45ffff640 1 -- 172.21.15.42:0/2419607491 reap_dead start 2024-08-30T08:59:27.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:27.694+0000 7fa466153640 1 -- 172.21.15.42:0/2419607491 shutdown_connections 2024-08-30T08:59:27.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:27.694+0000 7fa466153640 1 -- 172.21.15.42:0/2419607491 >> 172.21.15.42:0/2419607491 conn(0x7fa4600794d0 msgr2=0x7fa460103ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:59:27.697 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T08:59:27.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:27.696+0000 7fa466153640 1 -- 172.21.15.42:0/2419607491 shutdown_connections 2024-08-30T08:59:27.700 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T08:59:27.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:27.697+0000 7fa466153640 1 -- 172.21.15.42:0/2419607491 wait complete. 2024-08-30T08:59:27.711 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T08:59:27.711 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T08:59:28.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:28.369+0000 7fdd740d3640 1 -- 172.21.15.42:0/1711477457 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdd6c103900 msgr2=0x7fdd6c103d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:28.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:28.369+0000 7fdd740d3640 1 --2- 172.21.15.42:0/1711477457 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdd6c103900 0x7fdd6c103d10 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fdd68009900 tx=0x7fdd6802a410 comp rx=0 tx=0).stop 2024-08-30T08:59:28.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:28.369+0000 7fdd740d3640 1 -- 172.21.15.42:0/1711477457 shutdown_connections 2024-08-30T08:59:28.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:28.369+0000 7fdd740d3640 1 --2- 172.21.15.42:0/1711477457 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdd6c109b90 0x7fdd6c1103c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:28.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:28.369+0000 7fdd740d3640 1 --2- 172.21.15.42:0/1711477457 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdd6c109240 0x7fdd6c109650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:28.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:28.369+0000 7fdd740d3640 1 --2- 172.21.15.42:0/1711477457 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdd6c103900 0x7fdd6c103d10 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:28.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:28.369+0000 7fdd740d3640 1 -- 172.21.15.42:0/1711477457 >> 172.21.15.42:0/1711477457 conn(0x7fdd6c07b5f0 msgr2=0x7fdd6c07ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:59:28.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:28.370+0000 7fdd740d3640 1 -- 172.21.15.42:0/1711477457 shutdown_connections 2024-08-30T08:59:28.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:28.370+0000 7fdd740d3640 1 -- 172.21.15.42:0/1711477457 wait complete. 2024-08-30T08:59:28.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:28.370+0000 7fdd740d3640 1 Processor -- start 2024-08-30T08:59:28.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:28.370+0000 7fdd740d3640 1 -- start start 2024-08-30T08:59:28.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:28.370+0000 7fdd740d3640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdd6c103900 0x7fdd6c19a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:28.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:28.371+0000 7fdd740d3640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdd6c109240 0x7fdd6c19a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:28.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:28.371+0000 7fdd740d3640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdd6c109b90 0x7fdd6c19fb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:28.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:28.371+0000 7fdd740d3640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fdd6c113850 con 0x7fdd6c103900 2024-08-30T08:59:28.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:28.371+0000 7fdd740d3640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fdd6c1136d0 con 0x7fdd6c109240 2024-08-30T08:59:28.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:28.371+0000 7fdd740d3640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fdd6c1139d0 con 0x7fdd6c109b90 2024-08-30T08:59:28.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:28.371+0000 7fdd71e6f640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdd6c103900 0x7fdd6c19a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:28.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:28.371+0000 7fdd71e6f640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdd6c103900 0x7fdd6c19a2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:53248/0 (socket says 172.21.15.42:53248) 2024-08-30T08:59:28.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:28.371+0000 7fdd71e6f640 1 -- 172.21.15.42:0/2130177708 learned_addr learned my addr 172.21.15.42:0/2130177708 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T08:59:28.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:28.371+0000 7fdd71e6f640 1 -- 172.21.15.42:0/2130177708 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdd6c109b90 msgr2=0x7fdd6c19fb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:28.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:28.371+0000 7fdd71e6f640 1 --2- 172.21.15.42:0/2130177708 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdd6c109b90 0x7fdd6c19fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:28.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:28.371+0000 7fdd71e6f640 1 -- 172.21.15.42:0/2130177708 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdd6c109240 msgr2=0x7fdd6c19a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:28.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:28.371+0000 7fdd71e6f640 1 --2- 172.21.15.42:0/2130177708 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdd6c109240 0x7fdd6c19a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:28.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:28.371+0000 7fdd71e6f640 1 -- 172.21.15.42:0/2130177708 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd6c1a0230 con 0x7fdd6c103900 2024-08-30T08:59:28.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:28.371+0000 7fdd71e6f640 1 --2- 172.21.15.42:0/2130177708 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdd6c103900 0x7fdd6c19a2c0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fdd68005940 tx=0x7fdd680042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:59:28.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:28.371+0000 7fdd62ffd640 1 -- 172.21.15.42:0/2130177708 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd6803f030 con 0x7fdd6c103900 2024-08-30T08:59:28.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:28.371+0000 7fdd62ffd640 1 -- 172.21.15.42:0/2130177708 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd68003cb0 con 0x7fdd6c103900 2024-08-30T08:59:28.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:28.371+0000 7fdd740d3640 1 -- 172.21.15.42:0/2130177708 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd6c1a04c0 con 0x7fdd6c103900 2024-08-30T08:59:28.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:28.371+0000 7fdd62ffd640 1 -- 172.21.15.42:0/2130177708 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd68035070 con 0x7fdd6c103900 2024-08-30T08:59:28.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:28.371+0000 7fdd740d3640 1 -- 172.21.15.42:0/2130177708 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd6c1a6c80 con 0x7fdd6c103900 2024-08-30T08:59:28.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:28.372+0000 7fdd740d3640 1 -- 172.21.15.42:0/2130177708 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd6c103d10 con 0x7fdd6c103900 2024-08-30T08:59:28.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:28.374+0000 7fdd62ffd640 1 -- 172.21.15.42:0/2130177708 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) ==== 152628+0+0 (secure 0 0 0) 0x7fdd68042030 con 0x7fdd6c103900 2024-08-30T08:59:28.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:28.374+0000 7fdd62ffd640 1 --2- 172.21.15.42:0/2130177708 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fdd580b4bc0 0x7fdd580b70d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:28.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:28.375+0000 7fdd7166e640 1 --2- 172.21.15.42:0/2130177708 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fdd580b4bc0 0x7fdd580b70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:28.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:28.375+0000 7fdd62ffd640 1 -- 172.21.15.42:0/2130177708 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 9092+0+0 (secure 0 0 0) 0x7fdd68106200 con 0x7fdd6c103900 2024-08-30T08:59:28.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:28.375+0000 7fdd7166e640 1 --2- 172.21.15.42:0/2130177708 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fdd580b4bc0 0x7fdd580b70d0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fdd5c005fd0 tx=0x7fdd5c005c70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:59:28.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:28.378+0000 7fdd62ffd640 1 -- 172.21.15.42:0/2130177708 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7fdd680cce70 con 0x7fdd6c103900 2024-08-30T08:59:28.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:28.566+0000 7fdd740d3640 1 -- 172.21.15.42:0/2130177708 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7fdd6c02e430 con 0x7fdd6c103900 2024-08-30T08:59:28.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:28.566+0000 7fdd62ffd640 1 -- 172.21.15.42:0/2130177708 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fdd680cc770 con 0x7fdd6c103900 2024-08-30T08:59:28.569 INFO:teuthology.orchestra.run.smithi042.stdout:128849018946 2024-08-30T08:59:28.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:28.569+0000 7fdd740d3640 1 -- 172.21.15.42:0/2130177708 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fdd580b4bc0 msgr2=0x7fdd580b70d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:28.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:28.569+0000 7fdd740d3640 1 --2- 172.21.15.42:0/2130177708 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fdd580b4bc0 0x7fdd580b70d0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fdd5c005fd0 tx=0x7fdd5c005c70 comp rx=0 tx=0).stop 2024-08-30T08:59:28.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:28.569+0000 7fdd740d3640 1 -- 172.21.15.42:0/2130177708 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdd6c103900 msgr2=0x7fdd6c19a2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:28.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:28.569+0000 7fdd740d3640 1 --2- 172.21.15.42:0/2130177708 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdd6c103900 0x7fdd6c19a2c0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fdd68005940 tx=0x7fdd680042f0 comp rx=0 tx=0).stop 2024-08-30T08:59:28.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:28.569+0000 7fdd740d3640 1 -- 172.21.15.42:0/2130177708 shutdown_connections 2024-08-30T08:59:28.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:28.569+0000 7fdd740d3640 1 --2- 172.21.15.42:0/2130177708 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fdd580b4bc0 0x7fdd580b70d0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:28.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:28.569+0000 7fdd740d3640 1 --2- 172.21.15.42:0/2130177708 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdd6c109b90 0x7fdd6c19fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:28.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:28.569+0000 7fdd740d3640 1 --2- 172.21.15.42:0/2130177708 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdd6c109240 0x7fdd6c19a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:28.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:28.569+0000 7fdd740d3640 1 --2- 172.21.15.42:0/2130177708 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdd6c103900 0x7fdd6c19a2c0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:28.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:28.569+0000 7fdd740d3640 1 -- 172.21.15.42:0/2130177708 >> 172.21.15.42:0/2130177708 conn(0x7fdd6c07b5f0 msgr2=0x7fdd6c107330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:59:28.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:28.569+0000 7fdd740d3640 1 -- 172.21.15.42:0/2130177708 shutdown_connections 2024-08-30T08:59:28.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:28.569+0000 7fdd740d3640 1 -- 172.21.15.42:0/2130177708 wait complete. 2024-08-30T08:59:28.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:28.594+0000 7efff599d640 1 -- 172.21.15.42:0/3238723896 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efff0109bb0 msgr2=0x7efff01103e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:28.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:28.594+0000 7efff599d640 1 --2- 172.21.15.42:0/3238723896 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efff0109bb0 0x7efff01103e0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7effe8009ca0 tx=0x7effe802a8c0 comp rx=0 tx=0).stop 2024-08-30T08:59:28.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:28.594+0000 7efff599d640 1 -- 172.21.15.42:0/3238723896 shutdown_connections 2024-08-30T08:59:28.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:28.594+0000 7efff599d640 1 --2- 172.21.15.42:0/3238723896 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efff0109bb0 0x7efff01103e0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:28.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:28.594+0000 7efff599d640 1 --2- 172.21.15.42:0/3238723896 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efff0109260 0x7efff0109670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:28.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:28.594+0000 7efff599d640 1 --2- 172.21.15.42:0/3238723896 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efff0103920 0x7efff0103d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:28.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:28.594+0000 7efff599d640 1 -- 172.21.15.42:0/3238723896 >> 172.21.15.42:0/3238723896 conn(0x7efff007b670 msgr2=0x7efff007ba80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:59:28.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:28.594+0000 7efff599d640 1 -- 172.21.15.42:0/3238723896 shutdown_connections 2024-08-30T08:59:28.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:28.594+0000 7efff599d640 1 -- 172.21.15.42:0/3238723896 wait complete. 2024-08-30T08:59:28.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:28.595+0000 7efff599d640 1 Processor -- start 2024-08-30T08:59:28.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:28.595+0000 7efff599d640 1 -- start start 2024-08-30T08:59:28.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:28.595+0000 7efff599d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efff0103920 0x7efff019a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:28.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:28.595+0000 7efff599d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efff0109260 0x7efff019a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:28.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:28.595+0000 7effeeffd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efff0103920 0x7efff019a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:28.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:28.595+0000 7effeeffd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efff0103920 0x7efff019a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:53272/0 (socket says 172.21.15.42:53272) 2024-08-30T08:59:28.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:28.595+0000 7efff599d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efff0109bb0 0x7efff019fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:28.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:28.595+0000 7effeeffd640 1 -- 172.21.15.42:0/3492183680 learned_addr learned my addr 172.21.15.42:0/3492183680 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T08:59:28.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:28.595+0000 7efff599d640 1 -- 172.21.15.42:0/3492183680 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7efff01138c0 con 0x7efff0103920 2024-08-30T08:59:28.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:28.596+0000 7efff599d640 1 -- 172.21.15.42:0/3492183680 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7efff0113740 con 0x7efff0109260 2024-08-30T08:59:28.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:28.596+0000 7efff599d640 1 -- 172.21.15.42:0/3492183680 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7efff0113a40 con 0x7efff0109bb0 2024-08-30T08:59:28.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:28.596+0000 7effee7fc640 1 --2- 172.21.15.42:0/3492183680 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efff0109260 0x7efff019a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:28.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:28.596+0000 7effef7fe640 1 --2- 172.21.15.42:0/3492183680 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efff0109bb0 0x7efff019fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:28.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:28.596+0000 7effeeffd640 1 -- 172.21.15.42:0/3492183680 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efff0109bb0 msgr2=0x7efff019fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:28.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:28.596+0000 7effeeffd640 1 --2- 172.21.15.42:0/3492183680 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efff0109bb0 0x7efff019fb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:28.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:28.596+0000 7effeeffd640 1 -- 172.21.15.42:0/3492183680 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efff0109260 msgr2=0x7efff019a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:28.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:28.596+0000 7effeeffd640 1 --2- 172.21.15.42:0/3492183680 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efff0109260 0x7efff019a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:28.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:28.596+0000 7effeeffd640 1 -- 172.21.15.42:0/3492183680 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efff01a0260 con 0x7efff0103920 2024-08-30T08:59:28.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:28.596+0000 7effeeffd640 1 --2- 172.21.15.42:0/3492183680 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efff0103920 0x7efff019a2f0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7effe000e9f0 tx=0x7effe000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:59:28.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:28.596+0000 7efff499b640 1 -- 172.21.15.42:0/3492183680 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effe000cd80 con 0x7efff0103920 2024-08-30T08:59:28.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:28.596+0000 7efff499b640 1 -- 172.21.15.42:0/3492183680 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7effe00054d0 con 0x7efff0103920 2024-08-30T08:59:28.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:28.596+0000 7efff599d640 1 -- 172.21.15.42:0/3492183680 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efff01a0550 con 0x7efff0103920 2024-08-30T08:59:28.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:28.597+0000 7efff499b640 1 -- 172.21.15.42:0/3492183680 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effe0010690 con 0x7efff0103920 2024-08-30T08:59:28.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:28.597+0000 7efff599d640 1 -- 172.21.15.42:0/3492183680 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efff01a6cb0 con 0x7efff0103920 2024-08-30T08:59:28.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:28.598+0000 7efff599d640 1 -- 172.21.15.42:0/3492183680 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efff0103d30 con 0x7efff0103920 2024-08-30T08:59:28.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:28.600+0000 7efff499b640 1 -- 172.21.15.42:0/3492183680 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) ==== 152628+0+0 (secure 0 0 0) 0x7effe0004e30 con 0x7efff0103920 2024-08-30T08:59:28.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:28.600+0000 7efff499b640 1 --2- 172.21.15.42:0/3492183680 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7effd80b4bc0 0x7effd80b70d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:28.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:28.600+0000 7effee7fc640 1 --2- 172.21.15.42:0/3492183680 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7effd80b4bc0 0x7effd80b70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:28.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:28.600+0000 7efff499b640 1 -- 172.21.15.42:0/3492183680 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 9092+0+0 (secure 0 0 0) 0x7effe00e5d00 con 0x7efff0103920 2024-08-30T08:59:28.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:28.601+0000 7effee7fc640 1 --2- 172.21.15.42:0/3492183680 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7effd80b4bc0 0x7effd80b70d0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7efff0106d00 tx=0x7effe4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:59:28.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:28.604+0000 7efff499b640 1 -- 172.21.15.42:0/3492183680 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7effe0056130 con 0x7efff0103920 2024-08-30T08:59:28.793 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:28.790+0000 7efff599d640 1 -- 172.21.15.42:0/3492183680 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7efff010b410 con 0x7efff0103920 2024-08-30T08:59:28.793 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:59:28 smithi042 ceph-mon[18652]: pgmap v332: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-08-30T08:59:28.793 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:59:28 smithi042 ceph-mon[18652]: from='client.? 172.21.15.42:0/2130177708' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 4} : dispatch 2024-08-30T08:59:28.794 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:28.790+0000 7efff499b640 1 -- 172.21.15.42:0/3492183680 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7effe00ac910 con 0x7efff0103920 2024-08-30T08:59:28.794 INFO:teuthology.orchestra.run.smithi042.stdout:261993005090 2024-08-30T08:59:28.796 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:28.794+0000 7efff599d640 1 -- 172.21.15.42:0/3492183680 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7effd80b4bc0 msgr2=0x7effd80b70d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:28.797 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:28.794+0000 7efff599d640 1 --2- 172.21.15.42:0/3492183680 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7effd80b4bc0 0x7effd80b70d0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7efff0106d00 tx=0x7effe4005f90 comp rx=0 tx=0).stop 2024-08-30T08:59:28.797 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:28.794+0000 7efff599d640 1 -- 172.21.15.42:0/3492183680 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efff0103920 msgr2=0x7efff019a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:28.797 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:28.794+0000 7efff599d640 1 --2- 172.21.15.42:0/3492183680 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efff0103920 0x7efff019a2f0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7effe000e9f0 tx=0x7effe000edf0 comp rx=0 tx=0).stop 2024-08-30T08:59:28.797 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:28.794+0000 7efff599d640 1 -- 172.21.15.42:0/3492183680 shutdown_connections 2024-08-30T08:59:28.797 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:28.794+0000 7efff599d640 1 --2- 172.21.15.42:0/3492183680 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7effd80b4bc0 0x7effd80b70d0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:28.797 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:28.794+0000 7efff599d640 1 --2- 172.21.15.42:0/3492183680 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efff0109bb0 0x7efff019fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:28.797 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:28.794+0000 7efff599d640 1 --2- 172.21.15.42:0/3492183680 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efff0109260 0x7efff019a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:28.797 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:28.794+0000 7efff599d640 1 --2- 172.21.15.42:0/3492183680 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efff0103920 0x7efff019a2f0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:28.797 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:28.794+0000 7efff599d640 1 -- 172.21.15.42:0/3492183680 >> 172.21.15.42:0/3492183680 conn(0x7efff007b670 msgr2=0x7efff0105270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:59:28.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:28.794+0000 7efff599d640 1 -- 172.21.15.42:0/3492183680 shutdown_connections 2024-08-30T08:59:28.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:28.794+0000 7efff599d640 1 -- 172.21.15.42:0/3492183680 wait complete. 2024-08-30T08:59:28.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:59:28 smithi190 ceph-mon[22001]: pgmap v332: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-08-30T08:59:28.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:59:28 smithi190 ceph-mon[22001]: from='client.? 172.21.15.42:0/2130177708' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 4} : dispatch 2024-08-30T08:59:29.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:59:28 smithi167 ceph-mon[22063]: pgmap v332: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-08-30T08:59:29.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:59:28 smithi167 ceph-mon[22063]: from='client.? 172.21.15.42:0/2130177708' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 4} : dispatch 2024-08-30T08:59:29.466 INFO:tasks.cephadm.ceph_manager.ceph:need seq 128849018944 got 128849018946 for osd.4 2024-08-30T08:59:29.467 DEBUG:teuthology.parallel:result is None 2024-08-30T08:59:29.685 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:59:29 smithi042 ceph-mon[18652]: from='client.? 172.21.15.42:0/3492183680' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 10} : dispatch 2024-08-30T08:59:29.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:59:29 smithi190 ceph-mon[22001]: from='client.? 172.21.15.42:0/3492183680' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 10} : dispatch 2024-08-30T08:59:30.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:59:29 smithi167 ceph-mon[22063]: from='client.? 172.21.15.42:0/3492183680' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 10} : dispatch 2024-08-30T08:59:30.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:30.521+0000 7fac4051b640 1 -- 172.21.15.42:0/152895398 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fac38102f20 msgr2=0x7fac38103330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:30.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:30.521+0000 7fac4051b640 1 --2- 172.21.15.42:0/152895398 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fac38102f20 0x7fac38103330 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fac34009900 tx=0x7fac3402a410 comp rx=0 tx=0).stop 2024-08-30T08:59:30.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:30.521+0000 7fac4051b640 1 -- 172.21.15.42:0/152895398 shutdown_connections 2024-08-30T08:59:30.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:30.521+0000 7fac4051b640 1 --2- 172.21.15.42:0/152895398 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fac38108150 0x7fac38108560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:30.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:30.521+0000 7fac4051b640 1 --2- 172.21.15.42:0/152895398 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fac38108aa0 0x7fac3810f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:30.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:30.521+0000 7fac4051b640 1 --2- 172.21.15.42:0/152895398 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fac38102f20 0x7fac38103330 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:30.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:30.521+0000 7fac4051b640 1 -- 172.21.15.42:0/152895398 >> 172.21.15.42:0/152895398 conn(0x7fac38077a00 msgr2=0x7fac38077e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:59:30.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:30.521+0000 7fac4051b640 1 -- 172.21.15.42:0/152895398 shutdown_connections 2024-08-30T08:59:30.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:30.521+0000 7fac4051b640 1 -- 172.21.15.42:0/152895398 wait complete. 2024-08-30T08:59:30.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:30.522+0000 7fac4051b640 1 Processor -- start 2024-08-30T08:59:30.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:30.522+0000 7fac4051b640 1 -- start start 2024-08-30T08:59:30.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:30.523+0000 7fac4051b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fac38102f20 0x7fac3810fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:30.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:30.523+0000 7fac4051b640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fac38108150 0x7fac381102f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:30.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:30.523+0000 7fac4051b640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fac38108aa0 0x7fac38110830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:30.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:30.523+0000 7fac4051b640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fac38113880 con 0x7fac38108150 2024-08-30T08:59:30.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:30.523+0000 7fac4051b640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fac38113700 con 0x7fac38108aa0 2024-08-30T08:59:30.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:30.523+0000 7fac4051b640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fac38113a00 con 0x7fac38102f20 2024-08-30T08:59:30.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:30.523+0000 7fac3dab6640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fac38108150 0x7fac381102f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:30.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:30.523+0000 7fac3dab6640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fac38108150 0x7fac381102f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:53308/0 (socket says 172.21.15.42:53308) 2024-08-30T08:59:30.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:30.523+0000 7fac3e2b7640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fac38102f20 0x7fac3810fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:30.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:30.523+0000 7fac3dab6640 1 -- 172.21.15.42:0/2054534545 learned_addr learned my addr 172.21.15.42:0/2054534545 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T08:59:30.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:30.523+0000 7fac3eab8640 1 --2- 172.21.15.42:0/2054534545 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fac38108aa0 0x7fac38110830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:30.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:30.523+0000 7fac3e2b7640 1 -- 172.21.15.42:0/2054534545 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fac38108aa0 msgr2=0x7fac38110830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:30.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:30.523+0000 7fac3e2b7640 1 --2- 172.21.15.42:0/2054534545 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fac38108aa0 0x7fac38110830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:30.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:30.523+0000 7fac3e2b7640 1 -- 172.21.15.42:0/2054534545 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fac38108150 msgr2=0x7fac381102f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:30.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:30.523+0000 7fac3e2b7640 1 --2- 172.21.15.42:0/2054534545 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fac38108150 0x7fac381102f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:30.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:30.523+0000 7fac3e2b7640 1 -- 172.21.15.42:0/2054534545 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac381a8530 con 0x7fac38102f20 2024-08-30T08:59:30.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:30.523+0000 7fac3dab6640 1 --2- 172.21.15.42:0/2054534545 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fac38108150 0x7fac381102f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-30T08:59:30.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:30.524+0000 7fac3e2b7640 1 --2- 172.21.15.42:0/2054534545 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fac38102f20 0x7fac3810fdb0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fac3402a930 tx=0x7fac34004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:59:30.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:30.524+0000 7fac2f7fe640 1 -- 172.21.15.42:0/2054534545 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac3403f030 con 0x7fac38102f20 2024-08-30T08:59:30.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:30.524+0000 7fac4051b640 1 -- 172.21.15.42:0/2054534545 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac381a87c0 con 0x7fac38102f20 2024-08-30T08:59:30.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:30.524+0000 7fac2f7fe640 1 -- 172.21.15.42:0/2054534545 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac34003aa0 con 0x7fac38102f20 2024-08-30T08:59:30.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:30.524+0000 7fac2f7fe640 1 -- 172.21.15.42:0/2054534545 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac34035070 con 0x7fac38102f20 2024-08-30T08:59:30.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:30.524+0000 7fac4051b640 1 -- 172.21.15.42:0/2054534545 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac381a8cf0 con 0x7fac38102f20 2024-08-30T08:59:30.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:30.525+0000 7fac4051b640 1 -- 172.21.15.42:0/2054534545 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac38103330 con 0x7fac38102f20 2024-08-30T08:59:30.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:30.528+0000 7fac2f7fe640 1 -- 172.21.15.42:0/2054534545 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 17) ==== 152628+0+0 (secure 0 0 0) 0x7fac34042030 con 0x7fac38102f20 2024-08-30T08:59:30.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:30.529+0000 7fac2f7fe640 1 --2- 172.21.15.42:0/2054534545 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fac240b4bc0 0x7fac240b70d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:30.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:30.529+0000 7fac3dab6640 1 --2- 172.21.15.42:0/2054534545 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fac240b4bc0 0x7fac240b70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:30.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:30.530+0000 7fac2f7fe640 1 -- 172.21.15.42:0/2054534545 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 9092+0+0 (secure 0 0 0) 0x7fac34105e60 con 0x7fac38102f20 2024-08-30T08:59:30.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:30.530+0000 7fac3dab6640 1 --2- 172.21.15.42:0/2054534545 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fac240b4bc0 0x7fac240b70d0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fac38107cb0 tx=0x7fac2800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:59:30.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:30.531+0000 7fac2f7fe640 1 -- 172.21.15.42:0/2054534545 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7fac340cc420 con 0x7fac38102f20 2024-08-30T08:59:30.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:30.736+0000 7fac4051b640 1 -- 172.21.15.42:0/2054534545 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7fac3802e430 con 0x7fac38102f20 2024-08-30T08:59:30.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:30.737+0000 7fac2f7fe640 1 -- 172.21.15.42:0/2054534545 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fac340d0be0 con 0x7fac38102f20 2024-08-30T08:59:30.740 INFO:teuthology.orchestra.run.smithi042.stdout:150323855421 2024-08-30T08:59:30.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:30.739+0000 7fac4051b640 1 -- 172.21.15.42:0/2054534545 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fac240b4bc0 msgr2=0x7fac240b70d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:30.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:30.739+0000 7fac4051b640 1 --2- 172.21.15.42:0/2054534545 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fac240b4bc0 0x7fac240b70d0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fac38107cb0 tx=0x7fac2800b390 comp rx=0 tx=0).stop 2024-08-30T08:59:30.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:30.739+0000 7fac4051b640 1 -- 172.21.15.42:0/2054534545 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fac38102f20 msgr2=0x7fac3810fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:30.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:30.739+0000 7fac4051b640 1 --2- 172.21.15.42:0/2054534545 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fac38102f20 0x7fac3810fdb0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fac3402a930 tx=0x7fac34004270 comp rx=0 tx=0).stop 2024-08-30T08:59:30.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:30.739+0000 7fac4051b640 1 -- 172.21.15.42:0/2054534545 shutdown_connections 2024-08-30T08:59:30.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:30.740+0000 7fac4051b640 1 --2- 172.21.15.42:0/2054534545 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fac240b4bc0 0x7fac240b70d0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:30.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:30.740+0000 7fac4051b640 1 --2- 172.21.15.42:0/2054534545 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fac38108aa0 0x7fac38110830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:30.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:30.740+0000 7fac4051b640 1 --2- 172.21.15.42:0/2054534545 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fac38108150 0x7fac381102f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:30.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:30.740+0000 7fac4051b640 1 --2- 172.21.15.42:0/2054534545 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fac38102f20 0x7fac3810fdb0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:30.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:30.740+0000 7fac4051b640 1 -- 172.21.15.42:0/2054534545 >> 172.21.15.42:0/2054534545 conn(0x7fac38077a00 msgr2=0x7fac381061e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:59:30.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:30.740+0000 7fac4051b640 1 -- 172.21.15.42:0/2054534545 shutdown_connections 2024-08-30T08:59:30.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:30.740+0000 7fac4051b640 1 -- 172.21.15.42:0/2054534545 wait complete. 2024-08-30T08:59:30.771 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:59:30 smithi042 ceph-mon[18652]: pgmap v333: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-08-30T08:59:30.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:59:30 smithi190 ceph-mon[22001]: pgmap v333: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-08-30T08:59:31.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:59:30 smithi167 ceph-mon[22063]: pgmap v333: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-08-30T08:59:31.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:30.444+0000 7effdd78f640 1 -- 172.21.15.42:0/2635509874 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7effd810dc00 msgr2=0x7effd81100d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:31.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:30.444+0000 7effdd78f640 1 --2- 172.21.15.42:0/2635509874 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7effd810dc00 0x7effd81100d0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7effd0009ca0 tx=0x7effd002a8c0 comp rx=0 tx=0).stop 2024-08-30T08:59:31.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:30.444+0000 7effdd78f640 1 -- 172.21.15.42:0/2635509874 shutdown_connections 2024-08-30T08:59:31.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:30.444+0000 7effdd78f640 1 --2- 172.21.15.42:0/2635509874 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7effd8105120 0x7effd810d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:31.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:30.444+0000 7effdd78f640 1 --2- 172.21.15.42:0/2635509874 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7effd810dc00 0x7effd81100d0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:31.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:30.444+0000 7effdd78f640 1 --2- 172.21.15.42:0/2635509874 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7effd81047d0 0x7effd8104be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:31.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:30.444+0000 7effdd78f640 1 -- 172.21.15.42:0/2635509874 >> 172.21.15.42:0/2635509874 conn(0x7effd8079530 msgr2=0x7effd8079940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:59:31.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:30.444+0000 7effdd78f640 1 -- 172.21.15.42:0/2635509874 shutdown_connections 2024-08-30T08:59:31.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:30.444+0000 7effdd78f640 1 -- 172.21.15.42:0/2635509874 wait complete. 2024-08-30T08:59:31.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:30.445+0000 7effdd78f640 1 Processor -- start 2024-08-30T08:59:31.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:30.445+0000 7effdd78f640 1 -- start start 2024-08-30T08:59:31.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:30.445+0000 7effdd78f640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7effd81047d0 0x7effd819a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:31.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:30.445+0000 7effdd78f640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7effd8105120 0x7effd819a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:31.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:30.445+0000 7effdd78f640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7effd810dc00 0x7effd819f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:31.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:30.446+0000 7effdd78f640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7effd81138b0 con 0x7effd81047d0 2024-08-30T08:59:31.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:30.446+0000 7effdd78f640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7effd8113730 con 0x7effd8105120 2024-08-30T08:59:31.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:30.446+0000 7effdd78f640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7effd8113a30 con 0x7effd810dc00 2024-08-30T08:59:31.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:30.446+0000 7effd6ffd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7effd81047d0 0x7effd819a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:31.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:30.446+0000 7effd6ffd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7effd81047d0 0x7effd819a1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:53286/0 (socket says 172.21.15.42:53286) 2024-08-30T08:59:31.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:30.446+0000 7effd6ffd640 1 -- 172.21.15.42:0/2534681143 learned_addr learned my addr 172.21.15.42:0/2534681143 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T08:59:31.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:30.446+0000 7effd77fe640 1 --2- 172.21.15.42:0/2534681143 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7effd810dc00 0x7effd819f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:31.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:30.446+0000 7effd67fc640 1 --2- 172.21.15.42:0/2534681143 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7effd8105120 0x7effd819a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:31.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:30.446+0000 7effd6ffd640 1 -- 172.21.15.42:0/2534681143 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7effd810dc00 msgr2=0x7effd819f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:31.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:30.446+0000 7effd6ffd640 1 --2- 172.21.15.42:0/2534681143 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7effd810dc00 0x7effd819f9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:31.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:30.446+0000 7effd6ffd640 1 -- 172.21.15.42:0/2534681143 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7effd8105120 msgr2=0x7effd819a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:31.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:30.446+0000 7effd6ffd640 1 --2- 172.21.15.42:0/2534681143 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7effd8105120 0x7effd819a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:31.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:30.446+0000 7effd6ffd640 1 -- 172.21.15.42:0/2534681143 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effd81a0110 con 0x7effd81047d0 2024-08-30T08:59:31.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:30.446+0000 7effd6ffd640 1 --2- 172.21.15.42:0/2534681143 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7effd81047d0 0x7effd819a1a0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7effc800e9f0 tx=0x7effc800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:59:31.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:30.446+0000 7effbffff640 1 -- 172.21.15.42:0/2534681143 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effc800cd80 con 0x7effd81047d0 2024-08-30T08:59:31.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:30.447+0000 7effdd78f640 1 -- 172.21.15.42:0/2534681143 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effd81a0400 con 0x7effd81047d0 2024-08-30T08:59:31.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:30.447+0000 7effbffff640 1 -- 172.21.15.42:0/2534681143 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7effc80108e0 con 0x7effd81047d0 2024-08-30T08:59:31.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:30.447+0000 7effbffff640 1 -- 172.21.15.42:0/2534681143 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effc8018980 con 0x7effd81047d0 2024-08-30T08:59:31.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:30.447+0000 7effdd78f640 1 -- 172.21.15.42:0/2534681143 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effd81023f0 con 0x7effd81047d0 2024-08-30T08:59:31.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:30.448+0000 7effdd78f640 1 -- 172.21.15.42:0/2534681143 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7effd8104be0 con 0x7effd81047d0 2024-08-30T08:59:31.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:30.450+0000 7effbffff640 1 -- 172.21.15.42:0/2534681143 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) ==== 152628+0+0 (secure 0 0 0) 0x7effc8010430 con 0x7effd81047d0 2024-08-30T08:59:31.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:30.451+0000 7effbffff640 1 --2- 172.21.15.42:0/2534681143 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7effc00b4bd0 0x7effc00b70e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:31.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:30.451+0000 7effd67fc640 1 --2- 172.21.15.42:0/2534681143 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7effc00b4bd0 0x7effc00b70e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:31.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:30.451+0000 7effbffff640 1 -- 172.21.15.42:0/2534681143 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 9092+0+0 (secure 0 0 0) 0x7effc80e5ed0 con 0x7effd81047d0 2024-08-30T08:59:31.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:30.452+0000 7effd67fc640 1 --2- 172.21.15.42:0/2534681143 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7effc00b4bd0 0x7effc00b70e0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7effcc005fd0 tx=0x7effcc005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:59:31.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:30.455+0000 7effbffff640 1 -- 172.21.15.42:0/2534681143 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7effc80acb40 con 0x7effd81047d0 2024-08-30T08:59:31.178 INFO:teuthology.orchestra.run.smithi042.stdout:77309411404 2024-08-30T08:59:31.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:30.651+0000 7effdd78f640 1 -- 172.21.15.42:0/2534681143 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7effd81026e0 con 0x7effd81047d0 2024-08-30T08:59:31.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:30.651+0000 7effbffff640 1 -- 172.21.15.42:0/2534681143 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7effc80b5080 con 0x7effd81047d0 2024-08-30T08:59:31.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:30.654+0000 7effdd78f640 1 -- 172.21.15.42:0/2534681143 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7effc00b4bd0 msgr2=0x7effc00b70e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:31.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:30.654+0000 7effdd78f640 1 --2- 172.21.15.42:0/2534681143 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7effc00b4bd0 0x7effc00b70e0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7effcc005fd0 tx=0x7effcc005850 comp rx=0 tx=0).stop 2024-08-30T08:59:31.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:30.654+0000 7effdd78f640 1 -- 172.21.15.42:0/2534681143 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7effd81047d0 msgr2=0x7effd819a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:31.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:30.654+0000 7effdd78f640 1 --2- 172.21.15.42:0/2534681143 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7effd81047d0 0x7effd819a1a0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7effc800e9f0 tx=0x7effc800eeb0 comp rx=0 tx=0).stop 2024-08-30T08:59:31.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:30.654+0000 7effdd78f640 1 -- 172.21.15.42:0/2534681143 shutdown_connections 2024-08-30T08:59:31.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:30.654+0000 7effdd78f640 1 --2- 172.21.15.42:0/2534681143 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7effc00b4bd0 0x7effc00b70e0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:31.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:30.654+0000 7effdd78f640 1 --2- 172.21.15.42:0/2534681143 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7effd810dc00 0x7effd819f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:31.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:30.654+0000 7effdd78f640 1 --2- 172.21.15.42:0/2534681143 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7effd8105120 0x7effd819a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:31.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:30.654+0000 7effdd78f640 1 --2- 172.21.15.42:0/2534681143 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7effd81047d0 0x7effd819a1a0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:31.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:30.654+0000 7effdd78f640 1 -- 172.21.15.42:0/2534681143 >> 172.21.15.42:0/2534681143 conn(0x7effd8079530 msgr2=0x7effd80ff2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:59:31.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:30.654+0000 7effdd78f640 1 -- 172.21.15.42:0/2534681143 shutdown_connections 2024-08-30T08:59:31.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:30.654+0000 7effdd78f640 1 -- 172.21.15.42:0/2534681143 wait complete. 2024-08-30T08:59:31.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:59:31 smithi042 ceph-mon[18652]: from='client.? 172.21.15.42:0/2534681143' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 2} : dispatch 2024-08-30T08:59:31.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:59:31 smithi042 ceph-mon[18652]: from='client.? 172.21.15.42:0/2054534545' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 5} : dispatch 2024-08-30T08:59:31.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:59:31 smithi042 ceph-mon[18652]: from='client.? 172.21.15.42:0/4057169957' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 0} : dispatch 2024-08-30T08:59:31.833 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411401 got 77309411404 for osd.2 2024-08-30T08:59:31.834 DEBUG:teuthology.parallel:result is None 2024-08-30T08:59:31.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:31.172+0000 7fba79c53640 1 -- 172.21.15.42:0/3631328264 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fba74102800 msgr2=0x7fba74102c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:31.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:31.172+0000 7fba79c53640 1 --2- 172.21.15.42:0/3631328264 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fba74102800 0x7fba74102c10 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fba5c009900 tx=0x7fba5c02a410 comp rx=0 tx=0).stop 2024-08-30T08:59:31.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:31.172+0000 7fba79c53640 1 -- 172.21.15.42:0/3631328264 shutdown_connections 2024-08-30T08:59:31.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:31.172+0000 7fba79c53640 1 --2- 172.21.15.42:0/3631328264 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fba74108140 0x7fba74108550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:31.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:31.172+0000 7fba79c53640 1 --2- 172.21.15.42:0/3631328264 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fba74108a90 0x7fba7410f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:31.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:31.172+0000 7fba79c53640 1 --2- 172.21.15.42:0/3631328264 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fba74102800 0x7fba74102c10 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:31.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:31.172+0000 7fba79c53640 1 -- 172.21.15.42:0/3631328264 >> 172.21.15.42:0/3631328264 conn(0x7fba740794d0 msgr2=0x7fba740798e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:59:31.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:31.172+0000 7fba79c53640 1 -- 172.21.15.42:0/3631328264 shutdown_connections 2024-08-30T08:59:31.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:31.172+0000 7fba79c53640 1 -- 172.21.15.42:0/3631328264 wait complete. 2024-08-30T08:59:31.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:31.173+0000 7fba79c53640 1 Processor -- start 2024-08-30T08:59:31.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:31.173+0000 7fba79c53640 1 -- start start 2024-08-30T08:59:31.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:31.173+0000 7fba79c53640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fba74102800 0x7fba7419a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:31.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:31.173+0000 7fba79c53640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fba74108140 0x7fba7419a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:31.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:31.174+0000 7fba79c53640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fba74108a90 0x7fba7419fa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:31.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:31.174+0000 7fba79c53640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fba741126f0 con 0x7fba74102800 2024-08-30T08:59:31.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:31.174+0000 7fba79c53640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fba74112570 con 0x7fba74108140 2024-08-30T08:59:31.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:31.174+0000 7fba79c53640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fba74112870 con 0x7fba74108a90 2024-08-30T08:59:31.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:31.174+0000 7fba737fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fba74102800 0x7fba7419a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:31.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:31.174+0000 7fba737fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fba74102800 0x7fba7419a1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:53332/0 (socket says 172.21.15.42:53332) 2024-08-30T08:59:31.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:31.174+0000 7fba737fe640 1 -- 172.21.15.42:0/4057169957 learned_addr learned my addr 172.21.15.42:0/4057169957 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T08:59:31.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:31.174+0000 7fba73fff640 1 --2- 172.21.15.42:0/4057169957 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fba74108a90 0x7fba7419fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:31.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:31.174+0000 7fba72ffd640 1 --2- 172.21.15.42:0/4057169957 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fba74108140 0x7fba7419a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:31.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:31.174+0000 7fba737fe640 1 -- 172.21.15.42:0/4057169957 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fba74108a90 msgr2=0x7fba7419fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:31.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:31.174+0000 7fba737fe640 1 --2- 172.21.15.42:0/4057169957 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fba74108a90 0x7fba7419fa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:31.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:31.174+0000 7fba737fe640 1 -- 172.21.15.42:0/4057169957 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fba74108140 msgr2=0x7fba7419a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:31.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:31.174+0000 7fba737fe640 1 --2- 172.21.15.42:0/4057169957 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fba74108140 0x7fba7419a790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:31.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:31.174+0000 7fba737fe640 1 -- 172.21.15.42:0/4057169957 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba741a0160 con 0x7fba74102800 2024-08-30T08:59:31.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:31.174+0000 7fba737fe640 1 --2- 172.21.15.42:0/4057169957 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fba74102800 0x7fba7419a1f0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fba5c005940 tx=0x7fba5c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:59:31.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:31.174+0000 7fba70ff9640 1 -- 172.21.15.42:0/4057169957 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba5c03f030 con 0x7fba74102800 2024-08-30T08:59:31.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:31.175+0000 7fba79c53640 1 -- 172.21.15.42:0/4057169957 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba741a03f0 con 0x7fba74102800 2024-08-30T08:59:31.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:31.175+0000 7fba70ff9640 1 -- 172.21.15.42:0/4057169957 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fba5c003cb0 con 0x7fba74102800 2024-08-30T08:59:31.875 INFO:teuthology.orchestra.run.smithi042.stdout:34359738453 2024-08-30T08:59:31.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:31.175+0000 7fba70ff9640 1 -- 172.21.15.42:0/4057169957 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba5c035070 con 0x7fba74102800 2024-08-30T08:59:31.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:31.175+0000 7fba79c53640 1 -- 172.21.15.42:0/4057169957 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba741a6bb0 con 0x7fba74102800 2024-08-30T08:59:31.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:31.176+0000 7fba79c53640 1 -- 172.21.15.42:0/4057169957 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba74102c10 con 0x7fba74102800 2024-08-30T08:59:31.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:31.177+0000 7fba70ff9640 1 -- 172.21.15.42:0/4057169957 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) ==== 152628+0+0 (secure 0 0 0) 0x7fba5c042030 con 0x7fba74102800 2024-08-30T08:59:31.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:31.178+0000 7fba70ff9640 1 --2- 172.21.15.42:0/4057169957 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fba600b4bc0 0x7fba600b70d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:31.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:31.178+0000 7fba72ffd640 1 --2- 172.21.15.42:0/4057169957 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fba600b4bc0 0x7fba600b70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:31.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:31.178+0000 7fba70ff9640 1 -- 172.21.15.42:0/4057169957 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 9092+0+0 (secure 0 0 0) 0x7fba5c105e90 con 0x7fba74102800 2024-08-30T08:59:31.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:31.179+0000 7fba72ffd640 1 --2- 172.21.15.42:0/4057169957 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fba600b4bc0 0x7fba600b70d0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fba74100910 tx=0x7fba6400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:59:31.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:31.182+0000 7fba70ff9640 1 -- 172.21.15.42:0/4057169957 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7fba5c0cc450 con 0x7fba74102800 2024-08-30T08:59:31.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:31.371+0000 7fba79c53640 1 -- 172.21.15.42:0/4057169957 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7fba741a6e10 con 0x7fba74102800 2024-08-30T08:59:31.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:31.371+0000 7fba70ff9640 1 -- 172.21.15.42:0/4057169957 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fba5c0d0c10 con 0x7fba74102800 2024-08-30T08:59:31.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:31.373+0000 7fba79c53640 1 -- 172.21.15.42:0/4057169957 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fba600b4bc0 msgr2=0x7fba600b70d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:31.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:31.373+0000 7fba79c53640 1 --2- 172.21.15.42:0/4057169957 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fba600b4bc0 0x7fba600b70d0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fba74100910 tx=0x7fba6400b390 comp rx=0 tx=0).stop 2024-08-30T08:59:31.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:31.373+0000 7fba79c53640 1 -- 172.21.15.42:0/4057169957 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fba74102800 msgr2=0x7fba7419a1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:31.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:31.373+0000 7fba79c53640 1 --2- 172.21.15.42:0/4057169957 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fba74102800 0x7fba7419a1f0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fba5c005940 tx=0x7fba5c0042f0 comp rx=0 tx=0).stop 2024-08-30T08:59:31.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:31.373+0000 7fba79c53640 1 -- 172.21.15.42:0/4057169957 shutdown_connections 2024-08-30T08:59:31.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:31.373+0000 7fba79c53640 1 --2- 172.21.15.42:0/4057169957 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fba600b4bc0 0x7fba600b70d0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:31.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:31.373+0000 7fba79c53640 1 --2- 172.21.15.42:0/4057169957 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fba74108a90 0x7fba7419fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:31.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:31.373+0000 7fba79c53640 1 --2- 172.21.15.42:0/4057169957 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fba74108140 0x7fba7419a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:31.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:31.373+0000 7fba79c53640 1 --2- 172.21.15.42:0/4057169957 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fba74102800 0x7fba7419a1f0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:31.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:31.373+0000 7fba79c53640 1 -- 172.21.15.42:0/4057169957 >> 172.21.15.42:0/4057169957 conn(0x7fba740794d0 msgr2=0x7fba74101830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:59:31.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:31.373+0000 7fba79c53640 1 -- 172.21.15.42:0/4057169957 shutdown_connections 2024-08-30T08:59:31.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:31.373+0000 7fba79c53640 1 -- 172.21.15.42:0/4057169957 wait complete. 2024-08-30T08:59:31.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:59:31 smithi190 ceph-mon[22001]: from='client.? 172.21.15.42:0/2534681143' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 2} : dispatch 2024-08-30T08:59:31.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:59:31 smithi190 ceph-mon[22001]: from='client.? 172.21.15.42:0/2054534545' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 5} : dispatch 2024-08-30T08:59:31.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:59:31 smithi190 ceph-mon[22001]: from='client.? 172.21.15.42:0/4057169957' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 0} : dispatch 2024-08-30T08:59:32.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:59:31 smithi167 ceph-mon[22063]: from='client.? 172.21.15.42:0/2534681143' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 2} : dispatch 2024-08-30T08:59:32.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:59:31 smithi167 ceph-mon[22063]: from='client.? 172.21.15.42:0/2054534545' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 5} : dispatch 2024-08-30T08:59:32.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:59:31 smithi167 ceph-mon[22063]: from='client.? 172.21.15.42:0/4057169957' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 0} : dispatch 2024-08-30T08:59:32.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:32.652+0000 7fb92bb09640 1 -- 172.21.15.42:0/273323586 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb924104600 msgr2=0x7fb924104a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:32.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:32.652+0000 7fb92bb09640 1 --2- 172.21.15.42:0/273323586 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb924104600 0x7fb924104a10 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fb920009900 tx=0x7fb92002a410 comp rx=0 tx=0).stop 2024-08-30T08:59:32.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:32.652+0000 7fb92bb09640 1 -- 172.21.15.42:0/273323586 shutdown_connections 2024-08-30T08:59:32.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:32.652+0000 7fb92bb09640 1 --2- 172.21.15.42:0/273323586 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb924105030 0x7fb92410d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:32.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:32.652+0000 7fb92bb09640 1 --2- 172.21.15.42:0/273323586 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb92410dbb0 0x7fb924110090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:32.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:32.652+0000 7fb92bb09640 1 --2- 172.21.15.42:0/273323586 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb924104600 0x7fb924104a10 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:32.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:32.652+0000 7fb92bb09640 1 -- 172.21.15.42:0/273323586 >> 172.21.15.42:0/273323586 conn(0x7fb9240794d0 msgr2=0x7fb9240798e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:59:32.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:32.652+0000 7fb92bb09640 1 -- 172.21.15.42:0/273323586 shutdown_connections 2024-08-30T08:59:32.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:32.652+0000 7fb92bb09640 1 -- 172.21.15.42:0/273323586 wait complete. 2024-08-30T08:59:32.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:32.652+0000 7fb92bb09640 1 Processor -- start 2024-08-30T08:59:32.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:32.652+0000 7fb92bb09640 1 -- start start 2024-08-30T08:59:32.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:32.653+0000 7fb92bb09640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb924104600 0x7fb92419a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:32.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:32.653+0000 7fb92bb09640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb924105030 0x7fb92419a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:32.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:32.653+0000 7fb92bb09640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb92410dbb0 0x7fb92419fa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:32.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:32.653+0000 7fb92bb09640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fb924113810 con 0x7fb924105030 2024-08-30T08:59:32.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:32.653+0000 7fb92bb09640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb924113690 con 0x7fb924104600 2024-08-30T08:59:32.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:32.653+0000 7fb92bb09640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fb924113990 con 0x7fb92410dbb0 2024-08-30T08:59:32.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:32.653+0000 7fb9290a4640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb924105030 0x7fb92419a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:32.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:32.653+0000 7fb9290a4640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb924105030 0x7fb92419a7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:53350/0 (socket says 172.21.15.42:53350) 2024-08-30T08:59:32.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:32.653+0000 7fb9290a4640 1 -- 172.21.15.42:0/3237173874 learned_addr learned my addr 172.21.15.42:0/3237173874 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T08:59:32.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:32.653+0000 7fb92a0a6640 1 --2- 172.21.15.42:0/3237173874 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb92410dbb0 0x7fb92419fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:32.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:32.653+0000 7fb9298a5640 1 --2- 172.21.15.42:0/3237173874 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb924104600 0x7fb92419a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:32.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:32.653+0000 7fb9290a4640 1 -- 172.21.15.42:0/3237173874 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb92410dbb0 msgr2=0x7fb92419fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:32.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:32.653+0000 7fb9290a4640 1 --2- 172.21.15.42:0/3237173874 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb92410dbb0 0x7fb92419fa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:32.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:32.653+0000 7fb9290a4640 1 -- 172.21.15.42:0/3237173874 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb924104600 msgr2=0x7fb92419a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:32.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:32.653+0000 7fb9290a4640 1 --2- 172.21.15.42:0/3237173874 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb924104600 0x7fb92419a220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:32.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:32.653+0000 7fb9290a4640 1 -- 172.21.15.42:0/3237173874 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9241a00d0 con 0x7fb924105030 2024-08-30T08:59:32.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:32.653+0000 7fb9290a4640 1 --2- 172.21.15.42:0/3237173874 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb924105030 0x7fb92419a7c0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fb91400d740 tx=0x7fb91400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:59:32.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:32.654+0000 7fb91affd640 1 -- 172.21.15.42:0/3237173874 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb914017070 con 0x7fb924105030 2024-08-30T08:59:32.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:32.654+0000 7fb91affd640 1 -- 172.21.15.42:0/3237173874 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb91400de50 con 0x7fb924105030 2024-08-30T08:59:32.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:32.654+0000 7fb91affd640 1 -- 172.21.15.42:0/3237173874 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb91400f620 con 0x7fb924105030 2024-08-30T08:59:32.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:32.654+0000 7fb92bb09640 1 -- 172.21.15.42:0/3237173874 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9241a03c0 con 0x7fb924105030 2024-08-30T08:59:32.666 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:32.654+0000 7fb92bb09640 1 -- 172.21.15.42:0/3237173874 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9241a6be0 con 0x7fb924105030 2024-08-30T08:59:32.666 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:32.655+0000 7fb92bb09640 1 -- 172.21.15.42:0/3237173874 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb92406b280 con 0x7fb924105030 2024-08-30T08:59:32.666 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:32.657+0000 7fb91affd640 1 -- 172.21.15.42:0/3237173874 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) ==== 152628+0+0 (secure 0 0 0) 0x7fb91401a070 con 0x7fb924105030 2024-08-30T08:59:32.666 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:32.657+0000 7fb91affd640 1 --2- 172.21.15.42:0/3237173874 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fb9100b4bf0 0x7fb9100b7100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:32.666 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:32.658+0000 7fb9298a5640 1 --2- 172.21.15.42:0/3237173874 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fb9100b4bf0 0x7fb9100b7100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:32.666 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:32.658+0000 7fb91affd640 1 -- 172.21.15.42:0/3237173874 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 9092+0+0 (secure 0 0 0) 0x7fb9140e5010 con 0x7fb924105030 2024-08-30T08:59:32.666 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:32.658+0000 7fb9298a5640 1 --2- 172.21.15.42:0/3237173874 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fb9100b4bf0 0x7fb9100b7100 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fb920031440 tx=0x7fb920005d50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:59:32.666 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:32.661+0000 7fb91affd640 1 -- 172.21.15.42:0/3237173874 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7fb9140abc80 con 0x7fb924105030 2024-08-30T08:59:32.746 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:59:32 smithi042 ceph-mon[18652]: pgmap v334: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-08-30T08:59:32.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:32.856+0000 7fb92bb09640 1 -- 172.21.15.42:0/3237173874 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7fb924108d00 con 0x7fb924105030 2024-08-30T08:59:32.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:32.857+0000 7fb91affd640 1 -- 172.21.15.42:0/3237173874 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fb9140b41c0 con 0x7fb924105030 2024-08-30T08:59:32.860 INFO:teuthology.orchestra.run.smithi042.stdout:219043332142 2024-08-30T08:59:32.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:32.859+0000 7fb92bb09640 1 -- 172.21.15.42:0/3237173874 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fb9100b4bf0 msgr2=0x7fb9100b7100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:32.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:32.859+0000 7fb92bb09640 1 --2- 172.21.15.42:0/3237173874 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fb9100b4bf0 0x7fb9100b7100 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fb920031440 tx=0x7fb920005d50 comp rx=0 tx=0).stop 2024-08-30T08:59:32.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:32.859+0000 7fb92bb09640 1 -- 172.21.15.42:0/3237173874 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb924105030 msgr2=0x7fb92419a7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:32.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:32.859+0000 7fb92bb09640 1 --2- 172.21.15.42:0/3237173874 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb924105030 0x7fb92419a7c0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fb91400d740 tx=0x7fb91400dc00 comp rx=0 tx=0).stop 2024-08-30T08:59:32.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:32.859+0000 7fb92bb09640 1 -- 172.21.15.42:0/3237173874 shutdown_connections 2024-08-30T08:59:32.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:32.859+0000 7fb92bb09640 1 --2- 172.21.15.42:0/3237173874 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fb9100b4bf0 0x7fb9100b7100 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:32.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:32.859+0000 7fb92bb09640 1 --2- 172.21.15.42:0/3237173874 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb92410dbb0 0x7fb92419fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:32.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:32.859+0000 7fb92bb09640 1 --2- 172.21.15.42:0/3237173874 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb924105030 0x7fb92419a7c0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:32.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:32.859+0000 7fb92bb09640 1 --2- 172.21.15.42:0/3237173874 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb924104600 0x7fb92419a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:32.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:32.859+0000 7fb92bb09640 1 -- 172.21.15.42:0/3237173874 >> 172.21.15.42:0/3237173874 conn(0x7fb9240794d0 msgr2=0x7fb924100580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:59:32.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:32.859+0000 7fb92bb09640 1 -- 172.21.15.42:0/3237173874 shutdown_connections 2024-08-30T08:59:32.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:32.859+0000 7fb92bb09640 1 -- 172.21.15.42:0/3237173874 wait complete. 2024-08-30T08:59:32.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:59:32 smithi190 ceph-mon[22001]: pgmap v334: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-08-30T08:59:33.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:59:32 smithi167 ceph-mon[22063]: pgmap v334: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-08-30T08:59:33.104 INFO:tasks.cephadm.ceph_manager.ceph:need seq 150323855419 got 150323855421 for osd.5 2024-08-30T08:59:33.104 DEBUG:teuthology.parallel:result is None 2024-08-30T08:59:33.543 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738451 got 34359738453 for osd.0 2024-08-30T08:59:33.543 DEBUG:teuthology.parallel:result is None 2024-08-30T08:59:33.757 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T08:59:33.758 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T08:59:33.758 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T08:59:33.759 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T08:59:33.851 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:59:33 smithi042 ceph-mon[18652]: from='client.? 172.21.15.42:0/2747009913' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 6} : dispatch 2024-08-30T08:59:33.851 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:59:33 smithi042 ceph-mon[18652]: from='client.? 172.21.15.42:0/3237173874' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 8} : dispatch 2024-08-30T08:59:33.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:59:33 smithi190 ceph-mon[22001]: from='client.? 172.21.15.42:0/2747009913' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 6} : dispatch 2024-08-30T08:59:33.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:59:33 smithi190 ceph-mon[22001]: from='client.? 172.21.15.42:0/3237173874' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 8} : dispatch 2024-08-30T08:59:34.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:59:33 smithi167 ceph-mon[22063]: from='client.? 172.21.15.42:0/2747009913' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 6} : dispatch 2024-08-30T08:59:34.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:59:33 smithi167 ceph-mon[22063]: from='client.? 172.21.15.42:0/3237173874' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 8} : dispatch 2024-08-30T08:59:34.199 INFO:teuthology.orchestra.run.smithi042.stdout:287762808861 2024-08-30T08:59:34.199 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-08-30T08:59:34.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:32.543+0000 7fe742686640 1 -- 172.21.15.42:0/2933973733 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe73c102820 msgr2=0x7fe73c102c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:34.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:32.543+0000 7fe742686640 1 --2- 172.21.15.42:0/2933973733 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe73c102820 0x7fe73c102c30 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fe72c009900 tx=0x7fe72c02a410 comp rx=0 tx=0).stop 2024-08-30T08:59:34.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:32.543+0000 7fe742686640 1 -- 172.21.15.42:0/2933973733 shutdown_connections 2024-08-30T08:59:34.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:32.543+0000 7fe742686640 1 --2- 172.21.15.42:0/2933973733 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe73c108ab0 0x7fe73c10f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:34.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:32.543+0000 7fe742686640 1 --2- 172.21.15.42:0/2933973733 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe73c108160 0x7fe73c108570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:34.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:32.543+0000 7fe742686640 1 --2- 172.21.15.42:0/2933973733 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe73c102820 0x7fe73c102c30 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:34.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:32.543+0000 7fe742686640 1 -- 172.21.15.42:0/2933973733 >> 172.21.15.42:0/2933973733 conn(0x7fe73c0794e0 msgr2=0x7fe73c0798f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:59:34.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:32.543+0000 7fe742686640 1 -- 172.21.15.42:0/2933973733 shutdown_connections 2024-08-30T08:59:34.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:32.543+0000 7fe742686640 1 -- 172.21.15.42:0/2933973733 wait complete. 2024-08-30T08:59:34.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:32.544+0000 7fe742686640 1 Processor -- start 2024-08-30T08:59:34.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:32.544+0000 7fe742686640 1 -- start start 2024-08-30T08:59:34.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:32.544+0000 7fe742686640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe73c102820 0x7fe73c19a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:34.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:32.544+0000 7fe742686640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe73c108160 0x7fe73c19a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:34.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:32.544+0000 7fe742686640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe73c108ab0 0x7fe73c19fad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:34.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:32.544+0000 7fe742686640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fe73c113840 con 0x7fe73c108ab0 2024-08-30T08:59:34.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:32.544+0000 7fe742686640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe73c1136c0 con 0x7fe73c108160 2024-08-30T08:59:34.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:32.544+0000 7fe742686640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fe73c1139c0 con 0x7fe73c102820 2024-08-30T08:59:34.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:32.544+0000 7fe740c23640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe73c108ab0 0x7fe73c19fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:34.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:32.544+0000 7fe740c23640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe73c108ab0 0x7fe73c19fad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:53342/0 (socket says 172.21.15.42:53342) 2024-08-30T08:59:34.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:32.544+0000 7fe740c23640 1 -- 172.21.15.42:0/2747009913 learned_addr learned my addr 172.21.15.42:0/2747009913 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T08:59:34.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:32.544+0000 7fe733fff640 1 --2- 172.21.15.42:0/2747009913 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe73c108160 0x7fe73c19a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:34.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:32.544+0000 7fe73bfff640 1 --2- 172.21.15.42:0/2747009913 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe73c102820 0x7fe73c19a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:34.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:32.545+0000 7fe740c23640 1 -- 172.21.15.42:0/2747009913 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe73c102820 msgr2=0x7fe73c19a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:34.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:32.545+0000 7fe740c23640 1 --2- 172.21.15.42:0/2747009913 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe73c102820 0x7fe73c19a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:34.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:32.545+0000 7fe740c23640 1 -- 172.21.15.42:0/2747009913 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe73c108160 msgr2=0x7fe73c19a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:34.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:32.545+0000 7fe740c23640 1 --2- 172.21.15.42:0/2747009913 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe73c108160 0x7fe73c19a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:34.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:32.545+0000 7fe740c23640 1 -- 172.21.15.42:0/2747009913 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe73c1a0200 con 0x7fe73c108ab0 2024-08-30T08:59:34.279 INFO:teuthology.orchestra.run.smithi042.stdout:176093659192 2024-08-30T08:59:34.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:32.545+0000 7fe740c23640 1 --2- 172.21.15.42:0/2747009913 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe73c108ab0 0x7fe73c19fad0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fe73400cad0 tx=0x7fe73400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:59:34.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:32.545+0000 7fe739ffb640 1 -- 172.21.15.42:0/2747009913 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe73400ae20 con 0x7fe73c108ab0 2024-08-30T08:59:34.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:32.545+0000 7fe742686640 1 -- 172.21.15.42:0/2747009913 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe73c1a04f0 con 0x7fe73c108ab0 2024-08-30T08:59:34.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:32.545+0000 7fe739ffb640 1 -- 172.21.15.42:0/2747009913 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe734005020 con 0x7fe73c108ab0 2024-08-30T08:59:34.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:32.545+0000 7fe739ffb640 1 -- 172.21.15.42:0/2747009913 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe73400f6b0 con 0x7fe73c108ab0 2024-08-30T08:59:34.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:32.545+0000 7fe742686640 1 -- 172.21.15.42:0/2747009913 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe73c1a6c50 con 0x7fe73c108ab0 2024-08-30T08:59:34.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:32.546+0000 7fe742686640 1 -- 172.21.15.42:0/2747009913 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe73c103aa0 con 0x7fe73c108ab0 2024-08-30T08:59:34.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:32.548+0000 7fe739ffb640 1 -- 172.21.15.42:0/2747009913 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) ==== 152628+0+0 (secure 0 0 0) 0x7fe7340051d0 con 0x7fe73c108ab0 2024-08-30T08:59:34.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:32.549+0000 7fe739ffb640 1 --2- 172.21.15.42:0/2747009913 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fe7280b4bc0 0x7fe7280b70d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:34.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:32.549+0000 7fe73bfff640 1 --2- 172.21.15.42:0/2747009913 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fe7280b4bc0 0x7fe7280b70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:34.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:32.549+0000 7fe739ffb640 1 -- 172.21.15.42:0/2747009913 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 9092+0+0 (secure 0 0 0) 0x7fe7340e59d0 con 0x7fe73c108ab0 2024-08-30T08:59:34.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:32.549+0000 7fe73bfff640 1 --2- 172.21.15.42:0/2747009913 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fe7280b4bc0 0x7fe7280b70d0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fe72c031440 tx=0x7fe72c005d50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:59:34.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:32.552+0000 7fe739ffb640 1 -- 172.21.15.42:0/2747009913 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7fe734014030 con 0x7fe73c108ab0 2024-08-30T08:59:34.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:32.743+0000 7fe742686640 1 -- 172.21.15.42:0/2747009913 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7fe73c102c30 con 0x7fe73c108ab0 2024-08-30T08:59:34.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:32.744+0000 7fe739ffb640 1 -- 172.21.15.42:0/2747009913 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fe73c102c30 con 0x7fe73c108ab0 2024-08-30T08:59:34.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:32.747+0000 7fe742686640 1 -- 172.21.15.42:0/2747009913 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fe7280b4bc0 msgr2=0x7fe7280b70d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:34.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:32.747+0000 7fe742686640 1 --2- 172.21.15.42:0/2747009913 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fe7280b4bc0 0x7fe7280b70d0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fe72c031440 tx=0x7fe72c005d50 comp rx=0 tx=0).stop 2024-08-30T08:59:34.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:32.747+0000 7fe742686640 1 -- 172.21.15.42:0/2747009913 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe73c108ab0 msgr2=0x7fe73c19fad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:34.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:32.747+0000 7fe742686640 1 --2- 172.21.15.42:0/2747009913 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe73c108ab0 0x7fe73c19fad0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fe73400cad0 tx=0x7fe73400cf90 comp rx=0 tx=0).stop 2024-08-30T08:59:34.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:32.747+0000 7fe742686640 1 -- 172.21.15.42:0/2747009913 shutdown_connections 2024-08-30T08:59:34.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:32.747+0000 7fe742686640 1 --2- 172.21.15.42:0/2747009913 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fe7280b4bc0 0x7fe7280b70d0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:34.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:32.747+0000 7fe742686640 1 --2- 172.21.15.42:0/2747009913 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe73c108ab0 0x7fe73c19fad0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:34.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:32.747+0000 7fe742686640 1 --2- 172.21.15.42:0/2747009913 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe73c108160 0x7fe73c19a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:34.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:32.747+0000 7fe742686640 1 --2- 172.21.15.42:0/2747009913 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe73c102820 0x7fe73c19a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:34.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:32.747+0000 7fe742686640 1 -- 172.21.15.42:0/2747009913 >> 172.21.15.42:0/2747009913 conn(0x7fe73c0794e0 msgr2=0x7fe73c101270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:59:34.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:32.747+0000 7fe742686640 1 -- 172.21.15.42:0/2747009913 shutdown_connections 2024-08-30T08:59:34.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:32.747+0000 7fe742686640 1 -- 172.21.15.42:0/2747009913 wait complete. 2024-08-30T08:59:34.808 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:59:34 smithi042 ceph-mon[18652]: pgmap v335: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-08-30T08:59:34.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:59:34 smithi190 ceph-mon[22001]: pgmap v335: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-08-30T08:59:34.978 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:59:34 smithi167 ceph-mon[22063]: pgmap v335: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-08-30T08:59:35.126 INFO:tasks.cephadm.ceph_manager.ceph:need seq 219043332140 got 219043332142 for osd.8 2024-08-30T08:59:35.126 DEBUG:teuthology.parallel:result is None 2024-08-30T08:59:36.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:36.472+0000 7fa6b4609640 1 -- 172.21.15.42:0/2425898867 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa6ac109270 msgr2=0x7fa6ac109680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:36.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:36.472+0000 7fa6b4609640 1 --2- 172.21.15.42:0/2425898867 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa6ac109270 0x7fa6ac109680 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fa69c009900 tx=0x7fa69c02a410 comp rx=0 tx=0).stop 2024-08-30T08:59:36.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:36.473+0000 7fa6b4609640 1 -- 172.21.15.42:0/2425898867 shutdown_connections 2024-08-30T08:59:36.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:36.473+0000 7fa6b4609640 1 --2- 172.21.15.42:0/2425898867 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa6ac109bc0 0x7fa6ac1103f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:36.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:36.473+0000 7fa6b4609640 1 --2- 172.21.15.42:0/2425898867 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa6ac109270 0x7fa6ac109680 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:36.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:36.473+0000 7fa6b4609640 1 --2- 172.21.15.42:0/2425898867 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa6ac103930 0x7fa6ac103d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:36.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:36.473+0000 7fa6b4609640 1 -- 172.21.15.42:0/2425898867 >> 172.21.15.42:0/2425898867 conn(0x7fa6ac0ff3a0 msgr2=0x7fa6ac1017f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:59:36.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:36.473+0000 7fa6b4609640 1 -- 172.21.15.42:0/2425898867 shutdown_connections 2024-08-30T08:59:36.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:36.473+0000 7fa6b4609640 1 -- 172.21.15.42:0/2425898867 wait complete. 2024-08-30T08:59:36.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:36.473+0000 7fa6b4609640 1 Processor -- start 2024-08-30T08:59:36.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:36.473+0000 7fa6b4609640 1 -- start start 2024-08-30T08:59:36.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:36.474+0000 7fa6b4609640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa6ac103930 0x7fa6ac19a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:36.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:36.474+0000 7fa6b4609640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa6ac109270 0x7fa6ac19a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:36.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:36.474+0000 7fa6b4609640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa6ac109bc0 0x7fa6ac19fb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:36.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:36.474+0000 7fa6b4609640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fa6ac1138a0 con 0x7fa6ac109270 2024-08-30T08:59:36.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:36.474+0000 7fa6b4609640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa6ac113720 con 0x7fa6ac109bc0 2024-08-30T08:59:36.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:36.474+0000 7fa6b4609640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fa6ac113a20 con 0x7fa6ac103930 2024-08-30T08:59:36.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:36.474+0000 7fa6b1ba4640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa6ac109270 0x7fa6ac19a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:36.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:36.474+0000 7fa6b1ba4640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa6ac109270 0x7fa6ac19a870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:53364/0 (socket says 172.21.15.42:53364) 2024-08-30T08:59:36.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:36.474+0000 7fa6b1ba4640 1 -- 172.21.15.42:0/3747971199 learned_addr learned my addr 172.21.15.42:0/3747971199 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T08:59:36.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:36.474+0000 7fa6b23a5640 1 --2- 172.21.15.42:0/3747971199 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa6ac103930 0x7fa6ac19a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:36.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:36.474+0000 7fa6b2ba6640 1 --2- 172.21.15.42:0/3747971199 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa6ac109bc0 0x7fa6ac19fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:36.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:36.474+0000 7fa6b1ba4640 1 -- 172.21.15.42:0/3747971199 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa6ac103930 msgr2=0x7fa6ac19a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:36.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:36.474+0000 7fa6b1ba4640 1 --2- 172.21.15.42:0/3747971199 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa6ac103930 0x7fa6ac19a2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:36.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:36.474+0000 7fa6b1ba4640 1 -- 172.21.15.42:0/3747971199 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa6ac109bc0 msgr2=0x7fa6ac19fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:36.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:36.474+0000 7fa6b1ba4640 1 --2- 172.21.15.42:0/3747971199 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa6ac109bc0 0x7fa6ac19fb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:36.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:36.474+0000 7fa6b1ba4640 1 -- 172.21.15.42:0/3747971199 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6ac1a0180 con 0x7fa6ac109270 2024-08-30T08:59:36.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:36.474+0000 7fa6b2ba6640 1 --2- 172.21.15.42:0/3747971199 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa6ac109bc0 0x7fa6ac19fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-30T08:59:36.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:36.474+0000 7fa6b1ba4640 1 --2- 172.21.15.42:0/3747971199 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa6ac109270 0x7fa6ac19a870 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fa69c02a930 tx=0x7fa69c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:59:36.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:36.474+0000 7fa6a37fe640 1 -- 172.21.15.42:0/3747971199 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa69c03f030 con 0x7fa6ac109270 2024-08-30T08:59:36.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:36.475+0000 7fa6a37fe640 1 -- 172.21.15.42:0/3747971199 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa69c003aa0 con 0x7fa6ac109270 2024-08-30T08:59:36.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:36.475+0000 7fa6b4609640 1 -- 172.21.15.42:0/3747971199 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6ac1a0410 con 0x7fa6ac109270 2024-08-30T08:59:36.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:36.475+0000 7fa6a37fe640 1 -- 172.21.15.42:0/3747971199 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa69c035070 con 0x7fa6ac109270 2024-08-30T08:59:36.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:36.475+0000 7fa6b4609640 1 -- 172.21.15.42:0/3747971199 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6ac1a6c90 con 0x7fa6ac109270 2024-08-30T08:59:36.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:36.476+0000 7fa6b4609640 1 -- 172.21.15.42:0/3747971199 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6ac104bb0 con 0x7fa6ac109270 2024-08-30T08:59:36.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:36.478+0000 7fa6a37fe640 1 -- 172.21.15.42:0/3747971199 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) ==== 152628+0+0 (secure 0 0 0) 0x7fa69c042030 con 0x7fa6ac109270 2024-08-30T08:59:36.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:36.479+0000 7fa6a37fe640 1 --2- 172.21.15.42:0/3747971199 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fa6980b4bf0 0x7fa6980b7100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:36.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:36.479+0000 7fa6b23a5640 1 --2- 172.21.15.42:0/3747971199 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fa6980b4bf0 0x7fa6980b7100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:36.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:36.479+0000 7fa6a37fe640 1 -- 172.21.15.42:0/3747971199 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 9092+0+0 (secure 0 0 0) 0x7fa69c106270 con 0x7fa6ac109270 2024-08-30T08:59:36.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:36.479+0000 7fa6b23a5640 1 --2- 172.21.15.42:0/3747971199 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fa6980b4bf0 0x7fa6980b7100 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fa6a800abe0 tx=0x7fa6a800a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:59:36.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:36.482+0000 7fa6a37fe640 1 -- 172.21.15.42:0/3747971199 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7fa69c0cce80 con 0x7fa6ac109270 2024-08-30T08:59:36.616 INFO:tasks.cephadm.ceph_manager.ceph:need seq 176093659189 got 176093659192 for osd.6 2024-08-30T08:59:36.616 DEBUG:teuthology.parallel:result is None 2024-08-30T08:59:36.632 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:59:36 smithi042 ceph-mon[18652]: pgmap v336: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s 2024-08-30T08:59:36.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:36.670+0000 7fa6b4609640 1 -- 172.21.15.42:0/3747971199 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7fa6ac10b9b0 con 0x7fa6ac109270 2024-08-30T08:59:36.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:36.671+0000 7fa6a37fe640 1 -- 172.21.15.42:0/3747971199 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fa69c0d53c0 con 0x7fa6ac109270 2024-08-30T08:59:36.674 INFO:teuthology.orchestra.run.smithi042.stdout:244813135914 2024-08-30T08:59:36.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:36.673+0000 7fa6b4609640 1 -- 172.21.15.42:0/3747971199 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fa6980b4bf0 msgr2=0x7fa6980b7100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:36.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:36.673+0000 7fa6b4609640 1 --2- 172.21.15.42:0/3747971199 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fa6980b4bf0 0x7fa6980b7100 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fa6a800abe0 tx=0x7fa6a800a2d0 comp rx=0 tx=0).stop 2024-08-30T08:59:36.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:36.673+0000 7fa6b4609640 1 -- 172.21.15.42:0/3747971199 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa6ac109270 msgr2=0x7fa6ac19a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:36.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:36.673+0000 7fa6b4609640 1 --2- 172.21.15.42:0/3747971199 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa6ac109270 0x7fa6ac19a870 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fa69c02a930 tx=0x7fa69c004270 comp rx=0 tx=0).stop 2024-08-30T08:59:36.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:36.673+0000 7fa6b4609640 1 -- 172.21.15.42:0/3747971199 shutdown_connections 2024-08-30T08:59:36.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:36.673+0000 7fa6b4609640 1 --2- 172.21.15.42:0/3747971199 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fa6980b4bf0 0x7fa6980b7100 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:36.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:36.673+0000 7fa6b4609640 1 --2- 172.21.15.42:0/3747971199 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa6ac109bc0 0x7fa6ac19fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:36.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:36.673+0000 7fa6b4609640 1 --2- 172.21.15.42:0/3747971199 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa6ac109270 0x7fa6ac19a870 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:36.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:36.673+0000 7fa6b4609640 1 --2- 172.21.15.42:0/3747971199 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa6ac103930 0x7fa6ac19a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:36.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:36.673+0000 7fa6b4609640 1 -- 172.21.15.42:0/3747971199 >> 172.21.15.42:0/3747971199 conn(0x7fa6ac0ff3a0 msgr2=0x7fa6ac1017c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:59:36.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:36.673+0000 7fa6b4609640 1 -- 172.21.15.42:0/3747971199 shutdown_connections 2024-08-30T08:59:36.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:36.673+0000 7fa6b4609640 1 -- 172.21.15.42:0/3747971199 wait complete. 2024-08-30T08:59:36.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:36.814+0000 7fc5d5ac0640 1 -- 172.21.15.42:0/2751395513 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc5d0104750 msgr2=0x7fc5d0104b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:36.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:36.814+0000 7fc5d5ac0640 1 --2- 172.21.15.42:0/2751395513 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc5d0104750 0x7fc5d0104b60 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fc5c0009900 tx=0x7fc5c002a410 comp rx=0 tx=0).stop 2024-08-30T08:59:36.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:36.814+0000 7fc5d5ac0640 1 -- 172.21.15.42:0/2751395513 shutdown_connections 2024-08-30T08:59:36.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:36.814+0000 7fc5d5ac0640 1 --2- 172.21.15.42:0/2751395513 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc5d010dbb0 0x7fc5d01100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:36.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:36.814+0000 7fc5d5ac0640 1 --2- 172.21.15.42:0/2751395513 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc5d01050a0 0x7fc5d010d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:36.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:36.814+0000 7fc5d5ac0640 1 --2- 172.21.15.42:0/2751395513 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc5d0104750 0x7fc5d0104b60 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:36.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:36.814+0000 7fc5d5ac0640 1 -- 172.21.15.42:0/2751395513 >> 172.21.15.42:0/2751395513 conn(0x7fc5d00794d0 msgr2=0x7fc5d00798e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:59:36.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:36.814+0000 7fc5d5ac0640 1 -- 172.21.15.42:0/2751395513 shutdown_connections 2024-08-30T08:59:36.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:36.814+0000 7fc5d5ac0640 1 -- 172.21.15.42:0/2751395513 wait complete. 2024-08-30T08:59:36.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:36.815+0000 7fc5d5ac0640 1 Processor -- start 2024-08-30T08:59:36.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:36.815+0000 7fc5d5ac0640 1 -- start start 2024-08-30T08:59:36.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:36.815+0000 7fc5d5ac0640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc5d0104750 0x7fc5d019a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:36.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:36.815+0000 7fc5d5ac0640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc5d01050a0 0x7fc5d019a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:36.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:36.815+0000 7fc5d5ac0640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc5d010dbb0 0x7fc5d019faa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:36.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:36.815+0000 7fc5d5ac0640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fc5d0113810 con 0x7fc5d0104750 2024-08-30T08:59:36.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:36.815+0000 7fc5d5ac0640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc5d0113690 con 0x7fc5d010dbb0 2024-08-30T08:59:36.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:36.815+0000 7fc5d5ac0640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fc5d0113990 con 0x7fc5d01050a0 2024-08-30T08:59:36.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:36.815+0000 7fc5cf7fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc5d0104750 0x7fc5d019a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:36.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:36.815+0000 7fc5cf7fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc5d0104750 0x7fc5d019a260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:53384/0 (socket says 172.21.15.42:53384) 2024-08-30T08:59:36.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:36.815+0000 7fc5cf7fe640 1 -- 172.21.15.42:0/1759901006 learned_addr learned my addr 172.21.15.42:0/1759901006 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T08:59:36.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:36.815+0000 7fc5cffff640 1 --2- 172.21.15.42:0/1759901006 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc5d010dbb0 0x7fc5d019faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:36.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:36.815+0000 7fc5ceffd640 1 --2- 172.21.15.42:0/1759901006 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc5d01050a0 0x7fc5d019a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:36.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:36.816+0000 7fc5cf7fe640 1 -- 172.21.15.42:0/1759901006 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc5d01050a0 msgr2=0x7fc5d019a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:36.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:36.816+0000 7fc5cf7fe640 1 --2- 172.21.15.42:0/1759901006 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc5d01050a0 0x7fc5d019a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:36.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:36.816+0000 7fc5cf7fe640 1 -- 172.21.15.42:0/1759901006 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc5d010dbb0 msgr2=0x7fc5d019faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:36.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:36.816+0000 7fc5cf7fe640 1 --2- 172.21.15.42:0/1759901006 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc5d010dbb0 0x7fc5d019faa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:36.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:36.816+0000 7fc5cf7fe640 1 -- 172.21.15.42:0/1759901006 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5d01a01d0 con 0x7fc5d0104750 2024-08-30T08:59:36.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:36.816+0000 7fc5ceffd640 1 --2- 172.21.15.42:0/1759901006 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc5d01050a0 0x7fc5d019a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-30T08:59:36.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:36.816+0000 7fc5cffff640 1 --2- 172.21.15.42:0/1759901006 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc5d010dbb0 0x7fc5d019faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-30T08:59:36.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:36.816+0000 7fc5cf7fe640 1 --2- 172.21.15.42:0/1759901006 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc5d0104750 0x7fc5d019a260 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fc5c0032fd0 tx=0x7fc5c0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:59:36.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:36.816+0000 7fc5ccff9640 1 -- 172.21.15.42:0/1759901006 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5c003f030 con 0x7fc5d0104750 2024-08-30T08:59:36.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:36.816+0000 7fc5d5ac0640 1 -- 172.21.15.42:0/1759901006 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5d01a0460 con 0x7fc5d0104750 2024-08-30T08:59:36.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:36.816+0000 7fc5ccff9640 1 -- 172.21.15.42:0/1759901006 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc5c0004490 con 0x7fc5d0104750 2024-08-30T08:59:36.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:36.816+0000 7fc5ccff9640 1 -- 172.21.15.42:0/1759901006 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5c0035070 con 0x7fc5d0104750 2024-08-30T08:59:36.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:36.816+0000 7fc5d5ac0640 1 -- 172.21.15.42:0/1759901006 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5d01a6c20 con 0x7fc5d0104750 2024-08-30T08:59:36.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:36.817+0000 7fc5d5ac0640 1 -- 172.21.15.42:0/1759901006 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5d0104b60 con 0x7fc5d0104750 2024-08-30T08:59:36.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:36.819+0000 7fc5ccff9640 1 -- 172.21.15.42:0/1759901006 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) ==== 152628+0+0 (secure 0 0 0) 0x7fc5c0042030 con 0x7fc5d0104750 2024-08-30T08:59:36.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:36.819+0000 7fc5ccff9640 1 --2- 172.21.15.42:0/1759901006 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fc5b80b4bc0 0x7fc5b80b70d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:36.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:36.819+0000 7fc5ccff9640 1 -- 172.21.15.42:0/1759901006 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 9092+0+0 (secure 0 0 0) 0x7fc5c0106020 con 0x7fc5d0104750 2024-08-30T08:59:36.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:36.819+0000 7fc5ceffd640 1 --2- 172.21.15.42:0/1759901006 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fc5b80b4bc0 0x7fc5b80b70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:36.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:36.820+0000 7fc5ceffd640 1 --2- 172.21.15.42:0/1759901006 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fc5b80b4bc0 0x7fc5b80b70d0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fc5d010fa30 tx=0x7fc5c4005d30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:59:36.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:36.822+0000 7fc5ccff9640 1 -- 172.21.15.42:0/1759901006 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7fc5c00ccc90 con 0x7fc5d0104750 2024-08-30T08:59:36.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:59:36 smithi190 ceph-mon[22001]: pgmap v336: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s 2024-08-30T08:59:37.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:37.010+0000 7fc5d5ac0640 1 -- 172.21.15.42:0/1759901006 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7fc5d01a6e50 con 0x7fc5d0104750 2024-08-30T08:59:37.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:37.011+0000 7fc5ccff9640 1 -- 172.21.15.42:0/1759901006 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fc5c00cc590 con 0x7fc5d0104750 2024-08-30T08:59:37.014 INFO:teuthology.orchestra.run.smithi042.stdout:197568495668 2024-08-30T08:59:37.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:37.013+0000 7fc5d5ac0640 1 -- 172.21.15.42:0/1759901006 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fc5b80b4bc0 msgr2=0x7fc5b80b70d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:37.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:37.013+0000 7fc5d5ac0640 1 --2- 172.21.15.42:0/1759901006 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fc5b80b4bc0 0x7fc5b80b70d0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fc5d010fa30 tx=0x7fc5c4005d30 comp rx=0 tx=0).stop 2024-08-30T08:59:37.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:37.013+0000 7fc5d5ac0640 1 -- 172.21.15.42:0/1759901006 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc5d0104750 msgr2=0x7fc5d019a260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:37.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:37.013+0000 7fc5d5ac0640 1 --2- 172.21.15.42:0/1759901006 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc5d0104750 0x7fc5d019a260 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fc5c0032fd0 tx=0x7fc5c0004270 comp rx=0 tx=0).stop 2024-08-30T08:59:37.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:37.013+0000 7fc5d5ac0640 1 -- 172.21.15.42:0/1759901006 shutdown_connections 2024-08-30T08:59:37.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:37.013+0000 7fc5d5ac0640 1 --2- 172.21.15.42:0/1759901006 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fc5b80b4bc0 0x7fc5b80b70d0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:37.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:37.013+0000 7fc5d5ac0640 1 --2- 172.21.15.42:0/1759901006 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc5d01050a0 0x7fc5d019a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:37.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:37.013+0000 7fc5d5ac0640 1 --2- 172.21.15.42:0/1759901006 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc5d010dbb0 0x7fc5d019faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:37.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:37.013+0000 7fc5d5ac0640 1 --2- 172.21.15.42:0/1759901006 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc5d0104750 0x7fc5d019a260 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:37.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:37.013+0000 7fc5d5ac0640 1 -- 172.21.15.42:0/1759901006 >> 172.21.15.42:0/1759901006 conn(0x7fc5d00794d0 msgr2=0x7fc5d00ff2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:59:37.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:37.013+0000 7fc5d5ac0640 1 -- 172.21.15.42:0/1759901006 shutdown_connections 2024-08-30T08:59:37.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:37.013+0000 7fc5d5ac0640 1 -- 172.21.15.42:0/1759901006 wait complete. 2024-08-30T08:59:37.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:59:36 smithi167 ceph-mon[22063]: pgmap v336: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s 2024-08-30T08:59:37.713 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:59:37 smithi042 ceph-mon[18652]: from='client.? 172.21.15.42:0/3747971199' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 9} : dispatch 2024-08-30T08:59:37.714 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:59:37 smithi042 ceph-mon[18652]: from='client.? 172.21.15.42:0/1759901006' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 7} : dispatch 2024-08-30T08:59:37.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:59:37 smithi190 ceph-mon[22001]: from='client.? 172.21.15.42:0/3747971199' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 9} : dispatch 2024-08-30T08:59:37.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:59:37 smithi190 ceph-mon[22001]: from='client.? 172.21.15.42:0/1759901006' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 7} : dispatch 2024-08-30T08:59:37.986 INFO:tasks.cephadm.ceph_manager.ceph:need seq 261993005090 got 261993005090 for osd.10 2024-08-30T08:59:37.987 DEBUG:teuthology.parallel:result is None 2024-08-30T08:59:38.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:59:37 smithi167 ceph-mon[22063]: from='client.? 172.21.15.42:0/3747971199' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 9} : dispatch 2024-08-30T08:59:38.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:59:37 smithi167 ceph-mon[22063]: from='client.? 172.21.15.42:0/1759901006' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 7} : dispatch 2024-08-30T08:59:38.677 INFO:tasks.cephadm.ceph_manager.ceph:need seq 244813135911 got 244813135914 for osd.9 2024-08-30T08:59:38.678 DEBUG:teuthology.parallel:result is None 2024-08-30T08:59:38.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:37.539+0000 7f9da8959640 1 -- 172.21.15.42:0/778694064 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9da4103900 msgr2=0x7f9da4103d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:38.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:37.539+0000 7f9da8959640 1 --2- 172.21.15.42:0/778694064 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9da4103900 0x7f9da4103d10 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f9d94009ca0 tx=0x7f9d9402a8c0 comp rx=0 tx=0).stop 2024-08-30T08:59:38.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:37.539+0000 7f9da8959640 1 -- 172.21.15.42:0/778694064 shutdown_connections 2024-08-30T08:59:38.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:37.539+0000 7f9da8959640 1 --2- 172.21.15.42:0/778694064 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9da4109b90 0x7f9da41103c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:38.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:37.539+0000 7f9da8959640 1 --2- 172.21.15.42:0/778694064 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9da4109240 0x7f9da4109650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:38.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:37.539+0000 7f9da8959640 1 --2- 172.21.15.42:0/778694064 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9da4103900 0x7f9da4103d10 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:38.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:37.539+0000 7f9da8959640 1 -- 172.21.15.42:0/778694064 >> 172.21.15.42:0/778694064 conn(0x7f9da407b5f0 msgr2=0x7f9da407ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:59:38.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:37.539+0000 7f9da8959640 1 -- 172.21.15.42:0/778694064 shutdown_connections 2024-08-30T08:59:38.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:37.539+0000 7f9da8959640 1 -- 172.21.15.42:0/778694064 wait complete. 2024-08-30T08:59:38.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:37.540+0000 7f9da8959640 1 Processor -- start 2024-08-30T08:59:38.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:37.540+0000 7f9da8959640 1 -- start start 2024-08-30T08:59:38.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:37.540+0000 7f9da8959640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9da4103900 0x7f9da419a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:38.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:37.540+0000 7f9da8959640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9da4109240 0x7f9da419a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:38.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:37.540+0000 7f9da8959640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9da4109b90 0x7f9da419fa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:38.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:37.540+0000 7f9da8959640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f9da4113850 con 0x7f9da4109240 2024-08-30T08:59:38.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:37.540+0000 7f9da8959640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f9da41136d0 con 0x7f9da4109b90 2024-08-30T08:59:38.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:37.540+0000 7f9da8959640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f9da41139d0 con 0x7f9da4103900 2024-08-30T08:59:38.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:37.540+0000 7f9da1d9b640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9da4109240 0x7f9da419a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:38.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:37.540+0000 7f9da1d9b640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9da4109240 0x7f9da419a7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:53402/0 (socket says 172.21.15.42:53402) 2024-08-30T08:59:38.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:37.540+0000 7f9da1d9b640 1 -- 172.21.15.42:0/2478529885 learned_addr learned my addr 172.21.15.42:0/2478529885 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T08:59:38.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:37.540+0000 7f9da2d9d640 1 --2- 172.21.15.42:0/2478529885 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9da4109b90 0x7f9da419fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:38.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:37.540+0000 7f9da259c640 1 --2- 172.21.15.42:0/2478529885 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9da4103900 0x7f9da419a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:38.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:37.541+0000 7f9da1d9b640 1 -- 172.21.15.42:0/2478529885 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9da4103900 msgr2=0x7f9da419a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:38.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:37.541+0000 7f9da1d9b640 1 --2- 172.21.15.42:0/2478529885 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9da4103900 0x7f9da419a200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:38.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:37.541+0000 7f9da1d9b640 1 -- 172.21.15.42:0/2478529885 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9da4109b90 msgr2=0x7f9da419fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:38.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:37.541+0000 7f9da1d9b640 1 --2- 172.21.15.42:0/2478529885 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9da4109b90 0x7f9da419fa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:38.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:37.541+0000 7f9da1d9b640 1 -- 172.21.15.42:0/2478529885 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9da41a00b0 con 0x7f9da4109240 2024-08-30T08:59:38.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:37.541+0000 7f9da1d9b640 1 --2- 172.21.15.42:0/2478529885 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9da4109240 0x7f9da419a7a0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f9d8c00e9f0 tx=0x7f9d8c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:59:38.761 INFO:teuthology.orchestra.run.smithi042.stdout:103079215176 2024-08-30T08:59:38.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:37.541+0000 7f9d9b7fe640 1 -- 172.21.15.42:0/2478529885 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d8c00ccc0 con 0x7f9da4109240 2024-08-30T08:59:38.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:37.541+0000 7f9da8959640 1 -- 172.21.15.42:0/2478529885 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9da41a03a0 con 0x7f9da4109240 2024-08-30T08:59:38.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:37.541+0000 7f9d9b7fe640 1 -- 172.21.15.42:0/2478529885 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9d8c0108e0 con 0x7f9da4109240 2024-08-30T08:59:38.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:37.541+0000 7f9d9b7fe640 1 -- 172.21.15.42:0/2478529885 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d8c018860 con 0x7f9da4109240 2024-08-30T08:59:38.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:37.541+0000 7f9da8959640 1 -- 172.21.15.42:0/2478529885 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9da41a6bc0 con 0x7f9da4109240 2024-08-30T08:59:38.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:37.542+0000 7f9da8959640 1 -- 172.21.15.42:0/2478529885 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9da4103d10 con 0x7f9da4109240 2024-08-30T08:59:38.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:37.544+0000 7f9d9b7fe640 1 -- 172.21.15.42:0/2478529885 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) ==== 152628+0+0 (secure 0 0 0) 0x7f9d8c010430 con 0x7f9da4109240 2024-08-30T08:59:38.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:37.544+0000 7f9d9b7fe640 1 --2- 172.21.15.42:0/2478529885 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f9d900b4bf0 0x7f9d900b7100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:38.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:37.545+0000 7f9da259c640 1 --2- 172.21.15.42:0/2478529885 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f9d900b4bf0 0x7f9d900b7100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:38.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:37.545+0000 7f9d9b7fe640 1 -- 172.21.15.42:0/2478529885 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 9092+0+0 (secure 0 0 0) 0x7f9d8c0e5ef0 con 0x7f9da4109240 2024-08-30T08:59:38.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:37.545+0000 7f9da259c640 1 --2- 172.21.15.42:0/2478529885 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f9d900b4bf0 0x7f9d900b7100 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f9d94000f80 tx=0x7f9d94009360 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:59:38.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:37.548+0000 7f9d9b7fe640 1 -- 172.21.15.42:0/2478529885 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f9d8c0acb60 con 0x7f9da4109240 2024-08-30T08:59:38.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:37.741+0000 7f9da8959640 1 -- 172.21.15.42:0/2478529885 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f9da410b960 con 0x7f9da4109240 2024-08-30T08:59:38.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:37.742+0000 7f9d9b7fe640 1 -- 172.21.15.42:0/2478529885 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f9d8c0b50a0 con 0x7f9da4109240 2024-08-30T08:59:38.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:37.744+0000 7f9da8959640 1 -- 172.21.15.42:0/2478529885 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f9d900b4bf0 msgr2=0x7f9d900b7100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:38.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:37.744+0000 7f9da8959640 1 --2- 172.21.15.42:0/2478529885 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f9d900b4bf0 0x7f9d900b7100 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f9d94000f80 tx=0x7f9d94009360 comp rx=0 tx=0).stop 2024-08-30T08:59:38.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:37.744+0000 7f9da8959640 1 -- 172.21.15.42:0/2478529885 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9da4109240 msgr2=0x7f9da419a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:38.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:37.744+0000 7f9da8959640 1 --2- 172.21.15.42:0/2478529885 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9da4109240 0x7f9da419a7a0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f9d8c00e9f0 tx=0x7f9d8c00eeb0 comp rx=0 tx=0).stop 2024-08-30T08:59:38.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:37.744+0000 7f9da8959640 1 -- 172.21.15.42:0/2478529885 shutdown_connections 2024-08-30T08:59:38.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:37.744+0000 7f9da8959640 1 --2- 172.21.15.42:0/2478529885 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f9d900b4bf0 0x7f9d900b7100 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:38.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:37.744+0000 7f9da8959640 1 --2- 172.21.15.42:0/2478529885 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9da4109b90 0x7f9da419fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:38.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:37.744+0000 7f9da8959640 1 --2- 172.21.15.42:0/2478529885 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9da4109240 0x7f9da419a7a0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:38.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:37.744+0000 7f9da8959640 1 --2- 172.21.15.42:0/2478529885 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9da4103900 0x7f9da419a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:38.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:37.744+0000 7f9da8959640 1 -- 172.21.15.42:0/2478529885 >> 172.21.15.42:0/2478529885 conn(0x7f9da407b5f0 msgr2=0x7f9da4107330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:59:38.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:37.744+0000 7f9da8959640 1 -- 172.21.15.42:0/2478529885 shutdown_connections 2024-08-30T08:59:38.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:37.744+0000 7f9da8959640 1 -- 172.21.15.42:0/2478529885 wait complete. 2024-08-30T08:59:38.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:59:38 smithi042 ceph-mon[18652]: pgmap v337: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 5 op/s 2024-08-30T08:59:38.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:59:38 smithi042 ceph-mon[18652]: from='client.? 172.21.15.42:0/2478529885' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 3} : dispatch 2024-08-30T08:59:38.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:59:38 smithi190 ceph-mon[22001]: pgmap v337: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 5 op/s 2024-08-30T08:59:38.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:59:38 smithi190 ceph-mon[22001]: from='client.? 172.21.15.42:0/2478529885' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 3} : dispatch 2024-08-30T08:59:39.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:59:38 smithi167 ceph-mon[22063]: pgmap v337: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 5 op/s 2024-08-30T08:59:39.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:59:38 smithi167 ceph-mon[22063]: from='client.? 172.21.15.42:0/2478529885' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 3} : dispatch 2024-08-30T08:59:39.567 INFO:tasks.cephadm.ceph_manager.ceph:need seq 103079215173 got 103079215176 for osd.3 2024-08-30T08:59:39.567 DEBUG:teuthology.parallel:result is None 2024-08-30T08:59:39.738 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T08:59:39.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:39.739+0000 7fb8b95fc640 1 -- 172.21.15.42:0/333712970 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb8b4104f30 msgr2=0x7fb8b4105340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:39.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:39.739+0000 7fb8b95fc640 1 --2- 172.21.15.42:0/333712970 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb8b4104f30 0x7fb8b4105340 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fb8a8009900 tx=0x7fb8a802a410 comp rx=0 tx=0).stop 2024-08-30T08:59:39.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:39.739+0000 7fb8b95fc640 1 -- 172.21.15.42:0/333712970 shutdown_connections 2024-08-30T08:59:39.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:39.739+0000 7fb8b95fc640 1 --2- 172.21.15.42:0/333712970 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb8b4105880 0x7fb8b410f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:39.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:39.739+0000 7fb8b95fc640 1 --2- 172.21.15.42:0/333712970 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb8b4104f30 0x7fb8b4105340 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:39.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:39.739+0000 7fb8b95fc640 1 --2- 172.21.15.42:0/333712970 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb8b4102f60 0x7fb8b4103370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:39.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:39.739+0000 7fb8b95fc640 1 -- 172.21.15.42:0/333712970 >> 172.21.15.42:0/333712970 conn(0x7fb8b4077a00 msgr2=0x7fb8b4077e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:59:39.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:39.739+0000 7fb8b95fc640 1 -- 172.21.15.42:0/333712970 shutdown_connections 2024-08-30T08:59:39.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:39.739+0000 7fb8b95fc640 1 -- 172.21.15.42:0/333712970 wait complete. 2024-08-30T08:59:39.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:39.740+0000 7fb8b95fc640 1 Processor -- start 2024-08-30T08:59:39.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:39.741+0000 7fb8b95fc640 1 -- start start 2024-08-30T08:59:39.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:39.741+0000 7fb8b95fc640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb8b4102f60 0x7fb8b410fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:39.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:39.741+0000 7fb8b95fc640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb8b4104f30 0x7fb8b41102f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:39.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:39.741+0000 7fb8b95fc640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb8b4105880 0x7fb8b4110830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:39.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:39.741+0000 7fb8b95fc640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fb8b4113840 con 0x7fb8b4105880 2024-08-30T08:59:39.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:39.741+0000 7fb8b95fc640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb8b41136c0 con 0x7fb8b4104f30 2024-08-30T08:59:39.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:39.741+0000 7fb8b95fc640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fb8b41139c0 con 0x7fb8b4102f60 2024-08-30T08:59:39.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:39.741+0000 7fb8b37fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb8b4105880 0x7fb8b4110830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:39.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:39.741+0000 7fb8b37fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb8b4105880 0x7fb8b4110830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:40062/0 (socket says 172.21.15.42:40062) 2024-08-30T08:59:39.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:39.741+0000 7fb8b37fe640 1 -- 172.21.15.42:0/3091270176 learned_addr learned my addr 172.21.15.42:0/3091270176 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T08:59:39.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:39.741+0000 7fb8b27fc640 1 --2- 172.21.15.42:0/3091270176 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb8b4104f30 0x7fb8b41102f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:39.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:39.741+0000 7fb8b2ffd640 1 --2- 172.21.15.42:0/3091270176 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb8b4102f60 0x7fb8b410fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:39.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:39.742+0000 7fb8b37fe640 1 -- 172.21.15.42:0/3091270176 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb8b4102f60 msgr2=0x7fb8b410fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:39.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:39.742+0000 7fb8b37fe640 1 --2- 172.21.15.42:0/3091270176 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb8b4102f60 0x7fb8b410fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:39.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:39.742+0000 7fb8b37fe640 1 -- 172.21.15.42:0/3091270176 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb8b4104f30 msgr2=0x7fb8b41102f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:39.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:39.742+0000 7fb8b37fe640 1 --2- 172.21.15.42:0/3091270176 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb8b4104f30 0x7fb8b41102f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:39.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:39.742+0000 7fb8b37fe640 1 -- 172.21.15.42:0/3091270176 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8b41a8420 con 0x7fb8b4105880 2024-08-30T08:59:39.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:39.742+0000 7fb8b27fc640 1 --2- 172.21.15.42:0/3091270176 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb8b4104f30 0x7fb8b41102f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-30T08:59:39.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:39.742+0000 7fb8b37fe640 1 --2- 172.21.15.42:0/3091270176 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb8b4105880 0x7fb8b4110830 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fb8ac00d750 tx=0x7fb8ac00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:59:39.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:39.742+0000 7fb89bfff640 1 -- 172.21.15.42:0/3091270176 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8ac017070 con 0x7fb8b4105880 2024-08-30T08:59:39.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:39.742+0000 7fb8b95fc640 1 -- 172.21.15.42:0/3091270176 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8b41a8710 con 0x7fb8b4105880 2024-08-30T08:59:39.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:39.742+0000 7fb89bfff640 1 -- 172.21.15.42:0/3091270176 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb8ac0054d0 con 0x7fb8b4105880 2024-08-30T08:59:39.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:39.742+0000 7fb89bfff640 1 -- 172.21.15.42:0/3091270176 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8ac004c60 con 0x7fb8b4105880 2024-08-30T08:59:39.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:39.742+0000 7fb8b95fc640 1 -- 172.21.15.42:0/3091270176 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8b41a8c20 con 0x7fb8b4105880 2024-08-30T08:59:39.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:39.744+0000 7fb8b95fc640 1 -- 172.21.15.42:0/3091270176 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8b4103370 con 0x7fb8b4105880 2024-08-30T08:59:39.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:39.745+0000 7fb89bfff640 1 -- 172.21.15.42:0/3091270176 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) ==== 152628+0+0 (secure 0 0 0) 0x7fb8ac004e00 con 0x7fb8b4105880 2024-08-30T08:59:39.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:39.746+0000 7fb89bfff640 1 --2- 172.21.15.42:0/3091270176 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fb89c0b4c40 0x7fb89c0b7150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:39.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:39.746+0000 7fb8b2ffd640 1 --2- 172.21.15.42:0/3091270176 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fb89c0b4c40 0x7fb89c0b7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:39.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:39.746+0000 7fb89bfff640 1 -- 172.21.15.42:0/3091270176 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 9092+0+0 (secure 0 0 0) 0x7fb8ac0e4fe0 con 0x7fb8b4105880 2024-08-30T08:59:39.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:39.747+0000 7fb8b2ffd640 1 --2- 172.21.15.42:0/3091270176 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fb89c0b4c40 0x7fb89c0b7150 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fb8b4111800 tx=0x7fb8a40075b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:59:39.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:39.751+0000 7fb89bfff640 1 -- 172.21.15.42:0/3091270176 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7fb8ac0abc50 con 0x7fb8b4105880 2024-08-30T08:59:39.947 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:39.944+0000 7fb8b95fc640 1 -- 172.21.15.42:0/3091270176 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7fb8b410a300 con 0x7fb8b4105880 2024-08-30T08:59:39.947 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:39.944+0000 7fb89bfff640 1 -- 172.21.15.42:0/3091270176 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fb8ac0ab550 con 0x7fb8b4105880 2024-08-30T08:59:39.947 INFO:teuthology.orchestra.run.smithi042.stdout:55834574931 2024-08-30T08:59:39.949 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:39.946+0000 7fb8b95fc640 1 -- 172.21.15.42:0/3091270176 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fb89c0b4c40 msgr2=0x7fb89c0b7150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:39.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:39.946+0000 7fb8b95fc640 1 --2- 172.21.15.42:0/3091270176 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fb89c0b4c40 0x7fb89c0b7150 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fb8b4111800 tx=0x7fb8a40075b0 comp rx=0 tx=0).stop 2024-08-30T08:59:39.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:39.946+0000 7fb8b95fc640 1 -- 172.21.15.42:0/3091270176 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb8b4105880 msgr2=0x7fb8b4110830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:39.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:39.946+0000 7fb8b95fc640 1 --2- 172.21.15.42:0/3091270176 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb8b4105880 0x7fb8b4110830 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fb8ac00d750 tx=0x7fb8ac00dc10 comp rx=0 tx=0).stop 2024-08-30T08:59:39.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:39.946+0000 7fb8b95fc640 1 -- 172.21.15.42:0/3091270176 shutdown_connections 2024-08-30T08:59:39.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:39.946+0000 7fb8b95fc640 1 --2- 172.21.15.42:0/3091270176 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fb89c0b4c40 0x7fb89c0b7150 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:39.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:39.946+0000 7fb8b95fc640 1 --2- 172.21.15.42:0/3091270176 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb8b4105880 0x7fb8b4110830 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:39.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:39.946+0000 7fb8b95fc640 1 --2- 172.21.15.42:0/3091270176 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb8b4104f30 0x7fb8b41102f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:39.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:39.946+0000 7fb8b95fc640 1 --2- 172.21.15.42:0/3091270176 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb8b4102f60 0x7fb8b410fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:39.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:39.946+0000 7fb8b95fc640 1 -- 172.21.15.42:0/3091270176 >> 172.21.15.42:0/3091270176 conn(0x7fb8b4077a00 msgr2=0x7fb8b41061e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:59:39.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:39.947+0000 7fb8b95fc640 1 -- 172.21.15.42:0/3091270176 shutdown_connections 2024-08-30T08:59:39.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:39.947+0000 7fb8b95fc640 1 -- 172.21.15.42:0/3091270176 wait complete. 2024-08-30T08:59:40.190 INFO:tasks.cephadm.ceph_manager.ceph:need seq 197568495666 got 197568495668 for osd.7 2024-08-30T08:59:40.190 DEBUG:teuthology.parallel:result is None 2024-08-30T08:59:40.666 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:40.664+0000 7f9a6947c640 1 -- 172.21.15.42:0/1786049230 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9a64108ab0 msgr2=0x7f9a6410f2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:40.667 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:40.664+0000 7f9a6947c640 1 --2- 172.21.15.42:0/1786049230 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9a64108ab0 0x7f9a6410f2e0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f9a5c009ca0 tx=0x7f9a5c02a8c0 comp rx=0 tx=0).stop 2024-08-30T08:59:40.667 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:40.664+0000 7f9a6947c640 1 -- 172.21.15.42:0/1786049230 shutdown_connections 2024-08-30T08:59:40.667 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:40.664+0000 7f9a6947c640 1 --2- 172.21.15.42:0/1786049230 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9a64108ab0 0x7f9a6410f2e0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:40.667 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:40.664+0000 7f9a6947c640 1 --2- 172.21.15.42:0/1786049230 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9a64108160 0x7f9a64108570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:40.667 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:40.664+0000 7f9a6947c640 1 --2- 172.21.15.42:0/1786049230 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9a64102820 0x7f9a64102c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:40.667 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:40.664+0000 7f9a6947c640 1 -- 172.21.15.42:0/1786049230 >> 172.21.15.42:0/1786049230 conn(0x7f9a640794e0 msgr2=0x7f9a640798f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:59:40.667 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:40.664+0000 7f9a6947c640 1 -- 172.21.15.42:0/1786049230 shutdown_connections 2024-08-30T08:59:40.667 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:40.664+0000 7f9a6947c640 1 -- 172.21.15.42:0/1786049230 wait complete. 2024-08-30T08:59:40.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:40.665+0000 7f9a6947c640 1 Processor -- start 2024-08-30T08:59:40.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:40.666+0000 7f9a6947c640 1 -- start start 2024-08-30T08:59:40.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:40.666+0000 7f9a6947c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9a64102820 0x7f9a6419b060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:40.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:40.666+0000 7f9a6947c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9a64108160 0x7f9a6419b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:40.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:40.666+0000 7f9a6947c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9a64108ab0 0x7f9a6419bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:40.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:40.666+0000 7f9a6947c640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f9a64112710 con 0x7f9a64108ab0 2024-08-30T08:59:40.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:40.666+0000 7f9a6947c640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f9a64112590 con 0x7f9a64108160 2024-08-30T08:59:40.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:40.666+0000 7f9a6947c640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f9a64112890 con 0x7f9a64102820 2024-08-30T08:59:40.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:40.666+0000 7f9a637fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9a64108ab0 0x7f9a6419bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:40.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:40.666+0000 7f9a637fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9a64108ab0 0x7f9a6419bae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:40080/0 (socket says 172.21.15.42:40080) 2024-08-30T08:59:40.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:40.666+0000 7f9a637fe640 1 -- 172.21.15.42:0/1242711136 learned_addr learned my addr 172.21.15.42:0/1242711136 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T08:59:40.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:40.666+0000 7f9a62ffd640 1 --2- 172.21.15.42:0/1242711136 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9a64102820 0x7f9a6419b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:40.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:40.666+0000 7f9a627fc640 1 --2- 172.21.15.42:0/1242711136 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9a64108160 0x7f9a6419b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:40.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:40.666+0000 7f9a637fe640 1 -- 172.21.15.42:0/1242711136 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9a64102820 msgr2=0x7f9a6419b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:40.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:40.666+0000 7f9a637fe640 1 --2- 172.21.15.42:0/1242711136 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9a64102820 0x7f9a6419b060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:40.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:40.666+0000 7f9a637fe640 1 -- 172.21.15.42:0/1242711136 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9a64108160 msgr2=0x7f9a6419b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:40.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:40.666+0000 7f9a637fe640 1 --2- 172.21.15.42:0/1242711136 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9a64108160 0x7f9a6419b5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:40.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:40.666+0000 7f9a637fe640 1 -- 172.21.15.42:0/1242711136 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a641a62f0 con 0x7f9a64108ab0 2024-08-30T08:59:40.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:40.666+0000 7f9a627fc640 1 --2- 172.21.15.42:0/1242711136 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9a64108160 0x7f9a6419b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-30T08:59:40.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:40.667+0000 7f9a637fe640 1 --2- 172.21.15.42:0/1242711136 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9a64108ab0 0x7f9a6419bae0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f9a5c005cc0 tx=0x7f9a5c02f470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:59:40.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:40.667+0000 7f9a4bfff640 1 -- 172.21.15.42:0/1242711136 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a5c003cf0 con 0x7f9a64108ab0 2024-08-30T08:59:40.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:40.667+0000 7f9a4bfff640 1 -- 172.21.15.42:0/1242711136 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9a5c0044f0 con 0x7f9a64108ab0 2024-08-30T08:59:40.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:40.667+0000 7f9a4bfff640 1 -- 172.21.15.42:0/1242711136 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a5c02fd10 con 0x7f9a64108ab0 2024-08-30T08:59:40.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:40.667+0000 7f9a6947c640 1 -- 172.21.15.42:0/1242711136 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a641a6580 con 0x7f9a64108ab0 2024-08-30T08:59:40.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:40.667+0000 7f9a6947c640 1 -- 172.21.15.42:0/1242711136 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a641a6b10 con 0x7f9a64108ab0 2024-08-30T08:59:40.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:40.668+0000 7f9a6947c640 1 -- 172.21.15.42:0/1242711136 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a64103aa0 con 0x7f9a64108ab0 2024-08-30T08:59:40.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:40.669+0000 7f9a4bfff640 1 -- 172.21.15.42:0/1242711136 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) ==== 152628+0+0 (secure 0 0 0) 0x7f9a5c004040 con 0x7f9a64108ab0 2024-08-30T08:59:40.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:40.670+0000 7f9a4bfff640 1 --2- 172.21.15.42:0/1242711136 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f9a4c0b4c70 0x7f9a4c0b7180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:40.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:40.670+0000 7f9a62ffd640 1 --2- 172.21.15.42:0/1242711136 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f9a4c0b4c70 0x7f9a4c0b7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:40.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:40.670+0000 7f9a4bfff640 1 -- 172.21.15.42:0/1242711136 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 9092+0+0 (secure 0 0 0) 0x7f9a5c106be0 con 0x7f9a64108ab0 2024-08-30T08:59:40.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:40.671+0000 7f9a62ffd640 1 --2- 172.21.15.42:0/1242711136 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f9a4c0b4c70 0x7f9a4c0b7180 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f9a540058c0 tx=0x7f9a5400a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:59:40.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:40.673+0000 7f9a4bfff640 1 -- 172.21.15.42:0/1242711136 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f9a5c0cd850 con 0x7f9a64108ab0 2024-08-30T08:59:40.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:40.860+0000 7f9a6947c640 1 -- 172.21.15.42:0/1242711136 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7f9a6406bcf0 con 0x7f9a64108ab0 2024-08-30T08:59:40.862 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:59:40 smithi042 ceph-mon[18652]: pgmap v338: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s 2024-08-30T08:59:40.863 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:59:40 smithi042 ceph-mon[18652]: from='client.? 172.21.15.42:0/3091270176' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 1} : dispatch 2024-08-30T08:59:40.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:40.861+0000 7f9a4bfff640 1 -- 172.21.15.42:0/1242711136 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f9a6406bcf0 con 0x7f9a64108ab0 2024-08-30T08:59:40.863 INFO:teuthology.orchestra.run.smithi042.stdout:287762808863 2024-08-30T08:59:40.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:40.863+0000 7f9a6947c640 1 -- 172.21.15.42:0/1242711136 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f9a4c0b4c70 msgr2=0x7f9a4c0b7180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:40.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:40.863+0000 7f9a6947c640 1 --2- 172.21.15.42:0/1242711136 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f9a4c0b4c70 0x7f9a4c0b7180 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f9a540058c0 tx=0x7f9a5400a2d0 comp rx=0 tx=0).stop 2024-08-30T08:59:40.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:40.863+0000 7f9a6947c640 1 -- 172.21.15.42:0/1242711136 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9a64108ab0 msgr2=0x7f9a6419bae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:40.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:40.863+0000 7f9a6947c640 1 --2- 172.21.15.42:0/1242711136 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9a64108ab0 0x7f9a6419bae0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f9a5c005cc0 tx=0x7f9a5c02f470 comp rx=0 tx=0).stop 2024-08-30T08:59:40.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:40.863+0000 7f9a6947c640 1 -- 172.21.15.42:0/1242711136 shutdown_connections 2024-08-30T08:59:40.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:40.863+0000 7f9a6947c640 1 --2- 172.21.15.42:0/1242711136 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f9a4c0b4c70 0x7f9a4c0b7180 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:40.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:40.863+0000 7f9a6947c640 1 --2- 172.21.15.42:0/1242711136 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9a64108ab0 0x7f9a6419bae0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:40.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:40.863+0000 7f9a6947c640 1 --2- 172.21.15.42:0/1242711136 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9a64108160 0x7f9a6419b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:40.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:40.863+0000 7f9a6947c640 1 --2- 172.21.15.42:0/1242711136 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9a64102820 0x7f9a6419b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:40.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:40.863+0000 7f9a6947c640 1 -- 172.21.15.42:0/1242711136 >> 172.21.15.42:0/1242711136 conn(0x7f9a640794e0 msgr2=0x7f9a641041d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:59:40.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:40.863+0000 7f9a6947c640 1 -- 172.21.15.42:0/1242711136 shutdown_connections 2024-08-30T08:59:40.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:40.863+0000 7f9a6947c640 1 -- 172.21.15.42:0/1242711136 wait complete. 2024-08-30T08:59:40.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:59:40 smithi190 ceph-mon[22001]: pgmap v338: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s 2024-08-30T08:59:40.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:59:40 smithi190 ceph-mon[22001]: from='client.? 172.21.15.42:0/3091270176' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 1} : dispatch 2024-08-30T08:59:41.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:59:40 smithi167 ceph-mon[22063]: pgmap v338: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s 2024-08-30T08:59:41.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:59:40 smithi167 ceph-mon[22063]: from='client.? 172.21.15.42:0/3091270176' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 1} : dispatch 2024-08-30T08:59:41.636 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574927 got 55834574931 for osd.1 2024-08-30T08:59:41.637 DEBUG:teuthology.parallel:result is None 2024-08-30T08:59:41.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:59:41 smithi042 ceph-mon[18652]: from='client.? 172.21.15.42:0/1242711136' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 11} : dispatch 2024-08-30T08:59:41.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:59:41 smithi190 ceph-mon[22001]: from='client.? 172.21.15.42:0/1242711136' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 11} : dispatch 2024-08-30T08:59:42.038 INFO:tasks.cephadm.ceph_manager.ceph:need seq 287762808861 got 287762808863 for osd.11 2024-08-30T08:59:42.039 DEBUG:teuthology.parallel:result is None 2024-08-30T08:59:42.039 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-08-30T08:59:42.039 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph pg dump --format=json 2024-08-30T08:59:42.076 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:59:41 smithi167 ceph-mon[22063]: from='client.? 172.21.15.42:0/1242711136' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 11} : dispatch 2024-08-30T08:59:42.293 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T08:59:42.749 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:59:42 smithi042 ceph-mon[18652]: pgmap v339: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-08-30T08:59:42.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:42.839+0000 7f220d540640 1 -- 172.21.15.42:0/724876589 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2208102840 msgr2=0x7f2208102c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:42.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:42.839+0000 7f220d540640 1 --2- 172.21.15.42:0/724876589 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2208102840 0x7f2208102c50 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f21f8009900 tx=0x7f21f802a410 comp rx=0 tx=0).stop 2024-08-30T08:59:42.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:42.840+0000 7f220d540640 1 -- 172.21.15.42:0/724876589 shutdown_connections 2024-08-30T08:59:42.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:42.840+0000 7f220d540640 1 --2- 172.21.15.42:0/724876589 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2208108ad0 0x7f220810f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:42.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:42.840+0000 7f220d540640 1 --2- 172.21.15.42:0/724876589 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2208108180 0x7f2208108590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:42.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:42.840+0000 7f220d540640 1 --2- 172.21.15.42:0/724876589 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2208102840 0x7f2208102c50 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:42.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:42.840+0000 7f220d540640 1 -- 172.21.15.42:0/724876589 >> 172.21.15.42:0/724876589 conn(0x7f22080794f0 msgr2=0x7f2208079900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:59:42.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:42.840+0000 7f220d540640 1 -- 172.21.15.42:0/724876589 shutdown_connections 2024-08-30T08:59:42.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:42.840+0000 7f220d540640 1 -- 172.21.15.42:0/724876589 wait complete. 2024-08-30T08:59:42.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:42.840+0000 7f220d540640 1 Processor -- start 2024-08-30T08:59:42.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:42.840+0000 7f220d540640 1 -- start start 2024-08-30T08:59:42.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:42.841+0000 7f220d540640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2208102840 0x7f2208198ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:42.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:42.841+0000 7f220d540640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2208108180 0x7f2208199420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:42.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:42.841+0000 7f220d540640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2208108ad0 0x7f2208199960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:42.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:42.841+0000 7f220d540640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f2208112780 con 0x7f2208108ad0 2024-08-30T08:59:42.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:42.841+0000 7f220d540640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f2208112600 con 0x7f2208102840 2024-08-30T08:59:42.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:42.841+0000 7f220d540640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f2208112900 con 0x7f2208108180 2024-08-30T08:59:42.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:42.841+0000 7f22077fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2208108ad0 0x7f2208199960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:42.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:42.841+0000 7f2206ffd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2208102840 0x7f2208198ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:42.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:42.841+0000 7f2206ffd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2208102840 0x7f2208198ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:60706/0 (socket says 172.21.15.42:60706) 2024-08-30T08:59:42.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:42.841+0000 7f22077fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2208108ad0 0x7f2208199960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:40098/0 (socket says 172.21.15.42:40098) 2024-08-30T08:59:42.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:42.841+0000 7f2206ffd640 1 -- 172.21.15.42:0/2730145643 learned_addr learned my addr 172.21.15.42:0/2730145643 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T08:59:42.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:42.841+0000 7f21fffff640 1 --2- 172.21.15.42:0/2730145643 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2208108180 0x7f2208199420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:42.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:42.841+0000 7f22077fe640 1 -- 172.21.15.42:0/2730145643 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2208108180 msgr2=0x7f2208199420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:42.846 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:42.841+0000 7f22077fe640 1 --2- 172.21.15.42:0/2730145643 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2208108180 0x7f2208199420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:42.846 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:42.841+0000 7f22077fe640 1 -- 172.21.15.42:0/2730145643 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2208102840 msgr2=0x7f2208198ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:42.846 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:42.841+0000 7f22077fe640 1 --2- 172.21.15.42:0/2730145643 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2208102840 0x7f2208198ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:42.846 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:42.841+0000 7f22077fe640 1 -- 172.21.15.42:0/2730145643 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22081a8590 con 0x7f2208108ad0 2024-08-30T08:59:42.846 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:42.841+0000 7f22077fe640 1 --2- 172.21.15.42:0/2730145643 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2208108ad0 0x7f2208199960 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f220000ca30 tx=0x7f220000cef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:59:42.846 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:42.842+0000 7f2204ff9640 1 -- 172.21.15.42:0/2730145643 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2200007d90 con 0x7f2208108ad0 2024-08-30T08:59:42.846 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:42.842+0000 7f220d540640 1 -- 172.21.15.42:0/2730145643 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22081a8880 con 0x7f2208108ad0 2024-08-30T08:59:42.846 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:42.842+0000 7f2204ff9640 1 -- 172.21.15.42:0/2730145643 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2200005020 con 0x7f2208108ad0 2024-08-30T08:59:42.846 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:42.842+0000 7f2204ff9640 1 -- 172.21.15.42:0/2730145643 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f220000f690 con 0x7f2208108ad0 2024-08-30T08:59:42.846 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:42.842+0000 7f220d540640 1 -- 172.21.15.42:0/2730145643 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22081a8d70 con 0x7f2208108ad0 2024-08-30T08:59:42.846 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:42.843+0000 7f220d540640 1 -- 172.21.15.42:0/2730145643 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2208103ac0 con 0x7f2208108ad0 2024-08-30T08:59:42.847 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:42.845+0000 7f2204ff9640 1 -- 172.21.15.42:0/2730145643 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) ==== 152628+0+0 (secure 0 0 0) 0x7f22000051d0 con 0x7f2208108ad0 2024-08-30T08:59:42.847 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:42.846+0000 7f2204ff9640 1 --2- 172.21.15.42:0/2730145643 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f21f40b4bc0 0x7f21f40b70d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:42.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:42.846+0000 7f2206ffd640 1 --2- 172.21.15.42:0/2730145643 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f21f40b4bc0 0x7f21f40b70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:42.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:42.846+0000 7f2204ff9640 1 -- 172.21.15.42:0/2730145643 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 9092+0+0 (secure 0 0 0) 0x7f22000e5c80 con 0x7f2208108ad0 2024-08-30T08:59:42.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:42.847+0000 7f2206ffd640 1 --2- 172.21.15.42:0/2730145643 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f21f40b4bc0 0x7f21f40b70d0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f21f8031440 tx=0x7f21f80058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:59:42.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:42.849+0000 7f2204ff9640 1 -- 172.21.15.42:0/2730145643 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f2200014070 con 0x7f2208108ad0 2024-08-30T08:59:42.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:59:42 smithi190 ceph-mon[22001]: pgmap v339: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-08-30T08:59:43.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:43.030+0000 7f220d540640 1 -- 172.21.15.42:0/2730145643 --> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f22080635a0 con 0x7f21f40b4bc0 2024-08-30T08:59:43.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:43.035+0000 7f2204ff9640 1 -- 172.21.15.42:0/2730145643 <== mgr.14150 v2:172.21.15.42:6800/3005140902 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340326 (secure 0 0 0) 0x7f22080635a0 con 0x7f21f40b4bc0 2024-08-30T08:59:43.041 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-08-30T08:59:43.042 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-08-30T08:59:43.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:43.040+0000 7f220d540640 1 -- 172.21.15.42:0/2730145643 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f21f40b4bc0 msgr2=0x7f21f40b70d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:43.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:43.040+0000 7f220d540640 1 --2- 172.21.15.42:0/2730145643 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f21f40b4bc0 0x7f21f40b70d0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f21f8031440 tx=0x7f21f80058d0 comp rx=0 tx=0).stop 2024-08-30T08:59:43.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:43.040+0000 7f220d540640 1 -- 172.21.15.42:0/2730145643 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2208108ad0 msgr2=0x7f2208199960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:43.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:43.040+0000 7f220d540640 1 --2- 172.21.15.42:0/2730145643 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2208108ad0 0x7f2208199960 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f220000ca30 tx=0x7f220000cef0 comp rx=0 tx=0).stop 2024-08-30T08:59:43.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:43.040+0000 7f220d540640 1 -- 172.21.15.42:0/2730145643 shutdown_connections 2024-08-30T08:59:43.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:43.040+0000 7f220d540640 1 --2- 172.21.15.42:0/2730145643 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f21f40b4bc0 0x7f21f40b70d0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:43.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:43.040+0000 7f220d540640 1 --2- 172.21.15.42:0/2730145643 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2208108180 0x7f2208199420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:43.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:43.040+0000 7f220d540640 1 --2- 172.21.15.42:0/2730145643 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2208108ad0 0x7f2208199960 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:43.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:43.040+0000 7f220d540640 1 --2- 172.21.15.42:0/2730145643 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2208102840 0x7f2208198ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:43.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:43.040+0000 7f220d540640 1 -- 172.21.15.42:0/2730145643 >> 172.21.15.42:0/2730145643 conn(0x7f22080794f0 msgr2=0x7f2208106330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:59:43.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:43.040+0000 7f220d540640 1 -- 172.21.15.42:0/2730145643 shutdown_connections 2024-08-30T08:59:43.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:43.040+0000 7f220d540640 1 -- 172.21.15.42:0/2730145643 wait complete. 2024-08-30T08:59:43.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:59:42 smithi167 ceph-mon[22063]: pgmap v339: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-08-30T08:59:43.711 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":340,"stamp":"2024-08-30T08:59:42.951644+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":659054,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":415,"num_read_kb":306,"num_write":296,"num_write_kb":2272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":240,"ondisk_log_size":240,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":374580,"kb_used_data":12848,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124567244,"statfs":{"total":1151940427776,"available":1151556857856,"internally_reserved":0,"allocated":13156352,"data_stored":7417580,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":370193721},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":90,"num_read_kb":45,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003698"},"pg_stats":[{"pgid":"2.1f","version":"80'9","reported_seq":45,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.559175+0000","last_change":"2024-08-30T08:58:00.388595+0000","last_active":"2024-08-30T08:58:46.559175+0000","last_peered":"2024-08-30T08:58:46.559175+0000","last_clean":"2024-08-30T08:58:46.559175+0000","last_became_active":"2024-08-30T08:58:00.388215+0000","last_became_peered":"2024-08-30T08:58:00.388215+0000","last_unstale":"2024-08-30T08:58:46.559175+0000","last_undegraded":"2024-08-30T08:58:46.559175+0000","last_fullsized":"2024-08-30T08:58:46.559175+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T13:00:06.109935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.555091+0000","last_change":"2024-08-30T08:58:02.416042+0000","last_active":"2024-08-30T08:58:46.555091+0000","last_peered":"2024-08-30T08:58:46.555091+0000","last_clean":"2024-08-30T08:58:46.555091+0000","last_became_active":"2024-08-30T08:58:02.415507+0000","last_became_peered":"2024-08-30T08:58:02.415507+0000","last_unstale":"2024-08-30T08:58:46.555091+0000","last_undegraded":"2024-08-30T08:58:46.555091+0000","last_fullsized":"2024-08-30T08:58:46.555091+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T14:33:57.192004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.554665+0000","last_change":"2024-08-30T08:58:09.448785+0000","last_active":"2024-08-30T08:58:46.554665+0000","last_peered":"2024-08-30T08:58:46.554665+0000","last_clean":"2024-08-30T08:58:46.554665+0000","last_became_active":"2024-08-30T08:58:09.448191+0000","last_became_peered":"2024-08-30T08:58:09.448191+0000","last_unstale":"2024-08-30T08:58:46.554665+0000","last_undegraded":"2024-08-30T08:58:46.554665+0000","last_fullsized":"2024-08-30T08:58:46.554665+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T13:37:19.148201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"80'2","reported_seq":37,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.555518+0000","last_change":"2024-08-30T08:58:00.480406+0000","last_active":"2024-08-30T08:58:46.555518+0000","last_peered":"2024-08-30T08:58:46.555518+0000","last_clean":"2024-08-30T08:58:46.555518+0000","last_became_active":"2024-08-30T08:58:00.479896+0000","last_became_peered":"2024-08-30T08:58:00.479896+0000","last_unstale":"2024-08-30T08:58:46.555518+0000","last_undegraded":"2024-08-30T08:58:46.555518+0000","last_fullsized":"2024-08-30T08:58:46.555518+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T15:16:15.316343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.557006+0000","last_change":"2024-08-30T08:58:02.406915+0000","last_active":"2024-08-30T08:58:46.557006+0000","last_peered":"2024-08-30T08:58:46.557006+0000","last_clean":"2024-08-30T08:58:46.557006+0000","last_became_active":"2024-08-30T08:58:02.406557+0000","last_became_peered":"2024-08-30T08:58:02.406557+0000","last_unstale":"2024-08-30T08:58:46.557006+0000","last_undegraded":"2024-08-30T08:58:46.557006+0000","last_fullsized":"2024-08-30T08:58:46.557006+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T19:15:30.847212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:47.089607+0000","last_change":"2024-08-30T08:58:09.446974+0000","last_active":"2024-08-30T08:58:47.089607+0000","last_peered":"2024-08-30T08:58:47.089607+0000","last_clean":"2024-08-30T08:58:47.089607+0000","last_became_active":"2024-08-30T08:58:09.446173+0000","last_became_peered":"2024-08-30T08:58:09.446173+0000","last_unstale":"2024-08-30T08:58:47.089607+0000","last_undegraded":"2024-08-30T08:58:47.089607+0000","last_fullsized":"2024-08-30T08:58:47.089607+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T12:13:41.276825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"80'4","reported_seq":39,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.557615+0000","last_change":"2024-08-30T08:58:00.390609+0000","last_active":"2024-08-30T08:58:46.557615+0000","last_peered":"2024-08-30T08:58:46.557615+0000","last_clean":"2024-08-30T08:58:46.557615+0000","last_became_active":"2024-08-30T08:58:00.390229+0000","last_became_peered":"2024-08-30T08:58:00.390229+0000","last_unstale":"2024-08-30T08:58:46.557615+0000","last_undegraded":"2024-08-30T08:58:46.557615+0000","last_fullsized":"2024-08-30T08:58:46.557615+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T11:58:25.037431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.822738+0000","last_change":"2024-08-30T08:58:02.415594+0000","last_active":"2024-08-30T08:58:46.822738+0000","last_peered":"2024-08-30T08:58:46.822738+0000","last_clean":"2024-08-30T08:58:46.822738+0000","last_became_active":"2024-08-30T08:58:02.415066+0000","last_became_peered":"2024-08-30T08:58:02.415066+0000","last_unstale":"2024-08-30T08:58:46.822738+0000","last_undegraded":"2024-08-30T08:58:46.822738+0000","last_fullsized":"2024-08-30T08:58:46.822738+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T12:07:27.668442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:47.089672+0000","last_change":"2024-08-30T08:58:09.444403+0000","last_active":"2024-08-30T08:58:47.089672+0000","last_peered":"2024-08-30T08:58:47.089672+0000","last_clean":"2024-08-30T08:58:47.089672+0000","last_became_active":"2024-08-30T08:58:09.443914+0000","last_became_peered":"2024-08-30T08:58:09.443914+0000","last_unstale":"2024-08-30T08:58:47.089672+0000","last_undegraded":"2024-08-30T08:58:47.089672+0000","last_fullsized":"2024-08-30T08:58:47.089672+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T14:17:43.866352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"73'2","reported_seq":37,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.557996+0000","last_change":"2024-08-30T08:58:00.392108+0000","last_active":"2024-08-30T08:58:46.557996+0000","last_peered":"2024-08-30T08:58:46.557996+0000","last_clean":"2024-08-30T08:58:46.557996+0000","last_became_active":"2024-08-30T08:58:00.391157+0000","last_became_peered":"2024-08-30T08:58:00.391157+0000","last_unstale":"2024-08-30T08:58:46.557996+0000","last_undegraded":"2024-08-30T08:58:46.557996+0000","last_fullsized":"2024-08-30T08:58:46.557996+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T17:21:03.369776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.774221+0000","last_change":"2024-08-30T08:58:02.405550+0000","last_active":"2024-08-30T08:58:46.774221+0000","last_peered":"2024-08-30T08:58:46.774221+0000","last_clean":"2024-08-30T08:58:46.774221+0000","last_became_active":"2024-08-30T08:58:02.405129+0000","last_became_peered":"2024-08-30T08:58:02.405129+0000","last_unstale":"2024-08-30T08:58:46.774221+0000","last_undegraded":"2024-08-30T08:58:46.774221+0000","last_fullsized":"2024-08-30T08:58:46.774221+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T11:36:47.624026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:47.089364+0000","last_change":"2024-08-30T08:58:09.445746+0000","last_active":"2024-08-30T08:58:47.089364+0000","last_peered":"2024-08-30T08:58:47.089364+0000","last_clean":"2024-08-30T08:58:47.089364+0000","last_became_active":"2024-08-30T08:58:09.445257+0000","last_became_peered":"2024-08-30T08:58:09.445257+0000","last_unstale":"2024-08-30T08:58:47.089364+0000","last_undegraded":"2024-08-30T08:58:47.089364+0000","last_fullsized":"2024-08-30T08:58:47.089364+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T18:02:52.007131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:47.090324+0000","last_change":"2024-08-30T08:58:02.408213+0000","last_active":"2024-08-30T08:58:47.090324+0000","last_peered":"2024-08-30T08:58:47.090324+0000","last_clean":"2024-08-30T08:58:47.090324+0000","last_became_active":"2024-08-30T08:58:02.407856+0000","last_became_peered":"2024-08-30T08:58:02.407856+0000","last_unstale":"2024-08-30T08:58:47.090324+0000","last_undegraded":"2024-08-30T08:58:47.090324+0000","last_fullsized":"2024-08-30T08:58:47.090324+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T12:00:22.226653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"79'4","reported_seq":40,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.557487+0000","last_change":"2024-08-30T08:58:00.391714+0000","last_active":"2024-08-30T08:58:46.557487+0000","last_peered":"2024-08-30T08:58:46.557487+0000","last_clean":"2024-08-30T08:58:46.557487+0000","last_became_active":"2024-08-30T08:58:00.391211+0000","last_became_peered":"2024-08-30T08:58:00.391211+0000","last_unstale":"2024-08-30T08:58:46.557487+0000","last_undegraded":"2024-08-30T08:58:46.557487+0000","last_fullsized":"2024-08-30T08:58:46.557487+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T11:55:37.469983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:47.089359+0000","last_change":"2024-08-30T08:58:09.728177+0000","last_active":"2024-08-30T08:58:47.089359+0000","last_peered":"2024-08-30T08:58:47.089359+0000","last_clean":"2024-08-30T08:58:47.089359+0000","last_became_active":"2024-08-30T08:58:09.727630+0000","last_became_peered":"2024-08-30T08:58:09.727630+0000","last_unstale":"2024-08-30T08:58:47.089359+0000","last_undegraded":"2024-08-30T08:58:47.089359+0000","last_fullsized":"2024-08-30T08:58:47.089359+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T12:07:52.965014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.554094+0000","last_change":"2024-08-30T08:58:02.403378+0000","last_active":"2024-08-30T08:58:46.554094+0000","last_peered":"2024-08-30T08:58:46.554094+0000","last_clean":"2024-08-30T08:58:46.554094+0000","last_became_active":"2024-08-30T08:58:02.403036+0000","last_became_peered":"2024-08-30T08:58:02.403036+0000","last_unstale":"2024-08-30T08:58:46.554094+0000","last_undegraded":"2024-08-30T08:58:46.554094+0000","last_fullsized":"2024-08-30T08:58:46.554094+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T16:37:14.876384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"80'1","reported_seq":36,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.555381+0000","last_change":"2024-08-30T08:58:00.393860+0000","last_active":"2024-08-30T08:58:46.555381+0000","last_peered":"2024-08-30T08:58:46.555381+0000","last_clean":"2024-08-30T08:58:46.555381+0000","last_became_active":"2024-08-30T08:58:00.392486+0000","last_became_peered":"2024-08-30T08:58:00.392486+0000","last_unstale":"2024-08-30T08:58:46.555381+0000","last_undegraded":"2024-08-30T08:58:46.555381+0000","last_fullsized":"2024-08-30T08:58:46.555381+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T12:50:27.337817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"81'1","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:47.089678+0000","last_change":"2024-08-30T08:58:09.441476+0000","last_active":"2024-08-30T08:58:47.089678+0000","last_peered":"2024-08-30T08:58:47.089678+0000","last_clean":"2024-08-30T08:58:47.089678+0000","last_became_active":"2024-08-30T08:58:09.440344+0000","last_became_peered":"2024-08-30T08:58:09.440344+0000","last_unstale":"2024-08-30T08:58:47.089678+0000","last_undegraded":"2024-08-30T08:58:47.089678+0000","last_fullsized":"2024-08-30T08:58:47.089678+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T12:06:47.113611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.553171+0000","last_change":"2024-08-30T08:58:02.407098+0000","last_active":"2024-08-30T08:58:46.553171+0000","last_peered":"2024-08-30T08:58:46.553171+0000","last_clean":"2024-08-30T08:58:46.553171+0000","last_became_active":"2024-08-30T08:58:02.406690+0000","last_became_peered":"2024-08-30T08:58:02.406690+0000","last_unstale":"2024-08-30T08:58:46.553171+0000","last_undegraded":"2024-08-30T08:58:46.553171+0000","last_fullsized":"2024-08-30T08:58:46.553171+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T18:27:39.534923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"79'3","reported_seq":38,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.554200+0000","last_change":"2024-08-30T08:58:00.386434+0000","last_active":"2024-08-30T08:58:46.554200+0000","last_peered":"2024-08-30T08:58:46.554200+0000","last_clean":"2024-08-30T08:58:46.554200+0000","last_became_active":"2024-08-30T08:58:00.385930+0000","last_became_peered":"2024-08-30T08:58:00.385930+0000","last_unstale":"2024-08-30T08:58:46.554200+0000","last_undegraded":"2024-08-30T08:58:46.554200+0000","last_fullsized":"2024-08-30T08:58:46.554200+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T10:39:22.441568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.775800+0000","last_change":"2024-08-30T08:58:46.775674+0000","last_active":"2024-08-30T08:58:46.775800+0000","last_peered":"2024-08-30T08:58:46.775800+0000","last_clean":"2024-08-30T08:58:46.775800+0000","last_became_active":"2024-08-30T08:58:46.775157+0000","last_became_peered":"2024-08-30T08:58:46.775157+0000","last_unstale":"2024-08-30T08:58:46.775800+0000","last_undegraded":"2024-08-30T08:58:46.775800+0000","last_fullsized":"2024-08-30T08:58:46.775800+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T17:13:47.066475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,2,5],"acting":[8,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.559287+0000","last_change":"2024-08-30T08:58:02.410500+0000","last_active":"2024-08-30T08:58:46.559287+0000","last_peered":"2024-08-30T08:58:46.559287+0000","last_clean":"2024-08-30T08:58:46.559287+0000","last_became_active":"2024-08-30T08:58:02.409260+0000","last_became_peered":"2024-08-30T08:58:02.409260+0000","last_unstale":"2024-08-30T08:58:46.559287+0000","last_undegraded":"2024-08-30T08:58:46.559287+0000","last_fullsized":"2024-08-30T08:58:46.559287+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T16:47:59.701956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"79'2","reported_seq":37,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.822713+0000","last_change":"2024-08-30T08:58:00.482296+0000","last_active":"2024-08-30T08:58:46.822713+0000","last_peered":"2024-08-30T08:58:46.822713+0000","last_clean":"2024-08-30T08:58:46.822713+0000","last_became_active":"2024-08-30T08:58:00.481920+0000","last_became_peered":"2024-08-30T08:58:00.481920+0000","last_unstale":"2024-08-30T08:58:46.822713+0000","last_undegraded":"2024-08-30T08:58:46.822713+0000","last_fullsized":"2024-08-30T08:58:46.822713+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T09:29:30.724410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:47.090717+0000","last_change":"2024-08-30T08:58:09.727879+0000","last_active":"2024-08-30T08:58:47.090717+0000","last_peered":"2024-08-30T08:58:47.090717+0000","last_clean":"2024-08-30T08:58:47.090717+0000","last_became_active":"2024-08-30T08:58:09.727339+0000","last_became_peered":"2024-08-30T08:58:09.727339+0000","last_unstale":"2024-08-30T08:58:47.090717+0000","last_undegraded":"2024-08-30T08:58:47.090717+0000","last_fullsized":"2024-08-30T08:58:47.090717+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T13:19:18.582668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":37,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.560099+0000","last_change":"2024-08-30T08:58:46.559988+0000","last_active":"2024-08-30T08:58:46.560099+0000","last_peered":"2024-08-30T08:58:46.560099+0000","last_clean":"2024-08-30T08:58:46.560099+0000","last_became_active":"2024-08-30T08:58:46.559442+0000","last_became_peered":"2024-08-30T08:58:46.559442+0000","last_unstale":"2024-08-30T08:58:46.560099+0000","last_undegraded":"2024-08-30T08:58:46.560099+0000","last_fullsized":"2024-08-30T08:58:46.560099+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T09:29:09.747756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"79'4","reported_seq":39,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:47.091026+0000","last_change":"2024-08-30T08:58:00.380338+0000","last_active":"2024-08-30T08:58:47.091026+0000","last_peered":"2024-08-30T08:58:47.091026+0000","last_clean":"2024-08-30T08:58:47.091026+0000","last_became_active":"2024-08-30T08:58:00.379860+0000","last_became_peered":"2024-08-30T08:58:00.379860+0000","last_unstale":"2024-08-30T08:58:47.091026+0000","last_undegraded":"2024-08-30T08:58:47.091026+0000","last_fullsized":"2024-08-30T08:58:47.091026+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T19:57:15.349906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.555200+0000","last_change":"2024-08-30T08:58:09.443631+0000","last_active":"2024-08-30T08:58:46.555200+0000","last_peered":"2024-08-30T08:58:46.555200+0000","last_clean":"2024-08-30T08:58:46.555200+0000","last_became_active":"2024-08-30T08:58:09.443103+0000","last_became_peered":"2024-08-30T08:58:09.443103+0000","last_unstale":"2024-08-30T08:58:46.555200+0000","last_undegraded":"2024-08-30T08:58:46.555200+0000","last_fullsized":"2024-08-30T08:58:46.555200+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T19:58:11.031826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.555619+0000","last_change":"2024-08-30T08:58:02.406216+0000","last_active":"2024-08-30T08:58:46.555619+0000","last_peered":"2024-08-30T08:58:46.555619+0000","last_clean":"2024-08-30T08:58:46.555619+0000","last_became_active":"2024-08-30T08:58:02.405040+0000","last_became_peered":"2024-08-30T08:58:02.405040+0000","last_unstale":"2024-08-30T08:58:46.555619+0000","last_undegraded":"2024-08-30T08:58:46.555619+0000","last_fullsized":"2024-08-30T08:58:46.555619+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T14:28:28.721386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"81'12","reported_seq":47,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:59:01.413507+0000","last_change":"2024-08-30T08:58:00.387350+0000","last_active":"2024-08-30T08:59:01.413507+0000","last_peered":"2024-08-30T08:59:01.413507+0000","last_clean":"2024-08-30T08:59:01.413507+0000","last_became_active":"2024-08-30T08:58:00.387004+0000","last_became_peered":"2024-08-30T08:58:00.387004+0000","last_unstale":"2024-08-30T08:59:01.413507+0000","last_undegraded":"2024-08-30T08:59:01.413507+0000","last_fullsized":"2024-08-30T08:59:01.413507+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T19:24:25.604544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.553684+0000","last_change":"2024-08-30T08:58:09.727388+0000","last_active":"2024-08-30T08:58:46.553684+0000","last_peered":"2024-08-30T08:58:46.553684+0000","last_clean":"2024-08-30T08:58:46.553684+0000","last_became_active":"2024-08-30T08:58:09.726935+0000","last_became_peered":"2024-08-30T08:58:09.726935+0000","last_unstale":"2024-08-30T08:58:46.553684+0000","last_undegraded":"2024-08-30T08:58:46.553684+0000","last_fullsized":"2024-08-30T08:58:46.553684+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T17:53:16.066602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.552226+0000","last_change":"2024-08-30T08:58:02.402661+0000","last_active":"2024-08-30T08:58:46.552226+0000","last_peered":"2024-08-30T08:58:46.552226+0000","last_clean":"2024-08-30T08:58:46.552226+0000","last_became_active":"2024-08-30T08:58:02.402314+0000","last_became_peered":"2024-08-30T08:58:02.402314+0000","last_unstale":"2024-08-30T08:58:46.552226+0000","last_undegraded":"2024-08-30T08:58:46.552226+0000","last_fullsized":"2024-08-30T08:58:46.552226+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T10:56:41.336312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"79'1","reported_seq":37,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.557749+0000","last_change":"2024-08-30T08:58:00.482720+0000","last_active":"2024-08-30T08:58:46.557749+0000","last_peered":"2024-08-30T08:58:46.557749+0000","last_clean":"2024-08-30T08:58:46.557749+0000","last_became_active":"2024-08-30T08:58:00.482193+0000","last_became_peered":"2024-08-30T08:58:00.482193+0000","last_unstale":"2024-08-30T08:58:46.557749+0000","last_undegraded":"2024-08-30T08:58:46.557749+0000","last_fullsized":"2024-08-30T08:58:46.557749+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T18:42:06.654806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.553347+0000","last_change":"2024-08-30T08:58:09.449970+0000","last_active":"2024-08-30T08:58:46.553347+0000","last_peered":"2024-08-30T08:58:46.553347+0000","last_clean":"2024-08-30T08:58:46.553347+0000","last_became_active":"2024-08-30T08:58:09.447891+0000","last_became_peered":"2024-08-30T08:58:09.447891+0000","last_unstale":"2024-08-30T08:58:46.553347+0000","last_undegraded":"2024-08-30T08:58:46.553347+0000","last_fullsized":"2024-08-30T08:58:46.553347+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T15:06:45.414470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.553365+0000","last_change":"2024-08-30T08:58:02.409756+0000","last_active":"2024-08-30T08:58:46.553365+0000","last_peered":"2024-08-30T08:58:46.553365+0000","last_clean":"2024-08-30T08:58:46.553365+0000","last_became_active":"2024-08-30T08:58:02.409357+0000","last_became_peered":"2024-08-30T08:58:02.409357+0000","last_unstale":"2024-08-30T08:58:46.553365+0000","last_undegraded":"2024-08-30T08:58:46.553365+0000","last_fullsized":"2024-08-30T08:58:46.553365+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T13:27:41.696527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":35,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.558745+0000","last_change":"2024-08-30T08:58:00.482342+0000","last_active":"2024-08-30T08:58:46.558745+0000","last_peered":"2024-08-30T08:58:46.558745+0000","last_clean":"2024-08-30T08:58:46.558745+0000","last_became_active":"2024-08-30T08:58:00.481902+0000","last_became_peered":"2024-08-30T08:58:00.481902+0000","last_unstale":"2024-08-30T08:58:46.558745+0000","last_undegraded":"2024-08-30T08:58:46.558745+0000","last_fullsized":"2024-08-30T08:58:46.558745+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T14:12:36.789356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.556901+0000","last_change":"2024-08-30T08:58:09.728008+0000","last_active":"2024-08-30T08:58:46.556901+0000","last_peered":"2024-08-30T08:58:46.556901+0000","last_clean":"2024-08-30T08:58:46.556901+0000","last_became_active":"2024-08-30T08:58:09.727513+0000","last_became_peered":"2024-08-30T08:58:09.727513+0000","last_unstale":"2024-08-30T08:58:46.556901+0000","last_undegraded":"2024-08-30T08:58:46.556901+0000","last_fullsized":"2024-08-30T08:58:46.556901+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T14:34:04.165300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.553329+0000","last_change":"2024-08-30T08:58:02.409383+0000","last_active":"2024-08-30T08:58:46.553329+0000","last_peered":"2024-08-30T08:58:46.553329+0000","last_clean":"2024-08-30T08:58:46.553329+0000","last_became_active":"2024-08-30T08:58:02.409027+0000","last_became_peered":"2024-08-30T08:58:02.409027+0000","last_unstale":"2024-08-30T08:58:46.553329+0000","last_undegraded":"2024-08-30T08:58:46.553329+0000","last_fullsized":"2024-08-30T08:58:46.553329+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T10:44:23.467130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"80'1","reported_seq":36,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:47.090595+0000","last_change":"2024-08-30T08:58:00.389083+0000","last_active":"2024-08-30T08:58:47.090595+0000","last_peered":"2024-08-30T08:58:47.090595+0000","last_clean":"2024-08-30T08:58:47.090595+0000","last_became_active":"2024-08-30T08:58:00.387552+0000","last_became_peered":"2024-08-30T08:58:00.387552+0000","last_unstale":"2024-08-30T08:58:47.090595+0000","last_undegraded":"2024-08-30T08:58:47.090595+0000","last_fullsized":"2024-08-30T08:58:47.090595+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T11:56:38.827406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.552061+0000","last_change":"2024-08-30T08:58:09.447758+0000","last_active":"2024-08-30T08:58:46.552061+0000","last_peered":"2024-08-30T08:58:46.552061+0000","last_clean":"2024-08-30T08:58:46.552061+0000","last_became_active":"2024-08-30T08:58:09.447333+0000","last_became_peered":"2024-08-30T08:58:09.447333+0000","last_unstale":"2024-08-30T08:58:46.552061+0000","last_undegraded":"2024-08-30T08:58:46.552061+0000","last_fullsized":"2024-08-30T08:58:46.552061+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T12:43:35.289967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:47.091640+0000","last_change":"2024-08-30T08:58:47.091191+0000","last_active":"2024-08-30T08:58:47.091640+0000","last_peered":"2024-08-30T08:58:47.091640+0000","last_clean":"2024-08-30T08:58:47.091640+0000","last_became_active":"2024-08-30T08:58:47.090826+0000","last_became_peered":"2024-08-30T08:58:47.090826+0000","last_unstale":"2024-08-30T08:58:47.091640+0000","last_undegraded":"2024-08-30T08:58:47.091640+0000","last_fullsized":"2024-08-30T08:58:47.091640+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T17:47:22.862514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"80'4","reported_seq":92,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:59:41.693522+0000","last_change":"2024-08-30T08:58:00.480651+0000","last_active":"2024-08-30T08:59:41.693522+0000","last_peered":"2024-08-30T08:59:41.693522+0000","last_clean":"2024-08-30T08:59:41.693522+0000","last_became_active":"2024-08-30T08:58:00.480205+0000","last_became_peered":"2024-08-30T08:58:00.480205+0000","last_unstale":"2024-08-30T08:59:41.693522+0000","last_undegraded":"2024-08-30T08:59:41.693522+0000","last_fullsized":"2024-08-30T08:59:41.693522+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T16:28:30.687076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":53,"num_read_kb":53,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.552066+0000","last_change":"2024-08-30T08:58:09.727530+0000","last_active":"2024-08-30T08:58:46.552066+0000","last_peered":"2024-08-30T08:58:46.552066+0000","last_clean":"2024-08-30T08:58:46.552066+0000","last_became_active":"2024-08-30T08:58:09.726965+0000","last_became_peered":"2024-08-30T08:58:09.726965+0000","last_unstale":"2024-08-30T08:58:46.552066+0000","last_undegraded":"2024-08-30T08:58:46.552066+0000","last_fullsized":"2024-08-30T08:58:46.552066+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T12:56:42.885615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.551578+0000","last_change":"2024-08-30T08:58:02.401673+0000","last_active":"2024-08-30T08:58:46.551578+0000","last_peered":"2024-08-30T08:58:46.551578+0000","last_clean":"2024-08-30T08:58:46.551578+0000","last_became_active":"2024-08-30T08:58:02.398876+0000","last_became_peered":"2024-08-30T08:58:02.398876+0000","last_unstale":"2024-08-30T08:58:46.551578+0000","last_undegraded":"2024-08-30T08:58:46.551578+0000","last_fullsized":"2024-08-30T08:58:46.551578+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T16:13:20.893737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"80'3","reported_seq":38,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:47.091220+0000","last_change":"2024-08-30T08:58:00.391722+0000","last_active":"2024-08-30T08:58:47.091220+0000","last_peered":"2024-08-30T08:58:47.091220+0000","last_clean":"2024-08-30T08:58:47.091220+0000","last_became_active":"2024-08-30T08:58:00.391325+0000","last_became_peered":"2024-08-30T08:58:00.391325+0000","last_unstale":"2024-08-30T08:58:47.091220+0000","last_undegraded":"2024-08-30T08:58:47.091220+0000","last_fullsized":"2024-08-30T08:58:47.091220+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T11:58:08.189202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.774243+0000","last_change":"2024-08-30T08:58:09.728324+0000","last_active":"2024-08-30T08:58:46.774243+0000","last_peered":"2024-08-30T08:58:46.774243+0000","last_clean":"2024-08-30T08:58:46.774243+0000","last_became_active":"2024-08-30T08:58:09.727520+0000","last_became_peered":"2024-08-30T08:58:09.727520+0000","last_unstale":"2024-08-30T08:58:46.774243+0000","last_undegraded":"2024-08-30T08:58:46.774243+0000","last_fullsized":"2024-08-30T08:58:46.774243+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T10:09:39.046820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.554063+0000","last_change":"2024-08-30T08:58:02.410826+0000","last_active":"2024-08-30T08:58:46.554063+0000","last_peered":"2024-08-30T08:58:46.554063+0000","last_clean":"2024-08-30T08:58:46.554063+0000","last_became_active":"2024-08-30T08:58:02.410464+0000","last_became_peered":"2024-08-30T08:58:02.410464+0000","last_unstale":"2024-08-30T08:58:46.554063+0000","last_undegraded":"2024-08-30T08:58:46.554063+0000","last_fullsized":"2024-08-30T08:58:46.554063+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T12:14:59.666614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"80'3","reported_seq":40,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:50.596022+0000","last_change":"2024-08-30T08:58:00.385464+0000","last_active":"2024-08-30T08:58:50.596022+0000","last_peered":"2024-08-30T08:58:50.596022+0000","last_clean":"2024-08-30T08:58:50.596022+0000","last_became_active":"2024-08-30T08:58:00.385083+0000","last_became_peered":"2024-08-30T08:58:00.385083+0000","last_unstale":"2024-08-30T08:58:50.596022+0000","last_undegraded":"2024-08-30T08:58:50.596022+0000","last_fullsized":"2024-08-30T08:58:50.596022+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T16:13:52.304755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":108,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:47.091487+0000","last_change":"2024-08-30T08:56:49.645080+0000","last_active":"2024-08-30T08:58:47.091487+0000","last_peered":"2024-08-30T08:58:47.091487+0000","last_clean":"2024-08-30T08:58:47.091487+0000","last_became_active":"2024-08-30T08:56:49.639066+0000","last_became_peered":"2024-08-30T08:56:49.639066+0000","last_unstale":"2024-08-30T08:58:47.091487+0000","last_undegraded":"2024-08-30T08:58:47.091487+0000","last_fullsized":"2024-08-30T08:58:47.091487+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:53:27.566846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:53:27.566846+0000","last_clean_scrub_stamp":"2024-08-30T08:53:27.566846+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T20:23:45.723824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"81'1","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:47.090916+0000","last_change":"2024-08-30T08:58:09.444007+0000","last_active":"2024-08-30T08:58:47.090916+0000","last_peered":"2024-08-30T08:58:47.090916+0000","last_clean":"2024-08-30T08:58:47.090916+0000","last_became_active":"2024-08-30T08:58:09.443580+0000","last_became_peered":"2024-08-30T08:58:09.443580+0000","last_unstale":"2024-08-30T08:58:47.090916+0000","last_undegraded":"2024-08-30T08:58:47.090916+0000","last_fullsized":"2024-08-30T08:58:47.090916+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T17:00:58.654026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:47.091022+0000","last_change":"2024-08-30T08:58:02.406973+0000","last_active":"2024-08-30T08:58:47.091022+0000","last_peered":"2024-08-30T08:58:47.091022+0000","last_clean":"2024-08-30T08:58:47.091022+0000","last_became_active":"2024-08-30T08:58:02.406576+0000","last_became_peered":"2024-08-30T08:58:02.406576+0000","last_unstale":"2024-08-30T08:58:47.091022+0000","last_undegraded":"2024-08-30T08:58:47.091022+0000","last_fullsized":"2024-08-30T08:58:47.091022+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T16:04:48.785310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"80'3","reported_seq":39,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.556017+0000","last_change":"2024-08-30T08:58:00.481019+0000","last_active":"2024-08-30T08:58:46.556017+0000","last_peered":"2024-08-30T08:58:46.556017+0000","last_clean":"2024-08-30T08:58:46.556017+0000","last_became_active":"2024-08-30T08:58:00.480590+0000","last_became_peered":"2024-08-30T08:58:00.480590+0000","last_unstale":"2024-08-30T08:58:46.556017+0000","last_undegraded":"2024-08-30T08:58:46.556017+0000","last_fullsized":"2024-08-30T08:58:46.556017+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T13:20:46.731591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.822612+0000","last_change":"2024-08-30T08:58:09.445723+0000","last_active":"2024-08-30T08:58:46.822612+0000","last_peered":"2024-08-30T08:58:46.822612+0000","last_clean":"2024-08-30T08:58:46.822612+0000","last_became_active":"2024-08-30T08:58:09.445252+0000","last_became_peered":"2024-08-30T08:58:09.445252+0000","last_unstale":"2024-08-30T08:58:46.822612+0000","last_undegraded":"2024-08-30T08:58:46.822612+0000","last_fullsized":"2024-08-30T08:58:46.822612+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T12:16:12.255990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:47.090625+0000","last_change":"2024-08-30T08:58:02.414250+0000","last_active":"2024-08-30T08:58:47.090625+0000","last_peered":"2024-08-30T08:58:47.090625+0000","last_clean":"2024-08-30T08:58:47.090625+0000","last_became_active":"2024-08-30T08:58:02.413684+0000","last_became_peered":"2024-08-30T08:58:02.413684+0000","last_unstale":"2024-08-30T08:58:47.090625+0000","last_undegraded":"2024-08-30T08:58:47.090625+0000","last_fullsized":"2024-08-30T08:58:47.090625+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T16:55:12.878806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"73'1","reported_seq":37,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.774624+0000","last_change":"2024-08-30T08:58:00.388869+0000","last_active":"2024-08-30T08:58:46.774624+0000","last_peered":"2024-08-30T08:58:46.774624+0000","last_clean":"2024-08-30T08:58:46.774624+0000","last_became_active":"2024-08-30T08:58:00.388116+0000","last_became_peered":"2024-08-30T08:58:00.388116+0000","last_unstale":"2024-08-30T08:58:46.774624+0000","last_undegraded":"2024-08-30T08:58:46.774624+0000","last_fullsized":"2024-08-30T08:58:46.774624+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T09:50:48.449205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:47.092091+0000","last_change":"2024-08-30T08:58:47.092004+0000","last_active":"2024-08-30T08:58:47.092091+0000","last_peered":"2024-08-30T08:58:47.092091+0000","last_clean":"2024-08-30T08:58:47.092091+0000","last_became_active":"2024-08-30T08:58:47.091661+0000","last_became_peered":"2024-08-30T08:58:47.091661+0000","last_unstale":"2024-08-30T08:58:47.092091+0000","last_undegraded":"2024-08-30T08:58:47.092091+0000","last_fullsized":"2024-08-30T08:58:47.092091+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T09:14:55.353465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,1,4],"acting":[0,8,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:47.090626+0000","last_change":"2024-08-30T08:58:02.410891+0000","last_active":"2024-08-30T08:58:47.090626+0000","last_peered":"2024-08-30T08:58:47.090626+0000","last_clean":"2024-08-30T08:58:47.090626+0000","last_became_active":"2024-08-30T08:58:02.410522+0000","last_became_peered":"2024-08-30T08:58:02.410522+0000","last_unstale":"2024-08-30T08:58:47.090626+0000","last_undegraded":"2024-08-30T08:58:47.090626+0000","last_fullsized":"2024-08-30T08:58:47.090626+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T09:32:21.540298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"73'2","reported_seq":48,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:47.095913+0000","last_change":"2024-08-30T08:58:47.095801+0000","last_active":"2024-08-30T08:58:47.095913+0000","last_peered":"2024-08-30T08:58:47.095913+0000","last_clean":"2024-08-30T08:58:47.095913+0000","last_became_active":"2024-08-30T08:58:47.092756+0000","last_became_peered":"2024-08-30T08:58:47.092756+0000","last_unstale":"2024-08-30T08:58:47.095913+0000","last_undegraded":"2024-08-30T08:58:47.095913+0000","last_fullsized":"2024-08-30T08:58:47.095913+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T19:10:02.064649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.773596+0000","last_change":"2024-08-30T08:58:09.449210+0000","last_active":"2024-08-30T08:58:46.773596+0000","last_peered":"2024-08-30T08:58:46.773596+0000","last_clean":"2024-08-30T08:58:46.773596+0000","last_became_active":"2024-08-30T08:58:09.448769+0000","last_became_peered":"2024-08-30T08:58:09.448769+0000","last_unstale":"2024-08-30T08:58:46.773596+0000","last_undegraded":"2024-08-30T08:58:46.773596+0000","last_fullsized":"2024-08-30T08:58:46.773596+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T09:26:13.845520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.553484+0000","last_change":"2024-08-30T08:58:02.414152+0000","last_active":"2024-08-30T08:58:46.553484+0000","last_peered":"2024-08-30T08:58:46.553484+0000","last_clean":"2024-08-30T08:58:46.553484+0000","last_became_active":"2024-08-30T08:58:02.413768+0000","last_became_peered":"2024-08-30T08:58:02.413768+0000","last_unstale":"2024-08-30T08:58:46.553484+0000","last_undegraded":"2024-08-30T08:58:46.553484+0000","last_fullsized":"2024-08-30T08:58:46.553484+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T18:34:09.447071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"81'2","reported_seq":38,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:50.596425+0000","last_change":"2024-08-30T08:58:00.390839+0000","last_active":"2024-08-30T08:58:50.596425+0000","last_peered":"2024-08-30T08:58:50.596425+0000","last_clean":"2024-08-30T08:58:50.596425+0000","last_became_active":"2024-08-30T08:58:00.390422+0000","last_became_peered":"2024-08-30T08:58:00.390422+0000","last_unstale":"2024-08-30T08:58:50.596425+0000","last_undegraded":"2024-08-30T08:58:50.596425+0000","last_fullsized":"2024-08-30T08:58:50.596425+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T15:05:23.005075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.824657+0000","last_change":"2024-08-30T08:58:46.824563+0000","last_active":"2024-08-30T08:58:46.824657+0000","last_peered":"2024-08-30T08:58:46.824657+0000","last_clean":"2024-08-30T08:58:46.824657+0000","last_became_active":"2024-08-30T08:58:46.824163+0000","last_became_peered":"2024-08-30T08:58:46.824163+0000","last_unstale":"2024-08-30T08:58:46.824657+0000","last_undegraded":"2024-08-30T08:58:46.824657+0000","last_fullsized":"2024-08-30T08:58:46.824657+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T19:25:13.202762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,3,1],"acting":[8,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.773839+0000","last_change":"2024-08-30T08:58:02.405122+0000","last_active":"2024-08-30T08:58:46.773839+0000","last_peered":"2024-08-30T08:58:46.773839+0000","last_clean":"2024-08-30T08:58:46.773839+0000","last_became_active":"2024-08-30T08:58:02.404695+0000","last_became_peered":"2024-08-30T08:58:02.404695+0000","last_unstale":"2024-08-30T08:58:46.773839+0000","last_undegraded":"2024-08-30T08:58:46.773839+0000","last_fullsized":"2024-08-30T08:58:46.773839+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T15:50:01.447032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"81'15","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:47.395221+0000","last_change":"2024-08-30T08:58:46.573221+0000","last_active":"2024-08-30T08:58:47.395221+0000","last_peered":"2024-08-30T08:58:47.395221+0000","last_clean":"2024-08-30T08:58:47.395221+0000","last_became_active":"2024-08-30T08:58:46.559562+0000","last_became_peered":"2024-08-30T08:58:46.559562+0000","last_unstale":"2024-08-30T08:58:47.395221+0000","last_undegraded":"2024-08-30T08:58:47.395221+0000","last_fullsized":"2024-08-30T08:58:47.395221+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T16:21:23.182676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.552177+0000","last_change":"2024-08-30T08:58:09.446063+0000","last_active":"2024-08-30T08:58:46.552177+0000","last_peered":"2024-08-30T08:58:46.552177+0000","last_clean":"2024-08-30T08:58:46.552177+0000","last_became_active":"2024-08-30T08:58:09.445615+0000","last_became_peered":"2024-08-30T08:58:09.445615+0000","last_unstale":"2024-08-30T08:58:46.552177+0000","last_undegraded":"2024-08-30T08:58:46.552177+0000","last_fullsized":"2024-08-30T08:58:46.552177+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T10:25:10.904892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:47.090684+0000","last_change":"2024-08-30T08:58:02.412178+0000","last_active":"2024-08-30T08:58:47.090684+0000","last_peered":"2024-08-30T08:58:47.090684+0000","last_clean":"2024-08-30T08:58:47.090684+0000","last_became_active":"2024-08-30T08:58:02.411814+0000","last_became_peered":"2024-08-30T08:58:02.411814+0000","last_unstale":"2024-08-30T08:58:47.090684+0000","last_undegraded":"2024-08-30T08:58:47.090684+0000","last_fullsized":"2024-08-30T08:58:47.090684+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T10:02:59.070204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"81'4","reported_seq":92,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:59:39.691500+0000","last_change":"2024-08-30T08:58:00.390231+0000","last_active":"2024-08-30T08:59:39.691500+0000","last_peered":"2024-08-30T08:59:39.691500+0000","last_clean":"2024-08-30T08:59:39.691500+0000","last_became_active":"2024-08-30T08:58:00.389852+0000","last_became_peered":"2024-08-30T08:58:00.389852+0000","last_unstale":"2024-08-30T08:59:39.691500+0000","last_undegraded":"2024-08-30T08:59:39.691500+0000","last_fullsized":"2024-08-30T08:59:39.691500+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T12:09:12.105416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":54,"num_read_kb":3,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.558813+0000","last_change":"2024-08-30T08:58:09.441652+0000","last_active":"2024-08-30T08:58:46.558813+0000","last_peered":"2024-08-30T08:58:46.558813+0000","last_clean":"2024-08-30T08:58:46.558813+0000","last_became_active":"2024-08-30T08:58:09.441264+0000","last_became_peered":"2024-08-30T08:58:09.441264+0000","last_unstale":"2024-08-30T08:58:46.558813+0000","last_undegraded":"2024-08-30T08:58:46.558813+0000","last_fullsized":"2024-08-30T08:58:46.558813+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T09:42:25.636669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:47.090297+0000","last_change":"2024-08-30T08:58:02.416347+0000","last_active":"2024-08-30T08:58:47.090297+0000","last_peered":"2024-08-30T08:58:47.090297+0000","last_clean":"2024-08-30T08:58:47.090297+0000","last_became_active":"2024-08-30T08:58:02.415898+0000","last_became_peered":"2024-08-30T08:58:02.415898+0000","last_unstale":"2024-08-30T08:58:47.090297+0000","last_undegraded":"2024-08-30T08:58:47.090297+0000","last_fullsized":"2024-08-30T08:58:47.090297+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T10:18:15.507076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"79'3","reported_seq":38,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.774168+0000","last_change":"2024-08-30T08:58:00.393616+0000","last_active":"2024-08-30T08:58:46.774168+0000","last_peered":"2024-08-30T08:58:46.774168+0000","last_clean":"2024-08-30T08:58:46.774168+0000","last_became_active":"2024-08-30T08:58:00.393218+0000","last_became_peered":"2024-08-30T08:58:00.393218+0000","last_unstale":"2024-08-30T08:58:46.774168+0000","last_undegraded":"2024-08-30T08:58:46.774168+0000","last_fullsized":"2024-08-30T08:58:46.774168+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T10:56:37.175968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:47.093131+0000","last_change":"2024-08-30T08:58:47.092990+0000","last_active":"2024-08-30T08:58:47.093131+0000","last_peered":"2024-08-30T08:58:47.093131+0000","last_clean":"2024-08-30T08:58:47.093131+0000","last_became_active":"2024-08-30T08:58:47.092448+0000","last_became_peered":"2024-08-30T08:58:47.092448+0000","last_unstale":"2024-08-30T08:58:47.093131+0000","last_undegraded":"2024-08-30T08:58:47.093131+0000","last_fullsized":"2024-08-30T08:58:47.093131+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T20:23:06.527336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,7,6],"acting":[11,10,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.551703+0000","last_change":"2024-08-30T08:58:02.405783+0000","last_active":"2024-08-30T08:58:46.551703+0000","last_peered":"2024-08-30T08:58:46.551703+0000","last_clean":"2024-08-30T08:58:46.551703+0000","last_became_active":"2024-08-30T08:58:02.405430+0000","last_became_peered":"2024-08-30T08:58:02.405430+0000","last_unstale":"2024-08-30T08:58:46.551703+0000","last_undegraded":"2024-08-30T08:58:46.551703+0000","last_fullsized":"2024-08-30T08:58:46.551703+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T19:40:08.377259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"80'6","reported_seq":42,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.555590+0000","last_change":"2024-08-30T08:58:00.389617+0000","last_active":"2024-08-30T08:58:46.555590+0000","last_peered":"2024-08-30T08:58:46.555590+0000","last_clean":"2024-08-30T08:58:46.555590+0000","last_became_active":"2024-08-30T08:58:00.389031+0000","last_became_peered":"2024-08-30T08:58:00.389031+0000","last_unstale":"2024-08-30T08:58:46.555590+0000","last_undegraded":"2024-08-30T08:58:46.555590+0000","last_fullsized":"2024-08-30T08:58:46.555590+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T15:42:38.050768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.774094+0000","last_change":"2024-08-30T08:58:09.451925+0000","last_active":"2024-08-30T08:58:46.774094+0000","last_peered":"2024-08-30T08:58:46.774094+0000","last_clean":"2024-08-30T08:58:46.774094+0000","last_became_active":"2024-08-30T08:58:09.446893+0000","last_became_peered":"2024-08-30T08:58:09.446893+0000","last_unstale":"2024-08-30T08:58:46.774094+0000","last_undegraded":"2024-08-30T08:58:46.774094+0000","last_fullsized":"2024-08-30T08:58:46.774094+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T14:34:25.134006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.557378+0000","last_change":"2024-08-30T08:58:02.411194+0000","last_active":"2024-08-30T08:58:46.557378+0000","last_peered":"2024-08-30T08:58:46.557378+0000","last_clean":"2024-08-30T08:58:46.557378+0000","last_became_active":"2024-08-30T08:58:02.410183+0000","last_became_peered":"2024-08-30T08:58:02.410183+0000","last_unstale":"2024-08-30T08:58:46.557378+0000","last_undegraded":"2024-08-30T08:58:46.557378+0000","last_fullsized":"2024-08-30T08:58:46.557378+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T10:56:19.016115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"81'34","reported_seq":182,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:59:41.687527+0000","last_change":"2024-08-30T08:58:00.389808+0000","last_active":"2024-08-30T08:59:41.687527+0000","last_peered":"2024-08-30T08:59:41.687527+0000","last_clean":"2024-08-30T08:59:41.687527+0000","last_became_active":"2024-08-30T08:58:00.389411+0000","last_became_peered":"2024-08-30T08:58:00.389411+0000","last_unstale":"2024-08-30T08:59:41.687527+0000","last_undegraded":"2024-08-30T08:59:41.687527+0000","last_fullsized":"2024-08-30T08:59:41.687527+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T18:59:45.468456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64060,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":113,"num_read_kb":119,"num_write":35,"num_write_kb":78,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.557593+0000","last_change":"2024-08-30T08:58:09.447360+0000","last_active":"2024-08-30T08:58:46.557593+0000","last_peered":"2024-08-30T08:58:46.557593+0000","last_clean":"2024-08-30T08:58:46.557593+0000","last_became_active":"2024-08-30T08:58:09.446913+0000","last_became_peered":"2024-08-30T08:58:09.446913+0000","last_unstale":"2024-08-30T08:58:46.557593+0000","last_undegraded":"2024-08-30T08:58:46.557593+0000","last_fullsized":"2024-08-30T08:58:46.557593+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T13:59:43.147370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.822816+0000","last_change":"2024-08-30T08:58:02.408920+0000","last_active":"2024-08-30T08:58:46.822816+0000","last_peered":"2024-08-30T08:58:46.822816+0000","last_clean":"2024-08-30T08:58:46.822816+0000","last_became_active":"2024-08-30T08:58:02.408204+0000","last_became_peered":"2024-08-30T08:58:02.408204+0000","last_unstale":"2024-08-30T08:58:46.822816+0000","last_undegraded":"2024-08-30T08:58:46.822816+0000","last_fullsized":"2024-08-30T08:58:46.822816+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T11:44:00.765672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"79'4","reported_seq":40,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.774530+0000","last_change":"2024-08-30T08:58:00.482348+0000","last_active":"2024-08-30T08:58:46.774530+0000","last_peered":"2024-08-30T08:58:46.774530+0000","last_clean":"2024-08-30T08:58:46.774530+0000","last_became_active":"2024-08-30T08:58:00.481929+0000","last_became_peered":"2024-08-30T08:58:00.481929+0000","last_unstale":"2024-08-30T08:58:46.774530+0000","last_undegraded":"2024-08-30T08:58:46.774530+0000","last_fullsized":"2024-08-30T08:58:46.774530+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T11:13:55.332639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:47.089847+0000","last_change":"2024-08-30T08:58:09.443259+0000","last_active":"2024-08-30T08:58:47.089847+0000","last_peered":"2024-08-30T08:58:47.089847+0000","last_clean":"2024-08-30T08:58:47.089847+0000","last_became_active":"2024-08-30T08:58:09.442209+0000","last_became_peered":"2024-08-30T08:58:09.442209+0000","last_unstale":"2024-08-30T08:58:47.089847+0000","last_undegraded":"2024-08-30T08:58:47.089847+0000","last_fullsized":"2024-08-30T08:58:47.089847+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T09:25:56.233525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.557115+0000","last_change":"2024-08-30T08:58:02.407183+0000","last_active":"2024-08-30T08:58:46.557115+0000","last_peered":"2024-08-30T08:58:46.557115+0000","last_clean":"2024-08-30T08:58:46.557115+0000","last_became_active":"2024-08-30T08:58:02.406838+0000","last_became_peered":"2024-08-30T08:58:02.406838+0000","last_unstale":"2024-08-30T08:58:46.557115+0000","last_undegraded":"2024-08-30T08:58:46.557115+0000","last_fullsized":"2024-08-30T08:58:46.557115+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T13:40:32.937626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"79'2","reported_seq":37,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:47.090297+0000","last_change":"2024-08-30T08:58:00.392322+0000","last_active":"2024-08-30T08:58:47.090297+0000","last_peered":"2024-08-30T08:58:47.090297+0000","last_clean":"2024-08-30T08:58:47.090297+0000","last_became_active":"2024-08-30T08:58:00.391935+0000","last_became_peered":"2024-08-30T08:58:00.391935+0000","last_unstale":"2024-08-30T08:58:47.090297+0000","last_undegraded":"2024-08-30T08:58:47.090297+0000","last_fullsized":"2024-08-30T08:58:47.090297+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T12:20:52.368285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.557659+0000","last_change":"2024-08-30T08:58:09.441824+0000","last_active":"2024-08-30T08:58:46.557659+0000","last_peered":"2024-08-30T08:58:46.557659+0000","last_clean":"2024-08-30T08:58:46.557659+0000","last_became_active":"2024-08-30T08:58:09.441398+0000","last_became_peered":"2024-08-30T08:58:09.441398+0000","last_unstale":"2024-08-30T08:58:46.557659+0000","last_undegraded":"2024-08-30T08:58:46.557659+0000","last_fullsized":"2024-08-30T08:58:46.557659+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T14:05:45.710280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.823379+0000","last_change":"2024-08-30T08:58:02.403431+0000","last_active":"2024-08-30T08:58:46.823379+0000","last_peered":"2024-08-30T08:58:46.823379+0000","last_clean":"2024-08-30T08:58:46.823379+0000","last_became_active":"2024-08-30T08:58:02.403029+0000","last_became_peered":"2024-08-30T08:58:02.403029+0000","last_unstale":"2024-08-30T08:58:46.823379+0000","last_undegraded":"2024-08-30T08:58:46.823379+0000","last_fullsized":"2024-08-30T08:58:46.823379+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T09:34:56.452675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"79'3","reported_seq":90,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:59:40.685815+0000","last_change":"2024-08-30T08:58:00.389856+0000","last_active":"2024-08-30T08:59:40.685815+0000","last_peered":"2024-08-30T08:59:40.685815+0000","last_clean":"2024-08-30T08:59:40.685815+0000","last_became_active":"2024-08-30T08:58:00.389459+0000","last_became_peered":"2024-08-30T08:58:00.389459+0000","last_unstale":"2024-08-30T08:59:40.685815+0000","last_undegraded":"2024-08-30T08:59:40.685815+0000","last_fullsized":"2024-08-30T08:59:40.685815+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T18:32:06.214570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":52,"num_read_kb":52,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.553852+0000","last_change":"2024-08-30T08:58:09.445317+0000","last_active":"2024-08-30T08:58:46.553852+0000","last_peered":"2024-08-30T08:58:46.553852+0000","last_clean":"2024-08-30T08:58:46.553852+0000","last_became_active":"2024-08-30T08:58:09.444921+0000","last_became_peered":"2024-08-30T08:58:09.444921+0000","last_unstale":"2024-08-30T08:58:46.553852+0000","last_undegraded":"2024-08-30T08:58:46.553852+0000","last_fullsized":"2024-08-30T08:58:46.553852+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T09:22:13.038437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.556314+0000","last_change":"2024-08-30T08:58:09.447469+0000","last_active":"2024-08-30T08:58:46.556314+0000","last_peered":"2024-08-30T08:58:46.556314+0000","last_clean":"2024-08-30T08:58:46.556314+0000","last_became_active":"2024-08-30T08:58:09.447015+0000","last_became_peered":"2024-08-30T08:58:09.447015+0000","last_unstale":"2024-08-30T08:58:46.556314+0000","last_undegraded":"2024-08-30T08:58:46.556314+0000","last_fullsized":"2024-08-30T08:58:46.556314+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T09:18:36.734380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.557549+0000","last_change":"2024-08-30T08:58:02.403371+0000","last_active":"2024-08-30T08:58:46.557549+0000","last_peered":"2024-08-30T08:58:46.557549+0000","last_clean":"2024-08-30T08:58:46.557549+0000","last_became_active":"2024-08-30T08:58:02.403012+0000","last_became_peered":"2024-08-30T08:58:02.403012+0000","last_unstale":"2024-08-30T08:58:46.557549+0000","last_undegraded":"2024-08-30T08:58:46.557549+0000","last_fullsized":"2024-08-30T08:58:46.557549+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T10:29:29.683081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"79'1","reported_seq":36,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.774090+0000","last_change":"2024-08-30T08:58:00.394450+0000","last_active":"2024-08-30T08:58:46.774090+0000","last_peered":"2024-08-30T08:58:46.774090+0000","last_clean":"2024-08-30T08:58:46.774090+0000","last_became_active":"2024-08-30T08:58:00.391843+0000","last_became_peered":"2024-08-30T08:58:00.391843+0000","last_unstale":"2024-08-30T08:58:46.774090+0000","last_undegraded":"2024-08-30T08:58:46.774090+0000","last_fullsized":"2024-08-30T08:58:46.774090+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T10:29:18.947898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.553483+0000","last_change":"2024-08-30T08:58:09.449172+0000","last_active":"2024-08-30T08:58:46.553483+0000","last_peered":"2024-08-30T08:58:46.553483+0000","last_clean":"2024-08-30T08:58:46.553483+0000","last_became_active":"2024-08-30T08:58:09.448687+0000","last_became_peered":"2024-08-30T08:58:09.448687+0000","last_unstale":"2024-08-30T08:58:46.553483+0000","last_undegraded":"2024-08-30T08:58:46.553483+0000","last_fullsized":"2024-08-30T08:58:46.553483+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T13:23:16.090251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.553825+0000","last_change":"2024-08-30T08:58:02.400834+0000","last_active":"2024-08-30T08:58:46.553825+0000","last_peered":"2024-08-30T08:58:46.553825+0000","last_clean":"2024-08-30T08:58:46.553825+0000","last_became_active":"2024-08-30T08:58:02.400477+0000","last_became_peered":"2024-08-30T08:58:02.400477+0000","last_unstale":"2024-08-30T08:58:46.553825+0000","last_undegraded":"2024-08-30T08:58:46.553825+0000","last_fullsized":"2024-08-30T08:58:46.553825+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T13:40:45.239732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"80'2","reported_seq":37,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.551581+0000","last_change":"2024-08-30T08:58:00.381699+0000","last_active":"2024-08-30T08:58:46.551581+0000","last_peered":"2024-08-30T08:58:46.551581+0000","last_clean":"2024-08-30T08:58:46.551581+0000","last_became_active":"2024-08-30T08:58:00.381189+0000","last_became_peered":"2024-08-30T08:58:00.381189+0000","last_unstale":"2024-08-30T08:58:46.551581+0000","last_undegraded":"2024-08-30T08:58:46.551581+0000","last_fullsized":"2024-08-30T08:58:46.551581+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T14:09:16.939062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"80'3","reported_seq":38,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.553179+0000","last_change":"2024-08-30T08:58:00.387430+0000","last_active":"2024-08-30T08:58:46.553179+0000","last_peered":"2024-08-30T08:58:46.553179+0000","last_clean":"2024-08-30T08:58:46.553179+0000","last_became_active":"2024-08-30T08:58:00.387043+0000","last_became_peered":"2024-08-30T08:58:00.387043+0000","last_unstale":"2024-08-30T08:58:46.553179+0000","last_undegraded":"2024-08-30T08:58:46.553179+0000","last_fullsized":"2024-08-30T08:58:46.553179+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T13:32:25.418105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.774175+0000","last_change":"2024-08-30T08:58:02.404798+0000","last_active":"2024-08-30T08:58:46.774175+0000","last_peered":"2024-08-30T08:58:46.774175+0000","last_clean":"2024-08-30T08:58:46.774175+0000","last_became_active":"2024-08-30T08:58:02.404435+0000","last_became_peered":"2024-08-30T08:58:02.404435+0000","last_unstale":"2024-08-30T08:58:46.774175+0000","last_undegraded":"2024-08-30T08:58:46.774175+0000","last_fullsized":"2024-08-30T08:58:46.774175+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T20:31:49.472832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.555057+0000","last_change":"2024-08-30T08:58:09.448297+0000","last_active":"2024-08-30T08:58:46.555057+0000","last_peered":"2024-08-30T08:58:46.555057+0000","last_clean":"2024-08-30T08:58:46.555057+0000","last_became_active":"2024-08-30T08:58:09.447444+0000","last_became_peered":"2024-08-30T08:58:09.447444+0000","last_unstale":"2024-08-30T08:58:46.555057+0000","last_undegraded":"2024-08-30T08:58:46.555057+0000","last_fullsized":"2024-08-30T08:58:46.555057+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T11:40:03.507824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"81'6","reported_seq":95,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:59:40.686450+0000","last_change":"2024-08-30T08:58:00.392418+0000","last_active":"2024-08-30T08:59:40.686450+0000","last_peered":"2024-08-30T08:59:40.686450+0000","last_clean":"2024-08-30T08:59:40.686450+0000","last_became_active":"2024-08-30T08:58:00.391985+0000","last_became_peered":"2024-08-30T08:58:00.391985+0000","last_unstale":"2024-08-30T08:59:40.686450+0000","last_undegraded":"2024-08-30T08:59:40.686450+0000","last_fullsized":"2024-08-30T08:59:40.686450+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T17:42:18.147754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":54,"num_read_kb":2,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.557148+0000","last_change":"2024-08-30T08:58:02.408754+0000","last_active":"2024-08-30T08:58:46.557148+0000","last_peered":"2024-08-30T08:58:46.557148+0000","last_clean":"2024-08-30T08:58:46.557148+0000","last_became_active":"2024-08-30T08:58:02.407127+0000","last_became_peered":"2024-08-30T08:58:02.407127+0000","last_unstale":"2024-08-30T08:58:46.557148+0000","last_undegraded":"2024-08-30T08:58:46.557148+0000","last_fullsized":"2024-08-30T08:58:46.557148+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T16:15:41.796091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.555654+0000","last_change":"2024-08-30T08:58:09.445389+0000","last_active":"2024-08-30T08:58:46.555654+0000","last_peered":"2024-08-30T08:58:46.555654+0000","last_clean":"2024-08-30T08:58:46.555654+0000","last_became_active":"2024-08-30T08:58:09.444831+0000","last_became_peered":"2024-08-30T08:58:09.444831+0000","last_unstale":"2024-08-30T08:58:46.555654+0000","last_undegraded":"2024-08-30T08:58:46.555654+0000","last_fullsized":"2024-08-30T08:58:46.555654+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T09:55:58.162397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.553020+0000","last_change":"2024-08-30T08:58:02.415184+0000","last_active":"2024-08-30T08:58:46.553020+0000","last_peered":"2024-08-30T08:58:46.553020+0000","last_clean":"2024-08-30T08:58:46.553020+0000","last_became_active":"2024-08-30T08:58:02.414625+0000","last_became_peered":"2024-08-30T08:58:02.414625+0000","last_unstale":"2024-08-30T08:58:46.553020+0000","last_undegraded":"2024-08-30T08:58:46.553020+0000","last_fullsized":"2024-08-30T08:58:46.553020+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T11:10:31.130524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.555567+0000","last_change":"2024-08-30T08:58:02.405392+0000","last_active":"2024-08-30T08:58:46.555567+0000","last_peered":"2024-08-30T08:58:46.555567+0000","last_clean":"2024-08-30T08:58:46.555567+0000","last_became_active":"2024-08-30T08:58:02.405009+0000","last_became_peered":"2024-08-30T08:58:02.405009+0000","last_unstale":"2024-08-30T08:58:46.555567+0000","last_undegraded":"2024-08-30T08:58:46.555567+0000","last_fullsized":"2024-08-30T08:58:46.555567+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T13:22:46.375598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.555680+0000","last_change":"2024-08-30T08:58:02.402360+0000","last_active":"2024-08-30T08:58:46.555680+0000","last_peered":"2024-08-30T08:58:46.555680+0000","last_clean":"2024-08-30T08:58:46.555680+0000","last_became_active":"2024-08-30T08:58:02.401856+0000","last_became_peered":"2024-08-30T08:58:02.401856+0000","last_unstale":"2024-08-30T08:58:46.555680+0000","last_undegraded":"2024-08-30T08:58:46.555680+0000","last_fullsized":"2024-08-30T08:58:46.555680+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T10:25:14.491345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.557920+0000","last_change":"2024-08-30T08:58:02.415940+0000","last_active":"2024-08-30T08:58:46.557920+0000","last_peered":"2024-08-30T08:58:46.557920+0000","last_clean":"2024-08-30T08:58:46.557920+0000","last_became_active":"2024-08-30T08:58:02.415567+0000","last_became_peered":"2024-08-30T08:58:02.415567+0000","last_unstale":"2024-08-30T08:58:46.557920+0000","last_undegraded":"2024-08-30T08:58:46.557920+0000","last_fullsized":"2024-08-30T08:58:46.557920+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T15:02:26.270343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.552160+0000","last_change":"2024-08-30T08:58:02.406766+0000","last_active":"2024-08-30T08:58:46.552160+0000","last_peered":"2024-08-30T08:58:46.552160+0000","last_clean":"2024-08-30T08:58:46.552160+0000","last_became_active":"2024-08-30T08:58:02.406345+0000","last_became_peered":"2024-08-30T08:58:02.406345+0000","last_unstale":"2024-08-30T08:58:46.552160+0000","last_undegraded":"2024-08-30T08:58:46.552160+0000","last_fullsized":"2024-08-30T08:58:46.552160+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T13:10:32.986817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.553714+0000","last_change":"2024-08-30T08:58:02.412891+0000","last_active":"2024-08-30T08:58:46.553714+0000","last_peered":"2024-08-30T08:58:46.553714+0000","last_clean":"2024-08-30T08:58:46.553714+0000","last_became_active":"2024-08-30T08:58:02.412532+0000","last_became_peered":"2024-08-30T08:58:02.412532+0000","last_unstale":"2024-08-30T08:58:46.553714+0000","last_undegraded":"2024-08-30T08:58:46.553714+0000","last_fullsized":"2024-08-30T08:58:46.553714+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T14:40:30.663678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.556273+0000","last_change":"2024-08-30T08:58:02.405862+0000","last_active":"2024-08-30T08:58:46.556273+0000","last_peered":"2024-08-30T08:58:46.556273+0000","last_clean":"2024-08-30T08:58:46.556273+0000","last_became_active":"2024-08-30T08:58:02.405319+0000","last_became_peered":"2024-08-30T08:58:02.405319+0000","last_unstale":"2024-08-30T08:58:46.556273+0000","last_undegraded":"2024-08-30T08:58:46.556273+0000","last_fullsized":"2024-08-30T08:58:46.556273+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T20:36:10.315814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.555535+0000","last_change":"2024-08-30T08:58:02.401410+0000","last_active":"2024-08-30T08:58:46.555535+0000","last_peered":"2024-08-30T08:58:46.555535+0000","last_clean":"2024-08-30T08:58:46.555535+0000","last_became_active":"2024-08-30T08:58:02.401060+0000","last_became_peered":"2024-08-30T08:58:02.401060+0000","last_unstale":"2024-08-30T08:58:46.555535+0000","last_undegraded":"2024-08-30T08:58:46.555535+0000","last_fullsized":"2024-08-30T08:58:46.555535+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T16:00:41.275367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.773667+0000","last_change":"2024-08-30T08:58:02.401703+0000","last_active":"2024-08-30T08:58:46.773667+0000","last_peered":"2024-08-30T08:58:46.773667+0000","last_clean":"2024-08-30T08:58:46.773667+0000","last_became_active":"2024-08-30T08:58:02.401218+0000","last_became_peered":"2024-08-30T08:58:02.401218+0000","last_unstale":"2024-08-30T08:58:46.773667+0000","last_undegraded":"2024-08-30T08:58:46.773667+0000","last_fullsized":"2024-08-30T08:58:46.773667+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T18:36:25.249814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.554007+0000","last_change":"2024-08-30T08:58:02.403609+0000","last_active":"2024-08-30T08:58:46.554007+0000","last_peered":"2024-08-30T08:58:46.554007+0000","last_clean":"2024-08-30T08:58:46.554007+0000","last_became_active":"2024-08-30T08:58:02.402195+0000","last_became_peered":"2024-08-30T08:58:02.402195+0000","last_unstale":"2024-08-30T08:58:46.554007+0000","last_undegraded":"2024-08-30T08:58:46.554007+0000","last_fullsized":"2024-08-30T08:58:46.554007+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T09:59:05.877523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.823042+0000","last_change":"2024-08-30T08:58:02.404389+0000","last_active":"2024-08-30T08:58:46.823042+0000","last_peered":"2024-08-30T08:58:46.823042+0000","last_clean":"2024-08-30T08:58:46.823042+0000","last_became_active":"2024-08-30T08:58:02.404039+0000","last_became_peered":"2024-08-30T08:58:02.404039+0000","last_unstale":"2024-08-30T08:58:46.823042+0000","last_undegraded":"2024-08-30T08:58:46.823042+0000","last_fullsized":"2024-08-30T08:58:46.823042+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T20:55:51.739542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.552060+0000","last_change":"2024-08-30T08:58:02.411807+0000","last_active":"2024-08-30T08:58:46.552060+0000","last_peered":"2024-08-30T08:58:46.552060+0000","last_clean":"2024-08-30T08:58:46.552060+0000","last_became_active":"2024-08-30T08:58:02.410977+0000","last_became_peered":"2024-08-30T08:58:02.410977+0000","last_unstale":"2024-08-30T08:58:46.552060+0000","last_undegraded":"2024-08-30T08:58:46.552060+0000","last_fullsized":"2024-08-30T08:58:46.552060+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T13:04:29.931373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":37,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.560319+0000","last_change":"2024-08-30T08:58:46.559661+0000","last_active":"2024-08-30T08:58:46.560319+0000","last_peered":"2024-08-30T08:58:46.560319+0000","last_clean":"2024-08-30T08:58:46.560319+0000","last_became_active":"2024-08-30T08:58:46.558824+0000","last_became_peered":"2024-08-30T08:58:46.558824+0000","last_unstale":"2024-08-30T08:58:46.560319+0000","last_undegraded":"2024-08-30T08:58:46.560319+0000","last_fullsized":"2024-08-30T08:58:46.560319+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T09:53:07.650921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.556117+0000","last_change":"2024-08-30T08:58:02.396186+0000","last_active":"2024-08-30T08:58:46.556117+0000","last_peered":"2024-08-30T08:58:46.556117+0000","last_clean":"2024-08-30T08:58:46.556117+0000","last_became_active":"2024-08-30T08:58:02.395776+0000","last_became_peered":"2024-08-30T08:58:02.395776+0000","last_unstale":"2024-08-30T08:58:46.556117+0000","last_undegraded":"2024-08-30T08:58:46.556117+0000","last_fullsized":"2024-08-30T08:58:46.556117+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T09:33:30.068673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.559487+0000","last_change":"2024-08-30T08:58:02.408378+0000","last_active":"2024-08-30T08:58:46.559487+0000","last_peered":"2024-08-30T08:58:46.559487+0000","last_clean":"2024-08-30T08:58:46.559487+0000","last_became_active":"2024-08-30T08:58:02.407991+0000","last_became_peered":"2024-08-30T08:58:02.407991+0000","last_unstale":"2024-08-30T08:58:46.559487+0000","last_undegraded":"2024-08-30T08:58:46.559487+0000","last_fullsized":"2024-08-30T08:58:46.559487+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T20:57:02.411176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.553823+0000","last_change":"2024-08-30T08:58:02.411829+0000","last_active":"2024-08-30T08:58:46.553823+0000","last_peered":"2024-08-30T08:58:46.553823+0000","last_clean":"2024-08-30T08:58:46.553823+0000","last_became_active":"2024-08-30T08:58:02.411416+0000","last_became_peered":"2024-08-30T08:58:02.411416+0000","last_unstale":"2024-08-30T08:58:46.553823+0000","last_undegraded":"2024-08-30T08:58:46.553823+0000","last_fullsized":"2024-08-30T08:58:46.553823+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T17:12:08.855150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.822489+0000","last_change":"2024-08-30T08:58:02.405122+0000","last_active":"2024-08-30T08:58:46.822489+0000","last_peered":"2024-08-30T08:58:46.822489+0000","last_clean":"2024-08-30T08:58:46.822489+0000","last_became_active":"2024-08-30T08:58:02.403312+0000","last_became_peered":"2024-08-30T08:58:02.403312+0000","last_unstale":"2024-08-30T08:58:46.822489+0000","last_undegraded":"2024-08-30T08:58:46.822489+0000","last_fullsized":"2024-08-30T08:58:46.822489+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T18:11:53.077433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.556239+0000","last_change":"2024-08-30T08:58:02.404263+0000","last_active":"2024-08-30T08:58:46.556239+0000","last_peered":"2024-08-30T08:58:46.556239+0000","last_clean":"2024-08-30T08:58:46.556239+0000","last_became_active":"2024-08-30T08:58:02.403897+0000","last_became_peered":"2024-08-30T08:58:02.403897+0000","last_unstale":"2024-08-30T08:58:46.556239+0000","last_undegraded":"2024-08-30T08:58:46.556239+0000","last_fullsized":"2024-08-30T08:58:46.556239+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T18:38:31.439434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.556196+0000","last_change":"2024-08-30T08:58:02.411578+0000","last_active":"2024-08-30T08:58:46.556196+0000","last_peered":"2024-08-30T08:58:46.556196+0000","last_clean":"2024-08-30T08:58:46.556196+0000","last_became_active":"2024-08-30T08:58:02.411271+0000","last_became_peered":"2024-08-30T08:58:02.411271+0000","last_unstale":"2024-08-30T08:58:46.556196+0000","last_undegraded":"2024-08-30T08:58:46.556196+0000","last_fullsized":"2024-08-30T08:58:46.556196+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T10:02:46.382906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.552115+0000","last_change":"2024-08-30T08:58:02.412168+0000","last_active":"2024-08-30T08:58:46.552115+0000","last_peered":"2024-08-30T08:58:46.552115+0000","last_clean":"2024-08-30T08:58:46.552115+0000","last_became_active":"2024-08-30T08:58:02.411420+0000","last_became_peered":"2024-08-30T08:58:02.411420+0000","last_unstale":"2024-08-30T08:58:46.552115+0000","last_undegraded":"2024-08-30T08:58:46.552115+0000","last_fullsized":"2024-08-30T08:58:46.552115+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T10:35:53.654344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:47.089927+0000","last_change":"2024-08-30T08:58:02.409735+0000","last_active":"2024-08-30T08:58:47.089927+0000","last_peered":"2024-08-30T08:58:47.089927+0000","last_clean":"2024-08-30T08:58:47.089927+0000","last_became_active":"2024-08-30T08:58:02.409349+0000","last_became_peered":"2024-08-30T08:58:02.409349+0000","last_unstale":"2024-08-30T08:58:47.089927+0000","last_undegraded":"2024-08-30T08:58:47.089927+0000","last_fullsized":"2024-08-30T08:58:47.089927+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T20:10:22.956554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:47.090762+0000","last_change":"2024-08-30T08:58:02.416261+0000","last_active":"2024-08-30T08:58:47.090762+0000","last_peered":"2024-08-30T08:58:47.090762+0000","last_clean":"2024-08-30T08:58:47.090762+0000","last_became_active":"2024-08-30T08:58:02.415194+0000","last_became_peered":"2024-08-30T08:58:02.415194+0000","last_unstale":"2024-08-30T08:58:47.090762+0000","last_undegraded":"2024-08-30T08:58:47.090762+0000","last_fullsized":"2024-08-30T08:58:47.090762+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T17:59:04.339699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":37,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.560326+0000","last_change":"2024-08-30T08:58:46.560190+0000","last_active":"2024-08-30T08:58:46.560326+0000","last_peered":"2024-08-30T08:58:46.560326+0000","last_clean":"2024-08-30T08:58:46.560326+0000","last_became_active":"2024-08-30T08:58:46.559674+0000","last_became_peered":"2024-08-30T08:58:46.559674+0000","last_unstale":"2024-08-30T08:58:46.560326+0000","last_undegraded":"2024-08-30T08:58:46.560326+0000","last_fullsized":"2024-08-30T08:58:46.560326+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T10:08:02.835142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.823261+0000","last_change":"2024-08-30T08:58:02.406763+0000","last_active":"2024-08-30T08:58:46.823261+0000","last_peered":"2024-08-30T08:58:46.823261+0000","last_clean":"2024-08-30T08:58:46.823261+0000","last_became_active":"2024-08-30T08:58:02.406377+0000","last_became_peered":"2024-08-30T08:58:02.406377+0000","last_unstale":"2024-08-30T08:58:46.823261+0000","last_undegraded":"2024-08-30T08:58:46.823261+0000","last_fullsized":"2024-08-30T08:58:46.823261+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T13:32:50.492921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.553816+0000","last_change":"2024-08-30T08:58:02.416638+0000","last_active":"2024-08-30T08:58:46.553816+0000","last_peered":"2024-08-30T08:58:46.553816+0000","last_clean":"2024-08-30T08:58:46.553816+0000","last_became_active":"2024-08-30T08:58:02.415518+0000","last_became_peered":"2024-08-30T08:58:02.415518+0000","last_unstale":"2024-08-30T08:58:46.553816+0000","last_undegraded":"2024-08-30T08:58:46.553816+0000","last_fullsized":"2024-08-30T08:58:46.553816+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T17:05:07.531585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:47.089992+0000","last_change":"2024-08-30T08:58:02.412973+0000","last_active":"2024-08-30T08:58:47.089992+0000","last_peered":"2024-08-30T08:58:47.089992+0000","last_clean":"2024-08-30T08:58:47.089992+0000","last_became_active":"2024-08-30T08:58:02.412609+0000","last_became_peered":"2024-08-30T08:58:02.412609+0000","last_unstale":"2024-08-30T08:58:47.089992+0000","last_undegraded":"2024-08-30T08:58:47.089992+0000","last_fullsized":"2024-08-30T08:58:47.089992+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T14:22:37.704098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.553161+0000","last_change":"2024-08-30T08:58:02.402779+0000","last_active":"2024-08-30T08:58:46.553161+0000","last_peered":"2024-08-30T08:58:46.553161+0000","last_clean":"2024-08-30T08:58:46.553161+0000","last_became_active":"2024-08-30T08:58:02.401514+0000","last_became_peered":"2024-08-30T08:58:02.401514+0000","last_unstale":"2024-08-30T08:58:46.553161+0000","last_undegraded":"2024-08-30T08:58:46.553161+0000","last_fullsized":"2024-08-30T08:58:46.553161+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T11:37:51.450334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.557162+0000","last_change":"2024-08-30T08:58:02.406607+0000","last_active":"2024-08-30T08:58:46.557162+0000","last_peered":"2024-08-30T08:58:46.557162+0000","last_clean":"2024-08-30T08:58:46.557162+0000","last_became_active":"2024-08-30T08:58:02.406237+0000","last_became_peered":"2024-08-30T08:58:02.406237+0000","last_unstale":"2024-08-30T08:58:46.557162+0000","last_undegraded":"2024-08-30T08:58:46.557162+0000","last_fullsized":"2024-08-30T08:58:46.557162+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T12:09:03.247424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:47.089986+0000","last_change":"2024-08-30T08:58:02.410749+0000","last_active":"2024-08-30T08:58:47.089986+0000","last_peered":"2024-08-30T08:58:47.089986+0000","last_clean":"2024-08-30T08:58:47.089986+0000","last_became_active":"2024-08-30T08:58:02.410350+0000","last_became_peered":"2024-08-30T08:58:02.410350+0000","last_unstale":"2024-08-30T08:58:47.089986+0000","last_undegraded":"2024-08-30T08:58:47.089986+0000","last_fullsized":"2024-08-30T08:58:47.089986+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T15:22:51.451789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.557090+0000","last_change":"2024-08-30T08:58:02.405946+0000","last_active":"2024-08-30T08:58:46.557090+0000","last_peered":"2024-08-30T08:58:46.557090+0000","last_clean":"2024-08-30T08:58:46.557090+0000","last_became_active":"2024-08-30T08:58:02.405590+0000","last_became_peered":"2024-08-30T08:58:02.405590+0000","last_unstale":"2024-08-30T08:58:46.557090+0000","last_undegraded":"2024-08-30T08:58:46.557090+0000","last_fullsized":"2024-08-30T08:58:46.557090+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T11:42:14.966065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.556440+0000","last_change":"2024-08-30T08:58:02.409649+0000","last_active":"2024-08-30T08:58:46.556440+0000","last_peered":"2024-08-30T08:58:46.556440+0000","last_clean":"2024-08-30T08:58:46.556440+0000","last_became_active":"2024-08-30T08:58:02.409282+0000","last_became_peered":"2024-08-30T08:58:02.409282+0000","last_unstale":"2024-08-30T08:58:46.556440+0000","last_undegraded":"2024-08-30T08:58:46.556440+0000","last_fullsized":"2024-08-30T08:58:46.556440+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T15:33:52.166290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.553800+0000","last_change":"2024-08-30T08:58:02.410711+0000","last_active":"2024-08-30T08:58:46.553800+0000","last_peered":"2024-08-30T08:58:46.553800+0000","last_clean":"2024-08-30T08:58:46.553800+0000","last_became_active":"2024-08-30T08:58:02.410367+0000","last_became_peered":"2024-08-30T08:58:02.410367+0000","last_unstale":"2024-08-30T08:58:46.553800+0000","last_undegraded":"2024-08-30T08:58:46.553800+0000","last_fullsized":"2024-08-30T08:58:46.553800+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T15:27:40.392588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":68452,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":339,"num_read_kb":242,"num_write":171,"num_write_kb":120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":205356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":146,"ondisk_log_size":146,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":67,"seq":287762808864,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":925696,"data_stored":455486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":61,"seq":261993005094,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28596,"kb_used_data":1572,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716556,"statfs":{"total":95995035648,"available":95965753344,"internally_reserved":0,"allocated":1609728,"data_stored":1107093,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":57,"seq":244813135915,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36260,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708892,"statfs":{"total":95995035648,"available":95957905408,"internally_reserved":0,"allocated":937984,"data_stored":459706,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":51,"seq":219043332145,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":925696,"data_stored":455418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":46,"seq":197568495669,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28516,"kb_used_data":1492,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716636,"statfs":{"total":95995035648,"available":95965835264,"internally_reserved":0,"allocated":1527808,"data_stored":1051830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":41,"seq":176093659195,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":913408,"data_stored":450786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":35,"seq":150323855424,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36844,"kb_used_data":1500,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708308,"statfs":{"total":95995035648,"available":95957307392,"internally_reserved":0,"allocated":1536000,"data_stored":1047027,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":30,"seq":128849018949,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32088,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713064,"statfs":{"total":95995035648,"available":95962177536,"internally_reserved":0,"allocated":925696,"data_stored":455827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215177,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36248,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708904,"statfs":{"total":95995035648,"available":95957917696,"internally_reserved":0,"allocated":925696,"data_stored":450980,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411407,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36328,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708824,"statfs":{"total":95995035648,"available":95957835776,"internally_reserved":0,"allocated":1007616,"data_stored":516497,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574932,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":983040,"data_stored":514756,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738456,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":937984,"data_stored":452174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":64060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":65801,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":66029,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-30T08:59:43.717 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph pg dump --format=json 2024-08-30T08:59:43.974 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T08:59:44.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:44.440+0000 7fcd8ee93640 1 -- 172.21.15.42:0/1151597113 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcd8810fec0 msgr2=0x7fcd881122e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:44.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:44.440+0000 7fcd8ee93640 1 --2- 172.21.15.42:0/1151597113 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcd8810fec0 0x7fcd881122e0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fcd80009ca0 tx=0x7fcd8002a8c0 comp rx=0 tx=0).stop 2024-08-30T08:59:44.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:44.440+0000 7fcd8ee93640 1 -- 172.21.15.42:0/1151597113 shutdown_connections 2024-08-30T08:59:44.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:44.440+0000 7fcd8ee93640 1 --2- 172.21.15.42:0/1151597113 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcd88075ea0 0x7fcd8810f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:44.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:44.440+0000 7fcd8ee93640 1 --2- 172.21.15.42:0/1151597113 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcd8810fec0 0x7fcd881122e0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:44.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:44.440+0000 7fcd8ee93640 1 --2- 172.21.15.42:0/1151597113 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcd88075550 0x7fcd88075960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:44.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:44.440+0000 7fcd8ee93640 1 -- 172.21.15.42:0/1151597113 >> 172.21.15.42:0/1151597113 conn(0x7fcd8807b6b0 msgr2=0x7fcd8807bac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:59:44.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:44.440+0000 7fcd8ee93640 1 -- 172.21.15.42:0/1151597113 shutdown_connections 2024-08-30T08:59:44.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:44.440+0000 7fcd8ee93640 1 -- 172.21.15.42:0/1151597113 wait complete. 2024-08-30T08:59:44.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:44.441+0000 7fcd8ee93640 1 Processor -- start 2024-08-30T08:59:44.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:44.441+0000 7fcd8ee93640 1 -- start start 2024-08-30T08:59:44.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:44.441+0000 7fcd8ee93640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcd88075550 0x7fcd8819c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:44.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:44.441+0000 7fcd8ee93640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcd88075ea0 0x7fcd8819c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:44.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:44.441+0000 7fcd8ee93640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcd8810fec0 0x7fcd881a1c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:44.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:44.441+0000 7fcd8ee93640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fcd88115a30 con 0x7fcd88075550 2024-08-30T08:59:44.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:44.441+0000 7fcd8ee93640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fcd881158b0 con 0x7fcd8810fec0 2024-08-30T08:59:44.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:44.441+0000 7fcd8ee93640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fcd88115bb0 con 0x7fcd88075ea0 2024-08-30T08:59:44.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:44.441+0000 7fcd8cc2f640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcd88075550 0x7fcd8819c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:44.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:44.441+0000 7fcd8cc2f640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcd88075550 0x7fcd8819c3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:40118/0 (socket says 172.21.15.42:40118) 2024-08-30T08:59:44.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:44.441+0000 7fcd8cc2f640 1 -- 172.21.15.42:0/206210931 learned_addr learned my addr 172.21.15.42:0/206210931 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T08:59:44.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:44.441+0000 7fcd8d430640 1 --2- 172.21.15.42:0/206210931 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcd8810fec0 0x7fcd881a1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:44.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:44.442+0000 7fcd87fff640 1 --2- 172.21.15.42:0/206210931 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcd88075ea0 0x7fcd8819c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:44.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:44.442+0000 7fcd8cc2f640 1 -- 172.21.15.42:0/206210931 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcd88075ea0 msgr2=0x7fcd8819c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:44.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:44.442+0000 7fcd8cc2f640 1 --2- 172.21.15.42:0/206210931 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcd88075ea0 0x7fcd8819c980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:44.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:44.442+0000 7fcd8cc2f640 1 -- 172.21.15.42:0/206210931 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcd8810fec0 msgr2=0x7fcd881a1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:44.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:44.442+0000 7fcd8cc2f640 1 --2- 172.21.15.42:0/206210931 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcd8810fec0 0x7fcd881a1c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:44.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:44.442+0000 7fcd8cc2f640 1 -- 172.21.15.42:0/206210931 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd881a2290 con 0x7fcd88075550 2024-08-30T08:59:44.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:44.442+0000 7fcd8d430640 1 --2- 172.21.15.42:0/206210931 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcd8810fec0 0x7fcd881a1c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-30T08:59:44.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:44.442+0000 7fcd8cc2f640 1 --2- 172.21.15.42:0/206210931 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcd88075550 0x7fcd8819c3e0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fcd7800c920 tx=0x7fcd7800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:59:44.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:44.442+0000 7fcd85ffb640 1 -- 172.21.15.42:0/206210931 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd78007c70 con 0x7fcd88075550 2024-08-30T08:59:44.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:44.442+0000 7fcd85ffb640 1 -- 172.21.15.42:0/206210931 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcd78007e10 con 0x7fcd88075550 2024-08-30T08:59:44.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:44.442+0000 7fcd8ee93640 1 -- 172.21.15.42:0/206210931 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd881a2580 con 0x7fcd88075550 2024-08-30T08:59:44.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:44.442+0000 7fcd85ffb640 1 -- 172.21.15.42:0/206210931 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd78018820 con 0x7fcd88075550 2024-08-30T08:59:44.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:44.442+0000 7fcd8ee93640 1 -- 172.21.15.42:0/206210931 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd881a8da0 con 0x7fcd88075550 2024-08-30T08:59:44.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:44.443+0000 7fcd8ee93640 1 -- 172.21.15.42:0/206210931 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd88075960 con 0x7fcd88075550 2024-08-30T08:59:44.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:44.445+0000 7fcd85ffb640 1 -- 172.21.15.42:0/206210931 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) ==== 152628+0+0 (secure 0 0 0) 0x7fcd7800f450 con 0x7fcd88075550 2024-08-30T08:59:44.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:44.445+0000 7fcd85ffb640 1 --2- 172.21.15.42:0/206210931 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fcd700b4bf0 0x7fcd700b7100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:44.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:44.445+0000 7fcd87fff640 1 --2- 172.21.15.42:0/206210931 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fcd700b4bf0 0x7fcd700b7100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:44.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:44.446+0000 7fcd85ffb640 1 -- 172.21.15.42:0/206210931 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 9092+0+0 (secure 0 0 0) 0x7fcd780e57e0 con 0x7fcd88075550 2024-08-30T08:59:44.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:44.446+0000 7fcd87fff640 1 --2- 172.21.15.42:0/206210931 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fcd700b4bf0 0x7fcd700b7100 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fcd88111c40 tx=0x7fcd7c005be0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:59:44.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:44.450+0000 7fcd85ffb640 1 -- 172.21.15.42:0/206210931 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7fcd78014030 con 0x7fcd88075550 2024-08-30T08:59:44.638 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:44.636+0000 7fcd8ee93640 1 -- 172.21.15.42:0/206210931 --> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fcd88102af0 con 0x7fcd700b4bf0 2024-08-30T08:59:44.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:44.642+0000 7fcd85ffb640 1 -- 172.21.15.42:0/206210931 <== mgr.14150 v2:172.21.15.42:6800/3005140902 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340326 (secure 0 0 0) 0x7fcd88102af0 con 0x7fcd700b4bf0 2024-08-30T08:59:44.647 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-08-30T08:59:44.649 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-08-30T08:59:44.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:44.645+0000 7fcd8ee93640 1 -- 172.21.15.42:0/206210931 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fcd700b4bf0 msgr2=0x7fcd700b7100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:44.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:44.645+0000 7fcd8ee93640 1 --2- 172.21.15.42:0/206210931 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fcd700b4bf0 0x7fcd700b7100 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fcd88111c40 tx=0x7fcd7c005be0 comp rx=0 tx=0).stop 2024-08-30T08:59:44.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:44.645+0000 7fcd8ee93640 1 -- 172.21.15.42:0/206210931 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcd88075550 msgr2=0x7fcd8819c3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:44.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:44.645+0000 7fcd8ee93640 1 --2- 172.21.15.42:0/206210931 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcd88075550 0x7fcd8819c3e0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fcd7800c920 tx=0x7fcd7800cde0 comp rx=0 tx=0).stop 2024-08-30T08:59:44.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:44.645+0000 7fcd8ee93640 1 -- 172.21.15.42:0/206210931 shutdown_connections 2024-08-30T08:59:44.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:44.645+0000 7fcd8ee93640 1 --2- 172.21.15.42:0/206210931 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fcd700b4bf0 0x7fcd700b7100 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:44.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:44.645+0000 7fcd8ee93640 1 --2- 172.21.15.42:0/206210931 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcd88075ea0 0x7fcd8819c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:44.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:44.645+0000 7fcd8ee93640 1 --2- 172.21.15.42:0/206210931 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcd8810fec0 0x7fcd881a1c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:44.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:44.645+0000 7fcd8ee93640 1 --2- 172.21.15.42:0/206210931 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcd88075550 0x7fcd8819c3e0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:44.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:44.645+0000 7fcd8ee93640 1 -- 172.21.15.42:0/206210931 >> 172.21.15.42:0/206210931 conn(0x7fcd8807b6b0 msgr2=0x7fcd881023e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:59:44.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:44.645+0000 7fcd8ee93640 1 -- 172.21.15.42:0/206210931 shutdown_connections 2024-08-30T08:59:44.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:44.645+0000 7fcd8ee93640 1 -- 172.21.15.42:0/206210931 wait complete. 2024-08-30T08:59:44.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:59:44 smithi190 ceph-mon[22001]: pgmap v340: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-08-30T08:59:44.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:59:44 smithi190 ceph-mon[22001]: from='client.15189 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-30T08:59:45.005 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:59:44 smithi042 ceph-mon[18652]: pgmap v340: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-08-30T08:59:45.005 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:59:44 smithi042 ceph-mon[18652]: from='client.15189 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-30T08:59:45.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:59:44 smithi167 ceph-mon[22063]: pgmap v340: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-08-30T08:59:45.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:59:44 smithi167 ceph-mon[22063]: from='client.15189 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-30T08:59:45.380 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":340,"stamp":"2024-08-30T08:59:42.951644+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":659054,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":415,"num_read_kb":306,"num_write":296,"num_write_kb":2272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":240,"ondisk_log_size":240,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":374580,"kb_used_data":12848,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124567244,"statfs":{"total":1151940427776,"available":1151556857856,"internally_reserved":0,"allocated":13156352,"data_stored":7417580,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":370193721},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":90,"num_read_kb":45,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003698"},"pg_stats":[{"pgid":"2.1f","version":"80'9","reported_seq":45,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.559175+0000","last_change":"2024-08-30T08:58:00.388595+0000","last_active":"2024-08-30T08:58:46.559175+0000","last_peered":"2024-08-30T08:58:46.559175+0000","last_clean":"2024-08-30T08:58:46.559175+0000","last_became_active":"2024-08-30T08:58:00.388215+0000","last_became_peered":"2024-08-30T08:58:00.388215+0000","last_unstale":"2024-08-30T08:58:46.559175+0000","last_undegraded":"2024-08-30T08:58:46.559175+0000","last_fullsized":"2024-08-30T08:58:46.559175+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T13:00:06.109935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.555091+0000","last_change":"2024-08-30T08:58:02.416042+0000","last_active":"2024-08-30T08:58:46.555091+0000","last_peered":"2024-08-30T08:58:46.555091+0000","last_clean":"2024-08-30T08:58:46.555091+0000","last_became_active":"2024-08-30T08:58:02.415507+0000","last_became_peered":"2024-08-30T08:58:02.415507+0000","last_unstale":"2024-08-30T08:58:46.555091+0000","last_undegraded":"2024-08-30T08:58:46.555091+0000","last_fullsized":"2024-08-30T08:58:46.555091+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T14:33:57.192004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.554665+0000","last_change":"2024-08-30T08:58:09.448785+0000","last_active":"2024-08-30T08:58:46.554665+0000","last_peered":"2024-08-30T08:58:46.554665+0000","last_clean":"2024-08-30T08:58:46.554665+0000","last_became_active":"2024-08-30T08:58:09.448191+0000","last_became_peered":"2024-08-30T08:58:09.448191+0000","last_unstale":"2024-08-30T08:58:46.554665+0000","last_undegraded":"2024-08-30T08:58:46.554665+0000","last_fullsized":"2024-08-30T08:58:46.554665+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T13:37:19.148201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"80'2","reported_seq":37,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.555518+0000","last_change":"2024-08-30T08:58:00.480406+0000","last_active":"2024-08-30T08:58:46.555518+0000","last_peered":"2024-08-30T08:58:46.555518+0000","last_clean":"2024-08-30T08:58:46.555518+0000","last_became_active":"2024-08-30T08:58:00.479896+0000","last_became_peered":"2024-08-30T08:58:00.479896+0000","last_unstale":"2024-08-30T08:58:46.555518+0000","last_undegraded":"2024-08-30T08:58:46.555518+0000","last_fullsized":"2024-08-30T08:58:46.555518+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T15:16:15.316343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.557006+0000","last_change":"2024-08-30T08:58:02.406915+0000","last_active":"2024-08-30T08:58:46.557006+0000","last_peered":"2024-08-30T08:58:46.557006+0000","last_clean":"2024-08-30T08:58:46.557006+0000","last_became_active":"2024-08-30T08:58:02.406557+0000","last_became_peered":"2024-08-30T08:58:02.406557+0000","last_unstale":"2024-08-30T08:58:46.557006+0000","last_undegraded":"2024-08-30T08:58:46.557006+0000","last_fullsized":"2024-08-30T08:58:46.557006+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T19:15:30.847212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:47.089607+0000","last_change":"2024-08-30T08:58:09.446974+0000","last_active":"2024-08-30T08:58:47.089607+0000","last_peered":"2024-08-30T08:58:47.089607+0000","last_clean":"2024-08-30T08:58:47.089607+0000","last_became_active":"2024-08-30T08:58:09.446173+0000","last_became_peered":"2024-08-30T08:58:09.446173+0000","last_unstale":"2024-08-30T08:58:47.089607+0000","last_undegraded":"2024-08-30T08:58:47.089607+0000","last_fullsized":"2024-08-30T08:58:47.089607+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T12:13:41.276825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"80'4","reported_seq":39,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.557615+0000","last_change":"2024-08-30T08:58:00.390609+0000","last_active":"2024-08-30T08:58:46.557615+0000","last_peered":"2024-08-30T08:58:46.557615+0000","last_clean":"2024-08-30T08:58:46.557615+0000","last_became_active":"2024-08-30T08:58:00.390229+0000","last_became_peered":"2024-08-30T08:58:00.390229+0000","last_unstale":"2024-08-30T08:58:46.557615+0000","last_undegraded":"2024-08-30T08:58:46.557615+0000","last_fullsized":"2024-08-30T08:58:46.557615+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T11:58:25.037431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.822738+0000","last_change":"2024-08-30T08:58:02.415594+0000","last_active":"2024-08-30T08:58:46.822738+0000","last_peered":"2024-08-30T08:58:46.822738+0000","last_clean":"2024-08-30T08:58:46.822738+0000","last_became_active":"2024-08-30T08:58:02.415066+0000","last_became_peered":"2024-08-30T08:58:02.415066+0000","last_unstale":"2024-08-30T08:58:46.822738+0000","last_undegraded":"2024-08-30T08:58:46.822738+0000","last_fullsized":"2024-08-30T08:58:46.822738+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T12:07:27.668442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:47.089672+0000","last_change":"2024-08-30T08:58:09.444403+0000","last_active":"2024-08-30T08:58:47.089672+0000","last_peered":"2024-08-30T08:58:47.089672+0000","last_clean":"2024-08-30T08:58:47.089672+0000","last_became_active":"2024-08-30T08:58:09.443914+0000","last_became_peered":"2024-08-30T08:58:09.443914+0000","last_unstale":"2024-08-30T08:58:47.089672+0000","last_undegraded":"2024-08-30T08:58:47.089672+0000","last_fullsized":"2024-08-30T08:58:47.089672+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T14:17:43.866352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"73'2","reported_seq":37,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.557996+0000","last_change":"2024-08-30T08:58:00.392108+0000","last_active":"2024-08-30T08:58:46.557996+0000","last_peered":"2024-08-30T08:58:46.557996+0000","last_clean":"2024-08-30T08:58:46.557996+0000","last_became_active":"2024-08-30T08:58:00.391157+0000","last_became_peered":"2024-08-30T08:58:00.391157+0000","last_unstale":"2024-08-30T08:58:46.557996+0000","last_undegraded":"2024-08-30T08:58:46.557996+0000","last_fullsized":"2024-08-30T08:58:46.557996+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T17:21:03.369776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.774221+0000","last_change":"2024-08-30T08:58:02.405550+0000","last_active":"2024-08-30T08:58:46.774221+0000","last_peered":"2024-08-30T08:58:46.774221+0000","last_clean":"2024-08-30T08:58:46.774221+0000","last_became_active":"2024-08-30T08:58:02.405129+0000","last_became_peered":"2024-08-30T08:58:02.405129+0000","last_unstale":"2024-08-30T08:58:46.774221+0000","last_undegraded":"2024-08-30T08:58:46.774221+0000","last_fullsized":"2024-08-30T08:58:46.774221+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T11:36:47.624026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:47.089364+0000","last_change":"2024-08-30T08:58:09.445746+0000","last_active":"2024-08-30T08:58:47.089364+0000","last_peered":"2024-08-30T08:58:47.089364+0000","last_clean":"2024-08-30T08:58:47.089364+0000","last_became_active":"2024-08-30T08:58:09.445257+0000","last_became_peered":"2024-08-30T08:58:09.445257+0000","last_unstale":"2024-08-30T08:58:47.089364+0000","last_undegraded":"2024-08-30T08:58:47.089364+0000","last_fullsized":"2024-08-30T08:58:47.089364+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T18:02:52.007131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:47.090324+0000","last_change":"2024-08-30T08:58:02.408213+0000","last_active":"2024-08-30T08:58:47.090324+0000","last_peered":"2024-08-30T08:58:47.090324+0000","last_clean":"2024-08-30T08:58:47.090324+0000","last_became_active":"2024-08-30T08:58:02.407856+0000","last_became_peered":"2024-08-30T08:58:02.407856+0000","last_unstale":"2024-08-30T08:58:47.090324+0000","last_undegraded":"2024-08-30T08:58:47.090324+0000","last_fullsized":"2024-08-30T08:58:47.090324+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T12:00:22.226653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"79'4","reported_seq":40,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.557487+0000","last_change":"2024-08-30T08:58:00.391714+0000","last_active":"2024-08-30T08:58:46.557487+0000","last_peered":"2024-08-30T08:58:46.557487+0000","last_clean":"2024-08-30T08:58:46.557487+0000","last_became_active":"2024-08-30T08:58:00.391211+0000","last_became_peered":"2024-08-30T08:58:00.391211+0000","last_unstale":"2024-08-30T08:58:46.557487+0000","last_undegraded":"2024-08-30T08:58:46.557487+0000","last_fullsized":"2024-08-30T08:58:46.557487+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T11:55:37.469983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:47.089359+0000","last_change":"2024-08-30T08:58:09.728177+0000","last_active":"2024-08-30T08:58:47.089359+0000","last_peered":"2024-08-30T08:58:47.089359+0000","last_clean":"2024-08-30T08:58:47.089359+0000","last_became_active":"2024-08-30T08:58:09.727630+0000","last_became_peered":"2024-08-30T08:58:09.727630+0000","last_unstale":"2024-08-30T08:58:47.089359+0000","last_undegraded":"2024-08-30T08:58:47.089359+0000","last_fullsized":"2024-08-30T08:58:47.089359+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T12:07:52.965014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.554094+0000","last_change":"2024-08-30T08:58:02.403378+0000","last_active":"2024-08-30T08:58:46.554094+0000","last_peered":"2024-08-30T08:58:46.554094+0000","last_clean":"2024-08-30T08:58:46.554094+0000","last_became_active":"2024-08-30T08:58:02.403036+0000","last_became_peered":"2024-08-30T08:58:02.403036+0000","last_unstale":"2024-08-30T08:58:46.554094+0000","last_undegraded":"2024-08-30T08:58:46.554094+0000","last_fullsized":"2024-08-30T08:58:46.554094+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T16:37:14.876384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"80'1","reported_seq":36,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.555381+0000","last_change":"2024-08-30T08:58:00.393860+0000","last_active":"2024-08-30T08:58:46.555381+0000","last_peered":"2024-08-30T08:58:46.555381+0000","last_clean":"2024-08-30T08:58:46.555381+0000","last_became_active":"2024-08-30T08:58:00.392486+0000","last_became_peered":"2024-08-30T08:58:00.392486+0000","last_unstale":"2024-08-30T08:58:46.555381+0000","last_undegraded":"2024-08-30T08:58:46.555381+0000","last_fullsized":"2024-08-30T08:58:46.555381+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T12:50:27.337817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"81'1","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:47.089678+0000","last_change":"2024-08-30T08:58:09.441476+0000","last_active":"2024-08-30T08:58:47.089678+0000","last_peered":"2024-08-30T08:58:47.089678+0000","last_clean":"2024-08-30T08:58:47.089678+0000","last_became_active":"2024-08-30T08:58:09.440344+0000","last_became_peered":"2024-08-30T08:58:09.440344+0000","last_unstale":"2024-08-30T08:58:47.089678+0000","last_undegraded":"2024-08-30T08:58:47.089678+0000","last_fullsized":"2024-08-30T08:58:47.089678+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T12:06:47.113611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.553171+0000","last_change":"2024-08-30T08:58:02.407098+0000","last_active":"2024-08-30T08:58:46.553171+0000","last_peered":"2024-08-30T08:58:46.553171+0000","last_clean":"2024-08-30T08:58:46.553171+0000","last_became_active":"2024-08-30T08:58:02.406690+0000","last_became_peered":"2024-08-30T08:58:02.406690+0000","last_unstale":"2024-08-30T08:58:46.553171+0000","last_undegraded":"2024-08-30T08:58:46.553171+0000","last_fullsized":"2024-08-30T08:58:46.553171+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T18:27:39.534923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"79'3","reported_seq":38,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.554200+0000","last_change":"2024-08-30T08:58:00.386434+0000","last_active":"2024-08-30T08:58:46.554200+0000","last_peered":"2024-08-30T08:58:46.554200+0000","last_clean":"2024-08-30T08:58:46.554200+0000","last_became_active":"2024-08-30T08:58:00.385930+0000","last_became_peered":"2024-08-30T08:58:00.385930+0000","last_unstale":"2024-08-30T08:58:46.554200+0000","last_undegraded":"2024-08-30T08:58:46.554200+0000","last_fullsized":"2024-08-30T08:58:46.554200+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T10:39:22.441568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.775800+0000","last_change":"2024-08-30T08:58:46.775674+0000","last_active":"2024-08-30T08:58:46.775800+0000","last_peered":"2024-08-30T08:58:46.775800+0000","last_clean":"2024-08-30T08:58:46.775800+0000","last_became_active":"2024-08-30T08:58:46.775157+0000","last_became_peered":"2024-08-30T08:58:46.775157+0000","last_unstale":"2024-08-30T08:58:46.775800+0000","last_undegraded":"2024-08-30T08:58:46.775800+0000","last_fullsized":"2024-08-30T08:58:46.775800+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T17:13:47.066475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,2,5],"acting":[8,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.559287+0000","last_change":"2024-08-30T08:58:02.410500+0000","last_active":"2024-08-30T08:58:46.559287+0000","last_peered":"2024-08-30T08:58:46.559287+0000","last_clean":"2024-08-30T08:58:46.559287+0000","last_became_active":"2024-08-30T08:58:02.409260+0000","last_became_peered":"2024-08-30T08:58:02.409260+0000","last_unstale":"2024-08-30T08:58:46.559287+0000","last_undegraded":"2024-08-30T08:58:46.559287+0000","last_fullsized":"2024-08-30T08:58:46.559287+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T16:47:59.701956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"79'2","reported_seq":37,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.822713+0000","last_change":"2024-08-30T08:58:00.482296+0000","last_active":"2024-08-30T08:58:46.822713+0000","last_peered":"2024-08-30T08:58:46.822713+0000","last_clean":"2024-08-30T08:58:46.822713+0000","last_became_active":"2024-08-30T08:58:00.481920+0000","last_became_peered":"2024-08-30T08:58:00.481920+0000","last_unstale":"2024-08-30T08:58:46.822713+0000","last_undegraded":"2024-08-30T08:58:46.822713+0000","last_fullsized":"2024-08-30T08:58:46.822713+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T09:29:30.724410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:47.090717+0000","last_change":"2024-08-30T08:58:09.727879+0000","last_active":"2024-08-30T08:58:47.090717+0000","last_peered":"2024-08-30T08:58:47.090717+0000","last_clean":"2024-08-30T08:58:47.090717+0000","last_became_active":"2024-08-30T08:58:09.727339+0000","last_became_peered":"2024-08-30T08:58:09.727339+0000","last_unstale":"2024-08-30T08:58:47.090717+0000","last_undegraded":"2024-08-30T08:58:47.090717+0000","last_fullsized":"2024-08-30T08:58:47.090717+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T13:19:18.582668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":37,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.560099+0000","last_change":"2024-08-30T08:58:46.559988+0000","last_active":"2024-08-30T08:58:46.560099+0000","last_peered":"2024-08-30T08:58:46.560099+0000","last_clean":"2024-08-30T08:58:46.560099+0000","last_became_active":"2024-08-30T08:58:46.559442+0000","last_became_peered":"2024-08-30T08:58:46.559442+0000","last_unstale":"2024-08-30T08:58:46.560099+0000","last_undegraded":"2024-08-30T08:58:46.560099+0000","last_fullsized":"2024-08-30T08:58:46.560099+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T09:29:09.747756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"79'4","reported_seq":39,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:47.091026+0000","last_change":"2024-08-30T08:58:00.380338+0000","last_active":"2024-08-30T08:58:47.091026+0000","last_peered":"2024-08-30T08:58:47.091026+0000","last_clean":"2024-08-30T08:58:47.091026+0000","last_became_active":"2024-08-30T08:58:00.379860+0000","last_became_peered":"2024-08-30T08:58:00.379860+0000","last_unstale":"2024-08-30T08:58:47.091026+0000","last_undegraded":"2024-08-30T08:58:47.091026+0000","last_fullsized":"2024-08-30T08:58:47.091026+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T19:57:15.349906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.555200+0000","last_change":"2024-08-30T08:58:09.443631+0000","last_active":"2024-08-30T08:58:46.555200+0000","last_peered":"2024-08-30T08:58:46.555200+0000","last_clean":"2024-08-30T08:58:46.555200+0000","last_became_active":"2024-08-30T08:58:09.443103+0000","last_became_peered":"2024-08-30T08:58:09.443103+0000","last_unstale":"2024-08-30T08:58:46.555200+0000","last_undegraded":"2024-08-30T08:58:46.555200+0000","last_fullsized":"2024-08-30T08:58:46.555200+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T19:58:11.031826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.555619+0000","last_change":"2024-08-30T08:58:02.406216+0000","last_active":"2024-08-30T08:58:46.555619+0000","last_peered":"2024-08-30T08:58:46.555619+0000","last_clean":"2024-08-30T08:58:46.555619+0000","last_became_active":"2024-08-30T08:58:02.405040+0000","last_became_peered":"2024-08-30T08:58:02.405040+0000","last_unstale":"2024-08-30T08:58:46.555619+0000","last_undegraded":"2024-08-30T08:58:46.555619+0000","last_fullsized":"2024-08-30T08:58:46.555619+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T14:28:28.721386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"81'12","reported_seq":47,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:59:01.413507+0000","last_change":"2024-08-30T08:58:00.387350+0000","last_active":"2024-08-30T08:59:01.413507+0000","last_peered":"2024-08-30T08:59:01.413507+0000","last_clean":"2024-08-30T08:59:01.413507+0000","last_became_active":"2024-08-30T08:58:00.387004+0000","last_became_peered":"2024-08-30T08:58:00.387004+0000","last_unstale":"2024-08-30T08:59:01.413507+0000","last_undegraded":"2024-08-30T08:59:01.413507+0000","last_fullsized":"2024-08-30T08:59:01.413507+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T19:24:25.604544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.553684+0000","last_change":"2024-08-30T08:58:09.727388+0000","last_active":"2024-08-30T08:58:46.553684+0000","last_peered":"2024-08-30T08:58:46.553684+0000","last_clean":"2024-08-30T08:58:46.553684+0000","last_became_active":"2024-08-30T08:58:09.726935+0000","last_became_peered":"2024-08-30T08:58:09.726935+0000","last_unstale":"2024-08-30T08:58:46.553684+0000","last_undegraded":"2024-08-30T08:58:46.553684+0000","last_fullsized":"2024-08-30T08:58:46.553684+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T17:53:16.066602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.552226+0000","last_change":"2024-08-30T08:58:02.402661+0000","last_active":"2024-08-30T08:58:46.552226+0000","last_peered":"2024-08-30T08:58:46.552226+0000","last_clean":"2024-08-30T08:58:46.552226+0000","last_became_active":"2024-08-30T08:58:02.402314+0000","last_became_peered":"2024-08-30T08:58:02.402314+0000","last_unstale":"2024-08-30T08:58:46.552226+0000","last_undegraded":"2024-08-30T08:58:46.552226+0000","last_fullsized":"2024-08-30T08:58:46.552226+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T10:56:41.336312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"79'1","reported_seq":37,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.557749+0000","last_change":"2024-08-30T08:58:00.482720+0000","last_active":"2024-08-30T08:58:46.557749+0000","last_peered":"2024-08-30T08:58:46.557749+0000","last_clean":"2024-08-30T08:58:46.557749+0000","last_became_active":"2024-08-30T08:58:00.482193+0000","last_became_peered":"2024-08-30T08:58:00.482193+0000","last_unstale":"2024-08-30T08:58:46.557749+0000","last_undegraded":"2024-08-30T08:58:46.557749+0000","last_fullsized":"2024-08-30T08:58:46.557749+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T18:42:06.654806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.553347+0000","last_change":"2024-08-30T08:58:09.449970+0000","last_active":"2024-08-30T08:58:46.553347+0000","last_peered":"2024-08-30T08:58:46.553347+0000","last_clean":"2024-08-30T08:58:46.553347+0000","last_became_active":"2024-08-30T08:58:09.447891+0000","last_became_peered":"2024-08-30T08:58:09.447891+0000","last_unstale":"2024-08-30T08:58:46.553347+0000","last_undegraded":"2024-08-30T08:58:46.553347+0000","last_fullsized":"2024-08-30T08:58:46.553347+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T15:06:45.414470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.553365+0000","last_change":"2024-08-30T08:58:02.409756+0000","last_active":"2024-08-30T08:58:46.553365+0000","last_peered":"2024-08-30T08:58:46.553365+0000","last_clean":"2024-08-30T08:58:46.553365+0000","last_became_active":"2024-08-30T08:58:02.409357+0000","last_became_peered":"2024-08-30T08:58:02.409357+0000","last_unstale":"2024-08-30T08:58:46.553365+0000","last_undegraded":"2024-08-30T08:58:46.553365+0000","last_fullsized":"2024-08-30T08:58:46.553365+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T13:27:41.696527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":35,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.558745+0000","last_change":"2024-08-30T08:58:00.482342+0000","last_active":"2024-08-30T08:58:46.558745+0000","last_peered":"2024-08-30T08:58:46.558745+0000","last_clean":"2024-08-30T08:58:46.558745+0000","last_became_active":"2024-08-30T08:58:00.481902+0000","last_became_peered":"2024-08-30T08:58:00.481902+0000","last_unstale":"2024-08-30T08:58:46.558745+0000","last_undegraded":"2024-08-30T08:58:46.558745+0000","last_fullsized":"2024-08-30T08:58:46.558745+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T14:12:36.789356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.556901+0000","last_change":"2024-08-30T08:58:09.728008+0000","last_active":"2024-08-30T08:58:46.556901+0000","last_peered":"2024-08-30T08:58:46.556901+0000","last_clean":"2024-08-30T08:58:46.556901+0000","last_became_active":"2024-08-30T08:58:09.727513+0000","last_became_peered":"2024-08-30T08:58:09.727513+0000","last_unstale":"2024-08-30T08:58:46.556901+0000","last_undegraded":"2024-08-30T08:58:46.556901+0000","last_fullsized":"2024-08-30T08:58:46.556901+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T14:34:04.165300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.553329+0000","last_change":"2024-08-30T08:58:02.409383+0000","last_active":"2024-08-30T08:58:46.553329+0000","last_peered":"2024-08-30T08:58:46.553329+0000","last_clean":"2024-08-30T08:58:46.553329+0000","last_became_active":"2024-08-30T08:58:02.409027+0000","last_became_peered":"2024-08-30T08:58:02.409027+0000","last_unstale":"2024-08-30T08:58:46.553329+0000","last_undegraded":"2024-08-30T08:58:46.553329+0000","last_fullsized":"2024-08-30T08:58:46.553329+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T10:44:23.467130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"80'1","reported_seq":36,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:47.090595+0000","last_change":"2024-08-30T08:58:00.389083+0000","last_active":"2024-08-30T08:58:47.090595+0000","last_peered":"2024-08-30T08:58:47.090595+0000","last_clean":"2024-08-30T08:58:47.090595+0000","last_became_active":"2024-08-30T08:58:00.387552+0000","last_became_peered":"2024-08-30T08:58:00.387552+0000","last_unstale":"2024-08-30T08:58:47.090595+0000","last_undegraded":"2024-08-30T08:58:47.090595+0000","last_fullsized":"2024-08-30T08:58:47.090595+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T11:56:38.827406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.552061+0000","last_change":"2024-08-30T08:58:09.447758+0000","last_active":"2024-08-30T08:58:46.552061+0000","last_peered":"2024-08-30T08:58:46.552061+0000","last_clean":"2024-08-30T08:58:46.552061+0000","last_became_active":"2024-08-30T08:58:09.447333+0000","last_became_peered":"2024-08-30T08:58:09.447333+0000","last_unstale":"2024-08-30T08:58:46.552061+0000","last_undegraded":"2024-08-30T08:58:46.552061+0000","last_fullsized":"2024-08-30T08:58:46.552061+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T12:43:35.289967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:47.091640+0000","last_change":"2024-08-30T08:58:47.091191+0000","last_active":"2024-08-30T08:58:47.091640+0000","last_peered":"2024-08-30T08:58:47.091640+0000","last_clean":"2024-08-30T08:58:47.091640+0000","last_became_active":"2024-08-30T08:58:47.090826+0000","last_became_peered":"2024-08-30T08:58:47.090826+0000","last_unstale":"2024-08-30T08:58:47.091640+0000","last_undegraded":"2024-08-30T08:58:47.091640+0000","last_fullsized":"2024-08-30T08:58:47.091640+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T17:47:22.862514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"80'4","reported_seq":92,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:59:41.693522+0000","last_change":"2024-08-30T08:58:00.480651+0000","last_active":"2024-08-30T08:59:41.693522+0000","last_peered":"2024-08-30T08:59:41.693522+0000","last_clean":"2024-08-30T08:59:41.693522+0000","last_became_active":"2024-08-30T08:58:00.480205+0000","last_became_peered":"2024-08-30T08:58:00.480205+0000","last_unstale":"2024-08-30T08:59:41.693522+0000","last_undegraded":"2024-08-30T08:59:41.693522+0000","last_fullsized":"2024-08-30T08:59:41.693522+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T16:28:30.687076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":53,"num_read_kb":53,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.552066+0000","last_change":"2024-08-30T08:58:09.727530+0000","last_active":"2024-08-30T08:58:46.552066+0000","last_peered":"2024-08-30T08:58:46.552066+0000","last_clean":"2024-08-30T08:58:46.552066+0000","last_became_active":"2024-08-30T08:58:09.726965+0000","last_became_peered":"2024-08-30T08:58:09.726965+0000","last_unstale":"2024-08-30T08:58:46.552066+0000","last_undegraded":"2024-08-30T08:58:46.552066+0000","last_fullsized":"2024-08-30T08:58:46.552066+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T12:56:42.885615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.551578+0000","last_change":"2024-08-30T08:58:02.401673+0000","last_active":"2024-08-30T08:58:46.551578+0000","last_peered":"2024-08-30T08:58:46.551578+0000","last_clean":"2024-08-30T08:58:46.551578+0000","last_became_active":"2024-08-30T08:58:02.398876+0000","last_became_peered":"2024-08-30T08:58:02.398876+0000","last_unstale":"2024-08-30T08:58:46.551578+0000","last_undegraded":"2024-08-30T08:58:46.551578+0000","last_fullsized":"2024-08-30T08:58:46.551578+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T16:13:20.893737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"80'3","reported_seq":38,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:47.091220+0000","last_change":"2024-08-30T08:58:00.391722+0000","last_active":"2024-08-30T08:58:47.091220+0000","last_peered":"2024-08-30T08:58:47.091220+0000","last_clean":"2024-08-30T08:58:47.091220+0000","last_became_active":"2024-08-30T08:58:00.391325+0000","last_became_peered":"2024-08-30T08:58:00.391325+0000","last_unstale":"2024-08-30T08:58:47.091220+0000","last_undegraded":"2024-08-30T08:58:47.091220+0000","last_fullsized":"2024-08-30T08:58:47.091220+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T11:58:08.189202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.774243+0000","last_change":"2024-08-30T08:58:09.728324+0000","last_active":"2024-08-30T08:58:46.774243+0000","last_peered":"2024-08-30T08:58:46.774243+0000","last_clean":"2024-08-30T08:58:46.774243+0000","last_became_active":"2024-08-30T08:58:09.727520+0000","last_became_peered":"2024-08-30T08:58:09.727520+0000","last_unstale":"2024-08-30T08:58:46.774243+0000","last_undegraded":"2024-08-30T08:58:46.774243+0000","last_fullsized":"2024-08-30T08:58:46.774243+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T10:09:39.046820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.554063+0000","last_change":"2024-08-30T08:58:02.410826+0000","last_active":"2024-08-30T08:58:46.554063+0000","last_peered":"2024-08-30T08:58:46.554063+0000","last_clean":"2024-08-30T08:58:46.554063+0000","last_became_active":"2024-08-30T08:58:02.410464+0000","last_became_peered":"2024-08-30T08:58:02.410464+0000","last_unstale":"2024-08-30T08:58:46.554063+0000","last_undegraded":"2024-08-30T08:58:46.554063+0000","last_fullsized":"2024-08-30T08:58:46.554063+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T12:14:59.666614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"80'3","reported_seq":40,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:50.596022+0000","last_change":"2024-08-30T08:58:00.385464+0000","last_active":"2024-08-30T08:58:50.596022+0000","last_peered":"2024-08-30T08:58:50.596022+0000","last_clean":"2024-08-30T08:58:50.596022+0000","last_became_active":"2024-08-30T08:58:00.385083+0000","last_became_peered":"2024-08-30T08:58:00.385083+0000","last_unstale":"2024-08-30T08:58:50.596022+0000","last_undegraded":"2024-08-30T08:58:50.596022+0000","last_fullsized":"2024-08-30T08:58:50.596022+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T16:13:52.304755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":108,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:47.091487+0000","last_change":"2024-08-30T08:56:49.645080+0000","last_active":"2024-08-30T08:58:47.091487+0000","last_peered":"2024-08-30T08:58:47.091487+0000","last_clean":"2024-08-30T08:58:47.091487+0000","last_became_active":"2024-08-30T08:56:49.639066+0000","last_became_peered":"2024-08-30T08:56:49.639066+0000","last_unstale":"2024-08-30T08:58:47.091487+0000","last_undegraded":"2024-08-30T08:58:47.091487+0000","last_fullsized":"2024-08-30T08:58:47.091487+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:53:27.566846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:53:27.566846+0000","last_clean_scrub_stamp":"2024-08-30T08:53:27.566846+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T20:23:45.723824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"81'1","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:47.090916+0000","last_change":"2024-08-30T08:58:09.444007+0000","last_active":"2024-08-30T08:58:47.090916+0000","last_peered":"2024-08-30T08:58:47.090916+0000","last_clean":"2024-08-30T08:58:47.090916+0000","last_became_active":"2024-08-30T08:58:09.443580+0000","last_became_peered":"2024-08-30T08:58:09.443580+0000","last_unstale":"2024-08-30T08:58:47.090916+0000","last_undegraded":"2024-08-30T08:58:47.090916+0000","last_fullsized":"2024-08-30T08:58:47.090916+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T17:00:58.654026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:47.091022+0000","last_change":"2024-08-30T08:58:02.406973+0000","last_active":"2024-08-30T08:58:47.091022+0000","last_peered":"2024-08-30T08:58:47.091022+0000","last_clean":"2024-08-30T08:58:47.091022+0000","last_became_active":"2024-08-30T08:58:02.406576+0000","last_became_peered":"2024-08-30T08:58:02.406576+0000","last_unstale":"2024-08-30T08:58:47.091022+0000","last_undegraded":"2024-08-30T08:58:47.091022+0000","last_fullsized":"2024-08-30T08:58:47.091022+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T16:04:48.785310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"80'3","reported_seq":39,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.556017+0000","last_change":"2024-08-30T08:58:00.481019+0000","last_active":"2024-08-30T08:58:46.556017+0000","last_peered":"2024-08-30T08:58:46.556017+0000","last_clean":"2024-08-30T08:58:46.556017+0000","last_became_active":"2024-08-30T08:58:00.480590+0000","last_became_peered":"2024-08-30T08:58:00.480590+0000","last_unstale":"2024-08-30T08:58:46.556017+0000","last_undegraded":"2024-08-30T08:58:46.556017+0000","last_fullsized":"2024-08-30T08:58:46.556017+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T13:20:46.731591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.822612+0000","last_change":"2024-08-30T08:58:09.445723+0000","last_active":"2024-08-30T08:58:46.822612+0000","last_peered":"2024-08-30T08:58:46.822612+0000","last_clean":"2024-08-30T08:58:46.822612+0000","last_became_active":"2024-08-30T08:58:09.445252+0000","last_became_peered":"2024-08-30T08:58:09.445252+0000","last_unstale":"2024-08-30T08:58:46.822612+0000","last_undegraded":"2024-08-30T08:58:46.822612+0000","last_fullsized":"2024-08-30T08:58:46.822612+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T12:16:12.255990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:47.090625+0000","last_change":"2024-08-30T08:58:02.414250+0000","last_active":"2024-08-30T08:58:47.090625+0000","last_peered":"2024-08-30T08:58:47.090625+0000","last_clean":"2024-08-30T08:58:47.090625+0000","last_became_active":"2024-08-30T08:58:02.413684+0000","last_became_peered":"2024-08-30T08:58:02.413684+0000","last_unstale":"2024-08-30T08:58:47.090625+0000","last_undegraded":"2024-08-30T08:58:47.090625+0000","last_fullsized":"2024-08-30T08:58:47.090625+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T16:55:12.878806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"73'1","reported_seq":37,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.774624+0000","last_change":"2024-08-30T08:58:00.388869+0000","last_active":"2024-08-30T08:58:46.774624+0000","last_peered":"2024-08-30T08:58:46.774624+0000","last_clean":"2024-08-30T08:58:46.774624+0000","last_became_active":"2024-08-30T08:58:00.388116+0000","last_became_peered":"2024-08-30T08:58:00.388116+0000","last_unstale":"2024-08-30T08:58:46.774624+0000","last_undegraded":"2024-08-30T08:58:46.774624+0000","last_fullsized":"2024-08-30T08:58:46.774624+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T09:50:48.449205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:47.092091+0000","last_change":"2024-08-30T08:58:47.092004+0000","last_active":"2024-08-30T08:58:47.092091+0000","last_peered":"2024-08-30T08:58:47.092091+0000","last_clean":"2024-08-30T08:58:47.092091+0000","last_became_active":"2024-08-30T08:58:47.091661+0000","last_became_peered":"2024-08-30T08:58:47.091661+0000","last_unstale":"2024-08-30T08:58:47.092091+0000","last_undegraded":"2024-08-30T08:58:47.092091+0000","last_fullsized":"2024-08-30T08:58:47.092091+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T09:14:55.353465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,1,4],"acting":[0,8,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:47.090626+0000","last_change":"2024-08-30T08:58:02.410891+0000","last_active":"2024-08-30T08:58:47.090626+0000","last_peered":"2024-08-30T08:58:47.090626+0000","last_clean":"2024-08-30T08:58:47.090626+0000","last_became_active":"2024-08-30T08:58:02.410522+0000","last_became_peered":"2024-08-30T08:58:02.410522+0000","last_unstale":"2024-08-30T08:58:47.090626+0000","last_undegraded":"2024-08-30T08:58:47.090626+0000","last_fullsized":"2024-08-30T08:58:47.090626+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T09:32:21.540298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"73'2","reported_seq":48,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:47.095913+0000","last_change":"2024-08-30T08:58:47.095801+0000","last_active":"2024-08-30T08:58:47.095913+0000","last_peered":"2024-08-30T08:58:47.095913+0000","last_clean":"2024-08-30T08:58:47.095913+0000","last_became_active":"2024-08-30T08:58:47.092756+0000","last_became_peered":"2024-08-30T08:58:47.092756+0000","last_unstale":"2024-08-30T08:58:47.095913+0000","last_undegraded":"2024-08-30T08:58:47.095913+0000","last_fullsized":"2024-08-30T08:58:47.095913+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T19:10:02.064649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.773596+0000","last_change":"2024-08-30T08:58:09.449210+0000","last_active":"2024-08-30T08:58:46.773596+0000","last_peered":"2024-08-30T08:58:46.773596+0000","last_clean":"2024-08-30T08:58:46.773596+0000","last_became_active":"2024-08-30T08:58:09.448769+0000","last_became_peered":"2024-08-30T08:58:09.448769+0000","last_unstale":"2024-08-30T08:58:46.773596+0000","last_undegraded":"2024-08-30T08:58:46.773596+0000","last_fullsized":"2024-08-30T08:58:46.773596+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T09:26:13.845520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.553484+0000","last_change":"2024-08-30T08:58:02.414152+0000","last_active":"2024-08-30T08:58:46.553484+0000","last_peered":"2024-08-30T08:58:46.553484+0000","last_clean":"2024-08-30T08:58:46.553484+0000","last_became_active":"2024-08-30T08:58:02.413768+0000","last_became_peered":"2024-08-30T08:58:02.413768+0000","last_unstale":"2024-08-30T08:58:46.553484+0000","last_undegraded":"2024-08-30T08:58:46.553484+0000","last_fullsized":"2024-08-30T08:58:46.553484+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T18:34:09.447071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"81'2","reported_seq":38,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:50.596425+0000","last_change":"2024-08-30T08:58:00.390839+0000","last_active":"2024-08-30T08:58:50.596425+0000","last_peered":"2024-08-30T08:58:50.596425+0000","last_clean":"2024-08-30T08:58:50.596425+0000","last_became_active":"2024-08-30T08:58:00.390422+0000","last_became_peered":"2024-08-30T08:58:00.390422+0000","last_unstale":"2024-08-30T08:58:50.596425+0000","last_undegraded":"2024-08-30T08:58:50.596425+0000","last_fullsized":"2024-08-30T08:58:50.596425+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T15:05:23.005075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.824657+0000","last_change":"2024-08-30T08:58:46.824563+0000","last_active":"2024-08-30T08:58:46.824657+0000","last_peered":"2024-08-30T08:58:46.824657+0000","last_clean":"2024-08-30T08:58:46.824657+0000","last_became_active":"2024-08-30T08:58:46.824163+0000","last_became_peered":"2024-08-30T08:58:46.824163+0000","last_unstale":"2024-08-30T08:58:46.824657+0000","last_undegraded":"2024-08-30T08:58:46.824657+0000","last_fullsized":"2024-08-30T08:58:46.824657+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T19:25:13.202762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,3,1],"acting":[8,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.773839+0000","last_change":"2024-08-30T08:58:02.405122+0000","last_active":"2024-08-30T08:58:46.773839+0000","last_peered":"2024-08-30T08:58:46.773839+0000","last_clean":"2024-08-30T08:58:46.773839+0000","last_became_active":"2024-08-30T08:58:02.404695+0000","last_became_peered":"2024-08-30T08:58:02.404695+0000","last_unstale":"2024-08-30T08:58:46.773839+0000","last_undegraded":"2024-08-30T08:58:46.773839+0000","last_fullsized":"2024-08-30T08:58:46.773839+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T15:50:01.447032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"81'15","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:47.395221+0000","last_change":"2024-08-30T08:58:46.573221+0000","last_active":"2024-08-30T08:58:47.395221+0000","last_peered":"2024-08-30T08:58:47.395221+0000","last_clean":"2024-08-30T08:58:47.395221+0000","last_became_active":"2024-08-30T08:58:46.559562+0000","last_became_peered":"2024-08-30T08:58:46.559562+0000","last_unstale":"2024-08-30T08:58:47.395221+0000","last_undegraded":"2024-08-30T08:58:47.395221+0000","last_fullsized":"2024-08-30T08:58:47.395221+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T16:21:23.182676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.552177+0000","last_change":"2024-08-30T08:58:09.446063+0000","last_active":"2024-08-30T08:58:46.552177+0000","last_peered":"2024-08-30T08:58:46.552177+0000","last_clean":"2024-08-30T08:58:46.552177+0000","last_became_active":"2024-08-30T08:58:09.445615+0000","last_became_peered":"2024-08-30T08:58:09.445615+0000","last_unstale":"2024-08-30T08:58:46.552177+0000","last_undegraded":"2024-08-30T08:58:46.552177+0000","last_fullsized":"2024-08-30T08:58:46.552177+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T10:25:10.904892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:47.090684+0000","last_change":"2024-08-30T08:58:02.412178+0000","last_active":"2024-08-30T08:58:47.090684+0000","last_peered":"2024-08-30T08:58:47.090684+0000","last_clean":"2024-08-30T08:58:47.090684+0000","last_became_active":"2024-08-30T08:58:02.411814+0000","last_became_peered":"2024-08-30T08:58:02.411814+0000","last_unstale":"2024-08-30T08:58:47.090684+0000","last_undegraded":"2024-08-30T08:58:47.090684+0000","last_fullsized":"2024-08-30T08:58:47.090684+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T10:02:59.070204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"81'4","reported_seq":92,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:59:39.691500+0000","last_change":"2024-08-30T08:58:00.390231+0000","last_active":"2024-08-30T08:59:39.691500+0000","last_peered":"2024-08-30T08:59:39.691500+0000","last_clean":"2024-08-30T08:59:39.691500+0000","last_became_active":"2024-08-30T08:58:00.389852+0000","last_became_peered":"2024-08-30T08:58:00.389852+0000","last_unstale":"2024-08-30T08:59:39.691500+0000","last_undegraded":"2024-08-30T08:59:39.691500+0000","last_fullsized":"2024-08-30T08:59:39.691500+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T12:09:12.105416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":54,"num_read_kb":3,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.558813+0000","last_change":"2024-08-30T08:58:09.441652+0000","last_active":"2024-08-30T08:58:46.558813+0000","last_peered":"2024-08-30T08:58:46.558813+0000","last_clean":"2024-08-30T08:58:46.558813+0000","last_became_active":"2024-08-30T08:58:09.441264+0000","last_became_peered":"2024-08-30T08:58:09.441264+0000","last_unstale":"2024-08-30T08:58:46.558813+0000","last_undegraded":"2024-08-30T08:58:46.558813+0000","last_fullsized":"2024-08-30T08:58:46.558813+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T09:42:25.636669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:47.090297+0000","last_change":"2024-08-30T08:58:02.416347+0000","last_active":"2024-08-30T08:58:47.090297+0000","last_peered":"2024-08-30T08:58:47.090297+0000","last_clean":"2024-08-30T08:58:47.090297+0000","last_became_active":"2024-08-30T08:58:02.415898+0000","last_became_peered":"2024-08-30T08:58:02.415898+0000","last_unstale":"2024-08-30T08:58:47.090297+0000","last_undegraded":"2024-08-30T08:58:47.090297+0000","last_fullsized":"2024-08-30T08:58:47.090297+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T10:18:15.507076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"79'3","reported_seq":38,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.774168+0000","last_change":"2024-08-30T08:58:00.393616+0000","last_active":"2024-08-30T08:58:46.774168+0000","last_peered":"2024-08-30T08:58:46.774168+0000","last_clean":"2024-08-30T08:58:46.774168+0000","last_became_active":"2024-08-30T08:58:00.393218+0000","last_became_peered":"2024-08-30T08:58:00.393218+0000","last_unstale":"2024-08-30T08:58:46.774168+0000","last_undegraded":"2024-08-30T08:58:46.774168+0000","last_fullsized":"2024-08-30T08:58:46.774168+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T10:56:37.175968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:47.093131+0000","last_change":"2024-08-30T08:58:47.092990+0000","last_active":"2024-08-30T08:58:47.093131+0000","last_peered":"2024-08-30T08:58:47.093131+0000","last_clean":"2024-08-30T08:58:47.093131+0000","last_became_active":"2024-08-30T08:58:47.092448+0000","last_became_peered":"2024-08-30T08:58:47.092448+0000","last_unstale":"2024-08-30T08:58:47.093131+0000","last_undegraded":"2024-08-30T08:58:47.093131+0000","last_fullsized":"2024-08-30T08:58:47.093131+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T20:23:06.527336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,7,6],"acting":[11,10,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.551703+0000","last_change":"2024-08-30T08:58:02.405783+0000","last_active":"2024-08-30T08:58:46.551703+0000","last_peered":"2024-08-30T08:58:46.551703+0000","last_clean":"2024-08-30T08:58:46.551703+0000","last_became_active":"2024-08-30T08:58:02.405430+0000","last_became_peered":"2024-08-30T08:58:02.405430+0000","last_unstale":"2024-08-30T08:58:46.551703+0000","last_undegraded":"2024-08-30T08:58:46.551703+0000","last_fullsized":"2024-08-30T08:58:46.551703+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T19:40:08.377259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"80'6","reported_seq":42,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.555590+0000","last_change":"2024-08-30T08:58:00.389617+0000","last_active":"2024-08-30T08:58:46.555590+0000","last_peered":"2024-08-30T08:58:46.555590+0000","last_clean":"2024-08-30T08:58:46.555590+0000","last_became_active":"2024-08-30T08:58:00.389031+0000","last_became_peered":"2024-08-30T08:58:00.389031+0000","last_unstale":"2024-08-30T08:58:46.555590+0000","last_undegraded":"2024-08-30T08:58:46.555590+0000","last_fullsized":"2024-08-30T08:58:46.555590+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T15:42:38.050768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.774094+0000","last_change":"2024-08-30T08:58:09.451925+0000","last_active":"2024-08-30T08:58:46.774094+0000","last_peered":"2024-08-30T08:58:46.774094+0000","last_clean":"2024-08-30T08:58:46.774094+0000","last_became_active":"2024-08-30T08:58:09.446893+0000","last_became_peered":"2024-08-30T08:58:09.446893+0000","last_unstale":"2024-08-30T08:58:46.774094+0000","last_undegraded":"2024-08-30T08:58:46.774094+0000","last_fullsized":"2024-08-30T08:58:46.774094+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T14:34:25.134006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.557378+0000","last_change":"2024-08-30T08:58:02.411194+0000","last_active":"2024-08-30T08:58:46.557378+0000","last_peered":"2024-08-30T08:58:46.557378+0000","last_clean":"2024-08-30T08:58:46.557378+0000","last_became_active":"2024-08-30T08:58:02.410183+0000","last_became_peered":"2024-08-30T08:58:02.410183+0000","last_unstale":"2024-08-30T08:58:46.557378+0000","last_undegraded":"2024-08-30T08:58:46.557378+0000","last_fullsized":"2024-08-30T08:58:46.557378+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T10:56:19.016115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"81'34","reported_seq":182,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:59:41.687527+0000","last_change":"2024-08-30T08:58:00.389808+0000","last_active":"2024-08-30T08:59:41.687527+0000","last_peered":"2024-08-30T08:59:41.687527+0000","last_clean":"2024-08-30T08:59:41.687527+0000","last_became_active":"2024-08-30T08:58:00.389411+0000","last_became_peered":"2024-08-30T08:58:00.389411+0000","last_unstale":"2024-08-30T08:59:41.687527+0000","last_undegraded":"2024-08-30T08:59:41.687527+0000","last_fullsized":"2024-08-30T08:59:41.687527+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T18:59:45.468456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64060,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":113,"num_read_kb":119,"num_write":35,"num_write_kb":78,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.557593+0000","last_change":"2024-08-30T08:58:09.447360+0000","last_active":"2024-08-30T08:58:46.557593+0000","last_peered":"2024-08-30T08:58:46.557593+0000","last_clean":"2024-08-30T08:58:46.557593+0000","last_became_active":"2024-08-30T08:58:09.446913+0000","last_became_peered":"2024-08-30T08:58:09.446913+0000","last_unstale":"2024-08-30T08:58:46.557593+0000","last_undegraded":"2024-08-30T08:58:46.557593+0000","last_fullsized":"2024-08-30T08:58:46.557593+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T13:59:43.147370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.822816+0000","last_change":"2024-08-30T08:58:02.408920+0000","last_active":"2024-08-30T08:58:46.822816+0000","last_peered":"2024-08-30T08:58:46.822816+0000","last_clean":"2024-08-30T08:58:46.822816+0000","last_became_active":"2024-08-30T08:58:02.408204+0000","last_became_peered":"2024-08-30T08:58:02.408204+0000","last_unstale":"2024-08-30T08:58:46.822816+0000","last_undegraded":"2024-08-30T08:58:46.822816+0000","last_fullsized":"2024-08-30T08:58:46.822816+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T11:44:00.765672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"79'4","reported_seq":40,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.774530+0000","last_change":"2024-08-30T08:58:00.482348+0000","last_active":"2024-08-30T08:58:46.774530+0000","last_peered":"2024-08-30T08:58:46.774530+0000","last_clean":"2024-08-30T08:58:46.774530+0000","last_became_active":"2024-08-30T08:58:00.481929+0000","last_became_peered":"2024-08-30T08:58:00.481929+0000","last_unstale":"2024-08-30T08:58:46.774530+0000","last_undegraded":"2024-08-30T08:58:46.774530+0000","last_fullsized":"2024-08-30T08:58:46.774530+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T11:13:55.332639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:47.089847+0000","last_change":"2024-08-30T08:58:09.443259+0000","last_active":"2024-08-30T08:58:47.089847+0000","last_peered":"2024-08-30T08:58:47.089847+0000","last_clean":"2024-08-30T08:58:47.089847+0000","last_became_active":"2024-08-30T08:58:09.442209+0000","last_became_peered":"2024-08-30T08:58:09.442209+0000","last_unstale":"2024-08-30T08:58:47.089847+0000","last_undegraded":"2024-08-30T08:58:47.089847+0000","last_fullsized":"2024-08-30T08:58:47.089847+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T09:25:56.233525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.557115+0000","last_change":"2024-08-30T08:58:02.407183+0000","last_active":"2024-08-30T08:58:46.557115+0000","last_peered":"2024-08-30T08:58:46.557115+0000","last_clean":"2024-08-30T08:58:46.557115+0000","last_became_active":"2024-08-30T08:58:02.406838+0000","last_became_peered":"2024-08-30T08:58:02.406838+0000","last_unstale":"2024-08-30T08:58:46.557115+0000","last_undegraded":"2024-08-30T08:58:46.557115+0000","last_fullsized":"2024-08-30T08:58:46.557115+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T13:40:32.937626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"79'2","reported_seq":37,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:47.090297+0000","last_change":"2024-08-30T08:58:00.392322+0000","last_active":"2024-08-30T08:58:47.090297+0000","last_peered":"2024-08-30T08:58:47.090297+0000","last_clean":"2024-08-30T08:58:47.090297+0000","last_became_active":"2024-08-30T08:58:00.391935+0000","last_became_peered":"2024-08-30T08:58:00.391935+0000","last_unstale":"2024-08-30T08:58:47.090297+0000","last_undegraded":"2024-08-30T08:58:47.090297+0000","last_fullsized":"2024-08-30T08:58:47.090297+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T12:20:52.368285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.557659+0000","last_change":"2024-08-30T08:58:09.441824+0000","last_active":"2024-08-30T08:58:46.557659+0000","last_peered":"2024-08-30T08:58:46.557659+0000","last_clean":"2024-08-30T08:58:46.557659+0000","last_became_active":"2024-08-30T08:58:09.441398+0000","last_became_peered":"2024-08-30T08:58:09.441398+0000","last_unstale":"2024-08-30T08:58:46.557659+0000","last_undegraded":"2024-08-30T08:58:46.557659+0000","last_fullsized":"2024-08-30T08:58:46.557659+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T14:05:45.710280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.823379+0000","last_change":"2024-08-30T08:58:02.403431+0000","last_active":"2024-08-30T08:58:46.823379+0000","last_peered":"2024-08-30T08:58:46.823379+0000","last_clean":"2024-08-30T08:58:46.823379+0000","last_became_active":"2024-08-30T08:58:02.403029+0000","last_became_peered":"2024-08-30T08:58:02.403029+0000","last_unstale":"2024-08-30T08:58:46.823379+0000","last_undegraded":"2024-08-30T08:58:46.823379+0000","last_fullsized":"2024-08-30T08:58:46.823379+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T09:34:56.452675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"79'3","reported_seq":90,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:59:40.685815+0000","last_change":"2024-08-30T08:58:00.389856+0000","last_active":"2024-08-30T08:59:40.685815+0000","last_peered":"2024-08-30T08:59:40.685815+0000","last_clean":"2024-08-30T08:59:40.685815+0000","last_became_active":"2024-08-30T08:58:00.389459+0000","last_became_peered":"2024-08-30T08:58:00.389459+0000","last_unstale":"2024-08-30T08:59:40.685815+0000","last_undegraded":"2024-08-30T08:59:40.685815+0000","last_fullsized":"2024-08-30T08:59:40.685815+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T18:32:06.214570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":52,"num_read_kb":52,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.553852+0000","last_change":"2024-08-30T08:58:09.445317+0000","last_active":"2024-08-30T08:58:46.553852+0000","last_peered":"2024-08-30T08:58:46.553852+0000","last_clean":"2024-08-30T08:58:46.553852+0000","last_became_active":"2024-08-30T08:58:09.444921+0000","last_became_peered":"2024-08-30T08:58:09.444921+0000","last_unstale":"2024-08-30T08:58:46.553852+0000","last_undegraded":"2024-08-30T08:58:46.553852+0000","last_fullsized":"2024-08-30T08:58:46.553852+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T09:22:13.038437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.556314+0000","last_change":"2024-08-30T08:58:09.447469+0000","last_active":"2024-08-30T08:58:46.556314+0000","last_peered":"2024-08-30T08:58:46.556314+0000","last_clean":"2024-08-30T08:58:46.556314+0000","last_became_active":"2024-08-30T08:58:09.447015+0000","last_became_peered":"2024-08-30T08:58:09.447015+0000","last_unstale":"2024-08-30T08:58:46.556314+0000","last_undegraded":"2024-08-30T08:58:46.556314+0000","last_fullsized":"2024-08-30T08:58:46.556314+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T09:18:36.734380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.557549+0000","last_change":"2024-08-30T08:58:02.403371+0000","last_active":"2024-08-30T08:58:46.557549+0000","last_peered":"2024-08-30T08:58:46.557549+0000","last_clean":"2024-08-30T08:58:46.557549+0000","last_became_active":"2024-08-30T08:58:02.403012+0000","last_became_peered":"2024-08-30T08:58:02.403012+0000","last_unstale":"2024-08-30T08:58:46.557549+0000","last_undegraded":"2024-08-30T08:58:46.557549+0000","last_fullsized":"2024-08-30T08:58:46.557549+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T10:29:29.683081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"79'1","reported_seq":36,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.774090+0000","last_change":"2024-08-30T08:58:00.394450+0000","last_active":"2024-08-30T08:58:46.774090+0000","last_peered":"2024-08-30T08:58:46.774090+0000","last_clean":"2024-08-30T08:58:46.774090+0000","last_became_active":"2024-08-30T08:58:00.391843+0000","last_became_peered":"2024-08-30T08:58:00.391843+0000","last_unstale":"2024-08-30T08:58:46.774090+0000","last_undegraded":"2024-08-30T08:58:46.774090+0000","last_fullsized":"2024-08-30T08:58:46.774090+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T10:29:18.947898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.553483+0000","last_change":"2024-08-30T08:58:09.449172+0000","last_active":"2024-08-30T08:58:46.553483+0000","last_peered":"2024-08-30T08:58:46.553483+0000","last_clean":"2024-08-30T08:58:46.553483+0000","last_became_active":"2024-08-30T08:58:09.448687+0000","last_became_peered":"2024-08-30T08:58:09.448687+0000","last_unstale":"2024-08-30T08:58:46.553483+0000","last_undegraded":"2024-08-30T08:58:46.553483+0000","last_fullsized":"2024-08-30T08:58:46.553483+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T13:23:16.090251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.553825+0000","last_change":"2024-08-30T08:58:02.400834+0000","last_active":"2024-08-30T08:58:46.553825+0000","last_peered":"2024-08-30T08:58:46.553825+0000","last_clean":"2024-08-30T08:58:46.553825+0000","last_became_active":"2024-08-30T08:58:02.400477+0000","last_became_peered":"2024-08-30T08:58:02.400477+0000","last_unstale":"2024-08-30T08:58:46.553825+0000","last_undegraded":"2024-08-30T08:58:46.553825+0000","last_fullsized":"2024-08-30T08:58:46.553825+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T13:40:45.239732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"80'2","reported_seq":37,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.551581+0000","last_change":"2024-08-30T08:58:00.381699+0000","last_active":"2024-08-30T08:58:46.551581+0000","last_peered":"2024-08-30T08:58:46.551581+0000","last_clean":"2024-08-30T08:58:46.551581+0000","last_became_active":"2024-08-30T08:58:00.381189+0000","last_became_peered":"2024-08-30T08:58:00.381189+0000","last_unstale":"2024-08-30T08:58:46.551581+0000","last_undegraded":"2024-08-30T08:58:46.551581+0000","last_fullsized":"2024-08-30T08:58:46.551581+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T14:09:16.939062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"80'3","reported_seq":38,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.553179+0000","last_change":"2024-08-30T08:58:00.387430+0000","last_active":"2024-08-30T08:58:46.553179+0000","last_peered":"2024-08-30T08:58:46.553179+0000","last_clean":"2024-08-30T08:58:46.553179+0000","last_became_active":"2024-08-30T08:58:00.387043+0000","last_became_peered":"2024-08-30T08:58:00.387043+0000","last_unstale":"2024-08-30T08:58:46.553179+0000","last_undegraded":"2024-08-30T08:58:46.553179+0000","last_fullsized":"2024-08-30T08:58:46.553179+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T13:32:25.418105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.774175+0000","last_change":"2024-08-30T08:58:02.404798+0000","last_active":"2024-08-30T08:58:46.774175+0000","last_peered":"2024-08-30T08:58:46.774175+0000","last_clean":"2024-08-30T08:58:46.774175+0000","last_became_active":"2024-08-30T08:58:02.404435+0000","last_became_peered":"2024-08-30T08:58:02.404435+0000","last_unstale":"2024-08-30T08:58:46.774175+0000","last_undegraded":"2024-08-30T08:58:46.774175+0000","last_fullsized":"2024-08-30T08:58:46.774175+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T20:31:49.472832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.555057+0000","last_change":"2024-08-30T08:58:09.448297+0000","last_active":"2024-08-30T08:58:46.555057+0000","last_peered":"2024-08-30T08:58:46.555057+0000","last_clean":"2024-08-30T08:58:46.555057+0000","last_became_active":"2024-08-30T08:58:09.447444+0000","last_became_peered":"2024-08-30T08:58:09.447444+0000","last_unstale":"2024-08-30T08:58:46.555057+0000","last_undegraded":"2024-08-30T08:58:46.555057+0000","last_fullsized":"2024-08-30T08:58:46.555057+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T11:40:03.507824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"81'6","reported_seq":95,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:59:40.686450+0000","last_change":"2024-08-30T08:58:00.392418+0000","last_active":"2024-08-30T08:59:40.686450+0000","last_peered":"2024-08-30T08:59:40.686450+0000","last_clean":"2024-08-30T08:59:40.686450+0000","last_became_active":"2024-08-30T08:58:00.391985+0000","last_became_peered":"2024-08-30T08:58:00.391985+0000","last_unstale":"2024-08-30T08:59:40.686450+0000","last_undegraded":"2024-08-30T08:59:40.686450+0000","last_fullsized":"2024-08-30T08:59:40.686450+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T17:42:18.147754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":54,"num_read_kb":2,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.557148+0000","last_change":"2024-08-30T08:58:02.408754+0000","last_active":"2024-08-30T08:58:46.557148+0000","last_peered":"2024-08-30T08:58:46.557148+0000","last_clean":"2024-08-30T08:58:46.557148+0000","last_became_active":"2024-08-30T08:58:02.407127+0000","last_became_peered":"2024-08-30T08:58:02.407127+0000","last_unstale":"2024-08-30T08:58:46.557148+0000","last_undegraded":"2024-08-30T08:58:46.557148+0000","last_fullsized":"2024-08-30T08:58:46.557148+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T16:15:41.796091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.555654+0000","last_change":"2024-08-30T08:58:09.445389+0000","last_active":"2024-08-30T08:58:46.555654+0000","last_peered":"2024-08-30T08:58:46.555654+0000","last_clean":"2024-08-30T08:58:46.555654+0000","last_became_active":"2024-08-30T08:58:09.444831+0000","last_became_peered":"2024-08-30T08:58:09.444831+0000","last_unstale":"2024-08-30T08:58:46.555654+0000","last_undegraded":"2024-08-30T08:58:46.555654+0000","last_fullsized":"2024-08-30T08:58:46.555654+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T09:55:58.162397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.553020+0000","last_change":"2024-08-30T08:58:02.415184+0000","last_active":"2024-08-30T08:58:46.553020+0000","last_peered":"2024-08-30T08:58:46.553020+0000","last_clean":"2024-08-30T08:58:46.553020+0000","last_became_active":"2024-08-30T08:58:02.414625+0000","last_became_peered":"2024-08-30T08:58:02.414625+0000","last_unstale":"2024-08-30T08:58:46.553020+0000","last_undegraded":"2024-08-30T08:58:46.553020+0000","last_fullsized":"2024-08-30T08:58:46.553020+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T11:10:31.130524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.555567+0000","last_change":"2024-08-30T08:58:02.405392+0000","last_active":"2024-08-30T08:58:46.555567+0000","last_peered":"2024-08-30T08:58:46.555567+0000","last_clean":"2024-08-30T08:58:46.555567+0000","last_became_active":"2024-08-30T08:58:02.405009+0000","last_became_peered":"2024-08-30T08:58:02.405009+0000","last_unstale":"2024-08-30T08:58:46.555567+0000","last_undegraded":"2024-08-30T08:58:46.555567+0000","last_fullsized":"2024-08-30T08:58:46.555567+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T13:22:46.375598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.555680+0000","last_change":"2024-08-30T08:58:02.402360+0000","last_active":"2024-08-30T08:58:46.555680+0000","last_peered":"2024-08-30T08:58:46.555680+0000","last_clean":"2024-08-30T08:58:46.555680+0000","last_became_active":"2024-08-30T08:58:02.401856+0000","last_became_peered":"2024-08-30T08:58:02.401856+0000","last_unstale":"2024-08-30T08:58:46.555680+0000","last_undegraded":"2024-08-30T08:58:46.555680+0000","last_fullsized":"2024-08-30T08:58:46.555680+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T10:25:14.491345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.557920+0000","last_change":"2024-08-30T08:58:02.415940+0000","last_active":"2024-08-30T08:58:46.557920+0000","last_peered":"2024-08-30T08:58:46.557920+0000","last_clean":"2024-08-30T08:58:46.557920+0000","last_became_active":"2024-08-30T08:58:02.415567+0000","last_became_peered":"2024-08-30T08:58:02.415567+0000","last_unstale":"2024-08-30T08:58:46.557920+0000","last_undegraded":"2024-08-30T08:58:46.557920+0000","last_fullsized":"2024-08-30T08:58:46.557920+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T15:02:26.270343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.552160+0000","last_change":"2024-08-30T08:58:02.406766+0000","last_active":"2024-08-30T08:58:46.552160+0000","last_peered":"2024-08-30T08:58:46.552160+0000","last_clean":"2024-08-30T08:58:46.552160+0000","last_became_active":"2024-08-30T08:58:02.406345+0000","last_became_peered":"2024-08-30T08:58:02.406345+0000","last_unstale":"2024-08-30T08:58:46.552160+0000","last_undegraded":"2024-08-30T08:58:46.552160+0000","last_fullsized":"2024-08-30T08:58:46.552160+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T13:10:32.986817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.553714+0000","last_change":"2024-08-30T08:58:02.412891+0000","last_active":"2024-08-30T08:58:46.553714+0000","last_peered":"2024-08-30T08:58:46.553714+0000","last_clean":"2024-08-30T08:58:46.553714+0000","last_became_active":"2024-08-30T08:58:02.412532+0000","last_became_peered":"2024-08-30T08:58:02.412532+0000","last_unstale":"2024-08-30T08:58:46.553714+0000","last_undegraded":"2024-08-30T08:58:46.553714+0000","last_fullsized":"2024-08-30T08:58:46.553714+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T14:40:30.663678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.556273+0000","last_change":"2024-08-30T08:58:02.405862+0000","last_active":"2024-08-30T08:58:46.556273+0000","last_peered":"2024-08-30T08:58:46.556273+0000","last_clean":"2024-08-30T08:58:46.556273+0000","last_became_active":"2024-08-30T08:58:02.405319+0000","last_became_peered":"2024-08-30T08:58:02.405319+0000","last_unstale":"2024-08-30T08:58:46.556273+0000","last_undegraded":"2024-08-30T08:58:46.556273+0000","last_fullsized":"2024-08-30T08:58:46.556273+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T20:36:10.315814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.555535+0000","last_change":"2024-08-30T08:58:02.401410+0000","last_active":"2024-08-30T08:58:46.555535+0000","last_peered":"2024-08-30T08:58:46.555535+0000","last_clean":"2024-08-30T08:58:46.555535+0000","last_became_active":"2024-08-30T08:58:02.401060+0000","last_became_peered":"2024-08-30T08:58:02.401060+0000","last_unstale":"2024-08-30T08:58:46.555535+0000","last_undegraded":"2024-08-30T08:58:46.555535+0000","last_fullsized":"2024-08-30T08:58:46.555535+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T16:00:41.275367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.773667+0000","last_change":"2024-08-30T08:58:02.401703+0000","last_active":"2024-08-30T08:58:46.773667+0000","last_peered":"2024-08-30T08:58:46.773667+0000","last_clean":"2024-08-30T08:58:46.773667+0000","last_became_active":"2024-08-30T08:58:02.401218+0000","last_became_peered":"2024-08-30T08:58:02.401218+0000","last_unstale":"2024-08-30T08:58:46.773667+0000","last_undegraded":"2024-08-30T08:58:46.773667+0000","last_fullsized":"2024-08-30T08:58:46.773667+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T18:36:25.249814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.554007+0000","last_change":"2024-08-30T08:58:02.403609+0000","last_active":"2024-08-30T08:58:46.554007+0000","last_peered":"2024-08-30T08:58:46.554007+0000","last_clean":"2024-08-30T08:58:46.554007+0000","last_became_active":"2024-08-30T08:58:02.402195+0000","last_became_peered":"2024-08-30T08:58:02.402195+0000","last_unstale":"2024-08-30T08:58:46.554007+0000","last_undegraded":"2024-08-30T08:58:46.554007+0000","last_fullsized":"2024-08-30T08:58:46.554007+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T09:59:05.877523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.823042+0000","last_change":"2024-08-30T08:58:02.404389+0000","last_active":"2024-08-30T08:58:46.823042+0000","last_peered":"2024-08-30T08:58:46.823042+0000","last_clean":"2024-08-30T08:58:46.823042+0000","last_became_active":"2024-08-30T08:58:02.404039+0000","last_became_peered":"2024-08-30T08:58:02.404039+0000","last_unstale":"2024-08-30T08:58:46.823042+0000","last_undegraded":"2024-08-30T08:58:46.823042+0000","last_fullsized":"2024-08-30T08:58:46.823042+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T20:55:51.739542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.552060+0000","last_change":"2024-08-30T08:58:02.411807+0000","last_active":"2024-08-30T08:58:46.552060+0000","last_peered":"2024-08-30T08:58:46.552060+0000","last_clean":"2024-08-30T08:58:46.552060+0000","last_became_active":"2024-08-30T08:58:02.410977+0000","last_became_peered":"2024-08-30T08:58:02.410977+0000","last_unstale":"2024-08-30T08:58:46.552060+0000","last_undegraded":"2024-08-30T08:58:46.552060+0000","last_fullsized":"2024-08-30T08:58:46.552060+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T13:04:29.931373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":37,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.560319+0000","last_change":"2024-08-30T08:58:46.559661+0000","last_active":"2024-08-30T08:58:46.560319+0000","last_peered":"2024-08-30T08:58:46.560319+0000","last_clean":"2024-08-30T08:58:46.560319+0000","last_became_active":"2024-08-30T08:58:46.558824+0000","last_became_peered":"2024-08-30T08:58:46.558824+0000","last_unstale":"2024-08-30T08:58:46.560319+0000","last_undegraded":"2024-08-30T08:58:46.560319+0000","last_fullsized":"2024-08-30T08:58:46.560319+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T09:53:07.650921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.556117+0000","last_change":"2024-08-30T08:58:02.396186+0000","last_active":"2024-08-30T08:58:46.556117+0000","last_peered":"2024-08-30T08:58:46.556117+0000","last_clean":"2024-08-30T08:58:46.556117+0000","last_became_active":"2024-08-30T08:58:02.395776+0000","last_became_peered":"2024-08-30T08:58:02.395776+0000","last_unstale":"2024-08-30T08:58:46.556117+0000","last_undegraded":"2024-08-30T08:58:46.556117+0000","last_fullsized":"2024-08-30T08:58:46.556117+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T09:33:30.068673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.559487+0000","last_change":"2024-08-30T08:58:02.408378+0000","last_active":"2024-08-30T08:58:46.559487+0000","last_peered":"2024-08-30T08:58:46.559487+0000","last_clean":"2024-08-30T08:58:46.559487+0000","last_became_active":"2024-08-30T08:58:02.407991+0000","last_became_peered":"2024-08-30T08:58:02.407991+0000","last_unstale":"2024-08-30T08:58:46.559487+0000","last_undegraded":"2024-08-30T08:58:46.559487+0000","last_fullsized":"2024-08-30T08:58:46.559487+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T20:57:02.411176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.553823+0000","last_change":"2024-08-30T08:58:02.411829+0000","last_active":"2024-08-30T08:58:46.553823+0000","last_peered":"2024-08-30T08:58:46.553823+0000","last_clean":"2024-08-30T08:58:46.553823+0000","last_became_active":"2024-08-30T08:58:02.411416+0000","last_became_peered":"2024-08-30T08:58:02.411416+0000","last_unstale":"2024-08-30T08:58:46.553823+0000","last_undegraded":"2024-08-30T08:58:46.553823+0000","last_fullsized":"2024-08-30T08:58:46.553823+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T17:12:08.855150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.822489+0000","last_change":"2024-08-30T08:58:02.405122+0000","last_active":"2024-08-30T08:58:46.822489+0000","last_peered":"2024-08-30T08:58:46.822489+0000","last_clean":"2024-08-30T08:58:46.822489+0000","last_became_active":"2024-08-30T08:58:02.403312+0000","last_became_peered":"2024-08-30T08:58:02.403312+0000","last_unstale":"2024-08-30T08:58:46.822489+0000","last_undegraded":"2024-08-30T08:58:46.822489+0000","last_fullsized":"2024-08-30T08:58:46.822489+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T18:11:53.077433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.556239+0000","last_change":"2024-08-30T08:58:02.404263+0000","last_active":"2024-08-30T08:58:46.556239+0000","last_peered":"2024-08-30T08:58:46.556239+0000","last_clean":"2024-08-30T08:58:46.556239+0000","last_became_active":"2024-08-30T08:58:02.403897+0000","last_became_peered":"2024-08-30T08:58:02.403897+0000","last_unstale":"2024-08-30T08:58:46.556239+0000","last_undegraded":"2024-08-30T08:58:46.556239+0000","last_fullsized":"2024-08-30T08:58:46.556239+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T18:38:31.439434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.556196+0000","last_change":"2024-08-30T08:58:02.411578+0000","last_active":"2024-08-30T08:58:46.556196+0000","last_peered":"2024-08-30T08:58:46.556196+0000","last_clean":"2024-08-30T08:58:46.556196+0000","last_became_active":"2024-08-30T08:58:02.411271+0000","last_became_peered":"2024-08-30T08:58:02.411271+0000","last_unstale":"2024-08-30T08:58:46.556196+0000","last_undegraded":"2024-08-30T08:58:46.556196+0000","last_fullsized":"2024-08-30T08:58:46.556196+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T10:02:46.382906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.552115+0000","last_change":"2024-08-30T08:58:02.412168+0000","last_active":"2024-08-30T08:58:46.552115+0000","last_peered":"2024-08-30T08:58:46.552115+0000","last_clean":"2024-08-30T08:58:46.552115+0000","last_became_active":"2024-08-30T08:58:02.411420+0000","last_became_peered":"2024-08-30T08:58:02.411420+0000","last_unstale":"2024-08-30T08:58:46.552115+0000","last_undegraded":"2024-08-30T08:58:46.552115+0000","last_fullsized":"2024-08-30T08:58:46.552115+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T10:35:53.654344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:47.089927+0000","last_change":"2024-08-30T08:58:02.409735+0000","last_active":"2024-08-30T08:58:47.089927+0000","last_peered":"2024-08-30T08:58:47.089927+0000","last_clean":"2024-08-30T08:58:47.089927+0000","last_became_active":"2024-08-30T08:58:02.409349+0000","last_became_peered":"2024-08-30T08:58:02.409349+0000","last_unstale":"2024-08-30T08:58:47.089927+0000","last_undegraded":"2024-08-30T08:58:47.089927+0000","last_fullsized":"2024-08-30T08:58:47.089927+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T20:10:22.956554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:47.090762+0000","last_change":"2024-08-30T08:58:02.416261+0000","last_active":"2024-08-30T08:58:47.090762+0000","last_peered":"2024-08-30T08:58:47.090762+0000","last_clean":"2024-08-30T08:58:47.090762+0000","last_became_active":"2024-08-30T08:58:02.415194+0000","last_became_peered":"2024-08-30T08:58:02.415194+0000","last_unstale":"2024-08-30T08:58:47.090762+0000","last_undegraded":"2024-08-30T08:58:47.090762+0000","last_fullsized":"2024-08-30T08:58:47.090762+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T17:59:04.339699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":37,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.560326+0000","last_change":"2024-08-30T08:58:46.560190+0000","last_active":"2024-08-30T08:58:46.560326+0000","last_peered":"2024-08-30T08:58:46.560326+0000","last_clean":"2024-08-30T08:58:46.560326+0000","last_became_active":"2024-08-30T08:58:46.559674+0000","last_became_peered":"2024-08-30T08:58:46.559674+0000","last_unstale":"2024-08-30T08:58:46.560326+0000","last_undegraded":"2024-08-30T08:58:46.560326+0000","last_fullsized":"2024-08-30T08:58:46.560326+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T10:08:02.835142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.823261+0000","last_change":"2024-08-30T08:58:02.406763+0000","last_active":"2024-08-30T08:58:46.823261+0000","last_peered":"2024-08-30T08:58:46.823261+0000","last_clean":"2024-08-30T08:58:46.823261+0000","last_became_active":"2024-08-30T08:58:02.406377+0000","last_became_peered":"2024-08-30T08:58:02.406377+0000","last_unstale":"2024-08-30T08:58:46.823261+0000","last_undegraded":"2024-08-30T08:58:46.823261+0000","last_fullsized":"2024-08-30T08:58:46.823261+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T13:32:50.492921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.553816+0000","last_change":"2024-08-30T08:58:02.416638+0000","last_active":"2024-08-30T08:58:46.553816+0000","last_peered":"2024-08-30T08:58:46.553816+0000","last_clean":"2024-08-30T08:58:46.553816+0000","last_became_active":"2024-08-30T08:58:02.415518+0000","last_became_peered":"2024-08-30T08:58:02.415518+0000","last_unstale":"2024-08-30T08:58:46.553816+0000","last_undegraded":"2024-08-30T08:58:46.553816+0000","last_fullsized":"2024-08-30T08:58:46.553816+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T17:05:07.531585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:47.089992+0000","last_change":"2024-08-30T08:58:02.412973+0000","last_active":"2024-08-30T08:58:47.089992+0000","last_peered":"2024-08-30T08:58:47.089992+0000","last_clean":"2024-08-30T08:58:47.089992+0000","last_became_active":"2024-08-30T08:58:02.412609+0000","last_became_peered":"2024-08-30T08:58:02.412609+0000","last_unstale":"2024-08-30T08:58:47.089992+0000","last_undegraded":"2024-08-30T08:58:47.089992+0000","last_fullsized":"2024-08-30T08:58:47.089992+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T14:22:37.704098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.553161+0000","last_change":"2024-08-30T08:58:02.402779+0000","last_active":"2024-08-30T08:58:46.553161+0000","last_peered":"2024-08-30T08:58:46.553161+0000","last_clean":"2024-08-30T08:58:46.553161+0000","last_became_active":"2024-08-30T08:58:02.401514+0000","last_became_peered":"2024-08-30T08:58:02.401514+0000","last_unstale":"2024-08-30T08:58:46.553161+0000","last_undegraded":"2024-08-30T08:58:46.553161+0000","last_fullsized":"2024-08-30T08:58:46.553161+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T11:37:51.450334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.557162+0000","last_change":"2024-08-30T08:58:02.406607+0000","last_active":"2024-08-30T08:58:46.557162+0000","last_peered":"2024-08-30T08:58:46.557162+0000","last_clean":"2024-08-30T08:58:46.557162+0000","last_became_active":"2024-08-30T08:58:02.406237+0000","last_became_peered":"2024-08-30T08:58:02.406237+0000","last_unstale":"2024-08-30T08:58:46.557162+0000","last_undegraded":"2024-08-30T08:58:46.557162+0000","last_fullsized":"2024-08-30T08:58:46.557162+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T12:09:03.247424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:47.089986+0000","last_change":"2024-08-30T08:58:02.410749+0000","last_active":"2024-08-30T08:58:47.089986+0000","last_peered":"2024-08-30T08:58:47.089986+0000","last_clean":"2024-08-30T08:58:47.089986+0000","last_became_active":"2024-08-30T08:58:02.410350+0000","last_became_peered":"2024-08-30T08:58:02.410350+0000","last_unstale":"2024-08-30T08:58:47.089986+0000","last_undegraded":"2024-08-30T08:58:47.089986+0000","last_fullsized":"2024-08-30T08:58:47.089986+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T15:22:51.451789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.557090+0000","last_change":"2024-08-30T08:58:02.405946+0000","last_active":"2024-08-30T08:58:46.557090+0000","last_peered":"2024-08-30T08:58:46.557090+0000","last_clean":"2024-08-30T08:58:46.557090+0000","last_became_active":"2024-08-30T08:58:02.405590+0000","last_became_peered":"2024-08-30T08:58:02.405590+0000","last_unstale":"2024-08-30T08:58:46.557090+0000","last_undegraded":"2024-08-30T08:58:46.557090+0000","last_fullsized":"2024-08-30T08:58:46.557090+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T11:42:14.966065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.556440+0000","last_change":"2024-08-30T08:58:02.409649+0000","last_active":"2024-08-30T08:58:46.556440+0000","last_peered":"2024-08-30T08:58:46.556440+0000","last_clean":"2024-08-30T08:58:46.556440+0000","last_became_active":"2024-08-30T08:58:02.409282+0000","last_became_peered":"2024-08-30T08:58:02.409282+0000","last_unstale":"2024-08-30T08:58:46.556440+0000","last_undegraded":"2024-08-30T08:58:46.556440+0000","last_fullsized":"2024-08-30T08:58:46.556440+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T15:33:52.166290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-08-30T08:58:46.553800+0000","last_change":"2024-08-30T08:58:02.410711+0000","last_active":"2024-08-30T08:58:46.553800+0000","last_peered":"2024-08-30T08:58:46.553800+0000","last_clean":"2024-08-30T08:58:46.553800+0000","last_became_active":"2024-08-30T08:58:02.410367+0000","last_became_peered":"2024-08-30T08:58:02.410367+0000","last_unstale":"2024-08-30T08:58:46.553800+0000","last_undegraded":"2024-08-30T08:58:46.553800+0000","last_fullsized":"2024-08-30T08:58:46.553800+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T15:27:40.392588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":68452,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":339,"num_read_kb":242,"num_write":171,"num_write_kb":120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":205356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":146,"ondisk_log_size":146,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":67,"seq":287762808864,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":925696,"data_stored":455486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":61,"seq":261993005094,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28596,"kb_used_data":1572,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716556,"statfs":{"total":95995035648,"available":95965753344,"internally_reserved":0,"allocated":1609728,"data_stored":1107093,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":57,"seq":244813135915,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36260,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708892,"statfs":{"total":95995035648,"available":95957905408,"internally_reserved":0,"allocated":937984,"data_stored":459706,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":51,"seq":219043332145,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":925696,"data_stored":455418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":46,"seq":197568495669,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28516,"kb_used_data":1492,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716636,"statfs":{"total":95995035648,"available":95965835264,"internally_reserved":0,"allocated":1527808,"data_stored":1051830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":41,"seq":176093659195,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":913408,"data_stored":450786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":35,"seq":150323855424,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36844,"kb_used_data":1500,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708308,"statfs":{"total":95995035648,"available":95957307392,"internally_reserved":0,"allocated":1536000,"data_stored":1047027,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":30,"seq":128849018949,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32088,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713064,"statfs":{"total":95995035648,"available":95962177536,"internally_reserved":0,"allocated":925696,"data_stored":455827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215177,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36248,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708904,"statfs":{"total":95995035648,"available":95957917696,"internally_reserved":0,"allocated":925696,"data_stored":450980,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411407,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36328,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708824,"statfs":{"total":95995035648,"available":95957835776,"internally_reserved":0,"allocated":1007616,"data_stored":516497,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574932,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":983040,"data_stored":514756,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738456,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":937984,"data_stored":452174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":64060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":65801,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":66029,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-30T08:59:45.385 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-08-30T08:59:45.385 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-08-30T08:59:45.385 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-08-30T08:59:45.385 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph health --format=json 2024-08-30T08:59:45.644 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T08:59:45.901 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:59:45 smithi042 ceph-mon[18652]: from='client.15195 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-30T08:59:45.901 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:59:45 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]} : dispatch 2024-08-30T08:59:45.901 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:59:45 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 1]} : dispatch 2024-08-30T08:59:45.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:59:45 smithi190 ceph-mon[22001]: from='client.15195 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-30T08:59:45.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:59:45 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]} : dispatch 2024-08-30T08:59:45.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:59:45 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 1]} : dispatch 2024-08-30T08:59:46.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:59:45 smithi167 ceph-mon[22063]: from='client.15195 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-30T08:59:46.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:59:45 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]} : dispatch 2024-08-30T08:59:46.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:59:45 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 1]} : dispatch 2024-08-30T08:59:46.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:46.132+0000 7f744e195640 1 -- 172.21.15.42:0/1388906406 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7448108ac0 msgr2=0x7f744810f2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:46.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:46.132+0000 7f744e195640 1 --2- 172.21.15.42:0/1388906406 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7448108ac0 0x7f744810f2f0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f7440009900 tx=0x7f744002a410 comp rx=0 tx=0).stop 2024-08-30T08:59:46.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:46.132+0000 7f744e195640 1 -- 172.21.15.42:0/1388906406 shutdown_connections 2024-08-30T08:59:46.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:46.132+0000 7f744e195640 1 --2- 172.21.15.42:0/1388906406 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7448108ac0 0x7f744810f2f0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:46.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:46.132+0000 7f744e195640 1 --2- 172.21.15.42:0/1388906406 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7448108170 0x7f7448108580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:46.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:46.132+0000 7f744e195640 1 --2- 172.21.15.42:0/1388906406 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7448102880 0x7f7448102c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:46.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:46.132+0000 7f744e195640 1 -- 172.21.15.42:0/1388906406 >> 172.21.15.42:0/1388906406 conn(0x7f7448079500 msgr2=0x7f7448079910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:59:46.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:46.133+0000 7f744e195640 1 -- 172.21.15.42:0/1388906406 shutdown_connections 2024-08-30T08:59:46.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:46.133+0000 7f744e195640 1 -- 172.21.15.42:0/1388906406 wait complete. 2024-08-30T08:59:46.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:46.133+0000 7f744e195640 1 Processor -- start 2024-08-30T08:59:46.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:46.133+0000 7f744e195640 1 -- start start 2024-08-30T08:59:46.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:46.134+0000 7f744e195640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7448102880 0x7f74480ff410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:46.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:46.134+0000 7f744e195640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7448108170 0x7f74480ff950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:46.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:46.134+0000 7f744e195640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7448108ac0 0x7f74480ffe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:46.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:46.134+0000 7f744e195640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f7448112720 con 0x7f7448108170 2024-08-30T08:59:46.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:46.134+0000 7f744e195640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f74481125a0 con 0x7f7448108ac0 2024-08-30T08:59:46.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:46.134+0000 7f744e195640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f74481128a0 con 0x7f7448102880 2024-08-30T08:59:46.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:46.134+0000 7f7446ffd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7448108170 0x7f74480ff950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:46.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:46.134+0000 7f7446ffd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7448108170 0x7f74480ff950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:40138/0 (socket says 172.21.15.42:40138) 2024-08-30T08:59:46.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:46.134+0000 7f7446ffd640 1 -- 172.21.15.42:0/2442890786 learned_addr learned my addr 172.21.15.42:0/2442890786 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T08:59:46.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:46.134+0000 7f74477fe640 1 --2- 172.21.15.42:0/2442890786 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7448102880 0x7f74480ff410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:46.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:46.134+0000 7f7446ffd640 1 -- 172.21.15.42:0/2442890786 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7448102880 msgr2=0x7f74480ff410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:46.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:46.134+0000 7f7446ffd640 1 --2- 172.21.15.42:0/2442890786 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7448102880 0x7f74480ff410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:46.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:46.134+0000 7f7446ffd640 1 -- 172.21.15.42:0/2442890786 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7448108ac0 msgr2=0x7f74480ffe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:46.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:46.134+0000 7f7447fff640 1 --2- 172.21.15.42:0/2442890786 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7448108ac0 0x7f74480ffe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:46.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:46.134+0000 7f7446ffd640 1 --2- 172.21.15.42:0/2442890786 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7448108ac0 0x7f74480ffe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:46.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:46.134+0000 7f7446ffd640 1 -- 172.21.15.42:0/2442890786 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74481a8460 con 0x7f7448108170 2024-08-30T08:59:46.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:46.134+0000 7f7446ffd640 1 --2- 172.21.15.42:0/2442890786 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7448108170 0x7f74480ff950 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f743000e9c0 tx=0x7f743000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:59:46.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:46.134+0000 7f7444ff9640 1 -- 172.21.15.42:0/2442890786 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f743000cd80 con 0x7f7448108170 2024-08-30T08:59:46.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:46.135+0000 7f744e195640 1 -- 172.21.15.42:0/2442890786 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74481a8750 con 0x7f7448108170 2024-08-30T08:59:46.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:46.135+0000 7f7444ff9640 1 -- 172.21.15.42:0/2442890786 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f74300054d0 con 0x7f7448108170 2024-08-30T08:59:46.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:46.135+0000 7f7444ff9640 1 -- 172.21.15.42:0/2442890786 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7430010430 con 0x7f7448108170 2024-08-30T08:59:46.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:46.135+0000 7f744e195640 1 -- 172.21.15.42:0/2442890786 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74481a8c90 con 0x7f7448108170 2024-08-30T08:59:46.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:46.136+0000 7f744e195640 1 -- 172.21.15.42:0/2442890786 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7448103ab0 con 0x7f7448108170 2024-08-30T08:59:46.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:46.138+0000 7f7444ff9640 1 -- 172.21.15.42:0/2442890786 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) ==== 152628+0+0 (secure 0 0 0) 0x7f7430004fb0 con 0x7f7448108170 2024-08-30T08:59:46.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:46.139+0000 7f7444ff9640 1 --2- 172.21.15.42:0/2442890786 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f74340b4bc0 0x7f74340b70d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:46.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:46.139+0000 7f74477fe640 1 --2- 172.21.15.42:0/2442890786 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f74340b4bc0 0x7f74340b70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:46.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:46.139+0000 7f7444ff9640 1 -- 172.21.15.42:0/2442890786 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 9150+0+0 (secure 0 0 0) 0x7f74300e5e40 con 0x7f7448108170 2024-08-30T08:59:46.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:46.140+0000 7f74477fe640 1 --2- 172.21.15.42:0/2442890786 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f74340b4bc0 0x7f74340b70d0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f7448100e60 tx=0x7f7438009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:59:46.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:46.142+0000 7f7444ff9640 1 -- 172.21.15.42:0/2442890786 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f7430056130 con 0x7f7448108170 2024-08-30T08:59:46.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:46.369+0000 7f744e195640 1 -- 172.21.15.42:0/2442890786 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f7448100710 con 0x7f7448108170 2024-08-30T08:59:46.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:46.370+0000 7f7444ff9640 1 -- 172.21.15.42:0/2442890786 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f74300aca20 con 0x7f7448108170 2024-08-30T08:59:46.371 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-08-30T08:59:46.372 INFO:teuthology.orchestra.run.smithi042.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-08-30T08:59:46.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:46.373+0000 7f744e195640 1 -- 172.21.15.42:0/2442890786 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f74340b4bc0 msgr2=0x7f74340b70d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:46.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:46.373+0000 7f744e195640 1 --2- 172.21.15.42:0/2442890786 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f74340b4bc0 0x7f74340b70d0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f7448100e60 tx=0x7f7438009500 comp rx=0 tx=0).stop 2024-08-30T08:59:46.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:46.373+0000 7f744e195640 1 -- 172.21.15.42:0/2442890786 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7448108170 msgr2=0x7f74480ff950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:46.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:46.373+0000 7f744e195640 1 --2- 172.21.15.42:0/2442890786 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7448108170 0x7f74480ff950 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f743000e9c0 tx=0x7f743000ee80 comp rx=0 tx=0).stop 2024-08-30T08:59:46.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:46.373+0000 7f744e195640 1 -- 172.21.15.42:0/2442890786 shutdown_connections 2024-08-30T08:59:46.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:46.373+0000 7f744e195640 1 --2- 172.21.15.42:0/2442890786 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f74340b4bc0 0x7f74340b70d0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:46.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:46.373+0000 7f744e195640 1 --2- 172.21.15.42:0/2442890786 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7448108ac0 0x7f74480ffe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:46.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:46.373+0000 7f744e195640 1 --2- 172.21.15.42:0/2442890786 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7448108170 0x7f74480ff950 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:46.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:46.373+0000 7f744e195640 1 --2- 172.21.15.42:0/2442890786 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7448102880 0x7f74480ff410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:46.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:46.373+0000 7f744e195640 1 -- 172.21.15.42:0/2442890786 >> 172.21.15.42:0/2442890786 conn(0x7f7448079500 msgr2=0x7f7448101830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:59:46.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:46.374+0000 7f744e195640 1 -- 172.21.15.42:0/2442890786 shutdown_connections 2024-08-30T08:59:46.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:46.374+0000 7f744e195640 1 -- 172.21.15.42:0/2442890786 wait complete. 2024-08-30T08:59:46.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:59:46 smithi190 ceph-mon[22001]: pgmap v341: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-08-30T08:59:46.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:59:46 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]': finished 2024-08-30T08:59:46.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:59:46 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 1]}]': finished 2024-08-30T08:59:46.937 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:59:46 smithi190 ceph-mon[22001]: osdmap e82: 12 total, 12 up, 12 in 2024-08-30T08:59:46.937 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:59:46 smithi190 ceph-mon[22001]: from='client.? 172.21.15.42:0/2442890786' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2024-08-30T08:59:46.966 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:59:46 smithi042 ceph-mon[18652]: pgmap v341: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-08-30T08:59:46.967 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:59:46 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]': finished 2024-08-30T08:59:46.967 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:59:46 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 1]}]': finished 2024-08-30T08:59:46.967 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:59:46 smithi042 ceph-mon[18652]: osdmap e82: 12 total, 12 up, 12 in 2024-08-30T08:59:46.967 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:59:46 smithi042 ceph-mon[18652]: from='client.? 172.21.15.42:0/2442890786' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2024-08-30T08:59:46.994 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-08-30T08:59:46.995 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph fs dump --format=json 2024-08-30T08:59:47.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:59:46 smithi167 ceph-mon[22063]: pgmap v341: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-08-30T08:59:47.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:59:46 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]': finished 2024-08-30T08:59:47.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:59:46 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 1]}]': finished 2024-08-30T08:59:47.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:59:46 smithi167 ceph-mon[22063]: osdmap e82: 12 total, 12 up, 12 in 2024-08-30T08:59:47.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:59:46 smithi167 ceph-mon[22063]: from='client.? 172.21.15.42:0/2442890786' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2024-08-30T08:59:47.246 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T08:59:47.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:47.768+0000 7f659fbd3640 1 -- 172.21.15.42:0/658985808 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6598101c30 msgr2=0x7f6598102040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:47.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:47.768+0000 7f659fbd3640 1 --2- 172.21.15.42:0/658985808 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6598101c30 0x7f6598102040 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f6588009900 tx=0x7f658802a410 comp rx=0 tx=0).stop 2024-08-30T08:59:47.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:47.768+0000 7f659fbd3640 1 -- 172.21.15.42:0/658985808 shutdown_connections 2024-08-30T08:59:47.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:47.768+0000 7f659fbd3640 1 --2- 172.21.15.42:0/658985808 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6598102580 0x7f65981103b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:47.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:47.768+0000 7f659fbd3640 1 --2- 172.21.15.42:0/658985808 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6598101c30 0x7f6598102040 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:47.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:47.768+0000 7f659fbd3640 1 --2- 172.21.15.42:0/658985808 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6598108540 0x7f6598108950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:47.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:47.768+0000 7f659fbd3640 1 -- 172.21.15.42:0/658985808 >> 172.21.15.42:0/658985808 conn(0x7f6598079c60 msgr2=0x7f659807a070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:59:47.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:47.768+0000 7f659fbd3640 1 -- 172.21.15.42:0/658985808 shutdown_connections 2024-08-30T08:59:47.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:47.768+0000 7f659fbd3640 1 -- 172.21.15.42:0/658985808 wait complete. 2024-08-30T08:59:47.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:47.769+0000 7f659fbd3640 1 Processor -- start 2024-08-30T08:59:47.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:47.769+0000 7f659fbd3640 1 -- start start 2024-08-30T08:59:47.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:47.769+0000 7f659fbd3640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6598101c30 0x7f659819a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:47.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:47.769+0000 7f659fbd3640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6598102580 0x7f659819a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:47.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:47.769+0000 7f659fbd3640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6598108540 0x7f659819fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:47.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:47.769+0000 7f659fbd3640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f6598113930 con 0x7f6598102580 2024-08-30T08:59:47.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:47.769+0000 7f659fbd3640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f65981137b0 con 0x7f6598101c30 2024-08-30T08:59:47.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:47.769+0000 7f659fbd3640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f6598113ab0 con 0x7f6598108540 2024-08-30T08:59:47.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:47.769+0000 7f659d16e640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6598102580 0x7f659819a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:47.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:47.769+0000 7f659d16e640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6598102580 0x7f659819a900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:40148/0 (socket says 172.21.15.42:40148) 2024-08-30T08:59:47.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:47.769+0000 7f659d16e640 1 -- 172.21.15.42:0/1628114685 learned_addr learned my addr 172.21.15.42:0/1628114685 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T08:59:47.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:47.769+0000 7f659d96f640 1 --2- 172.21.15.42:0/1628114685 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6598101c30 0x7f659819a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:47.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:47.769+0000 7f659e170640 1 --2- 172.21.15.42:0/1628114685 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6598108540 0x7f659819fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:47.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:47.769+0000 7f659d16e640 1 -- 172.21.15.42:0/1628114685 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6598108540 msgr2=0x7f659819fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:47.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:47.769+0000 7f659d16e640 1 --2- 172.21.15.42:0/1628114685 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6598108540 0x7f659819fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:47.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:47.770+0000 7f659d16e640 1 -- 172.21.15.42:0/1628114685 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6598101c30 msgr2=0x7f659819a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:47.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:47.770+0000 7f659d16e640 1 --2- 172.21.15.42:0/1628114685 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6598101c30 0x7f659819a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:47.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:47.770+0000 7f659d16e640 1 -- 172.21.15.42:0/1628114685 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65981a0210 con 0x7f6598102580 2024-08-30T08:59:47.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:47.770+0000 7f659d16e640 1 --2- 172.21.15.42:0/1628114685 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6598102580 0x7f659819a900 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f658802a930 tx=0x7f6588004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:59:47.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:47.770+0000 7f658effd640 1 -- 172.21.15.42:0/1628114685 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f658803f030 con 0x7f6598102580 2024-08-30T08:59:47.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:47.770+0000 7f658effd640 1 -- 172.21.15.42:0/1628114685 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6588003aa0 con 0x7f6598102580 2024-08-30T08:59:47.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:47.770+0000 7f658effd640 1 -- 172.21.15.42:0/1628114685 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6588035070 con 0x7f6598102580 2024-08-30T08:59:47.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:47.770+0000 7f659fbd3640 1 -- 172.21.15.42:0/1628114685 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65981a04a0 con 0x7f6598102580 2024-08-30T08:59:47.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:47.770+0000 7f659fbd3640 1 -- 172.21.15.42:0/1628114685 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65981a0930 con 0x7f6598102580 2024-08-30T08:59:47.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:47.771+0000 7f659fbd3640 1 -- 172.21.15.42:0/1628114685 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6598102040 con 0x7f6598102580 2024-08-30T08:59:47.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:47.772+0000 7f658effd640 1 -- 172.21.15.42:0/1628114685 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) ==== 152628+0+0 (secure 0 0 0) 0x7f6588042030 con 0x7f6598102580 2024-08-30T08:59:47.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:47.773+0000 7f658effd640 1 --2- 172.21.15.42:0/1628114685 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f65840b4bf0 0x7f65840b7100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:47.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:47.773+0000 7f659d96f640 1 --2- 172.21.15.42:0/1628114685 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f65840b4bf0 0x7f65840b7100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:47.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:47.773+0000 7f658effd640 1 -- 172.21.15.42:0/1628114685 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f65881061b0 con 0x7f6598102580 2024-08-30T08:59:47.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:47.774+0000 7f659d96f640 1 --2- 172.21.15.42:0/1628114685 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f65840b4bf0 0x7f65840b7100 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f6594009990 tx=0x7f6594009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:59:47.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:47.777+0000 7f658effd640 1 -- 172.21.15.42:0/1628114685 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f65880ccd90 con 0x7f6598102580 2024-08-30T08:59:47.937 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:59:47 smithi190 ceph-mon[22001]: osdmap e83: 12 total, 12 up, 12 in 2024-08-30T08:59:47.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:47.998+0000 7f659fbd3640 1 -- 172.21.15.42:0/1628114685 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f659819b490 con 0x7f6598102580 2024-08-30T08:59:48.000 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:59:47 smithi042 ceph-mon[18652]: osdmap e83: 12 total, 12 up, 12 in 2024-08-30T08:59:48.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:47.998+0000 7f658effd640 1 -- 172.21.15.42:0/1628114685 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11915 (secure 0 0 0) 0x7f65880cc690 con 0x7f6598102580 2024-08-30T08:59:48.001 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-08-30T08:59:48.001 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":26,"btime":"2024-08-30T08:58:50:586137+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14703,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/3188400000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":3188400000},{"type":"v1","addr":"172.21.15.42:6835","nonce":3188400000}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14715,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/3356537674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":3356537674},{"type":"v1","addr":"172.21.15.42:6837","nonce":3356537674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14766,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6839/2107785513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6838","nonce":2107785513},{"type":"v1","addr":"172.21.15.42:6839","nonce":2107785513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14778,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6841/2298438674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6840","nonce":2298438674},{"type":"v1","addr":"172.21.15.42:6841","nonce":2298438674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24553,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/543555570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":543555570},{"type":"v1","addr":"172.21.15.190:6833","nonce":543555570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24554,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/3907818941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":3907818941},{"type":"v1","addr":"172.21.15.167:6833","nonce":3907818941}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-30T08:58:03.055084+0000","modified":"2024-08-30T08:58:50.586122+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24629,"mds_1":24610,"mds_2":24599},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24572":{"gid":24572,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6835/234596845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":234596845},{"type":"v1","addr":"172.21.15.167:6835","nonce":234596845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24592":{"gid":24592,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6835/247905709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":247905709},{"type":"v1","addr":"172.21.15.190:6835","nonce":247905709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24598":{"gid":24598,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6837/863256349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6836","nonce":863256349},{"type":"v1","addr":"172.21.15.190:6837","nonce":863256349}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24599":{"gid":24599,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.167:6837/3011837856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":3011837856},{"type":"v1","addr":"172.21.15.167:6837","nonce":3011837856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24610":{"gid":24610,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.190:6839/719981110","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6838","nonce":719981110},{"type":"v1","addr":"172.21.15.190:6839","nonce":719981110}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24629":{"gid":24629,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.167:6839/3604645207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":3604645207},{"type":"v1","addr":"172.21.15.167:6839","nonce":3604645207}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24629,"qdb_cluster":[24629,24610,24599]},"id":1}]} 2024-08-30T08:59:48.001 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 26 2024-08-30T08:59:48.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:48.000+0000 7f659fbd3640 1 -- 172.21.15.42:0/1628114685 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f65840b4bf0 msgr2=0x7f65840b7100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:48.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:48.000+0000 7f659fbd3640 1 --2- 172.21.15.42:0/1628114685 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f65840b4bf0 0x7f65840b7100 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f6594009990 tx=0x7f6594009110 comp rx=0 tx=0).stop 2024-08-30T08:59:48.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:48.001+0000 7f659fbd3640 1 -- 172.21.15.42:0/1628114685 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6598102580 msgr2=0x7f659819a900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:48.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:48.001+0000 7f659fbd3640 1 --2- 172.21.15.42:0/1628114685 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6598102580 0x7f659819a900 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f658802a930 tx=0x7f6588004270 comp rx=0 tx=0).stop 2024-08-30T08:59:48.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:48.001+0000 7f659fbd3640 1 -- 172.21.15.42:0/1628114685 shutdown_connections 2024-08-30T08:59:48.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:48.001+0000 7f659fbd3640 1 --2- 172.21.15.42:0/1628114685 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f65840b4bf0 0x7f65840b7100 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:48.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:48.001+0000 7f659fbd3640 1 --2- 172.21.15.42:0/1628114685 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6598108540 0x7f659819fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:48.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:48.001+0000 7f659fbd3640 1 --2- 172.21.15.42:0/1628114685 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6598102580 0x7f659819a900 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:48.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:48.001+0000 7f659fbd3640 1 --2- 172.21.15.42:0/1628114685 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6598101c30 0x7f659819a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:48.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:48.001+0000 7f659fbd3640 1 -- 172.21.15.42:0/1628114685 >> 172.21.15.42:0/1628114685 conn(0x7f6598079c60 msgr2=0x7f6598105190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:59:48.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:48.001+0000 7f659fbd3640 1 -- 172.21.15.42:0/1628114685 shutdown_connections 2024-08-30T08:59:48.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:48.001+0000 7f659fbd3640 1 -- 172.21.15.42:0/1628114685 wait complete. 2024-08-30T08:59:48.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:59:47 smithi167 ceph-mon[22063]: osdmap e83: 12 total, 12 up, 12 in 2024-08-30T08:59:48.791 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph fs dump --format=json 2024-08-30T08:59:48.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:59:48 smithi190 ceph-mon[22001]: pgmap v344: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 4 op/s; 0 B/s, 1 keys/s, 0 objects/s recovering 2024-08-30T08:59:48.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:59:48 smithi190 ceph-mon[22001]: from='client.? 172.21.15.42:0/1628114685' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-30T08:59:49.047 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T08:59:49.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:59:48 smithi167 ceph-mon[22063]: pgmap v344: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 4 op/s; 0 B/s, 1 keys/s, 0 objects/s recovering 2024-08-30T08:59:49.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:59:48 smithi167 ceph-mon[22063]: from='client.? 172.21.15.42:0/1628114685' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-30T08:59:49.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:59:48 smithi042 ceph-mon[18652]: pgmap v344: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 4 op/s; 0 B/s, 1 keys/s, 0 objects/s recovering 2024-08-30T08:59:49.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:59:48 smithi042 ceph-mon[18652]: from='client.? 172.21.15.42:0/1628114685' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-30T08:59:49.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:49.560+0000 7fbdf7a76640 1 -- 172.21.15.42:0/3852278156 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbdf010ffd0 msgr2=0x7fbdf01123f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:49.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:49.560+0000 7fbdf7a76640 1 --2- 172.21.15.42:0/3852278156 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbdf010ffd0 0x7fbdf01123f0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fbde8009ca0 tx=0x7fbde802a950 comp rx=0 tx=0).stop 2024-08-30T08:59:49.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:49.560+0000 7fbdf7a76640 1 -- 172.21.15.42:0/3852278156 shutdown_connections 2024-08-30T08:59:49.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:49.560+0000 7fbdf7a76640 1 --2- 172.21.15.42:0/3852278156 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbdf010ffd0 0x7fbdf01123f0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:49.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:49.560+0000 7fbdf7a76640 1 --2- 172.21.15.42:0/3852278156 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbdf0102d70 0x7fbdf010fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:49.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:49.560+0000 7fbdf7a76640 1 --2- 172.21.15.42:0/3852278156 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbdf0102420 0x7fbdf0102830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:49.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:49.560+0000 7fbdf7a76640 1 -- 172.21.15.42:0/3852278156 >> 172.21.15.42:0/3852278156 conn(0x7fbdf0079470 msgr2=0x7fbdf0079880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:59:49.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:49.560+0000 7fbdf7a76640 1 -- 172.21.15.42:0/3852278156 shutdown_connections 2024-08-30T08:59:49.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:49.560+0000 7fbdf7a76640 1 -- 172.21.15.42:0/3852278156 wait complete. 2024-08-30T08:59:49.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:49.561+0000 7fbdf7a76640 1 Processor -- start 2024-08-30T08:59:49.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:49.561+0000 7fbdf7a76640 1 -- start start 2024-08-30T08:59:49.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:49.561+0000 7fbdf7a76640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbdf0102420 0x7fbdf019c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:49.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:49.561+0000 7fbdf7a76640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbdf0102d70 0x7fbdf019ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:49.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:49.562+0000 7fbdf7a76640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbdf010ffd0 0x7fbdf01a1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:49.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:49.562+0000 7fbdf7a76640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fbdf0115ac0 con 0x7fbdf010ffd0 2024-08-30T08:59:49.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:49.562+0000 7fbdf7a76640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fbdf0115940 con 0x7fbdf0102d70 2024-08-30T08:59:49.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:49.562+0000 7fbdf7a76640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fbdf0115c40 con 0x7fbdf0102420 2024-08-30T08:59:49.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:49.562+0000 7fbdf6013640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbdf010ffd0 0x7fbdf01a1cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:49.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:49.562+0000 7fbdf6013640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbdf010ffd0 0x7fbdf01a1cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:45308/0 (socket says 172.21.15.42:45308) 2024-08-30T08:59:49.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:49.562+0000 7fbdf6013640 1 -- 172.21.15.42:0/2828372868 learned_addr learned my addr 172.21.15.42:0/2828372868 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T08:59:49.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:49.562+0000 7fbdf5011640 1 --2- 172.21.15.42:0/2828372868 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbdf0102d70 0x7fbdf019ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:49.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:49.562+0000 7fbdf5812640 1 --2- 172.21.15.42:0/2828372868 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbdf0102420 0x7fbdf019c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:49.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:49.562+0000 7fbdf6013640 1 -- 172.21.15.42:0/2828372868 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbdf0102420 msgr2=0x7fbdf019c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:49.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:49.562+0000 7fbdf6013640 1 --2- 172.21.15.42:0/2828372868 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbdf0102420 0x7fbdf019c4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:49.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:49.562+0000 7fbdf6013640 1 -- 172.21.15.42:0/2828372868 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbdf0102d70 msgr2=0x7fbdf019ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:49.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:49.562+0000 7fbdf6013640 1 --2- 172.21.15.42:0/2828372868 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbdf0102d70 0x7fbdf019ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:49.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:49.562+0000 7fbdf6013640 1 -- 172.21.15.42:0/2828372868 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdf01a2420 con 0x7fbdf010ffd0 2024-08-30T08:59:49.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:49.562+0000 7fbdf6013640 1 --2- 172.21.15.42:0/2828372868 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbdf010ffd0 0x7fbdf01a1cf0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fbde802ae70 tx=0x7fbde802f5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:59:49.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:49.562+0000 7fbde6ffd640 1 -- 172.21.15.42:0/2828372868 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbde8034040 con 0x7fbdf010ffd0 2024-08-30T08:59:49.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:49.562+0000 7fbde6ffd640 1 -- 172.21.15.42:0/2828372868 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbde80308f0 con 0x7fbdf010ffd0 2024-08-30T08:59:49.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:49.562+0000 7fbdf7a76640 1 -- 172.21.15.42:0/2828372868 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbdf01a26b0 con 0x7fbdf010ffd0 2024-08-30T08:59:49.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:49.562+0000 7fbde6ffd640 1 -- 172.21.15.42:0/2828372868 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbde80047f0 con 0x7fbdf010ffd0 2024-08-30T08:59:49.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:49.562+0000 7fbdf7a76640 1 -- 172.21.15.42:0/2828372868 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbdf01a8e70 con 0x7fbdf010ffd0 2024-08-30T08:59:49.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:49.563+0000 7fbdf7a76640 1 -- 172.21.15.42:0/2828372868 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbdf006c000 con 0x7fbdf010ffd0 2024-08-30T08:59:49.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:49.565+0000 7fbde6ffd640 1 -- 172.21.15.42:0/2828372868 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) ==== 152628+0+0 (secure 0 0 0) 0x7fbde8037020 con 0x7fbdf010ffd0 2024-08-30T08:59:49.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:49.566+0000 7fbde6ffd640 1 --2- 172.21.15.42:0/2828372868 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fbddc0b4bc0 0x7fbddc0b70d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:49.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:49.566+0000 7fbdf5812640 1 --2- 172.21.15.42:0/2828372868 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fbddc0b4bc0 0x7fbddc0b70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:49.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:49.566+0000 7fbde6ffd640 1 -- 172.21.15.42:0/2828372868 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7fbde81066e0 con 0x7fbdf010ffd0 2024-08-30T08:59:49.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:49.567+0000 7fbdf5812640 1 --2- 172.21.15.42:0/2828372868 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fbddc0b4bc0 0x7fbddc0b70d0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fbdec005fd0 tx=0x7fbdec005e20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:59:49.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:49.569+0000 7fbde6ffd640 1 -- 172.21.15.42:0/2828372868 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7fbde8083130 con 0x7fbdf010ffd0 2024-08-30T08:59:49.796 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:49.794+0000 7fbdf7a76640 1 -- 172.21.15.42:0/2828372868 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbdf01a9040 con 0x7fbdf010ffd0 2024-08-30T08:59:49.796 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:49.795+0000 7fbde6ffd640 1 -- 172.21.15.42:0/2828372868 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11915 (secure 0 0 0) 0x7fbde80cd320 con 0x7fbdf010ffd0 2024-08-30T08:59:49.797 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-08-30T08:59:49.797 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":26,"btime":"2024-08-30T08:58:50:586137+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14703,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/3188400000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":3188400000},{"type":"v1","addr":"172.21.15.42:6835","nonce":3188400000}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14715,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/3356537674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":3356537674},{"type":"v1","addr":"172.21.15.42:6837","nonce":3356537674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14766,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6839/2107785513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6838","nonce":2107785513},{"type":"v1","addr":"172.21.15.42:6839","nonce":2107785513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14778,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6841/2298438674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6840","nonce":2298438674},{"type":"v1","addr":"172.21.15.42:6841","nonce":2298438674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24553,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/543555570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":543555570},{"type":"v1","addr":"172.21.15.190:6833","nonce":543555570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24554,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/3907818941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":3907818941},{"type":"v1","addr":"172.21.15.167:6833","nonce":3907818941}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-30T08:58:03.055084+0000","modified":"2024-08-30T08:58:50.586122+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24629,"mds_1":24610,"mds_2":24599},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24572":{"gid":24572,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6835/234596845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":234596845},{"type":"v1","addr":"172.21.15.167:6835","nonce":234596845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24592":{"gid":24592,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6835/247905709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":247905709},{"type":"v1","addr":"172.21.15.190:6835","nonce":247905709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24598":{"gid":24598,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6837/863256349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6836","nonce":863256349},{"type":"v1","addr":"172.21.15.190:6837","nonce":863256349}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24599":{"gid":24599,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.167:6837/3011837856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":3011837856},{"type":"v1","addr":"172.21.15.167:6837","nonce":3011837856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24610":{"gid":24610,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.190:6839/719981110","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6838","nonce":719981110},{"type":"v1","addr":"172.21.15.190:6839","nonce":719981110}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24629":{"gid":24629,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.167:6839/3604645207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":3604645207},{"type":"v1","addr":"172.21.15.167:6839","nonce":3604645207}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24629,"qdb_cluster":[24629,24610,24599]},"id":1}]} 2024-08-30T08:59:49.797 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 26 2024-08-30T08:59:49.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:49.797+0000 7fbdf7a76640 1 -- 172.21.15.42:0/2828372868 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fbddc0b4bc0 msgr2=0x7fbddc0b70d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:49.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:49.797+0000 7fbdf7a76640 1 --2- 172.21.15.42:0/2828372868 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fbddc0b4bc0 0x7fbddc0b70d0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fbdec005fd0 tx=0x7fbdec005e20 comp rx=0 tx=0).stop 2024-08-30T08:59:49.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:49.797+0000 7fbdf7a76640 1 -- 172.21.15.42:0/2828372868 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbdf010ffd0 msgr2=0x7fbdf01a1cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:49.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:49.797+0000 7fbdf7a76640 1 --2- 172.21.15.42:0/2828372868 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbdf010ffd0 0x7fbdf01a1cf0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fbde802ae70 tx=0x7fbde802f5d0 comp rx=0 tx=0).stop 2024-08-30T08:59:49.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:49.797+0000 7fbdf7a76640 1 -- 172.21.15.42:0/2828372868 shutdown_connections 2024-08-30T08:59:49.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:49.797+0000 7fbdf7a76640 1 --2- 172.21.15.42:0/2828372868 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fbddc0b4bc0 0x7fbddc0b70d0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:49.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:49.797+0000 7fbdf7a76640 1 --2- 172.21.15.42:0/2828372868 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbdf010ffd0 0x7fbdf01a1cf0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:49.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:49.797+0000 7fbdf7a76640 1 --2- 172.21.15.42:0/2828372868 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbdf0102d70 0x7fbdf019ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:49.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:49.797+0000 7fbdf7a76640 1 --2- 172.21.15.42:0/2828372868 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbdf0102420 0x7fbdf019c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:49.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:49.797+0000 7fbdf7a76640 1 -- 172.21.15.42:0/2828372868 >> 172.21.15.42:0/2828372868 conn(0x7fbdf0079470 msgr2=0x7fbdf0109ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:59:49.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:49.798+0000 7fbdf7a76640 1 -- 172.21.15.42:0/2828372868 shutdown_connections 2024-08-30T08:59:49.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:49.798+0000 7fbdf7a76640 1 -- 172.21.15.42:0/2828372868 wait complete. 2024-08-30T08:59:50.636 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph osd dump --format=json 2024-08-30T08:59:50.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:59:50 smithi042 ceph-mon[18652]: pgmap v345: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s; 0 B/s, 1 keys/s, 0 objects/s recovering 2024-08-30T08:59:50.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:59:50 smithi042 ceph-mon[18652]: from='client.? 172.21.15.42:0/2828372868' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-30T08:59:50.890 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T08:59:50.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:59:50 smithi190 ceph-mon[22001]: pgmap v345: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s; 0 B/s, 1 keys/s, 0 objects/s recovering 2024-08-30T08:59:50.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:59:50 smithi190 ceph-mon[22001]: from='client.? 172.21.15.42:0/2828372868' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-30T08:59:51.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:59:50 smithi167 ceph-mon[22063]: pgmap v345: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s; 0 B/s, 1 keys/s, 0 objects/s recovering 2024-08-30T08:59:51.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:59:50 smithi167 ceph-mon[22063]: from='client.? 172.21.15.42:0/2828372868' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-30T08:59:51.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:51.417+0000 7f00d43aa640 1 -- 172.21.15.42:0/1546776695 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f00cc109bc0 msgr2=0x7f00cc1103f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:51.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:51.417+0000 7f00d43aa640 1 --2- 172.21.15.42:0/1546776695 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f00cc109bc0 0x7f00cc1103f0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f00c4009ca0 tx=0x7f00c402a8c0 comp rx=0 tx=0).stop 2024-08-30T08:59:51.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:51.417+0000 7f00d43aa640 1 -- 172.21.15.42:0/1546776695 shutdown_connections 2024-08-30T08:59:51.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:51.417+0000 7f00d43aa640 1 --2- 172.21.15.42:0/1546776695 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f00cc109bc0 0x7f00cc1103f0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:51.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:51.417+0000 7f00d43aa640 1 --2- 172.21.15.42:0/1546776695 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f00cc109270 0x7f00cc109680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:51.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:51.417+0000 7f00d43aa640 1 --2- 172.21.15.42:0/1546776695 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f00cc103930 0x7f00cc103d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:51.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:51.417+0000 7f00d43aa640 1 -- 172.21.15.42:0/1546776695 >> 172.21.15.42:0/1546776695 conn(0x7f00cc0ff3a0 msgr2=0x7f00cc1017f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:59:51.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:51.417+0000 7f00d43aa640 1 -- 172.21.15.42:0/1546776695 shutdown_connections 2024-08-30T08:59:51.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:51.417+0000 7f00d43aa640 1 -- 172.21.15.42:0/1546776695 wait complete. 2024-08-30T08:59:51.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:51.418+0000 7f00d43aa640 1 Processor -- start 2024-08-30T08:59:51.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:51.418+0000 7f00d43aa640 1 -- start start 2024-08-30T08:59:51.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:51.418+0000 7f00d43aa640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f00cc103930 0x7f00cc19a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:51.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:51.418+0000 7f00d43aa640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f00cc109270 0x7f00cc19a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:51.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:51.418+0000 7f00d43aa640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f00cc109bc0 0x7f00cc19fa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:51.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:51.418+0000 7f00d43aa640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f00cc113880 con 0x7f00cc109bc0 2024-08-30T08:59:51.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:51.418+0000 7f00d43aa640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f00cc113700 con 0x7f00cc103930 2024-08-30T08:59:51.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:51.418+0000 7f00d43aa640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f00cc113a00 con 0x7f00cc109270 2024-08-30T08:59:51.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:51.418+0000 7f00d2947640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f00cc109bc0 0x7f00cc19fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:51.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:51.418+0000 7f00d2947640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f00cc109bc0 0x7f00cc19fa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:45322/0 (socket says 172.21.15.42:45322) 2024-08-30T08:59:51.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:51.418+0000 7f00d2947640 1 -- 172.21.15.42:0/3602586576 learned_addr learned my addr 172.21.15.42:0/3602586576 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T08:59:51.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:51.418+0000 7f00d2146640 1 --2- 172.21.15.42:0/3602586576 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f00cc103930 0x7f00cc19a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:51.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:51.418+0000 7f00d1945640 1 --2- 172.21.15.42:0/3602586576 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f00cc109270 0x7f00cc19a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:51.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:51.419+0000 7f00d2947640 1 -- 172.21.15.42:0/3602586576 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f00cc109270 msgr2=0x7f00cc19a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:51.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:51.419+0000 7f00d2947640 1 --2- 172.21.15.42:0/3602586576 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f00cc109270 0x7f00cc19a7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:51.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:51.419+0000 7f00d2947640 1 -- 172.21.15.42:0/3602586576 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f00cc103930 msgr2=0x7f00cc19a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:51.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:51.419+0000 7f00d2947640 1 --2- 172.21.15.42:0/3602586576 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f00cc103930 0x7f00cc19a250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:51.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:51.419+0000 7f00d2947640 1 -- 172.21.15.42:0/3602586576 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00cc1a01c0 con 0x7f00cc109bc0 2024-08-30T08:59:51.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:51.419+0000 7f00d2947640 1 --2- 172.21.15.42:0/3602586576 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f00cc109bc0 0x7f00cc19fa90 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f00c4005cc0 tx=0x7f00c402f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:59:51.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:51.419+0000 7f00c37fe640 1 -- 172.21.15.42:0/3602586576 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00c4003bf0 con 0x7f00cc109bc0 2024-08-30T08:59:51.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:51.419+0000 7f00d43aa640 1 -- 172.21.15.42:0/3602586576 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00cc1a0450 con 0x7f00cc109bc0 2024-08-30T08:59:51.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:51.419+0000 7f00c37fe640 1 -- 172.21.15.42:0/3602586576 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f00c4003d90 con 0x7f00cc109bc0 2024-08-30T08:59:51.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:51.419+0000 7f00c37fe640 1 -- 172.21.15.42:0/3602586576 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00c402fbc0 con 0x7f00cc109bc0 2024-08-30T08:59:51.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:51.419+0000 7f00d43aa640 1 -- 172.21.15.42:0/3602586576 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00cc1a6c10 con 0x7f00cc109bc0 2024-08-30T08:59:51.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:51.420+0000 7f00d43aa640 1 -- 172.21.15.42:0/3602586576 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00cc103d40 con 0x7f00cc109bc0 2024-08-30T08:59:51.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:51.422+0000 7f00c37fe640 1 -- 172.21.15.42:0/3602586576 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) ==== 152628+0+0 (secure 0 0 0) 0x7f00c402fd60 con 0x7f00cc109bc0 2024-08-30T08:59:51.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:51.423+0000 7f00c37fe640 1 --2- 172.21.15.42:0/3602586576 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f00b80b4bc0 0x7f00b80b70d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:51.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:51.423+0000 7f00d2146640 1 --2- 172.21.15.42:0/3602586576 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f00b80b4bc0 0x7f00b80b70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:51.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:51.423+0000 7f00c37fe640 1 -- 172.21.15.42:0/3602586576 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f00c4106ca0 con 0x7f00cc109bc0 2024-08-30T08:59:51.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:51.423+0000 7f00d2146640 1 --2- 172.21.15.42:0/3602586576 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f00b80b4bc0 0x7f00b80b70d0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f00c8005fd0 tx=0x7f00c800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:59:51.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:51.426+0000 7f00c37fe640 1 -- 172.21.15.42:0/3602586576 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f00c40cd880 con 0x7f00cc109bc0 2024-08-30T08:59:51.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:51.612+0000 7f00d43aa640 1 -- 172.21.15.42:0/3602586576 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f00cc19b380 con 0x7f00cc109bc0 2024-08-30T08:59:51.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:51.614+0000 7f00c37fe640 1 -- 172.21.15.42:0/3602586576 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+24867 (secure 0 0 0) 0x7f00cc19b380 con 0x7f00cc109bc0 2024-08-30T08:59:51.616 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-08-30T08:59:51.616 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":83,"fsid":"b0895c1a-66ac-11ef-bcd4-c7b262605968","created":"2024-08-30T08:49:44.858265+0000","modified":"2024-08-30T08:59:46.645920+0000","last_up_change":"2024-08-30T08:57:12.367791+0000","last_in_change":"2024-08-30T08:56:50.274678+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-30T08:53:27.297022+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-30T08:57:58.489964+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-30T08:58:01.064783+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-08-30T08:58:07.416534+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"029d62d7-a6c6-4905-8c76-6dffa9d57b92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":2617890537},{"type":"v1","addr":"172.21.15.42:6803","nonce":2617890537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":2617890537},{"type":"v1","addr":"172.21.15.42:6805","nonce":2617890537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2617890537},{"type":"v1","addr":"172.21.15.42:6809","nonce":2617890537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":2617890537},{"type":"v1","addr":"172.21.15.42:6807","nonce":2617890537}]},"public_addr":"172.21.15.42:6803/2617890537","cluster_addr":"172.21.15.42:6805/2617890537","heartbeat_back_addr":"172.21.15.42:6809/2617890537","heartbeat_front_addr":"172.21.15.42:6807/2617890537","state":["exists","up"]},{"osd":1,"uuid":"3a78384b-0207-41df-9384-22665007d5b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":2791960082},{"type":"v1","addr":"172.21.15.167:6801","nonce":2791960082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":2791960082},{"type":"v1","addr":"172.21.15.167:6803","nonce":2791960082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":2791960082},{"type":"v1","addr":"172.21.15.167:6807","nonce":2791960082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":2791960082},{"type":"v1","addr":"172.21.15.167:6805","nonce":2791960082}]},"public_addr":"172.21.15.167:6801/2791960082","cluster_addr":"172.21.15.167:6803/2791960082","heartbeat_back_addr":"172.21.15.167:6807/2791960082","heartbeat_front_addr":"172.21.15.167:6805/2791960082","state":["exists","up"]},{"osd":2,"uuid":"16001809-c27c-4f52-948a-9998261acb4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":1101326404},{"type":"v1","addr":"172.21.15.190:6801","nonce":1101326404}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":1101326404},{"type":"v1","addr":"172.21.15.190:6803","nonce":1101326404}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":1101326404},{"type":"v1","addr":"172.21.15.190:6807","nonce":1101326404}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":1101326404},{"type":"v1","addr":"172.21.15.190:6805","nonce":1101326404}]},"public_addr":"172.21.15.190:6801/1101326404","cluster_addr":"172.21.15.190:6803/1101326404","heartbeat_back_addr":"172.21.15.190:6807/1101326404","heartbeat_front_addr":"172.21.15.190:6805/1101326404","state":["exists","up"]},{"osd":3,"uuid":"87bb3e36-9e9e-4b19-9d8c-0cdf6e089d3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":1135599916},{"type":"v1","addr":"172.21.15.42:6811","nonce":1135599916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":1135599916},{"type":"v1","addr":"172.21.15.42:6813","nonce":1135599916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":1135599916},{"type":"v1","addr":"172.21.15.42:6817","nonce":1135599916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":1135599916},{"type":"v1","addr":"172.21.15.42:6815","nonce":1135599916}]},"public_addr":"172.21.15.42:6811/1135599916","cluster_addr":"172.21.15.42:6813/1135599916","heartbeat_back_addr":"172.21.15.42:6817/1135599916","heartbeat_front_addr":"172.21.15.42:6815/1135599916","state":["exists","up"]},{"osd":4,"uuid":"3ca1aae3-c972-4fb9-ac86-236279c30d64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":2927511419},{"type":"v1","addr":"172.21.15.167:6809","nonce":2927511419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":2927511419},{"type":"v1","addr":"172.21.15.167:6811","nonce":2927511419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":2927511419},{"type":"v1","addr":"172.21.15.167:6815","nonce":2927511419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":2927511419},{"type":"v1","addr":"172.21.15.167:6813","nonce":2927511419}]},"public_addr":"172.21.15.167:6809/2927511419","cluster_addr":"172.21.15.167:6811/2927511419","heartbeat_back_addr":"172.21.15.167:6815/2927511419","heartbeat_front_addr":"172.21.15.167:6813/2927511419","state":["exists","up"]},{"osd":5,"uuid":"5d07f54b-0d07-4efe-9bcf-390755ce0258","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":2557145333},{"type":"v1","addr":"172.21.15.190:6809","nonce":2557145333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":2557145333},{"type":"v1","addr":"172.21.15.190:6811","nonce":2557145333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":2557145333},{"type":"v1","addr":"172.21.15.190:6815","nonce":2557145333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":2557145333},{"type":"v1","addr":"172.21.15.190:6813","nonce":2557145333}]},"public_addr":"172.21.15.190:6809/2557145333","cluster_addr":"172.21.15.190:6811/2557145333","heartbeat_back_addr":"172.21.15.190:6815/2557145333","heartbeat_front_addr":"172.21.15.190:6813/2557145333","state":["exists","up"]},{"osd":6,"uuid":"96ad64af-44ff-4197-97d4-156f0cb88e81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":3213526100},{"type":"v1","addr":"172.21.15.42:6819","nonce":3213526100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":3213526100},{"type":"v1","addr":"172.21.15.42:6821","nonce":3213526100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":3213526100},{"type":"v1","addr":"172.21.15.42:6825","nonce":3213526100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":3213526100},{"type":"v1","addr":"172.21.15.42:6823","nonce":3213526100}]},"public_addr":"172.21.15.42:6819/3213526100","cluster_addr":"172.21.15.42:6821/3213526100","heartbeat_back_addr":"172.21.15.42:6825/3213526100","heartbeat_front_addr":"172.21.15.42:6823/3213526100","state":["exists","up"]},{"osd":7,"uuid":"337a2112-eaf7-4e6b-86be-4e40d6ddf835","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":2962799814},{"type":"v1","addr":"172.21.15.167:6817","nonce":2962799814}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":2962799814},{"type":"v1","addr":"172.21.15.167:6819","nonce":2962799814}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":2962799814},{"type":"v1","addr":"172.21.15.167:6823","nonce":2962799814}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":2962799814},{"type":"v1","addr":"172.21.15.167:6821","nonce":2962799814}]},"public_addr":"172.21.15.167:6817/2962799814","cluster_addr":"172.21.15.167:6819/2962799814","heartbeat_back_addr":"172.21.15.167:6823/2962799814","heartbeat_front_addr":"172.21.15.167:6821/2962799814","state":["exists","up"]},{"osd":8,"uuid":"219a7224-4331-464b-ac86-c51054d51d91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":3242198843},{"type":"v1","addr":"172.21.15.190:6817","nonce":3242198843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":3242198843},{"type":"v1","addr":"172.21.15.190:6819","nonce":3242198843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":3242198843},{"type":"v1","addr":"172.21.15.190:6823","nonce":3242198843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":3242198843},{"type":"v1","addr":"172.21.15.190:6821","nonce":3242198843}]},"public_addr":"172.21.15.190:6817/3242198843","cluster_addr":"172.21.15.190:6819/3242198843","heartbeat_back_addr":"172.21.15.190:6823/3242198843","heartbeat_front_addr":"172.21.15.190:6821/3242198843","state":["exists","up"]},{"osd":9,"uuid":"2f2f5177-a009-40d9-991c-76aec8a5cba6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":1123536325},{"type":"v1","addr":"172.21.15.42:6827","nonce":1123536325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":1123536325},{"type":"v1","addr":"172.21.15.42:6829","nonce":1123536325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":1123536325},{"type":"v1","addr":"172.21.15.42:6833","nonce":1123536325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":1123536325},{"type":"v1","addr":"172.21.15.42:6831","nonce":1123536325}]},"public_addr":"172.21.15.42:6827/1123536325","cluster_addr":"172.21.15.42:6829/1123536325","heartbeat_back_addr":"172.21.15.42:6833/1123536325","heartbeat_front_addr":"172.21.15.42:6831/1123536325","state":["exists","up"]},{"osd":10,"uuid":"96ef5058-2518-4862-a165-17666b33a3e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":3451198319},{"type":"v1","addr":"172.21.15.167:6825","nonce":3451198319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":3451198319},{"type":"v1","addr":"172.21.15.167:6827","nonce":3451198319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":3451198319},{"type":"v1","addr":"172.21.15.167:6831","nonce":3451198319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":3451198319},{"type":"v1","addr":"172.21.15.167:6829","nonce":3451198319}]},"public_addr":"172.21.15.167:6825/3451198319","cluster_addr":"172.21.15.167:6827/3451198319","heartbeat_back_addr":"172.21.15.167:6831/3451198319","heartbeat_front_addr":"172.21.15.167:6829/3451198319","state":["exists","up"]},{"osd":11,"uuid":"a4eb9bd3-3faa-4c81-ba6d-c2084b72c9ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":380064457},{"type":"v1","addr":"172.21.15.190:6825","nonce":380064457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":380064457},{"type":"v1","addr":"172.21.15.190:6827","nonce":380064457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":380064457},{"type":"v1","addr":"172.21.15.190:6831","nonce":380064457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":380064457},{"type":"v1","addr":"172.21.15.190:6829","nonce":380064457}]},"public_addr":"172.21.15.190:6825/380064457","cluster_addr":"172.21.15.190:6827/380064457","heartbeat_back_addr":"172.21.15.190:6831/380064457","heartbeat_front_addr":"172.21.15.190:6829/380064457","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-30T08:52:36.511448+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-30T08:53:00.027277+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-30T08:53:23.770245+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-30T08:53:49.322226+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-30T08:54:14.350234+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-30T08:54:39.491793+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-30T08:55:04.870596+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-30T08:55:30.314150+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-30T08:55:54.483843+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-30T08:56:20.586115+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-30T08:56:45.666783+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-30T08:57:10.718387+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"2.1d","mappings":[{"from":2,"to":1}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"4.9","mappings":[{"from":5,"to":8}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.42:0/2691233260":"2024-08-31T08:50:44.821070+0000","172.21.15.42:0/1518053351":"2024-08-31T08:50:44.821070+0000","172.21.15.42:0/352001020":"2024-08-31T08:50:44.821070+0000","172.21.15.42:6801/1733768634":"2024-08-31T08:50:44.821070+0000","172.21.15.42:0/1000597888":"2024-08-31T08:50:18.561851+0000","172.21.15.42:6800/1733768634":"2024-08-31T08:50:44.821070+0000","172.21.15.42:6801/3538882081":"2024-08-31T08:50:18.561851+0000","172.21.15.42:6800/3538882081":"2024-08-31T08:50:18.561851+0000","172.21.15.42:0/145366508":"2024-08-31T08:50:18.561851+0000","172.21.15.42:0/3346355213":"2024-08-31T08:50:18.561851+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-30T08:59:51.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:51.617+0000 7f00d43aa640 1 -- 172.21.15.42:0/3602586576 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f00b80b4bc0 msgr2=0x7f00b80b70d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:51.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:51.617+0000 7f00d43aa640 1 --2- 172.21.15.42:0/3602586576 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f00b80b4bc0 0x7f00b80b70d0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f00c8005fd0 tx=0x7f00c800b390 comp rx=0 tx=0).stop 2024-08-30T08:59:51.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:51.617+0000 7f00d43aa640 1 -- 172.21.15.42:0/3602586576 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f00cc109bc0 msgr2=0x7f00cc19fa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:51.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:51.617+0000 7f00d43aa640 1 --2- 172.21.15.42:0/3602586576 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f00cc109bc0 0x7f00cc19fa90 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f00c4005cc0 tx=0x7f00c402f380 comp rx=0 tx=0).stop 2024-08-30T08:59:51.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:51.617+0000 7f00d43aa640 1 -- 172.21.15.42:0/3602586576 shutdown_connections 2024-08-30T08:59:51.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:51.617+0000 7f00d43aa640 1 --2- 172.21.15.42:0/3602586576 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f00b80b4bc0 0x7f00b80b70d0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:51.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:51.617+0000 7f00d43aa640 1 --2- 172.21.15.42:0/3602586576 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f00cc109270 0x7f00cc19a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:51.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:51.617+0000 7f00d43aa640 1 --2- 172.21.15.42:0/3602586576 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f00cc109bc0 0x7f00cc19fa90 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:51.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:51.617+0000 7f00d43aa640 1 --2- 172.21.15.42:0/3602586576 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f00cc103930 0x7f00cc19a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:51.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:51.617+0000 7f00d43aa640 1 -- 172.21.15.42:0/3602586576 >> 172.21.15.42:0/3602586576 conn(0x7f00cc0ff3a0 msgr2=0x7f00cc106800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:59:51.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:51.617+0000 7f00d43aa640 1 -- 172.21.15.42:0/3602586576 shutdown_connections 2024-08-30T08:59:51.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:51.617+0000 7f00d43aa640 1 -- 172.21.15.42:0/3602586576 wait complete. 2024-08-30T08:59:51.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:59:51 smithi042 ceph-mon[18652]: from='client.? 172.21.15.42:0/3602586576' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-30T08:59:51.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:59:51 smithi190 ceph-mon[22001]: from='client.? 172.21.15.42:0/3602586576' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-30T08:59:52.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:59:51 smithi167 ceph-mon[22063]: from='client.? 172.21.15.42:0/3602586576' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-30T08:59:52.351 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph fs dump --format=json 2024-08-30T08:59:52.603 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T08:59:52.929 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:59:52 smithi042 ceph-mon[18652]: pgmap v346: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 3 op/s; 0 B/s, 1 keys/s, 0 objects/s recovering 2024-08-30T08:59:52.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:59:52 smithi190 ceph-mon[22001]: pgmap v346: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 3 op/s; 0 B/s, 1 keys/s, 0 objects/s recovering 2024-08-30T08:59:53.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:59:52 smithi167 ceph-mon[22063]: pgmap v346: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 3 op/s; 0 B/s, 1 keys/s, 0 objects/s recovering 2024-08-30T08:59:53.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:53.124+0000 7fdf37013640 1 -- 172.21.15.42:0/4052257264 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdf30109b90 msgr2=0x7fdf301103c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:53.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:53.124+0000 7fdf37013640 1 --2- 172.21.15.42:0/4052257264 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdf30109b90 0x7fdf301103c0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fdf28009ca0 tx=0x7fdf2802a8c0 comp rx=0 tx=0).stop 2024-08-30T08:59:53.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:53.124+0000 7fdf37013640 1 -- 172.21.15.42:0/4052257264 shutdown_connections 2024-08-30T08:59:53.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:53.124+0000 7fdf37013640 1 --2- 172.21.15.42:0/4052257264 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdf30109b90 0x7fdf301103c0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:53.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:53.124+0000 7fdf37013640 1 --2- 172.21.15.42:0/4052257264 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdf30109240 0x7fdf30109650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:53.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:53.124+0000 7fdf37013640 1 --2- 172.21.15.42:0/4052257264 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdf30103900 0x7fdf30103d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:53.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:53.124+0000 7fdf37013640 1 -- 172.21.15.42:0/4052257264 >> 172.21.15.42:0/4052257264 conn(0x7fdf3007b5f0 msgr2=0x7fdf3007ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:59:53.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:53.124+0000 7fdf37013640 1 -- 172.21.15.42:0/4052257264 shutdown_connections 2024-08-30T08:59:53.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:53.124+0000 7fdf37013640 1 -- 172.21.15.42:0/4052257264 wait complete. 2024-08-30T08:59:53.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:53.125+0000 7fdf37013640 1 Processor -- start 2024-08-30T08:59:53.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:53.125+0000 7fdf37013640 1 -- start start 2024-08-30T08:59:53.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:53.125+0000 7fdf37013640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdf30103900 0x7fdf3019a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:53.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:53.125+0000 7fdf37013640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdf30109240 0x7fdf3019a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:53.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:53.125+0000 7fdf37013640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdf30109b90 0x7fdf3019fa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:53.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:53.125+0000 7fdf37013640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fdf30113850 con 0x7fdf30109b90 2024-08-30T08:59:53.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:53.125+0000 7fdf37013640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fdf301136d0 con 0x7fdf30103900 2024-08-30T08:59:53.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:53.125+0000 7fdf37013640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fdf301139d0 con 0x7fdf30109240 2024-08-30T08:59:53.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:53.126+0000 7fdf355b0640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdf30109b90 0x7fdf3019fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:53.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:53.126+0000 7fdf355b0640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdf30109b90 0x7fdf3019fa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:45338/0 (socket says 172.21.15.42:45338) 2024-08-30T08:59:53.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:53.126+0000 7fdf355b0640 1 -- 172.21.15.42:0/628453725 learned_addr learned my addr 172.21.15.42:0/628453725 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T08:59:53.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:53.126+0000 7fdf34daf640 1 --2- 172.21.15.42:0/628453725 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdf30103900 0x7fdf3019a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:53.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:53.126+0000 7fdf2ffff640 1 --2- 172.21.15.42:0/628453725 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdf30109240 0x7fdf3019a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:53.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:53.126+0000 7fdf355b0640 1 -- 172.21.15.42:0/628453725 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdf30109240 msgr2=0x7fdf3019a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:53.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:53.126+0000 7fdf355b0640 1 --2- 172.21.15.42:0/628453725 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdf30109240 0x7fdf3019a7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:53.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:53.126+0000 7fdf355b0640 1 -- 172.21.15.42:0/628453725 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdf30103900 msgr2=0x7fdf3019a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:53.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:53.126+0000 7fdf355b0640 1 --2- 172.21.15.42:0/628453725 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdf30103900 0x7fdf3019a220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:53.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:53.126+0000 7fdf355b0640 1 -- 172.21.15.42:0/628453725 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf301a00d0 con 0x7fdf30109b90 2024-08-30T08:59:53.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:53.126+0000 7fdf355b0640 1 --2- 172.21.15.42:0/628453725 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdf30109b90 0x7fdf3019fa60 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fdf28005cc0 tx=0x7fdf2802f470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:59:53.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:53.126+0000 7fdf2dffb640 1 -- 172.21.15.42:0/628453725 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf28003cf0 con 0x7fdf30109b90 2024-08-30T08:59:53.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:53.126+0000 7fdf37013640 1 -- 172.21.15.42:0/628453725 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf301a0360 con 0x7fdf30109b90 2024-08-30T08:59:53.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:53.126+0000 7fdf2dffb640 1 -- 172.21.15.42:0/628453725 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdf280044f0 con 0x7fdf30109b90 2024-08-30T08:59:53.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:53.126+0000 7fdf2dffb640 1 -- 172.21.15.42:0/628453725 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf2802fd10 con 0x7fdf30109b90 2024-08-30T08:59:53.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:53.127+0000 7fdf37013640 1 -- 172.21.15.42:0/628453725 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf301a6be0 con 0x7fdf30109b90 2024-08-30T08:59:53.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:53.128+0000 7fdf37013640 1 -- 172.21.15.42:0/628453725 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf30103d10 con 0x7fdf30109b90 2024-08-30T08:59:53.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:53.129+0000 7fdf2dffb640 1 -- 172.21.15.42:0/628453725 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) ==== 152628+0+0 (secure 0 0 0) 0x7fdf28004040 con 0x7fdf30109b90 2024-08-30T08:59:53.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:53.130+0000 7fdf2dffb640 1 --2- 172.21.15.42:0/628453725 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fdf200b4bf0 0x7fdf200b7100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:53.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:53.130+0000 7fdf34daf640 1 --2- 172.21.15.42:0/628453725 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fdf200b4bf0 0x7fdf200b7100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:53.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:53.130+0000 7fdf2dffb640 1 -- 172.21.15.42:0/628453725 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7fdf28106d80 con 0x7fdf30109b90 2024-08-30T08:59:53.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:53.130+0000 7fdf34daf640 1 --2- 172.21.15.42:0/628453725 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fdf200b4bf0 0x7fdf200b7100 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fdf240058c0 tx=0x7fdf2400a250 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:59:53.134 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:53.133+0000 7fdf2dffb640 1 -- 172.21.15.42:0/628453725 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7fdf280cd960 con 0x7fdf30109b90 2024-08-30T08:59:53.356 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:53.354+0000 7fdf37013640 1 -- 172.21.15.42:0/628453725 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdf3010b9e0 con 0x7fdf30109b90 2024-08-30T08:59:53.356 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:53.355+0000 7fdf2dffb640 1 -- 172.21.15.42:0/628453725 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11915 (secure 0 0 0) 0x7fdf280cd260 con 0x7fdf30109b90 2024-08-30T08:59:53.357 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-08-30T08:59:53.357 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":26,"btime":"2024-08-30T08:58:50:586137+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14703,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/3188400000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":3188400000},{"type":"v1","addr":"172.21.15.42:6835","nonce":3188400000}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14715,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/3356537674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":3356537674},{"type":"v1","addr":"172.21.15.42:6837","nonce":3356537674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14766,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6839/2107785513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6838","nonce":2107785513},{"type":"v1","addr":"172.21.15.42:6839","nonce":2107785513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14778,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6841/2298438674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6840","nonce":2298438674},{"type":"v1","addr":"172.21.15.42:6841","nonce":2298438674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24553,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/543555570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":543555570},{"type":"v1","addr":"172.21.15.190:6833","nonce":543555570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24554,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/3907818941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":3907818941},{"type":"v1","addr":"172.21.15.167:6833","nonce":3907818941}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-30T08:58:03.055084+0000","modified":"2024-08-30T08:58:50.586122+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24629,"mds_1":24610,"mds_2":24599},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24572":{"gid":24572,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6835/234596845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":234596845},{"type":"v1","addr":"172.21.15.167:6835","nonce":234596845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24592":{"gid":24592,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6835/247905709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":247905709},{"type":"v1","addr":"172.21.15.190:6835","nonce":247905709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24598":{"gid":24598,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6837/863256349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6836","nonce":863256349},{"type":"v1","addr":"172.21.15.190:6837","nonce":863256349}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24599":{"gid":24599,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.167:6837/3011837856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":3011837856},{"type":"v1","addr":"172.21.15.167:6837","nonce":3011837856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24610":{"gid":24610,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.190:6839/719981110","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6838","nonce":719981110},{"type":"v1","addr":"172.21.15.190:6839","nonce":719981110}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24629":{"gid":24629,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.167:6839/3604645207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":3604645207},{"type":"v1","addr":"172.21.15.167:6839","nonce":3604645207}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24629,"qdb_cluster":[24629,24610,24599]},"id":1}]} 2024-08-30T08:59:53.357 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 26 2024-08-30T08:59:53.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:53.357+0000 7fdf37013640 1 -- 172.21.15.42:0/628453725 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fdf200b4bf0 msgr2=0x7fdf200b7100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:53.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:53.357+0000 7fdf37013640 1 --2- 172.21.15.42:0/628453725 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fdf200b4bf0 0x7fdf200b7100 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fdf240058c0 tx=0x7fdf2400a250 comp rx=0 tx=0).stop 2024-08-30T08:59:53.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:53.357+0000 7fdf37013640 1 -- 172.21.15.42:0/628453725 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdf30109b90 msgr2=0x7fdf3019fa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:53.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:53.357+0000 7fdf37013640 1 --2- 172.21.15.42:0/628453725 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdf30109b90 0x7fdf3019fa60 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fdf28005cc0 tx=0x7fdf2802f470 comp rx=0 tx=0).stop 2024-08-30T08:59:53.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:53.357+0000 7fdf37013640 1 -- 172.21.15.42:0/628453725 shutdown_connections 2024-08-30T08:59:53.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:53.357+0000 7fdf37013640 1 --2- 172.21.15.42:0/628453725 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fdf200b4bf0 0x7fdf200b7100 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:53.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:53.358+0000 7fdf37013640 1 --2- 172.21.15.42:0/628453725 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdf30109240 0x7fdf3019a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:53.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:53.358+0000 7fdf37013640 1 --2- 172.21.15.42:0/628453725 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdf30109b90 0x7fdf3019fa60 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:53.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:53.358+0000 7fdf37013640 1 --2- 172.21.15.42:0/628453725 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdf30103900 0x7fdf3019a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:53.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:53.358+0000 7fdf37013640 1 -- 172.21.15.42:0/628453725 >> 172.21.15.42:0/628453725 conn(0x7fdf3007b5f0 msgr2=0x7fdf30105210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:59:53.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:53.358+0000 7fdf37013640 1 -- 172.21.15.42:0/628453725 shutdown_connections 2024-08-30T08:59:53.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:53.358+0000 7fdf37013640 1 -- 172.21.15.42:0/628453725 wait complete. 2024-08-30T08:59:53.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:59:53 smithi190 ceph-mon[22001]: from='client.? 172.21.15.42:0/628453725' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-30T08:59:54.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:59:53 smithi167 ceph-mon[22063]: from='client.? 172.21.15.42:0/628453725' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-30T08:59:54.119 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:59:53 smithi042 ceph-mon[18652]: from='client.? 172.21.15.42:0/628453725' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-30T08:59:54.149 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 26, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-08-30T08:58:03.055084+0000', 'modified': '2024-08-30T08:58:50.586122+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24629, 'mds_1': 24610, 'mds_2': 24599}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24572': {'gid': 24572, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.167:6835/234596845', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.167:6834', 'nonce': 234596845}, {'type': 'v1', 'addr': '172.21.15.167:6835', 'nonce': 234596845}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24592': {'gid': 24592, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.190:6835/247905709', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.190:6834', 'nonce': 247905709}, {'type': 'v1', 'addr': '172.21.15.190:6835', 'nonce': 247905709}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24598': {'gid': 24598, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.190:6837/863256349', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.190:6836', 'nonce': 863256349}, {'type': 'v1', 'addr': '172.21.15.190:6837', 'nonce': 863256349}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24599': {'gid': 24599, 'name': 'h', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 19, 'addr': '172.21.15.167:6837/3011837856', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.167:6836', 'nonce': 3011837856}, {'type': 'v1', 'addr': '172.21.15.167:6837', 'nonce': 3011837856}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24610': {'gid': 24610, 'name': 'i', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 17, 'addr': '172.21.15.190:6839/719981110', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.190:6838', 'nonce': 719981110}, {'type': 'v1', 'addr': '172.21.15.190:6839', 'nonce': 719981110}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24629': {'gid': 24629, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.167:6839/3604645207', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.167:6838', 'nonce': 3604645207}, {'type': 'v1', 'addr': '172.21.15.167:6839', 'nonce': 3604645207}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24629, 'qdb_cluster': [24629, 24610, 24599]} 2024-08-30T08:59:54.149 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-08-30T08:59:54.150 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph tell mds.h status 2024-08-30T08:59:54.404 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T08:59:54.789 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:59:54 smithi042 ceph-mon[18652]: pgmap v347: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s; 0 B/s, 1 keys/s, 0 objects/s recovering 2024-08-30T08:59:54.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.879+0000 7f1b3cee1640 1 -- 172.21.15.42:0/2434842272 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1b38109240 msgr2=0x7f1b38109650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:54.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.879+0000 7f1b3cee1640 1 --2- 172.21.15.42:0/2434842272 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1b38109240 0x7f1b38109650 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f1b2c009900 tx=0x7f1b2c02a410 comp rx=0 tx=0).stop 2024-08-30T08:59:54.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.879+0000 7f1b3cee1640 1 -- 172.21.15.42:0/2434842272 shutdown_connections 2024-08-30T08:59:54.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.879+0000 7f1b3cee1640 1 --2- 172.21.15.42:0/2434842272 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1b38109b90 0x7f1b381103c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:54.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.879+0000 7f1b3cee1640 1 --2- 172.21.15.42:0/2434842272 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1b38109240 0x7f1b38109650 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:54.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.879+0000 7f1b3cee1640 1 --2- 172.21.15.42:0/2434842272 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1b38103900 0x7f1b38103d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:54.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.879+0000 7f1b3cee1640 1 -- 172.21.15.42:0/2434842272 >> 172.21.15.42:0/2434842272 conn(0x7f1b3807b5f0 msgr2=0x7f1b3807ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:59:54.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.879+0000 7f1b3cee1640 1 -- 172.21.15.42:0/2434842272 shutdown_connections 2024-08-30T08:59:54.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.879+0000 7f1b3cee1640 1 -- 172.21.15.42:0/2434842272 wait complete. 2024-08-30T08:59:54.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.880+0000 7f1b3cee1640 1 Processor -- start 2024-08-30T08:59:54.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.880+0000 7f1b3cee1640 1 -- start start 2024-08-30T08:59:54.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.880+0000 7f1b3cee1640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1b38103900 0x7f1b3819a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:54.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.880+0000 7f1b3cee1640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1b38109240 0x7f1b3819a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:54.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.880+0000 7f1b3cee1640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1b38109b90 0x7f1b3819fa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:54.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.880+0000 7f1b3cee1640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f1b38113850 con 0x7f1b38109240 2024-08-30T08:59:54.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.880+0000 7f1b3cee1640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f1b381136d0 con 0x7f1b38109b90 2024-08-30T08:59:54.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.880+0000 7f1b3cee1640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f1b381139d0 con 0x7f1b38103900 2024-08-30T08:59:54.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.880+0000 7f1b35d9b640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1b38109240 0x7f1b3819a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:54.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.880+0000 7f1b35d9b640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1b38109240 0x7f1b3819a7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:45354/0 (socket says 172.21.15.42:45354) 2024-08-30T08:59:54.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.880+0000 7f1b35d9b640 1 -- 172.21.15.42:0/2495012568 learned_addr learned my addr 172.21.15.42:0/2495012568 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T08:59:54.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.880+0000 7f1b3659c640 1 --2- 172.21.15.42:0/2495012568 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1b38103900 0x7f1b3819a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:54.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.880+0000 7f1b36d9d640 1 --2- 172.21.15.42:0/2495012568 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1b38109b90 0x7f1b3819fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:54.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.881+0000 7f1b35d9b640 1 -- 172.21.15.42:0/2495012568 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1b38103900 msgr2=0x7f1b3819a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:54.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.881+0000 7f1b35d9b640 1 --2- 172.21.15.42:0/2495012568 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1b38103900 0x7f1b3819a220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:54.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.881+0000 7f1b35d9b640 1 -- 172.21.15.42:0/2495012568 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1b38109b90 msgr2=0x7f1b3819fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:54.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.881+0000 7f1b35d9b640 1 --2- 172.21.15.42:0/2495012568 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1b38109b90 0x7f1b3819fa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:54.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.881+0000 7f1b35d9b640 1 -- 172.21.15.42:0/2495012568 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b381a0190 con 0x7f1b38109240 2024-08-30T08:59:54.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.881+0000 7f1b36d9d640 1 --2- 172.21.15.42:0/2495012568 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1b38109b90 0x7f1b3819fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-30T08:59:54.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.881+0000 7f1b35d9b640 1 --2- 172.21.15.42:0/2495012568 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1b38109240 0x7f1b3819a7c0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f1b2c02a930 tx=0x7f1b2c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:59:54.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.881+0000 7f1b277fe640 1 -- 172.21.15.42:0/2495012568 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b2c03f030 con 0x7f1b38109240 2024-08-30T08:59:54.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.881+0000 7f1b3cee1640 1 -- 172.21.15.42:0/2495012568 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b381a0420 con 0x7f1b38109240 2024-08-30T08:59:54.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.881+0000 7f1b277fe640 1 -- 172.21.15.42:0/2495012568 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b2c003aa0 con 0x7f1b38109240 2024-08-30T08:59:54.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.881+0000 7f1b277fe640 1 -- 172.21.15.42:0/2495012568 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b2c035070 con 0x7f1b38109240 2024-08-30T08:59:54.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.881+0000 7f1b3cee1640 1 -- 172.21.15.42:0/2495012568 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b381a6be0 con 0x7f1b38109240 2024-08-30T08:59:54.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.884+0000 7f1b43456740 1 Processor -- start 2024-08-30T08:59:54.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.884+0000 7f1b43456740 1 -- start start 2024-08-30T08:59:54.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.885+0000 7f1b43456740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556b1d2d3f60 0x556b1d6797b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:54.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.885+0000 7f1b43456740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556b1d2ab0a0 0x556b1d679cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:54.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.885+0000 7f1b43456740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556b1d2b3280 0x556b1d67a230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:54.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.885+0000 7f1b43456740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x556b1d5f03f0 con 0x556b1d2b3280 2024-08-30T08:59:54.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.885+0000 7f1b43456740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x556b1d5df980 con 0x556b1d2d3f60 2024-08-30T08:59:54.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.885+0000 7f1b43456740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x556b1d5e3930 con 0x556b1d2ab0a0 2024-08-30T08:59:54.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.885+0000 7f1b277fe640 1 -- 172.21.15.42:0/2495012568 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) ==== 152628+0+0 (secure 0 0 0) 0x7f1b2c042030 con 0x7f1b38109240 2024-08-30T08:59:54.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.885+0000 7f1b35d9b640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556b1d2b3280 0x556b1d67a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:54.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.885+0000 7f1b36d9d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556b1d2d3f60 0x556b1d6797b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:54.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.885+0000 7f1b36d9d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556b1d2d3f60 0x556b1d6797b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:40030/0 (socket says 172.21.15.42:40030) 2024-08-30T08:59:54.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.885+0000 7f1b36d9d640 1 -- 172.21.15.42:0/3294835974 learned_addr learned my addr 172.21.15.42:0/3294835974 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T08:59:54.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.885+0000 7f1b3659c640 1 --2- 172.21.15.42:0/3294835974 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556b1d2ab0a0 0x556b1d679cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:54.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.885+0000 7f1b36d9d640 1 -- 172.21.15.42:0/3294835974 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556b1d2ab0a0 msgr2=0x556b1d679cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T08:59:54.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.885+0000 7f1b36d9d640 1 --2- 172.21.15.42:0/3294835974 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556b1d2ab0a0 0x556b1d679cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:54.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.885+0000 7f1b36d9d640 1 -- 172.21.15.42:0/3294835974 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556b1d2b3280 msgr2=0x556b1d67a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T08:59:54.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.885+0000 7f1b36d9d640 1 --2- 172.21.15.42:0/3294835974 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556b1d2b3280 0x556b1d67a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:54.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.885+0000 7f1b36d9d640 1 -- 172.21.15.42:0/3294835974 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b1d33f540 con 0x556b1d2d3f60 2024-08-30T08:59:54.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.885+0000 7f1b35d9b640 1 --2- 172.21.15.42:0/3294835974 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556b1d2b3280 0x556b1d67a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-30T08:59:54.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.886+0000 7f1b277fe640 1 --2- 172.21.15.42:0/2495012568 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f1b200b4bc0 0x7f1b200b70d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:54.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.886+0000 7f1b36d9d640 1 --2- 172.21.15.42:0/3294835974 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556b1d2d3f60 0x556b1d6797b0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f1b30009fb0 tx=0x7f1b30010800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ad16bf502c7bfbdc server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:59:54.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.886+0000 7f1b3659c640 1 --2- 172.21.15.42:0/2495012568 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f1b200b4bc0 0x7f1b200b70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:54.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.886+0000 7f1b277fe640 1 -- 172.21.15.42:0/2495012568 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f1b2c105b30 con 0x7f1b38109240 2024-08-30T08:59:54.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.886+0000 7f1b24ff9640 1 -- 172.21.15.42:0/3294835974 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b30016070 con 0x556b1d2d3f60 2024-08-30T08:59:54.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.886+0000 7f1b24ff9640 1 -- 172.21.15.42:0/3294835974 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b30012bc0 con 0x556b1d2d3f60 2024-08-30T08:59:54.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.886+0000 7f1b24ff9640 1 -- 172.21.15.42:0/3294835974 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b30011ba0 con 0x556b1d2d3f60 2024-08-30T08:59:54.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.887+0000 7f1b43456740 1 -- 172.21.15.42:0/3294835974 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556b1d2d3f60 msgr2=0x556b1d6797b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:54.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.887+0000 7f1b3659c640 1 --2- 172.21.15.42:0/2495012568 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f1b200b4bc0 0x7f1b200b70d0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x556b1d678430 tx=0x7f1b2800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:59:54.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.887+0000 7f1b43456740 1 --2- 172.21.15.42:0/3294835974 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556b1d2d3f60 0x556b1d6797b0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f1b30009fb0 tx=0x7f1b30010800 comp rx=0 tx=0).stop 2024-08-30T08:59:54.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.887+0000 7f1b43456740 1 -- 172.21.15.42:0/3294835974 shutdown_connections 2024-08-30T08:59:54.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.887+0000 7f1b43456740 1 --2- 172.21.15.42:0/3294835974 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556b1d2ab0a0 0x556b1d679cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:54.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.887+0000 7f1b43456740 1 --2- 172.21.15.42:0/3294835974 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556b1d2b3280 0x556b1d67a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:54.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.887+0000 7f1b43456740 1 --2- 172.21.15.42:0/3294835974 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556b1d2d3f60 0x556b1d6797b0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:54.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.887+0000 7f1b43456740 1 -- 172.21.15.42:0/3294835974 >> 172.21.15.42:0/3294835974 conn(0x556b1d2c8350 msgr2=0x556b1d6776a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:59:54.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.887+0000 7f1b43456740 1 -- 172.21.15.42:0/3294835974 shutdown_connections 2024-08-30T08:59:54.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.887+0000 7f1b43456740 1 -- 172.21.15.42:0/3294835974 wait complete. 2024-08-30T08:59:54.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.888+0000 7f1b43456740 1 -- start start 2024-08-30T08:59:54.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.888+0000 7f1b43456740 1 Processor -- start 2024-08-30T08:59:54.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.888+0000 7f1b43456740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556b1d2ab0a0 0x556b1d6892d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:54.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.888+0000 7f1b43456740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556b1d2b3280 0x556b1d689810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:54.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.888+0000 7f1b43456740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556b1d2d3f60 0x556b1d6955f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:54.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.888+0000 7f1b43456740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x556b1d4425f0 con 0x556b1d2b3280 2024-08-30T08:59:54.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.888+0000 7f1b43456740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x556b1d566600 con 0x556b1d2d3f60 2024-08-30T08:59:54.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.888+0000 7f1b43456740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x556b1d68c290 con 0x556b1d2ab0a0 2024-08-30T08:59:54.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.888+0000 7f1b43456740 10 client.0 _ll_register_callbacks cb 0x556b1d58caf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T08:59:54.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.888+0000 7f1b35d9b640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556b1d2b3280 0x556b1d689810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:54.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.888+0000 7f1b35d9b640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556b1d2b3280 0x556b1d689810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:45380/0 (socket says 172.21.15.42:45380) 2024-08-30T08:59:54.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.888+0000 7f1b35d9b640 1 -- 172.21.15.42:0/960440929 learned_addr learned my addr 172.21.15.42:0/960440929 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T08:59:54.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.888+0000 7f1b36d9d640 1 --2- 172.21.15.42:0/960440929 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556b1d2d3f60 0x556b1d6955f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:54.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.888+0000 7f1b3659c640 1 --2- 172.21.15.42:0/960440929 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556b1d2ab0a0 0x556b1d6892d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:54.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.889+0000 7f1b35d9b640 1 -- 172.21.15.42:0/960440929 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556b1d2ab0a0 msgr2=0x556b1d6892d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T08:59:54.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.889+0000 7f1b35d9b640 1 --2- 172.21.15.42:0/960440929 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556b1d2ab0a0 0x556b1d6892d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:54.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.889+0000 7f1b35d9b640 1 -- 172.21.15.42:0/960440929 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556b1d2d3f60 msgr2=0x556b1d6955f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T08:59:54.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.889+0000 7f1b35d9b640 1 --2- 172.21.15.42:0/960440929 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556b1d2d3f60 0x556b1d6955f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:54.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.889+0000 7f1b35d9b640 1 -- 172.21.15.42:0/960440929 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b1d59f7b0 con 0x556b1d2b3280 2024-08-30T08:59:54.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.889+0000 7f1b3659c640 1 --2- 172.21.15.42:0/960440929 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556b1d2ab0a0 0x556b1d6892d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-30T08:59:54.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.889+0000 7f1b35d9b640 1 --2- 172.21.15.42:0/960440929 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556b1d2b3280 0x556b1d689810 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f1b2c031440 tx=0x7f1b2c031a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf2f072174419931 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:59:54.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.889+0000 7f1b1dffb640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-08-30T08:59:54.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.889+0000 7f1b1dffb640 1 -- 172.21.15.42:0/960440929 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b2c049070 con 0x556b1d2b3280 2024-08-30T08:59:54.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.889+0000 7f1b1dffb640 1 -- 172.21.15.42:0/960440929 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f1b2c0c8300 con 0x556b1d2b3280 2024-08-30T08:59:54.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.889+0000 7f1b43456740 1 -- 172.21.15.42:0/960440929 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1b30012bc0 con 0x556b1d2b3280 2024-08-30T08:59:54.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.889+0000 7f1b1dffb640 1 -- 172.21.15.42:0/960440929 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f1b2c0cb9c0 con 0x556b1d2b3280 2024-08-30T08:59:54.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.889+0000 7f1b43456740 10 client.15237 fetch_fsmap learned FSMap version 26 2024-08-30T08:59:54.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.889+0000 7f1b43456740 1 -- 172.21.15.42:0/960440929 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x556b1d698040 con 0x556b1d2b3280 2024-08-30T08:59:54.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.889+0000 7f1b1dffb640 1 -- 172.21.15.42:0/960440929 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b30012bc0 con 0x556b1d2b3280 2024-08-30T08:59:54.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.890+0000 7f1b1dffb640 1 -- 172.21.15.42:0/960440929 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f1b2c0cc2d0 con 0x556b1d2b3280 2024-08-30T08:59:54.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.890+0000 7f1b43456740 10 client.15237 fetch_fsmap finished waiting for FSMap version 26 2024-08-30T08:59:54.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.890+0000 7f1b43456740 10 client.15237 resolve_mds: resolved name 'h' to daemon mds.h 2024-08-30T08:59:54.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.890+0000 7f1b43456740 20 client.15237 populate_metadata read hostname 'smithi042' 2024-08-30T08:59:54.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.890+0000 7f1b43456740 1 --2- 172.21.15.42:0/960440929 >> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] conn(0x556b1d2928e0 0x556b1d69bb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:54.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.890+0000 7f1b43456740 4 client.15237 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-30T08:59:54.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.890+0000 7f1b43456740 1 -- 172.21.15.42:0/960440929 --> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556b1d5f2340 con 0x556b1d2928e0 2024-08-30T08:59:54.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.890+0000 7f1b3659c640 1 --2- 172.21.15.42:0/960440929 >> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] conn(0x556b1d2928e0 0x556b1d69bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:54.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.891+0000 7f1b3659c640 1 --2- 172.21.15.42:0/960440929 >> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] conn(0x556b1d2928e0 0x556b1d69bb70 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4abdd4f63edac45f server_cookie=bd6ab8b6dd61d962 in_seq=0 out_seq=0 2024-08-30T08:59:54.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.891+0000 7f1b1dffb640 10 client.15237 ms_handle_connect on v2:172.21.15.167:6836/3011837856 2024-08-30T08:59:54.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.891+0000 7f1b1dffb640 1 -- 172.21.15.42:0/960440929 <== mds.2 v2:172.21.15.167:6836/3011837856 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x556b1d5f2340 con 0x556b1d2928e0 2024-08-30T08:59:54.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.891+0000 7f1b1dffb640 10 client.15237 handle_command_reply: tid=0 2024-08-30T08:59:54.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.892+0000 7f1b43456740 1 client.15237 shutdown 2024-08-30T08:59:54.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.892+0000 7f1b43456740 1 -- 172.21.15.42:0/960440929 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556b1d2b3280 msgr2=0x556b1d689810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:54.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.892+0000 7f1b43456740 1 --2- 172.21.15.42:0/960440929 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556b1d2b3280 0x556b1d689810 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f1b2c031440 tx=0x7f1b2c031a10 comp rx=0 tx=0).stop 2024-08-30T08:59:54.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.892+0000 7f1b43456740 1 -- 172.21.15.42:0/960440929 shutdown_connections 2024-08-30T08:59:54.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.892+0000 7f1b43456740 1 --2- 172.21.15.42:0/960440929 >> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] conn(0x556b1d2928e0 0x556b1d69bb70 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:54.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.892+0000 7f1b43456740 1 --2- 172.21.15.42:0/960440929 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556b1d2d3f60 0x556b1d6955f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:54.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.892+0000 7f1b43456740 1 --2- 172.21.15.42:0/960440929 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556b1d2b3280 0x556b1d689810 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:54.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.892+0000 7f1b43456740 1 --2- 172.21.15.42:0/960440929 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556b1d2ab0a0 0x556b1d6892d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:54.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.892+0000 7f1b43456740 1 -- 172.21.15.42:0/960440929 >> 172.21.15.42:0/960440929 conn(0x556b1d2c8350 msgr2=0x556b1d67b1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:59:54.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.892+0000 7f1b1dffb640 0 client.15237 ms_handle_reset on v2:172.21.15.167:6836/3011837856 2024-08-30T08:59:54.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.892+0000 7f1b43456740 1 -- 172.21.15.42:0/960440929 shutdown_connections 2024-08-30T08:59:54.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.892+0000 7f1b43456740 1 -- 172.21.15.42:0/960440929 wait complete. 2024-08-30T08:59:54.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.892+0000 7f1b43456740 20 client.15237 trim_cache size 0 max 16384 2024-08-30T08:59:54.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.910+0000 7f1b43456740 1 Processor -- start 2024-08-30T08:59:54.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.910+0000 7f1b43456740 1 -- start start 2024-08-30T08:59:54.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.910+0000 7f1b43456740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556b1d2ab0a0 0x556b1d6844b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:54.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.910+0000 7f1b43456740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556b1d2b3280 0x556b1d6849f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:54.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.910+0000 7f1b43456740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556b1d2d3f60 0x556b1d687b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:54.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.910+0000 7f1b43456740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x556b1d699eb0 con 0x556b1d2ab0a0 2024-08-30T08:59:54.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.910+0000 7f1b43456740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x556b1d69a580 con 0x556b1d2d3f60 2024-08-30T08:59:54.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.910+0000 7f1b43456740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x556b1d69ac50 con 0x556b1d2b3280 2024-08-30T08:59:54.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.911+0000 7f1b3659c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556b1d2ab0a0 0x556b1d6844b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:54.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.911+0000 7f1b3659c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556b1d2ab0a0 0x556b1d6844b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:45382/0 (socket says 172.21.15.42:45382) 2024-08-30T08:59:54.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.911+0000 7f1b3659c640 1 -- 172.21.15.42:0/632827980 learned_addr learned my addr 172.21.15.42:0/632827980 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T08:59:54.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.911+0000 7f1b35d9b640 1 --2- 172.21.15.42:0/632827980 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556b1d2b3280 0x556b1d6849f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:54.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.911+0000 7f1b36d9d640 1 --2- 172.21.15.42:0/632827980 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556b1d2d3f60 0x556b1d687b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:54.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.911+0000 7f1b3659c640 1 -- 172.21.15.42:0/632827980 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556b1d2b3280 msgr2=0x556b1d6849f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T08:59:54.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.911+0000 7f1b3659c640 1 --2- 172.21.15.42:0/632827980 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556b1d2b3280 0x556b1d6849f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:54.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.911+0000 7f1b3659c640 1 -- 172.21.15.42:0/632827980 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556b1d2d3f60 msgr2=0x556b1d687b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T08:59:54.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.911+0000 7f1b3659c640 1 --2- 172.21.15.42:0/632827980 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556b1d2d3f60 0x556b1d687b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:54.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.911+0000 7f1b3659c640 1 -- 172.21.15.42:0/632827980 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b1d67c570 con 0x556b1d2ab0a0 2024-08-30T08:59:54.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.911+0000 7f1b3659c640 1 --2- 172.21.15.42:0/632827980 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556b1d2ab0a0 0x556b1d6844b0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f1b2800bed0 tx=0x7f1b28016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9d718f4c9f6683b server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:59:54.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.911+0000 7f1b1ffff640 1 -- 172.21.15.42:0/632827980 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b28003d80 con 0x556b1d2ab0a0 2024-08-30T08:59:54.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.911+0000 7f1b1ffff640 1 -- 172.21.15.42:0/632827980 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b280200d0 con 0x556b1d2ab0a0 2024-08-30T08:59:54.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.911+0000 7f1b1ffff640 1 -- 172.21.15.42:0/632827980 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b28029da0 con 0x556b1d2ab0a0 2024-08-30T08:59:54.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.912+0000 7f1b43456740 1 -- 172.21.15.42:0/632827980 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556b1d2ab0a0 msgr2=0x556b1d6844b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:54.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.912+0000 7f1b43456740 1 --2- 172.21.15.42:0/632827980 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556b1d2ab0a0 0x556b1d6844b0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f1b2800bed0 tx=0x7f1b28016000 comp rx=0 tx=0).stop 2024-08-30T08:59:54.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.912+0000 7f1b43456740 1 -- 172.21.15.42:0/632827980 shutdown_connections 2024-08-30T08:59:54.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.912+0000 7f1b43456740 1 --2- 172.21.15.42:0/632827980 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556b1d2b3280 0x556b1d6849f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:54.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.912+0000 7f1b43456740 1 --2- 172.21.15.42:0/632827980 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556b1d2d3f60 0x556b1d687b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:54.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.912+0000 7f1b43456740 1 --2- 172.21.15.42:0/632827980 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556b1d2ab0a0 0x556b1d6844b0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:54.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.912+0000 7f1b43456740 1 -- 172.21.15.42:0/632827980 >> 172.21.15.42:0/632827980 conn(0x556b1d2c8350 msgr2=0x556b1d67e820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:59:54.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.912+0000 7f1b43456740 1 -- 172.21.15.42:0/632827980 shutdown_connections 2024-08-30T08:59:54.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.912+0000 7f1b43456740 1 -- 172.21.15.42:0/632827980 wait complete. 2024-08-30T08:59:54.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.913+0000 7f1b43456740 1 -- start start 2024-08-30T08:59:54.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.913+0000 7f1b43456740 1 Processor -- start 2024-08-30T08:59:54.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.913+0000 7f1b43456740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556b1d2ab0a0 0x556b1d69f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:54.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.913+0000 7f1b43456740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556b1d2b3280 0x556b1d69fed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:54.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.913+0000 7f1b43456740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556b1d2d3f60 0x556b1d6a3d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:54.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.913+0000 7f1b43456740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x556b1d69b9f0 con 0x556b1d2d3f60 2024-08-30T08:59:54.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.913+0000 7f1b43456740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x556b1d69b320 con 0x556b1d2b3280 2024-08-30T08:59:54.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.913+0000 7f1b43456740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x556b1d69c0c0 con 0x556b1d2ab0a0 2024-08-30T08:59:54.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.913+0000 7f1b43456740 10 client.0 _ll_register_callbacks cb 0x7f1b2c009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T08:59:54.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.913+0000 7f1b36d9d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556b1d2d3f60 0x556b1d6a3d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:54.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.913+0000 7f1b36d9d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556b1d2d3f60 0x556b1d6a3d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:45394/0 (socket says 172.21.15.42:45394) 2024-08-30T08:59:54.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.913+0000 7f1b36d9d640 1 -- 172.21.15.42:0/4126355780 learned_addr learned my addr 172.21.15.42:0/4126355780 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T08:59:54.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.913+0000 7f1b35d9b640 1 --2- 172.21.15.42:0/4126355780 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556b1d2b3280 0x556b1d69fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:54.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.913+0000 7f1b3659c640 1 --2- 172.21.15.42:0/4126355780 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556b1d2ab0a0 0x556b1d69f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:54.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.913+0000 7f1b36d9d640 1 -- 172.21.15.42:0/4126355780 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556b1d2ab0a0 msgr2=0x556b1d69f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T08:59:54.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.913+0000 7f1b36d9d640 1 --2- 172.21.15.42:0/4126355780 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556b1d2ab0a0 0x556b1d69f910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:54.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.913+0000 7f1b36d9d640 1 -- 172.21.15.42:0/4126355780 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556b1d2b3280 msgr2=0x556b1d69fed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T08:59:54.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.913+0000 7f1b36d9d640 1 --2- 172.21.15.42:0/4126355780 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556b1d2b3280 0x556b1d69fed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:54.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.913+0000 7f1b36d9d640 1 -- 172.21.15.42:0/4126355780 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b1d6a43e0 con 0x556b1d2d3f60 2024-08-30T08:59:54.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.914+0000 7f1b36d9d640 1 --2- 172.21.15.42:0/4126355780 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556b1d2d3f60 0x556b1d6a3d70 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f1b30010d00 tx=0x7f1b30011610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ceb4bd59bd26ece3 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:59:54.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.914+0000 7f1b1dffb640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-08-30T08:59:54.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.914+0000 7f1b1dffb640 1 -- 172.21.15.42:0/4126355780 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b3002c060 con 0x556b1d2d3f60 2024-08-30T08:59:54.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.914+0000 7f1b1dffb640 1 -- 172.21.15.42:0/4126355780 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f1b300118b0 con 0x556b1d2d3f60 2024-08-30T08:59:54.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.914+0000 7f1b43456740 1 -- 172.21.15.42:0/4126355780 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1b280200d0 con 0x556b1d2d3f60 2024-08-30T08:59:54.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.914+0000 7f1b1dffb640 1 -- 172.21.15.42:0/4126355780 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f1b30033050 con 0x556b1d2d3f60 2024-08-30T08:59:54.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.914+0000 7f1b43456740 10 client.15243 fetch_fsmap learned FSMap version 26 2024-08-30T08:59:54.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.914+0000 7f1b43456740 1 -- 172.21.15.42:0/4126355780 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x556b1d706b60 con 0x556b1d2d3f60 2024-08-30T08:59:54.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.914+0000 7f1b1dffb640 1 -- 172.21.15.42:0/4126355780 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b280200d0 con 0x556b1d2d3f60 2024-08-30T08:59:54.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.914+0000 7f1b1dffb640 1 -- 172.21.15.42:0/4126355780 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f1b3000f040 con 0x556b1d2d3f60 2024-08-30T08:59:54.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.914+0000 7f1b43456740 10 client.15243 fetch_fsmap finished waiting for FSMap version 26 2024-08-30T08:59:54.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.914+0000 7f1b43456740 10 client.15243 resolve_mds: resolved name 'h' to daemon mds.h 2024-08-30T08:59:54.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.914+0000 7f1b43456740 20 client.15243 populate_metadata read hostname 'smithi042' 2024-08-30T08:59:54.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.915+0000 7f1b43456740 1 --2- 172.21.15.42:0/4126355780 >> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] conn(0x556b1d267c90 0x556b1d70a130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:54.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.915+0000 7f1b43456740 4 client.15243 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "status"}] 2024-08-30T08:59:54.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.915+0000 7f1b43456740 1 -- 172.21.15.42:0/4126355780 --> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] -- command(tid 0: {"prefix": "status"}) -- 0x556b1d5edde0 con 0x556b1d267c90 2024-08-30T08:59:54.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.915+0000 7f1b3659c640 1 --2- 172.21.15.42:0/4126355780 >> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] conn(0x556b1d267c90 0x556b1d70a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:54.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.915+0000 7f1b1dffb640 10 client.15243 ms_handle_connect on v2:172.21.15.167:6836/3011837856 2024-08-30T08:59:54.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.915+0000 7f1b3659c640 1 --2- 172.21.15.42:0/4126355780 >> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] conn(0x556b1d267c90 0x556b1d70a130 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1bfbaa275ed1c446 server_cookie=a6f4c07e81431883 in_seq=0 out_seq=0 2024-08-30T08:59:54.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.916+0000 7f1b1dffb640 1 -- 172.21.15.42:0/4126355780 <== mds.2 v2:172.21.15.167:6836/3011837856 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x556b1d5edde0 con 0x556b1d267c90 2024-08-30T08:59:54.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.916+0000 7f1b1dffb640 10 client.15243 handle_command_reply: tid=0 2024-08-30T08:59:54.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.916+0000 7f1b43456740 1 client.15243 shutdown 2024-08-30T08:59:54.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.916+0000 7f1b43456740 1 -- 172.21.15.42:0/4126355780 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556b1d2d3f60 msgr2=0x556b1d6a3d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:54.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.916+0000 7f1b43456740 1 --2- 172.21.15.42:0/4126355780 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556b1d2d3f60 0x556b1d6a3d70 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f1b30010d00 tx=0x7f1b30011610 comp rx=0 tx=0).stop 2024-08-30T08:59:54.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.916+0000 7f1b43456740 1 -- 172.21.15.42:0/4126355780 shutdown_connections 2024-08-30T08:59:54.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.916+0000 7f1b43456740 1 --2- 172.21.15.42:0/4126355780 >> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] conn(0x556b1d267c90 0x556b1d70a130 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:54.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.916+0000 7f1b43456740 1 --2- 172.21.15.42:0/4126355780 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556b1d2d3f60 0x556b1d6a3d70 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:54.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.916+0000 7f1b1dffb640 0 client.15243 ms_handle_reset on v2:172.21.15.167:6836/3011837856 2024-08-30T08:59:54.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.916+0000 7f1b43456740 1 --2- 172.21.15.42:0/4126355780 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556b1d2b3280 0x556b1d69fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:54.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.916+0000 7f1b43456740 1 --2- 172.21.15.42:0/4126355780 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556b1d2ab0a0 0x556b1d69f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:54.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.916+0000 7f1b43456740 1 -- 172.21.15.42:0/4126355780 >> 172.21.15.42:0/4126355780 conn(0x556b1d2c8350 msgr2=0x556b1d6844b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:59:54.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.916+0000 7f1b43456740 1 -- 172.21.15.42:0/4126355780 shutdown_connections 2024-08-30T08:59:54.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.916+0000 7f1b43456740 1 -- 172.21.15.42:0/4126355780 wait complete. 2024-08-30T08:59:54.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.916+0000 7f1b43456740 20 client.15243 trim_cache size 0 max 16384 2024-08-30T08:59:54.923 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-08-30T08:59:54.923 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_fsid": "b0895c1a-66ac-11ef-bcd4-c7b262605968", 2024-08-30T08:59:54.923 INFO:teuthology.orchestra.run.smithi042.stdout: "whoami": 2, 2024-08-30T08:59:54.923 INFO:teuthology.orchestra.run.smithi042.stdout: "id": 24599, 2024-08-30T08:59:54.923 INFO:teuthology.orchestra.run.smithi042.stdout: "want_state": "up:active", 2024-08-30T08:59:54.923 INFO:teuthology.orchestra.run.smithi042.stdout: "state": "up:active", 2024-08-30T08:59:54.923 INFO:teuthology.orchestra.run.smithi042.stdout: "fs_name": "cephfs", 2024-08-30T08:59:54.923 INFO:teuthology.orchestra.run.smithi042.stdout: "rank_uptime": 69.353440923999997, 2024-08-30T08:59:54.923 INFO:teuthology.orchestra.run.smithi042.stdout: "mdsmap_epoch": 26, 2024-08-30T08:59:54.924 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_epoch": 83, 2024-08-30T08:59:54.924 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_epoch_barrier": 81, 2024-08-30T08:59:54.924 INFO:teuthology.orchestra.run.smithi042.stdout: "uptime": 134.25848678 2024-08-30T08:59:54.924 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-08-30T08:59:54.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.916+0000 7f1b3cee1640 1 -- 172.21.15.42:0/2495012568 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f1b200b4bc0 msgr2=0x7f1b200b70d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:54.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.916+0000 7f1b3cee1640 1 --2- 172.21.15.42:0/2495012568 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f1b200b4bc0 0x7f1b200b70d0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x556b1d678430 tx=0x7f1b2800b390 comp rx=0 tx=0).stop 2024-08-30T08:59:54.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.916+0000 7f1b3cee1640 1 -- 172.21.15.42:0/2495012568 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1b38109240 msgr2=0x7f1b3819a7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:54.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.916+0000 7f1b3cee1640 1 --2- 172.21.15.42:0/2495012568 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1b38109240 0x7f1b3819a7c0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f1b2c02a930 tx=0x7f1b2c004270 comp rx=0 tx=0).stop 2024-08-30T08:59:54.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.917+0000 7f1b3cee1640 1 -- 172.21.15.42:0/2495012568 shutdown_connections 2024-08-30T08:59:54.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.917+0000 7f1b3cee1640 1 --2- 172.21.15.42:0/2495012568 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f1b200b4bc0 0x7f1b200b70d0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:54.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.917+0000 7f1b3cee1640 1 --2- 172.21.15.42:0/2495012568 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1b38109b90 0x7f1b3819fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:54.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.917+0000 7f1b3cee1640 1 --2- 172.21.15.42:0/2495012568 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1b38109240 0x7f1b3819a7c0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:54.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.917+0000 7f1b3cee1640 1 --2- 172.21.15.42:0/2495012568 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1b38103900 0x7f1b3819a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:54.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.917+0000 7f1b3cee1640 1 -- 172.21.15.42:0/2495012568 >> 172.21.15.42:0/2495012568 conn(0x7f1b3807b5f0 msgr2=0x7f1b381052b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:59:54.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.917+0000 7f1b3cee1640 1 -- 172.21.15.42:0/2495012568 shutdown_connections 2024-08-30T08:59:54.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:54.917+0000 7f1b3cee1640 1 -- 172.21.15.42:0/2495012568 wait complete. 2024-08-30T08:59:54.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:59:54 smithi190 ceph-mon[22001]: pgmap v347: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s; 0 B/s, 1 keys/s, 0 objects/s recovering 2024-08-30T08:59:55.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:59:54 smithi167 ceph-mon[22063]: pgmap v347: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s; 0 B/s, 1 keys/s, 0 objects/s recovering 2024-08-30T08:59:55.744 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph tell mds.i status 2024-08-30T08:59:56.001 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T08:59:56.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.469+0000 7f156b55c640 1 -- 172.21.15.42:0/896110851 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1564103950 msgr2=0x7f1564103d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:56.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.469+0000 7f156b55c640 1 --2- 172.21.15.42:0/896110851 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1564103950 0x7f1564103d60 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f1558009900 tx=0x7f155802a410 comp rx=0 tx=0).stop 2024-08-30T08:59:56.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.469+0000 7f156b55c640 1 -- 172.21.15.42:0/896110851 shutdown_connections 2024-08-30T08:59:56.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.469+0000 7f156b55c640 1 --2- 172.21.15.42:0/896110851 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1564109290 0x7f15641096a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:56.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.469+0000 7f156b55c640 1 --2- 172.21.15.42:0/896110851 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1564109be0 0x7f1564110410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:56.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.469+0000 7f156b55c640 1 --2- 172.21.15.42:0/896110851 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1564103950 0x7f1564103d60 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:56.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.469+0000 7f156b55c640 1 -- 172.21.15.42:0/896110851 >> 172.21.15.42:0/896110851 conn(0x7f15640ff3a0 msgr2=0x7f15641017f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:59:56.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.469+0000 7f156b55c640 1 -- 172.21.15.42:0/896110851 shutdown_connections 2024-08-30T08:59:56.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.469+0000 7f156b55c640 1 -- 172.21.15.42:0/896110851 wait complete. 2024-08-30T08:59:56.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.470+0000 7f156b55c640 1 Processor -- start 2024-08-30T08:59:56.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.470+0000 7f156b55c640 1 -- start start 2024-08-30T08:59:56.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.470+0000 7f156b55c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1564103950 0x7f156419a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:56.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.470+0000 7f156b55c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1564109290 0x7f156419a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:56.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.470+0000 7f156b55c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1564109be0 0x7f156419fad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:56.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.470+0000 7f156b55c640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f15641138a0 con 0x7f1564109be0 2024-08-30T08:59:56.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.470+0000 7f156b55c640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f1564113720 con 0x7f1564103950 2024-08-30T08:59:56.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.470+0000 7f156b55c640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f1564113a20 con 0x7f1564109290 2024-08-30T08:59:56.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.470+0000 7f1569af9640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1564109be0 0x7f156419fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:56.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.470+0000 7f1569af9640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1564109be0 0x7f156419fad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:45408/0 (socket says 172.21.15.42:45408) 2024-08-30T08:59:56.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.471+0000 7f1569af9640 1 -- 172.21.15.42:0/1008138834 learned_addr learned my addr 172.21.15.42:0/1008138834 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T08:59:56.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.471+0000 7f15692f8640 1 --2- 172.21.15.42:0/1008138834 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1564103950 0x7f156419a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:56.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.471+0000 7f1568af7640 1 --2- 172.21.15.42:0/1008138834 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1564109290 0x7f156419a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:56.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.471+0000 7f1569af9640 1 -- 172.21.15.42:0/1008138834 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1564109290 msgr2=0x7f156419a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:56.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.471+0000 7f1569af9640 1 --2- 172.21.15.42:0/1008138834 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1564109290 0x7f156419a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:56.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.471+0000 7f1569af9640 1 -- 172.21.15.42:0/1008138834 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1564103950 msgr2=0x7f156419a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:56.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.471+0000 7f1569af9640 1 --2- 172.21.15.42:0/1008138834 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1564103950 0x7f156419a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:56.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.471+0000 7f1569af9640 1 -- 172.21.15.42:0/1008138834 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15641a0200 con 0x7f1564109be0 2024-08-30T08:59:56.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.471+0000 7f1568af7640 1 --2- 172.21.15.42:0/1008138834 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1564109290 0x7f156419a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-30T08:59:56.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.471+0000 7f1569af9640 1 --2- 172.21.15.42:0/1008138834 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1564109be0 0x7f156419fad0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f155c009fd0 tx=0x7f155c00b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:59:56.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.471+0000 7f15567fc640 1 -- 172.21.15.42:0/1008138834 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f155c017070 con 0x7f1564109be0 2024-08-30T08:59:56.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.471+0000 7f15567fc640 1 -- 172.21.15.42:0/1008138834 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f155c003e20 con 0x7f1564109be0 2024-08-30T08:59:56.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.471+0000 7f156b55c640 1 -- 172.21.15.42:0/1008138834 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15641a04f0 con 0x7f1564109be0 2024-08-30T08:59:56.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.471+0000 7f15567fc640 1 -- 172.21.15.42:0/1008138834 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f155c010460 con 0x7f1564109be0 2024-08-30T08:59:56.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.471+0000 7f156b55c640 1 -- 172.21.15.42:0/1008138834 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15641a6c50 con 0x7f1564109be0 2024-08-30T08:59:56.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.474+0000 7f15567fc640 1 -- 172.21.15.42:0/1008138834 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) ==== 152628+0+0 (secure 0 0 0) 0x7f155c003970 con 0x7f1564109be0 2024-08-30T08:59:56.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.474+0000 7f1571ad1740 1 Processor -- start 2024-08-30T08:59:56.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.474+0000 7f1571ad1740 1 -- start start 2024-08-30T08:59:56.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.475+0000 7f1571ad1740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56212d55af60 0x56212d9007b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:56.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.475+0000 7f1571ad1740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56212d5320a0 0x56212d900cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:56.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.475+0000 7f1571ad1740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56212d53a280 0x56212d901230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:56.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.475+0000 7f1571ad1740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x56212d8773f0 con 0x56212d53a280 2024-08-30T08:59:56.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.475+0000 7f1571ad1740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x56212d866980 con 0x56212d55af60 2024-08-30T08:59:56.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.475+0000 7f1571ad1740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x56212d86a930 con 0x56212d5320a0 2024-08-30T08:59:56.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.475+0000 7f1568af7640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56212d53a280 0x56212d901230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:56.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.475+0000 7f1568af7640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56212d53a280 0x56212d901230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:45422/0 (socket says 172.21.15.42:45422) 2024-08-30T08:59:56.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.475+0000 7f1568af7640 1 -- 172.21.15.42:0/2013609840 learned_addr learned my addr 172.21.15.42:0/2013609840 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T08:59:56.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.475+0000 7f15692f8640 1 --2- 172.21.15.42:0/2013609840 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56212d5320a0 0x56212d900cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:56.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.475+0000 7f1568af7640 1 --2- 172.21.15.42:0/2013609840 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56212d55af60 0x56212d9007b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:56.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.475+0000 7f15567fc640 1 --2- 172.21.15.42:0/1008138834 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f15500b4bc0 0x7f15500b70d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:56.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.475+0000 7f1569af9640 1 --2- 172.21.15.42:0/1008138834 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f15500b4bc0 0x7f15500b70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:56.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.475+0000 7f1568af7640 1 -- 172.21.15.42:0/2013609840 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56212d5320a0 msgr2=0x56212d900cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T08:59:56.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.475+0000 7f1568af7640 1 --2- 172.21.15.42:0/2013609840 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56212d5320a0 0x56212d900cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:56.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.475+0000 7f1568af7640 1 -- 172.21.15.42:0/2013609840 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56212d55af60 msgr2=0x56212d9007b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T08:59:56.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.475+0000 7f1568af7640 1 --2- 172.21.15.42:0/2013609840 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56212d55af60 0x56212d9007b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:56.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.475+0000 7f1568af7640 1 -- 172.21.15.42:0/2013609840 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56212d5c6540 con 0x56212d53a280 2024-08-30T08:59:56.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.475+0000 7f15567fc640 1 -- 172.21.15.42:0/1008138834 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f155c0e53d0 con 0x7f1564109be0 2024-08-30T08:59:56.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.476+0000 7f1568af7640 1 --2- 172.21.15.42:0/2013609840 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56212d53a280 0x56212d901230 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f1560005fd0 tx=0x7f156000b460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1a43ecfc5a21011 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:59:56.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.476+0000 7f154f7fe640 1 -- 172.21.15.42:0/2013609840 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1560005220 con 0x56212d53a280 2024-08-30T08:59:56.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.476+0000 7f1569af9640 1 --2- 172.21.15.42:0/1008138834 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f15500b4bc0 0x7f15500b70d0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f155c00dfd0 tx=0x7f155c0c1040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:59:56.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.476+0000 7f154f7fe640 1 -- 172.21.15.42:0/2013609840 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1560013e20 con 0x56212d53a280 2024-08-30T08:59:56.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.476+0000 7f154f7fe640 1 -- 172.21.15.42:0/2013609840 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f156000e950 con 0x56212d53a280 2024-08-30T08:59:56.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.477+0000 7f1571ad1740 1 -- 172.21.15.42:0/2013609840 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56212d53a280 msgr2=0x56212d901230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:56.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.477+0000 7f1571ad1740 1 --2- 172.21.15.42:0/2013609840 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56212d53a280 0x56212d901230 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f1560005fd0 tx=0x7f156000b460 comp rx=0 tx=0).stop 2024-08-30T08:59:56.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.477+0000 7f1571ad1740 1 -- 172.21.15.42:0/2013609840 shutdown_connections 2024-08-30T08:59:56.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.477+0000 7f1571ad1740 1 --2- 172.21.15.42:0/2013609840 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56212d5320a0 0x56212d900cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:56.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.477+0000 7f1571ad1740 1 --2- 172.21.15.42:0/2013609840 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56212d53a280 0x56212d901230 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:56.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.477+0000 7f1571ad1740 1 --2- 172.21.15.42:0/2013609840 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56212d55af60 0x56212d9007b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:56.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.477+0000 7f1571ad1740 1 -- 172.21.15.42:0/2013609840 >> 172.21.15.42:0/2013609840 conn(0x56212d54f350 msgr2=0x56212d8fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:59:56.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.477+0000 7f1571ad1740 1 -- 172.21.15.42:0/2013609840 shutdown_connections 2024-08-30T08:59:56.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.477+0000 7f1571ad1740 1 -- 172.21.15.42:0/2013609840 wait complete. 2024-08-30T08:59:56.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.477+0000 7f1571ad1740 1 -- start start 2024-08-30T08:59:56.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.477+0000 7f1571ad1740 1 Processor -- start 2024-08-30T08:59:56.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.478+0000 7f1571ad1740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56212d5320a0 0x56212d90be40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:56.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.478+0000 7f1571ad1740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56212d53a280 0x56212d90c380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:56.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.478+0000 7f1571ad1740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56212d55af60 0x56212d918280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:56.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.478+0000 7f1571ad1740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x56212d6c95f0 con 0x56212d5320a0 2024-08-30T08:59:56.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.478+0000 7f1571ad1740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x56212d7ed600 con 0x56212d55af60 2024-08-30T08:59:56.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.478+0000 7f1571ad1740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x56212d90ef90 con 0x56212d53a280 2024-08-30T08:59:56.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.478+0000 7f1571ad1740 10 client.0 _ll_register_callbacks cb 0x56212d813af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T08:59:56.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.478+0000 7f1568af7640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56212d5320a0 0x56212d90be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:56.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.478+0000 7f1568af7640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56212d5320a0 0x56212d90be40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:45436/0 (socket says 172.21.15.42:45436) 2024-08-30T08:59:56.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.478+0000 7f1568af7640 1 -- 172.21.15.42:0/751221556 learned_addr learned my addr 172.21.15.42:0/751221556 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T08:59:56.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.478+0000 7f15692f8640 1 --2- 172.21.15.42:0/751221556 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56212d53a280 0x56212d90c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:56.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.478+0000 7f1568af7640 1 --2- 172.21.15.42:0/751221556 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56212d55af60 0x56212d918280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:56.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.478+0000 7f1568af7640 1 -- 172.21.15.42:0/751221556 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56212d53a280 msgr2=0x56212d90c380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T08:59:56.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.478+0000 7f1568af7640 1 --2- 172.21.15.42:0/751221556 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56212d53a280 0x56212d90c380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:56.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.478+0000 7f1568af7640 1 -- 172.21.15.42:0/751221556 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56212d55af60 msgr2=0x56212d918280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T08:59:56.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.478+0000 7f1568af7640 1 --2- 172.21.15.42:0/751221556 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56212d55af60 0x56212d918280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:56.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.478+0000 7f1568af7640 1 -- 172.21.15.42:0/751221556 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56212d8267b0 con 0x56212d5320a0 2024-08-30T08:59:56.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.478+0000 7f1568af7640 1 --2- 172.21.15.42:0/751221556 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56212d5320a0 0x56212d90be40 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f15600055a0 tx=0x7f1560005c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c387b830f8963b5 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:59:56.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.478+0000 7f154cff9640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-08-30T08:59:56.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.479+0000 7f154cff9640 1 -- 172.21.15.42:0/751221556 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1560004020 con 0x56212d5320a0 2024-08-30T08:59:56.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.479+0000 7f154cff9640 1 -- 172.21.15.42:0/751221556 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f15600041c0 con 0x56212d5320a0 2024-08-30T08:59:56.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.479+0000 7f1571ad1740 1 -- 172.21.15.42:0/751221556 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1560013e20 con 0x56212d5320a0 2024-08-30T08:59:56.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.479+0000 7f154cff9640 1 -- 172.21.15.42:0/751221556 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f1560033050 con 0x56212d5320a0 2024-08-30T08:59:56.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.479+0000 7f1571ad1740 10 client.15255 fetch_fsmap learned FSMap version 26 2024-08-30T08:59:56.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.479+0000 7f1571ad1740 1 -- 172.21.15.42:0/751221556 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56212d91ac70 con 0x56212d5320a0 2024-08-30T08:59:56.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.479+0000 7f154cff9640 1 -- 172.21.15.42:0/751221556 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1560013e20 con 0x56212d5320a0 2024-08-30T08:59:56.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.479+0000 7f154cff9640 1 -- 172.21.15.42:0/751221556 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f156000f040 con 0x56212d5320a0 2024-08-30T08:59:56.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.479+0000 7f1571ad1740 10 client.15255 fetch_fsmap finished waiting for FSMap version 26 2024-08-30T08:59:56.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.479+0000 7f1571ad1740 10 client.15255 resolve_mds: resolved name 'i' to daemon mds.i 2024-08-30T08:59:56.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.479+0000 7f1571ad1740 20 client.15255 populate_metadata read hostname 'smithi042' 2024-08-30T08:59:56.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.479+0000 7f1571ad1740 1 --2- 172.21.15.42:0/751221556 >> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] conn(0x56212d5198e0 0x56212d91e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:56.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.479+0000 7f1571ad1740 4 client.15255 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-30T08:59:56.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.479+0000 7f1571ad1740 1 -- 172.21.15.42:0/751221556 --> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56212d879340 con 0x56212d5198e0 2024-08-30T08:59:56.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.480+0000 7f15692f8640 1 --2- 172.21.15.42:0/751221556 >> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] conn(0x56212d5198e0 0x56212d91e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:56.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.480+0000 7f15692f8640 1 --2- 172.21.15.42:0/751221556 >> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] conn(0x56212d5198e0 0x56212d91e250 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=912038a8f0513c42 server_cookie=c682c9fc4f2db2f4 in_seq=0 out_seq=0 2024-08-30T08:59:56.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.480+0000 7f154cff9640 10 client.15255 ms_handle_connect on v2:172.21.15.190:6838/719981110 2024-08-30T08:59:56.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.481+0000 7f154cff9640 1 -- 172.21.15.42:0/751221556 <== mds.1 v2:172.21.15.190:6838/719981110 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56212d879340 con 0x56212d5198e0 2024-08-30T08:59:56.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.481+0000 7f154cff9640 10 client.15255 handle_command_reply: tid=0 2024-08-30T08:59:56.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.481+0000 7f1571ad1740 1 client.15255 shutdown 2024-08-30T08:59:56.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.481+0000 7f1571ad1740 1 -- 172.21.15.42:0/751221556 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56212d5320a0 msgr2=0x56212d90be40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:56.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.481+0000 7f1571ad1740 1 --2- 172.21.15.42:0/751221556 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56212d5320a0 0x56212d90be40 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f15600055a0 tx=0x7f1560005c70 comp rx=0 tx=0).stop 2024-08-30T08:59:56.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.481+0000 7f1571ad1740 1 -- 172.21.15.42:0/751221556 shutdown_connections 2024-08-30T08:59:56.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.481+0000 7f1571ad1740 1 --2- 172.21.15.42:0/751221556 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56212d53a280 0x56212d90c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:56.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.481+0000 7f1571ad1740 1 --2- 172.21.15.42:0/751221556 >> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] conn(0x56212d5198e0 0x56212d91e250 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:56.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.481+0000 7f1571ad1740 1 --2- 172.21.15.42:0/751221556 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56212d55af60 0x56212d918280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:56.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.481+0000 7f1571ad1740 1 --2- 172.21.15.42:0/751221556 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56212d5320a0 0x56212d90be40 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:56.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.481+0000 7f1571ad1740 1 -- 172.21.15.42:0/751221556 >> 172.21.15.42:0/751221556 conn(0x56212d54f350 msgr2=0x56212d903990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:59:56.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.481+0000 7f154cff9640 0 client.15255 ms_handle_reset on v2:172.21.15.190:6838/719981110 2024-08-30T08:59:56.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.481+0000 7f1571ad1740 1 -- 172.21.15.42:0/751221556 shutdown_connections 2024-08-30T08:59:56.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.481+0000 7f1571ad1740 1 -- 172.21.15.42:0/751221556 wait complete. 2024-08-30T08:59:56.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.482+0000 7f1571ad1740 20 client.15255 trim_cache size 0 max 16384 2024-08-30T08:59:56.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.500+0000 7f1571ad1740 1 Processor -- start 2024-08-30T08:59:56.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.500+0000 7f1571ad1740 1 -- start start 2024-08-30T08:59:56.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.500+0000 7f1571ad1740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56212d5320a0 0x56212d91cde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:56.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.500+0000 7f1571ad1740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56212d53a280 0x56212d902290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:56.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.500+0000 7f1571ad1740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56212d55af60 0x56212d9181f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:56.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.500+0000 7f1571ad1740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x56212d91c590 con 0x56212d55af60 2024-08-30T08:59:56.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.500+0000 7f1571ad1740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x56212d91cc60 con 0x56212d53a280 2024-08-30T08:59:56.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.500+0000 7f1571ad1740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x56212d91d330 con 0x56212d5320a0 2024-08-30T08:59:56.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.500+0000 7f1568af7640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56212d55af60 0x56212d9181f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:56.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.500+0000 7f1568af7640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56212d55af60 0x56212d9181f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:45452/0 (socket says 172.21.15.42:45452) 2024-08-30T08:59:56.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.500+0000 7f1568af7640 1 -- 172.21.15.42:0/2326170578 learned_addr learned my addr 172.21.15.42:0/2326170578 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T08:59:56.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.500+0000 7f1568af7640 1 --2- 172.21.15.42:0/2326170578 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56212d5320a0 0x56212d91cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:56.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.500+0000 7f15692f8640 1 --2- 172.21.15.42:0/2326170578 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56212d53a280 0x56212d902290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:56.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.501+0000 7f1568af7640 1 -- 172.21.15.42:0/2326170578 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56212d5320a0 msgr2=0x56212d91cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T08:59:56.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.501+0000 7f1568af7640 1 --2- 172.21.15.42:0/2326170578 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56212d5320a0 0x56212d91cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:56.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.501+0000 7f1568af7640 1 -- 172.21.15.42:0/2326170578 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56212d53a280 msgr2=0x56212d902290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T08:59:56.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.501+0000 7f1568af7640 1 --2- 172.21.15.42:0/2326170578 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56212d53a280 0x56212d902290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:56.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.501+0000 7f1568af7640 1 -- 172.21.15.42:0/2326170578 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56212d90a340 con 0x56212d55af60 2024-08-30T08:59:56.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.501+0000 7f15692f8640 1 --2- 172.21.15.42:0/2326170578 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56212d53a280 0x56212d902290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-30T08:59:56.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.501+0000 7f1568af7640 1 --2- 172.21.15.42:0/2326170578 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56212d55af60 0x56212d9181f0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f1560005d00 tx=0x7f1560020040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38a27a9603b2eec0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:59:56.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.501+0000 7f154effd640 1 -- 172.21.15.42:0/2326170578 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f156003c070 con 0x56212d55af60 2024-08-30T08:59:56.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.501+0000 7f154effd640 1 -- 172.21.15.42:0/2326170578 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f156000b900 con 0x56212d55af60 2024-08-30T08:59:56.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.501+0000 7f154effd640 1 -- 172.21.15.42:0/2326170578 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1560004d90 con 0x56212d55af60 2024-08-30T08:59:56.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.502+0000 7f1571ad1740 1 -- 172.21.15.42:0/2326170578 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56212d55af60 msgr2=0x56212d9181f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:56.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.502+0000 7f1571ad1740 1 --2- 172.21.15.42:0/2326170578 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56212d55af60 0x56212d9181f0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f1560005d00 tx=0x7f1560020040 comp rx=0 tx=0).stop 2024-08-30T08:59:56.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.502+0000 7f1571ad1740 1 -- 172.21.15.42:0/2326170578 shutdown_connections 2024-08-30T08:59:56.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.502+0000 7f1571ad1740 1 --2- 172.21.15.42:0/2326170578 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56212d55af60 0x56212d9181f0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:56.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.502+0000 7f1571ad1740 1 --2- 172.21.15.42:0/2326170578 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56212d53a280 0x56212d902290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:56.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.502+0000 7f1571ad1740 1 --2- 172.21.15.42:0/2326170578 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56212d5320a0 0x56212d91cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:56.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.502+0000 7f1571ad1740 1 -- 172.21.15.42:0/2326170578 >> 172.21.15.42:0/2326170578 conn(0x56212d54f350 msgr2=0x56212d8f9400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:59:56.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.502+0000 7f1571ad1740 1 -- 172.21.15.42:0/2326170578 shutdown_connections 2024-08-30T08:59:56.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.502+0000 7f1571ad1740 1 -- 172.21.15.42:0/2326170578 wait complete. 2024-08-30T08:59:56.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.502+0000 7f1571ad1740 1 -- start start 2024-08-30T08:59:56.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.502+0000 7f1571ad1740 1 Processor -- start 2024-08-30T08:59:56.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.503+0000 7f1571ad1740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56212d5320a0 0x56212d925510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:56.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.503+0000 7f1571ad1740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56212d53a280 0x56212d925ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:56.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.503+0000 7f1571ad1740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56212d55af60 0x56212d91fb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:56.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.503+0000 7f1571ad1740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x56212d91e0d0 con 0x56212d55af60 2024-08-30T08:59:56.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.503+0000 7f1571ad1740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x56212d91da00 con 0x56212d53a280 2024-08-30T08:59:56.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.503+0000 7f1571ad1740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x56212d91e7a0 con 0x56212d5320a0 2024-08-30T08:59:56.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.503+0000 7f1571ad1740 10 client.0 _ll_register_callbacks cb 0x7f15600137d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T08:59:56.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.503+0000 7f1568af7640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56212d55af60 0x56212d91fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:56.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.503+0000 7f1568af7640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56212d55af60 0x56212d91fb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:45456/0 (socket says 172.21.15.42:45456) 2024-08-30T08:59:56.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.503+0000 7f1568af7640 1 -- 172.21.15.42:0/3348111757 learned_addr learned my addr 172.21.15.42:0/3348111757 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T08:59:56.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.503+0000 7f1568af7640 1 --2- 172.21.15.42:0/3348111757 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56212d5320a0 0x56212d925510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:56.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.503+0000 7f15692f8640 1 --2- 172.21.15.42:0/3348111757 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56212d53a280 0x56212d925ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:56.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.503+0000 7f1568af7640 1 -- 172.21.15.42:0/3348111757 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56212d5320a0 msgr2=0x56212d925510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T08:59:56.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.503+0000 7f1568af7640 1 --2- 172.21.15.42:0/3348111757 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56212d5320a0 0x56212d925510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:56.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.503+0000 7f1568af7640 1 -- 172.21.15.42:0/3348111757 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56212d53a280 msgr2=0x56212d925ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T08:59:56.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.503+0000 7f1568af7640 1 --2- 172.21.15.42:0/3348111757 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56212d53a280 0x56212d925ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:56.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.503+0000 7f1568af7640 1 -- 172.21.15.42:0/3348111757 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56212d920180 con 0x56212d55af60 2024-08-30T08:59:56.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.503+0000 7f15692f8640 1 --2- 172.21.15.42:0/3348111757 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56212d53a280 0x56212d925ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-30T08:59:56.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.503+0000 7f1568af7640 1 --2- 172.21.15.42:0/3348111757 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56212d55af60 0x56212d91fb10 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f1560005cd0 tx=0x7f1560000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4d592396f936dfc server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:59:56.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.504+0000 7f154cff9640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-08-30T08:59:56.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.504+0000 7f154cff9640 1 -- 172.21.15.42:0/3348111757 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f156003c040 con 0x56212d55af60 2024-08-30T08:59:56.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.504+0000 7f154cff9640 1 -- 172.21.15.42:0/3348111757 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f1560004680 con 0x56212d55af60 2024-08-30T08:59:56.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.504+0000 7f1571ad1740 1 -- 172.21.15.42:0/3348111757 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f156000b900 con 0x56212d55af60 2024-08-30T08:59:56.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.504+0000 7f154cff9640 1 -- 172.21.15.42:0/3348111757 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f156004d050 con 0x56212d55af60 2024-08-30T08:59:56.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.504+0000 7f1571ad1740 10 client.15261 fetch_fsmap learned FSMap version 26 2024-08-30T08:59:56.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.504+0000 7f1571ad1740 1 -- 172.21.15.42:0/3348111757 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56212d926d40 con 0x56212d55af60 2024-08-30T08:59:56.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.504+0000 7f154cff9640 1 -- 172.21.15.42:0/3348111757 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f156000b900 con 0x56212d55af60 2024-08-30T08:59:56.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.504+0000 7f154cff9640 1 -- 172.21.15.42:0/3348111757 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f1560042070 con 0x56212d55af60 2024-08-30T08:59:56.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.504+0000 7f1571ad1740 10 client.15261 fetch_fsmap finished waiting for FSMap version 26 2024-08-30T08:59:56.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.504+0000 7f1571ad1740 10 client.15261 resolve_mds: resolved name 'i' to daemon mds.i 2024-08-30T08:59:56.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.504+0000 7f1571ad1740 20 client.15261 populate_metadata read hostname 'smithi042' 2024-08-30T08:59:56.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.504+0000 7f1571ad1740 1 --2- 172.21.15.42:0/3348111757 >> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] conn(0x56212d4eec90 0x56212d98c450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:56.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.504+0000 7f1571ad1740 4 client.15261 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "status"}] 2024-08-30T08:59:56.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.504+0000 7f1571ad1740 1 -- 172.21.15.42:0/3348111757 --> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] -- command(tid 0: {"prefix": "status"}) -- 0x56212d874de0 con 0x56212d4eec90 2024-08-30T08:59:56.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.505+0000 7f15692f8640 1 --2- 172.21.15.42:0/3348111757 >> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] conn(0x56212d4eec90 0x56212d98c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:56.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.505+0000 7f15692f8640 1 --2- 172.21.15.42:0/3348111757 >> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] conn(0x56212d4eec90 0x56212d98c450 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=cb5bdabec98017a6 server_cookie=14bd967b02d1166 in_seq=0 out_seq=0 2024-08-30T08:59:56.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.505+0000 7f154cff9640 10 client.15261 ms_handle_connect on v2:172.21.15.190:6838/719981110 2024-08-30T08:59:56.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.505+0000 7f154cff9640 1 -- 172.21.15.42:0/3348111757 <== mds.1 v2:172.21.15.190:6838/719981110 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x56212d874de0 con 0x56212d4eec90 2024-08-30T08:59:56.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.505+0000 7f154cff9640 10 client.15261 handle_command_reply: tid=0 2024-08-30T08:59:56.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.505+0000 7f1571ad1740 1 client.15261 shutdown 2024-08-30T08:59:56.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.506+0000 7f1571ad1740 1 -- 172.21.15.42:0/3348111757 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56212d55af60 msgr2=0x56212d91fb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:56.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.506+0000 7f1571ad1740 1 --2- 172.21.15.42:0/3348111757 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56212d55af60 0x56212d91fb10 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f1560005cd0 tx=0x7f1560000bc0 comp rx=0 tx=0).stop 2024-08-30T08:59:56.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.506+0000 7f1571ad1740 1 -- 172.21.15.42:0/3348111757 shutdown_connections 2024-08-30T08:59:56.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.506+0000 7f1571ad1740 1 --2- 172.21.15.42:0/3348111757 >> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] conn(0x56212d4eec90 0x56212d98c450 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:56.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.506+0000 7f1571ad1740 1 --2- 172.21.15.42:0/3348111757 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56212d55af60 0x56212d91fb10 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:56.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.506+0000 7f1571ad1740 1 --2- 172.21.15.42:0/3348111757 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56212d53a280 0x56212d925ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:56.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.506+0000 7f1571ad1740 1 --2- 172.21.15.42:0/3348111757 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56212d5320a0 0x56212d925510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:56.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.506+0000 7f1571ad1740 1 -- 172.21.15.42:0/3348111757 >> 172.21.15.42:0/3348111757 conn(0x56212d54f350 msgr2=0x56212d8f9400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:59:56.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.506+0000 7f154cff9640 0 client.15261 ms_handle_reset on v2:172.21.15.190:6838/719981110 2024-08-30T08:59:56.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.506+0000 7f1571ad1740 1 -- 172.21.15.42:0/3348111757 shutdown_connections 2024-08-30T08:59:56.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.506+0000 7f1571ad1740 1 -- 172.21.15.42:0/3348111757 wait complete. 2024-08-30T08:59:56.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.506+0000 7f1571ad1740 20 client.15261 trim_cache size 0 max 16384 2024-08-30T08:59:56.514 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-08-30T08:59:56.514 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_fsid": "b0895c1a-66ac-11ef-bcd4-c7b262605968", 2024-08-30T08:59:56.514 INFO:teuthology.orchestra.run.smithi042.stdout: "whoami": 1, 2024-08-30T08:59:56.514 INFO:teuthology.orchestra.run.smithi042.stdout: "id": 24610, 2024-08-30T08:59:56.515 INFO:teuthology.orchestra.run.smithi042.stdout: "want_state": "up:active", 2024-08-30T08:59:56.515 INFO:teuthology.orchestra.run.smithi042.stdout: "state": "up:active", 2024-08-30T08:59:56.515 INFO:teuthology.orchestra.run.smithi042.stdout: "fs_name": "cephfs", 2024-08-30T08:59:56.515 INFO:teuthology.orchestra.run.smithi042.stdout: "rank_uptime": 71.961909702, 2024-08-30T08:59:56.515 INFO:teuthology.orchestra.run.smithi042.stdout: "mdsmap_epoch": 26, 2024-08-30T08:59:56.515 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_epoch": 83, 2024-08-30T08:59:56.515 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_epoch_barrier": 81, 2024-08-30T08:59:56.515 INFO:teuthology.orchestra.run.smithi042.stdout: "uptime": 133.19971176799999 2024-08-30T08:59:56.515 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-08-30T08:59:56.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.506+0000 7f156b55c640 1 -- 172.21.15.42:0/1008138834 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f15500b4bc0 msgr2=0x7f15500b70d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:56.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.506+0000 7f156b55c640 1 --2- 172.21.15.42:0/1008138834 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f15500b4bc0 0x7f15500b70d0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f155c00dfd0 tx=0x7f155c0c1040 comp rx=0 tx=0).stop 2024-08-30T08:59:56.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.506+0000 7f156b55c640 1 -- 172.21.15.42:0/1008138834 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1564109be0 msgr2=0x7f156419fad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:56.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.506+0000 7f156b55c640 1 --2- 172.21.15.42:0/1008138834 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1564109be0 0x7f156419fad0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f155c009fd0 tx=0x7f155c00b260 comp rx=0 tx=0).stop 2024-08-30T08:59:56.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.506+0000 7f156b55c640 1 -- 172.21.15.42:0/1008138834 shutdown_connections 2024-08-30T08:59:56.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.506+0000 7f156b55c640 1 --2- 172.21.15.42:0/1008138834 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f15500b4bc0 0x7f15500b70d0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:56.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.506+0000 7f156b55c640 1 --2- 172.21.15.42:0/1008138834 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1564109290 0x7f156419a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:56.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.506+0000 7f156b55c640 1 --2- 172.21.15.42:0/1008138834 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1564109be0 0x7f156419fad0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:56.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.507+0000 7f156b55c640 1 --2- 172.21.15.42:0/1008138834 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1564103950 0x7f156419a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:56.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.507+0000 7f156b55c640 1 -- 172.21.15.42:0/1008138834 >> 172.21.15.42:0/1008138834 conn(0x7f15640ff3a0 msgr2=0x7f15641017c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:59:56.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.507+0000 7f156b55c640 1 -- 172.21.15.42:0/1008138834 shutdown_connections 2024-08-30T08:59:56.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:56.507+0000 7f156b55c640 1 -- 172.21.15.42:0/1008138834 wait complete. 2024-08-30T08:59:56.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:59:56 smithi190 ceph-mon[22001]: pgmap v348: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s; 0 B/s, 1 keys/s, 0 objects/s recovering 2024-08-30T08:59:56.968 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:59:56 smithi042 ceph-mon[18652]: pgmap v348: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s; 0 B/s, 1 keys/s, 0 objects/s recovering 2024-08-30T08:59:57.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:59:56 smithi167 ceph-mon[22063]: pgmap v348: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s; 0 B/s, 1 keys/s, 0 objects/s recovering 2024-08-30T08:59:57.401 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph tell mds.b status 2024-08-30T08:59:57.655 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T08:59:58.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.139+0000 7f2e0dbca640 1 -- 172.21.15.42:0/1289250364 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2e08107100 msgr2=0x7f2e08107510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:58.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.139+0000 7f2e0dbca640 1 --2- 172.21.15.42:0/1289250364 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2e08107100 0x7f2e08107510 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f2dfc009900 tx=0x7f2dfc02a410 comp rx=0 tx=0).stop 2024-08-30T08:59:58.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.139+0000 7f2e0dbca640 1 -- 172.21.15.42:0/1289250364 shutdown_connections 2024-08-30T08:59:58.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.139+0000 7f2e0dbca640 1 --2- 172.21.15.42:0/1289250364 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2e08107a50 0x7f2e08110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:58.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.139+0000 7f2e0dbca640 1 --2- 172.21.15.42:0/1289250364 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2e08107100 0x7f2e08107510 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:58.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.139+0000 7f2e0dbca640 1 --2- 172.21.15.42:0/1289250364 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2e08101f00 0x7f2e08102310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:58.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.139+0000 7f2e0dbca640 1 -- 172.21.15.42:0/1289250364 >> 172.21.15.42:0/1289250364 conn(0x7f2e08079c40 msgr2=0x7f2e0807a050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:59:58.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.139+0000 7f2e0dbca640 1 -- 172.21.15.42:0/1289250364 shutdown_connections 2024-08-30T08:59:58.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.139+0000 7f2e0dbca640 1 -- 172.21.15.42:0/1289250364 wait complete. 2024-08-30T08:59:58.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.139+0000 7f2e0dbca640 1 Processor -- start 2024-08-30T08:59:58.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.140+0000 7f2e0dbca640 1 -- start start 2024-08-30T08:59:58.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.140+0000 7f2e0dbca640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2e08101f00 0x7f2e0819a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:58.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.140+0000 7f2e0dbca640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2e08107100 0x7f2e0819a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:58.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.140+0000 7f2e0dbca640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2e08107a50 0x7f2e0819fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:58.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.140+0000 7f2e0dbca640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f2e081138c0 con 0x7f2e08107a50 2024-08-30T08:59:58.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.140+0000 7f2e0dbca640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f2e08113740 con 0x7f2e08107100 2024-08-30T08:59:58.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.140+0000 7f2e0dbca640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f2e08113a40 con 0x7f2e08101f00 2024-08-30T08:59:58.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.140+0000 7f2e07fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2e08107a50 0x7f2e0819fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:58.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.140+0000 7f2e07fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2e08107a50 0x7f2e0819fb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43088/0 (socket says 172.21.15.42:43088) 2024-08-30T08:59:58.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.140+0000 7f2e07fff640 1 -- 172.21.15.42:0/2462416288 learned_addr learned my addr 172.21.15.42:0/2462416288 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T08:59:58.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.140+0000 7f2e077fe640 1 --2- 172.21.15.42:0/2462416288 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2e08101f00 0x7f2e0819a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:58.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.140+0000 7f2e06ffd640 1 --2- 172.21.15.42:0/2462416288 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2e08107100 0x7f2e0819a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:58.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.140+0000 7f2e07fff640 1 -- 172.21.15.42:0/2462416288 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2e08101f00 msgr2=0x7f2e0819a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:58.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.140+0000 7f2e07fff640 1 --2- 172.21.15.42:0/2462416288 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2e08101f00 0x7f2e0819a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:58.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.140+0000 7f2e07fff640 1 -- 172.21.15.42:0/2462416288 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2e08107100 msgr2=0x7f2e0819a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:58.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.140+0000 7f2e07fff640 1 --2- 172.21.15.42:0/2462416288 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2e08107100 0x7f2e0819a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:58.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.140+0000 7f2e07fff640 1 -- 172.21.15.42:0/2462416288 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e081a0260 con 0x7f2e08107a50 2024-08-30T08:59:58.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.141+0000 7f2e07fff640 1 --2- 172.21.15.42:0/2462416288 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2e08107a50 0x7f2e0819fb30 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f2e0000cad0 tx=0x7f2e0000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:59:58.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.141+0000 7f2e04ff9640 1 -- 172.21.15.42:0/2462416288 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e0000ae20 con 0x7f2e08107a50 2024-08-30T08:59:58.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.141+0000 7f2e04ff9640 1 -- 172.21.15.42:0/2462416288 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e00005020 con 0x7f2e08107a50 2024-08-30T08:59:58.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.141+0000 7f2e0dbca640 1 -- 172.21.15.42:0/2462416288 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e081a0550 con 0x7f2e08107a50 2024-08-30T08:59:58.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.141+0000 7f2e04ff9640 1 -- 172.21.15.42:0/2462416288 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e0000f6b0 con 0x7f2e08107a50 2024-08-30T08:59:58.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.141+0000 7f2e0dbca640 1 -- 172.21.15.42:0/2462416288 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e081a6cb0 con 0x7f2e08107a50 2024-08-30T08:59:58.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.143+0000 7f2e1413f740 1 Processor -- start 2024-08-30T08:59:58.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.144+0000 7f2e04ff9640 1 -- 172.21.15.42:0/2462416288 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) ==== 152628+0+0 (secure 0 0 0) 0x7f2e000051d0 con 0x7f2e08107a50 2024-08-30T08:59:58.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.144+0000 7f2e1413f740 1 -- start start 2024-08-30T08:59:58.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.144+0000 7f2e1413f740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559b0e2a3f60 0x559b0e6497b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:58.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.144+0000 7f2e1413f740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559b0e27b0a0 0x559b0e649cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:58.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.144+0000 7f2e1413f740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559b0e283280 0x559b0e64a230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:58.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.144+0000 7f2e1413f740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x559b0e5c03f0 con 0x559b0e2a3f60 2024-08-30T08:59:58.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.144+0000 7f2e1413f740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x559b0e5af980 con 0x559b0e283280 2024-08-30T08:59:58.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.144+0000 7f2e1413f740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x559b0e5b3930 con 0x559b0e27b0a0 2024-08-30T08:59:58.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.144+0000 7f2e06ffd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559b0e2a3f60 0x559b0e6497b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:58.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.144+0000 7f2e06ffd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559b0e2a3f60 0x559b0e6497b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43102/0 (socket says 172.21.15.42:43102) 2024-08-30T08:59:58.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.144+0000 7f2e06ffd640 1 -- 172.21.15.42:0/3999789102 learned_addr learned my addr 172.21.15.42:0/3999789102 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T08:59:58.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.144+0000 7f2e077fe640 1 --2- 172.21.15.42:0/3999789102 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559b0e27b0a0 0x559b0e649cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:58.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.144+0000 7f2e04ff9640 1 --2- 172.21.15.42:0/2462416288 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f2df00b4c40 0x7f2df00b7150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:58.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.144+0000 7f2e06ffd640 1 -- 172.21.15.42:0/3999789102 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559b0e27b0a0 msgr2=0x559b0e649cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T08:59:58.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.144+0000 7f2e06ffd640 1 --2- 172.21.15.42:0/3999789102 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559b0e27b0a0 0x559b0e649cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:58.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.144+0000 7f2e06ffd640 1 -- 172.21.15.42:0/3999789102 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559b0e283280 msgr2=0x559b0e64a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T08:59:58.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.144+0000 7f2e06ffd640 1 --2- 172.21.15.42:0/3999789102 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559b0e283280 0x559b0e64a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:58.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.144+0000 7f2e06ffd640 1 -- 172.21.15.42:0/3999789102 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559b0e30f540 con 0x559b0e2a3f60 2024-08-30T08:59:58.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.145+0000 7f2e07fff640 1 --2- 172.21.15.42:0/2462416288 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f2df00b4c40 0x7f2df00b7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:58.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.145+0000 7f2e06ffd640 1 --2- 172.21.15.42:0/3999789102 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559b0e2a3f60 0x559b0e6497b0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f2dfc009900 tx=0x7f2dfc005f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d1e4b3b8b41e347 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:59:58.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.145+0000 7f2e04ff9640 1 -- 172.21.15.42:0/2462416288 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f2e000e5700 con 0x7f2e08107a50 2024-08-30T08:59:58.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.145+0000 7f2dedffb640 1 -- 172.21.15.42:0/3999789102 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dfc035050 con 0x559b0e2a3f60 2024-08-30T08:59:58.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.145+0000 7f2dedffb640 1 -- 172.21.15.42:0/3999789102 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2dfc0316a0 con 0x559b0e2a3f60 2024-08-30T08:59:58.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.145+0000 7f2dedffb640 1 -- 172.21.15.42:0/3999789102 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dfc0047a0 con 0x559b0e2a3f60 2024-08-30T08:59:58.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.146+0000 7f2e1413f740 1 -- 172.21.15.42:0/3999789102 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559b0e2a3f60 msgr2=0x559b0e6497b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:58.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.146+0000 7f2e1413f740 1 --2- 172.21.15.42:0/3999789102 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559b0e2a3f60 0x559b0e6497b0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f2dfc009900 tx=0x7f2dfc005f00 comp rx=0 tx=0).stop 2024-08-30T08:59:58.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.146+0000 7f2e07fff640 1 --2- 172.21.15.42:0/2462416288 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f2df00b4c40 0x7f2df00b7150 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f2e000a6290 tx=0x7f2e00077e90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:59:58.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.146+0000 7f2e1413f740 1 -- 172.21.15.42:0/3999789102 shutdown_connections 2024-08-30T08:59:58.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.146+0000 7f2e1413f740 1 --2- 172.21.15.42:0/3999789102 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559b0e27b0a0 0x559b0e649cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:58.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.146+0000 7f2e1413f740 1 --2- 172.21.15.42:0/3999789102 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559b0e283280 0x559b0e64a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:58.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.146+0000 7f2e1413f740 1 --2- 172.21.15.42:0/3999789102 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559b0e2a3f60 0x559b0e6497b0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:58.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.146+0000 7f2e1413f740 1 -- 172.21.15.42:0/3999789102 >> 172.21.15.42:0/3999789102 conn(0x559b0e298350 msgr2=0x559b0e6476a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:59:58.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.146+0000 7f2e1413f740 1 -- 172.21.15.42:0/3999789102 shutdown_connections 2024-08-30T08:59:58.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.146+0000 7f2e1413f740 1 -- 172.21.15.42:0/3999789102 wait complete. 2024-08-30T08:59:58.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.146+0000 7f2e1413f740 1 -- start start 2024-08-30T08:59:58.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.147+0000 7f2e1413f740 1 Processor -- start 2024-08-30T08:59:58.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.147+0000 7f2e1413f740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559b0e27b0a0 0x559b0e659340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:58.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.147+0000 7f2e1413f740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559b0e283280 0x559b0e659880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:58.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.147+0000 7f2e1413f740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559b0e2a3f60 0x559b0e6655c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:58.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.147+0000 7f2e1413f740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x559b0e4125f0 con 0x559b0e27b0a0 2024-08-30T08:59:58.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.147+0000 7f2e1413f740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x559b0e536600 con 0x559b0e283280 2024-08-30T08:59:58.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.147+0000 7f2e1413f740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x559b0e65c290 con 0x559b0e2a3f60 2024-08-30T08:59:58.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.147+0000 7f2e1413f740 10 client.0 _ll_register_callbacks cb 0x559b0e55caf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T08:59:58.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.147+0000 7f2e06ffd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559b0e27b0a0 0x559b0e659340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:58.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.147+0000 7f2e06ffd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559b0e27b0a0 0x559b0e659340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43110/0 (socket says 172.21.15.42:43110) 2024-08-30T08:59:58.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.147+0000 7f2e06ffd640 1 -- 172.21.15.42:0/1851295519 learned_addr learned my addr 172.21.15.42:0/1851295519 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T08:59:58.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.147+0000 7f2e077fe640 1 --2- 172.21.15.42:0/1851295519 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559b0e283280 0x559b0e659880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:58.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.147+0000 7f2e06ffd640 1 --2- 172.21.15.42:0/1851295519 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559b0e2a3f60 0x559b0e6655c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:58.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.147+0000 7f2e06ffd640 1 -- 172.21.15.42:0/1851295519 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559b0e2a3f60 msgr2=0x559b0e6655c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T08:59:58.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.147+0000 7f2e06ffd640 1 --2- 172.21.15.42:0/1851295519 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559b0e2a3f60 0x559b0e6655c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:58.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.147+0000 7f2e06ffd640 1 -- 172.21.15.42:0/1851295519 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559b0e283280 msgr2=0x559b0e659880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T08:59:58.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.147+0000 7f2e06ffd640 1 --2- 172.21.15.42:0/1851295519 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559b0e283280 0x559b0e659880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:58.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.148+0000 7f2e06ffd640 1 -- 172.21.15.42:0/1851295519 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559b0e56f7b0 con 0x559b0e27b0a0 2024-08-30T08:59:58.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.148+0000 7f2e06ffd640 1 --2- 172.21.15.42:0/1851295519 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559b0e27b0a0 0x559b0e659340 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f2dfc02a450 tx=0x7f2dfc02aa90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b708573d2d107937 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:59:58.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.148+0000 7f2de6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-08-30T08:59:58.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.148+0000 7f2de6ffd640 1 -- 172.21.15.42:0/1851295519 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dfc04c020 con 0x559b0e27b0a0 2024-08-30T08:59:58.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.148+0000 7f2de6ffd640 1 -- 172.21.15.42:0/1851295519 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f2dfc004320 con 0x559b0e27b0a0 2024-08-30T08:59:58.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.148+0000 7f2e1413f740 1 -- 172.21.15.42:0/1851295519 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2dfc0316a0 con 0x559b0e27b0a0 2024-08-30T08:59:58.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.148+0000 7f2de6ffd640 1 -- 172.21.15.42:0/1851295519 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f2dfc058050 con 0x559b0e27b0a0 2024-08-30T08:59:58.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.148+0000 7f2e1413f740 10 client.15273 fetch_fsmap learned FSMap version 26 2024-08-30T08:59:58.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.148+0000 7f2e1413f740 1 -- 172.21.15.42:0/1851295519 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x559b0e668080 con 0x559b0e27b0a0 2024-08-30T08:59:58.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.148+0000 7f2de6ffd640 1 -- 172.21.15.42:0/1851295519 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2dfc0316a0 con 0x559b0e27b0a0 2024-08-30T08:59:58.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.148+0000 7f2de6ffd640 1 -- 172.21.15.42:0/1851295519 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f2dfc003850 con 0x559b0e27b0a0 2024-08-30T08:59:58.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.149+0000 7f2e1413f740 10 client.15273 fetch_fsmap finished waiting for FSMap version 26 2024-08-30T08:59:58.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.149+0000 7f2e1413f740 10 client.15273 resolve_mds: resolved name 'b' to daemon mds.b 2024-08-30T08:59:58.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.149+0000 7f2e1413f740 20 client.15273 populate_metadata read hostname 'smithi042' 2024-08-30T08:59:58.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.149+0000 7f2e1413f740 1 --2- 172.21.15.42:0/1851295519 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x559b0e2628e0 0x559b0e66b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:58.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.149+0000 7f2e1413f740 4 client.15273 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-30T08:59:58.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.149+0000 7f2e1413f740 1 -- 172.21.15.42:0/1851295519 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559b0e5c2340 con 0x559b0e2628e0 2024-08-30T08:59:58.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.149+0000 7f2e077fe640 1 --2- 172.21.15.42:0/1851295519 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x559b0e2628e0 0x559b0e66b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:58.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.150+0000 7f2e077fe640 1 --2- 172.21.15.42:0/1851295519 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x559b0e2628e0 0x559b0e66b5f0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be09252e9cae9cb5 server_cookie=e0d19f2c410e7a9 in_seq=0 out_seq=0 2024-08-30T08:59:58.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.150+0000 7f2de6ffd640 10 client.15273 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T08:59:58.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.151+0000 7f2de6ffd640 1 -- 172.21.15.42:0/1851295519 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x559b0e5c2340 con 0x559b0e2628e0 2024-08-30T08:59:58.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.151+0000 7f2de6ffd640 10 client.15273 handle_command_reply: tid=0 2024-08-30T08:59:58.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.151+0000 7f2e1413f740 1 client.15273 shutdown 2024-08-30T08:59:58.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.151+0000 7f2e1413f740 1 -- 172.21.15.42:0/1851295519 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559b0e27b0a0 msgr2=0x559b0e659340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:58.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.151+0000 7f2e1413f740 1 --2- 172.21.15.42:0/1851295519 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559b0e27b0a0 0x559b0e659340 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f2dfc02a450 tx=0x7f2dfc02aa90 comp rx=0 tx=0).stop 2024-08-30T08:59:58.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.151+0000 7f2e1413f740 1 -- 172.21.15.42:0/1851295519 shutdown_connections 2024-08-30T08:59:58.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.151+0000 7f2e1413f740 1 --2- 172.21.15.42:0/1851295519 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x559b0e2628e0 0x559b0e66b5f0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:58.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.151+0000 7f2e1413f740 1 --2- 172.21.15.42:0/1851295519 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559b0e2a3f60 0x559b0e6655c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:58.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.151+0000 7f2e1413f740 1 --2- 172.21.15.42:0/1851295519 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559b0e283280 0x559b0e659880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:58.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.151+0000 7f2de6ffd640 0 client.15273 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T08:59:58.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.151+0000 7f2e1413f740 1 --2- 172.21.15.42:0/1851295519 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559b0e27b0a0 0x559b0e659340 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:58.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.151+0000 7f2e1413f740 1 -- 172.21.15.42:0/1851295519 >> 172.21.15.42:0/1851295519 conn(0x559b0e298350 msgr2=0x559b0e64b1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:59:58.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.151+0000 7f2e1413f740 1 -- 172.21.15.42:0/1851295519 shutdown_connections 2024-08-30T08:59:58.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.151+0000 7f2e1413f740 1 -- 172.21.15.42:0/1851295519 wait complete. 2024-08-30T08:59:58.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.151+0000 7f2e1413f740 20 client.15273 trim_cache size 0 max 16384 2024-08-30T08:59:58.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.169+0000 7f2e1413f740 1 Processor -- start 2024-08-30T08:59:58.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.169+0000 7f2e1413f740 1 -- start start 2024-08-30T08:59:58.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.169+0000 7f2e1413f740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559b0e27b0a0 0x559b0e64e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:58.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.169+0000 7f2e1413f740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559b0e283280 0x559b0e665530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:58.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.169+0000 7f2e1413f740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559b0e2a3f60 0x559b0e656530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:58.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.170+0000 7f2e1413f740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x559b0e669930 con 0x559b0e2a3f60 2024-08-30T08:59:58.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.170+0000 7f2e1413f740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x559b0e66a000 con 0x559b0e283280 2024-08-30T08:59:58.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.170+0000 7f2e1413f740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x559b0e66a6d0 con 0x559b0e27b0a0 2024-08-30T08:59:58.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.170+0000 7f2e06ffd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559b0e2a3f60 0x559b0e656530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:58.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.170+0000 7f2e06ffd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559b0e2a3f60 0x559b0e656530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43126/0 (socket says 172.21.15.42:43126) 2024-08-30T08:59:58.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.170+0000 7f2e06ffd640 1 -- 172.21.15.42:0/3438133384 learned_addr learned my addr 172.21.15.42:0/3438133384 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T08:59:58.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.170+0000 7f2e06ffd640 1 --2- 172.21.15.42:0/3438133384 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559b0e27b0a0 0x559b0e64e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:58.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.170+0000 7f2e077fe640 1 --2- 172.21.15.42:0/3438133384 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559b0e283280 0x559b0e665530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:58.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.170+0000 7f2e06ffd640 1 -- 172.21.15.42:0/3438133384 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559b0e27b0a0 msgr2=0x559b0e64e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T08:59:58.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.170+0000 7f2e06ffd640 1 --2- 172.21.15.42:0/3438133384 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559b0e27b0a0 0x559b0e64e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:58.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.170+0000 7f2e06ffd640 1 -- 172.21.15.42:0/3438133384 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559b0e283280 msgr2=0x559b0e665530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T08:59:58.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.170+0000 7f2e06ffd640 1 --2- 172.21.15.42:0/3438133384 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559b0e283280 0x559b0e665530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:58.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.170+0000 7f2e06ffd640 1 -- 172.21.15.42:0/3438133384 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559b0e657840 con 0x559b0e2a3f60 2024-08-30T08:59:58.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.170+0000 7f2e06ffd640 1 --2- 172.21.15.42:0/3438133384 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559b0e2a3f60 0x559b0e656530 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f2dfc003ca0 tx=0x7f2dfc031850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffb5b4a865fc4aa3 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:59:58.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.170+0000 7f2ded7fa640 1 -- 172.21.15.42:0/3438133384 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dfc04c040 con 0x559b0e2a3f60 2024-08-30T08:59:58.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.170+0000 7f2ded7fa640 1 -- 172.21.15.42:0/3438133384 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2dfc004de0 con 0x559b0e2a3f60 2024-08-30T08:59:58.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.170+0000 7f2ded7fa640 1 -- 172.21.15.42:0/3438133384 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dfc035040 con 0x559b0e2a3f60 2024-08-30T08:59:58.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.171+0000 7f2e1413f740 1 -- 172.21.15.42:0/3438133384 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559b0e2a3f60 msgr2=0x559b0e656530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:58.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.171+0000 7f2e1413f740 1 --2- 172.21.15.42:0/3438133384 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559b0e2a3f60 0x559b0e656530 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f2dfc003ca0 tx=0x7f2dfc031850 comp rx=0 tx=0).stop 2024-08-30T08:59:58.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.171+0000 7f2e1413f740 1 -- 172.21.15.42:0/3438133384 shutdown_connections 2024-08-30T08:59:58.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.171+0000 7f2e1413f740 1 --2- 172.21.15.42:0/3438133384 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559b0e2a3f60 0x559b0e656530 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:58.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.171+0000 7f2e1413f740 1 --2- 172.21.15.42:0/3438133384 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559b0e283280 0x559b0e665530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:58.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.171+0000 7f2e1413f740 1 --2- 172.21.15.42:0/3438133384 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559b0e27b0a0 0x559b0e64e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:58.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.171+0000 7f2e1413f740 1 -- 172.21.15.42:0/3438133384 >> 172.21.15.42:0/3438133384 conn(0x559b0e298350 msgr2=0x559b0e6544f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:59:58.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.171+0000 7f2e1413f740 1 -- 172.21.15.42:0/3438133384 shutdown_connections 2024-08-30T08:59:58.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.171+0000 7f2e1413f740 1 -- 172.21.15.42:0/3438133384 wait complete. 2024-08-30T08:59:58.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.172+0000 7f2e1413f740 1 -- start start 2024-08-30T08:59:58.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.172+0000 7f2e1413f740 1 Processor -- start 2024-08-30T08:59:58.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.172+0000 7f2e1413f740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559b0e27b0a0 0x559b0e66f960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:58.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.172+0000 7f2e1413f740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559b0e283280 0x559b0e66fea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:58.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.172+0000 7f2e1413f740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559b0e2a3f60 0x559b0e6703e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:58.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.172+0000 7f2e1413f740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x559b0e66b470 con 0x559b0e283280 2024-08-30T08:59:58.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.172+0000 7f2e1413f740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x559b0e66ada0 con 0x559b0e2a3f60 2024-08-30T08:59:58.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.172+0000 7f2e1413f740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x559b0e66bb40 con 0x559b0e27b0a0 2024-08-30T08:59:58.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.172+0000 7f2e1413f740 10 client.0 _ll_register_callbacks cb 0x7f2dfc031970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T08:59:58.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.172+0000 7f2e077fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559b0e283280 0x559b0e66fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:58.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.172+0000 7f2e077fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559b0e283280 0x559b0e66fea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43138/0 (socket says 172.21.15.42:43138) 2024-08-30T08:59:58.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.172+0000 7f2e077fe640 1 -- 172.21.15.42:0/2265941766 learned_addr learned my addr 172.21.15.42:0/2265941766 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T08:59:58.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.173+0000 7f2e06ffd640 1 --2- 172.21.15.42:0/2265941766 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559b0e2a3f60 0x559b0e6703e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:58.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.173+0000 7f2e06ffd640 1 --2- 172.21.15.42:0/2265941766 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559b0e27b0a0 0x559b0e66f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:58.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.173+0000 7f2e077fe640 1 -- 172.21.15.42:0/2265941766 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559b0e27b0a0 msgr2=0x559b0e66f960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T08:59:58.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.173+0000 7f2e077fe640 1 --2- 172.21.15.42:0/2265941766 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559b0e27b0a0 0x559b0e66f960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:58.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.173+0000 7f2e077fe640 1 -- 172.21.15.42:0/2265941766 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559b0e2a3f60 msgr2=0x559b0e6703e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T08:59:58.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.173+0000 7f2e077fe640 1 --2- 172.21.15.42:0/2265941766 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559b0e2a3f60 0x559b0e6703e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:58.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.173+0000 7f2e077fe640 1 -- 172.21.15.42:0/2265941766 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559b0e6d5b90 con 0x559b0e283280 2024-08-30T08:59:58.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.173+0000 7f2e077fe640 1 --2- 172.21.15.42:0/2265941766 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559b0e283280 0x559b0e66fea0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f2df8004700 tx=0x7f2df8010a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9da0ab34c810098d server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:59:58.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.173+0000 7f2de6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-08-30T08:59:58.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.173+0000 7f2de6ffd640 1 -- 172.21.15.42:0/2265941766 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2df8005020 con 0x559b0e283280 2024-08-30T08:59:58.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.173+0000 7f2de6ffd640 1 -- 172.21.15.42:0/2265941766 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f2df80051c0 con 0x559b0e283280 2024-08-30T08:59:58.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.173+0000 7f2e1413f740 1 -- 172.21.15.42:0/2265941766 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2dfc004de0 con 0x559b0e283280 2024-08-30T08:59:58.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.173+0000 7f2de6ffd640 1 -- 172.21.15.42:0/2265941766 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f2df80053c0 con 0x559b0e283280 2024-08-30T08:59:58.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.173+0000 7f2e1413f740 10 client.15279 fetch_fsmap learned FSMap version 26 2024-08-30T08:59:58.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.173+0000 7f2e1413f740 1 -- 172.21.15.42:0/2265941766 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x559b0e6d6e70 con 0x559b0e283280 2024-08-30T08:59:58.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.174+0000 7f2de6ffd640 1 -- 172.21.15.42:0/2265941766 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2dfc004de0 con 0x559b0e283280 2024-08-30T08:59:58.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.174+0000 7f2de6ffd640 1 -- 172.21.15.42:0/2265941766 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f2df8004aa0 con 0x559b0e283280 2024-08-30T08:59:58.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.174+0000 7f2e1413f740 10 client.15279 fetch_fsmap finished waiting for FSMap version 26 2024-08-30T08:59:58.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.174+0000 7f2e1413f740 10 client.15279 resolve_mds: resolved name 'b' to daemon mds.b 2024-08-30T08:59:58.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.174+0000 7f2e1413f740 20 client.15279 populate_metadata read hostname 'smithi042' 2024-08-30T08:59:58.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.174+0000 7f2e1413f740 1 --2- 172.21.15.42:0/2265941766 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x559b0e237c90 0x559b0e6da7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:58.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.174+0000 7f2e1413f740 4 client.15279 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "status"}] 2024-08-30T08:59:58.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.174+0000 7f2e1413f740 1 -- 172.21.15.42:0/2265941766 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "status"}) -- 0x559b0e5bdde0 con 0x559b0e237c90 2024-08-30T08:59:58.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.174+0000 7f2e06ffd640 1 --2- 172.21.15.42:0/2265941766 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x559b0e237c90 0x559b0e6da7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:58.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.175+0000 7f2e06ffd640 1 --2- 172.21.15.42:0/2265941766 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x559b0e237c90 0x559b0e6da7b0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ab98224a7388c63d server_cookie=6ddd402279c7ea69 in_seq=0 out_seq=0 2024-08-30T08:59:58.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.175+0000 7f2de6ffd640 10 client.15279 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T08:59:58.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.175+0000 7f2de6ffd640 1 -- 172.21.15.42:0/2265941766 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x559b0e5bdde0 con 0x559b0e237c90 2024-08-30T08:59:58.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.175+0000 7f2de6ffd640 10 client.15279 handle_command_reply: tid=0 2024-08-30T08:59:58.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.175+0000 7f2e1413f740 1 client.15279 shutdown 2024-08-30T08:59:58.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.175+0000 7f2e1413f740 1 -- 172.21.15.42:0/2265941766 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559b0e283280 msgr2=0x559b0e66fea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:58.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.175+0000 7f2e1413f740 1 --2- 172.21.15.42:0/2265941766 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559b0e283280 0x559b0e66fea0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f2df8004700 tx=0x7f2df8010a30 comp rx=0 tx=0).stop 2024-08-30T08:59:58.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.175+0000 7f2e1413f740 1 -- 172.21.15.42:0/2265941766 shutdown_connections 2024-08-30T08:59:58.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.175+0000 7f2e1413f740 1 --2- 172.21.15.42:0/2265941766 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x559b0e237c90 0x559b0e6da7b0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:58.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.175+0000 7f2e1413f740 1 --2- 172.21.15.42:0/2265941766 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559b0e2a3f60 0x559b0e6703e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:58.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.175+0000 7f2e1413f740 1 --2- 172.21.15.42:0/2265941766 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559b0e283280 0x559b0e66fea0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:58.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.175+0000 7f2e1413f740 1 --2- 172.21.15.42:0/2265941766 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559b0e27b0a0 0x559b0e66f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:58.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.175+0000 7f2e1413f740 1 -- 172.21.15.42:0/2265941766 >> 172.21.15.42:0/2265941766 conn(0x559b0e298350 msgr2=0x559b0e65a6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:59:58.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.176+0000 7f2de6ffd640 0 client.15279 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T08:59:58.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.176+0000 7f2e1413f740 1 -- 172.21.15.42:0/2265941766 shutdown_connections 2024-08-30T08:59:58.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.176+0000 7f2e1413f740 1 -- 172.21.15.42:0/2265941766 wait complete. 2024-08-30T08:59:58.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.176+0000 7f2e1413f740 20 client.15279 trim_cache size 0 max 16384 2024-08-30T08:59:58.183 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-08-30T08:59:58.183 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_fsid": "b0895c1a-66ac-11ef-bcd4-c7b262605968", 2024-08-30T08:59:58.183 INFO:teuthology.orchestra.run.smithi042.stdout: "whoami": 0, 2024-08-30T08:59:58.183 INFO:teuthology.orchestra.run.smithi042.stdout: "id": 24629, 2024-08-30T08:59:58.183 INFO:teuthology.orchestra.run.smithi042.stdout: "want_state": "up:active", 2024-08-30T08:59:58.183 INFO:teuthology.orchestra.run.smithi042.stdout: "state": "up:active", 2024-08-30T08:59:58.183 INFO:teuthology.orchestra.run.smithi042.stdout: "fs_name": "cephfs", 2024-08-30T08:59:58.183 INFO:teuthology.orchestra.run.smithi042.stdout: "rank_uptime": 114.776168986, 2024-08-30T08:59:58.183 INFO:teuthology.orchestra.run.smithi042.stdout: "mdsmap_epoch": 26, 2024-08-30T08:59:58.183 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_epoch": 83, 2024-08-30T08:59:58.183 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_epoch_barrier": 81, 2024-08-30T08:59:58.183 INFO:teuthology.orchestra.run.smithi042.stdout: "uptime": 125.54450105799999 2024-08-30T08:59:58.183 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-08-30T08:59:58.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.176+0000 7f2e0dbca640 1 -- 172.21.15.42:0/2462416288 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f2df00b4c40 msgr2=0x7f2df00b7150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:58.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.176+0000 7f2e0dbca640 1 --2- 172.21.15.42:0/2462416288 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f2df00b4c40 0x7f2df00b7150 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f2e000a6290 tx=0x7f2e00077e90 comp rx=0 tx=0).stop 2024-08-30T08:59:58.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.176+0000 7f2e0dbca640 1 -- 172.21.15.42:0/2462416288 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2e08107a50 msgr2=0x7f2e0819fb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:58.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.176+0000 7f2e0dbca640 1 --2- 172.21.15.42:0/2462416288 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2e08107a50 0x7f2e0819fb30 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f2e0000cad0 tx=0x7f2e0000cf90 comp rx=0 tx=0).stop 2024-08-30T08:59:58.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.176+0000 7f2e0dbca640 1 -- 172.21.15.42:0/2462416288 shutdown_connections 2024-08-30T08:59:58.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.176+0000 7f2e0dbca640 1 --2- 172.21.15.42:0/2462416288 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f2df00b4c40 0x7f2df00b7150 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:58.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.176+0000 7f2e0dbca640 1 --2- 172.21.15.42:0/2462416288 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2e08107a50 0x7f2e0819fb30 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:58.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.176+0000 7f2e0dbca640 1 --2- 172.21.15.42:0/2462416288 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2e08107100 0x7f2e0819a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:58.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.176+0000 7f2e0dbca640 1 --2- 172.21.15.42:0/2462416288 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2e08101f00 0x7f2e0819a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:58.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.176+0000 7f2e0dbca640 1 -- 172.21.15.42:0/2462416288 >> 172.21.15.42:0/2462416288 conn(0x7f2e08079c40 msgr2=0x7f2e08105fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:59:58.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.176+0000 7f2e0dbca640 1 -- 172.21.15.42:0/2462416288 shutdown_connections 2024-08-30T08:59:58.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:58.176+0000 7f2e0dbca640 1 -- 172.21.15.42:0/2462416288 wait complete. 2024-08-30T08:59:58.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 08:59:58 smithi042 ceph-mon[18652]: pgmap v349: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 6 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-08-30T08:59:58.978 INFO:tasks.cephadm:Setup complete, yielding 2024-08-30T08:59:58.978 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-08-30T08:59:58.989 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi042.front.sepia.ceph.com 2024-08-30T08:59:58.989 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- bash -c 'ceph orch status' 2024-08-30T08:59:59.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 08:59:58 smithi167 ceph-mon[22063]: pgmap v349: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 6 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-08-30T08:59:59.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 08:59:58 smithi190 ceph-mon[22001]: pgmap v349: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 6 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-08-30T08:59:59.248 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T08:59:59.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:59.746+0000 7f67331b8640 1 -- 172.21.15.42:0/3238218736 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f672c1034a0 msgr2=0x7f672c1038b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:59.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:59.746+0000 7f67331b8640 1 --2- 172.21.15.42:0/3238218736 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f672c1034a0 0x7f672c1038b0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f6718009900 tx=0x7f671802a480 comp rx=0 tx=0).stop 2024-08-30T08:59:59.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:59.746+0000 7f67331b8640 1 -- 172.21.15.42:0/3238218736 shutdown_connections 2024-08-30T08:59:59.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:59.746+0000 7f67331b8640 1 --2- 172.21.15.42:0/3238218736 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f672c103e40 0x7f672c10a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:59.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:59.746+0000 7f67331b8640 1 --2- 172.21.15.42:0/3238218736 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f672c1034a0 0x7f672c1038b0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:59.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:59.746+0000 7f67331b8640 1 --2- 172.21.15.42:0/3238218736 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f672c102410 0x7f672c102880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:59.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:59.746+0000 7f67331b8640 1 -- 172.21.15.42:0/3238218736 >> 172.21.15.42:0/3238218736 conn(0x7f672c0fdd80 msgr2=0x7f672c100230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:59:59.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:59.746+0000 7f67331b8640 1 -- 172.21.15.42:0/3238218736 shutdown_connections 2024-08-30T08:59:59.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:59.746+0000 7f67331b8640 1 -- 172.21.15.42:0/3238218736 wait complete. 2024-08-30T08:59:59.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:59.747+0000 7f67331b8640 1 Processor -- start 2024-08-30T08:59:59.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:59.747+0000 7f67331b8640 1 -- start start 2024-08-30T08:59:59.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:59.747+0000 7f67331b8640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f672c102410 0x7f672c198cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:59.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:59.747+0000 7f67331b8640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f672c1034a0 0x7f672c199340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:59.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:59.747+0000 7f67331b8640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f672c103e40 0x7f672c199e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:59.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:59.747+0000 7f67331b8640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f672c10e070 con 0x7f672c103e40 2024-08-30T08:59:59.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:59.747+0000 7f67331b8640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f672c10def0 con 0x7f672c102410 2024-08-30T08:59:59.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:59.747+0000 7f67331b8640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f672c10e1f0 con 0x7f672c1034a0 2024-08-30T08:59:59.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:59.747+0000 7f6731755640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f672c103e40 0x7f672c199e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:59.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:59.747+0000 7f6731755640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f672c103e40 0x7f672c199e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43146/0 (socket says 172.21.15.42:43146) 2024-08-30T08:59:59.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:59.747+0000 7f6731755640 1 -- 172.21.15.42:0/3955799321 learned_addr learned my addr 172.21.15.42:0/3955799321 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T08:59:59.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:59.747+0000 7f6730f54640 1 --2- 172.21.15.42:0/3955799321 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f672c102410 0x7f672c198cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:59.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:59.747+0000 7f672bfff640 1 --2- 172.21.15.42:0/3955799321 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f672c1034a0 0x7f672c199340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:59.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:59.748+0000 7f6731755640 1 -- 172.21.15.42:0/3955799321 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f672c1034a0 msgr2=0x7f672c199340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:59.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:59.748+0000 7f6731755640 1 --2- 172.21.15.42:0/3955799321 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f672c1034a0 0x7f672c199340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:59.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:59.748+0000 7f6731755640 1 -- 172.21.15.42:0/3955799321 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f672c102410 msgr2=0x7f672c198cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:59.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:59.748+0000 7f6731755640 1 --2- 172.21.15.42:0/3955799321 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f672c102410 0x7f672c198cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:59.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:59.748+0000 7f6731755640 1 -- 172.21.15.42:0/3955799321 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f672c19a690 con 0x7f672c103e40 2024-08-30T08:59:59.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:59.748+0000 7f6731755640 1 --2- 172.21.15.42:0/3955799321 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f672c103e40 0x7f672c199e70 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f672400ebb0 tx=0x7f6724007590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:59:59.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:59.748+0000 7f6729ffb640 1 -- 172.21.15.42:0/3955799321 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6724007ca0 con 0x7f672c103e40 2024-08-30T08:59:59.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:59.748+0000 7f6729ffb640 1 -- 172.21.15.42:0/3955799321 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6724007e40 con 0x7f672c103e40 2024-08-30T08:59:59.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:59.748+0000 7f67331b8640 1 -- 172.21.15.42:0/3955799321 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f672c19a920 con 0x7f672c103e40 2024-08-30T08:59:59.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:59.748+0000 7f6729ffb640 1 -- 172.21.15.42:0/3955799321 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67240057d0 con 0x7f672c103e40 2024-08-30T08:59:59.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:59.748+0000 7f67331b8640 1 -- 172.21.15.42:0/3955799321 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f672c06bab0 con 0x7f672c103e40 2024-08-30T08:59:59.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:59.750+0000 7f67331b8640 1 -- 172.21.15.42:0/3955799321 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f672c102880 con 0x7f672c103e40 2024-08-30T08:59:59.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:59.751+0000 7f6729ffb640 1 -- 172.21.15.42:0/3955799321 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) ==== 152628+0+0 (secure 0 0 0) 0x7f6724004a40 con 0x7f672c103e40 2024-08-30T08:59:59.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:59.752+0000 7f6729ffb640 1 --2- 172.21.15.42:0/3955799321 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f671c0b4bc0 0x7f671c0b70d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T08:59:59.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:59.752+0000 7f6730f54640 1 --2- 172.21.15.42:0/3955799321 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f671c0b4bc0 0x7f671c0b70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T08:59:59.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:59.752+0000 7f6729ffb640 1 -- 172.21.15.42:0/3955799321 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f67240e6b30 con 0x7f672c103e40 2024-08-30T08:59:59.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:59.752+0000 7f6730f54640 1 --2- 172.21.15.42:0/3955799321 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f671c0b4bc0 0x7f671c0b70d0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f672000bbe0 tx=0x7f672000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T08:59:59.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:59.757+0000 7f6729ffb640 1 -- 172.21.15.42:0/3955799321 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f67240ad120 con 0x7f672c103e40 2024-08-30T08:59:59.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:59.959+0000 7f67331b8640 1 -- 172.21.15.42:0/3955799321 --> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7f672c109100 con 0x7f671c0b4bc0 2024-08-30T08:59:59.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:59.960+0000 7f6729ffb640 1 -- 172.21.15.42:0/3955799321 <== mgr.14150 v2:172.21.15.42:6800/3005140902 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f672c109100 con 0x7f671c0b4bc0 2024-08-30T08:59:59.962 INFO:teuthology.orchestra.run.smithi042.stdout:Backend: cephadm 2024-08-30T08:59:59.962 INFO:teuthology.orchestra.run.smithi042.stdout:Available: Yes 2024-08-30T08:59:59.962 INFO:teuthology.orchestra.run.smithi042.stdout:Paused: No 2024-08-30T08:59:59.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:59.963+0000 7f67331b8640 1 -- 172.21.15.42:0/3955799321 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f671c0b4bc0 msgr2=0x7f671c0b70d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:59.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:59.963+0000 7f67331b8640 1 --2- 172.21.15.42:0/3955799321 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f671c0b4bc0 0x7f671c0b70d0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f672000bbe0 tx=0x7f672000b390 comp rx=0 tx=0).stop 2024-08-30T08:59:59.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:59.963+0000 7f67331b8640 1 -- 172.21.15.42:0/3955799321 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f672c103e40 msgr2=0x7f672c199e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T08:59:59.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:59.963+0000 7f67331b8640 1 --2- 172.21.15.42:0/3955799321 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f672c103e40 0x7f672c199e70 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f672400ebb0 tx=0x7f6724007590 comp rx=0 tx=0).stop 2024-08-30T08:59:59.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:59.963+0000 7f67331b8640 1 -- 172.21.15.42:0/3955799321 shutdown_connections 2024-08-30T08:59:59.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:59.963+0000 7f67331b8640 1 --2- 172.21.15.42:0/3955799321 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f671c0b4bc0 0x7f671c0b70d0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:59.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:59.963+0000 7f67331b8640 1 --2- 172.21.15.42:0/3955799321 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f672c1034a0 0x7f672c199340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:59.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:59.963+0000 7f67331b8640 1 --2- 172.21.15.42:0/3955799321 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f672c103e40 0x7f672c199e70 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:59.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:59.963+0000 7f67331b8640 1 --2- 172.21.15.42:0/3955799321 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f672c102410 0x7f672c198cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T08:59:59.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:59.963+0000 7f67331b8640 1 -- 172.21.15.42:0/3955799321 >> 172.21.15.42:0/3955799321 conn(0x7f672c0fdd80 msgr2=0x7f672c1067c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T08:59:59.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:59.963+0000 7f67331b8640 1 -- 172.21.15.42:0/3955799321 shutdown_connections 2024-08-30T08:59:59.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T08:59:59.963+0000 7f67331b8640 1 -- 172.21.15.42:0/3955799321 wait complete. 2024-08-30T09:00:00.641 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- bash -c 'ceph orch ps' 2024-08-30T09:00:00.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:00:00 smithi042 ceph-mon[18652]: pgmap v350: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s; 0 B/s, 0 objects/s recovering 2024-08-30T09:00:00.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:00:00 smithi042 ceph-mon[18652]: overall HEALTH_OK 2024-08-30T09:00:00.895 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T09:00:01.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:00:00 smithi167 ceph-mon[22063]: pgmap v350: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s; 0 B/s, 0 objects/s recovering 2024-08-30T09:00:01.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:00:00 smithi167 ceph-mon[22063]: overall HEALTH_OK 2024-08-30T09:00:01.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:00:00 smithi190 ceph-mon[22001]: pgmap v350: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s; 0 B/s, 0 objects/s recovering 2024-08-30T09:00:01.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:00:00 smithi190 ceph-mon[22001]: overall HEALTH_OK 2024-08-30T09:00:01.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:01.400+0000 7f24c8c9c640 1 -- 172.21.15.42:0/2382208610 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f24c4103510 msgr2=0x7f24c4103980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:00:01.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:01.400+0000 7f24c8c9c640 1 --2- 172.21.15.42:0/2382208610 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f24c4103510 0x7f24c4103980 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f24b4009900 tx=0x7f24b402a490 comp rx=0 tx=0).stop 2024-08-30T09:00:01.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:01.400+0000 7f24c8c9c640 1 -- 172.21.15.42:0/2382208610 shutdown_connections 2024-08-30T09:00:01.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:01.400+0000 7f24c8c9c640 1 --2- 172.21.15.42:0/2382208610 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f24c41045a0 0x7f24c41049b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:01.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:01.400+0000 7f24c8c9c640 1 --2- 172.21.15.42:0/2382208610 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f24c4104f40 0x7f24c410b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:01.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:01.400+0000 7f24c8c9c640 1 --2- 172.21.15.42:0/2382208610 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f24c4103510 0x7f24c4103980 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:01.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:01.400+0000 7f24c8c9c640 1 -- 172.21.15.42:0/2382208610 >> 172.21.15.42:0/2382208610 conn(0x7f24c40fee80 msgr2=0x7f24c4101330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:00:01.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:01.400+0000 7f24c8c9c640 1 -- 172.21.15.42:0/2382208610 shutdown_connections 2024-08-30T09:00:01.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:01.400+0000 7f24c8c9c640 1 -- 172.21.15.42:0/2382208610 wait complete. 2024-08-30T09:00:01.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:01.400+0000 7f24c8c9c640 1 Processor -- start 2024-08-30T09:00:01.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:01.401+0000 7f24c8c9c640 1 -- start start 2024-08-30T09:00:01.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:01.401+0000 7f24c8c9c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f24c4103510 0x7f24c4076be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:01.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:01.401+0000 7f24c8c9c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f24c41045a0 0x7f24c40754b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:01.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:01.401+0000 7f24c8c9c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f24c4104f40 0x7f24c40759f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:01.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:01.401+0000 7f24c8c9c640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f24c410f130 con 0x7f24c4103510 2024-08-30T09:00:01.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:01.401+0000 7f24c8c9c640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f24c410efb0 con 0x7f24c4104f40 2024-08-30T09:00:01.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:01.401+0000 7f24c8c9c640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f24c410f2b0 con 0x7f24c41045a0 2024-08-30T09:00:01.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:01.401+0000 7f24c259c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f24c4103510 0x7f24c4076be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:01.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:01.401+0000 7f24c259c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f24c4103510 0x7f24c4076be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43158/0 (socket says 172.21.15.42:43158) 2024-08-30T09:00:01.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:01.401+0000 7f24c259c640 1 -- 172.21.15.42:0/4210683762 learned_addr learned my addr 172.21.15.42:0/4210683762 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:00:01.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:01.401+0000 7f24bbfff640 1 --2- 172.21.15.42:0/4210683762 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f24c41045a0 0x7f24c40754b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:01.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:01.401+0000 7f24c2d9d640 1 --2- 172.21.15.42:0/4210683762 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f24c4104f40 0x7f24c40759f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:01.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:01.401+0000 7f24c259c640 1 -- 172.21.15.42:0/4210683762 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f24c41045a0 msgr2=0x7f24c40754b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:00:01.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:01.401+0000 7f24c259c640 1 --2- 172.21.15.42:0/4210683762 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f24c41045a0 0x7f24c40754b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:01.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:01.401+0000 7f24c259c640 1 -- 172.21.15.42:0/4210683762 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f24c4104f40 msgr2=0x7f24c40759f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:00:01.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:01.401+0000 7f24c259c640 1 --2- 172.21.15.42:0/4210683762 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f24c4104f40 0x7f24c40759f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:01.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:01.401+0000 7f24c259c640 1 -- 172.21.15.42:0/4210683762 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24c40760f0 con 0x7f24c4103510 2024-08-30T09:00:01.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:01.402+0000 7f24c259c640 1 --2- 172.21.15.42:0/4210683762 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f24c4103510 0x7f24c4076be0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f24b4006010 tx=0x7f24b4004340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:00:01.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:01.402+0000 7f24bb7fe640 1 -- 172.21.15.42:0/4210683762 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24b403f030 con 0x7f24c4103510 2024-08-30T09:00:01.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:01.402+0000 7f24c8c9c640 1 -- 172.21.15.42:0/4210683762 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24c41a3df0 con 0x7f24c4103510 2024-08-30T09:00:01.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:01.402+0000 7f24bb7fe640 1 -- 172.21.15.42:0/4210683762 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f24b4003cb0 con 0x7f24c4103510 2024-08-30T09:00:01.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:01.402+0000 7f24bb7fe640 1 -- 172.21.15.42:0/4210683762 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24b4035070 con 0x7f24c4103510 2024-08-30T09:00:01.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:01.402+0000 7f24c8c9c640 1 -- 172.21.15.42:0/4210683762 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24c41a4190 con 0x7f24c4103510 2024-08-30T09:00:01.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:01.403+0000 7f24c8c9c640 1 -- 172.21.15.42:0/4210683762 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24c4103980 con 0x7f24c4103510 2024-08-30T09:00:01.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:01.406+0000 7f24bb7fe640 1 -- 172.21.15.42:0/4210683762 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) ==== 152628+0+0 (secure 0 0 0) 0x7f24b4042030 con 0x7f24c4103510 2024-08-30T09:00:01.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:01.407+0000 7f24bb7fe640 1 --2- 172.21.15.42:0/4210683762 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f24b00b4bc0 0x7f24b00b70d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:01.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:01.407+0000 7f24bb7fe640 1 -- 172.21.15.42:0/4210683762 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f24b41062e0 con 0x7f24c4103510 2024-08-30T09:00:01.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:01.407+0000 7f24bbfff640 1 --2- 172.21.15.42:0/4210683762 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f24b00b4bc0 0x7f24b00b70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:01.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:01.407+0000 7f24bbfff640 1 --2- 172.21.15.42:0/4210683762 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f24b00b4bc0 0x7f24b00b70d0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f24ac00bbe0 tx=0x7f24ac00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:00:01.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:01.410+0000 7f24bb7fe640 1 -- 172.21.15.42:0/4210683762 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f24b4003e60 con 0x7f24c4103510 2024-08-30T09:00:01.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:01.607+0000 7f24c8c9c640 1 -- 172.21.15.42:0/4210683762 --> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f24c40635a0 con 0x7f24b00b4bc0 2024-08-30T09:00:01.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:01.620+0000 7f24bb7fe640 1 -- 172.21.15.42:0/4210683762 <== mgr.14150 v2:172.21.15.42:6800/3005140902 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4123 (secure 0 0 0) 0x7f24c40635a0 con 0x7f24b00b4bc0 2024-08-30T09:00:01.622 INFO:teuthology.orchestra.run.smithi042.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-08-30T09:00:01.622 INFO:teuthology.orchestra.run.smithi042.stdout:mds.a smithi042 running (2m) 60s ago 2m 82.0M - 19.3.0-4560-gd19e6118 79f4cde5e369 f14af3600daf 2024-08-30T09:00:01.622 INFO:teuthology.orchestra.run.smithi042.stdout:mds.b smithi167 running (2m) 61s ago 2m 93.1M - 19.3.0-4560-gd19e6118 79f4cde5e369 7ec7569892e3 2024-08-30T09:00:01.622 INFO:teuthology.orchestra.run.smithi042.stdout:mds.c smithi190 running (2m) 61s ago 2m 83.1M - 19.3.0-4560-gd19e6118 79f4cde5e369 0526a2005b1e 2024-08-30T09:00:01.622 INFO:teuthology.orchestra.run.smithi042.stdout:mds.d smithi042 running (2m) 60s ago 2m 81.8M - 19.3.0-4560-gd19e6118 79f4cde5e369 e53c84315db0 2024-08-30T09:00:01.622 INFO:teuthology.orchestra.run.smithi042.stdout:mds.e smithi167 running (2m) 61s ago 2m 82.2M - 19.3.0-4560-gd19e6118 79f4cde5e369 7bbee5d4b3ee 2024-08-30T09:00:01.622 INFO:teuthology.orchestra.run.smithi042.stdout:mds.f smithi190 running (2m) 61s ago 2m 83.0M - 19.3.0-4560-gd19e6118 79f4cde5e369 5f43b96cf342 2024-08-30T09:00:01.623 INFO:teuthology.orchestra.run.smithi042.stdout:mds.g smithi042 running (2m) 60s ago 2m 81.4M - 19.3.0-4560-gd19e6118 79f4cde5e369 83975ccb3d1e 2024-08-30T09:00:01.623 INFO:teuthology.orchestra.run.smithi042.stdout:mds.h smithi167 running (2m) 61s ago 2m 84.6M - 19.3.0-4560-gd19e6118 79f4cde5e369 ed5d7ebb5670 2024-08-30T09:00:01.623 INFO:teuthology.orchestra.run.smithi042.stdout:mds.i smithi190 running (2m) 61s ago 2m 84.6M - 19.3.0-4560-gd19e6118 79f4cde5e369 0987ddbec8ab 2024-08-30T09:00:01.623 INFO:teuthology.orchestra.run.smithi042.stdout:mds.j smithi042 running (2m) 60s ago 2m 81.7M - 19.3.0-4560-gd19e6118 79f4cde5e369 f4e7191e9658 2024-08-30T09:00:01.623 INFO:teuthology.orchestra.run.smithi042.stdout:mds.k smithi167 running (2m) 61s ago 2m 82.2M - 19.3.0-4560-gd19e6118 79f4cde5e369 d26c861d4752 2024-08-30T09:00:01.623 INFO:teuthology.orchestra.run.smithi042.stdout:mds.l smithi190 running (2m) 61s ago 2m 81.7M - 19.3.0-4560-gd19e6118 79f4cde5e369 b0102237447b 2024-08-30T09:00:01.623 INFO:teuthology.orchestra.run.smithi042.stdout:mgr.x smithi042 *:9283,8765 running (9m) 60s ago 9m 591M - 19.3.0-4560-gd19e6118 79f4cde5e369 c24e99374a5d 2024-08-30T09:00:01.623 INFO:teuthology.orchestra.run.smithi042.stdout:mgr.y smithi167 *:8443,8765 running (8m) 61s ago 8m 522M - 19.3.0-4560-gd19e6118 79f4cde5e369 f497860332d6 2024-08-30T09:00:01.623 INFO:teuthology.orchestra.run.smithi042.stdout:mgr.z smithi190 *:8443,8765 running (8m) 61s ago 8m 522M - 19.3.0-4560-gd19e6118 79f4cde5e369 4d9568cd493d 2024-08-30T09:00:01.623 INFO:teuthology.orchestra.run.smithi042.stdout:mon.a smithi042 running (10m) 60s ago 10m 118M 2048M 19.3.0-4560-gd19e6118 79f4cde5e369 e677a01ba2f7 2024-08-30T09:00:01.623 INFO:teuthology.orchestra.run.smithi042.stdout:mon.b smithi167 running (8m) 61s ago 8m 109M 2048M 19.3.0-4560-gd19e6118 79f4cde5e369 124e20337385 2024-08-30T09:00:01.623 INFO:teuthology.orchestra.run.smithi042.stdout:mon.c smithi190 running (8m) 61s ago 8m 109M 2048M 19.3.0-4560-gd19e6118 79f4cde5e369 a523197617ba 2024-08-30T09:00:01.623 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0 smithi042 running (7m) 60s ago 7m 112M 4273M 19.3.0-4560-gd19e6118 79f4cde5e369 f5c7b845b9a7 2024-08-30T09:00:01.624 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1 smithi167 running (7m) 61s ago 7m 112M 4272M 19.3.0-4560-gd19e6118 79f4cde5e369 9f014f5eae10 2024-08-30T09:00:01.624 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2 smithi190 running (6m) 61s ago 6m 121M 4272M 19.3.0-4560-gd19e6118 79f4cde5e369 ad614b27c306 2024-08-30T09:00:01.624 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3 smithi042 running (6m) 60s ago 6m 119M 4273M 19.3.0-4560-gd19e6118 79f4cde5e369 91558ede3e88 2024-08-30T09:00:01.624 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4 smithi167 running (5m) 61s ago 5m 120M 4272M 19.3.0-4560-gd19e6118 79f4cde5e369 69bb5c8c83a5 2024-08-30T09:00:01.624 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5 smithi190 running (5m) 61s ago 5m 121M 4272M 19.3.0-4560-gd19e6118 79f4cde5e369 a3343420eb38 2024-08-30T09:00:01.624 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6 smithi042 running (5m) 60s ago 5m 109M 4273M 19.3.0-4560-gd19e6118 79f4cde5e369 838c2fa5133a 2024-08-30T09:00:01.624 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7 smithi167 running (4m) 61s ago 4m 114M 4272M 19.3.0-4560-gd19e6118 79f4cde5e369 2d8d08314073 2024-08-30T09:00:01.624 INFO:teuthology.orchestra.run.smithi042.stdout:osd.8 smithi190 running (4m) 61s ago 4m 113M 4272M 19.3.0-4560-gd19e6118 79f4cde5e369 b5580defe988 2024-08-30T09:00:01.624 INFO:teuthology.orchestra.run.smithi042.stdout:osd.9 smithi042 running (3m) 60s ago 3m 117M 4273M 19.3.0-4560-gd19e6118 79f4cde5e369 440b157a7c90 2024-08-30T09:00:01.624 INFO:teuthology.orchestra.run.smithi042.stdout:osd.10 smithi167 running (3m) 61s ago 3m 114M 4272M 19.3.0-4560-gd19e6118 79f4cde5e369 192dcc3ee590 2024-08-30T09:00:01.624 INFO:teuthology.orchestra.run.smithi042.stdout:osd.11 smithi190 running (2m) 61s ago 2m 112M 4272M 19.3.0-4560-gd19e6118 79f4cde5e369 d39072212a6f 2024-08-30T09:00:01.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:01.622+0000 7f24c8c9c640 1 -- 172.21.15.42:0/4210683762 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f24b00b4bc0 msgr2=0x7f24b00b70d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:00:01.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:01.622+0000 7f24c8c9c640 1 --2- 172.21.15.42:0/4210683762 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f24b00b4bc0 0x7f24b00b70d0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f24ac00bbe0 tx=0x7f24ac00b390 comp rx=0 tx=0).stop 2024-08-30T09:00:01.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:01.622+0000 7f24c8c9c640 1 -- 172.21.15.42:0/4210683762 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f24c4103510 msgr2=0x7f24c4076be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:00:01.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:01.622+0000 7f24c8c9c640 1 --2- 172.21.15.42:0/4210683762 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f24c4103510 0x7f24c4076be0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f24b4006010 tx=0x7f24b4004340 comp rx=0 tx=0).stop 2024-08-30T09:00:01.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:01.622+0000 7f24c8c9c640 1 -- 172.21.15.42:0/4210683762 shutdown_connections 2024-08-30T09:00:01.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:01.622+0000 7f24c8c9c640 1 --2- 172.21.15.42:0/4210683762 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f24b00b4bc0 0x7f24b00b70d0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:01.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:01.622+0000 7f24c8c9c640 1 --2- 172.21.15.42:0/4210683762 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f24c41045a0 0x7f24c40754b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:01.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:01.622+0000 7f24c8c9c640 1 --2- 172.21.15.42:0/4210683762 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f24c4104f40 0x7f24c40759f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:01.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:01.622+0000 7f24c8c9c640 1 --2- 172.21.15.42:0/4210683762 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f24c4103510 0x7f24c4076be0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:01.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:01.622+0000 7f24c8c9c640 1 -- 172.21.15.42:0/4210683762 >> 172.21.15.42:0/4210683762 conn(0x7f24c40fee80 msgr2=0x7f24c41078c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:00:01.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:01.622+0000 7f24c8c9c640 1 -- 172.21.15.42:0/4210683762 shutdown_connections 2024-08-30T09:00:01.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:01.622+0000 7f24c8c9c640 1 -- 172.21.15.42:0/4210683762 wait complete. 2024-08-30T09:00:01.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:00:01 smithi042 ceph-mon[18652]: from='client.15285 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-08-30T09:00:02.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:00:01 smithi167 ceph-mon[22063]: from='client.15285 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-08-30T09:00:02.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:00:01 smithi190 ceph-mon[22001]: from='client.15285 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-08-30T09:00:02.360 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- bash -c 'ceph orch ls' 2024-08-30T09:00:02.659 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T09:00:02.962 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:00:02 smithi042 ceph-mon[18652]: pgmap v351: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-08-30T09:00:02.963 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:00:02 smithi042 ceph-mon[18652]: from='client.15291 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-08-30T09:00:02.963 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:00:02 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-30T09:00:03.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:00:02 smithi167 ceph-mon[22063]: pgmap v351: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-08-30T09:00:03.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:00:02 smithi167 ceph-mon[22063]: from='client.15291 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-08-30T09:00:03.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:00:02 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-30T09:00:03.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:03.179+0000 7f72b51b0640 1 -- 172.21.15.42:0/2979029177 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f72b01018a0 msgr2=0x7f72b0101cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:00:03.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:03.179+0000 7f72b51b0640 1 --2- 172.21.15.42:0/2979029177 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f72b01018a0 0x7f72b0101cb0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f72a0009900 tx=0x7f72a002a410 comp rx=0 tx=0).stop 2024-08-30T09:00:03.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:03.179+0000 7f72b51b0640 1 -- 172.21.15.42:0/2979029177 shutdown_connections 2024-08-30T09:00:03.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:03.179+0000 7f72b51b0640 1 --2- 172.21.15.42:0/2979029177 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f72b01022d0 0x7f72b0108ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:03.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:03.179+0000 7f72b51b0640 1 --2- 172.21.15.42:0/2979029177 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f72b0109210 0x7f72b010b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:03.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:03.179+0000 7f72b51b0640 1 --2- 172.21.15.42:0/2979029177 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f72b01018a0 0x7f72b0101cb0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:03.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:03.179+0000 7f72b51b0640 1 -- 172.21.15.42:0/2979029177 >> 172.21.15.42:0/2979029177 conn(0x7f72b00fd510 msgr2=0x7f72b00ff980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:00:03.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:03.179+0000 7f72b51b0640 1 -- 172.21.15.42:0/2979029177 shutdown_connections 2024-08-30T09:00:03.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:03.179+0000 7f72b51b0640 1 -- 172.21.15.42:0/2979029177 wait complete. 2024-08-30T09:00:03.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:03.180+0000 7f72b51b0640 1 Processor -- start 2024-08-30T09:00:03.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:03.180+0000 7f72b51b0640 1 -- start start 2024-08-30T09:00:03.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:03.180+0000 7f72b51b0640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f72b01018a0 0x7f72b0199d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:03.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:03.180+0000 7f72b51b0640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f72b01022d0 0x7f72b019a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:03.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:03.180+0000 7f72b51b0640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f72b0109210 0x7f72b019af30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:03.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:03.180+0000 7f72b51b0640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f72b010f0f0 con 0x7f72b0109210 2024-08-30T09:00:03.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:03.180+0000 7f72b51b0640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f72b010ef70 con 0x7f72b01022d0 2024-08-30T09:00:03.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:03.180+0000 7f72b51b0640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f72b010f270 con 0x7f72b01018a0 2024-08-30T09:00:03.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:03.181+0000 7f72af59e640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f72b0109210 0x7f72b019af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:03.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:03.181+0000 7f72ae59c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f72b01022d0 0x7f72b019a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:03.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:03.181+0000 7f72af59e640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f72b0109210 0x7f72b019af30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43172/0 (socket says 172.21.15.42:43172) 2024-08-30T09:00:03.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:03.181+0000 7f72ae59c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f72b01022d0 0x7f72b019a390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:40196/0 (socket says 172.21.15.42:40196) 2024-08-30T09:00:03.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:03.181+0000 7f72af59e640 1 -- 172.21.15.42:0/1731367218 learned_addr learned my addr 172.21.15.42:0/1731367218 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:00:03.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:03.181+0000 7f72aed9d640 1 --2- 172.21.15.42:0/1731367218 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f72b01018a0 0x7f72b0199d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:03.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:03.181+0000 7f72af59e640 1 -- 172.21.15.42:0/1731367218 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f72b01018a0 msgr2=0x7f72b0199d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:00:03.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:03.181+0000 7f72af59e640 1 --2- 172.21.15.42:0/1731367218 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f72b01018a0 0x7f72b0199d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:03.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:03.181+0000 7f72af59e640 1 -- 172.21.15.42:0/1731367218 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f72b01022d0 msgr2=0x7f72b019a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:00:03.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:03.181+0000 7f72af59e640 1 --2- 172.21.15.42:0/1731367218 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f72b01022d0 0x7f72b019a390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:03.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:03.181+0000 7f72af59e640 1 -- 172.21.15.42:0/1731367218 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72b019b750 con 0x7f72b0109210 2024-08-30T09:00:03.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:03.181+0000 7f72ae59c640 1 --2- 172.21.15.42:0/1731367218 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f72b01022d0 0x7f72b019a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-30T09:00:03.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:03.181+0000 7f72aed9d640 1 --2- 172.21.15.42:0/1731367218 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f72b01018a0 0x7f72b0199d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-30T09:00:03.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:03.181+0000 7f72af59e640 1 --2- 172.21.15.42:0/1731367218 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f72b0109210 0x7f72b019af30 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f72a800eb10 tx=0x7f72a800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:00:03.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:03.182+0000 7f72a7fff640 1 -- 172.21.15.42:0/1731367218 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72a8004b20 con 0x7f72b0109210 2024-08-30T09:00:03.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:03.182+0000 7f72a7fff640 1 -- 172.21.15.42:0/1731367218 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f72a8004cc0 con 0x7f72b0109210 2024-08-30T09:00:03.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:03.182+0000 7f72b51b0640 1 -- 172.21.15.42:0/1731367218 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72b019ba40 con 0x7f72b0109210 2024-08-30T09:00:03.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:00:02 smithi190 ceph-mon[22001]: pgmap v351: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-08-30T09:00:03.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:00:02 smithi190 ceph-mon[22001]: from='client.15291 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-08-30T09:00:03.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:00:02 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-30T09:00:03.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:03.182+0000 7f72a7fff640 1 -- 172.21.15.42:0/1731367218 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72a8010670 con 0x7f72b0109210 2024-08-30T09:00:03.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:03.182+0000 7f72b51b0640 1 -- 172.21.15.42:0/1731367218 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72b01029b0 con 0x7f72b0109210 2024-08-30T09:00:03.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:03.183+0000 7f72b51b0640 1 -- 172.21.15.42:0/1731367218 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72b0101cb0 con 0x7f72b0109210 2024-08-30T09:00:03.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:03.186+0000 7f72a7fff640 1 -- 172.21.15.42:0/1731367218 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) ==== 152628+0+0 (secure 0 0 0) 0x7f72a8005000 con 0x7f72b0109210 2024-08-30T09:00:03.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:03.187+0000 7f72a7fff640 1 --2- 172.21.15.42:0/1731367218 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f729c0b4c40 0x7f729c0b7150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:03.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:03.187+0000 7f72aed9d640 1 --2- 172.21.15.42:0/1731367218 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f729c0b4c40 0x7f729c0b7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:03.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:03.187+0000 7f72a7fff640 1 -- 172.21.15.42:0/1731367218 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f72a80e5d70 con 0x7f72b0109210 2024-08-30T09:00:03.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:03.188+0000 7f72aed9d640 1 --2- 172.21.15.42:0/1731367218 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f729c0b4c40 0x7f729c0b7150 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f72a0000c00 tx=0x7f72a0031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:00:03.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:03.190+0000 7f72a7fff640 1 -- 172.21.15.42:0/1731367218 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f72a8056130 con 0x7f72b0109210 2024-08-30T09:00:03.397 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:03.395+0000 7f72b51b0640 1 -- 172.21.15.42:0/1731367218 --> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7f72b0108690 con 0x7f729c0b4c40 2024-08-30T09:00:03.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:03.401+0000 7f72a7fff640 1 -- 172.21.15.42:0/1731367218 <== mgr.14150 v2:172.21.15.42:6800/3005140902 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7f72b0108690 con 0x7f729c0b4c40 2024-08-30T09:00:03.403 INFO:teuthology.orchestra.run.smithi042.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-08-30T09:00:03.403 INFO:teuthology.orchestra.run.smithi042.stdout:mds.all 12/12 63s ago 2m smithi042=a;smithi042=d;smithi042=g;smithi042=j;smithi167=b;smithi167=e;smithi167=h;smithi167=k;smithi190=c;smithi190=f;smithi190=i;smithi190=l;count:12 2024-08-30T09:00:03.403 INFO:teuthology.orchestra.run.smithi042.stdout:mgr 3/3 63s ago 8m smithi042=x;smithi167=y;smithi190=z;count:3 2024-08-30T09:00:03.403 INFO:teuthology.orchestra.run.smithi042.stdout:mon 3/3 63s ago 8m smithi042:172.21.15.42=a;smithi167:172.21.15.167=b;smithi190:172.21.15.190=c;count:3 2024-08-30T09:00:03.403 INFO:teuthology.orchestra.run.smithi042.stdout:osd 12 63s ago - 2024-08-30T09:00:03.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:03.403+0000 7f72b51b0640 1 -- 172.21.15.42:0/1731367218 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f729c0b4c40 msgr2=0x7f729c0b7150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:00:03.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:03.403+0000 7f72b51b0640 1 --2- 172.21.15.42:0/1731367218 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f729c0b4c40 0x7f729c0b7150 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f72a0000c00 tx=0x7f72a0031400 comp rx=0 tx=0).stop 2024-08-30T09:00:03.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:03.403+0000 7f72b51b0640 1 -- 172.21.15.42:0/1731367218 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f72b0109210 msgr2=0x7f72b019af30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:00:03.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:03.403+0000 7f72b51b0640 1 --2- 172.21.15.42:0/1731367218 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f72b0109210 0x7f72b019af30 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f72a800eb10 tx=0x7f72a800c2d0 comp rx=0 tx=0).stop 2024-08-30T09:00:03.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:03.403+0000 7f72b51b0640 1 -- 172.21.15.42:0/1731367218 shutdown_connections 2024-08-30T09:00:03.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:03.403+0000 7f72b51b0640 1 --2- 172.21.15.42:0/1731367218 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f729c0b4c40 0x7f729c0b7150 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:03.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:03.403+0000 7f72b51b0640 1 --2- 172.21.15.42:0/1731367218 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f72b0109210 0x7f72b019af30 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:03.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:03.403+0000 7f72b51b0640 1 --2- 172.21.15.42:0/1731367218 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f72b01022d0 0x7f72b019a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:03.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:03.403+0000 7f72b51b0640 1 --2- 172.21.15.42:0/1731367218 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f72b01018a0 0x7f72b0199d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:03.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:03.403+0000 7f72b51b0640 1 -- 172.21.15.42:0/1731367218 >> 172.21.15.42:0/1731367218 conn(0x7f72b00fd510 msgr2=0x7f72b010b630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:00:03.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:03.403+0000 7f72b51b0640 1 -- 172.21.15.42:0/1731367218 shutdown_connections 2024-08-30T09:00:03.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:03.403+0000 7f72b51b0640 1 -- 172.21.15.42:0/1731367218 wait complete. 2024-08-30T09:00:04.714 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- bash -c 'ceph orch host ls' 2024-08-30T09:00:05.019 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T09:00:05.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:00:04 smithi167 ceph-mon[22063]: pgmap v352: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-08-30T09:00:05.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:00:04 smithi167 ceph-mon[22063]: from='client.15297 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-08-30T09:00:05.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:00:04 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T09:00:05.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:00:04 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T09:00:05.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:00:04 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T09:00:05.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:00:04 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T09:00:05.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:00:04 smithi042 ceph-mon[18652]: pgmap v352: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-08-30T09:00:05.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:00:04 smithi042 ceph-mon[18652]: from='client.15297 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-08-30T09:00:05.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:00:04 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T09:00:05.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:00:04 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T09:00:05.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:00:04 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T09:00:05.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:00:04 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T09:00:05.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:00:04 smithi190 ceph-mon[22001]: pgmap v352: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-08-30T09:00:05.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:00:04 smithi190 ceph-mon[22001]: from='client.15297 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-08-30T09:00:05.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:00:04 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T09:00:05.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:00:04 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T09:00:05.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:00:04 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T09:00:05.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:00:04 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T09:00:05.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:05.511+0000 7f7dd7802640 1 -- 172.21.15.42:0/956533260 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7dd0103e10 msgr2=0x7f7dd010a7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:00:05.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:05.511+0000 7f7dd7802640 1 --2- 172.21.15.42:0/956533260 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7dd0103e10 0x7f7dd010a7b0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f7dc8009ca0 tx=0x7f7dc802a950 comp rx=0 tx=0).stop 2024-08-30T09:00:05.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:05.511+0000 7f7dd7802640 1 -- 172.21.15.42:0/956533260 shutdown_connections 2024-08-30T09:00:05.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:05.511+0000 7f7dd7802640 1 --2- 172.21.15.42:0/956533260 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7dd0103470 0x7f7dd0103880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:05.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:05.511+0000 7f7dd7802640 1 --2- 172.21.15.42:0/956533260 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7dd0103e10 0x7f7dd010a7b0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:05.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:05.511+0000 7f7dd7802640 1 --2- 172.21.15.42:0/956533260 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7dd01023e0 0x7f7dd0102850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:05.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:05.511+0000 7f7dd7802640 1 -- 172.21.15.42:0/956533260 >> 172.21.15.42:0/956533260 conn(0x7f7dd00fdd50 msgr2=0x7f7dd0100200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:00:05.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:05.511+0000 7f7dd7802640 1 -- 172.21.15.42:0/956533260 shutdown_connections 2024-08-30T09:00:05.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:05.511+0000 7f7dd7802640 1 -- 172.21.15.42:0/956533260 wait complete. 2024-08-30T09:00:05.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:05.512+0000 7f7dd7802640 1 Processor -- start 2024-08-30T09:00:05.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:05.512+0000 7f7dd7802640 1 -- start start 2024-08-30T09:00:05.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:05.512+0000 7f7dd7802640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7dd01023e0 0x7f7dd0198bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:05.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:05.512+0000 7f7dd7802640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7dd0103470 0x7f7dd0199230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:05.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:05.512+0000 7f7dd7802640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7dd0103e10 0x7f7dd0199dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:05.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:05.512+0000 7f7dd7802640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f7dd010e040 con 0x7f7dd01023e0 2024-08-30T09:00:05.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:05.512+0000 7f7dd7802640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f7dd010dec0 con 0x7f7dd0103470 2024-08-30T09:00:05.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:05.512+0000 7f7dd7802640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f7dd010e1c0 con 0x7f7dd0103e10 2024-08-30T09:00:05.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:05.513+0000 7f7dd559e640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7dd01023e0 0x7f7dd0198bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:05.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:05.513+0000 7f7dd559e640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7dd01023e0 0x7f7dd0198bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43194/0 (socket says 172.21.15.42:43194) 2024-08-30T09:00:05.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:05.513+0000 7f7dd559e640 1 -- 172.21.15.42:0/3294435304 learned_addr learned my addr 172.21.15.42:0/3294435304 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:00:05.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:05.513+0000 7f7dd5d9f640 1 --2- 172.21.15.42:0/3294435304 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7dd0103e10 0x7f7dd0199dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:05.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:05.513+0000 7f7dd4d9d640 1 --2- 172.21.15.42:0/3294435304 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7dd0103470 0x7f7dd0199230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:05.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:05.513+0000 7f7dd559e640 1 -- 172.21.15.42:0/3294435304 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7dd0103e10 msgr2=0x7f7dd0199dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:00:05.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:05.513+0000 7f7dd559e640 1 --2- 172.21.15.42:0/3294435304 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7dd0103e10 0x7f7dd0199dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:05.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:05.513+0000 7f7dd559e640 1 -- 172.21.15.42:0/3294435304 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7dd0103470 msgr2=0x7f7dd0199230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:00:05.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:05.513+0000 7f7dd559e640 1 --2- 172.21.15.42:0/3294435304 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7dd0103470 0x7f7dd0199230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:05.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:05.513+0000 7f7dd559e640 1 -- 172.21.15.42:0/3294435304 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7dd019a560 con 0x7f7dd01023e0 2024-08-30T09:00:05.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:05.513+0000 7f7dd4d9d640 1 --2- 172.21.15.42:0/3294435304 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7dd0103470 0x7f7dd0199230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-30T09:00:05.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:05.513+0000 7f7dd559e640 1 --2- 172.21.15.42:0/3294435304 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7dd01023e0 0x7f7dd0198bc0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f7dcc00c920 tx=0x7f7dcc00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:00:05.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:05.513+0000 7f7dc27fc640 1 -- 172.21.15.42:0/3294435304 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dcc007c70 con 0x7f7dd01023e0 2024-08-30T09:00:05.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:05.513+0000 7f7dc27fc640 1 -- 172.21.15.42:0/3294435304 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7dcc007e10 con 0x7f7dd01023e0 2024-08-30T09:00:05.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:05.513+0000 7f7dd7802640 1 -- 172.21.15.42:0/3294435304 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7dd019a850 con 0x7f7dd01023e0 2024-08-30T09:00:05.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:05.513+0000 7f7dc27fc640 1 -- 172.21.15.42:0/3294435304 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dcc018820 con 0x7f7dd01023e0 2024-08-30T09:00:05.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:05.514+0000 7f7dd7802640 1 -- 172.21.15.42:0/3294435304 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7dd01a0fe0 con 0x7f7dd01023e0 2024-08-30T09:00:05.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:05.515+0000 7f7dd7802640 1 -- 172.21.15.42:0/3294435304 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7dd0102850 con 0x7f7dd01023e0 2024-08-30T09:00:05.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:05.516+0000 7f7dc27fc640 1 -- 172.21.15.42:0/3294435304 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) ==== 152628+0+0 (secure 0 0 0) 0x7f7dcc00f450 con 0x7f7dd01023e0 2024-08-30T09:00:05.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:05.517+0000 7f7dc27fc640 1 --2- 172.21.15.42:0/3294435304 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f7dbc0b4bc0 0x7f7dbc0b70d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:05.519 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:05.517+0000 7f7dc27fc640 1 -- 172.21.15.42:0/3294435304 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f7dcc0e5a60 con 0x7f7dd01023e0 2024-08-30T09:00:05.519 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:05.517+0000 7f7dd4d9d640 1 --2- 172.21.15.42:0/3294435304 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f7dbc0b4bc0 0x7f7dbc0b70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:05.519 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:05.517+0000 7f7dd4d9d640 1 --2- 172.21.15.42:0/3294435304 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f7dbc0b4bc0 0x7f7dbc0b70d0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f7dc4005fd0 tx=0x7f7dc4005ea0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:00:05.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:05.520+0000 7f7dc27fc640 1 -- 172.21.15.42:0/3294435304 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f7dcc014030 con 0x7f7dd01023e0 2024-08-30T09:00:05.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:05.717+0000 7f7dd7802640 1 -- 172.21.15.42:0/3294435304 --> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7f7dd002e7c0 con 0x7f7dbc0b4bc0 2024-08-30T09:00:05.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:05.719+0000 7f7dc27fc640 1 -- 172.21.15.42:0/3294435304 <== mgr.14150 v2:172.21.15.42:6800/3005140902 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7f7dd002e7c0 con 0x7f7dbc0b4bc0 2024-08-30T09:00:05.721 INFO:teuthology.orchestra.run.smithi042.stdout:HOST ADDR LABELS STATUS 2024-08-30T09:00:05.721 INFO:teuthology.orchestra.run.smithi042.stdout:smithi042 172.21.15.42 2024-08-30T09:00:05.721 INFO:teuthology.orchestra.run.smithi042.stdout:smithi167 172.21.15.167 2024-08-30T09:00:05.721 INFO:teuthology.orchestra.run.smithi042.stdout:smithi190 172.21.15.190 2024-08-30T09:00:05.721 INFO:teuthology.orchestra.run.smithi042.stdout:3 hosts in cluster 2024-08-30T09:00:05.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:05.721+0000 7f7dd7802640 1 -- 172.21.15.42:0/3294435304 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f7dbc0b4bc0 msgr2=0x7f7dbc0b70d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:00:05.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:05.721+0000 7f7dd7802640 1 --2- 172.21.15.42:0/3294435304 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f7dbc0b4bc0 0x7f7dbc0b70d0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f7dc4005fd0 tx=0x7f7dc4005ea0 comp rx=0 tx=0).stop 2024-08-30T09:00:05.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:05.721+0000 7f7dd7802640 1 -- 172.21.15.42:0/3294435304 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7dd01023e0 msgr2=0x7f7dd0198bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:00:05.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:05.721+0000 7f7dd7802640 1 --2- 172.21.15.42:0/3294435304 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7dd01023e0 0x7f7dd0198bc0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f7dcc00c920 tx=0x7f7dcc00cde0 comp rx=0 tx=0).stop 2024-08-30T09:00:05.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:05.721+0000 7f7dd7802640 1 -- 172.21.15.42:0/3294435304 shutdown_connections 2024-08-30T09:00:05.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:05.721+0000 7f7dd7802640 1 --2- 172.21.15.42:0/3294435304 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f7dbc0b4bc0 0x7f7dbc0b70d0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:05.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:05.721+0000 7f7dd7802640 1 --2- 172.21.15.42:0/3294435304 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7dd0103e10 0x7f7dd0199dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:05.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:05.721+0000 7f7dd7802640 1 --2- 172.21.15.42:0/3294435304 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7dd0103470 0x7f7dd0199230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:05.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:05.721+0000 7f7dd7802640 1 --2- 172.21.15.42:0/3294435304 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7dd01023e0 0x7f7dd0198bc0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:05.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:05.721+0000 7f7dd7802640 1 -- 172.21.15.42:0/3294435304 >> 172.21.15.42:0/3294435304 conn(0x7f7dd00fdd50 msgr2=0x7f7dd0106790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:00:05.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:05.721+0000 7f7dd7802640 1 -- 172.21.15.42:0/3294435304 shutdown_connections 2024-08-30T09:00:05.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:05.721+0000 7f7dd7802640 1 -- 172.21.15.42:0/3294435304 wait complete. 2024-08-30T09:00:06.374 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- bash -c 'ceph orch device ls' 2024-08-30T09:00:06.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:00:06 smithi167 ceph-mon[22063]: pgmap v353: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-08-30T09:00:06.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:00:06 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T09:00:06.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:00:06 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T09:00:06.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:00:06 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T09:00:06.578 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:00:06 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-30T09:00:06.578 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:00:06 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T09:00:06.578 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:00:06 smithi167 ceph-mon[22063]: from='client.15303 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-08-30T09:00:06.632 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T09:00:06.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:00:06 smithi042 ceph-mon[18652]: pgmap v353: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-08-30T09:00:06.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:00:06 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T09:00:06.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:00:06 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T09:00:06.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:00:06 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T09:00:06.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:00:06 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-30T09:00:06.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:00:06 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T09:00:06.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:00:06 smithi042 ceph-mon[18652]: from='client.15303 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-08-30T09:00:06.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:00:06 smithi190 ceph-mon[22001]: pgmap v353: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-08-30T09:00:06.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:00:06 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T09:00:06.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:00:06 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T09:00:06.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:00:06 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T09:00:06.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:00:06 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-30T09:00:06.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:00:06 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T09:00:06.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:00:06 smithi190 ceph-mon[22001]: from='client.15303 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-08-30T09:00:07.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:07.138+0000 7f0533e9e640 1 -- 172.21.15.42:0/2803615072 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f052c102400 msgr2=0x7f052c102870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:00:07.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:07.138+0000 7f0533e9e640 1 --2- 172.21.15.42:0/2803615072 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f052c102400 0x7f052c102870 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f0528009900 tx=0x7f052802a480 comp rx=0 tx=0).stop 2024-08-30T09:00:07.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:07.138+0000 7f0533e9e640 1 -- 172.21.15.42:0/2803615072 shutdown_connections 2024-08-30T09:00:07.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:07.138+0000 7f0533e9e640 1 --2- 172.21.15.42:0/2803615072 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f052c103e30 0x7f052c10a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:07.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:07.138+0000 7f0533e9e640 1 --2- 172.21.15.42:0/2803615072 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f052c103490 0x7f052c1038a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:07.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:07.138+0000 7f0533e9e640 1 --2- 172.21.15.42:0/2803615072 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f052c102400 0x7f052c102870 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:07.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:07.138+0000 7f0533e9e640 1 -- 172.21.15.42:0/2803615072 >> 172.21.15.42:0/2803615072 conn(0x7f052c0fdd70 msgr2=0x7f052c100220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:00:07.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:07.138+0000 7f0533e9e640 1 -- 172.21.15.42:0/2803615072 shutdown_connections 2024-08-30T09:00:07.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:07.138+0000 7f0533e9e640 1 -- 172.21.15.42:0/2803615072 wait complete. 2024-08-30T09:00:07.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:07.138+0000 7f0533e9e640 1 Processor -- start 2024-08-30T09:00:07.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:07.139+0000 7f0533e9e640 1 -- start start 2024-08-30T09:00:07.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:07.139+0000 7f0533e9e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f052c102400 0x7f052c06be00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:07.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:07.139+0000 7f0533e9e640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f052c103490 0x7f052c19eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:07.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:07.139+0000 7f0533e9e640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f052c103e30 0x7f052c1a1050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:07.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:07.139+0000 7f0533e9e640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f052c10e060 con 0x7f052c103490 2024-08-30T09:00:07.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:07.139+0000 7f0533e9e640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f052c10dee0 con 0x7f052c103e30 2024-08-30T09:00:07.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:07.139+0000 7f0533e9e640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f052c10e1e0 con 0x7f052c102400 2024-08-30T09:00:07.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:07.139+0000 7f0531439640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f052c103490 0x7f052c19eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:07.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:07.139+0000 7f0531439640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f052c103490 0x7f052c19eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43210/0 (socket says 172.21.15.42:43210) 2024-08-30T09:00:07.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:07.139+0000 7f0531439640 1 -- 172.21.15.42:0/1025966200 learned_addr learned my addr 172.21.15.42:0/1025966200 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:00:07.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:07.139+0000 7f0531c3a640 1 --2- 172.21.15.42:0/1025966200 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f052c102400 0x7f052c06be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:07.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:07.139+0000 7f053243b640 1 --2- 172.21.15.42:0/1025966200 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f052c103e30 0x7f052c1a1050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:07.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:07.139+0000 7f0531439640 1 -- 172.21.15.42:0/1025966200 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f052c102400 msgr2=0x7f052c06be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:00:07.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:07.139+0000 7f0531439640 1 --2- 172.21.15.42:0/1025966200 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f052c102400 0x7f052c06be00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:07.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:07.139+0000 7f0531439640 1 -- 172.21.15.42:0/1025966200 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f052c103e30 msgr2=0x7f052c1a1050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:00:07.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:07.139+0000 7f0531439640 1 --2- 172.21.15.42:0/1025966200 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f052c103e30 0x7f052c1a1050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:07.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:07.139+0000 7f0531439640 1 -- 172.21.15.42:0/1025966200 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f052c1a1850 con 0x7f052c103490 2024-08-30T09:00:07.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:07.139+0000 7f0531439640 1 --2- 172.21.15.42:0/1025966200 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f052c103490 0x7f052c19eb00 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f051c00e9f0 tx=0x7f051c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:00:07.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:07.140+0000 7f0522ffd640 1 -- 172.21.15.42:0/1025966200 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f051c00cd80 con 0x7f052c103490 2024-08-30T09:00:07.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:07.140+0000 7f0522ffd640 1 -- 172.21.15.42:0/1025966200 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f051c0108e0 con 0x7f052c103490 2024-08-30T09:00:07.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:07.140+0000 7f0533e9e640 1 -- 172.21.15.42:0/1025966200 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f052c1a1b40 con 0x7f052c103490 2024-08-30T09:00:07.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:07.140+0000 7f0522ffd640 1 -- 172.21.15.42:0/1025966200 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f051c018980 con 0x7f052c103490 2024-08-30T09:00:07.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:07.140+0000 7f0533e9e640 1 -- 172.21.15.42:0/1025966200 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f052c1a2080 con 0x7f052c103490 2024-08-30T09:00:07.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:07.141+0000 7f0533e9e640 1 -- 172.21.15.42:0/1025966200 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f052c1095e0 con 0x7f052c103490 2024-08-30T09:00:07.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:07.142+0000 7f0522ffd640 1 -- 172.21.15.42:0/1025966200 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) ==== 152628+0+0 (secure 0 0 0) 0x7f051c010430 con 0x7f052c103490 2024-08-30T09:00:07.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:07.143+0000 7f0522ffd640 1 --2- 172.21.15.42:0/1025966200 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f05180b4bc0 0x7f05180b70d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:07.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:07.143+0000 7f0531c3a640 1 --2- 172.21.15.42:0/1025966200 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f05180b4bc0 0x7f05180b70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:07.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:07.144+0000 7f0522ffd640 1 -- 172.21.15.42:0/1025966200 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f051c0e5ea0 con 0x7f052c103490 2024-08-30T09:00:07.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:07.144+0000 7f0531c3a640 1 --2- 172.21.15.42:0/1025966200 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f05180b4bc0 0x7f05180b70d0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f0528000c00 tx=0x7f0528031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:00:07.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:07.147+0000 7f0522ffd640 1 -- 172.21.15.42:0/1025966200 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f051c0ac430 con 0x7f052c103490 2024-08-30T09:00:07.345 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:07.343+0000 7f0533e9e640 1 -- 172.21.15.42:0/1025966200 --> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7f052c0635a0 con 0x7f05180b4bc0 2024-08-30T09:00:07.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:07.345+0000 7f0522ffd640 1 -- 172.21.15.42:0/1025966200 <== mgr.14150 v2:172.21.15.42:6800/3005140902 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7f052c0635a0 con 0x7f05180b4bc0 2024-08-30T09:00:07.347 INFO:teuthology.orchestra.run.smithi042.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-08-30T09:00:07.347 INFO:teuthology.orchestra.run.smithi042.stdout:smithi042 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB75130021480DGN 447G No 3m ago Has a FileSystem, LVM detected 2024-08-30T09:00:07.347 INFO:teuthology.orchestra.run.smithi042.stdout:smithi167 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT62330080400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-08-30T09:00:07.347 INFO:teuthology.orchestra.run.smithi042.stdout:smithi190 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT63210012400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-08-30T09:00:07.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:07.348+0000 7f0533e9e640 1 -- 172.21.15.42:0/1025966200 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f05180b4bc0 msgr2=0x7f05180b70d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:00:07.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:07.348+0000 7f0533e9e640 1 --2- 172.21.15.42:0/1025966200 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f05180b4bc0 0x7f05180b70d0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f0528000c00 tx=0x7f0528031400 comp rx=0 tx=0).stop 2024-08-30T09:00:07.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:07.348+0000 7f0533e9e640 1 -- 172.21.15.42:0/1025966200 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f052c103490 msgr2=0x7f052c19eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:00:07.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:07.348+0000 7f0533e9e640 1 --2- 172.21.15.42:0/1025966200 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f052c103490 0x7f052c19eb00 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f051c00e9f0 tx=0x7f051c00eeb0 comp rx=0 tx=0).stop 2024-08-30T09:00:07.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:07.348+0000 7f0533e9e640 1 -- 172.21.15.42:0/1025966200 shutdown_connections 2024-08-30T09:00:07.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:07.348+0000 7f0533e9e640 1 --2- 172.21.15.42:0/1025966200 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f05180b4bc0 0x7f05180b70d0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:07.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:07.348+0000 7f0533e9e640 1 --2- 172.21.15.42:0/1025966200 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f052c103e30 0x7f052c1a1050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:07.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:07.348+0000 7f0533e9e640 1 --2- 172.21.15.42:0/1025966200 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f052c103490 0x7f052c19eb00 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:07.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:07.348+0000 7f0533e9e640 1 --2- 172.21.15.42:0/1025966200 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f052c102400 0x7f052c06be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:07.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:07.348+0000 7f0533e9e640 1 -- 172.21.15.42:0/1025966200 >> 172.21.15.42:0/1025966200 conn(0x7f052c0fdd70 msgr2=0x7f052c1067b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:00:07.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:07.348+0000 7f0533e9e640 1 -- 172.21.15.42:0/1025966200 shutdown_connections 2024-08-30T09:00:07.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:07.348+0000 7f0533e9e640 1 -- 172.21.15.42:0/1025966200 wait complete. 2024-08-30T09:00:07.990 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-08-30T09:00:07.999 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi042.front.sepia.ceph.com 2024-08-30T09:00:07.999 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- bash -c 'ceph fs dump' 2024-08-30T09:00:08.254 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T09:00:08.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:00:08 smithi167 ceph-mon[22063]: pgmap v354: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-30T09:00:08.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:00:08 smithi167 ceph-mon[22063]: from='client.15309 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-08-30T09:00:08.579 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:00:08 smithi042 ceph-mon[18652]: pgmap v354: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-30T09:00:08.579 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:00:08 smithi042 ceph-mon[18652]: from='client.15309 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-08-30T09:00:08.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:00:08 smithi190 ceph-mon[22001]: pgmap v354: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-30T09:00:08.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:00:08 smithi190 ceph-mon[22001]: from='client.15309 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-08-30T09:00:08.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:08.788+0000 7f8c47669640 1 -- 172.21.15.42:0/3066256985 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8c40104580 msgr2=0x7f8c40104990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:00:08.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:08.788+0000 7f8c47669640 1 --2- 172.21.15.42:0/3066256985 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8c40104580 0x7f8c40104990 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f8c30009900 tx=0x7f8c3002a480 comp rx=0 tx=0).stop 2024-08-30T09:00:08.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:08.788+0000 7f8c47669640 1 -- 172.21.15.42:0/3066256985 shutdown_connections 2024-08-30T09:00:08.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:08.788+0000 7f8c47669640 1 --2- 172.21.15.42:0/3066256985 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8c40104f20 0x7f8c4010b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:08.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:08.788+0000 7f8c47669640 1 --2- 172.21.15.42:0/3066256985 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8c40104580 0x7f8c40104990 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:08.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:08.788+0000 7f8c47669640 1 --2- 172.21.15.42:0/3066256985 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8c401034f0 0x7f8c40103960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:08.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:08.788+0000 7f8c47669640 1 -- 172.21.15.42:0/3066256985 >> 172.21.15.42:0/3066256985 conn(0x7f8c400fee60 msgr2=0x7f8c40101310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:00:08.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:08.788+0000 7f8c47669640 1 -- 172.21.15.42:0/3066256985 shutdown_connections 2024-08-30T09:00:08.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:08.788+0000 7f8c47669640 1 -- 172.21.15.42:0/3066256985 wait complete. 2024-08-30T09:00:08.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:08.789+0000 7f8c47669640 1 Processor -- start 2024-08-30T09:00:08.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:08.789+0000 7f8c47669640 1 -- start start 2024-08-30T09:00:08.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:08.789+0000 7f8c47669640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8c401034f0 0x7f8c40199d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:08.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:08.789+0000 7f8c47669640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8c40104580 0x7f8c4019a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:08.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:08.789+0000 7f8c47669640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8c40104f20 0x7f8c4019af70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:08.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:08.789+0000 7f8c47669640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f8c4010f110 con 0x7f8c40104580 2024-08-30T09:00:08.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:08.789+0000 7f8c47669640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f8c4010ef90 con 0x7f8c40104f20 2024-08-30T09:00:08.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:08.789+0000 7f8c47669640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f8c4010f290 con 0x7f8c401034f0 2024-08-30T09:00:08.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:08.789+0000 7f8c44c04640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8c40104580 0x7f8c4019a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:08.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:08.789+0000 7f8c44c04640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8c40104580 0x7f8c4019a3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:51438/0 (socket says 172.21.15.42:51438) 2024-08-30T09:00:08.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:08.789+0000 7f8c44c04640 1 -- 172.21.15.42:0/3805421824 learned_addr learned my addr 172.21.15.42:0/3805421824 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:00:08.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:08.789+0000 7f8c45c06640 1 --2- 172.21.15.42:0/3805421824 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8c40104f20 0x7f8c4019af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:08.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:08.789+0000 7f8c45405640 1 --2- 172.21.15.42:0/3805421824 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8c401034f0 0x7f8c40199d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:08.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:08.789+0000 7f8c44c04640 1 -- 172.21.15.42:0/3805421824 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8c401034f0 msgr2=0x7f8c40199d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:00:08.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:08.789+0000 7f8c44c04640 1 --2- 172.21.15.42:0/3805421824 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8c401034f0 0x7f8c40199d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:08.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:08.789+0000 7f8c44c04640 1 -- 172.21.15.42:0/3805421824 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8c40104f20 msgr2=0x7f8c4019af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:00:08.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:08.789+0000 7f8c44c04640 1 --2- 172.21.15.42:0/3805421824 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8c40104f20 0x7f8c4019af70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:08.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:08.789+0000 7f8c44c04640 1 -- 172.21.15.42:0/3805421824 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c4019b790 con 0x7f8c40104580 2024-08-30T09:00:08.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:08.789+0000 7f8c45c06640 1 --2- 172.21.15.42:0/3805421824 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8c40104f20 0x7f8c4019af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-30T09:00:08.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:08.789+0000 7f8c45405640 1 --2- 172.21.15.42:0/3805421824 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8c401034f0 0x7f8c40199d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-30T09:00:08.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:08.790+0000 7f8c44c04640 1 --2- 172.21.15.42:0/3805421824 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8c40104580 0x7f8c4019a3d0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f8c30006010 tx=0x7f8c30031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:00:08.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:08.790+0000 7f8c367fc640 1 -- 172.21.15.42:0/3805421824 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c3003f030 con 0x7f8c40104580 2024-08-30T09:00:08.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:08.790+0000 7f8c47669640 1 -- 172.21.15.42:0/3805421824 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c4019ba20 con 0x7f8c40104580 2024-08-30T09:00:08.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:08.790+0000 7f8c367fc640 1 -- 172.21.15.42:0/3805421824 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8c30003aa0 con 0x7f8c40104580 2024-08-30T09:00:08.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:08.790+0000 7f8c367fc640 1 -- 172.21.15.42:0/3805421824 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c30035070 con 0x7f8c40104580 2024-08-30T09:00:08.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:08.790+0000 7f8c47669640 1 -- 172.21.15.42:0/3805421824 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c401a2180 con 0x7f8c40104580 2024-08-30T09:00:08.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:08.791+0000 7f8c47669640 1 -- 172.21.15.42:0/3805421824 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c40103960 con 0x7f8c40104580 2024-08-30T09:00:08.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:08.792+0000 7f8c367fc640 1 -- 172.21.15.42:0/3805421824 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) ==== 152628+0+0 (secure 0 0 0) 0x7f8c30042030 con 0x7f8c40104580 2024-08-30T09:00:08.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:08.793+0000 7f8c367fc640 1 --2- 172.21.15.42:0/3805421824 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f8c2c0b4bc0 0x7f8c2c0b70d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:08.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:08.794+0000 7f8c45405640 1 --2- 172.21.15.42:0/3805421824 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f8c2c0b4bc0 0x7f8c2c0b70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:08.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:08.794+0000 7f8c367fc640 1 -- 172.21.15.42:0/3805421824 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f8c30105d10 con 0x7f8c40104580 2024-08-30T09:00:08.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:08.794+0000 7f8c45405640 1 --2- 172.21.15.42:0/3805421824 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f8c2c0b4bc0 0x7f8c2c0b70d0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f8c3c009b50 tx=0x7f8c3c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:00:08.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:08.798+0000 7f8c367fc640 1 -- 172.21.15.42:0/3805421824 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f8c300cc950 con 0x7f8c40104580 2024-08-30T09:00:09.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:09.022+0000 7f8c47669640 1 -- 172.21.15.42:0/3805421824 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f8c4002e730 con 0x7f8c40104580 2024-08-30T09:00:09.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:09.023+0000 7f8c367fc640 1 -- 172.21.15.42:0/3805421824 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 26 v26) ==== 76+0+3090 (secure 0 0 0) 0x7f8c300cc250 con 0x7f8c40104580 2024-08-30T09:00:09.025 INFO:teuthology.orchestra.run.smithi042.stdout:e26 2024-08-30T09:00:09.025 INFO:teuthology.orchestra.run.smithi042.stdout:btime 2024-08-30T08:58:50:586137+0000 2024-08-30T09:00:09.025 INFO:teuthology.orchestra.run.smithi042.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-08-30T09:00:09.025 INFO:teuthology.orchestra.run.smithi042.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-08-30T09:00:09.025 INFO:teuthology.orchestra.run.smithi042.stdout:legacy client fscid: 1 2024-08-30T09:00:09.025 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-08-30T09:00:09.025 INFO:teuthology.orchestra.run.smithi042.stdout:Filesystem 'cephfs' (1) 2024-08-30T09:00:09.026 INFO:teuthology.orchestra.run.smithi042.stdout:fs_name cephfs 2024-08-30T09:00:09.026 INFO:teuthology.orchestra.run.smithi042.stdout:epoch 26 2024-08-30T09:00:09.026 INFO:teuthology.orchestra.run.smithi042.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-08-30T09:00:09.026 INFO:teuthology.orchestra.run.smithi042.stdout:created 2024-08-30T08:58:03.055084+0000 2024-08-30T09:00:09.026 INFO:teuthology.orchestra.run.smithi042.stdout:modified 2024-08-30T08:58:50.586122+0000 2024-08-30T09:00:09.026 INFO:teuthology.orchestra.run.smithi042.stdout:tableserver 0 2024-08-30T09:00:09.026 INFO:teuthology.orchestra.run.smithi042.stdout:root 0 2024-08-30T09:00:09.026 INFO:teuthology.orchestra.run.smithi042.stdout:session_timeout 300 2024-08-30T09:00:09.026 INFO:teuthology.orchestra.run.smithi042.stdout:session_autoclose 300 2024-08-30T09:00:09.026 INFO:teuthology.orchestra.run.smithi042.stdout:max_file_size 1099511627776 2024-08-30T09:00:09.026 INFO:teuthology.orchestra.run.smithi042.stdout:max_xattr_size 65536 2024-08-30T09:00:09.026 INFO:teuthology.orchestra.run.smithi042.stdout:required_client_features {} 2024-08-30T09:00:09.026 INFO:teuthology.orchestra.run.smithi042.stdout:last_failure 0 2024-08-30T09:00:09.026 INFO:teuthology.orchestra.run.smithi042.stdout:last_failure_osd_epoch 0 2024-08-30T09:00:09.026 INFO:teuthology.orchestra.run.smithi042.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-08-30T09:00:09.027 INFO:teuthology.orchestra.run.smithi042.stdout:max_mds 3 2024-08-30T09:00:09.027 INFO:teuthology.orchestra.run.smithi042.stdout:in 0,1,2 2024-08-30T09:00:09.027 INFO:teuthology.orchestra.run.smithi042.stdout:up {0=24629,1=24610,2=24599} 2024-08-30T09:00:09.027 INFO:teuthology.orchestra.run.smithi042.stdout:failed 2024-08-30T09:00:09.027 INFO:teuthology.orchestra.run.smithi042.stdout:damaged 2024-08-30T09:00:09.027 INFO:teuthology.orchestra.run.smithi042.stdout:stopped 2024-08-30T09:00:09.027 INFO:teuthology.orchestra.run.smithi042.stdout:data_pools [3,4] 2024-08-30T09:00:09.027 INFO:teuthology.orchestra.run.smithi042.stdout:metadata_pool 2 2024-08-30T09:00:09.027 INFO:teuthology.orchestra.run.smithi042.stdout:inline_data disabled 2024-08-30T09:00:09.027 INFO:teuthology.orchestra.run.smithi042.stdout:balancer 2024-08-30T09:00:09.027 INFO:teuthology.orchestra.run.smithi042.stdout:bal_rank_mask -1 2024-08-30T09:00:09.027 INFO:teuthology.orchestra.run.smithi042.stdout:standby_count_wanted 0 2024-08-30T09:00:09.027 INFO:teuthology.orchestra.run.smithi042.stdout:qdb_cluster leader: 24629 members: 24629,24610,24599 2024-08-30T09:00:09.028 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.b{0:24629} state up:active seq 4 addr [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] compat {c=[1],r=[1],i=[1fff]}] 2024-08-30T09:00:09.028 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.c{0:24598} state up:standby-replay seq 1 addr [v2:172.21.15.190:6836/863256349,v1:172.21.15.190:6837/863256349] compat {c=[1],r=[1],i=[1fff]}] 2024-08-30T09:00:09.028 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.i{1:24610} state up:active seq 17 addr [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] compat {c=[1],r=[1],i=[1fff]}] 2024-08-30T09:00:09.028 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.f{1:24592} state up:standby-replay seq 1 addr [v2:172.21.15.190:6834/247905709,v1:172.21.15.190:6835/247905709] compat {c=[1],r=[1],i=[1fff]}] 2024-08-30T09:00:09.028 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.h{2:24599} state up:active seq 19 addr [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] compat {c=[1],r=[1],i=[1fff]}] 2024-08-30T09:00:09.028 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.e{2:24572} state up:standby-replay seq 1 addr [v2:172.21.15.167:6834/234596845,v1:172.21.15.167:6835/234596845] compat {c=[1],r=[1],i=[1fff]}] 2024-08-30T09:00:09.028 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-08-30T09:00:09.028 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-08-30T09:00:09.028 INFO:teuthology.orchestra.run.smithi042.stdout:Standby daemons: 2024-08-30T09:00:09.028 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-08-30T09:00:09.028 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.j{-1:14703} state up:standby seq 1 addr [v2:172.21.15.42:6834/3188400000,v1:172.21.15.42:6835/3188400000] compat {c=[1],r=[1],i=[1fff]}] 2024-08-30T09:00:09.028 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.d{-1:14715} state up:standby seq 1 addr [v2:172.21.15.42:6836/3356537674,v1:172.21.15.42:6837/3356537674] compat {c=[1],r=[1],i=[1fff]}] 2024-08-30T09:00:09.028 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.g{-1:14766} state up:standby seq 1 addr [v2:172.21.15.42:6838/2107785513,v1:172.21.15.42:6839/2107785513] compat {c=[1],r=[1],i=[1fff]}] 2024-08-30T09:00:09.028 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.a{-1:14778} state up:standby seq 1 addr [v2:172.21.15.42:6840/2298438674,v1:172.21.15.42:6841/2298438674] compat {c=[1],r=[1],i=[1fff]}] 2024-08-30T09:00:09.029 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.l{-1:24553} state up:standby seq 1 addr [v2:172.21.15.190:6832/543555570,v1:172.21.15.190:6833/543555570] compat {c=[1],r=[1],i=[1fff]}] 2024-08-30T09:00:09.029 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.k{-1:24554} state up:standby seq 1 addr [v2:172.21.15.167:6832/3907818941,v1:172.21.15.167:6833/3907818941] compat {c=[1],r=[1],i=[1fff]}] 2024-08-30T09:00:09.029 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 26 2024-08-30T09:00:09.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:09.025+0000 7f8c47669640 1 -- 172.21.15.42:0/3805421824 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f8c2c0b4bc0 msgr2=0x7f8c2c0b70d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:00:09.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:09.025+0000 7f8c47669640 1 --2- 172.21.15.42:0/3805421824 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f8c2c0b4bc0 0x7f8c2c0b70d0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f8c3c009b50 tx=0x7f8c3c005f90 comp rx=0 tx=0).stop 2024-08-30T09:00:09.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:09.025+0000 7f8c47669640 1 -- 172.21.15.42:0/3805421824 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8c40104580 msgr2=0x7f8c4019a3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:00:09.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:09.025+0000 7f8c47669640 1 --2- 172.21.15.42:0/3805421824 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8c40104580 0x7f8c4019a3d0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f8c30006010 tx=0x7f8c30031a50 comp rx=0 tx=0).stop 2024-08-30T09:00:09.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:09.025+0000 7f8c47669640 1 -- 172.21.15.42:0/3805421824 shutdown_connections 2024-08-30T09:00:09.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:09.025+0000 7f8c47669640 1 --2- 172.21.15.42:0/3805421824 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f8c2c0b4bc0 0x7f8c2c0b70d0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:09.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:09.025+0000 7f8c47669640 1 --2- 172.21.15.42:0/3805421824 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8c40104f20 0x7f8c4019af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:09.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:09.025+0000 7f8c47669640 1 --2- 172.21.15.42:0/3805421824 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8c40104580 0x7f8c4019a3d0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:09.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:09.025+0000 7f8c47669640 1 --2- 172.21.15.42:0/3805421824 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8c401034f0 0x7f8c40199d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:09.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:09.025+0000 7f8c47669640 1 -- 172.21.15.42:0/3805421824 >> 172.21.15.42:0/3805421824 conn(0x7f8c400fee60 msgr2=0x7f8c40107880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:00:09.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:09.026+0000 7f8c47669640 1 -- 172.21.15.42:0/3805421824 shutdown_connections 2024-08-30T09:00:09.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:09.026+0000 7f8c47669640 1 -- 172.21.15.42:0/3805421824 wait complete. 2024-08-30T09:00:09.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:00:09 smithi167 ceph-mon[22063]: from='client.? 172.21.15.42:0/3805421824' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-08-30T09:00:09.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:00:09 smithi042 ceph-mon[18652]: from='client.? 172.21.15.42:0/3805421824' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-08-30T09:00:09.666 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- bash -c 'ceph osd dump' 2024-08-30T09:00:09.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:00:09 smithi190 ceph-mon[22001]: from='client.? 172.21.15.42:0/3805421824' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-08-30T09:00:09.919 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T09:00:10.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:10.410+0000 7f9748f25640 1 -- 172.21.15.42:0/1235200889 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f974406b6a0 msgr2=0x7f97441061b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:00:10.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:10.410+0000 7f9748f25640 1 --2- 172.21.15.42:0/1235200889 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f974406b6a0 0x7f97441061b0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f9734009900 tx=0x7f973402a410 comp rx=0 tx=0).stop 2024-08-30T09:00:10.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:10.410+0000 7f9748f25640 1 -- 172.21.15.42:0/1235200889 shutdown_connections 2024-08-30T09:00:10.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:10.410+0000 7f9748f25640 1 --2- 172.21.15.42:0/1235200889 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9744109230 0x7f974410b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:10.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:10.410+0000 7f9748f25640 1 --2- 172.21.15.42:0/1235200889 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f97441066f0 0x7f9744108b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:10.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:10.410+0000 7f9748f25640 1 --2- 172.21.15.42:0/1235200889 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f974406b6a0 0x7f97441061b0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:10.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:10.410+0000 7f9748f25640 1 -- 172.21.15.42:0/1235200889 >> 172.21.15.42:0/1235200889 conn(0x7f97440fdd50 msgr2=0x7f9744100200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:00:10.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:10.410+0000 7f9748f25640 1 -- 172.21.15.42:0/1235200889 shutdown_connections 2024-08-30T09:00:10.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:10.410+0000 7f9748f25640 1 -- 172.21.15.42:0/1235200889 wait complete. 2024-08-30T09:00:10.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:10.411+0000 7f9748f25640 1 Processor -- start 2024-08-30T09:00:10.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:10.411+0000 7f9748f25640 1 -- start start 2024-08-30T09:00:10.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:10.411+0000 7f9748f25640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f974406b6a0 0x7f9744199d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:10.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:10.411+0000 7f9748f25640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f97441066f0 0x7f974419a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:10.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:10.411+0000 7f9748f25640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9744109230 0x7f974419af70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:10.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:10.411+0000 7f9748f25640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f974410f170 con 0x7f97441066f0 2024-08-30T09:00:10.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:10.411+0000 7f9748f25640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f974410eff0 con 0x7f9744109230 2024-08-30T09:00:10.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:10.411+0000 7f9748f25640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f974410f2f0 con 0x7f974406b6a0 2024-08-30T09:00:10.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:10.411+0000 7f9741d9b640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f97441066f0 0x7f974419a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:10.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:10.411+0000 7f9741d9b640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f97441066f0 0x7f974419a3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:51454/0 (socket says 172.21.15.42:51454) 2024-08-30T09:00:10.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:10.411+0000 7f9741d9b640 1 -- 172.21.15.42:0/2048512799 learned_addr learned my addr 172.21.15.42:0/2048512799 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:00:10.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:10.411+0000 7f974259c640 1 --2- 172.21.15.42:0/2048512799 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f974406b6a0 0x7f9744199d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:10.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:10.411+0000 7f9742d9d640 1 --2- 172.21.15.42:0/2048512799 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9744109230 0x7f974419af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:10.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:10.411+0000 7f9741d9b640 1 -- 172.21.15.42:0/2048512799 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f974406b6a0 msgr2=0x7f9744199d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:00:10.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:10.411+0000 7f9741d9b640 1 --2- 172.21.15.42:0/2048512799 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f974406b6a0 0x7f9744199d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:10.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:10.411+0000 7f9741d9b640 1 -- 172.21.15.42:0/2048512799 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9744109230 msgr2=0x7f974419af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:00:10.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:10.411+0000 7f9741d9b640 1 --2- 172.21.15.42:0/2048512799 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9744109230 0x7f974419af70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:10.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:10.411+0000 7f9741d9b640 1 -- 172.21.15.42:0/2048512799 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f974419b790 con 0x7f97441066f0 2024-08-30T09:00:10.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:10.411+0000 7f9742d9d640 1 --2- 172.21.15.42:0/2048512799 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9744109230 0x7f974419af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-30T09:00:10.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:10.412+0000 7f9741d9b640 1 --2- 172.21.15.42:0/2048512799 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f97441066f0 0x7f974419a3d0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f973800e9f0 tx=0x7f973800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:00:10.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:10.412+0000 7f97337fe640 1 -- 172.21.15.42:0/2048512799 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f973800cd80 con 0x7f97441066f0 2024-08-30T09:00:10.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:10.412+0000 7f97337fe640 1 -- 172.21.15.42:0/2048512799 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f97380108e0 con 0x7f97441066f0 2024-08-30T09:00:10.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:10.412+0000 7f97337fe640 1 -- 172.21.15.42:0/2048512799 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9738018980 con 0x7f97441066f0 2024-08-30T09:00:10.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:10.412+0000 7f9748f25640 1 -- 172.21.15.42:0/2048512799 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f974419ba80 con 0x7f97441066f0 2024-08-30T09:00:10.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:10.412+0000 7f9748f25640 1 -- 172.21.15.42:0/2048512799 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9744102160 con 0x7f97441066f0 2024-08-30T09:00:10.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:10.414+0000 7f9748f25640 1 -- 172.21.15.42:0/2048512799 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9744106b00 con 0x7f97441066f0 2024-08-30T09:00:10.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:10.416+0000 7f97337fe640 1 -- 172.21.15.42:0/2048512799 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) ==== 152628+0+0 (secure 0 0 0) 0x7f9738010430 con 0x7f97441066f0 2024-08-30T09:00:10.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:10.417+0000 7f97337fe640 1 --2- 172.21.15.42:0/2048512799 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f972c0b4c40 0x7f972c0b7150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:10.421 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:10.417+0000 7f974259c640 1 --2- 172.21.15.42:0/2048512799 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f972c0b4c40 0x7f972c0b7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:10.421 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:10.417+0000 7f97337fe640 1 -- 172.21.15.42:0/2048512799 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f97380e5e50 con 0x7f97441066f0 2024-08-30T09:00:10.421 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:10.418+0000 7f974259c640 1 --2- 172.21.15.42:0/2048512799 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f972c0b4c40 0x7f972c0b7150 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f9734004770 tx=0x7f97340314a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:00:10.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:10.420+0000 7f97337fe640 1 -- 172.21.15.42:0/2048512799 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f97380aca90 con 0x7f97441066f0 2024-08-30T09:00:10.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:00:10 smithi167 ceph-mon[22063]: pgmap v355: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-08-30T09:00:10.587 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:00:10 smithi190 ceph-mon[22001]: pgmap v355: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-08-30T09:00:10.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:10.608+0000 7f9748f25640 1 -- 172.21.15.42:0/2048512799 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7f97440ff870 con 0x7f97441066f0 2024-08-30T09:00:10.610 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:00:10 smithi042 ceph-mon[18652]: pgmap v355: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-08-30T09:00:10.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:10.609+0000 7f97337fe640 1 -- 172.21.15.42:0/2048512799 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v83) ==== 56+0+5660 (secure 0 0 0) 0x7f97440ff870 con 0x7f97441066f0 2024-08-30T09:00:10.611 INFO:teuthology.orchestra.run.smithi042.stdout:epoch 83 2024-08-30T09:00:10.611 INFO:teuthology.orchestra.run.smithi042.stdout:fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 2024-08-30T09:00:10.611 INFO:teuthology.orchestra.run.smithi042.stdout:created 2024-08-30T08:49:44.858265+0000 2024-08-30T09:00:10.611 INFO:teuthology.orchestra.run.smithi042.stdout:modified 2024-08-30T08:59:46.645920+0000 2024-08-30T09:00:10.611 INFO:teuthology.orchestra.run.smithi042.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-08-30T09:00:10.611 INFO:teuthology.orchestra.run.smithi042.stdout:crush_version 27 2024-08-30T09:00:10.611 INFO:teuthology.orchestra.run.smithi042.stdout:full_ratio 0.95 2024-08-30T09:00:10.611 INFO:teuthology.orchestra.run.smithi042.stdout:backfillfull_ratio 0.9 2024-08-30T09:00:10.611 INFO:teuthology.orchestra.run.smithi042.stdout:nearfull_ratio 0.85 2024-08-30T09:00:10.611 INFO:teuthology.orchestra.run.smithi042.stdout:require_min_compat_client luminous 2024-08-30T09:00:10.611 INFO:teuthology.orchestra.run.smithi042.stdout:min_compat_client luminous 2024-08-30T09:00:10.612 INFO:teuthology.orchestra.run.smithi042.stdout:require_osd_release squid 2024-08-30T09:00:10.612 INFO:teuthology.orchestra.run.smithi042.stdout:stretch_mode_enabled false 2024-08-30T09:00:10.612 INFO:teuthology.orchestra.run.smithi042.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-08-30T09:00:10.612 INFO:teuthology.orchestra.run.smithi042.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 73 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-08-30T09:00:10.612 INFO:teuthology.orchestra.run.smithi042.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 73 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.05 application cephfs read_balance_score 1.50 2024-08-30T09:00:10.612 INFO:teuthology.orchestra.run.smithi042.stdout:pool 4 'cephfs_data_ec' erasure profile cephfs_data_ec size 4 min_size 3 crush_rule 1 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 79 flags hashpspool,ec_overwrites stripe_width 8192 pg_num_min 64 target_size_ratio 0.9 application cephfs 2024-08-30T09:00:10.612 INFO:teuthology.orchestra.run.smithi042.stdout:max_osd 12 2024-08-30T09:00:10.612 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0 up in weight 1 up_from 8 up_thru 80 down_at 0 last_clean_interval [0,0) [v2:172.21.15.42:6802/2617890537,v1:172.21.15.42:6803/2617890537] [v2:172.21.15.42:6804/2617890537,v1:172.21.15.42:6805/2617890537] exists,up 029d62d7-a6c6-4905-8c76-6dffa9d57b92 2024-08-30T09:00:10.612 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1 up in weight 1 up_from 13 up_thru 76 down_at 0 last_clean_interval [0,0) [v2:172.21.15.167:6800/2791960082,v1:172.21.15.167:6801/2791960082] [v2:172.21.15.167:6802/2791960082,v1:172.21.15.167:6803/2791960082] exists,up 3a78384b-0207-41df-9384-22665007d5b3 2024-08-30T09:00:10.612 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2 up in weight 1 up_from 18 up_thru 76 down_at 0 last_clean_interval [0,0) [v2:172.21.15.190:6800/1101326404,v1:172.21.15.190:6801/1101326404] [v2:172.21.15.190:6802/1101326404,v1:172.21.15.190:6803/1101326404] exists,up 16001809-c27c-4f52-948a-9998261acb4b 2024-08-30T09:00:10.612 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3 up in weight 1 up_from 24 up_thru 76 down_at 0 last_clean_interval [0,0) [v2:172.21.15.42:6810/1135599916,v1:172.21.15.42:6811/1135599916] [v2:172.21.15.42:6812/1135599916,v1:172.21.15.42:6813/1135599916] exists,up 87bb3e36-9e9e-4b19-9d8c-0cdf6e089d3e 2024-08-30T09:00:10.612 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4 up in weight 1 up_from 30 up_thru 76 down_at 0 last_clean_interval [0,0) [v2:172.21.15.167:6808/2927511419,v1:172.21.15.167:6809/2927511419] [v2:172.21.15.167:6810/2927511419,v1:172.21.15.167:6811/2927511419] exists,up 3ca1aae3-c972-4fb9-ac86-236279c30d64 2024-08-30T09:00:10.612 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5 up in weight 1 up_from 35 up_thru 76 down_at 0 last_clean_interval [0,0) [v2:172.21.15.190:6808/2557145333,v1:172.21.15.190:6809/2557145333] [v2:172.21.15.190:6810/2557145333,v1:172.21.15.190:6811/2557145333] exists,up 5d07f54b-0d07-4efe-9bcf-390755ce0258 2024-08-30T09:00:10.612 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6 up in weight 1 up_from 41 up_thru 80 down_at 0 last_clean_interval [0,0) [v2:172.21.15.42:6818/3213526100,v1:172.21.15.42:6819/3213526100] [v2:172.21.15.42:6820/3213526100,v1:172.21.15.42:6821/3213526100] exists,up 96ad64af-44ff-4197-97d4-156f0cb88e81 2024-08-30T09:00:10.613 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7 up in weight 1 up_from 46 up_thru 76 down_at 0 last_clean_interval [0,0) [v2:172.21.15.167:6816/2962799814,v1:172.21.15.167:6817/2962799814] [v2:172.21.15.167:6818/2962799814,v1:172.21.15.167:6819/2962799814] exists,up 337a2112-eaf7-4e6b-86be-4e40d6ddf835 2024-08-30T09:00:10.613 INFO:teuthology.orchestra.run.smithi042.stdout:osd.8 up in weight 1 up_from 51 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.190:6816/3242198843,v1:172.21.15.190:6817/3242198843] [v2:172.21.15.190:6818/3242198843,v1:172.21.15.190:6819/3242198843] exists,up 219a7224-4331-464b-ac86-c51054d51d91 2024-08-30T09:00:10.613 INFO:teuthology.orchestra.run.smithi042.stdout:osd.9 up in weight 1 up_from 57 up_thru 80 down_at 0 last_clean_interval [0,0) [v2:172.21.15.42:6826/1123536325,v1:172.21.15.42:6827/1123536325] [v2:172.21.15.42:6828/1123536325,v1:172.21.15.42:6829/1123536325] exists,up 2f2f5177-a009-40d9-991c-76aec8a5cba6 2024-08-30T09:00:10.613 INFO:teuthology.orchestra.run.smithi042.stdout:osd.10 up in weight 1 up_from 61 up_thru 80 down_at 0 last_clean_interval [0,0) [v2:172.21.15.167:6824/3451198319,v1:172.21.15.167:6825/3451198319] [v2:172.21.15.167:6826/3451198319,v1:172.21.15.167:6827/3451198319] exists,up 96ef5058-2518-4862-a165-17666b33a3e6 2024-08-30T09:00:10.613 INFO:teuthology.orchestra.run.smithi042.stdout:osd.11 up in weight 1 up_from 67 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.190:6824/380064457,v1:172.21.15.190:6825/380064457] [v2:172.21.15.190:6826/380064457,v1:172.21.15.190:6827/380064457] exists,up a4eb9bd3-3faa-4c81-ba6d-c2084b72c9ee 2024-08-30T09:00:10.613 INFO:teuthology.orchestra.run.smithi042.stdout:pg_upmap_items 2.e [2,4] 2024-08-30T09:00:10.613 INFO:teuthology.orchestra.run.smithi042.stdout:pg_upmap_items 2.10 [8,11] 2024-08-30T09:00:10.613 INFO:teuthology.orchestra.run.smithi042.stdout:pg_upmap_items 2.1c [8,11] 2024-08-30T09:00:10.613 INFO:teuthology.orchestra.run.smithi042.stdout:pg_upmap_items 2.1d [2,1] 2024-08-30T09:00:10.613 INFO:teuthology.orchestra.run.smithi042.stdout:pg_upmap_items 3.0 [5,9] 2024-08-30T09:00:10.613 INFO:teuthology.orchestra.run.smithi042.stdout:pg_upmap_items 3.6 [4,6] 2024-08-30T09:00:10.613 INFO:teuthology.orchestra.run.smithi042.stdout:pg_upmap_items 3.2c [4,9] 2024-08-30T09:00:10.613 INFO:teuthology.orchestra.run.smithi042.stdout:pg_upmap_items 3.36 [4,9] 2024-08-30T09:00:10.613 INFO:teuthology.orchestra.run.smithi042.stdout:pg_upmap_items 4.9 [5,8] 2024-08-30T09:00:10.614 INFO:teuthology.orchestra.run.smithi042.stdout:pg_upmap_items 4.b [9,8] 2024-08-30T09:00:10.614 INFO:teuthology.orchestra.run.smithi042.stdout:pg_upmap_items 4.f [9,8] 2024-08-30T09:00:10.614 INFO:teuthology.orchestra.run.smithi042.stdout:pg_upmap_items 4.14 [9,10] 2024-08-30T09:00:10.614 INFO:teuthology.orchestra.run.smithi042.stdout:blocklist 172.21.15.42:0/2691233260 expires 2024-08-31T08:50:44.821070+0000 2024-08-30T09:00:10.614 INFO:teuthology.orchestra.run.smithi042.stdout:blocklist 172.21.15.42:0/1518053351 expires 2024-08-31T08:50:44.821070+0000 2024-08-30T09:00:10.614 INFO:teuthology.orchestra.run.smithi042.stdout:blocklist 172.21.15.42:0/352001020 expires 2024-08-31T08:50:44.821070+0000 2024-08-30T09:00:10.614 INFO:teuthology.orchestra.run.smithi042.stdout:blocklist 172.21.15.42:6801/1733768634 expires 2024-08-31T08:50:44.821070+0000 2024-08-30T09:00:10.614 INFO:teuthology.orchestra.run.smithi042.stdout:blocklist 172.21.15.42:0/1000597888 expires 2024-08-31T08:50:18.561851+0000 2024-08-30T09:00:10.614 INFO:teuthology.orchestra.run.smithi042.stdout:blocklist 172.21.15.42:6800/1733768634 expires 2024-08-31T08:50:44.821070+0000 2024-08-30T09:00:10.614 INFO:teuthology.orchestra.run.smithi042.stdout:blocklist 172.21.15.42:6801/3538882081 expires 2024-08-31T08:50:18.561851+0000 2024-08-30T09:00:10.614 INFO:teuthology.orchestra.run.smithi042.stdout:blocklist 172.21.15.42:6800/3538882081 expires 2024-08-31T08:50:18.561851+0000 2024-08-30T09:00:10.614 INFO:teuthology.orchestra.run.smithi042.stdout:blocklist 172.21.15.42:0/145366508 expires 2024-08-31T08:50:18.561851+0000 2024-08-30T09:00:10.614 INFO:teuthology.orchestra.run.smithi042.stdout:blocklist 172.21.15.42:0/3346355213 expires 2024-08-31T08:50:18.561851+0000 2024-08-30T09:00:10.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:10.611+0000 7f9748f25640 1 -- 172.21.15.42:0/2048512799 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f972c0b4c40 msgr2=0x7f972c0b7150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:00:10.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:10.611+0000 7f9748f25640 1 --2- 172.21.15.42:0/2048512799 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f972c0b4c40 0x7f972c0b7150 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f9734004770 tx=0x7f97340314a0 comp rx=0 tx=0).stop 2024-08-30T09:00:10.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:10.611+0000 7f9748f25640 1 -- 172.21.15.42:0/2048512799 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f97441066f0 msgr2=0x7f974419a3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:00:10.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:10.611+0000 7f9748f25640 1 --2- 172.21.15.42:0/2048512799 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f97441066f0 0x7f974419a3d0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f973800e9f0 tx=0x7f973800eeb0 comp rx=0 tx=0).stop 2024-08-30T09:00:10.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:10.611+0000 7f9748f25640 1 -- 172.21.15.42:0/2048512799 shutdown_connections 2024-08-30T09:00:10.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:10.611+0000 7f9748f25640 1 --2- 172.21.15.42:0/2048512799 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f972c0b4c40 0x7f972c0b7150 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:10.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:10.611+0000 7f9748f25640 1 --2- 172.21.15.42:0/2048512799 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9744109230 0x7f974419af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:10.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:10.611+0000 7f9748f25640 1 --2- 172.21.15.42:0/2048512799 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f97441066f0 0x7f974419a3d0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:10.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:10.611+0000 7f9748f25640 1 --2- 172.21.15.42:0/2048512799 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f974406b6a0 0x7f9744199d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:10.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:10.611+0000 7f9748f25640 1 -- 172.21.15.42:0/2048512799 >> 172.21.15.42:0/2048512799 conn(0x7f97440fdd50 msgr2=0x7f9744104c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:00:10.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:10.611+0000 7f9748f25640 1 -- 172.21.15.42:0/2048512799 shutdown_connections 2024-08-30T09:00:10.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:10.611+0000 7f9748f25640 1 -- 172.21.15.42:0/2048512799 wait complete. 2024-08-30T09:00:11.315 INFO:teuthology.run_tasks:Running task fs.ready... 2024-08-30T09:00:11.327 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph fs dump --format=json 2024-08-30T09:00:11.586 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T09:00:11.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:00:11 smithi042 ceph-mon[18652]: from='client.? 172.21.15.42:0/2048512799' entity='client.admin' cmd={"prefix": "osd dump"} : dispatch 2024-08-30T09:00:11.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:00:11 smithi190 ceph-mon[22001]: from='client.? 172.21.15.42:0/2048512799' entity='client.admin' cmd={"prefix": "osd dump"} : dispatch 2024-08-30T09:00:11.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:00:11 smithi167 ceph-mon[22063]: from='client.? 172.21.15.42:0/2048512799' entity='client.admin' cmd={"prefix": "osd dump"} : dispatch 2024-08-30T09:00:12.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:12.116+0000 7f40bc73a640 1 -- 172.21.15.42:0/1635683220 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f40b41028b0 msgr2=0x7f40b4102cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:00:12.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:12.116+0000 7f40bc73a640 1 --2- 172.21.15.42:0/1635683220 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f40b41028b0 0x7f40b4102cc0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f40b0009900 tx=0x7f40b002a410 comp rx=0 tx=0).stop 2024-08-30T09:00:12.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:12.116+0000 7f40bc73a640 1 -- 172.21.15.42:0/1635683220 shutdown_connections 2024-08-30T09:00:12.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:12.116+0000 7f40bc73a640 1 --2- 172.21.15.42:0/1635683220 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f40b4108b40 0x7f40b410f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:12.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:12.116+0000 7f40bc73a640 1 --2- 172.21.15.42:0/1635683220 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f40b41081f0 0x7f40b4108600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:12.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:12.116+0000 7f40bc73a640 1 --2- 172.21.15.42:0/1635683220 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f40b41028b0 0x7f40b4102cc0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:12.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:12.116+0000 7f40bc73a640 1 -- 172.21.15.42:0/1635683220 >> 172.21.15.42:0/1635683220 conn(0x7f40b4079470 msgr2=0x7f40b4079880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:00:12.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:12.116+0000 7f40bc73a640 1 -- 172.21.15.42:0/1635683220 shutdown_connections 2024-08-30T09:00:12.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:12.116+0000 7f40bc73a640 1 -- 172.21.15.42:0/1635683220 wait complete. 2024-08-30T09:00:12.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:12.117+0000 7f40bc73a640 1 Processor -- start 2024-08-30T09:00:12.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:12.117+0000 7f40bc73a640 1 -- start start 2024-08-30T09:00:12.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:12.117+0000 7f40bc73a640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f40b41028b0 0x7f40b419b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:12.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:12.117+0000 7f40bc73a640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f40b41081f0 0x7f40b419b630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:12.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:12.117+0000 7f40bc73a640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f40b4108b40 0x7f40b419bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:12.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:12.117+0000 7f40bc73a640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f40b41127e0 con 0x7f40b41028b0 2024-08-30T09:00:12.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:12.117+0000 7f40bc73a640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f40b4112660 con 0x7f40b41081f0 2024-08-30T09:00:12.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:12.117+0000 7f40bc73a640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f40b4112960 con 0x7f40b4108b40 2024-08-30T09:00:12.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:12.117+0000 7f40ba4d6640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f40b41028b0 0x7f40b419b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:12.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:12.117+0000 7f40ba4d6640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f40b41028b0 0x7f40b419b0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:51486/0 (socket says 172.21.15.42:51486) 2024-08-30T09:00:12.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:12.117+0000 7f40ba4d6640 1 -- 172.21.15.42:0/1935003447 learned_addr learned my addr 172.21.15.42:0/1935003447 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:00:12.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:12.117+0000 7f40b9cd5640 1 --2- 172.21.15.42:0/1935003447 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f40b41081f0 0x7f40b419b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:12.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:12.118+0000 7f40bacd7640 1 --2- 172.21.15.42:0/1935003447 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f40b4108b40 0x7f40b419bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:12.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:12.118+0000 7f40ba4d6640 1 -- 172.21.15.42:0/1935003447 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f40b4108b40 msgr2=0x7f40b419bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:00:12.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:12.118+0000 7f40ba4d6640 1 --2- 172.21.15.42:0/1935003447 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f40b4108b40 0x7f40b419bb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:12.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:12.118+0000 7f40ba4d6640 1 -- 172.21.15.42:0/1935003447 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f40b41081f0 msgr2=0x7f40b419b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:00:12.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:12.118+0000 7f40ba4d6640 1 --2- 172.21.15.42:0/1935003447 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f40b41081f0 0x7f40b419b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:12.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:12.118+0000 7f40ba4d6640 1 -- 172.21.15.42:0/1935003447 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40b41a63b0 con 0x7f40b41028b0 2024-08-30T09:00:12.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:12.118+0000 7f40bacd7640 1 --2- 172.21.15.42:0/1935003447 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f40b4108b40 0x7f40b419bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-30T09:00:12.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:12.118+0000 7f40ba4d6640 1 --2- 172.21.15.42:0/1935003447 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f40b41028b0 0x7f40b419b0f0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f40b0005940 tx=0x7f40b00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:00:12.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:12.118+0000 7f40ab7fe640 1 -- 172.21.15.42:0/1935003447 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40b003f030 con 0x7f40b41028b0 2024-08-30T09:00:12.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:12.118+0000 7f40bc73a640 1 -- 172.21.15.42:0/1935003447 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40b41a6640 con 0x7f40b41028b0 2024-08-30T09:00:12.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:12.118+0000 7f40ab7fe640 1 -- 172.21.15.42:0/1935003447 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f40b0003cb0 con 0x7f40b41028b0 2024-08-30T09:00:12.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:12.118+0000 7f40ab7fe640 1 -- 172.21.15.42:0/1935003447 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40b0035070 con 0x7f40b41028b0 2024-08-30T09:00:12.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:12.118+0000 7f40bc73a640 1 -- 172.21.15.42:0/1935003447 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40b41a6b20 con 0x7f40b41028b0 2024-08-30T09:00:12.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:12.119+0000 7f40bc73a640 1 -- 172.21.15.42:0/1935003447 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40b4102cc0 con 0x7f40b41028b0 2024-08-30T09:00:12.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:12.121+0000 7f40ab7fe640 1 -- 172.21.15.42:0/1935003447 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) ==== 152628+0+0 (secure 0 0 0) 0x7f40b0042030 con 0x7f40b41028b0 2024-08-30T09:00:12.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:12.122+0000 7f40ab7fe640 1 --2- 172.21.15.42:0/1935003447 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f40a00b4bc0 0x7f40a00b70d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:12.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:12.122+0000 7f40b9cd5640 1 --2- 172.21.15.42:0/1935003447 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f40a00b4bc0 0x7f40a00b70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:12.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:12.122+0000 7f40ab7fe640 1 -- 172.21.15.42:0/1935003447 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f40b0106250 con 0x7f40b41028b0 2024-08-30T09:00:12.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:12.122+0000 7f40b9cd5640 1 --2- 172.21.15.42:0/1935003447 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f40a00b4bc0 0x7f40a00b70d0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f40b4105210 tx=0x7f40a400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:00:12.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:12.125+0000 7f40ab7fe640 1 -- 172.21.15.42:0/1935003447 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f40b00cce30 con 0x7f40b41028b0 2024-08-30T09:00:12.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:12.347+0000 7f40bc73a640 1 -- 172.21.15.42:0/1935003447 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f40b410a920 con 0x7f40b41028b0 2024-08-30T09:00:12.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:12.347+0000 7f40ab7fe640 1 -- 172.21.15.42:0/1935003447 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11915 (secure 0 0 0) 0x7f40b00cc730 con 0x7f40b41028b0 2024-08-30T09:00:12.349 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-08-30T09:00:12.350 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":26,"btime":"2024-08-30T08:58:50:586137+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14703,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/3188400000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":3188400000},{"type":"v1","addr":"172.21.15.42:6835","nonce":3188400000}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14715,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/3356537674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":3356537674},{"type":"v1","addr":"172.21.15.42:6837","nonce":3356537674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14766,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6839/2107785513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6838","nonce":2107785513},{"type":"v1","addr":"172.21.15.42:6839","nonce":2107785513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14778,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6841/2298438674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6840","nonce":2298438674},{"type":"v1","addr":"172.21.15.42:6841","nonce":2298438674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24553,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/543555570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":543555570},{"type":"v1","addr":"172.21.15.190:6833","nonce":543555570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24554,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/3907818941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":3907818941},{"type":"v1","addr":"172.21.15.167:6833","nonce":3907818941}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-30T08:58:03.055084+0000","modified":"2024-08-30T08:58:50.586122+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24629,"mds_1":24610,"mds_2":24599},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24572":{"gid":24572,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6835/234596845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":234596845},{"type":"v1","addr":"172.21.15.167:6835","nonce":234596845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24592":{"gid":24592,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6835/247905709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":247905709},{"type":"v1","addr":"172.21.15.190:6835","nonce":247905709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24598":{"gid":24598,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6837/863256349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6836","nonce":863256349},{"type":"v1","addr":"172.21.15.190:6837","nonce":863256349}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24599":{"gid":24599,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.167:6837/3011837856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":3011837856},{"type":"v1","addr":"172.21.15.167:6837","nonce":3011837856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24610":{"gid":24610,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.190:6839/719981110","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6838","nonce":719981110},{"type":"v1","addr":"172.21.15.190:6839","nonce":719981110}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24629":{"gid":24629,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.167:6839/3604645207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":3604645207},{"type":"v1","addr":"172.21.15.167:6839","nonce":3604645207}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24629,"qdb_cluster":[24629,24610,24599]},"id":1}]} 2024-08-30T09:00:12.350 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 26 2024-08-30T09:00:12.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:12.349+0000 7f40bc73a640 1 -- 172.21.15.42:0/1935003447 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f40a00b4bc0 msgr2=0x7f40a00b70d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:00:12.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:12.349+0000 7f40bc73a640 1 --2- 172.21.15.42:0/1935003447 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f40a00b4bc0 0x7f40a00b70d0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f40b4105210 tx=0x7f40a400b390 comp rx=0 tx=0).stop 2024-08-30T09:00:12.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:12.349+0000 7f40bc73a640 1 -- 172.21.15.42:0/1935003447 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f40b41028b0 msgr2=0x7f40b419b0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:00:12.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:12.350+0000 7f40bc73a640 1 --2- 172.21.15.42:0/1935003447 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f40b41028b0 0x7f40b419b0f0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f40b0005940 tx=0x7f40b00042f0 comp rx=0 tx=0).stop 2024-08-30T09:00:12.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:12.350+0000 7f40bc73a640 1 -- 172.21.15.42:0/1935003447 shutdown_connections 2024-08-30T09:00:12.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:12.350+0000 7f40bc73a640 1 --2- 172.21.15.42:0/1935003447 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f40a00b4bc0 0x7f40a00b70d0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:12.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:12.350+0000 7f40bc73a640 1 --2- 172.21.15.42:0/1935003447 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f40b4108b40 0x7f40b419bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:12.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:12.350+0000 7f40bc73a640 1 --2- 172.21.15.42:0/1935003447 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f40b41081f0 0x7f40b419b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:12.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:12.350+0000 7f40bc73a640 1 --2- 172.21.15.42:0/1935003447 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f40b41028b0 0x7f40b419b0f0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:12.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:12.350+0000 7f40bc73a640 1 -- 172.21.15.42:0/1935003447 >> 172.21.15.42:0/1935003447 conn(0x7f40b4079470 msgr2=0x7f40b4106330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:00:12.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:12.350+0000 7f40bc73a640 1 -- 172.21.15.42:0/1935003447 shutdown_connections 2024-08-30T09:00:12.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:12.350+0000 7f40bc73a640 1 -- 172.21.15.42:0/1935003447 wait complete. 2024-08-30T09:00:12.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:00:12 smithi042 ceph-mon[18652]: pgmap v356: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-08-30T09:00:12.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:00:12 smithi190 ceph-mon[22001]: pgmap v356: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-08-30T09:00:12.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:00:12 smithi167 ceph-mon[22063]: pgmap v356: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-08-30T09:00:13.052 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph fs dump --format=json 2024-08-30T09:00:13.313 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T09:00:13.591 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:00:13 smithi042 ceph-mon[18652]: from='client.? 172.21.15.42:0/1935003447' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-30T09:00:13.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:00:13 smithi190 ceph-mon[22001]: from='client.? 172.21.15.42:0/1935003447' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-30T09:00:13.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:13.818+0000 7faa47931640 1 -- 172.21.15.42:0/3542303441 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faa40109b90 msgr2=0x7faa401103c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:00:13.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:13.818+0000 7faa47931640 1 --2- 172.21.15.42:0/3542303441 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faa40109b90 0x7faa401103c0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7faa38009ca0 tx=0x7faa3802a8c0 comp rx=0 tx=0).stop 2024-08-30T09:00:13.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:13.818+0000 7faa47931640 1 -- 172.21.15.42:0/3542303441 shutdown_connections 2024-08-30T09:00:13.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:13.818+0000 7faa47931640 1 --2- 172.21.15.42:0/3542303441 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faa40109b90 0x7faa401103c0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:13.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:13.818+0000 7faa47931640 1 --2- 172.21.15.42:0/3542303441 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faa40109240 0x7faa40109650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:13.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:13.818+0000 7faa47931640 1 --2- 172.21.15.42:0/3542303441 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faa40103900 0x7faa40103d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:13.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:13.818+0000 7faa47931640 1 -- 172.21.15.42:0/3542303441 >> 172.21.15.42:0/3542303441 conn(0x7faa4007b5f0 msgr2=0x7faa4007ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:00:13.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:13.818+0000 7faa47931640 1 -- 172.21.15.42:0/3542303441 shutdown_connections 2024-08-30T09:00:13.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:13.818+0000 7faa47931640 1 -- 172.21.15.42:0/3542303441 wait complete. 2024-08-30T09:00:13.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:13.819+0000 7faa47931640 1 Processor -- start 2024-08-30T09:00:13.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:13.819+0000 7faa47931640 1 -- start start 2024-08-30T09:00:13.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:13.819+0000 7faa47931640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faa40103900 0x7faa4019a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:13.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:13.819+0000 7faa47931640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faa40109240 0x7faa4019a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:13.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:13.819+0000 7faa47931640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faa40109b90 0x7faa4019fa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:13.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:13.819+0000 7faa47931640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7faa40113850 con 0x7faa40109240 2024-08-30T09:00:13.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:13.819+0000 7faa47931640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7faa401136d0 con 0x7faa40109b90 2024-08-30T09:00:13.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:13.819+0000 7faa47931640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7faa401139d0 con 0x7faa40103900 2024-08-30T09:00:13.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:13.819+0000 7faa44ecc640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faa40109240 0x7faa4019a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:13.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:13.819+0000 7faa44ecc640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faa40109240 0x7faa4019a780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:51498/0 (socket says 172.21.15.42:51498) 2024-08-30T09:00:13.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:13.819+0000 7faa44ecc640 1 -- 172.21.15.42:0/726212896 learned_addr learned my addr 172.21.15.42:0/726212896 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:00:13.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:13.819+0000 7faa456cd640 1 --2- 172.21.15.42:0/726212896 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faa40103900 0x7faa4019a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:13.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:13.819+0000 7faa45ece640 1 --2- 172.21.15.42:0/726212896 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faa40109b90 0x7faa4019fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:13.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:13.819+0000 7faa44ecc640 1 -- 172.21.15.42:0/726212896 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faa40103900 msgr2=0x7faa4019a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:00:13.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:13.819+0000 7faa44ecc640 1 --2- 172.21.15.42:0/726212896 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faa40103900 0x7faa4019a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:13.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:13.819+0000 7faa44ecc640 1 -- 172.21.15.42:0/726212896 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faa40109b90 msgr2=0x7faa4019fa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:00:13.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:13.819+0000 7faa44ecc640 1 --2- 172.21.15.42:0/726212896 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faa40109b90 0x7faa4019fa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:13.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:13.819+0000 7faa44ecc640 1 -- 172.21.15.42:0/726212896 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa401a0150 con 0x7faa40109240 2024-08-30T09:00:13.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:13.820+0000 7faa45ece640 1 --2- 172.21.15.42:0/726212896 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faa40109b90 0x7faa4019fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-30T09:00:13.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:13.820+0000 7faa44ecc640 1 --2- 172.21.15.42:0/726212896 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faa40109240 0x7faa4019a780 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7faa3000d740 tx=0x7faa3000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:00:13.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:13.820+0000 7faa367fc640 1 -- 172.21.15.42:0/726212896 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa30017070 con 0x7faa40109240 2024-08-30T09:00:13.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:13.820+0000 7faa367fc640 1 -- 172.21.15.42:0/726212896 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faa3000de50 con 0x7faa40109240 2024-08-30T09:00:13.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:13.820+0000 7faa47931640 1 -- 172.21.15.42:0/726212896 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa401a0440 con 0x7faa40109240 2024-08-30T09:00:13.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:13.820+0000 7faa367fc640 1 -- 172.21.15.42:0/726212896 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa3000f620 con 0x7faa40109240 2024-08-30T09:00:13.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:13.820+0000 7faa47931640 1 -- 172.21.15.42:0/726212896 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa401a6ba0 con 0x7faa40109240 2024-08-30T09:00:13.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:13.821+0000 7faa47931640 1 -- 172.21.15.42:0/726212896 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa40103d10 con 0x7faa40109240 2024-08-30T09:00:13.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:13.823+0000 7faa367fc640 1 -- 172.21.15.42:0/726212896 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) ==== 152628+0+0 (secure 0 0 0) 0x7faa3001a070 con 0x7faa40109240 2024-08-30T09:00:13.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:13.823+0000 7faa367fc640 1 --2- 172.21.15.42:0/726212896 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7faa2c0b4c40 0x7faa2c0b7150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:13.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:13.824+0000 7faa456cd640 1 --2- 172.21.15.42:0/726212896 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7faa2c0b4c40 0x7faa2c0b7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:13.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:13.824+0000 7faa367fc640 1 -- 172.21.15.42:0/726212896 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7faa300e4f50 con 0x7faa40109240 2024-08-30T09:00:13.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:13.824+0000 7faa456cd640 1 --2- 172.21.15.42:0/726212896 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7faa2c0b4c40 0x7faa2c0b7150 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7faa3c005d00 tx=0x7faa3c00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:00:13.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:00:13 smithi167 ceph-mon[22063]: from='client.? 172.21.15.42:0/1935003447' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-30T09:00:13.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:13.827+0000 7faa367fc640 1 -- 172.21.15.42:0/726212896 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7faa300abc10 con 0x7faa40109240 2024-08-30T09:00:14.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:14.050+0000 7faa47931640 1 -- 172.21.15.42:0/726212896 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7faa4019b310 con 0x7faa40109240 2024-08-30T09:00:14.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:14.050+0000 7faa367fc640 1 -- 172.21.15.42:0/726212896 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11915 (secure 0 0 0) 0x7faa300c0e20 con 0x7faa40109240 2024-08-30T09:00:14.052 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-08-30T09:00:14.053 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":26,"btime":"2024-08-30T08:58:50:586137+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14703,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/3188400000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":3188400000},{"type":"v1","addr":"172.21.15.42:6835","nonce":3188400000}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14715,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/3356537674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":3356537674},{"type":"v1","addr":"172.21.15.42:6837","nonce":3356537674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14766,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6839/2107785513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6838","nonce":2107785513},{"type":"v1","addr":"172.21.15.42:6839","nonce":2107785513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14778,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6841/2298438674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6840","nonce":2298438674},{"type":"v1","addr":"172.21.15.42:6841","nonce":2298438674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24553,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/543555570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":543555570},{"type":"v1","addr":"172.21.15.190:6833","nonce":543555570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24554,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/3907818941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":3907818941},{"type":"v1","addr":"172.21.15.167:6833","nonce":3907818941}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-30T08:58:03.055084+0000","modified":"2024-08-30T08:58:50.586122+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24629,"mds_1":24610,"mds_2":24599},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24572":{"gid":24572,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6835/234596845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":234596845},{"type":"v1","addr":"172.21.15.167:6835","nonce":234596845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24592":{"gid":24592,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6835/247905709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":247905709},{"type":"v1","addr":"172.21.15.190:6835","nonce":247905709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24598":{"gid":24598,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6837/863256349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6836","nonce":863256349},{"type":"v1","addr":"172.21.15.190:6837","nonce":863256349}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24599":{"gid":24599,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.167:6837/3011837856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":3011837856},{"type":"v1","addr":"172.21.15.167:6837","nonce":3011837856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24610":{"gid":24610,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.190:6839/719981110","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6838","nonce":719981110},{"type":"v1","addr":"172.21.15.190:6839","nonce":719981110}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24629":{"gid":24629,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.167:6839/3604645207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":3604645207},{"type":"v1","addr":"172.21.15.167:6839","nonce":3604645207}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24629,"qdb_cluster":[24629,24610,24599]},"id":1}]} 2024-08-30T09:00:14.053 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 26 2024-08-30T09:00:14.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:14.053+0000 7faa47931640 1 -- 172.21.15.42:0/726212896 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7faa2c0b4c40 msgr2=0x7faa2c0b7150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:00:14.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:14.053+0000 7faa47931640 1 --2- 172.21.15.42:0/726212896 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7faa2c0b4c40 0x7faa2c0b7150 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7faa3c005d00 tx=0x7faa3c00b410 comp rx=0 tx=0).stop 2024-08-30T09:00:14.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:14.053+0000 7faa47931640 1 -- 172.21.15.42:0/726212896 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faa40109240 msgr2=0x7faa4019a780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:00:14.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:14.053+0000 7faa47931640 1 --2- 172.21.15.42:0/726212896 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faa40109240 0x7faa4019a780 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7faa3000d740 tx=0x7faa3000dc00 comp rx=0 tx=0).stop 2024-08-30T09:00:14.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:14.053+0000 7faa47931640 1 -- 172.21.15.42:0/726212896 shutdown_connections 2024-08-30T09:00:14.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:14.053+0000 7faa47931640 1 --2- 172.21.15.42:0/726212896 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7faa2c0b4c40 0x7faa2c0b7150 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:14.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:14.053+0000 7faa47931640 1 --2- 172.21.15.42:0/726212896 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faa40109b90 0x7faa4019fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:14.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:14.053+0000 7faa47931640 1 --2- 172.21.15.42:0/726212896 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faa40109240 0x7faa4019a780 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:14.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:14.053+0000 7faa47931640 1 --2- 172.21.15.42:0/726212896 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faa40103900 0x7faa4019a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:14.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:14.053+0000 7faa47931640 1 -- 172.21.15.42:0/726212896 >> 172.21.15.42:0/726212896 conn(0x7faa4007b5f0 msgr2=0x7faa40107250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:00:14.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:14.053+0000 7faa47931640 1 -- 172.21.15.42:0/726212896 shutdown_connections 2024-08-30T09:00:14.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:14.053+0000 7faa47931640 1 -- 172.21.15.42:0/726212896 wait complete. 2024-08-30T09:00:14.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:00:14 smithi042 ceph-mon[18652]: pgmap v357: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-08-30T09:00:14.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:00:14 smithi042 ceph-mon[18652]: from='client.? 172.21.15.42:0/726212896' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-30T09:00:14.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:00:14 smithi190 ceph-mon[22001]: pgmap v357: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-08-30T09:00:14.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:00:14 smithi190 ceph-mon[22001]: from='client.? 172.21.15.42:0/726212896' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-30T09:00:14.725 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph osd dump --format=json 2024-08-30T09:00:14.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:00:14 smithi167 ceph-mon[22063]: pgmap v357: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-08-30T09:00:14.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:00:14 smithi167 ceph-mon[22063]: from='client.? 172.21.15.42:0/726212896' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-30T09:00:14.980 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T09:00:15.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:15.504+0000 7f63b7bbd640 1 -- 172.21.15.42:0/4172942389 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f63b0103960 msgr2=0x7f63b0103d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:00:15.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:15.504+0000 7f63b7bbd640 1 --2- 172.21.15.42:0/4172942389 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f63b0103960 0x7f63b0103d70 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f63ac009900 tx=0x7f63ac02a3f0 comp rx=0 tx=0).stop 2024-08-30T09:00:15.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:15.504+0000 7f63b7bbd640 1 -- 172.21.15.42:0/4172942389 shutdown_connections 2024-08-30T09:00:15.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:15.504+0000 7f63b7bbd640 1 --2- 172.21.15.42:0/4172942389 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f63b0109ba0 0x7f63b01103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:15.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:15.504+0000 7f63b7bbd640 1 --2- 172.21.15.42:0/4172942389 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f63b0109250 0x7f63b0109660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:15.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:15.504+0000 7f63b7bbd640 1 --2- 172.21.15.42:0/4172942389 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f63b0103960 0x7f63b0103d70 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:15.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:15.504+0000 7f63b7bbd640 1 -- 172.21.15.42:0/4172942389 >> 172.21.15.42:0/4172942389 conn(0x7f63b007b6b0 msgr2=0x7f63b007bac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:00:15.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:15.504+0000 7f63b7bbd640 1 -- 172.21.15.42:0/4172942389 shutdown_connections 2024-08-30T09:00:15.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:15.504+0000 7f63b7bbd640 1 -- 172.21.15.42:0/4172942389 wait complete. 2024-08-30T09:00:15.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:15.504+0000 7f63b7bbd640 1 Processor -- start 2024-08-30T09:00:15.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:15.505+0000 7f63b7bbd640 1 -- start start 2024-08-30T09:00:15.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:15.505+0000 7f63b7bbd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f63b0103960 0x7f63b019a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:15.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:15.505+0000 7f63b7bbd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f63b0109250 0x7f63b019a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:15.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:15.505+0000 7f63b7bbd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f63b0109ba0 0x7f63b019fad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:15.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:15.505+0000 7f63b7bbd640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f63b0113860 con 0x7f63b0103960 2024-08-30T09:00:15.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:15.505+0000 7f63b7bbd640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f63b01136e0 con 0x7f63b0109ba0 2024-08-30T09:00:15.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:15.505+0000 7f63b7bbd640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f63b01139e0 con 0x7f63b0109250 2024-08-30T09:00:15.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:15.505+0000 7f63b5959640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f63b0103960 0x7f63b019a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:15.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:15.505+0000 7f63b5959640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f63b0103960 0x7f63b019a290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:51510/0 (socket says 172.21.15.42:51510) 2024-08-30T09:00:15.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:15.505+0000 7f63b5959640 1 -- 172.21.15.42:0/2260053512 learned_addr learned my addr 172.21.15.42:0/2260053512 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:00:15.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:15.505+0000 7f63b5158640 1 --2- 172.21.15.42:0/2260053512 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f63b0109250 0x7f63b019a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:15.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:15.505+0000 7f63b615a640 1 --2- 172.21.15.42:0/2260053512 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f63b0109ba0 0x7f63b019fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:15.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:15.505+0000 7f63b5959640 1 -- 172.21.15.42:0/2260053512 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f63b0109250 msgr2=0x7f63b019a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:00:15.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:15.505+0000 7f63b5959640 1 --2- 172.21.15.42:0/2260053512 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f63b0109250 0x7f63b019a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:15.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:15.505+0000 7f63b5959640 1 -- 172.21.15.42:0/2260053512 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f63b0109ba0 msgr2=0x7f63b019fad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:00:15.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:15.505+0000 7f63b5959640 1 --2- 172.21.15.42:0/2260053512 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f63b0109ba0 0x7f63b019fad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:15.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:15.505+0000 7f63b5959640 1 -- 172.21.15.42:0/2260053512 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63b01a0140 con 0x7f63b0103960 2024-08-30T09:00:15.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:15.505+0000 7f63b615a640 1 --2- 172.21.15.42:0/2260053512 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f63b0109ba0 0x7f63b019fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-30T09:00:15.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:15.505+0000 7f63b5959640 1 --2- 172.21.15.42:0/2260053512 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f63b0103960 0x7f63b019a290 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f63ac005940 tx=0x7f63ac0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:00:15.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:15.506+0000 7f63a6ffd640 1 -- 172.21.15.42:0/2260053512 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63ac03f030 con 0x7f63b0103960 2024-08-30T09:00:15.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:15.506+0000 7f63a6ffd640 1 -- 172.21.15.42:0/2260053512 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f63ac003cb0 con 0x7f63b0103960 2024-08-30T09:00:15.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:15.506+0000 7f63b7bbd640 1 -- 172.21.15.42:0/2260053512 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63b01a03d0 con 0x7f63b0103960 2024-08-30T09:00:15.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:15.506+0000 7f63a6ffd640 1 -- 172.21.15.42:0/2260053512 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63ac035070 con 0x7f63b0103960 2024-08-30T09:00:15.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:15.506+0000 7f63b7bbd640 1 -- 172.21.15.42:0/2260053512 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63b01a08b0 con 0x7f63b0103960 2024-08-30T09:00:15.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:15.507+0000 7f63b7bbd640 1 -- 172.21.15.42:0/2260053512 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63b0103d70 con 0x7f63b0103960 2024-08-30T09:00:15.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:15.508+0000 7f63a6ffd640 1 -- 172.21.15.42:0/2260053512 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) ==== 152628+0+0 (secure 0 0 0) 0x7f63ac042030 con 0x7f63b0103960 2024-08-30T09:00:15.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:15.509+0000 7f63a6ffd640 1 --2- 172.21.15.42:0/2260053512 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f639c0b4bf0 0x7f639c0b7100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:15.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:15.509+0000 7f63b5158640 1 --2- 172.21.15.42:0/2260053512 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f639c0b4bf0 0x7f639c0b7100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:15.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:15.509+0000 7f63a6ffd640 1 -- 172.21.15.42:0/2260053512 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f63ac105e50 con 0x7f63b0103960 2024-08-30T09:00:15.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:15.509+0000 7f63b5158640 1 --2- 172.21.15.42:0/2260053512 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f639c0b4bf0 0x7f639c0b7100 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f63a0005e10 tx=0x7f63a0005da0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:00:15.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:15.512+0000 7f63a6ffd640 1 -- 172.21.15.42:0/2260053512 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f63ac0cca90 con 0x7f63b0103960 2024-08-30T09:00:15.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:15.698+0000 7f63b7bbd640 1 -- 172.21.15.42:0/2260053512 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f63b010b3e0 con 0x7f63b0103960 2024-08-30T09:00:15.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:15.700+0000 7f63a6ffd640 1 -- 172.21.15.42:0/2260053512 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+24867 (secure 0 0 0) 0x7f63ac0cc390 con 0x7f63b0103960 2024-08-30T09:00:15.702 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-08-30T09:00:15.702 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":83,"fsid":"b0895c1a-66ac-11ef-bcd4-c7b262605968","created":"2024-08-30T08:49:44.858265+0000","modified":"2024-08-30T08:59:46.645920+0000","last_up_change":"2024-08-30T08:57:12.367791+0000","last_in_change":"2024-08-30T08:56:50.274678+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-30T08:53:27.297022+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-30T08:57:58.489964+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-30T08:58:01.064783+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-08-30T08:58:07.416534+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"029d62d7-a6c6-4905-8c76-6dffa9d57b92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":2617890537},{"type":"v1","addr":"172.21.15.42:6803","nonce":2617890537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":2617890537},{"type":"v1","addr":"172.21.15.42:6805","nonce":2617890537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2617890537},{"type":"v1","addr":"172.21.15.42:6809","nonce":2617890537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":2617890537},{"type":"v1","addr":"172.21.15.42:6807","nonce":2617890537}]},"public_addr":"172.21.15.42:6803/2617890537","cluster_addr":"172.21.15.42:6805/2617890537","heartbeat_back_addr":"172.21.15.42:6809/2617890537","heartbeat_front_addr":"172.21.15.42:6807/2617890537","state":["exists","up"]},{"osd":1,"uuid":"3a78384b-0207-41df-9384-22665007d5b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":2791960082},{"type":"v1","addr":"172.21.15.167:6801","nonce":2791960082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":2791960082},{"type":"v1","addr":"172.21.15.167:6803","nonce":2791960082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":2791960082},{"type":"v1","addr":"172.21.15.167:6807","nonce":2791960082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":2791960082},{"type":"v1","addr":"172.21.15.167:6805","nonce":2791960082}]},"public_addr":"172.21.15.167:6801/2791960082","cluster_addr":"172.21.15.167:6803/2791960082","heartbeat_back_addr":"172.21.15.167:6807/2791960082","heartbeat_front_addr":"172.21.15.167:6805/2791960082","state":["exists","up"]},{"osd":2,"uuid":"16001809-c27c-4f52-948a-9998261acb4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":1101326404},{"type":"v1","addr":"172.21.15.190:6801","nonce":1101326404}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":1101326404},{"type":"v1","addr":"172.21.15.190:6803","nonce":1101326404}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":1101326404},{"type":"v1","addr":"172.21.15.190:6807","nonce":1101326404}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":1101326404},{"type":"v1","addr":"172.21.15.190:6805","nonce":1101326404}]},"public_addr":"172.21.15.190:6801/1101326404","cluster_addr":"172.21.15.190:6803/1101326404","heartbeat_back_addr":"172.21.15.190:6807/1101326404","heartbeat_front_addr":"172.21.15.190:6805/1101326404","state":["exists","up"]},{"osd":3,"uuid":"87bb3e36-9e9e-4b19-9d8c-0cdf6e089d3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":1135599916},{"type":"v1","addr":"172.21.15.42:6811","nonce":1135599916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":1135599916},{"type":"v1","addr":"172.21.15.42:6813","nonce":1135599916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":1135599916},{"type":"v1","addr":"172.21.15.42:6817","nonce":1135599916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":1135599916},{"type":"v1","addr":"172.21.15.42:6815","nonce":1135599916}]},"public_addr":"172.21.15.42:6811/1135599916","cluster_addr":"172.21.15.42:6813/1135599916","heartbeat_back_addr":"172.21.15.42:6817/1135599916","heartbeat_front_addr":"172.21.15.42:6815/1135599916","state":["exists","up"]},{"osd":4,"uuid":"3ca1aae3-c972-4fb9-ac86-236279c30d64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":2927511419},{"type":"v1","addr":"172.21.15.167:6809","nonce":2927511419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":2927511419},{"type":"v1","addr":"172.21.15.167:6811","nonce":2927511419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":2927511419},{"type":"v1","addr":"172.21.15.167:6815","nonce":2927511419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":2927511419},{"type":"v1","addr":"172.21.15.167:6813","nonce":2927511419}]},"public_addr":"172.21.15.167:6809/2927511419","cluster_addr":"172.21.15.167:6811/2927511419","heartbeat_back_addr":"172.21.15.167:6815/2927511419","heartbeat_front_addr":"172.21.15.167:6813/2927511419","state":["exists","up"]},{"osd":5,"uuid":"5d07f54b-0d07-4efe-9bcf-390755ce0258","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":2557145333},{"type":"v1","addr":"172.21.15.190:6809","nonce":2557145333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":2557145333},{"type":"v1","addr":"172.21.15.190:6811","nonce":2557145333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":2557145333},{"type":"v1","addr":"172.21.15.190:6815","nonce":2557145333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":2557145333},{"type":"v1","addr":"172.21.15.190:6813","nonce":2557145333}]},"public_addr":"172.21.15.190:6809/2557145333","cluster_addr":"172.21.15.190:6811/2557145333","heartbeat_back_addr":"172.21.15.190:6815/2557145333","heartbeat_front_addr":"172.21.15.190:6813/2557145333","state":["exists","up"]},{"osd":6,"uuid":"96ad64af-44ff-4197-97d4-156f0cb88e81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":3213526100},{"type":"v1","addr":"172.21.15.42:6819","nonce":3213526100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":3213526100},{"type":"v1","addr":"172.21.15.42:6821","nonce":3213526100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":3213526100},{"type":"v1","addr":"172.21.15.42:6825","nonce":3213526100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":3213526100},{"type":"v1","addr":"172.21.15.42:6823","nonce":3213526100}]},"public_addr":"172.21.15.42:6819/3213526100","cluster_addr":"172.21.15.42:6821/3213526100","heartbeat_back_addr":"172.21.15.42:6825/3213526100","heartbeat_front_addr":"172.21.15.42:6823/3213526100","state":["exists","up"]},{"osd":7,"uuid":"337a2112-eaf7-4e6b-86be-4e40d6ddf835","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":2962799814},{"type":"v1","addr":"172.21.15.167:6817","nonce":2962799814}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":2962799814},{"type":"v1","addr":"172.21.15.167:6819","nonce":2962799814}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":2962799814},{"type":"v1","addr":"172.21.15.167:6823","nonce":2962799814}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":2962799814},{"type":"v1","addr":"172.21.15.167:6821","nonce":2962799814}]},"public_addr":"172.21.15.167:6817/2962799814","cluster_addr":"172.21.15.167:6819/2962799814","heartbeat_back_addr":"172.21.15.167:6823/2962799814","heartbeat_front_addr":"172.21.15.167:6821/2962799814","state":["exists","up"]},{"osd":8,"uuid":"219a7224-4331-464b-ac86-c51054d51d91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":3242198843},{"type":"v1","addr":"172.21.15.190:6817","nonce":3242198843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":3242198843},{"type":"v1","addr":"172.21.15.190:6819","nonce":3242198843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":3242198843},{"type":"v1","addr":"172.21.15.190:6823","nonce":3242198843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":3242198843},{"type":"v1","addr":"172.21.15.190:6821","nonce":3242198843}]},"public_addr":"172.21.15.190:6817/3242198843","cluster_addr":"172.21.15.190:6819/3242198843","heartbeat_back_addr":"172.21.15.190:6823/3242198843","heartbeat_front_addr":"172.21.15.190:6821/3242198843","state":["exists","up"]},{"osd":9,"uuid":"2f2f5177-a009-40d9-991c-76aec8a5cba6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":1123536325},{"type":"v1","addr":"172.21.15.42:6827","nonce":1123536325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":1123536325},{"type":"v1","addr":"172.21.15.42:6829","nonce":1123536325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":1123536325},{"type":"v1","addr":"172.21.15.42:6833","nonce":1123536325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":1123536325},{"type":"v1","addr":"172.21.15.42:6831","nonce":1123536325}]},"public_addr":"172.21.15.42:6827/1123536325","cluster_addr":"172.21.15.42:6829/1123536325","heartbeat_back_addr":"172.21.15.42:6833/1123536325","heartbeat_front_addr":"172.21.15.42:6831/1123536325","state":["exists","up"]},{"osd":10,"uuid":"96ef5058-2518-4862-a165-17666b33a3e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":3451198319},{"type":"v1","addr":"172.21.15.167:6825","nonce":3451198319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":3451198319},{"type":"v1","addr":"172.21.15.167:6827","nonce":3451198319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":3451198319},{"type":"v1","addr":"172.21.15.167:6831","nonce":3451198319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":3451198319},{"type":"v1","addr":"172.21.15.167:6829","nonce":3451198319}]},"public_addr":"172.21.15.167:6825/3451198319","cluster_addr":"172.21.15.167:6827/3451198319","heartbeat_back_addr":"172.21.15.167:6831/3451198319","heartbeat_front_addr":"172.21.15.167:6829/3451198319","state":["exists","up"]},{"osd":11,"uuid":"a4eb9bd3-3faa-4c81-ba6d-c2084b72c9ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":380064457},{"type":"v1","addr":"172.21.15.190:6825","nonce":380064457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":380064457},{"type":"v1","addr":"172.21.15.190:6827","nonce":380064457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":380064457},{"type":"v1","addr":"172.21.15.190:6831","nonce":380064457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":380064457},{"type":"v1","addr":"172.21.15.190:6829","nonce":380064457}]},"public_addr":"172.21.15.190:6825/380064457","cluster_addr":"172.21.15.190:6827/380064457","heartbeat_back_addr":"172.21.15.190:6831/380064457","heartbeat_front_addr":"172.21.15.190:6829/380064457","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-30T08:52:36.511448+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-30T08:53:00.027277+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-30T08:53:23.770245+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-30T08:53:49.322226+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-30T08:54:14.350234+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-30T08:54:39.491793+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-30T08:55:04.870596+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-30T08:55:30.314150+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-30T08:55:54.483843+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-30T08:56:20.586115+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-30T08:56:45.666783+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-30T08:57:10.718387+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"2.1d","mappings":[{"from":2,"to":1}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"4.9","mappings":[{"from":5,"to":8}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.42:0/2691233260":"2024-08-31T08:50:44.821070+0000","172.21.15.42:0/1518053351":"2024-08-31T08:50:44.821070+0000","172.21.15.42:0/352001020":"2024-08-31T08:50:44.821070+0000","172.21.15.42:6801/1733768634":"2024-08-31T08:50:44.821070+0000","172.21.15.42:0/1000597888":"2024-08-31T08:50:18.561851+0000","172.21.15.42:6800/1733768634":"2024-08-31T08:50:44.821070+0000","172.21.15.42:6801/3538882081":"2024-08-31T08:50:18.561851+0000","172.21.15.42:6800/3538882081":"2024-08-31T08:50:18.561851+0000","172.21.15.42:0/145366508":"2024-08-31T08:50:18.561851+0000","172.21.15.42:0/3346355213":"2024-08-31T08:50:18.561851+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-30T09:00:15.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:15.702+0000 7f63b7bbd640 1 -- 172.21.15.42:0/2260053512 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f639c0b4bf0 msgr2=0x7f639c0b7100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:00:15.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:15.702+0000 7f63b7bbd640 1 --2- 172.21.15.42:0/2260053512 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f639c0b4bf0 0x7f639c0b7100 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f63a0005e10 tx=0x7f63a0005da0 comp rx=0 tx=0).stop 2024-08-30T09:00:15.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:15.702+0000 7f63b7bbd640 1 -- 172.21.15.42:0/2260053512 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f63b0103960 msgr2=0x7f63b019a290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:00:15.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:15.702+0000 7f63b7bbd640 1 --2- 172.21.15.42:0/2260053512 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f63b0103960 0x7f63b019a290 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f63ac005940 tx=0x7f63ac0042f0 comp rx=0 tx=0).stop 2024-08-30T09:00:15.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:15.702+0000 7f63b7bbd640 1 -- 172.21.15.42:0/2260053512 shutdown_connections 2024-08-30T09:00:15.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:15.702+0000 7f63b7bbd640 1 --2- 172.21.15.42:0/2260053512 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f639c0b4bf0 0x7f639c0b7100 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:15.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:15.702+0000 7f63b7bbd640 1 --2- 172.21.15.42:0/2260053512 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f63b0109250 0x7f63b019a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:15.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:15.702+0000 7f63b7bbd640 1 --2- 172.21.15.42:0/2260053512 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f63b0109ba0 0x7f63b019fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:15.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:15.702+0000 7f63b7bbd640 1 --2- 172.21.15.42:0/2260053512 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f63b0103960 0x7f63b019a290 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:15.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:15.702+0000 7f63b7bbd640 1 -- 172.21.15.42:0/2260053512 >> 172.21.15.42:0/2260053512 conn(0x7f63b007b6b0 msgr2=0x7f63b0107340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:00:15.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:15.702+0000 7f63b7bbd640 1 -- 172.21.15.42:0/2260053512 shutdown_connections 2024-08-30T09:00:15.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:15.702+0000 7f63b7bbd640 1 -- 172.21.15.42:0/2260053512 wait complete. 2024-08-30T09:00:16.411 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 26, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-08-30T08:58:03.055084+0000', 'modified': '2024-08-30T08:58:50.586122+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24629, 'mds_1': 24610, 'mds_2': 24599}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24572': {'gid': 24572, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.167:6835/234596845', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.167:6834', 'nonce': 234596845}, {'type': 'v1', 'addr': '172.21.15.167:6835', 'nonce': 234596845}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24592': {'gid': 24592, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.190:6835/247905709', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.190:6834', 'nonce': 247905709}, {'type': 'v1', 'addr': '172.21.15.190:6835', 'nonce': 247905709}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24598': {'gid': 24598, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.190:6837/863256349', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.190:6836', 'nonce': 863256349}, {'type': 'v1', 'addr': '172.21.15.190:6837', 'nonce': 863256349}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24599': {'gid': 24599, 'name': 'h', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 19, 'addr': '172.21.15.167:6837/3011837856', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.167:6836', 'nonce': 3011837856}, {'type': 'v1', 'addr': '172.21.15.167:6837', 'nonce': 3011837856}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24610': {'gid': 24610, 'name': 'i', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 17, 'addr': '172.21.15.190:6839/719981110', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.190:6838', 'nonce': 719981110}, {'type': 'v1', 'addr': '172.21.15.190:6839', 'nonce': 719981110}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24629': {'gid': 24629, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.167:6839/3604645207', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.167:6838', 'nonce': 3604645207}, {'type': 'v1', 'addr': '172.21.15.167:6839', 'nonce': 3604645207}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24629, 'qdb_cluster': [24629, 24610, 24599]} 2024-08-30T09:00:16.411 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-08-30T09:00:16.411 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph tell mds.h status 2024-08-30T09:00:16.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:00:16 smithi042 ceph-mon[18652]: pgmap v358: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-08-30T09:00:16.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:00:16 smithi042 ceph-mon[18652]: from='client.? 172.21.15.42:0/2260053512' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-30T09:00:16.665 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T09:00:16.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:00:16 smithi190 ceph-mon[22001]: pgmap v358: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-08-30T09:00:16.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:00:16 smithi190 ceph-mon[22001]: from='client.? 172.21.15.42:0/2260053512' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-30T09:00:16.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:00:16 smithi167 ceph-mon[22063]: pgmap v358: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-08-30T09:00:16.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:00:16 smithi167 ceph-mon[22063]: from='client.? 172.21.15.42:0/2260053512' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-30T09:00:17.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.106+0000 7fd058c4a640 1 -- 172.21.15.42:0/1704750162 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd0541050a0 msgr2=0x7fd05410d5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:00:17.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.106+0000 7fd058c4a640 1 --2- 172.21.15.42:0/1704750162 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd0541050a0 0x7fd05410d5f0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fd048009900 tx=0x7fd04802a410 comp rx=0 tx=0).stop 2024-08-30T09:00:17.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.106+0000 7fd058c4a640 1 -- 172.21.15.42:0/1704750162 shutdown_connections 2024-08-30T09:00:17.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.106+0000 7fd058c4a640 1 --2- 172.21.15.42:0/1704750162 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd05410dbb0 0x7fd0541100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:17.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.106+0000 7fd058c4a640 1 --2- 172.21.15.42:0/1704750162 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd0541050a0 0x7fd05410d5f0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:17.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.106+0000 7fd058c4a640 1 --2- 172.21.15.42:0/1704750162 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd054104750 0x7fd054104b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:17.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.106+0000 7fd058c4a640 1 -- 172.21.15.42:0/1704750162 >> 172.21.15.42:0/1704750162 conn(0x7fd0540794d0 msgr2=0x7fd0540798e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:00:17.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.106+0000 7fd058c4a640 1 -- 172.21.15.42:0/1704750162 shutdown_connections 2024-08-30T09:00:17.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.106+0000 7fd058c4a640 1 -- 172.21.15.42:0/1704750162 wait complete. 2024-08-30T09:00:17.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.107+0000 7fd058c4a640 1 Processor -- start 2024-08-30T09:00:17.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.107+0000 7fd058c4a640 1 -- start start 2024-08-30T09:00:17.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.107+0000 7fd058c4a640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd054104750 0x7fd05419c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:17.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.107+0000 7fd058c4a640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd0541050a0 0x7fd05419ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:17.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.107+0000 7fd058c4a640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd05410dbb0 0x7fd0541a1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:17.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.107+0000 7fd058c4a640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fd054113870 con 0x7fd054104750 2024-08-30T09:00:17.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.107+0000 7fd058c4a640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd0541136f0 con 0x7fd0541050a0 2024-08-30T09:00:17.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.107+0000 7fd058c4a640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fd0541139f0 con 0x7fd05410dbb0 2024-08-30T09:00:17.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.107+0000 7fd05259c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd054104750 0x7fd05419c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:17.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.108+0000 7fd05259c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd054104750 0x7fd05419c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:51532/0 (socket says 172.21.15.42:51532) 2024-08-30T09:00:17.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.108+0000 7fd05259c640 1 -- 172.21.15.42:0/3526207846 learned_addr learned my addr 172.21.15.42:0/3526207846 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:00:17.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.108+0000 7fd052d9d640 1 --2- 172.21.15.42:0/3526207846 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd05410dbb0 0x7fd0541a1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:17.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.108+0000 7fd051d9b640 1 --2- 172.21.15.42:0/3526207846 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd0541050a0 0x7fd05419ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:17.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.108+0000 7fd05259c640 1 -- 172.21.15.42:0/3526207846 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd05410dbb0 msgr2=0x7fd0541a1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:00:17.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.108+0000 7fd05259c640 1 --2- 172.21.15.42:0/3526207846 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd05410dbb0 0x7fd0541a1cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:17.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.108+0000 7fd05259c640 1 -- 172.21.15.42:0/3526207846 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd0541050a0 msgr2=0x7fd05419ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:00:17.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.108+0000 7fd05259c640 1 --2- 172.21.15.42:0/3526207846 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd0541050a0 0x7fd05419ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:17.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.108+0000 7fd05259c640 1 -- 172.21.15.42:0/3526207846 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0541a23f0 con 0x7fd054104750 2024-08-30T09:00:17.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.108+0000 7fd05259c640 1 --2- 172.21.15.42:0/3526207846 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd054104750 0x7fd05419c480 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fd04400d950 tx=0x7fd04400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:00:17.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.108+0000 7fd0437fe640 1 -- 172.21.15.42:0/3526207846 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd044017070 con 0x7fd054104750 2024-08-30T09:00:17.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.108+0000 7fd0437fe640 1 -- 172.21.15.42:0/3526207846 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd044009c30 con 0x7fd054104750 2024-08-30T09:00:17.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.108+0000 7fd058c4a640 1 -- 172.21.15.42:0/3526207846 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0541a26e0 con 0x7fd054104750 2024-08-30T09:00:17.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.108+0000 7fd0437fe640 1 -- 172.21.15.42:0/3526207846 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd04400f620 con 0x7fd054104750 2024-08-30T09:00:17.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.108+0000 7fd058c4a640 1 -- 172.21.15.42:0/3526207846 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0541a8e40 con 0x7fd054104750 2024-08-30T09:00:17.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.111+0000 7fd0437fe640 1 -- 172.21.15.42:0/3526207846 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) ==== 152628+0+0 (secure 0 0 0) 0x7fd04401a070 con 0x7fd054104750 2024-08-30T09:00:17.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.111+0000 7fd05f1bf740 1 Processor -- start 2024-08-30T09:00:17.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.111+0000 7fd05f1bf740 1 -- start start 2024-08-30T09:00:17.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.112+0000 7fd05f1bf740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557752143f60 0x5577524e97b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:17.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.112+0000 7fd05f1bf740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55775211b0a0 0x5577524e9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:17.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.112+0000 7fd05f1bf740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557752123280 0x5577524ea230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:17.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.112+0000 7fd05f1bf740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x5577524603f0 con 0x557752143f60 2024-08-30T09:00:17.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.112+0000 7fd05f1bf740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55775244f980 con 0x55775211b0a0 2024-08-30T09:00:17.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.112+0000 7fd05f1bf740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x557752453930 con 0x557752123280 2024-08-30T09:00:17.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.112+0000 7fd052d9d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557752143f60 0x5577524e97b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:17.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.112+0000 7fd052d9d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557752143f60 0x5577524e97b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:51536/0 (socket says 172.21.15.42:51536) 2024-08-30T09:00:17.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.112+0000 7fd052d9d640 1 -- 172.21.15.42:0/997168070 learned_addr learned my addr 172.21.15.42:0/997168070 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:00:17.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.112+0000 7fd051d9b640 1 --2- 172.21.15.42:0/997168070 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557752123280 0x5577524ea230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:17.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.112+0000 7fd05259c640 1 --2- 172.21.15.42:0/997168070 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55775211b0a0 0x5577524e9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:17.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.112+0000 7fd0437fe640 1 --2- 172.21.15.42:0/3526207846 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fd03c0b4c40 0x7fd03c0b7150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:17.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.112+0000 7fd052d9d640 1 -- 172.21.15.42:0/997168070 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557752123280 msgr2=0x5577524ea230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:00:17.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.112+0000 7fd052d9d640 1 --2- 172.21.15.42:0/997168070 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557752123280 0x5577524ea230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:17.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.112+0000 7fd052d9d640 1 -- 172.21.15.42:0/997168070 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55775211b0a0 msgr2=0x5577524e9cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:00:17.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.112+0000 7fd052d9d640 1 --2- 172.21.15.42:0/997168070 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55775211b0a0 0x5577524e9cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:17.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.112+0000 7fd052d9d640 1 -- 172.21.15.42:0/997168070 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5577521af540 con 0x557752143f60 2024-08-30T09:00:17.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.113+0000 7fd0437fe640 1 -- 172.21.15.42:0/3526207846 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7fd0440e5010 con 0x7fd054104750 2024-08-30T09:00:17.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.113+0000 7fd052d9d640 1 --2- 172.21.15.42:0/3526207846 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fd03c0b4c40 0x7fd03c0b7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:17.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.113+0000 7fd052d9d640 1 --2- 172.21.15.42:0/997168070 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557752143f60 0x5577524e97b0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fd04c00a5d0 tx=0x7fd04c00aa90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3499524fa9ad762 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:00:17.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.113+0000 7fd040ff9640 1 -- 172.21.15.42:0/997168070 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd04c01a040 con 0x557752143f60 2024-08-30T09:00:17.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.113+0000 7fd040ff9640 1 -- 172.21.15.42:0/997168070 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd04c021ba0 con 0x557752143f60 2024-08-30T09:00:17.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.113+0000 7fd040ff9640 1 -- 172.21.15.42:0/997168070 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd04c020e00 con 0x557752143f60 2024-08-30T09:00:17.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.113+0000 7fd052d9d640 1 --2- 172.21.15.42:0/3526207846 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fd03c0b4c40 0x7fd03c0b7150 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fd04c020000 tx=0x7fd04c021000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:00:17.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.114+0000 7fd05f1bf740 1 -- 172.21.15.42:0/997168070 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557752143f60 msgr2=0x5577524e97b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:00:17.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.114+0000 7fd05f1bf740 1 --2- 172.21.15.42:0/997168070 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557752143f60 0x5577524e97b0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fd04c00a5d0 tx=0x7fd04c00aa90 comp rx=0 tx=0).stop 2024-08-30T09:00:17.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.114+0000 7fd05f1bf740 1 -- 172.21.15.42:0/997168070 shutdown_connections 2024-08-30T09:00:17.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.114+0000 7fd05f1bf740 1 --2- 172.21.15.42:0/997168070 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557752123280 0x5577524ea230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:17.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.114+0000 7fd05f1bf740 1 --2- 172.21.15.42:0/997168070 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55775211b0a0 0x5577524e9cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:17.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.114+0000 7fd05f1bf740 1 --2- 172.21.15.42:0/997168070 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557752143f60 0x5577524e97b0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:17.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.114+0000 7fd05f1bf740 1 -- 172.21.15.42:0/997168070 >> 172.21.15.42:0/997168070 conn(0x557752138350 msgr2=0x5577524e76a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:00:17.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.114+0000 7fd05f1bf740 1 -- 172.21.15.42:0/997168070 shutdown_connections 2024-08-30T09:00:17.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.114+0000 7fd05f1bf740 1 -- 172.21.15.42:0/997168070 wait complete. 2024-08-30T09:00:17.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.114+0000 7fd05f1bf740 1 -- start start 2024-08-30T09:00:17.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.115+0000 7fd05f1bf740 1 Processor -- start 2024-08-30T09:00:17.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.115+0000 7fd05f1bf740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55775211b0a0 0x5577524f4e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:17.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.115+0000 7fd05f1bf740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557752123280 0x5577524f53b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:17.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.115+0000 7fd05f1bf740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557752143f60 0x557752501280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:17.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.115+0000 7fd05f1bf740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x5577522b25f0 con 0x557752143f60 2024-08-30T09:00:17.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.115+0000 7fd05f1bf740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5577523d6600 con 0x557752123280 2024-08-30T09:00:17.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.115+0000 7fd05f1bf740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5577524f7f90 con 0x55775211b0a0 2024-08-30T09:00:17.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.115+0000 7fd05f1bf740 10 client.0 _ll_register_callbacks cb 0x5577523fcaf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:00:17.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.115+0000 7fd052d9d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557752143f60 0x557752501280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:17.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.115+0000 7fd052d9d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557752143f60 0x557752501280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:51540/0 (socket says 172.21.15.42:51540) 2024-08-30T09:00:17.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.115+0000 7fd052d9d640 1 -- 172.21.15.42:0/1080808992 learned_addr learned my addr 172.21.15.42:0/1080808992 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:00:17.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.115+0000 7fd05259c640 1 --2- 172.21.15.42:0/1080808992 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55775211b0a0 0x5577524f4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:17.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.115+0000 7fd051d9b640 1 --2- 172.21.15.42:0/1080808992 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557752123280 0x5577524f53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:17.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.115+0000 7fd052d9d640 1 -- 172.21.15.42:0/1080808992 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55775211b0a0 msgr2=0x5577524f4e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:00:17.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.115+0000 7fd052d9d640 1 --2- 172.21.15.42:0/1080808992 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55775211b0a0 0x5577524f4e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:17.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.115+0000 7fd052d9d640 1 -- 172.21.15.42:0/1080808992 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557752123280 msgr2=0x5577524f53b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:00:17.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.115+0000 7fd052d9d640 1 --2- 172.21.15.42:0/1080808992 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557752123280 0x5577524f53b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:17.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.115+0000 7fd052d9d640 1 -- 172.21.15.42:0/1080808992 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55775240f7b0 con 0x557752143f60 2024-08-30T09:00:17.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.116+0000 7fd052d9d640 1 --2- 172.21.15.42:0/1080808992 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557752143f60 0x557752501280 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fd04c0165b0 tx=0x7fd04c009d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=146dc38712c8f0a1 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:00:17.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.116+0000 7fd039ffb640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-08-30T09:00:17.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.116+0000 7fd039ffb640 1 -- 172.21.15.42:0/1080808992 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd04c02d070 con 0x557752143f60 2024-08-30T09:00:17.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.116+0000 7fd05f1bf740 1 -- 172.21.15.42:0/1080808992 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd04c021ba0 con 0x557752143f60 2024-08-30T09:00:17.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.116+0000 7fd039ffb640 1 -- 172.21.15.42:0/1080808992 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7fd04c020b80 con 0x557752143f60 2024-08-30T09:00:17.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.116+0000 7fd039ffb640 1 -- 172.21.15.42:0/1080808992 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fd04c039050 con 0x557752143f60 2024-08-30T09:00:17.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.116+0000 7fd05f1bf740 10 client.15351 fetch_fsmap learned FSMap version 26 2024-08-30T09:00:17.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.116+0000 7fd05f1bf740 1 -- 172.21.15.42:0/1080808992 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x557752503c90 con 0x557752143f60 2024-08-30T09:00:17.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.116+0000 7fd039ffb640 1 -- 172.21.15.42:0/1080808992 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd04c021ba0 con 0x557752143f60 2024-08-30T09:00:17.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.116+0000 7fd039ffb640 1 -- 172.21.15.42:0/1080808992 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fd04c028040 con 0x557752143f60 2024-08-30T09:00:17.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.117+0000 7fd05f1bf740 10 client.15351 fetch_fsmap finished waiting for FSMap version 26 2024-08-30T09:00:17.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.117+0000 7fd05f1bf740 10 client.15351 resolve_mds: resolved name 'h' to daemon mds.h 2024-08-30T09:00:17.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.117+0000 7fd05f1bf740 20 client.15351 populate_metadata read hostname 'smithi042' 2024-08-30T09:00:17.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.117+0000 7fd05f1bf740 1 --2- 172.21.15.42:0/1080808992 >> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] conn(0x5577521028e0 0x557752507bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:17.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.117+0000 7fd05f1bf740 4 client.15351 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-30T09:00:17.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.117+0000 7fd05f1bf740 1 -- 172.21.15.42:0/1080808992 --> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557752462340 con 0x5577521028e0 2024-08-30T09:00:17.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.117+0000 7fd05259c640 1 --2- 172.21.15.42:0/1080808992 >> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] conn(0x5577521028e0 0x557752507bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:17.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.117+0000 7fd039ffb640 10 client.15351 ms_handle_connect on v2:172.21.15.167:6836/3011837856 2024-08-30T09:00:17.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.117+0000 7fd05259c640 1 --2- 172.21.15.42:0/1080808992 >> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] conn(0x5577521028e0 0x557752507bc0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=997e8c76a077e864 server_cookie=36737190507c806c in_seq=0 out_seq=0 2024-08-30T09:00:17.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.118+0000 7fd039ffb640 1 -- 172.21.15.42:0/1080808992 <== mds.2 v2:172.21.15.167:6836/3011837856 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x557752462340 con 0x5577521028e0 2024-08-30T09:00:17.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.118+0000 7fd039ffb640 10 client.15351 handle_command_reply: tid=0 2024-08-30T09:00:17.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.118+0000 7fd05f1bf740 1 client.15351 shutdown 2024-08-30T09:00:17.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.118+0000 7fd05f1bf740 1 -- 172.21.15.42:0/1080808992 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557752143f60 msgr2=0x557752501280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:00:17.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.118+0000 7fd05f1bf740 1 --2- 172.21.15.42:0/1080808992 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557752143f60 0x557752501280 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fd04c0165b0 tx=0x7fd04c009d50 comp rx=0 tx=0).stop 2024-08-30T09:00:17.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.118+0000 7fd05f1bf740 1 -- 172.21.15.42:0/1080808992 shutdown_connections 2024-08-30T09:00:17.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.118+0000 7fd05f1bf740 1 --2- 172.21.15.42:0/1080808992 >> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] conn(0x5577521028e0 0x557752507bc0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:17.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.118+0000 7fd05f1bf740 1 --2- 172.21.15.42:0/1080808992 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557752143f60 0x557752501280 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:17.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.118+0000 7fd05f1bf740 1 --2- 172.21.15.42:0/1080808992 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557752123280 0x5577524f53b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:17.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.119+0000 7fd05f1bf740 1 --2- 172.21.15.42:0/1080808992 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55775211b0a0 0x5577524f4e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:17.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.119+0000 7fd039ffb640 0 client.15351 ms_handle_reset on v2:172.21.15.167:6836/3011837856 2024-08-30T09:00:17.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.119+0000 7fd05f1bf740 1 -- 172.21.15.42:0/1080808992 >> 172.21.15.42:0/1080808992 conn(0x557752138350 msgr2=0x5577524ec990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:00:17.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.119+0000 7fd05f1bf740 1 -- 172.21.15.42:0/1080808992 shutdown_connections 2024-08-30T09:00:17.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.119+0000 7fd05f1bf740 1 -- 172.21.15.42:0/1080808992 wait complete. 2024-08-30T09:00:17.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.119+0000 7fd05f1bf740 20 client.15351 trim_cache size 0 max 16384 2024-08-30T09:00:17.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.137+0000 7fd05f1bf740 1 Processor -- start 2024-08-30T09:00:17.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.137+0000 7fd05f1bf740 1 -- start start 2024-08-30T09:00:17.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.137+0000 7fd05f1bf740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55775211b0a0 0x5577525011f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:17.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.137+0000 7fd05f1bf740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557752123280 0x5577524e6cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:17.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.137+0000 7fd05f1bf740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557752143f60 0x5577524e7230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:17.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.137+0000 7fd05f1bf740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x557752505f00 con 0x557752143f60 2024-08-30T09:00:17.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.137+0000 7fd05f1bf740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5577525065d0 con 0x557752123280 2024-08-30T09:00:17.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.137+0000 7fd05f1bf740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x557752506ca0 con 0x55775211b0a0 2024-08-30T09:00:17.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.137+0000 7fd052d9d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557752143f60 0x5577524e7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:17.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.137+0000 7fd052d9d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557752143f60 0x5577524e7230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:51550/0 (socket says 172.21.15.42:51550) 2024-08-30T09:00:17.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.137+0000 7fd052d9d640 1 -- 172.21.15.42:0/1784561468 learned_addr learned my addr 172.21.15.42:0/1784561468 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:00:17.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.138+0000 7fd05259c640 1 --2- 172.21.15.42:0/1784561468 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55775211b0a0 0x5577525011f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:17.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.138+0000 7fd051d9b640 1 --2- 172.21.15.42:0/1784561468 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557752123280 0x5577524e6cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:17.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.138+0000 7fd052d9d640 1 -- 172.21.15.42:0/1784561468 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55775211b0a0 msgr2=0x5577525011f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:00:17.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.138+0000 7fd052d9d640 1 --2- 172.21.15.42:0/1784561468 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55775211b0a0 0x5577525011f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:17.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.138+0000 7fd052d9d640 1 -- 172.21.15.42:0/1784561468 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557752123280 msgr2=0x5577524e6cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:00:17.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.138+0000 7fd052d9d640 1 --2- 172.21.15.42:0/1784561468 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557752123280 0x5577524e6cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:17.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.138+0000 7fd052d9d640 1 -- 172.21.15.42:0/1784561468 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557752102b10 con 0x557752143f60 2024-08-30T09:00:17.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.138+0000 7fd052d9d640 1 --2- 172.21.15.42:0/1784561468 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557752143f60 0x5577524e7230 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fd04c004b90 tx=0x7fd04c00aa90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19e83fb0d5ba5864 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:00:17.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.138+0000 7fd03bfff640 1 -- 172.21.15.42:0/1784561468 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd04c036070 con 0x557752143f60 2024-08-30T09:00:17.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.138+0000 7fd03bfff640 1 -- 172.21.15.42:0/1784561468 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd04c021cd0 con 0x557752143f60 2024-08-30T09:00:17.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.138+0000 7fd03bfff640 1 -- 172.21.15.42:0/1784561468 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd04c02d040 con 0x557752143f60 2024-08-30T09:00:17.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.139+0000 7fd05f1bf740 1 -- 172.21.15.42:0/1784561468 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557752143f60 msgr2=0x5577524e7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:00:17.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.139+0000 7fd05f1bf740 1 --2- 172.21.15.42:0/1784561468 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557752143f60 0x5577524e7230 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fd04c004b90 tx=0x7fd04c00aa90 comp rx=0 tx=0).stop 2024-08-30T09:00:17.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.139+0000 7fd05f1bf740 1 -- 172.21.15.42:0/1784561468 shutdown_connections 2024-08-30T09:00:17.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.139+0000 7fd05f1bf740 1 --2- 172.21.15.42:0/1784561468 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557752143f60 0x5577524e7230 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:17.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.139+0000 7fd05f1bf740 1 --2- 172.21.15.42:0/1784561468 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557752123280 0x5577524e6cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:17.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.139+0000 7fd05f1bf740 1 --2- 172.21.15.42:0/1784561468 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55775211b0a0 0x5577525011f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:17.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.139+0000 7fd05f1bf740 1 -- 172.21.15.42:0/1784561468 >> 172.21.15.42:0/1784561468 conn(0x557752138350 msgr2=0x5577524f36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:00:17.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.139+0000 7fd05f1bf740 1 -- 172.21.15.42:0/1784561468 shutdown_connections 2024-08-30T09:00:17.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.139+0000 7fd05f1bf740 1 -- 172.21.15.42:0/1784561468 wait complete. 2024-08-30T09:00:17.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.139+0000 7fd05f1bf740 1 -- start start 2024-08-30T09:00:17.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.139+0000 7fd05f1bf740 1 Processor -- start 2024-08-30T09:00:17.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.140+0000 7fd05f1bf740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55775211b0a0 0x5577525102f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:17.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.140+0000 7fd05f1bf740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557752123280 0x557752510850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:17.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.140+0000 7fd05f1bf740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557752143f60 0x55775250e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:17.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.140+0000 7fd05f1bf740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x557752507a40 con 0x557752143f60 2024-08-30T09:00:17.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.140+0000 7fd05f1bf740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x557752507370 con 0x557752123280 2024-08-30T09:00:17.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.140+0000 7fd05f1bf740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x557752508110 con 0x55775211b0a0 2024-08-30T09:00:17.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.140+0000 7fd05f1bf740 10 client.0 _ll_register_callbacks cb 0x7fd04c009b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:00:17.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.140+0000 7fd052d9d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557752143f60 0x55775250e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:17.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.140+0000 7fd052d9d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557752143f60 0x55775250e000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:51566/0 (socket says 172.21.15.42:51566) 2024-08-30T09:00:17.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.140+0000 7fd052d9d640 1 -- 172.21.15.42:0/3207848453 learned_addr learned my addr 172.21.15.42:0/3207848453 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:00:17.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.140+0000 7fd05259c640 1 --2- 172.21.15.42:0/3207848453 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55775211b0a0 0x5577525102f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:17.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.140+0000 7fd051d9b640 1 --2- 172.21.15.42:0/3207848453 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557752123280 0x557752510850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:17.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.140+0000 7fd052d9d640 1 -- 172.21.15.42:0/3207848453 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55775211b0a0 msgr2=0x5577525102f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:00:17.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.140+0000 7fd052d9d640 1 --2- 172.21.15.42:0/3207848453 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55775211b0a0 0x5577525102f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:17.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.140+0000 7fd052d9d640 1 -- 172.21.15.42:0/3207848453 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557752123280 msgr2=0x557752510850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:00:17.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.140+0000 7fd052d9d640 1 --2- 172.21.15.42:0/3207848453 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557752123280 0x557752510850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:17.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.140+0000 7fd052d9d640 1 -- 172.21.15.42:0/3207848453 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55775250e790 con 0x557752143f60 2024-08-30T09:00:17.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.140+0000 7fd051d9b640 1 --2- 172.21.15.42:0/3207848453 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557752123280 0x557752510850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-30T09:00:17.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.140+0000 7fd052d9d640 1 --2- 172.21.15.42:0/3207848453 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557752143f60 0x55775250e000 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fd04c009e00 tx=0x7fd04c00a400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71841ac269e30068 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:00:17.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.140+0000 7fd039ffb640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-08-30T09:00:17.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.141+0000 7fd039ffb640 1 -- 172.21.15.42:0/3207848453 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd04c036070 con 0x557752143f60 2024-08-30T09:00:17.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.141+0000 7fd039ffb640 1 -- 172.21.15.42:0/3207848453 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7fd04c021a20 con 0x557752143f60 2024-08-30T09:00:17.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.141+0000 7fd05f1bf740 1 -- 172.21.15.42:0/3207848453 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd04c021cd0 con 0x557752143f60 2024-08-30T09:00:17.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.141+0000 7fd039ffb640 1 -- 172.21.15.42:0/3207848453 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fd04c04e050 con 0x557752143f60 2024-08-30T09:00:17.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.141+0000 7fd05f1bf740 10 client.15357 fetch_fsmap learned FSMap version 26 2024-08-30T09:00:17.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.141+0000 7fd05f1bf740 1 -- 172.21.15.42:0/3207848453 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x557752571be0 con 0x557752143f60 2024-08-30T09:00:17.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.141+0000 7fd039ffb640 1 -- 172.21.15.42:0/3207848453 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd04c021cd0 con 0x557752143f60 2024-08-30T09:00:17.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.141+0000 7fd039ffb640 1 -- 172.21.15.42:0/3207848453 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fd04c028040 con 0x557752143f60 2024-08-30T09:00:17.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.141+0000 7fd05f1bf740 10 client.15357 fetch_fsmap finished waiting for FSMap version 26 2024-08-30T09:00:17.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.141+0000 7fd05f1bf740 10 client.15357 resolve_mds: resolved name 'h' to daemon mds.h 2024-08-30T09:00:17.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.141+0000 7fd05f1bf740 20 client.15357 populate_metadata read hostname 'smithi042' 2024-08-30T09:00:17.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.141+0000 7fd05f1bf740 1 --2- 172.21.15.42:0/3207848453 >> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] conn(0x5577520d7c90 0x557752574fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:17.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.141+0000 7fd05f1bf740 4 client.15357 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "status"}] 2024-08-30T09:00:17.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.141+0000 7fd05f1bf740 1 -- 172.21.15.42:0/3207848453 --> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] -- command(tid 0: {"prefix": "status"}) -- 0x55775245dde0 con 0x5577520d7c90 2024-08-30T09:00:17.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.142+0000 7fd05259c640 1 --2- 172.21.15.42:0/3207848453 >> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] conn(0x5577520d7c90 0x557752574fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:17.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.142+0000 7fd039ffb640 10 client.15357 ms_handle_connect on v2:172.21.15.167:6836/3011837856 2024-08-30T09:00:17.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.142+0000 7fd05259c640 1 --2- 172.21.15.42:0/3207848453 >> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] conn(0x5577520d7c90 0x557752574fc0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c0f8f7b2aebb4787 server_cookie=c529f320758d43a0 in_seq=0 out_seq=0 2024-08-30T09:00:17.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.142+0000 7fd039ffb640 1 -- 172.21.15.42:0/3207848453 <== mds.2 v2:172.21.15.167:6836/3011837856 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55775245dde0 con 0x5577520d7c90 2024-08-30T09:00:17.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.142+0000 7fd039ffb640 10 client.15357 handle_command_reply: tid=0 2024-08-30T09:00:17.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.142+0000 7fd05f1bf740 1 client.15357 shutdown 2024-08-30T09:00:17.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.142+0000 7fd05f1bf740 1 -- 172.21.15.42:0/3207848453 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557752143f60 msgr2=0x55775250e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:00:17.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.142+0000 7fd05f1bf740 1 --2- 172.21.15.42:0/3207848453 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557752143f60 0x55775250e000 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fd04c009e00 tx=0x7fd04c00a400 comp rx=0 tx=0).stop 2024-08-30T09:00:17.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.142+0000 7fd05f1bf740 1 -- 172.21.15.42:0/3207848453 shutdown_connections 2024-08-30T09:00:17.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.142+0000 7fd05f1bf740 1 --2- 172.21.15.42:0/3207848453 >> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] conn(0x5577520d7c90 0x557752574fc0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:17.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.142+0000 7fd05f1bf740 1 --2- 172.21.15.42:0/3207848453 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557752143f60 0x55775250e000 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:17.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.142+0000 7fd05f1bf740 1 --2- 172.21.15.42:0/3207848453 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557752123280 0x557752510850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:17.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.142+0000 7fd039ffb640 0 client.15357 ms_handle_reset on v2:172.21.15.167:6836/3011837856 2024-08-30T09:00:17.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.142+0000 7fd05f1bf740 1 --2- 172.21.15.42:0/3207848453 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55775211b0a0 0x5577525102f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:17.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.142+0000 7fd05f1bf740 1 -- 172.21.15.42:0/3207848453 >> 172.21.15.42:0/3207848453 conn(0x557752138350 msgr2=0x557752505100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:00:17.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.142+0000 7fd05f1bf740 1 -- 172.21.15.42:0/3207848453 shutdown_connections 2024-08-30T09:00:17.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.142+0000 7fd05f1bf740 1 -- 172.21.15.42:0/3207848453 wait complete. 2024-08-30T09:00:17.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.143+0000 7fd05f1bf740 20 client.15357 trim_cache size 0 max 16384 2024-08-30T09:00:17.150 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-08-30T09:00:17.150 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_fsid": "b0895c1a-66ac-11ef-bcd4-c7b262605968", 2024-08-30T09:00:17.150 INFO:teuthology.orchestra.run.smithi042.stdout: "whoami": 2, 2024-08-30T09:00:17.150 INFO:teuthology.orchestra.run.smithi042.stdout: "id": 24599, 2024-08-30T09:00:17.150 INFO:teuthology.orchestra.run.smithi042.stdout: "want_state": "up:active", 2024-08-30T09:00:17.150 INFO:teuthology.orchestra.run.smithi042.stdout: "state": "up:active", 2024-08-30T09:00:17.150 INFO:teuthology.orchestra.run.smithi042.stdout: "fs_name": "cephfs", 2024-08-30T09:00:17.150 INFO:teuthology.orchestra.run.smithi042.stdout: "rank_uptime": 91.579888838000002, 2024-08-30T09:00:17.150 INFO:teuthology.orchestra.run.smithi042.stdout: "mdsmap_epoch": 26, 2024-08-30T09:00:17.151 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_epoch": 83, 2024-08-30T09:00:17.151 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_epoch_barrier": 81, 2024-08-30T09:00:17.151 INFO:teuthology.orchestra.run.smithi042.stdout: "uptime": 156.48493414199999 2024-08-30T09:00:17.151 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-08-30T09:00:17.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.143+0000 7fd058c4a640 1 -- 172.21.15.42:0/3526207846 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fd03c0b4c40 msgr2=0x7fd03c0b7150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:00:17.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.143+0000 7fd058c4a640 1 --2- 172.21.15.42:0/3526207846 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fd03c0b4c40 0x7fd03c0b7150 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fd04c020000 tx=0x7fd04c021000 comp rx=0 tx=0).stop 2024-08-30T09:00:17.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.143+0000 7fd058c4a640 1 -- 172.21.15.42:0/3526207846 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd054104750 msgr2=0x7fd05419c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:00:17.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.143+0000 7fd058c4a640 1 --2- 172.21.15.42:0/3526207846 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd054104750 0x7fd05419c480 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fd04400d950 tx=0x7fd04400de10 comp rx=0 tx=0).stop 2024-08-30T09:00:17.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.143+0000 7fd058c4a640 1 -- 172.21.15.42:0/3526207846 shutdown_connections 2024-08-30T09:00:17.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.143+0000 7fd058c4a640 1 --2- 172.21.15.42:0/3526207846 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fd03c0b4c40 0x7fd03c0b7150 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:17.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.143+0000 7fd058c4a640 1 --2- 172.21.15.42:0/3526207846 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd05410dbb0 0x7fd0541a1cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:17.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.143+0000 7fd058c4a640 1 --2- 172.21.15.42:0/3526207846 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd0541050a0 0x7fd05419ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:17.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.143+0000 7fd058c4a640 1 --2- 172.21.15.42:0/3526207846 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd054104750 0x7fd05419c480 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:17.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.143+0000 7fd058c4a640 1 -- 172.21.15.42:0/3526207846 >> 172.21.15.42:0/3526207846 conn(0x7fd0540794d0 msgr2=0x7fd0541099a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:00:17.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.143+0000 7fd058c4a640 1 -- 172.21.15.42:0/3526207846 shutdown_connections 2024-08-30T09:00:17.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:17.143+0000 7fd058c4a640 1 -- 172.21.15.42:0/3526207846 wait complete. 2024-08-30T09:00:17.800 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph tell mds.i status 2024-08-30T09:00:18.050 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T09:00:18.460 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:00:18 smithi042 ceph-mon[18652]: pgmap v359: 129 pgs: 129 active+clean; 644 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-08-30T09:00:18.460 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:00:18 smithi042 ceph-mon[18652]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-08-30T09:00:18.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.549+0000 7fa91f138640 1 -- 172.21.15.42:0/1689226329 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa918102820 msgr2=0x7fa918102c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:00:18.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.549+0000 7fa91f138640 1 --2- 172.21.15.42:0/1689226329 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa918102820 0x7fa918102c30 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fa90c009900 tx=0x7fa90c02a410 comp rx=0 tx=0).stop 2024-08-30T09:00:18.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.549+0000 7fa91f138640 1 -- 172.21.15.42:0/1689226329 shutdown_connections 2024-08-30T09:00:18.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.549+0000 7fa91f138640 1 --2- 172.21.15.42:0/1689226329 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa918108160 0x7fa918108570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:18.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.549+0000 7fa91f138640 1 --2- 172.21.15.42:0/1689226329 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa918108ab0 0x7fa91810f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:18.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.549+0000 7fa91f138640 1 --2- 172.21.15.42:0/1689226329 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa918102820 0x7fa918102c30 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:18.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.549+0000 7fa91f138640 1 -- 172.21.15.42:0/1689226329 >> 172.21.15.42:0/1689226329 conn(0x7fa9180794e0 msgr2=0x7fa9180798f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:00:18.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.550+0000 7fa91f138640 1 -- 172.21.15.42:0/1689226329 shutdown_connections 2024-08-30T09:00:18.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.550+0000 7fa91f138640 1 -- 172.21.15.42:0/1689226329 wait complete. 2024-08-30T09:00:18.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.550+0000 7fa91f138640 1 Processor -- start 2024-08-30T09:00:18.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.550+0000 7fa91f138640 1 -- start start 2024-08-30T09:00:18.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.551+0000 7fa91f138640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa918102820 0x7fa91819c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:18.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.551+0000 7fa91f138640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa918108160 0x7fa91819ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:18.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.551+0000 7fa91f138640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa918108ab0 0x7fa9181a1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:18.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.551+0000 7fa91f138640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fa918113840 con 0x7fa918102820 2024-08-30T09:00:18.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.551+0000 7fa91f138640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa9181136c0 con 0x7fa918108ab0 2024-08-30T09:00:18.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.551+0000 7fa91f138640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fa9181139c0 con 0x7fa918108160 2024-08-30T09:00:18.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.551+0000 7fa91ced4640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa918102820 0x7fa91819c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:18.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.551+0000 7fa91ced4640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa918102820 0x7fa91819c4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38296/0 (socket says 172.21.15.42:38296) 2024-08-30T09:00:18.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.551+0000 7fa91ced4640 1 -- 172.21.15.42:0/537511652 learned_addr learned my addr 172.21.15.42:0/537511652 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:00:18.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.551+0000 7fa917fff640 1 --2- 172.21.15.42:0/537511652 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa918108160 0x7fa91819ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:18.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.551+0000 7fa91d6d5640 1 --2- 172.21.15.42:0/537511652 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa918108ab0 0x7fa9181a1cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:18.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.551+0000 7fa91ced4640 1 -- 172.21.15.42:0/537511652 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa918108160 msgr2=0x7fa91819ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:00:18.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.551+0000 7fa91ced4640 1 --2- 172.21.15.42:0/537511652 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa918108160 0x7fa91819ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:18.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.551+0000 7fa91ced4640 1 -- 172.21.15.42:0/537511652 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa918108ab0 msgr2=0x7fa9181a1cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:00:18.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.551+0000 7fa91ced4640 1 --2- 172.21.15.42:0/537511652 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa918108ab0 0x7fa9181a1cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:18.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.551+0000 7fa91ced4640 1 -- 172.21.15.42:0/537511652 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9181a2420 con 0x7fa918102820 2024-08-30T09:00:18.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.551+0000 7fa91ced4640 1 --2- 172.21.15.42:0/537511652 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa918102820 0x7fa91819c4b0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fa90c005940 tx=0x7fa90c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:00:18.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.552+0000 7fa915ffb640 1 -- 172.21.15.42:0/537511652 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa90c03f030 con 0x7fa918102820 2024-08-30T09:00:18.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.552+0000 7fa915ffb640 1 -- 172.21.15.42:0/537511652 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa90c003cb0 con 0x7fa918102820 2024-08-30T09:00:18.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.552+0000 7fa91f138640 1 -- 172.21.15.42:0/537511652 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9181a26b0 con 0x7fa918102820 2024-08-30T09:00:18.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.552+0000 7fa915ffb640 1 -- 172.21.15.42:0/537511652 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa90c035070 con 0x7fa918102820 2024-08-30T09:00:18.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.552+0000 7fa91f138640 1 -- 172.21.15.42:0/537511652 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9181a8e70 con 0x7fa918102820 2024-08-30T09:00:18.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.555+0000 7fa9256ad740 1 Processor -- start 2024-08-30T09:00:18.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.555+0000 7fa9256ad740 1 -- start start 2024-08-30T09:00:18.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.555+0000 7fa915ffb640 1 -- 172.21.15.42:0/537511652 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7fa90c042030 con 0x7fa918102820 2024-08-30T09:00:18.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.555+0000 7fa9256ad740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f0b2340f60 0x55f0b26e67b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:18.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.555+0000 7fa9256ad740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f0b23180a0 0x55f0b26e6cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:18.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.555+0000 7fa9256ad740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f0b2320280 0x55f0b26e7230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:18.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.555+0000 7fa9256ad740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55f0b265d3f0 con 0x55f0b2320280 2024-08-30T09:00:18.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.555+0000 7fa9256ad740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55f0b264c980 con 0x55f0b2340f60 2024-08-30T09:00:18.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.555+0000 7fa9256ad740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55f0b2650930 con 0x55f0b23180a0 2024-08-30T09:00:18.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.556+0000 7fa917fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f0b2320280 0x55f0b26e7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:18.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.556+0000 7fa917fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f0b2320280 0x55f0b26e7230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38306/0 (socket says 172.21.15.42:38306) 2024-08-30T09:00:18.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.556+0000 7fa917fff640 1 -- 172.21.15.42:0/3148141915 learned_addr learned my addr 172.21.15.42:0/3148141915 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:00:18.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.556+0000 7fa91ced4640 1 --2- 172.21.15.42:0/3148141915 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f0b23180a0 0x55f0b26e6cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:18.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.556+0000 7fa91d6d5640 1 --2- 172.21.15.42:0/3148141915 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f0b2340f60 0x55f0b26e67b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:18.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.556+0000 7fa917fff640 1 -- 172.21.15.42:0/3148141915 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f0b23180a0 msgr2=0x55f0b26e6cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:00:18.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.556+0000 7fa917fff640 1 --2- 172.21.15.42:0/3148141915 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f0b23180a0 0x55f0b26e6cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:18.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.556+0000 7fa917fff640 1 -- 172.21.15.42:0/3148141915 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f0b2340f60 msgr2=0x55f0b26e67b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:00:18.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.556+0000 7fa917fff640 1 --2- 172.21.15.42:0/3148141915 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f0b2340f60 0x55f0b26e67b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:18.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.556+0000 7fa917fff640 1 -- 172.21.15.42:0/3148141915 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f0b23ac540 con 0x55f0b2320280 2024-08-30T09:00:18.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.556+0000 7fa917fff640 1 --2- 172.21.15.42:0/3148141915 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f0b2320280 0x55f0b26e7230 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fa90400bc70 tx=0x7fa9040107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3792f63444b1715 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:00:18.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.556+0000 7fa915ffb640 1 --2- 172.21.15.42:0/537511652 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fa9080b4b30 0x7fa9080b7040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:18.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.556+0000 7fa91d6d5640 1 --2- 172.21.15.42:0/537511652 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fa9080b4b30 0x7fa9080b7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:18.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.556+0000 7fa902ffd640 1 -- 172.21.15.42:0/3148141915 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa904005220 con 0x55f0b2320280 2024-08-30T09:00:18.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.557+0000 7fa902ffd640 1 -- 172.21.15.42:0/3148141915 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa90400be20 con 0x55f0b2320280 2024-08-30T09:00:18.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.557+0000 7fa915ffb640 1 -- 172.21.15.42:0/537511652 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7fa90c105cf0 con 0x7fa918102820 2024-08-30T09:00:18.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.557+0000 7fa902ffd640 1 -- 172.21.15.42:0/3148141915 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa904011680 con 0x55f0b2320280 2024-08-30T09:00:18.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.557+0000 7fa9256ad740 1 -- 172.21.15.42:0/3148141915 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f0b2320280 msgr2=0x55f0b26e7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:00:18.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.557+0000 7fa91d6d5640 1 --2- 172.21.15.42:0/537511652 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fa9080b4b30 0x7fa9080b7040 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fa910005fd0 tx=0x7fa910009600 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:00:18.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.557+0000 7fa9256ad740 1 --2- 172.21.15.42:0/3148141915 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f0b2320280 0x55f0b26e7230 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fa90400bc70 tx=0x7fa9040107a0 comp rx=0 tx=0).stop 2024-08-30T09:00:18.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.557+0000 7fa9256ad740 1 -- 172.21.15.42:0/3148141915 shutdown_connections 2024-08-30T09:00:18.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.557+0000 7fa9256ad740 1 --2- 172.21.15.42:0/3148141915 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f0b23180a0 0x55f0b26e6cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:18.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.557+0000 7fa9256ad740 1 --2- 172.21.15.42:0/3148141915 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f0b2320280 0x55f0b26e7230 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:18.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.557+0000 7fa9256ad740 1 --2- 172.21.15.42:0/3148141915 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f0b2340f60 0x55f0b26e67b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:18.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.557+0000 7fa9256ad740 1 -- 172.21.15.42:0/3148141915 >> 172.21.15.42:0/3148141915 conn(0x55f0b2335350 msgr2=0x55f0b26e46a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:00:18.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.557+0000 7fa9256ad740 1 -- 172.21.15.42:0/3148141915 shutdown_connections 2024-08-30T09:00:18.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.558+0000 7fa9256ad740 1 -- 172.21.15.42:0/3148141915 wait complete. 2024-08-30T09:00:18.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.558+0000 7fa9256ad740 1 -- start start 2024-08-30T09:00:18.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.558+0000 7fa9256ad740 1 Processor -- start 2024-08-30T09:00:18.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.558+0000 7fa9256ad740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f0b23180a0 0x55f0b26f1e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:18.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.558+0000 7fa9256ad740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f0b2320280 0x55f0b26f23b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:18.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.558+0000 7fa9256ad740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f0b2340f60 0x55f0b26fe2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:18.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.558+0000 7fa9256ad740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55f0b24af5f0 con 0x55f0b2340f60 2024-08-30T09:00:18.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.558+0000 7fa9256ad740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55f0b25d3600 con 0x55f0b23180a0 2024-08-30T09:00:18.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.558+0000 7fa9256ad740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55f0b26f4f90 con 0x55f0b2320280 2024-08-30T09:00:18.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.559+0000 7fa9256ad740 10 client.0 _ll_register_callbacks cb 0x55f0b25f9af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:00:18.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.559+0000 7fa91d6d5640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f0b2340f60 0x55f0b26fe2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:18.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.559+0000 7fa91d6d5640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f0b2340f60 0x55f0b26fe2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38312/0 (socket says 172.21.15.42:38312) 2024-08-30T09:00:18.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.559+0000 7fa91d6d5640 1 -- 172.21.15.42:0/3624104934 learned_addr learned my addr 172.21.15.42:0/3624104934 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:00:18.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.559+0000 7fa917fff640 1 --2- 172.21.15.42:0/3624104934 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f0b2320280 0x55f0b26f23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:18.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.559+0000 7fa91ced4640 1 --2- 172.21.15.42:0/3624104934 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f0b23180a0 0x55f0b26f1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:18.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.559+0000 7fa91d6d5640 1 -- 172.21.15.42:0/3624104934 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f0b2320280 msgr2=0x55f0b26f23b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:00:18.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.559+0000 7fa91d6d5640 1 --2- 172.21.15.42:0/3624104934 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f0b2320280 0x55f0b26f23b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:18.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.559+0000 7fa91d6d5640 1 -- 172.21.15.42:0/3624104934 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f0b23180a0 msgr2=0x55f0b26f1e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:00:18.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.559+0000 7fa91d6d5640 1 --2- 172.21.15.42:0/3624104934 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f0b23180a0 0x55f0b26f1e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:18.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.559+0000 7fa91d6d5640 1 -- 172.21.15.42:0/3624104934 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f0b260c7b0 con 0x55f0b2340f60 2024-08-30T09:00:18.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.559+0000 7fa91d6d5640 1 --2- 172.21.15.42:0/3624104934 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f0b2340f60 0x55f0b26fe2b0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fa910011d40 tx=0x7fa910004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a01047fc198d7438 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:00:18.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.559+0000 7fa8f7fff640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-08-30T09:00:18.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.559+0000 7fa8f7fff640 1 -- 172.21.15.42:0/3624104934 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa910004d20 con 0x55f0b2340f60 2024-08-30T09:00:18.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.559+0000 7fa8f7fff640 1 -- 172.21.15.42:0/3624104934 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7fa910005020 con 0x55f0b2340f60 2024-08-30T09:00:18.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.559+0000 7fa9256ad740 1 -- 172.21.15.42:0/3624104934 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa90400be20 con 0x55f0b2340f60 2024-08-30T09:00:18.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.560+0000 7fa8f7fff640 1 -- 172.21.15.42:0/3624104934 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fa910005260 con 0x55f0b2340f60 2024-08-30T09:00:18.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.560+0000 7fa9256ad740 10 client.15369 fetch_fsmap learned FSMap version 26 2024-08-30T09:00:18.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.560+0000 7fa9256ad740 1 -- 172.21.15.42:0/3624104934 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f0b2700d30 con 0x55f0b2340f60 2024-08-30T09:00:18.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.560+0000 7fa8f7fff640 1 -- 172.21.15.42:0/3624104934 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa90400be20 con 0x55f0b2340f60 2024-08-30T09:00:18.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.560+0000 7fa8f7fff640 1 -- 172.21.15.42:0/3624104934 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fa91000f040 con 0x55f0b2340f60 2024-08-30T09:00:18.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.560+0000 7fa9256ad740 10 client.15369 fetch_fsmap finished waiting for FSMap version 26 2024-08-30T09:00:18.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.560+0000 7fa9256ad740 10 client.15369 resolve_mds: resolved name 'i' to daemon mds.i 2024-08-30T09:00:18.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.560+0000 7fa9256ad740 20 client.15369 populate_metadata read hostname 'smithi042' 2024-08-30T09:00:18.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.560+0000 7fa9256ad740 1 --2- 172.21.15.42:0/3624104934 >> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] conn(0x55f0b22ff8e0 0x55f0b2704820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:18.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.560+0000 7fa9256ad740 4 client.15369 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-30T09:00:18.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.560+0000 7fa9256ad740 1 -- 172.21.15.42:0/3624104934 --> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f0b265f340 con 0x55f0b22ff8e0 2024-08-30T09:00:18.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.560+0000 7fa91ced4640 1 --2- 172.21.15.42:0/3624104934 >> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] conn(0x55f0b22ff8e0 0x55f0b2704820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:18.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.561+0000 7fa91ced4640 1 --2- 172.21.15.42:0/3624104934 >> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] conn(0x55f0b22ff8e0 0x55f0b2704820 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=75d698095fd6905d server_cookie=a68091e9ebd7a39f in_seq=0 out_seq=0 2024-08-30T09:00:18.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.561+0000 7fa8f7fff640 10 client.15369 ms_handle_connect on v2:172.21.15.190:6838/719981110 2024-08-30T09:00:18.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.562+0000 7fa8f7fff640 1 -- 172.21.15.42:0/3624104934 <== mds.1 v2:172.21.15.190:6838/719981110 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f0b265f340 con 0x55f0b22ff8e0 2024-08-30T09:00:18.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.562+0000 7fa8f7fff640 10 client.15369 handle_command_reply: tid=0 2024-08-30T09:00:18.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.562+0000 7fa9256ad740 1 client.15369 shutdown 2024-08-30T09:00:18.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.562+0000 7fa9256ad740 1 -- 172.21.15.42:0/3624104934 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f0b2340f60 msgr2=0x55f0b26fe2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:00:18.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.562+0000 7fa9256ad740 1 --2- 172.21.15.42:0/3624104934 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f0b2340f60 0x55f0b26fe2b0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fa910011d40 tx=0x7fa910004a40 comp rx=0 tx=0).stop 2024-08-30T09:00:18.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.562+0000 7fa9256ad740 1 -- 172.21.15.42:0/3624104934 shutdown_connections 2024-08-30T09:00:18.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.562+0000 7fa9256ad740 1 --2- 172.21.15.42:0/3624104934 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f0b2320280 0x55f0b26f23b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:18.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.562+0000 7fa9256ad740 1 --2- 172.21.15.42:0/3624104934 >> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] conn(0x55f0b22ff8e0 0x55f0b2704820 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:18.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.562+0000 7fa9256ad740 1 --2- 172.21.15.42:0/3624104934 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f0b2340f60 0x55f0b26fe2b0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:18.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.562+0000 7fa9256ad740 1 --2- 172.21.15.42:0/3624104934 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f0b23180a0 0x55f0b26f1e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:18.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.562+0000 7fa9256ad740 1 -- 172.21.15.42:0/3624104934 >> 172.21.15.42:0/3624104934 conn(0x55f0b2335350 msgr2=0x55f0b26e9940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:00:18.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.562+0000 7fa8f7fff640 0 client.15369 ms_handle_reset on v2:172.21.15.190:6838/719981110 2024-08-30T09:00:18.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.562+0000 7fa9256ad740 1 -- 172.21.15.42:0/3624104934 shutdown_connections 2024-08-30T09:00:18.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.562+0000 7fa9256ad740 1 -- 172.21.15.42:0/3624104934 wait complete. 2024-08-30T09:00:18.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.562+0000 7fa9256ad740 20 client.15369 trim_cache size 0 max 16384 2024-08-30T09:00:18.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.580+0000 7fa9256ad740 1 Processor -- start 2024-08-30T09:00:18.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.580+0000 7fa9256ad740 1 -- start start 2024-08-30T09:00:18.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.581+0000 7fa9256ad740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f0b23180a0 0x55f0b27021f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:18.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.581+0000 7fa9256ad740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f0b2320280 0x55f0b26fe220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:18.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.581+0000 7fa9256ad740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f0b2340f60 0x55f0b26ed050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:18.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.581+0000 7fa9256ad740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55f0b2702b60 con 0x55f0b2340f60 2024-08-30T09:00:18.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.581+0000 7fa9256ad740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55f0b2703230 con 0x55f0b2320280 2024-08-30T09:00:18.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.581+0000 7fa9256ad740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55f0b2703900 con 0x55f0b23180a0 2024-08-30T09:00:18.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.581+0000 7fa91d6d5640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f0b2340f60 0x55f0b26ed050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:18.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.581+0000 7fa91d6d5640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f0b2340f60 0x55f0b26ed050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38328/0 (socket says 172.21.15.42:38328) 2024-08-30T09:00:18.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.581+0000 7fa91d6d5640 1 -- 172.21.15.42:0/3747799455 learned_addr learned my addr 172.21.15.42:0/3747799455 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:00:18.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.581+0000 7fa91ced4640 1 --2- 172.21.15.42:0/3747799455 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f0b23180a0 0x55f0b27021f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:18.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.581+0000 7fa917fff640 1 --2- 172.21.15.42:0/3747799455 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f0b2320280 0x55f0b26fe220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:18.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.581+0000 7fa91d6d5640 1 -- 172.21.15.42:0/3747799455 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f0b23180a0 msgr2=0x55f0b27021f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:00:18.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.581+0000 7fa91d6d5640 1 --2- 172.21.15.42:0/3747799455 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f0b23180a0 0x55f0b27021f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:18.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.581+0000 7fa91d6d5640 1 -- 172.21.15.42:0/3747799455 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f0b2320280 msgr2=0x55f0b26fe220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:00:18.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.581+0000 7fa91d6d5640 1 --2- 172.21.15.42:0/3747799455 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f0b2320280 0x55f0b26fe220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:18.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.581+0000 7fa91d6d5640 1 -- 172.21.15.42:0/3747799455 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f0b26e8aa0 con 0x55f0b2340f60 2024-08-30T09:00:18.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.581+0000 7fa91d6d5640 1 --2- 172.21.15.42:0/3747799455 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f0b2340f60 0x55f0b26ed050 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fa910013dd0 tx=0x7fa910005cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d146217961764e4 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:00:18.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.582+0000 7fa9027fc640 1 -- 172.21.15.42:0/3747799455 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa91003b070 con 0x55f0b2340f60 2024-08-30T09:00:18.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.582+0000 7fa9027fc640 1 -- 172.21.15.42:0/3747799455 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa910020ce0 con 0x55f0b2340f60 2024-08-30T09:00:18.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.582+0000 7fa9027fc640 1 -- 172.21.15.42:0/3747799455 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa910004ff0 con 0x55f0b2340f60 2024-08-30T09:00:18.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.582+0000 7fa9256ad740 1 -- 172.21.15.42:0/3747799455 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f0b2340f60 msgr2=0x55f0b26ed050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:00:18.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.582+0000 7fa9256ad740 1 --2- 172.21.15.42:0/3747799455 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f0b2340f60 0x55f0b26ed050 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fa910013dd0 tx=0x7fa910005cd0 comp rx=0 tx=0).stop 2024-08-30T09:00:18.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.582+0000 7fa9256ad740 1 -- 172.21.15.42:0/3747799455 shutdown_connections 2024-08-30T09:00:18.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.582+0000 7fa9256ad740 1 --2- 172.21.15.42:0/3747799455 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f0b2340f60 0x55f0b26ed050 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:18.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.582+0000 7fa9256ad740 1 --2- 172.21.15.42:0/3747799455 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f0b2320280 0x55f0b26fe220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:18.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.582+0000 7fa9256ad740 1 --2- 172.21.15.42:0/3747799455 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f0b23180a0 0x55f0b27021f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:18.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.582+0000 7fa9256ad740 1 -- 172.21.15.42:0/3747799455 >> 172.21.15.42:0/3747799455 conn(0x55f0b2335350 msgr2=0x55f0b26df400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:00:18.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.583+0000 7fa9256ad740 1 -- 172.21.15.42:0/3747799455 shutdown_connections 2024-08-30T09:00:18.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.583+0000 7fa9256ad740 1 -- 172.21.15.42:0/3747799455 wait complete. 2024-08-30T09:00:18.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.583+0000 7fa9256ad740 1 -- start start 2024-08-30T09:00:18.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.583+0000 7fa9256ad740 1 Processor -- start 2024-08-30T09:00:18.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.584+0000 7fa9256ad740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f0b23180a0 0x55f0b2705e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:18.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.584+0000 7fa9256ad740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f0b2320280 0x55f0b27063d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:18.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.584+0000 7fa9256ad740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f0b2340f60 0x55f0b270b410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:18.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.584+0000 7fa9256ad740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55f0b27046a0 con 0x55f0b2340f60 2024-08-30T09:00:18.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.584+0000 7fa9256ad740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55f0b2703fd0 con 0x55f0b2320280 2024-08-30T09:00:18.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.584+0000 7fa9256ad740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55f0b2704d70 con 0x55f0b23180a0 2024-08-30T09:00:18.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.584+0000 7fa9256ad740 10 client.0 _ll_register_callbacks cb 0x7fa9100119f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:00:18.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.584+0000 7fa91d6d5640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f0b2340f60 0x55f0b270b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:18.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.584+0000 7fa91d6d5640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f0b2340f60 0x55f0b270b410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38332/0 (socket says 172.21.15.42:38332) 2024-08-30T09:00:18.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.584+0000 7fa91d6d5640 1 -- 172.21.15.42:0/3115631490 learned_addr learned my addr 172.21.15.42:0/3115631490 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:00:18.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.584+0000 7fa917fff640 1 --2- 172.21.15.42:0/3115631490 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f0b2320280 0x55f0b27063d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:18.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.584+0000 7fa91ced4640 1 --2- 172.21.15.42:0/3115631490 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f0b23180a0 0x55f0b2705e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:18.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.584+0000 7fa91d6d5640 1 -- 172.21.15.42:0/3115631490 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f0b23180a0 msgr2=0x55f0b2705e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:00:18.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.584+0000 7fa91d6d5640 1 --2- 172.21.15.42:0/3115631490 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f0b23180a0 0x55f0b2705e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:18.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.584+0000 7fa91d6d5640 1 -- 172.21.15.42:0/3115631490 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f0b2320280 msgr2=0x55f0b27063d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:00:18.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.584+0000 7fa91d6d5640 1 --2- 172.21.15.42:0/3115631490 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f0b2320280 0x55f0b27063d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:18.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.584+0000 7fa91d6d5640 1 -- 172.21.15.42:0/3115631490 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f0b270ba80 con 0x55f0b2340f60 2024-08-30T09:00:18.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.584+0000 7fa91d6d5640 1 --2- 172.21.15.42:0/3115631490 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f0b2340f60 0x55f0b270b410 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fa910009e40 tx=0x7fa910004d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a18032ee492135a5 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:00:18.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.584+0000 7fa8f7fff640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-08-30T09:00:18.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.584+0000 7fa8f7fff640 1 -- 172.21.15.42:0/3115631490 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa91003b040 con 0x55f0b2340f60 2024-08-30T09:00:18.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.584+0000 7fa8f7fff640 1 -- 172.21.15.42:0/3115631490 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7fa910003970 con 0x55f0b2340f60 2024-08-30T09:00:18.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.584+0000 7fa9256ad740 1 -- 172.21.15.42:0/3115631490 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa910020ce0 con 0x55f0b2340f60 2024-08-30T09:00:18.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.585+0000 7fa8f7fff640 1 -- 172.21.15.42:0/3115631490 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fa910003b70 con 0x55f0b2340f60 2024-08-30T09:00:18.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.585+0000 7fa9256ad740 10 client.15375 fetch_fsmap learned FSMap version 26 2024-08-30T09:00:18.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.585+0000 7fa9256ad740 1 -- 172.21.15.42:0/3115631490 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f0b270ce40 con 0x55f0b2340f60 2024-08-30T09:00:18.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.585+0000 7fa8f7fff640 1 -- 172.21.15.42:0/3115631490 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa910020ce0 con 0x55f0b2340f60 2024-08-30T09:00:18.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.585+0000 7fa8f7fff640 1 -- 172.21.15.42:0/3115631490 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fa91000f040 con 0x55f0b2340f60 2024-08-30T09:00:18.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.585+0000 7fa9256ad740 10 client.15375 fetch_fsmap finished waiting for FSMap version 26 2024-08-30T09:00:18.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.585+0000 7fa9256ad740 10 client.15375 resolve_mds: resolved name 'i' to daemon mds.i 2024-08-30T09:00:18.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.585+0000 7fa9256ad740 20 client.15375 populate_metadata read hostname 'smithi042' 2024-08-30T09:00:18.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.585+0000 7fa9256ad740 1 --2- 172.21.15.42:0/3115631490 >> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] conn(0x55f0b22d4c90 0x55f0b2772770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:18.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.585+0000 7fa9256ad740 4 client.15375 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "status"}] 2024-08-30T09:00:18.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.585+0000 7fa9256ad740 1 -- 172.21.15.42:0/3115631490 --> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] -- command(tid 0: {"prefix": "status"}) -- 0x55f0b265ade0 con 0x55f0b22d4c90 2024-08-30T09:00:18.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.585+0000 7fa91ced4640 1 --2- 172.21.15.42:0/3115631490 >> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] conn(0x55f0b22d4c90 0x55f0b2772770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:18.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.586+0000 7fa91ced4640 1 --2- 172.21.15.42:0/3115631490 >> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] conn(0x55f0b22d4c90 0x55f0b2772770 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=de536371700147d9 server_cookie=53f8f4d6ab41e63c in_seq=0 out_seq=0 2024-08-30T09:00:18.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.586+0000 7fa8f7fff640 10 client.15375 ms_handle_connect on v2:172.21.15.190:6838/719981110 2024-08-30T09:00:18.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.586+0000 7fa8f7fff640 1 -- 172.21.15.42:0/3115631490 <== mds.1 v2:172.21.15.190:6838/719981110 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55f0b265ade0 con 0x55f0b22d4c90 2024-08-30T09:00:18.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.586+0000 7fa8f7fff640 10 client.15375 handle_command_reply: tid=0 2024-08-30T09:00:18.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.586+0000 7fa9256ad740 1 client.15375 shutdown 2024-08-30T09:00:18.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.586+0000 7fa9256ad740 1 -- 172.21.15.42:0/3115631490 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f0b2340f60 msgr2=0x55f0b270b410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:00:18.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.586+0000 7fa9256ad740 1 --2- 172.21.15.42:0/3115631490 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f0b2340f60 0x55f0b270b410 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fa910009e40 tx=0x7fa910004d30 comp rx=0 tx=0).stop 2024-08-30T09:00:18.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.586+0000 7fa9256ad740 1 -- 172.21.15.42:0/3115631490 shutdown_connections 2024-08-30T09:00:18.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.586+0000 7fa9256ad740 1 --2- 172.21.15.42:0/3115631490 >> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] conn(0x55f0b22d4c90 0x55f0b2772770 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:18.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.586+0000 7fa9256ad740 1 --2- 172.21.15.42:0/3115631490 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f0b2340f60 0x55f0b270b410 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:18.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.586+0000 7fa9256ad740 1 --2- 172.21.15.42:0/3115631490 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f0b2320280 0x55f0b27063d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:18.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.586+0000 7fa9256ad740 1 --2- 172.21.15.42:0/3115631490 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f0b23180a0 0x55f0b2705e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:18.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.586+0000 7fa8f7fff640 0 client.15375 ms_handle_reset on v2:172.21.15.190:6838/719981110 2024-08-30T09:00:18.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.586+0000 7fa9256ad740 1 -- 172.21.15.42:0/3115631490 >> 172.21.15.42:0/3115631490 conn(0x55f0b2335350 msgr2=0x55f0b26df400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:00:18.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.587+0000 7fa9256ad740 1 -- 172.21.15.42:0/3115631490 shutdown_connections 2024-08-30T09:00:18.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.587+0000 7fa9256ad740 1 -- 172.21.15.42:0/3115631490 wait complete. 2024-08-30T09:00:18.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.587+0000 7fa9256ad740 20 client.15375 trim_cache size 0 max 16384 2024-08-30T09:00:18.594 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-08-30T09:00:18.594 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_fsid": "b0895c1a-66ac-11ef-bcd4-c7b262605968", 2024-08-30T09:00:18.594 INFO:teuthology.orchestra.run.smithi042.stdout: "whoami": 1, 2024-08-30T09:00:18.595 INFO:teuthology.orchestra.run.smithi042.stdout: "id": 24610, 2024-08-30T09:00:18.595 INFO:teuthology.orchestra.run.smithi042.stdout: "want_state": "up:active", 2024-08-30T09:00:18.595 INFO:teuthology.orchestra.run.smithi042.stdout: "state": "up:active", 2024-08-30T09:00:18.595 INFO:teuthology.orchestra.run.smithi042.stdout: "fs_name": "cephfs", 2024-08-30T09:00:18.595 INFO:teuthology.orchestra.run.smithi042.stdout: "rank_uptime": 94.042655281999998, 2024-08-30T09:00:18.595 INFO:teuthology.orchestra.run.smithi042.stdout: "mdsmap_epoch": 26, 2024-08-30T09:00:18.595 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_epoch": 83, 2024-08-30T09:00:18.595 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_epoch_barrier": 81, 2024-08-30T09:00:18.595 INFO:teuthology.orchestra.run.smithi042.stdout: "uptime": 155.28045677899999 2024-08-30T09:00:18.596 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-08-30T09:00:18.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.587+0000 7fa91f138640 1 -- 172.21.15.42:0/537511652 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fa9080b4b30 msgr2=0x7fa9080b7040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:00:18.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.587+0000 7fa91f138640 1 --2- 172.21.15.42:0/537511652 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fa9080b4b30 0x7fa9080b7040 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fa910005fd0 tx=0x7fa910009600 comp rx=0 tx=0).stop 2024-08-30T09:00:18.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.587+0000 7fa91f138640 1 -- 172.21.15.42:0/537511652 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa918102820 msgr2=0x7fa91819c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:00:18.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.587+0000 7fa91f138640 1 --2- 172.21.15.42:0/537511652 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa918102820 0x7fa91819c4b0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fa90c005940 tx=0x7fa90c0042f0 comp rx=0 tx=0).stop 2024-08-30T09:00:18.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.587+0000 7fa91f138640 1 -- 172.21.15.42:0/537511652 shutdown_connections 2024-08-30T09:00:18.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.587+0000 7fa91f138640 1 --2- 172.21.15.42:0/537511652 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fa9080b4b30 0x7fa9080b7040 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:18.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.587+0000 7fa91f138640 1 --2- 172.21.15.42:0/537511652 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa918108160 0x7fa91819ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:18.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.587+0000 7fa91f138640 1 --2- 172.21.15.42:0/537511652 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa918108ab0 0x7fa9181a1cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:18.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.587+0000 7fa91f138640 1 --2- 172.21.15.42:0/537511652 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa918102820 0x7fa91819c4b0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:18.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.587+0000 7fa91f138640 1 -- 172.21.15.42:0/537511652 >> 172.21.15.42:0/537511652 conn(0x7fa9180794e0 msgr2=0x7fa91810e620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:00:18.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.588+0000 7fa91f138640 1 -- 172.21.15.42:0/537511652 shutdown_connections 2024-08-30T09:00:18.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:18.588+0000 7fa91f138640 1 -- 172.21.15.42:0/537511652 wait complete. 2024-08-30T09:00:18.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:00:18 smithi190 ceph-mon[22001]: pgmap v359: 129 pgs: 129 active+clean; 644 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-08-30T09:00:18.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:00:18 smithi190 ceph-mon[22001]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-08-30T09:00:18.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:00:18 smithi167 ceph-mon[22063]: pgmap v359: 129 pgs: 129 active+clean; 644 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-08-30T09:00:18.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:00:18 smithi167 ceph-mon[22063]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-08-30T09:00:19.302 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph tell mds.b status 2024-08-30T09:00:19.553 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T09:00:20.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.088+0000 7f6da93b5640 1 -- 172.21.15.42:0/3430160523 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6da4102830 msgr2=0x7f6da4102c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:00:20.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.088+0000 7f6da93b5640 1 --2- 172.21.15.42:0/3430160523 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6da4102830 0x7f6da4102c40 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f6d94009900 tx=0x7f6d9402a410 comp rx=0 tx=0).stop 2024-08-30T09:00:20.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.088+0000 7f6da93b5640 1 -- 172.21.15.42:0/3430160523 shutdown_connections 2024-08-30T09:00:20.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.088+0000 7f6da93b5640 1 --2- 172.21.15.42:0/3430160523 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6da4108170 0x7f6da4108580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:20.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.088+0000 7f6da93b5640 1 --2- 172.21.15.42:0/3430160523 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6da4108ac0 0x7f6da410f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:20.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.088+0000 7f6da93b5640 1 --2- 172.21.15.42:0/3430160523 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6da4102830 0x7f6da4102c40 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:20.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.088+0000 7f6da93b5640 1 -- 172.21.15.42:0/3430160523 >> 172.21.15.42:0/3430160523 conn(0x7f6da40794f0 msgr2=0x7f6da4079900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:00:20.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.088+0000 7f6da93b5640 1 -- 172.21.15.42:0/3430160523 shutdown_connections 2024-08-30T09:00:20.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.088+0000 7f6da93b5640 1 -- 172.21.15.42:0/3430160523 wait complete. 2024-08-30T09:00:20.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.089+0000 7f6da93b5640 1 Processor -- start 2024-08-30T09:00:20.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.089+0000 7f6da93b5640 1 -- start start 2024-08-30T09:00:20.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.089+0000 7f6da93b5640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6da4102830 0x7f6da419aff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:20.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.089+0000 7f6da93b5640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6da4108170 0x7f6da419b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:20.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.089+0000 7f6da93b5640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6da4108ac0 0x7f6da419ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:20.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.089+0000 7f6da93b5640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f6da4112740 con 0x7f6da4108ac0 2024-08-30T09:00:20.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.089+0000 7f6da93b5640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f6da41125c0 con 0x7f6da4108170 2024-08-30T09:00:20.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.089+0000 7f6da93b5640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f6da41128c0 con 0x7f6da4102830 2024-08-30T09:00:20.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.089+0000 7f6da37fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6da4108ac0 0x7f6da419ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:20.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.089+0000 7f6da37fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6da4108ac0 0x7f6da419ba70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38358/0 (socket says 172.21.15.42:38358) 2024-08-30T09:00:20.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.089+0000 7f6da37fe640 1 -- 172.21.15.42:0/2351139867 learned_addr learned my addr 172.21.15.42:0/2351139867 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:00:20.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.089+0000 7f6da27fc640 1 --2- 172.21.15.42:0/2351139867 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6da4108170 0x7f6da419b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:20.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.089+0000 7f6da2ffd640 1 --2- 172.21.15.42:0/2351139867 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6da4102830 0x7f6da419aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:20.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.090+0000 7f6da37fe640 1 -- 172.21.15.42:0/2351139867 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6da4102830 msgr2=0x7f6da419aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:00:20.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.090+0000 7f6da37fe640 1 --2- 172.21.15.42:0/2351139867 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6da4102830 0x7f6da419aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:20.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.090+0000 7f6da37fe640 1 -- 172.21.15.42:0/2351139867 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6da4108170 msgr2=0x7f6da419b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:00:20.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.090+0000 7f6da37fe640 1 --2- 172.21.15.42:0/2351139867 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6da4108170 0x7f6da419b530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:20.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.090+0000 7f6da37fe640 1 -- 172.21.15.42:0/2351139867 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6da41a62b0 con 0x7f6da4108ac0 2024-08-30T09:00:20.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.090+0000 7f6da37fe640 1 --2- 172.21.15.42:0/2351139867 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6da4108ac0 0x7f6da419ba70 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f6d9c00eaa0 tx=0x7f6d9c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:00:20.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.090+0000 7f6d8bfff640 1 -- 172.21.15.42:0/2351139867 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d9c00cdb0 con 0x7f6da4108ac0 2024-08-30T09:00:20.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.090+0000 7f6d8bfff640 1 -- 172.21.15.42:0/2351139867 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d9c005020 con 0x7f6da4108ac0 2024-08-30T09:00:20.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.090+0000 7f6da93b5640 1 -- 172.21.15.42:0/2351139867 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6da41a65a0 con 0x7f6da4108ac0 2024-08-30T09:00:20.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.090+0000 7f6d8bfff640 1 -- 172.21.15.42:0/2351139867 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d9c0116d0 con 0x7f6da4108ac0 2024-08-30T09:00:20.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.090+0000 7f6da93b5640 1 -- 172.21.15.42:0/2351139867 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6da41a6a90 con 0x7f6da4108ac0 2024-08-30T09:00:20.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.092+0000 7f6daf92a740 1 Processor -- start 2024-08-30T09:00:20.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.092+0000 7f6d8bfff640 1 -- 172.21.15.42:0/2351139867 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7f6d9c011870 con 0x7f6da4108ac0 2024-08-30T09:00:20.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.093+0000 7f6daf92a740 1 -- start start 2024-08-30T09:00:20.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.093+0000 7f6daf92a740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dd61470f60 0x55dd618167b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:20.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.093+0000 7f6daf92a740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dd614480a0 0x55dd61816cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:20.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.093+0000 7f6daf92a740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dd61450280 0x55dd61817230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:20.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.093+0000 7f6daf92a740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55dd6178d3f0 con 0x55dd61450280 2024-08-30T09:00:20.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.093+0000 7f6daf92a740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55dd6177c980 con 0x55dd614480a0 2024-08-30T09:00:20.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.093+0000 7f6daf92a740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55dd61780930 con 0x55dd61470f60 2024-08-30T09:00:20.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.093+0000 7f6da27fc640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dd61450280 0x55dd61817230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:20.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.093+0000 7f6da27fc640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dd61450280 0x55dd61817230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38362/0 (socket says 172.21.15.42:38362) 2024-08-30T09:00:20.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.093+0000 7f6da27fc640 1 -- 172.21.15.42:0/1303369898 learned_addr learned my addr 172.21.15.42:0/1303369898 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:00:20.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.093+0000 7f6da27fc640 1 --2- 172.21.15.42:0/1303369898 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dd61470f60 0x55dd618167b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:20.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.093+0000 7f6da2ffd640 1 --2- 172.21.15.42:0/1303369898 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dd614480a0 0x55dd61816cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:20.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.093+0000 7f6d8bfff640 1 --2- 172.21.15.42:0/2351139867 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f6d8c0b4b30 0x7f6d8c0b7040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:20.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.093+0000 7f6da27fc640 1 -- 172.21.15.42:0/1303369898 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dd61470f60 msgr2=0x55dd618167b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:00:20.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.093+0000 7f6da27fc640 1 --2- 172.21.15.42:0/1303369898 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dd61470f60 0x55dd618167b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:20.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.093+0000 7f6da27fc640 1 -- 172.21.15.42:0/1303369898 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dd614480a0 msgr2=0x55dd61816cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:00:20.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.093+0000 7f6da27fc640 1 --2- 172.21.15.42:0/1303369898 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dd614480a0 0x55dd61816cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:20.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.093+0000 7f6da27fc640 1 -- 172.21.15.42:0/1303369898 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dd614dc540 con 0x55dd61450280 2024-08-30T09:00:20.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.093+0000 7f6da2ffd640 1 --2- 172.21.15.42:0/1303369898 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dd614480a0 0x55dd61816cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-30T09:00:20.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.094+0000 7f6da37fe640 1 --2- 172.21.15.42:0/2351139867 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f6d8c0b4b30 0x7f6d8c0b7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:20.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.094+0000 7f6da27fc640 1 --2- 172.21.15.42:0/1303369898 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dd61450280 0x55dd61817230 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f6d98005880 tx=0x7f6d98004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4447e06c049f6fdb server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:00:20.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.094+0000 7f6d8bfff640 1 -- 172.21.15.42:0/2351139867 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f6d9c0e5670 con 0x7f6da4108ac0 2024-08-30T09:00:20.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.094+0000 7f6d897fa640 1 -- 172.21.15.42:0/1303369898 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d98012ce0 con 0x55dd61450280 2024-08-30T09:00:20.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.094+0000 7f6d897fa640 1 -- 172.21.15.42:0/1303369898 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d98005020 con 0x55dd61450280 2024-08-30T09:00:20.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.094+0000 7f6d897fa640 1 -- 172.21.15.42:0/1303369898 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d980044e0 con 0x55dd61450280 2024-08-30T09:00:20.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.095+0000 7f6daf92a740 1 -- 172.21.15.42:0/1303369898 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dd61450280 msgr2=0x55dd61817230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:00:20.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.095+0000 7f6daf92a740 1 --2- 172.21.15.42:0/1303369898 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dd61450280 0x55dd61817230 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f6d98005880 tx=0x7f6d98004730 comp rx=0 tx=0).stop 2024-08-30T09:00:20.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.095+0000 7f6da37fe640 1 --2- 172.21.15.42:0/2351139867 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f6d8c0b4b30 0x7f6d8c0b7040 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f6d9c0a7e40 tx=0x7f6d9c0792b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:00:20.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.095+0000 7f6daf92a740 1 -- 172.21.15.42:0/1303369898 shutdown_connections 2024-08-30T09:00:20.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.095+0000 7f6daf92a740 1 --2- 172.21.15.42:0/1303369898 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dd61450280 0x55dd61817230 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:20.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.095+0000 7f6daf92a740 1 --2- 172.21.15.42:0/1303369898 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dd614480a0 0x55dd61816cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:20.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.095+0000 7f6daf92a740 1 --2- 172.21.15.42:0/1303369898 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dd61470f60 0x55dd618167b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:20.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.095+0000 7f6daf92a740 1 -- 172.21.15.42:0/1303369898 >> 172.21.15.42:0/1303369898 conn(0x55dd61465350 msgr2=0x55dd618146a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:00:20.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.095+0000 7f6daf92a740 1 -- 172.21.15.42:0/1303369898 shutdown_connections 2024-08-30T09:00:20.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.095+0000 7f6daf92a740 1 -- 172.21.15.42:0/1303369898 wait complete. 2024-08-30T09:00:20.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.095+0000 7f6daf92a740 1 -- start start 2024-08-30T09:00:20.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.095+0000 7f6daf92a740 1 Processor -- start 2024-08-30T09:00:20.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.096+0000 7f6daf92a740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dd614480a0 0x55dd61821eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:20.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.096+0000 7f6daf92a740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dd61450280 0x55dd618223f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:20.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.096+0000 7f6daf92a740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dd61470f60 0x55dd6182e280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:20.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.096+0000 7f6daf92a740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55dd615df5f0 con 0x55dd61450280 2024-08-30T09:00:20.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.096+0000 7f6daf92a740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55dd61703600 con 0x55dd614480a0 2024-08-30T09:00:20.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.096+0000 7f6daf92a740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55dd61824f90 con 0x55dd61470f60 2024-08-30T09:00:20.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.096+0000 7f6daf92a740 10 client.0 _ll_register_callbacks cb 0x55dd61729af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:00:20.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.096+0000 7f6da2ffd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dd61450280 0x55dd618223f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:20.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.096+0000 7f6da2ffd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dd61450280 0x55dd618223f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38372/0 (socket says 172.21.15.42:38372) 2024-08-30T09:00:20.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.096+0000 7f6da2ffd640 1 -- 172.21.15.42:0/256480749 learned_addr learned my addr 172.21.15.42:0/256480749 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:00:20.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.096+0000 7f6da27fc640 1 --2- 172.21.15.42:0/256480749 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dd61470f60 0x55dd6182e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:20.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.096+0000 7f6da27fc640 1 --2- 172.21.15.42:0/256480749 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dd614480a0 0x55dd61821eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:20.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.096+0000 7f6da2ffd640 1 -- 172.21.15.42:0/256480749 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dd61470f60 msgr2=0x55dd6182e280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:00:20.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.096+0000 7f6da2ffd640 1 --2- 172.21.15.42:0/256480749 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dd61470f60 0x55dd6182e280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:20.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.096+0000 7f6da2ffd640 1 -- 172.21.15.42:0/256480749 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dd614480a0 msgr2=0x55dd61821eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:00:20.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.096+0000 7f6da2ffd640 1 --2- 172.21.15.42:0/256480749 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dd614480a0 0x55dd61821eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:20.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.096+0000 7f6da2ffd640 1 -- 172.21.15.42:0/256480749 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dd6173c7b0 con 0x55dd61450280 2024-08-30T09:00:20.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.096+0000 7f6da27fc640 1 --2- 172.21.15.42:0/256480749 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dd614480a0 0x55dd61821eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-30T09:00:20.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.096+0000 7f6da2ffd640 1 --2- 172.21.15.42:0/256480749 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dd61450280 0x55dd618223f0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f6d9402a930 tx=0x7f6d94032f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb3672f27a93256a server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:00:20.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.096+0000 7f6d86ff5640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-08-30T09:00:20.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.096+0000 7f6d86ff5640 1 -- 172.21.15.42:0/256480749 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d94040070 con 0x55dd61450280 2024-08-30T09:00:20.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.097+0000 7f6d86ff5640 1 -- 172.21.15.42:0/256480749 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f6d94004040 con 0x55dd61450280 2024-08-30T09:00:20.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.097+0000 7f6daf92a740 1 -- 172.21.15.42:0/256480749 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6d98005020 con 0x55dd61450280 2024-08-30T09:00:20.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.097+0000 7f6d86ff5640 1 -- 172.21.15.42:0/256480749 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f6d9404d050 con 0x55dd61450280 2024-08-30T09:00:20.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.097+0000 7f6daf92a740 10 client.15387 fetch_fsmap learned FSMap version 26 2024-08-30T09:00:20.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.097+0000 7f6daf92a740 1 -- 172.21.15.42:0/256480749 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55dd61830c70 con 0x55dd61450280 2024-08-30T09:00:20.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.097+0000 7f6d86ff5640 1 -- 172.21.15.42:0/256480749 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d98005020 con 0x55dd61450280 2024-08-30T09:00:20.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.097+0000 7f6d86ff5640 1 -- 172.21.15.42:0/256480749 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f6d940048e0 con 0x55dd61450280 2024-08-30T09:00:20.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.097+0000 7f6daf92a740 10 client.15387 fetch_fsmap finished waiting for FSMap version 26 2024-08-30T09:00:20.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.097+0000 7f6daf92a740 10 client.15387 resolve_mds: resolved name 'b' to daemon mds.b 2024-08-30T09:00:20.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.097+0000 7f6daf92a740 20 client.15387 populate_metadata read hostname 'smithi042' 2024-08-30T09:00:20.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.097+0000 7f6daf92a740 1 --2- 172.21.15.42:0/256480749 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55dd6142f8e0 0x55dd618346c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:20.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.097+0000 7f6daf92a740 4 client.15387 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-30T09:00:20.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.097+0000 7f6daf92a740 1 -- 172.21.15.42:0/256480749 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dd6178f340 con 0x55dd6142f8e0 2024-08-30T09:00:20.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.098+0000 7f6da27fc640 1 --2- 172.21.15.42:0/256480749 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55dd6142f8e0 0x55dd618346c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:20.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.098+0000 7f6da27fc640 1 --2- 172.21.15.42:0/256480749 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55dd6142f8e0 0x55dd618346c0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4df108855f0a1da3 server_cookie=151ad225d34a8849 in_seq=0 out_seq=0 2024-08-30T09:00:20.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.098+0000 7f6d86ff5640 10 client.15387 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:00:20.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.099+0000 7f6d86ff5640 1 -- 172.21.15.42:0/256480749 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55dd6178f340 con 0x55dd6142f8e0 2024-08-30T09:00:20.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.099+0000 7f6d86ff5640 10 client.15387 handle_command_reply: tid=0 2024-08-30T09:00:20.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.099+0000 7f6daf92a740 1 client.15387 shutdown 2024-08-30T09:00:20.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.099+0000 7f6daf92a740 1 -- 172.21.15.42:0/256480749 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dd61450280 msgr2=0x55dd618223f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:00:20.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.099+0000 7f6daf92a740 1 --2- 172.21.15.42:0/256480749 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dd61450280 0x55dd618223f0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f6d9402a930 tx=0x7f6d94032f20 comp rx=0 tx=0).stop 2024-08-30T09:00:20.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.099+0000 7f6daf92a740 1 -- 172.21.15.42:0/256480749 shutdown_connections 2024-08-30T09:00:20.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.099+0000 7f6daf92a740 1 --2- 172.21.15.42:0/256480749 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55dd6142f8e0 0x55dd618346c0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:20.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.099+0000 7f6daf92a740 1 --2- 172.21.15.42:0/256480749 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dd61470f60 0x55dd6182e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:20.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.099+0000 7f6daf92a740 1 --2- 172.21.15.42:0/256480749 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dd61450280 0x55dd618223f0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:20.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.099+0000 7f6d86ff5640 0 client.15387 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:00:20.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.099+0000 7f6daf92a740 1 --2- 172.21.15.42:0/256480749 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dd614480a0 0x55dd61821eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:20.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.099+0000 7f6daf92a740 1 -- 172.21.15.42:0/256480749 >> 172.21.15.42:0/256480749 conn(0x55dd61465350 msgr2=0x55dd61819940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:00:20.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.099+0000 7f6daf92a740 1 -- 172.21.15.42:0/256480749 shutdown_connections 2024-08-30T09:00:20.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.099+0000 7f6daf92a740 1 -- 172.21.15.42:0/256480749 wait complete. 2024-08-30T09:00:20.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.099+0000 7f6daf92a740 20 client.15387 trim_cache size 0 max 16384 2024-08-30T09:00:20.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.117+0000 7f6daf92a740 1 Processor -- start 2024-08-30T09:00:20.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.117+0000 7f6daf92a740 1 -- start start 2024-08-30T09:00:20.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.118+0000 7f6daf92a740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dd614480a0 0x55dd61832130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:20.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.118+0000 7f6daf92a740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dd61450280 0x55dd61820730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:20.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.118+0000 7f6daf92a740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dd61470f60 0x55dd61820c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:20.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.118+0000 7f6daf92a740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55dd61832a00 con 0x55dd61450280 2024-08-30T09:00:20.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.118+0000 7f6daf92a740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55dd618330d0 con 0x55dd61470f60 2024-08-30T09:00:20.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.118+0000 7f6daf92a740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55dd618337a0 con 0x55dd614480a0 2024-08-30T09:00:20.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.118+0000 7f6da2ffd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dd61450280 0x55dd61820730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:20.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.118+0000 7f6da2ffd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dd61450280 0x55dd61820730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38374/0 (socket says 172.21.15.42:38374) 2024-08-30T09:00:20.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.118+0000 7f6da2ffd640 1 -- 172.21.15.42:0/2495589166 learned_addr learned my addr 172.21.15.42:0/2495589166 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:00:20.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.118+0000 7f6da27fc640 1 --2- 172.21.15.42:0/2495589166 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dd61470f60 0x55dd61820c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:20.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.118+0000 7f6da2ffd640 1 -- 172.21.15.42:0/2495589166 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dd614480a0 msgr2=0x55dd61832130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:00:20.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.118+0000 7f6da2ffd640 1 --2- 172.21.15.42:0/2495589166 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dd614480a0 0x55dd61832130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:20.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.118+0000 7f6da2ffd640 1 -- 172.21.15.42:0/2495589166 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dd61470f60 msgr2=0x55dd61820c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:00:20.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.118+0000 7f6da2ffd640 1 --2- 172.21.15.42:0/2495589166 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dd61470f60 0x55dd61820c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:20.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.118+0000 7f6da2ffd640 1 -- 172.21.15.42:0/2495589166 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dd61823170 con 0x55dd61450280 2024-08-30T09:00:20.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.118+0000 7f6da27fc640 1 --2- 172.21.15.42:0/2495589166 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dd61470f60 0x55dd61820c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-30T09:00:20.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.118+0000 7f6da2ffd640 1 --2- 172.21.15.42:0/2495589166 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dd61450280 0x55dd61820730 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f6d94005f70 tx=0x7f6d9402a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9425ecfe3b692ef server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:00:20.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.118+0000 7f6d88ff9640 1 -- 172.21.15.42:0/2495589166 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d94056070 con 0x55dd61450280 2024-08-30T09:00:20.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.119+0000 7f6d88ff9640 1 -- 172.21.15.42:0/2495589166 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d94004ed0 con 0x55dd61450280 2024-08-30T09:00:20.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.119+0000 7f6d88ff9640 1 -- 172.21.15.42:0/2495589166 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d94040070 con 0x55dd61450280 2024-08-30T09:00:20.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.119+0000 7f6daf92a740 1 -- 172.21.15.42:0/2495589166 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dd61450280 msgr2=0x55dd61820730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:00:20.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.119+0000 7f6daf92a740 1 --2- 172.21.15.42:0/2495589166 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dd61450280 0x55dd61820730 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f6d94005f70 tx=0x7f6d9402a410 comp rx=0 tx=0).stop 2024-08-30T09:00:20.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.119+0000 7f6daf92a740 1 -- 172.21.15.42:0/2495589166 shutdown_connections 2024-08-30T09:00:20.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.119+0000 7f6daf92a740 1 --2- 172.21.15.42:0/2495589166 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dd61470f60 0x55dd61820c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:20.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.119+0000 7f6daf92a740 1 --2- 172.21.15.42:0/2495589166 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dd61450280 0x55dd61820730 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:20.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.119+0000 7f6daf92a740 1 --2- 172.21.15.42:0/2495589166 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dd614480a0 0x55dd61832130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:20.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.119+0000 7f6daf92a740 1 -- 172.21.15.42:0/2495589166 >> 172.21.15.42:0/2495589166 conn(0x55dd61465350 msgr2=0x55dd6180f490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:00:20.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.119+0000 7f6daf92a740 1 -- 172.21.15.42:0/2495589166 shutdown_connections 2024-08-30T09:00:20.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.119+0000 7f6daf92a740 1 -- 172.21.15.42:0/2495589166 wait complete. 2024-08-30T09:00:20.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.120+0000 7f6daf92a740 1 -- start start 2024-08-30T09:00:20.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.120+0000 7f6daf92a740 1 Processor -- start 2024-08-30T09:00:20.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.120+0000 7f6daf92a740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dd614480a0 0x55dd6183cea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:20.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.120+0000 7f6daf92a740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dd61450280 0x55dd61836350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:20.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.120+0000 7f6daf92a740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dd61470f60 0x55dd61836890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:20.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.120+0000 7f6daf92a740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55dd61834540 con 0x55dd61450280 2024-08-30T09:00:20.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.120+0000 7f6daf92a740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55dd61833e70 con 0x55dd614480a0 2024-08-30T09:00:20.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.120+0000 7f6daf92a740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55dd61834c10 con 0x55dd61470f60 2024-08-30T09:00:20.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.120+0000 7f6daf92a740 10 client.0 _ll_register_callbacks cb 0x7f6d94032bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:00:20.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.120+0000 7f6da2ffd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dd61450280 0x55dd61836350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:20.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.120+0000 7f6da2ffd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dd61450280 0x55dd61836350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38386/0 (socket says 172.21.15.42:38386) 2024-08-30T09:00:20.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.120+0000 7f6da2ffd640 1 -- 172.21.15.42:0/565625182 learned_addr learned my addr 172.21.15.42:0/565625182 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:00:20.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.120+0000 7f6da27fc640 1 --2- 172.21.15.42:0/565625182 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dd614480a0 0x55dd6183cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:20.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.121+0000 7f6da27fc640 1 --2- 172.21.15.42:0/565625182 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dd61470f60 0x55dd61836890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:20.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.121+0000 7f6da2ffd640 1 -- 172.21.15.42:0/565625182 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dd61470f60 msgr2=0x55dd61836890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:00:20.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.121+0000 7f6da2ffd640 1 --2- 172.21.15.42:0/565625182 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dd61470f60 0x55dd61836890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:20.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.121+0000 7f6da2ffd640 1 -- 172.21.15.42:0/565625182 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dd614480a0 msgr2=0x55dd6183cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:00:20.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.121+0000 7f6da2ffd640 1 --2- 172.21.15.42:0/565625182 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dd614480a0 0x55dd6183cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:20.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.121+0000 7f6da2ffd640 1 -- 172.21.15.42:0/565625182 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dd61836dd0 con 0x55dd61450280 2024-08-30T09:00:20.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.121+0000 7f6da27fc640 1 --2- 172.21.15.42:0/565625182 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dd61470f60 0x55dd61836890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-30T09:00:20.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.121+0000 7f6da2ffd640 1 --2- 172.21.15.42:0/565625182 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dd61450280 0x55dd61836350 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f6d94004dd0 tx=0x7f6d94032f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc766ca0d3d53c2b server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:00:20.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.121+0000 7f6d86ff5640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-08-30T09:00:20.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.121+0000 7f6d86ff5640 1 -- 172.21.15.42:0/565625182 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d94060060 con 0x55dd61450280 2024-08-30T09:00:20.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.121+0000 7f6d86ff5640 1 -- 172.21.15.42:0/565625182 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f6d94004950 con 0x55dd61450280 2024-08-30T09:00:20.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.121+0000 7f6daf92a740 1 -- 172.21.15.42:0/565625182 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6d94004ed0 con 0x55dd61450280 2024-08-30T09:00:20.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.121+0000 7f6d86ff5640 1 -- 172.21.15.42:0/565625182 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f6d94067050 con 0x55dd61450280 2024-08-30T09:00:20.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.121+0000 7f6daf92a740 10 client.15393 fetch_fsmap learned FSMap version 26 2024-08-30T09:00:20.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.121+0000 7f6daf92a740 1 -- 172.21.15.42:0/565625182 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55dd6183c6e0 con 0x55dd61450280 2024-08-30T09:00:20.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.121+0000 7f6d86ff5640 1 -- 172.21.15.42:0/565625182 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d94004ed0 con 0x55dd61450280 2024-08-30T09:00:20.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.122+0000 7f6d86ff5640 1 -- 172.21.15.42:0/565625182 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f6d94050070 con 0x55dd61450280 2024-08-30T09:00:20.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.122+0000 7f6daf92a740 10 client.15393 fetch_fsmap finished waiting for FSMap version 26 2024-08-30T09:00:20.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.122+0000 7f6daf92a740 10 client.15393 resolve_mds: resolved name 'b' to daemon mds.b 2024-08-30T09:00:20.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.122+0000 7f6daf92a740 20 client.15393 populate_metadata read hostname 'smithi042' 2024-08-30T09:00:20.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.122+0000 7f6daf92a740 1 --2- 172.21.15.42:0/565625182 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55dd61404c90 0x55dd618a2c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:20.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.122+0000 7f6daf92a740 4 client.15393 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "status"}] 2024-08-30T09:00:20.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.122+0000 7f6daf92a740 1 -- 172.21.15.42:0/565625182 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "status"}) -- 0x55dd6178ade0 con 0x55dd61404c90 2024-08-30T09:00:20.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.122+0000 7f6da27fc640 1 --2- 172.21.15.42:0/565625182 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55dd61404c90 0x55dd618a2c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:20.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.122+0000 7f6d86ff5640 10 client.15393 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:00:20.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.122+0000 7f6da27fc640 1 --2- 172.21.15.42:0/565625182 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55dd61404c90 0x55dd618a2c20 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fbcff336b3954755 server_cookie=2265e71696b5f696 in_seq=0 out_seq=0 2024-08-30T09:00:20.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.123+0000 7f6d86ff5640 1 -- 172.21.15.42:0/565625182 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55dd6178ade0 con 0x55dd61404c90 2024-08-30T09:00:20.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.123+0000 7f6d86ff5640 10 client.15393 handle_command_reply: tid=0 2024-08-30T09:00:20.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.123+0000 7f6daf92a740 1 client.15393 shutdown 2024-08-30T09:00:20.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.123+0000 7f6daf92a740 1 -- 172.21.15.42:0/565625182 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dd61450280 msgr2=0x55dd61836350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:00:20.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.123+0000 7f6daf92a740 1 --2- 172.21.15.42:0/565625182 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dd61450280 0x55dd61836350 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f6d94004dd0 tx=0x7f6d94032f00 comp rx=0 tx=0).stop 2024-08-30T09:00:20.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.123+0000 7f6daf92a740 1 -- 172.21.15.42:0/565625182 shutdown_connections 2024-08-30T09:00:20.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.123+0000 7f6daf92a740 1 --2- 172.21.15.42:0/565625182 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55dd61404c90 0x55dd618a2c20 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:20.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.123+0000 7f6daf92a740 1 --2- 172.21.15.42:0/565625182 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dd61470f60 0x55dd61836890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:20.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.123+0000 7f6daf92a740 1 --2- 172.21.15.42:0/565625182 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dd61450280 0x55dd61836350 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:20.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.123+0000 7f6daf92a740 1 --2- 172.21.15.42:0/565625182 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dd614480a0 0x55dd6183cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:20.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.123+0000 7f6d86ff5640 0 client.15393 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:00:20.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.123+0000 7f6daf92a740 1 -- 172.21.15.42:0/565625182 >> 172.21.15.42:0/565625182 conn(0x55dd61465350 msgr2=0x55dd61832130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:00:20.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.123+0000 7f6daf92a740 1 -- 172.21.15.42:0/565625182 shutdown_connections 2024-08-30T09:00:20.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.123+0000 7f6daf92a740 1 -- 172.21.15.42:0/565625182 wait complete. 2024-08-30T09:00:20.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.123+0000 7f6daf92a740 20 client.15393 trim_cache size 0 max 16384 2024-08-30T09:00:20.130 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-08-30T09:00:20.130 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_fsid": "b0895c1a-66ac-11ef-bcd4-c7b262605968", 2024-08-30T09:00:20.130 INFO:teuthology.orchestra.run.smithi042.stdout: "whoami": 0, 2024-08-30T09:00:20.130 INFO:teuthology.orchestra.run.smithi042.stdout: "id": 24629, 2024-08-30T09:00:20.130 INFO:teuthology.orchestra.run.smithi042.stdout: "want_state": "up:active", 2024-08-30T09:00:20.130 INFO:teuthology.orchestra.run.smithi042.stdout: "state": "up:active", 2024-08-30T09:00:20.131 INFO:teuthology.orchestra.run.smithi042.stdout: "fs_name": "cephfs", 2024-08-30T09:00:20.131 INFO:teuthology.orchestra.run.smithi042.stdout: "rank_uptime": 136.723683833, 2024-08-30T09:00:20.131 INFO:teuthology.orchestra.run.smithi042.stdout: "mdsmap_epoch": 26, 2024-08-30T09:00:20.131 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_epoch": 83, 2024-08-30T09:00:20.131 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_epoch_barrier": 81, 2024-08-30T09:00:20.131 INFO:teuthology.orchestra.run.smithi042.stdout: "uptime": 147.49201583999999 2024-08-30T09:00:20.131 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-08-30T09:00:20.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.124+0000 7f6da93b5640 1 -- 172.21.15.42:0/2351139867 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f6d8c0b4b30 msgr2=0x7f6d8c0b7040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:00:20.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.124+0000 7f6da93b5640 1 --2- 172.21.15.42:0/2351139867 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f6d8c0b4b30 0x7f6d8c0b7040 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f6d9c0a7e40 tx=0x7f6d9c0792b0 comp rx=0 tx=0).stop 2024-08-30T09:00:20.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.124+0000 7f6da93b5640 1 -- 172.21.15.42:0/2351139867 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6da4108ac0 msgr2=0x7f6da419ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:00:20.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.124+0000 7f6da93b5640 1 --2- 172.21.15.42:0/2351139867 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6da4108ac0 0x7f6da419ba70 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f6d9c00eaa0 tx=0x7f6d9c00ef60 comp rx=0 tx=0).stop 2024-08-30T09:00:20.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.124+0000 7f6da93b5640 1 -- 172.21.15.42:0/2351139867 shutdown_connections 2024-08-30T09:00:20.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.124+0000 7f6da93b5640 1 --2- 172.21.15.42:0/2351139867 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f6d8c0b4b30 0x7f6d8c0b7040 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:20.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.124+0000 7f6da93b5640 1 --2- 172.21.15.42:0/2351139867 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6da4108ac0 0x7f6da419ba70 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:20.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.124+0000 7f6da93b5640 1 --2- 172.21.15.42:0/2351139867 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6da4108170 0x7f6da419b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:20.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.124+0000 7f6da93b5640 1 --2- 172.21.15.42:0/2351139867 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6da4102830 0x7f6da419aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:20.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.124+0000 7f6da93b5640 1 -- 172.21.15.42:0/2351139867 >> 172.21.15.42:0/2351139867 conn(0x7f6da40794f0 msgr2=0x7f6da4101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:00:20.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.124+0000 7f6da93b5640 1 -- 172.21.15.42:0/2351139867 shutdown_connections 2024-08-30T09:00:20.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:20.124+0000 7f6da93b5640 1 -- 172.21.15.42:0/2351139867 wait complete. 2024-08-30T09:00:20.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:00:20 smithi042 ceph-mon[18652]: pgmap v360: 129 pgs: 129 active+clean; 644 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 597 B/s wr, 8 op/s 2024-08-30T09:00:20.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:00:20 smithi190 ceph-mon[22001]: pgmap v360: 129 pgs: 129 active+clean; 644 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 597 B/s wr, 8 op/s 2024-08-30T09:00:20.799 INFO:teuthology.run_tasks:Running task kclient... 2024-08-30T09:00:20.814 INFO:tasks.kclient:Mounting kernel clients... 2024-08-30T09:00:20.814 INFO:tasks.kclient:config is {'client.0': {}} 2024-08-30T09:00:20.814 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-30T09:00:20.814 DEBUG:teuthology.orchestra.run.smithi042:> ip netns list 2024-08-30T09:00:20.826 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:00:20 smithi167 ceph-mon[22063]: pgmap v360: 129 pgs: 129 active+clean; 644 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 597 B/s wr, 8 op/s 2024-08-30T09:00:20.830 INFO:teuthology.orchestra.run.smithi042.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-08-30T09:00:20.831 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-30T09:00:20.831 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-08-30T09:00:20.895 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-30T09:00:20.896 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip link delete ceph-brx 2024-08-30T09:00:20.999 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=secure', 'nowsync', 'context=system_u:object_r:postgresql_db_t:s0'], 'syntax': 'v2', 'mount_subvol_num': 1, 'mountpoint': '/tmp/cephfs'} 2024-08-30T09:00:20.999 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-08-30T09:00:20.999 INFO:tasks.cephfs.mount:using client_config["hostfs_mntpt"] = /tmp/cephfs 2024-08-30T09:00:20.999 INFO:tasks.cephfs.mount:hostfs_mntpt = /tmp/cephfs 2024-08-30T09:00:21.000 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph fs ls 2024-08-30T09:00:21.254 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T09:00:21.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:21.743+0000 7f2e1bb16640 1 -- 172.21.15.42:0/107966764 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2e14108a90 msgr2=0x7f2e1410f2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:00:21.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:21.743+0000 7f2e1bb16640 1 --2- 172.21.15.42:0/107966764 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2e14108a90 0x7f2e1410f2c0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f2e0c009ca0 tx=0x7f2e0c02a8c0 comp rx=0 tx=0).stop 2024-08-30T09:00:21.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:21.743+0000 7f2e1bb16640 1 -- 172.21.15.42:0/107966764 shutdown_connections 2024-08-30T09:00:21.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:21.743+0000 7f2e1bb16640 1 --2- 172.21.15.42:0/107966764 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2e14108a90 0x7f2e1410f2c0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:21.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:21.743+0000 7f2e1bb16640 1 --2- 172.21.15.42:0/107966764 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2e14108140 0x7f2e14108550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:21.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:21.743+0000 7f2e1bb16640 1 --2- 172.21.15.42:0/107966764 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2e14102800 0x7f2e14102c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:21.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:21.743+0000 7f2e1bb16640 1 -- 172.21.15.42:0/107966764 >> 172.21.15.42:0/107966764 conn(0x7f2e140794d0 msgr2=0x7f2e140798e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:00:21.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:21.743+0000 7f2e1bb16640 1 -- 172.21.15.42:0/107966764 shutdown_connections 2024-08-30T09:00:21.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:21.744+0000 7f2e1bb16640 1 -- 172.21.15.42:0/107966764 wait complete. 2024-08-30T09:00:21.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:21.744+0000 7f2e1bb16640 1 Processor -- start 2024-08-30T09:00:21.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:21.744+0000 7f2e1bb16640 1 -- start start 2024-08-30T09:00:21.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:21.744+0000 7f2e1bb16640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2e14102800 0x7f2e1410fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:21.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:21.744+0000 7f2e1bb16640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2e14108140 0x7f2e14110290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:21.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:21.744+0000 7f2e1bb16640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2e14108a90 0x7f2e141107d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:21.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:21.744+0000 7f2e1bb16640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f2e14113820 con 0x7f2e14108140 2024-08-30T09:00:21.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:21.744+0000 7f2e1bb16640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f2e141136a0 con 0x7f2e14108a90 2024-08-30T09:00:21.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:21.744+0000 7f2e1bb16640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f2e141139a0 con 0x7f2e14102800 2024-08-30T09:00:21.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:21.745+0000 7f2e190b1640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2e14108140 0x7f2e14110290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:21.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:21.745+0000 7f2e190b1640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2e14108140 0x7f2e14110290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38408/0 (socket says 172.21.15.42:38408) 2024-08-30T09:00:21.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:21.745+0000 7f2e190b1640 1 -- 172.21.15.42:0/2120349060 learned_addr learned my addr 172.21.15.42:0/2120349060 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:00:21.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:21.745+0000 7f2e198b2640 1 --2- 172.21.15.42:0/2120349060 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2e14102800 0x7f2e1410fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:21.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:21.745+0000 7f2e190b1640 1 -- 172.21.15.42:0/2120349060 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2e14102800 msgr2=0x7f2e1410fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:00:21.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:21.745+0000 7f2e1a0b3640 1 --2- 172.21.15.42:0/2120349060 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2e14108a90 0x7f2e141107d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:21.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:21.745+0000 7f2e190b1640 1 --2- 172.21.15.42:0/2120349060 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2e14102800 0x7f2e1410fd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:21.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:21.745+0000 7f2e190b1640 1 -- 172.21.15.42:0/2120349060 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2e14108a90 msgr2=0x7f2e141107d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:00:21.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:21.745+0000 7f2e190b1640 1 --2- 172.21.15.42:0/2120349060 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2e14108a90 0x7f2e141107d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:21.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:21.745+0000 7f2e190b1640 1 -- 172.21.15.42:0/2120349060 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e141a8440 con 0x7f2e14108140 2024-08-30T09:00:21.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:21.745+0000 7f2e198b2640 1 --2- 172.21.15.42:0/2120349060 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2e14102800 0x7f2e1410fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-30T09:00:21.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:21.745+0000 7f2e1a0b3640 1 --2- 172.21.15.42:0/2120349060 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2e14108a90 0x7f2e141107d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-30T09:00:21.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:21.745+0000 7f2e190b1640 1 --2- 172.21.15.42:0/2120349060 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2e14108140 0x7f2e14110290 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f2e0400d770 tx=0x7f2e0400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:00:21.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:21.745+0000 7f2e0affd640 1 -- 172.21.15.42:0/2120349060 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e04017070 con 0x7f2e14108140 2024-08-30T09:00:21.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:21.745+0000 7f2e0affd640 1 -- 172.21.15.42:0/2120349060 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e04009c30 con 0x7f2e14108140 2024-08-30T09:00:21.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:21.745+0000 7f2e1bb16640 1 -- 172.21.15.42:0/2120349060 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e141a8730 con 0x7f2e14108140 2024-08-30T09:00:21.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:21.745+0000 7f2e0affd640 1 -- 172.21.15.42:0/2120349060 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e0400f620 con 0x7f2e14108140 2024-08-30T09:00:21.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:21.745+0000 7f2e1bb16640 1 -- 172.21.15.42:0/2120349060 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e141a8c70 con 0x7f2e14108140 2024-08-30T09:00:21.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:21.746+0000 7f2e1bb16640 1 -- 172.21.15.42:0/2120349060 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e14102c10 con 0x7f2e14108140 2024-08-30T09:00:21.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:21.748+0000 7f2e0affd640 1 -- 172.21.15.42:0/2120349060 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7f2e0401a070 con 0x7f2e14108140 2024-08-30T09:00:21.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:21.748+0000 7f2e0affd640 1 --2- 172.21.15.42:0/2120349060 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f2e000b4b30 0x7f2e000b7040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:21.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:21.749+0000 7f2e198b2640 1 --2- 172.21.15.42:0/2120349060 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f2e000b4b30 0x7f2e000b7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:21.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:21.749+0000 7f2e0affd640 1 -- 172.21.15.42:0/2120349060 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f2e040e5030 con 0x7f2e14108140 2024-08-30T09:00:21.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:21.749+0000 7f2e198b2640 1 --2- 172.21.15.42:0/2120349060 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f2e000b4b30 0x7f2e000b7040 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f2e141117a0 tx=0x7f2e10005db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:00:21.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:21.752+0000 7f2e0affd640 1 -- 172.21.15.42:0/2120349060 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f2e04009de0 con 0x7f2e14108140 2024-08-30T09:00:21.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:21.975+0000 7f2e1bb16640 1 -- 172.21.15.42:0/2120349060 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f2e1402e430 con 0x7f2e14108140 2024-08-30T09:00:21.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:21.975+0000 7f2e0affd640 1 -- 172.21.15.42:0/2120349060 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v26) ==== 53+0+88 (secure 0 0 0) 0x7f2e040abdd0 con 0x7f2e14108140 2024-08-30T09:00:21.977 INFO:teuthology.orchestra.run.smithi042.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-08-30T09:00:21.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:21.977+0000 7f2e1bb16640 1 -- 172.21.15.42:0/2120349060 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f2e000b4b30 msgr2=0x7f2e000b7040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:00:21.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:21.977+0000 7f2e1bb16640 1 --2- 172.21.15.42:0/2120349060 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f2e000b4b30 0x7f2e000b7040 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f2e141117a0 tx=0x7f2e10005db0 comp rx=0 tx=0).stop 2024-08-30T09:00:21.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:21.977+0000 7f2e1bb16640 1 -- 172.21.15.42:0/2120349060 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2e14108140 msgr2=0x7f2e14110290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:00:21.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:21.977+0000 7f2e1bb16640 1 --2- 172.21.15.42:0/2120349060 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2e14108140 0x7f2e14110290 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f2e0400d770 tx=0x7f2e0400dc30 comp rx=0 tx=0).stop 2024-08-30T09:00:21.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:21.977+0000 7f2e1bb16640 1 -- 172.21.15.42:0/2120349060 shutdown_connections 2024-08-30T09:00:21.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:21.977+0000 7f2e1bb16640 1 --2- 172.21.15.42:0/2120349060 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f2e000b4b30 0x7f2e000b7040 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:21.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:21.977+0000 7f2e1bb16640 1 --2- 172.21.15.42:0/2120349060 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2e14108a90 0x7f2e141107d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:21.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:21.977+0000 7f2e1bb16640 1 --2- 172.21.15.42:0/2120349060 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2e14108140 0x7f2e14110290 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:21.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:21.977+0000 7f2e1bb16640 1 --2- 172.21.15.42:0/2120349060 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2e14102800 0x7f2e1410fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:21.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:21.977+0000 7f2e1bb16640 1 -- 172.21.15.42:0/2120349060 >> 172.21.15.42:0/2120349060 conn(0x7f2e140794d0 msgr2=0x7f2e14105060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:00:21.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:21.978+0000 7f2e1bb16640 1 -- 172.21.15.42:0/2120349060 shutdown_connections 2024-08-30T09:00:21.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:21.978+0000 7f2e1bb16640 1 -- 172.21.15.42:0/2120349060 wait complete. 2024-08-30T09:00:22.581 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:00:22 smithi042 ceph-mon[18652]: pgmap v361: 129 pgs: 129 active+clean; 644 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 5 op/s 2024-08-30T09:00:22.581 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:00:22 smithi042 ceph-mon[18652]: from='client.? 172.21.15.42:0/2120349060' entity='client.admin' cmd={"prefix": "fs ls"} : dispatch 2024-08-30T09:00:22.610 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-08-30T09:00:22.611 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-08-30T09:00:22.611 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi042.front.sepia.ceph.com 2024-08-30T09:00:22.611 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-08-30T09:00:22.611 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /tmp/cephfs 2024-08-30T09:00:22.611 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-08-30T09:00:22.611 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-08-30T09:00:22.611 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-08-30T09:00:22.611 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/tmp/cephfs' 2024-08-30T09:00:22.611 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-30T09:00:22.611 DEBUG:teuthology.orchestra.run.smithi042:> ip addr 2024-08-30T09:00:22.629 INFO:teuthology.orchestra.run.smithi042.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-08-30T09:00:22.629 INFO:teuthology.orchestra.run.smithi042.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-08-30T09:00:22.629 INFO:teuthology.orchestra.run.smithi042.stdout: inet 127.0.0.1/8 scope host lo 2024-08-30T09:00:22.629 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-08-30T09:00:22.629 INFO:teuthology.orchestra.run.smithi042.stdout: inet6 ::1/128 scope host 2024-08-30T09:00:22.629 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-08-30T09:00:22.629 INFO:teuthology.orchestra.run.smithi042.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-08-30T09:00:22.630 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether 0c:c4:7a:6b:fd:0e brd ff:ff:ff:ff:ff:ff 2024-08-30T09:00:22.630 INFO:teuthology.orchestra.run.smithi042.stdout: altname enp7s0f0 2024-08-30T09:00:22.630 INFO:teuthology.orchestra.run.smithi042.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-08-30T09:00:22.630 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether 0c:c4:7a:6b:fd:0f brd ff:ff:ff:ff:ff:ff 2024-08-30T09:00:22.630 INFO:teuthology.orchestra.run.smithi042.stdout: altname enp7s0f1 2024-08-30T09:00:22.630 INFO:teuthology.orchestra.run.smithi042.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-08-30T09:00:22.630 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether 0c:c4:7a:bd:14:40 brd ff:ff:ff:ff:ff:ff 2024-08-30T09:00:22.630 INFO:teuthology.orchestra.run.smithi042.stdout: altname enp1s0f0 2024-08-30T09:00:22.630 INFO:teuthology.orchestra.run.smithi042.stdout: inet 172.21.15.42/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-08-30T09:00:22.630 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft 42177sec preferred_lft 42177sec 2024-08-30T09:00:22.630 INFO:teuthology.orchestra.run.smithi042.stdout: inet6 fe80::666b:514f:12c9:e93f/64 scope link noprefixroute 2024-08-30T09:00:22.630 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-08-30T09:00:22.630 INFO:teuthology.orchestra.run.smithi042.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-08-30T09:00:22.631 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether 0c:c4:7a:bd:14:41 brd ff:ff:ff:ff:ff:ff 2024-08-30T09:00:22.631 INFO:teuthology.orchestra.run.smithi042.stdout: altname enp1s0f1 2024-08-30T09:00:22.631 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-08-30T09:00:22.631 DEBUG:teuthology.orchestra.run.smithi042:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-08-30T09:00:22.631 DEBUG:teuthology.orchestra.run.smithi042:> set -e 2024-08-30T09:00:22.631 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip link add name ceph-brx type bridge 2024-08-30T09:00:22.631 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip addr flush dev ceph-brx 2024-08-30T09:00:22.631 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip link set ceph-brx up 2024-08-30T09:00:22.631 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-08-30T09:00:22.631 DEBUG:teuthology.orchestra.run.smithi042:> ') 2024-08-30T09:00:22.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:00:22 smithi190 ceph-mon[22001]: pgmap v361: 129 pgs: 129 active+clean; 644 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 5 op/s 2024-08-30T09:00:22.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:00:22 smithi190 ceph-mon[22001]: from='client.? 172.21.15.42:0/2120349060' entity='client.admin' cmd={"prefix": "fs ls"} : dispatch 2024-08-30T09:00:22.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:22 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-08-30T09:00:22.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-08-30T09:00:22.773 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-30T09:00:22.773 DEBUG:teuthology.orchestra.run.smithi042:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-08-30T09:00:22.797 INFO:teuthology.orchestra.run.smithi042.stdout:1 2024-08-30T09:00:22.798 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-30T09:00:22.798 DEBUG:teuthology.orchestra.run.smithi042:> route 2024-08-30T09:00:22.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:00:22 smithi167 ceph-mon[22063]: pgmap v361: 129 pgs: 129 active+clean; 644 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 5 op/s 2024-08-30T09:00:22.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:00:22 smithi167 ceph-mon[22063]: from='client.? 172.21.15.42:0/2120349060' entity='client.admin' cmd={"prefix": "fs ls"} : dispatch 2024-08-30T09:00:22.855 INFO:teuthology.orchestra.run.smithi042.stdout:Kernel IP routing table 2024-08-30T09:00:22.855 INFO:teuthology.orchestra.run.smithi042.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-08-30T09:00:22.855 INFO:teuthology.orchestra.run.smithi042.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-08-30T09:00:22.855 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-08-30T09:00:22.855 INFO:teuthology.orchestra.run.smithi042.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-08-30T09:00:22.856 DEBUG:teuthology.orchestra.run.smithi042:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-08-30T09:00:22.856 DEBUG:teuthology.orchestra.run.smithi042:> set -e 2024-08-30T09:00:22.856 DEBUG:teuthology.orchestra.run.smithi042:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-08-30T09:00:22.856 DEBUG:teuthology.orchestra.run.smithi042:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-08-30T09:00:22.856 DEBUG:teuthology.orchestra.run.smithi042:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-08-30T09:00:22.856 DEBUG:teuthology.orchestra.run.smithi042:> ') 2024-08-30T09:00:22.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:22 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-08-30T09:00:22.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-08-30T09:00:22.982 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-30T09:00:22.982 DEBUG:teuthology.orchestra.run.smithi042:> ip netns list 2024-08-30T09:00:22.997 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-30T09:00:22.997 DEBUG:teuthology.orchestra.run.smithi042:> ip netns list-id 2024-08-30T09:00:23.053 DEBUG:teuthology.orchestra.run.smithi042:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-08-30T09:00:23.053 DEBUG:teuthology.orchestra.run.smithi042:> set -e 2024-08-30T09:00:23.053 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip netns add ceph-ns--tmp-cephfs 2024-08-30T09:00:23.053 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip netns set ceph-ns--tmp-cephfs 0 2024-08-30T09:00:23.053 DEBUG:teuthology.orchestra.run.smithi042:> ') 2024-08-30T09:00:23.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:23 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-08-30T09:00:23.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-08-30T09:00:23.170 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--tmp-cephfs' with 192.168.0.1/16 2024-08-30T09:00:23.170 DEBUG:teuthology.orchestra.run.smithi042:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-08-30T09:00:23.170 DEBUG:teuthology.orchestra.run.smithi042:> set -e 2024-08-30T09:00:23.171 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip link add veth0 netns ceph-ns--tmp-cephfs type veth peer name brx.0 2024-08-30T09:00:23.171 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip netns exec ceph-ns--tmp-cephfs ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-08-30T09:00:23.171 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip netns exec ceph-ns--tmp-cephfs ip link set veth0 up 2024-08-30T09:00:23.171 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip netns exec ceph-ns--tmp-cephfs ip link set lo up 2024-08-30T09:00:23.171 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip netns exec ceph-ns--tmp-cephfs ip route add default via 192.168.255.254 2024-08-30T09:00:23.171 DEBUG:teuthology.orchestra.run.smithi042:> ') 2024-08-30T09:00:23.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:23 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-08-30T09:00:23.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-08-30T09:00:23.284 DEBUG:teuthology.orchestra.run.smithi042:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-08-30T09:00:23.284 DEBUG:teuthology.orchestra.run.smithi042:> set -e 2024-08-30T09:00:23.285 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip link set brx.0 up 2024-08-30T09:00:23.285 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip link set dev brx.0 master ceph-brx 2024-08-30T09:00:23.285 DEBUG:teuthology.orchestra.run.smithi042:> ') 2024-08-30T09:00:23.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:23 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-08-30T09:00:23.356 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-08-30T09:00:23.362 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-08-30T09:00:23.362 DEBUG:teuthology.orchestra.run.smithi042:> mkdir -p -v /tmp/cephfs 2024-08-30T09:00:23.377 INFO:teuthology.orchestra.run.smithi042.stdout:mkdir: created directory '/tmp/cephfs' 2024-08-30T09:00:23.377 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-08-30T09:00:23.377 DEBUG:teuthology.orchestra.run.smithi042:> chmod 0000 /tmp/cephfs 2024-08-30T09:00:23.432 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-08-30T09:00:23.432 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-08-30T09:00:23.433 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/volumes/qa/sv_1/d3168875-d2e5-410d-8754-72d29955777b 2024-08-30T09:00:23.433 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-30T09:00:23.433 DEBUG:teuthology.orchestra.run.smithi042:> sudo nsenter --net=/var/run/netns/ceph-ns--tmp-cephfs adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/volumes/qa/sv_1/d3168875-d2e5-410d-8754-72d29955777b /tmp/cephfs -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=secure,nowsync,context=system_u:object_r:postgresql_db_t:s0,nofallback 2024-08-30T09:00:23.525 INFO:teuthology.orchestra.run.smithi042.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=secure,nowsync,context="system_u:object_r:postgresql_db_t:s0",nofallback 2024-08-30T09:00:23.525 INFO:teuthology.orchestra.run.smithi042.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=secure,nowsync,context="system_u:object_r:postgresql_db_t:s0"". 2024-08-30T09:00:23.655 INFO:teuthology.orchestra.run.smithi042.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=secure,nowsync,context="system_u:object_r:postgresql_db_t:s0",nofallback 2024-08-30T09:00:23.655 INFO:teuthology.orchestra.run.smithi042.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=secure,nowsync,context="system_u:object_r:postgresql_db_t:s0"". 2024-08-30T09:00:23.655 INFO:teuthology.orchestra.run.smithi042.stdout:mount.ceph: resolved to: "172.21.15.42:3300,172.21.15.167:3300,172.21.15.190:3300" 2024-08-30T09:00:23.655 INFO:teuthology.orchestra.run.smithi042.stdout:mount.ceph: trying mount with new device syntax: 0@b0895c1a-66ac-11ef-bcd4-c7b262605968.cephfs=/volumes/qa/sv_1/d3168875-d2e5-410d-8754-72d29955777b 2024-08-30T09:00:23.655 INFO:teuthology.orchestra.run.smithi042.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=secure,nowsync,context="system_u:object_r:postgresql_db_t:s0",name=0,key=0,mon_addr=172.21.15.42:3300/172.21.15.167:3300/172.21.15.190:3300" will pass to kernel 2024-08-30T09:00:23.658 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-08-30T09:00:23.658 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-30T09:00:23.658 DEBUG:teuthology.orchestra.run.smithi042:> sudo chmod 1777 /tmp/cephfs 2024-08-30T09:00:23.684 DEBUG:teuthology.orchestra.run.smithi042:> (cd /tmp/cephfs && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-08-30T09:00:23.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:23 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-08-30T09:00:23.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-08-30T09:00:23.780 INFO:teuthology.orchestra.run.smithi042.stdout:client15402 2024-08-30T09:00:23.781 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-08-30T09:00:23.781 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/proc/self/mounts of=/dev/stdout 2024-08-30T09:00:23.797 DEBUG:teuthology.orchestra.run.smithi042:> (cd /tmp/cephfs && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-08-30T09:00:23.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:23 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-08-30T09:00:23.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-08-30T09:00:23.893 INFO:teuthology.orchestra.run.smithi042.stdout:client15402 2024-08-30T09:00:23.894 DEBUG:teuthology.orchestra.run.smithi042:> (cd /tmp/cephfs && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-08-30T09:00:23.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:23 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-08-30T09:00:23.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-08-30T09:00:23.949 INFO:teuthology.orchestra.run.smithi042.stdout:client15402 2024-08-30T09:00:23.950 DEBUG:teuthology.orchestra.run.smithi042:> (cd /tmp/cephfs && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968.client15402/status') 2024-08-30T09:00:23.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:23 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-08-30T09:00:24.008 INFO:teuthology.orchestra.run.smithi042.stdout:instance: client.15402 (3)192.168.0.1:0/3096654597 2024-08-30T09:00:24.008 INFO:teuthology.orchestra.run.smithi042.stdout:blocklisted: false 2024-08-30T09:00:24.009 INFO:teuthology.orchestra.run.smithi042.stderr:0+1 records in 2024-08-30T09:00:24.009 INFO:teuthology.orchestra.run.smithi042.stderr:0+1 records out 2024-08-30T09:00:24.009 INFO:teuthology.orchestra.run.smithi042.stderr:70 bytes copied, 2.4087e-05 s, 2.9 MB/s 2024-08-30T09:00:24.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-08-30T09:00:24.015 INFO:teuthology.run_tasks:Running task exec... 2024-08-30T09:00:24.029 INFO:teuthology.task.exec:Executing custom commands... 2024-08-30T09:00:24.029 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi042.front.sepia.ceph.com 2024-08-30T09:00:24.029 DEBUG:teuthology.orchestra.run.smithi042:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate true' 2024-08-30T09:00:24.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:24.139+0000 7ff187e2e640 1 -- 172.21.15.42:0/356357404 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff180103d90 msgr2=0x7ff18010a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:00:24.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:24.139+0000 7ff187e2e640 1 --2- 172.21.15.42:0/356357404 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff180103d90 0x7ff18010a730 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7ff178009ca0 tx=0x7ff17802a8c0 comp rx=0 tx=0).stop 2024-08-30T09:00:24.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:24.139+0000 7ff187e2e640 1 -- 172.21.15.42:0/356357404 shutdown_connections 2024-08-30T09:00:24.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:24.139+0000 7ff187e2e640 1 --2- 172.21.15.42:0/356357404 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff180103d90 0x7ff18010a730 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:24.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:24.139+0000 7ff187e2e640 1 --2- 172.21.15.42:0/356357404 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff1801033f0 0x7ff180103800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:24.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:24.139+0000 7ff187e2e640 1 --2- 172.21.15.42:0/356357404 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff180102370 0x7ff1801027e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:24.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:24.139+0000 7ff187e2e640 1 -- 172.21.15.42:0/356357404 >> 172.21.15.42:0/356357404 conn(0x7ff1800fdcd0 msgr2=0x7ff180100160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:00:24.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:24.139+0000 7ff187e2e640 1 -- 172.21.15.42:0/356357404 shutdown_connections 2024-08-30T09:00:24.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:24.139+0000 7ff187e2e640 1 -- 172.21.15.42:0/356357404 wait complete. 2024-08-30T09:00:24.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:24.140+0000 7ff187e2e640 1 Processor -- start 2024-08-30T09:00:24.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:24.140+0000 7ff187e2e640 1 -- start start 2024-08-30T09:00:24.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:24.140+0000 7ff187e2e640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff180102370 0x7ff180198b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:24.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:24.140+0000 7ff187e2e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff1801033f0 0x7ff1801991c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:24.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:24.140+0000 7ff187e2e640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff180103d90 0x7ff180199d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:24.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:24.140+0000 7ff187e2e640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7ff18010dff0 con 0x7ff180103d90 2024-08-30T09:00:24.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:24.140+0000 7ff187e2e640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff18010de70 con 0x7ff180102370 2024-08-30T09:00:24.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:24.140+0000 7ff187e2e640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7ff18010e170 con 0x7ff1801033f0 2024-08-30T09:00:24.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:24.140+0000 7ff1863cb640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff180103d90 0x7ff180199d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:24.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:24.140+0000 7ff1863cb640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff180103d90 0x7ff180199d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38430/0 (socket says 172.21.15.42:38430) 2024-08-30T09:00:24.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:24.140+0000 7ff1863cb640 1 -- 172.21.15.42:0/1095114448 learned_addr learned my addr 172.21.15.42:0/1095114448 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:00:24.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:24.140+0000 7ff185bca640 1 --2- 172.21.15.42:0/1095114448 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff180102370 0x7ff180198b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:24.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:24.140+0000 7ff1853c9640 1 --2- 172.21.15.42:0/1095114448 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff1801033f0 0x7ff1801991c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:24.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:24.140+0000 7ff1863cb640 1 -- 172.21.15.42:0/1095114448 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff1801033f0 msgr2=0x7ff1801991c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:00:24.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:24.140+0000 7ff1863cb640 1 --2- 172.21.15.42:0/1095114448 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff1801033f0 0x7ff1801991c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:24.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:24.140+0000 7ff1863cb640 1 -- 172.21.15.42:0/1095114448 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff180102370 msgr2=0x7ff180198b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:00:24.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:24.140+0000 7ff1863cb640 1 --2- 172.21.15.42:0/1095114448 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff180102370 0x7ff180198b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:24.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:24.140+0000 7ff1863cb640 1 -- 172.21.15.42:0/1095114448 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff18019a590 con 0x7ff180103d90 2024-08-30T09:00:24.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:24.140+0000 7ff1853c9640 1 --2- 172.21.15.42:0/1095114448 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff1801033f0 0x7ff1801991c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-30T09:00:24.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:24.141+0000 7ff1863cb640 1 --2- 172.21.15.42:0/1095114448 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff180103d90 0x7ff180199d70 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7ff1780097b0 tx=0x7ff178003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:00:24.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:24.141+0000 7ff176ffd640 1 -- 172.21.15.42:0/1095114448 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff17804a070 con 0x7ff180103d90 2024-08-30T09:00:24.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:24.141+0000 7ff176ffd640 1 -- 172.21.15.42:0/1095114448 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff17802f310 con 0x7ff180103d90 2024-08-30T09:00:24.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:24.141+0000 7ff176ffd640 1 -- 172.21.15.42:0/1095114448 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff17802fa40 con 0x7ff180103d90 2024-08-30T09:00:24.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:24.141+0000 7ff187e2e640 1 -- 172.21.15.42:0/1095114448 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff18019a820 con 0x7ff180103d90 2024-08-30T09:00:24.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:24.141+0000 7ff187e2e640 1 -- 172.21.15.42:0/1095114448 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff18006b900 con 0x7ff180103d90 2024-08-30T09:00:24.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:24.142+0000 7ff187e2e640 1 -- 172.21.15.42:0/1095114448 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1801027e0 con 0x7ff180103d90 2024-08-30T09:00:24.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:24.143+0000 7ff176ffd640 1 -- 172.21.15.42:0/1095114448 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7ff17804d070 con 0x7ff180103d90 2024-08-30T09:00:24.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:24.144+0000 7ff176ffd640 1 --2- 172.21.15.42:0/1095114448 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7ff16c0b4bb0 0x7ff16c0b70c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:24.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:24.144+0000 7ff185bca640 1 --2- 172.21.15.42:0/1095114448 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7ff16c0b4bb0 0x7ff16c0b70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:24.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:24.144+0000 7ff176ffd640 1 -- 172.21.15.42:0/1095114448 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7ff178107d90 con 0x7ff180103d90 2024-08-30T09:00:24.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:24.145+0000 7ff185bca640 1 --2- 172.21.15.42:0/1095114448 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7ff16c0b4bb0 0x7ff16c0b70c0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7ff17c0058c0 tx=0x7ff17c00a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:00:24.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:24.147+0000 7ff176ffd640 1 -- 172.21.15.42:0/1095114448 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7ff178084130 con 0x7ff180103d90 2024-08-30T09:00:24.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:24.376+0000 7ff187e2e640 1 -- 172.21.15.42:0/1095114448 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} v 0) -- 0x7ff180109050 con 0x7ff180103d90 2024-08-30T09:00:24.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:00:24 smithi042 ceph-mon[18652]: pgmap v362: 129 pgs: 129 active+clean; 644 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 597 B/s wr, 8 op/s 2024-08-30T09:00:24.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:00:24 smithi190 ceph-mon[22001]: pgmap v362: 129 pgs: 129 active+clean; 644 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 597 B/s wr, 8 op/s 2024-08-30T09:00:24.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:00:24 smithi167 ceph-mon[22063]: pgmap v362: 129 pgs: 129 active+clean; 644 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 597 B/s wr, 8 op/s 2024-08-30T09:00:25.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:25.360+0000 7ff176ffd640 1 -- 172.21.15.42:0/1095114448 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]=0 v27) ==== 117+0+0 (secure 0 0 0) 0x7ff1780ceb30 con 0x7ff180103d90 2024-08-30T09:00:25.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:25.363+0000 7ff187e2e640 1 -- 172.21.15.42:0/1095114448 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7ff16c0b4bb0 msgr2=0x7ff16c0b70c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:00:25.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:25.363+0000 7ff187e2e640 1 --2- 172.21.15.42:0/1095114448 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7ff16c0b4bb0 0x7ff16c0b70c0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7ff17c0058c0 tx=0x7ff17c00a2d0 comp rx=0 tx=0).stop 2024-08-30T09:00:25.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:25.363+0000 7ff187e2e640 1 -- 172.21.15.42:0/1095114448 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff180103d90 msgr2=0x7ff180199d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:00:25.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:25.363+0000 7ff187e2e640 1 --2- 172.21.15.42:0/1095114448 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff180103d90 0x7ff180199d70 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7ff1780097b0 tx=0x7ff178003ba0 comp rx=0 tx=0).stop 2024-08-30T09:00:25.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:25.363+0000 7ff187e2e640 1 -- 172.21.15.42:0/1095114448 shutdown_connections 2024-08-30T09:00:25.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:25.363+0000 7ff187e2e640 1 --2- 172.21.15.42:0/1095114448 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7ff16c0b4bb0 0x7ff16c0b70c0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:25.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:25.363+0000 7ff187e2e640 1 --2- 172.21.15.42:0/1095114448 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff1801033f0 0x7ff1801991c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:25.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:25.363+0000 7ff187e2e640 1 --2- 172.21.15.42:0/1095114448 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff180103d90 0x7ff180199d70 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:25.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:25.363+0000 7ff187e2e640 1 --2- 172.21.15.42:0/1095114448 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff180102370 0x7ff180198b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:25.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:25.363+0000 7ff187e2e640 1 -- 172.21.15.42:0/1095114448 >> 172.21.15.42:0/1095114448 conn(0x7ff1800fdcd0 msgr2=0x7ff180106710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:00:25.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:25.363+0000 7ff187e2e640 1 -- 172.21.15.42:0/1095114448 shutdown_connections 2024-08-30T09:00:25.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:25.363+0000 7ff187e2e640 1 -- 172.21.15.42:0/1095114448 wait complete. 2024-08-30T09:00:25.379 INFO:teuthology.run_tasks:Running task check-counter... 2024-08-30T09:00:25.390 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-08-30T09:00:25.400 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-30T09:00:25.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:25.520+0000 7f36261f9640 1 -- 172.21.15.42:0/2087764310 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3620106890 msgr2=0x7f3620108cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:00:25.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:25.520+0000 7f36261f9640 1 --2- 172.21.15.42:0/2087764310 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3620106890 0x7f3620108cb0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f3614009900 tx=0x7f361402a410 comp rx=0 tx=0).stop 2024-08-30T09:00:25.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:25.520+0000 7f36261f9640 1 -- 172.21.15.42:0/2087764310 shutdown_connections 2024-08-30T09:00:25.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:25.520+0000 7f36261f9640 1 --2- 172.21.15.42:0/2087764310 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f36201091f0 0x7f362010b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:25.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:25.520+0000 7f36261f9640 1 --2- 172.21.15.42:0/2087764310 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3620106890 0x7f3620108cb0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:25.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:25.520+0000 7f36261f9640 1 --2- 172.21.15.42:0/2087764310 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3620101800 0x7f3620101c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:25.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:25.520+0000 7f36261f9640 1 -- 172.21.15.42:0/2087764310 >> 172.21.15.42:0/2087764310 conn(0x7f36200fd400 msgr2=0x7f36200ff8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:00:25.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:25.520+0000 7f36261f9640 1 -- 172.21.15.42:0/2087764310 shutdown_connections 2024-08-30T09:00:25.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:25.520+0000 7f36261f9640 1 -- 172.21.15.42:0/2087764310 wait complete. 2024-08-30T09:00:25.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:25.521+0000 7f36261f9640 1 Processor -- start 2024-08-30T09:00:25.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:25.521+0000 7f36261f9640 1 -- start start 2024-08-30T09:00:25.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:25.521+0000 7f36261f9640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3620101800 0x7f3620199cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:25.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:25.521+0000 7f36261f9640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3620106890 0x7f362019a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:25.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:25.521+0000 7f36261f9640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f36201091f0 0x7f362019aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:25.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:25.521+0000 7f36261f9640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f362010f0e0 con 0x7f36201091f0 2024-08-30T09:00:25.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:25.521+0000 7f36261f9640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f362010ef60 con 0x7f3620106890 2024-08-30T09:00:25.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:25.522+0000 7f36261f9640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f362010f260 con 0x7f3620101800 2024-08-30T09:00:25.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:25.522+0000 7f361ffff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f36201091f0 0x7f362019aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:25.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:25.522+0000 7f361ffff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f36201091f0 0x7f362019aef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38448/0 (socket says 172.21.15.42:38448) 2024-08-30T09:00:25.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:25.522+0000 7f361ffff640 1 -- 172.21.15.42:0/3320797259 learned_addr learned my addr 172.21.15.42:0/3320797259 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:00:25.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:25.522+0000 7f361f7fe640 1 --2- 172.21.15.42:0/3320797259 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3620101800 0x7f3620199cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:25.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:25.522+0000 7f361effd640 1 --2- 172.21.15.42:0/3320797259 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3620106890 0x7f362019a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:25.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:25.522+0000 7f361ffff640 1 -- 172.21.15.42:0/3320797259 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3620101800 msgr2=0x7f3620199cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:00:25.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:25.522+0000 7f361ffff640 1 --2- 172.21.15.42:0/3320797259 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3620101800 0x7f3620199cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:25.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:25.522+0000 7f361ffff640 1 -- 172.21.15.42:0/3320797259 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3620106890 msgr2=0x7f362019a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:00:25.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:25.522+0000 7f361ffff640 1 --2- 172.21.15.42:0/3320797259 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3620106890 0x7f362019a340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:25.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:25.522+0000 7f361ffff640 1 -- 172.21.15.42:0/3320797259 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f362019b680 con 0x7f36201091f0 2024-08-30T09:00:25.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:25.522+0000 7f361f7fe640 1 --2- 172.21.15.42:0/3320797259 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3620101800 0x7f3620199cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-30T09:00:25.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:25.522+0000 7f361ffff640 1 --2- 172.21.15.42:0/3320797259 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f36201091f0 0x7f362019aef0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f361800b970 tx=0x7f361800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:00:25.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:25.522+0000 7f361cff9640 1 -- 172.21.15.42:0/3320797259 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3618019070 con 0x7f36201091f0 2024-08-30T09:00:25.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:25.522+0000 7f361cff9640 1 -- 172.21.15.42:0/3320797259 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3618014440 con 0x7f36201091f0 2024-08-30T09:00:25.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:25.522+0000 7f36261f9640 1 -- 172.21.15.42:0/3320797259 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f362019b970 con 0x7f36201091f0 2024-08-30T09:00:25.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:25.522+0000 7f361cff9640 1 -- 172.21.15.42:0/3320797259 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36180075d0 con 0x7f36201091f0 2024-08-30T09:00:25.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:25.522+0000 7f36261f9640 1 -- 172.21.15.42:0/3320797259 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36201a20f0 con 0x7f36201091f0 2024-08-30T09:00:25.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:25.523+0000 7f36261f9640 1 -- 172.21.15.42:0/3320797259 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3620101c10 con 0x7f36201091f0 2024-08-30T09:00:25.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:25.526+0000 7f361cff9640 1 -- 172.21.15.42:0/3320797259 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7f361801c070 con 0x7f36201091f0 2024-08-30T09:00:25.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:25.527+0000 7f361cff9640 1 --2- 172.21.15.42:0/3320797259 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f36080b4bb0 0x7f36080b70c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:25.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:25.527+0000 7f361cff9640 1 -- 172.21.15.42:0/3320797259 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f36180e4e80 con 0x7f36201091f0 2024-08-30T09:00:25.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:25.527+0000 7f361f7fe640 1 --2- 172.21.15.42:0/3320797259 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f36080b4bb0 0x7f36080b70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:25.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:25.528+0000 7f361f7fe640 1 --2- 172.21.15.42:0/3320797259 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f36080b4bb0 0x7f36080b70c0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f361000baa0 tx=0x7f3610005d90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:00:25.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:25.530+0000 7f361cff9640 1 -- 172.21.15.42:0/3320797259 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f36180abc20 con 0x7f36201091f0 2024-08-30T09:00:25.632 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:00:25 smithi042 ceph-mon[18652]: from='client.? 172.21.15.42:0/1095114448' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} : dispatch 2024-08-30T09:00:25.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:00:25 smithi190 ceph-mon[22001]: from='client.? 172.21.15.42:0/1095114448' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} : dispatch 2024-08-30T09:00:25.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:25.719+0000 7f36261f9640 1 -- 172.21.15.42:0/3320797259 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f362002e430 con 0x7f36201091f0 2024-08-30T09:00:25.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:25.721+0000 7f361cff9640 1 -- 172.21.15.42:0/3320797259 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+24867 (secure 0 0 0) 0x7f36180ab520 con 0x7f36201091f0 2024-08-30T09:00:25.723 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-08-30T09:00:25.723 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":83,"fsid":"b0895c1a-66ac-11ef-bcd4-c7b262605968","created":"2024-08-30T08:49:44.858265+0000","modified":"2024-08-30T08:59:46.645920+0000","last_up_change":"2024-08-30T08:57:12.367791+0000","last_in_change":"2024-08-30T08:56:50.274678+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-30T08:53:27.297022+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-30T08:57:58.489964+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-30T08:58:01.064783+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-08-30T08:58:07.416534+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"029d62d7-a6c6-4905-8c76-6dffa9d57b92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":2617890537},{"type":"v1","addr":"172.21.15.42:6803","nonce":2617890537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":2617890537},{"type":"v1","addr":"172.21.15.42:6805","nonce":2617890537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2617890537},{"type":"v1","addr":"172.21.15.42:6809","nonce":2617890537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":2617890537},{"type":"v1","addr":"172.21.15.42:6807","nonce":2617890537}]},"public_addr":"172.21.15.42:6803/2617890537","cluster_addr":"172.21.15.42:6805/2617890537","heartbeat_back_addr":"172.21.15.42:6809/2617890537","heartbeat_front_addr":"172.21.15.42:6807/2617890537","state":["exists","up"]},{"osd":1,"uuid":"3a78384b-0207-41df-9384-22665007d5b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":2791960082},{"type":"v1","addr":"172.21.15.167:6801","nonce":2791960082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":2791960082},{"type":"v1","addr":"172.21.15.167:6803","nonce":2791960082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":2791960082},{"type":"v1","addr":"172.21.15.167:6807","nonce":2791960082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":2791960082},{"type":"v1","addr":"172.21.15.167:6805","nonce":2791960082}]},"public_addr":"172.21.15.167:6801/2791960082","cluster_addr":"172.21.15.167:6803/2791960082","heartbeat_back_addr":"172.21.15.167:6807/2791960082","heartbeat_front_addr":"172.21.15.167:6805/2791960082","state":["exists","up"]},{"osd":2,"uuid":"16001809-c27c-4f52-948a-9998261acb4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":1101326404},{"type":"v1","addr":"172.21.15.190:6801","nonce":1101326404}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":1101326404},{"type":"v1","addr":"172.21.15.190:6803","nonce":1101326404}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":1101326404},{"type":"v1","addr":"172.21.15.190:6807","nonce":1101326404}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":1101326404},{"type":"v1","addr":"172.21.15.190:6805","nonce":1101326404}]},"public_addr":"172.21.15.190:6801/1101326404","cluster_addr":"172.21.15.190:6803/1101326404","heartbeat_back_addr":"172.21.15.190:6807/1101326404","heartbeat_front_addr":"172.21.15.190:6805/1101326404","state":["exists","up"]},{"osd":3,"uuid":"87bb3e36-9e9e-4b19-9d8c-0cdf6e089d3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":1135599916},{"type":"v1","addr":"172.21.15.42:6811","nonce":1135599916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":1135599916},{"type":"v1","addr":"172.21.15.42:6813","nonce":1135599916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":1135599916},{"type":"v1","addr":"172.21.15.42:6817","nonce":1135599916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":1135599916},{"type":"v1","addr":"172.21.15.42:6815","nonce":1135599916}]},"public_addr":"172.21.15.42:6811/1135599916","cluster_addr":"172.21.15.42:6813/1135599916","heartbeat_back_addr":"172.21.15.42:6817/1135599916","heartbeat_front_addr":"172.21.15.42:6815/1135599916","state":["exists","up"]},{"osd":4,"uuid":"3ca1aae3-c972-4fb9-ac86-236279c30d64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":2927511419},{"type":"v1","addr":"172.21.15.167:6809","nonce":2927511419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":2927511419},{"type":"v1","addr":"172.21.15.167:6811","nonce":2927511419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":2927511419},{"type":"v1","addr":"172.21.15.167:6815","nonce":2927511419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":2927511419},{"type":"v1","addr":"172.21.15.167:6813","nonce":2927511419}]},"public_addr":"172.21.15.167:6809/2927511419","cluster_addr":"172.21.15.167:6811/2927511419","heartbeat_back_addr":"172.21.15.167:6815/2927511419","heartbeat_front_addr":"172.21.15.167:6813/2927511419","state":["exists","up"]},{"osd":5,"uuid":"5d07f54b-0d07-4efe-9bcf-390755ce0258","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":2557145333},{"type":"v1","addr":"172.21.15.190:6809","nonce":2557145333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":2557145333},{"type":"v1","addr":"172.21.15.190:6811","nonce":2557145333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":2557145333},{"type":"v1","addr":"172.21.15.190:6815","nonce":2557145333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":2557145333},{"type":"v1","addr":"172.21.15.190:6813","nonce":2557145333}]},"public_addr":"172.21.15.190:6809/2557145333","cluster_addr":"172.21.15.190:6811/2557145333","heartbeat_back_addr":"172.21.15.190:6815/2557145333","heartbeat_front_addr":"172.21.15.190:6813/2557145333","state":["exists","up"]},{"osd":6,"uuid":"96ad64af-44ff-4197-97d4-156f0cb88e81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":3213526100},{"type":"v1","addr":"172.21.15.42:6819","nonce":3213526100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":3213526100},{"type":"v1","addr":"172.21.15.42:6821","nonce":3213526100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":3213526100},{"type":"v1","addr":"172.21.15.42:6825","nonce":3213526100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":3213526100},{"type":"v1","addr":"172.21.15.42:6823","nonce":3213526100}]},"public_addr":"172.21.15.42:6819/3213526100","cluster_addr":"172.21.15.42:6821/3213526100","heartbeat_back_addr":"172.21.15.42:6825/3213526100","heartbeat_front_addr":"172.21.15.42:6823/3213526100","state":["exists","up"]},{"osd":7,"uuid":"337a2112-eaf7-4e6b-86be-4e40d6ddf835","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":2962799814},{"type":"v1","addr":"172.21.15.167:6817","nonce":2962799814}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":2962799814},{"type":"v1","addr":"172.21.15.167:6819","nonce":2962799814}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":2962799814},{"type":"v1","addr":"172.21.15.167:6823","nonce":2962799814}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":2962799814},{"type":"v1","addr":"172.21.15.167:6821","nonce":2962799814}]},"public_addr":"172.21.15.167:6817/2962799814","cluster_addr":"172.21.15.167:6819/2962799814","heartbeat_back_addr":"172.21.15.167:6823/2962799814","heartbeat_front_addr":"172.21.15.167:6821/2962799814","state":["exists","up"]},{"osd":8,"uuid":"219a7224-4331-464b-ac86-c51054d51d91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":3242198843},{"type":"v1","addr":"172.21.15.190:6817","nonce":3242198843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":3242198843},{"type":"v1","addr":"172.21.15.190:6819","nonce":3242198843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":3242198843},{"type":"v1","addr":"172.21.15.190:6823","nonce":3242198843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":3242198843},{"type":"v1","addr":"172.21.15.190:6821","nonce":3242198843}]},"public_addr":"172.21.15.190:6817/3242198843","cluster_addr":"172.21.15.190:6819/3242198843","heartbeat_back_addr":"172.21.15.190:6823/3242198843","heartbeat_front_addr":"172.21.15.190:6821/3242198843","state":["exists","up"]},{"osd":9,"uuid":"2f2f5177-a009-40d9-991c-76aec8a5cba6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":1123536325},{"type":"v1","addr":"172.21.15.42:6827","nonce":1123536325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":1123536325},{"type":"v1","addr":"172.21.15.42:6829","nonce":1123536325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":1123536325},{"type":"v1","addr":"172.21.15.42:6833","nonce":1123536325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":1123536325},{"type":"v1","addr":"172.21.15.42:6831","nonce":1123536325}]},"public_addr":"172.21.15.42:6827/1123536325","cluster_addr":"172.21.15.42:6829/1123536325","heartbeat_back_addr":"172.21.15.42:6833/1123536325","heartbeat_front_addr":"172.21.15.42:6831/1123536325","state":["exists","up"]},{"osd":10,"uuid":"96ef5058-2518-4862-a165-17666b33a3e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":3451198319},{"type":"v1","addr":"172.21.15.167:6825","nonce":3451198319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":3451198319},{"type":"v1","addr":"172.21.15.167:6827","nonce":3451198319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":3451198319},{"type":"v1","addr":"172.21.15.167:6831","nonce":3451198319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":3451198319},{"type":"v1","addr":"172.21.15.167:6829","nonce":3451198319}]},"public_addr":"172.21.15.167:6825/3451198319","cluster_addr":"172.21.15.167:6827/3451198319","heartbeat_back_addr":"172.21.15.167:6831/3451198319","heartbeat_front_addr":"172.21.15.167:6829/3451198319","state":["exists","up"]},{"osd":11,"uuid":"a4eb9bd3-3faa-4c81-ba6d-c2084b72c9ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":380064457},{"type":"v1","addr":"172.21.15.190:6825","nonce":380064457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":380064457},{"type":"v1","addr":"172.21.15.190:6827","nonce":380064457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":380064457},{"type":"v1","addr":"172.21.15.190:6831","nonce":380064457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":380064457},{"type":"v1","addr":"172.21.15.190:6829","nonce":380064457}]},"public_addr":"172.21.15.190:6825/380064457","cluster_addr":"172.21.15.190:6827/380064457","heartbeat_back_addr":"172.21.15.190:6831/380064457","heartbeat_front_addr":"172.21.15.190:6829/380064457","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-30T08:52:36.511448+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-30T08:53:00.027277+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-30T08:53:23.770245+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-30T08:53:49.322226+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-30T08:54:14.350234+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-30T08:54:39.491793+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-30T08:55:04.870596+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-30T08:55:30.314150+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-30T08:55:54.483843+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-30T08:56:20.586115+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-30T08:56:45.666783+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-30T08:57:10.718387+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"2.1d","mappings":[{"from":2,"to":1}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"4.9","mappings":[{"from":5,"to":8}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.42:0/2691233260":"2024-08-31T08:50:44.821070+0000","172.21.15.42:0/1518053351":"2024-08-31T08:50:44.821070+0000","172.21.15.42:0/352001020":"2024-08-31T08:50:44.821070+0000","172.21.15.42:6801/1733768634":"2024-08-31T08:50:44.821070+0000","172.21.15.42:0/1000597888":"2024-08-31T08:50:18.561851+0000","172.21.15.42:6800/1733768634":"2024-08-31T08:50:44.821070+0000","172.21.15.42:6801/3538882081":"2024-08-31T08:50:18.561851+0000","172.21.15.42:6800/3538882081":"2024-08-31T08:50:18.561851+0000","172.21.15.42:0/145366508":"2024-08-31T08:50:18.561851+0000","172.21.15.42:0/3346355213":"2024-08-31T08:50:18.561851+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-30T09:00:25.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:25.723+0000 7f36261f9640 1 -- 172.21.15.42:0/3320797259 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f36080b4bb0 msgr2=0x7f36080b70c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:00:25.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:25.723+0000 7f36261f9640 1 --2- 172.21.15.42:0/3320797259 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f36080b4bb0 0x7f36080b70c0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f361000baa0 tx=0x7f3610005d90 comp rx=0 tx=0).stop 2024-08-30T09:00:25.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:25.723+0000 7f36261f9640 1 -- 172.21.15.42:0/3320797259 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f36201091f0 msgr2=0x7f362019aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:00:25.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:25.723+0000 7f36261f9640 1 --2- 172.21.15.42:0/3320797259 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f36201091f0 0x7f362019aef0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f361800b970 tx=0x7f361800be30 comp rx=0 tx=0).stop 2024-08-30T09:00:25.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:25.723+0000 7f36261f9640 1 -- 172.21.15.42:0/3320797259 shutdown_connections 2024-08-30T09:00:25.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:25.723+0000 7f36261f9640 1 --2- 172.21.15.42:0/3320797259 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f36080b4bb0 0x7f36080b70c0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:25.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:25.723+0000 7f36261f9640 1 --2- 172.21.15.42:0/3320797259 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f36201091f0 0x7f362019aef0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:25.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:25.723+0000 7f36261f9640 1 --2- 172.21.15.42:0/3320797259 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3620106890 0x7f362019a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:25.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:25.723+0000 7f36261f9640 1 --2- 172.21.15.42:0/3320797259 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3620101800 0x7f3620199cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:25.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:25.723+0000 7f36261f9640 1 -- 172.21.15.42:0/3320797259 >> 172.21.15.42:0/3320797259 conn(0x7f36200fd400 msgr2=0x7f3620107660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:00:25.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:25.723+0000 7f36261f9640 1 -- 172.21.15.42:0/3320797259 shutdown_connections 2024-08-30T09:00:25.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:25.723+0000 7f36261f9640 1 -- 172.21.15.42:0/3320797259 wait complete. 2024-08-30T09:00:25.737 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-08-30T08:53:27.297022+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-08-30T08:57:58.489964+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '73', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-08-30T08:58:01.064783+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '73', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-08-30T08:58:07.416534+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-08-30T09:00:25.737 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-08-30T09:00:25.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:00:25 smithi167 ceph-mon[22063]: from='client.? 172.21.15.42:0/1095114448' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} : dispatch 2024-08-30T09:00:25.853 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:25.849+0000 7f1b78762640 1 -- 172.21.15.42:0/3368972169 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1b70102b40 msgr2=0x7f1b70102f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:00:25.853 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:25.849+0000 7f1b78762640 1 --2- 172.21.15.42:0/3368972169 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1b70102b40 0x7f1b70102f50 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f1b6c009900 tx=0x7f1b6c02a410 comp rx=0 tx=0).stop 2024-08-30T09:00:25.853 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:25.849+0000 7f1b78762640 1 -- 172.21.15.42:0/3368972169 shutdown_connections 2024-08-30T09:00:25.854 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:25.849+0000 7f1b78762640 1 --2- 172.21.15.42:0/3368972169 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1b701034e0 0x7f1b70109e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:25.854 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:25.849+0000 7f1b78762640 1 --2- 172.21.15.42:0/3368972169 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1b70102b40 0x7f1b70102f50 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:25.854 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:25.850+0000 7f1b78762640 1 --2- 172.21.15.42:0/3368972169 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1b70101ac0 0x7f1b70101f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:25.854 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:25.850+0000 7f1b78762640 1 -- 172.21.15.42:0/3368972169 >> 172.21.15.42:0/3368972169 conn(0x7f1b700fd400 msgr2=0x7f1b700ff8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:00:25.854 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:25.850+0000 7f1b78762640 1 -- 172.21.15.42:0/3368972169 shutdown_connections 2024-08-30T09:00:25.854 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:25.850+0000 7f1b78762640 1 -- 172.21.15.42:0/3368972169 wait complete. 2024-08-30T09:00:25.854 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:25.850+0000 7f1b78762640 1 Processor -- start 2024-08-30T09:00:25.854 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:25.850+0000 7f1b78762640 1 -- start start 2024-08-30T09:00:25.854 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:25.850+0000 7f1b78762640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1b70101ac0 0x7f1b70199ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:25.854 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:25.850+0000 7f1b78762640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1b70102b40 0x7f1b7019a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:25.854 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:25.850+0000 7f1b78762640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1b701034e0 0x7f1b7019af00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:25.854 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:25.850+0000 7f1b78762640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f1b7010f0c0 con 0x7f1b701034e0 2024-08-30T09:00:25.854 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:25.850+0000 7f1b78762640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f1b7010ef40 con 0x7f1b70102b40 2024-08-30T09:00:25.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:25.850+0000 7f1b78762640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f1b7010f240 con 0x7f1b70101ac0 2024-08-30T09:00:25.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:25.850+0000 7f1b76cff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1b701034e0 0x7f1b7019af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:25.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:25.851+0000 7f1b76cff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1b701034e0 0x7f1b7019af00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38468/0 (socket says 172.21.15.42:38468) 2024-08-30T09:00:25.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:25.851+0000 7f1b76cff640 1 -- 172.21.15.42:0/287032192 learned_addr learned my addr 172.21.15.42:0/287032192 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:00:25.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:25.851+0000 7f1b75cfd640 1 --2- 172.21.15.42:0/287032192 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1b70102b40 0x7f1b7019a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:25.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:25.851+0000 7f1b764fe640 1 --2- 172.21.15.42:0/287032192 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1b70101ac0 0x7f1b70199ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:25.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:25.851+0000 7f1b76cff640 1 -- 172.21.15.42:0/287032192 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1b70101ac0 msgr2=0x7f1b70199ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:00:25.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:25.851+0000 7f1b76cff640 1 --2- 172.21.15.42:0/287032192 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1b70101ac0 0x7f1b70199ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:25.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:25.851+0000 7f1b76cff640 1 -- 172.21.15.42:0/287032192 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1b70102b40 msgr2=0x7f1b7019a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:00:25.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:25.851+0000 7f1b76cff640 1 --2- 172.21.15.42:0/287032192 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1b70102b40 0x7f1b7019a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:25.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:25.851+0000 7f1b76cff640 1 -- 172.21.15.42:0/287032192 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b7019b690 con 0x7f1b701034e0 2024-08-30T09:00:25.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:25.851+0000 7f1b76cff640 1 --2- 172.21.15.42:0/287032192 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1b701034e0 0x7f1b7019af00 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f1b68009fd0 tx=0x7f1b6800b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:00:25.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:25.851+0000 7f1b677fe640 1 -- 172.21.15.42:0/287032192 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b68017070 con 0x7f1b701034e0 2024-08-30T09:00:25.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:25.851+0000 7f1b78762640 1 -- 172.21.15.42:0/287032192 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b7019b980 con 0x7f1b701034e0 2024-08-30T09:00:25.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:25.851+0000 7f1b677fe640 1 -- 172.21.15.42:0/287032192 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b68003e20 con 0x7f1b701034e0 2024-08-30T09:00:25.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:25.851+0000 7f1b677fe640 1 -- 172.21.15.42:0/287032192 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b68010460 con 0x7f1b701034e0 2024-08-30T09:00:25.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:25.851+0000 7f1b78762640 1 -- 172.21.15.42:0/287032192 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b701a2100 con 0x7f1b701034e0 2024-08-30T09:00:25.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:25.852+0000 7f1b78762640 1 -- 172.21.15.42:0/287032192 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b70101f30 con 0x7f1b701034e0 2024-08-30T09:00:25.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:25.854+0000 7f1b677fe640 1 -- 172.21.15.42:0/287032192 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7f1b68003970 con 0x7f1b701034e0 2024-08-30T09:00:25.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:25.854+0000 7f1b677fe640 1 --2- 172.21.15.42:0/287032192 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f1b5c0b4b30 0x7f1b5c0b7040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:25.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:25.854+0000 7f1b764fe640 1 --2- 172.21.15.42:0/287032192 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f1b5c0b4b30 0x7f1b5c0b7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:25.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:25.855+0000 7f1b677fe640 1 -- 172.21.15.42:0/287032192 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f1b680e5260 con 0x7f1b701034e0 2024-08-30T09:00:25.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:25.855+0000 7f1b764fe640 1 --2- 172.21.15.42:0/287032192 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f1b5c0b4b30 0x7f1b5c0b7040 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f1b6000ba20 tx=0x7f1b60005ca0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:00:25.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:25.857+0000 7f1b677fe640 1 -- 172.21.15.42:0/287032192 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f1b680ac000 con 0x7f1b701034e0 2024-08-30T09:00:26.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:26.046+0000 7f1b78762640 1 -- 172.21.15.42:0/287032192 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f1b7002e7a0 con 0x7f1b701034e0 2024-08-30T09:00:26.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:26.046+0000 7f1b677fe640 1 -- 172.21.15.42:0/287032192 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v83) ==== 93+0+10 (secure 0 0 0) 0x7f1b680ab900 con 0x7f1b701034e0 2024-08-30T09:00:26.048 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 1 2024-08-30T09:00:26.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:26.048+0000 7f1b78762640 1 -- 172.21.15.42:0/287032192 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f1b5c0b4b30 msgr2=0x7f1b5c0b7040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:00:26.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:26.048+0000 7f1b78762640 1 --2- 172.21.15.42:0/287032192 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f1b5c0b4b30 0x7f1b5c0b7040 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f1b6000ba20 tx=0x7f1b60005ca0 comp rx=0 tx=0).stop 2024-08-30T09:00:26.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:26.048+0000 7f1b78762640 1 -- 172.21.15.42:0/287032192 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1b701034e0 msgr2=0x7f1b7019af00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:00:26.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:26.048+0000 7f1b78762640 1 --2- 172.21.15.42:0/287032192 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1b701034e0 0x7f1b7019af00 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f1b68009fd0 tx=0x7f1b6800b260 comp rx=0 tx=0).stop 2024-08-30T09:00:26.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:26.048+0000 7f1b78762640 1 -- 172.21.15.42:0/287032192 shutdown_connections 2024-08-30T09:00:26.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:26.048+0000 7f1b78762640 1 --2- 172.21.15.42:0/287032192 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f1b5c0b4b30 0x7f1b5c0b7040 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:26.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:26.048+0000 7f1b78762640 1 --2- 172.21.15.42:0/287032192 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1b701034e0 0x7f1b7019af00 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:26.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:26.048+0000 7f1b78762640 1 --2- 172.21.15.42:0/287032192 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1b70102b40 0x7f1b7019a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:26.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:26.048+0000 7f1b78762640 1 --2- 172.21.15.42:0/287032192 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1b70101ac0 0x7f1b70199ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:26.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:26.048+0000 7f1b78762640 1 -- 172.21.15.42:0/287032192 >> 172.21.15.42:0/287032192 conn(0x7f1b700fd400 msgr2=0x7f1b70105e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:00:26.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:26.048+0000 7f1b78762640 1 -- 172.21.15.42:0/287032192 shutdown_connections 2024-08-30T09:00:26.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:26.048+0000 7f1b78762640 1 -- 172.21.15.42:0/287032192 wait complete. 2024-08-30T09:00:26.062 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-08-30T09:00:26.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:26.174+0000 7f9c7af77640 1 -- 172.21.15.42:0/2756247743 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9c74103e30 msgr2=0x7f9c7410a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:00:26.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:26.174+0000 7f9c7af77640 1 --2- 172.21.15.42:0/2756247743 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9c74103e30 0x7f9c7410a8c0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f9c6c009900 tx=0x7f9c6c02a410 comp rx=0 tx=0).stop 2024-08-30T09:00:26.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:26.174+0000 7f9c7af77640 1 -- 172.21.15.42:0/2756247743 shutdown_connections 2024-08-30T09:00:26.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:26.174+0000 7f9c7af77640 1 --2- 172.21.15.42:0/2756247743 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9c74103220 0x7f9c741036f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:26.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:26.174+0000 7f9c7af77640 1 --2- 172.21.15.42:0/2756247743 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9c74103e30 0x7f9c7410a8c0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:26.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:26.174+0000 7f9c7af77640 1 --2- 172.21.15.42:0/2756247743 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9c74102840 0x7f9c74102c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:26.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:26.174+0000 7f9c7af77640 1 -- 172.21.15.42:0/2756247743 >> 172.21.15.42:0/2756247743 conn(0x7f9c740fd430 msgr2=0x7f9c740ff8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:00:26.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:26.174+0000 7f9c7af77640 1 -- 172.21.15.42:0/2756247743 shutdown_connections 2024-08-30T09:00:26.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:26.174+0000 7f9c7af77640 1 -- 172.21.15.42:0/2756247743 wait complete. 2024-08-30T09:00:26.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:26.174+0000 7f9c7af77640 1 Processor -- start 2024-08-30T09:00:26.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:26.175+0000 7f9c7af77640 1 -- start start 2024-08-30T09:00:26.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:26.175+0000 7f9c7af77640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9c74102840 0x7f9c74199c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:26.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:26.175+0000 7f9c7af77640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9c74103220 0x7f9c7419a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:26.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:26.175+0000 7f9c7af77640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9c74103e30 0x7f9c7419ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:26.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:26.175+0000 7f9c7af77640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f9c7410dfa0 con 0x7f9c74103e30 2024-08-30T09:00:26.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:26.175+0000 7f9c7af77640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f9c7410de20 con 0x7f9c74103220 2024-08-30T09:00:26.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:26.175+0000 7f9c7af77640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f9c7410e120 con 0x7f9c74102840 2024-08-30T09:00:26.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:26.175+0000 7f9c79514640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9c74103e30 0x7f9c7419ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:26.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:26.175+0000 7f9c79514640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9c74103e30 0x7f9c7419ae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38488/0 (socket says 172.21.15.42:38488) 2024-08-30T09:00:26.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:26.175+0000 7f9c79514640 1 -- 172.21.15.42:0/2883630127 learned_addr learned my addr 172.21.15.42:0/2883630127 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:00:26.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:26.175+0000 7f9c78d13640 1 --2- 172.21.15.42:0/2883630127 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9c74102840 0x7f9c74199c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:26.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:26.175+0000 7f9c73fff640 1 --2- 172.21.15.42:0/2883630127 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9c74103220 0x7f9c7419a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:26.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:26.175+0000 7f9c79514640 1 -- 172.21.15.42:0/2883630127 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9c74102840 msgr2=0x7f9c74199c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:00:26.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:26.175+0000 7f9c79514640 1 --2- 172.21.15.42:0/2883630127 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9c74102840 0x7f9c74199c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:26.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:26.175+0000 7f9c79514640 1 -- 172.21.15.42:0/2883630127 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9c74103220 msgr2=0x7f9c7419a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:00:26.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:26.175+0000 7f9c79514640 1 --2- 172.21.15.42:0/2883630127 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9c74103220 0x7f9c7419a2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:26.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:26.175+0000 7f9c79514640 1 -- 172.21.15.42:0/2883630127 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c7419b690 con 0x7f9c74103e30 2024-08-30T09:00:26.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:26.175+0000 7f9c78d13640 1 --2- 172.21.15.42:0/2883630127 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9c74102840 0x7f9c74199c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-30T09:00:26.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:26.175+0000 7f9c73fff640 1 --2- 172.21.15.42:0/2883630127 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9c74103220 0x7f9c7419a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-30T09:00:26.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:26.175+0000 7f9c79514640 1 --2- 172.21.15.42:0/2883630127 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9c74103e30 0x7f9c7419ae70 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f9c6c009230 tx=0x7f9c6c031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:00:26.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:26.175+0000 7f9c71ffb640 1 -- 172.21.15.42:0/2883630127 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c6c03f030 con 0x7f9c74103e30 2024-08-30T09:00:26.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:26.176+0000 7f9c71ffb640 1 -- 172.21.15.42:0/2883630127 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9c6c003aa0 con 0x7f9c74103e30 2024-08-30T09:00:26.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:26.176+0000 7f9c7af77640 1 -- 172.21.15.42:0/2883630127 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c7419b920 con 0x7f9c74103e30 2024-08-30T09:00:26.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:26.176+0000 7f9c71ffb640 1 -- 172.21.15.42:0/2883630127 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c6c035070 con 0x7f9c74103e30 2024-08-30T09:00:26.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:26.176+0000 7f9c7af77640 1 -- 172.21.15.42:0/2883630127 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c7419bc30 con 0x7f9c74103e30 2024-08-30T09:00:26.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:26.177+0000 7f9c7af77640 1 -- 172.21.15.42:0/2883630127 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c741a2070 con 0x7f9c74103e30 2024-08-30T09:00:26.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:26.179+0000 7f9c71ffb640 1 -- 172.21.15.42:0/2883630127 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7f9c6c042030 con 0x7f9c74103e30 2024-08-30T09:00:26.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:26.180+0000 7f9c71ffb640 1 --2- 172.21.15.42:0/2883630127 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f9c600b4b30 0x7f9c600b7040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:26.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:26.180+0000 7f9c78d13640 1 --2- 172.21.15.42:0/2883630127 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f9c600b4b30 0x7f9c600b7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:26.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:26.180+0000 7f9c71ffb640 1 -- 172.21.15.42:0/2883630127 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f9c6c105ed0 con 0x7f9c74103e30 2024-08-30T09:00:26.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:26.180+0000 7f9c78d13640 1 --2- 172.21.15.42:0/2883630127 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f9c600b4b30 0x7f9c600b7040 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f9c64005fd0 tx=0x7f9c64009520 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:00:26.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:26.183+0000 7f9c71ffb640 1 -- 172.21.15.42:0/2883630127 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f9c6c0ccc70 con 0x7f9c74103e30 2024-08-30T09:00:26.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:26.368+0000 7f9c7af77640 1 -- 172.21.15.42:0/2883630127 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f9c741a2240 con 0x7f9c74103e30 2024-08-30T09:00:26.370 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:26.368+0000 7f9c71ffb640 1 -- 172.21.15.42:0/2883630127 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v83) ==== 104+0+11 (secure 0 0 0) 0x7f9c6c0d51b0 con 0x7f9c74103e30 2024-08-30T09:00:26.370 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 32 2024-08-30T09:00:26.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:26.370+0000 7f9c7af77640 1 -- 172.21.15.42:0/2883630127 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f9c600b4b30 msgr2=0x7f9c600b7040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:00:26.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:26.370+0000 7f9c7af77640 1 --2- 172.21.15.42:0/2883630127 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f9c600b4b30 0x7f9c600b7040 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f9c64005fd0 tx=0x7f9c64009520 comp rx=0 tx=0).stop 2024-08-30T09:00:26.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:26.370+0000 7f9c7af77640 1 -- 172.21.15.42:0/2883630127 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9c74103e30 msgr2=0x7f9c7419ae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:00:26.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:26.370+0000 7f9c7af77640 1 --2- 172.21.15.42:0/2883630127 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9c74103e30 0x7f9c7419ae70 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f9c6c009230 tx=0x7f9c6c031a50 comp rx=0 tx=0).stop 2024-08-30T09:00:26.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:26.370+0000 7f9c7af77640 1 -- 172.21.15.42:0/2883630127 shutdown_connections 2024-08-30T09:00:26.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:26.370+0000 7f9c7af77640 1 --2- 172.21.15.42:0/2883630127 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f9c600b4b30 0x7f9c600b7040 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:26.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:26.370+0000 7f9c7af77640 1 --2- 172.21.15.42:0/2883630127 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9c74103e30 0x7f9c7419ae70 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:26.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:26.370+0000 7f9c7af77640 1 --2- 172.21.15.42:0/2883630127 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9c74103220 0x7f9c7419a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:26.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:26.370+0000 7f9c7af77640 1 --2- 172.21.15.42:0/2883630127 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9c74102840 0x7f9c74199c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:26.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:26.370+0000 7f9c7af77640 1 -- 172.21.15.42:0/2883630127 >> 172.21.15.42:0/2883630127 conn(0x7f9c740fd430 msgr2=0x7f9c740fec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:00:26.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:26.370+0000 7f9c7af77640 1 -- 172.21.15.42:0/2883630127 shutdown_connections 2024-08-30T09:00:26.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:26.370+0000 7f9c7af77640 1 -- 172.21.15.42:0/2883630127 wait complete. 2024-08-30T09:00:26.384 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-08-30T09:00:26.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:26.498+0000 7f75ea939640 1 -- 172.21.15.42:0/4019782482 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f75e406c170 msgr2=0x7f75e4106080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:00:26.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:26.498+0000 7f75ea939640 1 --2- 172.21.15.42:0/4019782482 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f75e406c170 0x7f75e4106080 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f75d4009900 tx=0x7f75d402a410 comp rx=0 tx=0).stop 2024-08-30T09:00:26.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:26.498+0000 7f75ea939640 1 -- 172.21.15.42:0/4019782482 shutdown_connections 2024-08-30T09:00:26.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:26.498+0000 7f75ea939640 1 --2- 172.21.15.42:0/4019782482 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f75e41065c0 0x7f75e4108a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:26.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:26.498+0000 7f75ea939640 1 --2- 172.21.15.42:0/4019782482 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f75e4109180 0x7f75e410b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:26.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:26.498+0000 7f75ea939640 1 --2- 172.21.15.42:0/4019782482 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f75e406c170 0x7f75e4106080 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:26.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:26.498+0000 7f75ea939640 1 -- 172.21.15.42:0/4019782482 >> 172.21.15.42:0/4019782482 conn(0x7f75e40fdc80 msgr2=0x7f75e4100130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:00:26.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:26.499+0000 7f75ea939640 1 -- 172.21.15.42:0/4019782482 shutdown_connections 2024-08-30T09:00:26.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:26.499+0000 7f75ea939640 1 -- 172.21.15.42:0/4019782482 wait complete. 2024-08-30T09:00:26.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:26.499+0000 7f75ea939640 1 Processor -- start 2024-08-30T09:00:26.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:26.499+0000 7f75ea939640 1 -- start start 2024-08-30T09:00:26.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:26.500+0000 7f75ea939640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f75e406c170 0x7f75e4199d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:26.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:26.500+0000 7f75ea939640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f75e41065c0 0x7f75e419a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:26.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:26.500+0000 7f75ea939640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f75e4109180 0x7f75e419af80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:26.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:26.500+0000 7f75ea939640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f75e410f0a0 con 0x7f75e406c170 2024-08-30T09:00:26.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:26.500+0000 7f75ea939640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f75e410ef20 con 0x7f75e41065c0 2024-08-30T09:00:26.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:26.500+0000 7f75ea939640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f75e410f220 con 0x7f75e4109180 2024-08-30T09:00:26.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:26.500+0000 7f75e3fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f75e406c170 0x7f75e4199d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:26.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:26.500+0000 7f75e3fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f75e406c170 0x7f75e4199d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38508/0 (socket says 172.21.15.42:38508) 2024-08-30T09:00:26.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:26.500+0000 7f75e3fff640 1 -- 172.21.15.42:0/378650973 learned_addr learned my addr 172.21.15.42:0/378650973 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:00:26.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:26.500+0000 7f75e37fe640 1 --2- 172.21.15.42:0/378650973 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f75e41065c0 0x7f75e419a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:26.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:26.500+0000 7f75e8ed6640 1 --2- 172.21.15.42:0/378650973 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f75e4109180 0x7f75e419af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:26.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:26.500+0000 7f75e3fff640 1 -- 172.21.15.42:0/378650973 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f75e4109180 msgr2=0x7f75e419af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:00:26.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:26.500+0000 7f75e3fff640 1 --2- 172.21.15.42:0/378650973 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f75e4109180 0x7f75e419af80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:26.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:26.500+0000 7f75e3fff640 1 -- 172.21.15.42:0/378650973 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f75e41065c0 msgr2=0x7f75e419a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:00:26.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:26.500+0000 7f75e3fff640 1 --2- 172.21.15.42:0/378650973 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f75e41065c0 0x7f75e419a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:26.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:26.500+0000 7f75e3fff640 1 -- 172.21.15.42:0/378650973 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75e419b7a0 con 0x7f75e406c170 2024-08-30T09:00:26.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:26.500+0000 7f75e3fff640 1 --2- 172.21.15.42:0/378650973 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f75e406c170 0x7f75e4199d60 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f75d4006010 tx=0x7f75d4004340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:00:26.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:26.500+0000 7f75e17fa640 1 -- 172.21.15.42:0/378650973 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75d403f030 con 0x7f75e406c170 2024-08-30T09:00:26.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:26.500+0000 7f75ea939640 1 -- 172.21.15.42:0/378650973 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75e419ba30 con 0x7f75e406c170 2024-08-30T09:00:26.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:26.500+0000 7f75e17fa640 1 -- 172.21.15.42:0/378650973 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f75d4003cb0 con 0x7f75e406c170 2024-08-30T09:00:26.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:26.500+0000 7f75e17fa640 1 -- 172.21.15.42:0/378650973 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75d4035070 con 0x7f75e406c170 2024-08-30T09:00:26.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:26.501+0000 7f75ea939640 1 -- 172.21.15.42:0/378650973 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75e419bd40 con 0x7f75e406c170 2024-08-30T09:00:26.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:26.502+0000 7f75ea939640 1 -- 172.21.15.42:0/378650973 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75e40ff7a0 con 0x7f75e406c170 2024-08-30T09:00:26.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:26.503+0000 7f75e17fa640 1 -- 172.21.15.42:0/378650973 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7f75d4042030 con 0x7f75e406c170 2024-08-30T09:00:26.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:26.504+0000 7f75e17fa640 1 --2- 172.21.15.42:0/378650973 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f75cc0b4b30 0x7f75cc0b7040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:26.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:26.504+0000 7f75e37fe640 1 --2- 172.21.15.42:0/378650973 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f75cc0b4b30 0x7f75cc0b7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:26.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:26.505+0000 7f75e17fa640 1 -- 172.21.15.42:0/378650973 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f75d4106120 con 0x7f75e406c170 2024-08-30T09:00:26.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:26.505+0000 7f75e37fe640 1 --2- 172.21.15.42:0/378650973 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f75cc0b4b30 0x7f75cc0b7040 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f75e406c720 tx=0x7f75d800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:00:26.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:26.508+0000 7f75e17fa640 1 -- 172.21.15.42:0/378650973 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f75d40ccec0 con 0x7f75e406c170 2024-08-30T09:00:26.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:00:26 smithi042 ceph-mon[18652]: pgmap v363: 129 pgs: 129 active+clean; 644 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 5 op/s 2024-08-30T09:00:26.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:00:26 smithi042 ceph-mon[18652]: from='client.? 172.21.15.42:0/1095114448' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-08-30T09:00:26.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:00:26 smithi042 ceph-mon[18652]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-30T09:00:26.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:00:26 smithi042 ceph-mon[18652]: from='client.? 172.21.15.42:0/3320797259' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-30T09:00:26.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:00:26 smithi042 ceph-mon[18652]: from='client.? 172.21.15.42:0/287032192' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-08-30T09:00:26.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:00:26 smithi190 ceph-mon[22001]: pgmap v363: 129 pgs: 129 active+clean; 644 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 5 op/s 2024-08-30T09:00:26.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:00:26 smithi190 ceph-mon[22001]: from='client.? 172.21.15.42:0/1095114448' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-08-30T09:00:26.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:00:26 smithi190 ceph-mon[22001]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-30T09:00:26.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:00:26 smithi190 ceph-mon[22001]: from='client.? 172.21.15.42:0/3320797259' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-30T09:00:26.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:00:26 smithi190 ceph-mon[22001]: from='client.? 172.21.15.42:0/287032192' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-08-30T09:00:26.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:26.694+0000 7f75ea939640 1 -- 172.21.15.42:0/378650973 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f75e402e680 con 0x7f75e406c170 2024-08-30T09:00:26.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:26.694+0000 7f75e17fa640 1 -- 172.21.15.42:0/378650973 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v83) ==== 100+0+11 (secure 0 0 0) 0x7f75d40d5400 con 0x7f75e406c170 2024-08-30T09:00:26.696 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 64 2024-08-30T09:00:26.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:26.696+0000 7f75ea939640 1 -- 172.21.15.42:0/378650973 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f75cc0b4b30 msgr2=0x7f75cc0b7040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:00:26.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:26.696+0000 7f75ea939640 1 --2- 172.21.15.42:0/378650973 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f75cc0b4b30 0x7f75cc0b7040 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f75e406c720 tx=0x7f75d800b390 comp rx=0 tx=0).stop 2024-08-30T09:00:26.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:26.696+0000 7f75ea939640 1 -- 172.21.15.42:0/378650973 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f75e406c170 msgr2=0x7f75e4199d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:00:26.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:26.696+0000 7f75ea939640 1 --2- 172.21.15.42:0/378650973 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f75e406c170 0x7f75e4199d60 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f75d4006010 tx=0x7f75d4004340 comp rx=0 tx=0).stop 2024-08-30T09:00:26.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:26.696+0000 7f75ea939640 1 -- 172.21.15.42:0/378650973 shutdown_connections 2024-08-30T09:00:26.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:26.696+0000 7f75ea939640 1 --2- 172.21.15.42:0/378650973 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f75cc0b4b30 0x7f75cc0b7040 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:26.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:26.696+0000 7f75ea939640 1 --2- 172.21.15.42:0/378650973 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f75e4109180 0x7f75e419af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:26.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:26.696+0000 7f75ea939640 1 --2- 172.21.15.42:0/378650973 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f75e41065c0 0x7f75e419a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:26.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:26.696+0000 7f75ea939640 1 --2- 172.21.15.42:0/378650973 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f75e406c170 0x7f75e4199d60 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:26.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:26.696+0000 7f75ea939640 1 -- 172.21.15.42:0/378650973 >> 172.21.15.42:0/378650973 conn(0x7f75e40fdc80 msgr2=0x7f75e4104b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:00:26.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:26.697+0000 7f75ea939640 1 -- 172.21.15.42:0/378650973 shutdown_connections 2024-08-30T09:00:26.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:26.697+0000 7f75ea939640 1 -- 172.21.15.42:0/378650973 wait complete. 2024-08-30T09:00:26.710 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-08-30T09:00:26.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:26.822+0000 7f0d73071640 1 -- 172.21.15.42:0/4145437005 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0d6c1029e0 msgr2=0x7f0d6c104e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:00:26.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:26.822+0000 7f0d73071640 1 --2- 172.21.15.42:0/4145437005 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0d6c1029e0 0x7f0d6c104e20 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f0d60009900 tx=0x7f0d6002a410 comp rx=0 tx=0).stop 2024-08-30T09:00:26.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:26.822+0000 7f0d73071640 1 -- 172.21.15.42:0/4145437005 shutdown_connections 2024-08-30T09:00:26.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:26.822+0000 7f0d73071640 1 --2- 172.21.15.42:0/4145437005 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0d6c105460 0x7f0d6c107940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:26.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:26.822+0000 7f0d73071640 1 --2- 172.21.15.42:0/4145437005 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0d6c108080 0x7f0d6c10a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:26.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:26.822+0000 7f0d73071640 1 --2- 172.21.15.42:0/4145437005 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0d6c1029e0 0x7f0d6c104e20 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:26.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:26.822+0000 7f0d73071640 1 -- 172.21.15.42:0/4145437005 >> 172.21.15.42:0/4145437005 conn(0x7f0d6c0fd430 msgr2=0x7f0d6c0ff8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:00:26.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:26.823+0000 7f0d73071640 1 -- 172.21.15.42:0/4145437005 shutdown_connections 2024-08-30T09:00:26.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:26.823+0000 7f0d73071640 1 -- 172.21.15.42:0/4145437005 wait complete. 2024-08-30T09:00:26.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:26.823+0000 7f0d73071640 1 Processor -- start 2024-08-30T09:00:26.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:26.823+0000 7f0d73071640 1 -- start start 2024-08-30T09:00:26.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:26.823+0000 7f0d73071640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0d6c1029e0 0x7f0d6c198bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:26.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:26.823+0000 7f0d73071640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0d6c105460 0x7f0d6c199230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:26.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:26.823+0000 7f0d73071640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0d6c108080 0x7f0d6c199de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:26.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:26.823+0000 7f0d73071640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f0d6c10dff0 con 0x7f0d6c108080 2024-08-30T09:00:26.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:26.823+0000 7f0d73071640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f0d6c10de70 con 0x7f0d6c105460 2024-08-30T09:00:26.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:26.823+0000 7f0d73071640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f0d6c10e170 con 0x7f0d6c1029e0 2024-08-30T09:00:26.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:26.824+0000 7f0d7160e640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0d6c108080 0x7f0d6c199de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:26.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:26.824+0000 7f0d7160e640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0d6c108080 0x7f0d6c199de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38518/0 (socket says 172.21.15.42:38518) 2024-08-30T09:00:26.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:26.824+0000 7f0d7160e640 1 -- 172.21.15.42:0/3297294938 learned_addr learned my addr 172.21.15.42:0/3297294938 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:00:26.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:26.824+0000 7f0d6bfff640 1 --2- 172.21.15.42:0/3297294938 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0d6c105460 0x7f0d6c199230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:26.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:26.824+0000 7f0d70e0d640 1 --2- 172.21.15.42:0/3297294938 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0d6c1029e0 0x7f0d6c198bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:26.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:26.824+0000 7f0d7160e640 1 -- 172.21.15.42:0/3297294938 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0d6c1029e0 msgr2=0x7f0d6c198bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:00:26.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:26.824+0000 7f0d7160e640 1 --2- 172.21.15.42:0/3297294938 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0d6c1029e0 0x7f0d6c198bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:26.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:26.824+0000 7f0d7160e640 1 -- 172.21.15.42:0/3297294938 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0d6c105460 msgr2=0x7f0d6c199230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:00:26.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:26.824+0000 7f0d7160e640 1 --2- 172.21.15.42:0/3297294938 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0d6c105460 0x7f0d6c199230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:26.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:26.824+0000 7f0d7160e640 1 -- 172.21.15.42:0/3297294938 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d6c19a600 con 0x7f0d6c108080 2024-08-30T09:00:26.829 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:26.824+0000 7f0d7160e640 1 --2- 172.21.15.42:0/3297294938 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0d6c108080 0x7f0d6c199de0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f0d64009fd0 tx=0x7f0d6400b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:00:26.829 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:26.824+0000 7f0d69ffb640 1 -- 172.21.15.42:0/3297294938 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d64017070 con 0x7f0d6c108080 2024-08-30T09:00:26.829 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:26.824+0000 7f0d69ffb640 1 -- 172.21.15.42:0/3297294938 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d64003e20 con 0x7f0d6c108080 2024-08-30T09:00:26.829 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:26.824+0000 7f0d73071640 1 -- 172.21.15.42:0/3297294938 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d6c19a8f0 con 0x7f0d6c108080 2024-08-30T09:00:26.829 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:26.824+0000 7f0d69ffb640 1 -- 172.21.15.42:0/3297294938 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d64010460 con 0x7f0d6c108080 2024-08-30T09:00:26.829 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:26.825+0000 7f0d73071640 1 -- 172.21.15.42:0/3297294938 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d6c1a0fe0 con 0x7f0d6c108080 2024-08-30T09:00:26.830 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:00:26 smithi167 ceph-mon[22063]: pgmap v363: 129 pgs: 129 active+clean; 644 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 5 op/s 2024-08-30T09:00:26.830 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:00:26 smithi167 ceph-mon[22063]: from='client.? 172.21.15.42:0/1095114448' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-08-30T09:00:26.830 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:00:26 smithi167 ceph-mon[22063]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-30T09:00:26.830 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:00:26 smithi167 ceph-mon[22063]: from='client.? 172.21.15.42:0/3320797259' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-30T09:00:26.830 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:00:26 smithi167 ceph-mon[22063]: from='client.? 172.21.15.42:0/287032192' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-08-30T09:00:26.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:26.826+0000 7f0d73071640 1 -- 172.21.15.42:0/3297294938 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d6c0fef50 con 0x7f0d6c108080 2024-08-30T09:00:26.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:26.827+0000 7f0d69ffb640 1 -- 172.21.15.42:0/3297294938 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7f0d64003970 con 0x7f0d6c108080 2024-08-30T09:00:26.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:26.828+0000 7f0d69ffb640 1 --2- 172.21.15.42:0/3297294938 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f0d5c0b4bb0 0x7f0d5c0b70c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:26.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:26.828+0000 7f0d69ffb640 1 -- 172.21.15.42:0/3297294938 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f0d640e5210 con 0x7f0d6c108080 2024-08-30T09:00:26.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:26.828+0000 7f0d70e0d640 1 --2- 172.21.15.42:0/3297294938 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f0d5c0b4bb0 0x7f0d5c0b70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:26.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:26.829+0000 7f0d70e0d640 1 --2- 172.21.15.42:0/3297294938 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f0d5c0b4bb0 0x7f0d5c0b70c0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f0d60000c00 tx=0x7f0d60031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:00:26.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:26.832+0000 7f0d69ffb640 1 -- 172.21.15.42:0/3297294938 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f0d640abf50 con 0x7f0d6c108080 2024-08-30T09:00:27.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:27.019+0000 7f0d73071640 1 -- 172.21.15.42:0/3297294938 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f0d6c063d60 con 0x7f0d6c108080 2024-08-30T09:00:27.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:27.019+0000 7f0d69ffb640 1 -- 172.21.15.42:0/3297294938 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v83) ==== 103+0+11 (secure 0 0 0) 0x7f0d640ab850 con 0x7f0d6c108080 2024-08-30T09:00:27.021 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 32 2024-08-30T09:00:27.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:27.021+0000 7f0d73071640 1 -- 172.21.15.42:0/3297294938 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f0d5c0b4bb0 msgr2=0x7f0d5c0b70c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:00:27.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:27.021+0000 7f0d73071640 1 --2- 172.21.15.42:0/3297294938 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f0d5c0b4bb0 0x7f0d5c0b70c0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f0d60000c00 tx=0x7f0d60031400 comp rx=0 tx=0).stop 2024-08-30T09:00:27.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:27.021+0000 7f0d73071640 1 -- 172.21.15.42:0/3297294938 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0d6c108080 msgr2=0x7f0d6c199de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:00:27.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:27.021+0000 7f0d73071640 1 --2- 172.21.15.42:0/3297294938 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0d6c108080 0x7f0d6c199de0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f0d64009fd0 tx=0x7f0d6400b260 comp rx=0 tx=0).stop 2024-08-30T09:00:27.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:27.021+0000 7f0d73071640 1 -- 172.21.15.42:0/3297294938 shutdown_connections 2024-08-30T09:00:27.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:27.021+0000 7f0d73071640 1 --2- 172.21.15.42:0/3297294938 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f0d5c0b4bb0 0x7f0d5c0b70c0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:27.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:27.021+0000 7f0d73071640 1 --2- 172.21.15.42:0/3297294938 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0d6c108080 0x7f0d6c199de0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:27.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:27.021+0000 7f0d73071640 1 --2- 172.21.15.42:0/3297294938 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0d6c105460 0x7f0d6c199230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:27.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:27.021+0000 7f0d73071640 1 --2- 172.21.15.42:0/3297294938 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0d6c1029e0 0x7f0d6c198bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:27.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:27.021+0000 7f0d73071640 1 -- 172.21.15.42:0/3297294938 >> 172.21.15.42:0/3297294938 conn(0x7f0d6c0fd430 msgr2=0x7f0d6c108f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:00:27.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:27.021+0000 7f0d73071640 1 -- 172.21.15.42:0/3297294938 shutdown_connections 2024-08-30T09:00:27.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:27.022+0000 7f0d73071640 1 -- 172.21.15.42:0/3297294938 wait complete. 2024-08-30T09:00:27.035 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-08-30T09:00:27.035 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph fs dump --format=json 2024-08-30T09:00:27.287 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T09:00:27.634 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:00:27 smithi042 ceph-mon[18652]: from='client.? 172.21.15.42:0/2883630127' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} : dispatch 2024-08-30T09:00:27.634 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:00:27 smithi042 ceph-mon[18652]: from='client.? 172.21.15.42:0/378650973' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} : dispatch 2024-08-30T09:00:27.635 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:00:27 smithi042 ceph-mon[18652]: from='client.? 172.21.15.42:0/3297294938' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} : dispatch 2024-08-30T09:00:27.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:00:27 smithi190 ceph-mon[22001]: from='client.? 172.21.15.42:0/2883630127' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} : dispatch 2024-08-30T09:00:27.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:00:27 smithi190 ceph-mon[22001]: from='client.? 172.21.15.42:0/378650973' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} : dispatch 2024-08-30T09:00:27.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:00:27 smithi190 ceph-mon[22001]: from='client.? 172.21.15.42:0/3297294938' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} : dispatch 2024-08-30T09:00:27.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:27.796+0000 7efc7841d640 1 -- 172.21.15.42:0/3289706476 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efc70102880 msgr2=0x7efc70102c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:00:27.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:27.796+0000 7efc7841d640 1 --2- 172.21.15.42:0/3289706476 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efc70102880 0x7efc70102c90 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7efc6c009900 tx=0x7efc6c02a410 comp rx=0 tx=0).stop 2024-08-30T09:00:27.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:27.797+0000 7efc7841d640 1 -- 172.21.15.42:0/3289706476 shutdown_connections 2024-08-30T09:00:27.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:27.797+0000 7efc7841d640 1 --2- 172.21.15.42:0/3289706476 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efc70108b10 0x7efc7010f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:27.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:27.797+0000 7efc7841d640 1 --2- 172.21.15.42:0/3289706476 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efc701081c0 0x7efc701085d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:27.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:27.797+0000 7efc7841d640 1 --2- 172.21.15.42:0/3289706476 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efc70102880 0x7efc70102c90 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:27.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:27.797+0000 7efc7841d640 1 -- 172.21.15.42:0/3289706476 >> 172.21.15.42:0/3289706476 conn(0x7efc70079510 msgr2=0x7efc70079920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:00:27.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:27.797+0000 7efc7841d640 1 -- 172.21.15.42:0/3289706476 shutdown_connections 2024-08-30T09:00:27.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:27.797+0000 7efc7841d640 1 -- 172.21.15.42:0/3289706476 wait complete. 2024-08-30T09:00:27.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:27.797+0000 7efc7841d640 1 Processor -- start 2024-08-30T09:00:27.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:27.797+0000 7efc7841d640 1 -- start start 2024-08-30T09:00:27.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:27.798+0000 7efc7841d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efc70102880 0x7efc7019b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:27.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:27.798+0000 7efc7841d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efc701081c0 0x7efc7019b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:27.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:27.798+0000 7efc7841d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efc70108b10 0x7efc7019bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:27.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:27.798+0000 7efc7841d640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7efc701127c0 con 0x7efc70102880 2024-08-30T09:00:27.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:27.798+0000 7efc7841d640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7efc70112640 con 0x7efc701081c0 2024-08-30T09:00:27.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:27.798+0000 7efc7841d640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7efc70112940 con 0x7efc70108b10 2024-08-30T09:00:27.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:27.798+0000 7efc761b9640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efc70102880 0x7efc7019b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:27.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:27.798+0000 7efc761b9640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efc70102880 0x7efc7019b130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38544/0 (socket says 172.21.15.42:38544) 2024-08-30T09:00:27.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:27.798+0000 7efc761b9640 1 -- 172.21.15.42:0/1412622214 learned_addr learned my addr 172.21.15.42:0/1412622214 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:00:27.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:27.798+0000 7efc769ba640 1 --2- 172.21.15.42:0/1412622214 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efc70108b10 0x7efc7019bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:27.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:27.798+0000 7efc761b9640 1 -- 172.21.15.42:0/1412622214 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efc70108b10 msgr2=0x7efc7019bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:00:27.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:27.798+0000 7efc761b9640 1 --2- 172.21.15.42:0/1412622214 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efc70108b10 0x7efc7019bbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:27.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:27.798+0000 7efc761b9640 1 -- 172.21.15.42:0/1412622214 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efc701081c0 msgr2=0x7efc7019b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:00:27.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:27.798+0000 7efc759b8640 1 --2- 172.21.15.42:0/1412622214 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efc701081c0 0x7efc7019b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:27.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:27.798+0000 7efc761b9640 1 --2- 172.21.15.42:0/1412622214 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efc701081c0 0x7efc7019b670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:27.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:27.798+0000 7efc761b9640 1 -- 172.21.15.42:0/1412622214 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc701a63f0 con 0x7efc70102880 2024-08-30T09:00:27.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:27.798+0000 7efc769ba640 1 --2- 172.21.15.42:0/1412622214 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efc70108b10 0x7efc7019bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-30T09:00:27.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:27.798+0000 7efc761b9640 1 --2- 172.21.15.42:0/1412622214 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efc70102880 0x7efc7019b130 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7efc6c005940 tx=0x7efc6c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:00:27.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:27.798+0000 7efc677fe640 1 -- 172.21.15.42:0/1412622214 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc6c03f030 con 0x7efc70102880 2024-08-30T09:00:27.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:27.798+0000 7efc7841d640 1 -- 172.21.15.42:0/1412622214 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc701a6680 con 0x7efc70102880 2024-08-30T09:00:27.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:27.798+0000 7efc677fe640 1 -- 172.21.15.42:0/1412622214 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efc6c003cb0 con 0x7efc70102880 2024-08-30T09:00:27.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:27.798+0000 7efc677fe640 1 -- 172.21.15.42:0/1412622214 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc6c035070 con 0x7efc70102880 2024-08-30T09:00:27.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:27.799+0000 7efc7841d640 1 -- 172.21.15.42:0/1412622214 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc701a6bb0 con 0x7efc70102880 2024-08-30T09:00:27.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:27.800+0000 7efc7841d640 1 -- 172.21.15.42:0/1412622214 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc70103b00 con 0x7efc70102880 2024-08-30T09:00:27.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:27.801+0000 7efc677fe640 1 -- 172.21.15.42:0/1412622214 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7efc6c042030 con 0x7efc70102880 2024-08-30T09:00:27.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:27.802+0000 7efc677fe640 1 --2- 172.21.15.42:0/1412622214 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7efc5c0b4b30 0x7efc5c0b7040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:27.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:27.802+0000 7efc759b8640 1 --2- 172.21.15.42:0/1412622214 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7efc5c0b4b30 0x7efc5c0b7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:27.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:27.802+0000 7efc677fe640 1 -- 172.21.15.42:0/1412622214 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7efc6c106120 con 0x7efc70102880 2024-08-30T09:00:27.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:27.802+0000 7efc759b8640 1 --2- 172.21.15.42:0/1412622214 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7efc5c0b4b30 0x7efc5c0b7040 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7efc701051e0 tx=0x7efc60005c60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:00:27.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:27.805+0000 7efc677fe640 1 -- 172.21.15.42:0/1412622214 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7efc6c0cce60 con 0x7efc70102880 2024-08-30T09:00:27.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:00:27 smithi167 ceph-mon[22063]: from='client.? 172.21.15.42:0/2883630127' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} : dispatch 2024-08-30T09:00:27.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:00:27 smithi167 ceph-mon[22063]: from='client.? 172.21.15.42:0/378650973' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} : dispatch 2024-08-30T09:00:27.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:00:27 smithi167 ceph-mon[22063]: from='client.? 172.21.15.42:0/3297294938' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} : dispatch 2024-08-30T09:00:28.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:28.026+0000 7efc7841d640 1 -- 172.21.15.42:0/1412622214 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efc70102c90 con 0x7efc70102880 2024-08-30T09:00:28.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:28.027+0000 7efc677fe640 1 -- 172.21.15.42:0/1412622214 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11915 (secure 0 0 0) 0x7efc6c0d53a0 con 0x7efc70102880 2024-08-30T09:00:28.029 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-08-30T09:00:28.029 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":27,"btime":"2024-08-30T09:00:25:352021+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14703,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/3188400000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":3188400000},{"type":"v1","addr":"172.21.15.42:6835","nonce":3188400000}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14715,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/3356537674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":3356537674},{"type":"v1","addr":"172.21.15.42:6837","nonce":3356537674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14766,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6839/2107785513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6838","nonce":2107785513},{"type":"v1","addr":"172.21.15.42:6839","nonce":2107785513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14778,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6841/2298438674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6840","nonce":2298438674},{"type":"v1","addr":"172.21.15.42:6841","nonce":2298438674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24553,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/543555570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":543555570},{"type":"v1","addr":"172.21.15.190:6833","nonce":543555570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24554,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/3907818941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":3907818941},{"type":"v1","addr":"172.21.15.167:6833","nonce":3907818941}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":27,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-30T08:58:03.055084+0000","modified":"2024-08-30T09:00:24.377910+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24629,"mds_1":24610,"mds_2":24599},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24572":{"gid":24572,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6835/234596845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":234596845},{"type":"v1","addr":"172.21.15.167:6835","nonce":234596845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24592":{"gid":24592,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6835/247905709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":247905709},{"type":"v1","addr":"172.21.15.190:6835","nonce":247905709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24598":{"gid":24598,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6837/863256349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6836","nonce":863256349},{"type":"v1","addr":"172.21.15.190:6837","nonce":863256349}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24599":{"gid":24599,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.167:6837/3011837856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":3011837856},{"type":"v1","addr":"172.21.15.167:6837","nonce":3011837856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24610":{"gid":24610,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.190:6839/719981110","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6838","nonce":719981110},{"type":"v1","addr":"172.21.15.190:6839","nonce":719981110}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24629":{"gid":24629,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.167:6839/3604645207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":3604645207},{"type":"v1","addr":"172.21.15.167:6839","nonce":3604645207}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24629,"qdb_cluster":[24599,24610,24629]},"id":1}]} 2024-08-30T09:00:28.030 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 27 2024-08-30T09:00:28.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:28.030+0000 7efc7841d640 1 -- 172.21.15.42:0/1412622214 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7efc5c0b4b30 msgr2=0x7efc5c0b7040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:00:28.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:28.030+0000 7efc7841d640 1 --2- 172.21.15.42:0/1412622214 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7efc5c0b4b30 0x7efc5c0b7040 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7efc701051e0 tx=0x7efc60005c60 comp rx=0 tx=0).stop 2024-08-30T09:00:28.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:28.030+0000 7efc7841d640 1 -- 172.21.15.42:0/1412622214 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efc70102880 msgr2=0x7efc7019b130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:00:28.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:28.030+0000 7efc7841d640 1 --2- 172.21.15.42:0/1412622214 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efc70102880 0x7efc7019b130 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7efc6c005940 tx=0x7efc6c0042f0 comp rx=0 tx=0).stop 2024-08-30T09:00:28.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:28.030+0000 7efc7841d640 1 -- 172.21.15.42:0/1412622214 shutdown_connections 2024-08-30T09:00:28.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:28.030+0000 7efc7841d640 1 --2- 172.21.15.42:0/1412622214 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7efc5c0b4b30 0x7efc5c0b7040 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:28.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:28.030+0000 7efc7841d640 1 --2- 172.21.15.42:0/1412622214 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efc70108b10 0x7efc7019bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:28.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:28.030+0000 7efc7841d640 1 --2- 172.21.15.42:0/1412622214 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efc701081c0 0x7efc7019b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:28.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:28.030+0000 7efc7841d640 1 --2- 172.21.15.42:0/1412622214 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efc70102880 0x7efc7019b130 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:28.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:28.030+0000 7efc7841d640 1 -- 172.21.15.42:0/1412622214 >> 172.21.15.42:0/1412622214 conn(0x7efc70079510 msgr2=0x7efc70106360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:00:28.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:28.030+0000 7efc7841d640 1 -- 172.21.15.42:0/1412622214 shutdown_connections 2024-08-30T09:00:28.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:28.030+0000 7efc7841d640 1 -- 172.21.15.42:0/1412622214 wait complete. 2024-08-30T09:00:28.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:00:28 smithi042 ceph-mon[18652]: pgmap v364: 129 pgs: 129 active+clean; 644 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 6 op/s 2024-08-30T09:00:28.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:00:28 smithi042 ceph-mon[18652]: from='client.? 172.21.15.42:0/1412622214' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-30T09:00:28.658 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-08-30T09:00:28.658 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-08-30T09:00:28.658 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-08-30T09:00:28.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:00:28 smithi190 ceph-mon[22001]: pgmap v364: 129 pgs: 129 active+clean; 644 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 6 op/s 2024-08-30T09:00:28.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:00:28 smithi190 ceph-mon[22001]: from='client.? 172.21.15.42:0/1412622214' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-30T09:00:28.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:28.771+0000 7f0d2bc07640 1 -- 172.21.15.42:0/4270161982 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0d24103d30 msgr2=0x7f0d2410a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:00:28.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:28.771+0000 7f0d2bc07640 1 --2- 172.21.15.42:0/4270161982 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0d24103d30 0x7f0d2410a6d0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f0d1c009ca0 tx=0x7f0d1c02a8c0 comp rx=0 tx=0).stop 2024-08-30T09:00:28.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:28.771+0000 7f0d2bc07640 1 -- 172.21.15.42:0/4270161982 shutdown_connections 2024-08-30T09:00:28.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:28.771+0000 7f0d2bc07640 1 --2- 172.21.15.42:0/4270161982 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0d24103d30 0x7f0d2410a6d0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:28.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:28.771+0000 7f0d2bc07640 1 --2- 172.21.15.42:0/4270161982 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0d24103390 0x7f0d241037a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:28.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:28.771+0000 7f0d2bc07640 1 --2- 172.21.15.42:0/4270161982 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0d24102310 0x7f0d24102780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:28.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:28.771+0000 7f0d2bc07640 1 -- 172.21.15.42:0/4270161982 >> 172.21.15.42:0/4270161982 conn(0x7f0d240fdc50 msgr2=0x7f0d24100100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:00:28.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:28.771+0000 7f0d2bc07640 1 -- 172.21.15.42:0/4270161982 shutdown_connections 2024-08-30T09:00:28.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:28.771+0000 7f0d2bc07640 1 -- 172.21.15.42:0/4270161982 wait complete. 2024-08-30T09:00:28.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:28.772+0000 7f0d2bc07640 1 Processor -- start 2024-08-30T09:00:28.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:28.772+0000 7f0d2bc07640 1 -- start start 2024-08-30T09:00:28.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:28.772+0000 7f0d2bc07640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0d24102310 0x7f0d24199c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:28.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:28.772+0000 7f0d2bc07640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0d24103390 0x7f0d2419a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:28.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:28.772+0000 7f0d2bc07640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0d24103d30 0x7f0d2419ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:28.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:28.772+0000 7f0d2bc07640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f0d2410f0a0 con 0x7f0d24103390 2024-08-30T09:00:28.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:28.772+0000 7f0d2bc07640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f0d2410ef20 con 0x7f0d24103d30 2024-08-30T09:00:28.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:28.772+0000 7f0d2bc07640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f0d2410f220 con 0x7f0d24102310 2024-08-30T09:00:28.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:28.772+0000 7f0d291a2640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0d24103390 0x7f0d2419a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:28.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:28.772+0000 7f0d291a2640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0d24103390 0x7f0d2419a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:51422/0 (socket says 172.21.15.42:51422) 2024-08-30T09:00:28.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:28.772+0000 7f0d291a2640 1 -- 172.21.15.42:0/690510755 learned_addr learned my addr 172.21.15.42:0/690510755 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:00:28.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:28.772+0000 7f0d299a3640 1 --2- 172.21.15.42:0/690510755 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0d24102310 0x7f0d24199c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:28.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:28.773+0000 7f0d2a1a4640 1 --2- 172.21.15.42:0/690510755 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0d24103d30 0x7f0d2419ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:28.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:28.773+0000 7f0d291a2640 1 -- 172.21.15.42:0/690510755 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0d24102310 msgr2=0x7f0d24199c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:00:28.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:28.773+0000 7f0d291a2640 1 --2- 172.21.15.42:0/690510755 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0d24102310 0x7f0d24199c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:28.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:28.773+0000 7f0d291a2640 1 -- 172.21.15.42:0/690510755 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0d24103d30 msgr2=0x7f0d2419ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:00:28.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:28.773+0000 7f0d291a2640 1 --2- 172.21.15.42:0/690510755 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0d24103d30 0x7f0d2419ae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:28.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:28.773+0000 7f0d291a2640 1 -- 172.21.15.42:0/690510755 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d2419b6a0 con 0x7f0d24103390 2024-08-30T09:00:28.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:28.773+0000 7f0d291a2640 1 --2- 172.21.15.42:0/690510755 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0d24103390 0x7f0d2419a2d0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f0d1400d950 tx=0x7f0d1400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:00:28.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:28.773+0000 7f0d1affd640 1 -- 172.21.15.42:0/690510755 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d14017070 con 0x7f0d24103390 2024-08-30T09:00:28.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:28.773+0000 7f0d1affd640 1 -- 172.21.15.42:0/690510755 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d14009c30 con 0x7f0d24103390 2024-08-30T09:00:28.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:28.773+0000 7f0d1affd640 1 -- 172.21.15.42:0/690510755 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d1400f620 con 0x7f0d24103390 2024-08-30T09:00:28.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:28.773+0000 7f0d2bc07640 1 -- 172.21.15.42:0/690510755 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d2419b990 con 0x7f0d24103390 2024-08-30T09:00:28.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:28.773+0000 7f0d2bc07640 1 -- 172.21.15.42:0/690510755 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d241a2080 con 0x7f0d24103390 2024-08-30T09:00:28.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:28.774+0000 7f0d2bc07640 1 -- 172.21.15.42:0/690510755 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d24102780 con 0x7f0d24103390 2024-08-30T09:00:28.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:28.776+0000 7f0d1affd640 1 -- 172.21.15.42:0/690510755 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7f0d14010460 con 0x7f0d24103390 2024-08-30T09:00:28.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:28.776+0000 7f0d1affd640 1 --2- 172.21.15.42:0/690510755 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f0d100b4b30 0x7f0d100b7040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:28.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:28.776+0000 7f0d1affd640 1 -- 172.21.15.42:0/690510755 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f0d140e5050 con 0x7f0d24103390 2024-08-30T09:00:28.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:28.776+0000 7f0d299a3640 1 --2- 172.21.15.42:0/690510755 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f0d100b4b30 0x7f0d100b7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:28.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:28.777+0000 7f0d299a3640 1 --2- 172.21.15.42:0/690510755 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f0d100b4b30 0x7f0d100b7040 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f0d2000bbe0 tx=0x7f0d2000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:00:28.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:28.779+0000 7f0d1affd640 1 -- 172.21.15.42:0/690510755 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f0d14009de0 con 0x7f0d24103390 2024-08-30T09:00:28.826 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:00:28 smithi167 ceph-mon[22063]: pgmap v364: 129 pgs: 129 active+clean; 644 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 6 op/s 2024-08-30T09:00:28.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:00:28 smithi167 ceph-mon[22063]: from='client.? 172.21.15.42:0/1412622214' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-30T09:00:28.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:28.961+0000 7f0d2bc07640 1 -- 172.21.15.42:0/690510755 --> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0d240635a0 con 0x7f0d100b4b30 2024-08-30T09:00:28.969 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:28.967+0000 7f0d1affd640 1 -- 172.21.15.42:0/690510755 <== mgr.14150 v2:172.21.15.42:6800/3005140902 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340450 (secure 0 0 0) 0x7f0d240635a0 con 0x7f0d100b4b30 2024-08-30T09:00:28.973 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-08-30T09:00:28.975 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-08-30T09:00:28.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:28.971+0000 7f0d2bc07640 1 -- 172.21.15.42:0/690510755 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f0d100b4b30 msgr2=0x7f0d100b7040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:00:28.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:28.971+0000 7f0d2bc07640 1 --2- 172.21.15.42:0/690510755 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f0d100b4b30 0x7f0d100b7040 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f0d2000bbe0 tx=0x7f0d2000b390 comp rx=0 tx=0).stop 2024-08-30T09:00:28.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:28.971+0000 7f0d2bc07640 1 -- 172.21.15.42:0/690510755 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0d24103390 msgr2=0x7f0d2419a2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:00:28.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:28.971+0000 7f0d2bc07640 1 --2- 172.21.15.42:0/690510755 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0d24103390 0x7f0d2419a2d0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f0d1400d950 tx=0x7f0d1400de10 comp rx=0 tx=0).stop 2024-08-30T09:00:28.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:28.971+0000 7f0d2bc07640 1 -- 172.21.15.42:0/690510755 shutdown_connections 2024-08-30T09:00:28.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:28.971+0000 7f0d2bc07640 1 --2- 172.21.15.42:0/690510755 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f0d100b4b30 0x7f0d100b7040 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:28.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:28.971+0000 7f0d2bc07640 1 --2- 172.21.15.42:0/690510755 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0d24103d30 0x7f0d2419ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:28.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:28.971+0000 7f0d2bc07640 1 --2- 172.21.15.42:0/690510755 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0d24103390 0x7f0d2419a2d0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:28.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:28.971+0000 7f0d2bc07640 1 --2- 172.21.15.42:0/690510755 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0d24102310 0x7f0d24199c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:28.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:28.971+0000 7f0d2bc07640 1 -- 172.21.15.42:0/690510755 >> 172.21.15.42:0/690510755 conn(0x7f0d240fdc50 msgr2=0x7f0d24106690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:00:28.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:28.971+0000 7f0d2bc07640 1 -- 172.21.15.42:0/690510755 shutdown_connections 2024-08-30T09:00:28.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:28.971+0000 7f0d2bc07640 1 -- 172.21.15.42:0/690510755 wait complete. 2024-08-30T09:00:28.984 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":364,"stamp":"2024-08-30T09:00:26.964817+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":659585,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":662,"num_read_kb":428,"num_write":305,"num_write_kb":2279,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":2951840,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":748,"ondisk_log_size":748,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":379120,"kb_used_data":13232,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124562704,"statfs":{"total":1151940427776,"available":1151552208896,"internally_reserved":0,"allocated":13549568,"data_stored":7644769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":374453561},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":531,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":68,"num_read_kb":33,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003344"},"pg_stats":[{"pgid":"2.1f","version":"80'9","reported_seq":50,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T08:59:46.656396+0000","last_change":"2024-08-30T08:58:00.388595+0000","last_active":"2024-08-30T08:59:46.656396+0000","last_peered":"2024-08-30T08:59:46.656396+0000","last_clean":"2024-08-30T08:59:46.656396+0000","last_became_active":"2024-08-30T08:58:00.388215+0000","last_became_peered":"2024-08-30T08:58:00.388215+0000","last_unstale":"2024-08-30T08:59:46.656396+0000","last_undegraded":"2024-08-30T08:59:46.656396+0000","last_fullsized":"2024-08-30T08:59:46.656396+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T13:00:06.109935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"83'5","reported_seq":40,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:15.312524+0000","last_change":"2024-08-30T08:58:02.416042+0000","last_active":"2024-08-30T09:00:15.312524+0000","last_peered":"2024-08-30T09:00:15.312524+0000","last_clean":"2024-08-30T09:00:15.312524+0000","last_became_active":"2024-08-30T08:58:02.415507+0000","last_became_peered":"2024-08-30T08:58:02.415507+0000","last_unstale":"2024-08-30T09:00:15.312524+0000","last_undegraded":"2024-08-30T09:00:15.312524+0000","last_fullsized":"2024-08-30T09:00:15.312524+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T14:33:57.192004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T08:59:46.657037+0000","last_change":"2024-08-30T08:58:09.448785+0000","last_active":"2024-08-30T08:59:46.657037+0000","last_peered":"2024-08-30T08:59:46.657037+0000","last_clean":"2024-08-30T08:59:46.657037+0000","last_became_active":"2024-08-30T08:58:09.448191+0000","last_became_peered":"2024-08-30T08:58:09.448191+0000","last_unstale":"2024-08-30T08:59:46.657037+0000","last_undegraded":"2024-08-30T08:59:46.657037+0000","last_fullsized":"2024-08-30T08:59:46.657037+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T13:37:19.148201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"80'2","reported_seq":42,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T08:59:46.702141+0000","last_change":"2024-08-30T08:58:00.480406+0000","last_active":"2024-08-30T08:59:46.702141+0000","last_peered":"2024-08-30T08:59:46.702141+0000","last_clean":"2024-08-30T08:59:46.702141+0000","last_became_active":"2024-08-30T08:58:00.479896+0000","last_became_peered":"2024-08-30T08:58:00.479896+0000","last_unstale":"2024-08-30T08:59:46.702141+0000","last_undegraded":"2024-08-30T08:59:46.702141+0000","last_fullsized":"2024-08-30T08:59:46.702141+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T15:16:15.316343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"83'6","reported_seq":41,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:15.335025+0000","last_change":"2024-08-30T08:58:02.406915+0000","last_active":"2024-08-30T09:00:15.335025+0000","last_peered":"2024-08-30T09:00:15.335025+0000","last_clean":"2024-08-30T09:00:15.335025+0000","last_became_active":"2024-08-30T08:58:02.406557+0000","last_became_peered":"2024-08-30T08:58:02.406557+0000","last_unstale":"2024-08-30T09:00:15.335025+0000","last_undegraded":"2024-08-30T09:00:15.335025+0000","last_fullsized":"2024-08-30T09:00:15.335025+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T19:15:30.847212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T08:59:46.661576+0000","last_change":"2024-08-30T08:58:09.446974+0000","last_active":"2024-08-30T08:59:46.661576+0000","last_peered":"2024-08-30T08:59:46.661576+0000","last_clean":"2024-08-30T08:59:46.661576+0000","last_became_active":"2024-08-30T08:58:09.446173+0000","last_became_peered":"2024-08-30T08:58:09.446173+0000","last_unstale":"2024-08-30T08:59:46.661576+0000","last_undegraded":"2024-08-30T08:59:46.661576+0000","last_fullsized":"2024-08-30T08:59:46.661576+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T12:13:41.276825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"80'4","reported_seq":56,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T08:59:46.673167+0000","last_change":"2024-08-30T08:59:46.673048+0000","last_active":"2024-08-30T08:59:46.673167+0000","last_peered":"2024-08-30T08:59:46.673167+0000","last_clean":"2024-08-30T08:59:46.673167+0000","last_became_active":"2024-08-30T08:59:46.665588+0000","last_became_peered":"2024-08-30T08:59:46.665588+0000","last_unstale":"2024-08-30T08:59:46.673167+0000","last_undegraded":"2024-08-30T08:59:46.673167+0000","last_fullsized":"2024-08-30T08:59:46.673167+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T18:02:29.663191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,7],"acting":[8,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"83'9","reported_seq":43,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:15.334804+0000","last_change":"2024-08-30T08:58:02.415594+0000","last_active":"2024-08-30T09:00:15.334804+0000","last_peered":"2024-08-30T09:00:15.334804+0000","last_clean":"2024-08-30T09:00:15.334804+0000","last_became_active":"2024-08-30T08:58:02.415066+0000","last_became_peered":"2024-08-30T08:58:02.415066+0000","last_unstale":"2024-08-30T09:00:15.334804+0000","last_undegraded":"2024-08-30T09:00:15.334804+0000","last_fullsized":"2024-08-30T09:00:15.334804+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T12:07:27.668442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T08:59:46.662698+0000","last_change":"2024-08-30T08:58:09.444403+0000","last_active":"2024-08-30T08:59:46.662698+0000","last_peered":"2024-08-30T08:59:46.662698+0000","last_clean":"2024-08-30T08:59:46.662698+0000","last_became_active":"2024-08-30T08:58:09.443914+0000","last_became_peered":"2024-08-30T08:58:09.443914+0000","last_unstale":"2024-08-30T08:59:46.662698+0000","last_undegraded":"2024-08-30T08:59:46.662698+0000","last_fullsized":"2024-08-30T08:59:46.662698+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T14:17:43.866352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"73'2","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T08:59:46.669336+0000","last_change":"2024-08-30T08:59:46.668686+0000","last_active":"2024-08-30T08:59:46.669336+0000","last_peered":"2024-08-30T08:59:46.669336+0000","last_clean":"2024-08-30T08:59:46.669336+0000","last_became_active":"2024-08-30T08:59:46.665549+0000","last_became_peered":"2024-08-30T08:59:46.665549+0000","last_unstale":"2024-08-30T08:59:46.669336+0000","last_undegraded":"2024-08-30T08:59:46.669336+0000","last_fullsized":"2024-08-30T08:59:46.669336+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T12:25:56.088867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"83'15","reported_seq":49,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:15.315195+0000","last_change":"2024-08-30T08:58:02.405550+0000","last_active":"2024-08-30T09:00:15.315195+0000","last_peered":"2024-08-30T09:00:15.315195+0000","last_clean":"2024-08-30T09:00:15.315195+0000","last_became_active":"2024-08-30T08:58:02.405129+0000","last_became_peered":"2024-08-30T08:58:02.405129+0000","last_unstale":"2024-08-30T09:00:15.315195+0000","last_undegraded":"2024-08-30T09:00:15.315195+0000","last_fullsized":"2024-08-30T09:00:15.315195+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T11:36:47.624026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T08:59:46.661666+0000","last_change":"2024-08-30T08:58:09.445746+0000","last_active":"2024-08-30T08:59:46.661666+0000","last_peered":"2024-08-30T08:59:46.661666+0000","last_clean":"2024-08-30T08:59:46.661666+0000","last_became_active":"2024-08-30T08:58:09.445257+0000","last_became_peered":"2024-08-30T08:58:09.445257+0000","last_unstale":"2024-08-30T08:59:46.661666+0000","last_undegraded":"2024-08-30T08:59:46.661666+0000","last_fullsized":"2024-08-30T08:59:46.661666+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T18:02:52.007131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"83'5","reported_seq":40,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:15.335272+0000","last_change":"2024-08-30T08:58:02.408213+0000","last_active":"2024-08-30T09:00:15.335272+0000","last_peered":"2024-08-30T09:00:15.335272+0000","last_clean":"2024-08-30T09:00:15.335272+0000","last_became_active":"2024-08-30T08:58:02.407856+0000","last_became_peered":"2024-08-30T08:58:02.407856+0000","last_unstale":"2024-08-30T09:00:15.335272+0000","last_undegraded":"2024-08-30T09:00:15.335272+0000","last_fullsized":"2024-08-30T09:00:15.335272+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T12:00:22.226653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"79'4","reported_seq":45,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T08:59:46.659742+0000","last_change":"2024-08-30T08:58:00.391714+0000","last_active":"2024-08-30T08:59:46.659742+0000","last_peered":"2024-08-30T08:59:46.659742+0000","last_clean":"2024-08-30T08:59:46.659742+0000","last_became_active":"2024-08-30T08:58:00.391211+0000","last_became_peered":"2024-08-30T08:58:00.391211+0000","last_unstale":"2024-08-30T08:59:46.659742+0000","last_undegraded":"2024-08-30T08:59:46.659742+0000","last_fullsized":"2024-08-30T08:59:46.659742+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T11:55:37.469983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T08:59:46.662058+0000","last_change":"2024-08-30T08:58:09.728177+0000","last_active":"2024-08-30T08:59:46.662058+0000","last_peered":"2024-08-30T08:59:46.662058+0000","last_clean":"2024-08-30T08:59:46.662058+0000","last_became_active":"2024-08-30T08:58:09.727630+0000","last_became_peered":"2024-08-30T08:58:09.727630+0000","last_unstale":"2024-08-30T08:59:46.662058+0000","last_undegraded":"2024-08-30T08:59:46.662058+0000","last_fullsized":"2024-08-30T08:59:46.662058+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T12:07:52.965014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"83'8","reported_seq":43,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:15.319441+0000","last_change":"2024-08-30T08:58:02.403378+0000","last_active":"2024-08-30T09:00:15.319441+0000","last_peered":"2024-08-30T09:00:15.319441+0000","last_clean":"2024-08-30T09:00:15.319441+0000","last_became_active":"2024-08-30T08:58:02.403036+0000","last_became_peered":"2024-08-30T08:58:02.403036+0000","last_unstale":"2024-08-30T09:00:15.319441+0000","last_undegraded":"2024-08-30T09:00:15.319441+0000","last_fullsized":"2024-08-30T09:00:15.319441+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T16:37:14.876384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"80'1","reported_seq":41,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T08:59:46.657210+0000","last_change":"2024-08-30T08:58:00.393860+0000","last_active":"2024-08-30T08:59:46.657210+0000","last_peered":"2024-08-30T08:59:46.657210+0000","last_clean":"2024-08-30T08:59:46.657210+0000","last_became_active":"2024-08-30T08:58:00.392486+0000","last_became_peered":"2024-08-30T08:58:00.392486+0000","last_unstale":"2024-08-30T08:59:46.657210+0000","last_undegraded":"2024-08-30T08:59:46.657210+0000","last_fullsized":"2024-08-30T08:59:46.657210+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T12:50:27.337817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"81'1","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T08:59:46.662322+0000","last_change":"2024-08-30T08:58:09.441476+0000","last_active":"2024-08-30T08:59:46.662322+0000","last_peered":"2024-08-30T08:59:46.662322+0000","last_clean":"2024-08-30T08:59:46.662322+0000","last_became_active":"2024-08-30T08:58:09.440344+0000","last_became_peered":"2024-08-30T08:58:09.440344+0000","last_unstale":"2024-08-30T08:59:46.662322+0000","last_undegraded":"2024-08-30T08:59:46.662322+0000","last_fullsized":"2024-08-30T08:59:46.662322+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T12:06:47.113611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"83'9","reported_seq":44,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:15.314692+0000","last_change":"2024-08-30T08:58:02.407098+0000","last_active":"2024-08-30T09:00:15.314692+0000","last_peered":"2024-08-30T09:00:15.314692+0000","last_clean":"2024-08-30T09:00:15.314692+0000","last_became_active":"2024-08-30T08:58:02.406690+0000","last_became_peered":"2024-08-30T08:58:02.406690+0000","last_unstale":"2024-08-30T09:00:15.314692+0000","last_undegraded":"2024-08-30T09:00:15.314692+0000","last_fullsized":"2024-08-30T09:00:15.314692+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T18:27:39.534923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"79'3","reported_seq":43,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T08:59:46.701567+0000","last_change":"2024-08-30T08:58:00.386434+0000","last_active":"2024-08-30T08:59:46.701567+0000","last_peered":"2024-08-30T08:59:46.701567+0000","last_clean":"2024-08-30T08:59:46.701567+0000","last_became_active":"2024-08-30T08:58:00.385930+0000","last_became_peered":"2024-08-30T08:58:00.385930+0000","last_unstale":"2024-08-30T08:59:46.701567+0000","last_undegraded":"2024-08-30T08:59:46.701567+0000","last_fullsized":"2024-08-30T08:59:46.701567+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T10:39:22.441568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T08:59:46.659680+0000","last_change":"2024-08-30T08:58:46.775674+0000","last_active":"2024-08-30T08:59:46.659680+0000","last_peered":"2024-08-30T08:59:46.659680+0000","last_clean":"2024-08-30T08:59:46.659680+0000","last_became_active":"2024-08-30T08:58:46.775157+0000","last_became_peered":"2024-08-30T08:58:46.775157+0000","last_unstale":"2024-08-30T08:59:46.659680+0000","last_undegraded":"2024-08-30T08:59:46.659680+0000","last_fullsized":"2024-08-30T08:59:46.659680+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T17:13:47.066475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,2,5],"acting":[8,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.9","version":"83'2","reported_seq":37,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:15.283463+0000","last_change":"2024-08-30T08:58:02.410500+0000","last_active":"2024-08-30T09:00:15.283463+0000","last_peered":"2024-08-30T09:00:15.283463+0000","last_clean":"2024-08-30T09:00:15.283463+0000","last_became_active":"2024-08-30T08:58:02.409260+0000","last_became_peered":"2024-08-30T08:58:02.409260+0000","last_unstale":"2024-08-30T09:00:15.283463+0000","last_undegraded":"2024-08-30T09:00:15.283463+0000","last_fullsized":"2024-08-30T09:00:15.283463+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T16:47:59.701956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"79'2","reported_seq":42,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T08:59:46.663767+0000","last_change":"2024-08-30T08:58:00.482296+0000","last_active":"2024-08-30T08:59:46.663767+0000","last_peered":"2024-08-30T08:59:46.663767+0000","last_clean":"2024-08-30T08:59:46.663767+0000","last_became_active":"2024-08-30T08:58:00.481920+0000","last_became_peered":"2024-08-30T08:58:00.481920+0000","last_unstale":"2024-08-30T08:59:46.663767+0000","last_undegraded":"2024-08-30T08:59:46.663767+0000","last_fullsized":"2024-08-30T08:59:46.663767+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T09:29:30.724410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T08:59:46.662353+0000","last_change":"2024-08-30T08:58:09.727879+0000","last_active":"2024-08-30T08:59:46.662353+0000","last_peered":"2024-08-30T08:59:46.662353+0000","last_clean":"2024-08-30T08:59:46.662353+0000","last_became_active":"2024-08-30T08:58:09.727339+0000","last_became_peered":"2024-08-30T08:58:09.727339+0000","last_unstale":"2024-08-30T08:59:46.662353+0000","last_undegraded":"2024-08-30T08:59:46.662353+0000","last_fullsized":"2024-08-30T08:59:46.662353+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T13:19:18.582668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"83'7","reported_seq":49,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:15.332760+0000","last_change":"2024-08-30T08:58:46.559988+0000","last_active":"2024-08-30T09:00:15.332760+0000","last_peered":"2024-08-30T09:00:15.332760+0000","last_clean":"2024-08-30T09:00:15.332760+0000","last_became_active":"2024-08-30T08:58:46.559442+0000","last_became_peered":"2024-08-30T08:58:46.559442+0000","last_unstale":"2024-08-30T09:00:15.332760+0000","last_undegraded":"2024-08-30T09:00:15.332760+0000","last_fullsized":"2024-08-30T09:00:15.332760+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T09:29:09.747756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"79'4","reported_seq":44,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T08:59:46.662490+0000","last_change":"2024-08-30T08:58:00.380338+0000","last_active":"2024-08-30T08:59:46.662490+0000","last_peered":"2024-08-30T08:59:46.662490+0000","last_clean":"2024-08-30T08:59:46.662490+0000","last_became_active":"2024-08-30T08:58:00.379860+0000","last_became_peered":"2024-08-30T08:58:00.379860+0000","last_unstale":"2024-08-30T08:59:46.662490+0000","last_undegraded":"2024-08-30T08:59:46.662490+0000","last_fullsized":"2024-08-30T08:59:46.662490+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T19:57:15.349906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T08:59:46.657178+0000","last_change":"2024-08-30T08:58:09.443631+0000","last_active":"2024-08-30T08:59:46.657178+0000","last_peered":"2024-08-30T08:59:46.657178+0000","last_clean":"2024-08-30T08:59:46.657178+0000","last_became_active":"2024-08-30T08:58:09.443103+0000","last_became_peered":"2024-08-30T08:58:09.443103+0000","last_unstale":"2024-08-30T08:59:46.657178+0000","last_undegraded":"2024-08-30T08:59:46.657178+0000","last_fullsized":"2024-08-30T08:59:46.657178+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T19:58:11.031826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"83'5","reported_seq":40,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:15.328087+0000","last_change":"2024-08-30T08:58:02.406216+0000","last_active":"2024-08-30T09:00:15.328087+0000","last_peered":"2024-08-30T09:00:15.328087+0000","last_clean":"2024-08-30T09:00:15.328087+0000","last_became_active":"2024-08-30T08:58:02.405040+0000","last_became_peered":"2024-08-30T08:58:02.405040+0000","last_unstale":"2024-08-30T09:00:15.328087+0000","last_undegraded":"2024-08-30T09:00:15.328087+0000","last_fullsized":"2024-08-30T09:00:15.328087+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T14:28:28.721386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"83'13","reported_seq":53,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:13.525613+0000","last_change":"2024-08-30T08:58:00.387350+0000","last_active":"2024-08-30T09:00:13.525613+0000","last_peered":"2024-08-30T09:00:13.525613+0000","last_clean":"2024-08-30T09:00:13.525613+0000","last_became_active":"2024-08-30T08:58:00.387004+0000","last_became_peered":"2024-08-30T08:58:00.387004+0000","last_unstale":"2024-08-30T09:00:13.525613+0000","last_undegraded":"2024-08-30T09:00:13.525613+0000","last_fullsized":"2024-08-30T09:00:13.525613+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T19:24:25.604544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T08:59:46.700944+0000","last_change":"2024-08-30T08:58:09.727388+0000","last_active":"2024-08-30T08:59:46.700944+0000","last_peered":"2024-08-30T08:59:46.700944+0000","last_clean":"2024-08-30T08:59:46.700944+0000","last_became_active":"2024-08-30T08:58:09.726935+0000","last_became_peered":"2024-08-30T08:58:09.726935+0000","last_unstale":"2024-08-30T08:59:46.700944+0000","last_undegraded":"2024-08-30T08:59:46.700944+0000","last_fullsized":"2024-08-30T08:59:46.700944+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T17:53:16.066602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"83'9","reported_seq":44,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:15.307241+0000","last_change":"2024-08-30T08:58:02.402661+0000","last_active":"2024-08-30T09:00:15.307241+0000","last_peered":"2024-08-30T09:00:15.307241+0000","last_clean":"2024-08-30T09:00:15.307241+0000","last_became_active":"2024-08-30T08:58:02.402314+0000","last_became_peered":"2024-08-30T08:58:02.402314+0000","last_unstale":"2024-08-30T09:00:15.307241+0000","last_undegraded":"2024-08-30T09:00:15.307241+0000","last_fullsized":"2024-08-30T09:00:15.307241+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T10:56:41.336312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"79'1","reported_seq":42,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T08:59:46.660369+0000","last_change":"2024-08-30T08:58:00.482720+0000","last_active":"2024-08-30T08:59:46.660369+0000","last_peered":"2024-08-30T08:59:46.660369+0000","last_clean":"2024-08-30T08:59:46.660369+0000","last_became_active":"2024-08-30T08:58:00.482193+0000","last_became_peered":"2024-08-30T08:58:00.482193+0000","last_unstale":"2024-08-30T08:59:46.660369+0000","last_undegraded":"2024-08-30T08:59:46.660369+0000","last_fullsized":"2024-08-30T08:59:46.660369+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T18:42:06.654806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T08:59:46.660980+0000","last_change":"2024-08-30T08:58:09.449970+0000","last_active":"2024-08-30T08:59:46.660980+0000","last_peered":"2024-08-30T08:59:46.660980+0000","last_clean":"2024-08-30T08:59:46.660980+0000","last_became_active":"2024-08-30T08:58:09.447891+0000","last_became_peered":"2024-08-30T08:58:09.447891+0000","last_unstale":"2024-08-30T08:59:46.660980+0000","last_undegraded":"2024-08-30T08:59:46.660980+0000","last_fullsized":"2024-08-30T08:59:46.660980+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T15:06:45.414470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"83'4","reported_seq":39,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:15.322836+0000","last_change":"2024-08-30T08:58:02.409756+0000","last_active":"2024-08-30T09:00:15.322836+0000","last_peered":"2024-08-30T09:00:15.322836+0000","last_clean":"2024-08-30T09:00:15.322836+0000","last_became_active":"2024-08-30T08:58:02.409357+0000","last_became_peered":"2024-08-30T08:58:02.409357+0000","last_unstale":"2024-08-30T09:00:15.322836+0000","last_undegraded":"2024-08-30T09:00:15.322836+0000","last_fullsized":"2024-08-30T09:00:15.322836+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T13:27:41.696527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":40,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T08:59:46.656323+0000","last_change":"2024-08-30T08:58:00.482342+0000","last_active":"2024-08-30T08:59:46.656323+0000","last_peered":"2024-08-30T08:59:46.656323+0000","last_clean":"2024-08-30T08:59:46.656323+0000","last_became_active":"2024-08-30T08:58:00.481902+0000","last_became_peered":"2024-08-30T08:58:00.481902+0000","last_unstale":"2024-08-30T08:59:46.656323+0000","last_undegraded":"2024-08-30T08:59:46.656323+0000","last_fullsized":"2024-08-30T08:59:46.656323+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T14:12:36.789356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T08:59:46.659485+0000","last_change":"2024-08-30T08:58:09.728008+0000","last_active":"2024-08-30T08:59:46.659485+0000","last_peered":"2024-08-30T08:59:46.659485+0000","last_clean":"2024-08-30T08:59:46.659485+0000","last_became_active":"2024-08-30T08:58:09.727513+0000","last_became_peered":"2024-08-30T08:58:09.727513+0000","last_unstale":"2024-08-30T08:59:46.659485+0000","last_undegraded":"2024-08-30T08:59:46.659485+0000","last_fullsized":"2024-08-30T08:59:46.659485+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T14:34:04.165300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"83'7","reported_seq":42,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:15.310825+0000","last_change":"2024-08-30T08:58:02.409383+0000","last_active":"2024-08-30T09:00:15.310825+0000","last_peered":"2024-08-30T09:00:15.310825+0000","last_clean":"2024-08-30T09:00:15.310825+0000","last_became_active":"2024-08-30T08:58:02.409027+0000","last_became_peered":"2024-08-30T08:58:02.409027+0000","last_unstale":"2024-08-30T09:00:15.310825+0000","last_undegraded":"2024-08-30T09:00:15.310825+0000","last_fullsized":"2024-08-30T09:00:15.310825+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T10:44:23.467130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"80'1","reported_seq":41,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T08:59:46.663584+0000","last_change":"2024-08-30T08:58:00.389083+0000","last_active":"2024-08-30T08:59:46.663584+0000","last_peered":"2024-08-30T08:59:46.663584+0000","last_clean":"2024-08-30T08:59:46.663584+0000","last_became_active":"2024-08-30T08:58:00.387552+0000","last_became_peered":"2024-08-30T08:58:00.387552+0000","last_unstale":"2024-08-30T08:59:46.663584+0000","last_undegraded":"2024-08-30T08:59:46.663584+0000","last_fullsized":"2024-08-30T08:59:46.663584+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T11:56:38.827406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T08:59:46.701544+0000","last_change":"2024-08-30T08:58:09.447758+0000","last_active":"2024-08-30T08:59:46.701544+0000","last_peered":"2024-08-30T08:59:46.701544+0000","last_clean":"2024-08-30T08:59:46.701544+0000","last_became_active":"2024-08-30T08:58:09.447333+0000","last_became_peered":"2024-08-30T08:58:09.447333+0000","last_unstale":"2024-08-30T08:59:46.701544+0000","last_undegraded":"2024-08-30T08:59:46.701544+0000","last_fullsized":"2024-08-30T08:59:46.701544+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T12:43:35.289967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"83'8","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:15.309024+0000","last_change":"2024-08-30T08:58:47.091191+0000","last_active":"2024-08-30T09:00:15.309024+0000","last_peered":"2024-08-30T09:00:15.309024+0000","last_clean":"2024-08-30T09:00:15.309024+0000","last_became_active":"2024-08-30T08:58:47.090826+0000","last_became_peered":"2024-08-30T08:58:47.090826+0000","last_unstale":"2024-08-30T09:00:15.309024+0000","last_undegraded":"2024-08-30T09:00:15.309024+0000","last_fullsized":"2024-08-30T09:00:15.309024+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T17:47:22.862514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"83'5","reported_seq":137,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:20.757169+0000","last_change":"2024-08-30T08:58:00.480651+0000","last_active":"2024-08-30T09:00:20.757169+0000","last_peered":"2024-08-30T09:00:20.757169+0000","last_clean":"2024-08-30T09:00:20.757169+0000","last_became_active":"2024-08-30T08:58:00.480205+0000","last_became_peered":"2024-08-30T08:58:00.480205+0000","last_unstale":"2024-08-30T09:00:20.757169+0000","last_undegraded":"2024-08-30T09:00:20.757169+0000","last_fullsized":"2024-08-30T09:00:20.757169+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T16:28:30.687076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":92,"num_read_kb":92,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T08:59:46.701596+0000","last_change":"2024-08-30T08:58:09.727530+0000","last_active":"2024-08-30T08:59:46.701596+0000","last_peered":"2024-08-30T08:59:46.701596+0000","last_clean":"2024-08-30T08:59:46.701596+0000","last_became_active":"2024-08-30T08:58:09.726965+0000","last_became_peered":"2024-08-30T08:58:09.726965+0000","last_unstale":"2024-08-30T08:59:46.701596+0000","last_undegraded":"2024-08-30T08:59:46.701596+0000","last_fullsized":"2024-08-30T08:59:46.701596+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T12:56:42.885615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"83'5","reported_seq":40,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:15.312193+0000","last_change":"2024-08-30T08:58:02.401673+0000","last_active":"2024-08-30T09:00:15.312193+0000","last_peered":"2024-08-30T09:00:15.312193+0000","last_clean":"2024-08-30T09:00:15.312193+0000","last_became_active":"2024-08-30T08:58:02.398876+0000","last_became_peered":"2024-08-30T08:58:02.398876+0000","last_unstale":"2024-08-30T09:00:15.312193+0000","last_undegraded":"2024-08-30T09:00:15.312193+0000","last_fullsized":"2024-08-30T09:00:15.312193+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T16:13:20.893737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"80'3","reported_seq":43,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T08:59:46.663087+0000","last_change":"2024-08-30T08:58:00.391722+0000","last_active":"2024-08-30T08:59:46.663087+0000","last_peered":"2024-08-30T08:59:46.663087+0000","last_clean":"2024-08-30T08:59:46.663087+0000","last_became_active":"2024-08-30T08:58:00.391325+0000","last_became_peered":"2024-08-30T08:58:00.391325+0000","last_unstale":"2024-08-30T08:59:46.663087+0000","last_undegraded":"2024-08-30T08:59:46.663087+0000","last_fullsized":"2024-08-30T08:59:46.663087+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T11:58:08.189202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T08:59:46.663818+0000","last_change":"2024-08-30T08:58:09.728324+0000","last_active":"2024-08-30T08:59:46.663818+0000","last_peered":"2024-08-30T08:59:46.663818+0000","last_clean":"2024-08-30T08:59:46.663818+0000","last_became_active":"2024-08-30T08:58:09.727520+0000","last_became_peered":"2024-08-30T08:58:09.727520+0000","last_unstale":"2024-08-30T08:59:46.663818+0000","last_undegraded":"2024-08-30T08:59:46.663818+0000","last_fullsized":"2024-08-30T08:59:46.663818+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T10:09:39.046820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"83'8","reported_seq":43,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:15.325800+0000","last_change":"2024-08-30T08:58:02.410826+0000","last_active":"2024-08-30T09:00:15.325800+0000","last_peered":"2024-08-30T09:00:15.325800+0000","last_clean":"2024-08-30T09:00:15.325800+0000","last_became_active":"2024-08-30T08:58:02.410464+0000","last_became_peered":"2024-08-30T08:58:02.410464+0000","last_unstale":"2024-08-30T09:00:15.325800+0000","last_undegraded":"2024-08-30T09:00:15.325800+0000","last_fullsized":"2024-08-30T09:00:15.325800+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T12:14:59.666614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"80'3","reported_seq":45,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T08:59:46.663066+0000","last_change":"2024-08-30T08:58:00.385464+0000","last_active":"2024-08-30T08:59:46.663066+0000","last_peered":"2024-08-30T08:59:46.663066+0000","last_clean":"2024-08-30T08:59:46.663066+0000","last_became_active":"2024-08-30T08:58:00.385083+0000","last_became_peered":"2024-08-30T08:58:00.385083+0000","last_unstale":"2024-08-30T08:59:46.663066+0000","last_undegraded":"2024-08-30T08:59:46.663066+0000","last_fullsized":"2024-08-30T08:59:46.663066+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T16:13:52.304755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":112,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T08:59:46.663146+0000","last_change":"2024-08-30T08:56:49.645080+0000","last_active":"2024-08-30T08:59:46.663146+0000","last_peered":"2024-08-30T08:59:46.663146+0000","last_clean":"2024-08-30T08:59:46.663146+0000","last_became_active":"2024-08-30T08:56:49.639066+0000","last_became_peered":"2024-08-30T08:56:49.639066+0000","last_unstale":"2024-08-30T08:59:46.663146+0000","last_undegraded":"2024-08-30T08:59:46.663146+0000","last_fullsized":"2024-08-30T08:59:46.663146+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:53:27.566846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:53:27.566846+0000","last_clean_scrub_stamp":"2024-08-30T08:53:27.566846+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T20:23:45.723824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"81'1","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T08:59:46.663607+0000","last_change":"2024-08-30T08:58:09.444007+0000","last_active":"2024-08-30T08:59:46.663607+0000","last_peered":"2024-08-30T08:59:46.663607+0000","last_clean":"2024-08-30T08:59:46.663607+0000","last_became_active":"2024-08-30T08:58:09.443580+0000","last_became_peered":"2024-08-30T08:58:09.443580+0000","last_unstale":"2024-08-30T08:59:46.663607+0000","last_undegraded":"2024-08-30T08:59:46.663607+0000","last_fullsized":"2024-08-30T08:59:46.663607+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T17:00:58.654026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"83'7","reported_seq":42,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:15.335454+0000","last_change":"2024-08-30T08:58:02.406973+0000","last_active":"2024-08-30T09:00:15.335454+0000","last_peered":"2024-08-30T09:00:15.335454+0000","last_clean":"2024-08-30T09:00:15.335454+0000","last_became_active":"2024-08-30T08:58:02.406576+0000","last_became_peered":"2024-08-30T08:58:02.406576+0000","last_unstale":"2024-08-30T09:00:15.335454+0000","last_undegraded":"2024-08-30T09:00:15.335454+0000","last_fullsized":"2024-08-30T09:00:15.335454+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T16:04:48.785310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"80'3","reported_seq":44,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T08:59:46.657731+0000","last_change":"2024-08-30T08:58:00.481019+0000","last_active":"2024-08-30T08:59:46.657731+0000","last_peered":"2024-08-30T08:59:46.657731+0000","last_clean":"2024-08-30T08:59:46.657731+0000","last_became_active":"2024-08-30T08:58:00.480590+0000","last_became_peered":"2024-08-30T08:58:00.480590+0000","last_unstale":"2024-08-30T08:59:46.657731+0000","last_undegraded":"2024-08-30T08:59:46.657731+0000","last_fullsized":"2024-08-30T08:59:46.657731+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T13:20:46.731591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T08:59:46.664023+0000","last_change":"2024-08-30T08:58:09.445723+0000","last_active":"2024-08-30T08:59:46.664023+0000","last_peered":"2024-08-30T08:59:46.664023+0000","last_clean":"2024-08-30T08:59:46.664023+0000","last_became_active":"2024-08-30T08:58:09.445252+0000","last_became_peered":"2024-08-30T08:58:09.445252+0000","last_unstale":"2024-08-30T08:59:46.664023+0000","last_undegraded":"2024-08-30T08:59:46.664023+0000","last_fullsized":"2024-08-30T08:59:46.664023+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T12:16:12.255990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"83'6","reported_seq":41,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:15.331469+0000","last_change":"2024-08-30T08:58:02.414250+0000","last_active":"2024-08-30T09:00:15.331469+0000","last_peered":"2024-08-30T09:00:15.331469+0000","last_clean":"2024-08-30T09:00:15.331469+0000","last_became_active":"2024-08-30T08:58:02.413684+0000","last_became_peered":"2024-08-30T08:58:02.413684+0000","last_unstale":"2024-08-30T09:00:15.331469+0000","last_undegraded":"2024-08-30T09:00:15.331469+0000","last_fullsized":"2024-08-30T09:00:15.331469+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T16:55:12.878806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"73'1","reported_seq":42,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T08:59:46.664107+0000","last_change":"2024-08-30T08:58:00.388869+0000","last_active":"2024-08-30T08:59:46.664107+0000","last_peered":"2024-08-30T08:59:46.664107+0000","last_clean":"2024-08-30T08:59:46.664107+0000","last_became_active":"2024-08-30T08:58:00.388116+0000","last_became_peered":"2024-08-30T08:58:00.388116+0000","last_unstale":"2024-08-30T08:59:46.664107+0000","last_undegraded":"2024-08-30T08:59:46.664107+0000","last_fullsized":"2024-08-30T08:59:46.664107+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T09:50:48.449205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T08:59:46.842285+0000","last_change":"2024-08-30T08:58:47.092004+0000","last_active":"2024-08-30T08:59:46.842285+0000","last_peered":"2024-08-30T08:59:46.842285+0000","last_clean":"2024-08-30T08:59:46.842285+0000","last_became_active":"2024-08-30T08:58:47.091661+0000","last_became_peered":"2024-08-30T08:58:47.091661+0000","last_unstale":"2024-08-30T08:59:46.842285+0000","last_undegraded":"2024-08-30T08:59:46.842285+0000","last_fullsized":"2024-08-30T08:59:46.842285+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T09:14:55.353465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,1,4],"acting":[0,8,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"83'9","reported_seq":44,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:15.331298+0000","last_change":"2024-08-30T08:58:02.410891+0000","last_active":"2024-08-30T09:00:15.331298+0000","last_peered":"2024-08-30T09:00:15.331298+0000","last_clean":"2024-08-30T09:00:15.331298+0000","last_became_active":"2024-08-30T08:58:02.410522+0000","last_became_peered":"2024-08-30T08:58:02.410522+0000","last_unstale":"2024-08-30T09:00:15.331298+0000","last_undegraded":"2024-08-30T09:00:15.331298+0000","last_fullsized":"2024-08-30T09:00:15.331298+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T09:32:21.540298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"73'2","reported_seq":53,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T08:59:46.702026+0000","last_change":"2024-08-30T08:58:47.095801+0000","last_active":"2024-08-30T08:59:46.702026+0000","last_peered":"2024-08-30T08:59:46.702026+0000","last_clean":"2024-08-30T08:59:46.702026+0000","last_became_active":"2024-08-30T08:58:47.092756+0000","last_became_peered":"2024-08-30T08:58:47.092756+0000","last_unstale":"2024-08-30T08:59:46.702026+0000","last_undegraded":"2024-08-30T08:59:46.702026+0000","last_fullsized":"2024-08-30T08:59:46.702026+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T19:10:02.064649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T08:59:46.663000+0000","last_change":"2024-08-30T08:58:09.449210+0000","last_active":"2024-08-30T08:59:46.663000+0000","last_peered":"2024-08-30T08:59:46.663000+0000","last_clean":"2024-08-30T08:59:46.663000+0000","last_became_active":"2024-08-30T08:58:09.448769+0000","last_became_peered":"2024-08-30T08:58:09.448769+0000","last_unstale":"2024-08-30T08:59:46.663000+0000","last_undegraded":"2024-08-30T08:59:46.663000+0000","last_fullsized":"2024-08-30T08:59:46.663000+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T09:26:13.845520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"83'5","reported_seq":40,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:15.316813+0000","last_change":"2024-08-30T08:58:02.414152+0000","last_active":"2024-08-30T09:00:15.316813+0000","last_peered":"2024-08-30T09:00:15.316813+0000","last_clean":"2024-08-30T09:00:15.316813+0000","last_became_active":"2024-08-30T08:58:02.413768+0000","last_became_peered":"2024-08-30T08:58:02.413768+0000","last_unstale":"2024-08-30T09:00:15.316813+0000","last_undegraded":"2024-08-30T09:00:15.316813+0000","last_fullsized":"2024-08-30T09:00:15.316813+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T18:34:09.447071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"81'2","reported_seq":43,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T08:59:46.660350+0000","last_change":"2024-08-30T08:58:00.390839+0000","last_active":"2024-08-30T08:59:46.660350+0000","last_peered":"2024-08-30T08:59:46.660350+0000","last_clean":"2024-08-30T08:59:46.660350+0000","last_became_active":"2024-08-30T08:58:00.390422+0000","last_became_peered":"2024-08-30T08:58:00.390422+0000","last_unstale":"2024-08-30T08:59:46.660350+0000","last_undegraded":"2024-08-30T08:59:46.660350+0000","last_fullsized":"2024-08-30T08:59:46.660350+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T15:05:23.005075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T08:59:46.659347+0000","last_change":"2024-08-30T08:58:46.824563+0000","last_active":"2024-08-30T08:59:46.659347+0000","last_peered":"2024-08-30T08:59:46.659347+0000","last_clean":"2024-08-30T08:59:46.659347+0000","last_became_active":"2024-08-30T08:58:46.824163+0000","last_became_peered":"2024-08-30T08:58:46.824163+0000","last_unstale":"2024-08-30T08:59:46.659347+0000","last_undegraded":"2024-08-30T08:59:46.659347+0000","last_fullsized":"2024-08-30T08:59:46.659347+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T19:25:13.202762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,3,1],"acting":[8,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"83'6","reported_seq":41,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:15.315507+0000","last_change":"2024-08-30T08:58:02.405122+0000","last_active":"2024-08-30T09:00:15.315507+0000","last_peered":"2024-08-30T09:00:15.315507+0000","last_clean":"2024-08-30T09:00:15.315507+0000","last_became_active":"2024-08-30T08:58:02.404695+0000","last_became_peered":"2024-08-30T08:58:02.404695+0000","last_unstale":"2024-08-30T09:00:15.315507+0000","last_undegraded":"2024-08-30T09:00:15.315507+0000","last_fullsized":"2024-08-30T09:00:15.315507+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T15:50:01.447032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"81'15","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T08:59:46.660605+0000","last_change":"2024-08-30T08:58:46.573221+0000","last_active":"2024-08-30T08:59:46.660605+0000","last_peered":"2024-08-30T08:59:46.660605+0000","last_clean":"2024-08-30T08:59:46.660605+0000","last_became_active":"2024-08-30T08:58:46.559562+0000","last_became_peered":"2024-08-30T08:58:46.559562+0000","last_unstale":"2024-08-30T08:59:46.660605+0000","last_undegraded":"2024-08-30T08:59:46.660605+0000","last_fullsized":"2024-08-30T08:59:46.660605+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T16:21:23.182676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T08:59:46.701861+0000","last_change":"2024-08-30T08:58:09.446063+0000","last_active":"2024-08-30T08:59:46.701861+0000","last_peered":"2024-08-30T08:59:46.701861+0000","last_clean":"2024-08-30T08:59:46.701861+0000","last_became_active":"2024-08-30T08:58:09.445615+0000","last_became_peered":"2024-08-30T08:58:09.445615+0000","last_unstale":"2024-08-30T08:59:46.701861+0000","last_undegraded":"2024-08-30T08:59:46.701861+0000","last_fullsized":"2024-08-30T08:59:46.701861+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T10:25:10.904892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"83'6","reported_seq":41,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:15.330498+0000","last_change":"2024-08-30T08:58:02.412178+0000","last_active":"2024-08-30T09:00:15.330498+0000","last_peered":"2024-08-30T09:00:15.330498+0000","last_clean":"2024-08-30T09:00:15.330498+0000","last_became_active":"2024-08-30T08:58:02.411814+0000","last_became_peered":"2024-08-30T08:58:02.411814+0000","last_unstale":"2024-08-30T09:00:15.330498+0000","last_undegraded":"2024-08-30T09:00:15.330498+0000","last_fullsized":"2024-08-30T09:00:15.330498+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T10:02:59.070204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"83'5","reported_seq":144,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:24.764937+0000","last_change":"2024-08-30T08:58:00.390231+0000","last_active":"2024-08-30T09:00:24.764937+0000","last_peered":"2024-08-30T09:00:24.764937+0000","last_clean":"2024-08-30T09:00:24.764937+0000","last_became_active":"2024-08-30T08:58:00.389852+0000","last_became_peered":"2024-08-30T08:58:00.389852+0000","last_unstale":"2024-08-30T09:00:24.764937+0000","last_undegraded":"2024-08-30T09:00:24.764937+0000","last_fullsized":"2024-08-30T09:00:24.764937+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T12:09:12.105416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":100,"num_read_kb":4,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T08:59:46.656215+0000","last_change":"2024-08-30T08:58:09.441652+0000","last_active":"2024-08-30T08:59:46.656215+0000","last_peered":"2024-08-30T08:59:46.656215+0000","last_clean":"2024-08-30T08:59:46.656215+0000","last_became_active":"2024-08-30T08:58:09.441264+0000","last_became_peered":"2024-08-30T08:58:09.441264+0000","last_unstale":"2024-08-30T08:59:46.656215+0000","last_undegraded":"2024-08-30T08:59:46.656215+0000","last_fullsized":"2024-08-30T08:59:46.656215+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T09:42:25.636669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"83'15","reported_seq":50,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:15.332517+0000","last_change":"2024-08-30T08:58:02.416347+0000","last_active":"2024-08-30T09:00:15.332517+0000","last_peered":"2024-08-30T09:00:15.332517+0000","last_clean":"2024-08-30T09:00:15.332517+0000","last_became_active":"2024-08-30T08:58:02.415898+0000","last_became_peered":"2024-08-30T08:58:02.415898+0000","last_unstale":"2024-08-30T09:00:15.332517+0000","last_undegraded":"2024-08-30T09:00:15.332517+0000","last_fullsized":"2024-08-30T09:00:15.332517+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T10:18:15.507076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"79'3","reported_seq":43,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T08:59:46.663379+0000","last_change":"2024-08-30T08:58:00.393616+0000","last_active":"2024-08-30T08:59:46.663379+0000","last_peered":"2024-08-30T08:59:46.663379+0000","last_clean":"2024-08-30T08:59:46.663379+0000","last_became_active":"2024-08-30T08:58:00.393218+0000","last_became_peered":"2024-08-30T08:58:00.393218+0000","last_unstale":"2024-08-30T08:59:46.663379+0000","last_undegraded":"2024-08-30T08:59:46.663379+0000","last_fullsized":"2024-08-30T08:59:46.663379+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T10:56:37.175968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T08:59:46.661446+0000","last_change":"2024-08-30T08:58:47.092990+0000","last_active":"2024-08-30T08:59:46.661446+0000","last_peered":"2024-08-30T08:59:46.661446+0000","last_clean":"2024-08-30T08:59:46.661446+0000","last_became_active":"2024-08-30T08:58:47.092448+0000","last_became_peered":"2024-08-30T08:58:47.092448+0000","last_unstale":"2024-08-30T08:59:46.661446+0000","last_undegraded":"2024-08-30T08:59:46.661446+0000","last_fullsized":"2024-08-30T08:59:46.661446+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T20:23:06.527336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,7,6],"acting":[11,10,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"83'9","reported_seq":44,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:15.317768+0000","last_change":"2024-08-30T08:58:02.405783+0000","last_active":"2024-08-30T09:00:15.317768+0000","last_peered":"2024-08-30T09:00:15.317768+0000","last_clean":"2024-08-30T09:00:15.317768+0000","last_became_active":"2024-08-30T08:58:02.405430+0000","last_became_peered":"2024-08-30T08:58:02.405430+0000","last_unstale":"2024-08-30T09:00:15.317768+0000","last_undegraded":"2024-08-30T09:00:15.317768+0000","last_fullsized":"2024-08-30T09:00:15.317768+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T19:40:08.377259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"80'6","reported_seq":47,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T08:59:46.657435+0000","last_change":"2024-08-30T08:58:00.389617+0000","last_active":"2024-08-30T08:59:46.657435+0000","last_peered":"2024-08-30T08:59:46.657435+0000","last_clean":"2024-08-30T08:59:46.657435+0000","last_became_active":"2024-08-30T08:58:00.389031+0000","last_became_peered":"2024-08-30T08:58:00.389031+0000","last_unstale":"2024-08-30T08:59:46.657435+0000","last_undegraded":"2024-08-30T08:59:46.657435+0000","last_fullsized":"2024-08-30T08:59:46.657435+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T15:42:38.050768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T08:59:46.663773+0000","last_change":"2024-08-30T08:58:09.451925+0000","last_active":"2024-08-30T08:59:46.663773+0000","last_peered":"2024-08-30T08:59:46.663773+0000","last_clean":"2024-08-30T08:59:46.663773+0000","last_became_active":"2024-08-30T08:58:09.446893+0000","last_became_peered":"2024-08-30T08:58:09.446893+0000","last_unstale":"2024-08-30T08:59:46.663773+0000","last_undegraded":"2024-08-30T08:59:46.663773+0000","last_fullsized":"2024-08-30T08:59:46.663773+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T14:34:25.134006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"83'6","reported_seq":41,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:15.315731+0000","last_change":"2024-08-30T08:58:02.411194+0000","last_active":"2024-08-30T09:00:15.315731+0000","last_peered":"2024-08-30T09:00:15.315731+0000","last_clean":"2024-08-30T09:00:15.315731+0000","last_became_active":"2024-08-30T08:58:02.410183+0000","last_became_peered":"2024-08-30T08:58:02.410183+0000","last_unstale":"2024-08-30T09:00:15.315731+0000","last_undegraded":"2024-08-30T09:00:15.315731+0000","last_fullsized":"2024-08-30T09:00:15.315731+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T10:56:19.016115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"83'37","reported_seq":271,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:21.760384+0000","last_change":"2024-08-30T08:58:00.389808+0000","last_active":"2024-08-30T09:00:21.760384+0000","last_peered":"2024-08-30T09:00:21.760384+0000","last_clean":"2024-08-30T09:00:21.760384+0000","last_became_active":"2024-08-30T08:58:00.389411+0000","last_became_peered":"2024-08-30T08:58:00.389411+0000","last_unstale":"2024-08-30T09:00:21.760384+0000","last_undegraded":"2024-08-30T09:00:21.760384+0000","last_fullsized":"2024-08-30T09:00:21.760384+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T18:59:45.468456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64309,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":194,"num_read_kb":160,"num_write":38,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T08:59:46.655971+0000","last_change":"2024-08-30T08:58:09.447360+0000","last_active":"2024-08-30T08:59:46.655971+0000","last_peered":"2024-08-30T08:59:46.655971+0000","last_clean":"2024-08-30T08:59:46.655971+0000","last_became_active":"2024-08-30T08:58:09.446913+0000","last_became_peered":"2024-08-30T08:58:09.446913+0000","last_unstale":"2024-08-30T08:59:46.655971+0000","last_undegraded":"2024-08-30T08:59:46.655971+0000","last_fullsized":"2024-08-30T08:59:46.655971+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T13:59:43.147370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"83'11","reported_seq":46,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:15.334813+0000","last_change":"2024-08-30T08:58:02.408920+0000","last_active":"2024-08-30T09:00:15.334813+0000","last_peered":"2024-08-30T09:00:15.334813+0000","last_clean":"2024-08-30T09:00:15.334813+0000","last_became_active":"2024-08-30T08:58:02.408204+0000","last_became_peered":"2024-08-30T08:58:02.408204+0000","last_unstale":"2024-08-30T09:00:15.334813+0000","last_undegraded":"2024-08-30T09:00:15.334813+0000","last_fullsized":"2024-08-30T09:00:15.334813+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T11:44:00.765672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"79'4","reported_seq":44,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T08:59:46.663960+0000","last_change":"2024-08-30T08:58:00.482348+0000","last_active":"2024-08-30T08:59:46.663960+0000","last_peered":"2024-08-30T08:59:46.663960+0000","last_clean":"2024-08-30T08:59:46.663960+0000","last_became_active":"2024-08-30T08:58:00.481929+0000","last_became_peered":"2024-08-30T08:58:00.481929+0000","last_unstale":"2024-08-30T08:59:46.663960+0000","last_undegraded":"2024-08-30T08:59:46.663960+0000","last_fullsized":"2024-08-30T08:59:46.663960+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T11:13:55.332639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T08:59:46.662774+0000","last_change":"2024-08-30T08:58:09.443259+0000","last_active":"2024-08-30T08:59:46.662774+0000","last_peered":"2024-08-30T08:59:46.662774+0000","last_clean":"2024-08-30T08:59:46.662774+0000","last_became_active":"2024-08-30T08:58:09.442209+0000","last_became_peered":"2024-08-30T08:58:09.442209+0000","last_unstale":"2024-08-30T08:59:46.662774+0000","last_undegraded":"2024-08-30T08:59:46.662774+0000","last_fullsized":"2024-08-30T08:59:46.662774+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T09:25:56.233525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"83'9","reported_seq":43,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:15.333201+0000","last_change":"2024-08-30T08:58:02.407183+0000","last_active":"2024-08-30T09:00:15.333201+0000","last_peered":"2024-08-30T09:00:15.333201+0000","last_clean":"2024-08-30T09:00:15.333201+0000","last_became_active":"2024-08-30T08:58:02.406838+0000","last_became_peered":"2024-08-30T08:58:02.406838+0000","last_unstale":"2024-08-30T09:00:15.333201+0000","last_undegraded":"2024-08-30T09:00:15.333201+0000","last_fullsized":"2024-08-30T09:00:15.333201+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T13:40:32.937626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"79'2","reported_seq":41,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T08:59:46.663225+0000","last_change":"2024-08-30T08:58:00.392322+0000","last_active":"2024-08-30T08:59:46.663225+0000","last_peered":"2024-08-30T08:59:46.663225+0000","last_clean":"2024-08-30T08:59:46.663225+0000","last_became_active":"2024-08-30T08:58:00.391935+0000","last_became_peered":"2024-08-30T08:58:00.391935+0000","last_unstale":"2024-08-30T08:59:46.663225+0000","last_undegraded":"2024-08-30T08:59:46.663225+0000","last_fullsized":"2024-08-30T08:59:46.663225+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T12:20:52.368285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T08:59:46.655822+0000","last_change":"2024-08-30T08:58:09.441824+0000","last_active":"2024-08-30T08:59:46.655822+0000","last_peered":"2024-08-30T08:59:46.655822+0000","last_clean":"2024-08-30T08:59:46.655822+0000","last_became_active":"2024-08-30T08:58:09.441398+0000","last_became_peered":"2024-08-30T08:58:09.441398+0000","last_unstale":"2024-08-30T08:59:46.655822+0000","last_undegraded":"2024-08-30T08:59:46.655822+0000","last_fullsized":"2024-08-30T08:59:46.655822+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T14:05:45.710280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"83'10","reported_seq":44,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:15.334283+0000","last_change":"2024-08-30T08:58:02.403431+0000","last_active":"2024-08-30T09:00:15.334283+0000","last_peered":"2024-08-30T09:00:15.334283+0000","last_clean":"2024-08-30T09:00:15.334283+0000","last_became_active":"2024-08-30T08:58:02.403029+0000","last_became_peered":"2024-08-30T08:58:02.403029+0000","last_unstale":"2024-08-30T09:00:15.334283+0000","last_undegraded":"2024-08-30T09:00:15.334283+0000","last_fullsized":"2024-08-30T09:00:15.334283+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T09:34:56.452675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"83'4","reported_seq":135,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:20.757880+0000","last_change":"2024-08-30T08:58:00.389856+0000","last_active":"2024-08-30T09:00:20.757880+0000","last_peered":"2024-08-30T09:00:20.757880+0000","last_clean":"2024-08-30T09:00:20.757880+0000","last_became_active":"2024-08-30T08:58:00.389459+0000","last_became_peered":"2024-08-30T08:58:00.389459+0000","last_unstale":"2024-08-30T09:00:20.757880+0000","last_undegraded":"2024-08-30T09:00:20.757880+0000","last_fullsized":"2024-08-30T09:00:20.757880+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T18:32:06.214570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":92,"num_read_kb":92,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T08:59:46.701213+0000","last_change":"2024-08-30T08:58:09.445317+0000","last_active":"2024-08-30T08:59:46.701213+0000","last_peered":"2024-08-30T08:59:46.701213+0000","last_clean":"2024-08-30T08:59:46.701213+0000","last_became_active":"2024-08-30T08:58:09.444921+0000","last_became_peered":"2024-08-30T08:58:09.444921+0000","last_unstale":"2024-08-30T08:59:46.701213+0000","last_undegraded":"2024-08-30T08:59:46.701213+0000","last_fullsized":"2024-08-30T08:59:46.701213+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T09:22:13.038437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T08:59:46.842401+0000","last_change":"2024-08-30T08:58:09.447469+0000","last_active":"2024-08-30T08:59:46.842401+0000","last_peered":"2024-08-30T08:59:46.842401+0000","last_clean":"2024-08-30T08:59:46.842401+0000","last_became_active":"2024-08-30T08:58:09.447015+0000","last_became_peered":"2024-08-30T08:58:09.447015+0000","last_unstale":"2024-08-30T08:59:46.842401+0000","last_undegraded":"2024-08-30T08:59:46.842401+0000","last_fullsized":"2024-08-30T08:59:46.842401+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T09:18:36.734380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"83'11","reported_seq":45,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:15.317124+0000","last_change":"2024-08-30T08:58:02.403371+0000","last_active":"2024-08-30T09:00:15.317124+0000","last_peered":"2024-08-30T09:00:15.317124+0000","last_clean":"2024-08-30T09:00:15.317124+0000","last_became_active":"2024-08-30T08:58:02.403012+0000","last_became_peered":"2024-08-30T08:58:02.403012+0000","last_unstale":"2024-08-30T09:00:15.317124+0000","last_undegraded":"2024-08-30T09:00:15.317124+0000","last_fullsized":"2024-08-30T09:00:15.317124+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T10:29:29.683081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"79'1","reported_seq":40,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T08:59:46.663377+0000","last_change":"2024-08-30T08:58:00.394450+0000","last_active":"2024-08-30T08:59:46.663377+0000","last_peered":"2024-08-30T08:59:46.663377+0000","last_clean":"2024-08-30T08:59:46.663377+0000","last_became_active":"2024-08-30T08:58:00.391843+0000","last_became_peered":"2024-08-30T08:58:00.391843+0000","last_unstale":"2024-08-30T08:59:46.663377+0000","last_undegraded":"2024-08-30T08:59:46.663377+0000","last_fullsized":"2024-08-30T08:59:46.663377+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T10:29:18.947898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T08:59:46.661222+0000","last_change":"2024-08-30T08:58:09.449172+0000","last_active":"2024-08-30T08:59:46.661222+0000","last_peered":"2024-08-30T08:59:46.661222+0000","last_clean":"2024-08-30T08:59:46.661222+0000","last_became_active":"2024-08-30T08:58:09.448687+0000","last_became_peered":"2024-08-30T08:58:09.448687+0000","last_unstale":"2024-08-30T08:59:46.661222+0000","last_undegraded":"2024-08-30T08:59:46.661222+0000","last_fullsized":"2024-08-30T08:59:46.661222+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T13:23:16.090251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"83'8","reported_seq":42,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:15.315455+0000","last_change":"2024-08-30T08:58:02.400834+0000","last_active":"2024-08-30T09:00:15.315455+0000","last_peered":"2024-08-30T09:00:15.315455+0000","last_clean":"2024-08-30T09:00:15.315455+0000","last_became_active":"2024-08-30T08:58:02.400477+0000","last_became_peered":"2024-08-30T08:58:02.400477+0000","last_unstale":"2024-08-30T09:00:15.315455+0000","last_undegraded":"2024-08-30T09:00:15.315455+0000","last_fullsized":"2024-08-30T09:00:15.315455+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T13:40:45.239732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"80'2","reported_seq":41,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T08:59:46.700963+0000","last_change":"2024-08-30T08:58:00.381699+0000","last_active":"2024-08-30T08:59:46.700963+0000","last_peered":"2024-08-30T08:59:46.700963+0000","last_clean":"2024-08-30T08:59:46.700963+0000","last_became_active":"2024-08-30T08:58:00.381189+0000","last_became_peered":"2024-08-30T08:58:00.381189+0000","last_unstale":"2024-08-30T08:59:46.700963+0000","last_undegraded":"2024-08-30T08:59:46.700963+0000","last_fullsized":"2024-08-30T08:59:46.700963+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T14:09:16.939062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"80'3","reported_seq":42,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T08:59:46.697436+0000","last_change":"2024-08-30T08:58:00.387430+0000","last_active":"2024-08-30T08:59:46.697436+0000","last_peered":"2024-08-30T08:59:46.697436+0000","last_clean":"2024-08-30T08:59:46.697436+0000","last_became_active":"2024-08-30T08:58:00.387043+0000","last_became_peered":"2024-08-30T08:58:00.387043+0000","last_unstale":"2024-08-30T08:59:46.697436+0000","last_undegraded":"2024-08-30T08:59:46.697436+0000","last_fullsized":"2024-08-30T08:59:46.697436+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T13:32:25.418105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"83'8","reported_seq":42,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:15.316380+0000","last_change":"2024-08-30T08:58:02.404798+0000","last_active":"2024-08-30T09:00:15.316380+0000","last_peered":"2024-08-30T09:00:15.316380+0000","last_clean":"2024-08-30T09:00:15.316380+0000","last_became_active":"2024-08-30T08:58:02.404435+0000","last_became_peered":"2024-08-30T08:58:02.404435+0000","last_unstale":"2024-08-30T09:00:15.316380+0000","last_undegraded":"2024-08-30T09:00:15.316380+0000","last_fullsized":"2024-08-30T09:00:15.316380+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T20:31:49.472832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T08:59:46.657368+0000","last_change":"2024-08-30T08:58:09.448297+0000","last_active":"2024-08-30T08:59:46.657368+0000","last_peered":"2024-08-30T08:59:46.657368+0000","last_clean":"2024-08-30T08:59:46.657368+0000","last_became_active":"2024-08-30T08:58:09.447444+0000","last_became_peered":"2024-08-30T08:58:09.447444+0000","last_unstale":"2024-08-30T08:59:46.657368+0000","last_undegraded":"2024-08-30T08:59:46.657368+0000","last_fullsized":"2024-08-30T08:59:46.657368+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T11:40:03.507824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"83'7","reported_seq":141,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:20.758486+0000","last_change":"2024-08-30T08:58:00.392418+0000","last_active":"2024-08-30T09:00:20.758486+0000","last_peered":"2024-08-30T09:00:20.758486+0000","last_clean":"2024-08-30T09:00:20.758486+0000","last_became_active":"2024-08-30T08:58:00.391985+0000","last_became_peered":"2024-08-30T08:58:00.391985+0000","last_unstale":"2024-08-30T09:00:20.758486+0000","last_undegraded":"2024-08-30T09:00:20.758486+0000","last_fullsized":"2024-08-30T09:00:20.758486+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T17:42:18.147754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":95,"num_read_kb":3,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"83'7","reported_seq":41,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:15.300043+0000","last_change":"2024-08-30T08:58:02.408754+0000","last_active":"2024-08-30T09:00:15.300043+0000","last_peered":"2024-08-30T09:00:15.300043+0000","last_clean":"2024-08-30T09:00:15.300043+0000","last_became_active":"2024-08-30T08:58:02.407127+0000","last_became_peered":"2024-08-30T08:58:02.407127+0000","last_unstale":"2024-08-30T09:00:15.300043+0000","last_undegraded":"2024-08-30T09:00:15.300043+0000","last_fullsized":"2024-08-30T09:00:15.300043+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T16:15:41.796091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T08:59:46.657646+0000","last_change":"2024-08-30T08:58:09.445389+0000","last_active":"2024-08-30T08:59:46.657646+0000","last_peered":"2024-08-30T08:59:46.657646+0000","last_clean":"2024-08-30T08:59:46.657646+0000","last_became_active":"2024-08-30T08:58:09.444831+0000","last_became_peered":"2024-08-30T08:58:09.444831+0000","last_unstale":"2024-08-30T08:59:46.657646+0000","last_undegraded":"2024-08-30T08:59:46.657646+0000","last_fullsized":"2024-08-30T08:59:46.657646+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T09:55:58.162397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"83'9","reported_seq":44,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:15.317322+0000","last_change":"2024-08-30T08:58:02.415184+0000","last_active":"2024-08-30T09:00:15.317322+0000","last_peered":"2024-08-30T09:00:15.317322+0000","last_clean":"2024-08-30T09:00:15.317322+0000","last_became_active":"2024-08-30T08:58:02.414625+0000","last_became_peered":"2024-08-30T08:58:02.414625+0000","last_unstale":"2024-08-30T09:00:15.317322+0000","last_undegraded":"2024-08-30T09:00:15.317322+0000","last_fullsized":"2024-08-30T09:00:15.317322+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T11:10:31.130524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"83'12","reported_seq":47,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:15.315361+0000","last_change":"2024-08-30T08:58:02.405392+0000","last_active":"2024-08-30T09:00:15.315361+0000","last_peered":"2024-08-30T09:00:15.315361+0000","last_clean":"2024-08-30T09:00:15.315361+0000","last_became_active":"2024-08-30T08:58:02.405009+0000","last_became_peered":"2024-08-30T08:58:02.405009+0000","last_unstale":"2024-08-30T09:00:15.315361+0000","last_undegraded":"2024-08-30T09:00:15.315361+0000","last_fullsized":"2024-08-30T09:00:15.315361+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T13:22:46.375598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"83'8","reported_seq":43,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:15.315926+0000","last_change":"2024-08-30T08:58:02.402360+0000","last_active":"2024-08-30T09:00:15.315926+0000","last_peered":"2024-08-30T09:00:15.315926+0000","last_clean":"2024-08-30T09:00:15.315926+0000","last_became_active":"2024-08-30T08:58:02.401856+0000","last_became_peered":"2024-08-30T08:58:02.401856+0000","last_unstale":"2024-08-30T09:00:15.315926+0000","last_undegraded":"2024-08-30T09:00:15.315926+0000","last_fullsized":"2024-08-30T09:00:15.315926+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T10:25:14.491345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"83'7","reported_seq":42,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:15.332529+0000","last_change":"2024-08-30T08:58:02.415940+0000","last_active":"2024-08-30T09:00:15.332529+0000","last_peered":"2024-08-30T09:00:15.332529+0000","last_clean":"2024-08-30T09:00:15.332529+0000","last_became_active":"2024-08-30T08:58:02.415567+0000","last_became_peered":"2024-08-30T08:58:02.415567+0000","last_unstale":"2024-08-30T09:00:15.332529+0000","last_undegraded":"2024-08-30T09:00:15.332529+0000","last_fullsized":"2024-08-30T09:00:15.332529+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T15:02:26.270343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"83'4","reported_seq":39,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:15.312692+0000","last_change":"2024-08-30T08:58:02.406766+0000","last_active":"2024-08-30T09:00:15.312692+0000","last_peered":"2024-08-30T09:00:15.312692+0000","last_clean":"2024-08-30T09:00:15.312692+0000","last_became_active":"2024-08-30T08:58:02.406345+0000","last_became_peered":"2024-08-30T08:58:02.406345+0000","last_unstale":"2024-08-30T09:00:15.312692+0000","last_undegraded":"2024-08-30T09:00:15.312692+0000","last_fullsized":"2024-08-30T09:00:15.312692+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T13:10:32.986817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"83'6","reported_seq":41,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:15.313675+0000","last_change":"2024-08-30T08:58:02.412891+0000","last_active":"2024-08-30T09:00:15.313675+0000","last_peered":"2024-08-30T09:00:15.313675+0000","last_clean":"2024-08-30T09:00:15.313675+0000","last_became_active":"2024-08-30T08:58:02.412532+0000","last_became_peered":"2024-08-30T08:58:02.412532+0000","last_unstale":"2024-08-30T09:00:15.313675+0000","last_undegraded":"2024-08-30T09:00:15.313675+0000","last_fullsized":"2024-08-30T09:00:15.313675+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T14:40:30.663678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"83'15","reported_seq":50,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:15.325109+0000","last_change":"2024-08-30T08:58:02.405862+0000","last_active":"2024-08-30T09:00:15.325109+0000","last_peered":"2024-08-30T09:00:15.325109+0000","last_clean":"2024-08-30T09:00:15.325109+0000","last_became_active":"2024-08-30T08:58:02.405319+0000","last_became_peered":"2024-08-30T08:58:02.405319+0000","last_unstale":"2024-08-30T09:00:15.325109+0000","last_undegraded":"2024-08-30T09:00:15.325109+0000","last_fullsized":"2024-08-30T09:00:15.325109+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T20:36:10.315814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"83'2","reported_seq":37,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:15.316363+0000","last_change":"2024-08-30T08:58:02.401410+0000","last_active":"2024-08-30T09:00:15.316363+0000","last_peered":"2024-08-30T09:00:15.316363+0000","last_clean":"2024-08-30T09:00:15.316363+0000","last_became_active":"2024-08-30T08:58:02.401060+0000","last_became_peered":"2024-08-30T08:58:02.401060+0000","last_unstale":"2024-08-30T09:00:15.316363+0000","last_undegraded":"2024-08-30T09:00:15.316363+0000","last_fullsized":"2024-08-30T09:00:15.316363+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T16:00:41.275367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"83'6","reported_seq":41,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:15.306834+0000","last_change":"2024-08-30T08:58:02.401703+0000","last_active":"2024-08-30T09:00:15.306834+0000","last_peered":"2024-08-30T09:00:15.306834+0000","last_clean":"2024-08-30T09:00:15.306834+0000","last_became_active":"2024-08-30T08:58:02.401218+0000","last_became_peered":"2024-08-30T08:58:02.401218+0000","last_unstale":"2024-08-30T09:00:15.306834+0000","last_undegraded":"2024-08-30T09:00:15.306834+0000","last_fullsized":"2024-08-30T09:00:15.306834+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T18:36:25.249814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"83'8","reported_seq":43,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:15.315612+0000","last_change":"2024-08-30T08:58:02.403609+0000","last_active":"2024-08-30T09:00:15.315612+0000","last_peered":"2024-08-30T09:00:15.315612+0000","last_clean":"2024-08-30T09:00:15.315612+0000","last_became_active":"2024-08-30T08:58:02.402195+0000","last_became_peered":"2024-08-30T08:58:02.402195+0000","last_unstale":"2024-08-30T09:00:15.315612+0000","last_undegraded":"2024-08-30T09:00:15.315612+0000","last_fullsized":"2024-08-30T09:00:15.315612+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T09:59:05.877523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"83'10","reported_seq":45,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:15.333885+0000","last_change":"2024-08-30T08:58:02.404389+0000","last_active":"2024-08-30T09:00:15.333885+0000","last_peered":"2024-08-30T09:00:15.333885+0000","last_clean":"2024-08-30T09:00:15.333885+0000","last_became_active":"2024-08-30T08:58:02.404039+0000","last_became_peered":"2024-08-30T08:58:02.404039+0000","last_unstale":"2024-08-30T09:00:15.333885+0000","last_undegraded":"2024-08-30T09:00:15.333885+0000","last_fullsized":"2024-08-30T09:00:15.333885+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T20:55:51.739542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"83'11","reported_seq":46,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:15.301840+0000","last_change":"2024-08-30T08:58:02.411807+0000","last_active":"2024-08-30T09:00:15.301840+0000","last_peered":"2024-08-30T09:00:15.301840+0000","last_clean":"2024-08-30T09:00:15.301840+0000","last_became_active":"2024-08-30T08:58:02.410977+0000","last_became_peered":"2024-08-30T08:58:02.410977+0000","last_unstale":"2024-08-30T09:00:15.301840+0000","last_undegraded":"2024-08-30T09:00:15.301840+0000","last_fullsized":"2024-08-30T09:00:15.301840+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T13:04:29.931373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"83'8","reported_seq":50,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:15.315288+0000","last_change":"2024-08-30T08:58:46.559661+0000","last_active":"2024-08-30T09:00:15.315288+0000","last_peered":"2024-08-30T09:00:15.315288+0000","last_clean":"2024-08-30T09:00:15.315288+0000","last_became_active":"2024-08-30T08:58:46.558824+0000","last_became_peered":"2024-08-30T08:58:46.558824+0000","last_unstale":"2024-08-30T09:00:15.315288+0000","last_undegraded":"2024-08-30T09:00:15.315288+0000","last_fullsized":"2024-08-30T09:00:15.315288+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T09:53:07.650921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"83'7","reported_seq":42,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:15.316869+0000","last_change":"2024-08-30T08:58:02.396186+0000","last_active":"2024-08-30T09:00:15.316869+0000","last_peered":"2024-08-30T09:00:15.316869+0000","last_clean":"2024-08-30T09:00:15.316869+0000","last_became_active":"2024-08-30T08:58:02.395776+0000","last_became_peered":"2024-08-30T08:58:02.395776+0000","last_unstale":"2024-08-30T09:00:15.316869+0000","last_undegraded":"2024-08-30T09:00:15.316869+0000","last_fullsized":"2024-08-30T09:00:15.316869+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T09:33:30.068673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"83'10","reported_seq":45,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:15.318659+0000","last_change":"2024-08-30T08:58:02.408378+0000","last_active":"2024-08-30T09:00:15.318659+0000","last_peered":"2024-08-30T09:00:15.318659+0000","last_clean":"2024-08-30T09:00:15.318659+0000","last_became_active":"2024-08-30T08:58:02.407991+0000","last_became_peered":"2024-08-30T08:58:02.407991+0000","last_unstale":"2024-08-30T09:00:15.318659+0000","last_undegraded":"2024-08-30T09:00:15.318659+0000","last_fullsized":"2024-08-30T09:00:15.318659+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T20:57:02.411176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"83'8","reported_seq":43,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:15.333624+0000","last_change":"2024-08-30T08:58:02.411829+0000","last_active":"2024-08-30T09:00:15.333624+0000","last_peered":"2024-08-30T09:00:15.333624+0000","last_clean":"2024-08-30T09:00:15.333624+0000","last_became_active":"2024-08-30T08:58:02.411416+0000","last_became_peered":"2024-08-30T08:58:02.411416+0000","last_unstale":"2024-08-30T09:00:15.333624+0000","last_undegraded":"2024-08-30T09:00:15.333624+0000","last_fullsized":"2024-08-30T09:00:15.333624+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T17:12:08.855150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"83'9","reported_seq":44,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:15.333466+0000","last_change":"2024-08-30T08:58:02.405122+0000","last_active":"2024-08-30T09:00:15.333466+0000","last_peered":"2024-08-30T09:00:15.333466+0000","last_clean":"2024-08-30T09:00:15.333466+0000","last_became_active":"2024-08-30T08:58:02.403312+0000","last_became_peered":"2024-08-30T08:58:02.403312+0000","last_unstale":"2024-08-30T09:00:15.333466+0000","last_undegraded":"2024-08-30T09:00:15.333466+0000","last_fullsized":"2024-08-30T09:00:15.333466+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T18:11:53.077433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"83'7","reported_seq":42,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:15.332861+0000","last_change":"2024-08-30T08:58:02.404263+0000","last_active":"2024-08-30T09:00:15.332861+0000","last_peered":"2024-08-30T09:00:15.332861+0000","last_clean":"2024-08-30T09:00:15.332861+0000","last_became_active":"2024-08-30T08:58:02.403897+0000","last_became_peered":"2024-08-30T08:58:02.403897+0000","last_unstale":"2024-08-30T09:00:15.332861+0000","last_undegraded":"2024-08-30T09:00:15.332861+0000","last_fullsized":"2024-08-30T09:00:15.332861+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T18:38:31.439434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"83'7","reported_seq":42,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:15.332364+0000","last_change":"2024-08-30T08:58:02.411578+0000","last_active":"2024-08-30T09:00:15.332364+0000","last_peered":"2024-08-30T09:00:15.332364+0000","last_clean":"2024-08-30T09:00:15.332364+0000","last_became_active":"2024-08-30T08:58:02.411271+0000","last_became_peered":"2024-08-30T08:58:02.411271+0000","last_unstale":"2024-08-30T09:00:15.332364+0000","last_undegraded":"2024-08-30T09:00:15.332364+0000","last_fullsized":"2024-08-30T09:00:15.332364+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T10:02:46.382906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"83'10","reported_seq":44,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:15.310639+0000","last_change":"2024-08-30T08:58:02.412168+0000","last_active":"2024-08-30T09:00:15.310639+0000","last_peered":"2024-08-30T09:00:15.310639+0000","last_clean":"2024-08-30T09:00:15.310639+0000","last_became_active":"2024-08-30T08:58:02.411420+0000","last_became_peered":"2024-08-30T08:58:02.411420+0000","last_unstale":"2024-08-30T09:00:15.310639+0000","last_undegraded":"2024-08-30T09:00:15.310639+0000","last_fullsized":"2024-08-30T09:00:15.310639+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T10:35:53.654344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"83'7","reported_seq":42,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:15.318663+0000","last_change":"2024-08-30T08:58:02.409735+0000","last_active":"2024-08-30T09:00:15.318663+0000","last_peered":"2024-08-30T09:00:15.318663+0000","last_clean":"2024-08-30T09:00:15.318663+0000","last_became_active":"2024-08-30T08:58:02.409349+0000","last_became_peered":"2024-08-30T08:58:02.409349+0000","last_unstale":"2024-08-30T09:00:15.318663+0000","last_undegraded":"2024-08-30T09:00:15.318663+0000","last_fullsized":"2024-08-30T09:00:15.318663+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T20:10:22.956554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"83'6","reported_seq":40,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:15.333218+0000","last_change":"2024-08-30T08:58:02.416261+0000","last_active":"2024-08-30T09:00:15.333218+0000","last_peered":"2024-08-30T09:00:15.333218+0000","last_clean":"2024-08-30T09:00:15.333218+0000","last_became_active":"2024-08-30T08:58:02.415194+0000","last_became_peered":"2024-08-30T08:58:02.415194+0000","last_unstale":"2024-08-30T09:00:15.333218+0000","last_undegraded":"2024-08-30T09:00:15.333218+0000","last_fullsized":"2024-08-30T09:00:15.333218+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T17:59:04.339699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"83'16","reported_seq":57,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:15.320857+0000","last_change":"2024-08-30T08:58:46.560190+0000","last_active":"2024-08-30T09:00:15.320857+0000","last_peered":"2024-08-30T09:00:15.320857+0000","last_clean":"2024-08-30T09:00:15.320857+0000","last_became_active":"2024-08-30T08:58:46.559674+0000","last_became_peered":"2024-08-30T08:58:46.559674+0000","last_unstale":"2024-08-30T09:00:15.320857+0000","last_undegraded":"2024-08-30T09:00:15.320857+0000","last_fullsized":"2024-08-30T09:00:15.320857+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T10:08:02.835142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"83'11","reported_seq":45,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:15.332664+0000","last_change":"2024-08-30T08:58:02.406763+0000","last_active":"2024-08-30T09:00:15.332664+0000","last_peered":"2024-08-30T09:00:15.332664+0000","last_clean":"2024-08-30T09:00:15.332664+0000","last_became_active":"2024-08-30T08:58:02.406377+0000","last_became_peered":"2024-08-30T08:58:02.406377+0000","last_unstale":"2024-08-30T09:00:15.332664+0000","last_undegraded":"2024-08-30T09:00:15.332664+0000","last_fullsized":"2024-08-30T09:00:15.332664+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T13:32:50.492921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"83'9","reported_seq":43,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:15.319363+0000","last_change":"2024-08-30T08:58:02.416638+0000","last_active":"2024-08-30T09:00:15.319363+0000","last_peered":"2024-08-30T09:00:15.319363+0000","last_clean":"2024-08-30T09:00:15.319363+0000","last_became_active":"2024-08-30T08:58:02.415518+0000","last_became_peered":"2024-08-30T08:58:02.415518+0000","last_unstale":"2024-08-30T09:00:15.319363+0000","last_undegraded":"2024-08-30T09:00:15.319363+0000","last_fullsized":"2024-08-30T09:00:15.319363+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T17:05:07.531585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"83'5","reported_seq":39,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:15.330612+0000","last_change":"2024-08-30T08:58:02.412973+0000","last_active":"2024-08-30T09:00:15.330612+0000","last_peered":"2024-08-30T09:00:15.330612+0000","last_clean":"2024-08-30T09:00:15.330612+0000","last_became_active":"2024-08-30T08:58:02.412609+0000","last_became_peered":"2024-08-30T08:58:02.412609+0000","last_unstale":"2024-08-30T09:00:15.330612+0000","last_undegraded":"2024-08-30T09:00:15.330612+0000","last_fullsized":"2024-08-30T09:00:15.330612+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T14:22:37.704098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"83'6","reported_seq":40,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:15.310078+0000","last_change":"2024-08-30T08:58:02.402779+0000","last_active":"2024-08-30T09:00:15.310078+0000","last_peered":"2024-08-30T09:00:15.310078+0000","last_clean":"2024-08-30T09:00:15.310078+0000","last_became_active":"2024-08-30T08:58:02.401514+0000","last_became_peered":"2024-08-30T08:58:02.401514+0000","last_unstale":"2024-08-30T09:00:15.310078+0000","last_undegraded":"2024-08-30T09:00:15.310078+0000","last_fullsized":"2024-08-30T09:00:15.310078+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T11:37:51.450334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"83'4","reported_seq":38,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:15.295684+0000","last_change":"2024-08-30T08:58:02.406607+0000","last_active":"2024-08-30T09:00:15.295684+0000","last_peered":"2024-08-30T09:00:15.295684+0000","last_clean":"2024-08-30T09:00:15.295684+0000","last_became_active":"2024-08-30T08:58:02.406237+0000","last_became_peered":"2024-08-30T08:58:02.406237+0000","last_unstale":"2024-08-30T09:00:15.295684+0000","last_undegraded":"2024-08-30T09:00:15.295684+0000","last_fullsized":"2024-08-30T09:00:15.295684+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T12:09:03.247424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"83'8","reported_seq":43,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:15.323344+0000","last_change":"2024-08-30T08:58:02.410749+0000","last_active":"2024-08-30T09:00:15.323344+0000","last_peered":"2024-08-30T09:00:15.323344+0000","last_clean":"2024-08-30T09:00:15.323344+0000","last_became_active":"2024-08-30T08:58:02.410350+0000","last_became_peered":"2024-08-30T08:58:02.410350+0000","last_unstale":"2024-08-30T09:00:15.323344+0000","last_undegraded":"2024-08-30T09:00:15.323344+0000","last_fullsized":"2024-08-30T09:00:15.323344+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T15:22:51.451789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"83'7","reported_seq":42,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:15.334454+0000","last_change":"2024-08-30T08:58:02.405946+0000","last_active":"2024-08-30T09:00:15.334454+0000","last_peered":"2024-08-30T09:00:15.334454+0000","last_clean":"2024-08-30T09:00:15.334454+0000","last_became_active":"2024-08-30T08:58:02.405590+0000","last_became_peered":"2024-08-30T08:58:02.405590+0000","last_unstale":"2024-08-30T09:00:15.334454+0000","last_undegraded":"2024-08-30T09:00:15.334454+0000","last_fullsized":"2024-08-30T09:00:15.334454+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T11:42:14.966065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"83'4","reported_seq":39,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:15.330681+0000","last_change":"2024-08-30T08:58:02.409649+0000","last_active":"2024-08-30T09:00:15.330681+0000","last_peered":"2024-08-30T09:00:15.330681+0000","last_clean":"2024-08-30T09:00:15.330681+0000","last_became_active":"2024-08-30T08:58:02.409282+0000","last_became_peered":"2024-08-30T08:58:02.409282+0000","last_unstale":"2024-08-30T09:00:15.330681+0000","last_undegraded":"2024-08-30T09:00:15.330681+0000","last_fullsized":"2024-08-30T09:00:15.330681+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T15:33:52.166290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"83'8","reported_seq":43,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:15.333394+0000","last_change":"2024-08-30T08:58:02.410711+0000","last_active":"2024-08-30T09:00:15.333394+0000","last_peered":"2024-08-30T09:00:15.333394+0000","last_clean":"2024-08-30T09:00:15.333394+0000","last_became_active":"2024-08-30T08:58:02.410367+0000","last_became_peered":"2024-08-30T08:58:02.410367+0000","last_unstale":"2024-08-30T09:00:15.333394+0000","last_undegraded":"2024-08-30T09:00:15.333394+0000","last_fullsized":"2024-08-30T09:00:15.333394+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T15:27:40.392588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":68983,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":586,"num_read_kb":364,"num_write":180,"num_write_kb":127,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":207257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":154,"ondisk_log_size":154,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":67,"seq":287762808872,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":958464,"data_stored":474260,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":61,"seq":261993005102,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28628,"kb_used_data":1604,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716524,"statfs":{"total":95995035648,"available":95965720576,"internally_reserved":0,"allocated":1642496,"data_stored":1126257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":57,"seq":244813135924,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36292,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708860,"statfs":{"total":95995035648,"available":95957872640,"internally_reserved":0,"allocated":970752,"data_stored":478480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":51,"seq":219043332154,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":958464,"data_stored":474192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":46,"seq":197568495678,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28548,"kb_used_data":1524,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716604,"statfs":{"total":95995035648,"available":95965802496,"internally_reserved":0,"allocated":1560576,"data_stored":1070886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":41,"seq":176093659203,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":946176,"data_stored":469560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":35,"seq":150323855432,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36876,"kb_used_data":1532,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708276,"statfs":{"total":95995035648,"available":95957274624,"internally_reserved":0,"allocated":1568768,"data_stored":1065942,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":30,"seq":128849018958,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36280,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708872,"statfs":{"total":95995035648,"available":95957884928,"internally_reserved":0,"allocated":958464,"data_stored":474742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215186,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36280,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708872,"statfs":{"total":95995035648,"available":95957884928,"internally_reserved":0,"allocated":958464,"data_stored":469754,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411416,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36360,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708792,"statfs":{"total":95995035648,"available":95957803008,"internally_reserved":0,"allocated":1040384,"data_stored":535969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574940,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1015808,"data_stored":533779,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738465,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":970752,"data_stored":470948,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":64309,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":66499,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":66419,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-30T09:00:28.990 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-08-30T09:00:29.135 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:29.133+0000 7fe97b569640 1 -- 172.21.15.42:0/3039253458 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe974102340 msgr2=0x7fe9741027b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:00:29.135 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:29.133+0000 7fe97b569640 1 --2- 172.21.15.42:0/3039253458 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe974102340 0x7fe9741027b0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fe970009900 tx=0x7fe97002a410 comp rx=0 tx=0).stop 2024-08-30T09:00:29.135 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:29.133+0000 7fe97b569640 1 -- 172.21.15.42:0/3039253458 shutdown_connections 2024-08-30T09:00:29.135 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:29.133+0000 7fe97b569640 1 --2- 172.21.15.42:0/3039253458 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe974103d60 0x7fe97410a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:29.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:29.133+0000 7fe97b569640 1 --2- 172.21.15.42:0/3039253458 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe9741033c0 0x7fe9741037d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:29.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:29.133+0000 7fe97b569640 1 --2- 172.21.15.42:0/3039253458 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe974102340 0x7fe9741027b0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:29.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:29.133+0000 7fe97b569640 1 -- 172.21.15.42:0/3039253458 >> 172.21.15.42:0/3039253458 conn(0x7fe9740fdc80 msgr2=0x7fe974100130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:00:29.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:29.133+0000 7fe97b569640 1 -- 172.21.15.42:0/3039253458 shutdown_connections 2024-08-30T09:00:29.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:29.133+0000 7fe97b569640 1 -- 172.21.15.42:0/3039253458 wait complete. 2024-08-30T09:00:29.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:29.134+0000 7fe97b569640 1 Processor -- start 2024-08-30T09:00:29.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:29.134+0000 7fe97b569640 1 -- start start 2024-08-30T09:00:29.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:29.134+0000 7fe97b569640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe974102340 0x7fe974199c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:29.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:29.134+0000 7fe97b569640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe9741033c0 0x7fe97419a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:29.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:29.134+0000 7fe97b569640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe974103d60 0x7fe97419aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:29.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:29.134+0000 7fe97b569640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fe97410dfc0 con 0x7fe974102340 2024-08-30T09:00:29.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:29.134+0000 7fe97b569640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe97410de40 con 0x7fe9741033c0 2024-08-30T09:00:29.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:29.134+0000 7fe97b569640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fe97410e140 con 0x7fe974103d60 2024-08-30T09:00:29.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:29.134+0000 7fe979305640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe974102340 0x7fe974199c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:29.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:29.135+0000 7fe979305640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe974102340 0x7fe974199c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:51440/0 (socket says 172.21.15.42:51440) 2024-08-30T09:00:29.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:29.135+0000 7fe979305640 1 -- 172.21.15.42:0/209686579 learned_addr learned my addr 172.21.15.42:0/209686579 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:00:29.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:29.135+0000 7fe979305640 1 -- 172.21.15.42:0/209686579 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe974103d60 msgr2=0x7fe97419aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:00:29.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:29.135+0000 7fe978b04640 1 --2- 172.21.15.42:0/209686579 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe9741033c0 0x7fe97419a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:29.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:29.135+0000 7fe979305640 1 --2- 172.21.15.42:0/209686579 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe974103d60 0x7fe97419aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:29.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:29.135+0000 7fe979305640 1 -- 172.21.15.42:0/209686579 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe9741033c0 msgr2=0x7fe97419a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:00:29.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:29.135+0000 7fe979305640 1 --2- 172.21.15.42:0/209686579 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe9741033c0 0x7fe97419a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:29.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:29.135+0000 7fe979305640 1 -- 172.21.15.42:0/209686579 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe97419b6d0 con 0x7fe974102340 2024-08-30T09:00:29.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:29.135+0000 7fe979305640 1 --2- 172.21.15.42:0/209686579 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe974102340 0x7fe974199c90 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fe970006010 tx=0x7fe970004340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:00:29.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:29.135+0000 7fe96a7fc640 1 -- 172.21.15.42:0/209686579 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe97003f030 con 0x7fe974102340 2024-08-30T09:00:29.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:29.135+0000 7fe96a7fc640 1 -- 172.21.15.42:0/209686579 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe970003cb0 con 0x7fe974102340 2024-08-30T09:00:29.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:29.135+0000 7fe97b569640 1 -- 172.21.15.42:0/209686579 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe97419b960 con 0x7fe974102340 2024-08-30T09:00:29.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:29.135+0000 7fe96a7fc640 1 -- 172.21.15.42:0/209686579 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe970035070 con 0x7fe974102340 2024-08-30T09:00:29.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:29.135+0000 7fe97b569640 1 -- 172.21.15.42:0/209686579 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe97419bc70 con 0x7fe974102340 2024-08-30T09:00:29.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:29.136+0000 7fe97b569640 1 -- 172.21.15.42:0/209686579 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9741a20b0 con 0x7fe974102340 2024-08-30T09:00:29.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:29.138+0000 7fe96a7fc640 1 -- 172.21.15.42:0/209686579 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7fe970042030 con 0x7fe974102340 2024-08-30T09:00:29.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:29.138+0000 7fe96a7fc640 1 --2- 172.21.15.42:0/209686579 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fe9600b4b30 0x7fe9600b7040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:29.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:29.138+0000 7fe978b04640 1 --2- 172.21.15.42:0/209686579 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fe9600b4b30 0x7fe9600b7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:29.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:29.139+0000 7fe96a7fc640 1 -- 172.21.15.42:0/209686579 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7fe970105c40 con 0x7fe974102340 2024-08-30T09:00:29.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:29.139+0000 7fe978b04640 1 --2- 172.21.15.42:0/209686579 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fe9600b4b30 0x7fe9600b7040 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fe964005fd0 tx=0x7fe964005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:00:29.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:29.143+0000 7fe96a7fc640 1 -- 172.21.15.42:0/209686579 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7fe970003e60 con 0x7fe974102340 2024-08-30T09:00:29.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:29.326+0000 7fe97b569640 1 -- 172.21.15.42:0/209686579 --> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe9740ffc20 con 0x7fe9600b4b30 2024-08-30T09:00:29.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:29.331+0000 7fe96a7fc640 1 -- 172.21.15.42:0/209686579 <== mgr.14150 v2:172.21.15.42:6800/3005140902 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340452 (secure 0 0 0) 0x7fe9740ffc20 con 0x7fe9600b4b30 2024-08-30T09:00:29.336 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-08-30T09:00:29.338 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-08-30T09:00:29.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:29.334+0000 7fe97b569640 1 -- 172.21.15.42:0/209686579 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fe9600b4b30 msgr2=0x7fe9600b7040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:00:29.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:29.334+0000 7fe97b569640 1 --2- 172.21.15.42:0/209686579 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fe9600b4b30 0x7fe9600b7040 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fe964005fd0 tx=0x7fe964005850 comp rx=0 tx=0).stop 2024-08-30T09:00:29.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:29.335+0000 7fe97b569640 1 -- 172.21.15.42:0/209686579 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe974102340 msgr2=0x7fe974199c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:00:29.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:29.335+0000 7fe97b569640 1 --2- 172.21.15.42:0/209686579 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe974102340 0x7fe974199c90 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fe970006010 tx=0x7fe970004340 comp rx=0 tx=0).stop 2024-08-30T09:00:29.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:29.335+0000 7fe97b569640 1 -- 172.21.15.42:0/209686579 shutdown_connections 2024-08-30T09:00:29.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:29.335+0000 7fe97b569640 1 --2- 172.21.15.42:0/209686579 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fe9600b4b30 0x7fe9600b7040 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:29.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:29.335+0000 7fe97b569640 1 --2- 172.21.15.42:0/209686579 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe974103d60 0x7fe97419aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:29.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:29.335+0000 7fe97b569640 1 --2- 172.21.15.42:0/209686579 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe9741033c0 0x7fe97419a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:29.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:29.335+0000 7fe97b569640 1 --2- 172.21.15.42:0/209686579 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe974102340 0x7fe974199c90 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:29.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:29.335+0000 7fe97b569640 1 -- 172.21.15.42:0/209686579 >> 172.21.15.42:0/209686579 conn(0x7fe9740fdc80 msgr2=0x7fe9740ff510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:00:29.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:29.335+0000 7fe97b569640 1 -- 172.21.15.42:0/209686579 shutdown_connections 2024-08-30T09:00:29.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:29.335+0000 7fe97b569640 1 -- 172.21.15.42:0/209686579 wait complete. 2024-08-30T09:00:29.348 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":365,"stamp":"2024-08-30T09:00:28.967886+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":659893,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":688,"num_read_kb":444,"num_write":308,"num_write_kb":2281,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":2951840,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":750,"ondisk_log_size":750,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":379120,"kb_used_data":13232,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124562704,"statfs":{"total":1151940427776,"available":1151552208896,"internally_reserved":0,"allocated":13549568,"data_stored":7645385,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":374453561},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":698,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":88,"num_read_kb":48,"num_write":11,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005884"},"pg_stats":[{"pgid":"2.1f","version":"80'9","reported_seq":50,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T08:59:46.656396+0000","last_change":"2024-08-30T08:58:00.388595+0000","last_active":"2024-08-30T08:59:46.656396+0000","last_peered":"2024-08-30T08:59:46.656396+0000","last_clean":"2024-08-30T08:59:46.656396+0000","last_became_active":"2024-08-30T08:58:00.388215+0000","last_became_peered":"2024-08-30T08:58:00.388215+0000","last_unstale":"2024-08-30T08:59:46.656396+0000","last_undegraded":"2024-08-30T08:59:46.656396+0000","last_fullsized":"2024-08-30T08:59:46.656396+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T13:00:06.109935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"83'5","reported_seq":40,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:15.312524+0000","last_change":"2024-08-30T08:58:02.416042+0000","last_active":"2024-08-30T09:00:15.312524+0000","last_peered":"2024-08-30T09:00:15.312524+0000","last_clean":"2024-08-30T09:00:15.312524+0000","last_became_active":"2024-08-30T08:58:02.415507+0000","last_became_peered":"2024-08-30T08:58:02.415507+0000","last_unstale":"2024-08-30T09:00:15.312524+0000","last_undegraded":"2024-08-30T09:00:15.312524+0000","last_fullsized":"2024-08-30T09:00:15.312524+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T14:33:57.192004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T08:59:46.657037+0000","last_change":"2024-08-30T08:58:09.448785+0000","last_active":"2024-08-30T08:59:46.657037+0000","last_peered":"2024-08-30T08:59:46.657037+0000","last_clean":"2024-08-30T08:59:46.657037+0000","last_became_active":"2024-08-30T08:58:09.448191+0000","last_became_peered":"2024-08-30T08:58:09.448191+0000","last_unstale":"2024-08-30T08:59:46.657037+0000","last_undegraded":"2024-08-30T08:59:46.657037+0000","last_fullsized":"2024-08-30T08:59:46.657037+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T13:37:19.148201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"80'2","reported_seq":42,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T08:59:46.702141+0000","last_change":"2024-08-30T08:58:00.480406+0000","last_active":"2024-08-30T08:59:46.702141+0000","last_peered":"2024-08-30T08:59:46.702141+0000","last_clean":"2024-08-30T08:59:46.702141+0000","last_became_active":"2024-08-30T08:58:00.479896+0000","last_became_peered":"2024-08-30T08:58:00.479896+0000","last_unstale":"2024-08-30T08:59:46.702141+0000","last_undegraded":"2024-08-30T08:59:46.702141+0000","last_fullsized":"2024-08-30T08:59:46.702141+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T15:16:15.316343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"83'6","reported_seq":41,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:15.335025+0000","last_change":"2024-08-30T08:58:02.406915+0000","last_active":"2024-08-30T09:00:15.335025+0000","last_peered":"2024-08-30T09:00:15.335025+0000","last_clean":"2024-08-30T09:00:15.335025+0000","last_became_active":"2024-08-30T08:58:02.406557+0000","last_became_peered":"2024-08-30T08:58:02.406557+0000","last_unstale":"2024-08-30T09:00:15.335025+0000","last_undegraded":"2024-08-30T09:00:15.335025+0000","last_fullsized":"2024-08-30T09:00:15.335025+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T19:15:30.847212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T08:59:46.661576+0000","last_change":"2024-08-30T08:58:09.446974+0000","last_active":"2024-08-30T08:59:46.661576+0000","last_peered":"2024-08-30T08:59:46.661576+0000","last_clean":"2024-08-30T08:59:46.661576+0000","last_became_active":"2024-08-30T08:58:09.446173+0000","last_became_peered":"2024-08-30T08:58:09.446173+0000","last_unstale":"2024-08-30T08:59:46.661576+0000","last_undegraded":"2024-08-30T08:59:46.661576+0000","last_fullsized":"2024-08-30T08:59:46.661576+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T12:13:41.276825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"80'4","reported_seq":56,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T08:59:46.673167+0000","last_change":"2024-08-30T08:59:46.673048+0000","last_active":"2024-08-30T08:59:46.673167+0000","last_peered":"2024-08-30T08:59:46.673167+0000","last_clean":"2024-08-30T08:59:46.673167+0000","last_became_active":"2024-08-30T08:59:46.665588+0000","last_became_peered":"2024-08-30T08:59:46.665588+0000","last_unstale":"2024-08-30T08:59:46.673167+0000","last_undegraded":"2024-08-30T08:59:46.673167+0000","last_fullsized":"2024-08-30T08:59:46.673167+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T18:02:29.663191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,7],"acting":[8,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"83'9","reported_seq":43,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:15.334804+0000","last_change":"2024-08-30T08:58:02.415594+0000","last_active":"2024-08-30T09:00:15.334804+0000","last_peered":"2024-08-30T09:00:15.334804+0000","last_clean":"2024-08-30T09:00:15.334804+0000","last_became_active":"2024-08-30T08:58:02.415066+0000","last_became_peered":"2024-08-30T08:58:02.415066+0000","last_unstale":"2024-08-30T09:00:15.334804+0000","last_undegraded":"2024-08-30T09:00:15.334804+0000","last_fullsized":"2024-08-30T09:00:15.334804+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T12:07:27.668442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T08:59:46.662698+0000","last_change":"2024-08-30T08:58:09.444403+0000","last_active":"2024-08-30T08:59:46.662698+0000","last_peered":"2024-08-30T08:59:46.662698+0000","last_clean":"2024-08-30T08:59:46.662698+0000","last_became_active":"2024-08-30T08:58:09.443914+0000","last_became_peered":"2024-08-30T08:58:09.443914+0000","last_unstale":"2024-08-30T08:59:46.662698+0000","last_undegraded":"2024-08-30T08:59:46.662698+0000","last_fullsized":"2024-08-30T08:59:46.662698+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T14:17:43.866352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"73'2","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T08:59:46.669336+0000","last_change":"2024-08-30T08:59:46.668686+0000","last_active":"2024-08-30T08:59:46.669336+0000","last_peered":"2024-08-30T08:59:46.669336+0000","last_clean":"2024-08-30T08:59:46.669336+0000","last_became_active":"2024-08-30T08:59:46.665549+0000","last_became_peered":"2024-08-30T08:59:46.665549+0000","last_unstale":"2024-08-30T08:59:46.669336+0000","last_undegraded":"2024-08-30T08:59:46.669336+0000","last_fullsized":"2024-08-30T08:59:46.669336+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T12:25:56.088867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"83'15","reported_seq":49,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:15.315195+0000","last_change":"2024-08-30T08:58:02.405550+0000","last_active":"2024-08-30T09:00:15.315195+0000","last_peered":"2024-08-30T09:00:15.315195+0000","last_clean":"2024-08-30T09:00:15.315195+0000","last_became_active":"2024-08-30T08:58:02.405129+0000","last_became_peered":"2024-08-30T08:58:02.405129+0000","last_unstale":"2024-08-30T09:00:15.315195+0000","last_undegraded":"2024-08-30T09:00:15.315195+0000","last_fullsized":"2024-08-30T09:00:15.315195+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T11:36:47.624026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T08:59:46.661666+0000","last_change":"2024-08-30T08:58:09.445746+0000","last_active":"2024-08-30T08:59:46.661666+0000","last_peered":"2024-08-30T08:59:46.661666+0000","last_clean":"2024-08-30T08:59:46.661666+0000","last_became_active":"2024-08-30T08:58:09.445257+0000","last_became_peered":"2024-08-30T08:58:09.445257+0000","last_unstale":"2024-08-30T08:59:46.661666+0000","last_undegraded":"2024-08-30T08:59:46.661666+0000","last_fullsized":"2024-08-30T08:59:46.661666+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T18:02:52.007131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"83'5","reported_seq":40,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:15.335272+0000","last_change":"2024-08-30T08:58:02.408213+0000","last_active":"2024-08-30T09:00:15.335272+0000","last_peered":"2024-08-30T09:00:15.335272+0000","last_clean":"2024-08-30T09:00:15.335272+0000","last_became_active":"2024-08-30T08:58:02.407856+0000","last_became_peered":"2024-08-30T08:58:02.407856+0000","last_unstale":"2024-08-30T09:00:15.335272+0000","last_undegraded":"2024-08-30T09:00:15.335272+0000","last_fullsized":"2024-08-30T09:00:15.335272+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T12:00:22.226653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"79'4","reported_seq":45,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T08:59:46.659742+0000","last_change":"2024-08-30T08:58:00.391714+0000","last_active":"2024-08-30T08:59:46.659742+0000","last_peered":"2024-08-30T08:59:46.659742+0000","last_clean":"2024-08-30T08:59:46.659742+0000","last_became_active":"2024-08-30T08:58:00.391211+0000","last_became_peered":"2024-08-30T08:58:00.391211+0000","last_unstale":"2024-08-30T08:59:46.659742+0000","last_undegraded":"2024-08-30T08:59:46.659742+0000","last_fullsized":"2024-08-30T08:59:46.659742+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T11:55:37.469983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T08:59:46.662058+0000","last_change":"2024-08-30T08:58:09.728177+0000","last_active":"2024-08-30T08:59:46.662058+0000","last_peered":"2024-08-30T08:59:46.662058+0000","last_clean":"2024-08-30T08:59:46.662058+0000","last_became_active":"2024-08-30T08:58:09.727630+0000","last_became_peered":"2024-08-30T08:58:09.727630+0000","last_unstale":"2024-08-30T08:59:46.662058+0000","last_undegraded":"2024-08-30T08:59:46.662058+0000","last_fullsized":"2024-08-30T08:59:46.662058+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T12:07:52.965014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"83'8","reported_seq":43,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:15.319441+0000","last_change":"2024-08-30T08:58:02.403378+0000","last_active":"2024-08-30T09:00:15.319441+0000","last_peered":"2024-08-30T09:00:15.319441+0000","last_clean":"2024-08-30T09:00:15.319441+0000","last_became_active":"2024-08-30T08:58:02.403036+0000","last_became_peered":"2024-08-30T08:58:02.403036+0000","last_unstale":"2024-08-30T09:00:15.319441+0000","last_undegraded":"2024-08-30T09:00:15.319441+0000","last_fullsized":"2024-08-30T09:00:15.319441+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T16:37:14.876384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"80'1","reported_seq":41,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T08:59:46.657210+0000","last_change":"2024-08-30T08:58:00.393860+0000","last_active":"2024-08-30T08:59:46.657210+0000","last_peered":"2024-08-30T08:59:46.657210+0000","last_clean":"2024-08-30T08:59:46.657210+0000","last_became_active":"2024-08-30T08:58:00.392486+0000","last_became_peered":"2024-08-30T08:58:00.392486+0000","last_unstale":"2024-08-30T08:59:46.657210+0000","last_undegraded":"2024-08-30T08:59:46.657210+0000","last_fullsized":"2024-08-30T08:59:46.657210+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T12:50:27.337817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"81'1","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T08:59:46.662322+0000","last_change":"2024-08-30T08:58:09.441476+0000","last_active":"2024-08-30T08:59:46.662322+0000","last_peered":"2024-08-30T08:59:46.662322+0000","last_clean":"2024-08-30T08:59:46.662322+0000","last_became_active":"2024-08-30T08:58:09.440344+0000","last_became_peered":"2024-08-30T08:58:09.440344+0000","last_unstale":"2024-08-30T08:59:46.662322+0000","last_undegraded":"2024-08-30T08:59:46.662322+0000","last_fullsized":"2024-08-30T08:59:46.662322+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T12:06:47.113611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"83'9","reported_seq":44,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:15.314692+0000","last_change":"2024-08-30T08:58:02.407098+0000","last_active":"2024-08-30T09:00:15.314692+0000","last_peered":"2024-08-30T09:00:15.314692+0000","last_clean":"2024-08-30T09:00:15.314692+0000","last_became_active":"2024-08-30T08:58:02.406690+0000","last_became_peered":"2024-08-30T08:58:02.406690+0000","last_unstale":"2024-08-30T09:00:15.314692+0000","last_undegraded":"2024-08-30T09:00:15.314692+0000","last_fullsized":"2024-08-30T09:00:15.314692+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T18:27:39.534923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"79'3","reported_seq":43,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T08:59:46.701567+0000","last_change":"2024-08-30T08:58:00.386434+0000","last_active":"2024-08-30T08:59:46.701567+0000","last_peered":"2024-08-30T08:59:46.701567+0000","last_clean":"2024-08-30T08:59:46.701567+0000","last_became_active":"2024-08-30T08:58:00.385930+0000","last_became_peered":"2024-08-30T08:58:00.385930+0000","last_unstale":"2024-08-30T08:59:46.701567+0000","last_undegraded":"2024-08-30T08:59:46.701567+0000","last_fullsized":"2024-08-30T08:59:46.701567+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T10:39:22.441568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T08:59:46.659680+0000","last_change":"2024-08-30T08:58:46.775674+0000","last_active":"2024-08-30T08:59:46.659680+0000","last_peered":"2024-08-30T08:59:46.659680+0000","last_clean":"2024-08-30T08:59:46.659680+0000","last_became_active":"2024-08-30T08:58:46.775157+0000","last_became_peered":"2024-08-30T08:58:46.775157+0000","last_unstale":"2024-08-30T08:59:46.659680+0000","last_undegraded":"2024-08-30T08:59:46.659680+0000","last_fullsized":"2024-08-30T08:59:46.659680+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T17:13:47.066475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,2,5],"acting":[8,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.9","version":"83'2","reported_seq":37,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:15.283463+0000","last_change":"2024-08-30T08:58:02.410500+0000","last_active":"2024-08-30T09:00:15.283463+0000","last_peered":"2024-08-30T09:00:15.283463+0000","last_clean":"2024-08-30T09:00:15.283463+0000","last_became_active":"2024-08-30T08:58:02.409260+0000","last_became_peered":"2024-08-30T08:58:02.409260+0000","last_unstale":"2024-08-30T09:00:15.283463+0000","last_undegraded":"2024-08-30T09:00:15.283463+0000","last_fullsized":"2024-08-30T09:00:15.283463+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T16:47:59.701956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"79'2","reported_seq":42,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T08:59:46.663767+0000","last_change":"2024-08-30T08:58:00.482296+0000","last_active":"2024-08-30T08:59:46.663767+0000","last_peered":"2024-08-30T08:59:46.663767+0000","last_clean":"2024-08-30T08:59:46.663767+0000","last_became_active":"2024-08-30T08:58:00.481920+0000","last_became_peered":"2024-08-30T08:58:00.481920+0000","last_unstale":"2024-08-30T08:59:46.663767+0000","last_undegraded":"2024-08-30T08:59:46.663767+0000","last_fullsized":"2024-08-30T08:59:46.663767+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T09:29:30.724410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T08:59:46.662353+0000","last_change":"2024-08-30T08:58:09.727879+0000","last_active":"2024-08-30T08:59:46.662353+0000","last_peered":"2024-08-30T08:59:46.662353+0000","last_clean":"2024-08-30T08:59:46.662353+0000","last_became_active":"2024-08-30T08:58:09.727339+0000","last_became_peered":"2024-08-30T08:58:09.727339+0000","last_unstale":"2024-08-30T08:59:46.662353+0000","last_undegraded":"2024-08-30T08:59:46.662353+0000","last_fullsized":"2024-08-30T08:59:46.662353+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T13:19:18.582668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"83'7","reported_seq":49,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:15.332760+0000","last_change":"2024-08-30T08:58:46.559988+0000","last_active":"2024-08-30T09:00:15.332760+0000","last_peered":"2024-08-30T09:00:15.332760+0000","last_clean":"2024-08-30T09:00:15.332760+0000","last_became_active":"2024-08-30T08:58:46.559442+0000","last_became_peered":"2024-08-30T08:58:46.559442+0000","last_unstale":"2024-08-30T09:00:15.332760+0000","last_undegraded":"2024-08-30T09:00:15.332760+0000","last_fullsized":"2024-08-30T09:00:15.332760+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T09:29:09.747756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"79'4","reported_seq":44,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T08:59:46.662490+0000","last_change":"2024-08-30T08:58:00.380338+0000","last_active":"2024-08-30T08:59:46.662490+0000","last_peered":"2024-08-30T08:59:46.662490+0000","last_clean":"2024-08-30T08:59:46.662490+0000","last_became_active":"2024-08-30T08:58:00.379860+0000","last_became_peered":"2024-08-30T08:58:00.379860+0000","last_unstale":"2024-08-30T08:59:46.662490+0000","last_undegraded":"2024-08-30T08:59:46.662490+0000","last_fullsized":"2024-08-30T08:59:46.662490+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T19:57:15.349906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T08:59:46.657178+0000","last_change":"2024-08-30T08:58:09.443631+0000","last_active":"2024-08-30T08:59:46.657178+0000","last_peered":"2024-08-30T08:59:46.657178+0000","last_clean":"2024-08-30T08:59:46.657178+0000","last_became_active":"2024-08-30T08:58:09.443103+0000","last_became_peered":"2024-08-30T08:58:09.443103+0000","last_unstale":"2024-08-30T08:59:46.657178+0000","last_undegraded":"2024-08-30T08:59:46.657178+0000","last_fullsized":"2024-08-30T08:59:46.657178+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T19:58:11.031826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"83'5","reported_seq":40,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:15.328087+0000","last_change":"2024-08-30T08:58:02.406216+0000","last_active":"2024-08-30T09:00:15.328087+0000","last_peered":"2024-08-30T09:00:15.328087+0000","last_clean":"2024-08-30T09:00:15.328087+0000","last_became_active":"2024-08-30T08:58:02.405040+0000","last_became_peered":"2024-08-30T08:58:02.405040+0000","last_unstale":"2024-08-30T09:00:15.328087+0000","last_undegraded":"2024-08-30T09:00:15.328087+0000","last_fullsized":"2024-08-30T09:00:15.328087+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T14:28:28.721386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"83'14","reported_seq":54,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:15.423472+0000","last_change":"2024-08-30T08:58:00.387350+0000","last_active":"2024-08-30T09:00:15.423472+0000","last_peered":"2024-08-30T09:00:15.423472+0000","last_clean":"2024-08-30T09:00:15.423472+0000","last_became_active":"2024-08-30T08:58:00.387004+0000","last_became_peered":"2024-08-30T08:58:00.387004+0000","last_unstale":"2024-08-30T09:00:15.423472+0000","last_undegraded":"2024-08-30T09:00:15.423472+0000","last_fullsized":"2024-08-30T09:00:15.423472+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T19:24:25.604544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T08:59:46.700944+0000","last_change":"2024-08-30T08:58:09.727388+0000","last_active":"2024-08-30T08:59:46.700944+0000","last_peered":"2024-08-30T08:59:46.700944+0000","last_clean":"2024-08-30T08:59:46.700944+0000","last_became_active":"2024-08-30T08:58:09.726935+0000","last_became_peered":"2024-08-30T08:58:09.726935+0000","last_unstale":"2024-08-30T08:59:46.700944+0000","last_undegraded":"2024-08-30T08:59:46.700944+0000","last_fullsized":"2024-08-30T08:59:46.700944+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T17:53:16.066602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"83'9","reported_seq":44,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:15.307241+0000","last_change":"2024-08-30T08:58:02.402661+0000","last_active":"2024-08-30T09:00:15.307241+0000","last_peered":"2024-08-30T09:00:15.307241+0000","last_clean":"2024-08-30T09:00:15.307241+0000","last_became_active":"2024-08-30T08:58:02.402314+0000","last_became_peered":"2024-08-30T08:58:02.402314+0000","last_unstale":"2024-08-30T09:00:15.307241+0000","last_undegraded":"2024-08-30T09:00:15.307241+0000","last_fullsized":"2024-08-30T09:00:15.307241+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T10:56:41.336312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"79'1","reported_seq":42,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T08:59:46.660369+0000","last_change":"2024-08-30T08:58:00.482720+0000","last_active":"2024-08-30T08:59:46.660369+0000","last_peered":"2024-08-30T08:59:46.660369+0000","last_clean":"2024-08-30T08:59:46.660369+0000","last_became_active":"2024-08-30T08:58:00.482193+0000","last_became_peered":"2024-08-30T08:58:00.482193+0000","last_unstale":"2024-08-30T08:59:46.660369+0000","last_undegraded":"2024-08-30T08:59:46.660369+0000","last_fullsized":"2024-08-30T08:59:46.660369+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T18:42:06.654806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T08:59:46.660980+0000","last_change":"2024-08-30T08:58:09.449970+0000","last_active":"2024-08-30T08:59:46.660980+0000","last_peered":"2024-08-30T08:59:46.660980+0000","last_clean":"2024-08-30T08:59:46.660980+0000","last_became_active":"2024-08-30T08:58:09.447891+0000","last_became_peered":"2024-08-30T08:58:09.447891+0000","last_unstale":"2024-08-30T08:59:46.660980+0000","last_undegraded":"2024-08-30T08:59:46.660980+0000","last_fullsized":"2024-08-30T08:59:46.660980+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T15:06:45.414470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"83'4","reported_seq":39,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:15.322836+0000","last_change":"2024-08-30T08:58:02.409756+0000","last_active":"2024-08-30T09:00:15.322836+0000","last_peered":"2024-08-30T09:00:15.322836+0000","last_clean":"2024-08-30T09:00:15.322836+0000","last_became_active":"2024-08-30T08:58:02.409357+0000","last_became_peered":"2024-08-30T08:58:02.409357+0000","last_unstale":"2024-08-30T09:00:15.322836+0000","last_undegraded":"2024-08-30T09:00:15.322836+0000","last_fullsized":"2024-08-30T09:00:15.322836+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T13:27:41.696527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":40,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T08:59:46.656323+0000","last_change":"2024-08-30T08:58:00.482342+0000","last_active":"2024-08-30T08:59:46.656323+0000","last_peered":"2024-08-30T08:59:46.656323+0000","last_clean":"2024-08-30T08:59:46.656323+0000","last_became_active":"2024-08-30T08:58:00.481902+0000","last_became_peered":"2024-08-30T08:58:00.481902+0000","last_unstale":"2024-08-30T08:59:46.656323+0000","last_undegraded":"2024-08-30T08:59:46.656323+0000","last_fullsized":"2024-08-30T08:59:46.656323+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T14:12:36.789356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T08:59:46.659485+0000","last_change":"2024-08-30T08:58:09.728008+0000","last_active":"2024-08-30T08:59:46.659485+0000","last_peered":"2024-08-30T08:59:46.659485+0000","last_clean":"2024-08-30T08:59:46.659485+0000","last_became_active":"2024-08-30T08:58:09.727513+0000","last_became_peered":"2024-08-30T08:58:09.727513+0000","last_unstale":"2024-08-30T08:59:46.659485+0000","last_undegraded":"2024-08-30T08:59:46.659485+0000","last_fullsized":"2024-08-30T08:59:46.659485+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T14:34:04.165300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"83'7","reported_seq":42,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:15.310825+0000","last_change":"2024-08-30T08:58:02.409383+0000","last_active":"2024-08-30T09:00:15.310825+0000","last_peered":"2024-08-30T09:00:15.310825+0000","last_clean":"2024-08-30T09:00:15.310825+0000","last_became_active":"2024-08-30T08:58:02.409027+0000","last_became_peered":"2024-08-30T08:58:02.409027+0000","last_unstale":"2024-08-30T09:00:15.310825+0000","last_undegraded":"2024-08-30T09:00:15.310825+0000","last_fullsized":"2024-08-30T09:00:15.310825+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T10:44:23.467130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"80'1","reported_seq":41,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T08:59:46.663584+0000","last_change":"2024-08-30T08:58:00.389083+0000","last_active":"2024-08-30T08:59:46.663584+0000","last_peered":"2024-08-30T08:59:46.663584+0000","last_clean":"2024-08-30T08:59:46.663584+0000","last_became_active":"2024-08-30T08:58:00.387552+0000","last_became_peered":"2024-08-30T08:58:00.387552+0000","last_unstale":"2024-08-30T08:59:46.663584+0000","last_undegraded":"2024-08-30T08:59:46.663584+0000","last_fullsized":"2024-08-30T08:59:46.663584+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T11:56:38.827406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T08:59:46.701544+0000","last_change":"2024-08-30T08:58:09.447758+0000","last_active":"2024-08-30T08:59:46.701544+0000","last_peered":"2024-08-30T08:59:46.701544+0000","last_clean":"2024-08-30T08:59:46.701544+0000","last_became_active":"2024-08-30T08:58:09.447333+0000","last_became_peered":"2024-08-30T08:58:09.447333+0000","last_unstale":"2024-08-30T08:59:46.701544+0000","last_undegraded":"2024-08-30T08:59:46.701544+0000","last_fullsized":"2024-08-30T08:59:46.701544+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T12:43:35.289967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"83'8","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:15.309024+0000","last_change":"2024-08-30T08:58:47.091191+0000","last_active":"2024-08-30T09:00:15.309024+0000","last_peered":"2024-08-30T09:00:15.309024+0000","last_clean":"2024-08-30T09:00:15.309024+0000","last_became_active":"2024-08-30T08:58:47.090826+0000","last_became_peered":"2024-08-30T08:58:47.090826+0000","last_unstale":"2024-08-30T09:00:15.309024+0000","last_undegraded":"2024-08-30T09:00:15.309024+0000","last_fullsized":"2024-08-30T09:00:15.309024+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T17:47:22.862514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"83'5","reported_seq":142,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:25.765746+0000","last_change":"2024-08-30T08:58:00.480651+0000","last_active":"2024-08-30T09:00:25.765746+0000","last_peered":"2024-08-30T09:00:25.765746+0000","last_clean":"2024-08-30T09:00:25.765746+0000","last_became_active":"2024-08-30T08:58:00.480205+0000","last_became_peered":"2024-08-30T08:58:00.480205+0000","last_unstale":"2024-08-30T09:00:25.765746+0000","last_undegraded":"2024-08-30T09:00:25.765746+0000","last_fullsized":"2024-08-30T09:00:25.765746+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T16:28:30.687076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":97,"num_read_kb":97,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T08:59:46.701596+0000","last_change":"2024-08-30T08:58:09.727530+0000","last_active":"2024-08-30T08:59:46.701596+0000","last_peered":"2024-08-30T08:59:46.701596+0000","last_clean":"2024-08-30T08:59:46.701596+0000","last_became_active":"2024-08-30T08:58:09.726965+0000","last_became_peered":"2024-08-30T08:58:09.726965+0000","last_unstale":"2024-08-30T08:59:46.701596+0000","last_undegraded":"2024-08-30T08:59:46.701596+0000","last_fullsized":"2024-08-30T08:59:46.701596+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T12:56:42.885615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"83'5","reported_seq":40,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:15.312193+0000","last_change":"2024-08-30T08:58:02.401673+0000","last_active":"2024-08-30T09:00:15.312193+0000","last_peered":"2024-08-30T09:00:15.312193+0000","last_clean":"2024-08-30T09:00:15.312193+0000","last_became_active":"2024-08-30T08:58:02.398876+0000","last_became_peered":"2024-08-30T08:58:02.398876+0000","last_unstale":"2024-08-30T09:00:15.312193+0000","last_undegraded":"2024-08-30T09:00:15.312193+0000","last_fullsized":"2024-08-30T09:00:15.312193+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T16:13:20.893737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"80'3","reported_seq":43,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T08:59:46.663087+0000","last_change":"2024-08-30T08:58:00.391722+0000","last_active":"2024-08-30T08:59:46.663087+0000","last_peered":"2024-08-30T08:59:46.663087+0000","last_clean":"2024-08-30T08:59:46.663087+0000","last_became_active":"2024-08-30T08:58:00.391325+0000","last_became_peered":"2024-08-30T08:58:00.391325+0000","last_unstale":"2024-08-30T08:59:46.663087+0000","last_undegraded":"2024-08-30T08:59:46.663087+0000","last_fullsized":"2024-08-30T08:59:46.663087+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T11:58:08.189202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T08:59:46.663818+0000","last_change":"2024-08-30T08:58:09.728324+0000","last_active":"2024-08-30T08:59:46.663818+0000","last_peered":"2024-08-30T08:59:46.663818+0000","last_clean":"2024-08-30T08:59:46.663818+0000","last_became_active":"2024-08-30T08:58:09.727520+0000","last_became_peered":"2024-08-30T08:58:09.727520+0000","last_unstale":"2024-08-30T08:59:46.663818+0000","last_undegraded":"2024-08-30T08:59:46.663818+0000","last_fullsized":"2024-08-30T08:59:46.663818+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T10:09:39.046820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"83'8","reported_seq":43,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:15.325800+0000","last_change":"2024-08-30T08:58:02.410826+0000","last_active":"2024-08-30T09:00:15.325800+0000","last_peered":"2024-08-30T09:00:15.325800+0000","last_clean":"2024-08-30T09:00:15.325800+0000","last_became_active":"2024-08-30T08:58:02.410464+0000","last_became_peered":"2024-08-30T08:58:02.410464+0000","last_unstale":"2024-08-30T09:00:15.325800+0000","last_undegraded":"2024-08-30T09:00:15.325800+0000","last_fullsized":"2024-08-30T09:00:15.325800+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T12:14:59.666614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"80'3","reported_seq":45,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T08:59:46.663066+0000","last_change":"2024-08-30T08:58:00.385464+0000","last_active":"2024-08-30T08:59:46.663066+0000","last_peered":"2024-08-30T08:59:46.663066+0000","last_clean":"2024-08-30T08:59:46.663066+0000","last_became_active":"2024-08-30T08:58:00.385083+0000","last_became_peered":"2024-08-30T08:58:00.385083+0000","last_unstale":"2024-08-30T08:59:46.663066+0000","last_undegraded":"2024-08-30T08:59:46.663066+0000","last_fullsized":"2024-08-30T08:59:46.663066+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T16:13:52.304755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":112,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T08:59:46.663146+0000","last_change":"2024-08-30T08:56:49.645080+0000","last_active":"2024-08-30T08:59:46.663146+0000","last_peered":"2024-08-30T08:59:46.663146+0000","last_clean":"2024-08-30T08:59:46.663146+0000","last_became_active":"2024-08-30T08:56:49.639066+0000","last_became_peered":"2024-08-30T08:56:49.639066+0000","last_unstale":"2024-08-30T08:59:46.663146+0000","last_undegraded":"2024-08-30T08:59:46.663146+0000","last_fullsized":"2024-08-30T08:59:46.663146+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:53:27.566846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:53:27.566846+0000","last_clean_scrub_stamp":"2024-08-30T08:53:27.566846+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T20:23:45.723824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"81'1","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T08:59:46.663607+0000","last_change":"2024-08-30T08:58:09.444007+0000","last_active":"2024-08-30T08:59:46.663607+0000","last_peered":"2024-08-30T08:59:46.663607+0000","last_clean":"2024-08-30T08:59:46.663607+0000","last_became_active":"2024-08-30T08:58:09.443580+0000","last_became_peered":"2024-08-30T08:58:09.443580+0000","last_unstale":"2024-08-30T08:59:46.663607+0000","last_undegraded":"2024-08-30T08:59:46.663607+0000","last_fullsized":"2024-08-30T08:59:46.663607+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T17:00:58.654026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"83'7","reported_seq":42,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:15.335454+0000","last_change":"2024-08-30T08:58:02.406973+0000","last_active":"2024-08-30T09:00:15.335454+0000","last_peered":"2024-08-30T09:00:15.335454+0000","last_clean":"2024-08-30T09:00:15.335454+0000","last_became_active":"2024-08-30T08:58:02.406576+0000","last_became_peered":"2024-08-30T08:58:02.406576+0000","last_unstale":"2024-08-30T09:00:15.335454+0000","last_undegraded":"2024-08-30T09:00:15.335454+0000","last_fullsized":"2024-08-30T09:00:15.335454+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T16:04:48.785310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"80'3","reported_seq":44,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T08:59:46.657731+0000","last_change":"2024-08-30T08:58:00.481019+0000","last_active":"2024-08-30T08:59:46.657731+0000","last_peered":"2024-08-30T08:59:46.657731+0000","last_clean":"2024-08-30T08:59:46.657731+0000","last_became_active":"2024-08-30T08:58:00.480590+0000","last_became_peered":"2024-08-30T08:58:00.480590+0000","last_unstale":"2024-08-30T08:59:46.657731+0000","last_undegraded":"2024-08-30T08:59:46.657731+0000","last_fullsized":"2024-08-30T08:59:46.657731+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T13:20:46.731591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T08:59:46.664023+0000","last_change":"2024-08-30T08:58:09.445723+0000","last_active":"2024-08-30T08:59:46.664023+0000","last_peered":"2024-08-30T08:59:46.664023+0000","last_clean":"2024-08-30T08:59:46.664023+0000","last_became_active":"2024-08-30T08:58:09.445252+0000","last_became_peered":"2024-08-30T08:58:09.445252+0000","last_unstale":"2024-08-30T08:59:46.664023+0000","last_undegraded":"2024-08-30T08:59:46.664023+0000","last_fullsized":"2024-08-30T08:59:46.664023+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T12:16:12.255990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"83'6","reported_seq":41,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:15.331469+0000","last_change":"2024-08-30T08:58:02.414250+0000","last_active":"2024-08-30T09:00:15.331469+0000","last_peered":"2024-08-30T09:00:15.331469+0000","last_clean":"2024-08-30T09:00:15.331469+0000","last_became_active":"2024-08-30T08:58:02.413684+0000","last_became_peered":"2024-08-30T08:58:02.413684+0000","last_unstale":"2024-08-30T09:00:15.331469+0000","last_undegraded":"2024-08-30T09:00:15.331469+0000","last_fullsized":"2024-08-30T09:00:15.331469+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T16:55:12.878806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"73'1","reported_seq":42,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T08:59:46.664107+0000","last_change":"2024-08-30T08:58:00.388869+0000","last_active":"2024-08-30T08:59:46.664107+0000","last_peered":"2024-08-30T08:59:46.664107+0000","last_clean":"2024-08-30T08:59:46.664107+0000","last_became_active":"2024-08-30T08:58:00.388116+0000","last_became_peered":"2024-08-30T08:58:00.388116+0000","last_unstale":"2024-08-30T08:59:46.664107+0000","last_undegraded":"2024-08-30T08:59:46.664107+0000","last_fullsized":"2024-08-30T08:59:46.664107+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T09:50:48.449205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T08:59:46.842285+0000","last_change":"2024-08-30T08:58:47.092004+0000","last_active":"2024-08-30T08:59:46.842285+0000","last_peered":"2024-08-30T08:59:46.842285+0000","last_clean":"2024-08-30T08:59:46.842285+0000","last_became_active":"2024-08-30T08:58:47.091661+0000","last_became_peered":"2024-08-30T08:58:47.091661+0000","last_unstale":"2024-08-30T08:59:46.842285+0000","last_undegraded":"2024-08-30T08:59:46.842285+0000","last_fullsized":"2024-08-30T08:59:46.842285+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T09:14:55.353465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,1,4],"acting":[0,8,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"83'9","reported_seq":44,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:15.331298+0000","last_change":"2024-08-30T08:58:02.410891+0000","last_active":"2024-08-30T09:00:15.331298+0000","last_peered":"2024-08-30T09:00:15.331298+0000","last_clean":"2024-08-30T09:00:15.331298+0000","last_became_active":"2024-08-30T08:58:02.410522+0000","last_became_peered":"2024-08-30T08:58:02.410522+0000","last_unstale":"2024-08-30T09:00:15.331298+0000","last_undegraded":"2024-08-30T09:00:15.331298+0000","last_fullsized":"2024-08-30T09:00:15.331298+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T09:32:21.540298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"73'2","reported_seq":53,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T08:59:46.702026+0000","last_change":"2024-08-30T08:58:47.095801+0000","last_active":"2024-08-30T08:59:46.702026+0000","last_peered":"2024-08-30T08:59:46.702026+0000","last_clean":"2024-08-30T08:59:46.702026+0000","last_became_active":"2024-08-30T08:58:47.092756+0000","last_became_peered":"2024-08-30T08:58:47.092756+0000","last_unstale":"2024-08-30T08:59:46.702026+0000","last_undegraded":"2024-08-30T08:59:46.702026+0000","last_fullsized":"2024-08-30T08:59:46.702026+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T19:10:02.064649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T08:59:46.663000+0000","last_change":"2024-08-30T08:58:09.449210+0000","last_active":"2024-08-30T08:59:46.663000+0000","last_peered":"2024-08-30T08:59:46.663000+0000","last_clean":"2024-08-30T08:59:46.663000+0000","last_became_active":"2024-08-30T08:58:09.448769+0000","last_became_peered":"2024-08-30T08:58:09.448769+0000","last_unstale":"2024-08-30T08:59:46.663000+0000","last_undegraded":"2024-08-30T08:59:46.663000+0000","last_fullsized":"2024-08-30T08:59:46.663000+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T09:26:13.845520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"83'5","reported_seq":40,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:15.316813+0000","last_change":"2024-08-30T08:58:02.414152+0000","last_active":"2024-08-30T09:00:15.316813+0000","last_peered":"2024-08-30T09:00:15.316813+0000","last_clean":"2024-08-30T09:00:15.316813+0000","last_became_active":"2024-08-30T08:58:02.413768+0000","last_became_peered":"2024-08-30T08:58:02.413768+0000","last_unstale":"2024-08-30T09:00:15.316813+0000","last_undegraded":"2024-08-30T09:00:15.316813+0000","last_fullsized":"2024-08-30T09:00:15.316813+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T18:34:09.447071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"81'2","reported_seq":43,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T08:59:46.660350+0000","last_change":"2024-08-30T08:58:00.390839+0000","last_active":"2024-08-30T08:59:46.660350+0000","last_peered":"2024-08-30T08:59:46.660350+0000","last_clean":"2024-08-30T08:59:46.660350+0000","last_became_active":"2024-08-30T08:58:00.390422+0000","last_became_peered":"2024-08-30T08:58:00.390422+0000","last_unstale":"2024-08-30T08:59:46.660350+0000","last_undegraded":"2024-08-30T08:59:46.660350+0000","last_fullsized":"2024-08-30T08:59:46.660350+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T15:05:23.005075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T08:59:46.659347+0000","last_change":"2024-08-30T08:58:46.824563+0000","last_active":"2024-08-30T08:59:46.659347+0000","last_peered":"2024-08-30T08:59:46.659347+0000","last_clean":"2024-08-30T08:59:46.659347+0000","last_became_active":"2024-08-30T08:58:46.824163+0000","last_became_peered":"2024-08-30T08:58:46.824163+0000","last_unstale":"2024-08-30T08:59:46.659347+0000","last_undegraded":"2024-08-30T08:59:46.659347+0000","last_fullsized":"2024-08-30T08:59:46.659347+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T19:25:13.202762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,3,1],"acting":[8,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"83'6","reported_seq":41,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:15.315507+0000","last_change":"2024-08-30T08:58:02.405122+0000","last_active":"2024-08-30T09:00:15.315507+0000","last_peered":"2024-08-30T09:00:15.315507+0000","last_clean":"2024-08-30T09:00:15.315507+0000","last_became_active":"2024-08-30T08:58:02.404695+0000","last_became_peered":"2024-08-30T08:58:02.404695+0000","last_unstale":"2024-08-30T09:00:15.315507+0000","last_undegraded":"2024-08-30T09:00:15.315507+0000","last_fullsized":"2024-08-30T09:00:15.315507+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T15:50:01.447032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"81'15","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T08:59:46.660605+0000","last_change":"2024-08-30T08:58:46.573221+0000","last_active":"2024-08-30T08:59:46.660605+0000","last_peered":"2024-08-30T08:59:46.660605+0000","last_clean":"2024-08-30T08:59:46.660605+0000","last_became_active":"2024-08-30T08:58:46.559562+0000","last_became_peered":"2024-08-30T08:58:46.559562+0000","last_unstale":"2024-08-30T08:59:46.660605+0000","last_undegraded":"2024-08-30T08:59:46.660605+0000","last_fullsized":"2024-08-30T08:59:46.660605+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T16:21:23.182676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T08:59:46.701861+0000","last_change":"2024-08-30T08:58:09.446063+0000","last_active":"2024-08-30T08:59:46.701861+0000","last_peered":"2024-08-30T08:59:46.701861+0000","last_clean":"2024-08-30T08:59:46.701861+0000","last_became_active":"2024-08-30T08:58:09.445615+0000","last_became_peered":"2024-08-30T08:58:09.445615+0000","last_unstale":"2024-08-30T08:59:46.701861+0000","last_undegraded":"2024-08-30T08:59:46.701861+0000","last_fullsized":"2024-08-30T08:59:46.701861+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T10:25:10.904892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"83'6","reported_seq":41,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:15.330498+0000","last_change":"2024-08-30T08:58:02.412178+0000","last_active":"2024-08-30T09:00:15.330498+0000","last_peered":"2024-08-30T09:00:15.330498+0000","last_clean":"2024-08-30T09:00:15.330498+0000","last_became_active":"2024-08-30T08:58:02.411814+0000","last_became_peered":"2024-08-30T08:58:02.411814+0000","last_unstale":"2024-08-30T09:00:15.330498+0000","last_undegraded":"2024-08-30T09:00:15.330498+0000","last_fullsized":"2024-08-30T09:00:15.330498+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T10:02:59.070204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"83'5","reported_seq":144,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:24.764937+0000","last_change":"2024-08-30T08:58:00.390231+0000","last_active":"2024-08-30T09:00:24.764937+0000","last_peered":"2024-08-30T09:00:24.764937+0000","last_clean":"2024-08-30T09:00:24.764937+0000","last_became_active":"2024-08-30T08:58:00.389852+0000","last_became_peered":"2024-08-30T08:58:00.389852+0000","last_unstale":"2024-08-30T09:00:24.764937+0000","last_undegraded":"2024-08-30T09:00:24.764937+0000","last_fullsized":"2024-08-30T09:00:24.764937+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T12:09:12.105416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":100,"num_read_kb":4,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T08:59:46.656215+0000","last_change":"2024-08-30T08:58:09.441652+0000","last_active":"2024-08-30T08:59:46.656215+0000","last_peered":"2024-08-30T08:59:46.656215+0000","last_clean":"2024-08-30T08:59:46.656215+0000","last_became_active":"2024-08-30T08:58:09.441264+0000","last_became_peered":"2024-08-30T08:58:09.441264+0000","last_unstale":"2024-08-30T08:59:46.656215+0000","last_undegraded":"2024-08-30T08:59:46.656215+0000","last_fullsized":"2024-08-30T08:59:46.656215+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T09:42:25.636669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"83'15","reported_seq":50,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:15.332517+0000","last_change":"2024-08-30T08:58:02.416347+0000","last_active":"2024-08-30T09:00:15.332517+0000","last_peered":"2024-08-30T09:00:15.332517+0000","last_clean":"2024-08-30T09:00:15.332517+0000","last_became_active":"2024-08-30T08:58:02.415898+0000","last_became_peered":"2024-08-30T08:58:02.415898+0000","last_unstale":"2024-08-30T09:00:15.332517+0000","last_undegraded":"2024-08-30T09:00:15.332517+0000","last_fullsized":"2024-08-30T09:00:15.332517+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T10:18:15.507076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"79'3","reported_seq":43,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T08:59:46.663379+0000","last_change":"2024-08-30T08:58:00.393616+0000","last_active":"2024-08-30T08:59:46.663379+0000","last_peered":"2024-08-30T08:59:46.663379+0000","last_clean":"2024-08-30T08:59:46.663379+0000","last_became_active":"2024-08-30T08:58:00.393218+0000","last_became_peered":"2024-08-30T08:58:00.393218+0000","last_unstale":"2024-08-30T08:59:46.663379+0000","last_undegraded":"2024-08-30T08:59:46.663379+0000","last_fullsized":"2024-08-30T08:59:46.663379+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T10:56:37.175968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T08:59:46.661446+0000","last_change":"2024-08-30T08:58:47.092990+0000","last_active":"2024-08-30T08:59:46.661446+0000","last_peered":"2024-08-30T08:59:46.661446+0000","last_clean":"2024-08-30T08:59:46.661446+0000","last_became_active":"2024-08-30T08:58:47.092448+0000","last_became_peered":"2024-08-30T08:58:47.092448+0000","last_unstale":"2024-08-30T08:59:46.661446+0000","last_undegraded":"2024-08-30T08:59:46.661446+0000","last_fullsized":"2024-08-30T08:59:46.661446+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T20:23:06.527336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,7,6],"acting":[11,10,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"83'9","reported_seq":44,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:15.317768+0000","last_change":"2024-08-30T08:58:02.405783+0000","last_active":"2024-08-30T09:00:15.317768+0000","last_peered":"2024-08-30T09:00:15.317768+0000","last_clean":"2024-08-30T09:00:15.317768+0000","last_became_active":"2024-08-30T08:58:02.405430+0000","last_became_peered":"2024-08-30T08:58:02.405430+0000","last_unstale":"2024-08-30T09:00:15.317768+0000","last_undegraded":"2024-08-30T09:00:15.317768+0000","last_fullsized":"2024-08-30T09:00:15.317768+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T19:40:08.377259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"80'6","reported_seq":47,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T08:59:46.657435+0000","last_change":"2024-08-30T08:58:00.389617+0000","last_active":"2024-08-30T08:59:46.657435+0000","last_peered":"2024-08-30T08:59:46.657435+0000","last_clean":"2024-08-30T08:59:46.657435+0000","last_became_active":"2024-08-30T08:58:00.389031+0000","last_became_peered":"2024-08-30T08:58:00.389031+0000","last_unstale":"2024-08-30T08:59:46.657435+0000","last_undegraded":"2024-08-30T08:59:46.657435+0000","last_fullsized":"2024-08-30T08:59:46.657435+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T15:42:38.050768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T08:59:46.663773+0000","last_change":"2024-08-30T08:58:09.451925+0000","last_active":"2024-08-30T08:59:46.663773+0000","last_peered":"2024-08-30T08:59:46.663773+0000","last_clean":"2024-08-30T08:59:46.663773+0000","last_became_active":"2024-08-30T08:58:09.446893+0000","last_became_peered":"2024-08-30T08:58:09.446893+0000","last_unstale":"2024-08-30T08:59:46.663773+0000","last_undegraded":"2024-08-30T08:59:46.663773+0000","last_fullsized":"2024-08-30T08:59:46.663773+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T14:34:25.134006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"83'6","reported_seq":41,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:15.315731+0000","last_change":"2024-08-30T08:58:02.411194+0000","last_active":"2024-08-30T09:00:15.315731+0000","last_peered":"2024-08-30T09:00:15.315731+0000","last_clean":"2024-08-30T09:00:15.315731+0000","last_became_active":"2024-08-30T08:58:02.410183+0000","last_became_peered":"2024-08-30T08:58:02.410183+0000","last_unstale":"2024-08-30T09:00:15.315731+0000","last_undegraded":"2024-08-30T09:00:15.315731+0000","last_fullsized":"2024-08-30T09:00:15.315731+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T10:56:19.016115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"83'38","reported_seq":283,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:26.768557+0000","last_change":"2024-08-30T08:58:00.389808+0000","last_active":"2024-08-30T09:00:26.768557+0000","last_peered":"2024-08-30T09:00:26.768557+0000","last_clean":"2024-08-30T09:00:26.768557+0000","last_became_active":"2024-08-30T08:58:00.389411+0000","last_became_peered":"2024-08-30T08:58:00.389411+0000","last_unstale":"2024-08-30T09:00:26.768557+0000","last_undegraded":"2024-08-30T09:00:26.768557+0000","last_fullsized":"2024-08-30T09:00:26.768557+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T18:59:45.468456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64617,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":205,"num_read_kb":166,"num_write":39,"num_write_kb":82,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T08:59:46.656029+0000","last_change":"2024-08-30T08:58:09.447360+0000","last_active":"2024-08-30T08:59:46.656029+0000","last_peered":"2024-08-30T08:59:46.656029+0000","last_clean":"2024-08-30T08:59:46.656029+0000","last_became_active":"2024-08-30T08:58:09.446913+0000","last_became_peered":"2024-08-30T08:58:09.446913+0000","last_unstale":"2024-08-30T08:59:46.656029+0000","last_undegraded":"2024-08-30T08:59:46.656029+0000","last_fullsized":"2024-08-30T08:59:46.656029+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T13:59:43.147370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"83'11","reported_seq":46,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:15.334813+0000","last_change":"2024-08-30T08:58:02.408920+0000","last_active":"2024-08-30T09:00:15.334813+0000","last_peered":"2024-08-30T09:00:15.334813+0000","last_clean":"2024-08-30T09:00:15.334813+0000","last_became_active":"2024-08-30T08:58:02.408204+0000","last_became_peered":"2024-08-30T08:58:02.408204+0000","last_unstale":"2024-08-30T09:00:15.334813+0000","last_undegraded":"2024-08-30T09:00:15.334813+0000","last_fullsized":"2024-08-30T09:00:15.334813+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T11:44:00.765672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"79'4","reported_seq":45,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T08:59:46.664024+0000","last_change":"2024-08-30T08:58:00.482348+0000","last_active":"2024-08-30T08:59:46.664024+0000","last_peered":"2024-08-30T08:59:46.664024+0000","last_clean":"2024-08-30T08:59:46.664024+0000","last_became_active":"2024-08-30T08:58:00.481929+0000","last_became_peered":"2024-08-30T08:58:00.481929+0000","last_unstale":"2024-08-30T08:59:46.664024+0000","last_undegraded":"2024-08-30T08:59:46.664024+0000","last_fullsized":"2024-08-30T08:59:46.664024+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T11:13:55.332639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T08:59:46.662774+0000","last_change":"2024-08-30T08:58:09.443259+0000","last_active":"2024-08-30T08:59:46.662774+0000","last_peered":"2024-08-30T08:59:46.662774+0000","last_clean":"2024-08-30T08:59:46.662774+0000","last_became_active":"2024-08-30T08:58:09.442209+0000","last_became_peered":"2024-08-30T08:58:09.442209+0000","last_unstale":"2024-08-30T08:59:46.662774+0000","last_undegraded":"2024-08-30T08:59:46.662774+0000","last_fullsized":"2024-08-30T08:59:46.662774+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T09:25:56.233525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"83'9","reported_seq":43,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:15.333201+0000","last_change":"2024-08-30T08:58:02.407183+0000","last_active":"2024-08-30T09:00:15.333201+0000","last_peered":"2024-08-30T09:00:15.333201+0000","last_clean":"2024-08-30T09:00:15.333201+0000","last_became_active":"2024-08-30T08:58:02.406838+0000","last_became_peered":"2024-08-30T08:58:02.406838+0000","last_unstale":"2024-08-30T09:00:15.333201+0000","last_undegraded":"2024-08-30T09:00:15.333201+0000","last_fullsized":"2024-08-30T09:00:15.333201+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T13:40:32.937626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"79'2","reported_seq":42,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T08:59:46.663284+0000","last_change":"2024-08-30T08:58:00.392322+0000","last_active":"2024-08-30T08:59:46.663284+0000","last_peered":"2024-08-30T08:59:46.663284+0000","last_clean":"2024-08-30T08:59:46.663284+0000","last_became_active":"2024-08-30T08:58:00.391935+0000","last_became_peered":"2024-08-30T08:58:00.391935+0000","last_unstale":"2024-08-30T08:59:46.663284+0000","last_undegraded":"2024-08-30T08:59:46.663284+0000","last_fullsized":"2024-08-30T08:59:46.663284+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T12:20:52.368285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T08:59:46.655884+0000","last_change":"2024-08-30T08:58:09.441824+0000","last_active":"2024-08-30T08:59:46.655884+0000","last_peered":"2024-08-30T08:59:46.655884+0000","last_clean":"2024-08-30T08:59:46.655884+0000","last_became_active":"2024-08-30T08:58:09.441398+0000","last_became_peered":"2024-08-30T08:58:09.441398+0000","last_unstale":"2024-08-30T08:59:46.655884+0000","last_undegraded":"2024-08-30T08:59:46.655884+0000","last_fullsized":"2024-08-30T08:59:46.655884+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T14:05:45.710280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"83'10","reported_seq":45,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:15.334428+0000","last_change":"2024-08-30T08:58:02.403431+0000","last_active":"2024-08-30T09:00:15.334428+0000","last_peered":"2024-08-30T09:00:15.334428+0000","last_clean":"2024-08-30T09:00:15.334428+0000","last_became_active":"2024-08-30T08:58:02.403029+0000","last_became_peered":"2024-08-30T08:58:02.403029+0000","last_unstale":"2024-08-30T09:00:15.334428+0000","last_undegraded":"2024-08-30T09:00:15.334428+0000","last_fullsized":"2024-08-30T09:00:15.334428+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T09:34:56.452675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"83'4","reported_seq":141,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:26.768548+0000","last_change":"2024-08-30T08:58:00.389856+0000","last_active":"2024-08-30T09:00:26.768548+0000","last_peered":"2024-08-30T09:00:26.768548+0000","last_clean":"2024-08-30T09:00:26.768548+0000","last_became_active":"2024-08-30T08:58:00.389459+0000","last_became_peered":"2024-08-30T08:58:00.389459+0000","last_unstale":"2024-08-30T09:00:26.768548+0000","last_undegraded":"2024-08-30T09:00:26.768548+0000","last_fullsized":"2024-08-30T09:00:26.768548+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T18:32:06.214570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":97,"num_read_kb":97,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T08:59:46.701213+0000","last_change":"2024-08-30T08:58:09.445317+0000","last_active":"2024-08-30T08:59:46.701213+0000","last_peered":"2024-08-30T08:59:46.701213+0000","last_clean":"2024-08-30T08:59:46.701213+0000","last_became_active":"2024-08-30T08:58:09.444921+0000","last_became_peered":"2024-08-30T08:58:09.444921+0000","last_unstale":"2024-08-30T08:59:46.701213+0000","last_undegraded":"2024-08-30T08:59:46.701213+0000","last_fullsized":"2024-08-30T08:59:46.701213+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T09:22:13.038437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T08:59:46.842401+0000","last_change":"2024-08-30T08:58:09.447469+0000","last_active":"2024-08-30T08:59:46.842401+0000","last_peered":"2024-08-30T08:59:46.842401+0000","last_clean":"2024-08-30T08:59:46.842401+0000","last_became_active":"2024-08-30T08:58:09.447015+0000","last_became_peered":"2024-08-30T08:58:09.447015+0000","last_unstale":"2024-08-30T08:59:46.842401+0000","last_undegraded":"2024-08-30T08:59:46.842401+0000","last_fullsized":"2024-08-30T08:59:46.842401+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T09:18:36.734380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"83'11","reported_seq":46,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:15.317241+0000","last_change":"2024-08-30T08:58:02.403371+0000","last_active":"2024-08-30T09:00:15.317241+0000","last_peered":"2024-08-30T09:00:15.317241+0000","last_clean":"2024-08-30T09:00:15.317241+0000","last_became_active":"2024-08-30T08:58:02.403012+0000","last_became_peered":"2024-08-30T08:58:02.403012+0000","last_unstale":"2024-08-30T09:00:15.317241+0000","last_undegraded":"2024-08-30T09:00:15.317241+0000","last_fullsized":"2024-08-30T09:00:15.317241+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T10:29:29.683081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"79'1","reported_seq":41,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T08:59:46.663441+0000","last_change":"2024-08-30T08:58:00.394450+0000","last_active":"2024-08-30T08:59:46.663441+0000","last_peered":"2024-08-30T08:59:46.663441+0000","last_clean":"2024-08-30T08:59:46.663441+0000","last_became_active":"2024-08-30T08:58:00.391843+0000","last_became_peered":"2024-08-30T08:58:00.391843+0000","last_unstale":"2024-08-30T08:59:46.663441+0000","last_undegraded":"2024-08-30T08:59:46.663441+0000","last_fullsized":"2024-08-30T08:59:46.663441+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T10:29:18.947898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T08:59:46.661222+0000","last_change":"2024-08-30T08:58:09.449172+0000","last_active":"2024-08-30T08:59:46.661222+0000","last_peered":"2024-08-30T08:59:46.661222+0000","last_clean":"2024-08-30T08:59:46.661222+0000","last_became_active":"2024-08-30T08:58:09.448687+0000","last_became_peered":"2024-08-30T08:58:09.448687+0000","last_unstale":"2024-08-30T08:59:46.661222+0000","last_undegraded":"2024-08-30T08:59:46.661222+0000","last_fullsized":"2024-08-30T08:59:46.661222+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T13:23:16.090251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"83'8","reported_seq":42,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:15.315455+0000","last_change":"2024-08-30T08:58:02.400834+0000","last_active":"2024-08-30T09:00:15.315455+0000","last_peered":"2024-08-30T09:00:15.315455+0000","last_clean":"2024-08-30T09:00:15.315455+0000","last_became_active":"2024-08-30T08:58:02.400477+0000","last_became_peered":"2024-08-30T08:58:02.400477+0000","last_unstale":"2024-08-30T09:00:15.315455+0000","last_undegraded":"2024-08-30T09:00:15.315455+0000","last_fullsized":"2024-08-30T09:00:15.315455+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T13:40:45.239732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"80'2","reported_seq":42,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T08:59:46.701030+0000","last_change":"2024-08-30T08:58:00.381699+0000","last_active":"2024-08-30T08:59:46.701030+0000","last_peered":"2024-08-30T08:59:46.701030+0000","last_clean":"2024-08-30T08:59:46.701030+0000","last_became_active":"2024-08-30T08:58:00.381189+0000","last_became_peered":"2024-08-30T08:58:00.381189+0000","last_unstale":"2024-08-30T08:59:46.701030+0000","last_undegraded":"2024-08-30T08:59:46.701030+0000","last_fullsized":"2024-08-30T08:59:46.701030+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T14:09:16.939062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"80'3","reported_seq":42,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T08:59:46.697436+0000","last_change":"2024-08-30T08:58:00.387430+0000","last_active":"2024-08-30T08:59:46.697436+0000","last_peered":"2024-08-30T08:59:46.697436+0000","last_clean":"2024-08-30T08:59:46.697436+0000","last_became_active":"2024-08-30T08:58:00.387043+0000","last_became_peered":"2024-08-30T08:58:00.387043+0000","last_unstale":"2024-08-30T08:59:46.697436+0000","last_undegraded":"2024-08-30T08:59:46.697436+0000","last_fullsized":"2024-08-30T08:59:46.697436+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T13:32:25.418105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"83'8","reported_seq":42,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:15.316380+0000","last_change":"2024-08-30T08:58:02.404798+0000","last_active":"2024-08-30T09:00:15.316380+0000","last_peered":"2024-08-30T09:00:15.316380+0000","last_clean":"2024-08-30T09:00:15.316380+0000","last_became_active":"2024-08-30T08:58:02.404435+0000","last_became_peered":"2024-08-30T08:58:02.404435+0000","last_unstale":"2024-08-30T09:00:15.316380+0000","last_undegraded":"2024-08-30T09:00:15.316380+0000","last_fullsized":"2024-08-30T09:00:15.316380+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T20:31:49.472832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T08:59:46.657368+0000","last_change":"2024-08-30T08:58:09.448297+0000","last_active":"2024-08-30T08:59:46.657368+0000","last_peered":"2024-08-30T08:59:46.657368+0000","last_clean":"2024-08-30T08:59:46.657368+0000","last_became_active":"2024-08-30T08:58:09.447444+0000","last_became_peered":"2024-08-30T08:58:09.447444+0000","last_unstale":"2024-08-30T08:59:46.657368+0000","last_undegraded":"2024-08-30T08:59:46.657368+0000","last_fullsized":"2024-08-30T08:59:46.657368+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T11:40:03.507824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"83'7","reported_seq":146,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:25.767458+0000","last_change":"2024-08-30T08:58:00.392418+0000","last_active":"2024-08-30T09:00:25.767458+0000","last_peered":"2024-08-30T09:00:25.767458+0000","last_clean":"2024-08-30T09:00:25.767458+0000","last_became_active":"2024-08-30T08:58:00.391985+0000","last_became_peered":"2024-08-30T08:58:00.391985+0000","last_unstale":"2024-08-30T09:00:25.767458+0000","last_undegraded":"2024-08-30T09:00:25.767458+0000","last_fullsized":"2024-08-30T09:00:25.767458+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T17:42:18.147754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":100,"num_read_kb":3,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"83'7","reported_seq":41,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:15.300043+0000","last_change":"2024-08-30T08:58:02.408754+0000","last_active":"2024-08-30T09:00:15.300043+0000","last_peered":"2024-08-30T09:00:15.300043+0000","last_clean":"2024-08-30T09:00:15.300043+0000","last_became_active":"2024-08-30T08:58:02.407127+0000","last_became_peered":"2024-08-30T08:58:02.407127+0000","last_unstale":"2024-08-30T09:00:15.300043+0000","last_undegraded":"2024-08-30T09:00:15.300043+0000","last_fullsized":"2024-08-30T09:00:15.300043+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T16:15:41.796091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T08:59:46.657646+0000","last_change":"2024-08-30T08:58:09.445389+0000","last_active":"2024-08-30T08:59:46.657646+0000","last_peered":"2024-08-30T08:59:46.657646+0000","last_clean":"2024-08-30T08:59:46.657646+0000","last_became_active":"2024-08-30T08:58:09.444831+0000","last_became_peered":"2024-08-30T08:58:09.444831+0000","last_unstale":"2024-08-30T08:59:46.657646+0000","last_undegraded":"2024-08-30T08:59:46.657646+0000","last_fullsized":"2024-08-30T08:59:46.657646+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T09:55:58.162397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"83'9","reported_seq":44,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:15.317322+0000","last_change":"2024-08-30T08:58:02.415184+0000","last_active":"2024-08-30T09:00:15.317322+0000","last_peered":"2024-08-30T09:00:15.317322+0000","last_clean":"2024-08-30T09:00:15.317322+0000","last_became_active":"2024-08-30T08:58:02.414625+0000","last_became_peered":"2024-08-30T08:58:02.414625+0000","last_unstale":"2024-08-30T09:00:15.317322+0000","last_undegraded":"2024-08-30T09:00:15.317322+0000","last_fullsized":"2024-08-30T09:00:15.317322+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T11:10:31.130524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"83'12","reported_seq":47,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:15.315361+0000","last_change":"2024-08-30T08:58:02.405392+0000","last_active":"2024-08-30T09:00:15.315361+0000","last_peered":"2024-08-30T09:00:15.315361+0000","last_clean":"2024-08-30T09:00:15.315361+0000","last_became_active":"2024-08-30T08:58:02.405009+0000","last_became_peered":"2024-08-30T08:58:02.405009+0000","last_unstale":"2024-08-30T09:00:15.315361+0000","last_undegraded":"2024-08-30T09:00:15.315361+0000","last_fullsized":"2024-08-30T09:00:15.315361+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T13:22:46.375598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"83'8","reported_seq":43,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:15.315926+0000","last_change":"2024-08-30T08:58:02.402360+0000","last_active":"2024-08-30T09:00:15.315926+0000","last_peered":"2024-08-30T09:00:15.315926+0000","last_clean":"2024-08-30T09:00:15.315926+0000","last_became_active":"2024-08-30T08:58:02.401856+0000","last_became_peered":"2024-08-30T08:58:02.401856+0000","last_unstale":"2024-08-30T09:00:15.315926+0000","last_undegraded":"2024-08-30T09:00:15.315926+0000","last_fullsized":"2024-08-30T09:00:15.315926+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T10:25:14.491345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"83'7","reported_seq":42,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:15.332529+0000","last_change":"2024-08-30T08:58:02.415940+0000","last_active":"2024-08-30T09:00:15.332529+0000","last_peered":"2024-08-30T09:00:15.332529+0000","last_clean":"2024-08-30T09:00:15.332529+0000","last_became_active":"2024-08-30T08:58:02.415567+0000","last_became_peered":"2024-08-30T08:58:02.415567+0000","last_unstale":"2024-08-30T09:00:15.332529+0000","last_undegraded":"2024-08-30T09:00:15.332529+0000","last_fullsized":"2024-08-30T09:00:15.332529+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T15:02:26.270343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"83'4","reported_seq":39,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:15.312692+0000","last_change":"2024-08-30T08:58:02.406766+0000","last_active":"2024-08-30T09:00:15.312692+0000","last_peered":"2024-08-30T09:00:15.312692+0000","last_clean":"2024-08-30T09:00:15.312692+0000","last_became_active":"2024-08-30T08:58:02.406345+0000","last_became_peered":"2024-08-30T08:58:02.406345+0000","last_unstale":"2024-08-30T09:00:15.312692+0000","last_undegraded":"2024-08-30T09:00:15.312692+0000","last_fullsized":"2024-08-30T09:00:15.312692+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T13:10:32.986817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"83'6","reported_seq":41,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:15.313675+0000","last_change":"2024-08-30T08:58:02.412891+0000","last_active":"2024-08-30T09:00:15.313675+0000","last_peered":"2024-08-30T09:00:15.313675+0000","last_clean":"2024-08-30T09:00:15.313675+0000","last_became_active":"2024-08-30T08:58:02.412532+0000","last_became_peered":"2024-08-30T08:58:02.412532+0000","last_unstale":"2024-08-30T09:00:15.313675+0000","last_undegraded":"2024-08-30T09:00:15.313675+0000","last_fullsized":"2024-08-30T09:00:15.313675+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T14:40:30.663678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"83'15","reported_seq":50,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:15.325109+0000","last_change":"2024-08-30T08:58:02.405862+0000","last_active":"2024-08-30T09:00:15.325109+0000","last_peered":"2024-08-30T09:00:15.325109+0000","last_clean":"2024-08-30T09:00:15.325109+0000","last_became_active":"2024-08-30T08:58:02.405319+0000","last_became_peered":"2024-08-30T08:58:02.405319+0000","last_unstale":"2024-08-30T09:00:15.325109+0000","last_undegraded":"2024-08-30T09:00:15.325109+0000","last_fullsized":"2024-08-30T09:00:15.325109+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T20:36:10.315814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"83'2","reported_seq":37,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:15.316363+0000","last_change":"2024-08-30T08:58:02.401410+0000","last_active":"2024-08-30T09:00:15.316363+0000","last_peered":"2024-08-30T09:00:15.316363+0000","last_clean":"2024-08-30T09:00:15.316363+0000","last_became_active":"2024-08-30T08:58:02.401060+0000","last_became_peered":"2024-08-30T08:58:02.401060+0000","last_unstale":"2024-08-30T09:00:15.316363+0000","last_undegraded":"2024-08-30T09:00:15.316363+0000","last_fullsized":"2024-08-30T09:00:15.316363+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T16:00:41.275367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"83'6","reported_seq":41,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:15.306834+0000","last_change":"2024-08-30T08:58:02.401703+0000","last_active":"2024-08-30T09:00:15.306834+0000","last_peered":"2024-08-30T09:00:15.306834+0000","last_clean":"2024-08-30T09:00:15.306834+0000","last_became_active":"2024-08-30T08:58:02.401218+0000","last_became_peered":"2024-08-30T08:58:02.401218+0000","last_unstale":"2024-08-30T09:00:15.306834+0000","last_undegraded":"2024-08-30T09:00:15.306834+0000","last_fullsized":"2024-08-30T09:00:15.306834+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T18:36:25.249814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"83'8","reported_seq":43,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:15.315612+0000","last_change":"2024-08-30T08:58:02.403609+0000","last_active":"2024-08-30T09:00:15.315612+0000","last_peered":"2024-08-30T09:00:15.315612+0000","last_clean":"2024-08-30T09:00:15.315612+0000","last_became_active":"2024-08-30T08:58:02.402195+0000","last_became_peered":"2024-08-30T08:58:02.402195+0000","last_unstale":"2024-08-30T09:00:15.315612+0000","last_undegraded":"2024-08-30T09:00:15.315612+0000","last_fullsized":"2024-08-30T09:00:15.315612+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T09:59:05.877523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"83'10","reported_seq":45,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:15.333885+0000","last_change":"2024-08-30T08:58:02.404389+0000","last_active":"2024-08-30T09:00:15.333885+0000","last_peered":"2024-08-30T09:00:15.333885+0000","last_clean":"2024-08-30T09:00:15.333885+0000","last_became_active":"2024-08-30T08:58:02.404039+0000","last_became_peered":"2024-08-30T08:58:02.404039+0000","last_unstale":"2024-08-30T09:00:15.333885+0000","last_undegraded":"2024-08-30T09:00:15.333885+0000","last_fullsized":"2024-08-30T09:00:15.333885+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T20:55:51.739542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"83'11","reported_seq":46,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:15.301840+0000","last_change":"2024-08-30T08:58:02.411807+0000","last_active":"2024-08-30T09:00:15.301840+0000","last_peered":"2024-08-30T09:00:15.301840+0000","last_clean":"2024-08-30T09:00:15.301840+0000","last_became_active":"2024-08-30T08:58:02.410977+0000","last_became_peered":"2024-08-30T08:58:02.410977+0000","last_unstale":"2024-08-30T09:00:15.301840+0000","last_undegraded":"2024-08-30T09:00:15.301840+0000","last_fullsized":"2024-08-30T09:00:15.301840+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T13:04:29.931373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"83'8","reported_seq":50,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:15.315288+0000","last_change":"2024-08-30T08:58:46.559661+0000","last_active":"2024-08-30T09:00:15.315288+0000","last_peered":"2024-08-30T09:00:15.315288+0000","last_clean":"2024-08-30T09:00:15.315288+0000","last_became_active":"2024-08-30T08:58:46.558824+0000","last_became_peered":"2024-08-30T08:58:46.558824+0000","last_unstale":"2024-08-30T09:00:15.315288+0000","last_undegraded":"2024-08-30T09:00:15.315288+0000","last_fullsized":"2024-08-30T09:00:15.315288+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T09:53:07.650921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"83'7","reported_seq":42,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:15.316869+0000","last_change":"2024-08-30T08:58:02.396186+0000","last_active":"2024-08-30T09:00:15.316869+0000","last_peered":"2024-08-30T09:00:15.316869+0000","last_clean":"2024-08-30T09:00:15.316869+0000","last_became_active":"2024-08-30T08:58:02.395776+0000","last_became_peered":"2024-08-30T08:58:02.395776+0000","last_unstale":"2024-08-30T09:00:15.316869+0000","last_undegraded":"2024-08-30T09:00:15.316869+0000","last_fullsized":"2024-08-30T09:00:15.316869+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T09:33:30.068673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"83'10","reported_seq":45,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:15.318659+0000","last_change":"2024-08-30T08:58:02.408378+0000","last_active":"2024-08-30T09:00:15.318659+0000","last_peered":"2024-08-30T09:00:15.318659+0000","last_clean":"2024-08-30T09:00:15.318659+0000","last_became_active":"2024-08-30T08:58:02.407991+0000","last_became_peered":"2024-08-30T08:58:02.407991+0000","last_unstale":"2024-08-30T09:00:15.318659+0000","last_undegraded":"2024-08-30T09:00:15.318659+0000","last_fullsized":"2024-08-30T09:00:15.318659+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T20:57:02.411176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"83'8","reported_seq":43,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:15.333624+0000","last_change":"2024-08-30T08:58:02.411829+0000","last_active":"2024-08-30T09:00:15.333624+0000","last_peered":"2024-08-30T09:00:15.333624+0000","last_clean":"2024-08-30T09:00:15.333624+0000","last_became_active":"2024-08-30T08:58:02.411416+0000","last_became_peered":"2024-08-30T08:58:02.411416+0000","last_unstale":"2024-08-30T09:00:15.333624+0000","last_undegraded":"2024-08-30T09:00:15.333624+0000","last_fullsized":"2024-08-30T09:00:15.333624+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T17:12:08.855150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"83'9","reported_seq":44,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:15.333466+0000","last_change":"2024-08-30T08:58:02.405122+0000","last_active":"2024-08-30T09:00:15.333466+0000","last_peered":"2024-08-30T09:00:15.333466+0000","last_clean":"2024-08-30T09:00:15.333466+0000","last_became_active":"2024-08-30T08:58:02.403312+0000","last_became_peered":"2024-08-30T08:58:02.403312+0000","last_unstale":"2024-08-30T09:00:15.333466+0000","last_undegraded":"2024-08-30T09:00:15.333466+0000","last_fullsized":"2024-08-30T09:00:15.333466+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T18:11:53.077433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"83'7","reported_seq":42,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:15.332861+0000","last_change":"2024-08-30T08:58:02.404263+0000","last_active":"2024-08-30T09:00:15.332861+0000","last_peered":"2024-08-30T09:00:15.332861+0000","last_clean":"2024-08-30T09:00:15.332861+0000","last_became_active":"2024-08-30T08:58:02.403897+0000","last_became_peered":"2024-08-30T08:58:02.403897+0000","last_unstale":"2024-08-30T09:00:15.332861+0000","last_undegraded":"2024-08-30T09:00:15.332861+0000","last_fullsized":"2024-08-30T09:00:15.332861+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T18:38:31.439434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"83'7","reported_seq":42,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:15.332364+0000","last_change":"2024-08-30T08:58:02.411578+0000","last_active":"2024-08-30T09:00:15.332364+0000","last_peered":"2024-08-30T09:00:15.332364+0000","last_clean":"2024-08-30T09:00:15.332364+0000","last_became_active":"2024-08-30T08:58:02.411271+0000","last_became_peered":"2024-08-30T08:58:02.411271+0000","last_unstale":"2024-08-30T09:00:15.332364+0000","last_undegraded":"2024-08-30T09:00:15.332364+0000","last_fullsized":"2024-08-30T09:00:15.332364+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T10:02:46.382906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"83'10","reported_seq":45,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:15.310698+0000","last_change":"2024-08-30T08:58:02.412168+0000","last_active":"2024-08-30T09:00:15.310698+0000","last_peered":"2024-08-30T09:00:15.310698+0000","last_clean":"2024-08-30T09:00:15.310698+0000","last_became_active":"2024-08-30T08:58:02.411420+0000","last_became_peered":"2024-08-30T08:58:02.411420+0000","last_unstale":"2024-08-30T09:00:15.310698+0000","last_undegraded":"2024-08-30T09:00:15.310698+0000","last_fullsized":"2024-08-30T09:00:15.310698+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T10:35:53.654344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"83'7","reported_seq":42,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:15.318663+0000","last_change":"2024-08-30T08:58:02.409735+0000","last_active":"2024-08-30T09:00:15.318663+0000","last_peered":"2024-08-30T09:00:15.318663+0000","last_clean":"2024-08-30T09:00:15.318663+0000","last_became_active":"2024-08-30T08:58:02.409349+0000","last_became_peered":"2024-08-30T08:58:02.409349+0000","last_unstale":"2024-08-30T09:00:15.318663+0000","last_undegraded":"2024-08-30T09:00:15.318663+0000","last_fullsized":"2024-08-30T09:00:15.318663+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T20:10:22.956554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"83'6","reported_seq":41,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:15.333587+0000","last_change":"2024-08-30T08:58:02.416261+0000","last_active":"2024-08-30T09:00:15.333587+0000","last_peered":"2024-08-30T09:00:15.333587+0000","last_clean":"2024-08-30T09:00:15.333587+0000","last_became_active":"2024-08-30T08:58:02.415194+0000","last_became_peered":"2024-08-30T08:58:02.415194+0000","last_unstale":"2024-08-30T09:00:15.333587+0000","last_undegraded":"2024-08-30T09:00:15.333587+0000","last_fullsized":"2024-08-30T09:00:15.333587+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T17:59:04.339699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"83'16","reported_seq":58,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:15.320984+0000","last_change":"2024-08-30T08:58:46.560190+0000","last_active":"2024-08-30T09:00:15.320984+0000","last_peered":"2024-08-30T09:00:15.320984+0000","last_clean":"2024-08-30T09:00:15.320984+0000","last_became_active":"2024-08-30T08:58:46.559674+0000","last_became_peered":"2024-08-30T08:58:46.559674+0000","last_unstale":"2024-08-30T09:00:15.320984+0000","last_undegraded":"2024-08-30T09:00:15.320984+0000","last_fullsized":"2024-08-30T09:00:15.320984+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T10:08:02.835142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"83'11","reported_seq":46,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:15.334030+0000","last_change":"2024-08-30T08:58:02.406763+0000","last_active":"2024-08-30T09:00:15.334030+0000","last_peered":"2024-08-30T09:00:15.334030+0000","last_clean":"2024-08-30T09:00:15.334030+0000","last_became_active":"2024-08-30T08:58:02.406377+0000","last_became_peered":"2024-08-30T08:58:02.406377+0000","last_unstale":"2024-08-30T09:00:15.334030+0000","last_undegraded":"2024-08-30T09:00:15.334030+0000","last_fullsized":"2024-08-30T09:00:15.334030+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T13:32:50.492921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"83'9","reported_seq":43,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:15.319363+0000","last_change":"2024-08-30T08:58:02.416638+0000","last_active":"2024-08-30T09:00:15.319363+0000","last_peered":"2024-08-30T09:00:15.319363+0000","last_clean":"2024-08-30T09:00:15.319363+0000","last_became_active":"2024-08-30T08:58:02.415518+0000","last_became_peered":"2024-08-30T08:58:02.415518+0000","last_unstale":"2024-08-30T09:00:15.319363+0000","last_undegraded":"2024-08-30T09:00:15.319363+0000","last_fullsized":"2024-08-30T09:00:15.319363+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T17:05:07.531585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"83'5","reported_seq":39,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:15.330612+0000","last_change":"2024-08-30T08:58:02.412973+0000","last_active":"2024-08-30T09:00:15.330612+0000","last_peered":"2024-08-30T09:00:15.330612+0000","last_clean":"2024-08-30T09:00:15.330612+0000","last_became_active":"2024-08-30T08:58:02.412609+0000","last_became_peered":"2024-08-30T08:58:02.412609+0000","last_unstale":"2024-08-30T09:00:15.330612+0000","last_undegraded":"2024-08-30T09:00:15.330612+0000","last_fullsized":"2024-08-30T09:00:15.330612+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T14:22:37.704098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"83'6","reported_seq":40,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:15.310078+0000","last_change":"2024-08-30T08:58:02.402779+0000","last_active":"2024-08-30T09:00:15.310078+0000","last_peered":"2024-08-30T09:00:15.310078+0000","last_clean":"2024-08-30T09:00:15.310078+0000","last_became_active":"2024-08-30T08:58:02.401514+0000","last_became_peered":"2024-08-30T08:58:02.401514+0000","last_unstale":"2024-08-30T09:00:15.310078+0000","last_undegraded":"2024-08-30T09:00:15.310078+0000","last_fullsized":"2024-08-30T09:00:15.310078+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T11:37:51.450334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"83'4","reported_seq":38,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:15.295684+0000","last_change":"2024-08-30T08:58:02.406607+0000","last_active":"2024-08-30T09:00:15.295684+0000","last_peered":"2024-08-30T09:00:15.295684+0000","last_clean":"2024-08-30T09:00:15.295684+0000","last_became_active":"2024-08-30T08:58:02.406237+0000","last_became_peered":"2024-08-30T08:58:02.406237+0000","last_unstale":"2024-08-30T09:00:15.295684+0000","last_undegraded":"2024-08-30T09:00:15.295684+0000","last_fullsized":"2024-08-30T09:00:15.295684+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T12:09:03.247424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"83'8","reported_seq":43,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:15.323344+0000","last_change":"2024-08-30T08:58:02.410749+0000","last_active":"2024-08-30T09:00:15.323344+0000","last_peered":"2024-08-30T09:00:15.323344+0000","last_clean":"2024-08-30T09:00:15.323344+0000","last_became_active":"2024-08-30T08:58:02.410350+0000","last_became_peered":"2024-08-30T08:58:02.410350+0000","last_unstale":"2024-08-30T09:00:15.323344+0000","last_undegraded":"2024-08-30T09:00:15.323344+0000","last_fullsized":"2024-08-30T09:00:15.323344+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T15:22:51.451789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"83'7","reported_seq":42,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:15.334454+0000","last_change":"2024-08-30T08:58:02.405946+0000","last_active":"2024-08-30T09:00:15.334454+0000","last_peered":"2024-08-30T09:00:15.334454+0000","last_clean":"2024-08-30T09:00:15.334454+0000","last_became_active":"2024-08-30T08:58:02.405590+0000","last_became_peered":"2024-08-30T08:58:02.405590+0000","last_unstale":"2024-08-30T09:00:15.334454+0000","last_undegraded":"2024-08-30T09:00:15.334454+0000","last_fullsized":"2024-08-30T09:00:15.334454+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T11:42:14.966065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"83'4","reported_seq":39,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:15.330681+0000","last_change":"2024-08-30T08:58:02.409649+0000","last_active":"2024-08-30T09:00:15.330681+0000","last_peered":"2024-08-30T09:00:15.330681+0000","last_clean":"2024-08-30T09:00:15.330681+0000","last_became_active":"2024-08-30T08:58:02.409282+0000","last_became_peered":"2024-08-30T08:58:02.409282+0000","last_unstale":"2024-08-30T09:00:15.330681+0000","last_undegraded":"2024-08-30T09:00:15.330681+0000","last_fullsized":"2024-08-30T09:00:15.330681+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T15:33:52.166290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"83'8","reported_seq":43,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:15.333394+0000","last_change":"2024-08-30T08:58:02.410711+0000","last_active":"2024-08-30T09:00:15.333394+0000","last_peered":"2024-08-30T09:00:15.333394+0000","last_clean":"2024-08-30T09:00:15.333394+0000","last_became_active":"2024-08-30T08:58:02.410367+0000","last_became_peered":"2024-08-30T08:58:02.410367+0000","last_unstale":"2024-08-30T09:00:15.333394+0000","last_undegraded":"2024-08-30T09:00:15.333394+0000","last_fullsized":"2024-08-30T09:00:15.333394+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T15:27:40.392588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":69291,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":612,"num_read_kb":380,"num_write":183,"num_write_kb":129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":207873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":67,"seq":287762808873,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":958464,"data_stored":474260,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":61,"seq":261993005103,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28628,"kb_used_data":1604,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716524,"statfs":{"total":95995035648,"available":95965720576,"internally_reserved":0,"allocated":1642496,"data_stored":1126565,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":57,"seq":244813135925,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36292,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708860,"statfs":{"total":95995035648,"available":95957872640,"internally_reserved":0,"allocated":970752,"data_stored":478480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":51,"seq":219043332154,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":958464,"data_stored":474192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":46,"seq":197568495679,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28548,"kb_used_data":1524,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716604,"statfs":{"total":95995035648,"available":95965802496,"internally_reserved":0,"allocated":1560576,"data_stored":1070886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":41,"seq":176093659204,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":946176,"data_stored":469560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":35,"seq":150323855433,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36876,"kb_used_data":1532,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708276,"statfs":{"total":95995035648,"available":95957274624,"internally_reserved":0,"allocated":1568768,"data_stored":1065942,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":30,"seq":128849018958,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36280,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708872,"statfs":{"total":95995035648,"available":95957884928,"internally_reserved":0,"allocated":958464,"data_stored":474742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215186,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36280,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708872,"statfs":{"total":95995035648,"available":95957884928,"internally_reserved":0,"allocated":958464,"data_stored":469754,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411416,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36360,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708792,"statfs":{"total":95995035648,"available":95957803008,"internally_reserved":0,"allocated":1040384,"data_stored":535969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574941,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1015808,"data_stored":534087,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738465,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":970752,"data_stored":470948,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":64617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":66499,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":66727,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-30T09:00:29.353 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-08-30T09:00:29.353 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-08-30T09:00:29.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:29.497+0000 7f9e90c39640 1 -- 172.21.15.42:0/3243472635 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9e8c104e30 msgr2=0x7f9e8c10b7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:00:29.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:29.497+0000 7f9e90c39640 1 --2- 172.21.15.42:0/3243472635 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9e8c104e30 0x7f9e8c10b7d0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f9e84009ca0 tx=0x7f9e8402a8c0 comp rx=0 tx=0).stop 2024-08-30T09:00:29.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:29.497+0000 7f9e90c39640 1 -- 172.21.15.42:0/3243472635 shutdown_connections 2024-08-30T09:00:29.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:29.497+0000 7f9e90c39640 1 --2- 172.21.15.42:0/3243472635 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9e8c104490 0x7f9e8c1048a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:29.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:29.497+0000 7f9e90c39640 1 --2- 172.21.15.42:0/3243472635 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9e8c104e30 0x7f9e8c10b7d0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:29.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:29.497+0000 7f9e90c39640 1 --2- 172.21.15.42:0/3243472635 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9e8c103410 0x7f9e8c103880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:29.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:29.497+0000 7f9e90c39640 1 -- 172.21.15.42:0/3243472635 >> 172.21.15.42:0/3243472635 conn(0x7f9e8c0fed70 msgr2=0x7f9e8c101200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:00:29.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:29.497+0000 7f9e90c39640 1 -- 172.21.15.42:0/3243472635 shutdown_connections 2024-08-30T09:00:29.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:29.497+0000 7f9e90c39640 1 -- 172.21.15.42:0/3243472635 wait complete. 2024-08-30T09:00:29.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:29.498+0000 7f9e90c39640 1 Processor -- start 2024-08-30T09:00:29.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:29.498+0000 7f9e90c39640 1 -- start start 2024-08-30T09:00:29.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:29.498+0000 7f9e90c39640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9e8c103410 0x7f9e8c199c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:29.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:29.498+0000 7f9e90c39640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9e8c104490 0x7f9e8c19a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:29.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:29.498+0000 7f9e90c39640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9e8c104e30 0x7f9e8c19ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:29.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:29.498+0000 7f9e90c39640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f9e8c10f090 con 0x7f9e8c103410 2024-08-30T09:00:29.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:29.498+0000 7f9e90c39640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f9e8c10ef10 con 0x7f9e8c104490 2024-08-30T09:00:29.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:29.498+0000 7f9e90c39640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f9e8c10f210 con 0x7f9e8c104e30 2024-08-30T09:00:29.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:29.498+0000 7f9e8a59c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9e8c103410 0x7f9e8c199c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:29.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:29.498+0000 7f9e8a59c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9e8c103410 0x7f9e8c199c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:51456/0 (socket says 172.21.15.42:51456) 2024-08-30T09:00:29.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:29.498+0000 7f9e8a59c640 1 -- 172.21.15.42:0/2385234037 learned_addr learned my addr 172.21.15.42:0/2385234037 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:00:29.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:29.498+0000 7f9e89d9b640 1 --2- 172.21.15.42:0/2385234037 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9e8c104490 0x7f9e8c19a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:29.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:29.498+0000 7f9e8ad9d640 1 --2- 172.21.15.42:0/2385234037 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9e8c104e30 0x7f9e8c19ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:29.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:29.498+0000 7f9e8a59c640 1 -- 172.21.15.42:0/2385234037 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9e8c104e30 msgr2=0x7f9e8c19ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:00:29.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:29.498+0000 7f9e8a59c640 1 --2- 172.21.15.42:0/2385234037 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9e8c104e30 0x7f9e8c19ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:29.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:29.499+0000 7f9e8a59c640 1 -- 172.21.15.42:0/2385234037 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9e8c104490 msgr2=0x7f9e8c19a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:00:29.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:29.499+0000 7f9e8a59c640 1 --2- 172.21.15.42:0/2385234037 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9e8c104490 0x7f9e8c19a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:29.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:29.499+0000 7f9e8a59c640 1 -- 172.21.15.42:0/2385234037 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e8c19b680 con 0x7f9e8c103410 2024-08-30T09:00:29.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:29.499+0000 7f9e8a59c640 1 --2- 172.21.15.42:0/2385234037 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9e8c103410 0x7f9e8c199c40 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f9e7c00c890 tx=0x7f9e7c00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:00:29.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:29.499+0000 7f9e7b7fe640 1 -- 172.21.15.42:0/2385234037 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e7c004220 con 0x7f9e8c103410 2024-08-30T09:00:29.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:29.499+0000 7f9e7b7fe640 1 -- 172.21.15.42:0/2385234037 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9e7c0043c0 con 0x7f9e8c103410 2024-08-30T09:00:29.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:29.499+0000 7f9e90c39640 1 -- 172.21.15.42:0/2385234037 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e8c19b970 con 0x7f9e8c103410 2024-08-30T09:00:29.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:29.499+0000 7f9e7b7fe640 1 -- 172.21.15.42:0/2385234037 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e7c004a00 con 0x7f9e8c103410 2024-08-30T09:00:29.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:29.499+0000 7f9e90c39640 1 -- 172.21.15.42:0/2385234037 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e8c1a2060 con 0x7f9e8c103410 2024-08-30T09:00:29.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:29.500+0000 7f9e90c39640 1 -- 172.21.15.42:0/2385234037 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e8c10a5e0 con 0x7f9e8c103410 2024-08-30T09:00:29.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:29.502+0000 7f9e7b7fe640 1 -- 172.21.15.42:0/2385234037 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7f9e7c0079b0 con 0x7f9e8c103410 2024-08-30T09:00:29.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:29.502+0000 7f9e7b7fe640 1 --2- 172.21.15.42:0/2385234037 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f9e740b4b30 0x7f9e740b7040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:29.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:29.503+0000 7f9e7b7fe640 1 -- 172.21.15.42:0/2385234037 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f9e7c0e5720 con 0x7f9e8c103410 2024-08-30T09:00:29.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:29.503+0000 7f9e89d9b640 1 --2- 172.21.15.42:0/2385234037 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f9e740b4b30 0x7f9e740b7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:29.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:29.503+0000 7f9e89d9b640 1 --2- 172.21.15.42:0/2385234037 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f9e740b4b30 0x7f9e740b7040 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f9e80005fd0 tx=0x7f9e80005e20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:00:29.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:29.505+0000 7f9e7b7fe640 1 -- 172.21.15.42:0/2385234037 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f9e7c014030 con 0x7f9e8c103410 2024-08-30T09:00:29.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:29.686+0000 7f9e90c39640 1 -- 172.21.15.42:0/2385234037 --> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9e8c0635a0 con 0x7f9e740b4b30 2024-08-30T09:00:29.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:29.692+0000 7f9e7b7fe640 1 -- 172.21.15.42:0/2385234037 <== mgr.14150 v2:172.21.15.42:6800/3005140902 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340452 (secure 0 0 0) 0x7f9e8c0635a0 con 0x7f9e740b4b30 2024-08-30T09:00:29.697 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-08-30T09:00:29.699 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-08-30T09:00:29.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:29.696+0000 7f9e90c39640 1 -- 172.21.15.42:0/2385234037 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f9e740b4b30 msgr2=0x7f9e740b7040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:00:29.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:29.696+0000 7f9e90c39640 1 --2- 172.21.15.42:0/2385234037 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f9e740b4b30 0x7f9e740b7040 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f9e80005fd0 tx=0x7f9e80005e20 comp rx=0 tx=0).stop 2024-08-30T09:00:29.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:29.696+0000 7f9e90c39640 1 -- 172.21.15.42:0/2385234037 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9e8c103410 msgr2=0x7f9e8c199c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:00:29.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:29.696+0000 7f9e90c39640 1 --2- 172.21.15.42:0/2385234037 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9e8c103410 0x7f9e8c199c40 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f9e7c00c890 tx=0x7f9e7c00cd50 comp rx=0 tx=0).stop 2024-08-30T09:00:29.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:29.696+0000 7f9e90c39640 1 -- 172.21.15.42:0/2385234037 shutdown_connections 2024-08-30T09:00:29.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:29.696+0000 7f9e90c39640 1 --2- 172.21.15.42:0/2385234037 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f9e740b4b30 0x7f9e740b7040 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:29.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:29.696+0000 7f9e90c39640 1 --2- 172.21.15.42:0/2385234037 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9e8c104e30 0x7f9e8c19ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:29.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:29.696+0000 7f9e90c39640 1 --2- 172.21.15.42:0/2385234037 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9e8c104490 0x7f9e8c19a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:29.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:29.696+0000 7f9e90c39640 1 --2- 172.21.15.42:0/2385234037 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9e8c103410 0x7f9e8c199c40 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:29.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:29.696+0000 7f9e90c39640 1 -- 172.21.15.42:0/2385234037 >> 172.21.15.42:0/2385234037 conn(0x7f9e8c0fed70 msgr2=0x7f9e8c1077b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:00:29.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:29.696+0000 7f9e90c39640 1 -- 172.21.15.42:0/2385234037 shutdown_connections 2024-08-30T09:00:29.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:29.696+0000 7f9e90c39640 1 -- 172.21.15.42:0/2385234037 wait complete. 2024-08-30T09:00:29.709 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":365,"stamp":"2024-08-30T09:00:28.967886+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":659893,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":688,"num_read_kb":444,"num_write":308,"num_write_kb":2281,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":2951840,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":750,"ondisk_log_size":750,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":379120,"kb_used_data":13232,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124562704,"statfs":{"total":1151940427776,"available":1151552208896,"internally_reserved":0,"allocated":13549568,"data_stored":7645385,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":374453561},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":698,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":88,"num_read_kb":48,"num_write":11,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005884"},"pg_stats":[{"pgid":"2.1f","version":"80'9","reported_seq":50,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T08:59:46.656396+0000","last_change":"2024-08-30T08:58:00.388595+0000","last_active":"2024-08-30T08:59:46.656396+0000","last_peered":"2024-08-30T08:59:46.656396+0000","last_clean":"2024-08-30T08:59:46.656396+0000","last_became_active":"2024-08-30T08:58:00.388215+0000","last_became_peered":"2024-08-30T08:58:00.388215+0000","last_unstale":"2024-08-30T08:59:46.656396+0000","last_undegraded":"2024-08-30T08:59:46.656396+0000","last_fullsized":"2024-08-30T08:59:46.656396+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T13:00:06.109935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"83'5","reported_seq":40,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:15.312524+0000","last_change":"2024-08-30T08:58:02.416042+0000","last_active":"2024-08-30T09:00:15.312524+0000","last_peered":"2024-08-30T09:00:15.312524+0000","last_clean":"2024-08-30T09:00:15.312524+0000","last_became_active":"2024-08-30T08:58:02.415507+0000","last_became_peered":"2024-08-30T08:58:02.415507+0000","last_unstale":"2024-08-30T09:00:15.312524+0000","last_undegraded":"2024-08-30T09:00:15.312524+0000","last_fullsized":"2024-08-30T09:00:15.312524+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T14:33:57.192004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T08:59:46.657037+0000","last_change":"2024-08-30T08:58:09.448785+0000","last_active":"2024-08-30T08:59:46.657037+0000","last_peered":"2024-08-30T08:59:46.657037+0000","last_clean":"2024-08-30T08:59:46.657037+0000","last_became_active":"2024-08-30T08:58:09.448191+0000","last_became_peered":"2024-08-30T08:58:09.448191+0000","last_unstale":"2024-08-30T08:59:46.657037+0000","last_undegraded":"2024-08-30T08:59:46.657037+0000","last_fullsized":"2024-08-30T08:59:46.657037+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T13:37:19.148201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"80'2","reported_seq":42,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T08:59:46.702141+0000","last_change":"2024-08-30T08:58:00.480406+0000","last_active":"2024-08-30T08:59:46.702141+0000","last_peered":"2024-08-30T08:59:46.702141+0000","last_clean":"2024-08-30T08:59:46.702141+0000","last_became_active":"2024-08-30T08:58:00.479896+0000","last_became_peered":"2024-08-30T08:58:00.479896+0000","last_unstale":"2024-08-30T08:59:46.702141+0000","last_undegraded":"2024-08-30T08:59:46.702141+0000","last_fullsized":"2024-08-30T08:59:46.702141+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T15:16:15.316343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"83'6","reported_seq":41,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:15.335025+0000","last_change":"2024-08-30T08:58:02.406915+0000","last_active":"2024-08-30T09:00:15.335025+0000","last_peered":"2024-08-30T09:00:15.335025+0000","last_clean":"2024-08-30T09:00:15.335025+0000","last_became_active":"2024-08-30T08:58:02.406557+0000","last_became_peered":"2024-08-30T08:58:02.406557+0000","last_unstale":"2024-08-30T09:00:15.335025+0000","last_undegraded":"2024-08-30T09:00:15.335025+0000","last_fullsized":"2024-08-30T09:00:15.335025+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T19:15:30.847212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T08:59:46.661576+0000","last_change":"2024-08-30T08:58:09.446974+0000","last_active":"2024-08-30T08:59:46.661576+0000","last_peered":"2024-08-30T08:59:46.661576+0000","last_clean":"2024-08-30T08:59:46.661576+0000","last_became_active":"2024-08-30T08:58:09.446173+0000","last_became_peered":"2024-08-30T08:58:09.446173+0000","last_unstale":"2024-08-30T08:59:46.661576+0000","last_undegraded":"2024-08-30T08:59:46.661576+0000","last_fullsized":"2024-08-30T08:59:46.661576+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T12:13:41.276825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"80'4","reported_seq":56,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T08:59:46.673167+0000","last_change":"2024-08-30T08:59:46.673048+0000","last_active":"2024-08-30T08:59:46.673167+0000","last_peered":"2024-08-30T08:59:46.673167+0000","last_clean":"2024-08-30T08:59:46.673167+0000","last_became_active":"2024-08-30T08:59:46.665588+0000","last_became_peered":"2024-08-30T08:59:46.665588+0000","last_unstale":"2024-08-30T08:59:46.673167+0000","last_undegraded":"2024-08-30T08:59:46.673167+0000","last_fullsized":"2024-08-30T08:59:46.673167+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T18:02:29.663191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,7],"acting":[8,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"83'9","reported_seq":43,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:15.334804+0000","last_change":"2024-08-30T08:58:02.415594+0000","last_active":"2024-08-30T09:00:15.334804+0000","last_peered":"2024-08-30T09:00:15.334804+0000","last_clean":"2024-08-30T09:00:15.334804+0000","last_became_active":"2024-08-30T08:58:02.415066+0000","last_became_peered":"2024-08-30T08:58:02.415066+0000","last_unstale":"2024-08-30T09:00:15.334804+0000","last_undegraded":"2024-08-30T09:00:15.334804+0000","last_fullsized":"2024-08-30T09:00:15.334804+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T12:07:27.668442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T08:59:46.662698+0000","last_change":"2024-08-30T08:58:09.444403+0000","last_active":"2024-08-30T08:59:46.662698+0000","last_peered":"2024-08-30T08:59:46.662698+0000","last_clean":"2024-08-30T08:59:46.662698+0000","last_became_active":"2024-08-30T08:58:09.443914+0000","last_became_peered":"2024-08-30T08:58:09.443914+0000","last_unstale":"2024-08-30T08:59:46.662698+0000","last_undegraded":"2024-08-30T08:59:46.662698+0000","last_fullsized":"2024-08-30T08:59:46.662698+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T14:17:43.866352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"73'2","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T08:59:46.669336+0000","last_change":"2024-08-30T08:59:46.668686+0000","last_active":"2024-08-30T08:59:46.669336+0000","last_peered":"2024-08-30T08:59:46.669336+0000","last_clean":"2024-08-30T08:59:46.669336+0000","last_became_active":"2024-08-30T08:59:46.665549+0000","last_became_peered":"2024-08-30T08:59:46.665549+0000","last_unstale":"2024-08-30T08:59:46.669336+0000","last_undegraded":"2024-08-30T08:59:46.669336+0000","last_fullsized":"2024-08-30T08:59:46.669336+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T12:25:56.088867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"83'15","reported_seq":49,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:15.315195+0000","last_change":"2024-08-30T08:58:02.405550+0000","last_active":"2024-08-30T09:00:15.315195+0000","last_peered":"2024-08-30T09:00:15.315195+0000","last_clean":"2024-08-30T09:00:15.315195+0000","last_became_active":"2024-08-30T08:58:02.405129+0000","last_became_peered":"2024-08-30T08:58:02.405129+0000","last_unstale":"2024-08-30T09:00:15.315195+0000","last_undegraded":"2024-08-30T09:00:15.315195+0000","last_fullsized":"2024-08-30T09:00:15.315195+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T11:36:47.624026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T08:59:46.661666+0000","last_change":"2024-08-30T08:58:09.445746+0000","last_active":"2024-08-30T08:59:46.661666+0000","last_peered":"2024-08-30T08:59:46.661666+0000","last_clean":"2024-08-30T08:59:46.661666+0000","last_became_active":"2024-08-30T08:58:09.445257+0000","last_became_peered":"2024-08-30T08:58:09.445257+0000","last_unstale":"2024-08-30T08:59:46.661666+0000","last_undegraded":"2024-08-30T08:59:46.661666+0000","last_fullsized":"2024-08-30T08:59:46.661666+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T18:02:52.007131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"83'5","reported_seq":40,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:15.335272+0000","last_change":"2024-08-30T08:58:02.408213+0000","last_active":"2024-08-30T09:00:15.335272+0000","last_peered":"2024-08-30T09:00:15.335272+0000","last_clean":"2024-08-30T09:00:15.335272+0000","last_became_active":"2024-08-30T08:58:02.407856+0000","last_became_peered":"2024-08-30T08:58:02.407856+0000","last_unstale":"2024-08-30T09:00:15.335272+0000","last_undegraded":"2024-08-30T09:00:15.335272+0000","last_fullsized":"2024-08-30T09:00:15.335272+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T12:00:22.226653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"79'4","reported_seq":45,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T08:59:46.659742+0000","last_change":"2024-08-30T08:58:00.391714+0000","last_active":"2024-08-30T08:59:46.659742+0000","last_peered":"2024-08-30T08:59:46.659742+0000","last_clean":"2024-08-30T08:59:46.659742+0000","last_became_active":"2024-08-30T08:58:00.391211+0000","last_became_peered":"2024-08-30T08:58:00.391211+0000","last_unstale":"2024-08-30T08:59:46.659742+0000","last_undegraded":"2024-08-30T08:59:46.659742+0000","last_fullsized":"2024-08-30T08:59:46.659742+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T11:55:37.469983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T08:59:46.662058+0000","last_change":"2024-08-30T08:58:09.728177+0000","last_active":"2024-08-30T08:59:46.662058+0000","last_peered":"2024-08-30T08:59:46.662058+0000","last_clean":"2024-08-30T08:59:46.662058+0000","last_became_active":"2024-08-30T08:58:09.727630+0000","last_became_peered":"2024-08-30T08:58:09.727630+0000","last_unstale":"2024-08-30T08:59:46.662058+0000","last_undegraded":"2024-08-30T08:59:46.662058+0000","last_fullsized":"2024-08-30T08:59:46.662058+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T12:07:52.965014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"83'8","reported_seq":43,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:15.319441+0000","last_change":"2024-08-30T08:58:02.403378+0000","last_active":"2024-08-30T09:00:15.319441+0000","last_peered":"2024-08-30T09:00:15.319441+0000","last_clean":"2024-08-30T09:00:15.319441+0000","last_became_active":"2024-08-30T08:58:02.403036+0000","last_became_peered":"2024-08-30T08:58:02.403036+0000","last_unstale":"2024-08-30T09:00:15.319441+0000","last_undegraded":"2024-08-30T09:00:15.319441+0000","last_fullsized":"2024-08-30T09:00:15.319441+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T16:37:14.876384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"80'1","reported_seq":41,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T08:59:46.657210+0000","last_change":"2024-08-30T08:58:00.393860+0000","last_active":"2024-08-30T08:59:46.657210+0000","last_peered":"2024-08-30T08:59:46.657210+0000","last_clean":"2024-08-30T08:59:46.657210+0000","last_became_active":"2024-08-30T08:58:00.392486+0000","last_became_peered":"2024-08-30T08:58:00.392486+0000","last_unstale":"2024-08-30T08:59:46.657210+0000","last_undegraded":"2024-08-30T08:59:46.657210+0000","last_fullsized":"2024-08-30T08:59:46.657210+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T12:50:27.337817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"81'1","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T08:59:46.662322+0000","last_change":"2024-08-30T08:58:09.441476+0000","last_active":"2024-08-30T08:59:46.662322+0000","last_peered":"2024-08-30T08:59:46.662322+0000","last_clean":"2024-08-30T08:59:46.662322+0000","last_became_active":"2024-08-30T08:58:09.440344+0000","last_became_peered":"2024-08-30T08:58:09.440344+0000","last_unstale":"2024-08-30T08:59:46.662322+0000","last_undegraded":"2024-08-30T08:59:46.662322+0000","last_fullsized":"2024-08-30T08:59:46.662322+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T12:06:47.113611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"83'9","reported_seq":44,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:15.314692+0000","last_change":"2024-08-30T08:58:02.407098+0000","last_active":"2024-08-30T09:00:15.314692+0000","last_peered":"2024-08-30T09:00:15.314692+0000","last_clean":"2024-08-30T09:00:15.314692+0000","last_became_active":"2024-08-30T08:58:02.406690+0000","last_became_peered":"2024-08-30T08:58:02.406690+0000","last_unstale":"2024-08-30T09:00:15.314692+0000","last_undegraded":"2024-08-30T09:00:15.314692+0000","last_fullsized":"2024-08-30T09:00:15.314692+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T18:27:39.534923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"79'3","reported_seq":43,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T08:59:46.701567+0000","last_change":"2024-08-30T08:58:00.386434+0000","last_active":"2024-08-30T08:59:46.701567+0000","last_peered":"2024-08-30T08:59:46.701567+0000","last_clean":"2024-08-30T08:59:46.701567+0000","last_became_active":"2024-08-30T08:58:00.385930+0000","last_became_peered":"2024-08-30T08:58:00.385930+0000","last_unstale":"2024-08-30T08:59:46.701567+0000","last_undegraded":"2024-08-30T08:59:46.701567+0000","last_fullsized":"2024-08-30T08:59:46.701567+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T10:39:22.441568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T08:59:46.659680+0000","last_change":"2024-08-30T08:58:46.775674+0000","last_active":"2024-08-30T08:59:46.659680+0000","last_peered":"2024-08-30T08:59:46.659680+0000","last_clean":"2024-08-30T08:59:46.659680+0000","last_became_active":"2024-08-30T08:58:46.775157+0000","last_became_peered":"2024-08-30T08:58:46.775157+0000","last_unstale":"2024-08-30T08:59:46.659680+0000","last_undegraded":"2024-08-30T08:59:46.659680+0000","last_fullsized":"2024-08-30T08:59:46.659680+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T17:13:47.066475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,2,5],"acting":[8,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.9","version":"83'2","reported_seq":37,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:15.283463+0000","last_change":"2024-08-30T08:58:02.410500+0000","last_active":"2024-08-30T09:00:15.283463+0000","last_peered":"2024-08-30T09:00:15.283463+0000","last_clean":"2024-08-30T09:00:15.283463+0000","last_became_active":"2024-08-30T08:58:02.409260+0000","last_became_peered":"2024-08-30T08:58:02.409260+0000","last_unstale":"2024-08-30T09:00:15.283463+0000","last_undegraded":"2024-08-30T09:00:15.283463+0000","last_fullsized":"2024-08-30T09:00:15.283463+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T16:47:59.701956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"79'2","reported_seq":42,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T08:59:46.663767+0000","last_change":"2024-08-30T08:58:00.482296+0000","last_active":"2024-08-30T08:59:46.663767+0000","last_peered":"2024-08-30T08:59:46.663767+0000","last_clean":"2024-08-30T08:59:46.663767+0000","last_became_active":"2024-08-30T08:58:00.481920+0000","last_became_peered":"2024-08-30T08:58:00.481920+0000","last_unstale":"2024-08-30T08:59:46.663767+0000","last_undegraded":"2024-08-30T08:59:46.663767+0000","last_fullsized":"2024-08-30T08:59:46.663767+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T09:29:30.724410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T08:59:46.662353+0000","last_change":"2024-08-30T08:58:09.727879+0000","last_active":"2024-08-30T08:59:46.662353+0000","last_peered":"2024-08-30T08:59:46.662353+0000","last_clean":"2024-08-30T08:59:46.662353+0000","last_became_active":"2024-08-30T08:58:09.727339+0000","last_became_peered":"2024-08-30T08:58:09.727339+0000","last_unstale":"2024-08-30T08:59:46.662353+0000","last_undegraded":"2024-08-30T08:59:46.662353+0000","last_fullsized":"2024-08-30T08:59:46.662353+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T13:19:18.582668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"83'7","reported_seq":49,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:15.332760+0000","last_change":"2024-08-30T08:58:46.559988+0000","last_active":"2024-08-30T09:00:15.332760+0000","last_peered":"2024-08-30T09:00:15.332760+0000","last_clean":"2024-08-30T09:00:15.332760+0000","last_became_active":"2024-08-30T08:58:46.559442+0000","last_became_peered":"2024-08-30T08:58:46.559442+0000","last_unstale":"2024-08-30T09:00:15.332760+0000","last_undegraded":"2024-08-30T09:00:15.332760+0000","last_fullsized":"2024-08-30T09:00:15.332760+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T09:29:09.747756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"79'4","reported_seq":44,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T08:59:46.662490+0000","last_change":"2024-08-30T08:58:00.380338+0000","last_active":"2024-08-30T08:59:46.662490+0000","last_peered":"2024-08-30T08:59:46.662490+0000","last_clean":"2024-08-30T08:59:46.662490+0000","last_became_active":"2024-08-30T08:58:00.379860+0000","last_became_peered":"2024-08-30T08:58:00.379860+0000","last_unstale":"2024-08-30T08:59:46.662490+0000","last_undegraded":"2024-08-30T08:59:46.662490+0000","last_fullsized":"2024-08-30T08:59:46.662490+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T19:57:15.349906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T08:59:46.657178+0000","last_change":"2024-08-30T08:58:09.443631+0000","last_active":"2024-08-30T08:59:46.657178+0000","last_peered":"2024-08-30T08:59:46.657178+0000","last_clean":"2024-08-30T08:59:46.657178+0000","last_became_active":"2024-08-30T08:58:09.443103+0000","last_became_peered":"2024-08-30T08:58:09.443103+0000","last_unstale":"2024-08-30T08:59:46.657178+0000","last_undegraded":"2024-08-30T08:59:46.657178+0000","last_fullsized":"2024-08-30T08:59:46.657178+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T19:58:11.031826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"83'5","reported_seq":40,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:15.328087+0000","last_change":"2024-08-30T08:58:02.406216+0000","last_active":"2024-08-30T09:00:15.328087+0000","last_peered":"2024-08-30T09:00:15.328087+0000","last_clean":"2024-08-30T09:00:15.328087+0000","last_became_active":"2024-08-30T08:58:02.405040+0000","last_became_peered":"2024-08-30T08:58:02.405040+0000","last_unstale":"2024-08-30T09:00:15.328087+0000","last_undegraded":"2024-08-30T09:00:15.328087+0000","last_fullsized":"2024-08-30T09:00:15.328087+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T14:28:28.721386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"83'14","reported_seq":54,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:15.423472+0000","last_change":"2024-08-30T08:58:00.387350+0000","last_active":"2024-08-30T09:00:15.423472+0000","last_peered":"2024-08-30T09:00:15.423472+0000","last_clean":"2024-08-30T09:00:15.423472+0000","last_became_active":"2024-08-30T08:58:00.387004+0000","last_became_peered":"2024-08-30T08:58:00.387004+0000","last_unstale":"2024-08-30T09:00:15.423472+0000","last_undegraded":"2024-08-30T09:00:15.423472+0000","last_fullsized":"2024-08-30T09:00:15.423472+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T19:24:25.604544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T08:59:46.700944+0000","last_change":"2024-08-30T08:58:09.727388+0000","last_active":"2024-08-30T08:59:46.700944+0000","last_peered":"2024-08-30T08:59:46.700944+0000","last_clean":"2024-08-30T08:59:46.700944+0000","last_became_active":"2024-08-30T08:58:09.726935+0000","last_became_peered":"2024-08-30T08:58:09.726935+0000","last_unstale":"2024-08-30T08:59:46.700944+0000","last_undegraded":"2024-08-30T08:59:46.700944+0000","last_fullsized":"2024-08-30T08:59:46.700944+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T17:53:16.066602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"83'9","reported_seq":44,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:15.307241+0000","last_change":"2024-08-30T08:58:02.402661+0000","last_active":"2024-08-30T09:00:15.307241+0000","last_peered":"2024-08-30T09:00:15.307241+0000","last_clean":"2024-08-30T09:00:15.307241+0000","last_became_active":"2024-08-30T08:58:02.402314+0000","last_became_peered":"2024-08-30T08:58:02.402314+0000","last_unstale":"2024-08-30T09:00:15.307241+0000","last_undegraded":"2024-08-30T09:00:15.307241+0000","last_fullsized":"2024-08-30T09:00:15.307241+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T10:56:41.336312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"79'1","reported_seq":42,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T08:59:46.660369+0000","last_change":"2024-08-30T08:58:00.482720+0000","last_active":"2024-08-30T08:59:46.660369+0000","last_peered":"2024-08-30T08:59:46.660369+0000","last_clean":"2024-08-30T08:59:46.660369+0000","last_became_active":"2024-08-30T08:58:00.482193+0000","last_became_peered":"2024-08-30T08:58:00.482193+0000","last_unstale":"2024-08-30T08:59:46.660369+0000","last_undegraded":"2024-08-30T08:59:46.660369+0000","last_fullsized":"2024-08-30T08:59:46.660369+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T18:42:06.654806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T08:59:46.660980+0000","last_change":"2024-08-30T08:58:09.449970+0000","last_active":"2024-08-30T08:59:46.660980+0000","last_peered":"2024-08-30T08:59:46.660980+0000","last_clean":"2024-08-30T08:59:46.660980+0000","last_became_active":"2024-08-30T08:58:09.447891+0000","last_became_peered":"2024-08-30T08:58:09.447891+0000","last_unstale":"2024-08-30T08:59:46.660980+0000","last_undegraded":"2024-08-30T08:59:46.660980+0000","last_fullsized":"2024-08-30T08:59:46.660980+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T15:06:45.414470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"83'4","reported_seq":39,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:15.322836+0000","last_change":"2024-08-30T08:58:02.409756+0000","last_active":"2024-08-30T09:00:15.322836+0000","last_peered":"2024-08-30T09:00:15.322836+0000","last_clean":"2024-08-30T09:00:15.322836+0000","last_became_active":"2024-08-30T08:58:02.409357+0000","last_became_peered":"2024-08-30T08:58:02.409357+0000","last_unstale":"2024-08-30T09:00:15.322836+0000","last_undegraded":"2024-08-30T09:00:15.322836+0000","last_fullsized":"2024-08-30T09:00:15.322836+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T13:27:41.696527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":40,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T08:59:46.656323+0000","last_change":"2024-08-30T08:58:00.482342+0000","last_active":"2024-08-30T08:59:46.656323+0000","last_peered":"2024-08-30T08:59:46.656323+0000","last_clean":"2024-08-30T08:59:46.656323+0000","last_became_active":"2024-08-30T08:58:00.481902+0000","last_became_peered":"2024-08-30T08:58:00.481902+0000","last_unstale":"2024-08-30T08:59:46.656323+0000","last_undegraded":"2024-08-30T08:59:46.656323+0000","last_fullsized":"2024-08-30T08:59:46.656323+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T14:12:36.789356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T08:59:46.659485+0000","last_change":"2024-08-30T08:58:09.728008+0000","last_active":"2024-08-30T08:59:46.659485+0000","last_peered":"2024-08-30T08:59:46.659485+0000","last_clean":"2024-08-30T08:59:46.659485+0000","last_became_active":"2024-08-30T08:58:09.727513+0000","last_became_peered":"2024-08-30T08:58:09.727513+0000","last_unstale":"2024-08-30T08:59:46.659485+0000","last_undegraded":"2024-08-30T08:59:46.659485+0000","last_fullsized":"2024-08-30T08:59:46.659485+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T14:34:04.165300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"83'7","reported_seq":42,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:15.310825+0000","last_change":"2024-08-30T08:58:02.409383+0000","last_active":"2024-08-30T09:00:15.310825+0000","last_peered":"2024-08-30T09:00:15.310825+0000","last_clean":"2024-08-30T09:00:15.310825+0000","last_became_active":"2024-08-30T08:58:02.409027+0000","last_became_peered":"2024-08-30T08:58:02.409027+0000","last_unstale":"2024-08-30T09:00:15.310825+0000","last_undegraded":"2024-08-30T09:00:15.310825+0000","last_fullsized":"2024-08-30T09:00:15.310825+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T10:44:23.467130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"80'1","reported_seq":41,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T08:59:46.663584+0000","last_change":"2024-08-30T08:58:00.389083+0000","last_active":"2024-08-30T08:59:46.663584+0000","last_peered":"2024-08-30T08:59:46.663584+0000","last_clean":"2024-08-30T08:59:46.663584+0000","last_became_active":"2024-08-30T08:58:00.387552+0000","last_became_peered":"2024-08-30T08:58:00.387552+0000","last_unstale":"2024-08-30T08:59:46.663584+0000","last_undegraded":"2024-08-30T08:59:46.663584+0000","last_fullsized":"2024-08-30T08:59:46.663584+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T11:56:38.827406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T08:59:46.701544+0000","last_change":"2024-08-30T08:58:09.447758+0000","last_active":"2024-08-30T08:59:46.701544+0000","last_peered":"2024-08-30T08:59:46.701544+0000","last_clean":"2024-08-30T08:59:46.701544+0000","last_became_active":"2024-08-30T08:58:09.447333+0000","last_became_peered":"2024-08-30T08:58:09.447333+0000","last_unstale":"2024-08-30T08:59:46.701544+0000","last_undegraded":"2024-08-30T08:59:46.701544+0000","last_fullsized":"2024-08-30T08:59:46.701544+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T12:43:35.289967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"83'8","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:15.309024+0000","last_change":"2024-08-30T08:58:47.091191+0000","last_active":"2024-08-30T09:00:15.309024+0000","last_peered":"2024-08-30T09:00:15.309024+0000","last_clean":"2024-08-30T09:00:15.309024+0000","last_became_active":"2024-08-30T08:58:47.090826+0000","last_became_peered":"2024-08-30T08:58:47.090826+0000","last_unstale":"2024-08-30T09:00:15.309024+0000","last_undegraded":"2024-08-30T09:00:15.309024+0000","last_fullsized":"2024-08-30T09:00:15.309024+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T17:47:22.862514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"83'5","reported_seq":142,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:25.765746+0000","last_change":"2024-08-30T08:58:00.480651+0000","last_active":"2024-08-30T09:00:25.765746+0000","last_peered":"2024-08-30T09:00:25.765746+0000","last_clean":"2024-08-30T09:00:25.765746+0000","last_became_active":"2024-08-30T08:58:00.480205+0000","last_became_peered":"2024-08-30T08:58:00.480205+0000","last_unstale":"2024-08-30T09:00:25.765746+0000","last_undegraded":"2024-08-30T09:00:25.765746+0000","last_fullsized":"2024-08-30T09:00:25.765746+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T16:28:30.687076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":97,"num_read_kb":97,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T08:59:46.701596+0000","last_change":"2024-08-30T08:58:09.727530+0000","last_active":"2024-08-30T08:59:46.701596+0000","last_peered":"2024-08-30T08:59:46.701596+0000","last_clean":"2024-08-30T08:59:46.701596+0000","last_became_active":"2024-08-30T08:58:09.726965+0000","last_became_peered":"2024-08-30T08:58:09.726965+0000","last_unstale":"2024-08-30T08:59:46.701596+0000","last_undegraded":"2024-08-30T08:59:46.701596+0000","last_fullsized":"2024-08-30T08:59:46.701596+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T12:56:42.885615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"83'5","reported_seq":40,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:15.312193+0000","last_change":"2024-08-30T08:58:02.401673+0000","last_active":"2024-08-30T09:00:15.312193+0000","last_peered":"2024-08-30T09:00:15.312193+0000","last_clean":"2024-08-30T09:00:15.312193+0000","last_became_active":"2024-08-30T08:58:02.398876+0000","last_became_peered":"2024-08-30T08:58:02.398876+0000","last_unstale":"2024-08-30T09:00:15.312193+0000","last_undegraded":"2024-08-30T09:00:15.312193+0000","last_fullsized":"2024-08-30T09:00:15.312193+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T16:13:20.893737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"80'3","reported_seq":43,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T08:59:46.663087+0000","last_change":"2024-08-30T08:58:00.391722+0000","last_active":"2024-08-30T08:59:46.663087+0000","last_peered":"2024-08-30T08:59:46.663087+0000","last_clean":"2024-08-30T08:59:46.663087+0000","last_became_active":"2024-08-30T08:58:00.391325+0000","last_became_peered":"2024-08-30T08:58:00.391325+0000","last_unstale":"2024-08-30T08:59:46.663087+0000","last_undegraded":"2024-08-30T08:59:46.663087+0000","last_fullsized":"2024-08-30T08:59:46.663087+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T11:58:08.189202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T08:59:46.663818+0000","last_change":"2024-08-30T08:58:09.728324+0000","last_active":"2024-08-30T08:59:46.663818+0000","last_peered":"2024-08-30T08:59:46.663818+0000","last_clean":"2024-08-30T08:59:46.663818+0000","last_became_active":"2024-08-30T08:58:09.727520+0000","last_became_peered":"2024-08-30T08:58:09.727520+0000","last_unstale":"2024-08-30T08:59:46.663818+0000","last_undegraded":"2024-08-30T08:59:46.663818+0000","last_fullsized":"2024-08-30T08:59:46.663818+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T10:09:39.046820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"83'8","reported_seq":43,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:15.325800+0000","last_change":"2024-08-30T08:58:02.410826+0000","last_active":"2024-08-30T09:00:15.325800+0000","last_peered":"2024-08-30T09:00:15.325800+0000","last_clean":"2024-08-30T09:00:15.325800+0000","last_became_active":"2024-08-30T08:58:02.410464+0000","last_became_peered":"2024-08-30T08:58:02.410464+0000","last_unstale":"2024-08-30T09:00:15.325800+0000","last_undegraded":"2024-08-30T09:00:15.325800+0000","last_fullsized":"2024-08-30T09:00:15.325800+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T12:14:59.666614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"80'3","reported_seq":45,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T08:59:46.663066+0000","last_change":"2024-08-30T08:58:00.385464+0000","last_active":"2024-08-30T08:59:46.663066+0000","last_peered":"2024-08-30T08:59:46.663066+0000","last_clean":"2024-08-30T08:59:46.663066+0000","last_became_active":"2024-08-30T08:58:00.385083+0000","last_became_peered":"2024-08-30T08:58:00.385083+0000","last_unstale":"2024-08-30T08:59:46.663066+0000","last_undegraded":"2024-08-30T08:59:46.663066+0000","last_fullsized":"2024-08-30T08:59:46.663066+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T16:13:52.304755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":112,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T08:59:46.663146+0000","last_change":"2024-08-30T08:56:49.645080+0000","last_active":"2024-08-30T08:59:46.663146+0000","last_peered":"2024-08-30T08:59:46.663146+0000","last_clean":"2024-08-30T08:59:46.663146+0000","last_became_active":"2024-08-30T08:56:49.639066+0000","last_became_peered":"2024-08-30T08:56:49.639066+0000","last_unstale":"2024-08-30T08:59:46.663146+0000","last_undegraded":"2024-08-30T08:59:46.663146+0000","last_fullsized":"2024-08-30T08:59:46.663146+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:53:27.566846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:53:27.566846+0000","last_clean_scrub_stamp":"2024-08-30T08:53:27.566846+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T20:23:45.723824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"81'1","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T08:59:46.663607+0000","last_change":"2024-08-30T08:58:09.444007+0000","last_active":"2024-08-30T08:59:46.663607+0000","last_peered":"2024-08-30T08:59:46.663607+0000","last_clean":"2024-08-30T08:59:46.663607+0000","last_became_active":"2024-08-30T08:58:09.443580+0000","last_became_peered":"2024-08-30T08:58:09.443580+0000","last_unstale":"2024-08-30T08:59:46.663607+0000","last_undegraded":"2024-08-30T08:59:46.663607+0000","last_fullsized":"2024-08-30T08:59:46.663607+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T17:00:58.654026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"83'7","reported_seq":42,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:15.335454+0000","last_change":"2024-08-30T08:58:02.406973+0000","last_active":"2024-08-30T09:00:15.335454+0000","last_peered":"2024-08-30T09:00:15.335454+0000","last_clean":"2024-08-30T09:00:15.335454+0000","last_became_active":"2024-08-30T08:58:02.406576+0000","last_became_peered":"2024-08-30T08:58:02.406576+0000","last_unstale":"2024-08-30T09:00:15.335454+0000","last_undegraded":"2024-08-30T09:00:15.335454+0000","last_fullsized":"2024-08-30T09:00:15.335454+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T16:04:48.785310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"80'3","reported_seq":44,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T08:59:46.657731+0000","last_change":"2024-08-30T08:58:00.481019+0000","last_active":"2024-08-30T08:59:46.657731+0000","last_peered":"2024-08-30T08:59:46.657731+0000","last_clean":"2024-08-30T08:59:46.657731+0000","last_became_active":"2024-08-30T08:58:00.480590+0000","last_became_peered":"2024-08-30T08:58:00.480590+0000","last_unstale":"2024-08-30T08:59:46.657731+0000","last_undegraded":"2024-08-30T08:59:46.657731+0000","last_fullsized":"2024-08-30T08:59:46.657731+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T13:20:46.731591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T08:59:46.664023+0000","last_change":"2024-08-30T08:58:09.445723+0000","last_active":"2024-08-30T08:59:46.664023+0000","last_peered":"2024-08-30T08:59:46.664023+0000","last_clean":"2024-08-30T08:59:46.664023+0000","last_became_active":"2024-08-30T08:58:09.445252+0000","last_became_peered":"2024-08-30T08:58:09.445252+0000","last_unstale":"2024-08-30T08:59:46.664023+0000","last_undegraded":"2024-08-30T08:59:46.664023+0000","last_fullsized":"2024-08-30T08:59:46.664023+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T12:16:12.255990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"83'6","reported_seq":41,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:15.331469+0000","last_change":"2024-08-30T08:58:02.414250+0000","last_active":"2024-08-30T09:00:15.331469+0000","last_peered":"2024-08-30T09:00:15.331469+0000","last_clean":"2024-08-30T09:00:15.331469+0000","last_became_active":"2024-08-30T08:58:02.413684+0000","last_became_peered":"2024-08-30T08:58:02.413684+0000","last_unstale":"2024-08-30T09:00:15.331469+0000","last_undegraded":"2024-08-30T09:00:15.331469+0000","last_fullsized":"2024-08-30T09:00:15.331469+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T16:55:12.878806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"73'1","reported_seq":42,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T08:59:46.664107+0000","last_change":"2024-08-30T08:58:00.388869+0000","last_active":"2024-08-30T08:59:46.664107+0000","last_peered":"2024-08-30T08:59:46.664107+0000","last_clean":"2024-08-30T08:59:46.664107+0000","last_became_active":"2024-08-30T08:58:00.388116+0000","last_became_peered":"2024-08-30T08:58:00.388116+0000","last_unstale":"2024-08-30T08:59:46.664107+0000","last_undegraded":"2024-08-30T08:59:46.664107+0000","last_fullsized":"2024-08-30T08:59:46.664107+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T09:50:48.449205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T08:59:46.842285+0000","last_change":"2024-08-30T08:58:47.092004+0000","last_active":"2024-08-30T08:59:46.842285+0000","last_peered":"2024-08-30T08:59:46.842285+0000","last_clean":"2024-08-30T08:59:46.842285+0000","last_became_active":"2024-08-30T08:58:47.091661+0000","last_became_peered":"2024-08-30T08:58:47.091661+0000","last_unstale":"2024-08-30T08:59:46.842285+0000","last_undegraded":"2024-08-30T08:59:46.842285+0000","last_fullsized":"2024-08-30T08:59:46.842285+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T09:14:55.353465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,1,4],"acting":[0,8,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"83'9","reported_seq":44,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:15.331298+0000","last_change":"2024-08-30T08:58:02.410891+0000","last_active":"2024-08-30T09:00:15.331298+0000","last_peered":"2024-08-30T09:00:15.331298+0000","last_clean":"2024-08-30T09:00:15.331298+0000","last_became_active":"2024-08-30T08:58:02.410522+0000","last_became_peered":"2024-08-30T08:58:02.410522+0000","last_unstale":"2024-08-30T09:00:15.331298+0000","last_undegraded":"2024-08-30T09:00:15.331298+0000","last_fullsized":"2024-08-30T09:00:15.331298+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T09:32:21.540298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"73'2","reported_seq":53,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T08:59:46.702026+0000","last_change":"2024-08-30T08:58:47.095801+0000","last_active":"2024-08-30T08:59:46.702026+0000","last_peered":"2024-08-30T08:59:46.702026+0000","last_clean":"2024-08-30T08:59:46.702026+0000","last_became_active":"2024-08-30T08:58:47.092756+0000","last_became_peered":"2024-08-30T08:58:47.092756+0000","last_unstale":"2024-08-30T08:59:46.702026+0000","last_undegraded":"2024-08-30T08:59:46.702026+0000","last_fullsized":"2024-08-30T08:59:46.702026+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T19:10:02.064649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T08:59:46.663000+0000","last_change":"2024-08-30T08:58:09.449210+0000","last_active":"2024-08-30T08:59:46.663000+0000","last_peered":"2024-08-30T08:59:46.663000+0000","last_clean":"2024-08-30T08:59:46.663000+0000","last_became_active":"2024-08-30T08:58:09.448769+0000","last_became_peered":"2024-08-30T08:58:09.448769+0000","last_unstale":"2024-08-30T08:59:46.663000+0000","last_undegraded":"2024-08-30T08:59:46.663000+0000","last_fullsized":"2024-08-30T08:59:46.663000+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T09:26:13.845520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"83'5","reported_seq":40,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:15.316813+0000","last_change":"2024-08-30T08:58:02.414152+0000","last_active":"2024-08-30T09:00:15.316813+0000","last_peered":"2024-08-30T09:00:15.316813+0000","last_clean":"2024-08-30T09:00:15.316813+0000","last_became_active":"2024-08-30T08:58:02.413768+0000","last_became_peered":"2024-08-30T08:58:02.413768+0000","last_unstale":"2024-08-30T09:00:15.316813+0000","last_undegraded":"2024-08-30T09:00:15.316813+0000","last_fullsized":"2024-08-30T09:00:15.316813+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T18:34:09.447071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"81'2","reported_seq":43,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T08:59:46.660350+0000","last_change":"2024-08-30T08:58:00.390839+0000","last_active":"2024-08-30T08:59:46.660350+0000","last_peered":"2024-08-30T08:59:46.660350+0000","last_clean":"2024-08-30T08:59:46.660350+0000","last_became_active":"2024-08-30T08:58:00.390422+0000","last_became_peered":"2024-08-30T08:58:00.390422+0000","last_unstale":"2024-08-30T08:59:46.660350+0000","last_undegraded":"2024-08-30T08:59:46.660350+0000","last_fullsized":"2024-08-30T08:59:46.660350+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T15:05:23.005075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T08:59:46.659347+0000","last_change":"2024-08-30T08:58:46.824563+0000","last_active":"2024-08-30T08:59:46.659347+0000","last_peered":"2024-08-30T08:59:46.659347+0000","last_clean":"2024-08-30T08:59:46.659347+0000","last_became_active":"2024-08-30T08:58:46.824163+0000","last_became_peered":"2024-08-30T08:58:46.824163+0000","last_unstale":"2024-08-30T08:59:46.659347+0000","last_undegraded":"2024-08-30T08:59:46.659347+0000","last_fullsized":"2024-08-30T08:59:46.659347+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T19:25:13.202762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,3,1],"acting":[8,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"83'6","reported_seq":41,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:15.315507+0000","last_change":"2024-08-30T08:58:02.405122+0000","last_active":"2024-08-30T09:00:15.315507+0000","last_peered":"2024-08-30T09:00:15.315507+0000","last_clean":"2024-08-30T09:00:15.315507+0000","last_became_active":"2024-08-30T08:58:02.404695+0000","last_became_peered":"2024-08-30T08:58:02.404695+0000","last_unstale":"2024-08-30T09:00:15.315507+0000","last_undegraded":"2024-08-30T09:00:15.315507+0000","last_fullsized":"2024-08-30T09:00:15.315507+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T15:50:01.447032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"81'15","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T08:59:46.660605+0000","last_change":"2024-08-30T08:58:46.573221+0000","last_active":"2024-08-30T08:59:46.660605+0000","last_peered":"2024-08-30T08:59:46.660605+0000","last_clean":"2024-08-30T08:59:46.660605+0000","last_became_active":"2024-08-30T08:58:46.559562+0000","last_became_peered":"2024-08-30T08:58:46.559562+0000","last_unstale":"2024-08-30T08:59:46.660605+0000","last_undegraded":"2024-08-30T08:59:46.660605+0000","last_fullsized":"2024-08-30T08:59:46.660605+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T16:21:23.182676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T08:59:46.701861+0000","last_change":"2024-08-30T08:58:09.446063+0000","last_active":"2024-08-30T08:59:46.701861+0000","last_peered":"2024-08-30T08:59:46.701861+0000","last_clean":"2024-08-30T08:59:46.701861+0000","last_became_active":"2024-08-30T08:58:09.445615+0000","last_became_peered":"2024-08-30T08:58:09.445615+0000","last_unstale":"2024-08-30T08:59:46.701861+0000","last_undegraded":"2024-08-30T08:59:46.701861+0000","last_fullsized":"2024-08-30T08:59:46.701861+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T10:25:10.904892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"83'6","reported_seq":41,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:15.330498+0000","last_change":"2024-08-30T08:58:02.412178+0000","last_active":"2024-08-30T09:00:15.330498+0000","last_peered":"2024-08-30T09:00:15.330498+0000","last_clean":"2024-08-30T09:00:15.330498+0000","last_became_active":"2024-08-30T08:58:02.411814+0000","last_became_peered":"2024-08-30T08:58:02.411814+0000","last_unstale":"2024-08-30T09:00:15.330498+0000","last_undegraded":"2024-08-30T09:00:15.330498+0000","last_fullsized":"2024-08-30T09:00:15.330498+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T10:02:59.070204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"83'5","reported_seq":144,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:24.764937+0000","last_change":"2024-08-30T08:58:00.390231+0000","last_active":"2024-08-30T09:00:24.764937+0000","last_peered":"2024-08-30T09:00:24.764937+0000","last_clean":"2024-08-30T09:00:24.764937+0000","last_became_active":"2024-08-30T08:58:00.389852+0000","last_became_peered":"2024-08-30T08:58:00.389852+0000","last_unstale":"2024-08-30T09:00:24.764937+0000","last_undegraded":"2024-08-30T09:00:24.764937+0000","last_fullsized":"2024-08-30T09:00:24.764937+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T12:09:12.105416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":100,"num_read_kb":4,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T08:59:46.656215+0000","last_change":"2024-08-30T08:58:09.441652+0000","last_active":"2024-08-30T08:59:46.656215+0000","last_peered":"2024-08-30T08:59:46.656215+0000","last_clean":"2024-08-30T08:59:46.656215+0000","last_became_active":"2024-08-30T08:58:09.441264+0000","last_became_peered":"2024-08-30T08:58:09.441264+0000","last_unstale":"2024-08-30T08:59:46.656215+0000","last_undegraded":"2024-08-30T08:59:46.656215+0000","last_fullsized":"2024-08-30T08:59:46.656215+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T09:42:25.636669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"83'15","reported_seq":50,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:15.332517+0000","last_change":"2024-08-30T08:58:02.416347+0000","last_active":"2024-08-30T09:00:15.332517+0000","last_peered":"2024-08-30T09:00:15.332517+0000","last_clean":"2024-08-30T09:00:15.332517+0000","last_became_active":"2024-08-30T08:58:02.415898+0000","last_became_peered":"2024-08-30T08:58:02.415898+0000","last_unstale":"2024-08-30T09:00:15.332517+0000","last_undegraded":"2024-08-30T09:00:15.332517+0000","last_fullsized":"2024-08-30T09:00:15.332517+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T10:18:15.507076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"79'3","reported_seq":43,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T08:59:46.663379+0000","last_change":"2024-08-30T08:58:00.393616+0000","last_active":"2024-08-30T08:59:46.663379+0000","last_peered":"2024-08-30T08:59:46.663379+0000","last_clean":"2024-08-30T08:59:46.663379+0000","last_became_active":"2024-08-30T08:58:00.393218+0000","last_became_peered":"2024-08-30T08:58:00.393218+0000","last_unstale":"2024-08-30T08:59:46.663379+0000","last_undegraded":"2024-08-30T08:59:46.663379+0000","last_fullsized":"2024-08-30T08:59:46.663379+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T10:56:37.175968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T08:59:46.661446+0000","last_change":"2024-08-30T08:58:47.092990+0000","last_active":"2024-08-30T08:59:46.661446+0000","last_peered":"2024-08-30T08:59:46.661446+0000","last_clean":"2024-08-30T08:59:46.661446+0000","last_became_active":"2024-08-30T08:58:47.092448+0000","last_became_peered":"2024-08-30T08:58:47.092448+0000","last_unstale":"2024-08-30T08:59:46.661446+0000","last_undegraded":"2024-08-30T08:59:46.661446+0000","last_fullsized":"2024-08-30T08:59:46.661446+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T20:23:06.527336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,7,6],"acting":[11,10,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"83'9","reported_seq":44,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:15.317768+0000","last_change":"2024-08-30T08:58:02.405783+0000","last_active":"2024-08-30T09:00:15.317768+0000","last_peered":"2024-08-30T09:00:15.317768+0000","last_clean":"2024-08-30T09:00:15.317768+0000","last_became_active":"2024-08-30T08:58:02.405430+0000","last_became_peered":"2024-08-30T08:58:02.405430+0000","last_unstale":"2024-08-30T09:00:15.317768+0000","last_undegraded":"2024-08-30T09:00:15.317768+0000","last_fullsized":"2024-08-30T09:00:15.317768+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T19:40:08.377259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"80'6","reported_seq":47,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T08:59:46.657435+0000","last_change":"2024-08-30T08:58:00.389617+0000","last_active":"2024-08-30T08:59:46.657435+0000","last_peered":"2024-08-30T08:59:46.657435+0000","last_clean":"2024-08-30T08:59:46.657435+0000","last_became_active":"2024-08-30T08:58:00.389031+0000","last_became_peered":"2024-08-30T08:58:00.389031+0000","last_unstale":"2024-08-30T08:59:46.657435+0000","last_undegraded":"2024-08-30T08:59:46.657435+0000","last_fullsized":"2024-08-30T08:59:46.657435+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T15:42:38.050768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T08:59:46.663773+0000","last_change":"2024-08-30T08:58:09.451925+0000","last_active":"2024-08-30T08:59:46.663773+0000","last_peered":"2024-08-30T08:59:46.663773+0000","last_clean":"2024-08-30T08:59:46.663773+0000","last_became_active":"2024-08-30T08:58:09.446893+0000","last_became_peered":"2024-08-30T08:58:09.446893+0000","last_unstale":"2024-08-30T08:59:46.663773+0000","last_undegraded":"2024-08-30T08:59:46.663773+0000","last_fullsized":"2024-08-30T08:59:46.663773+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T14:34:25.134006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"83'6","reported_seq":41,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:15.315731+0000","last_change":"2024-08-30T08:58:02.411194+0000","last_active":"2024-08-30T09:00:15.315731+0000","last_peered":"2024-08-30T09:00:15.315731+0000","last_clean":"2024-08-30T09:00:15.315731+0000","last_became_active":"2024-08-30T08:58:02.410183+0000","last_became_peered":"2024-08-30T08:58:02.410183+0000","last_unstale":"2024-08-30T09:00:15.315731+0000","last_undegraded":"2024-08-30T09:00:15.315731+0000","last_fullsized":"2024-08-30T09:00:15.315731+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T10:56:19.016115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"83'38","reported_seq":283,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:26.768557+0000","last_change":"2024-08-30T08:58:00.389808+0000","last_active":"2024-08-30T09:00:26.768557+0000","last_peered":"2024-08-30T09:00:26.768557+0000","last_clean":"2024-08-30T09:00:26.768557+0000","last_became_active":"2024-08-30T08:58:00.389411+0000","last_became_peered":"2024-08-30T08:58:00.389411+0000","last_unstale":"2024-08-30T09:00:26.768557+0000","last_undegraded":"2024-08-30T09:00:26.768557+0000","last_fullsized":"2024-08-30T09:00:26.768557+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T18:59:45.468456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64617,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":205,"num_read_kb":166,"num_write":39,"num_write_kb":82,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T08:59:46.656029+0000","last_change":"2024-08-30T08:58:09.447360+0000","last_active":"2024-08-30T08:59:46.656029+0000","last_peered":"2024-08-30T08:59:46.656029+0000","last_clean":"2024-08-30T08:59:46.656029+0000","last_became_active":"2024-08-30T08:58:09.446913+0000","last_became_peered":"2024-08-30T08:58:09.446913+0000","last_unstale":"2024-08-30T08:59:46.656029+0000","last_undegraded":"2024-08-30T08:59:46.656029+0000","last_fullsized":"2024-08-30T08:59:46.656029+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T13:59:43.147370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"83'11","reported_seq":46,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:15.334813+0000","last_change":"2024-08-30T08:58:02.408920+0000","last_active":"2024-08-30T09:00:15.334813+0000","last_peered":"2024-08-30T09:00:15.334813+0000","last_clean":"2024-08-30T09:00:15.334813+0000","last_became_active":"2024-08-30T08:58:02.408204+0000","last_became_peered":"2024-08-30T08:58:02.408204+0000","last_unstale":"2024-08-30T09:00:15.334813+0000","last_undegraded":"2024-08-30T09:00:15.334813+0000","last_fullsized":"2024-08-30T09:00:15.334813+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T11:44:00.765672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"79'4","reported_seq":45,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T08:59:46.664024+0000","last_change":"2024-08-30T08:58:00.482348+0000","last_active":"2024-08-30T08:59:46.664024+0000","last_peered":"2024-08-30T08:59:46.664024+0000","last_clean":"2024-08-30T08:59:46.664024+0000","last_became_active":"2024-08-30T08:58:00.481929+0000","last_became_peered":"2024-08-30T08:58:00.481929+0000","last_unstale":"2024-08-30T08:59:46.664024+0000","last_undegraded":"2024-08-30T08:59:46.664024+0000","last_fullsized":"2024-08-30T08:59:46.664024+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T11:13:55.332639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T08:59:46.662774+0000","last_change":"2024-08-30T08:58:09.443259+0000","last_active":"2024-08-30T08:59:46.662774+0000","last_peered":"2024-08-30T08:59:46.662774+0000","last_clean":"2024-08-30T08:59:46.662774+0000","last_became_active":"2024-08-30T08:58:09.442209+0000","last_became_peered":"2024-08-30T08:58:09.442209+0000","last_unstale":"2024-08-30T08:59:46.662774+0000","last_undegraded":"2024-08-30T08:59:46.662774+0000","last_fullsized":"2024-08-30T08:59:46.662774+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T09:25:56.233525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"83'9","reported_seq":43,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:15.333201+0000","last_change":"2024-08-30T08:58:02.407183+0000","last_active":"2024-08-30T09:00:15.333201+0000","last_peered":"2024-08-30T09:00:15.333201+0000","last_clean":"2024-08-30T09:00:15.333201+0000","last_became_active":"2024-08-30T08:58:02.406838+0000","last_became_peered":"2024-08-30T08:58:02.406838+0000","last_unstale":"2024-08-30T09:00:15.333201+0000","last_undegraded":"2024-08-30T09:00:15.333201+0000","last_fullsized":"2024-08-30T09:00:15.333201+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T13:40:32.937626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"79'2","reported_seq":42,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T08:59:46.663284+0000","last_change":"2024-08-30T08:58:00.392322+0000","last_active":"2024-08-30T08:59:46.663284+0000","last_peered":"2024-08-30T08:59:46.663284+0000","last_clean":"2024-08-30T08:59:46.663284+0000","last_became_active":"2024-08-30T08:58:00.391935+0000","last_became_peered":"2024-08-30T08:58:00.391935+0000","last_unstale":"2024-08-30T08:59:46.663284+0000","last_undegraded":"2024-08-30T08:59:46.663284+0000","last_fullsized":"2024-08-30T08:59:46.663284+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T12:20:52.368285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T08:59:46.655884+0000","last_change":"2024-08-30T08:58:09.441824+0000","last_active":"2024-08-30T08:59:46.655884+0000","last_peered":"2024-08-30T08:59:46.655884+0000","last_clean":"2024-08-30T08:59:46.655884+0000","last_became_active":"2024-08-30T08:58:09.441398+0000","last_became_peered":"2024-08-30T08:58:09.441398+0000","last_unstale":"2024-08-30T08:59:46.655884+0000","last_undegraded":"2024-08-30T08:59:46.655884+0000","last_fullsized":"2024-08-30T08:59:46.655884+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T14:05:45.710280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"83'10","reported_seq":45,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:15.334428+0000","last_change":"2024-08-30T08:58:02.403431+0000","last_active":"2024-08-30T09:00:15.334428+0000","last_peered":"2024-08-30T09:00:15.334428+0000","last_clean":"2024-08-30T09:00:15.334428+0000","last_became_active":"2024-08-30T08:58:02.403029+0000","last_became_peered":"2024-08-30T08:58:02.403029+0000","last_unstale":"2024-08-30T09:00:15.334428+0000","last_undegraded":"2024-08-30T09:00:15.334428+0000","last_fullsized":"2024-08-30T09:00:15.334428+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T09:34:56.452675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"83'4","reported_seq":141,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:26.768548+0000","last_change":"2024-08-30T08:58:00.389856+0000","last_active":"2024-08-30T09:00:26.768548+0000","last_peered":"2024-08-30T09:00:26.768548+0000","last_clean":"2024-08-30T09:00:26.768548+0000","last_became_active":"2024-08-30T08:58:00.389459+0000","last_became_peered":"2024-08-30T08:58:00.389459+0000","last_unstale":"2024-08-30T09:00:26.768548+0000","last_undegraded":"2024-08-30T09:00:26.768548+0000","last_fullsized":"2024-08-30T09:00:26.768548+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T18:32:06.214570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":97,"num_read_kb":97,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T08:59:46.701213+0000","last_change":"2024-08-30T08:58:09.445317+0000","last_active":"2024-08-30T08:59:46.701213+0000","last_peered":"2024-08-30T08:59:46.701213+0000","last_clean":"2024-08-30T08:59:46.701213+0000","last_became_active":"2024-08-30T08:58:09.444921+0000","last_became_peered":"2024-08-30T08:58:09.444921+0000","last_unstale":"2024-08-30T08:59:46.701213+0000","last_undegraded":"2024-08-30T08:59:46.701213+0000","last_fullsized":"2024-08-30T08:59:46.701213+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T09:22:13.038437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T08:59:46.842401+0000","last_change":"2024-08-30T08:58:09.447469+0000","last_active":"2024-08-30T08:59:46.842401+0000","last_peered":"2024-08-30T08:59:46.842401+0000","last_clean":"2024-08-30T08:59:46.842401+0000","last_became_active":"2024-08-30T08:58:09.447015+0000","last_became_peered":"2024-08-30T08:58:09.447015+0000","last_unstale":"2024-08-30T08:59:46.842401+0000","last_undegraded":"2024-08-30T08:59:46.842401+0000","last_fullsized":"2024-08-30T08:59:46.842401+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T09:18:36.734380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"83'11","reported_seq":46,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:15.317241+0000","last_change":"2024-08-30T08:58:02.403371+0000","last_active":"2024-08-30T09:00:15.317241+0000","last_peered":"2024-08-30T09:00:15.317241+0000","last_clean":"2024-08-30T09:00:15.317241+0000","last_became_active":"2024-08-30T08:58:02.403012+0000","last_became_peered":"2024-08-30T08:58:02.403012+0000","last_unstale":"2024-08-30T09:00:15.317241+0000","last_undegraded":"2024-08-30T09:00:15.317241+0000","last_fullsized":"2024-08-30T09:00:15.317241+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T10:29:29.683081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"79'1","reported_seq":41,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T08:59:46.663441+0000","last_change":"2024-08-30T08:58:00.394450+0000","last_active":"2024-08-30T08:59:46.663441+0000","last_peered":"2024-08-30T08:59:46.663441+0000","last_clean":"2024-08-30T08:59:46.663441+0000","last_became_active":"2024-08-30T08:58:00.391843+0000","last_became_peered":"2024-08-30T08:58:00.391843+0000","last_unstale":"2024-08-30T08:59:46.663441+0000","last_undegraded":"2024-08-30T08:59:46.663441+0000","last_fullsized":"2024-08-30T08:59:46.663441+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T10:29:18.947898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T08:59:46.661222+0000","last_change":"2024-08-30T08:58:09.449172+0000","last_active":"2024-08-30T08:59:46.661222+0000","last_peered":"2024-08-30T08:59:46.661222+0000","last_clean":"2024-08-30T08:59:46.661222+0000","last_became_active":"2024-08-30T08:58:09.448687+0000","last_became_peered":"2024-08-30T08:58:09.448687+0000","last_unstale":"2024-08-30T08:59:46.661222+0000","last_undegraded":"2024-08-30T08:59:46.661222+0000","last_fullsized":"2024-08-30T08:59:46.661222+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T13:23:16.090251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"83'8","reported_seq":42,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:15.315455+0000","last_change":"2024-08-30T08:58:02.400834+0000","last_active":"2024-08-30T09:00:15.315455+0000","last_peered":"2024-08-30T09:00:15.315455+0000","last_clean":"2024-08-30T09:00:15.315455+0000","last_became_active":"2024-08-30T08:58:02.400477+0000","last_became_peered":"2024-08-30T08:58:02.400477+0000","last_unstale":"2024-08-30T09:00:15.315455+0000","last_undegraded":"2024-08-30T09:00:15.315455+0000","last_fullsized":"2024-08-30T09:00:15.315455+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T13:40:45.239732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"80'2","reported_seq":42,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T08:59:46.701030+0000","last_change":"2024-08-30T08:58:00.381699+0000","last_active":"2024-08-30T08:59:46.701030+0000","last_peered":"2024-08-30T08:59:46.701030+0000","last_clean":"2024-08-30T08:59:46.701030+0000","last_became_active":"2024-08-30T08:58:00.381189+0000","last_became_peered":"2024-08-30T08:58:00.381189+0000","last_unstale":"2024-08-30T08:59:46.701030+0000","last_undegraded":"2024-08-30T08:59:46.701030+0000","last_fullsized":"2024-08-30T08:59:46.701030+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T14:09:16.939062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"80'3","reported_seq":42,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T08:59:46.697436+0000","last_change":"2024-08-30T08:58:00.387430+0000","last_active":"2024-08-30T08:59:46.697436+0000","last_peered":"2024-08-30T08:59:46.697436+0000","last_clean":"2024-08-30T08:59:46.697436+0000","last_became_active":"2024-08-30T08:58:00.387043+0000","last_became_peered":"2024-08-30T08:58:00.387043+0000","last_unstale":"2024-08-30T08:59:46.697436+0000","last_undegraded":"2024-08-30T08:59:46.697436+0000","last_fullsized":"2024-08-30T08:59:46.697436+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T13:32:25.418105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"83'8","reported_seq":42,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:15.316380+0000","last_change":"2024-08-30T08:58:02.404798+0000","last_active":"2024-08-30T09:00:15.316380+0000","last_peered":"2024-08-30T09:00:15.316380+0000","last_clean":"2024-08-30T09:00:15.316380+0000","last_became_active":"2024-08-30T08:58:02.404435+0000","last_became_peered":"2024-08-30T08:58:02.404435+0000","last_unstale":"2024-08-30T09:00:15.316380+0000","last_undegraded":"2024-08-30T09:00:15.316380+0000","last_fullsized":"2024-08-30T09:00:15.316380+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T20:31:49.472832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T08:59:46.657368+0000","last_change":"2024-08-30T08:58:09.448297+0000","last_active":"2024-08-30T08:59:46.657368+0000","last_peered":"2024-08-30T08:59:46.657368+0000","last_clean":"2024-08-30T08:59:46.657368+0000","last_became_active":"2024-08-30T08:58:09.447444+0000","last_became_peered":"2024-08-30T08:58:09.447444+0000","last_unstale":"2024-08-30T08:59:46.657368+0000","last_undegraded":"2024-08-30T08:59:46.657368+0000","last_fullsized":"2024-08-30T08:59:46.657368+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T11:40:03.507824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"83'7","reported_seq":146,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:25.767458+0000","last_change":"2024-08-30T08:58:00.392418+0000","last_active":"2024-08-30T09:00:25.767458+0000","last_peered":"2024-08-30T09:00:25.767458+0000","last_clean":"2024-08-30T09:00:25.767458+0000","last_became_active":"2024-08-30T08:58:00.391985+0000","last_became_peered":"2024-08-30T08:58:00.391985+0000","last_unstale":"2024-08-30T09:00:25.767458+0000","last_undegraded":"2024-08-30T09:00:25.767458+0000","last_fullsized":"2024-08-30T09:00:25.767458+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:57:59.362691+0000","last_clean_scrub_stamp":"2024-08-30T08:57:59.362691+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T17:42:18.147754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":100,"num_read_kb":3,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"83'7","reported_seq":41,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:15.300043+0000","last_change":"2024-08-30T08:58:02.408754+0000","last_active":"2024-08-30T09:00:15.300043+0000","last_peered":"2024-08-30T09:00:15.300043+0000","last_clean":"2024-08-30T09:00:15.300043+0000","last_became_active":"2024-08-30T08:58:02.407127+0000","last_became_peered":"2024-08-30T08:58:02.407127+0000","last_unstale":"2024-08-30T09:00:15.300043+0000","last_undegraded":"2024-08-30T09:00:15.300043+0000","last_fullsized":"2024-08-30T09:00:15.300043+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T16:15:41.796091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T08:59:46.657646+0000","last_change":"2024-08-30T08:58:09.445389+0000","last_active":"2024-08-30T08:59:46.657646+0000","last_peered":"2024-08-30T08:59:46.657646+0000","last_clean":"2024-08-30T08:59:46.657646+0000","last_became_active":"2024-08-30T08:58:09.444831+0000","last_became_peered":"2024-08-30T08:58:09.444831+0000","last_unstale":"2024-08-30T08:59:46.657646+0000","last_undegraded":"2024-08-30T08:59:46.657646+0000","last_fullsized":"2024-08-30T08:59:46.657646+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:08.409016+0000","last_clean_scrub_stamp":"2024-08-30T08:58:08.409016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T09:55:58.162397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"83'9","reported_seq":44,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:15.317322+0000","last_change":"2024-08-30T08:58:02.415184+0000","last_active":"2024-08-30T09:00:15.317322+0000","last_peered":"2024-08-30T09:00:15.317322+0000","last_clean":"2024-08-30T09:00:15.317322+0000","last_became_active":"2024-08-30T08:58:02.414625+0000","last_became_peered":"2024-08-30T08:58:02.414625+0000","last_unstale":"2024-08-30T09:00:15.317322+0000","last_undegraded":"2024-08-30T09:00:15.317322+0000","last_fullsized":"2024-08-30T09:00:15.317322+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T11:10:31.130524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"83'12","reported_seq":47,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:15.315361+0000","last_change":"2024-08-30T08:58:02.405392+0000","last_active":"2024-08-30T09:00:15.315361+0000","last_peered":"2024-08-30T09:00:15.315361+0000","last_clean":"2024-08-30T09:00:15.315361+0000","last_became_active":"2024-08-30T08:58:02.405009+0000","last_became_peered":"2024-08-30T08:58:02.405009+0000","last_unstale":"2024-08-30T09:00:15.315361+0000","last_undegraded":"2024-08-30T09:00:15.315361+0000","last_fullsized":"2024-08-30T09:00:15.315361+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T13:22:46.375598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"83'8","reported_seq":43,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:15.315926+0000","last_change":"2024-08-30T08:58:02.402360+0000","last_active":"2024-08-30T09:00:15.315926+0000","last_peered":"2024-08-30T09:00:15.315926+0000","last_clean":"2024-08-30T09:00:15.315926+0000","last_became_active":"2024-08-30T08:58:02.401856+0000","last_became_peered":"2024-08-30T08:58:02.401856+0000","last_unstale":"2024-08-30T09:00:15.315926+0000","last_undegraded":"2024-08-30T09:00:15.315926+0000","last_fullsized":"2024-08-30T09:00:15.315926+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T10:25:14.491345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"83'7","reported_seq":42,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:15.332529+0000","last_change":"2024-08-30T08:58:02.415940+0000","last_active":"2024-08-30T09:00:15.332529+0000","last_peered":"2024-08-30T09:00:15.332529+0000","last_clean":"2024-08-30T09:00:15.332529+0000","last_became_active":"2024-08-30T08:58:02.415567+0000","last_became_peered":"2024-08-30T08:58:02.415567+0000","last_unstale":"2024-08-30T09:00:15.332529+0000","last_undegraded":"2024-08-30T09:00:15.332529+0000","last_fullsized":"2024-08-30T09:00:15.332529+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T15:02:26.270343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"83'4","reported_seq":39,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:15.312692+0000","last_change":"2024-08-30T08:58:02.406766+0000","last_active":"2024-08-30T09:00:15.312692+0000","last_peered":"2024-08-30T09:00:15.312692+0000","last_clean":"2024-08-30T09:00:15.312692+0000","last_became_active":"2024-08-30T08:58:02.406345+0000","last_became_peered":"2024-08-30T08:58:02.406345+0000","last_unstale":"2024-08-30T09:00:15.312692+0000","last_undegraded":"2024-08-30T09:00:15.312692+0000","last_fullsized":"2024-08-30T09:00:15.312692+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T13:10:32.986817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"83'6","reported_seq":41,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:15.313675+0000","last_change":"2024-08-30T08:58:02.412891+0000","last_active":"2024-08-30T09:00:15.313675+0000","last_peered":"2024-08-30T09:00:15.313675+0000","last_clean":"2024-08-30T09:00:15.313675+0000","last_became_active":"2024-08-30T08:58:02.412532+0000","last_became_peered":"2024-08-30T08:58:02.412532+0000","last_unstale":"2024-08-30T09:00:15.313675+0000","last_undegraded":"2024-08-30T09:00:15.313675+0000","last_fullsized":"2024-08-30T09:00:15.313675+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T14:40:30.663678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"83'15","reported_seq":50,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:15.325109+0000","last_change":"2024-08-30T08:58:02.405862+0000","last_active":"2024-08-30T09:00:15.325109+0000","last_peered":"2024-08-30T09:00:15.325109+0000","last_clean":"2024-08-30T09:00:15.325109+0000","last_became_active":"2024-08-30T08:58:02.405319+0000","last_became_peered":"2024-08-30T08:58:02.405319+0000","last_unstale":"2024-08-30T09:00:15.325109+0000","last_undegraded":"2024-08-30T09:00:15.325109+0000","last_fullsized":"2024-08-30T09:00:15.325109+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T20:36:10.315814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"83'2","reported_seq":37,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:15.316363+0000","last_change":"2024-08-30T08:58:02.401410+0000","last_active":"2024-08-30T09:00:15.316363+0000","last_peered":"2024-08-30T09:00:15.316363+0000","last_clean":"2024-08-30T09:00:15.316363+0000","last_became_active":"2024-08-30T08:58:02.401060+0000","last_became_peered":"2024-08-30T08:58:02.401060+0000","last_unstale":"2024-08-30T09:00:15.316363+0000","last_undegraded":"2024-08-30T09:00:15.316363+0000","last_fullsized":"2024-08-30T09:00:15.316363+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T16:00:41.275367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"83'6","reported_seq":41,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:15.306834+0000","last_change":"2024-08-30T08:58:02.401703+0000","last_active":"2024-08-30T09:00:15.306834+0000","last_peered":"2024-08-30T09:00:15.306834+0000","last_clean":"2024-08-30T09:00:15.306834+0000","last_became_active":"2024-08-30T08:58:02.401218+0000","last_became_peered":"2024-08-30T08:58:02.401218+0000","last_unstale":"2024-08-30T09:00:15.306834+0000","last_undegraded":"2024-08-30T09:00:15.306834+0000","last_fullsized":"2024-08-30T09:00:15.306834+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T18:36:25.249814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"83'8","reported_seq":43,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:15.315612+0000","last_change":"2024-08-30T08:58:02.403609+0000","last_active":"2024-08-30T09:00:15.315612+0000","last_peered":"2024-08-30T09:00:15.315612+0000","last_clean":"2024-08-30T09:00:15.315612+0000","last_became_active":"2024-08-30T08:58:02.402195+0000","last_became_peered":"2024-08-30T08:58:02.402195+0000","last_unstale":"2024-08-30T09:00:15.315612+0000","last_undegraded":"2024-08-30T09:00:15.315612+0000","last_fullsized":"2024-08-30T09:00:15.315612+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T09:59:05.877523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"83'10","reported_seq":45,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:15.333885+0000","last_change":"2024-08-30T08:58:02.404389+0000","last_active":"2024-08-30T09:00:15.333885+0000","last_peered":"2024-08-30T09:00:15.333885+0000","last_clean":"2024-08-30T09:00:15.333885+0000","last_became_active":"2024-08-30T08:58:02.404039+0000","last_became_peered":"2024-08-30T08:58:02.404039+0000","last_unstale":"2024-08-30T09:00:15.333885+0000","last_undegraded":"2024-08-30T09:00:15.333885+0000","last_fullsized":"2024-08-30T09:00:15.333885+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T20:55:51.739542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"83'11","reported_seq":46,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:15.301840+0000","last_change":"2024-08-30T08:58:02.411807+0000","last_active":"2024-08-30T09:00:15.301840+0000","last_peered":"2024-08-30T09:00:15.301840+0000","last_clean":"2024-08-30T09:00:15.301840+0000","last_became_active":"2024-08-30T08:58:02.410977+0000","last_became_peered":"2024-08-30T08:58:02.410977+0000","last_unstale":"2024-08-30T09:00:15.301840+0000","last_undegraded":"2024-08-30T09:00:15.301840+0000","last_fullsized":"2024-08-30T09:00:15.301840+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T13:04:29.931373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"83'8","reported_seq":50,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:15.315288+0000","last_change":"2024-08-30T08:58:46.559661+0000","last_active":"2024-08-30T09:00:15.315288+0000","last_peered":"2024-08-30T09:00:15.315288+0000","last_clean":"2024-08-30T09:00:15.315288+0000","last_became_active":"2024-08-30T08:58:46.558824+0000","last_became_peered":"2024-08-30T08:58:46.558824+0000","last_unstale":"2024-08-30T09:00:15.315288+0000","last_undegraded":"2024-08-30T09:00:15.315288+0000","last_fullsized":"2024-08-30T09:00:15.315288+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T09:53:07.650921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"83'7","reported_seq":42,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:15.316869+0000","last_change":"2024-08-30T08:58:02.396186+0000","last_active":"2024-08-30T09:00:15.316869+0000","last_peered":"2024-08-30T09:00:15.316869+0000","last_clean":"2024-08-30T09:00:15.316869+0000","last_became_active":"2024-08-30T08:58:02.395776+0000","last_became_peered":"2024-08-30T08:58:02.395776+0000","last_unstale":"2024-08-30T09:00:15.316869+0000","last_undegraded":"2024-08-30T09:00:15.316869+0000","last_fullsized":"2024-08-30T09:00:15.316869+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T09:33:30.068673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"83'10","reported_seq":45,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:15.318659+0000","last_change":"2024-08-30T08:58:02.408378+0000","last_active":"2024-08-30T09:00:15.318659+0000","last_peered":"2024-08-30T09:00:15.318659+0000","last_clean":"2024-08-30T09:00:15.318659+0000","last_became_active":"2024-08-30T08:58:02.407991+0000","last_became_peered":"2024-08-30T08:58:02.407991+0000","last_unstale":"2024-08-30T09:00:15.318659+0000","last_undegraded":"2024-08-30T09:00:15.318659+0000","last_fullsized":"2024-08-30T09:00:15.318659+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T20:57:02.411176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"83'8","reported_seq":43,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:15.333624+0000","last_change":"2024-08-30T08:58:02.411829+0000","last_active":"2024-08-30T09:00:15.333624+0000","last_peered":"2024-08-30T09:00:15.333624+0000","last_clean":"2024-08-30T09:00:15.333624+0000","last_became_active":"2024-08-30T08:58:02.411416+0000","last_became_peered":"2024-08-30T08:58:02.411416+0000","last_unstale":"2024-08-30T09:00:15.333624+0000","last_undegraded":"2024-08-30T09:00:15.333624+0000","last_fullsized":"2024-08-30T09:00:15.333624+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T17:12:08.855150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"83'9","reported_seq":44,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:15.333466+0000","last_change":"2024-08-30T08:58:02.405122+0000","last_active":"2024-08-30T09:00:15.333466+0000","last_peered":"2024-08-30T09:00:15.333466+0000","last_clean":"2024-08-30T09:00:15.333466+0000","last_became_active":"2024-08-30T08:58:02.403312+0000","last_became_peered":"2024-08-30T08:58:02.403312+0000","last_unstale":"2024-08-30T09:00:15.333466+0000","last_undegraded":"2024-08-30T09:00:15.333466+0000","last_fullsized":"2024-08-30T09:00:15.333466+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T18:11:53.077433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"83'7","reported_seq":42,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:15.332861+0000","last_change":"2024-08-30T08:58:02.404263+0000","last_active":"2024-08-30T09:00:15.332861+0000","last_peered":"2024-08-30T09:00:15.332861+0000","last_clean":"2024-08-30T09:00:15.332861+0000","last_became_active":"2024-08-30T08:58:02.403897+0000","last_became_peered":"2024-08-30T08:58:02.403897+0000","last_unstale":"2024-08-30T09:00:15.332861+0000","last_undegraded":"2024-08-30T09:00:15.332861+0000","last_fullsized":"2024-08-30T09:00:15.332861+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T18:38:31.439434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"83'7","reported_seq":42,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:15.332364+0000","last_change":"2024-08-30T08:58:02.411578+0000","last_active":"2024-08-30T09:00:15.332364+0000","last_peered":"2024-08-30T09:00:15.332364+0000","last_clean":"2024-08-30T09:00:15.332364+0000","last_became_active":"2024-08-30T08:58:02.411271+0000","last_became_peered":"2024-08-30T08:58:02.411271+0000","last_unstale":"2024-08-30T09:00:15.332364+0000","last_undegraded":"2024-08-30T09:00:15.332364+0000","last_fullsized":"2024-08-30T09:00:15.332364+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T10:02:46.382906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"83'10","reported_seq":45,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:15.310698+0000","last_change":"2024-08-30T08:58:02.412168+0000","last_active":"2024-08-30T09:00:15.310698+0000","last_peered":"2024-08-30T09:00:15.310698+0000","last_clean":"2024-08-30T09:00:15.310698+0000","last_became_active":"2024-08-30T08:58:02.411420+0000","last_became_peered":"2024-08-30T08:58:02.411420+0000","last_unstale":"2024-08-30T09:00:15.310698+0000","last_undegraded":"2024-08-30T09:00:15.310698+0000","last_fullsized":"2024-08-30T09:00:15.310698+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T10:35:53.654344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"83'7","reported_seq":42,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:15.318663+0000","last_change":"2024-08-30T08:58:02.409735+0000","last_active":"2024-08-30T09:00:15.318663+0000","last_peered":"2024-08-30T09:00:15.318663+0000","last_clean":"2024-08-30T09:00:15.318663+0000","last_became_active":"2024-08-30T08:58:02.409349+0000","last_became_peered":"2024-08-30T08:58:02.409349+0000","last_unstale":"2024-08-30T09:00:15.318663+0000","last_undegraded":"2024-08-30T09:00:15.318663+0000","last_fullsized":"2024-08-30T09:00:15.318663+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T20:10:22.956554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"83'6","reported_seq":41,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:15.333587+0000","last_change":"2024-08-30T08:58:02.416261+0000","last_active":"2024-08-30T09:00:15.333587+0000","last_peered":"2024-08-30T09:00:15.333587+0000","last_clean":"2024-08-30T09:00:15.333587+0000","last_became_active":"2024-08-30T08:58:02.415194+0000","last_became_peered":"2024-08-30T08:58:02.415194+0000","last_unstale":"2024-08-30T09:00:15.333587+0000","last_undegraded":"2024-08-30T09:00:15.333587+0000","last_fullsized":"2024-08-30T09:00:15.333587+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T17:59:04.339699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"83'16","reported_seq":58,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:15.320984+0000","last_change":"2024-08-30T08:58:46.560190+0000","last_active":"2024-08-30T09:00:15.320984+0000","last_peered":"2024-08-30T09:00:15.320984+0000","last_clean":"2024-08-30T09:00:15.320984+0000","last_became_active":"2024-08-30T08:58:46.559674+0000","last_became_peered":"2024-08-30T08:58:46.559674+0000","last_unstale":"2024-08-30T09:00:15.320984+0000","last_undegraded":"2024-08-30T09:00:15.320984+0000","last_fullsized":"2024-08-30T09:00:15.320984+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T10:08:02.835142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"83'11","reported_seq":46,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:15.334030+0000","last_change":"2024-08-30T08:58:02.406763+0000","last_active":"2024-08-30T09:00:15.334030+0000","last_peered":"2024-08-30T09:00:15.334030+0000","last_clean":"2024-08-30T09:00:15.334030+0000","last_became_active":"2024-08-30T08:58:02.406377+0000","last_became_peered":"2024-08-30T08:58:02.406377+0000","last_unstale":"2024-08-30T09:00:15.334030+0000","last_undegraded":"2024-08-30T09:00:15.334030+0000","last_fullsized":"2024-08-30T09:00:15.334030+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T13:32:50.492921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"83'9","reported_seq":43,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:15.319363+0000","last_change":"2024-08-30T08:58:02.416638+0000","last_active":"2024-08-30T09:00:15.319363+0000","last_peered":"2024-08-30T09:00:15.319363+0000","last_clean":"2024-08-30T09:00:15.319363+0000","last_became_active":"2024-08-30T08:58:02.415518+0000","last_became_peered":"2024-08-30T08:58:02.415518+0000","last_unstale":"2024-08-30T09:00:15.319363+0000","last_undegraded":"2024-08-30T09:00:15.319363+0000","last_fullsized":"2024-08-30T09:00:15.319363+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T17:05:07.531585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"83'5","reported_seq":39,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:15.330612+0000","last_change":"2024-08-30T08:58:02.412973+0000","last_active":"2024-08-30T09:00:15.330612+0000","last_peered":"2024-08-30T09:00:15.330612+0000","last_clean":"2024-08-30T09:00:15.330612+0000","last_became_active":"2024-08-30T08:58:02.412609+0000","last_became_peered":"2024-08-30T08:58:02.412609+0000","last_unstale":"2024-08-30T09:00:15.330612+0000","last_undegraded":"2024-08-30T09:00:15.330612+0000","last_fullsized":"2024-08-30T09:00:15.330612+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T14:22:37.704098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"83'6","reported_seq":40,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:15.310078+0000","last_change":"2024-08-30T08:58:02.402779+0000","last_active":"2024-08-30T09:00:15.310078+0000","last_peered":"2024-08-30T09:00:15.310078+0000","last_clean":"2024-08-30T09:00:15.310078+0000","last_became_active":"2024-08-30T08:58:02.401514+0000","last_became_peered":"2024-08-30T08:58:02.401514+0000","last_unstale":"2024-08-30T09:00:15.310078+0000","last_undegraded":"2024-08-30T09:00:15.310078+0000","last_fullsized":"2024-08-30T09:00:15.310078+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T11:37:51.450334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"83'4","reported_seq":38,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:15.295684+0000","last_change":"2024-08-30T08:58:02.406607+0000","last_active":"2024-08-30T09:00:15.295684+0000","last_peered":"2024-08-30T09:00:15.295684+0000","last_clean":"2024-08-30T09:00:15.295684+0000","last_became_active":"2024-08-30T08:58:02.406237+0000","last_became_peered":"2024-08-30T08:58:02.406237+0000","last_unstale":"2024-08-30T09:00:15.295684+0000","last_undegraded":"2024-08-30T09:00:15.295684+0000","last_fullsized":"2024-08-30T09:00:15.295684+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T12:09:03.247424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"83'8","reported_seq":43,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:15.323344+0000","last_change":"2024-08-30T08:58:02.410749+0000","last_active":"2024-08-30T09:00:15.323344+0000","last_peered":"2024-08-30T09:00:15.323344+0000","last_clean":"2024-08-30T09:00:15.323344+0000","last_became_active":"2024-08-30T08:58:02.410350+0000","last_became_peered":"2024-08-30T08:58:02.410350+0000","last_unstale":"2024-08-30T09:00:15.323344+0000","last_undegraded":"2024-08-30T09:00:15.323344+0000","last_fullsized":"2024-08-30T09:00:15.323344+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T15:22:51.451789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"83'7","reported_seq":42,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:15.334454+0000","last_change":"2024-08-30T08:58:02.405946+0000","last_active":"2024-08-30T09:00:15.334454+0000","last_peered":"2024-08-30T09:00:15.334454+0000","last_clean":"2024-08-30T09:00:15.334454+0000","last_became_active":"2024-08-30T08:58:02.405590+0000","last_became_peered":"2024-08-30T08:58:02.405590+0000","last_unstale":"2024-08-30T09:00:15.334454+0000","last_undegraded":"2024-08-30T09:00:15.334454+0000","last_fullsized":"2024-08-30T09:00:15.334454+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T11:42:14.966065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"83'4","reported_seq":39,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:15.330681+0000","last_change":"2024-08-30T08:58:02.409649+0000","last_active":"2024-08-30T09:00:15.330681+0000","last_peered":"2024-08-30T09:00:15.330681+0000","last_clean":"2024-08-30T09:00:15.330681+0000","last_became_active":"2024-08-30T08:58:02.409282+0000","last_became_peered":"2024-08-30T08:58:02.409282+0000","last_unstale":"2024-08-30T09:00:15.330681+0000","last_undegraded":"2024-08-30T09:00:15.330681+0000","last_fullsized":"2024-08-30T09:00:15.330681+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T15:33:52.166290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"83'8","reported_seq":43,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-08-30T09:00:15.333394+0000","last_change":"2024-08-30T08:58:02.410711+0000","last_active":"2024-08-30T09:00:15.333394+0000","last_peered":"2024-08-30T09:00:15.333394+0000","last_clean":"2024-08-30T09:00:15.333394+0000","last_became_active":"2024-08-30T08:58:02.410367+0000","last_became_peered":"2024-08-30T08:58:02.410367+0000","last_unstale":"2024-08-30T09:00:15.333394+0000","last_undegraded":"2024-08-30T09:00:15.333394+0000","last_fullsized":"2024-08-30T09:00:15.333394+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-30T08:58:01.376235+0000","last_clean_scrub_stamp":"2024-08-30T08:58:01.376235+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-31T15:27:40.392588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":69291,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":612,"num_read_kb":380,"num_write":183,"num_write_kb":129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":207873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":67,"seq":287762808873,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":958464,"data_stored":474260,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":61,"seq":261993005103,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28628,"kb_used_data":1604,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716524,"statfs":{"total":95995035648,"available":95965720576,"internally_reserved":0,"allocated":1642496,"data_stored":1126565,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":57,"seq":244813135925,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36292,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708860,"statfs":{"total":95995035648,"available":95957872640,"internally_reserved":0,"allocated":970752,"data_stored":478480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":51,"seq":219043332154,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":958464,"data_stored":474192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":46,"seq":197568495679,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28548,"kb_used_data":1524,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716604,"statfs":{"total":95995035648,"available":95965802496,"internally_reserved":0,"allocated":1560576,"data_stored":1070886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":41,"seq":176093659204,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":946176,"data_stored":469560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":35,"seq":150323855433,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36876,"kb_used_data":1532,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708276,"statfs":{"total":95995035648,"available":95957274624,"internally_reserved":0,"allocated":1568768,"data_stored":1065942,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":30,"seq":128849018958,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36280,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708872,"statfs":{"total":95995035648,"available":95957884928,"internally_reserved":0,"allocated":958464,"data_stored":474742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215186,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36280,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708872,"statfs":{"total":95995035648,"available":95957884928,"internally_reserved":0,"allocated":958464,"data_stored":469754,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411416,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36360,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708792,"statfs":{"total":95995035648,"available":95957803008,"internally_reserved":0,"allocated":1040384,"data_stored":535969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574941,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1015808,"data_stored":534087,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738465,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":970752,"data_stored":470948,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":64617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":66499,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":66727,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-30T09:00:29.714 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph fs dump --format=json 2024-08-30T09:00:30.001 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T09:00:30.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:30.522+0000 7fc850503640 1 -- 172.21.15.42:0/2365666504 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc848109bc0 msgr2=0x7fc8481103f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:00:30.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:30.522+0000 7fc850503640 1 --2- 172.21.15.42:0/2365666504 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc848109bc0 0x7fc8481103f0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fc840009ca0 tx=0x7fc84002a8c0 comp rx=0 tx=0).stop 2024-08-30T09:00:30.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:30.522+0000 7fc850503640 1 -- 172.21.15.42:0/2365666504 shutdown_connections 2024-08-30T09:00:30.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:30.522+0000 7fc850503640 1 --2- 172.21.15.42:0/2365666504 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc848109270 0x7fc848109680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:30.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:30.522+0000 7fc850503640 1 --2- 172.21.15.42:0/2365666504 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc848109bc0 0x7fc8481103f0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:30.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:30.522+0000 7fc850503640 1 --2- 172.21.15.42:0/2365666504 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc848103930 0x7fc848103d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:30.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:30.522+0000 7fc850503640 1 -- 172.21.15.42:0/2365666504 >> 172.21.15.42:0/2365666504 conn(0x7fc8480ff380 msgr2=0x7fc8481017d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:00:30.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:30.522+0000 7fc850503640 1 -- 172.21.15.42:0/2365666504 shutdown_connections 2024-08-30T09:00:30.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:30.522+0000 7fc850503640 1 -- 172.21.15.42:0/2365666504 wait complete. 2024-08-30T09:00:30.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:30.523+0000 7fc850503640 1 Processor -- start 2024-08-30T09:00:30.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:30.523+0000 7fc850503640 1 -- start start 2024-08-30T09:00:30.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:30.523+0000 7fc850503640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc848103930 0x7fc84819a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:30.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:30.523+0000 7fc850503640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc848109270 0x7fc84819a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:30.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:30.523+0000 7fc850503640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc848109bc0 0x7fc84819fa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:30.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:30.523+0000 7fc850503640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fc848113880 con 0x7fc848103930 2024-08-30T09:00:30.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:30.523+0000 7fc850503640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc848113700 con 0x7fc848109270 2024-08-30T09:00:30.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:30.523+0000 7fc850503640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fc848113a00 con 0x7fc848109bc0 2024-08-30T09:00:30.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:30.523+0000 7fc84e29f640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc848103930 0x7fc84819a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:30.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:30.524+0000 7fc84e29f640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc848103930 0x7fc84819a210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:51488/0 (socket says 172.21.15.42:51488) 2024-08-30T09:00:30.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:30.524+0000 7fc84e29f640 1 -- 172.21.15.42:0/2968774490 learned_addr learned my addr 172.21.15.42:0/2968774490 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:00:30.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:30.524+0000 7fc84da9e640 1 --2- 172.21.15.42:0/2968774490 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc848109270 0x7fc84819a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:30.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:30.524+0000 7fc84eaa0640 1 --2- 172.21.15.42:0/2968774490 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc848109bc0 0x7fc84819fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:30.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:30.524+0000 7fc84e29f640 1 -- 172.21.15.42:0/2968774490 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc848109bc0 msgr2=0x7fc84819fa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:00:30.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:30.524+0000 7fc84e29f640 1 --2- 172.21.15.42:0/2968774490 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc848109bc0 0x7fc84819fa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:30.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:30.524+0000 7fc84e29f640 1 -- 172.21.15.42:0/2968774490 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc848109270 msgr2=0x7fc84819a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:00:30.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:30.524+0000 7fc84e29f640 1 --2- 172.21.15.42:0/2968774490 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc848109270 0x7fc84819a7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:30.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:30.524+0000 7fc84e29f640 1 -- 172.21.15.42:0/2968774490 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8481a0180 con 0x7fc848103930 2024-08-30T09:00:30.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:30.524+0000 7fc84e29f640 1 --2- 172.21.15.42:0/2968774490 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc848103930 0x7fc84819a210 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fc84400d950 tx=0x7fc84400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:00:30.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:30.524+0000 7fc83f7fe640 1 -- 172.21.15.42:0/2968774490 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc844017030 con 0x7fc848103930 2024-08-30T09:00:30.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:30.524+0000 7fc850503640 1 -- 172.21.15.42:0/2968774490 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8481a0470 con 0x7fc848103930 2024-08-30T09:00:30.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:30.524+0000 7fc83f7fe640 1 -- 172.21.15.42:0/2968774490 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc844004020 con 0x7fc848103930 2024-08-30T09:00:30.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:30.524+0000 7fc83f7fe640 1 -- 172.21.15.42:0/2968774490 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8440047a0 con 0x7fc848103930 2024-08-30T09:00:30.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:30.524+0000 7fc850503640 1 -- 172.21.15.42:0/2968774490 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8481a6bd0 con 0x7fc848103930 2024-08-30T09:00:30.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:30.526+0000 7fc850503640 1 -- 172.21.15.42:0/2968774490 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc848103d40 con 0x7fc848103930 2024-08-30T09:00:30.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:30.527+0000 7fc83f7fe640 1 -- 172.21.15.42:0/2968774490 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7fc844009a30 con 0x7fc848103930 2024-08-30T09:00:30.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:30.528+0000 7fc83f7fe640 1 --2- 172.21.15.42:0/2968774490 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fc8340b4b30 0x7fc8340b7040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:30.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:30.528+0000 7fc83f7fe640 1 -- 172.21.15.42:0/2968774490 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7fc8440e4f90 con 0x7fc848103930 2024-08-30T09:00:30.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:30.528+0000 7fc84da9e640 1 --2- 172.21.15.42:0/2968774490 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fc8340b4b30 0x7fc8340b7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:30.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:30.528+0000 7fc84da9e640 1 --2- 172.21.15.42:0/2968774490 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fc8340b4b30 0x7fc8340b7040 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fc848108d70 tx=0x7fc838005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:00:30.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:30.533+0000 7fc83f7fe640 1 -- 172.21.15.42:0/2968774490 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7fc8440abd30 con 0x7fc848103930 2024-08-30T09:00:30.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:00:30 smithi042 ceph-mon[18652]: from='client.15450 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-30T09:00:30.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:00:30 smithi042 ceph-mon[18652]: pgmap v365: 129 pgs: 129 active+clean; 644 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 682 B/s wr, 8 op/s 2024-08-30T09:00:30.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:00:30 smithi042 ceph-mon[18652]: from='client.15456 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-30T09:00:30.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:00:30 smithi042 ceph-mon[18652]: from='client.15462 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-30T09:00:30.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:00:30 smithi190 ceph-mon[22001]: from='client.15450 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-30T09:00:30.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:00:30 smithi190 ceph-mon[22001]: pgmap v365: 129 pgs: 129 active+clean; 644 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 682 B/s wr, 8 op/s 2024-08-30T09:00:30.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:00:30 smithi190 ceph-mon[22001]: from='client.15456 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-30T09:00:30.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:00:30 smithi190 ceph-mon[22001]: from='client.15462 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-30T09:00:30.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:30.771+0000 7fc850503640 1 -- 172.21.15.42:0/2968774490 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc84819b340 con 0x7fc848103930 2024-08-30T09:00:30.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:30.772+0000 7fc83f7fe640 1 -- 172.21.15.42:0/2968774490 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11915 (secure 0 0 0) 0x7fc8440ab630 con 0x7fc848103930 2024-08-30T09:00:30.774 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-08-30T09:00:30.774 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":27,"btime":"2024-08-30T09:00:25:352021+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14703,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/3188400000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":3188400000},{"type":"v1","addr":"172.21.15.42:6835","nonce":3188400000}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14715,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/3356537674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":3356537674},{"type":"v1","addr":"172.21.15.42:6837","nonce":3356537674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14766,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6839/2107785513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6838","nonce":2107785513},{"type":"v1","addr":"172.21.15.42:6839","nonce":2107785513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14778,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6841/2298438674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6840","nonce":2298438674},{"type":"v1","addr":"172.21.15.42:6841","nonce":2298438674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24553,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/543555570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":543555570},{"type":"v1","addr":"172.21.15.190:6833","nonce":543555570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24554,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/3907818941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":3907818941},{"type":"v1","addr":"172.21.15.167:6833","nonce":3907818941}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":27,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-30T08:58:03.055084+0000","modified":"2024-08-30T09:00:24.377910+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24629,"mds_1":24610,"mds_2":24599},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24572":{"gid":24572,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6835/234596845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":234596845},{"type":"v1","addr":"172.21.15.167:6835","nonce":234596845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24592":{"gid":24592,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6835/247905709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":247905709},{"type":"v1","addr":"172.21.15.190:6835","nonce":247905709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24598":{"gid":24598,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6837/863256349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6836","nonce":863256349},{"type":"v1","addr":"172.21.15.190:6837","nonce":863256349}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24599":{"gid":24599,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.167:6837/3011837856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":3011837856},{"type":"v1","addr":"172.21.15.167:6837","nonce":3011837856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24610":{"gid":24610,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.190:6839/719981110","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6838","nonce":719981110},{"type":"v1","addr":"172.21.15.190:6839","nonce":719981110}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24629":{"gid":24629,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.167:6839/3604645207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":3604645207},{"type":"v1","addr":"172.21.15.167:6839","nonce":3604645207}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24629,"qdb_cluster":[24599,24610,24629]},"id":1}]} 2024-08-30T09:00:30.774 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 27 2024-08-30T09:00:30.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:30.774+0000 7fc850503640 1 -- 172.21.15.42:0/2968774490 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fc8340b4b30 msgr2=0x7fc8340b7040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:00:30.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:30.774+0000 7fc850503640 1 --2- 172.21.15.42:0/2968774490 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fc8340b4b30 0x7fc8340b7040 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fc848108d70 tx=0x7fc838005f90 comp rx=0 tx=0).stop 2024-08-30T09:00:30.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:30.774+0000 7fc850503640 1 -- 172.21.15.42:0/2968774490 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc848103930 msgr2=0x7fc84819a210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:00:30.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:30.774+0000 7fc850503640 1 --2- 172.21.15.42:0/2968774490 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc848103930 0x7fc84819a210 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fc84400d950 tx=0x7fc84400de10 comp rx=0 tx=0).stop 2024-08-30T09:00:30.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:30.774+0000 7fc850503640 1 -- 172.21.15.42:0/2968774490 shutdown_connections 2024-08-30T09:00:30.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:30.774+0000 7fc850503640 1 --2- 172.21.15.42:0/2968774490 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fc8340b4b30 0x7fc8340b7040 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:30.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:30.774+0000 7fc850503640 1 --2- 172.21.15.42:0/2968774490 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc848109bc0 0x7fc84819fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:30.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:30.774+0000 7fc850503640 1 --2- 172.21.15.42:0/2968774490 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc848109270 0x7fc84819a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:30.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:30.774+0000 7fc850503640 1 --2- 172.21.15.42:0/2968774490 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc848103930 0x7fc84819a210 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:30.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:30.774+0000 7fc850503640 1 -- 172.21.15.42:0/2968774490 >> 172.21.15.42:0/2968774490 conn(0x7fc8480ff380 msgr2=0x7fc8481017a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:00:30.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:30.775+0000 7fc850503640 1 -- 172.21.15.42:0/2968774490 shutdown_connections 2024-08-30T09:00:30.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:30.775+0000 7fc850503640 1 -- 172.21.15.42:0/2968774490 wait complete. 2024-08-30T09:00:30.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:00:30 smithi167 ceph-mon[22063]: from='client.15450 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-30T09:00:30.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:00:30 smithi167 ceph-mon[22063]: pgmap v365: 129 pgs: 129 active+clean; 644 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 682 B/s wr, 8 op/s 2024-08-30T09:00:30.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:00:30 smithi167 ceph-mon[22063]: from='client.15456 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-30T09:00:30.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:00:30 smithi167 ceph-mon[22063]: from='client.15462 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-30T09:00:31.413 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph osd dump --format=json 2024-08-30T09:00:31.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:00:31 smithi042 ceph-mon[18652]: from='client.? 172.21.15.42:0/2968774490' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-30T09:00:31.670 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T09:00:31.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:00:31 smithi190 ceph-mon[22001]: from='client.? 172.21.15.42:0/2968774490' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-30T09:00:31.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:00:31 smithi167 ceph-mon[22063]: from='client.? 172.21.15.42:0/2968774490' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-30T09:00:32.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:32.181+0000 7fd268209640 1 -- 172.21.15.42:0/2295446904 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd2601081c0 msgr2=0x7fd2601085d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:00:32.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:32.181+0000 7fd268209640 1 --2- 172.21.15.42:0/2295446904 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd2601081c0 0x7fd2601085d0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fd250009900 tx=0x7fd25002a410 comp rx=0 tx=0).stop 2024-08-30T09:00:32.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:32.181+0000 7fd268209640 1 -- 172.21.15.42:0/2295446904 shutdown_connections 2024-08-30T09:00:32.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:32.181+0000 7fd268209640 1 --2- 172.21.15.42:0/2295446904 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd260108b10 0x7fd26010f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:32.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:32.181+0000 7fd268209640 1 --2- 172.21.15.42:0/2295446904 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd2601081c0 0x7fd2601085d0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:32.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:32.181+0000 7fd268209640 1 --2- 172.21.15.42:0/2295446904 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd260102880 0x7fd260102c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:32.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:32.181+0000 7fd268209640 1 -- 172.21.15.42:0/2295446904 >> 172.21.15.42:0/2295446904 conn(0x7fd260079510 msgr2=0x7fd260079920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:00:32.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:32.181+0000 7fd268209640 1 -- 172.21.15.42:0/2295446904 shutdown_connections 2024-08-30T09:00:32.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:32.181+0000 7fd268209640 1 -- 172.21.15.42:0/2295446904 wait complete. 2024-08-30T09:00:32.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:32.181+0000 7fd268209640 1 Processor -- start 2024-08-30T09:00:32.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:32.182+0000 7fd268209640 1 -- start start 2024-08-30T09:00:32.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:32.182+0000 7fd268209640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd260102880 0x7fd26019a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:32.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:32.182+0000 7fd268209640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd2601081c0 0x7fd26019a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:32.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:32.182+0000 7fd268209640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd260108b10 0x7fd26019fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:32.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:32.182+0000 7fd268209640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fd2601127f0 con 0x7fd260108b10 2024-08-30T09:00:32.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:32.182+0000 7fd268209640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd260112670 con 0x7fd260102880 2024-08-30T09:00:32.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:32.182+0000 7fd268209640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fd260112970 con 0x7fd2601081c0 2024-08-30T09:00:32.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:32.182+0000 7fd2667a6640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd260108b10 0x7fd26019fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:32.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:32.182+0000 7fd2667a6640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd260108b10 0x7fd26019fb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:51512/0 (socket says 172.21.15.42:51512) 2024-08-30T09:00:32.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:32.182+0000 7fd2667a6640 1 -- 172.21.15.42:0/310164668 learned_addr learned my addr 172.21.15.42:0/310164668 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:00:32.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:32.182+0000 7fd2657a4640 1 --2- 172.21.15.42:0/310164668 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd2601081c0 0x7fd26019a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:32.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:32.182+0000 7fd265fa5640 1 --2- 172.21.15.42:0/310164668 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd260102880 0x7fd26019a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:32.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:32.182+0000 7fd2667a6640 1 -- 172.21.15.42:0/310164668 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd2601081c0 msgr2=0x7fd26019a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:00:32.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:32.182+0000 7fd2667a6640 1 --2- 172.21.15.42:0/310164668 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd2601081c0 0x7fd26019a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:32.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:32.182+0000 7fd2667a6640 1 -- 172.21.15.42:0/310164668 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd260102880 msgr2=0x7fd26019a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:00:32.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:32.182+0000 7fd2667a6640 1 --2- 172.21.15.42:0/310164668 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd260102880 0x7fd26019a330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:32.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:32.182+0000 7fd2667a6640 1 -- 172.21.15.42:0/310164668 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2601a02a0 con 0x7fd260108b10 2024-08-30T09:00:32.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:32.183+0000 7fd2667a6640 1 --2- 172.21.15.42:0/310164668 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd260108b10 0x7fd26019fb70 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fd25800b970 tx=0x7fd25800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:00:32.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:32.183+0000 7fd256ffd640 1 -- 172.21.15.42:0/310164668 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd258019070 con 0x7fd260108b10 2024-08-30T09:00:32.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:32.183+0000 7fd256ffd640 1 -- 172.21.15.42:0/310164668 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd258014440 con 0x7fd260108b10 2024-08-30T09:00:32.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:32.183+0000 7fd268209640 1 -- 172.21.15.42:0/310164668 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2601a0590 con 0x7fd260108b10 2024-08-30T09:00:32.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:32.183+0000 7fd256ffd640 1 -- 172.21.15.42:0/310164668 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2580075d0 con 0x7fd260108b10 2024-08-30T09:00:32.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:32.183+0000 7fd268209640 1 -- 172.21.15.42:0/310164668 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2601a6cf0 con 0x7fd260108b10 2024-08-30T09:00:32.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:32.184+0000 7fd268209640 1 -- 172.21.15.42:0/310164668 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd260103b00 con 0x7fd260108b10 2024-08-30T09:00:32.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:32.185+0000 7fd256ffd640 1 -- 172.21.15.42:0/310164668 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7fd25801c070 con 0x7fd260108b10 2024-08-30T09:00:32.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:32.186+0000 7fd256ffd640 1 --2- 172.21.15.42:0/310164668 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fd24c0b4b30 0x7fd24c0b7040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:32.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:32.186+0000 7fd265fa5640 1 --2- 172.21.15.42:0/310164668 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fd24c0b4b30 0x7fd24c0b7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:32.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:32.186+0000 7fd256ffd640 1 -- 172.21.15.42:0/310164668 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7fd2580e4e00 con 0x7fd260108b10 2024-08-30T09:00:32.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:32.187+0000 7fd265fa5640 1 --2- 172.21.15.42:0/310164668 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fd24c0b4b30 0x7fd24c0b7040 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fd25c009bd0 tx=0x7fd25c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:00:32.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:32.189+0000 7fd256ffd640 1 -- 172.21.15.42:0/310164668 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7fd2580ab550 con 0x7fd260108b10 2024-08-30T09:00:32.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:32.375+0000 7fd268209640 1 -- 172.21.15.42:0/310164668 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd260102c90 con 0x7fd260108b10 2024-08-30T09:00:32.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:32.376+0000 7fd256ffd640 1 -- 172.21.15.42:0/310164668 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+24867 (secure 0 0 0) 0x7fd260102c90 con 0x7fd260108b10 2024-08-30T09:00:32.379 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-08-30T09:00:32.379 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":83,"fsid":"b0895c1a-66ac-11ef-bcd4-c7b262605968","created":"2024-08-30T08:49:44.858265+0000","modified":"2024-08-30T08:59:46.645920+0000","last_up_change":"2024-08-30T08:57:12.367791+0000","last_in_change":"2024-08-30T08:56:50.274678+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-30T08:53:27.297022+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-30T08:57:58.489964+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-30T08:58:01.064783+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-08-30T08:58:07.416534+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"029d62d7-a6c6-4905-8c76-6dffa9d57b92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":2617890537},{"type":"v1","addr":"172.21.15.42:6803","nonce":2617890537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":2617890537},{"type":"v1","addr":"172.21.15.42:6805","nonce":2617890537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2617890537},{"type":"v1","addr":"172.21.15.42:6809","nonce":2617890537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":2617890537},{"type":"v1","addr":"172.21.15.42:6807","nonce":2617890537}]},"public_addr":"172.21.15.42:6803/2617890537","cluster_addr":"172.21.15.42:6805/2617890537","heartbeat_back_addr":"172.21.15.42:6809/2617890537","heartbeat_front_addr":"172.21.15.42:6807/2617890537","state":["exists","up"]},{"osd":1,"uuid":"3a78384b-0207-41df-9384-22665007d5b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":2791960082},{"type":"v1","addr":"172.21.15.167:6801","nonce":2791960082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":2791960082},{"type":"v1","addr":"172.21.15.167:6803","nonce":2791960082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":2791960082},{"type":"v1","addr":"172.21.15.167:6807","nonce":2791960082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":2791960082},{"type":"v1","addr":"172.21.15.167:6805","nonce":2791960082}]},"public_addr":"172.21.15.167:6801/2791960082","cluster_addr":"172.21.15.167:6803/2791960082","heartbeat_back_addr":"172.21.15.167:6807/2791960082","heartbeat_front_addr":"172.21.15.167:6805/2791960082","state":["exists","up"]},{"osd":2,"uuid":"16001809-c27c-4f52-948a-9998261acb4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":1101326404},{"type":"v1","addr":"172.21.15.190:6801","nonce":1101326404}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":1101326404},{"type":"v1","addr":"172.21.15.190:6803","nonce":1101326404}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":1101326404},{"type":"v1","addr":"172.21.15.190:6807","nonce":1101326404}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":1101326404},{"type":"v1","addr":"172.21.15.190:6805","nonce":1101326404}]},"public_addr":"172.21.15.190:6801/1101326404","cluster_addr":"172.21.15.190:6803/1101326404","heartbeat_back_addr":"172.21.15.190:6807/1101326404","heartbeat_front_addr":"172.21.15.190:6805/1101326404","state":["exists","up"]},{"osd":3,"uuid":"87bb3e36-9e9e-4b19-9d8c-0cdf6e089d3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":1135599916},{"type":"v1","addr":"172.21.15.42:6811","nonce":1135599916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":1135599916},{"type":"v1","addr":"172.21.15.42:6813","nonce":1135599916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":1135599916},{"type":"v1","addr":"172.21.15.42:6817","nonce":1135599916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":1135599916},{"type":"v1","addr":"172.21.15.42:6815","nonce":1135599916}]},"public_addr":"172.21.15.42:6811/1135599916","cluster_addr":"172.21.15.42:6813/1135599916","heartbeat_back_addr":"172.21.15.42:6817/1135599916","heartbeat_front_addr":"172.21.15.42:6815/1135599916","state":["exists","up"]},{"osd":4,"uuid":"3ca1aae3-c972-4fb9-ac86-236279c30d64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":2927511419},{"type":"v1","addr":"172.21.15.167:6809","nonce":2927511419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":2927511419},{"type":"v1","addr":"172.21.15.167:6811","nonce":2927511419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":2927511419},{"type":"v1","addr":"172.21.15.167:6815","nonce":2927511419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":2927511419},{"type":"v1","addr":"172.21.15.167:6813","nonce":2927511419}]},"public_addr":"172.21.15.167:6809/2927511419","cluster_addr":"172.21.15.167:6811/2927511419","heartbeat_back_addr":"172.21.15.167:6815/2927511419","heartbeat_front_addr":"172.21.15.167:6813/2927511419","state":["exists","up"]},{"osd":5,"uuid":"5d07f54b-0d07-4efe-9bcf-390755ce0258","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":2557145333},{"type":"v1","addr":"172.21.15.190:6809","nonce":2557145333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":2557145333},{"type":"v1","addr":"172.21.15.190:6811","nonce":2557145333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":2557145333},{"type":"v1","addr":"172.21.15.190:6815","nonce":2557145333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":2557145333},{"type":"v1","addr":"172.21.15.190:6813","nonce":2557145333}]},"public_addr":"172.21.15.190:6809/2557145333","cluster_addr":"172.21.15.190:6811/2557145333","heartbeat_back_addr":"172.21.15.190:6815/2557145333","heartbeat_front_addr":"172.21.15.190:6813/2557145333","state":["exists","up"]},{"osd":6,"uuid":"96ad64af-44ff-4197-97d4-156f0cb88e81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":3213526100},{"type":"v1","addr":"172.21.15.42:6819","nonce":3213526100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":3213526100},{"type":"v1","addr":"172.21.15.42:6821","nonce":3213526100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":3213526100},{"type":"v1","addr":"172.21.15.42:6825","nonce":3213526100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":3213526100},{"type":"v1","addr":"172.21.15.42:6823","nonce":3213526100}]},"public_addr":"172.21.15.42:6819/3213526100","cluster_addr":"172.21.15.42:6821/3213526100","heartbeat_back_addr":"172.21.15.42:6825/3213526100","heartbeat_front_addr":"172.21.15.42:6823/3213526100","state":["exists","up"]},{"osd":7,"uuid":"337a2112-eaf7-4e6b-86be-4e40d6ddf835","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":2962799814},{"type":"v1","addr":"172.21.15.167:6817","nonce":2962799814}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":2962799814},{"type":"v1","addr":"172.21.15.167:6819","nonce":2962799814}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":2962799814},{"type":"v1","addr":"172.21.15.167:6823","nonce":2962799814}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":2962799814},{"type":"v1","addr":"172.21.15.167:6821","nonce":2962799814}]},"public_addr":"172.21.15.167:6817/2962799814","cluster_addr":"172.21.15.167:6819/2962799814","heartbeat_back_addr":"172.21.15.167:6823/2962799814","heartbeat_front_addr":"172.21.15.167:6821/2962799814","state":["exists","up"]},{"osd":8,"uuid":"219a7224-4331-464b-ac86-c51054d51d91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":3242198843},{"type":"v1","addr":"172.21.15.190:6817","nonce":3242198843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":3242198843},{"type":"v1","addr":"172.21.15.190:6819","nonce":3242198843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":3242198843},{"type":"v1","addr":"172.21.15.190:6823","nonce":3242198843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":3242198843},{"type":"v1","addr":"172.21.15.190:6821","nonce":3242198843}]},"public_addr":"172.21.15.190:6817/3242198843","cluster_addr":"172.21.15.190:6819/3242198843","heartbeat_back_addr":"172.21.15.190:6823/3242198843","heartbeat_front_addr":"172.21.15.190:6821/3242198843","state":["exists","up"]},{"osd":9,"uuid":"2f2f5177-a009-40d9-991c-76aec8a5cba6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":1123536325},{"type":"v1","addr":"172.21.15.42:6827","nonce":1123536325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":1123536325},{"type":"v1","addr":"172.21.15.42:6829","nonce":1123536325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":1123536325},{"type":"v1","addr":"172.21.15.42:6833","nonce":1123536325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":1123536325},{"type":"v1","addr":"172.21.15.42:6831","nonce":1123536325}]},"public_addr":"172.21.15.42:6827/1123536325","cluster_addr":"172.21.15.42:6829/1123536325","heartbeat_back_addr":"172.21.15.42:6833/1123536325","heartbeat_front_addr":"172.21.15.42:6831/1123536325","state":["exists","up"]},{"osd":10,"uuid":"96ef5058-2518-4862-a165-17666b33a3e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":3451198319},{"type":"v1","addr":"172.21.15.167:6825","nonce":3451198319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":3451198319},{"type":"v1","addr":"172.21.15.167:6827","nonce":3451198319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":3451198319},{"type":"v1","addr":"172.21.15.167:6831","nonce":3451198319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":3451198319},{"type":"v1","addr":"172.21.15.167:6829","nonce":3451198319}]},"public_addr":"172.21.15.167:6825/3451198319","cluster_addr":"172.21.15.167:6827/3451198319","heartbeat_back_addr":"172.21.15.167:6831/3451198319","heartbeat_front_addr":"172.21.15.167:6829/3451198319","state":["exists","up"]},{"osd":11,"uuid":"a4eb9bd3-3faa-4c81-ba6d-c2084b72c9ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":380064457},{"type":"v1","addr":"172.21.15.190:6825","nonce":380064457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":380064457},{"type":"v1","addr":"172.21.15.190:6827","nonce":380064457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":380064457},{"type":"v1","addr":"172.21.15.190:6831","nonce":380064457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":380064457},{"type":"v1","addr":"172.21.15.190:6829","nonce":380064457}]},"public_addr":"172.21.15.190:6825/380064457","cluster_addr":"172.21.15.190:6827/380064457","heartbeat_back_addr":"172.21.15.190:6831/380064457","heartbeat_front_addr":"172.21.15.190:6829/380064457","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-30T08:52:36.511448+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-30T08:53:00.027277+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-30T08:53:23.770245+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-30T08:53:49.322226+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-30T08:54:14.350234+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-30T08:54:39.491793+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-30T08:55:04.870596+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-30T08:55:30.314150+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-30T08:55:54.483843+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-30T08:56:20.586115+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-30T08:56:45.666783+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-30T08:57:10.718387+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"2.1d","mappings":[{"from":2,"to":1}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"4.9","mappings":[{"from":5,"to":8}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.42:0/2691233260":"2024-08-31T08:50:44.821070+0000","172.21.15.42:0/1518053351":"2024-08-31T08:50:44.821070+0000","172.21.15.42:0/352001020":"2024-08-31T08:50:44.821070+0000","172.21.15.42:6801/1733768634":"2024-08-31T08:50:44.821070+0000","172.21.15.42:0/1000597888":"2024-08-31T08:50:18.561851+0000","172.21.15.42:6800/1733768634":"2024-08-31T08:50:44.821070+0000","172.21.15.42:6801/3538882081":"2024-08-31T08:50:18.561851+0000","172.21.15.42:6800/3538882081":"2024-08-31T08:50:18.561851+0000","172.21.15.42:0/145366508":"2024-08-31T08:50:18.561851+0000","172.21.15.42:0/3346355213":"2024-08-31T08:50:18.561851+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-30T09:00:32.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:32.379+0000 7fd268209640 1 -- 172.21.15.42:0/310164668 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fd24c0b4b30 msgr2=0x7fd24c0b7040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:00:32.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:32.379+0000 7fd268209640 1 --2- 172.21.15.42:0/310164668 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fd24c0b4b30 0x7fd24c0b7040 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fd25c009bd0 tx=0x7fd25c005f90 comp rx=0 tx=0).stop 2024-08-30T09:00:32.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:32.379+0000 7fd268209640 1 -- 172.21.15.42:0/310164668 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd260108b10 msgr2=0x7fd26019fb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:00:32.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:32.379+0000 7fd268209640 1 --2- 172.21.15.42:0/310164668 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd260108b10 0x7fd26019fb70 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fd25800b970 tx=0x7fd25800be30 comp rx=0 tx=0).stop 2024-08-30T09:00:32.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:32.379+0000 7fd268209640 1 -- 172.21.15.42:0/310164668 shutdown_connections 2024-08-30T09:00:32.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:32.379+0000 7fd268209640 1 --2- 172.21.15.42:0/310164668 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fd24c0b4b30 0x7fd24c0b7040 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:32.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:32.379+0000 7fd268209640 1 --2- 172.21.15.42:0/310164668 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd2601081c0 0x7fd26019a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:32.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:32.379+0000 7fd268209640 1 --2- 172.21.15.42:0/310164668 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd260108b10 0x7fd26019fb70 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:32.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:32.379+0000 7fd268209640 1 --2- 172.21.15.42:0/310164668 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd260102880 0x7fd26019a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:32.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:32.379+0000 7fd268209640 1 -- 172.21.15.42:0/310164668 >> 172.21.15.42:0/310164668 conn(0x7fd260079510 msgr2=0x7fd26006bce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:00:32.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:32.379+0000 7fd268209640 1 -- 172.21.15.42:0/310164668 shutdown_connections 2024-08-30T09:00:32.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:32.379+0000 7fd268209640 1 -- 172.21.15.42:0/310164668 wait complete. 2024-08-30T09:00:32.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:00:32 smithi042 ceph-mon[18652]: pgmap v366: 129 pgs: 129 active+clean; 644 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 170 B/s wr, 5 op/s 2024-08-30T09:00:32.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:00:32 smithi042 ceph-mon[18652]: from='client.? 172.21.15.42:0/310164668' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-30T09:00:32.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:00:32 smithi190 ceph-mon[22001]: pgmap v366: 129 pgs: 129 active+clean; 644 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 170 B/s wr, 5 op/s 2024-08-30T09:00:32.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:00:32 smithi190 ceph-mon[22001]: from='client.? 172.21.15.42:0/310164668' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-30T09:00:32.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:00:32 smithi167 ceph-mon[22063]: pgmap v366: 129 pgs: 129 active+clean; 644 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 170 B/s wr, 5 op/s 2024-08-30T09:00:32.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:00:32 smithi167 ceph-mon[22063]: from='client.? 172.21.15.42:0/310164668' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-30T09:00:33.005 DEBUG:tasks.fwd_scrub:Yielding 2024-08-30T09:00:33.006 INFO:teuthology.run_tasks:Running task background_exec... 2024-08-30T09:00:33.038 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi042.front.sepia.ceph.com 2024-08-30T09:00:33.038 DEBUG:teuthology.orchestra.run.smithi042:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-08-30T09:00:33.038 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-08-30T09:00:33.039 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-08-30T09:00:33.039 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-08-30T09:00:33.041 INFO:teuthology.run_tasks:Running task exec... 2024-08-30T09:00:33.050 INFO:teuthology.task.exec:Executing custom commands... 2024-08-30T09:00:33.050 INFO:teuthology.task.exec:Running commands on role client.0 host ubuntu@smithi042.front.sepia.ceph.com 2024-08-30T09:00:33.050 DEBUG:teuthology.orchestra.run.smithi042:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -l /tmp/cephfs/ && sudo df -h /tmp/cephfs/' 2024-08-30T09:00:33.085 INFO:teuthology.orchestra.run.smithi042.stdout:total 0 2024-08-30T09:00:33.099 INFO:teuthology.orchestra.run.smithi042.stdout:Filesystem Size Used Avail Use% Mounted on 2024-08-30T09:00:33.100 INFO:teuthology.orchestra.run.smithi042.stdout:0@b0895c1a-66ac-11ef-bcd4-c7b262605968.cephfs=/volumes/qa/sv_1/d3168875-d2e5-410d-8754-72d29955777b 1.1T 372M 1.1T 1% /tmp/cephfs 2024-08-30T09:00:33.102 DEBUG:teuthology.orchestra.run.smithi042:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo mkdir -m 755 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres && sudo chown postgres:postgres /tmp/cephfs/postgres' 2024-08-30T09:00:33.192 DEBUG:teuthology.orchestra.run.smithi042:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- mkdir -m 700 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres/data' 2024-08-30T09:00:33.231 DEBUG:teuthology.orchestra.run.smithi042:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo semanage fcontext -a -t postgresql_db_t "/tmp/cephfs/postgres(/.*)?"' 2024-08-30T09:00:33.300 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T09:00:33.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.891+0000 7f6e4267f640 1 -- 172.21.15.42:0/1173695709 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6e3c108140 msgr2=0x7f6e3c108550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:00:33.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.891+0000 7f6e4267f640 1 --2- 172.21.15.42:0/1173695709 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6e3c108140 0x7f6e3c108550 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f6e30009900 tx=0x7f6e3002a410 comp rx=0 tx=0).stop 2024-08-30T09:00:33.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.891+0000 7f6e4267f640 1 -- 172.21.15.42:0/1173695709 shutdown_connections 2024-08-30T09:00:33.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.891+0000 7f6e4267f640 1 --2- 172.21.15.42:0/1173695709 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6e3c108a90 0x7f6e3c10f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:33.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.891+0000 7f6e4267f640 1 --2- 172.21.15.42:0/1173695709 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6e3c108140 0x7f6e3c108550 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:33.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.891+0000 7f6e4267f640 1 --2- 172.21.15.42:0/1173695709 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6e3c102800 0x7f6e3c102c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:33.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.891+0000 7f6e4267f640 1 -- 172.21.15.42:0/1173695709 >> 172.21.15.42:0/1173695709 conn(0x7f6e3c0794d0 msgr2=0x7f6e3c0798e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:00:33.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.891+0000 7f6e4267f640 1 -- 172.21.15.42:0/1173695709 shutdown_connections 2024-08-30T09:00:33.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.891+0000 7f6e4267f640 1 -- 172.21.15.42:0/1173695709 wait complete. 2024-08-30T09:00:33.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.892+0000 7f6e4267f640 1 Processor -- start 2024-08-30T09:00:33.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.892+0000 7f6e4267f640 1 -- start start 2024-08-30T09:00:33.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.892+0000 7f6e4267f640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6e3c102800 0x7f6e3c19a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:33.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.892+0000 7f6e4267f640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6e3c108140 0x7f6e3c19a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:33.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.892+0000 7f6e4267f640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6e3c108a90 0x7f6e3c19fb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:33.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.892+0000 7f6e4267f640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f6e3c112760 con 0x7f6e3c102800 2024-08-30T09:00:33.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.892+0000 7f6e4267f640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f6e3c1125e0 con 0x7f6e3c108140 2024-08-30T09:00:33.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.892+0000 7f6e4267f640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f6e3c1128e0 con 0x7f6e3c108a90 2024-08-30T09:00:33.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.893+0000 7f6e3bfff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6e3c102800 0x7f6e3c19a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:33.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.893+0000 7f6e3bfff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6e3c102800 0x7f6e3c19a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:51534/0 (socket says 172.21.15.42:51534) 2024-08-30T09:00:33.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.893+0000 7f6e3bfff640 1 -- 172.21.15.42:0/2198579008 learned_addr learned my addr 172.21.15.42:0/2198579008 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:00:33.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.893+0000 7f6e40c1c640 1 --2- 172.21.15.42:0/2198579008 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6e3c108a90 0x7f6e3c19fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:33.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.893+0000 7f6e3b7fe640 1 --2- 172.21.15.42:0/2198579008 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6e3c108140 0x7f6e3c19a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:33.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.893+0000 7f6e3bfff640 1 -- 172.21.15.42:0/2198579008 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6e3c108a90 msgr2=0x7f6e3c19fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:00:33.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.893+0000 7f6e3bfff640 1 --2- 172.21.15.42:0/2198579008 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6e3c108a90 0x7f6e3c19fb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:33.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.893+0000 7f6e3bfff640 1 -- 172.21.15.42:0/2198579008 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6e3c108140 msgr2=0x7f6e3c19a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:00:33.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.893+0000 7f6e3bfff640 1 --2- 172.21.15.42:0/2198579008 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6e3c108140 0x7f6e3c19a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:33.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.893+0000 7f6e3bfff640 1 -- 172.21.15.42:0/2198579008 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e3c1a0270 con 0x7f6e3c102800 2024-08-30T09:00:33.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.893+0000 7f6e3b7fe640 1 --2- 172.21.15.42:0/2198579008 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6e3c108140 0x7f6e3c19a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-30T09:00:33.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.893+0000 7f6e3bfff640 1 --2- 172.21.15.42:0/2198579008 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6e3c102800 0x7f6e3c19a300 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f6e2c00b4a0 tx=0x7f6e2c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:00:33.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.893+0000 7f6e397fa640 1 -- 172.21.15.42:0/2198579008 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e2c014070 con 0x7f6e3c102800 2024-08-30T09:00:33.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.893+0000 7f6e4267f640 1 -- 172.21.15.42:0/2198579008 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e3c1a0560 con 0x7f6e3c102800 2024-08-30T09:00:33.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.893+0000 7f6e397fa640 1 -- 172.21.15.42:0/2198579008 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6e2c010ac0 con 0x7f6e3c102800 2024-08-30T09:00:33.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.893+0000 7f6e397fa640 1 -- 172.21.15.42:0/2198579008 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e2c017c30 con 0x7f6e3c102800 2024-08-30T09:00:33.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.894+0000 7f6e4267f640 1 -- 172.21.15.42:0/2198579008 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e3c1a6cc0 con 0x7f6e3c102800 2024-08-30T09:00:33.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.896+0000 7f6e48bf4740 1 Processor -- start 2024-08-30T09:00:33.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.896+0000 7f6e48bf4740 1 -- start start 2024-08-30T09:00:33.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.896+0000 7f6e48bf4740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563aa3bff350 0x563aa3fa5780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:33.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.896+0000 7f6e48bf4740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563aa3bd6490 0x563aa3fa5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:33.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.896+0000 7f6e48bf4740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563aa3bde670 0x563aa3fa6200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:33.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.896+0000 7f6e48bf4740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x563aa3f1b5f0 con 0x563aa3bde670 2024-08-30T09:00:33.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.896+0000 7f6e48bf4740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x563aa3f0ab80 con 0x563aa3bd6490 2024-08-30T09:00:33.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.896+0000 7f6e48bf4740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x563aa3f0eb30 con 0x563aa3bff350 2024-08-30T09:00:33.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.896+0000 7f6e397fa640 1 -- 172.21.15.42:0/2198579008 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7f6e2c017070 con 0x7f6e3c102800 2024-08-30T09:00:33.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.897+0000 7f6e3b7fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563aa3bde670 0x563aa3fa6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:33.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.897+0000 7f6e3b7fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563aa3bde670 0x563aa3fa6200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:51546/0 (socket says 172.21.15.42:51546) 2024-08-30T09:00:33.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.897+0000 7f6e3b7fe640 1 -- 172.21.15.42:0/2928582559 learned_addr learned my addr 172.21.15.42:0/2928582559 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:00:33.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.897+0000 7f6e40c1c640 1 --2- 172.21.15.42:0/2928582559 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563aa3bff350 0x563aa3fa5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:33.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.897+0000 7f6e3b7fe640 1 -- 172.21.15.42:0/2928582559 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563aa3bff350 msgr2=0x563aa3fa5780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:00:33.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.897+0000 7f6e3b7fe640 1 --2- 172.21.15.42:0/2928582559 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563aa3bff350 0x563aa3fa5780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:33.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.897+0000 7f6e3b7fe640 1 -- 172.21.15.42:0/2928582559 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563aa3bd6490 msgr2=0x563aa3fa5cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:00:33.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.897+0000 7f6e3bfff640 1 --2- 172.21.15.42:0/2928582559 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563aa3bd6490 0x563aa3fa5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:33.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.897+0000 7f6e3b7fe640 1 --2- 172.21.15.42:0/2928582559 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563aa3bd6490 0x563aa3fa5cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:33.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.897+0000 7f6e3b7fe640 1 -- 172.21.15.42:0/2928582559 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563aa3c6a930 con 0x563aa3bde670 2024-08-30T09:00:33.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.897+0000 7f6e3bfff640 1 --2- 172.21.15.42:0/2928582559 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563aa3bd6490 0x563aa3fa5cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-30T09:00:33.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.897+0000 7f6e3b7fe640 1 --2- 172.21.15.42:0/2928582559 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563aa3bde670 0x563aa3fa6200 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f6e30031440 tx=0x7f6e30032e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6afe9feb399973c server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:00:33.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.897+0000 7f6e397fa640 1 --2- 172.21.15.42:0/2198579008 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f6e240b4b30 0x7f6e240b7040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:33.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.897+0000 7f6e227fc640 1 -- 172.21.15.42:0/2928582559 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e3003f070 con 0x563aa3bde670 2024-08-30T09:00:33.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.898+0000 7f6e40c1c640 1 --2- 172.21.15.42:0/2198579008 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f6e240b4b30 0x7f6e240b7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:33.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.898+0000 7f6e227fc640 1 -- 172.21.15.42:0/2928582559 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6e300316a0 con 0x563aa3bde670 2024-08-30T09:00:33.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.898+0000 7f6e227fc640 1 -- 172.21.15.42:0/2928582559 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e30035070 con 0x563aa3bde670 2024-08-30T09:00:33.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.898+0000 7f6e397fa640 1 -- 172.21.15.42:0/2198579008 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f6e2c0e4a70 con 0x7f6e3c102800 2024-08-30T09:00:33.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.899+0000 7f6e40c1c640 1 --2- 172.21.15.42:0/2198579008 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f6e240b4b30 0x7f6e240b7040 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f6e34005fd0 tx=0x7f6e34009600 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:00:33.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.899+0000 7f6e48bf4740 1 -- 172.21.15.42:0/2928582559 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563aa3bde670 msgr2=0x563aa3fa6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:00:33.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.899+0000 7f6e48bf4740 1 --2- 172.21.15.42:0/2928582559 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563aa3bde670 0x563aa3fa6200 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f6e30031440 tx=0x7f6e30032e60 comp rx=0 tx=0).stop 2024-08-30T09:00:33.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.899+0000 7f6e48bf4740 1 -- 172.21.15.42:0/2928582559 shutdown_connections 2024-08-30T09:00:33.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.899+0000 7f6e48bf4740 1 --2- 172.21.15.42:0/2928582559 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563aa3bde670 0x563aa3fa6200 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:33.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.899+0000 7f6e48bf4740 1 --2- 172.21.15.42:0/2928582559 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563aa3bd6490 0x563aa3fa5cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:33.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.899+0000 7f6e48bf4740 1 --2- 172.21.15.42:0/2928582559 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563aa3bff350 0x563aa3fa5780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:33.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.899+0000 7f6e48bf4740 1 -- 172.21.15.42:0/2928582559 >> 172.21.15.42:0/2928582559 conn(0x563aa3bf3740 msgr2=0x563aa3fa3670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:00:33.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.899+0000 7f6e48bf4740 1 -- 172.21.15.42:0/2928582559 shutdown_connections 2024-08-30T09:00:33.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.899+0000 7f6e48bf4740 1 -- 172.21.15.42:0/2928582559 wait complete. 2024-08-30T09:00:33.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.900+0000 7f6e48bf4740 1 -- start start 2024-08-30T09:00:33.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.900+0000 7f6e48bf4740 1 Processor -- start 2024-08-30T09:00:33.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.900+0000 7f6e48bf4740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563aa3bd6490 0x563aa3fb0ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:33.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.900+0000 7f6e48bf4740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563aa3bde670 0x563aa3fb1400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:33.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.900+0000 7f6e48bf4740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563aa3bff350 0x563aa3fbd280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:33.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.900+0000 7f6e48bf4740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x563aa3d6d890 con 0x563aa3bd6490 2024-08-30T09:00:33.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.900+0000 7f6e48bf4740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x563aa3e91790 con 0x563aa3bff350 2024-08-30T09:00:33.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.900+0000 7f6e48bf4740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x563aa3fb3f00 con 0x563aa3bde670 2024-08-30T09:00:33.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.901+0000 7f6e48bf4740 10 client.0 _ll_register_callbacks cb 0x563aa3c16ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:00:33.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.901+0000 7f6e3bfff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563aa3bd6490 0x563aa3fb0ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:33.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.901+0000 7f6e3bfff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563aa3bd6490 0x563aa3fb0ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:51552/0 (socket says 172.21.15.42:51552) 2024-08-30T09:00:33.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.901+0000 7f6e3bfff640 1 -- 172.21.15.42:0/3063077479 learned_addr learned my addr 172.21.15.42:0/3063077479 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:00:33.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.901+0000 7f6e40c1c640 1 --2- 172.21.15.42:0/3063077479 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563aa3bff350 0x563aa3fbd280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:33.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.901+0000 7f6e3b7fe640 1 --2- 172.21.15.42:0/3063077479 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563aa3bde670 0x563aa3fb1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:33.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.901+0000 7f6e3bfff640 1 -- 172.21.15.42:0/3063077479 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563aa3bde670 msgr2=0x563aa3fb1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:00:33.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.901+0000 7f6e3bfff640 1 --2- 172.21.15.42:0/3063077479 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563aa3bde670 0x563aa3fb1400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:33.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.901+0000 7f6e3bfff640 1 -- 172.21.15.42:0/3063077479 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563aa3bff350 msgr2=0x563aa3fbd280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:00:33.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.901+0000 7f6e3bfff640 1 --2- 172.21.15.42:0/3063077479 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563aa3bff350 0x563aa3fbd280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:33.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.901+0000 7f6e3bfff640 1 -- 172.21.15.42:0/3063077479 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563aa3ec8f30 con 0x563aa3bd6490 2024-08-30T09:00:33.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.901+0000 7f6e3bfff640 1 --2- 172.21.15.42:0/3063077479 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563aa3bd6490 0x563aa3fb0ec0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f6e2c00be60 tx=0x7f6e2c017360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adbf1b7b0451255 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:00:33.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.901+0000 7f6e1b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-08-30T09:00:33.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.901+0000 7f6e1b7fe640 1 -- 172.21.15.42:0/3063077479 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e2c01e070 con 0x563aa3bd6490 2024-08-30T09:00:33.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.901+0000 7f6e1b7fe640 1 -- 172.21.15.42:0/3063077479 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f6e2c0a73a0 con 0x563aa3bd6490 2024-08-30T09:00:33.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.901+0000 7f6e48bf4740 1 -- 172.21.15.42:0/3063077479 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6e300316a0 con 0x563aa3bd6490 2024-08-30T09:00:33.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.901+0000 7f6e1b7fe640 1 -- 172.21.15.42:0/3063077479 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6e2c0ab110 con 0x563aa3bd6490 2024-08-30T09:00:33.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.901+0000 7f6e48bf4740 10 client.15486 fetch_fsmap learned FSMap version 27 2024-08-30T09:00:33.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.902+0000 7f6e48bf4740 1 -- 172.21.15.42:0/3063077479 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x563aa3fbfca0 con 0x563aa3bd6490 2024-08-30T09:00:33.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.902+0000 7f6e1b7fe640 1 -- 172.21.15.42:0/3063077479 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6e300316a0 con 0x563aa3bd6490 2024-08-30T09:00:33.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.902+0000 7f6e1b7fe640 1 -- 172.21.15.42:0/3063077479 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f6e2c0aa370 con 0x563aa3bd6490 2024-08-30T09:00:33.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.902+0000 7f6e48bf4740 10 client.15486 fetch_fsmap finished waiting for FSMap version 27 2024-08-30T09:00:33.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.902+0000 7f6e48bf4740 10 client.15486 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-30T09:00:33.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.902+0000 7f6e48bf4740 20 client.15486 populate_metadata read hostname 'smithi042' 2024-08-30T09:00:33.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.902+0000 7f6e48bf4740 1 --2- 172.21.15.42:0/3063077479 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x563aa3bbdcd0 0x563aa3fc3160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:33.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.902+0000 7f6e48bf4740 4 client.15486 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-30T09:00:33.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.902+0000 7f6e48bf4740 1 -- 172.21.15.42:0/3063077479 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563aa3f1d540 con 0x563aa3bbdcd0 2024-08-30T09:00:33.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.902+0000 7f6e3b7fe640 1 --2- 172.21.15.42:0/3063077479 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x563aa3bbdcd0 0x563aa3fc3160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:33.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.903+0000 7f6e3b7fe640 1 --2- 172.21.15.42:0/3063077479 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x563aa3bbdcd0 0x563aa3fc3160 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b213aa3389ca513d server_cookie=ab79c116df0352bb in_seq=0 out_seq=0 2024-08-30T09:00:33.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.903+0000 7f6e1b7fe640 10 client.15486 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:00:33.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.903+0000 7f6e1b7fe640 1 -- 172.21.15.42:0/3063077479 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x563aa3f1d540 con 0x563aa3bbdcd0 2024-08-30T09:00:33.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.904+0000 7f6e1b7fe640 10 client.15486 handle_command_reply: tid=0 2024-08-30T09:00:33.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.904+0000 7f6e48bf4740 1 client.15486 shutdown 2024-08-30T09:00:33.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.904+0000 7f6e48bf4740 1 -- 172.21.15.42:0/3063077479 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563aa3bd6490 msgr2=0x563aa3fb0ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:00:33.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.904+0000 7f6e48bf4740 1 --2- 172.21.15.42:0/3063077479 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563aa3bd6490 0x563aa3fb0ec0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f6e2c00be60 tx=0x7f6e2c017360 comp rx=0 tx=0).stop 2024-08-30T09:00:33.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.904+0000 7f6e48bf4740 1 -- 172.21.15.42:0/3063077479 shutdown_connections 2024-08-30T09:00:33.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.904+0000 7f6e48bf4740 1 --2- 172.21.15.42:0/3063077479 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x563aa3bbdcd0 0x563aa3fc3160 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:33.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.904+0000 7f6e48bf4740 1 --2- 172.21.15.42:0/3063077479 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563aa3bde670 0x563aa3fb1400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:33.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.904+0000 7f6e48bf4740 1 --2- 172.21.15.42:0/3063077479 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563aa3bff350 0x563aa3fbd280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:33.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.904+0000 7f6e1b7fe640 0 client.15486 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:00:33.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.904+0000 7f6e48bf4740 1 --2- 172.21.15.42:0/3063077479 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563aa3bd6490 0x563aa3fb0ec0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:33.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.904+0000 7f6e48bf4740 1 -- 172.21.15.42:0/3063077479 >> 172.21.15.42:0/3063077479 conn(0x563aa3bf3740 msgr2=0x563aa3fa8910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:00:33.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.904+0000 7f6e48bf4740 1 -- 172.21.15.42:0/3063077479 shutdown_connections 2024-08-30T09:00:33.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.904+0000 7f6e48bf4740 1 -- 172.21.15.42:0/3063077479 wait complete. 2024-08-30T09:00:33.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.904+0000 7f6e48bf4740 20 client.15486 trim_cache size 0 max 16384 2024-08-30T09:00:33.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.922+0000 7f6e48bf4740 1 Processor -- start 2024-08-30T09:00:33.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.922+0000 7f6e48bf4740 1 -- start start 2024-08-30T09:00:33.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.922+0000 7f6e48bf4740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563aa3bbdcd0 0x563aa3fac670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:33.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.922+0000 7f6e48bf4740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563aa3bd6490 0x563aa3fb2af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:33.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.922+0000 7f6e48bf4740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563aa3bde670 0x563aa3fa2ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:33.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.922+0000 7f6e48bf4740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f6e3c1b31f0 con 0x563aa3bbdcd0 2024-08-30T09:00:33.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.922+0000 7f6e48bf4740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f6e3c1b0ac0 con 0x563aa3bd6490 2024-08-30T09:00:33.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.922+0000 7f6e48bf4740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f6e3c1ae910 con 0x563aa3bde670 2024-08-30T09:00:33.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.923+0000 7f6e3bfff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563aa3bbdcd0 0x563aa3fac670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:33.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.923+0000 7f6e3bfff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563aa3bbdcd0 0x563aa3fac670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:51558/0 (socket says 172.21.15.42:51558) 2024-08-30T09:00:33.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.923+0000 7f6e3bfff640 1 -- 172.21.15.42:0/438352705 learned_addr learned my addr 172.21.15.42:0/438352705 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:00:33.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.923+0000 7f6e3b7fe640 1 --2- 172.21.15.42:0/438352705 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563aa3bd6490 0x563aa3fb2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:33.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.923+0000 7f6e40c1c640 1 --2- 172.21.15.42:0/438352705 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563aa3bde670 0x563aa3fa2ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:33.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.923+0000 7f6e3bfff640 1 -- 172.21.15.42:0/438352705 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563aa3bde670 msgr2=0x563aa3fa2ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:00:33.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.923+0000 7f6e3bfff640 1 --2- 172.21.15.42:0/438352705 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563aa3bde670 0x563aa3fa2ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:33.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.923+0000 7f6e3bfff640 1 -- 172.21.15.42:0/438352705 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563aa3bd6490 msgr2=0x563aa3fb2af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:00:33.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.923+0000 7f6e3bfff640 1 --2- 172.21.15.42:0/438352705 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563aa3bd6490 0x563aa3fb2af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:33.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.923+0000 7f6e3bfff640 1 -- 172.21.15.42:0/438352705 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563aa3faf3c0 con 0x563aa3bbdcd0 2024-08-30T09:00:33.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.923+0000 7f6e3bfff640 1 --2- 172.21.15.42:0/438352705 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563aa3bbdcd0 0x563aa3fac670 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f6e2c0ab0a0 tx=0x7f6e2c017650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d45c39789414b0a6 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:00:33.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.923+0000 7f6e21ffb640 1 -- 172.21.15.42:0/438352705 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e2c01e030 con 0x563aa3bbdcd0 2024-08-30T09:00:33.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.923+0000 7f6e21ffb640 1 -- 172.21.15.42:0/438352705 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6e2c0af980 con 0x563aa3bbdcd0 2024-08-30T09:00:33.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.923+0000 7f6e21ffb640 1 -- 172.21.15.42:0/438352705 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e2c0aa370 con 0x563aa3bbdcd0 2024-08-30T09:00:33.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.924+0000 7f6e48bf4740 1 -- 172.21.15.42:0/438352705 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563aa3bbdcd0 msgr2=0x563aa3fac670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:00:33.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.924+0000 7f6e48bf4740 1 --2- 172.21.15.42:0/438352705 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563aa3bbdcd0 0x563aa3fac670 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f6e2c0ab0a0 tx=0x7f6e2c017650 comp rx=0 tx=0).stop 2024-08-30T09:00:33.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.924+0000 7f6e48bf4740 1 -- 172.21.15.42:0/438352705 shutdown_connections 2024-08-30T09:00:33.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.924+0000 7f6e48bf4740 1 --2- 172.21.15.42:0/438352705 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563aa3bde670 0x563aa3fa2ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:33.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.924+0000 7f6e48bf4740 1 --2- 172.21.15.42:0/438352705 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563aa3bd6490 0x563aa3fb2af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:33.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.924+0000 7f6e48bf4740 1 --2- 172.21.15.42:0/438352705 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563aa3bbdcd0 0x563aa3fac670 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:33.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.924+0000 7f6e48bf4740 1 -- 172.21.15.42:0/438352705 >> 172.21.15.42:0/438352705 conn(0x563aa3bf3740 msgr2=0x563aa3fb23e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:00:33.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.924+0000 7f6e48bf4740 1 -- 172.21.15.42:0/438352705 shutdown_connections 2024-08-30T09:00:33.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.924+0000 7f6e48bf4740 1 -- 172.21.15.42:0/438352705 wait complete. 2024-08-30T09:00:33.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.925+0000 7f6e48bf4740 1 -- start start 2024-08-30T09:00:33.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.925+0000 7f6e48bf4740 1 Processor -- start 2024-08-30T09:00:33.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.925+0000 7f6e48bf4740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563aa3bbdcd0 0x563aa3fb9880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:33.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.925+0000 7f6e48bf4740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563aa3bd6490 0x563aa3fb5d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:33.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.925+0000 7f6e48bf4740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563aa3bde670 0x563aa3fb6280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:33.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.925+0000 7f6e48bf4740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f6e3c1aef80 con 0x563aa3bd6490 2024-08-30T09:00:33.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.925+0000 7f6e48bf4740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f6e3c1b03f0 con 0x563aa3bde670 2024-08-30T09:00:33.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.925+0000 7f6e48bf4740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f6e3c1af650 con 0x563aa3bbdcd0 2024-08-30T09:00:33.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.925+0000 7f6e48bf4740 10 client.0 _ll_register_callbacks cb 0x7f6e2c017550 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:00:33.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.925+0000 7f6e3b7fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563aa3bd6490 0x563aa3fb5d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:33.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.926+0000 7f6e3b7fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563aa3bd6490 0x563aa3fb5d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:51568/0 (socket says 172.21.15.42:51568) 2024-08-30T09:00:33.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.926+0000 7f6e3b7fe640 1 -- 172.21.15.42:0/2353945847 learned_addr learned my addr 172.21.15.42:0/2353945847 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:00:33.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.926+0000 7f6e3bfff640 1 --2- 172.21.15.42:0/2353945847 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563aa3bbdcd0 0x563aa3fb9880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:33.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.926+0000 7f6e40c1c640 1 --2- 172.21.15.42:0/2353945847 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563aa3bde670 0x563aa3fb6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:33.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.926+0000 7f6e3b7fe640 1 -- 172.21.15.42:0/2353945847 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563aa3bbdcd0 msgr2=0x563aa3fb9880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:00:33.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.926+0000 7f6e3b7fe640 1 --2- 172.21.15.42:0/2353945847 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563aa3bbdcd0 0x563aa3fb9880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:33.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.926+0000 7f6e3b7fe640 1 -- 172.21.15.42:0/2353945847 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563aa3bde670 msgr2=0x563aa3fb6280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:00:33.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.926+0000 7f6e3b7fe640 1 --2- 172.21.15.42:0/2353945847 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563aa3bde670 0x563aa3fb6280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:33.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.926+0000 7f6e3b7fe640 1 -- 172.21.15.42:0/2353945847 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563aa3fb68c0 con 0x563aa3bd6490 2024-08-30T09:00:33.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.926+0000 7f6e3b7fe640 1 --2- 172.21.15.42:0/2353945847 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563aa3bd6490 0x563aa3fb5d40 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f6e30006010 tx=0x7f6e3002aa90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=280bc67919f7f65a server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:00:33.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.926+0000 7f6e1b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-08-30T09:00:33.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.926+0000 7f6e1b7fe640 1 -- 172.21.15.42:0/2353945847 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e30058060 con 0x563aa3bd6490 2024-08-30T09:00:33.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.926+0000 7f6e1b7fe640 1 -- 172.21.15.42:0/2353945847 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f6e30031b10 con 0x563aa3bd6490 2024-08-30T09:00:33.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.926+0000 7f6e48bf4740 1 -- 172.21.15.42:0/2353945847 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6e2c0af980 con 0x563aa3bd6490 2024-08-30T09:00:33.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.927+0000 7f6e1b7fe640 1 -- 172.21.15.42:0/2353945847 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6e3005f050 con 0x563aa3bd6490 2024-08-30T09:00:33.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.927+0000 7f6e48bf4740 10 client.15492 fetch_fsmap learned FSMap version 27 2024-08-30T09:00:33.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.927+0000 7f6e48bf4740 1 -- 172.21.15.42:0/2353945847 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x563aa4032dd0 con 0x563aa3bd6490 2024-08-30T09:00:33.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.927+0000 7f6e1b7fe640 1 -- 172.21.15.42:0/2353945847 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6e2c0af980 con 0x563aa3bd6490 2024-08-30T09:00:33.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.927+0000 7f6e1b7fe640 1 -- 172.21.15.42:0/2353945847 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f6e300037f0 con 0x563aa3bd6490 2024-08-30T09:00:33.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.927+0000 7f6e48bf4740 10 client.15492 fetch_fsmap finished waiting for FSMap version 27 2024-08-30T09:00:33.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.927+0000 7f6e48bf4740 10 client.15492 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-30T09:00:33.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.927+0000 7f6e48bf4740 20 client.15492 populate_metadata read hostname 'smithi042' 2024-08-30T09:00:33.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.927+0000 7f6e48bf4740 1 --2- 172.21.15.42:0/2353945847 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x563aa3bff350 0x563aa4035070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:33.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.927+0000 7f6e48bf4740 4 client.15492 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-08-30T09:00:33.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.927+0000 7f6e48bf4740 1 -- 172.21.15.42:0/2353945847 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x563aa3f18fe0 con 0x563aa3bff350 2024-08-30T09:00:33.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.927+0000 7f6e3bfff640 1 --2- 172.21.15.42:0/2353945847 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x563aa3bff350 0x563aa4035070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:33.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.928+0000 7f6e1b7fe640 10 client.15492 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:00:33.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.928+0000 7f6e3bfff640 1 --2- 172.21.15.42:0/2353945847 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x563aa3bff350 0x563aa4035070 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=327254ea168100af server_cookie=1a64a5be21e8c14b in_seq=0 out_seq=0 2024-08-30T09:00:33.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.929+0000 7f6e1b7fe640 1 -- 172.21.15.42:0/2353945847 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x563aa3f18fe0 con 0x563aa3bff350 2024-08-30T09:00:33.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.929+0000 7f6e1b7fe640 10 client.15492 handle_command_reply: tid=0 2024-08-30T09:00:33.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.929+0000 7f6e48bf4740 1 client.15492 shutdown 2024-08-30T09:00:33.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.930+0000 7f6e48bf4740 1 -- 172.21.15.42:0/2353945847 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563aa3bd6490 msgr2=0x563aa3fb5d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:00:33.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.930+0000 7f6e48bf4740 1 --2- 172.21.15.42:0/2353945847 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563aa3bd6490 0x563aa3fb5d40 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f6e30006010 tx=0x7f6e3002aa90 comp rx=0 tx=0).stop 2024-08-30T09:00:33.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.930+0000 7f6e48bf4740 1 -- 172.21.15.42:0/2353945847 shutdown_connections 2024-08-30T09:00:33.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.930+0000 7f6e48bf4740 1 --2- 172.21.15.42:0/2353945847 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x563aa3bff350 0x563aa4035070 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:33.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.930+0000 7f6e48bf4740 1 --2- 172.21.15.42:0/2353945847 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563aa3bde670 0x563aa3fb6280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:33.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.930+0000 7f6e48bf4740 1 --2- 172.21.15.42:0/2353945847 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563aa3bd6490 0x563aa3fb5d40 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:33.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.930+0000 7f6e48bf4740 1 --2- 172.21.15.42:0/2353945847 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563aa3bbdcd0 0x563aa3fb9880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:33.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.930+0000 7f6e48bf4740 1 -- 172.21.15.42:0/2353945847 >> 172.21.15.42:0/2353945847 conn(0x563aa3bf3740 msgr2=0x563aa3fa8880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:00:33.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.930+0000 7f6e1b7fe640 0 client.15492 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:00:33.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.930+0000 7f6e48bf4740 1 -- 172.21.15.42:0/2353945847 shutdown_connections 2024-08-30T09:00:33.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.930+0000 7f6e48bf4740 1 -- 172.21.15.42:0/2353945847 wait complete. 2024-08-30T09:00:33.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.930+0000 7f6e48bf4740 20 client.15492 trim_cache size 0 max 16384 2024-08-30T09:00:33.936 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-08-30T09:00:33.936 INFO:teuthology.orchestra.run.smithi042.stdout: "return_code": 0, 2024-08-30T09:00:33.936 INFO:teuthology.orchestra.run.smithi042.stdout: "scrub_tag": "5e2ed104-0625-478a-a78b-584e7f4c2817", 2024-08-30T09:00:33.936 INFO:teuthology.orchestra.run.smithi042.stdout: "mode": "asynchronous" 2024-08-30T09:00:33.936 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-08-30T09:00:33.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.931+0000 7f6e4267f640 1 -- 172.21.15.42:0/2198579008 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f6e240b4b30 msgr2=0x7f6e240b7040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:00:33.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.931+0000 7f6e4267f640 1 --2- 172.21.15.42:0/2198579008 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f6e240b4b30 0x7f6e240b7040 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f6e34005fd0 tx=0x7f6e34009600 comp rx=0 tx=0).stop 2024-08-30T09:00:33.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.931+0000 7f6e4267f640 1 -- 172.21.15.42:0/2198579008 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6e3c102800 msgr2=0x7f6e3c19a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:00:33.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.931+0000 7f6e4267f640 1 --2- 172.21.15.42:0/2198579008 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6e3c102800 0x7f6e3c19a300 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f6e2c00b4a0 tx=0x7f6e2c00b960 comp rx=0 tx=0).stop 2024-08-30T09:00:33.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.931+0000 7f6e4267f640 1 -- 172.21.15.42:0/2198579008 shutdown_connections 2024-08-30T09:00:33.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.931+0000 7f6e4267f640 1 --2- 172.21.15.42:0/2198579008 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f6e240b4b30 0x7f6e240b7040 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:33.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.931+0000 7f6e4267f640 1 --2- 172.21.15.42:0/2198579008 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6e3c108a90 0x7f6e3c19fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:33.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.931+0000 7f6e4267f640 1 --2- 172.21.15.42:0/2198579008 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6e3c108140 0x7f6e3c19a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:33.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.931+0000 7f6e4267f640 1 --2- 172.21.15.42:0/2198579008 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6e3c102800 0x7f6e3c19a300 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:33.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.931+0000 7f6e4267f640 1 -- 172.21.15.42:0/2198579008 >> 172.21.15.42:0/2198579008 conn(0x7f6e3c0794d0 msgr2=0x7f6e3c06af90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:00:33.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.931+0000 7f6e4267f640 1 -- 172.21.15.42:0/2198579008 shutdown_connections 2024-08-30T09:00:33.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:33.931+0000 7f6e4267f640 1 -- 172.21.15.42:0/2198579008 wait complete. 2024-08-30T09:00:34.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:00:34 smithi190 ceph-mon[22001]: pgmap v367: 129 pgs: 129 active+clean; 650 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 682 B/s wr, 7 op/s 2024-08-30T09:00:34.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:00:34 smithi167 ceph-mon[22063]: pgmap v367: 129 pgs: 129 active+clean; 650 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 682 B/s wr, 7 op/s 2024-08-30T09:00:34.868 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:00:34 smithi042 ceph-mon[18652]: pgmap v367: 129 pgs: 129 active+clean; 650 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 682 B/s wr, 7 op/s 2024-08-30T09:00:35.369 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-30T09:00:35.403 DEBUG:teuthology.orchestra.run.smithi042:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo grep -i postgresql /etc/selinux/targeted/contexts/files/file_contexts.local' 2024-08-30T09:00:35.480 INFO:teuthology.orchestra.run.smithi042.stdout:/tmp/cephfs/postgres(/.*)? system_u:object_r:postgresql_db_t:s0 2024-08-30T09:00:35.482 DEBUG:teuthology.orchestra.run.smithi042:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo restorecon -R -v /tmp/cephfs/postgres' 2024-08-30T09:00:35.543 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:00:35 smithi042 ceph-mon[18652]: scrub queued for path: / 2024-08-30T09:00:35.544 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:00:35 smithi042 ceph-mon[18652]: scrub summary: idle+waiting paths [/] 2024-08-30T09:00:35.544 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:00:35 smithi042 ceph-mon[18652]: scrub summary: active paths [/] 2024-08-30T09:00:35.544 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:00:35 smithi042 ceph-mon[18652]: scrub summary: idle+waiting paths [/] 2024-08-30T09:00:35.560 DEBUG:teuthology.orchestra.run.smithi042:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -lZaR /tmp/cephfs/postgres/' 2024-08-30T09:00:35.637 INFO:teuthology.orchestra.run.smithi042.stdout:/tmp/cephfs/postgres/: 2024-08-30T09:00:35.637 INFO:teuthology.orchestra.run.smithi042.stdout:total 0 2024-08-30T09:00:35.638 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Aug 30 09:00 . 2024-08-30T09:00:35.638 INFO:teuthology.orchestra.run.smithi042.stdout:drwxrwxrwt. 3 root root system_u:object_r:postgresql_db_t:s0 1 Aug 30 09:00 .. 2024-08-30T09:00:35.638 INFO:teuthology.orchestra.run.smithi042.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 data 2024-08-30T09:00:35.638 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-08-30T09:00:35.638 INFO:teuthology.orchestra.run.smithi042.stdout:/tmp/cephfs/postgres/data: 2024-08-30T09:00:35.638 INFO:teuthology.orchestra.run.smithi042.stdout:total 0 2024-08-30T09:00:35.638 INFO:teuthology.orchestra.run.smithi042.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 . 2024-08-30T09:00:35.638 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Aug 30 09:00 .. 2024-08-30T09:00:35.640 DEBUG:teuthology.orchestra.run.smithi042:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo mkdir -p /etc/systemd/system/postgresql.service.d/ && printf '"'"'[Service]\nEnvironment=PGDATA=/tmp/cephfs/postgres/data\nEnvironment=PGLOG=/tmp/cephfs/postgres/pgstartup.log\n'"'"' | sudo tee /etc/systemd/system/postgresql.service.d/env.conf' 2024-08-30T09:00:35.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:00:35 smithi190 ceph-mon[22001]: scrub queued for path: / 2024-08-30T09:00:35.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:00:35 smithi190 ceph-mon[22001]: scrub summary: idle+waiting paths [/] 2024-08-30T09:00:35.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:00:35 smithi190 ceph-mon[22001]: scrub summary: active paths [/] 2024-08-30T09:00:35.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:00:35 smithi190 ceph-mon[22001]: scrub summary: idle+waiting paths [/] 2024-08-30T09:00:35.707 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T09:00:35.726 INFO:teuthology.orchestra.run.smithi042.stdout:[Service] 2024-08-30T09:00:35.726 INFO:teuthology.orchestra.run.smithi042.stdout:Environment=PGDATA=/tmp/cephfs/postgres/data 2024-08-30T09:00:35.726 INFO:teuthology.orchestra.run.smithi042.stdout:Environment=PGLOG=/tmp/cephfs/postgres/pgstartup.log 2024-08-30T09:00:35.728 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:00:35 smithi167 ceph-mon[22063]: scrub queued for path: / 2024-08-30T09:00:35.728 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:00:35 smithi167 ceph-mon[22063]: scrub summary: idle+waiting paths [/] 2024-08-30T09:00:35.728 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:00:35 smithi167 ceph-mon[22063]: scrub summary: active paths [/] 2024-08-30T09:00:35.729 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:00:35 smithi167 ceph-mon[22063]: scrub summary: idle+waiting paths [/] 2024-08-30T09:00:35.729 DEBUG:teuthology.orchestra.run.smithi042:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- postgresql-setup --initdb' 2024-08-30T09:00:35.863 INFO:teuthology.orchestra.run.smithi042.stderr: * Initializing database in '/tmp/cephfs/postgres/data' 2024-08-30T09:00:36.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.275+0000 7f8a3ff5a640 1 -- 172.21.15.42:0/937243352 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8a38108a90 msgr2=0x7f8a3810f2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:00:36.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.275+0000 7f8a3ff5a640 1 --2- 172.21.15.42:0/937243352 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8a38108a90 0x7f8a3810f2c0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f8a30009ca0 tx=0x7f8a3002a8c0 comp rx=0 tx=0).stop 2024-08-30T09:00:36.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.275+0000 7f8a3ff5a640 1 -- 172.21.15.42:0/937243352 shutdown_connections 2024-08-30T09:00:36.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.275+0000 7f8a3ff5a640 1 --2- 172.21.15.42:0/937243352 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8a38108140 0x7f8a38108550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:36.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.275+0000 7f8a3ff5a640 1 --2- 172.21.15.42:0/937243352 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8a38108a90 0x7f8a3810f2c0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:36.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.275+0000 7f8a3ff5a640 1 --2- 172.21.15.42:0/937243352 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8a38102800 0x7f8a38102c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:36.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.275+0000 7f8a3ff5a640 1 -- 172.21.15.42:0/937243352 >> 172.21.15.42:0/937243352 conn(0x7f8a380794d0 msgr2=0x7f8a380798e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:00:36.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.275+0000 7f8a3ff5a640 1 -- 172.21.15.42:0/937243352 shutdown_connections 2024-08-30T09:00:36.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.275+0000 7f8a3ff5a640 1 -- 172.21.15.42:0/937243352 wait complete. 2024-08-30T09:00:36.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.276+0000 7f8a3ff5a640 1 Processor -- start 2024-08-30T09:00:36.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.276+0000 7f8a3ff5a640 1 -- start start 2024-08-30T09:00:36.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.276+0000 7f8a3ff5a640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8a38102800 0x7f8a3819a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:36.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.276+0000 7f8a3ff5a640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8a38108140 0x7f8a3819a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:36.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.276+0000 7f8a3ff5a640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8a38108a90 0x7f8a3819fa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:36.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.277+0000 7f8a3ff5a640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f8a381126f0 con 0x7f8a38108a90 2024-08-30T09:00:36.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.277+0000 7f8a3ff5a640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f8a38112570 con 0x7f8a38108140 2024-08-30T09:00:36.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.277+0000 7f8a3ff5a640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f8a38112870 con 0x7f8a38102800 2024-08-30T09:00:36.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.277+0000 7f8a3e4f7640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8a38108a90 0x7f8a3819fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:36.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.277+0000 7f8a3e4f7640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8a38108a90 0x7f8a3819fa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:51596/0 (socket says 172.21.15.42:51596) 2024-08-30T09:00:36.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.277+0000 7f8a3e4f7640 1 -- 172.21.15.42:0/2973488386 learned_addr learned my addr 172.21.15.42:0/2973488386 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:00:36.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.277+0000 7f8a3d4f5640 1 --2- 172.21.15.42:0/2973488386 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8a38108140 0x7f8a3819a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:36.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.277+0000 7f8a3dcf6640 1 --2- 172.21.15.42:0/2973488386 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8a38102800 0x7f8a3819a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:36.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.277+0000 7f8a3d4f5640 1 -- 172.21.15.42:0/2973488386 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8a38102800 msgr2=0x7f8a3819a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:00:36.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.277+0000 7f8a3d4f5640 1 --2- 172.21.15.42:0/2973488386 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8a38102800 0x7f8a3819a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:36.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.277+0000 7f8a3d4f5640 1 -- 172.21.15.42:0/2973488386 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8a38108a90 msgr2=0x7f8a3819fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:00:36.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.277+0000 7f8a3d4f5640 1 --2- 172.21.15.42:0/2973488386 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8a38108a90 0x7f8a3819fa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:36.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.277+0000 7f8a3d4f5640 1 -- 172.21.15.42:0/2973488386 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a381a00e0 con 0x7f8a38108140 2024-08-30T09:00:36.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.277+0000 7f8a3e4f7640 1 --2- 172.21.15.42:0/2973488386 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8a38108a90 0x7f8a3819fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-30T09:00:36.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.277+0000 7f8a3d4f5640 1 --2- 172.21.15.42:0/2973488386 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8a38108140 0x7f8a3819a7d0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f8a2800c940 tx=0x7f8a2800ce00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:00:36.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.277+0000 7f8a2effd640 1 -- 172.21.15.42:0/2973488386 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a28007bb0 con 0x7f8a38108140 2024-08-30T09:00:36.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.278+0000 7f8a3ff5a640 1 -- 172.21.15.42:0/2973488386 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a381a03d0 con 0x7f8a38108140 2024-08-30T09:00:36.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.278+0000 7f8a2effd640 1 -- 172.21.15.42:0/2973488386 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a28007d50 con 0x7f8a38108140 2024-08-30T09:00:36.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.278+0000 7f8a2effd640 1 -- 172.21.15.42:0/2973488386 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a28018730 con 0x7f8a38108140 2024-08-30T09:00:36.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.278+0000 7f8a3ff5a640 1 -- 172.21.15.42:0/2973488386 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a381a6bf0 con 0x7f8a38108140 2024-08-30T09:00:36.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.280+0000 7f8a464cf740 1 Processor -- start 2024-08-30T09:00:36.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.280+0000 7f8a464cf740 1 -- start start 2024-08-30T09:00:36.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.281+0000 7f8a2effd640 1 -- 172.21.15.42:0/2973488386 <== mon.2 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7f8a280188d0 con 0x7f8a38108140 2024-08-30T09:00:36.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.281+0000 7f8a464cf740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5620936c2070 0x562093a672c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:36.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.281+0000 7f8a464cf740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5620936991b0 0x562093a67800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:36.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.281+0000 7f8a464cf740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5620936a1390 0x562093a67d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:36.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.281+0000 7f8a464cf740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x5620939de630 con 0x5620936a1390 2024-08-30T09:00:36.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.281+0000 7f8a464cf740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5620939cdbc0 con 0x5620936991b0 2024-08-30T09:00:36.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.281+0000 7f8a464cf740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5620939d1b70 con 0x5620936c2070 2024-08-30T09:00:36.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.281+0000 7f8a3d4f5640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5620936a1390 0x562093a67d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:36.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.281+0000 7f8a3dcf6640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5620936991b0 0x562093a67800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:36.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.281+0000 7f8a3d4f5640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5620936a1390 0x562093a67d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:51604/0 (socket says 172.21.15.42:51604) 2024-08-30T09:00:36.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.281+0000 7f8a3d4f5640 1 -- 172.21.15.42:0/2326530523 learned_addr learned my addr 172.21.15.42:0/2326530523 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:00:36.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.281+0000 7f8a3e4f7640 1 --2- 172.21.15.42:0/2326530523 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5620936c2070 0x562093a672c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:36.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.281+0000 7f8a3d4f5640 1 -- 172.21.15.42:0/2326530523 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5620936c2070 msgr2=0x562093a672c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:00:36.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.281+0000 7f8a3d4f5640 1 --2- 172.21.15.42:0/2326530523 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5620936c2070 0x562093a672c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:36.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.281+0000 7f8a3d4f5640 1 -- 172.21.15.42:0/2326530523 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5620936991b0 msgr2=0x562093a67800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:00:36.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.281+0000 7f8a3d4f5640 1 --2- 172.21.15.42:0/2326530523 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5620936991b0 0x562093a67800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:36.285 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.281+0000 7f8a3d4f5640 1 -- 172.21.15.42:0/2326530523 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56209372d650 con 0x5620936a1390 2024-08-30T09:00:36.285 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.281+0000 7f8a3dcf6640 1 --2- 172.21.15.42:0/2326530523 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5620936991b0 0x562093a67800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-30T09:00:36.285 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.282+0000 7f8a2effd640 1 --2- 172.21.15.42:0/2973488386 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f8a240b4b60 0x7f8a240b7070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:36.285 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.282+0000 7f8a3d4f5640 1 --2- 172.21.15.42:0/2326530523 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5620936a1390 0x562093a67d40 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f8a280eb600 tx=0x7f8a280eb6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fbc8eb26989ec5 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:00:36.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.282+0000 7f8a3e4f7640 1 --2- 172.21.15.42:0/2973488386 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f8a240b4b60 0x7f8a240b7070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:36.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.282+0000 7f8a23fff640 1 -- 172.21.15.42:0/2326530523 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a280ace60 con 0x5620936a1390 2024-08-30T09:00:36.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.282+0000 7f8a2effd640 1 -- 172.21.15.42:0/2973488386 <== mon.2 v2:172.21.15.167:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f8a280e5490 con 0x7f8a38108140 2024-08-30T09:00:36.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.282+0000 7f8a23fff640 1 -- 172.21.15.42:0/2326530523 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a280ab640 con 0x5620936a1390 2024-08-30T09:00:36.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.282+0000 7f8a23fff640 1 -- 172.21.15.42:0/2326530523 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a280a79f0 con 0x5620936a1390 2024-08-30T09:00:36.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.283+0000 7f8a464cf740 1 -- 172.21.15.42:0/2326530523 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5620936a1390 msgr2=0x562093a67d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:00:36.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.283+0000 7f8a464cf740 1 --2- 172.21.15.42:0/2326530523 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5620936a1390 0x562093a67d40 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f8a280eb600 tx=0x7f8a280eb6f0 comp rx=0 tx=0).stop 2024-08-30T09:00:36.288 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.283+0000 7f8a3e4f7640 1 --2- 172.21.15.42:0/2973488386 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f8a240b4b60 0x7f8a240b7070 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f8a300094e0 tx=0x7f8a30005c50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:00:36.288 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.283+0000 7f8a464cf740 1 -- 172.21.15.42:0/2326530523 shutdown_connections 2024-08-30T09:00:36.288 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.283+0000 7f8a464cf740 1 --2- 172.21.15.42:0/2326530523 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5620936a1390 0x562093a67d40 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:36.288 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.283+0000 7f8a464cf740 1 --2- 172.21.15.42:0/2326530523 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5620936991b0 0x562093a67800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:36.288 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.283+0000 7f8a464cf740 1 --2- 172.21.15.42:0/2326530523 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5620936c2070 0x562093a672c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:36.288 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.283+0000 7f8a464cf740 1 -- 172.21.15.42:0/2326530523 >> 172.21.15.42:0/2326530523 conn(0x5620936b6460 msgr2=0x562093a651b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:00:36.288 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.283+0000 7f8a464cf740 1 -- 172.21.15.42:0/2326530523 shutdown_connections 2024-08-30T09:00:36.288 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.283+0000 7f8a464cf740 1 -- 172.21.15.42:0/2326530523 wait complete. 2024-08-30T09:00:36.288 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.284+0000 7f8a464cf740 1 -- start start 2024-08-30T09:00:36.288 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.284+0000 7f8a464cf740 1 Processor -- start 2024-08-30T09:00:36.288 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.284+0000 7f8a464cf740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5620936991b0 0x562093a72a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:36.288 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.284+0000 7f8a464cf740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5620936a1390 0x562093a72fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:36.288 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.284+0000 7f8a464cf740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5620936c2070 0x562093a7ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:36.289 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.284+0000 7f8a464cf740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x562093830700 con 0x5620936991b0 2024-08-30T09:00:36.289 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.284+0000 7f8a464cf740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x562093954700 con 0x5620936c2070 2024-08-30T09:00:36.289 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.284+0000 7f8a464cf740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x562093a75aa0 con 0x5620936a1390 2024-08-30T09:00:36.289 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.284+0000 7f8a464cf740 10 client.0 _ll_register_callbacks cb 0x562093983dc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:00:36.289 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.284+0000 7f8a3dcf6640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5620936991b0 0x562093a72a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:36.289 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.284+0000 7f8a3dcf6640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5620936991b0 0x562093a72a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:51606/0 (socket says 172.21.15.42:51606) 2024-08-30T09:00:36.289 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.284+0000 7f8a3dcf6640 1 -- 172.21.15.42:0/2655385820 learned_addr learned my addr 172.21.15.42:0/2655385820 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:00:36.289 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.284+0000 7f8a3e4f7640 1 --2- 172.21.15.42:0/2655385820 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5620936c2070 0x562093a7ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:36.289 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.285+0000 7f8a3d4f5640 1 --2- 172.21.15.42:0/2655385820 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5620936a1390 0x562093a72fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:36.289 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.285+0000 7f8a3dcf6640 1 -- 172.21.15.42:0/2655385820 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5620936a1390 msgr2=0x562093a72fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:00:36.289 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.285+0000 7f8a3dcf6640 1 --2- 172.21.15.42:0/2655385820 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5620936a1390 0x562093a72fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:36.289 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.285+0000 7f8a3dcf6640 1 -- 172.21.15.42:0/2655385820 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5620936c2070 msgr2=0x562093a7ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:00:36.289 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.285+0000 7f8a3dcf6640 1 --2- 172.21.15.42:0/2655385820 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5620936c2070 0x562093a7ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:36.290 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.285+0000 7f8a3dcf6640 1 -- 172.21.15.42:0/2655385820 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56209398d8b0 con 0x5620936991b0 2024-08-30T09:00:36.290 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.285+0000 7f8a3d4f5640 1 --2- 172.21.15.42:0/2655385820 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5620936a1390 0x562093a72fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-30T09:00:36.290 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.285+0000 7f8a3dcf6640 1 --2- 172.21.15.42:0/2655385820 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5620936991b0 0x562093a72a70 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f8a34012700 tx=0x7f8a34012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=428c349e7b759c2d server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:00:36.290 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.285+0000 7f8a217fa640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-08-30T09:00:36.290 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.285+0000 7f8a217fa640 1 -- 172.21.15.42:0/2655385820 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a34011bb0 con 0x5620936991b0 2024-08-30T09:00:36.290 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.285+0000 7f8a217fa640 1 -- 172.21.15.42:0/2655385820 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f8a34011d50 con 0x5620936991b0 2024-08-30T09:00:36.290 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.285+0000 7f8a464cf740 1 -- 172.21.15.42:0/2655385820 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a280ab640 con 0x5620936991b0 2024-08-30T09:00:36.290 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.285+0000 7f8a217fa640 1 -- 172.21.15.42:0/2655385820 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f8a3400f460 con 0x5620936991b0 2024-08-30T09:00:36.290 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.285+0000 7f8a464cf740 10 client.15504 fetch_fsmap learned FSMap version 27 2024-08-30T09:00:36.290 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.285+0000 7f8a464cf740 1 -- 172.21.15.42:0/2655385820 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x562093a81900 con 0x5620936991b0 2024-08-30T09:00:36.290 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.286+0000 7f8a217fa640 1 -- 172.21.15.42:0/2655385820 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a280ab640 con 0x5620936991b0 2024-08-30T09:00:36.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.286+0000 7f8a217fa640 1 -- 172.21.15.42:0/2655385820 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f8a3401c7c0 con 0x5620936991b0 2024-08-30T09:00:36.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.286+0000 7f8a464cf740 10 client.15504 fetch_fsmap finished waiting for FSMap version 27 2024-08-30T09:00:36.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.286+0000 7f8a464cf740 10 client.15504 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-30T09:00:36.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.286+0000 7f8a464cf740 20 client.15504 populate_metadata read hostname 'smithi042' 2024-08-30T09:00:36.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.286+0000 7f8a464cf740 1 --2- 172.21.15.42:0/2655385820 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x5620936809f0 0x562093a84eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:36.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.286+0000 7f8a464cf740 4 client.15504 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-30T09:00:36.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.286+0000 7f8a464cf740 1 -- 172.21.15.42:0/2655385820 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5620939e0580 con 0x5620936809f0 2024-08-30T09:00:36.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.286+0000 7f8a3d4f5640 1 --2- 172.21.15.42:0/2655385820 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x5620936809f0 0x562093a84eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:36.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.286+0000 7f8a3d4f5640 1 --2- 172.21.15.42:0/2655385820 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x5620936809f0 0x562093a84eb0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=11d0bca876e11622 server_cookie=ee272eb7e43f6f9c in_seq=0 out_seq=0 2024-08-30T09:00:36.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.286+0000 7f8a217fa640 10 client.15504 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:00:36.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.287+0000 7f8a217fa640 1 -- 172.21.15.42:0/2655385820 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5620939e0580 con 0x5620936809f0 2024-08-30T09:00:36.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.287+0000 7f8a217fa640 10 client.15504 handle_command_reply: tid=0 2024-08-30T09:00:36.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.287+0000 7f8a464cf740 1 client.15504 shutdown 2024-08-30T09:00:36.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.288+0000 7f8a464cf740 1 -- 172.21.15.42:0/2655385820 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5620936991b0 msgr2=0x562093a72a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:00:36.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.288+0000 7f8a464cf740 1 --2- 172.21.15.42:0/2655385820 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5620936991b0 0x562093a72a70 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f8a34012700 tx=0x7f8a34012bc0 comp rx=0 tx=0).stop 2024-08-30T09:00:36.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.288+0000 7f8a464cf740 1 -- 172.21.15.42:0/2655385820 shutdown_connections 2024-08-30T09:00:36.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.288+0000 7f8a464cf740 1 --2- 172.21.15.42:0/2655385820 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x5620936809f0 0x562093a84eb0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:36.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.288+0000 7f8a464cf740 1 --2- 172.21.15.42:0/2655385820 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5620936a1390 0x562093a72fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:36.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.288+0000 7f8a464cf740 1 --2- 172.21.15.42:0/2655385820 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5620936c2070 0x562093a7ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:36.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.288+0000 7f8a464cf740 1 --2- 172.21.15.42:0/2655385820 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5620936991b0 0x562093a72a70 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:36.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.288+0000 7f8a217fa640 0 client.15504 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:00:36.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.288+0000 7f8a464cf740 1 -- 172.21.15.42:0/2655385820 >> 172.21.15.42:0/2655385820 conn(0x5620936b6460 msgr2=0x562093a6a450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:00:36.294 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.288+0000 7f8a464cf740 1 -- 172.21.15.42:0/2655385820 shutdown_connections 2024-08-30T09:00:36.294 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.288+0000 7f8a464cf740 1 -- 172.21.15.42:0/2655385820 wait complete. 2024-08-30T09:00:36.294 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.288+0000 7f8a464cf740 20 client.15504 trim_cache size 0 max 16384 2024-08-30T09:00:36.308 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.306+0000 7f8a464cf740 1 Processor -- start 2024-08-30T09:00:36.308 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.306+0000 7f8a464cf740 1 -- start start 2024-08-30T09:00:36.308 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.306+0000 7f8a464cf740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5620936991b0 0x562093a73050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:36.308 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.306+0000 7f8a464cf740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5620936a1390 0x562093a6a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:36.308 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.306+0000 7f8a464cf740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5620936c2070 0x562093a6a900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:36.308 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.306+0000 7f8a464cf740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f8a381b3180 con 0x5620936a1390 2024-08-30T09:00:36.308 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.306+0000 7f8a464cf740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f8a381b0a50 con 0x5620936c2070 2024-08-30T09:00:36.308 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.306+0000 7f8a464cf740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f8a381ae8a0 con 0x5620936991b0 2024-08-30T09:00:36.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.306+0000 7f8a3d4f5640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5620936a1390 0x562093a6a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:36.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.306+0000 7f8a3d4f5640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5620936a1390 0x562093a6a3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:51612/0 (socket says 172.21.15.42:51612) 2024-08-30T09:00:36.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.306+0000 7f8a3d4f5640 1 -- 172.21.15.42:0/270811269 learned_addr learned my addr 172.21.15.42:0/270811269 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:00:36.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.306+0000 7f8a3e4f7640 1 --2- 172.21.15.42:0/270811269 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5620936c2070 0x562093a6a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:36.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.306+0000 7f8a3dcf6640 1 --2- 172.21.15.42:0/270811269 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5620936991b0 0x562093a73050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:36.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.306+0000 7f8a3d4f5640 1 -- 172.21.15.42:0/270811269 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5620936991b0 msgr2=0x562093a73050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:00:36.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.306+0000 7f8a3d4f5640 1 --2- 172.21.15.42:0/270811269 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5620936991b0 0x562093a73050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:36.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.306+0000 7f8a3d4f5640 1 -- 172.21.15.42:0/270811269 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5620936c2070 msgr2=0x562093a6a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:00:36.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.306+0000 7f8a3d4f5640 1 --2- 172.21.15.42:0/270811269 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5620936c2070 0x562093a6a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:36.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.306+0000 7f8a3d4f5640 1 -- 172.21.15.42:0/270811269 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562093a695b0 con 0x5620936a1390 2024-08-30T09:00:36.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.306+0000 7f8a3dcf6640 1 --2- 172.21.15.42:0/270811269 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5620936991b0 0x562093a73050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-30T09:00:36.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.307+0000 7f8a3d4f5640 1 --2- 172.21.15.42:0/270811269 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5620936a1390 0x562093a6a3c0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f8a280e2f70 tx=0x7f8a28000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1797eb888f7148cf server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:00:36.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.307+0000 7f8a237fe640 1 -- 172.21.15.42:0/270811269 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a280ace60 con 0x5620936a1390 2024-08-30T09:00:36.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.307+0000 7f8a237fe640 1 -- 172.21.15.42:0/270811269 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a280ab640 con 0x5620936a1390 2024-08-30T09:00:36.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.307+0000 7f8a237fe640 1 -- 172.21.15.42:0/270811269 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a280ab2a0 con 0x5620936a1390 2024-08-30T09:00:36.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.308+0000 7f8a464cf740 1 -- 172.21.15.42:0/270811269 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5620936a1390 msgr2=0x562093a6a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:00:36.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.308+0000 7f8a464cf740 1 --2- 172.21.15.42:0/270811269 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5620936a1390 0x562093a6a3c0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f8a280e2f70 tx=0x7f8a28000bc0 comp rx=0 tx=0).stop 2024-08-30T09:00:36.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.308+0000 7f8a464cf740 1 -- 172.21.15.42:0/270811269 shutdown_connections 2024-08-30T09:00:36.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.308+0000 7f8a464cf740 1 --2- 172.21.15.42:0/270811269 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5620936c2070 0x562093a6a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:36.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.308+0000 7f8a464cf740 1 --2- 172.21.15.42:0/270811269 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5620936a1390 0x562093a6a3c0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:36.313 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.308+0000 7f8a464cf740 1 --2- 172.21.15.42:0/270811269 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5620936991b0 0x562093a73050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:36.313 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.308+0000 7f8a464cf740 1 -- 172.21.15.42:0/270811269 >> 172.21.15.42:0/270811269 conn(0x5620936b6460 msgr2=0x562093a712f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:00:36.313 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.308+0000 7f8a464cf740 1 -- 172.21.15.42:0/270811269 shutdown_connections 2024-08-30T09:00:36.313 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.308+0000 7f8a464cf740 1 -- 172.21.15.42:0/270811269 wait complete. 2024-08-30T09:00:36.313 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.308+0000 7f8a464cf740 1 -- start start 2024-08-30T09:00:36.313 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.308+0000 7f8a464cf740 1 Processor -- start 2024-08-30T09:00:36.313 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.309+0000 7f8a464cf740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5620936991b0 0x562093a7b680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:36.313 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.309+0000 7f8a464cf740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5620936a1390 0x562093a76b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:36.313 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.309+0000 7f8a464cf740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5620936c2070 0x562093a77220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:36.313 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.309+0000 7f8a464cf740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f8a381aef10 con 0x5620936c2070 2024-08-30T09:00:36.313 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.309+0000 7f8a464cf740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f8a381b0380 con 0x5620936991b0 2024-08-30T09:00:36.313 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.309+0000 7f8a464cf740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f8a381af5e0 con 0x5620936a1390 2024-08-30T09:00:36.313 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.309+0000 7f8a464cf740 10 client.0 _ll_register_callbacks cb 0x7f8a34009d80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:00:36.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.309+0000 7f8a3e4f7640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5620936c2070 0x562093a77220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:36.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.309+0000 7f8a3e4f7640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5620936c2070 0x562093a77220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:51624/0 (socket says 172.21.15.42:51624) 2024-08-30T09:00:36.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.309+0000 7f8a3dcf6640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5620936991b0 0x562093a7b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:36.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.309+0000 7f8a3e4f7640 1 -- 172.21.15.42:0/2934199066 learned_addr learned my addr 172.21.15.42:0/2934199066 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:00:36.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.309+0000 7f8a3d4f5640 1 --2- 172.21.15.42:0/2934199066 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5620936a1390 0x562093a76b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:36.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.309+0000 7f8a3e4f7640 1 -- 172.21.15.42:0/2934199066 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5620936a1390 msgr2=0x562093a76b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:00:36.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.309+0000 7f8a3e4f7640 1 --2- 172.21.15.42:0/2934199066 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5620936a1390 0x562093a76b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:36.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.309+0000 7f8a3e4f7640 1 -- 172.21.15.42:0/2934199066 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5620936991b0 msgr2=0x562093a7b680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:00:36.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.309+0000 7f8a3e4f7640 1 --2- 172.21.15.42:0/2934199066 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5620936991b0 0x562093a7b680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:36.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.309+0000 7f8a3e4f7640 1 -- 172.21.15.42:0/2934199066 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562093aee8b0 con 0x5620936c2070 2024-08-30T09:00:36.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.309+0000 7f8a3dcf6640 1 --2- 172.21.15.42:0/2934199066 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5620936991b0 0x562093a7b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-30T09:00:36.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.310+0000 7f8a3e4f7640 1 --2- 172.21.15.42:0/2934199066 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5620936c2070 0x562093a77220 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f8a30003b80 tx=0x7f8a300095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73d6ca22e588162f server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:00:36.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.310+0000 7f8a217fa640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-08-30T09:00:36.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.310+0000 7f8a217fa640 1 -- 172.21.15.42:0/2934199066 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a30059060 con 0x5620936c2070 2024-08-30T09:00:36.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.310+0000 7f8a464cf740 1 -- 172.21.15.42:0/2934199066 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a280ab640 con 0x5620936c2070 2024-08-30T09:00:36.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.310+0000 7f8a217fa640 1 -- 172.21.15.42:0/2934199066 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f8a300045c0 con 0x5620936c2070 2024-08-30T09:00:36.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.310+0000 7f8a217fa640 1 -- 172.21.15.42:0/2934199066 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f8a300047c0 con 0x5620936c2070 2024-08-30T09:00:36.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.310+0000 7f8a464cf740 10 client.15510 fetch_fsmap learned FSMap version 27 2024-08-30T09:00:36.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.310+0000 7f8a464cf740 1 -- 172.21.15.42:0/2934199066 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x562093aef920 con 0x5620936c2070 2024-08-30T09:00:36.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.310+0000 7f8a217fa640 1 -- 172.21.15.42:0/2934199066 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a280ab640 con 0x5620936c2070 2024-08-30T09:00:36.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.310+0000 7f8a217fa640 1 -- 172.21.15.42:0/2934199066 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f8a30030830 con 0x5620936c2070 2024-08-30T09:00:36.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.311+0000 7f8a464cf740 10 client.15510 fetch_fsmap finished waiting for FSMap version 27 2024-08-30T09:00:36.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.311+0000 7f8a464cf740 10 client.15510 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-30T09:00:36.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.311+0000 7f8a464cf740 20 client.15510 populate_metadata read hostname 'smithi042' 2024-08-30T09:00:36.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.311+0000 7f8a464cf740 1 --2- 172.21.15.42:0/2934199066 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x562093655da0 0x562093af2140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:36.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.311+0000 7f8a464cf740 4 client.15510 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-30T09:00:36.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.311+0000 7f8a464cf740 1 -- 172.21.15.42:0/2934199066 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5620939dc020 con 0x562093655da0 2024-08-30T09:00:36.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.311+0000 7f8a3dcf6640 1 --2- 172.21.15.42:0/2934199066 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x562093655da0 0x562093af2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:36.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.311+0000 7f8a217fa640 10 client.15510 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:00:36.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.311+0000 7f8a3dcf6640 1 --2- 172.21.15.42:0/2934199066 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x562093655da0 0x562093af2140 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b631f998bd2e394e server_cookie=a0079149218b0b71 in_seq=0 out_seq=0 2024-08-30T09:00:36.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.312+0000 7f8a217fa640 1 -- 172.21.15.42:0/2934199066 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+570 (crc 0 0 0) 0x5620939dc020 con 0x562093655da0 2024-08-30T09:00:36.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.312+0000 7f8a217fa640 10 client.15510 handle_command_reply: tid=0 2024-08-30T09:00:36.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.312+0000 7f8a464cf740 1 client.15510 shutdown 2024-08-30T09:00:36.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.312+0000 7f8a464cf740 1 -- 172.21.15.42:0/2934199066 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5620936c2070 msgr2=0x562093a77220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:00:36.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.312+0000 7f8a464cf740 1 --2- 172.21.15.42:0/2934199066 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5620936c2070 0x562093a77220 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f8a30003b80 tx=0x7f8a300095c0 comp rx=0 tx=0).stop 2024-08-30T09:00:36.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.312+0000 7f8a464cf740 1 -- 172.21.15.42:0/2934199066 shutdown_connections 2024-08-30T09:00:36.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.312+0000 7f8a464cf740 1 --2- 172.21.15.42:0/2934199066 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x562093655da0 0x562093af2140 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:36.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.312+0000 7f8a464cf740 1 --2- 172.21.15.42:0/2934199066 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5620936a1390 0x562093a76b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:36.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.312+0000 7f8a464cf740 1 --2- 172.21.15.42:0/2934199066 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5620936c2070 0x562093a77220 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:36.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.312+0000 7f8a217fa640 0 client.15510 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:00:36.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.312+0000 7f8a464cf740 1 --2- 172.21.15.42:0/2934199066 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5620936991b0 0x562093a7b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:36.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.312+0000 7f8a464cf740 1 -- 172.21.15.42:0/2934199066 >> 172.21.15.42:0/2934199066 conn(0x5620936b6460 msgr2=0x562093a6a3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:00:36.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.312+0000 7f8a464cf740 1 -- 172.21.15.42:0/2934199066 shutdown_connections 2024-08-30T09:00:36.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.312+0000 7f8a464cf740 1 -- 172.21.15.42:0/2934199066 wait complete. 2024-08-30T09:00:36.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.312+0000 7f8a464cf740 20 client.15510 trim_cache size 0 max 16384 2024-08-30T09:00:36.320 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-08-30T09:00:36.320 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "IDLE (waiting for more scrubs)", 2024-08-30T09:00:36.321 INFO:teuthology.orchestra.run.smithi042.stdout: "scrubs": { 2024-08-30T09:00:36.321 INFO:teuthology.orchestra.run.smithi042.stdout: "5e2ed104-0625-478a-a78b-584e7f4c2817": { 2024-08-30T09:00:36.321 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:00:36.321 INFO:teuthology.orchestra.run.smithi042.stdout: "tag": "5e2ed104-0625-478a-a78b-584e7f4c2817", 2024-08-30T09:00:36.321 INFO:teuthology.orchestra.run.smithi042.stdout: "options": "recursive,force" 2024-08-30T09:00:36.321 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:00:36.321 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:00:36.321 INFO:teuthology.orchestra.run.smithi042.stdout: "scrub_stats": { 2024-08-30T09:00:36.321 INFO:teuthology.orchestra.run.smithi042.stdout: "5e2ed104-0625-478a-a78b-584e7f4c2817": { 2024-08-30T09:00:36.321 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:00:33.928400+0000", 2024-08-30T09:00:36.321 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:00:36.321 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:00:36.321 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:00:36.322 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:00:36.322 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:00:36.322 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:00:36.322 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:00:36.322 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-08-30T09:00:36.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.313+0000 7f8a3ff5a640 1 -- 172.21.15.42:0/2973488386 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f8a240b4b60 msgr2=0x7f8a240b7070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:00:36.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.313+0000 7f8a3ff5a640 1 --2- 172.21.15.42:0/2973488386 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f8a240b4b60 0x7f8a240b7070 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f8a300094e0 tx=0x7f8a30005c50 comp rx=0 tx=0).stop 2024-08-30T09:00:36.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.313+0000 7f8a3ff5a640 1 -- 172.21.15.42:0/2973488386 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8a38108140 msgr2=0x7f8a3819a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:00:36.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.313+0000 7f8a3ff5a640 1 --2- 172.21.15.42:0/2973488386 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8a38108140 0x7f8a3819a7d0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f8a2800c940 tx=0x7f8a2800ce00 comp rx=0 tx=0).stop 2024-08-30T09:00:36.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.313+0000 7f8a3ff5a640 1 -- 172.21.15.42:0/2973488386 shutdown_connections 2024-08-30T09:00:36.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.313+0000 7f8a3ff5a640 1 --2- 172.21.15.42:0/2973488386 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f8a240b4b60 0x7f8a240b7070 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:36.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.313+0000 7f8a3ff5a640 1 --2- 172.21.15.42:0/2973488386 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8a38108a90 0x7f8a3819fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:36.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.313+0000 7f8a3ff5a640 1 --2- 172.21.15.42:0/2973488386 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8a38108140 0x7f8a3819a7d0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:36.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.313+0000 7f8a3ff5a640 1 --2- 172.21.15.42:0/2973488386 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8a38102800 0x7f8a3819a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:36.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.313+0000 7f8a3ff5a640 1 -- 172.21.15.42:0/2973488386 >> 172.21.15.42:0/2973488386 conn(0x7f8a380794d0 msgr2=0x7f8a3806af40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:00:36.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.313+0000 7f8a3ff5a640 1 -- 172.21.15.42:0/2973488386 shutdown_connections 2024-08-30T09:00:36.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:00:36.313+0000 7f8a3ff5a640 1 -- 172.21.15.42:0/2973488386 wait complete. 2024-08-30T09:00:36.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:00:36 smithi042 ceph-mon[18652]: pgmap v368: 129 pgs: 129 active+clean; 650 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 767 B/s wr, 5 op/s 2024-08-30T09:00:36.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:00:36 smithi190 ceph-mon[22001]: pgmap v368: 129 pgs: 129 active+clean; 650 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 767 B/s wr, 5 op/s 2024-08-30T09:00:36.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:00:36 smithi167 ceph-mon[22063]: pgmap v368: 129 pgs: 129 active+clean; 650 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 767 B/s wr, 5 op/s 2024-08-30T09:00:37.088 INFO:tasks.cephfs.filesystem:scrub status for tag:5e2ed104-0625-478a-a78b-584e7f4c2817 - {'path': '/', 'tag': '5e2ed104-0625-478a-a78b-584e7f4c2817', 'options': 'recursive,force'} 2024-08-30T09:00:38.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:00:38 smithi190 ceph-mon[22001]: pgmap v369: 129 pgs: 129 active+clean; 650 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 2.1 KiB/s wr, 6 op/s 2024-08-30T09:00:38.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:00:38 smithi167 ceph-mon[22063]: pgmap v369: 129 pgs: 129 active+clean; 650 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 2.1 KiB/s wr, 6 op/s 2024-08-30T09:00:38.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:00:38 smithi042 ceph-mon[18652]: pgmap v369: 129 pgs: 129 active+clean; 650 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 2.1 KiB/s wr, 6 op/s 2024-08-30T09:00:39.574 INFO:teuthology.orchestra.run.smithi042.stderr: * Initialized, logs are in /var/lib/pgsql/initdb_postgresql.log 2024-08-30T09:00:39.577 DEBUG:teuthology.orchestra.run.smithi042:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -lZaR /tmp/cephfs/postgres/' 2024-08-30T09:00:39.618 INFO:teuthology.orchestra.run.smithi042.stdout:/tmp/cephfs/postgres/: 2024-08-30T09:00:39.618 INFO:teuthology.orchestra.run.smithi042.stdout:total 0 2024-08-30T09:00:39.618 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Aug 30 09:00 . 2024-08-30T09:00:39.618 INFO:teuthology.orchestra.run.smithi042.stdout:drwxrwxrwt. 3 root root system_u:object_r:postgresql_db_t:s0 1 Aug 30 09:00 .. 2024-08-30T09:00:39.618 INFO:teuthology.orchestra.run.smithi042.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Aug 30 09:00 data 2024-08-30T09:00:39.618 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-08-30T09:00:39.618 INFO:teuthology.orchestra.run.smithi042.stdout:/tmp/cephfs/postgres/data: 2024-08-30T09:00:39.618 INFO:teuthology.orchestra.run.smithi042.stdout:total 35 2024-08-30T09:00:39.618 INFO:teuthology.orchestra.run.smithi042.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Aug 30 09:00 . 2024-08-30T09:00:39.618 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Aug 30 09:00 .. 2024-08-30T09:00:39.619 INFO:teuthology.orchestra.run.smithi042.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Aug 30 09:00 base 2024-08-30T09:00:39.619 INFO:teuthology.orchestra.run.smithi042.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 57 Aug 30 09:00 global 2024-08-30T09:00:39.619 INFO:teuthology.orchestra.run.smithi042.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 log 2024-08-30T09:00:39.619 INFO:teuthology.orchestra.run.smithi042.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 pg_commit_ts 2024-08-30T09:00:39.619 INFO:teuthology.orchestra.run.smithi042.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 pg_dynshmem 2024-08-30T09:00:39.619 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 4516 Aug 30 09:00 pg_hba.conf 2024-08-30T09:00:39.619 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1636 Aug 30 09:00 pg_ident.conf 2024-08-30T09:00:39.619 INFO:teuthology.orchestra.run.smithi042.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Aug 30 09:00 pg_logical 2024-08-30T09:00:39.619 INFO:teuthology.orchestra.run.smithi042.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Aug 30 09:00 pg_multixact 2024-08-30T09:00:39.619 INFO:teuthology.orchestra.run.smithi042.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 pg_notify 2024-08-30T09:00:39.619 INFO:teuthology.orchestra.run.smithi042.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 pg_replslot 2024-08-30T09:00:39.619 INFO:teuthology.orchestra.run.smithi042.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 pg_serial 2024-08-30T09:00:39.619 INFO:teuthology.orchestra.run.smithi042.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 pg_snapshots 2024-08-30T09:00:39.619 INFO:teuthology.orchestra.run.smithi042.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 pg_stat 2024-08-30T09:00:39.620 INFO:teuthology.orchestra.run.smithi042.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 pg_stat_tmp 2024-08-30T09:00:39.620 INFO:teuthology.orchestra.run.smithi042.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Aug 30 09:00 pg_subtrans 2024-08-30T09:00:39.620 INFO:teuthology.orchestra.run.smithi042.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 pg_tblspc 2024-08-30T09:00:39.620 INFO:teuthology.orchestra.run.smithi042.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 pg_twophase 2024-08-30T09:00:39.620 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Aug 30 09:00 PG_VERSION 2024-08-30T09:00:39.620 INFO:teuthology.orchestra.run.smithi042.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Aug 30 09:00 pg_wal 2024-08-30T09:00:39.620 INFO:teuthology.orchestra.run.smithi042.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Aug 30 09:00 pg_xact 2024-08-30T09:00:39.620 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 88 Aug 30 09:00 postgresql.auto.conf 2024-08-30T09:00:39.620 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 28074 Aug 30 09:00 postgresql.conf 2024-08-30T09:00:39.620 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-08-30T09:00:39.620 INFO:teuthology.orchestra.run.smithi042.stdout:/tmp/cephfs/postgres/data/base: 2024-08-30T09:00:39.620 INFO:teuthology.orchestra.run.smithi042.stdout:total 0 2024-08-30T09:00:39.620 INFO:teuthology.orchestra.run.smithi042.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Aug 30 09:00 . 2024-08-30T09:00:39.621 INFO:teuthology.orchestra.run.smithi042.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Aug 30 09:00 .. 2024-08-30T09:00:39.621 INFO:teuthology.orchestra.run.smithi042.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Aug 30 09:00 1 2024-08-30T09:00:39.621 INFO:teuthology.orchestra.run.smithi042.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Aug 30 09:00 13493 2024-08-30T09:00:39.621 INFO:teuthology.orchestra.run.smithi042.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Aug 30 09:00 13494 2024-08-30T09:00:39.621 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-08-30T09:00:39.621 INFO:teuthology.orchestra.run.smithi042.stdout:/tmp/cephfs/postgres/data/base/1: 2024-08-30T09:00:39.621 INFO:teuthology.orchestra.run.smithi042.stdout:total 7777 2024-08-30T09:00:39.621 INFO:teuthology.orchestra.run.smithi042.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Aug 30 09:00 . 2024-08-30T09:00:39.621 INFO:teuthology.orchestra.run.smithi042.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Aug 30 09:00 .. 2024-08-30T09:00:39.621 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 112 2024-08-30T09:00:39.621 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 113 2024-08-30T09:00:39.621 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Aug 30 09:00 1247 2024-08-30T09:00:39.621 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 1247_fsm 2024-08-30T09:00:39.621 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 1247_vm 2024-08-30T09:00:39.622 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 Aug 30 09:00 1249 2024-08-30T09:00:39.622 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 1249_fsm 2024-08-30T09:00:39.622 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 1249_vm 2024-08-30T09:00:39.622 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 Aug 30 09:00 1255 2024-08-30T09:00:39.622 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 1255_fsm 2024-08-30T09:00:39.622 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 1255_vm 2024-08-30T09:00:39.622 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Aug 30 09:00 1259 2024-08-30T09:00:39.622 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 1259_fsm 2024-08-30T09:00:39.622 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 1259_vm 2024-08-30T09:00:39.622 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 Aug 30 09:00 13344 2024-08-30T09:00:39.622 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 13344_fsm 2024-08-30T09:00:39.622 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 13344_vm 2024-08-30T09:00:39.622 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 13346 2024-08-30T09:00:39.622 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 13348 2024-08-30T09:00:39.622 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 13349 2024-08-30T09:00:39.623 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 13349_fsm 2024-08-30T09:00:39.623 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 13349_vm 2024-08-30T09:00:39.623 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 13351 2024-08-30T09:00:39.623 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 13353 2024-08-30T09:00:39.623 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 13354 2024-08-30T09:00:39.623 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 13354_fsm 2024-08-30T09:00:39.623 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 13354_vm 2024-08-30T09:00:39.623 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 13356 2024-08-30T09:00:39.623 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 13358 2024-08-30T09:00:39.623 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 13359 2024-08-30T09:00:39.623 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 13359_fsm 2024-08-30T09:00:39.623 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 13359_vm 2024-08-30T09:00:39.623 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 13361 2024-08-30T09:00:39.623 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 13363 2024-08-30T09:00:39.624 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 1417 2024-08-30T09:00:39.624 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 1418 2024-08-30T09:00:39.624 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 174 2024-08-30T09:00:39.624 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 175 2024-08-30T09:00:39.624 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2187 2024-08-30T09:00:39.624 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 2224 2024-08-30T09:00:39.624 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 2328 2024-08-30T09:00:39.624 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 2336 2024-08-30T09:00:39.624 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2337 2024-08-30T09:00:39.624 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2579 2024-08-30T09:00:39.624 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2600 2024-08-30T09:00:39.624 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 2600_fsm 2024-08-30T09:00:39.624 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2600_vm 2024-08-30T09:00:39.625 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2601 2024-08-30T09:00:39.625 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 2601_fsm 2024-08-30T09:00:39.625 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2601_vm 2024-08-30T09:00:39.625 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Aug 30 09:00 2602 2024-08-30T09:00:39.625 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 2602_fsm 2024-08-30T09:00:39.625 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2602_vm 2024-08-30T09:00:39.625 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 30 09:00 2603 2024-08-30T09:00:39.625 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 2603_fsm 2024-08-30T09:00:39.625 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2603_vm 2024-08-30T09:00:39.625 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 2604 2024-08-30T09:00:39.625 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2605 2024-08-30T09:00:39.625 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 2605_fsm 2024-08-30T09:00:39.625 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2605_vm 2024-08-30T09:00:39.625 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2606 2024-08-30T09:00:39.626 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 2606_fsm 2024-08-30T09:00:39.626 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2606_vm 2024-08-30T09:00:39.626 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2607 2024-08-30T09:00:39.626 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 2607_fsm 2024-08-30T09:00:39.626 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2607_vm 2024-08-30T09:00:39.626 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 Aug 30 09:00 2608 2024-08-30T09:00:39.626 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 2608_fsm 2024-08-30T09:00:39.626 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2608_vm 2024-08-30T09:00:39.626 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 Aug 30 09:00 2609 2024-08-30T09:00:39.626 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 2609_fsm 2024-08-30T09:00:39.626 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2609_vm 2024-08-30T09:00:39.626 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 30 09:00 2610 2024-08-30T09:00:39.626 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 2610_fsm 2024-08-30T09:00:39.626 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2610_vm 2024-08-30T09:00:39.627 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 2611 2024-08-30T09:00:39.627 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2612 2024-08-30T09:00:39.627 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 2612_fsm 2024-08-30T09:00:39.627 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2612_vm 2024-08-30T09:00:39.627 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 2613 2024-08-30T09:00:39.627 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2615 2024-08-30T09:00:39.627 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 2615_fsm 2024-08-30T09:00:39.627 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2615_vm 2024-08-30T09:00:39.627 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2616 2024-08-30T09:00:39.627 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 2616_fsm 2024-08-30T09:00:39.627 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2616_vm 2024-08-30T09:00:39.627 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 Aug 30 09:00 2617 2024-08-30T09:00:39.627 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 2617_fsm 2024-08-30T09:00:39.628 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2617_vm 2024-08-30T09:00:39.628 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Aug 30 09:00 2618 2024-08-30T09:00:39.628 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 2618_fsm 2024-08-30T09:00:39.628 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2618_vm 2024-08-30T09:00:39.628 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 147456 Aug 30 09:00 2619 2024-08-30T09:00:39.628 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 2619_fsm 2024-08-30T09:00:39.628 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2619_vm 2024-08-30T09:00:39.628 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 2620 2024-08-30T09:00:39.628 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2650 2024-08-30T09:00:39.628 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2651 2024-08-30T09:00:39.628 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2652 2024-08-30T09:00:39.628 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Aug 30 09:00 2653 2024-08-30T09:00:39.628 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Aug 30 09:00 2654 2024-08-30T09:00:39.629 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 30 09:00 2655 2024-08-30T09:00:39.629 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2656 2024-08-30T09:00:39.629 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2657 2024-08-30T09:00:39.629 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Aug 30 09:00 2658 2024-08-30T09:00:39.629 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Aug 30 09:00 2659 2024-08-30T09:00:39.629 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2660 2024-08-30T09:00:39.629 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2661 2024-08-30T09:00:39.629 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 30 09:00 2662 2024-08-30T09:00:39.629 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Aug 30 09:00 2663 2024-08-30T09:00:39.629 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2664 2024-08-30T09:00:39.629 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2665 2024-08-30T09:00:39.629 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2666 2024-08-30T09:00:39.629 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2667 2024-08-30T09:00:39.629 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2668 2024-08-30T09:00:39.629 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2669 2024-08-30T09:00:39.630 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2670 2024-08-30T09:00:39.630 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 Aug 30 09:00 2673 2024-08-30T09:00:39.630 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 Aug 30 09:00 2674 2024-08-30T09:00:39.630 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 Aug 30 09:00 2675 2024-08-30T09:00:39.630 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2678 2024-08-30T09:00:39.630 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2679 2024-08-30T09:00:39.630 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2680 2024-08-30T09:00:39.630 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2681 2024-08-30T09:00:39.630 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2682 2024-08-30T09:00:39.630 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2683 2024-08-30T09:00:39.630 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2684 2024-08-30T09:00:39.630 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2685 2024-08-30T09:00:39.630 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2686 2024-08-30T09:00:39.630 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2687 2024-08-30T09:00:39.630 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Aug 30 09:00 2688 2024-08-30T09:00:39.631 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Aug 30 09:00 2689 2024-08-30T09:00:39.631 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Aug 30 09:00 2690 2024-08-30T09:00:39.631 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 Aug 30 09:00 2691 2024-08-30T09:00:39.631 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2692 2024-08-30T09:00:39.631 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2693 2024-08-30T09:00:39.631 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2696 2024-08-30T09:00:39.631 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2699 2024-08-30T09:00:39.631 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2701 2024-08-30T09:00:39.631 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2702 2024-08-30T09:00:39.631 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 30 09:00 2703 2024-08-30T09:00:39.631 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Aug 30 09:00 2704 2024-08-30T09:00:39.631 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2753 2024-08-30T09:00:39.631 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 2753_fsm 2024-08-30T09:00:39.631 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2753_vm 2024-08-30T09:00:39.632 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2754 2024-08-30T09:00:39.632 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2755 2024-08-30T09:00:39.632 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 30 09:00 2756 2024-08-30T09:00:39.632 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 30 09:00 2757 2024-08-30T09:00:39.632 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 2830 2024-08-30T09:00:39.632 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2831 2024-08-30T09:00:39.632 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 2832 2024-08-30T09:00:39.632 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2833 2024-08-30T09:00:39.632 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 2834 2024-08-30T09:00:39.632 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2835 2024-08-30T09:00:39.632 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 2836 2024-08-30T09:00:39.632 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2837 2024-08-30T09:00:39.632 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 Aug 30 09:00 2838 2024-08-30T09:00:39.633 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 2838_fsm 2024-08-30T09:00:39.633 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2838_vm 2024-08-30T09:00:39.633 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2839 2024-08-30T09:00:39.633 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 2840 2024-08-30T09:00:39.633 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 2840_fsm 2024-08-30T09:00:39.633 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2840_vm 2024-08-30T09:00:39.633 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2841 2024-08-30T09:00:39.633 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 2995 2024-08-30T09:00:39.633 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2996 2024-08-30T09:00:39.633 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3079 2024-08-30T09:00:39.633 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 3079_fsm 2024-08-30T09:00:39.633 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3079_vm 2024-08-30T09:00:39.633 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 3080 2024-08-30T09:00:39.633 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 3081 2024-08-30T09:00:39.634 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Aug 30 09:00 3085 2024-08-30T09:00:39.634 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 3118 2024-08-30T09:00:39.634 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3119 2024-08-30T09:00:39.634 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Aug 30 09:00 3164 2024-08-30T09:00:39.634 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 3256 2024-08-30T09:00:39.634 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3257 2024-08-30T09:00:39.634 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3258 2024-08-30T09:00:39.634 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 3350 2024-08-30T09:00:39.634 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3351 2024-08-30T09:00:39.634 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3379 2024-08-30T09:00:39.634 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3380 2024-08-30T09:00:39.634 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 3381 2024-08-30T09:00:39.634 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 3394 2024-08-30T09:00:39.634 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 3394_fsm 2024-08-30T09:00:39.634 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3394_vm 2024-08-30T09:00:39.635 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 3395 2024-08-30T09:00:39.635 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 3429 2024-08-30T09:00:39.635 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 3430 2024-08-30T09:00:39.635 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3431 2024-08-30T09:00:39.635 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3433 2024-08-30T09:00:39.635 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 3439 2024-08-30T09:00:39.635 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3440 2024-08-30T09:00:39.635 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Aug 30 09:00 3455 2024-08-30T09:00:39.635 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 Aug 30 09:00 3456 2024-08-30T09:00:39.635 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 3456_fsm 2024-08-30T09:00:39.635 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3456_vm 2024-08-30T09:00:39.635 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 3466 2024-08-30T09:00:39.635 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3467 2024-08-30T09:00:39.635 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3468 2024-08-30T09:00:39.636 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 3501 2024-08-30T09:00:39.636 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3502 2024-08-30T09:00:39.636 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3503 2024-08-30T09:00:39.636 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3534 2024-08-30T09:00:39.636 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3541 2024-08-30T09:00:39.636 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 3541_fsm 2024-08-30T09:00:39.636 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3541_vm 2024-08-30T09:00:39.636 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 3542 2024-08-30T09:00:39.636 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3574 2024-08-30T09:00:39.636 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3575 2024-08-30T09:00:39.636 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 3576 2024-08-30T09:00:39.636 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 3596 2024-08-30T09:00:39.636 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3597 2024-08-30T09:00:39.636 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 3598 2024-08-30T09:00:39.636 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3599 2024-08-30T09:00:39.637 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3600 2024-08-30T09:00:39.637 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 3600_fsm 2024-08-30T09:00:39.637 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3600_vm 2024-08-30T09:00:39.637 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3601 2024-08-30T09:00:39.637 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 3601_fsm 2024-08-30T09:00:39.637 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3601_vm 2024-08-30T09:00:39.637 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3602 2024-08-30T09:00:39.637 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 3602_fsm 2024-08-30T09:00:39.637 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3602_vm 2024-08-30T09:00:39.637 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 3603 2024-08-30T09:00:39.637 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 3603_fsm 2024-08-30T09:00:39.637 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3603_vm 2024-08-30T09:00:39.637 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 3604 2024-08-30T09:00:39.638 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 3605 2024-08-30T09:00:39.638 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 3606 2024-08-30T09:00:39.638 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 3607 2024-08-30T09:00:39.638 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 3608 2024-08-30T09:00:39.638 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 30 09:00 3609 2024-08-30T09:00:39.638 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 3712 2024-08-30T09:00:39.638 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3764 2024-08-30T09:00:39.638 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 3764_fsm 2024-08-30T09:00:39.638 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3764_vm 2024-08-30T09:00:39.638 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 3766 2024-08-30T09:00:39.638 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 3767 2024-08-30T09:00:39.638 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3997 2024-08-30T09:00:39.638 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 4143 2024-08-30T09:00:39.638 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 4144 2024-08-30T09:00:39.639 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 4145 2024-08-30T09:00:39.639 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 4146 2024-08-30T09:00:39.639 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 4147 2024-08-30T09:00:39.639 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 4148 2024-08-30T09:00:39.639 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 4149 2024-08-30T09:00:39.639 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 4150 2024-08-30T09:00:39.639 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 4151 2024-08-30T09:00:39.639 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 4152 2024-08-30T09:00:39.639 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 4153 2024-08-30T09:00:39.639 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 4154 2024-08-30T09:00:39.639 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 4155 2024-08-30T09:00:39.639 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 4156 2024-08-30T09:00:39.639 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 4157 2024-08-30T09:00:39.639 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 4158 2024-08-30T09:00:39.640 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 4159 2024-08-30T09:00:39.640 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 4160 2024-08-30T09:00:39.640 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 4161 2024-08-30T09:00:39.640 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 4162 2024-08-30T09:00:39.640 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 4163 2024-08-30T09:00:39.640 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 4164 2024-08-30T09:00:39.640 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 4165 2024-08-30T09:00:39.643 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 4166 2024-08-30T09:00:39.643 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 4167 2024-08-30T09:00:39.643 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 4168 2024-08-30T09:00:39.643 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 4169 2024-08-30T09:00:39.643 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 4170 2024-08-30T09:00:39.643 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 4171 2024-08-30T09:00:39.643 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 4172 2024-08-30T09:00:39.643 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 4173 2024-08-30T09:00:39.643 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 4174 2024-08-30T09:00:39.643 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 5002 2024-08-30T09:00:39.644 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 548 2024-08-30T09:00:39.644 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 549 2024-08-30T09:00:39.644 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 6102 2024-08-30T09:00:39.644 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 6104 2024-08-30T09:00:39.644 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 6106 2024-08-30T09:00:39.644 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 6110 2024-08-30T09:00:39.644 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 6111 2024-08-30T09:00:39.644 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 6112 2024-08-30T09:00:39.644 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 6113 2024-08-30T09:00:39.644 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 6117 2024-08-30T09:00:39.644 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 826 2024-08-30T09:00:39.644 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 827 2024-08-30T09:00:39.644 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 828 2024-08-30T09:00:39.645 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Aug 30 09:00 pg_filenode.map 2024-08-30T09:00:39.645 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Aug 30 09:00 PG_VERSION 2024-08-30T09:00:39.645 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-08-30T09:00:39.645 INFO:teuthology.orchestra.run.smithi042.stdout:/tmp/cephfs/postgres/data/base/13493: 2024-08-30T09:00:39.645 INFO:teuthology.orchestra.run.smithi042.stdout:total 7777 2024-08-30T09:00:39.645 INFO:teuthology.orchestra.run.smithi042.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Aug 30 09:00 . 2024-08-30T09:00:39.645 INFO:teuthology.orchestra.run.smithi042.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Aug 30 09:00 .. 2024-08-30T09:00:39.645 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 112 2024-08-30T09:00:39.645 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 113 2024-08-30T09:00:39.645 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Aug 30 09:00 1247 2024-08-30T09:00:39.645 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 1247_fsm 2024-08-30T09:00:39.645 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 1247_vm 2024-08-30T09:00:39.645 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 Aug 30 09:00 1249 2024-08-30T09:00:39.646 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 1249_fsm 2024-08-30T09:00:39.646 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 1249_vm 2024-08-30T09:00:39.646 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 Aug 30 09:00 1255 2024-08-30T09:00:39.646 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 1255_fsm 2024-08-30T09:00:39.646 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 1255_vm 2024-08-30T09:00:39.646 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Aug 30 09:00 1259 2024-08-30T09:00:39.646 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 1259_fsm 2024-08-30T09:00:39.646 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 1259_vm 2024-08-30T09:00:39.646 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 Aug 30 09:00 13344 2024-08-30T09:00:39.646 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 13344_fsm 2024-08-30T09:00:39.646 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 13344_vm 2024-08-30T09:00:39.646 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 13346 2024-08-30T09:00:39.646 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 13348 2024-08-30T09:00:39.647 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 13349 2024-08-30T09:00:39.647 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 13349_fsm 2024-08-30T09:00:39.647 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 13349_vm 2024-08-30T09:00:39.647 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 13351 2024-08-30T09:00:39.647 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 13353 2024-08-30T09:00:39.647 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 13354 2024-08-30T09:00:39.647 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 13354_fsm 2024-08-30T09:00:39.647 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 13354_vm 2024-08-30T09:00:39.647 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 13356 2024-08-30T09:00:39.647 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 13358 2024-08-30T09:00:39.647 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 13359 2024-08-30T09:00:39.648 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 13359_fsm 2024-08-30T09:00:39.648 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 13359_vm 2024-08-30T09:00:39.648 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 13361 2024-08-30T09:00:39.648 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 13363 2024-08-30T09:00:39.648 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 1417 2024-08-30T09:00:39.648 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 1418 2024-08-30T09:00:39.648 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 174 2024-08-30T09:00:39.648 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 175 2024-08-30T09:00:39.648 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2187 2024-08-30T09:00:39.648 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 2224 2024-08-30T09:00:39.648 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 2328 2024-08-30T09:00:39.648 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 2336 2024-08-30T09:00:39.648 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2337 2024-08-30T09:00:39.648 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2579 2024-08-30T09:00:39.649 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2600 2024-08-30T09:00:39.649 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 2600_fsm 2024-08-30T09:00:39.649 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2600_vm 2024-08-30T09:00:39.649 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2601 2024-08-30T09:00:39.649 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 2601_fsm 2024-08-30T09:00:39.649 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2601_vm 2024-08-30T09:00:39.649 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Aug 30 09:00 2602 2024-08-30T09:00:39.649 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 2602_fsm 2024-08-30T09:00:39.649 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2602_vm 2024-08-30T09:00:39.649 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 30 09:00 2603 2024-08-30T09:00:39.649 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 2603_fsm 2024-08-30T09:00:39.649 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2603_vm 2024-08-30T09:00:39.649 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 2604 2024-08-30T09:00:39.649 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2605 2024-08-30T09:00:39.650 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 2605_fsm 2024-08-30T09:00:39.650 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2605_vm 2024-08-30T09:00:39.650 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2606 2024-08-30T09:00:39.650 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 2606_fsm 2024-08-30T09:00:39.650 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2606_vm 2024-08-30T09:00:39.650 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2607 2024-08-30T09:00:39.650 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 2607_fsm 2024-08-30T09:00:39.650 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2607_vm 2024-08-30T09:00:39.650 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 Aug 30 09:00 2608 2024-08-30T09:00:39.650 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 2608_fsm 2024-08-30T09:00:39.650 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2608_vm 2024-08-30T09:00:39.650 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 Aug 30 09:00 2609 2024-08-30T09:00:39.650 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 2609_fsm 2024-08-30T09:00:39.651 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2609_vm 2024-08-30T09:00:39.651 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 30 09:00 2610 2024-08-30T09:00:39.651 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 2610_fsm 2024-08-30T09:00:39.651 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2610_vm 2024-08-30T09:00:39.651 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 2611 2024-08-30T09:00:39.651 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2612 2024-08-30T09:00:39.651 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 2612_fsm 2024-08-30T09:00:39.651 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2612_vm 2024-08-30T09:00:39.651 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 2613 2024-08-30T09:00:39.651 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2615 2024-08-30T09:00:39.651 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 2615_fsm 2024-08-30T09:00:39.651 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2615_vm 2024-08-30T09:00:39.651 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2616 2024-08-30T09:00:39.651 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 2616_fsm 2024-08-30T09:00:39.652 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2616_vm 2024-08-30T09:00:39.652 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 Aug 30 09:00 2617 2024-08-30T09:00:39.652 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 2617_fsm 2024-08-30T09:00:39.652 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2617_vm 2024-08-30T09:00:39.652 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Aug 30 09:00 2618 2024-08-30T09:00:39.652 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 2618_fsm 2024-08-30T09:00:39.652 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2618_vm 2024-08-30T09:00:39.652 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 147456 Aug 30 09:00 2619 2024-08-30T09:00:39.652 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 2619_fsm 2024-08-30T09:00:39.652 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2619_vm 2024-08-30T09:00:39.652 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 2620 2024-08-30T09:00:39.652 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2650 2024-08-30T09:00:39.652 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2651 2024-08-30T09:00:39.652 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2652 2024-08-30T09:00:39.653 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Aug 30 09:00 2653 2024-08-30T09:00:39.653 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Aug 30 09:00 2654 2024-08-30T09:00:39.653 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 30 09:00 2655 2024-08-30T09:00:39.653 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2656 2024-08-30T09:00:39.653 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2657 2024-08-30T09:00:39.653 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Aug 30 09:00 2658 2024-08-30T09:00:39.653 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Aug 30 09:00 2659 2024-08-30T09:00:39.653 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2660 2024-08-30T09:00:39.653 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2661 2024-08-30T09:00:39.653 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 30 09:00 2662 2024-08-30T09:00:39.653 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Aug 30 09:00 2663 2024-08-30T09:00:39.653 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2664 2024-08-30T09:00:39.653 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2665 2024-08-30T09:00:39.653 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2666 2024-08-30T09:00:39.654 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2667 2024-08-30T09:00:39.654 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2668 2024-08-30T09:00:39.654 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2669 2024-08-30T09:00:39.654 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2670 2024-08-30T09:00:39.654 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 Aug 30 09:00 2673 2024-08-30T09:00:39.654 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 Aug 30 09:00 2674 2024-08-30T09:00:39.654 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 Aug 30 09:00 2675 2024-08-30T09:00:39.654 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2678 2024-08-30T09:00:39.654 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2679 2024-08-30T09:00:39.654 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2680 2024-08-30T09:00:39.654 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2681 2024-08-30T09:00:39.654 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2682 2024-08-30T09:00:39.654 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2683 2024-08-30T09:00:39.655 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2684 2024-08-30T09:00:39.655 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2685 2024-08-30T09:00:39.655 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2686 2024-08-30T09:00:39.655 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2687 2024-08-30T09:00:39.655 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Aug 30 09:00 2688 2024-08-30T09:00:39.655 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Aug 30 09:00 2689 2024-08-30T09:00:39.655 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Aug 30 09:00 2690 2024-08-30T09:00:39.655 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 Aug 30 09:00 2691 2024-08-30T09:00:39.655 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2692 2024-08-30T09:00:39.655 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2693 2024-08-30T09:00:39.655 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2696 2024-08-30T09:00:39.655 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2699 2024-08-30T09:00:39.655 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2701 2024-08-30T09:00:39.655 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2702 2024-08-30T09:00:39.656 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 30 09:00 2703 2024-08-30T09:00:39.656 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Aug 30 09:00 2704 2024-08-30T09:00:39.656 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2753 2024-08-30T09:00:39.656 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 2753_fsm 2024-08-30T09:00:39.656 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2753_vm 2024-08-30T09:00:39.656 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2754 2024-08-30T09:00:39.656 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2755 2024-08-30T09:00:39.656 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 30 09:00 2756 2024-08-30T09:00:39.656 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 30 09:00 2757 2024-08-30T09:00:39.656 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 2830 2024-08-30T09:00:39.656 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2831 2024-08-30T09:00:39.656 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 2832 2024-08-30T09:00:39.656 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2833 2024-08-30T09:00:39.657 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 2834 2024-08-30T09:00:39.657 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2835 2024-08-30T09:00:39.657 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 2836 2024-08-30T09:00:39.657 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2837 2024-08-30T09:00:39.657 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 Aug 30 09:00 2838 2024-08-30T09:00:39.657 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 2838_fsm 2024-08-30T09:00:39.657 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2838_vm 2024-08-30T09:00:39.657 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2839 2024-08-30T09:00:39.657 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 2840 2024-08-30T09:00:39.657 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 2840_fsm 2024-08-30T09:00:39.657 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2840_vm 2024-08-30T09:00:39.657 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2841 2024-08-30T09:00:39.657 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 2995 2024-08-30T09:00:39.657 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2996 2024-08-30T09:00:39.658 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3079 2024-08-30T09:00:39.658 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 3079_fsm 2024-08-30T09:00:39.658 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3079_vm 2024-08-30T09:00:39.658 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 3080 2024-08-30T09:00:39.658 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 3081 2024-08-30T09:00:39.658 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Aug 30 09:00 3085 2024-08-30T09:00:39.658 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 3118 2024-08-30T09:00:39.658 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3119 2024-08-30T09:00:39.658 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Aug 30 09:00 3164 2024-08-30T09:00:39.658 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 3256 2024-08-30T09:00:39.658 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3257 2024-08-30T09:00:39.658 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3258 2024-08-30T09:00:39.658 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 3350 2024-08-30T09:00:39.658 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3351 2024-08-30T09:00:39.659 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3379 2024-08-30T09:00:39.659 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3380 2024-08-30T09:00:39.659 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 3381 2024-08-30T09:00:39.659 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 3394 2024-08-30T09:00:39.659 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 3394_fsm 2024-08-30T09:00:39.659 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3394_vm 2024-08-30T09:00:39.659 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 3395 2024-08-30T09:00:39.659 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 3429 2024-08-30T09:00:39.659 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 3430 2024-08-30T09:00:39.659 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3431 2024-08-30T09:00:39.659 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3433 2024-08-30T09:00:39.659 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 3439 2024-08-30T09:00:39.659 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3440 2024-08-30T09:00:39.659 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Aug 30 09:00 3455 2024-08-30T09:00:39.660 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 Aug 30 09:00 3456 2024-08-30T09:00:39.660 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 3456_fsm 2024-08-30T09:00:39.660 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3456_vm 2024-08-30T09:00:39.660 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 3466 2024-08-30T09:00:39.660 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3467 2024-08-30T09:00:39.660 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3468 2024-08-30T09:00:39.660 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 3501 2024-08-30T09:00:39.660 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3502 2024-08-30T09:00:39.660 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3503 2024-08-30T09:00:39.660 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3534 2024-08-30T09:00:39.660 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3541 2024-08-30T09:00:39.660 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 3541_fsm 2024-08-30T09:00:39.660 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3541_vm 2024-08-30T09:00:39.660 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 3542 2024-08-30T09:00:39.661 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3574 2024-08-30T09:00:39.661 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3575 2024-08-30T09:00:39.661 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 3576 2024-08-30T09:00:39.661 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 3596 2024-08-30T09:00:39.661 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3597 2024-08-30T09:00:39.661 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 3598 2024-08-30T09:00:39.661 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3599 2024-08-30T09:00:39.661 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3600 2024-08-30T09:00:39.661 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 3600_fsm 2024-08-30T09:00:39.661 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3600_vm 2024-08-30T09:00:39.661 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3601 2024-08-30T09:00:39.661 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 3601_fsm 2024-08-30T09:00:39.661 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3601_vm 2024-08-30T09:00:39.661 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3602 2024-08-30T09:00:39.662 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 3602_fsm 2024-08-30T09:00:39.662 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3602_vm 2024-08-30T09:00:39.662 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 3603 2024-08-30T09:00:39.662 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 3603_fsm 2024-08-30T09:00:39.662 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3603_vm 2024-08-30T09:00:39.662 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 3604 2024-08-30T09:00:39.662 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 3605 2024-08-30T09:00:39.662 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 3606 2024-08-30T09:00:39.662 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 3607 2024-08-30T09:00:39.662 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 3608 2024-08-30T09:00:39.662 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 30 09:00 3609 2024-08-30T09:00:39.662 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 3712 2024-08-30T09:00:39.662 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3764 2024-08-30T09:00:39.663 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 3764_fsm 2024-08-30T09:00:39.663 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3764_vm 2024-08-30T09:00:39.663 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 3766 2024-08-30T09:00:39.663 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 3767 2024-08-30T09:00:39.663 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3997 2024-08-30T09:00:39.663 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 4143 2024-08-30T09:00:39.663 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 4144 2024-08-30T09:00:39.663 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 4145 2024-08-30T09:00:39.663 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 4146 2024-08-30T09:00:39.663 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 4147 2024-08-30T09:00:39.663 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 4148 2024-08-30T09:00:39.663 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 4149 2024-08-30T09:00:39.663 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 4150 2024-08-30T09:00:39.663 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 4151 2024-08-30T09:00:39.664 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 4152 2024-08-30T09:00:39.664 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 4153 2024-08-30T09:00:39.664 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 4154 2024-08-30T09:00:39.664 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 4155 2024-08-30T09:00:39.664 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 4156 2024-08-30T09:00:39.664 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 4157 2024-08-30T09:00:39.664 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 4158 2024-08-30T09:00:39.664 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 4159 2024-08-30T09:00:39.664 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 4160 2024-08-30T09:00:39.664 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 4161 2024-08-30T09:00:39.664 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 4162 2024-08-30T09:00:39.664 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 4163 2024-08-30T09:00:39.664 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 4164 2024-08-30T09:00:39.665 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 4165 2024-08-30T09:00:39.665 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 4166 2024-08-30T09:00:39.665 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 4167 2024-08-30T09:00:39.665 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 4168 2024-08-30T09:00:39.665 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 4169 2024-08-30T09:00:39.665 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 4170 2024-08-30T09:00:39.665 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 4171 2024-08-30T09:00:39.665 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 4172 2024-08-30T09:00:39.665 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 4173 2024-08-30T09:00:39.665 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 4174 2024-08-30T09:00:39.665 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 5002 2024-08-30T09:00:39.665 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 548 2024-08-30T09:00:39.665 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 549 2024-08-30T09:00:39.665 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 6102 2024-08-30T09:00:39.666 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 6104 2024-08-30T09:00:39.666 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 6106 2024-08-30T09:00:39.666 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 6110 2024-08-30T09:00:39.666 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 6111 2024-08-30T09:00:39.666 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 6112 2024-08-30T09:00:39.666 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 6113 2024-08-30T09:00:39.666 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 6117 2024-08-30T09:00:39.666 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 826 2024-08-30T09:00:39.666 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 827 2024-08-30T09:00:39.666 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 828 2024-08-30T09:00:39.666 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Aug 30 09:00 pg_filenode.map 2024-08-30T09:00:39.666 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Aug 30 09:00 PG_VERSION 2024-08-30T09:00:39.666 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-08-30T09:00:39.666 INFO:teuthology.orchestra.run.smithi042.stdout:/tmp/cephfs/postgres/data/base/13494: 2024-08-30T09:00:39.667 INFO:teuthology.orchestra.run.smithi042.stdout:total 7777 2024-08-30T09:00:39.667 INFO:teuthology.orchestra.run.smithi042.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Aug 30 09:00 . 2024-08-30T09:00:39.667 INFO:teuthology.orchestra.run.smithi042.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Aug 30 09:00 .. 2024-08-30T09:00:39.667 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 112 2024-08-30T09:00:39.667 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 113 2024-08-30T09:00:39.667 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Aug 30 09:00 1247 2024-08-30T09:00:39.667 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 1247_fsm 2024-08-30T09:00:39.667 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 1247_vm 2024-08-30T09:00:39.667 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 Aug 30 09:00 1249 2024-08-30T09:00:39.667 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 1249_fsm 2024-08-30T09:00:39.667 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 1249_vm 2024-08-30T09:00:39.667 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 Aug 30 09:00 1255 2024-08-30T09:00:39.667 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 1255_fsm 2024-08-30T09:00:39.667 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 1255_vm 2024-08-30T09:00:39.668 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Aug 30 09:00 1259 2024-08-30T09:00:39.668 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 1259_fsm 2024-08-30T09:00:39.668 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 1259_vm 2024-08-30T09:00:39.668 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 Aug 30 09:00 13344 2024-08-30T09:00:39.668 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 13344_fsm 2024-08-30T09:00:39.668 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 13344_vm 2024-08-30T09:00:39.668 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 13346 2024-08-30T09:00:39.668 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 13348 2024-08-30T09:00:39.668 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 13349 2024-08-30T09:00:39.668 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 13349_fsm 2024-08-30T09:00:39.668 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 13349_vm 2024-08-30T09:00:39.668 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 13351 2024-08-30T09:00:39.668 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 13353 2024-08-30T09:00:39.668 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 13354 2024-08-30T09:00:39.669 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 13354_fsm 2024-08-30T09:00:39.669 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 13354_vm 2024-08-30T09:00:39.669 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 13356 2024-08-30T09:00:39.669 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 13358 2024-08-30T09:00:39.669 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 13359 2024-08-30T09:00:39.669 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 13359_fsm 2024-08-30T09:00:39.669 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 13359_vm 2024-08-30T09:00:39.669 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 13361 2024-08-30T09:00:39.669 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 13363 2024-08-30T09:00:39.669 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 1417 2024-08-30T09:00:39.669 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 1418 2024-08-30T09:00:39.669 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 174 2024-08-30T09:00:39.669 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 175 2024-08-30T09:00:39.670 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2187 2024-08-30T09:00:39.670 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 2224 2024-08-30T09:00:39.670 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 2328 2024-08-30T09:00:39.670 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 2336 2024-08-30T09:00:39.670 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2337 2024-08-30T09:00:39.670 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2579 2024-08-30T09:00:39.670 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2600 2024-08-30T09:00:39.670 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 2600_fsm 2024-08-30T09:00:39.670 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2600_vm 2024-08-30T09:00:39.670 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2601 2024-08-30T09:00:39.670 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 2601_fsm 2024-08-30T09:00:39.670 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2601_vm 2024-08-30T09:00:39.670 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Aug 30 09:00 2602 2024-08-30T09:00:39.671 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 2602_fsm 2024-08-30T09:00:39.671 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2602_vm 2024-08-30T09:00:39.671 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 30 09:00 2603 2024-08-30T09:00:39.671 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 2603_fsm 2024-08-30T09:00:39.671 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2603_vm 2024-08-30T09:00:39.671 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 2604 2024-08-30T09:00:39.671 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2605 2024-08-30T09:00:39.671 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 2605_fsm 2024-08-30T09:00:39.671 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2605_vm 2024-08-30T09:00:39.671 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2606 2024-08-30T09:00:39.671 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 2606_fsm 2024-08-30T09:00:39.671 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2606_vm 2024-08-30T09:00:39.671 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2607 2024-08-30T09:00:39.671 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 2607_fsm 2024-08-30T09:00:39.672 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2607_vm 2024-08-30T09:00:39.672 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 Aug 30 09:00 2608 2024-08-30T09:00:39.672 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 2608_fsm 2024-08-30T09:00:39.672 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2608_vm 2024-08-30T09:00:39.672 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 Aug 30 09:00 2609 2024-08-30T09:00:39.672 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 2609_fsm 2024-08-30T09:00:39.672 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2609_vm 2024-08-30T09:00:39.672 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 30 09:00 2610 2024-08-30T09:00:39.672 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 2610_fsm 2024-08-30T09:00:39.672 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2610_vm 2024-08-30T09:00:39.672 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 2611 2024-08-30T09:00:39.672 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2612 2024-08-30T09:00:39.672 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 2612_fsm 2024-08-30T09:00:39.672 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2612_vm 2024-08-30T09:00:39.673 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 2613 2024-08-30T09:00:39.673 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2615 2024-08-30T09:00:39.673 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 2615_fsm 2024-08-30T09:00:39.673 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2615_vm 2024-08-30T09:00:39.673 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2616 2024-08-30T09:00:39.673 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 2616_fsm 2024-08-30T09:00:39.673 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2616_vm 2024-08-30T09:00:39.673 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 Aug 30 09:00 2617 2024-08-30T09:00:39.673 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 2617_fsm 2024-08-30T09:00:39.673 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2617_vm 2024-08-30T09:00:39.673 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Aug 30 09:00 2618 2024-08-30T09:00:39.673 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 2618_fsm 2024-08-30T09:00:39.673 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2618_vm 2024-08-30T09:00:39.674 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 147456 Aug 30 09:00 2619 2024-08-30T09:00:39.674 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 2619_fsm 2024-08-30T09:00:39.674 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2619_vm 2024-08-30T09:00:39.674 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 2620 2024-08-30T09:00:39.674 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2650 2024-08-30T09:00:39.674 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2651 2024-08-30T09:00:39.674 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2652 2024-08-30T09:00:39.674 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Aug 30 09:00 2653 2024-08-30T09:00:39.674 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Aug 30 09:00 2654 2024-08-30T09:00:39.674 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 30 09:00 2655 2024-08-30T09:00:39.674 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2656 2024-08-30T09:00:39.674 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2657 2024-08-30T09:00:39.674 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Aug 30 09:00 2658 2024-08-30T09:00:39.674 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Aug 30 09:00 2659 2024-08-30T09:00:39.675 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2660 2024-08-30T09:00:39.675 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2661 2024-08-30T09:00:39.675 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 30 09:00 2662 2024-08-30T09:00:39.675 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Aug 30 09:00 2663 2024-08-30T09:00:39.675 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2664 2024-08-30T09:00:39.675 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2665 2024-08-30T09:00:39.675 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2666 2024-08-30T09:00:39.675 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2667 2024-08-30T09:00:39.675 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2668 2024-08-30T09:00:39.675 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2669 2024-08-30T09:00:39.675 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2670 2024-08-30T09:00:39.675 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 Aug 30 09:00 2673 2024-08-30T09:00:39.675 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 Aug 30 09:00 2674 2024-08-30T09:00:39.676 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 Aug 30 09:00 2675 2024-08-30T09:00:39.676 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2678 2024-08-30T09:00:39.676 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2679 2024-08-30T09:00:39.676 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2680 2024-08-30T09:00:39.676 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2681 2024-08-30T09:00:39.676 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2682 2024-08-30T09:00:39.676 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2683 2024-08-30T09:00:39.676 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2684 2024-08-30T09:00:39.676 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2685 2024-08-30T09:00:39.676 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2686 2024-08-30T09:00:39.676 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2687 2024-08-30T09:00:39.676 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Aug 30 09:00 2688 2024-08-30T09:00:39.676 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Aug 30 09:00 2689 2024-08-30T09:00:39.676 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Aug 30 09:00 2690 2024-08-30T09:00:39.677 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 Aug 30 09:00 2691 2024-08-30T09:00:39.677 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2692 2024-08-30T09:00:39.677 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2693 2024-08-30T09:00:39.677 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2696 2024-08-30T09:00:39.677 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2699 2024-08-30T09:00:39.677 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2701 2024-08-30T09:00:39.677 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2702 2024-08-30T09:00:39.677 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 30 09:00 2703 2024-08-30T09:00:39.677 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Aug 30 09:00 2704 2024-08-30T09:00:39.677 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2753 2024-08-30T09:00:39.677 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 2753_fsm 2024-08-30T09:00:39.677 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2753_vm 2024-08-30T09:00:39.677 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2754 2024-08-30T09:00:39.678 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2755 2024-08-30T09:00:39.678 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 30 09:00 2756 2024-08-30T09:00:39.678 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 30 09:00 2757 2024-08-30T09:00:39.678 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 2830 2024-08-30T09:00:39.678 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2831 2024-08-30T09:00:39.678 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 2832 2024-08-30T09:00:39.678 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2833 2024-08-30T09:00:39.678 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 2834 2024-08-30T09:00:39.678 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2835 2024-08-30T09:00:39.678 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 2836 2024-08-30T09:00:39.678 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2837 2024-08-30T09:00:39.678 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 Aug 30 09:00 2838 2024-08-30T09:00:39.678 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 2838_fsm 2024-08-30T09:00:39.678 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2838_vm 2024-08-30T09:00:39.679 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2839 2024-08-30T09:00:39.679 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 2840 2024-08-30T09:00:39.679 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 2840_fsm 2024-08-30T09:00:39.679 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2840_vm 2024-08-30T09:00:39.679 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2841 2024-08-30T09:00:39.679 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 2995 2024-08-30T09:00:39.679 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2996 2024-08-30T09:00:39.679 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3079 2024-08-30T09:00:39.679 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 3079_fsm 2024-08-30T09:00:39.679 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3079_vm 2024-08-30T09:00:39.679 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 3080 2024-08-30T09:00:39.679 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 3081 2024-08-30T09:00:39.679 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Aug 30 09:00 3085 2024-08-30T09:00:39.679 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 3118 2024-08-30T09:00:39.680 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3119 2024-08-30T09:00:39.680 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Aug 30 09:00 3164 2024-08-30T09:00:39.680 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 3256 2024-08-30T09:00:39.680 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3257 2024-08-30T09:00:39.680 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3258 2024-08-30T09:00:39.680 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 3350 2024-08-30T09:00:39.680 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3351 2024-08-30T09:00:39.680 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3379 2024-08-30T09:00:39.680 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3380 2024-08-30T09:00:39.680 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 3381 2024-08-30T09:00:39.680 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 3394 2024-08-30T09:00:39.680 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 3394_fsm 2024-08-30T09:00:39.680 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3394_vm 2024-08-30T09:00:39.681 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 3395 2024-08-30T09:00:39.681 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 3429 2024-08-30T09:00:39.681 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 3430 2024-08-30T09:00:39.681 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3431 2024-08-30T09:00:39.681 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3433 2024-08-30T09:00:39.681 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 3439 2024-08-30T09:00:39.681 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3440 2024-08-30T09:00:39.681 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Aug 30 09:00 3455 2024-08-30T09:00:39.681 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 Aug 30 09:00 3456 2024-08-30T09:00:39.681 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 3456_fsm 2024-08-30T09:00:39.681 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3456_vm 2024-08-30T09:00:39.681 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 3466 2024-08-30T09:00:39.681 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3467 2024-08-30T09:00:39.681 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3468 2024-08-30T09:00:39.682 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 3501 2024-08-30T09:00:39.682 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3502 2024-08-30T09:00:39.682 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3503 2024-08-30T09:00:39.682 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3534 2024-08-30T09:00:39.682 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3541 2024-08-30T09:00:39.682 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 3541_fsm 2024-08-30T09:00:39.682 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3541_vm 2024-08-30T09:00:39.682 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 3542 2024-08-30T09:00:39.682 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3574 2024-08-30T09:00:39.682 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3575 2024-08-30T09:00:39.682 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 3576 2024-08-30T09:00:39.682 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 3596 2024-08-30T09:00:39.682 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3597 2024-08-30T09:00:39.683 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 3598 2024-08-30T09:00:39.683 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3599 2024-08-30T09:00:39.683 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3600 2024-08-30T09:00:39.683 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 3600_fsm 2024-08-30T09:00:39.683 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3600_vm 2024-08-30T09:00:39.683 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3601 2024-08-30T09:00:39.683 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 3601_fsm 2024-08-30T09:00:39.683 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3601_vm 2024-08-30T09:00:39.683 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3602 2024-08-30T09:00:39.683 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 3602_fsm 2024-08-30T09:00:39.683 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3602_vm 2024-08-30T09:00:39.683 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 3603 2024-08-30T09:00:39.683 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 3603_fsm 2024-08-30T09:00:39.683 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3603_vm 2024-08-30T09:00:39.684 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 3604 2024-08-30T09:00:39.684 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 3605 2024-08-30T09:00:39.684 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 3606 2024-08-30T09:00:39.684 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 3607 2024-08-30T09:00:39.684 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 3608 2024-08-30T09:00:39.684 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 30 09:00 3609 2024-08-30T09:00:39.684 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 3712 2024-08-30T09:00:39.684 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3764 2024-08-30T09:00:39.684 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 3764_fsm 2024-08-30T09:00:39.684 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3764_vm 2024-08-30T09:00:39.684 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 3766 2024-08-30T09:00:39.684 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 3767 2024-08-30T09:00:39.684 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3997 2024-08-30T09:00:39.684 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 4143 2024-08-30T09:00:39.685 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 4144 2024-08-30T09:00:39.685 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 4145 2024-08-30T09:00:39.685 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 4146 2024-08-30T09:00:39.685 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 4147 2024-08-30T09:00:39.685 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 4148 2024-08-30T09:00:39.685 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 4149 2024-08-30T09:00:39.685 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 4150 2024-08-30T09:00:39.685 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 4151 2024-08-30T09:00:39.685 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 4152 2024-08-30T09:00:39.685 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 4153 2024-08-30T09:00:39.685 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 4154 2024-08-30T09:00:39.685 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 4155 2024-08-30T09:00:39.685 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 4156 2024-08-30T09:00:39.686 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 4157 2024-08-30T09:00:39.686 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 4158 2024-08-30T09:00:39.686 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 4159 2024-08-30T09:00:39.686 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 4160 2024-08-30T09:00:39.686 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 4161 2024-08-30T09:00:39.686 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 4162 2024-08-30T09:00:39.686 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 4163 2024-08-30T09:00:39.686 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 4164 2024-08-30T09:00:39.686 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 4165 2024-08-30T09:00:39.686 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 4166 2024-08-30T09:00:39.686 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 4167 2024-08-30T09:00:39.686 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 4168 2024-08-30T09:00:39.686 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 4169 2024-08-30T09:00:39.686 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 4170 2024-08-30T09:00:39.687 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 4171 2024-08-30T09:00:39.687 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 4172 2024-08-30T09:00:39.687 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 4173 2024-08-30T09:00:39.687 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 4174 2024-08-30T09:00:39.687 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 5002 2024-08-30T09:00:39.687 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 548 2024-08-30T09:00:39.687 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 549 2024-08-30T09:00:39.687 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 6102 2024-08-30T09:00:39.687 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 6104 2024-08-30T09:00:39.687 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 6106 2024-08-30T09:00:39.687 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 6110 2024-08-30T09:00:39.687 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 6111 2024-08-30T09:00:39.687 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 6112 2024-08-30T09:00:39.687 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 6113 2024-08-30T09:00:39.688 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 6117 2024-08-30T09:00:39.688 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 826 2024-08-30T09:00:39.688 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 827 2024-08-30T09:00:39.688 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 828 2024-08-30T09:00:39.688 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Aug 30 09:00 pg_filenode.map 2024-08-30T09:00:39.688 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Aug 30 09:00 PG_VERSION 2024-08-30T09:00:39.688 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-08-30T09:00:39.688 INFO:teuthology.orchestra.run.smithi042.stdout:/tmp/cephfs/postgres/data/global: 2024-08-30T09:00:39.688 INFO:teuthology.orchestra.run.smithi042.stdout:total 537 2024-08-30T09:00:39.688 INFO:teuthology.orchestra.run.smithi042.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 57 Aug 30 09:00 . 2024-08-30T09:00:39.688 INFO:teuthology.orchestra.run.smithi042.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Aug 30 09:00 .. 2024-08-30T09:00:39.688 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 1213 2024-08-30T09:00:39.688 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 1213_fsm 2024-08-30T09:00:39.689 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 1213_vm 2024-08-30T09:00:39.689 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 1214 2024-08-30T09:00:39.689 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 1214_fsm 2024-08-30T09:00:39.689 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 1214_vm 2024-08-30T09:00:39.689 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 1232 2024-08-30T09:00:39.689 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 1233 2024-08-30T09:00:39.689 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 1260 2024-08-30T09:00:39.689 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 1260_fsm 2024-08-30T09:00:39.689 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 1260_vm 2024-08-30T09:00:39.689 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 1261 2024-08-30T09:00:39.689 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 1261_fsm 2024-08-30T09:00:39.689 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 1261_vm 2024-08-30T09:00:39.689 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 1262 2024-08-30T09:00:39.689 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 1262_fsm 2024-08-30T09:00:39.690 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 1262_vm 2024-08-30T09:00:39.690 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2396 2024-08-30T09:00:39.690 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 2396_fsm 2024-08-30T09:00:39.690 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2396_vm 2024-08-30T09:00:39.690 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2397 2024-08-30T09:00:39.690 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2671 2024-08-30T09:00:39.690 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2672 2024-08-30T09:00:39.690 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2676 2024-08-30T09:00:39.690 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2677 2024-08-30T09:00:39.690 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2694 2024-08-30T09:00:39.690 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2695 2024-08-30T09:00:39.690 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2697 2024-08-30T09:00:39.690 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2698 2024-08-30T09:00:39.691 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 2846 2024-08-30T09:00:39.691 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2847 2024-08-30T09:00:39.691 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 2964 2024-08-30T09:00:39.691 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2965 2024-08-30T09:00:39.691 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 2966 2024-08-30T09:00:39.691 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2967 2024-08-30T09:00:39.691 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 3592 2024-08-30T09:00:39.691 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3593 2024-08-30T09:00:39.691 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 4060 2024-08-30T09:00:39.691 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 4061 2024-08-30T09:00:39.691 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 4175 2024-08-30T09:00:39.691 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 4176 2024-08-30T09:00:39.691 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 4177 2024-08-30T09:00:39.691 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 4178 2024-08-30T09:00:39.692 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 4181 2024-08-30T09:00:39.692 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 4182 2024-08-30T09:00:39.692 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 4183 2024-08-30T09:00:39.692 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 4184 2024-08-30T09:00:39.692 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 4185 2024-08-30T09:00:39.692 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 4186 2024-08-30T09:00:39.692 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 6000 2024-08-30T09:00:39.692 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 6001 2024-08-30T09:00:39.692 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 6002 2024-08-30T09:00:39.692 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 6100 2024-08-30T09:00:39.692 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 6114 2024-08-30T09:00:39.692 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 6115 2024-08-30T09:00:39.692 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 pg_control 2024-08-30T09:00:39.692 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Aug 30 09:00 pg_filenode.map 2024-08-30T09:00:39.693 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-08-30T09:00:39.693 INFO:teuthology.orchestra.run.smithi042.stdout:/tmp/cephfs/postgres/data/log: 2024-08-30T09:00:39.693 INFO:teuthology.orchestra.run.smithi042.stdout:total 0 2024-08-30T09:00:39.693 INFO:teuthology.orchestra.run.smithi042.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 . 2024-08-30T09:00:39.693 INFO:teuthology.orchestra.run.smithi042.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Aug 30 09:00 .. 2024-08-30T09:00:39.693 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-08-30T09:00:39.693 INFO:teuthology.orchestra.run.smithi042.stdout:/tmp/cephfs/postgres/data/pg_commit_ts: 2024-08-30T09:00:39.693 INFO:teuthology.orchestra.run.smithi042.stdout:total 0 2024-08-30T09:00:39.693 INFO:teuthology.orchestra.run.smithi042.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 . 2024-08-30T09:00:39.693 INFO:teuthology.orchestra.run.smithi042.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Aug 30 09:00 .. 2024-08-30T09:00:39.693 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-08-30T09:00:39.693 INFO:teuthology.orchestra.run.smithi042.stdout:/tmp/cephfs/postgres/data/pg_dynshmem: 2024-08-30T09:00:39.693 INFO:teuthology.orchestra.run.smithi042.stdout:total 0 2024-08-30T09:00:39.694 INFO:teuthology.orchestra.run.smithi042.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 . 2024-08-30T09:00:39.694 INFO:teuthology.orchestra.run.smithi042.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Aug 30 09:00 .. 2024-08-30T09:00:39.694 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-08-30T09:00:39.694 INFO:teuthology.orchestra.run.smithi042.stdout:/tmp/cephfs/postgres/data/pg_logical: 2024-08-30T09:00:39.694 INFO:teuthology.orchestra.run.smithi042.stdout:total 1 2024-08-30T09:00:39.694 INFO:teuthology.orchestra.run.smithi042.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Aug 30 09:00 . 2024-08-30T09:00:39.694 INFO:teuthology.orchestra.run.smithi042.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Aug 30 09:00 .. 2024-08-30T09:00:39.694 INFO:teuthology.orchestra.run.smithi042.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 mappings 2024-08-30T09:00:39.694 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8 Aug 30 09:00 replorigin_checkpoint 2024-08-30T09:00:39.694 INFO:teuthology.orchestra.run.smithi042.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 snapshots 2024-08-30T09:00:39.694 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-08-30T09:00:39.694 INFO:teuthology.orchestra.run.smithi042.stdout:/tmp/cephfs/postgres/data/pg_logical/mappings: 2024-08-30T09:00:39.694 INFO:teuthology.orchestra.run.smithi042.stdout:total 0 2024-08-30T09:00:39.694 INFO:teuthology.orchestra.run.smithi042.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 . 2024-08-30T09:00:39.695 INFO:teuthology.orchestra.run.smithi042.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Aug 30 09:00 .. 2024-08-30T09:00:39.695 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-08-30T09:00:39.695 INFO:teuthology.orchestra.run.smithi042.stdout:/tmp/cephfs/postgres/data/pg_logical/snapshots: 2024-08-30T09:00:39.695 INFO:teuthology.orchestra.run.smithi042.stdout:total 0 2024-08-30T09:00:39.695 INFO:teuthology.orchestra.run.smithi042.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 . 2024-08-30T09:00:39.695 INFO:teuthology.orchestra.run.smithi042.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Aug 30 09:00 .. 2024-08-30T09:00:39.695 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-08-30T09:00:39.695 INFO:teuthology.orchestra.run.smithi042.stdout:/tmp/cephfs/postgres/data/pg_multixact: 2024-08-30T09:00:39.695 INFO:teuthology.orchestra.run.smithi042.stdout:total 0 2024-08-30T09:00:39.695 INFO:teuthology.orchestra.run.smithi042.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Aug 30 09:00 . 2024-08-30T09:00:39.695 INFO:teuthology.orchestra.run.smithi042.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Aug 30 09:00 .. 2024-08-30T09:00:39.695 INFO:teuthology.orchestra.run.smithi042.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Aug 30 09:00 members 2024-08-30T09:00:39.695 INFO:teuthology.orchestra.run.smithi042.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Aug 30 09:00 offsets 2024-08-30T09:00:39.695 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-08-30T09:00:39.696 INFO:teuthology.orchestra.run.smithi042.stdout:/tmp/cephfs/postgres/data/pg_multixact/members: 2024-08-30T09:00:39.696 INFO:teuthology.orchestra.run.smithi042.stdout:total 8 2024-08-30T09:00:39.696 INFO:teuthology.orchestra.run.smithi042.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Aug 30 09:00 . 2024-08-30T09:00:39.696 INFO:teuthology.orchestra.run.smithi042.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Aug 30 09:00 .. 2024-08-30T09:00:39.696 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 0000 2024-08-30T09:00:39.696 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-08-30T09:00:39.696 INFO:teuthology.orchestra.run.smithi042.stdout:/tmp/cephfs/postgres/data/pg_multixact/offsets: 2024-08-30T09:00:39.696 INFO:teuthology.orchestra.run.smithi042.stdout:total 8 2024-08-30T09:00:39.696 INFO:teuthology.orchestra.run.smithi042.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Aug 30 09:00 . 2024-08-30T09:00:39.696 INFO:teuthology.orchestra.run.smithi042.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Aug 30 09:00 .. 2024-08-30T09:00:39.696 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 0000 2024-08-30T09:00:39.696 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-08-30T09:00:39.696 INFO:teuthology.orchestra.run.smithi042.stdout:/tmp/cephfs/postgres/data/pg_notify: 2024-08-30T09:00:39.697 INFO:teuthology.orchestra.run.smithi042.stdout:total 0 2024-08-30T09:00:39.697 INFO:teuthology.orchestra.run.smithi042.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 . 2024-08-30T09:00:39.697 INFO:teuthology.orchestra.run.smithi042.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Aug 30 09:00 .. 2024-08-30T09:00:39.697 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-08-30T09:00:39.697 INFO:teuthology.orchestra.run.smithi042.stdout:/tmp/cephfs/postgres/data/pg_replslot: 2024-08-30T09:00:39.697 INFO:teuthology.orchestra.run.smithi042.stdout:total 0 2024-08-30T09:00:39.697 INFO:teuthology.orchestra.run.smithi042.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 . 2024-08-30T09:00:39.697 INFO:teuthology.orchestra.run.smithi042.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Aug 30 09:00 .. 2024-08-30T09:00:39.697 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-08-30T09:00:39.697 INFO:teuthology.orchestra.run.smithi042.stdout:/tmp/cephfs/postgres/data/pg_serial: 2024-08-30T09:00:39.697 INFO:teuthology.orchestra.run.smithi042.stdout:total 0 2024-08-30T09:00:39.697 INFO:teuthology.orchestra.run.smithi042.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 . 2024-08-30T09:00:39.697 INFO:teuthology.orchestra.run.smithi042.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Aug 30 09:00 .. 2024-08-30T09:00:39.697 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-08-30T09:00:39.698 INFO:teuthology.orchestra.run.smithi042.stdout:/tmp/cephfs/postgres/data/pg_snapshots: 2024-08-30T09:00:39.698 INFO:teuthology.orchestra.run.smithi042.stdout:total 0 2024-08-30T09:00:39.698 INFO:teuthology.orchestra.run.smithi042.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 . 2024-08-30T09:00:39.698 INFO:teuthology.orchestra.run.smithi042.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Aug 30 09:00 .. 2024-08-30T09:00:39.698 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-08-30T09:00:39.698 INFO:teuthology.orchestra.run.smithi042.stdout:/tmp/cephfs/postgres/data/pg_stat: 2024-08-30T09:00:39.698 INFO:teuthology.orchestra.run.smithi042.stdout:total 0 2024-08-30T09:00:39.698 INFO:teuthology.orchestra.run.smithi042.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 . 2024-08-30T09:00:39.698 INFO:teuthology.orchestra.run.smithi042.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Aug 30 09:00 .. 2024-08-30T09:00:39.698 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-08-30T09:00:39.698 INFO:teuthology.orchestra.run.smithi042.stdout:/tmp/cephfs/postgres/data/pg_stat_tmp: 2024-08-30T09:00:39.698 INFO:teuthology.orchestra.run.smithi042.stdout:total 0 2024-08-30T09:00:39.698 INFO:teuthology.orchestra.run.smithi042.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 . 2024-08-30T09:00:39.698 INFO:teuthology.orchestra.run.smithi042.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Aug 30 09:00 .. 2024-08-30T09:00:39.699 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-08-30T09:00:39.699 INFO:teuthology.orchestra.run.smithi042.stdout:/tmp/cephfs/postgres/data/pg_subtrans: 2024-08-30T09:00:39.699 INFO:teuthology.orchestra.run.smithi042.stdout:total 8 2024-08-30T09:00:39.699 INFO:teuthology.orchestra.run.smithi042.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Aug 30 09:00 . 2024-08-30T09:00:39.699 INFO:teuthology.orchestra.run.smithi042.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Aug 30 09:00 .. 2024-08-30T09:00:39.699 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 0000 2024-08-30T09:00:39.699 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-08-30T09:00:39.699 INFO:teuthology.orchestra.run.smithi042.stdout:/tmp/cephfs/postgres/data/pg_tblspc: 2024-08-30T09:00:39.699 INFO:teuthology.orchestra.run.smithi042.stdout:total 0 2024-08-30T09:00:39.699 INFO:teuthology.orchestra.run.smithi042.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 . 2024-08-30T09:00:39.699 INFO:teuthology.orchestra.run.smithi042.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Aug 30 09:00 .. 2024-08-30T09:00:39.699 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-08-30T09:00:39.699 INFO:teuthology.orchestra.run.smithi042.stdout:/tmp/cephfs/postgres/data/pg_twophase: 2024-08-30T09:00:39.699 INFO:teuthology.orchestra.run.smithi042.stdout:total 0 2024-08-30T09:00:39.700 INFO:teuthology.orchestra.run.smithi042.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 . 2024-08-30T09:00:39.700 INFO:teuthology.orchestra.run.smithi042.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Aug 30 09:00 .. 2024-08-30T09:00:39.700 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-08-30T09:00:39.700 INFO:teuthology.orchestra.run.smithi042.stdout:/tmp/cephfs/postgres/data/pg_wal: 2024-08-30T09:00:39.700 INFO:teuthology.orchestra.run.smithi042.stdout:total 16384 2024-08-30T09:00:39.700 INFO:teuthology.orchestra.run.smithi042.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Aug 30 09:00 . 2024-08-30T09:00:39.700 INFO:teuthology.orchestra.run.smithi042.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Aug 30 09:00 .. 2024-08-30T09:00:39.700 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 30 09:00 000000010000000000000001 2024-08-30T09:00:39.700 INFO:teuthology.orchestra.run.smithi042.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 archive_status 2024-08-30T09:00:39.700 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-08-30T09:00:39.700 INFO:teuthology.orchestra.run.smithi042.stdout:/tmp/cephfs/postgres/data/pg_wal/archive_status: 2024-08-30T09:00:39.700 INFO:teuthology.orchestra.run.smithi042.stdout:total 0 2024-08-30T09:00:39.700 INFO:teuthology.orchestra.run.smithi042.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 . 2024-08-30T09:00:39.701 INFO:teuthology.orchestra.run.smithi042.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Aug 30 09:00 .. 2024-08-30T09:00:39.701 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-08-30T09:00:39.701 INFO:teuthology.orchestra.run.smithi042.stdout:/tmp/cephfs/postgres/data/pg_xact: 2024-08-30T09:00:39.701 INFO:teuthology.orchestra.run.smithi042.stdout:total 8 2024-08-30T09:00:39.701 INFO:teuthology.orchestra.run.smithi042.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Aug 30 09:00 . 2024-08-30T09:00:39.701 INFO:teuthology.orchestra.run.smithi042.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Aug 30 09:00 .. 2024-08-30T09:00:39.701 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 0000 2024-08-30T09:00:39.701 DEBUG:teuthology.orchestra.run.smithi042:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo systemctl start postgresql' 2024-08-30T09:00:39.836 DEBUG:teuthology.orchestra.run.smithi042:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- pgbench -s 32 -i' 2024-08-30T09:00:39.881 INFO:teuthology.orchestra.run.smithi042.stderr:dropping old tables... 2024-08-30T09:00:39.881 INFO:teuthology.orchestra.run.smithi042.stderr:NOTICE: table "pgbench_accounts" does not exist, skipping 2024-08-30T09:00:39.881 INFO:teuthology.orchestra.run.smithi042.stderr:NOTICE: table "pgbench_branches" does not exist, skipping 2024-08-30T09:00:39.881 INFO:teuthology.orchestra.run.smithi042.stderr:NOTICE: table "pgbench_history" does not exist, skipping 2024-08-30T09:00:39.881 INFO:teuthology.orchestra.run.smithi042.stderr:NOTICE: table "pgbench_tellers" does not exist, skipping 2024-08-30T09:00:39.882 INFO:teuthology.orchestra.run.smithi042.stderr:creating tables... 2024-08-30T09:00:39.895 INFO:teuthology.orchestra.run.smithi042.stderr:generating data (client-side)... 2024-08-30T09:00:39.968 INFO:teuthology.orchestra.run.smithi042.stderr:100000 of 3200000 tuples (3%) done (elapsed 0.06 s, remaining 1.74 s) 2024-08-30T09:00:40.126 INFO:teuthology.orchestra.run.smithi042.stderr:200000 of 3200000 tuples (6%) done (elapsed 0.21 s, remaining 3.21 s) 2024-08-30T09:00:40.212 INFO:teuthology.orchestra.run.smithi042.stderr:300000 of 3200000 tuples (9%) done (elapsed 0.30 s, remaining 2.90 s) 2024-08-30T09:00:40.357 INFO:teuthology.orchestra.run.smithi042.stderr:400000 of 3200000 tuples (12%) done (elapsed 0.45 s, remaining 3.12 s) 2024-08-30T09:00:40.507 INFO:teuthology.orchestra.run.smithi042.stderr:500000 of 3200000 tuples (15%) done (elapsed 0.59 s, remaining 3.21 s) 2024-08-30T09:00:40.591 INFO:teuthology.orchestra.run.smithi042.stderr:600000 of 3200000 tuples (18%) done (elapsed 0.68 s, remaining 2.94 s) 2024-08-30T09:00:40.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:00:40 smithi190 ceph-mon[22001]: pgmap v370: 129 pgs: 129 active+clean; 1.5 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 91 KiB/s wr, 12 op/s 2024-08-30T09:00:40.766 INFO:teuthology.orchestra.run.smithi042.stderr:700000 of 3200000 tuples (21%) done (elapsed 0.85 s, remaining 3.05 s) 2024-08-30T09:00:40.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:00:40 smithi167 ceph-mon[22063]: pgmap v370: 129 pgs: 129 active+clean; 1.5 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 91 KiB/s wr, 12 op/s 2024-08-30T09:00:40.882 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:00:40 smithi042 ceph-mon[18652]: pgmap v370: 129 pgs: 129 active+clean; 1.5 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 91 KiB/s wr, 12 op/s 2024-08-30T09:00:40.911 INFO:teuthology.orchestra.run.smithi042.stderr:800000 of 3200000 tuples (25%) done (elapsed 1.00 s, remaining 3.00 s) 2024-08-30T09:00:41.000 INFO:teuthology.orchestra.run.smithi042.stderr:900000 of 3200000 tuples (28%) done (elapsed 1.09 s, remaining 2.78 s) 2024-08-30T09:00:41.155 INFO:teuthology.orchestra.run.smithi042.stderr:1000000 of 3200000 tuples (31%) done (elapsed 1.24 s, remaining 2.73 s) 2024-08-30T09:00:41.324 INFO:teuthology.orchestra.run.smithi042.stderr:1100000 of 3200000 tuples (34%) done (elapsed 1.41 s, remaining 2.70 s) 2024-08-30T09:00:41.417 INFO:teuthology.orchestra.run.smithi042.stderr:1200000 of 3200000 tuples (37%) done (elapsed 1.51 s, remaining 2.51 s) 2024-08-30T09:00:41.551 INFO:teuthology.orchestra.run.smithi042.stderr:1300000 of 3200000 tuples (40%) done (elapsed 1.64 s, remaining 2.40 s) 2024-08-30T09:00:41.730 INFO:teuthology.orchestra.run.smithi042.stderr:1400000 of 3200000 tuples (43%) done (elapsed 1.82 s, remaining 2.34 s) 2024-08-30T09:00:41.829 INFO:teuthology.orchestra.run.smithi042.stderr:1500000 of 3200000 tuples (46%) done (elapsed 1.92 s, remaining 2.17 s) 2024-08-30T09:00:41.981 INFO:teuthology.orchestra.run.smithi042.stderr:1600000 of 3200000 tuples (50%) done (elapsed 2.07 s, remaining 2.07 s) 2024-08-30T09:00:42.084 INFO:teuthology.orchestra.run.smithi042.stderr:1700000 of 3200000 tuples (53%) done (elapsed 2.17 s, remaining 1.92 s) 2024-08-30T09:00:42.248 INFO:teuthology.orchestra.run.smithi042.stderr:1800000 of 3200000 tuples (56%) done (elapsed 2.34 s, remaining 1.82 s) 2024-08-30T09:00:42.380 INFO:teuthology.orchestra.run.smithi042.stderr:1900000 of 3200000 tuples (59%) done (elapsed 2.47 s, remaining 1.69 s) 2024-08-30T09:00:42.470 INFO:teuthology.orchestra.run.smithi042.stderr:2000000 of 3200000 tuples (62%) done (elapsed 2.56 s, remaining 1.53 s) 2024-08-30T09:00:42.600 INFO:teuthology.orchestra.run.smithi042.stderr:2100000 of 3200000 tuples (65%) done (elapsed 2.69 s, remaining 1.41 s) 2024-08-30T09:00:42.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:00:42 smithi190 ceph-mon[22001]: pgmap v371: 129 pgs: 129 active+clean; 13 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 1.4 MiB/s wr, 17 op/s 2024-08-30T09:00:42.779 INFO:teuthology.orchestra.run.smithi042.stderr:2200000 of 3200000 tuples (68%) done (elapsed 2.87 s, remaining 1.30 s) 2024-08-30T09:00:42.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:00:42 smithi167 ceph-mon[22063]: pgmap v371: 129 pgs: 129 active+clean; 13 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 1.4 MiB/s wr, 17 op/s 2024-08-30T09:00:42.879 INFO:teuthology.orchestra.run.smithi042.stderr:2300000 of 3200000 tuples (71%) done (elapsed 2.97 s, remaining 1.16 s) 2024-08-30T09:00:42.882 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:00:42 smithi042 ceph-mon[18652]: pgmap v371: 129 pgs: 129 active+clean; 13 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 1.4 MiB/s wr, 17 op/s 2024-08-30T09:00:43.007 INFO:teuthology.orchestra.run.smithi042.stderr:2400000 of 3200000 tuples (75%) done (elapsed 3.09 s, remaining 1.03 s) 2024-08-30T09:00:43.165 INFO:teuthology.orchestra.run.smithi042.stderr:2500000 of 3200000 tuples (78%) done (elapsed 3.25 s, remaining 0.91 s) 2024-08-30T09:00:43.253 INFO:teuthology.orchestra.run.smithi042.stderr:2600000 of 3200000 tuples (81%) done (elapsed 3.34 s, remaining 0.77 s) 2024-08-30T09:00:43.399 INFO:teuthology.orchestra.run.smithi042.stderr:2700000 of 3200000 tuples (84%) done (elapsed 3.49 s, remaining 0.65 s) 2024-08-30T09:00:43.519 INFO:teuthology.orchestra.run.smithi042.stderr:2800000 of 3200000 tuples (87%) done (elapsed 3.61 s, remaining 0.52 s) 2024-08-30T09:00:43.676 INFO:teuthology.orchestra.run.smithi042.stderr:2900000 of 3200000 tuples (90%) done (elapsed 3.76 s, remaining 0.39 s) 2024-08-30T09:00:43.818 INFO:teuthology.orchestra.run.smithi042.stderr:3000000 of 3200000 tuples (93%) done (elapsed 3.91 s, remaining 0.26 s) 2024-08-30T09:00:43.946 INFO:teuthology.orchestra.run.smithi042.stderr:3100000 of 3200000 tuples (96%) done (elapsed 4.03 s, remaining 0.13 s) 2024-08-30T09:00:44.121 INFO:teuthology.orchestra.run.smithi042.stderr:3200000 of 3200000 tuples (100%) done (elapsed 4.21 s, remaining 0.00 s) 2024-08-30T09:00:44.136 INFO:teuthology.orchestra.run.smithi042.stderr:vacuuming... 2024-08-30T09:00:44.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:00:44 smithi190 ceph-mon[22001]: pgmap v372: 129 pgs: 129 active+clean; 148 MiB data, 621 MiB used, 1.0 TiB / 1.0 TiB avail; 342 KiB/s rd, 22 MiB/s wr, 97 op/s 2024-08-30T09:00:44.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:00:44 smithi190 ceph-mon[22001]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-30T09:00:44.741 INFO:teuthology.orchestra.run.smithi042.stderr:creating primary keys... 2024-08-30T09:00:44.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:00:44 smithi167 ceph-mon[22063]: pgmap v372: 129 pgs: 129 active+clean; 148 MiB data, 621 MiB used, 1.0 TiB / 1.0 TiB avail; 342 KiB/s rd, 22 MiB/s wr, 97 op/s 2024-08-30T09:00:44.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:00:44 smithi167 ceph-mon[22063]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-30T09:00:44.882 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:00:44 smithi042 ceph-mon[18652]: pgmap v372: 129 pgs: 129 active+clean; 148 MiB data, 621 MiB used, 1.0 TiB / 1.0 TiB avail; 342 KiB/s rd, 22 MiB/s wr, 97 op/s 2024-08-30T09:00:44.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:00:44 smithi042 ceph-mon[18652]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-30T09:00:45.893 INFO:teuthology.orchestra.run.smithi042.stderr:done in 6.01 s (drop tables 0.00 s, create tables 0.01 s, client-side generate 4.24 s, vacuum 0.61 s, primary keys 1.15 s). 2024-08-30T09:00:45.896 DEBUG:teuthology.orchestra.run.smithi042:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- pgbench -c 100 -j 4 --progress=5 --time=900' 2024-08-30T09:00:45.953 INFO:teuthology.orchestra.run.smithi042.stderr:starting vacuum...end. 2024-08-30T09:00:46.237 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.234+0000 7f61e99ad640 1 -- 172.21.15.42:0/1054623350 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f61e4101db0 msgr2=0x7f61e4102220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:00:46.237 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.234+0000 7f61e99ad640 1 --2- 172.21.15.42:0/1054623350 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f61e4101db0 0x7f61e4102220 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f61d400a910 tx=0x7f61d402a610 comp rx=0 tx=0).stop 2024-08-30T09:00:46.237 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.234+0000 7f61e99ad640 1 -- 172.21.15.42:0/1054623350 shutdown_connections 2024-08-30T09:00:46.237 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.234+0000 7f61e99ad640 1 --2- 172.21.15.42:0/1054623350 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f61e4102b60 0x7f61e4102f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:46.237 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.234+0000 7f61e99ad640 1 --2- 172.21.15.42:0/1054623350 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f61e41035c0 0x7f61e4109ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:46.237 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.234+0000 7f61e99ad640 1 --2- 172.21.15.42:0/1054623350 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f61e4101db0 0x7f61e4102220 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:46.237 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.234+0000 7f61e99ad640 1 -- 172.21.15.42:0/1054623350 >> 172.21.15.42:0/1054623350 conn(0x7f61e40fd010 msgr2=0x7f61e40ff480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:00:46.237 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.234+0000 7f61e99ad640 1 -- 172.21.15.42:0/1054623350 shutdown_connections 2024-08-30T09:00:46.237 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.235+0000 7f61e99ad640 1 -- 172.21.15.42:0/1054623350 wait complete. 2024-08-30T09:00:46.238 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.236+0000 7f61e99ad640 1 Processor -- start 2024-08-30T09:00:46.238 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.236+0000 7f61e99ad640 1 -- start start 2024-08-30T09:00:46.239 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.237+0000 7f61e99ad640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f61e4101db0 0x7f61e4112920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:46.239 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.237+0000 7f61e99ad640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f61e4102b60 0x7f61e4112ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:46.239 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.237+0000 7f61e99ad640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f61e41035c0 0x7f61e41114e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:46.239 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.237+0000 7f61e2ffd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f61e4101db0 0x7f61e4112920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:46.240 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.237+0000 7f61e2ffd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f61e4101db0 0x7f61e4112920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:39132/0 (socket says 172.21.15.42:39132) 2024-08-30T09:00:46.240 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.237+0000 7f61e2ffd640 1 -- 172.21.15.42:0/3516554644 learned_addr learned my addr 172.21.15.42:0/3516554644 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:00:46.240 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.237+0000 7f61e99ad640 1 -- 172.21.15.42:0/3516554644 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f61e410bab0 con 0x7f61e41035c0 2024-08-30T09:00:46.240 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.237+0000 7f61e99ad640 1 -- 172.21.15.42:0/3516554644 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f61e410b930 con 0x7f61e4102b60 2024-08-30T09:00:46.240 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.237+0000 7f61e99ad640 1 -- 172.21.15.42:0/3516554644 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f61e410bc30 con 0x7f61e4101db0 2024-08-30T09:00:46.240 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.237+0000 7f61e27fc640 1 --2- 172.21.15.42:0/3516554644 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f61e4102b60 0x7f61e4112ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:46.240 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.238+0000 7f61e37fe640 1 --2- 172.21.15.42:0/3516554644 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f61e41035c0 0x7f61e41114e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:46.240 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.238+0000 7f61e2ffd640 1 -- 172.21.15.42:0/3516554644 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f61e4102b60 msgr2=0x7f61e4112ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:00:46.241 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.238+0000 7f61e2ffd640 1 --2- 172.21.15.42:0/3516554644 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f61e4102b60 0x7f61e4112ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:46.241 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.238+0000 7f61e2ffd640 1 -- 172.21.15.42:0/3516554644 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f61e41035c0 msgr2=0x7f61e41114e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:00:46.241 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.238+0000 7f61e2ffd640 1 --2- 172.21.15.42:0/3516554644 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f61e41035c0 0x7f61e41114e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:46.241 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.238+0000 7f61e2ffd640 1 -- 172.21.15.42:0/3516554644 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61e4111a20 con 0x7f61e4101db0 2024-08-30T09:00:46.241 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.239+0000 7f61e2ffd640 1 --2- 172.21.15.42:0/3516554644 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f61e4101db0 0x7f61e4112920 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f61d400a8e0 tx=0x7f61d40316b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:00:46.242 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.239+0000 7f61e89ab640 1 -- 172.21.15.42:0/3516554644 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d4035070 con 0x7f61e4101db0 2024-08-30T09:00:46.242 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.239+0000 7f61e89ab640 1 -- 172.21.15.42:0/3516554644 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f61d4004460 con 0x7f61e4101db0 2024-08-30T09:00:46.242 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.239+0000 7f61e99ad640 1 -- 172.21.15.42:0/3516554644 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61e4111cb0 con 0x7f61e4101db0 2024-08-30T09:00:46.242 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.240+0000 7f61e99ad640 1 -- 172.21.15.42:0/3516554644 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61e41120e0 con 0x7f61e4101db0 2024-08-30T09:00:46.242 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.240+0000 7f61e89ab640 1 -- 172.21.15.42:0/3516554644 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d403c470 con 0x7f61e4101db0 2024-08-30T09:00:46.245 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.244+0000 7f61f09f7740 1 Processor -- start 2024-08-30T09:00:46.247 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.244+0000 7f61f09f7740 1 -- start start 2024-08-30T09:00:46.247 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.244+0000 7f61f09f7740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d03af1c1b0 0x55d03b2c1f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:46.247 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.244+0000 7f61f09f7740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d03aef32f0 0x55d03b2c25d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:46.247 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.244+0000 7f61f09f7740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d03aefb4d0 0x55d03b2c3120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:46.247 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.244+0000 7f61f09f7740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55d03b235000 con 0x55d03aef32f0 2024-08-30T09:00:46.247 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.244+0000 7f61f09f7740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55d03b224590 con 0x55d03aefb4d0 2024-08-30T09:00:46.247 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.244+0000 7f61f09f7740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55d03b228540 con 0x55d03af1c1b0 2024-08-30T09:00:46.247 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.245+0000 7f61e89ab640 1 -- 172.21.15.42:0/3516554644 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7f61d403c610 con 0x7f61e4101db0 2024-08-30T09:00:46.247 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.245+0000 7f61e37fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d03af1c1b0 0x55d03b2c1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:46.247 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.245+0000 7f61e37fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d03af1c1b0 0x55d03b2c1f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:39142/0 (socket says 172.21.15.42:39142) 2024-08-30T09:00:46.247 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.245+0000 7f61e37fe640 1 -- 172.21.15.42:0/1195247997 learned_addr learned my addr 172.21.15.42:0/1195247997 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:00:46.248 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.245+0000 7f61e27fc640 1 --2- 172.21.15.42:0/1195247997 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d03aefb4d0 0x55d03b2c3120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:46.248 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.245+0000 7f61e2ffd640 1 --2- 172.21.15.42:0/1195247997 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d03aef32f0 0x55d03b2c25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:46.248 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.245+0000 7f61e27fc640 1 -- 172.21.15.42:0/1195247997 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d03af1c1b0 msgr2=0x55d03b2c1f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:00:46.248 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.245+0000 7f61e27fc640 1 --2- 172.21.15.42:0/1195247997 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d03af1c1b0 0x55d03b2c1f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:46.248 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.245+0000 7f61e27fc640 1 -- 172.21.15.42:0/1195247997 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d03aef32f0 msgr2=0x55d03b2c25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:00:46.248 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.245+0000 7f61e27fc640 1 --2- 172.21.15.42:0/1195247997 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d03aef32f0 0x55d03b2c25d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:46.249 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.245+0000 7f61e27fc640 1 -- 172.21.15.42:0/1195247997 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d03af87790 con 0x55d03aefb4d0 2024-08-30T09:00:46.249 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.246+0000 7f61e27fc640 1 --2- 172.21.15.42:0/1195247997 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d03aefb4d0 0x55d03b2c3120 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f61d800dc10 tx=0x7f61d8005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d38eab8aa908df07 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:00:46.249 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.246+0000 7f61c9ffb640 1 -- 172.21.15.42:0/1195247997 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d800de20 con 0x55d03aefb4d0 2024-08-30T09:00:46.249 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.246+0000 7f61c9ffb640 1 -- 172.21.15.42:0/1195247997 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f61d80178f0 con 0x55d03aefb4d0 2024-08-30T09:00:46.249 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.246+0000 7f61c9ffb640 1 -- 172.21.15.42:0/1195247997 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d8016890 con 0x55d03aefb4d0 2024-08-30T09:00:46.250 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.248+0000 7f61f09f7740 1 -- 172.21.15.42:0/1195247997 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d03aefb4d0 msgr2=0x55d03b2c3120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:00:46.250 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.248+0000 7f61f09f7740 1 --2- 172.21.15.42:0/1195247997 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d03aefb4d0 0x55d03b2c3120 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f61d800dc10 tx=0x7f61d8005f90 comp rx=0 tx=0).stop 2024-08-30T09:00:46.250 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.248+0000 7f61f09f7740 1 -- 172.21.15.42:0/1195247997 shutdown_connections 2024-08-30T09:00:46.250 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.248+0000 7f61f09f7740 1 --2- 172.21.15.42:0/1195247997 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d03aefb4d0 0x55d03b2c3120 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:46.250 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.248+0000 7f61f09f7740 1 --2- 172.21.15.42:0/1195247997 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d03aef32f0 0x55d03b2c25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:46.250 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.248+0000 7f61f09f7740 1 --2- 172.21.15.42:0/1195247997 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d03af1c1b0 0x55d03b2c1f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:46.250 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.248+0000 7f61f09f7740 1 -- 172.21.15.42:0/1195247997 >> 172.21.15.42:0/1195247997 conn(0x55d03af10300 msgr2=0x55d03b2bd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:00:46.250 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.249+0000 7f61f09f7740 1 -- 172.21.15.42:0/1195247997 shutdown_connections 2024-08-30T09:00:46.251 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.249+0000 7f61f09f7740 1 -- 172.21.15.42:0/1195247997 wait complete. 2024-08-30T09:00:46.251 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.250+0000 7f61f09f7740 1 -- start start 2024-08-30T09:00:46.253 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.251+0000 7f61e89ab640 1 --2- 172.21.15.42:0/3516554644 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f61cc0b4c10 0x7f61cc0b7120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:46.253 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.251+0000 7f61e89ab640 1 -- 172.21.15.42:0/3516554644 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f61d4106660 con 0x7f61e4101db0 2024-08-30T09:00:46.253 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.252+0000 7f61f09f7740 1 Processor -- start 2024-08-30T09:00:46.254 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.252+0000 7f61e37fe640 1 --2- 172.21.15.42:0/3516554644 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f61cc0b4c10 0x7f61cc0b7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:46.255 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.253+0000 7f61e37fe640 1 --2- 172.21.15.42:0/3516554644 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f61cc0b4c10 0x7f61cc0b7120 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f61dc005fd0 tx=0x7f61dc00d040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:00:46.255 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.253+0000 7f61f09f7740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d03aef32f0 0x55d03b2dcab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:46.256 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.253+0000 7f61f09f7740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d03aefb4d0 0x55d03b2dd170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:46.256 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.254+0000 7f61f09f7740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d03af1c1b0 0x55d03b2ddce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:46.256 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.254+0000 7f61f09f7740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55d03b15dab0 con 0x55d03aefb4d0 2024-08-30T09:00:46.256 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.254+0000 7f61f09f7740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55d03b1ab680 con 0x55d03aef32f0 2024-08-30T09:00:46.256 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.254+0000 7f61e37fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d03af1c1b0 0x55d03b2ddce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:46.256 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.254+0000 7f61e37fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d03af1c1b0 0x55d03b2ddce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:39152/0 (socket says 172.21.15.42:39152) 2024-08-30T09:00:46.256 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.254+0000 7f61e37fe640 1 -- 172.21.15.42:0/1212242860 learned_addr learned my addr 172.21.15.42:0/1212242860 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:00:46.256 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.254+0000 7f61f09f7740 1 -- 172.21.15.42:0/1212242860 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55d03b2c3c40 con 0x55d03af1c1b0 2024-08-30T09:00:46.256 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.254+0000 7f61e2ffd640 1 --2- 172.21.15.42:0/1212242860 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d03aef32f0 0x55d03b2dcab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:46.257 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.254+0000 7f61f09f7740 10 client.0 _ll_register_callbacks cb 0x55d03af33a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:00:46.257 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.254+0000 7f61e27fc640 1 --2- 172.21.15.42:0/1212242860 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d03aefb4d0 0x55d03b2dd170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:46.257 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.254+0000 7f61e37fe640 1 -- 172.21.15.42:0/1212242860 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d03aef32f0 msgr2=0x55d03b2dcab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:00:46.257 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.254+0000 7f61e37fe640 1 --2- 172.21.15.42:0/1212242860 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d03aef32f0 0x55d03b2dcab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:46.258 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.254+0000 7f61e37fe640 1 -- 172.21.15.42:0/1212242860 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d03aefb4d0 msgr2=0x55d03b2dd170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:00:46.258 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.254+0000 7f61e37fe640 1 --2- 172.21.15.42:0/1212242860 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d03aefb4d0 0x55d03b2dd170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:46.258 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.254+0000 7f61e37fe640 1 -- 172.21.15.42:0/1212242860 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d03b000620 con 0x55d03af1c1b0 2024-08-30T09:00:46.258 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.255+0000 7f61e27fc640 1 --2- 172.21.15.42:0/1212242860 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d03aefb4d0 0x55d03b2dd170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-30T09:00:46.258 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.255+0000 7f61f09f7740 1 -- 172.21.15.42:0/1212242860 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f61d80178f0 con 0x55d03af1c1b0 2024-08-30T09:00:46.258 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.255+0000 7f61e37fe640 1 --2- 172.21.15.42:0/1212242860 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d03af1c1b0 0x55d03b2ddce0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f61dc00b440 tx=0x7f61dc004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=79b8f241536f5089 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:00:46.258 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.255+0000 7f61c2ffd640 10 client.25150 ms_handle_connect on v2:172.21.15.190:3300/0 2024-08-30T09:00:46.258 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.255+0000 7f61c2ffd640 1 -- 172.21.15.42:0/1212242860 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61dc028020 con 0x55d03af1c1b0 2024-08-30T09:00:46.258 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.255+0000 7f61c2ffd640 1 -- 172.21.15.42:0/1212242860 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f61dc003b70 con 0x55d03af1c1b0 2024-08-30T09:00:46.258 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.255+0000 7f61c2ffd640 1 -- 172.21.15.42:0/1212242860 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f61dc003d90 con 0x55d03af1c1b0 2024-08-30T09:00:46.259 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.256+0000 7f61f09f7740 10 client.25150 fetch_fsmap learned FSMap version 28 2024-08-30T09:00:46.259 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.256+0000 7f61f09f7740 1 -- 172.21.15.42:0/1212242860 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({fsmap=28}) -- 0x55d03b2e6c80 con 0x55d03af1c1b0 2024-08-30T09:00:46.259 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.257+0000 7f61c2ffd640 1 -- 172.21.15.42:0/1212242860 <== mon.1 v2:172.21.15.190:3300/0 4 ==== fsmap(e 28) ==== 8358+0+0 (secure 0 0 0) 0x7f61dc003f30 con 0x55d03af1c1b0 2024-08-30T09:00:46.259 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.257+0000 7f61f09f7740 10 client.25150 fetch_fsmap finished waiting for FSMap version 28 2024-08-30T09:00:46.259 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.257+0000 7f61f09f7740 10 client.25150 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-30T09:00:46.259 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.257+0000 7f61f09f7740 20 client.25150 populate_metadata read hostname 'smithi042' 2024-08-30T09:00:46.259 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.257+0000 7f61f09f7740 1 --2- 172.21.15.42:0/1212242860 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55d03aedad80 0x55d03b2ea6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:46.259 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.257+0000 7f61f09f7740 4 client.25150 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-30T09:00:46.259 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.257+0000 7f61f09f7740 1 -- 172.21.15.42:0/1212242860 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d03b236f50 con 0x55d03aedad80 2024-08-30T09:00:46.260 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.258+0000 7f61e2ffd640 1 --2- 172.21.15.42:0/1212242860 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55d03aedad80 0x55d03b2ea6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:46.264 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.259+0000 7f61c2ffd640 10 client.25150 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:00:46.264 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.259+0000 7f61e2ffd640 1 --2- 172.21.15.42:0/1212242860 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55d03aedad80 0x55d03b2ea6a0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=22c1f7884dbccc43 server_cookie=45b5019c46269931 in_seq=0 out_seq=0 2024-08-30T09:00:46.265 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.260+0000 7f61c2ffd640 1 -- 172.21.15.42:0/1212242860 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d03b236f50 con 0x55d03aedad80 2024-08-30T09:00:46.265 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.261+0000 7f61c2ffd640 10 client.25150 handle_command_reply: tid=0 2024-08-30T09:00:46.265 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.261+0000 7f61f09f7740 1 client.25150 shutdown 2024-08-30T09:00:46.265 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.261+0000 7f61f09f7740 1 -- 172.21.15.42:0/1212242860 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d03af1c1b0 msgr2=0x55d03b2ddce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:00:46.265 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.261+0000 7f61f09f7740 1 --2- 172.21.15.42:0/1212242860 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d03af1c1b0 0x55d03b2ddce0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f61dc00b440 tx=0x7f61dc004ab0 comp rx=0 tx=0).stop 2024-08-30T09:00:46.266 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.262+0000 7f61f09f7740 1 -- 172.21.15.42:0/1212242860 shutdown_connections 2024-08-30T09:00:46.266 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.262+0000 7f61f09f7740 1 --2- 172.21.15.42:0/1212242860 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55d03aedad80 0x55d03b2ea6a0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:46.266 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.262+0000 7f61f09f7740 1 --2- 172.21.15.42:0/1212242860 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d03af1c1b0 0x55d03b2ddce0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:46.266 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.262+0000 7f61f09f7740 1 --2- 172.21.15.42:0/1212242860 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d03aefb4d0 0x55d03b2dd170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:46.266 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.262+0000 7f61f09f7740 1 --2- 172.21.15.42:0/1212242860 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d03aef32f0 0x55d03b2dcab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:46.266 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.262+0000 7f61c2ffd640 0 client.25150 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:00:46.266 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.262+0000 7f61f09f7740 1 -- 172.21.15.42:0/1212242860 >> 172.21.15.42:0/1212242860 conn(0x55d03af10300 msgr2=0x55d03b2bbc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:00:46.266 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.262+0000 7f61f09f7740 1 -- 172.21.15.42:0/1212242860 shutdown_connections 2024-08-30T09:00:46.266 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.262+0000 7f61f09f7740 1 -- 172.21.15.42:0/1212242860 wait complete. 2024-08-30T09:00:46.266 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.262+0000 7f61f09f7740 20 client.25150 trim_cache size 0 max 16384 2024-08-30T09:00:46.292 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.288+0000 7f61f09f7740 1 Processor -- start 2024-08-30T09:00:46.292 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.288+0000 7f61f09f7740 1 -- start start 2024-08-30T09:00:46.292 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.288+0000 7f61f09f7740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d03aef32f0 0x55d03b2c6950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:46.292 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.288+0000 7f61f09f7740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d03aefb4d0 0x55d03b2c6e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:46.292 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.288+0000 7f61f09f7740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d03af1c1b0 0x55d03b2db170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:46.292 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.288+0000 7f61f09f7740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f61d0012470 con 0x55d03af1c1b0 2024-08-30T09:00:46.292 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.288+0000 7f61f09f7740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f61d000fd40 con 0x55d03aefb4d0 2024-08-30T09:00:46.292 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.288+0000 7f61f09f7740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f61d000db90 con 0x55d03aef32f0 2024-08-30T09:00:46.292 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.289+0000 7f61e37fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d03af1c1b0 0x55d03b2db170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:46.292 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.289+0000 7f61e37fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d03af1c1b0 0x55d03b2db170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34620/0 (socket says 172.21.15.42:34620) 2024-08-30T09:00:46.292 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.289+0000 7f61e37fe640 1 -- 172.21.15.42:0/3167213520 learned_addr learned my addr 172.21.15.42:0/3167213520 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:00:46.292 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.289+0000 7f61e27fc640 1 --2- 172.21.15.42:0/3167213520 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d03aefb4d0 0x55d03b2c6e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:46.293 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.289+0000 7f61e37fe640 1 -- 172.21.15.42:0/3167213520 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d03aef32f0 msgr2=0x55d03b2c6950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:00:46.293 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.289+0000 7f61e37fe640 1 --2- 172.21.15.42:0/3167213520 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d03aef32f0 0x55d03b2c6950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:46.293 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.290+0000 7f61e37fe640 1 -- 172.21.15.42:0/3167213520 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d03aefb4d0 msgr2=0x55d03b2c6e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:00:46.293 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.290+0000 7f61e37fe640 1 --2- 172.21.15.42:0/3167213520 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d03aefb4d0 0x55d03b2c6e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:46.293 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.290+0000 7f61e37fe640 1 -- 172.21.15.42:0/3167213520 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d03aedafb0 con 0x55d03af1c1b0 2024-08-30T09:00:46.293 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.290+0000 7f61e37fe640 1 --2- 172.21.15.42:0/3167213520 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d03af1c1b0 0x55d03b2db170 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f61dc005f90 tx=0x7f61dc024000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a77ad49b444f7ec9 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:00:46.294 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.290+0000 7f61c97fa640 1 -- 172.21.15.42:0/3167213520 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61dc028070 con 0x55d03af1c1b0 2024-08-30T09:00:46.294 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.290+0000 7f61c97fa640 1 -- 172.21.15.42:0/3167213520 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f61d80178f0 con 0x55d03af1c1b0 2024-08-30T09:00:46.294 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.290+0000 7f61c97fa640 1 -- 172.21.15.42:0/3167213520 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61dc005030 con 0x55d03af1c1b0 2024-08-30T09:00:46.294 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.292+0000 7f61f09f7740 1 -- 172.21.15.42:0/3167213520 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d03af1c1b0 msgr2=0x55d03b2db170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:00:46.294 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.292+0000 7f61f09f7740 1 --2- 172.21.15.42:0/3167213520 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d03af1c1b0 0x55d03b2db170 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f61dc005f90 tx=0x7f61dc024000 comp rx=0 tx=0).stop 2024-08-30T09:00:46.294 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.292+0000 7f61f09f7740 1 -- 172.21.15.42:0/3167213520 shutdown_connections 2024-08-30T09:00:46.294 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.292+0000 7f61f09f7740 1 --2- 172.21.15.42:0/3167213520 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d03af1c1b0 0x55d03b2db170 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:46.294 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.292+0000 7f61f09f7740 1 --2- 172.21.15.42:0/3167213520 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d03aefb4d0 0x55d03b2c6e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:46.294 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.292+0000 7f61f09f7740 1 --2- 172.21.15.42:0/3167213520 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d03aef32f0 0x55d03b2c6950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:46.295 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.292+0000 7f61f09f7740 1 -- 172.21.15.42:0/3167213520 >> 172.21.15.42:0/3167213520 conn(0x55d03af10300 msgr2=0x55d03b2ea5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:00:46.295 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.292+0000 7f61f09f7740 1 -- 172.21.15.42:0/3167213520 shutdown_connections 2024-08-30T09:00:46.295 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.292+0000 7f61f09f7740 1 -- 172.21.15.42:0/3167213520 wait complete. 2024-08-30T09:00:46.295 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.292+0000 7f61f09f7740 1 -- start start 2024-08-30T09:00:46.296 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.293+0000 7f61f09f7740 1 Processor -- start 2024-08-30T09:00:46.296 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.293+0000 7f61f09f7740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d03aef32f0 0x55d03b2e1290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:46.296 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.293+0000 7f61f09f7740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d03aefb4d0 0x55d03b2e1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:46.296 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.293+0000 7f61f09f7740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d03af1c1b0 0x55d03b2e2450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:46.296 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.293+0000 7f61f09f7740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f61d000e200 con 0x55d03af1c1b0 2024-08-30T09:00:46.296 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.293+0000 7f61f09f7740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f61d000f670 con 0x55d03aef32f0 2024-08-30T09:00:46.296 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.293+0000 7f61f09f7740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f61d000e8d0 con 0x55d03aefb4d0 2024-08-30T09:00:46.296 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.293+0000 7f61f09f7740 10 client.0 _ll_register_callbacks cb 0x7f61dc01b070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:00:46.296 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.293+0000 7f61e27fc640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d03aefb4d0 0x55d03b2e1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:46.296 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.293+0000 7f61e27fc640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d03aefb4d0 0x55d03b2e1950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:39178/0 (socket says 172.21.15.42:39178) 2024-08-30T09:00:46.296 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.294+0000 7f61e27fc640 1 -- 172.21.15.42:0/2090037061 learned_addr learned my addr 172.21.15.42:0/2090037061 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:00:46.297 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.294+0000 7f61e2ffd640 1 --2- 172.21.15.42:0/2090037061 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d03aef32f0 0x55d03b2e1290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:46.297 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.294+0000 7f61e27fc640 1 -- 172.21.15.42:0/2090037061 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d03aef32f0 msgr2=0x55d03b2e1290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:00:46.297 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.294+0000 7f61e27fc640 1 --2- 172.21.15.42:0/2090037061 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d03aef32f0 0x55d03b2e1290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:46.297 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.294+0000 7f61e27fc640 1 -- 172.21.15.42:0/2090037061 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d03af1c1b0 msgr2=0x55d03b2e2450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:00:46.297 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.294+0000 7f61e27fc640 1 --2- 172.21.15.42:0/2090037061 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d03af1c1b0 0x55d03b2e2450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:46.297 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.294+0000 7f61e27fc640 1 -- 172.21.15.42:0/2090037061 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d03b2e2bb0 con 0x55d03aefb4d0 2024-08-30T09:00:46.297 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.295+0000 7f61e27fc640 1 --2- 172.21.15.42:0/2090037061 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d03aefb4d0 0x55d03b2e1950 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f61d800dbe0 tx=0x7f61d8017440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cda7f3cee452ca47 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:00:46.297 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.295+0000 7f61c2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-08-30T09:00:46.298 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.296+0000 7f61c2ffd640 1 -- 172.21.15.42:0/2090037061 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d8017ca0 con 0x55d03aefb4d0 2024-08-30T09:00:46.298 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.296+0000 7f61f09f7740 1 -- 172.21.15.42:0/2090037061 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f61d80178f0 con 0x55d03aefb4d0 2024-08-30T09:00:46.298 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.296+0000 7f61c2ffd640 1 -- 172.21.15.42:0/2090037061 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f61d8005040 con 0x55d03aefb4d0 2024-08-30T09:00:46.298 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.297+0000 7f61c2ffd640 1 -- 172.21.15.42:0/2090037061 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f61d8034050 con 0x55d03aefb4d0 2024-08-30T09:00:46.300 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.298+0000 7f61f09f7740 10 client.25153 fetch_fsmap learned FSMap version 28 2024-08-30T09:00:46.300 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.298+0000 7f61f09f7740 1 -- 172.21.15.42:0/2090037061 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d03b2e3f40 con 0x55d03aefb4d0 2024-08-30T09:00:46.300 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.298+0000 7f61c2ffd640 1 -- 172.21.15.42:0/2090037061 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f61d80178f0 con 0x55d03aefb4d0 2024-08-30T09:00:46.301 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.299+0000 7f61c2ffd640 1 -- 172.21.15.42:0/2090037061 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 28) ==== 8358+0+0 (secure 0 0 0) 0x7f61d8003ba0 con 0x55d03aefb4d0 2024-08-30T09:00:46.301 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.299+0000 7f61f09f7740 10 client.25153 fetch_fsmap finished waiting for FSMap version 28 2024-08-30T09:00:46.301 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.299+0000 7f61f09f7740 10 client.25153 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-30T09:00:46.301 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.299+0000 7f61f09f7740 20 client.25153 populate_metadata read hostname 'smithi042' 2024-08-30T09:00:46.301 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.299+0000 7f61f09f7740 1 --2- 172.21.15.42:0/2090037061 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55d03aeb01c0 0x55d03b2d2110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:46.301 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.299+0000 7f61f09f7740 4 client.25153 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-30T09:00:46.301 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.299+0000 7f61f09f7740 1 -- 172.21.15.42:0/2090037061 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d03b2329f0 con 0x55d03aeb01c0 2024-08-30T09:00:46.302 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.300+0000 7f61e2ffd640 1 --2- 172.21.15.42:0/2090037061 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55d03aeb01c0 0x55d03b2d2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:46.304 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.302+0000 7f61e2ffd640 1 --2- 172.21.15.42:0/2090037061 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55d03aeb01c0 0x55d03b2d2110 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f5ed38fffac14865 server_cookie=90224f0a385f5647 in_seq=0 out_seq=0 2024-08-30T09:00:46.304 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.302+0000 7f61c2ffd640 10 client.25153 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:00:46.614 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.612+0000 7f61c2ffd640 1 -- 172.21.15.42:0/2090037061 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d03b2329f0 con 0x55d03aeb01c0 2024-08-30T09:00:46.614 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.612+0000 7f61c2ffd640 10 client.25153 handle_command_reply: tid=0 2024-08-30T09:00:46.614 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.612+0000 7f61f09f7740 1 client.25153 shutdown 2024-08-30T09:00:46.615 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.612+0000 7f61f09f7740 1 -- 172.21.15.42:0/2090037061 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d03aefb4d0 msgr2=0x55d03b2e1950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:00:46.615 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.612+0000 7f61f09f7740 1 --2- 172.21.15.42:0/2090037061 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d03aefb4d0 0x55d03b2e1950 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f61d800dbe0 tx=0x7f61d8017440 comp rx=0 tx=0).stop 2024-08-30T09:00:46.615 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.612+0000 7f61f09f7740 1 -- 172.21.15.42:0/2090037061 shutdown_connections 2024-08-30T09:00:46.615 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.612+0000 7f61f09f7740 1 --2- 172.21.15.42:0/2090037061 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55d03aeb01c0 0x55d03b2d2110 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:46.615 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.612+0000 7f61f09f7740 1 --2- 172.21.15.42:0/2090037061 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d03aefb4d0 0x55d03b2e1950 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:46.615 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.612+0000 7f61c2ffd640 0 client.25153 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:00:46.615 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.612+0000 7f61f09f7740 1 --2- 172.21.15.42:0/2090037061 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d03af1c1b0 0x55d03b2e2450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:46.615 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.612+0000 7f61f09f7740 1 --2- 172.21.15.42:0/2090037061 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d03aef32f0 0x55d03b2e1290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:46.615 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.612+0000 7f61f09f7740 1 -- 172.21.15.42:0/2090037061 >> 172.21.15.42:0/2090037061 conn(0x55d03af10300 msgr2=0x55d03b2ea5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:00:46.615 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.612+0000 7f61f09f7740 1 -- 172.21.15.42:0/2090037061 shutdown_connections 2024-08-30T09:00:46.615 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.612+0000 7f61f09f7740 1 -- 172.21.15.42:0/2090037061 wait complete. 2024-08-30T09:00:46.615 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.612+0000 7f61f09f7740 20 client.25153 trim_cache size 0 max 16384 2024-08-30T09:00:46.616 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-08-30T09:00:46.616 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2024-08-30T09:00:46.616 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-08-30T09:00:46.616 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-08-30T09:00:46.617 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.613+0000 7f61e99ad640 1 -- 172.21.15.42:0/3516554644 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f61cc0b4c10 msgr2=0x7f61cc0b7120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:00:46.617 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.613+0000 7f61e99ad640 1 --2- 172.21.15.42:0/3516554644 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f61cc0b4c10 0x7f61cc0b7120 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f61dc005fd0 tx=0x7f61dc00d040 comp rx=0 tx=0).stop 2024-08-30T09:00:46.617 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.613+0000 7f61e99ad640 1 -- 172.21.15.42:0/3516554644 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f61e4101db0 msgr2=0x7f61e4112920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:00:46.617 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.613+0000 7f61e99ad640 1 --2- 172.21.15.42:0/3516554644 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f61e4101db0 0x7f61e4112920 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f61d400a8e0 tx=0x7f61d40316b0 comp rx=0 tx=0).stop 2024-08-30T09:00:46.617 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.613+0000 7f61e99ad640 1 -- 172.21.15.42:0/3516554644 shutdown_connections 2024-08-30T09:00:46.617 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.613+0000 7f61e99ad640 1 --2- 172.21.15.42:0/3516554644 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f61cc0b4c10 0x7f61cc0b7120 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:46.617 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.613+0000 7f61e99ad640 1 --2- 172.21.15.42:0/3516554644 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f61e41035c0 0x7f61e41114e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:46.617 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.613+0000 7f61e99ad640 1 --2- 172.21.15.42:0/3516554644 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f61e4102b60 0x7f61e4112ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:46.617 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.613+0000 7f61e99ad640 1 --2- 172.21.15.42:0/3516554644 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f61e4101db0 0x7f61e4112920 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:46.617 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.613+0000 7f61e99ad640 1 -- 172.21.15.42:0/3516554644 >> 172.21.15.42:0/3516554644 conn(0x7f61e40fd010 msgr2=0x7f61e4105e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:00:46.618 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.614+0000 7f61e99ad640 1 -- 172.21.15.42:0/3516554644 shutdown_connections 2024-08-30T09:00:46.618 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:46.614+0000 7f61e99ad640 1 -- 172.21.15.42:0/3516554644 wait complete. 2024-08-30T09:00:46.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:00:46 smithi190 ceph-mon[22001]: pgmap v373: 129 pgs: 129 active+clean; 208 MiB data, 763 MiB used, 1.0 TiB / 1.0 TiB avail; 341 KiB/s rd, 32 MiB/s wr, 109 op/s 2024-08-30T09:00:46.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:00:46 smithi167 ceph-mon[22063]: pgmap v373: 129 pgs: 129 active+clean; 208 MiB data, 763 MiB used, 1.0 TiB / 1.0 TiB avail; 341 KiB/s rd, 32 MiB/s wr, 109 op/s 2024-08-30T09:00:46.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:00:46 smithi042 ceph-mon[18652]: pgmap v373: 129 pgs: 129 active+clean; 208 MiB data, 763 MiB used, 1.0 TiB / 1.0 TiB avail; 341 KiB/s rd, 32 MiB/s wr, 109 op/s 2024-08-30T09:00:48.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:00:48 smithi190 ceph-mon[22001]: pgmap v374: 129 pgs: 129 active+clean; 369 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 341 KiB/s rd, 56 MiB/s wr, 210 op/s 2024-08-30T09:00:48.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:00:48 smithi190 ceph-mon[22001]: scrub summary: idle 2024-08-30T09:00:48.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:00:48 smithi167 ceph-mon[22063]: pgmap v374: 129 pgs: 129 active+clean; 369 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 341 KiB/s rd, 56 MiB/s wr, 210 op/s 2024-08-30T09:00:48.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:00:48 smithi167 ceph-mon[22063]: scrub summary: idle 2024-08-30T09:00:48.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:00:48 smithi042 ceph-mon[18652]: pgmap v374: 129 pgs: 129 active+clean; 369 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 341 KiB/s rd, 56 MiB/s wr, 210 op/s 2024-08-30T09:00:48.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:00:48 smithi042 ceph-mon[18652]: scrub summary: idle 2024-08-30T09:00:49.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:00:49 smithi190 ceph-mon[22001]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-30T09:00:49.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:00:49 smithi167 ceph-mon[22063]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-30T09:00:49.882 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:00:49 smithi042 ceph-mon[18652]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-30T09:00:50.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:00:50 smithi167 ceph-mon[22063]: pgmap v375: 129 pgs: 129 active+clean; 489 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 400 KiB/s rd, 73 MiB/s wr, 349 op/s 2024-08-30T09:00:50.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:00:50 smithi042 ceph-mon[18652]: pgmap v375: 129 pgs: 129 active+clean; 489 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 400 KiB/s rd, 73 MiB/s wr, 349 op/s 2024-08-30T09:00:50.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:00:50 smithi190 ceph-mon[22001]: pgmap v375: 129 pgs: 129 active+clean; 489 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 400 KiB/s rd, 73 MiB/s wr, 349 op/s 2024-08-30T09:00:50.953 INFO:teuthology.orchestra.run.smithi042.stderr:progress: 5.0 s, 3733.4 tps, lat 26.084 ms stddev 36.019 2024-08-30T09:00:52.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:00:52 smithi167 ceph-mon[22063]: pgmap v376: 129 pgs: 129 active+clean; 551 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 341 KiB/s rd, 76 MiB/s wr, 390 op/s 2024-08-30T09:00:52.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:00:52 smithi042 ceph-mon[18652]: pgmap v376: 129 pgs: 129 active+clean; 551 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 341 KiB/s rd, 76 MiB/s wr, 390 op/s 2024-08-30T09:00:52.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:00:52 smithi190 ceph-mon[22001]: pgmap v376: 129 pgs: 129 active+clean; 551 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 341 KiB/s rd, 76 MiB/s wr, 390 op/s 2024-08-30T09:00:54.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:00:54 smithi167 ceph-mon[22063]: pgmap v377: 129 pgs: 129 active+clean; 841 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 352 KiB/s rd, 77 MiB/s wr, 627 op/s 2024-08-30T09:00:54.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:00:54 smithi042 ceph-mon[18652]: pgmap v377: 129 pgs: 129 active+clean; 841 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 352 KiB/s rd, 77 MiB/s wr, 627 op/s 2024-08-30T09:00:54.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:00:54 smithi190 ceph-mon[22001]: pgmap v377: 129 pgs: 129 active+clean; 841 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 352 KiB/s rd, 77 MiB/s wr, 627 op/s 2024-08-30T09:00:55.953 INFO:teuthology.orchestra.run.smithi042.stderr:progress: 10.0 s, 3562.1 tps, lat 27.975 ms stddev 31.124 2024-08-30T09:00:56.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:00:56 smithi167 ceph-mon[22063]: pgmap v378: 129 pgs: 129 active+clean; 888 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 57 MiB/s wr, 592 op/s 2024-08-30T09:00:56.882 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:00:56 smithi042 ceph-mon[18652]: pgmap v378: 129 pgs: 129 active+clean; 888 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 57 MiB/s wr, 592 op/s 2024-08-30T09:00:56.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:00:56 smithi190 ceph-mon[22001]: pgmap v378: 129 pgs: 129 active+clean; 888 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 57 MiB/s wr, 592 op/s 2024-08-30T09:00:58.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:00:58 smithi167 ceph-mon[22063]: pgmap v379: 129 pgs: 129 active+clean; 908 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 49 MiB/s wr, 691 op/s 2024-08-30T09:00:58.882 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:00:58 smithi042 ceph-mon[18652]: pgmap v379: 129 pgs: 129 active+clean; 908 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 49 MiB/s wr, 691 op/s 2024-08-30T09:00:58.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:00:58 smithi190 ceph-mon[22001]: pgmap v379: 129 pgs: 129 active+clean; 908 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 49 MiB/s wr, 691 op/s 2024-08-30T09:00:59.770 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.768+0000 7f8696fb6640 1 -- 172.21.15.42:0/1349131730 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8690109180 msgr2=0x7f869010b690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:00:59.770 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.768+0000 7f8696fb6640 1 --2- 172.21.15.42:0/1349131730 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8690109180 0x7f869010b690 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f8688009c90 tx=0x7f868802a860 comp rx=0 tx=0).stop 2024-08-30T09:00:59.771 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.768+0000 7f8696fb6640 1 -- 172.21.15.42:0/1349131730 shutdown_connections 2024-08-30T09:00:59.771 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.768+0000 7f8696fb6640 1 --2- 172.21.15.42:0/1349131730 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f86901065c0 0x7f8690108a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:59.771 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.768+0000 7f8696fb6640 1 --2- 172.21.15.42:0/1349131730 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8690109180 0x7f869010b690 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:59.771 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.768+0000 7f8696fb6640 1 --2- 172.21.15.42:0/1349131730 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f869006b4b0 0x7f8690106080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:59.771 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.768+0000 7f8696fb6640 1 -- 172.21.15.42:0/1349131730 >> 172.21.15.42:0/1349131730 conn(0x7f86900fdc80 msgr2=0x7f8690100130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:00:59.771 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.768+0000 7f8696fb6640 1 -- 172.21.15.42:0/1349131730 shutdown_connections 2024-08-30T09:00:59.771 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.769+0000 7f8696fb6640 1 -- 172.21.15.42:0/1349131730 wait complete. 2024-08-30T09:00:59.771 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.769+0000 7f8696fb6640 1 Processor -- start 2024-08-30T09:00:59.772 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.769+0000 7f8696fb6640 1 -- start start 2024-08-30T09:00:59.772 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.770+0000 7f8696fb6640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f869006b4b0 0x7f86901a0330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:59.772 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.770+0000 7f8696fb6640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f86901065c0 0x7f86901a09a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:59.772 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.770+0000 7f8696fb6640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8690109180 0x7f86901a1550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:59.772 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.770+0000 7f868ffff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f86901065c0 0x7f86901a09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:59.772 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.770+0000 7f868ffff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f86901065c0 0x7f86901a09a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:41490/0 (socket says 172.21.15.42:41490) 2024-08-30T09:00:59.772 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.770+0000 7f868ffff640 1 -- 172.21.15.42:0/4239431985 learned_addr learned my addr 172.21.15.42:0/4239431985 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:00:59.772 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.770+0000 7f8694d52640 1 --2- 172.21.15.42:0/4239431985 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f869006b4b0 0x7f86901a0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:59.772 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.770+0000 7f8695553640 1 --2- 172.21.15.42:0/4239431985 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8690109180 0x7f86901a1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:59.773 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.770+0000 7f8696fb6640 1 -- 172.21.15.42:0/4239431985 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f869010cf00 con 0x7f869006b4b0 2024-08-30T09:00:59.773 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.770+0000 7f8696fb6640 1 -- 172.21.15.42:0/4239431985 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f869010cd80 con 0x7f86901065c0 2024-08-30T09:00:59.773 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.770+0000 7f8696fb6640 1 -- 172.21.15.42:0/4239431985 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f869010d080 con 0x7f8690109180 2024-08-30T09:00:59.773 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.771+0000 7f868ffff640 1 -- 172.21.15.42:0/4239431985 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8690109180 msgr2=0x7f86901a1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:00:59.774 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.771+0000 7f868ffff640 1 --2- 172.21.15.42:0/4239431985 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8690109180 0x7f86901a1550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:59.774 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.771+0000 7f868ffff640 1 -- 172.21.15.42:0/4239431985 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f869006b4b0 msgr2=0x7f86901a0330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:00:59.774 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.771+0000 7f868ffff640 1 --2- 172.21.15.42:0/4239431985 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f869006b4b0 0x7f86901a0330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:59.774 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.771+0000 7f868ffff640 1 -- 172.21.15.42:0/4239431985 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86901a1d70 con 0x7f86901065c0 2024-08-30T09:00:59.774 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.771+0000 7f8695553640 1 --2- 172.21.15.42:0/4239431985 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8690109180 0x7f86901a1550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-30T09:00:59.774 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.771+0000 7f8694d52640 1 --2- 172.21.15.42:0/4239431985 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f869006b4b0 0x7f86901a0330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-30T09:00:59.774 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.771+0000 7f868ffff640 1 --2- 172.21.15.42:0/4239431985 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f86901065c0 0x7f86901a09a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f867c00d770 tx=0x7f867c00dc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:00:59.774 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.771+0000 7f868dffb640 1 -- 172.21.15.42:0/4239431985 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f867c017070 con 0x7f86901065c0 2024-08-30T09:00:59.774 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.771+0000 7f8696fb6640 1 -- 172.21.15.42:0/4239431985 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86901a2060 con 0x7f86901065c0 2024-08-30T09:00:59.774 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.771+0000 7f868dffb640 1 -- 172.21.15.42:0/4239431985 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f867c009c30 con 0x7f86901065c0 2024-08-30T09:00:59.774 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.771+0000 7f868dffb640 1 -- 172.21.15.42:0/4239431985 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f867c010e60 con 0x7f86901065c0 2024-08-30T09:00:59.774 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.772+0000 7f8696fb6640 1 -- 172.21.15.42:0/4239431985 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86901a8750 con 0x7f86901065c0 2024-08-30T09:00:59.779 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.776+0000 7f869dfff740 1 Processor -- start 2024-08-30T09:00:59.779 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.776+0000 7f868dffb640 1 -- 172.21.15.42:0/4239431985 <== mon.2 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7f867c01a070 con 0x7f86901065c0 2024-08-30T09:00:59.779 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.776+0000 7f869dfff740 1 -- start start 2024-08-30T09:00:59.779 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.777+0000 7f869dfff740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564c90fc51b0 0x564c9136af60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:59.779 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.777+0000 7f869dfff740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564c90f9c2f0 0x564c9136b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:59.779 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.777+0000 7f8695553640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564c90fc51b0 0x564c9136af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:59.779 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.777+0000 7f8695553640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564c90fc51b0 0x564c9136af60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:41496/0 (socket says 172.21.15.42:41496) 2024-08-30T09:00:59.780 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.777+0000 7f869dfff740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564c90fa44d0 0x564c9136c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:59.781 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.777+0000 7f869dfff740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x564c912de000 con 0x564c90f9c2f0 2024-08-30T09:00:59.781 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.777+0000 7f869dfff740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x564c912cd590 con 0x564c90fc51b0 2024-08-30T09:00:59.781 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.777+0000 7f869dfff740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x564c912d1540 con 0x564c90fa44d0 2024-08-30T09:00:59.781 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.777+0000 7f8694d52640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564c90f9c2f0 0x564c9136b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:59.781 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.778+0000 7f8694d52640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564c90f9c2f0 0x564c9136b5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:46442/0 (socket says 172.21.15.42:46442) 2024-08-30T09:00:59.781 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.778+0000 7f8694d52640 1 -- 172.21.15.42:0/513115388 learned_addr learned my addr 172.21.15.42:0/513115388 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:00:59.781 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.778+0000 7f868ffff640 1 --2- 172.21.15.42:0/513115388 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564c90fa44d0 0x564c9136c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:59.781 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.778+0000 7f8695553640 1 -- 172.21.15.42:0/513115388 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564c90fa44d0 msgr2=0x564c9136c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:00:59.781 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.778+0000 7f8695553640 1 --2- 172.21.15.42:0/513115388 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564c90fa44d0 0x564c9136c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:59.781 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.778+0000 7f8695553640 1 -- 172.21.15.42:0/513115388 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564c90f9c2f0 msgr2=0x564c9136b5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:00:59.781 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.778+0000 7f8695553640 1 --2- 172.21.15.42:0/513115388 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564c90f9c2f0 0x564c9136b5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:59.781 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.778+0000 7f8695553640 1 -- 172.21.15.42:0/513115388 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564c91030790 con 0x564c90fc51b0 2024-08-30T09:00:59.781 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.778+0000 7f868ffff640 1 --2- 172.21.15.42:0/513115388 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564c90fa44d0 0x564c9136c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-30T09:00:59.782 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.778+0000 7f8695553640 1 --2- 172.21.15.42:0/513115388 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564c90fc51b0 0x564c9136af60 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f868802ae50 tx=0x7f8688009370 comp rx=0 tx=0).ready entity=mon.2 client_cookie=faede89ec6d26e7d server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:00:59.782 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.779+0000 7f867affd640 1 -- 172.21.15.42:0/513115388 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8688031e30 con 0x564c90fc51b0 2024-08-30T09:00:59.782 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.779+0000 7f867affd640 1 -- 172.21.15.42:0/513115388 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8688036030 con 0x564c90fc51b0 2024-08-30T09:00:59.783 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.779+0000 7f867affd640 1 -- 172.21.15.42:0/513115388 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8688005040 con 0x564c90fc51b0 2024-08-30T09:00:59.783 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.779+0000 7f868dffb640 1 --2- 172.21.15.42:0/4239431985 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f86800b4b30 0x7f86800b7040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:59.783 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.779+0000 7f8694d52640 1 --2- 172.21.15.42:0/4239431985 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f86800b4b30 0x7f86800b7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:59.783 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.780+0000 7f868dffb640 1 -- 172.21.15.42:0/4239431985 <== mon.2 v2:172.21.15.167:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f867c0e4ff0 con 0x7f86901065c0 2024-08-30T09:00:59.783 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.780+0000 7f869dfff740 1 -- 172.21.15.42:0/513115388 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564c90fc51b0 msgr2=0x564c9136af60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:00:59.783 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.780+0000 7f869dfff740 1 --2- 172.21.15.42:0/513115388 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564c90fc51b0 0x564c9136af60 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f868802ae50 tx=0x7f8688009370 comp rx=0 tx=0).stop 2024-08-30T09:00:59.783 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.780+0000 7f869dfff740 1 -- 172.21.15.42:0/513115388 shutdown_connections 2024-08-30T09:00:59.784 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.780+0000 7f869dfff740 1 --2- 172.21.15.42:0/513115388 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564c90fa44d0 0x564c9136c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:59.784 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.780+0000 7f869dfff740 1 --2- 172.21.15.42:0/513115388 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564c90f9c2f0 0x564c9136b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:59.784 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.780+0000 7f869dfff740 1 --2- 172.21.15.42:0/513115388 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564c90fc51b0 0x564c9136af60 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:59.784 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.780+0000 7f869dfff740 1 -- 172.21.15.42:0/513115388 >> 172.21.15.42:0/513115388 conn(0x564c90fb9300 msgr2=0x564c913669f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:00:59.784 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.780+0000 7f869dfff740 1 -- 172.21.15.42:0/513115388 shutdown_connections 2024-08-30T09:00:59.784 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.780+0000 7f8694d52640 1 --2- 172.21.15.42:0/4239431985 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f86800b4b30 0x7f86800b7040 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f86840058c0 tx=0x7f868400b660 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:00:59.784 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.780+0000 7f869dfff740 1 -- 172.21.15.42:0/513115388 wait complete. 2024-08-30T09:00:59.784 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.781+0000 7f869dfff740 1 -- start start 2024-08-30T09:00:59.784 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.781+0000 7f869dfff740 1 Processor -- start 2024-08-30T09:00:59.786 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.782+0000 7f869dfff740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564c90f9c2f0 0x564c913749e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:59.786 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.782+0000 7f869dfff740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564c90fa44d0 0x564c913750a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:59.786 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.782+0000 7f869dfff740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564c90fc51b0 0x564c91375c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:59.786 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.782+0000 7f869dfff740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x564c91206ab0 con 0x564c90fc51b0 2024-08-30T09:00:59.786 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.782+0000 7f869dfff740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x564c91254680 con 0x564c90fa44d0 2024-08-30T09:00:59.786 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.782+0000 7f869dfff740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x564c9136cc80 con 0x564c90f9c2f0 2024-08-30T09:00:59.787 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.782+0000 7f8694d52640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564c90f9c2f0 0x564c913749e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:59.787 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.782+0000 7f869dfff740 10 client.0 _ll_register_callbacks cb 0x564c90fdca00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:00:59.787 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.782+0000 7f8694d52640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564c90f9c2f0 0x564c913749e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:44542/0 (socket says 172.21.15.42:44542) 2024-08-30T09:00:59.787 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.782+0000 7f8694d52640 1 -- 172.21.15.42:0/2829018014 learned_addr learned my addr 172.21.15.42:0/2829018014 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:00:59.787 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.782+0000 7f868ffff640 1 --2- 172.21.15.42:0/2829018014 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564c90fa44d0 0x564c913750a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:59.787 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.783+0000 7f8695553640 1 --2- 172.21.15.42:0/2829018014 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564c90fc51b0 0x564c91375c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:59.787 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.783+0000 7f868ffff640 1 -- 172.21.15.42:0/2829018014 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564c90f9c2f0 msgr2=0x564c913749e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:00:59.787 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.783+0000 7f868ffff640 1 --2- 172.21.15.42:0/2829018014 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564c90f9c2f0 0x564c913749e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:59.787 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.783+0000 7f868ffff640 1 -- 172.21.15.42:0/2829018014 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564c90fc51b0 msgr2=0x564c91375c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:00:59.787 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.783+0000 7f868ffff640 1 --2- 172.21.15.42:0/2829018014 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564c90fc51b0 0x564c91375c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:59.787 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.783+0000 7f868ffff640 1 -- 172.21.15.42:0/2829018014 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564c910a9620 con 0x564c90fa44d0 2024-08-30T09:00:59.787 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.783+0000 7f868ffff640 1 --2- 172.21.15.42:0/2829018014 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564c90fa44d0 0x564c913750a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f867c0ab370 tx=0x7f867c0ab530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=82356bfe79b6c42c server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:00:59.788 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.783+0000 7f866ffff640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-30T09:00:59.788 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.783+0000 7f866ffff640 1 -- 172.21.15.42:0/2829018014 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f867c021070 con 0x564c90fa44d0 2024-08-30T09:00:59.788 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.783+0000 7f866ffff640 1 -- 172.21.15.42:0/2829018014 <== mon.2 v2:172.21.15.167:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f867c0aaae0 con 0x564c90fa44d0 2024-08-30T09:00:59.788 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.783+0000 7f869dfff740 1 -- 172.21.15.42:0/2829018014 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8688036030 con 0x564c90fa44d0 2024-08-30T09:00:59.788 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.784+0000 7f866ffff640 1 -- 172.21.15.42:0/2829018014 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f867c0ab180 con 0x564c90fa44d0 2024-08-30T09:00:59.788 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.784+0000 7f869dfff740 10 client.25178 fetch_fsmap learned FSMap version 29 2024-08-30T09:00:59.788 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.784+0000 7f869dfff740 1 -- 172.21.15.42:0/2829018014 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x564c9137ec20 con 0x564c90fa44d0 2024-08-30T09:00:59.788 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.784+0000 7f866ffff640 1 -- 172.21.15.42:0/2829018014 <== mon.2 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8688036030 con 0x564c90fa44d0 2024-08-30T09:00:59.788 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.784+0000 7f866ffff640 1 -- 172.21.15.42:0/2829018014 <== mon.2 v2:172.21.15.167:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f867c0e70d0 con 0x564c90fa44d0 2024-08-30T09:00:59.788 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.784+0000 7f869dfff740 10 client.25178 fetch_fsmap finished waiting for FSMap version 29 2024-08-30T09:00:59.788 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.784+0000 7f869dfff740 10 client.25178 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-30T09:00:59.788 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.784+0000 7f869dfff740 20 client.25178 populate_metadata read hostname 'smithi042' 2024-08-30T09:00:59.788 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.784+0000 7f869dfff740 1 --2- 172.21.15.42:0/2829018014 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x564c90f83d80 0x564c91382440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:59.789 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.784+0000 7f869dfff740 4 client.25178 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-30T09:00:59.789 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.784+0000 7f869dfff740 1 -- 172.21.15.42:0/2829018014 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564c912dff50 con 0x564c90f83d80 2024-08-30T09:00:59.790 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.784+0000 7f8694d52640 1 --2- 172.21.15.42:0/2829018014 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x564c90f83d80 0x564c91382440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:59.790 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.786+0000 7f8694d52640 1 --2- 172.21.15.42:0/2829018014 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x564c90f83d80 0x564c91382440 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee904636c9a637be server_cookie=7d78e48921e8427 in_seq=0 out_seq=0 2024-08-30T09:00:59.790 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.786+0000 7f866ffff640 10 client.25178 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:00:59.790 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.786+0000 7f866ffff640 1 -- 172.21.15.42:0/2829018014 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x564c912dff50 con 0x564c90f83d80 2024-08-30T09:00:59.790 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.786+0000 7f866ffff640 10 client.25178 handle_command_reply: tid=0 2024-08-30T09:00:59.790 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.786+0000 7f869dfff740 1 client.25178 shutdown 2024-08-30T09:00:59.790 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.787+0000 7f869dfff740 1 -- 172.21.15.42:0/2829018014 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564c90fa44d0 msgr2=0x564c913750a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:00:59.790 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.787+0000 7f869dfff740 1 --2- 172.21.15.42:0/2829018014 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564c90fa44d0 0x564c913750a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f867c0ab370 tx=0x7f867c0ab530 comp rx=0 tx=0).stop 2024-08-30T09:00:59.791 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.787+0000 7f869dfff740 1 -- 172.21.15.42:0/2829018014 shutdown_connections 2024-08-30T09:00:59.791 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.787+0000 7f869dfff740 1 --2- 172.21.15.42:0/2829018014 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x564c90f83d80 0x564c91382440 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:59.791 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.787+0000 7f869dfff740 1 --2- 172.21.15.42:0/2829018014 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564c90fc51b0 0x564c91375c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:59.791 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.787+0000 7f866ffff640 0 client.25178 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:00:59.791 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.787+0000 7f869dfff740 1 --2- 172.21.15.42:0/2829018014 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564c90fa44d0 0x564c913750a0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:59.791 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.787+0000 7f869dfff740 1 --2- 172.21.15.42:0/2829018014 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564c90f9c2f0 0x564c913749e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:59.791 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.787+0000 7f869dfff740 1 -- 172.21.15.42:0/2829018014 >> 172.21.15.42:0/2829018014 conn(0x564c90fb9300 msgr2=0x564c91364bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:00:59.791 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.787+0000 7f869dfff740 1 -- 172.21.15.42:0/2829018014 shutdown_connections 2024-08-30T09:00:59.791 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.787+0000 7f869dfff740 1 -- 172.21.15.42:0/2829018014 wait complete. 2024-08-30T09:00:59.791 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.787+0000 7f869dfff740 20 client.25178 trim_cache size 0 max 16384 2024-08-30T09:00:59.816 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.813+0000 7f869dfff740 1 Processor -- start 2024-08-30T09:00:59.816 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.813+0000 7f869dfff740 1 -- start start 2024-08-30T09:00:59.816 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.813+0000 7f869dfff740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564c90f83d80 0x564c91373600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:59.816 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.813+0000 7f869dfff740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564c90f9c2f0 0x564c91373be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:59.816 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.813+0000 7f869dfff740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564c90fa44d0 0x564c91374120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:59.816 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.814+0000 7f869dfff740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f86901b5300 con 0x564c90fa44d0 2024-08-30T09:00:59.816 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.814+0000 7f869dfff740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f86901b2bd0 con 0x564c90f9c2f0 2024-08-30T09:00:59.816 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.814+0000 7f869dfff740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f86901b0a20 con 0x564c90f83d80 2024-08-30T09:00:59.816 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.814+0000 7f868ffff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564c90f9c2f0 0x564c91373be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:59.817 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.814+0000 7f8695553640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564c90fa44d0 0x564c91374120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:59.817 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.814+0000 7f8694d52640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564c90f83d80 0x564c91373600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:59.817 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.814+0000 7f868ffff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564c90f9c2f0 0x564c91373be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:41508/0 (socket says 172.21.15.42:41508) 2024-08-30T09:00:59.817 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.814+0000 7f868ffff640 1 -- 172.21.15.42:0/1352792702 learned_addr learned my addr 172.21.15.42:0/1352792702 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:00:59.817 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.814+0000 7f868ffff640 1 -- 172.21.15.42:0/1352792702 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564c90f83d80 msgr2=0x564c91373600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:00:59.817 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.814+0000 7f868ffff640 1 --2- 172.21.15.42:0/1352792702 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564c90f83d80 0x564c91373600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:59.817 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.814+0000 7f868ffff640 1 -- 172.21.15.42:0/1352792702 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564c90fa44d0 msgr2=0x564c91374120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:00:59.817 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.814+0000 7f868ffff640 1 --2- 172.21.15.42:0/1352792702 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564c90fa44d0 0x564c91374120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:59.817 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.814+0000 7f868ffff640 1 -- 172.21.15.42:0/1352792702 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564c9137ceb0 con 0x564c90f9c2f0 2024-08-30T09:00:59.817 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.814+0000 7f868ffff640 1 --2- 172.21.15.42:0/1352792702 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564c90f9c2f0 0x564c91373be0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f867c0ab320 tx=0x7f867c0ce230 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1444be3f8939d34c server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:00:59.818 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.815+0000 7f867a7fc640 1 -- 172.21.15.42:0/1352792702 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f867c021030 con 0x564c90f9c2f0 2024-08-30T09:00:59.818 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.815+0000 7f867a7fc640 1 -- 172.21.15.42:0/1352792702 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f867c009de0 con 0x564c90f9c2f0 2024-08-30T09:00:59.818 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.815+0000 7f867a7fc640 1 -- 172.21.15.42:0/1352792702 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f867c0ce390 con 0x564c90f9c2f0 2024-08-30T09:00:59.819 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.817+0000 7f869dfff740 1 -- 172.21.15.42:0/1352792702 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564c90f9c2f0 msgr2=0x564c91373be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:00:59.819 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.817+0000 7f869dfff740 1 --2- 172.21.15.42:0/1352792702 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564c90f9c2f0 0x564c91373be0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f867c0ab320 tx=0x7f867c0ce230 comp rx=0 tx=0).stop 2024-08-30T09:00:59.819 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.817+0000 7f869dfff740 1 -- 172.21.15.42:0/1352792702 shutdown_connections 2024-08-30T09:00:59.819 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.817+0000 7f869dfff740 1 --2- 172.21.15.42:0/1352792702 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564c90fa44d0 0x564c91374120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:59.819 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.817+0000 7f869dfff740 1 --2- 172.21.15.42:0/1352792702 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564c90f9c2f0 0x564c91373be0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:59.819 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.817+0000 7f869dfff740 1 --2- 172.21.15.42:0/1352792702 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564c90f83d80 0x564c91373600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:59.819 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.817+0000 7f869dfff740 1 -- 172.21.15.42:0/1352792702 >> 172.21.15.42:0/1352792702 conn(0x564c90fb9300 msgr2=0x564c91372ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:00:59.819 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.817+0000 7f869dfff740 1 -- 172.21.15.42:0/1352792702 shutdown_connections 2024-08-30T09:00:59.820 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.818+0000 7f869dfff740 1 -- 172.21.15.42:0/1352792702 wait complete. 2024-08-30T09:00:59.820 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.818+0000 7f869dfff740 1 -- start start 2024-08-30T09:00:59.820 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.818+0000 7f869dfff740 1 Processor -- start 2024-08-30T09:00:59.821 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.819+0000 7f869dfff740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564c90f83d80 0x564c91377f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:59.821 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.819+0000 7f869dfff740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564c90f9c2f0 0x564c913785f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:59.821 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.819+0000 7f869dfff740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564c90fa44d0 0x564c9137c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:59.821 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.819+0000 7f869dfff740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f86901b1090 con 0x564c90f83d80 2024-08-30T09:00:59.821 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.819+0000 7f869dfff740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f86901b2500 con 0x564c90f9c2f0 2024-08-30T09:00:59.821 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.819+0000 7f869dfff740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f86901b1760 con 0x564c90fa44d0 2024-08-30T09:00:59.821 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.819+0000 7f869dfff740 10 client.0 _ll_register_callbacks cb 0x7f867c00de80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:00:59.822 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.819+0000 7f868ffff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564c90f9c2f0 0x564c913785f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:59.822 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.819+0000 7f8694d52640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564c90f83d80 0x564c91377f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:59.822 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.819+0000 7f868ffff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564c90f9c2f0 0x564c913785f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:41510/0 (socket says 172.21.15.42:41510) 2024-08-30T09:00:59.822 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.819+0000 7f8695553640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564c90fa44d0 0x564c9137c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:59.822 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.819+0000 7f868ffff640 1 -- 172.21.15.42:0/361984828 learned_addr learned my addr 172.21.15.42:0/361984828 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:00:59.822 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.820+0000 7f8695553640 1 -- 172.21.15.42:0/361984828 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564c90f9c2f0 msgr2=0x564c913785f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:00:59.822 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.820+0000 7f8695553640 1 --2- 172.21.15.42:0/361984828 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564c90f9c2f0 0x564c913785f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:59.822 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.820+0000 7f8695553640 1 -- 172.21.15.42:0/361984828 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564c90f83d80 msgr2=0x564c91377f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:00:59.823 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.820+0000 7f8695553640 1 --2- 172.21.15.42:0/361984828 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564c90f83d80 0x564c91377f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:59.823 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.820+0000 7f8695553640 1 -- 172.21.15.42:0/361984828 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564c9137c940 con 0x564c90fa44d0 2024-08-30T09:00:59.823 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.821+0000 7f8695553640 1 --2- 172.21.15.42:0/361984828 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564c90fa44d0 0x564c9137c1e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f868802ad80 tx=0x7f8688009880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9a422cbf740957a1 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:00:59.823 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.821+0000 7f866ffff640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-08-30T09:00:59.824 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.821+0000 7f866ffff640 1 -- 172.21.15.42:0/361984828 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8688054070 con 0x564c90fa44d0 2024-08-30T09:00:59.824 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.821+0000 7f866ffff640 1 -- 172.21.15.42:0/361984828 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f8688004a90 con 0x564c90fa44d0 2024-08-30T09:00:59.824 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.821+0000 7f869dfff740 1 -- 172.21.15.42:0/361984828 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f867c009de0 con 0x564c90fa44d0 2024-08-30T09:00:59.824 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.821+0000 7f866ffff640 1 -- 172.21.15.42:0/361984828 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8688031e40 con 0x564c90fa44d0 2024-08-30T09:00:59.824 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.821+0000 7f869dfff740 10 client.25171 fetch_fsmap learned FSMap version 29 2024-08-30T09:00:59.824 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.821+0000 7f869dfff740 1 -- 172.21.15.42:0/361984828 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x564c91379e30 con 0x564c90fa44d0 2024-08-30T09:00:59.824 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.822+0000 7f866ffff640 1 -- 172.21.15.42:0/361984828 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f867c009de0 con 0x564c90fa44d0 2024-08-30T09:00:59.825 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.822+0000 7f866ffff640 1 -- 172.21.15.42:0/361984828 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f8688043070 con 0x564c90fa44d0 2024-08-30T09:00:59.825 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.822+0000 7f869dfff740 10 client.25171 fetch_fsmap finished waiting for FSMap version 29 2024-08-30T09:00:59.825 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.822+0000 7f869dfff740 10 client.25171 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-30T09:00:59.825 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.822+0000 7f869dfff740 20 client.25171 populate_metadata read hostname 'smithi042' 2024-08-30T09:00:59.825 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.822+0000 7f869dfff740 1 --2- 172.21.15.42:0/361984828 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x564c90fc51b0 0x564c913ef1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:00:59.825 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.822+0000 7f869dfff740 4 client.25171 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-30T09:00:59.825 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.822+0000 7f869dfff740 1 -- 172.21.15.42:0/361984828 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564c912db9f0 con 0x564c90fc51b0 2024-08-30T09:00:59.825 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.822+0000 7f8694d52640 1 --2- 172.21.15.42:0/361984828 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x564c90fc51b0 0x564c913ef1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:00:59.826 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.823+0000 7f8694d52640 1 --2- 172.21.15.42:0/361984828 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x564c90fc51b0 0x564c913ef1d0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d232a804aeb1eb8 server_cookie=3652ff69acb1cdaa in_seq=0 out_seq=0 2024-08-30T09:00:59.826 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.823+0000 7f866ffff640 10 client.25171 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:00:59.838 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.836+0000 7f866ffff640 1 -- 172.21.15.42:0/361984828 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564c912db9f0 con 0x564c90fc51b0 2024-08-30T09:00:59.838 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.836+0000 7f866ffff640 10 client.25171 handle_command_reply: tid=0 2024-08-30T09:00:59.838 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.836+0000 7f869dfff740 1 client.25171 shutdown 2024-08-30T09:00:59.838 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.836+0000 7f869dfff740 1 -- 172.21.15.42:0/361984828 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564c90fa44d0 msgr2=0x564c9137c1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:00:59.838 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.836+0000 7f869dfff740 1 --2- 172.21.15.42:0/361984828 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564c90fa44d0 0x564c9137c1e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f868802ad80 tx=0x7f8688009880 comp rx=0 tx=0).stop 2024-08-30T09:00:59.839 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.836+0000 7f869dfff740 1 -- 172.21.15.42:0/361984828 shutdown_connections 2024-08-30T09:00:59.839 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.836+0000 7f869dfff740 1 --2- 172.21.15.42:0/361984828 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x564c90fc51b0 0x564c913ef1d0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:59.839 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.836+0000 7f869dfff740 1 --2- 172.21.15.42:0/361984828 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564c90fa44d0 0x564c9137c1e0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:59.839 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.836+0000 7f869dfff740 1 --2- 172.21.15.42:0/361984828 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564c90f9c2f0 0x564c913785f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:59.839 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.836+0000 7f866ffff640 0 client.25171 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:00:59.839 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.836+0000 7f869dfff740 1 --2- 172.21.15.42:0/361984828 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564c90f83d80 0x564c91377f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:59.839 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.836+0000 7f869dfff740 1 -- 172.21.15.42:0/361984828 >> 172.21.15.42:0/361984828 conn(0x564c90fb9300 msgr2=0x564c91373be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:00:59.839 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.836+0000 7f869dfff740 1 -- 172.21.15.42:0/361984828 shutdown_connections 2024-08-30T09:00:59.839 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.836+0000 7f869dfff740 1 -- 172.21.15.42:0/361984828 wait complete. 2024-08-30T09:00:59.839 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.837+0000 7f869dfff740 20 client.25171 trim_cache size 0 max 16384 2024-08-30T09:00:59.839 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-08-30T09:00:59.839 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2024-08-30T09:00:59.840 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-08-30T09:00:59.840 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-08-30T09:00:59.840 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.838+0000 7f8696fb6640 1 -- 172.21.15.42:0/4239431985 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f86800b4b30 msgr2=0x7f86800b7040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:00:59.840 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.838+0000 7f8696fb6640 1 --2- 172.21.15.42:0/4239431985 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f86800b4b30 0x7f86800b7040 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f86840058c0 tx=0x7f868400b660 comp rx=0 tx=0).stop 2024-08-30T09:00:59.840 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.838+0000 7f8696fb6640 1 -- 172.21.15.42:0/4239431985 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f86901065c0 msgr2=0x7f86901a09a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:00:59.840 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.838+0000 7f8696fb6640 1 --2- 172.21.15.42:0/4239431985 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f86901065c0 0x7f86901a09a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f867c00d770 tx=0x7f867c00dc30 comp rx=0 tx=0).stop 2024-08-30T09:00:59.841 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.839+0000 7f8696fb6640 1 -- 172.21.15.42:0/4239431985 shutdown_connections 2024-08-30T09:00:59.841 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.839+0000 7f8696fb6640 1 --2- 172.21.15.42:0/4239431985 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f86800b4b30 0x7f86800b7040 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:59.841 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.839+0000 7f8696fb6640 1 --2- 172.21.15.42:0/4239431985 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8690109180 0x7f86901a1550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:59.841 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.839+0000 7f8696fb6640 1 --2- 172.21.15.42:0/4239431985 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f86901065c0 0x7f86901a09a0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:59.841 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.839+0000 7f8696fb6640 1 --2- 172.21.15.42:0/4239431985 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f869006b4b0 0x7f86901a0330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:00:59.841 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.839+0000 7f8696fb6640 1 -- 172.21.15.42:0/4239431985 >> 172.21.15.42:0/4239431985 conn(0x7f86900fdc80 msgr2=0x7f8690104af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:00:59.841 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.839+0000 7f8696fb6640 1 -- 172.21.15.42:0/4239431985 shutdown_connections 2024-08-30T09:00:59.841 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:00:59.839+0000 7f8696fb6640 1 -- 172.21.15.42:0/4239431985 wait complete. 2024-08-30T09:01:00.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:01:00 smithi167 ceph-mon[22063]: pgmap v380: 129 pgs: 129 active+clean; 920 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 27 MiB/s wr, 751 op/s 2024-08-30T09:01:00.882 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:01:00 smithi042 ceph-mon[18652]: pgmap v380: 129 pgs: 129 active+clean; 920 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 27 MiB/s wr, 751 op/s 2024-08-30T09:01:00.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:01:00 smithi190 ceph-mon[22001]: pgmap v380: 129 pgs: 129 active+clean; 920 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 27 MiB/s wr, 751 op/s 2024-08-30T09:01:00.953 INFO:teuthology.orchestra.run.smithi042.stderr:progress: 15.0 s, 3775.6 tps, lat 26.675 ms stddev 30.018 2024-08-30T09:01:02.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:01:02 smithi167 ceph-mon[22063]: pgmap v381: 129 pgs: 129 active+clean; 925 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 11 MiB/s wr, 693 op/s 2024-08-30T09:01:02.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:01:02 smithi042 ceph-mon[18652]: pgmap v381: 129 pgs: 129 active+clean; 925 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 11 MiB/s wr, 693 op/s 2024-08-30T09:01:02.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:01:02 smithi190 ceph-mon[22001]: pgmap v381: 129 pgs: 129 active+clean; 925 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 11 MiB/s wr, 693 op/s 2024-08-30T09:01:04.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:01:04 smithi167 ceph-mon[22063]: pgmap v382: 129 pgs: 129 active+clean; 930 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 11 MiB/s wr, 869 op/s 2024-08-30T09:01:04.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:01:04 smithi042 ceph-mon[18652]: pgmap v382: 129 pgs: 129 active+clean; 930 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 11 MiB/s wr, 869 op/s 2024-08-30T09:01:04.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:01:04 smithi190 ceph-mon[22001]: pgmap v382: 129 pgs: 129 active+clean; 930 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 11 MiB/s wr, 869 op/s 2024-08-30T09:01:05.953 INFO:teuthology.orchestra.run.smithi042.stderr:progress: 20.0 s, 3927.6 tps, lat 25.564 ms stddev 34.917 2024-08-30T09:01:06.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:01:06 smithi167 ceph-mon[22063]: pgmap v383: 129 pgs: 129 active+clean; 937 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 8.3 MiB/s wr, 664 op/s 2024-08-30T09:01:06.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:01:06 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-30T09:01:06.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:01:06 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T09:01:06.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:01:06 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T09:01:06.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:01:06 smithi042 ceph-mon[18652]: pgmap v383: 129 pgs: 129 active+clean; 937 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 8.3 MiB/s wr, 664 op/s 2024-08-30T09:01:06.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:01:06 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-30T09:01:06.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:01:06 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T09:01:06.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:01:06 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T09:01:06.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:01:06 smithi190 ceph-mon[22001]: pgmap v383: 129 pgs: 129 active+clean; 937 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 8.3 MiB/s wr, 664 op/s 2024-08-30T09:01:06.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:01:06 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-30T09:01:06.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:01:06 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T09:01:06.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:01:06 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T09:01:07.090 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-30T09:01:07.493 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:01:07 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T09:01:07.493 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:01:07 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-30T09:01:07.493 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:01:07 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T09:01:07.547 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T09:01:07.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:01:07 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T09:01:07.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:01:07 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-30T09:01:07.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:01:07 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T09:01:07.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:01:07 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T09:01:07.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:01:07 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-30T09:01:07.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:01:07 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T09:01:08.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:01:08 smithi167 ceph-mon[22063]: pgmap v384: 129 pgs: 129 active+clean; 948 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 18 MiB/s wr, 908 op/s 2024-08-30T09:01:08.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:01:08 smithi042 ceph-mon[18652]: pgmap v384: 129 pgs: 129 active+clean; 948 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 18 MiB/s wr, 908 op/s 2024-08-30T09:01:08.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:01:08 smithi190 ceph-mon[22001]: pgmap v384: 129 pgs: 129 active+clean; 948 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 18 MiB/s wr, 908 op/s 2024-08-30T09:01:10.040 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-08-30T09:01:10.040 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "no active scrubs running", 2024-08-30T09:01:10.040 INFO:teuthology.orchestra.run.smithi042.stdout: "scrubs": {}, 2024-08-30T09:01:10.041 INFO:teuthology.orchestra.run.smithi042.stdout: "scrub_stats": { 2024-08-30T09:01:10.041 INFO:teuthology.orchestra.run.smithi042.stdout: "5e2ed104-0625-478a-a78b-584e7f4c2817": { 2024-08-30T09:01:10.041 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:00:33.928400+0000", 2024-08-30T09:01:10.041 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:01:10.041 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:01:10.041 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:01:10.041 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:01:10.041 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:01:10.041 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:01:10.041 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:01:10.041 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-08-30T09:01:10.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.145+0000 7f8016f92640 1 -- 172.21.15.42:0/399253197 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f801010bdb0 msgr2=0x7f80101125e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:01:10.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.145+0000 7f8016f92640 1 --2- 172.21.15.42:0/399253197 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f801010bdb0 0x7f80101125e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f7ff8009860 tx=0x7f7ff802a380 comp rx=0 tx=0).stop 2024-08-30T09:01:10.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.145+0000 7f8016f92640 1 -- 172.21.15.42:0/399253197 shutdown_connections 2024-08-30T09:01:10.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.145+0000 7f8016f92640 1 --2- 172.21.15.42:0/399253197 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f801010b460 0x7f801010b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:10.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.145+0000 7f8016f92640 1 --2- 172.21.15.42:0/399253197 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f801010bdb0 0x7f80101125e0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:10.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.145+0000 7f8016f92640 1 --2- 172.21.15.42:0/399253197 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8010105b20 0x7f8010105f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:10.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.145+0000 7f8016f92640 1 -- 172.21.15.42:0/399253197 >> 172.21.15.42:0/399253197 conn(0x7f8010101590 msgr2=0x7f80101039e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:01:10.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.145+0000 7f8016f92640 1 -- 172.21.15.42:0/399253197 shutdown_connections 2024-08-30T09:01:10.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.145+0000 7f8016f92640 1 -- 172.21.15.42:0/399253197 wait complete. 2024-08-30T09:01:10.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.146+0000 7f8016f92640 1 Processor -- start 2024-08-30T09:01:10.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.146+0000 7f8016f92640 1 -- start start 2024-08-30T09:01:10.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.147+0000 7f8016f92640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8010105b20 0x7f801010f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:01:10.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.147+0000 7f8016f92640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f801010b460 0x7f801010ff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:01:10.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.147+0000 7f8016f92640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f801010bdb0 0x7f8010110440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:01:10.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.147+0000 7f8016f92640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f8010115ab0 con 0x7f801010b460 2024-08-30T09:01:10.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.147+0000 7f8016f92640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f8010115930 con 0x7f801010bdb0 2024-08-30T09:01:10.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.147+0000 7f8016f92640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f8010115c30 con 0x7f8010105b20 2024-08-30T09:01:10.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.147+0000 7f8014d2e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8010105b20 0x7f801010f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:01:10.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.147+0000 7f8014d2e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8010105b20 0x7f801010f9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:53450/0 (socket says 172.21.15.42:53450) 2024-08-30T09:01:10.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.147+0000 7f8014d2e640 1 -- 172.21.15.42:0/49285056 learned_addr learned my addr 172.21.15.42:0/49285056 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:01:10.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.147+0000 7f801552f640 1 --2- 172.21.15.42:0/49285056 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f801010bdb0 0x7f8010110440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:01:10.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.147+0000 7f800ffff640 1 --2- 172.21.15.42:0/49285056 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f801010b460 0x7f801010ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:01:10.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.148+0000 7f801552f640 1 -- 172.21.15.42:0/49285056 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8010105b20 msgr2=0x7f801010f9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:01:10.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.148+0000 7f801552f640 1 --2- 172.21.15.42:0/49285056 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8010105b20 0x7f801010f9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:10.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.148+0000 7f801552f640 1 -- 172.21.15.42:0/49285056 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f801010b460 msgr2=0x7f801010ff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:01:10.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.148+0000 7f801552f640 1 --2- 172.21.15.42:0/49285056 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f801010b460 0x7f801010ff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:10.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.148+0000 7f801552f640 1 -- 172.21.15.42:0/49285056 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80101a4090 con 0x7f801010bdb0 2024-08-30T09:01:10.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.148+0000 7f8014d2e640 1 --2- 172.21.15.42:0/49285056 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8010105b20 0x7f801010f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-30T09:01:10.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.148+0000 7f801552f640 1 --2- 172.21.15.42:0/49285056 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f801010bdb0 0x7f8010110440 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f7ff802a8a0 tx=0x7f7ff8032cd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:01:10.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.148+0000 7f800dffb640 1 -- 172.21.15.42:0/49285056 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ff8040020 con 0x7f801010bdb0 2024-08-30T09:01:10.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.148+0000 7f800dffb640 1 -- 172.21.15.42:0/49285056 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7ff8038040 con 0x7f801010bdb0 2024-08-30T09:01:10.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.148+0000 7f800dffb640 1 -- 172.21.15.42:0/49285056 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ff8037070 con 0x7f801010bdb0 2024-08-30T09:01:10.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.148+0000 7f8016f92640 1 -- 172.21.15.42:0/49285056 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80101a4320 con 0x7f801010bdb0 2024-08-30T09:01:10.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.149+0000 7f8016f92640 1 -- 172.21.15.42:0/49285056 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80101a47d0 con 0x7f801010bdb0 2024-08-30T09:01:10.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.152+0000 7f801d507740 1 Processor -- start 2024-08-30T09:01:10.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.153+0000 7f800dffb640 1 -- 172.21.15.42:0/49285056 <== mon.2 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7f7ff8043020 con 0x7f801010bdb0 2024-08-30T09:01:10.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.153+0000 7f801d507740 1 -- start start 2024-08-30T09:01:10.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.154+0000 7f801d507740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563529436070 0x5635297db2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:01:10.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.154+0000 7f801d507740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56352940d1b0 0x5635297db800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:01:10.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.154+0000 7f801d507740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563529415390 0x5635297dbd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:01:10.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.154+0000 7f801d507740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x563529752630 con 0x56352940d1b0 2024-08-30T09:01:10.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.154+0000 7f801d507740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x563529741bc0 con 0x563529436070 2024-08-30T09:01:10.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.154+0000 7f801d507740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x563529745b70 con 0x563529415390 2024-08-30T09:01:10.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.154+0000 7f800ffff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563529436070 0x5635297db2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:01:10.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.154+0000 7f800ffff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563529436070 0x5635297db2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:43246/0 (socket says 172.21.15.42:43246) 2024-08-30T09:01:10.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.154+0000 7f800ffff640 1 -- 172.21.15.42:0/2659236793 learned_addr learned my addr 172.21.15.42:0/2659236793 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:01:10.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.154+0000 7f800ffff640 1 --2- 172.21.15.42:0/2659236793 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563529415390 0x5635297dbd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:01:10.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.154+0000 7f8014d2e640 1 --2- 172.21.15.42:0/2659236793 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56352940d1b0 0x5635297db800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:01:10.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.154+0000 7f800ffff640 1 -- 172.21.15.42:0/2659236793 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563529415390 msgr2=0x5635297dbd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:01:10.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.154+0000 7f800ffff640 1 --2- 172.21.15.42:0/2659236793 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563529415390 0x5635297dbd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:10.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.154+0000 7f800ffff640 1 -- 172.21.15.42:0/2659236793 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56352940d1b0 msgr2=0x5635297db800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:01:10.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.154+0000 7f800ffff640 1 --2- 172.21.15.42:0/2659236793 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56352940d1b0 0x5635297db800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:10.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.155+0000 7f800ffff640 1 -- 172.21.15.42:0/2659236793 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5635294a1650 con 0x563529436070 2024-08-30T09:01:10.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.155+0000 7f8014d2e640 1 --2- 172.21.15.42:0/2659236793 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56352940d1b0 0x5635297db800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-30T09:01:10.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.155+0000 7f800dffb640 1 --2- 172.21.15.42:0/49285056 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f7ffc0b4c80 0x7f7ffc0b7190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:01:10.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.155+0000 7f800ffff640 1 --2- 172.21.15.42:0/2659236793 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563529436070 0x5635297db2c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f8000009440 tx=0x7f80000099c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c01708be50251672 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:01:10.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.155+0000 7f8014d2e640 1 --2- 172.21.15.42:0/49285056 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f7ffc0b4c80 0x7f7ffc0b7190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:01:10.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.155+0000 7f800dffb640 1 -- 172.21.15.42:0/49285056 <== mon.2 v2:172.21.15.167:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f7ff8105480 con 0x7f801010bdb0 2024-08-30T09:01:10.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.155+0000 7f8006ffd640 1 -- 172.21.15.42:0/2659236793 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f800000c2d0 con 0x563529436070 2024-08-30T09:01:10.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.155+0000 7f8006ffd640 1 -- 172.21.15.42:0/2659236793 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8000010030 con 0x563529436070 2024-08-30T09:01:10.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.156+0000 7f8006ffd640 1 -- 172.21.15.42:0/2659236793 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f800000aae0 con 0x563529436070 2024-08-30T09:01:10.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.157+0000 7f801d507740 1 -- 172.21.15.42:0/2659236793 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563529436070 msgr2=0x5635297db2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:01:10.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.157+0000 7f801d507740 1 --2- 172.21.15.42:0/2659236793 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563529436070 0x5635297db2c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f8000009440 tx=0x7f80000099c0 comp rx=0 tx=0).stop 2024-08-30T09:01:10.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.157+0000 7f801d507740 1 -- 172.21.15.42:0/2659236793 shutdown_connections 2024-08-30T09:01:10.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.157+0000 7f801d507740 1 --2- 172.21.15.42:0/2659236793 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563529415390 0x5635297dbd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:10.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.157+0000 7f801d507740 1 --2- 172.21.15.42:0/2659236793 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56352940d1b0 0x5635297db800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:10.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.157+0000 7f801d507740 1 --2- 172.21.15.42:0/2659236793 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563529436070 0x5635297db2c0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:10.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.157+0000 7f801d507740 1 -- 172.21.15.42:0/2659236793 >> 172.21.15.42:0/2659236793 conn(0x56352942a460 msgr2=0x5635297d91b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:01:10.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.157+0000 7f8014d2e640 1 --2- 172.21.15.42:0/49285056 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f7ffc0b4c80 0x7f7ffc0b7190 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x5635297d9f40 tx=0x7f800800b480 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:01:10.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.157+0000 7f801d507740 1 -- 172.21.15.42:0/2659236793 shutdown_connections 2024-08-30T09:01:10.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.157+0000 7f801d507740 1 -- 172.21.15.42:0/2659236793 wait complete. 2024-08-30T09:01:10.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.158+0000 7f801d507740 1 -- start start 2024-08-30T09:01:10.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.158+0000 7f801d507740 1 Processor -- start 2024-08-30T09:01:10.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.159+0000 7f801d507740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56352940d1b0 0x5635297e69b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:01:10.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.159+0000 7f801d507740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563529415390 0x5635297e6ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:01:10.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.159+0000 7f801d507740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563529436070 0x5635297f2e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:01:10.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.159+0000 7f801d507740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x5635295a4700 con 0x563529415390 2024-08-30T09:01:10.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.159+0000 7f800ffff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563529415390 0x5635297e6ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:01:10.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.159+0000 7f8014d2e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56352940d1b0 0x5635297e69b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:01:10.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.159+0000 7f800ffff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563529415390 0x5635297e6ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:59682/0 (socket says 172.21.15.42:59682) 2024-08-30T09:01:10.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.159+0000 7f800ffff640 1 -- 172.21.15.42:0/3747854165 learned_addr learned my addr 172.21.15.42:0/3747854165 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:01:10.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.159+0000 7f801d507740 1 -- 172.21.15.42:0/3747854165 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5635296c8700 con 0x563529436070 2024-08-30T09:01:10.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.159+0000 7f801d507740 1 -- 172.21.15.42:0/3747854165 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5635297e9b40 con 0x56352940d1b0 2024-08-30T09:01:10.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.160+0000 7f801552f640 1 --2- 172.21.15.42:0/3747854165 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563529436070 0x5635297f2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:01:10.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.160+0000 7f801d507740 10 client.0 _ll_register_callbacks cb 0x5635296f7dc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:01:10.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.160+0000 7f800ffff640 1 -- 172.21.15.42:0/3747854165 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56352940d1b0 msgr2=0x5635297e69b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:01:10.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.160+0000 7f800ffff640 1 --2- 172.21.15.42:0/3747854165 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56352940d1b0 0x5635297e69b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:10.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.160+0000 7f800ffff640 1 -- 172.21.15.42:0/3747854165 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563529436070 msgr2=0x5635297f2e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:01:10.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.160+0000 7f800ffff640 1 --2- 172.21.15.42:0/3747854165 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563529436070 0x5635297f2e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:10.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.160+0000 7f800ffff640 1 -- 172.21.15.42:0/3747854165 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5635297018b0 con 0x563529415390 2024-08-30T09:01:10.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.160+0000 7f801552f640 1 --2- 172.21.15.42:0/3747854165 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563529436070 0x5635297f2e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-30T09:01:10.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.160+0000 7f800ffff640 1 --2- 172.21.15.42:0/3747854165 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563529415390 0x5635297e6ef0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f800000c380 tx=0x7f800000a400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a21639a5b9ef0dca server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:01:10.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.160+0000 7f7feffff640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-08-30T09:01:10.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.162+0000 7f7feffff640 1 -- 172.21.15.42:0/3747854165 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8000007c10 con 0x563529415390 2024-08-30T09:01:10.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.162+0000 7f801d507740 1 -- 172.21.15.42:0/3747854165 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8000010030 con 0x563529415390 2024-08-30T09:01:10.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.162+0000 7f7feffff640 1 -- 172.21.15.42:0/3747854165 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f8000007db0 con 0x563529415390 2024-08-30T09:01:10.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.162+0000 7f7feffff640 1 -- 172.21.15.42:0/3747854165 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f800000a760 con 0x563529415390 2024-08-30T09:01:10.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.162+0000 7f801d507740 10 client.15549 fetch_fsmap learned FSMap version 29 2024-08-30T09:01:10.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.162+0000 7f801d507740 1 -- 172.21.15.42:0/3747854165 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5635297f57c0 con 0x563529415390 2024-08-30T09:01:10.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.163+0000 7f7feffff640 1 -- 172.21.15.42:0/3747854165 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8000016040 con 0x563529415390 2024-08-30T09:01:10.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.164+0000 7f7feffff640 1 -- 172.21.15.42:0/3747854165 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f8000027910 con 0x563529415390 2024-08-30T09:01:10.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.164+0000 7f801d507740 10 client.15549 fetch_fsmap finished waiting for FSMap version 29 2024-08-30T09:01:10.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.164+0000 7f801d507740 10 client.15549 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-30T09:01:10.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.164+0000 7f801d507740 20 client.15549 populate_metadata read hostname 'smithi042' 2024-08-30T09:01:10.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.164+0000 7f801d507740 1 --2- 172.21.15.42:0/3747854165 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x5635293f49f0 0x5635297f8b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:01:10.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.164+0000 7f801d507740 4 client.15549 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-30T09:01:10.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.164+0000 7f801d507740 1 -- 172.21.15.42:0/3747854165 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f8000010030 con 0x5635293f49f0 2024-08-30T09:01:10.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.164+0000 7f8014d2e640 1 --2- 172.21.15.42:0/3747854165 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x5635293f49f0 0x5635297f8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:01:10.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.165+0000 7f8014d2e640 1 --2- 172.21.15.42:0/3747854165 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x5635293f49f0 0x5635297f8b80 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=487fd5c6df59f17d server_cookie=ee7f99413c525690 in_seq=0 out_seq=0 2024-08-30T09:01:10.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.165+0000 7f7feffff640 10 client.15549 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:01:10.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.167+0000 7f7feffff640 1 -- 172.21.15.42:0/3747854165 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f8000010030 con 0x5635293f49f0 2024-08-30T09:01:10.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.167+0000 7f7feffff640 10 client.15549 handle_command_reply: tid=0 2024-08-30T09:01:10.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.167+0000 7f801d507740 1 client.15549 shutdown 2024-08-30T09:01:10.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.168+0000 7f801d507740 1 -- 172.21.15.42:0/3747854165 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563529415390 msgr2=0x5635297e6ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:01:10.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.168+0000 7f801d507740 1 --2- 172.21.15.42:0/3747854165 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563529415390 0x5635297e6ef0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f800000c380 tx=0x7f800000a400 comp rx=0 tx=0).stop 2024-08-30T09:01:10.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.168+0000 7f801d507740 1 -- 172.21.15.42:0/3747854165 shutdown_connections 2024-08-30T09:01:10.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.168+0000 7f801d507740 1 --2- 172.21.15.42:0/3747854165 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x5635293f49f0 0x5635297f8b80 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:10.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.168+0000 7f801d507740 1 --2- 172.21.15.42:0/3747854165 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563529436070 0x5635297f2e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:10.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.168+0000 7f801d507740 1 --2- 172.21.15.42:0/3747854165 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563529415390 0x5635297e6ef0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:10.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.168+0000 7f7feffff640 0 client.15549 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:01:10.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.168+0000 7f801d507740 1 --2- 172.21.15.42:0/3747854165 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56352940d1b0 0x5635297e69b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:10.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.168+0000 7f801d507740 1 -- 172.21.15.42:0/3747854165 >> 172.21.15.42:0/3747854165 conn(0x56352942a460 msgr2=0x5635297de4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:01:10.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.168+0000 7f801d507740 1 -- 172.21.15.42:0/3747854165 shutdown_connections 2024-08-30T09:01:10.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.168+0000 7f801d507740 1 -- 172.21.15.42:0/3747854165 wait complete. 2024-08-30T09:01:10.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.168+0000 7f801d507740 20 client.15549 trim_cache size 0 max 16384 2024-08-30T09:01:10.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.200+0000 7f801d507740 1 Processor -- start 2024-08-30T09:01:10.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.200+0000 7f801d507740 1 -- start start 2024-08-30T09:01:10.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.201+0000 7f801d507740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56352940d1b0 0x5635297e85e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:01:10.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.201+0000 7f801d507740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563529415390 0x5635297e8b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:01:10.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.201+0000 7f801d507740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563529436070 0x5635297d8780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:01:10.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.201+0000 7f801d507740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f80101b0e70 con 0x563529436070 2024-08-30T09:01:10.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.201+0000 7f801d507740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f80101ae740 con 0x563529415390 2024-08-30T09:01:10.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.201+0000 7f801d507740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f80101ac590 con 0x56352940d1b0 2024-08-30T09:01:10.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.201+0000 7f800ffff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563529415390 0x5635297e8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:01:10.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.201+0000 7f800ffff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563529415390 0x5635297e8b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:43268/0 (socket says 172.21.15.42:43268) 2024-08-30T09:01:10.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.201+0000 7f800ffff640 1 -- 172.21.15.42:0/1230859633 learned_addr learned my addr 172.21.15.42:0/1230859633 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:01:10.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.201+0000 7f801552f640 1 --2- 172.21.15.42:0/1230859633 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563529436070 0x5635297d8780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:01:10.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.201+0000 7f8014d2e640 1 --2- 172.21.15.42:0/1230859633 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56352940d1b0 0x5635297e85e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:01:10.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.202+0000 7f8014d2e640 1 -- 172.21.15.42:0/1230859633 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563529415390 msgr2=0x5635297e8b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:01:10.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.202+0000 7f8014d2e640 1 --2- 172.21.15.42:0/1230859633 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563529415390 0x5635297e8b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:10.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.202+0000 7f8014d2e640 1 -- 172.21.15.42:0/1230859633 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563529436070 msgr2=0x5635297d8780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:01:10.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.202+0000 7f8014d2e640 1 --2- 172.21.15.42:0/1230859633 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563529436070 0x5635297d8780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:10.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.202+0000 7f8014d2e640 1 -- 172.21.15.42:0/1230859633 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5635297ddf00 con 0x56352940d1b0 2024-08-30T09:01:10.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.202+0000 7f801552f640 1 --2- 172.21.15.42:0/1230859633 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563529436070 0x5635297d8780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-30T09:01:10.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.202+0000 7f8014d2e640 1 --2- 172.21.15.42:0/1230859633 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56352940d1b0 0x5635297e85e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x5635297e9200 tx=0x7f8008014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ce8207aebb87e841 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:01:10.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.204+0000 7f80067fc640 1 -- 172.21.15.42:0/1230859633 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8008018070 con 0x56352940d1b0 2024-08-30T09:01:10.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.204+0000 7f80067fc640 1 -- 172.21.15.42:0/1230859633 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8008012d90 con 0x56352940d1b0 2024-08-30T09:01:10.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.204+0000 7f80067fc640 1 -- 172.21.15.42:0/1230859633 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8008003850 con 0x56352940d1b0 2024-08-30T09:01:10.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.205+0000 7f801d507740 1 -- 172.21.15.42:0/1230859633 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56352940d1b0 msgr2=0x5635297e85e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:01:10.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.205+0000 7f801d507740 1 --2- 172.21.15.42:0/1230859633 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56352940d1b0 0x5635297e85e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x5635297e9200 tx=0x7f8008014000 comp rx=0 tx=0).stop 2024-08-30T09:01:10.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.208+0000 7f801d507740 1 -- 172.21.15.42:0/1230859633 shutdown_connections 2024-08-30T09:01:10.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.208+0000 7f801d507740 1 --2- 172.21.15.42:0/1230859633 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563529436070 0x5635297d8780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:10.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.208+0000 7f801d507740 1 --2- 172.21.15.42:0/1230859633 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563529415390 0x5635297e8b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:10.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.208+0000 7f801d507740 1 --2- 172.21.15.42:0/1230859633 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56352940d1b0 0x5635297e85e0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:10.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.208+0000 7f801d507740 1 -- 172.21.15.42:0/1230859633 >> 172.21.15.42:0/1230859633 conn(0x56352942a460 msgr2=0x5635297e7ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:01:10.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.209+0000 7f801d507740 1 -- 172.21.15.42:0/1230859633 shutdown_connections 2024-08-30T09:01:10.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.209+0000 7f801d507740 1 -- 172.21.15.42:0/1230859633 wait complete. 2024-08-30T09:01:10.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.209+0000 7f801d507740 1 -- start start 2024-08-30T09:01:10.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.210+0000 7f801d507740 1 Processor -- start 2024-08-30T09:01:10.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.211+0000 7f801d507740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56352940d1b0 0x5635297ef600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:01:10.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.211+0000 7f801d507740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563529415390 0x5635297eaab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:01:10.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.211+0000 7f801d507740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563529436070 0x5635297eb030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:01:10.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.211+0000 7f801d507740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f80101acc00 con 0x56352940d1b0 2024-08-30T09:01:10.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.211+0000 7f801d507740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f80101ae070 con 0x563529415390 2024-08-30T09:01:10.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.211+0000 7f801552f640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563529436070 0x5635297eb030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:01:10.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.211+0000 7f801552f640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563529436070 0x5635297eb030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:53480/0 (socket says 172.21.15.42:53480) 2024-08-30T09:01:10.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.211+0000 7f801552f640 1 -- 172.21.15.42:0/77724789 learned_addr learned my addr 172.21.15.42:0/77724789 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:01:10.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.212+0000 7f801d507740 1 -- 172.21.15.42:0/77724789 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f80101ad2d0 con 0x563529436070 2024-08-30T09:01:10.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.212+0000 7f801d507740 10 client.0 _ll_register_callbacks cb 0x7f800000c810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:01:10.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.212+0000 7f801552f640 1 -- 172.21.15.42:0/77724789 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563529415390 msgr2=0x5635297eaab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:01:10.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.212+0000 7f8014d2e640 1 --2- 172.21.15.42:0/77724789 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56352940d1b0 0x5635297ef600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:01:10.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.212+0000 7f801552f640 1 --2- 172.21.15.42:0/77724789 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563529415390 0x5635297eaab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:10.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.212+0000 7f801552f640 1 -- 172.21.15.42:0/77724789 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56352940d1b0 msgr2=0x5635297ef600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:01:10.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.212+0000 7f801552f640 1 --2- 172.21.15.42:0/77724789 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56352940d1b0 0x5635297ef600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:10.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.212+0000 7f801552f640 1 -- 172.21.15.42:0/77724789 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5635297eb7c0 con 0x563529436070 2024-08-30T09:01:10.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.213+0000 7f8014d2e640 1 --2- 172.21.15.42:0/77724789 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56352940d1b0 0x5635297ef600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-30T09:01:10.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.218+0000 7f801552f640 1 --2- 172.21.15.42:0/77724789 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563529436070 0x5635297eb030 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f7ff80c7b40 tx=0x7f7ff80052a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7039c052fe1dcc5a server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:01:10.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.218+0000 7f7feffff640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-08-30T09:01:10.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.218+0000 7f7feffff640 1 -- 172.21.15.42:0/77724789 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ff80cc1c0 con 0x563529436070 2024-08-30T09:01:10.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.218+0000 7f7feffff640 1 -- 172.21.15.42:0/77724789 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f7ff8005050 con 0x563529436070 2024-08-30T09:01:10.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.218+0000 7f801d507740 1 -- 172.21.15.42:0/77724789 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8008012d90 con 0x563529436070 2024-08-30T09:01:10.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.219+0000 7f7feffff640 1 -- 172.21.15.42:0/77724789 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f7ff80cbac0 con 0x563529436070 2024-08-30T09:01:10.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.219+0000 7f801d507740 10 client.25186 fetch_fsmap learned FSMap version 29 2024-08-30T09:01:10.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.219+0000 7f801d507740 1 -- 172.21.15.42:0/77724789 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563529863340 con 0x563529436070 2024-08-30T09:01:10.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.220+0000 7f7feffff640 1 -- 172.21.15.42:0/77724789 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7ff80d0330 con 0x563529436070 2024-08-30T09:01:10.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.221+0000 7f7feffff640 1 -- 172.21.15.42:0/77724789 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f7ff80c7430 con 0x563529436070 2024-08-30T09:01:10.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.221+0000 7f801d507740 10 client.25186 fetch_fsmap finished waiting for FSMap version 29 2024-08-30T09:01:10.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.221+0000 7f801d507740 10 client.25186 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-30T09:01:10.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.221+0000 7f801d507740 20 client.25186 populate_metadata read hostname 'smithi042' 2024-08-30T09:01:10.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.221+0000 7f801d507740 1 --2- 172.21.15.42:0/77724789 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x5635293c9da0 0x563529865b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:01:10.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.221+0000 7f801d507740 4 client.25186 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-30T09:01:10.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.221+0000 7f801d507740 1 -- 172.21.15.42:0/77724789 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f8008012d90 con 0x5635293c9da0 2024-08-30T09:01:10.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.223+0000 7f8014d2e640 1 --2- 172.21.15.42:0/77724789 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x5635293c9da0 0x563529865b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:01:10.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.226+0000 7f8014d2e640 1 --2- 172.21.15.42:0/77724789 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x5635293c9da0 0x563529865b60 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=96bf7bae8fa6721d server_cookie=5abfba32d7b54f6d in_seq=0 out_seq=0 2024-08-30T09:01:10.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.226+0000 7f7feffff640 10 client.25186 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:01:10.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.228+0000 7f7feffff640 1 -- 172.21.15.42:0/77724789 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+374 (crc 0 0 0) 0x7f8008012d90 con 0x5635293c9da0 2024-08-30T09:01:10.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.228+0000 7f7feffff640 10 client.25186 handle_command_reply: tid=0 2024-08-30T09:01:10.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.228+0000 7f801d507740 1 client.25186 shutdown 2024-08-30T09:01:10.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.228+0000 7f801d507740 1 -- 172.21.15.42:0/77724789 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563529436070 msgr2=0x5635297eb030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:01:10.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.228+0000 7f801d507740 1 --2- 172.21.15.42:0/77724789 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563529436070 0x5635297eb030 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f7ff80c7b40 tx=0x7f7ff80052a0 comp rx=0 tx=0).stop 2024-08-30T09:01:10.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.229+0000 7f801d507740 1 -- 172.21.15.42:0/77724789 shutdown_connections 2024-08-30T09:01:10.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.229+0000 7f801d507740 1 --2- 172.21.15.42:0/77724789 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x5635293c9da0 0x563529865b60 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:10.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.229+0000 7f801d507740 1 --2- 172.21.15.42:0/77724789 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563529436070 0x5635297eb030 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:10.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.229+0000 7f7feffff640 0 client.25186 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:01:10.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.229+0000 7f801d507740 1 --2- 172.21.15.42:0/77724789 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563529415390 0x5635297eaab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:10.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.229+0000 7f801d507740 1 --2- 172.21.15.42:0/77724789 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56352940d1b0 0x5635297ef600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:10.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.229+0000 7f801d507740 1 -- 172.21.15.42:0/77724789 >> 172.21.15.42:0/77724789 conn(0x56352942a460 msgr2=0x5635297dc940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:01:10.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.232+0000 7f801d507740 1 -- 172.21.15.42:0/77724789 shutdown_connections 2024-08-30T09:01:10.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.232+0000 7f801d507740 1 -- 172.21.15.42:0/77724789 wait complete. 2024-08-30T09:01:10.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.232+0000 7f801d507740 20 client.25186 trim_cache size 0 max 16384 2024-08-30T09:01:10.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.233+0000 7f8016f92640 1 -- 172.21.15.42:0/49285056 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f7ffc0b4c80 msgr2=0x7f7ffc0b7190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:01:10.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.233+0000 7f8016f92640 1 --2- 172.21.15.42:0/49285056 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f7ffc0b4c80 0x7f7ffc0b7190 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x5635297d9f40 tx=0x7f800800b480 comp rx=0 tx=0).stop 2024-08-30T09:01:10.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.233+0000 7f8016f92640 1 -- 172.21.15.42:0/49285056 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f801010bdb0 msgr2=0x7f8010110440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:01:10.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.233+0000 7f8016f92640 1 --2- 172.21.15.42:0/49285056 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f801010bdb0 0x7f8010110440 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f7ff802a8a0 tx=0x7f7ff8032cd0 comp rx=0 tx=0).stop 2024-08-30T09:01:10.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.233+0000 7f8016f92640 1 -- 172.21.15.42:0/49285056 shutdown_connections 2024-08-30T09:01:10.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.233+0000 7f8016f92640 1 --2- 172.21.15.42:0/49285056 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f7ffc0b4c80 0x7f7ffc0b7190 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:10.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.233+0000 7f8016f92640 1 --2- 172.21.15.42:0/49285056 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f801010bdb0 0x7f8010110440 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:10.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.233+0000 7f8016f92640 1 --2- 172.21.15.42:0/49285056 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f801010b460 0x7f801010ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:10.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.233+0000 7f8016f92640 1 --2- 172.21.15.42:0/49285056 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8010105b20 0x7f801010f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:10.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.233+0000 7f8016f92640 1 -- 172.21.15.42:0/49285056 >> 172.21.15.42:0/49285056 conn(0x7f8010101590 msgr2=0x7f801010aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:01:10.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.234+0000 7f8016f92640 1 -- 172.21.15.42:0/49285056 shutdown_connections 2024-08-30T09:01:10.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:09.234+0000 7f8016f92640 1 -- 172.21.15.42:0/49285056 wait complete. 2024-08-30T09:01:10.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:01:10 smithi167 ceph-mon[22063]: pgmap v385: 129 pgs: 129 active+clean; 956 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 23 MiB/s wr, 984 op/s 2024-08-30T09:01:10.882 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:01:10 smithi042 ceph-mon[18652]: pgmap v385: 129 pgs: 129 active+clean; 956 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 23 MiB/s wr, 984 op/s 2024-08-30T09:01:10.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:01:10 smithi190 ceph-mon[22001]: pgmap v385: 129 pgs: 129 active+clean; 956 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 23 MiB/s wr, 984 op/s 2024-08-30T09:01:10.953 INFO:teuthology.orchestra.run.smithi042.stderr:progress: 25.0 s, 2438.0 tps, lat 40.250 ms stddev 50.676 2024-08-30T09:01:12.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:01:12 smithi167 ceph-mon[22063]: pgmap v386: 129 pgs: 129 active+clean; 957 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 22 MiB/s wr, 863 op/s 2024-08-30T09:01:12.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:01:12 smithi042 ceph-mon[18652]: pgmap v386: 129 pgs: 129 active+clean; 957 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 22 MiB/s wr, 863 op/s 2024-08-30T09:01:12.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:01:12 smithi190 ceph-mon[22001]: pgmap v386: 129 pgs: 129 active+clean; 957 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 22 MiB/s wr, 863 op/s 2024-08-30T09:01:12.989 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:12.986+0000 7f2ed24ec640 1 -- 172.21.15.42:0/1362599942 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2ecc101820 msgr2=0x7f2ecc101c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:01:12.990 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:12.986+0000 7f2ed24ec640 1 --2- 172.21.15.42:0/1362599942 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2ecc101820 0x7f2ecc101c30 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f2ebc009900 tx=0x7f2ebc02a410 comp rx=0 tx=0).stop 2024-08-30T09:01:12.990 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:12.986+0000 7f2ed24ec640 1 -- 172.21.15.42:0/1362599942 shutdown_connections 2024-08-30T09:01:12.990 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:12.986+0000 7f2ed24ec640 1 --2- 172.21.15.42:0/1362599942 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2ecc106970 0x7f2ecc108d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:12.990 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:12.986+0000 7f2ed24ec640 1 --2- 172.21.15.42:0/1362599942 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2ecc1092d0 0x7f2ecc10b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:12.990 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:12.986+0000 7f2ed24ec640 1 --2- 172.21.15.42:0/1362599942 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2ecc101820 0x7f2ecc101c30 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:12.990 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:12.986+0000 7f2ed24ec640 1 -- 172.21.15.42:0/1362599942 >> 172.21.15.42:0/1362599942 conn(0x7f2ecc0fd420 msgr2=0x7f2ecc0ff8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:01:12.990 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:12.986+0000 7f2ed24ec640 1 -- 172.21.15.42:0/1362599942 shutdown_connections 2024-08-30T09:01:12.990 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:12.986+0000 7f2ed24ec640 1 -- 172.21.15.42:0/1362599942 wait complete. 2024-08-30T09:01:12.990 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:12.987+0000 7f2ed24ec640 1 Processor -- start 2024-08-30T09:01:12.990 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:12.987+0000 7f2ed24ec640 1 -- start start 2024-08-30T09:01:12.992 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:12.987+0000 7f2ed24ec640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2ecc101820 0x7f2ecc199ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:01:12.992 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:12.987+0000 7f2ed24ec640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2ecc106970 0x7f2ecc19a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:01:12.993 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:12.987+0000 7f2ed24ec640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2ecc1092d0 0x7f2ecc19af00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:01:12.993 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:12.987+0000 7f2ed24ec640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f2ecc10f0b0 con 0x7f2ecc101820 2024-08-30T09:01:12.993 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:12.987+0000 7f2ed24ec640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f2ecc10ef30 con 0x7f2ecc106970 2024-08-30T09:01:12.993 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:12.987+0000 7f2ed24ec640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f2ecc10f230 con 0x7f2ecc1092d0 2024-08-30T09:01:12.993 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:12.987+0000 7f2ecbfff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2ecc101820 0x7f2ecc199ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:01:12.993 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:12.987+0000 7f2ecbfff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2ecc101820 0x7f2ecc199ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:59724/0 (socket says 172.21.15.42:59724) 2024-08-30T09:01:12.993 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:12.988+0000 7f2ecbfff640 1 -- 172.21.15.42:0/3477568616 learned_addr learned my addr 172.21.15.42:0/3477568616 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:01:12.993 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:12.988+0000 7f2ecb7fe640 1 --2- 172.21.15.42:0/3477568616 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2ecc106970 0x7f2ecc19a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:01:12.993 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:12.988+0000 7f2ed0a89640 1 --2- 172.21.15.42:0/3477568616 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2ecc1092d0 0x7f2ecc19af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:01:12.993 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:12.988+0000 7f2ecbfff640 1 -- 172.21.15.42:0/3477568616 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2ecc1092d0 msgr2=0x7f2ecc19af00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:01:12.993 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:12.988+0000 7f2ecbfff640 1 --2- 172.21.15.42:0/3477568616 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2ecc1092d0 0x7f2ecc19af00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:12.993 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:12.988+0000 7f2ecbfff640 1 -- 172.21.15.42:0/3477568616 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2ecc106970 msgr2=0x7f2ecc19a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:01:12.994 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:12.988+0000 7f2ecbfff640 1 --2- 172.21.15.42:0/3477568616 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2ecc106970 0x7f2ecc19a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:12.994 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:12.988+0000 7f2ecbfff640 1 -- 172.21.15.42:0/3477568616 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ecc19b720 con 0x7f2ecc101820 2024-08-30T09:01:12.994 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:12.988+0000 7f2ecbfff640 1 --2- 172.21.15.42:0/3477568616 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2ecc101820 0x7f2ecc199ce0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f2ebc006010 tx=0x7f2ebc0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:01:12.994 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:12.988+0000 7f2ec97fa640 1 -- 172.21.15.42:0/3477568616 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ebc03f030 con 0x7f2ecc101820 2024-08-30T09:01:12.994 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:12.988+0000 7f2ed24ec640 1 -- 172.21.15.42:0/3477568616 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ecc19b9b0 con 0x7f2ecc101820 2024-08-30T09:01:12.994 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:12.988+0000 7f2ec97fa640 1 -- 172.21.15.42:0/3477568616 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2ebc031910 con 0x7f2ecc101820 2024-08-30T09:01:12.994 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:12.988+0000 7f2ec97fa640 1 -- 172.21.15.42:0/3477568616 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ebc035070 con 0x7f2ecc101820 2024-08-30T09:01:12.994 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:12.988+0000 7f2ed24ec640 1 -- 172.21.15.42:0/3477568616 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ecc1a2100 con 0x7f2ecc101820 2024-08-30T09:01:13.000 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:12.991+0000 7f2ed945b740 1 Processor -- start 2024-08-30T09:01:13.000 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:12.991+0000 7f2ed945b740 1 -- start start 2024-08-30T09:01:13.000 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:12.991+0000 7f2ec97fa640 1 -- 172.21.15.42:0/3477568616 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7f2ebc042030 con 0x7f2ecc101820 2024-08-30T09:01:13.000 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:12.991+0000 7f2ed945b740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55efc9e601b0 0x55efca206180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:01:13.000 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:12.991+0000 7f2ed945b740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55efc9e372f0 0x55efca2067f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:01:13.000 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:12.991+0000 7f2ed945b740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55efc9e3f4d0 0x55efca207340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:01:13.000 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:12.991+0000 7f2ed945b740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55efca179000 con 0x55efc9e601b0 2024-08-30T09:01:13.000 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:12.991+0000 7f2ed945b740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55efca168590 con 0x55efc9e372f0 2024-08-30T09:01:13.000 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:12.991+0000 7f2ed945b740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55efca16c540 con 0x55efc9e3f4d0 2024-08-30T09:01:13.000 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:12.991+0000 7f2ed0a89640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55efc9e601b0 0x55efca206180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:01:13.001 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:12.991+0000 7f2ed0a89640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55efc9e601b0 0x55efca206180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:59734/0 (socket says 172.21.15.42:59734) 2024-08-30T09:01:13.001 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:12.991+0000 7f2ed0a89640 1 -- 172.21.15.42:0/3842095675 learned_addr learned my addr 172.21.15.42:0/3842095675 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:01:13.001 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:12.991+0000 7f2ecb7fe640 1 --2- 172.21.15.42:0/3842095675 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55efc9e3f4d0 0x55efca207340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:01:13.001 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:12.991+0000 7f2ecbfff640 1 --2- 172.21.15.42:0/3842095675 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55efc9e372f0 0x55efca2067f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:01:13.001 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:12.991+0000 7f2ed0a89640 1 -- 172.21.15.42:0/3842095675 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55efc9e3f4d0 msgr2=0x55efca207340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:01:13.001 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:12.991+0000 7f2ed0a89640 1 --2- 172.21.15.42:0/3842095675 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55efc9e3f4d0 0x55efca207340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:13.001 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:12.991+0000 7f2ed0a89640 1 -- 172.21.15.42:0/3842095675 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55efc9e372f0 msgr2=0x55efca2067f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:01:13.001 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:12.991+0000 7f2ed0a89640 1 --2- 172.21.15.42:0/3842095675 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55efc9e372f0 0x55efca2067f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:13.001 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:12.991+0000 7f2ed0a89640 1 -- 172.21.15.42:0/3842095675 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55efc9ecb790 con 0x55efc9e601b0 2024-08-30T09:01:13.001 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:12.992+0000 7f2ed0a89640 1 --2- 172.21.15.42:0/3842095675 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55efc9e601b0 0x55efca206180 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f2ec4009fd0 tx=0x7f2ec40118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfa147030bf87555 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:01:13.001 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:12.992+0000 7f2eb27fc640 1 -- 172.21.15.42:0/3842095675 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ec4004d20 con 0x55efc9e601b0 2024-08-30T09:01:13.001 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:12.992+0000 7f2eb27fc640 1 -- 172.21.15.42:0/3842095675 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2ec4004050 con 0x55efc9e601b0 2024-08-30T09:01:13.001 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:12.992+0000 7f2eb27fc640 1 -- 172.21.15.42:0/3842095675 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ec4005240 con 0x55efc9e601b0 2024-08-30T09:01:13.002 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:12.992+0000 7f2ec97fa640 1 --2- 172.21.15.42:0/3477568616 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f2eb40b4b30 0x7f2eb40b7040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:01:13.002 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:12.992+0000 7f2ecbfff640 1 --2- 172.21.15.42:0/3477568616 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f2eb40b4b30 0x7f2eb40b7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:01:13.002 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:12.992+0000 7f2ec97fa640 1 -- 172.21.15.42:0/3477568616 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f2ebc1059c0 con 0x7f2ecc101820 2024-08-30T09:01:13.002 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:12.993+0000 7f2ed945b740 1 -- 172.21.15.42:0/3842095675 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55efc9e601b0 msgr2=0x55efca206180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:01:13.002 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:12.993+0000 7f2ed945b740 1 --2- 172.21.15.42:0/3842095675 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55efc9e601b0 0x55efca206180 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f2ec4009fd0 tx=0x7f2ec40118d0 comp rx=0 tx=0).stop 2024-08-30T09:01:13.002 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:12.993+0000 7f2ecbfff640 1 --2- 172.21.15.42:0/3477568616 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f2eb40b4b30 0x7f2eb40b7040 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f2ebc0cbe10 tx=0x7f2ebc02a410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:01:13.002 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:12.993+0000 7f2ed945b740 1 -- 172.21.15.42:0/3842095675 shutdown_connections 2024-08-30T09:01:13.002 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:12.993+0000 7f2ed945b740 1 --2- 172.21.15.42:0/3842095675 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55efc9e3f4d0 0x55efca207340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:13.002 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:12.993+0000 7f2ed945b740 1 --2- 172.21.15.42:0/3842095675 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55efc9e372f0 0x55efca2067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:13.002 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:12.993+0000 7f2ed945b740 1 --2- 172.21.15.42:0/3842095675 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55efc9e601b0 0x55efca206180 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:13.002 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:12.993+0000 7f2ed945b740 1 -- 172.21.15.42:0/3842095675 >> 172.21.15.42:0/3842095675 conn(0x55efc9e54300 msgr2=0x55efca201c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:01:13.002 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:12.993+0000 7f2ed945b740 1 -- 172.21.15.42:0/3842095675 shutdown_connections 2024-08-30T09:01:13.002 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:12.993+0000 7f2ed945b740 1 -- 172.21.15.42:0/3842095675 wait complete. 2024-08-30T09:01:13.002 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:12.993+0000 7f2ed945b740 1 -- start start 2024-08-30T09:01:13.003 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:12.994+0000 7f2ed945b740 1 Processor -- start 2024-08-30T09:01:13.003 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:12.994+0000 7f2ed945b740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55efc9e372f0 0x55efca214020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:01:13.003 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:12.994+0000 7f2ed945b740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55efc9e3f4d0 0x55efca2146e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:01:13.003 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:12.994+0000 7f2ed945b740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55efc9e601b0 0x55efca215250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:01:13.003 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:12.994+0000 7f2ed945b740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55efca0a1ab0 con 0x55efc9e601b0 2024-08-30T09:01:13.003 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:12.994+0000 7f2ed945b740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55efca0ef680 con 0x55efc9e3f4d0 2024-08-30T09:01:13.003 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:12.994+0000 7f2ed945b740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55efca207eb0 con 0x55efc9e372f0 2024-08-30T09:01:13.003 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:12.994+0000 7f2ed945b740 10 client.0 _ll_register_callbacks cb 0x55efc9e77a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:01:13.003 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:12.994+0000 7f2ed0a89640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55efc9e601b0 0x55efca215250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:01:13.003 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:12.994+0000 7f2ed0a89640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55efc9e601b0 0x55efca215250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:59746/0 (socket says 172.21.15.42:59746) 2024-08-30T09:01:13.003 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:12.994+0000 7f2ed0a89640 1 -- 172.21.15.42:0/3597634175 learned_addr learned my addr 172.21.15.42:0/3597634175 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:01:13.003 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:12.994+0000 7f2ecb7fe640 1 --2- 172.21.15.42:0/3597634175 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55efc9e3f4d0 0x55efca2146e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:01:13.003 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:12.994+0000 7f2ed0a89640 1 --2- 172.21.15.42:0/3597634175 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55efc9e372f0 0x55efca214020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:01:13.004 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:12.994+0000 7f2ed0a89640 1 -- 172.21.15.42:0/3597634175 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55efc9e372f0 msgr2=0x55efca214020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:01:13.004 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:12.994+0000 7f2ed0a89640 1 --2- 172.21.15.42:0/3597634175 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55efc9e372f0 0x55efca214020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:13.004 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:12.994+0000 7f2ed0a89640 1 -- 172.21.15.42:0/3597634175 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55efc9e3f4d0 msgr2=0x55efca2146e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:01:13.004 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:12.994+0000 7f2ed0a89640 1 --2- 172.21.15.42:0/3597634175 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55efc9e3f4d0 0x55efca2146e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:13.004 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:12.994+0000 7f2ed0a89640 1 -- 172.21.15.42:0/3597634175 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55efc9f44620 con 0x55efc9e601b0 2024-08-30T09:01:13.004 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:12.995+0000 7f2ed0a89640 1 --2- 172.21.15.42:0/3597634175 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55efc9e601b0 0x55efca215250 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f2ec40046d0 tx=0x7f2ec4000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=349fa467674735b2 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:01:13.004 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:12.995+0000 7f2eab7fe640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-08-30T09:01:13.004 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:12.995+0000 7f2eab7fe640 1 -- 172.21.15.42:0/3597634175 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ec402c060 con 0x55efc9e601b0 2024-08-30T09:01:13.004 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:12.995+0000 7f2eab7fe640 1 -- 172.21.15.42:0/3597634175 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f2ec400f770 con 0x55efc9e601b0 2024-08-30T09:01:13.004 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:12.995+0000 7f2ed945b740 1 -- 172.21.15.42:0/3597634175 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2ec4004050 con 0x55efc9e601b0 2024-08-30T09:01:13.004 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:12.995+0000 7f2eab7fe640 1 -- 172.21.15.42:0/3597634175 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2ec4033050 con 0x55efc9e601b0 2024-08-30T09:01:13.004 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:12.995+0000 7f2ed945b740 10 client.15564 fetch_fsmap learned FSMap version 29 2024-08-30T09:01:13.004 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:12.995+0000 7f2ed945b740 1 -- 172.21.15.42:0/3597634175 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55efca21e1a0 con 0x55efc9e601b0 2024-08-30T09:01:13.005 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:12.995+0000 7f2eab7fe640 1 -- 172.21.15.42:0/3597634175 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2ec4004050 con 0x55efc9e601b0 2024-08-30T09:01:13.005 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:12.995+0000 7f2eab7fe640 1 -- 172.21.15.42:0/3597634175 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f2ec4027a30 con 0x55efc9e601b0 2024-08-30T09:01:13.005 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:12.995+0000 7f2ed945b740 10 client.15564 fetch_fsmap finished waiting for FSMap version 29 2024-08-30T09:01:13.005 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:12.995+0000 7f2ed945b740 10 client.15564 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-30T09:01:13.005 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:12.995+0000 7f2ed945b740 20 client.15564 populate_metadata read hostname 'smithi042' 2024-08-30T09:01:13.005 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:12.995+0000 7f2ed945b740 1 --2- 172.21.15.42:0/3597634175 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55efc9e1ed80 0x55efca221750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:01:13.005 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:12.995+0000 7f2ed945b740 4 client.15564 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-30T09:01:13.005 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:12.995+0000 7f2ed945b740 1 -- 172.21.15.42:0/3597634175 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55efca17af50 con 0x55efc9e1ed80 2024-08-30T09:01:13.005 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:12.996+0000 7f2ecb7fe640 1 --2- 172.21.15.42:0/3597634175 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55efc9e1ed80 0x55efca221750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:01:13.005 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:12.996+0000 7f2eab7fe640 10 client.15564 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:01:13.005 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:12.996+0000 7f2ecb7fe640 1 --2- 172.21.15.42:0/3597634175 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55efc9e1ed80 0x55efca221750 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a8fb8d469204a467 server_cookie=be1577376e38acb6 in_seq=0 out_seq=0 2024-08-30T09:01:13.005 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:12.997+0000 7f2eab7fe640 1 -- 172.21.15.42:0/3597634175 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55efca17af50 con 0x55efc9e1ed80 2024-08-30T09:01:13.005 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:12.997+0000 7f2eab7fe640 10 client.15564 handle_command_reply: tid=0 2024-08-30T09:01:13.005 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:12.997+0000 7f2ed945b740 1 client.15564 shutdown 2024-08-30T09:01:13.006 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:12.997+0000 7f2ed945b740 1 -- 172.21.15.42:0/3597634175 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55efc9e601b0 msgr2=0x55efca215250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:01:13.006 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:12.997+0000 7f2ed945b740 1 --2- 172.21.15.42:0/3597634175 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55efc9e601b0 0x55efca215250 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f2ec40046d0 tx=0x7f2ec4000f40 comp rx=0 tx=0).stop 2024-08-30T09:01:13.006 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:12.997+0000 7f2ed945b740 1 -- 172.21.15.42:0/3597634175 shutdown_connections 2024-08-30T09:01:13.006 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:12.997+0000 7f2ed945b740 1 --2- 172.21.15.42:0/3597634175 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55efc9e1ed80 0x55efca221750 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:13.006 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:12.997+0000 7f2eab7fe640 0 client.15564 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:01:13.006 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:12.997+0000 7f2ed945b740 1 --2- 172.21.15.42:0/3597634175 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55efc9e601b0 0x55efca215250 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:13.006 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:12.997+0000 7f2ed945b740 1 --2- 172.21.15.42:0/3597634175 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55efc9e3f4d0 0x55efca2146e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:13.006 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:12.997+0000 7f2ed945b740 1 --2- 172.21.15.42:0/3597634175 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55efc9e372f0 0x55efca214020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:13.006 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:12.997+0000 7f2ed945b740 1 -- 172.21.15.42:0/3597634175 >> 172.21.15.42:0/3597634175 conn(0x55efc9e54300 msgr2=0x55efca2071f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:01:13.006 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:12.997+0000 7f2ed945b740 1 -- 172.21.15.42:0/3597634175 shutdown_connections 2024-08-30T09:01:13.006 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:12.997+0000 7f2ed945b740 1 -- 172.21.15.42:0/3597634175 wait complete. 2024-08-30T09:01:13.006 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:12.997+0000 7f2ed945b740 20 client.15564 trim_cache size 0 max 16384 2024-08-30T09:01:13.016 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:13.011+0000 7f2ed945b740 1 Processor -- start 2024-08-30T09:01:13.016 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:13.012+0000 7f2ed945b740 1 -- start start 2024-08-30T09:01:13.016 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:13.012+0000 7f2ed945b740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55efc9e1ed80 0x55efca201450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:01:13.016 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:13.012+0000 7f2ed945b740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55efc9e372f0 0x55efca201990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:01:13.017 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:13.012+0000 7f2ed945b740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55efc9e3f4d0 0x55efca200170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:01:13.017 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:13.012+0000 7f2ed945b740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f2ecc1acb30 con 0x55efc9e1ed80 2024-08-30T09:01:13.017 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:13.012+0000 7f2ed945b740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f2ecc1aa400 con 0x55efc9e372f0 2024-08-30T09:01:13.017 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:13.012+0000 7f2ed945b740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f2ecc1a8250 con 0x55efc9e3f4d0 2024-08-30T09:01:13.017 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:13.012+0000 7f2ed0a89640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55efc9e1ed80 0x55efca201450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:01:13.017 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:13.012+0000 7f2ed0a89640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55efc9e1ed80 0x55efca201450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:59760/0 (socket says 172.21.15.42:59760) 2024-08-30T09:01:13.017 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:13.012+0000 7f2ed0a89640 1 -- 172.21.15.42:0/1972069163 learned_addr learned my addr 172.21.15.42:0/1972069163 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:01:13.017 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:13.012+0000 7f2ecb7fe640 1 --2- 172.21.15.42:0/1972069163 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55efc9e372f0 0x55efca201990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:01:13.017 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:13.012+0000 7f2ed0a89640 1 --2- 172.21.15.42:0/1972069163 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55efc9e3f4d0 0x55efca200170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:01:13.017 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:13.012+0000 7f2ed0a89640 1 -- 172.21.15.42:0/1972069163 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55efc9e3f4d0 msgr2=0x55efca200170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:01:13.017 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:13.012+0000 7f2ed0a89640 1 --2- 172.21.15.42:0/1972069163 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55efc9e3f4d0 0x55efca200170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:13.017 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:13.012+0000 7f2ed0a89640 1 -- 172.21.15.42:0/1972069163 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55efc9e372f0 msgr2=0x55efca201990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:01:13.017 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:13.012+0000 7f2ed0a89640 1 --2- 172.21.15.42:0/1972069163 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55efc9e372f0 0x55efca201990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:13.018 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:13.012+0000 7f2ed0a89640 1 -- 172.21.15.42:0/1972069163 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55efca1fff60 con 0x55efc9e1ed80 2024-08-30T09:01:13.018 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:13.012+0000 7f2ecb7fe640 1 --2- 172.21.15.42:0/1972069163 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55efc9e372f0 0x55efca201990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-30T09:01:13.018 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:13.012+0000 7f2ed0a89640 1 --2- 172.21.15.42:0/1972069163 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55efc9e1ed80 0x55efca201450 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x55efca1686e0 tx=0x7f2ec4005cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a360b858384d0636 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:01:13.018 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:13.013+0000 7f2eb1ffb640 1 -- 172.21.15.42:0/1972069163 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ec402c040 con 0x55efc9e1ed80 2024-08-30T09:01:13.018 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:13.013+0000 7f2eb1ffb640 1 -- 172.21.15.42:0/1972069163 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2ec4004700 con 0x55efc9e1ed80 2024-08-30T09:01:13.018 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:13.013+0000 7f2eb1ffb640 1 -- 172.21.15.42:0/1972069163 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ec40197d0 con 0x55efc9e1ed80 2024-08-30T09:01:13.018 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:13.013+0000 7f2ed945b740 1 -- 172.21.15.42:0/1972069163 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55efc9e1ed80 msgr2=0x55efca201450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:01:13.018 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:13.013+0000 7f2ed945b740 1 --2- 172.21.15.42:0/1972069163 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55efc9e1ed80 0x55efca201450 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x55efca1686e0 tx=0x7f2ec4005cf0 comp rx=0 tx=0).stop 2024-08-30T09:01:13.018 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:13.013+0000 7f2ed945b740 1 -- 172.21.15.42:0/1972069163 shutdown_connections 2024-08-30T09:01:13.018 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:13.013+0000 7f2ed945b740 1 --2- 172.21.15.42:0/1972069163 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55efc9e3f4d0 0x55efca200170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:13.018 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:13.013+0000 7f2ed945b740 1 --2- 172.21.15.42:0/1972069163 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55efc9e372f0 0x55efca201990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:13.018 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:13.013+0000 7f2ed945b740 1 --2- 172.21.15.42:0/1972069163 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55efc9e1ed80 0x55efca201450 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:13.018 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:13.013+0000 7f2ed945b740 1 -- 172.21.15.42:0/1972069163 >> 172.21.15.42:0/1972069163 conn(0x55efc9e54300 msgr2=0x55efca2125b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:01:13.018 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:13.013+0000 7f2ed945b740 1 -- 172.21.15.42:0/1972069163 shutdown_connections 2024-08-30T09:01:13.019 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:13.013+0000 7f2ed945b740 1 -- 172.21.15.42:0/1972069163 wait complete. 2024-08-30T09:01:13.019 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:13.014+0000 7f2ed945b740 1 -- start start 2024-08-30T09:01:13.022 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:13.014+0000 7f2ed945b740 1 Processor -- start 2024-08-30T09:01:13.022 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:13.014+0000 7f2ed945b740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55efc9e1ed80 0x55efca21bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:01:13.022 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:13.014+0000 7f2ed945b740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55efc9e372f0 0x55efca217af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:01:13.022 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:13.014+0000 7f2ed945b740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55efc9e3f4d0 0x55efca218030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:01:13.022 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:13.014+0000 7f2ed945b740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f2ecc1a88c0 con 0x55efc9e1ed80 2024-08-30T09:01:13.022 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:13.014+0000 7f2ed945b740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f2ecc1a9d30 con 0x55efc9e3f4d0 2024-08-30T09:01:13.022 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:13.014+0000 7f2ed945b740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f2ecc1a8f90 con 0x55efc9e372f0 2024-08-30T09:01:13.022 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:13.014+0000 7f2ed945b740 10 client.0 _ll_register_callbacks cb 0x7f2ec0009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:01:13.022 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:13.014+0000 7f2ed0a89640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55efc9e1ed80 0x55efca21bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:01:13.022 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:13.014+0000 7f2ed0a89640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55efc9e1ed80 0x55efca21bbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:59770/0 (socket says 172.21.15.42:59770) 2024-08-30T09:01:13.023 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:13.014+0000 7f2ed0a89640 1 -- 172.21.15.42:0/1914920555 learned_addr learned my addr 172.21.15.42:0/1914920555 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:01:13.023 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:13.014+0000 7f2ecb7fe640 1 --2- 172.21.15.42:0/1914920555 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55efc9e372f0 0x55efca217af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:01:13.023 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:13.014+0000 7f2ed0a89640 1 --2- 172.21.15.42:0/1914920555 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55efc9e3f4d0 0x55efca218030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:01:13.023 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:13.015+0000 7f2ed0a89640 1 -- 172.21.15.42:0/1914920555 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55efc9e372f0 msgr2=0x55efca217af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:01:13.023 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:13.015+0000 7f2ed0a89640 1 --2- 172.21.15.42:0/1914920555 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55efc9e372f0 0x55efca217af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:13.023 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:13.015+0000 7f2ed0a89640 1 -- 172.21.15.42:0/1914920555 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55efc9e3f4d0 msgr2=0x55efca218030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:01:13.023 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:13.015+0000 7f2ed0a89640 1 --2- 172.21.15.42:0/1914920555 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55efc9e3f4d0 0x55efca218030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:13.023 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:13.015+0000 7f2ed0a89640 1 -- 172.21.15.42:0/1914920555 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55efca218570 con 0x55efc9e1ed80 2024-08-30T09:01:13.023 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:13.015+0000 7f2ecb7fe640 1 --2- 172.21.15.42:0/1914920555 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55efc9e372f0 0x55efca217af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-30T09:01:13.023 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:13.015+0000 7f2ed0a89640 1 --2- 172.21.15.42:0/1914920555 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55efc9e1ed80 0x55efca21bbb0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f2ec40046d0 tx=0x7f2ec4004660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c67c47f678c34cd7 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:01:13.023 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:13.015+0000 7f2eab7fe640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-08-30T09:01:13.023 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:13.015+0000 7f2eab7fe640 1 -- 172.21.15.42:0/1914920555 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ec4004cc0 con 0x55efc9e1ed80 2024-08-30T09:01:13.023 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:13.015+0000 7f2eab7fe640 1 -- 172.21.15.42:0/1914920555 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f2ec40197d0 con 0x55efc9e1ed80 2024-08-30T09:01:13.024 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:13.015+0000 7f2ed945b740 1 -- 172.21.15.42:0/1914920555 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2ec4004700 con 0x55efc9e1ed80 2024-08-30T09:01:13.024 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:13.015+0000 7f2eab7fe640 1 -- 172.21.15.42:0/1914920555 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2ec40199f0 con 0x55efc9e1ed80 2024-08-30T09:01:13.024 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:13.015+0000 7f2ed945b740 10 client.15570 fetch_fsmap learned FSMap version 29 2024-08-30T09:01:13.024 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:13.015+0000 7f2ed945b740 1 -- 172.21.15.42:0/1914920555 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55efca28c150 con 0x55efc9e1ed80 2024-08-30T09:01:13.024 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:13.015+0000 7f2eab7fe640 1 -- 172.21.15.42:0/1914920555 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2ec4004700 con 0x55efc9e1ed80 2024-08-30T09:01:13.024 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:13.016+0000 7f2eab7fe640 1 -- 172.21.15.42:0/1914920555 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f2ec4019b90 con 0x55efc9e1ed80 2024-08-30T09:01:13.024 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:13.016+0000 7f2ed945b740 10 client.15570 fetch_fsmap finished waiting for FSMap version 29 2024-08-30T09:01:13.024 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:13.016+0000 7f2ed945b740 10 client.15570 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-30T09:01:13.024 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:13.016+0000 7f2ed945b740 20 client.15570 populate_metadata read hostname 'smithi042' 2024-08-30T09:01:13.024 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:13.016+0000 7f2ed945b740 1 --2- 172.21.15.42:0/1914920555 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55efc9e601b0 0x55efca28ea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:01:13.024 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:13.016+0000 7f2ed945b740 4 client.15570 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-30T09:01:13.024 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:13.016+0000 7f2ed945b740 1 -- 172.21.15.42:0/1914920555 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55efca1fcac0 con 0x55efc9e601b0 2024-08-30T09:01:13.024 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:13.016+0000 7f2ecb7fe640 1 --2- 172.21.15.42:0/1914920555 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55efc9e601b0 0x55efca28ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:01:13.025 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:13.016+0000 7f2ecb7fe640 1 --2- 172.21.15.42:0/1914920555 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55efc9e601b0 0x55efca28ea80 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=804836dd08f54203 server_cookie=2777683b82db3f7b in_seq=0 out_seq=0 2024-08-30T09:01:13.025 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:13.016+0000 7f2eab7fe640 10 client.15570 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:01:13.073 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:13.071+0000 7f2eab7fe640 1 -- 172.21.15.42:0/1914920555 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55efca1fcac0 con 0x55efc9e601b0 2024-08-30T09:01:13.073 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:13.071+0000 7f2eab7fe640 10 client.15570 handle_command_reply: tid=0 2024-08-30T09:01:13.074 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:13.071+0000 7f2ed945b740 1 client.15570 shutdown 2024-08-30T09:01:13.075 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:13.071+0000 7f2ed945b740 1 -- 172.21.15.42:0/1914920555 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55efc9e1ed80 msgr2=0x55efca21bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:01:13.075 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:13.071+0000 7f2ed945b740 1 --2- 172.21.15.42:0/1914920555 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55efc9e1ed80 0x55efca21bbb0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f2ec40046d0 tx=0x7f2ec4004660 comp rx=0 tx=0).stop 2024-08-30T09:01:13.075 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:13.071+0000 7f2ed945b740 1 -- 172.21.15.42:0/1914920555 shutdown_connections 2024-08-30T09:01:13.075 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:13.071+0000 7f2ed945b740 1 --2- 172.21.15.42:0/1914920555 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55efc9e601b0 0x55efca28ea80 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:13.075 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:13.071+0000 7f2eab7fe640 0 client.15570 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:01:13.075 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:13.071+0000 7f2ed945b740 1 --2- 172.21.15.42:0/1914920555 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55efc9e372f0 0x55efca217af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:13.075 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:13.071+0000 7f2ed945b740 1 --2- 172.21.15.42:0/1914920555 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55efc9e3f4d0 0x55efca218030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:13.076 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:13.071+0000 7f2ed945b740 1 --2- 172.21.15.42:0/1914920555 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55efc9e1ed80 0x55efca21bbb0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:13.076 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:13.071+0000 7f2ed945b740 1 -- 172.21.15.42:0/1914920555 >> 172.21.15.42:0/1914920555 conn(0x55efc9e54300 msgr2=0x55efca205d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:01:13.076 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:13.072+0000 7f2ed945b740 1 -- 172.21.15.42:0/1914920555 shutdown_connections 2024-08-30T09:01:13.076 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:13.072+0000 7f2ed945b740 1 -- 172.21.15.42:0/1914920555 wait complete. 2024-08-30T09:01:13.076 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:13.072+0000 7f2ed945b740 20 client.15570 trim_cache size 0 max 16384 2024-08-30T09:01:13.076 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-08-30T09:01:13.076 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2024-08-30T09:01:13.076 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-08-30T09:01:13.076 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-08-30T09:01:13.077 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:13.072+0000 7f2ed24ec640 1 -- 172.21.15.42:0/3477568616 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f2eb40b4b30 msgr2=0x7f2eb40b7040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:01:13.077 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:13.073+0000 7f2ed24ec640 1 --2- 172.21.15.42:0/3477568616 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f2eb40b4b30 0x7f2eb40b7040 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f2ebc0cbe10 tx=0x7f2ebc02a410 comp rx=0 tx=0).stop 2024-08-30T09:01:13.077 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:13.073+0000 7f2ed24ec640 1 -- 172.21.15.42:0/3477568616 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2ecc101820 msgr2=0x7f2ecc199ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:01:13.077 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:13.073+0000 7f2ed24ec640 1 --2- 172.21.15.42:0/3477568616 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2ecc101820 0x7f2ecc199ce0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f2ebc006010 tx=0x7f2ebc0042f0 comp rx=0 tx=0).stop 2024-08-30T09:01:13.077 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:13.073+0000 7f2ed24ec640 1 -- 172.21.15.42:0/3477568616 shutdown_connections 2024-08-30T09:01:13.077 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:13.073+0000 7f2ed24ec640 1 --2- 172.21.15.42:0/3477568616 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f2eb40b4b30 0x7f2eb40b7040 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:13.078 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:13.073+0000 7f2ed24ec640 1 --2- 172.21.15.42:0/3477568616 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2ecc1092d0 0x7f2ecc19af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:13.078 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:13.073+0000 7f2ed24ec640 1 --2- 172.21.15.42:0/3477568616 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2ecc106970 0x7f2ecc19a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:13.078 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:13.073+0000 7f2ed24ec640 1 --2- 172.21.15.42:0/3477568616 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2ecc101820 0x7f2ecc199ce0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:13.078 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:13.073+0000 7f2ed24ec640 1 -- 172.21.15.42:0/3477568616 >> 172.21.15.42:0/3477568616 conn(0x7f2ecc0fd420 msgr2=0x7f2ecc1077c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:01:13.078 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:13.073+0000 7f2ed24ec640 1 -- 172.21.15.42:0/3477568616 shutdown_connections 2024-08-30T09:01:13.078 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:13.073+0000 7f2ed24ec640 1 -- 172.21.15.42:0/3477568616 wait complete. 2024-08-30T09:01:14.136 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-08-30T09:01:14.136 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph fs dump --format=json 2024-08-30T09:01:14.552 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T09:01:14.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:01:14 smithi042 ceph-mon[18652]: pgmap v387: 129 pgs: 129 active+clean; 958 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 23 MiB/s wr, 983 op/s 2024-08-30T09:01:14.826 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:01:14 smithi167 ceph-mon[22063]: pgmap v387: 129 pgs: 129 active+clean; 958 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 23 MiB/s wr, 983 op/s 2024-08-30T09:01:14.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:01:14 smithi190 ceph-mon[22001]: pgmap v387: 129 pgs: 129 active+clean; 958 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 23 MiB/s wr, 983 op/s 2024-08-30T09:01:15.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:15.266+0000 7f79ac2bf640 1 -- 172.21.15.42:0/3448857916 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f79a410db90 msgr2=0x7f79a41100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:01:15.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:15.266+0000 7f79ac2bf640 1 --2- 172.21.15.42:0/3448857916 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f79a410db90 0x7f79a41100a0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f7994009900 tx=0x7f799402a3f0 comp rx=0 tx=0).stop 2024-08-30T09:01:15.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:15.267+0000 7f79ac2bf640 1 -- 172.21.15.42:0/3448857916 shutdown_connections 2024-08-30T09:01:15.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:15.267+0000 7f79ac2bf640 1 --2- 172.21.15.42:0/3448857916 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f79a4105010 0x7f79a410d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:15.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:15.267+0000 7f79ac2bf640 1 --2- 172.21.15.42:0/3448857916 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f79a410db90 0x7f79a41100a0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:15.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:15.267+0000 7f79ac2bf640 1 --2- 172.21.15.42:0/3448857916 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f79a41045e0 0x7f79a41049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:15.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:15.267+0000 7f79ac2bf640 1 -- 172.21.15.42:0/3448857916 >> 172.21.15.42:0/3448857916 conn(0x7f79a40794d0 msgr2=0x7f79a40798e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:01:15.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:15.267+0000 7f79ac2bf640 1 -- 172.21.15.42:0/3448857916 shutdown_connections 2024-08-30T09:01:15.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:15.267+0000 7f79ac2bf640 1 -- 172.21.15.42:0/3448857916 wait complete. 2024-08-30T09:01:15.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:15.268+0000 7f79ac2bf640 1 Processor -- start 2024-08-30T09:01:15.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:15.268+0000 7f79ac2bf640 1 -- start start 2024-08-30T09:01:15.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:15.268+0000 7f79ac2bf640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f79a41045e0 0x7f79a4198130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:01:15.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:15.268+0000 7f79ac2bf640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f79a4105010 0x7f79a41986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:01:15.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:15.268+0000 7f79ac2bf640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f79a410db90 0x7f79a419d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:01:15.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:15.268+0000 7f79ac2bf640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f79a4111620 con 0x7f79a41045e0 2024-08-30T09:01:15.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:15.268+0000 7f79ac2bf640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f79a41114a0 con 0x7f79a410db90 2024-08-30T09:01:15.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:15.268+0000 7f79ac2bf640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f79a41117a0 con 0x7f79a4105010 2024-08-30T09:01:15.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:15.268+0000 7f79aa05b640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f79a41045e0 0x7f79a4198130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:01:15.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:15.268+0000 7f79a985a640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f79a4105010 0x7f79a41986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:01:15.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:15.268+0000 7f79aa05b640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f79a41045e0 0x7f79a4198130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:59784/0 (socket says 172.21.15.42:59784) 2024-08-30T09:01:15.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:15.269+0000 7f79aa05b640 1 -- 172.21.15.42:0/527414110 learned_addr learned my addr 172.21.15.42:0/527414110 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:01:15.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:15.269+0000 7f79aa85c640 1 --2- 172.21.15.42:0/527414110 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f79a410db90 0x7f79a419d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:01:15.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:15.269+0000 7f79a985a640 1 -- 172.21.15.42:0/527414110 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f79a410db90 msgr2=0x7f79a419d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:01:15.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:15.269+0000 7f79a985a640 1 --2- 172.21.15.42:0/527414110 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f79a410db90 0x7f79a419d970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:15.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:15.269+0000 7f79a985a640 1 -- 172.21.15.42:0/527414110 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f79a41045e0 msgr2=0x7f79a4198130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:01:15.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:15.269+0000 7f79a985a640 1 --2- 172.21.15.42:0/527414110 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f79a41045e0 0x7f79a4198130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:15.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:15.269+0000 7f79a985a640 1 -- 172.21.15.42:0/527414110 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79a419e0a0 con 0x7f79a4105010 2024-08-30T09:01:15.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:15.269+0000 7f79a985a640 1 --2- 172.21.15.42:0/527414110 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f79a4105010 0x7f79a41986d0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f79a406aad0 tx=0x7f79a000ee50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:01:15.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:15.271+0000 7f799b7fe640 1 -- 172.21.15.42:0/527414110 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79a000cc40 con 0x7f79a4105010 2024-08-30T09:01:15.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:15.271+0000 7f799b7fe640 1 -- 172.21.15.42:0/527414110 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f79a000cde0 con 0x7f79a4105010 2024-08-30T09:01:15.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:15.271+0000 7f799b7fe640 1 -- 172.21.15.42:0/527414110 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79a0018780 con 0x7f79a4105010 2024-08-30T09:01:15.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:15.271+0000 7f79ac2bf640 1 -- 172.21.15.42:0/527414110 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79a419e390 con 0x7f79a4105010 2024-08-30T09:01:15.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:15.271+0000 7f79ac2bf640 1 -- 172.21.15.42:0/527414110 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79a41a4af0 con 0x7f79a4105010 2024-08-30T09:01:15.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:15.273+0000 7f79ac2bf640 1 -- 172.21.15.42:0/527414110 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79a41049f0 con 0x7f79a4105010 2024-08-30T09:01:15.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:15.275+0000 7f799b7fe640 1 -- 172.21.15.42:0/527414110 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7f79a0010a90 con 0x7f79a4105010 2024-08-30T09:01:15.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:15.276+0000 7f799b7fe640 1 --2- 172.21.15.42:0/527414110 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f79900b90d0 0x7f79900bb5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:01:15.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:15.276+0000 7f799b7fe640 1 -- 172.21.15.42:0/527414110 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f79a00e5780 con 0x7f79a4105010 2024-08-30T09:01:15.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:15.276+0000 7f79aa05b640 1 --2- 172.21.15.42:0/527414110 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f79900b90d0 0x7f79900bb5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:01:15.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:15.276+0000 7f79aa05b640 1 --2- 172.21.15.42:0/527414110 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f79900b90d0 0x7f79900bb5e0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f799c005c40 tx=0x7f799c009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:01:15.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:15.279+0000 7f799b7fe640 1 -- 172.21.15.42:0/527414110 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f79a00ac5a0 con 0x7f79a4105010 2024-08-30T09:01:15.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:15.655+0000 7f79ac2bf640 1 -- 172.21.15.42:0/527414110 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f79a41a4d20 con 0x7f79a4105010 2024-08-30T09:01:15.659 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:15.656+0000 7f799b7fe640 1 -- 172.21.15.42:0/527414110 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11918 (secure 0 0 0) 0x7f79a00abea0 con 0x7f79a4105010 2024-08-30T09:01:15.659 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-08-30T09:01:15.659 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":29,"btime":"2024-08-30T09:00:48:414225+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14703,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/3188400000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":3188400000},{"type":"v1","addr":"172.21.15.42:6835","nonce":3188400000}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14715,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/3356537674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":3356537674},{"type":"v1","addr":"172.21.15.42:6837","nonce":3356537674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14766,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6839/2107785513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6838","nonce":2107785513},{"type":"v1","addr":"172.21.15.42:6839","nonce":2107785513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14778,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6841/2298438674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6840","nonce":2298438674},{"type":"v1","addr":"172.21.15.42:6841","nonce":2298438674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24553,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/543555570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":543555570},{"type":"v1","addr":"172.21.15.190:6833","nonce":543555570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24554,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/3907818941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":3907818941},{"type":"v1","addr":"172.21.15.167:6833","nonce":3907818941}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-30T08:58:03.055084+0000","modified":"2024-08-30T09:00:47.637894+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24629,"mds_1":24610,"mds_2":24599},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24572":{"gid":24572,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6835/234596845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":234596845},{"type":"v1","addr":"172.21.15.167:6835","nonce":234596845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24592":{"gid":24592,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6835/247905709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":247905709},{"type":"v1","addr":"172.21.15.190:6835","nonce":247905709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24598":{"gid":24598,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6837/863256349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6836","nonce":863256349},{"type":"v1","addr":"172.21.15.190:6837","nonce":863256349}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24599":{"gid":24599,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.167:6837/3011837856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":3011837856},{"type":"v1","addr":"172.21.15.167:6837","nonce":3011837856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24610":{"gid":24610,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.190:6839/719981110","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6838","nonce":719981110},{"type":"v1","addr":"172.21.15.190:6839","nonce":719981110}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24629":{"gid":24629,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.167:6839/3604645207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":3604645207},{"type":"v1","addr":"172.21.15.167:6839","nonce":3604645207}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24629,"qdb_cluster":[24599,24610,24629]},"id":1}]} 2024-08-30T09:01:15.660 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 29 2024-08-30T09:01:15.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:15.660+0000 7f79ac2bf640 1 -- 172.21.15.42:0/527414110 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f79900b90d0 msgr2=0x7f79900bb5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:01:15.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:15.660+0000 7f79ac2bf640 1 --2- 172.21.15.42:0/527414110 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f79900b90d0 0x7f79900bb5e0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f799c005c40 tx=0x7f799c009500 comp rx=0 tx=0).stop 2024-08-30T09:01:15.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:15.660+0000 7f79ac2bf640 1 -- 172.21.15.42:0/527414110 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f79a4105010 msgr2=0x7f79a41986d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:01:15.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:15.660+0000 7f79ac2bf640 1 --2- 172.21.15.42:0/527414110 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f79a4105010 0x7f79a41986d0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f79a406aad0 tx=0x7f79a000ee50 comp rx=0 tx=0).stop 2024-08-30T09:01:15.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:15.660+0000 7f79ac2bf640 1 -- 172.21.15.42:0/527414110 shutdown_connections 2024-08-30T09:01:15.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:15.660+0000 7f79ac2bf640 1 --2- 172.21.15.42:0/527414110 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f79900b90d0 0x7f79900bb5e0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:15.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:15.660+0000 7f79ac2bf640 1 --2- 172.21.15.42:0/527414110 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f79a4105010 0x7f79a41986d0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:15.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:15.660+0000 7f79ac2bf640 1 --2- 172.21.15.42:0/527414110 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f79a410db90 0x7f79a419d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:15.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:15.660+0000 7f79ac2bf640 1 --2- 172.21.15.42:0/527414110 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f79a41045e0 0x7f79a4198130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:15.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:15.660+0000 7f79ac2bf640 1 -- 172.21.15.42:0/527414110 >> 172.21.15.42:0/527414110 conn(0x7f79a40794d0 msgr2=0x7f79a4106790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:01:15.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:15.661+0000 7f79ac2bf640 1 -- 172.21.15.42:0/527414110 shutdown_connections 2024-08-30T09:01:15.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:15.661+0000 7f79ac2bf640 1 -- 172.21.15.42:0/527414110 wait complete. 2024-08-30T09:01:15.953 INFO:teuthology.orchestra.run.smithi042.stderr:progress: 30.0 s, 2501.2 tps, lat 40.491 ms stddev 47.455 2024-08-30T09:01:16.527 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-08-30T09:01:16.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:01:16 smithi167 ceph-mon[22063]: pgmap v388: 129 pgs: 129 active+clean; 962 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 22 MiB/s wr, 857 op/s 2024-08-30T09:01:16.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:01:16 smithi167 ceph-mon[22063]: from='client.? 172.21.15.42:0/527414110' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-30T09:01:16.884 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:01:16 smithi042 ceph-mon[18652]: pgmap v388: 129 pgs: 129 active+clean; 962 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 22 MiB/s wr, 857 op/s 2024-08-30T09:01:16.884 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:01:16 smithi042 ceph-mon[18652]: from='client.? 172.21.15.42:0/527414110' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-30T09:01:16.935 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:01:16 smithi190 ceph-mon[22001]: pgmap v388: 129 pgs: 129 active+clean; 962 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 22 MiB/s wr, 857 op/s 2024-08-30T09:01:16.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:01:16 smithi190 ceph-mon[22001]: from='client.? 172.21.15.42:0/527414110' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-30T09:01:16.946 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T09:01:17.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.597+0000 7f2891e32640 1 -- 172.21.15.42:0/799627654 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f288c108170 msgr2=0x7f288c108580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:01:17.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.597+0000 7f2891e32640 1 --2- 172.21.15.42:0/799627654 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f288c108170 0x7f288c108580 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f2880009900 tx=0x7f288002a3f0 comp rx=0 tx=0).stop 2024-08-30T09:01:17.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.597+0000 7f2891e32640 1 -- 172.21.15.42:0/799627654 shutdown_connections 2024-08-30T09:01:17.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.597+0000 7f2891e32640 1 --2- 172.21.15.42:0/799627654 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f288c108ac0 0x7f288c10f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:17.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.597+0000 7f2891e32640 1 --2- 172.21.15.42:0/799627654 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f288c108170 0x7f288c108580 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:17.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.597+0000 7f2891e32640 1 --2- 172.21.15.42:0/799627654 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f288c102880 0x7f288c102c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:17.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.597+0000 7f2891e32640 1 -- 172.21.15.42:0/799627654 >> 172.21.15.42:0/799627654 conn(0x7f288c079550 msgr2=0x7f288c079960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:01:17.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.597+0000 7f2891e32640 1 -- 172.21.15.42:0/799627654 shutdown_connections 2024-08-30T09:01:17.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.597+0000 7f2891e32640 1 -- 172.21.15.42:0/799627654 wait complete. 2024-08-30T09:01:17.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.599+0000 7f2891e32640 1 Processor -- start 2024-08-30T09:01:17.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.599+0000 7f2891e32640 1 -- start start 2024-08-30T09:01:17.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.599+0000 7f2891e32640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f288c102880 0x7f288c19c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:01:17.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.599+0000 7f2891e32640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f288c108170 0x7f288c19ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:01:17.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.600+0000 7f288b7fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f288c102880 0x7f288c19c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:01:17.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.600+0000 7f288affd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f288c108170 0x7f288c19ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:01:17.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.600+0000 7f288affd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f288c108170 0x7f288c19ca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:53584/0 (socket says 172.21.15.42:53584) 2024-08-30T09:01:17.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.600+0000 7f288b7fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f288c102880 0x7f288c19c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:59816/0 (socket says 172.21.15.42:59816) 2024-08-30T09:01:17.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.600+0000 7f2891e32640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f288c108ac0 0x7f288c1a1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:01:17.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.600+0000 7f288affd640 1 -- 172.21.15.42:0/3977744304 learned_addr learned my addr 172.21.15.42:0/3977744304 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:01:17.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.600+0000 7f2891e32640 1 -- 172.21.15.42:0/3977744304 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f288c112730 con 0x7f288c102880 2024-08-30T09:01:17.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.600+0000 7f2891e32640 1 -- 172.21.15.42:0/3977744304 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f288c1125b0 con 0x7f288c108ac0 2024-08-30T09:01:17.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.600+0000 7f288bfff640 1 --2- 172.21.15.42:0/3977744304 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f288c108ac0 0x7f288c1a1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:01:17.605 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.600+0000 7f2891e32640 1 -- 172.21.15.42:0/3977744304 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f288c1128b0 con 0x7f288c108170 2024-08-30T09:01:17.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.601+0000 7f288b7fe640 1 -- 172.21.15.42:0/3977744304 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f288c108170 msgr2=0x7f288c19ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:01:17.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.601+0000 7f288b7fe640 1 --2- 172.21.15.42:0/3977744304 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f288c108170 0x7f288c19ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:17.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.601+0000 7f288b7fe640 1 -- 172.21.15.42:0/3977744304 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f288c108ac0 msgr2=0x7f288c1a1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:01:17.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.601+0000 7f288b7fe640 1 --2- 172.21.15.42:0/3977744304 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f288c108ac0 0x7f288c1a1cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:17.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.601+0000 7f288b7fe640 1 -- 172.21.15.42:0/3977744304 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f288c1a2330 con 0x7f288c102880 2024-08-30T09:01:17.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.602+0000 7f288affd640 1 --2- 172.21.15.42:0/3977744304 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f288c108170 0x7f288c19ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-30T09:01:17.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.602+0000 7f288bfff640 1 --2- 172.21.15.42:0/3977744304 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f288c108ac0 0x7f288c1a1cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-30T09:01:17.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.602+0000 7f288b7fe640 1 --2- 172.21.15.42:0/3977744304 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f288c102880 0x7f288c19c480 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f287c00e9f0 tx=0x7f287c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:01:17.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.602+0000 7f2888ff9640 1 -- 172.21.15.42:0/3977744304 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f287c00cd80 con 0x7f288c102880 2024-08-30T09:01:17.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.602+0000 7f2888ff9640 1 -- 172.21.15.42:0/3977744304 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f287c005020 con 0x7f288c102880 2024-08-30T09:01:17.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.602+0000 7f2891e32640 1 -- 172.21.15.42:0/3977744304 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f288c1a2620 con 0x7f288c102880 2024-08-30T09:01:17.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.603+0000 7f2891e32640 1 -- 172.21.15.42:0/3977744304 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f288c1a8e40 con 0x7f288c102880 2024-08-30T09:01:17.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.603+0000 7f2888ff9640 1 -- 172.21.15.42:0/3977744304 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f287c004a90 con 0x7f288c102880 2024-08-30T09:01:17.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.606+0000 7f28983a7740 1 Processor -- start 2024-08-30T09:01:17.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.606+0000 7f28983a7740 1 -- start start 2024-08-30T09:01:17.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.607+0000 7f28983a7740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dc83bd6070 0x55dc83f7b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:01:17.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.607+0000 7f28983a7740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dc83bad1b0 0x55dc83f7b850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:01:17.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.607+0000 7f28983a7740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dc83bb5390 0x55dc83f7bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:01:17.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.607+0000 7f28983a7740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55dc83ef2680 con 0x55dc83bad1b0 2024-08-30T09:01:17.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.607+0000 7f28983a7740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55dc83ee1c10 con 0x55dc83bb5390 2024-08-30T09:01:17.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.607+0000 7f28983a7740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55dc83ee5bc0 con 0x55dc83bd6070 2024-08-30T09:01:17.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.607+0000 7f288bfff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dc83bd6070 0x55dc83f7b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:01:17.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.607+0000 7f288affd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dc83bb5390 0x55dc83f7bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:01:17.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.607+0000 7f288bfff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dc83bd6070 0x55dc83f7b310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:53590/0 (socket says 172.21.15.42:53590) 2024-08-30T09:01:17.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.607+0000 7f288bfff640 1 -- 172.21.15.42:0/2697487509 learned_addr learned my addr 172.21.15.42:0/2697487509 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:01:17.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.607+0000 7f288affd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dc83bb5390 0x55dc83f7bd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:43374/0 (socket says 172.21.15.42:43374) 2024-08-30T09:01:17.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.607+0000 7f288bfff640 1 -- 172.21.15.42:0/2697487509 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dc83bb5390 msgr2=0x55dc83f7bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:01:17.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.607+0000 7f288bfff640 1 --2- 172.21.15.42:0/2697487509 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dc83bb5390 0x55dc83f7bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:17.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.607+0000 7f288bfff640 1 -- 172.21.15.42:0/2697487509 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dc83bad1b0 msgr2=0x55dc83f7b850 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-30T09:01:17.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.608+0000 7f288bfff640 1 --2- 172.21.15.42:0/2697487509 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dc83bad1b0 0x55dc83f7b850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:17.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.608+0000 7f2888ff9640 1 -- 172.21.15.42:0/3977744304 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7f287c004c30 con 0x7f288c102880 2024-08-30T09:01:17.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.608+0000 7f288bfff640 1 -- 172.21.15.42:0/2697487509 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc83c41650 con 0x55dc83bd6070 2024-08-30T09:01:17.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.608+0000 7f288bfff640 1 --2- 172.21.15.42:0/2697487509 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dc83bd6070 0x55dc83f7b310 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f2884008e60 tx=0x7f288400d710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5461aca65f96373 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:01:17.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.608+0000 7f2871ffb640 1 -- 172.21.15.42:0/2697487509 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f288400f830 con 0x55dc83bd6070 2024-08-30T09:01:17.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.608+0000 7f2871ffb640 1 -- 172.21.15.42:0/2697487509 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2884018480 con 0x55dc83bd6070 2024-08-30T09:01:17.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.608+0000 7f2888ff9640 1 --2- 172.21.15.42:0/3977744304 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f28740b4be0 0x7f28740b70f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:01:17.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.609+0000 7f2871ffb640 1 -- 172.21.15.42:0/2697487509 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f288400e610 con 0x55dc83bd6070 2024-08-30T09:01:17.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.609+0000 7f2888ff9640 1 -- 172.21.15.42:0/3977744304 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f287c0e5360 con 0x7f288c102880 2024-08-30T09:01:17.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.609+0000 7f288b7fe640 1 --2- 172.21.15.42:0/3977744304 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f28740b4be0 0x7f28740b70f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:01:17.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.609+0000 7f28983a7740 1 -- 172.21.15.42:0/2697487509 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dc83bd6070 msgr2=0x55dc83f7b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:01:17.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.609+0000 7f28983a7740 1 --2- 172.21.15.42:0/2697487509 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dc83bd6070 0x55dc83f7b310 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f2884008e60 tx=0x7f288400d710 comp rx=0 tx=0).stop 2024-08-30T09:01:17.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.609+0000 7f288b7fe640 1 --2- 172.21.15.42:0/3977744304 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f28740b4be0 0x7f28740b70f0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f287c0a7b20 tx=0x7f287c000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:01:17.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.609+0000 7f28983a7740 1 -- 172.21.15.42:0/2697487509 shutdown_connections 2024-08-30T09:01:17.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.609+0000 7f28983a7740 1 --2- 172.21.15.42:0/2697487509 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dc83bb5390 0x55dc83f7bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:17.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.609+0000 7f28983a7740 1 --2- 172.21.15.42:0/2697487509 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dc83bad1b0 0x55dc83f7b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:17.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.609+0000 7f28983a7740 1 --2- 172.21.15.42:0/2697487509 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dc83bd6070 0x55dc83f7b310 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:17.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.609+0000 7f28983a7740 1 -- 172.21.15.42:0/2697487509 >> 172.21.15.42:0/2697487509 conn(0x55dc83bca460 msgr2=0x55dc83f79200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:01:17.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.610+0000 7f28983a7740 1 -- 172.21.15.42:0/2697487509 shutdown_connections 2024-08-30T09:01:17.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.610+0000 7f28983a7740 1 -- 172.21.15.42:0/2697487509 wait complete. 2024-08-30T09:01:17.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.610+0000 7f28983a7740 1 -- start start 2024-08-30T09:01:17.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.611+0000 7f28983a7740 1 Processor -- start 2024-08-30T09:01:17.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.612+0000 7f28983a7740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dc83bad1b0 0x55dc83f86a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:01:17.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.612+0000 7f28983a7740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dc83bb5390 0x55dc83f86f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:01:17.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.612+0000 7f28983a7740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dc83bd6070 0x55dc83f92ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:01:17.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.612+0000 7f28983a7740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55dc83d44700 con 0x55dc83bad1b0 2024-08-30T09:01:17.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.612+0000 7f28983a7740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55dc83e68750 con 0x55dc83bd6070 2024-08-30T09:01:17.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.612+0000 7f28983a7740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55dc83f89b90 con 0x55dc83bb5390 2024-08-30T09:01:17.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.612+0000 7f28983a7740 10 client.0 _ll_register_callbacks cb 0x55dc83e97e10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:01:17.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.614+0000 7f288bfff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dc83bd6070 0x55dc83f92ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:01:17.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.614+0000 7f288bfff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dc83bad1b0 0x55dc83f86a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:01:17.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.614+0000 7f288bfff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dc83bad1b0 0x55dc83f86a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:59818/0 (socket says 172.21.15.42:59818) 2024-08-30T09:01:17.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.614+0000 7f288bfff640 1 -- 172.21.15.42:0/2192457891 learned_addr learned my addr 172.21.15.42:0/2192457891 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:01:17.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.615+0000 7f288affd640 1 --2- 172.21.15.42:0/2192457891 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dc83bb5390 0x55dc83f86f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:01:17.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.615+0000 7f288bfff640 1 -- 172.21.15.42:0/2192457891 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dc83bb5390 msgr2=0x55dc83f86f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:01:17.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.615+0000 7f288bfff640 1 --2- 172.21.15.42:0/2192457891 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dc83bb5390 0x55dc83f86f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:17.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.615+0000 7f288bfff640 1 -- 172.21.15.42:0/2192457891 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dc83bad1b0 msgr2=0x55dc83f86a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:01:17.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.615+0000 7f288bfff640 1 --2- 172.21.15.42:0/2192457891 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dc83bad1b0 0x55dc83f86a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:17.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.615+0000 7f288bfff640 1 -- 172.21.15.42:0/2192457891 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc83ea1900 con 0x55dc83bd6070 2024-08-30T09:01:17.621 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.615+0000 7f288bfff640 1 --2- 172.21.15.42:0/2192457891 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dc83bd6070 0x55dc83f92ee0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f28840082d0 tx=0x7f2884018e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a92b479b8c2a002a server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:01:17.621 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.615+0000 7f286affd640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-30T09:01:17.621 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.616+0000 7f286affd640 1 -- 172.21.15.42:0/2192457891 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2884021040 con 0x55dc83bd6070 2024-08-30T09:01:17.621 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.616+0000 7f28983a7740 1 -- 172.21.15.42:0/2192457891 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2884018480 con 0x55dc83bd6070 2024-08-30T09:01:17.621 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.616+0000 7f286affd640 1 -- 172.21.15.42:0/2192457891 <== mon.2 v2:172.21.15.167:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f2884010870 con 0x55dc83bd6070 2024-08-30T09:01:17.621 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.616+0000 7f286affd640 1 -- 172.21.15.42:0/2192457891 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2884010a70 con 0x55dc83bd6070 2024-08-30T09:01:17.621 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.616+0000 7f28983a7740 10 client.25232 fetch_fsmap learned FSMap version 29 2024-08-30T09:01:17.621 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.616+0000 7f28983a7740 1 -- 172.21.15.42:0/2192457891 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55dc83f95900 con 0x55dc83bd6070 2024-08-30T09:01:17.621 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.616+0000 7f286affd640 1 -- 172.21.15.42:0/2192457891 <== mon.2 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2884018480 con 0x55dc83bd6070 2024-08-30T09:01:17.621 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.616+0000 7f286affd640 1 -- 172.21.15.42:0/2192457891 <== mon.2 v2:172.21.15.167:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f2884028ae0 con 0x55dc83bd6070 2024-08-30T09:01:17.621 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.616+0000 7f28983a7740 10 client.25232 fetch_fsmap finished waiting for FSMap version 29 2024-08-30T09:01:17.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.616+0000 7f28983a7740 10 client.25232 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-08-30T09:01:17.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.616+0000 7f28983a7740 20 client.25232 populate_metadata read hostname 'smithi042' 2024-08-30T09:01:17.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.616+0000 7f28983a7740 1 --2- 172.21.15.42:0/2192457891 >> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] conn(0x55dc83b949f0 0x55dc83f98d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:01:17.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.616+0000 7f28983a7740 4 client.25232 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-30T09:01:17.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.616+0000 7f28983a7740 1 -- 172.21.15.42:0/2192457891 --> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dc83ef45d0 con 0x55dc83b949f0 2024-08-30T09:01:17.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.617+0000 7f288affd640 1 --2- 172.21.15.42:0/2192457891 >> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] conn(0x55dc83b949f0 0x55dc83f98d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:01:17.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.617+0000 7f288affd640 1 --2- 172.21.15.42:0/2192457891 >> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] conn(0x55dc83b949f0 0x55dc83f98d20 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2d4fa354d99b6e0 server_cookie=979c7f95e8890a6b in_seq=0 out_seq=0 2024-08-30T09:01:17.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.617+0000 7f286affd640 10 client.25232 ms_handle_connect on v2:172.21.15.167:6836/3011837856 2024-08-30T09:01:17.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.618+0000 7f286affd640 1 -- 172.21.15.42:0/2192457891 <== mds.2 v2:172.21.15.167:6836/3011837856 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55dc83ef45d0 con 0x55dc83b949f0 2024-08-30T09:01:17.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.618+0000 7f286affd640 10 client.25232 handle_command_reply: tid=0 2024-08-30T09:01:17.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.618+0000 7f28983a7740 1 client.25232 shutdown 2024-08-30T09:01:17.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.618+0000 7f28983a7740 1 -- 172.21.15.42:0/2192457891 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dc83bd6070 msgr2=0x55dc83f92ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:01:17.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.618+0000 7f28983a7740 1 --2- 172.21.15.42:0/2192457891 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dc83bd6070 0x55dc83f92ee0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f28840082d0 tx=0x7f2884018e80 comp rx=0 tx=0).stop 2024-08-30T09:01:17.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.619+0000 7f28983a7740 1 -- 172.21.15.42:0/2192457891 shutdown_connections 2024-08-30T09:01:17.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.619+0000 7f28983a7740 1 --2- 172.21.15.42:0/2192457891 >> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] conn(0x55dc83b949f0 0x55dc83f98d20 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:17.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.619+0000 7f28983a7740 1 --2- 172.21.15.42:0/2192457891 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dc83bb5390 0x55dc83f86f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:17.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.619+0000 7f28983a7740 1 --2- 172.21.15.42:0/2192457891 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dc83bd6070 0x55dc83f92ee0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:17.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.619+0000 7f28983a7740 1 --2- 172.21.15.42:0/2192457891 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dc83bad1b0 0x55dc83f86a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:17.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.619+0000 7f28983a7740 1 -- 172.21.15.42:0/2192457891 >> 172.21.15.42:0/2192457891 conn(0x55dc83bca460 msgr2=0x55dc83f7e4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:01:17.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.619+0000 7f286affd640 0 client.25232 ms_handle_reset on v2:172.21.15.167:6836/3011837856 2024-08-30T09:01:17.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.619+0000 7f28983a7740 1 -- 172.21.15.42:0/2192457891 shutdown_connections 2024-08-30T09:01:17.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.620+0000 7f28983a7740 1 -- 172.21.15.42:0/2192457891 wait complete. 2024-08-30T09:01:17.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.620+0000 7f28983a7740 20 client.25232 trim_cache size 0 max 16384 2024-08-30T09:01:17.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.639+0000 7f28983a7740 1 Processor -- start 2024-08-30T09:01:17.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.640+0000 7f28983a7740 1 -- start start 2024-08-30T09:01:17.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.640+0000 7f28983a7740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dc83bad1b0 0x55dc83f86ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:01:17.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.640+0000 7f28983a7740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dc83bb5390 0x55dc83f92de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:01:17.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.640+0000 7f28983a7740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dc83bd6070 0x55dc83f7e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:01:17.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.640+0000 7f28983a7740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f288c1b53e0 con 0x55dc83bad1b0 2024-08-30T09:01:17.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.640+0000 7f28983a7740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f288c1b2cb0 con 0x55dc83bb5390 2024-08-30T09:01:17.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.640+0000 7f28983a7740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f288c1b0b00 con 0x55dc83bd6070 2024-08-30T09:01:17.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.640+0000 7f288bfff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dc83bad1b0 0x55dc83f86ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:01:17.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.640+0000 7f288bfff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dc83bad1b0 0x55dc83f86ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:59834/0 (socket says 172.21.15.42:59834) 2024-08-30T09:01:17.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.640+0000 7f288bfff640 1 -- 172.21.15.42:0/3900774121 learned_addr learned my addr 172.21.15.42:0/3900774121 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:01:17.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.640+0000 7f288bfff640 1 --2- 172.21.15.42:0/3900774121 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dc83bd6070 0x55dc83f7e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:01:17.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.640+0000 7f288affd640 1 --2- 172.21.15.42:0/3900774121 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dc83bb5390 0x55dc83f92de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:01:17.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.641+0000 7f288bfff640 1 -- 172.21.15.42:0/3900774121 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dc83bb5390 msgr2=0x55dc83f92de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:01:17.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.641+0000 7f288bfff640 1 --2- 172.21.15.42:0/3900774121 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dc83bb5390 0x55dc83f92de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:17.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.641+0000 7f288bfff640 1 -- 172.21.15.42:0/3900774121 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dc83bad1b0 msgr2=0x55dc83f86ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:01:17.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.641+0000 7f288bfff640 1 --2- 172.21.15.42:0/3900774121 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dc83bad1b0 0x55dc83f86ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:17.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.641+0000 7f288bfff640 1 -- 172.21.15.42:0/3900774121 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc83f7d830 con 0x55dc83bd6070 2024-08-30T09:01:17.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.641+0000 7f288bfff640 1 --2- 172.21.15.42:0/3900774121 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dc83bd6070 0x55dc83f7e3d0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f2884018e50 tx=0x7f2884018de0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8d92f1ac1e273c7b server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:01:17.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.641+0000 7f28717fa640 1 -- 172.21.15.42:0/3900774121 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2884018480 con 0x55dc83bd6070 2024-08-30T09:01:17.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.641+0000 7f28717fa640 1 -- 172.21.15.42:0/3900774121 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f288400dc10 con 0x55dc83bd6070 2024-08-30T09:01:17.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.641+0000 7f28717fa640 1 -- 172.21.15.42:0/3900774121 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2884005280 con 0x55dc83bd6070 2024-08-30T09:01:17.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.642+0000 7f28983a7740 1 -- 172.21.15.42:0/3900774121 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dc83bd6070 msgr2=0x55dc83f7e3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:01:17.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.642+0000 7f28983a7740 1 --2- 172.21.15.42:0/3900774121 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dc83bd6070 0x55dc83f7e3d0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f2884018e50 tx=0x7f2884018de0 comp rx=0 tx=0).stop 2024-08-30T09:01:17.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.643+0000 7f28983a7740 1 -- 172.21.15.42:0/3900774121 shutdown_connections 2024-08-30T09:01:17.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.643+0000 7f28983a7740 1 --2- 172.21.15.42:0/3900774121 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dc83bd6070 0x55dc83f7e3d0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:17.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.643+0000 7f28983a7740 1 --2- 172.21.15.42:0/3900774121 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dc83bb5390 0x55dc83f92de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:17.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.643+0000 7f28983a7740 1 --2- 172.21.15.42:0/3900774121 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dc83bad1b0 0x55dc83f86ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:17.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.643+0000 7f28983a7740 1 -- 172.21.15.42:0/3900774121 >> 172.21.15.42:0/3900774121 conn(0x55dc83bca460 msgr2=0x55dc83f867e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:01:17.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.643+0000 7f28983a7740 1 -- 172.21.15.42:0/3900774121 shutdown_connections 2024-08-30T09:01:17.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.643+0000 7f28983a7740 1 -- 172.21.15.42:0/3900774121 wait complete. 2024-08-30T09:01:17.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.644+0000 7f28983a7740 1 -- start start 2024-08-30T09:01:17.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.644+0000 7f28983a7740 1 Processor -- start 2024-08-30T09:01:17.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.644+0000 7f28983a7740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dc83bad1b0 0x55dc83f8f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:01:17.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.645+0000 7f28983a7740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dc83bb5390 0x55dc83f8a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:01:17.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.645+0000 7f28983a7740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dc83bd6070 0x55dc83f8aed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:01:17.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.645+0000 7f288bfff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dc83bad1b0 0x55dc83f8f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:01:17.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.645+0000 7f288bfff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dc83bad1b0 0x55dc83f8f4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:43402/0 (socket says 172.21.15.42:43402) 2024-08-30T09:01:17.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.645+0000 7f288bfff640 1 -- 172.21.15.42:0/995165292 learned_addr learned my addr 172.21.15.42:0/995165292 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:01:17.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.645+0000 7f28983a7740 1 -- 172.21.15.42:0/995165292 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f288c1b1170 con 0x55dc83bd6070 2024-08-30T09:01:17.650 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.645+0000 7f28983a7740 1 -- 172.21.15.42:0/995165292 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f288c1b25e0 con 0x55dc83bad1b0 2024-08-30T09:01:17.650 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.645+0000 7f28983a7740 1 -- 172.21.15.42:0/995165292 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f288c1b1840 con 0x55dc83bb5390 2024-08-30T09:01:17.650 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.645+0000 7f28983a7740 10 client.0 _ll_register_callbacks cb 0x7f287c005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:01:17.650 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.645+0000 7f288affd640 1 --2- 172.21.15.42:0/995165292 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dc83bb5390 0x55dc83f8a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:01:17.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.645+0000 7f288bfff640 1 --2- 172.21.15.42:0/995165292 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dc83bd6070 0x55dc83f8aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:01:17.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.645+0000 7f288bfff640 1 -- 172.21.15.42:0/995165292 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dc83bb5390 msgr2=0x55dc83f8a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:01:17.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.646+0000 7f288bfff640 1 --2- 172.21.15.42:0/995165292 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dc83bb5390 0x55dc83f8a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:17.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.646+0000 7f288bfff640 1 -- 172.21.15.42:0/995165292 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dc83bd6070 msgr2=0x55dc83f8aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:01:17.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.646+0000 7f288bfff640 1 --2- 172.21.15.42:0/995165292 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dc83bd6070 0x55dc83f8aed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:17.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.646+0000 7f288bfff640 1 -- 172.21.15.42:0/995165292 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc83f8b510 con 0x55dc83bad1b0 2024-08-30T09:01:17.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.646+0000 7f288bfff640 1 --2- 172.21.15.42:0/995165292 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dc83bad1b0 0x55dc83f8f4e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f288400e010 tx=0x7f288400dee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b7b8fdd3c55fa4ed server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:01:17.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.646+0000 7f286affd640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-30T09:01:17.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.646+0000 7f286affd640 1 -- 172.21.15.42:0/995165292 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2884010e20 con 0x55dc83bad1b0 2024-08-30T09:01:17.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.646+0000 7f286affd640 1 -- 172.21.15.42:0/995165292 <== mon.2 v2:172.21.15.167:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f2884004a00 con 0x55dc83bad1b0 2024-08-30T09:01:17.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.646+0000 7f28983a7740 1 -- 172.21.15.42:0/995165292 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f288400dc10 con 0x55dc83bad1b0 2024-08-30T09:01:17.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.646+0000 7f286affd640 1 -- 172.21.15.42:0/995165292 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2884004c00 con 0x55dc83bad1b0 2024-08-30T09:01:17.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.646+0000 7f28983a7740 10 client.25238 fetch_fsmap learned FSMap version 29 2024-08-30T09:01:17.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.646+0000 7f28983a7740 1 -- 172.21.15.42:0/995165292 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55dc840033f0 con 0x55dc83bad1b0 2024-08-30T09:01:17.655 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.647+0000 7f286affd640 1 -- 172.21.15.42:0/995165292 <== mon.2 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f288400dc10 con 0x55dc83bad1b0 2024-08-30T09:01:17.655 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.647+0000 7f286affd640 1 -- 172.21.15.42:0/995165292 <== mon.2 v2:172.21.15.167:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f28840274a0 con 0x55dc83bad1b0 2024-08-30T09:01:17.655 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.647+0000 7f28983a7740 10 client.25238 fetch_fsmap finished waiting for FSMap version 29 2024-08-30T09:01:17.655 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.647+0000 7f28983a7740 10 client.25238 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-08-30T09:01:17.655 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.647+0000 7f28983a7740 20 client.25238 populate_metadata read hostname 'smithi042' 2024-08-30T09:01:17.655 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.647+0000 7f28983a7740 1 --2- 172.21.15.42:0/995165292 >> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] conn(0x55dc83b69da0 0x55dc840059f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:01:17.655 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.647+0000 7f28983a7740 4 client.25238 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-30T09:01:17.655 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.647+0000 7f28983a7740 1 -- 172.21.15.42:0/995165292 --> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55dc83ef0070 con 0x55dc83b69da0 2024-08-30T09:01:17.655 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.648+0000 7f288affd640 1 --2- 172.21.15.42:0/995165292 >> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] conn(0x55dc83b69da0 0x55dc840059f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:01:17.655 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.648+0000 7f288affd640 1 --2- 172.21.15.42:0/995165292 >> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] conn(0x55dc83b69da0 0x55dc840059f0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=536247f8653e5376 server_cookie=22acb94a68bf1441 in_seq=0 out_seq=0 2024-08-30T09:01:17.655 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.648+0000 7f286affd640 10 client.25238 ms_handle_connect on v2:172.21.15.167:6836/3011837856 2024-08-30T09:01:17.656 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.648+0000 7f286affd640 1 -- 172.21.15.42:0/995165292 <== mds.2 v2:172.21.15.167:6836/3011837856 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55dc83ef0070 con 0x55dc83b69da0 2024-08-30T09:01:17.656 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.648+0000 7f286affd640 10 client.25238 handle_command_reply: tid=0 2024-08-30T09:01:17.656 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.649+0000 7f28983a7740 1 client.25238 shutdown 2024-08-30T09:01:17.656 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.649+0000 7f28983a7740 1 -- 172.21.15.42:0/995165292 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dc83bad1b0 msgr2=0x55dc83f8f4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:01:17.656 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.649+0000 7f28983a7740 1 --2- 172.21.15.42:0/995165292 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dc83bad1b0 0x55dc83f8f4e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f288400e010 tx=0x7f288400dee0 comp rx=0 tx=0).stop 2024-08-30T09:01:17.656 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.649+0000 7f28983a7740 1 -- 172.21.15.42:0/995165292 shutdown_connections 2024-08-30T09:01:17.656 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.649+0000 7f28983a7740 1 --2- 172.21.15.42:0/995165292 >> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] conn(0x55dc83b69da0 0x55dc840059f0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:17.656 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.649+0000 7f28983a7740 1 --2- 172.21.15.42:0/995165292 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dc83bb5390 0x55dc83f8a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:17.656 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.649+0000 7f286affd640 0 client.25238 ms_handle_reset on v2:172.21.15.167:6836/3011837856 2024-08-30T09:01:17.656 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.649+0000 7f28983a7740 1 --2- 172.21.15.42:0/995165292 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dc83bd6070 0x55dc83f8aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:17.656 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.649+0000 7f28983a7740 1 --2- 172.21.15.42:0/995165292 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dc83bad1b0 0x55dc83f8f4e0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:17.656 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.649+0000 7f28983a7740 1 -- 172.21.15.42:0/995165292 >> 172.21.15.42:0/995165292 conn(0x55dc83bca460 msgr2=0x55dc83f7e3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:01:17.657 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.649+0000 7f28983a7740 1 -- 172.21.15.42:0/995165292 shutdown_connections 2024-08-30T09:01:17.657 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.649+0000 7f28983a7740 1 -- 172.21.15.42:0/995165292 wait complete. 2024-08-30T09:01:17.657 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.650+0000 7f28983a7740 20 client.25238 trim_cache size 0 max 16384 2024-08-30T09:01:17.657 INFO:teuthology.orchestra.run.smithi042.stdout:[] 2024-08-30T09:01:17.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.650+0000 7f2891e32640 1 -- 172.21.15.42:0/3977744304 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f28740b4be0 msgr2=0x7f28740b70f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:01:17.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.650+0000 7f2891e32640 1 --2- 172.21.15.42:0/3977744304 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f28740b4be0 0x7f28740b70f0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f287c0a7b20 tx=0x7f287c000bc0 comp rx=0 tx=0).stop 2024-08-30T09:01:17.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.650+0000 7f2891e32640 1 -- 172.21.15.42:0/3977744304 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f288c102880 msgr2=0x7f288c19c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:01:17.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.650+0000 7f2891e32640 1 --2- 172.21.15.42:0/3977744304 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f288c102880 0x7f288c19c480 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f287c00e9f0 tx=0x7f287c00eeb0 comp rx=0 tx=0).stop 2024-08-30T09:01:17.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.650+0000 7f2891e32640 1 -- 172.21.15.42:0/3977744304 shutdown_connections 2024-08-30T09:01:17.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.650+0000 7f2891e32640 1 --2- 172.21.15.42:0/3977744304 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f28740b4be0 0x7f28740b70f0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:17.659 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.650+0000 7f2891e32640 1 --2- 172.21.15.42:0/3977744304 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f288c108170 0x7f288c19ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:17.659 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.650+0000 7f2891e32640 1 --2- 172.21.15.42:0/3977744304 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f288c108ac0 0x7f288c1a1cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:17.659 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.650+0000 7f2891e32640 1 --2- 172.21.15.42:0/3977744304 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f288c102880 0x7f288c19c480 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:17.659 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.650+0000 7f2891e32640 1 -- 172.21.15.42:0/3977744304 >> 172.21.15.42:0/3977744304 conn(0x7f288c079550 msgr2=0x7f288c06b6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:01:17.659 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.651+0000 7f2891e32640 1 -- 172.21.15.42:0/3977744304 shutdown_connections 2024-08-30T09:01:17.659 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:17.651+0000 7f2891e32640 1 -- 172.21.15.42:0/3977744304 wait complete. 2024-08-30T09:01:18.725 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-08-30T09:01:18.775 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:01:18 smithi042 ceph-mon[18652]: pgmap v389: 129 pgs: 129 active+clean; 971 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 23 MiB/s wr, 924 op/s 2024-08-30T09:01:18.826 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:01:18 smithi167 ceph-mon[22063]: pgmap v389: 129 pgs: 129 active+clean; 971 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 23 MiB/s wr, 924 op/s 2024-08-30T09:01:18.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:01:18 smithi190 ceph-mon[22001]: pgmap v389: 129 pgs: 129 active+clean; 971 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 23 MiB/s wr, 924 op/s 2024-08-30T09:01:19.139 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T09:01:19.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.823+0000 7f8b72967640 1 -- 172.21.15.42:0/1606943857 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8b6c11c9d0 msgr2=0x7f8b6c11eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:01:19.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.823+0000 7f8b72967640 1 --2- 172.21.15.42:0/1606943857 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8b6c11c9d0 0x7f8b6c11eee0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f8b5c01a910 tx=0x7f8b5c03b430 comp rx=0 tx=0).stop 2024-08-30T09:01:19.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.823+0000 7f8b72967640 1 -- 172.21.15.42:0/1606943857 shutdown_connections 2024-08-30T09:01:19.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.823+0000 7f8b72967640 1 --2- 172.21.15.42:0/1606943857 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8b6c10a780 0x7f8b6c10ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:19.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.823+0000 7f8b72967640 1 --2- 172.21.15.42:0/1606943857 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8b6c11c9d0 0x7f8b6c11eee0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:19.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.823+0000 7f8b72967640 1 --2- 172.21.15.42:0/1606943857 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8b6c111ce0 0x7f8b6c10a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:19.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.823+0000 7f8b72967640 1 -- 172.21.15.42:0/1606943857 >> 172.21.15.42:0/1606943857 conn(0x7f8b6c070c60 msgr2=0x7f8b6c071070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:01:19.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.825+0000 7f8b72967640 1 -- 172.21.15.42:0/1606943857 shutdown_connections 2024-08-30T09:01:19.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.825+0000 7f8b72967640 1 -- 172.21.15.42:0/1606943857 wait complete. 2024-08-30T09:01:19.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.826+0000 7f8b72967640 1 Processor -- start 2024-08-30T09:01:19.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.826+0000 7f8b72967640 1 -- start start 2024-08-30T09:01:19.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.826+0000 7f8b72967640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8b6c10a780 0x7f8b6c11c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:01:19.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.827+0000 7f8b72967640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8b6c111ce0 0x7f8b6c118480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:01:19.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.827+0000 7f8b6bfff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8b6c10a780 0x7f8b6c11c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:01:19.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.827+0000 7f8b72967640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8b6c11c9d0 0x7f8b6c1189e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:01:19.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.827+0000 7f8b72967640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f8b6c1149c0 con 0x7f8b6c10a780 2024-08-30T09:01:19.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.827+0000 7f8b72967640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f8b6c114840 con 0x7f8b6c111ce0 2024-08-30T09:01:19.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.827+0000 7f8b72967640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f8b6c114b40 con 0x7f8b6c11c9d0 2024-08-30T09:01:19.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.827+0000 7f8b6bfff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8b6c10a780 0x7f8b6c11c1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:55464/0 (socket says 172.21.15.42:55464) 2024-08-30T09:01:19.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.827+0000 7f8b6bfff640 1 -- 172.21.15.42:0/3523967513 learned_addr learned my addr 172.21.15.42:0/3523967513 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:01:19.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.827+0000 7f8b6b7fe640 1 --2- 172.21.15.42:0/3523967513 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8b6c111ce0 0x7f8b6c118480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:01:19.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.827+0000 7f8b70f04640 1 --2- 172.21.15.42:0/3523967513 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8b6c11c9d0 0x7f8b6c1189e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:01:19.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.828+0000 7f8b6bfff640 1 -- 172.21.15.42:0/3523967513 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8b6c11c9d0 msgr2=0x7f8b6c1189e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:01:19.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.828+0000 7f8b6bfff640 1 --2- 172.21.15.42:0/3523967513 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8b6c11c9d0 0x7f8b6c1189e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:19.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.828+0000 7f8b6bfff640 1 -- 172.21.15.42:0/3523967513 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8b6c111ce0 msgr2=0x7f8b6c118480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:01:19.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.828+0000 7f8b6bfff640 1 --2- 172.21.15.42:0/3523967513 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8b6c111ce0 0x7f8b6c118480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:19.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.828+0000 7f8b6bfff640 1 -- 172.21.15.42:0/3523967513 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b6c119200 con 0x7f8b6c10a780 2024-08-30T09:01:19.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.828+0000 7f8b70f04640 1 --2- 172.21.15.42:0/3523967513 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8b6c11c9d0 0x7f8b6c1189e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-30T09:01:19.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.828+0000 7f8b6bfff640 1 --2- 172.21.15.42:0/3523967513 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8b6c10a780 0x7f8b6c11c1a0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f8b6000b4a0 tx=0x7f8b6000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:01:19.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.828+0000 7f8b697fa640 1 -- 172.21.15.42:0/3523967513 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b60014070 con 0x7f8b6c10a780 2024-08-30T09:01:19.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.828+0000 7f8b697fa640 1 -- 172.21.15.42:0/3523967513 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8b60010ac0 con 0x7f8b6c10a780 2024-08-30T09:01:19.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.828+0000 7f8b697fa640 1 -- 172.21.15.42:0/3523967513 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b60017c30 con 0x7f8b6c10a780 2024-08-30T09:01:19.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.828+0000 7f8b72967640 1 -- 172.21.15.42:0/3523967513 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b6c1acbd0 con 0x7f8b6c10a780 2024-08-30T09:01:19.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.829+0000 7f8b72967640 1 -- 172.21.15.42:0/3523967513 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b6c1ad140 con 0x7f8b6c10a780 2024-08-30T09:01:19.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.832+0000 7f8b78edc740 1 Processor -- start 2024-08-30T09:01:19.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.832+0000 7f8b78edc740 1 -- start start 2024-08-30T09:01:19.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.833+0000 7f8b78edc740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a850883070 0x55a850c28350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:01:19.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.833+0000 7f8b78edc740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a85085a1b0 0x55a850c28890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:01:19.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.833+0000 7f8b78edc740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a850862390 0x55a850c28dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:01:19.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.833+0000 7f8b78edc740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55a850b9f680 con 0x55a850862390 2024-08-30T09:01:19.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.833+0000 7f8b78edc740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55a850b8ec10 con 0x55a85085a1b0 2024-08-30T09:01:19.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.833+0000 7f8b78edc740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55a850b92bc0 con 0x55a850883070 2024-08-30T09:01:19.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.833+0000 7f8b70f04640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a850883070 0x55a850c28350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:01:19.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.833+0000 7f8b70f04640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a850883070 0x55a850c28350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:36912/0 (socket says 172.21.15.42:36912) 2024-08-30T09:01:19.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.833+0000 7f8b70f04640 1 -- 172.21.15.42:0/2373421671 learned_addr learned my addr 172.21.15.42:0/2373421671 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:01:19.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.833+0000 7f8b6b7fe640 1 --2- 172.21.15.42:0/2373421671 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a850862390 0x55a850c28dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:01:19.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.834+0000 7f8b70f04640 1 -- 172.21.15.42:0/2373421671 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a85085a1b0 msgr2=0x55a850c28890 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-30T09:01:19.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.834+0000 7f8b70f04640 1 --2- 172.21.15.42:0/2373421671 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a85085a1b0 0x55a850c28890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:19.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.834+0000 7f8b70f04640 1 -- 172.21.15.42:0/2373421671 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a850862390 msgr2=0x55a850c28dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:01:19.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.834+0000 7f8b70f04640 1 --2- 172.21.15.42:0/2373421671 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a850862390 0x55a850c28dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:19.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.834+0000 7f8b70f04640 1 -- 172.21.15.42:0/2373421671 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a8508ee650 con 0x55a850883070 2024-08-30T09:01:19.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.834+0000 7f8b70f04640 1 --2- 172.21.15.42:0/2373421671 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a850883070 0x55a850c28350 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f8b5c03b9a0 tx=0x7f8b5c0048e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=92126108a0f9c755 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:01:19.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.835+0000 7f8b527fc640 1 -- 172.21.15.42:0/2373421671 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b5c046070 con 0x55a850883070 2024-08-30T09:01:19.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.835+0000 7f8b527fc640 1 -- 172.21.15.42:0/2373421671 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8b5c004ad0 con 0x55a850883070 2024-08-30T09:01:19.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.835+0000 7f8b527fc640 1 -- 172.21.15.42:0/2373421671 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b5c0035f0 con 0x55a850883070 2024-08-30T09:01:19.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.835+0000 7f8b697fa640 1 -- 172.21.15.42:0/3523967513 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7f8b60017070 con 0x7f8b6c10a780 2024-08-30T09:01:19.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.836+0000 7f8b78edc740 1 -- 172.21.15.42:0/2373421671 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a850883070 msgr2=0x55a850c28350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:01:19.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.836+0000 7f8b78edc740 1 --2- 172.21.15.42:0/2373421671 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a850883070 0x55a850c28350 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f8b5c03b9a0 tx=0x7f8b5c0048e0 comp rx=0 tx=0).stop 2024-08-30T09:01:19.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.837+0000 7f8b697fa640 1 --2- 172.21.15.42:0/3523967513 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f8b540b4bf0 0x7f8b540b7100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:01:19.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.837+0000 7f8b697fa640 1 -- 172.21.15.42:0/3523967513 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f8b600e4aa0 con 0x7f8b6c10a780 2024-08-30T09:01:19.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.837+0000 7f8b70f04640 1 --2- 172.21.15.42:0/3523967513 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f8b540b4bf0 0x7f8b540b7100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:01:19.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.838+0000 7f8b78edc740 1 -- 172.21.15.42:0/2373421671 shutdown_connections 2024-08-30T09:01:19.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.838+0000 7f8b78edc740 1 --2- 172.21.15.42:0/2373421671 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a850862390 0x55a850c28dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:19.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.838+0000 7f8b78edc740 1 --2- 172.21.15.42:0/2373421671 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a85085a1b0 0x55a850c28890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:19.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.838+0000 7f8b78edc740 1 --2- 172.21.15.42:0/2373421671 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a850883070 0x55a850c28350 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:19.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.838+0000 7f8b78edc740 1 -- 172.21.15.42:0/2373421671 >> 172.21.15.42:0/2373421671 conn(0x55a850877460 msgr2=0x55a850c26240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:01:19.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.839+0000 7f8b78edc740 1 -- 172.21.15.42:0/2373421671 shutdown_connections 2024-08-30T09:01:19.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.839+0000 7f8b78edc740 1 -- 172.21.15.42:0/2373421671 wait complete. 2024-08-30T09:01:19.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.839+0000 7f8b70f04640 1 --2- 172.21.15.42:0/3523967513 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f8b540b4bf0 0x7f8b540b7100 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f8b5c004dc0 tx=0x7f8b5c005670 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:01:19.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.839+0000 7f8b78edc740 1 -- start start 2024-08-30T09:01:19.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.840+0000 7f8b78edc740 1 Processor -- start 2024-08-30T09:01:19.847 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.841+0000 7f8b78edc740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a85085a1b0 0x55a850c33ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:01:19.847 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.841+0000 7f8b78edc740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a850862390 0x55a850c33ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:01:19.847 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.841+0000 7f8b78edc740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a850883070 0x55a850c3fe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:01:19.847 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.841+0000 7f8b78edc740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55a8509f1700 con 0x55a85085a1b0 2024-08-30T09:01:19.847 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.841+0000 7f8b78edc740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55a850b15750 con 0x55a850862390 2024-08-30T09:01:19.847 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.842+0000 7f8b78edc740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55a850c36b20 con 0x55a850883070 2024-08-30T09:01:19.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.842+0000 7f8b70f04640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a850883070 0x55a850c3fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:01:19.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.842+0000 7f8b70f04640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a850883070 0x55a850c3fe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:36916/0 (socket says 172.21.15.42:36916) 2024-08-30T09:01:19.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.842+0000 7f8b70f04640 1 -- 172.21.15.42:0/4052295681 learned_addr learned my addr 172.21.15.42:0/4052295681 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:01:19.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.842+0000 7f8b78edc740 10 client.0 _ll_register_callbacks cb 0x55a850b44e10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:01:19.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.842+0000 7f8b6bfff640 1 --2- 172.21.15.42:0/4052295681 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a85085a1b0 0x55a850c33ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:01:19.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.842+0000 7f8b6b7fe640 1 --2- 172.21.15.42:0/4052295681 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a850862390 0x55a850c33ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:01:19.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.842+0000 7f8b70f04640 1 -- 172.21.15.42:0/4052295681 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a850862390 msgr2=0x55a850c33ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:01:19.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.842+0000 7f8b70f04640 1 --2- 172.21.15.42:0/4052295681 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a850862390 0x55a850c33ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:19.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.842+0000 7f8b70f04640 1 -- 172.21.15.42:0/4052295681 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a85085a1b0 msgr2=0x55a850c33ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:01:19.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.842+0000 7f8b70f04640 1 --2- 172.21.15.42:0/4052295681 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a85085a1b0 0x55a850c33ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:19.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.842+0000 7f8b70f04640 1 -- 172.21.15.42:0/4052295681 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a850b4e900 con 0x55a850883070 2024-08-30T09:01:19.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.843+0000 7f8b70f04640 1 --2- 172.21.15.42:0/4052295681 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a850883070 0x55a850c3fe40 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f8b5c006010 tx=0x7f8b5c03b930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bb43e11d5dd3f84e server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:01:19.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.843+0000 7f8b4b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-08-30T09:01:19.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.843+0000 7f8b4b7fe640 1 -- 172.21.15.42:0/4052295681 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b5c089070 con 0x55a850883070 2024-08-30T09:01:19.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.843+0000 7f8b78edc740 1 -- 172.21.15.42:0/4052295681 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8b5c004ad0 con 0x55a850883070 2024-08-30T09:01:19.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.843+0000 7f8b4b7fe640 1 -- 172.21.15.42:0/4052295681 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f8b5c0035f0 con 0x55a850883070 2024-08-30T09:01:19.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.843+0000 7f8b4b7fe640 1 -- 172.21.15.42:0/4052295681 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8b5c095050 con 0x55a850883070 2024-08-30T09:01:19.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.843+0000 7f8b78edc740 10 client.25237 fetch_fsmap learned FSMap version 29 2024-08-30T09:01:19.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.843+0000 7f8b78edc740 1 -- 172.21.15.42:0/4052295681 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a850c42900 con 0x55a850883070 2024-08-30T09:01:19.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.844+0000 7f8b4b7fe640 1 -- 172.21.15.42:0/4052295681 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8b5c004ad0 con 0x55a850883070 2024-08-30T09:01:19.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.844+0000 7f8b4b7fe640 1 -- 172.21.15.42:0/4052295681 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f8b5c061070 con 0x55a850883070 2024-08-30T09:01:19.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.844+0000 7f8b78edc740 10 client.25237 fetch_fsmap finished waiting for FSMap version 29 2024-08-30T09:01:19.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.844+0000 7f8b78edc740 10 client.25237 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-08-30T09:01:19.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.844+0000 7f8b78edc740 20 client.25237 populate_metadata read hostname 'smithi042' 2024-08-30T09:01:19.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.844+0000 7f8b78edc740 1 --2- 172.21.15.42:0/4052295681 >> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] conn(0x55a8508419f0 0x55a850c45c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:01:19.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.844+0000 7f8b78edc740 4 client.25237 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-30T09:01:19.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.844+0000 7f8b78edc740 1 -- 172.21.15.42:0/4052295681 --> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a850ba15d0 con 0x55a8508419f0 2024-08-30T09:01:19.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.844+0000 7f8b6bfff640 1 --2- 172.21.15.42:0/4052295681 >> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] conn(0x55a8508419f0 0x55a850c45c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:01:19.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.845+0000 7f8b4b7fe640 10 client.25237 ms_handle_connect on v2:172.21.15.190:6838/719981110 2024-08-30T09:01:19.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.845+0000 7f8b6bfff640 1 --2- 172.21.15.42:0/4052295681 >> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] conn(0x55a8508419f0 0x55a850c45c60 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=12d1ac26bbe172 server_cookie=daccab3cfacd278e in_seq=0 out_seq=0 2024-08-30T09:01:19.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.846+0000 7f8b4b7fe640 1 -- 172.21.15.42:0/4052295681 <== mds.1 v2:172.21.15.190:6838/719981110 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a850ba15d0 con 0x55a8508419f0 2024-08-30T09:01:19.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.846+0000 7f8b4b7fe640 10 client.25237 handle_command_reply: tid=0 2024-08-30T09:01:19.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.846+0000 7f8b78edc740 1 client.25237 shutdown 2024-08-30T09:01:19.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.846+0000 7f8b78edc740 1 -- 172.21.15.42:0/4052295681 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a850883070 msgr2=0x55a850c3fe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:01:19.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.846+0000 7f8b78edc740 1 --2- 172.21.15.42:0/4052295681 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a850883070 0x55a850c3fe40 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f8b5c006010 tx=0x7f8b5c03b930 comp rx=0 tx=0).stop 2024-08-30T09:01:19.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.846+0000 7f8b78edc740 1 -- 172.21.15.42:0/4052295681 shutdown_connections 2024-08-30T09:01:19.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.846+0000 7f8b78edc740 1 --2- 172.21.15.42:0/4052295681 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a850883070 0x55a850c3fe40 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:19.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.846+0000 7f8b78edc740 1 --2- 172.21.15.42:0/4052295681 >> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] conn(0x55a8508419f0 0x55a850c45c60 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:19.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.846+0000 7f8b4b7fe640 0 client.25237 ms_handle_reset on v2:172.21.15.190:6838/719981110 2024-08-30T09:01:19.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.846+0000 7f8b78edc740 1 --2- 172.21.15.42:0/4052295681 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a850862390 0x55a850c33ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:19.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.847+0000 7f8b78edc740 1 --2- 172.21.15.42:0/4052295681 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a85085a1b0 0x55a850c33ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:19.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.847+0000 7f8b78edc740 1 -- 172.21.15.42:0/4052295681 >> 172.21.15.42:0/4052295681 conn(0x55a850877460 msgr2=0x55a850c2b4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:01:19.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.847+0000 7f8b78edc740 1 -- 172.21.15.42:0/4052295681 shutdown_connections 2024-08-30T09:01:19.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.847+0000 7f8b78edc740 1 -- 172.21.15.42:0/4052295681 wait complete. 2024-08-30T09:01:19.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.847+0000 7f8b78edc740 20 client.25237 trim_cache size 0 max 16384 2024-08-30T09:01:19.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.871+0000 7f8b78edc740 1 Processor -- start 2024-08-30T09:01:19.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.871+0000 7f8b78edc740 1 -- start start 2024-08-30T09:01:19.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.872+0000 7f8b78edc740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a85085a1b0 0x55a850c29a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:01:19.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.872+0000 7f8b78edc740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a850862390 0x55a850c3fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:01:19.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.872+0000 7f8b78edc740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a850883070 0x55a850c30ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:01:19.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.872+0000 7f8b78edc740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f8b6c0817a0 con 0x55a85085a1b0 2024-08-30T09:01:19.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.872+0000 7f8b78edc740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f8b6c07f070 con 0x55a850862390 2024-08-30T09:01:19.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.872+0000 7f8b78edc740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f8b6c07cec0 con 0x55a850883070 2024-08-30T09:01:19.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.872+0000 7f8b70f04640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a850883070 0x55a850c30ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:01:19.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.872+0000 7f8b70f04640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a850883070 0x55a850c30ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:36918/0 (socket says 172.21.15.42:36918) 2024-08-30T09:01:19.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.872+0000 7f8b70f04640 1 -- 172.21.15.42:0/1330063662 learned_addr learned my addr 172.21.15.42:0/1330063662 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:01:19.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.872+0000 7f8b6bfff640 1 --2- 172.21.15.42:0/1330063662 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a85085a1b0 0x55a850c29a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:01:19.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.872+0000 7f8b6b7fe640 1 --2- 172.21.15.42:0/1330063662 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a850862390 0x55a850c3fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:01:19.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.872+0000 7f8b70f04640 1 -- 172.21.15.42:0/1330063662 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a850862390 msgr2=0x55a850c3fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:01:19.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.872+0000 7f8b70f04640 1 --2- 172.21.15.42:0/1330063662 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a850862390 0x55a850c3fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:19.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.872+0000 7f8b70f04640 1 -- 172.21.15.42:0/1330063662 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a85085a1b0 msgr2=0x55a850c29a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:01:19.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.872+0000 7f8b70f04640 1 --2- 172.21.15.42:0/1330063662 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a85085a1b0 0x55a850c29a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:19.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.872+0000 7f8b70f04640 1 -- 172.21.15.42:0/1330063662 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a850c2a640 con 0x55a850883070 2024-08-30T09:01:19.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.873+0000 7f8b70f04640 1 --2- 172.21.15.42:0/1330063662 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a850883070 0x55a850c30ca0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f8b5c005620 tx=0x7f8b5c005a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=935bbe0b0ac6f2e0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:01:19.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.873+0000 7f8b51ffb640 1 -- 172.21.15.42:0/1330063662 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b5c089040 con 0x55a850883070 2024-08-30T09:01:19.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.874+0000 7f8b51ffb640 1 -- 172.21.15.42:0/1330063662 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8b5c004ad0 con 0x55a850883070 2024-08-30T09:01:19.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.874+0000 7f8b51ffb640 1 -- 172.21.15.42:0/1330063662 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b5c046070 con 0x55a850883070 2024-08-30T09:01:19.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.875+0000 7f8b78edc740 1 -- 172.21.15.42:0/1330063662 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a850883070 msgr2=0x55a850c30ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:01:19.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.875+0000 7f8b78edc740 1 --2- 172.21.15.42:0/1330063662 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a850883070 0x55a850c30ca0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f8b5c005620 tx=0x7f8b5c005a20 comp rx=0 tx=0).stop 2024-08-30T09:01:19.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.875+0000 7f8b78edc740 1 -- 172.21.15.42:0/1330063662 shutdown_connections 2024-08-30T09:01:19.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.875+0000 7f8b78edc740 1 --2- 172.21.15.42:0/1330063662 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a850883070 0x55a850c30ca0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:19.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.875+0000 7f8b78edc740 1 --2- 172.21.15.42:0/1330063662 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a850862390 0x55a850c3fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:19.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.876+0000 7f8b78edc740 1 --2- 172.21.15.42:0/1330063662 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a85085a1b0 0x55a850c29a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:19.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.876+0000 7f8b78edc740 1 -- 172.21.15.42:0/1330063662 >> 172.21.15.42:0/1330063662 conn(0x55a850877460 msgr2=0x55a850c29380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:01:19.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.876+0000 7f8b78edc740 1 -- 172.21.15.42:0/1330063662 shutdown_connections 2024-08-30T09:01:19.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.876+0000 7f8b78edc740 1 -- 172.21.15.42:0/1330063662 wait complete. 2024-08-30T09:01:19.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.877+0000 7f8b78edc740 1 -- start start 2024-08-30T09:01:19.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.877+0000 7f8b78edc740 1 Processor -- start 2024-08-30T09:01:19.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.878+0000 7f8b78edc740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a85085a1b0 0x55a850c46f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:01:19.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.878+0000 7f8b78edc740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a850862390 0x55a850c47470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:01:19.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.878+0000 7f8b78edc740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a850883070 0x55a850caf3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:01:19.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.878+0000 7f8b78edc740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f8b6c07d530 con 0x55a850862390 2024-08-30T09:01:19.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.878+0000 7f8b70f04640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a850883070 0x55a850caf3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:01:19.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.878+0000 7f8b70f04640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a850883070 0x55a850caf3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:36926/0 (socket says 172.21.15.42:36926) 2024-08-30T09:01:19.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.878+0000 7f8b70f04640 1 -- 172.21.15.42:0/616201676 learned_addr learned my addr 172.21.15.42:0/616201676 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:01:19.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.878+0000 7f8b78edc740 1 -- 172.21.15.42:0/616201676 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f8b6c07e9a0 con 0x55a85085a1b0 2024-08-30T09:01:19.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.878+0000 7f8b78edc740 1 -- 172.21.15.42:0/616201676 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f8b6c07dc00 con 0x55a850883070 2024-08-30T09:01:19.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.878+0000 7f8b6b7fe640 1 --2- 172.21.15.42:0/616201676 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a850862390 0x55a850c47470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:01:19.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.878+0000 7f8b6bfff640 1 --2- 172.21.15.42:0/616201676 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a85085a1b0 0x55a850c46f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:01:19.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.878+0000 7f8b78edc740 10 client.0 _ll_register_callbacks cb 0x7f8b64009b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:01:19.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.878+0000 7f8b70f04640 1 -- 172.21.15.42:0/616201676 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a85085a1b0 msgr2=0x55a850c46f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:01:19.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.878+0000 7f8b70f04640 1 --2- 172.21.15.42:0/616201676 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a85085a1b0 0x55a850c46f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:19.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.878+0000 7f8b70f04640 1 -- 172.21.15.42:0/616201676 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a850862390 msgr2=0x55a850c47470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:01:19.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.878+0000 7f8b70f04640 1 --2- 172.21.15.42:0/616201676 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a850862390 0x55a850c47470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:19.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.879+0000 7f8b70f04640 1 -- 172.21.15.42:0/616201676 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a850c47ca0 con 0x55a850883070 2024-08-30T09:01:19.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.879+0000 7f8b70f04640 1 --2- 172.21.15.42:0/616201676 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a850883070 0x55a850caf3e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f8b5c01a6d0 tx=0x7f8b5c03b930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ea1fca0b71980c1d server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:01:19.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.879+0000 7f8b4b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-08-30T09:01:19.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.879+0000 7f8b4b7fe640 1 -- 172.21.15.42:0/616201676 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b5c089070 con 0x55a850883070 2024-08-30T09:01:19.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.879+0000 7f8b4b7fe640 1 -- 172.21.15.42:0/616201676 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f8b5c0035f0 con 0x55a850883070 2024-08-30T09:01:19.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.879+0000 7f8b78edc740 1 -- 172.21.15.42:0/616201676 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8b5c004ad0 con 0x55a850883070 2024-08-30T09:01:19.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.880+0000 7f8b4b7fe640 1 -- 172.21.15.42:0/616201676 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8b5c0aa050 con 0x55a850883070 2024-08-30T09:01:19.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.880+0000 7f8b78edc740 10 client.25243 fetch_fsmap learned FSMap version 29 2024-08-30T09:01:19.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.880+0000 7f8b78edc740 1 -- 172.21.15.42:0/616201676 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a850cb08a0 con 0x55a850883070 2024-08-30T09:01:19.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.880+0000 7f8b4b7fe640 1 -- 172.21.15.42:0/616201676 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8b5c003ca0 con 0x55a850883070 2024-08-30T09:01:19.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.880+0000 7f8b4b7fe640 1 -- 172.21.15.42:0/616201676 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f8b5c061070 con 0x55a850883070 2024-08-30T09:01:19.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.880+0000 7f8b78edc740 10 client.25243 fetch_fsmap finished waiting for FSMap version 29 2024-08-30T09:01:19.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.880+0000 7f8b78edc740 10 client.25243 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-08-30T09:01:19.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.880+0000 7f8b78edc740 20 client.25243 populate_metadata read hostname 'smithi042' 2024-08-30T09:01:19.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.880+0000 7f8b78edc740 1 --2- 172.21.15.42:0/616201676 >> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] conn(0x55a850816da0 0x55a850cb2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:01:19.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.880+0000 7f8b78edc740 4 client.25243 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-30T09:01:19.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.880+0000 7f8b78edc740 1 -- 172.21.15.42:0/616201676 --> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f8b5c004ad0 con 0x55a850816da0 2024-08-30T09:01:19.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.881+0000 7f8b6bfff640 1 --2- 172.21.15.42:0/616201676 >> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] conn(0x55a850816da0 0x55a850cb2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:01:19.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.882+0000 7f8b6bfff640 1 --2- 172.21.15.42:0/616201676 >> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] conn(0x55a850816da0 0x55a850cb2fc0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=410522006be31d4a server_cookie=ee8edc8110258720 in_seq=0 out_seq=0 2024-08-30T09:01:19.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.882+0000 7f8b4b7fe640 10 client.25243 ms_handle_connect on v2:172.21.15.190:6838/719981110 2024-08-30T09:01:19.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.882+0000 7f8b4b7fe640 1 -- 172.21.15.42:0/616201676 <== mds.1 v2:172.21.15.190:6838/719981110 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f8b5c004ad0 con 0x55a850816da0 2024-08-30T09:01:19.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.882+0000 7f8b4b7fe640 10 client.25243 handle_command_reply: tid=0 2024-08-30T09:01:19.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.882+0000 7f8b78edc740 1 client.25243 shutdown 2024-08-30T09:01:19.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.882+0000 7f8b78edc740 1 -- 172.21.15.42:0/616201676 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a850883070 msgr2=0x55a850caf3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:01:19.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.882+0000 7f8b78edc740 1 --2- 172.21.15.42:0/616201676 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a850883070 0x55a850caf3e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f8b5c01a6d0 tx=0x7f8b5c03b930 comp rx=0 tx=0).stop 2024-08-30T09:01:19.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.882+0000 7f8b78edc740 1 -- 172.21.15.42:0/616201676 shutdown_connections 2024-08-30T09:01:19.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.883+0000 7f8b78edc740 1 --2- 172.21.15.42:0/616201676 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a850883070 0x55a850caf3e0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:19.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.883+0000 7f8b78edc740 1 --2- 172.21.15.42:0/616201676 >> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] conn(0x55a850816da0 0x55a850cb2fc0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:19.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.883+0000 7f8b78edc740 1 --2- 172.21.15.42:0/616201676 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a850862390 0x55a850c47470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:19.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.883+0000 7f8b4b7fe640 0 client.25243 ms_handle_reset on v2:172.21.15.190:6838/719981110 2024-08-30T09:01:19.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.883+0000 7f8b78edc740 1 --2- 172.21.15.42:0/616201676 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a85085a1b0 0x55a850c46f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:19.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.883+0000 7f8b78edc740 1 -- 172.21.15.42:0/616201676 >> 172.21.15.42:0/616201676 conn(0x55a850877460 msgr2=0x55a850c26200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:01:19.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.883+0000 7f8b78edc740 1 -- 172.21.15.42:0/616201676 shutdown_connections 2024-08-30T09:01:19.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.883+0000 7f8b78edc740 1 -- 172.21.15.42:0/616201676 wait complete. 2024-08-30T09:01:19.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.883+0000 7f8b78edc740 20 client.25243 trim_cache size 0 max 16384 2024-08-30T09:01:19.888 INFO:teuthology.orchestra.run.smithi042.stdout:[] 2024-08-30T09:01:19.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.886+0000 7f8b72967640 1 -- 172.21.15.42:0/3523967513 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f8b540b4bf0 msgr2=0x7f8b540b7100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:01:19.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.886+0000 7f8b72967640 1 --2- 172.21.15.42:0/3523967513 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f8b540b4bf0 0x7f8b540b7100 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f8b5c004dc0 tx=0x7f8b5c005670 comp rx=0 tx=0).stop 2024-08-30T09:01:19.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.886+0000 7f8b72967640 1 -- 172.21.15.42:0/3523967513 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8b6c10a780 msgr2=0x7f8b6c11c1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:01:19.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.886+0000 7f8b72967640 1 --2- 172.21.15.42:0/3523967513 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8b6c10a780 0x7f8b6c11c1a0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f8b6000b4a0 tx=0x7f8b6000b960 comp rx=0 tx=0).stop 2024-08-30T09:01:19.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.887+0000 7f8b72967640 1 -- 172.21.15.42:0/3523967513 shutdown_connections 2024-08-30T09:01:19.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.887+0000 7f8b72967640 1 --2- 172.21.15.42:0/3523967513 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f8b540b4bf0 0x7f8b540b7100 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:19.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.887+0000 7f8b72967640 1 --2- 172.21.15.42:0/3523967513 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8b6c11c9d0 0x7f8b6c1189e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:19.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.887+0000 7f8b72967640 1 --2- 172.21.15.42:0/3523967513 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8b6c111ce0 0x7f8b6c118480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:19.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.887+0000 7f8b72967640 1 --2- 172.21.15.42:0/3523967513 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8b6c10a780 0x7f8b6c11c1a0 secure :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f8b6000b4a0 tx=0x7f8b6000b960 comp rx=0 tx=0).stop 2024-08-30T09:01:19.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.887+0000 7f8b72967640 1 -- 172.21.15.42:0/3523967513 >> 172.21.15.42:0/3523967513 conn(0x7f8b6c070c60 msgr2=0x7f8b6c10b7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:01:19.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.887+0000 7f8b72967640 1 -- 172.21.15.42:0/3523967513 shutdown_connections 2024-08-30T09:01:19.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:19.887+0000 7f8b72967640 1 -- 172.21.15.42:0/3523967513 wait complete. 2024-08-30T09:01:20.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:01:20 smithi167 ceph-mon[22063]: pgmap v390: 129 pgs: 129 active+clean; 975 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 14 MiB/s wr, 717 op/s 2024-08-30T09:01:20.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:01:20 smithi042 ceph-mon[18652]: pgmap v390: 129 pgs: 129 active+clean; 975 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 14 MiB/s wr, 717 op/s 2024-08-30T09:01:20.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:01:20 smithi190 ceph-mon[22001]: pgmap v390: 129 pgs: 129 active+clean; 975 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 14 MiB/s wr, 717 op/s 2024-08-30T09:01:20.953 INFO:teuthology.orchestra.run.smithi042.stderr:progress: 35.0 s, 3034.0 tps, lat 32.794 ms stddev 39.728 2024-08-30T09:01:22.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:01:22 smithi167 ceph-mon[22063]: pgmap v391: 129 pgs: 129 active+clean; 976 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 7.9 MiB/s wr, 611 op/s 2024-08-30T09:01:22.882 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:01:22 smithi042 ceph-mon[18652]: pgmap v391: 129 pgs: 129 active+clean; 976 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 7.9 MiB/s wr, 611 op/s 2024-08-30T09:01:22.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:01:22 smithi190 ceph-mon[22001]: pgmap v391: 129 pgs: 129 active+clean; 976 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 7.9 MiB/s wr, 611 op/s 2024-08-30T09:01:23.025 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-08-30T09:01:23.454 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T09:01:24.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.716+0000 7f36cc03a640 1 -- 172.21.15.42:0/132223185 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f36c4104750 msgr2=0x7f36c4104b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:01:24.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.716+0000 7f36cc03a640 1 --2- 172.21.15.42:0/132223185 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f36c4104750 0x7f36c4104b60 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f36c0009900 tx=0x7f36c002a410 comp rx=0 tx=0).stop 2024-08-30T09:01:24.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.717+0000 7f36cc03a640 1 -- 172.21.15.42:0/132223185 shutdown_connections 2024-08-30T09:01:24.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.717+0000 7f36cc03a640 1 --2- 172.21.15.42:0/132223185 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f36c410dbb0 0x7f36c41100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:24.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.717+0000 7f36cc03a640 1 --2- 172.21.15.42:0/132223185 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f36c41050a0 0x7f36c410d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:24.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.717+0000 7f36cc03a640 1 --2- 172.21.15.42:0/132223185 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f36c4104750 0x7f36c4104b60 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:24.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.717+0000 7f36cc03a640 1 -- 172.21.15.42:0/132223185 >> 172.21.15.42:0/132223185 conn(0x7f36c40794d0 msgr2=0x7f36c40798e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:01:24.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.718+0000 7f36cc03a640 1 -- 172.21.15.42:0/132223185 shutdown_connections 2024-08-30T09:01:24.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.718+0000 7f36cc03a640 1 -- 172.21.15.42:0/132223185 wait complete. 2024-08-30T09:01:24.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.719+0000 7f36cc03a640 1 Processor -- start 2024-08-30T09:01:24.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.719+0000 7f36cc03a640 1 -- start start 2024-08-30T09:01:24.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.720+0000 7f36cc03a640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f36c4104750 0x7f36c419a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:01:24.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.720+0000 7f36cc03a640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f36c41050a0 0x7f36c419a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:01:24.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.720+0000 7f36cc03a640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f36c410dbb0 0x7f36c419fb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:01:24.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.720+0000 7f36cc03a640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f36c4113810 con 0x7f36c410dbb0 2024-08-30T09:01:24.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.720+0000 7f36cc03a640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f36c4113690 con 0x7f36c4104750 2024-08-30T09:01:24.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.720+0000 7f36cc03a640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f36c4113990 con 0x7f36c41050a0 2024-08-30T09:01:24.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.720+0000 7f36c95d5640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f36c41050a0 0x7f36c419a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:01:24.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.720+0000 7f36c95d5640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f36c41050a0 0x7f36c419a8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:36934/0 (socket says 172.21.15.42:36934) 2024-08-30T09:01:24.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.720+0000 7f36c95d5640 1 -- 172.21.15.42:0/1032315355 learned_addr learned my addr 172.21.15.42:0/1032315355 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:01:24.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.720+0000 7f36c9dd6640 1 --2- 172.21.15.42:0/1032315355 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f36c4104750 0x7f36c419a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:01:24.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.720+0000 7f36ca5d7640 1 --2- 172.21.15.42:0/1032315355 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f36c410dbb0 0x7f36c419fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:01:24.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.720+0000 7f36c95d5640 1 -- 172.21.15.42:0/1032315355 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f36c4104750 msgr2=0x7f36c419a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:01:24.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.720+0000 7f36c95d5640 1 --2- 172.21.15.42:0/1032315355 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f36c4104750 0x7f36c419a320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:24.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.720+0000 7f36c95d5640 1 -- 172.21.15.42:0/1032315355 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f36c410dbb0 msgr2=0x7f36c419fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:01:24.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.720+0000 7f36c95d5640 1 --2- 172.21.15.42:0/1032315355 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f36c410dbb0 0x7f36c419fb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:24.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.720+0000 7f36c95d5640 1 -- 172.21.15.42:0/1032315355 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36c41a0290 con 0x7f36c41050a0 2024-08-30T09:01:24.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.721+0000 7f36c95d5640 1 --2- 172.21.15.42:0/1032315355 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f36c41050a0 0x7f36c419a8c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f36b400e9f0 tx=0x7f36b400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:01:24.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.721+0000 7f36baffd640 1 -- 172.21.15.42:0/1032315355 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36b400ccc0 con 0x7f36c41050a0 2024-08-30T09:01:24.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.721+0000 7f36baffd640 1 -- 172.21.15.42:0/1032315355 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f36b40108e0 con 0x7f36c41050a0 2024-08-30T09:01:24.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.722+0000 7f36cc03a640 1 -- 172.21.15.42:0/1032315355 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36c41a0580 con 0x7f36c41050a0 2024-08-30T09:01:24.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.722+0000 7f36cc03a640 1 -- 172.21.15.42:0/1032315355 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36c41a6ce0 con 0x7f36c41050a0 2024-08-30T09:01:24.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.722+0000 7f36baffd640 1 -- 172.21.15.42:0/1032315355 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36b4018860 con 0x7f36c41050a0 2024-08-30T09:01:24.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.725+0000 7f36d25af740 1 Processor -- start 2024-08-30T09:01:24.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.726+0000 7f36d25af740 1 -- start start 2024-08-30T09:01:24.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.726+0000 7f36d25af740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562786f11070 0x5627872b6310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:01:24.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.726+0000 7f36d25af740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562786ee81b0 0x5627872b6850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:01:24.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.726+0000 7f36d25af740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562786ef0390 0x5627872b6d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:01:24.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.726+0000 7f36d25af740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x56278722d680 con 0x562786f11070 2024-08-30T09:01:24.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.726+0000 7f36d25af740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x56278721cc10 con 0x562786ee81b0 2024-08-30T09:01:24.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.726+0000 7f36d25af740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x562787220bc0 con 0x562786ef0390 2024-08-30T09:01:24.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.726+0000 7f36ca5d7640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562786f11070 0x5627872b6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:01:24.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.727+0000 7f36c9dd6640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562786ee81b0 0x5627872b6850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:01:24.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.727+0000 7f36ca5d7640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562786f11070 0x5627872b6310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:55520/0 (socket says 172.21.15.42:55520) 2024-08-30T09:01:24.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.727+0000 7f36ca5d7640 1 -- 172.21.15.42:0/2896818001 learned_addr learned my addr 172.21.15.42:0/2896818001 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:01:24.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.727+0000 7f36baffd640 1 -- 172.21.15.42:0/1032315355 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7f36b4010430 con 0x7f36c41050a0 2024-08-30T09:01:24.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.727+0000 7f36c9dd6640 1 -- 172.21.15.42:0/2896818001 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562786ef0390 msgr2=0x5627872b6d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:01:24.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.727+0000 7f36c9dd6640 1 --2- 172.21.15.42:0/2896818001 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562786ef0390 0x5627872b6d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:24.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.727+0000 7f36c9dd6640 1 -- 172.21.15.42:0/2896818001 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562786f11070 msgr2=0x5627872b6310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:01:24.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.727+0000 7f36c9dd6640 1 --2- 172.21.15.42:0/2896818001 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562786f11070 0x5627872b6310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:24.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.727+0000 7f36c9dd6640 1 -- 172.21.15.42:0/2896818001 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562786f7c650 con 0x562786ee81b0 2024-08-30T09:01:24.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.727+0000 7f36ca5d7640 1 --2- 172.21.15.42:0/2896818001 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562786f11070 0x5627872b6310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-30T09:01:24.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.728+0000 7f36c9dd6640 1 --2- 172.21.15.42:0/2896818001 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562786ee81b0 0x5627872b6850 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f36c0006010 tx=0x7f36c0032bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=931645a7ea177cfb server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:01:24.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.728+0000 7f36baffd640 1 --2- 172.21.15.42:0/1032315355 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f36b00b4bb0 0x7f36b00b70c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:01:24.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.728+0000 7f36baffd640 1 -- 172.21.15.42:0/1032315355 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f36b40e57e0 con 0x7f36c41050a0 2024-08-30T09:01:24.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.728+0000 7f36affff640 1 -- 172.21.15.42:0/2896818001 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36c0040070 con 0x562786ee81b0 2024-08-30T09:01:24.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.729+0000 7f36affff640 1 -- 172.21.15.42:0/2896818001 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f36c0004030 con 0x562786ee81b0 2024-08-30T09:01:24.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.729+0000 7f36affff640 1 -- 172.21.15.42:0/2896818001 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36c0035040 con 0x562786ee81b0 2024-08-30T09:01:24.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.730+0000 7f36ca5d7640 1 --2- 172.21.15.42:0/1032315355 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f36b00b4bb0 0x7f36b00b70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:01:24.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.730+0000 7f36d25af740 1 -- 172.21.15.42:0/2896818001 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562786ee81b0 msgr2=0x5627872b6850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:01:24.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.730+0000 7f36d25af740 1 --2- 172.21.15.42:0/2896818001 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562786ee81b0 0x5627872b6850 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f36c0006010 tx=0x7f36c0032bd0 comp rx=0 tx=0).stop 2024-08-30T09:01:24.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.730+0000 7f36d25af740 1 -- 172.21.15.42:0/2896818001 shutdown_connections 2024-08-30T09:01:24.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.730+0000 7f36d25af740 1 --2- 172.21.15.42:0/2896818001 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562786ef0390 0x5627872b6d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:24.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.730+0000 7f36d25af740 1 --2- 172.21.15.42:0/2896818001 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562786ee81b0 0x5627872b6850 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:24.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.730+0000 7f36d25af740 1 --2- 172.21.15.42:0/2896818001 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562786f11070 0x5627872b6310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:24.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.730+0000 7f36d25af740 1 -- 172.21.15.42:0/2896818001 >> 172.21.15.42:0/2896818001 conn(0x562786f05460 msgr2=0x5627872b4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:01:24.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.731+0000 7f36ca5d7640 1 --2- 172.21.15.42:0/1032315355 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f36b00b4bb0 0x7f36b00b70c0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f36bc009370 tx=0x7f36bc009800 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:01:24.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.731+0000 7f36d25af740 1 -- 172.21.15.42:0/2896818001 shutdown_connections 2024-08-30T09:01:24.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.731+0000 7f36d25af740 1 -- 172.21.15.42:0/2896818001 wait complete. 2024-08-30T09:01:24.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.731+0000 7f36d25af740 1 -- start start 2024-08-30T09:01:24.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.731+0000 7f36d25af740 1 Processor -- start 2024-08-30T09:01:24.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.732+0000 7f36d25af740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562786ee81b0 0x5627872bdb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:01:24.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.732+0000 7f36d25af740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562786ef0390 0x5627872be0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:01:24.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.732+0000 7f36d25af740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562786f11070 0x5627872d6530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:01:24.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.732+0000 7f36c9dd6640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562786ee81b0 0x5627872bdb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:01:24.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.733+0000 7f36c9dd6640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562786ee81b0 0x5627872bdb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:49794/0 (socket says 172.21.15.42:49794) 2024-08-30T09:01:24.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.733+0000 7f36c9dd6640 1 -- 172.21.15.42:0/923254475 learned_addr learned my addr 172.21.15.42:0/923254475 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:01:24.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.733+0000 7f36d25af740 1 -- 172.21.15.42:0/923254475 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x56278707f700 con 0x562786ef0390 2024-08-30T09:01:24.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.733+0000 7f36d25af740 1 -- 172.21.15.42:0/923254475 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5627871a3750 con 0x562786ee81b0 2024-08-30T09:01:24.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.733+0000 7f36d25af740 1 -- 172.21.15.42:0/923254475 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5627872cd280 con 0x562786f11070 2024-08-30T09:01:24.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.733+0000 7f36ca5d7640 1 --2- 172.21.15.42:0/923254475 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562786f11070 0x5627872d6530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:01:24.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.733+0000 7f36d25af740 10 client.0 _ll_register_callbacks cb 0x5627871d2e10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:01:24.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.733+0000 7f36ca5d7640 1 -- 172.21.15.42:0/923254475 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562786ee81b0 msgr2=0x5627872bdb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:01:24.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.733+0000 7f36ca5d7640 1 --2- 172.21.15.42:0/923254475 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562786ee81b0 0x5627872bdb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:24.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.733+0000 7f36ca5d7640 1 -- 172.21.15.42:0/923254475 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562786ef0390 msgr2=0x5627872be0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:01:24.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.733+0000 7f36ca5d7640 1 --2- 172.21.15.42:0/923254475 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562786ef0390 0x5627872be0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:24.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.733+0000 7f36ca5d7640 1 -- 172.21.15.42:0/923254475 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5627871dc900 con 0x562786f11070 2024-08-30T09:01:24.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.734+0000 7f36ca5d7640 1 --2- 172.21.15.42:0/923254475 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562786f11070 0x5627872d6530 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f36bc012e40 tx=0x7f36bc004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f58f42734a6e1de8 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:01:24.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.734+0000 7f36ad7fa640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-08-30T09:01:24.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.734+0000 7f36c9dd6640 1 --2- 172.21.15.42:0/923254475 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562786ee81b0 0x5627872bdb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-30T09:01:24.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.735+0000 7f36ad7fa640 1 -- 172.21.15.42:0/923254475 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36bc01ed40 con 0x562786f11070 2024-08-30T09:01:24.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.735+0000 7f36ad7fa640 1 -- 172.21.15.42:0/923254475 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f36bc003970 con 0x562786f11070 2024-08-30T09:01:24.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.735+0000 7f36d25af740 1 -- 172.21.15.42:0/923254475 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f36c0004030 con 0x562786f11070 2024-08-30T09:01:24.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.735+0000 7f36ad7fa640 1 -- 172.21.15.42:0/923254475 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f36bc003b70 con 0x562786f11070 2024-08-30T09:01:24.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.736+0000 7f36d25af740 10 client.25252 fetch_fsmap learned FSMap version 29 2024-08-30T09:01:24.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.736+0000 7f36d25af740 1 -- 172.21.15.42:0/923254475 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5627872d9130 con 0x562786f11070 2024-08-30T09:01:24.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.736+0000 7f36ad7fa640 1 -- 172.21.15.42:0/923254475 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f36c0004030 con 0x562786f11070 2024-08-30T09:01:24.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.736+0000 7f36ad7fa640 1 -- 172.21.15.42:0/923254475 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f36bc031490 con 0x562786f11070 2024-08-30T09:01:24.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.736+0000 7f36d25af740 10 client.25252 fetch_fsmap finished waiting for FSMap version 29 2024-08-30T09:01:24.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.736+0000 7f36d25af740 10 client.25252 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-30T09:01:24.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.736+0000 7f36d25af740 20 client.25252 populate_metadata read hostname 'smithi042' 2024-08-30T09:01:24.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.736+0000 7f36d25af740 1 --2- 172.21.15.42:0/923254475 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x562786ecf9f0 0x5627872dcaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:01:24.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.737+0000 7f36d25af740 4 client.25252 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-30T09:01:24.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.737+0000 7f36d25af740 1 -- 172.21.15.42:0/923254475 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56278722f5d0 con 0x562786ecf9f0 2024-08-30T09:01:24.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.737+0000 7f36c9dd6640 1 --2- 172.21.15.42:0/923254475 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x562786ecf9f0 0x5627872dcaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:01:24.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.737+0000 7f36ad7fa640 10 client.25252 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:01:24.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.737+0000 7f36c9dd6640 1 --2- 172.21.15.42:0/923254475 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x562786ecf9f0 0x5627872dcaf0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=316c8d1655940200 server_cookie=bf51b1729b6df5ae in_seq=0 out_seq=0 2024-08-30T09:01:24.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.738+0000 7f36ad7fa640 1 -- 172.21.15.42:0/923254475 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56278722f5d0 con 0x562786ecf9f0 2024-08-30T09:01:24.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.738+0000 7f36ad7fa640 10 client.25252 handle_command_reply: tid=0 2024-08-30T09:01:24.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.738+0000 7f36d25af740 1 client.25252 shutdown 2024-08-30T09:01:24.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.739+0000 7f36d25af740 1 -- 172.21.15.42:0/923254475 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562786f11070 msgr2=0x5627872d6530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:01:24.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.739+0000 7f36d25af740 1 --2- 172.21.15.42:0/923254475 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562786f11070 0x5627872d6530 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f36bc012e40 tx=0x7f36bc004a40 comp rx=0 tx=0).stop 2024-08-30T09:01:24.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.739+0000 7f36d25af740 1 -- 172.21.15.42:0/923254475 shutdown_connections 2024-08-30T09:01:24.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.739+0000 7f36d25af740 1 --2- 172.21.15.42:0/923254475 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x562786ecf9f0 0x5627872dcaf0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:24.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.739+0000 7f36d25af740 1 --2- 172.21.15.42:0/923254475 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562786f11070 0x5627872d6530 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:24.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.739+0000 7f36d25af740 1 --2- 172.21.15.42:0/923254475 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562786ef0390 0x5627872be0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:24.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.739+0000 7f36d25af740 1 --2- 172.21.15.42:0/923254475 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562786ee81b0 0x5627872bdb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:24.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.739+0000 7f36d25af740 1 -- 172.21.15.42:0/923254475 >> 172.21.15.42:0/923254475 conn(0x562786f05460 msgr2=0x5627872b27c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:01:24.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.739+0000 7f36ad7fa640 0 client.25252 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:01:24.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.739+0000 7f36d25af740 1 -- 172.21.15.42:0/923254475 shutdown_connections 2024-08-30T09:01:24.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.739+0000 7f36d25af740 1 -- 172.21.15.42:0/923254475 wait complete. 2024-08-30T09:01:24.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.739+0000 7f36d25af740 20 client.25252 trim_cache size 0 max 16384 2024-08-30T09:01:24.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.762+0000 7f36d25af740 1 Processor -- start 2024-08-30T09:01:24.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.762+0000 7f36d25af740 1 -- start start 2024-08-30T09:01:24.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.763+0000 7f36d25af740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562786ee81b0 0x5627872aedd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:01:24.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.763+0000 7f36d25af740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562786ef0390 0x5627872d6470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:01:24.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.763+0000 7f36d25af740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562786f11070 0x5627872d69b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:01:24.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.763+0000 7f36d25af740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f36c41b1190 con 0x562786ef0390 2024-08-30T09:01:24.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.763+0000 7f36ca5d7640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562786f11070 0x5627872d69b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:01:24.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.763+0000 7f36ca5d7640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562786f11070 0x5627872d69b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:36964/0 (socket says 172.21.15.42:36964) 2024-08-30T09:01:24.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.763+0000 7f36ca5d7640 1 -- 172.21.15.42:0/536727788 learned_addr learned my addr 172.21.15.42:0/536727788 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:01:24.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.763+0000 7f36c9dd6640 1 --2- 172.21.15.42:0/536727788 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562786ee81b0 0x5627872aedd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:01:24.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.764+0000 7f36d25af740 1 -- 172.21.15.42:0/536727788 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f36c41aea60 con 0x562786ee81b0 2024-08-30T09:01:24.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.764+0000 7f36d25af740 1 -- 172.21.15.42:0/536727788 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f36c41ac8b0 con 0x562786f11070 2024-08-30T09:01:24.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.764+0000 7f36c95d5640 1 --2- 172.21.15.42:0/536727788 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562786ef0390 0x5627872d6470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:01:24.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.764+0000 7f36c9dd6640 1 -- 172.21.15.42:0/536727788 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562786f11070 msgr2=0x5627872d69b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:01:24.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.764+0000 7f36c9dd6640 1 --2- 172.21.15.42:0/536727788 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562786f11070 0x5627872d69b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:24.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.764+0000 7f36c9dd6640 1 -- 172.21.15.42:0/536727788 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562786ef0390 msgr2=0x5627872d6470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:01:24.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.764+0000 7f36c9dd6640 1 --2- 172.21.15.42:0/536727788 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562786ef0390 0x5627872d6470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:24.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.764+0000 7f36c9dd6640 1 -- 172.21.15.42:0/536727788 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5627872be440 con 0x562786ee81b0 2024-08-30T09:01:24.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.765+0000 7f36c9dd6640 1 --2- 172.21.15.42:0/536727788 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562786ee81b0 0x5627872aedd0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x5627872dd230 tx=0x7f36c00051c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3853a1c893f71a88 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:01:24.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.765+0000 7f36af7fe640 1 -- 172.21.15.42:0/536727788 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36c0035040 con 0x562786ee81b0 2024-08-30T09:01:24.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.765+0000 7f36af7fe640 1 -- 172.21.15.42:0/536727788 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f36c00314a0 con 0x562786ee81b0 2024-08-30T09:01:24.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.765+0000 7f36af7fe640 1 -- 172.21.15.42:0/536727788 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36c0040040 con 0x562786ee81b0 2024-08-30T09:01:24.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.767+0000 7f36d25af740 1 -- 172.21.15.42:0/536727788 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562786ee81b0 msgr2=0x5627872aedd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:01:24.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.767+0000 7f36d25af740 1 --2- 172.21.15.42:0/536727788 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562786ee81b0 0x5627872aedd0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x5627872dd230 tx=0x7f36c00051c0 comp rx=0 tx=0).stop 2024-08-30T09:01:24.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.767+0000 7f36d25af740 1 -- 172.21.15.42:0/536727788 shutdown_connections 2024-08-30T09:01:24.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.767+0000 7f36d25af740 1 --2- 172.21.15.42:0/536727788 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562786f11070 0x5627872d69b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:24.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.767+0000 7f36d25af740 1 --2- 172.21.15.42:0/536727788 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562786ef0390 0x5627872d6470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:24.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.767+0000 7f36d25af740 1 --2- 172.21.15.42:0/536727788 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562786ee81b0 0x5627872aedd0 secure :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0x5627872dd230 tx=0x7f36c00051c0 comp rx=0 tx=0).stop 2024-08-30T09:01:24.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.767+0000 7f36d25af740 1 -- 172.21.15.42:0/536727788 >> 172.21.15.42:0/536727788 conn(0x562786f05460 msgr2=0x5627872cbce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:01:24.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.767+0000 7f36d25af740 1 -- 172.21.15.42:0/536727788 shutdown_connections 2024-08-30T09:01:24.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.767+0000 7f36d25af740 1 -- 172.21.15.42:0/536727788 wait complete. 2024-08-30T09:01:24.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.768+0000 7f36d25af740 1 -- start start 2024-08-30T09:01:24.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.768+0000 7f36d25af740 1 Processor -- start 2024-08-30T09:01:24.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.768+0000 7f36d25af740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562786ef0390 0x5627872d2dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:01:24.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.768+0000 7f36d25af740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562786f11070 0x5627872ce280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:01:24.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.768+0000 7f36d25af740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562786ea4da0 0x5627872ce8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:01:24.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.768+0000 7f36d25af740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f36c41acf20 con 0x562786ea4da0 2024-08-30T09:01:24.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.768+0000 7f36d25af740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f36c41ae390 con 0x562786ef0390 2024-08-30T09:01:24.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.768+0000 7f36d25af740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f36c41ad5f0 con 0x562786f11070 2024-08-30T09:01:24.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.768+0000 7f36d25af740 10 client.0 _ll_register_callbacks cb 0x7f36bc0092a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:01:24.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.769+0000 7f36ca5d7640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562786ea4da0 0x5627872ce8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:01:24.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.769+0000 7f36ca5d7640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562786ea4da0 0x5627872ce8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:55556/0 (socket says 172.21.15.42:55556) 2024-08-30T09:01:24.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.769+0000 7f36ca5d7640 1 -- 172.21.15.42:0/1394008000 learned_addr learned my addr 172.21.15.42:0/1394008000 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:01:24.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.769+0000 7f36c95d5640 1 --2- 172.21.15.42:0/1394008000 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562786f11070 0x5627872ce280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:01:24.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.769+0000 7f36c9dd6640 1 --2- 172.21.15.42:0/1394008000 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562786ef0390 0x5627872d2dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:01:24.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.769+0000 7f36ca5d7640 1 -- 172.21.15.42:0/1394008000 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562786f11070 msgr2=0x5627872ce280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:01:24.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.769+0000 7f36ca5d7640 1 --2- 172.21.15.42:0/1394008000 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562786f11070 0x5627872ce280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:24.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.769+0000 7f36ca5d7640 1 -- 172.21.15.42:0/1394008000 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562786ef0390 msgr2=0x5627872d2dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:01:24.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.769+0000 7f36ca5d7640 1 --2- 172.21.15.42:0/1394008000 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562786ef0390 0x5627872d2dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:24.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.769+0000 7f36ca5d7640 1 -- 172.21.15.42:0/1394008000 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5627872cf060 con 0x562786ea4da0 2024-08-30T09:01:24.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.769+0000 7f36c95d5640 1 --2- 172.21.15.42:0/1394008000 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562786f11070 0x5627872ce280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-30T09:01:24.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.769+0000 7f36c9dd6640 1 --2- 172.21.15.42:0/1394008000 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562786ef0390 0x5627872d2dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-30T09:01:24.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.769+0000 7f36ca5d7640 1 --2- 172.21.15.42:0/1394008000 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562786ea4da0 0x5627872ce8d0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f36bc017040 tx=0x7f36bc01edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a1980a51835435c server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:01:24.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.769+0000 7f36ad7fa640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-08-30T09:01:24.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.771+0000 7f36ad7fa640 1 -- 172.21.15.42:0/1394008000 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36bc031aa0 con 0x562786ea4da0 2024-08-30T09:01:24.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.771+0000 7f36ad7fa640 1 -- 172.21.15.42:0/1394008000 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f36bc031c40 con 0x562786ea4da0 2024-08-30T09:01:24.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.771+0000 7f36d25af740 1 -- 172.21.15.42:0/1394008000 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f36c00314a0 con 0x562786ea4da0 2024-08-30T09:01:24.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.771+0000 7f36ad7fa640 1 -- 172.21.15.42:0/1394008000 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f36bc031e40 con 0x562786ea4da0 2024-08-30T09:01:24.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.772+0000 7f36d25af740 10 client.15621 fetch_fsmap learned FSMap version 29 2024-08-30T09:01:24.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.772+0000 7f36d25af740 1 -- 172.21.15.42:0/1394008000 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x562787347540 con 0x562786ea4da0 2024-08-30T09:01:24.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.772+0000 7f36ad7fa640 1 -- 172.21.15.42:0/1394008000 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f36c00314a0 con 0x562786ea4da0 2024-08-30T09:01:24.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.772+0000 7f36ad7fa640 1 -- 172.21.15.42:0/1394008000 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f36bc03a4f0 con 0x562786ea4da0 2024-08-30T09:01:24.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.772+0000 7f36d25af740 10 client.15621 fetch_fsmap finished waiting for FSMap version 29 2024-08-30T09:01:24.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.772+0000 7f36d25af740 10 client.15621 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-30T09:01:24.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.772+0000 7f36d25af740 20 client.15621 populate_metadata read hostname 'smithi042' 2024-08-30T09:01:24.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.772+0000 7f36d25af740 1 --2- 172.21.15.42:0/1394008000 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x562787347d40 0x56278734a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:01:24.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.772+0000 7f36d25af740 4 client.15621 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-30T09:01:24.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.772+0000 7f36d25af740 1 -- 172.21.15.42:0/1394008000 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56278722b070 con 0x562787347d40 2024-08-30T09:01:24.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.773+0000 7f36c9dd6640 1 --2- 172.21.15.42:0/1394008000 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x562787347d40 0x56278734a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:01:24.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.773+0000 7f36ad7fa640 10 client.15621 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:01:24.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.773+0000 7f36c9dd6640 1 --2- 172.21.15.42:0/1394008000 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x562787347d40 0x56278734a160 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6427162273a1bc73 server_cookie=1c1c301e8cc5e2cd in_seq=0 out_seq=0 2024-08-30T09:01:24.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.774+0000 7f36ad7fa640 1 -- 172.21.15.42:0/1394008000 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56278722b070 con 0x562787347d40 2024-08-30T09:01:24.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.774+0000 7f36ad7fa640 10 client.15621 handle_command_reply: tid=0 2024-08-30T09:01:24.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.774+0000 7f36d25af740 1 client.15621 shutdown 2024-08-30T09:01:24.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.774+0000 7f36d25af740 1 -- 172.21.15.42:0/1394008000 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562786ea4da0 msgr2=0x5627872ce8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:01:24.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.774+0000 7f36d25af740 1 --2- 172.21.15.42:0/1394008000 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562786ea4da0 0x5627872ce8d0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f36bc017040 tx=0x7f36bc01edf0 comp rx=0 tx=0).stop 2024-08-30T09:01:24.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.774+0000 7f36d25af740 1 -- 172.21.15.42:0/1394008000 shutdown_connections 2024-08-30T09:01:24.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.774+0000 7f36d25af740 1 --2- 172.21.15.42:0/1394008000 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x562787347d40 0x56278734a160 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:24.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.774+0000 7f36d25af740 1 --2- 172.21.15.42:0/1394008000 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562786f11070 0x5627872ce280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:24.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.774+0000 7f36d25af740 1 --2- 172.21.15.42:0/1394008000 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562786ea4da0 0x5627872ce8d0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:24.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.774+0000 7f36d25af740 1 --2- 172.21.15.42:0/1394008000 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562786ef0390 0x5627872d2dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:24.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.774+0000 7f36d25af740 1 -- 172.21.15.42:0/1394008000 >> 172.21.15.42:0/1394008000 conn(0x562786f05460 msgr2=0x5627872b7340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:01:24.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.774+0000 7f36ad7fa640 0 client.15621 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:01:24.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.774+0000 7f36d25af740 1 -- 172.21.15.42:0/1394008000 shutdown_connections 2024-08-30T09:01:24.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.774+0000 7f36d25af740 1 -- 172.21.15.42:0/1394008000 wait complete. 2024-08-30T09:01:24.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.774+0000 7f36d25af740 20 client.15621 trim_cache size 0 max 16384 2024-08-30T09:01:24.779 INFO:teuthology.orchestra.run.smithi042.stdout:[] 2024-08-30T09:01:24.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.776+0000 7f36cc03a640 1 -- 172.21.15.42:0/1032315355 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f36b00b4bb0 msgr2=0x7f36b00b70c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:01:24.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.776+0000 7f36cc03a640 1 --2- 172.21.15.42:0/1032315355 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f36b00b4bb0 0x7f36b00b70c0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f36bc009370 tx=0x7f36bc009800 comp rx=0 tx=0).stop 2024-08-30T09:01:24.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.776+0000 7f36cc03a640 1 -- 172.21.15.42:0/1032315355 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f36c41050a0 msgr2=0x7f36c419a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:01:24.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.776+0000 7f36cc03a640 1 --2- 172.21.15.42:0/1032315355 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f36c41050a0 0x7f36c419a8c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f36b400e9f0 tx=0x7f36b400eeb0 comp rx=0 tx=0).stop 2024-08-30T09:01:24.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.776+0000 7f36cc03a640 1 -- 172.21.15.42:0/1032315355 shutdown_connections 2024-08-30T09:01:24.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.776+0000 7f36cc03a640 1 --2- 172.21.15.42:0/1032315355 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f36b00b4bb0 0x7f36b00b70c0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:24.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.776+0000 7f36cc03a640 1 --2- 172.21.15.42:0/1032315355 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f36c41050a0 0x7f36c419a8c0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:24.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.776+0000 7f36cc03a640 1 --2- 172.21.15.42:0/1032315355 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f36c410dbb0 0x7f36c419fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:24.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.776+0000 7f36cc03a640 1 --2- 172.21.15.42:0/1032315355 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f36c4104750 0x7f36c419a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:24.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.776+0000 7f36cc03a640 1 -- 172.21.15.42:0/1032315355 >> 172.21.15.42:0/1032315355 conn(0x7f36c40794d0 msgr2=0x7f36c40ff530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:01:24.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.776+0000 7f36cc03a640 1 -- 172.21.15.42:0/1032315355 shutdown_connections 2024-08-30T09:01:24.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:24.776+0000 7f36cc03a640 1 -- 172.21.15.42:0/1032315355 wait complete. 2024-08-30T09:01:24.793 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:01:24 smithi042 ceph-mon[18652]: pgmap v392: 129 pgs: 129 active+clean; 976 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 9.0 MiB/s wr, 799 op/s 2024-08-30T09:01:24.826 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:01:24 smithi167 ceph-mon[22063]: pgmap v392: 129 pgs: 129 active+clean; 976 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 9.0 MiB/s wr, 799 op/s 2024-08-30T09:01:24.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:01:24 smithi190 ceph-mon[22001]: pgmap v392: 129 pgs: 129 active+clean; 976 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 9.0 MiB/s wr, 799 op/s 2024-08-30T09:01:25.953 INFO:teuthology.orchestra.run.smithi042.stderr:progress: 40.0 s, 1921.4 tps, lat 46.642 ms stddev 65.607 2024-08-30T09:01:26.241 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.238+0000 7f884867f640 1 -- 172.21.15.42:0/759948159 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f884010b9a0 msgr2=0x7f88401123c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:01:26.242 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.238+0000 7f884867f640 1 --2- 172.21.15.42:0/759948159 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f884010b9a0 0x7f88401123c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f8838009ca0 tx=0x7f883802a8f0 comp rx=0 tx=0).stop 2024-08-30T09:01:26.242 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.238+0000 7f884867f640 1 -- 172.21.15.42:0/759948159 shutdown_connections 2024-08-30T09:01:26.242 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.238+0000 7f884867f640 1 --2- 172.21.15.42:0/759948159 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f884010b9a0 0x7f88401123c0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:26.242 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.238+0000 7f884867f640 1 --2- 172.21.15.42:0/759948159 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f884010ad90 0x7f884010b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:26.242 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.238+0000 7f884867f640 1 --2- 172.21.15.42:0/759948159 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f884010a340 0x7f884010a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:26.242 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.238+0000 7f884867f640 1 -- 172.21.15.42:0/759948159 >> 172.21.15.42:0/759948159 conn(0x7f8840070a10 msgr2=0x7f8840072ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:01:26.244 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.241+0000 7f884867f640 1 -- 172.21.15.42:0/759948159 shutdown_connections 2024-08-30T09:01:26.250 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.244+0000 7f884867f640 1 -- 172.21.15.42:0/759948159 wait complete. 2024-08-30T09:01:26.250 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.245+0000 7f884867f640 1 Processor -- start 2024-08-30T09:01:26.250 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.246+0000 7f884867f640 1 -- start start 2024-08-30T09:01:26.250 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.246+0000 7f884867f640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f884010a340 0x7f884011b420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:01:26.250 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.246+0000 7f884867f640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f884010ad90 0x7f884011ba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:01:26.250 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.246+0000 7f884867f640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f884010b9a0 0x7f884011c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:01:26.250 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.246+0000 7f884867f640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f8840114d30 con 0x7f884010a340 2024-08-30T09:01:26.250 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.246+0000 7f884867f640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f8840114bb0 con 0x7f884010b9a0 2024-08-30T09:01:26.251 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.246+0000 7f884867f640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f8840114eb0 con 0x7f884010ad90 2024-08-30T09:01:26.251 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.246+0000 7f884641b640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f884010a340 0x7f884011b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:01:26.251 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.246+0000 7f8846c1c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f884010b9a0 0x7f884011c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:01:26.251 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.246+0000 7f8845c1a640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f884010ad90 0x7f884011ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:01:26.251 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.246+0000 7f8845c1a640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f884010ad90 0x7f884011ba90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:36992/0 (socket says 172.21.15.42:36992) 2024-08-30T09:01:26.251 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.246+0000 7f8846c1c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f884010b9a0 0x7f884011c640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:49828/0 (socket says 172.21.15.42:49828) 2024-08-30T09:01:26.251 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.246+0000 7f8845c1a640 1 -- 172.21.15.42:0/474720154 learned_addr learned my addr 172.21.15.42:0/474720154 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:01:26.252 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.247+0000 7f8845c1a640 1 -- 172.21.15.42:0/474720154 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f884010b9a0 msgr2=0x7f884011c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:01:26.252 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.247+0000 7f8845c1a640 1 --2- 172.21.15.42:0/474720154 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f884010b9a0 0x7f884011c640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:26.252 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.247+0000 7f8845c1a640 1 -- 172.21.15.42:0/474720154 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f884010a340 msgr2=0x7f884011b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:01:26.252 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.247+0000 7f8845c1a640 1 --2- 172.21.15.42:0/474720154 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f884010a340 0x7f884011b420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:26.253 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.247+0000 7f8845c1a640 1 -- 172.21.15.42:0/474720154 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f884011cdd0 con 0x7f884010ad90 2024-08-30T09:01:26.253 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.248+0000 7f8845c1a640 1 --2- 172.21.15.42:0/474720154 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f884010ad90 0x7f884011ba90 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f883400cab0 tx=0x7f883400cf70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:01:26.253 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.249+0000 7f883f7fe640 1 -- 172.21.15.42:0/474720154 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8834007d30 con 0x7f884010ad90 2024-08-30T09:01:26.253 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.249+0000 7f883f7fe640 1 -- 172.21.15.42:0/474720154 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f88340035f0 con 0x7f884010ad90 2024-08-30T09:01:26.253 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.249+0000 7f883f7fe640 1 -- 172.21.15.42:0/474720154 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f883400f690 con 0x7f884010ad90 2024-08-30T09:01:26.253 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.249+0000 7f884867f640 1 -- 172.21.15.42:0/474720154 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f884011d0c0 con 0x7f884010ad90 2024-08-30T09:01:26.253 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.249+0000 7f884867f640 1 -- 172.21.15.42:0/474720154 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f884011f6c0 con 0x7f884010ad90 2024-08-30T09:01:26.254 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.252+0000 7f883f7fe640 1 -- 172.21.15.42:0/474720154 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7f8834004000 con 0x7f884010ad90 2024-08-30T09:01:26.255 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.252+0000 7f883f7fe640 1 --2- 172.21.15.42:0/474720154 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f88280b4b90 0x7f88280b70a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:01:26.255 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.253+0000 7f883f7fe640 1 -- 172.21.15.42:0/474720154 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f88340e5290 con 0x7f884010ad90 2024-08-30T09:01:26.255 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.253+0000 7f884641b640 1 --2- 172.21.15.42:0/474720154 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f88280b4b90 0x7f88280b70a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:01:26.257 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.253+0000 7f884f5e1740 1 Processor -- start 2024-08-30T09:01:26.257 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.253+0000 7f884f5e1740 1 -- start start 2024-08-30T09:01:26.257 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.253+0000 7f884f5e1740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b63fbcc1b0 0x55b63ff71f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:01:26.257 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.253+0000 7f884f5e1740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b63fba32f0 0x55b63ff725d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:01:26.257 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.253+0000 7f884f5e1740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b63fbab4d0 0x55b63ff73120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:01:26.257 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.253+0000 7f884f5e1740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55b63fee5000 con 0x55b63fbab4d0 2024-08-30T09:01:26.258 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.253+0000 7f884f5e1740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55b63fed4590 con 0x55b63fbcc1b0 2024-08-30T09:01:26.258 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.253+0000 7f884641b640 1 --2- 172.21.15.42:0/474720154 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f88280b4b90 0x7f88280b70a0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f8830006fd0 tx=0x7f88300090e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:01:26.258 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.253+0000 7f884f5e1740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55b63fed8540 con 0x55b63fba32f0 2024-08-30T09:01:26.258 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.254+0000 7f884641b640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b63fbcc1b0 0x55b63ff71f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:01:26.258 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.254+0000 7f8846c1c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b63fbab4d0 0x55b63ff73120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:01:26.258 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.254+0000 7f884641b640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b63fbcc1b0 0x55b63ff71f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:49836/0 (socket says 172.21.15.42:49836) 2024-08-30T09:01:26.258 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.254+0000 7f884641b640 1 -- 172.21.15.42:0/3571306234 learned_addr learned my addr 172.21.15.42:0/3571306234 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:01:26.258 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.254+0000 7f8846c1c640 1 --2- 172.21.15.42:0/3571306234 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b63fbab4d0 0x55b63ff73120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:55582/0 (socket says 172.21.15.42:55582) 2024-08-30T09:01:26.258 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.254+0000 7f8845c1a640 1 --2- 172.21.15.42:0/3571306234 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b63fba32f0 0x55b63ff725d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:01:26.258 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.255+0000 7f8846c1c640 1 -- 172.21.15.42:0/3571306234 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b63fba32f0 msgr2=0x55b63ff725d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:01:26.259 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.255+0000 7f8846c1c640 1 --2- 172.21.15.42:0/3571306234 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b63fba32f0 0x55b63ff725d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:26.259 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.255+0000 7f8846c1c640 1 -- 172.21.15.42:0/3571306234 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b63fbcc1b0 msgr2=0x55b63ff71f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:01:26.259 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.255+0000 7f8846c1c640 1 --2- 172.21.15.42:0/3571306234 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b63fbcc1b0 0x55b63ff71f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:26.259 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.255+0000 7f8846c1c640 1 -- 172.21.15.42:0/3571306234 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b63fc37790 con 0x55b63fbab4d0 2024-08-30T09:01:26.259 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.257+0000 7f8846c1c640 1 --2- 172.21.15.42:0/3571306234 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b63fbab4d0 0x55b63ff73120 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f883802aee0 tx=0x7f8838009a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c355013df6386ec6 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:01:26.260 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.257+0000 7f883cbfe640 1 -- 172.21.15.42:0/3571306234 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8838031480 con 0x55b63fbab4d0 2024-08-30T09:01:26.260 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.257+0000 7f883cbfe640 1 -- 172.21.15.42:0/3571306234 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8838036030 con 0x55b63fbab4d0 2024-08-30T09:01:26.260 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.257+0000 7f883cbfe640 1 -- 172.21.15.42:0/3571306234 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8838004360 con 0x55b63fbab4d0 2024-08-30T09:01:26.262 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.258+0000 7f884f5e1740 1 -- 172.21.15.42:0/3571306234 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b63fbab4d0 msgr2=0x55b63ff73120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:01:26.262 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.258+0000 7f884f5e1740 1 --2- 172.21.15.42:0/3571306234 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b63fbab4d0 0x55b63ff73120 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f883802aee0 tx=0x7f8838009a10 comp rx=0 tx=0).stop 2024-08-30T09:01:26.262 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.258+0000 7f884f5e1740 1 -- 172.21.15.42:0/3571306234 shutdown_connections 2024-08-30T09:01:26.262 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.258+0000 7f884f5e1740 1 --2- 172.21.15.42:0/3571306234 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b63fba32f0 0x55b63ff725d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:26.262 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.258+0000 7f884f5e1740 1 --2- 172.21.15.42:0/3571306234 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b63fbab4d0 0x55b63ff73120 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:26.262 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.258+0000 7f884f5e1740 1 --2- 172.21.15.42:0/3571306234 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b63fbcc1b0 0x55b63ff71f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:26.262 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.258+0000 7f884f5e1740 1 -- 172.21.15.42:0/3571306234 >> 172.21.15.42:0/3571306234 conn(0x55b63fbc0300 msgr2=0x55b63ff6d9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:01:26.262 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.259+0000 7f884f5e1740 1 -- 172.21.15.42:0/3571306234 shutdown_connections 2024-08-30T09:01:26.262 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.259+0000 7f884f5e1740 1 -- 172.21.15.42:0/3571306234 wait complete. 2024-08-30T09:01:26.262 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.259+0000 7f884f5e1740 1 -- start start 2024-08-30T09:01:26.262 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.259+0000 7f884f5e1740 1 Processor -- start 2024-08-30T09:01:26.264 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.259+0000 7f884f5e1740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b63fba32f0 0x55b63ff7b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:01:26.264 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.259+0000 7f884f5e1740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b63fbab4d0 0x55b63ff7c050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:01:26.265 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.259+0000 7f884f5e1740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b63fbcc1b0 0x55b63ff7cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:01:26.265 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.259+0000 7f884f5e1740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55b63fe0dab0 con 0x55b63fbab4d0 2024-08-30T09:01:26.265 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.259+0000 7f884f5e1740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55b63fe5b680 con 0x55b63fbcc1b0 2024-08-30T09:01:26.265 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.259+0000 7f884f5e1740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55b63ff73be0 con 0x55b63fba32f0 2024-08-30T09:01:26.265 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.260+0000 7f884f5e1740 10 client.0 _ll_register_callbacks cb 0x55b63fbe3a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:01:26.265 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.260+0000 7f8845c1a640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b63fbab4d0 0x55b63ff7c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:01:26.265 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.260+0000 7f884641b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b63fba32f0 0x55b63ff7b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:01:26.265 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.260+0000 7f884641b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b63fba32f0 0x55b63ff7b990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:37008/0 (socket says 172.21.15.42:37008) 2024-08-30T09:01:26.265 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.260+0000 7f8845c1a640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b63fbab4d0 0x55b63ff7c050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:55586/0 (socket says 172.21.15.42:55586) 2024-08-30T09:01:26.265 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.260+0000 7f884641b640 1 -- 172.21.15.42:0/568269048 learned_addr learned my addr 172.21.15.42:0/568269048 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:01:26.265 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.260+0000 7f8846c1c640 1 --2- 172.21.15.42:0/568269048 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b63fbcc1b0 0x55b63ff7cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:01:26.265 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.260+0000 7f884641b640 1 -- 172.21.15.42:0/568269048 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b63fbcc1b0 msgr2=0x55b63ff7cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:01:26.266 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.260+0000 7f884641b640 1 --2- 172.21.15.42:0/568269048 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b63fbcc1b0 0x55b63ff7cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:26.266 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.260+0000 7f884641b640 1 -- 172.21.15.42:0/568269048 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b63fbab4d0 msgr2=0x55b63ff7c050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:01:26.266 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.260+0000 7f884641b640 1 --2- 172.21.15.42:0/568269048 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b63fbab4d0 0x55b63ff7c050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:26.266 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.260+0000 7f884641b640 1 -- 172.21.15.42:0/568269048 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b63fcb0620 con 0x55b63fba32f0 2024-08-30T09:01:26.266 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.260+0000 7f8845c1a640 1 --2- 172.21.15.42:0/568269048 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b63fbab4d0 0x55b63ff7c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-30T09:01:26.266 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.260+0000 7f8846c1c640 1 --2- 172.21.15.42:0/568269048 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b63fbcc1b0 0x55b63ff7cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-30T09:01:26.266 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.260+0000 7f884641b640 1 --2- 172.21.15.42:0/568269048 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b63fba32f0 0x55b63ff7b990 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f8830015700 tx=0x7f8830004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fcc71e95f6396ec8 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:01:26.266 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.260+0000 7f8825ffb640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-08-30T09:01:26.266 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.261+0000 7f8825ffb640 1 -- 172.21.15.42:0/568269048 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8830025070 con 0x55b63fba32f0 2024-08-30T09:01:26.266 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.261+0000 7f8825ffb640 1 -- 172.21.15.42:0/568269048 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f88300137e0 con 0x55b63fba32f0 2024-08-30T09:01:26.266 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.261+0000 7f884f5e1740 1 -- 172.21.15.42:0/568269048 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8838036030 con 0x55b63fba32f0 2024-08-30T09:01:26.266 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.262+0000 7f8825ffb640 1 -- 172.21.15.42:0/568269048 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f883001b040 con 0x55b63fba32f0 2024-08-30T09:01:26.269 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.262+0000 7f884f5e1740 10 client.25270 fetch_fsmap learned FSMap version 29 2024-08-30T09:01:26.269 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.262+0000 7f884f5e1740 1 -- 172.21.15.42:0/568269048 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b63ff85be0 con 0x55b63fba32f0 2024-08-30T09:01:26.269 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.263+0000 7f8825ffb640 1 -- 172.21.15.42:0/568269048 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8830031070 con 0x55b63fba32f0 2024-08-30T09:01:26.269 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.263+0000 7f8825ffb640 1 -- 172.21.15.42:0/568269048 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f8830021a60 con 0x55b63fba32f0 2024-08-30T09:01:26.269 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.263+0000 7f884f5e1740 10 client.25270 fetch_fsmap finished waiting for FSMap version 29 2024-08-30T09:01:26.269 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.263+0000 7f884f5e1740 10 client.25270 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-30T09:01:26.270 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.263+0000 7f884f5e1740 20 client.25270 populate_metadata read hostname 'smithi042' 2024-08-30T09:01:26.270 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.263+0000 7f884f5e1740 1 --2- 172.21.15.42:0/568269048 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55b63fb8ad80 0x55b63ff893e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:01:26.270 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.263+0000 7f884f5e1740 4 client.25270 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-30T09:01:26.270 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.263+0000 7f884f5e1740 1 -- 172.21.15.42:0/568269048 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b63fee6f50 con 0x55b63fb8ad80 2024-08-30T09:01:26.270 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.264+0000 7f8845c1a640 1 --2- 172.21.15.42:0/568269048 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55b63fb8ad80 0x55b63ff893e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:01:26.270 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.264+0000 7f8825ffb640 10 client.25270 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:01:26.270 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.264+0000 7f8845c1a640 1 --2- 172.21.15.42:0/568269048 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55b63fb8ad80 0x55b63ff893e0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d934a4f839a6489f server_cookie=1fa0189f90e16058 in_seq=0 out_seq=0 2024-08-30T09:01:26.270 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.265+0000 7f8825ffb640 1 -- 172.21.15.42:0/568269048 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b63fee6f50 con 0x55b63fb8ad80 2024-08-30T09:01:26.270 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.265+0000 7f8825ffb640 10 client.25270 handle_command_reply: tid=0 2024-08-30T09:01:26.270 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.265+0000 7f884f5e1740 1 client.25270 shutdown 2024-08-30T09:01:26.270 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.265+0000 7f884f5e1740 1 -- 172.21.15.42:0/568269048 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b63fba32f0 msgr2=0x55b63ff7b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:01:26.270 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.266+0000 7f884f5e1740 1 --2- 172.21.15.42:0/568269048 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b63fba32f0 0x55b63ff7b990 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f8830015700 tx=0x7f8830004750 comp rx=0 tx=0).stop 2024-08-30T09:01:26.270 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.266+0000 7f884f5e1740 1 -- 172.21.15.42:0/568269048 shutdown_connections 2024-08-30T09:01:26.270 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.266+0000 7f884f5e1740 1 --2- 172.21.15.42:0/568269048 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55b63fb8ad80 0x55b63ff893e0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:26.271 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.266+0000 7f884f5e1740 1 --2- 172.21.15.42:0/568269048 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b63fbcc1b0 0x55b63ff7cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:26.271 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.266+0000 7f8825ffb640 0 client.25270 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:01:26.271 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.266+0000 7f884f5e1740 1 --2- 172.21.15.42:0/568269048 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b63fbab4d0 0x55b63ff7c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:26.271 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.266+0000 7f884f5e1740 1 --2- 172.21.15.42:0/568269048 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b63fba32f0 0x55b63ff7b990 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:26.271 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.266+0000 7f884f5e1740 1 -- 172.21.15.42:0/568269048 >> 172.21.15.42:0/568269048 conn(0x55b63fbc0300 msgr2=0x55b63ff6bdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:01:26.271 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.266+0000 7f884f5e1740 1 -- 172.21.15.42:0/568269048 shutdown_connections 2024-08-30T09:01:26.271 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.266+0000 7f884f5e1740 1 -- 172.21.15.42:0/568269048 wait complete. 2024-08-30T09:01:26.271 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.267+0000 7f884f5e1740 20 client.25270 trim_cache size 0 max 16384 2024-08-30T09:01:26.296 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.293+0000 7f884f5e1740 1 Processor -- start 2024-08-30T09:01:26.296 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.294+0000 7f884f5e1740 1 -- start start 2024-08-30T09:01:26.297 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.294+0000 7f884f5e1740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b63fb8ad80 0x55b63ff6d330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:01:26.297 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.294+0000 7f884f5e1740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b63fba32f0 0x55b63ff769d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:01:26.297 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.294+0000 7f884f5e1740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b63fbab4d0 0x55b63ff6bb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:01:26.297 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.294+0000 7f884f5e1740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f8840085380 con 0x55b63fb8ad80 2024-08-30T09:01:26.297 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.294+0000 7f884f5e1740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f8840082c50 con 0x55b63fba32f0 2024-08-30T09:01:26.297 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.294+0000 7f884f5e1740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f8840080a60 con 0x55b63fbab4d0 2024-08-30T09:01:26.297 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.294+0000 7f8846c1c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b63fbab4d0 0x55b63ff6bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:01:26.297 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.294+0000 7f8846c1c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b63fbab4d0 0x55b63ff6bb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:37010/0 (socket says 172.21.15.42:37010) 2024-08-30T09:01:26.298 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.294+0000 7f8846c1c640 1 -- 172.21.15.42:0/3952849185 learned_addr learned my addr 172.21.15.42:0/3952849185 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:01:26.298 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.294+0000 7f884641b640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b63fb8ad80 0x55b63ff6d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:01:26.298 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.295+0000 7f8845c1a640 1 --2- 172.21.15.42:0/3952849185 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b63fba32f0 0x55b63ff769d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:01:26.298 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.295+0000 7f8845c1a640 1 -- 172.21.15.42:0/3952849185 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b63fbab4d0 msgr2=0x55b63ff6bb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:01:26.298 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.295+0000 7f8845c1a640 1 --2- 172.21.15.42:0/3952849185 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b63fbab4d0 0x55b63ff6bb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:26.299 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.295+0000 7f8845c1a640 1 -- 172.21.15.42:0/3952849185 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b63fb8ad80 msgr2=0x55b63ff6d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:01:26.299 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.295+0000 7f8845c1a640 1 --2- 172.21.15.42:0/3952849185 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b63fb8ad80 0x55b63ff6d330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:26.299 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.295+0000 7f8845c1a640 1 -- 172.21.15.42:0/3952849185 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b63ff7cf30 con 0x55b63fba32f0 2024-08-30T09:01:26.299 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.296+0000 7f8845c1a640 1 --2- 172.21.15.42:0/3952849185 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b63fba32f0 0x55b63ff769d0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f88340e2d70 tx=0x7f88340e28d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4d20eada23b2c6ba server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:01:26.299 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.296+0000 7f8827fff640 1 -- 172.21.15.42:0/3952849185 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88340b4c50 con 0x55b63fba32f0 2024-08-30T09:01:26.299 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.296+0000 7f8827fff640 1 -- 172.21.15.42:0/3952849185 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f88340e2b10 con 0x55b63fba32f0 2024-08-30T09:01:26.300 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.296+0000 7f8827fff640 1 -- 172.21.15.42:0/3952849185 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88340a7620 con 0x55b63fba32f0 2024-08-30T09:01:26.300 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.297+0000 7f884f5e1740 1 -- 172.21.15.42:0/3952849185 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b63fba32f0 msgr2=0x55b63ff769d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:01:26.301 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.297+0000 7f884f5e1740 1 --2- 172.21.15.42:0/3952849185 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b63fba32f0 0x55b63ff769d0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f88340e2d70 tx=0x7f88340e28d0 comp rx=0 tx=0).stop 2024-08-30T09:01:26.301 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.297+0000 7f884f5e1740 1 -- 172.21.15.42:0/3952849185 shutdown_connections 2024-08-30T09:01:26.301 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.297+0000 7f884f5e1740 1 --2- 172.21.15.42:0/3952849185 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b63fbab4d0 0x55b63ff6bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:26.301 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.297+0000 7f884f5e1740 1 --2- 172.21.15.42:0/3952849185 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b63fba32f0 0x55b63ff769d0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:26.301 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.297+0000 7f884f5e1740 1 --2- 172.21.15.42:0/3952849185 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b63fb8ad80 0x55b63ff6d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:26.301 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.297+0000 7f884f5e1740 1 -- 172.21.15.42:0/3952849185 >> 172.21.15.42:0/3952849185 conn(0x55b63fbc0300 msgr2=0x55b63ff71b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:01:26.301 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.297+0000 7f884f5e1740 1 -- 172.21.15.42:0/3952849185 shutdown_connections 2024-08-30T09:01:26.301 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.297+0000 7f884f5e1740 1 -- 172.21.15.42:0/3952849185 wait complete. 2024-08-30T09:01:26.302 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.298+0000 7f884f5e1740 1 -- start start 2024-08-30T09:01:26.302 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.298+0000 7f884f5e1740 1 Processor -- start 2024-08-30T09:01:26.302 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.299+0000 7f884f5e1740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b63fb8ad80 0x55b63ff83140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:01:26.302 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.299+0000 7f884f5e1740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b63fba32f0 0x55b63ff81e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:01:26.302 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.300+0000 7f884f5e1740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b63fbab4d0 0x55b63ff82340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:01:26.303 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.300+0000 7f884f5e1740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f8840081110 con 0x55b63fb8ad80 2024-08-30T09:01:26.303 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.300+0000 7f884f5e1740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f8840082580 con 0x55b63fbab4d0 2024-08-30T09:01:26.304 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.300+0000 7f8845c1a640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b63fba32f0 0x55b63ff81e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:01:26.304 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.300+0000 7f884f5e1740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f88400817e0 con 0x55b63fba32f0 2024-08-30T09:01:26.304 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.300+0000 7f8845c1a640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b63fba32f0 0x55b63ff81e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:37024/0 (socket says 172.21.15.42:37024) 2024-08-30T09:01:26.304 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.300+0000 7f8845c1a640 1 -- 172.21.15.42:0/1521794881 learned_addr learned my addr 172.21.15.42:0/1521794881 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:01:26.304 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.300+0000 7f884f5e1740 10 client.0 _ll_register_callbacks cb 0x7f883000e930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:01:26.304 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.300+0000 7f8846c1c640 1 --2- 172.21.15.42:0/1521794881 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b63fbab4d0 0x55b63ff82340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:01:26.304 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.300+0000 7f884641b640 1 --2- 172.21.15.42:0/1521794881 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b63fb8ad80 0x55b63ff83140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:01:26.304 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.300+0000 7f8846c1c640 1 -- 172.21.15.42:0/1521794881 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b63fba32f0 msgr2=0x55b63ff81e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:01:26.304 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.300+0000 7f8846c1c640 1 --2- 172.21.15.42:0/1521794881 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b63fba32f0 0x55b63ff81e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:26.304 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.300+0000 7f8846c1c640 1 -- 172.21.15.42:0/1521794881 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b63fb8ad80 msgr2=0x55b63ff83140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:01:26.304 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.300+0000 7f8846c1c640 1 --2- 172.21.15.42:0/1521794881 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b63fb8ad80 0x55b63ff83140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:26.305 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.300+0000 7f8846c1c640 1 -- 172.21.15.42:0/1521794881 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b63ff82a00 con 0x55b63fbab4d0 2024-08-30T09:01:26.305 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.301+0000 7f8846c1c640 1 --2- 172.21.15.42:0/1521794881 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b63fbab4d0 0x55b63ff82340 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f88380097f0 tx=0x7f883802b0a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2531b7453127f265 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:01:26.305 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.301+0000 7f8825ffb640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-30T09:01:26.305 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.301+0000 7f8825ffb640 1 -- 172.21.15.42:0/1521794881 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8838009360 con 0x55b63fbab4d0 2024-08-30T09:01:26.305 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.301+0000 7f8825ffb640 1 -- 172.21.15.42:0/1521794881 <== mon.2 v2:172.21.15.167:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f8838000f40 con 0x55b63fbab4d0 2024-08-30T09:01:26.305 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.301+0000 7f884f5e1740 1 -- 172.21.15.42:0/1521794881 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f88340e2b10 con 0x55b63fbab4d0 2024-08-30T09:01:26.307 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.301+0000 7f8825ffb640 1 -- 172.21.15.42:0/1521794881 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8838004040 con 0x55b63fbab4d0 2024-08-30T09:01:26.307 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.302+0000 7f884f5e1740 10 client.25280 fetch_fsmap learned FSMap version 29 2024-08-30T09:01:26.307 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.302+0000 7f884f5e1740 1 -- 172.21.15.42:0/1521794881 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b63fff3de0 con 0x55b63fbab4d0 2024-08-30T09:01:26.307 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.303+0000 7f8825ffb640 1 -- 172.21.15.42:0/1521794881 <== mon.2 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f883805c020 con 0x55b63fbab4d0 2024-08-30T09:01:26.307 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.303+0000 7f8825ffb640 1 -- 172.21.15.42:0/1521794881 <== mon.2 v2:172.21.15.167:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f8838043070 con 0x55b63fbab4d0 2024-08-30T09:01:26.307 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.303+0000 7f884f5e1740 10 client.25280 fetch_fsmap finished waiting for FSMap version 29 2024-08-30T09:01:26.307 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.303+0000 7f884f5e1740 10 client.25280 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-30T09:01:26.307 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.303+0000 7f884f5e1740 20 client.25280 populate_metadata read hostname 'smithi042' 2024-08-30T09:01:26.307 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.303+0000 7f884f5e1740 1 --2- 172.21.15.42:0/1521794881 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55b63fbcc1b0 0x55b63fff6300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:01:26.307 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.303+0000 7f884f5e1740 4 client.25280 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-30T09:01:26.307 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.303+0000 7f884f5e1740 1 -- 172.21.15.42:0/1521794881 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b63fee29f0 con 0x55b63fbcc1b0 2024-08-30T09:01:26.307 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.303+0000 7f884641b640 1 --2- 172.21.15.42:0/1521794881 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55b63fbcc1b0 0x55b63fff6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:01:26.308 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.304+0000 7f8825ffb640 10 client.25280 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:01:26.308 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.304+0000 7f884641b640 1 --2- 172.21.15.42:0/1521794881 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55b63fbcc1b0 0x55b63fff6300 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d0c171ab552f542 server_cookie=c1e1aa168548055e in_seq=0 out_seq=0 2024-08-30T09:01:26.318 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.316+0000 7f8825ffb640 1 -- 172.21.15.42:0/1521794881 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b63fee29f0 con 0x55b63fbcc1b0 2024-08-30T09:01:26.318 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.316+0000 7f8825ffb640 10 client.25280 handle_command_reply: tid=0 2024-08-30T09:01:26.319 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.316+0000 7f884f5e1740 1 client.25280 shutdown 2024-08-30T09:01:26.319 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.316+0000 7f884f5e1740 1 -- 172.21.15.42:0/1521794881 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b63fbab4d0 msgr2=0x55b63ff82340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:01:26.319 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.316+0000 7f884f5e1740 1 --2- 172.21.15.42:0/1521794881 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b63fbab4d0 0x55b63ff82340 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f88380097f0 tx=0x7f883802b0a0 comp rx=0 tx=0).stop 2024-08-30T09:01:26.320 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.316+0000 7f884f5e1740 1 -- 172.21.15.42:0/1521794881 shutdown_connections 2024-08-30T09:01:26.320 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.316+0000 7f884f5e1740 1 --2- 172.21.15.42:0/1521794881 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55b63fbcc1b0 0x55b63fff6300 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:26.320 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.317+0000 7f884f5e1740 1 --2- 172.21.15.42:0/1521794881 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b63fba32f0 0x55b63ff81e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:26.320 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.317+0000 7f884f5e1740 1 --2- 172.21.15.42:0/1521794881 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b63fbab4d0 0x55b63ff82340 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:26.320 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.317+0000 7f8825ffb640 0 client.25280 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:01:26.320 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.317+0000 7f884f5e1740 1 --2- 172.21.15.42:0/1521794881 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b63fb8ad80 0x55b63ff83140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:26.320 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.317+0000 7f884f5e1740 1 -- 172.21.15.42:0/1521794881 >> 172.21.15.42:0/1521794881 conn(0x55b63fbc0300 msgr2=0x55b63ff7b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:01:26.321 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.317+0000 7f884f5e1740 1 -- 172.21.15.42:0/1521794881 shutdown_connections 2024-08-30T09:01:26.321 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.317+0000 7f884f5e1740 1 -- 172.21.15.42:0/1521794881 wait complete. 2024-08-30T09:01:26.321 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.317+0000 7f884f5e1740 20 client.25280 trim_cache size 0 max 16384 2024-08-30T09:01:26.321 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-08-30T09:01:26.321 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2024-08-30T09:01:26.321 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-08-30T09:01:26.321 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-08-30T09:01:26.322 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.319+0000 7f884867f640 1 -- 172.21.15.42:0/474720154 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f88280b4b90 msgr2=0x7f88280b70a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:01:26.322 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.319+0000 7f884867f640 1 --2- 172.21.15.42:0/474720154 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f88280b4b90 0x7f88280b70a0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f8830006fd0 tx=0x7f88300090e0 comp rx=0 tx=0).stop 2024-08-30T09:01:26.322 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.319+0000 7f884867f640 1 -- 172.21.15.42:0/474720154 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f884010ad90 msgr2=0x7f884011ba90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:01:26.322 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.319+0000 7f884867f640 1 --2- 172.21.15.42:0/474720154 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f884010ad90 0x7f884011ba90 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f883400cab0 tx=0x7f883400cf70 comp rx=0 tx=0).stop 2024-08-30T09:01:26.322 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.319+0000 7f884867f640 1 -- 172.21.15.42:0/474720154 shutdown_connections 2024-08-30T09:01:26.323 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.319+0000 7f884867f640 1 --2- 172.21.15.42:0/474720154 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f88280b4b90 0x7f88280b70a0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:26.323 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.319+0000 7f884867f640 1 --2- 172.21.15.42:0/474720154 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f884010ad90 0x7f884011ba90 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:26.323 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.319+0000 7f884867f640 1 --2- 172.21.15.42:0/474720154 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f884010b9a0 0x7f884011c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:26.323 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.319+0000 7f884867f640 1 --2- 172.21.15.42:0/474720154 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f884010a340 0x7f884011b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:26.323 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.319+0000 7f884867f640 1 -- 172.21.15.42:0/474720154 >> 172.21.15.42:0/474720154 conn(0x7f8840070a10 msgr2=0x7f88400717e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:01:26.323 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.319+0000 7f884867f640 1 -- 172.21.15.42:0/474720154 shutdown_connections 2024-08-30T09:01:26.323 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:26.319+0000 7f884867f640 1 -- 172.21.15.42:0/474720154 wait complete. 2024-08-30T09:01:26.631 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-08-30T09:01:26.632 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-08-30T09:01:26.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:01:26 smithi167 ceph-mon[22063]: pgmap v393: 129 pgs: 129 active+clean; 976 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 7.4 MiB/s wr, 638 op/s 2024-08-30T09:01:26.830 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:01:26 smithi042 ceph-mon[18652]: pgmap v393: 129 pgs: 129 active+clean; 976 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 7.4 MiB/s wr, 638 op/s 2024-08-30T09:01:26.935 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:01:26 smithi190 ceph-mon[22001]: pgmap v393: 129 pgs: 129 active+clean; 976 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 7.4 MiB/s wr, 638 op/s 2024-08-30T09:01:27.038 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T09:01:27.650 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.647+0000 7fb3854a9640 1 -- 172.21.15.42:0/2451672247 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb380108170 msgr2=0x7fb380108580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:01:27.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.647+0000 7fb3854a9640 1 --2- 172.21.15.42:0/2451672247 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb380108170 0x7fb380108580 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fb370009860 tx=0x7fb37002a340 comp rx=0 tx=0).stop 2024-08-30T09:01:27.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.647+0000 7fb3854a9640 1 -- 172.21.15.42:0/2451672247 shutdown_connections 2024-08-30T09:01:27.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.647+0000 7fb3854a9640 1 --2- 172.21.15.42:0/2451672247 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb380108170 0x7fb380108580 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:27.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.647+0000 7fb3854a9640 1 --2- 172.21.15.42:0/2451672247 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb380108ac0 0x7fb38010f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:27.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.647+0000 7fb3854a9640 1 --2- 172.21.15.42:0/2451672247 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb380102880 0x7fb380102c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:27.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.647+0000 7fb3854a9640 1 -- 172.21.15.42:0/2451672247 >> 172.21.15.42:0/2451672247 conn(0x7fb380079500 msgr2=0x7fb380079910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:01:27.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.647+0000 7fb3854a9640 1 -- 172.21.15.42:0/2451672247 shutdown_connections 2024-08-30T09:01:27.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.647+0000 7fb3854a9640 1 -- 172.21.15.42:0/2451672247 wait complete. 2024-08-30T09:01:27.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.648+0000 7fb3854a9640 1 Processor -- start 2024-08-30T09:01:27.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.650+0000 7fb3854a9640 1 -- start start 2024-08-30T09:01:27.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.651+0000 7fb3854a9640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb380102880 0x7fb380199120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:01:27.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.651+0000 7fb3854a9640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb380108170 0x7fb3801996c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:01:27.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.651+0000 7fb37effd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb380102880 0x7fb380199120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:01:27.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.651+0000 7fb37effd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb380102880 0x7fb380199120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:37044/0 (socket says 172.21.15.42:37044) 2024-08-30T09:01:27.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.651+0000 7fb37effd640 1 -- 172.21.15.42:0/1764842499 learned_addr learned my addr 172.21.15.42:0/1764842499 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:01:27.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.651+0000 7fb3854a9640 1 --2- 172.21.15.42:0/1764842499 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb380108ac0 0x7fb38019e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:01:27.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.651+0000 7fb3854a9640 1 -- 172.21.15.42:0/1764842499 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fb3801126f0 con 0x7fb380108170 2024-08-30T09:01:27.655 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.651+0000 7fb3854a9640 1 -- 172.21.15.42:0/1764842499 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb380112570 con 0x7fb380108ac0 2024-08-30T09:01:27.655 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.651+0000 7fb3854a9640 1 -- 172.21.15.42:0/1764842499 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fb380112870 con 0x7fb380102880 2024-08-30T09:01:27.656 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.652+0000 7fb37f7fe640 1 --2- 172.21.15.42:0/1764842499 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb380108ac0 0x7fb38019e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:01:27.656 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.652+0000 7fb37effd640 1 -- 172.21.15.42:0/1764842499 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb380108ac0 msgr2=0x7fb38019e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:01:27.656 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.652+0000 7fb37effd640 1 --2- 172.21.15.42:0/1764842499 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb380108ac0 0x7fb38019e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:27.656 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.652+0000 7fb37effd640 1 -- 172.21.15.42:0/1764842499 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb380108170 msgr2=0x7fb3801996c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:01:27.656 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.652+0000 7fb37effd640 1 --2- 172.21.15.42:0/1764842499 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb380108170 0x7fb3801996c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:27.656 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.652+0000 7fb37effd640 1 -- 172.21.15.42:0/1764842499 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb38019f090 con 0x7fb380102880 2024-08-30T09:01:27.657 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.653+0000 7fb37effd640 1 --2- 172.21.15.42:0/1764842499 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb380102880 0x7fb380199120 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb378009d50 tx=0x7fb37800c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:01:27.657 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.653+0000 7fb367fff640 1 -- 172.21.15.42:0/1764842499 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb378011270 con 0x7fb380102880 2024-08-30T09:01:27.657 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.653+0000 7fb367fff640 1 -- 172.21.15.42:0/1764842499 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb378011410 con 0x7fb380102880 2024-08-30T09:01:27.657 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.653+0000 7fb367fff640 1 -- 172.21.15.42:0/1764842499 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb378004310 con 0x7fb380102880 2024-08-30T09:01:27.657 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.653+0000 7fb3854a9640 1 -- 172.21.15.42:0/1764842499 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb38019f380 con 0x7fb380102880 2024-08-30T09:01:27.657 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.653+0000 7fb3854a9640 1 -- 172.21.15.42:0/1764842499 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb38006b8e0 con 0x7fb380102880 2024-08-30T09:01:27.659 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.656+0000 7fb367fff640 1 -- 172.21.15.42:0/1764842499 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7fb3780044b0 con 0x7fb380102880 2024-08-30T09:01:27.660 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.657+0000 7fb367fff640 1 --2- 172.21.15.42:0/1764842499 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fb3680b4c80 0x7fb3680b7190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:01:27.660 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.657+0000 7fb367fff640 1 -- 172.21.15.42:0/1764842499 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7fb3780e9bc0 con 0x7fb380102880 2024-08-30T09:01:27.660 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.658+0000 7fb38ba1e740 1 Processor -- start 2024-08-30T09:01:27.660 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.658+0000 7fb37e7fc640 1 --2- 172.21.15.42:0/1764842499 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fb3680b4c80 0x7fb3680b7190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:01:27.660 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.658+0000 7fb38ba1e740 1 -- start start 2024-08-30T09:01:27.661 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.658+0000 7fb38ba1e740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bf737ad350 0x55bf73b53780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:01:27.661 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.658+0000 7fb38ba1e740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bf73784490 0x55bf73b53cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:01:27.661 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.658+0000 7fb38ba1e740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bf7378c670 0x55bf73b54200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:01:27.661 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.658+0000 7fb38ba1e740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55bf73ac95f0 con 0x55bf7378c670 2024-08-30T09:01:27.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.658+0000 7fb38ba1e740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55bf73ab8b80 con 0x55bf737ad350 2024-08-30T09:01:27.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.658+0000 7fb38ba1e740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55bf73abcb30 con 0x55bf73784490 2024-08-30T09:01:27.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.658+0000 7fb37effd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bf737ad350 0x55bf73b53780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:01:27.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.658+0000 7fb37f7fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bf7378c670 0x55bf73b54200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:01:27.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.658+0000 7fb37e7fc640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bf73784490 0x55bf73b53cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:01:27.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.658+0000 7fb37f7fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bf7378c670 0x55bf73b54200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:55634/0 (socket says 172.21.15.42:55634) 2024-08-30T09:01:27.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.658+0000 7fb37effd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bf737ad350 0x55bf73b53780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:49888/0 (socket says 172.21.15.42:49888) 2024-08-30T09:01:27.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.658+0000 7fb37f7fe640 1 -- 172.21.15.42:0/318483851 learned_addr learned my addr 172.21.15.42:0/318483851 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:01:27.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.659+0000 7fb37e7fc640 1 --2- 172.21.15.42:0/1764842499 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fb3680b4c80 0x7fb3680b7190 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fb3700095f0 tx=0x7fb370009580 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:01:27.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.659+0000 7fb37effd640 1 -- 172.21.15.42:0/318483851 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bf73784490 msgr2=0x55bf73b53cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:01:27.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.659+0000 7fb37effd640 1 --2- 172.21.15.42:0/318483851 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bf73784490 0x55bf73b53cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:27.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.659+0000 7fb37effd640 1 -- 172.21.15.42:0/318483851 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bf7378c670 msgr2=0x55bf73b54200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:01:27.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.659+0000 7fb37effd640 1 --2- 172.21.15.42:0/318483851 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bf7378c670 0x55bf73b54200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:27.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.659+0000 7fb37effd640 1 -- 172.21.15.42:0/318483851 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf73818930 con 0x55bf737ad350 2024-08-30T09:01:27.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.659+0000 7fb37effd640 1 --2- 172.21.15.42:0/318483851 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bf737ad350 0x55bf73b53780 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb3780e7380 tx=0x7fb3780b0d60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f0377c91d4cc80be server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:01:27.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.659+0000 7fb3657fa640 1 -- 172.21.15.42:0/318483851 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3780b1110 con 0x55bf737ad350 2024-08-30T09:01:27.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.659+0000 7fb3657fa640 1 -- 172.21.15.42:0/318483851 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb3780b5980 con 0x55bf737ad350 2024-08-30T09:01:27.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.659+0000 7fb3657fa640 1 -- 172.21.15.42:0/318483851 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3780aca80 con 0x55bf737ad350 2024-08-30T09:01:27.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.661+0000 7fb38ba1e740 1 -- 172.21.15.42:0/318483851 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bf737ad350 msgr2=0x55bf73b53780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:01:27.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.661+0000 7fb38ba1e740 1 --2- 172.21.15.42:0/318483851 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bf737ad350 0x55bf73b53780 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb3780e7380 tx=0x7fb3780b0d60 comp rx=0 tx=0).stop 2024-08-30T09:01:27.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.661+0000 7fb38ba1e740 1 -- 172.21.15.42:0/318483851 shutdown_connections 2024-08-30T09:01:27.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.661+0000 7fb38ba1e740 1 --2- 172.21.15.42:0/318483851 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bf73784490 0x55bf73b53cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:27.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.661+0000 7fb38ba1e740 1 --2- 172.21.15.42:0/318483851 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bf7378c670 0x55bf73b54200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:27.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.661+0000 7fb38ba1e740 1 --2- 172.21.15.42:0/318483851 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bf737ad350 0x55bf73b53780 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:27.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.661+0000 7fb38ba1e740 1 -- 172.21.15.42:0/318483851 >> 172.21.15.42:0/318483851 conn(0x55bf737a1740 msgr2=0x55bf73b51670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:01:27.666 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.662+0000 7fb38ba1e740 1 -- 172.21.15.42:0/318483851 shutdown_connections 2024-08-30T09:01:27.666 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.662+0000 7fb38ba1e740 1 -- 172.21.15.42:0/318483851 wait complete. 2024-08-30T09:01:27.666 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.663+0000 7fb38ba1e740 1 -- start start 2024-08-30T09:01:27.666 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.663+0000 7fb38ba1e740 1 Processor -- start 2024-08-30T09:01:27.668 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.664+0000 7fb38ba1e740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bf73784490 0x55bf73b5f210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:01:27.668 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.664+0000 7fb38ba1e740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bf7378c670 0x55bf73b5f750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:01:27.668 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.664+0000 7fb38ba1e740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bf737ad350 0x55bf73b6b250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:01:27.668 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.665+0000 7fb38ba1e740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55bf7391b890 con 0x55bf73784490 2024-08-30T09:01:27.668 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.665+0000 7fb37effd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bf73784490 0x55bf73b5f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:01:27.668 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.665+0000 7fb37effd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bf73784490 0x55bf73b5f210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:55638/0 (socket says 172.21.15.42:55638) 2024-08-30T09:01:27.668 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.665+0000 7fb37effd640 1 -- 172.21.15.42:0/2203105123 learned_addr learned my addr 172.21.15.42:0/2203105123 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:01:27.668 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.665+0000 7fb37f7fe640 1 --2- 172.21.15.42:0/2203105123 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bf737ad350 0x55bf73b6b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:01:27.668 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.665+0000 7fb37e7fc640 1 --2- 172.21.15.42:0/2203105123 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bf7378c670 0x55bf73b5f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:01:27.668 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.665+0000 7fb38ba1e740 1 -- 172.21.15.42:0/2203105123 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55bf73a3f790 con 0x55bf7378c670 2024-08-30T09:01:27.668 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.665+0000 7fb38ba1e740 1 -- 172.21.15.42:0/2203105123 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55bf73b61f60 con 0x55bf737ad350 2024-08-30T09:01:27.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.665+0000 7fb38ba1e740 10 client.0 _ll_register_callbacks cb 0x55bf737c4ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:01:27.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.665+0000 7fb37f7fe640 1 -- 172.21.15.42:0/2203105123 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bf7378c670 msgr2=0x55bf73b5f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:01:27.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.665+0000 7fb37f7fe640 1 --2- 172.21.15.42:0/2203105123 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bf7378c670 0x55bf73b5f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:27.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.665+0000 7fb37f7fe640 1 -- 172.21.15.42:0/2203105123 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bf73784490 msgr2=0x55bf73b5f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:01:27.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.665+0000 7fb37f7fe640 1 --2- 172.21.15.42:0/2203105123 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bf73784490 0x55bf73b5f210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:27.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.666+0000 7fb37f7fe640 1 -- 172.21.15.42:0/2203105123 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf73a76f30 con 0x55bf737ad350 2024-08-30T09:01:27.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.666+0000 7fb38ba1e740 1 -- 172.21.15.42:0/2203105123 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb3780b5980 con 0x55bf737ad350 2024-08-30T09:01:27.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.666+0000 7fb37f7fe640 1 --2- 172.21.15.42:0/2203105123 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bf737ad350 0x55bf73b6b250 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fb3740126d0 tx=0x7fb374012b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ff74cc1b1c38ea9f server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:01:27.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.666+0000 7fb362ff5640 10 client.25288 ms_handle_connect on v2:172.21.15.190:3300/0 2024-08-30T09:01:27.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.666+0000 7fb362ff5640 1 -- 172.21.15.42:0/2203105123 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb374004300 con 0x55bf737ad350 2024-08-30T09:01:27.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.666+0000 7fb362ff5640 1 -- 172.21.15.42:0/2203105123 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7fb374011920 con 0x55bf737ad350 2024-08-30T09:01:27.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.667+0000 7fb362ff5640 1 -- 172.21.15.42:0/2203105123 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb374011b40 con 0x55bf737ad350 2024-08-30T09:01:27.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.667+0000 7fb38ba1e740 10 client.25288 fetch_fsmap learned FSMap version 29 2024-08-30T09:01:27.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.667+0000 7fb38ba1e740 1 -- 172.21.15.42:0/2203105123 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({fsmap=29}) -- 0x55bf73b6ddc0 con 0x55bf737ad350 2024-08-30T09:01:27.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.667+0000 7fb362ff5640 1 -- 172.21.15.42:0/2203105123 <== mon.1 v2:172.21.15.190:3300/0 4 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fb374011ce0 con 0x55bf737ad350 2024-08-30T09:01:27.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.667+0000 7fb38ba1e740 10 client.25288 fetch_fsmap finished waiting for FSMap version 29 2024-08-30T09:01:27.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.667+0000 7fb38ba1e740 10 client.25288 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-30T09:01:27.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.667+0000 7fb38ba1e740 20 client.25288 populate_metadata read hostname 'smithi042' 2024-08-30T09:01:27.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.667+0000 7fb38ba1e740 1 --2- 172.21.15.42:0/2203105123 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55bf7376bcd0 0x55bf73b71200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:01:27.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.667+0000 7fb38ba1e740 4 client.25288 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-30T09:01:27.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.667+0000 7fb38ba1e740 1 -- 172.21.15.42:0/2203105123 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bf73acb540 con 0x55bf7376bcd0 2024-08-30T09:01:27.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.667+0000 7fb37effd640 1 --2- 172.21.15.42:0/2203105123 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55bf7376bcd0 0x55bf73b71200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:01:27.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.668+0000 7fb362ff5640 10 client.25288 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:01:27.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.668+0000 7fb37effd640 1 --2- 172.21.15.42:0/2203105123 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55bf7376bcd0 0x55bf73b71200 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bcb83c8c544034e2 server_cookie=f2bba14f2dcbd8f4 in_seq=0 out_seq=0 2024-08-30T09:01:27.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.669+0000 7fb362ff5640 1 -- 172.21.15.42:0/2203105123 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55bf73acb540 con 0x55bf7376bcd0 2024-08-30T09:01:27.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.669+0000 7fb362ff5640 10 client.25288 handle_command_reply: tid=0 2024-08-30T09:01:27.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.669+0000 7fb38ba1e740 1 client.25288 shutdown 2024-08-30T09:01:27.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.669+0000 7fb38ba1e740 1 -- 172.21.15.42:0/2203105123 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bf737ad350 msgr2=0x55bf73b6b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:01:27.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.669+0000 7fb38ba1e740 1 --2- 172.21.15.42:0/2203105123 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bf737ad350 0x55bf73b6b250 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fb3740126d0 tx=0x7fb374012b90 comp rx=0 tx=0).stop 2024-08-30T09:01:27.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.669+0000 7fb38ba1e740 1 -- 172.21.15.42:0/2203105123 shutdown_connections 2024-08-30T09:01:27.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.669+0000 7fb38ba1e740 1 --2- 172.21.15.42:0/2203105123 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55bf7376bcd0 0x55bf73b71200 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:27.674 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.669+0000 7fb38ba1e740 1 --2- 172.21.15.42:0/2203105123 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bf737ad350 0x55bf73b6b250 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:27.674 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.669+0000 7fb362ff5640 0 client.25288 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:01:27.674 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.669+0000 7fb38ba1e740 1 --2- 172.21.15.42:0/2203105123 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bf7378c670 0x55bf73b5f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:27.674 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.669+0000 7fb38ba1e740 1 --2- 172.21.15.42:0/2203105123 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bf73784490 0x55bf73b5f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:27.674 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.669+0000 7fb38ba1e740 1 -- 172.21.15.42:0/2203105123 >> 172.21.15.42:0/2203105123 conn(0x55bf737a1740 msgr2=0x55bf73b56960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:01:27.674 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.669+0000 7fb38ba1e740 1 -- 172.21.15.42:0/2203105123 shutdown_connections 2024-08-30T09:01:27.674 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.670+0000 7fb38ba1e740 1 -- 172.21.15.42:0/2203105123 wait complete. 2024-08-30T09:01:27.674 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.670+0000 7fb38ba1e740 20 client.25288 trim_cache size 0 max 16384 2024-08-30T09:01:27.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.698+0000 7fb38ba1e740 1 Processor -- start 2024-08-30T09:01:27.703 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.699+0000 7fb38ba1e740 1 -- start start 2024-08-30T09:01:27.703 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.700+0000 7fb38ba1e740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bf7376bcd0 0x55bf73b60ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:01:27.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.700+0000 7fb38ba1e740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bf73784490 0x55bf73b5c200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:01:27.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.700+0000 7fb38ba1e740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bf7378c670 0x55bf73b5c740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:01:27.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.700+0000 7fb38ba1e740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fb3801b0ff0 con 0x55bf73784490 2024-08-30T09:01:27.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.700+0000 7fb38ba1e740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb3801ae8c0 con 0x55bf7376bcd0 2024-08-30T09:01:27.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.700+0000 7fb37f7fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bf7378c670 0x55bf73b5c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:01:27.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.700+0000 7fb37f7fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bf7378c670 0x55bf73b5c740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:37070/0 (socket says 172.21.15.42:37070) 2024-08-30T09:01:27.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.700+0000 7fb37f7fe640 1 -- 172.21.15.42:0/58255524 learned_addr learned my addr 172.21.15.42:0/58255524 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:01:27.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.701+0000 7fb38ba1e740 1 -- 172.21.15.42:0/58255524 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fb3801ac710 con 0x55bf7378c670 2024-08-30T09:01:27.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.701+0000 7fb37f7fe640 1 -- 172.21.15.42:0/58255524 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bf7376bcd0 msgr2=0x55bf73b60ef0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-30T09:01:27.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.701+0000 7fb37f7fe640 1 --2- 172.21.15.42:0/58255524 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bf7376bcd0 0x55bf73b60ef0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:27.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.701+0000 7fb37f7fe640 1 -- 172.21.15.42:0/58255524 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bf73784490 msgr2=0x55bf73b5c200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:01:27.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.701+0000 7fb37f7fe640 1 --2- 172.21.15.42:0/58255524 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bf73784490 0x55bf73b5c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:27.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.701+0000 7fb37f7fe640 1 -- 172.21.15.42:0/58255524 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf73b5d510 con 0x55bf7378c670 2024-08-30T09:01:27.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.702+0000 7fb37f7fe640 1 --2- 172.21.15.42:0/58255524 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bf7378c670 0x55bf73b5c740 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fb3740126a0 tx=0x7fb37400f000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=65c8ce882dbcfbf6 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:01:27.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.702+0000 7fb364ff9640 1 -- 172.21.15.42:0/58255524 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb374004300 con 0x55bf7378c670 2024-08-30T09:01:27.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.702+0000 7fb364ff9640 1 -- 172.21.15.42:0/58255524 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb3780b5980 con 0x55bf7378c670 2024-08-30T09:01:27.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.702+0000 7fb364ff9640 1 -- 172.21.15.42:0/58255524 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb374003cd0 con 0x55bf7378c670 2024-08-30T09:01:27.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.704+0000 7fb38ba1e740 1 -- 172.21.15.42:0/58255524 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bf7378c670 msgr2=0x55bf73b5c740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:01:27.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.704+0000 7fb38ba1e740 1 --2- 172.21.15.42:0/58255524 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bf7378c670 0x55bf73b5c740 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fb3740126a0 tx=0x7fb37400f000 comp rx=0 tx=0).stop 2024-08-30T09:01:27.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.705+0000 7fb38ba1e740 1 -- 172.21.15.42:0/58255524 shutdown_connections 2024-08-30T09:01:27.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.705+0000 7fb38ba1e740 1 --2- 172.21.15.42:0/58255524 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bf7378c670 0x55bf73b5c740 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:27.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.705+0000 7fb38ba1e740 1 --2- 172.21.15.42:0/58255524 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bf73784490 0x55bf73b5c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:27.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.705+0000 7fb38ba1e740 1 --2- 172.21.15.42:0/58255524 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bf7376bcd0 0x55bf73b60ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:27.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.705+0000 7fb38ba1e740 1 -- 172.21.15.42:0/58255524 >> 172.21.15.42:0/58255524 conn(0x55bf737a1740 msgr2=0x55bf73b56ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:01:27.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.705+0000 7fb38ba1e740 1 -- 172.21.15.42:0/58255524 shutdown_connections 2024-08-30T09:01:27.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.713+0000 7fb38ba1e740 1 -- 172.21.15.42:0/58255524 wait complete. 2024-08-30T09:01:27.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.714+0000 7fb38ba1e740 1 -- start start 2024-08-30T09:01:27.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.714+0000 7fb38ba1e740 1 Processor -- start 2024-08-30T09:01:27.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.716+0000 7fb38ba1e740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bf7376bcd0 0x55bf73b65d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:01:27.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.716+0000 7fb38ba1e740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bf73784490 0x55bf73b662c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:01:27.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.716+0000 7fb38ba1e740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bf7378c670 0x55bf73bd2b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:01:27.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.717+0000 7fb38ba1e740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fb3801acd80 con 0x55bf73784490 2024-08-30T09:01:27.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.717+0000 7fb37f7fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bf7378c670 0x55bf73bd2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:01:27.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.717+0000 7fb37f7fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bf7378c670 0x55bf73bd2b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:49920/0 (socket says 172.21.15.42:49920) 2024-08-30T09:01:27.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.717+0000 7fb37f7fe640 1 -- 172.21.15.42:0/1010283357 learned_addr learned my addr 172.21.15.42:0/1010283357 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:01:27.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.717+0000 7fb38ba1e740 1 -- 172.21.15.42:0/1010283357 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb3801ae1f0 con 0x55bf7378c670 2024-08-30T09:01:27.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.717+0000 7fb38ba1e740 1 -- 172.21.15.42:0/1010283357 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fb3801ad450 con 0x55bf7376bcd0 2024-08-30T09:01:27.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.717+0000 7fb38ba1e740 10 client.0 _ll_register_callbacks cb 0x7fb3740097d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:01:27.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.717+0000 7fb37f7fe640 1 -- 172.21.15.42:0/1010283357 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bf7376bcd0 msgr2=0x55bf73b65d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:01:27.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.717+0000 7fb37f7fe640 1 --2- 172.21.15.42:0/1010283357 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bf7376bcd0 0x55bf73b65d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:27.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.717+0000 7fb37f7fe640 1 -- 172.21.15.42:0/1010283357 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bf73784490 msgr2=0x55bf73b662c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:01:27.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.717+0000 7fb37f7fe640 1 --2- 172.21.15.42:0/1010283357 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bf73784490 0x55bf73b662c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:27.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.717+0000 7fb37f7fe640 1 -- 172.21.15.42:0/1010283357 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf73bd31f0 con 0x55bf7378c670 2024-08-30T09:01:27.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.721+0000 7fb37f7fe640 1 --2- 172.21.15.42:0/1010283357 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bf7378c670 0x55bf73bd2b80 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fb374001eb0 tx=0x7fb374009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=184badc43fcc22e0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:01:27.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.722+0000 7fb362ff5640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-30T09:01:27.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.723+0000 7fb362ff5640 1 -- 172.21.15.42:0/1010283357 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb374036070 con 0x55bf7378c670 2024-08-30T09:01:27.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.723+0000 7fb38ba1e740 1 -- 172.21.15.42:0/1010283357 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb3780b5980 con 0x55bf7378c670 2024-08-30T09:01:27.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.723+0000 7fb362ff5640 1 -- 172.21.15.42:0/1010283357 <== mon.2 v2:172.21.15.167:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7fb374005280 con 0x55bf7378c670 2024-08-30T09:01:27.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.723+0000 7fb362ff5640 1 -- 172.21.15.42:0/1010283357 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb37403f050 con 0x55bf7378c670 2024-08-30T09:01:27.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.724+0000 7fb38ba1e740 10 client.25295 fetch_fsmap learned FSMap version 29 2024-08-30T09:01:27.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.724+0000 7fb38ba1e740 1 -- 172.21.15.42:0/1010283357 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55bf73bdc7d0 con 0x55bf7378c670 2024-08-30T09:01:27.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.727+0000 7fb362ff5640 1 -- 172.21.15.42:0/1010283357 <== mon.2 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb374003cd0 con 0x55bf7378c670 2024-08-30T09:01:27.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.728+0000 7fb362ff5640 1 -- 172.21.15.42:0/1010283357 <== mon.2 v2:172.21.15.167:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fb374039020 con 0x55bf7378c670 2024-08-30T09:01:27.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.730+0000 7fb38ba1e740 10 client.25295 fetch_fsmap finished waiting for FSMap version 29 2024-08-30T09:01:27.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.730+0000 7fb38ba1e740 10 client.25295 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-30T09:01:27.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.730+0000 7fb38ba1e740 20 client.25295 populate_metadata read hostname 'smithi042' 2024-08-30T09:01:27.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.730+0000 7fb38ba1e740 1 --2- 172.21.15.42:0/1010283357 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55bf737ad350 0x55bf73bdef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:01:27.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.730+0000 7fb38ba1e740 4 client.25295 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-08-30T09:01:27.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.730+0000 7fb38ba1e740 1 -- 172.21.15.42:0/1010283357 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7fb3780b5980 con 0x55bf737ad350 2024-08-30T09:01:27.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.733+0000 7fb37effd640 1 --2- 172.21.15.42:0/1010283357 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55bf737ad350 0x55bf73bdef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:01:27.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.736+0000 7fb37effd640 1 --2- 172.21.15.42:0/1010283357 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55bf737ad350 0x55bf73bdef70 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cac0e821d28f9ea5 server_cookie=c410fa8d5d25013 in_seq=0 out_seq=0 2024-08-30T09:01:27.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.737+0000 7fb362ff5640 10 client.25295 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:01:27.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.741+0000 7fb362ff5640 1 -- 172.21.15.42:0/1010283357 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7fb3780b5980 con 0x55bf737ad350 2024-08-30T09:01:27.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.741+0000 7fb362ff5640 10 client.25295 handle_command_reply: tid=0 2024-08-30T09:01:27.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.744+0000 7fb38ba1e740 1 client.25295 shutdown 2024-08-30T09:01:27.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.750+0000 7fb38ba1e740 1 -- 172.21.15.42:0/1010283357 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bf7378c670 msgr2=0x55bf73bd2b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:01:27.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.750+0000 7fb38ba1e740 1 --2- 172.21.15.42:0/1010283357 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bf7378c670 0x55bf73bd2b80 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fb374001eb0 tx=0x7fb374009260 comp rx=0 tx=0).stop 2024-08-30T09:01:27.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.750+0000 7fb38ba1e740 1 -- 172.21.15.42:0/1010283357 shutdown_connections 2024-08-30T09:01:27.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.751+0000 7fb38ba1e740 1 --2- 172.21.15.42:0/1010283357 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55bf737ad350 0x55bf73bdef70 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:27.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.751+0000 7fb38ba1e740 1 --2- 172.21.15.42:0/1010283357 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bf7378c670 0x55bf73bd2b80 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:27.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.751+0000 7fb38ba1e740 1 --2- 172.21.15.42:0/1010283357 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bf73784490 0x55bf73b662c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:27.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.751+0000 7fb38ba1e740 1 --2- 172.21.15.42:0/1010283357 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bf7376bcd0 0x55bf73b65d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:27.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.751+0000 7fb38ba1e740 1 -- 172.21.15.42:0/1010283357 >> 172.21.15.42:0/1010283357 conn(0x55bf737a1740 msgr2=0x55bf73b54750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:01:27.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.751+0000 7fb362ff5640 0 client.25295 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:01:27.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.751+0000 7fb38ba1e740 1 -- 172.21.15.42:0/1010283357 shutdown_connections 2024-08-30T09:01:27.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.751+0000 7fb38ba1e740 1 -- 172.21.15.42:0/1010283357 wait complete. 2024-08-30T09:01:27.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.752+0000 7fb38ba1e740 20 client.25295 trim_cache size 0 max 16384 2024-08-30T09:01:27.755 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-08-30T09:01:27.755 INFO:teuthology.orchestra.run.smithi042.stdout: "return_code": 0, 2024-08-30T09:01:27.755 INFO:teuthology.orchestra.run.smithi042.stdout: "scrub_tag": "85134e4b-ab4a-4885-b07e-a3ad01eaf13d", 2024-08-30T09:01:27.755 INFO:teuthology.orchestra.run.smithi042.stdout: "mode": "asynchronous" 2024-08-30T09:01:27.755 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-08-30T09:01:27.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.753+0000 7fb3854a9640 1 -- 172.21.15.42:0/1764842499 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fb3680b4c80 msgr2=0x7fb3680b7190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:01:27.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.753+0000 7fb3854a9640 1 --2- 172.21.15.42:0/1764842499 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fb3680b4c80 0x7fb3680b7190 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fb3700095f0 tx=0x7fb370009580 comp rx=0 tx=0).stop 2024-08-30T09:01:27.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.753+0000 7fb3854a9640 1 -- 172.21.15.42:0/1764842499 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb380102880 msgr2=0x7fb380199120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:01:27.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.753+0000 7fb3854a9640 1 --2- 172.21.15.42:0/1764842499 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb380102880 0x7fb380199120 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb378009d50 tx=0x7fb37800c6a0 comp rx=0 tx=0).stop 2024-08-30T09:01:27.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.753+0000 7fb3854a9640 1 -- 172.21.15.42:0/1764842499 shutdown_connections 2024-08-30T09:01:27.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.753+0000 7fb3854a9640 1 --2- 172.21.15.42:0/1764842499 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fb3680b4c80 0x7fb3680b7190 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:27.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.753+0000 7fb3854a9640 1 --2- 172.21.15.42:0/1764842499 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb380108ac0 0x7fb38019e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:27.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.753+0000 7fb3854a9640 1 --2- 172.21.15.42:0/1764842499 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb380108170 0x7fb3801996c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:27.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.753+0000 7fb3854a9640 1 --2- 172.21.15.42:0/1764842499 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb380102880 0x7fb380199120 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:27.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.753+0000 7fb3854a9640 1 -- 172.21.15.42:0/1764842499 >> 172.21.15.42:0/1764842499 conn(0x7fb380079500 msgr2=0x7fb3801041e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:01:27.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.753+0000 7fb3854a9640 1 -- 172.21.15.42:0/1764842499 shutdown_connections 2024-08-30T09:01:27.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:27.753+0000 7fb3854a9640 1 -- 172.21.15.42:0/1764842499 wait complete. 2024-08-30T09:01:28.602 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-30T09:01:28.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:01:28 smithi167 ceph-mon[22063]: pgmap v394: 129 pgs: 129 active+clean; 980 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 13 MiB/s wr, 826 op/s 2024-08-30T09:01:28.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:01:28 smithi167 ceph-mon[22063]: scrub summary: active paths [/] 2024-08-30T09:01:28.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:01:28 smithi167 ceph-mon[22063]: scrub summary: idle+waiting paths [/] 2024-08-30T09:01:28.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:01:28 smithi167 ceph-mon[22063]: scrub summary: active paths [/] 2024-08-30T09:01:28.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:01:28 smithi167 ceph-mon[22063]: scrub summary: idle+waiting paths [/] 2024-08-30T09:01:28.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:01:28 smithi167 ceph-mon[22063]: scrub summary: active paths [/] 2024-08-30T09:01:28.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:01:28 smithi167 ceph-mon[22063]: scrub summary: idle+waiting paths [/] 2024-08-30T09:01:28.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:01:28 smithi042 ceph-mon[18652]: pgmap v394: 129 pgs: 129 active+clean; 980 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 13 MiB/s wr, 826 op/s 2024-08-30T09:01:28.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:01:28 smithi042 ceph-mon[18652]: scrub summary: active paths [/] 2024-08-30T09:01:28.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:01:28 smithi042 ceph-mon[18652]: scrub summary: idle+waiting paths [/] 2024-08-30T09:01:28.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:01:28 smithi042 ceph-mon[18652]: scrub summary: active paths [/] 2024-08-30T09:01:28.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:01:28 smithi042 ceph-mon[18652]: scrub summary: idle+waiting paths [/] 2024-08-30T09:01:28.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:01:28 smithi042 ceph-mon[18652]: scrub summary: active paths [/] 2024-08-30T09:01:28.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:01:28 smithi042 ceph-mon[18652]: scrub summary: idle+waiting paths [/] 2024-08-30T09:01:28.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:01:28 smithi190 ceph-mon[22001]: pgmap v394: 129 pgs: 129 active+clean; 980 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 13 MiB/s wr, 826 op/s 2024-08-30T09:01:28.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:01:28 smithi190 ceph-mon[22001]: scrub summary: active paths [/] 2024-08-30T09:01:28.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:01:28 smithi190 ceph-mon[22001]: scrub summary: idle+waiting paths [/] 2024-08-30T09:01:28.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:01:28 smithi190 ceph-mon[22001]: scrub summary: active paths [/] 2024-08-30T09:01:28.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:01:28 smithi190 ceph-mon[22001]: scrub summary: idle+waiting paths [/] 2024-08-30T09:01:28.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:01:28 smithi190 ceph-mon[22001]: scrub summary: active paths [/] 2024-08-30T09:01:28.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:01:28 smithi190 ceph-mon[22001]: scrub summary: idle+waiting paths [/] 2024-08-30T09:01:29.038 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T09:01:29.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.673+0000 7f224d360640 1 -- 172.21.15.42:0/672615566 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2248109250 msgr2=0x7f2248109660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:01:29.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.673+0000 7f224d360640 1 --2- 172.21.15.42:0/672615566 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2248109250 0x7f2248109660 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f223c009900 tx=0x7f223c02a410 comp rx=0 tx=0).stop 2024-08-30T09:01:29.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.674+0000 7f224d360640 1 -- 172.21.15.42:0/672615566 shutdown_connections 2024-08-30T09:01:29.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.674+0000 7f224d360640 1 --2- 172.21.15.42:0/672615566 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2248109ba0 0x7f22481103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:29.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.674+0000 7f224d360640 1 --2- 172.21.15.42:0/672615566 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2248109250 0x7f2248109660 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:29.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.674+0000 7f224d360640 1 --2- 172.21.15.42:0/672615566 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2248103910 0x7f2248103d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:29.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.674+0000 7f224d360640 1 -- 172.21.15.42:0/672615566 >> 172.21.15.42:0/672615566 conn(0x7f22480ff380 msgr2=0x7f22481017d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:01:29.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.674+0000 7f224d360640 1 -- 172.21.15.42:0/672615566 shutdown_connections 2024-08-30T09:01:29.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.674+0000 7f224d360640 1 -- 172.21.15.42:0/672615566 wait complete. 2024-08-30T09:01:29.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.675+0000 7f224d360640 1 Processor -- start 2024-08-30T09:01:29.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.676+0000 7f224d360640 1 -- start start 2024-08-30T09:01:29.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.676+0000 7f224d360640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2248103910 0x7f224819a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:01:29.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.676+0000 7f224d360640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2248109250 0x7f224819a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:01:29.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.676+0000 7f224d360640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2248109ba0 0x7f224819fad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:01:29.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.676+0000 7f224d360640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f2248113860 con 0x7f2248109ba0 2024-08-30T09:01:29.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.676+0000 7f224d360640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f22481136e0 con 0x7f2248109250 2024-08-30T09:01:29.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.676+0000 7f224d360640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f22481139e0 con 0x7f2248103910 2024-08-30T09:01:29.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.676+0000 7f22467fc640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2248109250 0x7f224819a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:01:29.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.677+0000 7f22467fc640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2248109250 0x7f224819a830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:41866/0 (socket says 172.21.15.42:41866) 2024-08-30T09:01:29.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.677+0000 7f22467fc640 1 -- 172.21.15.42:0/416630451 learned_addr learned my addr 172.21.15.42:0/416630451 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:01:29.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.677+0000 7f2246ffd640 1 --2- 172.21.15.42:0/416630451 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2248103910 0x7f224819a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:01:29.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.677+0000 7f22477fe640 1 --2- 172.21.15.42:0/416630451 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2248109ba0 0x7f224819fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:01:29.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.677+0000 7f22467fc640 1 -- 172.21.15.42:0/416630451 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2248103910 msgr2=0x7f224819a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:01:29.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.677+0000 7f22467fc640 1 --2- 172.21.15.42:0/416630451 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2248103910 0x7f224819a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:29.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.677+0000 7f22467fc640 1 -- 172.21.15.42:0/416630451 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2248109ba0 msgr2=0x7f224819fad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:01:29.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.677+0000 7f22467fc640 1 --2- 172.21.15.42:0/416630451 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2248109ba0 0x7f224819fad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:29.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.677+0000 7f22467fc640 1 -- 172.21.15.42:0/416630451 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22481a0200 con 0x7f2248109250 2024-08-30T09:01:29.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.678+0000 7f22467fc640 1 --2- 172.21.15.42:0/416630451 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2248109250 0x7f224819a830 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f223c006010 tx=0x7f223c004060 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:01:29.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.678+0000 7f222ffff640 1 -- 172.21.15.42:0/416630451 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f223c035070 con 0x7f2248109250 2024-08-30T09:01:29.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.678+0000 7f222ffff640 1 -- 172.21.15.42:0/416630451 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f223c03dc10 con 0x7f2248109250 2024-08-30T09:01:29.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.678+0000 7f222ffff640 1 -- 172.21.15.42:0/416630451 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f223c003db0 con 0x7f2248109250 2024-08-30T09:01:29.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.678+0000 7f224d360640 1 -- 172.21.15.42:0/416630451 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22481a0490 con 0x7f2248109250 2024-08-30T09:01:29.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.678+0000 7f224d360640 1 -- 172.21.15.42:0/416630451 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2248075390 con 0x7f2248109250 2024-08-30T09:01:29.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.682+0000 7f22538d5740 1 Processor -- start 2024-08-30T09:01:29.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.682+0000 7f22538d5740 1 -- start start 2024-08-30T09:01:29.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.682+0000 7f22538d5740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562afb395070 0x562afb73a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:01:29.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.682+0000 7f22538d5740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562afb36c1b0 0x562afb73a800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:01:29.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.682+0000 7f22538d5740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562afb374390 0x562afb73ad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:01:29.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.682+0000 7f22538d5740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x562afb6b1630 con 0x562afb395070 2024-08-30T09:01:29.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.682+0000 7f22538d5740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x562afb6a0bc0 con 0x562afb374390 2024-08-30T09:01:29.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.682+0000 7f22538d5740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x562afb6a4b70 con 0x562afb36c1b0 2024-08-30T09:01:29.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.682+0000 7f2246ffd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562afb36c1b0 0x562afb73a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:01:29.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.682+0000 7f2246ffd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562afb36c1b0 0x562afb73a800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:57394/0 (socket says 172.21.15.42:57394) 2024-08-30T09:01:29.687 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.682+0000 7f2246ffd640 1 -- 172.21.15.42:0/954824083 learned_addr learned my addr 172.21.15.42:0/954824083 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:01:29.687 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.682+0000 7f22477fe640 1 --2- 172.21.15.42:0/954824083 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562afb395070 0x562afb73a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:01:29.687 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.683+0000 7f2246ffd640 1 -- 172.21.15.42:0/954824083 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562afb374390 msgr2=0x562afb73ad40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-30T09:01:29.687 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.683+0000 7f2246ffd640 1 --2- 172.21.15.42:0/954824083 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562afb374390 0x562afb73ad40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:29.687 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.683+0000 7f2246ffd640 1 -- 172.21.15.42:0/954824083 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562afb395070 msgr2=0x562afb73a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:01:29.687 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.683+0000 7f2246ffd640 1 --2- 172.21.15.42:0/954824083 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562afb395070 0x562afb73a2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:29.687 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.683+0000 7f2246ffd640 1 -- 172.21.15.42:0/954824083 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562afb400650 con 0x562afb36c1b0 2024-08-30T09:01:29.687 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.683+0000 7f2246ffd640 1 --2- 172.21.15.42:0/954824083 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562afb36c1b0 0x562afb73a800 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f2238009cf0 tx=0x7f2238011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1b672eca38c583cc server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:01:29.687 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.683+0000 7f222d7fa640 1 -- 172.21.15.42:0/954824083 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2238004230 con 0x562afb36c1b0 2024-08-30T09:01:29.687 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.683+0000 7f222d7fa640 1 -- 172.21.15.42:0/954824083 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f22380043d0 con 0x562afb36c1b0 2024-08-30T09:01:29.687 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.683+0000 7f222d7fa640 1 -- 172.21.15.42:0/954824083 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2238019840 con 0x562afb36c1b0 2024-08-30T09:01:29.687 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.683+0000 7f222ffff640 1 -- 172.21.15.42:0/416630451 <== mon.2 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7f223c03d0b0 con 0x7f2248109250 2024-08-30T09:01:29.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.684+0000 7f222ffff640 1 --2- 172.21.15.42:0/416630451 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f22300bd3c0 0x7f22300bf8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:01:29.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.684+0000 7f222ffff640 1 -- 172.21.15.42:0/416630451 <== mon.2 v2:172.21.15.167:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f223c105a60 con 0x7f2248109250 2024-08-30T09:01:29.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.684+0000 7f22538d5740 1 -- 172.21.15.42:0/954824083 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562afb36c1b0 msgr2=0x562afb73a800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:01:29.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.684+0000 7f22538d5740 1 --2- 172.21.15.42:0/954824083 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562afb36c1b0 0x562afb73a800 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f2238009cf0 tx=0x7f2238011810 comp rx=0 tx=0).stop 2024-08-30T09:01:29.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.684+0000 7f22477fe640 1 --2- 172.21.15.42:0/416630451 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f22300bd3c0 0x7f22300bf8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:01:29.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.684+0000 7f22538d5740 1 -- 172.21.15.42:0/954824083 shutdown_connections 2024-08-30T09:01:29.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.684+0000 7f22538d5740 1 --2- 172.21.15.42:0/954824083 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562afb36c1b0 0x562afb73a800 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:29.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.684+0000 7f22538d5740 1 --2- 172.21.15.42:0/954824083 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562afb374390 0x562afb73ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:29.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.684+0000 7f22538d5740 1 --2- 172.21.15.42:0/954824083 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562afb395070 0x562afb73a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:29.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.684+0000 7f22538d5740 1 -- 172.21.15.42:0/954824083 >> 172.21.15.42:0/954824083 conn(0x562afb389460 msgr2=0x562afb7381b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:01:29.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.685+0000 7f22477fe640 1 --2- 172.21.15.42:0/416630451 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f22300bd3c0 0x7f22300bf8d0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f2240008dc0 tx=0x7f2240008370 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:01:29.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.686+0000 7f22538d5740 1 -- 172.21.15.42:0/954824083 shutdown_connections 2024-08-30T09:01:29.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.686+0000 7f22538d5740 1 -- 172.21.15.42:0/954824083 wait complete. 2024-08-30T09:01:29.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.687+0000 7f22538d5740 1 -- start start 2024-08-30T09:01:29.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.687+0000 7f22538d5740 1 Processor -- start 2024-08-30T09:01:29.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.688+0000 7f22538d5740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562afb36c1b0 0x562afb745aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:01:29.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.688+0000 7f22538d5740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562afb374390 0x562afb745fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:01:29.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.688+0000 7f22538d5740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562afb395070 0x562afb751dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:01:29.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.688+0000 7f22467fc640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562afb374390 0x562afb745fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:01:29.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.688+0000 7f2246ffd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562afb36c1b0 0x562afb745aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:01:29.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.688+0000 7f22467fc640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562afb374390 0x562afb745fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:41880/0 (socket says 172.21.15.42:41880) 2024-08-30T09:01:29.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.688+0000 7f22467fc640 1 -- 172.21.15.42:0/255951770 learned_addr learned my addr 172.21.15.42:0/255951770 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:01:29.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.688+0000 7f22538d5740 1 -- 172.21.15.42:0/255951770 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x562afb503700 con 0x562afb395070 2024-08-30T09:01:29.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.688+0000 7f22538d5740 1 -- 172.21.15.42:0/255951770 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x562afb627700 con 0x562afb374390 2024-08-30T09:01:29.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.688+0000 7f22538d5740 1 -- 172.21.15.42:0/255951770 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x562afb748aa0 con 0x562afb36c1b0 2024-08-30T09:01:29.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.689+0000 7f22538d5740 10 client.0 _ll_register_callbacks cb 0x562afb656dc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:01:29.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.689+0000 7f22467fc640 1 -- 172.21.15.42:0/255951770 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562afb36c1b0 msgr2=0x562afb745aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:01:29.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.689+0000 7f22467fc640 1 --2- 172.21.15.42:0/255951770 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562afb36c1b0 0x562afb745aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:29.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.689+0000 7f22467fc640 1 -- 172.21.15.42:0/255951770 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562afb395070 msgr2=0x562afb751dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:01:29.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.689+0000 7f22467fc640 1 --2- 172.21.15.42:0/255951770 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562afb395070 0x562afb751dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:29.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.689+0000 7f22467fc640 1 -- 172.21.15.42:0/255951770 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562afb6608b0 con 0x562afb374390 2024-08-30T09:01:29.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.689+0000 7f22467fc640 1 --2- 172.21.15.42:0/255951770 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562afb374390 0x562afb745fe0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f223c0cbe00 tx=0x7f223c003820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3c944a5164e5f17f server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:01:29.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.689+0000 7f222aff5640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-30T09:01:29.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.690+0000 7f222aff5640 1 -- 172.21.15.42:0/255951770 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f223c0cb870 con 0x562afb374390 2024-08-30T09:01:29.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.690+0000 7f222aff5640 1 -- 172.21.15.42:0/255951770 <== mon.2 v2:172.21.15.167:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f223c0cc360 con 0x562afb374390 2024-08-30T09:01:29.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.690+0000 7f22538d5740 1 -- 172.21.15.42:0/255951770 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f22380043d0 con 0x562afb374390 2024-08-30T09:01:29.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.691+0000 7f222aff5640 1 -- 172.21.15.42:0/255951770 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f223c0cc5e0 con 0x562afb374390 2024-08-30T09:01:29.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.691+0000 7f22538d5740 10 client.25301 fetch_fsmap learned FSMap version 29 2024-08-30T09:01:29.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.691+0000 7f22538d5740 1 -- 172.21.15.42:0/255951770 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x562afb754870 con 0x562afb374390 2024-08-30T09:01:29.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.691+0000 7f222aff5640 1 -- 172.21.15.42:0/255951770 <== mon.2 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f22380043d0 con 0x562afb374390 2024-08-30T09:01:29.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.692+0000 7f222aff5640 1 -- 172.21.15.42:0/255951770 <== mon.2 v2:172.21.15.167:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f223c0d4f90 con 0x562afb374390 2024-08-30T09:01:29.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.693+0000 7f22538d5740 10 client.25301 fetch_fsmap finished waiting for FSMap version 29 2024-08-30T09:01:29.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.693+0000 7f22538d5740 10 client.25301 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-30T09:01:29.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.693+0000 7f22538d5740 20 client.25301 populate_metadata read hostname 'smithi042' 2024-08-30T09:01:29.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.693+0000 7f22538d5740 1 --2- 172.21.15.42:0/255951770 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x562afb3539f0 0x562afb757e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:01:29.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.693+0000 7f22538d5740 4 client.25301 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-30T09:01:29.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.693+0000 7f22538d5740 1 -- 172.21.15.42:0/255951770 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562afb6b3580 con 0x562afb3539f0 2024-08-30T09:01:29.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.693+0000 7f2246ffd640 1 --2- 172.21.15.42:0/255951770 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x562afb3539f0 0x562afb757e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:01:29.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.694+0000 7f222aff5640 10 client.25301 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:01:29.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.694+0000 7f2246ffd640 1 --2- 172.21.15.42:0/255951770 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x562afb3539f0 0x562afb757e90 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b6d49ad3a4c26f9 server_cookie=4734f15c2cd1283b in_seq=0 out_seq=0 2024-08-30T09:01:29.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.695+0000 7f222aff5640 1 -- 172.21.15.42:0/255951770 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x562afb6b3580 con 0x562afb3539f0 2024-08-30T09:01:29.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.695+0000 7f222aff5640 10 client.25301 handle_command_reply: tid=0 2024-08-30T09:01:29.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.695+0000 7f22538d5740 1 client.25301 shutdown 2024-08-30T09:01:29.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.696+0000 7f22538d5740 1 -- 172.21.15.42:0/255951770 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562afb374390 msgr2=0x562afb745fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:01:29.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.696+0000 7f22538d5740 1 --2- 172.21.15.42:0/255951770 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562afb374390 0x562afb745fe0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f223c0cbe00 tx=0x7f223c003820 comp rx=0 tx=0).stop 2024-08-30T09:01:29.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.696+0000 7f22538d5740 1 -- 172.21.15.42:0/255951770 shutdown_connections 2024-08-30T09:01:29.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.696+0000 7f22538d5740 1 --2- 172.21.15.42:0/255951770 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x562afb3539f0 0x562afb757e90 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:29.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.697+0000 7f22538d5740 1 --2- 172.21.15.42:0/255951770 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562afb395070 0x562afb751dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:29.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.697+0000 7f22538d5740 1 --2- 172.21.15.42:0/255951770 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562afb374390 0x562afb745fe0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:29.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.697+0000 7f22538d5740 1 --2- 172.21.15.42:0/255951770 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562afb36c1b0 0x562afb745aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:29.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.697+0000 7f22538d5740 1 -- 172.21.15.42:0/255951770 >> 172.21.15.42:0/255951770 conn(0x562afb389460 msgr2=0x562afb73d4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:01:29.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.697+0000 7f222aff5640 0 client.25301 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:01:29.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.697+0000 7f22538d5740 1 -- 172.21.15.42:0/255951770 shutdown_connections 2024-08-30T09:01:29.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.697+0000 7f22538d5740 1 -- 172.21.15.42:0/255951770 wait complete. 2024-08-30T09:01:29.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.697+0000 7f22538d5740 20 client.25301 trim_cache size 0 max 16384 2024-08-30T09:01:29.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.728+0000 7f22538d5740 1 Processor -- start 2024-08-30T09:01:29.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.728+0000 7f22538d5740 1 -- start start 2024-08-30T09:01:29.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.728+0000 7f22538d5740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562afb36c1b0 0x562afb7381e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:01:29.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.728+0000 7f22538d5740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562afb374390 0x562afb751cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:01:29.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.728+0000 7f22538d5740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562afb395070 0x562afb752200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:01:29.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.728+0000 7f22538d5740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f22481b0f60 con 0x562afb36c1b0 2024-08-30T09:01:29.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.728+0000 7f22538d5740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f22481ae830 con 0x562afb395070 2024-08-30T09:01:29.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.728+0000 7f22538d5740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f22481ac680 con 0x562afb374390 2024-08-30T09:01:29.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.728+0000 7f2246ffd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562afb36c1b0 0x562afb7381e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:01:29.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.728+0000 7f2246ffd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562afb36c1b0 0x562afb7381e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42444/0 (socket says 172.21.15.42:42444) 2024-08-30T09:01:29.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.728+0000 7f2246ffd640 1 -- 172.21.15.42:0/2321828874 learned_addr learned my addr 172.21.15.42:0/2321828874 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:01:29.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.728+0000 7f22477fe640 1 --2- 172.21.15.42:0/2321828874 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562afb395070 0x562afb752200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:01:29.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.729+0000 7f22467fc640 1 --2- 172.21.15.42:0/2321828874 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562afb374390 0x562afb751cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:01:29.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.729+0000 7f22467fc640 1 -- 172.21.15.42:0/2321828874 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562afb395070 msgr2=0x562afb752200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:01:29.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.729+0000 7f22467fc640 1 --2- 172.21.15.42:0/2321828874 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562afb395070 0x562afb752200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:29.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.729+0000 7f22467fc640 1 -- 172.21.15.42:0/2321828874 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562afb36c1b0 msgr2=0x562afb7381e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:01:29.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.729+0000 7f22467fc640 1 --2- 172.21.15.42:0/2321828874 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562afb36c1b0 0x562afb7381e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:29.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.729+0000 7f22467fc640 1 -- 172.21.15.42:0/2321828874 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562afb746d60 con 0x562afb374390 2024-08-30T09:01:29.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.730+0000 7f22467fc640 1 --2- 172.21.15.42:0/2321828874 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562afb374390 0x562afb751cc0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f223c0c8180 tx=0x7f223c0b7630 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2c57003356411a2b server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:01:29.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.730+0000 7f222cff9640 1 -- 172.21.15.42:0/2321828874 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f223c0cbed0 con 0x562afb374390 2024-08-30T09:01:29.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.730+0000 7f222cff9640 1 -- 172.21.15.42:0/2321828874 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f223c0cbc10 con 0x562afb374390 2024-08-30T09:01:29.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.730+0000 7f222cff9640 1 -- 172.21.15.42:0/2321828874 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f223c0cc360 con 0x562afb374390 2024-08-30T09:01:29.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.732+0000 7f22538d5740 1 -- 172.21.15.42:0/2321828874 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562afb374390 msgr2=0x562afb751cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:01:29.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.732+0000 7f22538d5740 1 --2- 172.21.15.42:0/2321828874 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562afb374390 0x562afb751cc0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f223c0c8180 tx=0x7f223c0b7630 comp rx=0 tx=0).stop 2024-08-30T09:01:29.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.732+0000 7f22538d5740 1 -- 172.21.15.42:0/2321828874 shutdown_connections 2024-08-30T09:01:29.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.732+0000 7f22538d5740 1 --2- 172.21.15.42:0/2321828874 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562afb374390 0x562afb751cc0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:29.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.732+0000 7f22538d5740 1 --2- 172.21.15.42:0/2321828874 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562afb395070 0x562afb752200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:29.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.732+0000 7f22538d5740 1 --2- 172.21.15.42:0/2321828874 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562afb36c1b0 0x562afb7381e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:29.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.732+0000 7f22538d5740 1 -- 172.21.15.42:0/2321828874 >> 172.21.15.42:0/2321828874 conn(0x562afb389460 msgr2=0x562afb737ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:01:29.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.732+0000 7f22538d5740 1 -- 172.21.15.42:0/2321828874 shutdown_connections 2024-08-30T09:01:29.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.732+0000 7f22538d5740 1 -- 172.21.15.42:0/2321828874 wait complete. 2024-08-30T09:01:29.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.733+0000 7f22538d5740 1 -- start start 2024-08-30T09:01:29.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.733+0000 7f22538d5740 1 Processor -- start 2024-08-30T09:01:29.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.734+0000 7f22538d5740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562afb36c1b0 0x562afb74a470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:01:29.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.734+0000 7f22538d5740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562afb374390 0x562afb74a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:01:29.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.734+0000 7f22538d5740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562afb395070 0x562afb7c1890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:01:29.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.734+0000 7f22538d5740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f22481accf0 con 0x562afb374390 2024-08-30T09:01:29.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.734+0000 7f22538d5740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f22481ae160 con 0x562afb395070 2024-08-30T09:01:29.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.734+0000 7f22538d5740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f22481ad3c0 con 0x562afb36c1b0 2024-08-30T09:01:29.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.734+0000 7f22538d5740 10 client.0 _ll_register_callbacks cb 0x7f223c003f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:01:29.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.734+0000 7f2246ffd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562afb36c1b0 0x562afb74a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:01:29.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.734+0000 7f22477fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562afb395070 0x562afb7c1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:01:29.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.734+0000 7f22477fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562afb395070 0x562afb7c1890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:41902/0 (socket says 172.21.15.42:41902) 2024-08-30T09:01:29.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.734+0000 7f22477fe640 1 -- 172.21.15.42:0/181135796 learned_addr learned my addr 172.21.15.42:0/181135796 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:01:29.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.735+0000 7f22467fc640 1 --2- 172.21.15.42:0/181135796 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562afb374390 0x562afb74a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:01:29.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.735+0000 7f22477fe640 1 -- 172.21.15.42:0/181135796 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562afb36c1b0 msgr2=0x562afb74a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:01:29.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.735+0000 7f22477fe640 1 --2- 172.21.15.42:0/181135796 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562afb36c1b0 0x562afb74a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:29.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.735+0000 7f22477fe640 1 -- 172.21.15.42:0/181135796 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562afb374390 msgr2=0x562afb74a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:01:29.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.735+0000 7f22477fe640 1 --2- 172.21.15.42:0/181135796 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562afb374390 0x562afb74a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:29.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.735+0000 7f22477fe640 1 -- 172.21.15.42:0/181135796 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562afb7c1e20 con 0x562afb395070 2024-08-30T09:01:29.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.735+0000 7f2246ffd640 1 --2- 172.21.15.42:0/181135796 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562afb36c1b0 0x562afb74a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-30T09:01:29.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.735+0000 7f22467fc640 1 --2- 172.21.15.42:0/181135796 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562afb374390 0x562afb74a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-30T09:01:29.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.735+0000 7f22477fe640 1 --2- 172.21.15.42:0/181135796 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562afb395070 0x562afb7c1890 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f2240011e10 tx=0x7f2240004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=97357df594ed701c server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:01:29.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.735+0000 7f222aff5640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-30T09:01:29.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.736+0000 7f222aff5640 1 -- 172.21.15.42:0/181135796 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2240011e40 con 0x562afb395070 2024-08-30T09:01:29.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.736+0000 7f222aff5640 1 -- 172.21.15.42:0/181135796 <== mon.2 v2:172.21.15.167:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f224001f430 con 0x562afb395070 2024-08-30T09:01:29.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.736+0000 7f22538d5740 1 -- 172.21.15.42:0/181135796 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f223c0cbc10 con 0x562afb395070 2024-08-30T09:01:29.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.736+0000 7f222aff5640 1 -- 172.21.15.42:0/181135796 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f224001f630 con 0x562afb395070 2024-08-30T09:01:29.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.737+0000 7f22538d5740 10 client.25307 fetch_fsmap learned FSMap version 29 2024-08-30T09:01:29.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.737+0000 7f22538d5740 1 -- 172.21.15.42:0/181135796 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x562afb7c30f0 con 0x562afb395070 2024-08-30T09:01:29.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.737+0000 7f222aff5640 1 -- 172.21.15.42:0/181135796 <== mon.2 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f224001fc80 con 0x562afb395070 2024-08-30T09:01:29.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.737+0000 7f222aff5640 1 -- 172.21.15.42:0/181135796 <== mon.2 v2:172.21.15.167:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f2240032c00 con 0x562afb395070 2024-08-30T09:01:29.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.737+0000 7f22538d5740 10 client.25307 fetch_fsmap finished waiting for FSMap version 29 2024-08-30T09:01:29.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.737+0000 7f22538d5740 10 client.25307 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-30T09:01:29.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.737+0000 7f22538d5740 20 client.25307 populate_metadata read hostname 'smithi042' 2024-08-30T09:01:29.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.737+0000 7f22538d5740 1 --2- 172.21.15.42:0/181135796 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x562afb328da0 0x562afb7c5920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:01:29.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.738+0000 7f22538d5740 4 client.25307 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-30T09:01:29.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.738+0000 7f22538d5740 1 -- 172.21.15.42:0/181135796 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f223c0cbc10 con 0x562afb328da0 2024-08-30T09:01:29.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.738+0000 7f2246ffd640 1 --2- 172.21.15.42:0/181135796 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x562afb328da0 0x562afb7c5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:01:29.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.738+0000 7f222aff5640 10 client.25307 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:01:29.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.740+0000 7f2246ffd640 1 --2- 172.21.15.42:0/181135796 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x562afb328da0 0x562afb7c5920 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7c7b249eff57bae4 server_cookie=fbc4488ab7dfed91 in_seq=0 out_seq=0 2024-08-30T09:01:29.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.741+0000 7f222aff5640 1 -- 172.21.15.42:0/181135796 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+854 (crc 0 0 0) 0x7f223c0cbc10 con 0x562afb328da0 2024-08-30T09:01:29.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.741+0000 7f222aff5640 10 client.25307 handle_command_reply: tid=0 2024-08-30T09:01:29.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.741+0000 7f22538d5740 1 client.25307 shutdown 2024-08-30T09:01:29.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.741+0000 7f22538d5740 1 -- 172.21.15.42:0/181135796 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562afb395070 msgr2=0x562afb7c1890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:01:29.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.742+0000 7f22538d5740 1 --2- 172.21.15.42:0/181135796 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562afb395070 0x562afb7c1890 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f2240011e10 tx=0x7f2240004ab0 comp rx=0 tx=0).stop 2024-08-30T09:01:29.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.742+0000 7f22538d5740 1 -- 172.21.15.42:0/181135796 shutdown_connections 2024-08-30T09:01:29.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.742+0000 7f22538d5740 1 --2- 172.21.15.42:0/181135796 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x562afb328da0 0x562afb7c5920 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:29.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.742+0000 7f22538d5740 1 --2- 172.21.15.42:0/181135796 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562afb395070 0x562afb7c1890 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:29.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.742+0000 7f222aff5640 0 client.25307 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:01:29.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.742+0000 7f22538d5740 1 --2- 172.21.15.42:0/181135796 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562afb374390 0x562afb74a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:29.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.742+0000 7f22538d5740 1 --2- 172.21.15.42:0/181135796 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562afb36c1b0 0x562afb74a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:29.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.742+0000 7f22538d5740 1 -- 172.21.15.42:0/181135796 >> 172.21.15.42:0/181135796 conn(0x562afb389460 msgr2=0x562afb73d3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:01:29.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.742+0000 7f22538d5740 1 -- 172.21.15.42:0/181135796 shutdown_connections 2024-08-30T09:01:29.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.742+0000 7f22538d5740 1 -- 172.21.15.42:0/181135796 wait complete. 2024-08-30T09:01:29.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.742+0000 7f22538d5740 20 client.25307 trim_cache size 0 max 16384 2024-08-30T09:01:29.746 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-08-30T09:01:29.747 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "IDLE (waiting for more scrubs)", 2024-08-30T09:01:29.747 INFO:teuthology.orchestra.run.smithi042.stdout: "scrubs": { 2024-08-30T09:01:29.747 INFO:teuthology.orchestra.run.smithi042.stdout: "85134e4b-ab4a-4885-b07e-a3ad01eaf13d": { 2024-08-30T09:01:29.747 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:01:29.747 INFO:teuthology.orchestra.run.smithi042.stdout: "tag": "85134e4b-ab4a-4885-b07e-a3ad01eaf13d", 2024-08-30T09:01:29.747 INFO:teuthology.orchestra.run.smithi042.stdout: "options": "recursive,force" 2024-08-30T09:01:29.747 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:01:29.747 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:01:29.747 INFO:teuthology.orchestra.run.smithi042.stdout: "scrub_stats": { 2024-08-30T09:01:29.747 INFO:teuthology.orchestra.run.smithi042.stdout: "85134e4b-ab4a-4885-b07e-a3ad01eaf13d": { 2024-08-30T09:01:29.747 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:01:27.738462+0000", 2024-08-30T09:01:29.747 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:01:29.747 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:01:29.748 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:01:29.748 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:01:29.748 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:01:29.748 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:01:29.748 INFO:teuthology.orchestra.run.smithi042.stdout: "5e2ed104-0625-478a-a78b-584e7f4c2817": { 2024-08-30T09:01:29.748 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:00:33.928400+0000", 2024-08-30T09:01:29.748 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:01:29.748 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:01:29.748 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:01:29.748 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:01:29.748 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:01:29.748 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:01:29.748 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:01:29.749 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-08-30T09:01:29.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.744+0000 7f224d360640 1 -- 172.21.15.42:0/416630451 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f22300bd3c0 msgr2=0x7f22300bf8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:01:29.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.744+0000 7f224d360640 1 --2- 172.21.15.42:0/416630451 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f22300bd3c0 0x7f22300bf8d0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f2240008dc0 tx=0x7f2240008370 comp rx=0 tx=0).stop 2024-08-30T09:01:29.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.744+0000 7f224d360640 1 -- 172.21.15.42:0/416630451 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2248109250 msgr2=0x7f224819a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:01:29.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.744+0000 7f224d360640 1 --2- 172.21.15.42:0/416630451 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2248109250 0x7f224819a830 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f223c006010 tx=0x7f223c004060 comp rx=0 tx=0).stop 2024-08-30T09:01:29.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.744+0000 7f224d360640 1 -- 172.21.15.42:0/416630451 shutdown_connections 2024-08-30T09:01:29.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.744+0000 7f224d360640 1 --2- 172.21.15.42:0/416630451 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f22300bd3c0 0x7f22300bf8d0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:29.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.744+0000 7f224d360640 1 --2- 172.21.15.42:0/416630451 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2248109ba0 0x7f224819fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:29.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.744+0000 7f224d360640 1 --2- 172.21.15.42:0/416630451 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2248109250 0x7f224819a830 secure :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f223c006010 tx=0x7f223c004060 comp rx=0 tx=0).stop 2024-08-30T09:01:29.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.744+0000 7f224d360640 1 --2- 172.21.15.42:0/416630451 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2248103910 0x7f224819a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:29.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.744+0000 7f224d360640 1 -- 172.21.15.42:0/416630451 >> 172.21.15.42:0/416630451 conn(0x7f22480ff380 msgr2=0x7f22481017a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:01:29.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.745+0000 7f224d360640 1 -- 172.21.15.42:0/416630451 shutdown_connections 2024-08-30T09:01:29.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:01:29.745+0000 7f224d360640 1 -- 172.21.15.42:0/416630451 wait complete. 2024-08-30T09:01:29.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:01:29 smithi167 ceph-mon[22063]: scrub queued for path: / 2024-08-30T09:01:29.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:01:29 smithi167 ceph-mon[22063]: scrub summary: idle+waiting paths [/] 2024-08-30T09:01:29.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:01:29 smithi167 ceph-mon[22063]: scrub summary: active paths [/] 2024-08-30T09:01:29.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:01:29 smithi167 ceph-mon[22063]: scrub summary: active paths [/] 2024-08-30T09:01:29.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:01:29 smithi167 ceph-mon[22063]: scrub summary: idle+waiting paths [/] 2024-08-30T09:01:29.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:01:29 smithi042 ceph-mon[18652]: scrub queued for path: / 2024-08-30T09:01:29.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:01:29 smithi042 ceph-mon[18652]: scrub summary: idle+waiting paths [/] 2024-08-30T09:01:29.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:01:29 smithi042 ceph-mon[18652]: scrub summary: active paths [/] 2024-08-30T09:01:29.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:01:29 smithi042 ceph-mon[18652]: scrub summary: active paths [/] 2024-08-30T09:01:29.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:01:29 smithi042 ceph-mon[18652]: scrub summary: idle+waiting paths [/] 2024-08-30T09:01:29.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:01:29 smithi190 ceph-mon[22001]: scrub queued for path: / 2024-08-30T09:01:29.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:01:29 smithi190 ceph-mon[22001]: scrub summary: idle+waiting paths [/] 2024-08-30T09:01:29.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:01:29 smithi190 ceph-mon[22001]: scrub summary: active paths [/] 2024-08-30T09:01:29.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:01:29 smithi190 ceph-mon[22001]: scrub summary: active paths [/] 2024-08-30T09:01:29.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:01:29 smithi190 ceph-mon[22001]: scrub summary: idle+waiting paths [/] 2024-08-30T09:01:30.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:01:30 smithi167 ceph-mon[22063]: scrub summary: idle+waiting paths [/] 2024-08-30T09:01:30.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:01:30 smithi167 ceph-mon[22063]: pgmap v395: 129 pgs: 129 active+clean; 981 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 16 MiB/s wr, 951 op/s 2024-08-30T09:01:30.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:01:30 smithi042 ceph-mon[18652]: scrub summary: idle+waiting paths [/] 2024-08-30T09:01:30.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:01:30 smithi042 ceph-mon[18652]: pgmap v395: 129 pgs: 129 active+clean; 981 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 16 MiB/s wr, 951 op/s 2024-08-30T09:01:30.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:01:30 smithi190 ceph-mon[22001]: scrub summary: idle+waiting paths [/] 2024-08-30T09:01:30.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:01:30 smithi190 ceph-mon[22001]: pgmap v395: 129 pgs: 129 active+clean; 981 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 16 MiB/s wr, 951 op/s 2024-08-30T09:01:30.953 INFO:teuthology.orchestra.run.smithi042.stderr:progress: 45.0 s, 2245.6 tps, lat 49.307 ms stddev 91.166 2024-08-30T09:01:32.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:01:32 smithi167 ceph-mon[22063]: pgmap v396: 129 pgs: 129 active+clean; 981 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 17 MiB/s wr, 962 op/s 2024-08-30T09:01:32.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:01:32 smithi042 ceph-mon[18652]: pgmap v396: 129 pgs: 129 active+clean; 981 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 17 MiB/s wr, 962 op/s 2024-08-30T09:01:32.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:01:32 smithi190 ceph-mon[22001]: pgmap v396: 129 pgs: 129 active+clean; 981 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 17 MiB/s wr, 962 op/s 2024-08-30T09:01:33.605 INFO:tasks.cephfs.filesystem:scrub status for tag:85134e4b-ab4a-4885-b07e-a3ad01eaf13d - {'path': '/', 'tag': '85134e4b-ab4a-4885-b07e-a3ad01eaf13d', 'options': 'recursive,force'} 2024-08-30T09:01:34.826 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:01:34 smithi167 ceph-mon[22063]: pgmap v397: 129 pgs: 129 active+clean; 994 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 19 MiB/s wr, 1.21k op/s 2024-08-30T09:01:34.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:01:34 smithi042 ceph-mon[18652]: pgmap v397: 129 pgs: 129 active+clean; 994 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 19 MiB/s wr, 1.21k op/s 2024-08-30T09:01:34.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:01:34 smithi190 ceph-mon[22001]: pgmap v397: 129 pgs: 129 active+clean; 994 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 19 MiB/s wr, 1.21k op/s 2024-08-30T09:01:35.953 INFO:teuthology.orchestra.run.smithi042.stderr:progress: 50.0 s, 2272.4 tps, lat 44.272 ms stddev 49.855 2024-08-30T09:01:36.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:01:36 smithi167 ceph-mon[22063]: pgmap v398: 129 pgs: 129 active+clean; 998 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 17 MiB/s wr, 1.04k op/s 2024-08-30T09:01:36.882 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:01:36 smithi042 ceph-mon[18652]: pgmap v398: 129 pgs: 129 active+clean; 998 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 17 MiB/s wr, 1.04k op/s 2024-08-30T09:01:36.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:01:36 smithi190 ceph-mon[22001]: pgmap v398: 129 pgs: 129 active+clean; 998 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 17 MiB/s wr, 1.04k op/s 2024-08-30T09:01:38.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:01:38 smithi167 ceph-mon[22063]: pgmap v399: 129 pgs: 129 active+clean; 998 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 19 MiB/s wr, 1.14k op/s 2024-08-30T09:01:38.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:01:38 smithi167 ceph-mon[22063]: scrub summary: idle 2024-08-30T09:01:38.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:01:38 smithi167 ceph-mon[22063]: scrub summary: idle 2024-08-30T09:01:38.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:01:38 smithi167 ceph-mon[22063]: scrub summary: idle 2024-08-30T09:01:38.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:01:38 smithi042 ceph-mon[18652]: pgmap v399: 129 pgs: 129 active+clean; 998 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 19 MiB/s wr, 1.14k op/s 2024-08-30T09:01:38.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:01:38 smithi042 ceph-mon[18652]: scrub summary: idle 2024-08-30T09:01:38.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:01:38 smithi042 ceph-mon[18652]: scrub summary: idle 2024-08-30T09:01:38.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:01:38 smithi042 ceph-mon[18652]: scrub summary: idle 2024-08-30T09:01:38.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:01:38 smithi190 ceph-mon[22001]: pgmap v399: 129 pgs: 129 active+clean; 998 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 19 MiB/s wr, 1.14k op/s 2024-08-30T09:01:38.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:01:38 smithi190 ceph-mon[22001]: scrub summary: idle 2024-08-30T09:01:38.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:01:38 smithi190 ceph-mon[22001]: scrub summary: idle 2024-08-30T09:01:38.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:01:38 smithi190 ceph-mon[22001]: scrub summary: idle 2024-08-30T09:01:39.464 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.462+0000 7f23aa27e640 1 -- 172.21.15.42:0/2693858759 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f23a41033f0 msgr2=0x7f23a4103800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:01:39.464 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.462+0000 7f23aa27e640 1 --2- 172.21.15.42:0/2693858759 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f23a41033f0 0x7f23a4103800 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f238c009900 tx=0x7f238c02a400 comp rx=0 tx=0).stop 2024-08-30T09:01:39.468 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.466+0000 7f23aa27e640 1 -- 172.21.15.42:0/2693858759 shutdown_connections 2024-08-30T09:01:39.468 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.466+0000 7f23aa27e640 1 --2- 172.21.15.42:0/2693858759 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f23a4103d90 0x7f23a410a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:39.468 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.466+0000 7f23aa27e640 1 --2- 172.21.15.42:0/2693858759 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f23a41033f0 0x7f23a4103800 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:39.468 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.466+0000 7f23aa27e640 1 --2- 172.21.15.42:0/2693858759 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f23a4102370 0x7f23a41027e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:39.468 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.466+0000 7f23aa27e640 1 -- 172.21.15.42:0/2693858759 >> 172.21.15.42:0/2693858759 conn(0x7f23a40fdcd0 msgr2=0x7f23a4100160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:01:39.469 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.467+0000 7f23aa27e640 1 -- 172.21.15.42:0/2693858759 shutdown_connections 2024-08-30T09:01:39.472 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.470+0000 7f23aa27e640 1 -- 172.21.15.42:0/2693858759 wait complete. 2024-08-30T09:01:39.473 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.471+0000 7f23aa27e640 1 Processor -- start 2024-08-30T09:01:39.479 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.476+0000 7f23aa27e640 1 -- start start 2024-08-30T09:01:39.479 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.477+0000 7f23aa27e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f23a4102370 0x7f23a4113120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:01:39.479 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.477+0000 7f23aa27e640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f23a41033f0 0x7f23a4113790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:01:39.479 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.477+0000 7f23aa27e640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f23a4103d90 0x7f23a4111bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:01:39.479 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.477+0000 7f23aa27e640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f23a410bdd0 con 0x7f23a41033f0 2024-08-30T09:01:39.479 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.477+0000 7f23aa27e640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f23a410bc50 con 0x7f23a4103d90 2024-08-30T09:01:39.479 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.477+0000 7f23aa27e640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f23a410bf50 con 0x7f23a4102370 2024-08-30T09:01:39.479 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.477+0000 7f23a37fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f23a41033f0 0x7f23a4113790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:01:39.479 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.477+0000 7f23a37fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f23a41033f0 0x7f23a4113790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42016/0 (socket says 172.21.15.42:42016) 2024-08-30T09:01:39.479 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.477+0000 7f23a37fe640 1 -- 172.21.15.42:0/3896396753 learned_addr learned my addr 172.21.15.42:0/3896396753 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:01:39.480 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.478+0000 7f23a37fe640 1 -- 172.21.15.42:0/3896396753 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f23a4102370 msgr2=0x7f23a4113120 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-08-30T09:01:39.480 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.478+0000 7f23a37fe640 1 --2- 172.21.15.42:0/3896396753 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f23a4102370 0x7f23a4113120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:39.480 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.478+0000 7f23a37fe640 1 -- 172.21.15.42:0/3896396753 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f23a4103d90 msgr2=0x7f23a4111bc0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-08-30T09:01:39.480 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.478+0000 7f23a37fe640 1 --2- 172.21.15.42:0/3896396753 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f23a4103d90 0x7f23a4111bc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:39.480 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.478+0000 7f23a37fe640 1 -- 172.21.15.42:0/3896396753 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23a41123e0 con 0x7f23a41033f0 2024-08-30T09:01:39.480 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.478+0000 7f23a37fe640 1 --2- 172.21.15.42:0/3896396753 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f23a41033f0 0x7f23a4113790 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f238c031440 tx=0x7f238c003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:01:39.482 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.479+0000 7f23a17fa640 1 -- 172.21.15.42:0/3896396753 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f238c03f070 con 0x7f23a41033f0 2024-08-30T09:01:39.482 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.480+0000 7f23a17fa640 1 -- 172.21.15.42:0/3896396753 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f238c031c50 con 0x7f23a41033f0 2024-08-30T09:01:39.482 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.480+0000 7f23a17fa640 1 -- 172.21.15.42:0/3896396753 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f238c035030 con 0x7f23a41033f0 2024-08-30T09:01:39.482 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.480+0000 7f23aa27e640 1 -- 172.21.15.42:0/3896396753 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23a4112670 con 0x7f23a41033f0 2024-08-30T09:01:39.482 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.480+0000 7f23aa27e640 1 -- 172.21.15.42:0/3896396753 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23a41b0db0 con 0x7f23a41033f0 2024-08-30T09:01:39.492 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.490+0000 7f23a17fa640 1 -- 172.21.15.42:0/3896396753 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7f238c042070 con 0x7f23a41033f0 2024-08-30T09:01:39.493 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.491+0000 7f23a17fa640 1 --2- 172.21.15.42:0/3896396753 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f23900b4d40 0x7f23900b7250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:01:39.493 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.491+0000 7f23a3fff640 1 --2- 172.21.15.42:0/3896396753 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f23900b4d40 0x7f23900b7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:01:39.493 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.491+0000 7f23a17fa640 1 -- 172.21.15.42:0/3896396753 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f238c105d70 con 0x7f23a41033f0 2024-08-30T09:01:39.497 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.493+0000 7f23b11e3740 1 Processor -- start 2024-08-30T09:01:39.497 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.494+0000 7f23b11e3740 1 -- start start 2024-08-30T09:01:39.497 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.494+0000 7f23b11e3740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5600fa2351b0 0x5600fa5db180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:01:39.497 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.494+0000 7f23b11e3740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5600fa20c2f0 0x5600fa5db7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:01:39.498 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.495+0000 7f23a3fff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5600fa2351b0 0x5600fa5db180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:01:39.498 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.495+0000 7f23a3fff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5600fa2351b0 0x5600fa5db180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:33798/0 (socket says 172.21.15.42:33798) 2024-08-30T09:01:39.498 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.495+0000 7f23b11e3740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5600fa2144d0 0x5600fa5dc340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:01:39.498 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.495+0000 7f23a3fff640 1 -- 172.21.15.42:0/125387091 learned_addr learned my addr 172.21.15.42:0/125387091 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:01:39.498 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.495+0000 7f23b11e3740 1 -- 172.21.15.42:0/125387091 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x5600fa54e000 con 0x5600fa20c2f0 2024-08-30T09:01:39.498 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.495+0000 7f23b11e3740 1 -- 172.21.15.42:0/125387091 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5600fa53d590 con 0x5600fa2351b0 2024-08-30T09:01:39.498 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.495+0000 7f23b11e3740 1 -- 172.21.15.42:0/125387091 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5600fa541540 con 0x5600fa2144d0 2024-08-30T09:01:39.498 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.496+0000 7f23a37fe640 1 --2- 172.21.15.42:0/125387091 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5600fa20c2f0 0x5600fa5db7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:01:39.498 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.496+0000 7f23a3fff640 1 -- 172.21.15.42:0/125387091 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5600fa2144d0 msgr2=0x5600fa5dc340 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-30T09:01:39.499 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.496+0000 7f23a3fff640 1 --2- 172.21.15.42:0/125387091 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5600fa2144d0 0x5600fa5dc340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:39.499 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.496+0000 7f23a3fff640 1 -- 172.21.15.42:0/125387091 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5600fa20c2f0 msgr2=0x5600fa5db7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:01:39.499 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.496+0000 7f23a3fff640 1 --2- 172.21.15.42:0/125387091 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5600fa20c2f0 0x5600fa5db7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:39.499 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.496+0000 7f23a3fff640 1 -- 172.21.15.42:0/125387091 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5600fa2a0790 con 0x5600fa2351b0 2024-08-30T09:01:39.499 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.497+0000 7f23a3fff640 1 --2- 172.21.15.42:0/125387091 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5600fa2351b0 0x5600fa5db180 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f2394009560 tx=0x7f239401b000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1cb698043c929de8 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:01:39.499 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.497+0000 7f23a3fff640 1 --2- 172.21.15.42:0/3896396753 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f23900b4d40 0x7f23900b7250 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f2394009a60 tx=0x7f2394005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:01:39.501 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.498+0000 7f239a7fc640 1 -- 172.21.15.42:0/125387091 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2394020070 con 0x5600fa2351b0 2024-08-30T09:01:39.501 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.498+0000 7f239a7fc640 1 -- 172.21.15.42:0/125387091 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2394025460 con 0x5600fa2351b0 2024-08-30T09:01:39.501 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.498+0000 7f239a7fc640 1 -- 172.21.15.42:0/125387091 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f239401c660 con 0x5600fa2351b0 2024-08-30T09:01:39.502 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.500+0000 7f23b11e3740 1 -- 172.21.15.42:0/125387091 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5600fa2351b0 msgr2=0x5600fa5db180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:01:39.502 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.500+0000 7f23b11e3740 1 --2- 172.21.15.42:0/125387091 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5600fa2351b0 0x5600fa5db180 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f2394009560 tx=0x7f239401b000 comp rx=0 tx=0).stop 2024-08-30T09:01:39.503 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.501+0000 7f23b11e3740 1 -- 172.21.15.42:0/125387091 shutdown_connections 2024-08-30T09:01:39.503 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.501+0000 7f23b11e3740 1 --2- 172.21.15.42:0/125387091 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5600fa2144d0 0x5600fa5dc340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:39.503 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.501+0000 7f23b11e3740 1 --2- 172.21.15.42:0/125387091 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5600fa20c2f0 0x5600fa5db7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:39.503 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.501+0000 7f23b11e3740 1 --2- 172.21.15.42:0/125387091 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5600fa2351b0 0x5600fa5db180 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:39.503 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.501+0000 7f23b11e3740 1 -- 172.21.15.42:0/125387091 >> 172.21.15.42:0/125387091 conn(0x5600fa229300 msgr2=0x5600fa5d6c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:01:39.514 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.513+0000 7f23b11e3740 1 -- 172.21.15.42:0/125387091 shutdown_connections 2024-08-30T09:01:39.518 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.516+0000 7f23b11e3740 1 -- 172.21.15.42:0/125387091 wait complete. 2024-08-30T09:01:39.519 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.518+0000 7f23b11e3740 1 -- start start 2024-08-30T09:01:39.520 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.518+0000 7f23b11e3740 1 Processor -- start 2024-08-30T09:01:39.525 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.522+0000 7f23b11e3740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5600fa20c2f0 0x5600fa5e4c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:01:39.525 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.523+0000 7f23b11e3740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5600fa2144d0 0x5600fa5e5330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:01:39.525 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.523+0000 7f23b11e3740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5600fa2351b0 0x5600fa5e5ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:01:39.525 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.523+0000 7f23a3fff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5600fa20c2f0 0x5600fa5e4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:01:39.525 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.523+0000 7f23a3fff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5600fa20c2f0 0x5600fa5e4c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:47870/0 (socket says 172.21.15.42:47870) 2024-08-30T09:01:39.525 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.523+0000 7f23a3fff640 1 -- 172.21.15.42:0/3255856524 learned_addr learned my addr 172.21.15.42:0/3255856524 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:01:39.526 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.523+0000 7f23a37fe640 1 --2- 172.21.15.42:0/3255856524 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5600fa2144d0 0x5600fa5e5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:01:39.526 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.523+0000 7f23b11e3740 1 -- 172.21.15.42:0/3255856524 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x5600fa476ab0 con 0x5600fa2351b0 2024-08-30T09:01:39.526 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.523+0000 7f23b11e3740 1 -- 172.21.15.42:0/3255856524 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5600fa4c4680 con 0x5600fa2144d0 2024-08-30T09:01:39.526 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.523+0000 7f23b11e3740 1 -- 172.21.15.42:0/3255856524 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5600fa5dce60 con 0x5600fa20c2f0 2024-08-30T09:01:39.526 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.523+0000 7f23a881b640 1 --2- 172.21.15.42:0/3255856524 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5600fa2351b0 0x5600fa5e5ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:01:39.526 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.523+0000 7f23b11e3740 10 client.0 _ll_register_callbacks cb 0x5600fa24ca00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:01:39.527 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.524+0000 7f23a881b640 1 -- 172.21.15.42:0/3255856524 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5600fa20c2f0 msgr2=0x5600fa5e4c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:01:39.527 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.524+0000 7f23a881b640 1 --2- 172.21.15.42:0/3255856524 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5600fa20c2f0 0x5600fa5e4c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:39.527 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.525+0000 7f23a881b640 1 -- 172.21.15.42:0/3255856524 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5600fa2144d0 msgr2=0x5600fa5e5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:01:39.527 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.525+0000 7f23a881b640 1 --2- 172.21.15.42:0/3255856524 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5600fa2144d0 0x5600fa5e5330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:39.527 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.525+0000 7f23a881b640 1 -- 172.21.15.42:0/3255856524 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5600fa319620 con 0x5600fa2351b0 2024-08-30T09:01:39.528 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.526+0000 7f23a881b640 1 --2- 172.21.15.42:0/3255856524 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5600fa2351b0 0x5600fa5e5ea0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f239c00e640 tx=0x7f239c00eb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca3ecc8218d10385 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:01:39.528 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.526+0000 7f23837fe640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-08-30T09:01:39.529 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.527+0000 7f23837fe640 1 -- 172.21.15.42:0/3255856524 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f239c004d20 con 0x5600fa2351b0 2024-08-30T09:01:39.529 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.527+0000 7f23b11e3740 1 -- 172.21.15.42:0/3255856524 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2394025460 con 0x5600fa2351b0 2024-08-30T09:01:39.529 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.527+0000 7f23837fe640 1 -- 172.21.15.42:0/3255856524 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f239c00c790 con 0x5600fa2351b0 2024-08-30T09:01:39.529 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.527+0000 7f23837fe640 1 -- 172.21.15.42:0/3255856524 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f239c00c990 con 0x5600fa2351b0 2024-08-30T09:01:39.530 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.527+0000 7f23b11e3740 10 client.15672 fetch_fsmap learned FSMap version 29 2024-08-30T09:01:39.530 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.527+0000 7f23b11e3740 1 -- 172.21.15.42:0/3255856524 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5600fa5eef10 con 0x5600fa2351b0 2024-08-30T09:01:39.530 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.528+0000 7f23837fe640 1 -- 172.21.15.42:0/3255856524 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2394025460 con 0x5600fa2351b0 2024-08-30T09:01:39.530 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.528+0000 7f23837fe640 1 -- 172.21.15.42:0/3255856524 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f239c018470 con 0x5600fa2351b0 2024-08-30T09:01:39.531 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.529+0000 7f23b11e3740 10 client.15672 fetch_fsmap finished waiting for FSMap version 29 2024-08-30T09:01:39.531 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.529+0000 7f23b11e3740 10 client.15672 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-30T09:01:39.531 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.529+0000 7f23b11e3740 20 client.15672 populate_metadata read hostname 'smithi042' 2024-08-30T09:01:39.531 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.529+0000 7f23b11e3740 1 --2- 172.21.15.42:0/3255856524 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x5600fa1f3d80 0x5600fa5f2660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:01:39.531 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.529+0000 7f23b11e3740 4 client.15672 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-30T09:01:39.531 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.529+0000 7f23b11e3740 1 -- 172.21.15.42:0/3255856524 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5600fa54ff50 con 0x5600fa1f3d80 2024-08-30T09:01:39.531 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.530+0000 7f23a3fff640 1 --2- 172.21.15.42:0/3255856524 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x5600fa1f3d80 0x5600fa5f2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:01:39.532 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.531+0000 7f23a3fff640 1 --2- 172.21.15.42:0/3255856524 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x5600fa1f3d80 0x5600fa5f2660 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4e8280c979a1ade0 server_cookie=57aa9581a1d21e1a in_seq=0 out_seq=0 2024-08-30T09:01:39.533 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.531+0000 7f23837fe640 10 client.15672 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:01:39.534 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.532+0000 7f23837fe640 1 -- 172.21.15.42:0/3255856524 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5600fa54ff50 con 0x5600fa1f3d80 2024-08-30T09:01:39.534 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.532+0000 7f23837fe640 10 client.15672 handle_command_reply: tid=0 2024-08-30T09:01:39.534 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.532+0000 7f23b11e3740 1 client.15672 shutdown 2024-08-30T09:01:39.534 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.532+0000 7f23b11e3740 1 -- 172.21.15.42:0/3255856524 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5600fa2351b0 msgr2=0x5600fa5e5ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:01:39.534 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.532+0000 7f23b11e3740 1 --2- 172.21.15.42:0/3255856524 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5600fa2351b0 0x5600fa5e5ea0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f239c00e640 tx=0x7f239c00eb00 comp rx=0 tx=0).stop 2024-08-30T09:01:39.534 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.532+0000 7f23b11e3740 1 -- 172.21.15.42:0/3255856524 shutdown_connections 2024-08-30T09:01:39.534 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.532+0000 7f23b11e3740 1 --2- 172.21.15.42:0/3255856524 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x5600fa1f3d80 0x5600fa5f2660 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:39.535 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.533+0000 7f23b11e3740 1 --2- 172.21.15.42:0/3255856524 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5600fa2351b0 0x5600fa5e5ea0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:39.535 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.533+0000 7f23837fe640 0 client.15672 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:01:39.535 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.533+0000 7f23b11e3740 1 --2- 172.21.15.42:0/3255856524 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5600fa2144d0 0x5600fa5e5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:39.535 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.533+0000 7f23b11e3740 1 --2- 172.21.15.42:0/3255856524 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5600fa20c2f0 0x5600fa5e4c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:39.535 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.533+0000 7f23b11e3740 1 -- 172.21.15.42:0/3255856524 >> 172.21.15.42:0/3255856524 conn(0x5600fa229300 msgr2=0x5600fa5d5170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:01:39.535 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.533+0000 7f23b11e3740 1 -- 172.21.15.42:0/3255856524 shutdown_connections 2024-08-30T09:01:39.536 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.534+0000 7f23b11e3740 1 -- 172.21.15.42:0/3255856524 wait complete. 2024-08-30T09:01:39.536 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.534+0000 7f23b11e3740 20 client.15672 trim_cache size 0 max 16384 2024-08-30T09:01:39.565 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.561+0000 7f23b11e3740 1 Processor -- start 2024-08-30T09:01:39.565 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.561+0000 7f23b11e3740 1 -- start start 2024-08-30T09:01:39.565 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.561+0000 7f23b11e3740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5600fa20c2f0 0x5600fa5d5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:01:39.566 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.561+0000 7f23b11e3740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5600fa2144d0 0x5600fa5d5ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:01:39.566 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.561+0000 7f23b11e3740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5600fa2351b0 0x5600fa5e3180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:01:39.566 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.561+0000 7f23b11e3740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f23a41bdb20 con 0x5600fa20c2f0 2024-08-30T09:01:39.566 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.561+0000 7f23b11e3740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f23a41bb3f0 con 0x5600fa2351b0 2024-08-30T09:01:39.566 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.561+0000 7f23b11e3740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f23a41b9240 con 0x5600fa2144d0 2024-08-30T09:01:39.566 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.561+0000 7f23a37fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5600fa2144d0 0x5600fa5d5ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:01:39.566 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.562+0000 7f23a881b640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5600fa2351b0 0x5600fa5e3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:01:39.566 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.562+0000 7f23a3fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5600fa20c2f0 0x5600fa5d5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:01:39.566 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.562+0000 7f23a37fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5600fa2144d0 0x5600fa5d5ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:47876/0 (socket says 172.21.15.42:47876) 2024-08-30T09:01:39.566 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.562+0000 7f23a37fe640 1 -- 172.21.15.42:0/2848873904 learned_addr learned my addr 172.21.15.42:0/2848873904 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:01:39.566 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.562+0000 7f23a37fe640 1 -- 172.21.15.42:0/2848873904 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5600fa2351b0 msgr2=0x5600fa5e3180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:01:39.566 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.562+0000 7f23a37fe640 1 --2- 172.21.15.42:0/2848873904 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5600fa2351b0 0x5600fa5e3180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:39.566 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.562+0000 7f23a37fe640 1 -- 172.21.15.42:0/2848873904 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5600fa20c2f0 msgr2=0x5600fa5d5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:01:39.567 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.562+0000 7f23a37fe640 1 --2- 172.21.15.42:0/2848873904 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5600fa20c2f0 0x5600fa5d5980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:39.567 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.562+0000 7f23a37fe640 1 -- 172.21.15.42:0/2848873904 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5600fa5ed140 con 0x5600fa2144d0 2024-08-30T09:01:39.567 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.562+0000 7f23a3fff640 1 --2- 172.21.15.42:0/2848873904 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5600fa20c2f0 0x5600fa5d5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-30T09:01:39.567 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.563+0000 7f23a37fe640 1 --2- 172.21.15.42:0/2848873904 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5600fa2144d0 0x5600fa5d5ec0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f238c0c9590 tx=0x7f238c000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eb78bf105ab06ff1 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:01:39.567 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.563+0000 7f2399ffb640 1 -- 172.21.15.42:0/2848873904 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f238c049040 con 0x5600fa2144d0 2024-08-30T09:01:39.567 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.563+0000 7f2399ffb640 1 -- 172.21.15.42:0/2848873904 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f238c0ccab0 con 0x5600fa2144d0 2024-08-30T09:01:39.567 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.563+0000 7f2399ffb640 1 -- 172.21.15.42:0/2848873904 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f238c0c85b0 con 0x5600fa2144d0 2024-08-30T09:01:39.569 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.564+0000 7f23b11e3740 1 -- 172.21.15.42:0/2848873904 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5600fa2144d0 msgr2=0x5600fa5d5ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:01:39.569 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.564+0000 7f23b11e3740 1 --2- 172.21.15.42:0/2848873904 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5600fa2144d0 0x5600fa5d5ec0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f238c0c9590 tx=0x7f238c000bc0 comp rx=0 tx=0).stop 2024-08-30T09:01:39.569 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.564+0000 7f23b11e3740 1 -- 172.21.15.42:0/2848873904 shutdown_connections 2024-08-30T09:01:39.569 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.564+0000 7f23b11e3740 1 --2- 172.21.15.42:0/2848873904 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5600fa2144d0 0x5600fa5d5ec0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:39.569 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.564+0000 7f23b11e3740 1 --2- 172.21.15.42:0/2848873904 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5600fa2351b0 0x5600fa5e3180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:39.569 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.564+0000 7f23b11e3740 1 --2- 172.21.15.42:0/2848873904 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5600fa20c2f0 0x5600fa5d5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:39.569 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.564+0000 7f23b11e3740 1 -- 172.21.15.42:0/2848873904 >> 172.21.15.42:0/2848873904 conn(0x5600fa229300 msgr2=0x5600fa5dfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:01:39.570 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.564+0000 7f23b11e3740 1 -- 172.21.15.42:0/2848873904 shutdown_connections 2024-08-30T09:01:39.570 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.564+0000 7f23b11e3740 1 -- 172.21.15.42:0/2848873904 wait complete. 2024-08-30T09:01:39.570 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.565+0000 7f23b11e3740 1 -- start start 2024-08-30T09:01:39.570 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.565+0000 7f23b11e3740 1 Processor -- start 2024-08-30T09:01:39.570 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.566+0000 7f23b11e3740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5600fa20c2f0 0x5600fa5e9bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:01:39.570 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.566+0000 7f23b11e3740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5600fa2144d0 0x5600fa5ea290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:01:39.570 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.566+0000 7f23a3fff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5600fa20c2f0 0x5600fa5e9bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:01:39.570 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.566+0000 7f23a3fff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5600fa20c2f0 0x5600fa5e9bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:33822/0 (socket says 172.21.15.42:33822) 2024-08-30T09:01:39.570 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.566+0000 7f23b11e3740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5600fa2351b0 0x5600fa5ead90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:01:39.570 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.566+0000 7f23b11e3740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f23a41b98b0 con 0x5600fa2144d0 2024-08-30T09:01:39.570 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.566+0000 7f23b11e3740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f23a41bad20 con 0x5600fa20c2f0 2024-08-30T09:01:39.570 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.566+0000 7f23b11e3740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f23a41b9f80 con 0x5600fa2351b0 2024-08-30T09:01:39.570 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.566+0000 7f23b11e3740 10 client.0 _ll_register_callbacks cb 0x7f239c0098e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:01:39.570 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.567+0000 7f23a37fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5600fa2144d0 0x5600fa5ea290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:01:39.570 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.567+0000 7f23a37fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5600fa2144d0 0x5600fa5ea290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42040/0 (socket says 172.21.15.42:42040) 2024-08-30T09:01:39.571 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.567+0000 7f23a37fe640 1 -- 172.21.15.42:0/3990295624 learned_addr learned my addr 172.21.15.42:0/3990295624 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:01:39.571 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.567+0000 7f23a881b640 1 --2- 172.21.15.42:0/3990295624 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5600fa2351b0 0x5600fa5ead90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:01:39.571 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.567+0000 7f23a3fff640 1 -- 172.21.15.42:0/3990295624 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5600fa2351b0 msgr2=0x5600fa5ead90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:01:39.571 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.567+0000 7f23a3fff640 1 --2- 172.21.15.42:0/3990295624 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5600fa2351b0 0x5600fa5ead90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:39.571 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.567+0000 7f23a3fff640 1 -- 172.21.15.42:0/3990295624 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5600fa2144d0 msgr2=0x5600fa5ea290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:01:39.571 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.567+0000 7f23a3fff640 1 --2- 172.21.15.42:0/3990295624 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5600fa2144d0 0x5600fa5ea290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:39.571 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.567+0000 7f23a3fff640 1 -- 172.21.15.42:0/3990295624 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5600fa5eb4f0 con 0x5600fa20c2f0 2024-08-30T09:01:39.572 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.567+0000 7f23a3fff640 1 --2- 172.21.15.42:0/3990295624 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5600fa20c2f0 0x5600fa5e9bd0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x5600fa5f0600 tx=0x7f2394025d60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=154c690841d60d75 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:01:39.573 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.567+0000 7f23837fe640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-30T09:01:39.573 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.568+0000 7f23837fe640 1 -- 172.21.15.42:0/3990295624 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f239403b060 con 0x5600fa20c2f0 2024-08-30T09:01:39.573 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.568+0000 7f23837fe640 1 -- 172.21.15.42:0/3990295624 <== mon.2 v2:172.21.15.167:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f2394025870 con 0x5600fa20c2f0 2024-08-30T09:01:39.573 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.568+0000 7f23b11e3740 1 -- 172.21.15.42:0/3990295624 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f238c0ccab0 con 0x5600fa20c2f0 2024-08-30T09:01:39.573 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.568+0000 7f23837fe640 1 -- 172.21.15.42:0/3990295624 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2394042050 con 0x5600fa20c2f0 2024-08-30T09:01:39.573 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.568+0000 7f23b11e3740 10 client.25322 fetch_fsmap learned FSMap version 29 2024-08-30T09:01:39.573 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.568+0000 7f23b11e3740 1 -- 172.21.15.42:0/3990295624 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5600fa5ec850 con 0x5600fa20c2f0 2024-08-30T09:01:39.573 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.569+0000 7f23837fe640 1 -- 172.21.15.42:0/3990295624 <== mon.2 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2394014070 con 0x5600fa20c2f0 2024-08-30T09:01:39.573 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.569+0000 7f23837fe640 1 -- 172.21.15.42:0/3990295624 <== mon.2 v2:172.21.15.167:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f2394035030 con 0x5600fa20c2f0 2024-08-30T09:01:39.573 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.569+0000 7f23b11e3740 10 client.25322 fetch_fsmap finished waiting for FSMap version 29 2024-08-30T09:01:39.573 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.569+0000 7f23b11e3740 10 client.25322 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-30T09:01:39.573 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.569+0000 7f23b11e3740 20 client.25322 populate_metadata read hostname 'smithi042' 2024-08-30T09:01:39.573 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.569+0000 7f23b11e3740 1 --2- 172.21.15.42:0/3990295624 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x5600fa1c91c0 0x5600fa65f160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:01:39.573 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.569+0000 7f23b11e3740 4 client.25322 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-30T09:01:39.574 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.569+0000 7f23b11e3740 1 -- 172.21.15.42:0/3990295624 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f238c0ccab0 con 0x5600fa1c91c0 2024-08-30T09:01:39.574 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.570+0000 7f23a37fe640 1 --2- 172.21.15.42:0/3990295624 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x5600fa1c91c0 0x5600fa65f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:01:39.574 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.570+0000 7f23837fe640 10 client.25322 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:01:39.574 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.570+0000 7f23a37fe640 1 --2- 172.21.15.42:0/3990295624 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x5600fa1c91c0 0x5600fa65f160 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f210b1ee83e758b server_cookie=d279f56bbfda9d9b in_seq=0 out_seq=0 2024-08-30T09:01:39.602 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.599+0000 7f23837fe640 1 -- 172.21.15.42:0/3990295624 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f238c0ccab0 con 0x5600fa1c91c0 2024-08-30T09:01:39.602 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.600+0000 7f23837fe640 10 client.25322 handle_command_reply: tid=0 2024-08-30T09:01:39.602 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.600+0000 7f23b11e3740 1 client.25322 shutdown 2024-08-30T09:01:39.602 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.600+0000 7f23b11e3740 1 -- 172.21.15.42:0/3990295624 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5600fa20c2f0 msgr2=0x5600fa5e9bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:01:39.602 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.600+0000 7f23b11e3740 1 --2- 172.21.15.42:0/3990295624 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5600fa20c2f0 0x5600fa5e9bd0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x5600fa5f0600 tx=0x7f2394025d60 comp rx=0 tx=0).stop 2024-08-30T09:01:39.602 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.600+0000 7f23b11e3740 1 -- 172.21.15.42:0/3990295624 shutdown_connections 2024-08-30T09:01:39.602 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.600+0000 7f23b11e3740 1 --2- 172.21.15.42:0/3990295624 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x5600fa1c91c0 0x5600fa65f160 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:39.602 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.600+0000 7f23b11e3740 1 --2- 172.21.15.42:0/3990295624 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5600fa2351b0 0x5600fa5ead90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:39.603 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.600+0000 7f23837fe640 0 client.25322 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:01:39.603 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.600+0000 7f23b11e3740 1 --2- 172.21.15.42:0/3990295624 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5600fa2144d0 0x5600fa5ea290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:39.603 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.600+0000 7f23b11e3740 1 --2- 172.21.15.42:0/3990295624 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5600fa20c2f0 0x5600fa5e9bd0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:39.603 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.600+0000 7f23b11e3740 1 -- 172.21.15.42:0/3990295624 >> 172.21.15.42:0/3990295624 conn(0x5600fa229300 msgr2=0x5600fa5d5800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:01:39.603 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.601+0000 7f23b11e3740 1 -- 172.21.15.42:0/3990295624 shutdown_connections 2024-08-30T09:01:39.603 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.601+0000 7f23b11e3740 1 -- 172.21.15.42:0/3990295624 wait complete. 2024-08-30T09:01:39.604 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.601+0000 7f23b11e3740 20 client.25322 trim_cache size 0 max 16384 2024-08-30T09:01:39.604 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-08-30T09:01:39.604 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2024-08-30T09:01:39.604 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-08-30T09:01:39.604 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-08-30T09:01:39.605 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.602+0000 7f23aa27e640 1 -- 172.21.15.42:0/3896396753 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f23900b4d40 msgr2=0x7f23900b7250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:01:39.605 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.602+0000 7f23aa27e640 1 --2- 172.21.15.42:0/3896396753 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f23900b4d40 0x7f23900b7250 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f2394009a60 tx=0x7f2394005f90 comp rx=0 tx=0).stop 2024-08-30T09:01:39.605 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.602+0000 7f23aa27e640 1 -- 172.21.15.42:0/3896396753 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f23a41033f0 msgr2=0x7f23a4113790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:01:39.605 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.602+0000 7f23aa27e640 1 --2- 172.21.15.42:0/3896396753 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f23a41033f0 0x7f23a4113790 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f238c031440 tx=0x7f238c003b90 comp rx=0 tx=0).stop 2024-08-30T09:01:39.605 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.602+0000 7f23aa27e640 1 -- 172.21.15.42:0/3896396753 shutdown_connections 2024-08-30T09:01:39.605 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.602+0000 7f23aa27e640 1 --2- 172.21.15.42:0/3896396753 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f23900b4d40 0x7f23900b7250 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:39.605 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.602+0000 7f23aa27e640 1 --2- 172.21.15.42:0/3896396753 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f23a4103d90 0x7f23a4111bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:39.605 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.602+0000 7f23aa27e640 1 --2- 172.21.15.42:0/3896396753 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f23a41033f0 0x7f23a4113790 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:39.606 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.602+0000 7f23aa27e640 1 --2- 172.21.15.42:0/3896396753 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f23a4102370 0x7f23a4113120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:39.606 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.602+0000 7f23aa27e640 1 -- 172.21.15.42:0/3896396753 >> 172.21.15.42:0/3896396753 conn(0x7f23a40fdcd0 msgr2=0x7f23a41066f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:01:39.606 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.602+0000 7f23aa27e640 1 -- 172.21.15.42:0/3896396753 shutdown_connections 2024-08-30T09:01:39.606 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:39.602+0000 7f23aa27e640 1 -- 172.21.15.42:0/3896396753 wait complete. 2024-08-30T09:01:40.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:01:40 smithi042 ceph-mon[18652]: pgmap v400: 129 pgs: 129 active+clean; 999 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 13 MiB/s wr, 971 op/s 2024-08-30T09:01:40.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:01:40 smithi190 ceph-mon[22001]: pgmap v400: 129 pgs: 129 active+clean; 999 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 13 MiB/s wr, 971 op/s 2024-08-30T09:01:40.953 INFO:teuthology.orchestra.run.smithi042.stderr:progress: 55.0 s, 3277.4 tps, lat 29.834 ms stddev 38.542 2024-08-30T09:01:41.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:01:40 smithi167 ceph-mon[22063]: pgmap v400: 129 pgs: 129 active+clean; 999 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 13 MiB/s wr, 971 op/s 2024-08-30T09:01:42.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:01:42 smithi042 ceph-mon[18652]: pgmap v401: 129 pgs: 129 active+clean; 999 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 8.7 MiB/s wr, 771 op/s 2024-08-30T09:01:42.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:01:42 smithi190 ceph-mon[22001]: pgmap v401: 129 pgs: 129 active+clean; 999 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 8.7 MiB/s wr, 771 op/s 2024-08-30T09:01:43.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:01:42 smithi167 ceph-mon[22063]: pgmap v401: 129 pgs: 129 active+clean; 999 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 8.7 MiB/s wr, 771 op/s 2024-08-30T09:01:44.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:01:44 smithi042 ceph-mon[18652]: pgmap v402: 129 pgs: 129 active+clean; 1012 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 13 MiB/s wr, 1.04k op/s 2024-08-30T09:01:44.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:01:44 smithi190 ceph-mon[22001]: pgmap v402: 129 pgs: 129 active+clean; 1012 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 13 MiB/s wr, 1.04k op/s 2024-08-30T09:01:45.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:01:44 smithi167 ceph-mon[22063]: pgmap v402: 129 pgs: 129 active+clean; 1012 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 13 MiB/s wr, 1.04k op/s 2024-08-30T09:01:45.953 INFO:teuthology.orchestra.run.smithi042.stderr:progress: 60.0 s, 2963.2 tps, lat 33.959 ms stddev 42.082 2024-08-30T09:01:46.885 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:01:46 smithi042 ceph-mon[18652]: pgmap v403: 129 pgs: 129 active+clean; 1016 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 12 MiB/s wr, 807 op/s 2024-08-30T09:01:46.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:01:46 smithi190 ceph-mon[22001]: pgmap v403: 129 pgs: 129 active+clean; 1016 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 12 MiB/s wr, 807 op/s 2024-08-30T09:01:47.076 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:01:46 smithi167 ceph-mon[22063]: pgmap v403: 129 pgs: 129 active+clean; 1016 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 12 MiB/s wr, 807 op/s 2024-08-30T09:01:48.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:01:48 smithi042 ceph-mon[18652]: pgmap v404: 129 pgs: 129 active+clean; 1018 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 14 MiB/s wr, 946 op/s 2024-08-30T09:01:48.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:01:48 smithi190 ceph-mon[22001]: pgmap v404: 129 pgs: 129 active+clean; 1018 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 14 MiB/s wr, 946 op/s 2024-08-30T09:01:49.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:01:48 smithi167 ceph-mon[22063]: pgmap v404: 129 pgs: 129 active+clean; 1018 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 14 MiB/s wr, 946 op/s 2024-08-30T09:01:50.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:01:50 smithi190 ceph-mon[22001]: pgmap v405: 129 pgs: 129 active+clean; 1019 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 15 MiB/s wr, 1.01k op/s 2024-08-30T09:01:50.953 INFO:teuthology.orchestra.run.smithi042.stderr:progress: 65.0 s, 2422.6 tps, lat 41.587 ms stddev 50.302 2024-08-30T09:01:51.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:01:50 smithi167 ceph-mon[22063]: pgmap v405: 129 pgs: 129 active+clean; 1019 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 15 MiB/s wr, 1.01k op/s 2024-08-30T09:01:51.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:01:50 smithi042 ceph-mon[18652]: pgmap v405: 129 pgs: 129 active+clean; 1019 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 15 MiB/s wr, 1.01k op/s 2024-08-30T09:01:52.757 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.755+0000 7ff013fff640 1 -- 172.21.15.42:0/3175530435 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0180318a0 con 0x7ff01c103460 2024-08-30T09:01:52.758 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.757+0000 7ff023a0e640 1 -- 172.21.15.42:0/3175530435 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff01c103460 msgr2=0x7ff01c1038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:01:52.758 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.757+0000 7ff023a0e640 1 --2- 172.21.15.42:0/3175530435 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff01c103460 0x7ff01c1038d0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7ff018009900 tx=0x7ff01802a410 comp rx=0 tx=0).stop 2024-08-30T09:01:52.759 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.757+0000 7ff023a0e640 1 -- 172.21.15.42:0/3175530435 shutdown_connections 2024-08-30T09:01:52.759 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.757+0000 7ff023a0e640 1 --2- 172.21.15.42:0/3175530435 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff01c1044e0 0x7ff01c1048f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:52.759 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.757+0000 7ff023a0e640 1 --2- 172.21.15.42:0/3175530435 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff01c104e80 0x7ff01c10b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:52.759 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.757+0000 7ff023a0e640 1 --2- 172.21.15.42:0/3175530435 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff01c103460 0x7ff01c1038d0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:52.759 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.757+0000 7ff023a0e640 1 -- 172.21.15.42:0/3175530435 >> 172.21.15.42:0/3175530435 conn(0x7ff01c0fed80 msgr2=0x7ff01c101250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:01:52.759 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.757+0000 7ff023a0e640 1 -- 172.21.15.42:0/3175530435 shutdown_connections 2024-08-30T09:01:52.759 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.757+0000 7ff023a0e640 1 -- 172.21.15.42:0/3175530435 wait complete. 2024-08-30T09:01:52.761 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.758+0000 7ff023a0e640 1 Processor -- start 2024-08-30T09:01:52.762 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.758+0000 7ff023a0e640 1 -- start start 2024-08-30T09:01:52.762 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.758+0000 7ff023a0e640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff01c103460 0x7ff01c076bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:01:52.762 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.758+0000 7ff023a0e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff01c1044e0 0x7ff01c0754b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:01:52.762 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.758+0000 7ff023a0e640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff01c104e80 0x7ff01c0759f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:01:52.762 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.758+0000 7ff023a0e640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7ff01c10f0a0 con 0x7ff01c104e80 2024-08-30T09:01:52.762 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.758+0000 7ff023a0e640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff01c10ef20 con 0x7ff01c103460 2024-08-30T09:01:52.762 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.758+0000 7ff023a0e640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7ff01c10f220 con 0x7ff01c1044e0 2024-08-30T09:01:52.762 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.758+0000 7ff020fa9640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff01c1044e0 0x7ff01c0754b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:01:52.762 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.758+0000 7ff0217aa640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff01c103460 0x7ff01c076bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:01:52.762 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.759+0000 7ff020fa9640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff01c1044e0 0x7ff01c0754b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:50832/0 (socket says 172.21.15.42:50832) 2024-08-30T09:01:52.762 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.759+0000 7ff0217aa640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff01c103460 0x7ff01c076bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:60532/0 (socket says 172.21.15.42:60532) 2024-08-30T09:01:52.762 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.759+0000 7ff020fa9640 1 -- 172.21.15.42:0/231240027 learned_addr learned my addr 172.21.15.42:0/231240027 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:01:52.763 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.759+0000 7ff021fab640 1 --2- 172.21.15.42:0/231240027 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff01c104e80 0x7ff01c0759f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:01:52.763 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.759+0000 7ff020fa9640 1 -- 172.21.15.42:0/231240027 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff01c103460 msgr2=0x7ff01c076bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:01:52.763 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.759+0000 7ff020fa9640 1 --2- 172.21.15.42:0/231240027 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff01c103460 0x7ff01c076bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:52.763 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.759+0000 7ff020fa9640 1 -- 172.21.15.42:0/231240027 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff01c104e80 msgr2=0x7ff01c0759f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:01:52.763 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.759+0000 7ff020fa9640 1 --2- 172.21.15.42:0/231240027 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff01c104e80 0x7ff01c0759f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:52.763 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.759+0000 7ff020fa9640 1 -- 172.21.15.42:0/231240027 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff01c076120 con 0x7ff01c1044e0 2024-08-30T09:01:52.763 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.759+0000 7ff021fab640 1 --2- 172.21.15.42:0/231240027 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff01c104e80 0x7ff01c0759f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-30T09:01:52.763 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.759+0000 7ff020fa9640 1 --2- 172.21.15.42:0/231240027 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff01c1044e0 0x7ff01c0754b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ff00c00e9f0 tx=0x7ff00c00edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:01:52.763 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.759+0000 7ff0127fc640 1 -- 172.21.15.42:0/231240027 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff00c00cd80 con 0x7ff01c1044e0 2024-08-30T09:01:52.763 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.759+0000 7ff023a0e640 1 -- 172.21.15.42:0/231240027 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff01c1a3da0 con 0x7ff01c1044e0 2024-08-30T09:01:52.763 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.759+0000 7ff0127fc640 1 -- 172.21.15.42:0/231240027 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff00c0054d0 con 0x7ff01c1044e0 2024-08-30T09:01:52.763 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.759+0000 7ff0127fc640 1 -- 172.21.15.42:0/231240027 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff00c010690 con 0x7ff01c1044e0 2024-08-30T09:01:52.764 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.759+0000 7ff023a0e640 1 -- 172.21.15.42:0/231240027 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff01c1a41b0 con 0x7ff01c1044e0 2024-08-30T09:01:52.764 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.763+0000 7ff02aa56740 1 Processor -- start 2024-08-30T09:01:52.767 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.765+0000 7ff02aa56740 1 -- start start 2024-08-30T09:01:52.767 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.765+0000 7ff02aa56740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a33f6c91b0 0x55a33fa77990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:01:52.767 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.765+0000 7ff02aa56740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a33f6a02f0 0x55a33fa78000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:01:52.767 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.765+0000 7ff02aa56740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a33f6a84d0 0x55a33fa78b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:01:52.767 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.765+0000 7ff02aa56740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55a33f9e2000 con 0x55a33f6a02f0 2024-08-30T09:01:52.767 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.765+0000 7ff02aa56740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55a33f9d1590 con 0x55a33f6c91b0 2024-08-30T09:01:52.767 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.765+0000 7ff02aa56740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55a33f9d5540 con 0x55a33f6a84d0 2024-08-30T09:01:52.769 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.765+0000 7ff021fab640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a33f6c91b0 0x55a33fa77990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:01:52.769 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.766+0000 7ff021fab640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a33f6c91b0 0x55a33fa77990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:60548/0 (socket says 172.21.15.42:60548) 2024-08-30T09:01:52.769 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.766+0000 7ff021fab640 1 -- 172.21.15.42:0/471895325 learned_addr learned my addr 172.21.15.42:0/471895325 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:01:52.769 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.766+0000 7ff0217aa640 1 --2- 172.21.15.42:0/471895325 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a33f6a02f0 0x55a33fa78000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:01:52.769 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.766+0000 7ff0127fc640 1 -- 172.21.15.42:0/231240027 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7ff00c010830 con 0x7ff01c1044e0 2024-08-30T09:01:52.769 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.766+0000 7ff021fab640 1 -- 172.21.15.42:0/471895325 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a33f6a84d0 msgr2=0x55a33fa78b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:01:52.769 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.766+0000 7ff020fa9640 1 --2- 172.21.15.42:0/471895325 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a33f6a84d0 0x55a33fa78b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:01:52.769 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.766+0000 7ff021fab640 1 --2- 172.21.15.42:0/471895325 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a33f6a84d0 0x55a33fa78b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:52.770 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.766+0000 7ff021fab640 1 -- 172.21.15.42:0/471895325 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a33f6a02f0 msgr2=0x55a33fa78000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:01:52.770 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.766+0000 7ff021fab640 1 --2- 172.21.15.42:0/471895325 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a33f6a02f0 0x55a33fa78000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:52.770 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.766+0000 7ff021fab640 1 -- 172.21.15.42:0/471895325 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a33f734790 con 0x55a33f6c91b0 2024-08-30T09:01:52.770 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.766+0000 7ff0217aa640 1 --2- 172.21.15.42:0/471895325 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a33f6a02f0 0x55a33fa78000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-30T09:01:52.770 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.766+0000 7ff020fa9640 1 --2- 172.21.15.42:0/471895325 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a33f6a84d0 0x55a33fa78b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-30T09:01:52.770 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.766+0000 7ff021fab640 1 --2- 172.21.15.42:0/471895325 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a33f6c91b0 0x55a33fa77990 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ff01400f410 tx=0x7ff01400f8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c7affccac73e0f5a server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:01:52.770 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.766+0000 7ff0077fe640 1 -- 172.21.15.42:0/471895325 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff014015070 con 0x55a33f6c91b0 2024-08-30T09:01:52.770 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.767+0000 7ff0077fe640 1 -- 172.21.15.42:0/471895325 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff014011bb0 con 0x55a33f6c91b0 2024-08-30T09:01:52.770 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.767+0000 7ff0077fe640 1 -- 172.21.15.42:0/471895325 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff014010c70 con 0x55a33f6c91b0 2024-08-30T09:01:52.770 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.767+0000 7ff0127fc640 1 --2- 172.21.15.42:0/231240027 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7ff0080b4b30 0x7ff0080b7040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:01:52.770 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.767+0000 7ff0127fc640 1 -- 172.21.15.42:0/231240027 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7ff00c0e5a00 con 0x7ff01c1044e0 2024-08-30T09:01:52.770 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.767+0000 7ff0217aa640 1 --2- 172.21.15.42:0/231240027 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7ff0080b4b30 0x7ff0080b7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:01:52.772 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.768+0000 7ff02aa56740 1 -- 172.21.15.42:0/471895325 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a33f6c91b0 msgr2=0x55a33fa77990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:01:52.772 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.768+0000 7ff02aa56740 1 --2- 172.21.15.42:0/471895325 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a33f6c91b0 0x55a33fa77990 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ff01400f410 tx=0x7ff01400f8d0 comp rx=0 tx=0).stop 2024-08-30T09:01:52.773 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.768+0000 7ff02aa56740 1 -- 172.21.15.42:0/471895325 shutdown_connections 2024-08-30T09:01:52.773 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.768+0000 7ff02aa56740 1 --2- 172.21.15.42:0/471895325 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a33f6a84d0 0x55a33fa78b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:52.773 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.768+0000 7ff02aa56740 1 --2- 172.21.15.42:0/471895325 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a33f6a02f0 0x55a33fa78000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:52.773 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.768+0000 7ff02aa56740 1 --2- 172.21.15.42:0/471895325 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a33f6c91b0 0x55a33fa77990 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:52.773 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.768+0000 7ff02aa56740 1 -- 172.21.15.42:0/471895325 >> 172.21.15.42:0/471895325 conn(0x55a33f6bd300 msgr2=0x55a33fa6ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:01:52.773 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.768+0000 7ff02aa56740 1 -- 172.21.15.42:0/471895325 shutdown_connections 2024-08-30T09:01:52.773 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.768+0000 7ff02aa56740 1 -- 172.21.15.42:0/471895325 wait complete. 2024-08-30T09:01:52.773 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.768+0000 7ff0217aa640 1 --2- 172.21.15.42:0/231240027 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7ff0080b4b30 0x7ff0080b7040 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7ff018000c00 tx=0x7ff018031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:01:52.773 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.768+0000 7ff02aa56740 1 -- start start 2024-08-30T09:01:52.773 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.769+0000 7ff02aa56740 1 Processor -- start 2024-08-30T09:01:52.773 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.769+0000 7ff02aa56740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a33f6a02f0 0x55a33fa81410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:01:52.773 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.769+0000 7ff02aa56740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a33f6a84d0 0x55a33fa81ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:01:52.773 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.769+0000 7ff02aa56740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a33f6c91b0 0x55a33fa82640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:01:52.774 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.769+0000 7ff02aa56740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55a33f90aab0 con 0x55a33f6a02f0 2024-08-30T09:01:52.774 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.769+0000 7ff02aa56740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55a33f958680 con 0x55a33f6c91b0 2024-08-30T09:01:52.774 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.769+0000 7ff02aa56740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55a33fa79670 con 0x55a33f6a84d0 2024-08-30T09:01:52.774 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.769+0000 7ff02aa56740 10 client.0 _ll_register_callbacks cb 0x55a33f6e0a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:01:52.774 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.769+0000 7ff0217aa640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a33f6a02f0 0x55a33fa81410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:01:52.774 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.769+0000 7ff0217aa640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a33f6a02f0 0x55a33fa81410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:44488/0 (socket says 172.21.15.42:44488) 2024-08-30T09:01:52.774 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.769+0000 7ff021fab640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a33f6c91b0 0x55a33fa82640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:01:52.774 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.769+0000 7ff0217aa640 1 -- 172.21.15.42:0/1096248332 learned_addr learned my addr 172.21.15.42:0/1096248332 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:01:52.774 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.770+0000 7ff021fab640 1 -- 172.21.15.42:0/1096248332 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a33f6a84d0 msgr2=0x55a33fa81ad0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-30T09:01:52.774 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.770+0000 7ff021fab640 1 --2- 172.21.15.42:0/1096248332 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a33f6a84d0 0x55a33fa81ad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:52.774 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.770+0000 7ff021fab640 1 -- 172.21.15.42:0/1096248332 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a33f6a02f0 msgr2=0x55a33fa81410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:01:52.774 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.770+0000 7ff021fab640 1 --2- 172.21.15.42:0/1096248332 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a33f6a02f0 0x55a33fa81410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:52.775 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.770+0000 7ff021fab640 1 -- 172.21.15.42:0/1096248332 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a33f7ad620 con 0x55a33f6c91b0 2024-08-30T09:01:52.775 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.770+0000 7ff021fab640 1 --2- 172.21.15.42:0/1096248332 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a33f6c91b0 0x55a33fa82640 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ff01400f3e0 tx=0x7ff014012d70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a95c65f43ce40f71 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:01:52.775 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.770+0000 7ff004ff9640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-30T09:01:52.775 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.770+0000 7ff004ff9640 1 -- 172.21.15.42:0/1096248332 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff014027020 con 0x55a33f6c91b0 2024-08-30T09:01:52.775 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.770+0000 7ff004ff9640 1 -- 172.21.15.42:0/1096248332 <== mon.2 v2:172.21.15.167:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7ff0140109f0 con 0x55a33f6c91b0 2024-08-30T09:01:52.775 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.771+0000 7ff02aa56740 1 -- 172.21.15.42:0/1096248332 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff014011bb0 con 0x55a33f6c91b0 2024-08-30T09:01:52.777 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.772+0000 7ff004ff9640 1 -- 172.21.15.42:0/1096248332 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff014015040 con 0x55a33f6c91b0 2024-08-30T09:01:52.777 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.772+0000 7ff02aa56740 10 client.25334 fetch_fsmap learned FSMap version 29 2024-08-30T09:01:52.777 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.772+0000 7ff02aa56740 1 -- 172.21.15.42:0/1096248332 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a33fa8b550 con 0x55a33f6c91b0 2024-08-30T09:01:52.777 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.773+0000 7ff004ff9640 1 -- 172.21.15.42:0/1096248332 <== mon.2 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff014011bb0 con 0x55a33f6c91b0 2024-08-30T09:01:52.777 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.773+0000 7ff004ff9640 1 -- 172.21.15.42:0/1096248332 <== mon.2 v2:172.21.15.167:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7ff014022070 con 0x55a33f6c91b0 2024-08-30T09:01:52.777 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.773+0000 7ff02aa56740 10 client.25334 fetch_fsmap finished waiting for FSMap version 29 2024-08-30T09:01:52.777 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.773+0000 7ff02aa56740 10 client.25334 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-30T09:01:52.777 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.773+0000 7ff02aa56740 20 client.25334 populate_metadata read hostname 'smithi042' 2024-08-30T09:01:52.777 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.773+0000 7ff02aa56740 1 --2- 172.21.15.42:0/1096248332 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55a33f687d80 0x55a33fa8e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:01:52.778 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.773+0000 7ff02aa56740 4 client.25334 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-30T09:01:52.778 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.773+0000 7ff02aa56740 1 -- 172.21.15.42:0/1096248332 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a33f9e3f50 con 0x55a33f687d80 2024-08-30T09:01:52.778 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.774+0000 7ff0217aa640 1 --2- 172.21.15.42:0/1096248332 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55a33f687d80 0x55a33fa8e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:01:52.778 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.774+0000 7ff004ff9640 10 client.25334 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:01:52.778 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.774+0000 7ff0217aa640 1 --2- 172.21.15.42:0/1096248332 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55a33f687d80 0x55a33fa8e9c0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=49e934c3336a5bc7 server_cookie=8d19776fd17d32a7 in_seq=0 out_seq=0 2024-08-30T09:01:52.778 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.775+0000 7ff004ff9640 1 -- 172.21.15.42:0/1096248332 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a33f9e3f50 con 0x55a33f687d80 2024-08-30T09:01:52.778 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.775+0000 7ff004ff9640 10 client.25334 handle_command_reply: tid=0 2024-08-30T09:01:52.778 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.775+0000 7ff02aa56740 1 client.25334 shutdown 2024-08-30T09:01:52.778 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.775+0000 7ff02aa56740 1 -- 172.21.15.42:0/1096248332 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a33f6c91b0 msgr2=0x55a33fa82640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:01:52.778 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.775+0000 7ff02aa56740 1 --2- 172.21.15.42:0/1096248332 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a33f6c91b0 0x55a33fa82640 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ff01400f3e0 tx=0x7ff014012d70 comp rx=0 tx=0).stop 2024-08-30T09:01:52.778 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.775+0000 7ff02aa56740 1 -- 172.21.15.42:0/1096248332 shutdown_connections 2024-08-30T09:01:52.778 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.775+0000 7ff02aa56740 1 --2- 172.21.15.42:0/1096248332 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55a33f687d80 0x55a33fa8e9c0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:52.778 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.775+0000 7ff02aa56740 1 --2- 172.21.15.42:0/1096248332 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a33f6a84d0 0x55a33fa81ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:52.779 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.775+0000 7ff02aa56740 1 --2- 172.21.15.42:0/1096248332 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a33f6c91b0 0x55a33fa82640 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:52.779 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.775+0000 7ff02aa56740 1 --2- 172.21.15.42:0/1096248332 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a33f6a02f0 0x55a33fa81410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:52.779 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.775+0000 7ff02aa56740 1 -- 172.21.15.42:0/1096248332 >> 172.21.15.42:0/1096248332 conn(0x55a33f6bd300 msgr2=0x55a33fa69170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:01:52.779 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.775+0000 7ff004ff9640 0 client.25334 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:01:52.779 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.775+0000 7ff02aa56740 1 -- 172.21.15.42:0/1096248332 shutdown_connections 2024-08-30T09:01:52.780 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.776+0000 7ff02aa56740 1 -- 172.21.15.42:0/1096248332 wait complete. 2024-08-30T09:01:52.780 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.776+0000 7ff02aa56740 20 client.25334 trim_cache size 0 max 16384 2024-08-30T09:01:52.802 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.799+0000 7ff02aa56740 1 Processor -- start 2024-08-30T09:01:52.802 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.799+0000 7ff02aa56740 1 -- start start 2024-08-30T09:01:52.802 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.799+0000 7ff02aa56740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a33f6bd300 0x55a33fa65420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:01:52.802 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.799+0000 7ff02aa56740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a33f6a02f0 0x55a33fa775c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:01:52.802 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.799+0000 7ff02aa56740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a33f6a84d0 0x55a33fa698e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:01:52.803 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.799+0000 7ff02aa56740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7ff01c1acb70 con 0x55a33f6a02f0 2024-08-30T09:01:52.803 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.799+0000 7ff02aa56740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff01c1aa440 con 0x55a33f6bd300 2024-08-30T09:01:52.803 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.799+0000 7ff02aa56740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7ff01c1a8290 con 0x55a33f6a84d0 2024-08-30T09:01:52.803 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.800+0000 7ff0217aa640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a33f6bd300 0x55a33fa65420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:01:52.803 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.800+0000 7ff0217aa640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a33f6bd300 0x55a33fa65420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:60566/0 (socket says 172.21.15.42:60566) 2024-08-30T09:01:52.803 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.800+0000 7ff0217aa640 1 -- 172.21.15.42:0/2853149238 learned_addr learned my addr 172.21.15.42:0/2853149238 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:01:52.803 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.800+0000 7ff021fab640 1 --2- 172.21.15.42:0/2853149238 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a33f6a84d0 0x55a33fa698e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:01:52.803 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.800+0000 7ff020fa9640 1 --2- 172.21.15.42:0/2853149238 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a33f6a02f0 0x55a33fa775c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:01:52.803 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.800+0000 7ff0217aa640 1 -- 172.21.15.42:0/2853149238 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a33f6a84d0 msgr2=0x55a33fa698e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:01:52.803 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.800+0000 7ff0217aa640 1 --2- 172.21.15.42:0/2853149238 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a33f6a84d0 0x55a33fa698e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:52.803 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.800+0000 7ff0217aa640 1 -- 172.21.15.42:0/2853149238 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a33f6a02f0 msgr2=0x55a33fa775c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:01:52.803 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.800+0000 7ff0217aa640 1 --2- 172.21.15.42:0/2853149238 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a33f6a02f0 0x55a33fa775c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:52.803 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.800+0000 7ff0217aa640 1 -- 172.21.15.42:0/2853149238 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a33fa81e40 con 0x55a33f6bd300 2024-08-30T09:01:52.804 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.800+0000 7ff020fa9640 1 --2- 172.21.15.42:0/2853149238 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a33f6a02f0 0x55a33fa775c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-30T09:01:52.804 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.800+0000 7ff0217aa640 1 --2- 172.21.15.42:0/2853149238 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a33f6bd300 0x55a33fa65420 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ff01802a930 tx=0x7ff01803f000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6dedf4b30b020dbf server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:01:52.804 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.800+0000 7ff006ffd640 1 -- 172.21.15.42:0/2853149238 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff018054070 con 0x55a33f6bd300 2024-08-30T09:01:52.804 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.801+0000 7ff006ffd640 1 -- 172.21.15.42:0/2853149238 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff018004270 con 0x55a33f6bd300 2024-08-30T09:01:52.804 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.801+0000 7ff006ffd640 1 -- 172.21.15.42:0/2853149238 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff018035070 con 0x55a33f6bd300 2024-08-30T09:01:52.805 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.802+0000 7ff02aa56740 1 -- 172.21.15.42:0/2853149238 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a33f6bd300 msgr2=0x55a33fa65420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:01:52.805 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.802+0000 7ff02aa56740 1 --2- 172.21.15.42:0/2853149238 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a33f6bd300 0x55a33fa65420 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ff01802a930 tx=0x7ff01803f000 comp rx=0 tx=0).stop 2024-08-30T09:01:52.805 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.802+0000 7ff02aa56740 1 -- 172.21.15.42:0/2853149238 shutdown_connections 2024-08-30T09:01:52.805 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.802+0000 7ff02aa56740 1 --2- 172.21.15.42:0/2853149238 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a33f6a84d0 0x55a33fa698e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:52.805 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.802+0000 7ff02aa56740 1 --2- 172.21.15.42:0/2853149238 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a33f6a02f0 0x55a33fa775c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:52.805 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.802+0000 7ff02aa56740 1 --2- 172.21.15.42:0/2853149238 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a33f6bd300 0x55a33fa65420 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:52.806 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.802+0000 7ff02aa56740 1 -- 172.21.15.42:0/2853149238 >> 172.21.15.42:0/2853149238 conn(0x55a33fa78260 msgr2=0x55a33fa787d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:01:52.806 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.802+0000 7ff02aa56740 1 -- 172.21.15.42:0/2853149238 shutdown_connections 2024-08-30T09:01:52.806 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.802+0000 7ff02aa56740 1 -- 172.21.15.42:0/2853149238 wait complete. 2024-08-30T09:01:52.806 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.803+0000 7ff02aa56740 1 -- start start 2024-08-30T09:01:52.806 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.803+0000 7ff02aa56740 1 Processor -- start 2024-08-30T09:01:52.806 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.804+0000 7ff02aa56740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a33f6a02f0 0x55a33fa89540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:01:52.806 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.804+0000 7ff02aa56740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a33f6a84d0 0x55a33fa88360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:01:52.806 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.804+0000 7ff02aa56740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a33f6bd300 0x55a33fa88a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:01:52.807 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.804+0000 7ff02aa56740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7ff01c1a8900 con 0x55a33f6bd300 2024-08-30T09:01:52.807 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.804+0000 7ff02aa56740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff01c1a9d70 con 0x55a33f6a84d0 2024-08-30T09:01:52.807 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.804+0000 7ff02aa56740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7ff01c1a8fd0 con 0x55a33f6a02f0 2024-08-30T09:01:52.807 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.804+0000 7ff02aa56740 10 client.0 _ll_register_callbacks cb 0x7ff014009ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:01:52.808 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.805+0000 7ff0217aa640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a33f6a02f0 0x55a33fa89540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:01:52.808 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.805+0000 7ff0217aa640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a33f6a02f0 0x55a33fa89540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:50876/0 (socket says 172.21.15.42:50876) 2024-08-30T09:01:52.808 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.805+0000 7ff0217aa640 1 -- 172.21.15.42:0/3636656868 learned_addr learned my addr 172.21.15.42:0/3636656868 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:01:52.809 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.805+0000 7ff021fab640 1 --2- 172.21.15.42:0/3636656868 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a33f6bd300 0x55a33fa88a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:01:52.809 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.805+0000 7ff020fa9640 1 --2- 172.21.15.42:0/3636656868 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a33f6a84d0 0x55a33fa88360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:01:52.809 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.806+0000 7ff0217aa640 1 -- 172.21.15.42:0/3636656868 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a33f6a84d0 msgr2=0x55a33fa88360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:01:52.809 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.806+0000 7ff0217aa640 1 --2- 172.21.15.42:0/3636656868 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a33f6a84d0 0x55a33fa88360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:52.809 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.806+0000 7ff0217aa640 1 -- 172.21.15.42:0/3636656868 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a33f6bd300 msgr2=0x55a33fa88a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:01:52.809 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.806+0000 7ff0217aa640 1 --2- 172.21.15.42:0/3636656868 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a33f6bd300 0x55a33fa88a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:52.809 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.806+0000 7ff0217aa640 1 -- 172.21.15.42:0/3636656868 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a33fa86ea0 con 0x55a33f6a02f0 2024-08-30T09:01:52.809 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.806+0000 7ff0217aa640 1 --2- 172.21.15.42:0/3636656868 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a33f6a02f0 0x55a33fa89540 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ff018009a20 tx=0x7ff018005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=19af424b1b962853 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:01:52.809 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.806+0000 7ff004ff9640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-08-30T09:01:52.809 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.806+0000 7ff004ff9640 1 -- 172.21.15.42:0/3636656868 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff018054040 con 0x55a33f6a02f0 2024-08-30T09:01:52.809 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.806+0000 7ff004ff9640 1 -- 172.21.15.42:0/3636656868 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7ff0180039d0 con 0x55a33f6a02f0 2024-08-30T09:01:52.809 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.806+0000 7ff02aa56740 1 -- 172.21.15.42:0/3636656868 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff018004270 con 0x55a33f6a02f0 2024-08-30T09:01:52.809 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.807+0000 7ff004ff9640 1 -- 172.21.15.42:0/3636656868 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff018062050 con 0x55a33f6a02f0 2024-08-30T09:01:52.810 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.807+0000 7ff02aa56740 10 client.25333 fetch_fsmap learned FSMap version 29 2024-08-30T09:01:52.810 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.807+0000 7ff02aa56740 1 -- 172.21.15.42:0/3636656868 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a33fa6f040 con 0x55a33f6a02f0 2024-08-30T09:01:52.811 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.807+0000 7ff004ff9640 1 -- 172.21.15.42:0/3636656868 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff018004270 con 0x55a33f6a02f0 2024-08-30T09:01:52.811 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.807+0000 7ff004ff9640 1 -- 172.21.15.42:0/3636656868 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7ff018043070 con 0x55a33f6a02f0 2024-08-30T09:01:52.811 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.807+0000 7ff02aa56740 10 client.25333 fetch_fsmap finished waiting for FSMap version 29 2024-08-30T09:01:52.811 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.807+0000 7ff02aa56740 10 client.25333 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-30T09:01:52.811 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.807+0000 7ff02aa56740 20 client.25333 populate_metadata read hostname 'smithi042' 2024-08-30T09:01:52.811 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.807+0000 7ff02aa56740 1 --2- 172.21.15.42:0/3636656868 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55a33f6c91b0 0x55a33fa71aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:01:52.811 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.807+0000 7ff02aa56740 4 client.25333 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-30T09:01:52.811 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.807+0000 7ff02aa56740 1 -- 172.21.15.42:0/3636656868 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a33f9df9f0 con 0x55a33f6c91b0 2024-08-30T09:01:52.811 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.808+0000 7ff020fa9640 1 --2- 172.21.15.42:0/3636656868 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55a33f6c91b0 0x55a33fa71aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:01:52.811 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.808+0000 7ff004ff9640 10 client.25333 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:01:52.812 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.808+0000 7ff020fa9640 1 --2- 172.21.15.42:0/3636656868 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55a33f6c91b0 0x55a33fa71aa0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=50c4ec099db670b1 server_cookie=ea142667797dd2a9 in_seq=0 out_seq=0 2024-08-30T09:01:52.863 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.860+0000 7ff004ff9640 1 -- 172.21.15.42:0/3636656868 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a33f9df9f0 con 0x55a33f6c91b0 2024-08-30T09:01:52.863 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.860+0000 7ff004ff9640 10 client.25333 handle_command_reply: tid=0 2024-08-30T09:01:52.863 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.860+0000 7ff02aa56740 1 client.25333 shutdown 2024-08-30T09:01:52.863 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.860+0000 7ff02aa56740 1 -- 172.21.15.42:0/3636656868 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a33f6a02f0 msgr2=0x55a33fa89540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:01:52.863 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.860+0000 7ff02aa56740 1 --2- 172.21.15.42:0/3636656868 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a33f6a02f0 0x55a33fa89540 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ff018009a20 tx=0x7ff018005fd0 comp rx=0 tx=0).stop 2024-08-30T09:01:52.863 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.860+0000 7ff02aa56740 1 -- 172.21.15.42:0/3636656868 shutdown_connections 2024-08-30T09:01:52.863 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.860+0000 7ff02aa56740 1 --2- 172.21.15.42:0/3636656868 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55a33f6c91b0 0x55a33fa71aa0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:52.863 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.860+0000 7ff02aa56740 1 --2- 172.21.15.42:0/3636656868 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a33f6bd300 0x55a33fa88a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:52.863 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.860+0000 7ff02aa56740 1 --2- 172.21.15.42:0/3636656868 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a33f6a84d0 0x55a33fa88360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:52.863 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.860+0000 7ff02aa56740 1 --2- 172.21.15.42:0/3636656868 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a33f6a02f0 0x55a33fa89540 secure :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ff018009a20 tx=0x7ff018005fd0 comp rx=0 tx=0).stop 2024-08-30T09:01:52.863 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.860+0000 7ff02aa56740 1 -- 172.21.15.42:0/3636656868 >> 172.21.15.42:0/3636656868 conn(0x55a33fa78260 msgr2=0x55a33fa775c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:01:52.864 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.860+0000 7ff004ff9640 0 client.25333 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:01:52.864 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.860+0000 7ff02aa56740 1 -- 172.21.15.42:0/3636656868 shutdown_connections 2024-08-30T09:01:52.864 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.861+0000 7ff02aa56740 1 -- 172.21.15.42:0/3636656868 wait complete. 2024-08-30T09:01:52.864 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.861+0000 7ff02aa56740 20 client.25333 trim_cache size 0 max 16384 2024-08-30T09:01:52.864 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-08-30T09:01:52.864 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2024-08-30T09:01:52.864 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-08-30T09:01:52.865 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-08-30T09:01:52.865 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.862+0000 7ff023a0e640 1 -- 172.21.15.42:0/231240027 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7ff0080b4b30 msgr2=0x7ff0080b7040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:01:52.865 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.862+0000 7ff023a0e640 1 --2- 172.21.15.42:0/231240027 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7ff0080b4b30 0x7ff0080b7040 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7ff018000c00 tx=0x7ff018031400 comp rx=0 tx=0).stop 2024-08-30T09:01:52.865 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.862+0000 7ff023a0e640 1 -- 172.21.15.42:0/231240027 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff01c1044e0 msgr2=0x7ff01c0754b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:01:52.865 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.862+0000 7ff023a0e640 1 --2- 172.21.15.42:0/231240027 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff01c1044e0 0x7ff01c0754b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ff00c00e9f0 tx=0x7ff00c00edf0 comp rx=0 tx=0).stop 2024-08-30T09:01:52.866 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.862+0000 7ff023a0e640 1 -- 172.21.15.42:0/231240027 shutdown_connections 2024-08-30T09:01:52.866 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.862+0000 7ff023a0e640 1 --2- 172.21.15.42:0/231240027 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7ff0080b4b30 0x7ff0080b7040 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:52.866 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.862+0000 7ff023a0e640 1 --2- 172.21.15.42:0/231240027 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff01c1044e0 0x7ff01c0754b0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:52.866 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.862+0000 7ff023a0e640 1 --2- 172.21.15.42:0/231240027 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff01c104e80 0x7ff01c0759f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:52.866 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.862+0000 7ff023a0e640 1 --2- 172.21.15.42:0/231240027 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff01c103460 0x7ff01c076bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:01:52.866 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.862+0000 7ff023a0e640 1 -- 172.21.15.42:0/231240027 >> 172.21.15.42:0/231240027 conn(0x7ff01c0fed80 msgr2=0x7ff01c107800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:01:52.866 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.862+0000 7ff023a0e640 1 -- 172.21.15.42:0/231240027 shutdown_connections 2024-08-30T09:01:52.866 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:01:52.863+0000 7ff023a0e640 1 -- 172.21.15.42:0/231240027 wait complete. 2024-08-30T09:01:52.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:01:52 smithi190 ceph-mon[22001]: pgmap v406: 129 pgs: 129 active+clean; 1019 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 14 MiB/s wr, 945 op/s 2024-08-30T09:01:53.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:01:52 smithi167 ceph-mon[22063]: pgmap v406: 129 pgs: 129 active+clean; 1019 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 14 MiB/s wr, 945 op/s 2024-08-30T09:01:53.132 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:01:52 smithi042 ceph-mon[18652]: pgmap v406: 129 pgs: 129 active+clean; 1019 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 14 MiB/s wr, 945 op/s 2024-08-30T09:01:54.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:01:54 smithi190 ceph-mon[22001]: pgmap v407: 129 pgs: 129 active+clean; 1019 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 16 MiB/s wr, 1.11k op/s 2024-08-30T09:01:55.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:01:54 smithi167 ceph-mon[22063]: pgmap v407: 129 pgs: 129 active+clean; 1019 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 16 MiB/s wr, 1.11k op/s 2024-08-30T09:01:55.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:01:54 smithi042 ceph-mon[18652]: pgmap v407: 129 pgs: 129 active+clean; 1019 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 16 MiB/s wr, 1.11k op/s 2024-08-30T09:01:55.953 INFO:teuthology.orchestra.run.smithi042.stderr:progress: 70.0 s, 2529.4 tps, lat 39.868 ms stddev 49.552 2024-08-30T09:01:56.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:01:56 smithi190 ceph-mon[22001]: pgmap v408: 129 pgs: 129 active+clean; 1019 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 MiB/s wr, 829 op/s 2024-08-30T09:01:57.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:01:56 smithi167 ceph-mon[22063]: pgmap v408: 129 pgs: 129 active+clean; 1019 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 MiB/s wr, 829 op/s 2024-08-30T09:01:57.132 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:01:56 smithi042 ceph-mon[18652]: pgmap v408: 129 pgs: 129 active+clean; 1019 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 MiB/s wr, 829 op/s 2024-08-30T09:01:58.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:01:58 smithi190 ceph-mon[22001]: pgmap v409: 129 pgs: 129 active+clean; 1023 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 10 MiB/s wr, 842 op/s 2024-08-30T09:01:59.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:01:58 smithi167 ceph-mon[22063]: pgmap v409: 129 pgs: 129 active+clean; 1023 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 10 MiB/s wr, 842 op/s 2024-08-30T09:01:59.132 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:01:58 smithi042 ceph-mon[18652]: pgmap v409: 129 pgs: 129 active+clean; 1023 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 10 MiB/s wr, 842 op/s 2024-08-30T09:02:00.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:02:00 smithi190 ceph-mon[22001]: pgmap v410: 129 pgs: 129 active+clean; 1.0 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.5 MiB/s wr, 790 op/s 2024-08-30T09:02:00.937 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:02:00 smithi042 ceph-mon[18652]: pgmap v410: 129 pgs: 129 active+clean; 1.0 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.5 MiB/s wr, 790 op/s 2024-08-30T09:02:00.953 INFO:teuthology.orchestra.run.smithi042.stderr:progress: 75.0 s, 3279.8 tps, lat 30.327 ms stddev 36.238 2024-08-30T09:02:01.076 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:02:00 smithi167 ceph-mon[22063]: pgmap v410: 129 pgs: 129 active+clean; 1.0 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.5 MiB/s wr, 790 op/s 2024-08-30T09:02:02.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:02:02 smithi190 ceph-mon[22001]: pgmap v411: 129 pgs: 129 active+clean; 1.0 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.2 MiB/s wr, 619 op/s 2024-08-30T09:02:03.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:02:02 smithi167 ceph-mon[22063]: pgmap v411: 129 pgs: 129 active+clean; 1.0 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.2 MiB/s wr, 619 op/s 2024-08-30T09:02:03.132 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:02:02 smithi042 ceph-mon[18652]: pgmap v411: 129 pgs: 129 active+clean; 1.0 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.2 MiB/s wr, 619 op/s 2024-08-30T09:02:03.606 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-30T09:02:04.072 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T09:02:04.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.689+0000 7f7d0f5cf640 1 -- 172.21.15.42:0/2954422133 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7d08102d00 msgr2=0x7f7d0810fa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:02:04.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.689+0000 7f7d0f5cf640 1 --2- 172.21.15.42:0/2954422133 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7d08102d00 0x7f7d0810fa20 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f7cf8009860 tx=0x7f7cf802a3f0 comp rx=0 tx=0).stop 2024-08-30T09:02:04.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.689+0000 7f7d0f5cf640 1 -- 172.21.15.42:0/2954422133 shutdown_connections 2024-08-30T09:02:04.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.689+0000 7f7d0f5cf640 1 --2- 172.21.15.42:0/2954422133 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7d08102d00 0x7f7d0810fa20 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:04.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.689+0000 7f7d0f5cf640 1 --2- 172.21.15.42:0/2954422133 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7d0810ff60 0x7f7d08112380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:04.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.689+0000 7f7d0f5cf640 1 --2- 172.21.15.42:0/2954422133 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7d081023b0 0x7f7d081027c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:04.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.689+0000 7f7d0f5cf640 1 -- 172.21.15.42:0/2954422133 >> 172.21.15.42:0/2954422133 conn(0x7f7d08079580 msgr2=0x7f7d08079990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:02:04.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.689+0000 7f7d0f5cf640 1 -- 172.21.15.42:0/2954422133 shutdown_connections 2024-08-30T09:02:04.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.689+0000 7f7d0f5cf640 1 -- 172.21.15.42:0/2954422133 wait complete. 2024-08-30T09:02:04.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.690+0000 7f7d0f5cf640 1 Processor -- start 2024-08-30T09:02:04.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.692+0000 7f7d0f5cf640 1 -- start start 2024-08-30T09:02:04.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.693+0000 7f7d0f5cf640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7d081023b0 0x7f7d0819c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:04.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.693+0000 7f7d0f5cf640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7d08102d00 0x7f7d0819cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:04.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.693+0000 7f7d0f5cf640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7d0810ff60 0x7f7d081a1d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:04.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.693+0000 7f7d0f5cf640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f7d08115ac0 con 0x7f7d081023b0 2024-08-30T09:02:04.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.693+0000 7f7d0f5cf640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f7d08115940 con 0x7f7d0810ff60 2024-08-30T09:02:04.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.693+0000 7f7d0f5cf640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f7d08115c40 con 0x7f7d08102d00 2024-08-30T09:02:04.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.694+0000 7f7d0d36b640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7d081023b0 0x7f7d0819c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:02:04.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.694+0000 7f7d0d36b640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7d081023b0 0x7f7d0819c530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:44684/0 (socket says 172.21.15.42:44684) 2024-08-30T09:02:04.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.694+0000 7f7d0d36b640 1 -- 172.21.15.42:0/321154225 learned_addr learned my addr 172.21.15.42:0/321154225 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:02:04.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.694+0000 7f7d0db6c640 1 --2- 172.21.15.42:0/321154225 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7d0810ff60 0x7f7d081a1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:02:04.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.694+0000 7f7d0cb6a640 1 --2- 172.21.15.42:0/321154225 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7d08102d00 0x7f7d0819cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:02:04.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.694+0000 7f7d0d36b640 1 -- 172.21.15.42:0/321154225 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7d08102d00 msgr2=0x7f7d0819cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:02:04.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.694+0000 7f7d0d36b640 1 --2- 172.21.15.42:0/321154225 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7d08102d00 0x7f7d0819cad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:04.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.694+0000 7f7d0d36b640 1 -- 172.21.15.42:0/321154225 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7d0810ff60 msgr2=0x7f7d081a1d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:02:04.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.694+0000 7f7d0d36b640 1 --2- 172.21.15.42:0/321154225 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7d0810ff60 0x7f7d081a1d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:04.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.694+0000 7f7d0d36b640 1 -- 172.21.15.42:0/321154225 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d081a24a0 con 0x7f7d081023b0 2024-08-30T09:02:04.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.694+0000 7f7d0cb6a640 1 --2- 172.21.15.42:0/321154225 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7d08102d00 0x7f7d0819cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-30T09:02:04.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.695+0000 7f7d0db6c640 1 --2- 172.21.15.42:0/321154225 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7d0810ff60 0x7f7d081a1d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-30T09:02:04.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.695+0000 7f7d0d36b640 1 --2- 172.21.15.42:0/321154225 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7d081023b0 0x7f7d0819c530 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f7d0400e9c0 tx=0x7f7d0400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:02:04.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.695+0000 7f7cfe7fc640 1 -- 172.21.15.42:0/321154225 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d0400cde0 con 0x7f7d081023b0 2024-08-30T09:02:04.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.695+0000 7f7d0f5cf640 1 -- 172.21.15.42:0/321154225 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d081a2790 con 0x7f7d081023b0 2024-08-30T09:02:04.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.696+0000 7f7d0f5cf640 1 -- 172.21.15.42:0/321154225 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d081a8ef0 con 0x7f7d081023b0 2024-08-30T09:02:04.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.696+0000 7f7cfe7fc640 1 -- 172.21.15.42:0/321154225 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d04009460 con 0x7f7d081023b0 2024-08-30T09:02:04.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.696+0000 7f7cfe7fc640 1 -- 172.21.15.42:0/321154225 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d04005020 con 0x7f7d081023b0 2024-08-30T09:02:04.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.700+0000 7f7d15b44740 1 Processor -- start 2024-08-30T09:02:04.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.703+0000 7f7cfe7fc640 1 -- 172.21.15.42:0/321154225 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7f7d040035f0 con 0x7f7d081023b0 2024-08-30T09:02:04.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.704+0000 7f7d15b44740 1 -- start start 2024-08-30T09:02:04.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.704+0000 7f7d15b44740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bc5cfaa070 0x55bc5d357ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:04.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.704+0000 7f7d15b44740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bc5cf811b0 0x55bc5d358010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:04.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.704+0000 7f7d15b44740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bc5cf89390 0x55bc5d358550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:04.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.704+0000 7f7d15b44740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55bc5d2c6630 con 0x55bc5cf811b0 2024-08-30T09:02:04.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.704+0000 7f7d15b44740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55bc5d2b5bc0 con 0x55bc5cf89390 2024-08-30T09:02:04.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.704+0000 7f7d15b44740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55bc5d2b9b70 con 0x55bc5cfaa070 2024-08-30T09:02:04.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.704+0000 7f7d0db6c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bc5cfaa070 0x55bc5d357ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:02:04.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.704+0000 7f7d0db6c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bc5cfaa070 0x55bc5d357ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:33992/0 (socket says 172.21.15.42:33992) 2024-08-30T09:02:04.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.704+0000 7f7d0db6c640 1 -- 172.21.15.42:0/1799778268 learned_addr learned my addr 172.21.15.42:0/1799778268 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:02:04.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.704+0000 7f7d0d36b640 1 --2- 172.21.15.42:0/1799778268 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bc5cf811b0 0x55bc5d358010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:02:04.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.705+0000 7f7d0cb6a640 1 --2- 172.21.15.42:0/1799778268 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bc5cf89390 0x55bc5d358550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:02:04.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.705+0000 7f7d0db6c640 1 -- 172.21.15.42:0/1799778268 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bc5cf89390 msgr2=0x55bc5d358550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:02:04.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.705+0000 7f7d0db6c640 1 --2- 172.21.15.42:0/1799778268 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bc5cf89390 0x55bc5d358550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:04.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.705+0000 7f7d0db6c640 1 -- 172.21.15.42:0/1799778268 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bc5cf811b0 msgr2=0x55bc5d358010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:02:04.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.705+0000 7f7d0db6c640 1 --2- 172.21.15.42:0/1799778268 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bc5cf811b0 0x55bc5d358010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:04.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.705+0000 7f7d0db6c640 1 -- 172.21.15.42:0/1799778268 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc5d015650 con 0x55bc5cfaa070 2024-08-30T09:02:04.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.705+0000 7f7cfe7fc640 1 --2- 172.21.15.42:0/321154225 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f7cf40b4bf0 0x7f7cf40b7100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:04.710 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.706+0000 7f7cfe7fc640 1 -- 172.21.15.42:0/321154225 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f7d040e5740 con 0x7f7d081023b0 2024-08-30T09:02:04.710 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.706+0000 7f7d0db6c640 1 --2- 172.21.15.42:0/1799778268 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bc5cfaa070 0x55bc5d357ad0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f7d0000c450 tx=0x7f7d0000c910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8ff86eb2be77b651 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:02:04.710 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.706+0000 7f7d0d36b640 1 --2- 172.21.15.42:0/321154225 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f7cf40b4bf0 0x7f7cf40b7100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:02:04.710 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.707+0000 7f7cf37fe640 1 -- 172.21.15.42:0/1799778268 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d00007430 con 0x55bc5cfaa070 2024-08-30T09:02:04.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.707+0000 7f7cf37fe640 1 -- 172.21.15.42:0/1799778268 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d00007a80 con 0x55bc5cfaa070 2024-08-30T09:02:04.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.707+0000 7f7cf37fe640 1 -- 172.21.15.42:0/1799778268 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d0000aa60 con 0x55bc5cfaa070 2024-08-30T09:02:04.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.708+0000 7f7d0d36b640 1 --2- 172.21.15.42:0/321154225 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f7cf40b4bf0 0x7f7cf40b7100 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x55bc5d356750 tx=0x7f7d040eb550 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:02:04.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.708+0000 7f7d15b44740 1 -- 172.21.15.42:0/1799778268 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bc5cfaa070 msgr2=0x55bc5d357ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:02:04.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.708+0000 7f7d15b44740 1 --2- 172.21.15.42:0/1799778268 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bc5cfaa070 0x55bc5d357ad0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f7d0000c450 tx=0x7f7d0000c910 comp rx=0 tx=0).stop 2024-08-30T09:02:04.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.708+0000 7f7d15b44740 1 -- 172.21.15.42:0/1799778268 shutdown_connections 2024-08-30T09:02:04.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.708+0000 7f7d15b44740 1 --2- 172.21.15.42:0/1799778268 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bc5cf89390 0x55bc5d358550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:04.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.708+0000 7f7d15b44740 1 --2- 172.21.15.42:0/1799778268 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bc5cf811b0 0x55bc5d358010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:04.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.708+0000 7f7d15b44740 1 --2- 172.21.15.42:0/1799778268 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bc5cfaa070 0x55bc5d357ad0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:04.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.708+0000 7f7d15b44740 1 -- 172.21.15.42:0/1799778268 >> 172.21.15.42:0/1799778268 conn(0x55bc5cf9e460 msgr2=0x55bc5d3559c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:02:04.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.708+0000 7f7d15b44740 1 -- 172.21.15.42:0/1799778268 shutdown_connections 2024-08-30T09:02:04.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.708+0000 7f7d15b44740 1 -- 172.21.15.42:0/1799778268 wait complete. 2024-08-30T09:02:04.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.709+0000 7f7d15b44740 1 -- start start 2024-08-30T09:02:04.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.709+0000 7f7d15b44740 1 Processor -- start 2024-08-30T09:02:04.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.710+0000 7f7d15b44740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bc5cf811b0 0x55bc5d363280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:04.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.710+0000 7f7d15b44740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bc5cf89390 0x55bc5d3637c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:04.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.710+0000 7f7d15b44740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bc5cfaa070 0x55bc5d36f680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:04.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.710+0000 7f7d15b44740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55bc5d118700 con 0x55bc5cf89390 2024-08-30T09:02:04.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.710+0000 7f7d15b44740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55bc5d23c700 con 0x55bc5cf811b0 2024-08-30T09:02:04.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.710+0000 7f7d15b44740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55bc5d366300 con 0x55bc5cfaa070 2024-08-30T09:02:04.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.710+0000 7f7d0db6c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bc5cfaa070 0x55bc5d36f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:02:04.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.710+0000 7f7d15b44740 10 client.0 _ll_register_callbacks cb 0x55bc5d26bdc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:02:04.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.710+0000 7f7d0db6c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bc5cfaa070 0x55bc5d36f680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:33994/0 (socket says 172.21.15.42:33994) 2024-08-30T09:02:04.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.710+0000 7f7d0db6c640 1 -- 172.21.15.42:0/3206803734 learned_addr learned my addr 172.21.15.42:0/3206803734 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:02:04.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.710+0000 7f7d0db6c640 1 --2- 172.21.15.42:0/3206803734 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bc5cf811b0 0x55bc5d363280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:02:04.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.711+0000 7f7d0db6c640 1 -- 172.21.15.42:0/3206803734 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bc5cf811b0 msgr2=0x55bc5d363280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:02:04.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.711+0000 7f7d0db6c640 1 --2- 172.21.15.42:0/3206803734 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bc5cf811b0 0x55bc5d363280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:04.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.711+0000 7f7d0db6c640 1 -- 172.21.15.42:0/3206803734 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bc5cf89390 msgr2=0x55bc5d3637c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:02:04.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.711+0000 7f7d0db6c640 1 --2- 172.21.15.42:0/3206803734 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bc5cf89390 0x55bc5d3637c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:04.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.711+0000 7f7d0db6c640 1 -- 172.21.15.42:0/3206803734 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc5d2758b0 con 0x55bc5cfaa070 2024-08-30T09:02:04.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.712+0000 7f7d0db6c640 1 --2- 172.21.15.42:0/3206803734 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bc5cfaa070 0x55bc5d36f680 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x55bc5d356500 tx=0x7f7d0000a640 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a6e6d4cadd9fda6c server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:02:04.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.712+0000 7f7cf0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-08-30T09:02:04.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.712+0000 7f7cf0ff9640 1 -- 172.21.15.42:0/3206803734 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d00007730 con 0x55bc5cfaa070 2024-08-30T09:02:04.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.712+0000 7f7d15b44740 1 -- 172.21.15.42:0/3206803734 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7d00007a80 con 0x55bc5cfaa070 2024-08-30T09:02:04.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.713+0000 7f7cf0ff9640 1 -- 172.21.15.42:0/3206803734 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f7d00012040 con 0x55bc5cfaa070 2024-08-30T09:02:04.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.713+0000 7f7cf0ff9640 1 -- 172.21.15.42:0/3206803734 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f7d000078d0 con 0x55bc5cfaa070 2024-08-30T09:02:04.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.714+0000 7f7d15b44740 10 client.25345 fetch_fsmap learned FSMap version 29 2024-08-30T09:02:04.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.714+0000 7f7d15b44740 1 -- 172.21.15.42:0/3206803734 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55bc5d372140 con 0x55bc5cfaa070 2024-08-30T09:02:04.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.714+0000 7f7cf0ff9640 1 -- 172.21.15.42:0/3206803734 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d00007a80 con 0x55bc5cfaa070 2024-08-30T09:02:04.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.714+0000 7f7cf0ff9640 1 -- 172.21.15.42:0/3206803734 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f7d0001e070 con 0x55bc5cfaa070 2024-08-30T09:02:04.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.714+0000 7f7d15b44740 10 client.25345 fetch_fsmap finished waiting for FSMap version 29 2024-08-30T09:02:04.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.714+0000 7f7d15b44740 10 client.25345 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-30T09:02:04.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.714+0000 7f7d15b44740 20 client.25345 populate_metadata read hostname 'smithi042' 2024-08-30T09:02:04.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.714+0000 7f7d15b44740 1 --2- 172.21.15.42:0/3206803734 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55bc5cf689f0 0x55bc5d375a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:04.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.714+0000 7f7d15b44740 4 client.25345 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-30T09:02:04.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.714+0000 7f7d15b44740 1 -- 172.21.15.42:0/3206803734 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bc5d2c8580 con 0x55bc5cf689f0 2024-08-30T09:02:04.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.715+0000 7f7d0cb6a640 1 --2- 172.21.15.42:0/3206803734 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55bc5cf689f0 0x55bc5d375a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:02:04.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.715+0000 7f7cf0ff9640 10 client.25345 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:02:04.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.715+0000 7f7d0cb6a640 1 --2- 172.21.15.42:0/3206803734 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55bc5cf689f0 0x55bc5d375a30 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f08d2c9ac6a5feb0 server_cookie=91460e0d577358f6 in_seq=0 out_seq=0 2024-08-30T09:02:04.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.716+0000 7f7cf0ff9640 1 -- 172.21.15.42:0/3206803734 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55bc5d2c8580 con 0x55bc5cf689f0 2024-08-30T09:02:04.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.716+0000 7f7cf0ff9640 10 client.25345 handle_command_reply: tid=0 2024-08-30T09:02:04.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.716+0000 7f7d15b44740 1 client.25345 shutdown 2024-08-30T09:02:04.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.717+0000 7f7d15b44740 1 -- 172.21.15.42:0/3206803734 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bc5cfaa070 msgr2=0x55bc5d36f680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:02:04.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.717+0000 7f7d15b44740 1 --2- 172.21.15.42:0/3206803734 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bc5cfaa070 0x55bc5d36f680 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x55bc5d356500 tx=0x7f7d0000a640 comp rx=0 tx=0).stop 2024-08-30T09:02:04.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.717+0000 7f7d15b44740 1 -- 172.21.15.42:0/3206803734 shutdown_connections 2024-08-30T09:02:04.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.717+0000 7f7d15b44740 1 --2- 172.21.15.42:0/3206803734 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55bc5cf689f0 0x55bc5d375a30 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:04.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.717+0000 7f7d15b44740 1 --2- 172.21.15.42:0/3206803734 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bc5cfaa070 0x55bc5d36f680 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:04.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.717+0000 7f7d15b44740 1 --2- 172.21.15.42:0/3206803734 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bc5cf89390 0x55bc5d3637c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:04.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.717+0000 7f7d15b44740 1 --2- 172.21.15.42:0/3206803734 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bc5cf811b0 0x55bc5d363280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:04.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.717+0000 7f7d15b44740 1 -- 172.21.15.42:0/3206803734 >> 172.21.15.42:0/3206803734 conn(0x55bc5cf9e460 msgr2=0x55bc5d35ac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:02:04.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.717+0000 7f7cf0ff9640 0 client.25345 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:02:04.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.718+0000 7f7d15b44740 1 -- 172.21.15.42:0/3206803734 shutdown_connections 2024-08-30T09:02:04.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.718+0000 7f7d15b44740 1 -- 172.21.15.42:0/3206803734 wait complete. 2024-08-30T09:02:04.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.718+0000 7f7d15b44740 20 client.25345 trim_cache size 0 max 16384 2024-08-30T09:02:04.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.747+0000 7f7d15b44740 1 Processor -- start 2024-08-30T09:02:04.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.747+0000 7f7d15b44740 1 -- start start 2024-08-30T09:02:04.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.747+0000 7f7d15b44740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bc5cf811b0 0x55bc5d364eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:04.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.748+0000 7f7d15b44740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bc5cf89390 0x55bc5d36f580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:04.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.748+0000 7f7d15b44740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bc5cfaa070 0x55bc5d360470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:04.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.748+0000 7f7d15b44740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f7cec012550 con 0x55bc5cf811b0 2024-08-30T09:02:04.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.748+0000 7f7d15b44740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f7cec00fe20 con 0x55bc5cf89390 2024-08-30T09:02:04.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.748+0000 7f7d15b44740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f7cec00dc70 con 0x55bc5cfaa070 2024-08-30T09:02:04.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.749+0000 7f7d0db6c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bc5cfaa070 0x55bc5d360470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:02:04.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.749+0000 7f7d0db6c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bc5cf811b0 0x55bc5d364eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:02:04.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.749+0000 7f7d0db6c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bc5cf811b0 0x55bc5d364eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:44714/0 (socket says 172.21.15.42:44714) 2024-08-30T09:02:04.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.749+0000 7f7d0db6c640 1 -- 172.21.15.42:0/143518740 learned_addr learned my addr 172.21.15.42:0/143518740 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:02:04.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.749+0000 7f7d0cb6a640 1 --2- 172.21.15.42:0/143518740 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bc5cf89390 0x55bc5d36f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:02:04.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.750+0000 7f7d0db6c640 1 -- 172.21.15.42:0/143518740 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bc5cf89390 msgr2=0x55bc5d36f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:02:04.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.750+0000 7f7d0db6c640 1 --2- 172.21.15.42:0/143518740 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bc5cf89390 0x55bc5d36f580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:04.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.750+0000 7f7d0db6c640 1 -- 172.21.15.42:0/143518740 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bc5cf811b0 msgr2=0x55bc5d364eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:02:04.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.750+0000 7f7d0db6c640 1 --2- 172.21.15.42:0/143518740 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bc5cf811b0 0x55bc5d364eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:04.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.750+0000 7f7d0db6c640 1 -- 172.21.15.42:0/143518740 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc5d364540 con 0x55bc5cfaa070 2024-08-30T09:02:04.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.750+0000 7f7d0db6c640 1 --2- 172.21.15.42:0/143518740 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bc5cfaa070 0x55bc5d360470 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f7d000074f0 tx=0x7f7d0000c910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9fdf4a5a799a3d31 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:02:04.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.754+0000 7f7cf2ffd640 1 -- 172.21.15.42:0/143518740 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d00030070 con 0x55bc5cfaa070 2024-08-30T09:02:04.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.754+0000 7f7cf2ffd640 1 -- 172.21.15.42:0/143518740 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d00031040 con 0x55bc5cfaa070 2024-08-30T09:02:04.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.754+0000 7f7cf2ffd640 1 -- 172.21.15.42:0/143518740 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d00007a30 con 0x55bc5cfaa070 2024-08-30T09:02:04.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.756+0000 7f7d15b44740 1 -- 172.21.15.42:0/143518740 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bc5cfaa070 msgr2=0x55bc5d360470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:02:04.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.756+0000 7f7d15b44740 1 --2- 172.21.15.42:0/143518740 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bc5cfaa070 0x55bc5d360470 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f7d000074f0 tx=0x7f7d0000c910 comp rx=0 tx=0).stop 2024-08-30T09:02:04.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.757+0000 7f7d15b44740 1 -- 172.21.15.42:0/143518740 shutdown_connections 2024-08-30T09:02:04.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.757+0000 7f7d15b44740 1 --2- 172.21.15.42:0/143518740 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bc5cfaa070 0x55bc5d360470 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:04.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.757+0000 7f7d15b44740 1 --2- 172.21.15.42:0/143518740 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bc5cf89390 0x55bc5d36f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:04.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.758+0000 7f7d15b44740 1 --2- 172.21.15.42:0/143518740 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bc5cf811b0 0x55bc5d364eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:04.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.758+0000 7f7d15b44740 1 -- 172.21.15.42:0/143518740 >> 172.21.15.42:0/143518740 conn(0x55bc5cf9e460 msgr2=0x55bc5d3647a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:02:04.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.759+0000 7f7d15b44740 1 -- 172.21.15.42:0/143518740 shutdown_connections 2024-08-30T09:02:04.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.759+0000 7f7d15b44740 1 -- 172.21.15.42:0/143518740 wait complete. 2024-08-30T09:02:04.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.760+0000 7f7d15b44740 1 -- start start 2024-08-30T09:02:04.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.760+0000 7f7d15b44740 1 Processor -- start 2024-08-30T09:02:04.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.761+0000 7f7d15b44740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bc5cf811b0 0x55bc5d36bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:04.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.761+0000 7f7d15b44740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bc5cf89390 0x55bc5d366f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:04.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.762+0000 7f7d15b44740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bc5cfaa070 0x55bc5d3674a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:04.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.762+0000 7f7d15b44740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f7cec00e2e0 con 0x55bc5cfaa070 2024-08-30T09:02:04.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.762+0000 7f7d0db6c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bc5cf811b0 0x55bc5d36bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:02:04.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.762+0000 7f7d0db6c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bc5cf811b0 0x55bc5d36bab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:49960/0 (socket says 172.21.15.42:49960) 2024-08-30T09:02:04.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.762+0000 7f7d0db6c640 1 -- 172.21.15.42:0/2673817440 learned_addr learned my addr 172.21.15.42:0/2673817440 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:02:04.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.762+0000 7f7d15b44740 1 -- 172.21.15.42:0/2673817440 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f7cec00f750 con 0x55bc5cf811b0 2024-08-30T09:02:04.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.762+0000 7f7d15b44740 1 -- 172.21.15.42:0/2673817440 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f7cec00e9b0 con 0x55bc5cf89390 2024-08-30T09:02:04.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.762+0000 7f7d0cb6a640 1 --2- 172.21.15.42:0/2673817440 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bc5cf89390 0x55bc5d366f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:02:04.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.762+0000 7f7d15b44740 10 client.0 _ll_register_callbacks cb 0x7f7d040eb420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:02:04.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.763+0000 7f7d0cb6a640 1 -- 172.21.15.42:0/2673817440 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bc5cf811b0 msgr2=0x55bc5d36bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:02:04.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.763+0000 7f7d0cb6a640 1 --2- 172.21.15.42:0/2673817440 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bc5cf811b0 0x55bc5d36bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:04.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.763+0000 7f7d0cb6a640 1 -- 172.21.15.42:0/2673817440 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bc5cfaa070 msgr2=0x55bc5d3674a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:02:04.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.763+0000 7f7d0cb6a640 1 --2- 172.21.15.42:0/2673817440 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bc5cfaa070 0x55bc5d3674a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:04.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.763+0000 7f7d0cb6a640 1 -- 172.21.15.42:0/2673817440 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc5d3679e0 con 0x55bc5cf89390 2024-08-30T09:02:04.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.764+0000 7f7d0cb6a640 1 --2- 172.21.15.42:0/2673817440 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bc5cf89390 0x55bc5d366f60 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f7cf802a9e0 tx=0x7f7cf8030710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=22526aba9870efea server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:02:04.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.764+0000 7f7cf0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-08-30T09:02:04.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.764+0000 7f7cf0ff9640 1 -- 172.21.15.42:0/2673817440 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cf8005040 con 0x55bc5cf89390 2024-08-30T09:02:04.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.764+0000 7f7cf0ff9640 1 -- 172.21.15.42:0/2673817440 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f7cf80051e0 con 0x55bc5cf89390 2024-08-30T09:02:04.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.765+0000 7f7d15b44740 1 -- 172.21.15.42:0/2673817440 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7d00031040 con 0x55bc5cf89390 2024-08-30T09:02:04.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.768+0000 7f7cf0ff9640 1 -- 172.21.15.42:0/2673817440 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f7cf80053e0 con 0x55bc5cf89390 2024-08-30T09:02:04.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.769+0000 7f7d15b44740 10 client.25351 fetch_fsmap learned FSMap version 29 2024-08-30T09:02:04.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.769+0000 7f7d15b44740 1 -- 172.21.15.42:0/2673817440 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55bc5d3dffe0 con 0x55bc5cf89390 2024-08-30T09:02:04.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.769+0000 7f7cf0ff9640 1 -- 172.21.15.42:0/2673817440 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d00031040 con 0x55bc5cf89390 2024-08-30T09:02:04.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.770+0000 7f7cf0ff9640 1 -- 172.21.15.42:0/2673817440 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f7cf803d630 con 0x55bc5cf89390 2024-08-30T09:02:04.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.770+0000 7f7d15b44740 10 client.25351 fetch_fsmap finished waiting for FSMap version 29 2024-08-30T09:02:04.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.770+0000 7f7d15b44740 10 client.25351 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-30T09:02:04.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.770+0000 7f7d15b44740 20 client.25351 populate_metadata read hostname 'smithi042' 2024-08-30T09:02:04.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.770+0000 7f7d15b44740 1 --2- 172.21.15.42:0/2673817440 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55bc5cf3dda0 0x55bc5d3e27e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:04.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.770+0000 7f7d15b44740 4 client.25351 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-30T09:02:04.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.770+0000 7f7d15b44740 1 -- 172.21.15.42:0/2673817440 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55bc5d3641f0 con 0x55bc5cf3dda0 2024-08-30T09:02:04.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.771+0000 7f7d0db6c640 1 --2- 172.21.15.42:0/2673817440 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55bc5cf3dda0 0x55bc5d3e27e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:02:04.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.780+0000 7f7d0db6c640 1 --2- 172.21.15.42:0/2673817440 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55bc5cf3dda0 0x55bc5d3e27e0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ffc72f2a5cba3a49 server_cookie=11c1594a929542e6 in_seq=0 out_seq=0 2024-08-30T09:02:04.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.780+0000 7f7cf0ff9640 10 client.25351 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:02:04.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.782+0000 7f7cf0ff9640 1 -- 172.21.15.42:0/2673817440 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+658 (crc 0 0 0) 0x55bc5d3641f0 con 0x55bc5cf3dda0 2024-08-30T09:02:04.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.782+0000 7f7cf0ff9640 10 client.25351 handle_command_reply: tid=0 2024-08-30T09:02:04.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.782+0000 7f7d15b44740 1 client.25351 shutdown 2024-08-30T09:02:04.791 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:02:04 smithi042 ceph-mon[18652]: pgmap v412: 129 pgs: 129 active+clean; 1.0 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.7 MiB/s wr, 783 op/s 2024-08-30T09:02:04.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.788+0000 7f7d15b44740 1 -- 172.21.15.42:0/2673817440 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bc5cf89390 msgr2=0x55bc5d366f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:02:04.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.788+0000 7f7d15b44740 1 --2- 172.21.15.42:0/2673817440 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bc5cf89390 0x55bc5d366f60 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f7cf802a9e0 tx=0x7f7cf8030710 comp rx=0 tx=0).stop 2024-08-30T09:02:04.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.788+0000 7f7d15b44740 1 -- 172.21.15.42:0/2673817440 shutdown_connections 2024-08-30T09:02:04.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.788+0000 7f7d15b44740 1 --2- 172.21.15.42:0/2673817440 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55bc5cf3dda0 0x55bc5d3e27e0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:04.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.788+0000 7f7d15b44740 1 --2- 172.21.15.42:0/2673817440 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bc5cf89390 0x55bc5d366f60 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:04.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.788+0000 7f7d15b44740 1 --2- 172.21.15.42:0/2673817440 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bc5cfaa070 0x55bc5d3674a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:04.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.788+0000 7f7d15b44740 1 --2- 172.21.15.42:0/2673817440 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bc5cf811b0 0x55bc5d36bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:04.792 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.788+0000 7f7d15b44740 1 -- 172.21.15.42:0/2673817440 >> 172.21.15.42:0/2673817440 conn(0x55bc5cf9e460 msgr2=0x55bc5d3650d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:02:04.792 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.788+0000 7f7cf0ff9640 0 client.25351 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:02:04.792 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.789+0000 7f7d15b44740 1 -- 172.21.15.42:0/2673817440 shutdown_connections 2024-08-30T09:02:04.792 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.789+0000 7f7d15b44740 1 -- 172.21.15.42:0/2673817440 wait complete. 2024-08-30T09:02:04.792 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.790+0000 7f7d15b44740 20 client.25351 trim_cache size 0 max 16384 2024-08-30T09:02:04.794 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-08-30T09:02:04.794 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "no active scrubs running", 2024-08-30T09:02:04.794 INFO:teuthology.orchestra.run.smithi042.stdout: "scrubs": {}, 2024-08-30T09:02:04.794 INFO:teuthology.orchestra.run.smithi042.stdout: "scrub_stats": { 2024-08-30T09:02:04.794 INFO:teuthology.orchestra.run.smithi042.stdout: "85134e4b-ab4a-4885-b07e-a3ad01eaf13d": { 2024-08-30T09:02:04.794 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:01:27.738462+0000", 2024-08-30T09:02:04.794 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:02:04.794 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:02:04.794 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:02:04.794 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:02:04.795 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:02:04.795 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:02:04.795 INFO:teuthology.orchestra.run.smithi042.stdout: "5e2ed104-0625-478a-a78b-584e7f4c2817": { 2024-08-30T09:02:04.795 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:00:33.928400+0000", 2024-08-30T09:02:04.795 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:02:04.795 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:02:04.795 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:02:04.795 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:02:04.795 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:02:04.795 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:02:04.795 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:02:04.795 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-08-30T09:02:04.796 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.793+0000 7f7d0f5cf640 1 -- 172.21.15.42:0/321154225 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f7cf40b4bf0 msgr2=0x7f7cf40b7100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:02:04.796 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.793+0000 7f7d0f5cf640 1 --2- 172.21.15.42:0/321154225 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f7cf40b4bf0 0x7f7cf40b7100 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x55bc5d356750 tx=0x7f7d040eb550 comp rx=0 tx=0).stop 2024-08-30T09:02:04.796 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.793+0000 7f7d0f5cf640 1 -- 172.21.15.42:0/321154225 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7d081023b0 msgr2=0x7f7d0819c530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:02:04.796 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.793+0000 7f7d0f5cf640 1 --2- 172.21.15.42:0/321154225 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7d081023b0 0x7f7d0819c530 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f7d0400e9c0 tx=0x7f7d0400ee80 comp rx=0 tx=0).stop 2024-08-30T09:02:04.796 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.794+0000 7f7d0f5cf640 1 -- 172.21.15.42:0/321154225 shutdown_connections 2024-08-30T09:02:04.797 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.794+0000 7f7d0f5cf640 1 --2- 172.21.15.42:0/321154225 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f7cf40b4bf0 0x7f7cf40b7100 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:04.797 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.794+0000 7f7d0f5cf640 1 --2- 172.21.15.42:0/321154225 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7d08102d00 0x7f7d0819cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:04.797 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.794+0000 7f7d0f5cf640 1 --2- 172.21.15.42:0/321154225 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7d0810ff60 0x7f7d081a1d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:04.797 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.794+0000 7f7d0f5cf640 1 --2- 172.21.15.42:0/321154225 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7d081023b0 0x7f7d0819c530 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:04.797 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.794+0000 7f7d0f5cf640 1 -- 172.21.15.42:0/321154225 >> 172.21.15.42:0/321154225 conn(0x7f7d08079580 msgr2=0x7f7d08109a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:02:04.797 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.795+0000 7f7d0f5cf640 1 -- 172.21.15.42:0/321154225 shutdown_connections 2024-08-30T09:02:04.797 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:04.795+0000 7f7d0f5cf640 1 -- 172.21.15.42:0/321154225 wait complete. 2024-08-30T09:02:04.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:02:04 smithi190 ceph-mon[22001]: pgmap v412: 129 pgs: 129 active+clean; 1.0 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.7 MiB/s wr, 783 op/s 2024-08-30T09:02:05.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:02:04 smithi167 ceph-mon[22063]: pgmap v412: 129 pgs: 129 active+clean; 1.0 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.7 MiB/s wr, 783 op/s 2024-08-30T09:02:05.953 INFO:teuthology.orchestra.run.smithi042.stderr:progress: 80.0 s, 2420.0 tps, lat 40.993 ms stddev 51.599 2024-08-30T09:02:06.024 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.022+0000 7fb808778640 1 -- 172.21.15.42:0/2352057922 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb80010aae0 msgr2=0x7fb80010aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:02:06.025 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.022+0000 7fb808778640 1 --2- 172.21.15.42:0/2352057922 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb80010aae0 0x7fb80010aef0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fb7fc009900 tx=0x7fb7fc02a440 comp rx=0 tx=0).stop 2024-08-30T09:02:06.027 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.025+0000 7fb808778640 1 -- 172.21.15.42:0/2352057922 shutdown_connections 2024-08-30T09:02:06.027 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.025+0000 7fb808778640 1 --2- 172.21.15.42:0/2352057922 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb80010b480 0x7fb800111e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:06.027 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.025+0000 7fb808778640 1 --2- 172.21.15.42:0/2352057922 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb80010aae0 0x7fb80010aef0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:06.027 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.025+0000 7fb808778640 1 --2- 172.21.15.42:0/2352057922 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb800109a60 0x7fb800109ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:06.027 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.025+0000 7fb808778640 1 -- 172.21.15.42:0/2352057922 >> 172.21.15.42:0/2352057922 conn(0x7fb800078db0 msgr2=0x7fb80007b280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:02:06.029 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.027+0000 7fb808778640 1 -- 172.21.15.42:0/2352057922 shutdown_connections 2024-08-30T09:02:06.030 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.028+0000 7fb808778640 1 -- 172.21.15.42:0/2352057922 wait complete. 2024-08-30T09:02:06.030 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.029+0000 7fb808778640 1 Processor -- start 2024-08-30T09:02:06.032 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.029+0000 7fb808778640 1 -- start start 2024-08-30T09:02:06.032 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.029+0000 7fb808778640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb800109a60 0x7fb800072170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:06.032 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.029+0000 7fb808778640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb80010aae0 0x7fb8000727e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:06.032 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.029+0000 7fb808778640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb80010b480 0x7fb800073340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:06.032 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.029+0000 7fb808778640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fb800115810 con 0x7fb800109a60 2024-08-30T09:02:06.032 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.029+0000 7fb808778640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb800115690 con 0x7fb80010b480 2024-08-30T09:02:06.032 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.029+0000 7fb808778640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fb800115990 con 0x7fb80010aae0 2024-08-30T09:02:06.032 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.030+0000 7fb805d13640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb80010aae0 0x7fb8000727e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:02:06.032 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.030+0000 7fb805d13640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb80010aae0 0x7fb8000727e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:34022/0 (socket says 172.21.15.42:34022) 2024-08-30T09:02:06.032 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.030+0000 7fb805d13640 1 -- 172.21.15.42:0/3141724961 learned_addr learned my addr 172.21.15.42:0/3141724961 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:02:06.033 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.030+0000 7fb805d13640 1 -- 172.21.15.42:0/3141724961 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb80010b480 msgr2=0x7fb800073340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:02:06.033 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.030+0000 7fb805d13640 1 --2- 172.21.15.42:0/3141724961 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb80010b480 0x7fb800073340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:06.033 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.030+0000 7fb805d13640 1 -- 172.21.15.42:0/3141724961 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb800109a60 msgr2=0x7fb800072170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:02:06.034 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.030+0000 7fb805d13640 1 --2- 172.21.15.42:0/3141724961 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb800109a60 0x7fb800072170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:06.034 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.030+0000 7fb805d13640 1 -- 172.21.15.42:0/3141724961 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb800073b60 con 0x7fb80010aae0 2024-08-30T09:02:06.034 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.030+0000 7fb805d13640 1 --2- 172.21.15.42:0/3141724961 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb80010aae0 0x7fb8000727e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fb7fc031440 tx=0x7fb7fc003dc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:02:06.034 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.031+0000 7fb7f77fe640 1 -- 172.21.15.42:0/3141724961 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7fc03f070 con 0x7fb80010aae0 2024-08-30T09:02:06.034 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.031+0000 7fb808778640 1 -- 172.21.15.42:0/3141724961 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb800073df0 con 0x7fb80010aae0 2024-08-30T09:02:06.034 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.031+0000 7fb808778640 1 -- 172.21.15.42:0/3141724961 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb800074100 con 0x7fb80010aae0 2024-08-30T09:02:06.034 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.032+0000 7fb7f77fe640 1 -- 172.21.15.42:0/3141724961 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb7fc03b460 con 0x7fb80010aae0 2024-08-30T09:02:06.034 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.032+0000 7fb7f77fe640 1 -- 172.21.15.42:0/3141724961 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7fc035030 con 0x7fb80010aae0 2024-08-30T09:02:06.044 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.041+0000 7fb80f322740 1 Processor -- start 2024-08-30T09:02:06.045 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.043+0000 7fb80f322740 1 -- start start 2024-08-30T09:02:06.045 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.043+0000 7fb7f77fe640 1 -- 172.21.15.42:0/3141724961 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7fb7fc042070 con 0x7fb80010aae0 2024-08-30T09:02:06.045 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.043+0000 7fb80f322740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b0ad8eb1b0 0x55b0adc90f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:06.045 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.043+0000 7fb80f322740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b0ad8c22f0 0x55b0adc915d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:06.045 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.043+0000 7fb80f322740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b0ad8ca4d0 0x55b0adc92120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:06.046 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.043+0000 7fb80f322740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55b0adc04000 con 0x55b0ad8ca4d0 2024-08-30T09:02:06.046 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.043+0000 7fb80f322740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55b0adbf3590 con 0x55b0ad8c22f0 2024-08-30T09:02:06.046 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.043+0000 7fb80f322740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55b0adbf7540 con 0x55b0ad8eb1b0 2024-08-30T09:02:06.046 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.044+0000 7fb806d15640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b0ad8eb1b0 0x55b0adc90f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:02:06.046 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.044+0000 7fb806d15640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b0ad8eb1b0 0x55b0adc90f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:34024/0 (socket says 172.21.15.42:34024) 2024-08-30T09:02:06.046 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.044+0000 7fb806d15640 1 -- 172.21.15.42:0/2378886832 learned_addr learned my addr 172.21.15.42:0/2378886832 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:02:06.046 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.045+0000 7fb806d15640 1 -- 172.21.15.42:0/2378886832 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b0ad8c22f0 msgr2=0x55b0adc915d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-30T09:02:06.047 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.045+0000 7fb806d15640 1 --2- 172.21.15.42:0/2378886832 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b0ad8c22f0 0x55b0adc915d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:06.047 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.045+0000 7fb806d15640 1 -- 172.21.15.42:0/2378886832 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b0ad8ca4d0 msgr2=0x55b0adc92120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:02:06.047 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.045+0000 7fb806d15640 1 --2- 172.21.15.42:0/2378886832 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b0ad8ca4d0 0x55b0adc92120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:06.047 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.045+0000 7fb806d15640 1 -- 172.21.15.42:0/2378886832 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b0ad956790 con 0x55b0ad8eb1b0 2024-08-30T09:02:06.047 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.045+0000 7fb806d15640 1 --2- 172.21.15.42:0/2378886832 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b0ad8eb1b0 0x55b0adc90f60 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fb7f8009d00 tx=0x7fb7f800b420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=90175a2019e0cc98 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:02:06.047 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.045+0000 7fb7f4ff9640 1 -- 172.21.15.42:0/2378886832 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7f800bb30 con 0x55b0ad8eb1b0 2024-08-30T09:02:06.047 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.046+0000 7fb7f4ff9640 1 -- 172.21.15.42:0/2378886832 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb7f8017070 con 0x55b0ad8eb1b0 2024-08-30T09:02:06.049 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.046+0000 7fb7f77fe640 1 --2- 172.21.15.42:0/3141724961 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fb7ec0b4d40 0x7fb7ec0b7250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:06.049 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.047+0000 7fb7f77fe640 1 -- 172.21.15.42:0/3141724961 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7fb7fc105fd0 con 0x7fb80010aae0 2024-08-30T09:02:06.049 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.047+0000 7fb80f322740 1 -- 172.21.15.42:0/2378886832 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b0ad8eb1b0 msgr2=0x55b0adc90f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:02:06.049 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.047+0000 7fb80f322740 1 --2- 172.21.15.42:0/2378886832 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b0ad8eb1b0 0x55b0adc90f60 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fb7f8009d00 tx=0x7fb7f800b420 comp rx=0 tx=0).stop 2024-08-30T09:02:06.049 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.048+0000 7fb80f322740 1 -- 172.21.15.42:0/2378886832 shutdown_connections 2024-08-30T09:02:06.050 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.048+0000 7fb80f322740 1 --2- 172.21.15.42:0/2378886832 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b0ad8ca4d0 0x55b0adc92120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:06.050 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.048+0000 7fb80f322740 1 --2- 172.21.15.42:0/2378886832 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b0ad8c22f0 0x55b0adc915d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:06.050 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.048+0000 7fb80f322740 1 --2- 172.21.15.42:0/2378886832 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b0ad8eb1b0 0x55b0adc90f60 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:06.050 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.048+0000 7fb80f322740 1 -- 172.21.15.42:0/2378886832 >> 172.21.15.42:0/2378886832 conn(0x55b0ad8df300 msgr2=0x55b0adc8c9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:02:06.051 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.049+0000 7fb806514640 1 --2- 172.21.15.42:0/3141724961 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fb7ec0b4d40 0x7fb7ec0b7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:02:06.051 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.050+0000 7fb80f322740 1 -- 172.21.15.42:0/2378886832 shutdown_connections 2024-08-30T09:02:06.052 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.050+0000 7fb80f322740 1 -- 172.21.15.42:0/2378886832 wait complete. 2024-08-30T09:02:06.052 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.050+0000 7fb80f322740 1 -- start start 2024-08-30T09:02:06.052 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.051+0000 7fb80f322740 1 Processor -- start 2024-08-30T09:02:06.053 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.051+0000 7fb806514640 1 --2- 172.21.15.42:0/3141724961 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fb7ec0b4d40 0x7fb7ec0b7250 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fb7f0006fd0 tx=0x7fb7f0006c40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:02:06.053 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.051+0000 7fb80f322740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b0ad8c22f0 0x55b0adc9ab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:06.053 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.051+0000 7fb80f322740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b0ad8ca4d0 0x55b0adc9b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:06.053 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.051+0000 7fb80f322740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b0ad8eb1b0 0x55b0adc9bd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:06.053 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.051+0000 7fb80f322740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55b0adb2cab0 con 0x55b0ad8eb1b0 2024-08-30T09:02:06.053 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.051+0000 7fb80f322740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55b0adb7a680 con 0x55b0ad8c22f0 2024-08-30T09:02:06.054 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.051+0000 7fb80f322740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55b0adc92c90 con 0x55b0ad8ca4d0 2024-08-30T09:02:06.054 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.051+0000 7fb80f322740 10 client.0 _ll_register_callbacks cb 0x55b0ad902a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:02:06.054 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.051+0000 7fb806514640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b0ad8c22f0 0x55b0adc9ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:02:06.054 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.051+0000 7fb806d15640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b0ad8eb1b0 0x55b0adc9bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:02:06.054 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.051+0000 7fb805d13640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b0ad8ca4d0 0x55b0adc9b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:02:06.054 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.051+0000 7fb806514640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b0ad8c22f0 0x55b0adc9ab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:50000/0 (socket says 172.21.15.42:50000) 2024-08-30T09:02:06.054 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.051+0000 7fb806514640 1 -- 172.21.15.42:0/1771248918 learned_addr learned my addr 172.21.15.42:0/1771248918 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:02:06.054 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.051+0000 7fb806d15640 1 --2- 172.21.15.42:0/1771248918 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b0ad8eb1b0 0x55b0adc9bd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:44742/0 (socket says 172.21.15.42:44742) 2024-08-30T09:02:06.054 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.052+0000 7fb805d13640 1 -- 172.21.15.42:0/1771248918 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b0ad8c22f0 msgr2=0x55b0adc9ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:02:06.055 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.052+0000 7fb805d13640 1 --2- 172.21.15.42:0/1771248918 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b0ad8c22f0 0x55b0adc9ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:06.055 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.052+0000 7fb805d13640 1 -- 172.21.15.42:0/1771248918 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b0ad8eb1b0 msgr2=0x55b0adc9bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:02:06.055 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.052+0000 7fb805d13640 1 --2- 172.21.15.42:0/1771248918 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b0ad8eb1b0 0x55b0adc9bd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:06.055 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.052+0000 7fb805d13640 1 -- 172.21.15.42:0/1771248918 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b0ad9cf620 con 0x55b0ad8ca4d0 2024-08-30T09:02:06.055 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.052+0000 7fb806d15640 1 --2- 172.21.15.42:0/1771248918 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b0ad8eb1b0 0x55b0adc9bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-30T09:02:06.056 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.053+0000 7fb805d13640 1 --2- 172.21.15.42:0/1771248918 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b0ad8ca4d0 0x55b0adc9b1f0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fb7fc02aa30 tx=0x7fb7fc0cc020 comp rx=0 tx=0).ready entity=mon.1 client_cookie=761d39e99916fb65 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:02:06.056 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.054+0000 7fb7e9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-08-30T09:02:06.056 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.054+0000 7fb7e9ffb640 1 -- 172.21.15.42:0/1771248918 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7fc049070 con 0x55b0ad8ca4d0 2024-08-30T09:02:06.056 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.054+0000 7fb7e9ffb640 1 -- 172.21.15.42:0/1771248918 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7fb7fc03b8f0 con 0x55b0ad8ca4d0 2024-08-30T09:02:06.056 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.054+0000 7fb80f322740 1 -- 172.21.15.42:0/1771248918 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb7f8017070 con 0x55b0ad8ca4d0 2024-08-30T09:02:06.056 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.054+0000 7fb7e9ffb640 1 -- 172.21.15.42:0/1771248918 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb7fc0cc120 con 0x55b0ad8ca4d0 2024-08-30T09:02:06.056 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.054+0000 7fb80f322740 10 client.25360 fetch_fsmap learned FSMap version 29 2024-08-30T09:02:06.056 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.054+0000 7fb80f322740 1 -- 172.21.15.42:0/1771248918 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b0adcb5cc0 con 0x55b0ad8ca4d0 2024-08-30T09:02:06.057 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.054+0000 7fb7e9ffb640 1 -- 172.21.15.42:0/1771248918 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb7f8017070 con 0x55b0ad8ca4d0 2024-08-30T09:02:06.057 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.055+0000 7fb7e9ffb640 1 -- 172.21.15.42:0/1771248918 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fb7fc0f3170 con 0x55b0ad8ca4d0 2024-08-30T09:02:06.057 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.055+0000 7fb80f322740 10 client.25360 fetch_fsmap finished waiting for FSMap version 29 2024-08-30T09:02:06.057 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.055+0000 7fb80f322740 10 client.25360 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-30T09:02:06.057 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.055+0000 7fb80f322740 20 client.25360 populate_metadata read hostname 'smithi042' 2024-08-30T09:02:06.057 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.055+0000 7fb80f322740 1 --2- 172.21.15.42:0/1771248918 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55b0ad8a9d80 0x55b0adcb9440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:06.058 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.055+0000 7fb80f322740 4 client.25360 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-30T09:02:06.058 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.055+0000 7fb80f322740 1 -- 172.21.15.42:0/1771248918 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b0adc05f50 con 0x55b0ad8a9d80 2024-08-30T09:02:06.058 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.055+0000 7fb806514640 1 --2- 172.21.15.42:0/1771248918 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55b0ad8a9d80 0x55b0adcb9440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:02:06.058 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.056+0000 7fb7e9ffb640 10 client.25360 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:02:06.058 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.056+0000 7fb806514640 1 --2- 172.21.15.42:0/1771248918 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55b0ad8a9d80 0x55b0adcb9440 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=68bd043effd4806 server_cookie=b6ea91ed687d7ff8 in_seq=0 out_seq=0 2024-08-30T09:02:06.059 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.057+0000 7fb7e9ffb640 1 -- 172.21.15.42:0/1771248918 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b0adc05f50 con 0x55b0ad8a9d80 2024-08-30T09:02:06.059 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.057+0000 7fb7e9ffb640 10 client.25360 handle_command_reply: tid=0 2024-08-30T09:02:06.059 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.057+0000 7fb80f322740 1 client.25360 shutdown 2024-08-30T09:02:06.059 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.057+0000 7fb80f322740 1 -- 172.21.15.42:0/1771248918 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b0ad8ca4d0 msgr2=0x55b0adc9b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:02:06.059 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.057+0000 7fb80f322740 1 --2- 172.21.15.42:0/1771248918 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b0ad8ca4d0 0x55b0adc9b1f0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fb7fc02aa30 tx=0x7fb7fc0cc020 comp rx=0 tx=0).stop 2024-08-30T09:02:06.061 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.059+0000 7fb80f322740 1 -- 172.21.15.42:0/1771248918 shutdown_connections 2024-08-30T09:02:06.061 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.059+0000 7fb80f322740 1 --2- 172.21.15.42:0/1771248918 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55b0ad8a9d80 0x55b0adcb9440 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:06.061 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.059+0000 7fb80f322740 1 --2- 172.21.15.42:0/1771248918 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b0ad8ca4d0 0x55b0adc9b1f0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:06.061 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.059+0000 7fb80f322740 1 --2- 172.21.15.42:0/1771248918 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b0ad8eb1b0 0x55b0adc9bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:06.061 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.059+0000 7fb80f322740 1 --2- 172.21.15.42:0/1771248918 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b0ad8c22f0 0x55b0adc9ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:06.061 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.059+0000 7fb80f322740 1 -- 172.21.15.42:0/1771248918 >> 172.21.15.42:0/1771248918 conn(0x55b0ad8df300 msgr2=0x55b0adc8abb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:02:06.061 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.059+0000 7fb7e9ffb640 0 client.25360 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:02:06.061 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.059+0000 7fb80f322740 1 -- 172.21.15.42:0/1771248918 shutdown_connections 2024-08-30T09:02:06.061 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.059+0000 7fb80f322740 1 -- 172.21.15.42:0/1771248918 wait complete. 2024-08-30T09:02:06.062 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.059+0000 7fb80f322740 20 client.25360 trim_cache size 0 max 16384 2024-08-30T09:02:06.090 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.088+0000 7fb80f322740 1 Processor -- start 2024-08-30T09:02:06.090 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.088+0000 7fb80f322740 1 -- start start 2024-08-30T09:02:06.092 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.089+0000 7fb80f322740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b0ad8c22f0 0x55b0adc8bb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:06.092 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.089+0000 7fb80f322740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b0ad8ca4d0 0x55b0adc90b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:06.092 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.089+0000 7fb80f322740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b0ad8eb1b0 0x55b0adc957d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:06.092 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.089+0000 7fb80f322740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fb7e4012450 con 0x55b0ad8c22f0 2024-08-30T09:02:06.092 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.089+0000 7fb80f322740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb7e400fd20 con 0x55b0ad8ca4d0 2024-08-30T09:02:06.093 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.089+0000 7fb805d13640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b0ad8ca4d0 0x55b0adc90b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:02:06.093 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.089+0000 7fb805d13640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b0ad8ca4d0 0x55b0adc90b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:50006/0 (socket says 172.21.15.42:50006) 2024-08-30T09:02:06.093 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.089+0000 7fb805d13640 1 -- 172.21.15.42:0/1754366943 learned_addr learned my addr 172.21.15.42:0/1754366943 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:02:06.093 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.089+0000 7fb806d15640 1 --2- 172.21.15.42:0/1754366943 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b0ad8eb1b0 0x55b0adc957d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:02:06.093 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.089+0000 7fb80f322740 1 -- 172.21.15.42:0/1754366943 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fb7e400db70 con 0x55b0ad8eb1b0 2024-08-30T09:02:06.093 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.090+0000 7fb805d13640 1 -- 172.21.15.42:0/1754366943 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b0ad8eb1b0 msgr2=0x55b0adc957d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:02:06.093 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.090+0000 7fb805d13640 1 --2- 172.21.15.42:0/1754366943 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b0ad8eb1b0 0x55b0adc957d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:06.093 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.090+0000 7fb805d13640 1 -- 172.21.15.42:0/1754366943 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b0ad8c22f0 msgr2=0x55b0adc8bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:02:06.093 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.090+0000 7fb805d13640 1 --2- 172.21.15.42:0/1754366943 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b0ad8c22f0 0x55b0adc8bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:06.093 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.090+0000 7fb805d13640 1 -- 172.21.15.42:0/1754366943 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b0adc9c0d0 con 0x55b0ad8ca4d0 2024-08-30T09:02:06.093 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.090+0000 7fb805d13640 1 --2- 172.21.15.42:0/1754366943 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b0ad8ca4d0 0x55b0adc90b50 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fb7fc0d5570 tx=0x7fb7fc000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=af8cf0475db4cc6e server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:02:06.093 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.090+0000 7fb7ebfff640 1 -- 172.21.15.42:0/1754366943 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7fc07c070 con 0x55b0ad8ca4d0 2024-08-30T09:02:06.093 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.090+0000 7fb7ebfff640 1 -- 172.21.15.42:0/1754366943 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb7fc0ccd10 con 0x55b0ad8ca4d0 2024-08-30T09:02:06.094 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.090+0000 7fb7ebfff640 1 -- 172.21.15.42:0/1754366943 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7fc049070 con 0x55b0ad8ca4d0 2024-08-30T09:02:06.095 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.092+0000 7fb80f322740 1 -- 172.21.15.42:0/1754366943 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b0ad8ca4d0 msgr2=0x55b0adc90b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:02:06.095 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.092+0000 7fb80f322740 1 --2- 172.21.15.42:0/1754366943 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b0ad8ca4d0 0x55b0adc90b50 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fb7fc0d5570 tx=0x7fb7fc000bc0 comp rx=0 tx=0).stop 2024-08-30T09:02:06.095 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.092+0000 7fb80f322740 1 -- 172.21.15.42:0/1754366943 shutdown_connections 2024-08-30T09:02:06.095 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.092+0000 7fb80f322740 1 --2- 172.21.15.42:0/1754366943 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b0ad8eb1b0 0x55b0adc957d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:06.095 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.092+0000 7fb80f322740 1 --2- 172.21.15.42:0/1754366943 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b0ad8ca4d0 0x55b0adc90b50 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:06.095 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.092+0000 7fb80f322740 1 --2- 172.21.15.42:0/1754366943 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b0ad8c22f0 0x55b0adc8bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:06.095 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.092+0000 7fb80f322740 1 -- 172.21.15.42:0/1754366943 >> 172.21.15.42:0/1754366943 conn(0x55b0ad8df300 msgr2=0x55b0adc8c330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:02:06.096 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.092+0000 7fb80f322740 1 -- 172.21.15.42:0/1754366943 shutdown_connections 2024-08-30T09:02:06.096 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.092+0000 7fb80f322740 1 -- 172.21.15.42:0/1754366943 wait complete. 2024-08-30T09:02:06.096 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.092+0000 7fb80f322740 1 -- start start 2024-08-30T09:02:06.096 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.092+0000 7fb80f322740 1 Processor -- start 2024-08-30T09:02:06.096 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.093+0000 7fb80f322740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b0ad8c22f0 0x55b0adc9eac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:06.096 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.093+0000 7fb80f322740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b0ad8ca4d0 0x55b0adc9f020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:06.096 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.093+0000 7fb80f322740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b0ad8eb1b0 0x55b0adc9fb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:06.096 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.093+0000 7fb80f322740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fb7e400e1e0 con 0x55b0ad8eb1b0 2024-08-30T09:02:06.096 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.093+0000 7fb80f322740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb7e400f650 con 0x55b0ad8c22f0 2024-08-30T09:02:06.096 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.093+0000 7fb80f322740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fb7e400e8b0 con 0x55b0ad8ca4d0 2024-08-30T09:02:06.096 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.093+0000 7fb806514640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b0ad8c22f0 0x55b0adc9eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:02:06.096 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.093+0000 7fb80f322740 10 client.0 _ll_register_callbacks cb 0x7fb7fc005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:02:06.097 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.093+0000 7fb806514640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b0ad8c22f0 0x55b0adc9eac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:50020/0 (socket says 172.21.15.42:50020) 2024-08-30T09:02:06.097 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.093+0000 7fb806514640 1 -- 172.21.15.42:0/1460715289 learned_addr learned my addr 172.21.15.42:0/1460715289 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:02:06.097 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.093+0000 7fb805d13640 1 --2- 172.21.15.42:0/1460715289 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b0ad8ca4d0 0x55b0adc9f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:02:06.098 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.094+0000 7fb806514640 1 -- 172.21.15.42:0/1460715289 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b0ad8ca4d0 msgr2=0x55b0adc9f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:02:06.098 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.094+0000 7fb806514640 1 --2- 172.21.15.42:0/1460715289 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b0ad8ca4d0 0x55b0adc9f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:06.098 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.094+0000 7fb806514640 1 -- 172.21.15.42:0/1460715289 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b0ad8eb1b0 msgr2=0x55b0adc9fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:02:06.099 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.094+0000 7fb806514640 1 --2- 172.21.15.42:0/1460715289 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b0ad8eb1b0 0x55b0adc9fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:06.099 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.094+0000 7fb806514640 1 -- 172.21.15.42:0/1460715289 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b0adca01e0 con 0x55b0ad8c22f0 2024-08-30T09:02:06.099 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.095+0000 7fb806514640 1 --2- 172.21.15.42:0/1460715289 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b0ad8c22f0 0x55b0adc9eac0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb7f0006e60 tx=0x7fb7f0004770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3dc33308bd8b1e41 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:02:06.099 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.095+0000 7fb7e9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-30T09:02:06.099 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.095+0000 7fb7e9ffb640 1 -- 172.21.15.42:0/1460715289 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7f002c020 con 0x55b0ad8c22f0 2024-08-30T09:02:06.099 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.095+0000 7fb7e9ffb640 1 -- 172.21.15.42:0/1460715289 <== mon.2 v2:172.21.15.167:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7fb7f00209d0 con 0x55b0ad8c22f0 2024-08-30T09:02:06.099 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.095+0000 7fb80f322740 1 -- 172.21.15.42:0/1460715289 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb7fc0ccd10 con 0x55b0ad8c22f0 2024-08-30T09:02:06.099 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.095+0000 7fb7e9ffb640 1 -- 172.21.15.42:0/1460715289 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb7f0037050 con 0x55b0ad8c22f0 2024-08-30T09:02:06.099 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.095+0000 7fb80f322740 10 client.25370 fetch_fsmap learned FSMap version 29 2024-08-30T09:02:06.099 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.095+0000 7fb80f322740 1 -- 172.21.15.42:0/1460715289 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b0add24850 con 0x55b0ad8c22f0 2024-08-30T09:02:06.099 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.096+0000 7fb7e9ffb640 1 -- 172.21.15.42:0/1460715289 <== mon.2 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb7f0020bd0 con 0x55b0ad8c22f0 2024-08-30T09:02:06.099 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.096+0000 7fb7e9ffb640 1 -- 172.21.15.42:0/1460715289 <== mon.2 v2:172.21.15.167:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fb7f0012070 con 0x55b0ad8c22f0 2024-08-30T09:02:06.100 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.096+0000 7fb80f322740 10 client.25370 fetch_fsmap finished waiting for FSMap version 29 2024-08-30T09:02:06.100 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.096+0000 7fb80f322740 10 client.25370 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-30T09:02:06.100 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.096+0000 7fb80f322740 20 client.25370 populate_metadata read hostname 'smithi042' 2024-08-30T09:02:06.100 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.096+0000 7fb80f322740 1 --2- 172.21.15.42:0/1460715289 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55b0ad87f1c0 0x55b0add272f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:06.100 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.096+0000 7fb80f322740 4 client.25370 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-30T09:02:06.100 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.096+0000 7fb80f322740 1 -- 172.21.15.42:0/1460715289 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7fb7fc0ccd10 con 0x55b0ad87f1c0 2024-08-30T09:02:06.100 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.096+0000 7fb805d13640 1 --2- 172.21.15.42:0/1460715289 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55b0ad87f1c0 0x55b0add272f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:02:06.101 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.097+0000 7fb805d13640 1 --2- 172.21.15.42:0/1460715289 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55b0ad87f1c0 0x55b0add272f0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74f14132c21bb1f1 server_cookie=14bcdf8924fc0f86 in_seq=0 out_seq=0 2024-08-30T09:02:06.101 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.097+0000 7fb7e9ffb640 10 client.25370 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:02:06.113 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.111+0000 7fb7e9ffb640 1 -- 172.21.15.42:0/1460715289 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7fb7fc0ccd10 con 0x55b0ad87f1c0 2024-08-30T09:02:06.113 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.111+0000 7fb7e9ffb640 10 client.25370 handle_command_reply: tid=0 2024-08-30T09:02:06.113 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.111+0000 7fb80f322740 1 client.25370 shutdown 2024-08-30T09:02:06.115 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.112+0000 7fb80f322740 1 -- 172.21.15.42:0/1460715289 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b0ad8c22f0 msgr2=0x55b0adc9eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:02:06.115 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.112+0000 7fb80f322740 1 --2- 172.21.15.42:0/1460715289 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b0ad8c22f0 0x55b0adc9eac0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb7f0006e60 tx=0x7fb7f0004770 comp rx=0 tx=0).stop 2024-08-30T09:02:06.115 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.112+0000 7fb80f322740 1 -- 172.21.15.42:0/1460715289 shutdown_connections 2024-08-30T09:02:06.115 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.112+0000 7fb80f322740 1 --2- 172.21.15.42:0/1460715289 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55b0ad87f1c0 0x55b0add272f0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:06.115 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.112+0000 7fb80f322740 1 --2- 172.21.15.42:0/1460715289 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b0ad8ca4d0 0x55b0adc9f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:06.115 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.112+0000 7fb80f322740 1 --2- 172.21.15.42:0/1460715289 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b0ad8eb1b0 0x55b0adc9fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:06.115 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.112+0000 7fb80f322740 1 --2- 172.21.15.42:0/1460715289 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b0ad8c22f0 0x55b0adc9eac0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:06.115 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.112+0000 7fb7e9ffb640 0 client.25370 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:02:06.115 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.112+0000 7fb80f322740 1 -- 172.21.15.42:0/1460715289 >> 172.21.15.42:0/1460715289 conn(0x55b0ad8df300 msgr2=0x55b0adc957d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:02:06.115 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.112+0000 7fb80f322740 1 -- 172.21.15.42:0/1460715289 shutdown_connections 2024-08-30T09:02:06.115 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.112+0000 7fb80f322740 1 -- 172.21.15.42:0/1460715289 wait complete. 2024-08-30T09:02:06.115 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.112+0000 7fb80f322740 20 client.25370 trim_cache size 0 max 16384 2024-08-30T09:02:06.116 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-08-30T09:02:06.116 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2024-08-30T09:02:06.116 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-08-30T09:02:06.116 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-08-30T09:02:06.117 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.113+0000 7fb808778640 1 -- 172.21.15.42:0/3141724961 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fb7ec0b4d40 msgr2=0x7fb7ec0b7250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:02:06.117 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.113+0000 7fb808778640 1 --2- 172.21.15.42:0/3141724961 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fb7ec0b4d40 0x7fb7ec0b7250 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fb7f0006fd0 tx=0x7fb7f0006c40 comp rx=0 tx=0).stop 2024-08-30T09:02:06.117 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.113+0000 7fb808778640 1 -- 172.21.15.42:0/3141724961 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb80010aae0 msgr2=0x7fb8000727e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:02:06.117 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.113+0000 7fb808778640 1 --2- 172.21.15.42:0/3141724961 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb80010aae0 0x7fb8000727e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fb7fc031440 tx=0x7fb7fc003dc0 comp rx=0 tx=0).stop 2024-08-30T09:02:06.117 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.113+0000 7fb808778640 1 -- 172.21.15.42:0/3141724961 shutdown_connections 2024-08-30T09:02:06.117 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.113+0000 7fb808778640 1 --2- 172.21.15.42:0/3141724961 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fb7ec0b4d40 0x7fb7ec0b7250 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:06.117 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.113+0000 7fb808778640 1 --2- 172.21.15.42:0/3141724961 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb80010aae0 0x7fb8000727e0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:06.117 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.113+0000 7fb808778640 1 --2- 172.21.15.42:0/3141724961 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb80010b480 0x7fb800073340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:06.117 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.113+0000 7fb808778640 1 --2- 172.21.15.42:0/3141724961 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb800109a60 0x7fb800072170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:06.117 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.113+0000 7fb808778640 1 -- 172.21.15.42:0/3141724961 >> 172.21.15.42:0/3141724961 conn(0x7fb800078db0 msgr2=0x7fb80010de00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:02:06.117 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.113+0000 7fb808778640 1 -- 172.21.15.42:0/3141724961 shutdown_connections 2024-08-30T09:02:06.117 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:06.113+0000 7fb808778640 1 -- 172.21.15.42:0/3141724961 wait complete. 2024-08-30T09:02:07.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:02:06 smithi167 ceph-mon[22063]: pgmap v413: 129 pgs: 129 active+clean; 1.0 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 7.2 MiB/s wr, 622 op/s 2024-08-30T09:02:07.099 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:02:06 smithi042 ceph-mon[18652]: pgmap v413: 129 pgs: 129 active+clean; 1.0 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 7.2 MiB/s wr, 622 op/s 2024-08-30T09:02:07.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:02:06 smithi190 ceph-mon[22001]: pgmap v413: 129 pgs: 129 active+clean; 1.0 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 7.2 MiB/s wr, 622 op/s 2024-08-30T09:02:08.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:02:07 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-30T09:02:08.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:02:07 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T09:02:08.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:02:07 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T09:02:08.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:02:07 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T09:02:08.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:02:07 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T09:02:08.097 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:02:07 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-30T09:02:08.097 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:02:07 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T09:02:08.097 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:02:07 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T09:02:08.097 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:02:07 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T09:02:08.097 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:02:07 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T09:02:08.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:02:07 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-30T09:02:08.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:02:07 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T09:02:08.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:02:07 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T09:02:08.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:02:07 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T09:02:08.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:02:07 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T09:02:08.458 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-08-30T09:02:08.459 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph fs dump --format=json 2024-08-30T09:02:08.906 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T09:02:09.038 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:02:08 smithi042 ceph-mon[18652]: pgmap v414: 129 pgs: 129 active+clean; 1.0 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 13 MiB/s wr, 847 op/s 2024-08-30T09:02:09.038 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:02:08 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T09:02:09.038 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:02:08 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-30T09:02:09.038 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:02:08 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T09:02:09.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:02:08 smithi167 ceph-mon[22063]: pgmap v414: 129 pgs: 129 active+clean; 1.0 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 13 MiB/s wr, 847 op/s 2024-08-30T09:02:09.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:02:08 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T09:02:09.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:02:08 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-30T09:02:09.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:02:08 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T09:02:09.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:02:08 smithi190 ceph-mon[22001]: pgmap v414: 129 pgs: 129 active+clean; 1.0 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 13 MiB/s wr, 847 op/s 2024-08-30T09:02:09.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:02:08 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T09:02:09.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:02:08 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-30T09:02:09.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:02:08 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T09:02:09.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:09.527+0000 7f1e36da5640 1 -- 172.21.15.42:0/824980878 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1e30108220 msgr2=0x7f1e3010ea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:02:09.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:09.527+0000 7f1e36da5640 1 --2- 172.21.15.42:0/824980878 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1e30108220 0x7f1e3010ea50 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f1e28009ca0 tx=0x7f1e2802a8c0 comp rx=0 tx=0).stop 2024-08-30T09:02:09.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:09.527+0000 7f1e36da5640 1 -- 172.21.15.42:0/824980878 shutdown_connections 2024-08-30T09:02:09.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:09.527+0000 7f1e36da5640 1 --2- 172.21.15.42:0/824980878 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1e30108220 0x7f1e3010ea50 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:09.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:09.527+0000 7f1e36da5640 1 --2- 172.21.15.42:0/824980878 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1e301078d0 0x7f1e30107ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:09.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:09.527+0000 7f1e36da5640 1 --2- 172.21.15.42:0/824980878 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1e30101f90 0x7f1e301023a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:09.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:09.527+0000 7f1e36da5640 1 -- 172.21.15.42:0/824980878 >> 172.21.15.42:0/824980878 conn(0x7f1e30077a00 msgr2=0x7f1e30077e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:02:09.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:09.527+0000 7f1e36da5640 1 -- 172.21.15.42:0/824980878 shutdown_connections 2024-08-30T09:02:09.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:09.528+0000 7f1e36da5640 1 -- 172.21.15.42:0/824980878 wait complete. 2024-08-30T09:02:09.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:09.528+0000 7f1e36da5640 1 Processor -- start 2024-08-30T09:02:09.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:09.529+0000 7f1e36da5640 1 -- start start 2024-08-30T09:02:09.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:09.530+0000 7f1e36da5640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1e30101f90 0x7f1e3019a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:09.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:09.531+0000 7f1e36da5640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1e301078d0 0x7f1e3019a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:09.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:09.531+0000 7f1e36da5640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1e30108220 0x7f1e3019fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:09.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:09.531+0000 7f1e36da5640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f1e301116b0 con 0x7f1e30101f90 2024-08-30T09:02:09.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:09.531+0000 7f1e36da5640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f1e30111530 con 0x7f1e301078d0 2024-08-30T09:02:09.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:09.531+0000 7f1e36da5640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f1e30111830 con 0x7f1e30108220 2024-08-30T09:02:09.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:09.531+0000 7f1e35342640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1e30108220 0x7f1e3019fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:02:09.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:09.531+0000 7f1e35342640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1e30108220 0x7f1e3019fa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:49482/0 (socket says 172.21.15.42:49482) 2024-08-30T09:02:09.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:09.531+0000 7f1e35342640 1 -- 172.21.15.42:0/2833290366 learned_addr learned my addr 172.21.15.42:0/2833290366 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:02:09.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:09.531+0000 7f1e2ffff640 1 --2- 172.21.15.42:0/2833290366 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1e301078d0 0x7f1e3019a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:02:09.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:09.531+0000 7f1e34b41640 1 --2- 172.21.15.42:0/2833290366 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1e30101f90 0x7f1e3019a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:02:09.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:09.531+0000 7f1e2ffff640 1 -- 172.21.15.42:0/2833290366 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1e30108220 msgr2=0x7f1e3019fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:02:09.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:09.531+0000 7f1e2ffff640 1 --2- 172.21.15.42:0/2833290366 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1e30108220 0x7f1e3019fa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:09.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:09.531+0000 7f1e2ffff640 1 -- 172.21.15.42:0/2833290366 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1e30101f90 msgr2=0x7f1e3019a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:02:09.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:09.531+0000 7f1e2ffff640 1 --2- 172.21.15.42:0/2833290366 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1e30101f90 0x7f1e3019a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:09.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:09.532+0000 7f1e2ffff640 1 -- 172.21.15.42:0/2833290366 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e301a01b0 con 0x7f1e301078d0 2024-08-30T09:02:09.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:09.532+0000 7f1e34b41640 1 --2- 172.21.15.42:0/2833290366 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1e30101f90 0x7f1e3019a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-30T09:02:09.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:09.532+0000 7f1e2ffff640 1 --2- 172.21.15.42:0/2833290366 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1e301078d0 0x7f1e3019a7e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f1e30075850 tx=0x7f1e1c00ee50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:02:09.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:09.532+0000 7f1e2dffb640 1 -- 172.21.15.42:0/2833290366 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e1c00cc40 con 0x7f1e301078d0 2024-08-30T09:02:09.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:09.532+0000 7f1e36da5640 1 -- 172.21.15.42:0/2833290366 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e301a04a0 con 0x7f1e301078d0 2024-08-30T09:02:09.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:09.532+0000 7f1e2dffb640 1 -- 172.21.15.42:0/2833290366 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1e1c00cde0 con 0x7f1e301078d0 2024-08-30T09:02:09.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:09.532+0000 7f1e2dffb640 1 -- 172.21.15.42:0/2833290366 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e1c018780 con 0x7f1e301078d0 2024-08-30T09:02:09.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:09.532+0000 7f1e36da5640 1 -- 172.21.15.42:0/2833290366 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e30108a20 con 0x7f1e301078d0 2024-08-30T09:02:09.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:09.534+0000 7f1e36da5640 1 -- 172.21.15.42:0/2833290366 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e301023a0 con 0x7f1e301078d0 2024-08-30T09:02:09.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:09.536+0000 7f1e2dffb640 1 -- 172.21.15.42:0/2833290366 <== mon.2 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7f1e1c010a90 con 0x7f1e301078d0 2024-08-30T09:02:09.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:09.537+0000 7f1e2dffb640 1 --2- 172.21.15.42:0/2833290366 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f1e200b4bb0 0x7f1e200b70c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:09.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:09.538+0000 7f1e2dffb640 1 -- 172.21.15.42:0/2833290366 <== mon.2 v2:172.21.15.167:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f1e1c0e5ca0 con 0x7f1e301078d0 2024-08-30T09:02:09.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:09.538+0000 7f1e34b41640 1 --2- 172.21.15.42:0/2833290366 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f1e200b4bb0 0x7f1e200b70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:02:09.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:09.539+0000 7f1e34b41640 1 --2- 172.21.15.42:0/2833290366 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f1e200b4bb0 0x7f1e200b70c0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f1e2400abc0 tx=0x7f1e24005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:02:09.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:09.540+0000 7f1e2dffb640 1 -- 172.21.15.42:0/2833290366 <== mon.2 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f1e1c0aca40 con 0x7f1e301078d0 2024-08-30T09:02:09.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:09.941+0000 7f1e36da5640 1 -- 172.21.15.42:0/2833290366 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1e30109d70 con 0x7f1e301078d0 2024-08-30T09:02:09.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:09.942+0000 7f1e2dffb640 1 -- 172.21.15.42:0/2833290366 <== mon.2 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11918 (secure 0 0 0) 0x7f1e1c0ac340 con 0x7f1e301078d0 2024-08-30T09:02:09.945 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-08-30T09:02:09.945 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":29,"btime":"2024-08-30T09:00:48:414225+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14703,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/3188400000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":3188400000},{"type":"v1","addr":"172.21.15.42:6835","nonce":3188400000}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14715,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/3356537674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":3356537674},{"type":"v1","addr":"172.21.15.42:6837","nonce":3356537674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14766,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6839/2107785513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6838","nonce":2107785513},{"type":"v1","addr":"172.21.15.42:6839","nonce":2107785513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14778,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6841/2298438674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6840","nonce":2298438674},{"type":"v1","addr":"172.21.15.42:6841","nonce":2298438674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24553,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/543555570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":543555570},{"type":"v1","addr":"172.21.15.190:6833","nonce":543555570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24554,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/3907818941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":3907818941},{"type":"v1","addr":"172.21.15.167:6833","nonce":3907818941}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-30T08:58:03.055084+0000","modified":"2024-08-30T09:00:47.637894+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24629,"mds_1":24610,"mds_2":24599},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24572":{"gid":24572,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6835/234596845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":234596845},{"type":"v1","addr":"172.21.15.167:6835","nonce":234596845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24592":{"gid":24592,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6835/247905709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":247905709},{"type":"v1","addr":"172.21.15.190:6835","nonce":247905709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24598":{"gid":24598,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6837/863256349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6836","nonce":863256349},{"type":"v1","addr":"172.21.15.190:6837","nonce":863256349}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24599":{"gid":24599,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.167:6837/3011837856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":3011837856},{"type":"v1","addr":"172.21.15.167:6837","nonce":3011837856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24610":{"gid":24610,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.190:6839/719981110","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6838","nonce":719981110},{"type":"v1","addr":"172.21.15.190:6839","nonce":719981110}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24629":{"gid":24629,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.167:6839/3604645207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":3604645207},{"type":"v1","addr":"172.21.15.167:6839","nonce":3604645207}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24629,"qdb_cluster":[24599,24610,24629]},"id":1}]} 2024-08-30T09:02:09.945 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 29 2024-08-30T09:02:09.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:09.946+0000 7f1e36da5640 1 -- 172.21.15.42:0/2833290366 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f1e200b4bb0 msgr2=0x7f1e200b70c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:02:09.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:09.946+0000 7f1e36da5640 1 --2- 172.21.15.42:0/2833290366 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f1e200b4bb0 0x7f1e200b70c0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f1e2400abc0 tx=0x7f1e24005f90 comp rx=0 tx=0).stop 2024-08-30T09:02:09.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:09.946+0000 7f1e36da5640 1 -- 172.21.15.42:0/2833290366 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1e301078d0 msgr2=0x7f1e3019a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:02:09.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:09.946+0000 7f1e36da5640 1 --2- 172.21.15.42:0/2833290366 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1e301078d0 0x7f1e3019a7e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f1e30075850 tx=0x7f1e1c00ee50 comp rx=0 tx=0).stop 2024-08-30T09:02:09.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:09.946+0000 7f1e36da5640 1 -- 172.21.15.42:0/2833290366 shutdown_connections 2024-08-30T09:02:09.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:09.946+0000 7f1e36da5640 1 --2- 172.21.15.42:0/2833290366 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f1e200b4bb0 0x7f1e200b70c0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:09.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:09.946+0000 7f1e36da5640 1 --2- 172.21.15.42:0/2833290366 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1e30108220 0x7f1e3019fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:09.949 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:09.946+0000 7f1e36da5640 1 --2- 172.21.15.42:0/2833290366 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1e301078d0 0x7f1e3019a7e0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:09.949 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:09.946+0000 7f1e36da5640 1 --2- 172.21.15.42:0/2833290366 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1e30101f90 0x7f1e3019a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:09.949 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:09.946+0000 7f1e36da5640 1 -- 172.21.15.42:0/2833290366 >> 172.21.15.42:0/2833290366 conn(0x7f1e30077a00 msgr2=0x7f1e30103720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:02:09.949 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:09.947+0000 7f1e36da5640 1 -- 172.21.15.42:0/2833290366 shutdown_connections 2024-08-30T09:02:09.949 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:09.947+0000 7f1e36da5640 1 -- 172.21.15.42:0/2833290366 wait complete. 2024-08-30T09:02:10.755 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-08-30T09:02:10.953 INFO:teuthology.orchestra.run.smithi042.stderr:progress: 85.0 s, 1873.8 tps, lat 53.660 ms stddev 97.125 2024-08-30T09:02:10.987 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:02:10 smithi190 ceph-mon[22001]: pgmap v415: 129 pgs: 129 active+clean; 1.0 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 19 MiB/s wr, 1.01k op/s 2024-08-30T09:02:10.988 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:02:10 smithi190 ceph-mon[22001]: from='client.? 172.21.15.42:0/2833290366' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-30T09:02:11.076 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:02:10 smithi167 ceph-mon[22063]: pgmap v415: 129 pgs: 129 active+clean; 1.0 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 19 MiB/s wr, 1.01k op/s 2024-08-30T09:02:11.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:02:10 smithi167 ceph-mon[22063]: from='client.? 172.21.15.42:0/2833290366' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-30T09:02:11.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:02:10 smithi042 ceph-mon[18652]: pgmap v415: 129 pgs: 129 active+clean; 1.0 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 19 MiB/s wr, 1.01k op/s 2024-08-30T09:02:11.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:02:10 smithi042 ceph-mon[18652]: from='client.? 172.21.15.42:0/2833290366' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-30T09:02:11.202 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T09:02:11.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.765+0000 7f8a2a126640 1 -- 172.21.15.42:0/1585388755 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8a2411c940 msgr2=0x7f8a2411ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:02:11.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.765+0000 7f8a2a126640 1 --2- 172.21.15.42:0/1585388755 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8a2411c940 0x7f8a2411ed60 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f8a1401a910 tx=0x7f8a1403b470 comp rx=0 tx=0).stop 2024-08-30T09:02:11.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.774+0000 7f8a2a126640 1 -- 172.21.15.42:0/1585388755 shutdown_connections 2024-08-30T09:02:11.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.774+0000 7f8a2a126640 1 --2- 172.21.15.42:0/1585388755 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8a2411c940 0x7f8a2411ed60 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:11.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.774+0000 7f8a2a126640 1 --2- 172.21.15.42:0/1585388755 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8a2410a920 0x7f8a2410ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:11.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.774+0000 7f8a2a126640 1 --2- 172.21.15.42:0/1585388755 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8a24111ae0 0x7f8a2410a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:11.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.774+0000 7f8a2a126640 1 -- 172.21.15.42:0/1585388755 >> 172.21.15.42:0/1585388755 conn(0x7f8a2406d6f0 msgr2=0x7f8a2406db30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:02:11.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.778+0000 7f8a2a126640 1 -- 172.21.15.42:0/1585388755 shutdown_connections 2024-08-30T09:02:11.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.779+0000 7f8a2a126640 1 -- 172.21.15.42:0/1585388755 wait complete. 2024-08-30T09:02:11.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.781+0000 7f8a2a126640 1 Processor -- start 2024-08-30T09:02:11.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.781+0000 7f8a2a126640 1 -- start start 2024-08-30T09:02:11.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.781+0000 7f8a2a126640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8a2410a920 0x7f8a2411c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:11.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.781+0000 7f8a2a126640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8a24111ae0 0x7f8a24118650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:11.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.781+0000 7f8a2a126640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8a2411c940 0x7f8a24118d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:11.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.781+0000 7f8a2a126640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f8a24114a60 con 0x7f8a24111ae0 2024-08-30T09:02:11.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.781+0000 7f8a2a126640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f8a241148e0 con 0x7f8a2411c940 2024-08-30T09:02:11.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.781+0000 7f8a2a126640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f8a24114be0 con 0x7f8a2410a920 2024-08-30T09:02:11.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.782+0000 7f8a237fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8a2410a920 0x7f8a2411c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:02:11.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.782+0000 7f8a237fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8a2410a920 0x7f8a2411c370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:49490/0 (socket says 172.21.15.42:49490) 2024-08-30T09:02:11.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.782+0000 7f8a237fe640 1 -- 172.21.15.42:0/2728490244 learned_addr learned my addr 172.21.15.42:0/2728490244 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:02:11.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.783+0000 7f8a22ffd640 1 --2- 172.21.15.42:0/2728490244 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8a24111ae0 0x7f8a24118650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:02:11.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.785+0000 7f8a22ffd640 1 -- 172.21.15.42:0/2728490244 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8a2410a920 msgr2=0x7f8a2411c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:02:11.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.785+0000 7f8a23fff640 1 --2- 172.21.15.42:0/2728490244 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8a2411c940 0x7f8a24118d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:02:11.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.785+0000 7f8a22ffd640 1 --2- 172.21.15.42:0/2728490244 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8a2410a920 0x7f8a2411c370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:11.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.785+0000 7f8a22ffd640 1 -- 172.21.15.42:0/2728490244 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8a2411c940 msgr2=0x7f8a24118d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:02:11.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.785+0000 7f8a22ffd640 1 --2- 172.21.15.42:0/2728490244 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8a2411c940 0x7f8a24118d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:11.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.785+0000 7f8a22ffd640 1 -- 172.21.15.42:0/2728490244 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a2407b3c0 con 0x7f8a24111ae0 2024-08-30T09:02:11.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.785+0000 7f8a22ffd640 1 --2- 172.21.15.42:0/2728490244 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8a24111ae0 0x7f8a24118650 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f8a18009fd0 tx=0x7f8a1800b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:02:11.792 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.789+0000 7f8a20ff9640 1 -- 172.21.15.42:0/2728490244 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a18015020 con 0x7f8a24111ae0 2024-08-30T09:02:11.792 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.789+0000 7f8a20ff9640 1 -- 172.21.15.42:0/2728490244 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a18011ce0 con 0x7f8a24111ae0 2024-08-30T09:02:11.792 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.789+0000 7f8a20ff9640 1 -- 172.21.15.42:0/2728490244 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a18010e00 con 0x7f8a24111ae0 2024-08-30T09:02:11.792 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.790+0000 7f8a2a126640 1 -- 172.21.15.42:0/2728490244 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a2407b6b0 con 0x7f8a24111ae0 2024-08-30T09:02:11.792 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.790+0000 7f8a2a126640 1 -- 172.21.15.42:0/2728490244 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a2407bba0 con 0x7f8a24111ae0 2024-08-30T09:02:11.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.796+0000 7f8a3069b740 1 Processor -- start 2024-08-30T09:02:11.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.796+0000 7f8a3069b740 1 -- start start 2024-08-30T09:02:11.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.796+0000 7f8a3069b740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5583f69db070 0x5583f6d804b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:11.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.796+0000 7f8a3069b740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5583f69b21b0 0x5583f6d809f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:11.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.796+0000 7f8a3069b740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5583f69ba390 0x5583f6d80f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:11.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.796+0000 7f8a3069b740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x5583f6cf7680 con 0x5583f69ba390 2024-08-30T09:02:11.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.796+0000 7f8a3069b740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5583f6ce6c10 con 0x5583f69db070 2024-08-30T09:02:11.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.796+0000 7f8a3069b740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5583f6ceabc0 con 0x5583f69b21b0 2024-08-30T09:02:11.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.797+0000 7f8a237fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5583f69b21b0 0x5583f6d809f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:02:11.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.797+0000 7f8a237fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5583f69b21b0 0x5583f6d809f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:49498/0 (socket says 172.21.15.42:49498) 2024-08-30T09:02:11.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.797+0000 7f8a237fe640 1 -- 172.21.15.42:0/2581138516 learned_addr learned my addr 172.21.15.42:0/2581138516 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:02:11.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.797+0000 7f8a23fff640 1 --2- 172.21.15.42:0/2581138516 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5583f69db070 0x5583f6d804b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:02:11.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.798+0000 7f8a237fe640 1 -- 172.21.15.42:0/2581138516 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5583f69db070 msgr2=0x5583f6d804b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:02:11.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.798+0000 7f8a237fe640 1 --2- 172.21.15.42:0/2581138516 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5583f69db070 0x5583f6d804b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:11.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.798+0000 7f8a237fe640 1 -- 172.21.15.42:0/2581138516 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5583f69ba390 msgr2=0x5583f6d80f30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-30T09:02:11.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.798+0000 7f8a237fe640 1 --2- 172.21.15.42:0/2581138516 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5583f69ba390 0x5583f6d80f30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:11.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.798+0000 7f8a20ff9640 1 -- 172.21.15.42:0/2728490244 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7f8a18018020 con 0x7f8a24111ae0 2024-08-30T09:02:11.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.798+0000 7f8a237fe640 1 -- 172.21.15.42:0/2581138516 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5583f6a46650 con 0x5583f69b21b0 2024-08-30T09:02:11.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.799+0000 7f8a20ff9640 1 --2- 172.21.15.42:0/2728490244 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f8a0c0b4cc0 0x7f8a0c0b71d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:11.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.799+0000 7f8a237fe640 1 --2- 172.21.15.42:0/2581138516 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5583f69b21b0 0x5583f6d809f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f8a1c007370 tx=0x7f8a1c007830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6391cfa2e567dad9 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:02:11.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.800+0000 7f8a20ff9640 1 -- 172.21.15.42:0/2728490244 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f8a180a78e0 con 0x7f8a24111ae0 2024-08-30T09:02:11.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.800+0000 7f8a23fff640 1 --2- 172.21.15.42:0/2728490244 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f8a0c0b4cc0 0x7f8a0c0b71d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:02:11.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.801+0000 7f8a09ffb640 1 -- 172.21.15.42:0/2581138516 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a1c019070 con 0x5583f69b21b0 2024-08-30T09:02:11.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.801+0000 7f8a09ffb640 1 -- 172.21.15.42:0/2581138516 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a1c012bc0 con 0x5583f69b21b0 2024-08-30T09:02:11.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.802+0000 7f8a3069b740 1 -- 172.21.15.42:0/2581138516 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5583f69b21b0 msgr2=0x5583f6d809f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:02:11.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.802+0000 7f8a3069b740 1 --2- 172.21.15.42:0/2581138516 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5583f69b21b0 0x5583f6d809f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f8a1c007370 tx=0x7f8a1c007830 comp rx=0 tx=0).stop 2024-08-30T09:02:11.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.807+0000 7f8a3069b740 1 -- 172.21.15.42:0/2581138516 shutdown_connections 2024-08-30T09:02:11.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.807+0000 7f8a3069b740 1 --2- 172.21.15.42:0/2581138516 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5583f69b21b0 0x5583f6d809f0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:11.811 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.807+0000 7f8a3069b740 1 --2- 172.21.15.42:0/2581138516 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5583f69ba390 0x5583f6d80f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:11.811 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.807+0000 7f8a3069b740 1 --2- 172.21.15.42:0/2581138516 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5583f69db070 0x5583f6d804b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:11.811 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.807+0000 7f8a3069b740 1 -- 172.21.15.42:0/2581138516 >> 172.21.15.42:0/2581138516 conn(0x5583f69cf460 msgr2=0x5583f6d7e3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:02:11.811 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.809+0000 7f8a3069b740 1 -- 172.21.15.42:0/2581138516 shutdown_connections 2024-08-30T09:02:11.811 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.810+0000 7f8a23fff640 1 --2- 172.21.15.42:0/2728490244 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f8a0c0b4cc0 0x7f8a0c0b71d0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f8a14006010 tx=0x7f8a1401a620 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:02:11.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.810+0000 7f8a3069b740 1 -- 172.21.15.42:0/2581138516 wait complete. 2024-08-30T09:02:11.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.810+0000 7f8a3069b740 1 -- start start 2024-08-30T09:02:11.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.811+0000 7f8a3069b740 1 Processor -- start 2024-08-30T09:02:11.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.811+0000 7f8a3069b740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5583f69b21b0 0x5583f6d8bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:11.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.811+0000 7f8a3069b740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5583f69ba390 0x5583f6d8c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:11.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.811+0000 7f8a3069b740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5583f69db070 0x5583f6d98020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:11.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.811+0000 7f8a3069b740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x5583f6b49700 con 0x5583f69db070 2024-08-30T09:02:11.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.811+0000 7f8a3069b740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5583f6c6d750 con 0x5583f69ba390 2024-08-30T09:02:11.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.811+0000 7f8a3069b740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5583f6d8ece0 con 0x5583f69b21b0 2024-08-30T09:02:11.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.811+0000 7f8a3069b740 10 client.0 _ll_register_callbacks cb 0x5583f6c9ce10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:02:11.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.812+0000 7f8a237fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5583f69b21b0 0x5583f6d8bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:02:11.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.812+0000 7f8a237fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5583f69b21b0 0x5583f6d8bbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:49500/0 (socket says 172.21.15.42:49500) 2024-08-30T09:02:11.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.812+0000 7f8a237fe640 1 -- 172.21.15.42:0/1552529046 learned_addr learned my addr 172.21.15.42:0/1552529046 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:02:11.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.813+0000 7f8a22ffd640 1 --2- 172.21.15.42:0/1552529046 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5583f69ba390 0x5583f6d8c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:02:11.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.813+0000 7f8a23fff640 1 --2- 172.21.15.42:0/1552529046 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5583f69db070 0x5583f6d98020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:02:11.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.813+0000 7f8a22ffd640 1 -- 172.21.15.42:0/1552529046 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5583f69b21b0 msgr2=0x5583f6d8bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:02:11.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.813+0000 7f8a22ffd640 1 --2- 172.21.15.42:0/1552529046 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5583f69b21b0 0x5583f6d8bbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:11.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.813+0000 7f8a22ffd640 1 -- 172.21.15.42:0/1552529046 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5583f69db070 msgr2=0x5583f6d98020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:02:11.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.813+0000 7f8a22ffd640 1 --2- 172.21.15.42:0/1552529046 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5583f69db070 0x5583f6d98020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:11.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.813+0000 7f8a22ffd640 1 -- 172.21.15.42:0/1552529046 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5583f6ca6900 con 0x5583f69ba390 2024-08-30T09:02:11.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.814+0000 7f8a22ffd640 1 --2- 172.21.15.42:0/1552529046 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5583f69ba390 0x5583f6d8c0f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f8a180ab280 tx=0x7f8a180092b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5b55a1d1b890a0fd server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:02:11.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.814+0000 7f8a02ffd640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-30T09:02:11.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.814+0000 7f8a02ffd640 1 -- 172.21.15.42:0/1552529046 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a180ab2f0 con 0x5583f69ba390 2024-08-30T09:02:11.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.814+0000 7f8a3069b740 1 -- 172.21.15.42:0/1552529046 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a1c012bc0 con 0x5583f69ba390 2024-08-30T09:02:11.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.814+0000 7f8a02ffd640 1 -- 172.21.15.42:0/1552529046 <== mon.2 v2:172.21.15.167:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f8a180ce0b0 con 0x5583f69ba390 2024-08-30T09:02:11.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.814+0000 7f8a02ffd640 1 -- 172.21.15.42:0/1552529046 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8a180afb60 con 0x5583f69ba390 2024-08-30T09:02:11.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.814+0000 7f8a3069b740 10 client.25382 fetch_fsmap learned FSMap version 29 2024-08-30T09:02:11.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.814+0000 7f8a3069b740 1 -- 172.21.15.42:0/1552529046 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5583f6d9aaf0 con 0x5583f69ba390 2024-08-30T09:02:11.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.815+0000 7f8a02ffd640 1 -- 172.21.15.42:0/1552529046 <== mon.2 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a1c012bc0 con 0x5583f69ba390 2024-08-30T09:02:11.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.815+0000 7f8a02ffd640 1 -- 172.21.15.42:0/1552529046 <== mon.2 v2:172.21.15.167:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f8a180a6c60 con 0x5583f69ba390 2024-08-30T09:02:11.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.816+0000 7f8a3069b740 10 client.25382 fetch_fsmap finished waiting for FSMap version 29 2024-08-30T09:02:11.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.816+0000 7f8a3069b740 10 client.25382 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-08-30T09:02:11.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.816+0000 7f8a3069b740 20 client.25382 populate_metadata read hostname 'smithi042' 2024-08-30T09:02:11.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.816+0000 7f8a3069b740 1 --2- 172.21.15.42:0/1552529046 >> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] conn(0x5583f69999f0 0x5583f6d9e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:11.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.816+0000 7f8a3069b740 4 client.25382 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-30T09:02:11.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.816+0000 7f8a3069b740 1 -- 172.21.15.42:0/1552529046 --> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5583f6cf95d0 con 0x5583f69999f0 2024-08-30T09:02:11.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.816+0000 7f8a237fe640 1 --2- 172.21.15.42:0/1552529046 >> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] conn(0x5583f69999f0 0x5583f6d9e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:02:11.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.817+0000 7f8a237fe640 1 --2- 172.21.15.42:0/1552529046 >> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] conn(0x5583f69999f0 0x5583f6d9e3b0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5b29f4d9d26db920 server_cookie=9fb7c14a33096e69 in_seq=0 out_seq=0 2024-08-30T09:02:11.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.817+0000 7f8a02ffd640 10 client.25382 ms_handle_connect on v2:172.21.15.167:6836/3011837856 2024-08-30T09:02:11.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.818+0000 7f8a02ffd640 1 -- 172.21.15.42:0/1552529046 <== mds.2 v2:172.21.15.167:6836/3011837856 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5583f6cf95d0 con 0x5583f69999f0 2024-08-30T09:02:11.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.818+0000 7f8a02ffd640 10 client.25382 handle_command_reply: tid=0 2024-08-30T09:02:11.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.818+0000 7f8a3069b740 1 client.25382 shutdown 2024-08-30T09:02:11.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.821+0000 7f8a3069b740 1 -- 172.21.15.42:0/1552529046 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5583f69ba390 msgr2=0x5583f6d8c0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:02:11.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.821+0000 7f8a3069b740 1 --2- 172.21.15.42:0/1552529046 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5583f69ba390 0x5583f6d8c0f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f8a180ab280 tx=0x7f8a180092b0 comp rx=0 tx=0).stop 2024-08-30T09:02:11.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.821+0000 7f8a3069b740 1 -- 172.21.15.42:0/1552529046 shutdown_connections 2024-08-30T09:02:11.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.821+0000 7f8a3069b740 1 --2- 172.21.15.42:0/1552529046 >> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] conn(0x5583f69999f0 0x5583f6d9e3b0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:11.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.821+0000 7f8a3069b740 1 --2- 172.21.15.42:0/1552529046 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5583f69db070 0x5583f6d98020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:11.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.821+0000 7f8a3069b740 1 --2- 172.21.15.42:0/1552529046 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5583f69ba390 0x5583f6d8c0f0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:11.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.821+0000 7f8a3069b740 1 --2- 172.21.15.42:0/1552529046 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5583f69b21b0 0x5583f6d8bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:11.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.821+0000 7f8a02ffd640 0 client.25382 ms_handle_reset on v2:172.21.15.167:6836/3011837856 2024-08-30T09:02:11.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.821+0000 7f8a3069b740 1 -- 172.21.15.42:0/1552529046 >> 172.21.15.42:0/1552529046 conn(0x5583f69cf460 msgr2=0x5583f6d83690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:02:11.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.821+0000 7f8a3069b740 1 -- 172.21.15.42:0/1552529046 shutdown_connections 2024-08-30T09:02:11.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.821+0000 7f8a3069b740 1 -- 172.21.15.42:0/1552529046 wait complete. 2024-08-30T09:02:11.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.822+0000 7f8a3069b740 20 client.25382 trim_cache size 0 max 16384 2024-08-30T09:02:11.847 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.845+0000 7f8a3069b740 1 Processor -- start 2024-08-30T09:02:11.847 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.846+0000 7f8a3069b740 1 -- start start 2024-08-30T09:02:11.847 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.846+0000 7f8a3069b740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5583f69b21b0 0x5583f6d81ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:11.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.846+0000 7f8a3069b740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5583f69ba390 0x5583f6d7e110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:11.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.846+0000 7f8a237fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5583f69b21b0 0x5583f6d81ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:02:11.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.846+0000 7f8a237fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5583f69b21b0 0x5583f6d81ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:59930/0 (socket says 172.21.15.42:59930) 2024-08-30T09:02:11.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.846+0000 7f8a237fe640 1 -- 172.21.15.42:0/3716736510 learned_addr learned my addr 172.21.15.42:0/3716736510 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:02:11.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.846+0000 7f8a22ffd640 1 --2- 172.21.15.42:0/3716736510 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5583f69ba390 0x5583f6d7e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:02:11.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.846+0000 7f8a3069b740 1 --2- 172.21.15.42:0/3716736510 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5583f69db070 0x5583f6d97f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:11.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.847+0000 7f8a3069b740 1 -- 172.21.15.42:0/3716736510 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f8a241ca5c0 con 0x5583f69db070 2024-08-30T09:02:11.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.847+0000 7f8a3069b740 1 -- 172.21.15.42:0/3716736510 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f8a24085930 con 0x5583f69b21b0 2024-08-30T09:02:11.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.847+0000 7f8a3069b740 1 -- 172.21.15.42:0/3716736510 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f8a2410adb0 con 0x5583f69ba390 2024-08-30T09:02:11.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.847+0000 7f8a23fff640 1 --2- 172.21.15.42:0/3716736510 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5583f69db070 0x5583f6d97f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:02:11.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.847+0000 7f8a237fe640 1 -- 172.21.15.42:0/3716736510 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5583f69ba390 msgr2=0x5583f6d7e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:02:11.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.847+0000 7f8a237fe640 1 --2- 172.21.15.42:0/3716736510 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5583f69ba390 0x5583f6d7e110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:11.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.847+0000 7f8a237fe640 1 -- 172.21.15.42:0/3716736510 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5583f69db070 msgr2=0x5583f6d97f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:02:11.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.847+0000 7f8a237fe640 1 --2- 172.21.15.42:0/3716736510 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5583f69db070 0x5583f6d97f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:11.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.847+0000 7f8a237fe640 1 -- 172.21.15.42:0/3716736510 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5583f6d867e0 con 0x5583f69b21b0 2024-08-30T09:02:11.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.847+0000 7f8a23fff640 1 --2- 172.21.15.42:0/3716736510 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5583f69db070 0x5583f6d97f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-30T09:02:11.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.848+0000 7f8a237fe640 1 --2- 172.21.15.42:0/3716736510 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5583f69b21b0 0x5583f6d81ea0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f8a1c007370 tx=0x7f8a1c0096e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1468b338f9fa5010 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:02:11.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.848+0000 7f8a097fa640 1 -- 172.21.15.42:0/3716736510 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a1c019040 con 0x5583f69b21b0 2024-08-30T09:02:11.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.848+0000 7f8a097fa640 1 -- 172.21.15.42:0/3716736510 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a1c02d070 con 0x5583f69b21b0 2024-08-30T09:02:11.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.848+0000 7f8a097fa640 1 -- 172.21.15.42:0/3716736510 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a1c0128b0 con 0x5583f69b21b0 2024-08-30T09:02:11.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.850+0000 7f8a3069b740 1 -- 172.21.15.42:0/3716736510 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5583f69b21b0 msgr2=0x5583f6d81ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:02:11.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.850+0000 7f8a3069b740 1 --2- 172.21.15.42:0/3716736510 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5583f69b21b0 0x5583f6d81ea0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f8a1c007370 tx=0x7f8a1c0096e0 comp rx=0 tx=0).stop 2024-08-30T09:02:11.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.850+0000 7f8a3069b740 1 -- 172.21.15.42:0/3716736510 shutdown_connections 2024-08-30T09:02:11.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.850+0000 7f8a3069b740 1 --2- 172.21.15.42:0/3716736510 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5583f69ba390 0x5583f6d7e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:11.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.850+0000 7f8a3069b740 1 --2- 172.21.15.42:0/3716736510 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5583f69db070 0x5583f6d97f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:11.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.850+0000 7f8a3069b740 1 --2- 172.21.15.42:0/3716736510 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5583f69b21b0 0x5583f6d81ea0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:11.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.850+0000 7f8a3069b740 1 -- 172.21.15.42:0/3716736510 >> 172.21.15.42:0/3716736510 conn(0x5583f69cf460 msgr2=0x5583f6d7da00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:02:11.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.850+0000 7f8a3069b740 1 -- 172.21.15.42:0/3716736510 shutdown_connections 2024-08-30T09:02:11.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.850+0000 7f8a3069b740 1 -- 172.21.15.42:0/3716736510 wait complete. 2024-08-30T09:02:11.854 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.850+0000 7f8a3069b740 1 -- start start 2024-08-30T09:02:11.854 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.851+0000 7f8a3069b740 1 Processor -- start 2024-08-30T09:02:11.854 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.851+0000 7f8a3069b740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5583f69b21b0 0x5583f6d8f470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:11.854 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.851+0000 7f8a3069b740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5583f69ba390 0x5583f6d8f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:11.854 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.851+0000 7f8a3069b740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5583f69db070 0x5583f6d8fef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:11.854 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.851+0000 7f8a3069b740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f8a24083df0 con 0x5583f69db070 2024-08-30T09:02:11.854 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.851+0000 7f8a3069b740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f8a24085260 con 0x5583f69b21b0 2024-08-30T09:02:11.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.851+0000 7f8a3069b740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f8a240844c0 con 0x5583f69ba390 2024-08-30T09:02:11.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.851+0000 7f8a3069b740 10 client.0 _ll_register_callbacks cb 0x7f8a18010720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:02:11.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.851+0000 7f8a22ffd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5583f69ba390 0x5583f6d8f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:02:11.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.851+0000 7f8a23fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5583f69db070 0x5583f6d8fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:02:11.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.851+0000 7f8a22ffd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5583f69ba390 0x5583f6d8f9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:49520/0 (socket says 172.21.15.42:49520) 2024-08-30T09:02:11.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.851+0000 7f8a22ffd640 1 -- 172.21.15.42:0/279542357 learned_addr learned my addr 172.21.15.42:0/279542357 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:02:11.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.852+0000 7f8a237fe640 1 --2- 172.21.15.42:0/279542357 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5583f69b21b0 0x5583f6d8f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:02:11.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.852+0000 7f8a237fe640 1 -- 172.21.15.42:0/279542357 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5583f69ba390 msgr2=0x5583f6d8f9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:02:11.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.852+0000 7f8a237fe640 1 --2- 172.21.15.42:0/279542357 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5583f69ba390 0x5583f6d8f9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:11.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.852+0000 7f8a237fe640 1 -- 172.21.15.42:0/279542357 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5583f69db070 msgr2=0x5583f6d8fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:02:11.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.852+0000 7f8a237fe640 1 --2- 172.21.15.42:0/279542357 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5583f69db070 0x5583f6d8fef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:11.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.852+0000 7f8a237fe640 1 -- 172.21.15.42:0/279542357 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5583f6d93670 con 0x5583f69b21b0 2024-08-30T09:02:11.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.852+0000 7f8a237fe640 1 --2- 172.21.15.42:0/279542357 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5583f69b21b0 0x5583f6d8f470 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f8a1c007370 tx=0x7f8a1c007ed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5c3bfae85abce5ea server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:02:11.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.853+0000 7f8a02ffd640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-30T09:02:11.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.853+0000 7f8a02ffd640 1 -- 172.21.15.42:0/279542357 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a1c035020 con 0x5583f69b21b0 2024-08-30T09:02:11.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.853+0000 7f8a3069b740 1 -- 172.21.15.42:0/279542357 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a1c02d070 con 0x5583f69b21b0 2024-08-30T09:02:11.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.854+0000 7f8a02ffd640 1 -- 172.21.15.42:0/279542357 <== mon.2 v2:172.21.15.167:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f8a1c003c30 con 0x5583f69b21b0 2024-08-30T09:02:11.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.854+0000 7f8a02ffd640 1 -- 172.21.15.42:0/279542357 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8a1c040050 con 0x5583f69b21b0 2024-08-30T09:02:11.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.854+0000 7f8a3069b740 10 client.25388 fetch_fsmap learned FSMap version 29 2024-08-30T09:02:11.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.854+0000 7f8a3069b740 1 -- 172.21.15.42:0/279542357 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5583f6e08fc0 con 0x5583f69b21b0 2024-08-30T09:02:11.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.854+0000 7f8a02ffd640 1 -- 172.21.15.42:0/279542357 <== mon.2 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a1c003e50 con 0x5583f69b21b0 2024-08-30T09:02:11.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.854+0000 7f8a02ffd640 1 -- 172.21.15.42:0/279542357 <== mon.2 v2:172.21.15.167:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f8a1c0043e0 con 0x5583f69b21b0 2024-08-30T09:02:11.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.854+0000 7f8a3069b740 10 client.25388 fetch_fsmap finished waiting for FSMap version 29 2024-08-30T09:02:11.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.854+0000 7f8a3069b740 10 client.25388 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-08-30T09:02:11.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.854+0000 7f8a3069b740 20 client.25388 populate_metadata read hostname 'smithi042' 2024-08-30T09:02:11.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.855+0000 7f8a3069b740 1 --2- 172.21.15.42:0/279542357 >> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] conn(0x5583f696eda0 0x5583f6e0b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:11.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.855+0000 7f8a3069b740 4 client.25388 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-30T09:02:11.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.855+0000 7f8a3069b740 1 -- 172.21.15.42:0/279542357 --> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f8a1c02d070 con 0x5583f696eda0 2024-08-30T09:02:11.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.855+0000 7f8a22ffd640 1 --2- 172.21.15.42:0/279542357 >> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] conn(0x5583f696eda0 0x5583f6e0b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:02:11.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.855+0000 7f8a02ffd640 10 client.25388 ms_handle_connect on v2:172.21.15.167:6836/3011837856 2024-08-30T09:02:11.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.855+0000 7f8a22ffd640 1 --2- 172.21.15.42:0/279542357 >> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] conn(0x5583f696eda0 0x5583f6e0b4e0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=961ebb631ec83925 server_cookie=14447d64e804dd in_seq=0 out_seq=0 2024-08-30T09:02:11.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.856+0000 7f8a02ffd640 1 -- 172.21.15.42:0/279542357 <== mds.2 v2:172.21.15.167:6836/3011837856 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f8a1c02d070 con 0x5583f696eda0 2024-08-30T09:02:11.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.856+0000 7f8a02ffd640 10 client.25388 handle_command_reply: tid=0 2024-08-30T09:02:11.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.856+0000 7f8a3069b740 1 client.25388 shutdown 2024-08-30T09:02:11.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.856+0000 7f8a3069b740 1 -- 172.21.15.42:0/279542357 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5583f69b21b0 msgr2=0x5583f6d8f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:02:11.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.856+0000 7f8a3069b740 1 --2- 172.21.15.42:0/279542357 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5583f69b21b0 0x5583f6d8f470 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f8a1c007370 tx=0x7f8a1c007ed0 comp rx=0 tx=0).stop 2024-08-30T09:02:11.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.856+0000 7f8a3069b740 1 -- 172.21.15.42:0/279542357 shutdown_connections 2024-08-30T09:02:11.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.856+0000 7f8a3069b740 1 --2- 172.21.15.42:0/279542357 >> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] conn(0x5583f696eda0 0x5583f6e0b4e0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:11.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.856+0000 7f8a02ffd640 0 client.25388 ms_handle_reset on v2:172.21.15.167:6836/3011837856 2024-08-30T09:02:11.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.856+0000 7f8a3069b740 1 --2- 172.21.15.42:0/279542357 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5583f69ba390 0x5583f6d8f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:11.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.856+0000 7f8a3069b740 1 --2- 172.21.15.42:0/279542357 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5583f69db070 0x5583f6d8fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:11.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.856+0000 7f8a3069b740 1 --2- 172.21.15.42:0/279542357 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5583f69b21b0 0x5583f6d8f470 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:11.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.856+0000 7f8a3069b740 1 -- 172.21.15.42:0/279542357 >> 172.21.15.42:0/279542357 conn(0x5583f69cf460 msgr2=0x5583f6d7e360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:02:11.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.856+0000 7f8a3069b740 1 -- 172.21.15.42:0/279542357 shutdown_connections 2024-08-30T09:02:11.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.857+0000 7f8a3069b740 1 -- 172.21.15.42:0/279542357 wait complete. 2024-08-30T09:02:11.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.858+0000 7f8a3069b740 20 client.25388 trim_cache size 0 max 16384 2024-08-30T09:02:11.861 INFO:teuthology.orchestra.run.smithi042.stdout:[] 2024-08-30T09:02:11.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.859+0000 7f8a2a126640 1 -- 172.21.15.42:0/2728490244 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f8a0c0b4cc0 msgr2=0x7f8a0c0b71d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:02:11.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.859+0000 7f8a2a126640 1 --2- 172.21.15.42:0/2728490244 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f8a0c0b4cc0 0x7f8a0c0b71d0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f8a14006010 tx=0x7f8a1401a620 comp rx=0 tx=0).stop 2024-08-30T09:02:11.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.859+0000 7f8a2a126640 1 -- 172.21.15.42:0/2728490244 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8a24111ae0 msgr2=0x7f8a24118650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:02:11.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.859+0000 7f8a2a126640 1 --2- 172.21.15.42:0/2728490244 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8a24111ae0 0x7f8a24118650 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f8a18009fd0 tx=0x7f8a1800b870 comp rx=0 tx=0).stop 2024-08-30T09:02:11.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.859+0000 7f8a2a126640 1 -- 172.21.15.42:0/2728490244 shutdown_connections 2024-08-30T09:02:11.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.859+0000 7f8a2a126640 1 --2- 172.21.15.42:0/2728490244 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f8a0c0b4cc0 0x7f8a0c0b71d0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:11.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.859+0000 7f8a2a126640 1 --2- 172.21.15.42:0/2728490244 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8a2411c940 0x7f8a24118d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:11.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.859+0000 7f8a2a126640 1 --2- 172.21.15.42:0/2728490244 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8a24111ae0 0x7f8a24118650 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:11.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.859+0000 7f8a2a126640 1 --2- 172.21.15.42:0/2728490244 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8a2410a920 0x7f8a2411c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:11.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.859+0000 7f8a2a126640 1 -- 172.21.15.42:0/2728490244 >> 172.21.15.42:0/2728490244 conn(0x7f8a2406d6f0 msgr2=0x7f8a24110770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:02:11.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.859+0000 7f8a2a126640 1 -- 172.21.15.42:0/2728490244 shutdown_connections 2024-08-30T09:02:11.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:11.859+0000 7f8a2a126640 1 -- 172.21.15.42:0/2728490244 wait complete. 2024-08-30T09:02:12.803 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-08-30T09:02:13.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:02:12 smithi167 ceph-mon[22063]: pgmap v416: 129 pgs: 129 active+clean; 1.0 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 19 MiB/s wr, 969 op/s 2024-08-30T09:02:13.132 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:02:12 smithi042 ceph-mon[18652]: pgmap v416: 129 pgs: 129 active+clean; 1.0 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 19 MiB/s wr, 969 op/s 2024-08-30T09:02:13.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:02:12 smithi190 ceph-mon[22001]: pgmap v416: 129 pgs: 129 active+clean; 1.0 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 19 MiB/s wr, 969 op/s 2024-08-30T09:02:13.225 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T09:02:15.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:02:14 smithi167 ceph-mon[22063]: pgmap v417: 129 pgs: 129 active+clean; 1.0 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 MiB/s wr, 1.18k op/s 2024-08-30T09:02:15.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:02:14 smithi042 ceph-mon[18652]: pgmap v417: 129 pgs: 129 active+clean; 1.0 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 MiB/s wr, 1.18k op/s 2024-08-30T09:02:15.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:02:14 smithi190 ceph-mon[22001]: pgmap v417: 129 pgs: 129 active+clean; 1.0 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 MiB/s wr, 1.18k op/s 2024-08-30T09:02:15.953 INFO:teuthology.orchestra.run.smithi042.stderr:progress: 90.0 s, 2654.0 tps, lat 37.684 ms stddev 46.532 2024-08-30T09:02:16.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:15.986+0000 7f96c355c640 1 -- 172.21.15.42:0/1291047529 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f96bc109bb0 msgr2=0x7f96bc1103e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:02:16.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:15.986+0000 7f96c355c640 1 --2- 172.21.15.42:0/1291047529 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f96bc109bb0 0x7f96bc1103e0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f96b4009ca0 tx=0x7f96b402a8c0 comp rx=0 tx=0).stop 2024-08-30T09:02:16.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:15.986+0000 7f96c355c640 1 -- 172.21.15.42:0/1291047529 shutdown_connections 2024-08-30T09:02:16.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:15.986+0000 7f96c355c640 1 --2- 172.21.15.42:0/1291047529 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f96bc109bb0 0x7f96bc1103e0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:16.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:15.986+0000 7f96c355c640 1 --2- 172.21.15.42:0/1291047529 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f96bc109260 0x7f96bc109670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:16.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:15.986+0000 7f96c355c640 1 --2- 172.21.15.42:0/1291047529 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f96bc103920 0x7f96bc103d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:16.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:15.986+0000 7f96c355c640 1 -- 172.21.15.42:0/1291047529 >> 172.21.15.42:0/1291047529 conn(0x7f96bc0ff370 msgr2=0x7f96bc1017c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:02:16.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:15.986+0000 7f96c355c640 1 -- 172.21.15.42:0/1291047529 shutdown_connections 2024-08-30T09:02:16.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:15.986+0000 7f96c355c640 1 -- 172.21.15.42:0/1291047529 wait complete. 2024-08-30T09:02:16.226 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:15.987+0000 7f96c355c640 1 Processor -- start 2024-08-30T09:02:16.226 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:15.988+0000 7f96c355c640 1 -- start start 2024-08-30T09:02:16.226 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:15.988+0000 7f96c355c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f96bc103920 0x7f96bc19a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:16.226 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:15.988+0000 7f96c355c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f96bc109260 0x7f96bc19a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:16.226 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:15.988+0000 7f96c355c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f96bc109bb0 0x7f96bc19fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:16.226 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:15.988+0000 7f96c355c640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f96bc113870 con 0x7f96bc109bb0 2024-08-30T09:02:16.226 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:15.988+0000 7f96c355c640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f96bc1136f0 con 0x7f96bc103920 2024-08-30T09:02:16.226 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:15.988+0000 7f96c355c640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f96bc1139f0 con 0x7f96bc109260 2024-08-30T09:02:16.226 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:15.988+0000 7f96c12f8640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f96bc103920 0x7f96bc19a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:02:16.226 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:15.988+0000 7f96c12f8640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f96bc103920 0x7f96bc19a1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:59942/0 (socket says 172.21.15.42:59942) 2024-08-30T09:02:16.226 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:15.988+0000 7f96c12f8640 1 -- 172.21.15.42:0/3308567914 learned_addr learned my addr 172.21.15.42:0/3308567914 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:02:16.226 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:15.988+0000 7f96c0af7640 1 --2- 172.21.15.42:0/3308567914 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f96bc109260 0x7f96bc19a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:02:16.226 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:15.988+0000 7f96c1af9640 1 --2- 172.21.15.42:0/3308567914 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f96bc109bb0 0x7f96bc19fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:02:16.226 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:15.989+0000 7f96c0af7640 1 -- 172.21.15.42:0/3308567914 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f96bc103920 msgr2=0x7f96bc19a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:02:16.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:15.989+0000 7f96c0af7640 1 --2- 172.21.15.42:0/3308567914 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f96bc103920 0x7f96bc19a1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:16.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:15.989+0000 7f96c0af7640 1 -- 172.21.15.42:0/3308567914 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f96bc109bb0 msgr2=0x7f96bc19fa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:02:16.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:15.989+0000 7f96c0af7640 1 --2- 172.21.15.42:0/3308567914 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f96bc109bb0 0x7f96bc19fa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:16.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:15.989+0000 7f96c0af7640 1 -- 172.21.15.42:0/3308567914 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96bc1a0130 con 0x7f96bc109260 2024-08-30T09:02:16.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:15.993+0000 7f96c0af7640 1 --2- 172.21.15.42:0/3308567914 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f96bc109260 0x7f96bc19a760 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f96bc06c7b0 tx=0x7f96b000ee50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:02:16.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:15.995+0000 7f96ae7fc640 1 -- 172.21.15.42:0/3308567914 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96b000cc40 con 0x7f96bc109260 2024-08-30T09:02:16.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:15.995+0000 7f96ae7fc640 1 -- 172.21.15.42:0/3308567914 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f96b000cde0 con 0x7f96bc109260 2024-08-30T09:02:16.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:15.996+0000 7f96ae7fc640 1 -- 172.21.15.42:0/3308567914 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96b0018780 con 0x7f96bc109260 2024-08-30T09:02:16.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:15.996+0000 7f96c355c640 1 -- 172.21.15.42:0/3308567914 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96bc1a0420 con 0x7f96bc109260 2024-08-30T09:02:16.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:15.997+0000 7f96c355c640 1 -- 172.21.15.42:0/3308567914 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96bc1a0820 con 0x7f96bc109260 2024-08-30T09:02:16.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:16.000+0000 7f96c9ad1740 1 Processor -- start 2024-08-30T09:02:16.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:16.003+0000 7f96c9ad1740 1 -- start start 2024-08-30T09:02:16.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:16.003+0000 7f96ae7fc640 1 -- 172.21.15.42:0/3308567914 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7f96b0010a90 con 0x7f96bc109260 2024-08-30T09:02:16.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:16.003+0000 7f96c9ad1740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cbb8b8c070 0x55cbb8f31310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:16.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:16.004+0000 7f96c9ad1740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cbb8b631b0 0x55cbb8f31850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:16.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:16.004+0000 7f96c9ad1740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cbb8b6b390 0x55cbb8f31d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:16.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:16.004+0000 7f96c9ad1740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55cbb8ea8680 con 0x55cbb8b8c070 2024-08-30T09:02:16.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:16.004+0000 7f96c9ad1740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55cbb8e97c10 con 0x55cbb8b631b0 2024-08-30T09:02:16.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:16.004+0000 7f96c9ad1740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55cbb8e9bbc0 con 0x55cbb8b6b390 2024-08-30T09:02:16.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:16.005+0000 7f96c1af9640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cbb8b8c070 0x55cbb8f31310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:02:16.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:16.005+0000 7f96c1af9640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cbb8b8c070 0x55cbb8f31310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56738/0 (socket says 172.21.15.42:56738) 2024-08-30T09:02:16.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:16.005+0000 7f96c1af9640 1 -- 172.21.15.42:0/4233286161 learned_addr learned my addr 172.21.15.42:0/4233286161 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:02:16.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:16.010+0000 7f96c12f8640 1 --2- 172.21.15.42:0/4233286161 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cbb8b631b0 0x55cbb8f31850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:02:16.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:16.013+0000 7f96c0af7640 1 --2- 172.21.15.42:0/4233286161 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cbb8b6b390 0x55cbb8f31d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:02:16.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:16.013+0000 7f96c1af9640 1 -- 172.21.15.42:0/4233286161 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cbb8b6b390 msgr2=0x55cbb8f31d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:02:16.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:16.013+0000 7f96c1af9640 1 --2- 172.21.15.42:0/4233286161 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cbb8b6b390 0x55cbb8f31d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:16.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:16.013+0000 7f96c1af9640 1 -- 172.21.15.42:0/4233286161 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cbb8b631b0 msgr2=0x55cbb8f31850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:02:16.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:16.013+0000 7f96c1af9640 1 --2- 172.21.15.42:0/4233286161 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cbb8b631b0 0x55cbb8f31850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:16.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:16.013+0000 7f96c1af9640 1 -- 172.21.15.42:0/4233286161 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cbb8bf7650 con 0x55cbb8b8c070 2024-08-30T09:02:16.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:16.014+0000 7f96c1af9640 1 --2- 172.21.15.42:0/4233286161 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cbb8b8c070 0x55cbb8f31310 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f96b402f410 tx=0x7f96b402f440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16fa979260002112 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:02:16.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:16.014+0000 7f96ae7fc640 1 --2- 172.21.15.42:0/3308567914 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f96a80b4d00 0x7f96a80b7210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:16.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:16.015+0000 7f96a77fe640 1 -- 172.21.15.42:0/4233286161 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96b4031cf0 con 0x55cbb8b8c070 2024-08-30T09:02:16.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:16.015+0000 7f96a77fe640 1 -- 172.21.15.42:0/4233286161 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f96b4036030 con 0x55cbb8b8c070 2024-08-30T09:02:16.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:16.015+0000 7f96ae7fc640 1 -- 172.21.15.42:0/3308567914 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f96b00e5830 con 0x7f96bc109260 2024-08-30T09:02:16.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:16.015+0000 7f96a77fe640 1 -- 172.21.15.42:0/4233286161 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96b40042a0 con 0x55cbb8b8c070 2024-08-30T09:02:16.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:16.016+0000 7f96c9ad1740 1 -- 172.21.15.42:0/4233286161 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cbb8b8c070 msgr2=0x55cbb8f31310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:02:16.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:16.016+0000 7f96c9ad1740 1 --2- 172.21.15.42:0/4233286161 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cbb8b8c070 0x55cbb8f31310 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f96b402f410 tx=0x7f96b402f440 comp rx=0 tx=0).stop 2024-08-30T09:02:16.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:16.016+0000 7f96c9ad1740 1 -- 172.21.15.42:0/4233286161 shutdown_connections 2024-08-30T09:02:16.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:16.016+0000 7f96c9ad1740 1 --2- 172.21.15.42:0/4233286161 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cbb8b6b390 0x55cbb8f31d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:16.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:16.016+0000 7f96c9ad1740 1 --2- 172.21.15.42:0/4233286161 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cbb8b631b0 0x55cbb8f31850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:16.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:16.017+0000 7f96c9ad1740 1 --2- 172.21.15.42:0/4233286161 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cbb8b8c070 0x55cbb8f31310 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:16.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:16.017+0000 7f96c12f8640 1 --2- 172.21.15.42:0/3308567914 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f96a80b4d00 0x7f96a80b7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:02:16.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:16.017+0000 7f96c9ad1740 1 -- 172.21.15.42:0/4233286161 >> 172.21.15.42:0/4233286161 conn(0x55cbb8b80460 msgr2=0x55cbb8f2f200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:02:16.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:16.020+0000 7f96c9ad1740 1 -- 172.21.15.42:0/4233286161 shutdown_connections 2024-08-30T09:02:16.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:16.021+0000 7f96c9ad1740 1 -- 172.21.15.42:0/4233286161 wait complete. 2024-08-30T09:02:16.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:16.022+0000 7f96c9ad1740 1 -- start start 2024-08-30T09:02:16.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:16.022+0000 7f96c9ad1740 1 Processor -- start 2024-08-30T09:02:16.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:16.023+0000 7f96c12f8640 1 --2- 172.21.15.42:0/3308567914 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f96a80b4d00 0x7f96a80b7210 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x55cbb8f2ff90 tx=0x7f96b8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:02:16.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:16.023+0000 7f96c9ad1740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cbb8b631b0 0x55cbb8f3cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:16.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:16.023+0000 7f96c9ad1740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cbb8b6b390 0x55cbb8f3d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:16.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:16.023+0000 7f96c9ad1740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cbb8b8c070 0x55cbb8f48c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:16.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:16.024+0000 7f96c9ad1740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55cbb8cfa700 con 0x55cbb8b8c070 2024-08-30T09:02:16.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:16.024+0000 7f96c9ad1740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55cbb8e1e750 con 0x55cbb8b631b0 2024-08-30T09:02:16.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:16.024+0000 7f96c9ad1740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55cbb8f3f910 con 0x55cbb8b6b390 2024-08-30T09:02:16.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:16.024+0000 7f96c0af7640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cbb8b6b390 0x55cbb8f3d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:02:16.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:16.024+0000 7f96c0af7640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cbb8b6b390 0x55cbb8f3d040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:49554/0 (socket says 172.21.15.42:49554) 2024-08-30T09:02:16.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:16.024+0000 7f96c0af7640 1 -- 172.21.15.42:0/771729043 learned_addr learned my addr 172.21.15.42:0/771729043 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:02:16.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:16.024+0000 7f96c9ad1740 10 client.0 _ll_register_callbacks cb 0x55cbb8e4de10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:02:16.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:16.024+0000 7f96c0af7640 1 -- 172.21.15.42:0/771729043 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cbb8b631b0 msgr2=0x55cbb8f3cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:02:16.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:16.024+0000 7f96c0af7640 1 --2- 172.21.15.42:0/771729043 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cbb8b631b0 0x55cbb8f3cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:16.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:16.024+0000 7f96c0af7640 1 -- 172.21.15.42:0/771729043 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cbb8b8c070 msgr2=0x55cbb8f48c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:02:16.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:16.024+0000 7f96c0af7640 1 --2- 172.21.15.42:0/771729043 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cbb8b8c070 0x55cbb8f48c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:16.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:16.024+0000 7f96c0af7640 1 -- 172.21.15.42:0/771729043 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cbb8e57900 con 0x55cbb8b6b390 2024-08-30T09:02:16.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:16.029+0000 7f96c0af7640 1 --2- 172.21.15.42:0/771729043 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cbb8b6b390 0x55cbb8f3d040 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f96b00a7ef0 tx=0x7f96b0009700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cc338b0be710b76b server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:02:16.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:16.029+0000 7f96a4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-08-30T09:02:16.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:16.031+0000 7f96a4ff9640 1 -- 172.21.15.42:0/771729043 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96b00ab980 con 0x55cbb8b6b390 2024-08-30T09:02:16.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:16.031+0000 7f96a4ff9640 1 -- 172.21.15.42:0/771729043 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f96b00e5df0 con 0x55cbb8b6b390 2024-08-30T09:02:16.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:16.032+0000 7f96c9ad1740 1 -- 172.21.15.42:0/771729043 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f96b4036030 con 0x55cbb8b6b390 2024-08-30T09:02:16.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:16.034+0000 7f96a4ff9640 1 -- 172.21.15.42:0/771729043 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f96b00ac570 con 0x55cbb8b6b390 2024-08-30T09:02:16.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:16.034+0000 7f96c9ad1740 10 client.25399 fetch_fsmap learned FSMap version 29 2024-08-30T09:02:16.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:16.034+0000 7f96c9ad1740 1 -- 172.21.15.42:0/771729043 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55cbb8f4b780 con 0x55cbb8b6b390 2024-08-30T09:02:16.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:16.035+0000 7f96a4ff9640 1 -- 172.21.15.42:0/771729043 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f96b4036030 con 0x55cbb8b6b390 2024-08-30T09:02:16.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:16.035+0000 7f96a4ff9640 1 -- 172.21.15.42:0/771729043 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f96b00a77e0 con 0x55cbb8b6b390 2024-08-30T09:02:16.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:16.035+0000 7f96c9ad1740 10 client.25399 fetch_fsmap finished waiting for FSMap version 29 2024-08-30T09:02:16.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:16.035+0000 7f96c9ad1740 10 client.25399 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-08-30T09:02:16.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:16.035+0000 7f96c9ad1740 20 client.25399 populate_metadata read hostname 'smithi042' 2024-08-30T09:02:16.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:16.035+0000 7f96c9ad1740 1 --2- 172.21.15.42:0/771729043 >> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] conn(0x55cbb8b4a9f0 0x55cbb8f4ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:16.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:16.035+0000 7f96c9ad1740 4 client.25399 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-30T09:02:16.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:16.035+0000 7f96c9ad1740 1 -- 172.21.15.42:0/771729043 --> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cbb8eaa5d0 con 0x55cbb8b4a9f0 2024-08-30T09:02:16.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:16.037+0000 7f96c12f8640 1 --2- 172.21.15.42:0/771729043 >> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] conn(0x55cbb8b4a9f0 0x55cbb8f4ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:02:16.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:16.046+0000 7f96a4ff9640 10 client.25399 ms_handle_connect on v2:172.21.15.190:6838/719981110 2024-08-30T09:02:16.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:16.046+0000 7f96c12f8640 1 --2- 172.21.15.42:0/771729043 >> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] conn(0x55cbb8b4a9f0 0x55cbb8f4ecb0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=208f5618c27e1147 server_cookie=7baa43c8dfe93a32 in_seq=0 out_seq=0 2024-08-30T09:02:16.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:16.047+0000 7f96a4ff9640 1 -- 172.21.15.42:0/771729043 <== mds.1 v2:172.21.15.190:6838/719981110 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55cbb8eaa5d0 con 0x55cbb8b4a9f0 2024-08-30T09:02:16.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:16.047+0000 7f96a4ff9640 10 client.25399 handle_command_reply: tid=0 2024-08-30T09:02:16.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:16.048+0000 7f96c9ad1740 1 client.25399 shutdown 2024-08-30T09:02:16.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:16.050+0000 7f96c9ad1740 1 -- 172.21.15.42:0/771729043 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cbb8b6b390 msgr2=0x55cbb8f3d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:02:16.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:16.050+0000 7f96c9ad1740 1 --2- 172.21.15.42:0/771729043 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cbb8b6b390 0x55cbb8f3d040 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f96b00a7ef0 tx=0x7f96b0009700 comp rx=0 tx=0).stop 2024-08-30T09:02:16.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:16.050+0000 7f96c9ad1740 1 -- 172.21.15.42:0/771729043 shutdown_connections 2024-08-30T09:02:16.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:16.050+0000 7f96c9ad1740 1 --2- 172.21.15.42:0/771729043 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cbb8b6b390 0x55cbb8f3d040 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:16.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:16.050+0000 7f96c9ad1740 1 --2- 172.21.15.42:0/771729043 >> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] conn(0x55cbb8b4a9f0 0x55cbb8f4ecb0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:16.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:16.050+0000 7f96c9ad1740 1 --2- 172.21.15.42:0/771729043 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cbb8b8c070 0x55cbb8f48c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:16.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:16.050+0000 7f96c9ad1740 1 --2- 172.21.15.42:0/771729043 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cbb8b631b0 0x55cbb8f3cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:16.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:16.050+0000 7f96a4ff9640 0 client.25399 ms_handle_reset on v2:172.21.15.190:6838/719981110 2024-08-30T09:02:16.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:16.050+0000 7f96c9ad1740 1 -- 172.21.15.42:0/771729043 >> 172.21.15.42:0/771729043 conn(0x55cbb8b80460 msgr2=0x55cbb8f344f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:02:16.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:16.050+0000 7f96c9ad1740 1 -- 172.21.15.42:0/771729043 shutdown_connections 2024-08-30T09:02:16.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:16.051+0000 7f96c9ad1740 1 -- 172.21.15.42:0/771729043 wait complete. 2024-08-30T09:02:16.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:16.053+0000 7f96c9ad1740 20 client.25399 trim_cache size 0 max 16384 2024-08-30T09:02:16.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:16.080+0000 7f96c9ad1740 1 Processor -- start 2024-08-30T09:02:16.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:16.080+0000 7f96c9ad1740 1 -- start start 2024-08-30T09:02:16.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:16.081+0000 7f96c9ad1740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cbb8b631b0 0x55cbb8f2f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:16.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:16.081+0000 7f96c9ad1740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cbb8b6b390 0x55cbb8f48b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:16.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:16.081+0000 7f96c9ad1740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cbb8b8c070 0x55cbb8f49040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:16.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:16.081+0000 7f96c9ad1740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f96bc1b96c0 con 0x55cbb8b631b0 2024-08-30T09:02:16.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:16.081+0000 7f96c9ad1740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f96bc1b6f90 con 0x55cbb8b8c070 2024-08-30T09:02:16.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:16.081+0000 7f96c9ad1740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f96bc1b4de0 con 0x55cbb8b6b390 2024-08-30T09:02:16.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:16.081+0000 7f96c1af9640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cbb8b8c070 0x55cbb8f49040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:02:16.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:16.081+0000 7f96c12f8640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cbb8b631b0 0x55cbb8f2f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:02:16.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:16.081+0000 7f96c1af9640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cbb8b8c070 0x55cbb8f49040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:59962/0 (socket says 172.21.15.42:59962) 2024-08-30T09:02:16.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:16.081+0000 7f96c12f8640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cbb8b631b0 0x55cbb8f2f0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56756/0 (socket says 172.21.15.42:56756) 2024-08-30T09:02:16.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:16.081+0000 7f96c1af9640 1 -- 172.21.15.42:0/2304773359 learned_addr learned my addr 172.21.15.42:0/2304773359 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:02:16.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:16.082+0000 7f96c1af9640 1 -- 172.21.15.42:0/2304773359 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cbb8b6b390 msgr2=0x55cbb8f48b00 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-30T09:02:16.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:16.082+0000 7f96c1af9640 1 --2- 172.21.15.42:0/2304773359 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cbb8b6b390 0x55cbb8f48b00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:16.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:16.082+0000 7f96c1af9640 1 -- 172.21.15.42:0/2304773359 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cbb8b631b0 msgr2=0x55cbb8f2f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:02:16.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:16.082+0000 7f96c1af9640 1 --2- 172.21.15.42:0/2304773359 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cbb8b631b0 0x55cbb8f2f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:16.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:16.082+0000 7f96c1af9640 1 -- 172.21.15.42:0/2304773359 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cbb8f3f240 con 0x55cbb8b8c070 2024-08-30T09:02:16.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:16.082+0000 7f96c1af9640 1 --2- 172.21.15.42:0/2304773359 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cbb8b8c070 0x55cbb8f49040 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f96b4005e30 tx=0x7f96b40095c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ff53a4fe1b9fc51b server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:02:16.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:16.082+0000 7f96a6ffd640 1 -- 172.21.15.42:0/2304773359 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96b4031d00 con 0x55cbb8b8c070 2024-08-30T09:02:16.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:16.082+0000 7f96a6ffd640 1 -- 172.21.15.42:0/2304773359 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f96b4036070 con 0x55cbb8b8c070 2024-08-30T09:02:16.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:16.082+0000 7f96a6ffd640 1 -- 172.21.15.42:0/2304773359 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96b4004040 con 0x55cbb8b8c070 2024-08-30T09:02:16.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:16.084+0000 7f96c9ad1740 1 -- 172.21.15.42:0/2304773359 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cbb8b8c070 msgr2=0x55cbb8f49040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:02:16.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:16.084+0000 7f96c9ad1740 1 --2- 172.21.15.42:0/2304773359 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cbb8b8c070 0x55cbb8f49040 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f96b4005e30 tx=0x7f96b40095c0 comp rx=0 tx=0).stop 2024-08-30T09:02:16.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:16.084+0000 7f96c9ad1740 1 -- 172.21.15.42:0/2304773359 shutdown_connections 2024-08-30T09:02:16.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:16.084+0000 7f96c9ad1740 1 --2- 172.21.15.42:0/2304773359 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cbb8b6b390 0x55cbb8f48b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:16.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:16.084+0000 7f96c9ad1740 1 --2- 172.21.15.42:0/2304773359 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cbb8b8c070 0x55cbb8f49040 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:16.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:16.084+0000 7f96c9ad1740 1 --2- 172.21.15.42:0/2304773359 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cbb8b631b0 0x55cbb8f2f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:16.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:16.084+0000 7f96c9ad1740 1 -- 172.21.15.42:0/2304773359 >> 172.21.15.42:0/2304773359 conn(0x55cbb8b80460 msgr2=0x55cbb8f2e9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:02:16.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:16.084+0000 7f96c9ad1740 1 -- 172.21.15.42:0/2304773359 shutdown_connections 2024-08-30T09:02:16.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:16.084+0000 7f96c9ad1740 1 -- 172.21.15.42:0/2304773359 wait complete. 2024-08-30T09:02:16.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:16.084+0000 7f96c9ad1740 1 -- start start 2024-08-30T09:02:16.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:16.085+0000 7f96c9ad1740 1 Processor -- start 2024-08-30T09:02:16.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:16.085+0000 7f96c9ad1740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cbb8b631b0 0x55cbb8f44b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:16.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:16.085+0000 7f96c9ad1740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cbb8b6b390 0x55cbb8f450b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:16.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:16.085+0000 7f96c9ad1740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cbb8b8c070 0x55cbb8f3fb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:16.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:16.085+0000 7f96c9ad1740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f96bc1b5450 con 0x55cbb8b8c070 2024-08-30T09:02:16.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:16.085+0000 7f96c9ad1740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f96bc1b68c0 con 0x55cbb8b6b390 2024-08-30T09:02:16.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:16.085+0000 7f96c9ad1740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f96bc1b5b20 con 0x55cbb8b631b0 2024-08-30T09:02:16.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:16.085+0000 7f96c9ad1740 10 client.0 _ll_register_callbacks cb 0x7f96b00a7d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:02:16.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:16.086+0000 7f96c12f8640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cbb8b631b0 0x55cbb8f44b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:02:16.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:16.086+0000 7f96c12f8640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cbb8b631b0 0x55cbb8f44b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:49576/0 (socket says 172.21.15.42:49576) 2024-08-30T09:02:16.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:16.086+0000 7f96c12f8640 1 -- 172.21.15.42:0/2581415890 learned_addr learned my addr 172.21.15.42:0/2581415890 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:02:16.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:16.086+0000 7f96c0af7640 1 --2- 172.21.15.42:0/2581415890 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cbb8b6b390 0x55cbb8f450b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:02:16.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:16.086+0000 7f96c12f8640 1 -- 172.21.15.42:0/2581415890 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cbb8b6b390 msgr2=0x55cbb8f450b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:02:16.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:16.086+0000 7f96c12f8640 1 --2- 172.21.15.42:0/2581415890 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cbb8b6b390 0x55cbb8f450b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:16.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:16.086+0000 7f96c12f8640 1 -- 172.21.15.42:0/2581415890 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cbb8b8c070 msgr2=0x55cbb8f3fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:02:16.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:16.086+0000 7f96c12f8640 1 --2- 172.21.15.42:0/2581415890 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cbb8b8c070 0x55cbb8f3fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:16.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:16.086+0000 7f96c12f8640 1 -- 172.21.15.42:0/2581415890 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cbb8f457a0 con 0x55cbb8b631b0 2024-08-30T09:02:16.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:16.087+0000 7f96c12f8640 1 --2- 172.21.15.42:0/2581415890 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cbb8b631b0 0x55cbb8f44b70 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x55cbb8f4f420 tx=0x7f96b8015040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c467e186ee729ffa server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:02:16.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:16.087+0000 7f96a4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-08-30T09:02:16.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:16.087+0000 7f96a4ff9640 1 -- 172.21.15.42:0/2581415890 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96b8030680 con 0x55cbb8b631b0 2024-08-30T09:02:16.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:16.087+0000 7f96c9ad1740 1 -- 172.21.15.42:0/2581415890 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f96b4036070 con 0x55cbb8b631b0 2024-08-30T09:02:16.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:16.087+0000 7f96a4ff9640 1 -- 172.21.15.42:0/2581415890 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f96b8030820 con 0x55cbb8b631b0 2024-08-30T09:02:16.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:16.087+0000 7f96a4ff9640 1 -- 172.21.15.42:0/2581415890 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f96b8030a20 con 0x55cbb8b631b0 2024-08-30T09:02:16.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:16.088+0000 7f96c9ad1740 10 client.25402 fetch_fsmap learned FSMap version 29 2024-08-30T09:02:16.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:16.088+0000 7f96c9ad1740 1 -- 172.21.15.42:0/2581415890 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55cbb8fcabd0 con 0x55cbb8b631b0 2024-08-30T09:02:16.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:16.088+0000 7f96a4ff9640 1 -- 172.21.15.42:0/2581415890 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55cbb8eaa5d0 con 0x55cbb8b631b0 2024-08-30T09:02:16.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:16.088+0000 7f96a4ff9640 1 -- 172.21.15.42:0/2581415890 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f96b803aca0 con 0x55cbb8b631b0 2024-08-30T09:02:16.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:16.088+0000 7f96c9ad1740 10 client.25402 fetch_fsmap finished waiting for FSMap version 29 2024-08-30T09:02:16.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:16.088+0000 7f96c9ad1740 10 client.25402 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-08-30T09:02:16.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:16.088+0000 7f96c9ad1740 20 client.25402 populate_metadata read hostname 'smithi042' 2024-08-30T09:02:16.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:16.088+0000 7f96c9ad1740 1 --2- 172.21.15.42:0/2581415890 >> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] conn(0x55cbb8b1fda0 0x55cbb8fcd370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:16.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:16.088+0000 7f96c9ad1740 4 client.25402 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-30T09:02:16.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:16.088+0000 7f96c9ad1740 1 -- 172.21.15.42:0/2581415890 --> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f96b4036070 con 0x55cbb8b1fda0 2024-08-30T09:02:16.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:16.088+0000 7f96c0af7640 1 --2- 172.21.15.42:0/2581415890 >> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] conn(0x55cbb8b1fda0 0x55cbb8fcd370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:02:16.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:16.089+0000 7f96c0af7640 1 --2- 172.21.15.42:0/2581415890 >> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] conn(0x55cbb8b1fda0 0x55cbb8fcd370 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5c92fb247901ab9b server_cookie=dcc9d8bd1c45123e in_seq=0 out_seq=0 2024-08-30T09:02:16.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:16.089+0000 7f96a4ff9640 10 client.25402 ms_handle_connect on v2:172.21.15.190:6838/719981110 2024-08-30T09:02:16.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:16.089+0000 7f96a4ff9640 1 -- 172.21.15.42:0/2581415890 <== mds.1 v2:172.21.15.190:6838/719981110 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f96b4036070 con 0x55cbb8b1fda0 2024-08-30T09:02:16.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:16.089+0000 7f96a4ff9640 10 client.25402 handle_command_reply: tid=0 2024-08-30T09:02:16.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:16.089+0000 7f96c9ad1740 1 client.25402 shutdown 2024-08-30T09:02:16.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:16.090+0000 7f96c9ad1740 1 -- 172.21.15.42:0/2581415890 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cbb8b631b0 msgr2=0x55cbb8f44b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:02:16.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:16.090+0000 7f96c9ad1740 1 --2- 172.21.15.42:0/2581415890 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cbb8b631b0 0x55cbb8f44b70 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x55cbb8f4f420 tx=0x7f96b8015040 comp rx=0 tx=0).stop 2024-08-30T09:02:16.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:16.090+0000 7f96c9ad1740 1 -- 172.21.15.42:0/2581415890 shutdown_connections 2024-08-30T09:02:16.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:16.090+0000 7f96c9ad1740 1 --2- 172.21.15.42:0/2581415890 >> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] conn(0x55cbb8b1fda0 0x55cbb8fcd370 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:16.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:16.090+0000 7f96c9ad1740 1 --2- 172.21.15.42:0/2581415890 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cbb8b8c070 0x55cbb8f3fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:16.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:16.090+0000 7f96c9ad1740 1 --2- 172.21.15.42:0/2581415890 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cbb8b6b390 0x55cbb8f450b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:16.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:16.090+0000 7f96c9ad1740 1 --2- 172.21.15.42:0/2581415890 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cbb8b631b0 0x55cbb8f44b70 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:16.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:16.090+0000 7f96a4ff9640 0 client.25402 ms_handle_reset on v2:172.21.15.190:6838/719981110 2024-08-30T09:02:16.240 INFO:teuthology.orchestra.run.smithi042.stdout:[] 2024-08-30T09:02:16.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:16.090+0000 7f96c9ad1740 1 -- 172.21.15.42:0/2581415890 >> 172.21.15.42:0/2581415890 conn(0x55cbb8b80460 msgr2=0x55cbb8f2d7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:02:16.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:16.090+0000 7f96c9ad1740 1 -- 172.21.15.42:0/2581415890 shutdown_connections 2024-08-30T09:02:16.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:16.090+0000 7f96c9ad1740 1 -- 172.21.15.42:0/2581415890 wait complete. 2024-08-30T09:02:16.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:16.090+0000 7f96c9ad1740 20 client.25402 trim_cache size 0 max 16384 2024-08-30T09:02:16.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:16.091+0000 7f96c355c640 1 -- 172.21.15.42:0/3308567914 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f96a80b4d00 msgr2=0x7f96a80b7210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:02:16.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:16.091+0000 7f96c355c640 1 --2- 172.21.15.42:0/3308567914 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f96a80b4d00 0x7f96a80b7210 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x55cbb8f2ff90 tx=0x7f96b8005f90 comp rx=0 tx=0).stop 2024-08-30T09:02:16.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:16.091+0000 7f96c355c640 1 -- 172.21.15.42:0/3308567914 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f96bc109260 msgr2=0x7f96bc19a760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:02:16.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:16.091+0000 7f96c355c640 1 --2- 172.21.15.42:0/3308567914 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f96bc109260 0x7f96bc19a760 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f96bc06c7b0 tx=0x7f96b000ee50 comp rx=0 tx=0).stop 2024-08-30T09:02:16.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:16.091+0000 7f96c355c640 1 -- 172.21.15.42:0/3308567914 shutdown_connections 2024-08-30T09:02:16.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:16.091+0000 7f96c355c640 1 --2- 172.21.15.42:0/3308567914 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f96a80b4d00 0x7f96a80b7210 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:16.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:16.091+0000 7f96c355c640 1 --2- 172.21.15.42:0/3308567914 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f96bc109260 0x7f96bc19a760 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:16.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:16.091+0000 7f96c355c640 1 --2- 172.21.15.42:0/3308567914 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f96bc109bb0 0x7f96bc19fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:16.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:16.091+0000 7f96c355c640 1 --2- 172.21.15.42:0/3308567914 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f96bc103920 0x7f96bc19a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:16.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:16.091+0000 7f96c355c640 1 -- 172.21.15.42:0/3308567914 >> 172.21.15.42:0/3308567914 conn(0x7f96bc0ff370 msgr2=0x7f96bc101790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:02:16.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:16.091+0000 7f96c355c640 1 -- 172.21.15.42:0/3308567914 shutdown_connections 2024-08-30T09:02:16.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:16.091+0000 7f96c355c640 1 -- 172.21.15.42:0/3308567914 wait complete. 2024-08-30T09:02:16.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:02:16 smithi042 ceph-mon[18652]: pgmap v418: 129 pgs: 129 active+clean; 1.0 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 21 MiB/s wr, 1.00k op/s 2024-08-30T09:02:17.048 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-08-30T09:02:17.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:02:16 smithi167 ceph-mon[22063]: pgmap v418: 129 pgs: 129 active+clean; 1.0 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 21 MiB/s wr, 1.00k op/s 2024-08-30T09:02:17.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:02:16 smithi190 ceph-mon[22001]: pgmap v418: 129 pgs: 129 active+clean; 1.0 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 21 MiB/s wr, 1.00k op/s 2024-08-30T09:02:17.435 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T09:02:18.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.016+0000 7f1be1835640 1 -- 172.21.15.42:0/3284169407 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1bdc108a90 msgr2=0x7f1bdc10f2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:02:18.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.016+0000 7f1be1835640 1 --2- 172.21.15.42:0/3284169407 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1bdc108a90 0x7f1bdc10f2c0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f1bd4009900 tx=0x7f1bd402a410 comp rx=0 tx=0).stop 2024-08-30T09:02:18.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.016+0000 7f1be1835640 1 -- 172.21.15.42:0/3284169407 shutdown_connections 2024-08-30T09:02:18.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.016+0000 7f1be1835640 1 --2- 172.21.15.42:0/3284169407 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1bdc108140 0x7f1bdc108550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:18.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.016+0000 7f1be1835640 1 --2- 172.21.15.42:0/3284169407 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1bdc108a90 0x7f1bdc10f2c0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:18.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.016+0000 7f1be1835640 1 --2- 172.21.15.42:0/3284169407 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1bdc102800 0x7f1bdc102c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:18.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.016+0000 7f1be1835640 1 -- 172.21.15.42:0/3284169407 >> 172.21.15.42:0/3284169407 conn(0x7f1bdc0794d0 msgr2=0x7f1bdc0798e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:02:18.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.016+0000 7f1be1835640 1 -- 172.21.15.42:0/3284169407 shutdown_connections 2024-08-30T09:02:18.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.016+0000 7f1be1835640 1 -- 172.21.15.42:0/3284169407 wait complete. 2024-08-30T09:02:18.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.017+0000 7f1be1835640 1 Processor -- start 2024-08-30T09:02:18.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.017+0000 7f1be1835640 1 -- start start 2024-08-30T09:02:18.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.018+0000 7f1be1835640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1bdc102800 0x7f1bdc19b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:18.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.018+0000 7f1be1835640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1bdc108140 0x7f1bdc19b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:18.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.018+0000 7f1be1835640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1bdc108a90 0x7f1bdc19baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:18.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.018+0000 7f1be1835640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f1bdc112740 con 0x7f1bdc102800 2024-08-30T09:02:18.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.018+0000 7f1be1835640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f1bdc1125c0 con 0x7f1bdc108140 2024-08-30T09:02:18.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.018+0000 7f1be1835640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f1bdc1128c0 con 0x7f1bdc108a90 2024-08-30T09:02:18.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.018+0000 7f1bda7fc640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1bdc108140 0x7f1bdc19b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:02:18.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.018+0000 7f1bdaffd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1bdc102800 0x7f1bdc19b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:02:18.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.018+0000 7f1bdb7fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1bdc108a90 0x7f1bdc19baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:02:18.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.018+0000 7f1bdaffd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1bdc102800 0x7f1bdc19b070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:51530/0 (socket says 172.21.15.42:51530) 2024-08-30T09:02:18.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.018+0000 7f1bda7fc640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1bdc108140 0x7f1bdc19b5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:35068/0 (socket says 172.21.15.42:35068) 2024-08-30T09:02:18.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.018+0000 7f1bdaffd640 1 -- 172.21.15.42:0/4025380014 learned_addr learned my addr 172.21.15.42:0/4025380014 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:02:18.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.018+0000 7f1bdb7fe640 1 -- 172.21.15.42:0/4025380014 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1bdc108140 msgr2=0x7f1bdc19b5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:02:18.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.018+0000 7f1bdb7fe640 1 --2- 172.21.15.42:0/4025380014 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1bdc108140 0x7f1bdc19b5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:18.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.018+0000 7f1bdb7fe640 1 -- 172.21.15.42:0/4025380014 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1bdc102800 msgr2=0x7f1bdc19b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:02:18.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.018+0000 7f1bdb7fe640 1 --2- 172.21.15.42:0/4025380014 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1bdc102800 0x7f1bdc19b070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:18.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.018+0000 7f1bdb7fe640 1 -- 172.21.15.42:0/4025380014 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bdc1a6300 con 0x7f1bdc108a90 2024-08-30T09:02:18.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.019+0000 7f1bdb7fe640 1 --2- 172.21.15.42:0/4025380014 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1bdc108a90 0x7f1bdc19baf0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f1bd402a930 tx=0x7f1bd4003b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:02:18.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.019+0000 7f1be0833640 1 -- 172.21.15.42:0/4025380014 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bd403f070 con 0x7f1bdc108a90 2024-08-30T09:02:18.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.019+0000 7f1be0833640 1 -- 172.21.15.42:0/4025380014 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1bd4031b50 con 0x7f1bdc108a90 2024-08-30T09:02:18.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.019+0000 7f1be0833640 1 -- 172.21.15.42:0/4025380014 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bd4035030 con 0x7f1bdc108a90 2024-08-30T09:02:18.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.019+0000 7f1be1835640 1 -- 172.21.15.42:0/4025380014 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1bdc1a6590 con 0x7f1bdc108a90 2024-08-30T09:02:18.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.019+0000 7f1be1835640 1 -- 172.21.15.42:0/4025380014 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1bdc1a6a20 con 0x7f1bdc108a90 2024-08-30T09:02:18.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.023+0000 7f1be7daa740 1 Processor -- start 2024-08-30T09:02:18.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.024+0000 7f1be0833640 1 -- 172.21.15.42:0/4025380014 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7f1bd4042070 con 0x7f1bdc108a90 2024-08-30T09:02:18.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.024+0000 7f1be7daa740 1 -- start start 2024-08-30T09:02:18.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.024+0000 7f1be7daa740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5654d4675070 0x5654d4a1a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:18.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.024+0000 7f1be7daa740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5654d464c1b0 0x5654d4a1a850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:18.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.024+0000 7f1be7daa740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5654d4654390 0x5654d4a1ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:18.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.024+0000 7f1be7daa740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x5654d4991680 con 0x5654d4654390 2024-08-30T09:02:18.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.024+0000 7f1be7daa740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5654d4980c10 con 0x5654d4675070 2024-08-30T09:02:18.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.024+0000 7f1be7daa740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5654d4984bc0 con 0x5654d464c1b0 2024-08-30T09:02:18.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.025+0000 7f1bdaffd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5654d464c1b0 0x5654d4a1a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:02:18.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.025+0000 7f1bdaffd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5654d464c1b0 0x5654d4a1a850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:43794/0 (socket says 172.21.15.42:43794) 2024-08-30T09:02:18.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.025+0000 7f1bdaffd640 1 -- 172.21.15.42:0/3362371214 learned_addr learned my addr 172.21.15.42:0/3362371214 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:02:18.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.025+0000 7f1bda7fc640 1 --2- 172.21.15.42:0/3362371214 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5654d4675070 0x5654d4a1a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:02:18.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.025+0000 7f1be0833640 1 --2- 172.21.15.42:0/4025380014 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f1bc80b4be0 0x7f1bc80b70f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:18.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.025+0000 7f1bda7fc640 1 --2- 172.21.15.42:0/3362371214 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5654d4654390 0x5654d4a1ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:02:18.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.025+0000 7f1bdb7fe640 1 --2- 172.21.15.42:0/4025380014 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f1bc80b4be0 0x7f1bc80b70f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:02:18.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.025+0000 7f1bdaffd640 1 -- 172.21.15.42:0/3362371214 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5654d4675070 msgr2=0x5654d4a1a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:02:18.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.025+0000 7f1bdaffd640 1 --2- 172.21.15.42:0/3362371214 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5654d4675070 0x5654d4a1a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:18.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.025+0000 7f1bdaffd640 1 -- 172.21.15.42:0/3362371214 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5654d4654390 msgr2=0x5654d4a1ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:02:18.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.025+0000 7f1bdaffd640 1 --2- 172.21.15.42:0/3362371214 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5654d4654390 0x5654d4a1ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:18.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.025+0000 7f1bdaffd640 1 -- 172.21.15.42:0/3362371214 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5654d46e0650 con 0x5654d464c1b0 2024-08-30T09:02:18.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.025+0000 7f1bdaffd640 1 --2- 172.21.15.42:0/3362371214 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5654d464c1b0 0x5654d4a1a850 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f1bcc009f70 tx=0x7f1bcc011a50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b25316a3909cf9de server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:02:18.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.026+0000 7f1be0833640 1 -- 172.21.15.42:0/4025380014 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f1bd4105990 con 0x7f1bdc108a90 2024-08-30T09:02:18.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.026+0000 7f1bdb7fe640 1 --2- 172.21.15.42:0/4025380014 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f1bc80b4be0 0x7f1bc80b70f0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f1bd40cbde0 tx=0x7f1bd40c8820 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:02:18.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.026+0000 7f1bd1ffb640 1 -- 172.21.15.42:0/3362371214 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bcc023070 con 0x5654d464c1b0 2024-08-30T09:02:18.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.026+0000 7f1bd1ffb640 1 -- 172.21.15.42:0/3362371214 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1bcc00fc90 con 0x5654d464c1b0 2024-08-30T09:02:18.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.026+0000 7f1bd1ffb640 1 -- 172.21.15.42:0/3362371214 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bcc016c90 con 0x5654d464c1b0 2024-08-30T09:02:18.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.027+0000 7f1be7daa740 1 -- 172.21.15.42:0/3362371214 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5654d464c1b0 msgr2=0x5654d4a1a850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:02:18.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.027+0000 7f1be7daa740 1 --2- 172.21.15.42:0/3362371214 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5654d464c1b0 0x5654d4a1a850 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f1bcc009f70 tx=0x7f1bcc011a50 comp rx=0 tx=0).stop 2024-08-30T09:02:18.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.027+0000 7f1be7daa740 1 -- 172.21.15.42:0/3362371214 shutdown_connections 2024-08-30T09:02:18.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.027+0000 7f1be7daa740 1 --2- 172.21.15.42:0/3362371214 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5654d464c1b0 0x5654d4a1a850 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:18.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.027+0000 7f1be7daa740 1 --2- 172.21.15.42:0/3362371214 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5654d4654390 0x5654d4a1ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:18.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.027+0000 7f1be7daa740 1 --2- 172.21.15.42:0/3362371214 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5654d4675070 0x5654d4a1a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:18.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.027+0000 7f1be7daa740 1 -- 172.21.15.42:0/3362371214 >> 172.21.15.42:0/3362371214 conn(0x5654d4669460 msgr2=0x5654d4a18200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:02:18.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.027+0000 7f1be7daa740 1 -- 172.21.15.42:0/3362371214 shutdown_connections 2024-08-30T09:02:18.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.027+0000 7f1be7daa740 1 -- 172.21.15.42:0/3362371214 wait complete. 2024-08-30T09:02:18.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.027+0000 7f1be7daa740 1 -- start start 2024-08-30T09:02:18.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.027+0000 7f1be7daa740 1 Processor -- start 2024-08-30T09:02:18.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.028+0000 7f1be7daa740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5654d464c1b0 0x5654d4a259b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:18.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.028+0000 7f1be7daa740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5654d4654390 0x5654d4a25ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:18.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.028+0000 7f1be7daa740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5654d4675070 0x5654d4a31de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:18.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.028+0000 7f1be7daa740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x5654d47e3700 con 0x5654d464c1b0 2024-08-30T09:02:18.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.028+0000 7f1bda7fc640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5654d464c1b0 0x5654d4a259b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:02:18.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.028+0000 7f1bda7fc640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5654d464c1b0 0x5654d4a259b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:51552/0 (socket says 172.21.15.42:51552) 2024-08-30T09:02:18.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.028+0000 7f1bda7fc640 1 -- 172.21.15.42:0/2013742234 learned_addr learned my addr 172.21.15.42:0/2013742234 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:02:18.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.028+0000 7f1be7daa740 1 -- 172.21.15.42:0/2013742234 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5654d4907750 con 0x5654d4654390 2024-08-30T09:02:18.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.028+0000 7f1be7daa740 1 -- 172.21.15.42:0/2013742234 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5654d4a28af0 con 0x5654d4675070 2024-08-30T09:02:18.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.028+0000 7f1be7daa740 10 client.0 _ll_register_callbacks cb 0x5654d4936e10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:02:18.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.029+0000 7f1bdaffd640 1 --2- 172.21.15.42:0/2013742234 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5654d4654390 0x5654d4a25ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:02:18.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.029+0000 7f1bda7fc640 1 --2- 172.21.15.42:0/2013742234 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5654d4675070 0x5654d4a31de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:02:18.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.029+0000 7f1bda7fc640 1 -- 172.21.15.42:0/2013742234 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5654d4654390 msgr2=0x5654d4a25ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:02:18.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.029+0000 7f1bda7fc640 1 --2- 172.21.15.42:0/2013742234 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5654d4654390 0x5654d4a25ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:18.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.029+0000 7f1bda7fc640 1 -- 172.21.15.42:0/2013742234 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5654d464c1b0 msgr2=0x5654d4a259b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:02:18.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.029+0000 7f1bda7fc640 1 --2- 172.21.15.42:0/2013742234 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5654d464c1b0 0x5654d4a259b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:18.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.029+0000 7f1bda7fc640 1 -- 172.21.15.42:0/2013742234 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5654d4940900 con 0x5654d4675070 2024-08-30T09:02:18.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.029+0000 7f1bda7fc640 1 --2- 172.21.15.42:0/2013742234 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5654d4675070 0x5654d4a31de0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f1bc4010f80 tx=0x7f1bc4005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=873ad9483434fd11 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:02:18.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.029+0000 7f1bbaffd640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-08-30T09:02:18.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.030+0000 7f1bbaffd640 1 -- 172.21.15.42:0/2013742234 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bc4026060 con 0x5654d4675070 2024-08-30T09:02:18.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.030+0000 7f1bbaffd640 1 -- 172.21.15.42:0/2013742234 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f1bc4021400 con 0x5654d4675070 2024-08-30T09:02:18.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.030+0000 7f1be7daa740 1 -- 172.21.15.42:0/2013742234 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1bcc00fc90 con 0x5654d4675070 2024-08-30T09:02:18.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.030+0000 7f1bbaffd640 1 -- 172.21.15.42:0/2013742234 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f1bc402d050 con 0x5654d4675070 2024-08-30T09:02:18.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.030+0000 7f1be7daa740 10 client.25414 fetch_fsmap learned FSMap version 29 2024-08-30T09:02:18.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.030+0000 7f1be7daa740 1 -- 172.21.15.42:0/2013742234 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5654d4a3d050 con 0x5654d4675070 2024-08-30T09:02:18.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.030+0000 7f1bbaffd640 1 -- 172.21.15.42:0/2013742234 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1bc4021ab0 con 0x5654d4675070 2024-08-30T09:02:18.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.031+0000 7f1bbaffd640 1 -- 172.21.15.42:0/2013742234 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f1bc4020040 con 0x5654d4675070 2024-08-30T09:02:18.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.031+0000 7f1be7daa740 10 client.25414 fetch_fsmap finished waiting for FSMap version 29 2024-08-30T09:02:18.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.031+0000 7f1be7daa740 10 client.25414 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-30T09:02:18.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.031+0000 7f1be7daa740 20 client.25414 populate_metadata read hostname 'smithi042' 2024-08-30T09:02:18.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.031+0000 7f1be7daa740 1 --2- 172.21.15.42:0/2013742234 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x5654d46339f0 0x5654d4a40580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:18.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.031+0000 7f1be7daa740 4 client.25414 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-30T09:02:18.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.031+0000 7f1be7daa740 1 -- 172.21.15.42:0/2013742234 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f1bcc00fc90 con 0x5654d46339f0 2024-08-30T09:02:18.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.031+0000 7f1bdaffd640 1 --2- 172.21.15.42:0/2013742234 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x5654d46339f0 0x5654d4a40580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:02:18.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.031+0000 7f1bdaffd640 1 --2- 172.21.15.42:0/2013742234 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x5654d46339f0 0x5654d4a40580 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c421728c4f4c1a23 server_cookie=253fc73c6763f4d9 in_seq=0 out_seq=0 2024-08-30T09:02:18.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.031+0000 7f1bbaffd640 10 client.25414 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:02:18.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.032+0000 7f1bbaffd640 1 -- 172.21.15.42:0/2013742234 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f1bcc00fc90 con 0x5654d46339f0 2024-08-30T09:02:18.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.032+0000 7f1bbaffd640 10 client.25414 handle_command_reply: tid=0 2024-08-30T09:02:18.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.032+0000 7f1be7daa740 1 client.25414 shutdown 2024-08-30T09:02:18.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.032+0000 7f1be7daa740 1 -- 172.21.15.42:0/2013742234 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5654d4675070 msgr2=0x5654d4a31de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:02:18.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.032+0000 7f1be7daa740 1 --2- 172.21.15.42:0/2013742234 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5654d4675070 0x5654d4a31de0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f1bc4010f80 tx=0x7f1bc4005f90 comp rx=0 tx=0).stop 2024-08-30T09:02:18.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.032+0000 7f1be7daa740 1 -- 172.21.15.42:0/2013742234 shutdown_connections 2024-08-30T09:02:18.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.032+0000 7f1be7daa740 1 --2- 172.21.15.42:0/2013742234 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x5654d46339f0 0x5654d4a40580 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:18.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.032+0000 7f1be7daa740 1 --2- 172.21.15.42:0/2013742234 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5654d4675070 0x5654d4a31de0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:18.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.032+0000 7f1be7daa740 1 --2- 172.21.15.42:0/2013742234 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5654d4654390 0x5654d4a25ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:18.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.032+0000 7f1be7daa740 1 --2- 172.21.15.42:0/2013742234 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5654d464c1b0 0x5654d4a259b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:18.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.032+0000 7f1be7daa740 1 -- 172.21.15.42:0/2013742234 >> 172.21.15.42:0/2013742234 conn(0x5654d4669460 msgr2=0x5654d4a1d4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:02:18.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.032+0000 7f1bbaffd640 0 client.25414 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:02:18.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.033+0000 7f1be7daa740 1 -- 172.21.15.42:0/2013742234 shutdown_connections 2024-08-30T09:02:18.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.033+0000 7f1be7daa740 1 -- 172.21.15.42:0/2013742234 wait complete. 2024-08-30T09:02:18.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.033+0000 7f1be7daa740 20 client.25414 trim_cache size 0 max 16384 2024-08-30T09:02:18.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.052+0000 7f1be7daa740 1 Processor -- start 2024-08-30T09:02:18.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.052+0000 7f1be7daa740 1 -- start start 2024-08-30T09:02:18.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.052+0000 7f1be7daa740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5654d464c1b0 0x5654d4a1ba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:18.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.052+0000 7f1be7daa740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5654d4654390 0x5654d4a177d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:18.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.052+0000 7f1be7daa740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5654d4675070 0x5654d4a17d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:18.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.052+0000 7f1be7daa740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f1bdc1b10a0 con 0x5654d464c1b0 2024-08-30T09:02:18.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.052+0000 7f1be7daa740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f1bdc1ae970 con 0x5654d4654390 2024-08-30T09:02:18.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.052+0000 7f1be7daa740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f1bdc1ac7c0 con 0x5654d4675070 2024-08-30T09:02:18.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.052+0000 7f1bda7fc640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5654d464c1b0 0x5654d4a1ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:02:18.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.052+0000 7f1bda7fc640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5654d464c1b0 0x5654d4a1ba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:51564/0 (socket says 172.21.15.42:51564) 2024-08-30T09:02:18.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.052+0000 7f1bda7fc640 1 -- 172.21.15.42:0/2063391380 learned_addr learned my addr 172.21.15.42:0/2063391380 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:02:18.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.052+0000 7f1bdaffd640 1 --2- 172.21.15.42:0/2063391380 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5654d4654390 0x5654d4a177d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:02:18.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.052+0000 7f1bda7fc640 1 --2- 172.21.15.42:0/2063391380 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5654d4675070 0x5654d4a17d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:02:18.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.053+0000 7f1bdaffd640 1 -- 172.21.15.42:0/2063391380 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5654d4675070 msgr2=0x5654d4a17d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:02:18.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.053+0000 7f1bdaffd640 1 --2- 172.21.15.42:0/2063391380 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5654d4675070 0x5654d4a17d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:18.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.053+0000 7f1bdaffd640 1 -- 172.21.15.42:0/2063391380 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5654d464c1b0 msgr2=0x5654d4a1ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:02:18.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.053+0000 7f1bdaffd640 1 --2- 172.21.15.42:0/2063391380 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5654d464c1b0 0x5654d4a1ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:18.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.053+0000 7f1bdaffd640 1 -- 172.21.15.42:0/2063391380 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5654d4a1c600 con 0x5654d4654390 2024-08-30T09:02:18.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.053+0000 7f1bda7fc640 1 --2- 172.21.15.42:0/2063391380 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5654d464c1b0 0x5654d4a1ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-30T09:02:18.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.053+0000 7f1bdaffd640 1 --2- 172.21.15.42:0/2063391380 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5654d4654390 0x5654d4a177d0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f1bcc009cf0 tx=0x7f1bcc016840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ac08cd1fbb852742 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:02:18.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.053+0000 7f1bd17fa640 1 -- 172.21.15.42:0/2063391380 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bcc023040 con 0x5654d4654390 2024-08-30T09:02:18.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.053+0000 7f1bd17fa640 1 -- 172.21.15.42:0/2063391380 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1bcc003970 con 0x5654d4654390 2024-08-30T09:02:18.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.053+0000 7f1bd17fa640 1 -- 172.21.15.42:0/2063391380 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bcc028f90 con 0x5654d4654390 2024-08-30T09:02:18.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.054+0000 7f1be7daa740 1 -- 172.21.15.42:0/2063391380 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5654d4654390 msgr2=0x5654d4a177d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:02:18.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.054+0000 7f1be7daa740 1 --2- 172.21.15.42:0/2063391380 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5654d4654390 0x5654d4a177d0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f1bcc009cf0 tx=0x7f1bcc016840 comp rx=0 tx=0).stop 2024-08-30T09:02:18.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.054+0000 7f1be7daa740 1 -- 172.21.15.42:0/2063391380 shutdown_connections 2024-08-30T09:02:18.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.054+0000 7f1be7daa740 1 --2- 172.21.15.42:0/2063391380 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5654d4675070 0x5654d4a17d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:18.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.054+0000 7f1be7daa740 1 --2- 172.21.15.42:0/2063391380 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5654d4654390 0x5654d4a177d0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:18.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.054+0000 7f1be7daa740 1 --2- 172.21.15.42:0/2063391380 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5654d464c1b0 0x5654d4a1ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:18.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.054+0000 7f1be7daa740 1 -- 172.21.15.42:0/2063391380 >> 172.21.15.42:0/2063391380 conn(0x5654d4669460 msgr2=0x5654d4a1b340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:02:18.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.054+0000 7f1be7daa740 1 -- 172.21.15.42:0/2063391380 shutdown_connections 2024-08-30T09:02:18.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.054+0000 7f1be7daa740 1 -- 172.21.15.42:0/2063391380 wait complete. 2024-08-30T09:02:18.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.055+0000 7f1be7daa740 1 -- start start 2024-08-30T09:02:18.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.055+0000 7f1be7daa740 1 Processor -- start 2024-08-30T09:02:18.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.055+0000 7f1be7daa740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5654d464c1b0 0x5654d4a2e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:18.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.055+0000 7f1be7daa740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5654d4654390 0x5654d4a2a960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:18.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.055+0000 7f1be7daa740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5654d4675070 0x5654d4a2aea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:18.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.055+0000 7f1be7daa740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f1bdc1ace30 con 0x5654d4654390 2024-08-30T09:02:18.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.055+0000 7f1be7daa740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f1bdc1ae2a0 con 0x5654d464c1b0 2024-08-30T09:02:18.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.055+0000 7f1be7daa740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f1bdc1ad500 con 0x5654d4675070 2024-08-30T09:02:18.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.055+0000 7f1be7daa740 10 client.0 _ll_register_callbacks cb 0x7f1bc4010bc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:02:18.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.055+0000 7f1bda7fc640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5654d464c1b0 0x5654d4a2e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:02:18.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.055+0000 7f1bda7fc640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5654d464c1b0 0x5654d4a2e4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:35108/0 (socket says 172.21.15.42:35108) 2024-08-30T09:02:18.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.055+0000 7f1bda7fc640 1 -- 172.21.15.42:0/2203490139 learned_addr learned my addr 172.21.15.42:0/2203490139 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:02:18.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.055+0000 7f1bda7fc640 1 --2- 172.21.15.42:0/2203490139 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5654d4675070 0x5654d4a2aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:02:18.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.055+0000 7f1bdaffd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5654d4654390 0x5654d4a2a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:02:18.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.056+0000 7f1bda7fc640 1 -- 172.21.15.42:0/2203490139 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5654d464c1b0 msgr2=0x5654d4a2e4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:02:18.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.056+0000 7f1bda7fc640 1 --2- 172.21.15.42:0/2203490139 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5654d464c1b0 0x5654d4a2e4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:18.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.056+0000 7f1bda7fc640 1 -- 172.21.15.42:0/2203490139 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5654d4654390 msgr2=0x5654d4a2a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:02:18.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.056+0000 7f1bda7fc640 1 --2- 172.21.15.42:0/2203490139 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5654d4654390 0x5654d4a2a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:18.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.056+0000 7f1bda7fc640 1 -- 172.21.15.42:0/2203490139 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5654d4a2b5e0 con 0x5654d4675070 2024-08-30T09:02:18.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.056+0000 7f1bdaffd640 1 --2- 172.21.15.42:0/2203490139 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5654d4654390 0x5654d4a2a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-30T09:02:18.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.056+0000 7f1bda7fc640 1 --2- 172.21.15.42:0/2203490139 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5654d4675070 0x5654d4a2aea0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f1bc40097c0 tx=0x7f1bc4010c70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=54bbbd8710423b89 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:02:18.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.056+0000 7f1bbaffd640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-08-30T09:02:18.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.056+0000 7f1bbaffd640 1 -- 172.21.15.42:0/2203490139 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bc401a040 con 0x5654d4675070 2024-08-30T09:02:18.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.056+0000 7f1bbaffd640 1 -- 172.21.15.42:0/2203490139 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f1bc4005450 con 0x5654d4675070 2024-08-30T09:02:18.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.056+0000 7f1be7daa740 1 -- 172.21.15.42:0/2203490139 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1bcc003970 con 0x5654d4675070 2024-08-30T09:02:18.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.056+0000 7f1bbaffd640 1 -- 172.21.15.42:0/2203490139 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f1bc400e510 con 0x5654d4675070 2024-08-30T09:02:18.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.056+0000 7f1be7daa740 10 client.25420 fetch_fsmap learned FSMap version 29 2024-08-30T09:02:18.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.056+0000 7f1be7daa740 1 -- 172.21.15.42:0/2203490139 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5654d4aabe70 con 0x5654d4675070 2024-08-30T09:02:18.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.057+0000 7f1bbaffd640 1 -- 172.21.15.42:0/2203490139 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1bcc003970 con 0x5654d4675070 2024-08-30T09:02:18.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.057+0000 7f1bbaffd640 1 -- 172.21.15.42:0/2203490139 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f1bc4031430 con 0x5654d4675070 2024-08-30T09:02:18.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.057+0000 7f1be7daa740 10 client.25420 fetch_fsmap finished waiting for FSMap version 29 2024-08-30T09:02:18.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.057+0000 7f1be7daa740 10 client.25420 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-30T09:02:18.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.057+0000 7f1be7daa740 20 client.25420 populate_metadata read hostname 'smithi042' 2024-08-30T09:02:18.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.057+0000 7f1be7daa740 1 --2- 172.21.15.42:0/2203490139 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x5654d4608da0 0x5654d4aae660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:18.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.057+0000 7f1be7daa740 4 client.25420 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-30T09:02:18.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.057+0000 7f1be7daa740 1 -- 172.21.15.42:0/2203490139 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5654d49935d0 con 0x5654d4608da0 2024-08-30T09:02:18.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.057+0000 7f1bdaffd640 1 --2- 172.21.15.42:0/2203490139 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x5654d4608da0 0x5654d4aae660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:02:18.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.059+0000 7f1bdaffd640 1 --2- 172.21.15.42:0/2203490139 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x5654d4608da0 0x5654d4aae660 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e27d07b3f318ced7 server_cookie=95ad2799df53a597 in_seq=0 out_seq=0 2024-08-30T09:02:18.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.060+0000 7f1bbaffd640 10 client.25420 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:02:18.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.060+0000 7f1bbaffd640 1 -- 172.21.15.42:0/2203490139 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5654d49935d0 con 0x5654d4608da0 2024-08-30T09:02:18.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.060+0000 7f1bbaffd640 10 client.25420 handle_command_reply: tid=0 2024-08-30T09:02:18.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.060+0000 7f1be7daa740 1 client.25420 shutdown 2024-08-30T09:02:18.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.060+0000 7f1be7daa740 1 -- 172.21.15.42:0/2203490139 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5654d4675070 msgr2=0x5654d4a2aea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:02:18.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.060+0000 7f1be7daa740 1 --2- 172.21.15.42:0/2203490139 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5654d4675070 0x5654d4a2aea0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f1bc40097c0 tx=0x7f1bc4010c70 comp rx=0 tx=0).stop 2024-08-30T09:02:18.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.061+0000 7f1be7daa740 1 -- 172.21.15.42:0/2203490139 shutdown_connections 2024-08-30T09:02:18.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.061+0000 7f1be7daa740 1 --2- 172.21.15.42:0/2203490139 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x5654d4608da0 0x5654d4aae660 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:18.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.061+0000 7f1be7daa740 1 --2- 172.21.15.42:0/2203490139 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5654d4675070 0x5654d4a2aea0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:18.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.061+0000 7f1be7daa740 1 --2- 172.21.15.42:0/2203490139 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5654d4654390 0x5654d4a2a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:18.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.061+0000 7f1be7daa740 1 --2- 172.21.15.42:0/2203490139 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5654d464c1b0 0x5654d4a2e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:18.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.061+0000 7f1bbaffd640 0 client.25420 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:02:18.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.061+0000 7f1be7daa740 1 -- 172.21.15.42:0/2203490139 >> 172.21.15.42:0/2203490139 conn(0x5654d4669460 msgr2=0x5654d4a26df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:02:18.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.061+0000 7f1be7daa740 1 -- 172.21.15.42:0/2203490139 shutdown_connections 2024-08-30T09:02:18.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.061+0000 7f1be7daa740 1 -- 172.21.15.42:0/2203490139 wait complete. 2024-08-30T09:02:18.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.061+0000 7f1be7daa740 20 client.25420 trim_cache size 0 max 16384 2024-08-30T09:02:18.067 INFO:teuthology.orchestra.run.smithi042.stdout:[] 2024-08-30T09:02:18.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.061+0000 7f1be1835640 1 -- 172.21.15.42:0/4025380014 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f1bc80b4be0 msgr2=0x7f1bc80b70f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:02:18.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.061+0000 7f1be1835640 1 --2- 172.21.15.42:0/4025380014 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f1bc80b4be0 0x7f1bc80b70f0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f1bd40cbde0 tx=0x7f1bd40c8820 comp rx=0 tx=0).stop 2024-08-30T09:02:18.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.062+0000 7f1be1835640 1 -- 172.21.15.42:0/4025380014 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1bdc108a90 msgr2=0x7f1bdc19baf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:02:18.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.062+0000 7f1be1835640 1 --2- 172.21.15.42:0/4025380014 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1bdc108a90 0x7f1bdc19baf0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f1bd402a930 tx=0x7f1bd4003b90 comp rx=0 tx=0).stop 2024-08-30T09:02:18.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.062+0000 7f1be1835640 1 -- 172.21.15.42:0/4025380014 shutdown_connections 2024-08-30T09:02:18.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.062+0000 7f1be1835640 1 --2- 172.21.15.42:0/4025380014 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f1bc80b4be0 0x7f1bc80b70f0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:18.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.062+0000 7f1be1835640 1 --2- 172.21.15.42:0/4025380014 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1bdc108a90 0x7f1bdc19baf0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:18.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.062+0000 7f1be1835640 1 --2- 172.21.15.42:0/4025380014 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1bdc108140 0x7f1bdc19b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:18.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.062+0000 7f1be1835640 1 --2- 172.21.15.42:0/4025380014 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1bdc102800 0x7f1bdc19b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:18.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.062+0000 7f1be1835640 1 -- 172.21.15.42:0/4025380014 >> 172.21.15.42:0/4025380014 conn(0x7f1bdc0794d0 msgr2=0x7f1bdc105040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:02:18.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.062+0000 7f1be1835640 1 -- 172.21.15.42:0/4025380014 shutdown_connections 2024-08-30T09:02:18.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:18.062+0000 7f1be1835640 1 -- 172.21.15.42:0/4025380014 wait complete. 2024-08-30T09:02:18.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:02:18 smithi042 ceph-mon[18652]: pgmap v419: 129 pgs: 129 active+clean; 1.0 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 23 MiB/s wr, 1.14k op/s 2024-08-30T09:02:19.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:02:18 smithi167 ceph-mon[22063]: pgmap v419: 129 pgs: 129 active+clean; 1.0 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 23 MiB/s wr, 1.14k op/s 2024-08-30T09:02:19.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:02:18 smithi190 ceph-mon[22001]: pgmap v419: 129 pgs: 129 active+clean; 1.0 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 23 MiB/s wr, 1.14k op/s 2024-08-30T09:02:19.264 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.262+0000 7f7d35e4d640 1 -- 172.21.15.42:0/160741767 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7d30103d90 msgr2=0x7f7d3010a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:02:19.264 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.262+0000 7f7d35e4d640 1 --2- 172.21.15.42:0/160741767 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7d30103d90 0x7f7d3010a730 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f7d28009ca0 tx=0x7f7d2802a8c0 comp rx=0 tx=0).stop 2024-08-30T09:02:19.264 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.262+0000 7f7d35e4d640 1 -- 172.21.15.42:0/160741767 shutdown_connections 2024-08-30T09:02:19.265 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.262+0000 7f7d35e4d640 1 --2- 172.21.15.42:0/160741767 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7d301033f0 0x7f7d30103800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:19.265 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.262+0000 7f7d35e4d640 1 --2- 172.21.15.42:0/160741767 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7d30103d90 0x7f7d3010a730 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:19.265 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.262+0000 7f7d35e4d640 1 --2- 172.21.15.42:0/160741767 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7d30102370 0x7f7d301027e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:19.265 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.262+0000 7f7d35e4d640 1 -- 172.21.15.42:0/160741767 >> 172.21.15.42:0/160741767 conn(0x7f7d300fdcd0 msgr2=0x7f7d30100160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:02:19.265 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.262+0000 7f7d35e4d640 1 -- 172.21.15.42:0/160741767 shutdown_connections 2024-08-30T09:02:19.265 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.262+0000 7f7d35e4d640 1 -- 172.21.15.42:0/160741767 wait complete. 2024-08-30T09:02:19.265 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.263+0000 7f7d35e4d640 1 Processor -- start 2024-08-30T09:02:19.265 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.263+0000 7f7d35e4d640 1 -- start start 2024-08-30T09:02:19.267 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.263+0000 7f7d35e4d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7d30102370 0x7f7d301a0290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:19.267 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.263+0000 7f7d35e4d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7d301033f0 0x7f7d301a0900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:19.267 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.263+0000 7f7d35e4d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7d30103d90 0x7f7d301a14b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:19.267 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.263+0000 7f7d35e4d640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f7d3010cf30 con 0x7f7d301033f0 2024-08-30T09:02:19.267 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.263+0000 7f7d35e4d640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f7d3010cdb0 con 0x7f7d30103d90 2024-08-30T09:02:19.267 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.263+0000 7f7d35e4d640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f7d3010d0b0 con 0x7f7d30102370 2024-08-30T09:02:19.267 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.263+0000 7f7d2f7fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7d30102370 0x7f7d301a0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:02:19.267 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.263+0000 7f7d26ffd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7d301033f0 0x7f7d301a0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:02:19.267 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.263+0000 7f7d2ffff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7d30103d90 0x7f7d301a14b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:02:19.267 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.263+0000 7f7d2f7fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7d30102370 0x7f7d301a0290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:43824/0 (socket says 172.21.15.42:43824) 2024-08-30T09:02:19.268 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.263+0000 7f7d2f7fe640 1 -- 172.21.15.42:0/2288700994 learned_addr learned my addr 172.21.15.42:0/2288700994 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:02:19.268 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.264+0000 7f7d2f7fe640 1 -- 172.21.15.42:0/2288700994 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7d30103d90 msgr2=0x7f7d301a14b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:02:19.268 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.264+0000 7f7d2f7fe640 1 --2- 172.21.15.42:0/2288700994 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7d30103d90 0x7f7d301a14b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:19.268 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.264+0000 7f7d2f7fe640 1 -- 172.21.15.42:0/2288700994 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7d301033f0 msgr2=0x7f7d301a0900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:02:19.268 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.264+0000 7f7d2f7fe640 1 --2- 172.21.15.42:0/2288700994 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7d301033f0 0x7f7d301a0900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:19.268 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.264+0000 7f7d2f7fe640 1 -- 172.21.15.42:0/2288700994 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d301a1cd0 con 0x7f7d30102370 2024-08-30T09:02:19.268 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.264+0000 7f7d2f7fe640 1 --2- 172.21.15.42:0/2288700994 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7d30102370 0x7f7d301a0290 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f7d2000e9f0 tx=0x7f7d2000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:02:19.268 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.264+0000 7f7d2d7fa640 1 -- 172.21.15.42:0/2288700994 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d2000cd80 con 0x7f7d30102370 2024-08-30T09:02:19.268 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.264+0000 7f7d35e4d640 1 -- 172.21.15.42:0/2288700994 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d301a1fc0 con 0x7f7d30102370 2024-08-30T09:02:19.268 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.264+0000 7f7d2d7fa640 1 -- 172.21.15.42:0/2288700994 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d200108e0 con 0x7f7d30102370 2024-08-30T09:02:19.268 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.264+0000 7f7d2d7fa640 1 -- 172.21.15.42:0/2288700994 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d20018980 con 0x7f7d30102370 2024-08-30T09:02:19.268 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.264+0000 7f7d35e4d640 1 -- 172.21.15.42:0/2288700994 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d301a86b0 con 0x7f7d30102370 2024-08-30T09:02:19.270 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.267+0000 7f7d3cdbe740 1 Processor -- start 2024-08-30T09:02:19.271 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.267+0000 7f7d2d7fa640 1 -- 172.21.15.42:0/2288700994 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7f7d20018b20 con 0x7f7d30102370 2024-08-30T09:02:19.271 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.267+0000 7f7d3cdbe740 1 -- start start 2024-08-30T09:02:19.271 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.268+0000 7f7d3cdbe740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560bc2d591b0 0x560bc30fef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:19.271 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.268+0000 7f7d3cdbe740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560bc2d302f0 0x560bc30ff5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:19.271 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.268+0000 7f7d3cdbe740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560bc2d384d0 0x560bc3100120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:19.271 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.268+0000 7f7d3cdbe740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x560bc3072000 con 0x560bc2d384d0 2024-08-30T09:02:19.271 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.268+0000 7f7d3cdbe740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x560bc3061590 con 0x560bc2d302f0 2024-08-30T09:02:19.271 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.268+0000 7f7d3cdbe740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x560bc3065540 con 0x560bc2d591b0 2024-08-30T09:02:19.271 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.268+0000 7f7d2f7fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560bc2d302f0 0x560bc30ff5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:02:19.271 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.268+0000 7f7d26ffd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560bc2d384d0 0x560bc3100120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:02:19.271 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.268+0000 7f7d2ffff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560bc2d591b0 0x560bc30fef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:02:19.271 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.268+0000 7f7d26ffd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560bc2d384d0 0x560bc3100120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:51590/0 (socket says 172.21.15.42:51590) 2024-08-30T09:02:19.272 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.268+0000 7f7d26ffd640 1 -- 172.21.15.42:0/4148968080 learned_addr learned my addr 172.21.15.42:0/4148968080 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:02:19.272 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.268+0000 7f7d2ffff640 1 -- 172.21.15.42:0/4148968080 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560bc2d302f0 msgr2=0x560bc30ff5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:02:19.272 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.268+0000 7f7d2ffff640 1 --2- 172.21.15.42:0/4148968080 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560bc2d302f0 0x560bc30ff5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:19.272 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.268+0000 7f7d2ffff640 1 -- 172.21.15.42:0/4148968080 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560bc2d384d0 msgr2=0x560bc3100120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:02:19.272 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.268+0000 7f7d2ffff640 1 --2- 172.21.15.42:0/4148968080 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560bc2d384d0 0x560bc3100120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:19.272 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.268+0000 7f7d2ffff640 1 -- 172.21.15.42:0/4148968080 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560bc2dc4790 con 0x560bc2d591b0 2024-08-30T09:02:19.272 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.268+0000 7f7d2d7fa640 1 --2- 172.21.15.42:0/2288700994 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f7d1c0b4b30 0x7f7d1c0b7040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:19.272 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.268+0000 7f7d2f7fe640 1 --2- 172.21.15.42:0/2288700994 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f7d1c0b4b30 0x7f7d1c0b7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:02:19.272 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.268+0000 7f7d2ffff640 1 --2- 172.21.15.42:0/4148968080 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560bc2d591b0 0x560bc30fef60 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f7d28006010 tx=0x7f7d2802f3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=23d97f5fa90d582d server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:02:19.272 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.268+0000 7f7d2d7fa640 1 -- 172.21.15.42:0/2288700994 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f7d200e5830 con 0x7f7d30102370 2024-08-30T09:02:19.272 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.269+0000 7f7d25ffb640 1 -- 172.21.15.42:0/4148968080 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d28031650 con 0x560bc2d591b0 2024-08-30T09:02:19.272 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.269+0000 7f7d25ffb640 1 -- 172.21.15.42:0/4148968080 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d28036030 con 0x560bc2d591b0 2024-08-30T09:02:19.272 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.269+0000 7f7d25ffb640 1 -- 172.21.15.42:0/4148968080 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d28004650 con 0x560bc2d591b0 2024-08-30T09:02:19.274 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.270+0000 7f7d3cdbe740 1 -- 172.21.15.42:0/4148968080 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560bc2d591b0 msgr2=0x560bc30fef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:02:19.274 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.270+0000 7f7d2f7fe640 1 --2- 172.21.15.42:0/2288700994 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f7d1c0b4b30 0x7f7d1c0b7040 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f7d200e2fa0 tx=0x7f7d20000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:02:19.274 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.270+0000 7f7d3cdbe740 1 --2- 172.21.15.42:0/4148968080 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560bc2d591b0 0x560bc30fef60 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f7d28006010 tx=0x7f7d2802f3a0 comp rx=0 tx=0).stop 2024-08-30T09:02:19.274 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.270+0000 7f7d3cdbe740 1 -- 172.21.15.42:0/4148968080 shutdown_connections 2024-08-30T09:02:19.274 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.270+0000 7f7d3cdbe740 1 --2- 172.21.15.42:0/4148968080 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560bc2d384d0 0x560bc3100120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:19.274 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.270+0000 7f7d3cdbe740 1 --2- 172.21.15.42:0/4148968080 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560bc2d302f0 0x560bc30ff5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:19.274 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.270+0000 7f7d3cdbe740 1 --2- 172.21.15.42:0/4148968080 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560bc2d591b0 0x560bc30fef60 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:19.274 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.270+0000 7f7d3cdbe740 1 -- 172.21.15.42:0/4148968080 >> 172.21.15.42:0/4148968080 conn(0x560bc2d4d300 msgr2=0x560bc30fa9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:02:19.275 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.270+0000 7f7d3cdbe740 1 -- 172.21.15.42:0/4148968080 shutdown_connections 2024-08-30T09:02:19.275 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.270+0000 7f7d3cdbe740 1 -- 172.21.15.42:0/4148968080 wait complete. 2024-08-30T09:02:19.275 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.271+0000 7f7d3cdbe740 1 -- start start 2024-08-30T09:02:19.275 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.271+0000 7f7d3cdbe740 1 Processor -- start 2024-08-30T09:02:19.275 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.271+0000 7f7d3cdbe740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560bc2d302f0 0x560bc3108860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:19.275 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.271+0000 7f7d3cdbe740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560bc2d384d0 0x560bc3108f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:19.275 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.271+0000 7f7d3cdbe740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560bc2d591b0 0x560bc3109a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:19.275 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.271+0000 7f7d3cdbe740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x560bc2f9aab0 con 0x560bc2d384d0 2024-08-30T09:02:19.275 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.271+0000 7f7d3cdbe740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x560bc2fe8680 con 0x560bc2d302f0 2024-08-30T09:02:19.276 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.272+0000 7f7d2ffff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560bc2d302f0 0x560bc3108860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:02:19.276 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.272+0000 7f7d2ffff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560bc2d302f0 0x560bc3108860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:35158/0 (socket says 172.21.15.42:35158) 2024-08-30T09:02:19.276 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.272+0000 7f7d2ffff640 1 -- 172.21.15.42:0/1810086288 learned_addr learned my addr 172.21.15.42:0/1810086288 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:02:19.276 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.273+0000 7f7d26ffd640 1 --2- 172.21.15.42:0/1810086288 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560bc2d384d0 0x560bc3108f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:02:19.276 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.273+0000 7f7d3cdbe740 1 -- 172.21.15.42:0/1810086288 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x560bc3100a60 con 0x560bc2d591b0 2024-08-30T09:02:19.276 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.273+0000 7f7d2ffff640 1 --2- 172.21.15.42:0/1810086288 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560bc2d591b0 0x560bc3109a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:02:19.276 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.273+0000 7f7d3cdbe740 10 client.0 _ll_register_callbacks cb 0x560bc2d70a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:02:19.276 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.273+0000 7f7d2ffff640 1 -- 172.21.15.42:0/1810086288 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560bc2d591b0 msgr2=0x560bc3109a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:02:19.276 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.273+0000 7f7d2ffff640 1 --2- 172.21.15.42:0/1810086288 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560bc2d591b0 0x560bc3109a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:19.276 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.273+0000 7f7d2ffff640 1 -- 172.21.15.42:0/1810086288 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560bc2d384d0 msgr2=0x560bc3108f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:02:19.276 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.273+0000 7f7d2ffff640 1 --2- 172.21.15.42:0/1810086288 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560bc2d384d0 0x560bc3108f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:19.276 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.273+0000 7f7d2ffff640 1 -- 172.21.15.42:0/1810086288 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560bc2e3d620 con 0x560bc2d302f0 2024-08-30T09:02:19.276 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.274+0000 7f7d2ffff640 1 --2- 172.21.15.42:0/1810086288 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560bc2d302f0 0x560bc3108860 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f7d28009a60 tx=0x7f7d28005bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fc42cb73196ebec8 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:02:19.278 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.275+0000 7f7d0effd640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-30T09:02:19.278 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.275+0000 7f7d0effd640 1 -- 172.21.15.42:0/1810086288 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d2806a060 con 0x560bc2d302f0 2024-08-30T09:02:19.278 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.275+0000 7f7d3cdbe740 1 -- 172.21.15.42:0/1810086288 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7d28036030 con 0x560bc2d302f0 2024-08-30T09:02:19.278 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.275+0000 7f7d0effd640 1 -- 172.21.15.42:0/1810086288 <== mon.2 v2:172.21.15.167:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f7d28003d80 con 0x560bc2d302f0 2024-08-30T09:02:19.278 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.275+0000 7f7d0effd640 1 -- 172.21.15.42:0/1810086288 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f7d28003f80 con 0x560bc2d302f0 2024-08-30T09:02:19.279 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.275+0000 7f7d3cdbe740 10 client.25430 fetch_fsmap learned FSMap version 29 2024-08-30T09:02:19.279 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.275+0000 7f7d3cdbe740 1 -- 172.21.15.42:0/1810086288 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560bc31129a0 con 0x560bc2d302f0 2024-08-30T09:02:19.279 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.276+0000 7f7d0effd640 1 -- 172.21.15.42:0/1810086288 <== mon.2 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d28036030 con 0x560bc2d302f0 2024-08-30T09:02:19.279 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.276+0000 7f7d0effd640 1 -- 172.21.15.42:0/1810086288 <== mon.2 v2:172.21.15.167:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f7d28043070 con 0x560bc2d302f0 2024-08-30T09:02:19.279 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.276+0000 7f7d3cdbe740 10 client.25430 fetch_fsmap finished waiting for FSMap version 29 2024-08-30T09:02:19.279 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.276+0000 7f7d3cdbe740 10 client.25430 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-30T09:02:19.279 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.276+0000 7f7d3cdbe740 20 client.25430 populate_metadata read hostname 'smithi042' 2024-08-30T09:02:19.279 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.276+0000 7f7d3cdbe740 1 --2- 172.21.15.42:0/1810086288 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x560bc2d17d80 0x560bc3116170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:19.279 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.276+0000 7f7d3cdbe740 4 client.25430 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-30T09:02:19.279 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.276+0000 7f7d3cdbe740 1 -- 172.21.15.42:0/1810086288 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560bc3073f50 con 0x560bc2d17d80 2024-08-30T09:02:19.279 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.277+0000 7f7d26ffd640 1 --2- 172.21.15.42:0/1810086288 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x560bc2d17d80 0x560bc3116170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:02:19.281 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.277+0000 7f7d0effd640 10 client.25430 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:02:19.281 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.277+0000 7f7d26ffd640 1 --2- 172.21.15.42:0/1810086288 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x560bc2d17d80 0x560bc3116170 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f032802146339b0 server_cookie=c9138bdf203cea34 in_seq=0 out_seq=0 2024-08-30T09:02:19.281 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.278+0000 7f7d0effd640 1 -- 172.21.15.42:0/1810086288 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x560bc3073f50 con 0x560bc2d17d80 2024-08-30T09:02:19.281 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.278+0000 7f7d0effd640 10 client.25430 handle_command_reply: tid=0 2024-08-30T09:02:19.281 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.278+0000 7f7d3cdbe740 1 client.25430 shutdown 2024-08-30T09:02:19.281 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.278+0000 7f7d3cdbe740 1 -- 172.21.15.42:0/1810086288 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560bc2d302f0 msgr2=0x560bc3108860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:02:19.282 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.278+0000 7f7d3cdbe740 1 --2- 172.21.15.42:0/1810086288 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560bc2d302f0 0x560bc3108860 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f7d28009a60 tx=0x7f7d28005bd0 comp rx=0 tx=0).stop 2024-08-30T09:02:19.282 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.278+0000 7f7d3cdbe740 1 -- 172.21.15.42:0/1810086288 shutdown_connections 2024-08-30T09:02:19.282 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.278+0000 7f7d3cdbe740 1 --2- 172.21.15.42:0/1810086288 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x560bc2d17d80 0x560bc3116170 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:19.282 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.278+0000 7f7d3cdbe740 1 --2- 172.21.15.42:0/1810086288 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560bc2d591b0 0x560bc3109a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:19.282 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.278+0000 7f7d3cdbe740 1 --2- 172.21.15.42:0/1810086288 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560bc2d384d0 0x560bc3108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:19.282 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.279+0000 7f7d3cdbe740 1 --2- 172.21.15.42:0/1810086288 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560bc2d302f0 0x560bc3108860 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:19.282 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.279+0000 7f7d3cdbe740 1 -- 172.21.15.42:0/1810086288 >> 172.21.15.42:0/1810086288 conn(0x560bc2d4d300 msgr2=0x560bc30f8c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:02:19.282 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.279+0000 7f7d0effd640 0 client.25430 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:02:19.282 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.279+0000 7f7d3cdbe740 1 -- 172.21.15.42:0/1810086288 shutdown_connections 2024-08-30T09:02:19.282 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.279+0000 7f7d3cdbe740 1 -- 172.21.15.42:0/1810086288 wait complete. 2024-08-30T09:02:19.283 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.279+0000 7f7d3cdbe740 20 client.25430 trim_cache size 0 max 16384 2024-08-30T09:02:19.301 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.299+0000 7f7d3cdbe740 1 Processor -- start 2024-08-30T09:02:19.303 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.299+0000 7f7d3cdbe740 1 -- start start 2024-08-30T09:02:19.303 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.300+0000 7f7d3cdbe740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560bc2d302f0 0x560bc30ffda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:19.303 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.300+0000 7f7d3cdbe740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560bc2d384d0 0x560bc30fec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:19.303 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.300+0000 7f7d3cdbe740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560bc2d591b0 0x560bc30ff160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:19.303 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.300+0000 7f7d3cdbe740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f7d301aca70 con 0x560bc2d302f0 2024-08-30T09:02:19.303 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.300+0000 7f7d3cdbe740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f7d301aa340 con 0x560bc2d591b0 2024-08-30T09:02:19.303 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.300+0000 7f7d3cdbe740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f7d3011ab30 con 0x560bc2d384d0 2024-08-30T09:02:19.303 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.300+0000 7f7d2ffff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560bc2d302f0 0x560bc30ffda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:02:19.303 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.300+0000 7f7d2ffff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560bc2d302f0 0x560bc30ffda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:51608/0 (socket says 172.21.15.42:51608) 2024-08-30T09:02:19.303 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.300+0000 7f7d2ffff640 1 -- 172.21.15.42:0/2195120243 learned_addr learned my addr 172.21.15.42:0/2195120243 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:02:19.303 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.300+0000 7f7d26ffd640 1 --2- 172.21.15.42:0/2195120243 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560bc2d384d0 0x560bc30fec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:02:19.304 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.300+0000 7f7d2ffff640 1 --2- 172.21.15.42:0/2195120243 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560bc2d591b0 0x560bc30ff160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:02:19.304 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.301+0000 7f7d2ffff640 1 -- 172.21.15.42:0/2195120243 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560bc2d384d0 msgr2=0x560bc30fec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:02:19.304 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.301+0000 7f7d2ffff640 1 --2- 172.21.15.42:0/2195120243 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560bc2d384d0 0x560bc30fec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:19.304 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.301+0000 7f7d2ffff640 1 -- 172.21.15.42:0/2195120243 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560bc2d591b0 msgr2=0x560bc30ff160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:02:19.304 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.301+0000 7f7d2ffff640 1 --2- 172.21.15.42:0/2195120243 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560bc2d591b0 0x560bc30ff160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:19.304 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.301+0000 7f7d2ffff640 1 -- 172.21.15.42:0/2195120243 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560bc2d17fb0 con 0x560bc2d302f0 2024-08-30T09:02:19.304 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.301+0000 7f7d26ffd640 1 --2- 172.21.15.42:0/2195120243 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560bc2d384d0 0x560bc30fec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-30T09:02:19.304 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.301+0000 7f7d2ffff640 1 --2- 172.21.15.42:0/2195120243 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560bc2d302f0 0x560bc30ffda0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f7d28031b50 tx=0x7f7d2802b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37ea66f49c719cd4 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:02:19.304 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.301+0000 7f7d257fa640 1 -- 172.21.15.42:0/2195120243 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d280305c0 con 0x560bc2d302f0 2024-08-30T09:02:19.304 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.301+0000 7f7d257fa640 1 -- 172.21.15.42:0/2195120243 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d28036070 con 0x560bc2d302f0 2024-08-30T09:02:19.304 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.301+0000 7f7d257fa640 1 -- 172.21.15.42:0/2195120243 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d2802f610 con 0x560bc2d302f0 2024-08-30T09:02:19.308 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.302+0000 7f7d3cdbe740 1 -- 172.21.15.42:0/2195120243 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560bc2d302f0 msgr2=0x560bc30ffda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:02:19.308 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.302+0000 7f7d3cdbe740 1 --2- 172.21.15.42:0/2195120243 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560bc2d302f0 0x560bc30ffda0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f7d28031b50 tx=0x7f7d2802b040 comp rx=0 tx=0).stop 2024-08-30T09:02:19.308 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.302+0000 7f7d3cdbe740 1 -- 172.21.15.42:0/2195120243 shutdown_connections 2024-08-30T09:02:19.308 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.302+0000 7f7d3cdbe740 1 --2- 172.21.15.42:0/2195120243 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560bc2d384d0 0x560bc30fec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:19.308 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.302+0000 7f7d3cdbe740 1 --2- 172.21.15.42:0/2195120243 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560bc2d591b0 0x560bc30ff160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:19.308 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.302+0000 7f7d3cdbe740 1 --2- 172.21.15.42:0/2195120243 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560bc2d302f0 0x560bc30ffda0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:19.308 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.302+0000 7f7d3cdbe740 1 -- 172.21.15.42:0/2195120243 >> 172.21.15.42:0/2195120243 conn(0x560bc2d4d300 msgr2=0x560bc3106df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:02:19.309 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.302+0000 7f7d3cdbe740 1 -- 172.21.15.42:0/2195120243 shutdown_connections 2024-08-30T09:02:19.309 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.302+0000 7f7d3cdbe740 1 -- 172.21.15.42:0/2195120243 wait complete. 2024-08-30T09:02:19.309 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.303+0000 7f7d3cdbe740 1 -- start start 2024-08-30T09:02:19.309 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.303+0000 7f7d3cdbe740 1 Processor -- start 2024-08-30T09:02:19.309 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.303+0000 7f7d3cdbe740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560bc2d302f0 0x560bc3110610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:19.309 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.303+0000 7f7d3cdbe740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560bc2d384d0 0x560bc310f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:19.309 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.303+0000 7f7d3cdbe740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560bc2d591b0 0x560bc310f810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:19.309 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.303+0000 7f7d3cdbe740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f7d301a8880 con 0x560bc2d591b0 2024-08-30T09:02:19.309 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.303+0000 7f7d3cdbe740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f7d301a9c70 con 0x560bc2d302f0 2024-08-30T09:02:19.309 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.303+0000 7f7d3cdbe740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f7d301a8ed0 con 0x560bc2d384d0 2024-08-30T09:02:19.309 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.303+0000 7f7d3cdbe740 10 client.0 _ll_register_callbacks cb 0x7f7d180097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:02:19.309 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.303+0000 7f7d2ffff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560bc2d591b0 0x560bc310f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:02:19.309 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.303+0000 7f7d26ffd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560bc2d384d0 0x560bc310f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:02:19.309 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.303+0000 7f7d2ffff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560bc2d591b0 0x560bc310f810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:51620/0 (socket says 172.21.15.42:51620) 2024-08-30T09:02:19.310 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.303+0000 7f7d2ffff640 1 -- 172.21.15.42:0/2341826467 learned_addr learned my addr 172.21.15.42:0/2341826467 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:02:19.310 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.303+0000 7f7d2ffff640 1 --2- 172.21.15.42:0/2341826467 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560bc2d302f0 0x560bc3110610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:02:19.310 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.304+0000 7f7d26ffd640 1 -- 172.21.15.42:0/2341826467 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560bc2d302f0 msgr2=0x560bc3110610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:02:19.310 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.304+0000 7f7d26ffd640 1 --2- 172.21.15.42:0/2341826467 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560bc2d302f0 0x560bc3110610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:19.310 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.304+0000 7f7d26ffd640 1 -- 172.21.15.42:0/2341826467 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560bc2d591b0 msgr2=0x560bc310f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:02:19.310 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.304+0000 7f7d26ffd640 1 --2- 172.21.15.42:0/2341826467 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560bc2d591b0 0x560bc310f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:19.310 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.304+0000 7f7d26ffd640 1 -- 172.21.15.42:0/2341826467 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560bc310ff50 con 0x560bc2d384d0 2024-08-30T09:02:19.310 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.304+0000 7f7d26ffd640 1 --2- 172.21.15.42:0/2341826467 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560bc2d384d0 0x560bc310f2d0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f7d18000c00 tx=0x7f7d18005df0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=416101bf89f41f0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:02:19.310 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.304+0000 7f7d0effd640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-08-30T09:02:19.310 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.304+0000 7f7d0effd640 1 -- 172.21.15.42:0/2341826467 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d18003cf0 con 0x560bc2d384d0 2024-08-30T09:02:19.311 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.304+0000 7f7d0effd640 1 -- 172.21.15.42:0/2341826467 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f7d18005020 con 0x560bc2d384d0 2024-08-30T09:02:19.311 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.304+0000 7f7d3cdbe740 1 -- 172.21.15.42:0/2341826467 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7d28036070 con 0x560bc2d384d0 2024-08-30T09:02:19.311 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.304+0000 7f7d0effd640 1 -- 172.21.15.42:0/2341826467 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f7d18031050 con 0x560bc2d384d0 2024-08-30T09:02:19.311 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.304+0000 7f7d3cdbe740 10 client.25438 fetch_fsmap learned FSMap version 29 2024-08-30T09:02:19.311 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.304+0000 7f7d3cdbe740 1 -- 172.21.15.42:0/2341826467 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560bc3180b70 con 0x560bc2d384d0 2024-08-30T09:02:19.311 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.304+0000 7f7d0effd640 1 -- 172.21.15.42:0/2341826467 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d28036070 con 0x560bc2d384d0 2024-08-30T09:02:19.311 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.305+0000 7f7d0effd640 1 -- 172.21.15.42:0/2341826467 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f7d28031690 con 0x560bc2d384d0 2024-08-30T09:02:19.311 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.305+0000 7f7d3cdbe740 10 client.25438 fetch_fsmap finished waiting for FSMap version 29 2024-08-30T09:02:19.311 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.305+0000 7f7d3cdbe740 10 client.25438 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-30T09:02:19.311 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.305+0000 7f7d3cdbe740 20 client.25438 populate_metadata read hostname 'smithi042' 2024-08-30T09:02:19.311 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.305+0000 7f7d3cdbe740 1 --2- 172.21.15.42:0/2341826467 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x560bc2ced1c0 0x560bc3183640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:19.311 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.305+0000 7f7d3cdbe740 4 client.25438 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-30T09:02:19.311 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.305+0000 7f7d3cdbe740 1 -- 172.21.15.42:0/2341826467 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560bc306f9f0 con 0x560bc2ced1c0 2024-08-30T09:02:19.311 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.305+0000 7f7d2ffff640 1 --2- 172.21.15.42:0/2341826467 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x560bc2ced1c0 0x560bc3183640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:02:19.312 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.306+0000 7f7d2ffff640 1 --2- 172.21.15.42:0/2341826467 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x560bc2ced1c0 0x560bc3183640 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c13d6ee24b838d6 server_cookie=2b79f6a7fa79f475 in_seq=0 out_seq=0 2024-08-30T09:02:19.312 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.306+0000 7f7d0effd640 10 client.25438 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:02:19.322 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.320+0000 7f7d0effd640 1 -- 172.21.15.42:0/2341826467 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560bc306f9f0 con 0x560bc2ced1c0 2024-08-30T09:02:19.322 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.320+0000 7f7d0effd640 10 client.25438 handle_command_reply: tid=0 2024-08-30T09:02:19.323 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.320+0000 7f7d3cdbe740 1 client.25438 shutdown 2024-08-30T09:02:19.323 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.322+0000 7f7d3cdbe740 1 -- 172.21.15.42:0/2341826467 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560bc2d384d0 msgr2=0x560bc310f2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:02:19.324 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.322+0000 7f7d3cdbe740 1 --2- 172.21.15.42:0/2341826467 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560bc2d384d0 0x560bc310f2d0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f7d18000c00 tx=0x7f7d18005df0 comp rx=0 tx=0).stop 2024-08-30T09:02:19.324 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.322+0000 7f7d3cdbe740 1 -- 172.21.15.42:0/2341826467 shutdown_connections 2024-08-30T09:02:19.324 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.322+0000 7f7d3cdbe740 1 --2- 172.21.15.42:0/2341826467 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x560bc2ced1c0 0x560bc3183640 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:19.324 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.322+0000 7f7d0effd640 0 client.25438 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:02:19.324 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.322+0000 7f7d3cdbe740 1 --2- 172.21.15.42:0/2341826467 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560bc2d384d0 0x560bc310f2d0 secure :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f7d18000c00 tx=0x7f7d18005df0 comp rx=0 tx=0).stop 2024-08-30T09:02:19.324 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.322+0000 7f7d3cdbe740 1 --2- 172.21.15.42:0/2341826467 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560bc2d591b0 0x560bc310f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:19.325 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.322+0000 7f7d3cdbe740 1 --2- 172.21.15.42:0/2341826467 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560bc2d302f0 0x560bc3110610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:19.325 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.322+0000 7f7d3cdbe740 1 -- 172.21.15.42:0/2341826467 >> 172.21.15.42:0/2341826467 conn(0x560bc2d4d300 msgr2=0x560bc30feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:02:19.325 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.323+0000 7f7d3cdbe740 1 -- 172.21.15.42:0/2341826467 shutdown_connections 2024-08-30T09:02:19.325 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.323+0000 7f7d3cdbe740 1 -- 172.21.15.42:0/2341826467 wait complete. 2024-08-30T09:02:19.326 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.323+0000 7f7d3cdbe740 20 client.25438 trim_cache size 0 max 16384 2024-08-30T09:02:19.326 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-08-30T09:02:19.326 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2024-08-30T09:02:19.326 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-08-30T09:02:19.326 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-08-30T09:02:19.327 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.324+0000 7f7d35e4d640 1 -- 172.21.15.42:0/2288700994 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f7d1c0b4b30 msgr2=0x7f7d1c0b7040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:02:19.327 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.324+0000 7f7d35e4d640 1 --2- 172.21.15.42:0/2288700994 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f7d1c0b4b30 0x7f7d1c0b7040 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f7d200e2fa0 tx=0x7f7d20000bc0 comp rx=0 tx=0).stop 2024-08-30T09:02:19.327 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.324+0000 7f7d35e4d640 1 -- 172.21.15.42:0/2288700994 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7d30102370 msgr2=0x7f7d301a0290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:02:19.327 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.324+0000 7f7d35e4d640 1 --2- 172.21.15.42:0/2288700994 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7d30102370 0x7f7d301a0290 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f7d2000e9f0 tx=0x7f7d2000eeb0 comp rx=0 tx=0).stop 2024-08-30T09:02:19.327 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.324+0000 7f7d35e4d640 1 -- 172.21.15.42:0/2288700994 shutdown_connections 2024-08-30T09:02:19.327 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.324+0000 7f7d35e4d640 1 --2- 172.21.15.42:0/2288700994 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f7d1c0b4b30 0x7f7d1c0b7040 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:19.327 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.324+0000 7f7d35e4d640 1 --2- 172.21.15.42:0/2288700994 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7d30103d90 0x7f7d301a14b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:19.327 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.324+0000 7f7d35e4d640 1 --2- 172.21.15.42:0/2288700994 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7d301033f0 0x7f7d301a0900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:19.327 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.324+0000 7f7d35e4d640 1 --2- 172.21.15.42:0/2288700994 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7d30102370 0x7f7d301a0290 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:19.328 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.324+0000 7f7d35e4d640 1 -- 172.21.15.42:0/2288700994 >> 172.21.15.42:0/2288700994 conn(0x7f7d300fdcd0 msgr2=0x7f7d3010f8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:02:19.328 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.324+0000 7f7d35e4d640 1 -- 172.21.15.42:0/2288700994 shutdown_connections 2024-08-30T09:02:19.328 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:19.324+0000 7f7d35e4d640 1 -- 172.21.15.42:0/2288700994 wait complete. 2024-08-30T09:02:19.826 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-08-30T09:02:19.826 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-08-30T09:02:20.201 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T09:02:20.953 INFO:teuthology.orchestra.run.smithi042.stderr:progress: 95.0 s, 1567.2 tps, lat 63.077 ms stddev 70.302 2024-08-30T09:02:21.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:02:20 smithi167 ceph-mon[22063]: pgmap v420: 129 pgs: 129 active+clean; 1.0 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 16 MiB/s wr, 910 op/s 2024-08-30T09:02:21.132 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:02:20 smithi042 ceph-mon[18652]: pgmap v420: 129 pgs: 129 active+clean; 1.0 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 16 MiB/s wr, 910 op/s 2024-08-30T09:02:21.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:02:20 smithi190 ceph-mon[22001]: pgmap v420: 129 pgs: 129 active+clean; 1.0 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 16 MiB/s wr, 910 op/s 2024-08-30T09:02:22.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.651+0000 7f5162c83640 1 -- 172.21.15.42:0/643740213 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f515c108a90 msgr2=0x7f515c10f2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:02:22.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.651+0000 7f5162c83640 1 --2- 172.21.15.42:0/643740213 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f515c108a90 0x7f515c10f2c0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f5154009ca0 tx=0x7f515402a8c0 comp rx=0 tx=0).stop 2024-08-30T09:02:22.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.651+0000 7f5162c83640 1 -- 172.21.15.42:0/643740213 shutdown_connections 2024-08-30T09:02:22.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.651+0000 7f5162c83640 1 --2- 172.21.15.42:0/643740213 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f515c108a90 0x7f515c10f2c0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:22.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.651+0000 7f5162c83640 1 --2- 172.21.15.42:0/643740213 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f515c108140 0x7f515c108550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:22.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.651+0000 7f5162c83640 1 --2- 172.21.15.42:0/643740213 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f515c102800 0x7f515c102c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:22.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.651+0000 7f5162c83640 1 -- 172.21.15.42:0/643740213 >> 172.21.15.42:0/643740213 conn(0x7f515c0794d0 msgr2=0x7f515c0798e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:02:22.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.651+0000 7f5162c83640 1 -- 172.21.15.42:0/643740213 shutdown_connections 2024-08-30T09:02:22.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.651+0000 7f5162c83640 1 -- 172.21.15.42:0/643740213 wait complete. 2024-08-30T09:02:22.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.652+0000 7f5162c83640 1 Processor -- start 2024-08-30T09:02:22.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.652+0000 7f5162c83640 1 -- start start 2024-08-30T09:02:22.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.652+0000 7f5162c83640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f515c102800 0x7f515c19a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:22.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.652+0000 7f5162c83640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f515c108140 0x7f515c19a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:22.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.652+0000 7f5162c83640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f515c108a90 0x7f515c19fa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:22.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.652+0000 7f5162c83640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f515c112710 con 0x7f515c108a90 2024-08-30T09:02:22.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.652+0000 7f5162c83640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f515c112590 con 0x7f515c102800 2024-08-30T09:02:22.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.652+0000 7f5162c83640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f515c112890 con 0x7f515c108140 2024-08-30T09:02:22.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.652+0000 7f5160a1f640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f515c102800 0x7f515c19a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:02:22.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.652+0000 7f5160a1f640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f515c102800 0x7f515c19a1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:35198/0 (socket says 172.21.15.42:35198) 2024-08-30T09:02:22.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.652+0000 7f5160a1f640 1 -- 172.21.15.42:0/1593660017 learned_addr learned my addr 172.21.15.42:0/1593660017 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:02:22.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.652+0000 7f5161220640 1 --2- 172.21.15.42:0/1593660017 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f515c108a90 0x7f515c19fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:02:22.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.652+0000 7f515bfff640 1 --2- 172.21.15.42:0/1593660017 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f515c108140 0x7f515c19a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:02:22.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.653+0000 7f5160a1f640 1 -- 172.21.15.42:0/1593660017 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f515c108140 msgr2=0x7f515c19a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:02:22.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.653+0000 7f5160a1f640 1 --2- 172.21.15.42:0/1593660017 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f515c108140 0x7f515c19a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:22.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.653+0000 7f5160a1f640 1 -- 172.21.15.42:0/1593660017 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f515c108a90 msgr2=0x7f515c19fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:02:22.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.653+0000 7f5160a1f640 1 --2- 172.21.15.42:0/1593660017 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f515c108a90 0x7f515c19fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:22.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.653+0000 7f5160a1f640 1 -- 172.21.15.42:0/1593660017 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f515c1a0140 con 0x7f515c102800 2024-08-30T09:02:22.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.653+0000 7f5161220640 1 --2- 172.21.15.42:0/1593660017 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f515c108a90 0x7f515c19fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-30T09:02:22.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.653+0000 7f5160a1f640 1 --2- 172.21.15.42:0/1593660017 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f515c102800 0x7f515c19a1d0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f515000e9f0 tx=0x7f515000edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:02:22.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.653+0000 7f5159ffb640 1 -- 172.21.15.42:0/1593660017 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f515000cd80 con 0x7f515c102800 2024-08-30T09:02:22.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.653+0000 7f5162c83640 1 -- 172.21.15.42:0/1593660017 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f515c1a0430 con 0x7f515c102800 2024-08-30T09:02:22.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.653+0000 7f5159ffb640 1 -- 172.21.15.42:0/1593660017 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f51500054d0 con 0x7f515c102800 2024-08-30T09:02:22.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.653+0000 7f5159ffb640 1 -- 172.21.15.42:0/1593660017 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5150010690 con 0x7f515c102800 2024-08-30T09:02:22.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.653+0000 7f5162c83640 1 -- 172.21.15.42:0/1593660017 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f515c1a6b90 con 0x7f515c102800 2024-08-30T09:02:22.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.656+0000 7f5159ffb640 1 -- 172.21.15.42:0/1593660017 <== mon.2 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7f5150004e90 con 0x7f515c102800 2024-08-30T09:02:22.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.656+0000 7f51691f8740 1 Processor -- start 2024-08-30T09:02:22.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.656+0000 7f51691f8740 1 -- start start 2024-08-30T09:02:22.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.656+0000 7f51691f8740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ac0ad7b350 0x55ac0b121780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:22.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.656+0000 7f51691f8740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ac0ad52490 0x55ac0b121cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:22.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.656+0000 7f51691f8740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ac0ad5a670 0x55ac0b122200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:22.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.656+0000 7f51691f8740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55ac0b0975f0 con 0x55ac0ad52490 2024-08-30T09:02:22.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.656+0000 7f51691f8740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55ac0b086b80 con 0x55ac0ad7b350 2024-08-30T09:02:22.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.656+0000 7f51691f8740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55ac0b08ab30 con 0x55ac0ad5a670 2024-08-30T09:02:22.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.656+0000 7f5160a1f640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ac0ad52490 0x55ac0b121cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:02:22.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.657+0000 7f5161220640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ac0ad7b350 0x55ac0b121780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:02:22.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.657+0000 7f5160a1f640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ac0ad52490 0x55ac0b121cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:51658/0 (socket says 172.21.15.42:51658) 2024-08-30T09:02:22.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.657+0000 7f5160a1f640 1 -- 172.21.15.42:0/3810572968 learned_addr learned my addr 172.21.15.42:0/3810572968 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:02:22.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.657+0000 7f515bfff640 1 --2- 172.21.15.42:0/3810572968 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ac0ad5a670 0x55ac0b122200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:02:22.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.657+0000 7f5159ffb640 1 --2- 172.21.15.42:0/1593660017 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f514c0b4b30 0x7f514c0b7040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:22.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.657+0000 7f515bfff640 1 -- 172.21.15.42:0/3810572968 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ac0ad7b350 msgr2=0x55ac0b121780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:02:22.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.657+0000 7f515bfff640 1 --2- 172.21.15.42:0/3810572968 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ac0ad7b350 0x55ac0b121780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:22.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.657+0000 7f515bfff640 1 -- 172.21.15.42:0/3810572968 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ac0ad52490 msgr2=0x55ac0b121cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:02:22.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.657+0000 7f515bfff640 1 --2- 172.21.15.42:0/3810572968 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ac0ad52490 0x55ac0b121cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:22.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.657+0000 7f515bfff640 1 -- 172.21.15.42:0/3810572968 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac0ade6930 con 0x55ac0ad5a670 2024-08-30T09:02:22.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.657+0000 7f5161220640 1 --2- 172.21.15.42:0/3810572968 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ac0ad7b350 0x55ac0b121780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-30T09:02:22.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.657+0000 7f5160a1f640 1 --2- 172.21.15.42:0/3810572968 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ac0ad52490 0x55ac0b121cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-30T09:02:22.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.657+0000 7f5161220640 1 --2- 172.21.15.42:0/1593660017 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f514c0b4b30 0x7f514c0b7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:02:22.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.657+0000 7f5159ffb640 1 -- 172.21.15.42:0/1593660017 <== mon.2 v2:172.21.15.167:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f51500e5910 con 0x7f515c102800 2024-08-30T09:02:22.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.657+0000 7f515bfff640 1 --2- 172.21.15.42:0/3810572968 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ac0ad5a670 0x55ac0b122200 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f515c06a840 tx=0x7f51480107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=96c1a6ecac67edb1 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:02:22.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.657+0000 7f5146ffd640 1 -- 172.21.15.42:0/3810572968 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5148004030 con 0x55ac0ad5a670 2024-08-30T09:02:22.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.657+0000 7f5146ffd640 1 -- 172.21.15.42:0/3810572968 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f51480035f0 con 0x55ac0ad5a670 2024-08-30T09:02:22.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.657+0000 7f5146ffd640 1 -- 172.21.15.42:0/3810572968 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51480116a0 con 0x55ac0ad5a670 2024-08-30T09:02:22.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.658+0000 7f51691f8740 1 -- 172.21.15.42:0/3810572968 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ac0ad5a670 msgr2=0x55ac0b122200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:02:22.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.658+0000 7f5161220640 1 --2- 172.21.15.42:0/1593660017 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f514c0b4b30 0x7f514c0b7040 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f5154006010 tx=0x7f51540095c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:02:22.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.658+0000 7f51691f8740 1 --2- 172.21.15.42:0/3810572968 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ac0ad5a670 0x55ac0b122200 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f515c06a840 tx=0x7f51480107a0 comp rx=0 tx=0).stop 2024-08-30T09:02:22.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.658+0000 7f51691f8740 1 -- 172.21.15.42:0/3810572968 shutdown_connections 2024-08-30T09:02:22.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.658+0000 7f51691f8740 1 --2- 172.21.15.42:0/3810572968 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ac0ad5a670 0x55ac0b122200 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:22.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.658+0000 7f51691f8740 1 --2- 172.21.15.42:0/3810572968 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ac0ad52490 0x55ac0b121cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:22.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.658+0000 7f51691f8740 1 --2- 172.21.15.42:0/3810572968 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ac0ad7b350 0x55ac0b121780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:22.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.658+0000 7f51691f8740 1 -- 172.21.15.42:0/3810572968 >> 172.21.15.42:0/3810572968 conn(0x55ac0ad6f740 msgr2=0x55ac0b11f670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:02:22.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.658+0000 7f51691f8740 1 -- 172.21.15.42:0/3810572968 shutdown_connections 2024-08-30T09:02:22.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.658+0000 7f51691f8740 1 -- 172.21.15.42:0/3810572968 wait complete. 2024-08-30T09:02:22.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.659+0000 7f51691f8740 1 -- start start 2024-08-30T09:02:22.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.659+0000 7f51691f8740 1 Processor -- start 2024-08-30T09:02:22.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.659+0000 7f51691f8740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ac0ad52490 0x55ac0b1312c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:22.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.659+0000 7f51691f8740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ac0ad5a670 0x55ac0b131800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:22.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.659+0000 7f51691f8740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ac0ad7b350 0x55ac0b13d620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:22.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.659+0000 7f51691f8740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55ac0aee9890 con 0x55ac0ad5a670 2024-08-30T09:02:22.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.659+0000 7f51691f8740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55ac0b00d790 con 0x55ac0ad7b350 2024-08-30T09:02:22.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.659+0000 7f51691f8740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55ac0b1342c0 con 0x55ac0ad52490 2024-08-30T09:02:22.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.659+0000 7f51691f8740 10 client.0 _ll_register_callbacks cb 0x55ac0ad92ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:02:22.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.659+0000 7f515bfff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ac0ad5a670 0x55ac0b131800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:02:22.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.659+0000 7f515bfff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ac0ad5a670 0x55ac0b131800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:51662/0 (socket says 172.21.15.42:51662) 2024-08-30T09:02:22.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.659+0000 7f515bfff640 1 -- 172.21.15.42:0/1931112235 learned_addr learned my addr 172.21.15.42:0/1931112235 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:02:22.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.659+0000 7f5160a1f640 1 --2- 172.21.15.42:0/1931112235 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ac0ad52490 0x55ac0b1312c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:02:22.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.660+0000 7f5161220640 1 --2- 172.21.15.42:0/1931112235 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ac0ad7b350 0x55ac0b13d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:02:22.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.660+0000 7f515bfff640 1 -- 172.21.15.42:0/1931112235 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ac0ad52490 msgr2=0x55ac0b1312c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:02:22.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.660+0000 7f515bfff640 1 --2- 172.21.15.42:0/1931112235 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ac0ad52490 0x55ac0b1312c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:22.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.660+0000 7f515bfff640 1 -- 172.21.15.42:0/1931112235 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ac0ad7b350 msgr2=0x55ac0b13d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:02:22.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.660+0000 7f515bfff640 1 --2- 172.21.15.42:0/1931112235 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ac0ad7b350 0x55ac0b13d620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:22.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.660+0000 7f515bfff640 1 -- 172.21.15.42:0/1931112235 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac0b044f30 con 0x55ac0ad5a670 2024-08-30T09:02:22.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.660+0000 7f5161220640 1 --2- 172.21.15.42:0/1931112235 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ac0ad7b350 0x55ac0b13d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-30T09:02:22.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.660+0000 7f5160a1f640 1 --2- 172.21.15.42:0/1931112235 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ac0ad52490 0x55ac0b1312c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-30T09:02:22.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.660+0000 7f515bfff640 1 --2- 172.21.15.42:0/1931112235 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ac0ad5a670 0x55ac0b131800 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f5148003f90 tx=0x7f514800beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=297c7581ae1be9c9 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:02:22.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.660+0000 7f513bfff640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-08-30T09:02:22.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.660+0000 7f513bfff640 1 -- 172.21.15.42:0/1931112235 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5148029060 con 0x55ac0ad5a670 2024-08-30T09:02:22.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.660+0000 7f513bfff640 1 -- 172.21.15.42:0/1931112235 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f51480039b0 con 0x55ac0ad5a670 2024-08-30T09:02:22.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.660+0000 7f51691f8740 1 -- 172.21.15.42:0/1931112235 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f51480035f0 con 0x55ac0ad5a670 2024-08-30T09:02:22.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.660+0000 7f513bfff640 1 -- 172.21.15.42:0/1931112235 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f5148030050 con 0x55ac0ad5a670 2024-08-30T09:02:22.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.660+0000 7f51691f8740 10 client.15783 fetch_fsmap learned FSMap version 29 2024-08-30T09:02:22.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.660+0000 7f51691f8740 1 -- 172.21.15.42:0/1931112235 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ac0b1400a0 con 0x55ac0ad5a670 2024-08-30T09:02:22.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.661+0000 7f513bfff640 1 -- 172.21.15.42:0/1931112235 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f51480035f0 con 0x55ac0ad5a670 2024-08-30T09:02:22.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.661+0000 7f513bfff640 1 -- 172.21.15.42:0/1931112235 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f5148011970 con 0x55ac0ad5a670 2024-08-30T09:02:22.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.661+0000 7f51691f8740 10 client.15783 fetch_fsmap finished waiting for FSMap version 29 2024-08-30T09:02:22.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.661+0000 7f51691f8740 10 client.15783 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-30T09:02:22.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.661+0000 7f51691f8740 20 client.15783 populate_metadata read hostname 'smithi042' 2024-08-30T09:02:22.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.661+0000 7f51691f8740 1 --2- 172.21.15.42:0/1931112235 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55ac0ad39cd0 0x55ac0b143400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:22.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.661+0000 7f51691f8740 4 client.15783 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-30T09:02:22.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.661+0000 7f51691f8740 1 -- 172.21.15.42:0/1931112235 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ac0b099540 con 0x55ac0ad39cd0 2024-08-30T09:02:22.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.661+0000 7f5160a1f640 1 --2- 172.21.15.42:0/1931112235 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55ac0ad39cd0 0x55ac0b143400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:02:22.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.662+0000 7f513bfff640 10 client.15783 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:02:22.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.662+0000 7f5160a1f640 1 --2- 172.21.15.42:0/1931112235 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55ac0ad39cd0 0x55ac0b143400 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f237e9b4a0de5b2 server_cookie=5fc47d513aee56e4 in_seq=0 out_seq=0 2024-08-30T09:02:22.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.662+0000 7f513bfff640 1 -- 172.21.15.42:0/1931112235 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55ac0b099540 con 0x55ac0ad39cd0 2024-08-30T09:02:22.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.662+0000 7f513bfff640 10 client.15783 handle_command_reply: tid=0 2024-08-30T09:02:22.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.662+0000 7f51691f8740 1 client.15783 shutdown 2024-08-30T09:02:22.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.662+0000 7f51691f8740 1 -- 172.21.15.42:0/1931112235 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ac0ad5a670 msgr2=0x55ac0b131800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:02:22.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.662+0000 7f51691f8740 1 --2- 172.21.15.42:0/1931112235 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ac0ad5a670 0x55ac0b131800 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f5148003f90 tx=0x7f514800beb0 comp rx=0 tx=0).stop 2024-08-30T09:02:22.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.663+0000 7f51691f8740 1 -- 172.21.15.42:0/1931112235 shutdown_connections 2024-08-30T09:02:22.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.663+0000 7f51691f8740 1 --2- 172.21.15.42:0/1931112235 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55ac0ad39cd0 0x55ac0b143400 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:22.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.663+0000 7f51691f8740 1 --2- 172.21.15.42:0/1931112235 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ac0ad7b350 0x55ac0b13d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:22.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.663+0000 7f51691f8740 1 --2- 172.21.15.42:0/1931112235 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ac0ad5a670 0x55ac0b131800 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:22.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.663+0000 7f51691f8740 1 --2- 172.21.15.42:0/1931112235 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ac0ad52490 0x55ac0b1312c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:22.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.663+0000 7f51691f8740 1 -- 172.21.15.42:0/1931112235 >> 172.21.15.42:0/1931112235 conn(0x55ac0ad6f740 msgr2=0x55ac0b11dc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:02:22.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.663+0000 7f513bfff640 0 client.15783 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:02:22.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.663+0000 7f51691f8740 1 -- 172.21.15.42:0/1931112235 shutdown_connections 2024-08-30T09:02:22.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.663+0000 7f51691f8740 1 -- 172.21.15.42:0/1931112235 wait complete. 2024-08-30T09:02:22.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.663+0000 7f51691f8740 20 client.15783 trim_cache size 0 max 16384 2024-08-30T09:02:22.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.681+0000 7f51691f8740 1 Processor -- start 2024-08-30T09:02:22.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.681+0000 7f51691f8740 1 -- start start 2024-08-30T09:02:22.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.682+0000 7f51691f8740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ac0ad6f740 0x55ac0b13d520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:22.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.682+0000 7f51691f8740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ac0ad52490 0x55ac0b11ef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:22.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.682+0000 7f51691f8740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ac0ad5a670 0x55ac0b11f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:22.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.682+0000 7f51691f8740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f515c1b0f50 con 0x55ac0ad6f740 2024-08-30T09:02:22.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.682+0000 7f51691f8740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f515c1ae820 con 0x55ac0ad52490 2024-08-30T09:02:22.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.682+0000 7f51691f8740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f515c1ac670 con 0x55ac0ad5a670 2024-08-30T09:02:22.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.682+0000 7f5160a1f640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ac0ad6f740 0x55ac0b13d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:02:22.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.682+0000 7f5160a1f640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ac0ad6f740 0x55ac0b13d520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:51670/0 (socket says 172.21.15.42:51670) 2024-08-30T09:02:22.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.682+0000 7f5160a1f640 1 -- 172.21.15.42:0/499737639 learned_addr learned my addr 172.21.15.42:0/499737639 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:02:22.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.682+0000 7f515bfff640 1 --2- 172.21.15.42:0/499737639 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ac0ad52490 0x55ac0b11ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:02:22.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.682+0000 7f5161220640 1 --2- 172.21.15.42:0/499737639 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ac0ad5a670 0x55ac0b11f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:02:22.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.682+0000 7f5160a1f640 1 -- 172.21.15.42:0/499737639 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ac0ad5a670 msgr2=0x55ac0b11f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:02:22.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.682+0000 7f5160a1f640 1 --2- 172.21.15.42:0/499737639 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ac0ad5a670 0x55ac0b11f4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:22.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.682+0000 7f5160a1f640 1 -- 172.21.15.42:0/499737639 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ac0ad52490 msgr2=0x55ac0b11ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:02:22.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.682+0000 7f5160a1f640 1 --2- 172.21.15.42:0/499737639 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ac0ad52490 0x55ac0b11ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:22.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.682+0000 7f5160a1f640 1 -- 172.21.15.42:0/499737639 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac0b122570 con 0x55ac0ad6f740 2024-08-30T09:02:22.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.682+0000 7f5161220640 1 --2- 172.21.15.42:0/499737639 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ac0ad5a670 0x55ac0b11f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-30T09:02:22.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.682+0000 7f5160a1f640 1 --2- 172.21.15.42:0/499737639 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ac0ad6f740 0x55ac0b13d520 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f51500a9220 tx=0x7f51500a8680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e773e5fd3634b1ae server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:02:22.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.683+0000 7f51467fc640 1 -- 172.21.15.42:0/499737639 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5150005180 con 0x55ac0ad6f740 2024-08-30T09:02:22.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.683+0000 7f51467fc640 1 -- 172.21.15.42:0/499737639 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f51500abac0 con 0x55ac0ad6f740 2024-08-30T09:02:22.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.683+0000 7f51467fc640 1 -- 172.21.15.42:0/499737639 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51500ac210 con 0x55ac0ad6f740 2024-08-30T09:02:22.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.683+0000 7f51691f8740 1 -- 172.21.15.42:0/499737639 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ac0ad6f740 msgr2=0x55ac0b13d520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:02:22.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.683+0000 7f51691f8740 1 --2- 172.21.15.42:0/499737639 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ac0ad6f740 0x55ac0b13d520 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f51500a9220 tx=0x7f51500a8680 comp rx=0 tx=0).stop 2024-08-30T09:02:22.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.683+0000 7f51691f8740 1 -- 172.21.15.42:0/499737639 shutdown_connections 2024-08-30T09:02:22.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.683+0000 7f51691f8740 1 --2- 172.21.15.42:0/499737639 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ac0ad5a670 0x55ac0b11f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:22.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.683+0000 7f51691f8740 1 --2- 172.21.15.42:0/499737639 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ac0ad52490 0x55ac0b11ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:22.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.683+0000 7f51691f8740 1 --2- 172.21.15.42:0/499737639 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ac0ad6f740 0x55ac0b13d520 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:22.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.684+0000 7f51691f8740 1 -- 172.21.15.42:0/499737639 >> 172.21.15.42:0/499737639 conn(0x55ac0b12cc10 msgr2=0x55ac0b12fb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:02:22.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.684+0000 7f51691f8740 1 -- 172.21.15.42:0/499737639 shutdown_connections 2024-08-30T09:02:22.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.684+0000 7f51691f8740 1 -- 172.21.15.42:0/499737639 wait complete. 2024-08-30T09:02:22.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.684+0000 7f51691f8740 1 -- start start 2024-08-30T09:02:22.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.684+0000 7f51691f8740 1 Processor -- start 2024-08-30T09:02:22.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.684+0000 7f51691f8740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ac0ad52490 0x55ac0b139590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:22.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.684+0000 7f51691f8740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ac0ad5a670 0x55ac0b139ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:22.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.684+0000 7f51691f8740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ac0ad6f740 0x55ac0b134540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:22.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.684+0000 7f51691f8740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f515c1acce0 con 0x55ac0ad5a670 2024-08-30T09:02:22.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.684+0000 7f51691f8740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f515c1ae150 con 0x55ac0ad52490 2024-08-30T09:02:22.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.684+0000 7f51691f8740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f515c1ad3b0 con 0x55ac0ad6f740 2024-08-30T09:02:22.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.685+0000 7f51691f8740 10 client.0 _ll_register_callbacks cb 0x7f51500eb420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:02:22.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.685+0000 7f515bfff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ac0ad5a670 0x55ac0b139ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:02:22.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.685+0000 7f515bfff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ac0ad5a670 0x55ac0b139ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:51676/0 (socket says 172.21.15.42:51676) 2024-08-30T09:02:22.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.685+0000 7f515bfff640 1 -- 172.21.15.42:0/2471980660 learned_addr learned my addr 172.21.15.42:0/2471980660 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:02:22.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.685+0000 7f5160a1f640 1 --2- 172.21.15.42:0/2471980660 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ac0ad52490 0x55ac0b139590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:02:22.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.685+0000 7f5161220640 1 --2- 172.21.15.42:0/2471980660 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ac0ad6f740 0x55ac0b134540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:02:22.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.685+0000 7f515bfff640 1 -- 172.21.15.42:0/2471980660 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ac0ad6f740 msgr2=0x55ac0b134540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:02:22.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.685+0000 7f515bfff640 1 --2- 172.21.15.42:0/2471980660 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ac0ad6f740 0x55ac0b134540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:22.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.685+0000 7f515bfff640 1 -- 172.21.15.42:0/2471980660 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ac0ad52490 msgr2=0x55ac0b139590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:02:22.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.685+0000 7f515bfff640 1 --2- 172.21.15.42:0/2471980660 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ac0ad52490 0x55ac0b139590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:22.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.685+0000 7f515bfff640 1 -- 172.21.15.42:0/2471980660 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac0b134a80 con 0x55ac0ad5a670 2024-08-30T09:02:22.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.685+0000 7f513bfff640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-08-30T09:02:22.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.685+0000 7f515bfff640 1 --2- 172.21.15.42:0/2471980660 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ac0ad5a670 0x55ac0b139ad0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f5148011f30 tx=0x7f514801b650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b333a72e3b7de3f3 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:02:22.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.685+0000 7f513bfff640 1 -- 172.21.15.42:0/2471980660 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5148029040 con 0x55ac0ad5a670 2024-08-30T09:02:22.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.685+0000 7f513bfff640 1 -- 172.21.15.42:0/2471980660 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f5148003bc0 con 0x55ac0ad5a670 2024-08-30T09:02:22.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.685+0000 7f51691f8740 1 -- 172.21.15.42:0/2471980660 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f51500abac0 con 0x55ac0ad5a670 2024-08-30T09:02:22.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.686+0000 7f513bfff640 1 -- 172.21.15.42:0/2471980660 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f514803d050 con 0x55ac0ad5a670 2024-08-30T09:02:22.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.686+0000 7f51691f8740 10 client.15789 fetch_fsmap learned FSMap version 29 2024-08-30T09:02:22.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.686+0000 7f51691f8740 1 -- 172.21.15.42:0/2471980660 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ac0b1ae250 con 0x55ac0ad5a670 2024-08-30T09:02:22.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.686+0000 7f513bfff640 1 -- 172.21.15.42:0/2471980660 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f51500abac0 con 0x55ac0ad5a670 2024-08-30T09:02:22.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.686+0000 7f513bfff640 1 -- 172.21.15.42:0/2471980660 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f514801ba10 con 0x55ac0ad5a670 2024-08-30T09:02:22.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.686+0000 7f51691f8740 10 client.15789 fetch_fsmap finished waiting for FSMap version 29 2024-08-30T09:02:22.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.686+0000 7f51691f8740 10 client.15789 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-30T09:02:22.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.686+0000 7f51691f8740 20 client.15789 populate_metadata read hostname 'smithi042' 2024-08-30T09:02:22.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.686+0000 7f51691f8740 1 --2- 172.21.15.42:0/2471980660 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55ac0ad7b350 0x55ac0b1b0980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:22.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.686+0000 7f51691f8740 4 client.15789 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-08-30T09:02:22.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.686+0000 7f51691f8740 1 -- 172.21.15.42:0/2471980660 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55ac0b11edc0 con 0x55ac0ad7b350 2024-08-30T09:02:22.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.686+0000 7f5160a1f640 1 --2- 172.21.15.42:0/2471980660 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55ac0ad7b350 0x55ac0b1b0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:02:22.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.687+0000 7f513bfff640 10 client.15789 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:02:22.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.687+0000 7f5160a1f640 1 --2- 172.21.15.42:0/2471980660 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55ac0ad7b350 0x55ac0b1b0980 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6126924bd2049838 server_cookie=b55f69fa40d29925 in_seq=0 out_seq=0 2024-08-30T09:02:22.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.689+0000 7f513bfff640 1 -- 172.21.15.42:0/2471980660 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55ac0b11edc0 con 0x55ac0ad7b350 2024-08-30T09:02:22.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.689+0000 7f513bfff640 10 client.15789 handle_command_reply: tid=0 2024-08-30T09:02:22.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.689+0000 7f51691f8740 1 client.15789 shutdown 2024-08-30T09:02:22.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.690+0000 7f51691f8740 1 -- 172.21.15.42:0/2471980660 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ac0ad5a670 msgr2=0x55ac0b139ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:02:22.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.690+0000 7f51691f8740 1 --2- 172.21.15.42:0/2471980660 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ac0ad5a670 0x55ac0b139ad0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f5148011f30 tx=0x7f514801b650 comp rx=0 tx=0).stop 2024-08-30T09:02:22.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.690+0000 7f51691f8740 1 -- 172.21.15.42:0/2471980660 shutdown_connections 2024-08-30T09:02:22.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.690+0000 7f51691f8740 1 --2- 172.21.15.42:0/2471980660 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55ac0ad7b350 0x55ac0b1b0980 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:22.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.690+0000 7f51691f8740 1 --2- 172.21.15.42:0/2471980660 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ac0ad6f740 0x55ac0b134540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:22.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.690+0000 7f51691f8740 1 --2- 172.21.15.42:0/2471980660 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ac0ad5a670 0x55ac0b139ad0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:22.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.690+0000 7f513bfff640 0 client.15789 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:02:22.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.690+0000 7f51691f8740 1 --2- 172.21.15.42:0/2471980660 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ac0ad52490 0x55ac0b139590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:22.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.690+0000 7f51691f8740 1 -- 172.21.15.42:0/2471980660 >> 172.21.15.42:0/2471980660 conn(0x55ac0b12cc10 msgr2=0x55ac0b11ef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:02:22.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.690+0000 7f51691f8740 1 -- 172.21.15.42:0/2471980660 shutdown_connections 2024-08-30T09:02:22.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.690+0000 7f51691f8740 1 -- 172.21.15.42:0/2471980660 wait complete. 2024-08-30T09:02:22.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.690+0000 7f51691f8740 20 client.15789 trim_cache size 0 max 16384 2024-08-30T09:02:22.893 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-08-30T09:02:22.893 INFO:teuthology.orchestra.run.smithi042.stdout: "return_code": 0, 2024-08-30T09:02:22.893 INFO:teuthology.orchestra.run.smithi042.stdout: "scrub_tag": "c827ad7a-44ee-4e1d-9bb3-51fb7ab00c73", 2024-08-30T09:02:22.893 INFO:teuthology.orchestra.run.smithi042.stdout: "mode": "asynchronous" 2024-08-30T09:02:22.893 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-08-30T09:02:22.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.691+0000 7f5162c83640 1 -- 172.21.15.42:0/1593660017 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f514c0b4b30 msgr2=0x7f514c0b7040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:02:22.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.691+0000 7f5162c83640 1 --2- 172.21.15.42:0/1593660017 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f514c0b4b30 0x7f514c0b7040 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f5154006010 tx=0x7f51540095c0 comp rx=0 tx=0).stop 2024-08-30T09:02:22.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.691+0000 7f5162c83640 1 -- 172.21.15.42:0/1593660017 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f515c102800 msgr2=0x7f515c19a1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:02:22.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.691+0000 7f5162c83640 1 --2- 172.21.15.42:0/1593660017 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f515c102800 0x7f515c19a1d0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f515000e9f0 tx=0x7f515000edf0 comp rx=0 tx=0).stop 2024-08-30T09:02:22.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.691+0000 7f5162c83640 1 -- 172.21.15.42:0/1593660017 shutdown_connections 2024-08-30T09:02:22.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.691+0000 7f5162c83640 1 --2- 172.21.15.42:0/1593660017 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f514c0b4b30 0x7f514c0b7040 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:22.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.691+0000 7f5162c83640 1 --2- 172.21.15.42:0/1593660017 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f515c108140 0x7f515c19a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:22.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.691+0000 7f5162c83640 1 --2- 172.21.15.42:0/1593660017 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f515c108a90 0x7f515c19fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:22.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.691+0000 7f5162c83640 1 --2- 172.21.15.42:0/1593660017 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f515c102800 0x7f515c19a1d0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:22.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.691+0000 7f5162c83640 1 -- 172.21.15.42:0/1593660017 >> 172.21.15.42:0/1593660017 conn(0x7f515c0794d0 msgr2=0x7f515c06aef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:02:22.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.691+0000 7f5162c83640 1 -- 172.21.15.42:0/1593660017 shutdown_connections 2024-08-30T09:02:22.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:21.691+0000 7f5162c83640 1 -- 172.21.15.42:0/1593660017 wait complete. 2024-08-30T09:02:23.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:02:22 smithi167 ceph-mon[22063]: pgmap v421: 129 pgs: 129 active+clean; 1.0 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 9.9 MiB/s wr, 704 op/s 2024-08-30T09:02:23.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:02:22 smithi167 ceph-mon[22063]: scrub queued for path: / 2024-08-30T09:02:23.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:02:22 smithi167 ceph-mon[22063]: scrub summary: idle+waiting paths [/] 2024-08-30T09:02:23.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:02:22 smithi167 ceph-mon[22063]: scrub summary: active paths [/] 2024-08-30T09:02:23.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:02:22 smithi167 ceph-mon[22063]: scrub summary: active paths [/] 2024-08-30T09:02:23.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:02:22 smithi167 ceph-mon[22063]: scrub summary: idle+waiting paths [/] 2024-08-30T09:02:23.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:02:22 smithi167 ceph-mon[22063]: scrub summary: active paths [/] 2024-08-30T09:02:23.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:02:22 smithi167 ceph-mon[22063]: scrub summary: idle+waiting paths [/] 2024-08-30T09:02:23.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:02:22 smithi167 ceph-mon[22063]: scrub summary: active paths [/] 2024-08-30T09:02:23.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:02:22 smithi167 ceph-mon[22063]: scrub summary: active paths [/] 2024-08-30T09:02:23.078 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:02:22 smithi167 ceph-mon[22063]: scrub summary: idle+waiting paths [/] 2024-08-30T09:02:23.078 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:02:22 smithi167 ceph-mon[22063]: scrub summary: idle+waiting paths [/] 2024-08-30T09:02:23.078 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:02:22 smithi167 ceph-mon[22063]: scrub summary: idle+waiting paths [/] 2024-08-30T09:02:23.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:02:22 smithi042 ceph-mon[18652]: pgmap v421: 129 pgs: 129 active+clean; 1.0 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 9.9 MiB/s wr, 704 op/s 2024-08-30T09:02:23.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:02:22 smithi042 ceph-mon[18652]: scrub queued for path: / 2024-08-30T09:02:23.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:02:22 smithi042 ceph-mon[18652]: scrub summary: idle+waiting paths [/] 2024-08-30T09:02:23.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:02:22 smithi042 ceph-mon[18652]: scrub summary: active paths [/] 2024-08-30T09:02:23.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:02:22 smithi042 ceph-mon[18652]: scrub summary: active paths [/] 2024-08-30T09:02:23.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:02:22 smithi042 ceph-mon[18652]: scrub summary: idle+waiting paths [/] 2024-08-30T09:02:23.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:02:22 smithi042 ceph-mon[18652]: scrub summary: active paths [/] 2024-08-30T09:02:23.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:02:22 smithi042 ceph-mon[18652]: scrub summary: idle+waiting paths [/] 2024-08-30T09:02:23.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:02:22 smithi042 ceph-mon[18652]: scrub summary: active paths [/] 2024-08-30T09:02:23.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:02:22 smithi042 ceph-mon[18652]: scrub summary: active paths [/] 2024-08-30T09:02:23.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:02:22 smithi042 ceph-mon[18652]: scrub summary: idle+waiting paths [/] 2024-08-30T09:02:23.134 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:02:22 smithi042 ceph-mon[18652]: scrub summary: idle+waiting paths [/] 2024-08-30T09:02:23.134 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:02:22 smithi042 ceph-mon[18652]: scrub summary: idle+waiting paths [/] 2024-08-30T09:02:23.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:02:22 smithi190 ceph-mon[22001]: pgmap v421: 129 pgs: 129 active+clean; 1.0 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 9.9 MiB/s wr, 704 op/s 2024-08-30T09:02:23.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:02:22 smithi190 ceph-mon[22001]: scrub queued for path: / 2024-08-30T09:02:23.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:02:22 smithi190 ceph-mon[22001]: scrub summary: idle+waiting paths [/] 2024-08-30T09:02:23.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:02:22 smithi190 ceph-mon[22001]: scrub summary: active paths [/] 2024-08-30T09:02:23.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:02:22 smithi190 ceph-mon[22001]: scrub summary: active paths [/] 2024-08-30T09:02:23.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:02:22 smithi190 ceph-mon[22001]: scrub summary: idle+waiting paths [/] 2024-08-30T09:02:23.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:02:22 smithi190 ceph-mon[22001]: scrub summary: active paths [/] 2024-08-30T09:02:23.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:02:22 smithi190 ceph-mon[22001]: scrub summary: idle+waiting paths [/] 2024-08-30T09:02:23.187 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:02:22 smithi190 ceph-mon[22001]: scrub summary: active paths [/] 2024-08-30T09:02:23.187 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:02:22 smithi190 ceph-mon[22001]: scrub summary: active paths [/] 2024-08-30T09:02:23.187 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:02:22 smithi190 ceph-mon[22001]: scrub summary: idle+waiting paths [/] 2024-08-30T09:02:23.187 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:02:22 smithi190 ceph-mon[22001]: scrub summary: idle+waiting paths [/] 2024-08-30T09:02:23.187 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:02:22 smithi190 ceph-mon[22001]: scrub summary: idle+waiting paths [/] 2024-08-30T09:02:25.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:02:24 smithi167 ceph-mon[22063]: pgmap v422: 129 pgs: 129 active+clean; 1.0 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 9.5 MiB/s wr, 811 op/s 2024-08-30T09:02:25.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:02:24 smithi042 ceph-mon[18652]: pgmap v422: 129 pgs: 129 active+clean; 1.0 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 9.5 MiB/s wr, 811 op/s 2024-08-30T09:02:25.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:02:24 smithi190 ceph-mon[22001]: pgmap v422: 129 pgs: 129 active+clean; 1.0 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 9.5 MiB/s wr, 811 op/s 2024-08-30T09:02:25.953 INFO:teuthology.orchestra.run.smithi042.stderr:progress: 100.0 s, 1461.4 tps, lat 69.173 ms stddev 88.938 2024-08-30T09:02:26.003 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-30T09:02:26.427 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T09:02:26.855 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:02:26 smithi042 ceph-mon[18652]: pgmap v423: 129 pgs: 129 active+clean; 1.0 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 6.1 MiB/s wr, 660 op/s 2024-08-30T09:02:27.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.016+0000 7f855e18a640 1 -- 172.21.15.42:0/337148502 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f855810f870 msgr2=0x7f855810fc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:02:27.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.016+0000 7f855e18a640 1 --2- 172.21.15.42:0/337148502 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f855810f870 0x7f855810fc80 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f8540009860 tx=0x7f854002a380 comp rx=0 tx=0).stop 2024-08-30T09:02:27.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.017+0000 7f855e18a640 1 -- 172.21.15.42:0/337148502 shutdown_connections 2024-08-30T09:02:27.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.017+0000 7f855e18a640 1 --2- 172.21.15.42:0/337148502 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f85581101c0 0x7f85581169f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:27.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.017+0000 7f855e18a640 1 --2- 172.21.15.42:0/337148502 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f855810f870 0x7f855810fc80 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:27.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.017+0000 7f855e18a640 1 --2- 172.21.15.42:0/337148502 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8558109f30 0x7f855810a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:27.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.017+0000 7f855e18a640 1 -- 172.21.15.42:0/337148502 >> 172.21.15.42:0/337148502 conn(0x7f8558077e90 msgr2=0x7f8558107f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:02:27.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.020+0000 7f855e18a640 1 -- 172.21.15.42:0/337148502 shutdown_connections 2024-08-30T09:02:27.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.020+0000 7f855e18a640 1 -- 172.21.15.42:0/337148502 wait complete. 2024-08-30T09:02:27.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.021+0000 7f855e18a640 1 Processor -- start 2024-08-30T09:02:27.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.021+0000 7f855e18a640 1 -- start start 2024-08-30T09:02:27.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.022+0000 7f855e18a640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8558109f30 0x7f8558116830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:27.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.022+0000 7f855e18a640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f855810f870 0x7f8558112b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:27.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.022+0000 7f855e18a640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f85581101c0 0x7f8558113100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:27.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.022+0000 7f855e18a640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f855806f410 con 0x7f85581101c0 2024-08-30T09:02:27.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.022+0000 7f855e18a640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f855806f290 con 0x7f855810f870 2024-08-30T09:02:27.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.022+0000 7f855e18a640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f855806f590 con 0x7f8558109f30 2024-08-30T09:02:27.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.022+0000 7f854effd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f855810f870 0x7f8558112b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:02:27.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.022+0000 7f854effd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f855810f870 0x7f8558112b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:35260/0 (socket says 172.21.15.42:35260) 2024-08-30T09:02:27.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.022+0000 7f854effd640 1 -- 172.21.15.42:0/3705864180 learned_addr learned my addr 172.21.15.42:0/3705864180 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:02:27.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.022+0000 7f85577fe640 1 --2- 172.21.15.42:0/3705864180 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8558109f30 0x7f8558116830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:02:27.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.023+0000 7f85577fe640 1 -- 172.21.15.42:0/3705864180 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f855810f870 msgr2=0x7f8558112b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:02:27.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.023+0000 7f85577fe640 1 --2- 172.21.15.42:0/3705864180 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f855810f870 0x7f8558112b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:27.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.023+0000 7f85577fe640 1 -- 172.21.15.42:0/3705864180 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f85581101c0 msgr2=0x7f8558113100 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-30T09:02:27.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.023+0000 7f85577fe640 1 --2- 172.21.15.42:0/3705864180 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f85581101c0 0x7f8558113100 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:27.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.023+0000 7f85577fe640 1 -- 172.21.15.42:0/3705864180 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8558113920 con 0x7f8558109f30 2024-08-30T09:02:27.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.023+0000 7f85577fe640 1 --2- 172.21.15.42:0/3705864180 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8558109f30 0x7f8558116830 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f854800e9f0 tx=0x7f854800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:02:27.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.023+0000 7f85557fa640 1 -- 172.21.15.42:0/3705864180 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f854800cd80 con 0x7f8558109f30 2024-08-30T09:02:27.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.023+0000 7f85557fa640 1 -- 172.21.15.42:0/3705864180 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8548011920 con 0x7f8558109f30 2024-08-30T09:02:27.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.023+0000 7f85557fa640 1 -- 172.21.15.42:0/3705864180 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85480109c0 con 0x7f8558109f30 2024-08-30T09:02:27.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.023+0000 7f855e18a640 1 -- 172.21.15.42:0/3705864180 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85581a44c0 con 0x7f8558109f30 2024-08-30T09:02:27.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.023+0000 7f855e18a640 1 -- 172.21.15.42:0/3705864180 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85581a4980 con 0x7f8558109f30 2024-08-30T09:02:27.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.028+0000 7f85646ff740 1 Processor -- start 2024-08-30T09:02:27.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.028+0000 7f85646ff740 1 -- start start 2024-08-30T09:02:27.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.028+0000 7f85646ff740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a019275070 0x55a01961a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:27.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.028+0000 7f85646ff740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a01924c1b0 0x55a01961a800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:27.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.028+0000 7f85646ff740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a019254390 0x55a01961ad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:27.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.028+0000 7f85646ff740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55a019591630 con 0x55a01924c1b0 2024-08-30T09:02:27.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.028+0000 7f85646ff740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55a019580bc0 con 0x55a019254390 2024-08-30T09:02:27.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.028+0000 7f85646ff740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55a019584b70 con 0x55a019275070 2024-08-30T09:02:27.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.028+0000 7f85557fa640 1 -- 172.21.15.42:0/3705864180 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7f8548011ad0 con 0x7f8558109f30 2024-08-30T09:02:27.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.029+0000 7f8557fff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a019275070 0x55a01961a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:02:27.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.029+0000 7f854effd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a019254390 0x55a01961ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:02:27.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.029+0000 7f854effd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a019254390 0x55a01961ad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:35276/0 (socket says 172.21.15.42:35276) 2024-08-30T09:02:27.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.029+0000 7f85577fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a01924c1b0 0x55a01961a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:02:27.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.029+0000 7f854effd640 1 -- 172.21.15.42:0/222668796 learned_addr learned my addr 172.21.15.42:0/222668796 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:02:27.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.029+0000 7f8557fff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a019275070 0x55a01961a2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:43954/0 (socket says 172.21.15.42:43954) 2024-08-30T09:02:27.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.029+0000 7f854effd640 1 -- 172.21.15.42:0/222668796 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a019275070 msgr2=0x55a01961a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:02:27.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.029+0000 7f854effd640 1 --2- 172.21.15.42:0/222668796 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a019275070 0x55a01961a2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:27.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.029+0000 7f854effd640 1 -- 172.21.15.42:0/222668796 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a01924c1b0 msgr2=0x55a01961a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:02:27.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.029+0000 7f854effd640 1 --2- 172.21.15.42:0/222668796 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a01924c1b0 0x55a01961a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:27.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.029+0000 7f854effd640 1 -- 172.21.15.42:0/222668796 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a0192e0650 con 0x55a019254390 2024-08-30T09:02:27.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.029+0000 7f854effd640 1 --2- 172.21.15.42:0/222668796 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a019254390 0x55a01961ad40 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f8540009780 tx=0x7f8540030c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b81e92f6de0daa73 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:02:27.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.029+0000 7f854dffb640 1 -- 172.21.15.42:0/222668796 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8540037040 con 0x55a019254390 2024-08-30T09:02:27.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.030+0000 7f854dffb640 1 -- 172.21.15.42:0/222668796 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8540030340 con 0x55a019254390 2024-08-30T09:02:27.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.030+0000 7f854dffb640 1 -- 172.21.15.42:0/222668796 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85400053b0 con 0x55a019254390 2024-08-30T09:02:27.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.031+0000 7f85557fa640 1 --2- 172.21.15.42:0/3705864180 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f85440b4bf0 0x7f85440b7100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:27.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.032+0000 7f85646ff740 1 -- 172.21.15.42:0/222668796 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a019254390 msgr2=0x55a01961ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:02:27.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.032+0000 7f85646ff740 1 --2- 172.21.15.42:0/222668796 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a019254390 0x55a01961ad40 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f8540009780 tx=0x7f8540030c00 comp rx=0 tx=0).stop 2024-08-30T09:02:27.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.032+0000 7f85557fa640 1 -- 172.21.15.42:0/3705864180 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f85480e5a20 con 0x7f8558109f30 2024-08-30T09:02:27.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.032+0000 7f8557fff640 1 --2- 172.21.15.42:0/3705864180 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f85440b4bf0 0x7f85440b7100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:02:27.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.032+0000 7f85646ff740 1 -- 172.21.15.42:0/222668796 shutdown_connections 2024-08-30T09:02:27.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.032+0000 7f85646ff740 1 --2- 172.21.15.42:0/222668796 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a019254390 0x55a01961ad40 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:27.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.032+0000 7f85646ff740 1 --2- 172.21.15.42:0/222668796 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a01924c1b0 0x55a01961a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:27.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.032+0000 7f85646ff740 1 --2- 172.21.15.42:0/222668796 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a019275070 0x55a01961a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:27.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.032+0000 7f85646ff740 1 -- 172.21.15.42:0/222668796 >> 172.21.15.42:0/222668796 conn(0x55a019269460 msgr2=0x55a0196181b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:02:27.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.033+0000 7f8557fff640 1 --2- 172.21.15.42:0/3705864180 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f85440b4bf0 0x7f85440b7100 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f855001c9b0 tx=0x7f8550005e00 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:02:27.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.034+0000 7f85646ff740 1 -- 172.21.15.42:0/222668796 shutdown_connections 2024-08-30T09:02:27.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.034+0000 7f85646ff740 1 -- 172.21.15.42:0/222668796 wait complete. 2024-08-30T09:02:27.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.034+0000 7f85646ff740 1 -- start start 2024-08-30T09:02:27.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.034+0000 7f85646ff740 1 Processor -- start 2024-08-30T09:02:27.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.035+0000 7f85646ff740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a01924c1b0 0x55a019625ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:27.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.035+0000 7f85646ff740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a019254390 0x55a019626010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:27.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.035+0000 7f85646ff740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a019275070 0x55a019631d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:27.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.035+0000 7f85646ff740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55a0193e3700 con 0x55a019254390 2024-08-30T09:02:27.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.035+0000 7f85646ff740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55a019507700 con 0x55a01924c1b0 2024-08-30T09:02:27.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.035+0000 7f85646ff740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55a019628a40 con 0x55a019275070 2024-08-30T09:02:27.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.035+0000 7f8557fff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a019275070 0x55a019631d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:02:27.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.035+0000 7f8557fff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a019275070 0x55a019631d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:43960/0 (socket says 172.21.15.42:43960) 2024-08-30T09:02:27.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.035+0000 7f8557fff640 1 -- 172.21.15.42:0/3888509191 learned_addr learned my addr 172.21.15.42:0/3888509191 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:02:27.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.035+0000 7f85577fe640 1 --2- 172.21.15.42:0/3888509191 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a01924c1b0 0x55a019625ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:02:27.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.035+0000 7f854effd640 1 --2- 172.21.15.42:0/3888509191 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a019254390 0x55a019626010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:02:27.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.035+0000 7f85646ff740 10 client.0 _ll_register_callbacks cb 0x55a019536dc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:02:27.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.035+0000 7f85577fe640 1 -- 172.21.15.42:0/3888509191 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a019275070 msgr2=0x55a019631d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:02:27.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.035+0000 7f85577fe640 1 --2- 172.21.15.42:0/3888509191 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a019275070 0x55a019631d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:27.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.035+0000 7f85577fe640 1 -- 172.21.15.42:0/3888509191 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a019254390 msgr2=0x55a019626010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:02:27.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.035+0000 7f85577fe640 1 --2- 172.21.15.42:0/3888509191 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a019254390 0x55a019626010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:27.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.036+0000 7f85577fe640 1 -- 172.21.15.42:0/3888509191 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a0195408b0 con 0x55a01924c1b0 2024-08-30T09:02:27.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.036+0000 7f85577fe640 1 --2- 172.21.15.42:0/3888509191 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a01924c1b0 0x55a019625ad0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x55a019618f40 tx=0x7f85480abf30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b0377570d123879c server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:02:27.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.036+0000 7f8536ffd640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-30T09:02:27.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.036+0000 7f8536ffd640 1 -- 172.21.15.42:0/3888509191 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85480abb70 con 0x55a01924c1b0 2024-08-30T09:02:27.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.036+0000 7f8536ffd640 1 -- 172.21.15.42:0/3888509191 <== mon.2 v2:172.21.15.167:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f85480b5100 con 0x55a01924c1b0 2024-08-30T09:02:27.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.036+0000 7f85646ff740 1 -- 172.21.15.42:0/3888509191 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8540030340 con 0x55a01924c1b0 2024-08-30T09:02:27.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.036+0000 7f8536ffd640 1 -- 172.21.15.42:0/3888509191 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f85480ac760 con 0x55a01924c1b0 2024-08-30T09:02:27.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.036+0000 7f85646ff740 10 client.25466 fetch_fsmap learned FSMap version 29 2024-08-30T09:02:27.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.036+0000 7f85646ff740 1 -- 172.21.15.42:0/3888509191 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a019634860 con 0x55a01924c1b0 2024-08-30T09:02:27.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.036+0000 7f8536ffd640 1 -- 172.21.15.42:0/3888509191 <== mon.2 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8540030340 con 0x55a01924c1b0 2024-08-30T09:02:27.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.037+0000 7f8536ffd640 1 -- 172.21.15.42:0/3888509191 <== mon.2 v2:172.21.15.167:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f854800f040 con 0x55a01924c1b0 2024-08-30T09:02:27.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.037+0000 7f85646ff740 10 client.25466 fetch_fsmap finished waiting for FSMap version 29 2024-08-30T09:02:27.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.037+0000 7f85646ff740 10 client.25466 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-30T09:02:27.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.037+0000 7f85646ff740 20 client.25466 populate_metadata read hostname 'smithi042' 2024-08-30T09:02:27.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.037+0000 7f85646ff740 1 --2- 172.21.15.42:0/3888509191 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55a0192339f0 0x55a019637db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:27.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.037+0000 7f85646ff740 4 client.25466 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-30T09:02:27.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.037+0000 7f85646ff740 1 -- 172.21.15.42:0/3888509191 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a019593580 con 0x55a0192339f0 2024-08-30T09:02:27.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.037+0000 7f854effd640 1 --2- 172.21.15.42:0/3888509191 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55a0192339f0 0x55a019637db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:02:27.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.038+0000 7f854effd640 1 --2- 172.21.15.42:0/3888509191 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55a0192339f0 0x55a019637db0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2f3f1e6074cadd28 server_cookie=d39063938ab4aca3 in_seq=0 out_seq=0 2024-08-30T09:02:27.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.038+0000 7f8536ffd640 10 client.25466 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:02:27.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.039+0000 7f8536ffd640 1 -- 172.21.15.42:0/3888509191 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a019593580 con 0x55a0192339f0 2024-08-30T09:02:27.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.039+0000 7f8536ffd640 10 client.25466 handle_command_reply: tid=0 2024-08-30T09:02:27.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.039+0000 7f85646ff740 1 client.25466 shutdown 2024-08-30T09:02:27.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.039+0000 7f85646ff740 1 -- 172.21.15.42:0/3888509191 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a01924c1b0 msgr2=0x55a019625ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:02:27.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.039+0000 7f85646ff740 1 --2- 172.21.15.42:0/3888509191 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a01924c1b0 0x55a019625ad0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x55a019618f40 tx=0x7f85480abf30 comp rx=0 tx=0).stop 2024-08-30T09:02:27.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.039+0000 7f85646ff740 1 -- 172.21.15.42:0/3888509191 shutdown_connections 2024-08-30T09:02:27.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.039+0000 7f85646ff740 1 --2- 172.21.15.42:0/3888509191 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55a0192339f0 0x55a019637db0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:27.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.039+0000 7f85646ff740 1 --2- 172.21.15.42:0/3888509191 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a019275070 0x55a019631d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:27.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.039+0000 7f85646ff740 1 --2- 172.21.15.42:0/3888509191 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a019254390 0x55a019626010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:27.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.039+0000 7f85646ff740 1 --2- 172.21.15.42:0/3888509191 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a01924c1b0 0x55a019625ad0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:27.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.039+0000 7f85646ff740 1 -- 172.21.15.42:0/3888509191 >> 172.21.15.42:0/3888509191 conn(0x55a019269460 msgr2=0x55a01961d450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:02:27.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.040+0000 7f8536ffd640 0 client.25466 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:02:27.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.040+0000 7f85646ff740 1 -- 172.21.15.42:0/3888509191 shutdown_connections 2024-08-30T09:02:27.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.040+0000 7f85646ff740 1 -- 172.21.15.42:0/3888509191 wait complete. 2024-08-30T09:02:27.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.040+0000 7f85646ff740 20 client.25466 trim_cache size 0 max 16384 2024-08-30T09:02:27.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.065+0000 7f85646ff740 1 Processor -- start 2024-08-30T09:02:27.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.065+0000 7f85646ff740 1 -- start start 2024-08-30T09:02:27.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.066+0000 7f85646ff740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a01924c1b0 0x55a01961dad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:27.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.066+0000 7f85646ff740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a019254390 0x55a019631c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:27.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.066+0000 7f85646ff740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a019275070 0x55a019622cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:27.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.066+0000 7f85646ff740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f85581b10c0 con 0x55a019254390 2024-08-30T09:02:27.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.066+0000 7f85646ff740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f85581ae990 con 0x55a019275070 2024-08-30T09:02:27.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.066+0000 7f85646ff740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f85581ac7e0 con 0x55a01924c1b0 2024-08-30T09:02:27.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.066+0000 7f85577fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a01924c1b0 0x55a01961dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:02:27.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.066+0000 7f85577fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a01924c1b0 0x55a01961dad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:43968/0 (socket says 172.21.15.42:43968) 2024-08-30T09:02:27.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.066+0000 7f85577fe640 1 -- 172.21.15.42:0/169780967 learned_addr learned my addr 172.21.15.42:0/169780967 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:02:27.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.066+0000 7f8557fff640 1 --2- 172.21.15.42:0/169780967 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a019275070 0x55a019622cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:02:27.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.066+0000 7f854effd640 1 --2- 172.21.15.42:0/169780967 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a019254390 0x55a019631c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:02:27.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.067+0000 7f85577fe640 1 -- 172.21.15.42:0/169780967 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a019275070 msgr2=0x55a019622cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:02:27.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.067+0000 7f85577fe640 1 --2- 172.21.15.42:0/169780967 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a019275070 0x55a019622cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:27.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.067+0000 7f85577fe640 1 -- 172.21.15.42:0/169780967 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a019254390 msgr2=0x55a019631c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:02:27.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.067+0000 7f85577fe640 1 --2- 172.21.15.42:0/169780967 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a019254390 0x55a019631c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:27.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.067+0000 7f85577fe640 1 -- 172.21.15.42:0/169780967 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a01961c5b0 con 0x55a01924c1b0 2024-08-30T09:02:27.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.067+0000 7f8557fff640 1 --2- 172.21.15.42:0/169780967 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a019275070 0x55a019622cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-30T09:02:27.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.067+0000 7f85577fe640 1 --2- 172.21.15.42:0/169780967 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a01924c1b0 0x55a01961dad0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f85480e5de0 tx=0x7f8548010460 comp rx=0 tx=0).ready entity=mon.1 client_cookie=58acf6e22456a688 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:02:27.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.067+0000 7f854d7fa640 1 -- 172.21.15.42:0/169780967 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85480ab7d0 con 0x55a01924c1b0 2024-08-30T09:02:27.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.068+0000 7f854d7fa640 1 -- 172.21.15.42:0/169780967 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f85480abb70 con 0x55a01924c1b0 2024-08-30T09:02:27.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.068+0000 7f854d7fa640 1 -- 172.21.15.42:0/169780967 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85480a7f20 con 0x55a01924c1b0 2024-08-30T09:02:27.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.069+0000 7f85646ff740 1 -- 172.21.15.42:0/169780967 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a01924c1b0 msgr2=0x55a01961dad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:02:27.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.069+0000 7f85646ff740 1 --2- 172.21.15.42:0/169780967 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a01924c1b0 0x55a01961dad0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f85480e5de0 tx=0x7f8548010460 comp rx=0 tx=0).stop 2024-08-30T09:02:27.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.069+0000 7f85646ff740 1 -- 172.21.15.42:0/169780967 shutdown_connections 2024-08-30T09:02:27.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.069+0000 7f85646ff740 1 --2- 172.21.15.42:0/169780967 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a019275070 0x55a019622cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:27.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.069+0000 7f85646ff740 1 --2- 172.21.15.42:0/169780967 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a019254390 0x55a019631c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:27.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.069+0000 7f85646ff740 1 --2- 172.21.15.42:0/169780967 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a01924c1b0 0x55a01961dad0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:27.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.069+0000 7f85646ff740 1 -- 172.21.15.42:0/169780967 >> 172.21.15.42:0/169780967 conn(0x55a019269460 msgr2=0x55a01961d3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:02:27.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.069+0000 7f85646ff740 1 -- 172.21.15.42:0/169780967 shutdown_connections 2024-08-30T09:02:27.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.069+0000 7f85646ff740 1 -- 172.21.15.42:0/169780967 wait complete. 2024-08-30T09:02:27.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.070+0000 7f85646ff740 1 -- start start 2024-08-30T09:02:27.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.070+0000 7f85646ff740 1 Processor -- start 2024-08-30T09:02:27.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.071+0000 7f85646ff740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a01924c1b0 0x55a01962a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:27.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.071+0000 7f85646ff740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a019254390 0x55a01962adb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:27.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:02:26 smithi167 ceph-mon[22063]: pgmap v423: 129 pgs: 129 active+clean; 1.0 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 6.1 MiB/s wr, 660 op/s 2024-08-30T09:02:27.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.071+0000 7f85577fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a01924c1b0 0x55a01962a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:02:27.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.071+0000 7f85577fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a01924c1b0 0x55a01962a7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:43972/0 (socket says 172.21.15.42:43972) 2024-08-30T09:02:27.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.071+0000 7f854effd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a019254390 0x55a01962adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:02:27.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.071+0000 7f85577fe640 1 -- 172.21.15.42:0/3734446944 learned_addr learned my addr 172.21.15.42:0/3734446944 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:02:27.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.071+0000 7f85646ff740 1 --2- 172.21.15.42:0/3734446944 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a019275070 0x55a0196ac8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:27.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.071+0000 7f85646ff740 1 -- 172.21.15.42:0/3734446944 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f85581ace50 con 0x55a019254390 2024-08-30T09:02:27.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.071+0000 7f85646ff740 1 -- 172.21.15.42:0/3734446944 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f85581ae2c0 con 0x55a019275070 2024-08-30T09:02:27.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.072+0000 7f85646ff740 1 -- 172.21.15.42:0/3734446944 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f85581ad520 con 0x55a01924c1b0 2024-08-30T09:02:27.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.072+0000 7f85577fe640 1 -- 172.21.15.42:0/3734446944 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a019275070 msgr2=0x55a0196ac8e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-30T09:02:27.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.072+0000 7f85577fe640 1 --2- 172.21.15.42:0/3734446944 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a019275070 0x55a0196ac8e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:27.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.072+0000 7f85577fe640 1 -- 172.21.15.42:0/3734446944 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a019254390 msgr2=0x55a01962adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:02:27.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.072+0000 7f85577fe640 1 --2- 172.21.15.42:0/3734446944 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a019254390 0x55a01962adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:27.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.072+0000 7f85577fe640 1 -- 172.21.15.42:0/3734446944 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a0196acf50 con 0x55a01924c1b0 2024-08-30T09:02:27.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.072+0000 7f85646ff740 10 client.0 _ll_register_callbacks cb 0x7f8548005f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:02:27.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.072+0000 7f85577fe640 1 --2- 172.21.15.42:0/3734446944 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a01924c1b0 0x55a01962a7d0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f85480e9010 tx=0x7f8548011f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6c250d2fb876d567 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:02:27.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.072+0000 7f8536ffd640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-08-30T09:02:27.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.073+0000 7f8536ffd640 1 -- 172.21.15.42:0/3734446944 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85480a7f20 con 0x55a01924c1b0 2024-08-30T09:02:27.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.073+0000 7f85646ff740 1 -- 172.21.15.42:0/3734446944 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f85480abb70 con 0x55a01924c1b0 2024-08-30T09:02:27.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.073+0000 7f8536ffd640 1 -- 172.21.15.42:0/3734446944 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f85480a8650 con 0x55a01924c1b0 2024-08-30T09:02:27.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.073+0000 7f8536ffd640 1 -- 172.21.15.42:0/3734446944 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f85480ab7d0 con 0x55a01924c1b0 2024-08-30T09:02:27.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.073+0000 7f85646ff740 10 client.25474 fetch_fsmap learned FSMap version 29 2024-08-30T09:02:27.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.073+0000 7f85646ff740 1 -- 172.21.15.42:0/3734446944 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({fsmap=29}) -- 0x55a0196b4120 con 0x55a01924c1b0 2024-08-30T09:02:27.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.074+0000 7f8536ffd640 1 -- 172.21.15.42:0/3734446944 <== mon.1 v2:172.21.15.190:3300/0 4 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f85480cd3b0 con 0x55a01924c1b0 2024-08-30T09:02:27.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.074+0000 7f85646ff740 10 client.25474 fetch_fsmap finished waiting for FSMap version 29 2024-08-30T09:02:27.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.074+0000 7f85646ff740 10 client.25474 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-30T09:02:27.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.074+0000 7f85646ff740 20 client.25474 populate_metadata read hostname 'smithi042' 2024-08-30T09:02:27.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.074+0000 7f85646ff740 1 --2- 172.21.15.42:0/3734446944 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55a019208da0 0x55a0196b63e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:27.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.074+0000 7f85646ff740 4 client.25474 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-30T09:02:27.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.074+0000 7f85646ff740 1 -- 172.21.15.42:0/3734446944 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f85480abb70 con 0x55a019208da0 2024-08-30T09:02:27.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.074+0000 7f854effd640 1 --2- 172.21.15.42:0/3734446944 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55a019208da0 0x55a0196b63e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:02:27.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.075+0000 7f854effd640 1 --2- 172.21.15.42:0/3734446944 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55a019208da0 0x55a0196b63e0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3d0198137380db1 server_cookie=1a2e43163db44483 in_seq=0 out_seq=0 2024-08-30T09:02:27.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.075+0000 7f8536ffd640 10 client.25474 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:02:27.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.075+0000 7f8536ffd640 1 -- 172.21.15.42:0/3734446944 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+1138 (crc 0 0 0) 0x7f85480abb70 con 0x55a019208da0 2024-08-30T09:02:27.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.075+0000 7f8536ffd640 10 client.25474 handle_command_reply: tid=0 2024-08-30T09:02:27.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.075+0000 7f85646ff740 1 client.25474 shutdown 2024-08-30T09:02:27.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.075+0000 7f85646ff740 1 -- 172.21.15.42:0/3734446944 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a01924c1b0 msgr2=0x55a01962a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:02:27.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.075+0000 7f85646ff740 1 --2- 172.21.15.42:0/3734446944 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a01924c1b0 0x55a01962a7d0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f85480e9010 tx=0x7f8548011f90 comp rx=0 tx=0).stop 2024-08-30T09:02:27.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.075+0000 7f85646ff740 1 -- 172.21.15.42:0/3734446944 shutdown_connections 2024-08-30T09:02:27.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.075+0000 7f85646ff740 1 --2- 172.21.15.42:0/3734446944 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55a019208da0 0x55a0196b63e0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:27.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.075+0000 7f85646ff740 1 --2- 172.21.15.42:0/3734446944 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a019275070 0x55a0196ac8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:27.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.075+0000 7f85646ff740 1 --2- 172.21.15.42:0/3734446944 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a019254390 0x55a01962adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:27.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.075+0000 7f85646ff740 1 --2- 172.21.15.42:0/3734446944 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a01924c1b0 0x55a01962a7d0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:27.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.075+0000 7f85646ff740 1 -- 172.21.15.42:0/3734446944 >> 172.21.15.42:0/3734446944 conn(0x55a019269460 msgr2=0x55a01961b2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:02:27.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.075+0000 7f8536ffd640 0 client.25474 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:02:27.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.076+0000 7f85646ff740 1 -- 172.21.15.42:0/3734446944 shutdown_connections 2024-08-30T09:02:27.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.076+0000 7f85646ff740 1 -- 172.21.15.42:0/3734446944 wait complete. 2024-08-30T09:02:27.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.076+0000 7f85646ff740 20 client.25474 trim_cache size 0 max 16384 2024-08-30T09:02:27.087 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-08-30T09:02:27.087 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "IDLE (waiting for more scrubs)", 2024-08-30T09:02:27.087 INFO:teuthology.orchestra.run.smithi042.stdout: "scrubs": { 2024-08-30T09:02:27.087 INFO:teuthology.orchestra.run.smithi042.stdout: "c827ad7a-44ee-4e1d-9bb3-51fb7ab00c73": { 2024-08-30T09:02:27.088 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:02:27.088 INFO:teuthology.orchestra.run.smithi042.stdout: "tag": "c827ad7a-44ee-4e1d-9bb3-51fb7ab00c73", 2024-08-30T09:02:27.088 INFO:teuthology.orchestra.run.smithi042.stdout: "options": "recursive,force" 2024-08-30T09:02:27.088 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:02:27.088 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:02:27.088 INFO:teuthology.orchestra.run.smithi042.stdout: "scrub_stats": { 2024-08-30T09:02:27.088 INFO:teuthology.orchestra.run.smithi042.stdout: "85134e4b-ab4a-4885-b07e-a3ad01eaf13d": { 2024-08-30T09:02:27.088 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:01:27.738462+0000", 2024-08-30T09:02:27.088 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:02:27.088 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:02:27.089 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:02:27.089 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:02:27.089 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:02:27.089 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:02:27.089 INFO:teuthology.orchestra.run.smithi042.stdout: "c827ad7a-44ee-4e1d-9bb3-51fb7ab00c73": { 2024-08-30T09:02:27.089 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:02:21.686517+0000", 2024-08-30T09:02:27.089 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:02:27.089 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:02:27.089 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:02:27.089 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:02:27.090 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:02:27.090 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:02:27.090 INFO:teuthology.orchestra.run.smithi042.stdout: "5e2ed104-0625-478a-a78b-584e7f4c2817": { 2024-08-30T09:02:27.090 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:00:33.928400+0000", 2024-08-30T09:02:27.090 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:02:27.090 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:02:27.090 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:02:27.090 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:02:27.090 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:02:27.090 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:02:27.090 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:02:27.091 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-08-30T09:02:27.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.076+0000 7f855e18a640 1 -- 172.21.15.42:0/3705864180 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f85440b4bf0 msgr2=0x7f85440b7100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:02:27.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.076+0000 7f855e18a640 1 --2- 172.21.15.42:0/3705864180 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f85440b4bf0 0x7f85440b7100 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f855001c9b0 tx=0x7f8550005e00 comp rx=0 tx=0).stop 2024-08-30T09:02:27.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.076+0000 7f855e18a640 1 -- 172.21.15.42:0/3705864180 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8558109f30 msgr2=0x7f8558116830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:02:27.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.076+0000 7f855e18a640 1 --2- 172.21.15.42:0/3705864180 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8558109f30 0x7f8558116830 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f854800e9f0 tx=0x7f854800eeb0 comp rx=0 tx=0).stop 2024-08-30T09:02:27.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.077+0000 7f855e18a640 1 -- 172.21.15.42:0/3705864180 shutdown_connections 2024-08-30T09:02:27.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.077+0000 7f855e18a640 1 --2- 172.21.15.42:0/3705864180 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f85440b4bf0 0x7f85440b7100 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:27.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.077+0000 7f855e18a640 1 --2- 172.21.15.42:0/3705864180 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f85581101c0 0x7f8558113100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:27.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.077+0000 7f855e18a640 1 --2- 172.21.15.42:0/3705864180 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f855810f870 0x7f8558112b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:27.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.077+0000 7f855e18a640 1 --2- 172.21.15.42:0/3705864180 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8558109f30 0x7f8558116830 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:27.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.077+0000 7f855e18a640 1 -- 172.21.15.42:0/3705864180 >> 172.21.15.42:0/3705864180 conn(0x7f8558077e90 msgr2=0x7f8558108510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:02:27.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.077+0000 7f855e18a640 1 -- 172.21.15.42:0/3705864180 shutdown_connections 2024-08-30T09:02:27.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:02:27.077+0000 7f855e18a640 1 -- 172.21.15.42:0/3705864180 wait complete. 2024-08-30T09:02:27.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:02:26 smithi190 ceph-mon[22001]: pgmap v423: 129 pgs: 129 active+clean; 1.0 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 6.1 MiB/s wr, 660 op/s 2024-08-30T09:02:27.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:02:27 smithi042 ceph-mon[18652]: scrub summary: idle 2024-08-30T09:02:27.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:02:27 smithi042 ceph-mon[18652]: scrub summary: idle 2024-08-30T09:02:28.040 INFO:tasks.cephfs.filesystem:scrub status for tag:c827ad7a-44ee-4e1d-9bb3-51fb7ab00c73 - {'path': '/', 'tag': 'c827ad7a-44ee-4e1d-9bb3-51fb7ab00c73', 'options': 'recursive,force'} 2024-08-30T09:02:28.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:02:27 smithi167 ceph-mon[22063]: scrub summary: idle 2024-08-30T09:02:28.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:02:27 smithi167 ceph-mon[22063]: scrub summary: idle 2024-08-30T09:02:28.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:02:27 smithi190 ceph-mon[22001]: scrub summary: idle 2024-08-30T09:02:28.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:02:27 smithi190 ceph-mon[22001]: scrub summary: idle 2024-08-30T09:02:29.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:02:28 smithi167 ceph-mon[22063]: pgmap v424: 129 pgs: 129 active+clean; 1.0 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 6.8 MiB/s wr, 788 op/s 2024-08-30T09:02:29.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:02:28 smithi167 ceph-mon[22063]: scrub summary: idle 2024-08-30T09:02:29.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:02:28 smithi042 ceph-mon[18652]: pgmap v424: 129 pgs: 129 active+clean; 1.0 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 6.8 MiB/s wr, 788 op/s 2024-08-30T09:02:29.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:02:28 smithi042 ceph-mon[18652]: scrub summary: idle 2024-08-30T09:02:29.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:02:28 smithi190 ceph-mon[22001]: pgmap v424: 129 pgs: 129 active+clean; 1.0 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 6.8 MiB/s wr, 788 op/s 2024-08-30T09:02:29.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:02:28 smithi190 ceph-mon[22001]: scrub summary: idle 2024-08-30T09:02:30.953 INFO:teuthology.orchestra.run.smithi042.stderr:progress: 105.0 s, 1703.8 tps, lat 58.660 ms stddev 65.943 2024-08-30T09:02:31.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:02:30 smithi167 ceph-mon[22063]: pgmap v425: 129 pgs: 129 active+clean; 1.0 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 7.6 MiB/s wr, 781 op/s 2024-08-30T09:02:31.132 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:02:30 smithi042 ceph-mon[18652]: pgmap v425: 129 pgs: 129 active+clean; 1.0 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 7.6 MiB/s wr, 781 op/s 2024-08-30T09:02:31.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:02:30 smithi190 ceph-mon[22001]: pgmap v425: 129 pgs: 129 active+clean; 1.0 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 7.6 MiB/s wr, 781 op/s 2024-08-30T09:02:32.469 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.466+0000 7f660739f640 1 -- 172.21.15.42:0/1122112199 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6600102350 msgr2=0x7f66001027c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:02:32.469 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.466+0000 7f660739f640 1 --2- 172.21.15.42:0/1122112199 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6600102350 0x7f66001027c0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f65fc009900 tx=0x7f65fc02a410 comp rx=0 tx=0).stop 2024-08-30T09:02:32.469 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.466+0000 7f660739f640 1 -- 172.21.15.42:0/1122112199 shutdown_connections 2024-08-30T09:02:32.470 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.466+0000 7f660739f640 1 --2- 172.21.15.42:0/1122112199 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f66001033d0 0x7f66001037e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:32.470 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.466+0000 7f660739f640 1 --2- 172.21.15.42:0/1122112199 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6600103d70 0x7f660010a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:32.470 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.466+0000 7f660739f640 1 --2- 172.21.15.42:0/1122112199 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6600102350 0x7f66001027c0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:32.470 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.466+0000 7f660739f640 1 -- 172.21.15.42:0/1122112199 >> 172.21.15.42:0/1122112199 conn(0x7f66000fdcd0 msgr2=0x7f6600100140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:02:32.470 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.467+0000 7f660739f640 1 -- 172.21.15.42:0/1122112199 shutdown_connections 2024-08-30T09:02:32.470 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.467+0000 7f660739f640 1 -- 172.21.15.42:0/1122112199 wait complete. 2024-08-30T09:02:32.472 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.468+0000 7f660739f640 1 Processor -- start 2024-08-30T09:02:32.472 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.468+0000 7f660739f640 1 -- start start 2024-08-30T09:02:32.472 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.468+0000 7f660739f640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6600102350 0x7f660019bef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:32.472 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.469+0000 7f660739f640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f66001033d0 0x7f660019c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:32.472 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.469+0000 7f660739f640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6600103d70 0x7f660019d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:32.473 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.469+0000 7f660739f640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f660010dfd0 con 0x7f6600102350 2024-08-30T09:02:32.473 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.469+0000 7f660739f640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f660010de50 con 0x7f66001033d0 2024-08-30T09:02:32.473 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.469+0000 7f660739f640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f660010e150 con 0x7f6600103d70 2024-08-30T09:02:32.473 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.469+0000 7f660513b640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6600102350 0x7f660019bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:02:32.473 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.469+0000 7f660513b640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6600102350 0x7f660019bef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56338/0 (socket says 172.21.15.42:56338) 2024-08-30T09:02:32.473 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.469+0000 7f660493a640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f66001033d0 0x7f660019c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:02:32.473 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.469+0000 7f660513b640 1 -- 172.21.15.42:0/3291704045 learned_addr learned my addr 172.21.15.42:0/3291704045 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:02:32.473 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.469+0000 7f660593c640 1 --2- 172.21.15.42:0/3291704045 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6600103d70 0x7f660019d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:02:32.473 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.469+0000 7f660493a640 1 -- 172.21.15.42:0/3291704045 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6600103d70 msgr2=0x7f660019d110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:02:32.473 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.469+0000 7f660493a640 1 --2- 172.21.15.42:0/3291704045 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6600103d70 0x7f660019d110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:32.474 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.469+0000 7f660493a640 1 -- 172.21.15.42:0/3291704045 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6600102350 msgr2=0x7f660019bef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:02:32.474 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.469+0000 7f660493a640 1 --2- 172.21.15.42:0/3291704045 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6600102350 0x7f660019bef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:32.474 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.469+0000 7f660493a640 1 -- 172.21.15.42:0/3291704045 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f660019d930 con 0x7f66001033d0 2024-08-30T09:02:32.474 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.469+0000 7f660493a640 1 --2- 172.21.15.42:0/3291704045 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f66001033d0 0x7f660019c560 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f65f000e9f0 tx=0x7f65f000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:02:32.474 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.470+0000 7f65f67fc640 1 -- 172.21.15.42:0/3291704045 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65f000ccc0 con 0x7f66001033d0 2024-08-30T09:02:32.474 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.470+0000 7f660739f640 1 -- 172.21.15.42:0/3291704045 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f660019dc20 con 0x7f66001033d0 2024-08-30T09:02:32.474 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.470+0000 7f65f67fc640 1 -- 172.21.15.42:0/3291704045 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f65f00108e0 con 0x7f66001033d0 2024-08-30T09:02:32.474 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.470+0000 7f65f67fc640 1 -- 172.21.15.42:0/3291704045 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65f0018880 con 0x7f66001033d0 2024-08-30T09:02:32.474 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.470+0000 7f660739f640 1 -- 172.21.15.42:0/3291704045 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66001a4310 con 0x7f66001033d0 2024-08-30T09:02:32.476 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.473+0000 7f65f67fc640 1 -- 172.21.15.42:0/3291704045 <== mon.2 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7f65f0018a20 con 0x7f66001033d0 2024-08-30T09:02:32.477 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.475+0000 7f65f67fc640 1 --2- 172.21.15.42:0/3291704045 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f65ec0b4bb0 0x7f65ec0b70c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:32.477 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.475+0000 7f65f67fc640 1 -- 172.21.15.42:0/3291704045 <== mon.2 v2:172.21.15.167:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f65f00e6780 con 0x7f66001033d0 2024-08-30T09:02:32.477 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.475+0000 7f660513b640 1 --2- 172.21.15.42:0/3291704045 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f65ec0b4bb0 0x7f65ec0b70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:02:32.477 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.475+0000 7f660e3e8740 1 Processor -- start 2024-08-30T09:02:32.477 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.476+0000 7f660e3e8740 1 -- start start 2024-08-30T09:02:32.479 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.476+0000 7f660e3e8740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561ee96351b0 0x561ee99e3770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:32.479 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.476+0000 7f660e3e8740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561ee960c2f0 0x561ee99e3de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:32.479 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.476+0000 7f660e3e8740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561ee96144d0 0x561ee99e4930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:32.479 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.476+0000 7f660e3e8740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x561ee994e000 con 0x561ee96351b0 2024-08-30T09:02:32.479 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.476+0000 7f660e3e8740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x561ee993d590 con 0x561ee960c2f0 2024-08-30T09:02:32.479 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.476+0000 7f660e3e8740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x561ee9941540 con 0x561ee96144d0 2024-08-30T09:02:32.479 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.476+0000 7f660513b640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561ee96351b0 0x561ee99e3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:02:32.479 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.476+0000 7f660513b640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561ee96351b0 0x561ee99e3770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56348/0 (socket says 172.21.15.42:56348) 2024-08-30T09:02:32.479 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.476+0000 7f660493a640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561ee960c2f0 0x561ee99e3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:02:32.480 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.476+0000 7f660593c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561ee96144d0 0x561ee99e4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:02:32.480 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.476+0000 7f660593c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561ee96144d0 0x561ee99e4930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:33724/0 (socket says 172.21.15.42:33724) 2024-08-30T09:02:32.480 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.476+0000 7f660493a640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561ee960c2f0 0x561ee99e3de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:58630/0 (socket says 172.21.15.42:58630) 2024-08-30T09:02:32.480 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.476+0000 7f660593c640 1 -- 172.21.15.42:0/3202704861 learned_addr learned my addr 172.21.15.42:0/3202704861 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:02:32.480 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.476+0000 7f660513b640 1 --2- 172.21.15.42:0/3291704045 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f65ec0b4bb0 0x7f65ec0b70c0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f65fc0096c0 tx=0x7f65fc0314a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:02:32.480 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.477+0000 7f660493a640 1 -- 172.21.15.42:0/3202704861 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561ee96144d0 msgr2=0x561ee99e4930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:02:32.480 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.477+0000 7f660493a640 1 --2- 172.21.15.42:0/3202704861 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561ee96144d0 0x561ee99e4930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:32.480 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.477+0000 7f660493a640 1 -- 172.21.15.42:0/3202704861 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561ee96351b0 msgr2=0x561ee99e3770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:02:32.480 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.477+0000 7f660493a640 1 --2- 172.21.15.42:0/3202704861 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561ee96351b0 0x561ee99e3770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:32.481 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.477+0000 7f660493a640 1 -- 172.21.15.42:0/3202704861 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561ee96a0790 con 0x561ee960c2f0 2024-08-30T09:02:32.481 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.477+0000 7f660493a640 1 --2- 172.21.15.42:0/3202704861 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561ee960c2f0 0x561ee99e3de0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f65f00ace00 tx=0x7f65f000ce60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3b78ad1ff17a2b8d server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:02:32.481 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.477+0000 7f65eb7fe640 1 -- 172.21.15.42:0/3202704861 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65f00e3fa0 con 0x561ee960c2f0 2024-08-30T09:02:32.481 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.477+0000 7f65eb7fe640 1 -- 172.21.15.42:0/3202704861 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f65f00e6450 con 0x561ee960c2f0 2024-08-30T09:02:32.481 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.477+0000 7f65eb7fe640 1 -- 172.21.15.42:0/3202704861 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65f0094110 con 0x561ee960c2f0 2024-08-30T09:02:32.484 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.478+0000 7f660e3e8740 1 -- 172.21.15.42:0/3202704861 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561ee960c2f0 msgr2=0x561ee99e3de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:02:32.484 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.478+0000 7f660e3e8740 1 --2- 172.21.15.42:0/3202704861 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561ee960c2f0 0x561ee99e3de0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f65f00ace00 tx=0x7f65f000ce60 comp rx=0 tx=0).stop 2024-08-30T09:02:32.484 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.478+0000 7f660e3e8740 1 -- 172.21.15.42:0/3202704861 shutdown_connections 2024-08-30T09:02:32.484 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.478+0000 7f660e3e8740 1 --2- 172.21.15.42:0/3202704861 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561ee96144d0 0x561ee99e4930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:32.484 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.478+0000 7f660e3e8740 1 --2- 172.21.15.42:0/3202704861 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561ee960c2f0 0x561ee99e3de0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:32.484 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.478+0000 7f660e3e8740 1 --2- 172.21.15.42:0/3202704861 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561ee96351b0 0x561ee99e3770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:32.485 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.478+0000 7f660e3e8740 1 -- 172.21.15.42:0/3202704861 >> 172.21.15.42:0/3202704861 conn(0x561ee9629300 msgr2=0x561ee99df200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:02:32.485 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.478+0000 7f660e3e8740 1 -- 172.21.15.42:0/3202704861 shutdown_connections 2024-08-30T09:02:32.485 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.478+0000 7f660e3e8740 1 -- 172.21.15.42:0/3202704861 wait complete. 2024-08-30T09:02:32.485 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.478+0000 7f660e3e8740 1 -- start start 2024-08-30T09:02:32.485 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.478+0000 7f660e3e8740 1 Processor -- start 2024-08-30T09:02:32.485 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.479+0000 7f660e3e8740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561ee960c2f0 0x561ee99ed2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:32.485 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.479+0000 7f660e3e8740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561ee96144d0 0x561ee99ed970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:32.485 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.479+0000 7f660e3e8740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561ee96351b0 0x561ee99ee4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:32.485 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.479+0000 7f660e3e8740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x561ee9876ab0 con 0x561ee96144d0 2024-08-30T09:02:32.485 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.479+0000 7f660e3e8740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x561ee98c4680 con 0x561ee960c2f0 2024-08-30T09:02:32.485 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.479+0000 7f660e3e8740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x561ee99e5450 con 0x561ee96351b0 2024-08-30T09:02:32.485 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.479+0000 7f660593c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561ee96351b0 0x561ee99ee4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:02:32.486 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.479+0000 7f660593c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561ee96351b0 0x561ee99ee4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:33728/0 (socket says 172.21.15.42:33728) 2024-08-30T09:02:32.486 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.479+0000 7f660593c640 1 -- 172.21.15.42:0/2751571401 learned_addr learned my addr 172.21.15.42:0/2751571401 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:02:32.486 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.479+0000 7f660493a640 1 --2- 172.21.15.42:0/2751571401 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561ee96144d0 0x561ee99ed970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:02:32.486 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.479+0000 7f660513b640 1 --2- 172.21.15.42:0/2751571401 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561ee960c2f0 0x561ee99ed2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:02:32.486 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.479+0000 7f660593c640 1 -- 172.21.15.42:0/2751571401 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561ee960c2f0 msgr2=0x561ee99ed2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:02:32.486 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.479+0000 7f660593c640 1 --2- 172.21.15.42:0/2751571401 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561ee960c2f0 0x561ee99ed2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:32.486 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.479+0000 7f660593c640 1 -- 172.21.15.42:0/2751571401 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561ee96144d0 msgr2=0x561ee99ed970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:02:32.486 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.479+0000 7f660593c640 1 --2- 172.21.15.42:0/2751571401 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561ee96144d0 0x561ee99ed970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:32.486 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.479+0000 7f660593c640 1 -- 172.21.15.42:0/2751571401 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561ee9719620 con 0x561ee96351b0 2024-08-30T09:02:32.486 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.480+0000 7f660e3e8740 10 client.0 _ll_register_callbacks cb 0x561ee964ca00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:02:32.486 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.480+0000 7f660593c640 1 --2- 172.21.15.42:0/2751571401 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561ee96351b0 0x561ee99ee4e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f65f8011d10 tx=0x7f65f800f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=76e633b9ab04e3f9 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:02:32.486 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.480+0000 7f65e8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-08-30T09:02:32.486 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.480+0000 7f660e3e8740 1 -- 172.21.15.42:0/2751571401 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f65f00e6450 con 0x561ee96351b0 2024-08-30T09:02:32.486 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.480+0000 7f65e8ff9640 1 -- 172.21.15.42:0/2751571401 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65f801f060 con 0x561ee96351b0 2024-08-30T09:02:32.487 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.480+0000 7f65e8ff9640 1 -- 172.21.15.42:0/2751571401 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f65f8017050 con 0x561ee96351b0 2024-08-30T09:02:32.487 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.480+0000 7f65e8ff9640 1 -- 172.21.15.42:0/2751571401 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f65f801a430 con 0x561ee96351b0 2024-08-30T09:02:32.487 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.480+0000 7f660e3e8740 10 client.25486 fetch_fsmap learned FSMap version 29 2024-08-30T09:02:32.487 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.480+0000 7f660e3e8740 1 -- 172.21.15.42:0/2751571401 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({fsmap=29}) -- 0x561ee99f73c0 con 0x561ee96351b0 2024-08-30T09:02:32.487 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.481+0000 7f65e8ff9640 1 -- 172.21.15.42:0/2751571401 <== mon.1 v2:172.21.15.190:3300/0 4 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f65f8019070 con 0x561ee96351b0 2024-08-30T09:02:32.487 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.481+0000 7f660e3e8740 10 client.25486 fetch_fsmap finished waiting for FSMap version 29 2024-08-30T09:02:32.487 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.481+0000 7f660e3e8740 10 client.25486 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-30T09:02:32.487 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.481+0000 7f660e3e8740 20 client.25486 populate_metadata read hostname 'smithi042' 2024-08-30T09:02:32.487 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.481+0000 7f660e3e8740 1 --2- 172.21.15.42:0/2751571401 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x561ee95f3d80 0x561ee99faaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:32.487 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.481+0000 7f660e3e8740 4 client.25486 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-30T09:02:32.487 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.481+0000 7f660e3e8740 1 -- 172.21.15.42:0/2751571401 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561ee994ff50 con 0x561ee95f3d80 2024-08-30T09:02:32.487 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.482+0000 7f660513b640 1 --2- 172.21.15.42:0/2751571401 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x561ee95f3d80 0x561ee99faaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:02:32.487 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.482+0000 7f660513b640 1 --2- 172.21.15.42:0/2751571401 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x561ee95f3d80 0x561ee99faaf0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cd9a931a87f8788c server_cookie=ef83964791f3cefc in_seq=0 out_seq=0 2024-08-30T09:02:32.487 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.482+0000 7f65e8ff9640 10 client.25486 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:02:32.489 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.483+0000 7f65e8ff9640 1 -- 172.21.15.42:0/2751571401 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x561ee994ff50 con 0x561ee95f3d80 2024-08-30T09:02:32.489 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.483+0000 7f65e8ff9640 10 client.25486 handle_command_reply: tid=0 2024-08-30T09:02:32.489 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.483+0000 7f660e3e8740 1 client.25486 shutdown 2024-08-30T09:02:32.489 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.483+0000 7f660e3e8740 1 -- 172.21.15.42:0/2751571401 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561ee96351b0 msgr2=0x561ee99ee4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:02:32.489 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.483+0000 7f660e3e8740 1 --2- 172.21.15.42:0/2751571401 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561ee96351b0 0x561ee99ee4e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f65f8011d10 tx=0x7f65f800f790 comp rx=0 tx=0).stop 2024-08-30T09:02:32.489 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.483+0000 7f660e3e8740 1 -- 172.21.15.42:0/2751571401 shutdown_connections 2024-08-30T09:02:32.489 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.483+0000 7f660e3e8740 1 --2- 172.21.15.42:0/2751571401 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x561ee95f3d80 0x561ee99faaf0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:32.489 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.483+0000 7f660e3e8740 1 --2- 172.21.15.42:0/2751571401 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561ee96351b0 0x561ee99ee4e0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:32.489 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.483+0000 7f660e3e8740 1 --2- 172.21.15.42:0/2751571401 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561ee96144d0 0x561ee99ed970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:32.489 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.483+0000 7f660e3e8740 1 --2- 172.21.15.42:0/2751571401 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561ee960c2f0 0x561ee99ed2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:32.489 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.483+0000 7f660e3e8740 1 -- 172.21.15.42:0/2751571401 >> 172.21.15.42:0/2751571401 conn(0x561ee9629300 msgr2=0x561ee99e47e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:02:32.489 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.484+0000 7f65e8ff9640 0 client.25486 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:02:32.490 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.484+0000 7f660e3e8740 1 -- 172.21.15.42:0/2751571401 shutdown_connections 2024-08-30T09:02:32.490 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.484+0000 7f660e3e8740 1 -- 172.21.15.42:0/2751571401 wait complete. 2024-08-30T09:02:32.490 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.484+0000 7f660e3e8740 20 client.25486 trim_cache size 0 max 16384 2024-08-30T09:02:32.510 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.507+0000 7f660e3e8740 1 Processor -- start 2024-08-30T09:02:32.510 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.507+0000 7f660e3e8740 1 -- start start 2024-08-30T09:02:32.510 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.507+0000 7f660e3e8740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561ee960c2f0 0x561ee99e0850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:32.510 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.507+0000 7f660e3e8740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561ee96144d0 0x561ee99ec4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:32.511 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.507+0000 7f660e3e8740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561ee96351b0 0x561ee99e7320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:32.511 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.507+0000 7f660e3e8740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f66001acc00 con 0x561ee96351b0 2024-08-30T09:02:32.511 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.507+0000 7f660e3e8740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f66001aa4d0 con 0x561ee96144d0 2024-08-30T09:02:32.511 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.507+0000 7f660e3e8740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f66001a8320 con 0x561ee960c2f0 2024-08-30T09:02:32.514 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.510+0000 7f660593c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561ee96351b0 0x561ee99e7320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:02:32.514 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.510+0000 7f660513b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561ee960c2f0 0x561ee99e0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:02:32.514 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.510+0000 7f660513b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561ee960c2f0 0x561ee99e0850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:33740/0 (socket says 172.21.15.42:33740) 2024-08-30T09:02:32.514 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.510+0000 7f660593c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561ee96351b0 0x561ee99e7320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56362/0 (socket says 172.21.15.42:56362) 2024-08-30T09:02:32.514 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.510+0000 7f660513b640 1 -- 172.21.15.42:0/3042489260 learned_addr learned my addr 172.21.15.42:0/3042489260 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:02:32.515 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.510+0000 7f660593c640 1 -- 172.21.15.42:0/3042489260 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561ee960c2f0 msgr2=0x561ee99e0850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:02:32.515 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.510+0000 7f660593c640 1 --2- 172.21.15.42:0/3042489260 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561ee960c2f0 0x561ee99e0850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:32.515 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.510+0000 7f660593c640 1 -- 172.21.15.42:0/3042489260 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561ee96144d0 msgr2=0x561ee99ec4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:02:32.515 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.510+0000 7f660593c640 1 --2- 172.21.15.42:0/3042489260 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561ee96144d0 0x561ee99ec4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:32.515 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.510+0000 7f660593c640 1 -- 172.21.15.42:0/3042489260 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561ee99eea40 con 0x561ee96351b0 2024-08-30T09:02:32.515 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.511+0000 7f660593c640 1 --2- 172.21.15.42:0/3042489260 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561ee96351b0 0x561ee99e7320 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f65f800ffd0 tx=0x7f65f8018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da5aa99302feadf5 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:02:32.515 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.511+0000 7f65eaffd640 1 -- 172.21.15.42:0/3042489260 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65f8031070 con 0x561ee96351b0 2024-08-30T09:02:32.515 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.512+0000 7f65eaffd640 1 -- 172.21.15.42:0/3042489260 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f65f00e6450 con 0x561ee96351b0 2024-08-30T09:02:32.518 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.513+0000 7f660e3e8740 1 -- 172.21.15.42:0/3042489260 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561ee96351b0 msgr2=0x561ee99e7320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:02:32.518 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.513+0000 7f660e3e8740 1 --2- 172.21.15.42:0/3042489260 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561ee96351b0 0x561ee99e7320 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f65f800ffd0 tx=0x7f65f8018000 comp rx=0 tx=0).stop 2024-08-30T09:02:32.518 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.513+0000 7f660e3e8740 1 -- 172.21.15.42:0/3042489260 shutdown_connections 2024-08-30T09:02:32.518 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.513+0000 7f660e3e8740 1 --2- 172.21.15.42:0/3042489260 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561ee96351b0 0x561ee99e7320 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:32.518 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.513+0000 7f660e3e8740 1 --2- 172.21.15.42:0/3042489260 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561ee96144d0 0x561ee99ec4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:32.518 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.513+0000 7f660e3e8740 1 --2- 172.21.15.42:0/3042489260 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561ee960c2f0 0x561ee99e0850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:32.519 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.513+0000 7f660e3e8740 1 -- 172.21.15.42:0/3042489260 >> 172.21.15.42:0/3042489260 conn(0x561ee9629300 msgr2=0x561ee99e4600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:02:32.519 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.517+0000 7f660e3e8740 1 -- 172.21.15.42:0/3042489260 shutdown_connections 2024-08-30T09:02:32.520 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.518+0000 7f660e3e8740 1 -- 172.21.15.42:0/3042489260 wait complete. 2024-08-30T09:02:32.520 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.518+0000 7f660e3e8740 1 -- start start 2024-08-30T09:02:32.520 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.518+0000 7f660e3e8740 1 Processor -- start 2024-08-30T09:02:32.521 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.519+0000 7f660e3e8740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561ee960c2f0 0x561ee99e6520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:32.521 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.519+0000 7f660e3e8740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561ee96144d0 0x561ee99e6ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:32.521 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.519+0000 7f660e3e8740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561ee95c91c0 0x561ee99d61c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:32.521 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.519+0000 7f660e3e8740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f66001a8990 con 0x561ee960c2f0 2024-08-30T09:02:32.521 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.519+0000 7f660e3e8740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f66001a9e00 con 0x561ee96144d0 2024-08-30T09:02:32.521 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.519+0000 7f660e3e8740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f66001a9060 con 0x561ee95c91c0 2024-08-30T09:02:32.521 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.519+0000 7f660e3e8740 10 client.0 _ll_register_callbacks cb 0x7f65f8009cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:02:32.522 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.519+0000 7f660493a640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561ee96144d0 0x561ee99e6ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:02:32.522 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.519+0000 7f660493a640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561ee96144d0 0x561ee99e6ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:58644/0 (socket says 172.21.15.42:58644) 2024-08-30T09:02:32.522 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.519+0000 7f660493a640 1 -- 172.21.15.42:0/45472969 learned_addr learned my addr 172.21.15.42:0/45472969 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:02:32.522 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.520+0000 7f660593c640 1 --2- 172.21.15.42:0/45472969 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561ee95c91c0 0x561ee99d61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:02:32.522 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.520+0000 7f660513b640 1 --2- 172.21.15.42:0/45472969 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561ee960c2f0 0x561ee99e6520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:02:32.522 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.520+0000 7f660493a640 1 -- 172.21.15.42:0/45472969 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561ee95c91c0 msgr2=0x561ee99d61c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:02:32.522 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.520+0000 7f660493a640 1 --2- 172.21.15.42:0/45472969 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561ee95c91c0 0x561ee99d61c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:32.522 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.520+0000 7f660493a640 1 -- 172.21.15.42:0/45472969 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561ee960c2f0 msgr2=0x561ee99e6520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:02:32.522 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.520+0000 7f660493a640 1 --2- 172.21.15.42:0/45472969 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561ee960c2f0 0x561ee99e6520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:32.523 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.520+0000 7f660493a640 1 -- 172.21.15.42:0/45472969 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561ee99e71f0 con 0x561ee96144d0 2024-08-30T09:02:32.523 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.520+0000 7f660513b640 1 --2- 172.21.15.42:0/45472969 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561ee960c2f0 0x561ee99e6520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-30T09:02:32.523 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.521+0000 7f660493a640 1 --2- 172.21.15.42:0/45472969 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561ee96144d0 0x561ee99e6ae0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f65f00ad030 tx=0x7f65f00b1550 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2932db70ef02031 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:02:32.525 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.522+0000 7f65e8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-30T09:02:32.525 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.522+0000 7f65e8ff9640 1 -- 172.21.15.42:0/45472969 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65f0098070 con 0x561ee96144d0 2024-08-30T09:02:32.525 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.522+0000 7f65e8ff9640 1 -- 172.21.15.42:0/45472969 <== mon.2 v2:172.21.15.167:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f65f0098210 con 0x561ee96144d0 2024-08-30T09:02:32.525 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.523+0000 7f660e3e8740 1 -- 172.21.15.42:0/45472969 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x561ee994b9f0 con 0x561ee96144d0 2024-08-30T09:02:32.528 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.526+0000 7f65e8ff9640 1 -- 172.21.15.42:0/45472969 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f65f00a9170 con 0x561ee96144d0 2024-08-30T09:02:32.528 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.527+0000 7f660e3e8740 10 client.25484 fetch_fsmap learned FSMap version 29 2024-08-30T09:02:32.529 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.527+0000 7f660e3e8740 1 -- 172.21.15.42:0/45472969 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x561ee99d76c0 con 0x561ee96144d0 2024-08-30T09:02:32.532 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.531+0000 7f65e8ff9640 1 -- 172.21.15.42:0/45472969 <== mon.2 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x561ee994b9f0 con 0x561ee96144d0 2024-08-30T09:02:32.534 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.532+0000 7f65e8ff9640 1 -- 172.21.15.42:0/45472969 <== mon.2 v2:172.21.15.167:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f65f00c7b70 con 0x561ee96144d0 2024-08-30T09:02:32.537 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.536+0000 7f660e3e8740 10 client.25484 fetch_fsmap finished waiting for FSMap version 29 2024-08-30T09:02:32.538 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.536+0000 7f660e3e8740 10 client.25484 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-30T09:02:32.538 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.536+0000 7f660e3e8740 20 client.25484 populate_metadata read hostname 'smithi042' 2024-08-30T09:02:32.538 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.536+0000 7f660e3e8740 1 --2- 172.21.15.42:0/45472969 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x561ee99d7a40 0x561ee99d9e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:32.538 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.536+0000 7f660e3e8740 4 client.25484 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-30T09:02:32.538 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.536+0000 7f660e3e8740 1 -- 172.21.15.42:0/45472969 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561ee9949130 con 0x561ee99d7a40 2024-08-30T09:02:32.538 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.536+0000 7f660513b640 1 --2- 172.21.15.42:0/45472969 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x561ee99d7a40 0x561ee99d9e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:02:32.539 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.537+0000 7f65e8ff9640 10 client.25484 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:02:32.539 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.537+0000 7f660513b640 1 --2- 172.21.15.42:0/45472969 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x561ee99d7a40 0x561ee99d9e60 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b9bf56a91aaa9538 server_cookie=b58b858a5fad40f2 in_seq=0 out_seq=0 2024-08-30T09:02:32.562 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.560+0000 7f65e8ff9640 1 -- 172.21.15.42:0/45472969 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561ee9949130 con 0x561ee99d7a40 2024-08-30T09:02:32.563 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.561+0000 7f65e8ff9640 10 client.25484 handle_command_reply: tid=0 2024-08-30T09:02:32.563 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.561+0000 7f660e3e8740 1 client.25484 shutdown 2024-08-30T09:02:32.565 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.562+0000 7f660e3e8740 1 -- 172.21.15.42:0/45472969 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561ee96144d0 msgr2=0x561ee99e6ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:02:32.565 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.563+0000 7f660e3e8740 1 --2- 172.21.15.42:0/45472969 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561ee96144d0 0x561ee99e6ae0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f65f00ad030 tx=0x7f65f00b1550 comp rx=0 tx=0).stop 2024-08-30T09:02:32.565 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.563+0000 7f660e3e8740 1 -- 172.21.15.42:0/45472969 shutdown_connections 2024-08-30T09:02:32.565 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.563+0000 7f660e3e8740 1 --2- 172.21.15.42:0/45472969 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x561ee99d7a40 0x561ee99d9e60 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:32.565 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.563+0000 7f660e3e8740 1 --2- 172.21.15.42:0/45472969 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561ee95c91c0 0x561ee99d61c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:32.565 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.563+0000 7f660e3e8740 1 --2- 172.21.15.42:0/45472969 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561ee96144d0 0x561ee99e6ae0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:32.565 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.563+0000 7f660e3e8740 1 --2- 172.21.15.42:0/45472969 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561ee960c2f0 0x561ee99e6520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:32.565 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.563+0000 7f660e3e8740 1 -- 172.21.15.42:0/45472969 >> 172.21.15.42:0/45472969 conn(0x561ee9629300 msgr2=0x561ee99e4600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:02:32.565 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.563+0000 7f65e8ff9640 0 client.25484 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:02:32.565 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.563+0000 7f660e3e8740 1 -- 172.21.15.42:0/45472969 shutdown_connections 2024-08-30T09:02:32.566 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.563+0000 7f660e3e8740 1 -- 172.21.15.42:0/45472969 wait complete. 2024-08-30T09:02:32.566 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.564+0000 7f660e3e8740 20 client.25484 trim_cache size 0 max 16384 2024-08-30T09:02:32.566 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-08-30T09:02:32.566 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2024-08-30T09:02:32.566 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-08-30T09:02:32.566 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-08-30T09:02:32.567 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.564+0000 7f660739f640 1 -- 172.21.15.42:0/3291704045 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f65ec0b4bb0 msgr2=0x7f65ec0b70c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:02:32.567 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.564+0000 7f660739f640 1 --2- 172.21.15.42:0/3291704045 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f65ec0b4bb0 0x7f65ec0b70c0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f65fc0096c0 tx=0x7f65fc0314a0 comp rx=0 tx=0).stop 2024-08-30T09:02:32.567 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.564+0000 7f660739f640 1 -- 172.21.15.42:0/3291704045 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f66001033d0 msgr2=0x7f660019c560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:02:32.567 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.564+0000 7f660739f640 1 --2- 172.21.15.42:0/3291704045 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f66001033d0 0x7f660019c560 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f65f000e9f0 tx=0x7f65f000eeb0 comp rx=0 tx=0).stop 2024-08-30T09:02:32.567 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.564+0000 7f660739f640 1 -- 172.21.15.42:0/3291704045 shutdown_connections 2024-08-30T09:02:32.567 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.564+0000 7f660739f640 1 --2- 172.21.15.42:0/3291704045 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f65ec0b4bb0 0x7f65ec0b70c0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:32.567 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.564+0000 7f660739f640 1 --2- 172.21.15.42:0/3291704045 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6600103d70 0x7f660019d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:32.568 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.564+0000 7f660739f640 1 --2- 172.21.15.42:0/3291704045 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f66001033d0 0x7f660019c560 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:32.568 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.564+0000 7f660739f640 1 --2- 172.21.15.42:0/3291704045 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6600102350 0x7f660019bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:32.568 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.564+0000 7f660739f640 1 -- 172.21.15.42:0/3291704045 >> 172.21.15.42:0/3291704045 conn(0x7f66000fdcd0 msgr2=0x7f66001000f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:02:32.568 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.564+0000 7f660739f640 1 -- 172.21.15.42:0/3291704045 shutdown_connections 2024-08-30T09:02:32.568 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:32.565+0000 7f660739f640 1 -- 172.21.15.42:0/3291704045 wait complete. 2024-08-30T09:02:33.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:02:32 smithi167 ceph-mon[22063]: pgmap v426: 129 pgs: 129 active+clean; 1.0 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 7.9 MiB/s wr, 760 op/s 2024-08-30T09:02:33.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:02:32 smithi042 ceph-mon[18652]: pgmap v426: 129 pgs: 129 active+clean; 1.0 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 7.9 MiB/s wr, 760 op/s 2024-08-30T09:02:33.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:02:32 smithi190 ceph-mon[22001]: pgmap v426: 129 pgs: 129 active+clean; 1.0 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 7.9 MiB/s wr, 760 op/s 2024-08-30T09:02:35.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:02:34 smithi167 ceph-mon[22063]: pgmap v427: 129 pgs: 129 active+clean; 1.0 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 10 MiB/s wr, 886 op/s 2024-08-30T09:02:35.132 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:02:34 smithi042 ceph-mon[18652]: pgmap v427: 129 pgs: 129 active+clean; 1.0 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 10 MiB/s wr, 886 op/s 2024-08-30T09:02:35.185 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:02:34 smithi190 ceph-mon[22001]: pgmap v427: 129 pgs: 129 active+clean; 1.0 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 10 MiB/s wr, 886 op/s 2024-08-30T09:02:35.953 INFO:teuthology.orchestra.run.smithi042.stderr:progress: 110.0 s, 1825.6 tps, lat 54.650 ms stddev 63.061 2024-08-30T09:02:37.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:02:36 smithi167 ceph-mon[22063]: pgmap v428: 129 pgs: 129 active+clean; 1.0 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 9.3 MiB/s wr, 759 op/s 2024-08-30T09:02:37.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:02:36 smithi042 ceph-mon[18652]: pgmap v428: 129 pgs: 129 active+clean; 1.0 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 9.3 MiB/s wr, 759 op/s 2024-08-30T09:02:37.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:02:36 smithi190 ceph-mon[22001]: pgmap v428: 129 pgs: 129 active+clean; 1.0 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 9.3 MiB/s wr, 759 op/s 2024-08-30T09:02:39.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:02:38 smithi167 ceph-mon[22063]: pgmap v429: 129 pgs: 129 active+clean; 1.0 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 9.4 MiB/s wr, 778 op/s 2024-08-30T09:02:39.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:02:38 smithi042 ceph-mon[18652]: pgmap v429: 129 pgs: 129 active+clean; 1.0 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 9.4 MiB/s wr, 778 op/s 2024-08-30T09:02:39.185 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:02:38 smithi190 ceph-mon[22001]: pgmap v429: 129 pgs: 129 active+clean; 1.0 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 9.4 MiB/s wr, 778 op/s 2024-08-30T09:02:40.953 INFO:teuthology.orchestra.run.smithi042.stderr:progress: 115.0 s, 2401.2 tps, lat 42.002 ms stddev 49.287 2024-08-30T09:02:41.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:02:40 smithi167 ceph-mon[22063]: pgmap v430: 129 pgs: 129 active+clean; 1.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 MiB/s wr, 762 op/s 2024-08-30T09:02:41.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:02:40 smithi042 ceph-mon[18652]: pgmap v430: 129 pgs: 129 active+clean; 1.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 MiB/s wr, 762 op/s 2024-08-30T09:02:41.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:02:40 smithi190 ceph-mon[22001]: pgmap v430: 129 pgs: 129 active+clean; 1.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 MiB/s wr, 762 op/s 2024-08-30T09:02:43.068 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:02:42 smithi042 ceph-mon[18652]: pgmap v431: 129 pgs: 129 active+clean; 1.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.8 MiB/s wr, 618 op/s 2024-08-30T09:02:43.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:02:42 smithi167 ceph-mon[22063]: pgmap v431: 129 pgs: 129 active+clean; 1.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.8 MiB/s wr, 618 op/s 2024-08-30T09:02:43.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:02:42 smithi190 ceph-mon[22001]: pgmap v431: 129 pgs: 129 active+clean; 1.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.8 MiB/s wr, 618 op/s 2024-08-30T09:02:45.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:02:44 smithi167 ceph-mon[22063]: pgmap v432: 129 pgs: 129 active+clean; 1.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 9.2 MiB/s wr, 793 op/s 2024-08-30T09:02:45.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:02:44 smithi042 ceph-mon[18652]: pgmap v432: 129 pgs: 129 active+clean; 1.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 9.2 MiB/s wr, 793 op/s 2024-08-30T09:02:45.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:02:44 smithi190 ceph-mon[22001]: pgmap v432: 129 pgs: 129 active+clean; 1.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 9.2 MiB/s wr, 793 op/s 2024-08-30T09:02:45.712 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.710+0000 7fbee9607640 1 -- 172.21.15.42:0/2687902723 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbee4102a40 msgr2=0x7fbee4104e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:02:45.713 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.710+0000 7fbee9607640 1 --2- 172.21.15.42:0/2687902723 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbee4102a40 0x7fbee4104e60 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fbed400d960 tx=0x7fbed402e4b0 comp rx=0 tx=0).stop 2024-08-30T09:02:45.713 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.710+0000 7fbee9607640 1 -- 172.21.15.42:0/2687902723 shutdown_connections 2024-08-30T09:02:45.713 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.710+0000 7fbee9607640 1 --2- 172.21.15.42:0/2687902723 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbee41080a0 0x7fbee410a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:45.713 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.710+0000 7fbee9607640 1 --2- 172.21.15.42:0/2687902723 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbee4105480 0x7fbee4107960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:45.713 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.710+0000 7fbee9607640 1 --2- 172.21.15.42:0/2687902723 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbee4102a40 0x7fbee4104e60 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:45.713 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.710+0000 7fbee9607640 1 -- 172.21.15.42:0/2687902723 >> 172.21.15.42:0/2687902723 conn(0x7fbee40fd430 msgr2=0x7fbee40ff8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:02:45.713 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.710+0000 7fbee9607640 1 -- 172.21.15.42:0/2687902723 shutdown_connections 2024-08-30T09:02:45.713 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.711+0000 7fbee9607640 1 -- 172.21.15.42:0/2687902723 wait complete. 2024-08-30T09:02:45.714 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.711+0000 7fbee9607640 1 Processor -- start 2024-08-30T09:02:45.714 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.712+0000 7fbee9607640 1 -- start start 2024-08-30T09:02:45.715 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.712+0000 7fbee9607640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbee4102a40 0x7fbee4074860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:45.715 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.712+0000 7fbee9607640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbee4105480 0x7fbee4073120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:45.715 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.712+0000 7fbee9607640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbee41080a0 0x7fbee4073660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:45.715 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.712+0000 7fbee9607640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fbee406d440 con 0x7fbee41080a0 2024-08-30T09:02:45.715 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.712+0000 7fbee9607640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fbee406d2c0 con 0x7fbee4102a40 2024-08-30T09:02:45.715 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.712+0000 7fbee9607640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fbee406d5c0 con 0x7fbee4105480 2024-08-30T09:02:45.715 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.712+0000 7fbee37fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbee41080a0 0x7fbee4073660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:02:45.715 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.712+0000 7fbee37fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbee41080a0 0x7fbee4073660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:55102/0 (socket says 172.21.15.42:55102) 2024-08-30T09:02:45.715 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.712+0000 7fbee37fe640 1 -- 172.21.15.42:0/4191108543 learned_addr learned my addr 172.21.15.42:0/4191108543 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:02:45.715 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.712+0000 7fbee2ffd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbee4102a40 0x7fbee4074860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:02:45.715 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.712+0000 7fbee27fc640 1 --2- 172.21.15.42:0/4191108543 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbee4105480 0x7fbee4073120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:02:45.716 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.713+0000 7fbee2ffd640 1 -- 172.21.15.42:0/4191108543 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbee4105480 msgr2=0x7fbee4073120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:02:45.717 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.713+0000 7fbee2ffd640 1 --2- 172.21.15.42:0/4191108543 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbee4105480 0x7fbee4073120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:45.717 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.713+0000 7fbee2ffd640 1 -- 172.21.15.42:0/4191108543 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbee41080a0 msgr2=0x7fbee4073660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:02:45.717 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.713+0000 7fbee2ffd640 1 --2- 172.21.15.42:0/4191108543 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbee41080a0 0x7fbee4073660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:45.717 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.713+0000 7fbee2ffd640 1 -- 172.21.15.42:0/4191108543 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbee4073bf0 con 0x7fbee4102a40 2024-08-30T09:02:45.717 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.713+0000 7fbee2ffd640 1 --2- 172.21.15.42:0/4191108543 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbee4102a40 0x7fbee4074860 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fbed400d880 tx=0x7fbed4005040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:02:45.717 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.714+0000 7fbedbfff640 1 -- 172.21.15.42:0/4191108543 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbed4044070 con 0x7fbee4102a40 2024-08-30T09:02:45.717 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.714+0000 7fbedbfff640 1 -- 172.21.15.42:0/4191108543 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbed40055c0 con 0x7fbee4102a40 2024-08-30T09:02:45.717 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.714+0000 7fbedbfff640 1 -- 172.21.15.42:0/4191108543 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbed4004da0 con 0x7fbee4102a40 2024-08-30T09:02:45.717 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.714+0000 7fbee9607640 1 -- 172.21.15.42:0/4191108543 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbee4073e80 con 0x7fbee4102a40 2024-08-30T09:02:45.717 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.714+0000 7fbee9607640 1 -- 172.21.15.42:0/4191108543 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbee41b9580 con 0x7fbee4102a40 2024-08-30T09:02:45.724 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.720+0000 7fbef0651740 1 Processor -- start 2024-08-30T09:02:45.724 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.721+0000 7fbedbfff640 1 -- 172.21.15.42:0/4191108543 <== mon.2 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7fbed403d040 con 0x7fbee4102a40 2024-08-30T09:02:45.724 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.721+0000 7fbef0651740 1 -- start start 2024-08-30T09:02:45.728 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.726+0000 7fbef0651740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cd686781b0 0x55cd68a1df60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:45.728 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.726+0000 7fbef0651740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cd6864f2f0 0x55cd68a1e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:45.728 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.726+0000 7fbef0651740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cd686574d0 0x55cd68a1f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:45.728 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.726+0000 7fbef0651740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55cd68991000 con 0x55cd686781b0 2024-08-30T09:02:45.728 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.726+0000 7fbef0651740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55cd68980590 con 0x55cd6864f2f0 2024-08-30T09:02:45.728 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.726+0000 7fbef0651740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55cd68984540 con 0x55cd686574d0 2024-08-30T09:02:45.729 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.727+0000 7fbee27fc640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cd686574d0 0x55cd68a1f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:02:45.729 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.727+0000 7fbee37fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cd686781b0 0x55cd68a1df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:02:45.729 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.727+0000 7fbee37fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cd686781b0 0x55cd68a1df60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:55116/0 (socket says 172.21.15.42:55116) 2024-08-30T09:02:45.735 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.727+0000 7fbee37fe640 1 -- 172.21.15.42:0/3088667375 learned_addr learned my addr 172.21.15.42:0/3088667375 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:02:45.744 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.738+0000 7fbee2ffd640 1 --2- 172.21.15.42:0/3088667375 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cd6864f2f0 0x55cd68a1e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:02:45.744 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.739+0000 7fbee2ffd640 1 -- 172.21.15.42:0/3088667375 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cd686574d0 msgr2=0x55cd68a1f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:02:45.744 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.739+0000 7fbee2ffd640 1 --2- 172.21.15.42:0/3088667375 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cd686574d0 0x55cd68a1f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:45.744 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.739+0000 7fbee2ffd640 1 -- 172.21.15.42:0/3088667375 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cd686781b0 msgr2=0x55cd68a1df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:02:45.745 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.739+0000 7fbee2ffd640 1 --2- 172.21.15.42:0/3088667375 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cd686781b0 0x55cd68a1df60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:45.745 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.739+0000 7fbee2ffd640 1 -- 172.21.15.42:0/3088667375 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cd686e3790 con 0x55cd6864f2f0 2024-08-30T09:02:45.745 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.739+0000 7fbee2ffd640 1 --2- 172.21.15.42:0/3088667375 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cd6864f2f0 0x55cd68a1e5d0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fbed4104130 tx=0x7fbed402e4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a6525109b346010a server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:02:45.745 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.740+0000 7fbedbfff640 1 --2- 172.21.15.42:0/4191108543 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fbed00b4d40 0x7fbed00b7250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:45.745 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.740+0000 7fbedbfff640 1 -- 172.21.15.42:0/4191108543 <== mon.2 v2:172.21.15.167:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7fbed4106a80 con 0x7fbee4102a40 2024-08-30T09:02:45.747 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.744+0000 7fbed97fa640 1 -- 172.21.15.42:0/3088667375 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbed410ee10 con 0x55cd6864f2f0 2024-08-30T09:02:45.748 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.744+0000 7fbed97fa640 1 -- 172.21.15.42:0/3088667375 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbed4109580 con 0x55cd6864f2f0 2024-08-30T09:02:45.748 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.745+0000 7fbef0651740 1 -- 172.21.15.42:0/3088667375 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cd6864f2f0 msgr2=0x55cd68a1e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:02:45.748 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.745+0000 7fbef0651740 1 --2- 172.21.15.42:0/3088667375 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cd6864f2f0 0x55cd68a1e5d0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fbed4104130 tx=0x7fbed402e4b0 comp rx=0 tx=0).stop 2024-08-30T09:02:45.753 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.748+0000 7fbee37fe640 1 --2- 172.21.15.42:0/4191108543 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fbed00b4d40 0x7fbed00b7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:02:45.753 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.751+0000 7fbef0651740 1 -- 172.21.15.42:0/3088667375 shutdown_connections 2024-08-30T09:02:45.753 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.751+0000 7fbef0651740 1 --2- 172.21.15.42:0/3088667375 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cd686574d0 0x55cd68a1f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:45.753 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.751+0000 7fbef0651740 1 --2- 172.21.15.42:0/3088667375 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cd6864f2f0 0x55cd68a1e5d0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:45.753 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.751+0000 7fbef0651740 1 --2- 172.21.15.42:0/3088667375 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cd686781b0 0x55cd68a1df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:45.754 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.751+0000 7fbef0651740 1 -- 172.21.15.42:0/3088667375 >> 172.21.15.42:0/3088667375 conn(0x55cd6866c300 msgr2=0x55cd68a199f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:02:45.757 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.755+0000 7fbee37fe640 1 --2- 172.21.15.42:0/4191108543 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fbed00b4d40 0x7fbed00b7250 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fbedc009fd0 tx=0x7fbedc00b040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:02:45.762 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.760+0000 7fbef0651740 1 -- 172.21.15.42:0/3088667375 shutdown_connections 2024-08-30T09:02:45.766 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.764+0000 7fbef0651740 1 -- 172.21.15.42:0/3088667375 wait complete. 2024-08-30T09:02:45.766 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.764+0000 7fbef0651740 1 -- start start 2024-08-30T09:02:45.767 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.765+0000 7fbef0651740 1 Processor -- start 2024-08-30T09:02:45.769 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.766+0000 7fbef0651740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cd6864f2f0 0x55cd68a27750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:45.770 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.766+0000 7fbef0651740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cd686574d0 0x55cd68a27e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:45.770 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.766+0000 7fbef0651740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cd686781b0 0x55cd68a28980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:45.770 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.766+0000 7fbef0651740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55cd688b9ab0 con 0x55cd6864f2f0 2024-08-30T09:02:45.770 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.766+0000 7fbef0651740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55cd68907680 con 0x55cd686574d0 2024-08-30T09:02:45.770 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.766+0000 7fbef0651740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55cd68a1fa00 con 0x55cd686781b0 2024-08-30T09:02:45.770 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.766+0000 7fbef0651740 10 client.0 _ll_register_callbacks cb 0x55cd6868fa00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:02:45.770 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.767+0000 7fbee2ffd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cd6864f2f0 0x55cd68a27750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:02:45.775 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.769+0000 7fbee2ffd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cd6864f2f0 0x55cd68a27750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:55124/0 (socket says 172.21.15.42:55124) 2024-08-30T09:02:45.775 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.769+0000 7fbee2ffd640 1 -- 172.21.15.42:0/1725303160 learned_addr learned my addr 172.21.15.42:0/1725303160 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:02:45.775 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.769+0000 7fbee27fc640 1 --2- 172.21.15.42:0/1725303160 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cd686574d0 0x55cd68a27e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:02:45.775 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.769+0000 7fbee37fe640 1 --2- 172.21.15.42:0/1725303160 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cd686781b0 0x55cd68a28980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:02:45.776 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.770+0000 7fbee27fc640 1 -- 172.21.15.42:0/1725303160 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cd686781b0 msgr2=0x55cd68a28980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:02:45.776 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.770+0000 7fbee27fc640 1 --2- 172.21.15.42:0/1725303160 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cd686781b0 0x55cd68a28980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:45.776 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.770+0000 7fbee27fc640 1 -- 172.21.15.42:0/1725303160 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cd6864f2f0 msgr2=0x55cd68a27750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:02:45.776 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.770+0000 7fbee27fc640 1 --2- 172.21.15.42:0/1725303160 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cd6864f2f0 0x55cd68a27750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:45.776 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.770+0000 7fbee27fc640 1 -- 172.21.15.42:0/1725303160 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cd6875c620 con 0x55cd686574d0 2024-08-30T09:02:45.776 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.771+0000 7fbee27fc640 1 --2- 172.21.15.42:0/1725303160 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cd686574d0 0x55cd68a27e10 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fbecc0116c0 tx=0x7fbecc011b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bd51a588015140b2 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:02:45.776 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.772+0000 7fbeca7fc640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-30T09:02:45.776 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.773+0000 7fbeca7fc640 1 -- 172.21.15.42:0/1725303160 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbecc00fbf0 con 0x55cd686574d0 2024-08-30T09:02:45.776 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.773+0000 7fbef0651740 1 -- 172.21.15.42:0/1725303160 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbed4109580 con 0x55cd686574d0 2024-08-30T09:02:45.776 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.773+0000 7fbeca7fc640 1 -- 172.21.15.42:0/1725303160 <== mon.2 v2:172.21.15.167:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7fbecc00fd90 con 0x55cd686574d0 2024-08-30T09:02:45.785 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.784+0000 7fbeca7fc640 1 -- 172.21.15.42:0/1725303160 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fbecc027050 con 0x55cd686574d0 2024-08-30T09:02:45.787 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.785+0000 7fbef0651740 10 client.25496 fetch_fsmap learned FSMap version 29 2024-08-30T09:02:45.787 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.785+0000 7fbef0651740 1 -- 172.21.15.42:0/1725303160 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55cd68a319f0 con 0x55cd686574d0 2024-08-30T09:02:45.789 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.787+0000 7fbeca7fc640 1 -- 172.21.15.42:0/1725303160 <== mon.2 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbed4109580 con 0x55cd686574d0 2024-08-30T09:02:45.789 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.787+0000 7fbeca7fc640 1 -- 172.21.15.42:0/1725303160 <== mon.2 v2:172.21.15.167:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fbecc022030 con 0x55cd686574d0 2024-08-30T09:02:45.790 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.789+0000 7fbef0651740 10 client.25496 fetch_fsmap finished waiting for FSMap version 29 2024-08-30T09:02:45.790 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.789+0000 7fbef0651740 10 client.25496 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-30T09:02:45.790 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.789+0000 7fbef0651740 20 client.25496 populate_metadata read hostname 'smithi042' 2024-08-30T09:02:45.791 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.789+0000 7fbef0651740 1 --2- 172.21.15.42:0/1725303160 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55cd68636d80 0x55cd68a351e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:45.791 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.789+0000 7fbef0651740 4 client.25496 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-30T09:02:45.791 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.789+0000 7fbef0651740 1 -- 172.21.15.42:0/1725303160 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cd68992f50 con 0x55cd68636d80 2024-08-30T09:02:45.791 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.789+0000 7fbee2ffd640 1 --2- 172.21.15.42:0/1725303160 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55cd68636d80 0x55cd68a351e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:02:45.795 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.790+0000 7fbee2ffd640 1 --2- 172.21.15.42:0/1725303160 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55cd68636d80 0x55cd68a351e0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e5824f948667bbfb server_cookie=d30a9a8fee2bc697 in_seq=0 out_seq=0 2024-08-30T09:02:45.795 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.790+0000 7fbeca7fc640 10 client.25496 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:02:45.795 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.793+0000 7fbeca7fc640 1 -- 172.21.15.42:0/1725303160 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55cd68992f50 con 0x55cd68636d80 2024-08-30T09:02:45.795 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.793+0000 7fbeca7fc640 10 client.25496 handle_command_reply: tid=0 2024-08-30T09:02:45.795 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.793+0000 7fbef0651740 1 client.25496 shutdown 2024-08-30T09:02:45.801 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.799+0000 7fbef0651740 1 -- 172.21.15.42:0/1725303160 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cd686574d0 msgr2=0x55cd68a27e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:02:45.801 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.799+0000 7fbef0651740 1 --2- 172.21.15.42:0/1725303160 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cd686574d0 0x55cd68a27e10 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fbecc0116c0 tx=0x7fbecc011b80 comp rx=0 tx=0).stop 2024-08-30T09:02:45.809 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.807+0000 7fbef0651740 1 -- 172.21.15.42:0/1725303160 shutdown_connections 2024-08-30T09:02:45.809 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.807+0000 7fbef0651740 1 --2- 172.21.15.42:0/1725303160 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55cd68636d80 0x55cd68a351e0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:45.809 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.807+0000 7fbef0651740 1 --2- 172.21.15.42:0/1725303160 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cd686781b0 0x55cd68a28980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:45.809 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.807+0000 7fbef0651740 1 --2- 172.21.15.42:0/1725303160 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cd686574d0 0x55cd68a27e10 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:45.810 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.807+0000 7fbef0651740 1 --2- 172.21.15.42:0/1725303160 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cd6864f2f0 0x55cd68a27750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:45.810 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.807+0000 7fbef0651740 1 -- 172.21.15.42:0/1725303160 >> 172.21.15.42:0/1725303160 conn(0x55cd6866c300 msgr2=0x55cd68a17bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:02:45.810 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.807+0000 7fbeca7fc640 0 client.25496 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:02:45.810 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.808+0000 7fbef0651740 1 -- 172.21.15.42:0/1725303160 shutdown_connections 2024-08-30T09:02:45.811 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.809+0000 7fbef0651740 1 -- 172.21.15.42:0/1725303160 wait complete. 2024-08-30T09:02:45.812 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.810+0000 7fbef0651740 20 client.25496 trim_cache size 0 max 16384 2024-08-30T09:02:45.840 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.838+0000 7fbef0651740 1 Processor -- start 2024-08-30T09:02:45.840 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.838+0000 7fbef0651740 1 -- start start 2024-08-30T09:02:45.841 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.839+0000 7fbef0651740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cd6864f2f0 0x55cd68a19330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:45.841 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.839+0000 7fbef0651740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cd686574d0 0x55cd68a217c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:45.841 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.839+0000 7fbef0651740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cd686781b0 0x55cd68a21d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:45.841 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.839+0000 7fbef0651740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fbee41c6450 con 0x55cd6864f2f0 2024-08-30T09:02:45.842 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.839+0000 7fbef0651740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fbee41c3d20 con 0x55cd686781b0 2024-08-30T09:02:45.842 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.839+0000 7fbef0651740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fbee41c1b70 con 0x55cd686574d0 2024-08-30T09:02:45.842 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.839+0000 7fbee2ffd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cd6864f2f0 0x55cd68a19330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:02:45.842 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.839+0000 7fbee2ffd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cd6864f2f0 0x55cd68a19330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:55134/0 (socket says 172.21.15.42:55134) 2024-08-30T09:02:45.842 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.839+0000 7fbee2ffd640 1 -- 172.21.15.42:0/1207480728 learned_addr learned my addr 172.21.15.42:0/1207480728 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:02:45.842 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.839+0000 7fbee27fc640 1 --2- 172.21.15.42:0/1207480728 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cd686574d0 0x55cd68a217c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:02:45.842 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.839+0000 7fbee37fe640 1 --2- 172.21.15.42:0/1207480728 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cd686781b0 0x55cd68a21d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:02:45.842 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.839+0000 7fbee2ffd640 1 -- 172.21.15.42:0/1207480728 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cd686574d0 msgr2=0x55cd68a217c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:02:45.842 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.839+0000 7fbee2ffd640 1 --2- 172.21.15.42:0/1207480728 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cd686574d0 0x55cd68a217c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:45.842 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.839+0000 7fbee2ffd640 1 -- 172.21.15.42:0/1207480728 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cd686781b0 msgr2=0x55cd68a21d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:02:45.842 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.839+0000 7fbee2ffd640 1 --2- 172.21.15.42:0/1207480728 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cd686781b0 0x55cd68a21d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:45.842 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.840+0000 7fbee2ffd640 1 -- 172.21.15.42:0/1207480728 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cd68a1a370 con 0x55cd6864f2f0 2024-08-30T09:02:45.843 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.840+0000 7fbee2ffd640 1 --2- 172.21.15.42:0/1207480728 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cd6864f2f0 0x55cd68a19330 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fbed402eaa0 tx=0x7fbed4000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d74201b1000d152 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:02:45.843 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.841+0000 7fbed8ff9640 1 -- 172.21.15.42:0/1207480728 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbed404e040 con 0x55cd6864f2f0 2024-08-30T09:02:45.843 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.841+0000 7fbed8ff9640 1 -- 172.21.15.42:0/1207480728 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbed40ccbd0 con 0x55cd6864f2f0 2024-08-30T09:02:45.843 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.841+0000 7fbed8ff9640 1 -- 172.21.15.42:0/1207480728 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbed40356a0 con 0x55cd6864f2f0 2024-08-30T09:02:45.844 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.842+0000 7fbef0651740 1 -- 172.21.15.42:0/1207480728 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cd6864f2f0 msgr2=0x55cd68a19330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:02:45.845 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.842+0000 7fbef0651740 1 --2- 172.21.15.42:0/1207480728 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cd6864f2f0 0x55cd68a19330 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fbed402eaa0 tx=0x7fbed4000bc0 comp rx=0 tx=0).stop 2024-08-30T09:02:45.845 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.842+0000 7fbef0651740 1 -- 172.21.15.42:0/1207480728 shutdown_connections 2024-08-30T09:02:45.845 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.842+0000 7fbef0651740 1 --2- 172.21.15.42:0/1207480728 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cd686574d0 0x55cd68a217c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:45.845 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.842+0000 7fbef0651740 1 --2- 172.21.15.42:0/1207480728 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cd686781b0 0x55cd68a21d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:45.845 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.842+0000 7fbef0651740 1 --2- 172.21.15.42:0/1207480728 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cd6864f2f0 0x55cd68a19330 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:45.845 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.842+0000 7fbef0651740 1 -- 172.21.15.42:0/1207480728 >> 172.21.15.42:0/1207480728 conn(0x55cd6866c300 msgr2=0x55cd68a26950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:02:45.845 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.843+0000 7fbef0651740 1 -- 172.21.15.42:0/1207480728 shutdown_connections 2024-08-30T09:02:45.845 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.843+0000 7fbef0651740 1 -- 172.21.15.42:0/1207480728 wait complete. 2024-08-30T09:02:45.846 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.843+0000 7fbef0651740 1 -- start start 2024-08-30T09:02:45.846 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.844+0000 7fbef0651740 1 Processor -- start 2024-08-30T09:02:45.847 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.844+0000 7fbef0651740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cd6864f2f0 0x55cd68a2a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:45.847 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.844+0000 7fbef0651740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cd686574d0 0x55cd68a2a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:45.847 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.844+0000 7fbef0651740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cd686781b0 0x55cd68a2b330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:45.847 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.844+0000 7fbef0651740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fbee41c21e0 con 0x55cd686574d0 2024-08-30T09:02:45.847 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.844+0000 7fbef0651740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fbee41c3650 con 0x55cd6864f2f0 2024-08-30T09:02:45.847 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.844+0000 7fbef0651740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fbee41c28b0 con 0x55cd686781b0 2024-08-30T09:02:45.847 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.844+0000 7fbef0651740 10 client.0 _ll_register_callbacks cb 0x7fbecc009890 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:02:45.847 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.844+0000 7fbee27fc640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cd686574d0 0x55cd68a2a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:02:45.847 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.844+0000 7fbee2ffd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cd6864f2f0 0x55cd68a2a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:02:45.847 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.844+0000 7fbee27fc640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cd686574d0 0x55cd68a2a8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:55144/0 (socket says 172.21.15.42:55144) 2024-08-30T09:02:45.847 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.844+0000 7fbee27fc640 1 -- 172.21.15.42:0/1651689724 learned_addr learned my addr 172.21.15.42:0/1651689724 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:02:45.847 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.844+0000 7fbee2ffd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cd6864f2f0 0x55cd68a2a1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:44452/0 (socket says 172.21.15.42:44452) 2024-08-30T09:02:45.847 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.845+0000 7fbee37fe640 1 --2- 172.21.15.42:0/1651689724 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cd686781b0 0x55cd68a2b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:02:45.848 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.845+0000 7fbee2ffd640 1 -- 172.21.15.42:0/1651689724 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cd686781b0 msgr2=0x55cd68a2b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:02:45.848 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.845+0000 7fbee2ffd640 1 --2- 172.21.15.42:0/1651689724 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cd686781b0 0x55cd68a2b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:45.848 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.845+0000 7fbee2ffd640 1 -- 172.21.15.42:0/1651689724 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cd686574d0 msgr2=0x55cd68a2a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:02:45.848 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.845+0000 7fbee2ffd640 1 --2- 172.21.15.42:0/1651689724 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cd686574d0 0x55cd68a2a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:45.848 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.845+0000 7fbee2ffd640 1 -- 172.21.15.42:0/1651689724 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cd68a2ba90 con 0x55cd6864f2f0 2024-08-30T09:02:45.848 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.845+0000 7fbee27fc640 1 --2- 172.21.15.42:0/1651689724 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cd686574d0 0x55cd68a2a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-30T09:02:45.850 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.846+0000 7fbee2ffd640 1 --2- 172.21.15.42:0/1651689724 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cd6864f2f0 0x55cd68a2a1f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fbed404d000 tx=0x7fbed4043000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=78ba7995522069cc server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:02:45.850 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.846+0000 7fbeca7fc640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-30T09:02:45.850 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.846+0000 7fbeca7fc640 1 -- 172.21.15.42:0/1651689724 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbed404e040 con 0x55cd6864f2f0 2024-08-30T09:02:45.850 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.846+0000 7fbeca7fc640 1 -- 172.21.15.42:0/1651689724 <== mon.2 v2:172.21.15.167:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7fbed40c9620 con 0x55cd6864f2f0 2024-08-30T09:02:45.850 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.846+0000 7fbef0651740 1 -- 172.21.15.42:0/1651689724 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbed40ccbd0 con 0x55cd6864f2f0 2024-08-30T09:02:45.850 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.847+0000 7fbeca7fc640 1 -- 172.21.15.42:0/1651689724 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fbed40c8ef0 con 0x55cd6864f2f0 2024-08-30T09:02:45.850 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.847+0000 7fbef0651740 10 client.25502 fetch_fsmap learned FSMap version 29 2024-08-30T09:02:45.850 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.847+0000 7fbef0651740 1 -- 172.21.15.42:0/1651689724 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55cd68a2ed90 con 0x55cd6864f2f0 2024-08-30T09:02:45.850 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.847+0000 7fbeca7fc640 1 -- 172.21.15.42:0/1651689724 <== mon.2 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbed40356a0 con 0x55cd6864f2f0 2024-08-30T09:02:45.850 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.847+0000 7fbeca7fc640 1 -- 172.21.15.42:0/1651689724 <== mon.2 v2:172.21.15.167:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fbed40c92c0 con 0x55cd6864f2f0 2024-08-30T09:02:45.850 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.847+0000 7fbef0651740 10 client.25502 fetch_fsmap finished waiting for FSMap version 29 2024-08-30T09:02:45.851 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.847+0000 7fbef0651740 10 client.25502 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-30T09:02:45.851 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.847+0000 7fbef0651740 20 client.25502 populate_metadata read hostname 'smithi042' 2024-08-30T09:02:45.851 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.847+0000 7fbef0651740 1 --2- 172.21.15.42:0/1651689724 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55cd6860c1c0 0x55cd68aa1bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:45.851 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.848+0000 7fbef0651740 4 client.25502 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-30T09:02:45.851 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.848+0000 7fbef0651740 1 -- 172.21.15.42:0/1651689724 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7fbed40ccbd0 con 0x55cd6860c1c0 2024-08-30T09:02:45.851 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.848+0000 7fbee27fc640 1 --2- 172.21.15.42:0/1651689724 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55cd6860c1c0 0x55cd68aa1bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:02:45.851 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.849+0000 7fbee27fc640 1 --2- 172.21.15.42:0/1651689724 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55cd6860c1c0 0x55cd68aa1bb0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=17e1be61ebad887a server_cookie=5ad79d6597e01544 in_seq=0 out_seq=0 2024-08-30T09:02:45.851 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.850+0000 7fbeca7fc640 10 client.25502 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:02:45.882 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.881+0000 7fbeca7fc640 1 -- 172.21.15.42:0/1651689724 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7fbed40ccbd0 con 0x55cd6860c1c0 2024-08-30T09:02:45.883 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.881+0000 7fbeca7fc640 10 client.25502 handle_command_reply: tid=0 2024-08-30T09:02:45.883 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.881+0000 7fbef0651740 1 client.25502 shutdown 2024-08-30T09:02:45.884 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.881+0000 7fbef0651740 1 -- 172.21.15.42:0/1651689724 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cd6864f2f0 msgr2=0x55cd68a2a1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:02:45.884 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.881+0000 7fbef0651740 1 --2- 172.21.15.42:0/1651689724 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cd6864f2f0 0x55cd68a2a1f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fbed404d000 tx=0x7fbed4043000 comp rx=0 tx=0).stop 2024-08-30T09:02:45.884 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.882+0000 7fbef0651740 1 -- 172.21.15.42:0/1651689724 shutdown_connections 2024-08-30T09:02:45.884 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.882+0000 7fbef0651740 1 --2- 172.21.15.42:0/1651689724 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55cd6860c1c0 0x55cd68aa1bb0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:45.884 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.882+0000 7fbeca7fc640 0 client.25502 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:02:45.884 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.882+0000 7fbef0651740 1 --2- 172.21.15.42:0/1651689724 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cd686781b0 0x55cd68a2b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:45.884 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.882+0000 7fbef0651740 1 --2- 172.21.15.42:0/1651689724 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cd686574d0 0x55cd68a2a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:45.884 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.882+0000 7fbef0651740 1 --2- 172.21.15.42:0/1651689724 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cd6864f2f0 0x55cd68a2a1f0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:45.884 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.882+0000 7fbef0651740 1 -- 172.21.15.42:0/1651689724 >> 172.21.15.42:0/1651689724 conn(0x55cd6866c300 msgr2=0x55cd68a19330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:02:45.885 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.882+0000 7fbef0651740 1 -- 172.21.15.42:0/1651689724 shutdown_connections 2024-08-30T09:02:45.885 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.882+0000 7fbef0651740 1 -- 172.21.15.42:0/1651689724 wait complete. 2024-08-30T09:02:45.885 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.882+0000 7fbef0651740 20 client.25502 trim_cache size 0 max 16384 2024-08-30T09:02:45.885 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-08-30T09:02:45.885 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2024-08-30T09:02:45.885 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-08-30T09:02:45.885 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-08-30T09:02:45.886 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.883+0000 7fbee9607640 1 -- 172.21.15.42:0/4191108543 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fbed00b4d40 msgr2=0x7fbed00b7250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:02:45.886 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.883+0000 7fbee9607640 1 --2- 172.21.15.42:0/4191108543 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fbed00b4d40 0x7fbed00b7250 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fbedc009fd0 tx=0x7fbedc00b040 comp rx=0 tx=0).stop 2024-08-30T09:02:45.886 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.883+0000 7fbee9607640 1 -- 172.21.15.42:0/4191108543 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbee4102a40 msgr2=0x7fbee4074860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:02:45.886 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.883+0000 7fbee9607640 1 --2- 172.21.15.42:0/4191108543 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbee4102a40 0x7fbee4074860 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fbed400d880 tx=0x7fbed4005040 comp rx=0 tx=0).stop 2024-08-30T09:02:45.886 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.883+0000 7fbee9607640 1 -- 172.21.15.42:0/4191108543 shutdown_connections 2024-08-30T09:02:45.886 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.883+0000 7fbee9607640 1 --2- 172.21.15.42:0/4191108543 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fbed00b4d40 0x7fbed00b7250 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:45.886 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.883+0000 7fbee9607640 1 --2- 172.21.15.42:0/4191108543 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbee4105480 0x7fbee4073120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:45.887 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.883+0000 7fbee9607640 1 --2- 172.21.15.42:0/4191108543 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbee41080a0 0x7fbee4073660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:45.887 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.883+0000 7fbee9607640 1 --2- 172.21.15.42:0/4191108543 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbee4102a40 0x7fbee4074860 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:45.887 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.883+0000 7fbee9607640 1 -- 172.21.15.42:0/4191108543 >> 172.21.15.42:0/4191108543 conn(0x7fbee40fd430 msgr2=0x7fbee4103950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:02:45.887 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.883+0000 7fbee9607640 1 -- 172.21.15.42:0/4191108543 shutdown_connections 2024-08-30T09:02:45.887 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:45.883+0000 7fbee9607640 1 -- 172.21.15.42:0/4191108543 wait complete. 2024-08-30T09:02:45.953 INFO:teuthology.orchestra.run.smithi042.stderr:progress: 120.0 s, 1980.6 tps, lat 49.652 ms stddev 56.383 2024-08-30T09:02:47.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:02:46 smithi167 ceph-mon[22063]: pgmap v433: 129 pgs: 129 active+clean; 1.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.4 MiB/s wr, 650 op/s 2024-08-30T09:02:47.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:02:46 smithi042 ceph-mon[18652]: pgmap v433: 129 pgs: 129 active+clean; 1.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.4 MiB/s wr, 650 op/s 2024-08-30T09:02:47.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:02:46 smithi190 ceph-mon[22001]: pgmap v433: 129 pgs: 129 active+clean; 1.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.4 MiB/s wr, 650 op/s 2024-08-30T09:02:48.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:02:47 smithi167 ceph-mon[22063]: pgmap v434: 129 pgs: 129 active+clean; 1.1 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 7.7 MiB/s wr, 712 op/s 2024-08-30T09:02:48.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:02:47 smithi042 ceph-mon[18652]: pgmap v434: 129 pgs: 129 active+clean; 1.1 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 7.7 MiB/s wr, 712 op/s 2024-08-30T09:02:48.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:02:47 smithi190 ceph-mon[22001]: pgmap v434: 129 pgs: 129 active+clean; 1.1 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 7.7 MiB/s wr, 712 op/s 2024-08-30T09:02:50.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:02:50 smithi042 ceph-mon[18652]: pgmap v435: 129 pgs: 129 active+clean; 1.1 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 9.1 MiB/s wr, 767 op/s 2024-08-30T09:02:50.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:02:50 smithi190 ceph-mon[22001]: pgmap v435: 129 pgs: 129 active+clean; 1.1 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 9.1 MiB/s wr, 767 op/s 2024-08-30T09:02:50.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:02:50 smithi167 ceph-mon[22063]: pgmap v435: 129 pgs: 129 active+clean; 1.1 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 9.1 MiB/s wr, 767 op/s 2024-08-30T09:02:50.953 INFO:teuthology.orchestra.run.smithi042.stderr:progress: 125.0 s, 1695.8 tps, lat 59.902 ms stddev 71.591 2024-08-30T09:02:52.382 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:02:52 smithi042 ceph-mon[18652]: pgmap v436: 129 pgs: 129 active+clean; 1.1 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 7.6 MiB/s wr, 641 op/s 2024-08-30T09:02:52.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:02:52 smithi190 ceph-mon[22001]: pgmap v436: 129 pgs: 129 active+clean; 1.1 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 7.6 MiB/s wr, 641 op/s 2024-08-30T09:02:52.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:02:52 smithi167 ceph-mon[22063]: pgmap v436: 129 pgs: 129 active+clean; 1.1 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 7.6 MiB/s wr, 641 op/s 2024-08-30T09:02:54.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:02:54 smithi042 ceph-mon[18652]: pgmap v437: 129 pgs: 129 active+clean; 1.1 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 8.8 MiB/s wr, 789 op/s 2024-08-30T09:02:54.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:02:54 smithi190 ceph-mon[22001]: pgmap v437: 129 pgs: 129 active+clean; 1.1 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 8.8 MiB/s wr, 789 op/s 2024-08-30T09:02:54.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:02:54 smithi167 ceph-mon[22063]: pgmap v437: 129 pgs: 129 active+clean; 1.1 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 8.8 MiB/s wr, 789 op/s 2024-08-30T09:02:55.953 INFO:teuthology.orchestra.run.smithi042.stderr:progress: 130.0 s, 1809.2 tps, lat 54.822 ms stddev 60.139 2024-08-30T09:02:56.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:02:56 smithi042 ceph-mon[18652]: pgmap v438: 129 pgs: 129 active+clean; 1.1 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 7.2 MiB/s wr, 617 op/s 2024-08-30T09:02:56.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:02:56 smithi190 ceph-mon[22001]: pgmap v438: 129 pgs: 129 active+clean; 1.1 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 7.2 MiB/s wr, 617 op/s 2024-08-30T09:02:56.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:02:56 smithi167 ceph-mon[22063]: pgmap v438: 129 pgs: 129 active+clean; 1.1 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 7.2 MiB/s wr, 617 op/s 2024-08-30T09:02:58.041 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-30T09:02:58.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:02:58 smithi042 ceph-mon[18652]: pgmap v439: 129 pgs: 129 active+clean; 1.1 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 7.8 MiB/s wr, 690 op/s 2024-08-30T09:02:58.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:02:58 smithi190 ceph-mon[22001]: pgmap v439: 129 pgs: 129 active+clean; 1.1 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 7.8 MiB/s wr, 690 op/s 2024-08-30T09:02:58.474 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T09:02:58.576 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:02:58 smithi167 ceph-mon[22063]: pgmap v439: 129 pgs: 129 active+clean; 1.1 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 7.8 MiB/s wr, 690 op/s 2024-08-30T09:02:59.029 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.027+0000 7f32bc194640 1 -- 172.21.15.42:0/1102937103 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f32b4075a90 msgr2=0x7f32b4075ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:02:59.029 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.027+0000 7f32bc194640 1 --2- 172.21.15.42:0/1102937103 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f32b4075a90 0x7f32b4075ea0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f32b0009900 tx=0x7f32b002a440 comp rx=0 tx=0).stop 2024-08-30T09:02:59.034 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.031+0000 7f32bc194640 1 -- 172.21.15.42:0/1102937103 shutdown_connections 2024-08-30T09:02:59.035 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.031+0000 7f32bc194640 1 --2- 172.21.15.42:0/1102937103 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f32b4075a90 0x7f32b4075ea0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:59.035 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.031+0000 7f32bc194640 1 --2- 172.21.15.42:0/1102937103 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f32b410fc10 0x7f32b4112030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:59.035 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.031+0000 7f32bc194640 1 --2- 172.21.15.42:0/1102937103 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f32b4076e90 0x7f32b4075550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:59.035 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.031+0000 7f32bc194640 1 -- 172.21.15.42:0/1102937103 >> 172.21.15.42:0/1102937103 conn(0x7f32b40fedd0 msgr2=0x7f32b4101280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:02:59.035 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.031+0000 7f32bc194640 1 -- 172.21.15.42:0/1102937103 shutdown_connections 2024-08-30T09:02:59.035 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.031+0000 7f32bc194640 1 -- 172.21.15.42:0/1102937103 wait complete. 2024-08-30T09:02:59.035 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.032+0000 7f32bc194640 1 Processor -- start 2024-08-30T09:02:59.035 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.032+0000 7f32bc194640 1 -- start start 2024-08-30T09:02:59.035 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.032+0000 7f32bc194640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f32b4075a90 0x7f32b4074160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:59.035 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.032+0000 7f32bc194640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f32b4076e90 0x7f32b40747b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:59.035 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.032+0000 7f32bc194640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f32b410fc10 0x7f32b4072d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:59.035 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.032+0000 7f32bc194640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f32b406d050 con 0x7f32b4075a90 2024-08-30T09:02:59.035 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.032+0000 7f32bc194640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f32b406ced0 con 0x7f32b4076e90 2024-08-30T09:02:59.035 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.032+0000 7f32bc194640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f32b406d1d0 con 0x7f32b410fc10 2024-08-30T09:02:59.038 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.033+0000 7f32ba731640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f32b410fc10 0x7f32b4072d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:02:59.038 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.033+0000 7f32ba731640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f32b410fc10 0x7f32b4072d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:49852/0 (socket says 172.21.15.42:49852) 2024-08-30T09:02:59.038 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.033+0000 7f32ba731640 1 -- 172.21.15.42:0/3306136158 learned_addr learned my addr 172.21.15.42:0/3306136158 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:02:59.038 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.033+0000 7f32b972f640 1 --2- 172.21.15.42:0/3306136158 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f32b4076e90 0x7f32b40747b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:02:59.039 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.034+0000 7f32b972f640 1 -- 172.21.15.42:0/3306136158 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f32b410fc10 msgr2=0x7f32b4072d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:02:59.039 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.034+0000 7f32b972f640 1 --2- 172.21.15.42:0/3306136158 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f32b410fc10 0x7f32b4072d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:59.039 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.034+0000 7f32b972f640 1 -- 172.21.15.42:0/3306136158 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f32b4075a90 msgr2=0x7f32b4074160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:02:59.039 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.034+0000 7f32b972f640 1 --2- 172.21.15.42:0/3306136158 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f32b4075a90 0x7f32b4074160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:59.039 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.034+0000 7f32b972f640 1 -- 172.21.15.42:0/3306136158 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32b4073310 con 0x7f32b4076e90 2024-08-30T09:02:59.039 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.034+0000 7f32b972f640 1 --2- 172.21.15.42:0/3306136158 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f32b4076e90 0x7f32b40747b0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f32b0000c00 tx=0x7f32b00309d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:02:59.039 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.035+0000 7f32aaffd640 1 -- 172.21.15.42:0/3306136158 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32b0040030 con 0x7f32b4076e90 2024-08-30T09:02:59.039 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.035+0000 7f32bc194640 1 -- 172.21.15.42:0/3306136158 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32b40735a0 con 0x7f32b4076e90 2024-08-30T09:02:59.039 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.036+0000 7f32bc194640 1 -- 172.21.15.42:0/3306136158 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32b4073a90 con 0x7f32b4076e90 2024-08-30T09:02:59.039 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.036+0000 7f32aaffd640 1 -- 172.21.15.42:0/3306136158 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f32b0030d50 con 0x7f32b4076e90 2024-08-30T09:02:59.039 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.036+0000 7f32aaffd640 1 -- 172.21.15.42:0/3306136158 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32b0035070 con 0x7f32b4076e90 2024-08-30T09:02:59.041 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.039+0000 7f32c31d9740 1 Processor -- start 2024-08-30T09:02:59.047 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.044+0000 7f32aaffd640 1 -- 172.21.15.42:0/3306136158 <== mon.2 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7f32b0043070 con 0x7f32b4076e90 2024-08-30T09:02:59.047 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.045+0000 7f32aaffd640 1 --2- 172.21.15.42:0/3306136158 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f32a00c5d40 0x7f32a00c8250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:59.047 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.045+0000 7f32aaffd640 1 -- 172.21.15.42:0/3306136158 <== mon.2 v2:172.21.15.167:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f32b0105d90 con 0x7f32b4076e90 2024-08-30T09:02:59.047 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.046+0000 7f32c31d9740 1 -- start start 2024-08-30T09:02:59.048 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.046+0000 7f32c31d9740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5576f25561b0 0x5576f28fc180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:59.048 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.046+0000 7f32c31d9740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5576f252d2f0 0x5576f28fc7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:59.048 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.046+0000 7f32c31d9740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5576f25354d0 0x5576f28fd340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:59.048 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.046+0000 7f32c31d9740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x5576f286f000 con 0x5576f252d2f0 2024-08-30T09:02:59.048 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.046+0000 7f32c31d9740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5576f285e590 con 0x5576f25354d0 2024-08-30T09:02:59.049 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.046+0000 7f32c31d9740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5576f2862540 con 0x5576f25561b0 2024-08-30T09:02:59.049 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.046+0000 7f32b9f30640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5576f25561b0 0x5576f28fc180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:02:59.049 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.046+0000 7f32b9f30640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5576f25561b0 0x5576f28fc180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:49868/0 (socket says 172.21.15.42:49868) 2024-08-30T09:02:59.049 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.046+0000 7f32b9f30640 1 -- 172.21.15.42:0/3537770237 learned_addr learned my addr 172.21.15.42:0/3537770237 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:02:59.049 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.047+0000 7f32ba731640 1 --2- 172.21.15.42:0/3306136158 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f32a00c5d40 0x7f32a00c8250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:02:59.049 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.047+0000 7f32ba731640 1 --2- 172.21.15.42:0/3537770237 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5576f25354d0 0x5576f28fd340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:02:59.049 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.047+0000 7f32b972f640 1 --2- 172.21.15.42:0/3537770237 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5576f252d2f0 0x5576f28fc7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:02:59.050 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.048+0000 7f32b9f30640 1 -- 172.21.15.42:0/3537770237 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5576f25354d0 msgr2=0x5576f28fd340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:02:59.050 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.048+0000 7f32b9f30640 1 --2- 172.21.15.42:0/3537770237 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5576f25354d0 0x5576f28fd340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:59.050 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.048+0000 7f32b9f30640 1 -- 172.21.15.42:0/3537770237 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5576f252d2f0 msgr2=0x5576f28fc7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:02:59.050 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.048+0000 7f32b9f30640 1 --2- 172.21.15.42:0/3537770237 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5576f252d2f0 0x5576f28fc7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:59.050 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.048+0000 7f32b9f30640 1 -- 172.21.15.42:0/3537770237 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5576f25c1790 con 0x5576f25561b0 2024-08-30T09:02:59.050 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.048+0000 7f32ba731640 1 --2- 172.21.15.42:0/3537770237 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5576f25354d0 0x5576f28fd340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-30T09:02:59.051 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.048+0000 7f32ba731640 1 --2- 172.21.15.42:0/3306136158 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f32a00c5d40 0x7f32a00c8250 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f32a4014580 tx=0x7f32a4014980 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:02:59.051 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.049+0000 7f32b9f30640 1 --2- 172.21.15.42:0/3537770237 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5576f25561b0 0x5576f28fc180 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f32ac009d70 tx=0x7f32ac00b420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ed95ba65cbadd953 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:02:59.051 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.049+0000 7f329ffff640 1 -- 172.21.15.42:0/3537770237 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32ac00bd40 con 0x5576f25561b0 2024-08-30T09:02:59.051 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.049+0000 7f329ffff640 1 -- 172.21.15.42:0/3537770237 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f32ac017070 con 0x5576f25561b0 2024-08-30T09:02:59.052 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.050+0000 7f329ffff640 1 -- 172.21.15.42:0/3537770237 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32ac0116c0 con 0x5576f25561b0 2024-08-30T09:02:59.052 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.051+0000 7f32c31d9740 1 -- 172.21.15.42:0/3537770237 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5576f25561b0 msgr2=0x5576f28fc180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:02:59.052 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.051+0000 7f32c31d9740 1 --2- 172.21.15.42:0/3537770237 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5576f25561b0 0x5576f28fc180 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f32ac009d70 tx=0x7f32ac00b420 comp rx=0 tx=0).stop 2024-08-30T09:02:59.054 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.052+0000 7f32c31d9740 1 -- 172.21.15.42:0/3537770237 shutdown_connections 2024-08-30T09:02:59.054 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.052+0000 7f32c31d9740 1 --2- 172.21.15.42:0/3537770237 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5576f25354d0 0x5576f28fd340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:59.054 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.052+0000 7f32c31d9740 1 --2- 172.21.15.42:0/3537770237 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5576f252d2f0 0x5576f28fc7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:59.054 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.052+0000 7f32c31d9740 1 --2- 172.21.15.42:0/3537770237 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5576f25561b0 0x5576f28fc180 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:59.054 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.052+0000 7f32c31d9740 1 -- 172.21.15.42:0/3537770237 >> 172.21.15.42:0/3537770237 conn(0x5576f254a300 msgr2=0x5576f28f7c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:02:59.054 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.052+0000 7f32c31d9740 1 -- 172.21.15.42:0/3537770237 shutdown_connections 2024-08-30T09:02:59.055 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.052+0000 7f32c31d9740 1 -- 172.21.15.42:0/3537770237 wait complete. 2024-08-30T09:02:59.055 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.053+0000 7f32c31d9740 1 -- start start 2024-08-30T09:02:59.055 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.053+0000 7f32c31d9740 1 Processor -- start 2024-08-30T09:02:59.055 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.054+0000 7f32c31d9740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5576f252d2f0 0x5576f2905cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:59.055 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.054+0000 7f32c31d9740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5576f25354d0 0x5576f29063b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:59.056 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.054+0000 7f32c31d9740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5576f25561b0 0x5576f2906f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:59.056 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.054+0000 7f32c31d9740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x5576f2797ab0 con 0x5576f25354d0 2024-08-30T09:02:59.056 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.054+0000 7f32c31d9740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5576f27e5680 con 0x5576f252d2f0 2024-08-30T09:02:59.056 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.054+0000 7f32c31d9740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5576f28fdeb0 con 0x5576f25561b0 2024-08-30T09:02:59.056 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.054+0000 7f32c31d9740 10 client.0 _ll_register_callbacks cb 0x5576f256da00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:02:59.056 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.054+0000 7f32b9f30640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5576f252d2f0 0x5576f2905cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:02:59.057 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.054+0000 7f32b9f30640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5576f252d2f0 0x5576f2905cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:52100/0 (socket says 172.21.15.42:52100) 2024-08-30T09:02:59.057 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.054+0000 7f32b9f30640 1 -- 172.21.15.42:0/3775196312 learned_addr learned my addr 172.21.15.42:0/3775196312 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:02:59.057 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.054+0000 7f32b972f640 1 --2- 172.21.15.42:0/3775196312 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5576f25354d0 0x5576f29063b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:02:59.057 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.054+0000 7f32ba731640 1 --2- 172.21.15.42:0/3775196312 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5576f25561b0 0x5576f2906f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:02:59.057 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.055+0000 7f32b9f30640 1 -- 172.21.15.42:0/3775196312 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5576f25561b0 msgr2=0x5576f2906f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:02:59.057 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.055+0000 7f32b9f30640 1 --2- 172.21.15.42:0/3775196312 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5576f25561b0 0x5576f2906f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:59.057 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.055+0000 7f32b9f30640 1 -- 172.21.15.42:0/3775196312 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5576f25354d0 msgr2=0x5576f29063b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:02:59.057 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.055+0000 7f32b9f30640 1 --2- 172.21.15.42:0/3775196312 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5576f25354d0 0x5576f29063b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:59.057 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.055+0000 7f32b9f30640 1 -- 172.21.15.42:0/3775196312 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5576f263a620 con 0x5576f252d2f0 2024-08-30T09:02:59.058 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.055+0000 7f32b9f30640 1 --2- 172.21.15.42:0/3775196312 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5576f252d2f0 0x5576f2905cf0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f32ac011660 tx=0x7f32ac011690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f243a8d7a6f8df6e server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:02:59.058 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.055+0000 7f329d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-30T09:02:59.058 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.056+0000 7f329d7fa640 1 -- 172.21.15.42:0/3775196312 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32ac022030 con 0x5576f252d2f0 2024-08-30T09:02:59.058 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.056+0000 7f329d7fa640 1 -- 172.21.15.42:0/3775196312 <== mon.2 v2:172.21.15.167:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f32ac003a10 con 0x5576f252d2f0 2024-08-30T09:02:59.058 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.056+0000 7f32c31d9740 1 -- 172.21.15.42:0/3775196312 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f32ac017070 con 0x5576f252d2f0 2024-08-30T09:02:59.059 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.057+0000 7f329d7fa640 1 -- 172.21.15.42:0/3775196312 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f32ac012be0 con 0x5576f252d2f0 2024-08-30T09:02:59.059 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.057+0000 7f32c31d9740 10 client.25511 fetch_fsmap learned FSMap version 29 2024-08-30T09:02:59.059 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.057+0000 7f32c31d9740 1 -- 172.21.15.42:0/3775196312 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5576f290fea0 con 0x5576f252d2f0 2024-08-30T09:02:59.059 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.057+0000 7f329d7fa640 1 -- 172.21.15.42:0/3775196312 <== mon.2 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f32ac017070 con 0x5576f252d2f0 2024-08-30T09:02:59.059 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.057+0000 7f329d7fa640 1 -- 172.21.15.42:0/3775196312 <== mon.2 v2:172.21.15.167:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f32ac011070 con 0x5576f252d2f0 2024-08-30T09:02:59.060 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.057+0000 7f32c31d9740 10 client.25511 fetch_fsmap finished waiting for FSMap version 29 2024-08-30T09:02:59.060 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.058+0000 7f32c31d9740 10 client.25511 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-30T09:02:59.060 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.058+0000 7f32c31d9740 20 client.25511 populate_metadata read hostname 'smithi042' 2024-08-30T09:02:59.060 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.058+0000 7f32c31d9740 1 --2- 172.21.15.42:0/3775196312 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x5576f2514d80 0x5576f29133a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:59.060 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.058+0000 7f32c31d9740 4 client.25511 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-30T09:02:59.060 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.058+0000 7f32c31d9740 1 -- 172.21.15.42:0/3775196312 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5576f2870f50 con 0x5576f2514d80 2024-08-30T09:02:59.060 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.058+0000 7f32b972f640 1 --2- 172.21.15.42:0/3775196312 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x5576f2514d80 0x5576f29133a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:02:59.061 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.058+0000 7f329d7fa640 10 client.25511 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:02:59.061 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.058+0000 7f32b972f640 1 --2- 172.21.15.42:0/3775196312 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x5576f2514d80 0x5576f29133a0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee806631a56ca596 server_cookie=34511b35205eb548 in_seq=0 out_seq=0 2024-08-30T09:02:59.061 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.059+0000 7f329d7fa640 1 -- 172.21.15.42:0/3775196312 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5576f2870f50 con 0x5576f2514d80 2024-08-30T09:02:59.061 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.059+0000 7f329d7fa640 10 client.25511 handle_command_reply: tid=0 2024-08-30T09:02:59.061 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.060+0000 7f32c31d9740 1 client.25511 shutdown 2024-08-30T09:02:59.064 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.062+0000 7f32c31d9740 1 -- 172.21.15.42:0/3775196312 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5576f252d2f0 msgr2=0x5576f2905cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:02:59.064 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.062+0000 7f32c31d9740 1 --2- 172.21.15.42:0/3775196312 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5576f252d2f0 0x5576f2905cf0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f32ac011660 tx=0x7f32ac011690 comp rx=0 tx=0).stop 2024-08-30T09:02:59.064 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.062+0000 7f32c31d9740 1 -- 172.21.15.42:0/3775196312 shutdown_connections 2024-08-30T09:02:59.065 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.062+0000 7f32c31d9740 1 --2- 172.21.15.42:0/3775196312 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x5576f2514d80 0x5576f29133a0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:59.065 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.062+0000 7f32c31d9740 1 --2- 172.21.15.42:0/3775196312 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5576f25561b0 0x5576f2906f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:59.065 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.062+0000 7f32c31d9740 1 --2- 172.21.15.42:0/3775196312 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5576f25354d0 0x5576f29063b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:59.065 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.062+0000 7f32c31d9740 1 --2- 172.21.15.42:0/3775196312 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5576f252d2f0 0x5576f2905cf0 secure :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f32ac011660 tx=0x7f32ac011690 comp rx=0 tx=0).stop 2024-08-30T09:02:59.065 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.062+0000 7f32c31d9740 1 -- 172.21.15.42:0/3775196312 >> 172.21.15.42:0/3775196312 conn(0x5576f254a300 msgr2=0x5576f28f6170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:02:59.065 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.062+0000 7f329d7fa640 0 client.25511 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:02:59.065 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.062+0000 7f32c31d9740 1 -- 172.21.15.42:0/3775196312 shutdown_connections 2024-08-30T09:02:59.065 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.063+0000 7f32c31d9740 1 -- 172.21.15.42:0/3775196312 wait complete. 2024-08-30T09:02:59.065 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.063+0000 7f32c31d9740 20 client.25511 trim_cache size 0 max 16384 2024-08-30T09:02:59.098 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.096+0000 7f32c31d9740 1 Processor -- start 2024-08-30T09:02:59.100 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.098+0000 7f32c31d9740 1 -- start start 2024-08-30T09:02:59.100 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.098+0000 7f32c31d9740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5576f25354d0 0x5576f28f9060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:59.100 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.098+0000 7f32c31d9740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5576f25561b0 0x5576f28f2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:59.100 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.098+0000 7f32c31d9740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5576f24ea1c0 0x5576f2904440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:59.100 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.098+0000 7f32c31d9740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f3298012560 con 0x5576f25561b0 2024-08-30T09:02:59.100 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.098+0000 7f32c31d9740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f329800fe30 con 0x5576f24ea1c0 2024-08-30T09:02:59.100 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.098+0000 7f32c31d9740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f329800dc80 con 0x5576f25354d0 2024-08-30T09:02:59.103 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.101+0000 7f32b972f640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5576f25561b0 0x5576f28f2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:02:59.103 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.101+0000 7f32b972f640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5576f25561b0 0x5576f28f2400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:54732/0 (socket says 172.21.15.42:54732) 2024-08-30T09:02:59.103 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.101+0000 7f32b972f640 1 -- 172.21.15.42:0/3507739758 learned_addr learned my addr 172.21.15.42:0/3507739758 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:02:59.103 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.101+0000 7f32b9f30640 1 --2- 172.21.15.42:0/3507739758 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5576f25354d0 0x5576f28f9060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:02:59.104 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.102+0000 7f32ba731640 1 --2- 172.21.15.42:0/3507739758 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5576f24ea1c0 0x5576f2904440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:02:59.105 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.104+0000 7f32b972f640 1 -- 172.21.15.42:0/3507739758 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5576f25354d0 msgr2=0x5576f28f9060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:02:59.105 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.104+0000 7f32b972f640 1 --2- 172.21.15.42:0/3507739758 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5576f25354d0 0x5576f28f9060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:59.106 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.104+0000 7f32b972f640 1 -- 172.21.15.42:0/3507739758 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5576f24ea1c0 msgr2=0x5576f2904440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:02:59.106 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.104+0000 7f32b972f640 1 --2- 172.21.15.42:0/3507739758 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5576f24ea1c0 0x5576f2904440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:59.106 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.104+0000 7f32b972f640 1 -- 172.21.15.42:0/3507739758 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5576f2907480 con 0x5576f25561b0 2024-08-30T09:02:59.108 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.107+0000 7f32b972f640 1 --2- 172.21.15.42:0/3507739758 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5576f25561b0 0x5576f28f2400 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f32b0006010 tx=0x7f32b0103370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48dd8cfdef320cc3 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:02:59.113 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.110+0000 7f329f7fe640 1 -- 172.21.15.42:0/3507739758 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32b00cc3d0 con 0x5576f25561b0 2024-08-30T09:02:59.113 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.110+0000 7f329f7fe640 1 -- 172.21.15.42:0/3507739758 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f32b00d0c40 con 0x5576f25561b0 2024-08-30T09:02:59.113 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.111+0000 7f32c31d9740 1 -- 172.21.15.42:0/3507739758 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5576f25561b0 msgr2=0x5576f28f2400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:02:59.113 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.111+0000 7f329f7fe640 1 -- 172.21.15.42:0/3507739758 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32b00ef0b0 con 0x5576f25561b0 2024-08-30T09:02:59.113 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.111+0000 7f32c31d9740 1 --2- 172.21.15.42:0/3507739758 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5576f25561b0 0x5576f28f2400 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f32b0006010 tx=0x7f32b0103370 comp rx=0 tx=0).stop 2024-08-30T09:02:59.114 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.111+0000 7f32c31d9740 1 -- 172.21.15.42:0/3507739758 shutdown_connections 2024-08-30T09:02:59.115 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.111+0000 7f32c31d9740 1 --2- 172.21.15.42:0/3507739758 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5576f24ea1c0 0x5576f2904440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:59.115 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.111+0000 7f32c31d9740 1 --2- 172.21.15.42:0/3507739758 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5576f25561b0 0x5576f28f2400 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:59.115 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.111+0000 7f32c31d9740 1 --2- 172.21.15.42:0/3507739758 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5576f25354d0 0x5576f28f9060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:59.115 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.111+0000 7f32c31d9740 1 -- 172.21.15.42:0/3507739758 >> 172.21.15.42:0/3507739758 conn(0x5576f254a300 msgr2=0x5576f28fcfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:02:59.117 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.116+0000 7f32c31d9740 1 -- 172.21.15.42:0/3507739758 shutdown_connections 2024-08-30T09:02:59.118 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.116+0000 7f32c31d9740 1 -- 172.21.15.42:0/3507739758 wait complete. 2024-08-30T09:02:59.118 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.116+0000 7f32c31d9740 1 -- start start 2024-08-30T09:02:59.118 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.116+0000 7f32c31d9740 1 Processor -- start 2024-08-30T09:02:59.119 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.117+0000 7f32c31d9740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5576f24ea1c0 0x5576f290b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:59.120 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.117+0000 7f32c31d9740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5576f25354d0 0x5576f290b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:59.120 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.118+0000 7f32b9f30640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5576f24ea1c0 0x5576f290b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:02:59.120 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.118+0000 7f32b9f30640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5576f24ea1c0 0x5576f290b210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:52112/0 (socket says 172.21.15.42:52112) 2024-08-30T09:02:59.120 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.118+0000 7f32b9f30640 1 -- 172.21.15.42:0/4047473977 learned_addr learned my addr 172.21.15.42:0/4047473977 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:02:59.120 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.118+0000 7f32c31d9740 1 --2- 172.21.15.42:0/4047473977 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5576f25561b0 0x5576f290c360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:59.120 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.118+0000 7f32c31d9740 1 -- 172.21.15.42:0/4047473977 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f329800e2f0 con 0x5576f25561b0 2024-08-30T09:02:59.120 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.118+0000 7f32c31d9740 1 -- 172.21.15.42:0/4047473977 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f329800f760 con 0x5576f24ea1c0 2024-08-30T09:02:59.120 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.118+0000 7f32c31d9740 1 -- 172.21.15.42:0/4047473977 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f329800e9c0 con 0x5576f25354d0 2024-08-30T09:02:59.121 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.118+0000 7f32c31d9740 10 client.0 _ll_register_callbacks cb 0x7f32ac009c40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:02:59.121 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.119+0000 7f32ba731640 1 --2- 172.21.15.42:0/4047473977 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5576f25561b0 0x5576f290c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:02:59.121 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.119+0000 7f32b9f30640 1 -- 172.21.15.42:0/4047473977 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5576f25354d0 msgr2=0x5576f290b8d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-30T09:02:59.121 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.119+0000 7f32b9f30640 1 --2- 172.21.15.42:0/4047473977 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5576f25354d0 0x5576f290b8d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:59.121 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.119+0000 7f32b9f30640 1 -- 172.21.15.42:0/4047473977 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5576f25561b0 msgr2=0x5576f290c360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:02:59.122 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.119+0000 7f32b9f30640 1 --2- 172.21.15.42:0/4047473977 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5576f25561b0 0x5576f290c360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:59.122 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.119+0000 7f32b9f30640 1 -- 172.21.15.42:0/4047473977 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5576f290cac0 con 0x5576f24ea1c0 2024-08-30T09:02:59.122 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.119+0000 7f32ba731640 1 --2- 172.21.15.42:0/4047473977 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5576f25561b0 0x5576f290c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-30T09:02:59.122 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.119+0000 7f32b9f30640 1 --2- 172.21.15.42:0/4047473977 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5576f24ea1c0 0x5576f290b210 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f32ac000f80 tx=0x7f32ac011700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=627912f4ebb4456d server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:02:59.122 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.119+0000 7f32c31d9740 1 -- 172.21.15.42:0/4047473977 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f32b00d0c40 con 0x5576f24ea1c0 2024-08-30T09:02:59.123 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.120+0000 7f329d7fa640 10 client.25517 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-30T09:02:59.123 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.120+0000 7f329d7fa640 1 -- 172.21.15.42:0/4047473977 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32ac022030 con 0x5576f24ea1c0 2024-08-30T09:02:59.123 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.120+0000 7f329d7fa640 1 -- 172.21.15.42:0/4047473977 <== mon.2 v2:172.21.15.167:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f32ac00f020 con 0x5576f24ea1c0 2024-08-30T09:02:59.123 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.121+0000 7f329d7fa640 1 -- 172.21.15.42:0/4047473977 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f32ac01f040 con 0x5576f24ea1c0 2024-08-30T09:02:59.123 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.121+0000 7f32c31d9740 10 client.25517 fetch_fsmap learned FSMap version 29 2024-08-30T09:02:59.123 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.121+0000 7f32c31d9740 1 -- 172.21.15.42:0/4047473977 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({fsmap=29}) -- 0x5576f290dcd0 con 0x5576f24ea1c0 2024-08-30T09:02:59.123 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.121+0000 7f329d7fa640 1 -- 172.21.15.42:0/4047473977 <== mon.2 v2:172.21.15.167:3300/0 4 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f32ac011030 con 0x5576f24ea1c0 2024-08-30T09:02:59.124 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.121+0000 7f32c31d9740 10 client.25517 fetch_fsmap finished waiting for FSMap version 29 2024-08-30T09:02:59.124 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.121+0000 7f32c31d9740 10 client.25517 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-30T09:02:59.124 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.121+0000 7f32c31d9740 20 client.25517 populate_metadata read hostname 'smithi042' 2024-08-30T09:02:59.124 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.121+0000 7f32c31d9740 1 --2- 172.21.15.42:0/4047473977 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x5576f297e050 0x5576f2980470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:02:59.124 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.121+0000 7f32c31d9740 4 client.25517 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-30T09:02:59.124 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.121+0000 7f32c31d9740 1 -- 172.21.15.42:0/4047473977 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f32b00d0c40 con 0x5576f297e050 2024-08-30T09:02:59.124 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.122+0000 7f32b972f640 1 --2- 172.21.15.42:0/4047473977 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x5576f297e050 0x5576f2980470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:02:59.127 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.125+0000 7f32b972f640 1 --2- 172.21.15.42:0/4047473977 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x5576f297e050 0x5576f2980470 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e0973dde506434ae server_cookie=322a3fc81eda7e91 in_seq=0 out_seq=0 2024-08-30T09:02:59.128 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.126+0000 7f329d7fa640 10 client.25517 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:02:59.143 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.140+0000 7f329d7fa640 1 -- 172.21.15.42:0/4047473977 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f32b00d0c40 con 0x5576f297e050 2024-08-30T09:02:59.143 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.141+0000 7f329d7fa640 10 client.25517 handle_command_reply: tid=0 2024-08-30T09:02:59.143 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.141+0000 7f32c31d9740 1 client.25517 shutdown 2024-08-30T09:02:59.143 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.141+0000 7f32c31d9740 1 -- 172.21.15.42:0/4047473977 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5576f24ea1c0 msgr2=0x5576f290b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:02:59.143 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.141+0000 7f32c31d9740 1 --2- 172.21.15.42:0/4047473977 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5576f24ea1c0 0x5576f290b210 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f32ac000f80 tx=0x7f32ac011700 comp rx=0 tx=0).stop 2024-08-30T09:02:59.143 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.141+0000 7f32c31d9740 1 -- 172.21.15.42:0/4047473977 shutdown_connections 2024-08-30T09:02:59.143 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.141+0000 7f32c31d9740 1 --2- 172.21.15.42:0/4047473977 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x5576f297e050 0x5576f2980470 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:59.144 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.141+0000 7f32c31d9740 1 --2- 172.21.15.42:0/4047473977 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5576f25354d0 0x5576f290b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:59.144 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.141+0000 7f329d7fa640 0 client.25517 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:02:59.144 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.141+0000 7f32c31d9740 1 --2- 172.21.15.42:0/4047473977 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5576f25561b0 0x5576f290c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:59.144 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.141+0000 7f32c31d9740 1 --2- 172.21.15.42:0/4047473977 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5576f24ea1c0 0x5576f290b210 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:59.144 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.141+0000 7f32c31d9740 1 -- 172.21.15.42:0/4047473977 >> 172.21.15.42:0/4047473977 conn(0x5576f254a300 msgr2=0x5576f2904440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:02:59.144 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.141+0000 7f32c31d9740 1 -- 172.21.15.42:0/4047473977 shutdown_connections 2024-08-30T09:02:59.144 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.141+0000 7f32c31d9740 1 -- 172.21.15.42:0/4047473977 wait complete. 2024-08-30T09:02:59.144 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.141+0000 7f32c31d9740 20 client.25517 trim_cache size 0 max 16384 2024-08-30T09:02:59.144 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-08-30T09:02:59.145 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2024-08-30T09:02:59.145 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-08-30T09:02:59.145 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-08-30T09:02:59.145 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.142+0000 7f32bc194640 1 -- 172.21.15.42:0/3306136158 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f32a00c5d40 msgr2=0x7f32a00c8250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:02:59.145 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.142+0000 7f32bc194640 1 --2- 172.21.15.42:0/3306136158 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f32a00c5d40 0x7f32a00c8250 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f32a4014580 tx=0x7f32a4014980 comp rx=0 tx=0).stop 2024-08-30T09:02:59.146 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.142+0000 7f32bc194640 1 -- 172.21.15.42:0/3306136158 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f32b4076e90 msgr2=0x7f32b40747b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:02:59.146 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.142+0000 7f32bc194640 1 --2- 172.21.15.42:0/3306136158 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f32b4076e90 0x7f32b40747b0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f32b0000c00 tx=0x7f32b00309d0 comp rx=0 tx=0).stop 2024-08-30T09:02:59.146 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.142+0000 7f32bc194640 1 -- 172.21.15.42:0/3306136158 shutdown_connections 2024-08-30T09:02:59.146 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.142+0000 7f32bc194640 1 --2- 172.21.15.42:0/3306136158 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f32a00c5d40 0x7f32a00c8250 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:59.146 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.142+0000 7f32bc194640 1 --2- 172.21.15.42:0/3306136158 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f32b410fc10 0x7f32b4072d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:59.146 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.142+0000 7f32bc194640 1 --2- 172.21.15.42:0/3306136158 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f32b4076e90 0x7f32b40747b0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:59.146 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.142+0000 7f32bc194640 1 --2- 172.21.15.42:0/3306136158 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f32b4075a90 0x7f32b4074160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:02:59.146 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.142+0000 7f32bc194640 1 -- 172.21.15.42:0/3306136158 >> 172.21.15.42:0/3306136158 conn(0x7f32b40fedd0 msgr2=0x7f32b40ff490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:02:59.146 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.142+0000 7f32bc194640 1 -- 172.21.15.42:0/3306136158 shutdown_connections 2024-08-30T09:02:59.146 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:02:59.143+0000 7f32bc194640 1 -- 172.21.15.42:0/3306136158 wait complete. 2024-08-30T09:03:00.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:03:00 smithi042 ceph-mon[18652]: pgmap v440: 129 pgs: 129 active+clean; 1.1 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 6.8 MiB/s wr, 670 op/s 2024-08-30T09:03:00.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:03:00 smithi190 ceph-mon[22001]: pgmap v440: 129 pgs: 129 active+clean; 1.1 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 6.8 MiB/s wr, 670 op/s 2024-08-30T09:03:00.576 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:03:00 smithi167 ceph-mon[22063]: pgmap v440: 129 pgs: 129 active+clean; 1.1 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 6.8 MiB/s wr, 670 op/s 2024-08-30T09:03:00.953 INFO:teuthology.orchestra.run.smithi042.stderr:progress: 135.0 s, 2159.0 tps, lat 46.842 ms stddev 61.232 2024-08-30T09:03:01.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.037+0000 7fe3b3fff640 1 -- 172.21.15.42:0/1124464386 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe3ac09c3b0 msgr2=0x7fe3ac0a2be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:03:01.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.037+0000 7fe3b3fff640 1 --2- 172.21.15.42:0/1124464386 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe3ac09c3b0 0x7fe3ac0a2be0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fe3b4069c20 tx=0x7fe3b4097da0 comp rx=0 tx=0).stop 2024-08-30T09:03:01.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.037+0000 7fe3b3fff640 1 -- 172.21.15.42:0/1124464386 shutdown_connections 2024-08-30T09:03:01.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.037+0000 7fe3b3fff640 1 --2- 172.21.15.42:0/1124464386 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe3ac09c3b0 0x7fe3ac0a2be0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:01.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.037+0000 7fe3b3fff640 1 --2- 172.21.15.42:0/1124464386 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe3ac09ba60 0x7fe3ac09be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:01.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.037+0000 7fe3b3fff640 1 --2- 172.21.15.42:0/1124464386 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe3ac096120 0x7fe3ac096530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:01.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.037+0000 7fe3b3fff640 1 -- 172.21.15.42:0/1124464386 >> 172.21.15.42:0/1124464386 conn(0x7fe3ac00ba80 msgr2=0x7fe3ac00bec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:03:01.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.037+0000 7fe3b3fff640 1 -- 172.21.15.42:0/1124464386 shutdown_connections 2024-08-30T09:03:01.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.037+0000 7fe3b3fff640 1 -- 172.21.15.42:0/1124464386 wait complete. 2024-08-30T09:03:01.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.038+0000 7fe3b3fff640 1 Processor -- start 2024-08-30T09:03:01.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.038+0000 7fe3b3fff640 1 -- start start 2024-08-30T09:03:01.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.039+0000 7fe3b3fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe3ac096120 0x7fe3ac0092c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:01.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.039+0000 7fe3b3fff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe3ac09ba60 0x7fe3ac0078b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:01.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.039+0000 7fe3b3fff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe3ac09c3b0 0x7fe3ac007df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:01.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.039+0000 7fe3b3fff640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fe3ac0a5fb0 con 0x7fe3ac096120 2024-08-30T09:03:01.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.039+0000 7fe3b37fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe3ac09c3b0 0x7fe3ac007df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:03:01.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.039+0000 7fe3b37fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe3ac09c3b0 0x7fe3ac007df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:49910/0 (socket says 172.21.15.42:49910) 2024-08-30T09:03:01.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.039+0000 7fe3b37fe640 1 -- 172.21.15.42:0/218819350 learned_addr learned my addr 172.21.15.42:0/218819350 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:03:01.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.039+0000 7fe3b3fff640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe3ac0a5e30 con 0x7fe3ac09ba60 2024-08-30T09:03:01.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.039+0000 7fe3b3fff640 1 -- 172.21.15.42:0/218819350 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fe3ac0a6130 con 0x7fe3ac09c3b0 2024-08-30T09:03:01.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.040+0000 7fe3b37fe640 1 -- 172.21.15.42:0/218819350 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe3ac09ba60 msgr2=0x7fe3ac0078b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:03:01.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.040+0000 7fe3b37fe640 1 --2- 172.21.15.42:0/218819350 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe3ac09ba60 0x7fe3ac0078b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:01.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.040+0000 7fe3b37fe640 1 -- 172.21.15.42:0/218819350 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe3ac096120 msgr2=0x7fe3ac0092c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:03:01.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.040+0000 7fe3b37fe640 1 --2- 172.21.15.42:0/218819350 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe3ac096120 0x7fe3ac0092c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:01.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.040+0000 7fe3b37fe640 1 -- 172.21.15.42:0/218819350 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3ac008610 con 0x7fe3ac09c3b0 2024-08-30T09:03:01.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.040+0000 7fe3b37fe640 1 --2- 172.21.15.42:0/218819350 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe3ac09c3b0 0x7fe3ac007df0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fe3b4000b60 tx=0x7fe3b407c020 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:03:01.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.041+0000 7fe3b8958640 1 -- 172.21.15.42:0/218819350 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3b40ae070 con 0x7fe3ac09c3b0 2024-08-30T09:03:01.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.041+0000 7fe3b8958640 1 -- 172.21.15.42:0/218819350 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe3b409fd80 con 0x7fe3ac09c3b0 2024-08-30T09:03:01.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.041+0000 7fe3b8958640 1 -- 172.21.15.42:0/218819350 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3b40a9410 con 0x7fe3ac09c3b0 2024-08-30T09:03:01.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.041+0000 7fe3b3fff640 1 -- 172.21.15.42:0/218819350 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3ac130a80 con 0x7fe3ac09c3b0 2024-08-30T09:03:01.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.041+0000 7fe3b3fff640 1 -- 172.21.15.42:0/218819350 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3ac130e20 con 0x7fe3ac09c3b0 2024-08-30T09:03:01.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.045+0000 7fe3bfecf740 1 Processor -- start 2024-08-30T09:03:01.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.046+0000 7fe3b8958640 1 -- 172.21.15.42:0/218819350 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7fe3b40b1070 con 0x7fe3ac09c3b0 2024-08-30T09:03:01.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.047+0000 7fe3bfecf740 1 -- start start 2024-08-30T09:03:01.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.047+0000 7fe3bfecf740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a9d292f070 0x55a9d2cd4460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:01.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.047+0000 7fe3bfecf740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a9d29061b0 0x55a9d2cd49a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:01.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.047+0000 7fe3bfecf740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a9d290e390 0x55a9d2cd4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:01.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.047+0000 7fe3bfecf740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55a9d2c4b630 con 0x55a9d292f070 2024-08-30T09:03:01.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.047+0000 7fe3bfecf740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55a9d2c3abc0 con 0x55a9d29061b0 2024-08-30T09:03:01.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.047+0000 7fe3bfecf740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55a9d2c3eb70 con 0x55a9d290e390 2024-08-30T09:03:01.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.048+0000 7fe3b2ffd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a9d29061b0 0x55a9d2cd49a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:03:01.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.048+0000 7fe3b2ffd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a9d29061b0 0x55a9d2cd49a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:52138/0 (socket says 172.21.15.42:52138) 2024-08-30T09:03:01.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.048+0000 7fe3b2ffd640 1 -- 172.21.15.42:0/3701557047 learned_addr learned my addr 172.21.15.42:0/3701557047 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:03:01.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.048+0000 7fe3b27fc640 1 --2- 172.21.15.42:0/3701557047 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a9d292f070 0x55a9d2cd4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:03:01.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.048+0000 7fe3b27fc640 1 --2- 172.21.15.42:0/3701557047 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a9d290e390 0x55a9d2cd4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:03:01.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.048+0000 7fe3b2ffd640 1 -- 172.21.15.42:0/3701557047 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a9d290e390 msgr2=0x55a9d2cd4ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:03:01.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.048+0000 7fe3b2ffd640 1 --2- 172.21.15.42:0/3701557047 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a9d290e390 0x55a9d2cd4ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:01.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.048+0000 7fe3b2ffd640 1 -- 172.21.15.42:0/3701557047 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a9d292f070 msgr2=0x55a9d2cd4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:03:01.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.048+0000 7fe3b2ffd640 1 --2- 172.21.15.42:0/3701557047 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a9d292f070 0x55a9d2cd4460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:01.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.048+0000 7fe3b2ffd640 1 -- 172.21.15.42:0/3701557047 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a9d299a650 con 0x55a9d29061b0 2024-08-30T09:03:01.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.048+0000 7fe3b2ffd640 1 --2- 172.21.15.42:0/3701557047 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a9d29061b0 0x55a9d2cd49a0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fe3a4009b30 tx=0x7fe3a400e350 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3a16cfeaf6fe19d0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:03:01.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.049+0000 7fe399ffb640 1 -- 172.21.15.42:0/3701557047 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3a400ed20 con 0x55a9d29061b0 2024-08-30T09:03:01.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.049+0000 7fe3b8958640 1 --2- 172.21.15.42:0/218819350 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fe39c0b4c70 0x7fe39c0b7180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:01.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.049+0000 7fe399ffb640 1 -- 172.21.15.42:0/3701557047 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe3a40035f0 con 0x55a9d29061b0 2024-08-30T09:03:01.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.049+0000 7fe399ffb640 1 -- 172.21.15.42:0/3701557047 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3a40044e0 con 0x55a9d29061b0 2024-08-30T09:03:01.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.049+0000 7fe3b8958640 1 -- 172.21.15.42:0/218819350 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7fe3b4172430 con 0x7fe3ac09c3b0 2024-08-30T09:03:01.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.049+0000 7fe3b27fc640 1 --2- 172.21.15.42:0/218819350 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fe39c0b4c70 0x7fe39c0b7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:03:01.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.050+0000 7fe3b27fc640 1 --2- 172.21.15.42:0/218819350 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fe39c0b4c70 0x7fe39c0b7180 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fe3a80058c0 tx=0x7fe3a8015040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:03:01.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.050+0000 7fe3bfecf740 1 -- 172.21.15.42:0/3701557047 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a9d29061b0 msgr2=0x55a9d2cd49a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:03:01.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.050+0000 7fe3bfecf740 1 --2- 172.21.15.42:0/3701557047 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a9d29061b0 0x55a9d2cd49a0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fe3a4009b30 tx=0x7fe3a400e350 comp rx=0 tx=0).stop 2024-08-30T09:03:01.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.050+0000 7fe3bfecf740 1 -- 172.21.15.42:0/3701557047 shutdown_connections 2024-08-30T09:03:01.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.050+0000 7fe3bfecf740 1 --2- 172.21.15.42:0/3701557047 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a9d290e390 0x55a9d2cd4ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:01.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.050+0000 7fe3bfecf740 1 --2- 172.21.15.42:0/3701557047 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a9d29061b0 0x55a9d2cd49a0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:01.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.051+0000 7fe3bfecf740 1 --2- 172.21.15.42:0/3701557047 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a9d292f070 0x55a9d2cd4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:01.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.051+0000 7fe3bfecf740 1 -- 172.21.15.42:0/3701557047 >> 172.21.15.42:0/3701557047 conn(0x55a9d2923460 msgr2=0x55a9d2cd2350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:03:01.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.051+0000 7fe3bfecf740 1 -- 172.21.15.42:0/3701557047 shutdown_connections 2024-08-30T09:03:01.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.051+0000 7fe3bfecf740 1 -- 172.21.15.42:0/3701557047 wait complete. 2024-08-30T09:03:01.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.051+0000 7fe3bfecf740 1 -- start start 2024-08-30T09:03:01.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.051+0000 7fe3bfecf740 1 Processor -- start 2024-08-30T09:03:01.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.053+0000 7fe3bfecf740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a9d29061b0 0x55a9d2cdfb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:01.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.053+0000 7fe3bfecf740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a9d290e390 0x55a9d2ce00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:01.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.054+0000 7fe3bfecf740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a9d292f070 0x55a9d2ce0640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:01.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.054+0000 7fe3bfecf740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55a9d2a9d700 con 0x55a9d292f070 2024-08-30T09:03:01.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.054+0000 7fe3bfecf740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55a9d2bc1700 con 0x55a9d290e390 2024-08-30T09:03:01.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.054+0000 7fe3bfecf740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55a9d2ce2c40 con 0x55a9d29061b0 2024-08-30T09:03:01.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.054+0000 7fe3bfecf740 10 client.0 _ll_register_callbacks cb 0x55a9d2bf0dc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:03:01.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.054+0000 7fe3b2ffd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a9d29061b0 0x55a9d2cdfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:03:01.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.054+0000 7fe3b2ffd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a9d29061b0 0x55a9d2cdfb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:49924/0 (socket says 172.21.15.42:49924) 2024-08-30T09:03:01.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.054+0000 7fe3b2ffd640 1 -- 172.21.15.42:0/1835915520 learned_addr learned my addr 172.21.15.42:0/1835915520 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:03:01.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.054+0000 7fe3b37fe640 1 --2- 172.21.15.42:0/1835915520 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a9d292f070 0x55a9d2ce0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:03:01.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.054+0000 7fe3b27fc640 1 --2- 172.21.15.42:0/1835915520 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a9d290e390 0x55a9d2ce00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:03:01.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.054+0000 7fe3b2ffd640 1 -- 172.21.15.42:0/1835915520 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a9d290e390 msgr2=0x55a9d2ce00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:03:01.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.054+0000 7fe3b2ffd640 1 --2- 172.21.15.42:0/1835915520 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a9d290e390 0x55a9d2ce00c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:01.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.054+0000 7fe3b2ffd640 1 -- 172.21.15.42:0/1835915520 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a9d292f070 msgr2=0x55a9d2ce0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:03:01.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.054+0000 7fe3b2ffd640 1 --2- 172.21.15.42:0/1835915520 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a9d292f070 0x55a9d2ce0640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:01.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.054+0000 7fe3b2ffd640 1 -- 172.21.15.42:0/1835915520 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a9d2bfa8b0 con 0x55a9d29061b0 2024-08-30T09:03:01.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.054+0000 7fe3b37fe640 1 --2- 172.21.15.42:0/1835915520 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a9d292f070 0x55a9d2ce0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-30T09:03:01.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.054+0000 7fe3b2ffd640 1 --2- 172.21.15.42:0/1835915520 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a9d29061b0 0x55a9d2cdfb80 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fe3a4006010 tx=0x7fe3a4004250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cb2182305d51d45f server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:03:01.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.054+0000 7fe392ffd640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-08-30T09:03:01.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.055+0000 7fe392ffd640 1 -- 172.21.15.42:0/1835915520 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3a4026020 con 0x55a9d29061b0 2024-08-30T09:03:01.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.055+0000 7fe392ffd640 1 -- 172.21.15.42:0/1835915520 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7fe3a4010430 con 0x55a9d29061b0 2024-08-30T09:03:01.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.055+0000 7fe3bfecf740 1 -- 172.21.15.42:0/1835915520 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe3a40035f0 con 0x55a9d29061b0 2024-08-30T09:03:01.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.055+0000 7fe392ffd640 1 -- 172.21.15.42:0/1835915520 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe3a4031050 con 0x55a9d29061b0 2024-08-30T09:03:01.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.056+0000 7fe3bfecf740 10 client.25534 fetch_fsmap learned FSMap version 29 2024-08-30T09:03:01.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.056+0000 7fe3bfecf740 1 -- 172.21.15.42:0/1835915520 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a9d2cee970 con 0x55a9d29061b0 2024-08-30T09:03:01.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.056+0000 7fe392ffd640 1 -- 172.21.15.42:0/1835915520 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe3a40035f0 con 0x55a9d29061b0 2024-08-30T09:03:01.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.057+0000 7fe392ffd640 1 -- 172.21.15.42:0/1835915520 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fe3a4018420 con 0x55a9d29061b0 2024-08-30T09:03:01.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.057+0000 7fe3bfecf740 10 client.25534 fetch_fsmap finished waiting for FSMap version 29 2024-08-30T09:03:01.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.057+0000 7fe3bfecf740 10 client.25534 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-30T09:03:01.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.057+0000 7fe3bfecf740 20 client.25534 populate_metadata read hostname 'smithi042' 2024-08-30T09:03:01.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.057+0000 7fe3bfecf740 1 --2- 172.21.15.42:0/1835915520 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55a9d28ed9f0 0x55a9d2cf2320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:01.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.057+0000 7fe3bfecf740 4 client.25534 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-30T09:03:01.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.057+0000 7fe3bfecf740 1 -- 172.21.15.42:0/1835915520 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a9d2c4d580 con 0x55a9d28ed9f0 2024-08-30T09:03:01.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.057+0000 7fe3b27fc640 1 --2- 172.21.15.42:0/1835915520 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55a9d28ed9f0 0x55a9d2cf2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:03:01.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.060+0000 7fe3b27fc640 1 --2- 172.21.15.42:0/1835915520 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55a9d28ed9f0 0x55a9d2cf2320 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=35853e0ea74615e3 server_cookie=e1400105dd09d546 in_seq=0 out_seq=0 2024-08-30T09:03:01.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.060+0000 7fe392ffd640 10 client.25534 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:03:01.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.060+0000 7fe392ffd640 1 -- 172.21.15.42:0/1835915520 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a9d2c4d580 con 0x55a9d28ed9f0 2024-08-30T09:03:01.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.060+0000 7fe392ffd640 10 client.25534 handle_command_reply: tid=0 2024-08-30T09:03:01.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.060+0000 7fe3bfecf740 1 client.25534 shutdown 2024-08-30T09:03:01.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.061+0000 7fe3bfecf740 1 -- 172.21.15.42:0/1835915520 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a9d29061b0 msgr2=0x55a9d2cdfb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:03:01.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.061+0000 7fe3bfecf740 1 --2- 172.21.15.42:0/1835915520 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a9d29061b0 0x55a9d2cdfb80 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fe3a4006010 tx=0x7fe3a4004250 comp rx=0 tx=0).stop 2024-08-30T09:03:01.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.061+0000 7fe3bfecf740 1 -- 172.21.15.42:0/1835915520 shutdown_connections 2024-08-30T09:03:01.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.061+0000 7fe3bfecf740 1 --2- 172.21.15.42:0/1835915520 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55a9d28ed9f0 0x55a9d2cf2320 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:01.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.061+0000 7fe3bfecf740 1 --2- 172.21.15.42:0/1835915520 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a9d292f070 0x55a9d2ce0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:01.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.061+0000 7fe3bfecf740 1 --2- 172.21.15.42:0/1835915520 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a9d290e390 0x55a9d2ce00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:01.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.061+0000 7fe3bfecf740 1 --2- 172.21.15.42:0/1835915520 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a9d29061b0 0x55a9d2cdfb80 secure :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fe3a4006010 tx=0x7fe3a4004250 comp rx=0 tx=0).stop 2024-08-30T09:03:01.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.061+0000 7fe3bfecf740 1 -- 172.21.15.42:0/1835915520 >> 172.21.15.42:0/1835915520 conn(0x55a9d2923460 msgr2=0x55a9d2cd75f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:03:01.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.061+0000 7fe392ffd640 0 client.25534 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:03:01.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.062+0000 7fe3bfecf740 1 -- 172.21.15.42:0/1835915520 shutdown_connections 2024-08-30T09:03:01.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.062+0000 7fe3bfecf740 1 -- 172.21.15.42:0/1835915520 wait complete. 2024-08-30T09:03:01.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.062+0000 7fe3bfecf740 20 client.25534 trim_cache size 0 max 16384 2024-08-30T09:03:01.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.091+0000 7fe3bfecf740 1 Processor -- start 2024-08-30T09:03:01.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.091+0000 7fe3bfecf740 1 -- start start 2024-08-30T09:03:01.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.092+0000 7fe3bfecf740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a9d290e390 0x55a9d2cdedc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:01.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.092+0000 7fe3bfecf740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a9d292f070 0x55a9d2ceb5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:01.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.092+0000 7fe3bfecf740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a9d28c2da0 0x55a9d2cd5490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:01.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.092+0000 7fe3b2ffd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a9d290e390 0x55a9d2cdedc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:03:01.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.092+0000 7fe3b27fc640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a9d292f070 0x55a9d2ceb5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:03:01.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.092+0000 7fe3b2ffd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a9d290e390 0x55a9d2cdedc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:52164/0 (socket says 172.21.15.42:52164) 2024-08-30T09:03:01.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.092+0000 7fe3b2ffd640 1 -- 172.21.15.42:0/1076429024 learned_addr learned my addr 172.21.15.42:0/1076429024 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:03:01.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.092+0000 7fe3b37fe640 1 --2- 172.21.15.42:0/1076429024 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a9d28c2da0 0x55a9d2cd5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:03:01.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.092+0000 7fe3bfecf740 1 -- 172.21.15.42:0/1076429024 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fe3ac154660 con 0x55a9d28c2da0 2024-08-30T09:03:01.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.092+0000 7fe3bfecf740 1 -- 172.21.15.42:0/1076429024 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe3ac151f30 con 0x55a9d290e390 2024-08-30T09:03:01.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.092+0000 7fe3bfecf740 1 -- 172.21.15.42:0/1076429024 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fe3ac138ec0 con 0x55a9d292f070 2024-08-30T09:03:01.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.093+0000 7fe3b27fc640 1 -- 172.21.15.42:0/1076429024 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a9d290e390 msgr2=0x55a9d2cdedc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:03:01.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.093+0000 7fe3b27fc640 1 --2- 172.21.15.42:0/1076429024 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a9d290e390 0x55a9d2cdedc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:01.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.093+0000 7fe3b27fc640 1 -- 172.21.15.42:0/1076429024 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a9d28c2da0 msgr2=0x55a9d2cd5490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:03:01.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.093+0000 7fe3b27fc640 1 --2- 172.21.15.42:0/1076429024 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a9d28c2da0 0x55a9d2cd5490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:01.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.093+0000 7fe3b27fc640 1 -- 172.21.15.42:0/1076429024 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a9d2cd2c40 con 0x55a9d292f070 2024-08-30T09:03:01.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.093+0000 7fe3b27fc640 1 --2- 172.21.15.42:0/1076429024 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a9d292f070 0x55a9d2ceb5c0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x55a9d2cf2a20 tx=0x7fe3a8005850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3906b267693b4636 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:03:01.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.093+0000 7fe3997fa640 1 -- 172.21.15.42:0/1076429024 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3a8004410 con 0x55a9d292f070 2024-08-30T09:03:01.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.093+0000 7fe3997fa640 1 -- 172.21.15.42:0/1076429024 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe3a80054d0 con 0x55a9d292f070 2024-08-30T09:03:01.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.093+0000 7fe3997fa640 1 -- 172.21.15.42:0/1076429024 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3a8010430 con 0x55a9d292f070 2024-08-30T09:03:01.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.095+0000 7fe3bfecf740 1 -- 172.21.15.42:0/1076429024 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a9d292f070 msgr2=0x55a9d2ceb5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:03:01.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.095+0000 7fe3bfecf740 1 --2- 172.21.15.42:0/1076429024 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a9d292f070 0x55a9d2ceb5c0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x55a9d2cf2a20 tx=0x7fe3a8005850 comp rx=0 tx=0).stop 2024-08-30T09:03:01.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.095+0000 7fe3bfecf740 1 -- 172.21.15.42:0/1076429024 shutdown_connections 2024-08-30T09:03:01.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.095+0000 7fe3bfecf740 1 --2- 172.21.15.42:0/1076429024 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a9d292f070 0x55a9d2ceb5c0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:01.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.095+0000 7fe3bfecf740 1 --2- 172.21.15.42:0/1076429024 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a9d28c2da0 0x55a9d2cd5490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:01.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.095+0000 7fe3bfecf740 1 --2- 172.21.15.42:0/1076429024 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a9d290e390 0x55a9d2cdedc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:01.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.095+0000 7fe3bfecf740 1 -- 172.21.15.42:0/1076429024 >> 172.21.15.42:0/1076429024 conn(0x55a9d2923460 msgr2=0x55a9d2ceaeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:03:01.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.095+0000 7fe3bfecf740 1 -- 172.21.15.42:0/1076429024 shutdown_connections 2024-08-30T09:03:01.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.096+0000 7fe3bfecf740 1 -- 172.21.15.42:0/1076429024 wait complete. 2024-08-30T09:03:01.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.096+0000 7fe3bfecf740 1 -- start start 2024-08-30T09:03:01.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.097+0000 7fe3bfecf740 1 Processor -- start 2024-08-30T09:03:01.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.097+0000 7fe3bfecf740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a9d28c2da0 0x55a9d2cda3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:01.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.098+0000 7fe3bfecf740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a9d290e390 0x55a9d2ce7f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:01.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.098+0000 7fe3bfecf740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a9d292f070 0x55a9d2ce8660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:01.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.098+0000 7fe3b37fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a9d292f070 0x55a9d2ce8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:03:01.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.098+0000 7fe3b37fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a9d292f070 0x55a9d2ce8660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:52166/0 (socket says 172.21.15.42:52166) 2024-08-30T09:03:01.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.098+0000 7fe3b2ffd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a9d28c2da0 0x55a9d2cda3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:03:01.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.098+0000 7fe3b37fe640 1 -- 172.21.15.42:0/2837305062 learned_addr learned my addr 172.21.15.42:0/2837305062 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:03:01.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.098+0000 7fe3b27fc640 1 --2- 172.21.15.42:0/2837305062 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a9d290e390 0x55a9d2ce7f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:03:01.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.098+0000 7fe3bfecf740 1 -- 172.21.15.42:0/2837305062 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fe3ac1503f0 con 0x55a9d28c2da0 2024-08-30T09:03:01.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.099+0000 7fe3bfecf740 1 -- 172.21.15.42:0/2837305062 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe3ac151860 con 0x55a9d292f070 2024-08-30T09:03:01.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.099+0000 7fe3bfecf740 1 -- 172.21.15.42:0/2837305062 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fe3ac150ac0 con 0x55a9d290e390 2024-08-30T09:03:01.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.099+0000 7fe3bfecf740 10 client.0 _ll_register_callbacks cb 0x7fe3a8009920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:03:01.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.099+0000 7fe3b27fc640 1 -- 172.21.15.42:0/2837305062 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a9d292f070 msgr2=0x55a9d2ce8660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:03:01.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.099+0000 7fe3b27fc640 1 --2- 172.21.15.42:0/2837305062 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a9d292f070 0x55a9d2ce8660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:01.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.099+0000 7fe3b27fc640 1 -- 172.21.15.42:0/2837305062 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a9d28c2da0 msgr2=0x55a9d2cda3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:03:01.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.099+0000 7fe3b27fc640 1 --2- 172.21.15.42:0/2837305062 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a9d28c2da0 0x55a9d2cda3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:01.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.099+0000 7fe3b27fc640 1 -- 172.21.15.42:0/2837305062 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a9d2ce2ee0 con 0x55a9d290e390 2024-08-30T09:03:01.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.099+0000 7fe3b27fc640 1 --2- 172.21.15.42:0/2837305062 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a9d290e390 0x55a9d2ce7f50 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fe3a8004230 tx=0x7fe3a80046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2a17e372644c0513 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:03:01.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.100+0000 7fe392ffd640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-08-30T09:03:01.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.100+0000 7fe392ffd640 1 -- 172.21.15.42:0/2837305062 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3a8035060 con 0x55a9d290e390 2024-08-30T09:03:01.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.100+0000 7fe392ffd640 1 -- 172.21.15.42:0/2837305062 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7fe3a8004c80 con 0x55a9d290e390 2024-08-30T09:03:01.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.100+0000 7fe3bfecf740 1 -- 172.21.15.42:0/2837305062 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe3a80054d0 con 0x55a9d290e390 2024-08-30T09:03:01.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.100+0000 7fe392ffd640 1 -- 172.21.15.42:0/2837305062 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe3a8004e80 con 0x55a9d290e390 2024-08-30T09:03:01.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.100+0000 7fe3bfecf740 10 client.25540 fetch_fsmap learned FSMap version 29 2024-08-30T09:03:01.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.100+0000 7fe3bfecf740 1 -- 172.21.15.42:0/2837305062 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a9d2d6df00 con 0x55a9d290e390 2024-08-30T09:03:01.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.101+0000 7fe392ffd640 1 -- 172.21.15.42:0/2837305062 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe3a80054d0 con 0x55a9d290e390 2024-08-30T09:03:01.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.101+0000 7fe392ffd640 1 -- 172.21.15.42:0/2837305062 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fe3a801eba0 con 0x55a9d290e390 2024-08-30T09:03:01.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.101+0000 7fe3bfecf740 10 client.25540 fetch_fsmap finished waiting for FSMap version 29 2024-08-30T09:03:01.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.101+0000 7fe3bfecf740 10 client.25540 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-30T09:03:01.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.101+0000 7fe3bfecf740 20 client.25540 populate_metadata read hostname 'smithi042' 2024-08-30T09:03:01.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.101+0000 7fe3bfecf740 1 --2- 172.21.15.42:0/2837305062 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55a9d2d6e580 0x55a9d2d709a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:01.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.101+0000 7fe3bfecf740 4 client.25540 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-30T09:03:01.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.101+0000 7fe3bfecf740 1 -- 172.21.15.42:0/2837305062 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a9d2c49020 con 0x55a9d2d6e580 2024-08-30T09:03:01.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.101+0000 7fe3b2ffd640 1 --2- 172.21.15.42:0/2837305062 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55a9d2d6e580 0x55a9d2d709a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:03:01.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.102+0000 7fe3b2ffd640 1 --2- 172.21.15.42:0/2837305062 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55a9d2d6e580 0x55a9d2d709a0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd34c382b14f5c23 server_cookie=7a5be28fe380f361 in_seq=0 out_seq=0 2024-08-30T09:03:01.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.102+0000 7fe392ffd640 10 client.25540 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:03:01.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.103+0000 7fe392ffd640 1 -- 172.21.15.42:0/2837305062 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+942 (crc 0 0 0) 0x55a9d2c49020 con 0x55a9d2d6e580 2024-08-30T09:03:01.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.103+0000 7fe392ffd640 10 client.25540 handle_command_reply: tid=0 2024-08-30T09:03:01.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.103+0000 7fe3bfecf740 1 client.25540 shutdown 2024-08-30T09:03:01.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.103+0000 7fe3bfecf740 1 -- 172.21.15.42:0/2837305062 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a9d290e390 msgr2=0x55a9d2ce7f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:03:01.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.103+0000 7fe3bfecf740 1 --2- 172.21.15.42:0/2837305062 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a9d290e390 0x55a9d2ce7f50 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fe3a8004230 tx=0x7fe3a80046c0 comp rx=0 tx=0).stop 2024-08-30T09:03:01.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.103+0000 7fe3bfecf740 1 -- 172.21.15.42:0/2837305062 shutdown_connections 2024-08-30T09:03:01.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.103+0000 7fe3bfecf740 1 --2- 172.21.15.42:0/2837305062 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55a9d2d6e580 0x55a9d2d709a0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:01.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.103+0000 7fe3bfecf740 1 --2- 172.21.15.42:0/2837305062 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a9d290e390 0x55a9d2ce7f50 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:01.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.103+0000 7fe3bfecf740 1 --2- 172.21.15.42:0/2837305062 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a9d292f070 0x55a9d2ce8660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:01.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.103+0000 7fe3bfecf740 1 --2- 172.21.15.42:0/2837305062 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a9d28c2da0 0x55a9d2cda3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:01.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.103+0000 7fe3bfecf740 1 -- 172.21.15.42:0/2837305062 >> 172.21.15.42:0/2837305062 conn(0x55a9d2923460 msgr2=0x55a9d2cdeb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:03:01.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.103+0000 7fe392ffd640 0 client.25540 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:03:01.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.104+0000 7fe3bfecf740 1 -- 172.21.15.42:0/2837305062 shutdown_connections 2024-08-30T09:03:01.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.104+0000 7fe3bfecf740 1 -- 172.21.15.42:0/2837305062 wait complete. 2024-08-30T09:03:01.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.104+0000 7fe3bfecf740 20 client.25540 trim_cache size 0 max 16384 2024-08-30T09:03:01.110 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-08-30T09:03:01.110 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "no active scrubs running", 2024-08-30T09:03:01.110 INFO:teuthology.orchestra.run.smithi042.stdout: "scrubs": {}, 2024-08-30T09:03:01.110 INFO:teuthology.orchestra.run.smithi042.stdout: "scrub_stats": { 2024-08-30T09:03:01.110 INFO:teuthology.orchestra.run.smithi042.stdout: "85134e4b-ab4a-4885-b07e-a3ad01eaf13d": { 2024-08-30T09:03:01.110 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:01:27.738462+0000", 2024-08-30T09:03:01.110 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:03:01.110 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:03:01.110 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:03:01.110 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:03:01.110 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:03:01.110 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:03:01.111 INFO:teuthology.orchestra.run.smithi042.stdout: "c827ad7a-44ee-4e1d-9bb3-51fb7ab00c73": { 2024-08-30T09:03:01.111 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:02:21.686517+0000", 2024-08-30T09:03:01.111 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:03:01.111 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:03:01.111 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:03:01.111 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:03:01.111 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:03:01.111 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:03:01.111 INFO:teuthology.orchestra.run.smithi042.stdout: "5e2ed104-0625-478a-a78b-584e7f4c2817": { 2024-08-30T09:03:01.111 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:00:33.928400+0000", 2024-08-30T09:03:01.111 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:03:01.111 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:03:01.111 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:03:01.112 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:03:01.112 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:03:01.112 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:03:01.112 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:03:01.112 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-08-30T09:03:01.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.105+0000 7fe3b3fff640 1 -- 172.21.15.42:0/218819350 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fe39c0b4c70 msgr2=0x7fe39c0b7180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:03:01.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.105+0000 7fe3b3fff640 1 --2- 172.21.15.42:0/218819350 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fe39c0b4c70 0x7fe39c0b7180 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fe3a80058c0 tx=0x7fe3a8015040 comp rx=0 tx=0).stop 2024-08-30T09:03:01.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.105+0000 7fe3b3fff640 1 -- 172.21.15.42:0/218819350 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe3ac09c3b0 msgr2=0x7fe3ac007df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:03:01.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.105+0000 7fe3b3fff640 1 --2- 172.21.15.42:0/218819350 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe3ac09c3b0 0x7fe3ac007df0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fe3b4000b60 tx=0x7fe3b407c020 comp rx=0 tx=0).stop 2024-08-30T09:03:01.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.106+0000 7fe3b3fff640 1 -- 172.21.15.42:0/218819350 shutdown_connections 2024-08-30T09:03:01.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.106+0000 7fe3b3fff640 1 --2- 172.21.15.42:0/218819350 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fe39c0b4c70 0x7fe39c0b7180 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:01.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.106+0000 7fe3b3fff640 1 --2- 172.21.15.42:0/218819350 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe3ac09c3b0 0x7fe3ac007df0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:01.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.106+0000 7fe3b3fff640 1 --2- 172.21.15.42:0/218819350 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe3ac09ba60 0x7fe3ac0078b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:01.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.106+0000 7fe3b3fff640 1 --2- 172.21.15.42:0/218819350 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe3ac096120 0x7fe3ac0092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:01.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.106+0000 7fe3b3fff640 1 -- 172.21.15.42:0/218819350 >> 172.21.15.42:0/218819350 conn(0x7fe3ac00ba80 msgr2=0x7fe3ac099ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:03:01.114 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.106+0000 7fe3b3fff640 1 -- 172.21.15.42:0/218819350 shutdown_connections 2024-08-30T09:03:01.114 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:01.106+0000 7fe3b3fff640 1 -- 172.21.15.42:0/218819350 wait complete. 2024-08-30T09:03:01.992 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-08-30T09:03:01.992 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph fs dump --format=json 2024-08-30T09:03:02.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:03:02 smithi042 ceph-mon[18652]: pgmap v441: 129 pgs: 129 active+clean; 1.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 4.8 MiB/s wr, 543 op/s 2024-08-30T09:03:02.435 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:03:02 smithi190 ceph-mon[22001]: pgmap v441: 129 pgs: 129 active+clean; 1.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 4.8 MiB/s wr, 543 op/s 2024-08-30T09:03:02.468 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T09:03:02.576 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:03:02 smithi167 ceph-mon[22063]: pgmap v441: 129 pgs: 129 active+clean; 1.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 4.8 MiB/s wr, 543 op/s 2024-08-30T09:03:03.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:03.126+0000 7ffab05c1640 1 -- 172.21.15.42:0/3583738107 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffaa8111ce0 msgr2=0x7ffaa810a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:03:03.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:03.126+0000 7ffab05c1640 1 --2- 172.21.15.42:0/3583738107 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffaa8111ce0 0x7ffaa810a160 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7ffa98009900 tx=0x7ffa9802a400 comp rx=0 tx=0).stop 2024-08-30T09:03:03.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:03.127+0000 7ffab05c1640 1 -- 172.21.15.42:0/3583738107 shutdown_connections 2024-08-30T09:03:03.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:03.127+0000 7ffab05c1640 1 --2- 172.21.15.42:0/3583738107 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffaa811c9d0 0x7ffaa811eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:03.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:03.127+0000 7ffab05c1640 1 --2- 172.21.15.42:0/3583738107 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffaa810a780 0x7ffaa810ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:03.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:03.127+0000 7ffab05c1640 1 --2- 172.21.15.42:0/3583738107 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffaa8111ce0 0x7ffaa810a160 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:03.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:03.127+0000 7ffab05c1640 1 -- 172.21.15.42:0/3583738107 >> 172.21.15.42:0/3583738107 conn(0x7ffaa8070c60 msgr2=0x7ffaa8071070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:03:03.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:03.128+0000 7ffab05c1640 1 -- 172.21.15.42:0/3583738107 shutdown_connections 2024-08-30T09:03:03.134 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:03.130+0000 7ffab05c1640 1 -- 172.21.15.42:0/3583738107 wait complete. 2024-08-30T09:03:03.134 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:03.132+0000 7ffab05c1640 1 Processor -- start 2024-08-30T09:03:03.135 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:03.132+0000 7ffab05c1640 1 -- start start 2024-08-30T09:03:03.135 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:03.132+0000 7ffab05c1640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffaa810a780 0x7ffaa81a9070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:03.135 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:03.132+0000 7ffab05c1640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffaa8111ce0 0x7ffaa81a9610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:03.135 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:03.132+0000 7ffab05c1640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffaa811c9d0 0x7ffaa81ae8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:03.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:03.132+0000 7ffaadb5c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffaa8111ce0 0x7ffaa81a9610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:03:03.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:03.133+0000 7ffaadb5c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffaa8111ce0 0x7ffaa81a9610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:49968/0 (socket says 172.21.15.42:49968) 2024-08-30T09:03:03.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:03.133+0000 7ffaadb5c640 1 -- 172.21.15.42:0/3727038394 learned_addr learned my addr 172.21.15.42:0/3727038394 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:03:03.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:03.133+0000 7ffab05c1640 1 -- 172.21.15.42:0/3727038394 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7ffaa8122640 con 0x7ffaa810a780 2024-08-30T09:03:03.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:03.133+0000 7ffab05c1640 1 -- 172.21.15.42:0/3727038394 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ffaa81224c0 con 0x7ffaa811c9d0 2024-08-30T09:03:03.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:03.133+0000 7ffab05c1640 1 -- 172.21.15.42:0/3727038394 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7ffaa81227c0 con 0x7ffaa8111ce0 2024-08-30T09:03:03.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:03.133+0000 7ffaaeb5e640 1 --2- 172.21.15.42:0/3727038394 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffaa811c9d0 0x7ffaa81ae8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:03:03.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:03.133+0000 7ffaaeb5e640 1 -- 172.21.15.42:0/3727038394 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffaa8111ce0 msgr2=0x7ffaa81a9610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:03:03.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:03.133+0000 7ffaaeb5e640 1 --2- 172.21.15.42:0/3727038394 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffaa8111ce0 0x7ffaa81a9610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:03.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:03.133+0000 7ffaaeb5e640 1 -- 172.21.15.42:0/3727038394 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffaa810a780 msgr2=0x7ffaa81a9070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:03:03.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:03.133+0000 7ffaaeb5e640 1 --2- 172.21.15.42:0/3727038394 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffaa810a780 0x7ffaa81a9070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:03.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:03.133+0000 7ffaaeb5e640 1 -- 172.21.15.42:0/3727038394 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffaa81aefe0 con 0x7ffaa811c9d0 2024-08-30T09:03:03.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:03.134+0000 7ffaaeb5e640 1 --2- 172.21.15.42:0/3727038394 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffaa811c9d0 0x7ffaa81ae8b0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7ffaa000e940 tx=0x7ffaa000ee00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:03:03.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:03.134+0000 7ffaadb5c640 1 --2- 172.21.15.42:0/3727038394 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffaa8111ce0 0x7ffaa81a9610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-30T09:03:03.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:03.134+0000 7ffa9f7fe640 1 -- 172.21.15.42:0/3727038394 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaa0004d20 con 0x7ffaa811c9d0 2024-08-30T09:03:03.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:03.134+0000 7ffa9f7fe640 1 -- 172.21.15.42:0/3727038394 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffaa0005020 con 0x7ffaa811c9d0 2024-08-30T09:03:03.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:03.135+0000 7ffa9f7fe640 1 -- 172.21.15.42:0/3727038394 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaa00116d0 con 0x7ffaa811c9d0 2024-08-30T09:03:03.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:03.135+0000 7ffab05c1640 1 -- 172.21.15.42:0/3727038394 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffaa81af2d0 con 0x7ffaa811c9d0 2024-08-30T09:03:03.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:03.135+0000 7ffab05c1640 1 -- 172.21.15.42:0/3727038394 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffaa81b5a30 con 0x7ffaa811c9d0 2024-08-30T09:03:03.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:03.137+0000 7ffab05c1640 1 -- 172.21.15.42:0/3727038394 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffaa810b380 con 0x7ffaa811c9d0 2024-08-30T09:03:03.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:03.140+0000 7ffa9f7fe640 1 -- 172.21.15.42:0/3727038394 <== mon.2 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7ffaa0005440 con 0x7ffaa811c9d0 2024-08-30T09:03:03.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:03.141+0000 7ffa9f7fe640 1 --2- 172.21.15.42:0/3727038394 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7ffa940b4c80 0x7ffa940b7190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:03.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:03.142+0000 7ffa9f7fe640 1 -- 172.21.15.42:0/3727038394 <== mon.2 v2:172.21.15.167:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7ffaa00e5d90 con 0x7ffaa811c9d0 2024-08-30T09:03:03.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:03.142+0000 7ffaae35d640 1 --2- 172.21.15.42:0/3727038394 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7ffa940b4c80 0x7ffa940b7190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:03:03.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:03.145+0000 7ffa9f7fe640 1 -- 172.21.15.42:0/3727038394 <== mon.2 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7ffaa001f020 con 0x7ffaa811c9d0 2024-08-30T09:03:03.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:03.145+0000 7ffaae35d640 1 --2- 172.21.15.42:0/3727038394 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7ffa940b4c80 0x7ffa940b7190 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7ffa98005f70 tx=0x7ffa98005d50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:03:03.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:03.497+0000 7ffab05c1640 1 -- 172.21.15.42:0/3727038394 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ffaa810ab90 con 0x7ffaa811c9d0 2024-08-30T09:03:03.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:03.498+0000 7ffa9f7fe640 1 -- 172.21.15.42:0/3727038394 <== mon.2 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11918 (secure 0 0 0) 0x7ffaa0016030 con 0x7ffaa811c9d0 2024-08-30T09:03:03.502 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-08-30T09:03:03.502 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":29,"btime":"2024-08-30T09:00:48:414225+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14703,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/3188400000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":3188400000},{"type":"v1","addr":"172.21.15.42:6835","nonce":3188400000}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14715,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/3356537674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":3356537674},{"type":"v1","addr":"172.21.15.42:6837","nonce":3356537674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14766,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6839/2107785513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6838","nonce":2107785513},{"type":"v1","addr":"172.21.15.42:6839","nonce":2107785513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14778,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6841/2298438674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6840","nonce":2298438674},{"type":"v1","addr":"172.21.15.42:6841","nonce":2298438674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24553,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/543555570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":543555570},{"type":"v1","addr":"172.21.15.190:6833","nonce":543555570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24554,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/3907818941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":3907818941},{"type":"v1","addr":"172.21.15.167:6833","nonce":3907818941}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-30T08:58:03.055084+0000","modified":"2024-08-30T09:00:47.637894+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24629,"mds_1":24610,"mds_2":24599},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24572":{"gid":24572,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6835/234596845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":234596845},{"type":"v1","addr":"172.21.15.167:6835","nonce":234596845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24592":{"gid":24592,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6835/247905709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":247905709},{"type":"v1","addr":"172.21.15.190:6835","nonce":247905709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24598":{"gid":24598,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6837/863256349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6836","nonce":863256349},{"type":"v1","addr":"172.21.15.190:6837","nonce":863256349}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24599":{"gid":24599,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.167:6837/3011837856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":3011837856},{"type":"v1","addr":"172.21.15.167:6837","nonce":3011837856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24610":{"gid":24610,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.190:6839/719981110","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6838","nonce":719981110},{"type":"v1","addr":"172.21.15.190:6839","nonce":719981110}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24629":{"gid":24629,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.167:6839/3604645207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":3604645207},{"type":"v1","addr":"172.21.15.167:6839","nonce":3604645207}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24629,"qdb_cluster":[24599,24610,24629]},"id":1}]} 2024-08-30T09:03:03.502 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 29 2024-08-30T09:03:03.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:03.503+0000 7ffab05c1640 1 -- 172.21.15.42:0/3727038394 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7ffa940b4c80 msgr2=0x7ffa940b7190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:03:03.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:03.503+0000 7ffab05c1640 1 --2- 172.21.15.42:0/3727038394 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7ffa940b4c80 0x7ffa940b7190 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7ffa98005f70 tx=0x7ffa98005d50 comp rx=0 tx=0).stop 2024-08-30T09:03:03.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:03.503+0000 7ffab05c1640 1 -- 172.21.15.42:0/3727038394 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffaa811c9d0 msgr2=0x7ffaa81ae8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:03:03.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:03.503+0000 7ffab05c1640 1 --2- 172.21.15.42:0/3727038394 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffaa811c9d0 0x7ffaa81ae8b0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7ffaa000e940 tx=0x7ffaa000ee00 comp rx=0 tx=0).stop 2024-08-30T09:03:03.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:03.503+0000 7ffab05c1640 1 -- 172.21.15.42:0/3727038394 shutdown_connections 2024-08-30T09:03:03.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:03.503+0000 7ffab05c1640 1 --2- 172.21.15.42:0/3727038394 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7ffa940b4c80 0x7ffa940b7190 secure :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7ffa98005f70 tx=0x7ffa98005d50 comp rx=0 tx=0).stop 2024-08-30T09:03:03.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:03.503+0000 7ffab05c1640 1 --2- 172.21.15.42:0/3727038394 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffaa8111ce0 0x7ffaa81a9610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:03.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:03.503+0000 7ffab05c1640 1 --2- 172.21.15.42:0/3727038394 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffaa811c9d0 0x7ffaa81ae8b0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:03.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:03.503+0000 7ffab05c1640 1 --2- 172.21.15.42:0/3727038394 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffaa810a780 0x7ffaa81a9070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:03.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:03.503+0000 7ffab05c1640 1 -- 172.21.15.42:0/3727038394 >> 172.21.15.42:0/3727038394 conn(0x7ffaa8070c60 msgr2=0x7ffaa8071bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:03:03.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:03.508+0000 7ffab05c1640 1 -- 172.21.15.42:0/3727038394 shutdown_connections 2024-08-30T09:03:03.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:03.509+0000 7ffab05c1640 1 -- 172.21.15.42:0/3727038394 wait complete. 2024-08-30T09:03:04.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:03:04 smithi042 ceph-mon[18652]: pgmap v442: 129 pgs: 129 active+clean; 1.1 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 7.3 MiB/s wr, 765 op/s 2024-08-30T09:03:04.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:03:04 smithi042 ceph-mon[18652]: from='client.? 172.21.15.42:0/3727038394' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-30T09:03:04.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:03:04 smithi190 ceph-mon[22001]: pgmap v442: 129 pgs: 129 active+clean; 1.1 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 7.3 MiB/s wr, 765 op/s 2024-08-30T09:03:04.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:03:04 smithi190 ceph-mon[22001]: from='client.? 172.21.15.42:0/3727038394' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-30T09:03:04.465 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-08-30T09:03:04.576 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:03:04 smithi167 ceph-mon[22063]: pgmap v442: 129 pgs: 129 active+clean; 1.1 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 7.3 MiB/s wr, 765 op/s 2024-08-30T09:03:04.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:03:04 smithi167 ceph-mon[22063]: from='client.? 172.21.15.42:0/3727038394' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-30T09:03:04.884 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T09:03:05.953 INFO:teuthology.orchestra.run.smithi042.stderr:progress: 140.0 s, 2214.8 tps, lat 45.071 ms stddev 64.620 2024-08-30T09:03:06.378 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:03:06 smithi167 ceph-mon[22063]: pgmap v443: 129 pgs: 129 active+clean; 1.1 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 7.4 MiB/s wr, 681 op/s 2024-08-30T09:03:06.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:03:06 smithi042 ceph-mon[18652]: pgmap v443: 129 pgs: 129 active+clean; 1.1 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 7.4 MiB/s wr, 681 op/s 2024-08-30T09:03:06.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:03:06 smithi190 ceph-mon[22001]: pgmap v443: 129 pgs: 129 active+clean; 1.1 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 7.4 MiB/s wr, 681 op/s 2024-08-30T09:03:07.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.752+0000 7f6f6142f640 1 -- 172.21.15.42:0/3300663786 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6f540a29b0 msgr2=0x7f6f540a91e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:03:07.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.752+0000 7f6f6142f640 1 --2- 172.21.15.42:0/3300663786 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6f540a29b0 0x7f6f540a91e0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f6f48009900 tx=0x7f6f4802a440 comp rx=0 tx=0).stop 2024-08-30T09:03:07.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.755+0000 7f6f6142f640 1 -- 172.21.15.42:0/3300663786 shutdown_connections 2024-08-30T09:03:07.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.755+0000 7f6f6142f640 1 --2- 172.21.15.42:0/3300663786 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6f540a29b0 0x7f6f540a91e0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:07.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.755+0000 7f6f6142f640 1 --2- 172.21.15.42:0/3300663786 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6f540a2060 0x7f6f540a2470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:07.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.755+0000 7f6f6142f640 1 --2- 172.21.15.42:0/3300663786 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6f5409c720 0x7f6f5409cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:07.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.755+0000 7f6f6142f640 1 -- 172.21.15.42:0/3300663786 >> 172.21.15.42:0/3300663786 conn(0x7f6f54008ce0 msgr2=0x7f6f540090f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:03:07.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.757+0000 7f6f6142f640 1 -- 172.21.15.42:0/3300663786 shutdown_connections 2024-08-30T09:03:07.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.757+0000 7f6f6142f640 1 -- 172.21.15.42:0/3300663786 wait complete. 2024-08-30T09:03:07.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.758+0000 7f6f6142f640 1 Processor -- start 2024-08-30T09:03:07.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.758+0000 7f6f6142f640 1 -- start start 2024-08-30T09:03:07.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.758+0000 7f6f6142f640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6f5409c720 0x7f6f540158f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:07.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.758+0000 7f6f6142f640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6f540a2060 0x7f6f5400ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:07.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.758+0000 7f6f6142f640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6f540a29b0 0x7f6f5400f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:07.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.758+0000 7f6f6142f640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f6f540bd660 con 0x7f6f5409c720 2024-08-30T09:03:07.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.758+0000 7f6f6142f640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f6f540bd4e0 con 0x7f6f540a29b0 2024-08-30T09:03:07.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.758+0000 7f6f6142f640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f6f540bd7e0 con 0x7f6f540a2060 2024-08-30T09:03:07.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.761+0000 7f6f5b7fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6f540a2060 0x7f6f5400ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:03:07.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.761+0000 7f6f60c2e640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6f540a29b0 0x7f6f5400f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:03:07.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.761+0000 7f6f5b7fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6f540a2060 0x7f6f5400ebc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:49984/0 (socket says 172.21.15.42:49984) 2024-08-30T09:03:07.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.761+0000 7f6f5b7fe640 1 -- 172.21.15.42:0/309083820 learned_addr learned my addr 172.21.15.42:0/309083820 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:03:07.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.761+0000 7f6f5b7fe640 1 -- 172.21.15.42:0/309083820 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6f540a29b0 msgr2=0x7f6f5400f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:03:07.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.761+0000 7f6f5b7fe640 1 --2- 172.21.15.42:0/309083820 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6f540a29b0 0x7f6f5400f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:07.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.761+0000 7f6f5b7fe640 1 -- 172.21.15.42:0/309083820 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6f5409c720 msgr2=0x7f6f540158f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:03:07.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.761+0000 7f6f5b7fe640 1 --2- 172.21.15.42:0/309083820 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6f5409c720 0x7f6f540158f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:07.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.761+0000 7f6f5b7fe640 1 -- 172.21.15.42:0/309083820 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f5400f880 con 0x7f6f540a2060 2024-08-30T09:03:07.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.762+0000 7f6f5b7fe640 1 --2- 172.21.15.42:0/309083820 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6f540a2060 0x7f6f5400ebc0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f6f5c068b50 tx=0x7f6f5c06e7e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:03:07.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.762+0000 7f6f597fa640 1 -- 172.21.15.42:0/309083820 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f5c06b820 con 0x7f6f540a2060 2024-08-30T09:03:07.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.762+0000 7f6f597fa640 1 -- 172.21.15.42:0/309083820 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6f5c06a370 con 0x7f6f540a2060 2024-08-30T09:03:07.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.762+0000 7f6f597fa640 1 -- 172.21.15.42:0/309083820 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f5c075340 con 0x7f6f540a2060 2024-08-30T09:03:07.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.763+0000 7f6f6142f640 1 -- 172.21.15.42:0/309083820 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f54014560 con 0x7f6f540a2060 2024-08-30T09:03:07.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.763+0000 7f6f6142f640 1 -- 172.21.15.42:0/309083820 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f54014960 con 0x7f6f540a2060 2024-08-30T09:03:07.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.767+0000 7f6f68c06740 1 Processor -- start 2024-08-30T09:03:07.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.770+0000 7f6f597fa640 1 -- 172.21.15.42:0/309083820 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7f6f5c06b9c0 con 0x7f6f540a2060 2024-08-30T09:03:07.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.772+0000 7f6f597fa640 1 --2- 172.21.15.42:0/309083820 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f6f4c0b4d30 0x7f6f4c0b7240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:07.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.772+0000 7f6f68c06740 1 -- start start 2024-08-30T09:03:07.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.772+0000 7f6f597fa640 1 -- 172.21.15.42:0/309083820 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f6f5c14a090 con 0x7f6f540a2060 2024-08-30T09:03:07.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.772+0000 7f6f68c06740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a537a2f070 0x55a537dd44b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:07.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.772+0000 7f6f68c06740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a537a061b0 0x55a537dd49f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:07.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.772+0000 7f6f68c06740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a537a0e390 0x55a537dd4f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:07.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.772+0000 7f6f68c06740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55a537d4b680 con 0x55a537a061b0 2024-08-30T09:03:07.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.772+0000 7f6f68c06740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55a537d3ac10 con 0x55a537a2f070 2024-08-30T09:03:07.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.772+0000 7f6f68c06740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55a537d3ebc0 con 0x55a537a0e390 2024-08-30T09:03:07.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.772+0000 7f6f60c2e640 1 --2- 172.21.15.42:0/309083820 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f6f4c0b4d30 0x7f6f4c0b7240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:03:07.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.773+0000 7f6f60c2e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a537a0e390 0x55a537dd4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:03:07.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.773+0000 7f6f60c2e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a537a0e390 0x55a537dd4f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:49992/0 (socket says 172.21.15.42:49992) 2024-08-30T09:03:07.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.773+0000 7f6f60c2e640 1 -- 172.21.15.42:0/1473322648 learned_addr learned my addr 172.21.15.42:0/1473322648 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:03:07.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.773+0000 7f6f5b7fe640 1 --2- 172.21.15.42:0/1473322648 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a537a061b0 0x55a537dd49f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:03:07.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.773+0000 7f6f60c2e640 1 -- 172.21.15.42:0/1473322648 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a537a2f070 msgr2=0x55a537dd44b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-30T09:03:07.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.773+0000 7f6f60c2e640 1 --2- 172.21.15.42:0/1473322648 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a537a2f070 0x55a537dd44b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:07.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.773+0000 7f6f60c2e640 1 -- 172.21.15.42:0/1473322648 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a537a061b0 msgr2=0x55a537dd49f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:03:07.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.773+0000 7f6f60c2e640 1 --2- 172.21.15.42:0/1473322648 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a537a061b0 0x55a537dd49f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:07.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.773+0000 7f6f60c2e640 1 -- 172.21.15.42:0/1473322648 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a537a9a650 con 0x55a537a0e390 2024-08-30T09:03:07.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.773+0000 7f6f60c2e640 1 --2- 172.21.15.42:0/309083820 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f6f4c0b4d30 0x7f6f4c0b7240 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f6f48009230 tx=0x7f6f48031bb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:03:07.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.773+0000 7f6f60c2e640 1 --2- 172.21.15.42:0/1473322648 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a537a0e390 0x55a537dd4f30 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f6f48009850 tx=0x7f6f48046000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=96ff7bf7624be279 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:03:07.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.774+0000 7f6f467fc640 1 -- 172.21.15.42:0/1473322648 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f48035050 con 0x55a537a0e390 2024-08-30T09:03:07.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.774+0000 7f6f467fc640 1 -- 172.21.15.42:0/1473322648 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6f4803f270 con 0x55a537a0e390 2024-08-30T09:03:07.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.774+0000 7f6f467fc640 1 -- 172.21.15.42:0/1473322648 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f48003850 con 0x55a537a0e390 2024-08-30T09:03:07.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.776+0000 7f6f68c06740 1 -- 172.21.15.42:0/1473322648 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a537a0e390 msgr2=0x55a537dd4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:03:07.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.776+0000 7f6f68c06740 1 --2- 172.21.15.42:0/1473322648 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a537a0e390 0x55a537dd4f30 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f6f48009850 tx=0x7f6f48046000 comp rx=0 tx=0).stop 2024-08-30T09:03:07.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.776+0000 7f6f68c06740 1 -- 172.21.15.42:0/1473322648 shutdown_connections 2024-08-30T09:03:07.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.776+0000 7f6f68c06740 1 --2- 172.21.15.42:0/1473322648 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a537a0e390 0x55a537dd4f30 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:07.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.776+0000 7f6f68c06740 1 --2- 172.21.15.42:0/1473322648 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a537a061b0 0x55a537dd49f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:07.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.776+0000 7f6f68c06740 1 --2- 172.21.15.42:0/1473322648 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a537a2f070 0x55a537dd44b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:07.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.776+0000 7f6f68c06740 1 -- 172.21.15.42:0/1473322648 >> 172.21.15.42:0/1473322648 conn(0x55a537a23460 msgr2=0x55a537dd23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:03:07.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.778+0000 7f6f68c06740 1 -- 172.21.15.42:0/1473322648 shutdown_connections 2024-08-30T09:03:07.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.778+0000 7f6f68c06740 1 -- 172.21.15.42:0/1473322648 wait complete. 2024-08-30T09:03:07.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.779+0000 7f6f68c06740 1 -- start start 2024-08-30T09:03:07.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.779+0000 7f6f68c06740 1 Processor -- start 2024-08-30T09:03:07.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.779+0000 7f6f68c06740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a537a061b0 0x55a537ddfce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:07.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.779+0000 7f6f68c06740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a537a0e390 0x55a537de0220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:07.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.779+0000 7f6f68c06740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a537a2f070 0x55a537debf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:07.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.779+0000 7f6f68c06740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55a537b9d700 con 0x55a537a061b0 2024-08-30T09:03:07.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.779+0000 7f6f68c06740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55a537cc1750 con 0x55a537a2f070 2024-08-30T09:03:07.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.779+0000 7f6f68c06740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55a537de2c30 con 0x55a537a0e390 2024-08-30T09:03:07.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.780+0000 7f6f68c06740 10 client.0 _ll_register_callbacks cb 0x55a537cf0e10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:03:07.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.780+0000 7f6f60c2e640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a537a2f070 0x55a537debf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:03:07.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.780+0000 7f6f5b7fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a537a0e390 0x55a537de0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:03:07.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.780+0000 7f6f60c2e640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a537a2f070 0x55a537debf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:52198/0 (socket says 172.21.15.42:52198) 2024-08-30T09:03:07.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.780+0000 7f6f60c2e640 1 -- 172.21.15.42:0/727635387 learned_addr learned my addr 172.21.15.42:0/727635387 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:03:07.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.781+0000 7f6f60c2e640 1 -- 172.21.15.42:0/727635387 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a537a0e390 msgr2=0x55a537de0220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:03:07.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.781+0000 7f6f60c2e640 1 --2- 172.21.15.42:0/727635387 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a537a0e390 0x55a537de0220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:07.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.781+0000 7f6f60c2e640 1 -- 172.21.15.42:0/727635387 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a537a061b0 msgr2=0x55a537ddfce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:03:07.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.781+0000 7f6f60c2e640 1 --2- 172.21.15.42:0/727635387 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a537a061b0 0x55a537ddfce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:07.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.781+0000 7f6f60c2e640 1 -- 172.21.15.42:0/727635387 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a537cfa900 con 0x55a537a2f070 2024-08-30T09:03:07.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.781+0000 7f6f60c2e640 1 --2- 172.21.15.42:0/727635387 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a537a2f070 0x55a537debf50 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f6f48004400 tx=0x7f6f480042a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=41e9347d6346f0b server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:03:07.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.781+0000 7f6f3b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-30T09:03:07.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.783+0000 7f6f3b7fe640 1 -- 172.21.15.42:0/727635387 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f480044e0 con 0x55a537a2f070 2024-08-30T09:03:07.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.783+0000 7f6f68c06740 1 -- 172.21.15.42:0/727635387 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6f4803f270 con 0x55a537a2f070 2024-08-30T09:03:07.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.783+0000 7f6f3b7fe640 1 -- 172.21.15.42:0/727635387 <== mon.2 v2:172.21.15.167:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f6f480035f0 con 0x55a537a2f070 2024-08-30T09:03:07.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.784+0000 7f6f3b7fe640 1 -- 172.21.15.42:0/727635387 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f6f4803fea0 con 0x55a537a2f070 2024-08-30T09:03:07.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.785+0000 7f6f68c06740 10 client.25544 fetch_fsmap learned FSMap version 29 2024-08-30T09:03:07.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.785+0000 7f6f68c06740 1 -- 172.21.15.42:0/727635387 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a537e10a20 con 0x55a537a2f070 2024-08-30T09:03:07.794 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.789+0000 7f6f3b7fe640 1 -- 172.21.15.42:0/727635387 <== mon.2 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6f4803f270 con 0x55a537a2f070 2024-08-30T09:03:07.794 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.789+0000 7f6f3b7fe640 1 -- 172.21.15.42:0/727635387 <== mon.2 v2:172.21.15.167:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f6f4803a040 con 0x55a537a2f070 2024-08-30T09:03:07.794 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.791+0000 7f6f68c06740 10 client.25544 fetch_fsmap finished waiting for FSMap version 29 2024-08-30T09:03:07.794 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.791+0000 7f6f68c06740 10 client.25544 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-08-30T09:03:07.794 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.791+0000 7f6f68c06740 20 client.25544 populate_metadata read hostname 'smithi042' 2024-08-30T09:03:07.794 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.791+0000 7f6f68c06740 1 --2- 172.21.15.42:0/727635387 >> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] conn(0x55a5379ed9f0 0x55a537e13ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:07.794 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.791+0000 7f6f68c06740 4 client.25544 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-30T09:03:07.794 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.791+0000 7f6f68c06740 1 -- 172.21.15.42:0/727635387 --> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a537d4d5d0 con 0x55a5379ed9f0 2024-08-30T09:03:07.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.792+0000 7f6f5bfff640 1 --2- 172.21.15.42:0/727635387 >> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] conn(0x55a5379ed9f0 0x55a537e13ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:03:07.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.797+0000 7f6f5bfff640 1 --2- 172.21.15.42:0/727635387 >> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] conn(0x55a5379ed9f0 0x55a537e13ff0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=70e13385de7832c4 server_cookie=34f830d350bb1145 in_seq=0 out_seq=0 2024-08-30T09:03:07.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.798+0000 7f6f3b7fe640 10 client.25544 ms_handle_connect on v2:172.21.15.167:6836/3011837856 2024-08-30T09:03:07.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.798+0000 7f6f3b7fe640 1 -- 172.21.15.42:0/727635387 <== mds.2 v2:172.21.15.167:6836/3011837856 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a537d4d5d0 con 0x55a5379ed9f0 2024-08-30T09:03:07.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.798+0000 7f6f3b7fe640 10 client.25544 handle_command_reply: tid=0 2024-08-30T09:03:07.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.798+0000 7f6f68c06740 1 client.25544 shutdown 2024-08-30T09:03:07.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.799+0000 7f6f68c06740 1 -- 172.21.15.42:0/727635387 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a537a2f070 msgr2=0x55a537debf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:03:07.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.799+0000 7f6f68c06740 1 --2- 172.21.15.42:0/727635387 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a537a2f070 0x55a537debf50 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f6f48004400 tx=0x7f6f480042a0 comp rx=0 tx=0).stop 2024-08-30T09:03:07.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.799+0000 7f6f68c06740 1 -- 172.21.15.42:0/727635387 shutdown_connections 2024-08-30T09:03:07.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.799+0000 7f6f68c06740 1 --2- 172.21.15.42:0/727635387 >> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] conn(0x55a5379ed9f0 0x55a537e13ff0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:07.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.799+0000 7f6f68c06740 1 --2- 172.21.15.42:0/727635387 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a537a0e390 0x55a537de0220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:07.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.799+0000 7f6f68c06740 1 --2- 172.21.15.42:0/727635387 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a537a2f070 0x55a537debf50 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:07.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.799+0000 7f6f68c06740 1 --2- 172.21.15.42:0/727635387 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a537a061b0 0x55a537ddfce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:07.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.799+0000 7f6f68c06740 1 -- 172.21.15.42:0/727635387 >> 172.21.15.42:0/727635387 conn(0x55a537a23460 msgr2=0x55a537dd7640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:03:07.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.799+0000 7f6f3b7fe640 0 client.25544 ms_handle_reset on v2:172.21.15.167:6836/3011837856 2024-08-30T09:03:07.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.799+0000 7f6f68c06740 1 -- 172.21.15.42:0/727635387 shutdown_connections 2024-08-30T09:03:07.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.799+0000 7f6f68c06740 1 -- 172.21.15.42:0/727635387 wait complete. 2024-08-30T09:03:07.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.800+0000 7f6f68c06740 20 client.25544 trim_cache size 0 max 16384 2024-08-30T09:03:07.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.821+0000 7f6f68c06740 1 Processor -- start 2024-08-30T09:03:07.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.821+0000 7f6f68c06740 1 -- start start 2024-08-30T09:03:07.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.821+0000 7f6f68c06740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a537a061b0 0x55a537dda5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:07.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.821+0000 7f6f68c06740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a537a0e390 0x55a537de01a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:07.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.821+0000 7f6f68c06740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a537a2f070 0x55a537debe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:07.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.821+0000 7f6f68c06740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f6f540b7db0 con 0x55a537a061b0 2024-08-30T09:03:07.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.821+0000 7f6f68c06740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f6f540b56c0 con 0x55a537a0e390 2024-08-30T09:03:07.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.821+0000 7f6f68c06740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f6f5400fec0 con 0x55a537a2f070 2024-08-30T09:03:07.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.822+0000 7f6f5bfff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a537a061b0 0x55a537dda5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:03:07.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.822+0000 7f6f5bfff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a537a061b0 0x55a537dda5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:54866/0 (socket says 172.21.15.42:54866) 2024-08-30T09:03:07.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.822+0000 7f6f5bfff640 1 -- 172.21.15.42:0/2650413148 learned_addr learned my addr 172.21.15.42:0/2650413148 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:03:07.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.822+0000 7f6f5b7fe640 1 --2- 172.21.15.42:0/2650413148 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a537a0e390 0x55a537de01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:03:07.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.822+0000 7f6f60c2e640 1 --2- 172.21.15.42:0/2650413148 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a537a2f070 0x55a537debe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:03:07.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.822+0000 7f6f5b7fe640 1 -- 172.21.15.42:0/2650413148 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a537a2f070 msgr2=0x55a537debe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:03:07.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.822+0000 7f6f5b7fe640 1 --2- 172.21.15.42:0/2650413148 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a537a2f070 0x55a537debe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:07.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.822+0000 7f6f5b7fe640 1 -- 172.21.15.42:0/2650413148 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a537a061b0 msgr2=0x55a537dda5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:03:07.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.822+0000 7f6f5b7fe640 1 --2- 172.21.15.42:0/2650413148 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a537a061b0 0x55a537dda5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:07.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.822+0000 7f6f5b7fe640 1 -- 172.21.15.42:0/2650413148 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a537dd67a0 con 0x55a537a0e390 2024-08-30T09:03:07.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.822+0000 7f6f5bfff640 1 --2- 172.21.15.42:0/2650413148 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a537a061b0 0x55a537dda5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-30T09:03:07.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.822+0000 7f6f5b7fe640 1 --2- 172.21.15.42:0/2650413148 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a537a0e390 0x55a537de01a0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f6f5c1477a0 tx=0x7f6f5c07b040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=766ebf9b6811b72d server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:03:07.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.822+0000 7f6f45ffb640 1 -- 172.21.15.42:0/2650413148 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f5c06bcb0 con 0x55a537a0e390 2024-08-30T09:03:07.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.823+0000 7f6f45ffb640 1 -- 172.21.15.42:0/2650413148 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6f5c119330 con 0x55a537a0e390 2024-08-30T09:03:07.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.823+0000 7f6f45ffb640 1 -- 172.21.15.42:0/2650413148 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f5c10c960 con 0x55a537a0e390 2024-08-30T09:03:07.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.824+0000 7f6f68c06740 1 -- 172.21.15.42:0/2650413148 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a537a0e390 msgr2=0x55a537de01a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:03:07.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.824+0000 7f6f68c06740 1 --2- 172.21.15.42:0/2650413148 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a537a0e390 0x55a537de01a0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f6f5c1477a0 tx=0x7f6f5c07b040 comp rx=0 tx=0).stop 2024-08-30T09:03:07.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.824+0000 7f6f68c06740 1 -- 172.21.15.42:0/2650413148 shutdown_connections 2024-08-30T09:03:07.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.824+0000 7f6f68c06740 1 --2- 172.21.15.42:0/2650413148 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a537a2f070 0x55a537debe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:07.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.824+0000 7f6f68c06740 1 --2- 172.21.15.42:0/2650413148 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a537a0e390 0x55a537de01a0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:07.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.824+0000 7f6f68c06740 1 --2- 172.21.15.42:0/2650413148 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a537a061b0 0x55a537dda5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:07.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.824+0000 7f6f68c06740 1 -- 172.21.15.42:0/2650413148 >> 172.21.15.42:0/2650413148 conn(0x55a537a23460 msgr2=0x55a537ddfa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:03:07.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.824+0000 7f6f68c06740 1 -- 172.21.15.42:0/2650413148 shutdown_connections 2024-08-30T09:03:07.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.824+0000 7f6f68c06740 1 -- 172.21.15.42:0/2650413148 wait complete. 2024-08-30T09:03:07.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.824+0000 7f6f68c06740 1 -- start start 2024-08-30T09:03:07.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.824+0000 7f6f68c06740 1 Processor -- start 2024-08-30T09:03:07.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.825+0000 7f6f68c06740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a537a061b0 0x55a537de3700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:07.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.825+0000 7f6f68c06740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a537a0e390 0x55a537de3ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:07.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.825+0000 7f6f68c06740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a537a2f070 0x55a537e7c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:07.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.825+0000 7f6f68c06740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f6f540b3b80 con 0x55a537a2f070 2024-08-30T09:03:07.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.825+0000 7f6f68c06740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f6f540b4ff0 con 0x55a537a0e390 2024-08-30T09:03:07.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.825+0000 7f6f68c06740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f6f540b4250 con 0x55a537a061b0 2024-08-30T09:03:07.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.825+0000 7f6f68c06740 10 client.0 _ll_register_callbacks cb 0x7f6f48009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:03:07.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.825+0000 7f6f60c2e640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a537a2f070 0x55a537e7c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:03:07.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.825+0000 7f6f5b7fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a537a0e390 0x55a537de3ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:03:07.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.825+0000 7f6f5b7fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a537a0e390 0x55a537de3ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:52218/0 (socket says 172.21.15.42:52218) 2024-08-30T09:03:07.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.825+0000 7f6f5b7fe640 1 -- 172.21.15.42:0/1866898243 learned_addr learned my addr 172.21.15.42:0/1866898243 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:03:07.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.826+0000 7f6f5b7fe640 1 -- 172.21.15.42:0/1866898243 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a537a061b0 msgr2=0x55a537de3700 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-30T09:03:07.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.826+0000 7f6f5b7fe640 1 --2- 172.21.15.42:0/1866898243 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a537a061b0 0x55a537de3700 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:07.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.826+0000 7f6f5b7fe640 1 -- 172.21.15.42:0/1866898243 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a537a2f070 msgr2=0x55a537e7c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:03:07.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.826+0000 7f6f5b7fe640 1 --2- 172.21.15.42:0/1866898243 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a537a2f070 0x55a537e7c9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:07.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.826+0000 7f6f5b7fe640 1 -- 172.21.15.42:0/1866898243 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a537e7d050 con 0x55a537a0e390 2024-08-30T09:03:07.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.826+0000 7f6f5b7fe640 1 --2- 172.21.15.42:0/1866898243 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a537a0e390 0x55a537de3ce0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f6f5c110660 tx=0x7f6f5c06ede0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=735b00e0e9c3229c server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:03:07.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.826+0000 7f6f3b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-30T09:03:07.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.826+0000 7f6f3b7fe640 1 -- 172.21.15.42:0/1866898243 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f5c1106d0 con 0x55a537a0e390 2024-08-30T09:03:07.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.826+0000 7f6f3b7fe640 1 -- 172.21.15.42:0/1866898243 <== mon.2 v2:172.21.15.167:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f6f5c10ccc0 con 0x55a537a0e390 2024-08-30T09:03:07.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.826+0000 7f6f68c06740 1 -- 172.21.15.42:0/1866898243 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55a537d49070 con 0x55a537a0e390 2024-08-30T09:03:07.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.827+0000 7f6f3b7fe640 1 -- 172.21.15.42:0/1866898243 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f6f5c114f40 con 0x55a537a0e390 2024-08-30T09:03:07.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.827+0000 7f6f68c06740 10 client.25550 fetch_fsmap learned FSMap version 29 2024-08-30T09:03:07.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.827+0000 7f6f68c06740 1 -- 172.21.15.42:0/1866898243 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a537e7e2d0 con 0x55a537a0e390 2024-08-30T09:03:07.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.828+0000 7f6f3b7fe640 1 -- 172.21.15.42:0/1866898243 <== mon.2 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55a537d49070 con 0x55a537a0e390 2024-08-30T09:03:07.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.828+0000 7f6f3b7fe640 1 -- 172.21.15.42:0/1866898243 <== mon.2 v2:172.21.15.167:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f6f5c075020 con 0x55a537a0e390 2024-08-30T09:03:07.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.828+0000 7f6f68c06740 10 client.25550 fetch_fsmap finished waiting for FSMap version 29 2024-08-30T09:03:07.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.828+0000 7f6f68c06740 10 client.25550 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-08-30T09:03:07.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.828+0000 7f6f68c06740 20 client.25550 populate_metadata read hostname 'smithi042' 2024-08-30T09:03:07.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.828+0000 7f6f68c06740 1 --2- 172.21.15.42:0/1866898243 >> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] conn(0x55a5379c2da0 0x55a537e80980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:07.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.828+0000 7f6f68c06740 4 client.25550 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-30T09:03:07.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.828+0000 7f6f68c06740 1 -- 172.21.15.42:0/1866898243 --> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a537d467b0 con 0x55a5379c2da0 2024-08-30T09:03:07.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.828+0000 7f6f5bfff640 1 --2- 172.21.15.42:0/1866898243 >> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] conn(0x55a5379c2da0 0x55a537e80980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:03:07.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.829+0000 7f6f3b7fe640 10 client.25550 ms_handle_connect on v2:172.21.15.167:6836/3011837856 2024-08-30T09:03:07.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.829+0000 7f6f5bfff640 1 --2- 172.21.15.42:0/1866898243 >> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] conn(0x55a5379c2da0 0x55a537e80980 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=aafcb8403f58bf14 server_cookie=77d5a049f4c7f68e in_seq=0 out_seq=0 2024-08-30T09:03:07.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.829+0000 7f6f3b7fe640 1 -- 172.21.15.42:0/1866898243 <== mds.2 v2:172.21.15.167:6836/3011837856 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a537d467b0 con 0x55a5379c2da0 2024-08-30T09:03:07.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.829+0000 7f6f3b7fe640 10 client.25550 handle_command_reply: tid=0 2024-08-30T09:03:07.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.829+0000 7f6f68c06740 1 client.25550 shutdown 2024-08-30T09:03:07.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.830+0000 7f6f68c06740 1 -- 172.21.15.42:0/1866898243 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a537a0e390 msgr2=0x55a537de3ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:03:07.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.830+0000 7f6f68c06740 1 --2- 172.21.15.42:0/1866898243 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a537a0e390 0x55a537de3ce0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f6f5c110660 tx=0x7f6f5c06ede0 comp rx=0 tx=0).stop 2024-08-30T09:03:07.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.830+0000 7f6f68c06740 1 -- 172.21.15.42:0/1866898243 shutdown_connections 2024-08-30T09:03:07.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.830+0000 7f6f68c06740 1 --2- 172.21.15.42:0/1866898243 >> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] conn(0x55a5379c2da0 0x55a537e80980 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:07.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.830+0000 7f6f68c06740 1 --2- 172.21.15.42:0/1866898243 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a537a2f070 0x55a537e7c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:07.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.830+0000 7f6f68c06740 1 --2- 172.21.15.42:0/1866898243 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a537a0e390 0x55a537de3ce0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:07.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.830+0000 7f6f68c06740 1 --2- 172.21.15.42:0/1866898243 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a537a061b0 0x55a537de3700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:07.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.830+0000 7f6f68c06740 1 -- 172.21.15.42:0/1866898243 >> 172.21.15.42:0/1866898243 conn(0x55a537a23460 msgr2=0x55a537dd1a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:03:07.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.830+0000 7f6f3b7fe640 0 client.25550 ms_handle_reset on v2:172.21.15.167:6836/3011837856 2024-08-30T09:03:07.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.830+0000 7f6f68c06740 1 -- 172.21.15.42:0/1866898243 shutdown_connections 2024-08-30T09:03:07.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.830+0000 7f6f68c06740 1 -- 172.21.15.42:0/1866898243 wait complete. 2024-08-30T09:03:07.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.830+0000 7f6f68c06740 20 client.25550 trim_cache size 0 max 16384 2024-08-30T09:03:07.837 INFO:teuthology.orchestra.run.smithi042.stdout:[] 2024-08-30T09:03:07.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.831+0000 7f6f6142f640 1 -- 172.21.15.42:0/309083820 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f6f4c0b4d30 msgr2=0x7f6f4c0b7240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:03:07.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.831+0000 7f6f6142f640 1 --2- 172.21.15.42:0/309083820 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f6f4c0b4d30 0x7f6f4c0b7240 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f6f48009230 tx=0x7f6f48031bb0 comp rx=0 tx=0).stop 2024-08-30T09:03:07.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.831+0000 7f6f6142f640 1 -- 172.21.15.42:0/309083820 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6f540a2060 msgr2=0x7f6f5400ebc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:03:07.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.831+0000 7f6f6142f640 1 --2- 172.21.15.42:0/309083820 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6f540a2060 0x7f6f5400ebc0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f6f5c068b50 tx=0x7f6f5c06e7e0 comp rx=0 tx=0).stop 2024-08-30T09:03:07.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.831+0000 7f6f6142f640 1 -- 172.21.15.42:0/309083820 shutdown_connections 2024-08-30T09:03:07.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.831+0000 7f6f6142f640 1 --2- 172.21.15.42:0/309083820 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f6f4c0b4d30 0x7f6f4c0b7240 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:07.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.831+0000 7f6f6142f640 1 --2- 172.21.15.42:0/309083820 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6f540a2060 0x7f6f5400ebc0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:07.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.831+0000 7f6f6142f640 1 --2- 172.21.15.42:0/309083820 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6f540a29b0 0x7f6f5400f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:07.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.831+0000 7f6f6142f640 1 --2- 172.21.15.42:0/309083820 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6f5409c720 0x7f6f540158f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:07.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.831+0000 7f6f6142f640 1 -- 172.21.15.42:0/309083820 >> 172.21.15.42:0/309083820 conn(0x7f6f54008ce0 msgr2=0x7f6f5409a0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:03:07.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.831+0000 7f6f6142f640 1 -- 172.21.15.42:0/309083820 shutdown_connections 2024-08-30T09:03:07.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:07.831+0000 7f6f6142f640 1 -- 172.21.15.42:0/309083820 wait complete. 2024-08-30T09:03:08.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:03:08 smithi042 ceph-mon[18652]: pgmap v444: 129 pgs: 129 active+clean; 1.1 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 11 MiB/s wr, 839 op/s 2024-08-30T09:03:08.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:03:08 smithi190 ceph-mon[22001]: pgmap v444: 129 pgs: 129 active+clean; 1.1 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 11 MiB/s wr, 839 op/s 2024-08-30T09:03:08.576 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:03:08 smithi167 ceph-mon[22063]: pgmap v444: 129 pgs: 129 active+clean; 1.1 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 11 MiB/s wr, 839 op/s 2024-08-30T09:03:08.699 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-08-30T09:03:09.118 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T09:03:09.404 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:03:09 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-30T09:03:09.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:03:09 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-30T09:03:09.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:03:09 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-30T09:03:09.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.707+0000 7f0f16e28640 1 -- 172.21.15.42:0/1487519362 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0f10102540 msgr2=0x7f0f10102950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:03:09.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.708+0000 7f0f16e28640 1 --2- 172.21.15.42:0/1487519362 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0f10102540 0x7f0f10102950 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f0f04009860 tx=0x7f0f0402a380 comp rx=0 tx=0).stop 2024-08-30T09:03:09.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.708+0000 7f0f16e28640 1 -- 172.21.15.42:0/1487519362 shutdown_connections 2024-08-30T09:03:09.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.708+0000 7f0f16e28640 1 --2- 172.21.15.42:0/1487519362 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0f10107980 0x7f0f10107e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:09.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.708+0000 7f0f16e28640 1 --2- 172.21.15.42:0/1487519362 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0f10102e90 0x7f0f101073c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:09.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.708+0000 7f0f16e28640 1 --2- 172.21.15.42:0/1487519362 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0f10102540 0x7f0f10102950 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:09.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.708+0000 7f0f16e28640 1 -- 172.21.15.42:0/1487519362 >> 172.21.15.42:0/1487519362 conn(0x7f0f100794d0 msgr2=0x7f0f100798e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:03:09.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.708+0000 7f0f16e28640 1 -- 172.21.15.42:0/1487519362 shutdown_connections 2024-08-30T09:03:09.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.708+0000 7f0f16e28640 1 -- 172.21.15.42:0/1487519362 wait complete. 2024-08-30T09:03:09.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.709+0000 7f0f16e28640 1 Processor -- start 2024-08-30T09:03:09.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.709+0000 7f0f16e28640 1 -- start start 2024-08-30T09:03:09.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.710+0000 7f0f16e28640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0f10102540 0x7f0f101a0850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:09.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.710+0000 7f0f16e28640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0f10102e90 0x7f0f101a0df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:09.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.710+0000 7f0f16e28640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0f10107980 0x7f0f101a6090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:09.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.710+0000 7f0f14bc4640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0f10102540 0x7f0f101a0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:03:09.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.710+0000 7f0f14bc4640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0f10102540 0x7f0f101a0850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:41680/0 (socket says 172.21.15.42:41680) 2024-08-30T09:03:09.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.710+0000 7f0f14bc4640 1 -- 172.21.15.42:0/2757690320 learned_addr learned my addr 172.21.15.42:0/2757690320 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:03:09.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.710+0000 7f0f16e28640 1 -- 172.21.15.42:0/2757690320 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f0f10119e20 con 0x7f0f10102e90 2024-08-30T09:03:09.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.710+0000 7f0f16e28640 1 -- 172.21.15.42:0/2757690320 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f0f10119ca0 con 0x7f0f10107980 2024-08-30T09:03:09.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.710+0000 7f0f16e28640 1 -- 172.21.15.42:0/2757690320 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f0f10119fa0 con 0x7f0f10102540 2024-08-30T09:03:09.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.711+0000 7f0f0ffff640 1 --2- 172.21.15.42:0/2757690320 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0f10102e90 0x7f0f101a0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:03:09.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.711+0000 7f0f153c5640 1 --2- 172.21.15.42:0/2757690320 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0f10107980 0x7f0f101a6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:03:09.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.711+0000 7f0f14bc4640 1 -- 172.21.15.42:0/2757690320 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0f10107980 msgr2=0x7f0f101a6090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:03:09.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.711+0000 7f0f14bc4640 1 --2- 172.21.15.42:0/2757690320 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0f10107980 0x7f0f101a6090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:09.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.711+0000 7f0f14bc4640 1 -- 172.21.15.42:0/2757690320 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0f10102e90 msgr2=0x7f0f101a0df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:03:09.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.711+0000 7f0f14bc4640 1 --2- 172.21.15.42:0/2757690320 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0f10102e90 0x7f0f101a0df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:09.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.711+0000 7f0f14bc4640 1 -- 172.21.15.42:0/2757690320 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f101a67c0 con 0x7f0f10102540 2024-08-30T09:03:09.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.712+0000 7f0f14bc4640 1 --2- 172.21.15.42:0/2757690320 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0f10102540 0x7f0f101a0850 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f0f04009830 tx=0x7f0f04033d60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:03:09.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.712+0000 7f0f0dffb640 1 -- 172.21.15.42:0/2757690320 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f0403f030 con 0x7f0f10102540 2024-08-30T09:03:09.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.712+0000 7f0f16e28640 1 -- 172.21.15.42:0/2757690320 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f101a6a50 con 0x7f0f10102540 2024-08-30T09:03:09.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.712+0000 7f0f0dffb640 1 -- 172.21.15.42:0/2757690320 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f04004020 con 0x7f0f10102540 2024-08-30T09:03:09.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.712+0000 7f0f0dffb640 1 -- 172.21.15.42:0/2757690320 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f04037040 con 0x7f0f10102540 2024-08-30T09:03:09.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.712+0000 7f0f16e28640 1 -- 172.21.15.42:0/2757690320 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f101ad210 con 0x7f0f10102540 2024-08-30T09:03:09.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.716+0000 7f0f1d39d740 1 Processor -- start 2024-08-30T09:03:09.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.716+0000 7f0f1d39d740 1 -- start start 2024-08-30T09:03:09.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.716+0000 7f0f1d39d740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563cdcec7070 0x563cdd26c370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:09.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.716+0000 7f0f1d39d740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563cdce9e1b0 0x563cdd26c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:09.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.716+0000 7f0f1d39d740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563cdcea6390 0x563cdd26cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:09.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.716+0000 7f0f1d39d740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x563cdd1e3680 con 0x563cdce9e1b0 2024-08-30T09:03:09.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.716+0000 7f0f1d39d740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x563cdd1d2c10 con 0x563cdcea6390 2024-08-30T09:03:09.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.716+0000 7f0f1d39d740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x563cdd1d6bc0 con 0x563cdcec7070 2024-08-30T09:03:09.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.717+0000 7f0f153c5640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563cdcec7070 0x563cdd26c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:03:09.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.717+0000 7f0f153c5640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563cdcec7070 0x563cdd26c370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:41690/0 (socket says 172.21.15.42:41690) 2024-08-30T09:03:09.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.717+0000 7f0f153c5640 1 -- 172.21.15.42:0/2668589077 learned_addr learned my addr 172.21.15.42:0/2668589077 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:03:09.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.717+0000 7f0f0ffff640 1 --2- 172.21.15.42:0/2668589077 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563cdcea6390 0x563cdd26cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:03:09.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.717+0000 7f0f153c5640 1 -- 172.21.15.42:0/2668589077 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563cdcea6390 msgr2=0x563cdd26cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:03:09.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.717+0000 7f0f153c5640 1 --2- 172.21.15.42:0/2668589077 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563cdcea6390 0x563cdd26cdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:09.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.717+0000 7f0f153c5640 1 -- 172.21.15.42:0/2668589077 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563cdce9e1b0 msgr2=0x563cdd26c8b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-30T09:03:09.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.717+0000 7f0f153c5640 1 --2- 172.21.15.42:0/2668589077 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563cdce9e1b0 0x563cdd26c8b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:09.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.717+0000 7f0f153c5640 1 -- 172.21.15.42:0/2668589077 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563cdcf32650 con 0x563cdcec7070 2024-08-30T09:03:09.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.719+0000 7f0f153c5640 1 --2- 172.21.15.42:0/2668589077 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563cdcec7070 0x563cdd26c370 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f0f08009ee0 tx=0x7f0f08010740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3630c43d1e84ab84 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:03:09.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.719+0000 7f0f0dffb640 1 -- 172.21.15.42:0/2757690320 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7f0f04042070 con 0x7f0f10102540 2024-08-30T09:03:09.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.720+0000 7f0efaffd640 1 -- 172.21.15.42:0/2668589077 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f08016070 con 0x563cdcec7070 2024-08-30T09:03:09.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.720+0000 7f0efaffd640 1 -- 172.21.15.42:0/2668589077 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f08004d20 con 0x563cdcec7070 2024-08-30T09:03:09.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.720+0000 7f0efaffd640 1 -- 172.21.15.42:0/2668589077 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f08012690 con 0x563cdcec7070 2024-08-30T09:03:09.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.721+0000 7f0f0dffb640 1 --2- 172.21.15.42:0/2757690320 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f0f000b4c30 0x7f0f000b7140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:09.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.721+0000 7f0f0dffb640 1 -- 172.21.15.42:0/2757690320 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f0f041051d0 con 0x7f0f10102540 2024-08-30T09:03:09.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.721+0000 7f0f1d39d740 1 -- 172.21.15.42:0/2668589077 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563cdcec7070 msgr2=0x563cdd26c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:03:09.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.721+0000 7f0f1d39d740 1 --2- 172.21.15.42:0/2668589077 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563cdcec7070 0x563cdd26c370 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f0f08009ee0 tx=0x7f0f08010740 comp rx=0 tx=0).stop 2024-08-30T09:03:09.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.721+0000 7f0f1d39d740 1 -- 172.21.15.42:0/2668589077 shutdown_connections 2024-08-30T09:03:09.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.721+0000 7f0f1d39d740 1 --2- 172.21.15.42:0/2668589077 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563cdcea6390 0x563cdd26cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:09.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.721+0000 7f0f1d39d740 1 --2- 172.21.15.42:0/2668589077 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563cdce9e1b0 0x563cdd26c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:09.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.721+0000 7f0f1d39d740 1 --2- 172.21.15.42:0/2668589077 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563cdcec7070 0x563cdd26c370 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:09.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.721+0000 7f0f1d39d740 1 -- 172.21.15.42:0/2668589077 >> 172.21.15.42:0/2668589077 conn(0x563cdcebb460 msgr2=0x563cdd26a260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:03:09.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.721+0000 7f0f14bc4640 1 --2- 172.21.15.42:0/2757690320 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f0f000b4c30 0x7f0f000b7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:03:09.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.721+0000 7f0f1d39d740 1 -- 172.21.15.42:0/2668589077 shutdown_connections 2024-08-30T09:03:09.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.721+0000 7f0f1d39d740 1 -- 172.21.15.42:0/2668589077 wait complete. 2024-08-30T09:03:09.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.722+0000 7f0f1d39d740 1 -- start start 2024-08-30T09:03:09.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.722+0000 7f0f1d39d740 1 Processor -- start 2024-08-30T09:03:09.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.722+0000 7f0f14bc4640 1 --2- 172.21.15.42:0/2757690320 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f0f000b4c30 0x7f0f000b7140 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f0f040cb7a0 tx=0x7f0f0402a380 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:03:09.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.723+0000 7f0f1d39d740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563cdce9e1b0 0x563cdd277ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:09.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.723+0000 7f0f1d39d740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563cdcea6390 0x563cdd278000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:09.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.723+0000 7f0f1d39d740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563cdcec7070 0x563cdd283f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:09.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.723+0000 7f0f1d39d740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x563cdd035700 con 0x563cdce9e1b0 2024-08-30T09:03:09.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.723+0000 7f0f1d39d740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x563cdd159750 con 0x563cdcea6390 2024-08-30T09:03:09.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.723+0000 7f0f1d39d740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x563cdd27abf0 con 0x563cdcec7070 2024-08-30T09:03:09.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.723+0000 7f0f1d39d740 10 client.0 _ll_register_callbacks cb 0x563cdd188e10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:03:09.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.723+0000 7f0f153c5640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563cdce9e1b0 0x563cdd277ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:03:09.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.723+0000 7f0f153c5640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563cdce9e1b0 0x563cdd277ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:58672/0 (socket says 172.21.15.42:58672) 2024-08-30T09:03:09.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.723+0000 7f0f153c5640 1 -- 172.21.15.42:0/1667903559 learned_addr learned my addr 172.21.15.42:0/1667903559 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:03:09.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.723+0000 7f0f153c5640 1 --2- 172.21.15.42:0/1667903559 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563cdcec7070 0x563cdd283f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:03:09.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.723+0000 7f0f0ffff640 1 --2- 172.21.15.42:0/1667903559 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563cdcea6390 0x563cdd278000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:03:09.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.723+0000 7f0f153c5640 1 -- 172.21.15.42:0/1667903559 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563cdcec7070 msgr2=0x563cdd283f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:03:09.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.723+0000 7f0f153c5640 1 --2- 172.21.15.42:0/1667903559 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563cdcec7070 0x563cdd283f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:09.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.723+0000 7f0f153c5640 1 -- 172.21.15.42:0/1667903559 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563cdcea6390 msgr2=0x563cdd278000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:03:09.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.723+0000 7f0f153c5640 1 --2- 172.21.15.42:0/1667903559 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563cdcea6390 0x563cdd278000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:09.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.723+0000 7f0f153c5640 1 -- 172.21.15.42:0/1667903559 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563cdd192900 con 0x563cdce9e1b0 2024-08-30T09:03:09.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.723+0000 7f0f0ffff640 1 --2- 172.21.15.42:0/1667903559 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563cdcea6390 0x563cdd278000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-30T09:03:09.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.724+0000 7f0f153c5640 1 --2- 172.21.15.42:0/1667903559 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563cdce9e1b0 0x563cdd277ac0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f0f08004cf0 tx=0x7f0f080053c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=319590ea5e649972 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:03:09.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.724+0000 7f0eeffff640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-08-30T09:03:09.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.724+0000 7f0eeffff640 1 -- 172.21.15.42:0/1667903559 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f08022040 con 0x563cdce9e1b0 2024-08-30T09:03:09.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.724+0000 7f0f1d39d740 1 -- 172.21.15.42:0/1667903559 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0f08004d20 con 0x563cdce9e1b0 2024-08-30T09:03:09.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.724+0000 7f0eeffff640 1 -- 172.21.15.42:0/1667903559 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f0f08012450 con 0x563cdce9e1b0 2024-08-30T09:03:09.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.724+0000 7f0eeffff640 1 -- 172.21.15.42:0/1667903559 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f0f08016040 con 0x563cdce9e1b0 2024-08-30T09:03:09.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.724+0000 7f0f1d39d740 10 client.15885 fetch_fsmap learned FSMap version 29 2024-08-30T09:03:09.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.724+0000 7f0f1d39d740 1 -- 172.21.15.42:0/1667903559 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563cdd2869f0 con 0x563cdce9e1b0 2024-08-30T09:03:09.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.725+0000 7f0eeffff640 1 -- 172.21.15.42:0/1667903559 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f08004d20 con 0x563cdce9e1b0 2024-08-30T09:03:09.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.725+0000 7f0eeffff640 1 -- 172.21.15.42:0/1667903559 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f0f08028070 con 0x563cdce9e1b0 2024-08-30T09:03:09.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.725+0000 7f0f1d39d740 10 client.15885 fetch_fsmap finished waiting for FSMap version 29 2024-08-30T09:03:09.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.725+0000 7f0f1d39d740 10 client.15885 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-08-30T09:03:09.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.725+0000 7f0f1d39d740 20 client.15885 populate_metadata read hostname 'smithi042' 2024-08-30T09:03:09.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.725+0000 7f0f1d39d740 1 --2- 172.21.15.42:0/1667903559 >> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] conn(0x563cdce859f0 0x563cdd289db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:09.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.725+0000 7f0f1d39d740 4 client.15885 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-30T09:03:09.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.725+0000 7f0f1d39d740 1 -- 172.21.15.42:0/1667903559 --> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563cdd1e55d0 con 0x563cdce859f0 2024-08-30T09:03:09.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.725+0000 7f0f0ffff640 1 --2- 172.21.15.42:0/1667903559 >> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] conn(0x563cdce859f0 0x563cdd289db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:03:09.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.726+0000 7f0eeffff640 10 client.15885 ms_handle_connect on v2:172.21.15.190:6838/719981110 2024-08-30T09:03:09.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.726+0000 7f0f0ffff640 1 --2- 172.21.15.42:0/1667903559 >> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] conn(0x563cdce859f0 0x563cdd289db0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e5520be0c5cf0dc1 server_cookie=e99202a93b95c69 in_seq=0 out_seq=0 2024-08-30T09:03:09.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.726+0000 7f0eeffff640 1 -- 172.21.15.42:0/1667903559 <== mds.1 v2:172.21.15.190:6838/719981110 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x563cdd1e55d0 con 0x563cdce859f0 2024-08-30T09:03:09.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.726+0000 7f0eeffff640 10 client.15885 handle_command_reply: tid=0 2024-08-30T09:03:09.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.726+0000 7f0f1d39d740 1 client.15885 shutdown 2024-08-30T09:03:09.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.727+0000 7f0f1d39d740 1 -- 172.21.15.42:0/1667903559 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563cdce9e1b0 msgr2=0x563cdd277ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:03:09.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.727+0000 7f0f1d39d740 1 --2- 172.21.15.42:0/1667903559 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563cdce9e1b0 0x563cdd277ac0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f0f08004cf0 tx=0x7f0f080053c0 comp rx=0 tx=0).stop 2024-08-30T09:03:09.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.727+0000 7f0f1d39d740 1 -- 172.21.15.42:0/1667903559 shutdown_connections 2024-08-30T09:03:09.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.727+0000 7f0f1d39d740 1 --2- 172.21.15.42:0/1667903559 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563cdcec7070 0x563cdd283f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:09.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.727+0000 7f0f1d39d740 1 --2- 172.21.15.42:0/1667903559 >> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] conn(0x563cdce859f0 0x563cdd289db0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:09.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.727+0000 7f0f1d39d740 1 --2- 172.21.15.42:0/1667903559 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563cdcea6390 0x563cdd278000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:09.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.727+0000 7f0f1d39d740 1 --2- 172.21.15.42:0/1667903559 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563cdce9e1b0 0x563cdd277ac0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:09.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.727+0000 7f0f1d39d740 1 -- 172.21.15.42:0/1667903559 >> 172.21.15.42:0/1667903559 conn(0x563cdcebb460 msgr2=0x563cdd26f500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:03:09.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.727+0000 7f0eeffff640 0 client.15885 ms_handle_reset on v2:172.21.15.190:6838/719981110 2024-08-30T09:03:09.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.727+0000 7f0f1d39d740 1 -- 172.21.15.42:0/1667903559 shutdown_connections 2024-08-30T09:03:09.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.727+0000 7f0f1d39d740 1 -- 172.21.15.42:0/1667903559 wait complete. 2024-08-30T09:03:09.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.727+0000 7f0f1d39d740 20 client.15885 trim_cache size 0 max 16384 2024-08-30T09:03:09.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.747+0000 7f0f1d39d740 1 Processor -- start 2024-08-30T09:03:09.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.747+0000 7f0f1d39d740 1 -- start start 2024-08-30T09:03:09.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.747+0000 7f0f1d39d740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563cdce9e1b0 0x563cdd26fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:09.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.747+0000 7f0f1d39d740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563cdcea6390 0x563cdd283e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:09.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.747+0000 7f0f1d39d740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563cdcec7070 0x563cdd274cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:09.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.747+0000 7f0f153c5640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563cdce9e1b0 0x563cdd26fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:03:09.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.747+0000 7f0f153c5640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563cdce9e1b0 0x563cdd26fb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:41708/0 (socket says 172.21.15.42:41708) 2024-08-30T09:03:09.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.747+0000 7f0f153c5640 1 -- 172.21.15.42:0/873696064 learned_addr learned my addr 172.21.15.42:0/873696064 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:03:09.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.747+0000 7f0f1d39d740 1 -- 172.21.15.42:0/873696064 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f0f101b97f0 con 0x563cdcec7070 2024-08-30T09:03:09.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.747+0000 7f0f1d39d740 1 -- 172.21.15.42:0/873696064 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f0f101b70c0 con 0x563cdcea6390 2024-08-30T09:03:09.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.747+0000 7f0f1d39d740 1 -- 172.21.15.42:0/873696064 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f0f101b4f10 con 0x563cdce9e1b0 2024-08-30T09:03:09.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.748+0000 7f0f0ffff640 1 --2- 172.21.15.42:0/873696064 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563cdcea6390 0x563cdd283e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:03:09.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.748+0000 7f0f153c5640 1 --2- 172.21.15.42:0/873696064 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563cdcec7070 0x563cdd274cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:03:09.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.748+0000 7f0f153c5640 1 -- 172.21.15.42:0/873696064 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563cdcea6390 msgr2=0x563cdd283e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:03:09.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.748+0000 7f0f153c5640 1 --2- 172.21.15.42:0/873696064 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563cdcea6390 0x563cdd283e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:09.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.748+0000 7f0f153c5640 1 -- 172.21.15.42:0/873696064 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563cdcec7070 msgr2=0x563cdd274cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:03:09.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.748+0000 7f0f153c5640 1 --2- 172.21.15.42:0/873696064 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563cdcec7070 0x563cdd274cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:09.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.748+0000 7f0f153c5640 1 -- 172.21.15.42:0/873696064 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563cdd2700c0 con 0x563cdce9e1b0 2024-08-30T09:03:09.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.748+0000 7f0f153c5640 1 --2- 172.21.15.42:0/873696064 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563cdce9e1b0 0x563cdd26fb80 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f0f08009700 tx=0x7f0f08010740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d5d0c7f524af97a1 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:03:09.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.748+0000 7f0efa7fc640 1 -- 172.21.15.42:0/873696064 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f0801a3e0 con 0x563cdce9e1b0 2024-08-30T09:03:09.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.748+0000 7f0efa7fc640 1 -- 172.21.15.42:0/873696064 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f08004d20 con 0x563cdce9e1b0 2024-08-30T09:03:09.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.748+0000 7f0efa7fc640 1 -- 172.21.15.42:0/873696064 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f080126b0 con 0x563cdce9e1b0 2024-08-30T09:03:09.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.750+0000 7f0f1d39d740 1 -- 172.21.15.42:0/873696064 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563cdce9e1b0 msgr2=0x563cdd26fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:03:09.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.750+0000 7f0f1d39d740 1 --2- 172.21.15.42:0/873696064 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563cdce9e1b0 0x563cdd26fb80 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f0f08009700 tx=0x7f0f08010740 comp rx=0 tx=0).stop 2024-08-30T09:03:09.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.750+0000 7f0f1d39d740 1 -- 172.21.15.42:0/873696064 shutdown_connections 2024-08-30T09:03:09.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.750+0000 7f0f1d39d740 1 --2- 172.21.15.42:0/873696064 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563cdcec7070 0x563cdd274cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:09.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.750+0000 7f0f1d39d740 1 --2- 172.21.15.42:0/873696064 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563cdcea6390 0x563cdd283e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:09.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.750+0000 7f0f1d39d740 1 --2- 172.21.15.42:0/873696064 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563cdce9e1b0 0x563cdd26fb80 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:09.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.750+0000 7f0f1d39d740 1 -- 172.21.15.42:0/873696064 >> 172.21.15.42:0/873696064 conn(0x563cdcebb460 msgr2=0x563cdd26f470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:03:09.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.750+0000 7f0f1d39d740 1 -- 172.21.15.42:0/873696064 shutdown_connections 2024-08-30T09:03:09.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.751+0000 7f0f1d39d740 1 -- 172.21.15.42:0/873696064 wait complete. 2024-08-30T09:03:09.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.751+0000 7f0f1d39d740 1 -- start start 2024-08-30T09:03:09.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.751+0000 7f0f1d39d740 1 Processor -- start 2024-08-30T09:03:09.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.752+0000 7f0f1d39d740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563cdce9e1b0 0x563cdd280560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:09.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.752+0000 7f0f1d39d740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563cdcea6390 0x563cdd27ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:09.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.752+0000 7f0f1d39d740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563cdcec7070 0x563cdd27bf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:09.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.752+0000 7f0f1d39d740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f0f101b5580 con 0x563cdcec7070 2024-08-30T09:03:09.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.752+0000 7f0f1d39d740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f0f101b69f0 con 0x563cdcea6390 2024-08-30T09:03:09.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.752+0000 7f0f1d39d740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f0f101b5c50 con 0x563cdce9e1b0 2024-08-30T09:03:09.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.752+0000 7f0f1d39d740 10 client.0 _ll_register_callbacks cb 0x7f0f08009d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:03:09.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.752+0000 7f0f153c5640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563cdce9e1b0 0x563cdd280560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:03:09.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.752+0000 7f0f153c5640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563cdce9e1b0 0x563cdd280560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:41722/0 (socket says 172.21.15.42:41722) 2024-08-30T09:03:09.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.752+0000 7f0f153c5640 1 -- 172.21.15.42:0/2493713345 learned_addr learned my addr 172.21.15.42:0/2493713345 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:03:09.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.753+0000 7f0f153c5640 1 -- 172.21.15.42:0/2493713345 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563cdcea6390 msgr2=0x563cdd27ba10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-30T09:03:09.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.753+0000 7f0f153c5640 1 --2- 172.21.15.42:0/2493713345 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563cdcea6390 0x563cdd27ba10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:09.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.753+0000 7f0f153c5640 1 -- 172.21.15.42:0/2493713345 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563cdcec7070 msgr2=0x563cdd27bf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:03:09.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.753+0000 7f0f153c5640 1 --2- 172.21.15.42:0/2493713345 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563cdcec7070 0x563cdd27bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:09.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.753+0000 7f0f153c5640 1 -- 172.21.15.42:0/2493713345 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563cdd27c5c0 con 0x563cdce9e1b0 2024-08-30T09:03:09.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.753+0000 7f0f153c5640 1 --2- 172.21.15.42:0/2493713345 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563cdce9e1b0 0x563cdd280560 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f0f08006010 tx=0x7f0f08004f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2bf7129ea3f32db6 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:03:09.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.753+0000 7f0eeffff640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-08-30T09:03:09.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.754+0000 7f0eeffff640 1 -- 172.21.15.42:0/2493713345 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f08005830 con 0x563cdce9e1b0 2024-08-30T09:03:09.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.754+0000 7f0eeffff640 1 -- 172.21.15.42:0/2493713345 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f0f08012450 con 0x563cdce9e1b0 2024-08-30T09:03:09.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.754+0000 7f0f1d39d740 1 -- 172.21.15.42:0/2493713345 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0f08004d20 con 0x563cdce9e1b0 2024-08-30T09:03:09.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.754+0000 7f0eeffff640 1 -- 172.21.15.42:0/2493713345 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f0f08012650 con 0x563cdce9e1b0 2024-08-30T09:03:09.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.754+0000 7f0f1d39d740 10 client.25576 fetch_fsmap learned FSMap version 29 2024-08-30T09:03:09.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.754+0000 7f0f1d39d740 1 -- 172.21.15.42:0/2493713345 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563cdd3053c0 con 0x563cdce9e1b0 2024-08-30T09:03:09.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.755+0000 7f0eeffff640 1 -- 172.21.15.42:0/2493713345 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f08043070 con 0x563cdce9e1b0 2024-08-30T09:03:09.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.755+0000 7f0eeffff640 1 -- 172.21.15.42:0/2493713345 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f0f08028070 con 0x563cdce9e1b0 2024-08-30T09:03:09.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.755+0000 7f0f1d39d740 10 client.25576 fetch_fsmap finished waiting for FSMap version 29 2024-08-30T09:03:09.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.755+0000 7f0f1d39d740 10 client.25576 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-08-30T09:03:09.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.755+0000 7f0f1d39d740 20 client.25576 populate_metadata read hostname 'smithi042' 2024-08-30T09:03:09.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.755+0000 7f0f1d39d740 1 --2- 172.21.15.42:0/2493713345 >> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] conn(0x563cdce5ada0 0x563cdd3078b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:09.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.755+0000 7f0f1d39d740 4 client.25576 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-30T09:03:09.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.755+0000 7f0f1d39d740 1 -- 172.21.15.42:0/2493713345 --> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f0f08004d20 con 0x563cdce5ada0 2024-08-30T09:03:09.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.755+0000 7f0f0ffff640 1 --2- 172.21.15.42:0/2493713345 >> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] conn(0x563cdce5ada0 0x563cdd3078b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:03:09.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.756+0000 7f0eeffff640 10 client.25576 ms_handle_connect on v2:172.21.15.190:6838/719981110 2024-08-30T09:03:09.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.756+0000 7f0f0ffff640 1 --2- 172.21.15.42:0/2493713345 >> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] conn(0x563cdce5ada0 0x563cdd3078b0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ee3e5adcc688dfc0 server_cookie=2b2df3347ebcda44 in_seq=0 out_seq=0 2024-08-30T09:03:09.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.756+0000 7f0eeffff640 1 -- 172.21.15.42:0/2493713345 <== mds.1 v2:172.21.15.190:6838/719981110 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f0f08004d20 con 0x563cdce5ada0 2024-08-30T09:03:09.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.756+0000 7f0eeffff640 10 client.25576 handle_command_reply: tid=0 2024-08-30T09:03:09.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.756+0000 7f0f1d39d740 1 client.25576 shutdown 2024-08-30T09:03:09.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.756+0000 7f0f1d39d740 1 -- 172.21.15.42:0/2493713345 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563cdce9e1b0 msgr2=0x563cdd280560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:03:09.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.756+0000 7f0f1d39d740 1 --2- 172.21.15.42:0/2493713345 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563cdce9e1b0 0x563cdd280560 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f0f08006010 tx=0x7f0f08004f40 comp rx=0 tx=0).stop 2024-08-30T09:03:09.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.756+0000 7f0f1d39d740 1 -- 172.21.15.42:0/2493713345 shutdown_connections 2024-08-30T09:03:09.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.756+0000 7f0f1d39d740 1 --2- 172.21.15.42:0/2493713345 >> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] conn(0x563cdce5ada0 0x563cdd3078b0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:09.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.756+0000 7f0eeffff640 0 client.25576 ms_handle_reset on v2:172.21.15.190:6838/719981110 2024-08-30T09:03:09.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.756+0000 7f0f1d39d740 1 --2- 172.21.15.42:0/2493713345 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563cdcec7070 0x563cdd27bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:09.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.756+0000 7f0f1d39d740 1 --2- 172.21.15.42:0/2493713345 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563cdcea6390 0x563cdd27ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:09.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.756+0000 7f0f1d39d740 1 --2- 172.21.15.42:0/2493713345 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563cdce9e1b0 0x563cdd280560 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:09.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.756+0000 7f0f1d39d740 1 -- 172.21.15.42:0/2493713345 >> 172.21.15.42:0/2493713345 conn(0x563cdcebb460 msgr2=0x563cdd26dc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:03:09.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.757+0000 7f0f1d39d740 1 -- 172.21.15.42:0/2493713345 shutdown_connections 2024-08-30T09:03:09.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.757+0000 7f0f1d39d740 1 -- 172.21.15.42:0/2493713345 wait complete. 2024-08-30T09:03:09.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.757+0000 7f0f1d39d740 20 client.25576 trim_cache size 0 max 16384 2024-08-30T09:03:09.762 INFO:teuthology.orchestra.run.smithi042.stdout:[] 2024-08-30T09:03:09.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.758+0000 7f0f16e28640 1 -- 172.21.15.42:0/2757690320 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f0f000b4c30 msgr2=0x7f0f000b7140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:03:09.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.758+0000 7f0f16e28640 1 --2- 172.21.15.42:0/2757690320 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f0f000b4c30 0x7f0f000b7140 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f0f040cb7a0 tx=0x7f0f0402a380 comp rx=0 tx=0).stop 2024-08-30T09:03:09.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.758+0000 7f0f16e28640 1 -- 172.21.15.42:0/2757690320 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0f10102540 msgr2=0x7f0f101a0850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:03:09.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.758+0000 7f0f16e28640 1 --2- 172.21.15.42:0/2757690320 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0f10102540 0x7f0f101a0850 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f0f04009830 tx=0x7f0f04033d60 comp rx=0 tx=0).stop 2024-08-30T09:03:09.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.758+0000 7f0f16e28640 1 -- 172.21.15.42:0/2757690320 shutdown_connections 2024-08-30T09:03:09.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.758+0000 7f0f16e28640 1 --2- 172.21.15.42:0/2757690320 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f0f000b4c30 0x7f0f000b7140 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:09.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.758+0000 7f0f16e28640 1 --2- 172.21.15.42:0/2757690320 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0f10107980 0x7f0f101a6090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:09.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.758+0000 7f0f16e28640 1 --2- 172.21.15.42:0/2757690320 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0f10102e90 0x7f0f101a0df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:09.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.758+0000 7f0f16e28640 1 --2- 172.21.15.42:0/2757690320 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0f10102540 0x7f0f101a0850 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:09.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.758+0000 7f0f16e28640 1 -- 172.21.15.42:0/2757690320 >> 172.21.15.42:0/2757690320 conn(0x7f0f100794d0 msgr2=0x7f0f10105390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:03:09.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.758+0000 7f0f16e28640 1 -- 172.21.15.42:0/2757690320 shutdown_connections 2024-08-30T09:03:09.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:09.758+0000 7f0f16e28640 1 -- 172.21.15.42:0/2757690320 wait complete. 2024-08-30T09:03:10.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:03:10 smithi042 ceph-mon[18652]: pgmap v445: 129 pgs: 129 active+clean; 1.1 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 13 MiB/s wr, 897 op/s 2024-08-30T09:03:10.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:03:10 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T09:03:10.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:03:10 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-30T09:03:10.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:03:10 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T09:03:10.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:03:10 smithi190 ceph-mon[22001]: pgmap v445: 129 pgs: 129 active+clean; 1.1 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 13 MiB/s wr, 897 op/s 2024-08-30T09:03:10.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:03:10 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T09:03:10.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:03:10 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-30T09:03:10.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:03:10 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T09:03:10.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:03:10 smithi167 ceph-mon[22063]: pgmap v445: 129 pgs: 129 active+clean; 1.1 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 13 MiB/s wr, 897 op/s 2024-08-30T09:03:10.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:03:10 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T09:03:10.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:03:10 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-30T09:03:10.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:03:10 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T09:03:10.622 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-08-30T09:03:10.956 INFO:teuthology.orchestra.run.smithi042.stderr:progress: 145.0 s, 1773.0 tps, lat 55.755 ms stddev 68.289 2024-08-30T09:03:11.024 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T09:03:12.292 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.289+0000 7fabcfe5a640 1 -- 172.21.15.42:0/2107894275 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fabc8103d60 msgr2=0x7fabc810a700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:03:12.292 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.289+0000 7fabcfe5a640 1 --2- 172.21.15.42:0/2107894275 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fabc8103d60 0x7fabc810a700 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fabc0009bd0 tx=0x7fabc002a6a0 comp rx=0 tx=0).stop 2024-08-30T09:03:12.292 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.289+0000 7fabcfe5a640 1 -- 172.21.15.42:0/2107894275 shutdown_connections 2024-08-30T09:03:12.292 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.289+0000 7fabcfe5a640 1 --2- 172.21.15.42:0/2107894275 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fabc8103d60 0x7fabc810a700 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:12.293 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.289+0000 7fabcfe5a640 1 --2- 172.21.15.42:0/2107894275 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fabc81033c0 0x7fabc81037d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:12.293 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.289+0000 7fabcfe5a640 1 --2- 172.21.15.42:0/2107894275 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fabc8102340 0x7fabc81027b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:12.293 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.289+0000 7fabcfe5a640 1 -- 172.21.15.42:0/2107894275 >> 172.21.15.42:0/2107894275 conn(0x7fabc80fdc80 msgr2=0x7fabc8100130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:03:12.293 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.289+0000 7fabcfe5a640 1 -- 172.21.15.42:0/2107894275 shutdown_connections 2024-08-30T09:03:12.293 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.289+0000 7fabcfe5a640 1 -- 172.21.15.42:0/2107894275 wait complete. 2024-08-30T09:03:12.295 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.290+0000 7fabcfe5a640 1 Processor -- start 2024-08-30T09:03:12.295 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.290+0000 7fabcfe5a640 1 -- start start 2024-08-30T09:03:12.295 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.290+0000 7fabcfe5a640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fabc8102340 0x7fabc8111af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:12.295 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.290+0000 7fabcfe5a640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fabc81033c0 0x7fabc8112160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:12.295 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.290+0000 7fabcfe5a640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fabc8103d60 0x7fabc8112cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:12.295 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.291+0000 7fabcfe5a640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fabc810bda0 con 0x7fabc8103d60 2024-08-30T09:03:12.295 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.291+0000 7fabcfe5a640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fabc810bc20 con 0x7fabc81033c0 2024-08-30T09:03:12.295 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.291+0000 7fabcfe5a640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fabc810bf20 con 0x7fabc8102340 2024-08-30T09:03:12.295 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.291+0000 7fabce3f7640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fabc8103d60 0x7fabc8112cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:03:12.295 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.291+0000 7fabce3f7640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fabc8103d60 0x7fabc8112cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:58704/0 (socket says 172.21.15.42:58704) 2024-08-30T09:03:12.295 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.291+0000 7fabce3f7640 1 -- 172.21.15.42:0/1127866408 learned_addr learned my addr 172.21.15.42:0/1127866408 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:03:12.295 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.291+0000 7fabcdbf6640 1 --2- 172.21.15.42:0/1127866408 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fabc8102340 0x7fabc8111af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:03:12.296 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.291+0000 7fabcd3f5640 1 --2- 172.21.15.42:0/1127866408 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fabc81033c0 0x7fabc8112160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:03:12.296 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.291+0000 7fabce3f7640 1 -- 172.21.15.42:0/1127866408 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fabc8102340 msgr2=0x7fabc8111af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:03:12.296 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.291+0000 7fabce3f7640 1 --2- 172.21.15.42:0/1127866408 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fabc8102340 0x7fabc8111af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:12.296 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.291+0000 7fabce3f7640 1 -- 172.21.15.42:0/1127866408 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fabc81033c0 msgr2=0x7fabc8112160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:03:12.296 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.291+0000 7fabce3f7640 1 --2- 172.21.15.42:0/1127866408 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fabc81033c0 0x7fabc8112160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:12.296 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.291+0000 7fabce3f7640 1 -- 172.21.15.42:0/1127866408 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabc81134f0 con 0x7fabc8103d60 2024-08-30T09:03:12.296 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.291+0000 7fabcdbf6640 1 --2- 172.21.15.42:0/1127866408 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fabc8102340 0x7fabc8111af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-30T09:03:12.296 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.291+0000 7fabcd3f5640 1 --2- 172.21.15.42:0/1127866408 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fabc81033c0 0x7fabc8112160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-30T09:03:12.296 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.291+0000 7fabce3f7640 1 --2- 172.21.15.42:0/1127866408 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fabc8103d60 0x7fabc8112cd0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fabc00011e0 tx=0x7fabc0031440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:03:12.296 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.292+0000 7fabbeffd640 1 -- 172.21.15.42:0/1127866408 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabc0040020 con 0x7fabc8103d60 2024-08-30T09:03:12.296 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.292+0000 7fabcfe5a640 1 -- 172.21.15.42:0/1127866408 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabc8113780 con 0x7fabc8103d60 2024-08-30T09:03:12.296 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.292+0000 7fabbeffd640 1 -- 172.21.15.42:0/1127866408 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fabc00318f0 con 0x7fabc8103d60 2024-08-30T09:03:12.296 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.292+0000 7fabbeffd640 1 -- 172.21.15.42:0/1127866408 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabc0036040 con 0x7fabc8103d60 2024-08-30T09:03:12.297 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.292+0000 7fabcfe5a640 1 -- 172.21.15.42:0/1127866408 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabc81b0d40 con 0x7fabc8103d60 2024-08-30T09:03:12.298 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.295+0000 7fabd6dca740 1 Processor -- start 2024-08-30T09:03:12.298 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.295+0000 7fabbeffd640 1 -- 172.21.15.42:0/1127866408 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7fabc0043020 con 0x7fabc8103d60 2024-08-30T09:03:12.298 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.296+0000 7fabd6dca740 1 -- start start 2024-08-30T09:03:12.299 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.296+0000 7fabd6dca740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55666e3d11b0 0x55666e777180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:12.299 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.296+0000 7fabd6dca740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55666e3a82f0 0x55666e7777f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:12.299 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.296+0000 7fabd6dca740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55666e3b04d0 0x55666e778340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:12.299 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.296+0000 7fabd6dca740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55666e6ea000 con 0x55666e3d11b0 2024-08-30T09:03:12.299 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.296+0000 7fabd6dca740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55666e6d9590 con 0x55666e3b04d0 2024-08-30T09:03:12.299 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.296+0000 7fabd6dca740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55666e6dd540 con 0x55666e3a82f0 2024-08-30T09:03:12.300 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.296+0000 7fabcd3f5640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55666e3b04d0 0x55666e778340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:03:12.301 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.296+0000 7fabcd3f5640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55666e3d11b0 0x55666e777180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:03:12.301 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.296+0000 7fabcd3f5640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55666e3d11b0 0x55666e777180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:58720/0 (socket says 172.21.15.42:58720) 2024-08-30T09:03:12.301 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.296+0000 7fabcd3f5640 1 -- 172.21.15.42:0/1182144978 learned_addr learned my addr 172.21.15.42:0/1182144978 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:03:12.301 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.296+0000 7fabcdbf6640 1 --2- 172.21.15.42:0/1182144978 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55666e3a82f0 0x55666e7777f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:03:12.301 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.297+0000 7fabcd3f5640 1 -- 172.21.15.42:0/1182144978 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55666e3a82f0 msgr2=0x55666e7777f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:03:12.301 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.297+0000 7fabcd3f5640 1 --2- 172.21.15.42:0/1182144978 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55666e3a82f0 0x55666e7777f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:12.301 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.297+0000 7fabcd3f5640 1 -- 172.21.15.42:0/1182144978 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55666e3d11b0 msgr2=0x55666e777180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:03:12.301 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.297+0000 7fabcd3f5640 1 --2- 172.21.15.42:0/1182144978 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55666e3d11b0 0x55666e777180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:12.301 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.297+0000 7fabcd3f5640 1 -- 172.21.15.42:0/1182144978 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55666e43c790 con 0x55666e3b04d0 2024-08-30T09:03:12.301 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.297+0000 7fabcd3f5640 1 --2- 172.21.15.42:0/1182144978 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55666e3b04d0 0x55666e778340 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fabb8010d70 tx=0x7fabb80096c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bd6fd6b5fd36021d server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:03:12.301 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.297+0000 7fabbeffd640 1 --2- 172.21.15.42:0/1127866408 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fabb40b4b30 0x7fabb40b7040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:12.301 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.297+0000 7fabb3fff640 1 -- 172.21.15.42:0/1182144978 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabb8016030 con 0x55666e3b04d0 2024-08-30T09:03:12.301 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.297+0000 7fabcdbf6640 1 --2- 172.21.15.42:0/1127866408 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fabb40b4b30 0x7fabb40b7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:03:12.302 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.297+0000 7fabb3fff640 1 -- 172.21.15.42:0/1182144978 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fabb800fe50 con 0x55666e3b04d0 2024-08-30T09:03:12.302 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.297+0000 7fabb3fff640 1 -- 172.21.15.42:0/1182144978 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabb8019040 con 0x55666e3b04d0 2024-08-30T09:03:12.305 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.298+0000 7fabd6dca740 1 -- 172.21.15.42:0/1182144978 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55666e3b04d0 msgr2=0x55666e778340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:03:12.305 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.298+0000 7fabd6dca740 1 --2- 172.21.15.42:0/1182144978 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55666e3b04d0 0x55666e778340 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fabb8010d70 tx=0x7fabb80096c0 comp rx=0 tx=0).stop 2024-08-30T09:03:12.305 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.298+0000 7fabbeffd640 1 -- 172.21.15.42:0/1127866408 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7fabc0106020 con 0x7fabc8103d60 2024-08-30T09:03:12.305 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.299+0000 7fabcdbf6640 1 --2- 172.21.15.42:0/1127866408 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fabb40b4b30 0x7fabb40b7040 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fabc4011630 tx=0x7fabc400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:03:12.305 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.299+0000 7fabd6dca740 1 -- 172.21.15.42:0/1182144978 shutdown_connections 2024-08-30T09:03:12.305 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.299+0000 7fabd6dca740 1 --2- 172.21.15.42:0/1182144978 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55666e3a82f0 0x55666e7777f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:12.305 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.299+0000 7fabd6dca740 1 --2- 172.21.15.42:0/1182144978 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55666e3b04d0 0x55666e778340 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:12.305 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.299+0000 7fabd6dca740 1 --2- 172.21.15.42:0/1182144978 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55666e3d11b0 0x55666e777180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:12.305 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.299+0000 7fabd6dca740 1 -- 172.21.15.42:0/1182144978 >> 172.21.15.42:0/1182144978 conn(0x55666e3c5300 msgr2=0x55666e772c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:03:12.305 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.299+0000 7fabd6dca740 1 -- 172.21.15.42:0/1182144978 shutdown_connections 2024-08-30T09:03:12.305 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.299+0000 7fabd6dca740 1 -- 172.21.15.42:0/1182144978 wait complete. 2024-08-30T09:03:12.306 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.299+0000 7fabd6dca740 1 -- start start 2024-08-30T09:03:12.306 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.299+0000 7fabd6dca740 1 Processor -- start 2024-08-30T09:03:12.306 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.300+0000 7fabd6dca740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55666e3a82f0 0x55666e780c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:12.306 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.300+0000 7fabd6dca740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55666e3b04d0 0x55666e781330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:12.306 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.300+0000 7fabd6dca740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55666e3d11b0 0x55666e781ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:12.306 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.300+0000 7fabd6dca740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55666e612ab0 con 0x55666e3b04d0 2024-08-30T09:03:12.306 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.300+0000 7fabd6dca740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55666e660680 con 0x55666e3d11b0 2024-08-30T09:03:12.306 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.300+0000 7fabd6dca740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55666e778e60 con 0x55666e3a82f0 2024-08-30T09:03:12.306 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.300+0000 7fabd6dca740 10 client.0 _ll_register_callbacks cb 0x55666e3e8a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:03:12.306 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.300+0000 7fabcdbf6640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55666e3a82f0 0x55666e780c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:03:12.306 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.300+0000 7fabcd3f5640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55666e3b04d0 0x55666e781330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:03:12.306 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.300+0000 7fabcdbf6640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55666e3a82f0 0x55666e780c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:41766/0 (socket says 172.21.15.42:41766) 2024-08-30T09:03:12.307 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.300+0000 7fabcdbf6640 1 -- 172.21.15.42:0/3112344913 learned_addr learned my addr 172.21.15.42:0/3112344913 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:03:12.307 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.300+0000 7fabce3f7640 1 --2- 172.21.15.42:0/3112344913 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55666e3d11b0 0x55666e781ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:03:12.307 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.300+0000 7fabcdbf6640 1 -- 172.21.15.42:0/3112344913 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55666e3d11b0 msgr2=0x55666e781ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:03:12.307 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.300+0000 7fabcdbf6640 1 --2- 172.21.15.42:0/3112344913 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55666e3d11b0 0x55666e781ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:12.307 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.300+0000 7fabcdbf6640 1 -- 172.21.15.42:0/3112344913 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55666e3b04d0 msgr2=0x55666e781330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:03:12.307 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.300+0000 7fabcdbf6640 1 --2- 172.21.15.42:0/3112344913 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55666e3b04d0 0x55666e781330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:12.307 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.300+0000 7fabcdbf6640 1 -- 172.21.15.42:0/3112344913 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55666e4b5620 con 0x55666e3a82f0 2024-08-30T09:03:12.307 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.300+0000 7fabce3f7640 1 --2- 172.21.15.42:0/3112344913 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55666e3d11b0 0x55666e781ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-30T09:03:12.307 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.301+0000 7fabcdbf6640 1 --2- 172.21.15.42:0/3112344913 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55666e3a82f0 0x55666e780c70 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fabc40108d0 tx=0x7fabc40046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2e24c4552718f233 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:03:12.307 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.301+0000 7fabb17fa640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-08-30T09:03:12.307 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.301+0000 7fabb17fa640 1 -- 172.21.15.42:0/3112344913 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabc402c060 con 0x55666e3a82f0 2024-08-30T09:03:12.307 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.301+0000 7fabb17fa640 1 -- 172.21.15.42:0/3112344913 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7fabc4017030 con 0x55666e3a82f0 2024-08-30T09:03:12.307 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.301+0000 7fabd6dca740 1 -- 172.21.15.42:0/3112344913 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fabb800fe50 con 0x55666e3a82f0 2024-08-30T09:03:12.307 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.301+0000 7fabb17fa640 1 -- 172.21.15.42:0/3112344913 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fabc4033050 con 0x55666e3a82f0 2024-08-30T09:03:12.308 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.301+0000 7fabd6dca740 10 client.25588 fetch_fsmap learned FSMap version 29 2024-08-30T09:03:12.308 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.301+0000 7fabd6dca740 1 -- 172.21.15.42:0/3112344913 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55666e78aec0 con 0x55666e3a82f0 2024-08-30T09:03:12.308 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.301+0000 7fabb17fa640 1 -- 172.21.15.42:0/3112344913 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fabb800fe50 con 0x55666e3a82f0 2024-08-30T09:03:12.308 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.301+0000 7fabb17fa640 1 -- 172.21.15.42:0/3112344913 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fabc401cce0 con 0x55666e3a82f0 2024-08-30T09:03:12.308 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.301+0000 7fabd6dca740 10 client.25588 fetch_fsmap finished waiting for FSMap version 29 2024-08-30T09:03:12.308 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.301+0000 7fabd6dca740 10 client.25588 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-30T09:03:12.308 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.301+0000 7fabd6dca740 20 client.25588 populate_metadata read hostname 'smithi042' 2024-08-30T09:03:12.308 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.301+0000 7fabd6dca740 1 --2- 172.21.15.42:0/3112344913 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55666e38fd80 0x55666e78e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:12.308 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.302+0000 7fabd6dca740 4 client.25588 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-30T09:03:12.308 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.302+0000 7fabd6dca740 1 -- 172.21.15.42:0/3112344913 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55666e6ebf50 con 0x55666e38fd80 2024-08-30T09:03:12.308 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.302+0000 7fabcd3f5640 1 --2- 172.21.15.42:0/3112344913 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55666e38fd80 0x55666e78e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:03:12.310 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.302+0000 7fabb17fa640 10 client.25588 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:03:12.310 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.302+0000 7fabcd3f5640 1 --2- 172.21.15.42:0/3112344913 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55666e38fd80 0x55666e78e6b0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=610c15153c49c49 server_cookie=b74b7ab09b9c91b9 in_seq=0 out_seq=0 2024-08-30T09:03:12.310 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.303+0000 7fabb17fa640 1 -- 172.21.15.42:0/3112344913 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55666e6ebf50 con 0x55666e38fd80 2024-08-30T09:03:12.310 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.303+0000 7fabb17fa640 10 client.25588 handle_command_reply: tid=0 2024-08-30T09:03:12.310 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.303+0000 7fabd6dca740 1 client.25588 shutdown 2024-08-30T09:03:12.310 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.303+0000 7fabd6dca740 1 -- 172.21.15.42:0/3112344913 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55666e3a82f0 msgr2=0x55666e780c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:03:12.310 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.303+0000 7fabd6dca740 1 --2- 172.21.15.42:0/3112344913 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55666e3a82f0 0x55666e780c70 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fabc40108d0 tx=0x7fabc40046c0 comp rx=0 tx=0).stop 2024-08-30T09:03:12.310 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.304+0000 7fabd6dca740 1 -- 172.21.15.42:0/3112344913 shutdown_connections 2024-08-30T09:03:12.310 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.304+0000 7fabd6dca740 1 --2- 172.21.15.42:0/3112344913 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55666e38fd80 0x55666e78e6b0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:12.310 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.304+0000 7fabb17fa640 0 client.25588 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:03:12.311 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.304+0000 7fabd6dca740 1 --2- 172.21.15.42:0/3112344913 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55666e3d11b0 0x55666e781ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:12.311 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.304+0000 7fabd6dca740 1 --2- 172.21.15.42:0/3112344913 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55666e3b04d0 0x55666e781330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:12.311 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.304+0000 7fabd6dca740 1 --2- 172.21.15.42:0/3112344913 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55666e3a82f0 0x55666e780c70 secure :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fabc40108d0 tx=0x7fabc40046c0 comp rx=0 tx=0).stop 2024-08-30T09:03:12.311 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.304+0000 7fabd6dca740 1 -- 172.21.15.42:0/3112344913 >> 172.21.15.42:0/3112344913 conn(0x55666e3c5300 msgr2=0x55666e771170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:03:12.311 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.305+0000 7fabd6dca740 1 -- 172.21.15.42:0/3112344913 shutdown_connections 2024-08-30T09:03:12.311 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.306+0000 7fabd6dca740 1 -- 172.21.15.42:0/3112344913 wait complete. 2024-08-30T09:03:12.311 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.306+0000 7fabd6dca740 20 client.25588 trim_cache size 0 max 16384 2024-08-30T09:03:12.330 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.326+0000 7fabd6dca740 1 Processor -- start 2024-08-30T09:03:12.330 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.326+0000 7fabd6dca740 1 -- start start 2024-08-30T09:03:12.330 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.326+0000 7fabd6dca740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55666e38fd80 0x55666e76d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:12.330 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.326+0000 7fabd6dca740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55666e3b04d0 0x55666e7725c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:12.330 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.326+0000 7fabd6dca740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55666e3d11b0 0x55666e77bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:12.330 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.326+0000 7fabd6dca740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fabc81bdb30 con 0x55666e38fd80 2024-08-30T09:03:12.330 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.326+0000 7fabd6dca740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fabc81bb400 con 0x55666e3b04d0 2024-08-30T09:03:12.330 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.326+0000 7fabd6dca740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fabc81b9250 con 0x55666e3d11b0 2024-08-30T09:03:12.330 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.326+0000 7fabcdbf6640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55666e38fd80 0x55666e76d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:03:12.330 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.327+0000 7fabcd3f5640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55666e3b04d0 0x55666e7725c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:03:12.330 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.327+0000 7fabce3f7640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55666e3d11b0 0x55666e77bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:03:12.331 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.327+0000 7fabcd3f5640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55666e3b04d0 0x55666e7725c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:36446/0 (socket says 172.21.15.42:36446) 2024-08-30T09:03:12.331 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.327+0000 7fabcd3f5640 1 -- 172.21.15.42:0/410090238 learned_addr learned my addr 172.21.15.42:0/410090238 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:03:12.331 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.327+0000 7fabcd3f5640 1 -- 172.21.15.42:0/410090238 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55666e3d11b0 msgr2=0x55666e77bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:03:12.331 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.327+0000 7fabcd3f5640 1 --2- 172.21.15.42:0/410090238 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55666e3d11b0 0x55666e77bbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:12.331 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.327+0000 7fabcd3f5640 1 -- 172.21.15.42:0/410090238 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55666e38fd80 msgr2=0x55666e76d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:03:12.331 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.327+0000 7fabcd3f5640 1 --2- 172.21.15.42:0/410090238 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55666e38fd80 0x55666e76d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:12.331 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.327+0000 7fabcd3f5640 1 -- 172.21.15.42:0/410090238 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55666e76da30 con 0x55666e3b04d0 2024-08-30T09:03:12.331 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.327+0000 7fabcdbf6640 1 --2- 172.21.15.42:0/410090238 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55666e38fd80 0x55666e76d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-30T09:03:12.333 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.327+0000 7fabcd3f5640 1 --2- 172.21.15.42:0/410090238 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55666e3b04d0 0x55666e7725c0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fabb8000c00 tx=0x7fabb8001e50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6e7148af64f28ac3 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:03:12.333 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.327+0000 7fabb37fe640 1 -- 172.21.15.42:0/410090238 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabb8036070 con 0x55666e3b04d0 2024-08-30T09:03:12.333 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.328+0000 7fabb37fe640 1 -- 172.21.15.42:0/410090238 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fabb80054f0 con 0x55666e3b04d0 2024-08-30T09:03:12.333 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.328+0000 7fabb37fe640 1 -- 172.21.15.42:0/410090238 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabb8016040 con 0x55666e3b04d0 2024-08-30T09:03:12.333 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.328+0000 7fabd6dca740 1 -- 172.21.15.42:0/410090238 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55666e3b04d0 msgr2=0x55666e7725c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:03:12.334 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.328+0000 7fabd6dca740 1 --2- 172.21.15.42:0/410090238 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55666e3b04d0 0x55666e7725c0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fabb8000c00 tx=0x7fabb8001e50 comp rx=0 tx=0).stop 2024-08-30T09:03:12.334 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.329+0000 7fabd6dca740 1 -- 172.21.15.42:0/410090238 shutdown_connections 2024-08-30T09:03:12.334 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.329+0000 7fabd6dca740 1 --2- 172.21.15.42:0/410090238 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55666e3d11b0 0x55666e77bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:12.334 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.329+0000 7fabd6dca740 1 --2- 172.21.15.42:0/410090238 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55666e3b04d0 0x55666e7725c0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:12.334 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.329+0000 7fabd6dca740 1 --2- 172.21.15.42:0/410090238 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55666e38fd80 0x55666e76d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:12.334 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.329+0000 7fabd6dca740 1 -- 172.21.15.42:0/410090238 >> 172.21.15.42:0/410090238 conn(0x55666e3c5300 msgr2=0x55666e78e620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:03:12.334 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.329+0000 7fabd6dca740 1 -- 172.21.15.42:0/410090238 shutdown_connections 2024-08-30T09:03:12.334 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.329+0000 7fabd6dca740 1 -- 172.21.15.42:0/410090238 wait complete. 2024-08-30T09:03:12.334 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.329+0000 7fabd6dca740 1 -- start start 2024-08-30T09:03:12.334 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.329+0000 7fabd6dca740 1 Processor -- start 2024-08-30T09:03:12.334 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.330+0000 7fabd6dca740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55666e38fd80 0x55666e785db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:12.334 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.330+0000 7fabd6dca740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55666e3b04d0 0x55666e786470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:12.334 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.330+0000 7fabd6dca740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55666e3d11b0 0x55666e786f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:12.334 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.330+0000 7fabd6dca740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fabc81b98c0 con 0x55666e38fd80 2024-08-30T09:03:12.335 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.330+0000 7fabd6dca740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fabc81bad30 con 0x55666e3b04d0 2024-08-30T09:03:12.335 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.330+0000 7fabd6dca740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fabc81b9f90 con 0x55666e3d11b0 2024-08-30T09:03:12.335 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.330+0000 7fabd6dca740 10 client.0 _ll_register_callbacks cb 0x7fabc400be50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:03:12.335 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.330+0000 7fabcd3f5640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55666e3b04d0 0x55666e786470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:03:12.335 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.330+0000 7fabcd3f5640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55666e3b04d0 0x55666e786470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:36458/0 (socket says 172.21.15.42:36458) 2024-08-30T09:03:12.335 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.330+0000 7fabcd3f5640 1 -- 172.21.15.42:0/3559287808 learned_addr learned my addr 172.21.15.42:0/3559287808 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:03:12.335 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.330+0000 7fabce3f7640 1 --2- 172.21.15.42:0/3559287808 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55666e3d11b0 0x55666e786f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:03:12.335 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.330+0000 7fabcdbf6640 1 --2- 172.21.15.42:0/3559287808 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55666e38fd80 0x55666e785db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:03:12.335 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.330+0000 7fabcd3f5640 1 -- 172.21.15.42:0/3559287808 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55666e3d11b0 msgr2=0x55666e786f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:03:12.335 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.330+0000 7fabcd3f5640 1 --2- 172.21.15.42:0/3559287808 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55666e3d11b0 0x55666e786f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:12.335 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.330+0000 7fabcd3f5640 1 -- 172.21.15.42:0/3559287808 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55666e38fd80 msgr2=0x55666e785db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:03:12.335 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.330+0000 7fabcd3f5640 1 --2- 172.21.15.42:0/3559287808 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55666e38fd80 0x55666e785db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:12.335 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.330+0000 7fabcd3f5640 1 -- 172.21.15.42:0/3559287808 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55666e7876d0 con 0x55666e3b04d0 2024-08-30T09:03:12.335 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.330+0000 7fabcdbf6640 1 --2- 172.21.15.42:0/3559287808 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55666e38fd80 0x55666e785db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-30T09:03:12.338 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.334+0000 7fabcd3f5640 1 --2- 172.21.15.42:0/3559287808 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55666e3b04d0 0x55666e786470 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fabb8005750 tx=0x7fabb8005780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ae281eb9df54cf99 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:03:12.338 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.334+0000 7fabb17fa640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-30T09:03:12.338 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.334+0000 7fabb17fa640 1 -- 172.21.15.42:0/3559287808 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabb8016070 con 0x55666e3b04d0 2024-08-30T09:03:12.338 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.334+0000 7fabb17fa640 1 -- 172.21.15.42:0/3559287808 <== mon.2 v2:172.21.15.167:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7fabb8005040 con 0x55666e3b04d0 2024-08-30T09:03:12.338 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.334+0000 7fabd6dca740 1 -- 172.21.15.42:0/3559287808 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fabb80054f0 con 0x55666e3b04d0 2024-08-30T09:03:12.338 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.334+0000 7fabb17fa640 1 -- 172.21.15.42:0/3559287808 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fabb8036040 con 0x55666e3b04d0 2024-08-30T09:03:12.338 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.335+0000 7fabd6dca740 10 client.25580 fetch_fsmap learned FSMap version 29 2024-08-30T09:03:12.339 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.335+0000 7fabd6dca740 1 -- 172.21.15.42:0/3559287808 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55666e7889f0 con 0x55666e3b04d0 2024-08-30T09:03:12.339 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.335+0000 7fabb17fa640 1 -- 172.21.15.42:0/3559287808 <== mon.2 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fabb80054f0 con 0x55666e3b04d0 2024-08-30T09:03:12.339 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.335+0000 7fabb17fa640 1 -- 172.21.15.42:0/3559287808 <== mon.2 v2:172.21.15.167:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fabb8026040 con 0x55666e3b04d0 2024-08-30T09:03:12.339 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.335+0000 7fabd6dca740 10 client.25580 fetch_fsmap finished waiting for FSMap version 29 2024-08-30T09:03:12.339 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.335+0000 7fabd6dca740 10 client.25580 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-30T09:03:12.339 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.335+0000 7fabd6dca740 20 client.25580 populate_metadata read hostname 'smithi042' 2024-08-30T09:03:12.339 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.335+0000 7fabd6dca740 1 --2- 172.21.15.42:0/3559287808 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55666e3651c0 0x55666e7fb080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:12.339 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.335+0000 7fabd6dca740 4 client.25580 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-30T09:03:12.339 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.335+0000 7fabd6dca740 1 -- 172.21.15.42:0/3559287808 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55666e6e79f0 con 0x55666e3651c0 2024-08-30T09:03:12.339 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.335+0000 7fabcdbf6640 1 --2- 172.21.15.42:0/3559287808 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55666e3651c0 0x55666e7fb080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:03:12.340 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.336+0000 7fabb17fa640 10 client.25580 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:03:12.340 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.336+0000 7fabcdbf6640 1 --2- 172.21.15.42:0/3559287808 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55666e3651c0 0x55666e7fb080 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d6bff06a4b85acc server_cookie=86f36a7211d770ec in_seq=0 out_seq=0 2024-08-30T09:03:12.355 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.351+0000 7fabb17fa640 1 -- 172.21.15.42:0/3559287808 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55666e6e79f0 con 0x55666e3651c0 2024-08-30T09:03:12.355 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.351+0000 7fabb17fa640 10 client.25580 handle_command_reply: tid=0 2024-08-30T09:03:12.355 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.351+0000 7fabd6dca740 1 client.25580 shutdown 2024-08-30T09:03:12.355 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.351+0000 7fabd6dca740 1 -- 172.21.15.42:0/3559287808 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55666e3b04d0 msgr2=0x55666e786470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:03:12.355 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.351+0000 7fabd6dca740 1 --2- 172.21.15.42:0/3559287808 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55666e3b04d0 0x55666e786470 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fabb8005750 tx=0x7fabb8005780 comp rx=0 tx=0).stop 2024-08-30T09:03:12.355 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.352+0000 7fabd6dca740 1 -- 172.21.15.42:0/3559287808 shutdown_connections 2024-08-30T09:03:12.355 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.352+0000 7fabd6dca740 1 --2- 172.21.15.42:0/3559287808 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55666e3651c0 0x55666e7fb080 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:12.355 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.352+0000 7fabd6dca740 1 --2- 172.21.15.42:0/3559287808 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55666e3d11b0 0x55666e786f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:12.355 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.352+0000 7fabb17fa640 0 client.25580 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:03:12.356 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.352+0000 7fabd6dca740 1 --2- 172.21.15.42:0/3559287808 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55666e3b04d0 0x55666e786470 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:12.356 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.352+0000 7fabd6dca740 1 --2- 172.21.15.42:0/3559287808 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55666e38fd80 0x55666e785db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:12.356 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.352+0000 7fabd6dca740 1 -- 172.21.15.42:0/3559287808 >> 172.21.15.42:0/3559287808 conn(0x55666e3c5300 msgr2=0x55666e78e620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:03:12.356 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.352+0000 7fabd6dca740 1 -- 172.21.15.42:0/3559287808 shutdown_connections 2024-08-30T09:03:12.356 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.352+0000 7fabd6dca740 1 -- 172.21.15.42:0/3559287808 wait complete. 2024-08-30T09:03:12.356 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.352+0000 7fabd6dca740 20 client.25580 trim_cache size 0 max 16384 2024-08-30T09:03:12.356 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-08-30T09:03:12.356 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2024-08-30T09:03:12.356 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-08-30T09:03:12.357 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-08-30T09:03:12.357 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.353+0000 7fabcfe5a640 1 -- 172.21.15.42:0/1127866408 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fabb40b4b30 msgr2=0x7fabb40b7040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:03:12.357 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.353+0000 7fabcfe5a640 1 --2- 172.21.15.42:0/1127866408 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fabb40b4b30 0x7fabb40b7040 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fabc4011630 tx=0x7fabc400b390 comp rx=0 tx=0).stop 2024-08-30T09:03:12.357 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.353+0000 7fabcfe5a640 1 -- 172.21.15.42:0/1127866408 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fabc8103d60 msgr2=0x7fabc8112cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:03:12.358 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.353+0000 7fabcfe5a640 1 --2- 172.21.15.42:0/1127866408 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fabc8103d60 0x7fabc8112cd0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fabc00011e0 tx=0x7fabc0031440 comp rx=0 tx=0).stop 2024-08-30T09:03:12.358 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.354+0000 7fabcfe5a640 1 -- 172.21.15.42:0/1127866408 shutdown_connections 2024-08-30T09:03:12.358 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.354+0000 7fabcfe5a640 1 --2- 172.21.15.42:0/1127866408 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fabb40b4b30 0x7fabb40b7040 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:12.358 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.354+0000 7fabcfe5a640 1 --2- 172.21.15.42:0/1127866408 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fabc8103d60 0x7fabc8112cd0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:12.358 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.354+0000 7fabcfe5a640 1 --2- 172.21.15.42:0/1127866408 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fabc81033c0 0x7fabc8112160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:12.358 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.354+0000 7fabcfe5a640 1 --2- 172.21.15.42:0/1127866408 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fabc8102340 0x7fabc8111af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:12.358 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.354+0000 7fabcfe5a640 1 -- 172.21.15.42:0/1127866408 >> 172.21.15.42:0/1127866408 conn(0x7fabc80fdc80 msgr2=0x7fabc81066c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:03:12.358 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.354+0000 7fabcfe5a640 1 -- 172.21.15.42:0/1127866408 shutdown_connections 2024-08-30T09:03:12.358 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:12.354+0000 7fabcfe5a640 1 -- 172.21.15.42:0/1127866408 wait complete. 2024-08-30T09:03:12.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:03:12 smithi190 ceph-mon[22001]: pgmap v446: 129 pgs: 129 active+clean; 1.1 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 MiB/s wr, 797 op/s 2024-08-30T09:03:12.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:03:12 smithi167 ceph-mon[22063]: pgmap v446: 129 pgs: 129 active+clean; 1.1 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 MiB/s wr, 797 op/s 2024-08-30T09:03:12.632 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:03:12 smithi042 ceph-mon[18652]: pgmap v446: 129 pgs: 129 active+clean; 1.1 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 MiB/s wr, 797 op/s 2024-08-30T09:03:13.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.383+0000 7f40c6320640 1 -- 172.21.15.42:0/8894978 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f40c0108160 msgr2=0x7f40c0108570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:03:13.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.383+0000 7f40c6320640 1 --2- 172.21.15.42:0/8894978 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f40c0108160 0x7f40c0108570 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f40a8009900 tx=0x7f40a802a410 comp rx=0 tx=0).stop 2024-08-30T09:03:13.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.383+0000 7f40c6320640 1 -- 172.21.15.42:0/8894978 shutdown_connections 2024-08-30T09:03:13.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.383+0000 7f40c6320640 1 --2- 172.21.15.42:0/8894978 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f40c0108ab0 0x7f40c010f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:13.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.383+0000 7f40c6320640 1 --2- 172.21.15.42:0/8894978 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f40c0108160 0x7f40c0108570 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:13.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.383+0000 7f40c6320640 1 --2- 172.21.15.42:0/8894978 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f40c0102820 0x7f40c0102c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:13.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.383+0000 7f40c6320640 1 -- 172.21.15.42:0/8894978 >> 172.21.15.42:0/8894978 conn(0x7f40c00794e0 msgr2=0x7f40c00798f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:03:13.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.383+0000 7f40c6320640 1 -- 172.21.15.42:0/8894978 shutdown_connections 2024-08-30T09:03:13.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.383+0000 7f40c6320640 1 -- 172.21.15.42:0/8894978 wait complete. 2024-08-30T09:03:13.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.384+0000 7f40c6320640 1 Processor -- start 2024-08-30T09:03:13.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.384+0000 7f40c6320640 1 -- start start 2024-08-30T09:03:13.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.384+0000 7f40c6320640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f40c0102820 0x7f40c019b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:13.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.384+0000 7f40c6320640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f40c0108160 0x7f40c019b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:13.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.384+0000 7f40c6320640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f40c0108ab0 0x7f40c019bb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:13.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.384+0000 7f40c6320640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f40c01127b0 con 0x7f40c0108ab0 2024-08-30T09:03:13.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.384+0000 7f40c6320640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f40c0112630 con 0x7f40c0108160 2024-08-30T09:03:13.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.384+0000 7f40c6320640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f40c0112930 con 0x7f40c0102820 2024-08-30T09:03:13.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.384+0000 7f40bffff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f40c0102820 0x7f40c019b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:03:13.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.384+0000 7f40b7fff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f40c0108160 0x7f40c019b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:03:13.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.384+0000 7f40bffff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f40c0102820 0x7f40c019b080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:41794/0 (socket says 172.21.15.42:41794) 2024-08-30T09:03:13.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.384+0000 7f40bffff640 1 -- 172.21.15.42:0/4068920041 learned_addr learned my addr 172.21.15.42:0/4068920041 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:03:13.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.384+0000 7f40c48bd640 1 --2- 172.21.15.42:0/4068920041 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f40c0108ab0 0x7f40c019bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:03:13.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.385+0000 7f40bffff640 1 -- 172.21.15.42:0/4068920041 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f40c0108160 msgr2=0x7f40c019b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:03:13.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.385+0000 7f40bffff640 1 --2- 172.21.15.42:0/4068920041 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f40c0108160 0x7f40c019b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:13.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.385+0000 7f40bffff640 1 -- 172.21.15.42:0/4068920041 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f40c0108ab0 msgr2=0x7f40c019bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:03:13.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.385+0000 7f40bffff640 1 --2- 172.21.15.42:0/4068920041 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f40c0108ab0 0x7f40c019bb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:13.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.385+0000 7f40bffff640 1 -- 172.21.15.42:0/4068920041 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40c01a6340 con 0x7f40c0102820 2024-08-30T09:03:13.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.385+0000 7f40b7fff640 1 --2- 172.21.15.42:0/4068920041 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f40c0108160 0x7f40c019b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-30T09:03:13.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.385+0000 7f40bffff640 1 --2- 172.21.15.42:0/4068920041 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f40c0102820 0x7f40c019b080 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f40b000d950 tx=0x7f40b000de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:03:13.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.385+0000 7f40bdffb640 1 -- 172.21.15.42:0/4068920041 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40b0017070 con 0x7f40c0102820 2024-08-30T09:03:13.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.385+0000 7f40c6320640 1 -- 172.21.15.42:0/4068920041 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40c01a6630 con 0x7f40c0102820 2024-08-30T09:03:13.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.385+0000 7f40bdffb640 1 -- 172.21.15.42:0/4068920041 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f40b0009c30 con 0x7f40c0102820 2024-08-30T09:03:13.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.385+0000 7f40bdffb640 1 -- 172.21.15.42:0/4068920041 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40b000f620 con 0x7f40c0102820 2024-08-30T09:03:13.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.385+0000 7f40c6320640 1 -- 172.21.15.42:0/4068920041 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40c01a6c10 con 0x7f40c0102820 2024-08-30T09:03:13.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.388+0000 7f40bdffb640 1 -- 172.21.15.42:0/4068920041 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7f40b000f7c0 con 0x7f40c0102820 2024-08-30T09:03:13.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.389+0000 7f40cc895740 1 Processor -- start 2024-08-30T09:03:13.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.389+0000 7f40cc895740 1 -- start start 2024-08-30T09:03:13.394 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.389+0000 7f40cc895740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5627dc27e070 0x5627dc623310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:13.394 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.389+0000 7f40cc895740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5627dc2551b0 0x5627dc623850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:13.394 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.389+0000 7f40cc895740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5627dc25d390 0x5627dc623d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:13.394 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.390+0000 7f40cc895740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x5627dc59a680 con 0x5627dc25d390 2024-08-30T09:03:13.394 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.390+0000 7f40cc895740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5627dc589c10 con 0x5627dc2551b0 2024-08-30T09:03:13.394 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.390+0000 7f40cc895740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5627dc58dbc0 con 0x5627dc27e070 2024-08-30T09:03:13.394 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.390+0000 7f40bffff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5627dc2551b0 0x5627dc623850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:03:13.394 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.390+0000 7f40bffff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5627dc2551b0 0x5627dc623850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:36482/0 (socket says 172.21.15.42:36482) 2024-08-30T09:03:13.394 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.390+0000 7f40bffff640 1 -- 172.21.15.42:0/3405759284 learned_addr learned my addr 172.21.15.42:0/3405759284 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:03:13.394 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.390+0000 7f40b7fff640 1 --2- 172.21.15.42:0/3405759284 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5627dc25d390 0x5627dc623d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:03:13.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.390+0000 7f40c48bd640 1 --2- 172.21.15.42:0/3405759284 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5627dc27e070 0x5627dc623310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:03:13.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.390+0000 7f40bffff640 1 -- 172.21.15.42:0/3405759284 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5627dc27e070 msgr2=0x5627dc623310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:03:13.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.390+0000 7f40bffff640 1 --2- 172.21.15.42:0/3405759284 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5627dc27e070 0x5627dc623310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:13.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.390+0000 7f40bffff640 1 -- 172.21.15.42:0/3405759284 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5627dc25d390 msgr2=0x5627dc623d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:03:13.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.390+0000 7f40bffff640 1 --2- 172.21.15.42:0/3405759284 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5627dc25d390 0x5627dc623d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:13.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.390+0000 7f40bffff640 1 -- 172.21.15.42:0/3405759284 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5627dc2e9650 con 0x5627dc2551b0 2024-08-30T09:03:13.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.391+0000 7f40bffff640 1 --2- 172.21.15.42:0/3405759284 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5627dc2551b0 0x5627dc623850 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f40b00ed530 tx=0x7f40b00ed620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f84d4c235451fd98 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:03:13.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.391+0000 7f40bdffb640 1 --2- 172.21.15.42:0/4068920041 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f40ac0b4bb0 0x7f40ac0b70c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:13.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.391+0000 7f40b67fc640 1 -- 172.21.15.42:0/3405759284 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40b00a8390 con 0x5627dc2551b0 2024-08-30T09:03:13.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.391+0000 7f40b67fc640 1 -- 172.21.15.42:0/3405759284 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f40b00ede50 con 0x5627dc2551b0 2024-08-30T09:03:13.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.391+0000 7f40b67fc640 1 -- 172.21.15.42:0/3405759284 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40b00ab4b0 con 0x5627dc2551b0 2024-08-30T09:03:13.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.391+0000 7f40c48bd640 1 --2- 172.21.15.42:0/4068920041 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f40ac0b4bb0 0x7f40ac0b70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:03:13.396 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.391+0000 7f40bdffb640 1 -- 172.21.15.42:0/4068920041 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f40b00e53b0 con 0x7f40c0102820 2024-08-30T09:03:13.397 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.392+0000 7f40c48bd640 1 --2- 172.21.15.42:0/4068920041 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f40ac0b4bb0 0x7f40ac0b70c0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f40b800a510 tx=0x7f40b800d040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:03:13.397 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.392+0000 7f40cc895740 1 -- 172.21.15.42:0/3405759284 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5627dc2551b0 msgr2=0x5627dc623850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:03:13.397 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.392+0000 7f40cc895740 1 --2- 172.21.15.42:0/3405759284 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5627dc2551b0 0x5627dc623850 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f40b00ed530 tx=0x7f40b00ed620 comp rx=0 tx=0).stop 2024-08-30T09:03:13.397 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.392+0000 7f40cc895740 1 -- 172.21.15.42:0/3405759284 shutdown_connections 2024-08-30T09:03:13.397 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.392+0000 7f40cc895740 1 --2- 172.21.15.42:0/3405759284 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5627dc25d390 0x5627dc623d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:13.397 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.392+0000 7f40cc895740 1 --2- 172.21.15.42:0/3405759284 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5627dc2551b0 0x5627dc623850 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:13.397 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.392+0000 7f40cc895740 1 --2- 172.21.15.42:0/3405759284 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5627dc27e070 0x5627dc623310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:13.397 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.392+0000 7f40cc895740 1 -- 172.21.15.42:0/3405759284 >> 172.21.15.42:0/3405759284 conn(0x5627dc272460 msgr2=0x5627dc621200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:03:13.397 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.392+0000 7f40cc895740 1 -- 172.21.15.42:0/3405759284 shutdown_connections 2024-08-30T09:03:13.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.392+0000 7f40cc895740 1 -- 172.21.15.42:0/3405759284 wait complete. 2024-08-30T09:03:13.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.393+0000 7f40cc895740 1 -- start start 2024-08-30T09:03:13.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.393+0000 7f40cc895740 1 Processor -- start 2024-08-30T09:03:13.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.394+0000 7f40cc895740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5627dc2551b0 0x5627dc62e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:13.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.394+0000 7f40cc895740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5627dc25d390 0x5627dc62ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:13.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.395+0000 7f40cc895740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5627dc27e070 0x5627dc63ade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:13.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.395+0000 7f40b7fff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5627dc25d390 0x5627dc62ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:03:13.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.395+0000 7f40b7fff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5627dc25d390 0x5627dc62ef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:36486/0 (socket says 172.21.15.42:36486) 2024-08-30T09:03:13.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.395+0000 7f40b7fff640 1 -- 172.21.15.42:0/3069238919 learned_addr learned my addr 172.21.15.42:0/3069238919 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:03:13.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.395+0000 7f40cc895740 1 -- 172.21.15.42:0/3069238919 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x5627dc3ec700 con 0x5627dc27e070 2024-08-30T09:03:13.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.395+0000 7f40cc895740 1 -- 172.21.15.42:0/3069238919 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5627dc510750 con 0x5627dc25d390 2024-08-30T09:03:13.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.395+0000 7f40cc895740 1 -- 172.21.15.42:0/3069238919 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5627dc631af0 con 0x5627dc2551b0 2024-08-30T09:03:13.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.395+0000 7f40c48bd640 1 --2- 172.21.15.42:0/3069238919 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5627dc27e070 0x5627dc63ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:03:13.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.396+0000 7f40bffff640 1 --2- 172.21.15.42:0/3069238919 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5627dc2551b0 0x5627dc62e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:03:13.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.396+0000 7f40b7fff640 1 -- 172.21.15.42:0/3069238919 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5627dc2551b0 msgr2=0x5627dc62e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:03:13.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.396+0000 7f40b7fff640 1 --2- 172.21.15.42:0/3069238919 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5627dc2551b0 0x5627dc62e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:13.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.396+0000 7f40b7fff640 1 -- 172.21.15.42:0/3069238919 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5627dc27e070 msgr2=0x5627dc63ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:03:13.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.396+0000 7f40b7fff640 1 --2- 172.21.15.42:0/3069238919 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5627dc27e070 0x5627dc63ade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:13.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.396+0000 7f40b7fff640 1 -- 172.21.15.42:0/3069238919 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5627dc549900 con 0x5627dc25d390 2024-08-30T09:03:13.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.396+0000 7f40bffff640 1 --2- 172.21.15.42:0/3069238919 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5627dc2551b0 0x5627dc62e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-30T09:03:13.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.396+0000 7f40cc895740 10 client.0 _ll_register_callbacks cb 0x5627dc53fe10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:03:13.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.396+0000 7f40cc895740 1 -- 172.21.15.42:0/3069238919 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f40b00ede50 con 0x5627dc25d390 2024-08-30T09:03:13.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.397+0000 7f40b7fff640 1 --2- 172.21.15.42:0/3069238919 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5627dc25d390 0x5627dc62ef30 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f40a80096c0 tx=0x7f40a8005440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ec9cb7bc975656d4 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:03:13.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.397+0000 7f409f7fe640 10 client.25592 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-30T09:03:13.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.397+0000 7f409f7fe640 1 -- 172.21.15.42:0/3069238919 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40a8035040 con 0x5627dc25d390 2024-08-30T09:03:13.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.397+0000 7f409f7fe640 1 -- 172.21.15.42:0/3069238919 <== mon.2 v2:172.21.15.167:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f40a8004b80 con 0x5627dc25d390 2024-08-30T09:03:13.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.397+0000 7f409f7fe640 1 -- 172.21.15.42:0/3069238919 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f40a8004dc0 con 0x5627dc25d390 2024-08-30T09:03:13.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.397+0000 7f40cc895740 10 client.25592 fetch_fsmap learned FSMap version 29 2024-08-30T09:03:13.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.397+0000 7f40cc895740 1 -- 172.21.15.42:0/3069238919 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({fsmap=29}) -- 0x5627dc63d680 con 0x5627dc25d390 2024-08-30T09:03:13.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.397+0000 7f409f7fe640 1 -- 172.21.15.42:0/3069238919 <== mon.2 v2:172.21.15.167:3300/0 4 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f40a8044020 con 0x5627dc25d390 2024-08-30T09:03:13.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.397+0000 7f40cc895740 10 client.25592 fetch_fsmap finished waiting for FSMap version 29 2024-08-30T09:03:13.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.397+0000 7f40cc895740 10 client.25592 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-30T09:03:13.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.397+0000 7f40cc895740 20 client.25592 populate_metadata read hostname 'smithi042' 2024-08-30T09:03:13.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.398+0000 7f40cc895740 1 --2- 172.21.15.42:0/3069238919 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x5627dc23c9f0 0x5627dc640bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:13.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.398+0000 7f40cc895740 4 client.25592 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-30T09:03:13.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.398+0000 7f40cc895740 1 -- 172.21.15.42:0/3069238919 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5627dc59c5d0 con 0x5627dc23c9f0 2024-08-30T09:03:13.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.398+0000 7f40bffff640 1 --2- 172.21.15.42:0/3069238919 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x5627dc23c9f0 0x5627dc640bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:03:13.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.398+0000 7f40bffff640 1 --2- 172.21.15.42:0/3069238919 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x5627dc23c9f0 0x5627dc640bf0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b59e61ab8bc5c58f server_cookie=7fd2e5acde7fb65e in_seq=0 out_seq=0 2024-08-30T09:03:13.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.398+0000 7f409f7fe640 10 client.25592 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:03:13.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.399+0000 7f409f7fe640 1 -- 172.21.15.42:0/3069238919 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5627dc59c5d0 con 0x5627dc23c9f0 2024-08-30T09:03:13.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.399+0000 7f409f7fe640 10 client.25592 handle_command_reply: tid=0 2024-08-30T09:03:13.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.399+0000 7f40cc895740 1 client.25592 shutdown 2024-08-30T09:03:13.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.399+0000 7f40cc895740 1 -- 172.21.15.42:0/3069238919 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5627dc25d390 msgr2=0x5627dc62ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:03:13.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.399+0000 7f40cc895740 1 --2- 172.21.15.42:0/3069238919 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5627dc25d390 0x5627dc62ef30 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f40a80096c0 tx=0x7f40a8005440 comp rx=0 tx=0).stop 2024-08-30T09:03:13.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.399+0000 7f40cc895740 1 -- 172.21.15.42:0/3069238919 shutdown_connections 2024-08-30T09:03:13.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.399+0000 7f40cc895740 1 --2- 172.21.15.42:0/3069238919 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x5627dc23c9f0 0x5627dc640bf0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:13.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.399+0000 7f40cc895740 1 --2- 172.21.15.42:0/3069238919 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5627dc27e070 0x5627dc63ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:13.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.399+0000 7f40cc895740 1 --2- 172.21.15.42:0/3069238919 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5627dc25d390 0x5627dc62ef30 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:13.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.399+0000 7f409f7fe640 0 client.25592 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:03:13.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.399+0000 7f40cc895740 1 --2- 172.21.15.42:0/3069238919 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5627dc2551b0 0x5627dc62e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:13.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.399+0000 7f40cc895740 1 -- 172.21.15.42:0/3069238919 >> 172.21.15.42:0/3069238919 conn(0x5627dc272460 msgr2=0x5627dc6264a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:03:13.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.399+0000 7f40cc895740 1 -- 172.21.15.42:0/3069238919 shutdown_connections 2024-08-30T09:03:13.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.399+0000 7f40cc895740 1 -- 172.21.15.42:0/3069238919 wait complete. 2024-08-30T09:03:13.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.399+0000 7f40cc895740 20 client.25592 trim_cache size 0 max 16384 2024-08-30T09:03:13.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.414+0000 7f40cc895740 1 Processor -- start 2024-08-30T09:03:13.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.414+0000 7f40cc895740 1 -- start start 2024-08-30T09:03:13.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.414+0000 7f40cc895740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5627dc2551b0 0x5627dc6308d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:13.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.414+0000 7f40cc895740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5627dc25d390 0x5627dc624a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:13.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.414+0000 7f40cc895740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5627dc27e070 0x5627dc63ace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:13.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.414+0000 7f40cc895740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f40c01b0fa0 con 0x5627dc27e070 2024-08-30T09:03:13.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.414+0000 7f40cc895740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f40c01ae870 con 0x5627dc25d390 2024-08-30T09:03:13.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.414+0000 7f40cc895740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f40c01ac6c0 con 0x5627dc2551b0 2024-08-30T09:03:13.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.414+0000 7f40bffff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5627dc2551b0 0x5627dc6308d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:03:13.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.414+0000 7f40b7fff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5627dc25d390 0x5627dc624a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:03:13.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.414+0000 7f40bffff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5627dc2551b0 0x5627dc6308d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:41822/0 (socket says 172.21.15.42:41822) 2024-08-30T09:03:13.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.414+0000 7f40bffff640 1 -- 172.21.15.42:0/38041654 learned_addr learned my addr 172.21.15.42:0/38041654 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:03:13.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.414+0000 7f40c48bd640 1 --2- 172.21.15.42:0/38041654 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5627dc27e070 0x5627dc63ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:03:13.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.415+0000 7f40b7fff640 1 -- 172.21.15.42:0/38041654 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5627dc2551b0 msgr2=0x5627dc6308d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:03:13.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.415+0000 7f40b7fff640 1 --2- 172.21.15.42:0/38041654 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5627dc2551b0 0x5627dc6308d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:13.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.415+0000 7f40b7fff640 1 -- 172.21.15.42:0/38041654 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5627dc27e070 msgr2=0x5627dc63ace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:03:13.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.415+0000 7f40b7fff640 1 --2- 172.21.15.42:0/38041654 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5627dc27e070 0x5627dc63ace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:13.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.415+0000 7f40b7fff640 1 -- 172.21.15.42:0/38041654 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5627dc629620 con 0x5627dc25d390 2024-08-30T09:03:13.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.415+0000 7f40b7fff640 1 --2- 172.21.15.42:0/38041654 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5627dc25d390 0x5627dc624a50 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f40a8009900 tx=0x7f40a8040000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8945b7835b11aedd server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:03:13.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.415+0000 7f40b5ffb640 1 -- 172.21.15.42:0/38041654 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40a8035020 con 0x5627dc25d390 2024-08-30T09:03:13.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.415+0000 7f40b5ffb640 1 -- 172.21.15.42:0/38041654 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f40b00ede50 con 0x5627dc25d390 2024-08-30T09:03:13.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.415+0000 7f40b5ffb640 1 -- 172.21.15.42:0/38041654 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40a8000e60 con 0x5627dc25d390 2024-08-30T09:03:13.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.416+0000 7f40cc895740 1 -- 172.21.15.42:0/38041654 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5627dc25d390 msgr2=0x5627dc624a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:03:13.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.416+0000 7f40cc895740 1 --2- 172.21.15.42:0/38041654 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5627dc25d390 0x5627dc624a50 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f40a8009900 tx=0x7f40a8040000 comp rx=0 tx=0).stop 2024-08-30T09:03:13.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.416+0000 7f40cc895740 1 -- 172.21.15.42:0/38041654 shutdown_connections 2024-08-30T09:03:13.421 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.416+0000 7f40cc895740 1 --2- 172.21.15.42:0/38041654 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5627dc27e070 0x5627dc63ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:13.421 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.416+0000 7f40cc895740 1 --2- 172.21.15.42:0/38041654 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5627dc25d390 0x5627dc624a50 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:13.421 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.416+0000 7f40cc895740 1 --2- 172.21.15.42:0/38041654 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5627dc2551b0 0x5627dc6308d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:13.421 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.416+0000 7f40cc895740 1 -- 172.21.15.42:0/38041654 >> 172.21.15.42:0/38041654 conn(0x5627dc272460 msgr2=0x5627dc624340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:03:13.421 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.416+0000 7f40cc895740 1 -- 172.21.15.42:0/38041654 shutdown_connections 2024-08-30T09:03:13.421 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.416+0000 7f40cc895740 1 -- 172.21.15.42:0/38041654 wait complete. 2024-08-30T09:03:13.421 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.417+0000 7f40cc895740 1 -- start start 2024-08-30T09:03:13.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.419+0000 7f40cc895740 1 Processor -- start 2024-08-30T09:03:13.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.421+0000 7f40cc895740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5627dc2551b0 0x5627dc6354a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:13.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.421+0000 7f40cc895740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5627dc25d390 0x5627dc6359e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:13.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.421+0000 7f40cc895740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5627dc27e070 0x5627dc6aa610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:13.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.421+0000 7f40bffff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5627dc2551b0 0x5627dc6354a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:03:13.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.421+0000 7f40bffff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5627dc2551b0 0x5627dc6354a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:58812/0 (socket says 172.21.15.42:58812) 2024-08-30T09:03:13.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.421+0000 7f40bffff640 1 -- 172.21.15.42:0/3637558229 learned_addr learned my addr 172.21.15.42:0/3637558229 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:03:13.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.421+0000 7f40cc895740 1 -- 172.21.15.42:0/3637558229 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f40c01acd30 con 0x5627dc2551b0 2024-08-30T09:03:13.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.421+0000 7f40cc895740 1 -- 172.21.15.42:0/3637558229 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f40c01ae1a0 con 0x5627dc25d390 2024-08-30T09:03:13.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.421+0000 7f40cc895740 1 -- 172.21.15.42:0/3637558229 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f40c01ad400 con 0x5627dc27e070 2024-08-30T09:03:13.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.421+0000 7f40b7fff640 1 --2- 172.21.15.42:0/3637558229 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5627dc25d390 0x5627dc6359e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:03:13.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.421+0000 7f40cc895740 10 client.0 _ll_register_callbacks cb 0x7f40a80043f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:03:13.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.421+0000 7f40b7fff640 1 -- 172.21.15.42:0/3637558229 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5627dc27e070 msgr2=0x5627dc6aa610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:03:13.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.421+0000 7f40b7fff640 1 --2- 172.21.15.42:0/3637558229 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5627dc27e070 0x5627dc6aa610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:13.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.421+0000 7f40b7fff640 1 -- 172.21.15.42:0/3637558229 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5627dc2551b0 msgr2=0x5627dc6354a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:03:13.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.421+0000 7f40b7fff640 1 --2- 172.21.15.42:0/3637558229 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5627dc2551b0 0x5627dc6354a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:13.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.421+0000 7f40b7fff640 1 -- 172.21.15.42:0/3637558229 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5627dc6aab80 con 0x5627dc25d390 2024-08-30T09:03:13.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.422+0000 7f40b7fff640 1 --2- 172.21.15.42:0/3637558229 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5627dc25d390 0x5627dc6359e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f40a8000c00 tx=0x7f40a8003870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4bf8f180d98c26d1 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:03:13.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.422+0000 7f409f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-30T09:03:13.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.422+0000 7f409f7fe640 1 -- 172.21.15.42:0/3637558229 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40a8035030 con 0x5627dc25d390 2024-08-30T09:03:13.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.422+0000 7f40cc895740 1 -- 172.21.15.42:0/3637558229 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f40b00ede50 con 0x5627dc25d390 2024-08-30T09:03:13.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.422+0000 7f409f7fe640 1 -- 172.21.15.42:0/3637558229 <== mon.2 v2:172.21.15.167:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f40a805ca00 con 0x5627dc25d390 2024-08-30T09:03:13.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.422+0000 7f409f7fe640 1 -- 172.21.15.42:0/3637558229 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f40a8003e80 con 0x5627dc25d390 2024-08-30T09:03:13.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.422+0000 7f40cc895740 10 client.25598 fetch_fsmap learned FSMap version 29 2024-08-30T09:03:13.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.422+0000 7f40cc895740 1 -- 172.21.15.42:0/3637558229 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5627dc6abe00 con 0x5627dc25d390 2024-08-30T09:03:13.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.422+0000 7f409f7fe640 1 -- 172.21.15.42:0/3637558229 <== mon.2 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f40b00ede50 con 0x5627dc25d390 2024-08-30T09:03:13.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.423+0000 7f409f7fe640 1 -- 172.21.15.42:0/3637558229 <== mon.2 v2:172.21.15.167:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f40a8044040 con 0x5627dc25d390 2024-08-30T09:03:13.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.423+0000 7f40cc895740 10 client.25598 fetch_fsmap finished waiting for FSMap version 29 2024-08-30T09:03:13.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.423+0000 7f40cc895740 10 client.25598 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-30T09:03:13.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.423+0000 7f40cc895740 20 client.25598 populate_metadata read hostname 'smithi042' 2024-08-30T09:03:13.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.423+0000 7f40cc895740 1 --2- 172.21.15.42:0/3637558229 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x5627dc211da0 0x5627dc6ae5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:13.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.423+0000 7f40cc895740 4 client.25598 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-30T09:03:13.430 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.423+0000 7f40cc895740 1 -- 172.21.15.42:0/3637558229 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5627dc598070 con 0x5627dc211da0 2024-08-30T09:03:13.430 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.423+0000 7f40bffff640 1 --2- 172.21.15.42:0/3637558229 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x5627dc211da0 0x5627dc6ae5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:03:13.430 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.423+0000 7f409f7fe640 10 client.25598 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:03:13.430 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.423+0000 7f40bffff640 1 --2- 172.21.15.42:0/3637558229 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x5627dc211da0 0x5627dc6ae5e0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4658c0cbc0512c14 server_cookie=ed20b42147a4b9e8 in_seq=0 out_seq=0 2024-08-30T09:03:13.430 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.423+0000 7f409f7fe640 1 -- 172.21.15.42:0/3637558229 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5627dc598070 con 0x5627dc211da0 2024-08-30T09:03:13.430 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.423+0000 7f409f7fe640 10 client.25598 handle_command_reply: tid=0 2024-08-30T09:03:13.430 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.424+0000 7f40cc895740 1 client.25598 shutdown 2024-08-30T09:03:13.430 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.424+0000 7f40cc895740 1 -- 172.21.15.42:0/3637558229 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5627dc25d390 msgr2=0x5627dc6359e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:03:13.430 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.424+0000 7f40cc895740 1 --2- 172.21.15.42:0/3637558229 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5627dc25d390 0x5627dc6359e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f40a8000c00 tx=0x7f40a8003870 comp rx=0 tx=0).stop 2024-08-30T09:03:13.430 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.424+0000 7f40cc895740 1 -- 172.21.15.42:0/3637558229 shutdown_connections 2024-08-30T09:03:13.430 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.424+0000 7f40cc895740 1 --2- 172.21.15.42:0/3637558229 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x5627dc211da0 0x5627dc6ae5e0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:13.430 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.424+0000 7f40cc895740 1 --2- 172.21.15.42:0/3637558229 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5627dc27e070 0x5627dc6aa610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:13.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.424+0000 7f40cc895740 1 --2- 172.21.15.42:0/3637558229 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5627dc25d390 0x5627dc6359e0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:13.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.424+0000 7f40cc895740 1 --2- 172.21.15.42:0/3637558229 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5627dc2551b0 0x5627dc6354a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:13.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.424+0000 7f40cc895740 1 -- 172.21.15.42:0/3637558229 >> 172.21.15.42:0/3637558229 conn(0x5627dc272460 msgr2=0x5627dc624d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:03:13.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.424+0000 7f409f7fe640 0 client.25598 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:03:13.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.424+0000 7f40cc895740 1 -- 172.21.15.42:0/3637558229 shutdown_connections 2024-08-30T09:03:13.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.424+0000 7f40cc895740 1 -- 172.21.15.42:0/3637558229 wait complete. 2024-08-30T09:03:13.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.424+0000 7f40cc895740 20 client.25598 trim_cache size 0 max 16384 2024-08-30T09:03:13.431 INFO:teuthology.orchestra.run.smithi042.stdout:[] 2024-08-30T09:03:13.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.425+0000 7f40c6320640 1 -- 172.21.15.42:0/4068920041 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f40ac0b4bb0 msgr2=0x7f40ac0b70c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:03:13.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.425+0000 7f40c6320640 1 --2- 172.21.15.42:0/4068920041 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f40ac0b4bb0 0x7f40ac0b70c0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f40b800a510 tx=0x7f40b800d040 comp rx=0 tx=0).stop 2024-08-30T09:03:13.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.425+0000 7f40c6320640 1 -- 172.21.15.42:0/4068920041 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f40c0102820 msgr2=0x7f40c019b080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:03:13.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.425+0000 7f40c6320640 1 --2- 172.21.15.42:0/4068920041 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f40c0102820 0x7f40c019b080 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f40b000d950 tx=0x7f40b000de10 comp rx=0 tx=0).stop 2024-08-30T09:03:13.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.425+0000 7f40c6320640 1 -- 172.21.15.42:0/4068920041 shutdown_connections 2024-08-30T09:03:13.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.425+0000 7f40c6320640 1 --2- 172.21.15.42:0/4068920041 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f40ac0b4bb0 0x7f40ac0b70c0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:13.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.425+0000 7f40c6320640 1 --2- 172.21.15.42:0/4068920041 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f40c0108ab0 0x7f40c019bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:13.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.425+0000 7f40c6320640 1 --2- 172.21.15.42:0/4068920041 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f40c0108160 0x7f40c019b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:13.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.425+0000 7f40c6320640 1 --2- 172.21.15.42:0/4068920041 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f40c0102820 0x7f40c019b080 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:13.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.425+0000 7f40c6320640 1 -- 172.21.15.42:0/4068920041 >> 172.21.15.42:0/4068920041 conn(0x7f40c00794e0 msgr2=0x7f40c0104170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:03:13.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.425+0000 7f40c6320640 1 -- 172.21.15.42:0/4068920041 shutdown_connections 2024-08-30T09:03:13.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:13.425+0000 7f40c6320640 1 -- 172.21.15.42:0/4068920041 wait complete. 2024-08-30T09:03:14.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:03:14 smithi190 ceph-mon[22001]: pgmap v447: 129 pgs: 129 active+clean; 1.1 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 MiB/s wr, 972 op/s 2024-08-30T09:03:14.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:03:14 smithi167 ceph-mon[22063]: pgmap v447: 129 pgs: 129 active+clean; 1.1 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 MiB/s wr, 972 op/s 2024-08-30T09:03:14.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:03:14 smithi042 ceph-mon[18652]: pgmap v447: 129 pgs: 129 active+clean; 1.1 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 MiB/s wr, 972 op/s 2024-08-30T09:03:15.283 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-08-30T09:03:15.283 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-08-30T09:03:15.688 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T09:03:15.953 INFO:teuthology.orchestra.run.smithi042.stderr:progress: 150.0 s, 1316.1 tps, lat 76.269 ms stddev 86.318 2024-08-30T09:03:16.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.329+0000 7efcdffff640 1 -- 172.21.15.42:0/3591566517 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcd0030c00 con 0x7efce0102800 2024-08-30T09:03:16.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.330+0000 7efce759b640 1 -- 172.21.15.42:0/3591566517 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efce0102800 msgr2=0x7efce0102c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:03:16.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.330+0000 7efce759b640 1 --2- 172.21.15.42:0/3591566517 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efce0102800 0x7efce0102c10 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7efcd0009ca0 tx=0x7efcd002a8b0 comp rx=0 tx=0).stop 2024-08-30T09:03:16.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.330+0000 7efce759b640 1 -- 172.21.15.42:0/3591566517 shutdown_connections 2024-08-30T09:03:16.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.330+0000 7efce759b640 1 --2- 172.21.15.42:0/3591566517 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efce0108140 0x7efce0108550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:16.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.330+0000 7efce759b640 1 --2- 172.21.15.42:0/3591566517 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efce0108a90 0x7efce010f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:16.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.330+0000 7efce759b640 1 --2- 172.21.15.42:0/3591566517 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efce0102800 0x7efce0102c10 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:16.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.330+0000 7efce759b640 1 -- 172.21.15.42:0/3591566517 >> 172.21.15.42:0/3591566517 conn(0x7efce00794d0 msgr2=0x7efce00798e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:03:16.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.330+0000 7efce759b640 1 -- 172.21.15.42:0/3591566517 shutdown_connections 2024-08-30T09:03:16.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.330+0000 7efce759b640 1 -- 172.21.15.42:0/3591566517 wait complete. 2024-08-30T09:03:16.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.331+0000 7efce759b640 1 Processor -- start 2024-08-30T09:03:16.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.331+0000 7efce759b640 1 -- start start 2024-08-30T09:03:16.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.332+0000 7efce759b640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efce0102800 0x7efce0198000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:16.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.332+0000 7efce759b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efce0108140 0x7efce01985a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:16.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.332+0000 7efce759b640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efce0108a90 0x7efce019d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:16.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.332+0000 7efce759b640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7efce0111660 con 0x7efce0108a90 2024-08-30T09:03:16.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.332+0000 7efce759b640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7efce01114e0 con 0x7efce0102800 2024-08-30T09:03:16.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.332+0000 7efce759b640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7efce01117e0 con 0x7efce0108140 2024-08-30T09:03:16.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.332+0000 7efce5337640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efce0102800 0x7efce0198000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:03:16.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.332+0000 7efce4b36640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efce0108140 0x7efce01985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:03:16.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.332+0000 7efce5337640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efce0102800 0x7efce0198000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:36530/0 (socket says 172.21.15.42:36530) 2024-08-30T09:03:16.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.332+0000 7efce5337640 1 -- 172.21.15.42:0/467798627 learned_addr learned my addr 172.21.15.42:0/467798627 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:03:16.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.332+0000 7efce5b38640 1 --2- 172.21.15.42:0/467798627 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efce0108a90 0x7efce019d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:03:16.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.332+0000 7efce4b36640 1 -- 172.21.15.42:0/467798627 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efce0102800 msgr2=0x7efce0198000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:03:16.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.332+0000 7efce4b36640 1 --2- 172.21.15.42:0/467798627 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efce0102800 0x7efce0198000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:16.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.332+0000 7efce4b36640 1 -- 172.21.15.42:0/467798627 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efce0108a90 msgr2=0x7efce019d840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:03:16.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.332+0000 7efce4b36640 1 --2- 172.21.15.42:0/467798627 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efce0108a90 0x7efce019d840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:16.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.332+0000 7efce4b36640 1 -- 172.21.15.42:0/467798627 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efce019df70 con 0x7efce0108140 2024-08-30T09:03:16.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.332+0000 7efce5337640 1 --2- 172.21.15.42:0/467798627 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efce0102800 0x7efce0198000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-30T09:03:16.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.333+0000 7efce4b36640 1 --2- 172.21.15.42:0/467798627 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efce0108140 0x7efce01985a0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7efcd400a560 tx=0x7efcd400a960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:03:16.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.333+0000 7efce5b38640 1 --2- 172.21.15.42:0/467798627 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efce0108a90 0x7efce019d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-30T09:03:16.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.333+0000 7efcde7fc640 1 -- 172.21.15.42:0/467798627 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcd4024070 con 0x7efce0108140 2024-08-30T09:03:16.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.333+0000 7efcde7fc640 1 -- 172.21.15.42:0/467798627 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efcd400cb00 con 0x7efce0108140 2024-08-30T09:03:16.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.333+0000 7efcde7fc640 1 -- 172.21.15.42:0/467798627 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcd401fbf0 con 0x7efce0108140 2024-08-30T09:03:16.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.333+0000 7efce759b640 1 -- 172.21.15.42:0/467798627 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efce019e260 con 0x7efce0108140 2024-08-30T09:03:16.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.333+0000 7efce759b640 1 -- 172.21.15.42:0/467798627 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efce01a49c0 con 0x7efce0108140 2024-08-30T09:03:16.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.337+0000 7efcedb10740 1 Processor -- start 2024-08-30T09:03:16.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.339+0000 7efcde7fc640 1 -- 172.21.15.42:0/467798627 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7efcd4027070 con 0x7efce0108140 2024-08-30T09:03:16.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.339+0000 7efcedb10740 1 -- start start 2024-08-30T09:03:16.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.339+0000 7efcedb10740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55783aba4350 0x55783af4a780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:16.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.340+0000 7efcedb10740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55783ab7b490 0x55783af4acc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:16.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.340+0000 7efcedb10740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55783ab83670 0x55783af4b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:16.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.340+0000 7efcedb10740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55783aec05f0 con 0x55783ab7b490 2024-08-30T09:03:16.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.340+0000 7efcedb10740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55783aeafb80 con 0x55783ab83670 2024-08-30T09:03:16.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.340+0000 7efcedb10740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55783aeb3b30 con 0x55783aba4350 2024-08-30T09:03:16.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.340+0000 7efce5b38640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55783aba4350 0x55783af4a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:03:16.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.340+0000 7efce5b38640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55783aba4350 0x55783af4a780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:41870/0 (socket says 172.21.15.42:41870) 2024-08-30T09:03:16.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.340+0000 7efce5b38640 1 -- 172.21.15.42:0/3172099114 learned_addr learned my addr 172.21.15.42:0/3172099114 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:03:16.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.340+0000 7efce5337640 1 --2- 172.21.15.42:0/3172099114 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55783ab7b490 0x55783af4acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:03:16.345 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.340+0000 7efce5b38640 1 -- 172.21.15.42:0/3172099114 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55783ab83670 msgr2=0x55783af4b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:03:16.345 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.340+0000 7efcde7fc640 1 --2- 172.21.15.42:0/467798627 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7efcc80b4bb0 0x7efcc80b70c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:16.345 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.340+0000 7efce4b36640 1 --2- 172.21.15.42:0/3172099114 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55783ab83670 0x55783af4b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:03:16.345 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.340+0000 7efce5b38640 1 --2- 172.21.15.42:0/3172099114 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55783ab83670 0x55783af4b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:16.345 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.340+0000 7efce5b38640 1 -- 172.21.15.42:0/3172099114 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55783ab7b490 msgr2=0x55783af4acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:03:16.345 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.340+0000 7efce5b38640 1 --2- 172.21.15.42:0/3172099114 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55783ab7b490 0x55783af4acc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:16.345 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.340+0000 7efce5b38640 1 -- 172.21.15.42:0/3172099114 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55783ac0f930 con 0x55783aba4350 2024-08-30T09:03:16.345 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.340+0000 7efce5b38640 1 --2- 172.21.15.42:0/467798627 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7efcc80b4bb0 0x7efcc80b70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:03:16.345 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.340+0000 7efcde7fc640 1 -- 172.21.15.42:0/467798627 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7efcd40ed860 con 0x7efce0108140 2024-08-30T09:03:16.345 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.340+0000 7efce4b36640 1 --2- 172.21.15.42:0/3172099114 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55783ab83670 0x55783af4b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-30T09:03:16.345 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.341+0000 7efce5b38640 1 --2- 172.21.15.42:0/3172099114 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55783aba4350 0x55783af4a780 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7efcd8009bb0 tx=0x7efcd8010740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f006e6bbea14aeb0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:03:16.345 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.341+0000 7efcc77fe640 1 -- 172.21.15.42:0/3172099114 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcd8022070 con 0x55783aba4350 2024-08-30T09:03:16.345 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.341+0000 7efcc77fe640 1 -- 172.21.15.42:0/3172099114 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efcd8012ba0 con 0x55783aba4350 2024-08-30T09:03:16.345 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.341+0000 7efcc77fe640 1 -- 172.21.15.42:0/3172099114 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcd801ce30 con 0x55783aba4350 2024-08-30T09:03:16.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.342+0000 7efcedb10740 1 -- 172.21.15.42:0/3172099114 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55783aba4350 msgr2=0x55783af4a780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:03:16.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.342+0000 7efcedb10740 1 --2- 172.21.15.42:0/3172099114 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55783aba4350 0x55783af4a780 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7efcd8009bb0 tx=0x7efcd8010740 comp rx=0 tx=0).stop 2024-08-30T09:03:16.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.342+0000 7efce5b38640 1 --2- 172.21.15.42:0/467798627 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7efcc80b4bb0 0x7efcc80b70c0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7efcd801f000 tx=0x7efcd80118b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:03:16.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.342+0000 7efcedb10740 1 -- 172.21.15.42:0/3172099114 shutdown_connections 2024-08-30T09:03:16.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.342+0000 7efcedb10740 1 --2- 172.21.15.42:0/3172099114 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55783ab83670 0x55783af4b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:16.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.342+0000 7efcedb10740 1 --2- 172.21.15.42:0/3172099114 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55783ab7b490 0x55783af4acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:16.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.342+0000 7efcedb10740 1 --2- 172.21.15.42:0/3172099114 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55783aba4350 0x55783af4a780 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:16.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.342+0000 7efcedb10740 1 -- 172.21.15.42:0/3172099114 >> 172.21.15.42:0/3172099114 conn(0x55783ab98740 msgr2=0x55783af48670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:03:16.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.342+0000 7efcedb10740 1 -- 172.21.15.42:0/3172099114 shutdown_connections 2024-08-30T09:03:16.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.342+0000 7efcedb10740 1 -- 172.21.15.42:0/3172099114 wait complete. 2024-08-30T09:03:16.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.342+0000 7efcedb10740 1 -- start start 2024-08-30T09:03:16.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.343+0000 7efcedb10740 1 Processor -- start 2024-08-30T09:03:16.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.343+0000 7efcedb10740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55783ab7b490 0x55783af55f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:16.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.343+0000 7efcedb10740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55783ab83670 0x55783af564d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:16.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.343+0000 7efcedb10740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55783aba4350 0x55783af62280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:16.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.343+0000 7efcedb10740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55783ad12890 con 0x55783aba4350 2024-08-30T09:03:16.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.343+0000 7efcedb10740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55783ae36790 con 0x55783ab83670 2024-08-30T09:03:16.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.343+0000 7efcedb10740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55783af58f60 con 0x55783ab7b490 2024-08-30T09:03:16.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.343+0000 7efcedb10740 10 client.0 _ll_register_callbacks cb 0x55783abbbba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:03:16.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.343+0000 7efce5337640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55783ab7b490 0x55783af55f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:03:16.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.343+0000 7efce5b38640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55783aba4350 0x55783af62280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:03:16.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.343+0000 7efce5337640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55783ab7b490 0x55783af55f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:41880/0 (socket says 172.21.15.42:41880) 2024-08-30T09:03:16.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.343+0000 7efce5337640 1 -- 172.21.15.42:0/3821043825 learned_addr learned my addr 172.21.15.42:0/3821043825 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:03:16.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.343+0000 7efce5b38640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55783aba4350 0x55783af62280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:58834/0 (socket says 172.21.15.42:58834) 2024-08-30T09:03:16.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.344+0000 7efce4b36640 1 --2- 172.21.15.42:0/3821043825 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55783ab83670 0x55783af564d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:03:16.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.344+0000 7efce5b38640 1 -- 172.21.15.42:0/3821043825 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55783ab7b490 msgr2=0x55783af55f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:03:16.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.344+0000 7efce5b38640 1 --2- 172.21.15.42:0/3821043825 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55783ab7b490 0x55783af55f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:16.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.344+0000 7efce5b38640 1 -- 172.21.15.42:0/3821043825 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55783ab83670 msgr2=0x55783af564d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:03:16.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.344+0000 7efce5b38640 1 --2- 172.21.15.42:0/3821043825 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55783ab83670 0x55783af564d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:16.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.344+0000 7efce5b38640 1 -- 172.21.15.42:0/3821043825 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55783ae6df30 con 0x55783aba4350 2024-08-30T09:03:16.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.344+0000 7efce4b36640 1 --2- 172.21.15.42:0/3821043825 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55783ab83670 0x55783af564d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-30T09:03:16.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.344+0000 7efce5b38640 1 --2- 172.21.15.42:0/3821043825 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55783aba4350 0x55783af62280 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7efcd80093b0 tx=0x7efcd8016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acfc6b372bd6e187 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:03:16.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.345+0000 7efcc4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-08-30T09:03:16.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.345+0000 7efcc4ff9640 1 -- 172.21.15.42:0/3821043825 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcd8036060 con 0x55783aba4350 2024-08-30T09:03:16.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.345+0000 7efcedb10740 1 -- 172.21.15.42:0/3821043825 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efcd8012ba0 con 0x55783aba4350 2024-08-30T09:03:16.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.345+0000 7efcc4ff9640 1 -- 172.21.15.42:0/3821043825 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7efcd801cab0 con 0x55783aba4350 2024-08-30T09:03:16.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.345+0000 7efcc4ff9640 1 -- 172.21.15.42:0/3821043825 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7efcd803d050 con 0x55783aba4350 2024-08-30T09:03:16.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.345+0000 7efcedb10740 10 client.15930 fetch_fsmap learned FSMap version 29 2024-08-30T09:03:16.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.346+0000 7efcedb10740 1 -- 172.21.15.42:0/3821043825 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55783af64d00 con 0x55783aba4350 2024-08-30T09:03:16.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.346+0000 7efcc4ff9640 1 -- 172.21.15.42:0/3821043825 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efcd8012900 con 0x55783aba4350 2024-08-30T09:03:16.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.346+0000 7efcc4ff9640 1 -- 172.21.15.42:0/3821043825 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7efcd800f040 con 0x55783aba4350 2024-08-30T09:03:16.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.346+0000 7efcedb10740 10 client.15930 fetch_fsmap finished waiting for FSMap version 29 2024-08-30T09:03:16.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.346+0000 7efcedb10740 10 client.15930 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-30T09:03:16.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.346+0000 7efcedb10740 20 client.15930 populate_metadata read hostname 'smithi042' 2024-08-30T09:03:16.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.346+0000 7efcedb10740 1 --2- 172.21.15.42:0/3821043825 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55783ab62cd0 0x55783af68170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:16.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.346+0000 7efcedb10740 4 client.15930 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-30T09:03:16.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.346+0000 7efcedb10740 1 -- 172.21.15.42:0/3821043825 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7efcd8012ba0 con 0x55783ab62cd0 2024-08-30T09:03:16.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.346+0000 7efce5337640 1 --2- 172.21.15.42:0/3821043825 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55783ab62cd0 0x55783af68170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:03:16.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.347+0000 7efce5337640 1 --2- 172.21.15.42:0/3821043825 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55783ab62cd0 0x55783af68170 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=925cfb15635ef664 server_cookie=2474deb757c0ae97 in_seq=0 out_seq=0 2024-08-30T09:03:16.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.347+0000 7efcc4ff9640 10 client.15930 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:03:16.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.348+0000 7efcc4ff9640 1 -- 172.21.15.42:0/3821043825 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7efcd8012ba0 con 0x55783ab62cd0 2024-08-30T09:03:16.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.348+0000 7efcc4ff9640 10 client.15930 handle_command_reply: tid=0 2024-08-30T09:03:16.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.348+0000 7efcedb10740 1 client.15930 shutdown 2024-08-30T09:03:16.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.348+0000 7efcedb10740 1 -- 172.21.15.42:0/3821043825 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55783aba4350 msgr2=0x55783af62280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:03:16.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.348+0000 7efcedb10740 1 --2- 172.21.15.42:0/3821043825 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55783aba4350 0x55783af62280 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7efcd80093b0 tx=0x7efcd8016000 comp rx=0 tx=0).stop 2024-08-30T09:03:16.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.348+0000 7efcedb10740 1 -- 172.21.15.42:0/3821043825 shutdown_connections 2024-08-30T09:03:16.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.348+0000 7efcedb10740 1 --2- 172.21.15.42:0/3821043825 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55783ab62cd0 0x55783af68170 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:16.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.348+0000 7efcedb10740 1 --2- 172.21.15.42:0/3821043825 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55783aba4350 0x55783af62280 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:16.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.348+0000 7efcedb10740 1 --2- 172.21.15.42:0/3821043825 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55783ab83670 0x55783af564d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:16.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.348+0000 7efcedb10740 1 --2- 172.21.15.42:0/3821043825 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55783ab7b490 0x55783af55f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:16.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.348+0000 7efcedb10740 1 -- 172.21.15.42:0/3821043825 >> 172.21.15.42:0/3821043825 conn(0x55783ab98740 msgr2=0x55783af4d910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:03:16.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.348+0000 7efcc4ff9640 0 client.15930 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:03:16.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.348+0000 7efcedb10740 1 -- 172.21.15.42:0/3821043825 shutdown_connections 2024-08-30T09:03:16.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.349+0000 7efcedb10740 1 -- 172.21.15.42:0/3821043825 wait complete. 2024-08-30T09:03:16.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.349+0000 7efcedb10740 20 client.15930 trim_cache size 0 max 16384 2024-08-30T09:03:16.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.377+0000 7efcedb10740 1 Processor -- start 2024-08-30T09:03:16.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.377+0000 7efcedb10740 1 -- start start 2024-08-30T09:03:16.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.377+0000 7efcedb10740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55783ab7b490 0x55783af4df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:16.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.377+0000 7efcedb10740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55783ab83670 0x55783af53180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:16.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.377+0000 7efcedb10740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55783aba4350 0x55783af536c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:16.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.377+0000 7efcedb10740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7efce01b1090 con 0x55783aba4350 2024-08-30T09:03:16.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.377+0000 7efcedb10740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7efce01ae960 con 0x55783ab7b490 2024-08-30T09:03:16.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.377+0000 7efcedb10740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7efce01ac7b0 con 0x55783ab83670 2024-08-30T09:03:16.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.377+0000 7efce5337640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55783ab7b490 0x55783af4df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:03:16.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.377+0000 7efce5337640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55783ab7b490 0x55783af4df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:36560/0 (socket says 172.21.15.42:36560) 2024-08-30T09:03:16.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.377+0000 7efce5337640 1 -- 172.21.15.42:0/903622477 learned_addr learned my addr 172.21.15.42:0/903622477 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:03:16.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.378+0000 7efce5b38640 1 --2- 172.21.15.42:0/903622477 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55783aba4350 0x55783af536c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:03:16.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.378+0000 7efce4b36640 1 --2- 172.21.15.42:0/903622477 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55783ab83670 0x55783af53180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:03:16.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.378+0000 7efce5337640 1 -- 172.21.15.42:0/903622477 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55783ab83670 msgr2=0x55783af53180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:03:16.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.378+0000 7efce5337640 1 --2- 172.21.15.42:0/903622477 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55783ab83670 0x55783af53180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:16.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.378+0000 7efce5337640 1 -- 172.21.15.42:0/903622477 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55783aba4350 msgr2=0x55783af536c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:03:16.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.378+0000 7efce5337640 1 --2- 172.21.15.42:0/903622477 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55783aba4350 0x55783af536c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:16.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.378+0000 7efce5337640 1 -- 172.21.15.42:0/903622477 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55783ab62f00 con 0x55783ab7b490 2024-08-30T09:03:16.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.378+0000 7efce5337640 1 --2- 172.21.15.42:0/903622477 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55783ab7b490 0x55783af4df90 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7efcd0006010 tx=0x7efcd00094a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=519a2fc17d057ba server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:03:16.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.379+0000 7efcc6ffd640 1 -- 172.21.15.42:0/903622477 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcd0030ac0 con 0x55783ab7b490 2024-08-30T09:03:16.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.379+0000 7efcc6ffd640 1 -- 172.21.15.42:0/903622477 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efcd0036040 con 0x55783ab7b490 2024-08-30T09:03:16.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.379+0000 7efcc6ffd640 1 -- 172.21.15.42:0/903622477 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcd0004770 con 0x55783ab7b490 2024-08-30T09:03:16.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.380+0000 7efcedb10740 1 -- 172.21.15.42:0/903622477 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55783ab7b490 msgr2=0x55783af4df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:03:16.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.380+0000 7efcedb10740 1 --2- 172.21.15.42:0/903622477 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55783ab7b490 0x55783af4df90 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7efcd0006010 tx=0x7efcd00094a0 comp rx=0 tx=0).stop 2024-08-30T09:03:16.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.381+0000 7efcedb10740 1 -- 172.21.15.42:0/903622477 shutdown_connections 2024-08-30T09:03:16.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.381+0000 7efcedb10740 1 --2- 172.21.15.42:0/903622477 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55783ab83670 0x55783af53180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:16.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.381+0000 7efcedb10740 1 --2- 172.21.15.42:0/903622477 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55783aba4350 0x55783af536c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:16.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.381+0000 7efcedb10740 1 --2- 172.21.15.42:0/903622477 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55783ab7b490 0x55783af4df90 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:16.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.381+0000 7efcedb10740 1 -- 172.21.15.42:0/903622477 >> 172.21.15.42:0/903622477 conn(0x55783ab98740 msgr2=0x55783af4d880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:03:16.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.381+0000 7efcedb10740 1 -- 172.21.15.42:0/903622477 shutdown_connections 2024-08-30T09:03:16.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.381+0000 7efcedb10740 1 -- 172.21.15.42:0/903622477 wait complete. 2024-08-30T09:03:16.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.382+0000 7efcedb10740 1 -- start start 2024-08-30T09:03:16.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.382+0000 7efcedb10740 1 Processor -- start 2024-08-30T09:03:16.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.382+0000 7efcedb10740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55783ab7b490 0x55783af6b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:16.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.382+0000 7efcedb10740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55783ab83670 0x55783af6b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:16.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.382+0000 7efcedb10740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55783aba4350 0x55783af66160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:16.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.382+0000 7efcedb10740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7efce01ace20 con 0x55783ab83670 2024-08-30T09:03:16.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.382+0000 7efcedb10740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7efce01ae290 con 0x55783aba4350 2024-08-30T09:03:16.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.382+0000 7efce5337640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55783ab7b490 0x55783af6b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:03:16.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.382+0000 7efce4b36640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55783ab83670 0x55783af6b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:03:16.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.382+0000 7efce5337640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55783ab7b490 0x55783af6b1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:41888/0 (socket says 172.21.15.42:41888) 2024-08-30T09:03:16.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.383+0000 7efce5337640 1 -- 172.21.15.42:0/3303319474 learned_addr learned my addr 172.21.15.42:0/3303319474 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:03:16.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.382+0000 7efce4b36640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55783ab83670 0x55783af6b6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:58852/0 (socket says 172.21.15.42:58852) 2024-08-30T09:03:16.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.383+0000 7efcedb10740 1 -- 172.21.15.42:0/3303319474 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7efce01ad4f0 con 0x55783ab7b490 2024-08-30T09:03:16.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.383+0000 7efce5b38640 1 --2- 172.21.15.42:0/3303319474 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55783aba4350 0x55783af66160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:03:16.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.383+0000 7efcedb10740 10 client.0 _ll_register_callbacks cb 0x7efcd8020070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:03:16.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.383+0000 7efce5b38640 1 -- 172.21.15.42:0/3303319474 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55783ab7b490 msgr2=0x55783af6b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:03:16.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.383+0000 7efce5b38640 1 --2- 172.21.15.42:0/3303319474 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55783ab7b490 0x55783af6b1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:16.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.383+0000 7efce5b38640 1 -- 172.21.15.42:0/3303319474 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55783ab83670 msgr2=0x55783af6b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:03:16.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.383+0000 7efce5b38640 1 --2- 172.21.15.42:0/3303319474 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55783ab83670 0x55783af6b6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:16.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.383+0000 7efce5b38640 1 -- 172.21.15.42:0/3303319474 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55783af666a0 con 0x55783aba4350 2024-08-30T09:03:16.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.383+0000 7efce5337640 1 --2- 172.21.15.42:0/3303319474 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55783ab7b490 0x55783af6b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-30T09:03:16.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.383+0000 7efce4b36640 1 --2- 172.21.15.42:0/3303319474 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55783ab83670 0x55783af6b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-30T09:03:16.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.383+0000 7efce5b38640 1 --2- 172.21.15.42:0/3303319474 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55783aba4350 0x55783af66160 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7efcd8000c00 tx=0x7efcd80095b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=140b5c839ba05eb9 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:03:16.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.383+0000 7efcc4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-30T09:03:16.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.383+0000 7efcc4ff9640 1 -- 172.21.15.42:0/3303319474 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcd8036040 con 0x55783aba4350 2024-08-30T09:03:16.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.383+0000 7efcc4ff9640 1 -- 172.21.15.42:0/3303319474 <== mon.2 v2:172.21.15.167:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7efcd801caf0 con 0x55783aba4350 2024-08-30T09:03:16.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.383+0000 7efcedb10740 1 -- 172.21.15.42:0/3303319474 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efcd0036040 con 0x55783aba4350 2024-08-30T09:03:16.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.384+0000 7efcc4ff9640 1 -- 172.21.15.42:0/3303319474 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7efcd8048050 con 0x55783aba4350 2024-08-30T09:03:16.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.384+0000 7efcedb10740 10 client.25613 fetch_fsmap learned FSMap version 29 2024-08-30T09:03:16.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.384+0000 7efcedb10740 1 -- 172.21.15.42:0/3303319474 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55783afe4010 con 0x55783aba4350 2024-08-30T09:03:16.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.384+0000 7efcc4ff9640 1 -- 172.21.15.42:0/3303319474 <== mon.2 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efcd0036040 con 0x55783aba4350 2024-08-30T09:03:16.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.384+0000 7efcc4ff9640 1 -- 172.21.15.42:0/3303319474 <== mon.2 v2:172.21.15.167:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7efcd8040020 con 0x55783aba4350 2024-08-30T09:03:16.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.384+0000 7efcedb10740 10 client.25613 fetch_fsmap finished waiting for FSMap version 29 2024-08-30T09:03:16.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.384+0000 7efcedb10740 10 client.25613 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-30T09:03:16.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.384+0000 7efcedb10740 20 client.25613 populate_metadata read hostname 'smithi042' 2024-08-30T09:03:16.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.384+0000 7efcedb10740 1 --2- 172.21.15.42:0/3303319474 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55783ab38080 0x55783afe66c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:16.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.384+0000 7efcedb10740 4 client.25613 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-08-30T09:03:16.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.384+0000 7efcedb10740 1 -- 172.21.15.42:0/3303319474 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55783aec2540 con 0x55783ab38080 2024-08-30T09:03:16.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.384+0000 7efce5337640 1 --2- 172.21.15.42:0/3303319474 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55783ab38080 0x55783afe66c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:03:16.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.385+0000 7efcc4ff9640 10 client.25613 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:03:16.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.385+0000 7efce5337640 1 --2- 172.21.15.42:0/3303319474 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55783ab38080 0x55783afe66c0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c1db3bd038fe1176 server_cookie=e332e0bc35b8e587 in_seq=0 out_seq=0 2024-08-30T09:03:16.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.386+0000 7efcc4ff9640 1 -- 172.21.15.42:0/3303319474 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55783aec2540 con 0x55783ab38080 2024-08-30T09:03:16.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.386+0000 7efcc4ff9640 10 client.25613 handle_command_reply: tid=0 2024-08-30T09:03:16.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.386+0000 7efcedb10740 1 client.25613 shutdown 2024-08-30T09:03:16.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.386+0000 7efcedb10740 1 -- 172.21.15.42:0/3303319474 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55783aba4350 msgr2=0x55783af66160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:03:16.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.386+0000 7efcedb10740 1 --2- 172.21.15.42:0/3303319474 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55783aba4350 0x55783af66160 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7efcd8000c00 tx=0x7efcd80095b0 comp rx=0 tx=0).stop 2024-08-30T09:03:16.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.386+0000 7efcedb10740 1 -- 172.21.15.42:0/3303319474 shutdown_connections 2024-08-30T09:03:16.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.386+0000 7efcedb10740 1 --2- 172.21.15.42:0/3303319474 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55783ab38080 0x55783afe66c0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:16.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.386+0000 7efcedb10740 1 --2- 172.21.15.42:0/3303319474 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55783aba4350 0x55783af66160 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:16.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.386+0000 7efcedb10740 1 --2- 172.21.15.42:0/3303319474 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55783ab83670 0x55783af6b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:16.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.386+0000 7efcedb10740 1 --2- 172.21.15.42:0/3303319474 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55783ab7b490 0x55783af6b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:16.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.386+0000 7efcedb10740 1 -- 172.21.15.42:0/3303319474 >> 172.21.15.42:0/3303319474 conn(0x55783ab98740 msgr2=0x55783af4b760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:03:16.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.386+0000 7efcc4ff9640 0 client.25613 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:03:16.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.387+0000 7efcedb10740 1 -- 172.21.15.42:0/3303319474 shutdown_connections 2024-08-30T09:03:16.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.387+0000 7efcedb10740 1 -- 172.21.15.42:0/3303319474 wait complete. 2024-08-30T09:03:16.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.387+0000 7efcedb10740 20 client.25613 trim_cache size 0 max 16384 2024-08-30T09:03:16.393 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-08-30T09:03:16.393 INFO:teuthology.orchestra.run.smithi042.stdout: "return_code": 0, 2024-08-30T09:03:16.393 INFO:teuthology.orchestra.run.smithi042.stdout: "scrub_tag": "749f4867-0b70-49e5-af2d-8362a240ebc3", 2024-08-30T09:03:16.393 INFO:teuthology.orchestra.run.smithi042.stdout: "mode": "asynchronous" 2024-08-30T09:03:16.393 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-08-30T09:03:16.394 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.388+0000 7efce759b640 1 -- 172.21.15.42:0/467798627 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7efcc80b4bb0 msgr2=0x7efcc80b70c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:03:16.394 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.388+0000 7efce759b640 1 --2- 172.21.15.42:0/467798627 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7efcc80b4bb0 0x7efcc80b70c0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7efcd801f000 tx=0x7efcd80118b0 comp rx=0 tx=0).stop 2024-08-30T09:03:16.394 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.388+0000 7efce759b640 1 -- 172.21.15.42:0/467798627 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efce0108140 msgr2=0x7efce01985a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:03:16.394 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.388+0000 7efce759b640 1 --2- 172.21.15.42:0/467798627 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efce0108140 0x7efce01985a0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7efcd400a560 tx=0x7efcd400a960 comp rx=0 tx=0).stop 2024-08-30T09:03:16.394 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.388+0000 7efce759b640 1 -- 172.21.15.42:0/467798627 shutdown_connections 2024-08-30T09:03:16.394 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.388+0000 7efce759b640 1 --2- 172.21.15.42:0/467798627 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7efcc80b4bb0 0x7efcc80b70c0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:16.394 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.388+0000 7efce759b640 1 --2- 172.21.15.42:0/467798627 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efce0108140 0x7efce01985a0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:16.394 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.388+0000 7efce759b640 1 --2- 172.21.15.42:0/467798627 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efce0108a90 0x7efce019d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:16.394 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.388+0000 7efce759b640 1 --2- 172.21.15.42:0/467798627 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efce0102800 0x7efce0198000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:16.394 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.388+0000 7efce759b640 1 -- 172.21.15.42:0/467798627 >> 172.21.15.42:0/467798627 conn(0x7efce00794d0 msgr2=0x7efce0101130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:03:16.394 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.388+0000 7efce759b640 1 -- 172.21.15.42:0/467798627 shutdown_connections 2024-08-30T09:03:16.394 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:16.388+0000 7efce759b640 1 -- 172.21.15.42:0/467798627 wait complete. 2024-08-30T09:03:16.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:03:16 smithi190 ceph-mon[22001]: pgmap v448: 129 pgs: 129 active+clean; 1.1 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 12 MiB/s wr, 766 op/s 2024-08-30T09:03:16.443 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:03:16 smithi167 ceph-mon[22063]: pgmap v448: 129 pgs: 129 active+clean; 1.1 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 12 MiB/s wr, 766 op/s 2024-08-30T09:03:16.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:03:16 smithi042 ceph-mon[18652]: pgmap v448: 129 pgs: 129 active+clean; 1.1 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 12 MiB/s wr, 766 op/s 2024-08-30T09:03:16.827 INFO:journalctl@ceph.mds.b.smithi167.stdout:Aug 30 09:03:16 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mds-b[52473]: 2024-08-30T09:03:16.439+0000 7f9d3fee4640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000000f [...2,head] /volumes/qa/sv_1/d3168875-d2e5-410d-8754-72d29955777b/postgres/data/pg_subtrans/ auth{1=1} v1411 ap=2 f(v4 m2024-08-30T09:02:56.233030+0000 5=5+0) n(v10 rc2024-08-30T09:03:01.914981+0000 b1105920 6=5+1) (inest sync->mix g=1) caps={15402=pAsLsXsFs/pAsLsXsFsxcral@14} | dirtyscattered=0 request=0 lock=0 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x555a32f6d080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x1000000000f:[<0x10000000006/pg_subtrans v284>,<0x10000000005/data v125>,<0x100000001fb/postgres v31>,<0x100000001fa/d3168875-d2e5-410d-8754-72d29955777b v45>,<0x10000000001/sv_1 v63>,<0x10000000000/qa v49>,<0x1/volumes v44>]//[]","memoryvalue":"(2)0x1000000000f:[<0x10000000006/pg_subtrans v1411>,<0x10000000005/data v316>,<0x100000001fb/postgres v156>,<0x100000001fa/d3168875-d2e5-410d-8754-72d29955777b v173>,<0x10000000001/sv_1 v190>,<0x10000000000/qa v175>,<0x1/volumes v156>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-08-30T09:02:20.607338+0000 4=4+0)","ondisk_value.rstat":"n(v0 rc2024-08-30T09:03:01.914981+0000 b1105920 6=5+1)","memory_value.dirstat":"f(v4 m2024-08-30T09:02:56.233030+0000 5=5+0)","memory_value.rstat":"n(v10 rc2024-08-30T09:03:01.914981+0000 b1105920 6=5+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-08-30T09:03:17.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:03:17 smithi042 ceph-mon[18652]: scrub queued for path: / 2024-08-30T09:03:17.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:03:17 smithi042 ceph-mon[18652]: scrub summary: idle+waiting paths [/] 2024-08-30T09:03:17.384 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:03:17 smithi042 ceph-mon[18652]: scrub summary: active paths [/] 2024-08-30T09:03:17.384 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:03:17 smithi042 ceph-mon[18652]: scrub summary: active paths [/] 2024-08-30T09:03:17.384 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:03:17 smithi042 ceph-mon[18652]: Scrub error on inode 0x1000000000f (/volumes/qa/sv_1/d3168875-d2e5-410d-8754-72d29955777b/postgres/data/pg_subtrans) see mds.b log and `damage ls` output for details 2024-08-30T09:03:17.384 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:03:17 smithi042 ceph-mon[18652]: scrub summary: idle+waiting paths [/] 2024-08-30T09:03:17.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:03:17 smithi190 ceph-mon[22001]: scrub queued for path: / 2024-08-30T09:03:17.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:03:17 smithi190 ceph-mon[22001]: scrub summary: idle+waiting paths [/] 2024-08-30T09:03:17.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:03:17 smithi190 ceph-mon[22001]: scrub summary: active paths [/] 2024-08-30T09:03:17.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:03:17 smithi190 ceph-mon[22001]: scrub summary: active paths [/] 2024-08-30T09:03:17.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:03:17 smithi190 ceph-mon[22001]: Scrub error on inode 0x1000000000f (/volumes/qa/sv_1/d3168875-d2e5-410d-8754-72d29955777b/postgres/data/pg_subtrans) see mds.b log and `damage ls` output for details 2024-08-30T09:03:17.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:03:17 smithi190 ceph-mon[22001]: scrub summary: idle+waiting paths [/] 2024-08-30T09:03:17.512 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-30T09:03:17.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:03:17 smithi167 ceph-mon[22063]: scrub queued for path: / 2024-08-30T09:03:17.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:03:17 smithi167 ceph-mon[22063]: scrub summary: idle+waiting paths [/] 2024-08-30T09:03:17.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:03:17 smithi167 ceph-mon[22063]: scrub summary: active paths [/] 2024-08-30T09:03:17.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:03:17 smithi167 ceph-mon[22063]: scrub summary: active paths [/] 2024-08-30T09:03:17.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:03:17 smithi167 ceph-mon[22063]: Scrub error on inode 0x1000000000f (/volumes/qa/sv_1/d3168875-d2e5-410d-8754-72d29955777b/postgres/data/pg_subtrans) see mds.b log and `damage ls` output for details 2024-08-30T09:03:17.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:03:17 smithi167 ceph-mon[22063]: scrub summary: idle+waiting paths [/] 2024-08-30T09:03:17.929 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T09:03:18.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:03:18 smithi042 ceph-mon[18652]: scrub summary: active paths [/] 2024-08-30T09:03:18.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:03:18 smithi042 ceph-mon[18652]: scrub summary: idle+waiting paths [/] 2024-08-30T09:03:18.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:03:18 smithi042 ceph-mon[18652]: scrub summary: idle+waiting paths [/] 2024-08-30T09:03:18.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:03:18 smithi042 ceph-mon[18652]: pgmap v449: 129 pgs: 129 active+clean; 1.1 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 12 MiB/s wr, 835 op/s 2024-08-30T09:03:18.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:03:18 smithi190 ceph-mon[22001]: scrub summary: active paths [/] 2024-08-30T09:03:18.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:03:18 smithi190 ceph-mon[22001]: scrub summary: idle+waiting paths [/] 2024-08-30T09:03:18.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:03:18 smithi190 ceph-mon[22001]: scrub summary: idle+waiting paths [/] 2024-08-30T09:03:18.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:03:18 smithi190 ceph-mon[22001]: pgmap v449: 129 pgs: 129 active+clean; 1.1 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 12 MiB/s wr, 835 op/s 2024-08-30T09:03:18.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:03:18 smithi167 ceph-mon[22063]: scrub summary: active paths [/] 2024-08-30T09:03:18.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:03:18 smithi167 ceph-mon[22063]: scrub summary: idle+waiting paths [/] 2024-08-30T09:03:18.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:03:18 smithi167 ceph-mon[22063]: scrub summary: idle+waiting paths [/] 2024-08-30T09:03:18.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:03:18 smithi167 ceph-mon[22063]: pgmap v449: 129 pgs: 129 active+clean; 1.1 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 12 MiB/s wr, 835 op/s 2024-08-30T09:03:18.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.735+0000 7f4c18b26640 1 -- 172.21.15.42:0/1277882330 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4c0c09a960 msgr2=0x7f4c0c09ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:03:18.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.735+0000 7f4c18b26640 1 --2- 172.21.15.42:0/1277882330 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4c0c09a960 0x7f4c0c09ad70 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f4c00009900 tx=0x7f4c0002a410 comp rx=0 tx=0).stop 2024-08-30T09:03:18.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.735+0000 7f4c18b26640 1 -- 172.21.15.42:0/1277882330 shutdown_connections 2024-08-30T09:03:18.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.735+0000 7f4c18b26640 1 --2- 172.21.15.42:0/1277882330 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4c0c09b2b0 0x7f4c0c0a1ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:18.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.735+0000 7f4c18b26640 1 --2- 172.21.15.42:0/1277882330 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4c0c09a960 0x7f4c0c09ad70 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:18.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.735+0000 7f4c18b26640 1 --2- 172.21.15.42:0/1277882330 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4c0c095020 0x7f4c0c095430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:18.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.735+0000 7f4c18b26640 1 -- 172.21.15.42:0/1277882330 >> 172.21.15.42:0/1277882330 conn(0x7f4c0c00cd70 msgr2=0x7f4c0c00d180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:03:18.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.735+0000 7f4c18b26640 1 -- 172.21.15.42:0/1277882330 shutdown_connections 2024-08-30T09:03:18.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.735+0000 7f4c18b26640 1 -- 172.21.15.42:0/1277882330 wait complete. 2024-08-30T09:03:18.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.736+0000 7f4c18b26640 1 Processor -- start 2024-08-30T09:03:18.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.736+0000 7f4c18b26640 1 -- start start 2024-08-30T09:03:18.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.737+0000 7f4c18b26640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4c0c095020 0x7f4c0c12ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:18.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.737+0000 7f4c18b26640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4c0c09a960 0x7f4c0c12f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:18.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.737+0000 7f4c18b26640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4c0c09b2b0 0x7f4c0c134420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:18.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.737+0000 7f4c18b26640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f4c0c0a4f10 con 0x7f4c0c09a960 2024-08-30T09:03:18.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.737+0000 7f4c18b26640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4c0c0a4d90 con 0x7f4c0c095020 2024-08-30T09:03:18.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.737+0000 7f4c18b26640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f4c0c0a5090 con 0x7f4c0c09b2b0 2024-08-30T09:03:18.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.737+0000 7f4c137fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4c0c095020 0x7f4c0c12ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:03:18.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.737+0000 7f4c137fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4c0c095020 0x7f4c0c12ebe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:59720/0 (socket says 172.21.15.42:59720) 2024-08-30T09:03:18.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.737+0000 7f4c137fe640 1 -- 172.21.15.42:0/281184112 learned_addr learned my addr 172.21.15.42:0/281184112 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:03:18.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.737+0000 7f4c12ffd640 1 --2- 172.21.15.42:0/281184112 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4c0c09a960 0x7f4c0c12f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:03:18.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.737+0000 7f4c13fff640 1 --2- 172.21.15.42:0/281184112 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4c0c09b2b0 0x7f4c0c134420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:03:18.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.738+0000 7f4c137fe640 1 -- 172.21.15.42:0/281184112 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4c0c09b2b0 msgr2=0x7f4c0c134420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:03:18.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.738+0000 7f4c137fe640 1 --2- 172.21.15.42:0/281184112 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4c0c09b2b0 0x7f4c0c134420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:18.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.738+0000 7f4c137fe640 1 -- 172.21.15.42:0/281184112 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4c0c09a960 msgr2=0x7f4c0c12f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:03:18.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.738+0000 7f4c137fe640 1 --2- 172.21.15.42:0/281184112 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4c0c09a960 0x7f4c0c12f180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:18.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.738+0000 7f4c137fe640 1 -- 172.21.15.42:0/281184112 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c0c134b50 con 0x7f4c0c095020 2024-08-30T09:03:18.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.738+0000 7f4c137fe640 1 --2- 172.21.15.42:0/281184112 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4c0c095020 0x7f4c0c12ebe0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f4c0800d950 tx=0x7f4c0800de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:03:18.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.738+0000 7f4c10ff9640 1 -- 172.21.15.42:0/281184112 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c08017030 con 0x7f4c0c095020 2024-08-30T09:03:18.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.738+0000 7f4c10ff9640 1 -- 172.21.15.42:0/281184112 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4c08004020 con 0x7f4c0c095020 2024-08-30T09:03:18.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.738+0000 7f4c10ff9640 1 -- 172.21.15.42:0/281184112 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c080047a0 con 0x7f4c0c095020 2024-08-30T09:03:18.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.738+0000 7f4c18b26640 1 -- 172.21.15.42:0/281184112 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c0c134e40 con 0x7f4c0c095020 2024-08-30T09:03:18.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.739+0000 7f4c18b26640 1 -- 172.21.15.42:0/281184112 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c0c13b5a0 con 0x7f4c0c095020 2024-08-30T09:03:18.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.742+0000 7f4c202fd740 1 Processor -- start 2024-08-30T09:03:18.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.742+0000 7f4c202fd740 1 -- start start 2024-08-30T09:03:18.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.744+0000 7f4c202fd740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55adcfbf3070 0x55adcff984a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:18.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.744+0000 7f4c202fd740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55adcfbca1b0 0x55adcff989e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:18.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.744+0000 7f4c202fd740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55adcfbd2390 0x55adcff98f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:18.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.744+0000 7f4c202fd740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55adcff0f630 con 0x55adcfbf3070 2024-08-30T09:03:18.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.744+0000 7f4c202fd740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55adcfefebc0 con 0x55adcfbca1b0 2024-08-30T09:03:18.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.744+0000 7f4c202fd740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55adcff02b70 con 0x55adcfbd2390 2024-08-30T09:03:18.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.747+0000 7f4c13fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55adcfbf3070 0x55adcff984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:03:18.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.747+0000 7f4c13fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55adcfbf3070 0x55adcff984a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:50696/0 (socket says 172.21.15.42:50696) 2024-08-30T09:03:18.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.747+0000 7f4c13fff640 1 -- 172.21.15.42:0/1227684549 learned_addr learned my addr 172.21.15.42:0/1227684549 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:03:18.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.749+0000 7f4c12ffd640 1 --2- 172.21.15.42:0/1227684549 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55adcfbd2390 0x55adcff98f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:03:18.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.749+0000 7f4c13fff640 1 -- 172.21.15.42:0/1227684549 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55adcfbd2390 msgr2=0x55adcff98f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:03:18.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.749+0000 7f4c13fff640 1 --2- 172.21.15.42:0/1227684549 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55adcfbd2390 0x55adcff98f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:18.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.749+0000 7f4c13fff640 1 -- 172.21.15.42:0/1227684549 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55adcfbca1b0 msgr2=0x55adcff989e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-30T09:03:18.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.749+0000 7f4c13fff640 1 --2- 172.21.15.42:0/1227684549 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55adcfbca1b0 0x55adcff989e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:18.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.749+0000 7f4c13fff640 1 -- 172.21.15.42:0/1227684549 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55adcfc5e650 con 0x55adcfbf3070 2024-08-30T09:03:18.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.750+0000 7f4c10ff9640 1 -- 172.21.15.42:0/281184112 <== mon.2 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7f4c08009a30 con 0x7f4c0c095020 2024-08-30T09:03:18.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.751+0000 7f4c10ff9640 1 --2- 172.21.15.42:0/281184112 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f4c040b4c30 0x7f4c040b7140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:18.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.751+0000 7f4c10ff9640 1 -- 172.21.15.42:0/281184112 <== mon.2 v2:172.21.15.167:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f4c080e4ef0 con 0x7f4c0c095020 2024-08-30T09:03:18.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.752+0000 7f4c13fff640 1 --2- 172.21.15.42:0/1227684549 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55adcfbf3070 0x55adcff984a0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f4c1402e9f0 tx=0x7f4c14069310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b22668a9a6abc16c server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:03:18.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.753+0000 7f4bfdffb640 1 -- 172.21.15.42:0/1227684549 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c1408b070 con 0x55adcfbf3070 2024-08-30T09:03:18.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.753+0000 7f4bfdffb640 1 -- 172.21.15.42:0/1227684549 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4c14079310 con 0x55adcfbf3070 2024-08-30T09:03:18.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.753+0000 7f4bfdffb640 1 -- 172.21.15.42:0/1227684549 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c14077ca0 con 0x55adcfbf3070 2024-08-30T09:03:18.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.757+0000 7f4c202fd740 1 -- 172.21.15.42:0/1227684549 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55adcfbf3070 msgr2=0x55adcff984a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:03:18.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.757+0000 7f4c202fd740 1 --2- 172.21.15.42:0/1227684549 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55adcfbf3070 0x55adcff984a0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f4c1402e9f0 tx=0x7f4c14069310 comp rx=0 tx=0).stop 2024-08-30T09:03:18.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.757+0000 7f4c202fd740 1 -- 172.21.15.42:0/1227684549 shutdown_connections 2024-08-30T09:03:18.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.757+0000 7f4c202fd740 1 --2- 172.21.15.42:0/1227684549 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55adcfbd2390 0x55adcff98f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:18.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.757+0000 7f4c202fd740 1 --2- 172.21.15.42:0/1227684549 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55adcfbca1b0 0x55adcff989e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:18.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.757+0000 7f4c202fd740 1 --2- 172.21.15.42:0/1227684549 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55adcfbf3070 0x55adcff984a0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:18.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.757+0000 7f4c202fd740 1 -- 172.21.15.42:0/1227684549 >> 172.21.15.42:0/1227684549 conn(0x55adcfbe7460 msgr2=0x55adcff96390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:03:18.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.765+0000 7f4c137fe640 1 --2- 172.21.15.42:0/281184112 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f4c040b4c30 0x7f4c040b7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:03:18.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.766+0000 7f4c202fd740 1 -- 172.21.15.42:0/1227684549 shutdown_connections 2024-08-30T09:03:18.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.767+0000 7f4c202fd740 1 -- 172.21.15.42:0/1227684549 wait complete. 2024-08-30T09:03:18.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.768+0000 7f4c202fd740 1 -- start start 2024-08-30T09:03:18.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.768+0000 7f4c202fd740 1 Processor -- start 2024-08-30T09:03:18.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.772+0000 7f4c202fd740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55adcfbd2390 0x55adcffac540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:18.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.772+0000 7f4c202fd740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55adcfbf3070 0x55adcffacb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:18.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.772+0000 7f4c202fd740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55adcfbb19f0 0x55adcffb8740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:18.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.772+0000 7f4c202fd740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55adcfd61700 con 0x55adcfbf3070 2024-08-30T09:03:18.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.772+0000 7f4c202fd740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55adcfe85700 con 0x55adcfbd2390 2024-08-30T09:03:18.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.772+0000 7f4c202fd740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55adcffaf3b0 con 0x55adcfbb19f0 2024-08-30T09:03:18.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.772+0000 7f4c202fd740 10 client.0 _ll_register_callbacks cb 0x55adcfeb4dc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:03:18.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.776+0000 7f4c137fe640 1 --2- 172.21.15.42:0/281184112 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f4c040b4c30 0x7f4c040b7140 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x55adcff9a9c0 tx=0x7f4c080a73f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:03:18.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.780+0000 7f4c12ffd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55adcfbf3070 0x55adcffacb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:03:18.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.780+0000 7f4c12ffd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55adcfbf3070 0x55adcffacb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:50708/0 (socket says 172.21.15.42:50708) 2024-08-30T09:03:18.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.780+0000 7f4c12ffd640 1 -- 172.21.15.42:0/3532515769 learned_addr learned my addr 172.21.15.42:0/3532515769 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:03:18.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.780+0000 7f4c13fff640 1 --2- 172.21.15.42:0/3532515769 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55adcfbd2390 0x55adcffac540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:03:18.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.780+0000 7f4c13fff640 1 --2- 172.21.15.42:0/3532515769 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55adcfbb19f0 0x55adcffb8740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:03:18.794 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.789+0000 7f4c12ffd640 1 -- 172.21.15.42:0/3532515769 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55adcfbb19f0 msgr2=0x55adcffb8740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:03:18.794 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.789+0000 7f4c12ffd640 1 --2- 172.21.15.42:0/3532515769 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55adcfbb19f0 0x55adcffb8740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:18.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.789+0000 7f4c12ffd640 1 -- 172.21.15.42:0/3532515769 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55adcfbd2390 msgr2=0x55adcffac540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:03:18.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.789+0000 7f4c12ffd640 1 --2- 172.21.15.42:0/3532515769 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55adcfbd2390 0x55adcffac540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:18.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.789+0000 7f4c12ffd640 1 -- 172.21.15.42:0/3532515769 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55adcfebe8b0 con 0x55adcfbf3070 2024-08-30T09:03:18.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.795+0000 7f4c12ffd640 1 --2- 172.21.15.42:0/3532515769 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55adcfbf3070 0x55adcffacb00 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f4c0c008800 tx=0x7f4c00032d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57155dec9b3e8bc9 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:03:18.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.796+0000 7f4bf2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-08-30T09:03:18.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.799+0000 7f4bf2ffd640 1 -- 172.21.15.42:0/3532515769 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c00040070 con 0x55adcfbf3070 2024-08-30T09:03:18.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.799+0000 7f4c202fd740 1 -- 172.21.15.42:0/3532515769 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4c14079310 con 0x55adcfbf3070 2024-08-30T09:03:18.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.800+0000 7f4bf2ffd640 1 -- 172.21.15.42:0/3532515769 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f4c000318b0 con 0x55adcfbf3070 2024-08-30T09:03:18.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.807+0000 7f4bf2ffd640 1 -- 172.21.15.42:0/3532515769 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4c00035040 con 0x55adcfbf3070 2024-08-30T09:03:18.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.808+0000 7f4c202fd740 10 client.15945 fetch_fsmap learned FSMap version 29 2024-08-30T09:03:18.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.808+0000 7f4c202fd740 1 -- 172.21.15.42:0/3532515769 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55adcffbb340 con 0x55adcfbf3070 2024-08-30T09:03:18.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.810+0000 7f4bf2ffd640 1 -- 172.21.15.42:0/3532515769 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4c14079310 con 0x55adcfbf3070 2024-08-30T09:03:18.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.810+0000 7f4bf2ffd640 1 -- 172.21.15.42:0/3532515769 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f4c00004750 con 0x55adcfbf3070 2024-08-30T09:03:18.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.811+0000 7f4c202fd740 10 client.15945 fetch_fsmap finished waiting for FSMap version 29 2024-08-30T09:03:18.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.811+0000 7f4c202fd740 10 client.15945 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-30T09:03:18.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.811+0000 7f4c202fd740 20 client.15945 populate_metadata read hostname 'smithi042' 2024-08-30T09:03:18.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.811+0000 7f4c202fd740 1 --2- 172.21.15.42:0/3532515769 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55adcfb86da0 0x55adcffbe890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:18.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.811+0000 7f4c202fd740 4 client.15945 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-30T09:03:18.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.811+0000 7f4c202fd740 1 -- 172.21.15.42:0/3532515769 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55adcff11580 con 0x55adcfb86da0 2024-08-30T09:03:18.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.812+0000 7f4c13fff640 1 --2- 172.21.15.42:0/3532515769 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55adcfb86da0 0x55adcffbe890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:03:18.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.812+0000 7f4c13fff640 1 --2- 172.21.15.42:0/3532515769 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55adcfb86da0 0x55adcffbe890 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bfa04ef66cc49ab server_cookie=2b2134217546e5b9 in_seq=0 out_seq=0 2024-08-30T09:03:18.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.812+0000 7f4bf2ffd640 10 client.15945 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:03:18.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.813+0000 7f4bf2ffd640 1 -- 172.21.15.42:0/3532515769 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55adcff11580 con 0x55adcfb86da0 2024-08-30T09:03:18.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.813+0000 7f4bf2ffd640 10 client.15945 handle_command_reply: tid=0 2024-08-30T09:03:18.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.814+0000 7f4c202fd740 1 client.15945 shutdown 2024-08-30T09:03:18.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.819+0000 7f4c202fd740 1 -- 172.21.15.42:0/3532515769 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55adcfbf3070 msgr2=0x55adcffacb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:03:18.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.819+0000 7f4c202fd740 1 --2- 172.21.15.42:0/3532515769 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55adcfbf3070 0x55adcffacb00 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f4c0c008800 tx=0x7f4c00032d30 comp rx=0 tx=0).stop 2024-08-30T09:03:18.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.819+0000 7f4c202fd740 1 -- 172.21.15.42:0/3532515769 shutdown_connections 2024-08-30T09:03:18.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.819+0000 7f4c202fd740 1 --2- 172.21.15.42:0/3532515769 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55adcfb86da0 0x55adcffbe890 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:18.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.819+0000 7f4c202fd740 1 --2- 172.21.15.42:0/3532515769 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55adcfbb19f0 0x55adcffb8740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:18.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.819+0000 7f4c202fd740 1 --2- 172.21.15.42:0/3532515769 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55adcfbf3070 0x55adcffacb00 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:18.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.819+0000 7f4c202fd740 1 --2- 172.21.15.42:0/3532515769 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55adcfbd2390 0x55adcffac540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:18.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.819+0000 7f4c202fd740 1 -- 172.21.15.42:0/3532515769 >> 172.21.15.42:0/3532515769 conn(0x55adcfbe7460 msgr2=0x55adcff94800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:03:18.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.819+0000 7f4bf2ffd640 0 client.15945 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:03:18.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.821+0000 7f4c202fd740 1 -- 172.21.15.42:0/3532515769 shutdown_connections 2024-08-30T09:03:18.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.823+0000 7f4c202fd740 1 -- 172.21.15.42:0/3532515769 wait complete. 2024-08-30T09:03:18.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.823+0000 7f4c202fd740 20 client.15945 trim_cache size 0 max 16384 2024-08-30T09:03:18.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.860+0000 7f4c202fd740 1 Processor -- start 2024-08-30T09:03:18.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.862+0000 7f4c202fd740 1 -- start start 2024-08-30T09:03:18.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.863+0000 7f4c202fd740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55adcfbb19f0 0x55adcff94f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:18.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.863+0000 7f4c202fd740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55adcfbd2390 0x55adcffb8640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:18.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.863+0000 7f4c202fd740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55adcffb8b80 0x55adcffa9790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:18.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.863+0000 7f4c13fff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55adcfbb19f0 0x55adcff94f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:03:18.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.863+0000 7f4c13fff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55adcfbb19f0 0x55adcff94f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:59756/0 (socket says 172.21.15.42:59756) 2024-08-30T09:03:18.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.863+0000 7f4c13fff640 1 -- 172.21.15.42:0/3708740141 learned_addr learned my addr 172.21.15.42:0/3708740141 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:03:18.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.863+0000 7f4c202fd740 1 -- 172.21.15.42:0/3708740141 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f4c0c143880 con 0x55adcffb8b80 2024-08-30T09:03:18.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.863+0000 7f4c202fd740 1 -- 172.21.15.42:0/3708740141 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4c0c141150 con 0x55adcfbb19f0 2024-08-30T09:03:18.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.863+0000 7f4c202fd740 1 -- 172.21.15.42:0/3708740141 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f4c0c13efc0 con 0x55adcfbd2390 2024-08-30T09:03:18.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.863+0000 7f4c12ffd640 1 --2- 172.21.15.42:0/3708740141 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55adcfbd2390 0x55adcffb8640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:03:18.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.863+0000 7f4c13fff640 1 --2- 172.21.15.42:0/3708740141 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55adcffb8b80 0x55adcffa9790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:03:18.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.863+0000 7f4c13fff640 1 -- 172.21.15.42:0/3708740141 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55adcfbd2390 msgr2=0x55adcffb8640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:03:18.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.863+0000 7f4c13fff640 1 --2- 172.21.15.42:0/3708740141 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55adcfbd2390 0x55adcffb8640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:18.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.863+0000 7f4c13fff640 1 -- 172.21.15.42:0/3708740141 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55adcffb8b80 msgr2=0x55adcffa9790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:03:18.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.863+0000 7f4c13fff640 1 --2- 172.21.15.42:0/3708740141 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55adcffb8b80 0x55adcffa9790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:18.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.863+0000 7f4c13fff640 1 -- 172.21.15.42:0/3708740141 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55adcffab500 con 0x55adcfbb19f0 2024-08-30T09:03:18.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.863+0000 7f4c12ffd640 1 --2- 172.21.15.42:0/3708740141 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55adcfbd2390 0x55adcffb8640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-30T09:03:18.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.864+0000 7f4c13fff640 1 --2- 172.21.15.42:0/3708740141 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55adcfbb19f0 0x55adcff94f10 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f4c14052250 tx=0x7f4c140646e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=23c790c3f6c4cca8 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:03:18.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.864+0000 7f4bfd7fa640 1 -- 172.21.15.42:0/3708740141 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c1408b070 con 0x55adcfbb19f0 2024-08-30T09:03:18.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.864+0000 7f4bfd7fa640 1 -- 172.21.15.42:0/3708740141 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4c1408c040 con 0x55adcfbb19f0 2024-08-30T09:03:18.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.864+0000 7f4bfd7fa640 1 -- 172.21.15.42:0/3708740141 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c14096670 con 0x55adcfbb19f0 2024-08-30T09:03:18.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.866+0000 7f4c202fd740 1 -- 172.21.15.42:0/3708740141 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55adcfbb19f0 msgr2=0x55adcff94f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:03:18.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.866+0000 7f4c202fd740 1 --2- 172.21.15.42:0/3708740141 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55adcfbb19f0 0x55adcff94f10 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f4c14052250 tx=0x7f4c140646e0 comp rx=0 tx=0).stop 2024-08-30T09:03:18.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.866+0000 7f4c202fd740 1 -- 172.21.15.42:0/3708740141 shutdown_connections 2024-08-30T09:03:18.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.866+0000 7f4c202fd740 1 --2- 172.21.15.42:0/3708740141 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55adcfbd2390 0x55adcffb8640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:18.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.866+0000 7f4c202fd740 1 --2- 172.21.15.42:0/3708740141 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55adcffb8b80 0x55adcffa9790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:18.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.866+0000 7f4c202fd740 1 --2- 172.21.15.42:0/3708740141 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55adcfbb19f0 0x55adcff94f10 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:18.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.866+0000 7f4c202fd740 1 -- 172.21.15.42:0/3708740141 >> 172.21.15.42:0/3708740141 conn(0x55adcfbe7460 msgr2=0x55adcff94800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:03:18.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.866+0000 7f4c202fd740 1 -- 172.21.15.42:0/3708740141 shutdown_connections 2024-08-30T09:03:18.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.867+0000 7f4c202fd740 1 -- 172.21.15.42:0/3708740141 wait complete. 2024-08-30T09:03:18.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.867+0000 7f4c202fd740 1 -- start start 2024-08-30T09:03:18.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.868+0000 7f4c202fd740 1 Processor -- start 2024-08-30T09:03:18.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.868+0000 7f4c202fd740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55adcfbb19f0 0x55adcffb11d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:18.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.868+0000 7f4c202fd740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55adcfbd2390 0x55adcffb1770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:18.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.868+0000 7f4c202fd740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55adcffb8b80 0x55add0022440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:18.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.868+0000 7f4c202fd740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f4c0c13f610 con 0x55adcffb8b80 2024-08-30T09:03:18.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.868+0000 7f4c202fd740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4c0c140a80 con 0x55adcfbb19f0 2024-08-30T09:03:18.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.869+0000 7f4c12ffd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55adcfbd2390 0x55adcffb1770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:03:18.872 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.869+0000 7f4c12ffd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55adcfbd2390 0x55adcffb1770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:40052/0 (socket says 172.21.15.42:40052) 2024-08-30T09:03:18.872 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.869+0000 7f4c12ffd640 1 -- 172.21.15.42:0/448137732 learned_addr learned my addr 172.21.15.42:0/448137732 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:03:18.872 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.869+0000 7f4c202fd740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f4c0c13fce0 con 0x55adcfbd2390 2024-08-30T09:03:18.872 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.869+0000 7f4c202fd740 10 client.0 _ll_register_callbacks cb 0x7f4c00009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:03:18.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.869+0000 7f4c12ffd640 1 -- 172.21.15.42:0/448137732 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55adcfbb19f0 msgr2=0x55adcffb11d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-30T09:03:18.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.869+0000 7f4c12ffd640 1 --2- 172.21.15.42:0/448137732 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55adcfbb19f0 0x55adcffb11d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:18.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.869+0000 7f4c12ffd640 1 -- 172.21.15.42:0/448137732 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55adcffb8b80 msgr2=0x55add0022440 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-30T09:03:18.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.869+0000 7f4c12ffd640 1 --2- 172.21.15.42:0/448137732 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55adcffb8b80 0x55add0022440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:18.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.869+0000 7f4c12ffd640 1 -- 172.21.15.42:0/448137732 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55add0022ab0 con 0x55adcfbd2390 2024-08-30T09:03:18.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.870+0000 7f4c12ffd640 1 --2- 172.21.15.42:0/448137732 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55adcfbd2390 0x55adcffb1770 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f4c00004cc0 tx=0x7f4c000098b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1756d1b82c6a626d server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:03:18.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.871+0000 7f4bf2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-08-30T09:03:18.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.872+0000 7f4bf2ffd640 1 -- 172.21.15.42:0/448137732 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c00040050 con 0x55adcfbd2390 2024-08-30T09:03:18.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.872+0000 7f4c202fd740 1 -- 172.21.15.42:0/448137732 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4c1408c040 con 0x55adcfbd2390 2024-08-30T09:03:18.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.872+0000 7f4bf2ffd640 1 -- 172.21.15.42:0/448137732 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f4c00004d40 con 0x55adcfbd2390 2024-08-30T09:03:18.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.873+0000 7f4bf2ffd640 1 -- 172.21.15.42:0/448137732 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4c00004020 con 0x55adcfbd2390 2024-08-30T09:03:18.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.873+0000 7f4c202fd740 10 client.25642 fetch_fsmap learned FSMap version 29 2024-08-30T09:03:18.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.873+0000 7f4c202fd740 1 -- 172.21.15.42:0/448137732 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55add0029e70 con 0x55adcfbd2390 2024-08-30T09:03:18.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.873+0000 7f4bf2ffd640 1 -- 172.21.15.42:0/448137732 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4c1408c040 con 0x55adcfbd2390 2024-08-30T09:03:18.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.874+0000 7f4bf2ffd640 1 -- 172.21.15.42:0/448137732 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f4c000035f0 con 0x55adcfbd2390 2024-08-30T09:03:18.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.874+0000 7f4c202fd740 10 client.25642 fetch_fsmap finished waiting for FSMap version 29 2024-08-30T09:03:18.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.874+0000 7f4c202fd740 10 client.25642 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-30T09:03:18.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.874+0000 7f4c202fd740 20 client.25642 populate_metadata read hostname 'smithi042' 2024-08-30T09:03:18.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.874+0000 7f4c202fd740 1 --2- 172.21.15.42:0/448137732 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55add002a5d0 0x55add002c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:18.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.874+0000 7f4c202fd740 4 client.25642 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-30T09:03:18.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.874+0000 7f4c202fd740 1 -- 172.21.15.42:0/448137732 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55adcff0d020 con 0x55add002a5d0 2024-08-30T09:03:18.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.874+0000 7f4c13fff640 1 --2- 172.21.15.42:0/448137732 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55add002a5d0 0x55add002c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:03:18.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.875+0000 7f4bf2ffd640 10 client.25642 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:03:18.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.875+0000 7f4c13fff640 1 --2- 172.21.15.42:0/448137732 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55add002a5d0 0x55add002c9f0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=309941a278e3d59f server_cookie=b36caa09786ec87b in_seq=0 out_seq=0 2024-08-30T09:03:18.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.875+0000 7f4bf2ffd640 1 -- 172.21.15.42:0/448137732 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+1422 (crc 0 0 0) 0x55adcff0d020 con 0x55add002a5d0 2024-08-30T09:03:18.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.875+0000 7f4bf2ffd640 10 client.25642 handle_command_reply: tid=0 2024-08-30T09:03:18.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.875+0000 7f4c202fd740 1 client.25642 shutdown 2024-08-30T09:03:18.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.876+0000 7f4c202fd740 1 -- 172.21.15.42:0/448137732 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55adcfbd2390 msgr2=0x55adcffb1770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:03:18.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.877+0000 7f4c202fd740 1 --2- 172.21.15.42:0/448137732 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55adcfbd2390 0x55adcffb1770 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f4c00004cc0 tx=0x7f4c000098b0 comp rx=0 tx=0).stop 2024-08-30T09:03:18.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.877+0000 7f4c202fd740 1 -- 172.21.15.42:0/448137732 shutdown_connections 2024-08-30T09:03:18.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.877+0000 7f4c202fd740 1 --2- 172.21.15.42:0/448137732 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55add002a5d0 0x55add002c9f0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:18.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.877+0000 7f4c202fd740 1 --2- 172.21.15.42:0/448137732 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55adcfbd2390 0x55adcffb1770 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:18.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.877+0000 7f4c202fd740 1 --2- 172.21.15.42:0/448137732 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55adcffb8b80 0x55add0022440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:18.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.877+0000 7f4c202fd740 1 --2- 172.21.15.42:0/448137732 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55adcfbb19f0 0x55adcffb11d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:18.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.877+0000 7f4c202fd740 1 -- 172.21.15.42:0/448137732 >> 172.21.15.42:0/448137732 conn(0x55adcfbe7460 msgr2=0x55adcff994d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:03:18.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.877+0000 7f4bf2ffd640 0 client.25642 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:03:18.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.877+0000 7f4c202fd740 1 -- 172.21.15.42:0/448137732 shutdown_connections 2024-08-30T09:03:18.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.877+0000 7f4c202fd740 1 -- 172.21.15.42:0/448137732 wait complete. 2024-08-30T09:03:18.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.877+0000 7f4c202fd740 20 client.25642 trim_cache size 0 max 16384 2024-08-30T09:03:18.881 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-08-30T09:03:18.881 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "IDLE (waiting for more scrubs)", 2024-08-30T09:03:18.881 INFO:teuthology.orchestra.run.smithi042.stdout: "scrubs": { 2024-08-30T09:03:18.881 INFO:teuthology.orchestra.run.smithi042.stdout: "749f4867-0b70-49e5-af2d-8362a240ebc3": { 2024-08-30T09:03:18.881 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:03:18.881 INFO:teuthology.orchestra.run.smithi042.stdout: "tag": "749f4867-0b70-49e5-af2d-8362a240ebc3", 2024-08-30T09:03:18.882 INFO:teuthology.orchestra.run.smithi042.stdout: "options": "recursive,force" 2024-08-30T09:03:18.882 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:03:18.882 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:03:18.882 INFO:teuthology.orchestra.run.smithi042.stdout: "scrub_stats": { 2024-08-30T09:03:18.882 INFO:teuthology.orchestra.run.smithi042.stdout: "749f4867-0b70-49e5-af2d-8362a240ebc3": { 2024-08-30T09:03:18.882 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:03:16.386554+0000", 2024-08-30T09:03:18.882 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:03:18.882 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:03:18.882 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:03:18.882 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:03:18.882 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:03:18.882 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:03:18.882 INFO:teuthology.orchestra.run.smithi042.stdout: "85134e4b-ab4a-4885-b07e-a3ad01eaf13d": { 2024-08-30T09:03:18.883 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:01:27.738462+0000", 2024-08-30T09:03:18.883 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:03:18.883 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:03:18.883 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:03:18.883 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:03:18.883 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:03:18.883 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:03:18.883 INFO:teuthology.orchestra.run.smithi042.stdout: "c827ad7a-44ee-4e1d-9bb3-51fb7ab00c73": { 2024-08-30T09:03:18.883 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:02:21.686517+0000", 2024-08-30T09:03:18.883 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:03:18.883 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:03:18.883 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:03:18.883 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:03:18.884 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:03:18.884 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:03:18.884 INFO:teuthology.orchestra.run.smithi042.stdout: "5e2ed104-0625-478a-a78b-584e7f4c2817": { 2024-08-30T09:03:18.884 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:00:33.928400+0000", 2024-08-30T09:03:18.884 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:03:18.884 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:03:18.884 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:03:18.884 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:03:18.884 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:03:18.884 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:03:18.884 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:03:18.884 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-08-30T09:03:18.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.878+0000 7f4c18b26640 1 -- 172.21.15.42:0/281184112 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f4c040b4c30 msgr2=0x7f4c040b7140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:03:18.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.878+0000 7f4c18b26640 1 --2- 172.21.15.42:0/281184112 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f4c040b4c30 0x7f4c040b7140 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x55adcff9a9c0 tx=0x7f4c080a73f0 comp rx=0 tx=0).stop 2024-08-30T09:03:18.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.878+0000 7f4c18b26640 1 -- 172.21.15.42:0/281184112 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4c0c095020 msgr2=0x7f4c0c12ebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:03:18.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.878+0000 7f4c18b26640 1 --2- 172.21.15.42:0/281184112 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4c0c095020 0x7f4c0c12ebe0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f4c0800d950 tx=0x7f4c0800de10 comp rx=0 tx=0).stop 2024-08-30T09:03:18.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.878+0000 7f4c18b26640 1 -- 172.21.15.42:0/281184112 shutdown_connections 2024-08-30T09:03:18.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.878+0000 7f4c18b26640 1 --2- 172.21.15.42:0/281184112 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f4c040b4c30 0x7f4c040b7140 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:18.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.878+0000 7f4c18b26640 1 --2- 172.21.15.42:0/281184112 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4c0c09b2b0 0x7f4c0c134420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:18.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.878+0000 7f4c18b26640 1 --2- 172.21.15.42:0/281184112 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4c0c09a960 0x7f4c0c12f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:18.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.879+0000 7f4c18b26640 1 --2- 172.21.15.42:0/281184112 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4c0c095020 0x7f4c0c12ebe0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:18.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.879+0000 7f4c18b26640 1 -- 172.21.15.42:0/281184112 >> 172.21.15.42:0/281184112 conn(0x7f4c0c00cd70 msgr2=0x7f4c0c004ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:03:18.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.879+0000 7f4c18b26640 1 -- 172.21.15.42:0/281184112 shutdown_connections 2024-08-30T09:03:18.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:18.879+0000 7f4c18b26640 1 -- 172.21.15.42:0/281184112 wait complete. 2024-08-30T09:03:19.751 INFO:tasks.cephfs.filesystem:scrub status for tag:749f4867-0b70-49e5-af2d-8362a240ebc3 - {'path': '/', 'tag': '749f4867-0b70-49e5-af2d-8362a240ebc3', 'options': 'recursive,force'} 2024-08-30T09:03:20.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:03:20 smithi190 ceph-mon[22001]: pgmap v450: 129 pgs: 129 active+clean; 1.1 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 8.0 MiB/s wr, 758 op/s 2024-08-30T09:03:20.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:03:20 smithi167 ceph-mon[22063]: pgmap v450: 129 pgs: 129 active+clean; 1.1 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 8.0 MiB/s wr, 758 op/s 2024-08-30T09:03:20.632 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:03:20 smithi042 ceph-mon[18652]: pgmap v450: 129 pgs: 129 active+clean; 1.1 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 8.0 MiB/s wr, 758 op/s 2024-08-30T09:03:20.953 INFO:teuthology.orchestra.run.smithi042.stderr:progress: 155.0 s, 1326.0 tps, lat 75.464 ms stddev 79.704 2024-08-30T09:03:22.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:03:22 smithi167 ceph-mon[22063]: pgmap v451: 129 pgs: 129 active+clean; 1.1 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 5.1 MiB/s wr, 613 op/s 2024-08-30T09:03:22.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:03:22 smithi042 ceph-mon[18652]: pgmap v451: 129 pgs: 129 active+clean; 1.1 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 5.1 MiB/s wr, 613 op/s 2024-08-30T09:03:22.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:03:22 smithi190 ceph-mon[22001]: pgmap v451: 129 pgs: 129 active+clean; 1.1 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 5.1 MiB/s wr, 613 op/s 2024-08-30T09:03:24.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:03:24 smithi167 ceph-mon[22063]: pgmap v452: 129 pgs: 129 active+clean; 1.1 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 6.4 MiB/s wr, 755 op/s 2024-08-30T09:03:24.632 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:03:24 smithi042 ceph-mon[18652]: pgmap v452: 129 pgs: 129 active+clean; 1.1 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 6.4 MiB/s wr, 755 op/s 2024-08-30T09:03:24.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:03:24 smithi190 ceph-mon[22001]: pgmap v452: 129 pgs: 129 active+clean; 1.1 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 6.4 MiB/s wr, 755 op/s 2024-08-30T09:03:25.506 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.502+0000 7f30bcc4d640 1 -- 172.21.15.42:0/2586611990 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f30b8103510 msgr2=0x7f30b8109eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:03:25.506 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.502+0000 7f30bcc4d640 1 --2- 172.21.15.42:0/2586611990 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f30b8103510 0x7f30b8109eb0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f30b001acb0 tx=0x7f30b003b900 comp rx=0 tx=0).stop 2024-08-30T09:03:25.506 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.503+0000 7f30bcc4d640 1 -- 172.21.15.42:0/2586611990 shutdown_connections 2024-08-30T09:03:25.506 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.503+0000 7f30bcc4d640 1 --2- 172.21.15.42:0/2586611990 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f30b8103510 0x7f30b8109eb0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:25.506 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.503+0000 7f30bcc4d640 1 --2- 172.21.15.42:0/2586611990 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f30b8102b70 0x7f30b8102f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:25.506 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.503+0000 7f30bcc4d640 1 --2- 172.21.15.42:0/2586611990 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f30b8101af0 0x7f30b8101f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:25.506 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.503+0000 7f30bcc4d640 1 -- 172.21.15.42:0/2586611990 >> 172.21.15.42:0/2586611990 conn(0x7f30b80fd430 msgr2=0x7f30b80ff8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:03:25.509 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.506+0000 7f30bcc4d640 1 -- 172.21.15.42:0/2586611990 shutdown_connections 2024-08-30T09:03:25.509 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.506+0000 7f30bcc4d640 1 -- 172.21.15.42:0/2586611990 wait complete. 2024-08-30T09:03:25.512 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.507+0000 7f30bcc4d640 1 Processor -- start 2024-08-30T09:03:25.512 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.507+0000 7f30bcc4d640 1 -- start start 2024-08-30T09:03:25.512 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.507+0000 7f30bcc4d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f30b8101af0 0x7f30b81919b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:25.512 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.507+0000 7f30bcc4d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f30b8102b70 0x7f30b8193f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:25.512 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.507+0000 7f30bcc4d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f30b8103510 0x7f30b81965a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:25.512 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.507+0000 7f30bcc4d640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f30b806f770 con 0x7f30b8101af0 2024-08-30T09:03:25.512 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.507+0000 7f30bcc4d640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f30b806f5f0 con 0x7f30b8103510 2024-08-30T09:03:25.512 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.507+0000 7f30bcc4d640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f30b806f8f0 con 0x7f30b8102b70 2024-08-30T09:03:25.513 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.508+0000 7f30b5d9b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f30b8102b70 0x7f30b8193f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:03:25.513 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.508+0000 7f30b5d9b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f30b8102b70 0x7f30b8193f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:40068/0 (socket says 172.21.15.42:40068) 2024-08-30T09:03:25.513 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.508+0000 7f30b5d9b640 1 -- 172.21.15.42:0/2370733912 learned_addr learned my addr 172.21.15.42:0/2370733912 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:03:25.513 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.508+0000 7f30b6d9d640 1 --2- 172.21.15.42:0/2370733912 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f30b8103510 0x7f30b81965a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:03:25.513 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.508+0000 7f30b5d9b640 1 -- 172.21.15.42:0/2370733912 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f30b8103510 msgr2=0x7f30b81965a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:03:25.513 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.508+0000 7f30b5d9b640 1 --2- 172.21.15.42:0/2370733912 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f30b8103510 0x7f30b81965a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:25.513 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.508+0000 7f30b5d9b640 1 -- 172.21.15.42:0/2370733912 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f30b8101af0 msgr2=0x7f30b81919b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:03:25.513 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.508+0000 7f30b5d9b640 1 --2- 172.21.15.42:0/2370733912 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f30b8101af0 0x7f30b81919b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:25.513 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.508+0000 7f30b5d9b640 1 -- 172.21.15.42:0/2370733912 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30b8196dc0 con 0x7f30b8102b70 2024-08-30T09:03:25.514 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.510+0000 7f30b5d9b640 1 --2- 172.21.15.42:0/2370733912 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f30b8102b70 0x7f30b8193f00 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f30a800b4a0 tx=0x7f30a800b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:03:25.514 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.510+0000 7f30a77fe640 1 -- 172.21.15.42:0/2370733912 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30a8014070 con 0x7f30b8102b70 2024-08-30T09:03:25.514 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.510+0000 7f30a77fe640 1 -- 172.21.15.42:0/2370733912 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f30a8010ac0 con 0x7f30b8102b70 2024-08-30T09:03:25.515 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.511+0000 7f30a77fe640 1 -- 172.21.15.42:0/2370733912 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30a8017c30 con 0x7f30b8102b70 2024-08-30T09:03:25.515 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.511+0000 7f30bcc4d640 1 -- 172.21.15.42:0/2370733912 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30b81970b0 con 0x7f30b8102b70 2024-08-30T09:03:25.515 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.511+0000 7f30bcc4d640 1 -- 172.21.15.42:0/2370733912 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30b81975b0 con 0x7f30b8102b70 2024-08-30T09:03:25.520 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.516+0000 7f30c3bbb740 1 Processor -- start 2024-08-30T09:03:25.520 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.516+0000 7f30c3bbb740 1 -- start start 2024-08-30T09:03:25.520 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.516+0000 7f30c3bbb740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a25f2c91b0 0x55a25f66f180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:25.520 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.517+0000 7f30c3bbb740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a25f2a02f0 0x55a25f66f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:25.520 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.517+0000 7f30c3bbb740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a25f2a84d0 0x55a25f670340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:25.520 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.517+0000 7f30c3bbb740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55a25f5e2000 con 0x55a25f2c91b0 2024-08-30T09:03:25.520 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.517+0000 7f30c3bbb740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55a25f5d1590 con 0x55a25f2a02f0 2024-08-30T09:03:25.521 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.517+0000 7f30c3bbb740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55a25f5d5540 con 0x55a25f2a84d0 2024-08-30T09:03:25.523 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.519+0000 7f30b6d9d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a25f2c91b0 0x55a25f66f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:03:25.524 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.519+0000 7f30b6d9d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a25f2c91b0 0x55a25f66f180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:50738/0 (socket says 172.21.15.42:50738) 2024-08-30T09:03:25.524 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.519+0000 7f30b6d9d640 1 -- 172.21.15.42:0/350390622 learned_addr learned my addr 172.21.15.42:0/350390622 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:03:25.529 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.519+0000 7f30a77fe640 1 -- 172.21.15.42:0/2370733912 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7f30a8017070 con 0x7f30b8102b70 2024-08-30T09:03:25.529 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.520+0000 7f30b5d9b640 1 --2- 172.21.15.42:0/350390622 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a25f2a84d0 0x55a25f670340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:03:25.529 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.520+0000 7f30b5d9b640 1 -- 172.21.15.42:0/350390622 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a25f2a02f0 msgr2=0x55a25f66f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:03:25.529 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.520+0000 7f30b5d9b640 1 --2- 172.21.15.42:0/350390622 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a25f2a02f0 0x55a25f66f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:25.530 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.520+0000 7f30b5d9b640 1 -- 172.21.15.42:0/350390622 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a25f2c91b0 msgr2=0x55a25f66f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:03:25.530 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.520+0000 7f30b5d9b640 1 --2- 172.21.15.42:0/350390622 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a25f2c91b0 0x55a25f66f180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:25.530 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.520+0000 7f30b5d9b640 1 -- 172.21.15.42:0/350390622 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a25f334790 con 0x55a25f2a84d0 2024-08-30T09:03:25.530 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.521+0000 7f30b5d9b640 1 --2- 172.21.15.42:0/350390622 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a25f2a84d0 0x55a25f670340 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f30a80e2140 tx=0x7f30a80eac40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1995328dbdffa2f1 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:03:25.530 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.526+0000 7f30a77fe640 1 --2- 172.21.15.42:0/2370733912 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f30a00b4c60 0x7f30a00b7170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:25.531 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.527+0000 7f30a77fe640 1 -- 172.21.15.42:0/2370733912 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f30a80e4a90 con 0x7f30b8102b70 2024-08-30T09:03:25.531 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.527+0000 7f30a4ff9640 1 -- 172.21.15.42:0/350390622 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30a80f3070 con 0x55a25f2a84d0 2024-08-30T09:03:25.531 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.527+0000 7f30a4ff9640 1 -- 172.21.15.42:0/350390622 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f30a8017360 con 0x55a25f2a84d0 2024-08-30T09:03:25.531 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.527+0000 7f30a4ff9640 1 -- 172.21.15.42:0/350390622 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30a80aaea0 con 0x55a25f2a84d0 2024-08-30T09:03:25.535 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.528+0000 7f30b6d9d640 1 --2- 172.21.15.42:0/2370733912 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f30a00b4c60 0x7f30a00b7170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:03:25.535 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.528+0000 7f30c3bbb740 1 -- 172.21.15.42:0/350390622 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a25f2a84d0 msgr2=0x55a25f670340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:03:25.535 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.528+0000 7f30c3bbb740 1 --2- 172.21.15.42:0/350390622 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a25f2a84d0 0x55a25f670340 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f30a80e2140 tx=0x7f30a80eac40 comp rx=0 tx=0).stop 2024-08-30T09:03:25.535 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.528+0000 7f30c3bbb740 1 -- 172.21.15.42:0/350390622 shutdown_connections 2024-08-30T09:03:25.535 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.528+0000 7f30c3bbb740 1 --2- 172.21.15.42:0/350390622 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a25f2a84d0 0x55a25f670340 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:25.535 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.528+0000 7f30c3bbb740 1 --2- 172.21.15.42:0/350390622 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a25f2a02f0 0x55a25f66f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:25.535 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.528+0000 7f30c3bbb740 1 --2- 172.21.15.42:0/350390622 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a25f2c91b0 0x55a25f66f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:25.535 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.528+0000 7f30c3bbb740 1 -- 172.21.15.42:0/350390622 >> 172.21.15.42:0/350390622 conn(0x55a25f2bd300 msgr2=0x55a25f66ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:03:25.535 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.529+0000 7f30c3bbb740 1 -- 172.21.15.42:0/350390622 shutdown_connections 2024-08-30T09:03:25.535 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.529+0000 7f30c3bbb740 1 -- 172.21.15.42:0/350390622 wait complete. 2024-08-30T09:03:25.536 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.529+0000 7f30b6d9d640 1 --2- 172.21.15.42:0/2370733912 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f30a00b4c60 0x7f30a00b7170 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f30b00407a0 tx=0x7f30b0040730 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:03:25.536 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.529+0000 7f30c3bbb740 1 -- start start 2024-08-30T09:03:25.536 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.529+0000 7f30c3bbb740 1 Processor -- start 2024-08-30T09:03:25.536 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.530+0000 7f30c3bbb740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a25f2a02f0 0x55a25f678c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:25.536 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.530+0000 7f30c3bbb740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a25f2a84d0 0x55a25f679320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:25.536 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.530+0000 7f30b659c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a25f2a02f0 0x55a25f678c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:03:25.536 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.530+0000 7f30b659c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a25f2a02f0 0x55a25f678c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:40094/0 (socket says 172.21.15.42:40094) 2024-08-30T09:03:25.536 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.530+0000 7f30b5d9b640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a25f2a84d0 0x55a25f679320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:03:25.536 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.530+0000 7f30b5d9b640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a25f2a84d0 0x55a25f679320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:50744/0 (socket says 172.21.15.42:50744) 2024-08-30T09:03:25.536 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.530+0000 7f30b659c640 1 -- 172.21.15.42:0/2736699208 learned_addr learned my addr 172.21.15.42:0/2736699208 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:03:25.536 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.531+0000 7f30c3bbb740 1 --2- 172.21.15.42:0/2736699208 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a25f2c91b0 0x55a25f679e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:25.536 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.531+0000 7f30c3bbb740 1 -- 172.21.15.42:0/2736699208 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55a25f50aab0 con 0x55a25f2a84d0 2024-08-30T09:03:25.536 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.531+0000 7f30c3bbb740 1 -- 172.21.15.42:0/2736699208 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55a25f558680 con 0x55a25f2c91b0 2024-08-30T09:03:25.537 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.531+0000 7f30c3bbb740 1 -- 172.21.15.42:0/2736699208 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55a25f670e60 con 0x55a25f2a02f0 2024-08-30T09:03:25.537 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.531+0000 7f30b6d9d640 1 --2- 172.21.15.42:0/2736699208 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a25f2c91b0 0x55a25f679e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:03:25.537 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.531+0000 7f30b659c640 1 -- 172.21.15.42:0/2736699208 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a25f2c91b0 msgr2=0x55a25f679e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:03:25.537 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.531+0000 7f30b659c640 1 --2- 172.21.15.42:0/2736699208 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a25f2c91b0 0x55a25f679e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:25.537 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.531+0000 7f30b659c640 1 -- 172.21.15.42:0/2736699208 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a25f2a84d0 msgr2=0x55a25f679320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:03:25.537 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.531+0000 7f30b659c640 1 --2- 172.21.15.42:0/2736699208 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a25f2a84d0 0x55a25f679320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:25.537 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.531+0000 7f30b659c640 1 -- 172.21.15.42:0/2736699208 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a25f3ad620 con 0x55a25f2a02f0 2024-08-30T09:03:25.537 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.531+0000 7f30b659c640 1 --2- 172.21.15.42:0/2736699208 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a25f2a02f0 0x55a25f678c60 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f30ac009e70 tx=0x7f30ac00b4b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=488e8ca26b2d1f7f server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:03:25.537 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.532+0000 7f30c3bbb740 10 client.0 _ll_register_callbacks cb 0x55a25f2e0a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:03:25.537 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.532+0000 7f30c3bbb740 1 -- 172.21.15.42:0/2736699208 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55a25f5e3f50 con 0x55a25f2a02f0 2024-08-30T09:03:25.537 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.532+0000 7f309dffb640 10 client.25651 ms_handle_connect on v2:172.21.15.190:3300/0 2024-08-30T09:03:25.537 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.532+0000 7f309dffb640 1 -- 172.21.15.42:0/2736699208 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30ac015020 con 0x55a25f2a02f0 2024-08-30T09:03:25.537 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.532+0000 7f309dffb640 1 -- 172.21.15.42:0/2736699208 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f30ac011620 con 0x55a25f2a02f0 2024-08-30T09:03:25.538 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.533+0000 7f309dffb640 1 -- 172.21.15.42:0/2736699208 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f30ac011860 con 0x55a25f2a02f0 2024-08-30T09:03:25.538 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.535+0000 7f30c3bbb740 10 client.25651 fetch_fsmap learned FSMap version 29 2024-08-30T09:03:25.538 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.535+0000 7f30c3bbb740 1 -- 172.21.15.42:0/2736699208 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({fsmap=29}) -- 0x55a25f682ce0 con 0x55a25f2a02f0 2024-08-30T09:03:25.539 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.536+0000 7f309dffb640 1 -- 172.21.15.42:0/2736699208 <== mon.1 v2:172.21.15.190:3300/0 4 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f30ac022020 con 0x55a25f2a02f0 2024-08-30T09:03:25.539 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.536+0000 7f30c3bbb740 10 client.25651 fetch_fsmap finished waiting for FSMap version 29 2024-08-30T09:03:25.539 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.536+0000 7f30c3bbb740 10 client.25651 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-30T09:03:25.540 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.536+0000 7f30c3bbb740 20 client.25651 populate_metadata read hostname 'smithi042' 2024-08-30T09:03:25.540 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.536+0000 7f30c3bbb740 1 --2- 172.21.15.42:0/2736699208 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55a25f287d80 0x55a25f6864a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:25.540 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.536+0000 7f30c3bbb740 4 client.25651 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-30T09:03:25.540 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.536+0000 7f30c3bbb740 1 -- 172.21.15.42:0/2736699208 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a25f5e3f50 con 0x55a25f287d80 2024-08-30T09:03:25.540 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.536+0000 7f30b5d9b640 1 --2- 172.21.15.42:0/2736699208 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55a25f287d80 0x55a25f6864a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:03:25.541 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.539+0000 7f30b5d9b640 1 --2- 172.21.15.42:0/2736699208 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55a25f287d80 0x55a25f6864a0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=caa5b0bfdd5ab7db server_cookie=8096ae9dc6eb6178 in_seq=0 out_seq=0 2024-08-30T09:03:25.544 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.539+0000 7f309dffb640 10 client.25651 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:03:25.545 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.540+0000 7f309dffb640 1 -- 172.21.15.42:0/2736699208 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a25f5e3f50 con 0x55a25f287d80 2024-08-30T09:03:25.545 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.540+0000 7f309dffb640 10 client.25651 handle_command_reply: tid=0 2024-08-30T09:03:25.545 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.541+0000 7f30c3bbb740 1 client.25651 shutdown 2024-08-30T09:03:25.545 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.541+0000 7f30c3bbb740 1 -- 172.21.15.42:0/2736699208 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a25f2a02f0 msgr2=0x55a25f678c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:03:25.545 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.541+0000 7f30c3bbb740 1 --2- 172.21.15.42:0/2736699208 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a25f2a02f0 0x55a25f678c60 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f30ac009e70 tx=0x7f30ac00b4b0 comp rx=0 tx=0).stop 2024-08-30T09:03:25.552 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.549+0000 7f30c3bbb740 1 -- 172.21.15.42:0/2736699208 shutdown_connections 2024-08-30T09:03:25.552 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.549+0000 7f30c3bbb740 1 --2- 172.21.15.42:0/2736699208 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55a25f287d80 0x55a25f6864a0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:25.552 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.549+0000 7f30c3bbb740 1 --2- 172.21.15.42:0/2736699208 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a25f2c91b0 0x55a25f679e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:25.552 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.549+0000 7f30c3bbb740 1 --2- 172.21.15.42:0/2736699208 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a25f2a84d0 0x55a25f679320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:25.552 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.549+0000 7f30c3bbb740 1 --2- 172.21.15.42:0/2736699208 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a25f2a02f0 0x55a25f678c60 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:25.552 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.549+0000 7f309dffb640 0 client.25651 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:03:25.552 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.549+0000 7f30c3bbb740 1 -- 172.21.15.42:0/2736699208 >> 172.21.15.42:0/2736699208 conn(0x55a25f2bd300 msgr2=0x55a25f669170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:03:25.557 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.554+0000 7f30c3bbb740 1 -- 172.21.15.42:0/2736699208 shutdown_connections 2024-08-30T09:03:25.557 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.555+0000 7f30c3bbb740 1 -- 172.21.15.42:0/2736699208 wait complete. 2024-08-30T09:03:25.560 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.556+0000 7f30c3bbb740 20 client.25651 trim_cache size 0 max 16384 2024-08-30T09:03:25.596 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.590+0000 7f30c3bbb740 1 Processor -- start 2024-08-30T09:03:25.596 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.592+0000 7f30c3bbb740 1 -- start start 2024-08-30T09:03:25.596 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.592+0000 7f30c3bbb740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a25f287d80 0x55a25f673bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:25.596 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.592+0000 7f30c3bbb740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a25f2a02f0 0x55a25f677380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:25.596 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.592+0000 7f30c3bbb740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a25f2a84d0 0x55a25f6778c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:25.596 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.592+0000 7f30c3bbb740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f30b81b52d0 con 0x55a25f2a84d0 2024-08-30T09:03:25.596 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.592+0000 7f30c3bbb740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f30b81b2ba0 con 0x55a25f2a02f0 2024-08-30T09:03:25.596 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.592+0000 7f30c3bbb740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f30b81b09f0 con 0x55a25f287d80 2024-08-30T09:03:25.596 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.592+0000 7f30b6d9d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a25f2a84d0 0x55a25f6778c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:03:25.596 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.592+0000 7f30b659c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a25f287d80 0x55a25f673bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:03:25.596 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.592+0000 7f30b6d9d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a25f2a84d0 0x55a25f6778c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:50750/0 (socket says 172.21.15.42:50750) 2024-08-30T09:03:25.596 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.592+0000 7f30b6d9d640 1 -- 172.21.15.42:0/2792848901 learned_addr learned my addr 172.21.15.42:0/2792848901 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:03:25.596 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.593+0000 7f30b659c640 1 -- 172.21.15.42:0/2792848901 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a25f2a02f0 msgr2=0x55a25f677380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:03:25.597 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.593+0000 7f30b659c640 1 --2- 172.21.15.42:0/2792848901 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a25f2a02f0 0x55a25f677380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:25.597 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.593+0000 7f30b659c640 1 -- 172.21.15.42:0/2792848901 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a25f2a84d0 msgr2=0x55a25f6778c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:03:25.597 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.593+0000 7f30b659c640 1 --2- 172.21.15.42:0/2792848901 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a25f2a84d0 0x55a25f6778c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:25.597 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.593+0000 7f30b659c640 1 -- 172.21.15.42:0/2792848901 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a25f67a200 con 0x55a25f287d80 2024-08-30T09:03:25.597 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.593+0000 7f30b6d9d640 1 --2- 172.21.15.42:0/2792848901 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a25f2a84d0 0x55a25f6778c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-30T09:03:25.597 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.593+0000 7f30b659c640 1 --2- 172.21.15.42:0/2792848901 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a25f287d80 0x55a25f673bf0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f30ac000c00 tx=0x7f30ac010000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c3c96ee12bfa895b server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:03:25.597 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.593+0000 7f309ffff640 1 -- 172.21.15.42:0/2792848901 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30ac015070 con 0x55a25f287d80 2024-08-30T09:03:25.598 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.593+0000 7f309ffff640 1 -- 172.21.15.42:0/2792848901 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f30ac011620 con 0x55a25f287d80 2024-08-30T09:03:25.599 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.593+0000 7f309ffff640 1 -- 172.21.15.42:0/2792848901 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30ac0246d0 con 0x55a25f287d80 2024-08-30T09:03:25.599 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.594+0000 7f30c3bbb740 1 -- 172.21.15.42:0/2792848901 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a25f287d80 msgr2=0x55a25f673bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:03:25.599 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.594+0000 7f30c3bbb740 1 --2- 172.21.15.42:0/2792848901 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a25f287d80 0x55a25f673bf0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f30ac000c00 tx=0x7f30ac010000 comp rx=0 tx=0).stop 2024-08-30T09:03:25.599 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.594+0000 7f30c3bbb740 1 -- 172.21.15.42:0/2792848901 shutdown_connections 2024-08-30T09:03:25.599 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.594+0000 7f30c3bbb740 1 --2- 172.21.15.42:0/2792848901 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a25f2a84d0 0x55a25f6778c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:25.599 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.594+0000 7f30c3bbb740 1 --2- 172.21.15.42:0/2792848901 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a25f2a02f0 0x55a25f677380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:25.599 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.594+0000 7f30c3bbb740 1 --2- 172.21.15.42:0/2792848901 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a25f287d80 0x55a25f673bf0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:25.599 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.594+0000 7f30c3bbb740 1 -- 172.21.15.42:0/2792848901 >> 172.21.15.42:0/2792848901 conn(0x55a25f2bd300 msgr2=0x55a25f66ed70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:03:25.599 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.595+0000 7f30c3bbb740 1 -- 172.21.15.42:0/2792848901 shutdown_connections 2024-08-30T09:03:25.599 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.595+0000 7f30c3bbb740 1 -- 172.21.15.42:0/2792848901 wait complete. 2024-08-30T09:03:25.599 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.595+0000 7f30c3bbb740 1 -- start start 2024-08-30T09:03:25.600 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.595+0000 7f30c3bbb740 1 Processor -- start 2024-08-30T09:03:25.600 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.595+0000 7f30c3bbb740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a25f287d80 0x55a25f680580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:25.600 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.596+0000 7f30c3bbb740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a25f2a02f0 0x55a25f680c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:25.600 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.596+0000 7f30c3bbb740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a25f2a84d0 0x55a25f67f410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:25.600 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.596+0000 7f30c3bbb740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f30b81b1060 con 0x55a25f2a02f0 2024-08-30T09:03:25.600 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.596+0000 7f30c3bbb740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f30b81b24d0 con 0x55a25f287d80 2024-08-30T09:03:25.600 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.596+0000 7f30c3bbb740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f30b81b1730 con 0x55a25f2a84d0 2024-08-30T09:03:25.600 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.596+0000 7f30c3bbb740 10 client.0 _ll_register_callbacks cb 0x7f30ac006f20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:03:25.600 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.596+0000 7f30b659c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a25f287d80 0x55a25f680580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:03:25.600 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.596+0000 7f30b5d9b640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a25f2a02f0 0x55a25f680c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:03:25.600 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.596+0000 7f30b659c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a25f287d80 0x55a25f680580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:59824/0 (socket says 172.21.15.42:59824) 2024-08-30T09:03:25.600 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.596+0000 7f30b659c640 1 -- 172.21.15.42:0/2943106825 learned_addr learned my addr 172.21.15.42:0/2943106825 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:03:25.602 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.596+0000 7f30b659c640 1 -- 172.21.15.42:0/2943106825 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a25f2a84d0 msgr2=0x55a25f67f410 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-30T09:03:25.602 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.596+0000 7f30b659c640 1 --2- 172.21.15.42:0/2943106825 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a25f2a84d0 0x55a25f67f410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:25.602 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.596+0000 7f30b659c640 1 -- 172.21.15.42:0/2943106825 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a25f2a02f0 msgr2=0x55a25f680c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:03:25.602 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.596+0000 7f30b659c640 1 --2- 172.21.15.42:0/2943106825 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a25f2a02f0 0x55a25f680c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:25.602 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.596+0000 7f30b659c640 1 -- 172.21.15.42:0/2943106825 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a25f67fba0 con 0x55a25f287d80 2024-08-30T09:03:25.602 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.596+0000 7f30b659c640 1 --2- 172.21.15.42:0/2943106825 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a25f287d80 0x55a25f680580 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f30ac00b4f0 tx=0x7f30ac009740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aa110c036215ff26 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:03:25.602 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.596+0000 7f309dffb640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-30T09:03:25.602 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.597+0000 7f309dffb640 1 -- 172.21.15.42:0/2943106825 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30ac015040 con 0x55a25f287d80 2024-08-30T09:03:25.603 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.597+0000 7f309dffb640 1 -- 172.21.15.42:0/2943106825 <== mon.2 v2:172.21.15.167:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f30ac026890 con 0x55a25f287d80 2024-08-30T09:03:25.603 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.597+0000 7f30c3bbb740 1 -- 172.21.15.42:0/2943106825 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f30ac011620 con 0x55a25f287d80 2024-08-30T09:03:25.603 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.597+0000 7f309dffb640 1 -- 172.21.15.42:0/2943106825 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f30ac026a90 con 0x55a25f287d80 2024-08-30T09:03:25.603 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.597+0000 7f30c3bbb740 10 client.25634 fetch_fsmap learned FSMap version 29 2024-08-30T09:03:25.603 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.597+0000 7f30c3bbb740 1 -- 172.21.15.42:0/2943106825 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a25f67eaa0 con 0x55a25f287d80 2024-08-30T09:03:25.603 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.598+0000 7f309dffb640 1 -- 172.21.15.42:0/2943106825 <== mon.2 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f30ac011620 con 0x55a25f287d80 2024-08-30T09:03:25.603 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.598+0000 7f309dffb640 1 -- 172.21.15.42:0/2943106825 <== mon.2 v2:172.21.15.167:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f30ac02f070 con 0x55a25f287d80 2024-08-30T09:03:25.603 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.598+0000 7f30c3bbb740 10 client.25634 fetch_fsmap finished waiting for FSMap version 29 2024-08-30T09:03:25.603 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.598+0000 7f30c3bbb740 10 client.25634 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-30T09:03:25.603 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.598+0000 7f30c3bbb740 20 client.25634 populate_metadata read hostname 'smithi042' 2024-08-30T09:03:25.603 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.598+0000 7f30c3bbb740 1 --2- 172.21.15.42:0/2943106825 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55a25f2c91b0 0x55a25f6f3440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:25.603 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.598+0000 7f30c3bbb740 4 client.25634 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-30T09:03:25.603 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.598+0000 7f30c3bbb740 1 -- 172.21.15.42:0/2943106825 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a25f5df9f0 con 0x55a25f2c91b0 2024-08-30T09:03:25.603 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.598+0000 7f30b5d9b640 1 --2- 172.21.15.42:0/2943106825 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55a25f2c91b0 0x55a25f6f3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:03:25.604 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.600+0000 7f30b5d9b640 1 --2- 172.21.15.42:0/2943106825 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55a25f2c91b0 0x55a25f6f3440 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f11ffc8f91803973 server_cookie=534eba81de33cba1 in_seq=0 out_seq=0 2024-08-30T09:03:25.604 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.600+0000 7f309dffb640 10 client.25634 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:03:25.618 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.614+0000 7f309dffb640 1 -- 172.21.15.42:0/2943106825 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a25f5df9f0 con 0x55a25f2c91b0 2024-08-30T09:03:25.618 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.614+0000 7f309dffb640 10 client.25634 handle_command_reply: tid=0 2024-08-30T09:03:25.618 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.614+0000 7f30c3bbb740 1 client.25634 shutdown 2024-08-30T09:03:25.618 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.614+0000 7f30c3bbb740 1 -- 172.21.15.42:0/2943106825 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a25f287d80 msgr2=0x55a25f680580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:03:25.618 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.614+0000 7f30c3bbb740 1 --2- 172.21.15.42:0/2943106825 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a25f287d80 0x55a25f680580 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f30ac00b4f0 tx=0x7f30ac009740 comp rx=0 tx=0).stop 2024-08-30T09:03:25.618 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.614+0000 7f30c3bbb740 1 -- 172.21.15.42:0/2943106825 shutdown_connections 2024-08-30T09:03:25.618 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.614+0000 7f30c3bbb740 1 --2- 172.21.15.42:0/2943106825 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55a25f2c91b0 0x55a25f6f3440 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:25.618 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.615+0000 7f30c3bbb740 1 --2- 172.21.15.42:0/2943106825 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a25f2a84d0 0x55a25f67f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:25.619 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.615+0000 7f30c3bbb740 1 --2- 172.21.15.42:0/2943106825 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a25f2a02f0 0x55a25f680c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:25.619 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.615+0000 7f30c3bbb740 1 --2- 172.21.15.42:0/2943106825 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a25f287d80 0x55a25f680580 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:25.619 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.615+0000 7f30c3bbb740 1 -- 172.21.15.42:0/2943106825 >> 172.21.15.42:0/2943106825 conn(0x55a25f2bd300 msgr2=0x55a25f66ed70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:03:25.619 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.615+0000 7f309dffb640 0 client.25634 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:03:25.619 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.615+0000 7f30c3bbb740 1 -- 172.21.15.42:0/2943106825 shutdown_connections 2024-08-30T09:03:25.619 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.615+0000 7f30c3bbb740 1 -- 172.21.15.42:0/2943106825 wait complete. 2024-08-30T09:03:25.619 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.615+0000 7f30c3bbb740 20 client.25634 trim_cache size 0 max 16384 2024-08-30T09:03:25.619 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-08-30T09:03:25.619 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2024-08-30T09:03:25.619 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-08-30T09:03:25.619 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-08-30T09:03:25.620 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.615+0000 7f30bcc4d640 1 -- 172.21.15.42:0/2370733912 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f30a00b4c60 msgr2=0x7f30a00b7170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:03:25.620 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.615+0000 7f30bcc4d640 1 --2- 172.21.15.42:0/2370733912 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f30a00b4c60 0x7f30a00b7170 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f30b00407a0 tx=0x7f30b0040730 comp rx=0 tx=0).stop 2024-08-30T09:03:25.620 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.615+0000 7f30bcc4d640 1 -- 172.21.15.42:0/2370733912 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f30b8102b70 msgr2=0x7f30b8193f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:03:25.620 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.615+0000 7f30bcc4d640 1 --2- 172.21.15.42:0/2370733912 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f30b8102b70 0x7f30b8193f00 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f30a800b4a0 tx=0x7f30a800b960 comp rx=0 tx=0).stop 2024-08-30T09:03:25.620 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.616+0000 7f30bcc4d640 1 -- 172.21.15.42:0/2370733912 shutdown_connections 2024-08-30T09:03:25.621 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.616+0000 7f30bcc4d640 1 --2- 172.21.15.42:0/2370733912 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f30a00b4c60 0x7f30a00b7170 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:25.621 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.616+0000 7f30bcc4d640 1 --2- 172.21.15.42:0/2370733912 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f30b8102b70 0x7f30b8193f00 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:25.621 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.616+0000 7f30bcc4d640 1 --2- 172.21.15.42:0/2370733912 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f30b8103510 0x7f30b81965a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:25.621 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.616+0000 7f30bcc4d640 1 --2- 172.21.15.42:0/2370733912 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f30b8101af0 0x7f30b81919b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:25.621 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.616+0000 7f30bcc4d640 1 -- 172.21.15.42:0/2370733912 >> 172.21.15.42:0/2370733912 conn(0x7f30b80fd430 msgr2=0x7f30b80ff850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:03:25.621 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.616+0000 7f30bcc4d640 1 -- 172.21.15.42:0/2370733912 shutdown_connections 2024-08-30T09:03:25.621 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:25.616+0000 7f30bcc4d640 1 -- 172.21.15.42:0/2370733912 wait complete. 2024-08-30T09:03:25.953 INFO:teuthology.orchestra.run.smithi042.stderr:progress: 160.0 s, 1066.2 tps, lat 93.190 ms stddev 106.844 2024-08-30T09:03:26.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:03:26 smithi167 ceph-mon[22063]: pgmap v453: 129 pgs: 129 active+clean; 1.1 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 4.8 MiB/s wr, 614 op/s 2024-08-30T09:03:26.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:03:26 smithi042 ceph-mon[18652]: pgmap v453: 129 pgs: 129 active+clean; 1.1 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 4.8 MiB/s wr, 614 op/s 2024-08-30T09:03:26.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:03:26 smithi190 ceph-mon[22001]: pgmap v453: 129 pgs: 129 active+clean; 1.1 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 4.8 MiB/s wr, 614 op/s 2024-08-30T09:03:28.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:03:28 smithi167 ceph-mon[22063]: pgmap v454: 129 pgs: 129 active+clean; 1.1 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 6.5 MiB/s wr, 766 op/s 2024-08-30T09:03:28.639 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:03:28 smithi167 ceph-mon[22063]: scrub summary: idle 2024-08-30T09:03:28.639 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:03:28 smithi167 ceph-mon[22063]: scrub summary: idle 2024-08-30T09:03:28.639 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:03:28 smithi042 ceph-mon[18652]: pgmap v454: 129 pgs: 129 active+clean; 1.1 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 6.5 MiB/s wr, 766 op/s 2024-08-30T09:03:28.640 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:03:28 smithi042 ceph-mon[18652]: scrub summary: idle 2024-08-30T09:03:28.640 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:03:28 smithi042 ceph-mon[18652]: scrub summary: idle 2024-08-30T09:03:28.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:03:28 smithi190 ceph-mon[22001]: pgmap v454: 129 pgs: 129 active+clean; 1.1 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 6.5 MiB/s wr, 766 op/s 2024-08-30T09:03:28.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:03:28 smithi190 ceph-mon[22001]: scrub summary: idle 2024-08-30T09:03:28.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:03:28 smithi190 ceph-mon[22001]: scrub summary: idle 2024-08-30T09:03:29.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:03:29 smithi167 ceph-mon[22063]: scrub summary: idle 2024-08-30T09:03:29.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:03:29 smithi042 ceph-mon[18652]: scrub summary: idle 2024-08-30T09:03:29.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:03:29 smithi190 ceph-mon[22001]: scrub summary: idle 2024-08-30T09:03:30.534 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:03:30 smithi042 ceph-mon[18652]: pgmap v455: 129 pgs: 129 active+clean; 1.1 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 6.3 MiB/s wr, 708 op/s 2024-08-30T09:03:30.576 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:03:30 smithi167 ceph-mon[22063]: pgmap v455: 129 pgs: 129 active+clean; 1.1 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 6.3 MiB/s wr, 708 op/s 2024-08-30T09:03:30.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:03:30 smithi190 ceph-mon[22001]: pgmap v455: 129 pgs: 129 active+clean; 1.1 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 6.3 MiB/s wr, 708 op/s 2024-08-30T09:03:30.953 INFO:teuthology.orchestra.run.smithi042.stderr:progress: 165.0 s, 1080.2 tps, lat 92.381 ms stddev 99.688 2024-08-30T09:03:32.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:03:32 smithi167 ceph-mon[22063]: pgmap v456: 129 pgs: 129 active+clean; 1.1 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 5.5 MiB/s wr, 594 op/s 2024-08-30T09:03:32.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:03:32 smithi042 ceph-mon[18652]: pgmap v456: 129 pgs: 129 active+clean; 1.1 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 5.5 MiB/s wr, 594 op/s 2024-08-30T09:03:32.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:03:32 smithi190 ceph-mon[22001]: pgmap v456: 129 pgs: 129 active+clean; 1.1 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 5.5 MiB/s wr, 594 op/s 2024-08-30T09:03:34.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:03:34 smithi167 ceph-mon[22063]: pgmap v457: 129 pgs: 129 active+clean; 1.1 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 7.3 MiB/s wr, 709 op/s 2024-08-30T09:03:34.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:03:34 smithi042 ceph-mon[18652]: pgmap v457: 129 pgs: 129 active+clean; 1.1 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 7.3 MiB/s wr, 709 op/s 2024-08-30T09:03:34.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:03:34 smithi190 ceph-mon[22001]: pgmap v457: 129 pgs: 129 active+clean; 1.1 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 7.3 MiB/s wr, 709 op/s 2024-08-30T09:03:35.953 INFO:teuthology.orchestra.run.smithi042.stderr:progress: 170.0 s, 1137.0 tps, lat 87.723 ms stddev 97.409 2024-08-30T09:03:36.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:03:36 smithi167 ceph-mon[22063]: pgmap v458: 129 pgs: 129 active+clean; 1.1 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 7.2 MiB/s wr, 627 op/s 2024-08-30T09:03:36.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:03:36 smithi042 ceph-mon[18652]: pgmap v458: 129 pgs: 129 active+clean; 1.1 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 7.2 MiB/s wr, 627 op/s 2024-08-30T09:03:36.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:03:36 smithi190 ceph-mon[22001]: pgmap v458: 129 pgs: 129 active+clean; 1.1 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 7.2 MiB/s wr, 627 op/s 2024-08-30T09:03:38.578 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:03:38 smithi167 ceph-mon[22063]: pgmap v459: 129 pgs: 129 active+clean; 1.1 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 7.4 MiB/s wr, 661 op/s 2024-08-30T09:03:38.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:03:38 smithi042 ceph-mon[18652]: pgmap v459: 129 pgs: 129 active+clean; 1.1 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 7.4 MiB/s wr, 661 op/s 2024-08-30T09:03:38.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:03:38 smithi190 ceph-mon[22001]: pgmap v459: 129 pgs: 129 active+clean; 1.1 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 7.4 MiB/s wr, 661 op/s 2024-08-30T09:03:38.771 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.768+0000 7faac5053640 1 -- 172.21.15.42:0/1595632333 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faac0109190 msgr2=0x7faac010b680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:03:38.771 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.768+0000 7faac5053640 1 --2- 172.21.15.42:0/1595632333 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faac0109190 0x7faac010b680 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7faab8009c90 tx=0x7faab802a860 comp rx=0 tx=0).stop 2024-08-30T09:03:38.771 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.769+0000 7faac5053640 1 -- 172.21.15.42:0/1595632333 shutdown_connections 2024-08-30T09:03:38.771 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.769+0000 7faac5053640 1 --2- 172.21.15.42:0/1595632333 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faac0109190 0x7faac010b680 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:38.771 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.769+0000 7faac5053640 1 --2- 172.21.15.42:0/1595632333 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faac0106680 0x7faac0108aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:38.771 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.769+0000 7faac5053640 1 --2- 172.21.15.42:0/1595632333 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faac006b3f0 0x7faac0106140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:38.771 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.769+0000 7faac5053640 1 -- 172.21.15.42:0/1595632333 >> 172.21.15.42:0/1595632333 conn(0x7faac00fdc80 msgr2=0x7faac0100130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:03:38.771 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.769+0000 7faac5053640 1 -- 172.21.15.42:0/1595632333 shutdown_connections 2024-08-30T09:03:38.771 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.769+0000 7faac5053640 1 -- 172.21.15.42:0/1595632333 wait complete. 2024-08-30T09:03:38.773 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.770+0000 7faac5053640 1 Processor -- start 2024-08-30T09:03:38.773 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.770+0000 7faac5053640 1 -- start start 2024-08-30T09:03:38.774 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.770+0000 7faac5053640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faac006b3f0 0x7faac0103780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:38.774 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.770+0000 7faac5053640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faac0106680 0x7faac0102040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:38.774 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.770+0000 7faac5053640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faac0109190 0x7faac0102580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:38.774 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.770+0000 7faac5053640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7faac010f090 con 0x7faac006b3f0 2024-08-30T09:03:38.774 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.770+0000 7faac5053640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7faac010ef10 con 0x7faac0106680 2024-08-30T09:03:38.774 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.770+0000 7faac5053640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7faac010f210 con 0x7faac0109190 2024-08-30T09:03:38.774 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.770+0000 7faabed9d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faac006b3f0 0x7faac0103780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:03:38.774 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.770+0000 7faab7fff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faac0106680 0x7faac0102040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:03:38.774 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.770+0000 7faabf59e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faac0109190 0x7faac0102580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:03:38.774 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.770+0000 7faab7fff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faac0106680 0x7faac0102040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:49138/0 (socket says 172.21.15.42:49138) 2024-08-30T09:03:38.774 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.770+0000 7faabed9d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faac006b3f0 0x7faac0103780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34210/0 (socket says 172.21.15.42:34210) 2024-08-30T09:03:38.774 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.770+0000 7faab7fff640 1 -- 172.21.15.42:0/3752078536 learned_addr learned my addr 172.21.15.42:0/3752078536 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:03:38.775 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.771+0000 7faab7fff640 1 -- 172.21.15.42:0/3752078536 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faac0109190 msgr2=0x7faac0102580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:03:38.775 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.771+0000 7faab7fff640 1 --2- 172.21.15.42:0/3752078536 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faac0109190 0x7faac0102580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:38.775 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.771+0000 7faab7fff640 1 -- 172.21.15.42:0/3752078536 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faac006b3f0 msgr2=0x7faac0103780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:03:38.775 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.771+0000 7faab7fff640 1 --2- 172.21.15.42:0/3752078536 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faac006b3f0 0x7faac0103780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:38.775 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.771+0000 7faab7fff640 1 -- 172.21.15.42:0/3752078536 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faac0102cb0 con 0x7faac0106680 2024-08-30T09:03:38.775 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.771+0000 7faabed9d640 1 --2- 172.21.15.42:0/3752078536 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faac006b3f0 0x7faac0103780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-30T09:03:38.775 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.771+0000 7faab7fff640 1 --2- 172.21.15.42:0/3752078536 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faac0106680 0x7faac0102040 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7faaa800e9f0 tx=0x7faaa800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:03:38.775 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.771+0000 7faabcd99640 1 -- 172.21.15.42:0/3752078536 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaa800cd80 con 0x7faac0106680 2024-08-30T09:03:38.775 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.771+0000 7faabcd99640 1 -- 172.21.15.42:0/3752078536 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faaa80108e0 con 0x7faac0106680 2024-08-30T09:03:38.775 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.771+0000 7faac5053640 1 -- 172.21.15.42:0/3752078536 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faac01a3d80 con 0x7faac0106680 2024-08-30T09:03:38.775 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.771+0000 7faabcd99640 1 -- 172.21.15.42:0/3752078536 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaa8018980 con 0x7faac0106680 2024-08-30T09:03:38.775 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.771+0000 7faac5053640 1 -- 172.21.15.42:0/3752078536 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faac01a41c0 con 0x7faac0106680 2024-08-30T09:03:38.776 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.774+0000 7faacbfc7740 1 Processor -- start 2024-08-30T09:03:38.776 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.774+0000 7faacbfc7740 1 -- start start 2024-08-30T09:03:38.776 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.774+0000 7faacbfc7740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563b8025b1b0 0x563b80601180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:38.776 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.774+0000 7faacbfc7740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563b802322f0 0x563b806017f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:38.776 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.775+0000 7faacbfc7740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563b8023a4d0 0x563b80602340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:38.776 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.775+0000 7faacbfc7740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x563b80574000 con 0x563b802322f0 2024-08-30T09:03:38.776 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.775+0000 7faacbfc7740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x563b80563590 con 0x563b8025b1b0 2024-08-30T09:03:38.777 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.775+0000 7faacbfc7740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x563b80567540 con 0x563b8023a4d0 2024-08-30T09:03:38.778 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.775+0000 7faabf59e640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563b8025b1b0 0x563b80601180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:03:38.778 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.775+0000 7faabf59e640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563b8025b1b0 0x563b80601180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:49144/0 (socket says 172.21.15.42:49144) 2024-08-30T09:03:38.778 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.775+0000 7faabf59e640 1 -- 172.21.15.42:0/530887895 learned_addr learned my addr 172.21.15.42:0/530887895 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:03:38.778 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.775+0000 7faabf59e640 1 -- 172.21.15.42:0/530887895 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563b8023a4d0 msgr2=0x563b80602340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:03:38.778 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.775+0000 7faab7fff640 1 --2- 172.21.15.42:0/530887895 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563b8023a4d0 0x563b80602340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:03:38.778 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.775+0000 7faabf59e640 1 --2- 172.21.15.42:0/530887895 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563b8023a4d0 0x563b80602340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:38.778 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.775+0000 7faabf59e640 1 -- 172.21.15.42:0/530887895 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563b802322f0 msgr2=0x563b806017f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-30T09:03:38.778 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.775+0000 7faabf59e640 1 --2- 172.21.15.42:0/530887895 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563b802322f0 0x563b806017f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:38.778 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.775+0000 7faabf59e640 1 -- 172.21.15.42:0/530887895 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b802c6790 con 0x563b8025b1b0 2024-08-30T09:03:38.778 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.775+0000 7faabcd99640 1 -- 172.21.15.42:0/3752078536 <== mon.2 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7faaa8018b20 con 0x7faac0106680 2024-08-30T09:03:38.778 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.776+0000 7faab7fff640 1 --2- 172.21.15.42:0/530887895 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563b8023a4d0 0x563b80602340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-30T09:03:38.778 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.776+0000 7faabf59e640 1 --2- 172.21.15.42:0/530887895 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563b8025b1b0 0x563b80601180 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7faab8009c90 tx=0x7faab8009970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d6400b98b477182 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:03:38.779 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.776+0000 7faab57fa640 1 -- 172.21.15.42:0/530887895 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faab80313f0 con 0x563b8025b1b0 2024-08-30T09:03:38.779 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.776+0000 7faab57fa640 1 -- 172.21.15.42:0/530887895 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faab8036030 con 0x563b8025b1b0 2024-08-30T09:03:38.779 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.776+0000 7faab57fa640 1 -- 172.21.15.42:0/530887895 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faab8004030 con 0x563b8025b1b0 2024-08-30T09:03:38.783 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.777+0000 7faacbfc7740 1 -- 172.21.15.42:0/530887895 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563b8025b1b0 msgr2=0x563b80601180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:03:38.783 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.777+0000 7faacbfc7740 1 --2- 172.21.15.42:0/530887895 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563b8025b1b0 0x563b80601180 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7faab8009c90 tx=0x7faab8009970 comp rx=0 tx=0).stop 2024-08-30T09:03:38.783 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.777+0000 7faabcd99640 1 --2- 172.21.15.42:0/3752078536 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7faaac0b4c30 0x7faaac0b7140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:38.783 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.777+0000 7faacbfc7740 1 -- 172.21.15.42:0/530887895 shutdown_connections 2024-08-30T09:03:38.783 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.777+0000 7faacbfc7740 1 --2- 172.21.15.42:0/530887895 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563b8023a4d0 0x563b80602340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:38.783 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.777+0000 7faacbfc7740 1 --2- 172.21.15.42:0/530887895 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563b802322f0 0x563b806017f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:38.783 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.777+0000 7faacbfc7740 1 --2- 172.21.15.42:0/530887895 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563b8025b1b0 0x563b80601180 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:38.783 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.777+0000 7faacbfc7740 1 -- 172.21.15.42:0/530887895 >> 172.21.15.42:0/530887895 conn(0x563b8024f300 msgr2=0x563b805fcc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:03:38.783 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.777+0000 7faabed9d640 1 --2- 172.21.15.42:0/3752078536 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7faaac0b4c30 0x7faaac0b7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:03:38.784 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.777+0000 7faabcd99640 1 -- 172.21.15.42:0/3752078536 <== mon.2 v2:172.21.15.167:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7faaa80e5820 con 0x7faac0106680 2024-08-30T09:03:38.784 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.777+0000 7faacbfc7740 1 -- 172.21.15.42:0/530887895 shutdown_connections 2024-08-30T09:03:38.784 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.778+0000 7faacbfc7740 1 -- 172.21.15.42:0/530887895 wait complete. 2024-08-30T09:03:38.784 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.778+0000 7faabed9d640 1 --2- 172.21.15.42:0/3752078536 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7faaac0b4c30 0x7faaac0b7140 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7faab0009fa0 tx=0x7faab00046c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:03:38.784 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.778+0000 7faacbfc7740 1 -- start start 2024-08-30T09:03:38.784 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.778+0000 7faacbfc7740 1 Processor -- start 2024-08-30T09:03:38.784 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.778+0000 7faacbfc7740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563b802322f0 0x563b8060ac20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:38.784 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.778+0000 7faacbfc7740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563b8023a4d0 0x563b8060b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:38.784 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.778+0000 7faacbfc7740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563b8025b1b0 0x563b8060be50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:38.784 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.778+0000 7faacbfc7740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x563b8049cab0 con 0x563b802322f0 2024-08-30T09:03:38.785 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.778+0000 7faacbfc7740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x563b804ea680 con 0x563b8025b1b0 2024-08-30T09:03:38.785 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.778+0000 7faacbfc7740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x563b80602e00 con 0x563b8023a4d0 2024-08-30T09:03:38.785 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.779+0000 7faabed9d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563b802322f0 0x563b8060ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:03:38.785 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.779+0000 7faabed9d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563b802322f0 0x563b8060ac20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34228/0 (socket says 172.21.15.42:34228) 2024-08-30T09:03:38.785 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.779+0000 7faacbfc7740 10 client.0 _ll_register_callbacks cb 0x563b80272a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:03:38.785 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.779+0000 7faabed9d640 1 -- 172.21.15.42:0/799816239 learned_addr learned my addr 172.21.15.42:0/799816239 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:03:38.785 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.779+0000 7faab7fff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563b8023a4d0 0x563b8060b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:03:38.785 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.779+0000 7faabf59e640 1 --2- 172.21.15.42:0/799816239 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563b8025b1b0 0x563b8060be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:03:38.785 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.779+0000 7faabed9d640 1 -- 172.21.15.42:0/799816239 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563b8023a4d0 msgr2=0x563b8060b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:03:38.785 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.779+0000 7faabed9d640 1 --2- 172.21.15.42:0/799816239 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563b8023a4d0 0x563b8060b2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:38.785 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.779+0000 7faabed9d640 1 -- 172.21.15.42:0/799816239 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563b8025b1b0 msgr2=0x563b8060be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:03:38.785 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.779+0000 7faabed9d640 1 --2- 172.21.15.42:0/799816239 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563b8025b1b0 0x563b8060be50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:38.785 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.779+0000 7faabed9d640 1 -- 172.21.15.42:0/799816239 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563b8033f620 con 0x563b802322f0 2024-08-30T09:03:38.785 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.779+0000 7faab7fff640 1 --2- 172.21.15.42:0/799816239 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563b8023a4d0 0x563b8060b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-30T09:03:38.786 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.779+0000 7faabed9d640 1 --2- 172.21.15.42:0/799816239 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563b802322f0 0x563b8060ac20 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7faab0003870 tx=0x7faab0005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a905260cfdf86152 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:03:38.786 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.779+0000 7faaa67fc640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-08-30T09:03:38.786 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.779+0000 7faaa67fc640 1 -- 172.21.15.42:0/799816239 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faab000fb40 con 0x563b802322f0 2024-08-30T09:03:38.786 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.779+0000 7faaa67fc640 1 -- 172.21.15.42:0/799816239 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7faab000fce0 con 0x563b802322f0 2024-08-30T09:03:38.786 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.779+0000 7faacbfc7740 1 -- 172.21.15.42:0/799816239 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faab8036030 con 0x563b802322f0 2024-08-30T09:03:38.786 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.780+0000 7faaa67fc640 1 -- 172.21.15.42:0/799816239 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7faab001e420 con 0x563b802322f0 2024-08-30T09:03:38.786 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.780+0000 7faacbfc7740 10 client.15972 fetch_fsmap learned FSMap version 29 2024-08-30T09:03:38.786 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.780+0000 7faacbfc7740 1 -- 172.21.15.42:0/799816239 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563b80614de0 con 0x563b802322f0 2024-08-30T09:03:38.786 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.780+0000 7faaa67fc640 1 -- 172.21.15.42:0/799816239 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faab8036030 con 0x563b802322f0 2024-08-30T09:03:38.786 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.780+0000 7faaa67fc640 1 -- 172.21.15.42:0/799816239 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7faab0026620 con 0x563b802322f0 2024-08-30T09:03:38.786 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.780+0000 7faacbfc7740 10 client.15972 fetch_fsmap finished waiting for FSMap version 29 2024-08-30T09:03:38.786 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.780+0000 7faacbfc7740 10 client.15972 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-30T09:03:38.786 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.780+0000 7faacbfc7740 20 client.15972 populate_metadata read hostname 'smithi042' 2024-08-30T09:03:38.786 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.780+0000 7faacbfc7740 1 --2- 172.21.15.42:0/799816239 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x563b80219d80 0x563b80618590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:38.787 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.780+0000 7faacbfc7740 4 client.15972 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-30T09:03:38.787 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.780+0000 7faacbfc7740 1 -- 172.21.15.42:0/799816239 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563b80575f50 con 0x563b80219d80 2024-08-30T09:03:38.787 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.780+0000 7faab7fff640 1 --2- 172.21.15.42:0/799816239 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x563b80219d80 0x563b80618590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:03:38.787 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.781+0000 7faaa67fc640 10 client.15972 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:03:38.787 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.781+0000 7faab7fff640 1 --2- 172.21.15.42:0/799816239 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x563b80219d80 0x563b80618590 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3dd95f179c7f87ae server_cookie=4f541b06f829542 in_seq=0 out_seq=0 2024-08-30T09:03:38.788 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.782+0000 7faaa67fc640 1 -- 172.21.15.42:0/799816239 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x563b80575f50 con 0x563b80219d80 2024-08-30T09:03:38.788 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.782+0000 7faaa67fc640 10 client.15972 handle_command_reply: tid=0 2024-08-30T09:03:38.788 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.782+0000 7faacbfc7740 1 client.15972 shutdown 2024-08-30T09:03:38.788 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.783+0000 7faacbfc7740 1 -- 172.21.15.42:0/799816239 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563b802322f0 msgr2=0x563b8060ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:03:38.788 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.783+0000 7faacbfc7740 1 --2- 172.21.15.42:0/799816239 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563b802322f0 0x563b8060ac20 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7faab0003870 tx=0x7faab0005f90 comp rx=0 tx=0).stop 2024-08-30T09:03:38.788 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.783+0000 7faacbfc7740 1 -- 172.21.15.42:0/799816239 shutdown_connections 2024-08-30T09:03:38.788 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.783+0000 7faacbfc7740 1 --2- 172.21.15.42:0/799816239 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x563b80219d80 0x563b80618590 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:38.788 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.783+0000 7faacbfc7740 1 --2- 172.21.15.42:0/799816239 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563b8023a4d0 0x563b8060b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:38.789 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.783+0000 7faacbfc7740 1 --2- 172.21.15.42:0/799816239 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563b8025b1b0 0x563b8060be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:38.789 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.783+0000 7faacbfc7740 1 --2- 172.21.15.42:0/799816239 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563b802322f0 0x563b8060ac20 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:38.789 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.783+0000 7faacbfc7740 1 -- 172.21.15.42:0/799816239 >> 172.21.15.42:0/799816239 conn(0x563b8024f300 msgr2=0x563b805fb170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:03:38.789 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.783+0000 7faaa67fc640 0 client.15972 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:03:38.789 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.783+0000 7faacbfc7740 1 -- 172.21.15.42:0/799816239 shutdown_connections 2024-08-30T09:03:38.789 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.783+0000 7faacbfc7740 1 -- 172.21.15.42:0/799816239 wait complete. 2024-08-30T09:03:38.789 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.784+0000 7faacbfc7740 20 client.15972 trim_cache size 0 max 16384 2024-08-30T09:03:38.807 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.804+0000 7faacbfc7740 1 Processor -- start 2024-08-30T09:03:38.807 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.804+0000 7faacbfc7740 1 -- start start 2024-08-30T09:03:38.807 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.805+0000 7faacbfc7740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563b802322f0 0x563b806020e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:38.807 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.805+0000 7faacbfc7740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563b8023a4d0 0x563b805fc550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:38.808 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.805+0000 7faacbfc7740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563b8025b1b0 0x563b80609e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:38.808 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.805+0000 7faacbfc7740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7faac01b0f00 con 0x563b8025b1b0 2024-08-30T09:03:38.808 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.805+0000 7faacbfc7740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7faac01ae7d0 con 0x563b802322f0 2024-08-30T09:03:38.808 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.805+0000 7faacbfc7740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7faac01ac620 con 0x563b8023a4d0 2024-08-30T09:03:38.808 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.805+0000 7faabf59e640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563b8025b1b0 0x563b80609e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:03:38.808 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.805+0000 7faabed9d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563b802322f0 0x563b806020e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:03:38.808 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.805+0000 7faabf59e640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563b8025b1b0 0x563b80609e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34242/0 (socket says 172.21.15.42:34242) 2024-08-30T09:03:38.808 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.805+0000 7faab7fff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563b8023a4d0 0x563b805fc550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:03:38.808 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.805+0000 7faabf59e640 1 -- 172.21.15.42:0/2198780179 learned_addr learned my addr 172.21.15.42:0/2198780179 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:03:38.808 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.805+0000 7faabf59e640 1 -- 172.21.15.42:0/2198780179 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563b8023a4d0 msgr2=0x563b805fc550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:03:38.808 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.805+0000 7faabf59e640 1 --2- 172.21.15.42:0/2198780179 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563b8023a4d0 0x563b805fc550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:38.808 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.805+0000 7faabf59e640 1 -- 172.21.15.42:0/2198780179 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563b802322f0 msgr2=0x563b806020e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:03:38.808 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.805+0000 7faabf59e640 1 --2- 172.21.15.42:0/2198780179 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563b802322f0 0x563b806020e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:38.808 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.805+0000 7faabf59e640 1 -- 172.21.15.42:0/2198780179 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b8060a9b0 con 0x563b8025b1b0 2024-08-30T09:03:38.809 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.805+0000 7faabed9d640 1 --2- 172.21.15.42:0/2198780179 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563b802322f0 0x563b806020e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-30T09:03:38.809 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.805+0000 7faabf59e640 1 --2- 172.21.15.42:0/2198780179 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563b8025b1b0 0x563b80609e20 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7faab80094c0 tx=0x7faab80099e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42ed83dbe322bb32 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:03:38.809 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.806+0000 7faab4ff9640 1 -- 172.21.15.42:0/2198780179 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faab8054070 con 0x563b8025b1b0 2024-08-30T09:03:38.809 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.806+0000 7faab4ff9640 1 -- 172.21.15.42:0/2198780179 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faab8036070 con 0x563b8025b1b0 2024-08-30T09:03:38.809 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.806+0000 7faab4ff9640 1 -- 172.21.15.42:0/2198780179 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faab80044a0 con 0x563b8025b1b0 2024-08-30T09:03:38.815 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.807+0000 7faacbfc7740 1 -- 172.21.15.42:0/2198780179 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563b8025b1b0 msgr2=0x563b80609e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:03:38.815 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.807+0000 7faacbfc7740 1 --2- 172.21.15.42:0/2198780179 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563b8025b1b0 0x563b80609e20 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7faab80094c0 tx=0x7faab80099e0 comp rx=0 tx=0).stop 2024-08-30T09:03:38.815 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.807+0000 7faacbfc7740 1 -- 172.21.15.42:0/2198780179 shutdown_connections 2024-08-30T09:03:38.815 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.807+0000 7faacbfc7740 1 --2- 172.21.15.42:0/2198780179 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563b8023a4d0 0x563b805fc550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:38.816 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.807+0000 7faacbfc7740 1 --2- 172.21.15.42:0/2198780179 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563b8025b1b0 0x563b80609e20 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:38.816 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.807+0000 7faacbfc7740 1 --2- 172.21.15.42:0/2198780179 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563b802322f0 0x563b806020e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:38.816 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.807+0000 7faacbfc7740 1 -- 172.21.15.42:0/2198780179 >> 172.21.15.42:0/2198780179 conn(0x563b8024f300 msgr2=0x563b806184c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:03:38.816 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.807+0000 7faacbfc7740 1 -- 172.21.15.42:0/2198780179 shutdown_connections 2024-08-30T09:03:38.816 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.807+0000 7faacbfc7740 1 -- 172.21.15.42:0/2198780179 wait complete. 2024-08-30T09:03:38.816 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.807+0000 7faacbfc7740 1 -- start start 2024-08-30T09:03:38.816 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.807+0000 7faacbfc7740 1 Processor -- start 2024-08-30T09:03:38.816 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.807+0000 7faacbfc7740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563b802322f0 0x563b806125b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:38.816 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.807+0000 7faacbfc7740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563b8023a4d0 0x563b80612c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:38.816 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.807+0000 7faacbfc7740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563b8025b1b0 0x563b80611310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:38.816 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.807+0000 7faacbfc7740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7faac01acc90 con 0x563b8025b1b0 2024-08-30T09:03:38.816 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.807+0000 7faacbfc7740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7faac01ae100 con 0x563b802322f0 2024-08-30T09:03:38.816 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.807+0000 7faacbfc7740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7faac01ad360 con 0x563b8023a4d0 2024-08-30T09:03:38.817 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.807+0000 7faacbfc7740 10 client.0 _ll_register_callbacks cb 0x7faab0009d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:03:38.817 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.808+0000 7faabf59e640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563b8025b1b0 0x563b80611310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:03:38.817 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.808+0000 7faabf59e640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563b8025b1b0 0x563b80611310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34248/0 (socket says 172.21.15.42:34248) 2024-08-30T09:03:38.817 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.808+0000 7faabf59e640 1 -- 172.21.15.42:0/596333218 learned_addr learned my addr 172.21.15.42:0/596333218 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:03:38.817 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.808+0000 7faab7fff640 1 --2- 172.21.15.42:0/596333218 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563b8023a4d0 0x563b80612c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:03:38.817 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.808+0000 7faabed9d640 1 --2- 172.21.15.42:0/596333218 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563b802322f0 0x563b806125b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:03:38.817 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.808+0000 7faabf59e640 1 -- 172.21.15.42:0/596333218 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563b8023a4d0 msgr2=0x563b80612c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:03:38.817 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.808+0000 7faabf59e640 1 --2- 172.21.15.42:0/596333218 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563b8023a4d0 0x563b80612c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:38.817 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.808+0000 7faabf59e640 1 -- 172.21.15.42:0/596333218 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563b802322f0 msgr2=0x563b806125b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:03:38.817 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.808+0000 7faabf59e640 1 --2- 172.21.15.42:0/596333218 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563b802322f0 0x563b806125b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:38.817 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.808+0000 7faabf59e640 1 -- 172.21.15.42:0/596333218 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563b80611aa0 con 0x563b8025b1b0 2024-08-30T09:03:38.817 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.808+0000 7faabf59e640 1 --2- 172.21.15.42:0/596333218 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563b8025b1b0 0x563b80611310 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7faab8052040 tx=0x7faab8004d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8674b2c415384e1e server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:03:38.817 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.808+0000 7faaa67fc640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-08-30T09:03:38.817 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.808+0000 7faaa67fc640 1 -- 172.21.15.42:0/596333218 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faab805d060 con 0x563b8025b1b0 2024-08-30T09:03:38.818 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.808+0000 7faaa67fc640 1 -- 172.21.15.42:0/596333218 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7faab8003bc0 con 0x563b8025b1b0 2024-08-30T09:03:38.818 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.808+0000 7faacbfc7740 1 -- 172.21.15.42:0/596333218 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faab8036070 con 0x563b8025b1b0 2024-08-30T09:03:38.818 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.809+0000 7faaa67fc640 1 -- 172.21.15.42:0/596333218 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7faab8064050 con 0x563b8025b1b0 2024-08-30T09:03:38.818 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.809+0000 7faacbfc7740 10 client.15978 fetch_fsmap learned FSMap version 29 2024-08-30T09:03:38.818 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.809+0000 7faacbfc7740 1 -- 172.21.15.42:0/596333218 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563b80610950 con 0x563b8025b1b0 2024-08-30T09:03:38.818 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.810+0000 7faaa67fc640 1 -- 172.21.15.42:0/596333218 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faab8036070 con 0x563b8025b1b0 2024-08-30T09:03:38.818 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.810+0000 7faaa67fc640 1 -- 172.21.15.42:0/596333218 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7faab8030560 con 0x563b8025b1b0 2024-08-30T09:03:38.818 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.810+0000 7faacbfc7740 10 client.15978 fetch_fsmap finished waiting for FSMap version 29 2024-08-30T09:03:38.818 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.810+0000 7faacbfc7740 10 client.15978 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-30T09:03:38.818 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.810+0000 7faacbfc7740 20 client.15978 populate_metadata read hostname 'smithi042' 2024-08-30T09:03:38.818 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.810+0000 7faacbfc7740 1 --2- 172.21.15.42:0/596333218 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x563b801ef1c0 0x563b80684fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:38.818 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.810+0000 7faacbfc7740 4 client.15978 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-30T09:03:38.818 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.810+0000 7faacbfc7740 1 -- 172.21.15.42:0/596333218 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563b805719f0 con 0x563b801ef1c0 2024-08-30T09:03:38.819 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.811+0000 7faabed9d640 1 --2- 172.21.15.42:0/596333218 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x563b801ef1c0 0x563b80684fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:03:38.819 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.811+0000 7faaa67fc640 10 client.15978 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:03:38.819 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.811+0000 7faabed9d640 1 --2- 172.21.15.42:0/596333218 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x563b801ef1c0 0x563b80684fd0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7949a93240e184c1 server_cookie=b0061ac18177822e in_seq=0 out_seq=0 2024-08-30T09:03:38.825 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.823+0000 7faaa67fc640 1 -- 172.21.15.42:0/596333218 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563b805719f0 con 0x563b801ef1c0 2024-08-30T09:03:38.826 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.823+0000 7faaa67fc640 10 client.15978 handle_command_reply: tid=0 2024-08-30T09:03:38.826 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.823+0000 7faacbfc7740 1 client.15978 shutdown 2024-08-30T09:03:38.826 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.824+0000 7faacbfc7740 1 -- 172.21.15.42:0/596333218 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563b8025b1b0 msgr2=0x563b80611310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:03:38.826 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.824+0000 7faacbfc7740 1 --2- 172.21.15.42:0/596333218 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563b8025b1b0 0x563b80611310 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7faab8052040 tx=0x7faab8004d40 comp rx=0 tx=0).stop 2024-08-30T09:03:38.826 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.824+0000 7faacbfc7740 1 -- 172.21.15.42:0/596333218 shutdown_connections 2024-08-30T09:03:38.826 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.824+0000 7faacbfc7740 1 --2- 172.21.15.42:0/596333218 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x563b801ef1c0 0x563b80684fd0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:38.826 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.824+0000 7faacbfc7740 1 --2- 172.21.15.42:0/596333218 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563b8023a4d0 0x563b80612c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:38.826 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.824+0000 7faacbfc7740 1 --2- 172.21.15.42:0/596333218 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563b8025b1b0 0x563b80611310 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:38.826 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.824+0000 7faacbfc7740 1 --2- 172.21.15.42:0/596333218 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563b802322f0 0x563b806125b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:38.826 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.824+0000 7faaa67fc640 0 client.15978 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:03:38.826 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.824+0000 7faacbfc7740 1 -- 172.21.15.42:0/596333218 >> 172.21.15.42:0/596333218 conn(0x563b8024f300 msgr2=0x563b806184c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:03:38.826 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.824+0000 7faacbfc7740 1 -- 172.21.15.42:0/596333218 shutdown_connections 2024-08-30T09:03:38.827 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.824+0000 7faacbfc7740 1 -- 172.21.15.42:0/596333218 wait complete. 2024-08-30T09:03:38.827 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.824+0000 7faacbfc7740 20 client.15978 trim_cache size 0 max 16384 2024-08-30T09:03:38.827 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-08-30T09:03:38.827 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2024-08-30T09:03:38.827 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-08-30T09:03:38.827 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-08-30T09:03:38.828 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.825+0000 7faac5053640 1 -- 172.21.15.42:0/3752078536 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7faaac0b4c30 msgr2=0x7faaac0b7140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:03:38.828 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.825+0000 7faac5053640 1 --2- 172.21.15.42:0/3752078536 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7faaac0b4c30 0x7faaac0b7140 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7faab0009fa0 tx=0x7faab00046c0 comp rx=0 tx=0).stop 2024-08-30T09:03:38.828 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.825+0000 7faac5053640 1 -- 172.21.15.42:0/3752078536 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faac0106680 msgr2=0x7faac0102040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:03:38.828 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.825+0000 7faac5053640 1 --2- 172.21.15.42:0/3752078536 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faac0106680 0x7faac0102040 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7faaa800e9f0 tx=0x7faaa800eeb0 comp rx=0 tx=0).stop 2024-08-30T09:03:38.828 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.825+0000 7faac5053640 1 -- 172.21.15.42:0/3752078536 shutdown_connections 2024-08-30T09:03:38.829 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.825+0000 7faac5053640 1 --2- 172.21.15.42:0/3752078536 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7faaac0b4c30 0x7faaac0b7140 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:38.829 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.825+0000 7faac5053640 1 --2- 172.21.15.42:0/3752078536 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faac0109190 0x7faac0102580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:38.829 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.825+0000 7faac5053640 1 --2- 172.21.15.42:0/3752078536 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faac0106680 0x7faac0102040 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:38.829 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.825+0000 7faac5053640 1 --2- 172.21.15.42:0/3752078536 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faac006b3f0 0x7faac0103780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:38.829 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.825+0000 7faac5053640 1 -- 172.21.15.42:0/3752078536 >> 172.21.15.42:0/3752078536 conn(0x7faac00fdc80 msgr2=0x7faac0104bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:03:38.829 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.825+0000 7faac5053640 1 -- 172.21.15.42:0/3752078536 shutdown_connections 2024-08-30T09:03:38.829 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:38.825+0000 7faac5053640 1 -- 172.21.15.42:0/3752078536 wait complete. 2024-08-30T09:03:40.488 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:03:40 smithi190 ceph-mon[22001]: pgmap v460: 129 pgs: 129 active+clean; 1.1 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.4 MiB/s wr, 586 op/s 2024-08-30T09:03:40.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:03:40 smithi167 ceph-mon[22063]: pgmap v460: 129 pgs: 129 active+clean; 1.1 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.4 MiB/s wr, 586 op/s 2024-08-30T09:03:40.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:03:40 smithi042 ceph-mon[18652]: pgmap v460: 129 pgs: 129 active+clean; 1.1 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.4 MiB/s wr, 586 op/s 2024-08-30T09:03:40.953 INFO:teuthology.orchestra.run.smithi042.stderr:progress: 175.0 s, 1273.6 tps, lat 78.968 ms stddev 83.884 2024-08-30T09:03:42.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:03:42 smithi167 ceph-mon[22063]: pgmap v461: 129 pgs: 129 active+clean; 1.1 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 5.7 MiB/s wr, 508 op/s 2024-08-30T09:03:42.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:03:42 smithi042 ceph-mon[18652]: pgmap v461: 129 pgs: 129 active+clean; 1.1 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 5.7 MiB/s wr, 508 op/s 2024-08-30T09:03:42.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:03:42 smithi190 ceph-mon[22001]: pgmap v461: 129 pgs: 129 active+clean; 1.1 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 5.7 MiB/s wr, 508 op/s 2024-08-30T09:03:44.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:03:44 smithi167 ceph-mon[22063]: pgmap v462: 129 pgs: 129 active+clean; 1.1 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 7.0 MiB/s wr, 653 op/s 2024-08-30T09:03:44.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:03:44 smithi042 ceph-mon[18652]: pgmap v462: 129 pgs: 129 active+clean; 1.1 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 7.0 MiB/s wr, 653 op/s 2024-08-30T09:03:44.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:03:44 smithi190 ceph-mon[22001]: pgmap v462: 129 pgs: 129 active+clean; 1.1 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 7.0 MiB/s wr, 653 op/s 2024-08-30T09:03:45.953 INFO:teuthology.orchestra.run.smithi042.stderr:progress: 180.0 s, 1104.8 tps, lat 91.633 ms stddev 158.327 2024-08-30T09:03:46.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:03:46 smithi167 ceph-mon[22063]: pgmap v463: 129 pgs: 129 active+clean; 1.1 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.5 MiB/s wr, 603 op/s 2024-08-30T09:03:46.632 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:03:46 smithi042 ceph-mon[18652]: pgmap v463: 129 pgs: 129 active+clean; 1.1 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.5 MiB/s wr, 603 op/s 2024-08-30T09:03:46.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:03:46 smithi190 ceph-mon[22001]: pgmap v463: 129 pgs: 129 active+clean; 1.1 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.5 MiB/s wr, 603 op/s 2024-08-30T09:03:48.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:03:48 smithi167 ceph-mon[22063]: pgmap v464: 129 pgs: 129 active+clean; 1.1 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 MiB/s wr, 827 op/s 2024-08-30T09:03:48.632 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:03:48 smithi042 ceph-mon[18652]: pgmap v464: 129 pgs: 129 active+clean; 1.1 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 MiB/s wr, 827 op/s 2024-08-30T09:03:48.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:03:48 smithi190 ceph-mon[22001]: pgmap v464: 129 pgs: 129 active+clean; 1.1 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 MiB/s wr, 827 op/s 2024-08-30T09:03:49.752 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-30T09:03:50.153 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T09:03:50.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:03:50 smithi167 ceph-mon[22063]: pgmap v465: 129 pgs: 129 active+clean; 1.1 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 12 MiB/s wr, 888 op/s 2024-08-30T09:03:50.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:03:50 smithi042 ceph-mon[18652]: pgmap v465: 129 pgs: 129 active+clean; 1.1 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 12 MiB/s wr, 888 op/s 2024-08-30T09:03:50.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:03:50 smithi190 ceph-mon[22001]: pgmap v465: 129 pgs: 129 active+clean; 1.1 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 12 MiB/s wr, 888 op/s 2024-08-30T09:03:50.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.795+0000 7f07e42e3640 1 -- 172.21.15.42:0/4193583349 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f07dc111ce0 msgr2=0x7f07dc10a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:03:50.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.795+0000 7f07e42e3640 1 --2- 172.21.15.42:0/4193583349 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f07dc111ce0 0x7f07dc10a160 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f07d4009900 tx=0x7f07d402a3f0 comp rx=0 tx=0).stop 2024-08-30T09:03:50.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.796+0000 7f07e42e3640 1 -- 172.21.15.42:0/4193583349 shutdown_connections 2024-08-30T09:03:50.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.796+0000 7f07e42e3640 1 --2- 172.21.15.42:0/4193583349 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f07dc11c9d0 0x7f07dc11eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:50.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.796+0000 7f07e42e3640 1 --2- 172.21.15.42:0/4193583349 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f07dc10a780 0x7f07dc10ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:50.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.796+0000 7f07e42e3640 1 --2- 172.21.15.42:0/4193583349 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f07dc111ce0 0x7f07dc10a160 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:50.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.796+0000 7f07e42e3640 1 -- 172.21.15.42:0/4193583349 >> 172.21.15.42:0/4193583349 conn(0x7f07dc070c60 msgr2=0x7f07dc071070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:03:50.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.796+0000 7f07e42e3640 1 -- 172.21.15.42:0/4193583349 shutdown_connections 2024-08-30T09:03:50.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.796+0000 7f07e42e3640 1 -- 172.21.15.42:0/4193583349 wait complete. 2024-08-30T09:03:50.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.797+0000 7f07e42e3640 1 Processor -- start 2024-08-30T09:03:50.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.797+0000 7f07e42e3640 1 -- start start 2024-08-30T09:03:50.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.797+0000 7f07e42e3640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f07dc10a780 0x7f07dc1a9040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:50.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.797+0000 7f07e42e3640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f07dc111ce0 0x7f07dc1a95e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:50.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.797+0000 7f07e42e3640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f07dc11c9d0 0x7f07dc1ae880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:50.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.797+0000 7f07e42e3640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f07dc122690 con 0x7f07dc10a780 2024-08-30T09:03:50.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.797+0000 7f07e42e3640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f07dc122510 con 0x7f07dc111ce0 2024-08-30T09:03:50.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.797+0000 7f07e42e3640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f07dc122810 con 0x7f07dc11c9d0 2024-08-30T09:03:50.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.797+0000 7f07e187e640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f07dc111ce0 0x7f07dc1a95e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:03:50.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.797+0000 7f07e187e640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f07dc111ce0 0x7f07dc1a95e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:33076/0 (socket says 172.21.15.42:33076) 2024-08-30T09:03:50.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.797+0000 7f07e187e640 1 -- 172.21.15.42:0/2375333776 learned_addr learned my addr 172.21.15.42:0/2375333776 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:03:50.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.797+0000 7f07e2880640 1 --2- 172.21.15.42:0/2375333776 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f07dc11c9d0 0x7f07dc1ae880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:03:50.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.798+0000 7f07e187e640 1 -- 172.21.15.42:0/2375333776 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f07dc11c9d0 msgr2=0x7f07dc1ae880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:03:50.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.798+0000 7f07e207f640 1 --2- 172.21.15.42:0/2375333776 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f07dc10a780 0x7f07dc1a9040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:03:50.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.798+0000 7f07e187e640 1 --2- 172.21.15.42:0/2375333776 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f07dc11c9d0 0x7f07dc1ae880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:50.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.798+0000 7f07e187e640 1 -- 172.21.15.42:0/2375333776 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f07dc10a780 msgr2=0x7f07dc1a9040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:03:50.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.798+0000 7f07e187e640 1 --2- 172.21.15.42:0/2375333776 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f07dc10a780 0x7f07dc1a9040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:50.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.798+0000 7f07e187e640 1 -- 172.21.15.42:0/2375333776 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07dc1aefb0 con 0x7f07dc111ce0 2024-08-30T09:03:50.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.798+0000 7f07e2880640 1 --2- 172.21.15.42:0/2375333776 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f07dc11c9d0 0x7f07dc1ae880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-30T09:03:50.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.798+0000 7f07e187e640 1 --2- 172.21.15.42:0/2375333776 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f07dc111ce0 0x7f07dc1a95e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f07cc00eb90 tx=0x7f07cc00c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:03:50.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.798+0000 7f07db7fe640 1 -- 172.21.15.42:0/2375333776 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07cc003b70 con 0x7f07dc111ce0 2024-08-30T09:03:50.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.798+0000 7f07db7fe640 1 -- 172.21.15.42:0/2375333776 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f07cc00ee90 con 0x7f07dc111ce0 2024-08-30T09:03:50.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.798+0000 7f07db7fe640 1 -- 172.21.15.42:0/2375333776 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07cc0197b0 con 0x7f07dc111ce0 2024-08-30T09:03:50.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.798+0000 7f07e42e3640 1 -- 172.21.15.42:0/2375333776 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07dc1af2a0 con 0x7f07dc111ce0 2024-08-30T09:03:50.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.798+0000 7f07e42e3640 1 -- 172.21.15.42:0/2375333776 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07dc1b5a00 con 0x7f07dc111ce0 2024-08-30T09:03:50.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.805+0000 7f07ea858740 1 Processor -- start 2024-08-30T09:03:50.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.806+0000 7f07ea858740 1 -- start start 2024-08-30T09:03:50.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.806+0000 7f07db7fe640 1 -- 172.21.15.42:0/2375333776 <== mon.2 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7f07cc004030 con 0x7f07dc111ce0 2024-08-30T09:03:50.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.806+0000 7f07ea858740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5609d6358070 0x5609d66fd2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:50.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.806+0000 7f07ea858740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5609d632f1b0 0x5609d66fd800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:50.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.806+0000 7f07ea858740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5609d6337390 0x5609d66fdd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:50.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.806+0000 7f07ea858740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x5609d6674630 con 0x5609d632f1b0 2024-08-30T09:03:50.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.806+0000 7f07ea858740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5609d6663bc0 con 0x5609d6337390 2024-08-30T09:03:50.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.806+0000 7f07ea858740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5609d6667b70 con 0x5609d6358070 2024-08-30T09:03:50.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.806+0000 7f07e187e640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5609d6337390 0x5609d66fdd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:03:50.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.806+0000 7f07e2880640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5609d6358070 0x5609d66fd2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:03:50.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.806+0000 7f07e2880640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5609d6358070 0x5609d66fd2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:60882/0 (socket says 172.21.15.42:60882) 2024-08-30T09:03:50.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.806+0000 7f07e2880640 1 -- 172.21.15.42:0/53755047 learned_addr learned my addr 172.21.15.42:0/53755047 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:03:50.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.806+0000 7f07e207f640 1 --2- 172.21.15.42:0/53755047 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5609d632f1b0 0x5609d66fd800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:03:50.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.807+0000 7f07e2880640 1 -- 172.21.15.42:0/53755047 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5609d6337390 msgr2=0x5609d66fdd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:03:50.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.807+0000 7f07e2880640 1 --2- 172.21.15.42:0/53755047 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5609d6337390 0x5609d66fdd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:50.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.807+0000 7f07e2880640 1 -- 172.21.15.42:0/53755047 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5609d632f1b0 msgr2=0x5609d66fd800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:03:50.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.807+0000 7f07e2880640 1 --2- 172.21.15.42:0/53755047 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5609d632f1b0 0x5609d66fd800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:50.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.807+0000 7f07e2880640 1 -- 172.21.15.42:0/53755047 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5609d63c3650 con 0x5609d6358070 2024-08-30T09:03:50.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.807+0000 7f07e2880640 1 --2- 172.21.15.42:0/53755047 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5609d6358070 0x5609d66fd2c0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f07d0009cc0 tx=0x7f07d00117b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ae60db47f837fadf server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:03:50.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.807+0000 7f07db7fe640 1 --2- 172.21.15.42:0/2375333776 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f07c40b4bb0 0x7f07c40b70c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:50.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.807+0000 7f07d8d9a640 1 -- 172.21.15.42:0/53755047 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07d0012450 con 0x5609d6358070 2024-08-30T09:03:50.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.808+0000 7f07d8d9a640 1 -- 172.21.15.42:0/53755047 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f07d0012aa0 con 0x5609d6358070 2024-08-30T09:03:50.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.808+0000 7f07d8d9a640 1 -- 172.21.15.42:0/53755047 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07d000fb90 con 0x5609d6358070 2024-08-30T09:03:50.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.808+0000 7f07e207f640 1 --2- 172.21.15.42:0/2375333776 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f07c40b4bb0 0x7f07c40b70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:03:50.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.808+0000 7f07db7fe640 1 -- 172.21.15.42:0/2375333776 <== mon.2 v2:172.21.15.167:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f07cc0e5ec0 con 0x7f07dc111ce0 2024-08-30T09:03:50.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.808+0000 7f07ea858740 1 -- 172.21.15.42:0/53755047 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5609d6358070 msgr2=0x5609d66fd2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:03:50.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.808+0000 7f07ea858740 1 --2- 172.21.15.42:0/53755047 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5609d6358070 0x5609d66fd2c0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f07d0009cc0 tx=0x7f07d00117b0 comp rx=0 tx=0).stop 2024-08-30T09:03:50.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.808+0000 7f07e207f640 1 --2- 172.21.15.42:0/2375333776 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f07c40b4bb0 0x7f07c40b70c0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f07d4000c00 tx=0x7f07d4031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:03:50.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.809+0000 7f07ea858740 1 -- 172.21.15.42:0/53755047 shutdown_connections 2024-08-30T09:03:50.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.809+0000 7f07ea858740 1 --2- 172.21.15.42:0/53755047 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5609d6337390 0x5609d66fdd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:50.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.809+0000 7f07ea858740 1 --2- 172.21.15.42:0/53755047 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5609d632f1b0 0x5609d66fd800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:50.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.809+0000 7f07ea858740 1 --2- 172.21.15.42:0/53755047 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5609d6358070 0x5609d66fd2c0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:50.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.809+0000 7f07ea858740 1 -- 172.21.15.42:0/53755047 >> 172.21.15.42:0/53755047 conn(0x5609d634c460 msgr2=0x5609d66fb1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:03:50.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.809+0000 7f07ea858740 1 -- 172.21.15.42:0/53755047 shutdown_connections 2024-08-30T09:03:50.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.809+0000 7f07ea858740 1 -- 172.21.15.42:0/53755047 wait complete. 2024-08-30T09:03:50.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.809+0000 7f07ea858740 1 -- start start 2024-08-30T09:03:50.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.809+0000 7f07ea858740 1 Processor -- start 2024-08-30T09:03:50.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.810+0000 7f07ea858740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5609d632f1b0 0x5609d6708a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:50.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.810+0000 7f07ea858740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5609d6337390 0x5609d6708f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:50.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.810+0000 7f07ea858740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5609d6358070 0x5609d6714dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:50.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.810+0000 7f07ea858740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x5609d64c6700 con 0x5609d632f1b0 2024-08-30T09:03:50.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.810+0000 7f07ea858740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5609d65ea700 con 0x5609d6337390 2024-08-30T09:03:50.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.810+0000 7f07ea858740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5609d670baa0 con 0x5609d6358070 2024-08-30T09:03:50.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.810+0000 7f07ea858740 10 client.0 _ll_register_callbacks cb 0x5609d6619dc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:03:50.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.810+0000 7f07e187e640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5609d6337390 0x5609d6708f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:03:50.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.810+0000 7f07e187e640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5609d6337390 0x5609d6708f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:33086/0 (socket says 172.21.15.42:33086) 2024-08-30T09:03:50.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.810+0000 7f07e187e640 1 -- 172.21.15.42:0/3348955537 learned_addr learned my addr 172.21.15.42:0/3348955537 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:03:50.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.810+0000 7f07e2880640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5609d6358070 0x5609d6714dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:03:50.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.810+0000 7f07e207f640 1 --2- 172.21.15.42:0/3348955537 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5609d632f1b0 0x5609d6708a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:03:50.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.811+0000 7f07e187e640 1 -- 172.21.15.42:0/3348955537 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5609d6358070 msgr2=0x5609d6714dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:03:50.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.811+0000 7f07e187e640 1 --2- 172.21.15.42:0/3348955537 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5609d6358070 0x5609d6714dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:50.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.811+0000 7f07e187e640 1 -- 172.21.15.42:0/3348955537 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5609d632f1b0 msgr2=0x5609d6708a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:03:50.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.811+0000 7f07e187e640 1 --2- 172.21.15.42:0/3348955537 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5609d632f1b0 0x5609d6708a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:50.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.811+0000 7f07e187e640 1 -- 172.21.15.42:0/3348955537 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5609d66238b0 con 0x5609d6337390 2024-08-30T09:03:50.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.811+0000 7f07e187e640 1 --2- 172.21.15.42:0/3348955537 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5609d6337390 0x5609d6708f50 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f07cc0ac490 tx=0x7f07cc003d10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2158bb4a0b7da6e5 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:03:50.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.811+0000 7f07c1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-30T09:03:50.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.811+0000 7f07c1ffb640 1 -- 172.21.15.42:0/3348955537 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07cc0abc70 con 0x5609d6337390 2024-08-30T09:03:50.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.811+0000 7f07c1ffb640 1 -- 172.21.15.42:0/3348955537 <== mon.2 v2:172.21.15.167:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f07cc0cf0b0 con 0x5609d6337390 2024-08-30T09:03:50.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.811+0000 7f07ea858740 1 -- 172.21.15.42:0/3348955537 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f07d0012aa0 con 0x5609d6337390 2024-08-30T09:03:50.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.812+0000 7f07c1ffb640 1 -- 172.21.15.42:0/3348955537 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f07cc0ac010 con 0x5609d6337390 2024-08-30T09:03:50.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.812+0000 7f07ea858740 10 client.25664 fetch_fsmap learned FSMap version 29 2024-08-30T09:03:50.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.812+0000 7f07ea858740 1 -- 172.21.15.42:0/3348955537 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5609d6717810 con 0x5609d6337390 2024-08-30T09:03:50.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.812+0000 7f07c1ffb640 1 -- 172.21.15.42:0/3348955537 <== mon.2 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f07cc0acc00 con 0x5609d6337390 2024-08-30T09:03:50.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.813+0000 7f07c1ffb640 1 -- 172.21.15.42:0/3348955537 <== mon.2 v2:172.21.15.167:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f07cc0b5390 con 0x5609d6337390 2024-08-30T09:03:50.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.813+0000 7f07ea858740 10 client.25664 fetch_fsmap finished waiting for FSMap version 29 2024-08-30T09:03:50.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.813+0000 7f07ea858740 10 client.25664 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-30T09:03:50.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.813+0000 7f07ea858740 20 client.25664 populate_metadata read hostname 'smithi042' 2024-08-30T09:03:50.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.813+0000 7f07ea858740 1 --2- 172.21.15.42:0/3348955537 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x5609d63169f0 0x5609d671ace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:50.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.813+0000 7f07ea858740 4 client.25664 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-30T09:03:50.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.813+0000 7f07ea858740 1 -- 172.21.15.42:0/3348955537 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f07d0012aa0 con 0x5609d63169f0 2024-08-30T09:03:50.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.813+0000 7f07e207f640 1 --2- 172.21.15.42:0/3348955537 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x5609d63169f0 0x5609d671ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:03:50.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.814+0000 7f07c1ffb640 10 client.25664 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:03:50.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.814+0000 7f07e207f640 1 --2- 172.21.15.42:0/3348955537 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x5609d63169f0 0x5609d671ace0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e024e0dba9bd0a00 server_cookie=c7590de531dd902a in_seq=0 out_seq=0 2024-08-30T09:03:50.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.817+0000 7f07c1ffb640 1 -- 172.21.15.42:0/3348955537 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f07d0012aa0 con 0x5609d63169f0 2024-08-30T09:03:50.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.817+0000 7f07c1ffb640 10 client.25664 handle_command_reply: tid=0 2024-08-30T09:03:50.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.817+0000 7f07ea858740 1 client.25664 shutdown 2024-08-30T09:03:50.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.817+0000 7f07ea858740 1 -- 172.21.15.42:0/3348955537 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5609d6337390 msgr2=0x5609d6708f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:03:50.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.817+0000 7f07ea858740 1 --2- 172.21.15.42:0/3348955537 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5609d6337390 0x5609d6708f50 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f07cc0ac490 tx=0x7f07cc003d10 comp rx=0 tx=0).stop 2024-08-30T09:03:50.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.817+0000 7f07ea858740 1 -- 172.21.15.42:0/3348955537 shutdown_connections 2024-08-30T09:03:50.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.817+0000 7f07ea858740 1 --2- 172.21.15.42:0/3348955537 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x5609d63169f0 0x5609d671ace0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:50.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.817+0000 7f07ea858740 1 --2- 172.21.15.42:0/3348955537 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5609d6358070 0x5609d6714dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:50.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.817+0000 7f07ea858740 1 --2- 172.21.15.42:0/3348955537 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5609d6337390 0x5609d6708f50 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:50.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.817+0000 7f07ea858740 1 --2- 172.21.15.42:0/3348955537 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5609d632f1b0 0x5609d6708a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:50.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.817+0000 7f07c1ffb640 0 client.25664 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:03:50.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.817+0000 7f07ea858740 1 -- 172.21.15.42:0/3348955537 >> 172.21.15.42:0/3348955537 conn(0x5609d634c460 msgr2=0x5609d6700450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:03:50.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.817+0000 7f07ea858740 1 -- 172.21.15.42:0/3348955537 shutdown_connections 2024-08-30T09:03:50.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.817+0000 7f07ea858740 1 -- 172.21.15.42:0/3348955537 wait complete. 2024-08-30T09:03:50.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.817+0000 7f07ea858740 20 client.25664 trim_cache size 0 max 16384 2024-08-30T09:03:50.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.861+0000 7f07ea858740 1 Processor -- start 2024-08-30T09:03:50.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.861+0000 7f07ea858740 1 -- start start 2024-08-30T09:03:50.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.861+0000 7f07ea858740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5609d632f1b0 0x5609d6714cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:50.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.861+0000 7f07ea858740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5609d6337390 0x5609d67003c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:50.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.861+0000 7f07ea858740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5609d6358070 0x5609d6700900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:50.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.861+0000 7f07ea858740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f07dc1c20c0 con 0x5609d6358070 2024-08-30T09:03:50.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.861+0000 7f07ea858740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f07dc1bf990 con 0x5609d632f1b0 2024-08-30T09:03:50.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.861+0000 7f07ea858740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f07dc1bd7e0 con 0x5609d6337390 2024-08-30T09:03:50.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.861+0000 7f07e2880640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5609d6358070 0x5609d6700900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:03:50.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.861+0000 7f07e207f640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5609d632f1b0 0x5609d6714cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:03:50.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.861+0000 7f07e207f640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5609d632f1b0 0x5609d6714cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:33100/0 (socket says 172.21.15.42:33100) 2024-08-30T09:03:50.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.861+0000 7f07e207f640 1 -- 172.21.15.42:0/2090040634 learned_addr learned my addr 172.21.15.42:0/2090040634 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:03:50.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.861+0000 7f07e2880640 1 --2- 172.21.15.42:0/2090040634 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5609d6358070 0x5609d6700900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:55558/0 (socket says 172.21.15.42:55558) 2024-08-30T09:03:50.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.862+0000 7f07e187e640 1 --2- 172.21.15.42:0/2090040634 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5609d6337390 0x5609d67003c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:03:50.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.862+0000 7f07e207f640 1 -- 172.21.15.42:0/2090040634 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5609d6337390 msgr2=0x5609d67003c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:03:50.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.862+0000 7f07e207f640 1 --2- 172.21.15.42:0/2090040634 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5609d6337390 0x5609d67003c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:50.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.862+0000 7f07e207f640 1 -- 172.21.15.42:0/2090040634 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5609d6358070 msgr2=0x5609d6700900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:03:50.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.862+0000 7f07e207f640 1 --2- 172.21.15.42:0/2090040634 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5609d6358070 0x5609d6700900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:50.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.862+0000 7f07e207f640 1 -- 172.21.15.42:0/2090040634 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5609d6709580 con 0x5609d632f1b0 2024-08-30T09:03:50.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.862+0000 7f07e2880640 1 --2- 172.21.15.42:0/2090040634 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5609d6358070 0x5609d6700900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-30T09:03:50.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.862+0000 7f07e207f640 1 --2- 172.21.15.42:0/2090040634 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5609d632f1b0 0x5609d6714cc0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f07d402a910 tx=0x7f07d4040000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7ede2cf85d40844 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:03:50.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.862+0000 7f07c3fff640 1 -- 172.21.15.42:0/2090040634 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07d4057070 con 0x5609d632f1b0 2024-08-30T09:03:50.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.862+0000 7f07c3fff640 1 -- 172.21.15.42:0/2090040634 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f07d4004cf0 con 0x5609d632f1b0 2024-08-30T09:03:50.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.862+0000 7f07c3fff640 1 -- 172.21.15.42:0/2090040634 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07d4035040 con 0x5609d632f1b0 2024-08-30T09:03:50.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.864+0000 7f07ea858740 1 -- 172.21.15.42:0/2090040634 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5609d632f1b0 msgr2=0x5609d6714cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:03:50.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.864+0000 7f07ea858740 1 --2- 172.21.15.42:0/2090040634 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5609d632f1b0 0x5609d6714cc0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f07d402a910 tx=0x7f07d4040000 comp rx=0 tx=0).stop 2024-08-30T09:03:50.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.864+0000 7f07ea858740 1 -- 172.21.15.42:0/2090040634 shutdown_connections 2024-08-30T09:03:50.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.864+0000 7f07ea858740 1 --2- 172.21.15.42:0/2090040634 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5609d6337390 0x5609d67003c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:50.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.864+0000 7f07ea858740 1 --2- 172.21.15.42:0/2090040634 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5609d6358070 0x5609d6700900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:50.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.864+0000 7f07ea858740 1 --2- 172.21.15.42:0/2090040634 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5609d632f1b0 0x5609d6714cc0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:50.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.864+0000 7f07ea858740 1 -- 172.21.15.42:0/2090040634 >> 172.21.15.42:0/2090040634 conn(0x5609d634c460 msgr2=0x5609d6707290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:03:50.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.864+0000 7f07ea858740 1 -- 172.21.15.42:0/2090040634 shutdown_connections 2024-08-30T09:03:50.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.864+0000 7f07ea858740 1 -- 172.21.15.42:0/2090040634 wait complete. 2024-08-30T09:03:50.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.864+0000 7f07ea858740 1 -- start start 2024-08-30T09:03:50.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.865+0000 7f07ea858740 1 Processor -- start 2024-08-30T09:03:50.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.865+0000 7f07ea858740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5609d632f1b0 0x5609d670d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:50.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.866+0000 7f07ea858740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5609d6337390 0x5609d670dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:50.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.866+0000 7f07ea858740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5609d6358070 0x5609d677e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:50.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.866+0000 7f07ea858740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f07dc1bde50 con 0x5609d632f1b0 2024-08-30T09:03:50.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.866+0000 7f07ea858740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f07dc1bf2c0 con 0x5609d6337390 2024-08-30T09:03:50.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.866+0000 7f07ea858740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f07dc1be520 con 0x5609d6358070 2024-08-30T09:03:50.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.866+0000 7f07ea858740 10 client.0 _ll_register_callbacks cb 0x7f07cc009990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:03:50.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.866+0000 7f07e2880640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5609d6358070 0x5609d677e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:03:50.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.866+0000 7f07e187e640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5609d6337390 0x5609d670dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:03:50.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.866+0000 7f07e187e640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5609d6337390 0x5609d670dbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:33102/0 (socket says 172.21.15.42:33102) 2024-08-30T09:03:50.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.866+0000 7f07e187e640 1 -- 172.21.15.42:0/2931410242 learned_addr learned my addr 172.21.15.42:0/2931410242 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:03:50.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.867+0000 7f07e187e640 1 -- 172.21.15.42:0/2931410242 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5609d6358070 msgr2=0x5609d677e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:03:50.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.867+0000 7f07e187e640 1 --2- 172.21.15.42:0/2931410242 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5609d6358070 0x5609d677e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:50.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.867+0000 7f07e187e640 1 -- 172.21.15.42:0/2931410242 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5609d632f1b0 msgr2=0x5609d670d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:03:50.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.867+0000 7f07e187e640 1 --2- 172.21.15.42:0/2931410242 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5609d632f1b0 0x5609d670d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:50.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.867+0000 7f07e187e640 1 -- 172.21.15.42:0/2931410242 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5609d677ed10 con 0x5609d6337390 2024-08-30T09:03:50.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.867+0000 7f07e187e640 1 --2- 172.21.15.42:0/2931410242 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5609d6337390 0x5609d670dbf0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f07cc0ac200 tx=0x7f07cc0cf2c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bc1f97368b856769 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:03:50.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.867+0000 7f07c1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-30T09:03:50.872 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.868+0000 7f07c1ffb640 1 -- 172.21.15.42:0/2931410242 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07cc0abc70 con 0x5609d6337390 2024-08-30T09:03:50.872 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.868+0000 7f07ea858740 1 -- 172.21.15.42:0/2931410242 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f07d4004cf0 con 0x5609d6337390 2024-08-30T09:03:50.872 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.868+0000 7f07c1ffb640 1 -- 172.21.15.42:0/2931410242 <== mon.2 v2:172.21.15.167:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f07cc004d70 con 0x5609d6337390 2024-08-30T09:03:50.872 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.869+0000 7f07c1ffb640 1 -- 172.21.15.42:0/2931410242 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f07cc0a83c0 con 0x5609d6337390 2024-08-30T09:03:50.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.870+0000 7f07ea858740 10 client.25670 fetch_fsmap learned FSMap version 29 2024-08-30T09:03:50.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.870+0000 7f07ea858740 1 -- 172.21.15.42:0/2931410242 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5609d6786020 con 0x5609d6337390 2024-08-30T09:03:50.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.871+0000 7f07c1ffb640 1 -- 172.21.15.42:0/2931410242 <== mon.2 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f07d4004cf0 con 0x5609d6337390 2024-08-30T09:03:50.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.871+0000 7f07c1ffb640 1 -- 172.21.15.42:0/2931410242 <== mon.2 v2:172.21.15.167:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f07cc0a7e70 con 0x5609d6337390 2024-08-30T09:03:50.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.871+0000 7f07ea858740 10 client.25670 fetch_fsmap finished waiting for FSMap version 29 2024-08-30T09:03:50.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.871+0000 7f07ea858740 10 client.25670 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-30T09:03:50.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.871+0000 7f07ea858740 20 client.25670 populate_metadata read hostname 'smithi042' 2024-08-30T09:03:50.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.871+0000 7f07ea858740 1 --2- 172.21.15.42:0/2931410242 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x5609d62ebda0 0x5609d67887f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:50.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.871+0000 7f07ea858740 4 client.25670 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-30T09:03:50.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.871+0000 7f07ea858740 1 -- 172.21.15.42:0/2931410242 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5609d6676580 con 0x5609d62ebda0 2024-08-30T09:03:50.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.871+0000 7f07e207f640 1 --2- 172.21.15.42:0/2931410242 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x5609d62ebda0 0x5609d67887f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:03:50.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.872+0000 7f07e207f640 1 --2- 172.21.15.42:0/2931410242 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x5609d62ebda0 0x5609d67887f0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=72f63e423d0bc4e4 server_cookie=602a14195474451c in_seq=0 out_seq=0 2024-08-30T09:03:50.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.872+0000 7f07c1ffb640 10 client.25670 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:03:50.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.874+0000 7f07c1ffb640 1 -- 172.21.15.42:0/2931410242 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+1226 (crc 0 0 0) 0x5609d6676580 con 0x5609d62ebda0 2024-08-30T09:03:50.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.874+0000 7f07c1ffb640 10 client.25670 handle_command_reply: tid=0 2024-08-30T09:03:50.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.874+0000 7f07ea858740 1 client.25670 shutdown 2024-08-30T09:03:50.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.874+0000 7f07ea858740 1 -- 172.21.15.42:0/2931410242 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5609d6337390 msgr2=0x5609d670dbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:03:50.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.874+0000 7f07ea858740 1 --2- 172.21.15.42:0/2931410242 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5609d6337390 0x5609d670dbf0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f07cc0ac200 tx=0x7f07cc0cf2c0 comp rx=0 tx=0).stop 2024-08-30T09:03:50.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.874+0000 7f07ea858740 1 -- 172.21.15.42:0/2931410242 shutdown_connections 2024-08-30T09:03:50.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.874+0000 7f07ea858740 1 --2- 172.21.15.42:0/2931410242 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x5609d62ebda0 0x5609d67887f0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:50.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.874+0000 7f07ea858740 1 --2- 172.21.15.42:0/2931410242 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5609d6358070 0x5609d677e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:50.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.874+0000 7f07ea858740 1 --2- 172.21.15.42:0/2931410242 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5609d6337390 0x5609d670dbf0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:50.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.874+0000 7f07c1ffb640 0 client.25670 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:03:50.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.874+0000 7f07ea858740 1 --2- 172.21.15.42:0/2931410242 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5609d632f1b0 0x5609d670d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:50.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.874+0000 7f07ea858740 1 -- 172.21.15.42:0/2931410242 >> 172.21.15.42:0/2931410242 conn(0x5609d634c460 msgr2=0x5609d66fe2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:03:50.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.874+0000 7f07ea858740 1 -- 172.21.15.42:0/2931410242 shutdown_connections 2024-08-30T09:03:50.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.874+0000 7f07ea858740 1 -- 172.21.15.42:0/2931410242 wait complete. 2024-08-30T09:03:50.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.874+0000 7f07ea858740 20 client.25670 trim_cache size 0 max 16384 2024-08-30T09:03:50.878 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-08-30T09:03:50.878 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "no active scrubs running", 2024-08-30T09:03:50.878 INFO:teuthology.orchestra.run.smithi042.stdout: "scrubs": {}, 2024-08-30T09:03:50.878 INFO:teuthology.orchestra.run.smithi042.stdout: "scrub_stats": { 2024-08-30T09:03:50.878 INFO:teuthology.orchestra.run.smithi042.stdout: "749f4867-0b70-49e5-af2d-8362a240ebc3": { 2024-08-30T09:03:50.878 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:03:16.386554+0000", 2024-08-30T09:03:50.878 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:03:50.878 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:03:50.878 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:03:50.878 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:03:50.878 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:03:50.879 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:03:50.879 INFO:teuthology.orchestra.run.smithi042.stdout: "85134e4b-ab4a-4885-b07e-a3ad01eaf13d": { 2024-08-30T09:03:50.879 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:01:27.738462+0000", 2024-08-30T09:03:50.879 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:03:50.879 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:03:50.879 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:03:50.879 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:03:50.879 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:03:50.879 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:03:50.879 INFO:teuthology.orchestra.run.smithi042.stdout: "c827ad7a-44ee-4e1d-9bb3-51fb7ab00c73": { 2024-08-30T09:03:50.879 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:02:21.686517+0000", 2024-08-30T09:03:50.879 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:03:50.879 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:03:50.880 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:03:50.880 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:03:50.880 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:03:50.880 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:03:50.880 INFO:teuthology.orchestra.run.smithi042.stdout: "5e2ed104-0625-478a-a78b-584e7f4c2817": { 2024-08-30T09:03:50.880 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:00:33.928400+0000", 2024-08-30T09:03:50.880 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:03:50.880 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:03:50.880 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:03:50.880 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:03:50.880 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:03:50.880 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:03:50.880 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:03:50.881 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-08-30T09:03:50.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.875+0000 7f07e42e3640 1 -- 172.21.15.42:0/2375333776 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f07c40b4bb0 msgr2=0x7f07c40b70c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:03:50.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.875+0000 7f07e42e3640 1 --2- 172.21.15.42:0/2375333776 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f07c40b4bb0 0x7f07c40b70c0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f07d4000c00 tx=0x7f07d4031400 comp rx=0 tx=0).stop 2024-08-30T09:03:50.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.875+0000 7f07e42e3640 1 -- 172.21.15.42:0/2375333776 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f07dc111ce0 msgr2=0x7f07dc1a95e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:03:50.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.875+0000 7f07e42e3640 1 --2- 172.21.15.42:0/2375333776 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f07dc111ce0 0x7f07dc1a95e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f07cc00eb90 tx=0x7f07cc00c6a0 comp rx=0 tx=0).stop 2024-08-30T09:03:50.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.875+0000 7f07e42e3640 1 -- 172.21.15.42:0/2375333776 shutdown_connections 2024-08-30T09:03:50.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.875+0000 7f07e42e3640 1 --2- 172.21.15.42:0/2375333776 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f07c40b4bb0 0x7f07c40b70c0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:50.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.875+0000 7f07e42e3640 1 --2- 172.21.15.42:0/2375333776 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f07dc11c9d0 0x7f07dc1ae880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:50.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.875+0000 7f07e42e3640 1 --2- 172.21.15.42:0/2375333776 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f07dc111ce0 0x7f07dc1a95e0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:50.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.875+0000 7f07e42e3640 1 --2- 172.21.15.42:0/2375333776 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f07dc10a780 0x7f07dc1a9040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:50.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.875+0000 7f07e42e3640 1 -- 172.21.15.42:0/2375333776 >> 172.21.15.42:0/2375333776 conn(0x7f07dc070c60 msgr2=0x7f07dc11e140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:03:50.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.875+0000 7f07e42e3640 1 -- 172.21.15.42:0/2375333776 shutdown_connections 2024-08-30T09:03:50.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:50.875+0000 7f07e42e3640 1 -- 172.21.15.42:0/2375333776 wait complete. 2024-08-30T09:03:50.953 INFO:teuthology.orchestra.run.smithi042.stderr:progress: 185.0 s, 1689.8 tps, lat 58.017 ms stddev 73.962 2024-08-30T09:03:51.967 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:51.965+0000 7f7d6549b640 1 -- 172.21.15.42:0/2260495626 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7d601033c0 msgr2=0x7f7d601037d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:03:51.967 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:51.965+0000 7f7d6549b640 1 --2- 172.21.15.42:0/2260495626 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7d601033c0 0x7f7d601037d0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f7d48009900 tx=0x7f7d4802a410 comp rx=0 tx=0).stop 2024-08-30T09:03:51.968 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:51.965+0000 7f7d6549b640 1 -- 172.21.15.42:0/2260495626 shutdown_connections 2024-08-30T09:03:51.968 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:51.965+0000 7f7d6549b640 1 --2- 172.21.15.42:0/2260495626 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7d60103d60 0x7f7d6010a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:51.968 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:51.965+0000 7f7d6549b640 1 --2- 172.21.15.42:0/2260495626 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7d601033c0 0x7f7d601037d0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:51.968 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:51.965+0000 7f7d6549b640 1 --2- 172.21.15.42:0/2260495626 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7d60102340 0x7f7d601027b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:51.968 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:51.965+0000 7f7d6549b640 1 -- 172.21.15.42:0/2260495626 >> 172.21.15.42:0/2260495626 conn(0x7f7d600fdc80 msgr2=0x7f7d60100130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:03:51.968 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:51.965+0000 7f7d6549b640 1 -- 172.21.15.42:0/2260495626 shutdown_connections 2024-08-30T09:03:51.968 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:51.965+0000 7f7d6549b640 1 -- 172.21.15.42:0/2260495626 wait complete. 2024-08-30T09:03:51.968 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:51.966+0000 7f7d6549b640 1 Processor -- start 2024-08-30T09:03:51.968 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:51.966+0000 7f7d6549b640 1 -- start start 2024-08-30T09:03:51.969 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:51.966+0000 7f7d6549b640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7d60102340 0x7f7d60198be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:51.969 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:51.966+0000 7f7d6549b640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7d601033c0 0x7f7d60199250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:51.969 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:51.966+0000 7f7d6549b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7d60103d60 0x7f7d60199e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:51.969 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:51.966+0000 7f7d6549b640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f7d6010dfc0 con 0x7f7d601033c0 2024-08-30T09:03:51.969 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:51.966+0000 7f7d6549b640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f7d6010de40 con 0x7f7d60102340 2024-08-30T09:03:51.969 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:51.966+0000 7f7d6549b640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f7d6010e140 con 0x7f7d60103d60 2024-08-30T09:03:51.970 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:51.967+0000 7f7d5effd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7d60102340 0x7f7d60198be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:03:51.970 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:51.967+0000 7f7d5effd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7d60102340 0x7f7d60198be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:33116/0 (socket says 172.21.15.42:33116) 2024-08-30T09:03:51.970 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:51.967+0000 7f7d5effd640 1 -- 172.21.15.42:0/1105859758 learned_addr learned my addr 172.21.15.42:0/1105859758 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:03:51.971 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:51.967+0000 7f7d5effd640 1 -- 172.21.15.42:0/1105859758 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7d60103d60 msgr2=0x7f7d60199e00 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-08-30T09:03:51.971 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:51.967+0000 7f7d5effd640 1 --2- 172.21.15.42:0/1105859758 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7d60103d60 0x7f7d60199e00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:51.971 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:51.967+0000 7f7d5effd640 1 -- 172.21.15.42:0/1105859758 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7d601033c0 msgr2=0x7f7d60199250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:03:51.971 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:51.967+0000 7f7d5effd640 1 --2- 172.21.15.42:0/1105859758 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7d601033c0 0x7f7d60199250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:51.971 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:51.967+0000 7f7d5effd640 1 -- 172.21.15.42:0/1105859758 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d6019a620 con 0x7f7d60102340 2024-08-30T09:03:51.971 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:51.968+0000 7f7d5effd640 1 --2- 172.21.15.42:0/1105859758 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7d60102340 0x7f7d60198be0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f7d5000ebc0 tx=0x7f7d5000d6b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:03:51.971 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:51.969+0000 7f7d57fff640 1 -- 172.21.15.42:0/1105859758 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d5001f070 con 0x7f7d60102340 2024-08-30T09:03:51.971 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:51.969+0000 7f7d57fff640 1 -- 172.21.15.42:0/1105859758 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d5000de60 con 0x7f7d60102340 2024-08-30T09:03:51.972 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:51.969+0000 7f7d57fff640 1 -- 172.21.15.42:0/1105859758 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d50023610 con 0x7f7d60102340 2024-08-30T09:03:51.972 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:51.970+0000 7f7d6549b640 1 -- 172.21.15.42:0/1105859758 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d6019a910 con 0x7f7d60102340 2024-08-30T09:03:51.972 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:51.970+0000 7f7d6549b640 1 -- 172.21.15.42:0/1105859758 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d6006ab60 con 0x7f7d60102340 2024-08-30T09:03:51.975 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:51.972+0000 7f7d6c40c740 1 Processor -- start 2024-08-30T09:03:51.976 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:51.972+0000 7f7d6c40c740 1 -- start start 2024-08-30T09:03:51.976 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:51.973+0000 7f7d6c40c740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c67fe481b0 0x55c6801ee180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:51.976 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:51.973+0000 7f7d6c40c740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c67fe1f2f0 0x55c6801ee7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:51.976 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:51.973+0000 7f7d6c40c740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c67fe274d0 0x55c6801ef340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:51.976 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:51.973+0000 7f7d6c40c740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55c680161000 con 0x55c67fe481b0 2024-08-30T09:03:51.976 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:51.973+0000 7f7d6c40c740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55c680150590 con 0x55c67fe274d0 2024-08-30T09:03:51.976 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:51.973+0000 7f7d6c40c740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55c680154540 con 0x55c67fe1f2f0 2024-08-30T09:03:51.976 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:51.973+0000 7f7d5f7fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c67fe481b0 0x55c6801ee180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:03:51.976 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:51.973+0000 7f7d5f7fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c67fe481b0 0x55c6801ee180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:55582/0 (socket says 172.21.15.42:55582) 2024-08-30T09:03:51.976 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:51.973+0000 7f7d5f7fe640 1 -- 172.21.15.42:0/4073931133 learned_addr learned my addr 172.21.15.42:0/4073931133 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:03:51.976 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:51.973+0000 7f7d5e7fc640 1 --2- 172.21.15.42:0/4073931133 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c67fe274d0 0x55c6801ef340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:03:51.976 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:51.973+0000 7f7d5f7fe640 1 -- 172.21.15.42:0/4073931133 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c67fe1f2f0 msgr2=0x55c6801ee7f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-30T09:03:51.976 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:51.973+0000 7f7d5f7fe640 1 --2- 172.21.15.42:0/4073931133 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c67fe1f2f0 0x55c6801ee7f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:51.977 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:51.973+0000 7f7d5f7fe640 1 -- 172.21.15.42:0/4073931133 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c67fe274d0 msgr2=0x55c6801ef340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:03:51.977 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:51.973+0000 7f7d5f7fe640 1 --2- 172.21.15.42:0/4073931133 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c67fe274d0 0x55c6801ef340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:51.977 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:51.973+0000 7f7d5f7fe640 1 -- 172.21.15.42:0/4073931133 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c67feb3790 con 0x55c67fe481b0 2024-08-30T09:03:51.977 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:51.973+0000 7f7d57fff640 1 -- 172.21.15.42:0/1105859758 <== mon.2 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7f7d50022070 con 0x7f7d60102340 2024-08-30T09:03:51.977 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:51.974+0000 7f7d5f7fe640 1 --2- 172.21.15.42:0/4073931133 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c67fe481b0 0x55c6801ee180 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f7d58009b70 tx=0x7f7d5800e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=782049103eba53bc server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:03:51.977 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:51.974+0000 7f7d557fa640 1 -- 172.21.15.42:0/4073931133 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d58009e60 con 0x55c67fe481b0 2024-08-30T09:03:51.977 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:51.974+0000 7f7d557fa640 1 -- 172.21.15.42:0/4073931133 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d580044d0 con 0x55c67fe481b0 2024-08-30T09:03:51.977 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:51.974+0000 7f7d557fa640 1 -- 172.21.15.42:0/4073931133 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d58005290 con 0x55c67fe481b0 2024-08-30T09:03:51.978 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:51.975+0000 7f7d57fff640 1 --2- 172.21.15.42:0/1105859758 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f7d4c0b4c80 0x7f7d4c0b7190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:51.978 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:51.975+0000 7f7d5effd640 1 --2- 172.21.15.42:0/1105859758 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f7d4c0b4c80 0x7f7d4c0b7190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:03:51.978 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:51.975+0000 7f7d57fff640 1 -- 172.21.15.42:0/1105859758 <== mon.2 v2:172.21.15.167:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f7d500ed810 con 0x7f7d60102340 2024-08-30T09:03:51.979 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:51.975+0000 7f7d6c40c740 1 -- 172.21.15.42:0/4073931133 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c67fe481b0 msgr2=0x55c6801ee180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:03:51.979 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:51.975+0000 7f7d6c40c740 1 --2- 172.21.15.42:0/4073931133 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c67fe481b0 0x55c6801ee180 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f7d58009b70 tx=0x7f7d5800e720 comp rx=0 tx=0).stop 2024-08-30T09:03:51.979 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:51.975+0000 7f7d6c40c740 1 -- 172.21.15.42:0/4073931133 shutdown_connections 2024-08-30T09:03:51.979 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:51.975+0000 7f7d6c40c740 1 --2- 172.21.15.42:0/4073931133 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c67fe1f2f0 0x55c6801ee7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:51.979 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:51.975+0000 7f7d6c40c740 1 --2- 172.21.15.42:0/4073931133 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c67fe274d0 0x55c6801ef340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:51.979 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:51.975+0000 7f7d6c40c740 1 --2- 172.21.15.42:0/4073931133 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c67fe481b0 0x55c6801ee180 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:51.979 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:51.975+0000 7f7d6c40c740 1 -- 172.21.15.42:0/4073931133 >> 172.21.15.42:0/4073931133 conn(0x55c67fe3c300 msgr2=0x55c6801e9c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:03:51.979 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:51.975+0000 7f7d5effd640 1 --2- 172.21.15.42:0/1105859758 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f7d4c0b4c80 0x7f7d4c0b7190 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f7d5000dbb0 tx=0x7f7d500a8780 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:03:51.979 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:51.976+0000 7f7d6c40c740 1 -- 172.21.15.42:0/4073931133 shutdown_connections 2024-08-30T09:03:51.979 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:51.976+0000 7f7d6c40c740 1 -- 172.21.15.42:0/4073931133 wait complete. 2024-08-30T09:03:51.979 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:51.976+0000 7f7d6c40c740 1 -- start start 2024-08-30T09:03:51.979 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:51.976+0000 7f7d6c40c740 1 Processor -- start 2024-08-30T09:03:51.980 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:51.977+0000 7f7d6c40c740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c67fe1f2f0 0x55c6801f7cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:51.980 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:51.977+0000 7f7d6c40c740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c67fe274d0 0x55c6801f8380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:51.980 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:51.977+0000 7f7d6c40c740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c67fe481b0 0x55c6801f8e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:51.980 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:51.977+0000 7f7d6c40c740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55c680089ab0 con 0x55c67fe274d0 2024-08-30T09:03:51.980 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:51.977+0000 7f7d6c40c740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55c6800d7680 con 0x55c67fe1f2f0 2024-08-30T09:03:51.980 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:51.977+0000 7f7d6c40c740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55c6801efe60 con 0x55c67fe481b0 2024-08-30T09:03:51.981 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:51.977+0000 7f7d6c40c740 10 client.0 _ll_register_callbacks cb 0x55c67fe5fa00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:03:51.981 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:51.977+0000 7f7d5f7fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c67fe1f2f0 0x55c6801f7cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:03:51.981 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:51.977+0000 7f7d5f7fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c67fe1f2f0 0x55c6801f7cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:33122/0 (socket says 172.21.15.42:33122) 2024-08-30T09:03:51.981 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:51.977+0000 7f7d5f7fe640 1 -- 172.21.15.42:0/84145321 learned_addr learned my addr 172.21.15.42:0/84145321 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:03:51.982 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:51.980+0000 7f7d5f7fe640 1 --2- 172.21.15.42:0/84145321 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c67fe481b0 0x55c6801f8e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:03:51.987 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:51.986+0000 7f7d5f7fe640 1 -- 172.21.15.42:0/84145321 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c67fe481b0 msgr2=0x55c6801f8e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:03:51.987 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:51.986+0000 7f7d5f7fe640 1 --2- 172.21.15.42:0/84145321 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c67fe481b0 0x55c6801f8e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:51.988 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:51.986+0000 7f7d5f7fe640 1 -- 172.21.15.42:0/84145321 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c67fe274d0 msgr2=0x55c6801f8380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:03:51.988 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:51.986+0000 7f7d5f7fe640 1 --2- 172.21.15.42:0/84145321 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c67fe274d0 0x55c6801f8380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:51.988 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:51.986+0000 7f7d5f7fe640 1 -- 172.21.15.42:0/84145321 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c67ff2c620 con 0x55c67fe1f2f0 2024-08-30T09:03:51.990 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:51.988+0000 7f7d5f7fe640 1 --2- 172.21.15.42:0/84145321 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c67fe1f2f0 0x55c6801f7cc0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f7d580095f0 tx=0x7f7d58005d70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e425d00fcc69cf23 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:03:51.990 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:51.988+0000 7f7d467fc640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-30T09:03:51.991 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:51.990+0000 7f7d467fc640 1 -- 172.21.15.42:0/84145321 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d58004fe0 con 0x55c67fe1f2f0 2024-08-30T09:03:51.991 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:51.990+0000 7f7d467fc640 1 -- 172.21.15.42:0/84145321 <== mon.2 v2:172.21.15.167:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f7d58005180 con 0x55c67fe1f2f0 2024-08-30T09:03:51.993 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:51.991+0000 7f7d6c40c740 1 -- 172.21.15.42:0/84145321 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55c680162f50 con 0x55c67fe1f2f0 2024-08-30T09:03:51.994 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:51.993+0000 7f7d467fc640 1 -- 172.21.15.42:0/84145321 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f7d580053a0 con 0x55c67fe1f2f0 2024-08-30T09:03:51.996 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:51.994+0000 7f7d6c40c740 10 client.25682 fetch_fsmap learned FSMap version 29 2024-08-30T09:03:51.996 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:51.994+0000 7f7d6c40c740 1 -- 172.21.15.42:0/84145321 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c680201dd0 con 0x55c67fe1f2f0 2024-08-30T09:03:51.998 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:51.996+0000 7f7d467fc640 1 -- 172.21.15.42:0/84145321 <== mon.2 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55c680162f50 con 0x55c67fe1f2f0 2024-08-30T09:03:51.998 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:51.996+0000 7f7d467fc640 1 -- 172.21.15.42:0/84145321 <== mon.2 v2:172.21.15.167:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f7d58018470 con 0x55c67fe1f2f0 2024-08-30T09:03:52.003 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:52.000+0000 7f7d6c40c740 10 client.25682 fetch_fsmap finished waiting for FSMap version 29 2024-08-30T09:03:52.003 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:52.000+0000 7f7d6c40c740 10 client.25682 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-30T09:03:52.003 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:52.000+0000 7f7d6c40c740 20 client.25682 populate_metadata read hostname 'smithi042' 2024-08-30T09:03:52.003 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:52.000+0000 7f7d6c40c740 1 --2- 172.21.15.42:0/84145321 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55c67fe06d80 0x55c680205250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:52.004 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:52.000+0000 7f7d6c40c740 4 client.25682 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-30T09:03:52.004 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:52.000+0000 7f7d6c40c740 1 -- 172.21.15.42:0/84145321 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c68015e9f0 con 0x55c67fe06d80 2024-08-30T09:03:52.007 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:52.004+0000 7f7d5e7fc640 1 --2- 172.21.15.42:0/84145321 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55c67fe06d80 0x55c680205250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:03:52.021 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:52.019+0000 7f7d5e7fc640 1 --2- 172.21.15.42:0/84145321 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55c67fe06d80 0x55c680205250 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ff0b4bd48128fe0 server_cookie=817f9264b6cc0eed in_seq=0 out_seq=0 2024-08-30T09:03:52.021 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:52.020+0000 7f7d467fc640 10 client.25682 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:03:52.025 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:52.024+0000 7f7d467fc640 1 -- 172.21.15.42:0/84145321 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55c68015e9f0 con 0x55c67fe06d80 2024-08-30T09:03:52.025 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:52.024+0000 7f7d467fc640 10 client.25682 handle_command_reply: tid=0 2024-08-30T09:03:52.027 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:52.025+0000 7f7d6c40c740 1 client.25682 shutdown 2024-08-30T09:03:52.031 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:52.028+0000 7f7d6c40c740 1 -- 172.21.15.42:0/84145321 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c67fe1f2f0 msgr2=0x55c6801f7cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:03:52.031 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:52.029+0000 7f7d6c40c740 1 --2- 172.21.15.42:0/84145321 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c67fe1f2f0 0x55c6801f7cc0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f7d580095f0 tx=0x7f7d58005d70 comp rx=0 tx=0).stop 2024-08-30T09:03:52.034 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:52.032+0000 7f7d6c40c740 1 -- 172.21.15.42:0/84145321 shutdown_connections 2024-08-30T09:03:52.035 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:52.032+0000 7f7d6c40c740 1 --2- 172.21.15.42:0/84145321 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55c67fe06d80 0x55c680205250 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:52.035 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:52.032+0000 7f7d6c40c740 1 --2- 172.21.15.42:0/84145321 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c67fe481b0 0x55c6801f8e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:52.035 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:52.032+0000 7f7d6c40c740 1 --2- 172.21.15.42:0/84145321 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c67fe274d0 0x55c6801f8380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:52.035 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:52.032+0000 7f7d6c40c740 1 --2- 172.21.15.42:0/84145321 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c67fe1f2f0 0x55c6801f7cc0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:52.035 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:52.032+0000 7f7d6c40c740 1 -- 172.21.15.42:0/84145321 >> 172.21.15.42:0/84145321 conn(0x55c67fe3c300 msgr2=0x55c6801e8170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:03:52.035 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:52.033+0000 7f7d467fc640 0 client.25682 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:03:52.042 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:52.039+0000 7f7d6c40c740 1 -- 172.21.15.42:0/84145321 shutdown_connections 2024-08-30T09:03:52.045 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:52.042+0000 7f7d6c40c740 1 -- 172.21.15.42:0/84145321 wait complete. 2024-08-30T09:03:52.048 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:52.046+0000 7f7d6c40c740 20 client.25682 trim_cache size 0 max 16384 2024-08-30T09:03:52.084 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:52.082+0000 7f7d6c40c740 1 Processor -- start 2024-08-30T09:03:52.088 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:52.086+0000 7f7d6c40c740 1 -- start start 2024-08-30T09:03:52.088 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:52.086+0000 7f7d6c40c740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c67fe1f2f0 0x55c6801e8920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:52.088 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:52.086+0000 7f7d6c40c740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c67fe274d0 0x55c6801e8e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:52.088 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:52.086+0000 7f7d6c40c740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c67fe481b0 0x55c6801f7300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:52.088 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:52.086+0000 7f7d6c40c740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f7d601accd0 con 0x55c67fe1f2f0 2024-08-30T09:03:52.088 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:52.086+0000 7f7d6c40c740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f7d601aa5a0 con 0x55c67fe274d0 2024-08-30T09:03:52.088 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:52.086+0000 7f7d6c40c740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f7d601a83f0 con 0x55c67fe481b0 2024-08-30T09:03:52.091 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:52.089+0000 7f7d5f7fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c67fe481b0 0x55c6801f7300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:03:52.091 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:52.090+0000 7f7d5f7fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c67fe481b0 0x55c6801f7300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:60950/0 (socket says 172.21.15.42:60950) 2024-08-30T09:03:52.092 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:52.090+0000 7f7d5f7fe640 1 -- 172.21.15.42:0/3499977816 learned_addr learned my addr 172.21.15.42:0/3499977816 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:03:52.092 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:52.091+0000 7f7d5e7fc640 1 --2- 172.21.15.42:0/3499977816 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c67fe274d0 0x55c6801e8e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:03:52.093 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:52.092+0000 7f7d5f7fe640 1 -- 172.21.15.42:0/3499977816 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c67fe274d0 msgr2=0x55c6801e8e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:03:52.094 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:52.092+0000 7f7d5f7fe640 1 --2- 172.21.15.42:0/3499977816 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c67fe274d0 0x55c6801e8e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:52.094 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:52.092+0000 7f7d5f7fe640 1 -- 172.21.15.42:0/3499977816 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c67fe1f2f0 msgr2=0x55c6801e8920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:03:52.094 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:52.092+0000 7f7d5f7fe640 1 --2- 172.21.15.42:0/3499977816 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c67fe1f2f0 0x55c6801e8920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:52.094 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:52.092+0000 7f7d5f7fe640 1 -- 172.21.15.42:0/3499977816 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c6801f91b0 con 0x55c67fe481b0 2024-08-30T09:03:52.096 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:52.095+0000 7f7d5f7fe640 1 --2- 172.21.15.42:0/3499977816 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c67fe481b0 0x55c6801f7300 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f7d5800ef80 tx=0x7f7d58000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5c5369e05f4be1b2 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:03:52.100 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:52.098+0000 7f7d54ff9640 1 -- 172.21.15.42:0/3499977816 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d58005b80 con 0x55c67fe481b0 2024-08-30T09:03:52.100 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:52.098+0000 7f7d54ff9640 1 -- 172.21.15.42:0/3499977816 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d58005540 con 0x55c67fe481b0 2024-08-30T09:03:52.100 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:52.099+0000 7f7d54ff9640 1 -- 172.21.15.42:0/3499977816 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d58003640 con 0x55c67fe481b0 2024-08-30T09:03:52.105 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:52.102+0000 7f7d6c40c740 1 -- 172.21.15.42:0/3499977816 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c67fe481b0 msgr2=0x55c6801f7300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:03:52.105 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:52.102+0000 7f7d6c40c740 1 --2- 172.21.15.42:0/3499977816 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c67fe481b0 0x55c6801f7300 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f7d5800ef80 tx=0x7f7d58000bc0 comp rx=0 tx=0).stop 2024-08-30T09:03:52.106 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:52.104+0000 7f7d6c40c740 1 -- 172.21.15.42:0/3499977816 shutdown_connections 2024-08-30T09:03:52.106 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:52.104+0000 7f7d6c40c740 1 --2- 172.21.15.42:0/3499977816 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c67fe481b0 0x55c6801f7300 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:52.106 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:52.104+0000 7f7d6c40c740 1 --2- 172.21.15.42:0/3499977816 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c67fe274d0 0x55c6801e8e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:52.106 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:52.104+0000 7f7d6c40c740 1 --2- 172.21.15.42:0/3499977816 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c67fe1f2f0 0x55c6801e8920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:52.106 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:52.104+0000 7f7d6c40c740 1 -- 172.21.15.42:0/3499977816 >> 172.21.15.42:0/3499977816 conn(0x55c67fe3c300 msgr2=0x55c6801f2bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:03:52.111 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:52.108+0000 7f7d6c40c740 1 -- 172.21.15.42:0/3499977816 shutdown_connections 2024-08-30T09:03:52.113 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:52.111+0000 7f7d6c40c740 1 -- 172.21.15.42:0/3499977816 wait complete. 2024-08-30T09:03:52.113 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:52.111+0000 7f7d6c40c740 1 -- start start 2024-08-30T09:03:52.113 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:52.112+0000 7f7d6c40c740 1 Processor -- start 2024-08-30T09:03:52.117 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:52.115+0000 7f7d6c40c740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c67fe1f2f0 0x55c6801ffa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:52.117 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:52.115+0000 7f7d6c40c740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c67fe274d0 0x55c6801fe740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:52.117 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:52.115+0000 7f7d6c40c740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c67fe481b0 0x55c6801feca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:52.117 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:52.115+0000 7f7d6c40c740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f7d601a8a60 con 0x55c67fe1f2f0 2024-08-30T09:03:52.117 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:52.115+0000 7f7d6c40c740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f7d601a9ed0 con 0x55c67fe481b0 2024-08-30T09:03:52.117 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:52.115+0000 7f7d6c40c740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f7d601a9130 con 0x55c67fe274d0 2024-08-30T09:03:52.117 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:52.115+0000 7f7d6c40c740 10 client.0 _ll_register_callbacks cb 0x7f7d58005e20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:03:52.118 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:52.116+0000 7f7d5e7fc640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c67fe274d0 0x55c6801fe740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:03:52.118 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:52.116+0000 7f7d5e7fc640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c67fe274d0 0x55c6801fe740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:60964/0 (socket says 172.21.15.42:60964) 2024-08-30T09:03:52.118 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:52.116+0000 7f7d5e7fc640 1 -- 172.21.15.42:0/3866580369 learned_addr learned my addr 172.21.15.42:0/3866580369 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:03:52.118 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:52.117+0000 7f7d5e7fc640 1 -- 172.21.15.42:0/3866580369 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c67fe481b0 msgr2=0x55c6801feca0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-30T09:03:52.119 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:52.117+0000 7f7d5e7fc640 1 --2- 172.21.15.42:0/3866580369 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c67fe481b0 0x55c6801feca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:52.119 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:52.117+0000 7f7d5e7fc640 1 -- 172.21.15.42:0/3866580369 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c67fe1f2f0 msgr2=0x55c6801ffa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:03:52.119 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:52.117+0000 7f7d5e7fc640 1 --2- 172.21.15.42:0/3866580369 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c67fe1f2f0 0x55c6801ffa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:52.119 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:52.117+0000 7f7d5e7fc640 1 -- 172.21.15.42:0/3866580369 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c6801ff430 con 0x55c67fe274d0 2024-08-30T09:03:52.119 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:52.117+0000 7f7d5e7fc640 1 --2- 172.21.15.42:0/3866580369 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c67fe274d0 0x55c6801fe740 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f7d4802a930 tx=0x7f7d4802a9b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2df6381047dd1711 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:03:52.120 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:52.117+0000 7f7d467fc640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-08-30T09:03:52.120 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:52.117+0000 7f7d467fc640 1 -- 172.21.15.42:0/3866580369 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d48035040 con 0x55c67fe274d0 2024-08-30T09:03:52.120 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:52.118+0000 7f7d6c40c740 1 -- 172.21.15.42:0/3866580369 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7d58005540 con 0x55c67fe274d0 2024-08-30T09:03:52.120 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:52.118+0000 7f7d467fc640 1 -- 172.21.15.42:0/3866580369 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f7d48005040 con 0x55c67fe274d0 2024-08-30T09:03:52.120 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:52.118+0000 7f7d467fc640 1 -- 172.21.15.42:0/3866580369 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f7d48052050 con 0x55c67fe274d0 2024-08-30T09:03:52.120 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:52.118+0000 7f7d6c40c740 10 client.25699 fetch_fsmap learned FSMap version 29 2024-08-30T09:03:52.120 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:52.118+0000 7f7d6c40c740 1 -- 172.21.15.42:0/3866580369 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c68026fd00 con 0x55c67fe274d0 2024-08-30T09:03:52.121 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:52.118+0000 7f7d467fc640 1 -- 172.21.15.42:0/3866580369 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d58005540 con 0x55c67fe274d0 2024-08-30T09:03:52.121 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:52.119+0000 7f7d467fc640 1 -- 172.21.15.42:0/3866580369 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f7d48031640 con 0x55c67fe274d0 2024-08-30T09:03:52.121 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:52.119+0000 7f7d6c40c740 10 client.25699 fetch_fsmap finished waiting for FSMap version 29 2024-08-30T09:03:52.121 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:52.119+0000 7f7d6c40c740 10 client.25699 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-30T09:03:52.121 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:52.119+0000 7f7d6c40c740 20 client.25699 populate_metadata read hostname 'smithi042' 2024-08-30T09:03:52.121 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:52.119+0000 7f7d6c40c740 1 --2- 172.21.15.42:0/3866580369 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55c67fddc1c0 0x55c680272770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:52.121 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:52.119+0000 7f7d6c40c740 4 client.25699 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-30T09:03:52.121 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:52.119+0000 7f7d6c40c740 1 -- 172.21.15.42:0/3866580369 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c6801e8710 con 0x55c67fddc1c0 2024-08-30T09:03:52.122 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:52.120+0000 7f7d5f7fe640 1 --2- 172.21.15.42:0/3866580369 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55c67fddc1c0 0x55c680272770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:03:52.122 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:52.120+0000 7f7d5f7fe640 1 --2- 172.21.15.42:0/3866580369 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55c67fddc1c0 0x55c680272770 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=34857a557c2fd663 server_cookie=5b27b387aa5cf54 in_seq=0 out_seq=0 2024-08-30T09:03:52.122 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:52.120+0000 7f7d467fc640 10 client.25699 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:03:52.158 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:52.156+0000 7f7d467fc640 1 -- 172.21.15.42:0/3866580369 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c6801e8710 con 0x55c67fddc1c0 2024-08-30T09:03:52.158 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:52.156+0000 7f7d467fc640 10 client.25699 handle_command_reply: tid=0 2024-08-30T09:03:52.158 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:52.156+0000 7f7d6c40c740 1 client.25699 shutdown 2024-08-30T09:03:52.159 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:52.156+0000 7f7d6c40c740 1 -- 172.21.15.42:0/3866580369 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c67fe274d0 msgr2=0x55c6801fe740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:03:52.159 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:52.156+0000 7f7d6c40c740 1 --2- 172.21.15.42:0/3866580369 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c67fe274d0 0x55c6801fe740 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f7d4802a930 tx=0x7f7d4802a9b0 comp rx=0 tx=0).stop 2024-08-30T09:03:52.159 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:52.156+0000 7f7d6c40c740 1 -- 172.21.15.42:0/3866580369 shutdown_connections 2024-08-30T09:03:52.159 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:52.156+0000 7f7d6c40c740 1 --2- 172.21.15.42:0/3866580369 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55c67fddc1c0 0x55c680272770 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:52.159 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:52.156+0000 7f7d6c40c740 1 --2- 172.21.15.42:0/3866580369 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c67fe274d0 0x55c6801fe740 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:52.159 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:52.156+0000 7f7d6c40c740 1 --2- 172.21.15.42:0/3866580369 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c67fe481b0 0x55c6801feca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:52.159 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:52.156+0000 7f7d6c40c740 1 --2- 172.21.15.42:0/3866580369 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c67fe1f2f0 0x55c6801ffa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:52.159 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:52.156+0000 7f7d6c40c740 1 -- 172.21.15.42:0/3866580369 >> 172.21.15.42:0/3866580369 conn(0x55c67fe3c300 msgr2=0x55c6801e88c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:03:52.159 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:52.156+0000 7f7d467fc640 0 client.25699 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:03:52.159 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:52.156+0000 7f7d6c40c740 1 -- 172.21.15.42:0/3866580369 shutdown_connections 2024-08-30T09:03:52.159 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:52.156+0000 7f7d6c40c740 1 -- 172.21.15.42:0/3866580369 wait complete. 2024-08-30T09:03:52.160 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:52.156+0000 7f7d6c40c740 20 client.25699 trim_cache size 0 max 16384 2024-08-30T09:03:52.160 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-08-30T09:03:52.160 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2024-08-30T09:03:52.160 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-08-30T09:03:52.160 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-08-30T09:03:52.161 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:52.157+0000 7f7d6549b640 1 -- 172.21.15.42:0/1105859758 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f7d4c0b4c80 msgr2=0x7f7d4c0b7190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:03:52.161 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:52.157+0000 7f7d6549b640 1 --2- 172.21.15.42:0/1105859758 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f7d4c0b4c80 0x7f7d4c0b7190 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f7d5000dbb0 tx=0x7f7d500a8780 comp rx=0 tx=0).stop 2024-08-30T09:03:52.161 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:52.157+0000 7f7d6549b640 1 -- 172.21.15.42:0/1105859758 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7d60102340 msgr2=0x7f7d60198be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:03:52.161 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:52.157+0000 7f7d6549b640 1 --2- 172.21.15.42:0/1105859758 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7d60102340 0x7f7d60198be0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f7d5000ebc0 tx=0x7f7d5000d6b0 comp rx=0 tx=0).stop 2024-08-30T09:03:52.161 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:52.157+0000 7f7d6549b640 1 -- 172.21.15.42:0/1105859758 shutdown_connections 2024-08-30T09:03:52.161 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:52.157+0000 7f7d6549b640 1 --2- 172.21.15.42:0/1105859758 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f7d4c0b4c80 0x7f7d4c0b7190 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:52.161 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:52.157+0000 7f7d6549b640 1 --2- 172.21.15.42:0/1105859758 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7d60103d60 0x7f7d60199e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:52.161 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:52.157+0000 7f7d6549b640 1 --2- 172.21.15.42:0/1105859758 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7d601033c0 0x7f7d60199250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:52.161 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:52.157+0000 7f7d6549b640 1 --2- 172.21.15.42:0/1105859758 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7d60102340 0x7f7d60198be0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:52.161 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:52.157+0000 7f7d6549b640 1 -- 172.21.15.42:0/1105859758 >> 172.21.15.42:0/1105859758 conn(0x7f7d600fdc80 msgr2=0x7f7d601066c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:03:52.162 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:52.157+0000 7f7d6549b640 1 -- 172.21.15.42:0/1105859758 shutdown_connections 2024-08-30T09:03:52.162 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:03:52.157+0000 7f7d6549b640 1 -- 172.21.15.42:0/1105859758 wait complete. 2024-08-30T09:03:52.576 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:03:52 smithi167 ceph-mon[22063]: pgmap v466: 129 pgs: 129 active+clean; 1.1 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 MiB/s wr, 844 op/s 2024-08-30T09:03:52.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:03:52 smithi042 ceph-mon[18652]: pgmap v466: 129 pgs: 129 active+clean; 1.1 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 MiB/s wr, 844 op/s 2024-08-30T09:03:52.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:03:52 smithi190 ceph-mon[22001]: pgmap v466: 129 pgs: 129 active+clean; 1.1 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 MiB/s wr, 844 op/s 2024-08-30T09:03:52.802 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-08-30T09:03:52.802 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph fs dump --format=json 2024-08-30T09:03:53.272 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T09:03:54.297 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:54.291+0000 7f5d4c986640 1 -- 172.21.15.42:0/4181531162 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5d4811c9d0 msgr2=0x7f5d4811eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:03:54.297 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:54.291+0000 7f5d4c986640 1 --2- 172.21.15.42:0/4181531162 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5d4811c9d0 0x7f5d4811eee0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f5d38009900 tx=0x7f5d3802a440 comp rx=0 tx=0).stop 2024-08-30T09:03:54.297 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:54.292+0000 7f5d4c986640 1 -- 172.21.15.42:0/4181531162 shutdown_connections 2024-08-30T09:03:54.297 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:54.292+0000 7f5d4c986640 1 --2- 172.21.15.42:0/4181531162 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5d4810a780 0x7f5d4810ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:54.297 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:54.292+0000 7f5d4c986640 1 --2- 172.21.15.42:0/4181531162 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5d4811c9d0 0x7f5d4811eee0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:54.297 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:54.292+0000 7f5d4c986640 1 --2- 172.21.15.42:0/4181531162 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5d48111ce0 0x7f5d4810a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:54.298 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:54.292+0000 7f5d4c986640 1 -- 172.21.15.42:0/4181531162 >> 172.21.15.42:0/4181531162 conn(0x7f5d48070c60 msgr2=0x7f5d48071070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:03:54.298 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:54.292+0000 7f5d4c986640 1 -- 172.21.15.42:0/4181531162 shutdown_connections 2024-08-30T09:03:54.298 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:54.292+0000 7f5d4c986640 1 -- 172.21.15.42:0/4181531162 wait complete. 2024-08-30T09:03:54.298 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:54.293+0000 7f5d4c986640 1 Processor -- start 2024-08-30T09:03:54.298 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:54.293+0000 7f5d4c986640 1 -- start start 2024-08-30T09:03:54.298 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:54.293+0000 7f5d4c986640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5d4810a780 0x7f5d48075b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:54.298 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:54.293+0000 7f5d4c986640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5d48111ce0 0x7f5d48076080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:54.298 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:54.293+0000 7f5d4c986640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5d4811c9d0 0x7f5d4807b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:54.298 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:54.293+0000 7f5d4c986640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f5d481226b0 con 0x7f5d48111ce0 2024-08-30T09:03:54.298 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:54.293+0000 7f5d4c986640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f5d48122530 con 0x7f5d4811c9d0 2024-08-30T09:03:54.298 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:54.293+0000 7f5d4c986640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f5d48122830 con 0x7f5d4810a780 2024-08-30T09:03:54.298 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:54.294+0000 7f5d46d9d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5d4811c9d0 0x7f5d4807b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:03:54.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:54.294+0000 7f5d46d9d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5d4811c9d0 0x7f5d4807b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:33166/0 (socket says 172.21.15.42:33166) 2024-08-30T09:03:54.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:54.294+0000 7f5d46d9d640 1 -- 172.21.15.42:0/1283125458 learned_addr learned my addr 172.21.15.42:0/1283125458 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:03:54.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:54.295+0000 7f5d46d9d640 1 -- 172.21.15.42:0/1283125458 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5d4810a780 msgr2=0x7f5d48075b00 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-30T09:03:54.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:54.295+0000 7f5d46d9d640 1 --2- 172.21.15.42:0/1283125458 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5d4810a780 0x7f5d48075b00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:54.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:54.295+0000 7f5d46d9d640 1 -- 172.21.15.42:0/1283125458 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5d48111ce0 msgr2=0x7f5d48076080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:03:54.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:54.295+0000 7f5d46d9d640 1 --2- 172.21.15.42:0/1283125458 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5d48111ce0 0x7f5d48076080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:54.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:54.295+0000 7f5d46d9d640 1 -- 172.21.15.42:0/1283125458 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d4807ba70 con 0x7f5d4811c9d0 2024-08-30T09:03:54.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:54.295+0000 7f5d46d9d640 1 --2- 172.21.15.42:0/1283125458 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5d4811c9d0 0x7f5d4807b340 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f5d3802a9b0 tx=0x7f5d38003cd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:03:54.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:54.296+0000 7f5d377fe640 1 -- 172.21.15.42:0/1283125458 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d3803f070 con 0x7f5d4811c9d0 2024-08-30T09:03:54.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:54.296+0000 7f5d377fe640 1 -- 172.21.15.42:0/1283125458 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d3803bb10 con 0x7f5d4811c9d0 2024-08-30T09:03:54.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:54.296+0000 7f5d377fe640 1 -- 172.21.15.42:0/1283125458 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d38035030 con 0x7f5d4811c9d0 2024-08-30T09:03:54.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:54.297+0000 7f5d4c986640 1 -- 172.21.15.42:0/1283125458 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d4807bca0 con 0x7f5d4811c9d0 2024-08-30T09:03:54.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:54.297+0000 7f5d4c986640 1 -- 172.21.15.42:0/1283125458 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d4807c070 con 0x7f5d4811c9d0 2024-08-30T09:03:54.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:54.302+0000 7f5d377fe640 1 -- 172.21.15.42:0/1283125458 <== mon.2 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7f5d38042070 con 0x7f5d4811c9d0 2024-08-30T09:03:54.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:54.303+0000 7f5d377fe640 1 --2- 172.21.15.42:0/1283125458 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f5d300b4d40 0x7f5d300b7250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:54.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:54.303+0000 7f5d377fe640 1 -- 172.21.15.42:0/1283125458 <== mon.2 v2:172.21.15.167:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f5d38105e00 con 0x7f5d4811c9d0 2024-08-30T09:03:54.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:54.304+0000 7f5d357fa640 1 -- 172.21.15.42:0/1283125458 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d4810ab90 con 0x7f5d4811c9d0 2024-08-30T09:03:54.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:54.309+0000 7f5d377fe640 1 -- 172.21.15.42:0/1283125458 <== mon.2 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f5d380ccb40 con 0x7f5d4811c9d0 2024-08-30T09:03:54.313 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:54.312+0000 7f5d4659c640 1 --2- 172.21.15.42:0/1283125458 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f5d300b4d40 0x7f5d300b7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:03:54.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:54.313+0000 7f5d4659c640 1 --2- 172.21.15.42:0/1283125458 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f5d300b4d40 0x7f5d300b7250 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f5d4807c3d0 tx=0x7f5d3c00b370 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:03:54.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:03:54 smithi167 ceph-mon[22063]: pgmap v467: 129 pgs: 129 active+clean; 1.1 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 13 MiB/s wr, 965 op/s 2024-08-30T09:03:54.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:03:54 smithi042 ceph-mon[18652]: pgmap v467: 129 pgs: 129 active+clean; 1.1 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 13 MiB/s wr, 965 op/s 2024-08-30T09:03:54.661 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:54.659+0000 7f5d357fa640 1 -- 172.21.15.42:0/1283125458 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5d48063e30 con 0x7f5d4811c9d0 2024-08-30T09:03:54.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:54.661+0000 7f5d377fe640 1 -- 172.21.15.42:0/1283125458 <== mon.2 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11918 (secure 0 0 0) 0x7f5d380d5080 con 0x7f5d4811c9d0 2024-08-30T09:03:54.663 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-08-30T09:03:54.663 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":29,"btime":"2024-08-30T09:00:48:414225+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14703,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/3188400000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":3188400000},{"type":"v1","addr":"172.21.15.42:6835","nonce":3188400000}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14715,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/3356537674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":3356537674},{"type":"v1","addr":"172.21.15.42:6837","nonce":3356537674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14766,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6839/2107785513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6838","nonce":2107785513},{"type":"v1","addr":"172.21.15.42:6839","nonce":2107785513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14778,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6841/2298438674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6840","nonce":2298438674},{"type":"v1","addr":"172.21.15.42:6841","nonce":2298438674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24553,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/543555570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":543555570},{"type":"v1","addr":"172.21.15.190:6833","nonce":543555570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24554,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/3907818941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":3907818941},{"type":"v1","addr":"172.21.15.167:6833","nonce":3907818941}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-30T08:58:03.055084+0000","modified":"2024-08-30T09:00:47.637894+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24629,"mds_1":24610,"mds_2":24599},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24572":{"gid":24572,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6835/234596845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":234596845},{"type":"v1","addr":"172.21.15.167:6835","nonce":234596845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24592":{"gid":24592,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6835/247905709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":247905709},{"type":"v1","addr":"172.21.15.190:6835","nonce":247905709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24598":{"gid":24598,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6837/863256349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6836","nonce":863256349},{"type":"v1","addr":"172.21.15.190:6837","nonce":863256349}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24599":{"gid":24599,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.167:6837/3011837856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":3011837856},{"type":"v1","addr":"172.21.15.167:6837","nonce":3011837856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24610":{"gid":24610,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.190:6839/719981110","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6838","nonce":719981110},{"type":"v1","addr":"172.21.15.190:6839","nonce":719981110}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24629":{"gid":24629,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.167:6839/3604645207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":3604645207},{"type":"v1","addr":"172.21.15.167:6839","nonce":3604645207}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24629,"qdb_cluster":[24599,24610,24629]},"id":1}]} 2024-08-30T09:03:54.664 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 29 2024-08-30T09:03:54.667 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:54.665+0000 7f5d357fa640 1 -- 172.21.15.42:0/1283125458 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f5d300b4d40 msgr2=0x7f5d300b7250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:03:54.667 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:54.665+0000 7f5d357fa640 1 --2- 172.21.15.42:0/1283125458 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f5d300b4d40 0x7f5d300b7250 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f5d4807c3d0 tx=0x7f5d3c00b370 comp rx=0 tx=0).stop 2024-08-30T09:03:54.667 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:54.665+0000 7f5d357fa640 1 -- 172.21.15.42:0/1283125458 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5d4811c9d0 msgr2=0x7f5d4807b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:03:54.667 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:54.665+0000 7f5d357fa640 1 --2- 172.21.15.42:0/1283125458 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5d4811c9d0 0x7f5d4807b340 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f5d3802a9b0 tx=0x7f5d38003cd0 comp rx=0 tx=0).stop 2024-08-30T09:03:54.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:54.666+0000 7f5d357fa640 1 -- 172.21.15.42:0/1283125458 shutdown_connections 2024-08-30T09:03:54.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:54.666+0000 7f5d357fa640 1 --2- 172.21.15.42:0/1283125458 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f5d300b4d40 0x7f5d300b7250 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:54.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:54.666+0000 7f5d357fa640 1 --2- 172.21.15.42:0/1283125458 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5d4811c9d0 0x7f5d4807b340 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:54.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:54.666+0000 7f5d357fa640 1 --2- 172.21.15.42:0/1283125458 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5d48111ce0 0x7f5d48076080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:54.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:54.666+0000 7f5d357fa640 1 --2- 172.21.15.42:0/1283125458 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5d4810a780 0x7f5d48075b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:54.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:54.666+0000 7f5d357fa640 1 -- 172.21.15.42:0/1283125458 >> 172.21.15.42:0/1283125458 conn(0x7f5d48070c60 msgr2=0x7f5d4811cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:03:54.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:54.667+0000 7f5d357fa640 1 -- 172.21.15.42:0/1283125458 shutdown_connections 2024-08-30T09:03:54.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:54.668+0000 7f5d357fa640 1 -- 172.21.15.42:0/1283125458 wait complete. 2024-08-30T09:03:54.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:03:54 smithi190 ceph-mon[22001]: pgmap v467: 129 pgs: 129 active+clean; 1.1 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 13 MiB/s wr, 965 op/s 2024-08-30T09:03:55.546 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:03:55 smithi042 ceph-mon[18652]: from='client.? 172.21.15.42:0/1283125458' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-30T09:03:55.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:03:55 smithi167 ceph-mon[22063]: from='client.? 172.21.15.42:0/1283125458' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-30T09:03:55.582 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-08-30T09:03:55.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:03:55 smithi190 ceph-mon[22001]: from='client.? 172.21.15.42:0/1283125458' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-30T09:03:55.954 INFO:teuthology.orchestra.run.smithi042.stderr:progress: 190.0 s, 769.7 tps, lat 126.801 ms stddev 158.053 2024-08-30T09:03:56.000 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T09:03:56.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:03:56 smithi042 ceph-mon[18652]: pgmap v468: 129 pgs: 129 active+clean; 1.1 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 MiB/s wr, 903 op/s 2024-08-30T09:03:56.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.531+0000 7f5068a98640 1 -- 172.21.15.42:0/2958915986 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f50641088a0 msgr2=0x7f506406ac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:03:56.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.531+0000 7f5068a98640 1 --2- 172.21.15.42:0/2958915986 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f50641088a0 0x7f506406ac00 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f5058009900 tx=0x7f505802a440 comp rx=0 tx=0).stop 2024-08-30T09:03:56.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.533+0000 7f5068a98640 1 -- 172.21.15.42:0/2958915986 shutdown_connections 2024-08-30T09:03:56.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.533+0000 7f5068a98640 1 --2- 172.21.15.42:0/2958915986 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f50641088a0 0x7f506406ac00 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:56.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.533+0000 7f5068a98640 1 --2- 172.21.15.42:0/2958915986 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5064107f50 0x7f5064108360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:56.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.533+0000 7f5068a98640 1 --2- 172.21.15.42:0/2958915986 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5064102610 0x7f5064102a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:56.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.533+0000 7f5068a98640 1 -- 172.21.15.42:0/2958915986 >> 172.21.15.42:0/2958915986 conn(0x7f50640fe060 msgr2=0x7f50641004b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:03:56.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.534+0000 7f5068a98640 1 -- 172.21.15.42:0/2958915986 shutdown_connections 2024-08-30T09:03:56.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.534+0000 7f5068a98640 1 -- 172.21.15.42:0/2958915986 wait complete. 2024-08-30T09:03:56.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.535+0000 7f5068a98640 1 Processor -- start 2024-08-30T09:03:56.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.535+0000 7f5068a98640 1 -- start start 2024-08-30T09:03:56.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.535+0000 7f5068a98640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5064102610 0x7f5064074be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:56.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.535+0000 7f5068a98640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5064107f50 0x7f506406deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:56.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.535+0000 7f5068a98640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f50641088a0 0x7f506406e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:56.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.535+0000 7f5068a98640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f5064111550 con 0x7f5064107f50 2024-08-30T09:03:56.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.535+0000 7f5068a98640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f50641113d0 con 0x7f50641088a0 2024-08-30T09:03:56.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.535+0000 7f5068a98640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f50641116d0 con 0x7f5064102610 2024-08-30T09:03:56.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.536+0000 7f5063fff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f50641088a0 0x7f506406e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:03:56.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.536+0000 7f5063fff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f50641088a0 0x7f506406e410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:33184/0 (socket says 172.21.15.42:33184) 2024-08-30T09:03:56.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.536+0000 7f5063fff640 1 -- 172.21.15.42:0/2992542533 learned_addr learned my addr 172.21.15.42:0/2992542533 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:03:56.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.537+0000 7f50637fe640 1 --2- 172.21.15.42:0/2992542533 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5064102610 0x7f5064074be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:03:56.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.537+0000 7f5063fff640 1 -- 172.21.15.42:0/2992542533 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5064102610 msgr2=0x7f5064074be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:03:56.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.537+0000 7f5063fff640 1 --2- 172.21.15.42:0/2992542533 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5064102610 0x7f5064074be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:56.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.537+0000 7f5063fff640 1 -- 172.21.15.42:0/2992542533 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5064107f50 msgr2=0x7f506406deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:03:56.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.537+0000 7f5063fff640 1 --2- 172.21.15.42:0/2992542533 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5064107f50 0x7f506406deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:56.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.537+0000 7f5063fff640 1 -- 172.21.15.42:0/2992542533 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f506406ec30 con 0x7f50641088a0 2024-08-30T09:03:56.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.538+0000 7f5063fff640 1 --2- 172.21.15.42:0/2992542533 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f50641088a0 0x7f506406e410 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f505802a9b0 tx=0x7f5058003b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:03:56.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.538+0000 7f5060ff9640 1 -- 172.21.15.42:0/2992542533 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f505803f070 con 0x7f50641088a0 2024-08-30T09:03:56.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.538+0000 7f5060ff9640 1 -- 172.21.15.42:0/2992542533 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5058031b50 con 0x7f50641088a0 2024-08-30T09:03:56.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.538+0000 7f5060ff9640 1 -- 172.21.15.42:0/2992542533 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5058035030 con 0x7f50641088a0 2024-08-30T09:03:56.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.538+0000 7f5068a98640 1 -- 172.21.15.42:0/2992542533 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5064073850 con 0x7f50641088a0 2024-08-30T09:03:56.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.538+0000 7f5068a98640 1 -- 172.21.15.42:0/2992542533 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5064073ce0 con 0x7f50641088a0 2024-08-30T09:03:56.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.542+0000 7f507026f740 1 Processor -- start 2024-08-30T09:03:56.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.543+0000 7f5060ff9640 1 -- 172.21.15.42:0/2992542533 <== mon.2 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7f5058042070 con 0x7f50641088a0 2024-08-30T09:03:56.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.544+0000 7f5060ff9640 1 --2- 172.21.15.42:0/2992542533 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f504c0b4c70 0x7f504c0b7180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:56.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.545+0000 7f5060ff9640 1 -- 172.21.15.42:0/2992542533 <== mon.2 v2:172.21.15.167:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f5058105be0 con 0x7f50641088a0 2024-08-30T09:03:56.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.547+0000 7f507026f740 1 -- start start 2024-08-30T09:03:56.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.547+0000 7f507026f740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562469330070 0x5624696d54b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:56.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.547+0000 7f507026f740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5624693071b0 0x5624696d59f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:56.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.547+0000 7f507026f740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56246930f390 0x5624696d5f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:56.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.547+0000 7f507026f740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x56246964c680 con 0x56246930f390 2024-08-30T09:03:56.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.547+0000 7f507026f740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x56246963bc10 con 0x5624693071b0 2024-08-30T09:03:56.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.547+0000 7f507026f740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x56246963fbc0 con 0x562469330070 2024-08-30T09:03:56.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.548+0000 7f50637fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562469330070 0x5624696d54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:03:56.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.548+0000 7f50637fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562469330070 0x5624696d54b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:60990/0 (socket says 172.21.15.42:60990) 2024-08-30T09:03:56.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.548+0000 7f50637fe640 1 -- 172.21.15.42:0/1568233191 learned_addr learned my addr 172.21.15.42:0/1568233191 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:03:56.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.548+0000 7f5063fff640 1 --2- 172.21.15.42:0/1568233191 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56246930f390 0x5624696d5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:03:56.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.548+0000 7f50637fe640 1 -- 172.21.15.42:0/1568233191 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5624693071b0 msgr2=0x5624696d59f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-30T09:03:56.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.548+0000 7f50637fe640 1 --2- 172.21.15.42:0/1568233191 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5624693071b0 0x5624696d59f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:56.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.548+0000 7f50637fe640 1 -- 172.21.15.42:0/1568233191 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56246930f390 msgr2=0x5624696d5f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:03:56.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.548+0000 7f50637fe640 1 --2- 172.21.15.42:0/1568233191 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56246930f390 0x5624696d5f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:56.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.548+0000 7f50637fe640 1 -- 172.21.15.42:0/1568233191 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56246939b650 con 0x562469330070 2024-08-30T09:03:56.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.549+0000 7f50637fe640 1 --2- 172.21.15.42:0/1568233191 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562469330070 0x5624696d54b0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f5054009a40 tx=0x7f5054009f00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=457fecd98d2f1142 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:03:56.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.549+0000 7f5062ffd640 1 --2- 172.21.15.42:0/2992542533 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f504c0b4c70 0x7f504c0b7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:03:56.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.549+0000 7f5049ffb640 1 -- 172.21.15.42:0/1568233191 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5054017070 con 0x562469330070 2024-08-30T09:03:56.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.550+0000 7f5049ffb640 1 -- 172.21.15.42:0/1568233191 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5054013470 con 0x562469330070 2024-08-30T09:03:56.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.550+0000 7f5049ffb640 1 -- 172.21.15.42:0/1568233191 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5054012610 con 0x562469330070 2024-08-30T09:03:56.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.551+0000 7f507026f740 1 -- 172.21.15.42:0/1568233191 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562469330070 msgr2=0x5624696d54b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:03:56.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.551+0000 7f5062ffd640 1 --2- 172.21.15.42:0/2992542533 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f504c0b4c70 0x7f504c0b7180 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x5624696d4130 tx=0x7f505c005bd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:03:56.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.551+0000 7f507026f740 1 --2- 172.21.15.42:0/1568233191 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562469330070 0x5624696d54b0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f5054009a40 tx=0x7f5054009f00 comp rx=0 tx=0).stop 2024-08-30T09:03:56.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.551+0000 7f507026f740 1 -- 172.21.15.42:0/1568233191 shutdown_connections 2024-08-30T09:03:56.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.551+0000 7f507026f740 1 --2- 172.21.15.42:0/1568233191 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56246930f390 0x5624696d5f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:56.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.551+0000 7f507026f740 1 --2- 172.21.15.42:0/1568233191 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5624693071b0 0x5624696d59f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:56.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.551+0000 7f507026f740 1 --2- 172.21.15.42:0/1568233191 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562469330070 0x5624696d54b0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:56.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.551+0000 7f507026f740 1 -- 172.21.15.42:0/1568233191 >> 172.21.15.42:0/1568233191 conn(0x562469324460 msgr2=0x5624696d33a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:03:56.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.551+0000 7f507026f740 1 -- 172.21.15.42:0/1568233191 shutdown_connections 2024-08-30T09:03:56.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.551+0000 7f507026f740 1 -- 172.21.15.42:0/1568233191 wait complete. 2024-08-30T09:03:56.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.552+0000 7f507026f740 1 -- start start 2024-08-30T09:03:56.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.552+0000 7f507026f740 1 Processor -- start 2024-08-30T09:03:56.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.553+0000 7f507026f740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5624693071b0 0x5624696e0b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:56.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.553+0000 7f507026f740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56246930f390 0x5624696e1060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:56.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.553+0000 7f507026f740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562469330070 0x5624696ed070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:56.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.553+0000 7f507026f740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x56246949e700 con 0x5624693071b0 2024-08-30T09:03:56.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.553+0000 7f507026f740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5624695c2750 con 0x56246930f390 2024-08-30T09:03:56.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.553+0000 7f507026f740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5624696e3d30 con 0x562469330070 2024-08-30T09:03:56.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.553+0000 7f507026f740 10 client.0 _ll_register_callbacks cb 0x5624695f1e10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:03:56.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.554+0000 7f5063fff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562469330070 0x5624696ed070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:03:56.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.554+0000 7f5063fff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562469330070 0x5624696ed070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:60992/0 (socket says 172.21.15.42:60992) 2024-08-30T09:03:56.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.554+0000 7f5063fff640 1 -- 172.21.15.42:0/3832127605 learned_addr learned my addr 172.21.15.42:0/3832127605 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:03:56.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.554+0000 7f5062ffd640 1 --2- 172.21.15.42:0/3832127605 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56246930f390 0x5624696e1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:03:56.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.554+0000 7f5063fff640 1 -- 172.21.15.42:0/3832127605 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56246930f390 msgr2=0x5624696e1060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:03:56.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.554+0000 7f5063fff640 1 --2- 172.21.15.42:0/3832127605 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56246930f390 0x5624696e1060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:56.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.554+0000 7f5063fff640 1 -- 172.21.15.42:0/3832127605 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5624693071b0 msgr2=0x5624696e0b20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-30T09:03:56.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.554+0000 7f5063fff640 1 --2- 172.21.15.42:0/3832127605 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5624693071b0 0x5624696e0b20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:56.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.554+0000 7f5063fff640 1 -- 172.21.15.42:0/3832127605 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5624695fb900 con 0x562469330070 2024-08-30T09:03:56.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.555+0000 7f5063fff640 1 --2- 172.21.15.42:0/3832127605 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562469330070 0x5624696ed070 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f5058005ab0 tx=0x7f50580cb990 comp rx=0 tx=0).ready entity=mon.1 client_cookie=606a29f332292462 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:03:56.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.555+0000 7f5042ffd640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-08-30T09:03:56.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.555+0000 7f5042ffd640 1 -- 172.21.15.42:0/3832127605 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50580cc220 con 0x562469330070 2024-08-30T09:03:56.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.555+0000 7f507026f740 1 -- 172.21.15.42:0/3832127605 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5054013470 con 0x562469330070 2024-08-30T09:03:56.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.555+0000 7f5042ffd640 1 -- 172.21.15.42:0/3832127605 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f50580c84b0 con 0x562469330070 2024-08-30T09:03:56.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.556+0000 7f5042ffd640 1 -- 172.21.15.42:0/3832127605 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f50580d0a90 con 0x562469330070 2024-08-30T09:03:56.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.556+0000 7f507026f740 10 client.25708 fetch_fsmap learned FSMap version 29 2024-08-30T09:03:56.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.556+0000 7f507026f740 1 -- 172.21.15.42:0/3832127605 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5624696efa50 con 0x562469330070 2024-08-30T09:03:56.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.557+0000 7f5042ffd640 1 -- 172.21.15.42:0/3832127605 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5054013470 con 0x562469330070 2024-08-30T09:03:56.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.557+0000 7f5042ffd640 1 -- 172.21.15.42:0/3832127605 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f50580e1b30 con 0x562469330070 2024-08-30T09:03:56.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.557+0000 7f507026f740 10 client.25708 fetch_fsmap finished waiting for FSMap version 29 2024-08-30T09:03:56.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.557+0000 7f507026f740 10 client.25708 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-08-30T09:03:56.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.557+0000 7f507026f740 20 client.25708 populate_metadata read hostname 'smithi042' 2024-08-30T09:03:56.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.557+0000 7f507026f740 1 --2- 172.21.15.42:0/3832127605 >> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] conn(0x5624692ee9f0 0x5624696f3110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:56.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.557+0000 7f507026f740 4 client.25708 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-30T09:03:56.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.557+0000 7f507026f740 1 -- 172.21.15.42:0/3832127605 --> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56246964e5d0 con 0x5624692ee9f0 2024-08-30T09:03:56.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.557+0000 7f50637fe640 1 --2- 172.21.15.42:0/3832127605 >> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] conn(0x5624692ee9f0 0x5624696f3110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:03:56.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.558+0000 7f5042ffd640 10 client.25708 ms_handle_connect on v2:172.21.15.167:6836/3011837856 2024-08-30T09:03:56.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.558+0000 7f50637fe640 1 --2- 172.21.15.42:0/3832127605 >> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] conn(0x5624692ee9f0 0x5624696f3110 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f1ff6c6ba85423f0 server_cookie=7d03f44c29298c18 in_seq=0 out_seq=0 2024-08-30T09:03:56.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.559+0000 7f5042ffd640 1 -- 172.21.15.42:0/3832127605 <== mds.2 v2:172.21.15.167:6836/3011837856 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56246964e5d0 con 0x5624692ee9f0 2024-08-30T09:03:56.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.559+0000 7f5042ffd640 10 client.25708 handle_command_reply: tid=0 2024-08-30T09:03:56.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.559+0000 7f507026f740 1 client.25708 shutdown 2024-08-30T09:03:56.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.559+0000 7f507026f740 1 -- 172.21.15.42:0/3832127605 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562469330070 msgr2=0x5624696ed070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:03:56.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.559+0000 7f507026f740 1 --2- 172.21.15.42:0/3832127605 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562469330070 0x5624696ed070 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f5058005ab0 tx=0x7f50580cb990 comp rx=0 tx=0).stop 2024-08-30T09:03:56.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.559+0000 7f507026f740 1 -- 172.21.15.42:0/3832127605 shutdown_connections 2024-08-30T09:03:56.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.559+0000 7f507026f740 1 --2- 172.21.15.42:0/3832127605 >> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] conn(0x5624692ee9f0 0x5624696f3110 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:56.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.559+0000 7f507026f740 1 --2- 172.21.15.42:0/3832127605 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562469330070 0x5624696ed070 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:56.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.559+0000 7f507026f740 1 --2- 172.21.15.42:0/3832127605 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56246930f390 0x5624696e1060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:56.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.559+0000 7f507026f740 1 --2- 172.21.15.42:0/3832127605 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5624693071b0 0x5624696e0b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:56.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.559+0000 7f507026f740 1 -- 172.21.15.42:0/3832127605 >> 172.21.15.42:0/3832127605 conn(0x562469324460 msgr2=0x5624696d8640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:03:56.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.559+0000 7f5042ffd640 0 client.25708 ms_handle_reset on v2:172.21.15.167:6836/3011837856 2024-08-30T09:03:56.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.560+0000 7f507026f740 1 -- 172.21.15.42:0/3832127605 shutdown_connections 2024-08-30T09:03:56.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.560+0000 7f507026f740 1 -- 172.21.15.42:0/3832127605 wait complete. 2024-08-30T09:03:56.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.560+0000 7f507026f740 20 client.25708 trim_cache size 0 max 16384 2024-08-30T09:03:56.576 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:03:56 smithi167 ceph-mon[22063]: pgmap v468: 129 pgs: 129 active+clean; 1.1 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 MiB/s wr, 903 op/s 2024-08-30T09:03:56.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.577+0000 7f507026f740 1 Processor -- start 2024-08-30T09:03:56.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.577+0000 7f507026f740 1 -- start start 2024-08-30T09:03:56.580 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.577+0000 7f507026f740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5624693071b0 0x5624696d3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:56.580 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.578+0000 7f507026f740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56246930f390 0x5624696ecf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:56.580 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.578+0000 7f50637fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5624693071b0 0x5624696d3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:03:56.580 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.578+0000 7f50637fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5624693071b0 0x5624696d3410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:33208/0 (socket says 172.21.15.42:33208) 2024-08-30T09:03:56.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.578+0000 7f50637fe640 1 -- 172.21.15.42:0/814473487 learned_addr learned my addr 172.21.15.42:0/814473487 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:03:56.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.578+0000 7f507026f740 1 --2- 172.21.15.42:0/814473487 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562469330070 0x5624696ddd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:56.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.578+0000 7f5062ffd640 1 --2- 172.21.15.42:0/814473487 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56246930f390 0x5624696ecf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:03:56.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.578+0000 7f507026f740 1 -- 172.21.15.42:0/814473487 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f50641ca700 con 0x56246930f390 2024-08-30T09:03:56.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.578+0000 7f507026f740 1 -- 172.21.15.42:0/814473487 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f50641c7fd0 con 0x5624693071b0 2024-08-30T09:03:56.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.578+0000 7f507026f740 1 -- 172.21.15.42:0/814473487 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f506406f1b0 con 0x562469330070 2024-08-30T09:03:56.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.578+0000 7f5063fff640 1 --2- 172.21.15.42:0/814473487 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562469330070 0x5624696ddd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:03:56.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.578+0000 7f50637fe640 1 -- 172.21.15.42:0/814473487 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562469330070 msgr2=0x5624696ddd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:03:56.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.578+0000 7f50637fe640 1 --2- 172.21.15.42:0/814473487 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562469330070 0x5624696ddd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:56.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.578+0000 7f50637fe640 1 -- 172.21.15.42:0/814473487 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56246930f390 msgr2=0x5624696ecf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:03:56.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.578+0000 7f50637fe640 1 --2- 172.21.15.42:0/814473487 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56246930f390 0x5624696ecf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:56.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.578+0000 7f50637fe640 1 -- 172.21.15.42:0/814473487 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5624696d8310 con 0x5624693071b0 2024-08-30T09:03:56.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.579+0000 7f50637fe640 1 --2- 172.21.15.42:0/814473487 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5624693071b0 0x5624696d3410 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f505400a010 tx=0x7f5054009f00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2d87c79a24643284 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:03:56.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.579+0000 7f50497fa640 1 -- 172.21.15.42:0/814473487 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5054017040 con 0x5624693071b0 2024-08-30T09:03:56.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.579+0000 7f50497fa640 1 -- 172.21.15.42:0/814473487 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5054020070 con 0x5624693071b0 2024-08-30T09:03:56.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.579+0000 7f50497fa640 1 -- 172.21.15.42:0/814473487 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f505401c6b0 con 0x5624693071b0 2024-08-30T09:03:56.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.580+0000 7f507026f740 1 -- 172.21.15.42:0/814473487 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5624693071b0 msgr2=0x5624696d3410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:03:56.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.580+0000 7f507026f740 1 --2- 172.21.15.42:0/814473487 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5624693071b0 0x5624696d3410 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f505400a010 tx=0x7f5054009f00 comp rx=0 tx=0).stop 2024-08-30T09:03:56.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.580+0000 7f507026f740 1 -- 172.21.15.42:0/814473487 shutdown_connections 2024-08-30T09:03:56.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.580+0000 7f507026f740 1 --2- 172.21.15.42:0/814473487 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562469330070 0x5624696ddd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:56.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.580+0000 7f507026f740 1 --2- 172.21.15.42:0/814473487 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56246930f390 0x5624696ecf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:56.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.580+0000 7f507026f740 1 --2- 172.21.15.42:0/814473487 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5624693071b0 0x5624696d3410 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:56.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.580+0000 7f507026f740 1 -- 172.21.15.42:0/814473487 >> 172.21.15.42:0/814473487 conn(0x562469324460 msgr2=0x5624696d2d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:03:56.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.580+0000 7f507026f740 1 -- 172.21.15.42:0/814473487 shutdown_connections 2024-08-30T09:03:56.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.580+0000 7f507026f740 1 -- 172.21.15.42:0/814473487 wait complete. 2024-08-30T09:03:56.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.580+0000 7f507026f740 1 -- start start 2024-08-30T09:03:56.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.581+0000 7f507026f740 1 Processor -- start 2024-08-30T09:03:56.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.581+0000 7f507026f740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5624693071b0 0x5624696e9940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:56.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.581+0000 7f507026f740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56246930f390 0x5624696ddbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:56.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.581+0000 7f507026f740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562469330070 0x5624696de2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:56.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.581+0000 7f507026f740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f506411dc70 con 0x56246930f390 2024-08-30T09:03:56.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.581+0000 7f507026f740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f50641c7900 con 0x562469330070 2024-08-30T09:03:56.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.581+0000 7f507026f740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f50641c6b60 con 0x5624693071b0 2024-08-30T09:03:56.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.581+0000 7f507026f740 10 client.0 _ll_register_callbacks cb 0x7f5058009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:03:56.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.581+0000 7f50637fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5624693071b0 0x5624696e9940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:03:56.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.581+0000 7f50637fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5624693071b0 0x5624696e9940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:32784/0 (socket says 172.21.15.42:32784) 2024-08-30T09:03:56.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.581+0000 7f50637fe640 1 -- 172.21.15.42:0/3787085938 learned_addr learned my addr 172.21.15.42:0/3787085938 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:03:56.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.581+0000 7f5063fff640 1 --2- 172.21.15.42:0/3787085938 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562469330070 0x5624696de2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:03:56.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.581+0000 7f5062ffd640 1 --2- 172.21.15.42:0/3787085938 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56246930f390 0x5624696ddbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:03:56.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.582+0000 7f50637fe640 1 -- 172.21.15.42:0/3787085938 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562469330070 msgr2=0x5624696de2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:03:56.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.582+0000 7f50637fe640 1 --2- 172.21.15.42:0/3787085938 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562469330070 0x5624696de2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:56.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.582+0000 7f50637fe640 1 -- 172.21.15.42:0/3787085938 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56246930f390 msgr2=0x5624696ddbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:03:56.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.582+0000 7f50637fe640 1 --2- 172.21.15.42:0/3787085938 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56246930f390 0x5624696ddbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:56.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.582+0000 7f50637fe640 1 -- 172.21.15.42:0/3787085938 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5624696e8580 con 0x5624693071b0 2024-08-30T09:03:56.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.582+0000 7f5063fff640 1 --2- 172.21.15.42:0/3787085938 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562469330070 0x5624696de2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-30T09:03:56.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.582+0000 7f50637fe640 1 --2- 172.21.15.42:0/3787085938 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5624693071b0 0x5624696e9940 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f5054013f90 tx=0x7f5054009750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dda812ea94472cc9 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:03:56.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.582+0000 7f5042ffd640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-08-30T09:03:56.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.582+0000 7f5042ffd640 1 -- 172.21.15.42:0/3787085938 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f505402f020 con 0x5624693071b0 2024-08-30T09:03:56.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.582+0000 7f507026f740 1 -- 172.21.15.42:0/3787085938 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5054020070 con 0x5624693071b0 2024-08-30T09:03:56.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.582+0000 7f5042ffd640 1 -- 172.21.15.42:0/3787085938 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f5054013670 con 0x5624693071b0 2024-08-30T09:03:56.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.582+0000 7f5042ffd640 1 -- 172.21.15.42:0/3787085938 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f5054017040 con 0x5624693071b0 2024-08-30T09:03:56.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.582+0000 7f507026f740 10 client.25714 fetch_fsmap learned FSMap version 29 2024-08-30T09:03:56.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.582+0000 7f507026f740 1 -- 172.21.15.42:0/3787085938 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56246975dc40 con 0x5624693071b0 2024-08-30T09:03:56.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.583+0000 7f5042ffd640 1 -- 172.21.15.42:0/3787085938 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5054020070 con 0x5624693071b0 2024-08-30T09:03:56.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.583+0000 7f5042ffd640 1 -- 172.21.15.42:0/3787085938 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f5054006380 con 0x5624693071b0 2024-08-30T09:03:56.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.583+0000 7f507026f740 10 client.25714 fetch_fsmap finished waiting for FSMap version 29 2024-08-30T09:03:56.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.583+0000 7f507026f740 10 client.25714 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-08-30T09:03:56.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.583+0000 7f507026f740 20 client.25714 populate_metadata read hostname 'smithi042' 2024-08-30T09:03:56.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.583+0000 7f507026f740 1 --2- 172.21.15.42:0/3787085938 >> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] conn(0x5624692c3da0 0x562469760030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:03:56.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.583+0000 7f507026f740 4 client.25714 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-30T09:03:56.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.583+0000 7f507026f740 1 -- 172.21.15.42:0/3787085938 --> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56246964a070 con 0x5624692c3da0 2024-08-30T09:03:56.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.583+0000 7f5062ffd640 1 --2- 172.21.15.42:0/3787085938 >> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] conn(0x5624692c3da0 0x562469760030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:03:56.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.583+0000 7f5042ffd640 10 client.25714 ms_handle_connect on v2:172.21.15.167:6836/3011837856 2024-08-30T09:03:56.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.584+0000 7f5062ffd640 1 --2- 172.21.15.42:0/3787085938 >> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] conn(0x5624692c3da0 0x562469760030 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=333af7727babd672 server_cookie=436e86a90cdcb39e in_seq=0 out_seq=0 2024-08-30T09:03:56.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.584+0000 7f5042ffd640 1 -- 172.21.15.42:0/3787085938 <== mds.2 v2:172.21.15.167:6836/3011837856 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56246964a070 con 0x5624692c3da0 2024-08-30T09:03:56.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.584+0000 7f5042ffd640 10 client.25714 handle_command_reply: tid=0 2024-08-30T09:03:56.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.584+0000 7f507026f740 1 client.25714 shutdown 2024-08-30T09:03:56.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.585+0000 7f507026f740 1 -- 172.21.15.42:0/3787085938 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5624693071b0 msgr2=0x5624696e9940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:03:56.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.585+0000 7f507026f740 1 --2- 172.21.15.42:0/3787085938 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5624693071b0 0x5624696e9940 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f5054013f90 tx=0x7f5054009750 comp rx=0 tx=0).stop 2024-08-30T09:03:56.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.585+0000 7f507026f740 1 -- 172.21.15.42:0/3787085938 shutdown_connections 2024-08-30T09:03:56.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.585+0000 7f507026f740 1 --2- 172.21.15.42:0/3787085938 >> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] conn(0x5624692c3da0 0x562469760030 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:56.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.585+0000 7f507026f740 1 --2- 172.21.15.42:0/3787085938 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562469330070 0x5624696de2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:56.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.585+0000 7f507026f740 1 --2- 172.21.15.42:0/3787085938 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56246930f390 0x5624696ddbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:56.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.585+0000 7f507026f740 1 --2- 172.21.15.42:0/3787085938 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5624693071b0 0x5624696e9940 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:56.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.585+0000 7f507026f740 1 -- 172.21.15.42:0/3787085938 >> 172.21.15.42:0/3787085938 conn(0x562469324460 msgr2=0x5624696d1810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:03:56.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.585+0000 7f5042ffd640 0 client.25714 ms_handle_reset on v2:172.21.15.167:6836/3011837856 2024-08-30T09:03:56.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.585+0000 7f507026f740 1 -- 172.21.15.42:0/3787085938 shutdown_connections 2024-08-30T09:03:56.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.585+0000 7f507026f740 1 -- 172.21.15.42:0/3787085938 wait complete. 2024-08-30T09:03:56.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.585+0000 7f507026f740 20 client.25714 trim_cache size 0 max 16384 2024-08-30T09:03:56.591 INFO:teuthology.orchestra.run.smithi042.stdout:[] 2024-08-30T09:03:56.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.586+0000 7f5068a98640 1 -- 172.21.15.42:0/2992542533 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f504c0b4c70 msgr2=0x7f504c0b7180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:03:56.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.586+0000 7f5068a98640 1 --2- 172.21.15.42:0/2992542533 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f504c0b4c70 0x7f504c0b7180 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x5624696d4130 tx=0x7f505c005bd0 comp rx=0 tx=0).stop 2024-08-30T09:03:56.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.586+0000 7f5068a98640 1 -- 172.21.15.42:0/2992542533 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f50641088a0 msgr2=0x7f506406e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:03:56.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.586+0000 7f5068a98640 1 --2- 172.21.15.42:0/2992542533 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f50641088a0 0x7f506406e410 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f505802a9b0 tx=0x7f5058003b90 comp rx=0 tx=0).stop 2024-08-30T09:03:56.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.586+0000 7f5068a98640 1 -- 172.21.15.42:0/2992542533 shutdown_connections 2024-08-30T09:03:56.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.586+0000 7f5068a98640 1 --2- 172.21.15.42:0/2992542533 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f504c0b4c70 0x7f504c0b7180 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:56.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.586+0000 7f5068a98640 1 --2- 172.21.15.42:0/2992542533 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f50641088a0 0x7f506406e410 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:56.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.586+0000 7f5068a98640 1 --2- 172.21.15.42:0/2992542533 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5064107f50 0x7f506406deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:56.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.586+0000 7f5068a98640 1 --2- 172.21.15.42:0/2992542533 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5064102610 0x7f5064074be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:03:56.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.586+0000 7f5068a98640 1 -- 172.21.15.42:0/2992542533 >> 172.21.15.42:0/2992542533 conn(0x7f50640fe060 msgr2=0x7f5064105d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:03:56.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.586+0000 7f5068a98640 1 -- 172.21.15.42:0/2992542533 shutdown_connections 2024-08-30T09:03:56.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:56.586+0000 7f5068a98640 1 -- 172.21.15.42:0/2992542533 wait complete. 2024-08-30T09:03:56.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:03:56 smithi190 ceph-mon[22001]: pgmap v468: 129 pgs: 129 active+clean; 1.1 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 MiB/s wr, 903 op/s 2024-08-30T09:03:57.464 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-08-30T09:03:57.889 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T09:03:58.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:03:58 smithi167 ceph-mon[22063]: pgmap v469: 129 pgs: 129 active+clean; 1.1 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 15 MiB/s wr, 946 op/s 2024-08-30T09:03:58.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:03:58 smithi042 ceph-mon[18652]: pgmap v469: 129 pgs: 129 active+clean; 1.1 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 15 MiB/s wr, 946 op/s 2024-08-30T09:03:58.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:03:58 smithi190 ceph-mon[22001]: pgmap v469: 129 pgs: 129 active+clean; 1.1 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 15 MiB/s wr, 946 op/s 2024-08-30T09:04:00.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:04:00 smithi167 ceph-mon[22063]: pgmap v470: 129 pgs: 129 active+clean; 1.1 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 13 MiB/s wr, 821 op/s 2024-08-30T09:04:00.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:04:00 smithi042 ceph-mon[18652]: pgmap v470: 129 pgs: 129 active+clean; 1.1 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 13 MiB/s wr, 821 op/s 2024-08-30T09:04:00.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:04:00 smithi190 ceph-mon[22001]: pgmap v470: 129 pgs: 129 active+clean; 1.1 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 13 MiB/s wr, 821 op/s 2024-08-30T09:04:00.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.416+0000 7f552193b640 1 -- 172.21.15.42:0/745685907 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f551c103920 msgr2=0x7f551c103d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:04:00.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.416+0000 7f552193b640 1 --2- 172.21.15.42:0/745685907 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f551c103920 0x7f551c103d30 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f5504009900 tx=0x7f550402a410 comp rx=0 tx=0).stop 2024-08-30T09:04:00.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.416+0000 7f552193b640 1 -- 172.21.15.42:0/745685907 shutdown_connections 2024-08-30T09:04:00.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.416+0000 7f552193b640 1 --2- 172.21.15.42:0/745685907 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f551c109260 0x7f551c109670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:00.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.416+0000 7f552193b640 1 --2- 172.21.15.42:0/745685907 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f551c109bb0 0x7f551c1103e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:00.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.416+0000 7f552193b640 1 --2- 172.21.15.42:0/745685907 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f551c103920 0x7f551c103d30 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:00.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.416+0000 7f552193b640 1 -- 172.21.15.42:0/745685907 >> 172.21.15.42:0/745685907 conn(0x7f551c0ff3b0 msgr2=0x7f551c101800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:04:00.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.416+0000 7f552193b640 1 -- 172.21.15.42:0/745685907 shutdown_connections 2024-08-30T09:04:00.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.416+0000 7f552193b640 1 -- 172.21.15.42:0/745685907 wait complete. 2024-08-30T09:04:00.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.417+0000 7f552193b640 1 Processor -- start 2024-08-30T09:04:00.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.417+0000 7f552193b640 1 -- start start 2024-08-30T09:04:00.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.417+0000 7f552193b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f551c103920 0x7f551c19a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:00.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.417+0000 7f552193b640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f551c109260 0x7f551c19a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:00.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.417+0000 7f552193b640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f551c109bb0 0x7f551c19fac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:00.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.417+0000 7f552193b640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f551c113870 con 0x7f551c109260 2024-08-30T09:04:00.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.417+0000 7f552193b640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f551c1136f0 con 0x7f551c109bb0 2024-08-30T09:04:00.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.417+0000 7f552193b640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f551c1139f0 con 0x7f551c103920 2024-08-30T09:04:00.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.417+0000 7f551a7fc640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f551c109260 0x7f551c19a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:00.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.417+0000 7f551a7fc640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f551c109260 0x7f551c19a820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37152/0 (socket says 172.21.15.42:37152) 2024-08-30T09:04:00.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.417+0000 7f551a7fc640 1 -- 172.21.15.42:0/3609814251 learned_addr learned my addr 172.21.15.42:0/3609814251 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:04:00.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.418+0000 7f551affd640 1 --2- 172.21.15.42:0/3609814251 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f551c103920 0x7f551c19a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:00.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.418+0000 7f551b7fe640 1 --2- 172.21.15.42:0/3609814251 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f551c109bb0 0x7f551c19fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:00.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.418+0000 7f551a7fc640 1 -- 172.21.15.42:0/3609814251 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f551c103920 msgr2=0x7f551c19a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:04:00.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.418+0000 7f551a7fc640 1 --2- 172.21.15.42:0/3609814251 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f551c103920 0x7f551c19a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:00.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.418+0000 7f551a7fc640 1 -- 172.21.15.42:0/3609814251 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f551c109bb0 msgr2=0x7f551c19fac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:04:00.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.418+0000 7f551a7fc640 1 --2- 172.21.15.42:0/3609814251 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f551c109bb0 0x7f551c19fac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:00.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.418+0000 7f551a7fc640 1 -- 172.21.15.42:0/3609814251 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f551c1a0130 con 0x7f551c109260 2024-08-30T09:04:00.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.418+0000 7f551b7fe640 1 --2- 172.21.15.42:0/3609814251 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f551c109bb0 0x7f551c19fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-30T09:04:00.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.418+0000 7f551a7fc640 1 --2- 172.21.15.42:0/3609814251 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f551c109260 0x7f551c19a820 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f550c00e9c0 tx=0x7f550c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:04:00.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.418+0000 7f5520939640 1 -- 172.21.15.42:0/3609814251 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f550c00ccc0 con 0x7f551c109260 2024-08-30T09:04:00.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.418+0000 7f552193b640 1 -- 172.21.15.42:0/3609814251 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f551c1a0420 con 0x7f551c109260 2024-08-30T09:04:00.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.418+0000 7f5520939640 1 -- 172.21.15.42:0/3609814251 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f550c0054d0 con 0x7f551c109260 2024-08-30T09:04:00.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.418+0000 7f5520939640 1 -- 172.21.15.42:0/3609814251 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f550c010430 con 0x7f551c109260 2024-08-30T09:04:00.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.418+0000 7f552193b640 1 -- 172.21.15.42:0/3609814251 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f551c1a6c40 con 0x7f551c109260 2024-08-30T09:04:00.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.421+0000 7f5527eb0740 1 Processor -- start 2024-08-30T09:04:00.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.421+0000 7f5527eb0740 1 -- start start 2024-08-30T09:04:00.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.421+0000 7f5520939640 1 -- 172.21.15.42:0/3609814251 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7f550c0105d0 con 0x7f551c109260 2024-08-30T09:04:00.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.421+0000 7f5527eb0740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559bd7931070 0x559bd7cd6350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:00.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.421+0000 7f5527eb0740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559bd79081b0 0x559bd7cd6890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:00.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.421+0000 7f5527eb0740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559bd7910390 0x559bd7cd6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:00.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.421+0000 7f5527eb0740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x559bd7c4d680 con 0x559bd7931070 2024-08-30T09:04:00.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.421+0000 7f5527eb0740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x559bd7c3cc10 con 0x559bd7910390 2024-08-30T09:04:00.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.421+0000 7f5527eb0740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x559bd7c40bc0 con 0x559bd79081b0 2024-08-30T09:04:00.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.421+0000 7f551b7fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559bd7931070 0x559bd7cd6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:00.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.421+0000 7f551b7fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559bd7931070 0x559bd7cd6350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37168/0 (socket says 172.21.15.42:37168) 2024-08-30T09:04:00.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.421+0000 7f551b7fe640 1 -- 172.21.15.42:0/1091714475 learned_addr learned my addr 172.21.15.42:0/1091714475 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:04:00.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.421+0000 7f551affd640 1 --2- 172.21.15.42:0/1091714475 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559bd79081b0 0x559bd7cd6890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:00.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.421+0000 7f551a7fc640 1 --2- 172.21.15.42:0/1091714475 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559bd7910390 0x559bd7cd6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:00.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.422+0000 7f551b7fe640 1 -- 172.21.15.42:0/1091714475 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559bd79081b0 msgr2=0x559bd7cd6890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:04:00.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.422+0000 7f551b7fe640 1 --2- 172.21.15.42:0/1091714475 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559bd79081b0 0x559bd7cd6890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:00.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.422+0000 7f551b7fe640 1 -- 172.21.15.42:0/1091714475 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559bd7910390 msgr2=0x559bd7cd6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:04:00.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.422+0000 7f551b7fe640 1 --2- 172.21.15.42:0/1091714475 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559bd7910390 0x559bd7cd6dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:00.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.422+0000 7f551b7fe640 1 -- 172.21.15.42:0/1091714475 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559bd799c650 con 0x559bd7931070 2024-08-30T09:04:00.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.422+0000 7f551affd640 1 --2- 172.21.15.42:0/1091714475 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559bd79081b0 0x559bd7cd6890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-30T09:04:00.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.422+0000 7f5520939640 1 --2- 172.21.15.42:0/3609814251 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f55080b4b60 0x7f55080b7070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:00.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.422+0000 7f551b7fe640 1 --2- 172.21.15.42:0/1091714475 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559bd7931070 0x559bd7cd6350 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f5514008e60 tx=0x7f551400d710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf1c55ec3e2bad16 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:04:00.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.422+0000 7f551affd640 1 --2- 172.21.15.42:0/3609814251 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f55080b4b60 0x7f55080b7070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:00.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.422+0000 7f5520939640 1 -- 172.21.15.42:0/3609814251 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f550c0e58a0 con 0x7f551c109260 2024-08-30T09:04:00.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.422+0000 7f5511ffb640 1 -- 172.21.15.42:0/1091714475 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f551400f830 con 0x559bd7931070 2024-08-30T09:04:00.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.422+0000 7f5511ffb640 1 -- 172.21.15.42:0/1091714475 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5514018480 con 0x559bd7931070 2024-08-30T09:04:00.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.422+0000 7f5511ffb640 1 -- 172.21.15.42:0/1091714475 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f551400e610 con 0x559bd7931070 2024-08-30T09:04:00.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.423+0000 7f5527eb0740 1 -- 172.21.15.42:0/1091714475 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559bd7931070 msgr2=0x559bd7cd6350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:04:00.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.423+0000 7f551affd640 1 --2- 172.21.15.42:0/3609814251 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f55080b4b60 0x7f55080b7070 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f5504031440 tx=0x7f5504009640 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:04:00.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.423+0000 7f5527eb0740 1 --2- 172.21.15.42:0/1091714475 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559bd7931070 0x559bd7cd6350 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f5514008e60 tx=0x7f551400d710 comp rx=0 tx=0).stop 2024-08-30T09:04:00.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.423+0000 7f5527eb0740 1 -- 172.21.15.42:0/1091714475 shutdown_connections 2024-08-30T09:04:00.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.423+0000 7f5527eb0740 1 --2- 172.21.15.42:0/1091714475 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559bd79081b0 0x559bd7cd6890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:00.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.423+0000 7f5527eb0740 1 --2- 172.21.15.42:0/1091714475 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559bd7910390 0x559bd7cd6dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:00.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.423+0000 7f5527eb0740 1 --2- 172.21.15.42:0/1091714475 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559bd7931070 0x559bd7cd6350 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:00.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.423+0000 7f5527eb0740 1 -- 172.21.15.42:0/1091714475 >> 172.21.15.42:0/1091714475 conn(0x559bd7925460 msgr2=0x559bd7cd4240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:04:00.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.423+0000 7f5527eb0740 1 -- 172.21.15.42:0/1091714475 shutdown_connections 2024-08-30T09:04:00.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.423+0000 7f5527eb0740 1 -- 172.21.15.42:0/1091714475 wait complete. 2024-08-30T09:04:00.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.424+0000 7f5527eb0740 1 -- start start 2024-08-30T09:04:00.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.424+0000 7f5527eb0740 1 Processor -- start 2024-08-30T09:04:00.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.424+0000 7f5527eb0740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559bd79081b0 0x559bd7ce1b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:00.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.424+0000 7f5527eb0740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559bd7910390 0x559bd7ce2040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:00.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.424+0000 7f5527eb0740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559bd7931070 0x559bd7cede20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:00.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.424+0000 7f5527eb0740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x559bd7a9f700 con 0x559bd79081b0 2024-08-30T09:04:00.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.424+0000 7f5527eb0740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x559bd7bc3750 con 0x559bd7910390 2024-08-30T09:04:00.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.424+0000 7f5527eb0740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x559bd7ce4b30 con 0x559bd7931070 2024-08-30T09:04:00.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.424+0000 7f551affd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559bd79081b0 0x559bd7ce1b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:00.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.424+0000 7f5527eb0740 10 client.0 _ll_register_callbacks cb 0x559bd7bf2e10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:04:00.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.424+0000 7f551affd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559bd79081b0 0x559bd7ce1b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37182/0 (socket says 172.21.15.42:37182) 2024-08-30T09:04:00.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.424+0000 7f551affd640 1 -- 172.21.15.42:0/2921738655 learned_addr learned my addr 172.21.15.42:0/2921738655 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:04:00.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.424+0000 7f551a7fc640 1 --2- 172.21.15.42:0/2921738655 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559bd7910390 0x559bd7ce2040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:00.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.424+0000 7f551b7fe640 1 --2- 172.21.15.42:0/2921738655 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559bd7931070 0x559bd7cede20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:00.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.425+0000 7f551affd640 1 -- 172.21.15.42:0/2921738655 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559bd7931070 msgr2=0x559bd7cede20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:04:00.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.425+0000 7f551affd640 1 --2- 172.21.15.42:0/2921738655 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559bd7931070 0x559bd7cede20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:00.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.425+0000 7f551affd640 1 -- 172.21.15.42:0/2921738655 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559bd7910390 msgr2=0x559bd7ce2040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:04:00.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.425+0000 7f551affd640 1 --2- 172.21.15.42:0/2921738655 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559bd7910390 0x559bd7ce2040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:00.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.425+0000 7f551affd640 1 -- 172.21.15.42:0/2921738655 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559bd7bfc900 con 0x559bd79081b0 2024-08-30T09:04:00.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.425+0000 7f551a7fc640 1 --2- 172.21.15.42:0/2921738655 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559bd7910390 0x559bd7ce2040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-30T09:04:00.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.425+0000 7f551affd640 1 --2- 172.21.15.42:0/2921738655 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559bd79081b0 0x559bd7ce1b00 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f5504033000 tx=0x7f5504009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e78b35752a9c1276 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:04:00.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.425+0000 7f54faffd640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-08-30T09:04:00.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.425+0000 7f54faffd640 1 -- 172.21.15.42:0/2921738655 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f550404d030 con 0x559bd79081b0 2024-08-30T09:04:00.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.425+0000 7f54faffd640 1 -- 172.21.15.42:0/2921738655 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f5504004310 con 0x559bd79081b0 2024-08-30T09:04:00.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.425+0000 7f5527eb0740 1 -- 172.21.15.42:0/2921738655 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5514018480 con 0x559bd79081b0 2024-08-30T09:04:00.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.425+0000 7f54faffd640 1 -- 172.21.15.42:0/2921738655 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f550405a050 con 0x559bd79081b0 2024-08-30T09:04:00.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.425+0000 7f5527eb0740 10 client.16023 fetch_fsmap learned FSMap version 29 2024-08-30T09:04:00.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.425+0000 7f5527eb0740 1 -- 172.21.15.42:0/2921738655 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x559bd7cf0930 con 0x559bd79081b0 2024-08-30T09:04:00.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.425+0000 7f54faffd640 1 -- 172.21.15.42:0/2921738655 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5504003aa0 con 0x559bd79081b0 2024-08-30T09:04:00.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.426+0000 7f54faffd640 1 -- 172.21.15.42:0/2921738655 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f550403d800 con 0x559bd79081b0 2024-08-30T09:04:00.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.426+0000 7f5527eb0740 10 client.16023 fetch_fsmap finished waiting for FSMap version 29 2024-08-30T09:04:00.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.426+0000 7f5527eb0740 10 client.16023 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-08-30T09:04:00.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.426+0000 7f5527eb0740 20 client.16023 populate_metadata read hostname 'smithi042' 2024-08-30T09:04:00.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.426+0000 7f5527eb0740 1 --2- 172.21.15.42:0/2921738655 >> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] conn(0x559bd78ef9f0 0x559bd7cf3f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:00.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.426+0000 7f5527eb0740 4 client.16023 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-30T09:04:00.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.426+0000 7f5527eb0740 1 -- 172.21.15.42:0/2921738655 --> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f5514018480 con 0x559bd78ef9f0 2024-08-30T09:04:00.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.426+0000 7f551a7fc640 1 --2- 172.21.15.42:0/2921738655 >> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] conn(0x559bd78ef9f0 0x559bd7cf3f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:00.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.426+0000 7f551a7fc640 1 --2- 172.21.15.42:0/2921738655 >> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] conn(0x559bd78ef9f0 0x559bd7cf3f00 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3d5272d6ea99f959 server_cookie=a06f6e5965b0227b in_seq=0 out_seq=0 2024-08-30T09:04:00.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.426+0000 7f54faffd640 10 client.16023 ms_handle_connect on v2:172.21.15.190:6838/719981110 2024-08-30T09:04:00.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.427+0000 7f54faffd640 1 -- 172.21.15.42:0/2921738655 <== mds.1 v2:172.21.15.190:6838/719981110 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f5514018480 con 0x559bd78ef9f0 2024-08-30T09:04:00.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.427+0000 7f54faffd640 10 client.16023 handle_command_reply: tid=0 2024-08-30T09:04:00.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.427+0000 7f5527eb0740 1 client.16023 shutdown 2024-08-30T09:04:00.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.428+0000 7f5527eb0740 1 -- 172.21.15.42:0/2921738655 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559bd79081b0 msgr2=0x559bd7ce1b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:04:00.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.428+0000 7f5527eb0740 1 --2- 172.21.15.42:0/2921738655 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559bd79081b0 0x559bd7ce1b00 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f5504033000 tx=0x7f5504009260 comp rx=0 tx=0).stop 2024-08-30T09:04:00.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.428+0000 7f5527eb0740 1 -- 172.21.15.42:0/2921738655 shutdown_connections 2024-08-30T09:04:00.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.428+0000 7f5527eb0740 1 --2- 172.21.15.42:0/2921738655 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559bd7931070 0x559bd7cede20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:00.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.428+0000 7f5527eb0740 1 --2- 172.21.15.42:0/2921738655 >> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] conn(0x559bd78ef9f0 0x559bd7cf3f00 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:00.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.428+0000 7f54faffd640 0 client.16023 ms_handle_reset on v2:172.21.15.190:6838/719981110 2024-08-30T09:04:00.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.428+0000 7f5527eb0740 1 --2- 172.21.15.42:0/2921738655 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559bd7910390 0x559bd7ce2040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:00.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.428+0000 7f5527eb0740 1 --2- 172.21.15.42:0/2921738655 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559bd79081b0 0x559bd7ce1b00 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:00.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.428+0000 7f5527eb0740 1 -- 172.21.15.42:0/2921738655 >> 172.21.15.42:0/2921738655 conn(0x559bd7925460 msgr2=0x559bd7cd9530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:04:00.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.429+0000 7f5527eb0740 1 -- 172.21.15.42:0/2921738655 shutdown_connections 2024-08-30T09:04:00.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.430+0000 7f5527eb0740 1 -- 172.21.15.42:0/2921738655 wait complete. 2024-08-30T09:04:00.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.430+0000 7f5527eb0740 20 client.16023 trim_cache size 0 max 16384 2024-08-30T09:04:00.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.461+0000 7f5527eb0740 1 Processor -- start 2024-08-30T09:04:00.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.465+0000 7f5527eb0740 1 -- start start 2024-08-30T09:04:00.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.465+0000 7f5527eb0740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559bd78ef9f0 0x559bd7cd7a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:00.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.465+0000 7f5527eb0740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559bd79081b0 0x559bd7cedd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:00.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.466+0000 7f5527eb0740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559bd7910390 0x559bd7cee260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:00.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.466+0000 7f5527eb0740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f551c1b30b0 con 0x559bd79081b0 2024-08-30T09:04:00.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.466+0000 7f5527eb0740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f551c1b0980 con 0x559bd78ef9f0 2024-08-30T09:04:00.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.466+0000 7f5527eb0740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f551c1ae7d0 con 0x559bd7910390 2024-08-30T09:04:00.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.466+0000 7f551b7fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559bd7910390 0x559bd7cee260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:00.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.466+0000 7f551b7fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559bd7910390 0x559bd7cee260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:35686/0 (socket says 172.21.15.42:35686) 2024-08-30T09:04:00.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.466+0000 7f551b7fe640 1 -- 172.21.15.42:0/3203445970 learned_addr learned my addr 172.21.15.42:0/3203445970 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:04:00.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.466+0000 7f551affd640 1 --2- 172.21.15.42:0/3203445970 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559bd78ef9f0 0x559bd7cd7a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:00.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.467+0000 7f551a7fc640 1 --2- 172.21.15.42:0/3203445970 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559bd79081b0 0x559bd7cedd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:00.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.469+0000 7f551affd640 1 -- 172.21.15.42:0/3203445970 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559bd7910390 msgr2=0x559bd7cee260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:04:00.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.469+0000 7f551affd640 1 --2- 172.21.15.42:0/3203445970 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559bd7910390 0x559bd7cee260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:00.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.469+0000 7f551affd640 1 -- 172.21.15.42:0/3203445970 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559bd79081b0 msgr2=0x559bd7cedd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:04:00.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.469+0000 7f551affd640 1 --2- 172.21.15.42:0/3203445970 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559bd79081b0 0x559bd7cedd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:00.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.469+0000 7f551affd640 1 -- 172.21.15.42:0/3203445970 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559bd7cd91b0 con 0x559bd78ef9f0 2024-08-30T09:04:00.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.469+0000 7f551affd640 1 --2- 172.21.15.42:0/3203445970 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559bd78ef9f0 0x559bd7cd7a90 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f550403dca0 tx=0x7f550402a410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=de1b1daf140ad78c server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:04:00.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.471+0000 7f551b7fe640 1 --2- 172.21.15.42:0/3203445970 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559bd7910390 0x559bd7cee260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-30T09:04:00.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.472+0000 7f55117fa640 1 -- 172.21.15.42:0/3203445970 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5504066070 con 0x559bd78ef9f0 2024-08-30T09:04:00.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.472+0000 7f55117fa640 1 -- 172.21.15.42:0/3203445970 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f55040054e0 con 0x559bd78ef9f0 2024-08-30T09:04:00.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.472+0000 7f55117fa640 1 -- 172.21.15.42:0/3203445970 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f550404d070 con 0x559bd78ef9f0 2024-08-30T09:04:00.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.474+0000 7f5527eb0740 1 -- 172.21.15.42:0/3203445970 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559bd78ef9f0 msgr2=0x559bd7cd7a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:04:00.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.474+0000 7f5527eb0740 1 --2- 172.21.15.42:0/3203445970 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559bd78ef9f0 0x559bd7cd7a90 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f550403dca0 tx=0x7f550402a410 comp rx=0 tx=0).stop 2024-08-30T09:04:00.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.479+0000 7f5527eb0740 1 -- 172.21.15.42:0/3203445970 shutdown_connections 2024-08-30T09:04:00.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.479+0000 7f5527eb0740 1 --2- 172.21.15.42:0/3203445970 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559bd7910390 0x559bd7cee260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:00.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.479+0000 7f5527eb0740 1 --2- 172.21.15.42:0/3203445970 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559bd79081b0 0x559bd7cedd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:00.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.479+0000 7f5527eb0740 1 --2- 172.21.15.42:0/3203445970 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559bd78ef9f0 0x559bd7cd7a90 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:00.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.479+0000 7f5527eb0740 1 -- 172.21.15.42:0/3203445970 >> 172.21.15.42:0/3203445970 conn(0x559bd7925460 msgr2=0x559bd7cd7380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:04:00.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.486+0000 7f5527eb0740 1 -- 172.21.15.42:0/3203445970 shutdown_connections 2024-08-30T09:04:00.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.486+0000 7f5527eb0740 1 -- 172.21.15.42:0/3203445970 wait complete. 2024-08-30T09:04:00.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.487+0000 7f5527eb0740 1 -- start start 2024-08-30T09:04:00.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.487+0000 7f5527eb0740 1 Processor -- start 2024-08-30T09:04:00.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.488+0000 7f5527eb0740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559bd78ef9f0 0x559bd7cea2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:00.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.488+0000 7f5527eb0740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559bd79081b0 0x559bd7ce6790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:00.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.488+0000 7f551affd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559bd78ef9f0 0x559bd7cea2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:00.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.488+0000 7f551affd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559bd78ef9f0 0x559bd7cea2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:35700/0 (socket says 172.21.15.42:35700) 2024-08-30T09:04:00.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.489+0000 7f551affd640 1 -- 172.21.15.42:0/2228540434 learned_addr learned my addr 172.21.15.42:0/2228540434 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:04:00.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.489+0000 7f5527eb0740 1 --2- 172.21.15.42:0/2228540434 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559bd7910390 0x559bd7ce6cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:00.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.489+0000 7f551a7fc640 1 --2- 172.21.15.42:0/2228540434 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559bd79081b0 0x559bd7ce6790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:00.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.489+0000 7f5527eb0740 1 -- 172.21.15.42:0/2228540434 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f551c1aee40 con 0x559bd7910390 2024-08-30T09:04:00.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.489+0000 7f551b7fe640 1 --2- 172.21.15.42:0/2228540434 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559bd7910390 0x559bd7ce6cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:00.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.489+0000 7f5527eb0740 1 -- 172.21.15.42:0/2228540434 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f551c1b02b0 con 0x559bd79081b0 2024-08-30T09:04:00.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.489+0000 7f5527eb0740 1 -- 172.21.15.42:0/2228540434 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f551c1af510 con 0x559bd78ef9f0 2024-08-30T09:04:00.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.490+0000 7f551a7fc640 1 -- 172.21.15.42:0/2228540434 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559bd78ef9f0 msgr2=0x559bd7cea2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:04:00.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.490+0000 7f551a7fc640 1 --2- 172.21.15.42:0/2228540434 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559bd78ef9f0 0x559bd7cea2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:00.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.490+0000 7f551a7fc640 1 -- 172.21.15.42:0/2228540434 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559bd7910390 msgr2=0x559bd7ce6cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:04:00.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.490+0000 7f551a7fc640 1 --2- 172.21.15.42:0/2228540434 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559bd7910390 0x559bd7ce6cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:00.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.490+0000 7f551a7fc640 1 -- 172.21.15.42:0/2228540434 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559bd7ce7210 con 0x559bd79081b0 2024-08-30T09:04:00.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.490+0000 7f5527eb0740 10 client.0 _ll_register_callbacks cb 0x7f5504005e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:04:00.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.490+0000 7f551a7fc640 1 --2- 172.21.15.42:0/2228540434 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559bd79081b0 0x559bd7ce6790 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f550c0e3060 tx=0x7f550c0ab6b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7cb2fa8921f29067 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:04:00.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.490+0000 7f5527eb0740 1 -- 172.21.15.42:0/2228540434 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f55040054e0 con 0x559bd79081b0 2024-08-30T09:04:00.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.491+0000 7f54faffd640 10 client.25718 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-30T09:04:00.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.491+0000 7f54faffd640 1 -- 172.21.15.42:0/2228540434 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f550c005680 con 0x559bd79081b0 2024-08-30T09:04:00.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.491+0000 7f54faffd640 1 -- 172.21.15.42:0/2228540434 <== mon.2 v2:172.21.15.167:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f550c0a81d0 con 0x559bd79081b0 2024-08-30T09:04:00.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.492+0000 7f54faffd640 1 -- 172.21.15.42:0/2228540434 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f550c0ac640 con 0x559bd79081b0 2024-08-30T09:04:00.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.494+0000 7f5527eb0740 10 client.25718 fetch_fsmap learned FSMap version 29 2024-08-30T09:04:00.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.494+0000 7f5527eb0740 1 -- 172.21.15.42:0/2228540434 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({fsmap=29}) -- 0x559bd7d5f430 con 0x559bd79081b0 2024-08-30T09:04:00.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.496+0000 7f54faffd640 1 -- 172.21.15.42:0/2228540434 <== mon.2 v2:172.21.15.167:3300/0 4 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f550c0b4dd0 con 0x559bd79081b0 2024-08-30T09:04:00.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.496+0000 7f5527eb0740 10 client.25718 fetch_fsmap finished waiting for FSMap version 29 2024-08-30T09:04:00.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.496+0000 7f5527eb0740 10 client.25718 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-08-30T09:04:00.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.496+0000 7f5527eb0740 20 client.25718 populate_metadata read hostname 'smithi042' 2024-08-30T09:04:00.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.496+0000 7f5527eb0740 1 --2- 172.21.15.42:0/2228540434 >> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] conn(0x559bd7931070 0x559bd7d619f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:00.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.496+0000 7f5527eb0740 4 client.25718 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-30T09:04:00.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.496+0000 7f5527eb0740 1 -- 172.21.15.42:0/2228540434 --> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f55040054e0 con 0x559bd7931070 2024-08-30T09:04:00.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.496+0000 7f551affd640 1 --2- 172.21.15.42:0/2228540434 >> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] conn(0x559bd7931070 0x559bd7d619f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:00.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.497+0000 7f551affd640 1 --2- 172.21.15.42:0/2228540434 >> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] conn(0x559bd7931070 0x559bd7d619f0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1f30f76612c158c7 server_cookie=da4fd38a745675f5 in_seq=0 out_seq=0 2024-08-30T09:04:00.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.497+0000 7f54faffd640 10 client.25718 ms_handle_connect on v2:172.21.15.190:6838/719981110 2024-08-30T09:04:00.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.497+0000 7f54faffd640 1 -- 172.21.15.42:0/2228540434 <== mds.1 v2:172.21.15.190:6838/719981110 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f55040054e0 con 0x559bd7931070 2024-08-30T09:04:00.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.497+0000 7f54faffd640 10 client.25718 handle_command_reply: tid=0 2024-08-30T09:04:00.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.497+0000 7f5527eb0740 1 client.25718 shutdown 2024-08-30T09:04:00.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.499+0000 7f5527eb0740 1 -- 172.21.15.42:0/2228540434 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559bd79081b0 msgr2=0x559bd7ce6790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:04:00.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.499+0000 7f5527eb0740 1 --2- 172.21.15.42:0/2228540434 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559bd79081b0 0x559bd7ce6790 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f550c0e3060 tx=0x7f550c0ab6b0 comp rx=0 tx=0).stop 2024-08-30T09:04:00.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.500+0000 7f5527eb0740 1 -- 172.21.15.42:0/2228540434 shutdown_connections 2024-08-30T09:04:00.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.500+0000 7f5527eb0740 1 --2- 172.21.15.42:0/2228540434 >> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] conn(0x559bd7931070 0x559bd7d619f0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:00.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.500+0000 7f5527eb0740 1 --2- 172.21.15.42:0/2228540434 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559bd7910390 0x559bd7ce6cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:00.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.500+0000 7f5527eb0740 1 --2- 172.21.15.42:0/2228540434 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559bd79081b0 0x559bd7ce6790 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:00.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.500+0000 7f5527eb0740 1 --2- 172.21.15.42:0/2228540434 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559bd78ef9f0 0x559bd7cea2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:00.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.500+0000 7f5527eb0740 1 -- 172.21.15.42:0/2228540434 >> 172.21.15.42:0/2228540434 conn(0x559bd7925460 msgr2=0x559bd7ce2f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:04:00.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.500+0000 7f54faffd640 0 client.25718 ms_handle_reset on v2:172.21.15.190:6838/719981110 2024-08-30T09:04:00.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.501+0000 7f5527eb0740 1 -- 172.21.15.42:0/2228540434 shutdown_connections 2024-08-30T09:04:00.732 INFO:teuthology.orchestra.run.smithi042.stdout:[] 2024-08-30T09:04:00.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.501+0000 7f5527eb0740 1 -- 172.21.15.42:0/2228540434 wait complete. 2024-08-30T09:04:00.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.502+0000 7f5527eb0740 20 client.25718 trim_cache size 0 max 16384 2024-08-30T09:04:00.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.502+0000 7f552193b640 1 -- 172.21.15.42:0/3609814251 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f55080b4b60 msgr2=0x7f55080b7070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:04:00.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.503+0000 7f552193b640 1 --2- 172.21.15.42:0/3609814251 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f55080b4b60 0x7f55080b7070 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f5504031440 tx=0x7f5504009640 comp rx=0 tx=0).stop 2024-08-30T09:04:00.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.503+0000 7f552193b640 1 -- 172.21.15.42:0/3609814251 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f551c109260 msgr2=0x7f551c19a820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:04:00.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.503+0000 7f552193b640 1 --2- 172.21.15.42:0/3609814251 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f551c109260 0x7f551c19a820 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f550c00e9c0 tx=0x7f550c00ee80 comp rx=0 tx=0).stop 2024-08-30T09:04:00.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.504+0000 7f552193b640 1 -- 172.21.15.42:0/3609814251 shutdown_connections 2024-08-30T09:04:00.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.504+0000 7f552193b640 1 --2- 172.21.15.42:0/3609814251 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f55080b4b60 0x7f55080b7070 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:00.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.504+0000 7f552193b640 1 --2- 172.21.15.42:0/3609814251 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f551c109bb0 0x7f551c19fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:00.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.504+0000 7f552193b640 1 --2- 172.21.15.42:0/3609814251 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f551c109260 0x7f551c19a820 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:00.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.504+0000 7f552193b640 1 --2- 172.21.15.42:0/3609814251 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f551c103920 0x7f551c19a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:00.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.504+0000 7f552193b640 1 -- 172.21.15.42:0/3609814251 >> 172.21.15.42:0/3609814251 conn(0x7f551c0ff3b0 msgr2=0x7f551c1017d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:04:00.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.504+0000 7f552193b640 1 -- 172.21.15.42:0/3609814251 shutdown_connections 2024-08-30T09:04:00.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:03:59.505+0000 7f552193b640 1 -- 172.21.15.42:0/3609814251 wait complete. 2024-08-30T09:04:00.953 INFO:teuthology.orchestra.run.smithi042.stderr:progress: 195.0 s, 862.9 tps, lat 118.350 ms stddev 142.720 2024-08-30T09:04:02.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:04:02 smithi167 ceph-mon[22063]: pgmap v471: 129 pgs: 129 active+clean; 1.1 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 12 MiB/s wr, 711 op/s 2024-08-30T09:04:02.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:04:02 smithi042 ceph-mon[18652]: pgmap v471: 129 pgs: 129 active+clean; 1.1 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 12 MiB/s wr, 711 op/s 2024-08-30T09:04:02.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:04:02 smithi190 ceph-mon[22001]: pgmap v471: 129 pgs: 129 active+clean; 1.1 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 12 MiB/s wr, 711 op/s 2024-08-30T09:04:04.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:04:04 smithi167 ceph-mon[22063]: pgmap v472: 129 pgs: 129 active+clean; 1.1 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 13 MiB/s wr, 798 op/s 2024-08-30T09:04:04.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:04:04 smithi042 ceph-mon[18652]: pgmap v472: 129 pgs: 129 active+clean; 1.1 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 13 MiB/s wr, 798 op/s 2024-08-30T09:04:04.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:04:04 smithi190 ceph-mon[22001]: pgmap v472: 129 pgs: 129 active+clean; 1.1 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 13 MiB/s wr, 798 op/s 2024-08-30T09:04:04.866 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-08-30T09:04:05.301 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T09:04:05.357 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.355+0000 7f1554eeb640 1 -- 172.21.15.42:0/1276332523 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1548030d70 con 0x7f155010f5d0 2024-08-30T09:04:05.358 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.356+0000 7f1558151640 1 -- 172.21.15.42:0/1276332523 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f155010f5d0 msgr2=0x7f155010fad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:04:05.358 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.356+0000 7f1558151640 1 --2- 172.21.15.42:0/1276332523 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f155010f5d0 0x7f155010fad0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f1548009be0 tx=0x7f154802a6d0 comp rx=0 tx=0).stop 2024-08-30T09:04:05.358 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.356+0000 7f1558151640 1 -- 172.21.15.42:0/1276332523 shutdown_connections 2024-08-30T09:04:05.358 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.356+0000 7f1558151640 1 --2- 172.21.15.42:0/1276332523 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f155010f5d0 0x7f155010fad0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:05.358 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.356+0000 7f1558151640 1 --2- 172.21.15.42:0/1276332523 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f155010cac0 0x7f155010eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:05.358 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.356+0000 7f1558151640 1 --2- 172.21.15.42:0/1276332523 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f155010a160 0x7f155010c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:05.358 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.356+0000 7f1558151640 1 -- 172.21.15.42:0/1276332523 >> 172.21.15.42:0/1276332523 conn(0x7f1550070410 msgr2=0x7f15500728c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:04:05.361 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.359+0000 7f1558151640 1 -- 172.21.15.42:0/1276332523 shutdown_connections 2024-08-30T09:04:05.363 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.360+0000 7f1558151640 1 -- 172.21.15.42:0/1276332523 wait complete. 2024-08-30T09:04:05.363 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.360+0000 7f1558151640 1 Processor -- start 2024-08-30T09:04:05.363 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.360+0000 7f1558151640 1 -- start start 2024-08-30T09:04:05.363 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.361+0000 7f1558151640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f155010a160 0x7f155011ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:05.363 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.361+0000 7f1558151640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f155010cac0 0x7f155011b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:05.363 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.361+0000 7f1558151640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f155010f5d0 0x7f1550119960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:05.364 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.361+0000 7f1558151640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f1550111d60 con 0x7f155010f5d0 2024-08-30T09:04:05.364 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.361+0000 7f1558151640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f1550111be0 con 0x7f155010cac0 2024-08-30T09:04:05.364 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.361+0000 7f1558151640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f1550111ee0 con 0x7f155010a160 2024-08-30T09:04:05.364 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.361+0000 7f1555eed640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f155010a160 0x7f155011ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:05.364 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.361+0000 7f1555eed640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f155010a160 0x7f155011ada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:35710/0 (socket says 172.21.15.42:35710) 2024-08-30T09:04:05.364 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.361+0000 7f1555eed640 1 -- 172.21.15.42:0/2056765597 learned_addr learned my addr 172.21.15.42:0/2056765597 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:04:05.364 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.361+0000 7f15556ec640 1 --2- 172.21.15.42:0/2056765597 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f155010cac0 0x7f155011b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:05.365 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.362+0000 7f1555eed640 1 -- 172.21.15.42:0/2056765597 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f155010cac0 msgr2=0x7f155011b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:04:05.365 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.362+0000 7f1555eed640 1 --2- 172.21.15.42:0/2056765597 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f155010cac0 0x7f155011b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:05.365 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.362+0000 7f1555eed640 1 -- 172.21.15.42:0/2056765597 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f155010f5d0 msgr2=0x7f1550119960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:04:05.365 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.362+0000 7f1555eed640 1 --2- 172.21.15.42:0/2056765597 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f155010f5d0 0x7f1550119960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:05.365 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.362+0000 7f1555eed640 1 -- 172.21.15.42:0/2056765597 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1550119f50 con 0x7f155010a160 2024-08-30T09:04:05.365 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.362+0000 7f1555eed640 1 --2- 172.21.15.42:0/2056765597 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f155010a160 0x7f155011ada0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f154c00e9c0 tx=0x7f154c00edc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:04:05.367 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.363+0000 7f1546ffd640 1 -- 172.21.15.42:0/2056765597 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f154c00ccc0 con 0x7f155010a160 2024-08-30T09:04:05.367 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.363+0000 7f1546ffd640 1 -- 172.21.15.42:0/2056765597 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f154c0054d0 con 0x7f155010a160 2024-08-30T09:04:05.367 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.363+0000 7f1546ffd640 1 -- 172.21.15.42:0/2056765597 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f154c010430 con 0x7f155010a160 2024-08-30T09:04:05.368 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.363+0000 7f1558151640 1 -- 172.21.15.42:0/2056765597 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f155011a240 con 0x7f155010a160 2024-08-30T09:04:05.368 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.363+0000 7f1558151640 1 -- 172.21.15.42:0/2056765597 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15501b9610 con 0x7f155010a160 2024-08-30T09:04:05.375 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.370+0000 7f155f0b8740 1 Processor -- start 2024-08-30T09:04:05.375 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.373+0000 7f155f0b8740 1 -- start start 2024-08-30T09:04:05.379 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.373+0000 7f155f0b8740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56055d22a1b0 0x56055d5d0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:05.380 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.373+0000 7f155f0b8740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56055d2012f0 0x56055d5d07d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:05.380 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.373+0000 7f155f0b8740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56055d2094d0 0x56055d5d1320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:05.380 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.373+0000 7f155f0b8740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x56055d543000 con 0x56055d2012f0 2024-08-30T09:04:05.380 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.373+0000 7f155f0b8740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x56055d532590 con 0x56055d22a1b0 2024-08-30T09:04:05.380 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.373+0000 7f155f0b8740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x56055d536540 con 0x56055d2094d0 2024-08-30T09:04:05.380 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.373+0000 7f1546ffd640 1 -- 172.21.15.42:0/2056765597 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7f154c004f20 con 0x7f155010a160 2024-08-30T09:04:05.380 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.373+0000 7f15566ee640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56055d22a1b0 0x56055d5d0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:05.380 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.374+0000 7f15566ee640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56055d22a1b0 0x56055d5d0160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:40460/0 (socket says 172.21.15.42:40460) 2024-08-30T09:04:05.380 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.374+0000 7f15566ee640 1 -- 172.21.15.42:0/4135635413 learned_addr learned my addr 172.21.15.42:0/4135635413 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:04:05.380 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.374+0000 7f1555eed640 1 --2- 172.21.15.42:0/4135635413 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56055d2012f0 0x56055d5d07d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:05.380 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.375+0000 7f15556ec640 1 --2- 172.21.15.42:0/4135635413 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56055d2094d0 0x56055d5d1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:05.380 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.375+0000 7f1546ffd640 1 --2- 172.21.15.42:0/2056765597 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f153c0b4c30 0x7f153c0b7140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:05.380 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.375+0000 7f1546ffd640 1 -- 172.21.15.42:0/2056765597 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f154c0e5a00 con 0x7f155010a160 2024-08-30T09:04:05.381 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.376+0000 7f15566ee640 1 -- 172.21.15.42:0/4135635413 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56055d2094d0 msgr2=0x56055d5d1320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:04:05.381 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.376+0000 7f15566ee640 1 --2- 172.21.15.42:0/4135635413 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56055d2094d0 0x56055d5d1320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:05.381 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.376+0000 7f15566ee640 1 -- 172.21.15.42:0/4135635413 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56055d2012f0 msgr2=0x56055d5d07d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:04:05.381 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.376+0000 7f15566ee640 1 --2- 172.21.15.42:0/4135635413 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56055d2012f0 0x56055d5d07d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:05.381 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.376+0000 7f15566ee640 1 -- 172.21.15.42:0/4135635413 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56055d295790 con 0x56055d22a1b0 2024-08-30T09:04:05.381 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.376+0000 7f15566ee640 1 --2- 172.21.15.42:0/2056765597 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f153c0b4c30 0x7f153c0b7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:05.381 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.376+0000 7f15566ee640 1 --2- 172.21.15.42:0/4135635413 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56055d22a1b0 0x56055d5d0160 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f1548034040 tx=0x7f15480304c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=652f1249d5f895c7 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:04:05.381 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.377+0000 7f153bfff640 1 -- 172.21.15.42:0/4135635413 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1548036040 con 0x56055d22a1b0 2024-08-30T09:04:05.381 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.377+0000 7f153bfff640 1 -- 172.21.15.42:0/4135635413 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1548041040 con 0x56055d22a1b0 2024-08-30T09:04:05.381 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.377+0000 7f153bfff640 1 -- 172.21.15.42:0/4135635413 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15480316a0 con 0x56055d22a1b0 2024-08-30T09:04:05.383 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.378+0000 7f155f0b8740 1 -- 172.21.15.42:0/4135635413 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56055d22a1b0 msgr2=0x56055d5d0160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:04:05.383 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.378+0000 7f15566ee640 1 --2- 172.21.15.42:0/2056765597 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f153c0b4c30 0x7f153c0b7140 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f1548046000 tx=0x7f1548031980 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:04:05.383 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.378+0000 7f155f0b8740 1 --2- 172.21.15.42:0/4135635413 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56055d22a1b0 0x56055d5d0160 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f1548034040 tx=0x7f15480304c0 comp rx=0 tx=0).stop 2024-08-30T09:04:05.383 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.378+0000 7f155f0b8740 1 -- 172.21.15.42:0/4135635413 shutdown_connections 2024-08-30T09:04:05.383 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.378+0000 7f155f0b8740 1 --2- 172.21.15.42:0/4135635413 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56055d2094d0 0x56055d5d1320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:05.383 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.378+0000 7f155f0b8740 1 --2- 172.21.15.42:0/4135635413 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56055d2012f0 0x56055d5d07d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:05.383 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.379+0000 7f155f0b8740 1 --2- 172.21.15.42:0/4135635413 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56055d22a1b0 0x56055d5d0160 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:05.383 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.379+0000 7f155f0b8740 1 -- 172.21.15.42:0/4135635413 >> 172.21.15.42:0/4135635413 conn(0x56055d21e300 msgr2=0x56055d5cbbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:04:05.384 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.379+0000 7f155f0b8740 1 -- 172.21.15.42:0/4135635413 shutdown_connections 2024-08-30T09:04:05.384 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.379+0000 7f155f0b8740 1 -- 172.21.15.42:0/4135635413 wait complete. 2024-08-30T09:04:05.384 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.379+0000 7f155f0b8740 1 -- start start 2024-08-30T09:04:05.384 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.380+0000 7f155f0b8740 1 Processor -- start 2024-08-30T09:04:05.384 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.381+0000 7f155f0b8740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56055d2012f0 0x56055d5d9bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:05.384 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.381+0000 7f155f0b8740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56055d2094d0 0x56055d5da290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:05.384 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.381+0000 7f155f0b8740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56055d22a1b0 0x56055d5dae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:05.384 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.381+0000 7f155f0b8740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x56055d46bab0 con 0x56055d2094d0 2024-08-30T09:04:05.384 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.381+0000 7f155f0b8740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x56055d4b9680 con 0x56055d22a1b0 2024-08-30T09:04:05.384 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.381+0000 7f155f0b8740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x56055d5d1e40 con 0x56055d2012f0 2024-08-30T09:04:05.384 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.381+0000 7f155f0b8740 10 client.0 _ll_register_callbacks cb 0x56055d241a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:04:05.384 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.381+0000 7f1555eed640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56055d2012f0 0x56055d5d9bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:05.384 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.381+0000 7f1555eed640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56055d2012f0 0x56055d5d9bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:35730/0 (socket says 172.21.15.42:35730) 2024-08-30T09:04:05.384 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.381+0000 7f1555eed640 1 -- 172.21.15.42:0/975469423 learned_addr learned my addr 172.21.15.42:0/975469423 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:04:05.386 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.381+0000 7f15566ee640 1 --2- 172.21.15.42:0/975469423 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56055d22a1b0 0x56055d5dae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:05.386 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.382+0000 7f15556ec640 1 --2- 172.21.15.42:0/975469423 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56055d2094d0 0x56055d5da290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:05.386 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.382+0000 7f15566ee640 1 -- 172.21.15.42:0/975469423 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56055d2012f0 msgr2=0x56055d5d9bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:04:05.386 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.382+0000 7f15566ee640 1 --2- 172.21.15.42:0/975469423 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56055d2012f0 0x56055d5d9bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:05.386 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.382+0000 7f15566ee640 1 -- 172.21.15.42:0/975469423 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56055d2094d0 msgr2=0x56055d5da290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:04:05.386 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.382+0000 7f15566ee640 1 --2- 172.21.15.42:0/975469423 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56055d2094d0 0x56055d5da290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:05.386 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.382+0000 7f15566ee640 1 -- 172.21.15.42:0/975469423 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56055d30e620 con 0x56055d22a1b0 2024-08-30T09:04:05.386 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.383+0000 7f15566ee640 1 --2- 172.21.15.42:0/975469423 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56055d22a1b0 0x56055d5dae00 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f1548031fd0 tx=0x7f154803f000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f64aba806f0d796a server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:04:05.386 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.383+0000 7f15397fa640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-30T09:04:05.386 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.383+0000 7f15397fa640 1 -- 172.21.15.42:0/975469423 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1548036040 con 0x56055d22a1b0 2024-08-30T09:04:05.386 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.383+0000 7f155f0b8740 1 -- 172.21.15.42:0/975469423 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1548041040 con 0x56055d22a1b0 2024-08-30T09:04:05.386 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.383+0000 7f15397fa640 1 -- 172.21.15.42:0/975469423 <== mon.2 v2:172.21.15.167:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f1548031440 con 0x56055d22a1b0 2024-08-30T09:04:05.386 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.384+0000 7f15397fa640 1 -- 172.21.15.42:0/975469423 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f1548062050 con 0x56055d22a1b0 2024-08-30T09:04:05.386 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.384+0000 7f155f0b8740 10 client.25730 fetch_fsmap learned FSMap version 29 2024-08-30T09:04:05.387 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.384+0000 7f155f0b8740 1 -- 172.21.15.42:0/975469423 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56055d5e3d50 con 0x56055d22a1b0 2024-08-30T09:04:05.387 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.385+0000 7f15397fa640 1 -- 172.21.15.42:0/975469423 <== mon.2 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1548055030 con 0x56055d22a1b0 2024-08-30T09:04:05.387 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.385+0000 7f15397fa640 1 -- 172.21.15.42:0/975469423 <== mon.2 v2:172.21.15.167:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f1548044040 con 0x56055d22a1b0 2024-08-30T09:04:05.388 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.386+0000 7f155f0b8740 10 client.25730 fetch_fsmap finished waiting for FSMap version 29 2024-08-30T09:04:05.388 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.386+0000 7f155f0b8740 10 client.25730 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-30T09:04:05.388 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.386+0000 7f155f0b8740 20 client.25730 populate_metadata read hostname 'smithi042' 2024-08-30T09:04:05.388 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.386+0000 7f155f0b8740 1 --2- 172.21.15.42:0/975469423 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x56055d1e8d80 0x56055d5e7680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:05.388 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.386+0000 7f155f0b8740 4 client.25730 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-30T09:04:05.388 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.386+0000 7f155f0b8740 1 -- 172.21.15.42:0/975469423 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f1548041040 con 0x56055d1e8d80 2024-08-30T09:04:05.389 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.388+0000 7f1555eed640 1 --2- 172.21.15.42:0/975469423 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x56055d1e8d80 0x56055d5e7680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:05.390 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.388+0000 7f1555eed640 1 --2- 172.21.15.42:0/975469423 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x56055d1e8d80 0x56055d5e7680 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f84b62a7594a0bbe server_cookie=b921c893cd6a81df in_seq=0 out_seq=0 2024-08-30T09:04:05.390 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.388+0000 7f15397fa640 10 client.25730 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:04:05.391 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.389+0000 7f15397fa640 1 -- 172.21.15.42:0/975469423 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f1548041040 con 0x56055d1e8d80 2024-08-30T09:04:05.392 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.389+0000 7f15397fa640 10 client.25730 handle_command_reply: tid=0 2024-08-30T09:04:05.392 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.389+0000 7f155f0b8740 1 client.25730 shutdown 2024-08-30T09:04:05.392 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.389+0000 7f155f0b8740 1 -- 172.21.15.42:0/975469423 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56055d22a1b0 msgr2=0x56055d5dae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:04:05.392 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.389+0000 7f155f0b8740 1 --2- 172.21.15.42:0/975469423 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56055d22a1b0 0x56055d5dae00 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f1548031fd0 tx=0x7f154803f000 comp rx=0 tx=0).stop 2024-08-30T09:04:05.392 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.389+0000 7f155f0b8740 1 -- 172.21.15.42:0/975469423 shutdown_connections 2024-08-30T09:04:05.392 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.389+0000 7f155f0b8740 1 --2- 172.21.15.42:0/975469423 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x56055d1e8d80 0x56055d5e7680 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:05.392 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.390+0000 7f155f0b8740 1 --2- 172.21.15.42:0/975469423 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56055d22a1b0 0x56055d5dae00 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:05.392 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.390+0000 7f15397fa640 0 client.25730 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:04:05.392 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.390+0000 7f155f0b8740 1 --2- 172.21.15.42:0/975469423 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56055d2094d0 0x56055d5da290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:05.392 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.390+0000 7f155f0b8740 1 --2- 172.21.15.42:0/975469423 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56055d2012f0 0x56055d5d9bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:05.392 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.390+0000 7f155f0b8740 1 -- 172.21.15.42:0/975469423 >> 172.21.15.42:0/975469423 conn(0x56055d21e300 msgr2=0x56055d5ca150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:04:05.392 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.390+0000 7f155f0b8740 1 -- 172.21.15.42:0/975469423 shutdown_connections 2024-08-30T09:04:05.392 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.390+0000 7f155f0b8740 1 -- 172.21.15.42:0/975469423 wait complete. 2024-08-30T09:04:05.393 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.390+0000 7f155f0b8740 20 client.25730 trim_cache size 0 max 16384 2024-08-30T09:04:05.415 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.412+0000 7f155f0b8740 1 Processor -- start 2024-08-30T09:04:05.415 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.412+0000 7f155f0b8740 1 -- start start 2024-08-30T09:04:05.415 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.413+0000 7f155f0b8740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56055d1e8d80 0x56055d5cb530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:05.415 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.413+0000 7f155f0b8740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56055d2012f0 0x56055d5ca8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:05.415 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.413+0000 7f155f0b8740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56055d2094d0 0x56055d5cae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:05.415 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.413+0000 7f155f0b8740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f15501c6270 con 0x56055d2012f0 2024-08-30T09:04:05.415 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.413+0000 7f155f0b8740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f15501c3b40 con 0x56055d2094d0 2024-08-30T09:04:05.416 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.413+0000 7f155f0b8740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f15501c1990 con 0x56055d1e8d80 2024-08-30T09:04:05.416 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.413+0000 7f1555eed640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56055d1e8d80 0x56055d5cb530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:05.416 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.413+0000 7f1555eed640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56055d1e8d80 0x56055d5cb530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:35734/0 (socket says 172.21.15.42:35734) 2024-08-30T09:04:05.416 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.413+0000 7f1555eed640 1 -- 172.21.15.42:0/2841632674 learned_addr learned my addr 172.21.15.42:0/2841632674 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:04:05.416 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.413+0000 7f15566ee640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56055d2094d0 0x56055d5cae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:05.417 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.414+0000 7f1555eed640 1 -- 172.21.15.42:0/2841632674 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56055d2094d0 msgr2=0x56055d5cae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:04:05.417 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.414+0000 7f1555eed640 1 --2- 172.21.15.42:0/2841632674 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56055d2094d0 0x56055d5cae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:05.418 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.414+0000 7f1555eed640 1 -- 172.21.15.42:0/2841632674 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56055d2012f0 msgr2=0x56055d5ca8c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-30T09:04:05.418 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.414+0000 7f1555eed640 1 --2- 172.21.15.42:0/2841632674 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56055d2012f0 0x56055d5ca8c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:05.418 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.414+0000 7f1555eed640 1 -- 172.21.15.42:0/2841632674 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56055d5cb340 con 0x56055d1e8d80 2024-08-30T09:04:05.418 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.414+0000 7f1555eed640 1 --2- 172.21.15.42:0/2841632674 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56055d1e8d80 0x56055d5cb530 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f154c000c00 tx=0x7f154c004980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5f5832a8316c469b server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:04:05.418 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.414+0000 7f153b7fe640 1 -- 172.21.15.42:0/2841632674 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f154c0ac040 con 0x56055d1e8d80 2024-08-30T09:04:05.418 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.414+0000 7f153b7fe640 1 -- 172.21.15.42:0/2841632674 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f154c0b08b0 con 0x56055d1e8d80 2024-08-30T09:04:05.418 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.414+0000 7f153b7fe640 1 -- 172.21.15.42:0/2841632674 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f154c0ac2a0 con 0x56055d1e8d80 2024-08-30T09:04:05.418 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.415+0000 7f155f0b8740 1 -- 172.21.15.42:0/2841632674 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56055d1e8d80 msgr2=0x56055d5cb530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:04:05.418 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.415+0000 7f155f0b8740 1 --2- 172.21.15.42:0/2841632674 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56055d1e8d80 0x56055d5cb530 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f154c000c00 tx=0x7f154c004980 comp rx=0 tx=0).stop 2024-08-30T09:04:05.418 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.415+0000 7f155f0b8740 1 -- 172.21.15.42:0/2841632674 shutdown_connections 2024-08-30T09:04:05.418 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.415+0000 7f155f0b8740 1 --2- 172.21.15.42:0/2841632674 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56055d2094d0 0x56055d5cae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:05.418 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.415+0000 7f155f0b8740 1 --2- 172.21.15.42:0/2841632674 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56055d2012f0 0x56055d5ca8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:05.418 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.415+0000 7f155f0b8740 1 --2- 172.21.15.42:0/2841632674 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56055d1e8d80 0x56055d5cb530 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:05.418 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.415+0000 7f155f0b8740 1 -- 172.21.15.42:0/2841632674 >> 172.21.15.42:0/2841632674 conn(0x56055d21e300 msgr2=0x56055d5d1190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:04:05.419 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.415+0000 7f155f0b8740 1 -- 172.21.15.42:0/2841632674 shutdown_connections 2024-08-30T09:04:05.419 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.415+0000 7f155f0b8740 1 -- 172.21.15.42:0/2841632674 wait complete. 2024-08-30T09:04:05.419 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.416+0000 7f155f0b8740 1 -- start start 2024-08-30T09:04:05.419 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.416+0000 7f155f0b8740 1 Processor -- start 2024-08-30T09:04:05.421 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.416+0000 7f155f0b8740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56055d1e8d80 0x56055d5e1130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:05.422 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.416+0000 7f155f0b8740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56055d2012f0 0x56055d5dfdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:05.422 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.417+0000 7f155f0b8740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56055d2094d0 0x56055d5e0330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:05.422 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.417+0000 7f155f0b8740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f15501c2000 con 0x56055d2012f0 2024-08-30T09:04:05.422 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.417+0000 7f1555eed640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56055d1e8d80 0x56055d5e1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:05.422 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.417+0000 7f15556ec640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56055d2012f0 0x56055d5dfdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:05.422 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.417+0000 7f1555eed640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56055d1e8d80 0x56055d5e1130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:35746/0 (socket says 172.21.15.42:35746) 2024-08-30T09:04:05.422 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.417+0000 7f15556ec640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56055d2012f0 0x56055d5dfdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37238/0 (socket says 172.21.15.42:37238) 2024-08-30T09:04:05.422 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.417+0000 7f1555eed640 1 -- 172.21.15.42:0/4002102046 learned_addr learned my addr 172.21.15.42:0/4002102046 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:04:05.422 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.417+0000 7f155f0b8740 1 -- 172.21.15.42:0/4002102046 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f15501c3470 con 0x56055d2094d0 2024-08-30T09:04:05.422 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.417+0000 7f155f0b8740 1 -- 172.21.15.42:0/4002102046 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f15501c26d0 con 0x56055d1e8d80 2024-08-30T09:04:05.422 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.417+0000 7f155f0b8740 10 client.0 _ll_register_callbacks cb 0x7f154c00ef70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:04:05.423 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.417+0000 7f15566ee640 1 --2- 172.21.15.42:0/4002102046 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56055d2094d0 0x56055d5e0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:05.423 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.417+0000 7f1555eed640 1 -- 172.21.15.42:0/4002102046 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56055d2094d0 msgr2=0x56055d5e0330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:04:05.423 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.417+0000 7f1555eed640 1 --2- 172.21.15.42:0/4002102046 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56055d2094d0 0x56055d5e0330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:05.423 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.417+0000 7f1555eed640 1 -- 172.21.15.42:0/4002102046 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56055d2012f0 msgr2=0x56055d5dfdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:04:05.423 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.417+0000 7f1555eed640 1 --2- 172.21.15.42:0/4002102046 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56055d2012f0 0x56055d5dfdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:05.423 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.417+0000 7f1555eed640 1 -- 172.21.15.42:0/4002102046 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56055d5e0870 con 0x56055d1e8d80 2024-08-30T09:04:05.423 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.417+0000 7f1555eed640 1 --2- 172.21.15.42:0/4002102046 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56055d1e8d80 0x56055d5e1130 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f154c0abd40 tx=0x7f154c0048f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c2ec2c5cb2fb80ab server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:04:05.423 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.417+0000 7f15397fa640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-08-30T09:04:05.423 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.417+0000 7f15397fa640 1 -- 172.21.15.42:0/4002102046 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f154c0b5360 con 0x56055d1e8d80 2024-08-30T09:04:05.423 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.418+0000 7f15397fa640 1 -- 172.21.15.42:0/4002102046 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f154c0ac2a0 con 0x56055d1e8d80 2024-08-30T09:04:05.423 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.418+0000 7f155f0b8740 1 -- 172.21.15.42:0/4002102046 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f154c0b08b0 con 0x56055d1e8d80 2024-08-30T09:04:05.423 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.418+0000 7f15397fa640 1 -- 172.21.15.42:0/4002102046 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f154c0ac4a0 con 0x56055d1e8d80 2024-08-30T09:04:05.423 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.418+0000 7f155f0b8740 10 client.25747 fetch_fsmap learned FSMap version 29 2024-08-30T09:04:05.424 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.418+0000 7f155f0b8740 1 -- 172.21.15.42:0/4002102046 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56055d5df720 con 0x56055d1e8d80 2024-08-30T09:04:05.424 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.418+0000 7f15397fa640 1 -- 172.21.15.42:0/4002102046 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f154c0b08b0 con 0x56055d1e8d80 2024-08-30T09:04:05.424 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.418+0000 7f15397fa640 1 -- 172.21.15.42:0/4002102046 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f154c0ab2a0 con 0x56055d1e8d80 2024-08-30T09:04:05.424 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.419+0000 7f155f0b8740 10 client.25747 fetch_fsmap finished waiting for FSMap version 29 2024-08-30T09:04:05.424 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.419+0000 7f155f0b8740 10 client.25747 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-30T09:04:05.424 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.419+0000 7f155f0b8740 20 client.25747 populate_metadata read hostname 'smithi042' 2024-08-30T09:04:05.424 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.419+0000 7f155f0b8740 1 --2- 172.21.15.42:0/4002102046 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x56055d22a1b0 0x56055d654700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:05.424 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.419+0000 7f155f0b8740 4 client.25747 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-30T09:04:05.424 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.419+0000 7f155f0b8740 1 -- 172.21.15.42:0/4002102046 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56055d544f50 con 0x56055d22a1b0 2024-08-30T09:04:05.424 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.419+0000 7f15556ec640 1 --2- 172.21.15.42:0/4002102046 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x56055d22a1b0 0x56055d654700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:05.424 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.419+0000 7f15397fa640 10 client.25747 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:04:05.424 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.419+0000 7f15556ec640 1 --2- 172.21.15.42:0/4002102046 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x56055d22a1b0 0x56055d654700 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=52016176e9476c1d server_cookie=51466018018568e7 in_seq=0 out_seq=0 2024-08-30T09:04:05.482 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.480+0000 7f15397fa640 1 -- 172.21.15.42:0/4002102046 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56055d544f50 con 0x56055d22a1b0 2024-08-30T09:04:05.483 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.480+0000 7f15397fa640 10 client.25747 handle_command_reply: tid=0 2024-08-30T09:04:05.483 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.480+0000 7f155f0b8740 1 client.25747 shutdown 2024-08-30T09:04:05.484 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.481+0000 7f155f0b8740 1 -- 172.21.15.42:0/4002102046 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56055d1e8d80 msgr2=0x56055d5e1130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:04:05.484 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.481+0000 7f155f0b8740 1 --2- 172.21.15.42:0/4002102046 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56055d1e8d80 0x56055d5e1130 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f154c0abd40 tx=0x7f154c0048f0 comp rx=0 tx=0).stop 2024-08-30T09:04:05.485 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.483+0000 7f155f0b8740 1 -- 172.21.15.42:0/4002102046 shutdown_connections 2024-08-30T09:04:05.485 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.483+0000 7f155f0b8740 1 --2- 172.21.15.42:0/4002102046 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x56055d22a1b0 0x56055d654700 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:05.485 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.483+0000 7f155f0b8740 1 --2- 172.21.15.42:0/4002102046 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56055d2094d0 0x56055d5e0330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:05.485 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.483+0000 7f155f0b8740 1 --2- 172.21.15.42:0/4002102046 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56055d2012f0 0x56055d5dfdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:05.485 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.483+0000 7f155f0b8740 1 --2- 172.21.15.42:0/4002102046 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56055d1e8d80 0x56055d5e1130 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:05.485 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.483+0000 7f155f0b8740 1 -- 172.21.15.42:0/4002102046 >> 172.21.15.42:0/4002102046 conn(0x56055d21e300 msgr2=0x56055d5cb530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:04:05.485 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.483+0000 7f15397fa640 0 client.25747 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:04:05.488 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.485+0000 7f155f0b8740 1 -- 172.21.15.42:0/4002102046 shutdown_connections 2024-08-30T09:04:05.488 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.486+0000 7f155f0b8740 1 -- 172.21.15.42:0/4002102046 wait complete. 2024-08-30T09:04:05.488 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.486+0000 7f155f0b8740 20 client.25747 trim_cache size 0 max 16384 2024-08-30T09:04:05.490 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-08-30T09:04:05.490 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2024-08-30T09:04:05.490 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-08-30T09:04:05.490 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-08-30T09:04:05.491 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.489+0000 7f1558151640 1 -- 172.21.15.42:0/2056765597 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f153c0b4c30 msgr2=0x7f153c0b7140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:04:05.491 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.489+0000 7f1558151640 1 --2- 172.21.15.42:0/2056765597 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f153c0b4c30 0x7f153c0b7140 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f1548046000 tx=0x7f1548031980 comp rx=0 tx=0).stop 2024-08-30T09:04:05.491 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.489+0000 7f1558151640 1 -- 172.21.15.42:0/2056765597 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f155010a160 msgr2=0x7f155011ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:04:05.491 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.489+0000 7f1558151640 1 --2- 172.21.15.42:0/2056765597 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f155010a160 0x7f155011ada0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f154c00e9c0 tx=0x7f154c00edc0 comp rx=0 tx=0).stop 2024-08-30T09:04:05.493 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.491+0000 7f1558151640 1 -- 172.21.15.42:0/2056765597 shutdown_connections 2024-08-30T09:04:05.493 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.491+0000 7f1558151640 1 --2- 172.21.15.42:0/2056765597 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f153c0b4c30 0x7f153c0b7140 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:05.493 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.491+0000 7f1558151640 1 --2- 172.21.15.42:0/2056765597 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f155010f5d0 0x7f1550119960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:05.493 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.491+0000 7f1558151640 1 --2- 172.21.15.42:0/2056765597 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f155010cac0 0x7f155011b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:05.493 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.491+0000 7f1558151640 1 --2- 172.21.15.42:0/2056765597 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f155010a160 0x7f155011ada0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:05.494 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.491+0000 7f1558151640 1 -- 172.21.15.42:0/2056765597 >> 172.21.15.42:0/2056765597 conn(0x7f1550070410 msgr2=0x7f1550070820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:04:05.494 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.492+0000 7f1558151640 1 -- 172.21.15.42:0/2056765597 shutdown_connections 2024-08-30T09:04:05.496 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:05.494+0000 7f1558151640 1 -- 172.21.15.42:0/2056765597 wait complete. 2024-08-30T09:04:05.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.919+0000 7fa477fff640 1 -- 172.21.15.42:0/3691463020 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa47810d990 msgr2=0x7fa47810dda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:04:05.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.919+0000 7fa477fff640 1 --2- 172.21.15.42:0/3691463020 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa47810d990 0x7fa47810dda0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fa46c009900 tx=0x7fa46c02a440 comp rx=0 tx=0).stop 2024-08-30T09:04:05.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.924+0000 7fa477fff640 1 -- 172.21.15.42:0/3691463020 shutdown_connections 2024-08-30T09:04:05.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.924+0000 7fa477fff640 1 --2- 172.21.15.42:0/3691463020 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa47811c940 0x7fa47811ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:05.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.924+0000 7fa477fff640 1 --2- 172.21.15.42:0/3691463020 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa47810d990 0x7fa47810dda0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:05.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.924+0000 7fa477fff640 1 --2- 172.21.15.42:0/3691463020 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa478111a70 0x7fa47810d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:05.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.924+0000 7fa477fff640 1 -- 172.21.15.42:0/3691463020 >> 172.21.15.42:0/3691463020 conn(0x7fa47806e790 msgr2=0x7fa47806ebd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:04:05.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.926+0000 7fa477fff640 1 -- 172.21.15.42:0/3691463020 shutdown_connections 2024-08-30T09:04:05.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.927+0000 7fa477fff640 1 -- 172.21.15.42:0/3691463020 wait complete. 2024-08-30T09:04:05.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.928+0000 7fa477fff640 1 Processor -- start 2024-08-30T09:04:05.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.929+0000 7fa477fff640 1 -- start start 2024-08-30T09:04:05.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.929+0000 7fa477fff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa47810d990 0x7fa4781a8ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:05.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.929+0000 7fa477fff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa478111a70 0x7fa4781a9440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:05.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.929+0000 7fa477fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa47811c940 0x7fa4781ae6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:05.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.929+0000 7fa477fff640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fa478122510 con 0x7fa47811c940 2024-08-30T09:04:05.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.929+0000 7fa477fff640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa478122390 con 0x7fa478111a70 2024-08-30T09:04:05.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.929+0000 7fa477fff640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fa478122690 con 0x7fa47810d990 2024-08-30T09:04:05.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.930+0000 7fa476ffd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa47810d990 0x7fa4781a8ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:05.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.930+0000 7fa476ffd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa47810d990 0x7fa4781a8ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:35758/0 (socket says 172.21.15.42:35758) 2024-08-30T09:04:05.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.930+0000 7fa476ffd640 1 -- 172.21.15.42:0/1959461317 learned_addr learned my addr 172.21.15.42:0/1959461317 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:04:05.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.930+0000 7fa4777fe640 1 --2- 172.21.15.42:0/1959461317 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa47811c940 0x7fa4781ae6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:05.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.931+0000 7fa4767fc640 1 --2- 172.21.15.42:0/1959461317 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa478111a70 0x7fa4781a9440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:05.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.931+0000 7fa476ffd640 1 -- 172.21.15.42:0/1959461317 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa478111a70 msgr2=0x7fa4781a9440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:04:05.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.931+0000 7fa476ffd640 1 --2- 172.21.15.42:0/1959461317 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa478111a70 0x7fa4781a9440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:05.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.931+0000 7fa476ffd640 1 -- 172.21.15.42:0/1959461317 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa47811c940 msgr2=0x7fa4781ae6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:04:05.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.931+0000 7fa476ffd640 1 --2- 172.21.15.42:0/1959461317 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa47811c940 0x7fa4781ae6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:05.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.931+0000 7fa476ffd640 1 -- 172.21.15.42:0/1959461317 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4781aede0 con 0x7fa47810d990 2024-08-30T09:04:05.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.932+0000 7fa476ffd640 1 --2- 172.21.15.42:0/1959461317 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa47810d990 0x7fa4781a8ea0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fa46800b4a0 tx=0x7fa46800b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:04:05.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.932+0000 7fa45ffff640 1 -- 172.21.15.42:0/1959461317 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa468014070 con 0x7fa47810d990 2024-08-30T09:04:05.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.932+0000 7fa45ffff640 1 -- 172.21.15.42:0/1959461317 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa468010ac0 con 0x7fa47810d990 2024-08-30T09:04:05.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.933+0000 7fa45ffff640 1 -- 172.21.15.42:0/1959461317 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa468017c20 con 0x7fa47810d990 2024-08-30T09:04:05.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.933+0000 7fa477fff640 1 -- 172.21.15.42:0/1959461317 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4781af0d0 con 0x7fa47810d990 2024-08-30T09:04:05.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.933+0000 7fa477fff640 1 -- 172.21.15.42:0/1959461317 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa478112120 con 0x7fa47810d990 2024-08-30T09:04:05.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.933+0000 7fa4777fe640 1 --2- 172.21.15.42:0/1959461317 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa47811c940 0x7fa4781ae6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-30T09:04:05.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.936+0000 7fa45ffff640 1 -- 172.21.15.42:0/1959461317 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7fa468017070 con 0x7fa47810d990 2024-08-30T09:04:05.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.937+0000 7fa483b3d740 1 Processor -- start 2024-08-30T09:04:05.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.937+0000 7fa483b3d740 1 -- start start 2024-08-30T09:04:05.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.937+0000 7fa483b3d740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558f81211070 0x558f815b64b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:05.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.937+0000 7fa483b3d740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558f811e81b0 0x558f815b69f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:05.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.937+0000 7fa483b3d740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558f811f0390 0x558f815b6f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:05.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.938+0000 7fa483b3d740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x558f8152d680 con 0x558f811f0390 2024-08-30T09:04:05.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.938+0000 7fa483b3d740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x558f8151cc10 con 0x558f81211070 2024-08-30T09:04:05.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.938+0000 7fa483b3d740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x558f81520bc0 con 0x558f811e81b0 2024-08-30T09:04:05.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.938+0000 7fa4777fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558f81211070 0x558f815b64b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:05.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.938+0000 7fa4777fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558f81211070 0x558f815b64b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:40492/0 (socket says 172.21.15.42:40492) 2024-08-30T09:04:05.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.938+0000 7fa4777fe640 1 -- 172.21.15.42:0/1507755360 learned_addr learned my addr 172.21.15.42:0/1507755360 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:04:05.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.938+0000 7fa45ffff640 1 --2- 172.21.15.42:0/1959461317 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fa4600b4be0 0x7fa4600b70f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:05.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.938+0000 7fa476ffd640 1 --2- 172.21.15.42:0/1507755360 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558f811e81b0 0x558f815b69f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:05.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.939+0000 7fa45ffff640 1 -- 172.21.15.42:0/1959461317 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7fa4680e4620 con 0x7fa47810d990 2024-08-30T09:04:05.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.939+0000 7fa476ffd640 1 -- 172.21.15.42:0/1507755360 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558f81211070 msgr2=0x558f815b64b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:04:05.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.939+0000 7fa476ffd640 1 --2- 172.21.15.42:0/1507755360 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558f81211070 0x558f815b64b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:05.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.939+0000 7fa476ffd640 1 -- 172.21.15.42:0/1507755360 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558f811f0390 msgr2=0x558f815b6f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:04:05.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.939+0000 7fa476ffd640 1 --2- 172.21.15.42:0/1507755360 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558f811f0390 0x558f815b6f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:05.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.939+0000 7fa476ffd640 1 -- 172.21.15.42:0/1507755360 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558f8127c650 con 0x558f811e81b0 2024-08-30T09:04:05.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.939+0000 7fa476ffd640 1 --2- 172.21.15.42:0/1507755360 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558f811e81b0 0x558f815b69f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fa4680e1de0 tx=0x7fa4680176c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f0777a2fa6136906 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:04:05.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.939+0000 7fa4777fe640 1 --2- 172.21.15.42:0/1959461317 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fa4600b4be0 0x7fa4600b70f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:05.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.939+0000 7fa45d7fa640 1 -- 172.21.15.42:0/1507755360 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa46801e070 con 0x558f811e81b0 2024-08-30T09:04:05.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.940+0000 7fa45d7fa640 1 -- 172.21.15.42:0/1507755360 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa4680af530 con 0x558f811e81b0 2024-08-30T09:04:05.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.940+0000 7fa45d7fa640 1 -- 172.21.15.42:0/1507755360 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4680a7000 con 0x558f811e81b0 2024-08-30T09:04:05.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.941+0000 7fa483b3d740 1 -- 172.21.15.42:0/1507755360 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558f811e81b0 msgr2=0x558f815b69f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:04:05.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.941+0000 7fa4777fe640 1 --2- 172.21.15.42:0/1959461317 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fa4600b4be0 0x7fa4600b70f0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fa470009fd0 tx=0x7fa470004ab0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:04:05.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.941+0000 7fa483b3d740 1 --2- 172.21.15.42:0/1507755360 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558f811e81b0 0x558f815b69f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fa4680e1de0 tx=0x7fa4680176c0 comp rx=0 tx=0).stop 2024-08-30T09:04:05.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.941+0000 7fa483b3d740 1 -- 172.21.15.42:0/1507755360 shutdown_connections 2024-08-30T09:04:05.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.941+0000 7fa483b3d740 1 --2- 172.21.15.42:0/1507755360 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558f811e81b0 0x558f815b69f0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:05.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.941+0000 7fa483b3d740 1 --2- 172.21.15.42:0/1507755360 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558f811f0390 0x558f815b6f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:05.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.941+0000 7fa483b3d740 1 --2- 172.21.15.42:0/1507755360 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558f81211070 0x558f815b64b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:05.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.941+0000 7fa483b3d740 1 -- 172.21.15.42:0/1507755360 >> 172.21.15.42:0/1507755360 conn(0x558f81205460 msgr2=0x558f815b43a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:04:05.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.942+0000 7fa483b3d740 1 -- 172.21.15.42:0/1507755360 shutdown_connections 2024-08-30T09:04:05.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.942+0000 7fa483b3d740 1 -- 172.21.15.42:0/1507755360 wait complete. 2024-08-30T09:04:05.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.942+0000 7fa483b3d740 1 -- start start 2024-08-30T09:04:05.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.943+0000 7fa483b3d740 1 Processor -- start 2024-08-30T09:04:05.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.943+0000 7fa483b3d740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558f811e81b0 0x558f815c1c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:05.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.943+0000 7fa483b3d740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558f811f0390 0x558f815c2190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:05.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.943+0000 7fa483b3d740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558f81211070 0x558f815ce020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:05.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.943+0000 7fa483b3d740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x558f8137f700 con 0x558f811e81b0 2024-08-30T09:04:05.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.943+0000 7fa483b3d740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x558f814a3750 con 0x558f811f0390 2024-08-30T09:04:05.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.943+0000 7fa483b3d740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x558f815c4d30 con 0x558f81211070 2024-08-30T09:04:05.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.943+0000 7fa483b3d740 10 client.0 _ll_register_callbacks cb 0x558f814d2e10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:04:05.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.943+0000 7fa476ffd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558f811e81b0 0x558f815c1c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:05.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.943+0000 7fa4767fc640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558f811f0390 0x558f815c2190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:05.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.943+0000 7fa4767fc640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558f811f0390 0x558f815c2190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:40500/0 (socket says 172.21.15.42:40500) 2024-08-30T09:04:05.949 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.943+0000 7fa4767fc640 1 -- 172.21.15.42:0/2104858376 learned_addr learned my addr 172.21.15.42:0/2104858376 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:04:05.949 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.943+0000 7fa476ffd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558f811e81b0 0x558f815c1c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37270/0 (socket says 172.21.15.42:37270) 2024-08-30T09:04:05.949 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.944+0000 7fa4777fe640 1 --2- 172.21.15.42:0/2104858376 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558f81211070 0x558f815ce020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:05.949 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.944+0000 7fa4767fc640 1 -- 172.21.15.42:0/2104858376 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558f81211070 msgr2=0x558f815ce020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:04:05.949 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.944+0000 7fa4767fc640 1 --2- 172.21.15.42:0/2104858376 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558f81211070 0x558f815ce020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:05.949 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.944+0000 7fa4767fc640 1 -- 172.21.15.42:0/2104858376 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558f811e81b0 msgr2=0x558f815c1c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:04:05.949 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.944+0000 7fa4767fc640 1 --2- 172.21.15.42:0/2104858376 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558f811e81b0 0x558f815c1c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:05.949 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.944+0000 7fa4767fc640 1 -- 172.21.15.42:0/2104858376 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558f814dc900 con 0x558f811f0390 2024-08-30T09:04:05.949 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.944+0000 7fa4767fc640 1 --2- 172.21.15.42:0/2104858376 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558f811f0390 0x558f815c2190 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fa46c006010 tx=0x7fa46c031670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cb0f289af42c6e95 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:04:05.949 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.944+0000 7fa45aff5640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-30T09:04:05.949 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.944+0000 7fa45aff5640 1 -- 172.21.15.42:0/2104858376 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa46c031bf0 con 0x558f811f0390 2024-08-30T09:04:05.949 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.944+0000 7fa45aff5640 1 -- 172.21.15.42:0/2104858376 <== mon.2 v2:172.21.15.167:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7fa46c031d90 con 0x558f811f0390 2024-08-30T09:04:05.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.944+0000 7fa483b3d740 1 -- 172.21.15.42:0/2104858376 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa4680af530 con 0x558f811f0390 2024-08-30T09:04:05.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.945+0000 7fa45aff5640 1 -- 172.21.15.42:0/2104858376 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa46c047470 con 0x558f811f0390 2024-08-30T09:04:05.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.945+0000 7fa483b3d740 10 client.25745 fetch_fsmap learned FSMap version 29 2024-08-30T09:04:05.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.945+0000 7fa483b3d740 1 -- 172.21.15.42:0/2104858376 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558f815d0b30 con 0x558f811f0390 2024-08-30T09:04:05.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.945+0000 7fa45aff5640 1 -- 172.21.15.42:0/2104858376 <== mon.2 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa4680af530 con 0x558f811f0390 2024-08-30T09:04:05.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.945+0000 7fa45aff5640 1 -- 172.21.15.42:0/2104858376 <== mon.2 v2:172.21.15.167:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fa46c03d4c0 con 0x558f811f0390 2024-08-30T09:04:05.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.945+0000 7fa483b3d740 10 client.25745 fetch_fsmap finished waiting for FSMap version 29 2024-08-30T09:04:05.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.945+0000 7fa483b3d740 10 client.25745 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-30T09:04:05.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.945+0000 7fa483b3d740 20 client.25745 populate_metadata read hostname 'smithi042' 2024-08-30T09:04:05.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.945+0000 7fa483b3d740 1 --2- 172.21.15.42:0/2104858376 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x558f811cf9f0 0x558f815d4040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:05.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.945+0000 7fa483b3d740 4 client.25745 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-30T09:04:05.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.945+0000 7fa483b3d740 1 -- 172.21.15.42:0/2104858376 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558f8152f5d0 con 0x558f811cf9f0 2024-08-30T09:04:05.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.946+0000 7fa476ffd640 1 --2- 172.21.15.42:0/2104858376 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x558f811cf9f0 0x558f815d4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:05.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.946+0000 7fa45aff5640 10 client.25745 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:04:05.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.946+0000 7fa476ffd640 1 --2- 172.21.15.42:0/2104858376 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x558f811cf9f0 0x558f815d4040 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4c7eac366ebf0051 server_cookie=8cb657b37b989044 in_seq=0 out_seq=0 2024-08-30T09:04:05.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.947+0000 7fa45aff5640 1 -- 172.21.15.42:0/2104858376 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x558f8152f5d0 con 0x558f811cf9f0 2024-08-30T09:04:05.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.947+0000 7fa45aff5640 10 client.25745 handle_command_reply: tid=0 2024-08-30T09:04:05.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.947+0000 7fa483b3d740 1 client.25745 shutdown 2024-08-30T09:04:05.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.947+0000 7fa483b3d740 1 -- 172.21.15.42:0/2104858376 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558f811f0390 msgr2=0x558f815c2190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:04:05.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.947+0000 7fa483b3d740 1 --2- 172.21.15.42:0/2104858376 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558f811f0390 0x558f815c2190 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fa46c006010 tx=0x7fa46c031670 comp rx=0 tx=0).stop 2024-08-30T09:04:05.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.947+0000 7fa483b3d740 1 -- 172.21.15.42:0/2104858376 shutdown_connections 2024-08-30T09:04:05.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.947+0000 7fa483b3d740 1 --2- 172.21.15.42:0/2104858376 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x558f811cf9f0 0x558f815d4040 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:05.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.947+0000 7fa483b3d740 1 --2- 172.21.15.42:0/2104858376 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558f81211070 0x558f815ce020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:05.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.947+0000 7fa483b3d740 1 --2- 172.21.15.42:0/2104858376 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558f811f0390 0x558f815c2190 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:05.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.947+0000 7fa45aff5640 0 client.25745 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:04:05.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.947+0000 7fa483b3d740 1 --2- 172.21.15.42:0/2104858376 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558f811e81b0 0x558f815c1c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:05.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.947+0000 7fa483b3d740 1 -- 172.21.15.42:0/2104858376 >> 172.21.15.42:0/2104858376 conn(0x558f81205460 msgr2=0x558f815b74e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:04:05.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.947+0000 7fa483b3d740 1 -- 172.21.15.42:0/2104858376 shutdown_connections 2024-08-30T09:04:05.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.947+0000 7fa483b3d740 1 -- 172.21.15.42:0/2104858376 wait complete. 2024-08-30T09:04:05.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.948+0000 7fa483b3d740 20 client.25745 trim_cache size 0 max 16384 2024-08-30T09:04:05.953 INFO:teuthology.orchestra.run.smithi042.stderr:progress: 200.0 s, 1268.2 tps, lat 79.082 ms stddev 106.585 2024-08-30T09:04:05.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.967+0000 7fa483b3d740 1 Processor -- start 2024-08-30T09:04:05.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.967+0000 7fa483b3d740 1 -- start start 2024-08-30T09:04:05.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.967+0000 7fa483b3d740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558f811e81b0 0x558f815b4410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:05.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.967+0000 7fa483b3d740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558f811f0390 0x558f815cdf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:05.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.967+0000 7fa483b3d740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558f81211070 0x558f815ce460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:05.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.967+0000 7fa483b3d740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fa4781c9e60 con 0x558f81211070 2024-08-30T09:04:05.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.967+0000 7fa483b3d740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa4781ca530 con 0x558f811f0390 2024-08-30T09:04:05.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.967+0000 7fa483b3d740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fa4781c7e00 con 0x558f811e81b0 2024-08-30T09:04:05.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.968+0000 7fa4777fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558f81211070 0x558f815ce460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:05.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.968+0000 7fa4777fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558f81211070 0x558f815ce460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37286/0 (socket says 172.21.15.42:37286) 2024-08-30T09:04:05.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.968+0000 7fa4777fe640 1 -- 172.21.15.42:0/226727959 learned_addr learned my addr 172.21.15.42:0/226727959 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:04:05.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.968+0000 7fa476ffd640 1 --2- 172.21.15.42:0/226727959 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558f811e81b0 0x558f815b4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:05.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.968+0000 7fa4767fc640 1 --2- 172.21.15.42:0/226727959 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558f811f0390 0x558f815cdf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:05.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.968+0000 7fa4777fe640 1 -- 172.21.15.42:0/226727959 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558f811e81b0 msgr2=0x558f815b4410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:04:05.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.968+0000 7fa4777fe640 1 --2- 172.21.15.42:0/226727959 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558f811e81b0 0x558f815b4410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:05.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.968+0000 7fa4777fe640 1 -- 172.21.15.42:0/226727959 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558f811f0390 msgr2=0x558f815cdf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:04:05.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.968+0000 7fa4777fe640 1 --2- 172.21.15.42:0/226727959 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558f811f0390 0x558f815cdf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:05.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.968+0000 7fa4777fe640 1 -- 172.21.15.42:0/226727959 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558f815c2f10 con 0x558f81211070 2024-08-30T09:04:05.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.968+0000 7fa476ffd640 1 --2- 172.21.15.42:0/226727959 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558f811e81b0 0x558f815b4410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-30T09:04:05.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.968+0000 7fa4767fc640 1 --2- 172.21.15.42:0/226727959 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558f811f0390 0x558f815cdf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-30T09:04:05.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.969+0000 7fa4777fe640 1 --2- 172.21.15.42:0/226727959 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558f81211070 0x558f815ce460 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fa470011a40 tx=0x7fa470018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26ebc135e404e52f server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:04:05.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.969+0000 7fa45cff9640 1 -- 172.21.15.42:0/226727959 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa470021a20 con 0x558f81211070 2024-08-30T09:04:05.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.969+0000 7fa45cff9640 1 -- 172.21.15.42:0/226727959 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa470021bc0 con 0x558f81211070 2024-08-30T09:04:05.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.969+0000 7fa45cff9640 1 -- 172.21.15.42:0/226727959 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa47001c680 con 0x558f81211070 2024-08-30T09:04:05.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.970+0000 7fa483b3d740 1 -- 172.21.15.42:0/226727959 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558f81211070 msgr2=0x558f815ce460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:04:05.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.970+0000 7fa483b3d740 1 --2- 172.21.15.42:0/226727959 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558f81211070 0x558f815ce460 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fa470011a40 tx=0x7fa470018000 comp rx=0 tx=0).stop 2024-08-30T09:04:05.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.970+0000 7fa483b3d740 1 -- 172.21.15.42:0/226727959 shutdown_connections 2024-08-30T09:04:05.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.970+0000 7fa483b3d740 1 --2- 172.21.15.42:0/226727959 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558f81211070 0x558f815ce460 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:05.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.970+0000 7fa483b3d740 1 --2- 172.21.15.42:0/226727959 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558f811f0390 0x558f815cdf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:05.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.970+0000 7fa483b3d740 1 --2- 172.21.15.42:0/226727959 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558f811e81b0 0x558f815b4410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:05.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.970+0000 7fa483b3d740 1 -- 172.21.15.42:0/226727959 >> 172.21.15.42:0/226727959 conn(0x558f81205460 msgr2=0x558f815b3d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:04:05.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.970+0000 7fa483b3d740 1 -- 172.21.15.42:0/226727959 shutdown_connections 2024-08-30T09:04:05.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.970+0000 7fa483b3d740 1 -- 172.21.15.42:0/226727959 wait complete. 2024-08-30T09:04:05.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.970+0000 7fa483b3d740 1 -- start start 2024-08-30T09:04:05.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.971+0000 7fa483b3d740 1 Processor -- start 2024-08-30T09:04:05.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.971+0000 7fa483b3d740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558f811e81b0 0x558f815ca2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:05.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.971+0000 7fa483b3d740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558f811f0390 0x558f815ca890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:05.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.971+0000 7fa483b3d740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558f81211070 0x558f815c4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:05.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.971+0000 7fa483b3d740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fa47811a3c0 con 0x558f811f0390 2024-08-30T09:04:05.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.971+0000 7fa483b3d740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa4781c7730 con 0x558f81211070 2024-08-30T09:04:05.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.971+0000 7fa483b3d740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fa4781c6990 con 0x558f811e81b0 2024-08-30T09:04:05.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.971+0000 7fa483b3d740 10 client.0 _ll_register_callbacks cb 0x7fa46c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:04:05.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.971+0000 7fa476ffd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558f811e81b0 0x558f815ca2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:05.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.971+0000 7fa476ffd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558f811e81b0 0x558f815ca2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:35796/0 (socket says 172.21.15.42:35796) 2024-08-30T09:04:05.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.971+0000 7fa476ffd640 1 -- 172.21.15.42:0/1477357295 learned_addr learned my addr 172.21.15.42:0/1477357295 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:04:05.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.971+0000 7fa4767fc640 1 --2- 172.21.15.42:0/1477357295 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558f811f0390 0x558f815ca890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:05.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.971+0000 7fa4777fe640 1 --2- 172.21.15.42:0/1477357295 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558f81211070 0x558f815c4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:05.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.971+0000 7fa476ffd640 1 -- 172.21.15.42:0/1477357295 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558f81211070 msgr2=0x558f815c4fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:04:05.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.971+0000 7fa476ffd640 1 --2- 172.21.15.42:0/1477357295 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558f81211070 0x558f815c4fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:05.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.971+0000 7fa476ffd640 1 -- 172.21.15.42:0/1477357295 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558f811f0390 msgr2=0x558f815ca890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:04:05.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.971+0000 7fa476ffd640 1 --2- 172.21.15.42:0/1477357295 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558f811f0390 0x558f815ca890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:05.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.971+0000 7fa476ffd640 1 -- 172.21.15.42:0/1477357295 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558f815c5770 con 0x558f811e81b0 2024-08-30T09:04:05.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.972+0000 7fa476ffd640 1 --2- 172.21.15.42:0/1477357295 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558f811e81b0 0x558f815ca2d0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fa4680aaa70 tx=0x7fa468010ee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9c28cc6c0468c6f1 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:04:05.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.972+0000 7fa45aff5640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-08-30T09:04:05.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.972+0000 7fa45aff5640 1 -- 172.21.15.42:0/1477357295 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa46801e040 con 0x558f811e81b0 2024-08-30T09:04:05.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.972+0000 7fa45aff5640 1 -- 172.21.15.42:0/1477357295 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7fa468017dc0 con 0x558f811e81b0 2024-08-30T09:04:05.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.972+0000 7fa483b3d740 1 -- 172.21.15.42:0/1477357295 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa470021bc0 con 0x558f811e81b0 2024-08-30T09:04:05.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.972+0000 7fa45aff5640 1 -- 172.21.15.42:0/1477357295 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa4680e4330 con 0x558f811e81b0 2024-08-30T09:04:05.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.972+0000 7fa483b3d740 10 client.25765 fetch_fsmap learned FSMap version 29 2024-08-30T09:04:05.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.973+0000 7fa483b3d740 1 -- 172.21.15.42:0/1477357295 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558f8163f4d0 con 0x558f811e81b0 2024-08-30T09:04:05.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.973+0000 7fa45aff5640 1 -- 172.21.15.42:0/1477357295 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa470021bc0 con 0x558f811e81b0 2024-08-30T09:04:05.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.973+0000 7fa45aff5640 1 -- 172.21.15.42:0/1477357295 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fa4680af0d0 con 0x558f811e81b0 2024-08-30T09:04:05.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.973+0000 7fa483b3d740 10 client.25765 fetch_fsmap finished waiting for FSMap version 29 2024-08-30T09:04:05.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.973+0000 7fa483b3d740 10 client.25765 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-30T09:04:05.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.973+0000 7fa483b3d740 20 client.25765 populate_metadata read hostname 'smithi042' 2024-08-30T09:04:05.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.973+0000 7fa483b3d740 1 --2- 172.21.15.42:0/1477357295 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x558f811a4da0 0x558f81641810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:05.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.973+0000 7fa483b3d740 4 client.25765 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-30T09:04:05.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.973+0000 7fa483b3d740 1 -- 172.21.15.42:0/1477357295 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558f8152b070 con 0x558f811a4da0 2024-08-30T09:04:05.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.974+0000 7fa4767fc640 1 --2- 172.21.15.42:0/1477357295 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x558f811a4da0 0x558f81641810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:05.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.974+0000 7fa45aff5640 10 client.25765 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:04:05.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.974+0000 7fa4767fc640 1 --2- 172.21.15.42:0/1477357295 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x558f811a4da0 0x558f81641810 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f2f3aff26d7dbef1 server_cookie=87a2fa999b5b538b in_seq=0 out_seq=0 2024-08-30T09:04:05.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.974+0000 7fa45aff5640 1 -- 172.21.15.42:0/1477357295 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558f8152b070 con 0x558f811a4da0 2024-08-30T09:04:05.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.974+0000 7fa45aff5640 10 client.25765 handle_command_reply: tid=0 2024-08-30T09:04:05.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.975+0000 7fa483b3d740 1 client.25765 shutdown 2024-08-30T09:04:05.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.975+0000 7fa483b3d740 1 -- 172.21.15.42:0/1477357295 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558f811e81b0 msgr2=0x558f815ca2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:04:05.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.975+0000 7fa483b3d740 1 --2- 172.21.15.42:0/1477357295 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558f811e81b0 0x558f815ca2d0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fa4680aaa70 tx=0x7fa468010ee0 comp rx=0 tx=0).stop 2024-08-30T09:04:05.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.975+0000 7fa483b3d740 1 -- 172.21.15.42:0/1477357295 shutdown_connections 2024-08-30T09:04:05.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.975+0000 7fa483b3d740 1 --2- 172.21.15.42:0/1477357295 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x558f811a4da0 0x558f81641810 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:05.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.975+0000 7fa483b3d740 1 --2- 172.21.15.42:0/1477357295 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558f81211070 0x558f815c4fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:05.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.975+0000 7fa483b3d740 1 --2- 172.21.15.42:0/1477357295 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558f811f0390 0x558f815ca890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:05.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.975+0000 7fa483b3d740 1 --2- 172.21.15.42:0/1477357295 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558f811e81b0 0x558f815ca2d0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:05.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.975+0000 7fa483b3d740 1 -- 172.21.15.42:0/1477357295 >> 172.21.15.42:0/1477357295 conn(0x558f81205460 msgr2=0x558f815b74e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:04:05.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.975+0000 7fa45aff5640 0 client.25765 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:04:05.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.975+0000 7fa483b3d740 1 -- 172.21.15.42:0/1477357295 shutdown_connections 2024-08-30T09:04:05.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.975+0000 7fa483b3d740 1 -- 172.21.15.42:0/1477357295 wait complete. 2024-08-30T09:04:05.981 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.975+0000 7fa483b3d740 20 client.25765 trim_cache size 0 max 16384 2024-08-30T09:04:05.981 INFO:teuthology.orchestra.run.smithi042.stdout:[] 2024-08-30T09:04:05.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.976+0000 7fa477fff640 1 -- 172.21.15.42:0/1959461317 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fa4600b4be0 msgr2=0x7fa4600b70f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:04:05.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.976+0000 7fa477fff640 1 --2- 172.21.15.42:0/1959461317 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fa4600b4be0 0x7fa4600b70f0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fa470009fd0 tx=0x7fa470004ab0 comp rx=0 tx=0).stop 2024-08-30T09:04:05.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.976+0000 7fa477fff640 1 -- 172.21.15.42:0/1959461317 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa47810d990 msgr2=0x7fa4781a8ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:04:05.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.976+0000 7fa477fff640 1 --2- 172.21.15.42:0/1959461317 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa47810d990 0x7fa4781a8ea0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fa46800b4a0 tx=0x7fa46800b960 comp rx=0 tx=0).stop 2024-08-30T09:04:05.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.976+0000 7fa477fff640 1 -- 172.21.15.42:0/1959461317 shutdown_connections 2024-08-30T09:04:05.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.976+0000 7fa477fff640 1 --2- 172.21.15.42:0/1959461317 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fa4600b4be0 0x7fa4600b70f0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:05.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.976+0000 7fa477fff640 1 --2- 172.21.15.42:0/1959461317 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa47811c940 0x7fa4781ae6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:05.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.976+0000 7fa477fff640 1 --2- 172.21.15.42:0/1959461317 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa478111a70 0x7fa4781a9440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:05.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.976+0000 7fa477fff640 1 --2- 172.21.15.42:0/1959461317 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa47810d990 0x7fa4781a8ea0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:05.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.976+0000 7fa477fff640 1 -- 172.21.15.42:0/1959461317 >> 172.21.15.42:0/1959461317 conn(0x7fa47806e790 msgr2=0x7fa478110720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:04:05.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.976+0000 7fa477fff640 1 -- 172.21.15.42:0/1959461317 shutdown_connections 2024-08-30T09:04:05.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:05.976+0000 7fa477fff640 1 -- 172.21.15.42:0/1959461317 wait complete. 2024-08-30T09:04:06.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:04:06 smithi167 ceph-mon[22063]: pgmap v473: 129 pgs: 129 active+clean; 1.1 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 MiB/s wr, 714 op/s 2024-08-30T09:04:06.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:04:06 smithi042 ceph-mon[18652]: pgmap v473: 129 pgs: 129 active+clean; 1.1 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 MiB/s wr, 714 op/s 2024-08-30T09:04:06.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:04:06 smithi190 ceph-mon[22001]: pgmap v473: 129 pgs: 129 active+clean; 1.1 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 MiB/s wr, 714 op/s 2024-08-30T09:04:07.862 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-08-30T09:04:07.862 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-08-30T09:04:08.284 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T09:04:08.572 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:04:08 smithi042 ceph-mon[18652]: pgmap v474: 129 pgs: 129 active+clean; 1.1 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 MiB/s wr, 781 op/s 2024-08-30T09:04:08.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:04:08 smithi167 ceph-mon[22063]: pgmap v474: 129 pgs: 129 active+clean; 1.1 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 MiB/s wr, 781 op/s 2024-08-30T09:04:08.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:04:08 smithi190 ceph-mon[22001]: pgmap v474: 129 pgs: 129 active+clean; 1.1 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 MiB/s wr, 781 op/s 2024-08-30T09:04:08.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.860+0000 7f45ee3d0640 1 -- 172.21.15.42:0/3837818160 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f45e81163e0 msgr2=0x7f45e81188f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:04:08.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.860+0000 7f45ee3d0640 1 --2- 172.21.15.42:0/3837818160 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f45e81163e0 0x7f45e81188f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f45e0009be0 tx=0x7f45e002a6d0 comp rx=0 tx=0).stop 2024-08-30T09:04:08.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.862+0000 7f45ee3d0640 1 -- 172.21.15.42:0/3837818160 shutdown_connections 2024-08-30T09:04:08.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.862+0000 7f45ee3d0640 1 --2- 172.21.15.42:0/3837818160 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f45e81163e0 0x7f45e81188f0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:08.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.862+0000 7f45ee3d0640 1 --2- 172.21.15.42:0/3837818160 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f45e810ac80 0x7f45e8115d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:08.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.862+0000 7f45ee3d0640 1 --2- 172.21.15.42:0/3837818160 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f45e810a330 0x7f45e810a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:08.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.862+0000 7f45ee3d0640 1 -- 172.21.15.42:0/3837818160 >> 172.21.15.42:0/3837818160 conn(0x7f45e8070c60 msgr2=0x7f45e8071070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:04:08.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.865+0000 7f45ee3d0640 1 -- 172.21.15.42:0/3837818160 shutdown_connections 2024-08-30T09:04:08.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.867+0000 7f45ee3d0640 1 -- 172.21.15.42:0/3837818160 wait complete. 2024-08-30T09:04:08.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.867+0000 7f45ee3d0640 1 Processor -- start 2024-08-30T09:04:08.872 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.869+0000 7f45ee3d0640 1 -- start start 2024-08-30T09:04:08.872 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.869+0000 7f45ee3d0640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f45e810a330 0x7f45e8115bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:08.872 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.869+0000 7f45ee3d0640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f45e810ac80 0x7f45e8112ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:08.872 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.869+0000 7f45ee3d0640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f45e81163e0 0x7f45e81134d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:08.872 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.869+0000 7f45ee3d0640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f45e810d190 con 0x7f45e810ac80 2024-08-30T09:04:08.872 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.869+0000 7f45ee3d0640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f45e810d010 con 0x7f45e81163e0 2024-08-30T09:04:08.872 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.869+0000 7f45ee3d0640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f45e810d310 con 0x7f45e810a330 2024-08-30T09:04:08.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.870+0000 7f45e7fff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f45e810a330 0x7f45e8115bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:08.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.871+0000 7f45e7fff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f45e810a330 0x7f45e8115bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:45402/0 (socket says 172.21.15.42:45402) 2024-08-30T09:04:08.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.871+0000 7f45e7fff640 1 -- 172.21.15.42:0/2149826634 learned_addr learned my addr 172.21.15.42:0/2149826634 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:04:08.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.871+0000 7f45e7fff640 1 -- 172.21.15.42:0/2149826634 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f45e81163e0 msgr2=0x7f45e81134d0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-08-30T09:04:08.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.871+0000 7f45e7fff640 1 --2- 172.21.15.42:0/2149826634 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f45e81163e0 0x7f45e81134d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:08.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.871+0000 7f45e7fff640 1 -- 172.21.15.42:0/2149826634 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f45e810ac80 msgr2=0x7f45e8112ee0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-30T09:04:08.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.871+0000 7f45e7fff640 1 --2- 172.21.15.42:0/2149826634 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f45e810ac80 0x7f45e8112ee0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:08.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.871+0000 7f45e7fff640 1 -- 172.21.15.42:0/2149826634 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45e8113cf0 con 0x7f45e810a330 2024-08-30T09:04:08.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.873+0000 7f45e7fff640 1 --2- 172.21.15.42:0/2149826634 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f45e810a330 0x7f45e8115bf0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f45d800e9c0 tx=0x7f45d800ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:04:08.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.873+0000 7f45e57fa640 1 -- 172.21.15.42:0/2149826634 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45d800ccc0 con 0x7f45e810a330 2024-08-30T09:04:08.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.873+0000 7f45e57fa640 1 -- 172.21.15.42:0/2149826634 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f45d80054d0 con 0x7f45e810a330 2024-08-30T09:04:08.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.873+0000 7f45e57fa640 1 -- 172.21.15.42:0/2149826634 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45d8010430 con 0x7f45e810a330 2024-08-30T09:04:08.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.873+0000 7f45ee3d0640 1 -- 172.21.15.42:0/2149826634 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45e811ef00 con 0x7f45e810a330 2024-08-30T09:04:08.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.874+0000 7f45ee3d0640 1 -- 172.21.15.42:0/2149826634 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45e811f360 con 0x7f45e810a330 2024-08-30T09:04:08.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.877+0000 7f45e57fa640 1 -- 172.21.15.42:0/2149826634 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7f45d80046c0 con 0x7f45e810a330 2024-08-30T09:04:08.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.878+0000 7f45f4945740 1 Processor -- start 2024-08-30T09:04:08.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.879+0000 7f45f4945740 1 -- start start 2024-08-30T09:04:08.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.879+0000 7f45e57fa640 1 --2- 172.21.15.42:0/2149826634 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f45d00b4d40 0x7f45d00b7250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:08.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.879+0000 7f45e57fa640 1 -- 172.21.15.42:0/2149826634 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f45d80e5950 con 0x7f45e810a330 2024-08-30T09:04:08.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.880+0000 7f45f4945740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560530140350 0x5605304e6780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:08.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.880+0000 7f45ec96d640 1 --2- 172.21.15.42:0/2149826634 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f45d00b4d40 0x7f45d00b7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:08.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.880+0000 7f45e7fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560530140350 0x5605304e6780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:08.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.880+0000 7f45e7fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560530140350 0x5605304e6780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36020/0 (socket says 172.21.15.42:36020) 2024-08-30T09:04:08.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.880+0000 7f45f4945740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560530117490 0x5605304e6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:08.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.880+0000 7f45e7fff640 1 -- 172.21.15.42:0/3308785436 learned_addr learned my addr 172.21.15.42:0/3308785436 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:04:08.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.880+0000 7f45f4945740 1 --2- 172.21.15.42:0/3308785436 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56053011f670 0x5605304e7200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:08.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.881+0000 7f45f4945740 1 -- 172.21.15.42:0/3308785436 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x56053045c5f0 con 0x560530140350 2024-08-30T09:04:08.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.881+0000 7f45f4945740 1 -- 172.21.15.42:0/3308785436 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x56053044bb80 con 0x56053011f670 2024-08-30T09:04:08.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.881+0000 7f45f4945740 1 -- 172.21.15.42:0/3308785436 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x56053044fb30 con 0x560530117490 2024-08-30T09:04:08.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.881+0000 7f45ec96d640 1 --2- 172.21.15.42:0/3308785436 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56053011f670 0x5605304e7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:08.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.881+0000 7f45e77fe640 1 --2- 172.21.15.42:0/3308785436 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560530117490 0x5605304e6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:08.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.881+0000 7f45e7fff640 1 -- 172.21.15.42:0/3308785436 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560530117490 msgr2=0x5605304e6cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:04:08.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.881+0000 7f45e7fff640 1 --2- 172.21.15.42:0/3308785436 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560530117490 0x5605304e6cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:08.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.881+0000 7f45e7fff640 1 -- 172.21.15.42:0/3308785436 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56053011f670 msgr2=0x5605304e7200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:04:08.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.881+0000 7f45e7fff640 1 --2- 172.21.15.42:0/3308785436 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56053011f670 0x5605304e7200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:08.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.881+0000 7f45e7fff640 1 -- 172.21.15.42:0/3308785436 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5605301ab930 con 0x560530140350 2024-08-30T09:04:08.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.881+0000 7f45ec96d640 1 --2- 172.21.15.42:0/2149826634 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f45d00b4d40 0x7f45d00b7250 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f45e00099a0 tx=0x7f45e0005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:04:08.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.881+0000 7f45e7fff640 1 --2- 172.21.15.42:0/3308785436 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560530140350 0x5605304e6780 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f45d80a8010 tx=0x7f45d800efb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6c1501bbfe244bf server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:04:08.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.882+0000 7f45ce7fc640 1 -- 172.21.15.42:0/3308785436 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45d80b0800 con 0x560530140350 2024-08-30T09:04:08.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.882+0000 7f45ce7fc640 1 -- 172.21.15.42:0/3308785436 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f45d80b52b0 con 0x560530140350 2024-08-30T09:04:08.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.882+0000 7f45ce7fc640 1 -- 172.21.15.42:0/3308785436 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45d80a7b60 con 0x560530140350 2024-08-30T09:04:08.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.883+0000 7f45f4945740 1 -- 172.21.15.42:0/3308785436 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560530140350 msgr2=0x5605304e6780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:04:08.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.883+0000 7f45f4945740 1 --2- 172.21.15.42:0/3308785436 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560530140350 0x5605304e6780 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f45d80a8010 tx=0x7f45d800efb0 comp rx=0 tx=0).stop 2024-08-30T09:04:08.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.883+0000 7f45f4945740 1 -- 172.21.15.42:0/3308785436 shutdown_connections 2024-08-30T09:04:08.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.883+0000 7f45f4945740 1 --2- 172.21.15.42:0/3308785436 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560530117490 0x5605304e6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:08.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.883+0000 7f45f4945740 1 --2- 172.21.15.42:0/3308785436 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56053011f670 0x5605304e7200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:08.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.883+0000 7f45f4945740 1 --2- 172.21.15.42:0/3308785436 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560530140350 0x5605304e6780 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:08.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.883+0000 7f45f4945740 1 -- 172.21.15.42:0/3308785436 >> 172.21.15.42:0/3308785436 conn(0x560530134740 msgr2=0x5605304e4670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:04:08.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.883+0000 7f45f4945740 1 -- 172.21.15.42:0/3308785436 shutdown_connections 2024-08-30T09:04:08.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.883+0000 7f45f4945740 1 -- 172.21.15.42:0/3308785436 wait complete. 2024-08-30T09:04:08.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.883+0000 7f45f4945740 1 -- start start 2024-08-30T09:04:08.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.884+0000 7f45f4945740 1 Processor -- start 2024-08-30T09:04:08.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.884+0000 7f45f4945740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560530117490 0x5605304f1f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:08.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.884+0000 7f45f4945740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56053011f670 0x5605304f2500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:08.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.884+0000 7f45f4945740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560530140350 0x5605304fe250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:08.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.884+0000 7f45f4945740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x5605302ae890 con 0x560530140350 2024-08-30T09:04:08.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.884+0000 7f45f4945740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5605303d2790 con 0x56053011f670 2024-08-30T09:04:08.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.884+0000 7f45f4945740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5605304f4f60 con 0x560530117490 2024-08-30T09:04:08.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.884+0000 7f45ec96d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560530140350 0x5605304fe250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:08.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.884+0000 7f45ec96d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560530140350 0x5605304fe250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36022/0 (socket says 172.21.15.42:36022) 2024-08-30T09:04:08.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.884+0000 7f45f4945740 10 client.0 _ll_register_callbacks cb 0x560530157ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:04:08.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.884+0000 7f45ec96d640 1 -- 172.21.15.42:0/2525864970 learned_addr learned my addr 172.21.15.42:0/2525864970 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:04:08.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.884+0000 7f45e7fff640 1 --2- 172.21.15.42:0/2525864970 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560530117490 0x5605304f1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:08.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.884+0000 7f45e77fe640 1 --2- 172.21.15.42:0/2525864970 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56053011f670 0x5605304f2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:08.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.885+0000 7f45ec96d640 1 -- 172.21.15.42:0/2525864970 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560530117490 msgr2=0x5605304f1f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:04:08.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.885+0000 7f45ec96d640 1 --2- 172.21.15.42:0/2525864970 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560530117490 0x5605304f1f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:08.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.885+0000 7f45ec96d640 1 -- 172.21.15.42:0/2525864970 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56053011f670 msgr2=0x5605304f2500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:04:08.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.885+0000 7f45ec96d640 1 --2- 172.21.15.42:0/2525864970 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56053011f670 0x5605304f2500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:08.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.885+0000 7f45ec96d640 1 -- 172.21.15.42:0/2525864970 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560530409f30 con 0x560530140350 2024-08-30T09:04:08.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.885+0000 7f45ec96d640 1 --2- 172.21.15.42:0/2525864970 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560530140350 0x5605304fe250 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f45e0030880 tx=0x7f45e002a6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7602ddf994a6093 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:04:08.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.885+0000 7f45c77fe640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-08-30T09:04:08.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.885+0000 7f45c77fe640 1 -- 172.21.15.42:0/2525864970 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45e0036040 con 0x560530140350 2024-08-30T09:04:08.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.885+0000 7f45f4945740 1 -- 172.21.15.42:0/2525864970 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f45d80b52b0 con 0x560530140350 2024-08-30T09:04:08.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.885+0000 7f45c77fe640 1 -- 172.21.15.42:0/2525864970 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f45e0030c40 con 0x560530140350 2024-08-30T09:04:08.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.886+0000 7f45c77fe640 1 -- 172.21.15.42:0/2525864970 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f45e0058050 con 0x560530140350 2024-08-30T09:04:08.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.886+0000 7f45f4945740 10 client.16062 fetch_fsmap learned FSMap version 29 2024-08-30T09:04:08.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.886+0000 7f45f4945740 1 -- 172.21.15.42:0/2525864970 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560530500da0 con 0x560530140350 2024-08-30T09:04:08.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.887+0000 7f45c77fe640 1 -- 172.21.15.42:0/2525864970 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f45e0030e40 con 0x560530140350 2024-08-30T09:04:08.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.887+0000 7f45c77fe640 1 -- 172.21.15.42:0/2525864970 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f45e0049740 con 0x560530140350 2024-08-30T09:04:08.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.887+0000 7f45f4945740 10 client.16062 fetch_fsmap finished waiting for FSMap version 29 2024-08-30T09:04:08.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.887+0000 7f45f4945740 10 client.16062 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-30T09:04:08.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.887+0000 7f45f4945740 20 client.16062 populate_metadata read hostname 'smithi042' 2024-08-30T09:04:08.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.887+0000 7f45f4945740 1 --2- 172.21.15.42:0/2525864970 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x5605300fecd0 0x560530504360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:08.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.888+0000 7f45f4945740 4 client.16062 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-30T09:04:08.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.888+0000 7f45f4945740 1 -- 172.21.15.42:0/2525864970 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f45d80b52b0 con 0x5605300fecd0 2024-08-30T09:04:08.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.888+0000 7f45e7fff640 1 --2- 172.21.15.42:0/2525864970 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x5605300fecd0 0x560530504360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:08.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.888+0000 7f45c77fe640 10 client.16062 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:04:08.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.888+0000 7f45e7fff640 1 --2- 172.21.15.42:0/2525864970 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x5605300fecd0 0x560530504360 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=84f85b39867ca90 server_cookie=9639c2b1fb55c317 in_seq=0 out_seq=0 2024-08-30T09:04:08.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.889+0000 7f45c77fe640 1 -- 172.21.15.42:0/2525864970 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f45d80b52b0 con 0x5605300fecd0 2024-08-30T09:04:08.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.889+0000 7f45c77fe640 10 client.16062 handle_command_reply: tid=0 2024-08-30T09:04:08.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.889+0000 7f45f4945740 1 client.16062 shutdown 2024-08-30T09:04:08.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.889+0000 7f45f4945740 1 -- 172.21.15.42:0/2525864970 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560530140350 msgr2=0x5605304fe250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:04:08.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.889+0000 7f45f4945740 1 --2- 172.21.15.42:0/2525864970 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560530140350 0x5605304fe250 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f45e0030880 tx=0x7f45e002a6d0 comp rx=0 tx=0).stop 2024-08-30T09:04:08.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.890+0000 7f45f4945740 1 -- 172.21.15.42:0/2525864970 shutdown_connections 2024-08-30T09:04:08.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.890+0000 7f45f4945740 1 --2- 172.21.15.42:0/2525864970 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x5605300fecd0 0x560530504360 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:08.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.890+0000 7f45f4945740 1 --2- 172.21.15.42:0/2525864970 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560530140350 0x5605304fe250 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:08.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.890+0000 7f45f4945740 1 --2- 172.21.15.42:0/2525864970 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56053011f670 0x5605304f2500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:08.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.890+0000 7f45c77fe640 0 client.16062 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:04:08.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.890+0000 7f45f4945740 1 --2- 172.21.15.42:0/2525864970 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560530117490 0x5605304f1f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:08.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.890+0000 7f45f4945740 1 -- 172.21.15.42:0/2525864970 >> 172.21.15.42:0/2525864970 conn(0x560530134740 msgr2=0x5605304e9960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:04:08.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.890+0000 7f45f4945740 1 -- 172.21.15.42:0/2525864970 shutdown_connections 2024-08-30T09:04:08.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.890+0000 7f45f4945740 1 -- 172.21.15.42:0/2525864970 wait complete. 2024-08-30T09:04:08.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.890+0000 7f45f4945740 20 client.16062 trim_cache size 0 max 16384 2024-08-30T09:04:08.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.915+0000 7f45f4945740 1 Processor -- start 2024-08-30T09:04:08.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.915+0000 7f45f4945740 1 -- start start 2024-08-30T09:04:08.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.916+0000 7f45f4945740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560530117490 0x5605304e7ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:08.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.916+0000 7f45f4945740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56053011f670 0x5605304fe150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:08.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.916+0000 7f45f4945740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560530140350 0x5605304fe690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:08.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.916+0000 7f45f4945740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f45e81ca8e0 con 0x56053011f670 2024-08-30T09:04:08.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.916+0000 7f45e77fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56053011f670 0x5605304fe150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:08.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.916+0000 7f45ec96d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560530140350 0x5605304fe690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:08.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.916+0000 7f45e77fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56053011f670 0x5605304fe150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36032/0 (socket says 172.21.15.42:36032) 2024-08-30T09:04:08.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.916+0000 7f45ec96d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560530140350 0x5605304fe690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:45428/0 (socket says 172.21.15.42:45428) 2024-08-30T09:04:08.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.916+0000 7f45e77fe640 1 -- 172.21.15.42:0/309061776 learned_addr learned my addr 172.21.15.42:0/309061776 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:04:08.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.916+0000 7f45f4945740 1 -- 172.21.15.42:0/309061776 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f45e81c81b0 con 0x560530117490 2024-08-30T09:04:08.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.916+0000 7f45f4945740 1 -- 172.21.15.42:0/309061776 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f45e81141e0 con 0x560530140350 2024-08-30T09:04:08.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.916+0000 7f45e7fff640 1 --2- 172.21.15.42:0/309061776 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560530117490 0x5605304e7ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:08.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.916+0000 7f45e77fe640 1 -- 172.21.15.42:0/309061776 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560530140350 msgr2=0x5605304fe690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:04:08.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.916+0000 7f45e77fe640 1 --2- 172.21.15.42:0/309061776 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560530140350 0x5605304fe690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:08.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.916+0000 7f45e77fe640 1 -- 172.21.15.42:0/309061776 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560530117490 msgr2=0x5605304e7ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:04:08.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.916+0000 7f45e77fe640 1 --2- 172.21.15.42:0/309061776 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560530117490 0x5605304e7ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:08.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.916+0000 7f45e77fe640 1 -- 172.21.15.42:0/309061776 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5605304f3240 con 0x56053011f670 2024-08-30T09:04:08.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.916+0000 7f45ec96d640 1 --2- 172.21.15.42:0/309061776 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560530140350 0x5605304fe690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-30T09:04:08.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.917+0000 7f45e77fe640 1 --2- 172.21.15.42:0/309061776 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56053011f670 0x5605304fe150 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f45dc009b30 tx=0x7f45dc012450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9abf898cbec3127 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:04:08.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.917+0000 7f45cdffb640 1 -- 172.21.15.42:0/309061776 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45dc012c40 con 0x56053011f670 2024-08-30T09:04:08.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.917+0000 7f45cdffb640 1 -- 172.21.15.42:0/309061776 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f45dc012de0 con 0x56053011f670 2024-08-30T09:04:08.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.917+0000 7f45cdffb640 1 -- 172.21.15.42:0/309061776 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45dc00f440 con 0x56053011f670 2024-08-30T09:04:08.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.918+0000 7f45f4945740 1 -- 172.21.15.42:0/309061776 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56053011f670 msgr2=0x5605304fe150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:04:08.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.918+0000 7f45f4945740 1 --2- 172.21.15.42:0/309061776 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56053011f670 0x5605304fe150 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f45dc009b30 tx=0x7f45dc012450 comp rx=0 tx=0).stop 2024-08-30T09:04:08.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.918+0000 7f45f4945740 1 -- 172.21.15.42:0/309061776 shutdown_connections 2024-08-30T09:04:08.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.918+0000 7f45f4945740 1 --2- 172.21.15.42:0/309061776 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560530140350 0x5605304fe690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:08.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.918+0000 7f45f4945740 1 --2- 172.21.15.42:0/309061776 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56053011f670 0x5605304fe150 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:08.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.918+0000 7f45f4945740 1 --2- 172.21.15.42:0/309061776 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560530117490 0x5605304e7ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:08.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.918+0000 7f45f4945740 1 -- 172.21.15.42:0/309061776 >> 172.21.15.42:0/309061776 conn(0x560530134740 msgr2=0x5605304e77b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:04:08.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.918+0000 7f45f4945740 1 -- 172.21.15.42:0/309061776 shutdown_connections 2024-08-30T09:04:08.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.918+0000 7f45f4945740 1 -- 172.21.15.42:0/309061776 wait complete. 2024-08-30T09:04:08.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.919+0000 7f45f4945740 1 -- start start 2024-08-30T09:04:08.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.919+0000 7f45f4945740 1 Processor -- start 2024-08-30T09:04:08.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.919+0000 7f45f4945740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560530117490 0x5605304fa7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:08.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.919+0000 7f45f4945740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56053011f670 0x5605304f8c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:08.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.920+0000 7f45f4945740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560530140350 0x5605304f91d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:08.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.920+0000 7f45f4945740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f45e81c6670 con 0x560530140350 2024-08-30T09:04:08.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.920+0000 7f45f4945740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f45e81c7ae0 con 0x560530117490 2024-08-30T09:04:08.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.920+0000 7f45f4945740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f45e81c6d40 con 0x56053011f670 2024-08-30T09:04:08.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.920+0000 7f45f4945740 10 client.0 _ll_register_callbacks cb 0x7f45dc005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:04:08.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.920+0000 7f45e7fff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560530117490 0x5605304fa7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:08.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.920+0000 7f45e7fff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560530117490 0x5605304fa7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:32854/0 (socket says 172.21.15.42:32854) 2024-08-30T09:04:08.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.920+0000 7f45e7fff640 1 -- 172.21.15.42:0/2837767677 learned_addr learned my addr 172.21.15.42:0/2837767677 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:04:08.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.920+0000 7f45e77fe640 1 --2- 172.21.15.42:0/2837767677 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56053011f670 0x5605304f8c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:08.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.920+0000 7f45ec96d640 1 --2- 172.21.15.42:0/2837767677 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560530140350 0x5605304f91d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:08.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.920+0000 7f45e7fff640 1 -- 172.21.15.42:0/2837767677 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56053011f670 msgr2=0x5605304f8c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:04:08.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.920+0000 7f45e7fff640 1 --2- 172.21.15.42:0/2837767677 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56053011f670 0x5605304f8c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:08.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.920+0000 7f45e7fff640 1 -- 172.21.15.42:0/2837767677 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560530140350 msgr2=0x5605304f91d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:04:08.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.920+0000 7f45e7fff640 1 --2- 172.21.15.42:0/2837767677 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560530140350 0x5605304f91d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:08.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.920+0000 7f45e7fff640 1 -- 172.21.15.42:0/2837767677 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5605304f9790 con 0x560530117490 2024-08-30T09:04:08.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.920+0000 7f45e77fe640 1 --2- 172.21.15.42:0/2837767677 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56053011f670 0x5605304f8c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-30T09:04:08.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.921+0000 7f45e7fff640 1 --2- 172.21.15.42:0/2837767677 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560530117490 0x5605304fa7b0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f45d80a7fc0 tx=0x7f45d80ab830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d9169b955229e8e2 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:04:08.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.921+0000 7f45c77fe640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-30T09:04:08.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.921+0000 7f45c77fe640 1 -- 172.21.15.42:0/2837767677 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45d80c6060 con 0x560530117490 2024-08-30T09:04:08.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.921+0000 7f45c77fe640 1 -- 172.21.15.42:0/2837767677 <== mon.2 v2:172.21.15.167:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f45d80b3b80 con 0x560530117490 2024-08-30T09:04:08.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.921+0000 7f45f4945740 1 -- 172.21.15.42:0/2837767677 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f45dc012de0 con 0x560530117490 2024-08-30T09:04:08.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.921+0000 7f45c77fe640 1 -- 172.21.15.42:0/2837767677 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f45d80abd60 con 0x560530117490 2024-08-30T09:04:08.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.921+0000 7f45f4945740 10 client.25766 fetch_fsmap learned FSMap version 29 2024-08-30T09:04:08.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.921+0000 7f45f4945740 1 -- 172.21.15.42:0/2837767677 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56053056fad0 con 0x560530117490 2024-08-30T09:04:08.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.921+0000 7f45c77fe640 1 -- 172.21.15.42:0/2837767677 <== mon.2 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f45dc012de0 con 0x560530117490 2024-08-30T09:04:08.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.921+0000 7f45c77fe640 1 -- 172.21.15.42:0/2837767677 <== mon.2 v2:172.21.15.167:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f45d80cd320 con 0x560530117490 2024-08-30T09:04:08.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.921+0000 7f45f4945740 10 client.25766 fetch_fsmap finished waiting for FSMap version 29 2024-08-30T09:04:08.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.921+0000 7f45f4945740 10 client.25766 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-30T09:04:08.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.921+0000 7f45f4945740 20 client.25766 populate_metadata read hostname 'smithi042' 2024-08-30T09:04:08.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.921+0000 7f45f4945740 1 --2- 172.21.15.42:0/2837767677 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x5605300d4080 0x560530571e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:08.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.921+0000 7f45f4945740 4 client.25766 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-08-30T09:04:08.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.921+0000 7f45f4945740 1 -- 172.21.15.42:0/2837767677 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x56053045e540 con 0x5605300d4080 2024-08-30T09:04:08.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.922+0000 7f45e77fe640 1 --2- 172.21.15.42:0/2837767677 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x5605300d4080 0x560530571e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:08.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.922+0000 7f45e77fe640 1 --2- 172.21.15.42:0/2837767677 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x5605300d4080 0x560530571e50 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=51c0a47eb5dda285 server_cookie=d709614c7583b2ad in_seq=0 out_seq=0 2024-08-30T09:04:08.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.923+0000 7f45c77fe640 10 client.25766 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:04:08.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.924+0000 7f45c77fe640 1 -- 172.21.15.42:0/2837767677 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56053045e540 con 0x5605300d4080 2024-08-30T09:04:08.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.924+0000 7f45c77fe640 10 client.25766 handle_command_reply: tid=0 2024-08-30T09:04:08.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.924+0000 7f45f4945740 1 client.25766 shutdown 2024-08-30T09:04:08.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.925+0000 7f45f4945740 1 -- 172.21.15.42:0/2837767677 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560530117490 msgr2=0x5605304fa7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:04:08.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.925+0000 7f45f4945740 1 --2- 172.21.15.42:0/2837767677 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560530117490 0x5605304fa7b0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f45d80a7fc0 tx=0x7f45d80ab830 comp rx=0 tx=0).stop 2024-08-30T09:04:08.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.925+0000 7f45f4945740 1 -- 172.21.15.42:0/2837767677 shutdown_connections 2024-08-30T09:04:08.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.925+0000 7f45f4945740 1 --2- 172.21.15.42:0/2837767677 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x5605300d4080 0x560530571e50 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:08.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.925+0000 7f45c77fe640 0 client.25766 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:04:08.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.925+0000 7f45f4945740 1 --2- 172.21.15.42:0/2837767677 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56053011f670 0x5605304f8c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:08.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.925+0000 7f45f4945740 1 --2- 172.21.15.42:0/2837767677 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560530140350 0x5605304f91d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:08.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.925+0000 7f45f4945740 1 --2- 172.21.15.42:0/2837767677 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560530117490 0x5605304fa7b0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:08.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.925+0000 7f45f4945740 1 -- 172.21.15.42:0/2837767677 >> 172.21.15.42:0/2837767677 conn(0x560530134740 msgr2=0x5605304ea010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:04:08.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.925+0000 7f45f4945740 1 -- 172.21.15.42:0/2837767677 shutdown_connections 2024-08-30T09:04:08.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.925+0000 7f45f4945740 1 -- 172.21.15.42:0/2837767677 wait complete. 2024-08-30T09:04:08.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.925+0000 7f45f4945740 20 client.25766 trim_cache size 0 max 16384 2024-08-30T09:04:08.930 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-08-30T09:04:08.930 INFO:teuthology.orchestra.run.smithi042.stdout: "return_code": 0, 2024-08-30T09:04:08.930 INFO:teuthology.orchestra.run.smithi042.stdout: "scrub_tag": "88cd7478-2b20-47de-ba51-256479749232", 2024-08-30T09:04:08.930 INFO:teuthology.orchestra.run.smithi042.stdout: "mode": "asynchronous" 2024-08-30T09:04:08.930 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-08-30T09:04:08.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.926+0000 7f45ee3d0640 1 -- 172.21.15.42:0/2149826634 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f45d00b4d40 msgr2=0x7f45d00b7250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:04:08.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.926+0000 7f45ee3d0640 1 --2- 172.21.15.42:0/2149826634 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f45d00b4d40 0x7f45d00b7250 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f45e00099a0 tx=0x7f45e0005fd0 comp rx=0 tx=0).stop 2024-08-30T09:04:08.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.926+0000 7f45ee3d0640 1 -- 172.21.15.42:0/2149826634 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f45e810a330 msgr2=0x7f45e8115bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:04:08.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.926+0000 7f45ee3d0640 1 --2- 172.21.15.42:0/2149826634 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f45e810a330 0x7f45e8115bf0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f45d800e9c0 tx=0x7f45d800ee80 comp rx=0 tx=0).stop 2024-08-30T09:04:08.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.927+0000 7f45ee3d0640 1 -- 172.21.15.42:0/2149826634 shutdown_connections 2024-08-30T09:04:08.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.927+0000 7f45ee3d0640 1 --2- 172.21.15.42:0/2149826634 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f45d00b4d40 0x7f45d00b7250 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:08.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.927+0000 7f45ee3d0640 1 --2- 172.21.15.42:0/2149826634 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f45e81163e0 0x7f45e81134d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:08.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.927+0000 7f45ee3d0640 1 --2- 172.21.15.42:0/2149826634 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f45e810ac80 0x7f45e8112ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:08.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.927+0000 7f45ee3d0640 1 --2- 172.21.15.42:0/2149826634 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f45e810a330 0x7f45e8115bf0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:08.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.927+0000 7f45ee3d0640 1 -- 172.21.15.42:0/2149826634 >> 172.21.15.42:0/2149826634 conn(0x7f45e8070c60 msgr2=0x7f45e8072cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:04:08.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.927+0000 7f45ee3d0640 1 -- 172.21.15.42:0/2149826634 shutdown_connections 2024-08-30T09:04:08.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:08.927+0000 7f45ee3d0640 1 -- 172.21.15.42:0/2149826634 wait complete. 2024-08-30T09:04:10.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:04:10 smithi167 ceph-mon[22063]: scrub queued for path: / 2024-08-30T09:04:10.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:04:10 smithi167 ceph-mon[22063]: scrub summary: idle+waiting paths [/] 2024-08-30T09:04:10.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:04:10 smithi167 ceph-mon[22063]: scrub summary: active paths [/] 2024-08-30T09:04:10.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:04:10 smithi167 ceph-mon[22063]: scrub summary: active paths [/] 2024-08-30T09:04:10.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:04:10 smithi167 ceph-mon[22063]: scrub summary: active paths [/] 2024-08-30T09:04:10.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:04:10 smithi167 ceph-mon[22063]: scrub summary: idle+waiting paths [/] 2024-08-30T09:04:10.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:04:10 smithi167 ceph-mon[22063]: pgmap v475: 129 pgs: 129 active+clean; 1.1 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 11 MiB/s wr, 732 op/s 2024-08-30T09:04:10.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:04:10 smithi167 ceph-mon[22063]: scrub summary: idle+waiting paths [/] 2024-08-30T09:04:10.578 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:04:10 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-30T09:04:10.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:04:10 smithi042 ceph-mon[18652]: scrub queued for path: / 2024-08-30T09:04:10.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:04:10 smithi042 ceph-mon[18652]: scrub summary: idle+waiting paths [/] 2024-08-30T09:04:10.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:04:10 smithi042 ceph-mon[18652]: scrub summary: active paths [/] 2024-08-30T09:04:10.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:04:10 smithi042 ceph-mon[18652]: scrub summary: active paths [/] 2024-08-30T09:04:10.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:04:10 smithi042 ceph-mon[18652]: scrub summary: active paths [/] 2024-08-30T09:04:10.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:04:10 smithi042 ceph-mon[18652]: scrub summary: idle+waiting paths [/] 2024-08-30T09:04:10.634 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:04:10 smithi042 ceph-mon[18652]: pgmap v475: 129 pgs: 129 active+clean; 1.1 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 11 MiB/s wr, 732 op/s 2024-08-30T09:04:10.634 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:04:10 smithi042 ceph-mon[18652]: scrub summary: idle+waiting paths [/] 2024-08-30T09:04:10.634 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:04:10 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-30T09:04:10.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:04:10 smithi190 ceph-mon[22001]: scrub queued for path: / 2024-08-30T09:04:10.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:04:10 smithi190 ceph-mon[22001]: scrub summary: idle+waiting paths [/] 2024-08-30T09:04:10.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:04:10 smithi190 ceph-mon[22001]: scrub summary: active paths [/] 2024-08-30T09:04:10.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:04:10 smithi190 ceph-mon[22001]: scrub summary: active paths [/] 2024-08-30T09:04:10.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:04:10 smithi190 ceph-mon[22001]: scrub summary: active paths [/] 2024-08-30T09:04:10.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:04:10 smithi190 ceph-mon[22001]: scrub summary: idle+waiting paths [/] 2024-08-30T09:04:10.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:04:10 smithi190 ceph-mon[22001]: pgmap v475: 129 pgs: 129 active+clean; 1.1 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 11 MiB/s wr, 732 op/s 2024-08-30T09:04:10.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:04:10 smithi190 ceph-mon[22001]: scrub summary: idle+waiting paths [/] 2024-08-30T09:04:10.687 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:04:10 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-30T09:04:10.953 INFO:teuthology.orchestra.run.smithi042.stderr:progress: 205.0 s, 1011.4 tps, lat 97.194 ms stddev 129.289 2024-08-30T09:04:11.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:04:11 smithi167 ceph-mon[22063]: scrub summary: idle+waiting paths [/] 2024-08-30T09:04:11.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:04:11 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T09:04:11.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:04:11 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-30T09:04:11.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:04:11 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T09:04:11.583 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:04:11 smithi042 ceph-mon[18652]: scrub summary: idle+waiting paths [/] 2024-08-30T09:04:11.583 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:04:11 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T09:04:11.583 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:04:11 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-30T09:04:11.583 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:04:11 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T09:04:11.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:04:11 smithi190 ceph-mon[22001]: scrub summary: idle+waiting paths [/] 2024-08-30T09:04:11.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:04:11 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T09:04:11.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:04:11 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-30T09:04:11.687 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:04:11 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T09:04:12.127 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-30T09:04:12.577 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T09:04:12.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:04:12 smithi167 ceph-mon[22063]: pgmap v476: 129 pgs: 129 active+clean; 1.1 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 7.5 MiB/s wr, 606 op/s 2024-08-30T09:04:12.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:04:12 smithi042 ceph-mon[18652]: pgmap v476: 129 pgs: 129 active+clean; 1.1 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 7.5 MiB/s wr, 606 op/s 2024-08-30T09:04:12.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:04:12 smithi190 ceph-mon[22001]: pgmap v476: 129 pgs: 129 active+clean; 1.1 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 7.5 MiB/s wr, 606 op/s 2024-08-30T09:04:13.847 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.845+0000 7f16bb6f3640 1 -- 172.21.15.42:0/2022013434 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f16b410a780 msgr2=0x7f16b410ab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:04:13.847 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.845+0000 7f16bb6f3640 1 --2- 172.21.15.42:0/2022013434 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f16b410a780 0x7f16b410ab90 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f16b0009900 tx=0x7f16b002a440 comp rx=0 tx=0).stop 2024-08-30T09:04:13.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.850+0000 7f16bb6f3640 1 -- 172.21.15.42:0/2022013434 shutdown_connections 2024-08-30T09:04:13.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.850+0000 7f16bb6f3640 1 --2- 172.21.15.42:0/2022013434 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f16b411c9d0 0x7f16b411eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:13.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.850+0000 7f16bb6f3640 1 --2- 172.21.15.42:0/2022013434 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f16b410a780 0x7f16b410ab90 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:13.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.850+0000 7f16bb6f3640 1 --2- 172.21.15.42:0/2022013434 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f16b4111ce0 0x7f16b410a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:13.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.850+0000 7f16bb6f3640 1 -- 172.21.15.42:0/2022013434 >> 172.21.15.42:0/2022013434 conn(0x7f16b4070c60 msgr2=0x7f16b4071070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:04:13.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.850+0000 7f16bb6f3640 1 -- 172.21.15.42:0/2022013434 shutdown_connections 2024-08-30T09:04:13.853 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.850+0000 7f16bb6f3640 1 -- 172.21.15.42:0/2022013434 wait complete. 2024-08-30T09:04:13.853 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.851+0000 7f16bb6f3640 1 Processor -- start 2024-08-30T09:04:13.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.855+0000 7f16bb6f3640 1 -- start start 2024-08-30T09:04:13.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.855+0000 7f16bb6f3640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f16b410a780 0x7f16b41a9250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:13.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.856+0000 7f16bb6f3640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f16b4111ce0 0x7f16b41a97f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:13.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.856+0000 7f16bb6f3640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f16b411c9d0 0x7f16b41aea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:13.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.856+0000 7f16bb6f3640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f16b41227a0 con 0x7f16b411c9d0 2024-08-30T09:04:13.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.856+0000 7f16bb6f3640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f16b4122620 con 0x7f16b4111ce0 2024-08-30T09:04:13.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.856+0000 7f16bb6f3640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f16b4122920 con 0x7f16b410a780 2024-08-30T09:04:13.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.856+0000 7f16b948f640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f16b410a780 0x7f16b41a9250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:13.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.856+0000 7f16b948f640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f16b410a780 0x7f16b41a9250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:45460/0 (socket says 172.21.15.42:45460) 2024-08-30T09:04:13.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.856+0000 7f16b948f640 1 -- 172.21.15.42:0/1651511381 learned_addr learned my addr 172.21.15.42:0/1651511381 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:04:13.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.858+0000 7f16b9c90640 1 --2- 172.21.15.42:0/1651511381 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f16b411c9d0 0x7f16b41aea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:13.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.859+0000 7f16b8c8e640 1 --2- 172.21.15.42:0/1651511381 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f16b4111ce0 0x7f16b41a97f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:13.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.859+0000 7f16b9c90640 1 -- 172.21.15.42:0/1651511381 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f16b410a780 msgr2=0x7f16b41a9250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:04:13.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.859+0000 7f16b9c90640 1 --2- 172.21.15.42:0/1651511381 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f16b410a780 0x7f16b41a9250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:13.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.859+0000 7f16b9c90640 1 -- 172.21.15.42:0/1651511381 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f16b4111ce0 msgr2=0x7f16b41a97f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:04:13.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.859+0000 7f16b9c90640 1 --2- 172.21.15.42:0/1651511381 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f16b4111ce0 0x7f16b41a97f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:13.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.859+0000 7f16b9c90640 1 -- 172.21.15.42:0/1651511381 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16b41af1c0 con 0x7f16b411c9d0 2024-08-30T09:04:13.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.859+0000 7f16b9c90640 1 --2- 172.21.15.42:0/1651511381 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f16b411c9d0 0x7f16b41aea90 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f16a400b410 tx=0x7f16a400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:04:13.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.863+0000 7f16aa7fc640 1 -- 172.21.15.42:0/1651511381 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16a4015020 con 0x7f16b411c9d0 2024-08-30T09:04:13.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.863+0000 7f16bb6f3640 1 -- 172.21.15.42:0/1651511381 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16b41af4b0 con 0x7f16b411c9d0 2024-08-30T09:04:13.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.863+0000 7f16bb6f3640 1 -- 172.21.15.42:0/1651511381 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16b41b5c10 con 0x7f16b411c9d0 2024-08-30T09:04:13.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.867+0000 7f16c1c68740 1 Processor -- start 2024-08-30T09:04:13.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.867+0000 7f16aa7fc640 1 -- 172.21.15.42:0/1651511381 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f16a4011ce0 con 0x7f16b411c9d0 2024-08-30T09:04:13.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.867+0000 7f16aa7fc640 1 -- 172.21.15.42:0/1651511381 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16a4010db0 con 0x7f16b411c9d0 2024-08-30T09:04:13.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.874+0000 7f16aa7fc640 1 -- 172.21.15.42:0/1651511381 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7f16a4018020 con 0x7f16b411c9d0 2024-08-30T09:04:13.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.874+0000 7f16c1c68740 1 -- start start 2024-08-30T09:04:13.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.875+0000 7f16c1c68740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563049753070 0x563049afaf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:13.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.875+0000 7f16c1c68740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56304972a1b0 0x563049af85a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:13.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.875+0000 7f16c1c68740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563049732390 0x563049af8b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:13.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.875+0000 7f16c1c68740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x563049a6f630 con 0x563049753070 2024-08-30T09:04:13.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.875+0000 7f16c1c68740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x563049a5ebc0 con 0x56304972a1b0 2024-08-30T09:04:13.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.875+0000 7f16c1c68740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x563049a62b70 con 0x563049732390 2024-08-30T09:04:13.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.876+0000 7f16aa7fc640 1 --2- 172.21.15.42:0/1651511381 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f16a00b4cc0 0x7f16a00b71d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:13.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.876+0000 7f16b948f640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56304972a1b0 0x563049af85a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:13.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.876+0000 7f16b948f640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56304972a1b0 0x563049af85a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:32880/0 (socket says 172.21.15.42:32880) 2024-08-30T09:04:13.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.876+0000 7f16b948f640 1 -- 172.21.15.42:0/2851279867 learned_addr learned my addr 172.21.15.42:0/2851279867 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:04:13.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.877+0000 7f16aa7fc640 1 -- 172.21.15.42:0/1651511381 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f16a40e4db0 con 0x7f16b411c9d0 2024-08-30T09:04:13.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.877+0000 7f16b948f640 1 -- 172.21.15.42:0/2851279867 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563049732390 msgr2=0x563049af8b40 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-30T09:04:13.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.877+0000 7f16b948f640 1 --2- 172.21.15.42:0/2851279867 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563049732390 0x563049af8b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:13.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.877+0000 7f16b948f640 1 -- 172.21.15.42:0/2851279867 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563049753070 msgr2=0x563049afaf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:04:13.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.877+0000 7f16b948f640 1 --2- 172.21.15.42:0/2851279867 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563049753070 0x563049afaf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:13.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.877+0000 7f16b948f640 1 -- 172.21.15.42:0/2851279867 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5630497be650 con 0x56304972a1b0 2024-08-30T09:04:13.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.880+0000 7f16b948f640 1 --2- 172.21.15.42:0/2851279867 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56304972a1b0 0x563049af85a0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f16ac00e2d0 tx=0x7f16ac009790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5eccb040a55d9da1 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:04:13.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.881+0000 7f16b9c90640 1 --2- 172.21.15.42:0/1651511381 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f16a00b4cc0 0x7f16a00b71d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:13.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.883+0000 7f16bb6f3640 1 -- 172.21.15.42:0/2851279867 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16ac018020 con 0x56304972a1b0 2024-08-30T09:04:13.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.883+0000 7f16bb6f3640 1 -- 172.21.15.42:0/2851279867 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f16ac014d40 con 0x56304972a1b0 2024-08-30T09:04:13.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.883+0000 7f16bb6f3640 1 -- 172.21.15.42:0/2851279867 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16ac013cc0 con 0x56304972a1b0 2024-08-30T09:04:13.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.885+0000 7f16c1c68740 1 -- 172.21.15.42:0/2851279867 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56304972a1b0 msgr2=0x563049af85a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:04:13.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.885+0000 7f16c1c68740 1 --2- 172.21.15.42:0/2851279867 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56304972a1b0 0x563049af85a0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f16ac00e2d0 tx=0x7f16ac009790 comp rx=0 tx=0).stop 2024-08-30T09:04:13.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.888+0000 7f16b9c90640 1 --2- 172.21.15.42:0/1651511381 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f16a00b4cc0 0x7f16a00b71d0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f16a4010a10 tx=0x7f16a40047d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:04:13.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.891+0000 7f16c1c68740 1 -- 172.21.15.42:0/2851279867 shutdown_connections 2024-08-30T09:04:13.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.891+0000 7f16c1c68740 1 --2- 172.21.15.42:0/2851279867 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563049732390 0x563049af8b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:13.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.891+0000 7f16c1c68740 1 --2- 172.21.15.42:0/2851279867 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56304972a1b0 0x563049af85a0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:13.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.891+0000 7f16c1c68740 1 --2- 172.21.15.42:0/2851279867 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563049753070 0x563049afaf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:13.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.891+0000 7f16c1c68740 1 -- 172.21.15.42:0/2851279867 >> 172.21.15.42:0/2851279867 conn(0x563049747460 msgr2=0x563049af6490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:04:13.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.896+0000 7f16c1c68740 1 -- 172.21.15.42:0/2851279867 shutdown_connections 2024-08-30T09:04:13.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.897+0000 7f16c1c68740 1 -- 172.21.15.42:0/2851279867 wait complete. 2024-08-30T09:04:13.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.897+0000 7f16c1c68740 1 -- start start 2024-08-30T09:04:13.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.897+0000 7f16c1c68740 1 Processor -- start 2024-08-30T09:04:13.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.898+0000 7f16c1c68740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56304972a1b0 0x563049b03ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:13.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.898+0000 7f16c1c68740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563049732390 0x563049b041e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:13.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.898+0000 7f16c1c68740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563049753070 0x563049b0fe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:13.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.898+0000 7f16c1c68740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x5630498c1700 con 0x563049732390 2024-08-30T09:04:13.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.898+0000 7f16c1c68740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5630499e5700 con 0x56304972a1b0 2024-08-30T09:04:13.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.898+0000 7f16c1c68740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x563049b06a40 con 0x563049753070 2024-08-30T09:04:13.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.898+0000 7f16c1c68740 10 client.0 _ll_register_callbacks cb 0x563049a14dc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:04:13.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.900+0000 7f16b8c8e640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56304972a1b0 0x563049b03ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:13.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.900+0000 7f16b8c8e640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56304972a1b0 0x563049b03ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:32896/0 (socket says 172.21.15.42:32896) 2024-08-30T09:04:13.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.900+0000 7f16b8c8e640 1 -- 172.21.15.42:0/2915073348 learned_addr learned my addr 172.21.15.42:0/2915073348 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:04:13.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.900+0000 7f16b8c8e640 1 --2- 172.21.15.42:0/2915073348 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563049753070 0x563049b0fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:13.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.901+0000 7f16b948f640 1 --2- 172.21.15.42:0/2915073348 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563049732390 0x563049b041e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:13.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.901+0000 7f16b8c8e640 1 -- 172.21.15.42:0/2915073348 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563049753070 msgr2=0x563049b0fe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:04:13.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.901+0000 7f16b8c8e640 1 --2- 172.21.15.42:0/2915073348 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563049753070 0x563049b0fe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:13.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.901+0000 7f16b8c8e640 1 -- 172.21.15.42:0/2915073348 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563049732390 msgr2=0x563049b041e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:04:13.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.901+0000 7f16b8c8e640 1 --2- 172.21.15.42:0/2915073348 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563049732390 0x563049b041e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:13.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.901+0000 7f16b8c8e640 1 -- 172.21.15.42:0/2915073348 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563049a1e8b0 con 0x56304972a1b0 2024-08-30T09:04:13.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.902+0000 7f16b8c8e640 1 --2- 172.21.15.42:0/2915073348 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56304972a1b0 0x563049b03ca0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f16b0004650 tx=0x7f16b0000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e559e41bcbb9bfd7 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:04:13.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.903+0000 7f169cff9640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-30T09:04:13.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.903+0000 7f169cff9640 1 -- 172.21.15.42:0/2915073348 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16b0035040 con 0x56304972a1b0 2024-08-30T09:04:13.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.903+0000 7f169cff9640 1 -- 172.21.15.42:0/2915073348 <== mon.2 v2:172.21.15.167:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f16b00314a0 con 0x56304972a1b0 2024-08-30T09:04:13.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.903+0000 7f16c1c68740 1 -- 172.21.15.42:0/2915073348 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f16ac014d40 con 0x56304972a1b0 2024-08-30T09:04:13.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.903+0000 7f169cff9640 1 -- 172.21.15.42:0/2915073348 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f16b004d050 con 0x56304972a1b0 2024-08-30T09:04:13.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.905+0000 7f16c1c68740 10 client.25772 fetch_fsmap learned FSMap version 29 2024-08-30T09:04:13.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.905+0000 7f16c1c68740 1 -- 172.21.15.42:0/2915073348 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563049b128b0 con 0x56304972a1b0 2024-08-30T09:04:13.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.906+0000 7f169cff9640 1 -- 172.21.15.42:0/2915073348 <== mon.2 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f16ac014d40 con 0x56304972a1b0 2024-08-30T09:04:13.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.906+0000 7f169cff9640 1 -- 172.21.15.42:0/2915073348 <== mon.2 v2:172.21.15.167:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f16b0048030 con 0x56304972a1b0 2024-08-30T09:04:13.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.906+0000 7f16c1c68740 10 client.25772 fetch_fsmap finished waiting for FSMap version 29 2024-08-30T09:04:13.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.906+0000 7f16c1c68740 10 client.25772 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-30T09:04:13.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.906+0000 7f16c1c68740 20 client.25772 populate_metadata read hostname 'smithi042' 2024-08-30T09:04:13.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.906+0000 7f16c1c68740 1 --2- 172.21.15.42:0/2915073348 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x5630497119f0 0x563049b16200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:13.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.906+0000 7f16c1c68740 4 client.25772 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-30T09:04:13.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.906+0000 7f16c1c68740 1 -- 172.21.15.42:0/2915073348 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563049a71580 con 0x5630497119f0 2024-08-30T09:04:13.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.908+0000 7f16b948f640 1 --2- 172.21.15.42:0/2915073348 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x5630497119f0 0x563049b16200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:13.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.917+0000 7f169cff9640 10 client.25772 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:04:13.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.917+0000 7f16b948f640 1 --2- 172.21.15.42:0/2915073348 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x5630497119f0 0x563049b16200 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c974f99f178fe418 server_cookie=98145ad96061f6e9 in_seq=0 out_seq=0 2024-08-30T09:04:13.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.918+0000 7f169cff9640 1 -- 172.21.15.42:0/2915073348 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x563049a71580 con 0x5630497119f0 2024-08-30T09:04:13.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.918+0000 7f169cff9640 10 client.25772 handle_command_reply: tid=0 2024-08-30T09:04:13.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.918+0000 7f16c1c68740 1 client.25772 shutdown 2024-08-30T09:04:13.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.920+0000 7f16c1c68740 1 -- 172.21.15.42:0/2915073348 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56304972a1b0 msgr2=0x563049b03ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:04:13.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.920+0000 7f16c1c68740 1 --2- 172.21.15.42:0/2915073348 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56304972a1b0 0x563049b03ca0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f16b0004650 tx=0x7f16b0000bc0 comp rx=0 tx=0).stop 2024-08-30T09:04:13.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.920+0000 7f16c1c68740 1 -- 172.21.15.42:0/2915073348 shutdown_connections 2024-08-30T09:04:13.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.920+0000 7f16c1c68740 1 --2- 172.21.15.42:0/2915073348 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x5630497119f0 0x563049b16200 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:13.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.920+0000 7f16c1c68740 1 --2- 172.21.15.42:0/2915073348 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563049753070 0x563049b0fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:13.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.920+0000 7f16c1c68740 1 --2- 172.21.15.42:0/2915073348 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563049732390 0x563049b041e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:13.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.920+0000 7f16c1c68740 1 --2- 172.21.15.42:0/2915073348 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56304972a1b0 0x563049b03ca0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:13.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.920+0000 7f16c1c68740 1 -- 172.21.15.42:0/2915073348 >> 172.21.15.42:0/2915073348 conn(0x563049747460 msgr2=0x563049af9f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:04:13.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.920+0000 7f169cff9640 0 client.25772 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:04:13.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.921+0000 7f16c1c68740 1 -- 172.21.15.42:0/2915073348 shutdown_connections 2024-08-30T09:04:13.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.921+0000 7f16c1c68740 1 -- 172.21.15.42:0/2915073348 wait complete. 2024-08-30T09:04:13.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.921+0000 7f16c1c68740 20 client.25772 trim_cache size 0 max 16384 2024-08-30T09:04:13.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.954+0000 7f16c1c68740 1 Processor -- start 2024-08-30T09:04:13.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.954+0000 7f16c1c68740 1 -- start start 2024-08-30T09:04:13.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.954+0000 7f16c1c68740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56304972a1b0 0x563049afe750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:13.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.954+0000 7f16c1c68740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563049732390 0x563049af84d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:13.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.954+0000 7f16c1c68740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563049753070 0x563049b0fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:13.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.954+0000 7f16c1c68740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f16980125a0 con 0x563049732390 2024-08-30T09:04:13.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.954+0000 7f16c1c68740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f169800fe70 con 0x563049753070 2024-08-30T09:04:13.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.954+0000 7f16c1c68740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f169800dcc0 con 0x56304972a1b0 2024-08-30T09:04:13.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.954+0000 7f16b8c8e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56304972a1b0 0x563049afe750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:13.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.955+0000 7f16b8c8e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56304972a1b0 0x563049afe750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:45488/0 (socket says 172.21.15.42:45488) 2024-08-30T09:04:13.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.955+0000 7f16b8c8e640 1 -- 172.21.15.42:0/2551356427 learned_addr learned my addr 172.21.15.42:0/2551356427 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:04:13.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.955+0000 7f16b8c8e640 1 --2- 172.21.15.42:0/2551356427 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563049753070 0x563049b0fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:13.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.955+0000 7f16b948f640 1 --2- 172.21.15.42:0/2551356427 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563049732390 0x563049af84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:13.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.955+0000 7f16b8c8e640 1 -- 172.21.15.42:0/2551356427 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563049753070 msgr2=0x563049b0fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:04:13.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.955+0000 7f16b8c8e640 1 --2- 172.21.15.42:0/2551356427 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563049753070 0x563049b0fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:13.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.955+0000 7f16b8c8e640 1 -- 172.21.15.42:0/2551356427 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563049732390 msgr2=0x563049af84d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:04:13.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.955+0000 7f16b8c8e640 1 --2- 172.21.15.42:0/2551356427 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563049732390 0x563049af84d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:13.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.955+0000 7f16b8c8e640 1 -- 172.21.15.42:0/2551356427 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563049af1a70 con 0x56304972a1b0 2024-08-30T09:04:13.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.955+0000 7f16b8c8e640 1 --2- 172.21.15.42:0/2551356427 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56304972a1b0 0x563049afe750 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f16b00059d0 tx=0x7f16b0005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c7c5497db3a5d42e server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:04:13.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.955+0000 7f169effd640 1 -- 172.21.15.42:0/2551356427 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16b0035030 con 0x56304972a1b0 2024-08-30T09:04:13.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.956+0000 7f169effd640 1 -- 172.21.15.42:0/2551356427 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f16b0003e70 con 0x56304972a1b0 2024-08-30T09:04:13.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.956+0000 7f169effd640 1 -- 172.21.15.42:0/2551356427 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16b0004ae0 con 0x56304972a1b0 2024-08-30T09:04:13.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.957+0000 7f16c1c68740 1 -- 172.21.15.42:0/2551356427 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56304972a1b0 msgr2=0x563049afe750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:04:13.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.957+0000 7f16c1c68740 1 --2- 172.21.15.42:0/2551356427 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56304972a1b0 0x563049afe750 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f16b00059d0 tx=0x7f16b0005fd0 comp rx=0 tx=0).stop 2024-08-30T09:04:13.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.957+0000 7f16c1c68740 1 -- 172.21.15.42:0/2551356427 shutdown_connections 2024-08-30T09:04:13.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.957+0000 7f16c1c68740 1 --2- 172.21.15.42:0/2551356427 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563049753070 0x563049b0fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:13.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.957+0000 7f16c1c68740 1 --2- 172.21.15.42:0/2551356427 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563049732390 0x563049af84d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:13.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.957+0000 7f16c1c68740 1 --2- 172.21.15.42:0/2551356427 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56304972a1b0 0x563049afe750 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:13.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.957+0000 7f16c1c68740 1 -- 172.21.15.42:0/2551356427 >> 172.21.15.42:0/2551356427 conn(0x563049747460 msgr2=0x563049b03a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:04:13.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.957+0000 7f16c1c68740 1 -- 172.21.15.42:0/2551356427 shutdown_connections 2024-08-30T09:04:13.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.957+0000 7f16c1c68740 1 -- 172.21.15.42:0/2551356427 wait complete. 2024-08-30T09:04:13.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.957+0000 7f16c1c68740 1 -- start start 2024-08-30T09:04:13.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.958+0000 7f16c1c68740 1 Processor -- start 2024-08-30T09:04:13.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.958+0000 7f16c1c68740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56304972a1b0 0x563049b0c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:13.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.958+0000 7f16c1c68740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563049732390 0x563049b1b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:13.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.958+0000 7f16c1c68740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563049753070 0x563049b1b800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:13.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.958+0000 7f16b8c8e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56304972a1b0 0x563049b0c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:13.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.958+0000 7f16c1c68740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f169800e330 con 0x563049753070 2024-08-30T09:04:13.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.958+0000 7f16c1c68740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f169800f7a0 con 0x563049732390 2024-08-30T09:04:13.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.958+0000 7f16c1c68740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f169800ea00 con 0x56304972a1b0 2024-08-30T09:04:13.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.958+0000 7f16b8c8e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56304972a1b0 0x563049b0c1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:45504/0 (socket says 172.21.15.42:45504) 2024-08-30T09:04:13.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.958+0000 7f16b8c8e640 1 -- 172.21.15.42:0/1077286363 learned_addr learned my addr 172.21.15.42:0/1077286363 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:04:13.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.958+0000 7f16b948f640 1 --2- 172.21.15.42:0/1077286363 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563049732390 0x563049b1b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:13.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.959+0000 7f16c1c68740 10 client.0 _ll_register_callbacks cb 0x7f16b0009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:04:13.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.959+0000 7f16b8c8e640 1 -- 172.21.15.42:0/1077286363 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563049732390 msgr2=0x563049b1b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:04:13.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.959+0000 7f16b8c8e640 1 --2- 172.21.15.42:0/1077286363 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563049732390 0x563049b1b2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:13.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.959+0000 7f16b8c8e640 1 -- 172.21.15.42:0/1077286363 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563049753070 msgr2=0x563049b1b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:04:13.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.959+0000 7f16b8c8e640 1 --2- 172.21.15.42:0/1077286363 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563049753070 0x563049b1b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:13.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.959+0000 7f16b8c8e640 1 -- 172.21.15.42:0/1077286363 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563049b1bd40 con 0x56304972a1b0 2024-08-30T09:04:13.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.959+0000 7f16b8c8e640 1 --2- 172.21.15.42:0/1077286363 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56304972a1b0 0x563049b0c1f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f16b003c830 tx=0x7f16b002a440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5af09ed36ad6cfd server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:04:13.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.959+0000 7f169cff9640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-08-30T09:04:13.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.960+0000 7f169cff9640 1 -- 172.21.15.42:0/1077286363 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16b005a060 con 0x56304972a1b0 2024-08-30T09:04:13.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.960+0000 7f169cff9640 1 -- 172.21.15.42:0/1077286363 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f16b00314a0 con 0x56304972a1b0 2024-08-30T09:04:13.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.960+0000 7f16c1c68740 1 -- 172.21.15.42:0/1077286363 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f16b0003e70 con 0x56304972a1b0 2024-08-30T09:04:13.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.960+0000 7f169cff9640 1 -- 172.21.15.42:0/1077286363 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f16b0035040 con 0x56304972a1b0 2024-08-30T09:04:13.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.960+0000 7f16c1c68740 10 client.25795 fetch_fsmap learned FSMap version 29 2024-08-30T09:04:13.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.960+0000 7f16c1c68740 1 -- 172.21.15.42:0/1077286363 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563049b0a920 con 0x56304972a1b0 2024-08-30T09:04:13.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.961+0000 7f169cff9640 1 -- 172.21.15.42:0/1077286363 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f16b0003e70 con 0x56304972a1b0 2024-08-30T09:04:13.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.962+0000 7f169cff9640 1 -- 172.21.15.42:0/1077286363 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f16b0048070 con 0x56304972a1b0 2024-08-30T09:04:13.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.962+0000 7f16c1c68740 10 client.25795 fetch_fsmap finished waiting for FSMap version 29 2024-08-30T09:04:13.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.962+0000 7f16c1c68740 10 client.25795 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-30T09:04:13.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.962+0000 7f16c1c68740 20 client.25795 populate_metadata read hostname 'smithi042' 2024-08-30T09:04:13.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.962+0000 7f16c1c68740 1 --2- 172.21.15.42:0/1077286363 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x5630496e6da0 0x563049b82fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:13.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.962+0000 7f16c1c68740 4 client.25795 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-30T09:04:13.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.962+0000 7f16c1c68740 1 -- 172.21.15.42:0/1077286363 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563049a6d020 con 0x5630496e6da0 2024-08-30T09:04:13.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.963+0000 7f16b948f640 1 --2- 172.21.15.42:0/1077286363 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x5630496e6da0 0x563049b82fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:13.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.963+0000 7f16b948f640 1 --2- 172.21.15.42:0/1077286363 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x5630496e6da0 0x563049b82fe0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f5ed108665ff68c6 server_cookie=9d31ffee066d3195 in_seq=0 out_seq=0 2024-08-30T09:04:13.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.964+0000 7f169cff9640 10 client.25795 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:04:13.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.964+0000 7f169cff9640 1 -- 172.21.15.42:0/1077286363 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+1706 (crc 0 0 0) 0x563049a6d020 con 0x5630496e6da0 2024-08-30T09:04:13.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.964+0000 7f169cff9640 10 client.25795 handle_command_reply: tid=0 2024-08-30T09:04:13.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.964+0000 7f16c1c68740 1 client.25795 shutdown 2024-08-30T09:04:13.969 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.967+0000 7f16c1c68740 1 -- 172.21.15.42:0/1077286363 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56304972a1b0 msgr2=0x563049b0c1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:04:13.969 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.967+0000 7f16c1c68740 1 --2- 172.21.15.42:0/1077286363 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56304972a1b0 0x563049b0c1f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f16b003c830 tx=0x7f16b002a440 comp rx=0 tx=0).stop 2024-08-30T09:04:13.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.969+0000 7f16c1c68740 1 -- 172.21.15.42:0/1077286363 shutdown_connections 2024-08-30T09:04:13.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.969+0000 7f16c1c68740 1 --2- 172.21.15.42:0/1077286363 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x5630496e6da0 0x563049b82fe0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:13.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.969+0000 7f169cff9640 0 client.25795 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:04:13.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.969+0000 7f16c1c68740 1 --2- 172.21.15.42:0/1077286363 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563049753070 0x563049b1b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:13.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.969+0000 7f16c1c68740 1 --2- 172.21.15.42:0/1077286363 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563049732390 0x563049b1b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:13.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.969+0000 7f16c1c68740 1 --2- 172.21.15.42:0/1077286363 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56304972a1b0 0x563049b0c1f0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:13.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.969+0000 7f16c1c68740 1 -- 172.21.15.42:0/1077286363 >> 172.21.15.42:0/1077286363 conn(0x563049747460 msgr2=0x563049af5c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:04:13.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.970+0000 7f16c1c68740 1 -- 172.21.15.42:0/1077286363 shutdown_connections 2024-08-30T09:04:13.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.970+0000 7f16c1c68740 1 -- 172.21.15.42:0/1077286363 wait complete. 2024-08-30T09:04:13.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.971+0000 7f16c1c68740 20 client.25795 trim_cache size 0 max 16384 2024-08-30T09:04:13.974 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-08-30T09:04:13.974 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "IDLE (waiting for more scrubs)", 2024-08-30T09:04:13.974 INFO:teuthology.orchestra.run.smithi042.stdout: "scrubs": { 2024-08-30T09:04:13.974 INFO:teuthology.orchestra.run.smithi042.stdout: "88cd7478-2b20-47de-ba51-256479749232": { 2024-08-30T09:04:13.974 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:04:13.974 INFO:teuthology.orchestra.run.smithi042.stdout: "tag": "88cd7478-2b20-47de-ba51-256479749232", 2024-08-30T09:04:13.975 INFO:teuthology.orchestra.run.smithi042.stdout: "options": "recursive,force" 2024-08-30T09:04:13.975 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:04:13.975 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:04:13.975 INFO:teuthology.orchestra.run.smithi042.stdout: "scrub_stats": { 2024-08-30T09:04:13.975 INFO:teuthology.orchestra.run.smithi042.stdout: "88cd7478-2b20-47de-ba51-256479749232": { 2024-08-30T09:04:13.975 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:04:08.922625+0000", 2024-08-30T09:04:13.975 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:04:13.975 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:04:13.975 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:04:13.975 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:04:13.975 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:04:13.975 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:04:13.976 INFO:teuthology.orchestra.run.smithi042.stdout: "749f4867-0b70-49e5-af2d-8362a240ebc3": { 2024-08-30T09:04:13.976 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:03:16.386554+0000", 2024-08-30T09:04:13.976 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:04:13.976 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:04:13.976 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:04:13.976 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:04:13.976 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:04:13.976 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:04:13.976 INFO:teuthology.orchestra.run.smithi042.stdout: "85134e4b-ab4a-4885-b07e-a3ad01eaf13d": { 2024-08-30T09:04:13.976 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:01:27.738462+0000", 2024-08-30T09:04:13.976 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:04:13.976 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:04:13.977 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:04:13.977 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:04:13.977 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:04:13.977 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:04:13.977 INFO:teuthology.orchestra.run.smithi042.stdout: "c827ad7a-44ee-4e1d-9bb3-51fb7ab00c73": { 2024-08-30T09:04:13.977 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:02:21.686517+0000", 2024-08-30T09:04:13.977 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:04:13.977 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:04:13.977 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:04:13.977 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:04:13.977 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:04:13.977 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:04:13.978 INFO:teuthology.orchestra.run.smithi042.stdout: "5e2ed104-0625-478a-a78b-584e7f4c2817": { 2024-08-30T09:04:13.978 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:00:33.928400+0000", 2024-08-30T09:04:13.978 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:04:13.978 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:04:13.978 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:04:13.978 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:04:13.978 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:04:13.978 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:04:13.978 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:04:13.978 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-08-30T09:04:13.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.973+0000 7f169ffff640 1 -- 172.21.15.42:0/1651511381 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f16a00b4cc0 msgr2=0x7f16a00b71d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:04:13.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.973+0000 7f169ffff640 1 --2- 172.21.15.42:0/1651511381 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f16a00b4cc0 0x7f16a00b71d0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f16a4010a10 tx=0x7f16a40047d0 comp rx=0 tx=0).stop 2024-08-30T09:04:13.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.973+0000 7f169ffff640 1 -- 172.21.15.42:0/1651511381 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f16b411c9d0 msgr2=0x7f16b41aea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:04:13.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.973+0000 7f169ffff640 1 --2- 172.21.15.42:0/1651511381 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f16b411c9d0 0x7f16b41aea90 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f16a400b410 tx=0x7f16a400b8d0 comp rx=0 tx=0).stop 2024-08-30T09:04:13.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.973+0000 7f169ffff640 1 -- 172.21.15.42:0/1651511381 shutdown_connections 2024-08-30T09:04:13.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.973+0000 7f169ffff640 1 --2- 172.21.15.42:0/1651511381 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f16a00b4cc0 0x7f16a00b71d0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:13.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.973+0000 7f169ffff640 1 --2- 172.21.15.42:0/1651511381 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f16b411c9d0 0x7f16b41aea90 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:13.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.973+0000 7f169ffff640 1 --2- 172.21.15.42:0/1651511381 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f16b4111ce0 0x7f16b41a97f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:13.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.973+0000 7f169ffff640 1 --2- 172.21.15.42:0/1651511381 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f16b410a780 0x7f16b41a9250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:13.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.973+0000 7f169ffff640 1 -- 172.21.15.42:0/1651511381 >> 172.21.15.42:0/1651511381 conn(0x7f16b4070c60 msgr2=0x7f16b411e180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:04:13.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.973+0000 7f169ffff640 1 -- 172.21.15.42:0/1651511381 shutdown_connections 2024-08-30T09:04:13.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:13.974+0000 7f169ffff640 1 -- 172.21.15.42:0/1651511381 wait complete. 2024-08-30T09:04:14.770 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:04:14 smithi042 ceph-mon[18652]: pgmap v477: 129 pgs: 129 active+clean; 1.1 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 8.0 MiB/s wr, 758 op/s 2024-08-30T09:04:14.828 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:04:14 smithi167 ceph-mon[22063]: pgmap v477: 129 pgs: 129 active+clean; 1.1 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 8.0 MiB/s wr, 758 op/s 2024-08-30T09:04:14.829 INFO:tasks.cephfs.filesystem:scrub status for tag:88cd7478-2b20-47de-ba51-256479749232 - {'path': '/', 'tag': '88cd7478-2b20-47de-ba51-256479749232', 'options': 'recursive,force'} 2024-08-30T09:04:14.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:04:14 smithi190 ceph-mon[22001]: pgmap v477: 129 pgs: 129 active+clean; 1.1 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 8.0 MiB/s wr, 758 op/s 2024-08-30T09:04:15.953 INFO:teuthology.orchestra.run.smithi042.stderr:progress: 210.0 s, 951.2 tps, lat 107.295 ms stddev 133.356 2024-08-30T09:04:16.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:04:16 smithi167 ceph-mon[22063]: pgmap v478: 129 pgs: 129 active+clean; 1.1 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 6.1 MiB/s wr, 670 op/s 2024-08-30T09:04:16.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:04:16 smithi042 ceph-mon[18652]: pgmap v478: 129 pgs: 129 active+clean; 1.1 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 6.1 MiB/s wr, 670 op/s 2024-08-30T09:04:16.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:04:16 smithi190 ceph-mon[22001]: pgmap v478: 129 pgs: 129 active+clean; 1.1 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 6.1 MiB/s wr, 670 op/s 2024-08-30T09:04:18.645 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.642+0000 7f337c03f640 1 -- 172.21.15.42:0/4056522444 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3374103d60 msgr2=0x7f337410a700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:04:18.645 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.642+0000 7f337c03f640 1 --2- 172.21.15.42:0/4056522444 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3374103d60 0x7f337410a700 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f336c009c90 tx=0x7f336c02a880 comp rx=0 tx=0).stop 2024-08-30T09:04:18.645 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.642+0000 7f337c03f640 1 -- 172.21.15.42:0/4056522444 shutdown_connections 2024-08-30T09:04:18.645 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.642+0000 7f337c03f640 1 --2- 172.21.15.42:0/4056522444 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3374103d60 0x7f337410a700 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:18.645 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.642+0000 7f337c03f640 1 --2- 172.21.15.42:0/4056522444 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f33741033c0 0x7f33741037d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:18.645 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.642+0000 7f337c03f640 1 --2- 172.21.15.42:0/4056522444 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3374102340 0x7f33741027b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:18.645 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.642+0000 7f337c03f640 1 -- 172.21.15.42:0/4056522444 >> 172.21.15.42:0/4056522444 conn(0x7f33740fdc80 msgr2=0x7f3374100130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:04:18.645 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.642+0000 7f337c03f640 1 -- 172.21.15.42:0/4056522444 shutdown_connections 2024-08-30T09:04:18.645 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.642+0000 7f337c03f640 1 -- 172.21.15.42:0/4056522444 wait complete. 2024-08-30T09:04:18.645 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.643+0000 7f337c03f640 1 Processor -- start 2024-08-30T09:04:18.645 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.643+0000 7f337c03f640 1 -- start start 2024-08-30T09:04:18.647 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.643+0000 7f337c03f640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3374102340 0x7f337406b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:18.647 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.643+0000 7f337c03f640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f33741033c0 0x7f337419ea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:18.647 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.643+0000 7f337c03f640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3374103d60 0x7f33741a0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:18.647 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.643+0000 7f337c03f640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f337410dfe0 con 0x7f33741033c0 2024-08-30T09:04:18.647 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.643+0000 7f337c03f640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f337410de60 con 0x7f3374103d60 2024-08-30T09:04:18.647 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.643+0000 7f337c03f640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f337410e160 con 0x7f3374102340 2024-08-30T09:04:18.647 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.643+0000 7f3379ddb640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3374102340 0x7f337406b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:18.647 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.643+0000 7f33795da640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f33741033c0 0x7f337419ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:18.648 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.644+0000 7f3379ddb640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3374102340 0x7f337406b1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:41180/0 (socket says 172.21.15.42:41180) 2024-08-30T09:04:18.648 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.644+0000 7f3379ddb640 1 -- 172.21.15.42:0/1645793 learned_addr learned my addr 172.21.15.42:0/1645793 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:04:18.648 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.644+0000 7f337a5dc640 1 --2- 172.21.15.42:0/1645793 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3374103d60 0x7f33741a0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:18.648 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.644+0000 7f3379ddb640 1 -- 172.21.15.42:0/1645793 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3374103d60 msgr2=0x7f33741a0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:04:18.648 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.644+0000 7f3379ddb640 1 --2- 172.21.15.42:0/1645793 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3374103d60 0x7f33741a0fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:18.648 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.644+0000 7f3379ddb640 1 -- 172.21.15.42:0/1645793 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f33741033c0 msgr2=0x7f337419ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:04:18.648 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.644+0000 7f3379ddb640 1 --2- 172.21.15.42:0/1645793 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f33741033c0 0x7f337419ea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:18.648 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.644+0000 7f3379ddb640 1 -- 172.21.15.42:0/1645793 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33741a17d0 con 0x7f3374102340 2024-08-30T09:04:18.648 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.644+0000 7f3379ddb640 1 --2- 172.21.15.42:0/1645793 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3374102340 0x7f337406b1d0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f337000e9f0 tx=0x7f337000edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:04:18.648 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.644+0000 7f336affd640 1 -- 172.21.15.42:0/1645793 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f337000ccc0 con 0x7f3374102340 2024-08-30T09:04:18.648 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.644+0000 7f336affd640 1 -- 172.21.15.42:0/1645793 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f33700108e0 con 0x7f3374102340 2024-08-30T09:04:18.648 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.645+0000 7f337c03f640 1 -- 172.21.15.42:0/1645793 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33741a1ac0 con 0x7f3374102340 2024-08-30T09:04:18.648 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.645+0000 7f337c03f640 1 -- 172.21.15.42:0/1645793 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33741a2010 con 0x7f3374102340 2024-08-30T09:04:18.649 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.646+0000 7f336affd640 1 -- 172.21.15.42:0/1645793 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3370018880 con 0x7f3374102340 2024-08-30T09:04:18.650 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.649+0000 7f3383087740 1 Processor -- start 2024-08-30T09:04:18.651 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.649+0000 7f3383087740 1 -- start start 2024-08-30T09:04:18.652 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.649+0000 7f3383087740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ee8f6bd1b0 0x55ee8fa63180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:18.652 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.649+0000 7f3383087740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ee8f6942f0 0x55ee8fa637f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:18.652 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.649+0000 7f3383087740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ee8f69c4d0 0x55ee8fa64340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:18.652 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.649+0000 7f3383087740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55ee8f9d6000 con 0x55ee8f69c4d0 2024-08-30T09:04:18.652 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.649+0000 7f3383087740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55ee8f9c5590 con 0x55ee8f6942f0 2024-08-30T09:04:18.652 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.649+0000 7f3383087740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55ee8f9c9540 con 0x55ee8f6bd1b0 2024-08-30T09:04:18.652 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.649+0000 7f33795da640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ee8f69c4d0 0x55ee8fa64340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:18.652 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.649+0000 7f337a5dc640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ee8f6bd1b0 0x55ee8fa63180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:18.652 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.649+0000 7f33795da640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ee8f69c4d0 0x55ee8fa64340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:55084/0 (socket says 172.21.15.42:55084) 2024-08-30T09:04:18.652 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.649+0000 7f337a5dc640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ee8f6bd1b0 0x55ee8fa63180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:41190/0 (socket says 172.21.15.42:41190) 2024-08-30T09:04:18.652 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.649+0000 7f33795da640 1 -- 172.21.15.42:0/3773046923 learned_addr learned my addr 172.21.15.42:0/3773046923 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:04:18.653 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.650+0000 7f336affd640 1 -- 172.21.15.42:0/1645793 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7f3370010430 con 0x7f3374102340 2024-08-30T09:04:18.653 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.650+0000 7f33795da640 1 -- 172.21.15.42:0/3773046923 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ee8f6bd1b0 msgr2=0x55ee8fa63180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:04:18.653 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.650+0000 7f33795da640 1 --2- 172.21.15.42:0/3773046923 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ee8f6bd1b0 0x55ee8fa63180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:18.653 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.650+0000 7f33795da640 1 -- 172.21.15.42:0/3773046923 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ee8f6942f0 msgr2=0x55ee8fa637f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:04:18.653 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.650+0000 7f33795da640 1 --2- 172.21.15.42:0/3773046923 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ee8f6942f0 0x55ee8fa637f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:18.653 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.650+0000 7f33795da640 1 -- 172.21.15.42:0/3773046923 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee8f728790 con 0x55ee8f69c4d0 2024-08-30T09:04:18.653 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.651+0000 7f33795da640 1 --2- 172.21.15.42:0/3773046923 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ee8f69c4d0 0x55ee8fa64340 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f3364009a70 tx=0x7f3364009f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9961de286afb5a15 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:04:18.653 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.651+0000 7f335ffff640 1 -- 172.21.15.42:0/3773046923 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3364017070 con 0x55ee8f69c4d0 2024-08-30T09:04:18.654 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.652+0000 7f335ffff640 1 -- 172.21.15.42:0/3773046923 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3364013470 con 0x55ee8f69c4d0 2024-08-30T09:04:18.654 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.652+0000 7f335ffff640 1 -- 172.21.15.42:0/3773046923 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3364012630 con 0x55ee8f69c4d0 2024-08-30T09:04:18.655 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.652+0000 7f336affd640 1 --2- 172.21.15.42:0/1645793 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f33600b4b30 0x7f33600b7040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:18.655 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.652+0000 7f337a5dc640 1 --2- 172.21.15.42:0/1645793 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f33600b4b30 0x7f33600b7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:18.655 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.653+0000 7f336affd640 1 -- 172.21.15.42:0/1645793 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f33700e59c0 con 0x7f3374102340 2024-08-30T09:04:18.656 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.653+0000 7f3383087740 1 -- 172.21.15.42:0/3773046923 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ee8f69c4d0 msgr2=0x55ee8fa64340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:04:18.656 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.653+0000 7f3383087740 1 --2- 172.21.15.42:0/3773046923 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ee8f69c4d0 0x55ee8fa64340 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f3364009a70 tx=0x7f3364009f30 comp rx=0 tx=0).stop 2024-08-30T09:04:18.656 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.653+0000 7f3383087740 1 -- 172.21.15.42:0/3773046923 shutdown_connections 2024-08-30T09:04:18.656 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.653+0000 7f3383087740 1 --2- 172.21.15.42:0/3773046923 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ee8f69c4d0 0x55ee8fa64340 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:18.656 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.653+0000 7f3383087740 1 --2- 172.21.15.42:0/3773046923 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ee8f6942f0 0x55ee8fa637f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:18.656 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.653+0000 7f3383087740 1 --2- 172.21.15.42:0/3773046923 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ee8f6bd1b0 0x55ee8fa63180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:18.656 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.653+0000 7f337a5dc640 1 --2- 172.21.15.42:0/1645793 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f33600b4b30 0x7f33600b7040 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f336c005c90 tx=0x7f336c005c20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:04:18.656 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.653+0000 7f3383087740 1 -- 172.21.15.42:0/3773046923 >> 172.21.15.42:0/3773046923 conn(0x55ee8f6b1300 msgr2=0x55ee8fa5ec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:04:18.656 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.653+0000 7f3383087740 1 -- 172.21.15.42:0/3773046923 shutdown_connections 2024-08-30T09:04:18.656 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.653+0000 7f3383087740 1 -- 172.21.15.42:0/3773046923 wait complete. 2024-08-30T09:04:18.656 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.654+0000 7f3383087740 1 -- start start 2024-08-30T09:04:18.656 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.654+0000 7f3383087740 1 Processor -- start 2024-08-30T09:04:18.659 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.654+0000 7f3383087740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ee8f6942f0 0x55ee8fa6cc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:18.659 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.654+0000 7f3383087740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ee8f69c4d0 0x55ee8fa6d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:18.660 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.654+0000 7f3383087740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ee8f6bd1b0 0x55ee8fa6dec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:18.660 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.654+0000 7f3383087740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55ee8f8feab0 con 0x55ee8f6942f0 2024-08-30T09:04:18.660 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.654+0000 7f3383087740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55ee8f94c680 con 0x55ee8f6bd1b0 2024-08-30T09:04:18.660 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.654+0000 7f3383087740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55ee8fa64e60 con 0x55ee8f69c4d0 2024-08-30T09:04:18.660 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.654+0000 7f3379ddb640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ee8f6942f0 0x55ee8fa6cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:18.660 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.654+0000 7f337a5dc640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ee8f6bd1b0 0x55ee8fa6dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:18.660 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.654+0000 7f337a5dc640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ee8f6bd1b0 0x55ee8fa6dec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:50310/0 (socket says 172.21.15.42:50310) 2024-08-30T09:04:18.660 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.654+0000 7f33795da640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ee8f69c4d0 0x55ee8fa6d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:18.660 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.654+0000 7f337a5dc640 1 -- 172.21.15.42:0/1762458236 learned_addr learned my addr 172.21.15.42:0/1762458236 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:04:18.660 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.654+0000 7f3383087740 10 client.0 _ll_register_callbacks cb 0x55ee8f6d4a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:04:18.660 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.655+0000 7f337a5dc640 1 -- 172.21.15.42:0/1762458236 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ee8f69c4d0 msgr2=0x55ee8fa6d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:04:18.660 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.655+0000 7f337a5dc640 1 --2- 172.21.15.42:0/1762458236 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ee8f69c4d0 0x55ee8fa6d350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:18.660 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.655+0000 7f337a5dc640 1 -- 172.21.15.42:0/1762458236 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ee8f6942f0 msgr2=0x55ee8fa6cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:04:18.661 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.655+0000 7f337a5dc640 1 --2- 172.21.15.42:0/1762458236 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ee8f6942f0 0x55ee8fa6cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:18.661 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.655+0000 7f337a5dc640 1 -- 172.21.15.42:0/1762458236 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee8f7a1620 con 0x55ee8f6bd1b0 2024-08-30T09:04:18.661 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.655+0000 7f33795da640 1 --2- 172.21.15.42:0/1762458236 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ee8f69c4d0 0x55ee8fa6d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-30T09:04:18.661 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.655+0000 7f337a5dc640 1 --2- 172.21.15.42:0/1762458236 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ee8f6bd1b0 0x55ee8fa6dec0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f336c035000 tx=0x7f336c0095a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=559579ba3a80fdcc server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:04:18.661 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.655+0000 7f335d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-30T09:04:18.661 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.655+0000 7f335d7fa640 1 -- 172.21.15.42:0/1762458236 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f336c030d40 con 0x55ee8f6bd1b0 2024-08-30T09:04:18.661 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.655+0000 7f335d7fa640 1 -- 172.21.15.42:0/1762458236 <== mon.2 v2:172.21.15.167:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f336c03d420 con 0x55ee8f6bd1b0 2024-08-30T09:04:18.661 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.655+0000 7f3383087740 1 -- 172.21.15.42:0/1762458236 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3364013470 con 0x55ee8f6bd1b0 2024-08-30T09:04:18.661 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.655+0000 7f335d7fa640 1 -- 172.21.15.42:0/1762458236 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f336c03d620 con 0x55ee8f6bd1b0 2024-08-30T09:04:18.661 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.655+0000 7f3383087740 10 client.25787 fetch_fsmap learned FSMap version 29 2024-08-30T09:04:18.661 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.656+0000 7f3383087740 1 -- 172.21.15.42:0/1762458236 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ee8fa76e80 con 0x55ee8f6bd1b0 2024-08-30T09:04:18.661 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.657+0000 7f335d7fa640 1 -- 172.21.15.42:0/1762458236 <== mon.2 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3364013470 con 0x55ee8f6bd1b0 2024-08-30T09:04:18.662 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.657+0000 7f335d7fa640 1 -- 172.21.15.42:0/1762458236 <== mon.2 v2:172.21.15.167:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f336c044070 con 0x55ee8f6bd1b0 2024-08-30T09:04:18.662 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.657+0000 7f3383087740 10 client.25787 fetch_fsmap finished waiting for FSMap version 29 2024-08-30T09:04:18.662 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.657+0000 7f3383087740 10 client.25787 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-30T09:04:18.662 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.657+0000 7f3383087740 20 client.25787 populate_metadata read hostname 'smithi042' 2024-08-30T09:04:18.662 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.657+0000 7f3383087740 1 --2- 172.21.15.42:0/1762458236 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55ee8f67bd80 0x55ee8fa7a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:18.662 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.657+0000 7f3383087740 4 client.25787 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-30T09:04:18.662 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.657+0000 7f3383087740 1 -- 172.21.15.42:0/1762458236 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ee8f9d7f50 con 0x55ee8f67bd80 2024-08-30T09:04:18.662 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.658+0000 7f3379ddb640 1 --2- 172.21.15.42:0/1762458236 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55ee8f67bd80 0x55ee8fa7a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:18.663 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.658+0000 7f335d7fa640 10 client.25787 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:04:18.663 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.658+0000 7f3379ddb640 1 --2- 172.21.15.42:0/1762458236 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55ee8f67bd80 0x55ee8fa7a5f0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d99aa07f86d7130d server_cookie=d3c1c7665f1080d4 in_seq=0 out_seq=0 2024-08-30T09:04:18.663 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.659+0000 7f335d7fa640 1 -- 172.21.15.42:0/1762458236 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55ee8f9d7f50 con 0x55ee8f67bd80 2024-08-30T09:04:18.663 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.659+0000 7f335d7fa640 10 client.25787 handle_command_reply: tid=0 2024-08-30T09:04:18.663 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.659+0000 7f3383087740 1 client.25787 shutdown 2024-08-30T09:04:18.664 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.659+0000 7f3383087740 1 -- 172.21.15.42:0/1762458236 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ee8f6bd1b0 msgr2=0x55ee8fa6dec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:04:18.664 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.659+0000 7f3383087740 1 --2- 172.21.15.42:0/1762458236 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ee8f6bd1b0 0x55ee8fa6dec0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f336c035000 tx=0x7f336c0095a0 comp rx=0 tx=0).stop 2024-08-30T09:04:18.664 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.659+0000 7f3383087740 1 -- 172.21.15.42:0/1762458236 shutdown_connections 2024-08-30T09:04:18.664 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.659+0000 7f3383087740 1 --2- 172.21.15.42:0/1762458236 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55ee8f67bd80 0x55ee8fa7a5f0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:18.664 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.659+0000 7f3383087740 1 --2- 172.21.15.42:0/1762458236 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ee8f69c4d0 0x55ee8fa6d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:18.664 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.659+0000 7f3383087740 1 --2- 172.21.15.42:0/1762458236 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ee8f6bd1b0 0x55ee8fa6dec0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:18.664 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.659+0000 7f3383087740 1 --2- 172.21.15.42:0/1762458236 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ee8f6942f0 0x55ee8fa6cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:18.664 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.659+0000 7f335d7fa640 0 client.25787 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:04:18.664 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.659+0000 7f3383087740 1 -- 172.21.15.42:0/1762458236 >> 172.21.15.42:0/1762458236 conn(0x55ee8f6b1300 msgr2=0x55ee8fa5d170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:04:18.664 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.659+0000 7f3383087740 1 -- 172.21.15.42:0/1762458236 shutdown_connections 2024-08-30T09:04:18.664 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.659+0000 7f3383087740 1 -- 172.21.15.42:0/1762458236 wait complete. 2024-08-30T09:04:18.664 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.659+0000 7f3383087740 20 client.25787 trim_cache size 0 max 16384 2024-08-30T09:04:18.689 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.687+0000 7f3383087740 1 Processor -- start 2024-08-30T09:04:18.689 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.688+0000 7f3383087740 1 -- start start 2024-08-30T09:04:18.691 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.688+0000 7f3383087740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ee8f6942f0 0x55ee8fa601a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:18.691 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.688+0000 7f3383087740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ee8f69c4d0 0x55ee8fa6e420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:18.691 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.688+0000 7f3383087740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ee8f6bd1b0 0x55ee8fa5e550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:18.691 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.688+0000 7f3383087740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f33741b10b0 con 0x55ee8f6bd1b0 2024-08-30T09:04:18.691 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.688+0000 7f3383087740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f33741ae980 con 0x55ee8f69c4d0 2024-08-30T09:04:18.691 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.688+0000 7f3383087740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f33741ac7d0 con 0x55ee8f6942f0 2024-08-30T09:04:18.691 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.688+0000 7f337a5dc640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ee8f6bd1b0 0x55ee8fa5e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:18.692 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.688+0000 7f337a5dc640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ee8f6bd1b0 0x55ee8fa5e550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:55100/0 (socket says 172.21.15.42:55100) 2024-08-30T09:04:18.692 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.688+0000 7f337a5dc640 1 -- 172.21.15.42:0/3724677194 learned_addr learned my addr 172.21.15.42:0/3724677194 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:04:18.692 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.689+0000 7f337a5dc640 1 -- 172.21.15.42:0/3724677194 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ee8f6942f0 msgr2=0x55ee8fa601a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-30T09:04:18.692 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.689+0000 7f337a5dc640 1 --2- 172.21.15.42:0/3724677194 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ee8f6942f0 0x55ee8fa601a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:18.692 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.689+0000 7f337a5dc640 1 -- 172.21.15.42:0/3724677194 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ee8f69c4d0 msgr2=0x55ee8fa6e420 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-30T09:04:18.692 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.689+0000 7f337a5dc640 1 --2- 172.21.15.42:0/3724677194 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ee8f69c4d0 0x55ee8fa6e420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:18.692 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.689+0000 7f337a5dc640 1 -- 172.21.15.42:0/3724677194 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee8fa6d6c0 con 0x55ee8f6bd1b0 2024-08-30T09:04:18.692 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.689+0000 7f337a5dc640 1 --2- 172.21.15.42:0/3724677194 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ee8f6bd1b0 0x55ee8fa5e550 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f336c0508e0 tx=0x7f336c0316f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b199e82a184d5d81 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:04:18.693 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.690+0000 7f335f7fe640 1 -- 172.21.15.42:0/3724677194 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f336c050630 con 0x55ee8f6bd1b0 2024-08-30T09:04:18.693 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.690+0000 7f335f7fe640 1 -- 172.21.15.42:0/3724677194 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f336c031e50 con 0x55ee8f6bd1b0 2024-08-30T09:04:18.693 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.690+0000 7f335f7fe640 1 -- 172.21.15.42:0/3724677194 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f336c03de40 con 0x55ee8f6bd1b0 2024-08-30T09:04:18.693 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.691+0000 7f3383087740 1 -- 172.21.15.42:0/3724677194 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ee8f6bd1b0 msgr2=0x55ee8fa5e550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:04:18.693 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.691+0000 7f3383087740 1 --2- 172.21.15.42:0/3724677194 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ee8f6bd1b0 0x55ee8fa5e550 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f336c0508e0 tx=0x7f336c0316f0 comp rx=0 tx=0).stop 2024-08-30T09:04:18.698 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.695+0000 7f3383087740 1 -- 172.21.15.42:0/3724677194 shutdown_connections 2024-08-30T09:04:18.698 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.695+0000 7f3383087740 1 --2- 172.21.15.42:0/3724677194 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ee8f6bd1b0 0x55ee8fa5e550 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:18.698 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.695+0000 7f3383087740 1 --2- 172.21.15.42:0/3724677194 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ee8f69c4d0 0x55ee8fa6e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:18.698 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.695+0000 7f3383087740 1 --2- 172.21.15.42:0/3724677194 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ee8f6942f0 0x55ee8fa601a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:18.698 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.695+0000 7f3383087740 1 -- 172.21.15.42:0/3724677194 >> 172.21.15.42:0/3724677194 conn(0x55ee8f6b1300 msgr2=0x55ee8fa5d170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:04:18.706 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.704+0000 7f3383087740 1 -- 172.21.15.42:0/3724677194 shutdown_connections 2024-08-30T09:04:18.711 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.709+0000 7f3383087740 1 -- 172.21.15.42:0/3724677194 wait complete. 2024-08-30T09:04:18.711 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.709+0000 7f3383087740 1 -- start start 2024-08-30T09:04:18.711 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.709+0000 7f3383087740 1 Processor -- start 2024-08-30T09:04:18.712 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.710+0000 7f3383087740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ee8f6942f0 0x55ee8fa74260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:18.712 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.710+0000 7f3383087740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ee8f69c4d0 0x55ee8fa72f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:18.712 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.710+0000 7f3383087740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ee8f6bd1b0 0x55ee8fa73460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:18.713 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.710+0000 7f3383087740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f33741ace40 con 0x55ee8f6bd1b0 2024-08-30T09:04:18.713 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.711+0000 7f3383087740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f33741ae2b0 con 0x55ee8f6942f0 2024-08-30T09:04:18.713 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.711+0000 7f3383087740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f33741ad510 con 0x55ee8f69c4d0 2024-08-30T09:04:18.713 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.711+0000 7f3383087740 10 client.0 _ll_register_callbacks cb 0x7f336c009400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:04:18.714 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.712+0000 7f337a5dc640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ee8f6bd1b0 0x55ee8fa73460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:18.714 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.712+0000 7f337a5dc640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ee8f6bd1b0 0x55ee8fa73460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:55110/0 (socket says 172.21.15.42:55110) 2024-08-30T09:04:18.714 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.712+0000 7f337a5dc640 1 -- 172.21.15.42:0/4247822125 learned_addr learned my addr 172.21.15.42:0/4247822125 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:04:18.715 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.714+0000 7f337a5dc640 1 -- 172.21.15.42:0/4247822125 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ee8f69c4d0 msgr2=0x55ee8fa72f20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-30T09:04:18.716 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.714+0000 7f337a5dc640 1 --2- 172.21.15.42:0/4247822125 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ee8f69c4d0 0x55ee8fa72f20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:18.716 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.714+0000 7f337a5dc640 1 -- 172.21.15.42:0/4247822125 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ee8f6942f0 msgr2=0x55ee8fa74260 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-30T09:04:18.716 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.714+0000 7f337a5dc640 1 --2- 172.21.15.42:0/4247822125 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ee8f6942f0 0x55ee8fa74260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:18.716 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.714+0000 7f337a5dc640 1 -- 172.21.15.42:0/4247822125 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee8fa739a0 con 0x55ee8f6bd1b0 2024-08-30T09:04:18.718 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.716+0000 7f337a5dc640 1 --2- 172.21.15.42:0/4247822125 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ee8f6bd1b0 0x55ee8fa73460 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f336c031900 tx=0x7f336c050660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd952ed4a419583a server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:04:18.720 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.718+0000 7f335d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-08-30T09:04:18.720 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.718+0000 7f335d7fa640 1 -- 172.21.15.42:0/4247822125 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f336c03d7c0 con 0x55ee8f6bd1b0 2024-08-30T09:04:18.720 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.718+0000 7f335d7fa640 1 -- 172.21.15.42:0/4247822125 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f336c03d420 con 0x55ee8f6bd1b0 2024-08-30T09:04:18.720 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.718+0000 7f3383087740 1 -- 172.21.15.42:0/4247822125 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f336c031e50 con 0x55ee8f6bd1b0 2024-08-30T09:04:18.720 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.719+0000 7f335d7fa640 1 -- 172.21.15.42:0/4247822125 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f336c03dd50 con 0x55ee8f6bd1b0 2024-08-30T09:04:18.724 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.722+0000 7f3383087740 10 client.16095 fetch_fsmap learned FSMap version 29 2024-08-30T09:04:18.724 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.722+0000 7f3383087740 1 -- 172.21.15.42:0/4247822125 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ee8fa72730 con 0x55ee8f6bd1b0 2024-08-30T09:04:18.728 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.726+0000 7f335d7fa640 1 -- 172.21.15.42:0/4247822125 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f336c031e50 con 0x55ee8f6bd1b0 2024-08-30T09:04:18.728 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.726+0000 7f335d7fa640 1 -- 172.21.15.42:0/4247822125 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f336c044070 con 0x55ee8f6bd1b0 2024-08-30T09:04:18.729 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.727+0000 7f3383087740 10 client.16095 fetch_fsmap finished waiting for FSMap version 29 2024-08-30T09:04:18.730 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.727+0000 7f3383087740 10 client.16095 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-30T09:04:18.730 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.727+0000 7f3383087740 20 client.16095 populate_metadata read hostname 'smithi042' 2024-08-30T09:04:18.730 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.727+0000 7f3383087740 1 --2- 172.21.15.42:0/4247822125 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55ee8f6511c0 0x55ee8fae75e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:18.730 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.727+0000 7f3383087740 4 client.16095 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-30T09:04:18.730 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.727+0000 7f3383087740 1 -- 172.21.15.42:0/4247822125 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ee8f9d39f0 con 0x55ee8f6511c0 2024-08-30T09:04:18.739 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.736+0000 7f3379ddb640 1 --2- 172.21.15.42:0/4247822125 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55ee8f6511c0 0x55ee8fae75e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:18.750 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.748+0000 7f3379ddb640 1 --2- 172.21.15.42:0/4247822125 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55ee8f6511c0 0x55ee8fae75e0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=688f65369a3fd8c0 server_cookie=2f17f6ae60294d8c in_seq=0 out_seq=0 2024-08-30T09:04:18.750 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.748+0000 7f335d7fa640 10 client.16095 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:04:18.785 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.784+0000 7f335d7fa640 1 -- 172.21.15.42:0/4247822125 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ee8f9d39f0 con 0x55ee8f6511c0 2024-08-30T09:04:18.786 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.784+0000 7f335d7fa640 10 client.16095 handle_command_reply: tid=0 2024-08-30T09:04:18.786 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.784+0000 7f3383087740 1 client.16095 shutdown 2024-08-30T09:04:18.788 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.786+0000 7f3383087740 1 -- 172.21.15.42:0/4247822125 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ee8f6bd1b0 msgr2=0x55ee8fa73460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:04:18.788 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.786+0000 7f3383087740 1 --2- 172.21.15.42:0/4247822125 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ee8f6bd1b0 0x55ee8fa73460 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f336c031900 tx=0x7f336c050660 comp rx=0 tx=0).stop 2024-08-30T09:04:18.788 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.786+0000 7f3383087740 1 -- 172.21.15.42:0/4247822125 shutdown_connections 2024-08-30T09:04:18.788 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.786+0000 7f3383087740 1 --2- 172.21.15.42:0/4247822125 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55ee8f6511c0 0x55ee8fae75e0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:18.788 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.786+0000 7f3383087740 1 --2- 172.21.15.42:0/4247822125 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ee8f69c4d0 0x55ee8fa72f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:18.788 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.786+0000 7f3383087740 1 --2- 172.21.15.42:0/4247822125 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ee8f6bd1b0 0x55ee8fa73460 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:18.789 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.786+0000 7f3383087740 1 --2- 172.21.15.42:0/4247822125 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ee8f6942f0 0x55ee8fa74260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:18.789 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.786+0000 7f3383087740 1 -- 172.21.15.42:0/4247822125 >> 172.21.15.42:0/4247822125 conn(0x55ee8f6b1300 msgr2=0x55ee8fa5d170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:04:18.789 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.787+0000 7f335d7fa640 0 client.16095 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:04:18.789 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.787+0000 7f3383087740 1 -- 172.21.15.42:0/4247822125 shutdown_connections 2024-08-30T09:04:18.789 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.787+0000 7f3383087740 1 -- 172.21.15.42:0/4247822125 wait complete. 2024-08-30T09:04:18.789 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.788+0000 7f3383087740 20 client.16095 trim_cache size 0 max 16384 2024-08-30T09:04:18.791 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-08-30T09:04:18.791 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2024-08-30T09:04:18.791 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-08-30T09:04:18.791 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-08-30T09:04:18.792 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.790+0000 7f337c03f640 1 -- 172.21.15.42:0/1645793 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f33600b4b30 msgr2=0x7f33600b7040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:04:18.792 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.790+0000 7f337c03f640 1 --2- 172.21.15.42:0/1645793 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f33600b4b30 0x7f33600b7040 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f336c005c90 tx=0x7f336c005c20 comp rx=0 tx=0).stop 2024-08-30T09:04:18.792 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.790+0000 7f337c03f640 1 -- 172.21.15.42:0/1645793 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3374102340 msgr2=0x7f337406b1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:04:18.792 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.790+0000 7f337c03f640 1 --2- 172.21.15.42:0/1645793 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3374102340 0x7f337406b1d0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f337000e9f0 tx=0x7f337000edf0 comp rx=0 tx=0).stop 2024-08-30T09:04:18.792 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.790+0000 7f337c03f640 1 -- 172.21.15.42:0/1645793 shutdown_connections 2024-08-30T09:04:18.792 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.790+0000 7f337c03f640 1 --2- 172.21.15.42:0/1645793 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f33600b4b30 0x7f33600b7040 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:18.792 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.790+0000 7f337c03f640 1 --2- 172.21.15.42:0/1645793 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3374103d60 0x7f33741a0fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:18.793 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.790+0000 7f337c03f640 1 --2- 172.21.15.42:0/1645793 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f33741033c0 0x7f337419ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:18.793 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.790+0000 7f337c03f640 1 --2- 172.21.15.42:0/1645793 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3374102340 0x7f337406b1d0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:18.793 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.790+0000 7f337c03f640 1 -- 172.21.15.42:0/1645793 >> 172.21.15.42:0/1645793 conn(0x7f33740fdc80 msgr2=0x7f33741066e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:04:18.793 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.790+0000 7f337c03f640 1 -- 172.21.15.42:0/1645793 shutdown_connections 2024-08-30T09:04:18.793 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:18.790+0000 7f337c03f640 1 -- 172.21.15.42:0/1645793 wait complete. 2024-08-30T09:04:18.826 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:04:18 smithi167 ceph-mon[22063]: pgmap v479: 129 pgs: 129 active+clean; 1.1 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 6.2 MiB/s wr, 688 op/s 2024-08-30T09:04:18.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:04:18 smithi042 ceph-mon[18652]: pgmap v479: 129 pgs: 129 active+clean; 1.1 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 6.2 MiB/s wr, 688 op/s 2024-08-30T09:04:18.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:04:18 smithi190 ceph-mon[22001]: pgmap v479: 129 pgs: 129 active+clean; 1.1 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 6.2 MiB/s wr, 688 op/s 2024-08-30T09:04:20.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:04:20 smithi167 ceph-mon[22063]: pgmap v480: 129 pgs: 129 active+clean; 1.1 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 5.4 MiB/s wr, 630 op/s 2024-08-30T09:04:20.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:04:20 smithi042 ceph-mon[18652]: pgmap v480: 129 pgs: 129 active+clean; 1.1 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 5.4 MiB/s wr, 630 op/s 2024-08-30T09:04:20.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:04:20 smithi190 ceph-mon[22001]: pgmap v480: 129 pgs: 129 active+clean; 1.1 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 5.4 MiB/s wr, 630 op/s 2024-08-30T09:04:20.953 INFO:teuthology.orchestra.run.smithi042.stderr:progress: 215.0 s, 1454.4 tps, lat 69.642 ms stddev 77.190 2024-08-30T09:04:22.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:04:22 smithi167 ceph-mon[22063]: pgmap v481: 129 pgs: 129 active+clean; 1.1 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 4.2 MiB/s wr, 571 op/s 2024-08-30T09:04:22.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:04:22 smithi042 ceph-mon[18652]: pgmap v481: 129 pgs: 129 active+clean; 1.1 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 4.2 MiB/s wr, 571 op/s 2024-08-30T09:04:22.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:04:22 smithi190 ceph-mon[22001]: pgmap v481: 129 pgs: 129 active+clean; 1.1 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 4.2 MiB/s wr, 571 op/s 2024-08-30T09:04:23.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:04:23 smithi167 ceph-mon[22063]: scrub summary: idle 2024-08-30T09:04:23.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:04:23 smithi167 ceph-mon[22063]: scrub summary: idle 2024-08-30T09:04:23.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:04:23 smithi167 ceph-mon[22063]: scrub summary: idle 2024-08-30T09:04:23.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:04:23 smithi042 ceph-mon[18652]: scrub summary: idle 2024-08-30T09:04:23.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:04:23 smithi042 ceph-mon[18652]: scrub summary: idle 2024-08-30T09:04:23.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:04:23 smithi042 ceph-mon[18652]: scrub summary: idle 2024-08-30T09:04:23.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:04:23 smithi190 ceph-mon[22001]: scrub summary: idle 2024-08-30T09:04:23.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:04:23 smithi190 ceph-mon[22001]: scrub summary: idle 2024-08-30T09:04:23.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:04:23 smithi190 ceph-mon[22001]: scrub summary: idle 2024-08-30T09:04:24.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:04:24 smithi167 ceph-mon[22063]: pgmap v482: 129 pgs: 129 active+clean; 1.1 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 5.6 MiB/s wr, 703 op/s 2024-08-30T09:04:24.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:04:24 smithi042 ceph-mon[18652]: pgmap v482: 129 pgs: 129 active+clean; 1.1 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 5.6 MiB/s wr, 703 op/s 2024-08-30T09:04:24.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:04:24 smithi190 ceph-mon[22001]: pgmap v482: 129 pgs: 129 active+clean; 1.1 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 5.6 MiB/s wr, 703 op/s 2024-08-30T09:04:25.953 INFO:teuthology.orchestra.run.smithi042.stderr:progress: 220.0 s, 1229.0 tps, lat 79.352 ms stddev 87.892 2024-08-30T09:04:26.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:04:26 smithi167 ceph-mon[22063]: pgmap v483: 129 pgs: 129 active+clean; 1.1 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 4.7 MiB/s wr, 554 op/s 2024-08-30T09:04:26.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:04:26 smithi042 ceph-mon[18652]: pgmap v483: 129 pgs: 129 active+clean; 1.1 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 4.7 MiB/s wr, 554 op/s 2024-08-30T09:04:26.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:04:26 smithi190 ceph-mon[22001]: pgmap v483: 129 pgs: 129 active+clean; 1.1 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 4.7 MiB/s wr, 554 op/s 2024-08-30T09:04:28.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:04:28 smithi042 ceph-mon[18652]: pgmap v484: 129 pgs: 129 active+clean; 1.1 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 5.3 MiB/s wr, 593 op/s 2024-08-30T09:04:28.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:04:28 smithi190 ceph-mon[22001]: pgmap v484: 129 pgs: 129 active+clean; 1.1 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 5.3 MiB/s wr, 593 op/s 2024-08-30T09:04:29.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:04:28 smithi167 ceph-mon[22063]: pgmap v484: 129 pgs: 129 active+clean; 1.1 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 5.3 MiB/s wr, 593 op/s 2024-08-30T09:04:30.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:04:30 smithi042 ceph-mon[18652]: pgmap v485: 129 pgs: 129 active+clean; 1.1 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 5.6 MiB/s wr, 610 op/s 2024-08-30T09:04:30.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:04:30 smithi190 ceph-mon[22001]: pgmap v485: 129 pgs: 129 active+clean; 1.1 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 5.6 MiB/s wr, 610 op/s 2024-08-30T09:04:30.953 INFO:teuthology.orchestra.run.smithi042.stderr:progress: 225.0 s, 1119.4 tps, lat 90.495 ms stddev 95.029 2024-08-30T09:04:31.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:04:30 smithi167 ceph-mon[22063]: pgmap v485: 129 pgs: 129 active+clean; 1.1 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 5.6 MiB/s wr, 610 op/s 2024-08-30T09:04:31.964 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:31.960+0000 7f4b62af2640 1 -- 172.21.15.42:0/1702552898 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4b5c10a160 msgr2=0x7f4b5c10c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:04:31.964 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:31.960+0000 7f4b62af2640 1 --2- 172.21.15.42:0/1702552898 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4b5c10a160 0x7f4b5c10c580 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f4b4401a910 tx=0x7f4b4403b470 comp rx=0 tx=0).stop 2024-08-30T09:04:31.970 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:31.968+0000 7f4b62af2640 1 -- 172.21.15.42:0/1702552898 shutdown_connections 2024-08-30T09:04:31.970 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:31.968+0000 7f4b62af2640 1 --2- 172.21.15.42:0/1702552898 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4b5c10f770 0x7f4b5c11a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:31.970 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:31.968+0000 7f4b62af2640 1 --2- 172.21.15.42:0/1702552898 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4b5c10cb50 0x7f4b5c10f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:31.971 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:31.968+0000 7f4b62af2640 1 --2- 172.21.15.42:0/1702552898 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4b5c10a160 0x7f4b5c10c580 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:31.971 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:31.968+0000 7f4b62af2640 1 -- 172.21.15.42:0/1702552898 >> 172.21.15.42:0/1702552898 conn(0x7f4b5c0705b0 msgr2=0x7f4b5c072a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:04:31.976 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:31.974+0000 7f4b62af2640 1 -- 172.21.15.42:0/1702552898 shutdown_connections 2024-08-30T09:04:31.980 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:31.978+0000 7f4b62af2640 1 -- 172.21.15.42:0/1702552898 wait complete. 2024-08-30T09:04:31.980 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:31.978+0000 7f4b62af2640 1 Processor -- start 2024-08-30T09:04:31.984 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:31.980+0000 7f4b62af2640 1 -- start start 2024-08-30T09:04:31.984 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:31.980+0000 7f4b62af2640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4b5c10a160 0x7f4b5c1a03b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:31.984 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:31.980+0000 7f4b62af2640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4b5c10cb50 0x7f4b5c1a0a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:31.984 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:31.980+0000 7f4b62af2640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4b5c10f770 0x7f4b5c1a15d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:31.984 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:31.980+0000 7f4b62af2640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f4b5c114c70 con 0x7f4b5c10f770 2024-08-30T09:04:31.984 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:31.980+0000 7f4b62af2640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4b5c114af0 con 0x7f4b5c10cb50 2024-08-30T09:04:31.984 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:31.980+0000 7f4b62af2640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f4b5c114df0 con 0x7f4b5c10a160 2024-08-30T09:04:31.985 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:31.982+0000 7f4b6088e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4b5c10a160 0x7f4b5c1a03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:31.985 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:31.983+0000 7f4b6088e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4b5c10a160 0x7f4b5c1a03b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:50688/0 (socket says 172.21.15.42:50688) 2024-08-30T09:04:31.986 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:31.983+0000 7f4b6088e640 1 -- 172.21.15.42:0/578426731 learned_addr learned my addr 172.21.15.42:0/578426731 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:04:31.986 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:31.983+0000 7f4b6088e640 1 -- 172.21.15.42:0/578426731 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4b5c10cb50 msgr2=0x7f4b5c1a0a20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-30T09:04:31.986 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:31.983+0000 7f4b6088e640 1 --2- 172.21.15.42:0/578426731 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4b5c10cb50 0x7f4b5c1a0a20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:31.986 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:31.983+0000 7f4b6088e640 1 -- 172.21.15.42:0/578426731 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4b5c10f770 msgr2=0x7f4b5c1a15d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:04:31.986 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:31.983+0000 7f4b6088e640 1 --2- 172.21.15.42:0/578426731 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4b5c10f770 0x7f4b5c1a15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:31.986 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:31.983+0000 7f4b6088e640 1 -- 172.21.15.42:0/578426731 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b5c1a1df0 con 0x7f4b5c10a160 2024-08-30T09:04:31.986 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:31.983+0000 7f4b6088e640 1 --2- 172.21.15.42:0/578426731 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4b5c10a160 0x7f4b5c1a03b0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f4b44004e80 tx=0x7f4b44004b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:04:31.987 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:31.984+0000 7f4b59ffb640 1 -- 172.21.15.42:0/578426731 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b4404f070 con 0x7f4b5c10a160 2024-08-30T09:04:31.987 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:31.984+0000 7f4b62af2640 1 -- 172.21.15.42:0/578426731 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b5c1a2080 con 0x7f4b5c10a160 2024-08-30T09:04:31.987 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:31.985+0000 7f4b62af2640 1 -- 172.21.15.42:0/578426731 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b5c1a87d0 con 0x7f4b5c10a160 2024-08-30T09:04:31.987 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:31.984+0000 7f4b59ffb640 1 -- 172.21.15.42:0/578426731 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b44040d10 con 0x7f4b5c10a160 2024-08-30T09:04:31.987 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:31.985+0000 7f4b59ffb640 1 -- 172.21.15.42:0/578426731 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b44046030 con 0x7f4b5c10a160 2024-08-30T09:04:31.997 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:31.994+0000 7f4b69a5f740 1 Processor -- start 2024-08-30T09:04:31.997 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:31.995+0000 7f4b69a5f740 1 -- start start 2024-08-30T09:04:31.997 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:31.995+0000 7f4b69a5f740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ec59ef61b0 0x55ec5a29bf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:31.997 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:31.995+0000 7f4b69a5f740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ec59ecd2f0 0x55ec5a29c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:31.997 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:31.995+0000 7f4b69a5f740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ec59ed54d0 0x55ec5a29d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:31.998 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:31.995+0000 7f4b69a5f740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55ec5a20f000 con 0x55ec59ed54d0 2024-08-30T09:04:31.998 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:31.995+0000 7f4b69a5f740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55ec5a1fe590 con 0x55ec59ecd2f0 2024-08-30T09:04:31.998 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:31.995+0000 7f4b69a5f740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55ec5a202540 con 0x55ec59ef61b0 2024-08-30T09:04:31.998 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:31.995+0000 7f4b6108f640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ec59ef61b0 0x55ec5a29bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:31.998 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:31.995+0000 7f4b6108f640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ec59ef61b0 0x55ec5a29bf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:50694/0 (socket says 172.21.15.42:50694) 2024-08-30T09:04:31.998 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:31.995+0000 7f4b6108f640 1 -- 172.21.15.42:0/222212382 learned_addr learned my addr 172.21.15.42:0/222212382 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:04:31.999 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:31.995+0000 7f4b6088e640 1 --2- 172.21.15.42:0/222212382 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ec59ecd2f0 0x55ec5a29c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:31.999 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:31.996+0000 7f4b6108f640 1 -- 172.21.15.42:0/222212382 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ec59ecd2f0 msgr2=0x55ec5a29c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:04:31.999 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:31.996+0000 7f4b6108f640 1 --2- 172.21.15.42:0/222212382 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ec59ecd2f0 0x55ec5a29c5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:31.999 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:31.996+0000 7f4b6108f640 1 -- 172.21.15.42:0/222212382 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ec59ed54d0 msgr2=0x55ec5a29d120 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-30T09:04:31.999 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:31.996+0000 7f4b6108f640 1 --2- 172.21.15.42:0/222212382 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ec59ed54d0 0x55ec5a29d120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:32.000 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:31.996+0000 7f4b6108f640 1 -- 172.21.15.42:0/222212382 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec59f61790 con 0x55ec59ef61b0 2024-08-30T09:04:32.000 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:31.996+0000 7f4b6108f640 1 --2- 172.21.15.42:0/222212382 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ec59ef61b0 0x55ec5a29bf60 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f4b4c007c50 tx=0x7f4b4c00c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=376530ed4b4a6946 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:04:32.000 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:31.996+0000 7f4b59ffb640 1 -- 172.21.15.42:0/578426731 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7f4b44052070 con 0x7f4b5c10a160 2024-08-30T09:04:32.001 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:31.998+0000 7f4b59ffb640 1 --2- 172.21.15.42:0/578426731 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f4b480b4d40 0x7f4b480b7250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:32.001 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:31.998+0000 7f4b59ffb640 1 -- 172.21.15.42:0/578426731 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f4b44117060 con 0x7f4b5c10a160 2024-08-30T09:04:32.003 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:32.000+0000 7f4b52ffd640 1 -- 172.21.15.42:0/222212382 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b4c014070 con 0x55ec59ef61b0 2024-08-30T09:04:32.003 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:32.000+0000 7f4b52ffd640 1 -- 172.21.15.42:0/222212382 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b4c00fb00 con 0x55ec59ef61b0 2024-08-30T09:04:32.003 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:32.000+0000 7f4b52ffd640 1 -- 172.21.15.42:0/222212382 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b4c00eb00 con 0x55ec59ef61b0 2024-08-30T09:04:32.003 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:32.001+0000 7f4b69a5f740 1 -- 172.21.15.42:0/222212382 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ec59ef61b0 msgr2=0x55ec5a29bf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:04:32.003 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:32.001+0000 7f4b69a5f740 1 --2- 172.21.15.42:0/222212382 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ec59ef61b0 0x55ec5a29bf60 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f4b4c007c50 tx=0x7f4b4c00c6a0 comp rx=0 tx=0).stop 2024-08-30T09:04:32.003 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:32.001+0000 7f4b6088e640 1 --2- 172.21.15.42:0/578426731 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f4b480b4d40 0x7f4b480b7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:32.005 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:32.002+0000 7f4b69a5f740 1 -- 172.21.15.42:0/222212382 shutdown_connections 2024-08-30T09:04:32.005 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:32.002+0000 7f4b69a5f740 1 --2- 172.21.15.42:0/222212382 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ec59ed54d0 0x55ec5a29d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:32.005 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:32.002+0000 7f4b69a5f740 1 --2- 172.21.15.42:0/222212382 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ec59ecd2f0 0x55ec5a29c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:32.005 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:32.002+0000 7f4b69a5f740 1 --2- 172.21.15.42:0/222212382 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ec59ef61b0 0x55ec5a29bf60 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:32.005 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:32.002+0000 7f4b69a5f740 1 -- 172.21.15.42:0/222212382 >> 172.21.15.42:0/222212382 conn(0x55ec59eea300 msgr2=0x55ec5a2979f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:04:32.005 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:32.002+0000 7f4b6088e640 1 --2- 172.21.15.42:0/578426731 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f4b480b4d40 0x7f4b480b7250 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f4b440da180 tx=0x7f4b4403b470 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:04:32.008 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:32.007+0000 7f4b69a5f740 1 -- 172.21.15.42:0/222212382 shutdown_connections 2024-08-30T09:04:32.010 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:32.007+0000 7f4b69a5f740 1 -- 172.21.15.42:0/222212382 wait complete. 2024-08-30T09:04:32.010 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:32.008+0000 7f4b69a5f740 1 -- start start 2024-08-30T09:04:32.010 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:32.008+0000 7f4b69a5f740 1 Processor -- start 2024-08-30T09:04:32.011 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:32.009+0000 7f4b69a5f740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ec59ecd2f0 0x55ec5a2a5a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:32.011 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:32.009+0000 7f4b69a5f740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ec59ed54d0 0x55ec5a2a6150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:32.011 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:32.009+0000 7f4b69a5f740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ec59ef61b0 0x55ec5a2a6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:32.012 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:32.009+0000 7f4b6108f640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ec59ecd2f0 0x55ec5a2a5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:32.012 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:32.009+0000 7f4b6108f640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ec59ecd2f0 0x55ec5a2a5a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:50698/0 (socket says 172.21.15.42:50698) 2024-08-30T09:04:32.012 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:32.009+0000 7f4b6108f640 1 -- 172.21.15.42:0/349515788 learned_addr learned my addr 172.21.15.42:0/349515788 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:04:32.012 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:32.009+0000 7f4b69a5f740 1 -- 172.21.15.42:0/349515788 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55ec5a137ab0 con 0x55ec59ef61b0 2024-08-30T09:04:32.012 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:32.010+0000 7f4b69a5f740 1 -- 172.21.15.42:0/349515788 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55ec5a185680 con 0x55ec59ed54d0 2024-08-30T09:04:32.012 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:32.010+0000 7f4b69a5f740 1 -- 172.21.15.42:0/349515788 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55ec5a29dc40 con 0x55ec59ecd2f0 2024-08-30T09:04:32.012 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:32.010+0000 7f4b5bfff640 1 --2- 172.21.15.42:0/349515788 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ec59ed54d0 0x55ec5a2a6150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:32.013 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:32.010+0000 7f4b6108f640 1 --2- 172.21.15.42:0/349515788 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ec59ef61b0 0x55ec5a2a6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:32.013 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:32.010+0000 7f4b6108f640 1 -- 172.21.15.42:0/349515788 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ec59ed54d0 msgr2=0x55ec5a2a6150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:04:32.013 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:32.010+0000 7f4b6108f640 1 --2- 172.21.15.42:0/349515788 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ec59ed54d0 0x55ec5a2a6150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:32.013 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:32.010+0000 7f4b6108f640 1 -- 172.21.15.42:0/349515788 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ec59ef61b0 msgr2=0x55ec5a2a6cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:04:32.013 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:32.010+0000 7f4b6108f640 1 --2- 172.21.15.42:0/349515788 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ec59ef61b0 0x55ec5a2a6cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:32.013 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:32.010+0000 7f4b6108f640 1 -- 172.21.15.42:0/349515788 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ec59fda620 con 0x55ec59ecd2f0 2024-08-30T09:04:32.013 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:32.010+0000 7f4b69a5f740 10 client.0 _ll_register_callbacks cb 0x55ec59f0da00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:04:32.013 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:32.010+0000 7f4b6108f640 1 --2- 172.21.15.42:0/349515788 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ec59ecd2f0 0x55ec5a2a5a90 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f4b4c0073a0 tx=0x7f4b4c00ced0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3a716d73efccbec7 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:04:32.013 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:32.010+0000 7f4b3bfff640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-08-30T09:04:32.014 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:32.011+0000 7f4b69a5f740 1 -- 172.21.15.42:0/349515788 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4b4c00fb00 con 0x55ec59ecd2f0 2024-08-30T09:04:32.014 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:32.012+0000 7f4b3bfff640 1 -- 172.21.15.42:0/349515788 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b4c014040 con 0x55ec59ecd2f0 2024-08-30T09:04:32.014 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:32.012+0000 7f4b3bfff640 1 -- 172.21.15.42:0/349515788 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f4b4c003b70 con 0x55ec59ecd2f0 2024-08-30T09:04:32.014 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:32.013+0000 7f4b3bfff640 1 -- 172.21.15.42:0/349515788 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4b4c02f050 con 0x55ec59ecd2f0 2024-08-30T09:04:32.015 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:32.013+0000 7f4b69a5f740 10 client.25816 fetch_fsmap learned FSMap version 29 2024-08-30T09:04:32.015 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:32.013+0000 7f4b69a5f740 1 -- 172.21.15.42:0/349515788 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({fsmap=29}) -- 0x55ec5a2afb00 con 0x55ec59ecd2f0 2024-08-30T09:04:32.020 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:32.017+0000 7f4b3bfff640 1 -- 172.21.15.42:0/349515788 <== mon.1 v2:172.21.15.190:3300/0 4 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f4b4c00e800 con 0x55ec59ecd2f0 2024-08-30T09:04:32.020 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:32.018+0000 7f4b69a5f740 10 client.25816 fetch_fsmap finished waiting for FSMap version 29 2024-08-30T09:04:32.020 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:32.018+0000 7f4b69a5f740 10 client.25816 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-30T09:04:32.020 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:32.018+0000 7f4b69a5f740 20 client.25816 populate_metadata read hostname 'smithi042' 2024-08-30T09:04:32.020 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:32.018+0000 7f4b69a5f740 1 --2- 172.21.15.42:0/349515788 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55ec59eb4d80 0x55ec5a2b2f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:32.020 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:32.018+0000 7f4b69a5f740 4 client.25816 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-30T09:04:32.020 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:32.018+0000 7f4b69a5f740 1 -- 172.21.15.42:0/349515788 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f4b4c00fb00 con 0x55ec59eb4d80 2024-08-30T09:04:32.021 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:32.019+0000 7f4b5bfff640 1 --2- 172.21.15.42:0/349515788 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55ec59eb4d80 0x55ec5a2b2f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:32.021 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:32.020+0000 7f4b5bfff640 1 --2- 172.21.15.42:0/349515788 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55ec59eb4d80 0x55ec5a2b2f10 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d82420d6f742f158 server_cookie=ba313f5785a8e1f8 in_seq=0 out_seq=0 2024-08-30T09:04:32.022 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:32.021+0000 7f4b3bfff640 10 client.25816 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:04:32.022 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:32.021+0000 7f4b3bfff640 1 -- 172.21.15.42:0/349515788 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f4b4c00fb00 con 0x55ec59eb4d80 2024-08-30T09:04:32.023 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:32.021+0000 7f4b3bfff640 10 client.25816 handle_command_reply: tid=0 2024-08-30T09:04:32.024 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:32.021+0000 7f4b69a5f740 1 client.25816 shutdown 2024-08-30T09:04:32.029 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:32.027+0000 7f4b69a5f740 1 -- 172.21.15.42:0/349515788 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ec59ecd2f0 msgr2=0x55ec5a2a5a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:04:32.029 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:32.027+0000 7f4b69a5f740 1 --2- 172.21.15.42:0/349515788 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ec59ecd2f0 0x55ec5a2a5a90 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f4b4c0073a0 tx=0x7f4b4c00ced0 comp rx=0 tx=0).stop 2024-08-30T09:04:32.030 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:32.027+0000 7f4b69a5f740 1 -- 172.21.15.42:0/349515788 shutdown_connections 2024-08-30T09:04:32.030 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:32.027+0000 7f4b69a5f740 1 --2- 172.21.15.42:0/349515788 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55ec59eb4d80 0x55ec5a2b2f10 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:32.030 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:32.027+0000 7f4b69a5f740 1 --2- 172.21.15.42:0/349515788 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ec59ef61b0 0x55ec5a2a6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:32.030 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:32.027+0000 7f4b69a5f740 1 --2- 172.21.15.42:0/349515788 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ec59ed54d0 0x55ec5a2a6150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:32.030 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:32.027+0000 7f4b69a5f740 1 --2- 172.21.15.42:0/349515788 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ec59ecd2f0 0x55ec5a2a5a90 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:32.030 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:32.027+0000 7f4b69a5f740 1 -- 172.21.15.42:0/349515788 >> 172.21.15.42:0/349515788 conn(0x55ec59eea300 msgr2=0x55ec5a295c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:04:32.030 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:32.027+0000 7f4b3bfff640 0 client.25816 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:04:32.030 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:32.028+0000 7f4b69a5f740 1 -- 172.21.15.42:0/349515788 shutdown_connections 2024-08-30T09:04:32.030 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:32.028+0000 7f4b69a5f740 1 -- 172.21.15.42:0/349515788 wait complete. 2024-08-30T09:04:32.033 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:32.030+0000 7f4b69a5f740 20 client.25816 trim_cache size 0 max 16384 2024-08-30T09:04:32.060 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:32.058+0000 7f4b69a5f740 1 Processor -- start 2024-08-30T09:04:32.060 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:32.058+0000 7f4b69a5f740 1 -- start start 2024-08-30T09:04:32.061 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:32.059+0000 7f4b69a5f740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ec59ecd2f0 0x55ec5a29bc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:32.061 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:32.059+0000 7f4b69a5f740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ec59ed54d0 0x55ec5a2adb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:32.061 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:32.059+0000 7f4b69a5f740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ec59ef61b0 0x55ec5a2a09d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:32.061 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:32.059+0000 7f4b69a5f740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f4b3c012550 con 0x55ec59ed54d0 2024-08-30T09:04:32.061 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:32.059+0000 7f4b69a5f740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4b3c00fe20 con 0x55ec59ef61b0 2024-08-30T09:04:32.061 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:32.059+0000 7f4b69a5f740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f4b3c00dc70 con 0x55ec59ecd2f0 2024-08-30T09:04:32.062 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:32.059+0000 7f4b6108f640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ec59ecd2f0 0x55ec5a29bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:32.062 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:32.059+0000 7f4b6108f640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ec59ecd2f0 0x55ec5a29bc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:50704/0 (socket says 172.21.15.42:50704) 2024-08-30T09:04:32.062 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:32.059+0000 7f4b6108f640 1 -- 172.21.15.42:0/2298578452 learned_addr learned my addr 172.21.15.42:0/2298578452 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:04:32.062 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:32.059+0000 7f4b6108f640 1 --2- 172.21.15.42:0/2298578452 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ec59ef61b0 0x55ec5a2a09d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:32.062 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:32.060+0000 7f4b6108f640 1 -- 172.21.15.42:0/2298578452 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ec59ef61b0 msgr2=0x55ec5a2a09d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:04:32.062 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:32.060+0000 7f4b6108f640 1 --2- 172.21.15.42:0/2298578452 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ec59ef61b0 0x55ec5a2a09d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:32.062 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:32.060+0000 7f4b6108f640 1 -- 172.21.15.42:0/2298578452 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ec59ed54d0 msgr2=0x55ec5a2adb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:04:32.062 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:32.060+0000 7f4b6108f640 1 --2- 172.21.15.42:0/2298578452 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ec59ed54d0 0x55ec5a2adb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:32.062 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:32.060+0000 7f4b6108f640 1 -- 172.21.15.42:0/2298578452 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec5a297f30 con 0x55ec59ecd2f0 2024-08-30T09:04:32.063 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:32.060+0000 7f4b6108f640 1 --2- 172.21.15.42:0/2298578452 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ec59ecd2f0 0x55ec5a29bc40 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f4b4c0078c0 tx=0x7f4b4c020000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7a8a5f103aa6b236 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:04:32.063 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:32.060+0000 7f4b527fc640 1 -- 172.21.15.42:0/2298578452 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b4c00f910 con 0x55ec59ecd2f0 2024-08-30T09:04:32.063 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:32.061+0000 7f4b527fc640 1 -- 172.21.15.42:0/2298578452 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b4c003e20 con 0x55ec59ecd2f0 2024-08-30T09:04:32.064 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:32.061+0000 7f4b527fc640 1 -- 172.21.15.42:0/2298578452 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b4c004020 con 0x55ec59ecd2f0 2024-08-30T09:04:32.064 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:32.062+0000 7f4b69a5f740 1 -- 172.21.15.42:0/2298578452 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ec59ecd2f0 msgr2=0x55ec5a29bc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:04:32.064 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:32.062+0000 7f4b69a5f740 1 --2- 172.21.15.42:0/2298578452 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ec59ecd2f0 0x55ec5a29bc40 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f4b4c0078c0 tx=0x7f4b4c020000 comp rx=0 tx=0).stop 2024-08-30T09:04:32.064 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:32.062+0000 7f4b69a5f740 1 -- 172.21.15.42:0/2298578452 shutdown_connections 2024-08-30T09:04:32.064 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:32.062+0000 7f4b69a5f740 1 --2- 172.21.15.42:0/2298578452 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ec59ef61b0 0x55ec5a2a09d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:32.064 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:32.062+0000 7f4b69a5f740 1 --2- 172.21.15.42:0/2298578452 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ec59ed54d0 0x55ec5a2adb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:32.064 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:32.062+0000 7f4b69a5f740 1 --2- 172.21.15.42:0/2298578452 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ec59ecd2f0 0x55ec5a29bc40 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:32.064 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:32.062+0000 7f4b69a5f740 1 -- 172.21.15.42:0/2298578452 >> 172.21.15.42:0/2298578452 conn(0x55ec59eea300 msgr2=0x55ec5a2a4c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:04:32.066 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:32.064+0000 7f4b69a5f740 1 -- 172.21.15.42:0/2298578452 shutdown_connections 2024-08-30T09:04:32.067 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:32.065+0000 7f4b69a5f740 1 -- 172.21.15.42:0/2298578452 wait complete. 2024-08-30T09:04:32.067 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:32.065+0000 7f4b69a5f740 1 -- start start 2024-08-30T09:04:32.067 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:32.065+0000 7f4b69a5f740 1 Processor -- start 2024-08-30T09:04:32.068 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:32.066+0000 7f4b69a5f740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ec59ecd2f0 0x55ec5a2aad60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:32.068 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:32.066+0000 7f4b69a5f740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ec59ed54d0 0x55ec5a2ab420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:32.069 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:32.066+0000 7f4b69a5f740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ec59ef61b0 0x55ec5a2abf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:32.069 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:32.066+0000 7f4b69a5f740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f4b3c00e2e0 con 0x55ec59ecd2f0 2024-08-30T09:04:32.069 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:32.066+0000 7f4b69a5f740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4b3c00f750 con 0x55ec59ef61b0 2024-08-30T09:04:32.069 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:32.066+0000 7f4b69a5f740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f4b3c00e9b0 con 0x55ec59ed54d0 2024-08-30T09:04:32.069 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:32.066+0000 7f4b69a5f740 10 client.0 _ll_register_callbacks cb 0x7f4b4c007aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:04:32.069 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:32.066+0000 7f4b5bfff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ec59ed54d0 0x55ec5a2ab420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:32.069 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:32.066+0000 7f4b5bfff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ec59ed54d0 0x55ec5a2ab420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:50710/0 (socket says 172.21.15.42:50710) 2024-08-30T09:04:32.069 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:32.066+0000 7f4b5bfff640 1 -- 172.21.15.42:0/2653315280 learned_addr learned my addr 172.21.15.42:0/2653315280 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:04:32.069 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:32.067+0000 7f4b6108f640 1 --2- 172.21.15.42:0/2653315280 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ec59ecd2f0 0x55ec5a2aad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:32.069 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:32.067+0000 7f4b6108f640 1 --2- 172.21.15.42:0/2653315280 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ec59ef61b0 0x55ec5a2abf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:32.071 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:32.067+0000 7f4b5bfff640 1 -- 172.21.15.42:0/2653315280 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ec59ef61b0 msgr2=0x55ec5a2abf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:04:32.071 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:32.067+0000 7f4b5bfff640 1 --2- 172.21.15.42:0/2653315280 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ec59ef61b0 0x55ec5a2abf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:32.071 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:32.067+0000 7f4b5bfff640 1 -- 172.21.15.42:0/2653315280 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ec59ecd2f0 msgr2=0x55ec5a2aad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:04:32.071 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:32.067+0000 7f4b5bfff640 1 --2- 172.21.15.42:0/2653315280 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ec59ecd2f0 0x55ec5a2aad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:32.071 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:32.067+0000 7f4b5bfff640 1 -- 172.21.15.42:0/2653315280 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ec5a291800 con 0x55ec59ed54d0 2024-08-30T09:04:32.071 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:32.067+0000 7f4b5bfff640 1 --2- 172.21.15.42:0/2653315280 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ec59ed54d0 0x55ec5a2ab420 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f4b5400ba30 tx=0x7f4b5400bb20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ae57de41dc24c51a server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:04:32.072 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:32.067+0000 7f4b3bfff640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-08-30T09:04:32.072 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:32.067+0000 7f4b3bfff640 1 -- 172.21.15.42:0/2653315280 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b54005ba0 con 0x55ec59ed54d0 2024-08-30T09:04:32.072 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:32.068+0000 7f4b69a5f740 1 -- 172.21.15.42:0/2653315280 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4b4c003e20 con 0x55ec59ed54d0 2024-08-30T09:04:32.072 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:32.068+0000 7f4b3bfff640 1 -- 172.21.15.42:0/2653315280 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f4b54005d40 con 0x55ec59ed54d0 2024-08-30T09:04:32.072 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:32.068+0000 7f4b3bfff640 1 -- 172.21.15.42:0/2653315280 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4b5402a050 con 0x55ec59ed54d0 2024-08-30T09:04:32.072 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:32.068+0000 7f4b69a5f740 10 client.25822 fetch_fsmap learned FSMap version 29 2024-08-30T09:04:32.072 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:32.068+0000 7f4b69a5f740 1 -- 172.21.15.42:0/2653315280 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ec5a2ad960 con 0x55ec59ed54d0 2024-08-30T09:04:32.072 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:32.068+0000 7f4b3bfff640 1 -- 172.21.15.42:0/2653315280 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b4c003e20 con 0x55ec59ed54d0 2024-08-30T09:04:32.072 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:32.068+0000 7f4b3bfff640 1 -- 172.21.15.42:0/2653315280 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f4b54006460 con 0x55ec59ed54d0 2024-08-30T09:04:32.072 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:32.068+0000 7f4b69a5f740 10 client.25822 fetch_fsmap finished waiting for FSMap version 29 2024-08-30T09:04:32.072 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:32.068+0000 7f4b69a5f740 10 client.25822 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-30T09:04:32.072 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:32.068+0000 7f4b69a5f740 20 client.25822 populate_metadata read hostname 'smithi042' 2024-08-30T09:04:32.073 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:32.068+0000 7f4b69a5f740 1 --2- 172.21.15.42:0/2653315280 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55ec59e8a1c0 0x55ec5a31fca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:32.073 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:32.068+0000 7f4b69a5f740 4 client.25822 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-30T09:04:32.073 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:32.068+0000 7f4b69a5f740 1 -- 172.21.15.42:0/2653315280 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ec5a210f50 con 0x55ec59e8a1c0 2024-08-30T09:04:32.073 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:32.069+0000 7f4b6108f640 1 --2- 172.21.15.42:0/2653315280 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55ec59e8a1c0 0x55ec5a31fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:32.073 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:32.069+0000 7f4b3bfff640 10 client.25822 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:04:32.073 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:32.070+0000 7f4b6108f640 1 --2- 172.21.15.42:0/2653315280 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55ec59e8a1c0 0x55ec5a31fca0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ca57f9b102372035 server_cookie=553e1adeebb53f7f in_seq=0 out_seq=0 2024-08-30T09:04:32.084 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:32.082+0000 7f4b3bfff640 1 -- 172.21.15.42:0/2653315280 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ec5a210f50 con 0x55ec59e8a1c0 2024-08-30T09:04:32.084 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:32.082+0000 7f4b3bfff640 10 client.25822 handle_command_reply: tid=0 2024-08-30T09:04:32.084 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:32.082+0000 7f4b69a5f740 1 client.25822 shutdown 2024-08-30T09:04:32.085 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:32.083+0000 7f4b69a5f740 1 -- 172.21.15.42:0/2653315280 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ec59ed54d0 msgr2=0x55ec5a2ab420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:04:32.085 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:32.083+0000 7f4b69a5f740 1 --2- 172.21.15.42:0/2653315280 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ec59ed54d0 0x55ec5a2ab420 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f4b5400ba30 tx=0x7f4b5400bb20 comp rx=0 tx=0).stop 2024-08-30T09:04:32.085 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:32.083+0000 7f4b69a5f740 1 -- 172.21.15.42:0/2653315280 shutdown_connections 2024-08-30T09:04:32.085 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:32.083+0000 7f4b69a5f740 1 --2- 172.21.15.42:0/2653315280 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55ec59e8a1c0 0x55ec5a31fca0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:32.085 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:32.083+0000 7f4b69a5f740 1 --2- 172.21.15.42:0/2653315280 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ec59ed54d0 0x55ec5a2ab420 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:32.085 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:32.083+0000 7f4b69a5f740 1 --2- 172.21.15.42:0/2653315280 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ec59ef61b0 0x55ec5a2abf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:32.085 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:32.083+0000 7f4b69a5f740 1 --2- 172.21.15.42:0/2653315280 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ec59ecd2f0 0x55ec5a2aad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:32.086 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:32.083+0000 7f4b69a5f740 1 -- 172.21.15.42:0/2653315280 >> 172.21.15.42:0/2653315280 conn(0x55ec59eea300 msgr2=0x55ec5a2a4c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:04:32.086 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:32.083+0000 7f4b3bfff640 0 client.25822 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:04:32.086 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:32.083+0000 7f4b69a5f740 1 -- 172.21.15.42:0/2653315280 shutdown_connections 2024-08-30T09:04:32.086 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:32.083+0000 7f4b69a5f740 1 -- 172.21.15.42:0/2653315280 wait complete. 2024-08-30T09:04:32.086 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:32.083+0000 7f4b69a5f740 20 client.25822 trim_cache size 0 max 16384 2024-08-30T09:04:32.086 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-08-30T09:04:32.086 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2024-08-30T09:04:32.086 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-08-30T09:04:32.086 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-08-30T09:04:32.087 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:32.084+0000 7f4b62af2640 1 -- 172.21.15.42:0/578426731 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f4b480b4d40 msgr2=0x7f4b480b7250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:04:32.087 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:32.084+0000 7f4b62af2640 1 --2- 172.21.15.42:0/578426731 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f4b480b4d40 0x7f4b480b7250 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f4b440da180 tx=0x7f4b4403b470 comp rx=0 tx=0).stop 2024-08-30T09:04:32.087 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:32.084+0000 7f4b62af2640 1 -- 172.21.15.42:0/578426731 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4b5c10a160 msgr2=0x7f4b5c1a03b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:04:32.088 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:32.084+0000 7f4b62af2640 1 --2- 172.21.15.42:0/578426731 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4b5c10a160 0x7f4b5c1a03b0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f4b44004e80 tx=0x7f4b44004b90 comp rx=0 tx=0).stop 2024-08-30T09:04:32.088 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:32.084+0000 7f4b62af2640 1 -- 172.21.15.42:0/578426731 shutdown_connections 2024-08-30T09:04:32.088 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:32.084+0000 7f4b62af2640 1 --2- 172.21.15.42:0/578426731 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f4b480b4d40 0x7f4b480b7250 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:32.088 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:32.084+0000 7f4b62af2640 1 --2- 172.21.15.42:0/578426731 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4b5c10f770 0x7f4b5c1a15d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:32.088 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:32.084+0000 7f4b62af2640 1 --2- 172.21.15.42:0/578426731 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4b5c10cb50 0x7f4b5c1a0a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:32.088 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:32.084+0000 7f4b62af2640 1 --2- 172.21.15.42:0/578426731 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4b5c10a160 0x7f4b5c1a03b0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:32.088 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:32.084+0000 7f4b62af2640 1 -- 172.21.15.42:0/578426731 >> 172.21.15.42:0/578426731 conn(0x7f4b5c0705b0 msgr2=0x7f4b5c10b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:04:32.088 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:32.084+0000 7f4b62af2640 1 -- 172.21.15.42:0/578426731 shutdown_connections 2024-08-30T09:04:32.088 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:32.084+0000 7f4b62af2640 1 -- 172.21.15.42:0/578426731 wait complete. 2024-08-30T09:04:32.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:04:32 smithi042 ceph-mon[18652]: pgmap v486: 129 pgs: 129 active+clean; 1.1 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 4.7 MiB/s wr, 529 op/s 2024-08-30T09:04:32.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:04:32 smithi190 ceph-mon[22001]: pgmap v486: 129 pgs: 129 active+clean; 1.1 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 4.7 MiB/s wr, 529 op/s 2024-08-30T09:04:33.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:04:32 smithi167 ceph-mon[22063]: pgmap v486: 129 pgs: 129 active+clean; 1.1 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 4.7 MiB/s wr, 529 op/s 2024-08-30T09:04:34.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:04:34 smithi042 ceph-mon[18652]: pgmap v487: 129 pgs: 129 active+clean; 1.1 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 7.1 MiB/s wr, 680 op/s 2024-08-30T09:04:34.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:04:34 smithi190 ceph-mon[22001]: pgmap v487: 129 pgs: 129 active+clean; 1.1 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 7.1 MiB/s wr, 680 op/s 2024-08-30T09:04:35.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:04:34 smithi167 ceph-mon[22063]: pgmap v487: 129 pgs: 129 active+clean; 1.1 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 7.1 MiB/s wr, 680 op/s 2024-08-30T09:04:35.953 INFO:teuthology.orchestra.run.smithi042.stderr:progress: 230.0 s, 644.2 tps, lat 151.620 ms stddev 167.166 2024-08-30T09:04:36.879 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:04:36 smithi167 ceph-mon[22063]: pgmap v488: 129 pgs: 129 active+clean; 1.1 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 6.2 MiB/s wr, 547 op/s 2024-08-30T09:04:36.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:04:36 smithi042 ceph-mon[18652]: pgmap v488: 129 pgs: 129 active+clean; 1.1 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 6.2 MiB/s wr, 547 op/s 2024-08-30T09:04:36.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:04:36 smithi190 ceph-mon[22001]: pgmap v488: 129 pgs: 129 active+clean; 1.1 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 6.2 MiB/s wr, 547 op/s 2024-08-30T09:04:38.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:04:38 smithi042 ceph-mon[18652]: pgmap v489: 129 pgs: 129 active+clean; 1.1 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 6.8 MiB/s wr, 610 op/s 2024-08-30T09:04:38.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:04:38 smithi190 ceph-mon[22001]: pgmap v489: 129 pgs: 129 active+clean; 1.1 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 6.8 MiB/s wr, 610 op/s 2024-08-30T09:04:39.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:04:38 smithi167 ceph-mon[22063]: pgmap v489: 129 pgs: 129 active+clean; 1.1 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 6.8 MiB/s wr, 610 op/s 2024-08-30T09:04:40.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:04:40 smithi042 ceph-mon[18652]: pgmap v490: 129 pgs: 129 active+clean; 1.1 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 6.7 MiB/s wr, 592 op/s 2024-08-30T09:04:40.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:04:40 smithi190 ceph-mon[22001]: pgmap v490: 129 pgs: 129 active+clean; 1.1 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 6.7 MiB/s wr, 592 op/s 2024-08-30T09:04:40.953 INFO:teuthology.orchestra.run.smithi042.stderr:progress: 235.0 s, 689.6 tps, lat 145.852 ms stddev 169.198 2024-08-30T09:04:41.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:04:40 smithi167 ceph-mon[22063]: pgmap v490: 129 pgs: 129 active+clean; 1.1 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 6.7 MiB/s wr, 592 op/s 2024-08-30T09:04:42.865 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:04:42 smithi042 ceph-mon[18652]: pgmap v491: 129 pgs: 129 active+clean; 1.1 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.1 MiB/s wr, 544 op/s 2024-08-30T09:04:42.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:04:42 smithi190 ceph-mon[22001]: pgmap v491: 129 pgs: 129 active+clean; 1.1 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.1 MiB/s wr, 544 op/s 2024-08-30T09:04:43.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:04:42 smithi167 ceph-mon[22063]: pgmap v491: 129 pgs: 129 active+clean; 1.1 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.1 MiB/s wr, 544 op/s 2024-08-30T09:04:44.831 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-30T09:04:44.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:04:44 smithi042 ceph-mon[18652]: pgmap v492: 129 pgs: 129 active+clean; 1.1 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 11 MiB/s wr, 847 op/s 2024-08-30T09:04:44.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:04:44 smithi190 ceph-mon[22001]: pgmap v492: 129 pgs: 129 active+clean; 1.1 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 11 MiB/s wr, 847 op/s 2024-08-30T09:04:45.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:04:44 smithi167 ceph-mon[22063]: pgmap v492: 129 pgs: 129 active+clean; 1.1 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 11 MiB/s wr, 847 op/s 2024-08-30T09:04:45.260 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.257+0000 7f7b4d4ce640 1 -- 172.21.15.42:0/905337240 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7b48104310 msgr2=0x7f7b48104720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:04:45.260 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.257+0000 7f7b4d4ce640 1 --2- 172.21.15.42:0/905337240 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7b48104310 0x7f7b48104720 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f7b40009ca0 tx=0x7f7b4002a8c0 comp rx=0 tx=0).stop 2024-08-30T09:04:45.260 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.257+0000 7f7b4d4ce640 1 -- 172.21.15.42:0/905337240 shutdown_connections 2024-08-30T09:04:45.260 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.257+0000 7f7b4d4ce640 1 --2- 172.21.15.42:0/905337240 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7b48104310 0x7f7b48104720 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:45.260 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.257+0000 7f7b4d4ce640 1 --2- 172.21.15.42:0/905337240 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7b48104cb0 0x7f7b4810b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:45.260 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.257+0000 7f7b4d4ce640 1 --2- 172.21.15.42:0/905337240 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7b48103290 0x7f7b48103700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:45.260 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.257+0000 7f7b4d4ce640 1 -- 172.21.15.42:0/905337240 >> 172.21.15.42:0/905337240 conn(0x7f7b480febf0 msgr2=0x7f7b48101080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:04:45.260 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.257+0000 7f7b4d4ce640 1 -- 172.21.15.42:0/905337240 shutdown_connections 2024-08-30T09:04:45.260 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.257+0000 7f7b4d4ce640 1 -- 172.21.15.42:0/905337240 wait complete. 2024-08-30T09:04:45.261 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.258+0000 7f7b4d4ce640 1 Processor -- start 2024-08-30T09:04:45.261 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.259+0000 7f7b4d4ce640 1 -- start start 2024-08-30T09:04:45.261 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.259+0000 7f7b4d4ce640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7b48103290 0x7f7b481a0160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:45.261 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.259+0000 7f7b4d4ce640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7b48104310 0x7f7b481a07d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:45.261 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.259+0000 7f7b46ffd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7b48103290 0x7f7b481a0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:45.261 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.259+0000 7f7b467fc640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7b48104310 0x7f7b481a07d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:45.262 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.259+0000 7f7b467fc640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7b48104310 0x7f7b481a07d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35490/0 (socket says 172.21.15.42:35490) 2024-08-30T09:04:45.262 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.259+0000 7f7b46ffd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7b48103290 0x7f7b481a0160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:43268/0 (socket says 172.21.15.42:43268) 2024-08-30T09:04:45.262 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.259+0000 7f7b4d4ce640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7b48104cb0 0x7f7b481a1380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:45.262 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.259+0000 7f7b4d4ce640 1 -- 172.21.15.42:0/762751704 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f7b48111160 con 0x7f7b48104310 2024-08-30T09:04:45.262 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.259+0000 7f7b467fc640 1 -- 172.21.15.42:0/762751704 learned_addr learned my addr 172.21.15.42:0/762751704 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:04:45.262 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.259+0000 7f7b4d4ce640 1 -- 172.21.15.42:0/762751704 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f7b48110fe0 con 0x7f7b48104cb0 2024-08-30T09:04:45.262 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.259+0000 7f7b4d4ce640 1 -- 172.21.15.42:0/762751704 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f7b481112e0 con 0x7f7b48103290 2024-08-30T09:04:45.263 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.260+0000 7f7b47fff640 1 --2- 172.21.15.42:0/762751704 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7b48104cb0 0x7f7b481a1380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:45.263 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.260+0000 7f7b46ffd640 1 -- 172.21.15.42:0/762751704 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7b48104cb0 msgr2=0x7f7b481a1380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:04:45.263 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.260+0000 7f7b46ffd640 1 --2- 172.21.15.42:0/762751704 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7b48104cb0 0x7f7b481a1380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:45.263 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.260+0000 7f7b46ffd640 1 -- 172.21.15.42:0/762751704 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7b48104310 msgr2=0x7f7b481a07d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:04:45.264 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.260+0000 7f7b46ffd640 1 --2- 172.21.15.42:0/762751704 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7b48104310 0x7f7b481a07d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:45.264 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.260+0000 7f7b46ffd640 1 -- 172.21.15.42:0/762751704 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b481a1ba0 con 0x7f7b48103290 2024-08-30T09:04:45.264 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.260+0000 7f7b47fff640 1 --2- 172.21.15.42:0/762751704 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7b48104cb0 0x7f7b481a1380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-30T09:04:45.264 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.260+0000 7f7b46ffd640 1 --2- 172.21.15.42:0/762751704 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7b48103290 0x7f7b481a0160 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f7b3c00e9f0 tx=0x7f7b3c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:04:45.264 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.260+0000 7f7b2ffff640 1 -- 172.21.15.42:0/762751704 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b3c00ccc0 con 0x7f7b48103290 2024-08-30T09:04:45.264 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.261+0000 7f7b4d4ce640 1 -- 172.21.15.42:0/762751704 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b481a1e90 con 0x7f7b48103290 2024-08-30T09:04:45.264 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.261+0000 7f7b2ffff640 1 -- 172.21.15.42:0/762751704 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7b3c0108e0 con 0x7f7b48103290 2024-08-30T09:04:45.264 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.261+0000 7f7b4d4ce640 1 -- 172.21.15.42:0/762751704 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b481a8580 con 0x7f7b48103290 2024-08-30T09:04:45.264 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.261+0000 7f7b2ffff640 1 -- 172.21.15.42:0/762751704 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b3c018880 con 0x7f7b48103290 2024-08-30T09:04:45.265 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.263+0000 7f7b54434740 1 Processor -- start 2024-08-30T09:04:45.266 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.263+0000 7f7b54434740 1 -- start start 2024-08-30T09:04:45.266 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.263+0000 7f7b54434740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f5e2e761b0 0x55f5e321c380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:45.267 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.263+0000 7f7b54434740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f5e2e4d2f0 0x55f5e321c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:45.267 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.263+0000 7f7b54434740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f5e2e554d0 0x55f5e321d540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:45.267 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.264+0000 7f7b54434740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55f5e318f000 con 0x55f5e2e4d2f0 2024-08-30T09:04:45.267 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.264+0000 7f7b54434740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55f5e317e590 con 0x55f5e2e761b0 2024-08-30T09:04:45.267 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.264+0000 7f7b54434740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55f5e3182540 con 0x55f5e2e554d0 2024-08-30T09:04:45.267 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.264+0000 7f7b47fff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f5e2e761b0 0x55f5e321c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:45.267 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.264+0000 7f7b467fc640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f5e2e554d0 0x55f5e321d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:45.267 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.264+0000 7f7b47fff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f5e2e761b0 0x55f5e321c380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:47918/0 (socket says 172.21.15.42:47918) 2024-08-30T09:04:45.267 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.264+0000 7f7b47fff640 1 -- 172.21.15.42:0/4217702434 learned_addr learned my addr 172.21.15.42:0/4217702434 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:04:45.267 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.264+0000 7f7b47fff640 1 -- 172.21.15.42:0/4217702434 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f5e2e554d0 msgr2=0x55f5e321d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:04:45.267 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.264+0000 7f7b47fff640 1 --2- 172.21.15.42:0/4217702434 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f5e2e554d0 0x55f5e321d540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:45.267 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.264+0000 7f7b47fff640 1 -- 172.21.15.42:0/4217702434 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f5e2e4d2f0 msgr2=0x55f5e321c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:04:45.268 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.264+0000 7f7b47fff640 1 --2- 172.21.15.42:0/4217702434 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f5e2e4d2f0 0x55f5e321c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:45.268 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.264+0000 7f7b47fff640 1 -- 172.21.15.42:0/4217702434 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f5e2ee1790 con 0x55f5e2e761b0 2024-08-30T09:04:45.268 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.264+0000 7f7b467fc640 1 --2- 172.21.15.42:0/4217702434 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f5e2e554d0 0x55f5e321d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-30T09:04:45.268 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.264+0000 7f7b2ffff640 1 -- 172.21.15.42:0/762751704 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7f7b3c010430 con 0x7f7b48103290 2024-08-30T09:04:45.268 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.264+0000 7f7b47fff640 1 --2- 172.21.15.42:0/4217702434 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f5e2e761b0 0x55f5e321c380 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f7b38009bb0 tx=0x7f7b38010740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=133710e83b923212 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:04:45.268 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.265+0000 7f7b2d7fa640 1 -- 172.21.15.42:0/4217702434 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b38004020 con 0x55f5e2e761b0 2024-08-30T09:04:45.268 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.265+0000 7f7b2d7fa640 1 -- 172.21.15.42:0/4217702434 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7b380041c0 con 0x55f5e2e761b0 2024-08-30T09:04:45.268 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.265+0000 7f7b2d7fa640 1 -- 172.21.15.42:0/4217702434 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b380126b0 con 0x55f5e2e761b0 2024-08-30T09:04:45.270 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.266+0000 7f7b2ffff640 1 --2- 172.21.15.42:0/762751704 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f7b300b4c30 0x7f7b300b7140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:45.270 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.266+0000 7f7b54434740 1 -- 172.21.15.42:0/4217702434 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f5e2e761b0 msgr2=0x55f5e321c380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:04:45.270 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.266+0000 7f7b54434740 1 --2- 172.21.15.42:0/4217702434 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f5e2e761b0 0x55f5e321c380 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f7b38009bb0 tx=0x7f7b38010740 comp rx=0 tx=0).stop 2024-08-30T09:04:45.270 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.267+0000 7f7b54434740 1 -- 172.21.15.42:0/4217702434 shutdown_connections 2024-08-30T09:04:45.270 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.267+0000 7f7b54434740 1 --2- 172.21.15.42:0/4217702434 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f5e2e554d0 0x55f5e321d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:45.270 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.267+0000 7f7b54434740 1 --2- 172.21.15.42:0/4217702434 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f5e2e4d2f0 0x55f5e321c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:45.271 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.267+0000 7f7b54434740 1 --2- 172.21.15.42:0/4217702434 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f5e2e761b0 0x55f5e321c380 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:45.271 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.267+0000 7f7b54434740 1 -- 172.21.15.42:0/4217702434 >> 172.21.15.42:0/4217702434 conn(0x55f5e2e6a300 msgr2=0x55f5e3217e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:04:45.271 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.267+0000 7f7b46ffd640 1 --2- 172.21.15.42:0/762751704 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f7b300b4c30 0x7f7b300b7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:45.271 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.267+0000 7f7b54434740 1 -- 172.21.15.42:0/4217702434 shutdown_connections 2024-08-30T09:04:45.271 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.267+0000 7f7b2ffff640 1 -- 172.21.15.42:0/762751704 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f7b3c0e57e0 con 0x7f7b48103290 2024-08-30T09:04:45.271 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.267+0000 7f7b54434740 1 -- 172.21.15.42:0/4217702434 wait complete. 2024-08-30T09:04:45.271 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.267+0000 7f7b46ffd640 1 --2- 172.21.15.42:0/762751704 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f7b300b4c30 0x7f7b300b7140 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f7b3c0a7ea0 tx=0x7f7b3c013d50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:04:45.271 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T09:04:45.275 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.268+0000 7f7b54434740 1 -- start start 2024-08-30T09:04:45.275 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.268+0000 7f7b54434740 1 Processor -- start 2024-08-30T09:04:45.275 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.268+0000 7f7b54434740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f5e2e4d2f0 0x55f5e3236ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:45.275 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.268+0000 7f7b54434740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f5e2e554d0 0x55f5e3237580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:45.276 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.268+0000 7f7b54434740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f5e2e761b0 0x55f5e32380f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:45.276 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.268+0000 7f7b54434740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55f5e30b7ab0 con 0x55f5e2e4d2f0 2024-08-30T09:04:45.276 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.268+0000 7f7b54434740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55f5e3105680 con 0x55f5e2e761b0 2024-08-30T09:04:45.276 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.268+0000 7f7b54434740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55f5e321e060 con 0x55f5e2e554d0 2024-08-30T09:04:45.276 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.268+0000 7f7b54434740 10 client.0 _ll_register_callbacks cb 0x55f5e2e8da00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:04:45.276 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.268+0000 7f7b47fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f5e2e4d2f0 0x55f5e3236ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:45.276 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.268+0000 7f7b47fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f5e2e4d2f0 0x55f5e3236ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35502/0 (socket says 172.21.15.42:35502) 2024-08-30T09:04:45.276 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.268+0000 7f7b467fc640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f5e2e554d0 0x55f5e3237580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:45.276 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.268+0000 7f7b47fff640 1 -- 172.21.15.42:0/3707896999 learned_addr learned my addr 172.21.15.42:0/3707896999 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:04:45.276 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.269+0000 7f7b47fff640 1 --2- 172.21.15.42:0/3707896999 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f5e2e761b0 0x55f5e32380f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:45.276 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.269+0000 7f7b47fff640 1 -- 172.21.15.42:0/3707896999 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f5e2e554d0 msgr2=0x55f5e3237580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:04:45.276 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.269+0000 7f7b47fff640 1 --2- 172.21.15.42:0/3707896999 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f5e2e554d0 0x55f5e3237580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:45.277 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.269+0000 7f7b47fff640 1 -- 172.21.15.42:0/3707896999 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f5e2e761b0 msgr2=0x55f5e32380f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:04:45.277 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.269+0000 7f7b47fff640 1 --2- 172.21.15.42:0/3707896999 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f5e2e761b0 0x55f5e32380f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:45.277 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.269+0000 7f7b47fff640 1 -- 172.21.15.42:0/3707896999 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f5e2f5a620 con 0x55f5e2e4d2f0 2024-08-30T09:04:45.277 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.269+0000 7f7b47fff640 1 --2- 172.21.15.42:0/3707896999 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f5e2e4d2f0 0x55f5e3236ec0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f7b38005e00 tx=0x7f7b38005d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d16b326e63d0d5a server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:04:45.277 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.269+0000 7f7b2aff5640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-08-30T09:04:45.277 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.269+0000 7f7b2aff5640 1 -- 172.21.15.42:0/3707896999 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b38022040 con 0x55f5e2e4d2f0 2024-08-30T09:04:45.277 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.269+0000 7f7b2aff5640 1 -- 172.21.15.42:0/3707896999 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f7b38003870 con 0x55f5e2e4d2f0 2024-08-30T09:04:45.277 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.269+0000 7f7b54434740 1 -- 172.21.15.42:0/3707896999 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b380041c0 con 0x55f5e2e4d2f0 2024-08-30T09:04:45.277 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.269+0000 7f7b2aff5640 1 -- 172.21.15.42:0/3707896999 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f7b38012450 con 0x55f5e2e4d2f0 2024-08-30T09:04:45.277 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.269+0000 7f7b54434740 10 client.16110 fetch_fsmap learned FSMap version 29 2024-08-30T09:04:45.277 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.269+0000 7f7b54434740 1 -- 172.21.15.42:0/3707896999 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f5e3241070 con 0x55f5e2e4d2f0 2024-08-30T09:04:45.277 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.270+0000 7f7b2aff5640 1 -- 172.21.15.42:0/3707896999 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7b380041c0 con 0x55f5e2e4d2f0 2024-08-30T09:04:45.277 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.270+0000 7f7b2aff5640 1 -- 172.21.15.42:0/3707896999 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f7b38028070 con 0x55f5e2e4d2f0 2024-08-30T09:04:45.277 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.270+0000 7f7b54434740 10 client.16110 fetch_fsmap finished waiting for FSMap version 29 2024-08-30T09:04:45.278 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.270+0000 7f7b54434740 10 client.16110 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-30T09:04:45.278 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.270+0000 7f7b54434740 20 client.16110 populate_metadata read hostname 'smithi042' 2024-08-30T09:04:45.278 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.270+0000 7f7b54434740 1 --2- 172.21.15.42:0/3707896999 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55f5e2e34d80 0x55f5e32445f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:45.278 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.270+0000 7f7b54434740 4 client.16110 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-30T09:04:45.278 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.270+0000 7f7b54434740 1 -- 172.21.15.42:0/3707896999 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f5e3190f50 con 0x55f5e2e34d80 2024-08-30T09:04:45.278 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.270+0000 7f7b467fc640 1 --2- 172.21.15.42:0/3707896999 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55f5e2e34d80 0x55f5e32445f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:45.278 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.271+0000 7f7b2aff5640 10 client.16110 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:04:45.278 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.271+0000 7f7b467fc640 1 --2- 172.21.15.42:0/3707896999 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55f5e2e34d80 0x55f5e32445f0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ce7c3f694467a1d server_cookie=92f3dac3c371eaf0 in_seq=0 out_seq=0 2024-08-30T09:04:45.278 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.271+0000 7f7b2aff5640 1 -- 172.21.15.42:0/3707896999 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f5e3190f50 con 0x55f5e2e34d80 2024-08-30T09:04:45.278 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.271+0000 7f7b2aff5640 10 client.16110 handle_command_reply: tid=0 2024-08-30T09:04:45.278 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.271+0000 7f7b54434740 1 client.16110 shutdown 2024-08-30T09:04:45.278 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.272+0000 7f7b54434740 1 -- 172.21.15.42:0/3707896999 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f5e2e4d2f0 msgr2=0x55f5e3236ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:04:45.279 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.272+0000 7f7b54434740 1 --2- 172.21.15.42:0/3707896999 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f5e2e4d2f0 0x55f5e3236ec0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f7b38005e00 tx=0x7f7b38005d70 comp rx=0 tx=0).stop 2024-08-30T09:04:45.279 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.272+0000 7f7b54434740 1 -- 172.21.15.42:0/3707896999 shutdown_connections 2024-08-30T09:04:45.279 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.272+0000 7f7b54434740 1 --2- 172.21.15.42:0/3707896999 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55f5e2e34d80 0x55f5e32445f0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:45.279 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.272+0000 7f7b2aff5640 0 client.16110 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:04:45.279 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.272+0000 7f7b54434740 1 --2- 172.21.15.42:0/3707896999 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f5e2e554d0 0x55f5e3237580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:45.279 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.272+0000 7f7b54434740 1 --2- 172.21.15.42:0/3707896999 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f5e2e761b0 0x55f5e32380f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:45.279 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.272+0000 7f7b54434740 1 --2- 172.21.15.42:0/3707896999 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f5e2e4d2f0 0x55f5e3236ec0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:45.279 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.272+0000 7f7b54434740 1 -- 172.21.15.42:0/3707896999 >> 172.21.15.42:0/3707896999 conn(0x55f5e2e6a300 msgr2=0x55f5e3216370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:04:45.279 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.272+0000 7f7b54434740 1 -- 172.21.15.42:0/3707896999 shutdown_connections 2024-08-30T09:04:45.279 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.272+0000 7f7b54434740 1 -- 172.21.15.42:0/3707896999 wait complete. 2024-08-30T09:04:45.279 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.272+0000 7f7b54434740 20 client.16110 trim_cache size 0 max 16384 2024-08-30T09:04:45.296 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.293+0000 7f7b54434740 1 Processor -- start 2024-08-30T09:04:45.296 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.293+0000 7f7b54434740 1 -- start start 2024-08-30T09:04:45.296 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.293+0000 7f7b54434740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f5e2e34d80 0x55f5e321bf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:45.296 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.293+0000 7f7b54434740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f5e2e4d2f0 0x55f5e3235590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:45.297 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.293+0000 7f7b54434740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f5e2e554d0 0x55f5e3235ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:45.297 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.293+0000 7f7b54434740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f7b34012490 con 0x55f5e2e554d0 2024-08-30T09:04:45.297 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.293+0000 7f7b54434740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f7b3400fd60 con 0x55f5e2e34d80 2024-08-30T09:04:45.297 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.293+0000 7f7b54434740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f7b3400dbb0 con 0x55f5e2e4d2f0 2024-08-30T09:04:45.297 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.293+0000 7f7b47fff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f5e2e34d80 0x55f5e321bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:45.297 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.293+0000 7f7b467fc640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f5e2e4d2f0 0x55f5e3235590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:45.297 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.293+0000 7f7b47fff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f5e2e34d80 0x55f5e321bf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:47936/0 (socket says 172.21.15.42:47936) 2024-08-30T09:04:45.297 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.293+0000 7f7b467fc640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f5e2e4d2f0 0x55f5e3235590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:43282/0 (socket says 172.21.15.42:43282) 2024-08-30T09:04:45.297 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.293+0000 7f7b47fff640 1 -- 172.21.15.42:0/2708646816 learned_addr learned my addr 172.21.15.42:0/2708646816 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:04:45.297 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.294+0000 7f7b47fff640 1 --2- 172.21.15.42:0/2708646816 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f5e2e554d0 0x55f5e3235ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:45.297 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.294+0000 7f7b467fc640 1 -- 172.21.15.42:0/2708646816 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f5e2e34d80 msgr2=0x55f5e321bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:04:45.298 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.294+0000 7f7b467fc640 1 --2- 172.21.15.42:0/2708646816 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f5e2e34d80 0x55f5e321bf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:45.298 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.294+0000 7f7b467fc640 1 -- 172.21.15.42:0/2708646816 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f5e2e554d0 msgr2=0x55f5e3235ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:04:45.298 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.294+0000 7f7b467fc640 1 --2- 172.21.15.42:0/2708646816 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f5e2e554d0 0x55f5e3235ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:45.298 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.294+0000 7f7b467fc640 1 -- 172.21.15.42:0/2708646816 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f5e3218540 con 0x55f5e2e4d2f0 2024-08-30T09:04:45.298 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.294+0000 7f7b467fc640 1 --2- 172.21.15.42:0/2708646816 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f5e2e4d2f0 0x55f5e3235590 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f7b40000f80 tx=0x7f7b40031af0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6593f6d514160902 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:04:45.298 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.294+0000 7f7b2cff9640 1 -- 172.21.15.42:0/2708646816 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b40053070 con 0x55f5e2e4d2f0 2024-08-30T09:04:45.298 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.294+0000 7f7b2cff9640 1 -- 172.21.15.42:0/2708646816 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7b40036040 con 0x55f5e2e4d2f0 2024-08-30T09:04:45.298 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.294+0000 7f7b2cff9640 1 -- 172.21.15.42:0/2708646816 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b400046e0 con 0x55f5e2e4d2f0 2024-08-30T09:04:45.303 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.295+0000 7f7b54434740 1 -- 172.21.15.42:0/2708646816 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f5e2e4d2f0 msgr2=0x55f5e3235590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:04:45.303 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.295+0000 7f7b54434740 1 --2- 172.21.15.42:0/2708646816 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f5e2e4d2f0 0x55f5e3235590 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f7b40000f80 tx=0x7f7b40031af0 comp rx=0 tx=0).stop 2024-08-30T09:04:45.303 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.295+0000 7f7b54434740 1 -- 172.21.15.42:0/2708646816 shutdown_connections 2024-08-30T09:04:45.303 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.295+0000 7f7b54434740 1 --2- 172.21.15.42:0/2708646816 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f5e2e4d2f0 0x55f5e3235590 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:45.303 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.295+0000 7f7b54434740 1 --2- 172.21.15.42:0/2708646816 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f5e2e554d0 0x55f5e3235ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:45.303 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.295+0000 7f7b54434740 1 --2- 172.21.15.42:0/2708646816 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f5e2e34d80 0x55f5e321bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:45.303 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.295+0000 7f7b54434740 1 -- 172.21.15.42:0/2708646816 >> 172.21.15.42:0/2708646816 conn(0x55f5e2e6a300 msgr2=0x55f5e3220df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:04:45.303 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.296+0000 7f7b54434740 1 -- 172.21.15.42:0/2708646816 shutdown_connections 2024-08-30T09:04:45.303 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.296+0000 7f7b54434740 1 -- 172.21.15.42:0/2708646816 wait complete. 2024-08-30T09:04:45.303 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.296+0000 7f7b54434740 1 -- start start 2024-08-30T09:04:45.303 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.296+0000 7f7b54434740 1 Processor -- start 2024-08-30T09:04:45.303 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.297+0000 7f7b54434740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f5e2e34d80 0x55f5e323ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:45.303 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.297+0000 7f7b54434740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f5e2e4d2f0 0x55f5e323d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:45.304 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.297+0000 7f7b54434740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f5e2e554d0 0x55f5e323de90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:45.304 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.297+0000 7f7b54434740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f7b3400e220 con 0x55f5e2e4d2f0 2024-08-30T09:04:45.304 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.297+0000 7f7b54434740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f7b3400f690 con 0x55f5e2e554d0 2024-08-30T09:04:45.304 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.297+0000 7f7b54434740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f7b3400e8f0 con 0x55f5e2e34d80 2024-08-30T09:04:45.304 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.297+0000 7f7b54434740 10 client.0 _ll_register_callbacks cb 0x7f7b38005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:04:45.304 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.297+0000 7f7b47fff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f5e2e34d80 0x55f5e323ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:45.304 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.297+0000 7f7b467fc640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f5e2e4d2f0 0x55f5e323d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:45.304 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.297+0000 7f7b467fc640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f5e2e4d2f0 0x55f5e323d950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35520/0 (socket says 172.21.15.42:35520) 2024-08-30T09:04:45.304 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.297+0000 7f7b467fc640 1 -- 172.21.15.42:0/472561862 learned_addr learned my addr 172.21.15.42:0/472561862 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:04:45.304 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.297+0000 7f7b47fff640 1 --2- 172.21.15.42:0/472561862 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f5e2e34d80 0x55f5e323ec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:43294/0 (socket says 172.21.15.42:43294) 2024-08-30T09:04:45.304 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.297+0000 7f7b47fff640 1 --2- 172.21.15.42:0/472561862 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f5e2e554d0 0x55f5e323de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:45.304 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.297+0000 7f7b467fc640 1 -- 172.21.15.42:0/472561862 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f5e2e34d80 msgr2=0x55f5e323ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:04:45.304 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.297+0000 7f7b467fc640 1 --2- 172.21.15.42:0/472561862 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f5e2e34d80 0x55f5e323ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:45.305 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.297+0000 7f7b467fc640 1 -- 172.21.15.42:0/472561862 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f5e2e554d0 msgr2=0x55f5e323de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:04:45.305 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.297+0000 7f7b467fc640 1 --2- 172.21.15.42:0/472561862 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f5e2e554d0 0x55f5e323de90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:45.305 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.297+0000 7f7b467fc640 1 -- 172.21.15.42:0/472561862 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f5e323e600 con 0x55f5e2e4d2f0 2024-08-30T09:04:45.305 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.298+0000 7f7b467fc640 1 --2- 172.21.15.42:0/472561862 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f5e2e4d2f0 0x55f5e323d950 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f7b40031540 tx=0x7f7b40031570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d1729e72324a772 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:04:45.305 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.298+0000 7f7b2aff5640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-08-30T09:04:45.305 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.298+0000 7f7b2aff5640 1 -- 172.21.15.42:0/472561862 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b40053040 con 0x55f5e2e4d2f0 2024-08-30T09:04:45.305 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.298+0000 7f7b2aff5640 1 -- 172.21.15.42:0/472561862 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f7b40004460 con 0x55f5e2e4d2f0 2024-08-30T09:04:45.305 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.298+0000 7f7b54434740 1 -- 172.21.15.42:0/472561862 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b40036040 con 0x55f5e2e4d2f0 2024-08-30T09:04:45.305 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.298+0000 7f7b2aff5640 1 -- 172.21.15.42:0/472561862 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f7b40004660 con 0x55f5e2e4d2f0 2024-08-30T09:04:45.305 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.298+0000 7f7b54434740 10 client.16116 fetch_fsmap learned FSMap version 29 2024-08-30T09:04:45.305 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.298+0000 7f7b54434740 1 -- 172.21.15.42:0/472561862 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f5e32aef90 con 0x55f5e2e4d2f0 2024-08-30T09:04:45.305 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.298+0000 7f7b2aff5640 1 -- 172.21.15.42:0/472561862 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7b40004800 con 0x55f5e2e4d2f0 2024-08-30T09:04:45.305 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.299+0000 7f7b2aff5640 1 -- 172.21.15.42:0/472561862 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f7b4005f030 con 0x55f5e2e4d2f0 2024-08-30T09:04:45.305 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.299+0000 7f7b54434740 10 client.16116 fetch_fsmap finished waiting for FSMap version 29 2024-08-30T09:04:45.305 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.299+0000 7f7b54434740 10 client.16116 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-30T09:04:45.306 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.299+0000 7f7b54434740 20 client.16116 populate_metadata read hostname 'smithi042' 2024-08-30T09:04:45.306 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.299+0000 7f7b54434740 1 --2- 172.21.15.42:0/472561862 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55f5e2e761b0 0x55f5e32b1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:45.306 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.299+0000 7f7b54434740 4 client.16116 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-30T09:04:45.306 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.299+0000 7f7b54434740 1 -- 172.21.15.42:0/472561862 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f7b40036040 con 0x55f5e2e761b0 2024-08-30T09:04:45.306 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.299+0000 7f7b47fff640 1 --2- 172.21.15.42:0/472561862 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55f5e2e761b0 0x55f5e32b1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:45.306 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.299+0000 7f7b2aff5640 10 client.16116 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:04:45.306 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.299+0000 7f7b47fff640 1 --2- 172.21.15.42:0/472561862 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55f5e2e761b0 0x55f5e32b1a10 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=55d98a3aa29f4311 server_cookie=b58c045320b303c in_seq=0 out_seq=0 2024-08-30T09:04:45.471 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.468+0000 7f7b2aff5640 1 -- 172.21.15.42:0/472561862 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f7b40036040 con 0x55f5e2e761b0 2024-08-30T09:04:45.471 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.468+0000 7f7b2aff5640 10 client.16116 handle_command_reply: tid=0 2024-08-30T09:04:45.471 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.468+0000 7f7b54434740 1 client.16116 shutdown 2024-08-30T09:04:45.471 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.469+0000 7f7b54434740 1 -- 172.21.15.42:0/472561862 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f5e2e4d2f0 msgr2=0x55f5e323d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:04:45.471 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.469+0000 7f7b54434740 1 --2- 172.21.15.42:0/472561862 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f5e2e4d2f0 0x55f5e323d950 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f7b40031540 tx=0x7f7b40031570 comp rx=0 tx=0).stop 2024-08-30T09:04:45.471 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.469+0000 7f7b54434740 1 -- 172.21.15.42:0/472561862 shutdown_connections 2024-08-30T09:04:45.471 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.469+0000 7f7b54434740 1 --2- 172.21.15.42:0/472561862 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55f5e2e761b0 0x55f5e32b1a10 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:45.471 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.469+0000 7f7b54434740 1 --2- 172.21.15.42:0/472561862 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f5e2e554d0 0x55f5e323de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:45.472 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.469+0000 7f7b54434740 1 --2- 172.21.15.42:0/472561862 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f5e2e4d2f0 0x55f5e323d950 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:45.472 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.469+0000 7f7b2aff5640 0 client.16116 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:04:45.472 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.469+0000 7f7b54434740 1 --2- 172.21.15.42:0/472561862 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f5e2e34d80 0x55f5e323ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:45.472 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.469+0000 7f7b54434740 1 -- 172.21.15.42:0/472561862 >> 172.21.15.42:0/472561862 conn(0x55f5e2e6a300 msgr2=0x55f5e321bf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:04:45.472 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.469+0000 7f7b54434740 1 -- 172.21.15.42:0/472561862 shutdown_connections 2024-08-30T09:04:45.472 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.469+0000 7f7b54434740 1 -- 172.21.15.42:0/472561862 wait complete. 2024-08-30T09:04:45.472 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.469+0000 7f7b54434740 20 client.16116 trim_cache size 0 max 16384 2024-08-30T09:04:45.472 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-08-30T09:04:45.472 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2024-08-30T09:04:45.472 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-08-30T09:04:45.472 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-08-30T09:04:45.473 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.469+0000 7f7b4d4ce640 1 -- 172.21.15.42:0/762751704 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f7b300b4c30 msgr2=0x7f7b300b7140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:04:45.473 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.469+0000 7f7b4d4ce640 1 --2- 172.21.15.42:0/762751704 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f7b300b4c30 0x7f7b300b7140 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f7b3c0a7ea0 tx=0x7f7b3c013d50 comp rx=0 tx=0).stop 2024-08-30T09:04:45.473 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.469+0000 7f7b4d4ce640 1 -- 172.21.15.42:0/762751704 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7b48103290 msgr2=0x7f7b481a0160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:04:45.473 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.469+0000 7f7b4d4ce640 1 --2- 172.21.15.42:0/762751704 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7b48103290 0x7f7b481a0160 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f7b3c00e9f0 tx=0x7f7b3c00eeb0 comp rx=0 tx=0).stop 2024-08-30T09:04:45.474 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.469+0000 7f7b4d4ce640 1 -- 172.21.15.42:0/762751704 shutdown_connections 2024-08-30T09:04:45.474 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.469+0000 7f7b4d4ce640 1 --2- 172.21.15.42:0/762751704 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f7b300b4c30 0x7f7b300b7140 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:45.474 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.469+0000 7f7b4d4ce640 1 --2- 172.21.15.42:0/762751704 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7b48104cb0 0x7f7b481a1380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:45.474 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.469+0000 7f7b4d4ce640 1 --2- 172.21.15.42:0/762751704 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7b48104310 0x7f7b481a07d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:45.474 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.469+0000 7f7b4d4ce640 1 --2- 172.21.15.42:0/762751704 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7b48103290 0x7f7b481a0160 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:45.474 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.470+0000 7f7b4d4ce640 1 -- 172.21.15.42:0/762751704 >> 172.21.15.42:0/762751704 conn(0x7f7b480febf0 msgr2=0x7f7b4810dc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:04:45.474 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.470+0000 7f7b4d4ce640 1 -- 172.21.15.42:0/762751704 shutdown_connections 2024-08-30T09:04:45.474 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:45.470+0000 7f7b4d4ce640 1 -- 172.21.15.42:0/762751704 wait complete. 2024-08-30T09:04:45.953 INFO:teuthology.orchestra.run.smithi042.stderr:progress: 240.0 s, 852.6 tps, lat 119.623 ms stddev 146.222 2024-08-30T09:04:46.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:45.999+0000 7f93ff3a8640 1 -- 172.21.15.42:0/3674144301 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f93f8107030 msgr2=0x7f93f8107440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:04:46.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:45.999+0000 7f93ff3a8640 1 --2- 172.21.15.42:0/3674144301 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f93f8107030 0x7f93f8107440 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f93e8009ca0 tx=0x7f93e802a8f0 comp rx=0 tx=0).stop 2024-08-30T09:04:46.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:45.999+0000 7f93ff3a8640 1 -- 172.21.15.42:0/3674144301 shutdown_connections 2024-08-30T09:04:46.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:45.999+0000 7f93ff3a8640 1 --2- 172.21.15.42:0/3674144301 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f93f8107030 0x7f93f8107440 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:46.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:45.999+0000 7f93ff3a8640 1 --2- 172.21.15.42:0/3674144301 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f93f8107980 0x7f93f81081a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:46.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:45.999+0000 7f93ff3a8640 1 --2- 172.21.15.42:0/3674144301 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f93f8101900 0x7f93f8101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:46.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:45.999+0000 7f93ff3a8640 1 -- 172.21.15.42:0/3674144301 >> 172.21.15.42:0/3674144301 conn(0x7f93f80775a0 msgr2=0x7f93f80779e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:04:46.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.005+0000 7f93ff3a8640 1 -- 172.21.15.42:0/3674144301 shutdown_connections 2024-08-30T09:04:46.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.007+0000 7f93ff3a8640 1 -- 172.21.15.42:0/3674144301 wait complete. 2024-08-30T09:04:46.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.008+0000 7f93ff3a8640 1 Processor -- start 2024-08-30T09:04:46.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.012+0000 7f93ff3a8640 1 -- start start 2024-08-30T09:04:46.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.012+0000 7f93ff3a8640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f93f8101900 0x7f93f8109800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:46.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.012+0000 7f93ff3a8640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f93f8107030 0x7f93f8109d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:46.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.012+0000 7f93ff3a8640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f93f8107980 0x7f93f810db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:46.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.012+0000 7f93ff3a8640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f93f806d460 con 0x7f93f8107030 2024-08-30T09:04:46.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.012+0000 7f93ff3a8640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f93f806d2e0 con 0x7f93f8107980 2024-08-30T09:04:46.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.012+0000 7f93ff3a8640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f93f806d5e0 con 0x7f93f8101900 2024-08-30T09:04:46.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.013+0000 7f93fc943640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f93f8107030 0x7f93f8109d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:46.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.013+0000 7f93fc943640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f93f8107030 0x7f93f8109d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35530/0 (socket says 172.21.15.42:35530) 2024-08-30T09:04:46.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.013+0000 7f93fc943640 1 -- 172.21.15.42:0/2322147461 learned_addr learned my addr 172.21.15.42:0/2322147461 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:04:46.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.013+0000 7f93fd144640 1 --2- 172.21.15.42:0/2322147461 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f93f8101900 0x7f93f8109800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:46.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.014+0000 7f93fd945640 1 --2- 172.21.15.42:0/2322147461 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f93f8107980 0x7f93f810db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:46.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.021+0000 7f93fd144640 1 -- 172.21.15.42:0/2322147461 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f93f8107980 msgr2=0x7f93f810db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:04:46.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.021+0000 7f93fd144640 1 --2- 172.21.15.42:0/2322147461 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f93f8107980 0x7f93f810db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:46.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.021+0000 7f93fd144640 1 -- 172.21.15.42:0/2322147461 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f93f8107030 msgr2=0x7f93f8109d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:04:46.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.021+0000 7f93fd144640 1 --2- 172.21.15.42:0/2322147461 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f93f8107030 0x7f93f8109d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:46.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.021+0000 7f93fd144640 1 -- 172.21.15.42:0/2322147461 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93f810e220 con 0x7f93f8101900 2024-08-30T09:04:46.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.028+0000 7f93fd144640 1 --2- 172.21.15.42:0/2322147461 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f93f8101900 0x7f93f8109800 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f93f000b3f0 tx=0x7f93f000b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:04:46.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.029+0000 7f93f67fc640 1 -- 172.21.15.42:0/2322147461 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93f0015020 con 0x7f93f8101900 2024-08-30T09:04:46.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.030+0000 7f93f67fc640 1 -- 172.21.15.42:0/2322147461 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f93f0011ce0 con 0x7f93f8101900 2024-08-30T09:04:46.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.030+0000 7f93f67fc640 1 -- 172.21.15.42:0/2322147461 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93f0010d80 con 0x7f93f8101900 2024-08-30T09:04:46.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.032+0000 7f93ff3a8640 1 -- 172.21.15.42:0/2322147461 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93f810e510 con 0x7f93f8101900 2024-08-30T09:04:46.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.032+0000 7f93ff3a8640 1 -- 172.21.15.42:0/2322147461 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93f8115e70 con 0x7f93f8101900 2024-08-30T09:04:46.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.036+0000 7f940591d740 1 Processor -- start 2024-08-30T09:04:46.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.038+0000 7f93f67fc640 1 -- 172.21.15.42:0/2322147461 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7f93f0018020 con 0x7f93f8101900 2024-08-30T09:04:46.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.043+0000 7f940591d740 1 -- start start 2024-08-30T09:04:46.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.043+0000 7f93f67fc640 1 --2- 172.21.15.42:0/2322147461 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f93e00b4d40 0x7f93e00b7250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:46.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.044+0000 7f93f67fc640 1 -- 172.21.15.42:0/2322147461 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f93f00e4a80 con 0x7f93f8101900 2024-08-30T09:04:46.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.045+0000 7f940591d740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f5a6c06070 0x55f5a6fab2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:46.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.045+0000 7f940591d740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f5a6bdd1b0 0x55f5a6fab800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:46.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.045+0000 7f940591d740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f5a6be5390 0x55f5a6fabd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:46.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.045+0000 7f940591d740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55f5a6f22630 con 0x55f5a6be5390 2024-08-30T09:04:46.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.045+0000 7f940591d740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55f5a6f11bc0 con 0x55f5a6c06070 2024-08-30T09:04:46.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.045+0000 7f940591d740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55f5a6f15b70 con 0x55f5a6bdd1b0 2024-08-30T09:04:46.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.046+0000 7f93fd945640 1 --2- 172.21.15.42:0/2322147461 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f93e00b4d40 0x7f93e00b7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:46.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.046+0000 7f93fc943640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f5a6bdd1b0 0x55f5a6fab800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:46.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.047+0000 7f93fc943640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f5a6bdd1b0 0x55f5a6fab800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:43312/0 (socket says 172.21.15.42:43312) 2024-08-30T09:04:46.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.047+0000 7f93fc943640 1 -- 172.21.15.42:0/2303841339 learned_addr learned my addr 172.21.15.42:0/2303841339 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:04:46.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.047+0000 7f93fd945640 1 --2- 172.21.15.42:0/2303841339 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f5a6be5390 0x55f5a6fabd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:46.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.047+0000 7f93fc943640 1 -- 172.21.15.42:0/2303841339 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f5a6c06070 msgr2=0x55f5a6fab2c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-30T09:04:46.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.047+0000 7f93fc943640 1 --2- 172.21.15.42:0/2303841339 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f5a6c06070 0x55f5a6fab2c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:46.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.047+0000 7f93fc943640 1 -- 172.21.15.42:0/2303841339 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f5a6be5390 msgr2=0x55f5a6fabd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:04:46.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.047+0000 7f93fc943640 1 --2- 172.21.15.42:0/2303841339 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f5a6be5390 0x55f5a6fabd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:46.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.047+0000 7f93fc943640 1 -- 172.21.15.42:0/2303841339 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f5a6c71650 con 0x55f5a6bdd1b0 2024-08-30T09:04:46.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.048+0000 7f93fc943640 1 --2- 172.21.15.42:0/2303841339 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f5a6bdd1b0 0x55f5a6fab800 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f93e8009bf0 tx=0x7f93e80099e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a058628713fa41b0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:04:46.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.049+0000 7f93df7fe640 1 -- 172.21.15.42:0/2303841339 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93e8031740 con 0x55f5a6bdd1b0 2024-08-30T09:04:46.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.049+0000 7f93df7fe640 1 -- 172.21.15.42:0/2303841339 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f93e8040070 con 0x55f5a6bdd1b0 2024-08-30T09:04:46.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.050+0000 7f940591d740 1 -- 172.21.15.42:0/2303841339 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f5a6bdd1b0 msgr2=0x55f5a6fab800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:04:46.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.050+0000 7f940591d740 1 --2- 172.21.15.42:0/2303841339 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f5a6bdd1b0 0x55f5a6fab800 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f93e8009bf0 tx=0x7f93e80099e0 comp rx=0 tx=0).stop 2024-08-30T09:04:46.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.051+0000 7f940591d740 1 -- 172.21.15.42:0/2303841339 shutdown_connections 2024-08-30T09:04:46.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.051+0000 7f940591d740 1 --2- 172.21.15.42:0/2303841339 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f5a6bdd1b0 0x55f5a6fab800 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:46.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.051+0000 7f940591d740 1 --2- 172.21.15.42:0/2303841339 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f5a6be5390 0x55f5a6fabd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:46.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.051+0000 7f940591d740 1 --2- 172.21.15.42:0/2303841339 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f5a6c06070 0x55f5a6fab2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:46.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.051+0000 7f940591d740 1 -- 172.21.15.42:0/2303841339 >> 172.21.15.42:0/2303841339 conn(0x55f5a6bfa460 msgr2=0x55f5a6fa91b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:04:46.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.054+0000 7f93fd945640 1 --2- 172.21.15.42:0/2322147461 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f93e00b4d40 0x7f93e00b7250 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f93ec00f560 tx=0x7f93ec009830 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:04:46.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.055+0000 7f940591d740 1 -- 172.21.15.42:0/2303841339 shutdown_connections 2024-08-30T09:04:46.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.055+0000 7f940591d740 1 -- 172.21.15.42:0/2303841339 wait complete. 2024-08-30T09:04:46.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.055+0000 7f940591d740 1 -- start start 2024-08-30T09:04:46.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.055+0000 7f940591d740 1 Processor -- start 2024-08-30T09:04:46.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.056+0000 7f940591d740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f5a6bdd1b0 0x55f5a6fb69e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:46.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.056+0000 7f940591d740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f5a6be5390 0x55f5a6fb6f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:46.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.056+0000 7f940591d740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f5a6c06070 0x55f5a6fc2dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:46.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.056+0000 7f940591d740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55f5a6d74700 con 0x55f5a6be5390 2024-08-30T09:04:46.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.056+0000 7f940591d740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55f5a6e98700 con 0x55f5a6c06070 2024-08-30T09:04:46.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.056+0000 7f940591d740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55f5a6fb9a90 con 0x55f5a6bdd1b0 2024-08-30T09:04:46.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.056+0000 7f940591d740 10 client.0 _ll_register_callbacks cb 0x55f5a6ec7dc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:04:46.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.057+0000 7f93fd144640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f5a6bdd1b0 0x55f5a6fb69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:46.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.057+0000 7f93fd144640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f5a6bdd1b0 0x55f5a6fb69e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:43318/0 (socket says 172.21.15.42:43318) 2024-08-30T09:04:46.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.057+0000 7f93fd144640 1 -- 172.21.15.42:0/3570958773 learned_addr learned my addr 172.21.15.42:0/3570958773 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:04:46.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.057+0000 7f93fd945640 1 --2- 172.21.15.42:0/3570958773 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f5a6c06070 0x55f5a6fc2dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:46.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.057+0000 7f93fc943640 1 --2- 172.21.15.42:0/3570958773 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f5a6be5390 0x55f5a6fb6f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:46.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.057+0000 7f93fd144640 1 -- 172.21.15.42:0/3570958773 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f5a6c06070 msgr2=0x55f5a6fc2dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:04:46.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.057+0000 7f93fd144640 1 --2- 172.21.15.42:0/3570958773 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f5a6c06070 0x55f5a6fc2dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:46.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.057+0000 7f93fd144640 1 -- 172.21.15.42:0/3570958773 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f5a6be5390 msgr2=0x55f5a6fb6f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:04:46.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.057+0000 7f93fd144640 1 --2- 172.21.15.42:0/3570958773 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f5a6be5390 0x55f5a6fb6f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:46.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.057+0000 7f93fd144640 1 -- 172.21.15.42:0/3570958773 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f5a6ed18b0 con 0x55f5a6bdd1b0 2024-08-30T09:04:46.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.058+0000 7f93fc943640 1 --2- 172.21.15.42:0/3570958773 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f5a6be5390 0x55f5a6fb6f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-30T09:04:46.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.058+0000 7f93fd144640 1 --2- 172.21.15.42:0/3570958773 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f5a6bdd1b0 0x55f5a6fb69e0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f93f000bdb0 tx=0x7f93f00092c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b722f10551724522 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:04:46.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.058+0000 7f93dcff9640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-08-30T09:04:46.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.058+0000 7f93dcff9640 1 -- 172.21.15.42:0/3570958773 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93f00ab0c0 con 0x55f5a6bdd1b0 2024-08-30T09:04:46.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.059+0000 7f940591d740 1 -- 172.21.15.42:0/3570958773 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f93e8040070 con 0x55f5a6bdd1b0 2024-08-30T09:04:46.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.059+0000 7f93dcff9640 1 -- 172.21.15.42:0/3570958773 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f93f00a7350 con 0x55f5a6bdd1b0 2024-08-30T09:04:46.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.059+0000 7f93dcff9640 1 -- 172.21.15.42:0/3570958773 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f93f00af930 con 0x55f5a6bdd1b0 2024-08-30T09:04:46.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.059+0000 7f940591d740 10 client.25852 fetch_fsmap learned FSMap version 29 2024-08-30T09:04:46.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.059+0000 7f940591d740 1 -- 172.21.15.42:0/3570958773 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f5a6fc5850 con 0x55f5a6bdd1b0 2024-08-30T09:04:46.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.060+0000 7f93dcff9640 1 -- 172.21.15.42:0/3570958773 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f93e8040070 con 0x55f5a6bdd1b0 2024-08-30T09:04:46.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.060+0000 7f93dcff9640 1 -- 172.21.15.42:0/3570958773 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f93f0022060 con 0x55f5a6bdd1b0 2024-08-30T09:04:46.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.060+0000 7f940591d740 10 client.25852 fetch_fsmap finished waiting for FSMap version 29 2024-08-30T09:04:46.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.060+0000 7f940591d740 10 client.25852 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-30T09:04:46.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.060+0000 7f940591d740 20 client.25852 populate_metadata read hostname 'smithi042' 2024-08-30T09:04:46.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.060+0000 7f940591d740 1 --2- 172.21.15.42:0/3570958773 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55f5a6bc49f0 0x55f5a6fc9050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:46.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.060+0000 7f940591d740 4 client.25852 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-30T09:04:46.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.060+0000 7f940591d740 1 -- 172.21.15.42:0/3570958773 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f5a6f24580 con 0x55f5a6bc49f0 2024-08-30T09:04:46.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.065+0000 7f93fc943640 1 --2- 172.21.15.42:0/3570958773 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55f5a6bc49f0 0x55f5a6fc9050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:46.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.067+0000 7f93fc943640 1 --2- 172.21.15.42:0/3570958773 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55f5a6bc49f0 0x55f5a6fc9050 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=97a64c9aef47dce7 server_cookie=89ab054ecf290cf2 in_seq=0 out_seq=0 2024-08-30T09:04:46.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.067+0000 7f93dcff9640 10 client.25852 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:04:46.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.070+0000 7f93dcff9640 1 -- 172.21.15.42:0/3570958773 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f5a6f24580 con 0x55f5a6bc49f0 2024-08-30T09:04:46.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.070+0000 7f93dcff9640 10 client.25852 handle_command_reply: tid=0 2024-08-30T09:04:46.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.070+0000 7f940591d740 1 client.25852 shutdown 2024-08-30T09:04:46.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.071+0000 7f940591d740 1 -- 172.21.15.42:0/3570958773 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f5a6bdd1b0 msgr2=0x55f5a6fb69e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:04:46.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.071+0000 7f940591d740 1 --2- 172.21.15.42:0/3570958773 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f5a6bdd1b0 0x55f5a6fb69e0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f93f000bdb0 tx=0x7f93f00092c0 comp rx=0 tx=0).stop 2024-08-30T09:04:46.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.071+0000 7f940591d740 1 -- 172.21.15.42:0/3570958773 shutdown_connections 2024-08-30T09:04:46.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.071+0000 7f940591d740 1 --2- 172.21.15.42:0/3570958773 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55f5a6bc49f0 0x55f5a6fc9050 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:46.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.071+0000 7f940591d740 1 --2- 172.21.15.42:0/3570958773 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f5a6c06070 0x55f5a6fc2dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:46.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.071+0000 7f940591d740 1 --2- 172.21.15.42:0/3570958773 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f5a6be5390 0x55f5a6fb6f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:46.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.071+0000 7f940591d740 1 --2- 172.21.15.42:0/3570958773 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f5a6bdd1b0 0x55f5a6fb69e0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:46.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.071+0000 7f940591d740 1 -- 172.21.15.42:0/3570958773 >> 172.21.15.42:0/3570958773 conn(0x55f5a6bfa460 msgr2=0x55f5a6fae4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:04:46.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.071+0000 7f93dcff9640 0 client.25852 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:04:46.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.072+0000 7f940591d740 1 -- 172.21.15.42:0/3570958773 shutdown_connections 2024-08-30T09:04:46.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.072+0000 7f940591d740 1 -- 172.21.15.42:0/3570958773 wait complete. 2024-08-30T09:04:46.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.072+0000 7f940591d740 20 client.25852 trim_cache size 0 max 16384 2024-08-30T09:04:46.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.103+0000 7f940591d740 1 Processor -- start 2024-08-30T09:04:46.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.104+0000 7f940591d740 1 -- start start 2024-08-30T09:04:46.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.104+0000 7f940591d740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f5a6bdd1b0 0x55f5a6fb8610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:46.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.104+0000 7f940591d740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f5a6be5390 0x55f5a6fa8780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:46.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.104+0000 7f940591d740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f5a6c06070 0x55f5a6fa8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:46.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.104+0000 7f940591d740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f93f81ca990 con 0x55f5a6c06070 2024-08-30T09:04:46.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.104+0000 7f940591d740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f93f81c8260 con 0x55f5a6bdd1b0 2024-08-30T09:04:46.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.104+0000 7f940591d740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f93f81c60b0 con 0x55f5a6be5390 2024-08-30T09:04:46.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.105+0000 7f93fd945640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f5a6c06070 0x55f5a6fa8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:46.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.105+0000 7f93fd144640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f5a6bdd1b0 0x55f5a6fb8610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:46.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.105+0000 7f93fd945640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f5a6c06070 0x55f5a6fa8cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35550/0 (socket says 172.21.15.42:35550) 2024-08-30T09:04:46.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.105+0000 7f93fd144640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f5a6bdd1b0 0x55f5a6fb8610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:47988/0 (socket says 172.21.15.42:47988) 2024-08-30T09:04:46.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.105+0000 7f93fd945640 1 -- 172.21.15.42:0/2042320428 learned_addr learned my addr 172.21.15.42:0/2042320428 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:04:46.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.105+0000 7f93fc943640 1 --2- 172.21.15.42:0/2042320428 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f5a6be5390 0x55f5a6fa8780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:46.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.105+0000 7f93fd945640 1 -- 172.21.15.42:0/2042320428 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f5a6be5390 msgr2=0x55f5a6fa8780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:04:46.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.105+0000 7f93fd945640 1 --2- 172.21.15.42:0/2042320428 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f5a6be5390 0x55f5a6fa8780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:46.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.105+0000 7f93fd945640 1 -- 172.21.15.42:0/2042320428 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f5a6bdd1b0 msgr2=0x55f5a6fb8610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:04:46.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.105+0000 7f93fd945640 1 --2- 172.21.15.42:0/2042320428 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f5a6bdd1b0 0x55f5a6fb8610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:46.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.105+0000 7f93fd945640 1 -- 172.21.15.42:0/2042320428 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f5a6fb15f0 con 0x55f5a6c06070 2024-08-30T09:04:46.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.105+0000 7f93fd144640 1 --2- 172.21.15.42:0/2042320428 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f5a6bdd1b0 0x55f5a6fb8610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-30T09:04:46.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.105+0000 7f93fd945640 1 --2- 172.21.15.42:0/2042320428 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f5a6c06070 0x55f5a6fa8cc0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f93ec004340 tx=0x7f93ec004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6bd5c5e59042984 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:04:46.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.105+0000 7f93deffd640 1 -- 172.21.15.42:0/2042320428 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93ec026070 con 0x55f5a6c06070 2024-08-30T09:04:46.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.106+0000 7f93deffd640 1 -- 172.21.15.42:0/2042320428 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f93ec02b910 con 0x55f5a6c06070 2024-08-30T09:04:46.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.106+0000 7f93deffd640 1 -- 172.21.15.42:0/2042320428 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93ec0226f0 con 0x55f5a6c06070 2024-08-30T09:04:46.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.106+0000 7f940591d740 1 -- 172.21.15.42:0/2042320428 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f5a6c06070 msgr2=0x55f5a6fa8cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:04:46.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.106+0000 7f940591d740 1 --2- 172.21.15.42:0/2042320428 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f5a6c06070 0x55f5a6fa8cc0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f93ec004340 tx=0x7f93ec004730 comp rx=0 tx=0).stop 2024-08-30T09:04:46.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.107+0000 7f940591d740 1 -- 172.21.15.42:0/2042320428 shutdown_connections 2024-08-30T09:04:46.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.107+0000 7f940591d740 1 --2- 172.21.15.42:0/2042320428 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f5a6be5390 0x55f5a6fa8780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:46.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.107+0000 7f940591d740 1 --2- 172.21.15.42:0/2042320428 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f5a6c06070 0x55f5a6fa8cc0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:46.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.107+0000 7f940591d740 1 --2- 172.21.15.42:0/2042320428 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f5a6bdd1b0 0x55f5a6fb8610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:46.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.107+0000 7f940591d740 1 -- 172.21.15.42:0/2042320428 >> 172.21.15.42:0/2042320428 conn(0x55f5a6bfa460 msgr2=0x55f5a6fb7f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:04:46.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.107+0000 7f940591d740 1 -- 172.21.15.42:0/2042320428 shutdown_connections 2024-08-30T09:04:46.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.107+0000 7f940591d740 1 -- 172.21.15.42:0/2042320428 wait complete. 2024-08-30T09:04:46.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.108+0000 7f940591d740 1 -- start start 2024-08-30T09:04:46.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.108+0000 7f940591d740 1 Processor -- start 2024-08-30T09:04:46.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.109+0000 7f940591d740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f5a6bdd1b0 0x55f5a6fbf390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:46.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.109+0000 7f940591d740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f5a6be5390 0x55f5a6fba840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:46.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.109+0000 7f940591d740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f5a6c06070 0x55f5a6fbad80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:46.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.109+0000 7f940591d740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f93f81c6720 con 0x55f5a6be5390 2024-08-30T09:04:46.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.109+0000 7f940591d740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f93f81c7b90 con 0x55f5a6c06070 2024-08-30T09:04:46.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.109+0000 7f940591d740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f93f81c6df0 con 0x55f5a6bdd1b0 2024-08-30T09:04:46.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.109+0000 7f940591d740 10 client.0 _ll_register_callbacks cb 0x7f93f0010fa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:04:46.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.109+0000 7f93fd144640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f5a6bdd1b0 0x55f5a6fbf390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:46.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.109+0000 7f93fd144640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f5a6bdd1b0 0x55f5a6fbf390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:43328/0 (socket says 172.21.15.42:43328) 2024-08-30T09:04:46.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.109+0000 7f93fd144640 1 -- 172.21.15.42:0/1482533232 learned_addr learned my addr 172.21.15.42:0/1482533232 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:04:46.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.109+0000 7f93fc943640 1 --2- 172.21.15.42:0/1482533232 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f5a6be5390 0x55f5a6fba840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:46.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.109+0000 7f93fd945640 1 --2- 172.21.15.42:0/1482533232 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f5a6c06070 0x55f5a6fbad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:46.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.109+0000 7f93fd144640 1 -- 172.21.15.42:0/1482533232 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f5a6c06070 msgr2=0x55f5a6fbad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:04:46.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.109+0000 7f93fd144640 1 --2- 172.21.15.42:0/1482533232 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f5a6c06070 0x55f5a6fbad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:46.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.109+0000 7f93fd144640 1 -- 172.21.15.42:0/1482533232 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f5a6be5390 msgr2=0x55f5a6fba840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:04:46.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.109+0000 7f93fd144640 1 --2- 172.21.15.42:0/1482533232 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f5a6be5390 0x55f5a6fba840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:46.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.109+0000 7f93fd144640 1 -- 172.21.15.42:0/1482533232 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f5a6fbb3c0 con 0x55f5a6bdd1b0 2024-08-30T09:04:46.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.110+0000 7f93fc943640 1 --2- 172.21.15.42:0/1482533232 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f5a6be5390 0x55f5a6fba840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-30T09:04:46.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.110+0000 7f93fd144640 1 --2- 172.21.15.42:0/1482533232 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f5a6bdd1b0 0x55f5a6fbf390 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f93f00aadc0 tx=0x7f93f0096580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8a697639f9fe3449 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:04:46.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.110+0000 7f93dcff9640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-08-30T09:04:46.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.110+0000 7f93dcff9640 1 -- 172.21.15.42:0/1482533232 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93f00c5020 con 0x55f5a6bdd1b0 2024-08-30T09:04:46.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.110+0000 7f93dcff9640 1 -- 172.21.15.42:0/1482533232 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f93f00a7350 con 0x55f5a6bdd1b0 2024-08-30T09:04:46.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.110+0000 7f940591d740 1 -- 172.21.15.42:0/1482533232 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f93ec02b910 con 0x55f5a6bdd1b0 2024-08-30T09:04:46.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.110+0000 7f93dcff9640 1 -- 172.21.15.42:0/1482533232 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f93f00bc050 con 0x55f5a6bdd1b0 2024-08-30T09:04:46.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.110+0000 7f940591d740 10 client.25858 fetch_fsmap learned FSMap version 29 2024-08-30T09:04:46.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.110+0000 7f940591d740 1 -- 172.21.15.42:0/1482533232 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f5a6fbb3c0 con 0x55f5a6bdd1b0 2024-08-30T09:04:46.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.111+0000 7f93dcff9640 1 -- 172.21.15.42:0/1482533232 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f93ec02b910 con 0x55f5a6bdd1b0 2024-08-30T09:04:46.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.111+0000 7f93dcff9640 1 -- 172.21.15.42:0/1482533232 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f93f00cc350 con 0x55f5a6bdd1b0 2024-08-30T09:04:46.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.111+0000 7f940591d740 10 client.25858 fetch_fsmap finished waiting for FSMap version 29 2024-08-30T09:04:46.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.111+0000 7f940591d740 10 client.25858 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-30T09:04:46.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.111+0000 7f940591d740 20 client.25858 populate_metadata read hostname 'smithi042' 2024-08-30T09:04:46.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.111+0000 7f940591d740 1 --2- 172.21.15.42:0/1482533232 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55f5a6b99da0 0x55f5a7035cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:46.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.111+0000 7f940591d740 4 client.25858 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-30T09:04:46.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.111+0000 7f940591d740 1 -- 172.21.15.42:0/1482533232 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f5a6f20020 con 0x55f5a6b99da0 2024-08-30T09:04:46.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.111+0000 7f93fc943640 1 --2- 172.21.15.42:0/1482533232 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55f5a6b99da0 0x55f5a7035cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:46.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.112+0000 7f93dcff9640 10 client.25858 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:04:46.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.112+0000 7f93fc943640 1 --2- 172.21.15.42:0/1482533232 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55f5a6b99da0 0x55f5a7035cc0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e8308ce7f62ea33 server_cookie=ffb0a546cda6f3ef in_seq=0 out_seq=0 2024-08-30T09:04:46.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.112+0000 7f93dcff9640 1 -- 172.21.15.42:0/1482533232 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+1510 (crc 0 0 0) 0x55f5a6f20020 con 0x55f5a6b99da0 2024-08-30T09:04:46.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.112+0000 7f93dcff9640 10 client.25858 handle_command_reply: tid=0 2024-08-30T09:04:46.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.112+0000 7f940591d740 1 client.25858 shutdown 2024-08-30T09:04:46.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.112+0000 7f940591d740 1 -- 172.21.15.42:0/1482533232 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f5a6bdd1b0 msgr2=0x55f5a6fbf390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:04:46.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.112+0000 7f940591d740 1 --2- 172.21.15.42:0/1482533232 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f5a6bdd1b0 0x55f5a6fbf390 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f93f00aadc0 tx=0x7f93f0096580 comp rx=0 tx=0).stop 2024-08-30T09:04:46.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.112+0000 7f940591d740 1 -- 172.21.15.42:0/1482533232 shutdown_connections 2024-08-30T09:04:46.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.112+0000 7f940591d740 1 --2- 172.21.15.42:0/1482533232 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55f5a6b99da0 0x55f5a7035cc0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:46.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.112+0000 7f940591d740 1 --2- 172.21.15.42:0/1482533232 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f5a6c06070 0x55f5a6fbad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:46.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.112+0000 7f940591d740 1 --2- 172.21.15.42:0/1482533232 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f5a6be5390 0x55f5a6fba840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:46.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.112+0000 7f940591d740 1 --2- 172.21.15.42:0/1482533232 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f5a6bdd1b0 0x55f5a6fbf390 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:46.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.112+0000 7f940591d740 1 -- 172.21.15.42:0/1482533232 >> 172.21.15.42:0/1482533232 conn(0x55f5a6bfa460 msgr2=0x55f5a6facbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:04:46.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.112+0000 7f93dcff9640 0 client.25858 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:04:46.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.112+0000 7f940591d740 1 -- 172.21.15.42:0/1482533232 shutdown_connections 2024-08-30T09:04:46.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.112+0000 7f940591d740 1 -- 172.21.15.42:0/1482533232 wait complete. 2024-08-30T09:04:46.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.113+0000 7f940591d740 20 client.25858 trim_cache size 0 max 16384 2024-08-30T09:04:46.122 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-08-30T09:04:46.122 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "no active scrubs running", 2024-08-30T09:04:46.122 INFO:teuthology.orchestra.run.smithi042.stdout: "scrubs": {}, 2024-08-30T09:04:46.122 INFO:teuthology.orchestra.run.smithi042.stdout: "scrub_stats": { 2024-08-30T09:04:46.122 INFO:teuthology.orchestra.run.smithi042.stdout: "88cd7478-2b20-47de-ba51-256479749232": { 2024-08-30T09:04:46.122 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:04:08.922625+0000", 2024-08-30T09:04:46.122 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:04:46.122 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:04:46.122 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:04:46.122 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:04:46.123 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:04:46.123 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:04:46.123 INFO:teuthology.orchestra.run.smithi042.stdout: "749f4867-0b70-49e5-af2d-8362a240ebc3": { 2024-08-30T09:04:46.123 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:03:16.386554+0000", 2024-08-30T09:04:46.123 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:04:46.123 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:04:46.123 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:04:46.123 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:04:46.123 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:04:46.123 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:04:46.123 INFO:teuthology.orchestra.run.smithi042.stdout: "85134e4b-ab4a-4885-b07e-a3ad01eaf13d": { 2024-08-30T09:04:46.123 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:01:27.738462+0000", 2024-08-30T09:04:46.123 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:04:46.123 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:04:46.124 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:04:46.124 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:04:46.124 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:04:46.124 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:04:46.124 INFO:teuthology.orchestra.run.smithi042.stdout: "c827ad7a-44ee-4e1d-9bb3-51fb7ab00c73": { 2024-08-30T09:04:46.124 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:02:21.686517+0000", 2024-08-30T09:04:46.124 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:04:46.124 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:04:46.124 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:04:46.124 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:04:46.124 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:04:46.124 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:04:46.124 INFO:teuthology.orchestra.run.smithi042.stdout: "5e2ed104-0625-478a-a78b-584e7f4c2817": { 2024-08-30T09:04:46.124 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:00:33.928400+0000", 2024-08-30T09:04:46.125 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:04:46.125 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:04:46.125 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:04:46.125 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:04:46.125 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:04:46.125 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:04:46.125 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:04:46.125 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-08-30T09:04:46.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.113+0000 7f93ff3a8640 1 -- 172.21.15.42:0/2322147461 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f93e00b4d40 msgr2=0x7f93e00b7250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:04:46.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.113+0000 7f93ff3a8640 1 --2- 172.21.15.42:0/2322147461 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f93e00b4d40 0x7f93e00b7250 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f93ec00f560 tx=0x7f93ec009830 comp rx=0 tx=0).stop 2024-08-30T09:04:46.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.113+0000 7f93ff3a8640 1 -- 172.21.15.42:0/2322147461 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f93f8101900 msgr2=0x7f93f8109800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:04:46.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.113+0000 7f93ff3a8640 1 --2- 172.21.15.42:0/2322147461 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f93f8101900 0x7f93f8109800 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f93f000b3f0 tx=0x7f93f000b8b0 comp rx=0 tx=0).stop 2024-08-30T09:04:46.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.113+0000 7f93ff3a8640 1 -- 172.21.15.42:0/2322147461 shutdown_connections 2024-08-30T09:04:46.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.113+0000 7f93ff3a8640 1 --2- 172.21.15.42:0/2322147461 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f93e00b4d40 0x7f93e00b7250 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:46.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.113+0000 7f93ff3a8640 1 --2- 172.21.15.42:0/2322147461 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f93f8107980 0x7f93f810db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:46.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.113+0000 7f93ff3a8640 1 --2- 172.21.15.42:0/2322147461 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f93f8107030 0x7f93f8109d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:46.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.113+0000 7f93ff3a8640 1 --2- 172.21.15.42:0/2322147461 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f93f8101900 0x7f93f8109800 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:46.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.113+0000 7f93ff3a8640 1 -- 172.21.15.42:0/2322147461 >> 172.21.15.42:0/2322147461 conn(0x7f93f80775a0 msgr2=0x7f93f8100660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:04:46.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.114+0000 7f93ff3a8640 1 -- 172.21.15.42:0/2322147461 shutdown_connections 2024-08-30T09:04:46.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:46.114+0000 7f93ff3a8640 1 -- 172.21.15.42:0/2322147461 wait complete. 2024-08-30T09:04:46.842 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:04:46 smithi042 ceph-mon[18652]: pgmap v493: 129 pgs: 129 active+clean; 1.1 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 9.4 MiB/s wr, 775 op/s 2024-08-30T09:04:46.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:04:46 smithi190 ceph-mon[22001]: pgmap v493: 129 pgs: 129 active+clean; 1.1 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 9.4 MiB/s wr, 775 op/s 2024-08-30T09:04:47.063 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-08-30T09:04:47.064 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph fs dump --format=json 2024-08-30T09:04:47.076 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:04:46 smithi167 ceph-mon[22063]: pgmap v493: 129 pgs: 129 active+clean; 1.1 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 9.4 MiB/s wr, 775 op/s 2024-08-30T09:04:47.531 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T09:04:48.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:48.142+0000 7f1adf692640 1 -- 172.21.15.42:0/119831807 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1ad810a330 msgr2=0x7f1ad810a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:04:48.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:48.142+0000 7f1adf692640 1 --2- 172.21.15.42:0/119831807 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1ad810a330 0x7f1ad810a740 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f1ad4009900 tx=0x7f1ad402a410 comp rx=0 tx=0).stop 2024-08-30T09:04:48.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:48.143+0000 7f1adf692640 1 -- 172.21.15.42:0/119831807 shutdown_connections 2024-08-30T09:04:48.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:48.143+0000 7f1adf692640 1 --2- 172.21.15.42:0/119831807 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1ad810ac80 0x7f1ad8115d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:48.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:48.143+0000 7f1adf692640 1 --2- 172.21.15.42:0/119831807 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1ad81163e0 0x7f1ad81188f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:48.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:48.143+0000 7f1adf692640 1 --2- 172.21.15.42:0/119831807 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1ad810a330 0x7f1ad810a740 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:48.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:48.143+0000 7f1adf692640 1 -- 172.21.15.42:0/119831807 >> 172.21.15.42:0/119831807 conn(0x7f1ad8070c60 msgr2=0x7f1ad8071070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:04:48.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:48.143+0000 7f1adf692640 1 -- 172.21.15.42:0/119831807 shutdown_connections 2024-08-30T09:04:48.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:48.143+0000 7f1adf692640 1 -- 172.21.15.42:0/119831807 wait complete. 2024-08-30T09:04:48.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:48.144+0000 7f1adf692640 1 Processor -- start 2024-08-30T09:04:48.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:48.144+0000 7f1adf692640 1 -- start start 2024-08-30T09:04:48.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:48.144+0000 7f1adf692640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1ad810a330 0x7f1ad81a4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:48.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:48.144+0000 7f1adf692640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1ad810ac80 0x7f1ad81a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:48.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:48.144+0000 7f1adf692640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1ad81163e0 0x7f1ad81aa4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:48.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:48.144+0000 7f1adf692640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f1ad811e2c0 con 0x7f1ad810ac80 2024-08-30T09:04:48.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:48.144+0000 7f1adf692640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f1ad811e140 con 0x7f1ad810a330 2024-08-30T09:04:48.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:48.144+0000 7f1adf692640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f1ad811e440 con 0x7f1ad81163e0 2024-08-30T09:04:48.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:48.144+0000 7f1add42e640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1ad810a330 0x7f1ad81a4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:48.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:48.144+0000 7f1add42e640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1ad810a330 0x7f1ad81a4cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:53164/0 (socket says 172.21.15.42:53164) 2024-08-30T09:04:48.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:48.144+0000 7f1add42e640 1 -- 172.21.15.42:0/3492318212 learned_addr learned my addr 172.21.15.42:0/3492318212 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:04:48.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:48.145+0000 7f1adcc2d640 1 --2- 172.21.15.42:0/3492318212 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1ad810ac80 0x7f1ad81a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:48.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:48.145+0000 7f1addc2f640 1 --2- 172.21.15.42:0/3492318212 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1ad81163e0 0x7f1ad81aa4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:48.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:48.145+0000 7f1add42e640 1 -- 172.21.15.42:0/3492318212 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1ad81163e0 msgr2=0x7f1ad81aa4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:04:48.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:48.145+0000 7f1add42e640 1 --2- 172.21.15.42:0/3492318212 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1ad81163e0 0x7f1ad81aa4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:48.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:48.145+0000 7f1add42e640 1 -- 172.21.15.42:0/3492318212 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1ad810ac80 msgr2=0x7f1ad81a5250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:04:48.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:48.145+0000 7f1add42e640 1 --2- 172.21.15.42:0/3492318212 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1ad810ac80 0x7f1ad81a5250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:48.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:48.145+0000 7f1add42e640 1 -- 172.21.15.42:0/3492318212 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ad81aab60 con 0x7f1ad810a330 2024-08-30T09:04:48.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:48.145+0000 7f1adcc2d640 1 --2- 172.21.15.42:0/3492318212 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1ad810ac80 0x7f1ad81a5250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-30T09:04:48.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:48.145+0000 7f1add42e640 1 --2- 172.21.15.42:0/3492318212 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1ad810a330 0x7f1ad81a4cb0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f1ad4000c00 tx=0x7f1ad4032c40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:04:48.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:48.145+0000 7f1ace7fc640 1 -- 172.21.15.42:0/3492318212 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ad403f030 con 0x7f1ad810a330 2024-08-30T09:04:48.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:48.145+0000 7f1adf692640 1 -- 172.21.15.42:0/3492318212 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ad81aadf0 con 0x7f1ad810a330 2024-08-30T09:04:48.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:48.145+0000 7f1ace7fc640 1 -- 172.21.15.42:0/3492318212 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ad4003cb0 con 0x7f1ad810a330 2024-08-30T09:04:48.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:48.146+0000 7f1ace7fc640 1 -- 172.21.15.42:0/3492318212 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ad4035070 con 0x7f1ad810a330 2024-08-30T09:04:48.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:48.146+0000 7f1adf692640 1 -- 172.21.15.42:0/3492318212 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ad81b1670 con 0x7f1ad810a330 2024-08-30T09:04:48.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:48.147+0000 7f1adf692640 1 -- 172.21.15.42:0/3492318212 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ad810b380 con 0x7f1ad810a330 2024-08-30T09:04:48.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:48.150+0000 7f1ace7fc640 1 -- 172.21.15.42:0/3492318212 <== mon.2 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7f1ad4042030 con 0x7f1ad810a330 2024-08-30T09:04:48.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:48.150+0000 7f1ace7fc640 1 --2- 172.21.15.42:0/3492318212 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f1ac40b4b60 0x7f1ac40b7070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:48.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:48.151+0000 7f1adcc2d640 1 --2- 172.21.15.42:0/3492318212 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f1ac40b4b60 0x7f1ac40b7070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:48.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:48.151+0000 7f1ace7fc640 1 -- 172.21.15.42:0/3492318212 <== mon.2 v2:172.21.15.167:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f1ad40c8830 con 0x7f1ad810a330 2024-08-30T09:04:48.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:48.151+0000 7f1adcc2d640 1 --2- 172.21.15.42:0/3492318212 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f1ac40b4b60 0x7f1ac40b7070 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f1ad8071570 tx=0x7f1ac8008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:04:48.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:48.156+0000 7f1ace7fc640 1 -- 172.21.15.42:0/3492318212 <== mon.2 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f1ad40cc940 con 0x7f1ad810a330 2024-08-30T09:04:48.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:48.507+0000 7f1adf692640 1 -- 172.21.15.42:0/3492318212 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1ad8111460 con 0x7f1ad810a330 2024-08-30T09:04:48.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:48.510+0000 7f1ace7fc640 1 -- 172.21.15.42:0/3492318212 <== mon.2 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11918 (secure 0 0 0) 0x7f1ad4099c30 con 0x7f1ad810a330 2024-08-30T09:04:48.513 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-08-30T09:04:48.513 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":29,"btime":"2024-08-30T09:00:48:414225+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14703,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/3188400000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":3188400000},{"type":"v1","addr":"172.21.15.42:6835","nonce":3188400000}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14715,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/3356537674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":3356537674},{"type":"v1","addr":"172.21.15.42:6837","nonce":3356537674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14766,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6839/2107785513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6838","nonce":2107785513},{"type":"v1","addr":"172.21.15.42:6839","nonce":2107785513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14778,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6841/2298438674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6840","nonce":2298438674},{"type":"v1","addr":"172.21.15.42:6841","nonce":2298438674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24553,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/543555570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":543555570},{"type":"v1","addr":"172.21.15.190:6833","nonce":543555570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24554,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/3907818941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":3907818941},{"type":"v1","addr":"172.21.15.167:6833","nonce":3907818941}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-30T08:58:03.055084+0000","modified":"2024-08-30T09:00:47.637894+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24629,"mds_1":24610,"mds_2":24599},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24572":{"gid":24572,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6835/234596845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":234596845},{"type":"v1","addr":"172.21.15.167:6835","nonce":234596845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24592":{"gid":24592,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6835/247905709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":247905709},{"type":"v1","addr":"172.21.15.190:6835","nonce":247905709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24598":{"gid":24598,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6837/863256349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6836","nonce":863256349},{"type":"v1","addr":"172.21.15.190:6837","nonce":863256349}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24599":{"gid":24599,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.167:6837/3011837856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":3011837856},{"type":"v1","addr":"172.21.15.167:6837","nonce":3011837856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24610":{"gid":24610,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.190:6839/719981110","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6838","nonce":719981110},{"type":"v1","addr":"172.21.15.190:6839","nonce":719981110}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24629":{"gid":24629,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.167:6839/3604645207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":3604645207},{"type":"v1","addr":"172.21.15.167:6839","nonce":3604645207}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24629,"qdb_cluster":[24599,24610,24629]},"id":1}]} 2024-08-30T09:04:48.513 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 29 2024-08-30T09:04:48.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:48.515+0000 7f1adf692640 1 -- 172.21.15.42:0/3492318212 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f1ac40b4b60 msgr2=0x7f1ac40b7070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:04:48.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:48.515+0000 7f1adf692640 1 --2- 172.21.15.42:0/3492318212 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f1ac40b4b60 0x7f1ac40b7070 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f1ad8071570 tx=0x7f1ac8008040 comp rx=0 tx=0).stop 2024-08-30T09:04:48.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:48.515+0000 7f1adf692640 1 -- 172.21.15.42:0/3492318212 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1ad810a330 msgr2=0x7f1ad81a4cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:04:48.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:48.515+0000 7f1adf692640 1 --2- 172.21.15.42:0/3492318212 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1ad810a330 0x7f1ad81a4cb0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f1ad4000c00 tx=0x7f1ad4032c40 comp rx=0 tx=0).stop 2024-08-30T09:04:48.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:48.515+0000 7f1adf692640 1 -- 172.21.15.42:0/3492318212 shutdown_connections 2024-08-30T09:04:48.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:48.515+0000 7f1adf692640 1 --2- 172.21.15.42:0/3492318212 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f1ac40b4b60 0x7f1ac40b7070 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:48.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:48.515+0000 7f1adf692640 1 --2- 172.21.15.42:0/3492318212 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1ad81163e0 0x7f1ad81aa4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:48.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:48.515+0000 7f1adf692640 1 --2- 172.21.15.42:0/3492318212 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1ad810ac80 0x7f1ad81a5250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:48.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:48.515+0000 7f1adf692640 1 --2- 172.21.15.42:0/3492318212 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1ad810a330 0x7f1ad81a4cb0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:48.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:48.515+0000 7f1adf692640 1 -- 172.21.15.42:0/3492318212 >> 172.21.15.42:0/3492318212 conn(0x7f1ad8070c60 msgr2=0x7f1ad8072e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:04:48.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:48.515+0000 7f1adf692640 1 -- 172.21.15.42:0/3492318212 shutdown_connections 2024-08-30T09:04:48.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:48.515+0000 7f1adf692640 1 -- 172.21.15.42:0/3492318212 wait complete. 2024-08-30T09:04:48.886 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:04:48 smithi042 ceph-mon[18652]: pgmap v494: 129 pgs: 129 active+clean; 1.1 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 9.7 MiB/s wr, 829 op/s 2024-08-30T09:04:48.886 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:04:48 smithi042 ceph-mon[18652]: from='client.? 172.21.15.42:0/3492318212' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-30T09:04:48.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:04:48 smithi190 ceph-mon[22001]: pgmap v494: 129 pgs: 129 active+clean; 1.1 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 9.7 MiB/s wr, 829 op/s 2024-08-30T09:04:48.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:04:48 smithi190 ceph-mon[22001]: from='client.? 172.21.15.42:0/3492318212' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-30T09:04:49.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:04:48 smithi167 ceph-mon[22063]: pgmap v494: 129 pgs: 129 active+clean; 1.1 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 9.7 MiB/s wr, 829 op/s 2024-08-30T09:04:49.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:04:48 smithi167 ceph-mon[22063]: from='client.? 172.21.15.42:0/3492318212' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-30T09:04:49.314 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-08-30T09:04:49.715 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T09:04:50.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.363+0000 7f6e0f025640 1 -- 172.21.15.42:0/494400936 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6e08102510 msgr2=0x7f6e08110340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:04:50.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.363+0000 7f6e0f025640 1 --2- 172.21.15.42:0/494400936 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6e08102510 0x7f6e08110340 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f6e00009930 tx=0x7f6e0002a420 comp rx=0 tx=0).stop 2024-08-30T09:04:50.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.363+0000 7f6e0f025640 1 -- 172.21.15.42:0/494400936 shutdown_connections 2024-08-30T09:04:50.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.363+0000 7f6e0f025640 1 --2- 172.21.15.42:0/494400936 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6e08102510 0x7f6e08110340 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:50.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.363+0000 7f6e0f025640 1 --2- 172.21.15.42:0/494400936 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6e08101bc0 0x7f6e08101fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:50.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.363+0000 7f6e0f025640 1 --2- 172.21.15.42:0/494400936 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6e081084d0 0x7f6e081088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:50.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.363+0000 7f6e0f025640 1 -- 172.21.15.42:0/494400936 >> 172.21.15.42:0/494400936 conn(0x7f6e08077a60 msgr2=0x7f6e08077ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:04:50.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.363+0000 7f6e0f025640 1 -- 172.21.15.42:0/494400936 shutdown_connections 2024-08-30T09:04:50.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.363+0000 7f6e0f025640 1 -- 172.21.15.42:0/494400936 wait complete. 2024-08-30T09:04:50.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.365+0000 7f6e0f025640 1 Processor -- start 2024-08-30T09:04:50.370 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.369+0000 7f6e0f025640 1 -- start start 2024-08-30T09:04:50.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.369+0000 7f6e0f025640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6e08101bc0 0x7f6e0819a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:50.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.369+0000 7f6e0f025640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6e08102510 0x7f6e0819a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:50.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.370+0000 7f6e0f025640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6e081084d0 0x7f6e0819fa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:50.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.370+0000 7f6e0f025640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f6e08113850 con 0x7f6e08102510 2024-08-30T09:04:50.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.370+0000 7f6e0f025640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f6e081136d0 con 0x7f6e081084d0 2024-08-30T09:04:50.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.370+0000 7f6e0f025640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f6e081139d0 con 0x7f6e08101bc0 2024-08-30T09:04:50.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.372+0000 7f6e07fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6e08102510 0x7f6e0819a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:50.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.372+0000 7f6e07fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6e08102510 0x7f6e0819a7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:39572/0 (socket says 172.21.15.42:39572) 2024-08-30T09:04:50.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.372+0000 7f6e07fff640 1 -- 172.21.15.42:0/392719990 learned_addr learned my addr 172.21.15.42:0/392719990 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:04:50.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.372+0000 7f6e0d5c2640 1 --2- 172.21.15.42:0/392719990 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6e081084d0 0x7f6e0819fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:50.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.376+0000 7f6e0cdc1640 1 --2- 172.21.15.42:0/392719990 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6e08101bc0 0x7f6e0819a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:50.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.378+0000 7f6e0d5c2640 1 -- 172.21.15.42:0/392719990 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6e08101bc0 msgr2=0x7f6e0819a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:04:50.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.378+0000 7f6e0d5c2640 1 --2- 172.21.15.42:0/392719990 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6e08101bc0 0x7f6e0819a220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:50.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.378+0000 7f6e0d5c2640 1 -- 172.21.15.42:0/392719990 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6e08102510 msgr2=0x7f6e0819a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:04:50.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.378+0000 7f6e0d5c2640 1 --2- 172.21.15.42:0/392719990 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6e08102510 0x7f6e0819a7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:50.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.378+0000 7f6e0d5c2640 1 -- 172.21.15.42:0/392719990 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e081a0190 con 0x7f6e081084d0 2024-08-30T09:04:50.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.385+0000 7f6e0d5c2640 1 --2- 172.21.15.42:0/392719990 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6e081084d0 0x7f6e0819fa60 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f6e00009900 tx=0x7f6e000040c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:04:50.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.391+0000 7f6e05ffb640 1 -- 172.21.15.42:0/392719990 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e00049020 con 0x7f6e081084d0 2024-08-30T09:04:50.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.391+0000 7f6e05ffb640 1 -- 172.21.15.42:0/392719990 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6e000044c0 con 0x7f6e081084d0 2024-08-30T09:04:50.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.392+0000 7f6e05ffb640 1 -- 172.21.15.42:0/392719990 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e00035070 con 0x7f6e081084d0 2024-08-30T09:04:50.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.393+0000 7f6e0f025640 1 -- 172.21.15.42:0/392719990 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e081a0420 con 0x7f6e081084d0 2024-08-30T09:04:50.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.393+0000 7f6e0f025640 1 -- 172.21.15.42:0/392719990 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e081a0880 con 0x7f6e081084d0 2024-08-30T09:04:50.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.397+0000 7f6e1559a740 1 Processor -- start 2024-08-30T09:04:50.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.399+0000 7f6e05ffb640 1 -- 172.21.15.42:0/392719990 <== mon.2 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7f6e0004c020 con 0x7f6e081084d0 2024-08-30T09:04:50.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.401+0000 7f6e05ffb640 1 --2- 172.21.15.42:0/392719990 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f6df00b4b50 0x7f6df00b7060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:50.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.401+0000 7f6e05ffb640 1 -- 172.21.15.42:0/392719990 <== mon.2 v2:172.21.15.167:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f6e0010e190 con 0x7f6e081084d0 2024-08-30T09:04:50.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.403+0000 7f6e07fff640 1 --2- 172.21.15.42:0/392719990 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f6df00b4b50 0x7f6df00b7060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:50.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.405+0000 7f6e1559a740 1 -- start start 2024-08-30T09:04:50.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.405+0000 7f6e1559a740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558be169b070 0x558be1a40310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:50.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.405+0000 7f6e1559a740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558be16721b0 0x558be1a40850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:50.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.405+0000 7f6e1559a740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558be167a390 0x558be1a40d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:50.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.405+0000 7f6e1559a740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x558be19b7680 con 0x558be167a390 2024-08-30T09:04:50.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.405+0000 7f6e1559a740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x558be19a6c10 con 0x558be169b070 2024-08-30T09:04:50.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.405+0000 7f6e1559a740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x558be19aabc0 con 0x558be16721b0 2024-08-30T09:04:50.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.406+0000 7f6e07fff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558be16721b0 0x558be1a40850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:50.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.406+0000 7f6e07fff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558be16721b0 0x558be1a40850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:34582/0 (socket says 172.21.15.42:34582) 2024-08-30T09:04:50.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.406+0000 7f6e07fff640 1 -- 172.21.15.42:0/3512156836 learned_addr learned my addr 172.21.15.42:0/3512156836 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:04:50.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.406+0000 7f6e0d5c2640 1 --2- 172.21.15.42:0/3512156836 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558be167a390 0x558be1a40d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:50.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.407+0000 7f6e07fff640 1 --2- 172.21.15.42:0/392719990 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f6df00b4b50 0x7f6df00b7060 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f6e08105df0 tx=0x7f6dfc011810 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:04:50.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.407+0000 7f6e07fff640 1 -- 172.21.15.42:0/3512156836 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558be169b070 msgr2=0x558be1a40310 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-30T09:04:50.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.407+0000 7f6e07fff640 1 --2- 172.21.15.42:0/3512156836 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558be169b070 0x558be1a40310 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:50.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.407+0000 7f6e07fff640 1 -- 172.21.15.42:0/3512156836 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558be167a390 msgr2=0x558be1a40d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:04:50.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.407+0000 7f6e07fff640 1 --2- 172.21.15.42:0/3512156836 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558be167a390 0x558be1a40d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:50.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.407+0000 7f6e07fff640 1 -- 172.21.15.42:0/3512156836 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558be1706650 con 0x558be16721b0 2024-08-30T09:04:50.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.410+0000 7f6e07fff640 1 --2- 172.21.15.42:0/3512156836 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558be16721b0 0x558be1a40850 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f6dfc0041d0 tx=0x7f6dfc01c000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bfdd01e30ddcc842 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:04:50.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.412+0000 7f6deeffd640 1 -- 172.21.15.42:0/3512156836 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dfc017040 con 0x558be16721b0 2024-08-30T09:04:50.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.412+0000 7f6deeffd640 1 -- 172.21.15.42:0/3512156836 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6dfc019050 con 0x558be16721b0 2024-08-30T09:04:50.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.413+0000 7f6deeffd640 1 -- 172.21.15.42:0/3512156836 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dfc01d440 con 0x558be16721b0 2024-08-30T09:04:50.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.414+0000 7f6e1559a740 1 -- 172.21.15.42:0/3512156836 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558be16721b0 msgr2=0x558be1a40850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:04:50.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.414+0000 7f6e1559a740 1 --2- 172.21.15.42:0/3512156836 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558be16721b0 0x558be1a40850 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f6dfc0041d0 tx=0x7f6dfc01c000 comp rx=0 tx=0).stop 2024-08-30T09:04:50.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.416+0000 7f6e1559a740 1 -- 172.21.15.42:0/3512156836 shutdown_connections 2024-08-30T09:04:50.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.416+0000 7f6e1559a740 1 --2- 172.21.15.42:0/3512156836 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558be16721b0 0x558be1a40850 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:50.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.416+0000 7f6e1559a740 1 --2- 172.21.15.42:0/3512156836 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558be167a390 0x558be1a40d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:50.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.416+0000 7f6e1559a740 1 --2- 172.21.15.42:0/3512156836 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558be169b070 0x558be1a40310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:50.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.416+0000 7f6e1559a740 1 -- 172.21.15.42:0/3512156836 >> 172.21.15.42:0/3512156836 conn(0x558be168f460 msgr2=0x558be1a3e200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:04:50.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.420+0000 7f6e1559a740 1 -- 172.21.15.42:0/3512156836 shutdown_connections 2024-08-30T09:04:50.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.420+0000 7f6e1559a740 1 -- 172.21.15.42:0/3512156836 wait complete. 2024-08-30T09:04:50.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.421+0000 7f6e1559a740 1 -- start start 2024-08-30T09:04:50.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.421+0000 7f6e1559a740 1 Processor -- start 2024-08-30T09:04:50.424 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.422+0000 7f6e1559a740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558be16721b0 0x558be1a4baa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:50.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.422+0000 7f6e1559a740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558be167a390 0x558be1a4bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:50.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.422+0000 7f6e1559a740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558be169b070 0x558be1a57e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:50.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.422+0000 7f6e1559a740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x558be1809700 con 0x558be169b070 2024-08-30T09:04:50.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.422+0000 7f6e1559a740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x558be192d750 con 0x558be16721b0 2024-08-30T09:04:50.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.422+0000 7f6e1559a740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x558be1a4eaf0 con 0x558be167a390 2024-08-30T09:04:50.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.422+0000 7f6e1559a740 10 client.0 _ll_register_callbacks cb 0x558be195ce10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:04:50.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.422+0000 7f6e07fff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558be167a390 0x558be1a4bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:50.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.422+0000 7f6e07fff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558be167a390 0x558be1a4bfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:34592/0 (socket says 172.21.15.42:34592) 2024-08-30T09:04:50.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.422+0000 7f6e07fff640 1 -- 172.21.15.42:0/2878948067 learned_addr learned my addr 172.21.15.42:0/2878948067 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:04:50.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.422+0000 7f6e0cdc1640 1 --2- 172.21.15.42:0/2878948067 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558be16721b0 0x558be1a4baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:50.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.423+0000 7f6e0cdc1640 1 -- 172.21.15.42:0/2878948067 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558be167a390 msgr2=0x558be1a4bfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:04:50.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.423+0000 7f6e0cdc1640 1 --2- 172.21.15.42:0/2878948067 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558be167a390 0x558be1a4bfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:50.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.423+0000 7f6e0cdc1640 1 -- 172.21.15.42:0/2878948067 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558be169b070 msgr2=0x558be1a57e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:04:50.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.423+0000 7f6e0cdc1640 1 --2- 172.21.15.42:0/2878948067 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558be169b070 0x558be1a57e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:50.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.423+0000 7f6e0cdc1640 1 -- 172.21.15.42:0/2878948067 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558be1966900 con 0x558be16721b0 2024-08-30T09:04:50.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.424+0000 7f6e0cdc1640 1 --2- 172.21.15.42:0/2878948067 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558be16721b0 0x558be1a4baa0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f6df800f3e0 tx=0x7f6df800f8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a8b3a8b3b345589f server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:04:50.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.424+0000 7f6de7fff640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-30T09:04:50.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.424+0000 7f6de7fff640 1 -- 172.21.15.42:0/2878948067 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6df8004d20 con 0x558be16721b0 2024-08-30T09:04:50.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.424+0000 7f6e1559a740 1 -- 172.21.15.42:0/2878948067 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6dfc019050 con 0x558be16721b0 2024-08-30T09:04:50.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.425+0000 7f6de7fff640 1 -- 172.21.15.42:0/2878948067 <== mon.2 v2:172.21.15.167:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f6df8011710 con 0x558be16721b0 2024-08-30T09:04:50.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.425+0000 7f6de7fff640 1 -- 172.21.15.42:0/2878948067 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f6df8023050 con 0x558be16721b0 2024-08-30T09:04:50.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.425+0000 7f6e1559a740 10 client.25844 fetch_fsmap learned FSMap version 29 2024-08-30T09:04:50.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.425+0000 7f6e1559a740 1 -- 172.21.15.42:0/2878948067 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558be1a6b970 con 0x558be16721b0 2024-08-30T09:04:50.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.427+0000 7f6de7fff640 1 -- 172.21.15.42:0/2878948067 <== mon.2 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6dfc019050 con 0x558be16721b0 2024-08-30T09:04:50.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.427+0000 7f6de7fff640 1 -- 172.21.15.42:0/2878948067 <== mon.2 v2:172.21.15.167:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f6df80055c0 con 0x558be16721b0 2024-08-30T09:04:50.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.429+0000 7f6e1559a740 10 client.25844 fetch_fsmap finished waiting for FSMap version 29 2024-08-30T09:04:50.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.429+0000 7f6e1559a740 10 client.25844 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-08-30T09:04:50.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.429+0000 7f6e1559a740 20 client.25844 populate_metadata read hostname 'smithi042' 2024-08-30T09:04:50.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.429+0000 7f6e1559a740 1 --2- 172.21.15.42:0/2878948067 >> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] conn(0x558be16599f0 0x558be1a6ef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:50.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.429+0000 7f6e1559a740 4 client.25844 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-30T09:04:50.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.429+0000 7f6e1559a740 1 -- 172.21.15.42:0/2878948067 --> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558be19b95d0 con 0x558be16599f0 2024-08-30T09:04:50.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.430+0000 7f6e07fff640 1 --2- 172.21.15.42:0/2878948067 >> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] conn(0x558be16599f0 0x558be1a6ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:50.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.434+0000 7f6e07fff640 1 --2- 172.21.15.42:0/2878948067 >> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] conn(0x558be16599f0 0x558be1a6ef20 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=dfb83e8c73477015 server_cookie=425bb3049b1e5777 in_seq=0 out_seq=0 2024-08-30T09:04:50.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.435+0000 7f6de7fff640 10 client.25844 ms_handle_connect on v2:172.21.15.167:6836/3011837856 2024-08-30T09:04:50.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.435+0000 7f6de7fff640 1 -- 172.21.15.42:0/2878948067 <== mds.2 v2:172.21.15.167:6836/3011837856 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x558be19b95d0 con 0x558be16599f0 2024-08-30T09:04:50.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.435+0000 7f6de7fff640 10 client.25844 handle_command_reply: tid=0 2024-08-30T09:04:50.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.435+0000 7f6e1559a740 1 client.25844 shutdown 2024-08-30T09:04:50.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.436+0000 7f6e1559a740 1 -- 172.21.15.42:0/2878948067 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558be16721b0 msgr2=0x558be1a4baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:04:50.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.436+0000 7f6e1559a740 1 --2- 172.21.15.42:0/2878948067 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558be16721b0 0x558be1a4baa0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f6df800f3e0 tx=0x7f6df800f8a0 comp rx=0 tx=0).stop 2024-08-30T09:04:50.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.438+0000 7f6e1559a740 1 -- 172.21.15.42:0/2878948067 shutdown_connections 2024-08-30T09:04:50.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.438+0000 7f6e1559a740 1 --2- 172.21.15.42:0/2878948067 >> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] conn(0x558be16599f0 0x558be1a6ef20 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:50.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.438+0000 7f6e1559a740 1 --2- 172.21.15.42:0/2878948067 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558be167a390 0x558be1a4bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:50.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.438+0000 7f6e1559a740 1 --2- 172.21.15.42:0/2878948067 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558be169b070 0x558be1a57e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:50.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.438+0000 7f6e1559a740 1 --2- 172.21.15.42:0/2878948067 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558be16721b0 0x558be1a4baa0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:50.441 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.438+0000 7f6e1559a740 1 -- 172.21.15.42:0/2878948067 >> 172.21.15.42:0/2878948067 conn(0x558be168f460 msgr2=0x558be1a434a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:04:50.441 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.438+0000 7f6de7fff640 0 client.25844 ms_handle_reset on v2:172.21.15.167:6836/3011837856 2024-08-30T09:04:50.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.440+0000 7f6e1559a740 1 -- 172.21.15.42:0/2878948067 shutdown_connections 2024-08-30T09:04:50.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.440+0000 7f6e1559a740 1 -- 172.21.15.42:0/2878948067 wait complete. 2024-08-30T09:04:50.443 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.441+0000 7f6e1559a740 20 client.25844 trim_cache size 0 max 16384 2024-08-30T09:04:50.465 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.464+0000 7f6e1559a740 1 Processor -- start 2024-08-30T09:04:50.465 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.464+0000 7f6e1559a740 1 -- start start 2024-08-30T09:04:50.466 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.464+0000 7f6e1559a740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558be16721b0 0x558be1a47250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:50.466 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.464+0000 7f6e1559a740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558be167a390 0x558be1a41a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:50.466 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.464+0000 7f6e1559a740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558be169b070 0x558be1a48c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:50.466 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.464+0000 7f6e1559a740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f6e081b0d40 con 0x558be169b070 2024-08-30T09:04:50.466 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.464+0000 7f6e1559a740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f6e081ae610 con 0x558be167a390 2024-08-30T09:04:50.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.464+0000 7f6e1559a740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f6e081ac460 con 0x558be16721b0 2024-08-30T09:04:50.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.464+0000 7f6e0cdc1640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558be16721b0 0x558be1a47250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:50.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.464+0000 7f6e0d5c2640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558be169b070 0x558be1a48c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:50.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.464+0000 7f6e0cdc1640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558be16721b0 0x558be1a47250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:34596/0 (socket says 172.21.15.42:34596) 2024-08-30T09:04:50.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.464+0000 7f6e0cdc1640 1 -- 172.21.15.42:0/2512187781 learned_addr learned my addr 172.21.15.42:0/2512187781 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:04:50.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.464+0000 7f6e07fff640 1 --2- 172.21.15.42:0/2512187781 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558be167a390 0x558be1a41a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:50.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.465+0000 7f6e0cdc1640 1 -- 172.21.15.42:0/2512187781 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558be167a390 msgr2=0x558be1a41a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:04:50.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.465+0000 7f6e0cdc1640 1 --2- 172.21.15.42:0/2512187781 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558be167a390 0x558be1a41a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:50.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.465+0000 7f6e0cdc1640 1 -- 172.21.15.42:0/2512187781 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558be169b070 msgr2=0x558be1a48c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:04:50.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.465+0000 7f6e0cdc1640 1 --2- 172.21.15.42:0/2512187781 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558be169b070 0x558be1a48c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:50.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.465+0000 7f6e0cdc1640 1 -- 172.21.15.42:0/2512187781 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558be1a43170 con 0x558be16721b0 2024-08-30T09:04:50.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.465+0000 7f6e0d5c2640 1 --2- 172.21.15.42:0/2512187781 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558be169b070 0x558be1a48c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-30T09:04:50.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.465+0000 7f6e0cdc1640 1 --2- 172.21.15.42:0/2512187781 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558be16721b0 0x558be1a47250 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f6df8000f80 tx=0x7f6df8010700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b6dbed7ee61d6c78 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:04:50.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.465+0000 7f6dee7fc640 1 -- 172.21.15.42:0/2512187781 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6df8020040 con 0x558be16721b0 2024-08-30T09:04:50.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.465+0000 7f6dee7fc640 1 -- 172.21.15.42:0/2512187781 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6df8003e20 con 0x558be16721b0 2024-08-30T09:04:50.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.465+0000 7f6dee7fc640 1 -- 172.21.15.42:0/2512187781 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6df8011970 con 0x558be16721b0 2024-08-30T09:04:50.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.466+0000 7f6e1559a740 1 -- 172.21.15.42:0/2512187781 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558be16721b0 msgr2=0x558be1a47250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:04:50.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.466+0000 7f6e1559a740 1 --2- 172.21.15.42:0/2512187781 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558be16721b0 0x558be1a47250 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f6df8000f80 tx=0x7f6df8010700 comp rx=0 tx=0).stop 2024-08-30T09:04:50.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.466+0000 7f6e1559a740 1 -- 172.21.15.42:0/2512187781 shutdown_connections 2024-08-30T09:04:50.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.466+0000 7f6e1559a740 1 --2- 172.21.15.42:0/2512187781 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558be169b070 0x558be1a48c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:50.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.466+0000 7f6e1559a740 1 --2- 172.21.15.42:0/2512187781 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558be167a390 0x558be1a41a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:50.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.466+0000 7f6e1559a740 1 --2- 172.21.15.42:0/2512187781 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558be16721b0 0x558be1a47250 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:50.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.466+0000 7f6e1559a740 1 -- 172.21.15.42:0/2512187781 >> 172.21.15.42:0/2512187781 conn(0x558be168f460 msgr2=0x558be1a41340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:04:50.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.466+0000 7f6e1559a740 1 -- 172.21.15.42:0/2512187781 shutdown_connections 2024-08-30T09:04:50.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.466+0000 7f6e1559a740 1 -- 172.21.15.42:0/2512187781 wait complete. 2024-08-30T09:04:50.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.467+0000 7f6e1559a740 1 -- start start 2024-08-30T09:04:50.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.467+0000 7f6e1559a740 1 Processor -- start 2024-08-30T09:04:50.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.467+0000 7f6e1559a740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558be16721b0 0x558be1a54410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:50.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.467+0000 7f6e1559a740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558be167a390 0x558be1a518e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:50.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.467+0000 7f6e1559a740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558be169b070 0x558be1a51e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:50.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.467+0000 7f6e1559a740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f6e081acad0 con 0x558be167a390 2024-08-30T09:04:50.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.467+0000 7f6e1559a740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f6e081adf40 con 0x558be16721b0 2024-08-30T09:04:50.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.467+0000 7f6e1559a740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f6e081ad1a0 con 0x558be169b070 2024-08-30T09:04:50.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.467+0000 7f6e0cdc1640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558be16721b0 0x558be1a54410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:50.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.467+0000 7f6e07fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558be167a390 0x558be1a518e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:50.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.467+0000 7f6e0cdc1640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558be16721b0 0x558be1a54410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:53222/0 (socket says 172.21.15.42:53222) 2024-08-30T09:04:50.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.467+0000 7f6e0cdc1640 1 -- 172.21.15.42:0/2503364496 learned_addr learned my addr 172.21.15.42:0/2503364496 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:04:50.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.467+0000 7f6e1559a740 10 client.0 _ll_register_callbacks cb 0x7f6df8009600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:04:50.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.467+0000 7f6e0d5c2640 1 --2- 172.21.15.42:0/2503364496 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558be169b070 0x558be1a51e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:50.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.468+0000 7f6e0cdc1640 1 -- 172.21.15.42:0/2503364496 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558be169b070 msgr2=0x558be1a51e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:04:50.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.468+0000 7f6e0cdc1640 1 --2- 172.21.15.42:0/2503364496 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558be169b070 0x558be1a51e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:50.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.468+0000 7f6e0cdc1640 1 -- 172.21.15.42:0/2503364496 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558be167a390 msgr2=0x558be1a518e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:04:50.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.468+0000 7f6e0cdc1640 1 --2- 172.21.15.42:0/2503364496 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558be167a390 0x558be1a518e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:50.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.468+0000 7f6e0cdc1640 1 -- 172.21.15.42:0/2503364496 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558be1a52460 con 0x558be16721b0 2024-08-30T09:04:50.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.468+0000 7f6e0cdc1640 1 --2- 172.21.15.42:0/2503364496 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558be16721b0 0x558be1a54410 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f6df800a010 tx=0x7f6df8005d10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1de13a4b63efdca0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:04:50.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.468+0000 7f6de7fff640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-30T09:04:50.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.468+0000 7f6de7fff640 1 -- 172.21.15.42:0/2503364496 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6df8003970 con 0x558be16721b0 2024-08-30T09:04:50.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.468+0000 7f6de7fff640 1 -- 172.21.15.42:0/2503364496 <== mon.2 v2:172.21.15.167:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f6df8003b10 con 0x558be16721b0 2024-08-30T09:04:50.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.468+0000 7f6e1559a740 1 -- 172.21.15.42:0/2503364496 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6df8003e20 con 0x558be16721b0 2024-08-30T09:04:50.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.468+0000 7f6de7fff640 1 -- 172.21.15.42:0/2503364496 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f6df803f050 con 0x558be16721b0 2024-08-30T09:04:50.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.469+0000 7f6e1559a740 10 client.25850 fetch_fsmap learned FSMap version 29 2024-08-30T09:04:50.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.469+0000 7f6e1559a740 1 -- 172.21.15.42:0/2503364496 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558be1ad96c0 con 0x558be16721b0 2024-08-30T09:04:50.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.469+0000 7f6de7fff640 1 -- 172.21.15.42:0/2503364496 <== mon.2 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6df8011dd0 con 0x558be16721b0 2024-08-30T09:04:50.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.469+0000 7f6de7fff640 1 -- 172.21.15.42:0/2503364496 <== mon.2 v2:172.21.15.167:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f6df8026050 con 0x558be16721b0 2024-08-30T09:04:50.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.469+0000 7f6e1559a740 10 client.25850 fetch_fsmap finished waiting for FSMap version 29 2024-08-30T09:04:50.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.469+0000 7f6e1559a740 10 client.25850 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-08-30T09:04:50.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.469+0000 7f6e1559a740 20 client.25850 populate_metadata read hostname 'smithi042' 2024-08-30T09:04:50.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.469+0000 7f6e1559a740 1 --2- 172.21.15.42:0/2503364496 >> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] conn(0x558be162eda0 0x558be1adbd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:50.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.469+0000 7f6e1559a740 4 client.25850 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-30T09:04:50.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.469+0000 7f6e1559a740 1 -- 172.21.15.42:0/2503364496 --> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f6df8003e20 con 0x558be162eda0 2024-08-30T09:04:50.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.469+0000 7f6e07fff640 1 --2- 172.21.15.42:0/2503364496 >> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] conn(0x558be162eda0 0x558be1adbd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:50.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.470+0000 7f6de7fff640 10 client.25850 ms_handle_connect on v2:172.21.15.167:6836/3011837856 2024-08-30T09:04:50.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.470+0000 7f6e07fff640 1 --2- 172.21.15.42:0/2503364496 >> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] conn(0x558be162eda0 0x558be1adbd30 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=bdca6afb71446f11 server_cookie=77cc9d79d5bcb665 in_seq=0 out_seq=0 2024-08-30T09:04:50.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.470+0000 7f6de7fff640 1 -- 172.21.15.42:0/2503364496 <== mds.2 v2:172.21.15.167:6836/3011837856 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f6df8003e20 con 0x558be162eda0 2024-08-30T09:04:50.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.470+0000 7f6de7fff640 10 client.25850 handle_command_reply: tid=0 2024-08-30T09:04:50.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.470+0000 7f6e1559a740 1 client.25850 shutdown 2024-08-30T09:04:50.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.470+0000 7f6e1559a740 1 -- 172.21.15.42:0/2503364496 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558be16721b0 msgr2=0x558be1a54410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:04:50.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.470+0000 7f6e1559a740 1 --2- 172.21.15.42:0/2503364496 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558be16721b0 0x558be1a54410 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f6df800a010 tx=0x7f6df8005d10 comp rx=0 tx=0).stop 2024-08-30T09:04:50.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.470+0000 7f6e1559a740 1 -- 172.21.15.42:0/2503364496 shutdown_connections 2024-08-30T09:04:50.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.470+0000 7f6e1559a740 1 --2- 172.21.15.42:0/2503364496 >> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] conn(0x558be162eda0 0x558be1adbd30 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:50.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.470+0000 7f6e1559a740 1 --2- 172.21.15.42:0/2503364496 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558be169b070 0x558be1a51e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:50.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.470+0000 7f6e1559a740 1 --2- 172.21.15.42:0/2503364496 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558be167a390 0x558be1a518e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:50.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.470+0000 7f6de7fff640 0 client.25850 ms_handle_reset on v2:172.21.15.167:6836/3011837856 2024-08-30T09:04:50.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.470+0000 7f6e1559a740 1 --2- 172.21.15.42:0/2503364496 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558be16721b0 0x558be1a54410 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:50.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.470+0000 7f6e1559a740 1 -- 172.21.15.42:0/2503364496 >> 172.21.15.42:0/2503364496 conn(0x558be168f460 msgr2=0x558be1a4cf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:04:50.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.470+0000 7f6e1559a740 1 -- 172.21.15.42:0/2503364496 shutdown_connections 2024-08-30T09:04:50.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.470+0000 7f6e1559a740 1 -- 172.21.15.42:0/2503364496 wait complete. 2024-08-30T09:04:50.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.471+0000 7f6e1559a740 20 client.25850 trim_cache size 0 max 16384 2024-08-30T09:04:50.477 INFO:teuthology.orchestra.run.smithi042.stdout:[] 2024-08-30T09:04:50.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.471+0000 7f6e0f025640 1 -- 172.21.15.42:0/392719990 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f6df00b4b50 msgr2=0x7f6df00b7060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:04:50.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.471+0000 7f6e0f025640 1 --2- 172.21.15.42:0/392719990 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f6df00b4b50 0x7f6df00b7060 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f6e08105df0 tx=0x7f6dfc011810 comp rx=0 tx=0).stop 2024-08-30T09:04:50.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.471+0000 7f6e0f025640 1 -- 172.21.15.42:0/392719990 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6e081084d0 msgr2=0x7f6e0819fa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:04:50.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.471+0000 7f6e0f025640 1 --2- 172.21.15.42:0/392719990 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6e081084d0 0x7f6e0819fa60 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f6e00009900 tx=0x7f6e000040c0 comp rx=0 tx=0).stop 2024-08-30T09:04:50.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.471+0000 7f6e0f025640 1 -- 172.21.15.42:0/392719990 shutdown_connections 2024-08-30T09:04:50.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.471+0000 7f6e0f025640 1 --2- 172.21.15.42:0/392719990 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f6df00b4b50 0x7f6df00b7060 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:50.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.471+0000 7f6e0f025640 1 --2- 172.21.15.42:0/392719990 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6e081084d0 0x7f6e0819fa60 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:50.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.471+0000 7f6e0f025640 1 --2- 172.21.15.42:0/392719990 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6e08102510 0x7f6e0819a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:50.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.471+0000 7f6e0f025640 1 --2- 172.21.15.42:0/392719990 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6e08101bc0 0x7f6e0819a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:50.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.471+0000 7f6e0f025640 1 -- 172.21.15.42:0/392719990 >> 172.21.15.42:0/392719990 conn(0x7f6e08077a60 msgr2=0x7f6e081062a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:04:50.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.472+0000 7f6e0f025640 1 -- 172.21.15.42:0/392719990 shutdown_connections 2024-08-30T09:04:50.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:50.472+0000 7f6e0f025640 1 -- 172.21.15.42:0/392719990 wait complete. 2024-08-30T09:04:50.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:04:50 smithi042 ceph-mon[18652]: pgmap v495: 129 pgs: 129 active+clean; 1.1 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 9.3 MiB/s wr, 798 op/s 2024-08-30T09:04:50.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:04:50 smithi190 ceph-mon[22001]: pgmap v495: 129 pgs: 129 active+clean; 1.1 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 9.3 MiB/s wr, 798 op/s 2024-08-30T09:04:50.953 INFO:teuthology.orchestra.run.smithi042.stderr:progress: 245.0 s, 1219.6 tps, lat 82.446 ms stddev 91.258 2024-08-30T09:04:51.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:04:50 smithi167 ceph-mon[22063]: pgmap v495: 129 pgs: 129 active+clean; 1.1 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 9.3 MiB/s wr, 798 op/s 2024-08-30T09:04:51.389 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-08-30T09:04:51.803 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T09:04:52.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.374+0000 7f2ec665b640 1 -- 172.21.15.42:0/2786543752 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2ec01163e0 msgr2=0x7f2ec01188f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:04:52.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.374+0000 7f2ec665b640 1 --2- 172.21.15.42:0/2786543752 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2ec01163e0 0x7f2ec01188f0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f2eb8009be0 tx=0x7f2eb802a6e0 comp rx=0 tx=0).stop 2024-08-30T09:04:52.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.374+0000 7f2ec665b640 1 -- 172.21.15.42:0/2786543752 shutdown_connections 2024-08-30T09:04:52.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.374+0000 7f2ec665b640 1 --2- 172.21.15.42:0/2786543752 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2ec010ac80 0x7f2ec0115d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:52.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.374+0000 7f2ec665b640 1 --2- 172.21.15.42:0/2786543752 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2ec01163e0 0x7f2ec01188f0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:52.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.374+0000 7f2ec665b640 1 --2- 172.21.15.42:0/2786543752 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2ec010a330 0x7f2ec010a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:52.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.374+0000 7f2ec665b640 1 -- 172.21.15.42:0/2786543752 >> 172.21.15.42:0/2786543752 conn(0x7f2ec0070c60 msgr2=0x7f2ec0071070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:04:52.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.374+0000 7f2ec665b640 1 -- 172.21.15.42:0/2786543752 shutdown_connections 2024-08-30T09:04:52.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.375+0000 7f2ec665b640 1 -- 172.21.15.42:0/2786543752 wait complete. 2024-08-30T09:04:52.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.376+0000 7f2ec665b640 1 Processor -- start 2024-08-30T09:04:52.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.376+0000 7f2ec665b640 1 -- start start 2024-08-30T09:04:52.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.377+0000 7f2ec665b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2ec010a330 0x7f2ec01a9d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:52.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.377+0000 7f2ec665b640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2ec010ac80 0x7f2ec01aa280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:52.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.377+0000 7f2ec665b640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2ec01163e0 0x7f2ec01b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:52.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.377+0000 7f2ec665b640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f2ec011c060 con 0x7f2ec010ac80 2024-08-30T09:04:52.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.377+0000 7f2ec665b640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f2ec011bee0 con 0x7f2ec01163e0 2024-08-30T09:04:52.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.377+0000 7f2ec665b640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f2ec011c1e0 con 0x7f2ec010a330 2024-08-30T09:04:52.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.377+0000 7f2ebffff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2ec010a330 0x7f2ec01a9d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:52.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.377+0000 7f2ec4bf8640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2ec01163e0 0x7f2ec01b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:52.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.377+0000 7f2ec4bf8640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2ec01163e0 0x7f2ec01b4dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:53238/0 (socket says 172.21.15.42:53238) 2024-08-30T09:04:52.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.377+0000 7f2ec4bf8640 1 -- 172.21.15.42:0/4213453670 learned_addr learned my addr 172.21.15.42:0/4213453670 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:04:52.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.378+0000 7f2ec4bf8640 1 -- 172.21.15.42:0/4213453670 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2ec010a330 msgr2=0x7f2ec01a9d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:04:52.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.378+0000 7f2ec4bf8640 1 --2- 172.21.15.42:0/4213453670 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2ec010a330 0x7f2ec01a9d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:52.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.378+0000 7f2ec4bf8640 1 -- 172.21.15.42:0/4213453670 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2ec010ac80 msgr2=0x7f2ec01aa280 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-30T09:04:52.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.378+0000 7f2ec4bf8640 1 --2- 172.21.15.42:0/4213453670 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2ec010ac80 0x7f2ec01aa280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:52.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.378+0000 7f2ec4bf8640 1 -- 172.21.15.42:0/4213453670 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ec01aab20 con 0x7f2ec01163e0 2024-08-30T09:04:52.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.378+0000 7f2ebffff640 1 --2- 172.21.15.42:0/4213453670 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2ec010a330 0x7f2ec01a9d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-30T09:04:52.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.378+0000 7f2ec4bf8640 1 --2- 172.21.15.42:0/4213453670 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2ec01163e0 0x7f2ec01b4dc0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f2eb8004af0 tx=0x7f2eb8003ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:04:52.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.378+0000 7f2ebd7fa640 1 -- 172.21.15.42:0/4213453670 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2eb8041020 con 0x7f2ec01163e0 2024-08-30T09:04:52.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.378+0000 7f2ec665b640 1 -- 172.21.15.42:0/4213453670 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ec01b5360 con 0x7f2ec01163e0 2024-08-30T09:04:52.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.378+0000 7f2ebd7fa640 1 -- 172.21.15.42:0/4213453670 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2eb80318f0 con 0x7f2ec01163e0 2024-08-30T09:04:52.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.378+0000 7f2ebd7fa640 1 -- 172.21.15.42:0/4213453670 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2eb8036040 con 0x7f2ec01163e0 2024-08-30T09:04:52.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.379+0000 7f2ec665b640 1 -- 172.21.15.42:0/4213453670 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ec01b5840 con 0x7f2ec01163e0 2024-08-30T09:04:52.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.382+0000 7f2eccbd0740 1 Processor -- start 2024-08-30T09:04:52.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.383+0000 7f2ebd7fa640 1 -- 172.21.15.42:0/4213453670 <== mon.2 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7f2eb8044020 con 0x7f2ec01163e0 2024-08-30T09:04:52.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.384+0000 7f2eccbd0740 1 -- start start 2024-08-30T09:04:52.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.384+0000 7f2eccbd0740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557401bb1070 0x557401f56310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:52.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.384+0000 7f2ebd7fa640 1 --2- 172.21.15.42:0/4213453670 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f2ea80b4d30 0x7f2ea80b7240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:52.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.384+0000 7f2eccbd0740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557401b881b0 0x557401f56850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:52.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.384+0000 7f2eccbd0740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557401b90390 0x557401f56d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:52.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.384+0000 7f2eccbd0740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x557401ecd680 con 0x557401b90390 2024-08-30T09:04:52.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.384+0000 7f2eccbd0740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x557401ebcc10 con 0x557401b881b0 2024-08-30T09:04:52.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.384+0000 7f2eccbd0740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x557401ec0bc0 con 0x557401bb1070 2024-08-30T09:04:52.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.385+0000 7f2ebd7fa640 1 -- 172.21.15.42:0/4213453670 <== mon.2 v2:172.21.15.167:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f2eb8004160 con 0x7f2ec01163e0 2024-08-30T09:04:52.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.385+0000 7f2ebffff640 1 --2- 172.21.15.42:0/4213453670 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f2ea80b4d30 0x7f2ea80b7240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:52.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.385+0000 7f2ec4bf8640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557401b90390 0x557401f56d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:52.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.385+0000 7f2ec4bf8640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557401b90390 0x557401f56d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:39612/0 (socket says 172.21.15.42:39612) 2024-08-30T09:04:52.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.385+0000 7f2ec4bf8640 1 -- 172.21.15.42:0/3673978227 learned_addr learned my addr 172.21.15.42:0/3673978227 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:04:52.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.385+0000 7f2ebf7fe640 1 --2- 172.21.15.42:0/3673978227 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557401bb1070 0x557401f56310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:52.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.385+0000 7f2ebf7fe640 1 --2- 172.21.15.42:0/3673978227 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557401b881b0 0x557401f56850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:52.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.385+0000 7f2ec4bf8640 1 -- 172.21.15.42:0/3673978227 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557401bb1070 msgr2=0x557401f56310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:04:52.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.385+0000 7f2ec4bf8640 1 --2- 172.21.15.42:0/3673978227 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557401bb1070 0x557401f56310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:52.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.385+0000 7f2ec4bf8640 1 -- 172.21.15.42:0/3673978227 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557401b881b0 msgr2=0x557401f56850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:04:52.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.385+0000 7f2ec4bf8640 1 --2- 172.21.15.42:0/3673978227 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557401b881b0 0x557401f56850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:52.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.385+0000 7f2ec4bf8640 1 -- 172.21.15.42:0/3673978227 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557401c1c650 con 0x557401b90390 2024-08-30T09:04:52.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.385+0000 7f2ebf7fe640 1 --2- 172.21.15.42:0/3673978227 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557401b881b0 0x557401f56850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-30T09:04:52.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.385+0000 7f2ec4bf8640 1 --2- 172.21.15.42:0/3673978227 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557401b90390 0x557401f56d90 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f2eb80c9560 tx=0x7f2eb8003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d37c02cf974f3a1 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:04:52.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.385+0000 7f2ebffff640 1 --2- 172.21.15.42:0/4213453670 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f2ea80b4d30 0x7f2ea80b7240 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f2ec0115e50 tx=0x7f2eb0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:04:52.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.386+0000 7f2ea67fc640 1 -- 172.21.15.42:0/3673978227 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2eb80ef0d0 con 0x557401b90390 2024-08-30T09:04:52.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.386+0000 7f2ea67fc640 1 -- 172.21.15.42:0/3673978227 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2eb80cc2f0 con 0x557401b90390 2024-08-30T09:04:52.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.386+0000 7f2ea67fc640 1 -- 172.21.15.42:0/3673978227 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2eb80d53e0 con 0x557401b90390 2024-08-30T09:04:52.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.387+0000 7f2eccbd0740 1 -- 172.21.15.42:0/3673978227 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557401b90390 msgr2=0x557401f56d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:04:52.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.387+0000 7f2eccbd0740 1 --2- 172.21.15.42:0/3673978227 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557401b90390 0x557401f56d90 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f2eb80c9560 tx=0x7f2eb8003f90 comp rx=0 tx=0).stop 2024-08-30T09:04:52.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.387+0000 7f2eccbd0740 1 -- 172.21.15.42:0/3673978227 shutdown_connections 2024-08-30T09:04:52.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.387+0000 7f2eccbd0740 1 --2- 172.21.15.42:0/3673978227 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557401b90390 0x557401f56d90 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:52.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.387+0000 7f2eccbd0740 1 --2- 172.21.15.42:0/3673978227 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557401b881b0 0x557401f56850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:52.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.387+0000 7f2eccbd0740 1 --2- 172.21.15.42:0/3673978227 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557401bb1070 0x557401f56310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:52.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.387+0000 7f2eccbd0740 1 -- 172.21.15.42:0/3673978227 >> 172.21.15.42:0/3673978227 conn(0x557401ba5460 msgr2=0x557401f54200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:04:52.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.387+0000 7f2eccbd0740 1 -- 172.21.15.42:0/3673978227 shutdown_connections 2024-08-30T09:04:52.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.387+0000 7f2eccbd0740 1 -- 172.21.15.42:0/3673978227 wait complete. 2024-08-30T09:04:52.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.387+0000 7f2eccbd0740 1 -- start start 2024-08-30T09:04:52.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.388+0000 7f2eccbd0740 1 Processor -- start 2024-08-30T09:04:52.394 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.388+0000 7f2eccbd0740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557401b881b0 0x557401f61ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:52.394 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.389+0000 7f2eccbd0740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557401b90390 0x557401f62010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:52.394 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.389+0000 7f2eccbd0740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557401bb1070 0x557401f6db60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:52.394 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.389+0000 7f2eccbd0740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x557401d1f700 con 0x557401bb1070 2024-08-30T09:04:52.394 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.389+0000 7f2eccbd0740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x557401e43750 con 0x557401b881b0 2024-08-30T09:04:52.394 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.389+0000 7f2eccbd0740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x557401f648b0 con 0x557401b90390 2024-08-30T09:04:52.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.389+0000 7f2eccbd0740 10 client.0 _ll_register_callbacks cb 0x557401e72e10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:04:52.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.390+0000 7f2ebffff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557401b881b0 0x557401f61ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:52.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.390+0000 7f2ebffff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557401b881b0 0x557401f61ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:53244/0 (socket says 172.21.15.42:53244) 2024-08-30T09:04:52.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.390+0000 7f2ebffff640 1 -- 172.21.15.42:0/3518036459 learned_addr learned my addr 172.21.15.42:0/3518036459 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:04:52.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.390+0000 7f2ec4bf8640 1 --2- 172.21.15.42:0/3518036459 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557401bb1070 0x557401f6db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:52.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.390+0000 7f2ec4bf8640 1 -- 172.21.15.42:0/3518036459 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557401b90390 msgr2=0x557401f62010 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-30T09:04:52.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.390+0000 7f2ec4bf8640 1 --2- 172.21.15.42:0/3518036459 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557401b90390 0x557401f62010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:52.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.390+0000 7f2ec4bf8640 1 -- 172.21.15.42:0/3518036459 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557401b881b0 msgr2=0x557401f61ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:04:52.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.390+0000 7f2ec4bf8640 1 --2- 172.21.15.42:0/3518036459 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557401b881b0 0x557401f61ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:52.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.390+0000 7f2ec4bf8640 1 -- 172.21.15.42:0/3518036459 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557401e7c900 con 0x557401bb1070 2024-08-30T09:04:52.396 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.391+0000 7f2ec4bf8640 1 --2- 172.21.15.42:0/3518036459 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557401bb1070 0x557401f6db60 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f2eb80c9560 tx=0x7f2eb80d5430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8eaba51f131ef5d9 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:04:52.396 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.391+0000 7f2e9f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-08-30T09:04:52.396 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.391+0000 7f2e9f7fe640 1 -- 172.21.15.42:0/3518036459 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2eb80d5610 con 0x557401bb1070 2024-08-30T09:04:52.396 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.391+0000 7f2eccbd0740 1 -- 172.21.15.42:0/3518036459 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2eb80cc2f0 con 0x557401bb1070 2024-08-30T09:04:52.396 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.392+0000 7f2e9f7fe640 1 -- 172.21.15.42:0/3518036459 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f2eb8030c20 con 0x557401bb1070 2024-08-30T09:04:52.396 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.392+0000 7f2e9f7fe640 1 -- 172.21.15.42:0/3518036459 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2eb80ef0d0 con 0x557401bb1070 2024-08-30T09:04:52.396 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.392+0000 7f2eccbd0740 10 client.16152 fetch_fsmap learned FSMap version 29 2024-08-30T09:04:52.396 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.392+0000 7f2eccbd0740 1 -- 172.21.15.42:0/3518036459 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x557401f706a0 con 0x557401bb1070 2024-08-30T09:04:52.397 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.393+0000 7f2e9f7fe640 1 -- 172.21.15.42:0/3518036459 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2eb80cc2f0 con 0x557401bb1070 2024-08-30T09:04:52.397 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.393+0000 7f2e9f7fe640 1 -- 172.21.15.42:0/3518036459 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f2eb80cafc0 con 0x557401bb1070 2024-08-30T09:04:52.397 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.393+0000 7f2eccbd0740 10 client.16152 fetch_fsmap finished waiting for FSMap version 29 2024-08-30T09:04:52.397 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.393+0000 7f2eccbd0740 10 client.16152 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-08-30T09:04:52.397 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.393+0000 7f2eccbd0740 20 client.16152 populate_metadata read hostname 'smithi042' 2024-08-30T09:04:52.397 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.393+0000 7f2eccbd0740 1 --2- 172.21.15.42:0/3518036459 >> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] conn(0x557401b6f9f0 0x557401f73a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:52.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.394+0000 7f2eccbd0740 4 client.16152 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-30T09:04:52.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.394+0000 7f2eccbd0740 1 -- 172.21.15.42:0/3518036459 --> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557401ecf5d0 con 0x557401b6f9f0 2024-08-30T09:04:52.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.394+0000 7f2ebffff640 1 --2- 172.21.15.42:0/3518036459 >> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] conn(0x557401b6f9f0 0x557401f73a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:52.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.394+0000 7f2ebffff640 1 --2- 172.21.15.42:0/3518036459 >> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] conn(0x557401b6f9f0 0x557401f73a00 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2b59b0ba6d5d352d server_cookie=2058623d13d2411c in_seq=0 out_seq=0 2024-08-30T09:04:52.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.394+0000 7f2e9f7fe640 10 client.16152 ms_handle_connect on v2:172.21.15.190:6838/719981110 2024-08-30T09:04:52.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.396+0000 7f2e9f7fe640 1 -- 172.21.15.42:0/3518036459 <== mds.1 v2:172.21.15.190:6838/719981110 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x557401ecf5d0 con 0x557401b6f9f0 2024-08-30T09:04:52.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.396+0000 7f2e9f7fe640 10 client.16152 handle_command_reply: tid=0 2024-08-30T09:04:52.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.396+0000 7f2eccbd0740 1 client.16152 shutdown 2024-08-30T09:04:52.400 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.397+0000 7f2eccbd0740 1 -- 172.21.15.42:0/3518036459 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557401bb1070 msgr2=0x557401f6db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:04:52.400 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.397+0000 7f2eccbd0740 1 --2- 172.21.15.42:0/3518036459 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557401bb1070 0x557401f6db60 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f2eb80c9560 tx=0x7f2eb80d5430 comp rx=0 tx=0).stop 2024-08-30T09:04:52.400 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.397+0000 7f2eccbd0740 1 -- 172.21.15.42:0/3518036459 shutdown_connections 2024-08-30T09:04:52.400 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.397+0000 7f2eccbd0740 1 --2- 172.21.15.42:0/3518036459 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557401b90390 0x557401f62010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:52.400 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.397+0000 7f2eccbd0740 1 --2- 172.21.15.42:0/3518036459 >> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] conn(0x557401b6f9f0 0x557401f73a00 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:52.400 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.397+0000 7f2eccbd0740 1 --2- 172.21.15.42:0/3518036459 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557401bb1070 0x557401f6db60 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:52.400 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.397+0000 7f2eccbd0740 1 --2- 172.21.15.42:0/3518036459 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557401b881b0 0x557401f61ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:52.400 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.397+0000 7f2eccbd0740 1 -- 172.21.15.42:0/3518036459 >> 172.21.15.42:0/3518036459 conn(0x557401ba5460 msgr2=0x557401f594f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:04:52.400 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.397+0000 7f2e9f7fe640 0 client.16152 ms_handle_reset on v2:172.21.15.190:6838/719981110 2024-08-30T09:04:52.400 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.397+0000 7f2eccbd0740 1 -- 172.21.15.42:0/3518036459 shutdown_connections 2024-08-30T09:04:52.400 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.397+0000 7f2eccbd0740 1 -- 172.21.15.42:0/3518036459 wait complete. 2024-08-30T09:04:52.400 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.397+0000 7f2eccbd0740 20 client.16152 trim_cache size 0 max 16384 2024-08-30T09:04:52.421 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.418+0000 7f2eccbd0740 1 Processor -- start 2024-08-30T09:04:52.421 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.418+0000 7f2eccbd0740 1 -- start start 2024-08-30T09:04:52.421 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.418+0000 7f2eccbd0740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557401b881b0 0x557401f61f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:52.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.418+0000 7f2eccbd0740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557401b90390 0x557401f593d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:52.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.418+0000 7f2eccbd0740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557401bb1070 0x557401f59910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:52.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.418+0000 7f2eccbd0740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f2ec01c1ec0 con 0x557401b90390 2024-08-30T09:04:52.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.418+0000 7f2eccbd0740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f2ec01bf790 con 0x557401bb1070 2024-08-30T09:04:52.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.418+0000 7f2eccbd0740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f2ec01bd5e0 con 0x557401b881b0 2024-08-30T09:04:52.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.418+0000 7f2ebf7fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557401b90390 0x557401f593d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:52.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.418+0000 7f2ebf7fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557401b90390 0x557401f593d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:39626/0 (socket says 172.21.15.42:39626) 2024-08-30T09:04:52.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.418+0000 7f2ebf7fe640 1 -- 172.21.15.42:0/1637432594 learned_addr learned my addr 172.21.15.42:0/1637432594 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:04:52.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.419+0000 7f2ec4bf8640 1 --2- 172.21.15.42:0/1637432594 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557401bb1070 0x557401f59910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:52.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.419+0000 7f2ebffff640 1 --2- 172.21.15.42:0/1637432594 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557401b881b0 0x557401f61f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:52.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.419+0000 7f2ec4bf8640 1 -- 172.21.15.42:0/1637432594 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557401b881b0 msgr2=0x557401f61f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:04:52.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.419+0000 7f2ec4bf8640 1 --2- 172.21.15.42:0/1637432594 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557401b881b0 0x557401f61f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:52.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.419+0000 7f2ec4bf8640 1 -- 172.21.15.42:0/1637432594 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557401b90390 msgr2=0x557401f593d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:04:52.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.419+0000 7f2ec4bf8640 1 --2- 172.21.15.42:0/1637432594 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557401b90390 0x557401f593d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:52.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.419+0000 7f2ec4bf8640 1 -- 172.21.15.42:0/1637432594 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557401f62d90 con 0x557401bb1070 2024-08-30T09:04:52.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.419+0000 7f2ebf7fe640 1 --2- 172.21.15.42:0/1637432594 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557401b90390 0x557401f593d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-30T09:04:52.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.419+0000 7f2ec4bf8640 1 --2- 172.21.15.42:0/1637432594 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557401bb1070 0x557401f59910 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f2eb8031040 tx=0x7f2eb8030e70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=86d32be308822e75 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:04:52.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.419+0000 7f2ea5ffb640 1 -- 172.21.15.42:0/1637432594 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2eb80d5610 con 0x557401bb1070 2024-08-30T09:04:52.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.419+0000 7f2ea5ffb640 1 -- 172.21.15.42:0/1637432594 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2eb80cc2f0 con 0x557401bb1070 2024-08-30T09:04:52.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.419+0000 7f2ea5ffb640 1 -- 172.21.15.42:0/1637432594 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2eb80cafc0 con 0x557401bb1070 2024-08-30T09:04:52.424 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.421+0000 7f2eccbd0740 1 -- 172.21.15.42:0/1637432594 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557401bb1070 msgr2=0x557401f59910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:04:52.424 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.421+0000 7f2eccbd0740 1 --2- 172.21.15.42:0/1637432594 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557401bb1070 0x557401f59910 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f2eb8031040 tx=0x7f2eb8030e70 comp rx=0 tx=0).stop 2024-08-30T09:04:52.425 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.422+0000 7f2eccbd0740 1 -- 172.21.15.42:0/1637432594 shutdown_connections 2024-08-30T09:04:52.425 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.422+0000 7f2eccbd0740 1 --2- 172.21.15.42:0/1637432594 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557401bb1070 0x557401f59910 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:52.425 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.422+0000 7f2eccbd0740 1 --2- 172.21.15.42:0/1637432594 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557401b90390 0x557401f593d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:52.425 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.422+0000 7f2eccbd0740 1 --2- 172.21.15.42:0/1637432594 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557401b881b0 0x557401f61f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:52.425 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.422+0000 7f2eccbd0740 1 -- 172.21.15.42:0/1637432594 >> 172.21.15.42:0/1637432594 conn(0x557401ba5460 msgr2=0x557401f61880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:04:52.425 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.422+0000 7f2eccbd0740 1 -- 172.21.15.42:0/1637432594 shutdown_connections 2024-08-30T09:04:52.425 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.422+0000 7f2eccbd0740 1 -- 172.21.15.42:0/1637432594 wait complete. 2024-08-30T09:04:52.425 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.422+0000 7f2eccbd0740 1 -- start start 2024-08-30T09:04:52.425 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.423+0000 7f2eccbd0740 1 Processor -- start 2024-08-30T09:04:52.430 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.423+0000 7f2eccbd0740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557401b881b0 0x557401f6a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:52.430 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.423+0000 7f2eccbd0740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557401b90390 0x557401f65850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:52.430 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.423+0000 7f2ebffff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557401b881b0 0x557401f6a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:52.430 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.423+0000 7f2eccbd0740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557401bb1070 0x557401f65e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:52.430 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.423+0000 7f2ebffff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557401b881b0 0x557401f6a3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:34648/0 (socket says 172.21.15.42:34648) 2024-08-30T09:04:52.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.423+0000 7f2ebffff640 1 -- 172.21.15.42:0/2686090559 learned_addr learned my addr 172.21.15.42:0/2686090559 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:04:52.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.423+0000 7f2eccbd0740 1 -- 172.21.15.42:0/2686090559 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f2ec01bdc50 con 0x557401b90390 2024-08-30T09:04:52.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.423+0000 7f2eccbd0740 1 -- 172.21.15.42:0/2686090559 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f2ec01bf0c0 con 0x557401bb1070 2024-08-30T09:04:52.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.423+0000 7f2eccbd0740 1 -- 172.21.15.42:0/2686090559 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f2ec01be320 con 0x557401b881b0 2024-08-30T09:04:52.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.423+0000 7f2eccbd0740 10 client.0 _ll_register_callbacks cb 0x7f2eb80d0070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:04:52.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.423+0000 7f2ec4bf8640 1 --2- 172.21.15.42:0/2686090559 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557401bb1070 0x557401f65e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:52.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.424+0000 7f2ebf7fe640 1 --2- 172.21.15.42:0/2686090559 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557401b90390 0x557401f65850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:52.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.424+0000 7f2ebffff640 1 -- 172.21.15.42:0/2686090559 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557401bb1070 msgr2=0x557401f65e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:04:52.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.424+0000 7f2ebffff640 1 --2- 172.21.15.42:0/2686090559 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557401bb1070 0x557401f65e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:52.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.424+0000 7f2ebffff640 1 -- 172.21.15.42:0/2686090559 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557401b90390 msgr2=0x557401f65850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:04:52.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.424+0000 7f2ebffff640 1 --2- 172.21.15.42:0/2686090559 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557401b90390 0x557401f65850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:52.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.424+0000 7f2ebffff640 1 -- 172.21.15.42:0/2686090559 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557401f66630 con 0x557401b881b0 2024-08-30T09:04:52.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.424+0000 7f2ebffff640 1 --2- 172.21.15.42:0/2686090559 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557401b881b0 0x557401f6a3a0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f2eb000f7b0 tx=0x7f2eb0013040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ac58cc92c25cd7d2 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:04:52.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.424+0000 7f2e9f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-08-30T09:04:52.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.424+0000 7f2e9f7fe640 1 -- 172.21.15.42:0/2686090559 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2eb002b070 con 0x557401b881b0 2024-08-30T09:04:52.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.424+0000 7f2eccbd0740 1 -- 172.21.15.42:0/2686090559 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2eb80cc2f0 con 0x557401b881b0 2024-08-30T09:04:52.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.424+0000 7f2e9f7fe640 1 -- 172.21.15.42:0/2686090559 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f2eb001d820 con 0x557401b881b0 2024-08-30T09:04:52.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.425+0000 7f2e9f7fe640 1 -- 172.21.15.42:0/2686090559 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2eb0037050 con 0x557401b881b0 2024-08-30T09:04:52.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.425+0000 7f2eccbd0740 10 client.25891 fetch_fsmap learned FSMap version 29 2024-08-30T09:04:52.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.425+0000 7f2eccbd0740 1 -- 172.21.15.42:0/2686090559 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x557401fde300 con 0x557401b881b0 2024-08-30T09:04:52.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.425+0000 7f2e9f7fe640 1 -- 172.21.15.42:0/2686090559 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2eb80cc2f0 con 0x557401b881b0 2024-08-30T09:04:52.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.425+0000 7f2e9f7fe640 1 -- 172.21.15.42:0/2686090559 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f2eb001cb30 con 0x557401b881b0 2024-08-30T09:04:52.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.425+0000 7f2eccbd0740 10 client.25891 fetch_fsmap finished waiting for FSMap version 29 2024-08-30T09:04:52.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.425+0000 7f2eccbd0740 10 client.25891 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-08-30T09:04:52.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.425+0000 7f2eccbd0740 20 client.25891 populate_metadata read hostname 'smithi042' 2024-08-30T09:04:52.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.425+0000 7f2eccbd0740 1 --2- 172.21.15.42:0/2686090559 >> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] conn(0x557401b44da0 0x557401fe0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:52.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.426+0000 7f2eccbd0740 4 client.25891 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-30T09:04:52.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.426+0000 7f2eccbd0740 1 -- 172.21.15.42:0/2686090559 --> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557401ecb070 con 0x557401b44da0 2024-08-30T09:04:52.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.426+0000 7f2ebf7fe640 1 --2- 172.21.15.42:0/2686090559 >> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] conn(0x557401b44da0 0x557401fe0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:52.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.426+0000 7f2e9f7fe640 10 client.25891 ms_handle_connect on v2:172.21.15.190:6838/719981110 2024-08-30T09:04:52.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.426+0000 7f2ebf7fe640 1 --2- 172.21.15.42:0/2686090559 >> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] conn(0x557401b44da0 0x557401fe0ad0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=35a3fcc36bbfd555 server_cookie=3235b9e081a65332 in_seq=0 out_seq=0 2024-08-30T09:04:52.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.426+0000 7f2e9f7fe640 1 -- 172.21.15.42:0/2686090559 <== mds.1 v2:172.21.15.190:6838/719981110 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557401ecb070 con 0x557401b44da0 2024-08-30T09:04:52.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.426+0000 7f2e9f7fe640 10 client.25891 handle_command_reply: tid=0 2024-08-30T09:04:52.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.426+0000 7f2eccbd0740 1 client.25891 shutdown 2024-08-30T09:04:52.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.427+0000 7f2eccbd0740 1 -- 172.21.15.42:0/2686090559 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557401b881b0 msgr2=0x557401f6a3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:04:52.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.427+0000 7f2eccbd0740 1 --2- 172.21.15.42:0/2686090559 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557401b881b0 0x557401f6a3a0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f2eb000f7b0 tx=0x7f2eb0013040 comp rx=0 tx=0).stop 2024-08-30T09:04:52.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.427+0000 7f2eccbd0740 1 -- 172.21.15.42:0/2686090559 shutdown_connections 2024-08-30T09:04:52.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.427+0000 7f2eccbd0740 1 --2- 172.21.15.42:0/2686090559 >> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] conn(0x557401b44da0 0x557401fe0ad0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:52.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.427+0000 7f2e9f7fe640 0 client.25891 ms_handle_reset on v2:172.21.15.190:6838/719981110 2024-08-30T09:04:52.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.427+0000 7f2eccbd0740 1 --2- 172.21.15.42:0/2686090559 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557401bb1070 0x557401f65e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:52.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.427+0000 7f2eccbd0740 1 --2- 172.21.15.42:0/2686090559 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557401b90390 0x557401f65850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:52.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.427+0000 7f2eccbd0740 1 --2- 172.21.15.42:0/2686090559 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557401b881b0 0x557401f6a3a0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:52.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.427+0000 7f2eccbd0740 1 -- 172.21.15.42:0/2686090559 >> 172.21.15.42:0/2686090559 conn(0x557401ba5460 msgr2=0x557401f59280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:04:52.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.427+0000 7f2eccbd0740 1 -- 172.21.15.42:0/2686090559 shutdown_connections 2024-08-30T09:04:52.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.427+0000 7f2eccbd0740 1 -- 172.21.15.42:0/2686090559 wait complete. 2024-08-30T09:04:52.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.427+0000 7f2eccbd0740 20 client.25891 trim_cache size 0 max 16384 2024-08-30T09:04:52.434 INFO:teuthology.orchestra.run.smithi042.stdout:[] 2024-08-30T09:04:52.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.428+0000 7f2ec665b640 1 -- 172.21.15.42:0/4213453670 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f2ea80b4d30 msgr2=0x7f2ea80b7240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:04:52.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.428+0000 7f2ec665b640 1 --2- 172.21.15.42:0/4213453670 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f2ea80b4d30 0x7f2ea80b7240 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f2ec0115e50 tx=0x7f2eb0009260 comp rx=0 tx=0).stop 2024-08-30T09:04:52.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.429+0000 7f2ec665b640 1 -- 172.21.15.42:0/4213453670 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2ec01163e0 msgr2=0x7f2ec01b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:04:52.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.429+0000 7f2ec665b640 1 --2- 172.21.15.42:0/4213453670 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2ec01163e0 0x7f2ec01b4dc0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f2eb8004af0 tx=0x7f2eb8003ba0 comp rx=0 tx=0).stop 2024-08-30T09:04:52.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.429+0000 7f2ec665b640 1 -- 172.21.15.42:0/4213453670 shutdown_connections 2024-08-30T09:04:52.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.429+0000 7f2ec665b640 1 --2- 172.21.15.42:0/4213453670 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f2ea80b4d30 0x7f2ea80b7240 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:52.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.429+0000 7f2ec665b640 1 --2- 172.21.15.42:0/4213453670 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2ec01163e0 0x7f2ec01b4dc0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:52.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.429+0000 7f2ec665b640 1 --2- 172.21.15.42:0/4213453670 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2ec010ac80 0x7f2ec01aa280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:52.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.429+0000 7f2ec665b640 1 --2- 172.21.15.42:0/4213453670 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2ec010a330 0x7f2ec01a9d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:52.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.429+0000 7f2ec665b640 1 -- 172.21.15.42:0/4213453670 >> 172.21.15.42:0/4213453670 conn(0x7f2ec0070c60 msgr2=0x7f2ec010cb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:04:52.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.429+0000 7f2ec665b640 1 -- 172.21.15.42:0/4213453670 shutdown_connections 2024-08-30T09:04:52.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:52.429+0000 7f2ec665b640 1 -- 172.21.15.42:0/4213453670 wait complete. 2024-08-30T09:04:52.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:04:52 smithi042 ceph-mon[18652]: pgmap v496: 129 pgs: 129 active+clean; 1.1 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 8.6 MiB/s wr, 745 op/s 2024-08-30T09:04:52.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:04:52 smithi190 ceph-mon[22001]: pgmap v496: 129 pgs: 129 active+clean; 1.1 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 8.6 MiB/s wr, 745 op/s 2024-08-30T09:04:53.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:04:52 smithi167 ceph-mon[22063]: pgmap v496: 129 pgs: 129 active+clean; 1.1 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 8.6 MiB/s wr, 745 op/s 2024-08-30T09:04:53.468 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-08-30T09:04:53.886 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T09:04:54.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.511+0000 7fcba277c640 1 -- 172.21.15.42:0/4047362465 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcb9c10eef0 msgr2=0x7fcb9c10f300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:04:54.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.511+0000 7fcba277c640 1 --2- 172.21.15.42:0/4047362465 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcb9c10eef0 0x7fcb9c10f300 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fcb9000aa10 tx=0x7fcb9002b720 comp rx=0 tx=0).stop 2024-08-30T09:04:54.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.511+0000 7fcba277c640 1 -- 172.21.15.42:0/4047362465 shutdown_connections 2024-08-30T09:04:54.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.511+0000 7fcba277c640 1 --2- 172.21.15.42:0/4047362465 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcb9c10a160 0x7fcb9c10a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:54.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.511+0000 7fcba277c640 1 --2- 172.21.15.42:0/4047362465 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcb9c10aab0 0x7fcb9c116d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:54.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.511+0000 7fcba277c640 1 --2- 172.21.15.42:0/4047362465 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcb9c10eef0 0x7fcb9c10f300 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:54.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.511+0000 7fcba277c640 1 -- 172.21.15.42:0/4047362465 >> 172.21.15.42:0/4047362465 conn(0x7fcb9c071090 msgr2=0x7fcb9c073bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:04:54.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.511+0000 7fcba277c640 1 -- 172.21.15.42:0/4047362465 shutdown_connections 2024-08-30T09:04:54.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.511+0000 7fcba277c640 1 -- 172.21.15.42:0/4047362465 wait complete. 2024-08-30T09:04:54.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.512+0000 7fcba277c640 1 Processor -- start 2024-08-30T09:04:54.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.513+0000 7fcba277c640 1 -- start start 2024-08-30T09:04:54.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.513+0000 7fcba277c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcb9c10a160 0x7fcb9c1a9d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:54.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.513+0000 7fcba277c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcb9c10aab0 0x7fcb9c1aa260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:54.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.513+0000 7fcba277c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcb9c10eef0 0x7fcb9c1b4da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:54.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.513+0000 7fcba277c640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fcb9c11c080 con 0x7fcb9c10a160 2024-08-30T09:04:54.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.513+0000 7fcba277c640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fcb9c11bf00 con 0x7fcb9c10eef0 2024-08-30T09:04:54.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.514+0000 7fcb9b7fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcb9c10aab0 0x7fcb9c1aa260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:54.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.514+0000 7fcb9b7fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcb9c10aab0 0x7fcb9c1aa260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:34668/0 (socket says 172.21.15.42:34668) 2024-08-30T09:04:54.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.514+0000 7fcb9b7fe640 1 -- 172.21.15.42:0/299773529 learned_addr learned my addr 172.21.15.42:0/299773529 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:04:54.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.514+0000 7fcba0d19640 1 --2- 172.21.15.42:0/299773529 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcb9c10eef0 0x7fcb9c1b4da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:54.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.514+0000 7fcb9bfff640 1 --2- 172.21.15.42:0/299773529 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcb9c10a160 0x7fcb9c1a9d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:54.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.514+0000 7fcba277c640 1 -- 172.21.15.42:0/299773529 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fcb9c11c200 con 0x7fcb9c10aab0 2024-08-30T09:04:54.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.514+0000 7fcba0d19640 1 -- 172.21.15.42:0/299773529 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcb9c10aab0 msgr2=0x7fcb9c1aa260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:04:54.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.514+0000 7fcba0d19640 1 --2- 172.21.15.42:0/299773529 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcb9c10aab0 0x7fcb9c1aa260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:54.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.514+0000 7fcba0d19640 1 -- 172.21.15.42:0/299773529 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcb9c10a160 msgr2=0x7fcb9c1a9d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:04:54.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.514+0000 7fcba0d19640 1 --2- 172.21.15.42:0/299773529 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcb9c10a160 0x7fcb9c1a9d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:54.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.514+0000 7fcba0d19640 1 -- 172.21.15.42:0/299773529 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb9c1aab50 con 0x7fcb9c10eef0 2024-08-30T09:04:54.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.514+0000 7fcba0d19640 1 --2- 172.21.15.42:0/299773529 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcb9c10eef0 0x7fcb9c1b4da0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fcb94009fd0 tx=0x7fcb9400ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:04:54.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.515+0000 7fcb997fa640 1 -- 172.21.15.42:0/299773529 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb9400ce20 con 0x7fcb9c10eef0 2024-08-30T09:04:54.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.515+0000 7fcb997fa640 1 -- 172.21.15.42:0/299773529 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb94005020 con 0x7fcb9c10eef0 2024-08-30T09:04:54.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.515+0000 7fcba277c640 1 -- 172.21.15.42:0/299773529 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb9c1b5400 con 0x7fcb9c10eef0 2024-08-30T09:04:54.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.515+0000 7fcb997fa640 1 -- 172.21.15.42:0/299773529 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb94010690 con 0x7fcb9c10eef0 2024-08-30T09:04:54.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.515+0000 7fcba277c640 1 -- 172.21.15.42:0/299773529 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb9c1b58e0 con 0x7fcb9c10eef0 2024-08-30T09:04:54.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.519+0000 7fcba8cf1740 1 Processor -- start 2024-08-30T09:04:54.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.519+0000 7fcb997fa640 1 -- 172.21.15.42:0/299773529 <== mon.2 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7fcb94010830 con 0x7fcb9c10eef0 2024-08-30T09:04:54.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.519+0000 7fcba8cf1740 1 -- start start 2024-08-30T09:04:54.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.520+0000 7fcba8cf1740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561e44e78070 0x561e4521d330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:54.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.520+0000 7fcba8cf1740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561e44e4f1b0 0x561e4521d870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:54.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.520+0000 7fcba8cf1740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561e44e57390 0x561e4521ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:54.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.520+0000 7fcba8cf1740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x561e45194680 con 0x561e44e57390 2024-08-30T09:04:54.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.520+0000 7fcba8cf1740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x561e45183c10 con 0x561e44e4f1b0 2024-08-30T09:04:54.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.520+0000 7fcba8cf1740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x561e45187bc0 con 0x561e44e78070 2024-08-30T09:04:54.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.520+0000 7fcb9b7fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561e44e78070 0x561e4521d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:54.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.520+0000 7fcb9b7fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561e44e78070 0x561e4521d330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:34680/0 (socket says 172.21.15.42:34680) 2024-08-30T09:04:54.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.520+0000 7fcb9b7fe640 1 -- 172.21.15.42:0/79311896 learned_addr learned my addr 172.21.15.42:0/79311896 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:04:54.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.520+0000 7fcb9b7fe640 1 --2- 172.21.15.42:0/79311896 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561e44e57390 0x561e4521ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:54.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.520+0000 7fcb9bfff640 1 --2- 172.21.15.42:0/79311896 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561e44e4f1b0 0x561e4521d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:54.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.520+0000 7fcb997fa640 1 --2- 172.21.15.42:0/299773529 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fcb840b4bb0 0x7fcb840b70c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:54.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.520+0000 7fcb9b7fe640 1 -- 172.21.15.42:0/79311896 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561e44e4f1b0 msgr2=0x561e4521d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:04:54.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.520+0000 7fcb9b7fe640 1 --2- 172.21.15.42:0/79311896 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561e44e4f1b0 0x561e4521d870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:54.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.520+0000 7fcb997fa640 1 -- 172.21.15.42:0/299773529 <== mon.2 v2:172.21.15.167:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7fcb940e5b30 con 0x7fcb9c10eef0 2024-08-30T09:04:54.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.520+0000 7fcb9b7fe640 1 -- 172.21.15.42:0/79311896 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561e44e57390 msgr2=0x561e4521ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:04:54.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.520+0000 7fcb9b7fe640 1 --2- 172.21.15.42:0/79311896 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561e44e57390 0x561e4521ddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:54.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.520+0000 7fcb9b7fe640 1 -- 172.21.15.42:0/79311896 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e44ee3650 con 0x561e44e78070 2024-08-30T09:04:54.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.521+0000 7fcb9bfff640 1 --2- 172.21.15.42:0/79311896 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561e44e4f1b0 0x561e4521d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-30T09:04:54.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.521+0000 7fcba0d19640 1 --2- 172.21.15.42:0/299773529 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fcb840b4bb0 0x7fcb840b70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:54.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.521+0000 7fcb9b7fe640 1 --2- 172.21.15.42:0/79311896 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561e44e78070 0x561e4521d330 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fcb8c005bc0 tx=0x7fcb8c0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a30d4d20f052a155 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:04:54.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.521+0000 7fcb827fc640 1 -- 172.21.15.42:0/79311896 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb8c012d80 con 0x561e44e78070 2024-08-30T09:04:54.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.521+0000 7fcb827fc640 1 -- 172.21.15.42:0/79311896 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb8c010770 con 0x561e44e78070 2024-08-30T09:04:54.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.521+0000 7fcb827fc640 1 -- 172.21.15.42:0/79311896 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb8c003830 con 0x561e44e78070 2024-08-30T09:04:54.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.522+0000 7fcba8cf1740 1 -- 172.21.15.42:0/79311896 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561e44e78070 msgr2=0x561e4521d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:04:54.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.522+0000 7fcba0d19640 1 --2- 172.21.15.42:0/299773529 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fcb840b4bb0 0x7fcb840b70c0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fcb940abed0 tx=0x7fcb94005f70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:04:54.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.522+0000 7fcba8cf1740 1 --2- 172.21.15.42:0/79311896 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561e44e78070 0x561e4521d330 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fcb8c005bc0 tx=0x7fcb8c0046c0 comp rx=0 tx=0).stop 2024-08-30T09:04:54.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.522+0000 7fcba8cf1740 1 -- 172.21.15.42:0/79311896 shutdown_connections 2024-08-30T09:04:54.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.522+0000 7fcba8cf1740 1 --2- 172.21.15.42:0/79311896 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561e44e57390 0x561e4521ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:54.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.522+0000 7fcba8cf1740 1 --2- 172.21.15.42:0/79311896 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561e44e4f1b0 0x561e4521d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:54.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.522+0000 7fcba8cf1740 1 --2- 172.21.15.42:0/79311896 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561e44e78070 0x561e4521d330 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:54.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.522+0000 7fcba8cf1740 1 -- 172.21.15.42:0/79311896 >> 172.21.15.42:0/79311896 conn(0x561e44e6c460 msgr2=0x561e4521b220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:04:54.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.522+0000 7fcba8cf1740 1 -- 172.21.15.42:0/79311896 shutdown_connections 2024-08-30T09:04:54.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.522+0000 7fcba8cf1740 1 -- 172.21.15.42:0/79311896 wait complete. 2024-08-30T09:04:54.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.523+0000 7fcba8cf1740 1 -- start start 2024-08-30T09:04:54.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.524+0000 7fcba8cf1740 1 Processor -- start 2024-08-30T09:04:54.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.524+0000 7fcba8cf1740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561e44e4f1b0 0x561e45228af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:54.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.524+0000 7fcba8cf1740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561e44e57390 0x561e45229030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:54.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.525+0000 7fcba8cf1740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561e44e78070 0x561e45234e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:54.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.525+0000 7fcba8cf1740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x561e44fe6700 con 0x561e44e78070 2024-08-30T09:04:54.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.525+0000 7fcba8cf1740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x561e4510a750 con 0x561e44e4f1b0 2024-08-30T09:04:54.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.525+0000 7fcba8cf1740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x561e4522bb10 con 0x561e44e57390 2024-08-30T09:04:54.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.525+0000 7fcb9bfff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561e44e57390 0x561e45229030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:54.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.525+0000 7fcb9bfff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561e44e57390 0x561e45229030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:34694/0 (socket says 172.21.15.42:34694) 2024-08-30T09:04:54.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.525+0000 7fcb9bfff640 1 -- 172.21.15.42:0/1999170135 learned_addr learned my addr 172.21.15.42:0/1999170135 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:04:54.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.525+0000 7fcba8cf1740 10 client.0 _ll_register_callbacks cb 0x561e45139e10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:04:54.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.525+0000 7fcb9b7fe640 1 --2- 172.21.15.42:0/1999170135 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561e44e4f1b0 0x561e45228af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:54.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.525+0000 7fcb9bfff640 1 -- 172.21.15.42:0/1999170135 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561e44e4f1b0 msgr2=0x561e45228af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:04:54.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.525+0000 7fcb9bfff640 1 --2- 172.21.15.42:0/1999170135 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561e44e4f1b0 0x561e45228af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:54.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.525+0000 7fcb9bfff640 1 -- 172.21.15.42:0/1999170135 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561e44e78070 msgr2=0x561e45234e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:04:54.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.525+0000 7fcb9bfff640 1 --2- 172.21.15.42:0/1999170135 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561e44e78070 0x561e45234e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:54.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.525+0000 7fcb9bfff640 1 -- 172.21.15.42:0/1999170135 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e45143900 con 0x561e44e57390 2024-08-30T09:04:54.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.526+0000 7fcb9bfff640 1 --2- 172.21.15.42:0/1999170135 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561e44e57390 0x561e45229030 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fcb9000aa10 tx=0x7fcb90030870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c4212f34acc2dc58 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:04:54.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.526+0000 7fcb7b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-08-30T09:04:54.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.526+0000 7fcb7b7fe640 1 -- 172.21.15.42:0/1999170135 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb90036040 con 0x561e44e57390 2024-08-30T09:04:54.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.526+0000 7fcba8cf1740 1 -- 172.21.15.42:0/1999170135 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcb8c010770 con 0x561e44e57390 2024-08-30T09:04:54.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.526+0000 7fcb7b7fe640 1 -- 172.21.15.42:0/1999170135 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7fcb90004450 con 0x561e44e57390 2024-08-30T09:04:54.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.526+0000 7fcb7b7fe640 1 -- 172.21.15.42:0/1999170135 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fcb9004e050 con 0x561e44e57390 2024-08-30T09:04:54.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.526+0000 7fcba8cf1740 10 client.25903 fetch_fsmap learned FSMap version 29 2024-08-30T09:04:54.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.526+0000 7fcba8cf1740 1 -- 172.21.15.42:0/1999170135 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x561e45237950 con 0x561e44e57390 2024-08-30T09:04:54.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.527+0000 7fcb7b7fe640 1 -- 172.21.15.42:0/1999170135 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb900054e0 con 0x561e44e57390 2024-08-30T09:04:54.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.527+0000 7fcb7b7fe640 1 -- 172.21.15.42:0/1999170135 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fcb9003b040 con 0x561e44e57390 2024-08-30T09:04:54.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.527+0000 7fcba8cf1740 10 client.25903 fetch_fsmap finished waiting for FSMap version 29 2024-08-30T09:04:54.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.528+0000 7fcba8cf1740 10 client.25903 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-30T09:04:54.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.528+0000 7fcba8cf1740 20 client.25903 populate_metadata read hostname 'smithi042' 2024-08-30T09:04:54.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.528+0000 7fcba8cf1740 1 --2- 172.21.15.42:0/1999170135 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x561e44e369f0 0x561e4523b260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:54.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.528+0000 7fcba8cf1740 4 client.25903 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-30T09:04:54.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.528+0000 7fcba8cf1740 1 -- 172.21.15.42:0/1999170135 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fcb8c010770 con 0x561e44e369f0 2024-08-30T09:04:54.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.528+0000 7fcb9b7fe640 1 --2- 172.21.15.42:0/1999170135 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x561e44e369f0 0x561e4523b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:54.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.529+0000 7fcb7b7fe640 10 client.25903 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:04:54.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.529+0000 7fcb9b7fe640 1 --2- 172.21.15.42:0/1999170135 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x561e44e369f0 0x561e4523b260 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a6a3fdb0163b1cff server_cookie=46ae67a42d743081 in_seq=0 out_seq=0 2024-08-30T09:04:54.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.531+0000 7fcb7b7fe640 1 -- 172.21.15.42:0/1999170135 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7fcb8c010770 con 0x561e44e369f0 2024-08-30T09:04:54.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.531+0000 7fcb7b7fe640 10 client.25903 handle_command_reply: tid=0 2024-08-30T09:04:54.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.531+0000 7fcba8cf1740 1 client.25903 shutdown 2024-08-30T09:04:54.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.531+0000 7fcba8cf1740 1 -- 172.21.15.42:0/1999170135 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561e44e57390 msgr2=0x561e45229030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:04:54.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.531+0000 7fcba8cf1740 1 --2- 172.21.15.42:0/1999170135 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561e44e57390 0x561e45229030 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fcb9000aa10 tx=0x7fcb90030870 comp rx=0 tx=0).stop 2024-08-30T09:04:54.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.532+0000 7fcba8cf1740 1 -- 172.21.15.42:0/1999170135 shutdown_connections 2024-08-30T09:04:54.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.532+0000 7fcba8cf1740 1 --2- 172.21.15.42:0/1999170135 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x561e44e369f0 0x561e4523b260 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:54.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.532+0000 7fcba8cf1740 1 --2- 172.21.15.42:0/1999170135 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561e44e57390 0x561e45229030 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:54.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.532+0000 7fcba8cf1740 1 --2- 172.21.15.42:0/1999170135 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561e44e78070 0x561e45234e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:54.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.532+0000 7fcba8cf1740 1 --2- 172.21.15.42:0/1999170135 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561e44e4f1b0 0x561e45228af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:54.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.532+0000 7fcba8cf1740 1 -- 172.21.15.42:0/1999170135 >> 172.21.15.42:0/1999170135 conn(0x561e44e6c460 msgr2=0x561e45220510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:04:54.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.532+0000 7fcb7b7fe640 0 client.25903 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:04:54.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.532+0000 7fcba8cf1740 1 -- 172.21.15.42:0/1999170135 shutdown_connections 2024-08-30T09:04:54.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.532+0000 7fcba8cf1740 1 -- 172.21.15.42:0/1999170135 wait complete. 2024-08-30T09:04:54.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.532+0000 7fcba8cf1740 20 client.25903 trim_cache size 0 max 16384 2024-08-30T09:04:54.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.550+0000 7fcba8cf1740 1 Processor -- start 2024-08-30T09:04:54.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.550+0000 7fcba8cf1740 1 -- start start 2024-08-30T09:04:54.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.551+0000 7fcba8cf1740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561e44e4f1b0 0x561e4521b320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:54.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.551+0000 7fcba8cf1740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561e44e57390 0x561e45234d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:54.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.551+0000 7fcba8cf1740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561e44e78070 0x561e45235240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:54.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.551+0000 7fcba8cf1740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fcb9c1c1db0 con 0x561e44e57390 2024-08-30T09:04:54.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.551+0000 7fcba8cf1740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fcb9c1bf680 con 0x561e44e4f1b0 2024-08-30T09:04:54.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.551+0000 7fcba8cf1740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fcb9c1bd580 con 0x561e44e78070 2024-08-30T09:04:54.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.551+0000 7fcb9b7fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561e44e4f1b0 0x561e4521b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:54.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.551+0000 7fcb9b7fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561e44e4f1b0 0x561e4521b320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:53292/0 (socket says 172.21.15.42:53292) 2024-08-30T09:04:54.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.551+0000 7fcb9b7fe640 1 -- 172.21.15.42:0/1530183266 learned_addr learned my addr 172.21.15.42:0/1530183266 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:04:54.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.551+0000 7fcb9b7fe640 1 --2- 172.21.15.42:0/1530183266 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561e44e78070 0x561e45235240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:54.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.551+0000 7fcb9bfff640 1 --2- 172.21.15.42:0/1530183266 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561e44e57390 0x561e45234d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:54.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.551+0000 7fcb9b7fe640 1 -- 172.21.15.42:0/1530183266 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561e44e78070 msgr2=0x561e45235240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:04:54.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.551+0000 7fcb9b7fe640 1 --2- 172.21.15.42:0/1530183266 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561e44e78070 0x561e45235240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:54.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.551+0000 7fcb9b7fe640 1 -- 172.21.15.42:0/1530183266 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561e44e57390 msgr2=0x561e45234d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:04:54.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.551+0000 7fcb9b7fe640 1 --2- 172.21.15.42:0/1530183266 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561e44e57390 0x561e45234d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:54.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.551+0000 7fcb9b7fe640 1 -- 172.21.15.42:0/1530183266 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e45229db0 con 0x561e44e4f1b0 2024-08-30T09:04:54.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.551+0000 7fcb9b7fe640 1 --2- 172.21.15.42:0/1530183266 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561e44e4f1b0 0x561e4521b320 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fcb8c00bcd0 tx=0x7fcb8c00bfd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a6625fd06f7a1ba1 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:04:54.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.552+0000 7fcb81ffb640 1 -- 172.21.15.42:0/1530183266 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb8c010b00 con 0x561e44e4f1b0 2024-08-30T09:04:54.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.552+0000 7fcb81ffb640 1 -- 172.21.15.42:0/1530183266 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb8c016040 con 0x561e44e4f1b0 2024-08-30T09:04:54.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.552+0000 7fcb81ffb640 1 -- 172.21.15.42:0/1530183266 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb8c005630 con 0x561e44e4f1b0 2024-08-30T09:04:54.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.553+0000 7fcba8cf1740 1 -- 172.21.15.42:0/1530183266 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561e44e4f1b0 msgr2=0x561e4521b320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:04:54.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.553+0000 7fcba8cf1740 1 --2- 172.21.15.42:0/1530183266 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561e44e4f1b0 0x561e4521b320 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fcb8c00bcd0 tx=0x7fcb8c00bfd0 comp rx=0 tx=0).stop 2024-08-30T09:04:54.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.556+0000 7fcba8cf1740 1 -- 172.21.15.42:0/1530183266 shutdown_connections 2024-08-30T09:04:54.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.556+0000 7fcba8cf1740 1 --2- 172.21.15.42:0/1530183266 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561e44e78070 0x561e45235240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:54.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.556+0000 7fcba8cf1740 1 --2- 172.21.15.42:0/1530183266 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561e44e57390 0x561e45234d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:54.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.556+0000 7fcba8cf1740 1 --2- 172.21.15.42:0/1530183266 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561e44e4f1b0 0x561e4521b320 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:54.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.556+0000 7fcba8cf1740 1 -- 172.21.15.42:0/1530183266 >> 172.21.15.42:0/1530183266 conn(0x561e44e6c460 msgr2=0x561e4521ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:04:54.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.560+0000 7fcba8cf1740 1 -- 172.21.15.42:0/1530183266 shutdown_connections 2024-08-30T09:04:54.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.560+0000 7fcba8cf1740 1 -- 172.21.15.42:0/1530183266 wait complete. 2024-08-30T09:04:54.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.560+0000 7fcba8cf1740 1 -- start start 2024-08-30T09:04:54.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.560+0000 7fcba8cf1740 1 Processor -- start 2024-08-30T09:04:54.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.561+0000 7fcba8cf1740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561e44e4f1b0 0x561e4522cf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:54.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.561+0000 7fcba8cf1740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561e44e57390 0x561e4522d490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:54.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.561+0000 7fcba8cf1740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561e44e78070 0x561e4522d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:54.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.561+0000 7fcba8cf1740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fcb9c1bdb40 con 0x561e44e78070 2024-08-30T09:04:54.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.561+0000 7fcba8cf1740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fcb9c1befb0 con 0x561e44e57390 2024-08-30T09:04:54.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.561+0000 7fcba8cf1740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fcb9c1be210 con 0x561e44e4f1b0 2024-08-30T09:04:54.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.561+0000 7fcba8cf1740 10 client.0 _ll_register_callbacks cb 0x7fcb9000a750 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:04:54.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.561+0000 7fcb9bfff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561e44e57390 0x561e4522d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:54.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.561+0000 7fcb9bfff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561e44e57390 0x561e4522d490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:53296/0 (socket says 172.21.15.42:53296) 2024-08-30T09:04:54.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.561+0000 7fcb9bfff640 1 -- 172.21.15.42:0/932310664 learned_addr learned my addr 172.21.15.42:0/932310664 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:04:54.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.564+0000 7fcb9b7fe640 1 --2- 172.21.15.42:0/932310664 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561e44e4f1b0 0x561e4522cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:54.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.566+0000 7fcb9bfff640 1 -- 172.21.15.42:0/932310664 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561e44e4f1b0 msgr2=0x561e4522cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:04:54.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.566+0000 7fcb9bfff640 1 --2- 172.21.15.42:0/932310664 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561e44e4f1b0 0x561e4522cf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:54.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.566+0000 7fcb9bfff640 1 -- 172.21.15.42:0/932310664 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561e44e78070 msgr2=0x561e4522d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:04:54.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.566+0000 7fcb9bfff640 1 --2- 172.21.15.42:0/932310664 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561e44e78070 0x561e4522d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:54.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.566+0000 7fcb9bfff640 1 -- 172.21.15.42:0/932310664 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e452a4e70 con 0x561e44e57390 2024-08-30T09:04:54.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.566+0000 7fcb9bfff640 1 --2- 172.21.15.42:0/932310664 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561e44e57390 0x561e4522d490 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fcb90000c00 tx=0x7fcb90004480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c6e9f09b35c23a6 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:04:54.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.566+0000 7fcb7b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-30T09:04:54.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.570+0000 7fcb7b7fe640 1 -- 172.21.15.42:0/932310664 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb90036040 con 0x561e44e57390 2024-08-30T09:04:54.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.570+0000 7fcb7b7fe640 1 -- 172.21.15.42:0/932310664 <== mon.2 v2:172.21.15.167:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7fcb90003b60 con 0x561e44e57390 2024-08-30T09:04:54.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.572+0000 7fcba8cf1740 1 -- 172.21.15.42:0/932310664 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcb8c016040 con 0x561e44e57390 2024-08-30T09:04:54.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.575+0000 7fcb7b7fe640 1 -- 172.21.15.42:0/932310664 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fcb90005850 con 0x561e44e57390 2024-08-30T09:04:54.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.588+0000 7fcba8cf1740 10 client.25886 fetch_fsmap learned FSMap version 29 2024-08-30T09:04:54.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.588+0000 7fcba8cf1740 1 -- 172.21.15.42:0/932310664 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x561e452c8100 con 0x561e44e57390 2024-08-30T09:04:54.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.589+0000 7fcb7b7fe640 1 -- 172.21.15.42:0/932310664 <== mon.2 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb90057070 con 0x561e44e57390 2024-08-30T09:04:54.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.589+0000 7fcb7b7fe640 1 -- 172.21.15.42:0/932310664 <== mon.2 v2:172.21.15.167:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fcb9000b040 con 0x561e44e57390 2024-08-30T09:04:54.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.590+0000 7fcba8cf1740 10 client.25886 fetch_fsmap finished waiting for FSMap version 29 2024-08-30T09:04:54.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.590+0000 7fcba8cf1740 10 client.25886 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-30T09:04:54.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.591+0000 7fcba8cf1740 20 client.25886 populate_metadata read hostname 'smithi042' 2024-08-30T09:04:54.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.591+0000 7fcba8cf1740 1 --2- 172.21.15.42:0/932310664 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x561e44e0bda0 0x561e452ca8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:54.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.591+0000 7fcba8cf1740 4 client.25886 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-30T09:04:54.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.591+0000 7fcba8cf1740 1 -- 172.21.15.42:0/932310664 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fcb8c016040 con 0x561e44e0bda0 2024-08-30T09:04:54.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.593+0000 7fcb9b7fe640 1 --2- 172.21.15.42:0/932310664 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x561e44e0bda0 0x561e452ca8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:54.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.600+0000 7fcb9b7fe640 1 --2- 172.21.15.42:0/932310664 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x561e44e0bda0 0x561e452ca8f0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=90660a4f74c80924 server_cookie=a48b9e3802551a29 in_seq=0 out_seq=0 2024-08-30T09:04:54.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.602+0000 7fcb7b7fe640 10 client.25886 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:04:54.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.602+0000 7fcb7b7fe640 1 -- 172.21.15.42:0/932310664 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fcb8c016040 con 0x561e44e0bda0 2024-08-30T09:04:54.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.602+0000 7fcb7b7fe640 10 client.25886 handle_command_reply: tid=0 2024-08-30T09:04:54.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.603+0000 7fcba8cf1740 1 client.25886 shutdown 2024-08-30T09:04:54.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.606+0000 7fcba8cf1740 1 -- 172.21.15.42:0/932310664 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561e44e57390 msgr2=0x561e4522d490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:04:54.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.606+0000 7fcba8cf1740 1 --2- 172.21.15.42:0/932310664 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561e44e57390 0x561e4522d490 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fcb90000c00 tx=0x7fcb90004480 comp rx=0 tx=0).stop 2024-08-30T09:04:54.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.606+0000 7fcba8cf1740 1 -- 172.21.15.42:0/932310664 shutdown_connections 2024-08-30T09:04:54.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.606+0000 7fcba8cf1740 1 --2- 172.21.15.42:0/932310664 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x561e44e0bda0 0x561e452ca8f0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:54.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.606+0000 7fcba8cf1740 1 --2- 172.21.15.42:0/932310664 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561e44e78070 0x561e4522d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:54.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.606+0000 7fcba8cf1740 1 --2- 172.21.15.42:0/932310664 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561e44e57390 0x561e4522d490 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:54.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.606+0000 7fcba8cf1740 1 --2- 172.21.15.42:0/932310664 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561e44e4f1b0 0x561e4522cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:54.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.606+0000 7fcba8cf1740 1 -- 172.21.15.42:0/932310664 >> 172.21.15.42:0/932310664 conn(0x561e44e6c460 msgr2=0x561e45220190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:04:54.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.606+0000 7fcb7b7fe640 0 client.25886 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:04:54.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.607+0000 7fcba8cf1740 1 -- 172.21.15.42:0/932310664 shutdown_connections 2024-08-30T09:04:54.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.607+0000 7fcba8cf1740 1 -- 172.21.15.42:0/932310664 wait complete. 2024-08-30T09:04:54.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.607+0000 7fcba8cf1740 20 client.25886 trim_cache size 0 max 16384 2024-08-30T09:04:54.611 INFO:teuthology.orchestra.run.smithi042.stdout:[] 2024-08-30T09:04:54.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.610+0000 7fcba277c640 1 -- 172.21.15.42:0/299773529 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fcb840b4bb0 msgr2=0x7fcb840b70c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:04:54.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.610+0000 7fcba277c640 1 --2- 172.21.15.42:0/299773529 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fcb840b4bb0 0x7fcb840b70c0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fcb940abed0 tx=0x7fcb94005f70 comp rx=0 tx=0).stop 2024-08-30T09:04:54.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.610+0000 7fcba277c640 1 -- 172.21.15.42:0/299773529 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcb9c10eef0 msgr2=0x7fcb9c1b4da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:04:54.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.610+0000 7fcba277c640 1 --2- 172.21.15.42:0/299773529 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcb9c10eef0 0x7fcb9c1b4da0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fcb94009fd0 tx=0x7fcb9400ef10 comp rx=0 tx=0).stop 2024-08-30T09:04:54.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.610+0000 7fcba277c640 1 -- 172.21.15.42:0/299773529 shutdown_connections 2024-08-30T09:04:54.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.610+0000 7fcba277c640 1 --2- 172.21.15.42:0/299773529 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fcb840b4bb0 0x7fcb840b70c0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:54.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.610+0000 7fcba277c640 1 --2- 172.21.15.42:0/299773529 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcb9c10aab0 0x7fcb9c1aa260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:54.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.610+0000 7fcba277c640 1 --2- 172.21.15.42:0/299773529 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcb9c10eef0 0x7fcb9c1b4da0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:54.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.610+0000 7fcba277c640 1 --2- 172.21.15.42:0/299773529 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcb9c10a160 0x7fcb9c1a9d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:54.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.610+0000 7fcba277c640 1 -- 172.21.15.42:0/299773529 >> 172.21.15.42:0/299773529 conn(0x7fcb9c071090 msgr2=0x7fcb9c10e350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:04:54.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.610+0000 7fcba277c640 1 -- 172.21.15.42:0/299773529 shutdown_connections 2024-08-30T09:04:54.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:04:54.610+0000 7fcba277c640 1 -- 172.21.15.42:0/299773529 wait complete. 2024-08-30T09:04:54.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:04:54 smithi042 ceph-mon[18652]: pgmap v497: 129 pgs: 129 active+clean; 1.1 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 9.9 MiB/s wr, 876 op/s 2024-08-30T09:04:54.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:04:54 smithi190 ceph-mon[22001]: pgmap v497: 129 pgs: 129 active+clean; 1.1 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 9.9 MiB/s wr, 876 op/s 2024-08-30T09:04:55.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:04:54 smithi167 ceph-mon[22063]: pgmap v497: 129 pgs: 129 active+clean; 1.1 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 9.9 MiB/s wr, 876 op/s 2024-08-30T09:04:55.953 INFO:teuthology.orchestra.run.smithi042.stderr:progress: 250.0 s, 1055.0 tps, lat 92.775 ms stddev 97.188 2024-08-30T09:04:56.503 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-08-30T09:04:56.503 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-08-30T09:04:56.884 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:04:56 smithi042 ceph-mon[18652]: pgmap v498: 129 pgs: 129 active+clean; 1.1 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 5.7 MiB/s wr, 605 op/s 2024-08-30T09:04:56.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:04:56 smithi190 ceph-mon[22001]: pgmap v498: 129 pgs: 129 active+clean; 1.1 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 5.7 MiB/s wr, 605 op/s 2024-08-30T09:04:57.017 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T09:04:57.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:04:56 smithi167 ceph-mon[22063]: pgmap v498: 129 pgs: 129 active+clean; 1.1 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 5.7 MiB/s wr, 605 op/s 2024-08-30T09:04:58.634 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.631+0000 7f1b5954a640 1 -- 172.21.15.42:0/3869882578 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1b5410a160 msgr2=0x7f1b5410a570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:04:58.634 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.631+0000 7f1b5954a640 1 --2- 172.21.15.42:0/3869882578 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1b5410a160 0x7f1b5410a570 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f1b44009860 tx=0x7f1b4402a340 comp rx=0 tx=0).stop 2024-08-30T09:04:58.635 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.631+0000 7f1b5954a640 1 -- 172.21.15.42:0/3869882578 shutdown_connections 2024-08-30T09:04:58.635 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.631+0000 7f1b5954a640 1 --2- 172.21.15.42:0/3869882578 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1b54111d30 0x7f1b54114150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:58.635 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.631+0000 7f1b5954a640 1 --2- 172.21.15.42:0/3869882578 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1b5410ab90 0x7f1b5410b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:58.635 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.631+0000 7f1b5954a640 1 --2- 172.21.15.42:0/3869882578 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1b5410a160 0x7f1b5410a570 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:58.635 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.631+0000 7f1b5954a640 1 -- 172.21.15.42:0/3869882578 >> 172.21.15.42:0/3869882578 conn(0x7f1b54070a10 msgr2=0x7f1b54072ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:04:58.635 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.631+0000 7f1b5954a640 1 -- 172.21.15.42:0/3869882578 shutdown_connections 2024-08-30T09:04:58.635 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.631+0000 7f1b5954a640 1 -- 172.21.15.42:0/3869882578 wait complete. 2024-08-30T09:04:58.635 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.632+0000 7f1b5954a640 1 Processor -- start 2024-08-30T09:04:58.637 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.632+0000 7f1b5954a640 1 -- start start 2024-08-30T09:04:58.637 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.632+0000 7f1b5954a640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1b5410a160 0x7f1b541163f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:58.637 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.632+0000 7f1b5954a640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1b5410ab90 0x7f1b54116a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:58.637 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.632+0000 7f1b5954a640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1b54111d30 0x7f1b54117610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:58.638 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.632+0000 7f1b5954a640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f1b5410d340 con 0x7f1b5410ab90 2024-08-30T09:04:58.638 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.632+0000 7f1b5954a640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f1b5410d1c0 con 0x7f1b54111d30 2024-08-30T09:04:58.638 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.632+0000 7f1b5954a640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f1b5410d4c0 con 0x7f1b5410a160 2024-08-30T09:04:58.638 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.632+0000 7f1b527fc640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1b5410ab90 0x7f1b54116a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:58.638 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.632+0000 7f1b537fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1b54111d30 0x7f1b54117610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:58.638 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.632+0000 7f1b527fc640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1b5410ab90 0x7f1b54116a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:39988/0 (socket says 172.21.15.42:39988) 2024-08-30T09:04:58.638 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.632+0000 7f1b537fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1b54111d30 0x7f1b54117610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:36146/0 (socket says 172.21.15.42:36146) 2024-08-30T09:04:58.638 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.632+0000 7f1b527fc640 1 -- 172.21.15.42:0/251149611 learned_addr learned my addr 172.21.15.42:0/251149611 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:04:58.639 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.632+0000 7f1b52ffd640 1 --2- 172.21.15.42:0/251149611 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1b5410a160 0x7f1b541163f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:58.639 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.632+0000 7f1b527fc640 1 -- 172.21.15.42:0/251149611 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1b5410a160 msgr2=0x7f1b541163f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:04:58.639 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.632+0000 7f1b527fc640 1 --2- 172.21.15.42:0/251149611 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1b5410a160 0x7f1b541163f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:58.639 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.632+0000 7f1b527fc640 1 -- 172.21.15.42:0/251149611 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1b54111d30 msgr2=0x7f1b54117610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:04:58.639 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.632+0000 7f1b527fc640 1 --2- 172.21.15.42:0/251149611 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1b54111d30 0x7f1b54117610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:58.639 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.633+0000 7f1b527fc640 1 -- 172.21.15.42:0/251149611 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b54117e30 con 0x7f1b5410ab90 2024-08-30T09:04:58.639 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.633+0000 7f1b537fe640 1 --2- 172.21.15.42:0/251149611 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1b54111d30 0x7f1b54117610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-30T09:04:58.639 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.633+0000 7f1b527fc640 1 --2- 172.21.15.42:0/251149611 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1b5410ab90 0x7f1b54116a60 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f1b4800e9c0 tx=0x7f1b4800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:04:58.639 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.633+0000 7f1b3bfff640 1 -- 172.21.15.42:0/251149611 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b4800ccc0 con 0x7f1b5410ab90 2024-08-30T09:04:58.639 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.633+0000 7f1b3bfff640 1 -- 172.21.15.42:0/251149611 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b480054d0 con 0x7f1b5410ab90 2024-08-30T09:04:58.640 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.633+0000 7f1b5954a640 1 -- 172.21.15.42:0/251149611 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b54118120 con 0x7f1b5410ab90 2024-08-30T09:04:58.640 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.633+0000 7f1b5954a640 1 -- 172.21.15.42:0/251149611 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b541187e0 con 0x7f1b5410ab90 2024-08-30T09:04:58.640 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.633+0000 7f1b3bfff640 1 -- 172.21.15.42:0/251149611 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b48010430 con 0x7f1b5410ab90 2024-08-30T09:04:58.646 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.635+0000 7f1b604b3740 1 Processor -- start 2024-08-30T09:04:58.646 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.635+0000 7f1b604b3740 1 -- start start 2024-08-30T09:04:58.646 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.636+0000 7f1b604b3740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5560a62371b0 0x5560a65dd180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:58.646 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.636+0000 7f1b604b3740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5560a620e2f0 0x5560a65dd7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:58.647 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.636+0000 7f1b604b3740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5560a62164d0 0x5560a65de340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:58.647 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.636+0000 7f1b604b3740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x5560a6550000 con 0x5560a62371b0 2024-08-30T09:04:58.647 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.636+0000 7f1b604b3740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5560a653f590 con 0x5560a620e2f0 2024-08-30T09:04:58.647 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.636+0000 7f1b604b3740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5560a6543540 con 0x5560a62164d0 2024-08-30T09:04:58.647 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.636+0000 7f1b3bfff640 1 -- 172.21.15.42:0/251149611 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7f1b480105d0 con 0x7f1b5410ab90 2024-08-30T09:04:58.647 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.636+0000 7f1b537fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5560a62371b0 0x5560a65dd180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:58.647 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.636+0000 7f1b537fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5560a62371b0 0x5560a65dd180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:40000/0 (socket says 172.21.15.42:40000) 2024-08-30T09:04:58.647 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.636+0000 7f1b537fe640 1 -- 172.21.15.42:0/828056554 learned_addr learned my addr 172.21.15.42:0/828056554 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:04:58.647 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.636+0000 7f1b52ffd640 1 --2- 172.21.15.42:0/828056554 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5560a620e2f0 0x5560a65dd7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:58.647 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.636+0000 7f1b527fc640 1 --2- 172.21.15.42:0/828056554 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5560a62164d0 0x5560a65de340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:58.647 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.636+0000 7f1b537fe640 1 -- 172.21.15.42:0/828056554 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5560a62164d0 msgr2=0x5560a65de340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:04:58.647 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.636+0000 7f1b537fe640 1 --2- 172.21.15.42:0/828056554 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5560a62164d0 0x5560a65de340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:58.647 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.636+0000 7f1b537fe640 1 -- 172.21.15.42:0/828056554 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5560a620e2f0 msgr2=0x5560a65dd7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:04:58.647 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.636+0000 7f1b537fe640 1 --2- 172.21.15.42:0/828056554 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5560a620e2f0 0x5560a65dd7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:58.648 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.636+0000 7f1b537fe640 1 -- 172.21.15.42:0/828056554 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5560a62a2790 con 0x5560a62371b0 2024-08-30T09:04:58.648 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.636+0000 7f1b52ffd640 1 --2- 172.21.15.42:0/828056554 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5560a620e2f0 0x5560a65dd7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-30T09:04:58.648 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.636+0000 7f1b527fc640 1 --2- 172.21.15.42:0/828056554 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5560a62164d0 0x5560a65de340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-30T09:04:58.648 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.636+0000 7f1b537fe640 1 --2- 172.21.15.42:0/828056554 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5560a62371b0 0x5560a65dd180 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f1b4c0117e0 tx=0x7f1b4c011ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3ab12ef5a2781c0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:04:58.648 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.636+0000 7f1b397fa640 1 -- 172.21.15.42:0/828056554 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b4c011310 con 0x5560a62371b0 2024-08-30T09:04:58.648 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.636+0000 7f1b397fa640 1 -- 172.21.15.42:0/828056554 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b4c003e20 con 0x5560a62371b0 2024-08-30T09:04:58.648 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.636+0000 7f1b397fa640 1 -- 172.21.15.42:0/828056554 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b4c0044f0 con 0x5560a62371b0 2024-08-30T09:04:58.648 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.637+0000 7f1b3bfff640 1 --2- 172.21.15.42:0/251149611 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f1b3c0b4bb0 0x7f1b3c0b70c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:58.648 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.637+0000 7f1b52ffd640 1 --2- 172.21.15.42:0/251149611 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f1b3c0b4bb0 0x7f1b3c0b70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:58.648 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.637+0000 7f1b3bfff640 1 -- 172.21.15.42:0/251149611 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f1b480e5830 con 0x7f1b5410ab90 2024-08-30T09:04:58.648 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.637+0000 7f1b604b3740 1 -- 172.21.15.42:0/828056554 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5560a62371b0 msgr2=0x5560a65dd180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:04:58.648 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.637+0000 7f1b604b3740 1 --2- 172.21.15.42:0/828056554 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5560a62371b0 0x5560a65dd180 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f1b4c0117e0 tx=0x7f1b4c011ca0 comp rx=0 tx=0).stop 2024-08-30T09:04:58.648 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.637+0000 7f1b604b3740 1 -- 172.21.15.42:0/828056554 shutdown_connections 2024-08-30T09:04:58.648 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.637+0000 7f1b604b3740 1 --2- 172.21.15.42:0/828056554 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5560a62164d0 0x5560a65de340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:58.649 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.637+0000 7f1b604b3740 1 --2- 172.21.15.42:0/828056554 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5560a620e2f0 0x5560a65dd7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:58.649 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.637+0000 7f1b604b3740 1 --2- 172.21.15.42:0/828056554 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5560a62371b0 0x5560a65dd180 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:58.649 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.637+0000 7f1b604b3740 1 -- 172.21.15.42:0/828056554 >> 172.21.15.42:0/828056554 conn(0x5560a622b300 msgr2=0x5560a65d8c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:04:58.649 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.638+0000 7f1b604b3740 1 -- 172.21.15.42:0/828056554 shutdown_connections 2024-08-30T09:04:58.649 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.638+0000 7f1b604b3740 1 -- 172.21.15.42:0/828056554 wait complete. 2024-08-30T09:04:58.649 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.638+0000 7f1b52ffd640 1 --2- 172.21.15.42:0/251149611 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f1b3c0b4bb0 0x7f1b3c0b70c0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f1b440304b0 tx=0x7f1b4402c040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:04:58.649 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.638+0000 7f1b604b3740 1 -- start start 2024-08-30T09:04:58.649 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.638+0000 7f1b604b3740 1 Processor -- start 2024-08-30T09:04:58.649 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.638+0000 7f1b604b3740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5560a620e2f0 0x5560a65e6c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:58.649 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.638+0000 7f1b604b3740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5560a62164d0 0x5560a65e72f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:58.649 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.638+0000 7f1b604b3740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5560a62371b0 0x5560a65e7e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:58.649 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.638+0000 7f1b604b3740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x5560a6478ab0 con 0x5560a62164d0 2024-08-30T09:04:58.649 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.638+0000 7f1b604b3740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5560a64c6680 con 0x5560a62371b0 2024-08-30T09:04:58.650 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.638+0000 7f1b604b3740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5560a65dee60 con 0x5560a620e2f0 2024-08-30T09:04:58.650 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.638+0000 7f1b527fc640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5560a62164d0 0x5560a65e72f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:58.650 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.638+0000 7f1b527fc640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5560a62164d0 0x5560a65e72f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:40014/0 (socket says 172.21.15.42:40014) 2024-08-30T09:04:58.650 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.638+0000 7f1b527fc640 1 -- 172.21.15.42:0/3800333707 learned_addr learned my addr 172.21.15.42:0/3800333707 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:04:58.650 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.638+0000 7f1b604b3740 10 client.0 _ll_register_callbacks cb 0x5560a624ea00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:04:58.650 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.639+0000 7f1b537fe640 1 --2- 172.21.15.42:0/3800333707 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5560a62371b0 0x5560a65e7e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:58.650 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.639+0000 7f1b527fc640 1 -- 172.21.15.42:0/3800333707 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5560a620e2f0 msgr2=0x5560a65e6c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:04:58.650 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.639+0000 7f1b527fc640 1 --2- 172.21.15.42:0/3800333707 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5560a620e2f0 0x5560a65e6c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:58.650 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.639+0000 7f1b527fc640 1 -- 172.21.15.42:0/3800333707 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5560a62371b0 msgr2=0x5560a65e7e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:04:58.650 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.639+0000 7f1b527fc640 1 --2- 172.21.15.42:0/3800333707 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5560a62371b0 0x5560a65e7e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:58.650 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.639+0000 7f1b527fc640 1 -- 172.21.15.42:0/3800333707 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5560a631b620 con 0x5560a62164d0 2024-08-30T09:04:58.650 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.639+0000 7f1b537fe640 1 --2- 172.21.15.42:0/3800333707 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5560a62371b0 0x5560a65e7e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-30T09:04:58.650 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.639+0000 7f1b527fc640 1 --2- 172.21.15.42:0/3800333707 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5560a62164d0 0x5560a65e72f0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f1b48009230 tx=0x7f1b4800ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=956f0e200656da13 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:04:58.650 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.639+0000 7f1b36ff5640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-08-30T09:04:58.651 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.639+0000 7f1b36ff5640 1 -- 172.21.15.42:0/3800333707 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b480abed0 con 0x5560a62164d0 2024-08-30T09:04:58.651 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.639+0000 7f1b36ff5640 1 -- 172.21.15.42:0/3800333707 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f1b480932f0 con 0x5560a62164d0 2024-08-30T09:04:58.651 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.639+0000 7f1b604b3740 1 -- 172.21.15.42:0/3800333707 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1b4c003e20 con 0x5560a62164d0 2024-08-30T09:04:58.651 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.639+0000 7f1b36ff5640 1 -- 172.21.15.42:0/3800333707 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f1b480b0740 con 0x5560a62164d0 2024-08-30T09:04:58.651 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.639+0000 7f1b604b3740 10 client.16182 fetch_fsmap learned FSMap version 29 2024-08-30T09:04:58.651 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.639+0000 7f1b604b3740 1 -- 172.21.15.42:0/3800333707 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5560a65f0de0 con 0x5560a62164d0 2024-08-30T09:04:58.651 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.639+0000 7f1b36ff5640 1 -- 172.21.15.42:0/3800333707 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b4c003e20 con 0x5560a62164d0 2024-08-30T09:04:58.651 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.640+0000 7f1b36ff5640 1 -- 172.21.15.42:0/3800333707 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f1b480e5ce0 con 0x5560a62164d0 2024-08-30T09:04:58.651 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.640+0000 7f1b604b3740 10 client.16182 fetch_fsmap finished waiting for FSMap version 29 2024-08-30T09:04:58.651 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.640+0000 7f1b604b3740 10 client.16182 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-30T09:04:58.651 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.640+0000 7f1b604b3740 20 client.16182 populate_metadata read hostname 'smithi042' 2024-08-30T09:04:58.651 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.640+0000 7f1b604b3740 1 --2- 172.21.15.42:0/3800333707 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x5560a61f5d80 0x5560a65f4640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:58.651 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.640+0000 7f1b604b3740 4 client.16182 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-30T09:04:58.652 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.640+0000 7f1b604b3740 1 -- 172.21.15.42:0/3800333707 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5560a6551f50 con 0x5560a61f5d80 2024-08-30T09:04:58.652 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.640+0000 7f1b52ffd640 1 --2- 172.21.15.42:0/3800333707 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x5560a61f5d80 0x5560a65f4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:58.652 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.640+0000 7f1b52ffd640 1 --2- 172.21.15.42:0/3800333707 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x5560a61f5d80 0x5560a65f4640 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e4b32905c9e48a6 server_cookie=a1e2455394e477e1 in_seq=0 out_seq=0 2024-08-30T09:04:58.652 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.640+0000 7f1b36ff5640 10 client.16182 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:04:58.652 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.641+0000 7f1b36ff5640 1 -- 172.21.15.42:0/3800333707 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5560a6551f50 con 0x5560a61f5d80 2024-08-30T09:04:58.652 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.641+0000 7f1b36ff5640 10 client.16182 handle_command_reply: tid=0 2024-08-30T09:04:58.652 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.641+0000 7f1b604b3740 1 client.16182 shutdown 2024-08-30T09:04:58.652 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.641+0000 7f1b604b3740 1 -- 172.21.15.42:0/3800333707 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5560a62164d0 msgr2=0x5560a65e72f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:04:58.652 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.641+0000 7f1b604b3740 1 --2- 172.21.15.42:0/3800333707 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5560a62164d0 0x5560a65e72f0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f1b48009230 tx=0x7f1b4800ce60 comp rx=0 tx=0).stop 2024-08-30T09:04:58.652 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.641+0000 7f1b604b3740 1 -- 172.21.15.42:0/3800333707 shutdown_connections 2024-08-30T09:04:58.652 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.641+0000 7f1b604b3740 1 --2- 172.21.15.42:0/3800333707 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x5560a61f5d80 0x5560a65f4640 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:58.652 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.641+0000 7f1b604b3740 1 --2- 172.21.15.42:0/3800333707 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5560a62371b0 0x5560a65e7e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:58.652 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.641+0000 7f1b604b3740 1 --2- 172.21.15.42:0/3800333707 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5560a62164d0 0x5560a65e72f0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:58.653 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.641+0000 7f1b604b3740 1 --2- 172.21.15.42:0/3800333707 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5560a620e2f0 0x5560a65e6c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:58.653 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.641+0000 7f1b604b3740 1 -- 172.21.15.42:0/3800333707 >> 172.21.15.42:0/3800333707 conn(0x5560a622b300 msgr2=0x5560a65dd6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:04:58.653 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.641+0000 7f1b36ff5640 0 client.16182 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:04:58.653 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.641+0000 7f1b604b3740 1 -- 172.21.15.42:0/3800333707 shutdown_connections 2024-08-30T09:04:58.653 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.641+0000 7f1b604b3740 1 -- 172.21.15.42:0/3800333707 wait complete. 2024-08-30T09:04:58.653 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.641+0000 7f1b604b3740 20 client.16182 trim_cache size 0 max 16384 2024-08-30T09:04:58.663 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.658+0000 7f1b604b3740 1 Processor -- start 2024-08-30T09:04:58.663 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.659+0000 7f1b604b3740 1 -- start start 2024-08-30T09:04:58.663 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.659+0000 7f1b604b3740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5560a620e2f0 0x5560a65ddfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:58.663 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.659+0000 7f1b604b3740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5560a62164d0 0x5560a65eed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:58.663 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.659+0000 7f1b604b3740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5560a62371b0 0x5560a65e19d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:58.663 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.659+0000 7f1b604b3740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f1b541c6280 con 0x5560a62371b0 2024-08-30T09:04:58.663 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.659+0000 7f1b604b3740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f1b541c3b80 con 0x5560a620e2f0 2024-08-30T09:04:58.663 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.659+0000 7f1b604b3740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f1b541aa950 con 0x5560a62164d0 2024-08-30T09:04:58.664 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.659+0000 7f1b537fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5560a62371b0 0x5560a65e19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:58.664 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.659+0000 7f1b537fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5560a62371b0 0x5560a65e19d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:40022/0 (socket says 172.21.15.42:40022) 2024-08-30T09:04:58.664 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.659+0000 7f1b537fe640 1 -- 172.21.15.42:0/3703199035 learned_addr learned my addr 172.21.15.42:0/3703199035 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:04:58.664 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.659+0000 7f1b52ffd640 1 --2- 172.21.15.42:0/3703199035 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5560a620e2f0 0x5560a65ddfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:58.664 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.659+0000 7f1b527fc640 1 --2- 172.21.15.42:0/3703199035 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5560a62164d0 0x5560a65eed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:58.664 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.659+0000 7f1b537fe640 1 -- 172.21.15.42:0/3703199035 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5560a62164d0 msgr2=0x5560a65eed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:04:58.664 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.659+0000 7f1b537fe640 1 --2- 172.21.15.42:0/3703199035 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5560a62164d0 0x5560a65eed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:58.664 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.659+0000 7f1b537fe640 1 -- 172.21.15.42:0/3703199035 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5560a620e2f0 msgr2=0x5560a65ddfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:04:58.665 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.659+0000 7f1b537fe640 1 --2- 172.21.15.42:0/3703199035 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5560a620e2f0 0x5560a65ddfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:58.665 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.659+0000 7f1b537fe640 1 -- 172.21.15.42:0/3703199035 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5560a61f5fb0 con 0x5560a62371b0 2024-08-30T09:04:58.665 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.659+0000 7f1b52ffd640 1 --2- 172.21.15.42:0/3703199035 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5560a620e2f0 0x5560a65ddfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-30T09:04:58.665 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.660+0000 7f1b537fe640 1 --2- 172.21.15.42:0/3703199035 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5560a62371b0 0x5560a65e19d0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f1b4c0117b0 tx=0x7f1b4c0113d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99ac39a58d4ac383 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:04:58.665 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.660+0000 7f1b38ff9640 1 -- 172.21.15.42:0/3703199035 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b4c029020 con 0x5560a62371b0 2024-08-30T09:04:58.665 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.660+0000 7f1b38ff9640 1 -- 172.21.15.42:0/3703199035 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b4c003e20 con 0x5560a62371b0 2024-08-30T09:04:58.665 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.660+0000 7f1b38ff9640 1 -- 172.21.15.42:0/3703199035 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b4c0053d0 con 0x5560a62371b0 2024-08-30T09:04:58.669 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.661+0000 7f1b604b3740 1 -- 172.21.15.42:0/3703199035 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5560a62371b0 msgr2=0x5560a65e19d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:04:58.669 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.661+0000 7f1b604b3740 1 --2- 172.21.15.42:0/3703199035 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5560a62371b0 0x5560a65e19d0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f1b4c0117b0 tx=0x7f1b4c0113d0 comp rx=0 tx=0).stop 2024-08-30T09:04:58.669 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.661+0000 7f1b604b3740 1 -- 172.21.15.42:0/3703199035 shutdown_connections 2024-08-30T09:04:58.669 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.661+0000 7f1b604b3740 1 --2- 172.21.15.42:0/3703199035 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5560a62164d0 0x5560a65eed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:58.669 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.661+0000 7f1b604b3740 1 --2- 172.21.15.42:0/3703199035 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5560a62371b0 0x5560a65e19d0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:58.669 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.661+0000 7f1b604b3740 1 --2- 172.21.15.42:0/3703199035 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5560a620e2f0 0x5560a65ddfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:58.669 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.661+0000 7f1b604b3740 1 -- 172.21.15.42:0/3703199035 >> 172.21.15.42:0/3703199035 conn(0x5560a622b300 msgr2=0x5560a65dce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:04:58.669 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.661+0000 7f1b604b3740 1 -- 172.21.15.42:0/3703199035 shutdown_connections 2024-08-30T09:04:58.669 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.661+0000 7f1b604b3740 1 -- 172.21.15.42:0/3703199035 wait complete. 2024-08-30T09:04:58.669 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.661+0000 7f1b604b3740 1 -- start start 2024-08-30T09:04:58.669 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.661+0000 7f1b604b3740 1 Processor -- start 2024-08-30T09:04:58.670 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.662+0000 7f1b604b3740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5560a620e2f0 0x5560a65e9650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:58.670 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.662+0000 7f1b604b3740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5560a62164d0 0x5560a65e9d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:58.670 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.662+0000 7f1b604b3740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5560a62371b0 0x5560a65ea880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:58.670 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.662+0000 7f1b527fc640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5560a62164d0 0x5560a65e9d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:58.670 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.662+0000 7f1b527fc640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5560a62164d0 0x5560a65e9d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:40036/0 (socket says 172.21.15.42:40036) 2024-08-30T09:04:58.670 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.662+0000 7f1b527fc640 1 -- 172.21.15.42:0/4232883205 learned_addr learned my addr 172.21.15.42:0/4232883205 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:04:58.670 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.662+0000 7f1b604b3740 1 -- 172.21.15.42:0/4232883205 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f1b541aafc0 con 0x5560a62164d0 2024-08-30T09:04:58.670 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.662+0000 7f1b604b3740 1 -- 172.21.15.42:0/4232883205 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f1b541ac430 con 0x5560a620e2f0 2024-08-30T09:04:58.670 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.662+0000 7f1b52ffd640 1 --2- 172.21.15.42:0/4232883205 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5560a620e2f0 0x5560a65e9650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:58.670 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.662+0000 7f1b537fe640 1 --2- 172.21.15.42:0/4232883205 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5560a62371b0 0x5560a65ea880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:58.670 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.662+0000 7f1b604b3740 1 -- 172.21.15.42:0/4232883205 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f1b541ab690 con 0x5560a62371b0 2024-08-30T09:04:58.670 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.662+0000 7f1b604b3740 10 client.0 _ll_register_callbacks cb 0x7f1b48004920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:04:58.670 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.662+0000 7f1b527fc640 1 -- 172.21.15.42:0/4232883205 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5560a62371b0 msgr2=0x5560a65ea880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:04:58.670 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.662+0000 7f1b527fc640 1 --2- 172.21.15.42:0/4232883205 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5560a62371b0 0x5560a65ea880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:58.671 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.662+0000 7f1b527fc640 1 -- 172.21.15.42:0/4232883205 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5560a620e2f0 msgr2=0x5560a65e9650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:04:58.671 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.662+0000 7f1b527fc640 1 --2- 172.21.15.42:0/4232883205 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5560a620e2f0 0x5560a65e9650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:58.671 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.662+0000 7f1b527fc640 1 -- 172.21.15.42:0/4232883205 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5560a65eafe0 con 0x5560a62164d0 2024-08-30T09:04:58.671 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.662+0000 7f1b52ffd640 1 --2- 172.21.15.42:0/4232883205 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5560a620e2f0 0x5560a65e9650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-30T09:04:58.671 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.662+0000 7f1b527fc640 1 --2- 172.21.15.42:0/4232883205 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5560a62164d0 0x5560a65e9d10 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f1b480a8470 tx=0x7f1b480ab8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26315db8cd3b2b4e server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:04:58.671 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.662+0000 7f1b36ff5640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-08-30T09:04:58.671 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.662+0000 7f1b36ff5640 1 -- 172.21.15.42:0/4232883205 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b480a7bc0 con 0x5560a62164d0 2024-08-30T09:04:58.671 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.662+0000 7f1b36ff5640 1 -- 172.21.15.42:0/4232883205 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f1b480ab130 con 0x5560a62164d0 2024-08-30T09:04:58.671 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.662+0000 7f1b604b3740 1 -- 172.21.15.42:0/4232883205 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1b4c003e20 con 0x5560a62164d0 2024-08-30T09:04:58.671 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.663+0000 7f1b36ff5640 1 -- 172.21.15.42:0/4232883205 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f1b480a7d90 con 0x5560a62164d0 2024-08-30T09:04:58.671 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.663+0000 7f1b604b3740 10 client.16188 fetch_fsmap learned FSMap version 29 2024-08-30T09:04:58.671 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.663+0000 7f1b604b3740 1 -- 172.21.15.42:0/4232883205 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5560a65ee340 con 0x5560a62164d0 2024-08-30T09:04:58.671 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.663+0000 7f1b36ff5640 1 -- 172.21.15.42:0/4232883205 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b4c003e20 con 0x5560a62164d0 2024-08-30T09:04:58.671 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.663+0000 7f1b36ff5640 1 -- 172.21.15.42:0/4232883205 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f1b48004f80 con 0x5560a62164d0 2024-08-30T09:04:58.672 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.663+0000 7f1b604b3740 10 client.16188 fetch_fsmap finished waiting for FSMap version 29 2024-08-30T09:04:58.672 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.663+0000 7f1b604b3740 10 client.16188 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-30T09:04:58.672 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.663+0000 7f1b604b3740 20 client.16188 populate_metadata read hostname 'smithi042' 2024-08-30T09:04:58.672 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.663+0000 7f1b604b3740 1 --2- 172.21.15.42:0/4232883205 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x5560a61cb1c0 0x5560a6661010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:04:58.672 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.663+0000 7f1b604b3740 4 client.16188 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-30T09:04:58.672 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.663+0000 7f1b604b3740 1 -- 172.21.15.42:0/4232883205 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5560a654d9f0 con 0x5560a61cb1c0 2024-08-30T09:04:58.672 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.663+0000 7f1b52ffd640 1 --2- 172.21.15.42:0/4232883205 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x5560a61cb1c0 0x5560a6661010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:04:58.672 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.664+0000 7f1b52ffd640 1 --2- 172.21.15.42:0/4232883205 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x5560a61cb1c0 0x5560a6661010 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b3bb76dee2073f03 server_cookie=db421140aff0b12f in_seq=0 out_seq=0 2024-08-30T09:04:58.672 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.664+0000 7f1b36ff5640 10 client.16188 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:04:58.733 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.728+0000 7f1b36ff5640 1 -- 172.21.15.42:0/4232883205 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5560a654d9f0 con 0x5560a61cb1c0 2024-08-30T09:04:58.733 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.728+0000 7f1b36ff5640 10 client.16188 handle_command_reply: tid=0 2024-08-30T09:04:58.733 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.729+0000 7f1b604b3740 1 client.16188 shutdown 2024-08-30T09:04:58.733 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.729+0000 7f1b604b3740 1 -- 172.21.15.42:0/4232883205 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5560a62164d0 msgr2=0x5560a65e9d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:04:58.734 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.729+0000 7f1b604b3740 1 --2- 172.21.15.42:0/4232883205 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5560a62164d0 0x5560a65e9d10 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f1b480a8470 tx=0x7f1b480ab8d0 comp rx=0 tx=0).stop 2024-08-30T09:04:58.734 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.729+0000 7f1b604b3740 1 -- 172.21.15.42:0/4232883205 shutdown_connections 2024-08-30T09:04:58.734 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.729+0000 7f1b604b3740 1 --2- 172.21.15.42:0/4232883205 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x5560a61cb1c0 0x5560a6661010 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:58.734 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.729+0000 7f1b36ff5640 0 client.16188 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:04:58.734 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.730+0000 7f1b604b3740 1 --2- 172.21.15.42:0/4232883205 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5560a62371b0 0x5560a65ea880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:58.734 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.730+0000 7f1b604b3740 1 --2- 172.21.15.42:0/4232883205 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5560a62164d0 0x5560a65e9d10 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:58.734 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.730+0000 7f1b604b3740 1 --2- 172.21.15.42:0/4232883205 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5560a620e2f0 0x5560a65e9650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:58.734 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.730+0000 7f1b604b3740 1 -- 172.21.15.42:0/4232883205 >> 172.21.15.42:0/4232883205 conn(0x5560a622b300 msgr2=0x5560a65dce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:04:58.734 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.730+0000 7f1b604b3740 1 -- 172.21.15.42:0/4232883205 shutdown_connections 2024-08-30T09:04:58.734 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.730+0000 7f1b604b3740 1 -- 172.21.15.42:0/4232883205 wait complete. 2024-08-30T09:04:58.734 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.730+0000 7f1b604b3740 20 client.16188 trim_cache size 0 max 16384 2024-08-30T09:04:58.735 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-08-30T09:04:58.735 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2024-08-30T09:04:58.735 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-08-30T09:04:58.735 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-08-30T09:04:58.735 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.731+0000 7f1b5954a640 1 -- 172.21.15.42:0/251149611 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f1b3c0b4bb0 msgr2=0x7f1b3c0b70c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:04:58.735 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.731+0000 7f1b5954a640 1 --2- 172.21.15.42:0/251149611 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f1b3c0b4bb0 0x7f1b3c0b70c0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f1b440304b0 tx=0x7f1b4402c040 comp rx=0 tx=0).stop 2024-08-30T09:04:58.735 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.731+0000 7f1b5954a640 1 -- 172.21.15.42:0/251149611 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1b5410ab90 msgr2=0x7f1b54116a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:04:58.735 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.731+0000 7f1b5954a640 1 --2- 172.21.15.42:0/251149611 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1b5410ab90 0x7f1b54116a60 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f1b4800e9c0 tx=0x7f1b4800ee80 comp rx=0 tx=0).stop 2024-08-30T09:04:58.736 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.732+0000 7f1b5954a640 1 -- 172.21.15.42:0/251149611 shutdown_connections 2024-08-30T09:04:58.736 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.732+0000 7f1b5954a640 1 --2- 172.21.15.42:0/251149611 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f1b3c0b4bb0 0x7f1b3c0b70c0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:58.736 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.732+0000 7f1b5954a640 1 --2- 172.21.15.42:0/251149611 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1b54111d30 0x7f1b54117610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:58.736 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.732+0000 7f1b5954a640 1 --2- 172.21.15.42:0/251149611 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1b5410ab90 0x7f1b54116a60 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:58.736 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.732+0000 7f1b5954a640 1 --2- 172.21.15.42:0/251149611 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1b5410a160 0x7f1b541163f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:04:58.736 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.732+0000 7f1b5954a640 1 -- 172.21.15.42:0/251149611 >> 172.21.15.42:0/251149611 conn(0x7f1b54070a10 msgr2=0x7f1b5410dd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:04:58.736 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.732+0000 7f1b5954a640 1 -- 172.21.15.42:0/251149611 shutdown_connections 2024-08-30T09:04:58.736 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:04:58.732+0000 7f1b5954a640 1 -- 172.21.15.42:0/251149611 wait complete. 2024-08-30T09:04:58.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:04:58 smithi190 ceph-mon[22001]: pgmap v499: 129 pgs: 129 active+clean; 1.1 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 5.0 MiB/s wr, 582 op/s 2024-08-30T09:04:59.018 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:04:58 smithi042 ceph-mon[18652]: pgmap v499: 129 pgs: 129 active+clean; 1.1 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 5.0 MiB/s wr, 582 op/s 2024-08-30T09:04:59.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:04:58 smithi167 ceph-mon[22063]: pgmap v499: 129 pgs: 129 active+clean; 1.1 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 5.0 MiB/s wr, 582 op/s 2024-08-30T09:05:00.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.287+0000 7fc54b3e8640 1 -- 172.21.15.42:0/3238487459 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc54411c9d0 msgr2=0x7fc54411eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:05:00.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.287+0000 7fc54b3e8640 1 --2- 172.21.15.42:0/3238487459 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc54411c9d0 0x7fc54411eee0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fc540009900 tx=0x7fc54002a420 comp rx=0 tx=0).stop 2024-08-30T09:05:00.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.293+0000 7fc54b3e8640 1 -- 172.21.15.42:0/3238487459 shutdown_connections 2024-08-30T09:05:00.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.293+0000 7fc54b3e8640 1 --2- 172.21.15.42:0/3238487459 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc54411c9d0 0x7fc54411eee0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:00.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.293+0000 7fc54b3e8640 1 --2- 172.21.15.42:0/3238487459 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc54410a780 0x7fc54410ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:00.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.293+0000 7fc54b3e8640 1 --2- 172.21.15.42:0/3238487459 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc544111ce0 0x7fc54410a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:00.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.293+0000 7fc54b3e8640 1 -- 172.21.15.42:0/3238487459 >> 172.21.15.42:0/3238487459 conn(0x7fc544070c60 msgr2=0x7fc544071070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:05:00.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.297+0000 7fc54b3e8640 1 -- 172.21.15.42:0/3238487459 shutdown_connections 2024-08-30T09:05:00.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.298+0000 7fc54b3e8640 1 -- 172.21.15.42:0/3238487459 wait complete. 2024-08-30T09:05:00.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.300+0000 7fc54b3e8640 1 Processor -- start 2024-08-30T09:05:00.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.304+0000 7fc54b3e8640 1 -- start start 2024-08-30T09:05:00.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.304+0000 7fc54b3e8640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc54410a780 0x7fc5441a9030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:00.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.304+0000 7fc54b3e8640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc544111ce0 0x7fc5441a95d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:00.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.304+0000 7fc54b3e8640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc54411c9d0 0x7fc5441ae870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:00.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.304+0000 7fc54b3e8640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fc5441226b0 con 0x7fc544111ce0 2024-08-30T09:05:00.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.304+0000 7fc54b3e8640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc544122530 con 0x7fc54411c9d0 2024-08-30T09:05:00.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.304+0000 7fc54b3e8640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fc544122830 con 0x7fc54410a780 2024-08-30T09:05:00.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.305+0000 7fc549184640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc54410a780 0x7fc5441a9030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:05:00.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.305+0000 7fc549184640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc54410a780 0x7fc5441a9030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:60070/0 (socket says 172.21.15.42:60070) 2024-08-30T09:05:00.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.305+0000 7fc549184640 1 -- 172.21.15.42:0/1846509886 learned_addr learned my addr 172.21.15.42:0/1846509886 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:05:00.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.306+0000 7fc549985640 1 --2- 172.21.15.42:0/1846509886 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc54411c9d0 0x7fc5441ae870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:05:00.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.306+0000 7fc549184640 1 -- 172.21.15.42:0/1846509886 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc54411c9d0 msgr2=0x7fc5441ae870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:05:00.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.306+0000 7fc549184640 1 --2- 172.21.15.42:0/1846509886 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc54411c9d0 0x7fc5441ae870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:00.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.306+0000 7fc549184640 1 -- 172.21.15.42:0/1846509886 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc544111ce0 msgr2=0x7fc5441a95d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:05:00.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.306+0000 7fc549184640 1 --2- 172.21.15.42:0/1846509886 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc544111ce0 0x7fc5441a95d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:00.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.306+0000 7fc549184640 1 -- 172.21.15.42:0/1846509886 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5441aefa0 con 0x7fc54410a780 2024-08-30T09:05:00.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.307+0000 7fc549184640 1 --2- 172.21.15.42:0/1846509886 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc54410a780 0x7fc5441a9030 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fc53400c890 tx=0x7fc53400cd50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:05:00.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.307+0000 7fc53a7fc640 1 -- 172.21.15.42:0/1846509886 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc534004220 con 0x7fc54410a780 2024-08-30T09:05:00.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.307+0000 7fc54b3e8640 1 -- 172.21.15.42:0/1846509886 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5441af290 con 0x7fc54410a780 2024-08-30T09:05:00.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.307+0000 7fc54b3e8640 1 -- 172.21.15.42:0/1846509886 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5441b59f0 con 0x7fc54410a780 2024-08-30T09:05:00.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.308+0000 7fc53a7fc640 1 -- 172.21.15.42:0/1846509886 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc5340043c0 con 0x7fc54410a780 2024-08-30T09:05:00.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.308+0000 7fc53a7fc640 1 -- 172.21.15.42:0/1846509886 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc534004a00 con 0x7fc54410a780 2024-08-30T09:05:00.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.312+0000 7fc53a7fc640 1 -- 172.21.15.42:0/1846509886 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7fc534004c60 con 0x7fc54410a780 2024-08-30T09:05:00.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.315+0000 7fc55195d740 1 Processor -- start 2024-08-30T09:05:00.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.316+0000 7fc55195d740 1 -- start start 2024-08-30T09:05:00.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.316+0000 7fc53a7fc640 1 --2- 172.21.15.42:0/1846509886 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fc5300b4d40 0x7fc5300b7250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:00.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.316+0000 7fc55195d740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559c5364f350 0x559c539f5780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:00.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.316+0000 7fc55195d740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559c53626490 0x559c539f5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:00.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.316+0000 7fc55195d740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559c5362e670 0x559c539f6200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:00.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.316+0000 7fc55195d740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x559c5396b5f0 con 0x559c53626490 2024-08-30T09:05:00.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.316+0000 7fc55195d740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x559c5395ab80 con 0x559c5364f350 2024-08-30T09:05:00.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.316+0000 7fc55195d740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x559c5395eb30 con 0x559c5362e670 2024-08-30T09:05:00.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.316+0000 7fc53a7fc640 1 -- 172.21.15.42:0/1846509886 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7fc5340e6190 con 0x7fc54410a780 2024-08-30T09:05:00.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.317+0000 7fc549985640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559c5362e670 0x559c539f6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:05:00.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.317+0000 7fc549985640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559c5362e670 0x559c539f6200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:60082/0 (socket says 172.21.15.42:60082) 2024-08-30T09:05:00.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.317+0000 7fc549985640 1 -- 172.21.15.42:0/2703551039 learned_addr learned my addr 172.21.15.42:0/2703551039 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:05:00.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.318+0000 7fc549985640 1 -- 172.21.15.42:0/2703551039 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559c5364f350 msgr2=0x559c539f5780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:05:00.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.318+0000 7fc549985640 1 --2- 172.21.15.42:0/2703551039 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559c5364f350 0x559c539f5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:00.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.318+0000 7fc549985640 1 -- 172.21.15.42:0/2703551039 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559c53626490 msgr2=0x559c539f5cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:05:00.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.318+0000 7fc549985640 1 --2- 172.21.15.42:0/2703551039 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559c53626490 0x559c539f5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:00.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.318+0000 7fc549985640 1 -- 172.21.15.42:0/2703551039 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559c536ba930 con 0x559c5362e670 2024-08-30T09:05:00.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.318+0000 7fc549985640 1 --2- 172.21.15.42:0/2703551039 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559c5362e670 0x559c539f6200 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fc540009850 tx=0x7fc540004180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bdb4da22e1e9c6e6 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:05:00.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.322+0000 7fc52f7fe640 1 -- 172.21.15.42:0/2703551039 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc540041020 con 0x559c5362e670 2024-08-30T09:05:00.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.322+0000 7fc52f7fe640 1 -- 172.21.15.42:0/2703551039 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc540003aa0 con 0x559c5362e670 2024-08-30T09:05:00.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.323+0000 7fc549985640 1 --2- 172.21.15.42:0/1846509886 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fc5300b4d40 0x7fc5300b7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:05:00.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.323+0000 7fc55195d740 1 -- 172.21.15.42:0/2703551039 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559c5362e670 msgr2=0x559c539f6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:05:00.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.323+0000 7fc55195d740 1 --2- 172.21.15.42:0/2703551039 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559c5362e670 0x559c539f6200 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fc540009850 tx=0x7fc540004180 comp rx=0 tx=0).stop 2024-08-30T09:05:00.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.325+0000 7fc55195d740 1 -- 172.21.15.42:0/2703551039 shutdown_connections 2024-08-30T09:05:00.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.325+0000 7fc55195d740 1 --2- 172.21.15.42:0/2703551039 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559c5362e670 0x559c539f6200 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:00.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.325+0000 7fc55195d740 1 --2- 172.21.15.42:0/2703551039 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559c53626490 0x559c539f5cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:00.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.325+0000 7fc55195d740 1 --2- 172.21.15.42:0/2703551039 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559c5364f350 0x559c539f5780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:00.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.325+0000 7fc55195d740 1 -- 172.21.15.42:0/2703551039 >> 172.21.15.42:0/2703551039 conn(0x559c53643740 msgr2=0x559c539f3670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:05:00.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.325+0000 7fc549985640 1 --2- 172.21.15.42:0/1846509886 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fc5300b4d40 0x7fc5300b7250 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fc540009850 tx=0x7fc540004180 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:05:00.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.331+0000 7fc55195d740 1 -- 172.21.15.42:0/2703551039 shutdown_connections 2024-08-30T09:05:00.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.331+0000 7fc55195d740 1 -- 172.21.15.42:0/2703551039 wait complete. 2024-08-30T09:05:00.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.331+0000 7fc55195d740 1 -- start start 2024-08-30T09:05:00.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.331+0000 7fc55195d740 1 Processor -- start 2024-08-30T09:05:00.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.341+0000 7fc55195d740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559c53626490 0x559c53a00f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:00.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.341+0000 7fc55195d740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559c5362e670 0x559c53a01480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:00.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.341+0000 7fc55195d740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559c5364f350 0x559c53a0d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:00.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.341+0000 7fc55195d740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x559c537bd890 con 0x559c5362e670 2024-08-30T09:05:00.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.341+0000 7fc55195d740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x559c538e1790 con 0x559c53626490 2024-08-30T09:05:00.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.341+0000 7fc55195d740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x559c53a03f50 con 0x559c5364f350 2024-08-30T09:05:00.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.341+0000 7fc55195d740 10 client.0 _ll_register_callbacks cb 0x559c53666ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:05:00.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.342+0000 7fc549184640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559c53626490 0x559c53a00f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:05:00.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.342+0000 7fc549184640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559c53626490 0x559c53a00f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:36222/0 (socket says 172.21.15.42:36222) 2024-08-30T09:05:00.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.342+0000 7fc549184640 1 -- 172.21.15.42:0/1251238431 learned_addr learned my addr 172.21.15.42:0/1251238431 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:05:00.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.342+0000 7fc548983640 1 --2- 172.21.15.42:0/1251238431 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559c5362e670 0x559c53a01480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:05:00.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.343+0000 7fc549184640 1 -- 172.21.15.42:0/1251238431 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559c5364f350 msgr2=0x559c53a0d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:05:00.564 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-08-30T09:05:00.564 INFO:teuthology.orchestra.run.smithi042.stdout: "return_code": 0, 2024-08-30T09:05:00.564 INFO:teuthology.orchestra.run.smithi042.stdout: "scrub_tag": "347d05ed-4bd3-4590-b099-314f719aab39", 2024-08-30T09:05:00.565 INFO:teuthology.orchestra.run.smithi042.stdout: "mode": "asynchronous" 2024-08-30T09:05:00.565 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-08-30T09:05:00.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.343+0000 7fc549184640 1 --2- 172.21.15.42:0/1251238431 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559c5364f350 0x559c53a0d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:00.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.343+0000 7fc549184640 1 -- 172.21.15.42:0/1251238431 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559c5362e670 msgr2=0x559c53a01480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:05:00.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.343+0000 7fc549184640 1 --2- 172.21.15.42:0/1251238431 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559c5362e670 0x559c53a01480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:00.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.343+0000 7fc549184640 1 -- 172.21.15.42:0/1251238431 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559c53918f30 con 0x559c53626490 2024-08-30T09:05:00.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.343+0000 7fc549184640 1 --2- 172.21.15.42:0/1251238431 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559c53626490 0x559c53a00f40 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fc534007590 tx=0x7fc5340ac620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=22d2aab3de3fd967 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:05:00.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.344+0000 7fc52cff9640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-30T09:05:00.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.344+0000 7fc52cff9640 1 -- 172.21.15.42:0/1251238431 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc53401d040 con 0x559c53626490 2024-08-30T09:05:00.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.344+0000 7fc55195d740 1 -- 172.21.15.42:0/1251238431 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc540003aa0 con 0x559c53626490 2024-08-30T09:05:00.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.344+0000 7fc52cff9640 1 -- 172.21.15.42:0/1251238431 <== mon.2 v2:172.21.15.167:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7fc53408bc70 con 0x559c53626490 2024-08-30T09:05:00.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.345+0000 7fc52cff9640 1 -- 172.21.15.42:0/1251238431 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc53408beb0 con 0x559c53626490 2024-08-30T09:05:00.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.345+0000 7fc55195d740 10 client.25910 fetch_fsmap learned FSMap version 29 2024-08-30T09:05:00.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.345+0000 7fc55195d740 1 -- 172.21.15.42:0/1251238431 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x559c53a0fe70 con 0x559c53626490 2024-08-30T09:05:00.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.350+0000 7fc52cff9640 1 -- 172.21.15.42:0/1251238431 <== mon.2 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc540003aa0 con 0x559c53626490 2024-08-30T09:05:00.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.350+0000 7fc52cff9640 1 -- 172.21.15.42:0/1251238431 <== mon.2 v2:172.21.15.167:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fc53408f160 con 0x559c53626490 2024-08-30T09:05:00.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.351+0000 7fc55195d740 10 client.25910 fetch_fsmap finished waiting for FSMap version 29 2024-08-30T09:05:00.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.351+0000 7fc55195d740 10 client.25910 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-30T09:05:00.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.351+0000 7fc55195d740 20 client.25910 populate_metadata read hostname 'smithi042' 2024-08-30T09:05:00.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.351+0000 7fc55195d740 1 --2- 172.21.15.42:0/1251238431 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x559c5360dcd0 0x559c53a137f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:00.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.351+0000 7fc55195d740 4 client.25910 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-30T09:05:00.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.351+0000 7fc55195d740 1 -- 172.21.15.42:0/1251238431 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559c5396d540 con 0x559c5360dcd0 2024-08-30T09:05:00.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.351+0000 7fc548983640 1 --2- 172.21.15.42:0/1251238431 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x559c5360dcd0 0x559c53a137f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:05:00.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.355+0000 7fc548983640 1 --2- 172.21.15.42:0/1251238431 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x559c5360dcd0 0x559c53a137f0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4466b78ad59e3a4b server_cookie=7d51d9d11c20f3da in_seq=0 out_seq=0 2024-08-30T09:05:00.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.356+0000 7fc52cff9640 10 client.25910 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:05:00.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.358+0000 7fc52cff9640 1 -- 172.21.15.42:0/1251238431 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x559c5396d540 con 0x559c5360dcd0 2024-08-30T09:05:00.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.358+0000 7fc52cff9640 10 client.25910 handle_command_reply: tid=0 2024-08-30T09:05:00.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.358+0000 7fc55195d740 1 client.25910 shutdown 2024-08-30T09:05:00.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.362+0000 7fc55195d740 1 -- 172.21.15.42:0/1251238431 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559c53626490 msgr2=0x559c53a00f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:05:00.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.362+0000 7fc55195d740 1 --2- 172.21.15.42:0/1251238431 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559c53626490 0x559c53a00f40 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fc534007590 tx=0x7fc5340ac620 comp rx=0 tx=0).stop 2024-08-30T09:05:00.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.366+0000 7fc55195d740 1 -- 172.21.15.42:0/1251238431 shutdown_connections 2024-08-30T09:05:00.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.366+0000 7fc55195d740 1 --2- 172.21.15.42:0/1251238431 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x559c5360dcd0 0x559c53a137f0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:00.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.366+0000 7fc55195d740 1 --2- 172.21.15.42:0/1251238431 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559c5364f350 0x559c53a0d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:00.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.366+0000 7fc55195d740 1 --2- 172.21.15.42:0/1251238431 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559c5362e670 0x559c53a01480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:00.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.366+0000 7fc55195d740 1 --2- 172.21.15.42:0/1251238431 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559c53626490 0x559c53a00f40 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:00.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.366+0000 7fc55195d740 1 -- 172.21.15.42:0/1251238431 >> 172.21.15.42:0/1251238431 conn(0x559c53643740 msgr2=0x559c539f8910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:05:00.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.366+0000 7fc52cff9640 0 client.25910 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:05:00.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.367+0000 7fc55195d740 1 -- 172.21.15.42:0/1251238431 shutdown_connections 2024-08-30T09:05:00.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.367+0000 7fc55195d740 1 -- 172.21.15.42:0/1251238431 wait complete. 2024-08-30T09:05:00.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.368+0000 7fc55195d740 20 client.25910 trim_cache size 0 max 16384 2024-08-30T09:05:00.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.407+0000 7fc55195d740 1 Processor -- start 2024-08-30T09:05:00.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.407+0000 7fc55195d740 1 -- start start 2024-08-30T09:05:00.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.407+0000 7fc55195d740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559c53626490 0x559c539fc6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:00.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.407+0000 7fc55195d740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559c5362e670 0x559c539f36a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:00.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.407+0000 7fc549184640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559c53626490 0x559c539fc6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:05:00.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.408+0000 7fc549184640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559c53626490 0x559c539fc6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:60102/0 (socket says 172.21.15.42:60102) 2024-08-30T09:05:00.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.408+0000 7fc549184640 1 -- 172.21.15.42:0/617084171 learned_addr learned my addr 172.21.15.42:0/617084171 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:05:00.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.408+0000 7fc55195d740 1 --2- 172.21.15.42:0/617084171 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559c5364f350 0x559c53a0d220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:00.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.408+0000 7fc55195d740 1 -- 172.21.15.42:0/617084171 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fc5280125e0 con 0x559c5362e670 2024-08-30T09:05:00.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.408+0000 7fc55195d740 1 -- 172.21.15.42:0/617084171 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc52800feb0 con 0x559c5364f350 2024-08-30T09:05:00.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.408+0000 7fc55195d740 1 -- 172.21.15.42:0/617084171 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fc52800dd00 con 0x559c53626490 2024-08-30T09:05:00.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.408+0000 7fc549985640 1 --2- 172.21.15.42:0/617084171 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559c5364f350 0x559c53a0d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:05:00.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.408+0000 7fc548983640 1 --2- 172.21.15.42:0/617084171 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559c5362e670 0x559c539f36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:05:00.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.408+0000 7fc549985640 1 -- 172.21.15.42:0/617084171 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559c53626490 msgr2=0x559c539fc6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:05:00.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.408+0000 7fc549985640 1 --2- 172.21.15.42:0/617084171 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559c53626490 0x559c539fc6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:00.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.408+0000 7fc549985640 1 -- 172.21.15.42:0/617084171 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559c5362e670 msgr2=0x559c539f36a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:05:00.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.408+0000 7fc549985640 1 --2- 172.21.15.42:0/617084171 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559c5362e670 0x559c539f36a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:00.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.408+0000 7fc549985640 1 -- 172.21.15.42:0/617084171 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559c539ffea0 con 0x559c5364f350 2024-08-30T09:05:00.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.409+0000 7fc549985640 1 --2- 172.21.15.42:0/617084171 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559c5364f350 0x559c53a0d220 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fc540009230 tx=0x7fc540031400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=54c7c89fbd6b3bd0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:05:00.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.413+0000 7fc52effd640 1 -- 172.21.15.42:0/617084171 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc540041040 con 0x559c5364f350 2024-08-30T09:05:00.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.413+0000 7fc52effd640 1 -- 172.21.15.42:0/617084171 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc540058070 con 0x559c5364f350 2024-08-30T09:05:00.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.414+0000 7fc55195d740 1 -- 172.21.15.42:0/617084171 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559c5364f350 msgr2=0x559c53a0d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:05:00.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.414+0000 7fc55195d740 1 --2- 172.21.15.42:0/617084171 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559c5364f350 0x559c53a0d220 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fc540009230 tx=0x7fc540031400 comp rx=0 tx=0).stop 2024-08-30T09:05:00.570 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.416+0000 7fc55195d740 1 -- 172.21.15.42:0/617084171 shutdown_connections 2024-08-30T09:05:00.570 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.416+0000 7fc55195d740 1 --2- 172.21.15.42:0/617084171 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559c5364f350 0x559c53a0d220 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:00.570 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.416+0000 7fc55195d740 1 --2- 172.21.15.42:0/617084171 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559c5362e670 0x559c539f36a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:00.570 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.416+0000 7fc55195d740 1 --2- 172.21.15.42:0/617084171 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559c53626490 0x559c539fc6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:00.570 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.416+0000 7fc55195d740 1 -- 172.21.15.42:0/617084171 >> 172.21.15.42:0/617084171 conn(0x559c53643740 msgr2=0x559c539f2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:05:00.570 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.420+0000 7fc55195d740 1 -- 172.21.15.42:0/617084171 shutdown_connections 2024-08-30T09:05:00.570 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.420+0000 7fc55195d740 1 -- 172.21.15.42:0/617084171 wait complete. 2024-08-30T09:05:00.570 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.420+0000 7fc55195d740 1 -- start start 2024-08-30T09:05:00.570 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.420+0000 7fc55195d740 1 Processor -- start 2024-08-30T09:05:00.570 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.421+0000 7fc55195d740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559c53626490 0x559c53a051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:00.570 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.421+0000 7fc55195d740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559c5362e670 0x559c53a05700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:00.570 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.421+0000 7fc55195d740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559c5364f350 0x559c53a05c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:00.570 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.421+0000 7fc55195d740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fc52800e370 con 0x559c5364f350 2024-08-30T09:05:00.570 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.421+0000 7fc55195d740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc52800f7e0 con 0x559c5362e670 2024-08-30T09:05:00.570 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.421+0000 7fc55195d740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fc52800ea40 con 0x559c53626490 2024-08-30T09:05:00.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.421+0000 7fc55195d740 10 client.0 _ll_register_callbacks cb 0x7fc534004f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:05:00.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.421+0000 7fc548983640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559c5362e670 0x559c53a05700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:05:00.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.422+0000 7fc548983640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559c5362e670 0x559c53a05700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:36244/0 (socket says 172.21.15.42:36244) 2024-08-30T09:05:00.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.422+0000 7fc548983640 1 -- 172.21.15.42:0/2826685586 learned_addr learned my addr 172.21.15.42:0/2826685586 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:05:00.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.422+0000 7fc548983640 1 -- 172.21.15.42:0/2826685586 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559c53626490 msgr2=0x559c53a051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:05:00.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.422+0000 7fc548983640 1 --2- 172.21.15.42:0/2826685586 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559c53626490 0x559c53a051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:00.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.422+0000 7fc548983640 1 -- 172.21.15.42:0/2826685586 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559c5364f350 msgr2=0x559c53a05c40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-30T09:05:00.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.422+0000 7fc548983640 1 --2- 172.21.15.42:0/2826685586 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559c5364f350 0x559c53a05c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:00.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.422+0000 7fc548983640 1 -- 172.21.15.42:0/2826685586 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559c53a7d1f0 con 0x559c5362e670 2024-08-30T09:05:00.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.423+0000 7fc548983640 1 --2- 172.21.15.42:0/2826685586 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559c5362e670 0x559c53a05700 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fc54406c710 tx=0x7fc53c014b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1e694cd92cc3e438 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:05:00.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.426+0000 7fc52cff9640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-30T09:05:00.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.428+0000 7fc52cff9640 1 -- 172.21.15.42:0/2826685586 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc53c013e40 con 0x559c5362e670 2024-08-30T09:05:00.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.428+0000 7fc55195d740 1 -- 172.21.15.42:0/2826685586 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc540058070 con 0x559c5362e670 2024-08-30T09:05:00.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.428+0000 7fc52cff9640 1 -- 172.21.15.42:0/2826685586 <== mon.2 v2:172.21.15.167:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7fc53c022420 con 0x559c5362e670 2024-08-30T09:05:00.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.429+0000 7fc52cff9640 1 -- 172.21.15.42:0/2826685586 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc53c02a050 con 0x559c5362e670 2024-08-30T09:05:00.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.433+0000 7fc55195d740 10 client.25916 fetch_fsmap learned FSMap version 29 2024-08-30T09:05:00.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.433+0000 7fc55195d740 1 -- 172.21.15.42:0/2826685586 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x559c53a7e470 con 0x559c5362e670 2024-08-30T09:05:00.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.434+0000 7fc52cff9640 1 -- 172.21.15.42:0/2826685586 <== mon.2 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc540058070 con 0x559c5362e670 2024-08-30T09:05:00.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.434+0000 7fc52cff9640 1 -- 172.21.15.42:0/2826685586 <== mon.2 v2:172.21.15.167:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fc53c019810 con 0x559c5362e670 2024-08-30T09:05:00.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.434+0000 7fc55195d740 10 client.25916 fetch_fsmap finished waiting for FSMap version 29 2024-08-30T09:05:00.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.434+0000 7fc55195d740 10 client.25916 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-30T09:05:00.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.434+0000 7fc55195d740 20 client.25916 populate_metadata read hostname 'smithi042' 2024-08-30T09:05:00.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.434+0000 7fc55195d740 1 --2- 172.21.15.42:0/2826685586 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x559c535e3080 0x559c53a80d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:00.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.434+0000 7fc55195d740 4 client.25916 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-08-30T09:05:00.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.434+0000 7fc55195d740 1 -- 172.21.15.42:0/2826685586 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x559c53a01eb0 con 0x559c535e3080 2024-08-30T09:05:00.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.434+0000 7fc549184640 1 --2- 172.21.15.42:0/2826685586 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x559c535e3080 0x559c53a80d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:05:00.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.437+0000 7fc52cff9640 10 client.25916 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:05:00.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.438+0000 7fc549184640 1 --2- 172.21.15.42:0/2826685586 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x559c535e3080 0x559c53a80d30 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=209b713d77e19760 server_cookie=cdf94c4d1c1ed906 in_seq=0 out_seq=0 2024-08-30T09:05:00.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.443+0000 7fc52cff9640 1 -- 172.21.15.42:0/2826685586 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x559c53a01eb0 con 0x559c535e3080 2024-08-30T09:05:00.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.443+0000 7fc52cff9640 10 client.25916 handle_command_reply: tid=0 2024-08-30T09:05:00.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.443+0000 7fc55195d740 1 client.25916 shutdown 2024-08-30T09:05:00.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.445+0000 7fc55195d740 1 -- 172.21.15.42:0/2826685586 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559c5362e670 msgr2=0x559c53a05700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:05:00.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.445+0000 7fc55195d740 1 --2- 172.21.15.42:0/2826685586 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559c5362e670 0x559c53a05700 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fc54406c710 tx=0x7fc53c014b60 comp rx=0 tx=0).stop 2024-08-30T09:05:00.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.446+0000 7fc55195d740 1 -- 172.21.15.42:0/2826685586 shutdown_connections 2024-08-30T09:05:00.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.446+0000 7fc55195d740 1 --2- 172.21.15.42:0/2826685586 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x559c535e3080 0x559c53a80d30 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:00.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.446+0000 7fc55195d740 1 --2- 172.21.15.42:0/2826685586 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559c5364f350 0x559c53a05c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:00.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.446+0000 7fc55195d740 1 --2- 172.21.15.42:0/2826685586 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559c5362e670 0x559c53a05700 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:00.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.446+0000 7fc55195d740 1 --2- 172.21.15.42:0/2826685586 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559c53626490 0x559c53a051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:00.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.446+0000 7fc55195d740 1 -- 172.21.15.42:0/2826685586 >> 172.21.15.42:0/2826685586 conn(0x559c53643740 msgr2=0x559c539f8880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:05:00.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.446+0000 7fc52cff9640 0 client.25916 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:05:00.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.447+0000 7fc55195d740 1 -- 172.21.15.42:0/2826685586 shutdown_connections 2024-08-30T09:05:00.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.449+0000 7fc55195d740 1 -- 172.21.15.42:0/2826685586 wait complete. 2024-08-30T09:05:00.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.449+0000 7fc55195d740 20 client.25916 trim_cache size 0 max 16384 2024-08-30T09:05:00.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.450+0000 7fc54b3e8640 1 -- 172.21.15.42:0/1846509886 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fc5300b4d40 msgr2=0x7fc5300b7250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:05:00.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.450+0000 7fc54b3e8640 1 --2- 172.21.15.42:0/1846509886 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fc5300b4d40 0x7fc5300b7250 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fc540009850 tx=0x7fc540004180 comp rx=0 tx=0).stop 2024-08-30T09:05:00.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.450+0000 7fc54b3e8640 1 -- 172.21.15.42:0/1846509886 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc54410a780 msgr2=0x7fc5441a9030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:05:00.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.450+0000 7fc54b3e8640 1 --2- 172.21.15.42:0/1846509886 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc54410a780 0x7fc5441a9030 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fc53400c890 tx=0x7fc53400cd50 comp rx=0 tx=0).stop 2024-08-30T09:05:00.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.450+0000 7fc54b3e8640 1 -- 172.21.15.42:0/1846509886 shutdown_connections 2024-08-30T09:05:00.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.450+0000 7fc54b3e8640 1 --2- 172.21.15.42:0/1846509886 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fc5300b4d40 0x7fc5300b7250 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:00.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.450+0000 7fc54b3e8640 1 --2- 172.21.15.42:0/1846509886 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc54411c9d0 0x7fc5441ae870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:00.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.450+0000 7fc54b3e8640 1 --2- 172.21.15.42:0/1846509886 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc544111ce0 0x7fc5441a95d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:00.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.450+0000 7fc54b3e8640 1 --2- 172.21.15.42:0/1846509886 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc54410a780 0x7fc5441a9030 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:00.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.450+0000 7fc54b3e8640 1 -- 172.21.15.42:0/1846509886 >> 172.21.15.42:0/1846509886 conn(0x7fc544070c60 msgr2=0x7fc54411cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:05:00.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.450+0000 7fc54b3e8640 1 -- 172.21.15.42:0/1846509886 shutdown_connections 2024-08-30T09:05:00.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:00.450+0000 7fc54b3e8640 1 -- 172.21.15.42:0/1846509886 wait complete. 2024-08-30T09:05:00.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:05:00 smithi190 ceph-mon[22001]: pgmap v500: 129 pgs: 129 active+clean; 1.1 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 5.2 MiB/s wr, 598 op/s 2024-08-30T09:05:00.949 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:05:00 smithi042 ceph-mon[18652]: pgmap v500: 129 pgs: 129 active+clean; 1.1 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 5.2 MiB/s wr, 598 op/s 2024-08-30T09:05:00.953 INFO:teuthology.orchestra.run.smithi042.stderr:progress: 255.0 s, 802.8 tps, lat 122.888 ms stddev 139.885 2024-08-30T09:05:01.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:05:00 smithi167 ceph-mon[22063]: pgmap v500: 129 pgs: 129 active+clean; 1.1 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 5.2 MiB/s wr, 598 op/s 2024-08-30T09:05:01.710 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-30T09:05:01.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:05:01 smithi190 ceph-mon[22001]: scrub queued for path: / 2024-08-30T09:05:01.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:05:01 smithi190 ceph-mon[22001]: scrub summary: idle+waiting paths [/] 2024-08-30T09:05:01.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:05:01 smithi190 ceph-mon[22001]: scrub summary: active paths [/] 2024-08-30T09:05:01.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:05:01 smithi190 ceph-mon[22001]: scrub summary: active paths [/] 2024-08-30T09:05:01.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:05:01 smithi190 ceph-mon[22001]: scrub summary: active paths [/] 2024-08-30T09:05:02.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:05:01 smithi167 ceph-mon[22063]: scrub queued for path: / 2024-08-30T09:05:02.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:05:01 smithi167 ceph-mon[22063]: scrub summary: idle+waiting paths [/] 2024-08-30T09:05:02.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:05:01 smithi167 ceph-mon[22063]: scrub summary: active paths [/] 2024-08-30T09:05:02.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:05:01 smithi167 ceph-mon[22063]: scrub summary: active paths [/] 2024-08-30T09:05:02.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:05:01 smithi167 ceph-mon[22063]: scrub summary: active paths [/] 2024-08-30T09:05:02.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:05:01 smithi042 ceph-mon[18652]: scrub queued for path: / 2024-08-30T09:05:02.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:05:01 smithi042 ceph-mon[18652]: scrub summary: idle+waiting paths [/] 2024-08-30T09:05:02.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:05:01 smithi042 ceph-mon[18652]: scrub summary: active paths [/] 2024-08-30T09:05:02.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:05:01 smithi042 ceph-mon[18652]: scrub summary: active paths [/] 2024-08-30T09:05:02.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:05:01 smithi042 ceph-mon[18652]: scrub summary: active paths [/] 2024-08-30T09:05:02.180 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T09:05:02.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:05:02 smithi190 ceph-mon[22001]: pgmap v501: 129 pgs: 129 active+clean; 1.1 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 4.9 MiB/s wr, 545 op/s 2024-08-30T09:05:02.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:05:02 smithi190 ceph-mon[22001]: scrub summary: idle+waiting paths [/] 2024-08-30T09:05:02.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:05:02 smithi190 ceph-mon[22001]: scrub summary: idle+waiting paths [/] 2024-08-30T09:05:02.959 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:05:02 smithi042 ceph-mon[18652]: pgmap v501: 129 pgs: 129 active+clean; 1.1 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 4.9 MiB/s wr, 545 op/s 2024-08-30T09:05:02.960 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:05:02 smithi042 ceph-mon[18652]: scrub summary: idle+waiting paths [/] 2024-08-30T09:05:02.960 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:05:02 smithi042 ceph-mon[18652]: scrub summary: idle+waiting paths [/] 2024-08-30T09:05:02.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:02.990+0000 7f72f3706640 1 -- 172.21.15.42:0/2353961796 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f72ec103960 msgr2=0x7f72ec103d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:05:02.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:02.990+0000 7f72f3706640 1 --2- 172.21.15.42:0/2353961796 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f72ec103960 0x7f72ec103d70 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f72dc009900 tx=0x7f72dc02a410 comp rx=0 tx=0).stop 2024-08-30T09:05:02.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:02.990+0000 7f72f3706640 1 -- 172.21.15.42:0/2353961796 shutdown_connections 2024-08-30T09:05:02.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:02.990+0000 7f72f3706640 1 --2- 172.21.15.42:0/2353961796 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f72ec109250 0x7f72ec109660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:02.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:02.990+0000 7f72f3706640 1 --2- 172.21.15.42:0/2353961796 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f72ec109ba0 0x7f72ec1103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:02.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:02.990+0000 7f72f3706640 1 --2- 172.21.15.42:0/2353961796 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f72ec103960 0x7f72ec103d70 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:02.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:02.990+0000 7f72f3706640 1 -- 172.21.15.42:0/2353961796 >> 172.21.15.42:0/2353961796 conn(0x7f72ec07b6b0 msgr2=0x7f72ec07bac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:05:02.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:02.990+0000 7f72f3706640 1 -- 172.21.15.42:0/2353961796 shutdown_connections 2024-08-30T09:05:02.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:02.990+0000 7f72f3706640 1 -- 172.21.15.42:0/2353961796 wait complete. 2024-08-30T09:05:02.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:02.991+0000 7f72f3706640 1 Processor -- start 2024-08-30T09:05:02.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:02.991+0000 7f72f3706640 1 -- start start 2024-08-30T09:05:02.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:02.991+0000 7f72f3706640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f72ec103960 0x7f72ec19a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:02.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:02.991+0000 7f72f3706640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f72ec109250 0x7f72ec19a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:02.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:02.991+0000 7f72f3706640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f72ec109ba0 0x7f72ec19fab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:02.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:02.991+0000 7f72f3706640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f72ec113860 con 0x7f72ec103960 2024-08-30T09:05:02.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:02.991+0000 7f72f3706640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f72ec1136e0 con 0x7f72ec109250 2024-08-30T09:05:02.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:02.991+0000 7f72f3706640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f72ec1139e0 con 0x7f72ec109ba0 2024-08-30T09:05:02.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:02.992+0000 7f72f14a2640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f72ec103960 0x7f72ec19a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:05:02.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:02.992+0000 7f72f1ca3640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f72ec109ba0 0x7f72ec19fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:05:02.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:02.992+0000 7f72f14a2640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f72ec103960 0x7f72ec19a270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:40100/0 (socket says 172.21.15.42:40100) 2024-08-30T09:05:02.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:02.992+0000 7f72f1ca3640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f72ec109ba0 0x7f72ec19fab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:60122/0 (socket says 172.21.15.42:60122) 2024-08-30T09:05:02.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:02.992+0000 7f72f14a2640 1 -- 172.21.15.42:0/3699909391 learned_addr learned my addr 172.21.15.42:0/3699909391 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:05:02.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:02.992+0000 7f72f0ca1640 1 --2- 172.21.15.42:0/3699909391 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f72ec109250 0x7f72ec19a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:05:02.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:02.992+0000 7f72f14a2640 1 -- 172.21.15.42:0/3699909391 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f72ec109ba0 msgr2=0x7f72ec19fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:05:02.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:02.992+0000 7f72f14a2640 1 --2- 172.21.15.42:0/3699909391 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f72ec109ba0 0x7f72ec19fab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:02.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:02.992+0000 7f72f14a2640 1 -- 172.21.15.42:0/3699909391 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f72ec109250 msgr2=0x7f72ec19a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:05:02.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:02.992+0000 7f72f14a2640 1 --2- 172.21.15.42:0/3699909391 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f72ec109250 0x7f72ec19a810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:02.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:02.992+0000 7f72f14a2640 1 -- 172.21.15.42:0/3699909391 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72ec1a01e0 con 0x7f72ec103960 2024-08-30T09:05:02.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:02.993+0000 7f72f14a2640 1 --2- 172.21.15.42:0/3699909391 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f72ec103960 0x7f72ec19a270 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f72dc005940 tx=0x7f72dc0040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:05:02.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:02.993+0000 7f72f0ca1640 1 --2- 172.21.15.42:0/3699909391 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f72ec109250 0x7f72ec19a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-30T09:05:02.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:02.994+0000 7f72e27fc640 1 -- 172.21.15.42:0/3699909391 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72dc03f030 con 0x7f72ec103960 2024-08-30T09:05:02.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:02.994+0000 7f72e27fc640 1 -- 172.21.15.42:0/3699909391 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f72dc0044c0 con 0x7f72ec103960 2024-08-30T09:05:02.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:02.994+0000 7f72f3706640 1 -- 172.21.15.42:0/3699909391 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72ec1a0470 con 0x7f72ec103960 2024-08-30T09:05:02.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:02.994+0000 7f72f3706640 1 -- 172.21.15.42:0/3699909391 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72ec075390 con 0x7f72ec103960 2024-08-30T09:05:02.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:02.995+0000 7f72e27fc640 1 -- 172.21.15.42:0/3699909391 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72dc035070 con 0x7f72ec103960 2024-08-30T09:05:03.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:02.997+0000 7f72f9c7b740 1 Processor -- start 2024-08-30T09:05:03.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:02.997+0000 7f72e27fc640 1 -- 172.21.15.42:0/3699909391 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7f72dc042030 con 0x7f72ec103960 2024-08-30T09:05:03.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:02.997+0000 7f72f9c7b740 1 -- start start 2024-08-30T09:05:03.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:02.998+0000 7f72f9c7b740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557598dcd070 0x5575991722c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:03.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:02.998+0000 7f72f9c7b740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557598da41b0 0x557599172800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:03.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:02.998+0000 7f72f9c7b740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557598dac390 0x557599172d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:03.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:02.998+0000 7f72f9c7b740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x5575990e9630 con 0x557598dcd070 2024-08-30T09:05:03.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:02.998+0000 7f72f9c7b740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5575990d8bc0 con 0x557598da41b0 2024-08-30T09:05:03.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:02.998+0000 7f72f9c7b740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5575990dcb70 con 0x557598dac390 2024-08-30T09:05:03.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:02.998+0000 7f72f1ca3640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557598dcd070 0x5575991722c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:05:03.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:02.998+0000 7f72f0ca1640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557598dac390 0x557599172d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:05:03.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:02.998+0000 7f72f14a2640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557598da41b0 0x557599172800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:05:03.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:02.998+0000 7f72f0ca1640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557598dac390 0x557599172d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:60134/0 (socket says 172.21.15.42:60134) 2024-08-30T09:05:03.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:02.998+0000 7f72f0ca1640 1 -- 172.21.15.42:0/3340653869 learned_addr learned my addr 172.21.15.42:0/3340653869 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:05:03.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:02.998+0000 7f72f0ca1640 1 -- 172.21.15.42:0/3340653869 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557598da41b0 msgr2=0x557599172800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:05:03.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:02.998+0000 7f72f0ca1640 1 --2- 172.21.15.42:0/3340653869 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557598da41b0 0x557599172800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:03.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:02.998+0000 7f72f0ca1640 1 -- 172.21.15.42:0/3340653869 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557598dcd070 msgr2=0x5575991722c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:05:03.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:02.998+0000 7f72f0ca1640 1 --2- 172.21.15.42:0/3340653869 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557598dcd070 0x5575991722c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:03.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:02.998+0000 7f72f0ca1640 1 -- 172.21.15.42:0/3340653869 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557598e38650 con 0x557598dac390 2024-08-30T09:05:03.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:02.998+0000 7f72f14a2640 1 --2- 172.21.15.42:0/3340653869 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557598da41b0 0x557599172800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-30T09:05:03.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:02.998+0000 7f72e27fc640 1 --2- 172.21.15.42:0/3699909391 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f72d80b4bb0 0x7f72d80b70c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:03.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:02.999+0000 7f72f1ca3640 1 --2- 172.21.15.42:0/3699909391 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f72d80b4bb0 0x7f72d80b70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:05:03.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:02.999+0000 7f72e27fc640 1 -- 172.21.15.42:0/3699909391 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f72dc1055b0 con 0x7f72ec103960 2024-08-30T09:05:03.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:02.999+0000 7f72f0ca1640 1 --2- 172.21.15.42:0/3340653869 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557598dac390 0x557599172d40 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f72e8011470 tx=0x7f72e8011930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4028fa3383e1f5cc server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:05:03.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:02.999+0000 7f72d77fe640 1 -- 172.21.15.42:0/3340653869 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72e801e070 con 0x557598dac390 2024-08-30T09:05:03.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:02.999+0000 7f72f1ca3640 1 --2- 172.21.15.42:0/3699909391 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f72d80b4bb0 0x7f72d80b70c0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f72e4005fd0 tx=0x7f72e4012040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:05:03.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:02.999+0000 7f72d77fe640 1 -- 172.21.15.42:0/3340653869 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f72e8004310 con 0x557598dac390 2024-08-30T09:05:03.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:02.999+0000 7f72d77fe640 1 -- 172.21.15.42:0/3340653869 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72e800f9a0 con 0x557598dac390 2024-08-30T09:05:03.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:03.000+0000 7f72f9c7b740 1 -- 172.21.15.42:0/3340653869 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557598dac390 msgr2=0x557599172d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:05:03.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:03.000+0000 7f72f9c7b740 1 --2- 172.21.15.42:0/3340653869 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557598dac390 0x557599172d40 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f72e8011470 tx=0x7f72e8011930 comp rx=0 tx=0).stop 2024-08-30T09:05:03.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:03.000+0000 7f72f9c7b740 1 -- 172.21.15.42:0/3340653869 shutdown_connections 2024-08-30T09:05:03.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:03.000+0000 7f72f9c7b740 1 --2- 172.21.15.42:0/3340653869 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557598dac390 0x557599172d40 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:03.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:03.000+0000 7f72f9c7b740 1 --2- 172.21.15.42:0/3340653869 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557598da41b0 0x557599172800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:03.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:03.000+0000 7f72f9c7b740 1 --2- 172.21.15.42:0/3340653869 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557598dcd070 0x5575991722c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:03.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:03.000+0000 7f72f9c7b740 1 -- 172.21.15.42:0/3340653869 >> 172.21.15.42:0/3340653869 conn(0x557598dc1460 msgr2=0x5575991701b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:05:03.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:03.000+0000 7f72f9c7b740 1 -- 172.21.15.42:0/3340653869 shutdown_connections 2024-08-30T09:05:03.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:03.000+0000 7f72f9c7b740 1 -- 172.21.15.42:0/3340653869 wait complete. 2024-08-30T09:05:03.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:03.001+0000 7f72f9c7b740 1 -- start start 2024-08-30T09:05:03.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:03.001+0000 7f72f9c7b740 1 Processor -- start 2024-08-30T09:05:03.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:03.001+0000 7f72f9c7b740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557598da41b0 0x55759917d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:03.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:03.001+0000 7f72f9c7b740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557598dac390 0x55759917dee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:03.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:03.002+0000 7f72f9c7b740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557598dcd070 0x557599189e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:03.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:03.002+0000 7f72f9c7b740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x557598f3b700 con 0x557598dcd070 2024-08-30T09:05:03.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:03.002+0000 7f72f9c7b740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55759905f700 con 0x557598dac390 2024-08-30T09:05:03.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:03.002+0000 7f72f9c7b740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x557599180af0 con 0x557598da41b0 2024-08-30T09:05:03.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:03.002+0000 7f72f14a2640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557598da41b0 0x55759917d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:05:03.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:03.002+0000 7f72f9c7b740 10 client.0 _ll_register_callbacks cb 0x55759908edc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:05:03.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:03.002+0000 7f72f1ca3640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557598dcd070 0x557599189e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:05:03.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:03.002+0000 7f72f1ca3640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557598dcd070 0x557599189e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:40116/0 (socket says 172.21.15.42:40116) 2024-08-30T09:05:03.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:03.002+0000 7f72f14a2640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557598da41b0 0x55759917d9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:60142/0 (socket says 172.21.15.42:60142) 2024-08-30T09:05:03.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:03.002+0000 7f72f1ca3640 1 -- 172.21.15.42:0/3781747396 learned_addr learned my addr 172.21.15.42:0/3781747396 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:05:03.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:03.002+0000 7f72f0ca1640 1 --2- 172.21.15.42:0/3781747396 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557598dac390 0x55759917dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:05:03.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:03.002+0000 7f72f0ca1640 1 -- 172.21.15.42:0/3781747396 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557598da41b0 msgr2=0x55759917d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:05:03.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:03.002+0000 7f72f0ca1640 1 --2- 172.21.15.42:0/3781747396 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557598da41b0 0x55759917d9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:03.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:03.002+0000 7f72f0ca1640 1 -- 172.21.15.42:0/3781747396 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557598dcd070 msgr2=0x557599189e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:05:03.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:03.002+0000 7f72f0ca1640 1 --2- 172.21.15.42:0/3781747396 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557598dcd070 0x557599189e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:03.012 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:03.002+0000 7f72f0ca1640 1 -- 172.21.15.42:0/3781747396 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5575990988b0 con 0x557598dac390 2024-08-30T09:05:03.012 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:03.002+0000 7f72f14a2640 1 --2- 172.21.15.42:0/3781747396 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557598da41b0 0x55759917d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-30T09:05:03.012 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:03.002+0000 7f72f1ca3640 1 --2- 172.21.15.42:0/3781747396 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557598dcd070 0x557599189e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-30T09:05:03.012 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:03.002+0000 7f72f0ca1640 1 --2- 172.21.15.42:0/3781747396 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557598dac390 0x55759917dee0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f72e8011440 tx=0x7f72e8003c90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fddf902ac969426 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:05:03.012 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:03.002+0000 7f72d4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-30T09:05:03.012 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:03.003+0000 7f72d4ff9640 1 -- 172.21.15.42:0/3781747396 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72e801e050 con 0x557598dac390 2024-08-30T09:05:03.012 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:03.003+0000 7f72d4ff9640 1 -- 172.21.15.42:0/3781747396 <== mon.2 v2:172.21.15.167:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f72e8005250 con 0x557598dac390 2024-08-30T09:05:03.012 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:03.003+0000 7f72f9c7b740 1 -- 172.21.15.42:0/3781747396 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f72e8004310 con 0x557598dac390 2024-08-30T09:05:03.012 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:03.003+0000 7f72d4ff9640 1 -- 172.21.15.42:0/3781747396 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f72e8033050 con 0x557598dac390 2024-08-30T09:05:03.012 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:03.003+0000 7f72f9c7b740 10 client.25928 fetch_fsmap learned FSMap version 30 2024-08-30T09:05:03.012 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:03.003+0000 7f72f9c7b740 1 -- 172.21.15.42:0/3781747396 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55759918c850 con 0x557598dac390 2024-08-30T09:05:03.012 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:03.004+0000 7f72d4ff9640 1 -- 172.21.15.42:0/3781747396 <== mon.2 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f72e802a070 con 0x557598dac390 2024-08-30T09:05:03.012 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:03.004+0000 7f72d4ff9640 1 -- 172.21.15.42:0/3781747396 <== mon.2 v2:172.21.15.167:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7f72e800f6e0 con 0x557598dac390 2024-08-30T09:05:03.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:03.004+0000 7f72f9c7b740 10 client.25928 fetch_fsmap finished waiting for FSMap version 30 2024-08-30T09:05:03.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:03.004+0000 7f72f9c7b740 10 client.25928 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-30T09:05:03.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:03.004+0000 7f72f9c7b740 20 client.25928 populate_metadata read hostname 'smithi042' 2024-08-30T09:05:03.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:03.004+0000 7f72f9c7b740 1 --2- 172.21.15.42:0/3781747396 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x557598d8b9f0 0x55759918fc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:03.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:03.004+0000 7f72f9c7b740 4 client.25928 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-30T09:05:03.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:03.004+0000 7f72f9c7b740 1 -- 172.21.15.42:0/3781747396 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f72e8004310 con 0x557598d8b9f0 2024-08-30T09:05:03.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:03.004+0000 7f72f14a2640 1 --2- 172.21.15.42:0/3781747396 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x557598d8b9f0 0x55759918fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:05:03.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:03.005+0000 7f72d4ff9640 10 client.25928 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:05:03.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:03.005+0000 7f72f14a2640 1 --2- 172.21.15.42:0/3781747396 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x557598d8b9f0 0x55759918fc60 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=839d5c0891974681 server_cookie=c1d95349be740713 in_seq=0 out_seq=0 2024-08-30T09:05:03.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:03.006+0000 7f72d4ff9640 1 -- 172.21.15.42:0/3781747396 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f72e8004310 con 0x557598d8b9f0 2024-08-30T09:05:03.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:03.006+0000 7f72d4ff9640 10 client.25928 handle_command_reply: tid=0 2024-08-30T09:05:03.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:03.006+0000 7f72f9c7b740 1 client.25928 shutdown 2024-08-30T09:05:03.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:03.006+0000 7f72f9c7b740 1 -- 172.21.15.42:0/3781747396 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557598dac390 msgr2=0x55759917dee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:05:03.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:03.006+0000 7f72f9c7b740 1 --2- 172.21.15.42:0/3781747396 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557598dac390 0x55759917dee0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f72e8011440 tx=0x7f72e8003c90 comp rx=0 tx=0).stop 2024-08-30T09:05:03.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:03.006+0000 7f72f9c7b740 1 -- 172.21.15.42:0/3781747396 shutdown_connections 2024-08-30T09:05:03.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:03.006+0000 7f72f9c7b740 1 --2- 172.21.15.42:0/3781747396 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x557598d8b9f0 0x55759918fc60 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:03.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:03.006+0000 7f72f9c7b740 1 --2- 172.21.15.42:0/3781747396 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557598dcd070 0x557599189e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:03.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:03.006+0000 7f72f9c7b740 1 --2- 172.21.15.42:0/3781747396 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557598dac390 0x55759917dee0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:03.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:03.006+0000 7f72d4ff9640 0 client.25928 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:05:03.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:03.006+0000 7f72f9c7b740 1 --2- 172.21.15.42:0/3781747396 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557598da41b0 0x55759917d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:03.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:03.006+0000 7f72f9c7b740 1 -- 172.21.15.42:0/3781747396 >> 172.21.15.42:0/3781747396 conn(0x557598dc1460 msgr2=0x557599175450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:05:03.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:03.007+0000 7f72f9c7b740 1 -- 172.21.15.42:0/3781747396 shutdown_connections 2024-08-30T09:05:03.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:03.007+0000 7f72f9c7b740 1 -- 172.21.15.42:0/3781747396 wait complete. 2024-08-30T09:05:03.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:03.007+0000 7f72f9c7b740 20 client.25928 trim_cache size 0 max 16384 2024-08-30T09:05:03.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:03.038+0000 7f72f9c7b740 1 Processor -- start 2024-08-30T09:05:03.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:03.039+0000 7f72f9c7b740 1 -- start start 2024-08-30T09:05:03.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:03.039+0000 7f72f9c7b740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557598d8b9f0 0x557599173cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:03.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:03.039+0000 7f72f9c7b740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557598da41b0 0x55759917f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:03.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:03.039+0000 7f72f9c7b740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557598dac390 0x55759916f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:03.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:03.039+0000 7f72f9c7b740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f72d0012450 con 0x557598dac390 2024-08-30T09:05:03.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:03.039+0000 7f72f9c7b740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f72d000fd20 con 0x557598da41b0 2024-08-30T09:05:03.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:03.039+0000 7f72f9c7b740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f72d000db70 con 0x557598d8b9f0 2024-08-30T09:05:03.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:03.040+0000 7f72f0ca1640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557598da41b0 0x55759917f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:05:03.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:03.040+0000 7f72f0ca1640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557598da41b0 0x55759917f5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:36300/0 (socket says 172.21.15.42:36300) 2024-08-30T09:05:03.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:03.040+0000 7f72f0ca1640 1 -- 172.21.15.42:0/2798491958 learned_addr learned my addr 172.21.15.42:0/2798491958 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:05:03.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:03.039+0000 7f72f1ca3640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557598dac390 0x55759916f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:05:03.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:03.042+0000 7f72f1ca3640 1 -- 172.21.15.42:0/2798491958 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557598d8b9f0 msgr2=0x557599173cb0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-30T09:05:03.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:03.042+0000 7f72f1ca3640 1 --2- 172.21.15.42:0/2798491958 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557598d8b9f0 0x557599173cb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:03.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:03.042+0000 7f72f1ca3640 1 -- 172.21.15.42:0/2798491958 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557598da41b0 msgr2=0x55759917f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:05:03.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:03.042+0000 7f72f1ca3640 1 --2- 172.21.15.42:0/2798491958 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557598da41b0 0x55759917f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:03.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:03.042+0000 7f72f1ca3640 1 -- 172.21.15.42:0/2798491958 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55759917bea0 con 0x557598dac390 2024-08-30T09:05:03.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:03.042+0000 7f72f1ca3640 1 --2- 172.21.15.42:0/2798491958 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557598dac390 0x55759916f780 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f72e400bcc0 tx=0x7f72e401a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ef70c755ecc2911 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:05:03.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:03.043+0000 7f72d6ffd640 1 -- 172.21.15.42:0/2798491958 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72e403f070 con 0x557598dac390 2024-08-30T09:05:03.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:03.043+0000 7f72d6ffd640 1 -- 172.21.15.42:0/2798491958 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f72e4024070 con 0x557598dac390 2024-08-30T09:05:03.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:03.043+0000 7f72d6ffd640 1 -- 172.21.15.42:0/2798491958 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72e40042b0 con 0x557598dac390 2024-08-30T09:05:03.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:03.045+0000 7f72f9c7b740 1 -- 172.21.15.42:0/2798491958 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557598dac390 msgr2=0x55759916f780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:05:03.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:03.045+0000 7f72f9c7b740 1 --2- 172.21.15.42:0/2798491958 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557598dac390 0x55759916f780 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f72e400bcc0 tx=0x7f72e401a000 comp rx=0 tx=0).stop 2024-08-30T09:05:03.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:03.051+0000 7f72f9c7b740 1 -- 172.21.15.42:0/2798491958 shutdown_connections 2024-08-30T09:05:03.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:03.051+0000 7f72f9c7b740 1 --2- 172.21.15.42:0/2798491958 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557598dac390 0x55759916f780 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:03.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:03.051+0000 7f72f9c7b740 1 --2- 172.21.15.42:0/2798491958 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557598da41b0 0x55759917f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:03.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:03.051+0000 7f72f9c7b740 1 --2- 172.21.15.42:0/2798491958 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557598d8b9f0 0x557599173cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:03.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:03.051+0000 7f72f9c7b740 1 -- 172.21.15.42:0/2798491958 >> 172.21.15.42:0/2798491958 conn(0x557598dc1460 msgr2=0x55759917eec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:05:03.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:03.056+0000 7f72f9c7b740 1 -- 172.21.15.42:0/2798491958 shutdown_connections 2024-08-30T09:05:03.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:03.060+0000 7f72f9c7b740 1 -- 172.21.15.42:0/2798491958 wait complete. 2024-08-30T09:05:03.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:03.060+0000 7f72f9c7b740 1 -- start start 2024-08-30T09:05:03.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:03.061+0000 7f72f9c7b740 1 Processor -- start 2024-08-30T09:05:03.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:03.062+0000 7f72f9c7b740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557598d8b9f0 0x557599181fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:03.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:03.062+0000 7f72f9c7b740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557598da41b0 0x5575991824e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:03.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:03.062+0000 7f72f9c7b740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557598dac390 0x557599182a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:03.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:03.062+0000 7f72f9c7b740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f72d000e1e0 con 0x557598dac390 2024-08-30T09:05:03.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:03.062+0000 7f72f9c7b740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f72d000f650 con 0x557598d8b9f0 2024-08-30T09:05:03.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:03.062+0000 7f72f9c7b740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f72d000e8b0 con 0x557598da41b0 2024-08-30T09:05:03.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:03.062+0000 7f72f9c7b740 10 client.0 _ll_register_callbacks cb 0x7f72e801c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:05:03.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:03.063+0000 7f72f1ca3640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557598dac390 0x557599182a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:05:03.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:03.063+0000 7f72f1ca3640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557598dac390 0x557599182a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:40136/0 (socket says 172.21.15.42:40136) 2024-08-30T09:05:03.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:03.063+0000 7f72f1ca3640 1 -- 172.21.15.42:0/682864221 learned_addr learned my addr 172.21.15.42:0/682864221 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:05:03.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:03.064+0000 7f72f1ca3640 1 -- 172.21.15.42:0/682864221 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557598da41b0 msgr2=0x5575991824e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:05:03.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:03.064+0000 7f72f1ca3640 1 --2- 172.21.15.42:0/682864221 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557598da41b0 0x5575991824e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:03.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:03.064+0000 7f72f1ca3640 1 -- 172.21.15.42:0/682864221 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557598d8b9f0 msgr2=0x557599181fa0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-30T09:05:03.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:03.064+0000 7f72f1ca3640 1 --2- 172.21.15.42:0/682864221 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557598d8b9f0 0x557599181fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:03.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:03.064+0000 7f72f1ca3640 1 -- 172.21.15.42:0/682864221 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557599201f90 con 0x557598dac390 2024-08-30T09:05:03.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:03.064+0000 7f72f1ca3640 1 --2- 172.21.15.42:0/682864221 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557598dac390 0x557599182a20 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f72e4003f80 tx=0x7f72e4003bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb4556849d54e553 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:05:03.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:03.064+0000 7f72d4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-08-30T09:05:03.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:03.067+0000 7f72d4ff9640 1 -- 172.21.15.42:0/682864221 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72e403f070 con 0x557598dac390 2024-08-30T09:05:03.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:03.067+0000 7f72f9c7b740 1 -- 172.21.15.42:0/682864221 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f72e4024070 con 0x557598dac390 2024-08-30T09:05:03.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:03.067+0000 7f72d4ff9640 1 -- 172.21.15.42:0/682864221 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f72e401cbc0 con 0x557598dac390 2024-08-30T09:05:03.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:03.068+0000 7f72d4ff9640 1 -- 172.21.15.42:0/682864221 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f72e400be70 con 0x557598dac390 2024-08-30T09:05:03.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:03.074+0000 7f72f9c7b740 10 client.16209 fetch_fsmap learned FSMap version 30 2024-08-30T09:05:03.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:03.074+0000 7f72f9c7b740 1 -- 172.21.15.42:0/682864221 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x557599203220 con 0x557598dac390 2024-08-30T09:05:03.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:05:02 smithi167 ceph-mon[22063]: pgmap v501: 129 pgs: 129 active+clean; 1.1 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 4.9 MiB/s wr, 545 op/s 2024-08-30T09:05:03.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:05:02 smithi167 ceph-mon[22063]: scrub summary: idle+waiting paths [/] 2024-08-30T09:05:03.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:05:02 smithi167 ceph-mon[22063]: scrub summary: idle+waiting paths [/] 2024-08-30T09:05:03.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:03.075+0000 7f72d4ff9640 1 -- 172.21.15.42:0/682864221 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f72e4024070 con 0x557598dac390 2024-08-30T09:05:03.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:03.075+0000 7f72d4ff9640 1 -- 172.21.15.42:0/682864221 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7f72e40050c0 con 0x557598dac390 2024-08-30T09:05:03.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:03.075+0000 7f72f9c7b740 10 client.16209 fetch_fsmap finished waiting for FSMap version 30 2024-08-30T09:05:03.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:03.075+0000 7f72f9c7b740 10 client.16209 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-30T09:05:03.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:03.075+0000 7f72f9c7b740 20 client.16209 populate_metadata read hostname 'smithi042' 2024-08-30T09:05:03.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:03.075+0000 7f72f9c7b740 1 --2- 172.21.15.42:0/682864221 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x557598dcd070 0x5575992058b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:03.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:03.075+0000 7f72f9c7b740 4 client.16209 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-30T09:05:03.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:03.075+0000 7f72f9c7b740 1 -- 172.21.15.42:0/682864221 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5575990eb580 con 0x557598dcd070 2024-08-30T09:05:03.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:03.077+0000 7f72f14a2640 1 --2- 172.21.15.42:0/682864221 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x557598dcd070 0x5575992058b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:05:03.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:03.087+0000 7f72f14a2640 1 --2- 172.21.15.42:0/682864221 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x557598dcd070 0x5575992058b0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a41f7e618ee61c8 server_cookie=a4c7047af8d4f2c4 in_seq=0 out_seq=0 2024-08-30T09:05:03.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:03.087+0000 7f72d4ff9640 10 client.16209 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:05:03.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:03.090+0000 7f72d4ff9640 1 -- 172.21.15.42:0/682864221 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+1990 (crc 0 0 0) 0x5575990eb580 con 0x557598dcd070 2024-08-30T09:05:03.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:03.090+0000 7f72d4ff9640 10 client.16209 handle_command_reply: tid=0 2024-08-30T09:05:03.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:03.090+0000 7f72f9c7b740 1 client.16209 shutdown 2024-08-30T09:05:03.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:03.095+0000 7f72f9c7b740 1 -- 172.21.15.42:0/682864221 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557598dac390 msgr2=0x557599182a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:05:03.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:03.095+0000 7f72f9c7b740 1 --2- 172.21.15.42:0/682864221 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557598dac390 0x557599182a20 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f72e4003f80 tx=0x7f72e4003bd0 comp rx=0 tx=0).stop 2024-08-30T09:05:03.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:03.097+0000 7f72f9c7b740 1 -- 172.21.15.42:0/682864221 shutdown_connections 2024-08-30T09:05:03.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:03.097+0000 7f72f9c7b740 1 --2- 172.21.15.42:0/682864221 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x557598dcd070 0x5575992058b0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:03.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:03.097+0000 7f72f9c7b740 1 --2- 172.21.15.42:0/682864221 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557598da41b0 0x5575991824e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:03.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:03.097+0000 7f72f9c7b740 1 --2- 172.21.15.42:0/682864221 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557598dac390 0x557599182a20 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:03.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:03.097+0000 7f72f9c7b740 1 --2- 172.21.15.42:0/682864221 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557598d8b9f0 0x557599181fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:03.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:03.097+0000 7f72f9c7b740 1 -- 172.21.15.42:0/682864221 >> 172.21.15.42:0/682864221 conn(0x557598dc1460 msgr2=0x557599173a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:05:03.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:03.097+0000 7f72d4ff9640 0 client.16209 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:05:03.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:03.097+0000 7f72f9c7b740 1 -- 172.21.15.42:0/682864221 shutdown_connections 2024-08-30T09:05:03.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:03.097+0000 7f72f9c7b740 1 -- 172.21.15.42:0/682864221 wait complete. 2024-08-30T09:05:03.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:03.098+0000 7f72f9c7b740 20 client.16209 trim_cache size 0 max 16384 2024-08-30T09:05:03.102 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-08-30T09:05:03.102 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "IDLE (waiting for more scrubs)", 2024-08-30T09:05:03.102 INFO:teuthology.orchestra.run.smithi042.stdout: "scrubs": { 2024-08-30T09:05:03.102 INFO:teuthology.orchestra.run.smithi042.stdout: "347d05ed-4bd3-4590-b099-314f719aab39": { 2024-08-30T09:05:03.102 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:05:03.102 INFO:teuthology.orchestra.run.smithi042.stdout: "tag": "347d05ed-4bd3-4590-b099-314f719aab39", 2024-08-30T09:05:03.102 INFO:teuthology.orchestra.run.smithi042.stdout: "options": "recursive,force" 2024-08-30T09:05:03.102 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:05:03.102 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:05:03.102 INFO:teuthology.orchestra.run.smithi042.stdout: "scrub_stats": { 2024-08-30T09:05:03.102 INFO:teuthology.orchestra.run.smithi042.stdout: "347d05ed-4bd3-4590-b099-314f719aab39": { 2024-08-30T09:05:03.102 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:05:00.439721+0000", 2024-08-30T09:05:03.102 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:05:03.103 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:05:03.103 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:05:03.103 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:05:03.103 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:05:03.103 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:05:03.103 INFO:teuthology.orchestra.run.smithi042.stdout: "88cd7478-2b20-47de-ba51-256479749232": { 2024-08-30T09:05:03.103 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:04:08.922625+0000", 2024-08-30T09:05:03.103 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:05:03.103 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:05:03.103 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:05:03.103 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:05:03.103 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:05:03.103 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:05:03.103 INFO:teuthology.orchestra.run.smithi042.stdout: "749f4867-0b70-49e5-af2d-8362a240ebc3": { 2024-08-30T09:05:03.103 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:03:16.386554+0000", 2024-08-30T09:05:03.104 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:05:03.104 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:05:03.104 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:05:03.104 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:05:03.104 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:05:03.104 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:05:03.104 INFO:teuthology.orchestra.run.smithi042.stdout: "85134e4b-ab4a-4885-b07e-a3ad01eaf13d": { 2024-08-30T09:05:03.104 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:01:27.738462+0000", 2024-08-30T09:05:03.104 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:05:03.104 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:05:03.104 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:05:03.104 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:05:03.104 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:05:03.105 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:05:03.105 INFO:teuthology.orchestra.run.smithi042.stdout: "c827ad7a-44ee-4e1d-9bb3-51fb7ab00c73": { 2024-08-30T09:05:03.105 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:02:21.686517+0000", 2024-08-30T09:05:03.105 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:05:03.105 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:05:03.105 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:05:03.105 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:05:03.105 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:05:03.105 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:05:03.105 INFO:teuthology.orchestra.run.smithi042.stdout: "5e2ed104-0625-478a-a78b-584e7f4c2817": { 2024-08-30T09:05:03.105 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:00:33.928400+0000", 2024-08-30T09:05:03.105 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:05:03.105 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:05:03.105 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:05:03.106 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:05:03.106 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:05:03.106 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:05:03.106 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:05:03.106 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-08-30T09:05:03.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:03.099+0000 7f72f3706640 1 -- 172.21.15.42:0/3699909391 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f72d80b4bb0 msgr2=0x7f72d80b70c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:05:03.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:03.099+0000 7f72f3706640 1 --2- 172.21.15.42:0/3699909391 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f72d80b4bb0 0x7f72d80b70c0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f72e4005fd0 tx=0x7f72e4012040 comp rx=0 tx=0).stop 2024-08-30T09:05:03.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:03.099+0000 7f72f3706640 1 -- 172.21.15.42:0/3699909391 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f72ec103960 msgr2=0x7f72ec19a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:05:03.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:03.099+0000 7f72f3706640 1 --2- 172.21.15.42:0/3699909391 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f72ec103960 0x7f72ec19a270 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f72dc005940 tx=0x7f72dc0040c0 comp rx=0 tx=0).stop 2024-08-30T09:05:03.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:03.099+0000 7f72f3706640 1 -- 172.21.15.42:0/3699909391 shutdown_connections 2024-08-30T09:05:03.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:03.099+0000 7f72f3706640 1 --2- 172.21.15.42:0/3699909391 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f72d80b4bb0 0x7f72d80b70c0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:03.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:03.099+0000 7f72f3706640 1 --2- 172.21.15.42:0/3699909391 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f72ec109ba0 0x7f72ec19fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:03.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:03.099+0000 7f72f3706640 1 --2- 172.21.15.42:0/3699909391 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f72ec109250 0x7f72ec19a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:03.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:03.099+0000 7f72f3706640 1 --2- 172.21.15.42:0/3699909391 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f72ec103960 0x7f72ec19a270 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:03.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:03.099+0000 7f72f3706640 1 -- 172.21.15.42:0/3699909391 >> 172.21.15.42:0/3699909391 conn(0x7f72ec07b6b0 msgr2=0x7f72ec107340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:05:03.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:03.100+0000 7f72f3706640 1 -- 172.21.15.42:0/3699909391 shutdown_connections 2024-08-30T09:05:03.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:03.103+0000 7f72f3706640 1 -- 172.21.15.42:0/3699909391 wait complete. 2024-08-30T09:05:03.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:05:03 smithi042 ceph-mon[18652]: scrub summary: idle+waiting paths [/] 2024-08-30T09:05:03.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:05:03 smithi042 ceph-mon[18652]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-30T09:05:03.933 INFO:tasks.cephfs.filesystem:scrub status for tag:347d05ed-4bd3-4590-b099-314f719aab39 - {'path': '/', 'tag': '347d05ed-4bd3-4590-b099-314f719aab39', 'options': 'recursive,force'} 2024-08-30T09:05:04.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:05:03 smithi167 ceph-mon[22063]: scrub summary: idle+waiting paths [/] 2024-08-30T09:05:04.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:05:03 smithi167 ceph-mon[22063]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-30T09:05:04.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:05:03 smithi190 ceph-mon[22001]: scrub summary: idle+waiting paths [/] 2024-08-30T09:05:04.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:05:03 smithi190 ceph-mon[22001]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-30T09:05:05.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:05:04 smithi167 ceph-mon[22063]: pgmap v502: 129 pgs: 129 active+clean; 1.1 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 8.0 MiB/s wr, 833 op/s 2024-08-30T09:05:05.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:05:04 smithi042 ceph-mon[18652]: pgmap v502: 129 pgs: 129 active+clean; 1.1 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 8.0 MiB/s wr, 833 op/s 2024-08-30T09:05:05.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:05:04 smithi190 ceph-mon[22001]: pgmap v502: 129 pgs: 129 active+clean; 1.1 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 8.0 MiB/s wr, 833 op/s 2024-08-30T09:05:05.953 INFO:teuthology.orchestra.run.smithi042.stderr:progress: 260.0 s, 666.4 tps, lat 152.389 ms stddev 166.521 2024-08-30T09:05:07.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:05:06 smithi167 ceph-mon[22063]: pgmap v503: 129 pgs: 129 active+clean; 1.1 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 8.2 MiB/s wr, 816 op/s 2024-08-30T09:05:07.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:05:06 smithi042 ceph-mon[18652]: pgmap v503: 129 pgs: 129 active+clean; 1.1 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 8.2 MiB/s wr, 816 op/s 2024-08-30T09:05:07.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:05:06 smithi190 ceph-mon[22001]: pgmap v503: 129 pgs: 129 active+clean; 1.1 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 8.2 MiB/s wr, 816 op/s 2024-08-30T09:05:09.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:05:08 smithi167 ceph-mon[22063]: pgmap v504: 129 pgs: 129 active+clean; 1.1 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 8.2 MiB/s wr, 828 op/s 2024-08-30T09:05:09.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:05:08 smithi042 ceph-mon[18652]: pgmap v504: 129 pgs: 129 active+clean; 1.1 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 8.2 MiB/s wr, 828 op/s 2024-08-30T09:05:09.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:05:08 smithi190 ceph-mon[22001]: pgmap v504: 129 pgs: 129 active+clean; 1.1 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 8.2 MiB/s wr, 828 op/s 2024-08-30T09:05:10.953 INFO:teuthology.orchestra.run.smithi042.stderr:progress: 265.0 s, 653.6 tps, lat 148.090 ms stddev 174.964 2024-08-30T09:05:10.988 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:05:10 smithi190 ceph-mon[22001]: pgmap v505: 129 pgs: 129 active+clean; 1.1 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 8.3 MiB/s wr, 844 op/s 2024-08-30T09:05:10.988 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:05:10 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-30T09:05:11.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:05:10 smithi167 ceph-mon[22063]: pgmap v505: 129 pgs: 129 active+clean; 1.1 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 8.3 MiB/s wr, 844 op/s 2024-08-30T09:05:11.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:05:10 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-30T09:05:11.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:05:10 smithi042 ceph-mon[18652]: pgmap v505: 129 pgs: 129 active+clean; 1.1 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 8.3 MiB/s wr, 844 op/s 2024-08-30T09:05:11.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:05:10 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-30T09:05:11.895 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:11.890+0000 7f1d61d80640 1 -- 172.21.15.42:0/3265471352 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1d5c10f770 msgr2=0x7f1d5c11a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:05:11.896 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:11.890+0000 7f1d61d80640 1 --2- 172.21.15.42:0/3265471352 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1d5c10f770 0x7f1d5c11a930 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f1d54009ca0 tx=0x7f1d5402a8f0 comp rx=0 tx=0).stop 2024-08-30T09:05:11.898 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:11.894+0000 7f1d61d80640 1 -- 172.21.15.42:0/3265471352 shutdown_connections 2024-08-30T09:05:11.898 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:11.894+0000 7f1d61d80640 1 --2- 172.21.15.42:0/3265471352 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1d5c10f770 0x7f1d5c11a930 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:11.898 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:11.894+0000 7f1d61d80640 1 --2- 172.21.15.42:0/3265471352 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1d5c10cb50 0x7f1d5c10f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:11.898 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:11.894+0000 7f1d61d80640 1 --2- 172.21.15.42:0/3265471352 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1d5c10a160 0x7f1d5c10c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:11.898 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:11.894+0000 7f1d61d80640 1 -- 172.21.15.42:0/3265471352 >> 172.21.15.42:0/3265471352 conn(0x7f1d5c0705b0 msgr2=0x7f1d5c072a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:05:11.898 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:11.895+0000 7f1d61d80640 1 -- 172.21.15.42:0/3265471352 shutdown_connections 2024-08-30T09:05:11.902 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:11.898+0000 7f1d61d80640 1 -- 172.21.15.42:0/3265471352 wait complete. 2024-08-30T09:05:11.902 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:11.899+0000 7f1d61d80640 1 Processor -- start 2024-08-30T09:05:11.905 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:11.901+0000 7f1d61d80640 1 -- start start 2024-08-30T09:05:11.905 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:11.901+0000 7f1d61d80640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1d5c10a160 0x7f1d5c117660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:11.905 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:11.901+0000 7f1d61d80640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1d5c10cb50 0x7f1d5c117cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:11.905 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:11.901+0000 7f1d61d80640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1d5c10f770 0x7f1d5c118880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:11.906 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:11.901+0000 7f1d61d80640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f1d5c11dea0 con 0x7f1d5c10a160 2024-08-30T09:05:11.906 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:11.901+0000 7f1d61d80640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f1d5c11dd20 con 0x7f1d5c10f770 2024-08-30T09:05:11.906 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:11.901+0000 7f1d61d80640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f1d5c11e020 con 0x7f1d5c10cb50 2024-08-30T09:05:11.906 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:11.902+0000 7f1d5b7fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1d5c10a160 0x7f1d5c117660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:05:11.906 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:11.902+0000 7f1d5b7fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1d5c10a160 0x7f1d5c117660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41376/0 (socket says 172.21.15.42:41376) 2024-08-30T09:05:11.906 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:11.902+0000 7f1d5b7fe640 1 -- 172.21.15.42:0/1558963248 learned_addr learned my addr 172.21.15.42:0/1558963248 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:05:11.906 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:11.902+0000 7f1d5b7fe640 1 -- 172.21.15.42:0/1558963248 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1d5c10cb50 msgr2=0x7f1d5c117cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:05:11.906 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:11.902+0000 7f1d5b7fe640 1 --2- 172.21.15.42:0/1558963248 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1d5c10cb50 0x7f1d5c117cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:11.906 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:11.902+0000 7f1d5b7fe640 1 -- 172.21.15.42:0/1558963248 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1d5c10f770 msgr2=0x7f1d5c118880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:05:11.906 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:11.902+0000 7f1d5b7fe640 1 --2- 172.21.15.42:0/1558963248 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1d5c10f770 0x7f1d5c118880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:11.906 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:11.902+0000 7f1d5b7fe640 1 -- 172.21.15.42:0/1558963248 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d5c119010 con 0x7f1d5c10a160 2024-08-30T09:05:11.907 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:11.903+0000 7f1d5b7fe640 1 --2- 172.21.15.42:0/1558963248 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1d5c10a160 0x7f1d5c117660 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f1d5000c920 tx=0x7f1d5000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:05:11.907 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:11.903+0000 7f1d58ff9640 1 -- 172.21.15.42:0/1558963248 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d50007c70 con 0x7f1d5c10a160 2024-08-30T09:05:11.908 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:11.903+0000 7f1d58ff9640 1 -- 172.21.15.42:0/1558963248 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d50007e10 con 0x7f1d5c10a160 2024-08-30T09:05:11.908 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:11.903+0000 7f1d58ff9640 1 -- 172.21.15.42:0/1558963248 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d5000e760 con 0x7f1d5c10a160 2024-08-30T09:05:11.908 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:11.905+0000 7f1d61d80640 1 -- 172.21.15.42:0/1558963248 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d5c119300 con 0x7f1d5c10a160 2024-08-30T09:05:11.908 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:11.905+0000 7f1d61d80640 1 -- 172.21.15.42:0/1558963248 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d5c119870 con 0x7f1d5c10a160 2024-08-30T09:05:11.912 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:11.908+0000 7f1d68dca740 1 Processor -- start 2024-08-30T09:05:11.912 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:11.909+0000 7f1d68dca740 1 -- start start 2024-08-30T09:05:11.912 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:11.909+0000 7f1d68dca740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56467e64d1b0 0x56467e9f2f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:11.912 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:11.909+0000 7f1d68dca740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56467e6242f0 0x56467e9f35d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:11.912 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:11.909+0000 7f1d68dca740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56467e62c4d0 0x56467e9f4120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:11.912 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:11.909+0000 7f1d68dca740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x56467e966000 con 0x56467e6242f0 2024-08-30T09:05:11.912 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:11.909+0000 7f1d68dca740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x56467e955590 con 0x56467e62c4d0 2024-08-30T09:05:11.912 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:11.909+0000 7f1d68dca740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x56467e959540 con 0x56467e64d1b0 2024-08-30T09:05:11.914 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:11.910+0000 7f1d5affd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56467e62c4d0 0x56467e9f4120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:05:11.914 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:11.910+0000 7f1d5affd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56467e62c4d0 0x56467e9f4120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:53352/0 (socket says 172.21.15.42:53352) 2024-08-30T09:05:11.915 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:11.910+0000 7f1d5affd640 1 -- 172.21.15.42:0/3322450249 learned_addr learned my addr 172.21.15.42:0/3322450249 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:05:11.915 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:11.910+0000 7f1d5bfff640 1 --2- 172.21.15.42:0/3322450249 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56467e64d1b0 0x56467e9f2f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:05:11.915 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:11.911+0000 7f1d5bfff640 1 -- 172.21.15.42:0/3322450249 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56467e62c4d0 msgr2=0x56467e9f4120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:05:11.915 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:11.911+0000 7f1d5bfff640 1 --2- 172.21.15.42:0/3322450249 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56467e62c4d0 0x56467e9f4120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:11.915 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:11.911+0000 7f1d5bfff640 1 -- 172.21.15.42:0/3322450249 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56467e6242f0 msgr2=0x56467e9f35d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-30T09:05:11.915 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:11.911+0000 7f1d5bfff640 1 --2- 172.21.15.42:0/3322450249 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56467e6242f0 0x56467e9f35d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:11.915 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:11.911+0000 7f1d5bfff640 1 -- 172.21.15.42:0/3322450249 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56467e6b8790 con 0x56467e64d1b0 2024-08-30T09:05:11.915 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:11.911+0000 7f1d5bfff640 1 --2- 172.21.15.42:0/3322450249 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56467e64d1b0 0x56467e9f2f60 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f1d54031ac0 tx=0x7f1d54004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2446f8174d782428 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:05:11.915 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:11.912+0000 7f1d41ffb640 1 -- 172.21.15.42:0/3322450249 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d54031d60 con 0x56467e64d1b0 2024-08-30T09:05:11.921 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:11.916+0000 7f1d58ff9640 1 -- 172.21.15.42:0/1558963248 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7f1d5000fab0 con 0x7f1d5c10a160 2024-08-30T09:05:11.921 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:11.917+0000 7f1d58ff9640 1 --2- 172.21.15.42:0/1558963248 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f1d440b4d40 0x7f1d440b7250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:11.921 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:11.917+0000 7f1d58ff9640 1 -- 172.21.15.42:0/1558963248 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f1d500e54d0 con 0x7f1d5c10a160 2024-08-30T09:05:11.921 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:11.918+0000 7f1d5b7fe640 1 --2- 172.21.15.42:0/1558963248 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f1d440b4d40 0x7f1d440b7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:05:11.921 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:11.918+0000 7f1d41ffb640 1 -- 172.21.15.42:0/3322450249 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d5403f070 con 0x56467e64d1b0 2024-08-30T09:05:11.922 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:11.920+0000 7f1d68dca740 1 -- 172.21.15.42:0/3322450249 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56467e64d1b0 msgr2=0x56467e9f2f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:05:11.922 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:11.920+0000 7f1d68dca740 1 --2- 172.21.15.42:0/3322450249 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56467e64d1b0 0x56467e9f2f60 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f1d54031ac0 tx=0x7f1d54004a40 comp rx=0 tx=0).stop 2024-08-30T09:05:11.927 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:11.922+0000 7f1d5b7fe640 1 --2- 172.21.15.42:0/1558963248 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f1d440b4d40 0x7f1d440b7250 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f1d500a7c10 tx=0x7f1d50005e60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:05:11.927 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:11.924+0000 7f1d68dca740 1 -- 172.21.15.42:0/3322450249 shutdown_connections 2024-08-30T09:05:11.927 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:11.924+0000 7f1d68dca740 1 --2- 172.21.15.42:0/3322450249 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56467e62c4d0 0x56467e9f4120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:11.927 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:11.924+0000 7f1d68dca740 1 --2- 172.21.15.42:0/3322450249 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56467e6242f0 0x56467e9f35d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:11.927 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:11.924+0000 7f1d68dca740 1 --2- 172.21.15.42:0/3322450249 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56467e64d1b0 0x56467e9f2f60 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:11.927 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:11.924+0000 7f1d68dca740 1 -- 172.21.15.42:0/3322450249 >> 172.21.15.42:0/3322450249 conn(0x56467e641300 msgr2=0x56467e9ee9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:05:11.934 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:11.931+0000 7f1d68dca740 1 -- 172.21.15.42:0/3322450249 shutdown_connections 2024-08-30T09:05:11.935 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:11.931+0000 7f1d68dca740 1 -- 172.21.15.42:0/3322450249 wait complete. 2024-08-30T09:05:11.935 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:11.932+0000 7f1d68dca740 1 -- start start 2024-08-30T09:05:11.935 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:11.932+0000 7f1d68dca740 1 Processor -- start 2024-08-30T09:05:11.936 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:11.933+0000 7f1d68dca740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56467e6242f0 0x56467e9fc9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:11.936 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:11.933+0000 7f1d68dca740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56467e62c4d0 0x56467e9fd0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:11.937 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:11.933+0000 7f1d68dca740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56467e64d1b0 0x56467e9fdc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:11.937 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:11.933+0000 7f1d68dca740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x56467e88eab0 con 0x56467e62c4d0 2024-08-30T09:05:11.937 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:11.933+0000 7f1d68dca740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x56467e8dc680 con 0x56467e64d1b0 2024-08-30T09:05:11.937 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:11.933+0000 7f1d68dca740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x56467e9f4c80 con 0x56467e6242f0 2024-08-30T09:05:11.937 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:11.933+0000 7f1d68dca740 10 client.0 _ll_register_callbacks cb 0x56467e664a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:05:11.937 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:11.933+0000 7f1d5bfff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56467e6242f0 0x56467e9fc9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:05:11.937 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:11.933+0000 7f1d5bfff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56467e64d1b0 0x56467e9fdc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:05:11.937 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:11.933+0000 7f1d5bfff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56467e6242f0 0x56467e9fc9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:39012/0 (socket says 172.21.15.42:39012) 2024-08-30T09:05:11.937 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:11.933+0000 7f1d5bfff640 1 -- 172.21.15.42:0/1252790677 learned_addr learned my addr 172.21.15.42:0/1252790677 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:05:11.938 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:11.935+0000 7f1d5affd640 1 --2- 172.21.15.42:0/1252790677 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56467e62c4d0 0x56467e9fd0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:05:11.939 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:11.936+0000 7f1d5bfff640 1 -- 172.21.15.42:0/1252790677 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56467e64d1b0 msgr2=0x56467e9fdc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:05:11.939 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:11.936+0000 7f1d5bfff640 1 --2- 172.21.15.42:0/1252790677 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56467e64d1b0 0x56467e9fdc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:11.939 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:11.936+0000 7f1d5bfff640 1 -- 172.21.15.42:0/1252790677 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56467e62c4d0 msgr2=0x56467e9fd0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:05:11.939 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:11.936+0000 7f1d5bfff640 1 --2- 172.21.15.42:0/1252790677 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56467e62c4d0 0x56467e9fd0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:11.939 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:11.936+0000 7f1d5bfff640 1 -- 172.21.15.42:0/1252790677 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56467e731620 con 0x56467e6242f0 2024-08-30T09:05:11.939 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:11.936+0000 7f1d5affd640 1 --2- 172.21.15.42:0/1252790677 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56467e62c4d0 0x56467e9fd0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-30T09:05:11.941 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:11.938+0000 7f1d5bfff640 1 --2- 172.21.15.42:0/1252790677 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56467e6242f0 0x56467e9fc9e0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f1d540096c0 tx=0x7f1d54005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6d8bf11546d327ae server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:05:11.941 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:11.938+0000 7f1d3affd640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-08-30T09:05:11.942 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:11.939+0000 7f1d3affd640 1 -- 172.21.15.42:0/1252790677 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d5404f040 con 0x56467e6242f0 2024-08-30T09:05:11.942 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:11.939+0000 7f1d68dca740 1 -- 172.21.15.42:0/1252790677 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1d5403f070 con 0x56467e6242f0 2024-08-30T09:05:11.942 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:11.939+0000 7f1d3affd640 1 -- 172.21.15.42:0/1252790677 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f1d54003b70 con 0x56467e6242f0 2024-08-30T09:05:11.942 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:11.940+0000 7f1d3affd640 1 -- 172.21.15.42:0/1252790677 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f1d54004b40 con 0x56467e6242f0 2024-08-30T09:05:11.943 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:11.940+0000 7f1d68dca740 10 client.25951 fetch_fsmap learned FSMap version 30 2024-08-30T09:05:11.943 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:11.940+0000 7f1d68dca740 1 -- 172.21.15.42:0/1252790677 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56467ea06b60 con 0x56467e6242f0 2024-08-30T09:05:11.947 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:11.944+0000 7f1d3affd640 1 -- 172.21.15.42:0/1252790677 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d5403f070 con 0x56467e6242f0 2024-08-30T09:05:11.947 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:11.944+0000 7f1d3affd640 1 -- 172.21.15.42:0/1252790677 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7f1d54030d50 con 0x56467e6242f0 2024-08-30T09:05:11.947 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:11.944+0000 7f1d68dca740 10 client.25951 fetch_fsmap finished waiting for FSMap version 30 2024-08-30T09:05:11.947 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:11.944+0000 7f1d68dca740 10 client.25951 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-30T09:05:11.947 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:11.944+0000 7f1d68dca740 20 client.25951 populate_metadata read hostname 'smithi042' 2024-08-30T09:05:11.947 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:11.944+0000 7f1d68dca740 1 --2- 172.21.15.42:0/1252790677 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x56467e60bd80 0x56467ea0a440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:11.947 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:11.944+0000 7f1d68dca740 4 client.25951 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-30T09:05:11.947 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:11.944+0000 7f1d68dca740 1 -- 172.21.15.42:0/1252790677 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56467e967f50 con 0x56467e60bd80 2024-08-30T09:05:11.948 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:11.945+0000 7f1d5affd640 1 --2- 172.21.15.42:0/1252790677 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x56467e60bd80 0x56467ea0a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:05:11.951 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:11.948+0000 7f1d5affd640 1 --2- 172.21.15.42:0/1252790677 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x56467e60bd80 0x56467ea0a440 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a06a51ec46aa23f4 server_cookie=a15fcc48ab3cd92d in_seq=0 out_seq=0 2024-08-30T09:05:11.951 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:11.948+0000 7f1d3affd640 10 client.25951 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:05:11.952 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:11.949+0000 7f1d3affd640 1 -- 172.21.15.42:0/1252790677 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56467e967f50 con 0x56467e60bd80 2024-08-30T09:05:11.952 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:11.949+0000 7f1d3affd640 10 client.25951 handle_command_reply: tid=0 2024-08-30T09:05:11.952 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:11.949+0000 7f1d68dca740 1 client.25951 shutdown 2024-08-30T09:05:11.953 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:11.950+0000 7f1d68dca740 1 -- 172.21.15.42:0/1252790677 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56467e6242f0 msgr2=0x56467e9fc9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:05:11.953 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:11.950+0000 7f1d68dca740 1 --2- 172.21.15.42:0/1252790677 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56467e6242f0 0x56467e9fc9e0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f1d540096c0 tx=0x7f1d54005fd0 comp rx=0 tx=0).stop 2024-08-30T09:05:11.953 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:11.951+0000 7f1d68dca740 1 -- 172.21.15.42:0/1252790677 shutdown_connections 2024-08-30T09:05:11.953 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:11.951+0000 7f1d68dca740 1 --2- 172.21.15.42:0/1252790677 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x56467e60bd80 0x56467ea0a440 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:11.953 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:11.951+0000 7f1d68dca740 1 --2- 172.21.15.42:0/1252790677 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56467e64d1b0 0x56467e9fdc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:11.954 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:11.951+0000 7f1d68dca740 1 --2- 172.21.15.42:0/1252790677 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56467e62c4d0 0x56467e9fd0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:11.954 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:11.951+0000 7f1d68dca740 1 --2- 172.21.15.42:0/1252790677 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56467e6242f0 0x56467e9fc9e0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:11.954 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:11.951+0000 7f1d68dca740 1 -- 172.21.15.42:0/1252790677 >> 172.21.15.42:0/1252790677 conn(0x56467e641300 msgr2=0x56467e9ecbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:05:11.954 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:11.951+0000 7f1d3affd640 0 client.25951 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:05:11.955 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:11.952+0000 7f1d68dca740 1 -- 172.21.15.42:0/1252790677 shutdown_connections 2024-08-30T09:05:11.957 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:11.954+0000 7f1d68dca740 1 -- 172.21.15.42:0/1252790677 wait complete. 2024-08-30T09:05:11.957 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:11.954+0000 7f1d68dca740 20 client.25951 trim_cache size 0 max 16384 2024-08-30T09:05:11.988 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:11.984+0000 7f1d68dca740 1 Processor -- start 2024-08-30T09:05:11.988 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:11.984+0000 7f1d68dca740 1 -- start start 2024-08-30T09:05:11.988 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:11.984+0000 7f1d68dca740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56467e6242f0 0x56467e9f2c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:11.988 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:11.984+0000 7f1d68dca740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56467e62c4d0 0x56467e9edb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:11.988 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:11.985+0000 7f1d5bfff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56467e6242f0 0x56467e9f2c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:05:11.989 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:11.985+0000 7f1d5affd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56467e62c4d0 0x56467e9edb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:05:11.989 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:11.985+0000 7f1d5bfff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56467e6242f0 0x56467e9f2c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:53374/0 (socket says 172.21.15.42:53374) 2024-08-30T09:05:11.989 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:11.985+0000 7f1d5affd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56467e62c4d0 0x56467e9edb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41402/0 (socket says 172.21.15.42:41402) 2024-08-30T09:05:11.989 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:11.985+0000 7f1d68dca740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56467e64d1b0 0x56467e9ee330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:11.989 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:11.985+0000 7f1d5bfff640 1 -- 172.21.15.42:0/674918798 learned_addr learned my addr 172.21.15.42:0/674918798 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:05:11.989 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:11.985+0000 7f1d68dca740 1 -- 172.21.15.42:0/674918798 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f1d5c1c63b0 con 0x56467e62c4d0 2024-08-30T09:05:11.989 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:11.985+0000 7f1d68dca740 1 -- 172.21.15.42:0/674918798 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f1d5c1c3c80 con 0x56467e6242f0 2024-08-30T09:05:11.989 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:11.985+0000 7f1d68dca740 1 -- 172.21.15.42:0/674918798 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f1d5c1c1ad0 con 0x56467e64d1b0 2024-08-30T09:05:11.989 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:11.985+0000 7f1d5bfff640 1 --2- 172.21.15.42:0/674918798 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56467e64d1b0 0x56467e9ee330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:05:11.989 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:11.985+0000 7f1d5affd640 1 -- 172.21.15.42:0/674918798 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56467e64d1b0 msgr2=0x56467e9ee330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:05:11.989 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:11.985+0000 7f1d5affd640 1 --2- 172.21.15.42:0/674918798 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56467e64d1b0 0x56467e9ee330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:11.989 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:11.985+0000 7f1d5affd640 1 -- 172.21.15.42:0/674918798 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56467e6242f0 msgr2=0x56467e9f2c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:05:11.989 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:11.985+0000 7f1d5affd640 1 --2- 172.21.15.42:0/674918798 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56467e6242f0 0x56467e9f2c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:11.990 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:11.985+0000 7f1d5affd640 1 -- 172.21.15.42:0/674918798 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56467e9fd500 con 0x56467e62c4d0 2024-08-30T09:05:11.990 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:11.985+0000 7f1d5bfff640 1 --2- 172.21.15.42:0/674918798 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56467e64d1b0 0x56467e9ee330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-30T09:05:11.990 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:11.985+0000 7f1d5affd640 1 --2- 172.21.15.42:0/674918798 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56467e62c4d0 0x56467e9edb70 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x56467ea0ab40 tx=0x7f1d4c012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ef535e6ad78943f server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:05:11.990 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:11.985+0000 7f1d417fa640 1 -- 172.21.15.42:0/674918798 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d4c01e070 con 0x56467e62c4d0 2024-08-30T09:05:11.990 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:11.986+0000 7f1d417fa640 1 -- 172.21.15.42:0/674918798 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d4c011890 con 0x56467e62c4d0 2024-08-30T09:05:11.990 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:11.986+0000 7f1d417fa640 1 -- 172.21.15.42:0/674918798 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d4c010770 con 0x56467e62c4d0 2024-08-30T09:05:11.993 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:11.986+0000 7f1d68dca740 1 -- 172.21.15.42:0/674918798 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56467e62c4d0 msgr2=0x56467e9edb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:05:11.993 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:11.986+0000 7f1d68dca740 1 --2- 172.21.15.42:0/674918798 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56467e62c4d0 0x56467e9edb70 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x56467ea0ab40 tx=0x7f1d4c012820 comp rx=0 tx=0).stop 2024-08-30T09:05:11.994 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:11.986+0000 7f1d68dca740 1 -- 172.21.15.42:0/674918798 shutdown_connections 2024-08-30T09:05:11.994 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:11.986+0000 7f1d68dca740 1 --2- 172.21.15.42:0/674918798 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56467e64d1b0 0x56467e9ee330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:11.994 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:11.986+0000 7f1d68dca740 1 --2- 172.21.15.42:0/674918798 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56467e62c4d0 0x56467e9edb70 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:11.994 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:11.986+0000 7f1d68dca740 1 --2- 172.21.15.42:0/674918798 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56467e6242f0 0x56467e9f2c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:11.994 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:11.986+0000 7f1d68dca740 1 -- 172.21.15.42:0/674918798 >> 172.21.15.42:0/674918798 conn(0x56467e641300 msgr2=0x56467e9f76f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:05:11.994 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:11.987+0000 7f1d68dca740 1 -- 172.21.15.42:0/674918798 shutdown_connections 2024-08-30T09:05:11.994 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:11.987+0000 7f1d68dca740 1 -- 172.21.15.42:0/674918798 wait complete. 2024-08-30T09:05:11.994 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:11.987+0000 7f1d68dca740 1 -- start start 2024-08-30T09:05:11.994 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:11.987+0000 7f1d68dca740 1 Processor -- start 2024-08-30T09:05:11.994 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:11.987+0000 7f1d68dca740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56467e6242f0 0x56467e9ff4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:11.994 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:11.987+0000 7f1d68dca740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56467e62c4d0 0x56467e9ffb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:11.994 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:11.987+0000 7f1d68dca740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56467e64d1b0 0x56467ea00510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:11.994 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:11.987+0000 7f1d68dca740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f1d5c1c2140 con 0x56467e6242f0 2024-08-30T09:05:11.995 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:11.987+0000 7f1d68dca740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f1d5c1c35b0 con 0x56467e64d1b0 2024-08-30T09:05:11.995 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:11.988+0000 7f1d68dca740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f1d5c1c2810 con 0x56467e62c4d0 2024-08-30T09:05:11.995 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:11.988+0000 7f1d5bfff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56467e6242f0 0x56467e9ff4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:05:11.995 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:11.988+0000 7f1d5affd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56467e62c4d0 0x56467e9ffb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:05:11.995 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:11.988+0000 7f1d5bfff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56467e6242f0 0x56467e9ff4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41418/0 (socket says 172.21.15.42:41418) 2024-08-30T09:05:11.995 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:11.988+0000 7f1d5bfff640 1 -- 172.21.15.42:0/2758749016 learned_addr learned my addr 172.21.15.42:0/2758749016 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:05:11.995 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:11.988+0000 7f1d5bfff640 1 --2- 172.21.15.42:0/2758749016 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56467e64d1b0 0x56467ea00510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:05:11.995 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:11.988+0000 7f1d68dca740 10 client.0 _ll_register_callbacks cb 0x7f1d540099a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:05:11.995 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:11.988+0000 7f1d5bfff640 1 -- 172.21.15.42:0/2758749016 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56467e62c4d0 msgr2=0x56467e9ffb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:05:11.995 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:11.988+0000 7f1d5bfff640 1 --2- 172.21.15.42:0/2758749016 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56467e62c4d0 0x56467e9ffb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:11.995 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:11.988+0000 7f1d5bfff640 1 -- 172.21.15.42:0/2758749016 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56467e64d1b0 msgr2=0x56467ea00510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:05:11.995 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:11.988+0000 7f1d5bfff640 1 --2- 172.21.15.42:0/2758749016 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56467e64d1b0 0x56467ea00510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:11.995 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:11.988+0000 7f1d5bfff640 1 -- 172.21.15.42:0/2758749016 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56467ea00c80 con 0x56467e6242f0 2024-08-30T09:05:11.995 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:11.988+0000 7f1d5bfff640 1 --2- 172.21.15.42:0/2758749016 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56467e6242f0 0x56467e9ff4d0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f1d54044660 tx=0x7f1d54004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c47d7b03823bb38d server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:05:11.995 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:11.988+0000 7f1d3affd640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-08-30T09:05:11.996 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:11.988+0000 7f1d3affd640 1 -- 172.21.15.42:0/2758749016 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d540446e0 con 0x56467e6242f0 2024-08-30T09:05:11.996 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:11.988+0000 7f1d68dca740 1 -- 172.21.15.42:0/2758749016 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1d4c011890 con 0x56467e6242f0 2024-08-30T09:05:11.996 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:11.988+0000 7f1d3affd640 1 -- 172.21.15.42:0/2758749016 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f1d54044880 con 0x56467e6242f0 2024-08-30T09:05:11.996 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:11.988+0000 7f1d3affd640 1 -- 172.21.15.42:0/2758749016 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f1d54003970 con 0x56467e6242f0 2024-08-30T09:05:11.996 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:11.989+0000 7f1d68dca740 10 client.16218 fetch_fsmap learned FSMap version 30 2024-08-30T09:05:11.996 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:11.989+0000 7f1d68dca740 1 -- 172.21.15.42:0/2758749016 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56467ea03f50 con 0x56467e6242f0 2024-08-30T09:05:11.996 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:11.989+0000 7f1d3affd640 1 -- 172.21.15.42:0/2758749016 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d4c011890 con 0x56467e6242f0 2024-08-30T09:05:11.996 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:11.989+0000 7f1d3affd640 1 -- 172.21.15.42:0/2758749016 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7f1d54030ab0 con 0x56467e6242f0 2024-08-30T09:05:11.996 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:11.989+0000 7f1d68dca740 10 client.16218 fetch_fsmap finished waiting for FSMap version 30 2024-08-30T09:05:11.996 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:11.989+0000 7f1d68dca740 10 client.16218 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-30T09:05:11.996 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:11.989+0000 7f1d68dca740 20 client.16218 populate_metadata read hostname 'smithi042' 2024-08-30T09:05:11.996 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:11.989+0000 7f1d68dca740 1 --2- 172.21.15.42:0/2758749016 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x56467e5e11c0 0x56467ea048b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:11.996 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:11.989+0000 7f1d68dca740 4 client.16218 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-30T09:05:11.997 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:11.989+0000 7f1d68dca740 1 -- 172.21.15.42:0/2758749016 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56467e9639f0 con 0x56467e5e11c0 2024-08-30T09:05:11.997 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:11.989+0000 7f1d5affd640 1 --2- 172.21.15.42:0/2758749016 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x56467e5e11c0 0x56467ea048b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:05:11.997 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:11.990+0000 7f1d3affd640 10 client.16218 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:05:11.997 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:11.990+0000 7f1d5affd640 1 --2- 172.21.15.42:0/2758749016 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x56467e5e11c0 0x56467ea048b0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a8c065b0bba970ce server_cookie=1c167cf220709410 in_seq=0 out_seq=0 2024-08-30T09:05:12.008 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:12.004+0000 7f1d3affd640 1 -- 172.21.15.42:0/2758749016 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56467e9639f0 con 0x56467e5e11c0 2024-08-30T09:05:12.008 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:12.004+0000 7f1d3affd640 10 client.16218 handle_command_reply: tid=0 2024-08-30T09:05:12.008 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:12.004+0000 7f1d68dca740 1 client.16218 shutdown 2024-08-30T09:05:12.008 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:12.005+0000 7f1d68dca740 1 -- 172.21.15.42:0/2758749016 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56467e6242f0 msgr2=0x56467e9ff4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:05:12.008 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:12.005+0000 7f1d68dca740 1 --2- 172.21.15.42:0/2758749016 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56467e6242f0 0x56467e9ff4d0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f1d54044660 tx=0x7f1d54004a40 comp rx=0 tx=0).stop 2024-08-30T09:05:12.008 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:12.005+0000 7f1d68dca740 1 -- 172.21.15.42:0/2758749016 shutdown_connections 2024-08-30T09:05:12.008 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:12.005+0000 7f1d68dca740 1 --2- 172.21.15.42:0/2758749016 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x56467e5e11c0 0x56467ea048b0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:12.009 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:12.005+0000 7f1d68dca740 1 --2- 172.21.15.42:0/2758749016 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56467e62c4d0 0x56467e9ffb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:12.009 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:12.005+0000 7f1d3affd640 0 client.16218 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:05:12.009 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:12.005+0000 7f1d68dca740 1 --2- 172.21.15.42:0/2758749016 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56467e64d1b0 0x56467ea00510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:12.009 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:12.005+0000 7f1d68dca740 1 --2- 172.21.15.42:0/2758749016 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56467e6242f0 0x56467e9ff4d0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:12.009 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:12.005+0000 7f1d68dca740 1 -- 172.21.15.42:0/2758749016 >> 172.21.15.42:0/2758749016 conn(0x56467e641300 msgr2=0x56467e9f2b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:05:12.009 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:12.005+0000 7f1d68dca740 1 -- 172.21.15.42:0/2758749016 shutdown_connections 2024-08-30T09:05:12.009 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:12.005+0000 7f1d68dca740 1 -- 172.21.15.42:0/2758749016 wait complete. 2024-08-30T09:05:12.009 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:12.005+0000 7f1d68dca740 20 client.16218 trim_cache size 0 max 16384 2024-08-30T09:05:12.009 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-08-30T09:05:12.009 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2024-08-30T09:05:12.010 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-08-30T09:05:12.010 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-08-30T09:05:12.010 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:12.006+0000 7f1d61d80640 1 -- 172.21.15.42:0/1558963248 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f1d440b4d40 msgr2=0x7f1d440b7250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:05:12.010 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:12.006+0000 7f1d61d80640 1 --2- 172.21.15.42:0/1558963248 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f1d440b4d40 0x7f1d440b7250 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f1d500a7c10 tx=0x7f1d50005e60 comp rx=0 tx=0).stop 2024-08-30T09:05:12.010 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:12.006+0000 7f1d61d80640 1 -- 172.21.15.42:0/1558963248 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1d5c10a160 msgr2=0x7f1d5c117660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:05:12.010 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:12.006+0000 7f1d61d80640 1 --2- 172.21.15.42:0/1558963248 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1d5c10a160 0x7f1d5c117660 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f1d5000c920 tx=0x7f1d5000cde0 comp rx=0 tx=0).stop 2024-08-30T09:05:12.011 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:12.006+0000 7f1d61d80640 1 -- 172.21.15.42:0/1558963248 shutdown_connections 2024-08-30T09:05:12.011 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:12.006+0000 7f1d61d80640 1 --2- 172.21.15.42:0/1558963248 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f1d440b4d40 0x7f1d440b7250 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:12.011 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:12.006+0000 7f1d61d80640 1 --2- 172.21.15.42:0/1558963248 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1d5c10cb50 0x7f1d5c117cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:12.011 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:12.006+0000 7f1d61d80640 1 --2- 172.21.15.42:0/1558963248 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1d5c10f770 0x7f1d5c118880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:12.011 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:12.006+0000 7f1d61d80640 1 --2- 172.21.15.42:0/1558963248 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1d5c10a160 0x7f1d5c117660 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:12.011 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:12.006+0000 7f1d61d80640 1 -- 172.21.15.42:0/1558963248 >> 172.21.15.42:0/1558963248 conn(0x7f1d5c0705b0 msgr2=0x7f1d5c10af30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:05:12.011 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:12.006+0000 7f1d61d80640 1 -- 172.21.15.42:0/1558963248 shutdown_connections 2024-08-30T09:05:12.011 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:12.006+0000 7f1d61d80640 1 -- 172.21.15.42:0/1558963248 wait complete. 2024-08-30T09:05:12.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:05:11 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T09:05:12.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:05:11 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-30T09:05:12.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:05:11 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T09:05:12.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:05:11 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T09:05:12.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:05:11 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-30T09:05:12.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:05:11 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T09:05:12.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:05:11 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T09:05:12.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:05:11 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-30T09:05:12.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:05:11 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T09:05:12.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:05:12 smithi042 ceph-mon[18652]: pgmap v506: 129 pgs: 129 active+clean; 1.1 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 7.4 MiB/s wr, 755 op/s 2024-08-30T09:05:12.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:05:12 smithi042 ceph-mon[18652]: scrub summary: idle 2024-08-30T09:05:12.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:05:12 smithi042 ceph-mon[18652]: scrub summary: idle 2024-08-30T09:05:13.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:05:12 smithi167 ceph-mon[22063]: pgmap v506: 129 pgs: 129 active+clean; 1.1 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 7.4 MiB/s wr, 755 op/s 2024-08-30T09:05:13.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:05:12 smithi167 ceph-mon[22063]: scrub summary: idle 2024-08-30T09:05:13.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:05:12 smithi167 ceph-mon[22063]: scrub summary: idle 2024-08-30T09:05:13.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:05:12 smithi190 ceph-mon[22001]: pgmap v506: 129 pgs: 129 active+clean; 1.1 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 7.4 MiB/s wr, 755 op/s 2024-08-30T09:05:13.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:05:12 smithi190 ceph-mon[22001]: scrub summary: idle 2024-08-30T09:05:13.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:05:12 smithi190 ceph-mon[22001]: scrub summary: idle 2024-08-30T09:05:14.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:05:13 smithi167 ceph-mon[22063]: scrub summary: idle 2024-08-30T09:05:14.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:05:13 smithi042 ceph-mon[18652]: scrub summary: idle 2024-08-30T09:05:14.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:05:13 smithi190 ceph-mon[22001]: scrub summary: idle 2024-08-30T09:05:15.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:05:14 smithi167 ceph-mon[22063]: pgmap v507: 129 pgs: 129 active+clean; 1.1 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 8.4 MiB/s wr, 877 op/s 2024-08-30T09:05:15.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:05:14 smithi042 ceph-mon[18652]: pgmap v507: 129 pgs: 129 active+clean; 1.1 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 8.4 MiB/s wr, 877 op/s 2024-08-30T09:05:15.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:05:14 smithi190 ceph-mon[22001]: pgmap v507: 129 pgs: 129 active+clean; 1.1 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 8.4 MiB/s wr, 877 op/s 2024-08-30T09:05:15.953 INFO:teuthology.orchestra.run.smithi042.stderr:progress: 270.0 s, 646.0 tps, lat 161.599 ms stddev 198.707 2024-08-30T09:05:17.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:05:16 smithi167 ceph-mon[22063]: pgmap v508: 129 pgs: 129 active+clean; 1.1 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 5.5 MiB/s wr, 613 op/s 2024-08-30T09:05:17.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:05:16 smithi042 ceph-mon[18652]: pgmap v508: 129 pgs: 129 active+clean; 1.1 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 5.5 MiB/s wr, 613 op/s 2024-08-30T09:05:17.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:05:16 smithi190 ceph-mon[22001]: pgmap v508: 129 pgs: 129 active+clean; 1.1 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 5.5 MiB/s wr, 613 op/s 2024-08-30T09:05:19.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:05:18 smithi167 ceph-mon[22063]: pgmap v509: 129 pgs: 129 active+clean; 1.1 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 4.8 MiB/s wr, 542 op/s 2024-08-30T09:05:19.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:05:18 smithi042 ceph-mon[18652]: pgmap v509: 129 pgs: 129 active+clean; 1.1 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 4.8 MiB/s wr, 542 op/s 2024-08-30T09:05:19.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:05:18 smithi190 ceph-mon[22001]: pgmap v509: 129 pgs: 129 active+clean; 1.1 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 4.8 MiB/s wr, 542 op/s 2024-08-30T09:05:20.953 INFO:teuthology.orchestra.run.smithi042.stderr:progress: 275.0 s, 1836.6 tps, lat 54.810 ms stddev 70.036 2024-08-30T09:05:21.076 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:05:20 smithi167 ceph-mon[22063]: pgmap v510: 129 pgs: 129 active+clean; 1.1 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 5.1 MiB/s wr, 566 op/s 2024-08-30T09:05:21.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:05:20 smithi042 ceph-mon[18652]: pgmap v510: 129 pgs: 129 active+clean; 1.1 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 5.1 MiB/s wr, 566 op/s 2024-08-30T09:05:21.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:05:20 smithi190 ceph-mon[22001]: pgmap v510: 129 pgs: 129 active+clean; 1.1 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 5.1 MiB/s wr, 566 op/s 2024-08-30T09:05:23.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:05:22 smithi167 ceph-mon[22063]: pgmap v511: 129 pgs: 129 active+clean; 1.2 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 5.4 MiB/s wr, 492 op/s 2024-08-30T09:05:23.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:05:22 smithi042 ceph-mon[18652]: pgmap v511: 129 pgs: 129 active+clean; 1.2 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 5.4 MiB/s wr, 492 op/s 2024-08-30T09:05:23.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:05:22 smithi190 ceph-mon[22001]: pgmap v511: 129 pgs: 129 active+clean; 1.2 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 5.4 MiB/s wr, 492 op/s 2024-08-30T09:05:25.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:05:24 smithi167 ceph-mon[22063]: pgmap v512: 129 pgs: 129 active+clean; 1.2 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 6.8 MiB/s wr, 635 op/s 2024-08-30T09:05:25.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:05:24 smithi042 ceph-mon[18652]: pgmap v512: 129 pgs: 129 active+clean; 1.2 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 6.8 MiB/s wr, 635 op/s 2024-08-30T09:05:25.178 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.175+0000 7f78e1bf1640 1 -- 172.21.15.42:0/1121136642 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f78dc10f770 msgr2=0x7f78dc11a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:05:25.178 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.175+0000 7f78e1bf1640 1 --2- 172.21.15.42:0/1121136642 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f78dc10f770 0x7f78dc11a930 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f78cc009860 tx=0x7f78cc02a380 comp rx=0 tx=0).stop 2024-08-30T09:05:25.185 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.180+0000 7f78e1bf1640 1 -- 172.21.15.42:0/1121136642 shutdown_connections 2024-08-30T09:05:25.185 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.180+0000 7f78e1bf1640 1 --2- 172.21.15.42:0/1121136642 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f78dc10cb50 0x7f78dc10f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:25.185 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.180+0000 7f78e1bf1640 1 --2- 172.21.15.42:0/1121136642 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f78dc10f770 0x7f78dc11a930 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:25.185 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.180+0000 7f78e1bf1640 1 --2- 172.21.15.42:0/1121136642 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f78dc10a160 0x7f78dc10c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:25.186 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.180+0000 7f78e1bf1640 1 -- 172.21.15.42:0/1121136642 >> 172.21.15.42:0/1121136642 conn(0x7f78dc0705b0 msgr2=0x7f78dc072a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:05:25.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:05:24 smithi190 ceph-mon[22001]: pgmap v512: 129 pgs: 129 active+clean; 1.2 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 6.8 MiB/s wr, 635 op/s 2024-08-30T09:05:25.189 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.186+0000 7f78e1bf1640 1 -- 172.21.15.42:0/1121136642 shutdown_connections 2024-08-30T09:05:25.192 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.190+0000 7f78e1bf1640 1 -- 172.21.15.42:0/1121136642 wait complete. 2024-08-30T09:05:25.194 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.191+0000 7f78e1bf1640 1 Processor -- start 2024-08-30T09:05:25.199 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.196+0000 7f78e1bf1640 1 -- start start 2024-08-30T09:05:25.200 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.196+0000 7f78e1bf1640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f78dc10a160 0x7f78dc1a03b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:25.200 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.197+0000 7f78e1bf1640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f78dc10cb50 0x7f78dc1a0a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:25.200 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.197+0000 7f78e1bf1640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f78dc10f770 0x7f78dc1a15a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:25.200 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.197+0000 7f78e1bf1640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f78dc114c00 con 0x7f78dc10cb50 2024-08-30T09:05:25.200 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.197+0000 7f78e1bf1640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f78dc114a80 con 0x7f78dc10f770 2024-08-30T09:05:25.200 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.197+0000 7f78e1bf1640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f78dc114d80 con 0x7f78dc10a160 2024-08-30T09:05:25.203 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.200+0000 7f78dbfff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f78dc10f770 0x7f78dc1a15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:05:25.203 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.200+0000 7f78dbfff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f78dc10f770 0x7f78dc1a15a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:33690/0 (socket says 172.21.15.42:33690) 2024-08-30T09:05:25.203 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.200+0000 7f78dbfff640 1 -- 172.21.15.42:0/2248446315 learned_addr learned my addr 172.21.15.42:0/2248446315 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:05:25.206 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.201+0000 7f78db7fe640 1 --2- 172.21.15.42:0/2248446315 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f78dc10a160 0x7f78dc1a03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:05:25.206 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.202+0000 7f78daffd640 1 --2- 172.21.15.42:0/2248446315 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f78dc10cb50 0x7f78dc1a0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:05:25.207 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.204+0000 7f78db7fe640 1 -- 172.21.15.42:0/2248446315 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f78dc10f770 msgr2=0x7f78dc1a15a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:05:25.207 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.204+0000 7f78db7fe640 1 --2- 172.21.15.42:0/2248446315 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f78dc10f770 0x7f78dc1a15a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:25.207 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.204+0000 7f78db7fe640 1 -- 172.21.15.42:0/2248446315 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f78dc10cb50 msgr2=0x7f78dc1a0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:05:25.207 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.204+0000 7f78db7fe640 1 --2- 172.21.15.42:0/2248446315 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f78dc10cb50 0x7f78dc1a0a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:25.207 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.204+0000 7f78db7fe640 1 -- 172.21.15.42:0/2248446315 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78dc1a1dc0 con 0x7f78dc10a160 2024-08-30T09:05:25.209 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.205+0000 7f78db7fe640 1 --2- 172.21.15.42:0/2248446315 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f78dc10a160 0x7f78dc1a03b0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f78d400bd70 tx=0x7f78d400be60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:05:25.213 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.208+0000 7f78d8ff9640 1 -- 172.21.15.42:0/2248446315 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78d4019070 con 0x7f78dc10a160 2024-08-30T09:05:25.213 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.208+0000 7f78d8ff9640 1 -- 172.21.15.42:0/2248446315 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f78d40148f0 con 0x7f78dc10a160 2024-08-30T09:05:25.214 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.208+0000 7f78d8ff9640 1 -- 172.21.15.42:0/2248446315 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78d4012920 con 0x7f78dc10a160 2024-08-30T09:05:25.214 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.209+0000 7f78e1bf1640 1 -- 172.21.15.42:0/2248446315 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78dc1a20b0 con 0x7f78dc10a160 2024-08-30T09:05:25.214 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.209+0000 7f78e1bf1640 1 -- 172.21.15.42:0/2248446315 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78dc1a8780 con 0x7f78dc10a160 2024-08-30T09:05:25.219 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.216+0000 7f78e8c34740 1 Processor -- start 2024-08-30T09:05:25.223 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.220+0000 7f78d8ff9640 1 -- 172.21.15.42:0/2248446315 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7f78d401c070 con 0x7f78dc10a160 2024-08-30T09:05:25.225 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.222+0000 7f78d8ff9640 1 --2- 172.21.15.42:0/2248446315 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f78c00b4d40 0x7f78c00b7250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:25.225 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.222+0000 7f78e8c34740 1 -- start start 2024-08-30T09:05:25.225 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.222+0000 7f78e8c34740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bc206631b0 0x55bc20a08f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:25.226 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.222+0000 7f78e8c34740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bc2063a2f0 0x55bc20a095d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:25.226 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.222+0000 7f78e8c34740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bc206424d0 0x55bc20a0a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:25.226 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.222+0000 7f78e8c34740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55bc2097c000 con 0x55bc206424d0 2024-08-30T09:05:25.226 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.222+0000 7f78e8c34740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55bc2096b590 con 0x55bc206631b0 2024-08-30T09:05:25.226 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.222+0000 7f78e8c34740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55bc2096f540 con 0x55bc2063a2f0 2024-08-30T09:05:25.226 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.222+0000 7f78d8ff9640 1 -- 172.21.15.42:0/2248446315 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f78d40e4f60 con 0x7f78dc10a160 2024-08-30T09:05:25.226 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.223+0000 7f78db7fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bc206631b0 0x55bc20a08f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:05:25.226 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.223+0000 7f78db7fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bc206631b0 0x55bc20a08f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:33698/0 (socket says 172.21.15.42:33698) 2024-08-30T09:05:25.226 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.223+0000 7f78db7fe640 1 -- 172.21.15.42:0/3710992706 learned_addr learned my addr 172.21.15.42:0/3710992706 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:05:25.227 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.223+0000 7f78db7fe640 1 -- 172.21.15.42:0/3710992706 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bc2063a2f0 msgr2=0x55bc20a095d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-30T09:05:25.227 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.223+0000 7f78db7fe640 1 --2- 172.21.15.42:0/3710992706 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bc2063a2f0 0x55bc20a095d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:25.227 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.223+0000 7f78db7fe640 1 -- 172.21.15.42:0/3710992706 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bc206424d0 msgr2=0x55bc20a0a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:05:25.227 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.223+0000 7f78db7fe640 1 --2- 172.21.15.42:0/3710992706 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bc206424d0 0x55bc20a0a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:25.227 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.223+0000 7f78db7fe640 1 -- 172.21.15.42:0/3710992706 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc206ce790 con 0x55bc206631b0 2024-08-30T09:05:25.227 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.224+0000 7f78db7fe640 1 --2- 172.21.15.42:0/3710992706 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bc206631b0 0x55bc20a08f60 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f78d40ab5e0 tx=0x7f78d4004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c8dcab62fb809224 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:05:25.229 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.225+0000 7f78d1ffb640 1 -- 172.21.15.42:0/3710992706 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78d40b48c0 con 0x55bc206631b0 2024-08-30T09:05:25.229 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.225+0000 7f78d1ffb640 1 -- 172.21.15.42:0/3710992706 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f78d40a78e0 con 0x55bc206631b0 2024-08-30T09:05:25.230 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.226+0000 7f78e8c34740 1 -- 172.21.15.42:0/3710992706 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bc206631b0 msgr2=0x55bc20a08f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:05:25.230 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.226+0000 7f78e8c34740 1 --2- 172.21.15.42:0/3710992706 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bc206631b0 0x55bc20a08f60 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f78d40ab5e0 tx=0x7f78d4004a40 comp rx=0 tx=0).stop 2024-08-30T09:05:25.230 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.227+0000 7f78dbfff640 1 --2- 172.21.15.42:0/2248446315 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f78c00b4d40 0x7f78c00b7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:05:25.231 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.228+0000 7f78e8c34740 1 -- 172.21.15.42:0/3710992706 shutdown_connections 2024-08-30T09:05:25.231 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.228+0000 7f78e8c34740 1 --2- 172.21.15.42:0/3710992706 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bc2063a2f0 0x55bc20a095d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:25.231 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.228+0000 7f78e8c34740 1 --2- 172.21.15.42:0/3710992706 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bc206424d0 0x55bc20a0a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:25.232 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.228+0000 7f78e8c34740 1 --2- 172.21.15.42:0/3710992706 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bc206631b0 0x55bc20a08f60 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:25.232 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.228+0000 7f78e8c34740 1 -- 172.21.15.42:0/3710992706 >> 172.21.15.42:0/3710992706 conn(0x55bc20657300 msgr2=0x55bc20a049f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:05:25.232 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.229+0000 7f78dbfff640 1 --2- 172.21.15.42:0/2248446315 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f78c00b4d40 0x7f78c00b7250 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f78cc005eb0 tx=0x7f78cc030bb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:05:25.235 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.232+0000 7f78e8c34740 1 -- 172.21.15.42:0/3710992706 shutdown_connections 2024-08-30T09:05:25.236 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.232+0000 7f78e8c34740 1 -- 172.21.15.42:0/3710992706 wait complete. 2024-08-30T09:05:25.236 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.233+0000 7f78e8c34740 1 -- start start 2024-08-30T09:05:25.236 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.233+0000 7f78e8c34740 1 Processor -- start 2024-08-30T09:05:25.237 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.234+0000 7f78e8c34740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bc2063a2f0 0x55bc20a129c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:25.237 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.234+0000 7f78e8c34740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bc206424d0 0x55bc20a13080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:25.237 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.234+0000 7f78e8c34740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bc206631b0 0x55bc20a13bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:25.237 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.234+0000 7f78e8c34740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55bc208a4ab0 con 0x55bc206424d0 2024-08-30T09:05:25.237 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.234+0000 7f78e8c34740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55bc208f2680 con 0x55bc2063a2f0 2024-08-30T09:05:25.237 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.234+0000 7f78e8c34740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55bc20a0ac90 con 0x55bc206631b0 2024-08-30T09:05:25.237 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.234+0000 7f78e8c34740 10 client.0 _ll_register_callbacks cb 0x55bc2067aa00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:05:25.238 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.234+0000 7f78dbfff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bc206631b0 0x55bc20a13bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:05:25.238 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.234+0000 7f78dbfff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bc206631b0 0x55bc20a13bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:33806/0 (socket says 172.21.15.42:33806) 2024-08-30T09:05:25.238 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.234+0000 7f78dbfff640 1 -- 172.21.15.42:0/4180009024 learned_addr learned my addr 172.21.15.42:0/4180009024 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:05:25.238 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.234+0000 7f78daffd640 1 --2- 172.21.15.42:0/4180009024 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bc206424d0 0x55bc20a13080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:05:25.238 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.234+0000 7f78db7fe640 1 --2- 172.21.15.42:0/4180009024 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bc2063a2f0 0x55bc20a129c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:05:25.240 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.237+0000 7f78dbfff640 1 -- 172.21.15.42:0/4180009024 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bc2063a2f0 msgr2=0x55bc20a129c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:05:25.240 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.237+0000 7f78dbfff640 1 --2- 172.21.15.42:0/4180009024 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bc2063a2f0 0x55bc20a129c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:25.240 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.237+0000 7f78dbfff640 1 -- 172.21.15.42:0/4180009024 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bc206424d0 msgr2=0x55bc20a13080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:05:25.240 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.237+0000 7f78dbfff640 1 --2- 172.21.15.42:0/4180009024 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bc206424d0 0x55bc20a13080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:25.240 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.237+0000 7f78dbfff640 1 -- 172.21.15.42:0/4180009024 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc20747620 con 0x55bc206631b0 2024-08-30T09:05:25.240 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.237+0000 7f78daffd640 1 --2- 172.21.15.42:0/4180009024 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bc206424d0 0x55bc20a13080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-30T09:05:25.241 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.238+0000 7f78dbfff640 1 --2- 172.21.15.42:0/4180009024 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bc206631b0 0x55bc20a13bf0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f78cc030960 tx=0x7f78cc005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=586a61acfebd244c server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:05:25.241 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.238+0000 7f78beffd640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-08-30T09:05:25.242 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.239+0000 7f78beffd640 1 -- 172.21.15.42:0/4180009024 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78cc043070 con 0x55bc206631b0 2024-08-30T09:05:25.242 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.239+0000 7f78beffd640 1 -- 172.21.15.42:0/4180009024 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f78cc005450 con 0x55bc206631b0 2024-08-30T09:05:25.242 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.240+0000 7f78e8c34740 1 -- 172.21.15.42:0/4180009024 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f78d40a78e0 con 0x55bc206631b0 2024-08-30T09:05:25.242 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.240+0000 7f78beffd640 1 -- 172.21.15.42:0/4180009024 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f78cc037050 con 0x55bc206631b0 2024-08-30T09:05:25.244 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.242+0000 7f78e8c34740 10 client.25963 fetch_fsmap learned FSMap version 30 2024-08-30T09:05:25.244 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.242+0000 7f78e8c34740 1 -- 172.21.15.42:0/4180009024 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55bc20a1cbe0 con 0x55bc206631b0 2024-08-30T09:05:25.247 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.244+0000 7f78beffd640 1 -- 172.21.15.42:0/4180009024 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f78d40a78e0 con 0x55bc206631b0 2024-08-30T09:05:25.247 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.244+0000 7f78beffd640 1 -- 172.21.15.42:0/4180009024 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7f78cc003910 con 0x55bc206631b0 2024-08-30T09:05:25.247 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.244+0000 7f78e8c34740 10 client.25963 fetch_fsmap finished waiting for FSMap version 30 2024-08-30T09:05:25.248 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.244+0000 7f78e8c34740 10 client.25963 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-30T09:05:25.248 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.244+0000 7f78e8c34740 20 client.25963 populate_metadata read hostname 'smithi042' 2024-08-30T09:05:25.248 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.244+0000 7f78e8c34740 1 --2- 172.21.15.42:0/4180009024 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55bc20621d80 0x55bc20a203e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:25.248 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.244+0000 7f78e8c34740 4 client.25963 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-30T09:05:25.248 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.244+0000 7f78e8c34740 1 -- 172.21.15.42:0/4180009024 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bc2097df50 con 0x55bc20621d80 2024-08-30T09:05:25.248 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.245+0000 7f78db7fe640 1 --2- 172.21.15.42:0/4180009024 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55bc20621d80 0x55bc20a203e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:05:25.253 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.251+0000 7f78db7fe640 1 --2- 172.21.15.42:0/4180009024 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55bc20621d80 0x55bc20a203e0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f6bb71df3d80d06 server_cookie=f1f159be70b436e0 in_seq=0 out_seq=0 2024-08-30T09:05:25.254 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.251+0000 7f78beffd640 10 client.25963 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:05:25.254 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.251+0000 7f78beffd640 1 -- 172.21.15.42:0/4180009024 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55bc2097df50 con 0x55bc20621d80 2024-08-30T09:05:25.255 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.251+0000 7f78beffd640 10 client.25963 handle_command_reply: tid=0 2024-08-30T09:05:25.255 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.251+0000 7f78e8c34740 1 client.25963 shutdown 2024-08-30T09:05:25.255 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.252+0000 7f78e8c34740 1 -- 172.21.15.42:0/4180009024 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bc206631b0 msgr2=0x55bc20a13bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:05:25.255 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.252+0000 7f78e8c34740 1 --2- 172.21.15.42:0/4180009024 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bc206631b0 0x55bc20a13bf0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f78cc030960 tx=0x7f78cc005f90 comp rx=0 tx=0).stop 2024-08-30T09:05:25.255 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.252+0000 7f78e8c34740 1 -- 172.21.15.42:0/4180009024 shutdown_connections 2024-08-30T09:05:25.255 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.252+0000 7f78e8c34740 1 --2- 172.21.15.42:0/4180009024 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55bc20621d80 0x55bc20a203e0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:25.255 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.252+0000 7f78e8c34740 1 --2- 172.21.15.42:0/4180009024 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bc206631b0 0x55bc20a13bf0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:25.256 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.252+0000 7f78e8c34740 1 --2- 172.21.15.42:0/4180009024 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bc206424d0 0x55bc20a13080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:25.256 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.252+0000 7f78e8c34740 1 --2- 172.21.15.42:0/4180009024 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bc2063a2f0 0x55bc20a129c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:25.256 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.252+0000 7f78e8c34740 1 -- 172.21.15.42:0/4180009024 >> 172.21.15.42:0/4180009024 conn(0x55bc20657300 msgr2=0x55bc20a02bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:05:25.256 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.252+0000 7f78beffd640 0 client.25963 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:05:25.258 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.255+0000 7f78e8c34740 1 -- 172.21.15.42:0/4180009024 shutdown_connections 2024-08-30T09:05:25.258 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.255+0000 7f78e8c34740 1 -- 172.21.15.42:0/4180009024 wait complete. 2024-08-30T09:05:25.258 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.255+0000 7f78e8c34740 20 client.25963 trim_cache size 0 max 16384 2024-08-30T09:05:25.288 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.285+0000 7f78e8c34740 1 Processor -- start 2024-08-30T09:05:25.288 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.285+0000 7f78e8c34740 1 -- start start 2024-08-30T09:05:25.289 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.285+0000 7f78e8c34740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bc2063a2f0 0x55bc20a09ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:25.289 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.286+0000 7f78e8c34740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bc206424d0 0x55bc20a0c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:25.289 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.286+0000 7f78db7fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bc2063a2f0 0x55bc20a09ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:05:25.290 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.286+0000 7f78db7fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bc2063a2f0 0x55bc20a09ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:33812/0 (socket says 172.21.15.42:33812) 2024-08-30T09:05:25.290 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.286+0000 7f78db7fe640 1 -- 172.21.15.42:0/2342583421 learned_addr learned my addr 172.21.15.42:0/2342583421 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:05:25.290 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.286+0000 7f78e8c34740 1 --2- 172.21.15.42:0/2342583421 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bc206631b0 0x55bc20a054f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:25.290 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.286+0000 7f78daffd640 1 --2- 172.21.15.42:0/2342583421 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bc206424d0 0x55bc20a0c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:05:25.290 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.286+0000 7f78e8c34740 1 -- 172.21.15.42:0/2342583421 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f78c40125f0 con 0x55bc206631b0 2024-08-30T09:05:25.290 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.287+0000 7f78e8c34740 1 -- 172.21.15.42:0/2342583421 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f78c400fec0 con 0x55bc206424d0 2024-08-30T09:05:25.290 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.287+0000 7f78e8c34740 1 -- 172.21.15.42:0/2342583421 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f78c400dd10 con 0x55bc2063a2f0 2024-08-30T09:05:25.290 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.287+0000 7f78dbfff640 1 --2- 172.21.15.42:0/2342583421 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bc206631b0 0x55bc20a054f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:05:25.291 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.287+0000 7f78db7fe640 1 -- 172.21.15.42:0/2342583421 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bc206424d0 msgr2=0x55bc20a0c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:05:25.291 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.287+0000 7f78db7fe640 1 --2- 172.21.15.42:0/2342583421 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bc206424d0 0x55bc20a0c070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:25.291 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.287+0000 7f78db7fe640 1 -- 172.21.15.42:0/2342583421 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bc206631b0 msgr2=0x55bc20a054f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:05:25.291 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.287+0000 7f78db7fe640 1 --2- 172.21.15.42:0/2342583421 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bc206631b0 0x55bc20a054f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:25.291 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.287+0000 7f78db7fe640 1 -- 172.21.15.42:0/2342583421 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc20a1ae90 con 0x55bc2063a2f0 2024-08-30T09:05:25.291 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.287+0000 7f78dbfff640 1 --2- 172.21.15.42:0/2342583421 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bc206631b0 0x55bc20a054f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-30T09:05:25.291 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.288+0000 7f78db7fe640 1 --2- 172.21.15.42:0/2342583421 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bc2063a2f0 0x55bc20a09ec0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f78d400bd40 tx=0x7f78d40093b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=82b46bb59b475494 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:05:25.291 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.288+0000 7f78d17fa640 1 -- 172.21.15.42:0/2342583421 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78d40b4430 con 0x55bc2063a2f0 2024-08-30T09:05:25.291 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.288+0000 7f78d17fa640 1 -- 172.21.15.42:0/2342583421 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f78d40d2170 con 0x55bc2063a2f0 2024-08-30T09:05:25.291 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.288+0000 7f78d17fa640 1 -- 172.21.15.42:0/2342583421 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78d40e2b30 con 0x55bc2063a2f0 2024-08-30T09:05:25.292 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.289+0000 7f78e8c34740 1 -- 172.21.15.42:0/2342583421 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bc2063a2f0 msgr2=0x55bc20a09ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:05:25.292 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.289+0000 7f78e8c34740 1 --2- 172.21.15.42:0/2342583421 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bc2063a2f0 0x55bc20a09ec0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f78d400bd40 tx=0x7f78d40093b0 comp rx=0 tx=0).stop 2024-08-30T09:05:25.293 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.290+0000 7f78e8c34740 1 -- 172.21.15.42:0/2342583421 shutdown_connections 2024-08-30T09:05:25.293 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.290+0000 7f78e8c34740 1 --2- 172.21.15.42:0/2342583421 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bc206631b0 0x55bc20a054f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:25.293 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.290+0000 7f78e8c34740 1 --2- 172.21.15.42:0/2342583421 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bc206424d0 0x55bc20a0c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:25.294 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.290+0000 7f78e8c34740 1 --2- 172.21.15.42:0/2342583421 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bc2063a2f0 0x55bc20a09ec0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:25.294 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.290+0000 7f78e8c34740 1 -- 172.21.15.42:0/2342583421 >> 172.21.15.42:0/2342583421 conn(0x55bc20657300 msgr2=0x55bc20a0c960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:05:25.294 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.290+0000 7f78e8c34740 1 -- 172.21.15.42:0/2342583421 shutdown_connections 2024-08-30T09:05:25.294 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.290+0000 7f78e8c34740 1 -- 172.21.15.42:0/2342583421 wait complete. 2024-08-30T09:05:25.294 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.291+0000 7f78e8c34740 1 -- start start 2024-08-30T09:05:25.294 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.291+0000 7f78e8c34740 1 Processor -- start 2024-08-30T09:05:25.296 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.292+0000 7f78e8c34740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bc2063a2f0 0x55bc20a1a660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:25.297 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.292+0000 7f78e8c34740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bc206424d0 0x55bc20a16470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:25.297 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.292+0000 7f78e8c34740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bc206631b0 0x55bc20a16a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:25.297 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.292+0000 7f78e8c34740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f78c400e380 con 0x55bc2063a2f0 2024-08-30T09:05:25.297 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.292+0000 7f78e8c34740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f78c400f7f0 con 0x55bc206631b0 2024-08-30T09:05:25.297 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.292+0000 7f78e8c34740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f78c400ea50 con 0x55bc206424d0 2024-08-30T09:05:25.297 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.292+0000 7f78daffd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bc206424d0 0x55bc20a16470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:05:25.297 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.292+0000 7f78daffd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bc206424d0 0x55bc20a16470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:33818/0 (socket says 172.21.15.42:33818) 2024-08-30T09:05:25.297 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.292+0000 7f78daffd640 1 -- 172.21.15.42:0/176603736 learned_addr learned my addr 172.21.15.42:0/176603736 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:05:25.297 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.293+0000 7f78e8c34740 10 client.0 _ll_register_callbacks cb 0x7f78cc0096b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:05:25.298 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.293+0000 7f78dbfff640 1 --2- 172.21.15.42:0/176603736 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bc206631b0 0x55bc20a16a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:05:25.298 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.293+0000 7f78daffd640 1 -- 172.21.15.42:0/176603736 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bc206631b0 msgr2=0x55bc20a16a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:05:25.298 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.293+0000 7f78daffd640 1 --2- 172.21.15.42:0/176603736 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bc206631b0 0x55bc20a16a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:25.298 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.293+0000 7f78daffd640 1 -- 172.21.15.42:0/176603736 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bc2063a2f0 msgr2=0x55bc20a1a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:05:25.298 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.293+0000 7f78daffd640 1 --2- 172.21.15.42:0/176603736 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bc2063a2f0 0x55bc20a1a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:25.298 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.293+0000 7f78daffd640 1 -- 172.21.15.42:0/176603736 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc20a175e0 con 0x55bc206424d0 2024-08-30T09:05:25.298 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.293+0000 7f78dbfff640 1 --2- 172.21.15.42:0/176603736 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bc206631b0 0x55bc20a16a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-30T09:05:25.298 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.293+0000 7f78daffd640 1 --2- 172.21.15.42:0/176603736 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bc206424d0 0x55bc20a16470 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f78c8022420 tx=0x7f78c80228e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=447dc210a6a9d71f server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:05:25.298 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.293+0000 7f78beffd640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-08-30T09:05:25.298 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.293+0000 7f78beffd640 1 -- 172.21.15.42:0/176603736 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78c8004030 con 0x55bc206424d0 2024-08-30T09:05:25.298 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.294+0000 7f78e8c34740 1 -- 172.21.15.42:0/176603736 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f78d40d2170 con 0x55bc206424d0 2024-08-30T09:05:25.298 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.294+0000 7f78beffd640 1 -- 172.21.15.42:0/176603736 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f78c8019040 con 0x55bc206424d0 2024-08-30T09:05:25.299 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.294+0000 7f78beffd640 1 -- 172.21.15.42:0/176603736 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f78c8036050 con 0x55bc206424d0 2024-08-30T09:05:25.300 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.294+0000 7f78e8c34740 10 client.25969 fetch_fsmap learned FSMap version 30 2024-08-30T09:05:25.300 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.294+0000 7f78e8c34740 1 -- 172.21.15.42:0/176603736 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55bc20a8b0e0 con 0x55bc206424d0 2024-08-30T09:05:25.300 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.294+0000 7f78beffd640 1 -- 172.21.15.42:0/176603736 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f78d40d2170 con 0x55bc206424d0 2024-08-30T09:05:25.300 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.295+0000 7f78beffd640 1 -- 172.21.15.42:0/176603736 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7f78c801f040 con 0x55bc206424d0 2024-08-30T09:05:25.300 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.295+0000 7f78e8c34740 10 client.25969 fetch_fsmap finished waiting for FSMap version 30 2024-08-30T09:05:25.300 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.295+0000 7f78e8c34740 10 client.25969 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-30T09:05:25.300 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.295+0000 7f78e8c34740 20 client.25969 populate_metadata read hostname 'smithi042' 2024-08-30T09:05:25.300 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.295+0000 7f78e8c34740 1 --2- 172.21.15.42:0/176603736 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55bc205f71c0 0x55bc20a8dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:25.300 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.295+0000 7f78e8c34740 4 client.25969 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-30T09:05:25.300 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.295+0000 7f78e8c34740 1 -- 172.21.15.42:0/176603736 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55bc20a0a5a0 con 0x55bc205f71c0 2024-08-30T09:05:25.300 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.295+0000 7f78db7fe640 1 --2- 172.21.15.42:0/176603736 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55bc205f71c0 0x55bc20a8dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:05:25.301 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.295+0000 7f78beffd640 10 client.25969 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:05:25.301 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.296+0000 7f78db7fe640 1 --2- 172.21.15.42:0/176603736 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55bc205f71c0 0x55bc20a8dbd0 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=87517b0a4f86053c server_cookie=9fd0f838216f194f in_seq=0 out_seq=0 2024-08-30T09:05:25.320 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.316+0000 7f78beffd640 1 -- 172.21.15.42:0/176603736 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55bc20a0a5a0 con 0x55bc205f71c0 2024-08-30T09:05:25.320 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.316+0000 7f78beffd640 10 client.25969 handle_command_reply: tid=0 2024-08-30T09:05:25.320 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.316+0000 7f78e8c34740 1 client.25969 shutdown 2024-08-30T09:05:25.320 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.316+0000 7f78e8c34740 1 -- 172.21.15.42:0/176603736 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bc206424d0 msgr2=0x55bc20a16470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:05:25.320 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.316+0000 7f78e8c34740 1 --2- 172.21.15.42:0/176603736 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bc206424d0 0x55bc20a16470 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f78c8022420 tx=0x7f78c80228e0 comp rx=0 tx=0).stop 2024-08-30T09:05:25.320 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.317+0000 7f78e8c34740 1 -- 172.21.15.42:0/176603736 shutdown_connections 2024-08-30T09:05:25.320 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.317+0000 7f78e8c34740 1 --2- 172.21.15.42:0/176603736 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55bc205f71c0 0x55bc20a8dbd0 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:25.320 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.317+0000 7f78e8c34740 1 --2- 172.21.15.42:0/176603736 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bc206424d0 0x55bc20a16470 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:25.320 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.317+0000 7f78e8c34740 1 --2- 172.21.15.42:0/176603736 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bc206631b0 0x55bc20a16a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:25.321 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.317+0000 7f78beffd640 0 client.25969 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:05:25.321 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.317+0000 7f78e8c34740 1 --2- 172.21.15.42:0/176603736 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bc2063a2f0 0x55bc20a1a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:25.321 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.317+0000 7f78e8c34740 1 -- 172.21.15.42:0/176603736 >> 172.21.15.42:0/176603736 conn(0x55bc20657300 msgr2=0x55bc20a0d6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:05:25.321 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.317+0000 7f78e8c34740 1 -- 172.21.15.42:0/176603736 shutdown_connections 2024-08-30T09:05:25.321 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.317+0000 7f78e8c34740 1 -- 172.21.15.42:0/176603736 wait complete. 2024-08-30T09:05:25.321 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.317+0000 7f78e8c34740 20 client.25969 trim_cache size 0 max 16384 2024-08-30T09:05:25.321 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-08-30T09:05:25.322 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2024-08-30T09:05:25.322 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-08-30T09:05:25.322 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-08-30T09:05:25.323 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.317+0000 7f78e1bf1640 1 -- 172.21.15.42:0/2248446315 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f78c00b4d40 msgr2=0x7f78c00b7250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:05:25.323 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.318+0000 7f78e1bf1640 1 --2- 172.21.15.42:0/2248446315 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f78c00b4d40 0x7f78c00b7250 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f78cc005eb0 tx=0x7f78cc030bb0 comp rx=0 tx=0).stop 2024-08-30T09:05:25.323 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.318+0000 7f78e1bf1640 1 -- 172.21.15.42:0/2248446315 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f78dc10a160 msgr2=0x7f78dc1a03b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:05:25.323 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.318+0000 7f78e1bf1640 1 --2- 172.21.15.42:0/2248446315 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f78dc10a160 0x7f78dc1a03b0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f78d400bd70 tx=0x7f78d400be60 comp rx=0 tx=0).stop 2024-08-30T09:05:25.323 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.319+0000 7f78e1bf1640 1 -- 172.21.15.42:0/2248446315 shutdown_connections 2024-08-30T09:05:25.323 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.319+0000 7f78e1bf1640 1 --2- 172.21.15.42:0/2248446315 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f78c00b4d40 0x7f78c00b7250 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:25.323 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.319+0000 7f78e1bf1640 1 --2- 172.21.15.42:0/2248446315 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f78dc10f770 0x7f78dc1a15a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:25.323 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.319+0000 7f78e1bf1640 1 --2- 172.21.15.42:0/2248446315 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f78dc10cb50 0x7f78dc1a0a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:25.324 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.319+0000 7f78e1bf1640 1 --2- 172.21.15.42:0/2248446315 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f78dc10a160 0x7f78dc1a03b0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:25.324 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.319+0000 7f78e1bf1640 1 -- 172.21.15.42:0/2248446315 >> 172.21.15.42:0/2248446315 conn(0x7f78dc0705b0 msgr2=0x7f78dc10b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:05:25.324 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.319+0000 7f78e1bf1640 1 -- 172.21.15.42:0/2248446315 shutdown_connections 2024-08-30T09:05:25.324 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:25.319+0000 7f78e1bf1640 1 -- 172.21.15.42:0/2248446315 wait complete. 2024-08-30T09:05:25.953 INFO:teuthology.orchestra.run.smithi042.stderr:progress: 280.0 s, 813.2 tps, lat 121.469 ms stddev 129.364 2024-08-30T09:05:27.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:05:26 smithi167 ceph-mon[22063]: pgmap v513: 129 pgs: 129 active+clean; 1.2 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 6.2 MiB/s wr, 558 op/s 2024-08-30T09:05:27.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:05:26 smithi042 ceph-mon[18652]: pgmap v513: 129 pgs: 129 active+clean; 1.2 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 6.2 MiB/s wr, 558 op/s 2024-08-30T09:05:27.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:05:26 smithi190 ceph-mon[22001]: pgmap v513: 129 pgs: 129 active+clean; 1.2 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 6.2 MiB/s wr, 558 op/s 2024-08-30T09:05:29.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:05:28 smithi167 ceph-mon[22063]: pgmap v514: 129 pgs: 129 active+clean; 1.2 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.7 MiB/s wr, 596 op/s 2024-08-30T09:05:29.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:05:28 smithi167 ceph-mon[22063]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-30T09:05:29.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:05:28 smithi042 ceph-mon[18652]: pgmap v514: 129 pgs: 129 active+clean; 1.2 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.7 MiB/s wr, 596 op/s 2024-08-30T09:05:29.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:05:28 smithi042 ceph-mon[18652]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-30T09:05:29.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:05:28 smithi190 ceph-mon[22001]: pgmap v514: 129 pgs: 129 active+clean; 1.2 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.7 MiB/s wr, 596 op/s 2024-08-30T09:05:29.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:05:28 smithi190 ceph-mon[22001]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-30T09:05:30.953 INFO:teuthology.orchestra.run.smithi042.stderr:progress: 285.0 s, 798.8 tps, lat 124.372 ms stddev 132.999 2024-08-30T09:05:31.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:05:30 smithi167 ceph-mon[22063]: pgmap v515: 129 pgs: 129 active+clean; 1.2 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.2 MiB/s wr, 590 op/s 2024-08-30T09:05:31.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:05:30 smithi042 ceph-mon[18652]: pgmap v515: 129 pgs: 129 active+clean; 1.2 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.2 MiB/s wr, 590 op/s 2024-08-30T09:05:31.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:05:30 smithi190 ceph-mon[22001]: pgmap v515: 129 pgs: 129 active+clean; 1.2 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.2 MiB/s wr, 590 op/s 2024-08-30T09:05:33.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:05:32 smithi167 ceph-mon[22063]: pgmap v516: 129 pgs: 129 active+clean; 1.2 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 6.0 MiB/s wr, 549 op/s 2024-08-30T09:05:33.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:05:32 smithi042 ceph-mon[18652]: pgmap v516: 129 pgs: 129 active+clean; 1.2 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 6.0 MiB/s wr, 549 op/s 2024-08-30T09:05:33.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:05:32 smithi190 ceph-mon[22001]: pgmap v516: 129 pgs: 129 active+clean; 1.2 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 6.0 MiB/s wr, 549 op/s 2024-08-30T09:05:33.935 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-30T09:05:34.346 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T09:05:35.024 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:05:34 smithi042 ceph-mon[18652]: pgmap v517: 129 pgs: 129 active+clean; 1.2 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 5.9 MiB/s wr, 673 op/s 2024-08-30T09:05:35.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.069+0000 7f4ecee4d640 1 -- 172.21.15.42:0/2948892833 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4ec811c9d0 msgr2=0x7f4ec811eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:05:35.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.069+0000 7f4ecee4d640 1 --2- 172.21.15.42:0/2948892833 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4ec811c9d0 0x7f4ec811eee0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f4eb8009900 tx=0x7f4eb802a420 comp rx=0 tx=0).stop 2024-08-30T09:05:35.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.071+0000 7f4ecee4d640 1 -- 172.21.15.42:0/2948892833 shutdown_connections 2024-08-30T09:05:35.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.071+0000 7f4ecee4d640 1 --2- 172.21.15.42:0/2948892833 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4ec811c9d0 0x7f4ec811eee0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:35.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.071+0000 7f4ecee4d640 1 --2- 172.21.15.42:0/2948892833 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4ec810a780 0x7f4ec810ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:35.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.071+0000 7f4ecee4d640 1 --2- 172.21.15.42:0/2948892833 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4ec8111ce0 0x7f4ec810a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:35.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.071+0000 7f4ecee4d640 1 -- 172.21.15.42:0/2948892833 >> 172.21.15.42:0/2948892833 conn(0x7f4ec8070c60 msgr2=0x7f4ec8071070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:05:35.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.071+0000 7f4ecee4d640 1 -- 172.21.15.42:0/2948892833 shutdown_connections 2024-08-30T09:05:35.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.073+0000 7f4ecee4d640 1 -- 172.21.15.42:0/2948892833 wait complete. 2024-08-30T09:05:35.076 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:05:34 smithi167 ceph-mon[22063]: pgmap v517: 129 pgs: 129 active+clean; 1.2 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 5.9 MiB/s wr, 673 op/s 2024-08-30T09:05:35.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.074+0000 7f4ecee4d640 1 Processor -- start 2024-08-30T09:05:35.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.077+0000 7f4ecee4d640 1 -- start start 2024-08-30T09:05:35.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.077+0000 7f4ecee4d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4ec810a780 0x7f4ec811a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:35.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.078+0000 7f4ecee4d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4ec8111ce0 0x7f4ec8113430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:35.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.078+0000 7f4eccbe9640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4ec810a780 0x7f4ec811a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:05:35.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.078+0000 7f4eccbe9640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4ec810a780 0x7f4ec811a160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:39368/0 (socket says 172.21.15.42:39368) 2024-08-30T09:05:35.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.078+0000 7f4ecee4d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4ec811c9d0 0x7f4ec8113970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:35.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.078+0000 7f4eccbe9640 1 -- 172.21.15.42:0/432751619 learned_addr learned my addr 172.21.15.42:0/432751619 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:05:35.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.078+0000 7f4ecee4d640 1 -- 172.21.15.42:0/432751619 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f4ec81226b0 con 0x7f4ec811c9d0 2024-08-30T09:05:35.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.078+0000 7f4ecee4d640 1 -- 172.21.15.42:0/432751619 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4ec8122530 con 0x7f4ec810a780 2024-08-30T09:05:35.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.078+0000 7f4ecee4d640 1 -- 172.21.15.42:0/432751619 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f4ec8122830 con 0x7f4ec8111ce0 2024-08-30T09:05:35.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.078+0000 7f4ec7fff640 1 --2- 172.21.15.42:0/432751619 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4ec8111ce0 0x7f4ec8113430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:05:35.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.079+0000 7f4ec7fff640 1 -- 172.21.15.42:0/432751619 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4ec810a780 msgr2=0x7f4ec811a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:05:35.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.079+0000 7f4ecd3ea640 1 --2- 172.21.15.42:0/432751619 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4ec811c9d0 0x7f4ec8113970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:05:35.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.079+0000 7f4ec7fff640 1 --2- 172.21.15.42:0/432751619 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4ec810a780 0x7f4ec811a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:35.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.079+0000 7f4ec7fff640 1 -- 172.21.15.42:0/432751619 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4ec811c9d0 msgr2=0x7f4ec8113970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:05:35.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.079+0000 7f4ec7fff640 1 --2- 172.21.15.42:0/432751619 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4ec811c9d0 0x7f4ec8113970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:35.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.079+0000 7f4ec7fff640 1 -- 172.21.15.42:0/432751619 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ec8114190 con 0x7f4ec8111ce0 2024-08-30T09:05:35.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.079+0000 7f4eccbe9640 1 --2- 172.21.15.42:0/432751619 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4ec810a780 0x7f4ec811a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-30T09:05:35.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.079+0000 7f4ecd3ea640 1 --2- 172.21.15.42:0/432751619 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4ec811c9d0 0x7f4ec8113970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-30T09:05:35.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.079+0000 7f4ec7fff640 1 --2- 172.21.15.42:0/432751619 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4ec8111ce0 0x7f4ec8113430 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f4ec806c480 tx=0x7f4ec000be00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:05:35.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.080+0000 7f4ec5ffb640 1 -- 172.21.15.42:0/432751619 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ec0019070 con 0x7f4ec8111ce0 2024-08-30T09:05:35.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.080+0000 7f4ecee4d640 1 -- 172.21.15.42:0/432751619 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ec8118e30 con 0x7f4ec8111ce0 2024-08-30T09:05:35.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.080+0000 7f4ecee4d640 1 -- 172.21.15.42:0/432751619 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ec81193f0 con 0x7f4ec8111ce0 2024-08-30T09:05:35.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.080+0000 7f4ec5ffb640 1 -- 172.21.15.42:0/432751619 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ec00148f0 con 0x7f4ec8111ce0 2024-08-30T09:05:35.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.080+0000 7f4ec5ffb640 1 -- 172.21.15.42:0/432751619 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ec0012860 con 0x7f4ec8111ce0 2024-08-30T09:05:35.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.089+0000 7f4ec5ffb640 1 -- 172.21.15.42:0/432751619 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7f4ec001c070 con 0x7f4ec8111ce0 2024-08-30T09:05:35.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.090+0000 7f4ec5ffb640 1 --2- 172.21.15.42:0/432751619 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f4eb00b4d40 0x7f4eb00b7250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:35.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.090+0000 7f4ec5ffb640 1 -- 172.21.15.42:0/432751619 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f4ec00e49f0 con 0x7f4ec8111ce0 2024-08-30T09:05:35.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.091+0000 7f4ed53c2740 1 Processor -- start 2024-08-30T09:05:35.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.091+0000 7f4ed53c2740 1 -- start start 2024-08-30T09:05:35.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.091+0000 7f4eccbe9640 1 --2- 172.21.15.42:0/432751619 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f4eb00b4d40 0x7f4eb00b7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:05:35.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.091+0000 7f4ed53c2740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563408236070 0x5634085db300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:35.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.091+0000 7f4ed53c2740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56340820d1b0 0x5634085db840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:35.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.091+0000 7f4ed53c2740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563408215390 0x5634085dbd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:35.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.091+0000 7f4ed53c2740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x563408552630 con 0x563408215390 2024-08-30T09:05:35.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.091+0000 7f4ed53c2740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x563408541bc0 con 0x56340820d1b0 2024-08-30T09:05:35.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.091+0000 7f4ed53c2740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x563408545b70 con 0x563408236070 2024-08-30T09:05:35.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.092+0000 7f4ecd3ea640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563408215390 0x5634085dbd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:05:35.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.092+0000 7f4ecd3ea640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563408215390 0x5634085dbd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37910/0 (socket says 172.21.15.42:37910) 2024-08-30T09:05:35.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.092+0000 7f4ecd3ea640 1 -- 172.21.15.42:0/3973232788 learned_addr learned my addr 172.21.15.42:0/3973232788 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:05:35.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.092+0000 7f4ecd3ea640 1 -- 172.21.15.42:0/3973232788 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563408236070 msgr2=0x5634085db300 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-30T09:05:35.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.092+0000 7f4ecd3ea640 1 --2- 172.21.15.42:0/3973232788 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563408236070 0x5634085db300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:35.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.092+0000 7f4ecd3ea640 1 -- 172.21.15.42:0/3973232788 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56340820d1b0 msgr2=0x5634085db840 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-30T09:05:35.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.092+0000 7f4ecd3ea640 1 --2- 172.21.15.42:0/3973232788 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56340820d1b0 0x5634085db840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:35.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.092+0000 7f4ecd3ea640 1 -- 172.21.15.42:0/3973232788 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5634082a1650 con 0x563408215390 2024-08-30T09:05:35.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.093+0000 7f4ecd3ea640 1 --2- 172.21.15.42:0/3973232788 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563408215390 0x5634085dbd80 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f4eb8006010 tx=0x7f4eb8004260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ffb9e0ce09eb5fe server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:05:35.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.093+0000 7f4eaeffd640 1 -- 172.21.15.42:0/3973232788 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eb803f070 con 0x563408215390 2024-08-30T09:05:35.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.093+0000 7f4eaeffd640 1 -- 172.21.15.42:0/3973232788 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4eb80054f0 con 0x563408215390 2024-08-30T09:05:35.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.094+0000 7f4eaeffd640 1 -- 172.21.15.42:0/3973232788 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eb8035030 con 0x563408215390 2024-08-30T09:05:35.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.094+0000 7f4ed53c2740 1 -- 172.21.15.42:0/3973232788 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563408215390 msgr2=0x5634085dbd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:05:35.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.094+0000 7f4ed53c2740 1 --2- 172.21.15.42:0/3973232788 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563408215390 0x5634085dbd80 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f4eb8006010 tx=0x7f4eb8004260 comp rx=0 tx=0).stop 2024-08-30T09:05:35.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.094+0000 7f4ed53c2740 1 -- 172.21.15.42:0/3973232788 shutdown_connections 2024-08-30T09:05:35.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.094+0000 7f4ed53c2740 1 --2- 172.21.15.42:0/3973232788 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563408215390 0x5634085dbd80 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:35.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.094+0000 7f4ed53c2740 1 --2- 172.21.15.42:0/3973232788 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56340820d1b0 0x5634085db840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:35.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.094+0000 7f4ed53c2740 1 --2- 172.21.15.42:0/3973232788 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563408236070 0x5634085db300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:35.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.094+0000 7f4ed53c2740 1 -- 172.21.15.42:0/3973232788 >> 172.21.15.42:0/3973232788 conn(0x56340822a460 msgr2=0x5634085d91f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:05:35.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.095+0000 7f4eccbe9640 1 --2- 172.21.15.42:0/432751619 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f4eb00b4d40 0x7f4eb00b7250 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f4ebc011550 tx=0x7f4ebc005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:05:35.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.095+0000 7f4ed53c2740 1 -- 172.21.15.42:0/3973232788 shutdown_connections 2024-08-30T09:05:35.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.095+0000 7f4ed53c2740 1 -- 172.21.15.42:0/3973232788 wait complete. 2024-08-30T09:05:35.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.096+0000 7f4ed53c2740 1 -- start start 2024-08-30T09:05:35.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.096+0000 7f4ed53c2740 1 Processor -- start 2024-08-30T09:05:35.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.096+0000 7f4ed53c2740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56340820d1b0 0x5634085e6a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:35.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.097+0000 7f4ed53c2740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563408215390 0x5634085e6f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:35.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.097+0000 7f4eccbe9640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56340820d1b0 0x5634085e6a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:05:35.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.097+0000 7f4eccbe9640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56340820d1b0 0x5634085e6a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:39384/0 (socket says 172.21.15.42:39384) 2024-08-30T09:05:35.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.097+0000 7f4eccbe9640 1 -- 172.21.15.42:0/4102099351 learned_addr learned my addr 172.21.15.42:0/4102099351 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:05:35.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.097+0000 7f4ed53c2740 1 --2- 172.21.15.42:0/4102099351 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5634081f49f0 0x5634085f2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:35.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.097+0000 7f4ed53c2740 1 -- 172.21.15.42:0/4102099351 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x5634083a4700 con 0x563408215390 2024-08-30T09:05:35.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.097+0000 7f4ed53c2740 1 -- 172.21.15.42:0/4102099351 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5634084c8700 con 0x56340820d1b0 2024-08-30T09:05:35.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.097+0000 7f4ed53c2740 1 -- 172.21.15.42:0/4102099351 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5634085e9b20 con 0x5634081f49f0 2024-08-30T09:05:35.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.097+0000 7f4ed53c2740 10 client.0 _ll_register_callbacks cb 0x5634084f7dc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:05:35.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.097+0000 7f4eccbe9640 1 -- 172.21.15.42:0/4102099351 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5634081f49f0 msgr2=0x5634085f2ef0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-30T09:05:35.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.097+0000 7f4eccbe9640 1 --2- 172.21.15.42:0/4102099351 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5634081f49f0 0x5634085f2ef0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:35.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.097+0000 7f4eccbe9640 1 -- 172.21.15.42:0/4102099351 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563408215390 msgr2=0x5634085e6f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:05:35.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.097+0000 7f4eccbe9640 1 --2- 172.21.15.42:0/4102099351 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563408215390 0x5634085e6f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:35.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.097+0000 7f4eccbe9640 1 -- 172.21.15.42:0/4102099351 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5634085018b0 con 0x56340820d1b0 2024-08-30T09:05:35.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.098+0000 7f4eccbe9640 1 --2- 172.21.15.42:0/4102099351 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56340820d1b0 0x5634085e6a50 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f4ebc009560 tx=0x7f4ebc004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=879f9b8160b84cf2 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:05:35.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.098+0000 7f4ea7fff640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-30T09:05:35.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.099+0000 7f4ea7fff640 1 -- 172.21.15.42:0/4102099351 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ebc02b060 con 0x56340820d1b0 2024-08-30T09:05:35.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.099+0000 7f4ea7fff640 1 -- 172.21.15.42:0/4102099351 <== mon.2 v2:172.21.15.167:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f4ebc01cdd0 con 0x56340820d1b0 2024-08-30T09:05:35.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.100+0000 7f4ed53c2740 1 -- 172.21.15.42:0/4102099351 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4eb80054f0 con 0x56340820d1b0 2024-08-30T09:05:35.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.100+0000 7f4ea7fff640 1 -- 172.21.15.42:0/4102099351 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f4ebc032050 con 0x56340820d1b0 2024-08-30T09:05:35.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.100+0000 7f4ed53c2740 10 client.25970 fetch_fsmap learned FSMap version 31 2024-08-30T09:05:35.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.100+0000 7f4ed53c2740 1 -- 172.21.15.42:0/4102099351 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5634085f5960 con 0x56340820d1b0 2024-08-30T09:05:35.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.100+0000 7f4ea7fff640 1 -- 172.21.15.42:0/4102099351 <== mon.2 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4eb80054f0 con 0x56340820d1b0 2024-08-30T09:05:35.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.100+0000 7f4ea7fff640 1 -- 172.21.15.42:0/4102099351 <== mon.2 v2:172.21.15.167:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f4ebc00e4d0 con 0x56340820d1b0 2024-08-30T09:05:35.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.100+0000 7f4ed53c2740 10 client.25970 fetch_fsmap finished waiting for FSMap version 31 2024-08-30T09:05:35.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.100+0000 7f4ed53c2740 10 client.25970 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-30T09:05:35.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.100+0000 7f4ed53c2740 20 client.25970 populate_metadata read hostname 'smithi042' 2024-08-30T09:05:35.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.100+0000 7f4ed53c2740 1 --2- 172.21.15.42:0/4102099351 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x5634081c9da0 0x5634085f93a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:35.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.101+0000 7f4ed53c2740 4 client.25970 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-30T09:05:35.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.101+0000 7f4ed53c2740 1 -- 172.21.15.42:0/4102099351 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563408554580 con 0x5634081c9da0 2024-08-30T09:05:35.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.101+0000 7f4ec7fff640 1 --2- 172.21.15.42:0/4102099351 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x5634081c9da0 0x5634085f93a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:05:35.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.102+0000 7f4ea7fff640 10 client.25970 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:05:35.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.103+0000 7f4ec7fff640 1 --2- 172.21.15.42:0/4102099351 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x5634081c9da0 0x5634085f93a0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc31a9dc5049bd78 server_cookie=8a90c9982505679d in_seq=0 out_seq=0 2024-08-30T09:05:35.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.105+0000 7f4ea7fff640 1 -- 172.21.15.42:0/4102099351 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x563408554580 con 0x5634081c9da0 2024-08-30T09:05:35.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.105+0000 7f4ea7fff640 10 client.25970 handle_command_reply: tid=0 2024-08-30T09:05:35.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.105+0000 7f4ed53c2740 1 client.25970 shutdown 2024-08-30T09:05:35.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.105+0000 7f4ed53c2740 1 -- 172.21.15.42:0/4102099351 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56340820d1b0 msgr2=0x5634085e6a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:05:35.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.105+0000 7f4ed53c2740 1 --2- 172.21.15.42:0/4102099351 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56340820d1b0 0x5634085e6a50 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f4ebc009560 tx=0x7f4ebc004730 comp rx=0 tx=0).stop 2024-08-30T09:05:35.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.105+0000 7f4ed53c2740 1 -- 172.21.15.42:0/4102099351 shutdown_connections 2024-08-30T09:05:35.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.105+0000 7f4ed53c2740 1 --2- 172.21.15.42:0/4102099351 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x5634081c9da0 0x5634085f93a0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:35.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.105+0000 7f4ed53c2740 1 --2- 172.21.15.42:0/4102099351 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5634081f49f0 0x5634085f2ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:35.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.105+0000 7f4ed53c2740 1 --2- 172.21.15.42:0/4102099351 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563408215390 0x5634085e6f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:35.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.105+0000 7f4ed53c2740 1 --2- 172.21.15.42:0/4102099351 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56340820d1b0 0x5634085e6a50 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:35.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.105+0000 7f4ed53c2740 1 -- 172.21.15.42:0/4102099351 >> 172.21.15.42:0/4102099351 conn(0x56340822a460 msgr2=0x5634085da2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:05:35.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.105+0000 7f4ea7fff640 0 client.25970 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:05:35.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.106+0000 7f4ed53c2740 1 -- 172.21.15.42:0/4102099351 shutdown_connections 2024-08-30T09:05:35.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.106+0000 7f4ed53c2740 1 -- 172.21.15.42:0/4102099351 wait complete. 2024-08-30T09:05:35.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.106+0000 7f4ed53c2740 20 client.25970 trim_cache size 0 max 16384 2024-08-30T09:05:35.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.133+0000 7f4ed53c2740 1 Processor -- start 2024-08-30T09:05:35.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.133+0000 7f4ed53c2740 1 -- start start 2024-08-30T09:05:35.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.133+0000 7f4ed53c2740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5634081f49f0 0x5634085d7af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:35.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.133+0000 7f4ed53c2740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56340820d1b0 0x5634085e3dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:35.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.133+0000 7f4ed53c2740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563408215390 0x5634085e4300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:35.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.133+0000 7f4ed53c2740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f4eb4012550 con 0x56340820d1b0 2024-08-30T09:05:35.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.133+0000 7f4ed53c2740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4eb400fe20 con 0x5634081f49f0 2024-08-30T09:05:35.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.133+0000 7f4ed53c2740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f4eb400dc70 con 0x563408215390 2024-08-30T09:05:35.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.133+0000 7f4eccbe9640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5634081f49f0 0x5634085d7af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:05:35.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.133+0000 7f4ec7fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56340820d1b0 0x5634085e3dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:05:35.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.133+0000 7f4ecd3ea640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563408215390 0x5634085e4300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:05:35.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.133+0000 7f4eccbe9640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5634081f49f0 0x5634085d7af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:39398/0 (socket says 172.21.15.42:39398) 2024-08-30T09:05:35.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.133+0000 7f4eccbe9640 1 -- 172.21.15.42:0/3171260308 learned_addr learned my addr 172.21.15.42:0/3171260308 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:05:35.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.134+0000 7f4ecd3ea640 1 -- 172.21.15.42:0/3171260308 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5634081f49f0 msgr2=0x5634085d7af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:05:35.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.134+0000 7f4ecd3ea640 1 --2- 172.21.15.42:0/3171260308 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5634081f49f0 0x5634085d7af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:35.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.134+0000 7f4ecd3ea640 1 -- 172.21.15.42:0/3171260308 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56340820d1b0 msgr2=0x5634085e3dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:05:35.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.134+0000 7f4ecd3ea640 1 --2- 172.21.15.42:0/3171260308 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56340820d1b0 0x5634085e3dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:35.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.134+0000 7f4ecd3ea640 1 -- 172.21.15.42:0/3171260308 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5634085d81d0 con 0x563408215390 2024-08-30T09:05:35.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.134+0000 7f4eccbe9640 1 --2- 172.21.15.42:0/3171260308 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5634081f49f0 0x5634085d7af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-30T09:05:35.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.134+0000 7f4ecd3ea640 1 --2- 172.21.15.42:0/3171260308 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563408215390 0x5634085e4300 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f4eb8031440 tx=0x7f4eb8004610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e3174a558fc8f84a server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:05:35.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.134+0000 7f4eae7fc640 1 -- 172.21.15.42:0/3171260308 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eb804d070 con 0x563408215390 2024-08-30T09:05:35.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.134+0000 7f4eae7fc640 1 -- 172.21.15.42:0/3171260308 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4eb80046c0 con 0x563408215390 2024-08-30T09:05:35.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.134+0000 7f4eae7fc640 1 -- 172.21.15.42:0/3171260308 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eb803f070 con 0x563408215390 2024-08-30T09:05:35.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.135+0000 7f4ed53c2740 1 -- 172.21.15.42:0/3171260308 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563408215390 msgr2=0x5634085e4300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:05:35.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.135+0000 7f4ed53c2740 1 --2- 172.21.15.42:0/3171260308 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563408215390 0x5634085e4300 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f4eb8031440 tx=0x7f4eb8004610 comp rx=0 tx=0).stop 2024-08-30T09:05:35.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.135+0000 7f4ed53c2740 1 -- 172.21.15.42:0/3171260308 shutdown_connections 2024-08-30T09:05:35.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.135+0000 7f4ed53c2740 1 --2- 172.21.15.42:0/3171260308 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563408215390 0x5634085e4300 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:35.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.135+0000 7f4ed53c2740 1 --2- 172.21.15.42:0/3171260308 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56340820d1b0 0x5634085e3dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:35.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.135+0000 7f4ed53c2740 1 --2- 172.21.15.42:0/3171260308 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5634081f49f0 0x5634085d7af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:35.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.135+0000 7f4ed53c2740 1 -- 172.21.15.42:0/3171260308 >> 172.21.15.42:0/3171260308 conn(0x56340822a460 msgr2=0x5634085e52d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:05:35.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.135+0000 7f4ed53c2740 1 -- 172.21.15.42:0/3171260308 shutdown_connections 2024-08-30T09:05:35.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.135+0000 7f4ed53c2740 1 -- 172.21.15.42:0/3171260308 wait complete. 2024-08-30T09:05:35.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.136+0000 7f4ed53c2740 1 -- start start 2024-08-30T09:05:35.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.136+0000 7f4ed53c2740 1 Processor -- start 2024-08-30T09:05:35.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.137+0000 7f4ed53c2740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5634081f49f0 0x5634085ef390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:35.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.137+0000 7f4ed53c2740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56340820d1b0 0x5634085ea8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:35.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.137+0000 7f4ed53c2740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563408215390 0x5634085eae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:35.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.137+0000 7f4ed53c2740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f4eb400e2e0 con 0x56340820d1b0 2024-08-30T09:05:35.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.137+0000 7f4ed53c2740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4eb400f750 con 0x5634081f49f0 2024-08-30T09:05:35.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.137+0000 7f4ed53c2740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f4eb400e9b0 con 0x563408215390 2024-08-30T09:05:35.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.138+0000 7f4ed53c2740 10 client.0 _ll_register_callbacks cb 0x7f4ebc009e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:05:35.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.138+0000 7f4ec7fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56340820d1b0 0x5634085ea8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:05:35.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.138+0000 7f4ec7fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56340820d1b0 0x5634085ea8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37956/0 (socket says 172.21.15.42:37956) 2024-08-30T09:05:35.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.138+0000 7f4ec7fff640 1 -- 172.21.15.42:0/1553482030 learned_addr learned my addr 172.21.15.42:0/1553482030 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:05:35.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.137+0000 7f4eccbe9640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5634081f49f0 0x5634085ef390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:05:35.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.138+0000 7f4ecd3ea640 1 --2- 172.21.15.42:0/1553482030 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563408215390 0x5634085eae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:05:35.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.138+0000 7f4ec7fff640 1 -- 172.21.15.42:0/1553482030 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563408215390 msgr2=0x5634085eae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:05:35.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.138+0000 7f4ec7fff640 1 --2- 172.21.15.42:0/1553482030 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563408215390 0x5634085eae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:35.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.138+0000 7f4ec7fff640 1 -- 172.21.15.42:0/1553482030 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5634081f49f0 msgr2=0x5634085ef390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:05:35.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.138+0000 7f4ec7fff640 1 --2- 172.21.15.42:0/1553482030 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5634081f49f0 0x5634085ef390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:35.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.138+0000 7f4ec7fff640 1 -- 172.21.15.42:0/1553482030 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5634085eb360 con 0x56340820d1b0 2024-08-30T09:05:35.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.138+0000 7f4ec7fff640 1 --2- 172.21.15.42:0/1553482030 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56340820d1b0 0x5634085ea8e0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f4ec00ab070 tx=0x7f4ec0009320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a201727d9217c32 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:05:35.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.138+0000 7f4ea7fff640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-08-30T09:05:35.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.138+0000 7f4ea7fff640 1 -- 172.21.15.42:0/1553482030 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ec00b4350 con 0x56340820d1b0 2024-08-30T09:05:35.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.139+0000 7f4ea7fff640 1 -- 172.21.15.42:0/1553482030 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f4ec00b3ec0 con 0x56340820d1b0 2024-08-30T09:05:35.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.139+0000 7f4ed53c2740 1 -- 172.21.15.42:0/1553482030 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4eb80046c0 con 0x56340820d1b0 2024-08-30T09:05:35.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.139+0000 7f4ea7fff640 1 -- 172.21.15.42:0/1553482030 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f4ec00b40c0 con 0x56340820d1b0 2024-08-30T09:05:35.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.139+0000 7f4ed53c2740 10 client.16236 fetch_fsmap learned FSMap version 31 2024-08-30T09:05:35.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.139+0000 7f4ed53c2740 1 -- 172.21.15.42:0/1553482030 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x563408663a60 con 0x56340820d1b0 2024-08-30T09:05:35.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.139+0000 7f4ea7fff640 1 -- 172.21.15.42:0/1553482030 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4eb80046c0 con 0x56340820d1b0 2024-08-30T09:05:35.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.139+0000 7f4ea7fff640 1 -- 172.21.15.42:0/1553482030 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f4ec0014bc0 con 0x56340820d1b0 2024-08-30T09:05:35.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.139+0000 7f4ed53c2740 10 client.16236 fetch_fsmap finished waiting for FSMap version 31 2024-08-30T09:05:35.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.140+0000 7f4ed53c2740 10 client.16236 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-30T09:05:35.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.140+0000 7f4ed53c2740 20 client.16236 populate_metadata read hostname 'smithi042' 2024-08-30T09:05:35.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.140+0000 7f4ed53c2740 1 --2- 172.21.15.42:0/1553482030 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x563408664210 0x563408666630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:35.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.140+0000 7f4ed53c2740 4 client.16236 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-30T09:05:35.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.140+0000 7f4ed53c2740 1 -- 172.21.15.42:0/1553482030 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563408550020 con 0x563408664210 2024-08-30T09:05:35.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.140+0000 7f4eccbe9640 1 --2- 172.21.15.42:0/1553482030 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x563408664210 0x563408666630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:05:35.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.140+0000 7f4ea7fff640 10 client.16236 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:05:35.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.140+0000 7f4eccbe9640 1 --2- 172.21.15.42:0/1553482030 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x563408664210 0x563408666630 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=39010d51041f0a2f server_cookie=530b4c1fc939bd08 in_seq=0 out_seq=0 2024-08-30T09:05:35.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.141+0000 7f4ea7fff640 1 -- 172.21.15.42:0/1553482030 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+1794 (crc 0 0 0) 0x563408550020 con 0x563408664210 2024-08-30T09:05:35.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.141+0000 7f4ea7fff640 10 client.16236 handle_command_reply: tid=0 2024-08-30T09:05:35.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.141+0000 7f4ed53c2740 1 client.16236 shutdown 2024-08-30T09:05:35.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.141+0000 7f4ed53c2740 1 -- 172.21.15.42:0/1553482030 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56340820d1b0 msgr2=0x5634085ea8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:05:35.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.141+0000 7f4ed53c2740 1 --2- 172.21.15.42:0/1553482030 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56340820d1b0 0x5634085ea8e0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f4ec00ab070 tx=0x7f4ec0009320 comp rx=0 tx=0).stop 2024-08-30T09:05:35.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.141+0000 7f4ed53c2740 1 -- 172.21.15.42:0/1553482030 shutdown_connections 2024-08-30T09:05:35.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.141+0000 7f4ed53c2740 1 --2- 172.21.15.42:0/1553482030 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x563408664210 0x563408666630 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:35.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.141+0000 7f4ed53c2740 1 --2- 172.21.15.42:0/1553482030 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563408215390 0x5634085eae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:35.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.141+0000 7f4ed53c2740 1 --2- 172.21.15.42:0/1553482030 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56340820d1b0 0x5634085ea8e0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:35.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.141+0000 7f4ed53c2740 1 --2- 172.21.15.42:0/1553482030 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5634081f49f0 0x5634085ef390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:35.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.141+0000 7f4ed53c2740 1 -- 172.21.15.42:0/1553482030 >> 172.21.15.42:0/1553482030 conn(0x56340822a460 msgr2=0x5634085dc330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:05:35.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.141+0000 7f4ea7fff640 0 client.16236 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:05:35.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.141+0000 7f4ed53c2740 1 -- 172.21.15.42:0/1553482030 shutdown_connections 2024-08-30T09:05:35.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.141+0000 7f4ed53c2740 1 -- 172.21.15.42:0/1553482030 wait complete. 2024-08-30T09:05:35.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.142+0000 7f4ed53c2740 20 client.16236 trim_cache size 0 max 16384 2024-08-30T09:05:35.147 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-08-30T09:05:35.147 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "no active scrubs running", 2024-08-30T09:05:35.147 INFO:teuthology.orchestra.run.smithi042.stdout: "scrubs": {}, 2024-08-30T09:05:35.148 INFO:teuthology.orchestra.run.smithi042.stdout: "scrub_stats": { 2024-08-30T09:05:35.148 INFO:teuthology.orchestra.run.smithi042.stdout: "347d05ed-4bd3-4590-b099-314f719aab39": { 2024-08-30T09:05:35.148 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:05:00.439721+0000", 2024-08-30T09:05:35.148 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:05:35.148 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:05:35.148 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:05:35.148 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:05:35.148 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:05:35.148 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:05:35.148 INFO:teuthology.orchestra.run.smithi042.stdout: "88cd7478-2b20-47de-ba51-256479749232": { 2024-08-30T09:05:35.148 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:04:08.922625+0000", 2024-08-30T09:05:35.148 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:05:35.148 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:05:35.149 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:05:35.149 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:05:35.149 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:05:35.149 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:05:35.149 INFO:teuthology.orchestra.run.smithi042.stdout: "749f4867-0b70-49e5-af2d-8362a240ebc3": { 2024-08-30T09:05:35.149 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:03:16.386554+0000", 2024-08-30T09:05:35.149 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:05:35.149 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:05:35.149 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:05:35.149 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:05:35.149 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:05:35.149 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:05:35.149 INFO:teuthology.orchestra.run.smithi042.stdout: "85134e4b-ab4a-4885-b07e-a3ad01eaf13d": { 2024-08-30T09:05:35.149 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:01:27.738462+0000", 2024-08-30T09:05:35.149 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:05:35.150 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:05:35.150 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:05:35.150 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:05:35.150 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:05:35.150 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:05:35.150 INFO:teuthology.orchestra.run.smithi042.stdout: "c827ad7a-44ee-4e1d-9bb3-51fb7ab00c73": { 2024-08-30T09:05:35.150 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:02:21.686517+0000", 2024-08-30T09:05:35.150 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:05:35.150 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:05:35.150 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:05:35.150 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:05:35.150 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:05:35.150 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:05:35.150 INFO:teuthology.orchestra.run.smithi042.stdout: "5e2ed104-0625-478a-a78b-584e7f4c2817": { 2024-08-30T09:05:35.151 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:00:33.928400+0000", 2024-08-30T09:05:35.151 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:05:35.151 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:05:35.151 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:05:35.151 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:05:35.151 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:05:35.151 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:05:35.151 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:05:35.151 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-08-30T09:05:35.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.142+0000 7f4ecee4d640 1 -- 172.21.15.42:0/432751619 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f4eb00b4d40 msgr2=0x7f4eb00b7250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:05:35.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.142+0000 7f4ecee4d640 1 --2- 172.21.15.42:0/432751619 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f4eb00b4d40 0x7f4eb00b7250 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f4ebc011550 tx=0x7f4ebc005f90 comp rx=0 tx=0).stop 2024-08-30T09:05:35.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.142+0000 7f4ecee4d640 1 -- 172.21.15.42:0/432751619 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4ec8111ce0 msgr2=0x7f4ec8113430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:05:35.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.142+0000 7f4ecee4d640 1 --2- 172.21.15.42:0/432751619 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4ec8111ce0 0x7f4ec8113430 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f4ec806c480 tx=0x7f4ec000be00 comp rx=0 tx=0).stop 2024-08-30T09:05:35.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.142+0000 7f4ecee4d640 1 -- 172.21.15.42:0/432751619 shutdown_connections 2024-08-30T09:05:35.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.142+0000 7f4ecee4d640 1 --2- 172.21.15.42:0/432751619 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f4eb00b4d40 0x7f4eb00b7250 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:35.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.142+0000 7f4ecee4d640 1 --2- 172.21.15.42:0/432751619 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4ec8111ce0 0x7f4ec8113430 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:35.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.142+0000 7f4ecee4d640 1 --2- 172.21.15.42:0/432751619 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4ec811c9d0 0x7f4ec8113970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:35.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.142+0000 7f4ecee4d640 1 --2- 172.21.15.42:0/432751619 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4ec810a780 0x7f4ec811a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:35.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.142+0000 7f4ecee4d640 1 -- 172.21.15.42:0/432751619 >> 172.21.15.42:0/432751619 conn(0x7f4ec8070c60 msgr2=0x7f4ec811e0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:05:35.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.143+0000 7f4ecee4d640 1 -- 172.21.15.42:0/432751619 shutdown_connections 2024-08-30T09:05:35.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:35.143+0000 7f4ecee4d640 1 -- 172.21.15.42:0/432751619 wait complete. 2024-08-30T09:05:35.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:05:34 smithi190 ceph-mon[22001]: pgmap v517: 129 pgs: 129 active+clean; 1.2 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 5.9 MiB/s wr, 673 op/s 2024-08-30T09:05:35.953 INFO:teuthology.orchestra.run.smithi042.stderr:progress: 290.0 s, 636.2 tps, lat 156.562 ms stddev 184.182 2024-08-30T09:05:35.981 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-08-30T09:05:35.982 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph fs dump --format=json 2024-08-30T09:05:36.405 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T09:05:37.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:37.012+0000 7f1359c06640 1 -- 172.21.15.42:0/3250081727 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f134c094970 msgr2=0x7f134c099c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:05:37.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:37.012+0000 7f1359c06640 1 --2- 172.21.15.42:0/3250081727 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f134c094970 0x7f134c099c90 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f1348013080 tx=0x7f1348032710 comp rx=0 tx=0).stop 2024-08-30T09:05:37.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:37.013+0000 7f1359c06640 1 -- 172.21.15.42:0/3250081727 shutdown_connections 2024-08-30T09:05:37.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:37.013+0000 7f1359c06640 1 --2- 172.21.15.42:0/3250081727 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f134c094970 0x7f134c099c90 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:37.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:37.013+0000 7f1359c06640 1 --2- 172.21.15.42:0/3250081727 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f134c09a1d0 0x7f134c09a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:37.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:37.013+0000 7f1359c06640 1 --2- 172.21.15.42:0/3250081727 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f134c004800 0x7f134c094350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:37.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:37.013+0000 7f1359c06640 1 -- 172.21.15.42:0/3250081727 >> 172.21.15.42:0/3250081727 conn(0x7f134c00c0a0 msgr2=0x7f134c00c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:05:37.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:37.014+0000 7f1359c06640 1 -- 172.21.15.42:0/3250081727 shutdown_connections 2024-08-30T09:05:37.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:37.014+0000 7f1359c06640 1 -- 172.21.15.42:0/3250081727 wait complete. 2024-08-30T09:05:37.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:37.015+0000 7f1359c06640 1 Processor -- start 2024-08-30T09:05:37.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:37.015+0000 7f1359c06640 1 -- start start 2024-08-30T09:05:37.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:37.015+0000 7f1359c06640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f134c004800 0x7f134c12a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:37.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:37.015+0000 7f1359c06640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f134c094970 0x7f134c12ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:37.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:37.015+0000 7f1359c06640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f134c09a1d0 0x7f134c130130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:37.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:37.015+0000 7f1359c06640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f134c0a1e90 con 0x7f134c004800 2024-08-30T09:05:37.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:37.015+0000 7f1359c06640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f134c0a1d10 con 0x7f134c094970 2024-08-30T09:05:37.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:37.015+0000 7f1359c06640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f134c0a4130 con 0x7f134c09a1d0 2024-08-30T09:05:37.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:37.017+0000 7f1352ffd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f134c094970 0x7f134c12ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:05:37.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:37.017+0000 7f1352ffd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f134c094970 0x7f134c12ae90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:39422/0 (socket says 172.21.15.42:39422) 2024-08-30T09:05:37.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:37.017+0000 7f1352ffd640 1 -- 172.21.15.42:0/3879987629 learned_addr learned my addr 172.21.15.42:0/3879987629 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:05:37.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:37.017+0000 7f1352ffd640 1 -- 172.21.15.42:0/3879987629 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f134c09a1d0 msgr2=0x7f134c130130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:05:37.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:37.017+0000 7f1352ffd640 1 --2- 172.21.15.42:0/3879987629 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f134c09a1d0 0x7f134c130130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:37.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:37.017+0000 7f1352ffd640 1 -- 172.21.15.42:0/3879987629 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f134c004800 msgr2=0x7f134c12a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:05:37.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:37.017+0000 7f1352ffd640 1 --2- 172.21.15.42:0/3879987629 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f134c004800 0x7f134c12a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:37.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:37.018+0000 7f1352ffd640 1 -- 172.21.15.42:0/3879987629 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f134c130830 con 0x7f134c094970 2024-08-30T09:05:37.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:37.018+0000 7f1352ffd640 1 --2- 172.21.15.42:0/3879987629 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f134c094970 0x7f134c12ae90 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f1348039980 tx=0x7f13480399b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:05:37.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:37.020+0000 7f1350ff9640 1 -- 172.21.15.42:0/3879987629 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1348047030 con 0x7f134c094970 2024-08-30T09:05:37.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:37.020+0000 7f1350ff9640 1 -- 172.21.15.42:0/3879987629 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f134803f040 con 0x7f134c094970 2024-08-30T09:05:37.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:37.021+0000 7f1350ff9640 1 -- 172.21.15.42:0/3879987629 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f134803e070 con 0x7f134c094970 2024-08-30T09:05:37.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:37.022+0000 7f1359c06640 1 -- 172.21.15.42:0/3879987629 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f134c130ac0 con 0x7f134c094970 2024-08-30T09:05:37.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:37.022+0000 7f1359c06640 1 -- 172.21.15.42:0/3879987629 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f134c1482c0 con 0x7f134c094970 2024-08-30T09:05:37.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:37.026+0000 7f1359c06640 1 -- 172.21.15.42:0/3879987629 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f134c0955d0 con 0x7f134c094970 2024-08-30T09:05:37.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:37.026+0000 7f1350ff9640 1 -- 172.21.15.42:0/3879987629 <== mon.2 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7f134804a070 con 0x7f134c094970 2024-08-30T09:05:37.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:37.028+0000 7f1350ff9640 1 --2- 172.21.15.42:0/3879987629 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f13440b4c10 0x7f13440b7120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:37.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:37.028+0000 7f1350ff9640 1 -- 172.21.15.42:0/3879987629 <== mon.2 v2:172.21.15.167:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f134810c830 con 0x7f134c094970 2024-08-30T09:05:37.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:37.032+0000 7f13537fe640 1 --2- 172.21.15.42:0/3879987629 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f13440b4c10 0x7f13440b7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:05:37.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:37.033+0000 7f13537fe640 1 --2- 172.21.15.42:0/3879987629 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f13440b4c10 0x7f13440b7120 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f1354051ab0 tx=0x7f1354069100 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:05:37.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:37.036+0000 7f1350ff9640 1 -- 172.21.15.42:0/3879987629 <== mon.2 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f1348039e20 con 0x7f134c094970 2024-08-30T09:05:37.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:05:36 smithi167 ceph-mon[22063]: pgmap v518: 129 pgs: 129 active+clean; 1.2 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 5.1 MiB/s wr, 582 op/s 2024-08-30T09:05:37.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:05:36 smithi042 ceph-mon[18652]: pgmap v518: 129 pgs: 129 active+clean; 1.2 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 5.1 MiB/s wr, 582 op/s 2024-08-30T09:05:37.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:05:36 smithi190 ceph-mon[22001]: pgmap v518: 129 pgs: 129 active+clean; 1.2 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 5.1 MiB/s wr, 582 op/s 2024-08-30T09:05:37.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:37.411+0000 7f1359c06640 1 -- 172.21.15.42:0/3879987629 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f134c12ba20 con 0x7f134c094970 2024-08-30T09:05:37.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:37.412+0000 7f1350ff9640 1 -- 172.21.15.42:0/3879987629 <== mon.2 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+11915 (secure 0 0 0) 0x7f13480d3570 con 0x7f134c094970 2024-08-30T09:05:37.415 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-08-30T09:05:37.415 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 31 2024-08-30T09:05:37.416 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":31,"btime":"2024-08-30T09:05:27:779984+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14703,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/3188400000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":3188400000},{"type":"v1","addr":"172.21.15.42:6835","nonce":3188400000}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14715,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/3356537674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":3356537674},{"type":"v1","addr":"172.21.15.42:6837","nonce":3356537674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14766,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6839/2107785513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6838","nonce":2107785513},{"type":"v1","addr":"172.21.15.42:6839","nonce":2107785513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14778,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6841/2298438674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6840","nonce":2298438674},{"type":"v1","addr":"172.21.15.42:6841","nonce":2298438674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24553,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/543555570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":543555570},{"type":"v1","addr":"172.21.15.190:6833","nonce":543555570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24554,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/3907818941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":3907818941},{"type":"v1","addr":"172.21.15.167:6833","nonce":3907818941}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":31,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-30T08:58:03.055084+0000","modified":"2024-08-30T09:05:27.642967+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24629,"mds_1":24610,"mds_2":24599},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24572":{"gid":24572,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6835/234596845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":234596845},{"type":"v1","addr":"172.21.15.167:6835","nonce":234596845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24592":{"gid":24592,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6835/247905709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":247905709},{"type":"v1","addr":"172.21.15.190:6835","nonce":247905709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24598":{"gid":24598,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6837/863256349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6836","nonce":863256349},{"type":"v1","addr":"172.21.15.190:6837","nonce":863256349}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24599":{"gid":24599,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.167:6837/3011837856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":3011837856},{"type":"v1","addr":"172.21.15.167:6837","nonce":3011837856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24610":{"gid":24610,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.190:6839/719981110","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6838","nonce":719981110},{"type":"v1","addr":"172.21.15.190:6839","nonce":719981110}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24629":{"gid":24629,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.167:6839/3604645207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":3604645207},{"type":"v1","addr":"172.21.15.167:6839","nonce":3604645207}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24629,"qdb_cluster":[24599,24610,24629]},"id":1}]} 2024-08-30T09:05:37.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:37.416+0000 7f1359c06640 1 -- 172.21.15.42:0/3879987629 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f13440b4c10 msgr2=0x7f13440b7120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:05:37.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:37.416+0000 7f1359c06640 1 --2- 172.21.15.42:0/3879987629 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f13440b4c10 0x7f13440b7120 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f1354051ab0 tx=0x7f1354069100 comp rx=0 tx=0).stop 2024-08-30T09:05:37.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:37.416+0000 7f1359c06640 1 -- 172.21.15.42:0/3879987629 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f134c094970 msgr2=0x7f134c12ae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:05:37.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:37.416+0000 7f1359c06640 1 --2- 172.21.15.42:0/3879987629 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f134c094970 0x7f134c12ae90 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f1348039980 tx=0x7f13480399b0 comp rx=0 tx=0).stop 2024-08-30T09:05:37.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:37.416+0000 7f1359c06640 1 -- 172.21.15.42:0/3879987629 shutdown_connections 2024-08-30T09:05:37.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:37.416+0000 7f1359c06640 1 --2- 172.21.15.42:0/3879987629 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f13440b4c10 0x7f13440b7120 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:37.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:37.416+0000 7f1359c06640 1 --2- 172.21.15.42:0/3879987629 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f134c09a1d0 0x7f134c130130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:37.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:37.416+0000 7f1359c06640 1 --2- 172.21.15.42:0/3879987629 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f134c094970 0x7f134c12ae90 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:37.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:37.416+0000 7f1359c06640 1 --2- 172.21.15.42:0/3879987629 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f134c004800 0x7f134c12a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:37.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:37.416+0000 7f1359c06640 1 -- 172.21.15.42:0/3879987629 >> 172.21.15.42:0/3879987629 conn(0x7f134c00c0a0 msgr2=0x7f134c098460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:05:37.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:37.417+0000 7f1359c06640 1 -- 172.21.15.42:0/3879987629 shutdown_connections 2024-08-30T09:05:37.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:37.417+0000 7f1359c06640 1 -- 172.21.15.42:0/3879987629 wait complete. 2024-08-30T09:05:38.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:05:37 smithi167 ceph-mon[22063]: from='client.? 172.21.15.42:0/3879987629' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-30T09:05:38.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:05:37 smithi042 ceph-mon[18652]: from='client.? 172.21.15.42:0/3879987629' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-30T09:05:38.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:05:37 smithi190 ceph-mon[22001]: from='client.? 172.21.15.42:0/3879987629' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-30T09:05:38.236 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-08-30T09:05:38.475 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.473+0000 7f17db59e640 1 -- 172.21.15.42:0/2055849930 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f17dc10a160 msgr2=0x7f17dc10c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:05:38.475 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.473+0000 7f17db59e640 1 --2- 172.21.15.42:0/2055849930 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f17dc10a160 0x7f17dc10c580 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f17d4009ca0 tx=0x7f17d402a8e0 comp rx=0 tx=0).stop 2024-08-30T09:05:38.475 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.473+0000 7f17db59e640 1 -- 172.21.15.42:0/2055849930 shutdown_connections 2024-08-30T09:05:38.475 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.473+0000 7f17db59e640 1 --2- 172.21.15.42:0/2055849930 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f17dc10f5d0 0x7f17dc10fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:38.475 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.473+0000 7f17db59e640 1 --2- 172.21.15.42:0/2055849930 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f17dc10cac0 0x7f17dc10eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:38.476 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.473+0000 7f17db59e640 1 --2- 172.21.15.42:0/2055849930 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f17dc10a160 0x7f17dc10c580 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:38.476 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.473+0000 7f17db59e640 1 -- 172.21.15.42:0/2055849930 >> 172.21.15.42:0/2055849930 conn(0x7f17dc0703e0 msgr2=0x7f17dc072890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:05:38.476 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.473+0000 7f17db59e640 1 -- 172.21.15.42:0/2055849930 shutdown_connections 2024-08-30T09:05:38.476 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.473+0000 7f17db59e640 1 -- 172.21.15.42:0/2055849930 wait complete. 2024-08-30T09:05:38.476 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.474+0000 7f17db59e640 1 Processor -- start 2024-08-30T09:05:38.476 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.474+0000 7f17db59e640 1 -- start start 2024-08-30T09:05:38.477 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.475+0000 7f17db59e640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f17dc10a160 0x7f17dc1a8960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:38.477 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.475+0000 7f17db59e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f17dc10cac0 0x7f17dc1a8fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:38.477 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.475+0000 7f17db59e640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f17dc10f5d0 0x7f17dc1a9b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:38.477 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.475+0000 7f17db59e640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f17dc11dd70 con 0x7f17dc10f5d0 2024-08-30T09:05:38.478 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.475+0000 7f17da59c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f17dc10a160 0x7f17dc1a8960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:05:38.478 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.475+0000 7f17db59e640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f17dc11dbf0 con 0x7f17dc10a160 2024-08-30T09:05:38.478 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.475+0000 7f17db59e640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f17dc11def0 con 0x7f17dc10cac0 2024-08-30T09:05:38.478 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.475+0000 7f17da59c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f17dc10a160 0x7f17dc1a8960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:48220/0 (socket says 172.21.15.42:48220) 2024-08-30T09:05:38.478 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.475+0000 7f17da59c640 1 -- 172.21.15.42:0/3120399519 learned_addr learned my addr 172.21.15.42:0/3120399519 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:05:38.478 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.475+0000 7f17dad9d640 1 --2- 172.21.15.42:0/3120399519 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f17dc10f5d0 0x7f17dc1a9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:05:38.478 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.475+0000 7f17d9d9b640 1 --2- 172.21.15.42:0/3120399519 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f17dc10cac0 0x7f17dc1a8fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:05:38.478 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.475+0000 7f17da59c640 1 -- 172.21.15.42:0/3120399519 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f17dc10cac0 msgr2=0x7f17dc1a8fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:05:38.478 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.475+0000 7f17da59c640 1 --2- 172.21.15.42:0/3120399519 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f17dc10cac0 0x7f17dc1a8fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:38.478 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.475+0000 7f17da59c640 1 -- 172.21.15.42:0/3120399519 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f17dc10f5d0 msgr2=0x7f17dc1a9b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:05:38.478 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.475+0000 7f17da59c640 1 --2- 172.21.15.42:0/3120399519 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f17dc10f5d0 0x7f17dc1a9b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:38.478 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.475+0000 7f17da59c640 1 -- 172.21.15.42:0/3120399519 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17dc1aa3a0 con 0x7f17dc10a160 2024-08-30T09:05:38.478 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.476+0000 7f17da59c640 1 --2- 172.21.15.42:0/3120399519 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f17dc10a160 0x7f17dc1a8960 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f17d402ae00 tx=0x7f17d4030870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:05:38.479 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.476+0000 7f17cb7fe640 1 -- 172.21.15.42:0/3120399519 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17d4046480 con 0x7f17dc10a160 2024-08-30T09:05:38.479 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.476+0000 7f17cb7fe640 1 -- 172.21.15.42:0/3120399519 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f17d4046ad0 con 0x7f17dc10a160 2024-08-30T09:05:38.479 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.476+0000 7f17cb7fe640 1 -- 172.21.15.42:0/3120399519 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17d403a740 con 0x7f17dc10a160 2024-08-30T09:05:38.479 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.476+0000 7f17db59e640 1 -- 172.21.15.42:0/3120399519 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17dc1aa630 con 0x7f17dc10a160 2024-08-30T09:05:38.480 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.476+0000 7f17db59e640 1 -- 172.21.15.42:0/3120399519 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17dc1b0d80 con 0x7f17dc10a160 2024-08-30T09:05:38.482 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.479+0000 7f17e77f8740 1 Processor -- start 2024-08-30T09:05:38.482 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.480+0000 7f17e77f8740 1 -- start start 2024-08-30T09:05:38.482 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.480+0000 7f17e77f8740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561a6dece1b0 0x561a6e274380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:38.482 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.480+0000 7f17e77f8740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561a6dea52f0 0x561a6e2749f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:38.482 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.480+0000 7f17e77f8740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561a6dead4d0 0x561a6e275540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:38.482 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.480+0000 7f17e77f8740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x561a6e1e7000 con 0x561a6dece1b0 2024-08-30T09:05:38.482 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.480+0000 7f17e77f8740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x561a6e1d6590 con 0x561a6dea52f0 2024-08-30T09:05:38.482 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.480+0000 7f17e77f8740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x561a6e1da540 con 0x561a6dead4d0 2024-08-30T09:05:38.483 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.480+0000 7f17d9d9b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561a6dead4d0 0x561a6e275540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:05:38.483 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.480+0000 7f17d9d9b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561a6dead4d0 0x561a6e275540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:58694/0 (socket says 172.21.15.42:58694) 2024-08-30T09:05:38.483 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.480+0000 7f17d9d9b640 1 -- 172.21.15.42:0/3346512274 learned_addr learned my addr 172.21.15.42:0/3346512274 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:05:38.483 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.481+0000 7f17dad9d640 1 --2- 172.21.15.42:0/3346512274 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561a6dece1b0 0x561a6e274380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:05:38.484 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.481+0000 7f17cb7fe640 1 -- 172.21.15.42:0/3120399519 <== mon.2 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7f17d403a940 con 0x7f17dc10a160 2024-08-30T09:05:38.484 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.481+0000 7f17d9d9b640 1 -- 172.21.15.42:0/3346512274 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561a6dea52f0 msgr2=0x561a6e2749f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-30T09:05:38.484 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.481+0000 7f17d9d9b640 1 --2- 172.21.15.42:0/3346512274 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561a6dea52f0 0x561a6e2749f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:38.484 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.481+0000 7f17d9d9b640 1 -- 172.21.15.42:0/3346512274 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561a6dece1b0 msgr2=0x561a6e274380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:05:38.484 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.481+0000 7f17d9d9b640 1 --2- 172.21.15.42:0/3346512274 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561a6dece1b0 0x561a6e274380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:38.484 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.481+0000 7f17d9d9b640 1 -- 172.21.15.42:0/3346512274 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a6df39790 con 0x561a6dead4d0 2024-08-30T09:05:38.484 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.481+0000 7f17dad9d640 1 --2- 172.21.15.42:0/3346512274 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561a6dece1b0 0x561a6e274380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-30T09:05:38.484 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.481+0000 7f17d9d9b640 1 --2- 172.21.15.42:0/3346512274 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561a6dead4d0 0x561a6e275540 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f17cc009c80 tx=0x7f17cc0117b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=24a8e22a848dc8b0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:05:38.484 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.481+0000 7f17c8ff9640 1 -- 172.21.15.42:0/3346512274 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17cc012450 con 0x561a6dead4d0 2024-08-30T09:05:38.484 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.481+0000 7f17c8ff9640 1 -- 172.21.15.42:0/3346512274 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f17cc012aa0 con 0x561a6dead4d0 2024-08-30T09:05:38.484 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.481+0000 7f17c8ff9640 1 -- 172.21.15.42:0/3346512274 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17cc00fae0 con 0x561a6dead4d0 2024-08-30T09:05:38.486 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.482+0000 7f17cb7fe640 1 --2- 172.21.15.42:0/3120399519 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f17c40b4b30 0x7f17c40b7040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:38.486 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.482+0000 7f17dad9d640 1 --2- 172.21.15.42:0/3120399519 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f17c40b4b30 0x7f17c40b7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:05:38.486 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.482+0000 7f17cb7fe640 1 -- 172.21.15.42:0/3120399519 <== mon.2 v2:172.21.15.167:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f17d4106950 con 0x7f17dc10a160 2024-08-30T09:05:38.486 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.483+0000 7f17dad9d640 1 --2- 172.21.15.42:0/3120399519 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f17c40b4b30 0x7f17c40b7040 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f17d0009ac0 tx=0x7f17d0008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:05:38.486 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.484+0000 7f17e77f8740 1 -- 172.21.15.42:0/3346512274 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561a6dead4d0 msgr2=0x561a6e275540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:05:38.486 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.484+0000 7f17e77f8740 1 --2- 172.21.15.42:0/3346512274 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561a6dead4d0 0x561a6e275540 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f17cc009c80 tx=0x7f17cc0117b0 comp rx=0 tx=0).stop 2024-08-30T09:05:38.486 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.484+0000 7f17e77f8740 1 -- 172.21.15.42:0/3346512274 shutdown_connections 2024-08-30T09:05:38.486 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.484+0000 7f17e77f8740 1 --2- 172.21.15.42:0/3346512274 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561a6dead4d0 0x561a6e275540 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:38.486 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.484+0000 7f17e77f8740 1 --2- 172.21.15.42:0/3346512274 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561a6dea52f0 0x561a6e2749f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:38.487 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.484+0000 7f17e77f8740 1 --2- 172.21.15.42:0/3346512274 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561a6dece1b0 0x561a6e274380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:38.487 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.484+0000 7f17e77f8740 1 -- 172.21.15.42:0/3346512274 >> 172.21.15.42:0/3346512274 conn(0x561a6dec2300 msgr2=0x561a6e26fe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:05:38.487 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.484+0000 7f17e77f8740 1 -- 172.21.15.42:0/3346512274 shutdown_connections 2024-08-30T09:05:38.487 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.484+0000 7f17e77f8740 1 -- 172.21.15.42:0/3346512274 wait complete. 2024-08-30T09:05:38.487 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.485+0000 7f17e77f8740 1 -- start start 2024-08-30T09:05:38.487 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.485+0000 7f17e77f8740 1 Processor -- start 2024-08-30T09:05:38.488 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.486+0000 7f17e77f8740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561a6dea52f0 0x561a6e27de00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:38.489 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.486+0000 7f17e77f8740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561a6dead4d0 0x561a6e27e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:38.489 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.486+0000 7f17e77f8740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561a6dece1b0 0x561a6e27f030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:38.489 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.486+0000 7f17e77f8740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x561a6e10fab0 con 0x561a6dea52f0 2024-08-30T09:05:38.489 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.486+0000 7f17e77f8740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x561a6e15d680 con 0x561a6dece1b0 2024-08-30T09:05:38.489 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.486+0000 7f17dad9d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561a6dece1b0 0x561a6e27f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:05:38.489 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.487+0000 7f17dad9d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561a6dece1b0 0x561a6e27f030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:48250/0 (socket says 172.21.15.42:48250) 2024-08-30T09:05:38.489 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.487+0000 7f17dad9d640 1 -- 172.21.15.42:0/1629053939 learned_addr learned my addr 172.21.15.42:0/1629053939 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:05:38.489 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.487+0000 7f17e77f8740 1 -- 172.21.15.42:0/1629053939 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x561a6e276060 con 0x561a6dead4d0 2024-08-30T09:05:38.489 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.487+0000 7f17da59c640 1 --2- 172.21.15.42:0/1629053939 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561a6dea52f0 0x561a6e27de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:05:38.489 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.487+0000 7f17e77f8740 10 client.0 _ll_register_callbacks cb 0x561a6dee5a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:05:38.491 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.487+0000 7f17dad9d640 1 -- 172.21.15.42:0/1629053939 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561a6dead4d0 msgr2=0x561a6e27e4c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-30T09:05:38.491 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.487+0000 7f17dad9d640 1 --2- 172.21.15.42:0/1629053939 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561a6dead4d0 0x561a6e27e4c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:38.491 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.487+0000 7f17dad9d640 1 -- 172.21.15.42:0/1629053939 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561a6dea52f0 msgr2=0x561a6e27de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:05:38.491 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.487+0000 7f17dad9d640 1 --2- 172.21.15.42:0/1629053939 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561a6dea52f0 0x561a6e27de00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:38.491 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.487+0000 7f17dad9d640 1 -- 172.21.15.42:0/1629053939 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561a6dfb2620 con 0x561a6dece1b0 2024-08-30T09:05:38.491 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.488+0000 7f17dad9d640 1 --2- 172.21.15.42:0/1629053939 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561a6dece1b0 0x561a6e27f030 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f17d00098a0 tx=0x7f17d00046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eeb3c40a7d4bd185 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:05:38.491 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.488+0000 7f17c1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-30T09:05:38.491 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.488+0000 7f17c1ffb640 1 -- 172.21.15.42:0/1629053939 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17d0025020 con 0x561a6dece1b0 2024-08-30T09:05:38.491 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.488+0000 7f17c1ffb640 1 -- 172.21.15.42:0/1629053939 <== mon.2 v2:172.21.15.167:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f17d00144a0 con 0x561a6dece1b0 2024-08-30T09:05:38.492 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.488+0000 7f17e77f8740 1 -- 172.21.15.42:0/1629053939 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f17cc012aa0 con 0x561a6dece1b0 2024-08-30T09:05:38.492 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.489+0000 7f17c1ffb640 1 -- 172.21.15.42:0/1629053939 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f17d00146a0 con 0x561a6dece1b0 2024-08-30T09:05:38.492 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.489+0000 7f17e77f8740 10 client.25991 fetch_fsmap learned FSMap version 31 2024-08-30T09:05:38.492 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.489+0000 7f17e77f8740 1 -- 172.21.15.42:0/1629053939 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x561a6e288050 con 0x561a6dece1b0 2024-08-30T09:05:38.492 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.489+0000 7f17c1ffb640 1 -- 172.21.15.42:0/1629053939 <== mon.2 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f17cc012aa0 con 0x561a6dece1b0 2024-08-30T09:05:38.492 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.489+0000 7f17c1ffb640 1 -- 172.21.15.42:0/1629053939 <== mon.2 v2:172.21.15.167:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f17d00219e0 con 0x561a6dece1b0 2024-08-30T09:05:38.492 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.489+0000 7f17e77f8740 10 client.25991 fetch_fsmap finished waiting for FSMap version 31 2024-08-30T09:05:38.492 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.489+0000 7f17e77f8740 10 client.25991 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-30T09:05:38.492 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.489+0000 7f17e77f8740 20 client.25991 populate_metadata read hostname 'smithi042' 2024-08-30T09:05:38.492 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.489+0000 7f17e77f8740 1 --2- 172.21.15.42:0/1629053939 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x561a6de8cd80 0x561a6e28b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:38.492 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.489+0000 7f17e77f8740 4 client.25991 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-30T09:05:38.492 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.489+0000 7f17e77f8740 1 -- 172.21.15.42:0/1629053939 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561a6e1e8f50 con 0x561a6de8cd80 2024-08-30T09:05:38.494 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.489+0000 7f17da59c640 1 --2- 172.21.15.42:0/1629053939 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x561a6de8cd80 0x561a6e28b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:05:38.494 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.490+0000 7f17c1ffb640 10 client.25991 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:05:38.494 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.490+0000 7f17da59c640 1 --2- 172.21.15.42:0/1629053939 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x561a6de8cd80 0x561a6e28b8c0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e7280493cbec8c61 server_cookie=26c804358ff9430b in_seq=0 out_seq=0 2024-08-30T09:05:38.494 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.491+0000 7f17c1ffb640 1 -- 172.21.15.42:0/1629053939 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x561a6e1e8f50 con 0x561a6de8cd80 2024-08-30T09:05:38.494 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.491+0000 7f17c1ffb640 10 client.25991 handle_command_reply: tid=0 2024-08-30T09:05:38.494 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.491+0000 7f17e77f8740 1 client.25991 shutdown 2024-08-30T09:05:38.494 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.491+0000 7f17e77f8740 1 -- 172.21.15.42:0/1629053939 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561a6dece1b0 msgr2=0x561a6e27f030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:05:38.494 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.491+0000 7f17e77f8740 1 --2- 172.21.15.42:0/1629053939 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561a6dece1b0 0x561a6e27f030 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f17d00098a0 tx=0x7f17d00046c0 comp rx=0 tx=0).stop 2024-08-30T09:05:38.494 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.491+0000 7f17e77f8740 1 -- 172.21.15.42:0/1629053939 shutdown_connections 2024-08-30T09:05:38.494 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.491+0000 7f17e77f8740 1 --2- 172.21.15.42:0/1629053939 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x561a6de8cd80 0x561a6e28b8c0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:38.494 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.491+0000 7f17e77f8740 1 --2- 172.21.15.42:0/1629053939 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561a6dead4d0 0x561a6e27e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:38.494 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.491+0000 7f17e77f8740 1 --2- 172.21.15.42:0/1629053939 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561a6dece1b0 0x561a6e27f030 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:38.494 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.491+0000 7f17e77f8740 1 --2- 172.21.15.42:0/1629053939 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561a6dea52f0 0x561a6e27de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:38.494 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.491+0000 7f17e77f8740 1 -- 172.21.15.42:0/1629053939 >> 172.21.15.42:0/1629053939 conn(0x561a6dec2300 msgr2=0x561a6e26e370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:05:38.495 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.491+0000 7f17c1ffb640 0 client.25991 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:05:38.495 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.491+0000 7f17e77f8740 1 -- 172.21.15.42:0/1629053939 shutdown_connections 2024-08-30T09:05:38.495 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.491+0000 7f17e77f8740 1 -- 172.21.15.42:0/1629053939 wait complete. 2024-08-30T09:05:38.495 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.491+0000 7f17e77f8740 20 client.25991 trim_cache size 0 max 16384 2024-08-30T09:05:38.509 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.507+0000 7f17e77f8740 1 Processor -- start 2024-08-30T09:05:38.509 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.507+0000 7f17e77f8740 1 -- start start 2024-08-30T09:05:38.512 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.508+0000 7f17e77f8740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561a6dea52f0 0x561a6e275210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:38.512 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.508+0000 7f17e77f8740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561a6dead4d0 0x561a6e27d500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:38.512 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.508+0000 7f17e77f8740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561a6dece1b0 0x561a6e26f750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:38.512 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.508+0000 7f17e77f8740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f17dc1bd920 con 0x561a6dece1b0 2024-08-30T09:05:38.512 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.508+0000 7f17da59c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561a6dea52f0 0x561a6e275210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:05:38.512 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.508+0000 7f17da59c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561a6dea52f0 0x561a6e275210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:58706/0 (socket says 172.21.15.42:58706) 2024-08-30T09:05:38.512 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.508+0000 7f17da59c640 1 -- 172.21.15.42:0/1922276577 learned_addr learned my addr 172.21.15.42:0/1922276577 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:05:38.512 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.508+0000 7f17e77f8740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f17dc1bb1f0 con 0x561a6dead4d0 2024-08-30T09:05:38.512 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.508+0000 7f17e77f8740 1 -- 172.21.15.42:0/1922276577 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f17dc1b9040 con 0x561a6dea52f0 2024-08-30T09:05:38.512 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.508+0000 7f17dad9d640 1 --2- 172.21.15.42:0/1922276577 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561a6dece1b0 0x561a6e26f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:05:38.512 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.509+0000 7f17da59c640 1 -- 172.21.15.42:0/1922276577 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561a6dead4d0 msgr2=0x561a6e27d500 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-30T09:05:38.513 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.509+0000 7f17da59c640 1 --2- 172.21.15.42:0/1922276577 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561a6dead4d0 0x561a6e27d500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:38.513 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.509+0000 7f17da59c640 1 -- 172.21.15.42:0/1922276577 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561a6dece1b0 msgr2=0x561a6e26f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:05:38.513 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.509+0000 7f17da59c640 1 --2- 172.21.15.42:0/1922276577 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561a6dece1b0 0x561a6e26f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:38.513 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.509+0000 7f17da59c640 1 -- 172.21.15.42:0/1922276577 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a6e274e10 con 0x561a6dea52f0 2024-08-30T09:05:38.513 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.509+0000 7f17da59c640 1 --2- 172.21.15.42:0/1922276577 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561a6dea52f0 0x561a6e275210 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f17d4006010 tx=0x7f17d402a8e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=58a60e569957887 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:05:38.513 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.510+0000 7f17c3fff640 1 -- 172.21.15.42:0/1922276577 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17d40f2030 con 0x561a6dea52f0 2024-08-30T09:05:38.513 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.510+0000 7f17c3fff640 1 -- 172.21.15.42:0/1922276577 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f17d4036040 con 0x561a6dea52f0 2024-08-30T09:05:38.513 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.510+0000 7f17c3fff640 1 -- 172.21.15.42:0/1922276577 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17d4046c80 con 0x561a6dea52f0 2024-08-30T09:05:38.517 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.511+0000 7f17e77f8740 1 -- 172.21.15.42:0/1922276577 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561a6dea52f0 msgr2=0x561a6e275210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:05:38.517 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.511+0000 7f17e77f8740 1 --2- 172.21.15.42:0/1922276577 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561a6dea52f0 0x561a6e275210 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f17d4006010 tx=0x7f17d402a8e0 comp rx=0 tx=0).stop 2024-08-30T09:05:38.517 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.511+0000 7f17e77f8740 1 -- 172.21.15.42:0/1922276577 shutdown_connections 2024-08-30T09:05:38.517 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.511+0000 7f17e77f8740 1 --2- 172.21.15.42:0/1922276577 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561a6dece1b0 0x561a6e26f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:38.517 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.511+0000 7f17e77f8740 1 --2- 172.21.15.42:0/1922276577 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561a6dead4d0 0x561a6e27d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:38.517 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.511+0000 7f17e77f8740 1 --2- 172.21.15.42:0/1922276577 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561a6dea52f0 0x561a6e275210 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:38.518 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.511+0000 7f17e77f8740 1 -- 172.21.15.42:0/1922276577 >> 172.21.15.42:0/1922276577 conn(0x561a6dec2300 msgr2=0x561a6e26a670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:05:38.518 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.511+0000 7f17e77f8740 1 -- 172.21.15.42:0/1922276577 shutdown_connections 2024-08-30T09:05:38.518 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.511+0000 7f17e77f8740 1 -- 172.21.15.42:0/1922276577 wait complete. 2024-08-30T09:05:38.518 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.512+0000 7f17e77f8740 1 -- start start 2024-08-30T09:05:38.518 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.512+0000 7f17e77f8740 1 Processor -- start 2024-08-30T09:05:38.518 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.512+0000 7f17e77f8740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561a6dea52f0 0x561a6e2857c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:38.518 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.512+0000 7f17e77f8740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561a6dead4d0 0x561a6e285e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:38.518 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.512+0000 7f17e77f8740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561a6dece1b0 0x561a6e284570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:38.518 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.512+0000 7f17e77f8740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f17dc1b96b0 con 0x561a6dea52f0 2024-08-30T09:05:38.518 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.512+0000 7f17e77f8740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f17dc1bab20 con 0x561a6dece1b0 2024-08-30T09:05:38.518 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.512+0000 7f17e77f8740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f17dc1b9d80 con 0x561a6dead4d0 2024-08-30T09:05:38.518 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.512+0000 7f17da59c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561a6dea52f0 0x561a6e2857c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:05:38.519 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.512+0000 7f17d9d9b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561a6dead4d0 0x561a6e285e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:05:38.519 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.512+0000 7f17da59c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561a6dea52f0 0x561a6e2857c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35578/0 (socket says 172.21.15.42:35578) 2024-08-30T09:05:38.519 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.512+0000 7f17d9d9b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561a6dead4d0 0x561a6e285e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:58718/0 (socket says 172.21.15.42:58718) 2024-08-30T09:05:38.519 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.512+0000 7f17da59c640 1 -- 172.21.15.42:0/1671641364 learned_addr learned my addr 172.21.15.42:0/1671641364 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:05:38.519 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.513+0000 7f17dad9d640 1 --2- 172.21.15.42:0/1671641364 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561a6dece1b0 0x561a6e284570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:05:38.519 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.513+0000 7f17e77f8740 10 client.0 _ll_register_callbacks cb 0x7f17d0009710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:05:38.519 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.513+0000 7f17d9d9b640 1 -- 172.21.15.42:0/1671641364 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561a6dece1b0 msgr2=0x561a6e284570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:05:38.519 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.513+0000 7f17d9d9b640 1 --2- 172.21.15.42:0/1671641364 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561a6dece1b0 0x561a6e284570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:38.519 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.513+0000 7f17d9d9b640 1 -- 172.21.15.42:0/1671641364 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561a6dea52f0 msgr2=0x561a6e2857c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:05:38.519 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.513+0000 7f17d9d9b640 1 --2- 172.21.15.42:0/1671641364 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561a6dea52f0 0x561a6e2857c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:38.519 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.513+0000 7f17d9d9b640 1 -- 172.21.15.42:0/1671641364 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561a6e284d00 con 0x561a6dead4d0 2024-08-30T09:05:38.519 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.513+0000 7f17da59c640 1 --2- 172.21.15.42:0/1671641364 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561a6dea52f0 0x561a6e2857c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-30T09:05:38.519 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.513+0000 7f17d9d9b640 1 --2- 172.21.15.42:0/1671641364 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561a6dead4d0 0x561a6e285e80 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f17cc000c00 tx=0x7f17cc009fa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b95a5086cb55b7e server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:05:38.520 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.513+0000 7f17c1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-08-30T09:05:38.520 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.513+0000 7f17c1ffb640 1 -- 172.21.15.42:0/1671641364 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17cc028020 con 0x561a6dead4d0 2024-08-30T09:05:38.520 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.513+0000 7f17c1ffb640 1 -- 172.21.15.42:0/1671641364 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f17cc003800 con 0x561a6dead4d0 2024-08-30T09:05:38.520 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.514+0000 7f17e77f8740 1 -- 172.21.15.42:0/1671641364 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f17d4036040 con 0x561a6dead4d0 2024-08-30T09:05:38.520 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.514+0000 7f17c1ffb640 1 -- 172.21.15.42:0/1671641364 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f17cc033050 con 0x561a6dead4d0 2024-08-30T09:05:38.520 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.514+0000 7f17e77f8740 10 client.25996 fetch_fsmap learned FSMap version 31 2024-08-30T09:05:38.520 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.514+0000 7f17e77f8740 1 -- 172.21.15.42:0/1671641364 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x561a6e283b60 con 0x561a6dead4d0 2024-08-30T09:05:38.520 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.514+0000 7f17c1ffb640 1 -- 172.21.15.42:0/1671641364 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f17d4036040 con 0x561a6dead4d0 2024-08-30T09:05:38.520 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.514+0000 7f17c1ffb640 1 -- 172.21.15.42:0/1671641364 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f17cc024800 con 0x561a6dead4d0 2024-08-30T09:05:38.520 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.514+0000 7f17e77f8740 10 client.25996 fetch_fsmap finished waiting for FSMap version 31 2024-08-30T09:05:38.520 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.514+0000 7f17e77f8740 10 client.25996 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-30T09:05:38.520 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.514+0000 7f17e77f8740 20 client.25996 populate_metadata read hostname 'smithi042' 2024-08-30T09:05:38.520 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.514+0000 7f17e77f8740 1 --2- 172.21.15.42:0/1671641364 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x561a6de621c0 0x561a6e2f84d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:38.521 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.514+0000 7f17e77f8740 4 client.25996 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-30T09:05:38.521 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.514+0000 7f17e77f8740 1 -- 172.21.15.42:0/1671641364 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561a6e1e49f0 con 0x561a6de621c0 2024-08-30T09:05:38.521 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.515+0000 7f17da59c640 1 --2- 172.21.15.42:0/1671641364 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x561a6de621c0 0x561a6e2f84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:05:38.521 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.515+0000 7f17c1ffb640 10 client.25996 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:05:38.521 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.515+0000 7f17da59c640 1 --2- 172.21.15.42:0/1671641364 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x561a6de621c0 0x561a6e2f84d0 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b36b372bba3c3c0 server_cookie=12a359af7a1d1000 in_seq=0 out_seq=0 2024-08-30T09:05:38.527 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.524+0000 7f17c1ffb640 1 -- 172.21.15.42:0/1671641364 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561a6e1e49f0 con 0x561a6de621c0 2024-08-30T09:05:38.527 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.524+0000 7f17c1ffb640 10 client.25996 handle_command_reply: tid=0 2024-08-30T09:05:38.527 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.524+0000 7f17e77f8740 1 client.25996 shutdown 2024-08-30T09:05:38.527 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.524+0000 7f17e77f8740 1 -- 172.21.15.42:0/1671641364 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561a6dead4d0 msgr2=0x561a6e285e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:05:38.527 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.524+0000 7f17e77f8740 1 --2- 172.21.15.42:0/1671641364 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561a6dead4d0 0x561a6e285e80 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f17cc000c00 tx=0x7f17cc009fa0 comp rx=0 tx=0).stop 2024-08-30T09:05:38.527 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.525+0000 7f17e77f8740 1 -- 172.21.15.42:0/1671641364 shutdown_connections 2024-08-30T09:05:38.527 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.525+0000 7f17e77f8740 1 --2- 172.21.15.42:0/1671641364 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x561a6de621c0 0x561a6e2f84d0 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:38.527 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.525+0000 7f17e77f8740 1 --2- 172.21.15.42:0/1671641364 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561a6dead4d0 0x561a6e285e80 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:38.527 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.525+0000 7f17c1ffb640 0 client.25996 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:05:38.527 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.525+0000 7f17e77f8740 1 --2- 172.21.15.42:0/1671641364 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561a6dece1b0 0x561a6e284570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:38.527 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.525+0000 7f17e77f8740 1 --2- 172.21.15.42:0/1671641364 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561a6dea52f0 0x561a6e2857c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:38.527 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.525+0000 7f17e77f8740 1 -- 172.21.15.42:0/1671641364 >> 172.21.15.42:0/1671641364 conn(0x561a6dec2300 msgr2=0x561a6e26a670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:05:38.528 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.525+0000 7f17e77f8740 1 -- 172.21.15.42:0/1671641364 shutdown_connections 2024-08-30T09:05:38.528 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.525+0000 7f17e77f8740 1 -- 172.21.15.42:0/1671641364 wait complete. 2024-08-30T09:05:38.528 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.525+0000 7f17e77f8740 20 client.25996 trim_cache size 0 max 16384 2024-08-30T09:05:38.528 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-08-30T09:05:38.528 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2024-08-30T09:05:38.528 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-08-30T09:05:38.528 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-08-30T09:05:38.529 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.525+0000 7f17db59e640 1 -- 172.21.15.42:0/3120399519 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f17c40b4b30 msgr2=0x7f17c40b7040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:05:38.529 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.525+0000 7f17db59e640 1 --2- 172.21.15.42:0/3120399519 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f17c40b4b30 0x7f17c40b7040 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f17d0009ac0 tx=0x7f17d0008040 comp rx=0 tx=0).stop 2024-08-30T09:05:38.529 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.525+0000 7f17db59e640 1 -- 172.21.15.42:0/3120399519 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f17dc10a160 msgr2=0x7f17dc1a8960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:05:38.529 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.525+0000 7f17db59e640 1 --2- 172.21.15.42:0/3120399519 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f17dc10a160 0x7f17dc1a8960 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f17d402ae00 tx=0x7f17d4030870 comp rx=0 tx=0).stop 2024-08-30T09:05:38.529 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.526+0000 7f17db59e640 1 -- 172.21.15.42:0/3120399519 shutdown_connections 2024-08-30T09:05:38.529 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.526+0000 7f17db59e640 1 --2- 172.21.15.42:0/3120399519 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f17c40b4b30 0x7f17c40b7040 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:38.529 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.526+0000 7f17db59e640 1 --2- 172.21.15.42:0/3120399519 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f17dc10cac0 0x7f17dc1a8fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:38.529 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.526+0000 7f17db59e640 1 --2- 172.21.15.42:0/3120399519 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f17dc10f5d0 0x7f17dc1a9b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:38.529 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.526+0000 7f17db59e640 1 --2- 172.21.15.42:0/3120399519 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f17dc10a160 0x7f17dc1a8960 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:38.530 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.526+0000 7f17db59e640 1 -- 172.21.15.42:0/3120399519 >> 172.21.15.42:0/3120399519 conn(0x7f17dc0703e0 msgr2=0x7f17dc10af30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:05:38.530 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.526+0000 7f17db59e640 1 -- 172.21.15.42:0/3120399519 shutdown_connections 2024-08-30T09:05:38.530 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:38.526+0000 7f17db59e640 1 -- 172.21.15.42:0/3120399519 wait complete. 2024-08-30T09:05:38.741 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T09:05:39.074 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:05:38 smithi042 ceph-mon[18652]: pgmap v519: 129 pgs: 129 active+clean; 1.2 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 5.1 MiB/s wr, 580 op/s 2024-08-30T09:05:39.076 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:05:38 smithi167 ceph-mon[22063]: pgmap v519: 129 pgs: 129 active+clean; 1.2 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 5.1 MiB/s wr, 580 op/s 2024-08-30T09:05:39.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:05:38 smithi190 ceph-mon[22001]: pgmap v519: 129 pgs: 129 active+clean; 1.2 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 5.1 MiB/s wr, 580 op/s 2024-08-30T09:05:40.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.570+0000 7fae8889c640 1 -- 172.21.15.42:0/1576444332 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fae8410dbd0 msgr2=0x7fae841100c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:05:40.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.570+0000 7fae8889c640 1 --2- 172.21.15.42:0/1576444332 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fae8410dbd0 0x7fae841100c0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fae7c009ca0 tx=0x7fae7c02a8c0 comp rx=0 tx=0).stop 2024-08-30T09:05:40.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.571+0000 7fae8889c640 1 -- 172.21.15.42:0/1576444332 shutdown_connections 2024-08-30T09:05:40.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.571+0000 7fae8889c640 1 --2- 172.21.15.42:0/1576444332 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fae8410dbd0 0x7fae841100c0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:40.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.571+0000 7fae8889c640 1 --2- 172.21.15.42:0/1576444332 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fae841050c0 0x7fae8410d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:40.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.571+0000 7fae8889c640 1 --2- 172.21.15.42:0/1576444332 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fae84104770 0x7fae84104b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:40.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.571+0000 7fae8889c640 1 -- 172.21.15.42:0/1576444332 >> 172.21.15.42:0/1576444332 conn(0x7fae840794e0 msgr2=0x7fae840798f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:05:40.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.571+0000 7fae8889c640 1 -- 172.21.15.42:0/1576444332 shutdown_connections 2024-08-30T09:05:40.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.571+0000 7fae8889c640 1 -- 172.21.15.42:0/1576444332 wait complete. 2024-08-30T09:05:40.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.571+0000 7fae8889c640 1 Processor -- start 2024-08-30T09:05:40.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.572+0000 7fae8889c640 1 -- start start 2024-08-30T09:05:40.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.572+0000 7fae8889c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fae84104770 0x7fae8419a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:40.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.572+0000 7fae8889c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fae841050c0 0x7fae8419a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:40.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.572+0000 7fae8889c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fae8410dbd0 0x7fae8419fa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:40.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.572+0000 7fae8889c640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fae84113830 con 0x7fae8410dbd0 2024-08-30T09:05:40.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.572+0000 7fae8889c640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fae841136b0 con 0x7fae84104770 2024-08-30T09:05:40.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.572+0000 7fae8889c640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fae841139b0 con 0x7fae841050c0 2024-08-30T09:05:40.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.572+0000 7fae81d9b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fae841050c0 0x7fae8419a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:05:40.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.572+0000 7fae82d9d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fae8410dbd0 0x7fae8419fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:05:40.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.572+0000 7fae81d9b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fae841050c0 0x7fae8419a780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:58740/0 (socket says 172.21.15.42:58740) 2024-08-30T09:05:40.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.572+0000 7fae81d9b640 1 -- 172.21.15.42:0/1246667401 learned_addr learned my addr 172.21.15.42:0/1246667401 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:05:40.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.572+0000 7fae8259c640 1 --2- 172.21.15.42:0/1246667401 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fae84104770 0x7fae8419a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:05:40.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.572+0000 7fae81d9b640 1 -- 172.21.15.42:0/1246667401 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fae84104770 msgr2=0x7fae8419a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:05:40.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.572+0000 7fae81d9b640 1 --2- 172.21.15.42:0/1246667401 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fae84104770 0x7fae8419a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:40.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.572+0000 7fae81d9b640 1 -- 172.21.15.42:0/1246667401 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fae8410dbd0 msgr2=0x7fae8419fa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:05:40.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.572+0000 7fae81d9b640 1 --2- 172.21.15.42:0/1246667401 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fae8410dbd0 0x7fae8419fa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:40.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.573+0000 7fae81d9b640 1 -- 172.21.15.42:0/1246667401 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae841a0090 con 0x7fae841050c0 2024-08-30T09:05:40.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.573+0000 7fae8259c640 1 --2- 172.21.15.42:0/1246667401 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fae84104770 0x7fae8419a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-30T09:05:40.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.573+0000 7fae82d9d640 1 --2- 172.21.15.42:0/1246667401 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fae8410dbd0 0x7fae8419fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-30T09:05:40.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.573+0000 7fae81d9b640 1 --2- 172.21.15.42:0/1246667401 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fae841050c0 0x7fae8419a780 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fae6c00d740 tx=0x7fae6c00dc00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:05:40.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.573+0000 7fae7b7fe640 1 -- 172.21.15.42:0/1246667401 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae6c017070 con 0x7fae841050c0 2024-08-30T09:05:40.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.573+0000 7fae8889c640 1 -- 172.21.15.42:0/1246667401 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae841a0380 con 0x7fae841050c0 2024-08-30T09:05:40.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.573+0000 7fae7b7fe640 1 -- 172.21.15.42:0/1246667401 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae6c00de50 con 0x7fae841050c0 2024-08-30T09:05:40.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.573+0000 7fae7b7fe640 1 -- 172.21.15.42:0/1246667401 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae6c00f620 con 0x7fae841050c0 2024-08-30T09:05:40.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.573+0000 7fae8889c640 1 -- 172.21.15.42:0/1246667401 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae84102390 con 0x7fae841050c0 2024-08-30T09:05:40.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.576+0000 7fae7b7fe640 1 -- 172.21.15.42:0/1246667401 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7fae6c01a070 con 0x7fae841050c0 2024-08-30T09:05:40.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.576+0000 7fae8ee11740 1 Processor -- start 2024-08-30T09:05:40.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.576+0000 7fae8ee11740 1 -- start start 2024-08-30T09:05:40.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.576+0000 7fae8ee11740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560b8c93d070 0x560b8cce2310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:40.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.576+0000 7fae8ee11740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560b8c9141b0 0x560b8cce2850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:40.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.576+0000 7fae8ee11740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560b8c91c390 0x560b8cce2d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:40.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.576+0000 7fae8ee11740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x560b8cc59680 con 0x560b8c93d070 2024-08-30T09:05:40.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.576+0000 7fae8ee11740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x560b8cc48c10 con 0x560b8c9141b0 2024-08-30T09:05:40.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.576+0000 7fae8ee11740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x560b8cc4cbc0 con 0x560b8c91c390 2024-08-30T09:05:40.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.576+0000 7fae82d9d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560b8c93d070 0x560b8cce2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:05:40.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.576+0000 7fae82d9d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560b8c93d070 0x560b8cce2310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35606/0 (socket says 172.21.15.42:35606) 2024-08-30T09:05:40.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.576+0000 7fae82d9d640 1 -- 172.21.15.42:0/186043418 learned_addr learned my addr 172.21.15.42:0/186043418 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:05:40.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.576+0000 7fae81d9b640 1 --2- 172.21.15.42:0/186043418 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560b8c91c390 0x560b8cce2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:05:40.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.577+0000 7fae8259c640 1 --2- 172.21.15.42:0/186043418 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560b8c9141b0 0x560b8cce2850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:05:40.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.577+0000 7fae82d9d640 1 -- 172.21.15.42:0/186043418 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560b8c91c390 msgr2=0x560b8cce2d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:05:40.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.577+0000 7fae82d9d640 1 --2- 172.21.15.42:0/186043418 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560b8c91c390 0x560b8cce2d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:40.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.577+0000 7fae82d9d640 1 -- 172.21.15.42:0/186043418 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560b8c9141b0 msgr2=0x560b8cce2850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:05:40.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.577+0000 7fae82d9d640 1 --2- 172.21.15.42:0/186043418 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560b8c9141b0 0x560b8cce2850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:40.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.577+0000 7fae82d9d640 1 -- 172.21.15.42:0/186043418 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b8c9a8650 con 0x560b8c93d070 2024-08-30T09:05:40.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.577+0000 7fae8259c640 1 --2- 172.21.15.42:0/186043418 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560b8c9141b0 0x560b8cce2850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-30T09:05:40.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.577+0000 7fae7b7fe640 1 --2- 172.21.15.42:0/1246667401 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fae700b4b60 0x7fae700b7070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:40.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.577+0000 7fae8259c640 1 --2- 172.21.15.42:0/1246667401 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fae700b4b60 0x7fae700b7070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:05:40.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.577+0000 7fae82d9d640 1 --2- 172.21.15.42:0/186043418 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560b8c93d070 0x560b8cce2310 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fae7c033010 tx=0x7fae7c009800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6c91106b9d849b5 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:05:40.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.577+0000 7fae7b7fe640 1 -- 172.21.15.42:0/1246667401 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7fae6c0e4e30 con 0x7fae841050c0 2024-08-30T09:05:40.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.577+0000 7fae78ff9640 1 -- 172.21.15.42:0/186043418 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae7c031650 con 0x560b8c93d070 2024-08-30T09:05:40.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.577+0000 7fae78ff9640 1 -- 172.21.15.42:0/186043418 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae7c036040 con 0x560b8c93d070 2024-08-30T09:05:40.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.577+0000 7fae78ff9640 1 -- 172.21.15.42:0/186043418 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae7c004620 con 0x560b8c93d070 2024-08-30T09:05:40.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.578+0000 7fae8ee11740 1 -- 172.21.15.42:0/186043418 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560b8c93d070 msgr2=0x560b8cce2310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:05:40.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.578+0000 7fae8ee11740 1 --2- 172.21.15.42:0/186043418 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560b8c93d070 0x560b8cce2310 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fae7c033010 tx=0x7fae7c009800 comp rx=0 tx=0).stop 2024-08-30T09:05:40.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.578+0000 7fae8259c640 1 --2- 172.21.15.42:0/1246667401 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fae700b4b60 0x7fae700b7070 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fae7400bfd0 tx=0x7fae7400d040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:05:40.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.578+0000 7fae8ee11740 1 -- 172.21.15.42:0/186043418 shutdown_connections 2024-08-30T09:05:40.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.578+0000 7fae8ee11740 1 --2- 172.21.15.42:0/186043418 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560b8c91c390 0x560b8cce2d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:40.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.578+0000 7fae8ee11740 1 --2- 172.21.15.42:0/186043418 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560b8c9141b0 0x560b8cce2850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:40.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.578+0000 7fae8ee11740 1 --2- 172.21.15.42:0/186043418 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560b8c93d070 0x560b8cce2310 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:40.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.578+0000 7fae8ee11740 1 -- 172.21.15.42:0/186043418 >> 172.21.15.42:0/186043418 conn(0x560b8c931460 msgr2=0x560b8cce0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:05:40.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.578+0000 7fae8ee11740 1 -- 172.21.15.42:0/186043418 shutdown_connections 2024-08-30T09:05:40.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.578+0000 7fae8ee11740 1 -- 172.21.15.42:0/186043418 wait complete. 2024-08-30T09:05:40.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.579+0000 7fae8ee11740 1 -- start start 2024-08-30T09:05:40.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.579+0000 7fae8ee11740 1 Processor -- start 2024-08-30T09:05:40.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.579+0000 7fae8ee11740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560b8c9141b0 0x560b8cceda80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:40.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.579+0000 7fae8ee11740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560b8c91c390 0x560b8ccedfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:40.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.579+0000 7fae8ee11740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560b8c93d070 0x560b8ccf9c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:40.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.579+0000 7fae8ee11740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x560b8caab700 con 0x560b8c9141b0 2024-08-30T09:05:40.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.579+0000 7fae8ee11740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x560b8cbcf750 con 0x560b8c93d070 2024-08-30T09:05:40.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.579+0000 7fae8ee11740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x560b8ccf0910 con 0x560b8c91c390 2024-08-30T09:05:40.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.579+0000 7fae8ee11740 10 client.0 _ll_register_callbacks cb 0x560b8cbfee10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:05:40.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.579+0000 7fae8259c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560b8c9141b0 0x560b8cceda80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:05:40.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.579+0000 7fae81d9b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560b8c91c390 0x560b8ccedfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:05:40.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.579+0000 7fae8259c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560b8c9141b0 0x560b8cceda80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35612/0 (socket says 172.21.15.42:35612) 2024-08-30T09:05:40.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.579+0000 7fae8259c640 1 -- 172.21.15.42:0/620503813 learned_addr learned my addr 172.21.15.42:0/620503813 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:05:40.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.579+0000 7fae81d9b640 1 --2- 172.21.15.42:0/620503813 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560b8c91c390 0x560b8ccedfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:58756/0 (socket says 172.21.15.42:58756) 2024-08-30T09:05:40.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.579+0000 7fae82d9d640 1 --2- 172.21.15.42:0/620503813 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560b8c93d070 0x560b8ccf9c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:05:40.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.580+0000 7fae8259c640 1 -- 172.21.15.42:0/620503813 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560b8c91c390 msgr2=0x560b8ccedfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:05:40.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.580+0000 7fae8259c640 1 --2- 172.21.15.42:0/620503813 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560b8c91c390 0x560b8ccedfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:40.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.580+0000 7fae8259c640 1 -- 172.21.15.42:0/620503813 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560b8c93d070 msgr2=0x560b8ccf9c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:05:40.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.580+0000 7fae8259c640 1 --2- 172.21.15.42:0/620503813 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560b8c93d070 0x560b8ccf9c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:40.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.580+0000 7fae8259c640 1 -- 172.21.15.42:0/620503813 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b8cc08900 con 0x560b8c9141b0 2024-08-30T09:05:40.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.580+0000 7fae8259c640 1 --2- 172.21.15.42:0/620503813 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560b8c9141b0 0x560b8cceda80 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fae7401be50 tx=0x7fae74004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8c8639cb4e9913d server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:05:40.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.580+0000 7fae69ffb640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-08-30T09:05:40.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.580+0000 7fae69ffb640 1 -- 172.21.15.42:0/620503813 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae74029070 con 0x560b8c9141b0 2024-08-30T09:05:40.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.580+0000 7fae69ffb640 1 -- 172.21.15.42:0/620503813 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7fae74015450 con 0x560b8c9141b0 2024-08-30T09:05:40.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.580+0000 7fae8ee11740 1 -- 172.21.15.42:0/620503813 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae7c036040 con 0x560b8c9141b0 2024-08-30T09:05:40.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.580+0000 7fae69ffb640 1 -- 172.21.15.42:0/620503813 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fae74035050 con 0x560b8c9141b0 2024-08-30T09:05:40.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.580+0000 7fae8ee11740 10 client.16266 fetch_fsmap learned FSMap version 31 2024-08-30T09:05:40.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.580+0000 7fae8ee11740 1 -- 172.21.15.42:0/620503813 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x560b8ccfc730 con 0x560b8c9141b0 2024-08-30T09:05:40.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.580+0000 7fae69ffb640 1 -- 172.21.15.42:0/620503813 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae7c036040 con 0x560b8c9141b0 2024-08-30T09:05:40.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.581+0000 7fae69ffb640 1 -- 172.21.15.42:0/620503813 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7fae74024070 con 0x560b8c9141b0 2024-08-30T09:05:40.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.581+0000 7fae8ee11740 10 client.16266 fetch_fsmap finished waiting for FSMap version 31 2024-08-30T09:05:40.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.581+0000 7fae8ee11740 10 client.16266 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-08-30T09:05:40.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.581+0000 7fae8ee11740 20 client.16266 populate_metadata read hostname 'smithi042' 2024-08-30T09:05:40.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.581+0000 7fae8ee11740 1 --2- 172.21.15.42:0/620503813 >> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] conn(0x560b8c8fb9f0 0x560b8ccffbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:40.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.581+0000 7fae8ee11740 4 client.16266 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-30T09:05:40.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.581+0000 7fae8ee11740 1 -- 172.21.15.42:0/620503813 --> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560b8cc5b5d0 con 0x560b8c8fb9f0 2024-08-30T09:05:40.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.581+0000 7fae81d9b640 1 --2- 172.21.15.42:0/620503813 >> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] conn(0x560b8c8fb9f0 0x560b8ccffbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:05:40.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.581+0000 7fae69ffb640 10 client.16266 ms_handle_connect on v2:172.21.15.167:6836/3011837856 2024-08-30T09:05:40.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.581+0000 7fae81d9b640 1 --2- 172.21.15.42:0/620503813 >> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] conn(0x560b8c8fb9f0 0x560b8ccffbc0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a93240791f8748b8 server_cookie=a4c602f074c0a13a in_seq=0 out_seq=0 2024-08-30T09:05:40.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.582+0000 7fae69ffb640 1 -- 172.21.15.42:0/620503813 <== mds.2 v2:172.21.15.167:6836/3011837856 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x560b8cc5b5d0 con 0x560b8c8fb9f0 2024-08-30T09:05:40.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.582+0000 7fae69ffb640 10 client.16266 handle_command_reply: tid=0 2024-08-30T09:05:40.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.582+0000 7fae8ee11740 1 client.16266 shutdown 2024-08-30T09:05:40.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.582+0000 7fae8ee11740 1 -- 172.21.15.42:0/620503813 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560b8c9141b0 msgr2=0x560b8cceda80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:05:40.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.582+0000 7fae8ee11740 1 --2- 172.21.15.42:0/620503813 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560b8c9141b0 0x560b8cceda80 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fae7401be50 tx=0x7fae74004730 comp rx=0 tx=0).stop 2024-08-30T09:05:40.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.582+0000 7fae8ee11740 1 -- 172.21.15.42:0/620503813 shutdown_connections 2024-08-30T09:05:40.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.582+0000 7fae8ee11740 1 --2- 172.21.15.42:0/620503813 >> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] conn(0x560b8c8fb9f0 0x560b8ccffbc0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:40.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.582+0000 7fae8ee11740 1 --2- 172.21.15.42:0/620503813 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560b8c91c390 0x560b8ccedfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:40.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.582+0000 7fae8ee11740 1 --2- 172.21.15.42:0/620503813 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560b8c93d070 0x560b8ccf9c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:40.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.582+0000 7fae69ffb640 0 client.16266 ms_handle_reset on v2:172.21.15.167:6836/3011837856 2024-08-30T09:05:40.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.582+0000 7fae8ee11740 1 --2- 172.21.15.42:0/620503813 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560b8c9141b0 0x560b8cceda80 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:40.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.582+0000 7fae8ee11740 1 -- 172.21.15.42:0/620503813 >> 172.21.15.42:0/620503813 conn(0x560b8c931460 msgr2=0x560b8cce54f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:05:40.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.582+0000 7fae8ee11740 1 -- 172.21.15.42:0/620503813 shutdown_connections 2024-08-30T09:05:40.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.582+0000 7fae8ee11740 1 -- 172.21.15.42:0/620503813 wait complete. 2024-08-30T09:05:40.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.582+0000 7fae8ee11740 20 client.16266 trim_cache size 0 max 16384 2024-08-30T09:05:40.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.595+0000 7fae8ee11740 1 Processor -- start 2024-08-30T09:05:40.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.595+0000 7fae8ee11740 1 -- start start 2024-08-30T09:05:40.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.596+0000 7fae8ee11740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560b8c9141b0 0x560b8cce0270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:40.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.596+0000 7fae8ee11740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560b8c91c390 0x560b8ccf9b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:40.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.596+0000 7fae8ee11740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560b8c93d070 0x560b8cce5410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:40.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.596+0000 7fae8ee11740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fae841b0e70 con 0x560b8c93d070 2024-08-30T09:05:40.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.596+0000 7fae8ee11740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fae841ae740 con 0x560b8c9141b0 2024-08-30T09:05:40.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.596+0000 7fae8ee11740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fae841ac590 con 0x560b8c91c390 2024-08-30T09:05:40.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.596+0000 7fae82d9d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560b8c93d070 0x560b8cce5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:05:40.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.596+0000 7fae82d9d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560b8c93d070 0x560b8cce5410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35628/0 (socket says 172.21.15.42:35628) 2024-08-30T09:05:40.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.596+0000 7fae82d9d640 1 -- 172.21.15.42:0/4271470448 learned_addr learned my addr 172.21.15.42:0/4271470448 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:05:40.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.596+0000 7fae81d9b640 1 --2- 172.21.15.42:0/4271470448 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560b8c91c390 0x560b8ccf9b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:05:40.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.596+0000 7fae8259c640 1 --2- 172.21.15.42:0/4271470448 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560b8c9141b0 0x560b8cce0270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:05:40.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.596+0000 7fae82d9d640 1 -- 172.21.15.42:0/4271470448 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560b8c91c390 msgr2=0x560b8ccf9b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:05:40.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.596+0000 7fae82d9d640 1 --2- 172.21.15.42:0/4271470448 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560b8c91c390 0x560b8ccf9b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:40.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.596+0000 7fae82d9d640 1 -- 172.21.15.42:0/4271470448 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560b8c9141b0 msgr2=0x560b8cce0270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:05:40.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.596+0000 7fae82d9d640 1 --2- 172.21.15.42:0/4271470448 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560b8c9141b0 0x560b8cce0270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:40.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.596+0000 7fae82d9d640 1 -- 172.21.15.42:0/4271470448 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b8cce4600 con 0x560b8c93d070 2024-08-30T09:05:40.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.596+0000 7fae81d9b640 1 --2- 172.21.15.42:0/4271470448 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560b8c91c390 0x560b8ccf9b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-30T09:05:40.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.596+0000 7fae82d9d640 1 --2- 172.21.15.42:0/4271470448 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560b8c93d070 0x560b8cce5410 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fae7c0042d0 tx=0x7fae7c004300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c5bd78431d785bb server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:05:40.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.597+0000 7fae6bfff640 1 -- 172.21.15.42:0/4271470448 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae7c055070 con 0x560b8c93d070 2024-08-30T09:05:40.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.597+0000 7fae6bfff640 1 -- 172.21.15.42:0/4271470448 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae7c036040 con 0x560b8c93d070 2024-08-30T09:05:40.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.597+0000 7fae6bfff640 1 -- 172.21.15.42:0/4271470448 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae7c003bd0 con 0x560b8c93d070 2024-08-30T09:05:40.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.597+0000 7fae8ee11740 1 -- 172.21.15.42:0/4271470448 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560b8c93d070 msgr2=0x560b8cce5410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:05:40.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.597+0000 7fae8ee11740 1 --2- 172.21.15.42:0/4271470448 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560b8c93d070 0x560b8cce5410 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fae7c0042d0 tx=0x7fae7c004300 comp rx=0 tx=0).stop 2024-08-30T09:05:40.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.597+0000 7fae8ee11740 1 -- 172.21.15.42:0/4271470448 shutdown_connections 2024-08-30T09:05:40.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.597+0000 7fae8ee11740 1 --2- 172.21.15.42:0/4271470448 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560b8c91c390 0x560b8ccf9b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:40.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.597+0000 7fae8ee11740 1 --2- 172.21.15.42:0/4271470448 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560b8c93d070 0x560b8cce5410 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:40.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.597+0000 7fae8ee11740 1 --2- 172.21.15.42:0/4271470448 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560b8c9141b0 0x560b8cce0270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:40.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.597+0000 7fae8ee11740 1 -- 172.21.15.42:0/4271470448 >> 172.21.15.42:0/4271470448 conn(0x560b8c931460 msgr2=0x560b8ccdfb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:05:40.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.597+0000 7fae8ee11740 1 -- 172.21.15.42:0/4271470448 shutdown_connections 2024-08-30T09:05:40.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.597+0000 7fae8ee11740 1 -- 172.21.15.42:0/4271470448 wait complete. 2024-08-30T09:05:40.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.598+0000 7fae8ee11740 1 -- start start 2024-08-30T09:05:40.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.598+0000 7fae8ee11740 1 Processor -- start 2024-08-30T09:05:40.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.598+0000 7fae8ee11740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560b8c9141b0 0x560b8ccf1dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:40.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.598+0000 7fae8ee11740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560b8c91c390 0x560b8ccf2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:40.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.598+0000 7fae8ee11740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560b8c93d070 0x560b8cd695c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:40.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.598+0000 7fae8ee11740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fae841acc00 con 0x560b8c91c390 2024-08-30T09:05:40.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.598+0000 7fae8ee11740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fae841ae070 con 0x560b8c93d070 2024-08-30T09:05:40.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.598+0000 7fae8ee11740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fae841ad2d0 con 0x560b8c9141b0 2024-08-30T09:05:40.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.598+0000 7fae8ee11740 10 client.0 _ll_register_callbacks cb 0x7fae74005e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:05:40.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.598+0000 7fae81d9b640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560b8c91c390 0x560b8ccf2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:05:40.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.598+0000 7fae81d9b640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560b8c91c390 0x560b8ccf2300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35642/0 (socket says 172.21.15.42:35642) 2024-08-30T09:05:40.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.598+0000 7fae81d9b640 1 -- 172.21.15.42:0/1757426901 learned_addr learned my addr 172.21.15.42:0/1757426901 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:05:40.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.598+0000 7fae82d9d640 1 --2- 172.21.15.42:0/1757426901 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560b8c93d070 0x560b8cd695c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:05:40.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.599+0000 7fae8259c640 1 --2- 172.21.15.42:0/1757426901 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560b8c9141b0 0x560b8ccf1dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:05:40.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.599+0000 7fae81d9b640 1 -- 172.21.15.42:0/1757426901 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560b8c9141b0 msgr2=0x560b8ccf1dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:05:40.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.599+0000 7fae81d9b640 1 --2- 172.21.15.42:0/1757426901 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560b8c9141b0 0x560b8ccf1dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:40.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.599+0000 7fae81d9b640 1 -- 172.21.15.42:0/1757426901 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560b8c93d070 msgr2=0x560b8cd695c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:05:40.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.599+0000 7fae81d9b640 1 --2- 172.21.15.42:0/1757426901 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560b8c93d070 0x560b8cd695c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:40.481 INFO:teuthology.orchestra.run.smithi042.stdout:[] 2024-08-30T09:05:40.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.599+0000 7fae81d9b640 1 -- 172.21.15.42:0/1757426901 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b8ccf2b10 con 0x560b8c91c390 2024-08-30T09:05:40.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.599+0000 7fae81d9b640 1 --2- 172.21.15.42:0/1757426901 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560b8c91c390 0x560b8ccf2300 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fae6c009610 tx=0x7fae6c010c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84cde5733ff08002 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:05:40.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.599+0000 7fae69ffb640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-08-30T09:05:40.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.599+0000 7fae69ffb640 1 -- 172.21.15.42:0/1757426901 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae6c021040 con 0x560b8c91c390 2024-08-30T09:05:40.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.599+0000 7fae69ffb640 1 -- 172.21.15.42:0/1757426901 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7fae6c0a7810 con 0x560b8c91c390 2024-08-30T09:05:40.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.599+0000 7fae8ee11740 1 -- 172.21.15.42:0/1757426901 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae7c036040 con 0x560b8c91c390 2024-08-30T09:05:40.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.599+0000 7fae69ffb640 1 -- 172.21.15.42:0/1757426901 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fae6c0b4130 con 0x560b8c91c390 2024-08-30T09:05:40.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.599+0000 7fae8ee11740 10 client.16272 fetch_fsmap learned FSMap version 31 2024-08-30T09:05:40.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.599+0000 7fae8ee11740 1 -- 172.21.15.42:0/1757426901 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x560b8cd6ab20 con 0x560b8c91c390 2024-08-30T09:05:40.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.599+0000 7fae69ffb640 1 -- 172.21.15.42:0/1757426901 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae7c036040 con 0x560b8c91c390 2024-08-30T09:05:40.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.599+0000 7fae69ffb640 1 -- 172.21.15.42:0/1757426901 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7fae6c05b1a0 con 0x560b8c91c390 2024-08-30T09:05:40.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.599+0000 7fae8ee11740 10 client.16272 fetch_fsmap finished waiting for FSMap version 31 2024-08-30T09:05:40.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.599+0000 7fae8ee11740 10 client.16272 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-08-30T09:05:40.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.599+0000 7fae8ee11740 20 client.16272 populate_metadata read hostname 'smithi042' 2024-08-30T09:05:40.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.599+0000 7fae8ee11740 1 --2- 172.21.15.42:0/1757426901 >> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] conn(0x560b8c8d0da0 0x560b8cd6d240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:40.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.600+0000 7fae8ee11740 4 client.16272 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-30T09:05:40.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.600+0000 7fae8ee11740 1 -- 172.21.15.42:0/1757426901 --> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560b8cc57070 con 0x560b8c8d0da0 2024-08-30T09:05:40.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.600+0000 7fae8259c640 1 --2- 172.21.15.42:0/1757426901 >> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] conn(0x560b8c8d0da0 0x560b8cd6d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:05:40.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.600+0000 7fae69ffb640 10 client.16272 ms_handle_connect on v2:172.21.15.167:6836/3011837856 2024-08-30T09:05:40.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.600+0000 7fae8259c640 1 --2- 172.21.15.42:0/1757426901 >> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] conn(0x560b8c8d0da0 0x560b8cd6d240 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=168aa47f7641243 server_cookie=7f1d3b0eb3ad89f8 in_seq=0 out_seq=0 2024-08-30T09:05:40.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.600+0000 7fae69ffb640 1 -- 172.21.15.42:0/1757426901 <== mds.2 v2:172.21.15.167:6836/3011837856 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560b8cc57070 con 0x560b8c8d0da0 2024-08-30T09:05:40.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.600+0000 7fae69ffb640 10 client.16272 handle_command_reply: tid=0 2024-08-30T09:05:40.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.600+0000 7fae8ee11740 1 client.16272 shutdown 2024-08-30T09:05:40.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.600+0000 7fae8ee11740 1 -- 172.21.15.42:0/1757426901 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560b8c91c390 msgr2=0x560b8ccf2300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:05:40.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.600+0000 7fae8ee11740 1 --2- 172.21.15.42:0/1757426901 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560b8c91c390 0x560b8ccf2300 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fae6c009610 tx=0x7fae6c010c70 comp rx=0 tx=0).stop 2024-08-30T09:05:40.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.600+0000 7fae8ee11740 1 -- 172.21.15.42:0/1757426901 shutdown_connections 2024-08-30T09:05:40.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.600+0000 7fae8ee11740 1 --2- 172.21.15.42:0/1757426901 >> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] conn(0x560b8c8d0da0 0x560b8cd6d240 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:40.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.600+0000 7fae8ee11740 1 --2- 172.21.15.42:0/1757426901 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560b8c93d070 0x560b8cd695c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:40.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.600+0000 7fae69ffb640 0 client.16272 ms_handle_reset on v2:172.21.15.167:6836/3011837856 2024-08-30T09:05:40.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.600+0000 7fae8ee11740 1 --2- 172.21.15.42:0/1757426901 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560b8c91c390 0x560b8ccf2300 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:40.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.600+0000 7fae8ee11740 1 --2- 172.21.15.42:0/1757426901 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560b8c9141b0 0x560b8ccf1dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:40.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.600+0000 7fae8ee11740 1 -- 172.21.15.42:0/1757426901 >> 172.21.15.42:0/1757426901 conn(0x560b8c931460 msgr2=0x560b8cce5410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:05:40.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.601+0000 7fae8ee11740 1 -- 172.21.15.42:0/1757426901 shutdown_connections 2024-08-30T09:05:40.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.601+0000 7fae8ee11740 1 -- 172.21.15.42:0/1757426901 wait complete. 2024-08-30T09:05:40.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.601+0000 7fae8ee11740 20 client.16272 trim_cache size 0 max 16384 2024-08-30T09:05:40.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.601+0000 7fae8889c640 1 -- 172.21.15.42:0/1246667401 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fae700b4b60 msgr2=0x7fae700b7070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:05:40.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.601+0000 7fae8889c640 1 --2- 172.21.15.42:0/1246667401 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fae700b4b60 0x7fae700b7070 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fae7400bfd0 tx=0x7fae7400d040 comp rx=0 tx=0).stop 2024-08-30T09:05:40.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.601+0000 7fae8889c640 1 -- 172.21.15.42:0/1246667401 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fae841050c0 msgr2=0x7fae8419a780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:05:40.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.601+0000 7fae8889c640 1 --2- 172.21.15.42:0/1246667401 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fae841050c0 0x7fae8419a780 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fae6c00d740 tx=0x7fae6c00dc00 comp rx=0 tx=0).stop 2024-08-30T09:05:40.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.601+0000 7fae8889c640 1 -- 172.21.15.42:0/1246667401 shutdown_connections 2024-08-30T09:05:40.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.601+0000 7fae8889c640 1 --2- 172.21.15.42:0/1246667401 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fae700b4b60 0x7fae700b7070 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:40.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.601+0000 7fae8889c640 1 --2- 172.21.15.42:0/1246667401 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fae841050c0 0x7fae8419a780 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:40.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.601+0000 7fae8889c640 1 --2- 172.21.15.42:0/1246667401 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fae8410dbd0 0x7fae8419fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:40.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.601+0000 7fae8889c640 1 --2- 172.21.15.42:0/1246667401 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fae84104770 0x7fae8419a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:40.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.601+0000 7fae8889c640 1 -- 172.21.15.42:0/1246667401 >> 172.21.15.42:0/1246667401 conn(0x7fae840794e0 msgr2=0x7fae840ff310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:05:40.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.601+0000 7fae8889c640 1 -- 172.21.15.42:0/1246667401 shutdown_connections 2024-08-30T09:05:40.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:39.601+0000 7fae8889c640 1 -- 172.21.15.42:0/1246667401 wait complete. 2024-08-30T09:05:40.953 INFO:teuthology.orchestra.run.smithi042.stderr:progress: 295.0 s, 557.2 tps, lat 182.803 ms stddev 212.632 2024-08-30T09:05:41.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:05:40 smithi042 ceph-mon[18652]: pgmap v520: 129 pgs: 129 active+clean; 1.2 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 4.9 MiB/s wr, 564 op/s 2024-08-30T09:05:41.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:05:40 smithi190 ceph-mon[22001]: pgmap v520: 129 pgs: 129 active+clean; 1.2 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 4.9 MiB/s wr, 564 op/s 2024-08-30T09:05:41.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:05:40 smithi167 ceph-mon[22063]: pgmap v520: 129 pgs: 129 active+clean; 1.2 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 4.9 MiB/s wr, 564 op/s 2024-08-30T09:05:41.850 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-08-30T09:05:42.358 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T09:05:42.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:42.976+0000 7f91ea914640 1 -- 172.21.15.42:0/3037793557 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f91e4102580 msgr2=0x7f91e4102990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:05:42.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:42.976+0000 7f91ea914640 1 --2- 172.21.15.42:0/3037793557 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f91e4102580 0x7f91e4102990 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f91dc009930 tx=0x7f91dc02a480 comp rx=0 tx=0).stop 2024-08-30T09:05:42.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:42.977+0000 7f91ea914640 1 -- 172.21.15.42:0/3037793557 shutdown_connections 2024-08-30T09:05:42.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:42.977+0000 7f91ea914640 1 --2- 172.21.15.42:0/3037793557 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f91e4102ed0 0x7f91e4107400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:42.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:42.977+0000 7f91ea914640 1 --2- 172.21.15.42:0/3037793557 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f91e41079c0 0x7f91e4107ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:42.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:42.977+0000 7f91ea914640 1 --2- 172.21.15.42:0/3037793557 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f91e4102580 0x7f91e4102990 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:42.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:42.977+0000 7f91ea914640 1 -- 172.21.15.42:0/3037793557 >> 172.21.15.42:0/3037793557 conn(0x7f91e4079580 msgr2=0x7f91e4079990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:05:42.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:42.982+0000 7f91ea914640 1 -- 172.21.15.42:0/3037793557 shutdown_connections 2024-08-30T09:05:42.987 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:42.982+0000 7f91ea914640 1 -- 172.21.15.42:0/3037793557 wait complete. 2024-08-30T09:05:42.987 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:42.983+0000 7f91ea914640 1 Processor -- start 2024-08-30T09:05:42.988 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:42.983+0000 7f91ea914640 1 -- start start 2024-08-30T09:05:42.988 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:42.983+0000 7f91ea914640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f91e4102ed0 0x7f91e41167a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:42.988 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:42.983+0000 7f91ea914640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f91e41079c0 0x7f91e410f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:42.988 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:42.983+0000 7f91ea914640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f91e410fef0 0x7f91e4110300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:42.988 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:42.983+0000 7f91ea914640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f91e4119fb0 con 0x7f91e4102ed0 2024-08-30T09:05:42.988 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:42.984+0000 7f91ea914640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f91e4119e30 con 0x7f91e410fef0 2024-08-30T09:05:42.988 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:42.984+0000 7f91ea914640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f91e411a130 con 0x7f91e41079c0 2024-08-30T09:05:42.988 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:42.984+0000 7f91e3fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f91e4102ed0 0x7f91e41167a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:05:42.988 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:42.984+0000 7f91e3fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f91e4102ed0 0x7f91e41167a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35652/0 (socket says 172.21.15.42:35652) 2024-08-30T09:05:42.988 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:42.984+0000 7f91e3fff640 1 -- 172.21.15.42:0/4053956055 learned_addr learned my addr 172.21.15.42:0/4053956055 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:05:42.988 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:42.984+0000 7f91e8eb1640 1 --2- 172.21.15.42:0/4053956055 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f91e410fef0 0x7f91e4110300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:05:42.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:42.985+0000 7f91e8eb1640 1 -- 172.21.15.42:0/4053956055 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f91e41079c0 msgr2=0x7f91e410f9b0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-08-30T09:05:42.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:42.985+0000 7f91e8eb1640 1 --2- 172.21.15.42:0/4053956055 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f91e41079c0 0x7f91e410f9b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:42.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:42.985+0000 7f91e8eb1640 1 -- 172.21.15.42:0/4053956055 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f91e4102ed0 msgr2=0x7f91e41167a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:05:42.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:42.985+0000 7f91e8eb1640 1 --2- 172.21.15.42:0/4053956055 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f91e4102ed0 0x7f91e41167a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:42.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:42.985+0000 7f91e8eb1640 1 -- 172.21.15.42:0/4053956055 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91e4115350 con 0x7f91e410fef0 2024-08-30T09:05:42.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:42.985+0000 7f91e8eb1640 1 --2- 172.21.15.42:0/4053956055 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f91e410fef0 0x7f91e4110300 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f91d400be20 tx=0x7f91d400bf10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:05:42.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:42.986+0000 7f91e17fa640 1 -- 172.21.15.42:0/4053956055 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91d4019070 con 0x7f91e410fef0 2024-08-30T09:05:42.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:42.989+0000 7f91ea914640 1 -- 172.21.15.42:0/4053956055 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91e4115640 con 0x7f91e410fef0 2024-08-30T09:05:42.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:42.989+0000 7f91ea914640 1 -- 172.21.15.42:0/4053956055 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91e4115b80 con 0x7f91e410fef0 2024-08-30T09:05:42.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:42.991+0000 7f91e17fa640 1 -- 172.21.15.42:0/4053956055 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f91d40148f0 con 0x7f91e410fef0 2024-08-30T09:05:42.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:42.991+0000 7f91e17fa640 1 -- 172.21.15.42:0/4053956055 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91d4012860 con 0x7f91e410fef0 2024-08-30T09:05:42.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:42.998+0000 7f91f0e89740 1 Processor -- start 2024-08-30T09:05:43.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:42.998+0000 7f91e17fa640 1 -- 172.21.15.42:0/4053956055 <== mon.2 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7f91d401c070 con 0x7f91e410fef0 2024-08-30T09:05:43.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:42.998+0000 7f91f0e89740 1 -- start start 2024-08-30T09:05:43.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:42.998+0000 7f91f0e89740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564031f32070 0x5640322d7350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:43.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:42.998+0000 7f91f0e89740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564031f091b0 0x5640322d7890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:43.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:43.000+0000 7f91e17fa640 1 --2- 172.21.15.42:0/4053956055 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f91d00b4c60 0x7f91d00b7170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:43.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:43.000+0000 7f91e17fa640 1 -- 172.21.15.42:0/4053956055 <== mon.2 v2:172.21.15.167:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f91d40e5100 con 0x7f91e410fef0 2024-08-30T09:05:43.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:43.000+0000 7f91e37fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564031f32070 0x5640322d7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:05:43.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:43.000+0000 7f91e37fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564031f32070 0x5640322d7350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:48322/0 (socket says 172.21.15.42:48322) 2024-08-30T09:05:43.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:43.000+0000 7f91e37fe640 1 -- 172.21.15.42:0/677611000 learned_addr learned my addr 172.21.15.42:0/677611000 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:05:43.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:43.001+0000 7f91f0e89740 1 --2- 172.21.15.42:0/677611000 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564031f11390 0x5640322d7dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:43.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:43.001+0000 7f91f0e89740 1 -- 172.21.15.42:0/677611000 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x56403224e680 con 0x564031f091b0 2024-08-30T09:05:43.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:43.001+0000 7f91f0e89740 1 -- 172.21.15.42:0/677611000 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x56403223dc10 con 0x564031f32070 2024-08-30T09:05:43.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:43.001+0000 7f91f0e89740 1 -- 172.21.15.42:0/677611000 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x564032241bc0 con 0x564031f11390 2024-08-30T09:05:43.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:43.001+0000 7f91e3fff640 1 --2- 172.21.15.42:0/677611000 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564031f091b0 0x5640322d7890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:05:43.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:43.002+0000 7f91e37fe640 1 --2- 172.21.15.42:0/4053956055 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f91d00b4c60 0x7f91d00b7170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:05:43.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:43.002+0000 7f91e8eb1640 1 --2- 172.21.15.42:0/677611000 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564031f11390 0x5640322d7dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:05:43.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:43.003+0000 7f91e8eb1640 1 -- 172.21.15.42:0/677611000 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564031f32070 msgr2=0x5640322d7350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:05:43.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:43.003+0000 7f91e8eb1640 1 --2- 172.21.15.42:0/677611000 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564031f32070 0x5640322d7350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:43.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:43.003+0000 7f91e8eb1640 1 -- 172.21.15.42:0/677611000 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564031f091b0 msgr2=0x5640322d7890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:05:43.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:43.003+0000 7f91e8eb1640 1 --2- 172.21.15.42:0/677611000 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564031f091b0 0x5640322d7890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:43.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:43.003+0000 7f91e8eb1640 1 -- 172.21.15.42:0/677611000 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564031f9d650 con 0x564031f11390 2024-08-30T09:05:43.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:43.006+0000 7f91e8eb1640 1 --2- 172.21.15.42:0/677611000 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564031f11390 0x5640322d7dd0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f91d4009fb0 tx=0x7f91d40aba60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3d64dbfddbbb53b2 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:05:43.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:43.008+0000 7f91da7fc640 1 -- 172.21.15.42:0/677611000 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91d40ac2f0 con 0x564031f11390 2024-08-30T09:05:43.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:43.008+0000 7f91da7fc640 1 -- 172.21.15.42:0/677611000 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f91d40b0b60 con 0x564031f11390 2024-08-30T09:05:43.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:43.008+0000 7f91da7fc640 1 -- 172.21.15.42:0/677611000 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91d40ab550 con 0x564031f11390 2024-08-30T09:05:43.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:43.009+0000 7f91f0e89740 1 -- 172.21.15.42:0/677611000 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564031f11390 msgr2=0x5640322d7dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:05:43.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:43.009+0000 7f91f0e89740 1 --2- 172.21.15.42:0/677611000 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564031f11390 0x5640322d7dd0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f91d4009fb0 tx=0x7f91d40aba60 comp rx=0 tx=0).stop 2024-08-30T09:05:43.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:43.009+0000 7f91f0e89740 1 -- 172.21.15.42:0/677611000 shutdown_connections 2024-08-30T09:05:43.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:43.009+0000 7f91f0e89740 1 --2- 172.21.15.42:0/677611000 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564031f11390 0x5640322d7dd0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:43.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:43.009+0000 7f91f0e89740 1 --2- 172.21.15.42:0/677611000 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564031f091b0 0x5640322d7890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:43.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:43.009+0000 7f91f0e89740 1 --2- 172.21.15.42:0/677611000 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564031f32070 0x5640322d7350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:43.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:43.009+0000 7f91f0e89740 1 -- 172.21.15.42:0/677611000 >> 172.21.15.42:0/677611000 conn(0x564031f26460 msgr2=0x5640322d5240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:05:43.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:43.011+0000 7f91f0e89740 1 -- 172.21.15.42:0/677611000 shutdown_connections 2024-08-30T09:05:43.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:43.012+0000 7f91e37fe640 1 --2- 172.21.15.42:0/4053956055 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f91d00b4c60 0x7f91d00b7170 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f91cc00bfd0 tx=0x7f91cc004750 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:05:43.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:43.012+0000 7f91f0e89740 1 -- 172.21.15.42:0/677611000 wait complete. 2024-08-30T09:05:43.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:43.012+0000 7f91f0e89740 1 -- start start 2024-08-30T09:05:43.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:43.013+0000 7f91f0e89740 1 Processor -- start 2024-08-30T09:05:43.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:43.013+0000 7f91f0e89740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564031f091b0 0x5640322e2b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:43.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:43.014+0000 7f91f0e89740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564031f11390 0x5640322e30c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:43.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:43.014+0000 7f91f0e89740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564031f32070 0x5640322eee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:43.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:43.014+0000 7f91f0e89740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x5640320a0700 con 0x564031f091b0 2024-08-30T09:05:43.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:43.014+0000 7f91f0e89740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5640321c4750 con 0x564031f32070 2024-08-30T09:05:43.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:43.014+0000 7f91f0e89740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5640322e5b30 con 0x564031f11390 2024-08-30T09:05:43.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:43.014+0000 7f91e3fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564031f091b0 0x5640322e2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:05:43.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:43.014+0000 7f91e3fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564031f091b0 0x5640322e2b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35668/0 (socket says 172.21.15.42:35668) 2024-08-30T09:05:43.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:43.014+0000 7f91e3fff640 1 -- 172.21.15.42:0/905906185 learned_addr learned my addr 172.21.15.42:0/905906185 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:05:43.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:43.014+0000 7f91e8eb1640 1 --2- 172.21.15.42:0/905906185 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564031f32070 0x5640322eee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:05:43.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:43.014+0000 7f91e8eb1640 1 -- 172.21.15.42:0/905906185 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564031f11390 msgr2=0x5640322e30c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-30T09:05:43.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:43.014+0000 7f91e8eb1640 1 --2- 172.21.15.42:0/905906185 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564031f11390 0x5640322e30c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:43.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:43.014+0000 7f91e8eb1640 1 -- 172.21.15.42:0/905906185 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564031f091b0 msgr2=0x5640322e2b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:05:43.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:43.014+0000 7f91e8eb1640 1 --2- 172.21.15.42:0/905906185 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564031f091b0 0x5640322e2b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:43.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:43.014+0000 7f91e8eb1640 1 -- 172.21.15.42:0/905906185 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5640321fd900 con 0x564031f32070 2024-08-30T09:05:43.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:43.015+0000 7f91e8eb1640 1 --2- 172.21.15.42:0/905906185 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564031f32070 0x5640322eee50 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x5640322e5800 tx=0x7f91d40b53b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1496dd0da6ba252c server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:05:43.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:43.015+0000 7f91f0e89740 10 client.0 _ll_register_callbacks cb 0x5640321f3e10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:05:43.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:43.015+0000 7f91f0e89740 1 -- 172.21.15.42:0/905906185 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f91d40b0b60 con 0x564031f32070 2024-08-30T09:05:43.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:43.015+0000 7f91c37fe640 10 client.26021 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-30T09:05:43.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:43.015+0000 7f91c37fe640 1 -- 172.21.15.42:0/905906185 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91d40cf0e0 con 0x564031f32070 2024-08-30T09:05:43.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:43.017+0000 7f91c37fe640 1 -- 172.21.15.42:0/905906185 <== mon.2 v2:172.21.15.167:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f91d4014650 con 0x564031f32070 2024-08-30T09:05:43.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:43.017+0000 7f91c37fe640 1 -- 172.21.15.42:0/905906185 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f91d40ac2f0 con 0x564031f32070 2024-08-30T09:05:43.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:43.017+0000 7f91f0e89740 10 client.26021 fetch_fsmap learned FSMap version 31 2024-08-30T09:05:43.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:43.017+0000 7f91f0e89740 1 -- 172.21.15.42:0/905906185 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({fsmap=31}) -- 0x5640322f13e0 con 0x564031f32070 2024-08-30T09:05:43.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:43.017+0000 7f91c37fe640 1 -- 172.21.15.42:0/905906185 <== mon.2 v2:172.21.15.167:3300/0 4 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f91d40ab550 con 0x564031f32070 2024-08-30T09:05:43.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:43.018+0000 7f91f0e89740 10 client.26021 fetch_fsmap finished waiting for FSMap version 31 2024-08-30T09:05:43.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:43.018+0000 7f91f0e89740 10 client.26021 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-08-30T09:05:43.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:43.018+0000 7f91f0e89740 20 client.26021 populate_metadata read hostname 'smithi042' 2024-08-30T09:05:43.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:43.018+0000 7f91f0e89740 1 --2- 172.21.15.42:0/905906185 >> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] conn(0x564031ef09f0 0x5640322f4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:43.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:43.018+0000 7f91f0e89740 4 client.26021 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-30T09:05:43.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:43.018+0000 7f91f0e89740 1 -- 172.21.15.42:0/905906185 --> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f91d40b0b60 con 0x564031ef09f0 2024-08-30T09:05:43.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:43.020+0000 7f91e3fff640 1 --2- 172.21.15.42:0/905906185 >> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] conn(0x564031ef09f0 0x5640322f4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:05:43.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:43.021+0000 7f91e3fff640 1 --2- 172.21.15.42:0/905906185 >> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] conn(0x564031ef09f0 0x5640322f4ca0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8d34f171e8dd7d57 server_cookie=fe90ea243db8334f in_seq=0 out_seq=0 2024-08-30T09:05:43.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:43.021+0000 7f91c37fe640 10 client.26021 ms_handle_connect on v2:172.21.15.190:6838/719981110 2024-08-30T09:05:43.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:43.022+0000 7f91c37fe640 1 -- 172.21.15.42:0/905906185 <== mds.1 v2:172.21.15.190:6838/719981110 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f91d40b0b60 con 0x564031ef09f0 2024-08-30T09:05:43.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:43.022+0000 7f91c37fe640 10 client.26021 handle_command_reply: tid=0 2024-08-30T09:05:43.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:43.022+0000 7f91f0e89740 1 client.26021 shutdown 2024-08-30T09:05:43.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:43.026+0000 7f91f0e89740 1 -- 172.21.15.42:0/905906185 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564031f32070 msgr2=0x5640322eee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:05:43.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:43.026+0000 7f91f0e89740 1 --2- 172.21.15.42:0/905906185 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564031f32070 0x5640322eee50 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x5640322e5800 tx=0x7f91d40b53b0 comp rx=0 tx=0).stop 2024-08-30T09:05:43.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:43.026+0000 7f91f0e89740 1 -- 172.21.15.42:0/905906185 shutdown_connections 2024-08-30T09:05:43.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:43.026+0000 7f91f0e89740 1 --2- 172.21.15.42:0/905906185 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564031f11390 0x5640322e30c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:43.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:43.026+0000 7f91f0e89740 1 --2- 172.21.15.42:0/905906185 >> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] conn(0x564031ef09f0 0x5640322f4ca0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:43.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:43.026+0000 7f91f0e89740 1 --2- 172.21.15.42:0/905906185 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564031f32070 0x5640322eee50 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:43.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:43.026+0000 7f91f0e89740 1 --2- 172.21.15.42:0/905906185 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564031f091b0 0x5640322e2b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:43.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:43.026+0000 7f91f0e89740 1 -- 172.21.15.42:0/905906185 >> 172.21.15.42:0/905906185 conn(0x564031f26460 msgr2=0x5640322da4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:05:43.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:43.027+0000 7f91c37fe640 0 client.26021 ms_handle_reset on v2:172.21.15.190:6838/719981110 2024-08-30T09:05:43.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:43.033+0000 7f91f0e89740 1 -- 172.21.15.42:0/905906185 shutdown_connections 2024-08-30T09:05:43.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:43.033+0000 7f91f0e89740 1 -- 172.21.15.42:0/905906185 wait complete. 2024-08-30T09:05:43.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:43.033+0000 7f91f0e89740 20 client.26021 trim_cache size 0 max 16384 2024-08-30T09:05:43.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:43.070+0000 7f91f0e89740 1 Processor -- start 2024-08-30T09:05:43.073 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:05:42 smithi042 ceph-mon[18652]: pgmap v521: 129 pgs: 129 active+clean; 1.2 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 4.8 MiB/s wr, 537 op/s 2024-08-30T09:05:43.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:43.071+0000 7f91f0e89740 1 -- start start 2024-08-30T09:05:43.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:43.071+0000 7f91f0e89740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564031f091b0 0x5640322dab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:43.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:43.071+0000 7f91f0e89740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564031f11390 0x5640322dfd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:43.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:43.071+0000 7f91f0e89740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564031f32070 0x5640322e02b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:43.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:43.071+0000 7f91f0e89740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f91c40124f0 con 0x564031f11390 2024-08-30T09:05:43.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:43.071+0000 7f91f0e89740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f91c400fdc0 con 0x564031f32070 2024-08-30T09:05:43.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:43.071+0000 7f91f0e89740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f91c400dc10 con 0x564031f091b0 2024-08-30T09:05:43.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:43.072+0000 7f91e3fff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564031f091b0 0x5640322dab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:05:43.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:43.072+0000 7f91e8eb1640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564031f32070 0x5640322e02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:05:43.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:43.072+0000 7f91e8eb1640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564031f32070 0x5640322e02b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:48334/0 (socket says 172.21.15.42:48334) 2024-08-30T09:05:43.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:43.072+0000 7f91e8eb1640 1 -- 172.21.15.42:0/4022690646 learned_addr learned my addr 172.21.15.42:0/4022690646 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:05:43.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:43.072+0000 7f91e37fe640 1 --2- 172.21.15.42:0/4022690646 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564031f11390 0x5640322dfd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:05:43.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:43.075+0000 7f91e3fff640 1 -- 172.21.15.42:0/4022690646 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564031f32070 msgr2=0x5640322e02b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:05:43.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:43.075+0000 7f91e3fff640 1 --2- 172.21.15.42:0/4022690646 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564031f32070 0x5640322e02b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:43.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:43.075+0000 7f91e3fff640 1 -- 172.21.15.42:0/4022690646 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564031f11390 msgr2=0x5640322dfd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:05:43.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:43.075+0000 7f91e3fff640 1 --2- 172.21.15.42:0/4022690646 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564031f11390 0x5640322dfd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:43.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:43.075+0000 7f91e3fff640 1 -- 172.21.15.42:0/4022690646 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5640322d9be0 con 0x564031f091b0 2024-08-30T09:05:43.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:43.077+0000 7f91e3fff640 1 --2- 172.21.15.42:0/4022690646 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564031f091b0 0x5640322dab60 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x5640322e53d0 tx=0x7f91dc02a980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8a6d773607aac6a server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:05:43.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:43.078+0000 7f91d9ffb640 1 -- 172.21.15.42:0/4022690646 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91dc049020 con 0x564031f091b0 2024-08-30T09:05:43.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:43.079+0000 7f91d9ffb640 1 -- 172.21.15.42:0/4022690646 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f91dc032e50 con 0x564031f091b0 2024-08-30T09:05:43.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:43.080+0000 7f91f0e89740 1 -- 172.21.15.42:0/4022690646 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564031f091b0 msgr2=0x5640322dab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:05:43.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:43.080+0000 7f91f0e89740 1 --2- 172.21.15.42:0/4022690646 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564031f091b0 0x5640322dab60 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x5640322e53d0 tx=0x7f91dc02a980 comp rx=0 tx=0).stop 2024-08-30T09:05:43.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:43.081+0000 7f91f0e89740 1 -- 172.21.15.42:0/4022690646 shutdown_connections 2024-08-30T09:05:43.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:43.081+0000 7f91f0e89740 1 --2- 172.21.15.42:0/4022690646 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564031f32070 0x5640322e02b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:43.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:43.081+0000 7f91f0e89740 1 --2- 172.21.15.42:0/4022690646 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564031f11390 0x5640322dfd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:43.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:43.081+0000 7f91f0e89740 1 --2- 172.21.15.42:0/4022690646 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564031f091b0 0x5640322dab60 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:43.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:43.081+0000 7f91f0e89740 1 -- 172.21.15.42:0/4022690646 >> 172.21.15.42:0/4022690646 conn(0x564031f26460 msgr2=0x5640322da450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:05:43.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:43.082+0000 7f91f0e89740 1 -- 172.21.15.42:0/4022690646 shutdown_connections 2024-08-30T09:05:43.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:43.082+0000 7f91f0e89740 1 -- 172.21.15.42:0/4022690646 wait complete. 2024-08-30T09:05:43.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:43.082+0000 7f91f0e89740 1 -- start start 2024-08-30T09:05:43.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:43.083+0000 7f91f0e89740 1 Processor -- start 2024-08-30T09:05:43.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:43.083+0000 7f91f0e89740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564031f091b0 0x5640322eb480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:43.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:43.083+0000 7f91f0e89740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564031f11390 0x5640322e8950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:43.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:43.083+0000 7f91f0e89740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564031f32070 0x5640322e8e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:43.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:43.083+0000 7f91f0e89740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f91c400e280 con 0x564031f11390 2024-08-30T09:05:43.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:43.083+0000 7f91f0e89740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f91c400f6f0 con 0x564031f32070 2024-08-30T09:05:43.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:43.083+0000 7f91f0e89740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f91c400e950 con 0x564031f091b0 2024-08-30T09:05:43.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:43.083+0000 7f91f0e89740 10 client.0 _ll_register_callbacks cb 0x7f91dc009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:05:43.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:43.084+0000 7f91e8eb1640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564031f32070 0x5640322e8e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:05:43.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:43.084+0000 7f91e8eb1640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564031f32070 0x5640322e8e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:48346/0 (socket says 172.21.15.42:48346) 2024-08-30T09:05:43.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:43.084+0000 7f91e8eb1640 1 -- 172.21.15.42:0/848170968 learned_addr learned my addr 172.21.15.42:0/848170968 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:05:43.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:43.088+0000 7f91e8eb1640 1 -- 172.21.15.42:0/848170968 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564031f091b0 msgr2=0x5640322eb480 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-30T09:05:43.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:43.088+0000 7f91e8eb1640 1 --2- 172.21.15.42:0/848170968 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564031f091b0 0x5640322eb480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:43.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:43.088+0000 7f91e8eb1640 1 -- 172.21.15.42:0/848170968 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564031f11390 msgr2=0x5640322e8950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:05:43.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:43.088+0000 7f91e8eb1640 1 --2- 172.21.15.42:0/848170968 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564031f11390 0x5640322e8950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:43.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:43.088+0000 7f91e8eb1640 1 -- 172.21.15.42:0/848170968 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5640322e9500 con 0x564031f32070 2024-08-30T09:05:43.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:43.088+0000 7f91e8eb1640 1 --2- 172.21.15.42:0/848170968 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564031f32070 0x5640322e8e90 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f91d40ac280 tx=0x7f91d40cf070 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c93dc749c8d996a8 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:05:43.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:43.088+0000 7f91c37fe640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-30T09:05:43.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:43.089+0000 7f91c37fe640 1 -- 172.21.15.42:0/848170968 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91d4014dd0 con 0x564031f32070 2024-08-30T09:05:43.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:43.089+0000 7f91f0e89740 1 -- 172.21.15.42:0/848170968 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5640322505d0 con 0x564031f32070 2024-08-30T09:05:43.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:43.089+0000 7f91c37fe640 1 -- 172.21.15.42:0/848170968 <== mon.2 v2:172.21.15.167:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f91d40a7e70 con 0x564031f32070 2024-08-30T09:05:43.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:43.091+0000 7f91c37fe640 1 -- 172.21.15.42:0/848170968 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f91d40ac0c0 con 0x564031f32070 2024-08-30T09:05:43.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:43.091+0000 7f91f0e89740 10 client.26027 fetch_fsmap learned FSMap version 31 2024-08-30T09:05:43.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:43.091+0000 7f91f0e89740 1 -- 172.21.15.42:0/848170968 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x56403235f570 con 0x564031f32070 2024-08-30T09:05:43.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:43.093+0000 7f91c37fe640 1 -- 172.21.15.42:0/848170968 <== mon.2 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5640322505d0 con 0x564031f32070 2024-08-30T09:05:43.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:43.093+0000 7f91c37fe640 1 -- 172.21.15.42:0/848170968 <== mon.2 v2:172.21.15.167:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f91d40aafc0 con 0x564031f32070 2024-08-30T09:05:43.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:43.094+0000 7f91f0e89740 10 client.26027 fetch_fsmap finished waiting for FSMap version 31 2024-08-30T09:05:43.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:43.094+0000 7f91f0e89740 10 client.26027 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-08-30T09:05:43.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:43.094+0000 7f91f0e89740 20 client.26027 populate_metadata read hostname 'smithi042' 2024-08-30T09:05:43.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:43.094+0000 7f91f0e89740 1 --2- 172.21.15.42:0/848170968 >> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] conn(0x564031ec5da0 0x564032361ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:43.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:43.094+0000 7f91f0e89740 4 client.26027 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-30T09:05:43.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:43.094+0000 7f91f0e89740 1 -- 172.21.15.42:0/848170968 --> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56403224c070 con 0x564031ec5da0 2024-08-30T09:05:43.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:43.095+0000 7f91e3fff640 1 --2- 172.21.15.42:0/848170968 >> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] conn(0x564031ec5da0 0x564032361ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:05:43.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:43.096+0000 7f91e3fff640 1 --2- 172.21.15.42:0/848170968 >> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] conn(0x564031ec5da0 0x564032361ba0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=78a41d1e5a58ca4f server_cookie=5ea4cc27b01ce33e in_seq=0 out_seq=0 2024-08-30T09:05:43.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:43.096+0000 7f91c37fe640 10 client.26027 ms_handle_connect on v2:172.21.15.190:6838/719981110 2024-08-30T09:05:43.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:43.096+0000 7f91c37fe640 1 -- 172.21.15.42:0/848170968 <== mds.1 v2:172.21.15.190:6838/719981110 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56403224c070 con 0x564031ec5da0 2024-08-30T09:05:43.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:43.096+0000 7f91c37fe640 10 client.26027 handle_command_reply: tid=0 2024-08-30T09:05:43.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:43.097+0000 7f91f0e89740 1 client.26027 shutdown 2024-08-30T09:05:43.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:43.098+0000 7f91f0e89740 1 -- 172.21.15.42:0/848170968 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564031f32070 msgr2=0x5640322e8e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:05:43.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:43.098+0000 7f91f0e89740 1 --2- 172.21.15.42:0/848170968 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564031f32070 0x5640322e8e90 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f91d40ac280 tx=0x7f91d40cf070 comp rx=0 tx=0).stop 2024-08-30T09:05:43.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:43.098+0000 7f91f0e89740 1 -- 172.21.15.42:0/848170968 shutdown_connections 2024-08-30T09:05:43.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:43.098+0000 7f91f0e89740 1 --2- 172.21.15.42:0/848170968 >> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] conn(0x564031ec5da0 0x564032361ba0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:43.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:43.098+0000 7f91f0e89740 1 --2- 172.21.15.42:0/848170968 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564031f32070 0x5640322e8e90 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:43.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:43.098+0000 7f91f0e89740 1 --2- 172.21.15.42:0/848170968 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564031f11390 0x5640322e8950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:43.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:43.098+0000 7f91c37fe640 0 client.26027 ms_handle_reset on v2:172.21.15.190:6838/719981110 2024-08-30T09:05:43.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:43.098+0000 7f91f0e89740 1 --2- 172.21.15.42:0/848170968 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564031f091b0 0x5640322eb480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:43.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:43.098+0000 7f91f0e89740 1 -- 172.21.15.42:0/848170968 >> 172.21.15.42:0/848170968 conn(0x564031f26460 msgr2=0x5640322dabe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:05:43.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:43.099+0000 7f91f0e89740 1 -- 172.21.15.42:0/848170968 shutdown_connections 2024-08-30T09:05:43.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:43.099+0000 7f91f0e89740 1 -- 172.21.15.42:0/848170968 wait complete. 2024-08-30T09:05:43.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:43.099+0000 7f91f0e89740 20 client.26027 trim_cache size 0 max 16384 2024-08-30T09:05:43.102 INFO:teuthology.orchestra.run.smithi042.stdout:[] 2024-08-30T09:05:43.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:43.101+0000 7f91ea914640 1 -- 172.21.15.42:0/4053956055 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f91d00b4c60 msgr2=0x7f91d00b7170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:05:43.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:43.101+0000 7f91ea914640 1 --2- 172.21.15.42:0/4053956055 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f91d00b4c60 0x7f91d00b7170 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f91cc00bfd0 tx=0x7f91cc004750 comp rx=0 tx=0).stop 2024-08-30T09:05:43.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:43.101+0000 7f91ea914640 1 -- 172.21.15.42:0/4053956055 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f91e410fef0 msgr2=0x7f91e4110300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:05:43.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:43.101+0000 7f91ea914640 1 --2- 172.21.15.42:0/4053956055 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f91e410fef0 0x7f91e4110300 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f91d400be20 tx=0x7f91d400bf10 comp rx=0 tx=0).stop 2024-08-30T09:05:43.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:43.101+0000 7f91ea914640 1 -- 172.21.15.42:0/4053956055 shutdown_connections 2024-08-30T09:05:43.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:43.101+0000 7f91ea914640 1 --2- 172.21.15.42:0/4053956055 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f91d00b4c60 0x7f91d00b7170 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:43.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:43.101+0000 7f91ea914640 1 --2- 172.21.15.42:0/4053956055 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f91e41079c0 0x7f91e410f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:43.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:43.101+0000 7f91ea914640 1 --2- 172.21.15.42:0/4053956055 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f91e410fef0 0x7f91e4110300 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:43.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:43.101+0000 7f91ea914640 1 --2- 172.21.15.42:0/4053956055 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f91e4102ed0 0x7f91e41167a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:43.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:43.101+0000 7f91ea914640 1 -- 172.21.15.42:0/4053956055 >> 172.21.15.42:0/4053956055 conn(0x7f91e4079580 msgr2=0x7f91e41053d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:05:43.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:43.102+0000 7f91ea914640 1 -- 172.21.15.42:0/4053956055 shutdown_connections 2024-08-30T09:05:43.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:43.102+0000 7f91ea914640 1 -- 172.21.15.42:0/4053956055 wait complete. 2024-08-30T09:05:43.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:05:42 smithi190 ceph-mon[22001]: pgmap v521: 129 pgs: 129 active+clean; 1.2 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 4.8 MiB/s wr, 537 op/s 2024-08-30T09:05:43.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:05:42 smithi167 ceph-mon[22063]: pgmap v521: 129 pgs: 129 active+clean; 1.2 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 4.8 MiB/s wr, 537 op/s 2024-08-30T09:05:43.824 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-08-30T09:05:44.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:05:43 smithi042 ceph-mon[18652]: pgmap v522: 129 pgs: 129 active+clean; 1.2 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 7.1 MiB/s wr, 636 op/s 2024-08-30T09:05:44.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:05:43 smithi190 ceph-mon[22001]: pgmap v522: 129 pgs: 129 active+clean; 1.2 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 7.1 MiB/s wr, 636 op/s 2024-08-30T09:05:44.305 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T09:05:44.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:05:43 smithi167 ceph-mon[22063]: pgmap v522: 129 pgs: 129 active+clean; 1.2 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 7.1 MiB/s wr, 636 op/s 2024-08-30T09:05:45.172 INFO:teuthology.orchestra.run.smithi042.stdout:[] 2024-08-30T09:05:45.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.869+0000 7f0588a05640 1 -- 172.21.15.42:0/4105700620 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0584114510 msgr2=0x7f0584116930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:05:45.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.869+0000 7f0588a05640 1 --2- 172.21.15.42:0/4105700620 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0584114510 0x7f0584116930 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f0574009900 tx=0x7f057402a410 comp rx=0 tx=0).stop 2024-08-30T09:05:45.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.869+0000 7f0588a05640 1 -- 172.21.15.42:0/4105700620 shutdown_connections 2024-08-30T09:05:45.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.869+0000 7f0588a05640 1 --2- 172.21.15.42:0/4105700620 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0584078030 0x7f0584113fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:45.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.869+0000 7f0588a05640 1 --2- 172.21.15.42:0/4105700620 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0584114510 0x7f0584116930 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:45.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.869+0000 7f0588a05640 1 --2- 172.21.15.42:0/4105700620 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f05840776e0 0x7f0584077af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:45.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.869+0000 7f0588a05640 1 -- 172.21.15.42:0/4105700620 >> 172.21.15.42:0/4105700620 conn(0x7f0584101560 msgr2=0x7f05841039b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:05:45.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.870+0000 7f0588a05640 1 -- 172.21.15.42:0/4105700620 shutdown_connections 2024-08-30T09:05:45.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.870+0000 7f0588a05640 1 -- 172.21.15.42:0/4105700620 wait complete. 2024-08-30T09:05:45.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.871+0000 7f0588a05640 1 Processor -- start 2024-08-30T09:05:45.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.871+0000 7f0588a05640 1 -- start start 2024-08-30T09:05:45.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.871+0000 7f0588a05640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f05840776e0 0x7f0584113cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:45.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.871+0000 7f0588a05640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0584078030 0x7f058410cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:45.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.871+0000 7f0588a05640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0584114510 0x7f058410d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:45.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.871+0000 7f0588a05640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f0584119ec0 con 0x7f0584078030 2024-08-30T09:05:45.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.871+0000 7f0588a05640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f0584119d40 con 0x7f0584114510 2024-08-30T09:05:45.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.871+0000 7f0588a05640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f058411a040 con 0x7f05840776e0 2024-08-30T09:05:45.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.871+0000 7f0581d9b640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0584078030 0x7f058410cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:05:45.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.871+0000 7f0581d9b640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0584078030 0x7f058410cf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35700/0 (socket says 172.21.15.42:35700) 2024-08-30T09:05:45.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.871+0000 7f0581d9b640 1 -- 172.21.15.42:0/3220772662 learned_addr learned my addr 172.21.15.42:0/3220772662 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:05:45.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.871+0000 7f0582d9d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0584114510 0x7f058410d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:05:45.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.871+0000 7f058259c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f05840776e0 0x7f0584113cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:05:45.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.872+0000 7f058259c640 1 -- 172.21.15.42:0/3220772662 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0584114510 msgr2=0x7f058410d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:05:45.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.872+0000 7f058259c640 1 --2- 172.21.15.42:0/3220772662 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0584114510 0x7f058410d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:45.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.872+0000 7f058259c640 1 -- 172.21.15.42:0/3220772662 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0584078030 msgr2=0x7f058410cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:05:45.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.872+0000 7f058259c640 1 --2- 172.21.15.42:0/3220772662 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0584078030 0x7f058410cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:45.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.872+0000 7f058259c640 1 -- 172.21.15.42:0/3220772662 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f058410dc90 con 0x7f05840776e0 2024-08-30T09:05:45.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.872+0000 7f058259c640 1 --2- 172.21.15.42:0/3220772662 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f05840776e0 0x7f0584113cb0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f057c00ec80 tx=0x7f057c00c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:05:45.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.872+0000 7f05737fe640 1 -- 172.21.15.42:0/3220772662 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f057c003b70 con 0x7f05840776e0 2024-08-30T09:05:45.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.873+0000 7f05737fe640 1 -- 172.21.15.42:0/3220772662 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f057c003d10 con 0x7f05840776e0 2024-08-30T09:05:45.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.873+0000 7f05737fe640 1 -- 172.21.15.42:0/3220772662 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f057c010910 con 0x7f05840776e0 2024-08-30T09:05:45.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.873+0000 7f0588a05640 1 -- 172.21.15.42:0/3220772662 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0584112920 con 0x7f05840776e0 2024-08-30T09:05:45.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.873+0000 7f0588a05640 1 -- 172.21.15.42:0/3220772662 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0584112e10 con 0x7f05840776e0 2024-08-30T09:05:45.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.876+0000 7f05737fe640 1 -- 172.21.15.42:0/3220772662 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7f057c010ab0 con 0x7f05840776e0 2024-08-30T09:05:45.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.877+0000 7f058ef7a740 1 Processor -- start 2024-08-30T09:05:45.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.877+0000 7f058ef7a740 1 -- start start 2024-08-30T09:05:45.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.878+0000 7f058ef7a740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562de712d070 0x562de74d2310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:45.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.878+0000 7f058ef7a740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562de71041b0 0x562de74d2850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:45.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.878+0000 7f058ef7a740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562de710c390 0x562de74d2d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:45.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.878+0000 7f058ef7a740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x562de7449680 con 0x562de71041b0 2024-08-30T09:05:45.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.878+0000 7f058ef7a740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x562de7438c10 con 0x562de710c390 2024-08-30T09:05:45.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.878+0000 7f058ef7a740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x562de743cbc0 con 0x562de712d070 2024-08-30T09:05:45.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.878+0000 7f058259c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562de71041b0 0x562de74d2850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:05:45.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.878+0000 7f058259c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562de71041b0 0x562de74d2850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35714/0 (socket says 172.21.15.42:35714) 2024-08-30T09:05:45.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.878+0000 7f058259c640 1 -- 172.21.15.42:0/2692673125 learned_addr learned my addr 172.21.15.42:0/2692673125 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:05:45.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.878+0000 7f0581d9b640 1 --2- 172.21.15.42:0/2692673125 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562de710c390 0x562de74d2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:05:45.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.878+0000 7f0582d9d640 1 --2- 172.21.15.42:0/2692673125 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562de712d070 0x562de74d2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:05:45.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.879+0000 7f058259c640 1 -- 172.21.15.42:0/2692673125 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562de712d070 msgr2=0x562de74d2310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:05:45.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.879+0000 7f058259c640 1 --2- 172.21.15.42:0/2692673125 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562de712d070 0x562de74d2310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:45.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.879+0000 7f058259c640 1 -- 172.21.15.42:0/2692673125 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562de710c390 msgr2=0x562de74d2d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:05:45.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.879+0000 7f058259c640 1 --2- 172.21.15.42:0/2692673125 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562de710c390 0x562de74d2d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:45.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.879+0000 7f05737fe640 1 --2- 172.21.15.42:0/3220772662 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f056c0b4c30 0x7f056c0b7140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:45.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.879+0000 7f058259c640 1 -- 172.21.15.42:0/2692673125 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562de7198650 con 0x562de71041b0 2024-08-30T09:05:45.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.879+0000 7f0582d9d640 1 --2- 172.21.15.42:0/3220772662 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f056c0b4c30 0x7f056c0b7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:05:45.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.880+0000 7f05737fe640 1 -- 172.21.15.42:0/3220772662 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f057c0e5bf0 con 0x7f05840776e0 2024-08-30T09:05:45.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.880+0000 7f0582d9d640 1 --2- 172.21.15.42:0/3220772662 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f056c0b4c30 0x7f056c0b7140 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f0574006010 tx=0x7f0574005a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:05:45.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.880+0000 7f058259c640 1 --2- 172.21.15.42:0/2692673125 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562de71041b0 0x562de74d2850 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f057c0af9f0 tx=0x7f057c0a8b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=581cc942b350d759 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:05:45.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.882+0000 7f0570ff9640 1 -- 172.21.15.42:0/2692673125 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f057c0ed040 con 0x562de71041b0 2024-08-30T09:05:45.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.882+0000 7f0570ff9640 1 -- 172.21.15.42:0/2692673125 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f057c0ac930 con 0x562de71041b0 2024-08-30T09:05:45.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.882+0000 7f0570ff9640 1 -- 172.21.15.42:0/2692673125 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f057c0afac0 con 0x562de71041b0 2024-08-30T09:05:45.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.883+0000 7f058ef7a740 1 -- 172.21.15.42:0/2692673125 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562de71041b0 msgr2=0x562de74d2850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:05:45.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.883+0000 7f058ef7a740 1 --2- 172.21.15.42:0/2692673125 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562de71041b0 0x562de74d2850 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f057c0af9f0 tx=0x7f057c0a8b00 comp rx=0 tx=0).stop 2024-08-30T09:05:45.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.884+0000 7f058ef7a740 1 -- 172.21.15.42:0/2692673125 shutdown_connections 2024-08-30T09:05:45.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.884+0000 7f058ef7a740 1 --2- 172.21.15.42:0/2692673125 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562de710c390 0x562de74d2d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:45.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.884+0000 7f058ef7a740 1 --2- 172.21.15.42:0/2692673125 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562de71041b0 0x562de74d2850 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:45.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.884+0000 7f058ef7a740 1 --2- 172.21.15.42:0/2692673125 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562de712d070 0x562de74d2310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:45.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.884+0000 7f058ef7a740 1 -- 172.21.15.42:0/2692673125 >> 172.21.15.42:0/2692673125 conn(0x562de7121460 msgr2=0x562de74d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:05:45.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.884+0000 7f058ef7a740 1 -- 172.21.15.42:0/2692673125 shutdown_connections 2024-08-30T09:05:45.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.884+0000 7f058ef7a740 1 -- 172.21.15.42:0/2692673125 wait complete. 2024-08-30T09:05:45.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.885+0000 7f058ef7a740 1 -- start start 2024-08-30T09:05:45.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.885+0000 7f058ef7a740 1 Processor -- start 2024-08-30T09:05:45.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.885+0000 7f058ef7a740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562de71041b0 0x562de74dda70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:45.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.885+0000 7f058ef7a740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562de710c390 0x562de74ddfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:45.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.885+0000 7f058ef7a740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562de712d070 0x562de74e9e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:45.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.885+0000 7f058ef7a740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x562de729b700 con 0x562de71041b0 2024-08-30T09:05:45.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.885+0000 7f058ef7a740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x562de73bf750 con 0x562de710c390 2024-08-30T09:05:45.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.885+0000 7f058ef7a740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x562de74e0af0 con 0x562de712d070 2024-08-30T09:05:45.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.885+0000 7f0581d9b640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562de710c390 0x562de74ddfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:05:45.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.885+0000 7f0581d9b640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562de710c390 0x562de74ddfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:48392/0 (socket says 172.21.15.42:48392) 2024-08-30T09:05:45.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.885+0000 7f058259c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562de71041b0 0x562de74dda70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:05:45.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.885+0000 7f0581d9b640 1 -- 172.21.15.42:0/1868038987 learned_addr learned my addr 172.21.15.42:0/1868038987 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:05:45.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.885+0000 7f0582d9d640 1 --2- 172.21.15.42:0/1868038987 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562de712d070 0x562de74e9e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:05:45.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.885+0000 7f058ef7a740 10 client.0 _ll_register_callbacks cb 0x562de73eee10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:05:45.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.886+0000 7f0581d9b640 1 -- 172.21.15.42:0/1868038987 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562de712d070 msgr2=0x562de74e9e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:05:45.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.886+0000 7f0581d9b640 1 --2- 172.21.15.42:0/1868038987 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562de712d070 0x562de74e9e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:45.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.886+0000 7f0581d9b640 1 -- 172.21.15.42:0/1868038987 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562de71041b0 msgr2=0x562de74dda70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:05:45.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.886+0000 7f0581d9b640 1 --2- 172.21.15.42:0/1868038987 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562de71041b0 0x562de74dda70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:45.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.886+0000 7f0581d9b640 1 -- 172.21.15.42:0/1868038987 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562de73f8900 con 0x562de710c390 2024-08-30T09:05:45.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.886+0000 7f0581d9b640 1 --2- 172.21.15.42:0/1868038987 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562de710c390 0x562de74ddfb0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f0578012680 tx=0x7f0578012b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1c3779600b259458 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:05:45.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.886+0000 7f0569ffb640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-30T09:05:45.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.886+0000 7f0569ffb640 1 -- 172.21.15.42:0/1868038987 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0578011bb0 con 0x562de710c390 2024-08-30T09:05:45.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.886+0000 7f0569ffb640 1 -- 172.21.15.42:0/1868038987 <== mon.2 v2:172.21.15.167:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f0578011d50 con 0x562de710c390 2024-08-30T09:05:45.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.886+0000 7f058ef7a740 1 -- 172.21.15.42:0/1868038987 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f057c0ac930 con 0x562de710c390 2024-08-30T09:05:45.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.887+0000 7f0569ffb640 1 -- 172.21.15.42:0/1868038987 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f057800f460 con 0x562de710c390 2024-08-30T09:05:45.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.887+0000 7f058ef7a740 10 client.26039 fetch_fsmap learned FSMap version 31 2024-08-30T09:05:45.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.887+0000 7f058ef7a740 1 -- 172.21.15.42:0/1868038987 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x562de74ec930 con 0x562de710c390 2024-08-30T09:05:45.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.887+0000 7f0569ffb640 1 -- 172.21.15.42:0/1868038987 <== mon.2 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f057c0ac930 con 0x562de710c390 2024-08-30T09:05:45.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.887+0000 7f0569ffb640 1 -- 172.21.15.42:0/1868038987 <== mon.2 v2:172.21.15.167:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f0578026710 con 0x562de710c390 2024-08-30T09:05:45.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.887+0000 7f058ef7a740 10 client.26039 fetch_fsmap finished waiting for FSMap version 31 2024-08-30T09:05:45.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.887+0000 7f058ef7a740 10 client.26039 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-30T09:05:45.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.887+0000 7f058ef7a740 20 client.26039 populate_metadata read hostname 'smithi042' 2024-08-30T09:05:45.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.887+0000 7f058ef7a740 1 --2- 172.21.15.42:0/1868038987 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x562de70eb9f0 0x562de74efec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:45.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.887+0000 7f058ef7a740 4 client.26039 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-30T09:05:45.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.887+0000 7f058ef7a740 1 -- 172.21.15.42:0/1868038987 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562de744b5d0 con 0x562de70eb9f0 2024-08-30T09:05:45.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.888+0000 7f058259c640 1 --2- 172.21.15.42:0/1868038987 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x562de70eb9f0 0x562de74efec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:05:45.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.888+0000 7f058259c640 1 --2- 172.21.15.42:0/1868038987 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x562de70eb9f0 0x562de74efec0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b0cb03ddfdac7ab5 server_cookie=379e8827b3808fea in_seq=0 out_seq=0 2024-08-30T09:05:45.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.888+0000 7f0569ffb640 10 client.26039 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:05:45.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.889+0000 7f0569ffb640 1 -- 172.21.15.42:0/1868038987 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x562de744b5d0 con 0x562de70eb9f0 2024-08-30T09:05:45.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.889+0000 7f0569ffb640 10 client.26039 handle_command_reply: tid=0 2024-08-30T09:05:45.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.889+0000 7f058ef7a740 1 client.26039 shutdown 2024-08-30T09:05:45.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.889+0000 7f058ef7a740 1 -- 172.21.15.42:0/1868038987 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562de710c390 msgr2=0x562de74ddfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:05:45.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.889+0000 7f058ef7a740 1 --2- 172.21.15.42:0/1868038987 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562de710c390 0x562de74ddfb0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f0578012680 tx=0x7f0578012b40 comp rx=0 tx=0).stop 2024-08-30T09:05:45.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.889+0000 7f058ef7a740 1 -- 172.21.15.42:0/1868038987 shutdown_connections 2024-08-30T09:05:45.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.889+0000 7f058ef7a740 1 --2- 172.21.15.42:0/1868038987 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x562de70eb9f0 0x562de74efec0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:45.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.889+0000 7f058ef7a740 1 --2- 172.21.15.42:0/1868038987 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562de712d070 0x562de74e9e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:45.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.889+0000 7f058ef7a740 1 --2- 172.21.15.42:0/1868038987 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562de710c390 0x562de74ddfb0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:45.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.889+0000 7f058ef7a740 1 --2- 172.21.15.42:0/1868038987 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562de71041b0 0x562de74dda70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:45.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.889+0000 7f058ef7a740 1 -- 172.21.15.42:0/1868038987 >> 172.21.15.42:0/1868038987 conn(0x562de7121460 msgr2=0x562de74d54a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:05:45.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.889+0000 7f0569ffb640 0 client.26039 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:05:45.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.889+0000 7f058ef7a740 1 -- 172.21.15.42:0/1868038987 shutdown_connections 2024-08-30T09:05:45.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.889+0000 7f058ef7a740 1 -- 172.21.15.42:0/1868038987 wait complete. 2024-08-30T09:05:45.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.890+0000 7f058ef7a740 20 client.26039 trim_cache size 0 max 16384 2024-08-30T09:05:45.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.907+0000 7f058ef7a740 1 Processor -- start 2024-08-30T09:05:45.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.907+0000 7f058ef7a740 1 -- start start 2024-08-30T09:05:45.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.907+0000 7f058ef7a740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562de71041b0 0x562de74cb230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:45.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.908+0000 7f058ef7a740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562de710c390 0x562de74de050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:45.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.908+0000 7f058ef7a740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562de712d070 0x562de74d5410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:45.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.908+0000 7f058ef7a740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f0564012470 con 0x562de710c390 2024-08-30T09:05:45.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.908+0000 7f058ef7a740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f056400fd40 con 0x562de712d070 2024-08-30T09:05:45.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.908+0000 7f058ef7a740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f056400db90 con 0x562de71041b0 2024-08-30T09:05:45.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.908+0000 7f0581d9b640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562de710c390 0x562de74de050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:05:45.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.908+0000 7f0582d9d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562de712d070 0x562de74d5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:05:45.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.908+0000 7f0581d9b640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562de710c390 0x562de74de050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35732/0 (socket says 172.21.15.42:35732) 2024-08-30T09:05:45.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.908+0000 7f0581d9b640 1 -- 172.21.15.42:0/191051459 learned_addr learned my addr 172.21.15.42:0/191051459 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:05:45.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.908+0000 7f058259c640 1 --2- 172.21.15.42:0/191051459 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562de71041b0 0x562de74cb230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:05:45.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.908+0000 7f0582d9d640 1 -- 172.21.15.42:0/191051459 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562de71041b0 msgr2=0x562de74cb230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:05:45.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.908+0000 7f0582d9d640 1 --2- 172.21.15.42:0/191051459 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562de71041b0 0x562de74cb230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:45.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.908+0000 7f0582d9d640 1 -- 172.21.15.42:0/191051459 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562de710c390 msgr2=0x562de74de050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:05:45.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.908+0000 7f0582d9d640 1 --2- 172.21.15.42:0/191051459 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562de710c390 0x562de74de050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:45.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.908+0000 7f0582d9d640 1 -- 172.21.15.42:0/191051459 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562de74e01b0 con 0x562de712d070 2024-08-30T09:05:45.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.908+0000 7f0582d9d640 1 --2- 172.21.15.42:0/191051459 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562de712d070 0x562de74d5410 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f057402a930 tx=0x7f0574040000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=be8fb89fa1080ee9 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:05:45.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.909+0000 7f056bfff640 1 -- 172.21.15.42:0/191051459 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0574035040 con 0x562de712d070 2024-08-30T09:05:45.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.909+0000 7f056bfff640 1 -- 172.21.15.42:0/191051459 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f05740314a0 con 0x562de712d070 2024-08-30T09:05:45.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.909+0000 7f056bfff640 1 -- 172.21.15.42:0/191051459 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05740526c0 con 0x562de712d070 2024-08-30T09:05:45.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.910+0000 7f058ef7a740 1 -- 172.21.15.42:0/191051459 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562de712d070 msgr2=0x562de74d5410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:05:45.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.910+0000 7f058ef7a740 1 --2- 172.21.15.42:0/191051459 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562de712d070 0x562de74d5410 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f057402a930 tx=0x7f0574040000 comp rx=0 tx=0).stop 2024-08-30T09:05:45.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.910+0000 7f058ef7a740 1 -- 172.21.15.42:0/191051459 shutdown_connections 2024-08-30T09:05:45.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.910+0000 7f058ef7a740 1 --2- 172.21.15.42:0/191051459 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562de712d070 0x562de74d5410 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:45.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.910+0000 7f058ef7a740 1 --2- 172.21.15.42:0/191051459 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562de710c390 0x562de74de050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:45.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.910+0000 7f058ef7a740 1 --2- 172.21.15.42:0/191051459 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562de71041b0 0x562de74cb230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:45.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.910+0000 7f058ef7a740 1 -- 172.21.15.42:0/191051459 >> 172.21.15.42:0/191051459 conn(0x562de7121460 msgr2=0x562de74dc2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:05:45.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.910+0000 7f058ef7a740 1 -- 172.21.15.42:0/191051459 shutdown_connections 2024-08-30T09:05:45.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.910+0000 7f058ef7a740 1 -- 172.21.15.42:0/191051459 wait complete. 2024-08-30T09:05:45.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.910+0000 7f058ef7a740 1 -- start start 2024-08-30T09:05:45.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.910+0000 7f058ef7a740 1 Processor -- start 2024-08-30T09:05:45.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.911+0000 7f058ef7a740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562de71041b0 0x562de74e6590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:45.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.911+0000 7f058ef7a740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562de710c390 0x562de74e1a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:45.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.911+0000 7f058ef7a740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562de712d070 0x562de74e1f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:45.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.911+0000 7f058ef7a740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f056400e200 con 0x562de71041b0 2024-08-30T09:05:45.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.911+0000 7f058ef7a740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f056400f670 con 0x562de712d070 2024-08-30T09:05:45.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.911+0000 7f058ef7a740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f056400e8d0 con 0x562de710c390 2024-08-30T09:05:45.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.911+0000 7f058ef7a740 10 client.0 _ll_register_callbacks cb 0x7f0578009760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:05:45.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.911+0000 7f058259c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562de71041b0 0x562de74e6590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:05:45.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.911+0000 7f0581d9b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562de710c390 0x562de74e1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:05:45.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.911+0000 7f058259c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562de71041b0 0x562de74e6590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35736/0 (socket says 172.21.15.42:35736) 2024-08-30T09:05:45.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.911+0000 7f0581d9b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562de710c390 0x562de74e1a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:58856/0 (socket says 172.21.15.42:58856) 2024-08-30T09:05:45.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.911+0000 7f058259c640 1 -- 172.21.15.42:0/2620121563 learned_addr learned my addr 172.21.15.42:0/2620121563 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:05:45.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.911+0000 7f0582d9d640 1 --2- 172.21.15.42:0/2620121563 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562de712d070 0x562de74e1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:05:45.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.911+0000 7f0581d9b640 1 -- 172.21.15.42:0/2620121563 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562de712d070 msgr2=0x562de74e1f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:05:45.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.911+0000 7f0581d9b640 1 --2- 172.21.15.42:0/2620121563 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562de712d070 0x562de74e1f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:45.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.911+0000 7f0581d9b640 1 -- 172.21.15.42:0/2620121563 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562de71041b0 msgr2=0x562de74e6590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:05:45.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.911+0000 7f0581d9b640 1 --2- 172.21.15.42:0/2620121563 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562de71041b0 0x562de74e6590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:45.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.911+0000 7f0581d9b640 1 -- 172.21.15.42:0/2620121563 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562de74e26f0 con 0x562de710c390 2024-08-30T09:05:45.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.911+0000 7f0582d9d640 1 --2- 172.21.15.42:0/2620121563 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562de712d070 0x562de74e1f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-30T09:05:45.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.912+0000 7f0581d9b640 1 --2- 172.21.15.42:0/2620121563 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562de710c390 0x562de74e1a40 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x562de74d5050 tx=0x7f0578026a10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=691dd098a4609a3 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:05:45.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.912+0000 7f0569ffb640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-08-30T09:05:45.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.912+0000 7f0569ffb640 1 -- 172.21.15.42:0/2620121563 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0578009e50 con 0x562de710c390 2024-08-30T09:05:45.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.912+0000 7f0569ffb640 1 -- 172.21.15.42:0/2620121563 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f05780052d0 con 0x562de710c390 2024-08-30T09:05:45.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.912+0000 7f058ef7a740 1 -- 172.21.15.42:0/2620121563 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f05740314a0 con 0x562de710c390 2024-08-30T09:05:45.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.912+0000 7f0569ffb640 1 -- 172.21.15.42:0/2620121563 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f057801c420 con 0x562de710c390 2024-08-30T09:05:45.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.912+0000 7f058ef7a740 10 client.26035 fetch_fsmap learned FSMap version 31 2024-08-30T09:05:45.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.912+0000 7f058ef7a740 1 -- 172.21.15.42:0/2620121563 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x562de755a7a0 con 0x562de710c390 2024-08-30T09:05:45.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.912+0000 7f0569ffb640 1 -- 172.21.15.42:0/2620121563 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f05740314a0 con 0x562de710c390 2024-08-30T09:05:45.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.913+0000 7f0569ffb640 1 -- 172.21.15.42:0/2620121563 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f05780035f0 con 0x562de710c390 2024-08-30T09:05:45.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.913+0000 7f058ef7a740 10 client.26035 fetch_fsmap finished waiting for FSMap version 31 2024-08-30T09:05:45.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.913+0000 7f058ef7a740 10 client.26035 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-30T09:05:45.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.913+0000 7f058ef7a740 20 client.26035 populate_metadata read hostname 'smithi042' 2024-08-30T09:05:45.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.913+0000 7f058ef7a740 1 --2- 172.21.15.42:0/2620121563 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x562de70c0da0 0x562de755cf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:45.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.913+0000 7f058ef7a740 4 client.26035 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-30T09:05:45.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.913+0000 7f058ef7a740 1 -- 172.21.15.42:0/2620121563 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562de7447070 con 0x562de70c0da0 2024-08-30T09:05:45.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.913+0000 7f058259c640 1 --2- 172.21.15.42:0/2620121563 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x562de70c0da0 0x562de755cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:05:45.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.913+0000 7f0569ffb640 10 client.26035 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:05:45.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.914+0000 7f058259c640 1 --2- 172.21.15.42:0/2620121563 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x562de70c0da0 0x562de755cf00 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de76cb6c1872c78c server_cookie=c832ac9bccfcd54a in_seq=0 out_seq=0 2024-08-30T09:05:45.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.914+0000 7f0569ffb640 1 -- 172.21.15.42:0/2620121563 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562de7447070 con 0x562de70c0da0 2024-08-30T09:05:45.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.914+0000 7f0569ffb640 10 client.26035 handle_command_reply: tid=0 2024-08-30T09:05:45.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.914+0000 7f058ef7a740 1 client.26035 shutdown 2024-08-30T09:05:45.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.914+0000 7f058ef7a740 1 -- 172.21.15.42:0/2620121563 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562de710c390 msgr2=0x562de74e1a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:05:45.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.914+0000 7f058ef7a740 1 --2- 172.21.15.42:0/2620121563 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562de710c390 0x562de74e1a40 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x562de74d5050 tx=0x7f0578026a10 comp rx=0 tx=0).stop 2024-08-30T09:05:45.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.914+0000 7f058ef7a740 1 -- 172.21.15.42:0/2620121563 shutdown_connections 2024-08-30T09:05:45.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.914+0000 7f058ef7a740 1 --2- 172.21.15.42:0/2620121563 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x562de70c0da0 0x562de755cf00 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:45.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.914+0000 7f058ef7a740 1 --2- 172.21.15.42:0/2620121563 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562de710c390 0x562de74e1a40 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:45.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.914+0000 7f058ef7a740 1 --2- 172.21.15.42:0/2620121563 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562de712d070 0x562de74e1f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:45.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.914+0000 7f058ef7a740 1 --2- 172.21.15.42:0/2620121563 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562de71041b0 0x562de74e6590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:45.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.914+0000 7f058ef7a740 1 -- 172.21.15.42:0/2620121563 >> 172.21.15.42:0/2620121563 conn(0x562de7121460 msgr2=0x562de74d5410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:05:45.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.914+0000 7f0569ffb640 0 client.26035 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:05:45.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.914+0000 7f058ef7a740 1 -- 172.21.15.42:0/2620121563 shutdown_connections 2024-08-30T09:05:45.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.914+0000 7f058ef7a740 1 -- 172.21.15.42:0/2620121563 wait complete. 2024-08-30T09:05:45.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.915+0000 7f058ef7a740 20 client.26035 trim_cache size 0 max 16384 2024-08-30T09:05:45.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.916+0000 7f0588a05640 1 -- 172.21.15.42:0/3220772662 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f056c0b4c30 msgr2=0x7f056c0b7140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:05:45.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.916+0000 7f0588a05640 1 --2- 172.21.15.42:0/3220772662 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f056c0b4c30 0x7f056c0b7140 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f0574006010 tx=0x7f0574005a40 comp rx=0 tx=0).stop 2024-08-30T09:05:45.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.916+0000 7f0588a05640 1 -- 172.21.15.42:0/3220772662 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f05840776e0 msgr2=0x7f0584113cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:05:45.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.916+0000 7f0588a05640 1 --2- 172.21.15.42:0/3220772662 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f05840776e0 0x7f0584113cb0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f057c00ec80 tx=0x7f057c00c6a0 comp rx=0 tx=0).stop 2024-08-30T09:05:45.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.916+0000 7f0588a05640 1 -- 172.21.15.42:0/3220772662 shutdown_connections 2024-08-30T09:05:45.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.916+0000 7f0588a05640 1 --2- 172.21.15.42:0/3220772662 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f056c0b4c30 0x7f056c0b7140 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:45.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.916+0000 7f0588a05640 1 --2- 172.21.15.42:0/3220772662 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0584114510 0x7f058410d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:45.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.916+0000 7f0588a05640 1 --2- 172.21.15.42:0/3220772662 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0584078030 0x7f058410cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:45.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.916+0000 7f0588a05640 1 --2- 172.21.15.42:0/3220772662 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f05840776e0 0x7f0584113cb0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:45.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.916+0000 7f0588a05640 1 -- 172.21.15.42:0/3220772662 >> 172.21.15.42:0/3220772662 conn(0x7f0584101560 msgr2=0x7f0584103980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:05:45.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.916+0000 7f0588a05640 1 -- 172.21.15.42:0/3220772662 shutdown_connections 2024-08-30T09:05:45.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:44.916+0000 7f0588a05640 1 -- 172.21.15.42:0/3220772662 wait complete. 2024-08-30T09:05:45.953 INFO:teuthology.orchestra.run.smithi042.stderr:progress: 300.0 s, 487.2 tps, lat 191.763 ms stddev 229.906 2024-08-30T09:05:46.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:05:46 smithi042 ceph-mon[18652]: pgmap v523: 129 pgs: 129 active+clean; 1.2 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 6.9 MiB/s wr, 545 op/s 2024-08-30T09:05:46.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:05:46 smithi190 ceph-mon[22001]: pgmap v523: 129 pgs: 129 active+clean; 1.2 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 6.9 MiB/s wr, 545 op/s 2024-08-30T09:05:46.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:05:46 smithi167 ceph-mon[22063]: pgmap v523: 129 pgs: 129 active+clean; 1.2 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 6.9 MiB/s wr, 545 op/s 2024-08-30T09:05:47.123 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-08-30T09:05:47.123 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-08-30T09:05:47.588 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T09:05:48.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:05:48 smithi190 ceph-mon[22001]: pgmap v524: 129 pgs: 129 active+clean; 1.2 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 8.6 MiB/s wr, 556 op/s 2024-08-30T09:05:48.445 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.443+0000 7f689a0ca640 1 -- 172.21.15.42:0/3251759934 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6894107bf0 msgr2=0x7f689410e420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:05:48.445 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.443+0000 7f689a0ca640 1 --2- 172.21.15.42:0/3251759934 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6894107bf0 0x7f689410e420 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f688c009ca0 tx=0x7f688c02a8b0 comp rx=0 tx=0).stop 2024-08-30T09:05:48.445 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:05:48 smithi042 ceph-mon[18652]: pgmap v524: 129 pgs: 129 active+clean; 1.2 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 8.6 MiB/s wr, 556 op/s 2024-08-30T09:05:48.446 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.443+0000 7f689a0ca640 1 -- 172.21.15.42:0/3251759934 shutdown_connections 2024-08-30T09:05:48.446 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.443+0000 7f689a0ca640 1 --2- 172.21.15.42:0/3251759934 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6894107bf0 0x7f689410e420 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:48.446 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.443+0000 7f689a0ca640 1 --2- 172.21.15.42:0/3251759934 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f68941072a0 0x7f68941076b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:48.446 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.443+0000 7f689a0ca640 1 --2- 172.21.15.42:0/3251759934 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6894101960 0x7f6894101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:48.446 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.443+0000 7f689a0ca640 1 -- 172.21.15.42:0/3251759934 >> 172.21.15.42:0/3251759934 conn(0x7f6894078420 msgr2=0x7f6894078860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:05:48.447 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.445+0000 7f689a0ca640 1 -- 172.21.15.42:0/3251759934 shutdown_connections 2024-08-30T09:05:48.447 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.445+0000 7f689a0ca640 1 -- 172.21.15.42:0/3251759934 wait complete. 2024-08-30T09:05:48.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.446+0000 7f689a0ca640 1 Processor -- start 2024-08-30T09:05:48.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.446+0000 7f689a0ca640 1 -- start start 2024-08-30T09:05:48.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.447+0000 7f689a0ca640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6894101960 0x7f6894197c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:48.450 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.447+0000 7f689a0ca640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f68941072a0 0x7f68941981c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:48.450 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.447+0000 7f689a0ca640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6894107bf0 0x7f689419d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:48.450 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.447+0000 7f68990c8640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6894101960 0x7f6894197c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:05:48.450 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.447+0000 7f68990c8640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6894101960 0x7f6894197c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60520/0 (socket says 172.21.15.42:60520) 2024-08-30T09:05:48.450 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.447+0000 7f68990c8640 1 -- 172.21.15.42:0/1942008346 learned_addr learned my addr 172.21.15.42:0/1942008346 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:05:48.450 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.447+0000 7f689a0ca640 1 -- 172.21.15.42:0/1942008346 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f6894111450 con 0x7f6894101960 2024-08-30T09:05:48.450 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.448+0000 7f689a0ca640 1 -- 172.21.15.42:0/1942008346 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f68941112d0 con 0x7f68941072a0 2024-08-30T09:05:48.450 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.448+0000 7f689a0ca640 1 -- 172.21.15.42:0/1942008346 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f68941115d0 con 0x7f6894107bf0 2024-08-30T09:05:48.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.448+0000 7f68988c7640 1 --2- 172.21.15.42:0/1942008346 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f68941072a0 0x7f68941981c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:05:48.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.448+0000 7f68990c8640 1 -- 172.21.15.42:0/1942008346 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6894107bf0 msgr2=0x7f689419d460 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-08-30T09:05:48.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.448+0000 7f68990c8640 1 --2- 172.21.15.42:0/1942008346 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6894107bf0 0x7f689419d460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:48.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.448+0000 7f68990c8640 1 -- 172.21.15.42:0/1942008346 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f68941072a0 msgr2=0x7f68941981c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:05:48.452 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.448+0000 7f68990c8640 1 --2- 172.21.15.42:0/1942008346 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f68941072a0 0x7f68941981c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:48.452 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.448+0000 7f68990c8640 1 -- 172.21.15.42:0/1942008346 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f689419db90 con 0x7f6894101960 2024-08-30T09:05:48.452 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.448+0000 7f68988c7640 1 --2- 172.21.15.42:0/1942008346 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f68941072a0 0x7f68941981c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-30T09:05:48.452 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.449+0000 7f68990c8640 1 --2- 172.21.15.42:0/1942008346 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6894101960 0x7f6894197c20 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f689000d950 tx=0x7f689000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:05:48.452 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.449+0000 7f688a7fc640 1 -- 172.21.15.42:0/1942008346 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6890017070 con 0x7f6894101960 2024-08-30T09:05:48.452 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.449+0000 7f688a7fc640 1 -- 172.21.15.42:0/1942008346 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6890009c30 con 0x7f6894101960 2024-08-30T09:05:48.452 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.449+0000 7f688a7fc640 1 -- 172.21.15.42:0/1942008346 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f689000f620 con 0x7f6894101960 2024-08-30T09:05:48.452 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.449+0000 7f689a0ca640 1 -- 172.21.15.42:0/1942008346 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f689419de80 con 0x7f6894101960 2024-08-30T09:05:48.452 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.450+0000 7f689a0ca640 1 -- 172.21.15.42:0/1942008346 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68941a45e0 con 0x7f6894101960 2024-08-30T09:05:48.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.455+0000 7f68a18a1740 1 Processor -- start 2024-08-30T09:05:48.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.455+0000 7f68a18a1740 1 -- start start 2024-08-30T09:05:48.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.455+0000 7f68a18a1740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56285c022350 0x56285c3d1170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:48.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.455+0000 7f68a18a1740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56285bff9490 0x56285c3d16b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:48.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.455+0000 7f68a18a1740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56285c001670 0x56285c3d1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:48.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.455+0000 7f68a18a1740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x56285c33e5f0 con 0x56285bff9490 2024-08-30T09:05:48.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.455+0000 7f68a18a1740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x56285c32db80 con 0x56285c022350 2024-08-30T09:05:48.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.455+0000 7f68a18a1740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x56285c331b30 con 0x56285c001670 2024-08-30T09:05:48.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.455+0000 7f68998c9640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56285c022350 0x56285c3d1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:05:48.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.455+0000 7f68998c9640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56285c022350 0x56285c3d1170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:40294/0 (socket says 172.21.15.42:40294) 2024-08-30T09:05:48.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.455+0000 7f68998c9640 1 -- 172.21.15.42:0/619313483 learned_addr learned my addr 172.21.15.42:0/619313483 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:05:48.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.455+0000 7f68988c7640 1 --2- 172.21.15.42:0/619313483 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56285c001670 0x56285c3d1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:05:48.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.456+0000 7f688a7fc640 1 -- 172.21.15.42:0/1942008346 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7f6890010460 con 0x7f6894101960 2024-08-30T09:05:48.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.456+0000 7f68990c8640 1 --2- 172.21.15.42:0/619313483 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56285bff9490 0x56285c3d16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:05:48.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.458+0000 7f68988c7640 1 -- 172.21.15.42:0/619313483 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56285c022350 msgr2=0x56285c3d1170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:05:48.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.458+0000 7f68988c7640 1 --2- 172.21.15.42:0/619313483 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56285c022350 0x56285c3d1170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:48.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.458+0000 7f68988c7640 1 -- 172.21.15.42:0/619313483 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56285bff9490 msgr2=0x56285c3d16b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:05:48.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.458+0000 7f68988c7640 1 --2- 172.21.15.42:0/619313483 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56285bff9490 0x56285c3d16b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:48.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.458+0000 7f68988c7640 1 -- 172.21.15.42:0/619313483 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56285c08d930 con 0x56285c001670 2024-08-30T09:05:48.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.459+0000 7f68990c8640 1 --2- 172.21.15.42:0/619313483 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56285bff9490 0x56285c3d16b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-30T09:05:48.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.459+0000 7f68988c7640 1 --2- 172.21.15.42:0/619313483 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56285c001670 0x56285c3d1bf0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f689406c7c0 tx=0x7f6884010740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4d2551b9659ce1da server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:05:48.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.459+0000 7f687f7fe640 1 -- 172.21.15.42:0/619313483 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6884004030 con 0x56285c001670 2024-08-30T09:05:48.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.460+0000 7f688a7fc640 1 --2- 172.21.15.42:0/1942008346 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f68800b4c00 0x7f68800b7110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:48.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.460+0000 7f687f7fe640 1 -- 172.21.15.42:0/619313483 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f68840035f0 con 0x56285c001670 2024-08-30T09:05:48.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.460+0000 7f687f7fe640 1 -- 172.21.15.42:0/619313483 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68840126b0 con 0x56285c001670 2024-08-30T09:05:48.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.460+0000 7f688a7fc640 1 -- 172.21.15.42:0/1942008346 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f68900e5110 con 0x7f6894101960 2024-08-30T09:05:48.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.460+0000 7f68998c9640 1 --2- 172.21.15.42:0/1942008346 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f68800b4c00 0x7f68800b7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:05:48.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.461+0000 7f68a18a1740 1 -- 172.21.15.42:0/619313483 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56285c001670 msgr2=0x56285c3d1bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:05:48.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.461+0000 7f68a18a1740 1 --2- 172.21.15.42:0/619313483 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56285c001670 0x56285c3d1bf0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f689406c7c0 tx=0x7f6884010740 comp rx=0 tx=0).stop 2024-08-30T09:05:48.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.461+0000 7f68998c9640 1 --2- 172.21.15.42:0/1942008346 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f68800b4c00 0x7f68800b7110 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f688c031040 tx=0x7f688c005d80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:05:48.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.461+0000 7f68a18a1740 1 -- 172.21.15.42:0/619313483 shutdown_connections 2024-08-30T09:05:48.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.461+0000 7f68a18a1740 1 --2- 172.21.15.42:0/619313483 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56285c001670 0x56285c3d1bf0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:48.465 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.461+0000 7f68a18a1740 1 --2- 172.21.15.42:0/619313483 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56285bff9490 0x56285c3d16b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:48.465 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.461+0000 7f68a18a1740 1 --2- 172.21.15.42:0/619313483 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56285c022350 0x56285c3d1170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:48.465 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.462+0000 7f68a18a1740 1 -- 172.21.15.42:0/619313483 >> 172.21.15.42:0/619313483 conn(0x56285c016740 msgr2=0x56285c3cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:05:48.465 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.462+0000 7f68a18a1740 1 -- 172.21.15.42:0/619313483 shutdown_connections 2024-08-30T09:05:48.465 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.462+0000 7f68a18a1740 1 -- 172.21.15.42:0/619313483 wait complete. 2024-08-30T09:05:48.466 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.463+0000 7f68a18a1740 1 -- start start 2024-08-30T09:05:48.466 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.463+0000 7f68a18a1740 1 Processor -- start 2024-08-30T09:05:48.466 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.464+0000 7f68a18a1740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56285bff9490 0x56285c3dc920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:48.466 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.464+0000 7f68a18a1740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56285c001670 0x56285c3dce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:48.466 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.464+0000 7f68a18a1740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56285c022350 0x56285c3f9c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:48.466 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.464+0000 7f68a18a1740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x56285c190890 con 0x56285bff9490 2024-08-30T09:05:48.466 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.464+0000 7f68a18a1740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x56285c2b4790 con 0x56285c022350 2024-08-30T09:05:48.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.464+0000 7f68a18a1740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x56285c3f0960 con 0x56285c001670 2024-08-30T09:05:48.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.464+0000 7f68a18a1740 10 client.0 _ll_register_callbacks cb 0x56285c039ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:05:48.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.464+0000 7f68990c8640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56285bff9490 0x56285c3dc920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:05:48.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.464+0000 7f68990c8640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56285bff9490 0x56285c3dc920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60534/0 (socket says 172.21.15.42:60534) 2024-08-30T09:05:48.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.464+0000 7f68990c8640 1 -- 172.21.15.42:0/2930206985 learned_addr learned my addr 172.21.15.42:0/2930206985 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:05:48.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.465+0000 7f68988c7640 1 --2- 172.21.15.42:0/2930206985 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56285c001670 0x56285c3dce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:05:48.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.465+0000 7f68998c9640 1 --2- 172.21.15.42:0/2930206985 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56285c022350 0x56285c3f9c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:05:48.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.465+0000 7f68998c9640 1 -- 172.21.15.42:0/2930206985 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56285c001670 msgr2=0x56285c3dce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:05:48.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.465+0000 7f68998c9640 1 --2- 172.21.15.42:0/2930206985 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56285c001670 0x56285c3dce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:48.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.465+0000 7f68998c9640 1 -- 172.21.15.42:0/2930206985 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56285bff9490 msgr2=0x56285c3dc920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:05:48.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.465+0000 7f68998c9640 1 --2- 172.21.15.42:0/2930206985 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56285bff9490 0x56285c3dc920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:48.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.465+0000 7f68998c9640 1 -- 172.21.15.42:0/2930206985 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56285c2ebf30 con 0x56285c022350 2024-08-30T09:05:48.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.466+0000 7f68998c9640 1 --2- 172.21.15.42:0/2930206985 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56285c022350 0x56285c3f9c80 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f688c009ca0 tx=0x7f688c02a8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=322f299ca3ef1bd server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:05:48.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.466+0000 7f687cff9640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-30T09:05:48.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.467+0000 7f687cff9640 1 -- 172.21.15.42:0/2930206985 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f688c004450 con 0x56285c022350 2024-08-30T09:05:48.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.467+0000 7f687cff9640 1 -- 172.21.15.42:0/2930206985 <== mon.2 v2:172.21.15.167:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f688c0045f0 con 0x56285c022350 2024-08-30T09:05:48.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.467+0000 7f68a18a1740 1 -- 172.21.15.42:0/2930206985 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f68840035f0 con 0x56285c022350 2024-08-30T09:05:48.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.467+0000 7f687cff9640 1 -- 172.21.15.42:0/2930206985 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f688c0047f0 con 0x56285c022350 2024-08-30T09:05:48.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.468+0000 7f68a18a1740 10 client.26054 fetch_fsmap learned FSMap version 31 2024-08-30T09:05:48.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.468+0000 7f68a18a1740 1 -- 172.21.15.42:0/2930206985 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x56285c3c8590 con 0x56285c022350 2024-08-30T09:05:48.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.468+0000 7f687cff9640 1 -- 172.21.15.42:0/2930206985 <== mon.2 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f68840035f0 con 0x56285c022350 2024-08-30T09:05:48.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.468+0000 7f687cff9640 1 -- 172.21.15.42:0/2930206985 <== mon.2 v2:172.21.15.167:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f688c04e070 con 0x56285c022350 2024-08-30T09:05:48.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.468+0000 7f68a18a1740 10 client.26054 fetch_fsmap finished waiting for FSMap version 31 2024-08-30T09:05:48.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.468+0000 7f68a18a1740 10 client.26054 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-30T09:05:48.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.468+0000 7f68a18a1740 20 client.26054 populate_metadata read hostname 'smithi042' 2024-08-30T09:05:48.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.468+0000 7f68a18a1740 1 --2- 172.21.15.42:0/2930206985 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x56285bfe0cd0 0x56285c3cbaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:48.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.468+0000 7f68a18a1740 4 client.26054 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-30T09:05:48.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.468+0000 7f68a18a1740 1 -- 172.21.15.42:0/2930206985 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56285c340540 con 0x56285bfe0cd0 2024-08-30T09:05:48.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.469+0000 7f68990c8640 1 --2- 172.21.15.42:0/2930206985 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x56285bfe0cd0 0x56285c3cbaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:05:48.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.470+0000 7f687cff9640 10 client.26054 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:05:48.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.472+0000 7f68990c8640 1 --2- 172.21.15.42:0/2930206985 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x56285bfe0cd0 0x56285c3cbaf0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b87174df5f9cf17d server_cookie=acf0f24b49a52943 in_seq=0 out_seq=0 2024-08-30T09:05:48.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.473+0000 7f687cff9640 1 -- 172.21.15.42:0/2930206985 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56285c340540 con 0x56285bfe0cd0 2024-08-30T09:05:48.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.473+0000 7f687cff9640 10 client.26054 handle_command_reply: tid=0 2024-08-30T09:05:48.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.474+0000 7f68a18a1740 1 client.26054 shutdown 2024-08-30T09:05:48.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.475+0000 7f68a18a1740 1 -- 172.21.15.42:0/2930206985 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56285c022350 msgr2=0x56285c3f9c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:05:48.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.475+0000 7f68a18a1740 1 --2- 172.21.15.42:0/2930206985 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56285c022350 0x56285c3f9c80 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f688c009ca0 tx=0x7f688c02a8b0 comp rx=0 tx=0).stop 2024-08-30T09:05:48.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.476+0000 7f68a18a1740 1 -- 172.21.15.42:0/2930206985 shutdown_connections 2024-08-30T09:05:48.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.476+0000 7f68a18a1740 1 --2- 172.21.15.42:0/2930206985 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x56285bfe0cd0 0x56285c3cbaf0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:48.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.476+0000 7f68a18a1740 1 --2- 172.21.15.42:0/2930206985 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56285c001670 0x56285c3dce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:48.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.476+0000 7f68a18a1740 1 --2- 172.21.15.42:0/2930206985 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56285c022350 0x56285c3f9c80 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:48.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.476+0000 7f68a18a1740 1 --2- 172.21.15.42:0/2930206985 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56285bff9490 0x56285c3dc920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:48.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.476+0000 7f68a18a1740 1 -- 172.21.15.42:0/2930206985 >> 172.21.15.42:0/2930206985 conn(0x56285c016740 msgr2=0x56285c3c4f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:05:48.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.476+0000 7f687cff9640 0 client.26054 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:05:48.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.476+0000 7f68a18a1740 1 -- 172.21.15.42:0/2930206985 shutdown_connections 2024-08-30T09:05:48.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.476+0000 7f68a18a1740 1 -- 172.21.15.42:0/2930206985 wait complete. 2024-08-30T09:05:48.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.476+0000 7f68a18a1740 20 client.26054 trim_cache size 0 max 16384 2024-08-30T09:05:48.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.510+0000 7f68a18a1740 1 Processor -- start 2024-08-30T09:05:48.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.513+0000 7f68a18a1740 1 -- start start 2024-08-30T09:05:48.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.513+0000 7f68a18a1740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56285bff9490 0x56285c3dc6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:48.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.513+0000 7f68a18a1740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56285c001670 0x56285c3dcc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:48.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.513+0000 7f68a18a1740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56285c022350 0x56285c3d9b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:48.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.513+0000 7f68a18a1740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f68941a86c0 con 0x56285c022350 2024-08-30T09:05:48.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.513+0000 7f68a18a1740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f68941a5f90 con 0x56285bff9490 2024-08-30T09:05:48.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.513+0000 7f68a18a1740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f6894074b40 con 0x56285c001670 2024-08-30T09:05:48.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.514+0000 7f68988c7640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56285c001670 0x56285c3dcc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:05:48.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.514+0000 7f68988c7640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56285c001670 0x56285c3dcc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:48988/0 (socket says 172.21.15.42:48988) 2024-08-30T09:05:48.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.514+0000 7f68988c7640 1 -- 172.21.15.42:0/2001021035 learned_addr learned my addr 172.21.15.42:0/2001021035 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:05:48.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.514+0000 7f68998c9640 1 --2- 172.21.15.42:0/2001021035 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56285c022350 0x56285c3d9b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:05:48.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.514+0000 7f68990c8640 1 --2- 172.21.15.42:0/2001021035 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56285bff9490 0x56285c3dc6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:05:48.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.514+0000 7f68998c9640 1 -- 172.21.15.42:0/2001021035 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56285c001670 msgr2=0x56285c3dcc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:05:48.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.514+0000 7f68998c9640 1 --2- 172.21.15.42:0/2001021035 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56285c001670 0x56285c3dcc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:48.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.514+0000 7f68998c9640 1 -- 172.21.15.42:0/2001021035 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56285bff9490 msgr2=0x56285c3dc6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:05:48.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.514+0000 7f68998c9640 1 --2- 172.21.15.42:0/2001021035 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56285bff9490 0x56285c3dc6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:48.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.514+0000 7f68998c9640 1 -- 172.21.15.42:0/2001021035 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56285bfe0f00 con 0x56285c022350 2024-08-30T09:05:48.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.515+0000 7f68998c9640 1 --2- 172.21.15.42:0/2001021035 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56285c022350 0x56285c3d9b10 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f688c050530 tx=0x7f688c02adb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe307c59c9a9d6e3 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:05:48.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.516+0000 7f687effd640 1 -- 172.21.15.42:0/2001021035 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f688c004d40 con 0x56285c022350 2024-08-30T09:05:48.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.516+0000 7f687effd640 1 -- 172.21.15.42:0/2001021035 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f688c036040 con 0x56285c022350 2024-08-30T09:05:48.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.516+0000 7f687effd640 1 -- 172.21.15.42:0/2001021035 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f688c050d40 con 0x56285c022350 2024-08-30T09:05:48.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.520+0000 7f68a18a1740 1 -- 172.21.15.42:0/2001021035 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56285c022350 msgr2=0x56285c3d9b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:05:48.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.520+0000 7f68a18a1740 1 --2- 172.21.15.42:0/2001021035 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56285c022350 0x56285c3d9b10 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f688c050530 tx=0x7f688c02adb0 comp rx=0 tx=0).stop 2024-08-30T09:05:48.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.524+0000 7f68a18a1740 1 -- 172.21.15.42:0/2001021035 shutdown_connections 2024-08-30T09:05:48.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.524+0000 7f68a18a1740 1 --2- 172.21.15.42:0/2001021035 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56285c001670 0x56285c3dcc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:48.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.524+0000 7f68a18a1740 1 --2- 172.21.15.42:0/2001021035 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56285c022350 0x56285c3d9b10 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:48.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.524+0000 7f68a18a1740 1 --2- 172.21.15.42:0/2001021035 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56285bff9490 0x56285c3dc6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:48.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.524+0000 7f68a18a1740 1 -- 172.21.15.42:0/2001021035 >> 172.21.15.42:0/2001021035 conn(0x56285c016740 msgr2=0x56285c3db1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:05:48.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.527+0000 7f68a18a1740 1 -- 172.21.15.42:0/2001021035 shutdown_connections 2024-08-30T09:05:48.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.527+0000 7f68a18a1740 1 -- 172.21.15.42:0/2001021035 wait complete. 2024-08-30T09:05:48.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.527+0000 7f68a18a1740 1 -- start start 2024-08-30T09:05:48.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.528+0000 7f68a18a1740 1 Processor -- start 2024-08-30T09:05:48.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.529+0000 7f68a18a1740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56285bff9490 0x56285c3f64c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:48.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.529+0000 7f68a18a1740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56285c001670 0x56285c3d9960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:48.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.529+0000 7f68a18a1740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56285c022350 0x56285c3d9fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:48.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.529+0000 7f68a18a1740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f6894075190 con 0x56285bff9490 2024-08-30T09:05:48.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.529+0000 7f68a18a1740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f68941a58c0 con 0x56285c022350 2024-08-30T09:05:48.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.529+0000 7f68a18a1740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f68941a4b20 con 0x56285c001670 2024-08-30T09:05:48.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.529+0000 7f68a18a1740 10 client.0 _ll_register_callbacks cb 0x7f688c009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:05:48.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.530+0000 7f68988c7640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56285c001670 0x56285c3d9960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:05:48.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.530+0000 7f68998c9640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56285c022350 0x56285c3d9fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:05:48.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.530+0000 7f68988c7640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56285c001670 0x56285c3d9960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:49004/0 (socket says 172.21.15.42:49004) 2024-08-30T09:05:48.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.530+0000 7f68988c7640 1 -- 172.21.15.42:0/195891127 learned_addr learned my addr 172.21.15.42:0/195891127 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:05:48.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.532+0000 7f68990c8640 1 --2- 172.21.15.42:0/195891127 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56285bff9490 0x56285c3f64c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:05:48.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.534+0000 7f68998c9640 1 -- 172.21.15.42:0/195891127 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56285c001670 msgr2=0x56285c3d9960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:05:48.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.534+0000 7f68998c9640 1 --2- 172.21.15.42:0/195891127 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56285c001670 0x56285c3d9960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:48.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.534+0000 7f68998c9640 1 -- 172.21.15.42:0/195891127 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56285bff9490 msgr2=0x56285c3f64c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:05:48.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.534+0000 7f68998c9640 1 --2- 172.21.15.42:0/195891127 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56285bff9490 0x56285c3f64c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:48.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.534+0000 7f68998c9640 1 -- 172.21.15.42:0/195891127 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56285c3da740 con 0x56285c022350 2024-08-30T09:05:48.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.535+0000 7f68998c9640 1 --2- 172.21.15.42:0/195891127 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56285c022350 0x56285c3d9fb0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f688c009620 tx=0x7f688c02a8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f3afb5875ab29bc4 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:05:48.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.536+0000 7f687cff9640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-30T09:05:48.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.543+0000 7f687cff9640 1 -- 172.21.15.42:0/195891127 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f688c045670 con 0x56285c022350 2024-08-30T09:05:48.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.543+0000 7f687cff9640 1 -- 172.21.15.42:0/195891127 <== mon.2 v2:172.21.15.167:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f688c045810 con 0x56285c022350 2024-08-30T09:05:48.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.544+0000 7f68a18a1740 1 -- 172.21.15.42:0/195891127 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f688c036040 con 0x56285c022350 2024-08-30T09:05:48.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.552+0000 7f687cff9640 1 -- 172.21.15.42:0/195891127 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f688c045a50 con 0x56285c022350 2024-08-30T09:05:48.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.556+0000 7f68a18a1740 10 client.26057 fetch_fsmap learned FSMap version 31 2024-08-30T09:05:48.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.556+0000 7f68a18a1740 1 -- 172.21.15.42:0/195891127 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x56285c4734b0 con 0x56285c022350 2024-08-30T09:05:48.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.562+0000 7f687cff9640 1 -- 172.21.15.42:0/195891127 <== mon.2 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f688c036040 con 0x56285c022350 2024-08-30T09:05:48.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.563+0000 7f687cff9640 1 -- 172.21.15.42:0/195891127 <== mon.2 v2:172.21.15.167:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f688c05c020 con 0x56285c022350 2024-08-30T09:05:48.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.564+0000 7f68a18a1740 10 client.26057 fetch_fsmap finished waiting for FSMap version 31 2024-08-30T09:05:48.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.564+0000 7f68a18a1740 10 client.26057 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-30T09:05:48.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.564+0000 7f68a18a1740 20 client.26057 populate_metadata read hostname 'smithi042' 2024-08-30T09:05:48.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.564+0000 7f68a18a1740 1 --2- 172.21.15.42:0/195891127 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x56285bfb6080 0x56285c475a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:48.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.564+0000 7f68a18a1740 4 client.26057 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-08-30T09:05:48.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.564+0000 7f68a18a1740 1 -- 172.21.15.42:0/195891127 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x56285c33bfe0 con 0x56285bfb6080 2024-08-30T09:05:48.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.564+0000 7f68990c8640 1 --2- 172.21.15.42:0/195891127 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x56285bfb6080 0x56285c475a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:05:48.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.565+0000 7f68990c8640 1 --2- 172.21.15.42:0/195891127 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x56285bfb6080 0x56285c475a70 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bcb331c3be467d68 server_cookie=c8d3b1fee14a4f33 in_seq=0 out_seq=0 2024-08-30T09:05:48.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.566+0000 7f687cff9640 10 client.26057 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:05:48.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.567+0000 7f687cff9640 1 -- 172.21.15.42:0/195891127 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56285c33bfe0 con 0x56285bfb6080 2024-08-30T09:05:48.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.567+0000 7f687cff9640 10 client.26057 handle_command_reply: tid=0 2024-08-30T09:05:48.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.570+0000 7f68a18a1740 1 client.26057 shutdown 2024-08-30T09:05:48.576 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:05:48 smithi167 ceph-mon[22063]: pgmap v524: 129 pgs: 129 active+clean; 1.2 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 8.6 MiB/s wr, 556 op/s 2024-08-30T09:05:48.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.576+0000 7f68a18a1740 1 -- 172.21.15.42:0/195891127 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56285c022350 msgr2=0x56285c3d9fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:05:48.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.576+0000 7f68a18a1740 1 --2- 172.21.15.42:0/195891127 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56285c022350 0x56285c3d9fb0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f688c009620 tx=0x7f688c02a8b0 comp rx=0 tx=0).stop 2024-08-30T09:05:48.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.577+0000 7f68a18a1740 1 -- 172.21.15.42:0/195891127 shutdown_connections 2024-08-30T09:05:48.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.577+0000 7f68a18a1740 1 --2- 172.21.15.42:0/195891127 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x56285bfb6080 0x56285c475a70 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:48.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.577+0000 7f68a18a1740 1 --2- 172.21.15.42:0/195891127 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56285c001670 0x56285c3d9960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:48.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.577+0000 7f68a18a1740 1 --2- 172.21.15.42:0/195891127 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56285c022350 0x56285c3d9fb0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:48.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.577+0000 7f68a18a1740 1 --2- 172.21.15.42:0/195891127 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56285bff9490 0x56285c3f64c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:48.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.577+0000 7f687cff9640 0 client.26057 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:05:48.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.577+0000 7f68a18a1740 1 -- 172.21.15.42:0/195891127 >> 172.21.15.42:0/195891127 conn(0x56285c016740 msgr2=0x56285c3dada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:05:48.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.578+0000 7f68a18a1740 1 -- 172.21.15.42:0/195891127 shutdown_connections 2024-08-30T09:05:48.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.580+0000 7f68a18a1740 1 -- 172.21.15.42:0/195891127 wait complete. 2024-08-30T09:05:48.585 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-08-30T09:05:48.585 INFO:teuthology.orchestra.run.smithi042.stdout: "return_code": 0, 2024-08-30T09:05:48.585 INFO:teuthology.orchestra.run.smithi042.stdout: "scrub_tag": "1af4252a-4a99-47e9-bbf0-becf5e573dd5", 2024-08-30T09:05:48.585 INFO:teuthology.orchestra.run.smithi042.stdout: "mode": "asynchronous" 2024-08-30T09:05:48.585 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-08-30T09:05:48.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.583+0000 7f68a18a1740 20 client.26057 trim_cache size 0 max 16384 2024-08-30T09:05:48.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.583+0000 7f689a0ca640 1 -- 172.21.15.42:0/1942008346 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f68800b4c00 msgr2=0x7f68800b7110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:05:48.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.583+0000 7f689a0ca640 1 --2- 172.21.15.42:0/1942008346 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f68800b4c00 0x7f68800b7110 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f688c031040 tx=0x7f688c005d80 comp rx=0 tx=0).stop 2024-08-30T09:05:48.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.583+0000 7f689a0ca640 1 -- 172.21.15.42:0/1942008346 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6894101960 msgr2=0x7f6894197c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:05:48.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.583+0000 7f689a0ca640 1 --2- 172.21.15.42:0/1942008346 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6894101960 0x7f6894197c20 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f689000d950 tx=0x7f689000de10 comp rx=0 tx=0).stop 2024-08-30T09:05:48.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.584+0000 7f689a0ca640 1 -- 172.21.15.42:0/1942008346 shutdown_connections 2024-08-30T09:05:48.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.584+0000 7f689a0ca640 1 --2- 172.21.15.42:0/1942008346 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f68800b4c00 0x7f68800b7110 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:48.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.584+0000 7f689a0ca640 1 --2- 172.21.15.42:0/1942008346 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6894107bf0 0x7f689419d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:48.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.584+0000 7f689a0ca640 1 --2- 172.21.15.42:0/1942008346 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f68941072a0 0x7f68941981c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:48.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.584+0000 7f689a0ca640 1 --2- 172.21.15.42:0/1942008346 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6894101960 0x7f6894197c20 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:48.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.584+0000 7f689a0ca640 1 -- 172.21.15.42:0/1942008346 >> 172.21.15.42:0/1942008346 conn(0x7f6894078420 msgr2=0x7f6894103090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:05:48.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.584+0000 7f689a0ca640 1 -- 172.21.15.42:0/1942008346 shutdown_connections 2024-08-30T09:05:48.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:48.584+0000 7f689a0ca640 1 -- 172.21.15.42:0/1942008346 wait complete. 2024-08-30T09:05:49.283 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:05:49 smithi042 ceph-mon[18652]: scrub queued for path: / 2024-08-30T09:05:49.283 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:05:49 smithi042 ceph-mon[18652]: scrub summary: idle+waiting paths [/] 2024-08-30T09:05:49.283 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:05:49 smithi042 ceph-mon[18652]: scrub summary: active paths [/] 2024-08-30T09:05:49.283 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:05:49 smithi042 ceph-mon[18652]: scrub summary: active paths [/] 2024-08-30T09:05:49.328 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-30T09:05:49.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:05:49 smithi190 ceph-mon[22001]: scrub queued for path: / 2024-08-30T09:05:49.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:05:49 smithi190 ceph-mon[22001]: scrub summary: idle+waiting paths [/] 2024-08-30T09:05:49.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:05:49 smithi190 ceph-mon[22001]: scrub summary: active paths [/] 2024-08-30T09:05:49.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:05:49 smithi190 ceph-mon[22001]: scrub summary: active paths [/] 2024-08-30T09:05:49.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:05:49 smithi167 ceph-mon[22063]: scrub queued for path: / 2024-08-30T09:05:49.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:05:49 smithi167 ceph-mon[22063]: scrub summary: idle+waiting paths [/] 2024-08-30T09:05:49.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:05:49 smithi167 ceph-mon[22063]: scrub summary: active paths [/] 2024-08-30T09:05:49.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:05:49 smithi167 ceph-mon[22063]: scrub summary: active paths [/] 2024-08-30T09:05:49.772 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T09:05:50.283 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:05:50 smithi042 ceph-mon[18652]: scrub summary: active paths [/] 2024-08-30T09:05:50.283 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:05:50 smithi042 ceph-mon[18652]: scrub summary: idle+waiting paths [/] 2024-08-30T09:05:50.283 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:05:50 smithi042 ceph-mon[18652]: pgmap v525: 129 pgs: 129 active+clean; 1.2 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 10 MiB/s wr, 570 op/s 2024-08-30T09:05:50.283 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:05:50 smithi042 ceph-mon[18652]: scrub summary: idle+waiting paths [/] 2024-08-30T09:05:50.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:05:50 smithi190 ceph-mon[22001]: scrub summary: active paths [/] 2024-08-30T09:05:50.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:05:50 smithi190 ceph-mon[22001]: scrub summary: idle+waiting paths [/] 2024-08-30T09:05:50.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:05:50 smithi190 ceph-mon[22001]: pgmap v525: 129 pgs: 129 active+clean; 1.2 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 10 MiB/s wr, 570 op/s 2024-08-30T09:05:50.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:05:50 smithi190 ceph-mon[22001]: scrub summary: idle+waiting paths [/] 2024-08-30T09:05:50.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.443+0000 7f4ac37e5640 1 -- 172.21.15.42:0/3668923114 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4abc10b890 msgr2=0x7f4abc10bca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:05:50.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.443+0000 7f4ac37e5640 1 --2- 172.21.15.42:0/3668923114 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4abc10b890 0x7f4abc10bca0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f4ab800d910 tx=0x7f4ab802e5c0 comp rx=0 tx=0).stop 2024-08-30T09:05:50.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.446+0000 7f4ac37e5640 1 -- 172.21.15.42:0/3668923114 shutdown_connections 2024-08-30T09:05:50.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.446+0000 7f4ac37e5640 1 --2- 172.21.15.42:0/3668923114 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4abc10f8e0 0x7f4abc10fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:50.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.446+0000 7f4ac37e5640 1 --2- 172.21.15.42:0/3668923114 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4abc110230 0x7f4abc07b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:50.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.446+0000 7f4ac37e5640 1 --2- 172.21.15.42:0/3668923114 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4abc10b890 0x7f4abc10bca0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:50.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.446+0000 7f4ac37e5640 1 -- 172.21.15.42:0/3668923114 >> 172.21.15.42:0/3668923114 conn(0x7f4abc070c60 msgr2=0x7f4abc071070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:05:50.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.447+0000 7f4ac37e5640 1 -- 172.21.15.42:0/3668923114 shutdown_connections 2024-08-30T09:05:50.454 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.449+0000 7f4ac37e5640 1 -- 172.21.15.42:0/3668923114 wait complete. 2024-08-30T09:05:50.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.450+0000 7f4ac37e5640 1 Processor -- start 2024-08-30T09:05:50.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.450+0000 7f4ac37e5640 1 -- start start 2024-08-30T09:05:50.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.450+0000 7f4ac37e5640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4abc10b890 0x7f4abc080420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:50.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.450+0000 7f4ac37e5640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4abc10f8e0 0x7f4abc080960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:50.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.451+0000 7f4ac37e5640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4abc110230 0x7f4abc080ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:50.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.451+0000 7f4ac37e5640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f4abc07e910 con 0x7f4abc10f8e0 2024-08-30T09:05:50.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.451+0000 7f4ac37e5640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4abc07e790 con 0x7f4abc110230 2024-08-30T09:05:50.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.451+0000 7f4ac37e5640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f4abc07ea90 con 0x7f4abc10b890 2024-08-30T09:05:50.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.452+0000 7f4ac1d82640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4abc110230 0x7f4abc080ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:05:50.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.452+0000 7f4ac1d82640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4abc110230 0x7f4abc080ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:40328/0 (socket says 172.21.15.42:40328) 2024-08-30T09:05:50.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.452+0000 7f4ac1d82640 1 -- 172.21.15.42:0/3474482618 learned_addr learned my addr 172.21.15.42:0/3474482618 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:05:50.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.452+0000 7f4ac1581640 1 --2- 172.21.15.42:0/3474482618 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4abc10b890 0x7f4abc080420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:05:50.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.452+0000 7f4ac0d80640 1 --2- 172.21.15.42:0/3474482618 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4abc10f8e0 0x7f4abc080960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:05:50.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.454+0000 7f4ac1581640 1 -- 172.21.15.42:0/3474482618 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4abc110230 msgr2=0x7f4abc080ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:05:50.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.454+0000 7f4ac1581640 1 --2- 172.21.15.42:0/3474482618 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4abc110230 0x7f4abc080ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:50.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.454+0000 7f4ac1581640 1 -- 172.21.15.42:0/3474482618 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4abc10f8e0 msgr2=0x7f4abc080960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:05:50.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.454+0000 7f4ac1581640 1 --2- 172.21.15.42:0/3474482618 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4abc10f8e0 0x7f4abc080960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:50.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.454+0000 7f4ac1581640 1 -- 172.21.15.42:0/3474482618 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4abc1cea20 con 0x7f4abc10b890 2024-08-30T09:05:50.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.454+0000 7f4ac1581640 1 --2- 172.21.15.42:0/3474482618 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4abc10b890 0x7f4abc080420 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f4ab8006010 tx=0x7f4ab800d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:05:50.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.454+0000 7f4ab27fc640 1 -- 172.21.15.42:0/3474482618 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ab8004020 con 0x7f4abc10b890 2024-08-30T09:05:50.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.455+0000 7f4ac37e5640 1 -- 172.21.15.42:0/3474482618 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4abc1cecb0 con 0x7f4abc10b890 2024-08-30T09:05:50.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.455+0000 7f4ac37e5640 1 -- 172.21.15.42:0/3474482618 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4abc1cf1b0 con 0x7f4abc10b890 2024-08-30T09:05:50.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.455+0000 7f4ab27fc640 1 -- 172.21.15.42:0/3474482618 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ab80053b0 con 0x7f4abc10b890 2024-08-30T09:05:50.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.455+0000 7f4ab27fc640 1 -- 172.21.15.42:0/3474482618 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ab80035f0 con 0x7f4abc10b890 2024-08-30T09:05:50.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.460+0000 7f4ab27fc640 1 -- 172.21.15.42:0/3474482618 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7f4ab8045060 con 0x7f4abc10b890 2024-08-30T09:05:50.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.461+0000 7f4ab27fc640 1 --2- 172.21.15.42:0/3474482618 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f4aa80b4c60 0x7f4aa80b7170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:50.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.461+0000 7f4ab27fc640 1 -- 172.21.15.42:0/3474482618 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f4ab81042d0 con 0x7f4abc10b890 2024-08-30T09:05:50.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.468+0000 7f4ac9d5a740 1 Processor -- start 2024-08-30T09:05:50.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.469+0000 7f4ac0d80640 1 --2- 172.21.15.42:0/3474482618 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f4aa80b4c60 0x7f4aa80b7170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:05:50.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.470+0000 7f4ac0d80640 1 --2- 172.21.15.42:0/3474482618 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f4aa80b4c60 0x7f4aa80b7170 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f4abc071260 tx=0x7f4ab40094f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:05:50.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.472+0000 7f4ac9d5a740 1 -- start start 2024-08-30T09:05:50.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.472+0000 7f4ac9d5a740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b45270a070 0x55b452aaf2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:50.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.472+0000 7f4ac9d5a740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b4526e11b0 0x55b452aaf800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:50.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.472+0000 7f4ac9d5a740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b4526e9390 0x55b452aafd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:50.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.472+0000 7f4ac9d5a740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55b452a26630 con 0x55b45270a070 2024-08-30T09:05:50.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.472+0000 7f4ac9d5a740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55b452a15bc0 con 0x55b4526e9390 2024-08-30T09:05:50.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.472+0000 7f4ac9d5a740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55b452a19b70 con 0x55b4526e11b0 2024-08-30T09:05:50.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.473+0000 7f4ac1d82640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b4526e9390 0x55b452aafd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:05:50.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.473+0000 7f4ac0d80640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b4526e11b0 0x55b452aaf800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:05:50.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.473+0000 7f4ac0d80640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b4526e11b0 0x55b452aaf800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:49038/0 (socket says 172.21.15.42:49038) 2024-08-30T09:05:50.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.473+0000 7f4ac0d80640 1 -- 172.21.15.42:0/4060369500 learned_addr learned my addr 172.21.15.42:0/4060369500 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:05:50.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.473+0000 7f4ac1581640 1 --2- 172.21.15.42:0/4060369500 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b45270a070 0x55b452aaf2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:05:50.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.473+0000 7f4ac1d82640 1 -- 172.21.15.42:0/4060369500 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b4526e11b0 msgr2=0x55b452aaf800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:05:50.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.473+0000 7f4ac1d82640 1 --2- 172.21.15.42:0/4060369500 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b4526e11b0 0x55b452aaf800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:50.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.473+0000 7f4ac1d82640 1 -- 172.21.15.42:0/4060369500 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b45270a070 msgr2=0x55b452aaf2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:05:50.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.473+0000 7f4ac1d82640 1 --2- 172.21.15.42:0/4060369500 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b45270a070 0x55b452aaf2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:50.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.473+0000 7f4ac1d82640 1 -- 172.21.15.42:0/4060369500 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b452775650 con 0x55b4526e9390 2024-08-30T09:05:50.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.475+0000 7f4ac1d82640 1 --2- 172.21.15.42:0/4060369500 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b4526e9390 0x55b452aafd40 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f4aac00aaa0 tx=0x7f4aac00af60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8d9066c550572258 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:05:50.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.476+0000 7f4aa77fe640 1 -- 172.21.15.42:0/4060369500 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4aac019030 con 0x55b4526e9390 2024-08-30T09:05:50.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.476+0000 7f4aa77fe640 1 -- 172.21.15.42:0/4060369500 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4aac0054d0 con 0x55b4526e9390 2024-08-30T09:05:50.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.477+0000 7f4ac9d5a740 1 -- 172.21.15.42:0/4060369500 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b4526e9390 msgr2=0x55b452aafd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:05:50.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.477+0000 7f4ac9d5a740 1 --2- 172.21.15.42:0/4060369500 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b4526e9390 0x55b452aafd40 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f4aac00aaa0 tx=0x7f4aac00af60 comp rx=0 tx=0).stop 2024-08-30T09:05:50.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.480+0000 7f4ac9d5a740 1 -- 172.21.15.42:0/4060369500 shutdown_connections 2024-08-30T09:05:50.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.480+0000 7f4ac9d5a740 1 --2- 172.21.15.42:0/4060369500 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b4526e11b0 0x55b452aaf800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:50.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.480+0000 7f4ac9d5a740 1 --2- 172.21.15.42:0/4060369500 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b4526e9390 0x55b452aafd40 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:50.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.480+0000 7f4ac9d5a740 1 --2- 172.21.15.42:0/4060369500 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b45270a070 0x55b452aaf2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:50.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.480+0000 7f4ac9d5a740 1 -- 172.21.15.42:0/4060369500 >> 172.21.15.42:0/4060369500 conn(0x55b4526fe460 msgr2=0x55b452aad1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:05:50.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.480+0000 7f4ac9d5a740 1 -- 172.21.15.42:0/4060369500 shutdown_connections 2024-08-30T09:05:50.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.480+0000 7f4ac9d5a740 1 -- 172.21.15.42:0/4060369500 wait complete. 2024-08-30T09:05:50.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.480+0000 7f4ac9d5a740 1 -- start start 2024-08-30T09:05:50.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.481+0000 7f4ac9d5a740 1 Processor -- start 2024-08-30T09:05:50.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.481+0000 7f4ac9d5a740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b4526e11b0 0x55b452abaa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:50.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.481+0000 7f4ac9d5a740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b4526e9390 0x55b452abaf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:50.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.481+0000 7f4ac9d5a740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b45270a070 0x55b452ac6ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:50.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.481+0000 7f4ac9d5a740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55b452878700 con 0x55b4526e9390 2024-08-30T09:05:50.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.481+0000 7f4ac9d5a740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55b45299c700 con 0x55b4526e11b0 2024-08-30T09:05:50.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.481+0000 7f4ac9d5a740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55b452abdaf0 con 0x55b45270a070 2024-08-30T09:05:50.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.482+0000 7f4ac1581640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b4526e11b0 0x55b452abaa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:05:50.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.482+0000 7f4ac1581640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b4526e11b0 0x55b452abaa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:40338/0 (socket says 172.21.15.42:40338) 2024-08-30T09:05:50.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.482+0000 7f4ac1581640 1 -- 172.21.15.42:0/2076031343 learned_addr learned my addr 172.21.15.42:0/2076031343 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:05:50.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.482+0000 7f4ac9d5a740 10 client.0 _ll_register_callbacks cb 0x55b4529cbdc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:05:50.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.482+0000 7f4ac1d82640 1 --2- 172.21.15.42:0/2076031343 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b45270a070 0x55b452ac6ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:05:50.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.482+0000 7f4ac1581640 1 -- 172.21.15.42:0/2076031343 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b45270a070 msgr2=0x55b452ac6ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:05:50.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.482+0000 7f4ac1581640 1 --2- 172.21.15.42:0/2076031343 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b45270a070 0x55b452ac6ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:50.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.482+0000 7f4ac1581640 1 -- 172.21.15.42:0/2076031343 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b4526e9390 msgr2=0x55b452abaf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:05:50.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.482+0000 7f4ac1581640 1 --2- 172.21.15.42:0/2076031343 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b4526e9390 0x55b452abaf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:50.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.482+0000 7f4ac1581640 1 -- 172.21.15.42:0/2076031343 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b4529d58b0 con 0x55b4526e11b0 2024-08-30T09:05:50.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.484+0000 7f4ac1581640 1 --2- 172.21.15.42:0/2076031343 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b4526e11b0 0x55b452abaa30 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f4ab80ca8a0 tx=0x7f4ab80cae60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=83f82ce224b6f23d server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:05:50.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.485+0000 7f4aa4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-30T09:05:50.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.485+0000 7f4aa4ff9640 1 -- 172.21.15.42:0/2076031343 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ab80ca420 con 0x55b4526e11b0 2024-08-30T09:05:50.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.485+0000 7f4aa4ff9640 1 -- 172.21.15.42:0/2076031343 <== mon.2 v2:172.21.15.167:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f4ab80c9d80 con 0x55b4526e11b0 2024-08-30T09:05:50.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.485+0000 7f4ac9d5a740 1 -- 172.21.15.42:0/2076031343 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4aac0054d0 con 0x55b4526e11b0 2024-08-30T09:05:50.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.487+0000 7f4aa4ff9640 1 -- 172.21.15.42:0/2076031343 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f4ab80cb010 con 0x55b4526e11b0 2024-08-30T09:05:50.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.487+0000 7f4ac9d5a740 10 client.26069 fetch_fsmap learned FSMap version 31 2024-08-30T09:05:50.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.487+0000 7f4ac9d5a740 1 -- 172.21.15.42:0/2076031343 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55b452ac9890 con 0x55b4526e11b0 2024-08-30T09:05:50.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.487+0000 7f4aa4ff9640 1 -- 172.21.15.42:0/2076031343 <== mon.2 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ab80ca910 con 0x55b4526e11b0 2024-08-30T09:05:50.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.487+0000 7f4aa4ff9640 1 -- 172.21.15.42:0/2076031343 <== mon.2 v2:172.21.15.167:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f4ab80c6310 con 0x55b4526e11b0 2024-08-30T09:05:50.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.487+0000 7f4ac9d5a740 10 client.26069 fetch_fsmap finished waiting for FSMap version 31 2024-08-30T09:05:50.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.487+0000 7f4ac9d5a740 10 client.26069 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-30T09:05:50.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.487+0000 7f4ac9d5a740 20 client.26069 populate_metadata read hostname 'smithi042' 2024-08-30T09:05:50.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.487+0000 7f4ac9d5a740 1 --2- 172.21.15.42:0/2076031343 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55b4526c89f0 0x55b452accdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:50.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.487+0000 7f4ac9d5a740 4 client.26069 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-30T09:05:50.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.487+0000 7f4ac9d5a740 1 -- 172.21.15.42:0/2076031343 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f4aac0054d0 con 0x55b4526c89f0 2024-08-30T09:05:50.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.488+0000 7f4ac0d80640 1 --2- 172.21.15.42:0/2076031343 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55b4526c89f0 0x55b452accdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:05:50.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.489+0000 7f4ac0d80640 1 --2- 172.21.15.42:0/2076031343 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55b4526c89f0 0x55b452accdc0 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5aeac8566c63905c server_cookie=cfa788bac1689c8c in_seq=0 out_seq=0 2024-08-30T09:05:50.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.490+0000 7f4aa4ff9640 10 client.26069 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:05:50.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.490+0000 7f4aa4ff9640 1 -- 172.21.15.42:0/2076031343 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f4aac0054d0 con 0x55b4526c89f0 2024-08-30T09:05:50.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.490+0000 7f4aa4ff9640 10 client.26069 handle_command_reply: tid=0 2024-08-30T09:05:50.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.491+0000 7f4ac9d5a740 1 client.26069 shutdown 2024-08-30T09:05:50.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.493+0000 7f4ac9d5a740 1 -- 172.21.15.42:0/2076031343 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b4526e11b0 msgr2=0x55b452abaa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:05:50.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.493+0000 7f4ac9d5a740 1 --2- 172.21.15.42:0/2076031343 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b4526e11b0 0x55b452abaa30 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f4ab80ca8a0 tx=0x7f4ab80cae60 comp rx=0 tx=0).stop 2024-08-30T09:05:50.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.493+0000 7f4ac9d5a740 1 -- 172.21.15.42:0/2076031343 shutdown_connections 2024-08-30T09:05:50.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.493+0000 7f4ac9d5a740 1 --2- 172.21.15.42:0/2076031343 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55b4526c89f0 0x55b452accdc0 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:50.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.493+0000 7f4ac9d5a740 1 --2- 172.21.15.42:0/2076031343 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b45270a070 0x55b452ac6ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:50.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.493+0000 7f4ac9d5a740 1 --2- 172.21.15.42:0/2076031343 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b4526e9390 0x55b452abaf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:50.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.493+0000 7f4ac9d5a740 1 --2- 172.21.15.42:0/2076031343 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b4526e11b0 0x55b452abaa30 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:50.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.493+0000 7f4ac9d5a740 1 -- 172.21.15.42:0/2076031343 >> 172.21.15.42:0/2076031343 conn(0x55b4526fe460 msgr2=0x55b452ab24a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:05:50.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.493+0000 7f4aa4ff9640 0 client.26069 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:05:50.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.493+0000 7f4ac9d5a740 1 -- 172.21.15.42:0/2076031343 shutdown_connections 2024-08-30T09:05:50.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.493+0000 7f4ac9d5a740 1 -- 172.21.15.42:0/2076031343 wait complete. 2024-08-30T09:05:50.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.494+0000 7f4ac9d5a740 20 client.26069 trim_cache size 0 max 16384 2024-08-30T09:05:50.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.524+0000 7f4ac9d5a740 1 Processor -- start 2024-08-30T09:05:50.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.525+0000 7f4ac9d5a740 1 -- start start 2024-08-30T09:05:50.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.525+0000 7f4ac9d5a740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b4526e11b0 0x55b452ab9920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:50.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.525+0000 7f4ac9d5a740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b4526e9390 0x55b452abc660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:50.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.525+0000 7f4ac9d5a740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b45270a070 0x55b452aac780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:50.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.525+0000 7f4ac9d5a740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f4aa0012590 con 0x55b4526e11b0 2024-08-30T09:05:50.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.525+0000 7f4ac1581640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b4526e11b0 0x55b452ab9920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:05:50.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.525+0000 7f4ac1581640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b4526e11b0 0x55b452ab9920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60602/0 (socket says 172.21.15.42:60602) 2024-08-30T09:05:50.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.525+0000 7f4ac1581640 1 -- 172.21.15.42:0/3227332908 learned_addr learned my addr 172.21.15.42:0/3227332908 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:05:50.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.525+0000 7f4ac0d80640 1 --2- 172.21.15.42:0/3227332908 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b4526e9390 0x55b452abc660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:05:50.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.525+0000 7f4ac1d82640 1 --2- 172.21.15.42:0/3227332908 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b45270a070 0x55b452aac780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:05:50.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.525+0000 7f4ac9d5a740 1 -- 172.21.15.42:0/3227332908 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4aa000fe60 con 0x55b4526e9390 2024-08-30T09:05:50.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.526+0000 7f4ac9d5a740 1 -- 172.21.15.42:0/3227332908 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f4aa000dcb0 con 0x55b45270a070 2024-08-30T09:05:50.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.526+0000 7f4ac1581640 1 -- 172.21.15.42:0/3227332908 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b45270a070 msgr2=0x55b452aac780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:05:50.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.526+0000 7f4ac1581640 1 --2- 172.21.15.42:0/3227332908 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b45270a070 0x55b452aac780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:50.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.526+0000 7f4ac1581640 1 -- 172.21.15.42:0/3227332908 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b4526e9390 msgr2=0x55b452abc660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:05:50.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.526+0000 7f4ac1581640 1 --2- 172.21.15.42:0/3227332908 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b4526e9390 0x55b452abc660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:50.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.526+0000 7f4ac1581640 1 -- 172.21.15.42:0/3227332908 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b452ab8f30 con 0x55b4526e11b0 2024-08-30T09:05:50.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.526+0000 7f4ac1581640 1 --2- 172.21.15.42:0/3227332908 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b4526e11b0 0x55b452ab9920 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f4ab80c7b80 tx=0x7f4ab8004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3464dd4fc268a22b server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:05:50.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.526+0000 7f4aa6ffd640 1 -- 172.21.15.42:0/3227332908 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ab80ca080 con 0x55b4526e11b0 2024-08-30T09:05:50.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.527+0000 7f4aa6ffd640 1 -- 172.21.15.42:0/3227332908 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ab80ca420 con 0x55b4526e11b0 2024-08-30T09:05:50.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.527+0000 7f4aa6ffd640 1 -- 172.21.15.42:0/3227332908 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ab80c60a0 con 0x55b4526e11b0 2024-08-30T09:05:50.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.528+0000 7f4ac9d5a740 1 -- 172.21.15.42:0/3227332908 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b4526e11b0 msgr2=0x55b452ab9920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:05:50.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.528+0000 7f4ac9d5a740 1 --2- 172.21.15.42:0/3227332908 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b4526e11b0 0x55b452ab9920 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f4ab80c7b80 tx=0x7f4ab8004610 comp rx=0 tx=0).stop 2024-08-30T09:05:50.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.528+0000 7f4ac9d5a740 1 -- 172.21.15.42:0/3227332908 shutdown_connections 2024-08-30T09:05:50.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.528+0000 7f4ac9d5a740 1 --2- 172.21.15.42:0/3227332908 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b45270a070 0x55b452aac780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:50.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.528+0000 7f4ac9d5a740 1 --2- 172.21.15.42:0/3227332908 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b4526e9390 0x55b452abc660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:50.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.528+0000 7f4ac9d5a740 1 --2- 172.21.15.42:0/3227332908 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b4526e11b0 0x55b452ab9920 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:50.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.528+0000 7f4ac9d5a740 1 -- 172.21.15.42:0/3227332908 >> 172.21.15.42:0/3227332908 conn(0x55b4526fe460 msgr2=0x55b452abbf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:05:50.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.528+0000 7f4ac9d5a740 1 -- 172.21.15.42:0/3227332908 shutdown_connections 2024-08-30T09:05:50.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.529+0000 7f4ac9d5a740 1 -- 172.21.15.42:0/3227332908 wait complete. 2024-08-30T09:05:50.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.529+0000 7f4ac9d5a740 1 -- start start 2024-08-30T09:05:50.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.529+0000 7f4ac9d5a740 1 Processor -- start 2024-08-30T09:05:50.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.530+0000 7f4ac9d5a740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b4526e11b0 0x55b452abf110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:50.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.530+0000 7f4ac9d5a740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b4526e9390 0x55b452abf650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:50.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.530+0000 7f4ac9d5a740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b45270a070 0x55b452b367c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:50.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.530+0000 7f4ac9d5a740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f4aa000e320 con 0x55b45270a070 2024-08-30T09:05:50.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.530+0000 7f4ac9d5a740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4aa000f790 con 0x55b4526e11b0 2024-08-30T09:05:50.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.530+0000 7f4ac9d5a740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f4aa000e9f0 con 0x55b4526e9390 2024-08-30T09:05:50.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.530+0000 7f4ac1d82640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b45270a070 0x55b452b367c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:05:50.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.530+0000 7f4ac9d5a740 10 client.0 _ll_register_callbacks cb 0x7f4ab800d890 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:05:50.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.530+0000 7f4ac1581640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b4526e11b0 0x55b452abf110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:05:50.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.530+0000 7f4ac0d80640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b4526e9390 0x55b452abf650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:05:50.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.530+0000 7f4ac1581640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b4526e11b0 0x55b452abf110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:40360/0 (socket says 172.21.15.42:40360) 2024-08-30T09:05:50.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.530+0000 7f4ac1581640 1 -- 172.21.15.42:0/2676498197 learned_addr learned my addr 172.21.15.42:0/2676498197 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:05:50.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.530+0000 7f4ac0d80640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b4526e9390 0x55b452abf650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:49060/0 (socket says 172.21.15.42:49060) 2024-08-30T09:05:50.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.530+0000 7f4ac1581640 1 -- 172.21.15.42:0/2676498197 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b4526e9390 msgr2=0x55b452abf650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:05:50.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.530+0000 7f4ac1581640 1 --2- 172.21.15.42:0/2676498197 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b4526e9390 0x55b452abf650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:50.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.530+0000 7f4ac1581640 1 -- 172.21.15.42:0/2676498197 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b45270a070 msgr2=0x55b452b367c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:05:50.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.530+0000 7f4ac1581640 1 --2- 172.21.15.42:0/2676498197 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b45270a070 0x55b452b367c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:50.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.530+0000 7f4ac1581640 1 -- 172.21.15.42:0/2676498197 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b452abfef0 con 0x55b4526e11b0 2024-08-30T09:05:50.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.531+0000 7f4ac1581640 1 --2- 172.21.15.42:0/2676498197 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b4526e11b0 0x55b452abf110 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f4ab80c6940 tx=0x7f4ab8004c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eaa2bff29b485669 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:05:50.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.531+0000 7f4aa4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-30T09:05:50.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.531+0000 7f4aa4ff9640 1 -- 172.21.15.42:0/2676498197 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ab80ca080 con 0x55b4526e11b0 2024-08-30T09:05:50.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.531+0000 7f4ac9d5a740 1 -- 172.21.15.42:0/2676498197 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ab80ca420 con 0x55b4526e11b0 2024-08-30T09:05:50.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.531+0000 7f4aa4ff9640 1 -- 172.21.15.42:0/2676498197 <== mon.2 v2:172.21.15.167:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f4ab80c6f00 con 0x55b4526e11b0 2024-08-30T09:05:50.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.531+0000 7f4aa4ff9640 1 -- 172.21.15.42:0/2676498197 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f4ab80ca910 con 0x55b4526e11b0 2024-08-30T09:05:50.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.531+0000 7f4ac9d5a740 10 client.26075 fetch_fsmap learned FSMap version 31 2024-08-30T09:05:50.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.531+0000 7f4ac9d5a740 1 -- 172.21.15.42:0/2676498197 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55b452abfef0 con 0x55b4526e11b0 2024-08-30T09:05:50.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.531+0000 7f4aa4ff9640 1 -- 172.21.15.42:0/2676498197 <== mon.2 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ab80ca420 con 0x55b4526e11b0 2024-08-30T09:05:50.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.532+0000 7f4aa4ff9640 1 -- 172.21.15.42:0/2676498197 <== mon.2 v2:172.21.15.167:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f4ab8076a50 con 0x55b4526e11b0 2024-08-30T09:05:50.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.532+0000 7f4ac9d5a740 10 client.26075 fetch_fsmap finished waiting for FSMap version 31 2024-08-30T09:05:50.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.532+0000 7f4ac9d5a740 10 client.26075 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-30T09:05:50.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.532+0000 7f4ac9d5a740 20 client.26075 populate_metadata read hostname 'smithi042' 2024-08-30T09:05:50.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.532+0000 7f4ac9d5a740 1 --2- 172.21.15.42:0/2676498197 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55b45269dda0 0x55b452b3a060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:50.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.532+0000 7f4ac9d5a740 4 client.26075 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-30T09:05:50.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.532+0000 7f4ac9d5a740 1 -- 172.21.15.42:0/2676498197 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b452a28580 con 0x55b45269dda0 2024-08-30T09:05:50.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.533+0000 7f4ac0d80640 1 --2- 172.21.15.42:0/2676498197 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55b45269dda0 0x55b452b3a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:05:50.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.535+0000 7f4ac0d80640 1 --2- 172.21.15.42:0/2676498197 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55b45269dda0 0x55b452b3a060 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8afea88afdde334c server_cookie=b30b8b3aa997f341 in_seq=0 out_seq=0 2024-08-30T09:05:50.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.535+0000 7f4aa4ff9640 10 client.26075 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:05:50.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.535+0000 7f4aa4ff9640 1 -- 172.21.15.42:0/2676498197 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+2274 (crc 0 0 0) 0x55b452a28580 con 0x55b45269dda0 2024-08-30T09:05:50.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.535+0000 7f4aa4ff9640 10 client.26075 handle_command_reply: tid=0 2024-08-30T09:05:50.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.535+0000 7f4ac9d5a740 1 client.26075 shutdown 2024-08-30T09:05:50.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.536+0000 7f4ac9d5a740 1 -- 172.21.15.42:0/2676498197 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b4526e11b0 msgr2=0x55b452abf110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:05:50.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.536+0000 7f4ac9d5a740 1 --2- 172.21.15.42:0/2676498197 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b4526e11b0 0x55b452abf110 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f4ab80c6940 tx=0x7f4ab8004c00 comp rx=0 tx=0).stop 2024-08-30T09:05:50.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.536+0000 7f4ac9d5a740 1 -- 172.21.15.42:0/2676498197 shutdown_connections 2024-08-30T09:05:50.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.536+0000 7f4ac9d5a740 1 --2- 172.21.15.42:0/2676498197 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55b45269dda0 0x55b452b3a060 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:50.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.536+0000 7f4ac9d5a740 1 --2- 172.21.15.42:0/2676498197 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b4526e9390 0x55b452abf650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:50.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.536+0000 7f4ac9d5a740 1 --2- 172.21.15.42:0/2676498197 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b45270a070 0x55b452b367c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:50.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.536+0000 7f4ac9d5a740 1 --2- 172.21.15.42:0/2676498197 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b4526e11b0 0x55b452abf110 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:50.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.536+0000 7f4aa4ff9640 0 client.26075 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:05:50.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.536+0000 7f4ac9d5a740 1 -- 172.21.15.42:0/2676498197 >> 172.21.15.42:0/2676498197 conn(0x55b4526fe460 msgr2=0x55b452abbee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:05:50.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.537+0000 7f4ac9d5a740 1 -- 172.21.15.42:0/2676498197 shutdown_connections 2024-08-30T09:05:50.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.537+0000 7f4ac9d5a740 1 -- 172.21.15.42:0/2676498197 wait complete. 2024-08-30T09:05:50.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.537+0000 7f4ac9d5a740 20 client.26075 trim_cache size 0 max 16384 2024-08-30T09:05:50.541 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-08-30T09:05:50.541 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "IDLE (waiting for more scrubs)", 2024-08-30T09:05:50.541 INFO:teuthology.orchestra.run.smithi042.stdout: "scrubs": { 2024-08-30T09:05:50.541 INFO:teuthology.orchestra.run.smithi042.stdout: "1af4252a-4a99-47e9-bbf0-becf5e573dd5": { 2024-08-30T09:05:50.541 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:05:50.541 INFO:teuthology.orchestra.run.smithi042.stdout: "tag": "1af4252a-4a99-47e9-bbf0-becf5e573dd5", 2024-08-30T09:05:50.541 INFO:teuthology.orchestra.run.smithi042.stdout: "options": "recursive,force" 2024-08-30T09:05:50.541 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:05:50.542 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:05:50.542 INFO:teuthology.orchestra.run.smithi042.stdout: "scrub_stats": { 2024-08-30T09:05:50.542 INFO:teuthology.orchestra.run.smithi042.stdout: "1af4252a-4a99-47e9-bbf0-becf5e573dd5": { 2024-08-30T09:05:50.542 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:05:48.565881+0000", 2024-08-30T09:05:50.542 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:05:50.542 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:05:50.542 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:05:50.542 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:05:50.542 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:05:50.542 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:05:50.542 INFO:teuthology.orchestra.run.smithi042.stdout: "347d05ed-4bd3-4590-b099-314f719aab39": { 2024-08-30T09:05:50.542 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:05:00.439721+0000", 2024-08-30T09:05:50.542 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:05:50.543 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:05:50.543 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:05:50.543 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:05:50.543 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:05:50.543 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:05:50.543 INFO:teuthology.orchestra.run.smithi042.stdout: "88cd7478-2b20-47de-ba51-256479749232": { 2024-08-30T09:05:50.543 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:04:08.922625+0000", 2024-08-30T09:05:50.543 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:05:50.543 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:05:50.543 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:05:50.543 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:05:50.543 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:05:50.543 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:05:50.543 INFO:teuthology.orchestra.run.smithi042.stdout: "749f4867-0b70-49e5-af2d-8362a240ebc3": { 2024-08-30T09:05:50.543 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:03:16.386554+0000", 2024-08-30T09:05:50.544 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:05:50.544 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:05:50.544 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:05:50.544 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:05:50.544 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:05:50.544 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:05:50.544 INFO:teuthology.orchestra.run.smithi042.stdout: "85134e4b-ab4a-4885-b07e-a3ad01eaf13d": { 2024-08-30T09:05:50.544 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:01:27.738462+0000", 2024-08-30T09:05:50.544 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:05:50.544 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:05:50.544 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:05:50.544 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:05:50.544 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:05:50.544 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:05:50.544 INFO:teuthology.orchestra.run.smithi042.stdout: "c827ad7a-44ee-4e1d-9bb3-51fb7ab00c73": { 2024-08-30T09:05:50.545 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:02:21.686517+0000", 2024-08-30T09:05:50.545 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:05:50.545 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:05:50.545 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:05:50.545 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:05:50.545 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:05:50.545 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:05:50.545 INFO:teuthology.orchestra.run.smithi042.stdout: "5e2ed104-0625-478a-a78b-584e7f4c2817": { 2024-08-30T09:05:50.545 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:00:33.928400+0000", 2024-08-30T09:05:50.545 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:05:50.545 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:05:50.545 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:05:50.545 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:05:50.545 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:05:50.546 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:05:50.546 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:05:50.546 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-08-30T09:05:50.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.538+0000 7f4ac37e5640 1 -- 172.21.15.42:0/3474482618 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f4aa80b4c60 msgr2=0x7f4aa80b7170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:05:50.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.538+0000 7f4ac37e5640 1 --2- 172.21.15.42:0/3474482618 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f4aa80b4c60 0x7f4aa80b7170 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f4abc071260 tx=0x7f4ab40094f0 comp rx=0 tx=0).stop 2024-08-30T09:05:50.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.538+0000 7f4ac37e5640 1 -- 172.21.15.42:0/3474482618 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4abc10b890 msgr2=0x7f4abc080420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:05:50.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.538+0000 7f4ac37e5640 1 --2- 172.21.15.42:0/3474482618 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4abc10b890 0x7f4abc080420 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f4ab8006010 tx=0x7f4ab800d3b0 comp rx=0 tx=0).stop 2024-08-30T09:05:50.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.539+0000 7f4ac37e5640 1 -- 172.21.15.42:0/3474482618 shutdown_connections 2024-08-30T09:05:50.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.539+0000 7f4ac37e5640 1 --2- 172.21.15.42:0/3474482618 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f4aa80b4c60 0x7f4aa80b7170 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:50.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.539+0000 7f4ac37e5640 1 --2- 172.21.15.42:0/3474482618 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4abc110230 0x7f4abc080ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:50.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.539+0000 7f4ac37e5640 1 --2- 172.21.15.42:0/3474482618 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4abc10f8e0 0x7f4abc080960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:50.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.539+0000 7f4ac37e5640 1 --2- 172.21.15.42:0/3474482618 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4abc10b890 0x7f4abc080420 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:50.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.539+0000 7f4ac37e5640 1 -- 172.21.15.42:0/3474482618 >> 172.21.15.42:0/3474482618 conn(0x7f4abc070c60 msgr2=0x7f4abc10d860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:05:50.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.539+0000 7f4ac37e5640 1 -- 172.21.15.42:0/3474482618 shutdown_connections 2024-08-30T09:05:50.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:05:50.539+0000 7f4ac37e5640 1 -- 172.21.15.42:0/3474482618 wait complete. 2024-08-30T09:05:50.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:05:50 smithi167 ceph-mon[22063]: scrub summary: active paths [/] 2024-08-30T09:05:50.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:05:50 smithi167 ceph-mon[22063]: scrub summary: idle+waiting paths [/] 2024-08-30T09:05:50.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:05:50 smithi167 ceph-mon[22063]: pgmap v525: 129 pgs: 129 active+clean; 1.2 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 10 MiB/s wr, 570 op/s 2024-08-30T09:05:50.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:05:50 smithi167 ceph-mon[22063]: scrub summary: idle+waiting paths [/] 2024-08-30T09:05:50.953 INFO:teuthology.orchestra.run.smithi042.stderr:progress: 305.0 s, 655.8 tps, lat 155.760 ms stddev 240.607 2024-08-30T09:05:51.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:05:51 smithi167 ceph-mon[22063]: scrub summary: idle+waiting paths [/] 2024-08-30T09:05:51.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:05:51 smithi042 ceph-mon[18652]: scrub summary: idle+waiting paths [/] 2024-08-30T09:05:51.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:05:51 smithi190 ceph-mon[22001]: scrub summary: idle+waiting paths [/] 2024-08-30T09:05:51.692 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.688+0000 7fabca08b640 1 -- 172.21.15.42:0/3035861736 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fabc410cb50 msgr2=0x7fabc410f030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:05:51.692 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.688+0000 7fabca08b640 1 --2- 172.21.15.42:0/3035861736 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fabc410cb50 0x7fabc410f030 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fabbc009ca0 tx=0x7fabbc02a8f0 comp rx=0 tx=0).stop 2024-08-30T09:05:51.692 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.689+0000 7fabca08b640 1 -- 172.21.15.42:0/3035861736 shutdown_connections 2024-08-30T09:05:51.692 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.689+0000 7fabca08b640 1 --2- 172.21.15.42:0/3035861736 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fabc410f770 0x7fabc411a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:51.692 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.689+0000 7fabca08b640 1 --2- 172.21.15.42:0/3035861736 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fabc410cb50 0x7fabc410f030 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:51.692 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.689+0000 7fabca08b640 1 --2- 172.21.15.42:0/3035861736 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fabc410a160 0x7fabc410c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:51.692 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.689+0000 7fabca08b640 1 -- 172.21.15.42:0/3035861736 >> 172.21.15.42:0/3035861736 conn(0x7fabc40705b0 msgr2=0x7fabc4072a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:05:51.697 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.694+0000 7fabca08b640 1 -- 172.21.15.42:0/3035861736 shutdown_connections 2024-08-30T09:05:51.701 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.697+0000 7fabca08b640 1 -- 172.21.15.42:0/3035861736 wait complete. 2024-08-30T09:05:51.701 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.697+0000 7fabca08b640 1 Processor -- start 2024-08-30T09:05:51.707 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.702+0000 7fabca08b640 1 -- start start 2024-08-30T09:05:51.707 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.702+0000 7fabca08b640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fabc410a160 0x7fabc41176c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:51.707 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.702+0000 7fabca08b640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fabc410cb50 0x7fabc4117d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:51.707 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.702+0000 7fabca08b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fabc410f770 0x7fabc41188e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:51.707 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.702+0000 7fabca08b640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fabc411def0 con 0x7fabc410a160 2024-08-30T09:05:51.707 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.702+0000 7fabca08b640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fabc411dd70 con 0x7fabc410cb50 2024-08-30T09:05:51.707 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.702+0000 7fabca08b640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fabc411e070 con 0x7fabc410f770 2024-08-30T09:05:51.708 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.703+0000 7fabc3fff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fabc410f770 0x7fabc41188e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:05:51.714 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.706+0000 7fabc2ffd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fabc410cb50 0x7fabc4117d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:05:51.714 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.706+0000 7fabc37fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fabc410a160 0x7fabc41176c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:05:51.714 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.706+0000 7fabc37fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fabc410a160 0x7fabc41176c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60636/0 (socket says 172.21.15.42:60636) 2024-08-30T09:05:51.714 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.706+0000 7fabc37fe640 1 -- 172.21.15.42:0/3555654152 learned_addr learned my addr 172.21.15.42:0/3555654152 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:05:51.714 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.711+0000 7fabc3fff640 1 -- 172.21.15.42:0/3555654152 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fabc410cb50 msgr2=0x7fabc4117d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:05:51.714 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.711+0000 7fabc3fff640 1 --2- 172.21.15.42:0/3555654152 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fabc410cb50 0x7fabc4117d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:51.715 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.711+0000 7fabc3fff640 1 -- 172.21.15.42:0/3555654152 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fabc410a160 msgr2=0x7fabc41176c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:05:51.715 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.711+0000 7fabc3fff640 1 --2- 172.21.15.42:0/3555654152 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fabc410a160 0x7fabc41176c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:51.715 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.711+0000 7fabc3fff640 1 -- 172.21.15.42:0/3555654152 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabc4119100 con 0x7fabc410f770 2024-08-30T09:05:51.716 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.715+0000 7fabc3fff640 1 --2- 172.21.15.42:0/3555654152 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fabc410f770 0x7fabc41188e0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fabb400e970 tx=0x7fabb400ee30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:05:51.729 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.717+0000 7fabc0ff9640 1 -- 172.21.15.42:0/3555654152 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabb400ccc0 con 0x7fabc410f770 2024-08-30T09:05:51.729 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.717+0000 7fabca08b640 1 -- 172.21.15.42:0/3555654152 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabc41193f0 con 0x7fabc410f770 2024-08-30T09:05:51.729 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.717+0000 7fabca08b640 1 -- 172.21.15.42:0/3555654152 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabc4119ab0 con 0x7fabc410f770 2024-08-30T09:05:51.729 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.717+0000 7fabc0ff9640 1 -- 172.21.15.42:0/3555654152 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fabb40035f0 con 0x7fabc410f770 2024-08-30T09:05:51.729 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.717+0000 7fabc0ff9640 1 -- 172.21.15.42:0/3555654152 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabb400ce60 con 0x7fabc410f770 2024-08-30T09:05:51.738 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.725+0000 7fabd0ff6740 1 Processor -- start 2024-08-30T09:05:51.738 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.726+0000 7fabd0ff6740 1 -- start start 2024-08-30T09:05:51.738 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.726+0000 7fabd0ff6740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a5b3f6f1b0 0x55a5b4314f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:51.738 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.726+0000 7fabd0ff6740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a5b3f462f0 0x55a5b43155d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:51.738 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.726+0000 7fabd0ff6740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a5b3f4e4d0 0x55a5b4316120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:51.738 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.726+0000 7fabd0ff6740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55a5b4288000 con 0x55a5b3f462f0 2024-08-30T09:05:51.738 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.726+0000 7fabd0ff6740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55a5b4277590 con 0x55a5b3f6f1b0 2024-08-30T09:05:51.738 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.726+0000 7fabd0ff6740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55a5b427b540 con 0x55a5b3f4e4d0 2024-08-30T09:05:51.738 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.726+0000 7fabc2ffd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a5b3f6f1b0 0x55a5b4314f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:05:51.738 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.726+0000 7fabc2ffd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a5b3f6f1b0 0x55a5b4314f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:40392/0 (socket says 172.21.15.42:40392) 2024-08-30T09:05:51.738 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.726+0000 7fabc2ffd640 1 -- 172.21.15.42:0/515239792 learned_addr learned my addr 172.21.15.42:0/515239792 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:05:51.738 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.729+0000 7fabc37fe640 1 --2- 172.21.15.42:0/515239792 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a5b3f462f0 0x55a5b43155d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:05:51.741 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.739+0000 7fabc2ffd640 1 --2- 172.21.15.42:0/515239792 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a5b3f4e4d0 0x55a5b4316120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:05:51.742 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.739+0000 7fabc2ffd640 1 -- 172.21.15.42:0/515239792 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a5b3f4e4d0 msgr2=0x55a5b4316120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:05:51.742 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.739+0000 7fabc2ffd640 1 --2- 172.21.15.42:0/515239792 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a5b3f4e4d0 0x55a5b4316120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:51.742 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.740+0000 7fabc2ffd640 1 -- 172.21.15.42:0/515239792 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a5b3f462f0 msgr2=0x55a5b43155d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:05:51.742 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.740+0000 7fabc2ffd640 1 --2- 172.21.15.42:0/515239792 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a5b3f462f0 0x55a5b43155d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:51.742 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.740+0000 7fabc2ffd640 1 -- 172.21.15.42:0/515239792 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a5b3fda790 con 0x55a5b3f6f1b0 2024-08-30T09:05:51.742 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.740+0000 7fabc2ffd640 1 --2- 172.21.15.42:0/515239792 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a5b3f6f1b0 0x55a5b4314f60 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fabbc009bf0 tx=0x7fabbc004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6dd6ba54a181db server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:05:51.745 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.741+0000 7fabc37fe640 1 --2- 172.21.15.42:0/515239792 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a5b3f462f0 0x55a5b43155d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-30T09:05:51.745 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.742+0000 7fabc0ff9640 1 -- 172.21.15.42:0/3555654152 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7fabb4004020 con 0x7fabc410f770 2024-08-30T09:05:51.746 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.742+0000 7fabb9ffb640 1 -- 172.21.15.42:0/515239792 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabbc031590 con 0x55a5b3f6f1b0 2024-08-30T09:05:51.751 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.742+0000 7fabb9ffb640 1 -- 172.21.15.42:0/515239792 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fabbc036070 con 0x55a5b3f6f1b0 2024-08-30T09:05:51.751 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.743+0000 7fabc0ff9640 1 --2- 172.21.15.42:0/3555654152 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7faba80b4d40 0x7faba80b7250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:51.751 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.743+0000 7fabd0ff6740 1 -- 172.21.15.42:0/515239792 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a5b3f6f1b0 msgr2=0x55a5b4314f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:05:51.751 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.743+0000 7fabd0ff6740 1 --2- 172.21.15.42:0/515239792 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a5b3f6f1b0 0x55a5b4314f60 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fabbc009bf0 tx=0x7fabbc004a40 comp rx=0 tx=0).stop 2024-08-30T09:05:51.751 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.743+0000 7fabc0ff9640 1 -- 172.21.15.42:0/3555654152 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7fabb40e5560 con 0x7fabc410f770 2024-08-30T09:05:51.751 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.746+0000 7fabc37fe640 1 --2- 172.21.15.42:0/3555654152 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7faba80b4d40 0x7faba80b7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:05:51.751 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.748+0000 7fabd0ff6740 1 -- 172.21.15.42:0/515239792 shutdown_connections 2024-08-30T09:05:51.752 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.748+0000 7fabd0ff6740 1 --2- 172.21.15.42:0/515239792 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a5b3f4e4d0 0x55a5b4316120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:51.752 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.748+0000 7fabd0ff6740 1 --2- 172.21.15.42:0/515239792 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a5b3f462f0 0x55a5b43155d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:51.752 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.748+0000 7fabd0ff6740 1 --2- 172.21.15.42:0/515239792 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a5b3f6f1b0 0x55a5b4314f60 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:51.752 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.748+0000 7fabd0ff6740 1 -- 172.21.15.42:0/515239792 >> 172.21.15.42:0/515239792 conn(0x55a5b3f63300 msgr2=0x55a5b43109f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:05:51.756 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.754+0000 7fabd0ff6740 1 -- 172.21.15.42:0/515239792 shutdown_connections 2024-08-30T09:05:51.762 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.758+0000 7fabd0ff6740 1 -- 172.21.15.42:0/515239792 wait complete. 2024-08-30T09:05:51.762 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.758+0000 7fabd0ff6740 1 -- start start 2024-08-30T09:05:51.762 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.758+0000 7fabd0ff6740 1 Processor -- start 2024-08-30T09:05:51.762 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.759+0000 7fabc37fe640 1 --2- 172.21.15.42:0/3555654152 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7faba80b4d40 0x7faba80b7250 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fabb0009d80 tx=0x7fabb0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:05:51.762 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.759+0000 7fabd0ff6740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a5b3f462f0 0x55a5b43409c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:51.762 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.759+0000 7fabd0ff6740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a5b3f4e4d0 0x55a5b4341080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:51.763 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.759+0000 7fabd0ff6740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a5b3f6f1b0 0x55a5b4341bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:51.763 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.759+0000 7fabd0ff6740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55a5b41b0ab0 con 0x55a5b3f6f1b0 2024-08-30T09:05:51.763 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.759+0000 7fabd0ff6740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55a5b41fe680 con 0x55a5b3f4e4d0 2024-08-30T09:05:51.763 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.759+0000 7fabd0ff6740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55a5b4316c30 con 0x55a5b3f462f0 2024-08-30T09:05:51.763 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.759+0000 7fabd0ff6740 10 client.0 _ll_register_callbacks cb 0x55a5b3f86a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:05:51.763 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.759+0000 7fabc2ffd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a5b3f4e4d0 0x55a5b4341080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:05:51.763 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.759+0000 7fabc2ffd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a5b3f4e4d0 0x55a5b4341080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:40394/0 (socket says 172.21.15.42:40394) 2024-08-30T09:05:51.763 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.759+0000 7fabc2ffd640 1 -- 172.21.15.42:0/3190749862 learned_addr learned my addr 172.21.15.42:0/3190749862 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:05:51.765 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.761+0000 7fabc2ffd640 1 -- 172.21.15.42:0/3190749862 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a5b3f462f0 msgr2=0x55a5b43409c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-30T09:05:51.765 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.761+0000 7fabc2ffd640 1 --2- 172.21.15.42:0/3190749862 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a5b3f462f0 0x55a5b43409c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:51.765 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.761+0000 7fabc2ffd640 1 -- 172.21.15.42:0/3190749862 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a5b3f6f1b0 msgr2=0x55a5b4341bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:05:51.765 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.761+0000 7fabc2ffd640 1 --2- 172.21.15.42:0/3190749862 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a5b3f6f1b0 0x55a5b4341bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:51.765 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.761+0000 7fabc2ffd640 1 -- 172.21.15.42:0/3190749862 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a5b4053620 con 0x55a5b3f4e4d0 2024-08-30T09:05:51.770 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.764+0000 7fabc2ffd640 1 --2- 172.21.15.42:0/3190749862 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a5b3f4e4d0 0x55a5b4341080 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fabbc02aee0 tx=0x7fabbc0095e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8d219f8a005e1833 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:05:51.770 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.768+0000 7faba6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-30T09:05:51.770 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.768+0000 7faba6ffd640 1 -- 172.21.15.42:0/3190749862 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabbc058060 con 0x55a5b3f4e4d0 2024-08-30T09:05:51.770 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.768+0000 7fabd0ff6740 1 -- 172.21.15.42:0/3190749862 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fabbc036070 con 0x55a5b3f4e4d0 2024-08-30T09:05:51.771 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.768+0000 7faba6ffd640 1 -- 172.21.15.42:0/3190749862 <== mon.2 v2:172.21.15.167:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7fabbc004780 con 0x55a5b3f4e4d0 2024-08-30T09:05:51.771 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.768+0000 7faba6ffd640 1 -- 172.21.15.42:0/3190749862 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fabbc05f050 con 0x55a5b3f4e4d0 2024-08-30T09:05:51.771 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.768+0000 7fabd0ff6740 10 client.26087 fetch_fsmap learned FSMap version 31 2024-08-30T09:05:51.771 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.768+0000 7fabd0ff6740 1 -- 172.21.15.42:0/3190749862 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55a5b434aab0 con 0x55a5b3f4e4d0 2024-08-30T09:05:51.772 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.769+0000 7faba6ffd640 1 -- 172.21.15.42:0/3190749862 <== mon.2 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fabbc036070 con 0x55a5b3f4e4d0 2024-08-30T09:05:51.772 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.769+0000 7faba6ffd640 1 -- 172.21.15.42:0/3190749862 <== mon.2 v2:172.21.15.167:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7fabbc030c40 con 0x55a5b3f4e4d0 2024-08-30T09:05:51.772 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.769+0000 7fabd0ff6740 10 client.26087 fetch_fsmap finished waiting for FSMap version 31 2024-08-30T09:05:51.772 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.769+0000 7fabd0ff6740 10 client.26087 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-30T09:05:51.772 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.769+0000 7fabd0ff6740 20 client.26087 populate_metadata read hostname 'smithi042' 2024-08-30T09:05:51.772 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.770+0000 7fabd0ff6740 1 --2- 172.21.15.42:0/3190749862 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55a5b3f2dd80 0x55a5b434e400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:51.772 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.770+0000 7fabd0ff6740 4 client.26087 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-30T09:05:51.772 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.770+0000 7fabd0ff6740 1 -- 172.21.15.42:0/3190749862 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a5b4289f50 con 0x55a5b3f2dd80 2024-08-30T09:05:51.774 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.773+0000 7fabc37fe640 1 --2- 172.21.15.42:0/3190749862 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55a5b3f2dd80 0x55a5b434e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:05:51.781 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.777+0000 7fabc37fe640 1 --2- 172.21.15.42:0/3190749862 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55a5b3f2dd80 0x55a5b434e400 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=358b3a4a3913b342 server_cookie=474e1ab0143d169d in_seq=0 out_seq=0 2024-08-30T09:05:51.781 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.780+0000 7faba6ffd640 10 client.26087 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:05:51.783 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.781+0000 7faba6ffd640 1 -- 172.21.15.42:0/3190749862 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a5b4289f50 con 0x55a5b3f2dd80 2024-08-30T09:05:51.783 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.781+0000 7faba6ffd640 10 client.26087 handle_command_reply: tid=0 2024-08-30T09:05:51.783 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.781+0000 7fabd0ff6740 1 client.26087 shutdown 2024-08-30T09:05:51.789 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.786+0000 7fabd0ff6740 1 -- 172.21.15.42:0/3190749862 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a5b3f4e4d0 msgr2=0x55a5b4341080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:05:51.790 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.786+0000 7fabd0ff6740 1 --2- 172.21.15.42:0/3190749862 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a5b3f4e4d0 0x55a5b4341080 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fabbc02aee0 tx=0x7fabbc0095e0 comp rx=0 tx=0).stop 2024-08-30T09:05:51.790 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.787+0000 7fabd0ff6740 1 -- 172.21.15.42:0/3190749862 shutdown_connections 2024-08-30T09:05:51.790 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.787+0000 7fabd0ff6740 1 --2- 172.21.15.42:0/3190749862 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55a5b3f2dd80 0x55a5b434e400 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:51.790 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.787+0000 7fabd0ff6740 1 --2- 172.21.15.42:0/3190749862 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a5b3f6f1b0 0x55a5b4341bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:51.790 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.787+0000 7fabd0ff6740 1 --2- 172.21.15.42:0/3190749862 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a5b3f4e4d0 0x55a5b4341080 secure :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fabbc02aee0 tx=0x7fabbc0095e0 comp rx=0 tx=0).stop 2024-08-30T09:05:51.790 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.787+0000 7fabd0ff6740 1 --2- 172.21.15.42:0/3190749862 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a5b3f462f0 0x55a5b43409c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:51.790 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.787+0000 7fabd0ff6740 1 -- 172.21.15.42:0/3190749862 >> 172.21.15.42:0/3190749862 conn(0x55a5b3f63300 msgr2=0x55a5b430ebb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:05:51.790 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.787+0000 7faba6ffd640 0 client.26087 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:05:51.791 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.789+0000 7fabd0ff6740 1 -- 172.21.15.42:0/3190749862 shutdown_connections 2024-08-30T09:05:51.794 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.790+0000 7fabd0ff6740 1 -- 172.21.15.42:0/3190749862 wait complete. 2024-08-30T09:05:51.794 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.792+0000 7fabd0ff6740 20 client.26087 trim_cache size 0 max 16384 2024-08-30T09:05:51.833 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.829+0000 7fabd0ff6740 1 Processor -- start 2024-08-30T09:05:51.833 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.829+0000 7fabd0ff6740 1 -- start start 2024-08-30T09:05:51.833 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.829+0000 7fabd0ff6740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a5b3f462f0 0x55a5b43409c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:51.833 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.829+0000 7fabd0ff6740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a5b3f6f1b0 0x55a5b4340f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:51.833 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.829+0000 7fabd0ff6740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a5b3f031c0 0x55a5b433b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:51.833 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.830+0000 7fabc37fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a5b3f462f0 0x55a5b43409c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:05:51.833 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.830+0000 7fabc37fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a5b3f462f0 0x55a5b43409c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:40410/0 (socket says 172.21.15.42:40410) 2024-08-30T09:05:51.833 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.830+0000 7fabc37fe640 1 -- 172.21.15.42:0/3072152857 learned_addr learned my addr 172.21.15.42:0/3072152857 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:05:51.834 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.830+0000 7fabc3fff640 1 --2- 172.21.15.42:0/3072152857 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a5b3f031c0 0x55a5b433b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:05:51.834 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.830+0000 7fabd0ff6740 1 -- 172.21.15.42:0/3072152857 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fabac0125f0 con 0x55a5b3f031c0 2024-08-30T09:05:51.834 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.830+0000 7fabd0ff6740 1 -- 172.21.15.42:0/3072152857 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fabac00fec0 con 0x55a5b3f462f0 2024-08-30T09:05:51.834 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.830+0000 7fabd0ff6740 1 -- 172.21.15.42:0/3072152857 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fabac00dd10 con 0x55a5b3f6f1b0 2024-08-30T09:05:51.834 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.830+0000 7fabc2ffd640 1 --2- 172.21.15.42:0/3072152857 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a5b3f6f1b0 0x55a5b4340f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:05:51.834 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.830+0000 7fabc37fe640 1 -- 172.21.15.42:0/3072152857 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a5b3f6f1b0 msgr2=0x55a5b4340f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:05:51.834 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.830+0000 7fabc37fe640 1 --2- 172.21.15.42:0/3072152857 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a5b3f6f1b0 0x55a5b4340f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:51.834 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.830+0000 7fabc37fe640 1 -- 172.21.15.42:0/3072152857 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a5b3f031c0 msgr2=0x55a5b433b6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:05:51.834 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.830+0000 7fabc37fe640 1 --2- 172.21.15.42:0/3072152857 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a5b3f031c0 0x55a5b433b6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:51.834 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.830+0000 7fabc37fe640 1 -- 172.21.15.42:0/3072152857 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a5b4315df0 con 0x55a5b3f462f0 2024-08-30T09:05:51.834 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.830+0000 7fabc2ffd640 1 --2- 172.21.15.42:0/3072152857 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a5b3f6f1b0 0x55a5b4340f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-30T09:05:51.834 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.831+0000 7fabc37fe640 1 --2- 172.21.15.42:0/3072152857 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a5b3f462f0 0x55a5b43409c0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fabb0009ef0 tx=0x7fabb0016000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fe828c765fa3223 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:05:51.834 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.831+0000 7fabb97fa640 1 -- 172.21.15.42:0/3072152857 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabb0005220 con 0x55a5b3f462f0 2024-08-30T09:05:51.834 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.831+0000 7fabb97fa640 1 -- 172.21.15.42:0/3072152857 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fabb0004c50 con 0x55a5b3f462f0 2024-08-30T09:05:51.835 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.831+0000 7fabb97fa640 1 -- 172.21.15.42:0/3072152857 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabb001d9b0 con 0x55a5b3f462f0 2024-08-30T09:05:51.835 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.832+0000 7fabd0ff6740 1 -- 172.21.15.42:0/3072152857 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a5b3f462f0 msgr2=0x55a5b43409c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:05:51.835 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.832+0000 7fabd0ff6740 1 --2- 172.21.15.42:0/3072152857 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a5b3f462f0 0x55a5b43409c0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fabb0009ef0 tx=0x7fabb0016000 comp rx=0 tx=0).stop 2024-08-30T09:05:51.837 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.835+0000 7fabd0ff6740 1 -- 172.21.15.42:0/3072152857 shutdown_connections 2024-08-30T09:05:51.837 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.835+0000 7fabd0ff6740 1 --2- 172.21.15.42:0/3072152857 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a5b3f6f1b0 0x55a5b4340f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:51.837 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.835+0000 7fabd0ff6740 1 --2- 172.21.15.42:0/3072152857 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a5b3f031c0 0x55a5b433b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:51.837 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.835+0000 7fabd0ff6740 1 --2- 172.21.15.42:0/3072152857 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a5b3f462f0 0x55a5b43409c0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:51.837 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.835+0000 7fabd0ff6740 1 -- 172.21.15.42:0/3072152857 >> 172.21.15.42:0/3072152857 conn(0x55a5b3f63300 msgr2=0x55a5b433a970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:05:51.839 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.837+0000 7fabd0ff6740 1 -- 172.21.15.42:0/3072152857 shutdown_connections 2024-08-30T09:05:51.840 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.837+0000 7fabd0ff6740 1 -- 172.21.15.42:0/3072152857 wait complete. 2024-08-30T09:05:51.840 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.838+0000 7fabd0ff6740 1 -- start start 2024-08-30T09:05:51.840 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.838+0000 7fabd0ff6740 1 Processor -- start 2024-08-30T09:05:51.841 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.839+0000 7fabd0ff6740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a5b3f031c0 0x55a5b43439b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:51.842 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.839+0000 7fabd0ff6740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a5b3f462f0 0x55a5b4344070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:51.842 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.840+0000 7fabc37fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a5b3f031c0 0x55a5b43439b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:05:51.842 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.840+0000 7fabc37fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a5b3f031c0 0x55a5b43439b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:40418/0 (socket says 172.21.15.42:40418) 2024-08-30T09:05:51.842 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.840+0000 7fabc37fe640 1 -- 172.21.15.42:0/2269116240 learned_addr learned my addr 172.21.15.42:0/2269116240 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:05:51.842 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.840+0000 7fabd0ff6740 1 --2- 172.21.15.42:0/2269116240 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a5b3f6f1b0 0x55a5b4344be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:51.843 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.840+0000 7fabd0ff6740 1 -- 172.21.15.42:0/2269116240 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fabac00e380 con 0x55a5b3f462f0 2024-08-30T09:05:51.843 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.840+0000 7fabc3fff640 1 --2- 172.21.15.42:0/2269116240 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a5b3f6f1b0 0x55a5b4344be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:05:51.843 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.840+0000 7fabd0ff6740 1 -- 172.21.15.42:0/2269116240 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fabac00f7f0 con 0x55a5b3f031c0 2024-08-30T09:05:51.843 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.840+0000 7fabd0ff6740 1 -- 172.21.15.42:0/2269116240 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fabac00ea50 con 0x55a5b3f6f1b0 2024-08-30T09:05:51.843 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.841+0000 7fabd0ff6740 10 client.0 _ll_register_callbacks cb 0x7fabbc009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:05:51.843 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.841+0000 7fabc37fe640 1 -- 172.21.15.42:0/2269116240 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a5b3f6f1b0 msgr2=0x55a5b4344be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:05:51.844 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.841+0000 7fabc37fe640 1 --2- 172.21.15.42:0/2269116240 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a5b3f6f1b0 0x55a5b4344be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:51.844 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.841+0000 7fabc37fe640 1 -- 172.21.15.42:0/2269116240 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a5b3f462f0 msgr2=0x55a5b4344070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:05:51.844 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.841+0000 7fabc37fe640 1 --2- 172.21.15.42:0/2269116240 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a5b3f462f0 0x55a5b4344070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:51.844 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.841+0000 7fabc37fe640 1 -- 172.21.15.42:0/2269116240 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a5b4345340 con 0x55a5b3f031c0 2024-08-30T09:05:51.844 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.841+0000 7fabc37fe640 1 --2- 172.21.15.42:0/2269116240 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a5b3f031c0 0x55a5b43439b0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fabb0009ec0 tx=0x7fabb0004b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e77bf06655db6f10 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:05:51.845 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.841+0000 7faba6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-30T09:05:51.845 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.842+0000 7faba6ffd640 1 -- 172.21.15.42:0/2269116240 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabb0034020 con 0x55a5b3f031c0 2024-08-30T09:05:51.846 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.844+0000 7fabd0ff6740 1 -- 172.21.15.42:0/2269116240 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fabb0004c50 con 0x55a5b3f031c0 2024-08-30T09:05:51.846 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.844+0000 7faba6ffd640 1 -- 172.21.15.42:0/2269116240 <== mon.2 v2:172.21.15.167:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7fabb001db40 con 0x55a5b3f031c0 2024-08-30T09:05:51.849 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.847+0000 7faba6ffd640 1 -- 172.21.15.42:0/2269116240 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fabb003f050 con 0x55a5b3f031c0 2024-08-30T09:05:51.849 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.847+0000 7fabd0ff6740 10 client.26093 fetch_fsmap learned FSMap version 31 2024-08-30T09:05:51.849 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.847+0000 7fabd0ff6740 1 -- 172.21.15.42:0/2269116240 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({fsmap=31}) -- 0x55a5b4348590 con 0x55a5b3f031c0 2024-08-30T09:05:51.853 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.851+0000 7faba6ffd640 1 -- 172.21.15.42:0/2269116240 <== mon.2 v2:172.21.15.167:3300/0 4 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7fabb001a040 con 0x55a5b3f031c0 2024-08-30T09:05:51.853 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.851+0000 7fabd0ff6740 10 client.26093 fetch_fsmap finished waiting for FSMap version 31 2024-08-30T09:05:51.853 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.851+0000 7fabd0ff6740 10 client.26093 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-30T09:05:51.853 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.851+0000 7fabd0ff6740 20 client.26093 populate_metadata read hostname 'smithi042' 2024-08-30T09:05:51.853 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.851+0000 7fabd0ff6740 1 --2- 172.21.15.42:0/2269116240 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55a5b43b8e80 0x55a5b43bb2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:05:51.853 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.851+0000 7fabd0ff6740 4 client.26093 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-30T09:05:51.854 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.851+0000 7fabd0ff6740 1 -- 172.21.15.42:0/2269116240 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7fabb0004c50 con 0x55a5b43b8e80 2024-08-30T09:05:51.854 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.852+0000 7fabc2ffd640 1 --2- 172.21.15.42:0/2269116240 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55a5b43b8e80 0x55a5b43bb2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:05:51.864 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.862+0000 7fabc2ffd640 1 --2- 172.21.15.42:0/2269116240 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55a5b43b8e80 0x55a5b43bb2a0 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=17d8d42ad941fe7 server_cookie=36431eda8831759e in_seq=0 out_seq=0 2024-08-30T09:05:51.864 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.862+0000 7faba6ffd640 10 client.26093 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:05:51.978 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.975+0000 7faba6ffd640 1 -- 172.21.15.42:0/2269116240 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7fabb0004c50 con 0x55a5b43b8e80 2024-08-30T09:05:51.978 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.975+0000 7faba6ffd640 10 client.26093 handle_command_reply: tid=0 2024-08-30T09:05:51.978 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.975+0000 7fabd0ff6740 1 client.26093 shutdown 2024-08-30T09:05:51.978 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.975+0000 7fabd0ff6740 1 -- 172.21.15.42:0/2269116240 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a5b3f031c0 msgr2=0x55a5b43439b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:05:51.978 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.975+0000 7fabd0ff6740 1 --2- 172.21.15.42:0/2269116240 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a5b3f031c0 0x55a5b43439b0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fabb0009ec0 tx=0x7fabb0004b60 comp rx=0 tx=0).stop 2024-08-30T09:05:51.978 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.975+0000 7fabd0ff6740 1 -- 172.21.15.42:0/2269116240 shutdown_connections 2024-08-30T09:05:51.978 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.975+0000 7fabd0ff6740 1 --2- 172.21.15.42:0/2269116240 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55a5b43b8e80 0x55a5b43bb2a0 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:51.978 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.975+0000 7fabd0ff6740 1 --2- 172.21.15.42:0/2269116240 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a5b3f6f1b0 0x55a5b4344be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:51.979 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.975+0000 7fabd0ff6740 1 --2- 172.21.15.42:0/2269116240 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a5b3f462f0 0x55a5b4344070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:51.979 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.975+0000 7faba6ffd640 0 client.26093 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:05:51.979 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.975+0000 7fabd0ff6740 1 --2- 172.21.15.42:0/2269116240 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a5b3f031c0 0x55a5b43439b0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:51.979 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.975+0000 7fabd0ff6740 1 -- 172.21.15.42:0/2269116240 >> 172.21.15.42:0/2269116240 conn(0x55a5b3f63300 msgr2=0x55a5b433b6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:05:51.979 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.976+0000 7fabd0ff6740 1 -- 172.21.15.42:0/2269116240 shutdown_connections 2024-08-30T09:05:51.979 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.976+0000 7fabd0ff6740 1 -- 172.21.15.42:0/2269116240 wait complete. 2024-08-30T09:05:51.979 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.976+0000 7fabd0ff6740 20 client.26093 trim_cache size 0 max 16384 2024-08-30T09:05:51.979 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-08-30T09:05:51.979 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2024-08-30T09:05:51.979 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-08-30T09:05:51.980 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-08-30T09:05:51.980 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.976+0000 7fabca08b640 1 -- 172.21.15.42:0/3555654152 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7faba80b4d40 msgr2=0x7faba80b7250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:05:51.981 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.976+0000 7fabca08b640 1 --2- 172.21.15.42:0/3555654152 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7faba80b4d40 0x7faba80b7250 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fabb0009d80 tx=0x7fabb0005f90 comp rx=0 tx=0).stop 2024-08-30T09:05:51.981 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.976+0000 7fabca08b640 1 -- 172.21.15.42:0/3555654152 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fabc410f770 msgr2=0x7fabc41188e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:05:51.981 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.976+0000 7fabca08b640 1 --2- 172.21.15.42:0/3555654152 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fabc410f770 0x7fabc41188e0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fabb400e970 tx=0x7fabb400ee30 comp rx=0 tx=0).stop 2024-08-30T09:05:51.981 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.977+0000 7fabca08b640 1 -- 172.21.15.42:0/3555654152 shutdown_connections 2024-08-30T09:05:51.981 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.977+0000 7fabca08b640 1 --2- 172.21.15.42:0/3555654152 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7faba80b4d40 0x7faba80b7250 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:51.981 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.977+0000 7fabca08b640 1 --2- 172.21.15.42:0/3555654152 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fabc410f770 0x7fabc41188e0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:51.981 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.977+0000 7fabca08b640 1 --2- 172.21.15.42:0/3555654152 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fabc410cb50 0x7fabc4117d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:51.981 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.977+0000 7fabca08b640 1 --2- 172.21.15.42:0/3555654152 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fabc410a160 0x7fabc41176c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:05:51.981 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.977+0000 7fabca08b640 1 -- 172.21.15.42:0/3555654152 >> 172.21.15.42:0/3555654152 conn(0x7fabc40705b0 msgr2=0x7fabc410af50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:05:51.981 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.977+0000 7fabca08b640 1 -- 172.21.15.42:0/3555654152 shutdown_connections 2024-08-30T09:05:51.981 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:05:51.977+0000 7fabca08b640 1 -- 172.21.15.42:0/3555654152 wait complete. 2024-08-30T09:05:52.172 INFO:tasks.cephfs.filesystem:scrub status for tag:1af4252a-4a99-47e9-bbf0-becf5e573dd5 - {'path': '/', 'tag': '1af4252a-4a99-47e9-bbf0-becf5e573dd5', 'options': 'recursive,force'} 2024-08-30T09:05:52.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:05:52 smithi042 ceph-mon[18652]: pgmap v526: 129 pgs: 129 active+clean; 1.2 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 9.9 MiB/s wr, 540 op/s 2024-08-30T09:05:52.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:05:52 smithi167 ceph-mon[22063]: pgmap v526: 129 pgs: 129 active+clean; 1.2 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 9.9 MiB/s wr, 540 op/s 2024-08-30T09:05:52.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:05:52 smithi190 ceph-mon[22001]: pgmap v526: 129 pgs: 129 active+clean; 1.2 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 9.9 MiB/s wr, 540 op/s 2024-08-30T09:05:54.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:05:54 smithi167 ceph-mon[22063]: pgmap v527: 129 pgs: 129 active+clean; 1.2 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 14 MiB/s wr, 721 op/s 2024-08-30T09:05:54.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:05:54 smithi042 ceph-mon[18652]: pgmap v527: 129 pgs: 129 active+clean; 1.2 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 14 MiB/s wr, 721 op/s 2024-08-30T09:05:54.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:05:54 smithi190 ceph-mon[22001]: pgmap v527: 129 pgs: 129 active+clean; 1.2 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 14 MiB/s wr, 721 op/s 2024-08-30T09:05:55.953 INFO:teuthology.orchestra.run.smithi042.stderr:progress: 310.0 s, 810.6 tps, lat 126.782 ms stddev 161.401 2024-08-30T09:05:56.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:05:56 smithi167 ceph-mon[22063]: pgmap v528: 129 pgs: 129 active+clean; 1.2 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 12 MiB/s wr, 624 op/s 2024-08-30T09:05:56.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:05:56 smithi042 ceph-mon[18652]: pgmap v528: 129 pgs: 129 active+clean; 1.2 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 12 MiB/s wr, 624 op/s 2024-08-30T09:05:56.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:05:56 smithi190 ceph-mon[22001]: pgmap v528: 129 pgs: 129 active+clean; 1.2 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 12 MiB/s wr, 624 op/s 2024-08-30T09:05:58.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:05:58 smithi167 ceph-mon[22063]: pgmap v529: 129 pgs: 129 active+clean; 1.2 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 15 MiB/s wr, 707 op/s 2024-08-30T09:05:58.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:05:58 smithi042 ceph-mon[18652]: pgmap v529: 129 pgs: 129 active+clean; 1.2 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 15 MiB/s wr, 707 op/s 2024-08-30T09:05:58.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:05:58 smithi190 ceph-mon[22001]: pgmap v529: 129 pgs: 129 active+clean; 1.2 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 15 MiB/s wr, 707 op/s 2024-08-30T09:06:00.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:06:00 smithi167 ceph-mon[22063]: pgmap v530: 129 pgs: 129 active+clean; 1.2 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 15 MiB/s wr, 730 op/s 2024-08-30T09:06:00.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:06:00 smithi042 ceph-mon[18652]: pgmap v530: 129 pgs: 129 active+clean; 1.2 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 15 MiB/s wr, 730 op/s 2024-08-30T09:06:00.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:06:00 smithi190 ceph-mon[22001]: pgmap v530: 129 pgs: 129 active+clean; 1.2 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 15 MiB/s wr, 730 op/s 2024-08-30T09:06:00.953 INFO:teuthology.orchestra.run.smithi042.stderr:progress: 315.0 s, 882.6 tps, lat 113.188 ms stddev 133.911 2024-08-30T09:06:02.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:06:02 smithi167 ceph-mon[22063]: pgmap v531: 129 pgs: 129 active+clean; 1.3 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 15 MiB/s wr, 681 op/s 2024-08-30T09:06:02.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:06:02 smithi042 ceph-mon[18652]: pgmap v531: 129 pgs: 129 active+clean; 1.3 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 15 MiB/s wr, 681 op/s 2024-08-30T09:06:02.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:06:02 smithi190 ceph-mon[22001]: pgmap v531: 129 pgs: 129 active+clean; 1.3 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 15 MiB/s wr, 681 op/s 2024-08-30T09:06:03.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:06:03 smithi167 ceph-mon[22063]: scrub summary: idle 2024-08-30T09:06:03.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:06:03 smithi167 ceph-mon[22063]: scrub summary: idle 2024-08-30T09:06:03.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:06:03 smithi042 ceph-mon[18652]: scrub summary: idle 2024-08-30T09:06:03.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:06:03 smithi042 ceph-mon[18652]: scrub summary: idle 2024-08-30T09:06:03.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:06:03 smithi190 ceph-mon[22001]: scrub summary: idle 2024-08-30T09:06:03.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:06:03 smithi190 ceph-mon[22001]: scrub summary: idle 2024-08-30T09:06:04.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:06:04 smithi167 ceph-mon[22063]: scrub summary: idle 2024-08-30T09:06:04.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:06:04 smithi167 ceph-mon[22063]: pgmap v532: 129 pgs: 129 active+clean; 1.3 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 19 MiB/s wr, 797 op/s 2024-08-30T09:06:04.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:06:04 smithi042 ceph-mon[18652]: scrub summary: idle 2024-08-30T09:06:04.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:06:04 smithi042 ceph-mon[18652]: pgmap v532: 129 pgs: 129 active+clean; 1.3 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 19 MiB/s wr, 797 op/s 2024-08-30T09:06:04.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:06:04 smithi190 ceph-mon[22001]: scrub summary: idle 2024-08-30T09:06:04.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:06:04 smithi190 ceph-mon[22001]: pgmap v532: 129 pgs: 129 active+clean; 1.3 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 19 MiB/s wr, 797 op/s 2024-08-30T09:06:05.136 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.134+0000 7ff436d39640 1 -- 172.21.15.42:0/2075059402 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff430104e80 msgr2=0x7ff43010b820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:06:05.136 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.134+0000 7ff436d39640 1 --2- 172.21.15.42:0/2075059402 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff430104e80 0x7ff43010b820 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7ff42800dc90 tx=0x7ff42802e3b0 comp rx=0 tx=0).stop 2024-08-30T09:06:05.144 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.142+0000 7ff436d39640 1 -- 172.21.15.42:0/2075059402 shutdown_connections 2024-08-30T09:06:05.144 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.142+0000 7ff436d39640 1 --2- 172.21.15.42:0/2075059402 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff430104e80 0x7ff43010b820 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:05.145 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.142+0000 7ff436d39640 1 --2- 172.21.15.42:0/2075059402 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff4301044e0 0x7ff4301048f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:05.145 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.142+0000 7ff436d39640 1 --2- 172.21.15.42:0/2075059402 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff430103460 0x7ff4301038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:05.145 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.142+0000 7ff436d39640 1 -- 172.21.15.42:0/2075059402 >> 172.21.15.42:0/2075059402 conn(0x7ff4300fed80 msgr2=0x7ff430101250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:06:05.152 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.149+0000 7ff436d39640 1 -- 172.21.15.42:0/2075059402 shutdown_connections 2024-08-30T09:06:05.155 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.153+0000 7ff436d39640 1 -- 172.21.15.42:0/2075059402 wait complete. 2024-08-30T09:06:05.155 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.154+0000 7ff436d39640 1 Processor -- start 2024-08-30T09:06:05.159 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.158+0000 7ff436d39640 1 -- start start 2024-08-30T09:06:05.161 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.158+0000 7ff436d39640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff430103460 0x7ff4301a0270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:05.161 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.158+0000 7ff436d39640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff4301044e0 0x7ff4301a08e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:05.161 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.159+0000 7ff436d39640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff430104e80 0x7ff4301a1490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:05.161 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.159+0000 7ff436d39640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7ff430115720 con 0x7ff430104e80 2024-08-30T09:06:05.161 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.159+0000 7ff436d39640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff4301155a0 con 0x7ff430103460 2024-08-30T09:06:05.161 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.159+0000 7ff436d39640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7ff4301158a0 con 0x7ff4301044e0 2024-08-30T09:06:05.161 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.160+0000 7ff4352d6640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff430104e80 0x7ff4301a1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:06:05.162 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.160+0000 7ff4352d6640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff430104e80 0x7ff4301a1490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:53646/0 (socket says 172.21.15.42:53646) 2024-08-30T09:06:05.162 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.160+0000 7ff4352d6640 1 -- 172.21.15.42:0/3100039680 learned_addr learned my addr 172.21.15.42:0/3100039680 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:06:05.164 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.162+0000 7ff4352d6640 1 -- 172.21.15.42:0/3100039680 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff4301044e0 msgr2=0x7ff4301a08e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-30T09:06:05.164 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.162+0000 7ff4352d6640 1 --2- 172.21.15.42:0/3100039680 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff4301044e0 0x7ff4301a08e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:05.164 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.162+0000 7ff4352d6640 1 -- 172.21.15.42:0/3100039680 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff430103460 msgr2=0x7ff4301a0270 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-08-30T09:06:05.164 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.162+0000 7ff4352d6640 1 --2- 172.21.15.42:0/3100039680 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff430103460 0x7ff4301a0270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:05.164 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.162+0000 7ff4352d6640 1 -- 172.21.15.42:0/3100039680 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4301a1cb0 con 0x7ff430104e80 2024-08-30T09:06:05.169 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.166+0000 7ff4352d6640 1 --2- 172.21.15.42:0/3100039680 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff430104e80 0x7ff4301a1490 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7ff428004c40 tx=0x7ff428004f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:06:05.178 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.176+0000 7ff42dffb640 1 -- 172.21.15.42:0/3100039680 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff42800d440 con 0x7ff430104e80 2024-08-30T09:06:05.178 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.176+0000 7ff42dffb640 1 -- 172.21.15.42:0/3100039680 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff4280035f0 con 0x7ff430104e80 2024-08-30T09:06:05.178 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.176+0000 7ff42dffb640 1 -- 172.21.15.42:0/3100039680 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff42803f6b0 con 0x7ff430104e80 2024-08-30T09:06:05.178 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.177+0000 7ff436d39640 1 -- 172.21.15.42:0/3100039680 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4301a1f40 con 0x7ff430104e80 2024-08-30T09:06:05.179 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.177+0000 7ff436d39640 1 -- 172.21.15.42:0/3100039680 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4301a84b0 con 0x7ff430104e80 2024-08-30T09:06:05.186 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.183+0000 7ff43dca9740 1 Processor -- start 2024-08-30T09:06:05.186 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.184+0000 7ff43dca9740 1 -- start start 2024-08-30T09:06:05.186 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.184+0000 7ff43dca9740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5642581b91b0 0x564258570190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:05.186 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.184+0000 7ff43dca9740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5642581902f0 0x564258570800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:05.186 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.184+0000 7ff43dca9740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5642581984d0 0x564258571350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:05.187 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.184+0000 7ff43dca9740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x5642584d2000 con 0x5642581902f0 2024-08-30T09:06:05.187 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.184+0000 7ff43dca9740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5642584c1590 con 0x5642581b91b0 2024-08-30T09:06:05.187 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.184+0000 7ff43dca9740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5642584c5540 con 0x5642581984d0 2024-08-30T09:06:05.187 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.184+0000 7ff42ffff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5642581b91b0 0x564258570190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:06:05.187 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.185+0000 7ff42ffff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5642581b91b0 0x564258570190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:35896/0 (socket says 172.21.15.42:35896) 2024-08-30T09:06:05.187 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.185+0000 7ff42ffff640 1 -- 172.21.15.42:0/2425718447 learned_addr learned my addr 172.21.15.42:0/2425718447 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:06:05.188 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.185+0000 7ff42ffff640 1 --2- 172.21.15.42:0/2425718447 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5642581984d0 0x564258571350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:06:05.191 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.187+0000 7ff434ad5640 1 --2- 172.21.15.42:0/2425718447 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5642581902f0 0x564258570800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:06:05.191 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.190+0000 7ff434ad5640 1 -- 172.21.15.42:0/2425718447 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5642581984d0 msgr2=0x564258571350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:06:05.192 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.190+0000 7ff434ad5640 1 --2- 172.21.15.42:0/2425718447 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5642581984d0 0x564258571350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:05.192 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.190+0000 7ff434ad5640 1 -- 172.21.15.42:0/2425718447 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5642581b91b0 msgr2=0x564258570190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:06:05.192 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.190+0000 7ff434ad5640 1 --2- 172.21.15.42:0/2425718447 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5642581b91b0 0x564258570190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:05.192 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.190+0000 7ff434ad5640 1 -- 172.21.15.42:0/2425718447 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564258224790 con 0x5642581902f0 2024-08-30T09:06:05.195 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.190+0000 7ff42dffb640 1 -- 172.21.15.42:0/3100039680 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7ff428003aa0 con 0x7ff430104e80 2024-08-30T09:06:05.195 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.191+0000 7ff434ad5640 1 --2- 172.21.15.42:0/2425718447 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5642581902f0 0x564258570800 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7ff420009d70 tx=0x7ff42000e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aaf0831ffca203c8 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:06:05.195 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.191+0000 7ff42dffb640 1 --2- 172.21.15.42:0/3100039680 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7ff41c0b4ee0 0x7ff41c0b73f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:05.195 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.192+0000 7ff42dffb640 1 -- 172.21.15.42:0/3100039680 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7ff4281057b0 con 0x7ff430104e80 2024-08-30T09:06:05.195 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.193+0000 7ff426ffd640 1 -- 172.21.15.42:0/2425718447 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff420004b20 con 0x5642581902f0 2024-08-30T09:06:05.195 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.193+0000 7ff426ffd640 1 -- 172.21.15.42:0/2425718447 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff420003e20 con 0x5642581902f0 2024-08-30T09:06:05.195 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.193+0000 7ff426ffd640 1 -- 172.21.15.42:0/2425718447 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4200047f0 con 0x5642581902f0 2024-08-30T09:06:05.196 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.194+0000 7ff42ffff640 1 --2- 172.21.15.42:0/3100039680 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7ff41c0b4ee0 0x7ff41c0b73f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:06:05.201 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.197+0000 7ff43dca9740 1 -- 172.21.15.42:0/2425718447 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5642581902f0 msgr2=0x564258570800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:06:05.201 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.197+0000 7ff43dca9740 1 --2- 172.21.15.42:0/2425718447 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5642581902f0 0x564258570800 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7ff420009d70 tx=0x7ff42000e720 comp rx=0 tx=0).stop 2024-08-30T09:06:05.201 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.197+0000 7ff42ffff640 1 --2- 172.21.15.42:0/3100039680 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7ff41c0b4ee0 0x7ff41c0b73f0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7ff418012510 tx=0x7ff4180046c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:06:05.201 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.197+0000 7ff43dca9740 1 -- 172.21.15.42:0/2425718447 shutdown_connections 2024-08-30T09:06:05.201 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.197+0000 7ff43dca9740 1 --2- 172.21.15.42:0/2425718447 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5642581984d0 0x564258571350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:05.201 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.197+0000 7ff43dca9740 1 --2- 172.21.15.42:0/2425718447 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5642581902f0 0x564258570800 secure :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7ff420009d70 tx=0x7ff42000e720 comp rx=0 tx=0).stop 2024-08-30T09:06:05.201 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.197+0000 7ff43dca9740 1 --2- 172.21.15.42:0/2425718447 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5642581b91b0 0x564258570190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:05.201 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.197+0000 7ff43dca9740 1 -- 172.21.15.42:0/2425718447 >> 172.21.15.42:0/2425718447 conn(0x5642581ad300 msgr2=0x56425855ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:06:05.207 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.205+0000 7ff43dca9740 1 -- 172.21.15.42:0/2425718447 shutdown_connections 2024-08-30T09:06:05.209 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.207+0000 7ff43dca9740 1 -- 172.21.15.42:0/2425718447 wait complete. 2024-08-30T09:06:05.209 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.207+0000 7ff43dca9740 1 -- start start 2024-08-30T09:06:05.209 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.208+0000 7ff43dca9740 1 Processor -- start 2024-08-30T09:06:05.210 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.208+0000 7ff43dca9740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5642581984d0 0x564258579d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:05.210 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.208+0000 7ff43dca9740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5642581b91b0 0x56425857a430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:05.210 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.208+0000 7ff43dca9740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564258177d80 0x56425857afa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:05.210 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.208+0000 7ff43dca9740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x5642583faab0 con 0x564258177d80 2024-08-30T09:06:05.210 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.208+0000 7ff43dca9740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x564258448680 con 0x5642581b91b0 2024-08-30T09:06:05.210 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.208+0000 7ff43dca9740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x564258571e70 con 0x5642581984d0 2024-08-30T09:06:05.211 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.208+0000 7ff43dca9740 10 client.0 _ll_register_callbacks cb 0x5642581d0a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:06:05.212 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.210+0000 7ff4352d6640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564258177d80 0x56425857afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:06:05.212 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.210+0000 7ff4352d6640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564258177d80 0x56425857afa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:53658/0 (socket says 172.21.15.42:53658) 2024-08-30T09:06:05.212 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.210+0000 7ff4352d6640 1 -- 172.21.15.42:0/260696192 learned_addr learned my addr 172.21.15.42:0/260696192 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:06:05.213 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.211+0000 7ff42ffff640 1 --2- 172.21.15.42:0/260696192 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5642581b91b0 0x56425857a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:06:05.213 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.211+0000 7ff4352d6640 1 -- 172.21.15.42:0/260696192 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5642581984d0 msgr2=0x564258579d70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-30T09:06:05.213 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.211+0000 7ff4352d6640 1 --2- 172.21.15.42:0/260696192 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5642581984d0 0x564258579d70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:05.213 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.211+0000 7ff4352d6640 1 -- 172.21.15.42:0/260696192 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5642581b91b0 msgr2=0x56425857a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:06:05.213 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.211+0000 7ff4352d6640 1 --2- 172.21.15.42:0/260696192 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5642581b91b0 0x56425857a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:05.213 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.211+0000 7ff4352d6640 1 -- 172.21.15.42:0/260696192 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56425829d620 con 0x564258177d80 2024-08-30T09:06:05.214 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.212+0000 7ff4352d6640 1 --2- 172.21.15.42:0/260696192 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564258177d80 0x56425857afa0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7ff428005f70 tx=0x7ff4280cb620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bc0b0d700337e0b server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:06:05.218 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.214+0000 7ff40ffff640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-08-30T09:06:05.218 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.214+0000 7ff40ffff640 1 -- 172.21.15.42:0/260696192 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4280f2060 con 0x564258177d80 2024-08-30T09:06:05.218 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.214+0000 7ff40ffff640 1 -- 172.21.15.42:0/260696192 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7ff428105400 con 0x564258177d80 2024-08-30T09:06:05.219 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.217+0000 7ff43dca9740 1 -- 172.21.15.42:0/260696192 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff420003e20 con 0x564258177d80 2024-08-30T09:06:05.219 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.217+0000 7ff40ffff640 1 -- 172.21.15.42:0/260696192 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7ff4280d0660 con 0x564258177d80 2024-08-30T09:06:05.219 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.217+0000 7ff43dca9740 10 client.16356 fetch_fsmap learned FSMap version 32 2024-08-30T09:06:05.219 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.217+0000 7ff43dca9740 1 -- 172.21.15.42:0/260696192 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x564258594f00 con 0x564258177d80 2024-08-30T09:06:05.220 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.218+0000 7ff40ffff640 1 -- 172.21.15.42:0/260696192 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff420003e20 con 0x564258177d80 2024-08-30T09:06:05.220 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.218+0000 7ff40ffff640 1 -- 172.21.15.42:0/260696192 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 32) ==== 8358+0+0 (secure 0 0 0) 0x7ff4280d4cc0 con 0x564258177d80 2024-08-30T09:06:05.220 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.218+0000 7ff43dca9740 10 client.16356 fetch_fsmap finished waiting for FSMap version 32 2024-08-30T09:06:05.220 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.218+0000 7ff43dca9740 10 client.16356 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-30T09:06:05.220 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.218+0000 7ff43dca9740 20 client.16356 populate_metadata read hostname 'smithi042' 2024-08-30T09:06:05.220 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.218+0000 7ff43dca9740 1 --2- 172.21.15.42:0/260696192 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x56425814d1c0 0x564258598940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:05.220 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.218+0000 7ff43dca9740 4 client.16356 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-30T09:06:05.221 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.218+0000 7ff43dca9740 1 -- 172.21.15.42:0/260696192 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5642584d3f50 con 0x56425814d1c0 2024-08-30T09:06:05.221 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.219+0000 7ff434ad5640 1 --2- 172.21.15.42:0/260696192 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x56425814d1c0 0x564258598940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:06:05.241 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.238+0000 7ff434ad5640 1 --2- 172.21.15.42:0/260696192 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x56425814d1c0 0x564258598940 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bde1068a8967d524 server_cookie=5f89566de4eeae64 in_seq=0 out_seq=0 2024-08-30T09:06:05.241 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.239+0000 7ff40ffff640 10 client.16356 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:06:05.241 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.239+0000 7ff40ffff640 1 -- 172.21.15.42:0/260696192 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5642584d3f50 con 0x56425814d1c0 2024-08-30T09:06:05.241 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.239+0000 7ff40ffff640 10 client.16356 handle_command_reply: tid=0 2024-08-30T09:06:05.241 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.239+0000 7ff43dca9740 1 client.16356 shutdown 2024-08-30T09:06:05.248 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.242+0000 7ff43dca9740 1 -- 172.21.15.42:0/260696192 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564258177d80 msgr2=0x56425857afa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:06:05.249 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.242+0000 7ff43dca9740 1 --2- 172.21.15.42:0/260696192 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564258177d80 0x56425857afa0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7ff428005f70 tx=0x7ff4280cb620 comp rx=0 tx=0).stop 2024-08-30T09:06:05.249 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.246+0000 7ff43dca9740 1 -- 172.21.15.42:0/260696192 shutdown_connections 2024-08-30T09:06:05.249 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.246+0000 7ff43dca9740 1 --2- 172.21.15.42:0/260696192 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x56425814d1c0 0x564258598940 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:05.249 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.246+0000 7ff43dca9740 1 --2- 172.21.15.42:0/260696192 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564258177d80 0x56425857afa0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:05.249 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.246+0000 7ff43dca9740 1 --2- 172.21.15.42:0/260696192 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5642581b91b0 0x56425857a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:05.249 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.246+0000 7ff43dca9740 1 --2- 172.21.15.42:0/260696192 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5642581984d0 0x564258579d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:05.249 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.246+0000 7ff43dca9740 1 -- 172.21.15.42:0/260696192 >> 172.21.15.42:0/260696192 conn(0x5642581ad300 msgr2=0x564258559170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:06:05.249 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.246+0000 7ff40ffff640 0 client.16356 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:06:05.250 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.246+0000 7ff43dca9740 1 -- 172.21.15.42:0/260696192 shutdown_connections 2024-08-30T09:06:05.250 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.246+0000 7ff43dca9740 1 -- 172.21.15.42:0/260696192 wait complete. 2024-08-30T09:06:05.250 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.247+0000 7ff43dca9740 20 client.16356 trim_cache size 0 max 16384 2024-08-30T09:06:05.273 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.270+0000 7ff43dca9740 1 Processor -- start 2024-08-30T09:06:05.274 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.271+0000 7ff43dca9740 1 -- start start 2024-08-30T09:06:05.274 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.271+0000 7ff43dca9740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564258177d80 0x564258574280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:05.274 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.271+0000 7ff43dca9740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5642581984d0 0x56425856fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:05.274 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.271+0000 7ff43dca9740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5642581b91b0 0x564258555400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:05.274 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.271+0000 7ff43dca9740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7ff4301108d0 con 0x564258177d80 2024-08-30T09:06:05.274 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.271+0000 7ff434ad5640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564258177d80 0x564258574280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:06:05.274 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.272+0000 7ff434ad5640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564258177d80 0x564258574280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:53666/0 (socket says 172.21.15.42:53666) 2024-08-30T09:06:05.274 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.272+0000 7ff434ad5640 1 -- 172.21.15.42:0/2900671340 learned_addr learned my addr 172.21.15.42:0/2900671340 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:06:05.274 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.272+0000 7ff4352d6640 1 --2- 172.21.15.42:0/2900671340 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5642581b91b0 0x564258555400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:06:05.274 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.272+0000 7ff43dca9740 1 -- 172.21.15.42:0/2900671340 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff43010e170 con 0x5642581984d0 2024-08-30T09:06:05.274 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.272+0000 7ff43dca9740 1 -- 172.21.15.42:0/2900671340 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7ff430074e00 con 0x5642581b91b0 2024-08-30T09:06:05.275 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.272+0000 7ff42ffff640 1 --2- 172.21.15.42:0/2900671340 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5642581984d0 0x56425856fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:06:05.275 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.272+0000 7ff42ffff640 1 -- 172.21.15.42:0/2900671340 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5642581b91b0 msgr2=0x564258555400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:06:05.275 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.272+0000 7ff42ffff640 1 --2- 172.21.15.42:0/2900671340 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5642581b91b0 0x564258555400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:05.275 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.272+0000 7ff42ffff640 1 -- 172.21.15.42:0/2900671340 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564258177d80 msgr2=0x564258574280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:06:05.276 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.272+0000 7ff42ffff640 1 --2- 172.21.15.42:0/2900671340 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564258177d80 0x564258574280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:05.276 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.272+0000 7ff42ffff640 1 -- 172.21.15.42:0/2900671340 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56425855b340 con 0x5642581984d0 2024-08-30T09:06:05.276 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.273+0000 7ff42ffff640 1 --2- 172.21.15.42:0/2900671340 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5642581984d0 0x56425856fe70 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7ff418005ea0 tx=0x7ff418009000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8bc897ba0da264b1 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:06:05.276 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.273+0000 7ff4267fc640 1 -- 172.21.15.42:0/2900671340 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff418019070 con 0x5642581984d0 2024-08-30T09:06:05.276 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.273+0000 7ff4267fc640 1 -- 172.21.15.42:0/2900671340 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff418024070 con 0x5642581984d0 2024-08-30T09:06:05.276 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.273+0000 7ff4267fc640 1 -- 172.21.15.42:0/2900671340 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff418029bb0 con 0x5642581984d0 2024-08-30T09:06:05.279 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.274+0000 7ff43dca9740 1 -- 172.21.15.42:0/2900671340 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5642581984d0 msgr2=0x56425856fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:06:05.279 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.274+0000 7ff43dca9740 1 --2- 172.21.15.42:0/2900671340 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5642581984d0 0x56425856fe70 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7ff418005ea0 tx=0x7ff418009000 comp rx=0 tx=0).stop 2024-08-30T09:06:05.279 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.274+0000 7ff43dca9740 1 -- 172.21.15.42:0/2900671340 shutdown_connections 2024-08-30T09:06:05.279 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.274+0000 7ff43dca9740 1 --2- 172.21.15.42:0/2900671340 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5642581b91b0 0x564258555400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:05.279 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.274+0000 7ff43dca9740 1 --2- 172.21.15.42:0/2900671340 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5642581984d0 0x56425856fe70 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:05.279 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.274+0000 7ff43dca9740 1 --2- 172.21.15.42:0/2900671340 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564258177d80 0x564258574280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:05.279 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.274+0000 7ff43dca9740 1 -- 172.21.15.42:0/2900671340 >> 172.21.15.42:0/2900671340 conn(0x5642581ad300 msgr2=0x564258598890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:06:05.279 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.274+0000 7ff43dca9740 1 -- 172.21.15.42:0/2900671340 shutdown_connections 2024-08-30T09:06:05.280 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.274+0000 7ff43dca9740 1 -- 172.21.15.42:0/2900671340 wait complete. 2024-08-30T09:06:05.280 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.275+0000 7ff43dca9740 1 -- start start 2024-08-30T09:06:05.280 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.275+0000 7ff43dca9740 1 Processor -- start 2024-08-30T09:06:05.280 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.275+0000 7ff43dca9740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564258177d80 0x56425857e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:05.280 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.275+0000 7ff43dca9740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5642581984d0 0x56425857edc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:05.280 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.275+0000 7ff434ad5640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564258177d80 0x56425857e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:06:05.280 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.275+0000 7ff434ad5640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564258177d80 0x56425857e700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:53670/0 (socket says 172.21.15.42:53670) 2024-08-30T09:06:05.280 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.275+0000 7ff434ad5640 1 -- 172.21.15.42:0/2380526302 learned_addr learned my addr 172.21.15.42:0/2380526302 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:06:05.280 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.276+0000 7ff43dca9740 1 --2- 172.21.15.42:0/2380526302 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5642581b91b0 0x56425857f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:05.280 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.276+0000 7ff43dca9740 1 -- 172.21.15.42:0/2380526302 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7ff43010c630 con 0x564258177d80 2024-08-30T09:06:05.280 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.276+0000 7ff43dca9740 1 -- 172.21.15.42:0/2380526302 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff43010daa0 con 0x5642581984d0 2024-08-30T09:06:05.280 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.276+0000 7ff43dca9740 1 -- 172.21.15.42:0/2380526302 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7ff43010cd00 con 0x5642581b91b0 2024-08-30T09:06:05.281 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.276+0000 7ff42ffff640 1 --2- 172.21.15.42:0/2380526302 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5642581984d0 0x56425857edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:06:05.281 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.276+0000 7ff43dca9740 10 client.0 _ll_register_callbacks cb 0x7ff42800d970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:06:05.281 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.276+0000 7ff4352d6640 1 --2- 172.21.15.42:0/2380526302 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5642581b91b0 0x56425857f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:06:05.281 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.276+0000 7ff42ffff640 1 -- 172.21.15.42:0/2380526302 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5642581b91b0 msgr2=0x56425857f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:06:05.281 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.276+0000 7ff42ffff640 1 --2- 172.21.15.42:0/2380526302 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5642581b91b0 0x56425857f930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:05.281 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.276+0000 7ff42ffff640 1 -- 172.21.15.42:0/2380526302 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564258177d80 msgr2=0x56425857e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:06:05.281 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.276+0000 7ff42ffff640 1 --2- 172.21.15.42:0/2380526302 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564258177d80 0x56425857e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:05.281 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.276+0000 7ff42ffff640 1 -- 172.21.15.42:0/2380526302 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564258580090 con 0x5642581984d0 2024-08-30T09:06:05.281 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.277+0000 7ff42ffff640 1 --2- 172.21.15.42:0/2380526302 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5642581984d0 0x56425857edc0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7ff418012fd0 tx=0x7ff418029b00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2cfa488f937b196a server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:06:05.281 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.277+0000 7ff40ffff640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-30T09:06:05.282 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.277+0000 7ff40ffff640 1 -- 172.21.15.42:0/2380526302 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff418019040 con 0x5642581984d0 2024-08-30T09:06:05.282 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.277+0000 7ff43dca9740 1 -- 172.21.15.42:0/2380526302 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff418024070 con 0x5642581984d0 2024-08-30T09:06:05.282 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.277+0000 7ff40ffff640 1 -- 172.21.15.42:0/2380526302 <== mon.2 v2:172.21.15.167:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7ff418005030 con 0x5642581984d0 2024-08-30T09:06:05.282 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.277+0000 7ff40ffff640 1 -- 172.21.15.42:0/2380526302 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7ff418005230 con 0x5642581984d0 2024-08-30T09:06:05.282 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.277+0000 7ff43dca9740 10 client.26108 fetch_fsmap learned FSMap version 32 2024-08-30T09:06:05.282 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.277+0000 7ff43dca9740 1 -- 172.21.15.42:0/2380526302 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x564258581390 con 0x5642581984d0 2024-08-30T09:06:05.283 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.277+0000 7ff40ffff640 1 -- 172.21.15.42:0/2380526302 <== mon.2 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff418024070 con 0x5642581984d0 2024-08-30T09:06:05.283 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.279+0000 7ff40ffff640 1 -- 172.21.15.42:0/2380526302 <== mon.2 v2:172.21.15.167:3300/0 5 ==== fsmap(e 32) ==== 8358+0+0 (secure 0 0 0) 0x7ff418004730 con 0x5642581984d0 2024-08-30T09:06:05.283 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.279+0000 7ff43dca9740 10 client.26108 fetch_fsmap finished waiting for FSMap version 32 2024-08-30T09:06:05.283 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.279+0000 7ff43dca9740 10 client.26108 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-30T09:06:05.283 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.279+0000 7ff43dca9740 20 client.26108 populate_metadata read hostname 'smithi042' 2024-08-30T09:06:05.284 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.279+0000 7ff43dca9740 1 --2- 172.21.15.42:0/2380526302 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x564258603300 0x564258605720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:05.284 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.279+0000 7ff43dca9740 4 client.26108 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-30T09:06:05.284 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.280+0000 7ff43dca9740 1 -- 172.21.15.42:0/2380526302 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5642584cf9f0 con 0x564258603300 2024-08-30T09:06:05.284 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.280+0000 7ff434ad5640 1 --2- 172.21.15.42:0/2380526302 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x564258603300 0x564258605720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:06:05.284 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.280+0000 7ff40ffff640 10 client.26108 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:06:05.284 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.280+0000 7ff434ad5640 1 --2- 172.21.15.42:0/2380526302 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x564258603300 0x564258605720 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c405b9c497ba99dc server_cookie=93bd9d1bbc37212f in_seq=0 out_seq=0 2024-08-30T09:06:05.529 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.526+0000 7ff40ffff640 1 -- 172.21.15.42:0/2380526302 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5642584cf9f0 con 0x564258603300 2024-08-30T09:06:05.530 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.526+0000 7ff40ffff640 10 client.26108 handle_command_reply: tid=0 2024-08-30T09:06:05.530 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.526+0000 7ff43dca9740 1 client.26108 shutdown 2024-08-30T09:06:05.530 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.526+0000 7ff43dca9740 1 -- 172.21.15.42:0/2380526302 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5642581984d0 msgr2=0x56425857edc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:06:05.530 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.526+0000 7ff43dca9740 1 --2- 172.21.15.42:0/2380526302 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5642581984d0 0x56425857edc0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7ff418012fd0 tx=0x7ff418029b00 comp rx=0 tx=0).stop 2024-08-30T09:06:05.530 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.526+0000 7ff43dca9740 1 -- 172.21.15.42:0/2380526302 shutdown_connections 2024-08-30T09:06:05.530 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.526+0000 7ff43dca9740 1 --2- 172.21.15.42:0/2380526302 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x564258603300 0x564258605720 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:05.530 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.527+0000 7ff40ffff640 0 client.26108 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:06:05.530 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.527+0000 7ff43dca9740 1 --2- 172.21.15.42:0/2380526302 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5642581b91b0 0x56425857f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:05.530 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.527+0000 7ff43dca9740 1 --2- 172.21.15.42:0/2380526302 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5642581984d0 0x56425857edc0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:05.531 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.527+0000 7ff43dca9740 1 --2- 172.21.15.42:0/2380526302 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564258177d80 0x56425857e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:05.531 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.527+0000 7ff43dca9740 1 -- 172.21.15.42:0/2380526302 >> 172.21.15.42:0/2380526302 conn(0x5642581ad300 msgr2=0x564258598890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:06:05.531 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.527+0000 7ff43dca9740 1 -- 172.21.15.42:0/2380526302 shutdown_connections 2024-08-30T09:06:05.531 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.527+0000 7ff43dca9740 1 -- 172.21.15.42:0/2380526302 wait complete. 2024-08-30T09:06:05.531 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.527+0000 7ff43dca9740 20 client.26108 trim_cache size 0 max 16384 2024-08-30T09:06:05.531 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-08-30T09:06:05.531 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2024-08-30T09:06:05.531 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-08-30T09:06:05.531 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-08-30T09:06:05.533 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.528+0000 7ff436d39640 1 -- 172.21.15.42:0/3100039680 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7ff41c0b4ee0 msgr2=0x7ff41c0b73f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:06:05.533 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.528+0000 7ff436d39640 1 --2- 172.21.15.42:0/3100039680 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7ff41c0b4ee0 0x7ff41c0b73f0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7ff418012510 tx=0x7ff4180046c0 comp rx=0 tx=0).stop 2024-08-30T09:06:05.533 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.528+0000 7ff436d39640 1 -- 172.21.15.42:0/3100039680 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff430104e80 msgr2=0x7ff4301a1490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:06:05.533 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.529+0000 7ff436d39640 1 --2- 172.21.15.42:0/3100039680 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff430104e80 0x7ff4301a1490 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7ff428004c40 tx=0x7ff428004f10 comp rx=0 tx=0).stop 2024-08-30T09:06:05.533 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.529+0000 7ff436d39640 1 -- 172.21.15.42:0/3100039680 shutdown_connections 2024-08-30T09:06:05.533 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.529+0000 7ff436d39640 1 --2- 172.21.15.42:0/3100039680 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7ff41c0b4ee0 0x7ff41c0b73f0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:05.533 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.529+0000 7ff436d39640 1 --2- 172.21.15.42:0/3100039680 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff4301044e0 0x7ff4301a08e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:05.533 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.529+0000 7ff436d39640 1 --2- 172.21.15.42:0/3100039680 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff430104e80 0x7ff4301a1490 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:05.533 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.529+0000 7ff436d39640 1 --2- 172.21.15.42:0/3100039680 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff430103460 0x7ff4301a0270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:05.533 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.529+0000 7ff436d39640 1 -- 172.21.15.42:0/3100039680 >> 172.21.15.42:0/3100039680 conn(0x7ff4300fed80 msgr2=0x7ff430107800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:06:05.533 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.529+0000 7ff436d39640 1 -- 172.21.15.42:0/3100039680 shutdown_connections 2024-08-30T09:06:05.533 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:05.529+0000 7ff436d39640 1 -- 172.21.15.42:0/3100039680 wait complete. 2024-08-30T09:06:05.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:06:05 smithi167 ceph-mon[22063]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-30T09:06:05.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:06:05 smithi042 ceph-mon[18652]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-30T09:06:05.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:06:05 smithi190 ceph-mon[22001]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-30T09:06:05.953 INFO:teuthology.orchestra.run.smithi042.stderr:progress: 320.0 s, 671.2 tps, lat 148.763 ms stddev 179.035 2024-08-30T09:06:06.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:06:06 smithi167 ceph-mon[22063]: pgmap v533: 129 pgs: 129 active+clean; 1.3 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 MiB/s wr, 600 op/s 2024-08-30T09:06:06.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:06:06 smithi042 ceph-mon[18652]: pgmap v533: 129 pgs: 129 active+clean; 1.3 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 MiB/s wr, 600 op/s 2024-08-30T09:06:06.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:06:06 smithi190 ceph-mon[22001]: pgmap v533: 129 pgs: 129 active+clean; 1.3 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 MiB/s wr, 600 op/s 2024-08-30T09:06:08.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:06:08 smithi167 ceph-mon[22063]: pgmap v534: 129 pgs: 129 active+clean; 1.3 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 15 MiB/s wr, 664 op/s 2024-08-30T09:06:08.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:06:08 smithi042 ceph-mon[18652]: pgmap v534: 129 pgs: 129 active+clean; 1.3 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 15 MiB/s wr, 664 op/s 2024-08-30T09:06:08.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:06:08 smithi190 ceph-mon[22001]: pgmap v534: 129 pgs: 129 active+clean; 1.3 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 15 MiB/s wr, 664 op/s 2024-08-30T09:06:10.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:06:10 smithi167 ceph-mon[22063]: pgmap v535: 129 pgs: 129 active+clean; 1.3 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 16 MiB/s wr, 635 op/s 2024-08-30T09:06:10.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:06:10 smithi167 ceph-mon[22063]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-30T09:06:10.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:06:10 smithi042 ceph-mon[18652]: pgmap v535: 129 pgs: 129 active+clean; 1.3 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 16 MiB/s wr, 635 op/s 2024-08-30T09:06:10.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:06:10 smithi042 ceph-mon[18652]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-30T09:06:10.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:06:10 smithi190 ceph-mon[22001]: pgmap v535: 129 pgs: 129 active+clean; 1.3 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 16 MiB/s wr, 635 op/s 2024-08-30T09:06:10.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:06:10 smithi190 ceph-mon[22001]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-30T09:06:10.953 INFO:teuthology.orchestra.run.smithi042.stderr:progress: 325.0 s, 628.6 tps, lat 159.605 ms stddev 196.420 2024-08-30T09:06:12.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:06:12 smithi167 ceph-mon[22063]: pgmap v536: 129 pgs: 129 active+clean; 1.3 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 14 MiB/s wr, 557 op/s 2024-08-30T09:06:12.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:06:12 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-30T09:06:12.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:06:12 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T09:06:12.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:06:12 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-30T09:06:12.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:06:12 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T09:06:12.599 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:06:12 smithi042 ceph-mon[18652]: pgmap v536: 129 pgs: 129 active+clean; 1.3 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 14 MiB/s wr, 557 op/s 2024-08-30T09:06:12.600 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:06:12 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-30T09:06:12.600 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:06:12 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T09:06:12.600 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:06:12 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-30T09:06:12.600 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:06:12 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T09:06:12.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:06:12 smithi190 ceph-mon[22001]: pgmap v536: 129 pgs: 129 active+clean; 1.3 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 14 MiB/s wr, 557 op/s 2024-08-30T09:06:12.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:06:12 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-30T09:06:12.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:06:12 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T09:06:12.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:06:12 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-30T09:06:12.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:06:12 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T09:06:14.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:06:14 smithi167 ceph-mon[22063]: pgmap v537: 129 pgs: 129 active+clean; 1.3 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 15 MiB/s wr, 662 op/s 2024-08-30T09:06:14.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:06:14 smithi042 ceph-mon[18652]: pgmap v537: 129 pgs: 129 active+clean; 1.3 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 15 MiB/s wr, 662 op/s 2024-08-30T09:06:14.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:06:14 smithi190 ceph-mon[22001]: pgmap v537: 129 pgs: 129 active+clean; 1.3 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 15 MiB/s wr, 662 op/s 2024-08-30T09:06:15.953 INFO:teuthology.orchestra.run.smithi042.stderr:progress: 330.0 s, 654.8 tps, lat 155.136 ms stddev 183.463 2024-08-30T09:06:16.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:06:16 smithi167 ceph-mon[22063]: pgmap v538: 129 pgs: 129 active+clean; 1.3 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 11 MiB/s wr, 534 op/s 2024-08-30T09:06:16.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:06:16 smithi042 ceph-mon[18652]: pgmap v538: 129 pgs: 129 active+clean; 1.3 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 11 MiB/s wr, 534 op/s 2024-08-30T09:06:16.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:06:16 smithi190 ceph-mon[22001]: pgmap v538: 129 pgs: 129 active+clean; 1.3 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 11 MiB/s wr, 534 op/s 2024-08-30T09:06:18.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:06:18 smithi167 ceph-mon[22063]: pgmap v539: 129 pgs: 129 active+clean; 1.3 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 15 MiB/s wr, 619 op/s 2024-08-30T09:06:18.634 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:06:18 smithi042 ceph-mon[18652]: pgmap v539: 129 pgs: 129 active+clean; 1.3 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 15 MiB/s wr, 619 op/s 2024-08-30T09:06:18.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:06:18 smithi190 ceph-mon[22001]: pgmap v539: 129 pgs: 129 active+clean; 1.3 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 15 MiB/s wr, 619 op/s 2024-08-30T09:06:18.699 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.697+0000 7f567660e640 1 -- 172.21.15.42:0/1475526323 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f567010f770 msgr2=0x7f567011a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:06:18.699 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.697+0000 7f567660e640 1 --2- 172.21.15.42:0/1475526323 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f567010f770 0x7f567011a930 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f5668009be0 tx=0x7f566802a710 comp rx=0 tx=0).stop 2024-08-30T09:06:18.705 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.703+0000 7f567660e640 1 -- 172.21.15.42:0/1475526323 shutdown_connections 2024-08-30T09:06:18.705 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.703+0000 7f567660e640 1 --2- 172.21.15.42:0/1475526323 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f567010f770 0x7f567011a930 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:18.705 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.703+0000 7f567660e640 1 --2- 172.21.15.42:0/1475526323 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f567010cb50 0x7f567010f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:18.705 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.703+0000 7f567660e640 1 --2- 172.21.15.42:0/1475526323 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f567010a160 0x7f567010c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:18.705 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.703+0000 7f567660e640 1 -- 172.21.15.42:0/1475526323 >> 172.21.15.42:0/1475526323 conn(0x7f56700705b0 msgr2=0x7f5670072a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:06:18.710 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.708+0000 7f567660e640 1 -- 172.21.15.42:0/1475526323 shutdown_connections 2024-08-30T09:06:18.713 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.709+0000 7f567660e640 1 -- 172.21.15.42:0/1475526323 wait complete. 2024-08-30T09:06:18.713 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.710+0000 7f567660e640 1 Processor -- start 2024-08-30T09:06:18.715 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.712+0000 7f567660e640 1 -- start start 2024-08-30T09:06:18.715 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.712+0000 7f567660e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f567010a160 0x7f56701a03a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:18.715 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.712+0000 7f567660e640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f567010cb50 0x7f56701a0a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:18.716 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.712+0000 7f567660e640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f567010f770 0x7f56701a14f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:18.716 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.712+0000 7f567660e640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f5670114bc0 con 0x7f567010f770 2024-08-30T09:06:18.716 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.712+0000 7f567660e640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f5670114a40 con 0x7f567010cb50 2024-08-30T09:06:18.716 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.712+0000 7f567660e640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f5670114d40 con 0x7f567010a160 2024-08-30T09:06:18.716 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.713+0000 7f566ffff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f567010a160 0x7f56701a03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:06:18.716 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.713+0000 7f566ffff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f567010a160 0x7f56701a03a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:50960/0 (socket says 172.21.15.42:50960) 2024-08-30T09:06:18.716 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.713+0000 7f566ffff640 1 -- 172.21.15.42:0/805101847 learned_addr learned my addr 172.21.15.42:0/805101847 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:06:18.716 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.713+0000 7f566ffff640 1 -- 172.21.15.42:0/805101847 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f567010cb50 msgr2=0x7f56701a0a10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-30T09:06:18.716 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.713+0000 7f566ffff640 1 --2- 172.21.15.42:0/805101847 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f567010cb50 0x7f56701a0a10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:18.716 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.713+0000 7f566ffff640 1 -- 172.21.15.42:0/805101847 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f567010f770 msgr2=0x7f56701a14f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-30T09:06:18.716 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.713+0000 7f566ffff640 1 --2- 172.21.15.42:0/805101847 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f567010f770 0x7f56701a14f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:18.716 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.713+0000 7f566ffff640 1 -- 172.21.15.42:0/805101847 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56701a1d10 con 0x7f567010a160 2024-08-30T09:06:18.716 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.713+0000 7f566ffff640 1 --2- 172.21.15.42:0/805101847 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f567010a160 0x7f56701a03a0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f566401ea20 tx=0x7f566401eee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:06:18.717 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.714+0000 7f566d7fa640 1 -- 172.21.15.42:0/805101847 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5664028070 con 0x7f567010a160 2024-08-30T09:06:18.717 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.714+0000 7f566d7fa640 1 -- 172.21.15.42:0/805101847 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f566401abc0 con 0x7f567010a160 2024-08-30T09:06:18.717 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.714+0000 7f566d7fa640 1 -- 172.21.15.42:0/805101847 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5664020630 con 0x7f567010a160 2024-08-30T09:06:18.718 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.714+0000 7f567660e640 1 -- 172.21.15.42:0/805101847 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56701a2000 con 0x7f567010a160 2024-08-30T09:06:18.718 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.714+0000 7f567660e640 1 -- 172.21.15.42:0/805101847 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56701a8530 con 0x7f567010a160 2024-08-30T09:06:18.721 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.718+0000 7f567d659740 1 Processor -- start 2024-08-30T09:06:18.721 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.720+0000 7f567d659740 1 -- start start 2024-08-30T09:06:18.722 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.720+0000 7f567d659740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5599fecb71b0 0x5599ff05cf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:18.723 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.720+0000 7f567d659740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5599fec8e2f0 0x5599ff05d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:18.723 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.720+0000 7f567d659740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5599fec964d0 0x5599ff05e120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:18.723 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.720+0000 7f567d659740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x5599fefd0000 con 0x5599fec8e2f0 2024-08-30T09:06:18.723 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.720+0000 7f567d659740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5599fefbf590 con 0x5599fecb71b0 2024-08-30T09:06:18.723 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.720+0000 7f567d659740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5599fefc3540 con 0x5599fec964d0 2024-08-30T09:06:18.723 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.721+0000 7f5674bab640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5599fecb71b0 0x5599ff05cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:06:18.723 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.721+0000 7f5674bab640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5599fecb71b0 0x5599ff05cf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:58928/0 (socket says 172.21.15.42:58928) 2024-08-30T09:06:18.723 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.721+0000 7f5674bab640 1 -- 172.21.15.42:0/2808680829 learned_addr learned my addr 172.21.15.42:0/2808680829 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:06:18.724 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.722+0000 7f566f7fe640 1 --2- 172.21.15.42:0/2808680829 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5599fec964d0 0x5599ff05e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:06:18.726 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.724+0000 7f566f7fe640 1 -- 172.21.15.42:0/2808680829 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5599fecb71b0 msgr2=0x5599ff05cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:06:18.726 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.724+0000 7f566f7fe640 1 --2- 172.21.15.42:0/2808680829 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5599fecb71b0 0x5599ff05cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:18.726 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.724+0000 7f566f7fe640 1 -- 172.21.15.42:0/2808680829 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5599fec8e2f0 msgr2=0x5599ff05d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:06:18.726 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.724+0000 7f566f7fe640 1 --2- 172.21.15.42:0/2808680829 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5599fec8e2f0 0x5599ff05d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:18.726 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.724+0000 7f566f7fe640 1 -- 172.21.15.42:0/2808680829 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5599fed22790 con 0x5599fec964d0 2024-08-30T09:06:18.729 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.725+0000 7f566f7fe640 1 --2- 172.21.15.42:0/2808680829 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5599fec964d0 0x5599ff05e120 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f5660009ad0 tx=0x7f5660009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4a9bab5239abced4 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:06:18.730 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.727+0000 7f56567fc640 1 -- 172.21.15.42:0/2808680829 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f566000ed10 con 0x5599fec964d0 2024-08-30T09:06:18.731 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.728+0000 7f56567fc640 1 -- 172.21.15.42:0/2808680829 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5660005020 con 0x5599fec964d0 2024-08-30T09:06:18.731 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.728+0000 7f56567fc640 1 -- 172.21.15.42:0/2808680829 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5660004a90 con 0x5599fec964d0 2024-08-30T09:06:18.731 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.728+0000 7f566d7fa640 1 -- 172.21.15.42:0/805101847 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7f5664021470 con 0x7f567010a160 2024-08-30T09:06:18.731 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.729+0000 7f566d7fa640 1 --2- 172.21.15.42:0/805101847 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f56580b4cc0 0x7f56580b71d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:18.731 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.729+0000 7f566d7fa640 1 -- 172.21.15.42:0/805101847 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f56640f8370 con 0x7f567010a160 2024-08-30T09:06:18.733 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.730+0000 7f567d659740 1 -- 172.21.15.42:0/2808680829 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5599fec964d0 msgr2=0x5599ff05e120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:06:18.733 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.730+0000 7f567d659740 1 --2- 172.21.15.42:0/2808680829 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5599fec964d0 0x5599ff05e120 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f5660009ad0 tx=0x7f5660009f90 comp rx=0 tx=0).stop 2024-08-30T09:06:18.733 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.731+0000 7f5674bab640 1 --2- 172.21.15.42:0/805101847 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f56580b4cc0 0x7f56580b71d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:06:18.734 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.732+0000 7f5674bab640 1 --2- 172.21.15.42:0/805101847 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f56580b4cc0 0x7f56580b71d0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f56680099a0 tx=0x7f5668004a60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:06:18.735 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.732+0000 7f567d659740 1 -- 172.21.15.42:0/2808680829 shutdown_connections 2024-08-30T09:06:18.735 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.732+0000 7f567d659740 1 --2- 172.21.15.42:0/2808680829 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5599fec964d0 0x5599ff05e120 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:18.735 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.732+0000 7f567d659740 1 --2- 172.21.15.42:0/2808680829 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5599fec8e2f0 0x5599ff05d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:18.735 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.732+0000 7f567d659740 1 --2- 172.21.15.42:0/2808680829 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5599fecb71b0 0x5599ff05cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:18.735 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.732+0000 7f567d659740 1 -- 172.21.15.42:0/2808680829 >> 172.21.15.42:0/2808680829 conn(0x5599fecab300 msgr2=0x5599ff0589f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:06:18.735 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.733+0000 7f567d659740 1 -- 172.21.15.42:0/2808680829 shutdown_connections 2024-08-30T09:06:18.735 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.733+0000 7f567d659740 1 -- 172.21.15.42:0/2808680829 wait complete. 2024-08-30T09:06:18.736 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.734+0000 7f567d659740 1 -- start start 2024-08-30T09:06:18.736 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.734+0000 7f567d659740 1 Processor -- start 2024-08-30T09:06:18.737 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.735+0000 7f567d659740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5599fec8e2f0 0x5599ff066aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:18.738 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.735+0000 7f567d659740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5599fec964d0 0x5599ff067160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:18.738 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.736+0000 7f567d659740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5599fecb71b0 0x5599ff067cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:18.738 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.736+0000 7f566f7fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5599fec964d0 0x5599ff067160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:06:18.738 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.736+0000 7f566f7fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5599fec964d0 0x5599ff067160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:50980/0 (socket says 172.21.15.42:50980) 2024-08-30T09:06:18.738 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.736+0000 7f566f7fe640 1 -- 172.21.15.42:0/1015205848 learned_addr learned my addr 172.21.15.42:0/1015205848 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:06:18.738 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.736+0000 7f567d659740 1 -- 172.21.15.42:0/1015205848 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x5599feef8ab0 con 0x5599fec8e2f0 2024-08-30T09:06:18.738 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.736+0000 7f567d659740 1 -- 172.21.15.42:0/1015205848 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5599fef46680 con 0x5599fecb71b0 2024-08-30T09:06:18.738 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.736+0000 7f567d659740 1 -- 172.21.15.42:0/1015205848 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5599ff05ec40 con 0x5599fec964d0 2024-08-30T09:06:18.738 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.736+0000 7f566ffff640 1 --2- 172.21.15.42:0/1015205848 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5599fec8e2f0 0x5599ff066aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:06:18.740 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.736+0000 7f5674bab640 1 --2- 172.21.15.42:0/1015205848 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5599fecb71b0 0x5599ff067cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:06:18.740 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.737+0000 7f567d659740 10 client.0 _ll_register_callbacks cb 0x5599feccea00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:06:18.740 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.737+0000 7f566f7fe640 1 -- 172.21.15.42:0/1015205848 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5599fecb71b0 msgr2=0x5599ff067cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:06:18.740 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.737+0000 7f566f7fe640 1 --2- 172.21.15.42:0/1015205848 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5599fecb71b0 0x5599ff067cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:18.740 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.737+0000 7f566f7fe640 1 -- 172.21.15.42:0/1015205848 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5599fec8e2f0 msgr2=0x5599ff066aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:06:18.740 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.737+0000 7f566f7fe640 1 --2- 172.21.15.42:0/1015205848 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5599fec8e2f0 0x5599ff066aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:18.740 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.737+0000 7f566f7fe640 1 -- 172.21.15.42:0/1015205848 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5599fed9b620 con 0x5599fec964d0 2024-08-30T09:06:18.740 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.737+0000 7f567d659740 1 -- 172.21.15.42:0/1015205848 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5660005020 con 0x5599fec964d0 2024-08-30T09:06:18.740 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.737+0000 7f566f7fe640 1 --2- 172.21.15.42:0/1015205848 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5599fec964d0 0x5599ff067160 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f5660009710 tx=0x7f5660009740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ddd16de549c9a8dc server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:06:18.740 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.737+0000 7f564f7fe640 10 client.26101 ms_handle_connect on v2:172.21.15.190:3300/0 2024-08-30T09:06:18.740 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.737+0000 7f564f7fe640 1 -- 172.21.15.42:0/1015205848 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5660004a20 con 0x5599fec964d0 2024-08-30T09:06:18.740 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.737+0000 7f564f7fe640 1 -- 172.21.15.42:0/1015205848 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f5660004bc0 con 0x5599fec964d0 2024-08-30T09:06:18.740 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.738+0000 7f564f7fe640 1 -- 172.21.15.42:0/1015205848 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f5660004e00 con 0x5599fec964d0 2024-08-30T09:06:18.741 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.738+0000 7f567d659740 10 client.26101 fetch_fsmap learned FSMap version 33 2024-08-30T09:06:18.741 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.738+0000 7f567d659740 1 -- 172.21.15.42:0/1015205848 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({fsmap=33}) -- 0x5599ff070b50 con 0x5599fec964d0 2024-08-30T09:06:18.742 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.738+0000 7f564f7fe640 1 -- 172.21.15.42:0/1015205848 <== mon.1 v2:172.21.15.190:3300/0 4 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f5660018470 con 0x5599fec964d0 2024-08-30T09:06:18.742 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.739+0000 7f567d659740 10 client.26101 fetch_fsmap finished waiting for FSMap version 33 2024-08-30T09:06:18.742 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.739+0000 7f567d659740 10 client.26101 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-30T09:06:18.742 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.739+0000 7f567d659740 20 client.26101 populate_metadata read hostname 'smithi042' 2024-08-30T09:06:18.742 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.739+0000 7f567d659740 1 --2- 172.21.15.42:0/1015205848 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x5599fec75d80 0x5599ff073eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:18.742 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.739+0000 7f567d659740 4 client.26101 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-30T09:06:18.742 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.739+0000 7f567d659740 1 -- 172.21.15.42:0/1015205848 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5599fefd1f50 con 0x5599fec75d80 2024-08-30T09:06:18.742 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.739+0000 7f566ffff640 1 --2- 172.21.15.42:0/1015205848 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x5599fec75d80 0x5599ff073eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:06:18.742 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.739+0000 7f566ffff640 1 --2- 172.21.15.42:0/1015205848 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x5599fec75d80 0x5599ff073eb0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5319c29c3409b294 server_cookie=f272101c733b0890 in_seq=0 out_seq=0 2024-08-30T09:06:18.743 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.739+0000 7f564f7fe640 10 client.26101 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:06:18.743 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.740+0000 7f564f7fe640 1 -- 172.21.15.42:0/1015205848 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5599fefd1f50 con 0x5599fec75d80 2024-08-30T09:06:18.743 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.740+0000 7f564f7fe640 10 client.26101 handle_command_reply: tid=0 2024-08-30T09:06:18.743 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.740+0000 7f567d659740 1 client.26101 shutdown 2024-08-30T09:06:18.744 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.741+0000 7f567d659740 1 -- 172.21.15.42:0/1015205848 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5599fec964d0 msgr2=0x5599ff067160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:06:18.744 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.741+0000 7f567d659740 1 --2- 172.21.15.42:0/1015205848 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5599fec964d0 0x5599ff067160 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f5660009710 tx=0x7f5660009740 comp rx=0 tx=0).stop 2024-08-30T09:06:18.744 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.741+0000 7f567d659740 1 -- 172.21.15.42:0/1015205848 shutdown_connections 2024-08-30T09:06:18.744 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.741+0000 7f567d659740 1 --2- 172.21.15.42:0/1015205848 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x5599fec75d80 0x5599ff073eb0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:18.744 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.741+0000 7f567d659740 1 --2- 172.21.15.42:0/1015205848 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5599fec964d0 0x5599ff067160 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:18.744 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.741+0000 7f567d659740 1 --2- 172.21.15.42:0/1015205848 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5599fecb71b0 0x5599ff067cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:18.744 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.741+0000 7f567d659740 1 --2- 172.21.15.42:0/1015205848 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5599fec8e2f0 0x5599ff066aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:18.744 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.741+0000 7f564f7fe640 0 client.26101 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:06:18.744 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.741+0000 7f567d659740 1 -- 172.21.15.42:0/1015205848 >> 172.21.15.42:0/1015205848 conn(0x5599fecab300 msgr2=0x5599ff056c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:06:18.744 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.741+0000 7f567d659740 1 -- 172.21.15.42:0/1015205848 shutdown_connections 2024-08-30T09:06:18.744 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.741+0000 7f567d659740 1 -- 172.21.15.42:0/1015205848 wait complete. 2024-08-30T09:06:18.745 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.741+0000 7f567d659740 20 client.26101 trim_cache size 0 max 16384 2024-08-30T09:06:18.769 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.766+0000 7f567d659740 1 Processor -- start 2024-08-30T09:06:18.769 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.766+0000 7f567d659740 1 -- start start 2024-08-30T09:06:18.769 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.766+0000 7f567d659740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5599fec75d80 0x5599ff05cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:18.769 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.766+0000 7f567d659740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5599fec8e2f0 0x5599ff058800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:18.769 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.766+0000 7f567d659740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5599fec964d0 0x5599ff064ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:18.769 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.766+0000 7f567d659740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f56701b54a0 con 0x5599fec75d80 2024-08-30T09:06:18.769 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.766+0000 7f567d659740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f56701b2d10 con 0x5599fec8e2f0 2024-08-30T09:06:18.769 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.766+0000 7f567d659740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f56701b0b60 con 0x5599fec964d0 2024-08-30T09:06:18.770 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.767+0000 7f566ffff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5599fec75d80 0x5599ff05cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:06:18.770 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.767+0000 7f566ffff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5599fec75d80 0x5599ff05cfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:58236/0 (socket says 172.21.15.42:58236) 2024-08-30T09:06:18.770 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.767+0000 7f566ffff640 1 -- 172.21.15.42:0/679181881 learned_addr learned my addr 172.21.15.42:0/679181881 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:06:18.770 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.767+0000 7f566f7fe640 1 --2- 172.21.15.42:0/679181881 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5599fec8e2f0 0x5599ff058800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:06:18.770 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.767+0000 7f5674bab640 1 --2- 172.21.15.42:0/679181881 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5599fec964d0 0x5599ff064ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:06:18.770 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.767+0000 7f566ffff640 1 -- 172.21.15.42:0/679181881 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5599fec964d0 msgr2=0x5599ff064ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:06:18.770 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.767+0000 7f566ffff640 1 --2- 172.21.15.42:0/679181881 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5599fec964d0 0x5599ff064ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:18.770 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.767+0000 7f566ffff640 1 -- 172.21.15.42:0/679181881 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5599fec8e2f0 msgr2=0x5599ff058800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:06:18.770 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.767+0000 7f566ffff640 1 --2- 172.21.15.42:0/679181881 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5599fec8e2f0 0x5599ff058800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:18.770 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.767+0000 7f566ffff640 1 -- 172.21.15.42:0/679181881 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5599ff074410 con 0x5599fec75d80 2024-08-30T09:06:18.774 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.767+0000 7f566ffff640 1 --2- 172.21.15.42:0/679181881 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5599fec75d80 0x5599ff05cfb0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f56640bbc20 tx=0x7f566402e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea992d11b25abe55 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:06:18.774 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.768+0000 7f5655ffb640 1 -- 172.21.15.42:0/679181881 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5664031040 con 0x5599fec75d80 2024-08-30T09:06:18.774 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.768+0000 7f5655ffb640 1 -- 172.21.15.42:0/679181881 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f566401ad70 con 0x5599fec75d80 2024-08-30T09:06:18.774 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.768+0000 7f5655ffb640 1 -- 172.21.15.42:0/679181881 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56640ba580 con 0x5599fec75d80 2024-08-30T09:06:18.774 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.769+0000 7f567d659740 1 -- 172.21.15.42:0/679181881 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5599fec75d80 msgr2=0x5599ff05cfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:06:18.774 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.769+0000 7f567d659740 1 --2- 172.21.15.42:0/679181881 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5599fec75d80 0x5599ff05cfb0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f56640bbc20 tx=0x7f566402e040 comp rx=0 tx=0).stop 2024-08-30T09:06:18.774 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.769+0000 7f567d659740 1 -- 172.21.15.42:0/679181881 shutdown_connections 2024-08-30T09:06:18.774 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.769+0000 7f567d659740 1 --2- 172.21.15.42:0/679181881 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5599fec964d0 0x5599ff064ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:18.774 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.769+0000 7f567d659740 1 --2- 172.21.15.42:0/679181881 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5599fec8e2f0 0x5599ff058800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:18.774 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.769+0000 7f567d659740 1 --2- 172.21.15.42:0/679181881 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5599fec75d80 0x5599ff05cfb0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:18.774 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.769+0000 7f567d659740 1 -- 172.21.15.42:0/679181881 >> 172.21.15.42:0/679181881 conn(0x5599fecab300 msgr2=0x5599ff056b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:06:18.774 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.769+0000 7f567d659740 1 -- 172.21.15.42:0/679181881 shutdown_connections 2024-08-30T09:06:18.775 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.769+0000 7f567d659740 1 -- 172.21.15.42:0/679181881 wait complete. 2024-08-30T09:06:18.775 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.769+0000 7f567d659740 1 -- start start 2024-08-30T09:06:18.775 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.769+0000 7f567d659740 1 Processor -- start 2024-08-30T09:06:18.775 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.770+0000 7f567d659740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5599fec75d80 0x5599ff06e120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:18.775 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.770+0000 7f567d659740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5599fec8e2f0 0x5599ff06cde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:18.775 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.770+0000 7f567d659740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5599fec964d0 0x5599ff06d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:18.775 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.770+0000 7f567d659740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f56701b11d0 con 0x5599fec75d80 2024-08-30T09:06:18.775 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.770+0000 7f567d659740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f56701b2640 con 0x5599fec8e2f0 2024-08-30T09:06:18.775 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.770+0000 7f567d659740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f56701b18a0 con 0x5599fec964d0 2024-08-30T09:06:18.775 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.770+0000 7f567d659740 10 client.0 _ll_register_callbacks cb 0x7f5668034070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:06:18.775 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.770+0000 7f566f7fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5599fec8e2f0 0x5599ff06cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:06:18.775 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.770+0000 7f566ffff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5599fec75d80 0x5599ff06e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:06:18.775 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.770+0000 7f566f7fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5599fec8e2f0 0x5599ff06cde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:58950/0 (socket says 172.21.15.42:58950) 2024-08-30T09:06:18.776 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.770+0000 7f566f7fe640 1 -- 172.21.15.42:0/524001278 learned_addr learned my addr 172.21.15.42:0/524001278 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:06:18.776 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.770+0000 7f5674bab640 1 --2- 172.21.15.42:0/524001278 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5599fec964d0 0x5599ff06d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:06:18.776 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.770+0000 7f566f7fe640 1 -- 172.21.15.42:0/524001278 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5599fec964d0 msgr2=0x5599ff06d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:06:18.776 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.770+0000 7f566f7fe640 1 --2- 172.21.15.42:0/524001278 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5599fec964d0 0x5599ff06d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:18.776 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.770+0000 7f566f7fe640 1 -- 172.21.15.42:0/524001278 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5599fec75d80 msgr2=0x5599ff06e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:06:18.776 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.770+0000 7f566f7fe640 1 --2- 172.21.15.42:0/524001278 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5599fec75d80 0x5599ff06e120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:18.776 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.770+0000 7f566f7fe640 1 -- 172.21.15.42:0/524001278 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5599ff06d860 con 0x5599fec8e2f0 2024-08-30T09:06:18.776 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.770+0000 7f5674bab640 1 --2- 172.21.15.42:0/524001278 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5599fec964d0 0x5599ff06d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-30T09:06:18.776 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.770+0000 7f566ffff640 1 --2- 172.21.15.42:0/524001278 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5599fec75d80 0x5599ff06e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-30T09:06:18.776 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.770+0000 7f566f7fe640 1 --2- 172.21.15.42:0/524001278 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5599fec8e2f0 0x5599ff06cde0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f5660004630 tx=0x7f5660004580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b4437532a07b7e31 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:06:18.776 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.770+0000 7f564f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-30T09:06:18.776 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.771+0000 7f564f7fe640 1 -- 172.21.15.42:0/524001278 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5660004660 con 0x5599fec8e2f0 2024-08-30T09:06:18.776 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.771+0000 7f567d659740 1 -- 172.21.15.42:0/524001278 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f566401ad70 con 0x5599fec8e2f0 2024-08-30T09:06:18.777 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.771+0000 7f564f7fe640 1 -- 172.21.15.42:0/524001278 <== mon.2 v2:172.21.15.167:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f5660004a20 con 0x5599fec8e2f0 2024-08-30T09:06:18.777 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.771+0000 7f564f7fe640 1 -- 172.21.15.42:0/524001278 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f5660009530 con 0x5599fec8e2f0 2024-08-30T09:06:18.777 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.771+0000 7f567d659740 10 client.26123 fetch_fsmap learned FSMap version 33 2024-08-30T09:06:18.777 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.771+0000 7f567d659740 1 -- 172.21.15.42:0/524001278 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5599ff06c770 con 0x5599fec8e2f0 2024-08-30T09:06:18.777 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.771+0000 7f564f7fe640 1 -- 172.21.15.42:0/524001278 <== mon.2 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f566401ad70 con 0x5599fec8e2f0 2024-08-30T09:06:18.777 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.771+0000 7f564f7fe640 1 -- 172.21.15.42:0/524001278 <== mon.2 v2:172.21.15.167:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f5660024a00 con 0x5599fec8e2f0 2024-08-30T09:06:18.777 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.772+0000 7f567d659740 10 client.26123 fetch_fsmap finished waiting for FSMap version 33 2024-08-30T09:06:18.777 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.772+0000 7f567d659740 10 client.26123 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-30T09:06:18.777 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.772+0000 7f567d659740 20 client.26123 populate_metadata read hostname 'smithi042' 2024-08-30T09:06:18.777 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.772+0000 7f567d659740 1 --2- 172.21.15.42:0/524001278 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x5599fecb71b0 0x5599ff0e1040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:18.777 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.772+0000 7f567d659740 4 client.26123 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-30T09:06:18.777 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.772+0000 7f567d659740 1 -- 172.21.15.42:0/524001278 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5599fefcd9f0 con 0x5599fecb71b0 2024-08-30T09:06:18.777 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.772+0000 7f566ffff640 1 --2- 172.21.15.42:0/524001278 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x5599fecb71b0 0x5599ff0e1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:06:18.778 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.772+0000 7f564f7fe640 10 client.26123 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:06:18.778 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.772+0000 7f566ffff640 1 --2- 172.21.15.42:0/524001278 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x5599fecb71b0 0x5599ff0e1040 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bdf8e1f94596cead server_cookie=6505f7614151d4e8 in_seq=0 out_seq=0 2024-08-30T09:06:18.787 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.785+0000 7f564f7fe640 1 -- 172.21.15.42:0/524001278 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5599fefcd9f0 con 0x5599fecb71b0 2024-08-30T09:06:18.787 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.785+0000 7f564f7fe640 10 client.26123 handle_command_reply: tid=0 2024-08-30T09:06:18.787 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.785+0000 7f567d659740 1 client.26123 shutdown 2024-08-30T09:06:18.788 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.785+0000 7f567d659740 1 -- 172.21.15.42:0/524001278 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5599fec8e2f0 msgr2=0x5599ff06cde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:06:18.788 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.785+0000 7f567d659740 1 --2- 172.21.15.42:0/524001278 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5599fec8e2f0 0x5599ff06cde0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f5660004630 tx=0x7f5660004580 comp rx=0 tx=0).stop 2024-08-30T09:06:18.788 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.786+0000 7f567d659740 1 -- 172.21.15.42:0/524001278 shutdown_connections 2024-08-30T09:06:18.788 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.786+0000 7f567d659740 1 --2- 172.21.15.42:0/524001278 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x5599fecb71b0 0x5599ff0e1040 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:18.788 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.786+0000 7f567d659740 1 --2- 172.21.15.42:0/524001278 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5599fec964d0 0x5599ff06d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:18.788 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.786+0000 7f567d659740 1 --2- 172.21.15.42:0/524001278 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5599fec8e2f0 0x5599ff06cde0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:18.788 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.786+0000 7f564f7fe640 0 client.26123 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:06:18.788 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.786+0000 7f567d659740 1 --2- 172.21.15.42:0/524001278 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5599fec75d80 0x5599ff06e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:18.789 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.786+0000 7f567d659740 1 -- 172.21.15.42:0/524001278 >> 172.21.15.42:0/524001278 conn(0x5599fecab300 msgr2=0x5599ff078630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:06:18.789 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.786+0000 7f567d659740 1 -- 172.21.15.42:0/524001278 shutdown_connections 2024-08-30T09:06:18.789 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.786+0000 7f567d659740 1 -- 172.21.15.42:0/524001278 wait complete. 2024-08-30T09:06:18.789 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.786+0000 7f567d659740 20 client.26123 trim_cache size 0 max 16384 2024-08-30T09:06:18.789 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-08-30T09:06:18.789 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2024-08-30T09:06:18.789 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-08-30T09:06:18.789 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-08-30T09:06:18.790 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.787+0000 7f567660e640 1 -- 172.21.15.42:0/805101847 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f56580b4cc0 msgr2=0x7f56580b71d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:06:18.790 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.787+0000 7f567660e640 1 --2- 172.21.15.42:0/805101847 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f56580b4cc0 0x7f56580b71d0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f56680099a0 tx=0x7f5668004a60 comp rx=0 tx=0).stop 2024-08-30T09:06:18.790 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.787+0000 7f567660e640 1 -- 172.21.15.42:0/805101847 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f567010a160 msgr2=0x7f56701a03a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:06:18.790 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.787+0000 7f567660e640 1 --2- 172.21.15.42:0/805101847 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f567010a160 0x7f56701a03a0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f566401ea20 tx=0x7f566401eee0 comp rx=0 tx=0).stop 2024-08-30T09:06:18.790 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.787+0000 7f567660e640 1 -- 172.21.15.42:0/805101847 shutdown_connections 2024-08-30T09:06:18.790 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.787+0000 7f567660e640 1 --2- 172.21.15.42:0/805101847 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f56580b4cc0 0x7f56580b71d0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:18.790 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.787+0000 7f567660e640 1 --2- 172.21.15.42:0/805101847 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f567010f770 0x7f56701a14f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:18.790 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.787+0000 7f567660e640 1 --2- 172.21.15.42:0/805101847 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f567010cb50 0x7f56701a0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:18.790 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.787+0000 7f567660e640 1 --2- 172.21.15.42:0/805101847 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f567010a160 0x7f56701a03a0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:18.791 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.787+0000 7f567660e640 1 -- 172.21.15.42:0/805101847 >> 172.21.15.42:0/805101847 conn(0x7f56700705b0 msgr2=0x7f567010b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:06:18.791 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.788+0000 7f567660e640 1 -- 172.21.15.42:0/805101847 shutdown_connections 2024-08-30T09:06:18.791 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:18.788+0000 7f567660e640 1 -- 172.21.15.42:0/805101847 wait complete. 2024-08-30T09:06:20.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:06:20 smithi167 ceph-mon[22063]: pgmap v540: 129 pgs: 129 active+clean; 1.3 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 14 MiB/s wr, 578 op/s 2024-08-30T09:06:20.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:06:20 smithi042 ceph-mon[18652]: pgmap v540: 129 pgs: 129 active+clean; 1.3 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 14 MiB/s wr, 578 op/s 2024-08-30T09:06:20.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:06:20 smithi190 ceph-mon[22001]: pgmap v540: 129 pgs: 129 active+clean; 1.3 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 14 MiB/s wr, 578 op/s 2024-08-30T09:06:20.953 INFO:teuthology.orchestra.run.smithi042.stderr:progress: 335.0 s, 606.4 tps, lat 161.083 ms stddev 206.598 2024-08-30T09:06:22.173 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-30T09:06:22.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:06:22 smithi167 ceph-mon[22063]: pgmap v541: 129 pgs: 129 active+clean; 1.3 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.8 MiB/s wr, 492 op/s 2024-08-30T09:06:22.634 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:06:22 smithi042 ceph-mon[18652]: pgmap v541: 129 pgs: 129 active+clean; 1.3 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.8 MiB/s wr, 492 op/s 2024-08-30T09:06:22.634 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T09:06:22.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:06:22 smithi190 ceph-mon[22001]: pgmap v541: 129 pgs: 129 active+clean; 1.3 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.8 MiB/s wr, 492 op/s 2024-08-30T09:06:23.308 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.305+0000 7f7a9f75d640 1 -- 172.21.15.42:0/4097613772 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7a9811c9d0 msgr2=0x7f7a9811eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:06:23.308 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.305+0000 7f7a9f75d640 1 --2- 172.21.15.42:0/4097613772 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7a9811c9d0 0x7f7a9811eee0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f7a9001acb0 tx=0x7f7a9003b920 comp rx=0 tx=0).stop 2024-08-30T09:06:23.308 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.306+0000 7f7a9f75d640 1 -- 172.21.15.42:0/4097613772 shutdown_connections 2024-08-30T09:06:23.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.306+0000 7f7a9f75d640 1 --2- 172.21.15.42:0/4097613772 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7a9811c9d0 0x7f7a9811eee0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:23.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.306+0000 7f7a9f75d640 1 --2- 172.21.15.42:0/4097613772 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7a9810a780 0x7f7a9810ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:23.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.306+0000 7f7a9f75d640 1 --2- 172.21.15.42:0/4097613772 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7a98111ce0 0x7f7a9810a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:23.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.306+0000 7f7a9f75d640 1 -- 172.21.15.42:0/4097613772 >> 172.21.15.42:0/4097613772 conn(0x7f7a98070c60 msgr2=0x7f7a98071070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:06:23.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.306+0000 7f7a9f75d640 1 -- 172.21.15.42:0/4097613772 shutdown_connections 2024-08-30T09:06:23.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.306+0000 7f7a9f75d640 1 -- 172.21.15.42:0/4097613772 wait complete. 2024-08-30T09:06:23.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.307+0000 7f7a9f75d640 1 Processor -- start 2024-08-30T09:06:23.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.310+0000 7f7a9f75d640 1 -- start start 2024-08-30T09:06:23.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.310+0000 7f7a9f75d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7a9810a780 0x7f7a9811c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:23.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.310+0000 7f7a9f75d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7a98111ce0 0x7f7a98118480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:23.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.310+0000 7f7a9f75d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7a9811c9d0 0x7f7a981189e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:23.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.310+0000 7f7a9f75d640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f7a98114960 con 0x7f7a9811c9d0 2024-08-30T09:06:23.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.310+0000 7f7a9f75d640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f7a981147e0 con 0x7f7a98111ce0 2024-08-30T09:06:23.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.310+0000 7f7a9f75d640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f7a98114ae0 con 0x7f7a9810a780 2024-08-30T09:06:23.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.310+0000 7f7a9d4f9640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7a9810a780 0x7f7a9811c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:06:23.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.310+0000 7f7a9d4f9640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7a9810a780 0x7f7a9811c1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:51006/0 (socket says 172.21.15.42:51006) 2024-08-30T09:06:23.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.310+0000 7f7a9d4f9640 1 -- 172.21.15.42:0/811797890 learned_addr learned my addr 172.21.15.42:0/811797890 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:06:23.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.311+0000 7f7a9ccf8640 1 --2- 172.21.15.42:0/811797890 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7a98111ce0 0x7f7a98118480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:06:23.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.311+0000 7f7a9dcfa640 1 --2- 172.21.15.42:0/811797890 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7a9811c9d0 0x7f7a981189e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:06:23.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.311+0000 7f7a9ccf8640 1 -- 172.21.15.42:0/811797890 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7a9810a780 msgr2=0x7f7a9811c1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:06:23.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.311+0000 7f7a9ccf8640 1 --2- 172.21.15.42:0/811797890 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7a9810a780 0x7f7a9811c1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:23.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.311+0000 7f7a9ccf8640 1 -- 172.21.15.42:0/811797890 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7a9811c9d0 msgr2=0x7f7a981189e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:06:23.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.311+0000 7f7a9ccf8640 1 --2- 172.21.15.42:0/811797890 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7a9811c9d0 0x7f7a981189e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:23.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.311+0000 7f7a9ccf8640 1 -- 172.21.15.42:0/811797890 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a98119200 con 0x7f7a98111ce0 2024-08-30T09:06:23.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.312+0000 7f7a9ccf8640 1 --2- 172.21.15.42:0/811797890 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7a98111ce0 0x7f7a98118480 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f7a9806c480 tx=0x7f7a8800db30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:06:23.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.313+0000 7f7a8e7fc640 1 -- 172.21.15.42:0/811797890 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a88039070 con 0x7f7a98111ce0 2024-08-30T09:06:23.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.313+0000 7f7a8e7fc640 1 -- 172.21.15.42:0/811797890 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7a88009cd0 con 0x7f7a98111ce0 2024-08-30T09:06:23.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.313+0000 7f7a8e7fc640 1 -- 172.21.15.42:0/811797890 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a88003c20 con 0x7f7a98111ce0 2024-08-30T09:06:23.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.313+0000 7f7a9f75d640 1 -- 172.21.15.42:0/811797890 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a981acbd0 con 0x7f7a98111ce0 2024-08-30T09:06:23.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.313+0000 7f7a9f75d640 1 -- 172.21.15.42:0/811797890 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a981ad0c0 con 0x7f7a98111ce0 2024-08-30T09:06:23.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.317+0000 7f7aa5cd2740 1 Processor -- start 2024-08-30T09:06:23.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.319+0000 7f7a8e7fc640 1 -- 172.21.15.42:0/811797890 <== mon.2 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7f7a88032470 con 0x7f7a98111ce0 2024-08-30T09:06:23.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.323+0000 7f7a8e7fc640 1 --2- 172.21.15.42:0/811797890 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f7a840b4d40 0x7f7a840b7250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:23.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.324+0000 7f7aa5cd2740 1 -- start start 2024-08-30T09:06:23.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.324+0000 7f7a8e7fc640 1 -- 172.21.15.42:0/811797890 <== mon.2 v2:172.21.15.167:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f7a88106f80 con 0x7f7a98111ce0 2024-08-30T09:06:23.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.324+0000 7f7aa5cd2740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a53140d070 0x55a5317b22c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:23.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.324+0000 7f7aa5cd2740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a5313e41b0 0x55a5317b2800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:23.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.324+0000 7f7aa5cd2740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a5313ec390 0x55a5317b2d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:23.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.324+0000 7f7aa5cd2740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55a531729630 con 0x55a5313ec390 2024-08-30T09:06:23.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.324+0000 7f7aa5cd2740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55a531718bc0 con 0x55a5313e41b0 2024-08-30T09:06:23.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.324+0000 7f7aa5cd2740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55a53171cb70 con 0x55a53140d070 2024-08-30T09:06:23.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.324+0000 7f7a9dcfa640 1 --2- 172.21.15.42:0/811797890 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f7a840b4d40 0x7f7a840b7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:06:23.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.325+0000 7f7a9d4f9640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a53140d070 0x55a5317b22c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:06:23.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.325+0000 7f7a9d4f9640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a53140d070 0x55a5317b22c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:51022/0 (socket says 172.21.15.42:51022) 2024-08-30T09:06:23.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.325+0000 7f7a9d4f9640 1 -- 172.21.15.42:0/4224983394 learned_addr learned my addr 172.21.15.42:0/4224983394 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:06:23.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.327+0000 7f7a9dcfa640 1 --2- 172.21.15.42:0/4224983394 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a5313ec390 0x55a5317b2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:06:23.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.327+0000 7f7a9ccf8640 1 --2- 172.21.15.42:0/4224983394 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a5313e41b0 0x55a5317b2800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:06:23.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.327+0000 7f7a9d4f9640 1 -- 172.21.15.42:0/4224983394 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a5313e41b0 msgr2=0x55a5317b2800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:06:23.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.327+0000 7f7a9d4f9640 1 --2- 172.21.15.42:0/4224983394 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a5313e41b0 0x55a5317b2800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:23.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.327+0000 7f7a9d4f9640 1 -- 172.21.15.42:0/4224983394 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a5313ec390 msgr2=0x55a5317b2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:06:23.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.327+0000 7f7a9d4f9640 1 --2- 172.21.15.42:0/4224983394 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a5313ec390 0x55a5317b2d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:23.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.327+0000 7f7a9d4f9640 1 -- 172.21.15.42:0/4224983394 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a531478650 con 0x55a53140d070 2024-08-30T09:06:23.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.330+0000 7f7a9d4f9640 1 --2- 172.21.15.42:0/4224983394 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a53140d070 0x55a5317b22c0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f7a9400bd40 tx=0x7f7a940107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ab9d72b62d3907d5 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:06:23.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.331+0000 7f7a837fe640 1 -- 172.21.15.42:0/4224983394 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a94011440 con 0x55a53140d070 2024-08-30T09:06:23.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.331+0000 7f7a837fe640 1 -- 172.21.15.42:0/4224983394 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7a94011a90 con 0x55a53140d070 2024-08-30T09:06:23.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.332+0000 7f7aa5cd2740 1 -- 172.21.15.42:0/4224983394 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a53140d070 msgr2=0x55a5317b22c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:06:23.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.332+0000 7f7aa5cd2740 1 --2- 172.21.15.42:0/4224983394 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a53140d070 0x55a5317b22c0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f7a9400bd40 tx=0x7f7a940107a0 comp rx=0 tx=0).stop 2024-08-30T09:06:23.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.333+0000 7f7aa5cd2740 1 -- 172.21.15.42:0/4224983394 shutdown_connections 2024-08-30T09:06:23.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.333+0000 7f7aa5cd2740 1 --2- 172.21.15.42:0/4224983394 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a5313ec390 0x55a5317b2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:23.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.333+0000 7f7aa5cd2740 1 --2- 172.21.15.42:0/4224983394 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a5313e41b0 0x55a5317b2800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:23.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.333+0000 7f7aa5cd2740 1 --2- 172.21.15.42:0/4224983394 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a53140d070 0x55a5317b22c0 secure :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f7a9400bd40 tx=0x7f7a940107a0 comp rx=0 tx=0).stop 2024-08-30T09:06:23.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.333+0000 7f7aa5cd2740 1 -- 172.21.15.42:0/4224983394 >> 172.21.15.42:0/4224983394 conn(0x55a531401460 msgr2=0x55a5317b01b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:06:23.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.334+0000 7f7a9dcfa640 1 --2- 172.21.15.42:0/811797890 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f7a840b4d40 0x7f7a840b7250 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f7a9001a4f0 tx=0x7f7a9001a7f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:06:23.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.334+0000 7f7aa5cd2740 1 -- 172.21.15.42:0/4224983394 shutdown_connections 2024-08-30T09:06:23.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.335+0000 7f7aa5cd2740 1 -- 172.21.15.42:0/4224983394 wait complete. 2024-08-30T09:06:23.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.335+0000 7f7aa5cd2740 1 -- start start 2024-08-30T09:06:23.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.335+0000 7f7aa5cd2740 1 Processor -- start 2024-08-30T09:06:23.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.336+0000 7f7aa5cd2740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a5313e41b0 0x55a5317bda00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:23.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.336+0000 7f7aa5cd2740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a5313ec390 0x55a5317bdf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:23.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.336+0000 7f7aa5cd2740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a5313cb9f0 0x55a5317c9ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:23.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.336+0000 7f7aa5cd2740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55a53157b700 con 0x55a5313e41b0 2024-08-30T09:06:23.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.336+0000 7f7aa5cd2740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55a53169f700 con 0x55a5313cb9f0 2024-08-30T09:06:23.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.336+0000 7f7aa5cd2740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55a5317c0b40 con 0x55a5313ec390 2024-08-30T09:06:23.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.336+0000 7f7aa5cd2740 10 client.0 _ll_register_callbacks cb 0x55a5316cedc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:06:23.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.337+0000 7f7a9dcfa640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a5313cb9f0 0x55a5317c9ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:06:23.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.337+0000 7f7a9dcfa640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a5313cb9f0 0x55a5317c9ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:58970/0 (socket says 172.21.15.42:58970) 2024-08-30T09:06:23.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.337+0000 7f7a9dcfa640 1 -- 172.21.15.42:0/1293536487 learned_addr learned my addr 172.21.15.42:0/1293536487 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:06:23.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.337+0000 7f7a9ccf8640 1 --2- 172.21.15.42:0/1293536487 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a5313ec390 0x55a5317bdf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:06:23.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.338+0000 7f7a9d4f9640 1 --2- 172.21.15.42:0/1293536487 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a5313e41b0 0x55a5317bda00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:06:23.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.338+0000 7f7a9ccf8640 1 -- 172.21.15.42:0/1293536487 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a5313cb9f0 msgr2=0x55a5317c9ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:06:23.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.338+0000 7f7a9ccf8640 1 --2- 172.21.15.42:0/1293536487 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a5313cb9f0 0x55a5317c9ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:23.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.338+0000 7f7a9ccf8640 1 -- 172.21.15.42:0/1293536487 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a5313e41b0 msgr2=0x55a5317bda00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:06:23.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.338+0000 7f7a9ccf8640 1 --2- 172.21.15.42:0/1293536487 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a5313e41b0 0x55a5317bda00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:23.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.338+0000 7f7a9ccf8640 1 -- 172.21.15.42:0/1293536487 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a5316d88b0 con 0x55a5313ec390 2024-08-30T09:06:23.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.338+0000 7f7a9ccf8640 1 --2- 172.21.15.42:0/1293536487 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a5313ec390 0x55a5317bdf40 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f7a88104690 tx=0x7f7a880d7350 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2bbfd8145107a39a server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:06:23.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.340+0000 7f7a80ff9640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-08-30T09:06:23.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.343+0000 7f7a80ff9640 1 -- 172.21.15.42:0/1293536487 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a88042040 con 0x55a5313ec390 2024-08-30T09:06:23.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.343+0000 7f7a80ff9640 1 -- 172.21.15.42:0/1293536487 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f7a880ce890 con 0x55a5313ec390 2024-08-30T09:06:23.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.343+0000 7f7aa5cd2740 1 -- 172.21.15.42:0/1293536487 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7a94011a90 con 0x55a5313ec390 2024-08-30T09:06:23.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.344+0000 7f7a80ff9640 1 -- 172.21.15.42:0/1293536487 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f7a880ce630 con 0x55a5313ec390 2024-08-30T09:06:23.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.344+0000 7f7aa5cd2740 10 client.26119 fetch_fsmap learned FSMap version 33 2024-08-30T09:06:23.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.344+0000 7f7aa5cd2740 1 -- 172.21.15.42:0/1293536487 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55a5317cc9e0 con 0x55a5313ec390 2024-08-30T09:06:23.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.349+0000 7f7a80ff9640 1 -- 172.21.15.42:0/1293536487 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7a94011a90 con 0x55a5313ec390 2024-08-30T09:06:23.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.349+0000 7f7a80ff9640 1 -- 172.21.15.42:0/1293536487 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f7a880cd890 con 0x55a5313ec390 2024-08-30T09:06:23.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.349+0000 7f7aa5cd2740 10 client.26119 fetch_fsmap finished waiting for FSMap version 33 2024-08-30T09:06:23.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.349+0000 7f7aa5cd2740 10 client.26119 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-30T09:06:23.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.349+0000 7f7aa5cd2740 20 client.26119 populate_metadata read hostname 'smithi042' 2024-08-30T09:06:23.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.349+0000 7f7aa5cd2740 1 --2- 172.21.15.42:0/1293536487 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55a5313a0da0 0x55a5317cfdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:23.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.349+0000 7f7aa5cd2740 4 client.26119 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-30T09:06:23.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.349+0000 7f7aa5cd2740 1 -- 172.21.15.42:0/1293536487 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a53172b580 con 0x55a5313a0da0 2024-08-30T09:06:23.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.353+0000 7f7a9d4f9640 1 --2- 172.21.15.42:0/1293536487 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55a5313a0da0 0x55a5317cfdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:06:23.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.359+0000 7f7a9d4f9640 1 --2- 172.21.15.42:0/1293536487 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55a5313a0da0 0x55a5317cfdd0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=462d908c9a24fcb2 server_cookie=18bb7dd51796b997 in_seq=0 out_seq=0 2024-08-30T09:06:23.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.360+0000 7f7a80ff9640 10 client.26119 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:06:23.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.363+0000 7f7a80ff9640 1 -- 172.21.15.42:0/1293536487 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a53172b580 con 0x55a5313a0da0 2024-08-30T09:06:23.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.363+0000 7f7a80ff9640 10 client.26119 handle_command_reply: tid=0 2024-08-30T09:06:23.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.363+0000 7f7aa5cd2740 1 client.26119 shutdown 2024-08-30T09:06:23.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.363+0000 7f7aa5cd2740 1 -- 172.21.15.42:0/1293536487 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a5313ec390 msgr2=0x55a5317bdf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:06:23.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.363+0000 7f7aa5cd2740 1 --2- 172.21.15.42:0/1293536487 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a5313ec390 0x55a5317bdf40 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f7a88104690 tx=0x7f7a880d7350 comp rx=0 tx=0).stop 2024-08-30T09:06:23.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.364+0000 7f7aa5cd2740 1 -- 172.21.15.42:0/1293536487 shutdown_connections 2024-08-30T09:06:23.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.364+0000 7f7aa5cd2740 1 --2- 172.21.15.42:0/1293536487 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55a5313a0da0 0x55a5317cfdd0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:23.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.364+0000 7f7aa5cd2740 1 --2- 172.21.15.42:0/1293536487 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a5313ec390 0x55a5317bdf40 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:23.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.364+0000 7f7aa5cd2740 1 --2- 172.21.15.42:0/1293536487 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a5313cb9f0 0x55a5317c9ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:23.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.364+0000 7f7aa5cd2740 1 --2- 172.21.15.42:0/1293536487 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a5313e41b0 0x55a5317bda00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:23.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.364+0000 7f7aa5cd2740 1 -- 172.21.15.42:0/1293536487 >> 172.21.15.42:0/1293536487 conn(0x55a531401460 msgr2=0x55a5317b5450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:06:23.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.364+0000 7f7a80ff9640 0 client.26119 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:06:23.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.364+0000 7f7aa5cd2740 1 -- 172.21.15.42:0/1293536487 shutdown_connections 2024-08-30T09:06:23.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.364+0000 7f7aa5cd2740 1 -- 172.21.15.42:0/1293536487 wait complete. 2024-08-30T09:06:23.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.364+0000 7f7aa5cd2740 20 client.26119 trim_cache size 0 max 16384 2024-08-30T09:06:23.401 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.399+0000 7f7aa5cd2740 1 Processor -- start 2024-08-30T09:06:23.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.399+0000 7f7aa5cd2740 1 -- start start 2024-08-30T09:06:23.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.400+0000 7f7aa5cd2740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a5313cb9f0 0x55a5317b5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:23.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.400+0000 7f7aa5cd2740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a5313e41b0 0x55a5317c9de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:23.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.400+0000 7f7aa5cd2740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a5313ec390 0x55a5317afad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:23.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.400+0000 7f7aa5cd2740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f7a7c0125c0 con 0x55a5313ec390 2024-08-30T09:06:23.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.400+0000 7f7a9ccf8640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a5313e41b0 0x55a5317c9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:06:23.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.400+0000 7f7a9ccf8640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a5313e41b0 0x55a5317c9de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:51042/0 (socket says 172.21.15.42:51042) 2024-08-30T09:06:23.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.400+0000 7f7a9ccf8640 1 -- 172.21.15.42:0/1495908297 learned_addr learned my addr 172.21.15.42:0/1495908297 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:06:23.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.400+0000 7f7a9d4f9640 1 --2- 172.21.15.42:0/1495908297 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a5313cb9f0 0x55a5317b5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:06:23.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.400+0000 7f7aa5cd2740 1 -- 172.21.15.42:0/1495908297 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f7a7c00fe30 con 0x55a5313cb9f0 2024-08-30T09:06:23.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.400+0000 7f7a9dcfa640 1 --2- 172.21.15.42:0/1495908297 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a5313ec390 0x55a5317afad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:06:23.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.400+0000 7f7aa5cd2740 1 -- 172.21.15.42:0/1495908297 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f7a7c00dc80 con 0x55a5313e41b0 2024-08-30T09:06:23.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.401+0000 7f7a9dcfa640 1 -- 172.21.15.42:0/1495908297 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a5313e41b0 msgr2=0x55a5317c9de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:06:23.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.401+0000 7f7a9dcfa640 1 --2- 172.21.15.42:0/1495908297 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a5313e41b0 0x55a5317c9de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:23.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.401+0000 7f7a9dcfa640 1 -- 172.21.15.42:0/1495908297 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a5313cb9f0 msgr2=0x55a5317b5ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:06:23.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.401+0000 7f7a9dcfa640 1 --2- 172.21.15.42:0/1495908297 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a5313cb9f0 0x55a5317b5ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:23.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.401+0000 7f7a9dcfa640 1 -- 172.21.15.42:0/1495908297 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a5317b8610 con 0x55a5313ec390 2024-08-30T09:06:23.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.401+0000 7f7a9dcfa640 1 --2- 172.21.15.42:0/1495908297 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a5313ec390 0x55a5317afad0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f7a9003bf10 tx=0x7f7a9000a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab068b49492d5ab8 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:06:23.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.402+0000 7f7a82ffd640 1 -- 172.21.15.42:0/1495908297 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a90018040 con 0x55a5313ec390 2024-08-30T09:06:23.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.402+0000 7f7a82ffd640 1 -- 172.21.15.42:0/1495908297 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7a90041c70 con 0x55a5313ec390 2024-08-30T09:06:23.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.402+0000 7f7a82ffd640 1 -- 172.21.15.42:0/1495908297 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a90045040 con 0x55a5313ec390 2024-08-30T09:06:23.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.403+0000 7f7aa5cd2740 1 -- 172.21.15.42:0/1495908297 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a5313ec390 msgr2=0x55a5317afad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:06:23.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.403+0000 7f7aa5cd2740 1 --2- 172.21.15.42:0/1495908297 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a5313ec390 0x55a5317afad0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f7a9003bf10 tx=0x7f7a9000a000 comp rx=0 tx=0).stop 2024-08-30T09:06:23.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.403+0000 7f7aa5cd2740 1 -- 172.21.15.42:0/1495908297 shutdown_connections 2024-08-30T09:06:23.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.403+0000 7f7aa5cd2740 1 --2- 172.21.15.42:0/1495908297 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a5313e41b0 0x55a5317c9de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:23.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.403+0000 7f7aa5cd2740 1 --2- 172.21.15.42:0/1495908297 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a5313ec390 0x55a5317afad0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:23.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.403+0000 7f7aa5cd2740 1 --2- 172.21.15.42:0/1495908297 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a5313cb9f0 0x55a5317b5ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:23.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.403+0000 7f7aa5cd2740 1 -- 172.21.15.42:0/1495908297 >> 172.21.15.42:0/1495908297 conn(0x55a531401460 msgr2=0x55a5317b53c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:06:23.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.404+0000 7f7aa5cd2740 1 -- 172.21.15.42:0/1495908297 shutdown_connections 2024-08-30T09:06:23.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.404+0000 7f7aa5cd2740 1 -- 172.21.15.42:0/1495908297 wait complete. 2024-08-30T09:06:23.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.404+0000 7f7aa5cd2740 1 -- start start 2024-08-30T09:06:23.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.405+0000 7f7aa5cd2740 1 Processor -- start 2024-08-30T09:06:23.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.406+0000 7f7aa5cd2740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a5313cb9f0 0x55a5317c2520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:23.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.406+0000 7f7aa5cd2740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a5313e41b0 0x55a5317c2a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:23.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.406+0000 7f7a9d4f9640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a5313cb9f0 0x55a5317c2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:06:23.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.406+0000 7f7a9d4f9640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a5313cb9f0 0x55a5317c2520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:51054/0 (socket says 172.21.15.42:51054) 2024-08-30T09:06:23.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.406+0000 7f7a9ccf8640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a5313e41b0 0x55a5317c2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:06:23.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.406+0000 7f7a9ccf8640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a5313e41b0 0x55a5317c2a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:58298/0 (socket says 172.21.15.42:58298) 2024-08-30T09:06:23.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.406+0000 7f7a9d4f9640 1 -- 172.21.15.42:0/3915129377 learned_addr learned my addr 172.21.15.42:0/3915129377 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:06:23.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.406+0000 7f7aa5cd2740 1 --2- 172.21.15.42:0/3915129377 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a5313ec390 0x55a531839790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:23.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.406+0000 7f7aa5cd2740 1 -- 172.21.15.42:0/3915129377 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f7a7c00e2f0 con 0x55a5313e41b0 2024-08-30T09:06:23.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.407+0000 7f7a9dcfa640 1 --2- 172.21.15.42:0/3915129377 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a5313ec390 0x55a531839790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:06:23.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.407+0000 7f7aa5cd2740 1 -- 172.21.15.42:0/3915129377 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f7a7c00f760 con 0x55a5313ec390 2024-08-30T09:06:23.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.407+0000 7f7aa5cd2740 1 -- 172.21.15.42:0/3915129377 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f7a7c00e9c0 con 0x55a5313cb9f0 2024-08-30T09:06:23.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.408+0000 7f7a9d4f9640 1 -- 172.21.15.42:0/3915129377 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a5313ec390 msgr2=0x55a531839790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:06:23.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.408+0000 7f7a9d4f9640 1 --2- 172.21.15.42:0/3915129377 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a5313ec390 0x55a531839790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:23.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.408+0000 7f7a9d4f9640 1 -- 172.21.15.42:0/3915129377 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a5313e41b0 msgr2=0x55a5317c2a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:06:23.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.408+0000 7f7a9d4f9640 1 --2- 172.21.15.42:0/3915129377 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a5313e41b0 0x55a5317c2a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:23.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.408+0000 7f7a9d4f9640 1 -- 172.21.15.42:0/3915129377 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a531839cd0 con 0x55a5313cb9f0 2024-08-30T09:06:23.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.408+0000 7f7a9dcfa640 1 --2- 172.21.15.42:0/3915129377 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a5313ec390 0x55a531839790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-30T09:06:23.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.409+0000 7f7a9d4f9640 1 --2- 172.21.15.42:0/3915129377 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a5313cb9f0 0x55a5317c2520 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f7a94010ca0 tx=0x7f7a9400b530 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a96702042dd15c1c server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:06:23.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.409+0000 7f7a80ff9640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-08-30T09:06:23.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.409+0000 7f7aa5cd2740 10 client.0 _ll_register_callbacks cb 0x7f7a88003630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:06:23.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.409+0000 7f7aa5cd2740 1 -- 172.21.15.42:0/3915129377 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7a90041c70 con 0x55a5313cb9f0 2024-08-30T09:06:23.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.409+0000 7f7a80ff9640 1 -- 172.21.15.42:0/3915129377 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a9401b630 con 0x55a5313cb9f0 2024-08-30T09:06:23.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.410+0000 7f7a80ff9640 1 -- 172.21.15.42:0/3915129377 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f7a94005040 con 0x55a5313cb9f0 2024-08-30T09:06:23.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.410+0000 7f7a80ff9640 1 -- 172.21.15.42:0/3915129377 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f7a94037050 con 0x55a5313cb9f0 2024-08-30T09:06:23.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.410+0000 7f7aa5cd2740 10 client.26125 fetch_fsmap learned FSMap version 33 2024-08-30T09:06:23.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.410+0000 7f7aa5cd2740 1 -- 172.21.15.42:0/3915129377 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({fsmap=33}) -- 0x55a53183ad50 con 0x55a5313cb9f0 2024-08-30T09:06:23.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.411+0000 7f7a80ff9640 1 -- 172.21.15.42:0/3915129377 <== mon.1 v2:172.21.15.190:3300/0 4 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f7a94005240 con 0x55a5313cb9f0 2024-08-30T09:06:23.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.411+0000 7f7aa5cd2740 10 client.26125 fetch_fsmap finished waiting for FSMap version 33 2024-08-30T09:06:23.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.411+0000 7f7aa5cd2740 10 client.26125 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-30T09:06:23.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.411+0000 7f7aa5cd2740 20 client.26125 populate_metadata read hostname 'smithi042' 2024-08-30T09:06:23.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.411+0000 7f7aa5cd2740 1 --2- 172.21.15.42:0/3915129377 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55a53183b150 0x55a53183d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:23.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.411+0000 7f7aa5cd2740 4 client.26125 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-30T09:06:23.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.411+0000 7f7aa5cd2740 1 -- 172.21.15.42:0/3915129377 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f7a90041c70 con 0x55a53183b150 2024-08-30T09:06:23.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.411+0000 7f7a9ccf8640 1 --2- 172.21.15.42:0/3915129377 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55a53183b150 0x55a53183d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:06:23.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.412+0000 7f7a9ccf8640 1 --2- 172.21.15.42:0/3915129377 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55a53183b150 0x55a53183d630 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f0203a9fe756e99 server_cookie=45c7b7cb5ec94fda in_seq=0 out_seq=0 2024-08-30T09:06:23.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.412+0000 7f7a80ff9640 10 client.26125 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:06:23.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.412+0000 7f7a80ff9640 1 -- 172.21.15.42:0/3915129377 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+2078 (crc 0 0 0) 0x7f7a90041c70 con 0x55a53183b150 2024-08-30T09:06:23.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.412+0000 7f7a80ff9640 10 client.26125 handle_command_reply: tid=0 2024-08-30T09:06:23.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.412+0000 7f7aa5cd2740 1 client.26125 shutdown 2024-08-30T09:06:23.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.412+0000 7f7aa5cd2740 1 -- 172.21.15.42:0/3915129377 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a5313cb9f0 msgr2=0x55a5317c2520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:06:23.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.412+0000 7f7aa5cd2740 1 --2- 172.21.15.42:0/3915129377 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a5313cb9f0 0x55a5317c2520 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f7a94010ca0 tx=0x7f7a9400b530 comp rx=0 tx=0).stop 2024-08-30T09:06:23.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.414+0000 7f7aa5cd2740 1 -- 172.21.15.42:0/3915129377 shutdown_connections 2024-08-30T09:06:23.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.414+0000 7f7aa5cd2740 1 --2- 172.21.15.42:0/3915129377 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55a53183b150 0x55a53183d630 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:23.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.414+0000 7f7aa5cd2740 1 --2- 172.21.15.42:0/3915129377 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a5313ec390 0x55a531839790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:23.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.414+0000 7f7a80ff9640 0 client.26125 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:06:23.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.414+0000 7f7aa5cd2740 1 --2- 172.21.15.42:0/3915129377 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a5313e41b0 0x55a5317c2a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:23.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.414+0000 7f7aa5cd2740 1 --2- 172.21.15.42:0/3915129377 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a5313cb9f0 0x55a5317c2520 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:23.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.414+0000 7f7aa5cd2740 1 -- 172.21.15.42:0/3915129377 >> 172.21.15.42:0/3915129377 conn(0x55a531401460 msgr2=0x55a5317afad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:06:23.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.414+0000 7f7aa5cd2740 1 -- 172.21.15.42:0/3915129377 shutdown_connections 2024-08-30T09:06:23.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.415+0000 7f7aa5cd2740 1 -- 172.21.15.42:0/3915129377 wait complete. 2024-08-30T09:06:23.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.415+0000 7f7aa5cd2740 20 client.26125 trim_cache size 0 max 16384 2024-08-30T09:06:23.418 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-08-30T09:06:23.418 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "no active scrubs running", 2024-08-30T09:06:23.418 INFO:teuthology.orchestra.run.smithi042.stdout: "scrubs": {}, 2024-08-30T09:06:23.418 INFO:teuthology.orchestra.run.smithi042.stdout: "scrub_stats": { 2024-08-30T09:06:23.418 INFO:teuthology.orchestra.run.smithi042.stdout: "1af4252a-4a99-47e9-bbf0-becf5e573dd5": { 2024-08-30T09:06:23.418 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:05:48.565881+0000", 2024-08-30T09:06:23.418 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:06:23.418 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:06:23.418 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:06:23.418 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:06:23.418 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:06:23.418 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:06:23.419 INFO:teuthology.orchestra.run.smithi042.stdout: "347d05ed-4bd3-4590-b099-314f719aab39": { 2024-08-30T09:06:23.419 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:05:00.439721+0000", 2024-08-30T09:06:23.419 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:06:23.419 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:06:23.419 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:06:23.419 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:06:23.419 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:06:23.419 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:06:23.419 INFO:teuthology.orchestra.run.smithi042.stdout: "88cd7478-2b20-47de-ba51-256479749232": { 2024-08-30T09:06:23.419 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:04:08.922625+0000", 2024-08-30T09:06:23.419 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:06:23.419 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:06:23.419 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:06:23.419 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:06:23.419 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:06:23.420 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:06:23.420 INFO:teuthology.orchestra.run.smithi042.stdout: "749f4867-0b70-49e5-af2d-8362a240ebc3": { 2024-08-30T09:06:23.420 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:03:16.386554+0000", 2024-08-30T09:06:23.420 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:06:23.420 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:06:23.420 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:06:23.420 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:06:23.420 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:06:23.420 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:06:23.420 INFO:teuthology.orchestra.run.smithi042.stdout: "85134e4b-ab4a-4885-b07e-a3ad01eaf13d": { 2024-08-30T09:06:23.420 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:01:27.738462+0000", 2024-08-30T09:06:23.420 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:06:23.420 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:06:23.420 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:06:23.421 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:06:23.421 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:06:23.421 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:06:23.421 INFO:teuthology.orchestra.run.smithi042.stdout: "c827ad7a-44ee-4e1d-9bb3-51fb7ab00c73": { 2024-08-30T09:06:23.421 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:02:21.686517+0000", 2024-08-30T09:06:23.421 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:06:23.421 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:06:23.421 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:06:23.421 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:06:23.421 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:06:23.421 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:06:23.421 INFO:teuthology.orchestra.run.smithi042.stdout: "5e2ed104-0625-478a-a78b-584e7f4c2817": { 2024-08-30T09:06:23.421 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:00:33.928400+0000", 2024-08-30T09:06:23.421 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:06:23.421 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:06:23.422 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:06:23.422 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:06:23.422 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:06:23.422 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:06:23.422 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:06:23.422 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-08-30T09:06:23.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.416+0000 7f7a9f75d640 1 -- 172.21.15.42:0/811797890 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f7a840b4d40 msgr2=0x7f7a840b7250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:06:23.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.416+0000 7f7a9f75d640 1 --2- 172.21.15.42:0/811797890 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f7a840b4d40 0x7f7a840b7250 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f7a9001a4f0 tx=0x7f7a9001a7f0 comp rx=0 tx=0).stop 2024-08-30T09:06:23.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.416+0000 7f7a9f75d640 1 -- 172.21.15.42:0/811797890 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7a98111ce0 msgr2=0x7f7a98118480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:06:23.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.416+0000 7f7a9f75d640 1 --2- 172.21.15.42:0/811797890 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7a98111ce0 0x7f7a98118480 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f7a9806c480 tx=0x7f7a8800db30 comp rx=0 tx=0).stop 2024-08-30T09:06:23.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.417+0000 7f7a9f75d640 1 -- 172.21.15.42:0/811797890 shutdown_connections 2024-08-30T09:06:23.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.417+0000 7f7a9f75d640 1 --2- 172.21.15.42:0/811797890 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f7a840b4d40 0x7f7a840b7250 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:23.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.417+0000 7f7a9f75d640 1 --2- 172.21.15.42:0/811797890 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7a9811c9d0 0x7f7a981189e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:23.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.417+0000 7f7a9f75d640 1 --2- 172.21.15.42:0/811797890 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7a98111ce0 0x7f7a98118480 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:23.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.417+0000 7f7a9f75d640 1 --2- 172.21.15.42:0/811797890 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7a9810a780 0x7f7a9811c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:23.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.417+0000 7f7a9f75d640 1 -- 172.21.15.42:0/811797890 >> 172.21.15.42:0/811797890 conn(0x7f7a98070c60 msgr2=0x7f7a98073bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:06:23.424 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.417+0000 7f7a9f75d640 1 -- 172.21.15.42:0/811797890 shutdown_connections 2024-08-30T09:06:23.424 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:23.417+0000 7f7a9f75d640 1 -- 172.21.15.42:0/811797890 wait complete. 2024-08-30T09:06:24.251 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-08-30T09:06:24.252 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph fs dump --format=json 2024-08-30T09:06:24.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:06:24 smithi167 ceph-mon[22063]: pgmap v542: 129 pgs: 129 active+clean; 1.3 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 12 MiB/s wr, 617 op/s 2024-08-30T09:06:24.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:06:24 smithi042 ceph-mon[18652]: pgmap v542: 129 pgs: 129 active+clean; 1.3 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 12 MiB/s wr, 617 op/s 2024-08-30T09:06:24.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:06:24 smithi190 ceph-mon[22001]: pgmap v542: 129 pgs: 129 active+clean; 1.3 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 12 MiB/s wr, 617 op/s 2024-08-30T09:06:24.716 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T09:06:25.954 INFO:teuthology.orchestra.run.smithi042.stderr:progress: 340.0 s, 594.2 tps, lat 163.966 ms stddev 205.380 2024-08-30T09:06:26.447 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:25.401+0000 7f11f0344640 1 -- 172.21.15.42:0/1206997996 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f11e8102800 msgr2=0x7f11e8102c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:06:26.447 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:25.401+0000 7f11f0344640 1 --2- 172.21.15.42:0/1206997996 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f11e8102800 0x7f11e8102c10 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f11d8009ca0 tx=0x7f11d802a8c0 comp rx=0 tx=0).stop 2024-08-30T09:06:26.447 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:25.401+0000 7f11f0344640 1 -- 172.21.15.42:0/1206997996 shutdown_connections 2024-08-30T09:06:26.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:25.401+0000 7f11f0344640 1 --2- 172.21.15.42:0/1206997996 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f11e8108140 0x7f11e8108550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:26.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:25.401+0000 7f11f0344640 1 --2- 172.21.15.42:0/1206997996 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f11e8108a90 0x7f11e810f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:26.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:25.401+0000 7f11f0344640 1 --2- 172.21.15.42:0/1206997996 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f11e8102800 0x7f11e8102c10 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:26.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:25.401+0000 7f11f0344640 1 -- 172.21.15.42:0/1206997996 >> 172.21.15.42:0/1206997996 conn(0x7f11e80794d0 msgr2=0x7f11e80798e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:06:26.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:25.401+0000 7f11f0344640 1 -- 172.21.15.42:0/1206997996 shutdown_connections 2024-08-30T09:06:26.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:25.401+0000 7f11f0344640 1 -- 172.21.15.42:0/1206997996 wait complete. 2024-08-30T09:06:26.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:25.402+0000 7f11f0344640 1 Processor -- start 2024-08-30T09:06:26.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:25.402+0000 7f11f0344640 1 -- start start 2024-08-30T09:06:26.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:25.402+0000 7f11f0344640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f11e8102800 0x7f11e819a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:26.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:25.403+0000 7f11f0344640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f11e8108140 0x7f11e819a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:26.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:25.403+0000 7f11f0344640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f11e8108a90 0x7f11e819fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:26.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:25.403+0000 7f11f0344640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f11e8112710 con 0x7f11e8108140 2024-08-30T09:06:26.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:25.403+0000 7f11f0344640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f11e8112590 con 0x7f11e8102800 2024-08-30T09:06:26.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:25.403+0000 7f11f0344640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f11e8112890 con 0x7f11e8108a90 2024-08-30T09:06:26.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:25.403+0000 7f11ed8df640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f11e8108140 0x7f11e819a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:06:26.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:25.403+0000 7f11ed8df640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f11e8108140 0x7f11e819a8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:58322/0 (socket says 172.21.15.42:58322) 2024-08-30T09:06:26.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:25.403+0000 7f11ed8df640 1 -- 172.21.15.42:0/2314697294 learned_addr learned my addr 172.21.15.42:0/2314697294 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:06:26.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:25.403+0000 7f11ee0e0640 1 --2- 172.21.15.42:0/2314697294 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f11e8102800 0x7f11e819a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:06:26.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:25.403+0000 7f11ee8e1640 1 --2- 172.21.15.42:0/2314697294 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f11e8108a90 0x7f11e819fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:06:26.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:25.403+0000 7f11ed8df640 1 -- 172.21.15.42:0/2314697294 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f11e8108a90 msgr2=0x7f11e819fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:06:26.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:25.403+0000 7f11ed8df640 1 --2- 172.21.15.42:0/2314697294 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f11e8108a90 0x7f11e819fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:26.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:25.403+0000 7f11ed8df640 1 -- 172.21.15.42:0/2314697294 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f11e8102800 msgr2=0x7f11e819a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:06:26.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:25.403+0000 7f11ed8df640 1 --2- 172.21.15.42:0/2314697294 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f11e8102800 0x7f11e819a330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:26.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:25.403+0000 7f11ed8df640 1 -- 172.21.15.42:0/2314697294 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11e81a01e0 con 0x7f11e8108140 2024-08-30T09:06:26.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:25.403+0000 7f11ee8e1640 1 --2- 172.21.15.42:0/2314697294 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f11e8108a90 0x7f11e819fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-30T09:06:26.450 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-08-30T09:06:26.450 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":33,"btime":"2024-08-30T09:06:09:250638+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14703,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/3188400000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":3188400000},{"type":"v1","addr":"172.21.15.42:6835","nonce":3188400000}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14715,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/3356537674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":3356537674},{"type":"v1","addr":"172.21.15.42:6837","nonce":3356537674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14766,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6839/2107785513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6838","nonce":2107785513},{"type":"v1","addr":"172.21.15.42:6839","nonce":2107785513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14778,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6841/2298438674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6840","nonce":2298438674},{"type":"v1","addr":"172.21.15.42:6841","nonce":2298438674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24553,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/543555570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":543555570},{"type":"v1","addr":"172.21.15.190:6833","nonce":543555570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24554,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/3907818941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":3907818941},{"type":"v1","addr":"172.21.15.167:6833","nonce":3907818941}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":33,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-30T08:58:03.055084+0000","modified":"2024-08-30T09:06:08.327698+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24629,"mds_1":24610,"mds_2":24599},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24572":{"gid":24572,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6835/234596845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":234596845},{"type":"v1","addr":"172.21.15.167:6835","nonce":234596845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24592":{"gid":24592,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6835/247905709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":247905709},{"type":"v1","addr":"172.21.15.190:6835","nonce":247905709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24598":{"gid":24598,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6837/863256349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6836","nonce":863256349},{"type":"v1","addr":"172.21.15.190:6837","nonce":863256349}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24599":{"gid":24599,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.167:6837/3011837856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":3011837856},{"type":"v1","addr":"172.21.15.167:6837","nonce":3011837856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24610":{"gid":24610,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.190:6839/719981110","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6838","nonce":719981110},{"type":"v1","addr":"172.21.15.190:6839","nonce":719981110}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24629":{"gid":24629,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.167:6839/3604645207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":3604645207},{"type":"v1","addr":"172.21.15.167:6839","nonce":3604645207}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24629,"qdb_cluster":[24599,24610,24629]},"id":1}]} 2024-08-30T09:06:26.450 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:25.403+0000 7f11ed8df640 1 --2- 172.21.15.42:0/2314697294 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f11e8108140 0x7f11e819a8d0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f11dc00c970 tx=0x7f11dc00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:06:26.450 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:25.403+0000 7f11e77fe640 1 -- 172.21.15.42:0/2314697294 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11dc007bb0 con 0x7f11e8108140 2024-08-30T09:06:26.450 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:25.404+0000 7f11f0344640 1 -- 172.21.15.42:0/2314697294 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11e81a04d0 con 0x7f11e8108140 2024-08-30T09:06:26.450 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:25.404+0000 7f11e77fe640 1 -- 172.21.15.42:0/2314697294 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11dc007d50 con 0x7f11e8108140 2024-08-30T09:06:26.450 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:25.404+0000 7f11e77fe640 1 -- 172.21.15.42:0/2314697294 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11dc018730 con 0x7f11e8108140 2024-08-30T09:06:26.450 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:25.404+0000 7f11f0344640 1 -- 172.21.15.42:0/2314697294 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11e81a6cf0 con 0x7f11e8108140 2024-08-30T09:06:26.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:25.405+0000 7f11f0344640 1 -- 172.21.15.42:0/2314697294 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11e8102c10 con 0x7f11e8108140 2024-08-30T09:06:26.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:25.408+0000 7f11e77fe640 1 -- 172.21.15.42:0/2314697294 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7f11dc00f990 con 0x7f11e8108140 2024-08-30T09:06:26.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:25.409+0000 7f11e77fe640 1 --2- 172.21.15.42:0/2314697294 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f11d00b4de0 0x7f11d00b72f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:26.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:25.409+0000 7f11e77fe640 1 -- 172.21.15.42:0/2314697294 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f11dc0184d0 con 0x7f11e8108140 2024-08-30T09:06:26.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:25.410+0000 7f11ee0e0640 1 --2- 172.21.15.42:0/2314697294 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f11d00b4de0 0x7f11d00b72f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:06:26.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:25.411+0000 7f11ee0e0640 1 --2- 172.21.15.42:0/2314697294 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f11d00b4de0 0x7f11d00b72f0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f11d8009a60 tx=0x7f11d80093e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:06:26.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:25.417+0000 7f11e77fe640 1 -- 172.21.15.42:0/2314697294 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f11dc014030 con 0x7f11e8108140 2024-08-30T09:06:26.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:25.781+0000 7f11f0344640 1 -- 172.21.15.42:0/2314697294 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f11e819b460 con 0x7f11e8108140 2024-08-30T09:06:26.451 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 33 2024-08-30T09:06:26.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:25.787+0000 7f11e77fe640 1 -- 172.21.15.42:0/2314697294 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+11918 (secure 0 0 0) 0x7f11e819b460 con 0x7f11e8108140 2024-08-30T09:06:26.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:25.791+0000 7f11f0344640 1 -- 172.21.15.42:0/2314697294 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f11d00b4de0 msgr2=0x7f11d00b72f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:06:26.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:25.791+0000 7f11f0344640 1 --2- 172.21.15.42:0/2314697294 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f11d00b4de0 0x7f11d00b72f0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f11d8009a60 tx=0x7f11d80093e0 comp rx=0 tx=0).stop 2024-08-30T09:06:26.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:25.791+0000 7f11f0344640 1 -- 172.21.15.42:0/2314697294 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f11e8108140 msgr2=0x7f11e819a8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:06:26.452 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:25.791+0000 7f11f0344640 1 --2- 172.21.15.42:0/2314697294 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f11e8108140 0x7f11e819a8d0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f11dc00c970 tx=0x7f11dc00ce30 comp rx=0 tx=0).stop 2024-08-30T09:06:26.452 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:25.792+0000 7f11f0344640 1 -- 172.21.15.42:0/2314697294 shutdown_connections 2024-08-30T09:06:26.452 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:25.792+0000 7f11f0344640 1 --2- 172.21.15.42:0/2314697294 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f11d00b4de0 0x7f11d00b72f0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:26.452 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:25.792+0000 7f11f0344640 1 --2- 172.21.15.42:0/2314697294 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f11e8108a90 0x7f11e819fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:26.452 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:25.792+0000 7f11f0344640 1 --2- 172.21.15.42:0/2314697294 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f11e8108140 0x7f11e819a8d0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:26.452 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:25.792+0000 7f11f0344640 1 --2- 172.21.15.42:0/2314697294 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f11e8102800 0x7f11e819a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:26.452 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:25.792+0000 7f11f0344640 1 -- 172.21.15.42:0/2314697294 >> 172.21.15.42:0/2314697294 conn(0x7f11e80794d0 msgr2=0x7f11e8100360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:06:26.452 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:25.792+0000 7f11f0344640 1 -- 172.21.15.42:0/2314697294 shutdown_connections 2024-08-30T09:06:26.452 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:25.792+0000 7f11f0344640 1 -- 172.21.15.42:0/2314697294 wait complete. 2024-08-30T09:06:26.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:06:26 smithi167 ceph-mon[22063]: pgmap v543: 129 pgs: 129 active+clean; 1.3 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 10 MiB/s wr, 504 op/s 2024-08-30T09:06:26.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:06:26 smithi167 ceph-mon[22063]: from='client.? 172.21.15.42:0/2314697294' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-30T09:06:26.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:06:26 smithi042 ceph-mon[18652]: pgmap v543: 129 pgs: 129 active+clean; 1.3 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 10 MiB/s wr, 504 op/s 2024-08-30T09:06:26.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:06:26 smithi042 ceph-mon[18652]: from='client.? 172.21.15.42:0/2314697294' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-30T09:06:26.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:06:26 smithi190 ceph-mon[22001]: pgmap v543: 129 pgs: 129 active+clean; 1.3 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 10 MiB/s wr, 504 op/s 2024-08-30T09:06:26.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:06:26 smithi190 ceph-mon[22001]: from='client.? 172.21.15.42:0/2314697294' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-30T09:06:27.263 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-08-30T09:06:27.717 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T09:06:28.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:06:28 smithi042 ceph-mon[18652]: pgmap v544: 129 pgs: 129 active+clean; 1.4 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 MiB/s wr, 568 op/s 2024-08-30T09:06:28.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.467+0000 7f3429a46640 1 -- 172.21.15.42:0/3049064423 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f342410a780 msgr2=0x7f342410ab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:06:28.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.467+0000 7f3429a46640 1 --2- 172.21.15.42:0/3049064423 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f342410a780 0x7f342410ab90 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f341c009ca0 tx=0x7f341c02a8f0 comp rx=0 tx=0).stop 2024-08-30T09:06:28.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.471+0000 7f3429a46640 1 -- 172.21.15.42:0/3049064423 shutdown_connections 2024-08-30T09:06:28.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.471+0000 7f3429a46640 1 --2- 172.21.15.42:0/3049064423 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f342411c9d0 0x7f342411eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:28.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.471+0000 7f3429a46640 1 --2- 172.21.15.42:0/3049064423 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f342410a780 0x7f342410ab90 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:28.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.471+0000 7f3429a46640 1 --2- 172.21.15.42:0/3049064423 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3424111ce0 0x7f342410a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:28.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.471+0000 7f3429a46640 1 -- 172.21.15.42:0/3049064423 >> 172.21.15.42:0/3049064423 conn(0x7f3424070c60 msgr2=0x7f3424071070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:06:28.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.475+0000 7f3429a46640 1 -- 172.21.15.42:0/3049064423 shutdown_connections 2024-08-30T09:06:28.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.476+0000 7f3429a46640 1 -- 172.21.15.42:0/3049064423 wait complete. 2024-08-30T09:06:28.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.478+0000 7f3429a46640 1 Processor -- start 2024-08-30T09:06:28.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.482+0000 7f3429a46640 1 -- start start 2024-08-30T09:06:28.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.482+0000 7f3429a46640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f342410a780 0x7f34241a90f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:28.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.482+0000 7f3429a46640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3424111ce0 0x7f34241a9690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:28.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.482+0000 7f3429a46640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f342411c9d0 0x7f34241ae930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:28.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.482+0000 7f3429a46640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f3424122700 con 0x7f3424111ce0 2024-08-30T09:06:28.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.482+0000 7f3429a46640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f3424122580 con 0x7f342411c9d0 2024-08-30T09:06:28.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.482+0000 7f3429a46640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f3424122880 con 0x7f342410a780 2024-08-30T09:06:28.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.483+0000 7f34227fc640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3424111ce0 0x7f34241a9690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:06:28.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.483+0000 7f34227fc640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3424111ce0 0x7f34241a9690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:45492/0 (socket says 172.21.15.42:45492) 2024-08-30T09:06:28.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.483+0000 7f34227fc640 1 -- 172.21.15.42:0/3628142501 learned_addr learned my addr 172.21.15.42:0/3628142501 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:06:28.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.483+0000 7f3422ffd640 1 --2- 172.21.15.42:0/3628142501 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f342410a780 0x7f34241a90f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:06:28.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.483+0000 7f34237fe640 1 --2- 172.21.15.42:0/3628142501 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f342411c9d0 0x7f34241ae930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:06:28.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.487+0000 7f34237fe640 1 -- 172.21.15.42:0/3628142501 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f342410a780 msgr2=0x7f34241a90f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:06:28.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.487+0000 7f34237fe640 1 --2- 172.21.15.42:0/3628142501 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f342410a780 0x7f34241a90f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:28.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.487+0000 7f34237fe640 1 -- 172.21.15.42:0/3628142501 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3424111ce0 msgr2=0x7f34241a9690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:06:28.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.487+0000 7f34237fe640 1 --2- 172.21.15.42:0/3628142501 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3424111ce0 0x7f34241a9690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:28.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.487+0000 7f34237fe640 1 -- 172.21.15.42:0/3628142501 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34241af060 con 0x7f342411c9d0 2024-08-30T09:06:28.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.487+0000 7f34237fe640 1 --2- 172.21.15.42:0/3628142501 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f342411c9d0 0x7f34241ae930 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f341800d680 tx=0x7f341800db40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:06:28.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.490+0000 7f3428a44640 1 -- 172.21.15.42:0/3628142501 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3418017070 con 0x7f342411c9d0 2024-08-30T09:06:28.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.491+0000 7f3428a44640 1 -- 172.21.15.42:0/3628142501 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3418009820 con 0x7f342411c9d0 2024-08-30T09:06:28.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.491+0000 7f3428a44640 1 -- 172.21.15.42:0/3628142501 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3418004ce0 con 0x7f342411c9d0 2024-08-30T09:06:28.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.491+0000 7f3429a46640 1 -- 172.21.15.42:0/3628142501 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34241af350 con 0x7f342411c9d0 2024-08-30T09:06:28.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.491+0000 7f3429a46640 1 -- 172.21.15.42:0/3628142501 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34241b5ab0 con 0x7f342411c9d0 2024-08-30T09:06:28.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.498+0000 7f342ffbb740 1 Processor -- start 2024-08-30T09:06:28.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.500+0000 7f342ffbb740 1 -- start start 2024-08-30T09:06:28.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.500+0000 7f342ffbb740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5561bc8d5070 0x5561bcc7a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:28.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.500+0000 7f342ffbb740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5561bc8ac1b0 0x5561bcc7a890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:28.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.501+0000 7f342ffbb740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5561bc8b4390 0x5561bcc7add0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:28.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.501+0000 7f342ffbb740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x5561bcbf1680 con 0x5561bc8b4390 2024-08-30T09:06:28.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.501+0000 7f342ffbb740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5561bcbe0c10 con 0x5561bc8ac1b0 2024-08-30T09:06:28.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.501+0000 7f342ffbb740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5561bcbe4bc0 con 0x5561bc8d5070 2024-08-30T09:06:28.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.502+0000 7f3422ffd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5561bc8ac1b0 0x5561bcc7a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:06:28.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.502+0000 7f3422ffd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5561bc8ac1b0 0x5561bcc7a890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:56502/0 (socket says 172.21.15.42:56502) 2024-08-30T09:06:28.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.502+0000 7f3428a44640 1 -- 172.21.15.42:0/3628142501 <== mon.2 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7f3418005220 con 0x7f342411c9d0 2024-08-30T09:06:28.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.502+0000 7f3422ffd640 1 -- 172.21.15.42:0/1454857819 learned_addr learned my addr 172.21.15.42:0/1454857819 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:06:28.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.503+0000 7f34227fc640 1 --2- 172.21.15.42:0/1454857819 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5561bc8d5070 0x5561bcc7a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:06:28.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.504+0000 7f3422ffd640 1 -- 172.21.15.42:0/1454857819 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5561bc8d5070 msgr2=0x5561bcc7a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:06:28.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.504+0000 7f3422ffd640 1 --2- 172.21.15.42:0/1454857819 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5561bc8d5070 0x5561bcc7a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:28.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.504+0000 7f3422ffd640 1 -- 172.21.15.42:0/1454857819 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5561bc8b4390 msgr2=0x5561bcc7add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:06:28.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.504+0000 7f3422ffd640 1 --2- 172.21.15.42:0/1454857819 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5561bc8b4390 0x5561bcc7add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:28.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.504+0000 7f3422ffd640 1 -- 172.21.15.42:0/1454857819 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5561bc940650 con 0x5561bc8ac1b0 2024-08-30T09:06:28.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.504+0000 7f3422ffd640 1 --2- 172.21.15.42:0/1454857819 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5561bc8ac1b0 0x5561bcc7a890 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f3414009a40 tx=0x7f3414009f00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6547a7a1717fc951 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:06:28.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.504+0000 7f3428a44640 1 --2- 172.21.15.42:0/3628142501 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f340c0b4d40 0x7f340c0b7250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:28.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.505+0000 7f3409ffb640 1 -- 172.21.15.42:0/1454857819 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3414017070 con 0x5561bc8ac1b0 2024-08-30T09:06:28.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.505+0000 7f3409ffb640 1 -- 172.21.15.42:0/1454857819 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3414013470 con 0x5561bc8ac1b0 2024-08-30T09:06:28.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.505+0000 7f3428a44640 1 -- 172.21.15.42:0/3628142501 <== mon.2 v2:172.21.15.167:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f34180e4bd0 con 0x7f342411c9d0 2024-08-30T09:06:28.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.506+0000 7f342ffbb740 1 -- 172.21.15.42:0/1454857819 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5561bc8ac1b0 msgr2=0x5561bcc7a890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:06:28.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.506+0000 7f342ffbb740 1 --2- 172.21.15.42:0/1454857819 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5561bc8ac1b0 0x5561bcc7a890 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f3414009a40 tx=0x7f3414009f00 comp rx=0 tx=0).stop 2024-08-30T09:06:28.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.506+0000 7f342ffbb740 1 -- 172.21.15.42:0/1454857819 shutdown_connections 2024-08-30T09:06:28.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.506+0000 7f342ffbb740 1 --2- 172.21.15.42:0/1454857819 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5561bc8b4390 0x5561bcc7add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:28.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.506+0000 7f342ffbb740 1 --2- 172.21.15.42:0/1454857819 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5561bc8ac1b0 0x5561bcc7a890 secure :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f3414009a40 tx=0x7f3414009f00 comp rx=0 tx=0).stop 2024-08-30T09:06:28.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.506+0000 7f342ffbb740 1 --2- 172.21.15.42:0/1454857819 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5561bc8d5070 0x5561bcc7a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:28.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.506+0000 7f342ffbb740 1 -- 172.21.15.42:0/1454857819 >> 172.21.15.42:0/1454857819 conn(0x5561bc8c9460 msgr2=0x5561bcc78240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:06:28.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.508+0000 7f34227fc640 1 --2- 172.21.15.42:0/3628142501 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f340c0b4d40 0x7f340c0b7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:06:28.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.510+0000 7f34227fc640 1 --2- 172.21.15.42:0/3628142501 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f340c0b4d40 0x7f340c0b7250 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f342411c180 tx=0x7f341c005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:06:28.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.510+0000 7f342ffbb740 1 -- 172.21.15.42:0/1454857819 shutdown_connections 2024-08-30T09:06:28.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.511+0000 7f342ffbb740 1 -- 172.21.15.42:0/1454857819 wait complete. 2024-08-30T09:06:28.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.511+0000 7f342ffbb740 1 -- start start 2024-08-30T09:06:28.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.512+0000 7f342ffbb740 1 Processor -- start 2024-08-30T09:06:28.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.512+0000 7f342ffbb740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5561bc8b4390 0x5561bcc85b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:28.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.512+0000 7f342ffbb740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5561bc8d5070 0x5561bcc86090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:28.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.512+0000 7f342ffbb740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5561bc8939f0 0x5561bcc91f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:28.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.512+0000 7f342ffbb740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x5561bca43700 con 0x5561bc8b4390 2024-08-30T09:06:28.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.512+0000 7f342ffbb740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5561bcb67750 con 0x5561bc8d5070 2024-08-30T09:06:28.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.512+0000 7f342ffbb740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5561bcc88b80 con 0x5561bc8939f0 2024-08-30T09:06:28.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.512+0000 7f342ffbb740 10 client.0 _ll_register_callbacks cb 0x5561bcb96e10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:06:28.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.513+0000 7f34227fc640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5561bc8d5070 0x5561bcc86090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:06:28.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.513+0000 7f34237fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5561bc8939f0 0x5561bcc91f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:06:28.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.513+0000 7f34237fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5561bc8939f0 0x5561bcc91f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:32904/0 (socket says 172.21.15.42:32904) 2024-08-30T09:06:28.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.513+0000 7f34237fe640 1 -- 172.21.15.42:0/1111119026 learned_addr learned my addr 172.21.15.42:0/1111119026 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:06:28.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.513+0000 7f34237fe640 1 -- 172.21.15.42:0/1111119026 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5561bc8d5070 msgr2=0x5561bcc86090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:06:28.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.513+0000 7f34237fe640 1 --2- 172.21.15.42:0/1111119026 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5561bc8d5070 0x5561bcc86090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:28.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.513+0000 7f34237fe640 1 -- 172.21.15.42:0/1111119026 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5561bc8b4390 msgr2=0x5561bcc85b30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-30T09:06:28.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.513+0000 7f34237fe640 1 --2- 172.21.15.42:0/1111119026 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5561bc8b4390 0x5561bcc85b30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:28.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.513+0000 7f34237fe640 1 -- 172.21.15.42:0/1111119026 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5561bcba0900 con 0x5561bc8939f0 2024-08-30T09:06:28.519 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.515+0000 7f34237fe640 1 --2- 172.21.15.42:0/1111119026 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5561bc8939f0 0x5561bcc91f50 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f34180a7290 tx=0x7f3418005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=31782222c6327d29 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:06:28.519 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.516+0000 7f3402ffd640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-08-30T09:06:28.519 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.517+0000 7f3402ffd640 1 -- 172.21.15.42:0/1111119026 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3418020040 con 0x5561bc8939f0 2024-08-30T09:06:28.519 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.517+0000 7f3402ffd640 1 -- 172.21.15.42:0/1111119026 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f34180a7800 con 0x5561bc8939f0 2024-08-30T09:06:28.519 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.517+0000 7f342ffbb740 1 -- 172.21.15.42:0/1111119026 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5561bcbf35d0 con 0x5561bc8939f0 2024-08-30T09:06:28.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.517+0000 7f3402ffd640 1 -- 172.21.15.42:0/1111119026 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f34180afa80 con 0x5561bc8939f0 2024-08-30T09:06:28.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.517+0000 7f342ffbb740 10 client.26137 fetch_fsmap learned FSMap version 33 2024-08-30T09:06:28.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.517+0000 7f342ffbb740 1 -- 172.21.15.42:0/1111119026 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5561bcc94b60 con 0x5561bc8939f0 2024-08-30T09:06:28.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.519+0000 7f3402ffd640 1 -- 172.21.15.42:0/1111119026 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5561bcbf35d0 con 0x5561bc8939f0 2024-08-30T09:06:28.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.519+0000 7f3402ffd640 1 -- 172.21.15.42:0/1111119026 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f34180cf060 con 0x5561bc8939f0 2024-08-30T09:06:28.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.519+0000 7f342ffbb740 10 client.26137 fetch_fsmap finished waiting for FSMap version 33 2024-08-30T09:06:28.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.519+0000 7f342ffbb740 10 client.26137 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-08-30T09:06:28.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.519+0000 7f342ffbb740 20 client.26137 populate_metadata read hostname 'smithi042' 2024-08-30T09:06:28.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.519+0000 7f342ffbb740 1 --2- 172.21.15.42:0/1111119026 >> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] conn(0x5561bc868da0 0x5561bcc98690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:28.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.519+0000 7f342ffbb740 4 client.26137 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-30T09:06:28.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.519+0000 7f342ffbb740 1 -- 172.21.15.42:0/1111119026 --> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5561bcbef070 con 0x5561bc868da0 2024-08-30T09:06:28.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.520+0000 7f3422ffd640 1 --2- 172.21.15.42:0/1111119026 >> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] conn(0x5561bc868da0 0x5561bcc98690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:06:28.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.521+0000 7f3422ffd640 1 --2- 172.21.15.42:0/1111119026 >> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] conn(0x5561bc868da0 0x5561bcc98690 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=21225cf429805fd1 server_cookie=4d05a79228637bce in_seq=0 out_seq=0 2024-08-30T09:06:28.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.521+0000 7f3402ffd640 10 client.26137 ms_handle_connect on v2:172.21.15.167:6836/3011837856 2024-08-30T09:06:28.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.523+0000 7f3402ffd640 1 -- 172.21.15.42:0/1111119026 <== mds.2 v2:172.21.15.167:6836/3011837856 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5561bcbef070 con 0x5561bc868da0 2024-08-30T09:06:28.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.523+0000 7f3402ffd640 10 client.26137 handle_command_reply: tid=0 2024-08-30T09:06:28.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.523+0000 7f342ffbb740 1 client.26137 shutdown 2024-08-30T09:06:28.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.523+0000 7f342ffbb740 1 -- 172.21.15.42:0/1111119026 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5561bc8939f0 msgr2=0x5561bcc91f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:06:28.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.523+0000 7f342ffbb740 1 --2- 172.21.15.42:0/1111119026 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5561bc8939f0 0x5561bcc91f50 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f34180a7290 tx=0x7f3418005740 comp rx=0 tx=0).stop 2024-08-30T09:06:28.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.523+0000 7f342ffbb740 1 -- 172.21.15.42:0/1111119026 shutdown_connections 2024-08-30T09:06:28.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.523+0000 7f342ffbb740 1 --2- 172.21.15.42:0/1111119026 >> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] conn(0x5561bc868da0 0x5561bcc98690 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:28.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.523+0000 7f342ffbb740 1 --2- 172.21.15.42:0/1111119026 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5561bc8939f0 0x5561bcc91f50 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:28.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.523+0000 7f342ffbb740 1 --2- 172.21.15.42:0/1111119026 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5561bc8d5070 0x5561bcc86090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:28.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.523+0000 7f342ffbb740 1 --2- 172.21.15.42:0/1111119026 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5561bc8b4390 0x5561bcc85b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:28.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.523+0000 7f342ffbb740 1 -- 172.21.15.42:0/1111119026 >> 172.21.15.42:0/1111119026 conn(0x5561bc8c9460 msgr2=0x5561bcc7d4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:06:28.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.523+0000 7f3402ffd640 0 client.26137 ms_handle_reset on v2:172.21.15.167:6836/3011837856 2024-08-30T09:06:28.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.524+0000 7f342ffbb740 1 -- 172.21.15.42:0/1111119026 shutdown_connections 2024-08-30T09:06:28.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.524+0000 7f342ffbb740 1 -- 172.21.15.42:0/1111119026 wait complete. 2024-08-30T09:06:28.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.524+0000 7f342ffbb740 20 client.26137 trim_cache size 0 max 16384 2024-08-30T09:06:28.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.547+0000 7f342ffbb740 1 Processor -- start 2024-08-30T09:06:28.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.547+0000 7f342ffbb740 1 -- start start 2024-08-30T09:06:28.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.547+0000 7f342ffbb740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5561bc8939f0 0x5561bcc91e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:28.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.547+0000 7f342ffbb740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5561bc8b4390 0x5561bcc92390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:28.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.547+0000 7f342ffbb740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5561bc8d5070 0x5561bcc7d450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:28.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.547+0000 7f342ffbb740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f34100125f0 con 0x5561bc8b4390 2024-08-30T09:06:28.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.547+0000 7f342ffbb740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f341000fe60 con 0x5561bc8d5070 2024-08-30T09:06:28.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.547+0000 7f342ffbb740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f341000dcb0 con 0x5561bc8939f0 2024-08-30T09:06:28.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.547+0000 7f34227fc640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5561bc8b4390 0x5561bcc92390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:06:28.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.547+0000 7f3422ffd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5561bc8939f0 0x5561bcc91e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:06:28.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.547+0000 7f34237fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5561bc8d5070 0x5561bcc7d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:06:28.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.547+0000 7f3422ffd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5561bc8939f0 0x5561bcc91e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:32920/0 (socket says 172.21.15.42:32920) 2024-08-30T09:06:28.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.547+0000 7f3422ffd640 1 -- 172.21.15.42:0/175467204 learned_addr learned my addr 172.21.15.42:0/175467204 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:06:28.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.548+0000 7f3422ffd640 1 -- 172.21.15.42:0/175467204 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5561bc8d5070 msgr2=0x5561bcc7d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:06:28.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.548+0000 7f3422ffd640 1 --2- 172.21.15.42:0/175467204 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5561bc8d5070 0x5561bcc7d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:28.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.548+0000 7f3422ffd640 1 -- 172.21.15.42:0/175467204 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5561bc8b4390 msgr2=0x5561bcc92390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:06:28.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.548+0000 7f3422ffd640 1 --2- 172.21.15.42:0/175467204 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5561bc8b4390 0x5561bcc92390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:28.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.548+0000 7f3422ffd640 1 -- 172.21.15.42:0/175467204 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5561bcc72380 con 0x5561bc8939f0 2024-08-30T09:06:28.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.549+0000 7f3422ffd640 1 --2- 172.21.15.42:0/175467204 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5561bc8939f0 0x5561bcc91e50 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f3414010010 tx=0x7f34140090e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=67008bf90dca92e0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:06:28.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.549+0000 7f34097fa640 1 -- 172.21.15.42:0/175467204 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3414017040 con 0x5561bc8939f0 2024-08-30T09:06:28.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.550+0000 7f34097fa640 1 -- 172.21.15.42:0/175467204 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3414027030 con 0x5561bc8939f0 2024-08-30T09:06:28.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.550+0000 7f34097fa640 1 -- 172.21.15.42:0/175467204 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3414012e50 con 0x5561bc8939f0 2024-08-30T09:06:28.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.550+0000 7f342ffbb740 1 -- 172.21.15.42:0/175467204 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5561bc8939f0 msgr2=0x5561bcc91e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:06:28.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.550+0000 7f342ffbb740 1 --2- 172.21.15.42:0/175467204 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5561bc8939f0 0x5561bcc91e50 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f3414010010 tx=0x7f34140090e0 comp rx=0 tx=0).stop 2024-08-30T09:06:28.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.551+0000 7f342ffbb740 1 -- 172.21.15.42:0/175467204 shutdown_connections 2024-08-30T09:06:28.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.551+0000 7f342ffbb740 1 --2- 172.21.15.42:0/175467204 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5561bc8d5070 0x5561bcc7d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:28.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.551+0000 7f342ffbb740 1 --2- 172.21.15.42:0/175467204 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5561bc8b4390 0x5561bcc92390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:28.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.551+0000 7f342ffbb740 1 --2- 172.21.15.42:0/175467204 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5561bc8939f0 0x5561bcc91e50 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:28.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.551+0000 7f342ffbb740 1 -- 172.21.15.42:0/175467204 >> 172.21.15.42:0/175467204 conn(0x5561bc8c9460 msgr2=0x5561bcc843b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:06:28.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.551+0000 7f342ffbb740 1 -- 172.21.15.42:0/175467204 shutdown_connections 2024-08-30T09:06:28.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.551+0000 7f342ffbb740 1 -- 172.21.15.42:0/175467204 wait complete. 2024-08-30T09:06:28.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.551+0000 7f342ffbb740 1 -- start start 2024-08-30T09:06:28.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.552+0000 7f342ffbb740 1 Processor -- start 2024-08-30T09:06:28.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.552+0000 7f342ffbb740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5561bc8939f0 0x5561bcc897f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:28.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.552+0000 7f342ffbb740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5561bc8b4390 0x5561bcc89dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:28.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.552+0000 7f3422ffd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5561bc8939f0 0x5561bcc897f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:06:28.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.552+0000 7f3422ffd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5561bc8939f0 0x5561bcc897f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:32924/0 (socket says 172.21.15.42:32924) 2024-08-30T09:06:28.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.552+0000 7f342ffbb740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5561bc8d5070 0x5561bcc8dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:28.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.553+0000 7f34227fc640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5561bc8b4390 0x5561bcc89dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:06:28.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.553+0000 7f34227fc640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5561bc8b4390 0x5561bcc89dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:56526/0 (socket says 172.21.15.42:56526) 2024-08-30T09:06:28.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.553+0000 7f3422ffd640 1 -- 172.21.15.42:0/1410957762 learned_addr learned my addr 172.21.15.42:0/1410957762 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:06:28.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.553+0000 7f342ffbb740 1 -- 172.21.15.42:0/1410957762 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f341000e320 con 0x5561bc8d5070 2024-08-30T09:06:28.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.553+0000 7f34237fe640 1 --2- 172.21.15.42:0/1410957762 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5561bc8d5070 0x5561bcc8dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:06:28.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.553+0000 7f342ffbb740 1 -- 172.21.15.42:0/1410957762 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f341000f790 con 0x5561bc8b4390 2024-08-30T09:06:28.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.553+0000 7f342ffbb740 1 -- 172.21.15.42:0/1410957762 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f341000e9f0 con 0x5561bc8939f0 2024-08-30T09:06:28.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.553+0000 7f3422ffd640 1 -- 172.21.15.42:0/1410957762 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5561bc8b4390 msgr2=0x5561bcc89dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:06:28.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.553+0000 7f3422ffd640 1 --2- 172.21.15.42:0/1410957762 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5561bc8b4390 0x5561bcc89dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:28.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.553+0000 7f3422ffd640 1 -- 172.21.15.42:0/1410957762 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5561bc8d5070 msgr2=0x5561bcc8dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:06:28.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.553+0000 7f3422ffd640 1 --2- 172.21.15.42:0/1410957762 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5561bc8d5070 0x5561bcc8dcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:28.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.553+0000 7f3422ffd640 1 -- 172.21.15.42:0/1410957762 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5561bcc8e330 con 0x5561bc8939f0 2024-08-30T09:06:28.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.554+0000 7f3422ffd640 1 --2- 172.21.15.42:0/1410957762 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5561bc8939f0 0x5561bcc897f0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f3414006fd0 tx=0x7f3414006a70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=10142a2ef76a90bb server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:06:28.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.554+0000 7f3402ffd640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-08-30T09:06:28.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.554+0000 7f3402ffd640 1 -- 172.21.15.42:0/1410957762 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3414030070 con 0x5561bc8939f0 2024-08-30T09:06:28.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.554+0000 7f3402ffd640 1 -- 172.21.15.42:0/1410957762 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f3414012b50 con 0x5561bc8939f0 2024-08-30T09:06:28.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.554+0000 7f342ffbb740 10 client.0 _ll_register_callbacks cb 0x7f34180aafe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:06:28.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.554+0000 7f342ffbb740 1 -- 172.21.15.42:0/1410957762 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3414027030 con 0x5561bc8939f0 2024-08-30T09:06:28.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.554+0000 7f3402ffd640 1 -- 172.21.15.42:0/1410957762 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f341403c050 con 0x5561bc8939f0 2024-08-30T09:06:28.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.554+0000 7f342ffbb740 10 client.26143 fetch_fsmap learned FSMap version 33 2024-08-30T09:06:28.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.554+0000 7f342ffbb740 1 -- 172.21.15.42:0/1410957762 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({fsmap=33}) -- 0x5561bcd02800 con 0x5561bc8939f0 2024-08-30T09:06:28.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.555+0000 7f3402ffd640 1 -- 172.21.15.42:0/1410957762 <== mon.1 v2:172.21.15.190:3300/0 4 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f3414012d50 con 0x5561bc8939f0 2024-08-30T09:06:28.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.555+0000 7f342ffbb740 10 client.26143 fetch_fsmap finished waiting for FSMap version 33 2024-08-30T09:06:28.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.555+0000 7f342ffbb740 10 client.26143 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-08-30T09:06:28.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.555+0000 7f342ffbb740 20 client.26143 populate_metadata read hostname 'smithi042' 2024-08-30T09:06:28.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.555+0000 7f342ffbb740 1 --2- 172.21.15.42:0/1410957762 >> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] conn(0x5561bcd02b40 0x5561bcd04f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:28.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.555+0000 7f342ffbb740 4 client.26143 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-30T09:06:28.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.555+0000 7f342ffbb740 1 -- 172.21.15.42:0/1410957762 --> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f3414027030 con 0x5561bcd02b40 2024-08-30T09:06:28.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.555+0000 7f34227fc640 1 --2- 172.21.15.42:0/1410957762 >> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] conn(0x5561bcd02b40 0x5561bcd04f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:06:28.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.556+0000 7f34227fc640 1 --2- 172.21.15.42:0/1410957762 >> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] conn(0x5561bcd02b40 0x5561bcd04f60 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=655524825c4cc3c3 server_cookie=db6140342dfade32 in_seq=0 out_seq=0 2024-08-30T09:06:28.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.556+0000 7f3402ffd640 10 client.26143 ms_handle_connect on v2:172.21.15.167:6836/3011837856 2024-08-30T09:06:28.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.556+0000 7f3402ffd640 1 -- 172.21.15.42:0/1410957762 <== mds.2 v2:172.21.15.167:6836/3011837856 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f3414027030 con 0x5561bcd02b40 2024-08-30T09:06:28.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.556+0000 7f3402ffd640 10 client.26143 handle_command_reply: tid=0 2024-08-30T09:06:28.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.556+0000 7f342ffbb740 1 client.26143 shutdown 2024-08-30T09:06:28.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.556+0000 7f342ffbb740 1 -- 172.21.15.42:0/1410957762 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5561bc8939f0 msgr2=0x5561bcc897f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:06:28.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.556+0000 7f342ffbb740 1 --2- 172.21.15.42:0/1410957762 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5561bc8939f0 0x5561bcc897f0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f3414006fd0 tx=0x7f3414006a70 comp rx=0 tx=0).stop 2024-08-30T09:06:28.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.556+0000 7f342ffbb740 1 -- 172.21.15.42:0/1410957762 shutdown_connections 2024-08-30T09:06:28.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.556+0000 7f342ffbb740 1 --2- 172.21.15.42:0/1410957762 >> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] conn(0x5561bcd02b40 0x5561bcd04f60 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:28.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.556+0000 7f342ffbb740 1 --2- 172.21.15.42:0/1410957762 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5561bc8d5070 0x5561bcc8dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:28.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.556+0000 7f342ffbb740 1 --2- 172.21.15.42:0/1410957762 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5561bc8b4390 0x5561bcc89dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:28.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.556+0000 7f342ffbb740 1 --2- 172.21.15.42:0/1410957762 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5561bc8939f0 0x5561bcc897f0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:28.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.556+0000 7f342ffbb740 1 -- 172.21.15.42:0/1410957762 >> 172.21.15.42:0/1410957762 conn(0x5561bc8c9460 msgr2=0x5561bcc77b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:06:28.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.556+0000 7f3402ffd640 0 client.26143 ms_handle_reset on v2:172.21.15.167:6836/3011837856 2024-08-30T09:06:28.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.556+0000 7f342ffbb740 1 -- 172.21.15.42:0/1410957762 shutdown_connections 2024-08-30T09:06:28.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.556+0000 7f342ffbb740 1 -- 172.21.15.42:0/1410957762 wait complete. 2024-08-30T09:06:28.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.556+0000 7f342ffbb740 20 client.26143 trim_cache size 0 max 16384 2024-08-30T09:06:28.562 INFO:teuthology.orchestra.run.smithi042.stdout:[] 2024-08-30T09:06:28.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.558+0000 7f3429a46640 1 -- 172.21.15.42:0/3628142501 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f340c0b4d40 msgr2=0x7f340c0b7250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:06:28.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.558+0000 7f3429a46640 1 --2- 172.21.15.42:0/3628142501 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f340c0b4d40 0x7f340c0b7250 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f342411c180 tx=0x7f341c005fd0 comp rx=0 tx=0).stop 2024-08-30T09:06:28.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.558+0000 7f3429a46640 1 -- 172.21.15.42:0/3628142501 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f342411c9d0 msgr2=0x7f34241ae930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:06:28.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.558+0000 7f3429a46640 1 --2- 172.21.15.42:0/3628142501 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f342411c9d0 0x7f34241ae930 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f341800d680 tx=0x7f341800db40 comp rx=0 tx=0).stop 2024-08-30T09:06:28.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.559+0000 7f3429a46640 1 -- 172.21.15.42:0/3628142501 shutdown_connections 2024-08-30T09:06:28.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.559+0000 7f3429a46640 1 --2- 172.21.15.42:0/3628142501 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f340c0b4d40 0x7f340c0b7250 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:28.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.559+0000 7f3429a46640 1 --2- 172.21.15.42:0/3628142501 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f342411c9d0 0x7f34241ae930 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:28.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.559+0000 7f3429a46640 1 --2- 172.21.15.42:0/3628142501 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3424111ce0 0x7f34241a9690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:28.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.559+0000 7f3429a46640 1 --2- 172.21.15.42:0/3628142501 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f342410a780 0x7f34241a90f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:28.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.559+0000 7f3429a46640 1 -- 172.21.15.42:0/3628142501 >> 172.21.15.42:0/3628142501 conn(0x7f3424070c60 msgr2=0x7f342411e220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:06:28.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.559+0000 7f3429a46640 1 -- 172.21.15.42:0/3628142501 shutdown_connections 2024-08-30T09:06:28.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:28.559+0000 7f3429a46640 1 -- 172.21.15.42:0/3628142501 wait complete. 2024-08-30T09:06:28.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:06:28 smithi167 ceph-mon[22063]: pgmap v544: 129 pgs: 129 active+clean; 1.4 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 MiB/s wr, 568 op/s 2024-08-30T09:06:28.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:06:28 smithi190 ceph-mon[22001]: pgmap v544: 129 pgs: 129 active+clean; 1.4 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 MiB/s wr, 568 op/s 2024-08-30T09:06:29.438 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-08-30T09:06:29.897 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T09:06:30.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.498+0000 7f59e5591640 1 -- 172.21.15.42:0/3919683795 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f59e0102820 msgr2=0x7f59e0102c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:06:30.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.498+0000 7f59e5591640 1 --2- 172.21.15.42:0/3919683795 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f59e0102820 0x7f59e0102c30 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f59d0009900 tx=0x7f59d002a410 comp rx=0 tx=0).stop 2024-08-30T09:06:30.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.498+0000 7f59e5591640 1 -- 172.21.15.42:0/3919683795 shutdown_connections 2024-08-30T09:06:30.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.498+0000 7f59e5591640 1 --2- 172.21.15.42:0/3919683795 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f59e0108160 0x7f59e0108570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:30.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.498+0000 7f59e5591640 1 --2- 172.21.15.42:0/3919683795 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f59e0108ab0 0x7f59e010f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:30.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.498+0000 7f59e5591640 1 --2- 172.21.15.42:0/3919683795 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f59e0102820 0x7f59e0102c30 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:30.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.498+0000 7f59e5591640 1 -- 172.21.15.42:0/3919683795 >> 172.21.15.42:0/3919683795 conn(0x7f59e00794e0 msgr2=0x7f59e00798f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:06:30.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.498+0000 7f59e5591640 1 -- 172.21.15.42:0/3919683795 shutdown_connections 2024-08-30T09:06:30.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.498+0000 7f59e5591640 1 -- 172.21.15.42:0/3919683795 wait complete. 2024-08-30T09:06:30.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.499+0000 7f59e5591640 1 Processor -- start 2024-08-30T09:06:30.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.499+0000 7f59e5591640 1 -- start start 2024-08-30T09:06:30.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.500+0000 7f59e5591640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f59e0102820 0x7f59e019a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:30.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.500+0000 7f59e5591640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f59e0108160 0x7f59e019a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:30.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.500+0000 7f59e5591640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f59e0108ab0 0x7f59e019fa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:30.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.500+0000 7f59e5591640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f59e0113840 con 0x7f59e0102820 2024-08-30T09:06:30.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.500+0000 7f59e5591640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f59e01136c0 con 0x7f59e0108ab0 2024-08-30T09:06:30.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.500+0000 7f59e5591640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f59e01139c0 con 0x7f59e0108160 2024-08-30T09:06:30.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.500+0000 7f59de7fc640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f59e0108160 0x7f59e019a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:06:30.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.500+0000 7f59de7fc640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f59e0108160 0x7f59e019a7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:32930/0 (socket says 172.21.15.42:32930) 2024-08-30T09:06:30.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.500+0000 7f59de7fc640 1 -- 172.21.15.42:0/3198775126 learned_addr learned my addr 172.21.15.42:0/3198775126 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:06:30.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.500+0000 7f59deffd640 1 --2- 172.21.15.42:0/3198775126 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f59e0102820 0x7f59e019a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:06:30.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.500+0000 7f59df7fe640 1 --2- 172.21.15.42:0/3198775126 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f59e0108ab0 0x7f59e019fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:06:30.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.500+0000 7f59de7fc640 1 -- 172.21.15.42:0/3198775126 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f59e0108ab0 msgr2=0x7f59e019fa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:06:30.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.500+0000 7f59de7fc640 1 --2- 172.21.15.42:0/3198775126 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f59e0108ab0 0x7f59e019fa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:30.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.500+0000 7f59de7fc640 1 -- 172.21.15.42:0/3198775126 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f59e0102820 msgr2=0x7f59e019a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:06:30.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.500+0000 7f59de7fc640 1 --2- 172.21.15.42:0/3198775126 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f59e0102820 0x7f59e019a210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:30.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.500+0000 7f59de7fc640 1 -- 172.21.15.42:0/3198775126 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59e01a0180 con 0x7f59e0108160 2024-08-30T09:06:30.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.501+0000 7f59de7fc640 1 --2- 172.21.15.42:0/3198775126 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f59e0108160 0x7f59e019a7b0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f59d400d740 tx=0x7f59d400dc00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:06:30.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.501+0000 7f59c7fff640 1 -- 172.21.15.42:0/3198775126 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59d4017070 con 0x7f59e0108160 2024-08-30T09:06:30.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.501+0000 7f59c7fff640 1 -- 172.21.15.42:0/3198775126 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f59d400de50 con 0x7f59e0108160 2024-08-30T09:06:30.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.501+0000 7f59e5591640 1 -- 172.21.15.42:0/3198775126 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59e01a0470 con 0x7f59e0108160 2024-08-30T09:06:30.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.501+0000 7f59c7fff640 1 -- 172.21.15.42:0/3198775126 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59d4004ce0 con 0x7f59e0108160 2024-08-30T09:06:30.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.501+0000 7f59e5591640 1 -- 172.21.15.42:0/3198775126 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59e01a6bd0 con 0x7f59e0108160 2024-08-30T09:06:30.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.505+0000 7f59c7fff640 1 -- 172.21.15.42:0/3198775126 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7f59d4010460 con 0x7f59e0108160 2024-08-30T09:06:30.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.505+0000 7f59ebb06740 1 Processor -- start 2024-08-30T09:06:30.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.506+0000 7f59ebb06740 1 -- start start 2024-08-30T09:06:30.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.506+0000 7f59c7fff640 1 --2- 172.21.15.42:0/3198775126 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f59c80b4b30 0x7f59c80b7040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:30.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.506+0000 7f59c7fff640 1 -- 172.21.15.42:0/3198775126 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f59d40e4e20 con 0x7f59e0108160 2024-08-30T09:06:30.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.506+0000 7f59df7fe640 1 --2- 172.21.15.42:0/3198775126 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f59c80b4b30 0x7f59c80b7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:06:30.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.506+0000 7f59ebb06740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564f5850b070 0x564f588b0310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:30.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.507+0000 7f59ebb06740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564f584e21b0 0x564f588b0850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:30.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.507+0000 7f59ebb06740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564f584ea390 0x564f588b0d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:30.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.507+0000 7f59ebb06740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x564f58827680 con 0x564f5850b070 2024-08-30T09:06:30.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.507+0000 7f59ebb06740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x564f58816c10 con 0x564f584ea390 2024-08-30T09:06:30.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.507+0000 7f59deffd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564f5850b070 0x564f588b0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:06:30.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.507+0000 7f59deffd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564f5850b070 0x564f588b0310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:45554/0 (socket says 172.21.15.42:45554) 2024-08-30T09:06:30.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.507+0000 7f59deffd640 1 -- 172.21.15.42:0/3064092346 learned_addr learned my addr 172.21.15.42:0/3064092346 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:06:30.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.507+0000 7f59de7fc640 1 --2- 172.21.15.42:0/3064092346 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564f584e21b0 0x564f588b0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:06:30.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.507+0000 7f59ebb06740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x564f5881abc0 con 0x564f584e21b0 2024-08-30T09:06:30.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.507+0000 7f59de7fc640 1 -- 172.21.15.42:0/3064092346 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564f584ea390 msgr2=0x564f588b0d90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-30T09:06:30.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.507+0000 7f59de7fc640 1 --2- 172.21.15.42:0/3064092346 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564f584ea390 0x564f588b0d90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:30.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.507+0000 7f59de7fc640 1 -- 172.21.15.42:0/3064092346 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564f5850b070 msgr2=0x564f588b0310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:06:30.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.507+0000 7f59de7fc640 1 --2- 172.21.15.42:0/3064092346 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564f5850b070 0x564f588b0310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:30.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.507+0000 7f59de7fc640 1 -- 172.21.15.42:0/3064092346 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564f58576650 con 0x564f584e21b0 2024-08-30T09:06:30.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.507+0000 7f59deffd640 1 --2- 172.21.15.42:0/3064092346 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564f5850b070 0x564f588b0310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-30T09:06:30.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.508+0000 7f59df7fe640 1 --2- 172.21.15.42:0/3198775126 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f59c80b4b30 0x7f59c80b7040 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f59d8005fd0 tx=0x7f59d8009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:06:30.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.508+0000 7f59de7fc640 1 --2- 172.21.15.42:0/3064092346 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564f584e21b0 0x564f588b0850 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f59d4009610 tx=0x7f59d40aaff0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d1d38715e548b614 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:06:30.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.508+0000 7f59c57fa640 1 -- 172.21.15.42:0/3064092346 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59d4020070 con 0x564f584e21b0 2024-08-30T09:06:30.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.508+0000 7f59c57fa640 1 -- 172.21.15.42:0/3064092346 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f59d40b47e0 con 0x564f584e21b0 2024-08-30T09:06:30.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.508+0000 7f59c57fa640 1 -- 172.21.15.42:0/3064092346 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59d40a6ee0 con 0x564f584e21b0 2024-08-30T09:06:30.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.509+0000 7f59ebb06740 1 -- 172.21.15.42:0/3064092346 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564f584e21b0 msgr2=0x564f588b0850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:06:30.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.509+0000 7f59ebb06740 1 --2- 172.21.15.42:0/3064092346 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564f584e21b0 0x564f588b0850 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f59d4009610 tx=0x7f59d40aaff0 comp rx=0 tx=0).stop 2024-08-30T09:06:30.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.509+0000 7f59ebb06740 1 -- 172.21.15.42:0/3064092346 shutdown_connections 2024-08-30T09:06:30.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.509+0000 7f59ebb06740 1 --2- 172.21.15.42:0/3064092346 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564f584e21b0 0x564f588b0850 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:30.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.509+0000 7f59ebb06740 1 --2- 172.21.15.42:0/3064092346 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564f584ea390 0x564f588b0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:30.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.509+0000 7f59ebb06740 1 --2- 172.21.15.42:0/3064092346 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564f5850b070 0x564f588b0310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:30.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.509+0000 7f59ebb06740 1 -- 172.21.15.42:0/3064092346 >> 172.21.15.42:0/3064092346 conn(0x564f584ff460 msgr2=0x564f588ae200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:06:30.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.511+0000 7f59ebb06740 1 -- 172.21.15.42:0/3064092346 shutdown_connections 2024-08-30T09:06:30.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.516+0000 7f59ebb06740 1 -- 172.21.15.42:0/3064092346 wait complete. 2024-08-30T09:06:30.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.516+0000 7f59ebb06740 1 -- start start 2024-08-30T09:06:30.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.519+0000 7f59ebb06740 1 Processor -- start 2024-08-30T09:06:30.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.525+0000 7f59ebb06740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564f584e21b0 0x564f588bbb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:30.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.525+0000 7f59ebb06740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564f584ea390 0x564f588bc0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:30.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.525+0000 7f59ebb06740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564f5850b070 0x564f588c1d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:30.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.525+0000 7f59ebb06740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x564f58679700 con 0x564f5850b070 2024-08-30T09:06:30.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.525+0000 7f59ebb06740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x564f5879d750 con 0x564f584ea390 2024-08-30T09:06:30.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.525+0000 7f59ebb06740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x564f588bea90 con 0x564f584e21b0 2024-08-30T09:06:30.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.525+0000 7f59ebb06740 10 client.0 _ll_register_callbacks cb 0x564f587cce10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:06:30.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.528+0000 7f59de7fc640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564f584ea390 0x564f588bc0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:06:30.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.531+0000 7f59de7fc640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564f584ea390 0x564f588bc0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:56566/0 (socket says 172.21.15.42:56566) 2024-08-30T09:06:30.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.531+0000 7f59de7fc640 1 -- 172.21.15.42:0/2818022409 learned_addr learned my addr 172.21.15.42:0/2818022409 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:06:30.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.534+0000 7f59df7fe640 1 --2- 172.21.15.42:0/2818022409 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564f5850b070 0x564f588c1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:06:30.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.534+0000 7f59de7fc640 1 -- 172.21.15.42:0/2818022409 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564f584e21b0 msgr2=0x564f588bbb40 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-30T09:06:30.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.534+0000 7f59de7fc640 1 --2- 172.21.15.42:0/2818022409 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564f584e21b0 0x564f588bbb40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:30.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.534+0000 7f59de7fc640 1 -- 172.21.15.42:0/2818022409 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564f5850b070 msgr2=0x564f588c1d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:06:30.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.534+0000 7f59de7fc640 1 --2- 172.21.15.42:0/2818022409 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564f5850b070 0x564f588c1d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:30.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.534+0000 7f59de7fc640 1 -- 172.21.15.42:0/2818022409 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564f587d6900 con 0x564f584ea390 2024-08-30T09:06:30.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.536+0000 7f59de7fc640 1 --2- 172.21.15.42:0/2818022409 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564f584ea390 0x564f588bc0e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f59d40a75f0 tx=0x7f59d40a74e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=db38935b51c5dc4e server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:06:30.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.536+0000 7f59c2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-30T09:06:30.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.538+0000 7f59c2ff5640 1 -- 172.21.15.42:0/2818022409 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59d4020070 con 0x564f584ea390 2024-08-30T09:06:30.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.538+0000 7f59c2ff5640 1 -- 172.21.15.42:0/2818022409 <== mon.2 v2:172.21.15.167:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f59d40d2180 con 0x564f584ea390 2024-08-30T09:06:30.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.540+0000 7f59ebb06740 1 -- 172.21.15.42:0/2818022409 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f59d40b47e0 con 0x564f584ea390 2024-08-30T09:06:30.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.540+0000 7f59c2ff5640 1 -- 172.21.15.42:0/2818022409 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f59d40a7800 con 0x564f584ea390 2024-08-30T09:06:30.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.541+0000 7f59ebb06740 10 client.26165 fetch_fsmap learned FSMap version 33 2024-08-30T09:06:30.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.541+0000 7f59ebb06740 1 -- 172.21.15.42:0/2818022409 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x564f588c48f0 con 0x564f584ea390 2024-08-30T09:06:30.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.546+0000 7f59c2ff5640 1 -- 172.21.15.42:0/2818022409 <== mon.2 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f59d40b47e0 con 0x564f584ea390 2024-08-30T09:06:30.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.548+0000 7f59c2ff5640 1 -- 172.21.15.42:0/2818022409 <== mon.2 v2:172.21.15.167:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f59d40aa240 con 0x564f584ea390 2024-08-30T09:06:30.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.548+0000 7f59ebb06740 10 client.26165 fetch_fsmap finished waiting for FSMap version 33 2024-08-30T09:06:30.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.548+0000 7f59ebb06740 10 client.26165 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-08-30T09:06:30.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.548+0000 7f59ebb06740 20 client.26165 populate_metadata read hostname 'smithi042' 2024-08-30T09:06:30.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.548+0000 7f59ebb06740 1 --2- 172.21.15.42:0/2818022409 >> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] conn(0x564f584c99f0 0x564f588c7dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:30.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.548+0000 7f59ebb06740 4 client.26165 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-30T09:06:30.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.548+0000 7f59ebb06740 1 -- 172.21.15.42:0/2818022409 --> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564f588295d0 con 0x564f584c99f0 2024-08-30T09:06:30.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.555+0000 7f59deffd640 1 --2- 172.21.15.42:0/2818022409 >> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] conn(0x564f584c99f0 0x564f588c7dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:06:30.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.561+0000 7f59deffd640 1 --2- 172.21.15.42:0/2818022409 >> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] conn(0x564f584c99f0 0x564f588c7dc0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c018194bdbec47da server_cookie=2c66b02261d14251 in_seq=0 out_seq=0 2024-08-30T09:06:30.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.561+0000 7f59c2ff5640 10 client.26165 ms_handle_connect on v2:172.21.15.190:6838/719981110 2024-08-30T09:06:30.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.564+0000 7f59c2ff5640 1 -- 172.21.15.42:0/2818022409 <== mds.1 v2:172.21.15.190:6838/719981110 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x564f588295d0 con 0x564f584c99f0 2024-08-30T09:06:30.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.564+0000 7f59c2ff5640 10 client.26165 handle_command_reply: tid=0 2024-08-30T09:06:30.568 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:06:30 smithi042 ceph-mon[18652]: pgmap v545: 129 pgs: 129 active+clean; 1.4 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 9.9 MiB/s wr, 562 op/s 2024-08-30T09:06:30.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.565+0000 7f59ebb06740 1 client.26165 shutdown 2024-08-30T09:06:30.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.566+0000 7f59ebb06740 1 -- 172.21.15.42:0/2818022409 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564f584ea390 msgr2=0x564f588bc0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:06:30.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.567+0000 7f59ebb06740 1 --2- 172.21.15.42:0/2818022409 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564f584ea390 0x564f588bc0e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f59d40a75f0 tx=0x7f59d40a74e0 comp rx=0 tx=0).stop 2024-08-30T09:06:30.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.567+0000 7f59ebb06740 1 -- 172.21.15.42:0/2818022409 shutdown_connections 2024-08-30T09:06:30.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.567+0000 7f59ebb06740 1 --2- 172.21.15.42:0/2818022409 >> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] conn(0x564f584c99f0 0x564f588c7dc0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:30.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.567+0000 7f59ebb06740 1 --2- 172.21.15.42:0/2818022409 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564f5850b070 0x564f588c1d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:30.570 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.567+0000 7f59ebb06740 1 --2- 172.21.15.42:0/2818022409 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564f584ea390 0x564f588bc0e0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:30.570 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.567+0000 7f59ebb06740 1 --2- 172.21.15.42:0/2818022409 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564f584e21b0 0x564f588bbb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:30.570 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.567+0000 7f59c2ff5640 0 client.26165 ms_handle_reset on v2:172.21.15.190:6838/719981110 2024-08-30T09:06:30.570 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.567+0000 7f59ebb06740 1 -- 172.21.15.42:0/2818022409 >> 172.21.15.42:0/2818022409 conn(0x564f584ff460 msgr2=0x564f588b34f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:06:30.570 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.567+0000 7f59ebb06740 1 -- 172.21.15.42:0/2818022409 shutdown_connections 2024-08-30T09:06:30.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.571+0000 7f59ebb06740 1 -- 172.21.15.42:0/2818022409 wait complete. 2024-08-30T09:06:30.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.571+0000 7f59ebb06740 20 client.26165 trim_cache size 0 max 16384 2024-08-30T09:06:30.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:06:30 smithi167 ceph-mon[22063]: pgmap v545: 129 pgs: 129 active+clean; 1.4 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 9.9 MiB/s wr, 562 op/s 2024-08-30T09:06:30.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.597+0000 7f59ebb06740 1 Processor -- start 2024-08-30T09:06:30.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.598+0000 7f59ebb06740 1 -- start start 2024-08-30T09:06:30.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.598+0000 7f59ebb06740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564f584e21b0 0x564f588b1d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:30.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.598+0000 7f59ebb06740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564f584ea390 0x564f588bd770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:30.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.598+0000 7f59ebb06740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564f5850b070 0x564f588c1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:30.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.599+0000 7f59ebb06740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f59e01b0f10 con 0x564f5850b070 2024-08-30T09:06:30.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.599+0000 7f59df7fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564f5850b070 0x564f588c1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:06:30.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.599+0000 7f59df7fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564f5850b070 0x564f588c1cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:45566/0 (socket says 172.21.15.42:45566) 2024-08-30T09:06:30.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.599+0000 7f59df7fe640 1 -- 172.21.15.42:0/451431822 learned_addr learned my addr 172.21.15.42:0/451431822 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:06:30.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.599+0000 7f59deffd640 1 --2- 172.21.15.42:0/451431822 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564f584e21b0 0x564f588b1d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:06:30.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.599+0000 7f59de7fc640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564f584ea390 0x564f588bd770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:06:30.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.599+0000 7f59ebb06740 1 -- 172.21.15.42:0/451431822 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f59e01ae780 con 0x564f584e21b0 2024-08-30T09:06:30.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.599+0000 7f59ebb06740 1 -- 172.21.15.42:0/451431822 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f59e01ac680 con 0x564f584ea390 2024-08-30T09:06:30.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.600+0000 7f59deffd640 1 -- 172.21.15.42:0/451431822 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564f584ea390 msgr2=0x564f588bd770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:06:30.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.600+0000 7f59deffd640 1 --2- 172.21.15.42:0/451431822 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564f584ea390 0x564f588bd770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:30.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.600+0000 7f59deffd640 1 -- 172.21.15.42:0/451431822 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564f5850b070 msgr2=0x564f588c1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:06:30.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.600+0000 7f59deffd640 1 --2- 172.21.15.42:0/451431822 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564f5850b070 0x564f588c1cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:30.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.600+0000 7f59deffd640 1 -- 172.21.15.42:0/451431822 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564f588b2ba0 con 0x564f584e21b0 2024-08-30T09:06:30.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.600+0000 7f59deffd640 1 --2- 172.21.15.42:0/451431822 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564f584e21b0 0x564f588b1d00 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f59d0009230 tx=0x7f59d0000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e854e2dd18a46b55 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:06:30.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.600+0000 7f59c4ff9640 1 -- 172.21.15.42:0/451431822 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59d004a070 con 0x564f584e21b0 2024-08-30T09:06:30.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.601+0000 7f59c4ff9640 1 -- 172.21.15.42:0/451431822 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f59d0005030 con 0x564f584e21b0 2024-08-30T09:06:30.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.602+0000 7f59ebb06740 1 -- 172.21.15.42:0/451431822 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564f584e21b0 msgr2=0x564f588b1d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:06:30.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.602+0000 7f59ebb06740 1 --2- 172.21.15.42:0/451431822 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564f584e21b0 0x564f588b1d00 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f59d0009230 tx=0x7f59d0000bc0 comp rx=0 tx=0).stop 2024-08-30T09:06:30.605 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.602+0000 7f59ebb06740 1 -- 172.21.15.42:0/451431822 shutdown_connections 2024-08-30T09:06:30.605 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.602+0000 7f59ebb06740 1 --2- 172.21.15.42:0/451431822 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564f584ea390 0x564f588bd770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:30.605 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.603+0000 7f59ebb06740 1 --2- 172.21.15.42:0/451431822 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564f5850b070 0x564f588c1cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:30.605 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.603+0000 7f59ebb06740 1 --2- 172.21.15.42:0/451431822 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564f584e21b0 0x564f588b1d00 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:30.605 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.603+0000 7f59ebb06740 1 -- 172.21.15.42:0/451431822 >> 172.21.15.42:0/451431822 conn(0x564f584ff460 msgr2=0x564f588bd060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:06:30.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.605+0000 7f59ebb06740 1 -- 172.21.15.42:0/451431822 shutdown_connections 2024-08-30T09:06:30.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.605+0000 7f59ebb06740 1 -- 172.21.15.42:0/451431822 wait complete. 2024-08-30T09:06:30.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.605+0000 7f59ebb06740 1 -- start start 2024-08-30T09:06:30.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.606+0000 7f59ebb06740 1 Processor -- start 2024-08-30T09:06:30.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.606+0000 7f59ebb06740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564f584e21b0 0x564f588cb370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:30.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.606+0000 7f59ebb06740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564f584ea390 0x564f588c7830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:30.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.607+0000 7f59ebb06740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564f5850b070 0x564f588c7d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:30.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.607+0000 7f59ebb06740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f59e01acc40 con 0x564f584ea390 2024-08-30T09:06:30.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.607+0000 7f59ebb06740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f59e01ae0b0 con 0x564f5850b070 2024-08-30T09:06:30.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.607+0000 7f59ebb06740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f59e01ad310 con 0x564f584e21b0 2024-08-30T09:06:30.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.607+0000 7f59ebb06740 10 client.0 _ll_register_callbacks cb 0x7f59d0009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:06:30.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.607+0000 7f59df7fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564f5850b070 0x564f588c7d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:06:30.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.607+0000 7f59df7fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564f5850b070 0x564f588c7d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:56580/0 (socket says 172.21.15.42:56580) 2024-08-30T09:06:30.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.607+0000 7f59df7fe640 1 -- 172.21.15.42:0/1670246119 learned_addr learned my addr 172.21.15.42:0/1670246119 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:06:30.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.607+0000 7f59deffd640 1 --2- 172.21.15.42:0/1670246119 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564f584e21b0 0x564f588cb370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:06:30.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.607+0000 7f59de7fc640 1 --2- 172.21.15.42:0/1670246119 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564f584ea390 0x564f588c7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:06:30.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.608+0000 7f59df7fe640 1 -- 172.21.15.42:0/1670246119 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564f584e21b0 msgr2=0x564f588cb370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:06:30.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.608+0000 7f59df7fe640 1 --2- 172.21.15.42:0/1670246119 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564f584e21b0 0x564f588cb370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:30.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.608+0000 7f59df7fe640 1 -- 172.21.15.42:0/1670246119 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564f584ea390 msgr2=0x564f588c7830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:06:30.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.608+0000 7f59df7fe640 1 --2- 172.21.15.42:0/1670246119 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564f584ea390 0x564f588c7830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:30.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.608+0000 7f59df7fe640 1 -- 172.21.15.42:0/1670246119 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564f588c82b0 con 0x564f5850b070 2024-08-30T09:06:30.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.614+0000 7f59df7fe640 1 --2- 172.21.15.42:0/1670246119 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564f5850b070 0x564f588c7d70 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f59d8009270 tx=0x7f59d8004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4714bb28a7c522b7 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:06:30.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.615+0000 7f59c2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-30T09:06:30.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.622+0000 7f59c2ff5640 1 -- 172.21.15.42:0/1670246119 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59d802d060 con 0x564f5850b070 2024-08-30T09:06:30.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.622+0000 7f59ebb06740 1 -- 172.21.15.42:0/1670246119 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f59d0005030 con 0x564f5850b070 2024-08-30T09:06:30.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.622+0000 7f59c2ff5640 1 -- 172.21.15.42:0/1670246119 <== mon.2 v2:172.21.15.167:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f59d80283f0 con 0x564f5850b070 2024-08-30T09:06:30.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.623+0000 7f59c2ff5640 1 -- 172.21.15.42:0/1670246119 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f59d8034050 con 0x564f5850b070 2024-08-30T09:06:30.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.626+0000 7f59ebb06740 10 client.26171 fetch_fsmap learned FSMap version 33 2024-08-30T09:06:30.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.626+0000 7f59ebb06740 1 -- 172.21.15.42:0/1670246119 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x564f589498e0 con 0x564f5850b070 2024-08-30T09:06:30.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.631+0000 7f59c2ff5640 1 -- 172.21.15.42:0/1670246119 <== mon.2 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f59d0005030 con 0x564f5850b070 2024-08-30T09:06:30.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.631+0000 7f59c2ff5640 1 -- 172.21.15.42:0/1670246119 <== mon.2 v2:172.21.15.167:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f59d8027030 con 0x564f5850b070 2024-08-30T09:06:30.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.631+0000 7f59ebb06740 10 client.26171 fetch_fsmap finished waiting for FSMap version 33 2024-08-30T09:06:30.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.631+0000 7f59ebb06740 10 client.26171 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-08-30T09:06:30.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.631+0000 7f59ebb06740 20 client.26171 populate_metadata read hostname 'smithi042' 2024-08-30T09:06:30.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.631+0000 7f59ebb06740 1 --2- 172.21.15.42:0/1670246119 >> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] conn(0x564f5849eda0 0x564f5894c110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:30.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.631+0000 7f59ebb06740 4 client.26171 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-30T09:06:30.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.631+0000 7f59ebb06740 1 -- 172.21.15.42:0/1670246119 --> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564f58825070 con 0x564f5849eda0 2024-08-30T09:06:30.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.631+0000 7f59deffd640 1 --2- 172.21.15.42:0/1670246119 >> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] conn(0x564f5849eda0 0x564f5894c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:06:30.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.632+0000 7f59c2ff5640 10 client.26171 ms_handle_connect on v2:172.21.15.190:6838/719981110 2024-08-30T09:06:30.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.633+0000 7f59deffd640 1 --2- 172.21.15.42:0/1670246119 >> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] conn(0x564f5849eda0 0x564f5894c110 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=87db0d3ddd1aa32c server_cookie=3c5062f591e13e47 in_seq=0 out_seq=0 2024-08-30T09:06:30.638 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.633+0000 7f59c2ff5640 1 -- 172.21.15.42:0/1670246119 <== mds.1 v2:172.21.15.190:6838/719981110 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564f58825070 con 0x564f5849eda0 2024-08-30T09:06:30.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.633+0000 7f59c2ff5640 10 client.26171 handle_command_reply: tid=0 2024-08-30T09:06:30.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.633+0000 7f59ebb06740 1 client.26171 shutdown 2024-08-30T09:06:30.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.635+0000 7f59ebb06740 1 -- 172.21.15.42:0/1670246119 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564f5850b070 msgr2=0x564f588c7d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:06:30.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.635+0000 7f59ebb06740 1 --2- 172.21.15.42:0/1670246119 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564f5850b070 0x564f588c7d70 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f59d8009270 tx=0x7f59d8004ab0 comp rx=0 tx=0).stop 2024-08-30T09:06:30.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.637+0000 7f59ebb06740 1 -- 172.21.15.42:0/1670246119 shutdown_connections 2024-08-30T09:06:30.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.638+0000 7f59ebb06740 1 --2- 172.21.15.42:0/1670246119 >> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] conn(0x564f5849eda0 0x564f5894c110 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:30.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.638+0000 7f59ebb06740 1 --2- 172.21.15.42:0/1670246119 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564f5850b070 0x564f588c7d70 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:30.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.638+0000 7f59ebb06740 1 --2- 172.21.15.42:0/1670246119 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564f584ea390 0x564f588c7830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:30.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.638+0000 7f59ebb06740 1 --2- 172.21.15.42:0/1670246119 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564f584e21b0 0x564f588cb370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:30.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.638+0000 7f59c2ff5640 0 client.26171 ms_handle_reset on v2:172.21.15.190:6838/719981110 2024-08-30T09:06:30.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.638+0000 7f59ebb06740 1 -- 172.21.15.42:0/1670246119 >> 172.21.15.42:0/1670246119 conn(0x564f584ff460 msgr2=0x564f588b1ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:06:30.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.640+0000 7f59ebb06740 1 -- 172.21.15.42:0/1670246119 shutdown_connections 2024-08-30T09:06:30.647 INFO:teuthology.orchestra.run.smithi042.stdout:[] 2024-08-30T09:06:30.647 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.641+0000 7f59ebb06740 1 -- 172.21.15.42:0/1670246119 wait complete. 2024-08-30T09:06:30.647 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.643+0000 7f59ebb06740 20 client.26171 trim_cache size 0 max 16384 2024-08-30T09:06:30.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.650+0000 7f59e5591640 1 -- 172.21.15.42:0/3198775126 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f59c80b4b30 msgr2=0x7f59c80b7040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:06:30.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.650+0000 7f59e5591640 1 --2- 172.21.15.42:0/3198775126 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f59c80b4b30 0x7f59c80b7040 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f59d8005fd0 tx=0x7f59d8009500 comp rx=0 tx=0).stop 2024-08-30T09:06:30.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.650+0000 7f59e5591640 1 -- 172.21.15.42:0/3198775126 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f59e0108160 msgr2=0x7f59e019a7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:06:30.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.650+0000 7f59e5591640 1 --2- 172.21.15.42:0/3198775126 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f59e0108160 0x7f59e019a7b0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f59d400d740 tx=0x7f59d400dc00 comp rx=0 tx=0).stop 2024-08-30T09:06:30.656 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.654+0000 7f59e5591640 1 -- 172.21.15.42:0/3198775126 shutdown_connections 2024-08-30T09:06:30.656 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.654+0000 7f59e5591640 1 --2- 172.21.15.42:0/3198775126 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f59c80b4b30 0x7f59c80b7040 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:30.656 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.654+0000 7f59e5591640 1 --2- 172.21.15.42:0/3198775126 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f59e0108160 0x7f59e019a7b0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:30.656 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.654+0000 7f59e5591640 1 --2- 172.21.15.42:0/3198775126 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f59e0108ab0 0x7f59e019fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:30.656 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.654+0000 7f59e5591640 1 --2- 172.21.15.42:0/3198775126 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f59e0102820 0x7f59e019a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:30.657 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.654+0000 7f59e5591640 1 -- 172.21.15.42:0/3198775126 >> 172.21.15.42:0/3198775126 conn(0x7f59e00794e0 msgr2=0x7f59e01011f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:06:30.657 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.655+0000 7f59e5591640 1 -- 172.21.15.42:0/3198775126 shutdown_connections 2024-08-30T09:06:30.657 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:30.655+0000 7f59e5591640 1 -- 172.21.15.42:0/3198775126 wait complete. 2024-08-30T09:06:30.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:06:30 smithi190 ceph-mon[22001]: pgmap v545: 129 pgs: 129 active+clean; 1.4 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 9.9 MiB/s wr, 562 op/s 2024-08-30T09:06:30.953 INFO:teuthology.orchestra.run.smithi042.stderr:progress: 345.0 s, 653.2 tps, lat 153.779 ms stddev 199.924 2024-08-30T09:06:31.345 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-08-30T09:06:31.821 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T09:06:31.941 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:31.939+0000 7faa53fad640 1 -- 172.21.15.42:0/2692256563 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faa4c1080a0 msgr2=0x7faa4c10a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:06:31.942 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:31.939+0000 7faa53fad640 1 --2- 172.21.15.42:0/2692256563 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faa4c1080a0 0x7faa4c10a5b0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7faa44009ca0 tx=0x7faa4402a8c0 comp rx=0 tx=0).stop 2024-08-30T09:06:31.942 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:31.939+0000 7faa53fad640 1 -- 172.21.15.42:0/2692256563 shutdown_connections 2024-08-30T09:06:31.942 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:31.939+0000 7faa53fad640 1 --2- 172.21.15.42:0/2692256563 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faa4c105480 0x7faa4c107960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:31.942 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:31.939+0000 7faa53fad640 1 --2- 172.21.15.42:0/2692256563 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faa4c1080a0 0x7faa4c10a5b0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:31.942 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:31.939+0000 7faa53fad640 1 --2- 172.21.15.42:0/2692256563 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faa4c102a40 0x7faa4c104e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:31.942 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:31.939+0000 7faa53fad640 1 -- 172.21.15.42:0/2692256563 >> 172.21.15.42:0/2692256563 conn(0x7faa4c0fd430 msgr2=0x7faa4c0ff8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:06:31.942 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:31.939+0000 7faa53fad640 1 -- 172.21.15.42:0/2692256563 shutdown_connections 2024-08-30T09:06:31.942 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:31.939+0000 7faa53fad640 1 -- 172.21.15.42:0/2692256563 wait complete. 2024-08-30T09:06:31.943 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:31.940+0000 7faa53fad640 1 Processor -- start 2024-08-30T09:06:31.944 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:31.940+0000 7faa53fad640 1 -- start start 2024-08-30T09:06:31.944 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:31.940+0000 7faa53fad640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faa4c102a40 0x7faa4c1a0270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:31.944 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:31.940+0000 7faa53fad640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faa4c105480 0x7faa4c1a08e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:31.944 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:31.940+0000 7faa53fad640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faa4c1080a0 0x7faa4c1a1490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:31.944 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:31.940+0000 7faa53fad640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7faa4c10c2a0 con 0x7faa4c102a40 2024-08-30T09:06:31.944 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:31.940+0000 7faa53fad640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7faa4c10c120 con 0x7faa4c1080a0 2024-08-30T09:06:31.944 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:31.940+0000 7faa53fad640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7faa4c10c420 con 0x7faa4c105480 2024-08-30T09:06:31.944 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:31.941+0000 7faa5254a640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faa4c1080a0 0x7faa4c1a1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:06:31.944 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:31.941+0000 7faa5254a640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faa4c1080a0 0x7faa4c1a1490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:56600/0 (socket says 172.21.15.42:56600) 2024-08-30T09:06:31.944 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:31.941+0000 7faa5254a640 1 -- 172.21.15.42:0/442590217 learned_addr learned my addr 172.21.15.42:0/442590217 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:06:31.944 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:31.941+0000 7faa51548640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faa4c105480 0x7faa4c1a08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:06:31.944 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:31.941+0000 7faa51d49640 1 --2- 172.21.15.42:0/442590217 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faa4c102a40 0x7faa4c1a0270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:06:31.944 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:31.941+0000 7faa51548640 1 -- 172.21.15.42:0/442590217 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faa4c1080a0 msgr2=0x7faa4c1a1490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:06:31.945 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:31.941+0000 7faa51548640 1 --2- 172.21.15.42:0/442590217 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faa4c1080a0 0x7faa4c1a1490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:31.945 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:31.941+0000 7faa51548640 1 -- 172.21.15.42:0/442590217 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faa4c102a40 msgr2=0x7faa4c1a0270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:06:31.945 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:31.941+0000 7faa51548640 1 --2- 172.21.15.42:0/442590217 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faa4c102a40 0x7faa4c1a0270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:31.945 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:31.941+0000 7faa51548640 1 -- 172.21.15.42:0/442590217 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa4c1a1cb0 con 0x7faa4c105480 2024-08-30T09:06:31.945 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:31.941+0000 7faa51548640 1 --2- 172.21.15.42:0/442590217 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faa4c105480 0x7faa4c1a08e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7faa3c00e9f0 tx=0x7faa3c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:06:31.945 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:31.941+0000 7faa42ffd640 1 -- 172.21.15.42:0/442590217 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa3c00ccc0 con 0x7faa4c105480 2024-08-30T09:06:31.945 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:31.942+0000 7faa42ffd640 1 -- 172.21.15.42:0/442590217 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faa3c005020 con 0x7faa4c105480 2024-08-30T09:06:31.945 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:31.942+0000 7faa53fad640 1 -- 172.21.15.42:0/442590217 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa4c1a1fa0 con 0x7faa4c105480 2024-08-30T09:06:31.945 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:31.942+0000 7faa42ffd640 1 -- 172.21.15.42:0/442590217 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa3c004a00 con 0x7faa4c105480 2024-08-30T09:06:31.945 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:31.942+0000 7faa53fad640 1 -- 172.21.15.42:0/442590217 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa4c1a8690 con 0x7faa4c105480 2024-08-30T09:06:31.947 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:31.945+0000 7faa42ffd640 1 -- 172.21.15.42:0/442590217 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7faa3c004ba0 con 0x7faa4c105480 2024-08-30T09:06:31.947 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:31.945+0000 7faa5aff5740 1 Processor -- start 2024-08-30T09:06:31.947 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:31.945+0000 7faa5aff5740 1 -- start start 2024-08-30T09:06:31.948 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:31.945+0000 7faa5aff5740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561949a391b0 0x561949ddf180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:31.948 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:31.945+0000 7faa5aff5740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561949a102f0 0x561949ddf7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:31.948 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:31.945+0000 7faa5aff5740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561949a184d0 0x561949de0340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:31.948 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:31.945+0000 7faa5aff5740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x561949d52000 con 0x561949a184d0 2024-08-30T09:06:31.948 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:31.945+0000 7faa5aff5740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x561949d41590 con 0x561949a391b0 2024-08-30T09:06:31.948 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:31.945+0000 7faa5aff5740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x561949d45540 con 0x561949a102f0 2024-08-30T09:06:31.948 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:31.945+0000 7faa5254a640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561949a391b0 0x561949ddf180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:06:31.948 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:31.946+0000 7faa5254a640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561949a391b0 0x561949ddf180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:56608/0 (socket says 172.21.15.42:56608) 2024-08-30T09:06:31.948 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:31.946+0000 7faa5254a640 1 -- 172.21.15.42:0/2568141254 learned_addr learned my addr 172.21.15.42:0/2568141254 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:06:31.948 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:31.946+0000 7faa51548640 1 --2- 172.21.15.42:0/2568141254 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561949a184d0 0x561949de0340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:06:31.949 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:31.946+0000 7faa51d49640 1 --2- 172.21.15.42:0/2568141254 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561949a102f0 0x561949ddf7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:06:31.949 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:31.946+0000 7faa42ffd640 1 --2- 172.21.15.42:0/442590217 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7faa380b4bb0 0x7faa380b70c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:31.950 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:31.946+0000 7faa51d49640 1 -- 172.21.15.42:0/2568141254 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561949a391b0 msgr2=0x561949ddf180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:06:31.950 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:31.946+0000 7faa51d49640 1 --2- 172.21.15.42:0/2568141254 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561949a391b0 0x561949ddf180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:31.950 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:31.946+0000 7faa51d49640 1 -- 172.21.15.42:0/2568141254 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561949a184d0 msgr2=0x561949de0340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:06:31.950 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:31.946+0000 7faa51d49640 1 --2- 172.21.15.42:0/2568141254 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561949a184d0 0x561949de0340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:31.950 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:31.946+0000 7faa51d49640 1 -- 172.21.15.42:0/2568141254 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561949aa4790 con 0x561949a102f0 2024-08-30T09:06:31.950 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:31.946+0000 7faa42ffd640 1 -- 172.21.15.42:0/442590217 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7faa3c0e5340 con 0x7faa4c105480 2024-08-30T09:06:31.950 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:31.947+0000 7faa5254a640 1 --2- 172.21.15.42:0/442590217 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7faa380b4bb0 0x7faa380b70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:06:31.950 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:31.947+0000 7faa51d49640 1 --2- 172.21.15.42:0/2568141254 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561949a102f0 0x561949ddf7f0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7faa48009c20 tx=0x7faa48011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e86c78df3138ce01 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:06:31.950 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:31.947+0000 7faa37fff640 1 -- 172.21.15.42:0/2568141254 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa48009dd0 con 0x561949a102f0 2024-08-30T09:06:31.950 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:31.947+0000 7faa37fff640 1 -- 172.21.15.42:0/2568141254 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faa48005020 con 0x561949a102f0 2024-08-30T09:06:31.950 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:31.947+0000 7faa37fff640 1 -- 172.21.15.42:0/2568141254 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa48004b20 con 0x561949a102f0 2024-08-30T09:06:31.951 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:31.948+0000 7faa5aff5740 1 -- 172.21.15.42:0/2568141254 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561949a102f0 msgr2=0x561949ddf7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:06:31.951 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:31.948+0000 7faa5aff5740 1 --2- 172.21.15.42:0/2568141254 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561949a102f0 0x561949ddf7f0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7faa48009c20 tx=0x7faa48011810 comp rx=0 tx=0).stop 2024-08-30T09:06:31.951 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:31.949+0000 7faa5254a640 1 --2- 172.21.15.42:0/442590217 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7faa380b4bb0 0x7faa380b70c0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7faa44009940 tx=0x7faa440095c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:06:31.951 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:31.949+0000 7faa5aff5740 1 -- 172.21.15.42:0/2568141254 shutdown_connections 2024-08-30T09:06:31.951 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:31.949+0000 7faa5aff5740 1 --2- 172.21.15.42:0/2568141254 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561949a102f0 0x561949ddf7f0 secure :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7faa48009c20 tx=0x7faa48011810 comp rx=0 tx=0).stop 2024-08-30T09:06:31.952 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:31.949+0000 7faa5aff5740 1 --2- 172.21.15.42:0/2568141254 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561949a184d0 0x561949de0340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:31.952 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:31.949+0000 7faa5aff5740 1 --2- 172.21.15.42:0/2568141254 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561949a391b0 0x561949ddf180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:31.952 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:31.949+0000 7faa5aff5740 1 -- 172.21.15.42:0/2568141254 >> 172.21.15.42:0/2568141254 conn(0x561949a2d300 msgr2=0x561949ddac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:06:31.952 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:31.949+0000 7faa5aff5740 1 -- 172.21.15.42:0/2568141254 shutdown_connections 2024-08-30T09:06:31.952 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:31.949+0000 7faa5aff5740 1 -- 172.21.15.42:0/2568141254 wait complete. 2024-08-30T09:06:31.953 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:31.950+0000 7faa5aff5740 1 -- start start 2024-08-30T09:06:31.953 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:31.950+0000 7faa5aff5740 1 Processor -- start 2024-08-30T09:06:31.953 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:31.951+0000 7faa5aff5740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561949a184d0 0x561949de8d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:31.953 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:31.951+0000 7faa5aff5740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561949a391b0 0x561949de93c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:31.953 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:31.951+0000 7faa5aff5740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5619499f7d80 0x561949de9f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:31.953 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:31.951+0000 7faa5aff5740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x561949c7aab0 con 0x561949a391b0 2024-08-30T09:06:31.953 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:31.951+0000 7faa5aff5740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x561949cc8680 con 0x5619499f7d80 2024-08-30T09:06:31.953 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:31.951+0000 7faa5aff5740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x561949de0e60 con 0x561949a184d0 2024-08-30T09:06:31.953 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:31.951+0000 7faa5aff5740 10 client.0 _ll_register_callbacks cb 0x561949a50a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:06:31.959 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:31.951+0000 7faa5254a640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5619499f7d80 0x561949de9f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:06:31.959 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:31.951+0000 7faa51d49640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561949a184d0 0x561949de8d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:06:31.959 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:31.951+0000 7faa51d49640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561949a184d0 0x561949de8d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:33000/0 (socket says 172.21.15.42:33000) 2024-08-30T09:06:31.959 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:31.951+0000 7faa51d49640 1 -- 172.21.15.42:0/2490685949 learned_addr learned my addr 172.21.15.42:0/2490685949 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:06:31.959 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:31.952+0000 7faa51d49640 1 -- 172.21.15.42:0/2490685949 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5619499f7d80 msgr2=0x561949de9f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:06:31.959 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:31.952+0000 7faa51d49640 1 --2- 172.21.15.42:0/2490685949 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5619499f7d80 0x561949de9f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:31.959 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:31.952+0000 7faa51d49640 1 -- 172.21.15.42:0/2490685949 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561949a391b0 msgr2=0x561949de93c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:06:31.959 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:31.952+0000 7faa51d49640 1 --2- 172.21.15.42:0/2490685949 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561949a391b0 0x561949de93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:31.959 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:31.952+0000 7faa51d49640 1 -- 172.21.15.42:0/2490685949 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561949b1d620 con 0x561949a184d0 2024-08-30T09:06:31.967 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:31.965+0000 7faa51d49640 1 --2- 172.21.15.42:0/2490685949 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561949a184d0 0x561949de8d00 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7faa4801b040 tx=0x7faa480098c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b9f0344a2a3408f4 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:06:31.970 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:31.966+0000 7faa357fa640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-08-30T09:06:31.973 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:31.970+0000 7faa357fa640 1 -- 172.21.15.42:0/2490685949 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa48029020 con 0x561949a184d0 2024-08-30T09:06:31.973 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:31.970+0000 7faa5aff5740 1 -- 172.21.15.42:0/2490685949 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faa48005020 con 0x561949a184d0 2024-08-30T09:06:31.974 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:31.972+0000 7faa357fa640 1 -- 172.21.15.42:0/2490685949 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7faa480054e0 con 0x561949a184d0 2024-08-30T09:06:31.977 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:31.976+0000 7faa357fa640 1 -- 172.21.15.42:0/2490685949 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7faa48034050 con 0x561949a184d0 2024-08-30T09:06:31.978 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:31.977+0000 7faa5aff5740 10 client.26167 fetch_fsmap learned FSMap version 33 2024-08-30T09:06:31.978 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:31.977+0000 7faa5aff5740 1 -- 172.21.15.42:0/2490685949 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x561949df2e80 con 0x561949a184d0 2024-08-30T09:06:31.987 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:31.983+0000 7faa357fa640 1 -- 172.21.15.42:0/2490685949 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faa48005020 con 0x561949a184d0 2024-08-30T09:06:31.987 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:31.983+0000 7faa357fa640 1 -- 172.21.15.42:0/2490685949 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7faa48003b50 con 0x561949a184d0 2024-08-30T09:06:31.987 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:31.983+0000 7faa5aff5740 10 client.26167 fetch_fsmap finished waiting for FSMap version 33 2024-08-30T09:06:31.988 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:31.983+0000 7faa5aff5740 10 client.26167 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-30T09:06:31.988 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:31.983+0000 7faa5aff5740 20 client.26167 populate_metadata read hostname 'smithi042' 2024-08-30T09:06:31.988 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:31.983+0000 7faa5aff5740 1 --2- 172.21.15.42:0/2490685949 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x5619499cd1c0 0x561949df6430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:31.988 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:31.983+0000 7faa5aff5740 4 client.26167 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-30T09:06:31.988 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:31.983+0000 7faa5aff5740 1 -- 172.21.15.42:0/2490685949 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561949d53f50 con 0x5619499cd1c0 2024-08-30T09:06:31.992 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:31.987+0000 7faa51548640 1 --2- 172.21.15.42:0/2490685949 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x5619499cd1c0 0x561949df6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:06:31.998 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:31.995+0000 7faa51548640 1 --2- 172.21.15.42:0/2490685949 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x5619499cd1c0 0x561949df6430 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=329d0af5fe01c453 server_cookie=ce0d745bf2721a71 in_seq=0 out_seq=0 2024-08-30T09:06:31.998 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:31.997+0000 7faa357fa640 10 client.26167 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:06:32.006 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:32.001+0000 7faa357fa640 1 -- 172.21.15.42:0/2490685949 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x561949d53f50 con 0x5619499cd1c0 2024-08-30T09:06:32.006 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:32.001+0000 7faa357fa640 10 client.26167 handle_command_reply: tid=0 2024-08-30T09:06:32.007 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:32.001+0000 7faa5aff5740 1 client.26167 shutdown 2024-08-30T09:06:32.009 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:32.005+0000 7faa5aff5740 1 -- 172.21.15.42:0/2490685949 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561949a184d0 msgr2=0x561949de8d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:06:32.009 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:32.005+0000 7faa5aff5740 1 --2- 172.21.15.42:0/2490685949 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561949a184d0 0x561949de8d00 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7faa4801b040 tx=0x7faa480098c0 comp rx=0 tx=0).stop 2024-08-30T09:06:32.012 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:32.010+0000 7faa5aff5740 1 -- 172.21.15.42:0/2490685949 shutdown_connections 2024-08-30T09:06:32.012 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:32.010+0000 7faa5aff5740 1 --2- 172.21.15.42:0/2490685949 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x5619499cd1c0 0x561949df6430 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:32.012 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:32.010+0000 7faa357fa640 0 client.26167 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:06:32.012 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:32.010+0000 7faa5aff5740 1 --2- 172.21.15.42:0/2490685949 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5619499f7d80 0x561949de9f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:32.012 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:32.010+0000 7faa5aff5740 1 --2- 172.21.15.42:0/2490685949 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561949a391b0 0x561949de93c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:32.012 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:32.010+0000 7faa5aff5740 1 --2- 172.21.15.42:0/2490685949 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561949a184d0 0x561949de8d00 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:32.012 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:32.010+0000 7faa5aff5740 1 -- 172.21.15.42:0/2490685949 >> 172.21.15.42:0/2490685949 conn(0x561949a2d300 msgr2=0x561949dd9170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:06:32.014 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:32.013+0000 7faa5aff5740 1 -- 172.21.15.42:0/2490685949 shutdown_connections 2024-08-30T09:06:32.017 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:32.013+0000 7faa5aff5740 1 -- 172.21.15.42:0/2490685949 wait complete. 2024-08-30T09:06:32.017 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:32.013+0000 7faa5aff5740 20 client.26167 trim_cache size 0 max 16384 2024-08-30T09:06:32.061 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:32.059+0000 7faa5aff5740 1 Processor -- start 2024-08-30T09:06:32.061 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:32.059+0000 7faa5aff5740 1 -- start start 2024-08-30T09:06:32.062 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:32.059+0000 7faa5aff5740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5619499f7d80 0x561949dd5440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:32.062 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:32.059+0000 7faa5aff5740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561949a184d0 0x561949de7f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:32.062 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:32.059+0000 7faa5aff5740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561949a391b0 0x561949de8440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:32.062 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:32.059+0000 7faa5aff5740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7faa4c1b5380 con 0x5619499f7d80 2024-08-30T09:06:32.062 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:32.059+0000 7faa5aff5740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7faa4c1b2bf0 con 0x561949a184d0 2024-08-30T09:06:32.062 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:32.059+0000 7faa5aff5740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7faa4c1b0a40 con 0x561949a391b0 2024-08-30T09:06:32.062 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:32.060+0000 7faa51548640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561949a184d0 0x561949de7f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:06:32.062 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:32.060+0000 7faa51548640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561949a184d0 0x561949de7f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:56630/0 (socket says 172.21.15.42:56630) 2024-08-30T09:06:32.062 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:32.060+0000 7faa51548640 1 -- 172.21.15.42:0/4109259853 learned_addr learned my addr 172.21.15.42:0/4109259853 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:06:32.062 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:32.060+0000 7faa5254a640 1 --2- 172.21.15.42:0/4109259853 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561949a391b0 0x561949de8440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:06:32.063 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:32.060+0000 7faa51548640 1 -- 172.21.15.42:0/4109259853 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561949a391b0 msgr2=0x561949de8440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:06:32.063 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:32.060+0000 7faa51548640 1 --2- 172.21.15.42:0/4109259853 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561949a391b0 0x561949de8440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:32.063 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:32.060+0000 7faa51548640 1 -- 172.21.15.42:0/4109259853 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5619499f7d80 msgr2=0x561949dd5440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:06:32.063 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:32.060+0000 7faa51548640 1 --2- 172.21.15.42:0/4109259853 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5619499f7d80 0x561949dd5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:32.063 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:32.060+0000 7faa51548640 1 -- 172.21.15.42:0/4109259853 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561949dd8f30 con 0x561949a184d0 2024-08-30T09:06:32.063 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:32.061+0000 7faa51548640 1 --2- 172.21.15.42:0/4109259853 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561949a184d0 0x561949de7f00 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7faa3c0ab940 tx=0x7faa3c0e5720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=16e5f73c4bfdc621 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:06:32.064 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:32.062+0000 7faa377fe640 1 -- 172.21.15.42:0/4109259853 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa3c0b0020 con 0x561949a184d0 2024-08-30T09:06:32.064 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:32.062+0000 7faa377fe640 1 -- 172.21.15.42:0/4109259853 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faa3c0b01c0 con 0x561949a184d0 2024-08-30T09:06:32.064 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:32.062+0000 7faa377fe640 1 -- 172.21.15.42:0/4109259853 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa3c01c040 con 0x561949a184d0 2024-08-30T09:06:32.065 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:32.063+0000 7faa5aff5740 1 -- 172.21.15.42:0/4109259853 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561949a184d0 msgr2=0x561949de7f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:06:32.065 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:32.063+0000 7faa5aff5740 1 --2- 172.21.15.42:0/4109259853 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561949a184d0 0x561949de7f00 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7faa3c0ab940 tx=0x7faa3c0e5720 comp rx=0 tx=0).stop 2024-08-30T09:06:32.066 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:32.063+0000 7faa5aff5740 1 -- 172.21.15.42:0/4109259853 shutdown_connections 2024-08-30T09:06:32.066 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:32.063+0000 7faa5aff5740 1 --2- 172.21.15.42:0/4109259853 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561949a391b0 0x561949de8440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:32.066 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:32.063+0000 7faa5aff5740 1 --2- 172.21.15.42:0/4109259853 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561949a184d0 0x561949de7f00 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:32.066 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:32.063+0000 7faa5aff5740 1 --2- 172.21.15.42:0/4109259853 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5619499f7d80 0x561949dd5440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:32.066 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:32.064+0000 7faa5aff5740 1 -- 172.21.15.42:0/4109259853 >> 172.21.15.42:0/4109259853 conn(0x561949a2d300 msgr2=0x561949dded70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:06:32.067 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:32.065+0000 7faa5aff5740 1 -- 172.21.15.42:0/4109259853 shutdown_connections 2024-08-30T09:06:32.067 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:32.065+0000 7faa5aff5740 1 -- 172.21.15.42:0/4109259853 wait complete. 2024-08-30T09:06:32.067 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:32.065+0000 7faa5aff5740 1 -- start start 2024-08-30T09:06:32.067 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:32.066+0000 7faa5aff5740 1 Processor -- start 2024-08-30T09:06:32.068 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:32.066+0000 7faa5aff5740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5619499f7d80 0x561949df0480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:32.068 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:32.066+0000 7faa5aff5740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561949a184d0 0x561949def140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:32.068 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:32.066+0000 7faa5aff5740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561949a391b0 0x561949def680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:32.068 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:32.066+0000 7faa5aff5740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7faa4c1b10b0 con 0x5619499f7d80 2024-08-30T09:06:32.068 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:32.066+0000 7faa5aff5740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7faa4c1b2520 con 0x561949a184d0 2024-08-30T09:06:32.068 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:32.066+0000 7faa5aff5740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7faa4c1b1780 con 0x561949a391b0 2024-08-30T09:06:32.069 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:32.066+0000 7faa5aff5740 10 client.0 _ll_register_callbacks cb 0x7faa3c005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:06:32.069 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:32.067+0000 7faa5254a640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561949a391b0 0x561949def680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:06:32.069 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:32.067+0000 7faa51548640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561949a184d0 0x561949def140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:06:32.069 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:32.067+0000 7faa51548640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561949a184d0 0x561949def140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:56642/0 (socket says 172.21.15.42:56642) 2024-08-30T09:06:32.069 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:32.067+0000 7faa51548640 1 -- 172.21.15.42:0/352911675 learned_addr learned my addr 172.21.15.42:0/352911675 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:06:32.069 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:32.067+0000 7faa51d49640 1 --2- 172.21.15.42:0/352911675 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5619499f7d80 0x561949df0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:06:32.070 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:32.067+0000 7faa51548640 1 -- 172.21.15.42:0/352911675 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561949a391b0 msgr2=0x561949def680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:06:32.070 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:32.067+0000 7faa51548640 1 --2- 172.21.15.42:0/352911675 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561949a391b0 0x561949def680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:32.070 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:32.067+0000 7faa51548640 1 -- 172.21.15.42:0/352911675 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5619499f7d80 msgr2=0x561949df0480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:06:32.070 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:32.067+0000 7faa51548640 1 --2- 172.21.15.42:0/352911675 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5619499f7d80 0x561949df0480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:32.070 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:32.067+0000 7faa51548640 1 -- 172.21.15.42:0/352911675 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561949defcc0 con 0x561949a184d0 2024-08-30T09:06:32.071 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:32.068+0000 7faa51548640 1 --2- 172.21.15.42:0/352911675 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561949a184d0 0x561949def140 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7faa3c0a7bf0 tx=0x7faa3c0aba80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=74a4a1a4f64fe896 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:06:32.071 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:32.068+0000 7faa357fa640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-30T09:06:32.071 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:32.068+0000 7faa357fa640 1 -- 172.21.15.42:0/352911675 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa3c01c040 con 0x561949a184d0 2024-08-30T09:06:32.071 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:32.069+0000 7faa357fa640 1 -- 172.21.15.42:0/352911675 <== mon.2 v2:172.21.15.167:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7faa3c0054d0 con 0x561949a184d0 2024-08-30T09:06:32.071 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:32.069+0000 7faa5aff5740 1 -- 172.21.15.42:0/352911675 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faa3c0b01c0 con 0x561949a184d0 2024-08-30T09:06:32.071 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:32.069+0000 7faa357fa640 1 -- 172.21.15.42:0/352911675 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7faa3c0051d0 con 0x561949a184d0 2024-08-30T09:06:32.071 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:32.069+0000 7faa5aff5740 10 client.26189 fetch_fsmap learned FSMap version 33 2024-08-30T09:06:32.071 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:32.069+0000 7faa5aff5740 1 -- 172.21.15.42:0/352911675 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x561949deec10 con 0x561949a184d0 2024-08-30T09:06:32.073 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:32.071+0000 7faa357fa640 1 -- 172.21.15.42:0/352911675 <== mon.2 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faa3c0b01c0 con 0x561949a184d0 2024-08-30T09:06:32.073 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:32.071+0000 7faa357fa640 1 -- 172.21.15.42:0/352911675 <== mon.2 v2:172.21.15.167:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7faa3c0ceb60 con 0x561949a184d0 2024-08-30T09:06:32.073 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:32.071+0000 7faa5aff5740 10 client.26189 fetch_fsmap finished waiting for FSMap version 33 2024-08-30T09:06:32.073 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:32.071+0000 7faa5aff5740 10 client.26189 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-30T09:06:32.073 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:32.071+0000 7faa5aff5740 20 client.26189 populate_metadata read hostname 'smithi042' 2024-08-30T09:06:32.073 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:32.071+0000 7faa5aff5740 1 --2- 172.21.15.42:0/352911675 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x561949e72660 0x561949e74a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:32.073 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:32.071+0000 7faa5aff5740 4 client.26189 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-30T09:06:32.073 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:32.071+0000 7faa5aff5740 1 -- 172.21.15.42:0/352911675 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561949d4f9f0 con 0x561949e72660 2024-08-30T09:06:32.074 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:32.073+0000 7faa51d49640 1 --2- 172.21.15.42:0/352911675 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x561949e72660 0x561949e74a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:06:32.078 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:32.076+0000 7faa51d49640 1 --2- 172.21.15.42:0/352911675 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x561949e72660 0x561949e74a80 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=88c136d2e6ea29d0 server_cookie=795e9ea200888591 in_seq=0 out_seq=0 2024-08-30T09:06:32.078 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:32.076+0000 7faa357fa640 10 client.26189 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:06:32.116 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:32.114+0000 7faa357fa640 1 -- 172.21.15.42:0/352911675 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561949d4f9f0 con 0x561949e72660 2024-08-30T09:06:32.117 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:32.114+0000 7faa357fa640 10 client.26189 handle_command_reply: tid=0 2024-08-30T09:06:32.117 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:32.114+0000 7faa5aff5740 1 client.26189 shutdown 2024-08-30T09:06:32.117 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:32.115+0000 7faa5aff5740 1 -- 172.21.15.42:0/352911675 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561949a184d0 msgr2=0x561949def140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:06:32.117 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:32.115+0000 7faa5aff5740 1 --2- 172.21.15.42:0/352911675 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561949a184d0 0x561949def140 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7faa3c0a7bf0 tx=0x7faa3c0aba80 comp rx=0 tx=0).stop 2024-08-30T09:06:32.117 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:32.115+0000 7faa5aff5740 1 -- 172.21.15.42:0/352911675 shutdown_connections 2024-08-30T09:06:32.118 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:32.115+0000 7faa5aff5740 1 --2- 172.21.15.42:0/352911675 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x561949e72660 0x561949e74a80 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:32.118 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:32.115+0000 7faa5aff5740 1 --2- 172.21.15.42:0/352911675 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561949a391b0 0x561949def680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:32.118 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:32.115+0000 7faa5aff5740 1 --2- 172.21.15.42:0/352911675 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561949a184d0 0x561949def140 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:32.118 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:32.115+0000 7faa5aff5740 1 --2- 172.21.15.42:0/352911675 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5619499f7d80 0x561949df0480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:32.118 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:32.115+0000 7faa5aff5740 1 -- 172.21.15.42:0/352911675 >> 172.21.15.42:0/352911675 conn(0x561949a2d300 msgr2=0x561949ddffc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:06:32.118 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:32.115+0000 7faa357fa640 0 client.26189 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:06:32.118 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:32.116+0000 7faa5aff5740 1 -- 172.21.15.42:0/352911675 shutdown_connections 2024-08-30T09:06:32.118 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:32.116+0000 7faa5aff5740 1 -- 172.21.15.42:0/352911675 wait complete. 2024-08-30T09:06:32.118 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:32.116+0000 7faa5aff5740 20 client.26189 trim_cache size 0 max 16384 2024-08-30T09:06:32.118 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-08-30T09:06:32.119 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2024-08-30T09:06:32.119 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-08-30T09:06:32.119 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-08-30T09:06:32.120 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:32.117+0000 7faa53fad640 1 -- 172.21.15.42:0/442590217 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7faa380b4bb0 msgr2=0x7faa380b70c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:06:32.120 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:32.117+0000 7faa53fad640 1 --2- 172.21.15.42:0/442590217 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7faa380b4bb0 0x7faa380b70c0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7faa44009940 tx=0x7faa440095c0 comp rx=0 tx=0).stop 2024-08-30T09:06:32.120 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:32.117+0000 7faa53fad640 1 -- 172.21.15.42:0/442590217 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faa4c105480 msgr2=0x7faa4c1a08e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:06:32.120 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:32.117+0000 7faa53fad640 1 --2- 172.21.15.42:0/442590217 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faa4c105480 0x7faa4c1a08e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7faa3c00e9f0 tx=0x7faa3c00eeb0 comp rx=0 tx=0).stop 2024-08-30T09:06:32.120 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:32.117+0000 7faa53fad640 1 -- 172.21.15.42:0/442590217 shutdown_connections 2024-08-30T09:06:32.120 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:32.117+0000 7faa53fad640 1 --2- 172.21.15.42:0/442590217 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7faa380b4bb0 0x7faa380b70c0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:32.120 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:32.117+0000 7faa53fad640 1 --2- 172.21.15.42:0/442590217 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faa4c105480 0x7faa4c1a08e0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:32.120 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:32.117+0000 7faa53fad640 1 --2- 172.21.15.42:0/442590217 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faa4c1080a0 0x7faa4c1a1490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:32.120 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:32.117+0000 7faa53fad640 1 --2- 172.21.15.42:0/442590217 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faa4c102a40 0x7faa4c1a0270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:32.120 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:32.117+0000 7faa53fad640 1 -- 172.21.15.42:0/442590217 >> 172.21.15.42:0/442590217 conn(0x7faa4c0fd430 msgr2=0x7faa4c1031a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:06:32.120 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:32.117+0000 7faa53fad640 1 -- 172.21.15.42:0/442590217 shutdown_connections 2024-08-30T09:06:32.120 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:32.118+0000 7faa53fad640 1 -- 172.21.15.42:0/442590217 wait complete. 2024-08-30T09:06:32.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:06:32 smithi167 ceph-mon[22063]: pgmap v546: 129 pgs: 129 active+clean; 1.4 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.5 MiB/s wr, 508 op/s 2024-08-30T09:06:32.587 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:06:32 smithi042 ceph-mon[18652]: pgmap v546: 129 pgs: 129 active+clean; 1.4 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.5 MiB/s wr, 508 op/s 2024-08-30T09:06:32.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:06:32 smithi190 ceph-mon[22001]: pgmap v546: 129 pgs: 129 active+clean; 1.4 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.5 MiB/s wr, 508 op/s 2024-08-30T09:06:34.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.116+0000 7f49f66ea640 1 -- 172.21.15.42:0/2905568109 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f49f0111ce0 msgr2=0x7f49f010a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:06:34.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.116+0000 7f49f66ea640 1 --2- 172.21.15.42:0/2905568109 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f49f0111ce0 0x7f49f010a160 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f49e4009860 tx=0x7f49e402a360 comp rx=0 tx=0).stop 2024-08-30T09:06:34.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.120+0000 7f49f66ea640 1 -- 172.21.15.42:0/2905568109 shutdown_connections 2024-08-30T09:06:34.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.120+0000 7f49f66ea640 1 --2- 172.21.15.42:0/2905568109 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f49f010a780 0x7f49f010ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:34.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.120+0000 7f49f66ea640 1 --2- 172.21.15.42:0/2905568109 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f49f011c9d0 0x7f49f011eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:34.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.120+0000 7f49f66ea640 1 --2- 172.21.15.42:0/2905568109 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f49f0111ce0 0x7f49f010a160 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:34.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.120+0000 7f49f66ea640 1 -- 172.21.15.42:0/2905568109 >> 172.21.15.42:0/2905568109 conn(0x7f49f0070c60 msgr2=0x7f49f0071070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:06:34.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.121+0000 7f49f66ea640 1 -- 172.21.15.42:0/2905568109 shutdown_connections 2024-08-30T09:06:34.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.122+0000 7f49f66ea640 1 -- 172.21.15.42:0/2905568109 wait complete. 2024-08-30T09:06:34.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.123+0000 7f49f66ea640 1 Processor -- start 2024-08-30T09:06:34.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.125+0000 7f49f66ea640 1 -- start start 2024-08-30T09:06:34.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.125+0000 7f49f66ea640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f49f010a780 0x7f49f01a9080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:34.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.125+0000 7f49f66ea640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f49f0111ce0 0x7f49f01a9620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:34.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.125+0000 7f49f66ea640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f49f011c9d0 0x7f49f01ae8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:34.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.125+0000 7f49f66ea640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f49f0122690 con 0x7f49f0111ce0 2024-08-30T09:06:34.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.125+0000 7f49f66ea640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f49f0122510 con 0x7f49f011c9d0 2024-08-30T09:06:34.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.125+0000 7f49f66ea640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f49f0122810 con 0x7f49f010a780 2024-08-30T09:06:34.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.126+0000 7f49ef7fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f49f0111ce0 0x7f49f01a9620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:06:34.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.126+0000 7f49ef7fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f49f0111ce0 0x7f49f01a9620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:45652/0 (socket says 172.21.15.42:45652) 2024-08-30T09:06:34.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.126+0000 7f49ef7fe640 1 -- 172.21.15.42:0/2444130898 learned_addr learned my addr 172.21.15.42:0/2444130898 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:06:34.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.126+0000 7f49f4c87640 1 --2- 172.21.15.42:0/2444130898 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f49f011c9d0 0x7f49f01ae8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:06:34.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.126+0000 7f49effff640 1 --2- 172.21.15.42:0/2444130898 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f49f010a780 0x7f49f01a9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:06:34.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.127+0000 7f49f4c87640 1 -- 172.21.15.42:0/2444130898 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f49f010a780 msgr2=0x7f49f01a9080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:06:34.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.127+0000 7f49f4c87640 1 --2- 172.21.15.42:0/2444130898 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f49f010a780 0x7f49f01a9080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:34.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.127+0000 7f49f4c87640 1 -- 172.21.15.42:0/2444130898 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f49f0111ce0 msgr2=0x7f49f01a9620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:06:34.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.127+0000 7f49f4c87640 1 --2- 172.21.15.42:0/2444130898 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f49f0111ce0 0x7f49f01a9620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:34.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.127+0000 7f49f4c87640 1 -- 172.21.15.42:0/2444130898 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49f01aefc0 con 0x7f49f011c9d0 2024-08-30T09:06:34.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.127+0000 7f49f4c87640 1 --2- 172.21.15.42:0/2444130898 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f49f011c9d0 0x7f49f01ae8c0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f49e000e9a0 tx=0x7f49e000ee60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:06:34.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.127+0000 7f49ed7fa640 1 -- 172.21.15.42:0/2444130898 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49e000ccc0 con 0x7f49f011c9d0 2024-08-30T09:06:34.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.128+0000 7f49f66ea640 1 -- 172.21.15.42:0/2444130898 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49f01af2b0 con 0x7f49f011c9d0 2024-08-30T09:06:34.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.128+0000 7f49f66ea640 1 -- 172.21.15.42:0/2444130898 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49f0112120 con 0x7f49f011c9d0 2024-08-30T09:06:34.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.128+0000 7f49ed7fa640 1 -- 172.21.15.42:0/2444130898 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f49e000ce60 con 0x7f49f011c9d0 2024-08-30T09:06:34.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.128+0000 7f49ed7fa640 1 -- 172.21.15.42:0/2444130898 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49e0004a00 con 0x7f49f011c9d0 2024-08-30T09:06:34.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.131+0000 7f49fcc5f740 1 Processor -- start 2024-08-30T09:06:34.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.134+0000 7f49fcc5f740 1 -- start start 2024-08-30T09:06:34.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.134+0000 7f49fcc5f740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a8b7dd9070 0x55a8b817e310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:34.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.134+0000 7f49fcc5f740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a8b7db01b0 0x55a8b817e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:34.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.134+0000 7f49fcc5f740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a8b7db8390 0x55a8b817ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:34.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.134+0000 7f49fcc5f740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55a8b80f5680 con 0x55a8b7db01b0 2024-08-30T09:06:34.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.134+0000 7f49fcc5f740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55a8b80e4c10 con 0x55a8b7db8390 2024-08-30T09:06:34.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.134+0000 7f49fcc5f740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55a8b80e8bc0 con 0x55a8b7dd9070 2024-08-30T09:06:34.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.135+0000 7f49ed7fa640 1 -- 172.21.15.42:0/2444130898 <== mon.2 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7f49e0004ba0 con 0x7f49f011c9d0 2024-08-30T09:06:34.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.135+0000 7f49effff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a8b7db01b0 0x55a8b817e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:06:34.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.135+0000 7f49effff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a8b7db01b0 0x55a8b817e850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:45668/0 (socket says 172.21.15.42:45668) 2024-08-30T09:06:34.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.135+0000 7f49effff640 1 -- 172.21.15.42:0/4024582102 learned_addr learned my addr 172.21.15.42:0/4024582102 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:06:34.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.135+0000 7f49effff640 1 -- 172.21.15.42:0/4024582102 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a8b7dd9070 msgr2=0x55a8b817e310 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-30T09:06:34.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.135+0000 7f49effff640 1 --2- 172.21.15.42:0/4024582102 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a8b7dd9070 0x55a8b817e310 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:34.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.136+0000 7f49effff640 1 -- 172.21.15.42:0/4024582102 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a8b7db8390 msgr2=0x55a8b817ed90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-30T09:06:34.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.136+0000 7f49effff640 1 --2- 172.21.15.42:0/4024582102 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a8b7db8390 0x55a8b817ed90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:34.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.136+0000 7f49effff640 1 -- 172.21.15.42:0/4024582102 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a8b7e44650 con 0x55a8b7db01b0 2024-08-30T09:06:34.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.137+0000 7f49effff640 1 --2- 172.21.15.42:0/4024582102 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a8b7db01b0 0x55a8b817e850 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f49e4009780 tx=0x7f49e40303b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69c7647b9c0ff2f2 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:06:34.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.138+0000 7f49ed7fa640 1 --2- 172.21.15.42:0/2444130898 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f49d80b4c60 0x7f49d80b7170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:34.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.138+0000 7f49ed7fa640 1 -- 172.21.15.42:0/2444130898 <== mon.2 v2:172.21.15.167:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f49e00e5180 con 0x7f49f011c9d0 2024-08-30T09:06:34.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.141+0000 7f49d67fc640 1 -- 172.21.15.42:0/4024582102 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49e4043020 con 0x55a8b7db01b0 2024-08-30T09:06:34.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.141+0000 7f49d67fc640 1 -- 172.21.15.42:0/4024582102 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f49e4038040 con 0x55a8b7db01b0 2024-08-30T09:06:34.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.141+0000 7f49d67fc640 1 -- 172.21.15.42:0/4024582102 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49e4037040 con 0x55a8b7db01b0 2024-08-30T09:06:34.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.144+0000 7f49fcc5f740 1 -- 172.21.15.42:0/4024582102 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a8b7db01b0 msgr2=0x55a8b817e850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:06:34.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.144+0000 7f49fcc5f740 1 --2- 172.21.15.42:0/4024582102 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a8b7db01b0 0x55a8b817e850 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f49e4009780 tx=0x7f49e40303b0 comp rx=0 tx=0).stop 2024-08-30T09:06:34.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.146+0000 7f49fcc5f740 1 -- 172.21.15.42:0/4024582102 shutdown_connections 2024-08-30T09:06:34.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.146+0000 7f49fcc5f740 1 --2- 172.21.15.42:0/4024582102 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a8b7db8390 0x55a8b817ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:34.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.146+0000 7f49fcc5f740 1 --2- 172.21.15.42:0/4024582102 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a8b7db01b0 0x55a8b817e850 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:34.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.146+0000 7f49fcc5f740 1 --2- 172.21.15.42:0/4024582102 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a8b7dd9070 0x55a8b817e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:34.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.146+0000 7f49fcc5f740 1 -- 172.21.15.42:0/4024582102 >> 172.21.15.42:0/4024582102 conn(0x55a8b7dcd460 msgr2=0x55a8b817c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:06:34.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.147+0000 7f49ef7fe640 1 --2- 172.21.15.42:0/2444130898 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f49d80b4c60 0x7f49d80b7170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:06:34.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.147+0000 7f49fcc5f740 1 -- 172.21.15.42:0/4024582102 shutdown_connections 2024-08-30T09:06:34.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.150+0000 7f49fcc5f740 1 -- 172.21.15.42:0/4024582102 wait complete. 2024-08-30T09:06:34.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.151+0000 7f49fcc5f740 1 -- start start 2024-08-30T09:06:34.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.151+0000 7f49fcc5f740 1 Processor -- start 2024-08-30T09:06:34.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.152+0000 7f49fcc5f740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a8b7db01b0 0x55a8b8189a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:34.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.152+0000 7f49fcc5f740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a8b7db8390 0x55a8b8189fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:34.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.152+0000 7f49fcc5f740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a8b7dd9070 0x55a8b8195e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:34.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.152+0000 7f49fcc5f740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55a8b7f47700 con 0x55a8b7db01b0 2024-08-30T09:06:34.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.152+0000 7f49fcc5f740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55a8b806b750 con 0x55a8b7db8390 2024-08-30T09:06:34.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.152+0000 7f49fcc5f740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55a8b818ca90 con 0x55a8b7dd9070 2024-08-30T09:06:34.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.152+0000 7f49fcc5f740 10 client.0 _ll_register_callbacks cb 0x55a8b809ae10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:06:34.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.153+0000 7f49f4c87640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a8b7dd9070 0x55a8b8195e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:06:34.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.153+0000 7f49f4c87640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a8b7dd9070 0x55a8b8195e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:33062/0 (socket says 172.21.15.42:33062) 2024-08-30T09:06:34.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.153+0000 7f49f4c87640 1 -- 172.21.15.42:0/1797940970 learned_addr learned my addr 172.21.15.42:0/1797940970 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:06:34.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.153+0000 7f49ef7fe640 1 --2- 172.21.15.42:0/1797940970 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a8b7db8390 0x55a8b8189fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:06:34.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.154+0000 7f49effff640 1 --2- 172.21.15.42:0/1797940970 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a8b7db01b0 0x55a8b8189a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:06:34.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.155+0000 7f49effff640 1 -- 172.21.15.42:0/1797940970 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a8b7dd9070 msgr2=0x55a8b8195e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:06:34.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.155+0000 7f49effff640 1 --2- 172.21.15.42:0/1797940970 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a8b7dd9070 0x55a8b8195e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:34.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.155+0000 7f49effff640 1 -- 172.21.15.42:0/1797940970 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a8b7db8390 msgr2=0x55a8b8189fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:06:34.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.155+0000 7f49effff640 1 --2- 172.21.15.42:0/1797940970 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a8b7db8390 0x55a8b8189fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:34.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.155+0000 7f49effff640 1 -- 172.21.15.42:0/1797940970 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a8b80a4900 con 0x55a8b7db01b0 2024-08-30T09:06:34.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.156+0000 7f49ef7fe640 1 --2- 172.21.15.42:0/2444130898 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f49d80b4c60 0x7f49d80b7170 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f49f011bf50 tx=0x7f49e800fb20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:06:34.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.156+0000 7f49effff640 1 --2- 172.21.15.42:0/1797940970 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a8b7db01b0 0x55a8b8189a90 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f49e402a950 tx=0x7f49e402a360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9771689ecf8e381 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:06:34.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.156+0000 7f49cf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-08-30T09:06:34.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.157+0000 7f49cf7fe640 1 -- 172.21.15.42:0/1797940970 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49e4037040 con 0x55a8b7db01b0 2024-08-30T09:06:34.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.157+0000 7f49fcc5f740 1 -- 172.21.15.42:0/1797940970 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f49e4038040 con 0x55a8b7db01b0 2024-08-30T09:06:34.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.157+0000 7f49cf7fe640 1 -- 172.21.15.42:0/1797940970 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f49e40306f0 con 0x55a8b7db01b0 2024-08-30T09:06:34.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.157+0000 7f49cf7fe640 1 -- 172.21.15.42:0/1797940970 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f49e4058050 con 0x55a8b7db01b0 2024-08-30T09:06:34.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.158+0000 7f49fcc5f740 10 client.16452 fetch_fsmap learned FSMap version 33 2024-08-30T09:06:34.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.158+0000 7f49fcc5f740 1 -- 172.21.15.42:0/1797940970 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55a8b8198890 con 0x55a8b7db01b0 2024-08-30T09:06:34.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.159+0000 7f49cf7fe640 1 -- 172.21.15.42:0/1797940970 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f49e4038040 con 0x55a8b7db01b0 2024-08-30T09:06:34.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.159+0000 7f49cf7fe640 1 -- 172.21.15.42:0/1797940970 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f49e404a4c0 con 0x55a8b7db01b0 2024-08-30T09:06:34.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.162+0000 7f49fcc5f740 10 client.16452 fetch_fsmap finished waiting for FSMap version 33 2024-08-30T09:06:34.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.162+0000 7f49fcc5f740 10 client.16452 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-30T09:06:34.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.162+0000 7f49fcc5f740 20 client.16452 populate_metadata read hostname 'smithi042' 2024-08-30T09:06:34.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.162+0000 7f49fcc5f740 1 --2- 172.21.15.42:0/1797940970 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55a8b7d979f0 0x55a8b819bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:34.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.162+0000 7f49fcc5f740 4 client.16452 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-30T09:06:34.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.162+0000 7f49fcc5f740 1 -- 172.21.15.42:0/1797940970 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a8b80f75d0 con 0x55a8b7d979f0 2024-08-30T09:06:34.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.163+0000 7f49ef7fe640 1 --2- 172.21.15.42:0/1797940970 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55a8b7d979f0 0x55a8b819bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:06:34.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.171+0000 7f49ef7fe640 1 --2- 172.21.15.42:0/1797940970 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55a8b7d979f0 0x55a8b819bbf0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a9dd491b567ee39 server_cookie=e7179f91637f1217 in_seq=0 out_seq=0 2024-08-30T09:06:34.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.171+0000 7f49cf7fe640 10 client.16452 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:06:34.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.174+0000 7f49cf7fe640 1 -- 172.21.15.42:0/1797940970 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a8b80f75d0 con 0x55a8b7d979f0 2024-08-30T09:06:34.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.174+0000 7f49cf7fe640 10 client.16452 handle_command_reply: tid=0 2024-08-30T09:06:34.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.175+0000 7f49fcc5f740 1 client.16452 shutdown 2024-08-30T09:06:34.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.175+0000 7f49fcc5f740 1 -- 172.21.15.42:0/1797940970 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a8b7db01b0 msgr2=0x55a8b8189a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:06:34.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.175+0000 7f49fcc5f740 1 --2- 172.21.15.42:0/1797940970 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a8b7db01b0 0x55a8b8189a90 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f49e402a950 tx=0x7f49e402a360 comp rx=0 tx=0).stop 2024-08-30T09:06:34.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.175+0000 7f49fcc5f740 1 -- 172.21.15.42:0/1797940970 shutdown_connections 2024-08-30T09:06:34.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.175+0000 7f49fcc5f740 1 --2- 172.21.15.42:0/1797940970 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55a8b7d979f0 0x55a8b819bbf0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:34.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.175+0000 7f49fcc5f740 1 --2- 172.21.15.42:0/1797940970 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a8b7dd9070 0x55a8b8195e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:34.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.175+0000 7f49fcc5f740 1 --2- 172.21.15.42:0/1797940970 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a8b7db8390 0x55a8b8189fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:34.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.175+0000 7f49fcc5f740 1 --2- 172.21.15.42:0/1797940970 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a8b7db01b0 0x55a8b8189a90 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:34.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.175+0000 7f49fcc5f740 1 -- 172.21.15.42:0/1797940970 >> 172.21.15.42:0/1797940970 conn(0x55a8b7dcd460 msgr2=0x55a8b81814a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:06:34.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.176+0000 7f49cf7fe640 0 client.16452 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:06:34.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.176+0000 7f49fcc5f740 1 -- 172.21.15.42:0/1797940970 shutdown_connections 2024-08-30T09:06:34.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.176+0000 7f49fcc5f740 1 -- 172.21.15.42:0/1797940970 wait complete. 2024-08-30T09:06:34.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.177+0000 7f49fcc5f740 20 client.16452 trim_cache size 0 max 16384 2024-08-30T09:06:34.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.203+0000 7f49fcc5f740 1 Processor -- start 2024-08-30T09:06:34.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.203+0000 7f49fcc5f740 1 -- start start 2024-08-30T09:06:34.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.204+0000 7f49fcc5f740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a8b7db01b0 0x55a8b8189f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:34.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.204+0000 7f49fcc5f740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a8b7db8390 0x55a8b8181410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:34.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.204+0000 7f49fcc5f740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a8b7dd9070 0x55a8b8181950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:34.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.204+0000 7f49fcc5f740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f49dc012600 con 0x55a8b7db8390 2024-08-30T09:06:34.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.204+0000 7f49fcc5f740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f49dc00fe70 con 0x55a8b7db01b0 2024-08-30T09:06:34.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.204+0000 7f49fcc5f740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f49dc00dcc0 con 0x55a8b7dd9070 2024-08-30T09:06:34.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.204+0000 7f49f4c87640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a8b7dd9070 0x55a8b8181950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:06:34.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.204+0000 7f49f4c87640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a8b7dd9070 0x55a8b8181950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:33066/0 (socket says 172.21.15.42:33066) 2024-08-30T09:06:34.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.204+0000 7f49f4c87640 1 -- 172.21.15.42:0/3772304586 learned_addr learned my addr 172.21.15.42:0/3772304586 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:06:34.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.205+0000 7f49effff640 1 --2- 172.21.15.42:0/3772304586 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a8b7db01b0 0x55a8b8189f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:06:34.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.205+0000 7f49f4c87640 1 -- 172.21.15.42:0/3772304586 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a8b7db01b0 msgr2=0x55a8b8189f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:06:34.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.205+0000 7f49f4c87640 1 --2- 172.21.15.42:0/3772304586 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a8b7db01b0 0x55a8b8189f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:34.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.205+0000 7f49f4c87640 1 -- 172.21.15.42:0/3772304586 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a8b7db8390 msgr2=0x55a8b8181410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:06:34.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.205+0000 7f49f4c87640 1 --2- 172.21.15.42:0/3772304586 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a8b7db8390 0x55a8b8181410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:34.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.205+0000 7f49f4c87640 1 -- 172.21.15.42:0/3772304586 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a8b8181170 con 0x55a8b7dd9070 2024-08-30T09:06:34.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.205+0000 7f49effff640 1 --2- 172.21.15.42:0/3772304586 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a8b7db01b0 0x55a8b8189f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-30T09:06:34.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.206+0000 7f49f4c87640 1 --2- 172.21.15.42:0/3772304586 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a8b7dd9070 0x55a8b8181950 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f49e0004770 tx=0x7f49e00aaf30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b7d657c152e02c32 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:06:34.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.206+0000 7f49d5ffb640 1 -- 172.21.15.42:0/3772304586 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49e001c040 con 0x55a8b7dd9070 2024-08-30T09:06:34.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.206+0000 7f49d5ffb640 1 -- 172.21.15.42:0/3772304586 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f49e00abec0 con 0x55a8b7dd9070 2024-08-30T09:06:34.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.206+0000 7f49d5ffb640 1 -- 172.21.15.42:0/3772304586 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49e00b48b0 con 0x55a8b7dd9070 2024-08-30T09:06:34.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.208+0000 7f49fcc5f740 1 -- 172.21.15.42:0/3772304586 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a8b7dd9070 msgr2=0x55a8b8181950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:06:34.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.208+0000 7f49fcc5f740 1 --2- 172.21.15.42:0/3772304586 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a8b7dd9070 0x55a8b8181950 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f49e0004770 tx=0x7f49e00aaf30 comp rx=0 tx=0).stop 2024-08-30T09:06:34.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.208+0000 7f49fcc5f740 1 -- 172.21.15.42:0/3772304586 shutdown_connections 2024-08-30T09:06:34.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.208+0000 7f49fcc5f740 1 --2- 172.21.15.42:0/3772304586 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a8b7dd9070 0x55a8b8181950 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:34.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.208+0000 7f49fcc5f740 1 --2- 172.21.15.42:0/3772304586 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a8b7db8390 0x55a8b8181410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:34.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.208+0000 7f49fcc5f740 1 --2- 172.21.15.42:0/3772304586 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a8b7db01b0 0x55a8b8189f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:34.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.208+0000 7f49fcc5f740 1 -- 172.21.15.42:0/3772304586 >> 172.21.15.42:0/3772304586 conn(0x55a8b7dcd460 msgr2=0x55a8b8189840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:06:34.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.208+0000 7f49fcc5f740 1 -- 172.21.15.42:0/3772304586 shutdown_connections 2024-08-30T09:06:34.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.208+0000 7f49fcc5f740 1 -- 172.21.15.42:0/3772304586 wait complete. 2024-08-30T09:06:34.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.209+0000 7f49fcc5f740 1 -- start start 2024-08-30T09:06:34.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.209+0000 7f49fcc5f740 1 Processor -- start 2024-08-30T09:06:34.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.210+0000 7f49fcc5f740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a8b7db01b0 0x55a8b819cde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:34.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.210+0000 7f49fcc5f740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a8b7db8390 0x55a8b819d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:34.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.210+0000 7f49fcc5f740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a8b7dd9070 0x55a8b819d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:34.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.210+0000 7f49fcc5f740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f49dc00e330 con 0x55a8b7db01b0 2024-08-30T09:06:34.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.210+0000 7f49fcc5f740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f49dc00f7a0 con 0x55a8b7db8390 2024-08-30T09:06:34.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.210+0000 7f49fcc5f740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f49dc00ea00 con 0x55a8b7dd9070 2024-08-30T09:06:34.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.210+0000 7f49fcc5f740 10 client.0 _ll_register_callbacks cb 0x7f49e40096b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:06:34.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.210+0000 7f49ef7fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a8b7db8390 0x55a8b819d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:06:34.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.210+0000 7f49ef7fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a8b7db8390 0x55a8b819d320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:56702/0 (socket says 172.21.15.42:56702) 2024-08-30T09:06:34.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.210+0000 7f49ef7fe640 1 -- 172.21.15.42:0/166406044 learned_addr learned my addr 172.21.15.42:0/166406044 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:06:34.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.210+0000 7f49f4c87640 1 --2- 172.21.15.42:0/166406044 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a8b7dd9070 0x55a8b819d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:06:34.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.210+0000 7f49ef7fe640 1 -- 172.21.15.42:0/166406044 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a8b7dd9070 msgr2=0x55a8b819d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:06:34.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.210+0000 7f49ef7fe640 1 --2- 172.21.15.42:0/166406044 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a8b7dd9070 0x55a8b819d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:34.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.210+0000 7f49ef7fe640 1 -- 172.21.15.42:0/166406044 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a8b7db01b0 msgr2=0x55a8b819cde0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-30T09:06:34.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.210+0000 7f49effff640 1 --2- 172.21.15.42:0/166406044 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a8b7db01b0 0x55a8b819cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:06:34.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.210+0000 7f49ef7fe640 1 --2- 172.21.15.42:0/166406044 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a8b7db01b0 0x55a8b819cde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:34.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.210+0000 7f49ef7fe640 1 -- 172.21.15.42:0/166406044 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a8b8205570 con 0x55a8b7db8390 2024-08-30T09:06:34.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.210+0000 7f49f4c87640 1 --2- 172.21.15.42:0/166406044 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a8b7dd9070 0x55a8b819d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-30T09:06:34.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.212+0000 7f49ef7fe640 1 --2- 172.21.15.42:0/166406044 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a8b7db8390 0x55a8b819d320 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f49e8009d80 tx=0x7f49e8004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d4261695509471d1 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:06:34.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.212+0000 7f49cf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-30T09:06:34.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.214+0000 7f49cf7fe640 1 -- 172.21.15.42:0/166406044 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49e800ed20 con 0x55a8b7db8390 2024-08-30T09:06:34.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.214+0000 7f49cf7fe640 1 -- 172.21.15.42:0/166406044 <== mon.2 v2:172.21.15.167:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f49e8003970 con 0x55a8b7db8390 2024-08-30T09:06:34.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.214+0000 7f49fcc5f740 1 -- 172.21.15.42:0/166406044 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f49e00abec0 con 0x55a8b7db8390 2024-08-30T09:06:34.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.214+0000 7f49cf7fe640 1 -- 172.21.15.42:0/166406044 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f49e8003b70 con 0x55a8b7db8390 2024-08-30T09:06:34.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.214+0000 7f49fcc5f740 10 client.26201 fetch_fsmap learned FSMap version 33 2024-08-30T09:06:34.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.214+0000 7f49fcc5f740 1 -- 172.21.15.42:0/166406044 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55a8b8206750 con 0x55a8b7db8390 2024-08-30T09:06:34.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.214+0000 7f49cf7fe640 1 -- 172.21.15.42:0/166406044 <== mon.2 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f49e00abec0 con 0x55a8b7db8390 2024-08-30T09:06:34.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.215+0000 7f49cf7fe640 1 -- 172.21.15.42:0/166406044 <== mon.2 v2:172.21.15.167:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f49e802f490 con 0x55a8b7db8390 2024-08-30T09:06:34.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.215+0000 7f49fcc5f740 10 client.26201 fetch_fsmap finished waiting for FSMap version 33 2024-08-30T09:06:34.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.215+0000 7f49fcc5f740 10 client.26201 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-30T09:06:34.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.215+0000 7f49fcc5f740 20 client.26201 populate_metadata read hostname 'smithi042' 2024-08-30T09:06:34.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.215+0000 7f49fcc5f740 1 --2- 172.21.15.42:0/166406044 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55a8b7d6cda0 0x55a8b8208f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:34.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.215+0000 7f49fcc5f740 4 client.26201 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-30T09:06:34.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.215+0000 7f49fcc5f740 1 -- 172.21.15.42:0/166406044 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a8b80f3070 con 0x55a8b7d6cda0 2024-08-30T09:06:34.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.215+0000 7f49effff640 1 --2- 172.21.15.42:0/166406044 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55a8b7d6cda0 0x55a8b8208f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:06:34.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.215+0000 7f49cf7fe640 10 client.26201 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:06:34.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.215+0000 7f49effff640 1 --2- 172.21.15.42:0/166406044 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55a8b7d6cda0 0x55a8b8208f40 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b737ba787a8d2ba server_cookie=75723816e5b9bc5d in_seq=0 out_seq=0 2024-08-30T09:06:34.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.216+0000 7f49cf7fe640 1 -- 172.21.15.42:0/166406044 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a8b80f3070 con 0x55a8b7d6cda0 2024-08-30T09:06:34.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.216+0000 7f49cf7fe640 10 client.26201 handle_command_reply: tid=0 2024-08-30T09:06:34.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.216+0000 7f49fcc5f740 1 client.26201 shutdown 2024-08-30T09:06:34.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.216+0000 7f49fcc5f740 1 -- 172.21.15.42:0/166406044 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a8b7db8390 msgr2=0x55a8b819d320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:06:34.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.216+0000 7f49fcc5f740 1 --2- 172.21.15.42:0/166406044 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a8b7db8390 0x55a8b819d320 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f49e8009d80 tx=0x7f49e8004a40 comp rx=0 tx=0).stop 2024-08-30T09:06:34.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.216+0000 7f49fcc5f740 1 -- 172.21.15.42:0/166406044 shutdown_connections 2024-08-30T09:06:34.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.216+0000 7f49fcc5f740 1 --2- 172.21.15.42:0/166406044 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55a8b7d6cda0 0x55a8b8208f40 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:34.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.216+0000 7f49fcc5f740 1 --2- 172.21.15.42:0/166406044 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a8b7dd9070 0x55a8b819d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:34.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.216+0000 7f49fcc5f740 1 --2- 172.21.15.42:0/166406044 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a8b7db8390 0x55a8b819d320 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:34.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.216+0000 7f49fcc5f740 1 --2- 172.21.15.42:0/166406044 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a8b7db01b0 0x55a8b819cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:34.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.216+0000 7f49fcc5f740 1 -- 172.21.15.42:0/166406044 >> 172.21.15.42:0/166406044 conn(0x55a8b7dcd460 msgr2=0x55a8b8181410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:06:34.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.216+0000 7f49cf7fe640 0 client.26201 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:06:34.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.216+0000 7f49fcc5f740 1 -- 172.21.15.42:0/166406044 shutdown_connections 2024-08-30T09:06:34.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.216+0000 7f49fcc5f740 1 -- 172.21.15.42:0/166406044 wait complete. 2024-08-30T09:06:34.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.216+0000 7f49fcc5f740 20 client.26201 trim_cache size 0 max 16384 2024-08-30T09:06:34.221 INFO:teuthology.orchestra.run.smithi042.stdout:[] 2024-08-30T09:06:34.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.217+0000 7f49f66ea640 1 -- 172.21.15.42:0/2444130898 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f49d80b4c60 msgr2=0x7f49d80b7170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:06:34.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.217+0000 7f49f66ea640 1 --2- 172.21.15.42:0/2444130898 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f49d80b4c60 0x7f49d80b7170 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f49f011bf50 tx=0x7f49e800fb20 comp rx=0 tx=0).stop 2024-08-30T09:06:34.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.217+0000 7f49f66ea640 1 -- 172.21.15.42:0/2444130898 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f49f011c9d0 msgr2=0x7f49f01ae8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:06:34.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.217+0000 7f49f66ea640 1 --2- 172.21.15.42:0/2444130898 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f49f011c9d0 0x7f49f01ae8c0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f49e000e9a0 tx=0x7f49e000ee60 comp rx=0 tx=0).stop 2024-08-30T09:06:34.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.217+0000 7f49f66ea640 1 -- 172.21.15.42:0/2444130898 shutdown_connections 2024-08-30T09:06:34.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.217+0000 7f49f66ea640 1 --2- 172.21.15.42:0/2444130898 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f49d80b4c60 0x7f49d80b7170 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:34.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.218+0000 7f49f66ea640 1 --2- 172.21.15.42:0/2444130898 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f49f011c9d0 0x7f49f01ae8c0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:34.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.218+0000 7f49f66ea640 1 --2- 172.21.15.42:0/2444130898 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f49f0111ce0 0x7f49f01a9620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:34.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.218+0000 7f49f66ea640 1 --2- 172.21.15.42:0/2444130898 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f49f010a780 0x7f49f01a9080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:34.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.218+0000 7f49f66ea640 1 -- 172.21.15.42:0/2444130898 >> 172.21.15.42:0/2444130898 conn(0x7f49f0070c60 msgr2=0x7f49f011cea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:06:34.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.218+0000 7f49f66ea640 1 -- 172.21.15.42:0/2444130898 shutdown_connections 2024-08-30T09:06:34.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:34.218+0000 7f49f66ea640 1 -- 172.21.15.42:0/2444130898 wait complete. 2024-08-30T09:06:34.572 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:06:34 smithi042 ceph-mon[18652]: pgmap v547: 129 pgs: 129 active+clean; 1.4 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 12 MiB/s wr, 647 op/s 2024-08-30T09:06:34.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:06:34 smithi190 ceph-mon[22001]: pgmap v547: 129 pgs: 129 active+clean; 1.4 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 12 MiB/s wr, 647 op/s 2024-08-30T09:06:34.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:06:34 smithi167 ceph-mon[22063]: pgmap v547: 129 pgs: 129 active+clean; 1.4 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 12 MiB/s wr, 647 op/s 2024-08-30T09:06:35.906 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-08-30T09:06:35.906 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-08-30T09:06:35.953 INFO:teuthology.orchestra.run.smithi042.stderr:progress: 350.0 s, 679.8 tps, lat 151.542 ms stddev 206.621 2024-08-30T09:06:36.394 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T09:06:36.628 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:06:36 smithi167 ceph-mon[22063]: pgmap v548: 129 pgs: 129 active+clean; 1.4 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 MiB/s wr, 528 op/s 2024-08-30T09:06:36.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:06:36 smithi042 ceph-mon[18652]: pgmap v548: 129 pgs: 129 active+clean; 1.4 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 MiB/s wr, 528 op/s 2024-08-30T09:06:36.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:06:36 smithi190 ceph-mon[22001]: pgmap v548: 129 pgs: 129 active+clean; 1.4 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 MiB/s wr, 528 op/s 2024-08-30T09:06:37.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.003+0000 7f95b27c6640 1 -- 172.21.15.42:0/1101988625 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f95ac077120 msgr2=0x7f95ac075550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:06:37.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.003+0000 7f95b27c6640 1 --2- 172.21.15.42:0/1101988625 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f95ac077120 0x7f95ac075550 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f95a4009be0 tx=0x7f95a402a710 comp rx=0 tx=0).stop 2024-08-30T09:06:37.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.006+0000 7f95b27c6640 1 -- 172.21.15.42:0/1101988625 shutdown_connections 2024-08-30T09:06:37.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.006+0000 7f95b27c6640 1 --2- 172.21.15.42:0/1101988625 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f95ac075b70 0x7f95ac075f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:37.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.006+0000 7f95b27c6640 1 --2- 172.21.15.42:0/1101988625 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f95ac1141f0 0x7f95ac1166c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:37.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.006+0000 7f95b27c6640 1 --2- 172.21.15.42:0/1101988625 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f95ac077120 0x7f95ac075550 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:37.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.006+0000 7f95b27c6640 1 -- 172.21.15.42:0/1101988625 >> 172.21.15.42:0/1101988625 conn(0x7f95ac0ff3b0 msgr2=0x7f95ac101800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:06:37.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.014+0000 7f95b27c6640 1 -- 172.21.15.42:0/1101988625 shutdown_connections 2024-08-30T09:06:37.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.014+0000 7f95b27c6640 1 -- 172.21.15.42:0/1101988625 wait complete. 2024-08-30T09:06:37.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.015+0000 7f95b27c6640 1 Processor -- start 2024-08-30T09:06:37.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.015+0000 7f95b27c6640 1 -- start start 2024-08-30T09:06:37.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.015+0000 7f95b27c6640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f95ac075b70 0x7f95ac113a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:37.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.015+0000 7f95b27c6640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f95ac077120 0x7f95ac10dd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:37.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.015+0000 7f95b27c6640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f95ac1141f0 0x7f95ac10e330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:37.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.015+0000 7f95b27c6640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f95ac1081a0 con 0x7f95ac1141f0 2024-08-30T09:06:37.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.016+0000 7f95abfff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f95ac075b70 0x7f95ac113a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:06:37.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.016+0000 7f95abfff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f95ac075b70 0x7f95ac113a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:33080/0 (socket says 172.21.15.42:33080) 2024-08-30T09:06:37.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.016+0000 7f95abfff640 1 -- 172.21.15.42:0/1153576684 learned_addr learned my addr 172.21.15.42:0/1153576684 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:06:37.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.016+0000 7f95ab7fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f95ac077120 0x7f95ac10dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:06:37.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.016+0000 7f95b27c6640 1 -- 172.21.15.42:0/1153576684 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f95ac108020 con 0x7f95ac077120 2024-08-30T09:06:37.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.016+0000 7f95b27c6640 1 -- 172.21.15.42:0/1153576684 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f95ac108320 con 0x7f95ac075b70 2024-08-30T09:06:37.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.016+0000 7f95b0d63640 1 --2- 172.21.15.42:0/1153576684 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f95ac1141f0 0x7f95ac10e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:06:37.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.017+0000 7f95b0d63640 1 -- 172.21.15.42:0/1153576684 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f95ac075b70 msgr2=0x7f95ac113a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:06:37.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.017+0000 7f95b0d63640 1 --2- 172.21.15.42:0/1153576684 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f95ac075b70 0x7f95ac113a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:37.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.017+0000 7f95b0d63640 1 -- 172.21.15.42:0/1153576684 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f95ac077120 msgr2=0x7f95ac10dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:06:37.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.017+0000 7f95b0d63640 1 --2- 172.21.15.42:0/1153576684 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f95ac077120 0x7f95ac10dd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:37.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.017+0000 7f95b0d63640 1 -- 172.21.15.42:0/1153576684 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95ac11f5f0 con 0x7f95ac1141f0 2024-08-30T09:06:37.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.017+0000 7f95b0d63640 1 --2- 172.21.15.42:0/1153576684 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f95ac1141f0 0x7f95ac10e330 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f95a0009fd0 tx=0x7f95a000eba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:06:37.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.018+0000 7f95a97fa640 1 -- 172.21.15.42:0/1153576684 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95a0004230 con 0x7f95ac1141f0 2024-08-30T09:06:37.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.018+0000 7f95a97fa640 1 -- 172.21.15.42:0/1153576684 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f95a00043d0 con 0x7f95ac1141f0 2024-08-30T09:06:37.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.018+0000 7f95b27c6640 1 -- 172.21.15.42:0/1153576684 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95ac11f880 con 0x7f95ac1141f0 2024-08-30T09:06:37.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.018+0000 7f95b27c6640 1 -- 172.21.15.42:0/1153576684 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95ac11fd70 con 0x7f95ac1141f0 2024-08-30T09:06:37.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.019+0000 7f95ab7fe640 1 --2- 172.21.15.42:0/1153576684 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f95ac077120 0x7f95ac10dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-30T09:06:37.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.019+0000 7f95abfff640 1 --2- 172.21.15.42:0/1153576684 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f95ac075b70 0x7f95ac113a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-30T09:06:37.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.021+0000 7f95a97fa640 1 -- 172.21.15.42:0/1153576684 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95a0005650 con 0x7f95ac1141f0 2024-08-30T09:06:37.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.022+0000 7f95b8d3b740 1 Processor -- start 2024-08-30T09:06:37.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.024+0000 7f95a97fa640 1 -- 172.21.15.42:0/1153576684 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7f95a000c8b0 con 0x7f95ac1141f0 2024-08-30T09:06:37.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.025+0000 7f95b8d3b740 1 -- start start 2024-08-30T09:06:37.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.026+0000 7f95b8d3b740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fec5abe350 0x55fec5e64780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:37.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.027+0000 7f95b8d3b740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fec5a95490 0x55fec5e64cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:37.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.027+0000 7f95b8d3b740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fec5a9d670 0x55fec5e65200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:37.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.027+0000 7f95b8d3b740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55fec5dda5f0 con 0x55fec5a9d670 2024-08-30T09:06:37.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.027+0000 7f95b8d3b740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55fec5dc9b80 con 0x55fec5abe350 2024-08-30T09:06:37.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.027+0000 7f95b8d3b740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55fec5dcdb30 con 0x55fec5a95490 2024-08-30T09:06:37.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.028+0000 7f95ab7fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fec5abe350 0x55fec5e64780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:06:37.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.028+0000 7f95a97fa640 1 --2- 172.21.15.42:0/1153576684 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f95940b4cc0 0x7f95940b71d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:37.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.028+0000 7f95ab7fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fec5abe350 0x55fec5e64780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:56730/0 (socket says 172.21.15.42:56730) 2024-08-30T09:06:37.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.028+0000 7f95ab7fe640 1 -- 172.21.15.42:0/2435601131 learned_addr learned my addr 172.21.15.42:0/2435601131 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:06:37.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.028+0000 7f95a97fa640 1 -- 172.21.15.42:0/1153576684 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f95a00e5530 con 0x7f95ac1141f0 2024-08-30T09:06:37.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.029+0000 7f95b0d63640 1 --2- 172.21.15.42:0/1153576684 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f95940b4cc0 0x7f95940b71d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:06:37.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.030+0000 7f95b0d63640 1 --2- 172.21.15.42:0/1153576684 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f95940b4cc0 0x7f95940b71d0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f95a00a8de0 tx=0x7f95a0004730 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:06:37.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.035+0000 7f95abfff640 1 --2- 172.21.15.42:0/2435601131 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fec5a95490 0x55fec5e64cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:06:37.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.038+0000 7f95ab7fe640 1 --2- 172.21.15.42:0/2435601131 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fec5a9d670 0x55fec5e65200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:06:37.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.038+0000 7f95ab7fe640 1 -- 172.21.15.42:0/2435601131 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fec5a95490 msgr2=0x55fec5e64cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:06:37.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.038+0000 7f95ab7fe640 1 --2- 172.21.15.42:0/2435601131 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fec5a95490 0x55fec5e64cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:37.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.038+0000 7f95ab7fe640 1 -- 172.21.15.42:0/2435601131 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fec5a9d670 msgr2=0x55fec5e65200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:06:37.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.038+0000 7f95ab7fe640 1 --2- 172.21.15.42:0/2435601131 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fec5a9d670 0x55fec5e65200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:37.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.038+0000 7f95ab7fe640 1 -- 172.21.15.42:0/2435601131 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fec5b29930 con 0x55fec5abe350 2024-08-30T09:06:37.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.043+0000 7f95ab7fe640 1 --2- 172.21.15.42:0/2435601131 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fec5abe350 0x55fec5e64780 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f959c0058c0 tx=0x7f959c0096e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dbdb80fe6078f713 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:06:37.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.045+0000 7f95927fc640 1 -- 172.21.15.42:0/2435601131 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f959c004030 con 0x55fec5abe350 2024-08-30T09:06:37.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.045+0000 7f95927fc640 1 -- 172.21.15.42:0/2435601131 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f959c012c40 con 0x55fec5abe350 2024-08-30T09:06:37.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.047+0000 7f95b8d3b740 1 -- 172.21.15.42:0/2435601131 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fec5abe350 msgr2=0x55fec5e64780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:06:37.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.047+0000 7f95b8d3b740 1 --2- 172.21.15.42:0/2435601131 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fec5abe350 0x55fec5e64780 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f959c0058c0 tx=0x7f959c0096e0 comp rx=0 tx=0).stop 2024-08-30T09:06:37.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.047+0000 7f95b8d3b740 1 -- 172.21.15.42:0/2435601131 shutdown_connections 2024-08-30T09:06:37.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.047+0000 7f95b8d3b740 1 --2- 172.21.15.42:0/2435601131 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fec5a95490 0x55fec5e64cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:37.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.047+0000 7f95b8d3b740 1 --2- 172.21.15.42:0/2435601131 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fec5a9d670 0x55fec5e65200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:37.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.047+0000 7f95b8d3b740 1 --2- 172.21.15.42:0/2435601131 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fec5abe350 0x55fec5e64780 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:37.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.047+0000 7f95b8d3b740 1 -- 172.21.15.42:0/2435601131 >> 172.21.15.42:0/2435601131 conn(0x55fec5ab2740 msgr2=0x55fec5e62670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:06:37.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.049+0000 7f95b8d3b740 1 -- 172.21.15.42:0/2435601131 shutdown_connections 2024-08-30T09:06:37.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.050+0000 7f95b8d3b740 1 -- 172.21.15.42:0/2435601131 wait complete. 2024-08-30T09:06:37.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.051+0000 7f95b8d3b740 1 -- start start 2024-08-30T09:06:37.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.051+0000 7f95b8d3b740 1 Processor -- start 2024-08-30T09:06:37.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.052+0000 7f95b8d3b740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fec5a95490 0x55fec5e6fe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:37.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.052+0000 7f95b8d3b740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fec5a9d670 0x55fec5e70360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:37.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.052+0000 7f95b8d3b740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fec5abe350 0x55fec5e7c380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:37.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.052+0000 7f95b8d3b740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55fec5c2c890 con 0x55fec5abe350 2024-08-30T09:06:37.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.052+0000 7f95b8d3b740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55fec5d50790 con 0x55fec5a95490 2024-08-30T09:06:37.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.052+0000 7f95b8d3b740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55fec5e72fb0 con 0x55fec5a9d670 2024-08-30T09:06:37.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.052+0000 7f95b8d3b740 10 client.0 _ll_register_callbacks cb 0x55fec5ad5ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:06:37.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.052+0000 7f95abfff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fec5a9d670 0x55fec5e70360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:06:37.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.052+0000 7f95abfff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fec5a9d670 0x55fec5e70360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:33100/0 (socket says 172.21.15.42:33100) 2024-08-30T09:06:37.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.052+0000 7f95abfff640 1 -- 172.21.15.42:0/1657404500 learned_addr learned my addr 172.21.15.42:0/1657404500 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:06:37.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.052+0000 7f95ab7fe640 1 --2- 172.21.15.42:0/1657404500 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fec5abe350 0x55fec5e7c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:06:37.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.052+0000 7f95ab7fe640 1 --2- 172.21.15.42:0/1657404500 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fec5a95490 0x55fec5e6fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:06:37.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.053+0000 7f95ab7fe640 1 -- 172.21.15.42:0/1657404500 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fec5a9d670 msgr2=0x55fec5e70360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:06:37.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.053+0000 7f95ab7fe640 1 --2- 172.21.15.42:0/1657404500 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fec5a9d670 0x55fec5e70360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:37.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.053+0000 7f95ab7fe640 1 -- 172.21.15.42:0/1657404500 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fec5a95490 msgr2=0x55fec5e6fe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:06:37.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.053+0000 7f95ab7fe640 1 --2- 172.21.15.42:0/1657404500 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fec5a95490 0x55fec5e6fe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:37.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.053+0000 7f95ab7fe640 1 -- 172.21.15.42:0/1657404500 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fec5d87f30 con 0x55fec5abe350 2024-08-30T09:06:37.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.053+0000 7f95ab7fe640 1 --2- 172.21.15.42:0/1657404500 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fec5abe350 0x55fec5e7c380 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f959c0146d0 tx=0x7f959c016040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fce16fe0756f149 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:06:37.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.057+0000 7f958b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-08-30T09:06:37.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.057+0000 7f958b7fe640 1 -- 172.21.15.42:0/1657404500 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f959c005020 con 0x55fec5abe350 2024-08-30T09:06:37.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.057+0000 7f958b7fe640 1 -- 172.21.15.42:0/1657404500 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f959c0051c0 con 0x55fec5abe350 2024-08-30T09:06:37.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.057+0000 7f95b8d3b740 1 -- 172.21.15.42:0/1657404500 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f959c012c40 con 0x55fec5abe350 2024-08-30T09:06:37.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.057+0000 7f958b7fe640 1 -- 172.21.15.42:0/1657404500 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f959c0053c0 con 0x55fec5abe350 2024-08-30T09:06:37.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.057+0000 7f95b8d3b740 10 client.16464 fetch_fsmap learned FSMap version 33 2024-08-30T09:06:37.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.057+0000 7f95b8d3b740 1 -- 172.21.15.42:0/1657404500 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55fec5e7ed80 con 0x55fec5abe350 2024-08-30T09:06:37.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.058+0000 7f958b7fe640 1 -- 172.21.15.42:0/1657404500 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f959c012c40 con 0x55fec5abe350 2024-08-30T09:06:37.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.058+0000 7f958b7fe640 1 -- 172.21.15.42:0/1657404500 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f959c003c80 con 0x55fec5abe350 2024-08-30T09:06:37.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.058+0000 7f95b8d3b740 10 client.16464 fetch_fsmap finished waiting for FSMap version 33 2024-08-30T09:06:37.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.058+0000 7f95b8d3b740 10 client.16464 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-30T09:06:37.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.058+0000 7f95b8d3b740 20 client.16464 populate_metadata read hostname 'smithi042' 2024-08-30T09:06:37.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.058+0000 7f95b8d3b740 1 --2- 172.21.15.42:0/1657404500 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55fec5a7ccd0 0x55fec5e823a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:37.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.058+0000 7f95b8d3b740 4 client.16464 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-30T09:06:37.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.058+0000 7f95b8d3b740 1 -- 172.21.15.42:0/1657404500 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fec5ddc540 con 0x55fec5a7ccd0 2024-08-30T09:06:37.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.059+0000 7f95abfff640 1 --2- 172.21.15.42:0/1657404500 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55fec5a7ccd0 0x55fec5e823a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:06:37.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.061+0000 7f958b7fe640 10 client.16464 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:06:37.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.062+0000 7f95abfff640 1 --2- 172.21.15.42:0/1657404500 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55fec5a7ccd0 0x55fec5e823a0 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f5cef4c97519ad05 server_cookie=b8e835ac469952a in_seq=0 out_seq=0 2024-08-30T09:06:37.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.066+0000 7f958b7fe640 1 -- 172.21.15.42:0/1657404500 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55fec5ddc540 con 0x55fec5a7ccd0 2024-08-30T09:06:37.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.066+0000 7f958b7fe640 10 client.16464 handle_command_reply: tid=0 2024-08-30T09:06:37.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.066+0000 7f95b8d3b740 1 client.16464 shutdown 2024-08-30T09:06:37.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.069+0000 7f95b8d3b740 1 -- 172.21.15.42:0/1657404500 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fec5abe350 msgr2=0x55fec5e7c380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:06:37.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.069+0000 7f95b8d3b740 1 --2- 172.21.15.42:0/1657404500 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fec5abe350 0x55fec5e7c380 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f959c0146d0 tx=0x7f959c016040 comp rx=0 tx=0).stop 2024-08-30T09:06:37.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.074+0000 7f95b8d3b740 1 -- 172.21.15.42:0/1657404500 shutdown_connections 2024-08-30T09:06:37.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.074+0000 7f95b8d3b740 1 --2- 172.21.15.42:0/1657404500 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55fec5a7ccd0 0x55fec5e823a0 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:37.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.074+0000 7f95b8d3b740 1 --2- 172.21.15.42:0/1657404500 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fec5a9d670 0x55fec5e70360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:37.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.074+0000 7f95b8d3b740 1 --2- 172.21.15.42:0/1657404500 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fec5abe350 0x55fec5e7c380 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:37.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.074+0000 7f95b8d3b740 1 --2- 172.21.15.42:0/1657404500 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fec5a95490 0x55fec5e6fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:37.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.074+0000 7f95b8d3b740 1 -- 172.21.15.42:0/1657404500 >> 172.21.15.42:0/1657404500 conn(0x55fec5ab2740 msgr2=0x55fec5e67910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:06:37.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.075+0000 7f958b7fe640 0 client.16464 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:06:37.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.079+0000 7f95b8d3b740 1 -- 172.21.15.42:0/1657404500 shutdown_connections 2024-08-30T09:06:37.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.080+0000 7f95b8d3b740 1 -- 172.21.15.42:0/1657404500 wait complete. 2024-08-30T09:06:37.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.080+0000 7f95b8d3b740 20 client.16464 trim_cache size 0 max 16384 2024-08-30T09:06:37.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.122+0000 7f95b8d3b740 1 Processor -- start 2024-08-30T09:06:37.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.122+0000 7f95b8d3b740 1 -- start start 2024-08-30T09:06:37.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.122+0000 7f95b8d3b740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fec5a95490 0x55fec5e7c280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:37.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.122+0000 7f95b8d3b740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fec5a9d670 0x55fec5e6d010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:37.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.122+0000 7f95b8d3b740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fec5abe350 0x55fec5e6d550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:37.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.122+0000 7f95b8d3b740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f95980125f0 con 0x55fec5a9d670 2024-08-30T09:06:37.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.122+0000 7f95b8d3b740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f959800fe60 con 0x55fec5a95490 2024-08-30T09:06:37.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.122+0000 7f95b8d3b740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f959800dcb0 con 0x55fec5abe350 2024-08-30T09:06:37.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.123+0000 7f95ab7fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fec5a95490 0x55fec5e7c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:06:37.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.124+0000 7f95ab7fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fec5abe350 0x55fec5e6d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:06:37.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.124+0000 7f95ab7fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fec5a95490 0x55fec5e7c280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:56746/0 (socket says 172.21.15.42:56746) 2024-08-30T09:06:37.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.124+0000 7f95ab7fe640 1 -- 172.21.15.42:0/3594768937 learned_addr learned my addr 172.21.15.42:0/3594768937 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:06:37.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.124+0000 7f95abfff640 1 --2- 172.21.15.42:0/3594768937 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fec5a9d670 0x55fec5e6d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:06:37.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.124+0000 7f95ab7fe640 1 -- 172.21.15.42:0/3594768937 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fec5abe350 msgr2=0x55fec5e6d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:06:37.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.124+0000 7f95ab7fe640 1 --2- 172.21.15.42:0/3594768937 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fec5abe350 0x55fec5e6d550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:37.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.124+0000 7f95ab7fe640 1 -- 172.21.15.42:0/3594768937 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fec5a9d670 msgr2=0x55fec5e6d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:06:37.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.124+0000 7f95ab7fe640 1 --2- 172.21.15.42:0/3594768937 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fec5a9d670 0x55fec5e6d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:37.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.124+0000 7f95ab7fe640 1 -- 172.21.15.42:0/3594768937 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fec5e6b6b0 con 0x55fec5a95490 2024-08-30T09:06:37.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.125+0000 7f95ab7fe640 1 --2- 172.21.15.42:0/3594768937 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fec5a95490 0x55fec5e7c280 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f959c006010 tx=0x7f959c0099a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7673722950cbfe3e server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:06:37.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.128+0000 7f9591ffb640 1 -- 172.21.15.42:0/3594768937 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f959c024040 con 0x55fec5a95490 2024-08-30T09:06:37.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.128+0000 7f9591ffb640 1 -- 172.21.15.42:0/3594768937 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f959c012c40 con 0x55fec5a95490 2024-08-30T09:06:37.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.129+0000 7f95b8d3b740 1 -- 172.21.15.42:0/3594768937 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fec5a95490 msgr2=0x55fec5e7c280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:06:37.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.129+0000 7f95b8d3b740 1 --2- 172.21.15.42:0/3594768937 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fec5a95490 0x55fec5e7c280 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f959c006010 tx=0x7f959c0099a0 comp rx=0 tx=0).stop 2024-08-30T09:06:37.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.130+0000 7f95b8d3b740 1 -- 172.21.15.42:0/3594768937 shutdown_connections 2024-08-30T09:06:37.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.130+0000 7f95b8d3b740 1 --2- 172.21.15.42:0/3594768937 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fec5abe350 0x55fec5e6d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:37.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.130+0000 7f95b8d3b740 1 --2- 172.21.15.42:0/3594768937 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fec5a9d670 0x55fec5e6d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:37.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.130+0000 7f95b8d3b740 1 --2- 172.21.15.42:0/3594768937 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fec5a95490 0x55fec5e7c280 secure :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f959c006010 tx=0x7f959c0099a0 comp rx=0 tx=0).stop 2024-08-30T09:06:37.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.130+0000 7f95b8d3b740 1 -- 172.21.15.42:0/3594768937 >> 172.21.15.42:0/3594768937 conn(0x55fec5ab2740 msgr2=0x55fec5e6e6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:06:37.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.134+0000 7f95b8d3b740 1 -- 172.21.15.42:0/3594768937 shutdown_connections 2024-08-30T09:06:37.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.134+0000 7f95b8d3b740 1 -- 172.21.15.42:0/3594768937 wait complete. 2024-08-30T09:06:37.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.135+0000 7f95b8d3b740 1 -- start start 2024-08-30T09:06:37.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.135+0000 7f95b8d3b740 1 Processor -- start 2024-08-30T09:06:37.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.136+0000 7f95b8d3b740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fec5a9d670 0x55fec5e78cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:37.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.136+0000 7f95b8d3b740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fec5abe350 0x55fec5e741b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:37.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.136+0000 7f95b8d3b740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fec5a52080 0x55fec5e748c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:37.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.136+0000 7f95b8d3b740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f959800e320 con 0x55fec5a9d670 2024-08-30T09:06:37.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.136+0000 7f95b8d3b740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f959800f790 con 0x55fec5abe350 2024-08-30T09:06:37.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.136+0000 7f95b8d3b740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f959800e9f0 con 0x55fec5a52080 2024-08-30T09:06:37.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.136+0000 7f95b8d3b740 10 client.0 _ll_register_callbacks cb 0x7f959c009d90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:06:37.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.136+0000 7f95ab7fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fec5a9d670 0x55fec5e78cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:06:37.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.136+0000 7f95ab7fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fec5a9d670 0x55fec5e78cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:45750/0 (socket says 172.21.15.42:45750) 2024-08-30T09:06:37.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.136+0000 7f95abfff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fec5abe350 0x55fec5e741b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:06:37.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.136+0000 7f95ab7fe640 1 -- 172.21.15.42:0/1174509404 learned_addr learned my addr 172.21.15.42:0/1174509404 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:06:37.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.137+0000 7f95ab7fe640 1 --2- 172.21.15.42:0/1174509404 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fec5a52080 0x55fec5e748c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:06:37.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.137+0000 7f95ab7fe640 1 -- 172.21.15.42:0/1174509404 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fec5abe350 msgr2=0x55fec5e741b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:06:37.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.137+0000 7f95ab7fe640 1 --2- 172.21.15.42:0/1174509404 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fec5abe350 0x55fec5e741b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:37.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.137+0000 7f95ab7fe640 1 -- 172.21.15.42:0/1174509404 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fec5a9d670 msgr2=0x55fec5e78cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:06:37.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.137+0000 7f95ab7fe640 1 --2- 172.21.15.42:0/1174509404 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fec5a9d670 0x55fec5e78cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:37.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.137+0000 7f95ab7fe640 1 -- 172.21.15.42:0/1174509404 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fec5eebda0 con 0x55fec5a52080 2024-08-30T09:06:37.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.137+0000 7f95ab7fe640 1 --2- 172.21.15.42:0/1174509404 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fec5a52080 0x55fec5e748c0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x55fec5e7d420 tx=0x7f959c000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=399343a680307ffc server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:06:37.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.137+0000 7f958b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-08-30T09:06:37.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.138+0000 7f958b7fe640 1 -- 172.21.15.42:0/1174509404 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f959c024040 con 0x55fec5a52080 2024-08-30T09:06:37.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.138+0000 7f958b7fe640 1 -- 172.21.15.42:0/1174509404 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f959c012be0 con 0x55fec5a52080 2024-08-30T09:06:37.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.138+0000 7f95b8d3b740 1 -- 172.21.15.42:0/1174509404 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55fec5dd7fe0 con 0x55fec5a52080 2024-08-30T09:06:37.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.140+0000 7f958b7fe640 1 -- 172.21.15.42:0/1174509404 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f959c005c50 con 0x55fec5a52080 2024-08-30T09:06:37.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.140+0000 7f95b8d3b740 10 client.26200 fetch_fsmap learned FSMap version 33 2024-08-30T09:06:37.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.140+0000 7f95b8d3b740 1 -- 172.21.15.42:0/1174509404 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55fec5eecfc0 con 0x55fec5a52080 2024-08-30T09:06:37.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.141+0000 7f958b7fe640 1 -- 172.21.15.42:0/1174509404 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55fec5dd7fe0 con 0x55fec5a52080 2024-08-30T09:06:37.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.141+0000 7f958b7fe640 1 -- 172.21.15.42:0/1174509404 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f959c0206a0 con 0x55fec5a52080 2024-08-30T09:06:37.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.141+0000 7f95b8d3b740 10 client.26200 fetch_fsmap finished waiting for FSMap version 33 2024-08-30T09:06:37.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.141+0000 7f95b8d3b740 10 client.26200 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-30T09:06:37.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.141+0000 7f95b8d3b740 20 client.26200 populate_metadata read hostname 'smithi042' 2024-08-30T09:06:37.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.142+0000 7f95b8d3b740 1 --2- 172.21.15.42:0/1174509404 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55fec5eed350 0x55fec5eef7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:37.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.142+0000 7f95b8d3b740 4 client.26200 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-08-30T09:06:37.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.142+0000 7f95b8d3b740 1 -- 172.21.15.42:0/1174509404 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55fec5dd5720 con 0x55fec5eed350 2024-08-30T09:06:37.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.142+0000 7f95abfff640 1 --2- 172.21.15.42:0/1174509404 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55fec5eed350 0x55fec5eef7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:06:37.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.144+0000 7f95abfff640 1 --2- 172.21.15.42:0/1174509404 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55fec5eed350 0x55fec5eef7a0 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9cd3f718ccb9a4a3 server_cookie=dad6f3fb40ffaa2a in_seq=0 out_seq=0 2024-08-30T09:06:37.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.144+0000 7f958b7fe640 10 client.26200 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:06:37.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.146+0000 7f958b7fe640 1 -- 172.21.15.42:0/1174509404 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55fec5dd5720 con 0x55fec5eed350 2024-08-30T09:06:37.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.146+0000 7f958b7fe640 10 client.26200 handle_command_reply: tid=0 2024-08-30T09:06:37.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.147+0000 7f95b8d3b740 1 client.26200 shutdown 2024-08-30T09:06:37.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.147+0000 7f95b8d3b740 1 -- 172.21.15.42:0/1174509404 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fec5a52080 msgr2=0x55fec5e748c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:06:37.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.147+0000 7f95b8d3b740 1 --2- 172.21.15.42:0/1174509404 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fec5a52080 0x55fec5e748c0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x55fec5e7d420 tx=0x7f959c000bc0 comp rx=0 tx=0).stop 2024-08-30T09:06:37.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.147+0000 7f95b8d3b740 1 -- 172.21.15.42:0/1174509404 shutdown_connections 2024-08-30T09:06:37.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.147+0000 7f95b8d3b740 1 --2- 172.21.15.42:0/1174509404 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55fec5eed350 0x55fec5eef7a0 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:37.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.147+0000 7f95b8d3b740 1 --2- 172.21.15.42:0/1174509404 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fec5a52080 0x55fec5e748c0 secure :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0x55fec5e7d420 tx=0x7f959c000bc0 comp rx=0 tx=0).stop 2024-08-30T09:06:37.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.147+0000 7f95b8d3b740 1 --2- 172.21.15.42:0/1174509404 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fec5abe350 0x55fec5e741b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:37.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.147+0000 7f95b8d3b740 1 --2- 172.21.15.42:0/1174509404 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fec5a9d670 0x55fec5e78cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:37.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.147+0000 7f95b8d3b740 1 -- 172.21.15.42:0/1174509404 >> 172.21.15.42:0/1174509404 conn(0x55fec5ab2740 msgr2=0x55fec5e657b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:06:37.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.148+0000 7f958b7fe640 0 client.26200 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:06:37.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.148+0000 7f95b8d3b740 1 -- 172.21.15.42:0/1174509404 shutdown_connections 2024-08-30T09:06:37.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.148+0000 7f95b8d3b740 1 -- 172.21.15.42:0/1174509404 wait complete. 2024-08-30T09:06:37.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.148+0000 7f95b8d3b740 20 client.26200 trim_cache size 0 max 16384 2024-08-30T09:06:37.151 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-08-30T09:06:37.151 INFO:teuthology.orchestra.run.smithi042.stdout: "return_code": 0, 2024-08-30T09:06:37.151 INFO:teuthology.orchestra.run.smithi042.stdout: "scrub_tag": "665ec740-7254-49cb-84ef-deff71c461b0", 2024-08-30T09:06:37.151 INFO:teuthology.orchestra.run.smithi042.stdout: "mode": "asynchronous" 2024-08-30T09:06:37.152 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-08-30T09:06:37.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.149+0000 7f95b27c6640 1 -- 172.21.15.42:0/1153576684 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f95940b4cc0 msgr2=0x7f95940b71d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:06:37.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.149+0000 7f95b27c6640 1 --2- 172.21.15.42:0/1153576684 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f95940b4cc0 0x7f95940b71d0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f95a00a8de0 tx=0x7f95a0004730 comp rx=0 tx=0).stop 2024-08-30T09:06:37.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.149+0000 7f95b27c6640 1 -- 172.21.15.42:0/1153576684 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f95ac1141f0 msgr2=0x7f95ac10e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:06:37.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.149+0000 7f95b27c6640 1 --2- 172.21.15.42:0/1153576684 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f95ac1141f0 0x7f95ac10e330 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f95a0009fd0 tx=0x7f95a000eba0 comp rx=0 tx=0).stop 2024-08-30T09:06:37.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.149+0000 7f95b27c6640 1 -- 172.21.15.42:0/1153576684 shutdown_connections 2024-08-30T09:06:37.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.149+0000 7f95b27c6640 1 --2- 172.21.15.42:0/1153576684 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f95940b4cc0 0x7f95940b71d0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:37.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.149+0000 7f95b27c6640 1 --2- 172.21.15.42:0/1153576684 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f95ac1141f0 0x7f95ac10e330 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:37.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.149+0000 7f95b27c6640 1 --2- 172.21.15.42:0/1153576684 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f95ac077120 0x7f95ac10dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:37.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.149+0000 7f95b27c6640 1 --2- 172.21.15.42:0/1153576684 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f95ac075b70 0x7f95ac113a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:37.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.149+0000 7f95b27c6640 1 -- 172.21.15.42:0/1153576684 >> 172.21.15.42:0/1153576684 conn(0x7f95ac0ff3b0 msgr2=0x7f95ac103350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:06:37.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.150+0000 7f95b27c6640 1 -- 172.21.15.42:0/1153576684 shutdown_connections 2024-08-30T09:06:37.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:37.150+0000 7f95b27c6640 1 -- 172.21.15.42:0/1153576684 wait complete. 2024-08-30T09:06:37.577 INFO:journalctl@ceph.mds.b.smithi167.stdout:Aug 30 09:06:37 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mds-b[52473]: 2024-08-30T09:06:37.175+0000 7f9d3fee4640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000000f [...2,head] /volumes/qa/sv_1/d3168875-d2e5-410d-8754-72d29955777b/postgres/data/pg_subtrans/ auth{1=1} v1981 ap=2 f(v7 m2024-08-30T09:05:39.951616+0000 8=8+0) n(v20 rc2024-08-30T09:05:39.955616+0000 b2039808 9=8+1) (inest sync->mix g=1) caps={15402=pAsLsXsFs/pAsLsXsFsxcral@20} mcw={1=AsLsXsFs} | dirtyscattered=0 request=0 lock=0 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x555a32f6d080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x1000000000f:[<0x10000000006/pg_subtrans v284>,<0x10000000005/data v125>,<0x100000001fb/postgres v31>,<0x100000001fa/d3168875-d2e5-410d-8754-72d29955777b v45>,<0x10000000001/sv_1 v63>,<0x10000000000/qa v49>,<0x1/volumes v44>]//[]","memoryvalue":"(2)0x1000000000f:[<0x10000000006/pg_subtrans v1981>,<0x10000000005/data v540>,<0x100000001fb/postgres v294>,<0x100000001fa/d3168875-d2e5-410d-8754-72d29955777b v309>,<0x10000000001/sv_1 v325>,<0x10000000000/qa v307>,<0x1/volumes v273>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-08-30T09:04:42.547112+0000 7=7+0)","ondisk_value.rstat":"n(v0 rc2024-08-30T09:05:39.955616+0000 b2039808 9=8+1)","memory_value.dirstat":"f(v7 m2024-08-30T09:05:39.951616+0000 8=8+0)","memory_value.rstat":"n(v20 rc2024-08-30T09:05:39.955616+0000 b2039808 9=8+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-08-30T09:06:38.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:06:38 smithi042 ceph-mon[18652]: pgmap v549: 129 pgs: 129 active+clean; 1.4 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 MiB/s wr, 606 op/s 2024-08-30T09:06:38.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:06:38 smithi042 ceph-mon[18652]: scrub queued for path: / 2024-08-30T09:06:38.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:06:38 smithi042 ceph-mon[18652]: scrub summary: idle+waiting paths [/] 2024-08-30T09:06:38.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:06:38 smithi042 ceph-mon[18652]: scrub summary: active paths [/] 2024-08-30T09:06:38.634 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:06:38 smithi042 ceph-mon[18652]: scrub summary: active paths [/] 2024-08-30T09:06:38.634 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:06:38 smithi042 ceph-mon[18652]: scrub summary: idle+waiting paths [/] 2024-08-30T09:06:38.634 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:06:38 smithi042 ceph-mon[18652]: scrub summary: active paths [/] 2024-08-30T09:06:38.634 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:06:38 smithi042 ceph-mon[18652]: scrub summary: active paths [/] 2024-08-30T09:06:38.634 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:06:38 smithi042 ceph-mon[18652]: scrub summary: idle+waiting paths [/] 2024-08-30T09:06:38.634 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:06:38 smithi042 ceph-mon[18652]: scrub summary: idle+waiting paths [/] 2024-08-30T09:06:38.634 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:06:38 smithi042 ceph-mon[18652]: scrub summary: active paths [/] 2024-08-30T09:06:38.634 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:06:38 smithi042 ceph-mon[18652]: scrub summary: active paths [/] 2024-08-30T09:06:38.634 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:06:38 smithi042 ceph-mon[18652]: scrub summary: idle+waiting paths [/] 2024-08-30T09:06:38.634 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:06:38 smithi042 ceph-mon[18652]: scrub summary: active paths [/] 2024-08-30T09:06:38.634 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:06:38 smithi042 ceph-mon[18652]: Scrub error on inode 0x1000000000f (/volumes/qa/sv_1/d3168875-d2e5-410d-8754-72d29955777b/postgres/data/pg_subtrans) see mds.b log and `damage ls` output for details 2024-08-30T09:06:38.634 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:06:38 smithi042 ceph-mon[18652]: scrub summary: idle+waiting paths [/] 2024-08-30T09:06:38.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:06:38 smithi190 ceph-mon[22001]: pgmap v549: 129 pgs: 129 active+clean; 1.4 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 MiB/s wr, 606 op/s 2024-08-30T09:06:38.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:06:38 smithi190 ceph-mon[22001]: scrub queued for path: / 2024-08-30T09:06:38.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:06:38 smithi190 ceph-mon[22001]: scrub summary: idle+waiting paths [/] 2024-08-30T09:06:38.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:06:38 smithi190 ceph-mon[22001]: scrub summary: active paths [/] 2024-08-30T09:06:38.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:06:38 smithi190 ceph-mon[22001]: scrub summary: active paths [/] 2024-08-30T09:06:38.687 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:06:38 smithi190 ceph-mon[22001]: scrub summary: idle+waiting paths [/] 2024-08-30T09:06:38.687 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:06:38 smithi190 ceph-mon[22001]: scrub summary: active paths [/] 2024-08-30T09:06:38.687 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:06:38 smithi190 ceph-mon[22001]: scrub summary: active paths [/] 2024-08-30T09:06:38.687 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:06:38 smithi190 ceph-mon[22001]: scrub summary: idle+waiting paths [/] 2024-08-30T09:06:38.687 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:06:38 smithi190 ceph-mon[22001]: scrub summary: idle+waiting paths [/] 2024-08-30T09:06:38.687 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:06:38 smithi190 ceph-mon[22001]: scrub summary: active paths [/] 2024-08-30T09:06:38.687 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:06:38 smithi190 ceph-mon[22001]: scrub summary: active paths [/] 2024-08-30T09:06:38.687 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:06:38 smithi190 ceph-mon[22001]: scrub summary: idle+waiting paths [/] 2024-08-30T09:06:38.687 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:06:38 smithi190 ceph-mon[22001]: scrub summary: active paths [/] 2024-08-30T09:06:38.687 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:06:38 smithi190 ceph-mon[22001]: Scrub error on inode 0x1000000000f (/volumes/qa/sv_1/d3168875-d2e5-410d-8754-72d29955777b/postgres/data/pg_subtrans) see mds.b log and `damage ls` output for details 2024-08-30T09:06:38.687 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:06:38 smithi190 ceph-mon[22001]: scrub summary: idle+waiting paths [/] 2024-08-30T09:06:38.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:06:38 smithi167 ceph-mon[22063]: pgmap v549: 129 pgs: 129 active+clean; 1.4 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 MiB/s wr, 606 op/s 2024-08-30T09:06:38.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:06:38 smithi167 ceph-mon[22063]: scrub queued for path: / 2024-08-30T09:06:38.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:06:38 smithi167 ceph-mon[22063]: scrub summary: idle+waiting paths [/] 2024-08-30T09:06:38.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:06:38 smithi167 ceph-mon[22063]: scrub summary: active paths [/] 2024-08-30T09:06:38.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:06:38 smithi167 ceph-mon[22063]: scrub summary: active paths [/] 2024-08-30T09:06:38.828 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:06:38 smithi167 ceph-mon[22063]: scrub summary: idle+waiting paths [/] 2024-08-30T09:06:38.828 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:06:38 smithi167 ceph-mon[22063]: scrub summary: active paths [/] 2024-08-30T09:06:38.828 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:06:38 smithi167 ceph-mon[22063]: scrub summary: active paths [/] 2024-08-30T09:06:38.828 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:06:38 smithi167 ceph-mon[22063]: scrub summary: idle+waiting paths [/] 2024-08-30T09:06:38.828 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:06:38 smithi167 ceph-mon[22063]: scrub summary: idle+waiting paths [/] 2024-08-30T09:06:38.828 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:06:38 smithi167 ceph-mon[22063]: scrub summary: active paths [/] 2024-08-30T09:06:38.828 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:06:38 smithi167 ceph-mon[22063]: scrub summary: active paths [/] 2024-08-30T09:06:38.828 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:06:38 smithi167 ceph-mon[22063]: scrub summary: idle+waiting paths [/] 2024-08-30T09:06:38.828 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:06:38 smithi167 ceph-mon[22063]: scrub summary: active paths [/] 2024-08-30T09:06:38.828 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:06:38 smithi167 ceph-mon[22063]: Scrub error on inode 0x1000000000f (/volumes/qa/sv_1/d3168875-d2e5-410d-8754-72d29955777b/postgres/data/pg_subtrans) see mds.b log and `damage ls` output for details 2024-08-30T09:06:38.828 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:06:38 smithi167 ceph-mon[22063]: scrub summary: idle+waiting paths [/] 2024-08-30T09:06:39.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:06:39 smithi042 ceph-mon[18652]: scrub summary: idle+waiting paths [/] 2024-08-30T09:06:39.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:06:39 smithi042 ceph-mon[18652]: scrub summary: idle+waiting paths [/] 2024-08-30T09:06:39.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:06:39 smithi190 ceph-mon[22001]: scrub summary: idle+waiting paths [/] 2024-08-30T09:06:39.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:06:39 smithi190 ceph-mon[22001]: scrub summary: idle+waiting paths [/] 2024-08-30T09:06:39.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:06:39 smithi167 ceph-mon[22063]: scrub summary: idle+waiting paths [/] 2024-08-30T09:06:39.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:06:39 smithi167 ceph-mon[22063]: scrub summary: idle+waiting paths [/] 2024-08-30T09:06:40.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:06:40 smithi042 ceph-mon[18652]: pgmap v550: 129 pgs: 129 active+clean; 1.4 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 12 MiB/s wr, 621 op/s 2024-08-30T09:06:40.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:06:40 smithi190 ceph-mon[22001]: pgmap v550: 129 pgs: 129 active+clean; 1.4 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 12 MiB/s wr, 621 op/s 2024-08-30T09:06:40.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:06:40 smithi167 ceph-mon[22063]: pgmap v550: 129 pgs: 129 active+clean; 1.4 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 12 MiB/s wr, 621 op/s 2024-08-30T09:06:40.953 INFO:teuthology.orchestra.run.smithi042.stderr:progress: 355.0 s, 650.8 tps, lat 153.750 ms stddev 211.810 2024-08-30T09:06:41.245 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-30T09:06:41.723 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T09:06:42.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.379+0000 7f5261596640 1 -- 172.21.15.42:0/1974943347 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f525c10a160 msgr2=0x7f525c10a570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:06:42.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.379+0000 7f5261596640 1 --2- 172.21.15.42:0/1974943347 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f525c10a160 0x7f525c10a570 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f5254009ca0 tx=0x7f525402a8f0 comp rx=0 tx=0).stop 2024-08-30T09:06:42.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.383+0000 7f5261596640 1 -- 172.21.15.42:0/1974943347 shutdown_connections 2024-08-30T09:06:42.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.383+0000 7f5261596640 1 --2- 172.21.15.42:0/1974943347 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f525c1186a0 0x7f525c11aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:42.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.383+0000 7f5261596640 1 --2- 172.21.15.42:0/1974943347 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f525c10aab0 0x7f525c118160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:42.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.383+0000 7f5261596640 1 --2- 172.21.15.42:0/1974943347 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f525c10a160 0x7f525c10a570 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:42.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.383+0000 7f5261596640 1 -- 172.21.15.42:0/1974943347 >> 172.21.15.42:0/1974943347 conn(0x7f525c070c60 msgr2=0x7f525c071070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:06:42.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.387+0000 7f5261596640 1 -- 172.21.15.42:0/1974943347 shutdown_connections 2024-08-30T09:06:42.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.388+0000 7f5261596640 1 -- 172.21.15.42:0/1974943347 wait complete. 2024-08-30T09:06:42.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.389+0000 7f5261596640 1 Processor -- start 2024-08-30T09:06:42.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.389+0000 7f5261596640 1 -- start start 2024-08-30T09:06:42.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.389+0000 7f5261596640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f525c10a160 0x7f525c116d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:42.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.389+0000 7f5261596640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f525c10aab0 0x7f525c1172f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:42.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.390+0000 7f5261596640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f525c1186a0 0x7f525c1aea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:42.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.390+0000 7f5261596640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f525c10eb10 con 0x7f525c10a160 2024-08-30T09:06:42.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.390+0000 7f5261596640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f525c10e990 con 0x7f525c10aab0 2024-08-30T09:06:42.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.390+0000 7f5261596640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f525c10ec90 con 0x7f525c1186a0 2024-08-30T09:06:42.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.390+0000 7f525a7fc640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f525c10aab0 0x7f525c1172f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:06:42.393 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.390+0000 7f525a7fc640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f525c10aab0 0x7f525c1172f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:44596/0 (socket says 172.21.15.42:44596) 2024-08-30T09:06:42.393 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.390+0000 7f525a7fc640 1 -- 172.21.15.42:0/3779692215 learned_addr learned my addr 172.21.15.42:0/3779692215 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:06:42.393 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.390+0000 7f525affd640 1 --2- 172.21.15.42:0/3779692215 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f525c10a160 0x7f525c116d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:06:42.393 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.390+0000 7f525b7fe640 1 --2- 172.21.15.42:0/3779692215 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f525c1186a0 0x7f525c1aea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:06:42.397 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.391+0000 7f525affd640 1 -- 172.21.15.42:0/3779692215 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f525c1186a0 msgr2=0x7f525c1aea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:06:42.397 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.391+0000 7f525affd640 1 --2- 172.21.15.42:0/3779692215 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f525c1186a0 0x7f525c1aea30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:42.397 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.391+0000 7f525affd640 1 -- 172.21.15.42:0/3779692215 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f525c10aab0 msgr2=0x7f525c1172f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:06:42.397 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.391+0000 7f525affd640 1 --2- 172.21.15.42:0/3779692215 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f525c10aab0 0x7f525c1172f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:42.397 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.391+0000 7f525affd640 1 -- 172.21.15.42:0/3779692215 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f525c1af160 con 0x7f525c10a160 2024-08-30T09:06:42.397 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.395+0000 7f525affd640 1 --2- 172.21.15.42:0/3779692215 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f525c10a160 0x7f525c116d50 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f5254000f80 tx=0x7f5254030820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:06:42.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.400+0000 7f5243fff640 1 -- 172.21.15.42:0/3779692215 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5254030b10 con 0x7f525c10a160 2024-08-30T09:06:42.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.400+0000 7f5243fff640 1 -- 172.21.15.42:0/3779692215 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f525402d070 con 0x7f525c10a160 2024-08-30T09:06:42.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.400+0000 7f5243fff640 1 -- 172.21.15.42:0/3779692215 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f525403a420 con 0x7f525c10a160 2024-08-30T09:06:42.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.402+0000 7f5261596640 1 -- 172.21.15.42:0/3779692215 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f525c1af3f0 con 0x7f525c10a160 2024-08-30T09:06:42.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.402+0000 7f5261596640 1 -- 172.21.15.42:0/3779692215 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f525c1b5ae0 con 0x7f525c10a160 2024-08-30T09:06:42.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.409+0000 7f5267b0b740 1 Processor -- start 2024-08-30T09:06:42.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.410+0000 7f5267b0b740 1 -- start start 2024-08-30T09:06:42.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.410+0000 7f5267b0b740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d74f3fe070 0x55d74f7a3300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:42.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.410+0000 7f5267b0b740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d74f3d51b0 0x55d74f7a3840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:42.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.410+0000 7f5267b0b740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d74f3dd390 0x55d74f7a3d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:42.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.410+0000 7f5267b0b740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55d74f71a630 con 0x55d74f3dd390 2024-08-30T09:06:42.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.410+0000 7f5267b0b740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55d74f709bc0 con 0x55d74f3fe070 2024-08-30T09:06:42.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.410+0000 7f5267b0b740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55d74f70db70 con 0x55d74f3d51b0 2024-08-30T09:06:42.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.413+0000 7f525affd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d74f3d51b0 0x55d74f7a3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:06:42.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.414+0000 7f525b7fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d74f3fe070 0x55d74f7a3300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:06:42.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.414+0000 7f525b7fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d74f3fe070 0x55d74f7a3300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:44612/0 (socket says 172.21.15.42:44612) 2024-08-30T09:06:42.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.414+0000 7f525b7fe640 1 -- 172.21.15.42:0/4256266592 learned_addr learned my addr 172.21.15.42:0/4256266592 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:06:42.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.415+0000 7f5243fff640 1 -- 172.21.15.42:0/3779692215 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7f5254004b40 con 0x7f525c10a160 2024-08-30T09:06:42.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.416+0000 7f5243fff640 1 --2- 172.21.15.42:0/3779692215 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f52440b4cc0 0x7f52440b71d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:42.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.417+0000 7f525b7fe640 1 -- 172.21.15.42:0/4256266592 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d74f3d51b0 msgr2=0x55d74f7a3840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:06:42.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.417+0000 7f525b7fe640 1 --2- 172.21.15.42:0/4256266592 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d74f3d51b0 0x55d74f7a3840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:42.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.417+0000 7f525b7fe640 1 -- 172.21.15.42:0/4256266592 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d74f3dd390 msgr2=0x55d74f7a3d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:06:42.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.417+0000 7f525b7fe640 1 --2- 172.21.15.42:0/4256266592 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d74f3dd390 0x55d74f7a3d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:42.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.417+0000 7f525b7fe640 1 -- 172.21.15.42:0/4256266592 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d74f469650 con 0x55d74f3fe070 2024-08-30T09:06:42.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.417+0000 7f525b7fe640 1 --2- 172.21.15.42:0/4256266592 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d74f3fe070 0x55d74f7a3300 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f5250009a60 tx=0x7f5250009f20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1cdd3a108ca9db2d server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:06:42.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.418+0000 7f5243fff640 1 -- 172.21.15.42:0/3779692215 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f52540ccad0 con 0x7f525c10a160 2024-08-30T09:06:42.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.418+0000 7f52417fa640 1 -- 172.21.15.42:0/4256266592 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5250016070 con 0x55d74f3fe070 2024-08-30T09:06:42.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.418+0000 7f525b7fe640 1 --2- 172.21.15.42:0/3779692215 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f52440b4cc0 0x7f52440b71d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:06:42.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.418+0000 7f52417fa640 1 -- 172.21.15.42:0/4256266592 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f525000ec50 con 0x55d74f3fe070 2024-08-30T09:06:42.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.420+0000 7f5267b0b740 1 -- 172.21.15.42:0/4256266592 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d74f3fe070 msgr2=0x55d74f7a3300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:06:42.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.420+0000 7f5267b0b740 1 --2- 172.21.15.42:0/4256266592 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d74f3fe070 0x55d74f7a3300 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f5250009a60 tx=0x7f5250009f20 comp rx=0 tx=0).stop 2024-08-30T09:06:42.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.421+0000 7f525b7fe640 1 --2- 172.21.15.42:0/3779692215 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f52440b4cc0 0x7f52440b71d0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f5250010fd0 tx=0x7f5250010b20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:06:42.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.421+0000 7f5267b0b740 1 -- 172.21.15.42:0/4256266592 shutdown_connections 2024-08-30T09:06:42.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.421+0000 7f5267b0b740 1 --2- 172.21.15.42:0/4256266592 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d74f3d51b0 0x55d74f7a3840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:42.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.421+0000 7f5267b0b740 1 --2- 172.21.15.42:0/4256266592 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d74f3dd390 0x55d74f7a3d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:42.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.421+0000 7f5267b0b740 1 --2- 172.21.15.42:0/4256266592 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d74f3fe070 0x55d74f7a3300 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:42.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.421+0000 7f5267b0b740 1 -- 172.21.15.42:0/4256266592 >> 172.21.15.42:0/4256266592 conn(0x55d74f3f2460 msgr2=0x55d74f7a11f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:06:42.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.424+0000 7f5267b0b740 1 -- 172.21.15.42:0/4256266592 shutdown_connections 2024-08-30T09:06:42.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.425+0000 7f5267b0b740 1 -- 172.21.15.42:0/4256266592 wait complete. 2024-08-30T09:06:42.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.426+0000 7f5267b0b740 1 -- start start 2024-08-30T09:06:42.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.426+0000 7f5267b0b740 1 Processor -- start 2024-08-30T09:06:42.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.427+0000 7f5267b0b740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d74f3d51b0 0x55d74f7aea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:42.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.427+0000 7f5267b0b740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d74f3dd390 0x55d74f7aef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:42.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.427+0000 7f5267b0b740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d74f3fe070 0x55d74f7cbeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:42.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.427+0000 7f5267b0b740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55d74f56c700 con 0x55d74f3d51b0 2024-08-30T09:06:42.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.427+0000 7f5267b0b740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55d74f690700 con 0x55d74f3dd390 2024-08-30T09:06:42.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.427+0000 7f5267b0b740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55d74f7c2b40 con 0x55d74f3fe070 2024-08-30T09:06:42.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.430+0000 7f5267b0b740 10 client.0 _ll_register_callbacks cb 0x55d74f6bfdc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:06:42.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.428+0000 7f525b7fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d74f3fe070 0x55d74f7cbeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:06:42.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.431+0000 7f525b7fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d74f3fe070 0x55d74f7cbeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:47104/0 (socket says 172.21.15.42:47104) 2024-08-30T09:06:42.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.431+0000 7f525b7fe640 1 -- 172.21.15.42:0/610867807 learned_addr learned my addr 172.21.15.42:0/610867807 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:06:42.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.430+0000 7f525a7fc640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d74f3dd390 0x55d74f7aef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:06:42.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.430+0000 7f525affd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d74f3d51b0 0x55d74f7aea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:06:42.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.434+0000 7f525a7fc640 1 -- 172.21.15.42:0/610867807 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d74f3fe070 msgr2=0x55d74f7cbeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:06:42.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.434+0000 7f525a7fc640 1 --2- 172.21.15.42:0/610867807 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d74f3fe070 0x55d74f7cbeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:42.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.434+0000 7f525a7fc640 1 -- 172.21.15.42:0/610867807 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d74f3d51b0 msgr2=0x55d74f7aea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:06:42.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.434+0000 7f525a7fc640 1 --2- 172.21.15.42:0/610867807 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d74f3d51b0 0x55d74f7aea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:42.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.434+0000 7f525a7fc640 1 -- 172.21.15.42:0/610867807 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d74f6c98b0 con 0x55d74f3dd390 2024-08-30T09:06:42.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.434+0000 7f525b7fe640 1 --2- 172.21.15.42:0/610867807 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d74f3fe070 0x55d74f7cbeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-30T09:06:42.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.435+0000 7f525a7fc640 1 --2- 172.21.15.42:0/610867807 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d74f3dd390 0x55d74f7aef80 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f524c005d00 tx=0x7f524c0107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4f8aa7fe720d2341 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:06:42.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.436+0000 7f523eff5640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-30T09:06:42.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.436+0000 7f523eff5640 1 -- 172.21.15.42:0/610867807 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f524c004230 con 0x55d74f3dd390 2024-08-30T09:06:42.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.436+0000 7f523eff5640 1 -- 172.21.15.42:0/610867807 <== mon.2 v2:172.21.15.167:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f524c0043d0 con 0x55d74f3dd390 2024-08-30T09:06:42.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.437+0000 7f5267b0b740 1 -- 172.21.15.42:0/610867807 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f525000ec50 con 0x55d74f3dd390 2024-08-30T09:06:42.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.441+0000 7f523eff5640 1 -- 172.21.15.42:0/610867807 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f524c011650 con 0x55d74f3dd390 2024-08-30T09:06:42.447 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.444+0000 7f5267b0b740 10 client.26228 fetch_fsmap learned FSMap version 33 2024-08-30T09:06:42.447 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.444+0000 7f5267b0b740 1 -- 172.21.15.42:0/610867807 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55d74f7f09f0 con 0x55d74f3dd390 2024-08-30T09:06:42.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.447+0000 7f523eff5640 1 -- 172.21.15.42:0/610867807 <== mon.2 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f525000ec50 con 0x55d74f3dd390 2024-08-30T09:06:42.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.447+0000 7f523eff5640 1 -- 172.21.15.42:0/610867807 <== mon.2 v2:172.21.15.167:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f524c003a60 con 0x55d74f3dd390 2024-08-30T09:06:42.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.450+0000 7f5267b0b740 10 client.26228 fetch_fsmap finished waiting for FSMap version 33 2024-08-30T09:06:42.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.450+0000 7f5267b0b740 10 client.26228 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-30T09:06:42.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.450+0000 7f5267b0b740 20 client.26228 populate_metadata read hostname 'smithi042' 2024-08-30T09:06:42.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.450+0000 7f5267b0b740 1 --2- 172.21.15.42:0/610867807 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55d74f3bc9f0 0x55d74f7f3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:42.454 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.450+0000 7f5267b0b740 4 client.26228 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-30T09:06:42.454 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.450+0000 7f5267b0b740 1 -- 172.21.15.42:0/610867807 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d74f71c580 con 0x55d74f3bc9f0 2024-08-30T09:06:42.454 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.451+0000 7f525affd640 1 --2- 172.21.15.42:0/610867807 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55d74f3bc9f0 0x55d74f7f3ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:06:42.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.455+0000 7f525affd640 1 --2- 172.21.15.42:0/610867807 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55d74f3bc9f0 0x55d74f7f3ec0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=433d038ba874b464 server_cookie=6ddbe83f496b59ca in_seq=0 out_seq=0 2024-08-30T09:06:42.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.455+0000 7f523eff5640 10 client.26228 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:06:42.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.456+0000 7f523eff5640 1 -- 172.21.15.42:0/610867807 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d74f71c580 con 0x55d74f3bc9f0 2024-08-30T09:06:42.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.456+0000 7f523eff5640 10 client.26228 handle_command_reply: tid=0 2024-08-30T09:06:42.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.456+0000 7f5267b0b740 1 client.26228 shutdown 2024-08-30T09:06:42.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.457+0000 7f5267b0b740 1 -- 172.21.15.42:0/610867807 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d74f3dd390 msgr2=0x55d74f7aef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:06:42.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.457+0000 7f5267b0b740 1 --2- 172.21.15.42:0/610867807 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d74f3dd390 0x55d74f7aef80 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f524c005d00 tx=0x7f524c0107a0 comp rx=0 tx=0).stop 2024-08-30T09:06:42.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.461+0000 7f5267b0b740 1 -- 172.21.15.42:0/610867807 shutdown_connections 2024-08-30T09:06:42.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.461+0000 7f5267b0b740 1 --2- 172.21.15.42:0/610867807 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55d74f3bc9f0 0x55d74f7f3ec0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:42.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.461+0000 7f5267b0b740 1 --2- 172.21.15.42:0/610867807 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d74f3fe070 0x55d74f7cbeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:42.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.461+0000 7f5267b0b740 1 --2- 172.21.15.42:0/610867807 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d74f3dd390 0x55d74f7aef80 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:42.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.461+0000 7f5267b0b740 1 --2- 172.21.15.42:0/610867807 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d74f3d51b0 0x55d74f7aea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:42.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.461+0000 7f5267b0b740 1 -- 172.21.15.42:0/610867807 >> 172.21.15.42:0/610867807 conn(0x55d74f3f2460 msgr2=0x55d74f7a64e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:06:42.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.461+0000 7f523eff5640 0 client.26228 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:06:42.465 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.463+0000 7f5267b0b740 1 -- 172.21.15.42:0/610867807 shutdown_connections 2024-08-30T09:06:42.465 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.463+0000 7f5267b0b740 1 -- 172.21.15.42:0/610867807 wait complete. 2024-08-30T09:06:42.465 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.463+0000 7f5267b0b740 20 client.26228 trim_cache size 0 max 16384 2024-08-30T09:06:42.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.500+0000 7f5267b0b740 1 Processor -- start 2024-08-30T09:06:42.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.500+0000 7f5267b0b740 1 -- start start 2024-08-30T09:06:42.503 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:06:42 smithi042 ceph-mon[18652]: pgmap v551: 129 pgs: 129 active+clean; 1.4 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 9.2 MiB/s wr, 548 op/s 2024-08-30T09:06:42.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.500+0000 7f5267b0b740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d74f3d51b0 0x55d74f7ad930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:42.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.500+0000 7f5267b0b740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d74f3dd390 0x55d74f7b0670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:42.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.500+0000 7f5267b0b740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d74f3fe070 0x55d74f7a07c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:42.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.500+0000 7f5267b0b740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f525c078fc0 con 0x55d74f3dd390 2024-08-30T09:06:42.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.500+0000 7f5267b0b740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f525c076830 con 0x55d74f3d51b0 2024-08-30T09:06:42.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.500+0000 7f5267b0b740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f525c124fd0 con 0x55d74f3fe070 2024-08-30T09:06:42.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.501+0000 7f525b7fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d74f3fe070 0x55d74f7a07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:06:42.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.501+0000 7f525b7fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d74f3fe070 0x55d74f7a07c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:47118/0 (socket says 172.21.15.42:47118) 2024-08-30T09:06:42.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.501+0000 7f525b7fe640 1 -- 172.21.15.42:0/797152489 learned_addr learned my addr 172.21.15.42:0/797152489 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:06:42.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.501+0000 7f525affd640 1 --2- 172.21.15.42:0/797152489 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d74f3d51b0 0x55d74f7ad930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:06:42.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.501+0000 7f525b7fe640 1 -- 172.21.15.42:0/797152489 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d74f3d51b0 msgr2=0x55d74f7ad930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:06:42.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.501+0000 7f525b7fe640 1 --2- 172.21.15.42:0/797152489 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d74f3d51b0 0x55d74f7ad930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:42.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.501+0000 7f525b7fe640 1 -- 172.21.15.42:0/797152489 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d74f3dd390 msgr2=0x55d74f7b0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:06:42.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.501+0000 7f525b7fe640 1 --2- 172.21.15.42:0/797152489 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d74f3dd390 0x55d74f7b0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:42.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.501+0000 7f525b7fe640 1 -- 172.21.15.42:0/797152489 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d74f7a5820 con 0x55d74f3fe070 2024-08-30T09:06:42.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.501+0000 7f525b7fe640 1 --2- 172.21.15.42:0/797152489 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d74f3fe070 0x55d74f7a07c0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f525000e780 tx=0x7f5250022000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bd03350ec716d268 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:06:42.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.502+0000 7f5240ff9640 1 -- 172.21.15.42:0/797152489 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5250016040 con 0x55d74f3fe070 2024-08-30T09:06:42.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.502+0000 7f5240ff9640 1 -- 172.21.15.42:0/797152489 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5250009650 con 0x55d74f3fe070 2024-08-30T09:06:42.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.503+0000 7f5267b0b740 1 -- 172.21.15.42:0/797152489 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d74f3fe070 msgr2=0x55d74f7a07c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:06:42.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.503+0000 7f5267b0b740 1 --2- 172.21.15.42:0/797152489 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d74f3fe070 0x55d74f7a07c0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f525000e780 tx=0x7f5250022000 comp rx=0 tx=0).stop 2024-08-30T09:06:42.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.503+0000 7f5267b0b740 1 -- 172.21.15.42:0/797152489 shutdown_connections 2024-08-30T09:06:42.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.503+0000 7f5267b0b740 1 --2- 172.21.15.42:0/797152489 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d74f3fe070 0x55d74f7a07c0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:42.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.503+0000 7f5267b0b740 1 --2- 172.21.15.42:0/797152489 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d74f3dd390 0x55d74f7b0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:42.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.503+0000 7f5267b0b740 1 --2- 172.21.15.42:0/797152489 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d74f3d51b0 0x55d74f7ad930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:42.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.503+0000 7f5267b0b740 1 -- 172.21.15.42:0/797152489 >> 172.21.15.42:0/797152489 conn(0x55d74f3f2460 msgr2=0x55d74f7aff60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:06:42.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.504+0000 7f5267b0b740 1 -- 172.21.15.42:0/797152489 shutdown_connections 2024-08-30T09:06:42.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.504+0000 7f5267b0b740 1 -- 172.21.15.42:0/797152489 wait complete. 2024-08-30T09:06:42.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.505+0000 7f5267b0b740 1 -- start start 2024-08-30T09:06:42.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.505+0000 7f5267b0b740 1 Processor -- start 2024-08-30T09:06:42.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.505+0000 7f5267b0b740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d74f3d51b0 0x55d74f7c87d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:42.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.505+0000 7f5267b0b740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d74f3dd390 0x55d74f7c3c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:42.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.505+0000 7f5267b0b740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d74f3fe070 0x55d74f7c4390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:42.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.505+0000 7f5267b0b740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f525c125640 con 0x55d74f3fe070 2024-08-30T09:06:42.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.505+0000 7f5267b0b740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f525c076160 con 0x55d74f3d51b0 2024-08-30T09:06:42.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.505+0000 7f5267b0b740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f525c0753e0 con 0x55d74f3dd390 2024-08-30T09:06:42.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.505+0000 7f5267b0b740 10 client.0 _ll_register_callbacks cb 0x7f524c00b810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:06:42.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.506+0000 7f525a7fc640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d74f3dd390 0x55d74f7c3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:06:42.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.506+0000 7f525a7fc640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d74f3dd390 0x55d74f7c3c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:47132/0 (socket says 172.21.15.42:47132) 2024-08-30T09:06:42.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.506+0000 7f525a7fc640 1 -- 172.21.15.42:0/3770521196 learned_addr learned my addr 172.21.15.42:0/3770521196 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:06:42.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.506+0000 7f525a7fc640 1 -- 172.21.15.42:0/3770521196 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d74f3d51b0 msgr2=0x55d74f7c87d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:06:42.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.506+0000 7f525a7fc640 1 --2- 172.21.15.42:0/3770521196 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d74f3d51b0 0x55d74f7c87d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:42.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.506+0000 7f525a7fc640 1 -- 172.21.15.42:0/3770521196 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d74f3fe070 msgr2=0x55d74f7c4390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:06:42.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.506+0000 7f525a7fc640 1 --2- 172.21.15.42:0/3770521196 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d74f3fe070 0x55d74f7c4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:42.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.506+0000 7f525a7fc640 1 -- 172.21.15.42:0/3770521196 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d74f85d840 con 0x55d74f3dd390 2024-08-30T09:06:42.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.506+0000 7f525a7fc640 1 --2- 172.21.15.42:0/3770521196 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d74f3dd390 0x55d74f7c3c80 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f524c00b9d0 tx=0x7f524c0041e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=74ecb2226b94e5a7 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:06:42.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.508+0000 7f523eff5640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-08-30T09:06:42.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.508+0000 7f523eff5640 1 -- 172.21.15.42:0/3770521196 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f524c033060 con 0x55d74f3dd390 2024-08-30T09:06:42.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.508+0000 7f523eff5640 1 -- 172.21.15.42:0/3770521196 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f524c025910 con 0x55d74f3dd390 2024-08-30T09:06:42.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.508+0000 7f5267b0b740 1 -- 172.21.15.42:0/3770521196 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5250009650 con 0x55d74f3dd390 2024-08-30T09:06:42.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.508+0000 7f523eff5640 1 -- 172.21.15.42:0/3770521196 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f524c03a050 con 0x55d74f3dd390 2024-08-30T09:06:42.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.508+0000 7f5267b0b740 10 client.26212 fetch_fsmap learned FSMap version 33 2024-08-30T09:06:42.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.508+0000 7f5267b0b740 1 -- 172.21.15.42:0/3770521196 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55d74f7b1940 con 0x55d74f3dd390 2024-08-30T09:06:42.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.509+0000 7f523eff5640 1 -- 172.21.15.42:0/3770521196 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5250009650 con 0x55d74f3dd390 2024-08-30T09:06:42.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.509+0000 7f523eff5640 1 -- 172.21.15.42:0/3770521196 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f524c0035f0 con 0x55d74f3dd390 2024-08-30T09:06:42.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.509+0000 7f5267b0b740 10 client.26212 fetch_fsmap finished waiting for FSMap version 33 2024-08-30T09:06:42.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.509+0000 7f5267b0b740 10 client.26212 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-30T09:06:42.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.509+0000 7f5267b0b740 20 client.26212 populate_metadata read hostname 'smithi042' 2024-08-30T09:06:42.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.509+0000 7f5267b0b740 1 --2- 172.21.15.42:0/3770521196 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55d74f391da0 0x55d74f7b4120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:42.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.509+0000 7f5267b0b740 4 client.26212 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-30T09:06:42.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.509+0000 7f5267b0b740 1 -- 172.21.15.42:0/3770521196 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f5250005640 con 0x55d74f391da0 2024-08-30T09:06:42.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.509+0000 7f525affd640 1 --2- 172.21.15.42:0/3770521196 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55d74f391da0 0x55d74f7b4120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:06:42.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.511+0000 7f525affd640 1 --2- 172.21.15.42:0/3770521196 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55d74f391da0 0x55d74f7b4120 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b9672946e1f40ee4 server_cookie=79a4cae07d754b6a in_seq=0 out_seq=0 2024-08-30T09:06:42.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.511+0000 7f523eff5640 10 client.26212 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:06:42.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.511+0000 7f523eff5640 1 -- 172.21.15.42:0/3770521196 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+2558 (crc 0 0 0) 0x7f5250005640 con 0x55d74f391da0 2024-08-30T09:06:42.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.511+0000 7f523eff5640 10 client.26212 handle_command_reply: tid=0 2024-08-30T09:06:42.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.511+0000 7f5267b0b740 1 client.26212 shutdown 2024-08-30T09:06:42.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.511+0000 7f5267b0b740 1 -- 172.21.15.42:0/3770521196 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d74f3dd390 msgr2=0x55d74f7c3c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:06:42.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.511+0000 7f5267b0b740 1 --2- 172.21.15.42:0/3770521196 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d74f3dd390 0x55d74f7c3c80 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f524c00b9d0 tx=0x7f524c0041e0 comp rx=0 tx=0).stop 2024-08-30T09:06:42.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.511+0000 7f5267b0b740 1 -- 172.21.15.42:0/3770521196 shutdown_connections 2024-08-30T09:06:42.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.511+0000 7f5267b0b740 1 --2- 172.21.15.42:0/3770521196 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55d74f391da0 0x55d74f7b4120 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:42.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.511+0000 7f5267b0b740 1 --2- 172.21.15.42:0/3770521196 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d74f3dd390 0x55d74f7c3c80 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:42.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.511+0000 7f5267b0b740 1 --2- 172.21.15.42:0/3770521196 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d74f3fe070 0x55d74f7c4390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:42.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.511+0000 7f5267b0b740 1 --2- 172.21.15.42:0/3770521196 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d74f3d51b0 0x55d74f7c87d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:42.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.511+0000 7f5267b0b740 1 -- 172.21.15.42:0/3770521196 >> 172.21.15.42:0/3770521196 conn(0x55d74f3f2460 msgr2=0x55d74f7aef80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:06:42.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.512+0000 7f523eff5640 0 client.26212 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:06:42.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.512+0000 7f5267b0b740 1 -- 172.21.15.42:0/3770521196 shutdown_connections 2024-08-30T09:06:42.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.512+0000 7f5267b0b740 1 -- 172.21.15.42:0/3770521196 wait complete. 2024-08-30T09:06:42.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.512+0000 7f5267b0b740 20 client.26212 trim_cache size 0 max 16384 2024-08-30T09:06:42.517 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-08-30T09:06:42.517 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "IDLE (waiting for more scrubs)", 2024-08-30T09:06:42.517 INFO:teuthology.orchestra.run.smithi042.stdout: "scrubs": { 2024-08-30T09:06:42.517 INFO:teuthology.orchestra.run.smithi042.stdout: "665ec740-7254-49cb-84ef-deff71c461b0": { 2024-08-30T09:06:42.517 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:06:42.517 INFO:teuthology.orchestra.run.smithi042.stdout: "tag": "665ec740-7254-49cb-84ef-deff71c461b0", 2024-08-30T09:06:42.517 INFO:teuthology.orchestra.run.smithi042.stdout: "options": "recursive,force" 2024-08-30T09:06:42.517 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:06:42.517 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:06:42.517 INFO:teuthology.orchestra.run.smithi042.stdout: "scrub_stats": { 2024-08-30T09:06:42.517 INFO:teuthology.orchestra.run.smithi042.stdout: "665ec740-7254-49cb-84ef-deff71c461b0": { 2024-08-30T09:06:42.517 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:06:37.144035+0000", 2024-08-30T09:06:42.518 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:06:42.518 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:06:42.518 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:06:42.518 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:06:42.518 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:06:42.518 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:06:42.518 INFO:teuthology.orchestra.run.smithi042.stdout: "1af4252a-4a99-47e9-bbf0-becf5e573dd5": { 2024-08-30T09:06:42.518 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:05:48.565881+0000", 2024-08-30T09:06:42.518 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:06:42.518 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:06:42.518 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:06:42.518 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:06:42.518 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:06:42.519 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:06:42.519 INFO:teuthology.orchestra.run.smithi042.stdout: "347d05ed-4bd3-4590-b099-314f719aab39": { 2024-08-30T09:06:42.519 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:05:00.439721+0000", 2024-08-30T09:06:42.519 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:06:42.519 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:06:42.519 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:06:42.519 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:06:42.519 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:06:42.519 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:06:42.519 INFO:teuthology.orchestra.run.smithi042.stdout: "88cd7478-2b20-47de-ba51-256479749232": { 2024-08-30T09:06:42.519 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:04:08.922625+0000", 2024-08-30T09:06:42.519 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:06:42.519 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:06:42.519 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:06:42.519 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:06:42.520 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:06:42.520 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:06:42.520 INFO:teuthology.orchestra.run.smithi042.stdout: "749f4867-0b70-49e5-af2d-8362a240ebc3": { 2024-08-30T09:06:42.520 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:03:16.386554+0000", 2024-08-30T09:06:42.520 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:06:42.520 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:06:42.520 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:06:42.520 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:06:42.520 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:06:42.520 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:06:42.520 INFO:teuthology.orchestra.run.smithi042.stdout: "85134e4b-ab4a-4885-b07e-a3ad01eaf13d": { 2024-08-30T09:06:42.520 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:01:27.738462+0000", 2024-08-30T09:06:42.520 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:06:42.521 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:06:42.521 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:06:42.521 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:06:42.521 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:06:42.521 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:06:42.521 INFO:teuthology.orchestra.run.smithi042.stdout: "c827ad7a-44ee-4e1d-9bb3-51fb7ab00c73": { 2024-08-30T09:06:42.521 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:02:21.686517+0000", 2024-08-30T09:06:42.521 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:06:42.521 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:06:42.521 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:06:42.521 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:06:42.521 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:06:42.521 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:06:42.521 INFO:teuthology.orchestra.run.smithi042.stdout: "5e2ed104-0625-478a-a78b-584e7f4c2817": { 2024-08-30T09:06:42.521 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:00:33.928400+0000", 2024-08-30T09:06:42.522 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:06:42.522 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:06:42.522 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:06:42.522 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:06:42.522 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:06:42.522 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:06:42.522 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:06:42.522 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-08-30T09:06:42.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.513+0000 7f5261596640 1 -- 172.21.15.42:0/3779692215 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f52440b4cc0 msgr2=0x7f52440b71d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:06:42.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.513+0000 7f5261596640 1 --2- 172.21.15.42:0/3779692215 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f52440b4cc0 0x7f52440b71d0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f5250010fd0 tx=0x7f5250010b20 comp rx=0 tx=0).stop 2024-08-30T09:06:42.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.513+0000 7f5261596640 1 -- 172.21.15.42:0/3779692215 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f525c10a160 msgr2=0x7f525c116d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:06:42.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.513+0000 7f5261596640 1 --2- 172.21.15.42:0/3779692215 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f525c10a160 0x7f525c116d50 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f5254000f80 tx=0x7f5254030820 comp rx=0 tx=0).stop 2024-08-30T09:06:42.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.513+0000 7f5261596640 1 -- 172.21.15.42:0/3779692215 shutdown_connections 2024-08-30T09:06:42.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.513+0000 7f5261596640 1 --2- 172.21.15.42:0/3779692215 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f52440b4cc0 0x7f52440b71d0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:42.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.513+0000 7f5261596640 1 --2- 172.21.15.42:0/3779692215 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f525c1186a0 0x7f525c1aea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:42.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.513+0000 7f5261596640 1 --2- 172.21.15.42:0/3779692215 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f525c10aab0 0x7f525c1172f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:42.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.513+0000 7f5261596640 1 --2- 172.21.15.42:0/3779692215 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f525c10a160 0x7f525c116d50 secure :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f5254000f80 tx=0x7f5254030820 comp rx=0 tx=0).stop 2024-08-30T09:06:42.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.513+0000 7f5261596640 1 -- 172.21.15.42:0/3779692215 >> 172.21.15.42:0/3779692215 conn(0x7f525c070c60 msgr2=0x7f525c118c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:06:42.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.514+0000 7f5261596640 1 -- 172.21.15.42:0/3779692215 shutdown_connections 2024-08-30T09:06:42.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:06:42.514+0000 7f5261596640 1 -- 172.21.15.42:0/3779692215 wait complete. 2024-08-30T09:06:42.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:06:42 smithi190 ceph-mon[22001]: pgmap v551: 129 pgs: 129 active+clean; 1.4 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 9.2 MiB/s wr, 548 op/s 2024-08-30T09:06:42.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:06:42 smithi167 ceph-mon[22063]: pgmap v551: 129 pgs: 129 active+clean; 1.4 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 9.2 MiB/s wr, 548 op/s 2024-08-30T09:06:43.172 INFO:tasks.cephfs.filesystem:scrub status for tag:665ec740-7254-49cb-84ef-deff71c461b0 - {'path': '/', 'tag': '665ec740-7254-49cb-84ef-deff71c461b0', 'options': 'recursive,force'} 2024-08-30T09:06:44.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:06:44 smithi042 ceph-mon[18652]: pgmap v552: 129 pgs: 129 active+clean; 1.4 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 12 MiB/s wr, 722 op/s 2024-08-30T09:06:44.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:06:44 smithi190 ceph-mon[22001]: pgmap v552: 129 pgs: 129 active+clean; 1.4 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 12 MiB/s wr, 722 op/s 2024-08-30T09:06:44.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:06:44 smithi167 ceph-mon[22063]: pgmap v552: 129 pgs: 129 active+clean; 1.4 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 12 MiB/s wr, 722 op/s 2024-08-30T09:06:45.264 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.262+0000 7feeb0a7e640 1 -- 172.21.15.42:0/4287908087 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7feeac110000 msgr2=0x7feeac112510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:06:45.264 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.262+0000 7feeb0a7e640 1 --2- 172.21.15.42:0/4287908087 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7feeac110000 0x7feeac112510 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7feea400adb0 tx=0x7feea402b7c0 comp rx=0 tx=0).stop 2024-08-30T09:06:45.264 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.262+0000 7feeb0a7e640 1 -- 172.21.15.42:0/4287908087 shutdown_connections 2024-08-30T09:06:45.264 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.262+0000 7feeb0a7e640 1 --2- 172.21.15.42:0/4287908087 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7feeac110000 0x7feeac112510 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:45.264 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.262+0000 7feeb0a7e640 1 --2- 172.21.15.42:0/4287908087 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feeac10d3e0 0x7feeac10f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:45.264 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.262+0000 7feeb0a7e640 1 --2- 172.21.15.42:0/4287908087 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feeac10a9f0 0x7feeac10ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:45.265 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.262+0000 7feeb0a7e640 1 -- 172.21.15.42:0/4287908087 >> 172.21.15.42:0/4287908087 conn(0x7feeac0705b0 msgr2=0x7feeac072a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:06:45.265 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.262+0000 7feeb0a7e640 1 -- 172.21.15.42:0/4287908087 shutdown_connections 2024-08-30T09:06:45.265 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.262+0000 7feeb0a7e640 1 -- 172.21.15.42:0/4287908087 wait complete. 2024-08-30T09:06:45.266 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.263+0000 7feeb0a7e640 1 Processor -- start 2024-08-30T09:06:45.266 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.263+0000 7feeb0a7e640 1 -- start start 2024-08-30T09:06:45.266 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.263+0000 7feeb0a7e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feeac10a9f0 0x7feeac1a24e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:45.266 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.264+0000 7feeb0a7e640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7feeac10d3e0 0x7feeac1a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:45.266 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.264+0000 7feeb0a7e640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feeac110000 0x7feeac1a3700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:45.266 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.264+0000 7feeb0a7e640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7feeac115690 con 0x7feeac10d3e0 2024-08-30T09:06:45.266 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.264+0000 7feeb0a7e640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7feeac115510 con 0x7feeac110000 2024-08-30T09:06:45.266 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.264+0000 7feeb0a7e640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7feeac115810 con 0x7feeac10a9f0 2024-08-30T09:06:45.266 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.264+0000 7feeaa59c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feeac10a9f0 0x7feeac1a24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:06:45.266 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.264+0000 7feeaad9d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feeac110000 0x7feeac1a3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:06:45.266 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.264+0000 7feeaa59c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feeac10a9f0 0x7feeac1a24e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:47154/0 (socket says 172.21.15.42:47154) 2024-08-30T09:06:45.267 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.264+0000 7feeaa59c640 1 -- 172.21.15.42:0/759151790 learned_addr learned my addr 172.21.15.42:0/759151790 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:06:45.267 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.265+0000 7feea9d9b640 1 --2- 172.21.15.42:0/759151790 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7feeac10d3e0 0x7feeac1a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:06:45.267 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.265+0000 7feeaa59c640 1 -- 172.21.15.42:0/759151790 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feeac110000 msgr2=0x7feeac1a3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:06:45.268 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.265+0000 7feeaa59c640 1 --2- 172.21.15.42:0/759151790 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feeac110000 0x7feeac1a3700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:45.268 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.265+0000 7feeaa59c640 1 -- 172.21.15.42:0/759151790 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7feeac10d3e0 msgr2=0x7feeac1a2b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:06:45.268 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.265+0000 7feeaa59c640 1 --2- 172.21.15.42:0/759151790 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7feeac10d3e0 0x7feeac1a2b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:45.268 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.265+0000 7feeaa59c640 1 -- 172.21.15.42:0/759151790 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feeac1a3f20 con 0x7feeac10a9f0 2024-08-30T09:06:45.268 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.266+0000 7feeaad9d640 1 --2- 172.21.15.42:0/759151790 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feeac110000 0x7feeac1a3700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-30T09:06:45.268 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.266+0000 7feea9d9b640 1 --2- 172.21.15.42:0/759151790 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7feeac10d3e0 0x7feeac1a2b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-30T09:06:45.269 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.266+0000 7feeaa59c640 1 --2- 172.21.15.42:0/759151790 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feeac10a9f0 0x7feeac1a24e0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fee9c00e9f0 tx=0x7fee9c00edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:06:45.269 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.266+0000 7fee9b7fe640 1 -- 172.21.15.42:0/759151790 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee9c00cd80 con 0x7feeac10a9f0 2024-08-30T09:06:45.269 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.266+0000 7feeb0a7e640 1 -- 172.21.15.42:0/759151790 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feeac1a4210 con 0x7feeac10a9f0 2024-08-30T09:06:45.269 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.266+0000 7fee9b7fe640 1 -- 172.21.15.42:0/759151790 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee9c0054d0 con 0x7feeac10a9f0 2024-08-30T09:06:45.269 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.266+0000 7fee9b7fe640 1 -- 172.21.15.42:0/759151790 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee9c00cd80 con 0x7feeac10a9f0 2024-08-30T09:06:45.269 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.266+0000 7feeb0a7e640 1 -- 172.21.15.42:0/759151790 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feeac1aa900 con 0x7feeac10a9f0 2024-08-30T09:06:45.271 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.269+0000 7fee9b7fe640 1 -- 172.21.15.42:0/759151790 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7fee9c004e50 con 0x7feeac10a9f0 2024-08-30T09:06:45.271 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.270+0000 7feeb79eb740 1 Processor -- start 2024-08-30T09:06:45.272 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.270+0000 7feeb79eb740 1 -- start start 2024-08-30T09:06:45.272 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.270+0000 7feeb79eb740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5605e081b1b0 0x5605e0bc5370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:45.273 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.270+0000 7feeb79eb740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5605e07f22f0 0x5605e0bc59e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:45.273 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.270+0000 7feeb79eb740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5605e07fa4d0 0x5605e0bc6530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:45.273 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.270+0000 7feeb79eb740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x5605e0b34000 con 0x5605e07fa4d0 2024-08-30T09:06:45.273 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.270+0000 7feeb79eb740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5605e0b23590 con 0x5605e07f22f0 2024-08-30T09:06:45.273 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.270+0000 7feeb79eb740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5605e0b27540 con 0x5605e081b1b0 2024-08-30T09:06:45.273 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.271+0000 7feeaad9d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5605e081b1b0 0x5605e0bc5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:06:45.273 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.271+0000 7feeaad9d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5605e081b1b0 0x5605e0bc5370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:47164/0 (socket says 172.21.15.42:47164) 2024-08-30T09:06:45.273 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.271+0000 7feeaad9d640 1 -- 172.21.15.42:0/2147539457 learned_addr learned my addr 172.21.15.42:0/2147539457 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:06:45.273 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.271+0000 7fee9b7fe640 1 --2- 172.21.15.42:0/759151790 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fee940b4f20 0x7fee940b7430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:45.273 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.271+0000 7feea9d9b640 1 --2- 172.21.15.42:0/2147539457 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5605e07fa4d0 0x5605e0bc6530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:06:45.274 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.271+0000 7feeaad9d640 1 --2- 172.21.15.42:0/759151790 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fee940b4f20 0x7fee940b7430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:06:45.274 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.271+0000 7fee9b7fe640 1 -- 172.21.15.42:0/759151790 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7fee9c00cb80 con 0x7feeac10a9f0 2024-08-30T09:06:45.274 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.271+0000 7feeaad9d640 1 -- 172.21.15.42:0/2147539457 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5605e07f22f0 msgr2=0x5605e0bc59e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-30T09:06:45.274 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.271+0000 7feeaad9d640 1 --2- 172.21.15.42:0/2147539457 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5605e07f22f0 0x5605e0bc59e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:45.274 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.271+0000 7feeaad9d640 1 -- 172.21.15.42:0/2147539457 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5605e07fa4d0 msgr2=0x5605e0bc6530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:06:45.274 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.271+0000 7feeaad9d640 1 --2- 172.21.15.42:0/2147539457 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5605e07fa4d0 0x5605e0bc6530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:45.275 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.271+0000 7feeaad9d640 1 -- 172.21.15.42:0/2147539457 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5605e0886790 con 0x5605e081b1b0 2024-08-30T09:06:45.275 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.272+0000 7feeaad9d640 1 --2- 172.21.15.42:0/2147539457 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5605e081b1b0 0x5605e0bc5370 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7feea4004af0 tx=0x7feea4032e40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e0a22c5eeb98e75e server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:06:45.275 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.272+0000 7fee98ff9640 1 -- 172.21.15.42:0/2147539457 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feea4031be0 con 0x5605e081b1b0 2024-08-30T09:06:45.275 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.272+0000 7fee98ff9640 1 -- 172.21.15.42:0/2147539457 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feea40327c0 con 0x5605e081b1b0 2024-08-30T09:06:45.275 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.272+0000 7fee98ff9640 1 -- 172.21.15.42:0/2147539457 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feea403f440 con 0x5605e081b1b0 2024-08-30T09:06:45.275 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.272+0000 7feeaad9d640 1 --2- 172.21.15.42:0/759151790 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fee940b4f20 0x7fee940b7430 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7feea4048000 tx=0x7feea404a000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:06:45.275 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.273+0000 7feeb79eb740 1 -- 172.21.15.42:0/2147539457 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5605e081b1b0 msgr2=0x5605e0bc5370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:06:45.276 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.273+0000 7feeb79eb740 1 --2- 172.21.15.42:0/2147539457 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5605e081b1b0 0x5605e0bc5370 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7feea4004af0 tx=0x7feea4032e40 comp rx=0 tx=0).stop 2024-08-30T09:06:45.277 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.275+0000 7feeb79eb740 1 -- 172.21.15.42:0/2147539457 shutdown_connections 2024-08-30T09:06:45.278 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.275+0000 7feeb79eb740 1 --2- 172.21.15.42:0/2147539457 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5605e07fa4d0 0x5605e0bc6530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:45.278 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.275+0000 7feeb79eb740 1 --2- 172.21.15.42:0/2147539457 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5605e07f22f0 0x5605e0bc59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:45.278 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.275+0000 7feeb79eb740 1 --2- 172.21.15.42:0/2147539457 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5605e081b1b0 0x5605e0bc5370 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:45.278 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.276+0000 7feeb79eb740 1 -- 172.21.15.42:0/2147539457 >> 172.21.15.42:0/2147539457 conn(0x5605e080f300 msgr2=0x5605e0bc0e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:06:45.279 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.277+0000 7feeb79eb740 1 -- 172.21.15.42:0/2147539457 shutdown_connections 2024-08-30T09:06:45.284 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.282+0000 7feeb79eb740 1 -- 172.21.15.42:0/2147539457 wait complete. 2024-08-30T09:06:45.285 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.283+0000 7feeb79eb740 1 -- start start 2024-08-30T09:06:45.285 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.284+0000 7feeb79eb740 1 Processor -- start 2024-08-30T09:06:45.290 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.288+0000 7feeb79eb740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5605e07f22f0 0x5605e0bbc5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:45.290 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.288+0000 7feeb79eb740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5605e07fa4d0 0x5605e0bbcc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:45.290 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.288+0000 7feeb79eb740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5605e081b1b0 0x5605e0bbd7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:45.290 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.288+0000 7feeb79eb740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x5605e0a5cab0 con 0x5605e07f22f0 2024-08-30T09:06:45.290 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.288+0000 7feeb79eb740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5605e0aaa680 con 0x5605e081b1b0 2024-08-30T09:06:45.290 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.288+0000 7feeb79eb740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5605e0bc6f90 con 0x5605e07fa4d0 2024-08-30T09:06:45.291 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.288+0000 7feeb79eb740 10 client.0 _ll_register_callbacks cb 0x5605e0832a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:06:45.293 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.290+0000 7feeaad9d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5605e081b1b0 0x5605e0bbd7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:06:45.293 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.290+0000 7feeaad9d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5605e081b1b0 0x5605e0bbd7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:44666/0 (socket says 172.21.15.42:44666) 2024-08-30T09:06:45.294 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.290+0000 7feeaad9d640 1 -- 172.21.15.42:0/1764847299 learned_addr learned my addr 172.21.15.42:0/1764847299 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:06:45.294 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.290+0000 7feea9d9b640 1 --2- 172.21.15.42:0/1764847299 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5605e07fa4d0 0x5605e0bbcc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:06:45.300 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.294+0000 7feeaa59c640 1 --2- 172.21.15.42:0/1764847299 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5605e07f22f0 0x5605e0bbc5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:06:45.300 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.294+0000 7feeaad9d640 1 -- 172.21.15.42:0/1764847299 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5605e07fa4d0 msgr2=0x5605e0bbcc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:06:45.301 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.294+0000 7feeaad9d640 1 --2- 172.21.15.42:0/1764847299 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5605e07fa4d0 0x5605e0bbcc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:45.301 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.294+0000 7feeaad9d640 1 -- 172.21.15.42:0/1764847299 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5605e07f22f0 msgr2=0x5605e0bbc5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:06:45.301 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.294+0000 7feeaad9d640 1 --2- 172.21.15.42:0/1764847299 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5605e07f22f0 0x5605e0bbc5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:45.301 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.294+0000 7feeaad9d640 1 -- 172.21.15.42:0/1764847299 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5605e08ff620 con 0x5605e081b1b0 2024-08-30T09:06:45.301 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.295+0000 7feeaad9d640 1 --2- 172.21.15.42:0/1764847299 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5605e081b1b0 0x5605e0bbd7d0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7feea4030910 tx=0x7feea4030940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=40757636bd73166 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:06:45.301 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.295+0000 7fee91ffb640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-30T09:06:45.303 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.299+0000 7fee91ffb640 1 -- 172.21.15.42:0/1764847299 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feea4031be0 con 0x5605e081b1b0 2024-08-30T09:06:45.303 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.299+0000 7fee91ffb640 1 -- 172.21.15.42:0/1764847299 <== mon.2 v2:172.21.15.167:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7feea4031d80 con 0x5605e081b1b0 2024-08-30T09:06:45.303 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.300+0000 7feeb79eb740 1 -- 172.21.15.42:0/1764847299 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feea40327c0 con 0x5605e081b1b0 2024-08-30T09:06:45.312 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.307+0000 7fee91ffb640 1 -- 172.21.15.42:0/1764847299 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7feea4005bd0 con 0x5605e081b1b0 2024-08-30T09:06:45.313 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.308+0000 7feeb79eb740 10 client.26240 fetch_fsmap learned FSMap version 33 2024-08-30T09:06:45.313 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.308+0000 7feeb79eb740 1 -- 172.21.15.42:0/1764847299 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5605e0bcec00 con 0x5605e081b1b0 2024-08-30T09:06:45.317 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.312+0000 7fee91ffb640 1 -- 172.21.15.42:0/1764847299 <== mon.2 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feea40327c0 con 0x5605e081b1b0 2024-08-30T09:06:45.317 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.312+0000 7fee91ffb640 1 -- 172.21.15.42:0/1764847299 <== mon.2 v2:172.21.15.167:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7feea403c5a0 con 0x5605e081b1b0 2024-08-30T09:06:45.317 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.312+0000 7feeb79eb740 10 client.26240 fetch_fsmap finished waiting for FSMap version 33 2024-08-30T09:06:45.317 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.312+0000 7feeb79eb740 10 client.26240 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-30T09:06:45.318 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.312+0000 7feeb79eb740 20 client.26240 populate_metadata read hostname 'smithi042' 2024-08-30T09:06:45.318 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.312+0000 7feeb79eb740 1 --2- 172.21.15.42:0/1764847299 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x5605e07d9d80 0x5605e0bd2500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:45.318 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.312+0000 7feeb79eb740 4 client.26240 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-30T09:06:45.318 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.312+0000 7feeb79eb740 1 -- 172.21.15.42:0/1764847299 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5605e0b35f50 con 0x5605e07d9d80 2024-08-30T09:06:45.321 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.317+0000 7feeaa59c640 1 --2- 172.21.15.42:0/1764847299 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x5605e07d9d80 0x5605e0bd2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:06:45.333 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.331+0000 7feeaa59c640 1 --2- 172.21.15.42:0/1764847299 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x5605e07d9d80 0x5605e0bd2500 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=41b71b7531fc18d3 server_cookie=34f91788470c6098 in_seq=0 out_seq=0 2024-08-30T09:06:45.333 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.331+0000 7fee91ffb640 10 client.26240 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:06:45.336 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.332+0000 7fee91ffb640 1 -- 172.21.15.42:0/1764847299 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5605e0b35f50 con 0x5605e07d9d80 2024-08-30T09:06:45.336 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.332+0000 7fee91ffb640 10 client.26240 handle_command_reply: tid=0 2024-08-30T09:06:45.336 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.332+0000 7feeb79eb740 1 client.26240 shutdown 2024-08-30T09:06:45.340 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.338+0000 7feeb79eb740 1 -- 172.21.15.42:0/1764847299 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5605e081b1b0 msgr2=0x5605e0bbd7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:06:45.340 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.338+0000 7feeb79eb740 1 --2- 172.21.15.42:0/1764847299 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5605e081b1b0 0x5605e0bbd7d0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7feea4030910 tx=0x7feea4030940 comp rx=0 tx=0).stop 2024-08-30T09:06:45.347 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.345+0000 7feeb79eb740 1 -- 172.21.15.42:0/1764847299 shutdown_connections 2024-08-30T09:06:45.347 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.345+0000 7feeb79eb740 1 --2- 172.21.15.42:0/1764847299 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x5605e07d9d80 0x5605e0bd2500 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:45.347 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.345+0000 7feeb79eb740 1 --2- 172.21.15.42:0/1764847299 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5605e07fa4d0 0x5605e0bbcc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:45.347 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.345+0000 7feeb79eb740 1 --2- 172.21.15.42:0/1764847299 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5605e081b1b0 0x5605e0bbd7d0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:45.347 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.345+0000 7feeb79eb740 1 --2- 172.21.15.42:0/1764847299 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5605e07f22f0 0x5605e0bbc5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:45.347 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.345+0000 7feeb79eb740 1 -- 172.21.15.42:0/1764847299 >> 172.21.15.42:0/1764847299 conn(0x5605e080f300 msgr2=0x5605e0bbac50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:06:45.348 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.345+0000 7fee91ffb640 0 client.26240 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:06:45.348 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.346+0000 7feeb79eb740 1 -- 172.21.15.42:0/1764847299 shutdown_connections 2024-08-30T09:06:45.348 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.346+0000 7feeb79eb740 1 -- 172.21.15.42:0/1764847299 wait complete. 2024-08-30T09:06:45.352 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.351+0000 7feeb79eb740 20 client.26240 trim_cache size 0 max 16384 2024-08-30T09:06:45.387 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.385+0000 7feeb79eb740 1 Processor -- start 2024-08-30T09:06:45.388 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.386+0000 7feeb79eb740 1 -- start start 2024-08-30T09:06:45.388 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.386+0000 7feeb79eb740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5605e07f22f0 0x5605e0bc0740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:45.388 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.386+0000 7feeb79eb740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5605e07fa4d0 0x5605e0bc0c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:45.388 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.386+0000 7feeb79eb740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5605e081b1b0 0x5605e0bc9d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:45.389 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.387+0000 7feea9d9b640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5605e07fa4d0 0x5605e0bc0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:06:45.390 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.387+0000 7feea9d9b640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5605e07fa4d0 0x5605e0bc0c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:44682/0 (socket says 172.21.15.42:44682) 2024-08-30T09:06:45.390 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.387+0000 7feeaad9d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5605e081b1b0 0x5605e0bc9d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:06:45.390 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.387+0000 7feea9d9b640 1 -- 172.21.15.42:0/2323832839 learned_addr learned my addr 172.21.15.42:0/2323832839 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:06:45.390 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.387+0000 7feeb79eb740 1 -- 172.21.15.42:0/2323832839 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7feeac1b5460 con 0x5605e07f22f0 2024-08-30T09:06:45.390 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.387+0000 7feeb79eb740 1 -- 172.21.15.42:0/2323832839 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7feeac1b2cd0 con 0x5605e07fa4d0 2024-08-30T09:06:45.390 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.387+0000 7feeb79eb740 1 -- 172.21.15.42:0/2323832839 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7feeac1b0b20 con 0x5605e081b1b0 2024-08-30T09:06:45.394 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.388+0000 7feeaa59c640 1 --2- 172.21.15.42:0/2323832839 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5605e07f22f0 0x5605e0bc0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:06:45.394 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.390+0000 7feeaad9d640 1 -- 172.21.15.42:0/2323832839 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5605e07fa4d0 msgr2=0x5605e0bc0c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:06:45.395 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.390+0000 7feeaad9d640 1 --2- 172.21.15.42:0/2323832839 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5605e07fa4d0 0x5605e0bc0c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:45.395 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.390+0000 7feeaad9d640 1 -- 172.21.15.42:0/2323832839 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5605e07f22f0 msgr2=0x5605e0bc0740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:06:45.395 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.390+0000 7feeaad9d640 1 --2- 172.21.15.42:0/2323832839 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5605e07f22f0 0x5605e0bc0740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:45.395 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.390+0000 7feeaad9d640 1 -- 172.21.15.42:0/2323832839 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5605e0bb6800 con 0x5605e081b1b0 2024-08-30T09:06:45.395 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.391+0000 7feeaad9d640 1 --2- 172.21.15.42:0/2323832839 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5605e081b1b0 0x5605e0bc9d20 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7feea4032a50 tx=0x7feea4032ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1d18d8619fe911d4 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:06:45.395 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.391+0000 7fee93fff640 1 -- 172.21.15.42:0/2323832839 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feea403f440 con 0x5605e081b1b0 2024-08-30T09:06:45.395 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.391+0000 7fee93fff640 1 -- 172.21.15.42:0/2323832839 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feea400a910 con 0x5605e081b1b0 2024-08-30T09:06:45.396 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.393+0000 7feeb79eb740 1 -- 172.21.15.42:0/2323832839 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5605e081b1b0 msgr2=0x5605e0bc9d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:06:45.396 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.393+0000 7feeb79eb740 1 --2- 172.21.15.42:0/2323832839 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5605e081b1b0 0x5605e0bc9d20 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7feea4032a50 tx=0x7feea4032ab0 comp rx=0 tx=0).stop 2024-08-30T09:06:45.396 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.393+0000 7feeb79eb740 1 -- 172.21.15.42:0/2323832839 shutdown_connections 2024-08-30T09:06:45.396 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.393+0000 7feeb79eb740 1 --2- 172.21.15.42:0/2323832839 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5605e081b1b0 0x5605e0bc9d20 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:45.396 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.393+0000 7feeb79eb740 1 --2- 172.21.15.42:0/2323832839 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5605e07fa4d0 0x5605e0bc0c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:45.396 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.393+0000 7feeb79eb740 1 --2- 172.21.15.42:0/2323832839 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5605e07f22f0 0x5605e0bc0740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:45.396 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.393+0000 7feeb79eb740 1 -- 172.21.15.42:0/2323832839 >> 172.21.15.42:0/2323832839 conn(0x5605e080f300 msgr2=0x5605e0bce0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:06:45.396 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.394+0000 7feeb79eb740 1 -- 172.21.15.42:0/2323832839 shutdown_connections 2024-08-30T09:06:45.397 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.394+0000 7feeb79eb740 1 -- 172.21.15.42:0/2323832839 wait complete. 2024-08-30T09:06:45.398 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.394+0000 7feeb79eb740 1 -- start start 2024-08-30T09:06:45.398 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.394+0000 7feeb79eb740 1 Processor -- start 2024-08-30T09:06:45.398 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.395+0000 7feeb79eb740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5605e07f22f0 0x5605e0bd5d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:45.398 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.395+0000 7feeb79eb740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5605e07fa4d0 0x5605e0bd8440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:45.398 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.395+0000 7feeb79eb740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5605e081b1b0 0x5605e0c52530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:45.398 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.395+0000 7feeb79eb740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7feeac1b1190 con 0x5605e07fa4d0 2024-08-30T09:06:45.398 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.395+0000 7feeb79eb740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7feeac1b2600 con 0x5605e081b1b0 2024-08-30T09:06:45.398 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.395+0000 7feeb79eb740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7feeac1b1860 con 0x5605e07f22f0 2024-08-30T09:06:45.398 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.395+0000 7feeb79eb740 10 client.0 _ll_register_callbacks cb 0x7feea000bb20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:06:45.398 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.395+0000 7feea9d9b640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5605e07fa4d0 0x5605e0bd8440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:06:45.398 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.395+0000 7feea9d9b640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5605e07fa4d0 0x5605e0bd8440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:47070/0 (socket says 172.21.15.42:47070) 2024-08-30T09:06:45.398 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.395+0000 7feea9d9b640 1 -- 172.21.15.42:0/1039254440 learned_addr learned my addr 172.21.15.42:0/1039254440 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:06:45.398 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.395+0000 7feeaad9d640 1 --2- 172.21.15.42:0/1039254440 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5605e081b1b0 0x5605e0c52530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:06:45.398 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.395+0000 7feeaa59c640 1 --2- 172.21.15.42:0/1039254440 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5605e07f22f0 0x5605e0bd5d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:06:45.399 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.396+0000 7feea9d9b640 1 -- 172.21.15.42:0/1039254440 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5605e07f22f0 msgr2=0x5605e0bd5d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:06:45.399 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.396+0000 7feea9d9b640 1 --2- 172.21.15.42:0/1039254440 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5605e07f22f0 0x5605e0bd5d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:45.399 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.396+0000 7feea9d9b640 1 -- 172.21.15.42:0/1039254440 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5605e081b1b0 msgr2=0x5605e0c52530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:06:45.399 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.396+0000 7feea9d9b640 1 --2- 172.21.15.42:0/1039254440 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5605e081b1b0 0x5605e0c52530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:45.399 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.396+0000 7feea9d9b640 1 -- 172.21.15.42:0/1039254440 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5605e0bbb520 con 0x5605e07fa4d0 2024-08-30T09:06:45.399 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.396+0000 7feea9d9b640 1 --2- 172.21.15.42:0/1039254440 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5605e07fa4d0 0x5605e0bd8440 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7feea0010b80 tx=0x7feea0012450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b4a895fe7d92994 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:06:45.400 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.398+0000 7fee91ffb640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-08-30T09:06:45.400 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.398+0000 7fee91ffb640 1 -- 172.21.15.42:0/1039254440 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feea0021020 con 0x5605e07fa4d0 2024-08-30T09:06:45.400 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.398+0000 7fee91ffb640 1 -- 172.21.15.42:0/1039254440 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7feea001d910 con 0x5605e07fa4d0 2024-08-30T09:06:45.400 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.398+0000 7feeb79eb740 1 -- 172.21.15.42:0/1039254440 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5605e0b319f0 con 0x5605e07fa4d0 2024-08-30T09:06:45.405 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.402+0000 7fee91ffb640 1 -- 172.21.15.42:0/1039254440 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7feea001db10 con 0x5605e07fa4d0 2024-08-30T09:06:45.407 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.404+0000 7feeb79eb740 10 client.16497 fetch_fsmap learned FSMap version 33 2024-08-30T09:06:45.407 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.404+0000 7feeb79eb740 1 -- 172.21.15.42:0/1039254440 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5605e0c53d90 con 0x5605e07fa4d0 2024-08-30T09:06:45.408 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.406+0000 7fee91ffb640 1 -- 172.21.15.42:0/1039254440 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5605e0b319f0 con 0x5605e07fa4d0 2024-08-30T09:06:45.408 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.406+0000 7fee91ffb640 1 -- 172.21.15.42:0/1039254440 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7feea002e020 con 0x5605e07fa4d0 2024-08-30T09:06:45.408 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.406+0000 7feeb79eb740 10 client.16497 fetch_fsmap finished waiting for FSMap version 33 2024-08-30T09:06:45.408 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.406+0000 7feeb79eb740 10 client.16497 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-30T09:06:45.408 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.406+0000 7feeb79eb740 20 client.16497 populate_metadata read hostname 'smithi042' 2024-08-30T09:06:45.409 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.406+0000 7feeb79eb740 1 --2- 172.21.15.42:0/1039254440 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x5605e07af1c0 0x5605e0c567e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:45.409 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.406+0000 7feeb79eb740 4 client.16497 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-30T09:06:45.409 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.406+0000 7feeb79eb740 1 -- 172.21.15.42:0/1039254440 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5605e0b2f130 con 0x5605e07af1c0 2024-08-30T09:06:45.410 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.408+0000 7feeaa59c640 1 --2- 172.21.15.42:0/1039254440 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x5605e07af1c0 0x5605e0c567e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:06:45.410 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.408+0000 7feeaa59c640 1 --2- 172.21.15.42:0/1039254440 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x5605e07af1c0 0x5605e0c567e0 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b9e5de045d1c4d9 server_cookie=8e46aad849dc8991 in_seq=0 out_seq=0 2024-08-30T09:06:45.410 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.408+0000 7fee91ffb640 10 client.16497 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:06:45.466 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.464+0000 7fee91ffb640 1 -- 172.21.15.42:0/1039254440 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5605e0b2f130 con 0x5605e07af1c0 2024-08-30T09:06:45.466 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.464+0000 7fee91ffb640 10 client.16497 handle_command_reply: tid=0 2024-08-30T09:06:45.466 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.464+0000 7feeb79eb740 1 client.16497 shutdown 2024-08-30T09:06:45.467 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.464+0000 7feeb79eb740 1 -- 172.21.15.42:0/1039254440 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5605e07fa4d0 msgr2=0x5605e0bd8440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:06:45.467 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.464+0000 7feeb79eb740 1 --2- 172.21.15.42:0/1039254440 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5605e07fa4d0 0x5605e0bd8440 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7feea0010b80 tx=0x7feea0012450 comp rx=0 tx=0).stop 2024-08-30T09:06:45.469 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.467+0000 7feeb79eb740 1 -- 172.21.15.42:0/1039254440 shutdown_connections 2024-08-30T09:06:45.469 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.467+0000 7feeb79eb740 1 --2- 172.21.15.42:0/1039254440 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x5605e07af1c0 0x5605e0c567e0 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:45.469 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.467+0000 7feeb79eb740 1 --2- 172.21.15.42:0/1039254440 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5605e081b1b0 0x5605e0c52530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:45.469 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.467+0000 7feeb79eb740 1 --2- 172.21.15.42:0/1039254440 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5605e07fa4d0 0x5605e0bd8440 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:45.469 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.467+0000 7feeb79eb740 1 --2- 172.21.15.42:0/1039254440 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5605e07f22f0 0x5605e0bd5d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:45.469 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.467+0000 7feeb79eb740 1 -- 172.21.15.42:0/1039254440 >> 172.21.15.42:0/1039254440 conn(0x5605e080f300 msgr2=0x5605e0bc0740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:06:45.469 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.467+0000 7fee91ffb640 0 client.16497 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:06:45.470 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.468+0000 7feeb79eb740 1 -- 172.21.15.42:0/1039254440 shutdown_connections 2024-08-30T09:06:45.470 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.468+0000 7feeb79eb740 1 -- 172.21.15.42:0/1039254440 wait complete. 2024-08-30T09:06:45.470 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.468+0000 7feeb79eb740 20 client.16497 trim_cache size 0 max 16384 2024-08-30T09:06:45.470 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-08-30T09:06:45.470 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2024-08-30T09:06:45.470 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-08-30T09:06:45.470 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-08-30T09:06:45.471 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.469+0000 7feeb0a7e640 1 -- 172.21.15.42:0/759151790 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fee940b4f20 msgr2=0x7fee940b7430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:06:45.471 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.469+0000 7feeb0a7e640 1 --2- 172.21.15.42:0/759151790 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fee940b4f20 0x7fee940b7430 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7feea4048000 tx=0x7feea404a000 comp rx=0 tx=0).stop 2024-08-30T09:06:45.471 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.469+0000 7feeb0a7e640 1 -- 172.21.15.42:0/759151790 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feeac10a9f0 msgr2=0x7feeac1a24e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:06:45.471 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.469+0000 7feeb0a7e640 1 --2- 172.21.15.42:0/759151790 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feeac10a9f0 0x7feeac1a24e0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fee9c00e9f0 tx=0x7fee9c00edf0 comp rx=0 tx=0).stop 2024-08-30T09:06:45.471 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.469+0000 7feeb0a7e640 1 -- 172.21.15.42:0/759151790 shutdown_connections 2024-08-30T09:06:45.472 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.469+0000 7feeb0a7e640 1 --2- 172.21.15.42:0/759151790 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fee940b4f20 0x7fee940b7430 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:45.472 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.469+0000 7feeb0a7e640 1 --2- 172.21.15.42:0/759151790 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feeac110000 0x7feeac1a3700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:45.472 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.469+0000 7feeb0a7e640 1 --2- 172.21.15.42:0/759151790 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7feeac10d3e0 0x7feeac1a2b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:45.472 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.469+0000 7feeb0a7e640 1 --2- 172.21.15.42:0/759151790 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feeac10a9f0 0x7feeac1a24e0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:45.472 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.469+0000 7feeb0a7e640 1 -- 172.21.15.42:0/759151790 >> 172.21.15.42:0/759151790 conn(0x7feeac0705b0 msgr2=0x7feeac0729d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:06:45.472 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.469+0000 7feeb0a7e640 1 -- 172.21.15.42:0/759151790 shutdown_connections 2024-08-30T09:06:45.472 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:45.469+0000 7feeb0a7e640 1 -- 172.21.15.42:0/759151790 wait complete. 2024-08-30T09:06:45.953 INFO:teuthology.orchestra.run.smithi042.stderr:progress: 360.0 s, 677.2 tps, lat 148.650 ms stddev 168.946 2024-08-30T09:06:46.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:06:46 smithi190 ceph-mon[22001]: pgmap v553: 129 pgs: 129 active+clean; 1.4 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 9.8 MiB/s wr, 586 op/s 2024-08-30T09:06:46.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:06:46 smithi167 ceph-mon[22063]: pgmap v553: 129 pgs: 129 active+clean; 1.4 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 9.8 MiB/s wr, 586 op/s 2024-08-30T09:06:46.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:06:46 smithi042 ceph-mon[18652]: pgmap v553: 129 pgs: 129 active+clean; 1.4 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 9.8 MiB/s wr, 586 op/s 2024-08-30T09:06:48.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:06:48 smithi190 ceph-mon[22001]: pgmap v554: 129 pgs: 129 active+clean; 1.4 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 9.6 MiB/s wr, 646 op/s 2024-08-30T09:06:48.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:06:48 smithi190 ceph-mon[22001]: scrub summary: idle 2024-08-30T09:06:48.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:06:48 smithi190 ceph-mon[22001]: scrub summary: idle 2024-08-30T09:06:48.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:06:48 smithi167 ceph-mon[22063]: pgmap v554: 129 pgs: 129 active+clean; 1.4 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 9.6 MiB/s wr, 646 op/s 2024-08-30T09:06:48.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:06:48 smithi167 ceph-mon[22063]: scrub summary: idle 2024-08-30T09:06:48.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:06:48 smithi167 ceph-mon[22063]: scrub summary: idle 2024-08-30T09:06:48.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:06:48 smithi042 ceph-mon[18652]: pgmap v554: 129 pgs: 129 active+clean; 1.4 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 9.6 MiB/s wr, 646 op/s 2024-08-30T09:06:48.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:06:48 smithi042 ceph-mon[18652]: scrub summary: idle 2024-08-30T09:06:48.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:06:48 smithi042 ceph-mon[18652]: scrub summary: idle 2024-08-30T09:06:49.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:06:49 smithi190 ceph-mon[22001]: scrub summary: idle 2024-08-30T09:06:49.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:06:49 smithi190 ceph-mon[22001]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-30T09:06:49.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:06:49 smithi167 ceph-mon[22063]: scrub summary: idle 2024-08-30T09:06:49.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:06:49 smithi167 ceph-mon[22063]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-30T09:06:49.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:06:49 smithi042 ceph-mon[18652]: scrub summary: idle 2024-08-30T09:06:49.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:06:49 smithi042 ceph-mon[18652]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-30T09:06:50.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:06:50 smithi190 ceph-mon[22001]: pgmap v555: 129 pgs: 129 active+clean; 1.4 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 10 MiB/s wr, 641 op/s 2024-08-30T09:06:50.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:06:50 smithi167 ceph-mon[22063]: pgmap v555: 129 pgs: 129 active+clean; 1.4 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 10 MiB/s wr, 641 op/s 2024-08-30T09:06:50.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:06:50 smithi042 ceph-mon[18652]: pgmap v555: 129 pgs: 129 active+clean; 1.4 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 10 MiB/s wr, 641 op/s 2024-08-30T09:06:50.953 INFO:teuthology.orchestra.run.smithi042.stderr:progress: 365.0 s, 778.6 tps, lat 127.738 ms stddev 150.813 2024-08-30T09:06:52.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:06:52 smithi190 ceph-mon[22001]: pgmap v556: 129 pgs: 129 active+clean; 1.4 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 8.5 MiB/s wr, 560 op/s 2024-08-30T09:06:52.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:06:52 smithi167 ceph-mon[22063]: pgmap v556: 129 pgs: 129 active+clean; 1.4 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 8.5 MiB/s wr, 560 op/s 2024-08-30T09:06:52.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:06:52 smithi042 ceph-mon[18652]: pgmap v556: 129 pgs: 129 active+clean; 1.4 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 8.5 MiB/s wr, 560 op/s 2024-08-30T09:06:54.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:06:54 smithi190 ceph-mon[22001]: pgmap v557: 129 pgs: 129 active+clean; 1.4 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 11 MiB/s wr, 671 op/s 2024-08-30T09:06:54.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:06:54 smithi167 ceph-mon[22063]: pgmap v557: 129 pgs: 129 active+clean; 1.4 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 11 MiB/s wr, 671 op/s 2024-08-30T09:06:54.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:06:54 smithi042 ceph-mon[18652]: pgmap v557: 129 pgs: 129 active+clean; 1.4 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 11 MiB/s wr, 671 op/s 2024-08-30T09:06:55.953 INFO:teuthology.orchestra.run.smithi042.stderr:progress: 370.0 s, 670.6 tps, lat 148.975 ms stddev 173.018 2024-08-30T09:06:56.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:06:56 smithi190 ceph-mon[22001]: pgmap v558: 129 pgs: 129 active+clean; 1.4 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 8.7 MiB/s wr, 516 op/s 2024-08-30T09:06:56.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:06:56 smithi167 ceph-mon[22063]: pgmap v558: 129 pgs: 129 active+clean; 1.4 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 8.7 MiB/s wr, 516 op/s 2024-08-30T09:06:56.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:06:56 smithi042 ceph-mon[18652]: pgmap v558: 129 pgs: 129 active+clean; 1.4 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 8.7 MiB/s wr, 516 op/s 2024-08-30T09:06:58.626 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.622+0000 7f13fd52b640 1 -- 172.21.15.42:0/2038363064 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f13f8102340 msgr2=0x7f13f81027b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:06:58.626 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.622+0000 7f13fd52b640 1 --2- 172.21.15.42:0/2038363064 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f13f8102340 0x7f13f81027b0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f13e8009900 tx=0x7f13e802a400 comp rx=0 tx=0).stop 2024-08-30T09:06:58.626 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.623+0000 7f13fd52b640 1 -- 172.21.15.42:0/2038363064 shutdown_connections 2024-08-30T09:06:58.626 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.623+0000 7f13fd52b640 1 --2- 172.21.15.42:0/2038363064 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f13f8103d60 0x7f13f806a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:58.626 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.623+0000 7f13fd52b640 1 --2- 172.21.15.42:0/2038363064 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f13f81033c0 0x7f13f81037d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:58.627 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.623+0000 7f13fd52b640 1 --2- 172.21.15.42:0/2038363064 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f13f8102340 0x7f13f81027b0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:58.627 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.623+0000 7f13fd52b640 1 -- 172.21.15.42:0/2038363064 >> 172.21.15.42:0/2038363064 conn(0x7f13f80fdc80 msgr2=0x7f13f8100130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:06:58.629 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.626+0000 7f13fd52b640 1 -- 172.21.15.42:0/2038363064 shutdown_connections 2024-08-30T09:06:58.629 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.626+0000 7f13fd52b640 1 -- 172.21.15.42:0/2038363064 wait complete. 2024-08-30T09:06:58.632 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.627+0000 7f13fd52b640 1 Processor -- start 2024-08-30T09:06:58.633 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.628+0000 7f13fd52b640 1 -- start start 2024-08-30T09:06:58.633 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.628+0000 7f13fd52b640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f13f8102340 0x7f13f81a02e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:58.633 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.628+0000 7f13fd52b640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f13f81033c0 0x7f13f81a0950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:58.633 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.628+0000 7f13fd52b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f13f8103d60 0x7f13f81a1500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:58.633 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.628+0000 7f13fd52b640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f13f810cf30 con 0x7f13f8102340 2024-08-30T09:06:58.633 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.628+0000 7f13fd52b640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f13f810cdb0 con 0x7f13f81033c0 2024-08-30T09:06:58.633 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.628+0000 7f13fd52b640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f13f810d0b0 con 0x7f13f8103d60 2024-08-30T09:06:58.633 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.628+0000 7f13f6ffd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f13f8102340 0x7f13f81a02e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:06:58.633 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.628+0000 7f13f6ffd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f13f8102340 0x7f13f81a02e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:32888/0 (socket says 172.21.15.42:32888) 2024-08-30T09:06:58.633 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.628+0000 7f13f6ffd640 1 -- 172.21.15.42:0/2556672058 learned_addr learned my addr 172.21.15.42:0/2556672058 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:06:58.633 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.628+0000 7f13effff640 1 --2- 172.21.15.42:0/2556672058 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f13f81033c0 0x7f13f81a0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:06:58.633 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.628+0000 7f13f77fe640 1 --2- 172.21.15.42:0/2556672058 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f13f8103d60 0x7f13f81a1500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:06:58.634 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.628+0000 7f13f77fe640 1 -- 172.21.15.42:0/2556672058 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f13f81033c0 msgr2=0x7f13f81a0950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:06:58.634 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.629+0000 7f13f77fe640 1 --2- 172.21.15.42:0/2556672058 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f13f81033c0 0x7f13f81a0950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:58.634 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.629+0000 7f13f77fe640 1 -- 172.21.15.42:0/2556672058 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f13f8102340 msgr2=0x7f13f81a02e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:06:58.634 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.629+0000 7f13f77fe640 1 --2- 172.21.15.42:0/2556672058 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f13f8102340 0x7f13f81a02e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:58.634 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.629+0000 7f13f77fe640 1 -- 172.21.15.42:0/2556672058 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13f81a1d20 con 0x7f13f8103d60 2024-08-30T09:06:58.634 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.629+0000 7f13f77fe640 1 --2- 172.21.15.42:0/2556672058 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f13f8103d60 0x7f13f81a1500 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f13f000eb20 tx=0x7f13f000c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:06:58.634 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.629+0000 7f13f4ff9640 1 -- 172.21.15.42:0/2556672058 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13f00039c0 con 0x7f13f8103d60 2024-08-30T09:06:58.634 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.629+0000 7f13f4ff9640 1 -- 172.21.15.42:0/2556672058 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f13f0003b60 con 0x7f13f8103d60 2024-08-30T09:06:58.634 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.629+0000 7f13fd52b640 1 -- 172.21.15.42:0/2556672058 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13f81a2010 con 0x7f13f8103d60 2024-08-30T09:06:58.634 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.629+0000 7f13f4ff9640 1 -- 172.21.15.42:0/2556672058 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13f0010690 con 0x7f13f8103d60 2024-08-30T09:06:58.634 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.629+0000 7f13fd52b640 1 -- 172.21.15.42:0/2556672058 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13f81a22c0 con 0x7f13f8103d60 2024-08-30T09:06:58.640 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.633+0000 7f13f4ff9640 1 -- 172.21.15.42:0/2556672058 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7f13f00108f0 con 0x7f13f8103d60 2024-08-30T09:06:58.640 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.633+0000 7f1404498740 1 Processor -- start 2024-08-30T09:06:58.640 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.633+0000 7f1404498740 1 -- start start 2024-08-30T09:06:58.640 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.634+0000 7f1404498740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55eb704721b0 0x55eb70818180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:58.640 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.634+0000 7f1404498740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55eb704492f0 0x55eb708187f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:58.640 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.634+0000 7f1404498740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eb704514d0 0x55eb70819340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:58.640 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.634+0000 7f1404498740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55eb7078b000 con 0x55eb704492f0 2024-08-30T09:06:58.640 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.634+0000 7f1404498740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55eb7077a590 con 0x55eb704514d0 2024-08-30T09:06:58.641 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.634+0000 7f1404498740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55eb7077e540 con 0x55eb704721b0 2024-08-30T09:06:58.641 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.634+0000 7f13effff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55eb704721b0 0x55eb70818180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:06:58.641 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.634+0000 7f13effff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55eb704721b0 0x55eb70818180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:36596/0 (socket says 172.21.15.42:36596) 2024-08-30T09:06:58.641 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.634+0000 7f13effff640 1 -- 172.21.15.42:0/3367631505 learned_addr learned my addr 172.21.15.42:0/3367631505 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:06:58.641 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.634+0000 7f13effff640 1 --2- 172.21.15.42:0/3367631505 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eb704514d0 0x55eb70819340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:06:58.641 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.634+0000 7f13f6ffd640 1 --2- 172.21.15.42:0/3367631505 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55eb704492f0 0x55eb708187f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:06:58.641 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.634+0000 7f13effff640 1 -- 172.21.15.42:0/3367631505 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eb704514d0 msgr2=0x55eb70819340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:06:58.641 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.634+0000 7f13effff640 1 --2- 172.21.15.42:0/3367631505 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eb704514d0 0x55eb70819340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:58.641 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.634+0000 7f13effff640 1 -- 172.21.15.42:0/3367631505 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55eb704492f0 msgr2=0x55eb708187f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:06:58.641 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.634+0000 7f13effff640 1 --2- 172.21.15.42:0/3367631505 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55eb704492f0 0x55eb708187f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:58.641 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.634+0000 7f13effff640 1 -- 172.21.15.42:0/3367631505 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb704dd790 con 0x55eb704721b0 2024-08-30T09:06:58.641 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.634+0000 7f13effff640 1 --2- 172.21.15.42:0/3367631505 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55eb704721b0 0x55eb70818180 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f13e0012820 tx=0x7f13e000b7e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d065e0505f3f6f06 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:06:58.641 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.635+0000 7f13f4ff9640 1 --2- 172.21.15.42:0/2556672058 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f13e40b4b70 0x7f13e40b7080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:58.642 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.635+0000 7f13ed7fa640 1 -- 172.21.15.42:0/3367631505 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13e0012da0 con 0x55eb704721b0 2024-08-30T09:06:58.642 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.635+0000 7f13ed7fa640 1 -- 172.21.15.42:0/3367631505 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f13e0010c20 con 0x55eb704721b0 2024-08-30T09:06:58.642 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.635+0000 7f13ed7fa640 1 -- 172.21.15.42:0/3367631505 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13e001eaa0 con 0x55eb704721b0 2024-08-30T09:06:58.642 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.635+0000 7f13f4ff9640 1 -- 172.21.15.42:0/2556672058 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f13f00e5bb0 con 0x7f13f8103d60 2024-08-30T09:06:58.642 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.635+0000 7f13f6ffd640 1 --2- 172.21.15.42:0/2556672058 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f13e40b4b70 0x7f13e40b7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:06:58.642 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.636+0000 7f1404498740 1 -- 172.21.15.42:0/3367631505 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55eb704721b0 msgr2=0x55eb70818180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:06:58.642 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.636+0000 7f13f6ffd640 1 --2- 172.21.15.42:0/2556672058 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f13e40b4b70 0x7f13e40b7080 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f13e8000c00 tx=0x7f13e80095e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:06:58.642 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.636+0000 7f1404498740 1 --2- 172.21.15.42:0/3367631505 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55eb704721b0 0x55eb70818180 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f13e0012820 tx=0x7f13e000b7e0 comp rx=0 tx=0).stop 2024-08-30T09:06:58.642 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.636+0000 7f1404498740 1 -- 172.21.15.42:0/3367631505 shutdown_connections 2024-08-30T09:06:58.642 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.636+0000 7f1404498740 1 --2- 172.21.15.42:0/3367631505 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eb704514d0 0x55eb70819340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:58.642 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.636+0000 7f1404498740 1 --2- 172.21.15.42:0/3367631505 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55eb704492f0 0x55eb708187f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:58.642 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.636+0000 7f1404498740 1 --2- 172.21.15.42:0/3367631505 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55eb704721b0 0x55eb70818180 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:58.642 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.636+0000 7f1404498740 1 -- 172.21.15.42:0/3367631505 >> 172.21.15.42:0/3367631505 conn(0x55eb70466300 msgr2=0x55eb70813c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:06:58.643 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.636+0000 7f1404498740 1 -- 172.21.15.42:0/3367631505 shutdown_connections 2024-08-30T09:06:58.643 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.636+0000 7f1404498740 1 -- 172.21.15.42:0/3367631505 wait complete. 2024-08-30T09:06:58.646 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.636+0000 7f1404498740 1 -- start start 2024-08-30T09:06:58.646 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.637+0000 7f1404498740 1 Processor -- start 2024-08-30T09:06:58.646 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.637+0000 7f1404498740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eb704492f0 0x55eb70821c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:58.646 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.637+0000 7f1404498740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55eb704514d0 0x55eb708222f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:58.646 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.638+0000 7f13effff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55eb704514d0 0x55eb708222f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:06:58.646 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.638+0000 7f13effff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55eb704514d0 0x55eb708222f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:32906/0 (socket says 172.21.15.42:32906) 2024-08-30T09:06:58.646 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.638+0000 7f1404498740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55eb704721b0 0x55eb70822e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:58.646 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.638+0000 7f1404498740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55eb706b3ab0 con 0x55eb704514d0 2024-08-30T09:06:58.646 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.638+0000 7f1404498740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55eb70701680 con 0x55eb704492f0 2024-08-30T09:06:58.646 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.638+0000 7f1404498740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55eb70819e60 con 0x55eb704721b0 2024-08-30T09:06:58.647 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.638+0000 7f13f77fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55eb704721b0 0x55eb70822e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:06:58.647 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.638+0000 7f13f77fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55eb704721b0 0x55eb70822e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:36610/0 (socket says 172.21.15.42:36610) 2024-08-30T09:06:58.647 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.638+0000 7f13f77fe640 1 -- 172.21.15.42:0/649411490 learned_addr learned my addr 172.21.15.42:0/649411490 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:06:58.647 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.639+0000 7f13f77fe640 1 -- 172.21.15.42:0/649411490 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eb704492f0 msgr2=0x55eb70821c30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-30T09:06:58.647 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.639+0000 7f13f77fe640 1 --2- 172.21.15.42:0/649411490 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eb704492f0 0x55eb70821c30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:58.647 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.639+0000 7f13f77fe640 1 -- 172.21.15.42:0/649411490 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55eb704514d0 msgr2=0x55eb708222f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:06:58.647 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.639+0000 7f13f77fe640 1 --2- 172.21.15.42:0/649411490 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55eb704514d0 0x55eb708222f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:58.647 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.639+0000 7f13f77fe640 1 -- 172.21.15.42:0/649411490 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb70556620 con 0x55eb704721b0 2024-08-30T09:06:58.647 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.639+0000 7f13effff640 1 --2- 172.21.15.42:0/649411490 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55eb704514d0 0x55eb708222f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-30T09:06:58.647 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.638+0000 7f1404498740 10 client.0 _ll_register_callbacks cb 0x55eb70489a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:06:58.647 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.639+0000 7f1404498740 1 -- 172.21.15.42:0/649411490 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f13e0010c20 con 0x55eb704721b0 2024-08-30T09:06:58.647 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.639+0000 7f13f77fe640 1 --2- 172.21.15.42:0/649411490 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55eb704721b0 0x55eb70822e60 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f13f00a87a0 tx=0x7f13f00acee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5d72305871c0a988 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:06:58.647 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.639+0000 7f13de7fc640 10 client.26242 ms_handle_connect on v2:172.21.15.190:3300/0 2024-08-30T09:06:58.648 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.639+0000 7f13de7fc640 1 -- 172.21.15.42:0/649411490 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13f00b5570 con 0x55eb704721b0 2024-08-30T09:06:58.648 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.639+0000 7f13de7fc640 1 -- 172.21.15.42:0/649411490 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f13f0004480 con 0x55eb704721b0 2024-08-30T09:06:58.648 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.640+0000 7f13de7fc640 1 -- 172.21.15.42:0/649411490 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f13f00ac980 con 0x55eb704721b0 2024-08-30T09:06:58.648 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.640+0000 7f1404498740 10 client.26242 fetch_fsmap learned FSMap version 34 2024-08-30T09:06:58.648 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.640+0000 7f1404498740 1 -- 172.21.15.42:0/649411490 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({fsmap=34}) -- 0x55eb7082b090 con 0x55eb704721b0 2024-08-30T09:06:58.648 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.640+0000 7f13de7fc640 1 -- 172.21.15.42:0/649411490 <== mon.1 v2:172.21.15.190:3300/0 4 ==== fsmap(e 34) ==== 8366+0+0 (secure 0 0 0) 0x7f13f00b50e0 con 0x55eb704721b0 2024-08-30T09:06:58.648 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.640+0000 7f1404498740 10 client.26242 fetch_fsmap finished waiting for FSMap version 34 2024-08-30T09:06:58.648 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.640+0000 7f1404498740 10 client.26242 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-30T09:06:58.648 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.640+0000 7f1404498740 20 client.26242 populate_metadata read hostname 'smithi042' 2024-08-30T09:06:58.648 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.640+0000 7f1404498740 1 --2- 172.21.15.42:0/649411490 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55eb70430d80 0x55eb7082e810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:58.648 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.640+0000 7f1404498740 4 client.26242 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-30T09:06:58.648 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.640+0000 7f1404498740 1 -- 172.21.15.42:0/649411490 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eb7078cf50 con 0x55eb70430d80 2024-08-30T09:06:58.649 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.640+0000 7f13f6ffd640 1 --2- 172.21.15.42:0/649411490 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55eb70430d80 0x55eb7082e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:06:58.649 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.641+0000 7f13de7fc640 10 client.26242 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:06:58.649 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.641+0000 7f13f6ffd640 1 --2- 172.21.15.42:0/649411490 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55eb70430d80 0x55eb7082e810 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8552b10b5a82227a server_cookie=a377cdc50eeffed0 in_seq=0 out_seq=0 2024-08-30T09:06:58.649 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.642+0000 7f13de7fc640 1 -- 172.21.15.42:0/649411490 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55eb7078cf50 con 0x55eb70430d80 2024-08-30T09:06:58.649 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.642+0000 7f13de7fc640 10 client.26242 handle_command_reply: tid=0 2024-08-30T09:06:58.649 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.642+0000 7f1404498740 1 client.26242 shutdown 2024-08-30T09:06:58.649 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.642+0000 7f1404498740 1 -- 172.21.15.42:0/649411490 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55eb704721b0 msgr2=0x55eb70822e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:06:58.649 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.642+0000 7f1404498740 1 --2- 172.21.15.42:0/649411490 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55eb704721b0 0x55eb70822e60 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f13f00a87a0 tx=0x7f13f00acee0 comp rx=0 tx=0).stop 2024-08-30T09:06:58.649 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.642+0000 7f1404498740 1 -- 172.21.15.42:0/649411490 shutdown_connections 2024-08-30T09:06:58.649 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.642+0000 7f1404498740 1 --2- 172.21.15.42:0/649411490 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55eb70430d80 0x55eb7082e810 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:58.649 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.642+0000 7f1404498740 1 --2- 172.21.15.42:0/649411490 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55eb704721b0 0x55eb70822e60 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:58.649 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.642+0000 7f1404498740 1 --2- 172.21.15.42:0/649411490 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55eb704514d0 0x55eb708222f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:58.649 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.642+0000 7f1404498740 1 --2- 172.21.15.42:0/649411490 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eb704492f0 0x55eb70821c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:58.649 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.642+0000 7f13de7fc640 0 client.26242 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:06:58.650 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.642+0000 7f1404498740 1 -- 172.21.15.42:0/649411490 >> 172.21.15.42:0/649411490 conn(0x55eb70466300 msgr2=0x55eb70812170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:06:58.650 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.642+0000 7f1404498740 1 -- 172.21.15.42:0/649411490 shutdown_connections 2024-08-30T09:06:58.650 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.642+0000 7f1404498740 1 -- 172.21.15.42:0/649411490 wait complete. 2024-08-30T09:06:58.650 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.642+0000 7f1404498740 20 client.26242 trim_cache size 0 max 16384 2024-08-30T09:06:58.663 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.661+0000 7f1404498740 1 Processor -- start 2024-08-30T09:06:58.664 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.661+0000 7f1404498740 1 -- start start 2024-08-30T09:06:58.665 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.661+0000 7f1404498740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55eb704492f0 0x55eb70817e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:58.665 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.661+0000 7f1404498740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55eb704514d0 0x55eb7081cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:58.665 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.661+0000 7f1404498740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eb704721b0 0x55eb70814150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:58.665 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.661+0000 7f13effff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55eb704514d0 0x55eb7081cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:06:58.665 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.662+0000 7f13effff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55eb704514d0 0x55eb7081cbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:32912/0 (socket says 172.21.15.42:32912) 2024-08-30T09:06:58.665 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.662+0000 7f13f6ffd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55eb704492f0 0x55eb70817e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:06:58.665 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.662+0000 7f13effff640 1 -- 172.21.15.42:0/2173853115 learned_addr learned my addr 172.21.15.42:0/2173853115 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:06:58.665 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.662+0000 7f1404498740 1 -- 172.21.15.42:0/2173853115 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f13f81acb60 con 0x55eb704514d0 2024-08-30T09:06:58.665 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.662+0000 7f1404498740 1 -- 172.21.15.42:0/2173853115 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f13f81aa3d0 con 0x55eb704721b0 2024-08-30T09:06:58.665 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.662+0000 7f1404498740 1 -- 172.21.15.42:0/2173853115 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f13f8074e20 con 0x55eb704492f0 2024-08-30T09:06:58.665 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.662+0000 7f13f77fe640 1 --2- 172.21.15.42:0/2173853115 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eb704721b0 0x55eb70814150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:06:58.666 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.662+0000 7f13f6ffd640 1 -- 172.21.15.42:0/2173853115 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eb704721b0 msgr2=0x55eb70814150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:06:58.666 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.662+0000 7f13f6ffd640 1 --2- 172.21.15.42:0/2173853115 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eb704721b0 0x55eb70814150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:58.666 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.662+0000 7f13f6ffd640 1 -- 172.21.15.42:0/2173853115 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55eb704514d0 msgr2=0x55eb7081cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:06:58.666 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.662+0000 7f13f6ffd640 1 --2- 172.21.15.42:0/2173853115 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55eb704514d0 0x55eb7081cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:58.666 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.662+0000 7f13f6ffd640 1 -- 172.21.15.42:0/2173853115 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb7081fdb0 con 0x55eb704492f0 2024-08-30T09:06:58.666 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.662+0000 7f13f6ffd640 1 --2- 172.21.15.42:0/2173853115 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55eb704492f0 0x55eb70817e60 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x55eb7082ef40 tx=0x7f13e803f000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8c0b326236071185 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:06:58.667 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.663+0000 7f13ecff9640 1 -- 172.21.15.42:0/2173853115 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13e8056070 con 0x55eb704492f0 2024-08-30T09:06:58.667 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.663+0000 7f13ecff9640 1 -- 172.21.15.42:0/2173853115 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f13e8031a50 con 0x55eb704492f0 2024-08-30T09:06:58.667 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.663+0000 7f13ecff9640 1 -- 172.21.15.42:0/2173853115 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13e8035040 con 0x55eb704492f0 2024-08-30T09:06:58.667 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.664+0000 7f1404498740 1 -- 172.21.15.42:0/2173853115 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55eb704492f0 msgr2=0x55eb70817e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:06:58.667 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.664+0000 7f1404498740 1 --2- 172.21.15.42:0/2173853115 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55eb704492f0 0x55eb70817e60 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x55eb7082ef40 tx=0x7f13e803f000 comp rx=0 tx=0).stop 2024-08-30T09:06:58.669 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.666+0000 7f1404498740 1 -- 172.21.15.42:0/2173853115 shutdown_connections 2024-08-30T09:06:58.669 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.666+0000 7f1404498740 1 --2- 172.21.15.42:0/2173853115 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eb704721b0 0x55eb70814150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:58.669 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.666+0000 7f1404498740 1 --2- 172.21.15.42:0/2173853115 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55eb704514d0 0x55eb7081cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:58.669 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.666+0000 7f1404498740 1 --2- 172.21.15.42:0/2173853115 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55eb704492f0 0x55eb70817e60 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:58.669 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.666+0000 7f1404498740 1 -- 172.21.15.42:0/2173853115 >> 172.21.15.42:0/2173853115 conn(0x55eb70466300 msgr2=0x55eb7080e4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:06:58.669 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.666+0000 7f1404498740 1 -- 172.21.15.42:0/2173853115 shutdown_connections 2024-08-30T09:06:58.669 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.666+0000 7f1404498740 1 -- 172.21.15.42:0/2173853115 wait complete. 2024-08-30T09:06:58.670 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.666+0000 7f1404498740 1 -- start start 2024-08-30T09:06:58.670 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.667+0000 7f1404498740 1 Processor -- start 2024-08-30T09:06:58.670 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.667+0000 7f1404498740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eb704492f0 0x55eb70828ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:58.670 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.667+0000 7f1404498740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55eb704514d0 0x55eb70829580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:58.671 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.667+0000 7f1404498740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55eb704721b0 0x55eb70891f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:58.671 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.668+0000 7f1404498740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f13f81a8890 con 0x55eb704514d0 2024-08-30T09:06:58.671 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.668+0000 7f13effff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55eb704514d0 0x55eb70829580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:06:58.671 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.668+0000 7f13effff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55eb704514d0 0x55eb70829580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:32914/0 (socket says 172.21.15.42:32914) 2024-08-30T09:06:58.671 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.668+0000 7f13effff640 1 -- 172.21.15.42:0/342839923 learned_addr learned my addr 172.21.15.42:0/342839923 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:06:58.672 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.668+0000 7f13f6ffd640 1 --2- 172.21.15.42:0/342839923 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eb704492f0 0x55eb70828ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:06:58.672 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.668+0000 7f1404498740 1 -- 172.21.15.42:0/342839923 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f13f81a9d00 con 0x55eb704492f0 2024-08-30T09:06:58.672 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.668+0000 7f1404498740 1 -- 172.21.15.42:0/342839923 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f13f81a8f60 con 0x55eb704721b0 2024-08-30T09:06:58.673 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.668+0000 7f13f77fe640 1 --2- 172.21.15.42:0/342839923 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55eb704721b0 0x55eb70891f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:06:58.673 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.668+0000 7f1404498740 10 client.0 _ll_register_callbacks cb 0x7f13f0005aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:06:58.673 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.668+0000 7f13effff640 1 -- 172.21.15.42:0/342839923 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55eb704721b0 msgr2=0x55eb70891f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:06:58.673 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.668+0000 7f13effff640 1 --2- 172.21.15.42:0/342839923 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55eb704721b0 0x55eb70891f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:58.673 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.668+0000 7f13effff640 1 -- 172.21.15.42:0/342839923 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eb704492f0 msgr2=0x55eb70828ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:06:58.674 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.668+0000 7f13effff640 1 --2- 172.21.15.42:0/342839923 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eb704492f0 0x55eb70828ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:58.674 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.668+0000 7f13effff640 1 -- 172.21.15.42:0/342839923 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb708926a0 con 0x55eb704514d0 2024-08-30T09:06:58.674 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.669+0000 7f13effff640 1 --2- 172.21.15.42:0/342839923 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55eb704514d0 0x55eb70829580 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f13e0012820 tx=0x7f13e0012490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22ce86cbbeadc053 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:06:58.674 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.669+0000 7f13de7fc640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-08-30T09:06:58.674 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.669+0000 7f13de7fc640 1 -- 172.21.15.42:0/342839923 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13e0024040 con 0x55eb704514d0 2024-08-30T09:06:58.674 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.670+0000 7f13de7fc640 1 -- 172.21.15.42:0/342839923 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f13e00050b0 con 0x55eb704514d0 2024-08-30T09:06:58.674 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.670+0000 7f1404498740 1 -- 172.21.15.42:0/342839923 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f13e8031a50 con 0x55eb704514d0 2024-08-30T09:06:58.674 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.670+0000 7f13de7fc640 1 -- 172.21.15.42:0/342839923 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f13e00052d0 con 0x55eb704514d0 2024-08-30T09:06:58.674 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.670+0000 7f1404498740 10 client.16515 fetch_fsmap learned FSMap version 34 2024-08-30T09:06:58.674 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.670+0000 7f1404498740 1 -- 172.21.15.42:0/342839923 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55eb70899980 con 0x55eb704514d0 2024-08-30T09:06:58.674 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.670+0000 7f13de7fc640 1 -- 172.21.15.42:0/342839923 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f13e8031a50 con 0x55eb704514d0 2024-08-30T09:06:58.675 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.671+0000 7f13de7fc640 1 -- 172.21.15.42:0/342839923 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 34) ==== 8366+0+0 (secure 0 0 0) 0x7f13e0027030 con 0x55eb704514d0 2024-08-30T09:06:58.675 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.671+0000 7f1404498740 10 client.16515 fetch_fsmap finished waiting for FSMap version 34 2024-08-30T09:06:58.675 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.671+0000 7f1404498740 10 client.16515 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-30T09:06:58.675 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.671+0000 7f1404498740 20 client.16515 populate_metadata read hostname 'smithi042' 2024-08-30T09:06:58.675 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.671+0000 7f1404498740 1 --2- 172.21.15.42:0/342839923 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55eb704061c0 0x55eb7089c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:06:58.675 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.671+0000 7f1404498740 4 client.16515 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-30T09:06:58.675 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.671+0000 7f1404498740 1 -- 172.21.15.42:0/342839923 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55eb707889f0 con 0x55eb704061c0 2024-08-30T09:06:58.676 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.673+0000 7f13f6ffd640 1 --2- 172.21.15.42:0/342839923 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55eb704061c0 0x55eb7089c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:06:58.680 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.677+0000 7f13de7fc640 10 client.16515 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:06:58.680 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.677+0000 7f13f6ffd640 1 --2- 172.21.15.42:0/342839923 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55eb704061c0 0x55eb7089c150 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=18b4c135a31ed69b server_cookie=581529d44a87fae in_seq=0 out_seq=0 2024-08-30T09:06:58.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:06:58 smithi190 ceph-mon[22001]: pgmap v559: 129 pgs: 129 active+clean; 1.4 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 10 MiB/s wr, 587 op/s 2024-08-30T09:06:58.804 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.800+0000 7f13de7fc640 1 -- 172.21.15.42:0/342839923 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55eb707889f0 con 0x55eb704061c0 2024-08-30T09:06:58.804 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.800+0000 7f13de7fc640 10 client.16515 handle_command_reply: tid=0 2024-08-30T09:06:58.804 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.800+0000 7f1404498740 1 client.16515 shutdown 2024-08-30T09:06:58.804 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.800+0000 7f1404498740 1 -- 172.21.15.42:0/342839923 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55eb704514d0 msgr2=0x55eb70829580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:06:58.805 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.800+0000 7f1404498740 1 --2- 172.21.15.42:0/342839923 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55eb704514d0 0x55eb70829580 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f13e0012820 tx=0x7f13e0012490 comp rx=0 tx=0).stop 2024-08-30T09:06:58.805 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.801+0000 7f1404498740 1 -- 172.21.15.42:0/342839923 shutdown_connections 2024-08-30T09:06:58.805 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.801+0000 7f1404498740 1 --2- 172.21.15.42:0/342839923 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55eb704061c0 0x55eb7089c150 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:58.805 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.801+0000 7f1404498740 1 --2- 172.21.15.42:0/342839923 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55eb704721b0 0x55eb70891f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:58.805 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.801+0000 7f13de7fc640 0 client.16515 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:06:58.805 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.801+0000 7f1404498740 1 --2- 172.21.15.42:0/342839923 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55eb704514d0 0x55eb70829580 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:58.805 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.801+0000 7f1404498740 1 --2- 172.21.15.42:0/342839923 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eb704492f0 0x55eb70828ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:58.805 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.801+0000 7f1404498740 1 -- 172.21.15.42:0/342839923 >> 172.21.15.42:0/342839923 conn(0x55eb70466300 msgr2=0x55eb70817d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:06:58.805 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.801+0000 7f1404498740 1 -- 172.21.15.42:0/342839923 shutdown_connections 2024-08-30T09:06:58.805 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.801+0000 7f1404498740 1 -- 172.21.15.42:0/342839923 wait complete. 2024-08-30T09:06:58.805 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.801+0000 7f1404498740 20 client.16515 trim_cache size 0 max 16384 2024-08-30T09:06:58.806 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-08-30T09:06:58.806 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2024-08-30T09:06:58.806 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-08-30T09:06:58.806 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-08-30T09:06:58.807 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.802+0000 7f13fd52b640 1 -- 172.21.15.42:0/2556672058 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f13e40b4b70 msgr2=0x7f13e40b7080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:06:58.807 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.802+0000 7f13fd52b640 1 --2- 172.21.15.42:0/2556672058 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f13e40b4b70 0x7f13e40b7080 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f13e8000c00 tx=0x7f13e80095e0 comp rx=0 tx=0).stop 2024-08-30T09:06:58.807 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.802+0000 7f13fd52b640 1 -- 172.21.15.42:0/2556672058 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f13f8103d60 msgr2=0x7f13f81a1500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:06:58.807 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.802+0000 7f13fd52b640 1 --2- 172.21.15.42:0/2556672058 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f13f8103d60 0x7f13f81a1500 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f13f000eb20 tx=0x7f13f000c2d0 comp rx=0 tx=0).stop 2024-08-30T09:06:58.807 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.802+0000 7f13fd52b640 1 -- 172.21.15.42:0/2556672058 shutdown_connections 2024-08-30T09:06:58.807 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.802+0000 7f13fd52b640 1 --2- 172.21.15.42:0/2556672058 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f13e40b4b70 0x7f13e40b7080 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:58.807 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.802+0000 7f13fd52b640 1 --2- 172.21.15.42:0/2556672058 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f13f8103d60 0x7f13f81a1500 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:58.807 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.802+0000 7f13fd52b640 1 --2- 172.21.15.42:0/2556672058 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f13f81033c0 0x7f13f81a0950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:58.807 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.802+0000 7f13fd52b640 1 --2- 172.21.15.42:0/2556672058 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f13f8102340 0x7f13f81a02e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:06:58.807 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.802+0000 7f13fd52b640 1 -- 172.21.15.42:0/2556672058 >> 172.21.15.42:0/2556672058 conn(0x7f13f80fdc80 msgr2=0x7f13f810f880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:06:58.807 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.802+0000 7f13fd52b640 1 -- 172.21.15.42:0/2556672058 shutdown_connections 2024-08-30T09:06:58.808 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:06:58.802+0000 7f13fd52b640 1 -- 172.21.15.42:0/2556672058 wait complete. 2024-08-30T09:06:58.826 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:06:58 smithi167 ceph-mon[22063]: pgmap v559: 129 pgs: 129 active+clean; 1.4 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 10 MiB/s wr, 587 op/s 2024-08-30T09:06:58.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:06:58 smithi042 ceph-mon[18652]: pgmap v559: 129 pgs: 129 active+clean; 1.4 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 10 MiB/s wr, 587 op/s 2024-08-30T09:07:00.685 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:07:00 smithi042 ceph-mon[18652]: pgmap v560: 129 pgs: 129 active+clean; 1.4 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 10 MiB/s wr, 579 op/s 2024-08-30T09:07:00.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:07:00 smithi190 ceph-mon[22001]: pgmap v560: 129 pgs: 129 active+clean; 1.4 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 10 MiB/s wr, 579 op/s 2024-08-30T09:07:00.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:07:00 smithi167 ceph-mon[22063]: pgmap v560: 129 pgs: 129 active+clean; 1.4 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 10 MiB/s wr, 579 op/s 2024-08-30T09:07:00.953 INFO:teuthology.orchestra.run.smithi042.stderr:progress: 375.0 s, 842.4 tps, lat 116.918 ms stddev 149.723 2024-08-30T09:07:02.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:07:02 smithi190 ceph-mon[22001]: pgmap v561: 129 pgs: 129 active+clean; 1.4 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.9 MiB/s wr, 515 op/s 2024-08-30T09:07:02.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:07:02 smithi167 ceph-mon[22063]: pgmap v561: 129 pgs: 129 active+clean; 1.4 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.9 MiB/s wr, 515 op/s 2024-08-30T09:07:02.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:07:02 smithi042 ceph-mon[18652]: pgmap v561: 129 pgs: 129 active+clean; 1.4 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.9 MiB/s wr, 515 op/s 2024-08-30T09:07:04.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:07:04 smithi167 ceph-mon[22063]: pgmap v562: 129 pgs: 129 active+clean; 1.5 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 MiB/s wr, 687 op/s 2024-08-30T09:07:04.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:07:04 smithi042 ceph-mon[18652]: pgmap v562: 129 pgs: 129 active+clean; 1.5 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 MiB/s wr, 687 op/s 2024-08-30T09:07:04.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:07:04 smithi190 ceph-mon[22001]: pgmap v562: 129 pgs: 129 active+clean; 1.5 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 MiB/s wr, 687 op/s 2024-08-30T09:07:05.953 INFO:teuthology.orchestra.run.smithi042.stderr:progress: 380.0 s, 639.0 tps, lat 156.348 ms stddev 200.397 2024-08-30T09:07:06.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:07:06 smithi167 ceph-mon[22063]: pgmap v563: 129 pgs: 129 active+clean; 1.5 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.6 MiB/s wr, 566 op/s 2024-08-30T09:07:06.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:07:06 smithi042 ceph-mon[18652]: pgmap v563: 129 pgs: 129 active+clean; 1.5 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.6 MiB/s wr, 566 op/s 2024-08-30T09:07:06.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:07:06 smithi190 ceph-mon[22001]: pgmap v563: 129 pgs: 129 active+clean; 1.5 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.6 MiB/s wr, 566 op/s 2024-08-30T09:07:08.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:07:08 smithi167 ceph-mon[22063]: pgmap v564: 129 pgs: 129 active+clean; 1.5 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.4 MiB/s wr, 568 op/s 2024-08-30T09:07:08.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:07:08 smithi042 ceph-mon[18652]: pgmap v564: 129 pgs: 129 active+clean; 1.5 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.4 MiB/s wr, 568 op/s 2024-08-30T09:07:08.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:07:08 smithi190 ceph-mon[22001]: pgmap v564: 129 pgs: 129 active+clean; 1.5 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.4 MiB/s wr, 568 op/s 2024-08-30T09:07:10.738 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:07:10 smithi190 ceph-mon[22001]: pgmap v565: 129 pgs: 129 active+clean; 1.5 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 MiB/s wr, 1.10k op/s 2024-08-30T09:07:10.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:07:10 smithi167 ceph-mon[22063]: pgmap v565: 129 pgs: 129 active+clean; 1.5 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 MiB/s wr, 1.10k op/s 2024-08-30T09:07:10.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:07:10 smithi042 ceph-mon[18652]: pgmap v565: 129 pgs: 129 active+clean; 1.5 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 MiB/s wr, 1.10k op/s 2024-08-30T09:07:10.953 INFO:teuthology.orchestra.run.smithi042.stderr:progress: 385.0 s, 479.2 tps, lat 205.347 ms stddev 284.761 2024-08-30T09:07:12.804 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:07:12 smithi042 ceph-mon[18652]: pgmap v566: 129 pgs: 129 active+clean; 1.5 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 14 MiB/s wr, 1.16k op/s 2024-08-30T09:07:12.804 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:07:12 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-30T09:07:12.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:07:12 smithi167 ceph-mon[22063]: pgmap v566: 129 pgs: 129 active+clean; 1.5 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 14 MiB/s wr, 1.16k op/s 2024-08-30T09:07:12.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:07:12 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-30T09:07:12.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:07:12 smithi190 ceph-mon[22001]: pgmap v566: 129 pgs: 129 active+clean; 1.5 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 14 MiB/s wr, 1.16k op/s 2024-08-30T09:07:12.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:07:12 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-30T09:07:13.173 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-30T09:07:13.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:07:13 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-08-30T09:07:13.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:07:13 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-08-30T09:07:13.634 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:07:13 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-08-30T09:07:13.634 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:07:13 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-08-30T09:07:13.634 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:07:13 smithi042 ceph-mon[18652]: Adjusting osd_memory_target on smithi167 to 176.0M 2024-08-30T09:07:13.634 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:07:13 smithi042 ceph-mon[18652]: Unable to set osd_memory_target on smithi167 to 184590336: error parsing value: Value '184590336' is below minimum 939524096 2024-08-30T09:07:13.634 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:07:13 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-08-30T09:07:13.634 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:07:13 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-08-30T09:07:13.634 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:07:13 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-08-30T09:07:13.634 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:07:13 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-08-30T09:07:13.634 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:07:13 smithi042 ceph-mon[18652]: Adjusting osd_memory_target on smithi190 to 176.0M 2024-08-30T09:07:13.634 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:07:13 smithi042 ceph-mon[18652]: Unable to set osd_memory_target on smithi190 to 184596070: error parsing value: Value '184596070' is below minimum 939524096 2024-08-30T09:07:13.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:07:13 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-08-30T09:07:13.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:07:13 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-08-30T09:07:13.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:07:13 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-08-30T09:07:13.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:07:13 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-08-30T09:07:13.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:07:13 smithi167 ceph-mon[22063]: Adjusting osd_memory_target on smithi167 to 176.0M 2024-08-30T09:07:13.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:07:13 smithi167 ceph-mon[22063]: Unable to set osd_memory_target on smithi167 to 184590336: error parsing value: Value '184590336' is below minimum 939524096 2024-08-30T09:07:13.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:07:13 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-08-30T09:07:13.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:07:13 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-08-30T09:07:13.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:07:13 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-08-30T09:07:13.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:07:13 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-08-30T09:07:13.828 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:07:13 smithi167 ceph-mon[22063]: Adjusting osd_memory_target on smithi190 to 176.0M 2024-08-30T09:07:13.828 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:07:13 smithi167 ceph-mon[22063]: Unable to set osd_memory_target on smithi190 to 184596070: error parsing value: Value '184596070' is below minimum 939524096 2024-08-30T09:07:13.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:07:13 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-08-30T09:07:13.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:07:13 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-08-30T09:07:13.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:07:13 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-08-30T09:07:13.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:07:13 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-08-30T09:07:13.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:07:13 smithi190 ceph-mon[22001]: Adjusting osd_memory_target on smithi167 to 176.0M 2024-08-30T09:07:13.937 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:07:13 smithi190 ceph-mon[22001]: Unable to set osd_memory_target on smithi167 to 184590336: error parsing value: Value '184590336' is below minimum 939524096 2024-08-30T09:07:13.937 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:07:13 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-08-30T09:07:13.937 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:07:13 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-08-30T09:07:13.937 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:07:13 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-08-30T09:07:13.937 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:07:13 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-08-30T09:07:13.937 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:07:13 smithi190 ceph-mon[22001]: Adjusting osd_memory_target on smithi190 to 176.0M 2024-08-30T09:07:13.937 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:07:13 smithi190 ceph-mon[22001]: Unable to set osd_memory_target on smithi190 to 184596070: error parsing value: Value '184596070' is below minimum 939524096 2024-08-30T09:07:14.660 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.656+0000 7f4c0ac69640 1 -- 172.21.15.42:0/2574758582 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4c0410a160 msgr2=0x7f4c0410c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:07:14.661 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.656+0000 7f4c0ac69640 1 --2- 172.21.15.42:0/2574758582 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4c0410a160 0x7f4c0410c580 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f4bfc009ca0 tx=0x7f4bfc02a8f0 comp rx=0 tx=0).stop 2024-08-30T09:07:14.661 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.657+0000 7f4c0ac69640 1 -- 172.21.15.42:0/2574758582 shutdown_connections 2024-08-30T09:07:14.661 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.657+0000 7f4c0ac69640 1 --2- 172.21.15.42:0/2574758582 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4c0410f770 0x7f4c0411a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:14.661 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.657+0000 7f4c0ac69640 1 --2- 172.21.15.42:0/2574758582 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4c0410cb50 0x7f4c0410f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:14.661 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.657+0000 7f4c0ac69640 1 --2- 172.21.15.42:0/2574758582 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4c0410a160 0x7f4c0410c580 secure :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f4bfc009ca0 tx=0x7f4bfc02a8f0 comp rx=0 tx=0).stop 2024-08-30T09:07:14.661 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.657+0000 7f4c0ac69640 1 -- 172.21.15.42:0/2574758582 >> 172.21.15.42:0/2574758582 conn(0x7f4c040705b0 msgr2=0x7f4c04072a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:07:14.669 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.664+0000 7f4c0ac69640 1 -- 172.21.15.42:0/2574758582 shutdown_connections 2024-08-30T09:07:14.669 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.666+0000 7f4c0ac69640 1 -- 172.21.15.42:0/2574758582 wait complete. 2024-08-30T09:07:14.674 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.671+0000 7f4c0ac69640 1 Processor -- start 2024-08-30T09:07:14.674 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.671+0000 7f4c0ac69640 1 -- start start 2024-08-30T09:07:14.675 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.672+0000 7f4c0ac69640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4c0410cb50 0x7f4c0411a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:07:14.675 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.672+0000 7f4c0ac69640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4c0410f770 0x7f4c04118cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:07:14.675 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.672+0000 7f4c0ac69640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4c04119200 0x7f4c04119610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:07:14.675 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.672+0000 7f4c0ac69640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f4c04112a20 con 0x7f4c0410cb50 2024-08-30T09:07:14.675 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.672+0000 7f4c0ac69640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4c041128a0 con 0x7f4c0410f770 2024-08-30T09:07:14.675 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.672+0000 7f4c0ac69640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f4c04112ba0 con 0x7f4c04119200 2024-08-30T09:07:14.676 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.672+0000 7f4c03fff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4c0410f770 0x7f4c04118cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:07:14.676 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.672+0000 7f4c09206640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4c04119200 0x7f4c04119610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:07:14.676 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.672+0000 7f4c03fff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4c0410f770 0x7f4c04118cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:56994/0 (socket says 172.21.15.42:56994) 2024-08-30T09:07:14.676 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.672+0000 7f4c03fff640 1 -- 172.21.15.42:0/2894941902 learned_addr learned my addr 172.21.15.42:0/2894941902 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:07:14.676 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.672+0000 7f4c08a05640 1 --2- 172.21.15.42:0/2894941902 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4c0410cb50 0x7f4c0411a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:07:14.677 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.673+0000 7f4c03fff640 1 -- 172.21.15.42:0/2894941902 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4c04119200 msgr2=0x7f4c04119610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:07:14.677 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.673+0000 7f4c03fff640 1 --2- 172.21.15.42:0/2894941902 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4c04119200 0x7f4c04119610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:14.677 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.673+0000 7f4c03fff640 1 -- 172.21.15.42:0/2894941902 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4c0410cb50 msgr2=0x7f4c0411a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:07:14.677 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.673+0000 7f4c03fff640 1 --2- 172.21.15.42:0/2894941902 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4c0410cb50 0x7f4c0411a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:14.677 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.673+0000 7f4c03fff640 1 -- 172.21.15.42:0/2894941902 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c041b9400 con 0x7f4c0410f770 2024-08-30T09:07:14.679 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.676+0000 7f4c03fff640 1 --2- 172.21.15.42:0/2894941902 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4c0410f770 0x7f4c04118cc0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f4bec00b4a0 tx=0x7f4bec00b960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:07:14.681 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.678+0000 7f4c01ffb640 1 -- 172.21.15.42:0/2894941902 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bec014070 con 0x7f4c0410f770 2024-08-30T09:07:14.681 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.678+0000 7f4c01ffb640 1 -- 172.21.15.42:0/2894941902 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4bec010ac0 con 0x7f4c0410f770 2024-08-30T09:07:14.682 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.679+0000 7f4c01ffb640 1 -- 172.21.15.42:0/2894941902 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bec017c20 con 0x7f4c0410f770 2024-08-30T09:07:14.682 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.679+0000 7f4c0ac69640 1 -- 172.21.15.42:0/2894941902 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c041b96f0 con 0x7f4c0410f770 2024-08-30T09:07:14.682 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.679+0000 7f4c0ac69640 1 -- 172.21.15.42:0/2894941902 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c041b9c70 con 0x7f4c0410f770 2024-08-30T09:07:14.687 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.683+0000 7f4c11bd8740 1 Processor -- start 2024-08-30T09:07:14.687 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.684+0000 7f4c01ffb640 1 -- 172.21.15.42:0/2894941902 <== mon.2 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7f4bec017070 con 0x7f4c0410f770 2024-08-30T09:07:14.688 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.685+0000 7f4c01ffb640 1 --2- 172.21.15.42:0/2894941902 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f4bf00b4d40 0x7f4bf00b7250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:07:14.689 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.686+0000 7f4c11bd8740 1 -- start start 2024-08-30T09:07:14.689 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.686+0000 7f4c09206640 1 --2- 172.21.15.42:0/2894941902 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f4bf00b4d40 0x7f4bf00b7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:07:14.689 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.686+0000 7f4c11bd8740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b2311301b0 0x55b2314d6180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:07:14.690 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.686+0000 7f4c11bd8740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b2311072f0 0x55b2314d67f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:07:14.690 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.686+0000 7f4c11bd8740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b23110f4d0 0x55b2314d7340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:07:14.690 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.686+0000 7f4c11bd8740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55b231449000 con 0x55b2311072f0 2024-08-30T09:07:14.690 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.686+0000 7f4c11bd8740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55b231438590 con 0x55b23110f4d0 2024-08-30T09:07:14.690 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.686+0000 7f4c11bd8740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55b23143c540 con 0x55b2311301b0 2024-08-30T09:07:14.690 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.686+0000 7f4c08a05640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b2311301b0 0x55b2314d6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:07:14.690 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.687+0000 7f4c08a05640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b2311301b0 0x55b2314d6180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:59116/0 (socket says 172.21.15.42:59116) 2024-08-30T09:07:14.690 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.687+0000 7f4c08a05640 1 -- 172.21.15.42:0/2442427921 learned_addr learned my addr 172.21.15.42:0/2442427921 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:07:14.692 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.687+0000 7f4c01ffb640 1 -- 172.21.15.42:0/2894941902 <== mon.2 v2:172.21.15.167:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f4bec0e4660 con 0x7f4c0410f770 2024-08-30T09:07:14.692 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.687+0000 7f4c09206640 1 --2- 172.21.15.42:0/2442427921 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b23110f4d0 0x55b2314d7340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:07:14.692 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.687+0000 7f4c08a05640 1 -- 172.21.15.42:0/2442427921 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b23110f4d0 msgr2=0x55b2314d7340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:07:14.692 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.687+0000 7f4c08a05640 1 --2- 172.21.15.42:0/2442427921 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b23110f4d0 0x55b2314d7340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:14.692 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.687+0000 7f4c08a05640 1 -- 172.21.15.42:0/2442427921 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b2311072f0 msgr2=0x55b2314d67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:07:14.692 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.687+0000 7f4c08a05640 1 --2- 172.21.15.42:0/2442427921 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b2311072f0 0x55b2314d67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:14.692 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.688+0000 7f4c08a05640 1 -- 172.21.15.42:0/2442427921 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b23119b790 con 0x55b2311301b0 2024-08-30T09:07:14.692 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.688+0000 7f4c09206640 1 --2- 172.21.15.42:0/2442427921 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b23110f4d0 0x55b2314d7340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-30T09:07:14.692 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.688+0000 7f4c08a05640 1 --2- 172.21.15.42:0/2442427921 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b2311301b0 0x55b2314d6180 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f4bfc031a00 tx=0x7f4bfc0305f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3b737f52498ac5a1 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:07:14.693 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.688+0000 7f4c09206640 1 --2- 172.21.15.42:0/2894941902 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f4bf00b4d40 0x7f4bf00b7250 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f4bf4009aa0 tx=0x7f4bf4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:07:14.693 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.688+0000 7f4bfaffd640 1 -- 172.21.15.42:0/2442427921 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bfc031b60 con 0x55b2311301b0 2024-08-30T09:07:14.693 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.689+0000 7f4bfaffd640 1 -- 172.21.15.42:0/2442427921 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4bfc036070 con 0x55b2311301b0 2024-08-30T09:07:14.693 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.689+0000 7f4bfaffd640 1 -- 172.21.15.42:0/2442427921 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bfc003bd0 con 0x55b2311301b0 2024-08-30T09:07:14.696 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.690+0000 7f4c11bd8740 1 -- 172.21.15.42:0/2442427921 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b2311301b0 msgr2=0x55b2314d6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:07:14.696 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.690+0000 7f4c11bd8740 1 --2- 172.21.15.42:0/2442427921 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b2311301b0 0x55b2314d6180 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f4bfc031a00 tx=0x7f4bfc0305f0 comp rx=0 tx=0).stop 2024-08-30T09:07:14.696 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.690+0000 7f4c11bd8740 1 -- 172.21.15.42:0/2442427921 shutdown_connections 2024-08-30T09:07:14.696 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.690+0000 7f4c11bd8740 1 --2- 172.21.15.42:0/2442427921 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b23110f4d0 0x55b2314d7340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:14.697 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.690+0000 7f4c11bd8740 1 --2- 172.21.15.42:0/2442427921 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b2311072f0 0x55b2314d67f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:14.697 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.690+0000 7f4c11bd8740 1 --2- 172.21.15.42:0/2442427921 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b2311301b0 0x55b2314d6180 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:14.697 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.690+0000 7f4c11bd8740 1 -- 172.21.15.42:0/2442427921 >> 172.21.15.42:0/2442427921 conn(0x55b231124300 msgr2=0x55b2314d1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:07:14.697 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.690+0000 7f4c11bd8740 1 -- 172.21.15.42:0/2442427921 shutdown_connections 2024-08-30T09:07:14.697 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.690+0000 7f4c11bd8740 1 -- 172.21.15.42:0/2442427921 wait complete. 2024-08-30T09:07:14.697 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.690+0000 7f4c11bd8740 1 -- start start 2024-08-30T09:07:14.697 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.690+0000 7f4c11bd8740 1 Processor -- start 2024-08-30T09:07:14.697 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.691+0000 7f4c11bd8740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b2311072f0 0x55b2314dfc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:07:14.697 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.691+0000 7f4c11bd8740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b23110f4d0 0x55b2314e02d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:07:14.697 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.691+0000 7f4c11bd8740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b2311301b0 0x55b2314e0e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:07:14.697 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.691+0000 7f4c11bd8740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55b231371ab0 con 0x55b2311301b0 2024-08-30T09:07:14.698 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.691+0000 7f4c11bd8740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55b2313bf680 con 0x55b2311072f0 2024-08-30T09:07:14.698 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.691+0000 7f4c11bd8740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55b2314d7e00 con 0x55b23110f4d0 2024-08-30T09:07:14.698 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.691+0000 7f4c03fff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b23110f4d0 0x55b2314e02d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:07:14.698 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.691+0000 7f4c09206640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b2311301b0 0x55b2314e0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:07:14.698 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.691+0000 7f4c09206640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b2311301b0 0x55b2314e0e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43146/0 (socket says 172.21.15.42:43146) 2024-08-30T09:07:14.698 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.691+0000 7f4c03fff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b23110f4d0 0x55b2314e02d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:59132/0 (socket says 172.21.15.42:59132) 2024-08-30T09:07:14.698 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.691+0000 7f4c09206640 1 -- 172.21.15.42:0/1447703603 learned_addr learned my addr 172.21.15.42:0/1447703603 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:07:14.698 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.691+0000 7f4c08a05640 1 --2- 172.21.15.42:0/1447703603 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b2311072f0 0x55b2314dfc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:07:14.698 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.691+0000 7f4c11bd8740 10 client.0 _ll_register_callbacks cb 0x55b231147a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:07:14.698 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.691+0000 7f4c03fff640 1 -- 172.21.15.42:0/1447703603 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b2311072f0 msgr2=0x55b2314dfc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:07:14.698 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.691+0000 7f4c03fff640 1 --2- 172.21.15.42:0/1447703603 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b2311072f0 0x55b2314dfc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:14.699 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.691+0000 7f4c03fff640 1 -- 172.21.15.42:0/1447703603 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b2311301b0 msgr2=0x55b2314e0e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:07:14.699 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.691+0000 7f4c03fff640 1 --2- 172.21.15.42:0/1447703603 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b2311301b0 0x55b2314e0e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:14.699 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.691+0000 7f4c03fff640 1 -- 172.21.15.42:0/1447703603 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b231214620 con 0x55b23110f4d0 2024-08-30T09:07:14.699 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.691+0000 7f4c09206640 1 --2- 172.21.15.42:0/1447703603 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b2311301b0 0x55b2314e0e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-30T09:07:14.699 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.692+0000 7f4c03fff640 1 --2- 172.21.15.42:0/1447703603 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b23110f4d0 0x55b2314e02d0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f4bec0aac30 tx=0x7f4bec0a6430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a1d933baee44169d server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:07:14.699 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.692+0000 7f4be3fff640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-08-30T09:07:14.699 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.692+0000 7f4be3fff640 1 -- 172.21.15.42:0/1447703603 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bec0aaca0 con 0x55b23110f4d0 2024-08-30T09:07:14.699 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.692+0000 7f4be3fff640 1 -- 172.21.15.42:0/1447703603 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f4bec0e4370 con 0x55b23110f4d0 2024-08-30T09:07:14.699 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.692+0000 7f4c11bd8740 1 -- 172.21.15.42:0/1447703603 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4bfc036070 con 0x55b23110f4d0 2024-08-30T09:07:14.699 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.692+0000 7f4be3fff640 1 -- 172.21.15.42:0/1447703603 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f4bec0af510 con 0x55b23110f4d0 2024-08-30T09:07:14.700 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.692+0000 7f4c11bd8740 10 client.26257 fetch_fsmap learned FSMap version 34 2024-08-30T09:07:14.700 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.692+0000 7f4c11bd8740 1 -- 172.21.15.42:0/1447703603 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55b2314e9dc0 con 0x55b23110f4d0 2024-08-30T09:07:14.700 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.692+0000 7f4be3fff640 1 -- 172.21.15.42:0/1447703603 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4bec0b3fc0 con 0x55b23110f4d0 2024-08-30T09:07:14.700 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.692+0000 7f4be3fff640 1 -- 172.21.15.42:0/1447703603 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 34) ==== 8366+0+0 (secure 0 0 0) 0x7f4bec0a6f30 con 0x55b23110f4d0 2024-08-30T09:07:14.700 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.692+0000 7f4c11bd8740 10 client.26257 fetch_fsmap finished waiting for FSMap version 34 2024-08-30T09:07:14.700 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.692+0000 7f4c11bd8740 10 client.26257 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-30T09:07:14.700 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.692+0000 7f4c11bd8740 20 client.26257 populate_metadata read hostname 'smithi042' 2024-08-30T09:07:14.700 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.692+0000 7f4c11bd8740 1 --2- 172.21.15.42:0/1447703603 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55b2310eed80 0x55b2314ed4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:07:14.700 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.692+0000 7f4c11bd8740 4 client.26257 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-30T09:07:14.700 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.692+0000 7f4c11bd8740 1 -- 172.21.15.42:0/1447703603 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f4bfc036070 con 0x55b2310eed80 2024-08-30T09:07:14.700 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.693+0000 7f4c08a05640 1 --2- 172.21.15.42:0/1447703603 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55b2310eed80 0x55b2314ed4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:07:14.701 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.693+0000 7f4be3fff640 10 client.26257 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:07:14.701 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.693+0000 7f4c08a05640 1 --2- 172.21.15.42:0/1447703603 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55b2310eed80 0x55b2314ed4b0 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f7a207167e7a6026 server_cookie=b0e03dc56070ae36 in_seq=0 out_seq=0 2024-08-30T09:07:14.702 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.694+0000 7f4be3fff640 1 -- 172.21.15.42:0/1447703603 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f4bfc036070 con 0x55b2310eed80 2024-08-30T09:07:14.702 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.694+0000 7f4be3fff640 10 client.26257 handle_command_reply: tid=0 2024-08-30T09:07:14.702 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.694+0000 7f4c11bd8740 1 client.26257 shutdown 2024-08-30T09:07:14.702 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.694+0000 7f4c11bd8740 1 -- 172.21.15.42:0/1447703603 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b23110f4d0 msgr2=0x55b2314e02d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:07:14.702 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.694+0000 7f4c11bd8740 1 --2- 172.21.15.42:0/1447703603 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b23110f4d0 0x55b2314e02d0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f4bec0aac30 tx=0x7f4bec0a6430 comp rx=0 tx=0).stop 2024-08-30T09:07:14.702 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.694+0000 7f4c11bd8740 1 -- 172.21.15.42:0/1447703603 shutdown_connections 2024-08-30T09:07:14.702 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.694+0000 7f4c11bd8740 1 --2- 172.21.15.42:0/1447703603 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55b2310eed80 0x55b2314ed4b0 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:14.703 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.694+0000 7f4c11bd8740 1 --2- 172.21.15.42:0/1447703603 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b23110f4d0 0x55b2314e02d0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:14.703 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.694+0000 7f4c11bd8740 1 --2- 172.21.15.42:0/1447703603 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b2311301b0 0x55b2314e0e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:14.703 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.694+0000 7f4be3fff640 0 client.26257 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:07:14.703 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.694+0000 7f4c11bd8740 1 --2- 172.21.15.42:0/1447703603 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b2311072f0 0x55b2314dfc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:14.703 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.694+0000 7f4c11bd8740 1 -- 172.21.15.42:0/1447703603 >> 172.21.15.42:0/1447703603 conn(0x55b231124300 msgr2=0x55b2314d0170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:07:14.703 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.694+0000 7f4c11bd8740 1 -- 172.21.15.42:0/1447703603 shutdown_connections 2024-08-30T09:07:14.703 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.694+0000 7f4c11bd8740 1 -- 172.21.15.42:0/1447703603 wait complete. 2024-08-30T09:07:14.703 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.694+0000 7f4c11bd8740 20 client.26257 trim_cache size 0 max 16384 2024-08-30T09:07:14.715 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.711+0000 7f4c11bd8740 1 Processor -- start 2024-08-30T09:07:14.715 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.711+0000 7f4c11bd8740 1 -- start start 2024-08-30T09:07:14.715 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.711+0000 7f4c11bd8740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b2311072f0 0x55b2314d03a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:07:14.715 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.711+0000 7f4c11bd8740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b23110f4d0 0x55b2314e7d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:07:14.715 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.711+0000 7f4c03fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b23110f4d0 0x55b2314e7d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:07:14.715 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.711+0000 7f4c03fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b23110f4d0 0x55b2314e7d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43158/0 (socket says 172.21.15.42:43158) 2024-08-30T09:07:14.716 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.711+0000 7f4c03fff640 1 -- 172.21.15.42:0/3885336855 learned_addr learned my addr 172.21.15.42:0/3885336855 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:07:14.716 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.711+0000 7f4c08a05640 1 --2- 172.21.15.42:0/3885336855 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b2311072f0 0x55b2314d03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:07:14.716 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.711+0000 7f4c11bd8740 1 --2- 172.21.15.42:0/3885336855 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b2311301b0 0x55b2314cc400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:07:14.716 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.711+0000 7f4c11bd8740 1 -- 172.21.15.42:0/3885336855 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f4c041c6950 con 0x55b23110f4d0 2024-08-30T09:07:14.716 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.711+0000 7f4c11bd8740 1 -- 172.21.15.42:0/3885336855 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4c041c41c0 con 0x55b2311301b0 2024-08-30T09:07:14.716 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.711+0000 7f4c11bd8740 1 -- 172.21.15.42:0/3885336855 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f4c041c2010 con 0x55b2311072f0 2024-08-30T09:07:14.716 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.711+0000 7f4c08a05640 1 -- 172.21.15.42:0/3885336855 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b2311301b0 msgr2=0x55b2314cc400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:07:14.716 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.711+0000 7f4c08a05640 1 --2- 172.21.15.42:0/3885336855 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b2311301b0 0x55b2314cc400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:14.716 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.711+0000 7f4c08a05640 1 -- 172.21.15.42:0/3885336855 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b23110f4d0 msgr2=0x55b2314e7d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:07:14.716 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.711+0000 7f4c08a05640 1 --2- 172.21.15.42:0/3885336855 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b23110f4d0 0x55b2314e7d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:14.716 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.712+0000 7f4c08a05640 1 -- 172.21.15.42:0/3885336855 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b2314e11b0 con 0x55b2311072f0 2024-08-30T09:07:14.716 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.712+0000 7f4c03fff640 1 --2- 172.21.15.42:0/3885336855 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b23110f4d0 0x55b2314e7d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-30T09:07:14.717 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.712+0000 7f4c08a05640 1 --2- 172.21.15.42:0/3885336855 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b2311072f0 0x55b2314d03a0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f4bfc02a930 tx=0x7f4bfc02b070 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a1841a44c9a3582a server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:07:14.717 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.712+0000 7f4bfa7fc640 1 -- 172.21.15.42:0/3885336855 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bfc031b60 con 0x55b2311072f0 2024-08-30T09:07:14.717 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.712+0000 7f4bfa7fc640 1 -- 172.21.15.42:0/3885336855 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4bfc036050 con 0x55b2311072f0 2024-08-30T09:07:14.717 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.712+0000 7f4bfa7fc640 1 -- 172.21.15.42:0/3885336855 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bfc003bd0 con 0x55b2311072f0 2024-08-30T09:07:14.720 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.713+0000 7f4c11bd8740 1 -- 172.21.15.42:0/3885336855 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b2311072f0 msgr2=0x55b2314d03a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:07:14.720 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.713+0000 7f4c11bd8740 1 --2- 172.21.15.42:0/3885336855 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b2311072f0 0x55b2314d03a0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f4bfc02a930 tx=0x7f4bfc02b070 comp rx=0 tx=0).stop 2024-08-30T09:07:14.720 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.713+0000 7f4c11bd8740 1 -- 172.21.15.42:0/3885336855 shutdown_connections 2024-08-30T09:07:14.720 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.713+0000 7f4c11bd8740 1 --2- 172.21.15.42:0/3885336855 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b2311301b0 0x55b2314cc400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:14.720 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.713+0000 7f4c11bd8740 1 --2- 172.21.15.42:0/3885336855 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b23110f4d0 0x55b2314e7d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:14.720 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.713+0000 7f4c11bd8740 1 --2- 172.21.15.42:0/3885336855 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b2311072f0 0x55b2314d03a0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:14.720 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.713+0000 7f4c11bd8740 1 -- 172.21.15.42:0/3885336855 >> 172.21.15.42:0/3885336855 conn(0x55b231124300 msgr2=0x55b2314d6fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:07:14.720 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.713+0000 7f4c11bd8740 1 -- 172.21.15.42:0/3885336855 shutdown_connections 2024-08-30T09:07:14.720 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.713+0000 7f4c11bd8740 1 -- 172.21.15.42:0/3885336855 wait complete. 2024-08-30T09:07:14.721 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.714+0000 7f4c11bd8740 1 -- start start 2024-08-30T09:07:14.721 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.714+0000 7f4c11bd8740 1 Processor -- start 2024-08-30T09:07:14.721 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.714+0000 7f4c11bd8740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b2311072f0 0x55b2314e77c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:07:14.721 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.715+0000 7f4c11bd8740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b23110f4d0 0x55b2314e6480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:07:14.721 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.715+0000 7f4c08a05640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b2311072f0 0x55b2314e77c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:07:14.721 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.715+0000 7f4c08a05640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b2311072f0 0x55b2314e77c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:59150/0 (socket says 172.21.15.42:59150) 2024-08-30T09:07:14.721 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.715+0000 7f4c08a05640 1 -- 172.21.15.42:0/4062386514 learned_addr learned my addr 172.21.15.42:0/4062386514 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:07:14.721 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.715+0000 7f4c11bd8740 1 --2- 172.21.15.42:0/4062386514 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b2311301b0 0x55b2314e69c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:07:14.721 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.715+0000 7f4c11bd8740 1 -- 172.21.15.42:0/4062386514 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f4c041c2680 con 0x55b23110f4d0 2024-08-30T09:07:14.721 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.715+0000 7f4c09206640 1 --2- 172.21.15.42:0/4062386514 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b2311301b0 0x55b2314e69c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:07:14.721 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.715+0000 7f4c11bd8740 1 -- 172.21.15.42:0/4062386514 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4c041c3af0 con 0x55b2311301b0 2024-08-30T09:07:14.721 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.715+0000 7f4c03fff640 1 --2- 172.21.15.42:0/4062386514 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b23110f4d0 0x55b2314e6480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:07:14.722 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.716+0000 7f4c11bd8740 1 -- 172.21.15.42:0/4062386514 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f4c041c2d50 con 0x55b2311072f0 2024-08-30T09:07:14.722 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.716+0000 7f4c08a05640 1 -- 172.21.15.42:0/4062386514 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b2311301b0 msgr2=0x55b2314e69c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:07:14.722 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.716+0000 7f4c08a05640 1 --2- 172.21.15.42:0/4062386514 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b2311301b0 0x55b2314e69c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:14.722 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.716+0000 7f4c08a05640 1 -- 172.21.15.42:0/4062386514 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b23110f4d0 msgr2=0x55b2314e6480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:07:14.722 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.716+0000 7f4c08a05640 1 --2- 172.21.15.42:0/4062386514 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b23110f4d0 0x55b2314e6480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:14.722 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.716+0000 7f4c08a05640 1 -- 172.21.15.42:0/4062386514 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b2314e6f80 con 0x55b2311072f0 2024-08-30T09:07:14.722 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.716+0000 7f4c11bd8740 10 client.0 _ll_register_callbacks cb 0x7f4bec0a6b60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:07:14.722 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.716+0000 7f4c08a05640 1 --2- 172.21.15.42:0/4062386514 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b2311072f0 0x55b2314e77c0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f4bfc0098e0 tx=0x7f4bfc0307e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5ca6d1a3a580eb0b server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:07:14.722 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.716+0000 7f4be3fff640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-08-30T09:07:14.722 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.716+0000 7f4be3fff640 1 -- 172.21.15.42:0/4062386514 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bfc031680 con 0x55b2311072f0 2024-08-30T09:07:14.722 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.717+0000 7f4c11bd8740 1 -- 172.21.15.42:0/4062386514 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4bfc036050 con 0x55b2311072f0 2024-08-30T09:07:14.723 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.717+0000 7f4be3fff640 1 -- 172.21.15.42:0/4062386514 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f4bfc003970 con 0x55b2311072f0 2024-08-30T09:07:14.723 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.717+0000 7f4be3fff640 1 -- 172.21.15.42:0/4062386514 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f4bfc003b70 con 0x55b2311072f0 2024-08-30T09:07:14.724 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.717+0000 7f4c11bd8740 10 client.26263 fetch_fsmap learned FSMap version 34 2024-08-30T09:07:14.724 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.717+0000 7f4c11bd8740 1 -- 172.21.15.42:0/4062386514 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({fsmap=34}) -- 0x55b2314e5b60 con 0x55b2311072f0 2024-08-30T09:07:14.724 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.718+0000 7f4be3fff640 1 -- 172.21.15.42:0/4062386514 <== mon.1 v2:172.21.15.190:3300/0 4 ==== fsmap(e 34) ==== 8366+0+0 (secure 0 0 0) 0x7f4bfc057020 con 0x55b2311072f0 2024-08-30T09:07:14.724 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.718+0000 7f4c11bd8740 10 client.26263 fetch_fsmap finished waiting for FSMap version 34 2024-08-30T09:07:14.724 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.718+0000 7f4c11bd8740 10 client.26263 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-30T09:07:14.724 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.718+0000 7f4c11bd8740 20 client.26263 populate_metadata read hostname 'smithi042' 2024-08-30T09:07:14.725 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.718+0000 7f4c11bd8740 1 --2- 172.21.15.42:0/4062386514 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55b2310c41c0 0x55b231559ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:07:14.725 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.718+0000 7f4c11bd8740 4 client.26263 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-30T09:07:14.725 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.718+0000 7f4c11bd8740 1 -- 172.21.15.42:0/4062386514 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f4bfc036050 con 0x55b2310c41c0 2024-08-30T09:07:14.725 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.718+0000 7f4c03fff640 1 --2- 172.21.15.42:0/4062386514 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55b2310c41c0 0x55b231559ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:07:14.725 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.719+0000 7f4c03fff640 1 --2- 172.21.15.42:0/4062386514 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55b2310c41c0 0x55b231559ec0 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c71d71349f73f368 server_cookie=d8483f907d6cbf9c in_seq=0 out_seq=0 2024-08-30T09:07:14.725 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.719+0000 7f4be3fff640 10 client.26263 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:07:14.741 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.737+0000 7f4be3fff640 1 -- 172.21.15.42:0/4062386514 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f4bfc036050 con 0x55b2310c41c0 2024-08-30T09:07:14.741 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.737+0000 7f4be3fff640 10 client.26263 handle_command_reply: tid=0 2024-08-30T09:07:14.741 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.737+0000 7f4c11bd8740 1 client.26263 shutdown 2024-08-30T09:07:14.741 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.737+0000 7f4c11bd8740 1 -- 172.21.15.42:0/4062386514 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b2311072f0 msgr2=0x55b2314e77c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:07:14.741 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.737+0000 7f4c11bd8740 1 --2- 172.21.15.42:0/4062386514 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b2311072f0 0x55b2314e77c0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f4bfc0098e0 tx=0x7f4bfc0307e0 comp rx=0 tx=0).stop 2024-08-30T09:07:14.741 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.738+0000 7f4c11bd8740 1 -- 172.21.15.42:0/4062386514 shutdown_connections 2024-08-30T09:07:14.742 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.738+0000 7f4c11bd8740 1 --2- 172.21.15.42:0/4062386514 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55b2310c41c0 0x55b231559ec0 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:14.742 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.738+0000 7f4be3fff640 0 client.26263 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:07:14.742 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.738+0000 7f4c11bd8740 1 --2- 172.21.15.42:0/4062386514 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b2311301b0 0x55b2314e69c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:14.742 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.738+0000 7f4c11bd8740 1 --2- 172.21.15.42:0/4062386514 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b23110f4d0 0x55b2314e6480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:14.742 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.738+0000 7f4c11bd8740 1 --2- 172.21.15.42:0/4062386514 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b2311072f0 0x55b2314e77c0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:14.742 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.738+0000 7f4c11bd8740 1 -- 172.21.15.42:0/4062386514 >> 172.21.15.42:0/4062386514 conn(0x55b231124300 msgr2=0x55b2314dab10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:07:14.742 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.738+0000 7f4c11bd8740 1 -- 172.21.15.42:0/4062386514 shutdown_connections 2024-08-30T09:07:14.742 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.738+0000 7f4c11bd8740 1 -- 172.21.15.42:0/4062386514 wait complete. 2024-08-30T09:07:14.742 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.738+0000 7f4c11bd8740 20 client.26263 trim_cache size 0 max 16384 2024-08-30T09:07:14.743 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-08-30T09:07:14.743 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2024-08-30T09:07:14.743 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-08-30T09:07:14.743 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-08-30T09:07:14.744 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.739+0000 7f4c0ac69640 1 -- 172.21.15.42:0/2894941902 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f4bf00b4d40 msgr2=0x7f4bf00b7250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:07:14.744 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.739+0000 7f4c0ac69640 1 --2- 172.21.15.42:0/2894941902 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f4bf00b4d40 0x7f4bf00b7250 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f4bf4009aa0 tx=0x7f4bf4005f90 comp rx=0 tx=0).stop 2024-08-30T09:07:14.744 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.739+0000 7f4c0ac69640 1 -- 172.21.15.42:0/2894941902 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4c0410f770 msgr2=0x7f4c04118cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:07:14.744 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.739+0000 7f4c0ac69640 1 --2- 172.21.15.42:0/2894941902 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4c0410f770 0x7f4c04118cc0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f4bec00b4a0 tx=0x7f4bec00b960 comp rx=0 tx=0).stop 2024-08-30T09:07:14.744 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.739+0000 7f4c0ac69640 1 -- 172.21.15.42:0/2894941902 shutdown_connections 2024-08-30T09:07:14.744 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.739+0000 7f4c0ac69640 1 --2- 172.21.15.42:0/2894941902 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f4bf00b4d40 0x7f4bf00b7250 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:14.744 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.739+0000 7f4c0ac69640 1 --2- 172.21.15.42:0/2894941902 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4c04119200 0x7f4c04119610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:14.744 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.739+0000 7f4c0ac69640 1 --2- 172.21.15.42:0/2894941902 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4c0410f770 0x7f4c04118cc0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:14.744 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.739+0000 7f4c0ac69640 1 --2- 172.21.15.42:0/2894941902 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4c0410cb50 0x7f4c0411a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:14.744 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.739+0000 7f4c0ac69640 1 -- 172.21.15.42:0/2894941902 >> 172.21.15.42:0/2894941902 conn(0x7f4c040705b0 msgr2=0x7f4c0410b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:07:14.745 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.739+0000 7f4c0ac69640 1 -- 172.21.15.42:0/2894941902 shutdown_connections 2024-08-30T09:07:14.745 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:14.739+0000 7f4c0ac69640 1 -- 172.21.15.42:0/2894941902 wait complete. 2024-08-30T09:07:14.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:07:14 smithi167 ceph-mon[22063]: pgmap v567: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 17 MiB/s wr, 1.35k op/s 2024-08-30T09:07:14.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:07:14 smithi042 ceph-mon[18652]: pgmap v567: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 17 MiB/s wr, 1.35k op/s 2024-08-30T09:07:14.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:07:14 smithi190 ceph-mon[22001]: pgmap v567: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 17 MiB/s wr, 1.35k op/s 2024-08-30T09:07:15.452 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T09:07:15.953 INFO:teuthology.orchestra.run.smithi042.stderr:progress: 390.0 s, 570.6 tps, lat 181.513 ms stddev 232.554 2024-08-30T09:07:16.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.124+0000 7f3b3c161640 1 -- 172.21.15.42:0/857710565 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3b34108250 msgr2=0x7f3b3410ea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:07:16.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.124+0000 7f3b3c161640 1 --2- 172.21.15.42:0/857710565 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3b34108250 0x7f3b3410ea80 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f3b2c005fd0 tx=0x7f3b2c02a6b0 comp rx=0 tx=0).stop 2024-08-30T09:07:16.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.127+0000 7f3b3c161640 1 -- 172.21.15.42:0/857710565 shutdown_connections 2024-08-30T09:07:16.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.127+0000 7f3b3c161640 1 --2- 172.21.15.42:0/857710565 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3b34107900 0x7f3b34107d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:16.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.127+0000 7f3b3c161640 1 --2- 172.21.15.42:0/857710565 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3b34108250 0x7f3b3410ea80 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:16.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.127+0000 7f3b3c161640 1 --2- 172.21.15.42:0/857710565 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3b34102010 0x7f3b34102420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:16.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.127+0000 7f3b3c161640 1 -- 172.21.15.42:0/857710565 >> 172.21.15.42:0/857710565 conn(0x7f3b34077a60 msgr2=0x7f3b34077ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:07:16.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.133+0000 7f3b3c161640 1 -- 172.21.15.42:0/857710565 shutdown_connections 2024-08-30T09:07:16.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.136+0000 7f3b3c161640 1 -- 172.21.15.42:0/857710565 wait complete. 2024-08-30T09:07:16.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.137+0000 7f3b3c161640 1 Processor -- start 2024-08-30T09:07:16.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.141+0000 7f3b3c161640 1 -- start start 2024-08-30T09:07:16.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.141+0000 7f3b3c161640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3b34102010 0x7f3b34074c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:07:16.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.141+0000 7f3b3c161640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3b34107900 0x7f3b3406eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:07:16.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.141+0000 7f3b3c161640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3b34108250 0x7f3b3406f450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:07:16.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.141+0000 7f3b3c161640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f3b3406d440 con 0x7f3b34107900 2024-08-30T09:07:16.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.141+0000 7f3b3c161640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f3b3406d2c0 con 0x7f3b34102010 2024-08-30T09:07:16.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.141+0000 7f3b3c161640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f3b3406d5c0 con 0x7f3b34108250 2024-08-30T09:07:16.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.145+0000 7f3b39efd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3b34102010 0x7f3b34074c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:07:16.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.145+0000 7f3b39efd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3b34102010 0x7f3b34074c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:57018/0 (socket says 172.21.15.42:57018) 2024-08-30T09:07:16.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.145+0000 7f3b39efd640 1 -- 172.21.15.42:0/2766617602 learned_addr learned my addr 172.21.15.42:0/2766617602 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:07:16.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.145+0000 7f3b3a6fe640 1 --2- 172.21.15.42:0/2766617602 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3b34108250 0x7f3b3406f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:07:16.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.145+0000 7f3b396fc640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3b34107900 0x7f3b3406eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:07:16.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.151+0000 7f3b39efd640 1 -- 172.21.15.42:0/2766617602 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3b34108250 msgr2=0x7f3b3406f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:07:16.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.151+0000 7f3b39efd640 1 --2- 172.21.15.42:0/2766617602 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3b34108250 0x7f3b3406f450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:16.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.151+0000 7f3b39efd640 1 -- 172.21.15.42:0/2766617602 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3b34107900 msgr2=0x7f3b3406eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:07:16.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.151+0000 7f3b39efd640 1 --2- 172.21.15.42:0/2766617602 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3b34107900 0x7f3b3406eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:16.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.151+0000 7f3b39efd640 1 -- 172.21.15.42:0/2766617602 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b3406fbe0 con 0x7f3b34102010 2024-08-30T09:07:16.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.153+0000 7f3b39efd640 1 --2- 172.21.15.42:0/2766617602 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3b34102010 0x7f3b34074c10 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f3b3000daf0 tx=0x7f3b3000af60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:07:16.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.158+0000 7f3b2affd640 1 -- 172.21.15.42:0/2766617602 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b3001a070 con 0x7f3b34102010 2024-08-30T09:07:16.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.158+0000 7f3b2affd640 1 -- 172.21.15.42:0/2766617602 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3b3000cc70 con 0x7f3b34102010 2024-08-30T09:07:16.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.158+0000 7f3b2affd640 1 -- 172.21.15.42:0/2766617602 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b30014610 con 0x7f3b34102010 2024-08-30T09:07:16.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.158+0000 7f3b3c161640 1 -- 172.21.15.42:0/2766617602 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b341a6af0 con 0x7f3b34102010 2024-08-30T09:07:16.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.158+0000 7f3b3c161640 1 -- 172.21.15.42:0/2766617602 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b341a6fb0 con 0x7f3b34102010 2024-08-30T09:07:16.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.168+0000 7f3b426d6740 1 Processor -- start 2024-08-30T09:07:16.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.169+0000 7f3b426d6740 1 -- start start 2024-08-30T09:07:16.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.169+0000 7f3b426d6740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a73a1d8070 0x55a73a57d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:07:16.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.169+0000 7f3b426d6740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a73a1af1b0 0x55a73a57d800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:07:16.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.169+0000 7f3b426d6740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a73a1b7390 0x55a73a57dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:07:16.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.169+0000 7f3b426d6740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55a73a4f4630 con 0x55a73a1d8070 2024-08-30T09:07:16.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.169+0000 7f3b426d6740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55a73a4e3bc0 con 0x55a73a1af1b0 2024-08-30T09:07:16.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.169+0000 7f3b426d6740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55a73a4e7b70 con 0x55a73a1b7390 2024-08-30T09:07:16.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.171+0000 7f3b2affd640 1 -- 172.21.15.42:0/2766617602 <== mon.2 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7f3b3001d070 con 0x7f3b34102010 2024-08-30T09:07:16.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.172+0000 7f3b3a6fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a73a1d8070 0x55a73a57d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:07:16.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.172+0000 7f3b3a6fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a73a1d8070 0x55a73a57d2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43186/0 (socket says 172.21.15.42:43186) 2024-08-30T09:07:16.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.172+0000 7f3b39efd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a73a1af1b0 0x55a73a57d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:07:16.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.172+0000 7f3b3a6fe640 1 -- 172.21.15.42:0/3897737014 learned_addr learned my addr 172.21.15.42:0/3897737014 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:07:16.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.172+0000 7f3b2affd640 1 --2- 172.21.15.42:0/2766617602 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f3b200b4d40 0x7f3b200b7250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:07:16.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.172+0000 7f3b2affd640 1 -- 172.21.15.42:0/2766617602 <== mon.2 v2:172.21.15.167:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f3b300e4710 con 0x7f3b34102010 2024-08-30T09:07:16.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.173+0000 7f3b396fc640 1 --2- 172.21.15.42:0/3897737014 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a73a1b7390 0x55a73a57dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:07:16.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.176+0000 7f3b3a6fe640 1 -- 172.21.15.42:0/3897737014 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a73a1b7390 msgr2=0x55a73a57dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:07:16.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.176+0000 7f3b3a6fe640 1 --2- 172.21.15.42:0/3897737014 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a73a1b7390 0x55a73a57dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:16.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.176+0000 7f3b3a6fe640 1 -- 172.21.15.42:0/3897737014 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a73a1af1b0 msgr2=0x55a73a57d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:07:16.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.176+0000 7f3b3a6fe640 1 --2- 172.21.15.42:0/3897737014 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a73a1af1b0 0x55a73a57d800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:16.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.176+0000 7f3b3a6fe640 1 -- 172.21.15.42:0/3897737014 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a73a243650 con 0x55a73a1d8070 2024-08-30T09:07:16.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.177+0000 7f3b3a6fe640 1 --2- 172.21.15.42:0/2766617602 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f3b200b4d40 0x7f3b200b7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:07:16.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.187+0000 7f3b3a6fe640 1 --2- 172.21.15.42:0/3897737014 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a73a1d8070 0x55a73a57d2c0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f3b2c005fd0 tx=0x7f3b2c031f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a03a069e022f8fa server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:07:16.195 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.192+0000 7f3b3a6fe640 1 --2- 172.21.15.42:0/2766617602 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f3b200b4d40 0x7f3b200b7250 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f3b2c034000 tx=0x7f3b2c047000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:07:16.195 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.192+0000 7f3b1ffff640 1 -- 172.21.15.42:0/3897737014 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b2c043040 con 0x55a73a1d8070 2024-08-30T09:07:16.195 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.192+0000 7f3b1ffff640 1 -- 172.21.15.42:0/3897737014 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3b2c0044f0 con 0x55a73a1d8070 2024-08-30T09:07:16.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.193+0000 7f3b426d6740 1 -- 172.21.15.42:0/3897737014 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a73a1d8070 msgr2=0x55a73a57d2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:07:16.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.194+0000 7f3b426d6740 1 --2- 172.21.15.42:0/3897737014 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a73a1d8070 0x55a73a57d2c0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f3b2c005fd0 tx=0x7f3b2c031f90 comp rx=0 tx=0).stop 2024-08-30T09:07:16.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.195+0000 7f3b426d6740 1 -- 172.21.15.42:0/3897737014 shutdown_connections 2024-08-30T09:07:16.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.195+0000 7f3b426d6740 1 --2- 172.21.15.42:0/3897737014 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a73a1b7390 0x55a73a57dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:16.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.195+0000 7f3b426d6740 1 --2- 172.21.15.42:0/3897737014 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a73a1af1b0 0x55a73a57d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:16.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.195+0000 7f3b426d6740 1 --2- 172.21.15.42:0/3897737014 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a73a1d8070 0x55a73a57d2c0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:16.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.195+0000 7f3b426d6740 1 -- 172.21.15.42:0/3897737014 >> 172.21.15.42:0/3897737014 conn(0x55a73a1cc460 msgr2=0x55a73a57b1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:07:16.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.203+0000 7f3b426d6740 1 -- 172.21.15.42:0/3897737014 shutdown_connections 2024-08-30T09:07:16.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.204+0000 7f3b426d6740 1 -- 172.21.15.42:0/3897737014 wait complete. 2024-08-30T09:07:16.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.205+0000 7f3b426d6740 1 -- start start 2024-08-30T09:07:16.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.205+0000 7f3b426d6740 1 Processor -- start 2024-08-30T09:07:16.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.206+0000 7f3b426d6740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a73a1af1b0 0x55a73a5889c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:07:16.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.206+0000 7f3b426d6740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a73a1b7390 0x55a73a588f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:07:16.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.206+0000 7f3b426d6740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a73a1d8070 0x55a73a594e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:07:16.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.206+0000 7f3b426d6740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55a73a346700 con 0x55a73a1af1b0 2024-08-30T09:07:16.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.206+0000 7f3b426d6740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55a73a46a700 con 0x55a73a1b7390 2024-08-30T09:07:16.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.206+0000 7f3b426d6740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55a73a58ba90 con 0x55a73a1d8070 2024-08-30T09:07:16.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.206+0000 7f3b426d6740 10 client.0 _ll_register_callbacks cb 0x55a73a499dc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:07:16.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.207+0000 7f3b396fc640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a73a1b7390 0x55a73a588f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:07:16.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.207+0000 7f3b396fc640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a73a1b7390 0x55a73a588f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:57030/0 (socket says 172.21.15.42:57030) 2024-08-30T09:07:16.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.207+0000 7f3b396fc640 1 -- 172.21.15.42:0/440860324 learned_addr learned my addr 172.21.15.42:0/440860324 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:07:16.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.207+0000 7f3b3a6fe640 1 --2- 172.21.15.42:0/440860324 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a73a1d8070 0x55a73a594e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:07:16.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.210+0000 7f3b396fc640 1 -- 172.21.15.42:0/440860324 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a73a1d8070 msgr2=0x55a73a594e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:07:16.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.210+0000 7f3b396fc640 1 --2- 172.21.15.42:0/440860324 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a73a1d8070 0x55a73a594e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:16.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.210+0000 7f3b396fc640 1 -- 172.21.15.42:0/440860324 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a73a1af1b0 msgr2=0x55a73a5889c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:07:16.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.210+0000 7f3b396fc640 1 --2- 172.21.15.42:0/440860324 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a73a1af1b0 0x55a73a5889c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:16.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.210+0000 7f3b396fc640 1 -- 172.21.15.42:0/440860324 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a73a4a38b0 con 0x55a73a1b7390 2024-08-30T09:07:16.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.210+0000 7f3b396fc640 1 --2- 172.21.15.42:0/440860324 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a73a1b7390 0x55a73a588f00 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f3b240107c0 tx=0x7f3b24010c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ad6096b10b0ae132 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:07:16.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.210+0000 7f3b1d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-30T09:07:16.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.218+0000 7f3b1d7fa640 1 -- 172.21.15.42:0/440860324 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b2400cb10 con 0x55a73a1b7390 2024-08-30T09:07:16.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.218+0000 7f3b426d6740 1 -- 172.21.15.42:0/440860324 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3b2c0044f0 con 0x55a73a1b7390 2024-08-30T09:07:16.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.218+0000 7f3b1d7fa640 1 -- 172.21.15.42:0/440860324 <== mon.2 v2:172.21.15.167:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f3b2400ccb0 con 0x55a73a1b7390 2024-08-30T09:07:16.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.218+0000 7f3b1d7fa640 1 -- 172.21.15.42:0/440860324 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f3b2401f520 con 0x55a73a1b7390 2024-08-30T09:07:16.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.218+0000 7f3b426d6740 10 client.26288 fetch_fsmap learned FSMap version 34 2024-08-30T09:07:16.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.218+0000 7f3b426d6740 1 -- 172.21.15.42:0/440860324 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55a73a597890 con 0x55a73a1b7390 2024-08-30T09:07:16.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.222+0000 7f3b1d7fa640 1 -- 172.21.15.42:0/440860324 <== mon.2 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3b2c0044f0 con 0x55a73a1b7390 2024-08-30T09:07:16.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.222+0000 7f3b1d7fa640 1 -- 172.21.15.42:0/440860324 <== mon.2 v2:172.21.15.167:3300/0 5 ==== fsmap(e 34) ==== 8366+0+0 (secure 0 0 0) 0x7f3b24011440 con 0x55a73a1b7390 2024-08-30T09:07:16.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.222+0000 7f3b426d6740 10 client.26288 fetch_fsmap finished waiting for FSMap version 34 2024-08-30T09:07:16.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.222+0000 7f3b426d6740 10 client.26288 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-30T09:07:16.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.222+0000 7f3b426d6740 20 client.26288 populate_metadata read hostname 'smithi042' 2024-08-30T09:07:16.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.222+0000 7f3b426d6740 1 --2- 172.21.15.42:0/440860324 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55a73a1969f0 0x55a73a59b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:07:16.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.222+0000 7f3b426d6740 4 client.26288 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-30T09:07:16.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.222+0000 7f3b426d6740 1 -- 172.21.15.42:0/440860324 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a73a4f6580 con 0x55a73a1969f0 2024-08-30T09:07:16.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.224+0000 7f3b39efd640 1 --2- 172.21.15.42:0/440860324 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55a73a1969f0 0x55a73a59b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:07:16.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.227+0000 7f3b39efd640 1 --2- 172.21.15.42:0/440860324 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55a73a1969f0 0x55a73a59b2b0 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=71378b3ea650f7f1 server_cookie=474162c30e71a50e in_seq=0 out_seq=0 2024-08-30T09:07:16.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.227+0000 7f3b1d7fa640 10 client.26288 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:07:16.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.228+0000 7f3b1d7fa640 1 -- 172.21.15.42:0/440860324 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a73a4f6580 con 0x55a73a1969f0 2024-08-30T09:07:16.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.228+0000 7f3b1d7fa640 10 client.26288 handle_command_reply: tid=0 2024-08-30T09:07:16.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.228+0000 7f3b426d6740 1 client.26288 shutdown 2024-08-30T09:07:16.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.237+0000 7f3b426d6740 1 -- 172.21.15.42:0/440860324 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a73a1b7390 msgr2=0x55a73a588f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:07:16.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.237+0000 7f3b426d6740 1 --2- 172.21.15.42:0/440860324 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a73a1b7390 0x55a73a588f00 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f3b240107c0 tx=0x7f3b24010c80 comp rx=0 tx=0).stop 2024-08-30T09:07:16.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.237+0000 7f3b426d6740 1 -- 172.21.15.42:0/440860324 shutdown_connections 2024-08-30T09:07:16.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.237+0000 7f3b426d6740 1 --2- 172.21.15.42:0/440860324 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55a73a1969f0 0x55a73a59b2b0 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:16.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.237+0000 7f3b426d6740 1 --2- 172.21.15.42:0/440860324 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a73a1d8070 0x55a73a594e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:16.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.237+0000 7f3b426d6740 1 --2- 172.21.15.42:0/440860324 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a73a1b7390 0x55a73a588f00 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:16.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.237+0000 7f3b426d6740 1 --2- 172.21.15.42:0/440860324 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a73a1af1b0 0x55a73a5889c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:16.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.237+0000 7f3b426d6740 1 -- 172.21.15.42:0/440860324 >> 172.21.15.42:0/440860324 conn(0x55a73a1cc460 msgr2=0x55a73a5804a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:07:16.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.239+0000 7f3b1d7fa640 0 client.26288 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:07:16.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.240+0000 7f3b426d6740 1 -- 172.21.15.42:0/440860324 shutdown_connections 2024-08-30T09:07:16.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.242+0000 7f3b426d6740 1 -- 172.21.15.42:0/440860324 wait complete. 2024-08-30T09:07:16.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.242+0000 7f3b426d6740 20 client.26288 trim_cache size 0 max 16384 2024-08-30T09:07:16.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.277+0000 7f3b426d6740 1 Processor -- start 2024-08-30T09:07:16.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.277+0000 7f3b426d6740 1 -- start start 2024-08-30T09:07:16.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.278+0000 7f3b426d6740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a73a1af1b0 0x55a73a5878b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:07:16.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.278+0000 7f3b426d6740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a73a1b7390 0x55a73a57ea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:07:16.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.278+0000 7f3b39efd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a73a1af1b0 0x55a73a5878b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:07:16.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.278+0000 7f3b39efd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a73a1af1b0 0x55a73a5878b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:57040/0 (socket says 172.21.15.42:57040) 2024-08-30T09:07:16.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.278+0000 7f3b39efd640 1 -- 172.21.15.42:0/3442359423 learned_addr learned my addr 172.21.15.42:0/3442359423 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:07:16.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.278+0000 7f3b426d6740 1 --2- 172.21.15.42:0/3442359423 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a73a1d8070 0x55a73a57a780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:07:16.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.279+0000 7f3b426d6740 1 -- 172.21.15.42:0/3442359423 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f3b18012640 con 0x55a73a1b7390 2024-08-30T09:07:16.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.279+0000 7f3b426d6740 1 -- 172.21.15.42:0/3442359423 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f3b1800feb0 con 0x55a73a1af1b0 2024-08-30T09:07:16.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.279+0000 7f3b426d6740 1 -- 172.21.15.42:0/3442359423 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f3b1800dd00 con 0x55a73a1d8070 2024-08-30T09:07:16.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.279+0000 7f3b3a6fe640 1 --2- 172.21.15.42:0/3442359423 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a73a1d8070 0x55a73a57a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:07:16.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.279+0000 7f3b396fc640 1 --2- 172.21.15.42:0/3442359423 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a73a1b7390 0x55a73a57ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:07:16.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.279+0000 7f3b39efd640 1 -- 172.21.15.42:0/3442359423 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a73a1d8070 msgr2=0x55a73a57a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:07:16.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.279+0000 7f3b39efd640 1 --2- 172.21.15.42:0/3442359423 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a73a1d8070 0x55a73a57a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:16.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.279+0000 7f3b39efd640 1 -- 172.21.15.42:0/3442359423 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a73a1b7390 msgr2=0x55a73a57ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:07:16.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.279+0000 7f3b39efd640 1 --2- 172.21.15.42:0/3442359423 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a73a1b7390 0x55a73a57ea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:16.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.280+0000 7f3b39efd640 1 -- 172.21.15.42:0/3442359423 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a73a589a60 con 0x55a73a1af1b0 2024-08-30T09:07:16.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.280+0000 7f3b39efd640 1 --2- 172.21.15.42:0/3442359423 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a73a1af1b0 0x55a73a5878b0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f3b30005c20 tx=0x7f3b300e4960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b84a5b56d23eb660 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:07:16.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.281+0000 7f3b1f7fe640 1 -- 172.21.15.42:0/3442359423 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b300af5c0 con 0x55a73a1af1b0 2024-08-30T09:07:16.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.281+0000 7f3b1f7fe640 1 -- 172.21.15.42:0/3442359423 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3b300b4070 con 0x55a73a1af1b0 2024-08-30T09:07:16.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.281+0000 7f3b1f7fe640 1 -- 172.21.15.42:0/3442359423 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b300aa060 con 0x55a73a1af1b0 2024-08-30T09:07:16.285 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.282+0000 7f3b426d6740 1 -- 172.21.15.42:0/3442359423 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a73a1af1b0 msgr2=0x55a73a5878b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:07:16.285 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.282+0000 7f3b426d6740 1 --2- 172.21.15.42:0/3442359423 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a73a1af1b0 0x55a73a5878b0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f3b30005c20 tx=0x7f3b300e4960 comp rx=0 tx=0).stop 2024-08-30T09:07:16.285 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.282+0000 7f3b426d6740 1 -- 172.21.15.42:0/3442359423 shutdown_connections 2024-08-30T09:07:16.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.282+0000 7f3b426d6740 1 --2- 172.21.15.42:0/3442359423 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a73a1d8070 0x55a73a57a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:16.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.282+0000 7f3b426d6740 1 --2- 172.21.15.42:0/3442359423 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a73a1b7390 0x55a73a57ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:16.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.282+0000 7f3b426d6740 1 --2- 172.21.15.42:0/3442359423 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a73a1af1b0 0x55a73a5878b0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:16.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.283+0000 7f3b426d6740 1 -- 172.21.15.42:0/3442359423 >> 172.21.15.42:0/3442359423 conn(0x55a73a1cc460 msgr2=0x55a73a57e2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:07:16.290 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.288+0000 7f3b426d6740 1 -- 172.21.15.42:0/3442359423 shutdown_connections 2024-08-30T09:07:16.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.288+0000 7f3b426d6740 1 -- 172.21.15.42:0/3442359423 wait complete. 2024-08-30T09:07:16.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.288+0000 7f3b426d6740 1 -- start start 2024-08-30T09:07:16.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.289+0000 7f3b426d6740 1 Processor -- start 2024-08-30T09:07:16.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.289+0000 7f3b426d6740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a73a1af1b0 0x55a73a591550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:07:16.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.290+0000 7f3b426d6740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a73a1b7390 0x55a73a58ca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:07:16.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.290+0000 7f3b426d6740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a73a1d8070 0x55a73a58cf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:07:16.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.290+0000 7f3b396fc640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a73a1b7390 0x55a73a58ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:07:16.294 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.290+0000 7f3b396fc640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a73a1b7390 0x55a73a58ca00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:59204/0 (socket says 172.21.15.42:59204) 2024-08-30T09:07:16.294 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.290+0000 7f3b396fc640 1 -- 172.21.15.42:0/1369635916 learned_addr learned my addr 172.21.15.42:0/1369635916 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:07:16.294 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.290+0000 7f3b426d6740 1 -- 172.21.15.42:0/1369635916 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f3b1800e370 con 0x55a73a1d8070 2024-08-30T09:07:16.294 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.290+0000 7f3b426d6740 1 -- 172.21.15.42:0/1369635916 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f3b1800f7e0 con 0x55a73a1af1b0 2024-08-30T09:07:16.294 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.290+0000 7f3b426d6740 1 -- 172.21.15.42:0/1369635916 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f3b1800ea40 con 0x55a73a1b7390 2024-08-30T09:07:16.294 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.291+0000 7f3b426d6740 10 client.0 _ll_register_callbacks cb 0x7f3b2400e3f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:07:16.297 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.295+0000 7f3b39efd640 1 --2- 172.21.15.42:0/1369635916 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a73a1af1b0 0x55a73a591550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:07:16.298 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.295+0000 7f3b396fc640 1 -- 172.21.15.42:0/1369635916 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a73a1af1b0 msgr2=0x55a73a591550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:07:16.298 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.295+0000 7f3b396fc640 1 --2- 172.21.15.42:0/1369635916 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a73a1af1b0 0x55a73a591550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:16.298 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.295+0000 7f3b396fc640 1 -- 172.21.15.42:0/1369635916 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a73a1d8070 msgr2=0x55a73a58cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:07:16.298 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.295+0000 7f3b396fc640 1 --2- 172.21.15.42:0/1369635916 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a73a1d8070 0x55a73a58cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:16.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.295+0000 7f3b396fc640 1 -- 172.21.15.42:0/1369635916 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a73a58d6b0 con 0x55a73a1b7390 2024-08-30T09:07:16.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.301+0000 7f3b396fc640 1 --2- 172.21.15.42:0/1369635916 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a73a1b7390 0x55a73a58ca00 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f3b240117e0 tx=0x7f3b2401f430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=909f23f2bd67db7c server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:07:16.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.301+0000 7f3b1d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-08-30T09:07:16.308 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.305+0000 7f3b1d7fa640 1 -- 172.21.15.42:0/1369635916 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b24011c50 con 0x55a73a1b7390 2024-08-30T09:07:16.308 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.305+0000 7f3b1d7fa640 1 -- 172.21.15.42:0/1369635916 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f3b24011df0 con 0x55a73a1b7390 2024-08-30T09:07:16.308 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.305+0000 7f3b426d6740 1 -- 172.21.15.42:0/1369635916 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3b300b4070 con 0x55a73a1b7390 2024-08-30T09:07:16.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.307+0000 7f3b1d7fa640 1 -- 172.21.15.42:0/1369635916 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f3b2403a050 con 0x55a73a1b7390 2024-08-30T09:07:16.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.307+0000 7f3b426d6740 10 client.26278 fetch_fsmap learned FSMap version 34 2024-08-30T09:07:16.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.308+0000 7f3b426d6740 1 -- 172.21.15.42:0/1369635916 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55a73a605c20 con 0x55a73a1b7390 2024-08-30T09:07:16.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.315+0000 7f3b1d7fa640 1 -- 172.21.15.42:0/1369635916 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3b2402d070 con 0x55a73a1b7390 2024-08-30T09:07:16.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.315+0000 7f3b1d7fa640 1 -- 172.21.15.42:0/1369635916 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 34) ==== 8366+0+0 (secure 0 0 0) 0x7f3b2401f6c0 con 0x55a73a1b7390 2024-08-30T09:07:16.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.315+0000 7f3b426d6740 10 client.26278 fetch_fsmap finished waiting for FSMap version 34 2024-08-30T09:07:16.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.315+0000 7f3b426d6740 10 client.26278 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-30T09:07:16.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.315+0000 7f3b426d6740 20 client.26278 populate_metadata read hostname 'smithi042' 2024-08-30T09:07:16.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.315+0000 7f3b426d6740 1 --2- 172.21.15.42:0/1369635916 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55a73a16bda0 0x55a73a6083c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:07:16.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.315+0000 7f3b426d6740 4 client.26278 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-30T09:07:16.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.315+0000 7f3b426d6740 1 -- 172.21.15.42:0/1369635916 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f3b300b4070 con 0x55a73a16bda0 2024-08-30T09:07:16.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.319+0000 7f3b39efd640 1 --2- 172.21.15.42:0/1369635916 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55a73a16bda0 0x55a73a6083c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:07:16.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.327+0000 7f3b39efd640 1 --2- 172.21.15.42:0/1369635916 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55a73a16bda0 0x55a73a6083c0 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fe7f900ed79b2886 server_cookie=45735ec29cd8398f in_seq=0 out_seq=0 2024-08-30T09:07:16.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.328+0000 7f3b1d7fa640 10 client.26278 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:07:16.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.330+0000 7f3b1d7fa640 1 -- 172.21.15.42:0/1369635916 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+2362 (crc 0 0 0) 0x7f3b300b4070 con 0x55a73a16bda0 2024-08-30T09:07:16.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.330+0000 7f3b1d7fa640 10 client.26278 handle_command_reply: tid=0 2024-08-30T09:07:16.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.330+0000 7f3b426d6740 1 client.26278 shutdown 2024-08-30T09:07:16.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.334+0000 7f3b426d6740 1 -- 172.21.15.42:0/1369635916 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a73a1b7390 msgr2=0x55a73a58ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:07:16.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.334+0000 7f3b426d6740 1 --2- 172.21.15.42:0/1369635916 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a73a1b7390 0x55a73a58ca00 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f3b240117e0 tx=0x7f3b2401f430 comp rx=0 tx=0).stop 2024-08-30T09:07:16.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.338+0000 7f3b426d6740 1 -- 172.21.15.42:0/1369635916 shutdown_connections 2024-08-30T09:07:16.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.338+0000 7f3b426d6740 1 --2- 172.21.15.42:0/1369635916 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55a73a16bda0 0x55a73a6083c0 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:16.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.338+0000 7f3b426d6740 1 --2- 172.21.15.42:0/1369635916 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a73a1b7390 0x55a73a58ca00 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:16.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.338+0000 7f3b426d6740 1 --2- 172.21.15.42:0/1369635916 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a73a1d8070 0x55a73a58cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:16.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.338+0000 7f3b426d6740 1 --2- 172.21.15.42:0/1369635916 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a73a1af1b0 0x55a73a591550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:16.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.338+0000 7f3b1d7fa640 0 client.26278 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:07:16.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.338+0000 7f3b426d6740 1 -- 172.21.15.42:0/1369635916 >> 172.21.15.42:0/1369635916 conn(0x55a73a1cc460 msgr2=0x55a73a57eb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:07:16.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.338+0000 7f3b426d6740 1 -- 172.21.15.42:0/1369635916 shutdown_connections 2024-08-30T09:07:16.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.338+0000 7f3b426d6740 1 -- 172.21.15.42:0/1369635916 wait complete. 2024-08-30T09:07:16.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.338+0000 7f3b426d6740 20 client.26278 trim_cache size 0 max 16384 2024-08-30T09:07:16.342 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-08-30T09:07:16.342 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "no active scrubs running", 2024-08-30T09:07:16.342 INFO:teuthology.orchestra.run.smithi042.stdout: "scrubs": {}, 2024-08-30T09:07:16.343 INFO:teuthology.orchestra.run.smithi042.stdout: "scrub_stats": { 2024-08-30T09:07:16.343 INFO:teuthology.orchestra.run.smithi042.stdout: "665ec740-7254-49cb-84ef-deff71c461b0": { 2024-08-30T09:07:16.343 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:06:37.144035+0000", 2024-08-30T09:07:16.343 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:07:16.343 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:07:16.343 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:07:16.343 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:07:16.343 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:07:16.343 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:07:16.343 INFO:teuthology.orchestra.run.smithi042.stdout: "1af4252a-4a99-47e9-bbf0-becf5e573dd5": { 2024-08-30T09:07:16.343 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:05:48.565881+0000", 2024-08-30T09:07:16.344 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:07:16.344 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:07:16.344 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:07:16.344 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:07:16.344 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:07:16.344 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:07:16.344 INFO:teuthology.orchestra.run.smithi042.stdout: "347d05ed-4bd3-4590-b099-314f719aab39": { 2024-08-30T09:07:16.344 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:05:00.439721+0000", 2024-08-30T09:07:16.344 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:07:16.344 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:07:16.344 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:07:16.345 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:07:16.345 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:07:16.345 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:07:16.345 INFO:teuthology.orchestra.run.smithi042.stdout: "88cd7478-2b20-47de-ba51-256479749232": { 2024-08-30T09:07:16.345 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:04:08.922625+0000", 2024-08-30T09:07:16.345 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:07:16.345 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:07:16.345 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:07:16.345 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:07:16.346 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:07:16.346 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:07:16.346 INFO:teuthology.orchestra.run.smithi042.stdout: "749f4867-0b70-49e5-af2d-8362a240ebc3": { 2024-08-30T09:07:16.346 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:03:16.386554+0000", 2024-08-30T09:07:16.346 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:07:16.346 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:07:16.346 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:07:16.346 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:07:16.346 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:07:16.346 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:07:16.346 INFO:teuthology.orchestra.run.smithi042.stdout: "85134e4b-ab4a-4885-b07e-a3ad01eaf13d": { 2024-08-30T09:07:16.346 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:01:27.738462+0000", 2024-08-30T09:07:16.347 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:07:16.347 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:07:16.347 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:07:16.347 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:07:16.347 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:07:16.347 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:07:16.347 INFO:teuthology.orchestra.run.smithi042.stdout: "c827ad7a-44ee-4e1d-9bb3-51fb7ab00c73": { 2024-08-30T09:07:16.347 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:02:21.686517+0000", 2024-08-30T09:07:16.347 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:07:16.347 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:07:16.347 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:07:16.347 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:07:16.348 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:07:16.348 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:07:16.348 INFO:teuthology.orchestra.run.smithi042.stdout: "5e2ed104-0625-478a-a78b-584e7f4c2817": { 2024-08-30T09:07:16.348 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:00:33.928400+0000", 2024-08-30T09:07:16.348 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:07:16.348 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:07:16.348 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:07:16.348 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:07:16.348 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:07:16.348 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:07:16.348 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:07:16.349 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-08-30T09:07:16.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.339+0000 7f3b3c161640 1 -- 172.21.15.42:0/2766617602 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f3b200b4d40 msgr2=0x7f3b200b7250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:07:16.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.339+0000 7f3b3c161640 1 --2- 172.21.15.42:0/2766617602 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f3b200b4d40 0x7f3b200b7250 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f3b2c034000 tx=0x7f3b2c047000 comp rx=0 tx=0).stop 2024-08-30T09:07:16.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.339+0000 7f3b3c161640 1 -- 172.21.15.42:0/2766617602 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3b34102010 msgr2=0x7f3b34074c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:07:16.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.339+0000 7f3b3c161640 1 --2- 172.21.15.42:0/2766617602 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3b34102010 0x7f3b34074c10 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f3b3000daf0 tx=0x7f3b3000af60 comp rx=0 tx=0).stop 2024-08-30T09:07:16.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.339+0000 7f3b3c161640 1 -- 172.21.15.42:0/2766617602 shutdown_connections 2024-08-30T09:07:16.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.339+0000 7f3b3c161640 1 --2- 172.21.15.42:0/2766617602 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f3b200b4d40 0x7f3b200b7250 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:16.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.339+0000 7f3b3c161640 1 --2- 172.21.15.42:0/2766617602 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3b34108250 0x7f3b3406f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:16.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.339+0000 7f3b3c161640 1 --2- 172.21.15.42:0/2766617602 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3b34107900 0x7f3b3406eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:16.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.339+0000 7f3b3c161640 1 --2- 172.21.15.42:0/2766617602 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3b34102010 0x7f3b34074c10 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:16.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.339+0000 7f3b3c161640 1 -- 172.21.15.42:0/2766617602 >> 172.21.15.42:0/2766617602 conn(0x7f3b34077a60 msgr2=0x7f3b341058f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:07:16.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.339+0000 7f3b3c161640 1 -- 172.21.15.42:0/2766617602 shutdown_connections 2024-08-30T09:07:16.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:16.339+0000 7f3b3c161640 1 -- 172.21.15.42:0/2766617602 wait complete. 2024-08-30T09:07:16.766 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:07:16 smithi042 ceph-mon[18652]: pgmap v568: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 14 MiB/s wr, 1.18k op/s 2024-08-30T09:07:16.766 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:07:16 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-08-30T09:07:16.767 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:07:16 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-08-30T09:07:16.767 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:07:16 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-08-30T09:07:16.767 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:07:16 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-08-30T09:07:16.767 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:07:16 smithi042 ceph-mon[18652]: Adjusting osd_memory_target on smithi042 to 177.0M 2024-08-30T09:07:16.767 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:07:16 smithi042 ceph-mon[18652]: Unable to set osd_memory_target on smithi042 to 185699942: error parsing value: Value '185699942' is below minimum 939524096 2024-08-30T09:07:16.767 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:07:16 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T09:07:16.767 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:07:16 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-30T09:07:16.767 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:07:16 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T09:07:16.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:07:16 smithi167 ceph-mon[22063]: pgmap v568: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 14 MiB/s wr, 1.18k op/s 2024-08-30T09:07:16.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:07:16 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-08-30T09:07:16.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:07:16 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-08-30T09:07:16.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:07:16 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-08-30T09:07:16.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:07:16 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-08-30T09:07:16.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:07:16 smithi167 ceph-mon[22063]: Adjusting osd_memory_target on smithi042 to 177.0M 2024-08-30T09:07:16.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:07:16 smithi167 ceph-mon[22063]: Unable to set osd_memory_target on smithi042 to 185699942: error parsing value: Value '185699942' is below minimum 939524096 2024-08-30T09:07:16.828 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:07:16 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T09:07:16.828 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:07:16 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-30T09:07:16.828 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:07:16 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T09:07:16.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:07:16 smithi190 ceph-mon[22001]: pgmap v568: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 14 MiB/s wr, 1.18k op/s 2024-08-30T09:07:16.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:07:16 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-08-30T09:07:16.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:07:16 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-08-30T09:07:16.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:07:16 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-08-30T09:07:16.937 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:07:16 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-08-30T09:07:16.937 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:07:16 smithi190 ceph-mon[22001]: Adjusting osd_memory_target on smithi042 to 177.0M 2024-08-30T09:07:16.937 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:07:16 smithi190 ceph-mon[22001]: Unable to set osd_memory_target on smithi042 to 185699942: error parsing value: Value '185699942' is below minimum 939524096 2024-08-30T09:07:16.937 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:07:16 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T09:07:16.937 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:07:16 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-30T09:07:16.937 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:07:16 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T09:07:17.286 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-08-30T09:07:17.286 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph fs dump --format=json 2024-08-30T09:07:17.857 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T09:07:18.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:07:18 smithi167 ceph-mon[22063]: pgmap v569: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 14 MiB/s wr, 1.18k op/s 2024-08-30T09:07:18.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:07:18 smithi042 ceph-mon[18652]: pgmap v569: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 14 MiB/s wr, 1.18k op/s 2024-08-30T09:07:18.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:07:18 smithi190 ceph-mon[22001]: pgmap v569: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 14 MiB/s wr, 1.18k op/s 2024-08-30T09:07:19.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:19.823+0000 7f8b5bd6c640 1 -- 172.21.15.42:0/1337340160 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8b54111ce0 msgr2=0x7f8b5410a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:07:19.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:19.823+0000 7f8b5bd6c640 1 --2- 172.21.15.42:0/1337340160 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8b54111ce0 0x7f8b5410a160 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f8b50009860 tx=0x7f8b5002a380 comp rx=0 tx=0).stop 2024-08-30T09:07:19.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:19.830+0000 7f8b5bd6c640 1 -- 172.21.15.42:0/1337340160 shutdown_connections 2024-08-30T09:07:19.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:19.830+0000 7f8b5bd6c640 1 --2- 172.21.15.42:0/1337340160 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8b5411c9d0 0x7f8b5411eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:19.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:19.830+0000 7f8b5bd6c640 1 --2- 172.21.15.42:0/1337340160 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8b5410a780 0x7f8b5410ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:19.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:19.830+0000 7f8b5bd6c640 1 --2- 172.21.15.42:0/1337340160 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8b54111ce0 0x7f8b5410a160 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:19.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:19.830+0000 7f8b5bd6c640 1 -- 172.21.15.42:0/1337340160 >> 172.21.15.42:0/1337340160 conn(0x7f8b54070c60 msgr2=0x7f8b54071070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:07:19.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:19.835+0000 7f8b5bd6c640 1 -- 172.21.15.42:0/1337340160 shutdown_connections 2024-08-30T09:07:19.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:19.837+0000 7f8b5bd6c640 1 -- 172.21.15.42:0/1337340160 wait complete. 2024-08-30T09:07:19.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:19.838+0000 7f8b5bd6c640 1 Processor -- start 2024-08-30T09:07:19.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:19.848+0000 7f8b5bd6c640 1 -- start start 2024-08-30T09:07:19.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:19.848+0000 7f8b5bd6c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8b5410a780 0x7f8b5411c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:07:19.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:19.848+0000 7f8b5bd6c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8b54111ce0 0x7f8b541185a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:07:19.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:19.848+0000 7f8b5bd6c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8b5411c9d0 0x7f8b54118c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:07:19.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:19.848+0000 7f8b5bd6c640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f8b54114ab0 con 0x7f8b5411c9d0 2024-08-30T09:07:19.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:19.848+0000 7f8b5bd6c640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f8b54114930 con 0x7f8b5410a780 2024-08-30T09:07:19.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:19.848+0000 7f8b5bd6c640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f8b54114c30 con 0x7f8b54111ce0 2024-08-30T09:07:19.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:19.849+0000 7f8b59b08640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8b5410a780 0x7f8b5411c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:07:19.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:19.849+0000 7f8b59b08640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8b5410a780 0x7f8b5411c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:44192/0 (socket says 172.21.15.42:44192) 2024-08-30T09:07:19.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:19.849+0000 7f8b59b08640 1 -- 172.21.15.42:0/4134129502 learned_addr learned my addr 172.21.15.42:0/4134129502 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:07:19.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:19.849+0000 7f8b59307640 1 --2- 172.21.15.42:0/4134129502 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8b54111ce0 0x7f8b541185a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:07:19.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:19.852+0000 7f8b59307640 1 -- 172.21.15.42:0/4134129502 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8b5410a780 msgr2=0x7f8b5411c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:07:19.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:19.852+0000 7f8b59307640 1 --2- 172.21.15.42:0/4134129502 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8b5410a780 0x7f8b5411c2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:19.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:19.852+0000 7f8b59307640 1 -- 172.21.15.42:0/4134129502 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8b5411c9d0 msgr2=0x7f8b54118c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:07:19.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:19.852+0000 7f8b59307640 1 --2- 172.21.15.42:0/4134129502 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8b5411c9d0 0x7f8b54118c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:19.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:19.852+0000 7f8b59307640 1 -- 172.21.15.42:0/4134129502 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b541acb70 con 0x7f8b54111ce0 2024-08-30T09:07:19.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:19.853+0000 7f8b59307640 1 --2- 172.21.15.42:0/4134129502 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8b54111ce0 0x7f8b541185a0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f8b5406c480 tx=0x7f8b4c01d750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:07:19.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:19.863+0000 7f8b4affd640 1 -- 172.21.15.42:0/4134129502 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b4c003970 con 0x7f8b54111ce0 2024-08-30T09:07:19.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:19.863+0000 7f8b4affd640 1 -- 172.21.15.42:0/4134129502 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8b4c004020 con 0x7f8b54111ce0 2024-08-30T09:07:19.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:19.863+0000 7f8b4affd640 1 -- 172.21.15.42:0/4134129502 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b4c01c850 con 0x7f8b54111ce0 2024-08-30T09:07:19.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:19.864+0000 7f8b5bd6c640 1 -- 172.21.15.42:0/4134129502 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b541acd40 con 0x7f8b54111ce0 2024-08-30T09:07:19.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:19.864+0000 7f8b5bd6c640 1 -- 172.21.15.42:0/4134129502 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b541ad1a0 con 0x7f8b54111ce0 2024-08-30T09:07:19.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:19.868+0000 7f8b5bd6c640 1 -- 172.21.15.42:0/4134129502 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b5410ab90 con 0x7f8b54111ce0 2024-08-30T09:07:19.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:19.872+0000 7f8b4affd640 1 -- 172.21.15.42:0/4134129502 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7f8b4c02e020 con 0x7f8b54111ce0 2024-08-30T09:07:19.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:19.873+0000 7f8b4affd640 1 --2- 172.21.15.42:0/4134129502 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f8b400b4d60 0x7f8b400b7270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:07:19.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:19.873+0000 7f8b59b08640 1 --2- 172.21.15.42:0/4134129502 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f8b400b4d60 0x7f8b400b7270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:07:19.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:19.874+0000 7f8b4affd640 1 -- 172.21.15.42:0/4134129502 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f8b4c0f63b0 con 0x7f8b54111ce0 2024-08-30T09:07:19.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:19.874+0000 7f8b59b08640 1 --2- 172.21.15.42:0/4134129502 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f8b400b4d60 0x7f8b400b7270 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f8b50009830 tx=0x7f8b5002a880 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:07:19.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:19.882+0000 7f8b4affd640 1 -- 172.21.15.42:0/4134129502 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f8b4c0bd0f0 con 0x7f8b54111ce0 2024-08-30T09:07:20.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:20.330+0000 7f8b5bd6c640 1 -- 172.21.15.42:0/4134129502 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8b541ad4c0 con 0x7f8b54111ce0 2024-08-30T09:07:20.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:20.333+0000 7f8b4affd640 1 -- 172.21.15.42:0/4134129502 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11919 (secure 0 0 0) 0x7f8b4c0bc9f0 con 0x7f8b54111ce0 2024-08-30T09:07:20.336 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-08-30T09:07:20.336 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":34,"btime":"2024-08-30T09:06:48:392460+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14703,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/3188400000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":3188400000},{"type":"v1","addr":"172.21.15.42:6835","nonce":3188400000}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14715,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/3356537674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":3356537674},{"type":"v1","addr":"172.21.15.42:6837","nonce":3356537674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14766,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6839/2107785513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6838","nonce":2107785513},{"type":"v1","addr":"172.21.15.42:6839","nonce":2107785513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14778,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6841/2298438674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6840","nonce":2298438674},{"type":"v1","addr":"172.21.15.42:6841","nonce":2298438674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24553,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/543555570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":543555570},{"type":"v1","addr":"172.21.15.190:6833","nonce":543555570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24554,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/3907818941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":3907818941},{"type":"v1","addr":"172.21.15.167:6833","nonce":3907818941}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":34,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-30T08:58:03.055084+0000","modified":"2024-08-30T09:06:47.644639+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24629,"mds_1":24610,"mds_2":24599},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24572":{"gid":24572,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6835/234596845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":234596845},{"type":"v1","addr":"172.21.15.167:6835","nonce":234596845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24592":{"gid":24592,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6835/247905709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":247905709},{"type":"v1","addr":"172.21.15.190:6835","nonce":247905709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24598":{"gid":24598,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6837/863256349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6836","nonce":863256349},{"type":"v1","addr":"172.21.15.190:6837","nonce":863256349}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24599":{"gid":24599,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.167:6837/3011837856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":3011837856},{"type":"v1","addr":"172.21.15.167:6837","nonce":3011837856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24610":{"gid":24610,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.190:6839/719981110","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6838","nonce":719981110},{"type":"v1","addr":"172.21.15.190:6839","nonce":719981110}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24629":{"gid":24629,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.167:6839/3604645207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":3604645207},{"type":"v1","addr":"172.21.15.167:6839","nonce":3604645207}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24629,"qdb_cluster":[24629,24610,24599]},"id":1}]} 2024-08-30T09:07:20.337 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 34 2024-08-30T09:07:20.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:20.338+0000 7f8b5bd6c640 1 -- 172.21.15.42:0/4134129502 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f8b400b4d60 msgr2=0x7f8b400b7270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:07:20.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:20.338+0000 7f8b5bd6c640 1 --2- 172.21.15.42:0/4134129502 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f8b400b4d60 0x7f8b400b7270 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f8b50009830 tx=0x7f8b5002a880 comp rx=0 tx=0).stop 2024-08-30T09:07:20.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:20.338+0000 7f8b5bd6c640 1 -- 172.21.15.42:0/4134129502 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8b54111ce0 msgr2=0x7f8b541185a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:07:20.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:20.338+0000 7f8b5bd6c640 1 --2- 172.21.15.42:0/4134129502 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8b54111ce0 0x7f8b541185a0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f8b5406c480 tx=0x7f8b4c01d750 comp rx=0 tx=0).stop 2024-08-30T09:07:20.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:20.338+0000 7f8b5bd6c640 1 -- 172.21.15.42:0/4134129502 shutdown_connections 2024-08-30T09:07:20.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:20.338+0000 7f8b5bd6c640 1 --2- 172.21.15.42:0/4134129502 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f8b400b4d60 0x7f8b400b7270 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:20.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:20.338+0000 7f8b5bd6c640 1 --2- 172.21.15.42:0/4134129502 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8b54111ce0 0x7f8b541185a0 secure :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f8b5406c480 tx=0x7f8b4c01d750 comp rx=0 tx=0).stop 2024-08-30T09:07:20.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:20.338+0000 7f8b5bd6c640 1 --2- 172.21.15.42:0/4134129502 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8b5411c9d0 0x7f8b54118c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:20.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:20.338+0000 7f8b5bd6c640 1 --2- 172.21.15.42:0/4134129502 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8b5410a780 0x7f8b5411c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:20.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:20.338+0000 7f8b5bd6c640 1 -- 172.21.15.42:0/4134129502 >> 172.21.15.42:0/4134129502 conn(0x7f8b54070c60 msgr2=0x7f8b54073bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:07:20.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:20.339+0000 7f8b5bd6c640 1 -- 172.21.15.42:0/4134129502 shutdown_connections 2024-08-30T09:07:20.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:20.339+0000 7f8b5bd6c640 1 -- 172.21.15.42:0/4134129502 wait complete. 2024-08-30T09:07:20.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:07:20 smithi042 ceph-mon[18652]: pgmap v570: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 16 MiB/s wr, 1.29k op/s 2024-08-30T09:07:20.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:07:20 smithi042 ceph-mon[18652]: from='client.? 172.21.15.42:0/4134129502' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-30T09:07:20.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:07:20 smithi167 ceph-mon[22063]: pgmap v570: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 16 MiB/s wr, 1.29k op/s 2024-08-30T09:07:20.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:07:20 smithi167 ceph-mon[22063]: from='client.? 172.21.15.42:0/4134129502' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-30T09:07:20.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:07:20 smithi190 ceph-mon[22001]: pgmap v570: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 16 MiB/s wr, 1.29k op/s 2024-08-30T09:07:20.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:07:20 smithi190 ceph-mon[22001]: from='client.? 172.21.15.42:0/4134129502' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-30T09:07:20.953 INFO:teuthology.orchestra.run.smithi042.stderr:progress: 395.0 s, 542.8 tps, lat 183.117 ms stddev 265.290 2024-08-30T09:07:21.077 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-08-30T09:07:21.565 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T09:07:22.267 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.263+0000 7fb837529640 1 -- 172.21.15.42:0/3951983871 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb830108140 msgr2=0x7fb830108550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:07:22.267 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.263+0000 7fb837529640 1 --2- 172.21.15.42:0/3951983871 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb830108140 0x7fb830108550 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fb820009900 tx=0x7fb82002a3f0 comp rx=0 tx=0).stop 2024-08-30T09:07:22.267 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.263+0000 7fb837529640 1 -- 172.21.15.42:0/3951983871 shutdown_connections 2024-08-30T09:07:22.267 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.263+0000 7fb837529640 1 --2- 172.21.15.42:0/3951983871 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb830108a90 0x7fb83010f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:22.267 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.263+0000 7fb837529640 1 --2- 172.21.15.42:0/3951983871 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb830108140 0x7fb830108550 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:22.267 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.263+0000 7fb837529640 1 --2- 172.21.15.42:0/3951983871 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb830102800 0x7fb830102c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:22.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.263+0000 7fb837529640 1 -- 172.21.15.42:0/3951983871 >> 172.21.15.42:0/3951983871 conn(0x7fb8300794d0 msgr2=0x7fb8300798e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:07:22.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.263+0000 7fb837529640 1 -- 172.21.15.42:0/3951983871 shutdown_connections 2024-08-30T09:07:22.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.263+0000 7fb837529640 1 -- 172.21.15.42:0/3951983871 wait complete. 2024-08-30T09:07:22.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.264+0000 7fb837529640 1 Processor -- start 2024-08-30T09:07:22.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.265+0000 7fb837529640 1 -- start start 2024-08-30T09:07:22.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.265+0000 7fb837529640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb830102800 0x7fb83019a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:07:22.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.265+0000 7fb837529640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb830108140 0x7fb83019a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:07:22.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.265+0000 7fb837529640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb830108a90 0x7fb83019f920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:07:22.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.265+0000 7fb837529640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fb830113810 con 0x7fb830108140 2024-08-30T09:07:22.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.265+0000 7fb837529640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb830113690 con 0x7fb830102800 2024-08-30T09:07:22.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.265+0000 7fb837529640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fb830113990 con 0x7fb830108a90 2024-08-30T09:07:22.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.265+0000 7fb8352c5640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb830102800 0x7fb83019a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:07:22.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.265+0000 7fb8352c5640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb830102800 0x7fb83019a0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:44214/0 (socket says 172.21.15.42:44214) 2024-08-30T09:07:22.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.265+0000 7fb8352c5640 1 -- 172.21.15.42:0/3913681153 learned_addr learned my addr 172.21.15.42:0/3913681153 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:07:22.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.265+0000 7fb835ac6640 1 --2- 172.21.15.42:0/3913681153 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb830108a90 0x7fb83019f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:07:22.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.266+0000 7fb835ac6640 1 -- 172.21.15.42:0/3913681153 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb830102800 msgr2=0x7fb83019a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:07:22.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.266+0000 7fb835ac6640 1 --2- 172.21.15.42:0/3913681153 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb830102800 0x7fb83019a0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:22.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.266+0000 7fb835ac6640 1 -- 172.21.15.42:0/3913681153 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb830108140 msgr2=0x7fb83019a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:07:22.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.266+0000 7fb835ac6640 1 --2- 172.21.15.42:0/3913681153 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb830108140 0x7fb83019a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:22.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.266+0000 7fb835ac6640 1 -- 172.21.15.42:0/3913681153 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8301a0050 con 0x7fb830108a90 2024-08-30T09:07:22.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.266+0000 7fb835ac6640 1 --2- 172.21.15.42:0/3913681153 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb830108a90 0x7fb83019f920 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fb828009fd0 tx=0x7fb82800ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:07:22.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.266+0000 7fb8267fc640 1 -- 172.21.15.42:0/3913681153 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb82800ce20 con 0x7fb830108a90 2024-08-30T09:07:22.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.266+0000 7fb837529640 1 -- 172.21.15.42:0/3913681153 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8301a0340 con 0x7fb830108a90 2024-08-30T09:07:22.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.266+0000 7fb8267fc640 1 -- 172.21.15.42:0/3913681153 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb828005020 con 0x7fb830108a90 2024-08-30T09:07:22.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.266+0000 7fb8267fc640 1 -- 172.21.15.42:0/3913681153 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb828010690 con 0x7fb830108a90 2024-08-30T09:07:22.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.267+0000 7fb837529640 1 -- 172.21.15.42:0/3913681153 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8301a6aa0 con 0x7fb830108a90 2024-08-30T09:07:22.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.270+0000 7fb83da9e740 1 Processor -- start 2024-08-30T09:07:22.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.271+0000 7fb8267fc640 1 -- 172.21.15.42:0/3913681153 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7fb8280054d0 con 0x7fb830108a90 2024-08-30T09:07:22.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.271+0000 7fb83da9e740 1 -- start start 2024-08-30T09:07:22.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.272+0000 7fb83da9e740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564519374070 0x564519719310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:07:22.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.272+0000 7fb83da9e740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56451934b1b0 0x564519719850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:07:22.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.272+0000 7fb83da9e740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564519353390 0x564519719d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:07:22.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.272+0000 7fb83da9e740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x564519690680 con 0x564519374070 2024-08-30T09:07:22.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.272+0000 7fb83da9e740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x56451967fc10 con 0x56451934b1b0 2024-08-30T09:07:22.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.272+0000 7fb83da9e740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x564519683bc0 con 0x564519353390 2024-08-30T09:07:22.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.272+0000 7fb8352c5640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56451934b1b0 0x564519719850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:07:22.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.272+0000 7fb834ac4640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564519374070 0x564519719310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:07:22.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.272+0000 7fb8352c5640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56451934b1b0 0x564519719850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:44218/0 (socket says 172.21.15.42:44218) 2024-08-30T09:07:22.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.272+0000 7fb8352c5640 1 -- 172.21.15.42:0/4130029442 learned_addr learned my addr 172.21.15.42:0/4130029442 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:07:22.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.272+0000 7fb8267fc640 1 --2- 172.21.15.42:0/3913681153 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fb81c0b4b30 0x7fb81c0b7040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:07:22.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.272+0000 7fb834ac4640 1 --2- 172.21.15.42:0/4130029442 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564519353390 0x564519719d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:07:22.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.272+0000 7fb8352c5640 1 -- 172.21.15.42:0/4130029442 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564519353390 msgr2=0x564519719d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:07:22.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.272+0000 7fb8352c5640 1 --2- 172.21.15.42:0/4130029442 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564519353390 0x564519719d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:22.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.272+0000 7fb8352c5640 1 -- 172.21.15.42:0/4130029442 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564519374070 msgr2=0x564519719310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:07:22.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.272+0000 7fb8352c5640 1 --2- 172.21.15.42:0/4130029442 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564519374070 0x564519719310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:22.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.272+0000 7fb8352c5640 1 -- 172.21.15.42:0/4130029442 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5645193df650 con 0x56451934b1b0 2024-08-30T09:07:22.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.272+0000 7fb8267fc640 1 -- 172.21.15.42:0/3913681153 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7fb8280e5bc0 con 0x7fb830108a90 2024-08-30T09:07:22.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.272+0000 7fb835ac6640 1 --2- 172.21.15.42:0/3913681153 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fb81c0b4b30 0x7fb81c0b7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:07:22.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.273+0000 7fb8352c5640 1 --2- 172.21.15.42:0/4130029442 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56451934b1b0 0x564519719850 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fb82c009dc0 tx=0x7fb82c011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1e64dcd66629fa55 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:07:22.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.273+0000 7fb81b7fe640 1 -- 172.21.15.42:0/4130029442 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb82c005220 con 0x56451934b1b0 2024-08-30T09:07:22.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.273+0000 7fb81b7fe640 1 -- 172.21.15.42:0/4130029442 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb82c004c50 con 0x56451934b1b0 2024-08-30T09:07:22.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.273+0000 7fb81b7fe640 1 -- 172.21.15.42:0/4130029442 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb82c019970 con 0x56451934b1b0 2024-08-30T09:07:22.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.274+0000 7fb83da9e740 1 -- 172.21.15.42:0/4130029442 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56451934b1b0 msgr2=0x564519719850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:07:22.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.274+0000 7fb835ac6640 1 --2- 172.21.15.42:0/3913681153 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fb81c0b4b30 0x7fb81c0b7040 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fb82801a040 tx=0x7fb8280057c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:07:22.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.274+0000 7fb83da9e740 1 --2- 172.21.15.42:0/4130029442 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56451934b1b0 0x564519719850 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fb82c009dc0 tx=0x7fb82c011810 comp rx=0 tx=0).stop 2024-08-30T09:07:22.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.274+0000 7fb83da9e740 1 -- 172.21.15.42:0/4130029442 shutdown_connections 2024-08-30T09:07:22.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.274+0000 7fb83da9e740 1 --2- 172.21.15.42:0/4130029442 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564519353390 0x564519719d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:22.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.274+0000 7fb83da9e740 1 --2- 172.21.15.42:0/4130029442 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56451934b1b0 0x564519719850 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:22.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.274+0000 7fb83da9e740 1 --2- 172.21.15.42:0/4130029442 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564519374070 0x564519719310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:22.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.274+0000 7fb83da9e740 1 -- 172.21.15.42:0/4130029442 >> 172.21.15.42:0/4130029442 conn(0x564519368460 msgr2=0x564519717200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:07:22.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.274+0000 7fb83da9e740 1 -- 172.21.15.42:0/4130029442 shutdown_connections 2024-08-30T09:07:22.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.274+0000 7fb83da9e740 1 -- 172.21.15.42:0/4130029442 wait complete. 2024-08-30T09:07:22.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.275+0000 7fb83da9e740 1 -- start start 2024-08-30T09:07:22.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.275+0000 7fb83da9e740 1 Processor -- start 2024-08-30T09:07:22.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.275+0000 7fb83da9e740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56451934b1b0 0x564519720b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:07:22.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.276+0000 7fb83da9e740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564519353390 0x564519721170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:07:22.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.276+0000 7fb834ac4640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56451934b1b0 0x564519720b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:07:22.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.276+0000 7fb834ac4640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56451934b1b0 0x564519720b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:39196/0 (socket says 172.21.15.42:39196) 2024-08-30T09:07:22.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.276+0000 7fb83da9e740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564519374070 0x564519726320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:07:22.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.276+0000 7fb834ac4640 1 -- 172.21.15.42:0/1950228774 learned_addr learned my addr 172.21.15.42:0/1950228774 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:07:22.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.276+0000 7fb8352c5640 1 --2- 172.21.15.42:0/1950228774 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564519353390 0x564519721170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:07:22.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.276+0000 7fb83da9e740 1 -- 172.21.15.42:0/1950228774 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x5645194e2700 con 0x564519374070 2024-08-30T09:07:22.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.276+0000 7fb83da9e740 1 -- 172.21.15.42:0/1950228774 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x564519606750 con 0x564519353390 2024-08-30T09:07:22.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.276+0000 7fb83da9e740 1 -- 172.21.15.42:0/1950228774 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x564519727930 con 0x56451934b1b0 2024-08-30T09:07:22.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.276+0000 7fb83da9e740 10 client.0 _ll_register_callbacks cb 0x564519635e10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:07:22.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.276+0000 7fb834ac4640 1 --2- 172.21.15.42:0/1950228774 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564519374070 0x564519726320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:07:22.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.276+0000 7fb834ac4640 1 -- 172.21.15.42:0/1950228774 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564519353390 msgr2=0x564519721170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:07:22.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.276+0000 7fb834ac4640 1 --2- 172.21.15.42:0/1950228774 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564519353390 0x564519721170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:22.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.276+0000 7fb834ac4640 1 -- 172.21.15.42:0/1950228774 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564519374070 msgr2=0x564519726320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:07:22.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.276+0000 7fb834ac4640 1 --2- 172.21.15.42:0/1950228774 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564519374070 0x564519726320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:22.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.276+0000 7fb834ac4640 1 -- 172.21.15.42:0/1950228774 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56451963f900 con 0x56451934b1b0 2024-08-30T09:07:22.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.276+0000 7fb834ac4640 1 --2- 172.21.15.42:0/1950228774 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56451934b1b0 0x564519720b90 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fb820009900 tx=0x7fb8200095e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7add1846a7571fcc server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:07:22.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.276+0000 7fb818ff9640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-08-30T09:07:22.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.277+0000 7fb818ff9640 1 -- 172.21.15.42:0/1950228774 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb820035040 con 0x56451934b1b0 2024-08-30T09:07:22.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.277+0000 7fb818ff9640 1 -- 172.21.15.42:0/1950228774 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7fb820047850 con 0x56451934b1b0 2024-08-30T09:07:22.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.277+0000 7fb83da9e740 1 -- 172.21.15.42:0/1950228774 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb82c004c50 con 0x56451934b1b0 2024-08-30T09:07:22.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.277+0000 7fb818ff9640 1 -- 172.21.15.42:0/1950228774 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fb820047a50 con 0x56451934b1b0 2024-08-30T09:07:22.285 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.277+0000 7fb83da9e740 10 client.26293 fetch_fsmap learned FSMap version 34 2024-08-30T09:07:22.285 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.277+0000 7fb83da9e740 1 -- 172.21.15.42:0/1950228774 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5645197338e0 con 0x56451934b1b0 2024-08-30T09:07:22.285 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.278+0000 7fb818ff9640 1 -- 172.21.15.42:0/1950228774 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb82c004c50 con 0x56451934b1b0 2024-08-30T09:07:22.285 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.278+0000 7fb818ff9640 1 -- 172.21.15.42:0/1950228774 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 34) ==== 8366+0+0 (secure 0 0 0) 0x7fb82003a040 con 0x56451934b1b0 2024-08-30T09:07:22.285 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.278+0000 7fb83da9e740 10 client.26293 fetch_fsmap finished waiting for FSMap version 34 2024-08-30T09:07:22.285 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.278+0000 7fb83da9e740 10 client.26293 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-08-30T09:07:22.285 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.278+0000 7fb83da9e740 20 client.26293 populate_metadata read hostname 'smithi042' 2024-08-30T09:07:22.285 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.278+0000 7fb83da9e740 1 --2- 172.21.15.42:0/1950228774 >> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] conn(0x5645193329f0 0x564519736e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:07:22.285 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.278+0000 7fb83da9e740 4 client.26293 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-30T09:07:22.285 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.278+0000 7fb83da9e740 1 -- 172.21.15.42:0/1950228774 --> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5645196925d0 con 0x5645193329f0 2024-08-30T09:07:22.285 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.278+0000 7fb8352c5640 1 --2- 172.21.15.42:0/1950228774 >> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] conn(0x5645193329f0 0x564519736e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:07:22.285 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.279+0000 7fb8352c5640 1 --2- 172.21.15.42:0/1950228774 >> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] conn(0x5645193329f0 0x564519736e40 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=feee8033bc8f82aa server_cookie=b22e178b9761f4c8 in_seq=0 out_seq=0 2024-08-30T09:07:22.285 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.279+0000 7fb818ff9640 10 client.26293 ms_handle_connect on v2:172.21.15.167:6836/3011837856 2024-08-30T09:07:22.285 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.279+0000 7fb818ff9640 1 -- 172.21.15.42:0/1950228774 <== mds.2 v2:172.21.15.167:6836/3011837856 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5645196925d0 con 0x5645193329f0 2024-08-30T09:07:22.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.279+0000 7fb818ff9640 10 client.26293 handle_command_reply: tid=0 2024-08-30T09:07:22.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.279+0000 7fb83da9e740 1 client.26293 shutdown 2024-08-30T09:07:22.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.280+0000 7fb83da9e740 1 -- 172.21.15.42:0/1950228774 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56451934b1b0 msgr2=0x564519720b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:07:22.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.280+0000 7fb83da9e740 1 --2- 172.21.15.42:0/1950228774 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56451934b1b0 0x564519720b90 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fb820009900 tx=0x7fb8200095e0 comp rx=0 tx=0).stop 2024-08-30T09:07:22.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.282+0000 7fb83da9e740 1 -- 172.21.15.42:0/1950228774 shutdown_connections 2024-08-30T09:07:22.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.282+0000 7fb83da9e740 1 --2- 172.21.15.42:0/1950228774 >> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] conn(0x5645193329f0 0x564519736e40 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:22.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.282+0000 7fb83da9e740 1 --2- 172.21.15.42:0/1950228774 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564519374070 0x564519726320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:22.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.282+0000 7fb83da9e740 1 --2- 172.21.15.42:0/1950228774 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564519353390 0x564519721170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:22.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.282+0000 7fb83da9e740 1 --2- 172.21.15.42:0/1950228774 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56451934b1b0 0x564519720b90 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:22.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.282+0000 7fb83da9e740 1 -- 172.21.15.42:0/1950228774 >> 172.21.15.42:0/1950228774 conn(0x564519368460 msgr2=0x56451971c4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:07:22.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.282+0000 7fb818ff9640 0 client.26293 ms_handle_reset on v2:172.21.15.167:6836/3011837856 2024-08-30T09:07:22.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.282+0000 7fb83da9e740 1 -- 172.21.15.42:0/1950228774 shutdown_connections 2024-08-30T09:07:22.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.282+0000 7fb83da9e740 1 -- 172.21.15.42:0/1950228774 wait complete. 2024-08-30T09:07:22.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.283+0000 7fb83da9e740 20 client.26293 trim_cache size 0 max 16384 2024-08-30T09:07:22.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.301+0000 7fb83da9e740 1 Processor -- start 2024-08-30T09:07:22.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.301+0000 7fb83da9e740 1 -- start start 2024-08-30T09:07:22.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.301+0000 7fb83da9e740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56451934b1b0 0x56451972b300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:07:22.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.301+0000 7fb83da9e740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564519353390 0x56451971f410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:07:22.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.301+0000 7fb83da9e740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564519374070 0x56451971f950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:07:22.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.301+0000 7fb83da9e740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fb8301b0fe0 con 0x56451934b1b0 2024-08-30T09:07:22.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.301+0000 7fb834ac4640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56451934b1b0 0x56451972b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:07:22.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.301+0000 7fb834ac4640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56451934b1b0 0x56451972b300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:50392/0 (socket says 172.21.15.42:50392) 2024-08-30T09:07:22.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.301+0000 7fb834ac4640 1 -- 172.21.15.42:0/3433102023 learned_addr learned my addr 172.21.15.42:0/3433102023 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:07:22.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.301+0000 7fb83da9e740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb8301ae850 con 0x564519374070 2024-08-30T09:07:22.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.301+0000 7fb83da9e740 1 -- 172.21.15.42:0/3433102023 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fb8301ac6a0 con 0x564519353390 2024-08-30T09:07:22.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.301+0000 7fb8352c5640 1 --2- 172.21.15.42:0/3433102023 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564519353390 0x56451971f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:07:22.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.302+0000 7fb834ac4640 1 --2- 172.21.15.42:0/3433102023 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564519374070 0x56451971f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:07:22.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.302+0000 7fb834ac4640 1 -- 172.21.15.42:0/3433102023 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564519353390 msgr2=0x56451971f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:07:22.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.302+0000 7fb834ac4640 1 --2- 172.21.15.42:0/3433102023 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564519353390 0x56451971f410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:22.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.302+0000 7fb834ac4640 1 -- 172.21.15.42:0/3433102023 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564519374070 msgr2=0x56451971f950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:07:22.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.302+0000 7fb834ac4640 1 --2- 172.21.15.42:0/3433102023 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564519374070 0x56451971f950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:22.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.302+0000 7fb834ac4640 1 -- 172.21.15.42:0/3433102023 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564519721e50 con 0x56451934b1b0 2024-08-30T09:07:22.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.302+0000 7fb834ac4640 1 --2- 172.21.15.42:0/3433102023 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56451934b1b0 0x56451972b300 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fb8200095a0 tx=0x7fb820000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b6093e2759856c4 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:07:22.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.302+0000 7fb81affd640 1 -- 172.21.15.42:0/3433102023 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb820035040 con 0x56451934b1b0 2024-08-30T09:07:22.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.302+0000 7fb81affd640 1 -- 172.21.15.42:0/3433102023 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb82004f070 con 0x56451934b1b0 2024-08-30T09:07:22.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.302+0000 7fb81affd640 1 -- 172.21.15.42:0/3433102023 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb820054700 con 0x56451934b1b0 2024-08-30T09:07:22.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.303+0000 7fb83da9e740 1 -- 172.21.15.42:0/3433102023 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56451934b1b0 msgr2=0x56451972b300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:07:22.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.303+0000 7fb83da9e740 1 --2- 172.21.15.42:0/3433102023 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56451934b1b0 0x56451972b300 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fb8200095a0 tx=0x7fb820000bc0 comp rx=0 tx=0).stop 2024-08-30T09:07:22.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.303+0000 7fb83da9e740 1 -- 172.21.15.42:0/3433102023 shutdown_connections 2024-08-30T09:07:22.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.303+0000 7fb83da9e740 1 --2- 172.21.15.42:0/3433102023 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564519353390 0x56451971f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:22.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.303+0000 7fb83da9e740 1 --2- 172.21.15.42:0/3433102023 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564519374070 0x56451971f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:22.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.303+0000 7fb83da9e740 1 --2- 172.21.15.42:0/3433102023 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56451934b1b0 0x56451972b300 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:22.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.303+0000 7fb83da9e740 1 -- 172.21.15.42:0/3433102023 >> 172.21.15.42:0/3433102023 conn(0x564519368460 msgr2=0x56451972a260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:07:22.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.304+0000 7fb83da9e740 1 -- 172.21.15.42:0/3433102023 shutdown_connections 2024-08-30T09:07:22.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.304+0000 7fb83da9e740 1 -- 172.21.15.42:0/3433102023 wait complete. 2024-08-30T09:07:22.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.304+0000 7fb83da9e740 1 -- start start 2024-08-30T09:07:22.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.304+0000 7fb83da9e740 1 Processor -- start 2024-08-30T09:07:22.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.305+0000 7fb83da9e740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56451934b1b0 0x56451972ce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:07:22.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.305+0000 7fb83da9e740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564519353390 0x56451972d360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:07:22.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.305+0000 7fb83da9e740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564519374070 0x56451972d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:07:22.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.305+0000 7fb83da9e740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fb8301acd10 con 0x56451934b1b0 2024-08-30T09:07:22.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.305+0000 7fb83da9e740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb8301ae180 con 0x564519374070 2024-08-30T09:07:22.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.305+0000 7fb83da9e740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fb8301ad3e0 con 0x564519353390 2024-08-30T09:07:22.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.305+0000 7fb83da9e740 10 client.0 _ll_register_callbacks cb 0x7fb820009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:07:22.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.305+0000 7fb834ac4640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56451934b1b0 0x56451972ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:07:22.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.305+0000 7fb834ac4640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56451934b1b0 0x56451972ce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:50404/0 (socket says 172.21.15.42:50404) 2024-08-30T09:07:22.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.305+0000 7fb834ac4640 1 -- 172.21.15.42:0/2566058465 learned_addr learned my addr 172.21.15.42:0/2566058465 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:07:22.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.305+0000 7fb8352c5640 1 --2- 172.21.15.42:0/2566058465 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564519353390 0x56451972d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:07:22.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.305+0000 7fb834ac4640 1 --2- 172.21.15.42:0/2566058465 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564519374070 0x56451972d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:07:22.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.305+0000 7fb834ac4640 1 -- 172.21.15.42:0/2566058465 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564519353390 msgr2=0x56451972d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:07:22.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.305+0000 7fb834ac4640 1 --2- 172.21.15.42:0/2566058465 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564519353390 0x56451972d360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:22.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.305+0000 7fb834ac4640 1 -- 172.21.15.42:0/2566058465 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564519374070 msgr2=0x56451972d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:07:22.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.305+0000 7fb834ac4640 1 --2- 172.21.15.42:0/2566058465 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564519374070 0x56451972d8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:22.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.305+0000 7fb834ac4640 1 -- 172.21.15.42:0/2566058465 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5645197a0870 con 0x56451934b1b0 2024-08-30T09:07:22.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.305+0000 7fb834ac4640 1 --2- 172.21.15.42:0/2566058465 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56451934b1b0 0x56451972ce20 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fb82000b0b0 tx=0x7fb820031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d745c466edb92b server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:07:22.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.305+0000 7fb818ff9640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-08-30T09:07:22.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.308+0000 7fb818ff9640 1 -- 172.21.15.42:0/2566058465 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb820035040 con 0x56451934b1b0 2024-08-30T09:07:22.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.308+0000 7fb818ff9640 1 -- 172.21.15.42:0/2566058465 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7fb820047850 con 0x56451934b1b0 2024-08-30T09:07:22.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.308+0000 7fb83da9e740 1 -- 172.21.15.42:0/2566058465 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb82004f070 con 0x56451934b1b0 2024-08-30T09:07:22.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.308+0000 7fb818ff9640 1 -- 172.21.15.42:0/2566058465 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fb8200035f0 con 0x56451934b1b0 2024-08-30T09:07:22.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.308+0000 7fb83da9e740 10 client.16557 fetch_fsmap learned FSMap version 34 2024-08-30T09:07:22.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.308+0000 7fb83da9e740 1 -- 172.21.15.42:0/2566058465 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5645197a1af0 con 0x56451934b1b0 2024-08-30T09:07:22.313 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.310+0000 7fb818ff9640 1 -- 172.21.15.42:0/2566058465 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb820055030 con 0x56451934b1b0 2024-08-30T09:07:22.313 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.310+0000 7fb818ff9640 1 -- 172.21.15.42:0/2566058465 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 34) ==== 8366+0+0 (secure 0 0 0) 0x7fb82005c040 con 0x56451934b1b0 2024-08-30T09:07:22.313 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.310+0000 7fb83da9e740 10 client.16557 fetch_fsmap finished waiting for FSMap version 34 2024-08-30T09:07:22.313 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.310+0000 7fb83da9e740 10 client.16557 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-08-30T09:07:22.313 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.310+0000 7fb83da9e740 20 client.16557 populate_metadata read hostname 'smithi042' 2024-08-30T09:07:22.313 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.310+0000 7fb83da9e740 1 --2- 172.21.15.42:0/2566058465 >> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] conn(0x564519307da0 0x5645197a4140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:07:22.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.311+0000 7fb83da9e740 4 client.16557 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-30T09:07:22.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.311+0000 7fb83da9e740 1 -- 172.21.15.42:0/2566058465 --> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fb82004f070 con 0x564519307da0 2024-08-30T09:07:22.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.315+0000 7fb8352c5640 1 --2- 172.21.15.42:0/2566058465 >> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] conn(0x564519307da0 0x5645197a4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:07:22.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.329+0000 7fb8352c5640 1 --2- 172.21.15.42:0/2566058465 >> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] conn(0x564519307da0 0x5645197a4140 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6aff6e2fa33cf21 server_cookie=ef1d22ca873d7571 in_seq=0 out_seq=0 2024-08-30T09:07:22.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.329+0000 7fb818ff9640 10 client.16557 ms_handle_connect on v2:172.21.15.167:6836/3011837856 2024-08-30T09:07:22.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.330+0000 7fb818ff9640 1 -- 172.21.15.42:0/2566058465 <== mds.2 v2:172.21.15.167:6836/3011837856 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fb82004f070 con 0x564519307da0 2024-08-30T09:07:22.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.330+0000 7fb818ff9640 10 client.16557 handle_command_reply: tid=0 2024-08-30T09:07:22.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.330+0000 7fb83da9e740 1 client.16557 shutdown 2024-08-30T09:07:22.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.337+0000 7fb83da9e740 1 -- 172.21.15.42:0/2566058465 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56451934b1b0 msgr2=0x56451972ce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:07:22.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.337+0000 7fb83da9e740 1 --2- 172.21.15.42:0/2566058465 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56451934b1b0 0x56451972ce20 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fb82000b0b0 tx=0x7fb820031400 comp rx=0 tx=0).stop 2024-08-30T09:07:22.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.338+0000 7fb83da9e740 1 -- 172.21.15.42:0/2566058465 shutdown_connections 2024-08-30T09:07:22.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.338+0000 7fb83da9e740 1 --2- 172.21.15.42:0/2566058465 >> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] conn(0x564519307da0 0x5645197a4140 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:22.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.338+0000 7fb83da9e740 1 --2- 172.21.15.42:0/2566058465 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564519353390 0x56451972d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:22.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.338+0000 7fb83da9e740 1 --2- 172.21.15.42:0/2566058465 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564519374070 0x56451972d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:22.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.338+0000 7fb83da9e740 1 --2- 172.21.15.42:0/2566058465 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56451934b1b0 0x56451972ce20 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:22.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.338+0000 7fb83da9e740 1 -- 172.21.15.42:0/2566058465 >> 172.21.15.42:0/2566058465 conn(0x564519368460 msgr2=0x564519726200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:07:22.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.338+0000 7fb818ff9640 0 client.16557 ms_handle_reset on v2:172.21.15.167:6836/3011837856 2024-08-30T09:07:22.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.338+0000 7fb83da9e740 1 -- 172.21.15.42:0/2566058465 shutdown_connections 2024-08-30T09:07:22.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.338+0000 7fb83da9e740 1 -- 172.21.15.42:0/2566058465 wait complete. 2024-08-30T09:07:22.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.338+0000 7fb83da9e740 20 client.16557 trim_cache size 0 max 16384 2024-08-30T09:07:22.345 INFO:teuthology.orchestra.run.smithi042.stdout:[] 2024-08-30T09:07:22.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.344+0000 7fb837529640 1 -- 172.21.15.42:0/3913681153 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fb81c0b4b30 msgr2=0x7fb81c0b7040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:07:22.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.344+0000 7fb837529640 1 --2- 172.21.15.42:0/3913681153 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fb81c0b4b30 0x7fb81c0b7040 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fb82801a040 tx=0x7fb8280057c0 comp rx=0 tx=0).stop 2024-08-30T09:07:22.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.344+0000 7fb837529640 1 -- 172.21.15.42:0/3913681153 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb830108a90 msgr2=0x7fb83019f920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:07:22.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.344+0000 7fb837529640 1 --2- 172.21.15.42:0/3913681153 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb830108a90 0x7fb83019f920 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fb828009fd0 tx=0x7fb82800ef10 comp rx=0 tx=0).stop 2024-08-30T09:07:22.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.344+0000 7fb837529640 1 -- 172.21.15.42:0/3913681153 shutdown_connections 2024-08-30T09:07:22.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.344+0000 7fb837529640 1 --2- 172.21.15.42:0/3913681153 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fb81c0b4b30 0x7fb81c0b7040 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:22.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.344+0000 7fb837529640 1 --2- 172.21.15.42:0/3913681153 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb830108a90 0x7fb83019f920 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:22.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.344+0000 7fb837529640 1 --2- 172.21.15.42:0/3913681153 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb830108140 0x7fb83019a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:22.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.344+0000 7fb837529640 1 --2- 172.21.15.42:0/3913681153 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb830102800 0x7fb83019a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:22.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.344+0000 7fb837529640 1 -- 172.21.15.42:0/3913681153 >> 172.21.15.42:0/3913681153 conn(0x7fb8300794d0 msgr2=0x7fb8301040b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:07:22.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.346+0000 7fb837529640 1 -- 172.21.15.42:0/3913681153 shutdown_connections 2024-08-30T09:07:22.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:22.346+0000 7fb837529640 1 -- 172.21.15.42:0/3913681153 wait complete. 2024-08-30T09:07:22.782 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:07:22 smithi042 ceph-mon[18652]: pgmap v571: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 9.8 MiB/s wr, 687 op/s 2024-08-30T09:07:22.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:07:22 smithi167 ceph-mon[22063]: pgmap v571: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 9.8 MiB/s wr, 687 op/s 2024-08-30T09:07:22.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:07:22 smithi190 ceph-mon[22001]: pgmap v571: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 9.8 MiB/s wr, 687 op/s 2024-08-30T09:07:23.233 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-08-30T09:07:23.771 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T09:07:24.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.511+0000 7f12f8a0a640 1 -- 172.21.15.42:0/2684316664 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f12f411c940 msgr2=0x7f12f411ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:07:24.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.511+0000 7f12f8a0a640 1 --2- 172.21.15.42:0/2684316664 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f12f411c940 0x7f12f411ed60 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f12e8009900 tx=0x7f12e802a440 comp rx=0 tx=0).stop 2024-08-30T09:07:24.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.511+0000 7f12f8a0a640 1 -- 172.21.15.42:0/2684316664 shutdown_connections 2024-08-30T09:07:24.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.511+0000 7f12f8a0a640 1 --2- 172.21.15.42:0/2684316664 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f12f411c940 0x7f12f411ed60 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:24.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.511+0000 7f12f8a0a640 1 --2- 172.21.15.42:0/2684316664 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f12f410d990 0x7f12f410dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:24.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.511+0000 7f12f8a0a640 1 --2- 172.21.15.42:0/2684316664 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f12f4111a70 0x7f12f410d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:24.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.511+0000 7f12f8a0a640 1 -- 172.21.15.42:0/2684316664 >> 172.21.15.42:0/2684316664 conn(0x7f12f406e790 msgr2=0x7f12f406ebd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:07:24.519 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.517+0000 7f12f8a0a640 1 -- 172.21.15.42:0/2684316664 shutdown_connections 2024-08-30T09:07:24.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.518+0000 7f12f8a0a640 1 -- 172.21.15.42:0/2684316664 wait complete. 2024-08-30T09:07:24.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.519+0000 7f12f8a0a640 1 Processor -- start 2024-08-30T09:07:24.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.519+0000 7f12f8a0a640 1 -- start start 2024-08-30T09:07:24.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.520+0000 7f12f8a0a640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f12f410d990 0x7f12f411c120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:07:24.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.520+0000 7f12f8a0a640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f12f4111a70 0x7f12f41153f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:07:24.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.520+0000 7f12f8a0a640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f12f411c940 0x7f12f4115970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:07:24.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.520+0000 7f12f8a0a640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f12f41225b0 con 0x7f12f411c940 2024-08-30T09:07:24.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.520+0000 7f12f8a0a640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f12f4122430 con 0x7f12f410d990 2024-08-30T09:07:24.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.520+0000 7f12f8a0a640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f12f4122730 con 0x7f12f4111a70 2024-08-30T09:07:24.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.521+0000 7f12f2d9d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f12f410d990 0x7f12f411c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:07:24.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.521+0000 7f12f2d9d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f12f410d990 0x7f12f411c120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:44246/0 (socket says 172.21.15.42:44246) 2024-08-30T09:07:24.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.522+0000 7f12f2d9d640 1 -- 172.21.15.42:0/1472167904 learned_addr learned my addr 172.21.15.42:0/1472167904 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:07:24.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.522+0000 7f12f259c640 1 --2- 172.21.15.42:0/1472167904 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f12f4111a70 0x7f12f41153f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:07:24.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.522+0000 7f12f2d9d640 1 -- 172.21.15.42:0/1472167904 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f12f4111a70 msgr2=0x7f12f41153f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:07:24.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.522+0000 7f12f2d9d640 1 --2- 172.21.15.42:0/1472167904 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f12f4111a70 0x7f12f41153f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:24.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.522+0000 7f12f2d9d640 1 -- 172.21.15.42:0/1472167904 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f12f411c940 msgr2=0x7f12f4115970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:07:24.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.522+0000 7f12f2d9d640 1 --2- 172.21.15.42:0/1472167904 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f12f411c940 0x7f12f4115970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:24.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.523+0000 7f12f2d9d640 1 -- 172.21.15.42:0/1472167904 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12f4116190 con 0x7f12f410d990 2024-08-30T09:07:24.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.523+0000 7f12f259c640 1 --2- 172.21.15.42:0/1472167904 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f12f4111a70 0x7f12f41153f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-30T09:07:24.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.523+0000 7f12f2d9d640 1 --2- 172.21.15.42:0/1472167904 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f12f410d990 0x7f12f411c120 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f12ec00b9e0 tx=0x7f12ec00bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:07:24.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.524+0000 7f12dbfff640 1 -- 172.21.15.42:0/1472167904 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12ec019070 con 0x7f12f410d990 2024-08-30T09:07:24.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.524+0000 7f12dbfff640 1 -- 172.21.15.42:0/1472167904 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f12ec00cba0 con 0x7f12f410d990 2024-08-30T09:07:24.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.524+0000 7f12f8a0a640 1 -- 172.21.15.42:0/1472167904 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12f411ae50 con 0x7f12f410d990 2024-08-30T09:07:24.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.524+0000 7f12dbfff640 1 -- 172.21.15.42:0/1472167904 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12ec012580 con 0x7f12f410d990 2024-08-30T09:07:24.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.524+0000 7f12f8a0a640 1 -- 172.21.15.42:0/1472167904 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12f411b410 con 0x7f12f410d990 2024-08-30T09:07:24.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.529+0000 7f12ff780740 1 Processor -- start 2024-08-30T09:07:24.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.530+0000 7f12dbfff640 1 -- 172.21.15.42:0/1472167904 <== mon.2 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7f12ec01c070 con 0x7f12f410d990 2024-08-30T09:07:24.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.532+0000 7f12ff780740 1 -- start start 2024-08-30T09:07:24.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.532+0000 7f12ff780740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e03f507070 0x55e03f8ac4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:07:24.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.532+0000 7f12ff780740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e03f4de1b0 0x55e03f8ac9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:07:24.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.532+0000 7f12ff780740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e03f4e6390 0x55e03f8acf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:07:24.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.532+0000 7f12ff780740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55e03f823680 con 0x55e03f507070 2024-08-30T09:07:24.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.532+0000 7f12ff780740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55e03f812c10 con 0x55e03f4e6390 2024-08-30T09:07:24.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.532+0000 7f12ff780740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55e03f816bc0 con 0x55e03f4de1b0 2024-08-30T09:07:24.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.532+0000 7f12f359e640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e03f507070 0x55e03f8ac4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:07:24.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.532+0000 7f12f359e640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e03f507070 0x55e03f8ac4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:50420/0 (socket says 172.21.15.42:50420) 2024-08-30T09:07:24.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.532+0000 7f12f359e640 1 -- 172.21.15.42:0/2699429294 learned_addr learned my addr 172.21.15.42:0/2699429294 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:07:24.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.533+0000 7f12f2d9d640 1 --2- 172.21.15.42:0/2699429294 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e03f4de1b0 0x55e03f8ac9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:07:24.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.533+0000 7f12dbfff640 1 --2- 172.21.15.42:0/1472167904 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f12dc0b4d30 0x7f12dc0b7240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:07:24.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.534+0000 7f12dbfff640 1 -- 172.21.15.42:0/1472167904 <== mon.2 v2:172.21.15.167:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f12ec0e4a40 con 0x7f12f410d990 2024-08-30T09:07:24.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.535+0000 7f12f259c640 1 --2- 172.21.15.42:0/2699429294 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e03f4e6390 0x55e03f8acf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:07:24.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.535+0000 7f12f2d9d640 1 -- 172.21.15.42:0/2699429294 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e03f4e6390 msgr2=0x55e03f8acf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:07:24.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.536+0000 7f12f359e640 1 --2- 172.21.15.42:0/1472167904 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f12dc0b4d30 0x7f12dc0b7240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:07:24.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.536+0000 7f12f2d9d640 1 --2- 172.21.15.42:0/2699429294 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e03f4e6390 0x55e03f8acf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:24.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.536+0000 7f12f2d9d640 1 -- 172.21.15.42:0/2699429294 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e03f507070 msgr2=0x55e03f8ac4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:07:24.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.536+0000 7f12f2d9d640 1 --2- 172.21.15.42:0/2699429294 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e03f507070 0x55e03f8ac4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:24.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.536+0000 7f12f2d9d640 1 -- 172.21.15.42:0/2699429294 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e03f572650 con 0x55e03f4de1b0 2024-08-30T09:07:24.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.536+0000 7f12f359e640 1 --2- 172.21.15.42:0/2699429294 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e03f507070 0x55e03f8ac4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-30T09:07:24.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.537+0000 7f12f2d9d640 1 --2- 172.21.15.42:0/2699429294 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e03f4de1b0 0x55e03f8ac9f0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f12ec0094d0 tx=0x7f12ec014d00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=136f5da09b1a877b server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:07:24.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.537+0000 7f12f359e640 1 --2- 172.21.15.42:0/1472167904 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f12dc0b4d30 0x7f12dc0b7240 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f12e80057c0 tx=0x7f12e8005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:07:24.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.537+0000 7f12d97fa640 1 -- 172.21.15.42:0/2699429294 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12ec0ab730 con 0x55e03f4de1b0 2024-08-30T09:07:24.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.537+0000 7f12d97fa640 1 -- 172.21.15.42:0/2699429294 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f12ec0abad0 con 0x55e03f4de1b0 2024-08-30T09:07:24.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.537+0000 7f12d97fa640 1 -- 172.21.15.42:0/2699429294 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12ec0b4e50 con 0x55e03f4de1b0 2024-08-30T09:07:24.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.538+0000 7f12ff780740 1 -- 172.21.15.42:0/2699429294 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e03f4de1b0 msgr2=0x55e03f8ac9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:07:24.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.538+0000 7f12ff780740 1 --2- 172.21.15.42:0/2699429294 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e03f4de1b0 0x55e03f8ac9f0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f12ec0094d0 tx=0x7f12ec014d00 comp rx=0 tx=0).stop 2024-08-30T09:07:24.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.541+0000 7f12ff780740 1 -- 172.21.15.42:0/2699429294 shutdown_connections 2024-08-30T09:07:24.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.541+0000 7f12ff780740 1 --2- 172.21.15.42:0/2699429294 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e03f4de1b0 0x55e03f8ac9f0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:24.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.541+0000 7f12ff780740 1 --2- 172.21.15.42:0/2699429294 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e03f4e6390 0x55e03f8acf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:24.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.541+0000 7f12ff780740 1 --2- 172.21.15.42:0/2699429294 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e03f507070 0x55e03f8ac4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:24.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.541+0000 7f12ff780740 1 -- 172.21.15.42:0/2699429294 >> 172.21.15.42:0/2699429294 conn(0x55e03f4fb460 msgr2=0x55e03f8aa3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:07:24.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.541+0000 7f12ff780740 1 -- 172.21.15.42:0/2699429294 shutdown_connections 2024-08-30T09:07:24.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.541+0000 7f12ff780740 1 -- 172.21.15.42:0/2699429294 wait complete. 2024-08-30T09:07:24.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.542+0000 7f12ff780740 1 -- start start 2024-08-30T09:07:24.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.542+0000 7f12ff780740 1 Processor -- start 2024-08-30T09:07:24.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.543+0000 7f12ff780740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e03f4de1b0 0x55e03f8b7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:07:24.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.543+0000 7f12ff780740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e03f4e6390 0x55e03f8b8270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:07:24.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.543+0000 7f12ff780740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e03f507070 0x55e03f8c3f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:07:24.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.543+0000 7f12ff780740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55e03f675700 con 0x55e03f4de1b0 2024-08-30T09:07:24.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.543+0000 7f12ff780740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55e03f799750 con 0x55e03f4e6390 2024-08-30T09:07:24.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.543+0000 7f12ff780740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55e03f8bac80 con 0x55e03f507070 2024-08-30T09:07:24.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.543+0000 7f12f359e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e03f507070 0x55e03f8c3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:07:24.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.543+0000 7f12f359e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e03f507070 0x55e03f8c3f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:39230/0 (socket says 172.21.15.42:39230) 2024-08-30T09:07:24.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.543+0000 7f12f359e640 1 -- 172.21.15.42:0/86504957 learned_addr learned my addr 172.21.15.42:0/86504957 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:07:24.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.543+0000 7f12ff780740 10 client.0 _ll_register_callbacks cb 0x55e03f7c8e10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:07:24.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.544+0000 7f12f2d9d640 1 --2- 172.21.15.42:0/86504957 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e03f4de1b0 0x55e03f8b7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:07:24.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.544+0000 7f12f259c640 1 --2- 172.21.15.42:0/86504957 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e03f4e6390 0x55e03f8b8270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:07:24.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.544+0000 7f12f359e640 1 -- 172.21.15.42:0/86504957 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e03f4e6390 msgr2=0x55e03f8b8270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:07:24.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.544+0000 7f12f359e640 1 --2- 172.21.15.42:0/86504957 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e03f4e6390 0x55e03f8b8270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:24.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.544+0000 7f12f359e640 1 -- 172.21.15.42:0/86504957 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e03f4de1b0 msgr2=0x55e03f8b7cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:07:24.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.544+0000 7f12f359e640 1 --2- 172.21.15.42:0/86504957 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e03f4de1b0 0x55e03f8b7cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:24.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.544+0000 7f12f359e640 1 -- 172.21.15.42:0/86504957 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e03f7d2900 con 0x55e03f507070 2024-08-30T09:07:24.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.545+0000 7f12d6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-08-30T09:07:24.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.545+0000 7f12f359e640 1 --2- 172.21.15.42:0/86504957 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e03f507070 0x55e03f8c3f70 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f12e8032fb0 tx=0x7f12e8009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f380aa97ba80e3d9 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:07:24.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.545+0000 7f12d6ff5640 1 -- 172.21.15.42:0/86504957 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12e804d020 con 0x55e03f507070 2024-08-30T09:07:24.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.545+0000 7f12ff780740 1 -- 172.21.15.42:0/86504957 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f12ec0abad0 con 0x55e03f507070 2024-08-30T09:07:24.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.545+0000 7f12d6ff5640 1 -- 172.21.15.42:0/86504957 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f12e8044050 con 0x55e03f507070 2024-08-30T09:07:24.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.546+0000 7f12d6ff5640 1 -- 172.21.15.42:0/86504957 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f12e804d020 con 0x55e03f507070 2024-08-30T09:07:24.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.546+0000 7f12ff780740 10 client.26308 fetch_fsmap learned FSMap version 34 2024-08-30T09:07:24.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.546+0000 7f12ff780740 1 -- 172.21.15.42:0/86504957 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55e03f8c6b60 con 0x55e03f507070 2024-08-30T09:07:24.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.547+0000 7f12d6ff5640 1 -- 172.21.15.42:0/86504957 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f12ec0abad0 con 0x55e03f507070 2024-08-30T09:07:24.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.547+0000 7f12d6ff5640 1 -- 172.21.15.42:0/86504957 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 34) ==== 8366+0+0 (secure 0 0 0) 0x7f12e8032020 con 0x55e03f507070 2024-08-30T09:07:24.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.547+0000 7f12ff780740 10 client.26308 fetch_fsmap finished waiting for FSMap version 34 2024-08-30T09:07:24.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.547+0000 7f12ff780740 10 client.26308 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-08-30T09:07:24.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.547+0000 7f12ff780740 20 client.26308 populate_metadata read hostname 'smithi042' 2024-08-30T09:07:24.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.547+0000 7f12ff780740 1 --2- 172.21.15.42:0/86504957 >> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] conn(0x55e03f4c59f0 0x55e03f8c9ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:07:24.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.547+0000 7f12ff780740 4 client.26308 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-30T09:07:24.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.547+0000 7f12ff780740 1 -- 172.21.15.42:0/86504957 --> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e03f8255d0 con 0x55e03f4c59f0 2024-08-30T09:07:24.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.547+0000 7f12f2d9d640 1 --2- 172.21.15.42:0/86504957 >> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] conn(0x55e03f4c59f0 0x55e03f8c9ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:07:24.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.548+0000 7f12d6ff5640 10 client.26308 ms_handle_connect on v2:172.21.15.190:6838/719981110 2024-08-30T09:07:24.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.549+0000 7f12f2d9d640 1 --2- 172.21.15.42:0/86504957 >> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] conn(0x55e03f4c59f0 0x55e03f8c9ff0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=170028673ab8fdaa server_cookie=eec5adf12d2a61b8 in_seq=0 out_seq=0 2024-08-30T09:07:24.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.550+0000 7f12d6ff5640 1 -- 172.21.15.42:0/86504957 <== mds.1 v2:172.21.15.190:6838/719981110 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55e03f8255d0 con 0x55e03f4c59f0 2024-08-30T09:07:24.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.550+0000 7f12d6ff5640 10 client.26308 handle_command_reply: tid=0 2024-08-30T09:07:24.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.550+0000 7f12ff780740 1 client.26308 shutdown 2024-08-30T09:07:24.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.550+0000 7f12ff780740 1 -- 172.21.15.42:0/86504957 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e03f507070 msgr2=0x55e03f8c3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:07:24.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.550+0000 7f12ff780740 1 --2- 172.21.15.42:0/86504957 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e03f507070 0x55e03f8c3f70 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f12e8032fb0 tx=0x7f12e8009260 comp rx=0 tx=0).stop 2024-08-30T09:07:24.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.550+0000 7f12ff780740 1 -- 172.21.15.42:0/86504957 shutdown_connections 2024-08-30T09:07:24.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.550+0000 7f12ff780740 1 --2- 172.21.15.42:0/86504957 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e03f507070 0x55e03f8c3f70 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:24.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.550+0000 7f12ff780740 1 --2- 172.21.15.42:0/86504957 >> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] conn(0x55e03f4c59f0 0x55e03f8c9ff0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:24.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.551+0000 7f12d6ff5640 0 client.26308 ms_handle_reset on v2:172.21.15.190:6838/719981110 2024-08-30T09:07:24.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.551+0000 7f12ff780740 1 --2- 172.21.15.42:0/86504957 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e03f4e6390 0x55e03f8b8270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:24.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.551+0000 7f12ff780740 1 --2- 172.21.15.42:0/86504957 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e03f4de1b0 0x55e03f8b7cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:24.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.551+0000 7f12ff780740 1 -- 172.21.15.42:0/86504957 >> 172.21.15.42:0/86504957 conn(0x55e03f4fb460 msgr2=0x55e03f8af690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:07:24.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.551+0000 7f12ff780740 1 -- 172.21.15.42:0/86504957 shutdown_connections 2024-08-30T09:07:24.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.551+0000 7f12ff780740 1 -- 172.21.15.42:0/86504957 wait complete. 2024-08-30T09:07:24.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.551+0000 7f12ff780740 20 client.26308 trim_cache size 0 max 16384 2024-08-30T09:07:24.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.572+0000 7f12ff780740 1 Processor -- start 2024-08-30T09:07:24.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.572+0000 7f12ff780740 1 -- start start 2024-08-30T09:07:24.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.573+0000 7f12ff780740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e03f4de1b0 0x55e03f8adea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:07:24.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.573+0000 7f12ff780740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e03f4e6390 0x55e03f8b9900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:07:24.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.573+0000 7f12ff780740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e03f507070 0x55e03f8c3e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:07:24.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.573+0000 7f12ff780740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f12f41ca820 con 0x55e03f4de1b0 2024-08-30T09:07:24.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.573+0000 7f12ff780740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f12f41c8090 con 0x55e03f507070 2024-08-30T09:07:24.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.573+0000 7f12ff780740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f12f41c5e80 con 0x55e03f4e6390 2024-08-30T09:07:24.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.573+0000 7f12f2d9d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e03f4de1b0 0x55e03f8adea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:07:24.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.573+0000 7f12f2d9d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e03f4de1b0 0x55e03f8adea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:50432/0 (socket says 172.21.15.42:50432) 2024-08-30T09:07:24.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.573+0000 7f12f2d9d640 1 -- 172.21.15.42:0/2849663523 learned_addr learned my addr 172.21.15.42:0/2849663523 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:07:24.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.573+0000 7f12f259c640 1 --2- 172.21.15.42:0/2849663523 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e03f4e6390 0x55e03f8b9900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:07:24.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.573+0000 7f12f359e640 1 --2- 172.21.15.42:0/2849663523 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e03f507070 0x55e03f8c3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:07:24.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.573+0000 7f12f2d9d640 1 -- 172.21.15.42:0/2849663523 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e03f4e6390 msgr2=0x55e03f8b9900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:07:24.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.573+0000 7f12f2d9d640 1 --2- 172.21.15.42:0/2849663523 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e03f4e6390 0x55e03f8b9900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:24.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.573+0000 7f12f2d9d640 1 -- 172.21.15.42:0/2849663523 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e03f507070 msgr2=0x55e03f8c3e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:07:24.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.573+0000 7f12f2d9d640 1 --2- 172.21.15.42:0/2849663523 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e03f507070 0x55e03f8c3e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:24.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.573+0000 7f12f2d9d640 1 -- 172.21.15.42:0/2849663523 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e03f8aed40 con 0x55e03f4de1b0 2024-08-30T09:07:24.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.574+0000 7f12f2d9d640 1 --2- 172.21.15.42:0/2849663523 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e03f4de1b0 0x55e03f8adea0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f12ec0a8040 tx=0x7f12ec00e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbcd65cae47a221a server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:07:24.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.574+0000 7f12d8ff9640 1 -- 172.21.15.42:0/2849663523 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12ec0abad0 con 0x55e03f4de1b0 2024-08-30T09:07:24.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.574+0000 7f12d8ff9640 1 -- 172.21.15.42:0/2849663523 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55e03f8255d0 con 0x55e03f4de1b0 2024-08-30T09:07:24.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.574+0000 7f12d8ff9640 1 -- 172.21.15.42:0/2849663523 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12ec0b4e50 con 0x55e03f4de1b0 2024-08-30T09:07:24.580 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.575+0000 7f12ff780740 1 -- 172.21.15.42:0/2849663523 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e03f4de1b0 msgr2=0x55e03f8adea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:07:24.580 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.575+0000 7f12ff780740 1 --2- 172.21.15.42:0/2849663523 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e03f4de1b0 0x55e03f8adea0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f12ec0a8040 tx=0x7f12ec00e040 comp rx=0 tx=0).stop 2024-08-30T09:07:24.580 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.575+0000 7f12ff780740 1 -- 172.21.15.42:0/2849663523 shutdown_connections 2024-08-30T09:07:24.580 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.575+0000 7f12ff780740 1 --2- 172.21.15.42:0/2849663523 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e03f4e6390 0x55e03f8b9900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:24.580 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.575+0000 7f12ff780740 1 --2- 172.21.15.42:0/2849663523 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e03f507070 0x55e03f8c3e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:24.580 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.575+0000 7f12ff780740 1 --2- 172.21.15.42:0/2849663523 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e03f4de1b0 0x55e03f8adea0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:24.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.575+0000 7f12ff780740 1 -- 172.21.15.42:0/2849663523 >> 172.21.15.42:0/2849663523 conn(0x55e03f4fb460 msgr2=0x55e03f8b91f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:07:24.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.575+0000 7f12ff780740 1 -- 172.21.15.42:0/2849663523 shutdown_connections 2024-08-30T09:07:24.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.575+0000 7f12ff780740 1 -- 172.21.15.42:0/2849663523 wait complete. 2024-08-30T09:07:24.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.576+0000 7f12ff780740 1 -- start start 2024-08-30T09:07:24.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.576+0000 7f12ff780740 1 Processor -- start 2024-08-30T09:07:24.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.576+0000 7f12ff780740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e03f4de1b0 0x55e03f8bc6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:07:24.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.576+0000 7f12ff780740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e03f4e6390 0x55e03f8bccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:07:24.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.576+0000 7f12ff780740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e03f507070 0x55e03f9339b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:07:24.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.576+0000 7f12ff780740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f12f41c6550 con 0x55e03f4e6390 2024-08-30T09:07:24.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.576+0000 7f12ff780740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f12f41c79c0 con 0x55e03f4de1b0 2024-08-30T09:07:24.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.576+0000 7f12ff780740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f12f41c6c20 con 0x55e03f507070 2024-08-30T09:07:24.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.577+0000 7f12ff780740 10 client.0 _ll_register_callbacks cb 0x7f12e8009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:07:24.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.577+0000 7f12f2d9d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e03f4de1b0 0x55e03f8bc6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:07:24.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.577+0000 7f12f2d9d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e03f4de1b0 0x55e03f8bc6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:44276/0 (socket says 172.21.15.42:44276) 2024-08-30T09:07:24.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.577+0000 7f12f2d9d640 1 -- 172.21.15.42:0/2002824502 learned_addr learned my addr 172.21.15.42:0/2002824502 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:07:24.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.577+0000 7f12f259c640 1 --2- 172.21.15.42:0/2002824502 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e03f4e6390 0x55e03f8bccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:07:24.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.577+0000 7f12f359e640 1 --2- 172.21.15.42:0/2002824502 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e03f507070 0x55e03f9339b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:07:24.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.577+0000 7f12f2d9d640 1 -- 172.21.15.42:0/2002824502 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e03f507070 msgr2=0x55e03f9339b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:07:24.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.577+0000 7f12f2d9d640 1 --2- 172.21.15.42:0/2002824502 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e03f507070 0x55e03f9339b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:24.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.577+0000 7f12f2d9d640 1 -- 172.21.15.42:0/2002824502 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e03f4e6390 msgr2=0x55e03f8bccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:07:24.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.577+0000 7f12f2d9d640 1 --2- 172.21.15.42:0/2002824502 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e03f4e6390 0x55e03f8bccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:24.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.577+0000 7f12f2d9d640 1 -- 172.21.15.42:0/2002824502 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e03f934020 con 0x55e03f4de1b0 2024-08-30T09:07:24.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.578+0000 7f12f2d9d640 1 --2- 172.21.15.42:0/2002824502 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e03f4de1b0 0x55e03f8bc6f0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f12ec0a9230 tx=0x7f12ec0b52e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4930822b7aa334bf server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:07:24.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.578+0000 7f12d6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-30T09:07:24.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.578+0000 7f12d6ff5640 1 -- 172.21.15.42:0/2002824502 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12ec0b9070 con 0x55e03f4de1b0 2024-08-30T09:07:24.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.579+0000 7f12d6ff5640 1 -- 172.21.15.42:0/2002824502 <== mon.2 v2:172.21.15.167:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f12ec0e2260 con 0x55e03f4de1b0 2024-08-30T09:07:24.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.579+0000 7f12ff780740 1 -- 172.21.15.42:0/2002824502 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55e03f8255d0 con 0x55e03f4de1b0 2024-08-30T09:07:24.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.579+0000 7f12d6ff5640 1 -- 172.21.15.42:0/2002824502 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f12ec0a82c0 con 0x55e03f4de1b0 2024-08-30T09:07:24.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.579+0000 7f12ff780740 10 client.26336 fetch_fsmap learned FSMap version 34 2024-08-30T09:07:24.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.579+0000 7f12ff780740 1 -- 172.21.15.42:0/2002824502 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55e03f935280 con 0x55e03f4de1b0 2024-08-30T09:07:24.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.579+0000 7f12d6ff5640 1 -- 172.21.15.42:0/2002824502 <== mon.2 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f12ec0b4e50 con 0x55e03f4de1b0 2024-08-30T09:07:24.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.580+0000 7f12d6ff5640 1 -- 172.21.15.42:0/2002824502 <== mon.2 v2:172.21.15.167:3300/0 5 ==== fsmap(e 34) ==== 8366+0+0 (secure 0 0 0) 0x7f12ec0bc020 con 0x55e03f4de1b0 2024-08-30T09:07:24.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.580+0000 7f12ff780740 10 client.26336 fetch_fsmap finished waiting for FSMap version 34 2024-08-30T09:07:24.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.580+0000 7f12ff780740 10 client.26336 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-08-30T09:07:24.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.580+0000 7f12ff780740 20 client.26336 populate_metadata read hostname 'smithi042' 2024-08-30T09:07:24.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.580+0000 7f12ff780740 1 --2- 172.21.15.42:0/2002824502 >> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] conn(0x55e03f49ada0 0x55e03f937510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:07:24.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.580+0000 7f12ff780740 4 client.26336 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-30T09:07:24.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.580+0000 7f12ff780740 1 -- 172.21.15.42:0/2002824502 --> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e03f8255d0 con 0x55e03f49ada0 2024-08-30T09:07:24.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.580+0000 7f12f259c640 1 --2- 172.21.15.42:0/2002824502 >> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] conn(0x55e03f49ada0 0x55e03f937510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:07:24.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.581+0000 7f12f259c640 1 --2- 172.21.15.42:0/2002824502 >> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] conn(0x55e03f49ada0 0x55e03f937510 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=11ca073706178ecc server_cookie=361277789479ed18 in_seq=0 out_seq=0 2024-08-30T09:07:24.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.581+0000 7f12d6ff5640 10 client.26336 ms_handle_connect on v2:172.21.15.190:6838/719981110 2024-08-30T09:07:24.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.581+0000 7f12d6ff5640 1 -- 172.21.15.42:0/2002824502 <== mds.1 v2:172.21.15.190:6838/719981110 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e03f8255d0 con 0x55e03f49ada0 2024-08-30T09:07:24.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.581+0000 7f12d6ff5640 10 client.26336 handle_command_reply: tid=0 2024-08-30T09:07:24.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.581+0000 7f12ff780740 1 client.26336 shutdown 2024-08-30T09:07:24.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.582+0000 7f12ff780740 1 -- 172.21.15.42:0/2002824502 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e03f4de1b0 msgr2=0x55e03f8bc6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:07:24.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.582+0000 7f12ff780740 1 --2- 172.21.15.42:0/2002824502 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e03f4de1b0 0x55e03f8bc6f0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f12ec0a9230 tx=0x7f12ec0b52e0 comp rx=0 tx=0).stop 2024-08-30T09:07:24.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.582+0000 7f12ff780740 1 -- 172.21.15.42:0/2002824502 shutdown_connections 2024-08-30T09:07:24.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.582+0000 7f12ff780740 1 --2- 172.21.15.42:0/2002824502 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e03f507070 0x55e03f9339b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:24.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.582+0000 7f12ff780740 1 --2- 172.21.15.42:0/2002824502 >> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] conn(0x55e03f49ada0 0x55e03f937510 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:24.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.582+0000 7f12ff780740 1 --2- 172.21.15.42:0/2002824502 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e03f4e6390 0x55e03f8bccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:24.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.582+0000 7f12ff780740 1 --2- 172.21.15.42:0/2002824502 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e03f4de1b0 0x55e03f8bc6f0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:24.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.582+0000 7f12d6ff5640 0 client.26336 ms_handle_reset on v2:172.21.15.190:6838/719981110 2024-08-30T09:07:24.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.582+0000 7f12ff780740 1 -- 172.21.15.42:0/2002824502 >> 172.21.15.42:0/2002824502 conn(0x55e03f4fb460 msgr2=0x55e03f8adc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:07:24.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.582+0000 7f12ff780740 1 -- 172.21.15.42:0/2002824502 shutdown_connections 2024-08-30T09:07:24.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.582+0000 7f12ff780740 1 -- 172.21.15.42:0/2002824502 wait complete. 2024-08-30T09:07:24.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.582+0000 7f12ff780740 20 client.26336 trim_cache size 0 max 16384 2024-08-30T09:07:24.587 INFO:teuthology.orchestra.run.smithi042.stdout:[] 2024-08-30T09:07:24.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.583+0000 7f12f8a0a640 1 -- 172.21.15.42:0/1472167904 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f12dc0b4d30 msgr2=0x7f12dc0b7240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:07:24.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.583+0000 7f12f8a0a640 1 --2- 172.21.15.42:0/1472167904 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f12dc0b4d30 0x7f12dc0b7240 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f12e80057c0 tx=0x7f12e8005fd0 comp rx=0 tx=0).stop 2024-08-30T09:07:24.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.583+0000 7f12f8a0a640 1 -- 172.21.15.42:0/1472167904 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f12f410d990 msgr2=0x7f12f411c120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:07:24.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.583+0000 7f12f8a0a640 1 --2- 172.21.15.42:0/1472167904 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f12f410d990 0x7f12f411c120 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f12ec00b9e0 tx=0x7f12ec00bea0 comp rx=0 tx=0).stop 2024-08-30T09:07:24.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.583+0000 7f12f8a0a640 1 -- 172.21.15.42:0/1472167904 shutdown_connections 2024-08-30T09:07:24.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.583+0000 7f12f8a0a640 1 --2- 172.21.15.42:0/1472167904 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f12dc0b4d30 0x7f12dc0b7240 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:24.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.583+0000 7f12f8a0a640 1 --2- 172.21.15.42:0/1472167904 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f12f4111a70 0x7f12f41153f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:24.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.583+0000 7f12f8a0a640 1 --2- 172.21.15.42:0/1472167904 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f12f411c940 0x7f12f4115970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:24.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.583+0000 7f12f8a0a640 1 --2- 172.21.15.42:0/1472167904 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f12f410d990 0x7f12f411c120 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:24.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.583+0000 7f12f8a0a640 1 -- 172.21.15.42:0/1472167904 >> 172.21.15.42:0/1472167904 conn(0x7f12f406e790 msgr2=0x7f12f4110720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:07:24.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.583+0000 7f12f8a0a640 1 -- 172.21.15.42:0/1472167904 shutdown_connections 2024-08-30T09:07:24.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:24.583+0000 7f12f8a0a640 1 -- 172.21.15.42:0/1472167904 wait complete. 2024-08-30T09:07:24.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:07:24 smithi042 ceph-mon[18652]: pgmap v572: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 9.9 MiB/s wr, 696 op/s 2024-08-30T09:07:24.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:07:24 smithi167 ceph-mon[22063]: pgmap v572: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 9.9 MiB/s wr, 696 op/s 2024-08-30T09:07:24.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:07:24 smithi190 ceph-mon[22001]: pgmap v572: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 9.9 MiB/s wr, 696 op/s 2024-08-30T09:07:25.294 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-08-30T09:07:25.770 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T09:07:25.953 INFO:teuthology.orchestra.run.smithi042.stderr:progress: 400.0 s, 729.6 tps, lat 132.826 ms stddev 154.642 2024-08-30T09:07:26.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.437+0000 7f8c76817640 1 -- 172.21.15.42:0/4025033885 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8c701028d0 msgr2=0x7f8c70102ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:07:26.441 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.437+0000 7f8c76817640 1 --2- 172.21.15.42:0/4025033885 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8c701028d0 0x7f8c70102ce0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f8c60009900 tx=0x7f8c6002a410 comp rx=0 tx=0).stop 2024-08-30T09:07:26.441 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.437+0000 7f8c76817640 1 -- 172.21.15.42:0/4025033885 shutdown_connections 2024-08-30T09:07:26.441 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.437+0000 7f8c76817640 1 --2- 172.21.15.42:0/4025033885 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8c701058a0 0x7f8c7010f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:26.441 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.437+0000 7f8c76817640 1 --2- 172.21.15.42:0/4025033885 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8c70104f50 0x7f8c70105360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:26.441 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.437+0000 7f8c76817640 1 --2- 172.21.15.42:0/4025033885 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8c701028d0 0x7f8c70102ce0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:26.441 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.437+0000 7f8c76817640 1 -- 172.21.15.42:0/4025033885 >> 172.21.15.42:0/4025033885 conn(0x7f8c70079740 msgr2=0x7f8c70079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:07:26.441 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.438+0000 7f8c76817640 1 -- 172.21.15.42:0/4025033885 shutdown_connections 2024-08-30T09:07:26.441 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.438+0000 7f8c76817640 1 -- 172.21.15.42:0/4025033885 wait complete. 2024-08-30T09:07:26.441 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.439+0000 7f8c76817640 1 Processor -- start 2024-08-30T09:07:26.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.439+0000 7f8c76817640 1 -- start start 2024-08-30T09:07:26.443 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.439+0000 7f8c76817640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8c701028d0 0x7f8c701a0880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:07:26.443 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.439+0000 7f8c76817640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8c70104f50 0x7f8c701a0e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:07:26.444 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.439+0000 7f8c76817640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8c701058a0 0x7f8c701a60c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:07:26.444 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.439+0000 7f8c76817640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f8c701127b0 con 0x7f8c701058a0 2024-08-30T09:07:26.444 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.439+0000 7f8c76817640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f8c70112630 con 0x7f8c701028d0 2024-08-30T09:07:26.444 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.439+0000 7f8c76817640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f8c70112930 con 0x7f8c70104f50 2024-08-30T09:07:26.444 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.439+0000 7f8c6ffff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8c701028d0 0x7f8c701a0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:07:26.444 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.439+0000 7f8c6f7fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8c70104f50 0x7f8c701a0e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:07:26.444 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.439+0000 7f8c6f7fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8c70104f50 0x7f8c701a0e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:39276/0 (socket says 172.21.15.42:39276) 2024-08-30T09:07:26.444 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.439+0000 7f8c6ffff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8c701028d0 0x7f8c701a0880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:44306/0 (socket says 172.21.15.42:44306) 2024-08-30T09:07:26.444 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.439+0000 7f8c6f7fe640 1 -- 172.21.15.42:0/2645944807 learned_addr learned my addr 172.21.15.42:0/2645944807 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:07:26.444 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.440+0000 7f8c74db4640 1 --2- 172.21.15.42:0/2645944807 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8c701058a0 0x7f8c701a60c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:07:26.444 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.440+0000 7f8c6f7fe640 1 -- 172.21.15.42:0/2645944807 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8c701028d0 msgr2=0x7f8c701a0880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:07:26.444 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.440+0000 7f8c6f7fe640 1 --2- 172.21.15.42:0/2645944807 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8c701028d0 0x7f8c701a0880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:26.444 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.440+0000 7f8c6f7fe640 1 -- 172.21.15.42:0/2645944807 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8c701058a0 msgr2=0x7f8c701a60c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:07:26.445 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.440+0000 7f8c6f7fe640 1 --2- 172.21.15.42:0/2645944807 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8c701058a0 0x7f8c701a60c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:26.445 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.440+0000 7f8c6f7fe640 1 -- 172.21.15.42:0/2645944807 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c701a6730 con 0x7f8c70104f50 2024-08-30T09:07:26.445 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.440+0000 7f8c74db4640 1 --2- 172.21.15.42:0/2645944807 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8c701058a0 0x7f8c701a60c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-30T09:07:26.445 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.440+0000 7f8c6f7fe640 1 --2- 172.21.15.42:0/2645944807 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8c70104f50 0x7f8c701a0e20 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f8c6400c8c0 tx=0x7f8c6400cd80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:07:26.445 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.440+0000 7f8c6d7fa640 1 -- 172.21.15.42:0/2645944807 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c64007bb0 con 0x7f8c70104f50 2024-08-30T09:07:26.445 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.440+0000 7f8c76817640 1 -- 172.21.15.42:0/2645944807 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c701a6a20 con 0x7f8c70104f50 2024-08-30T09:07:26.445 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.440+0000 7f8c6d7fa640 1 -- 172.21.15.42:0/2645944807 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8c64007d50 con 0x7f8c70104f50 2024-08-30T09:07:26.445 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.440+0000 7f8c6d7fa640 1 -- 172.21.15.42:0/2645944807 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c640187c0 con 0x7f8c70104f50 2024-08-30T09:07:26.445 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.440+0000 7f8c76817640 1 -- 172.21.15.42:0/2645944807 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c701ad240 con 0x7f8c70104f50 2024-08-30T09:07:26.446 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.444+0000 7f8c6d7fa640 1 -- 172.21.15.42:0/2645944807 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7f8c64018960 con 0x7f8c70104f50 2024-08-30T09:07:26.446 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.444+0000 7f8c7cd8c740 1 Processor -- start 2024-08-30T09:07:26.446 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.444+0000 7f8c6d7fa640 1 --2- 172.21.15.42:0/2645944807 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f8c580b4b60 0x7f8c580b7070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:07:26.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.445+0000 7f8c6d7fa640 1 -- 172.21.15.42:0/2645944807 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f8c640e5640 con 0x7f8c70104f50 2024-08-30T09:07:26.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.445+0000 7f8c7cd8c740 1 -- start start 2024-08-30T09:07:26.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.445+0000 7f8c74db4640 1 --2- 172.21.15.42:0/2645944807 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f8c580b4b60 0x7f8c580b7070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:07:26.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.445+0000 7f8c7cd8c740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e2bebe5070 0x55e2bef8a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:07:26.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.445+0000 7f8c7cd8c740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e2bebbc1b0 0x55e2bef8a850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:07:26.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.445+0000 7f8c7cd8c740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e2bebc4390 0x55e2bef8ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:07:26.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.445+0000 7f8c7cd8c740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55e2bef01680 con 0x55e2bebe5070 2024-08-30T09:07:26.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.445+0000 7f8c7cd8c740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55e2beef0c10 con 0x55e2bebc4390 2024-08-30T09:07:26.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.445+0000 7f8c7cd8c740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55e2beef4bc0 con 0x55e2bebbc1b0 2024-08-30T09:07:26.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.445+0000 7f8c6ffff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e2bebe5070 0x55e2bef8a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:07:26.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.445+0000 7f8c74db4640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e2bebc4390 0x55e2bef8ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:07:26.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.445+0000 7f8c6ffff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e2bebe5070 0x55e2bef8a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:50464/0 (socket says 172.21.15.42:50464) 2024-08-30T09:07:26.450 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.445+0000 7f8c6ffff640 1 -- 172.21.15.42:0/4086681338 learned_addr learned my addr 172.21.15.42:0/4086681338 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:07:26.450 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.445+0000 7f8c74db4640 1 --2- 172.21.15.42:0/4086681338 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e2bebc4390 0x55e2bef8ad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:44310/0 (socket says 172.21.15.42:44310) 2024-08-30T09:07:26.450 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.445+0000 7f8c6f7fe640 1 --2- 172.21.15.42:0/4086681338 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e2bebbc1b0 0x55e2bef8a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:07:26.450 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.446+0000 7f8c74db4640 1 --2- 172.21.15.42:0/2645944807 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f8c580b4b60 0x7f8c580b7070 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f8c68005fd0 tx=0x7f8c68010740 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:07:26.450 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.446+0000 7f8c74db4640 1 -- 172.21.15.42:0/4086681338 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e2bebbc1b0 msgr2=0x55e2bef8a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:07:26.450 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.446+0000 7f8c74db4640 1 --2- 172.21.15.42:0/4086681338 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e2bebbc1b0 0x55e2bef8a850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:26.450 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.446+0000 7f8c74db4640 1 -- 172.21.15.42:0/4086681338 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e2bebe5070 msgr2=0x55e2bef8a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:07:26.450 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.446+0000 7f8c74db4640 1 --2- 172.21.15.42:0/4086681338 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e2bebe5070 0x55e2bef8a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:26.450 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.446+0000 7f8c74db4640 1 -- 172.21.15.42:0/4086681338 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e2bec50650 con 0x55e2bebc4390 2024-08-30T09:07:26.450 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.446+0000 7f8c74db4640 1 --2- 172.21.15.42:0/4086681338 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e2bebc4390 0x55e2bef8ad90 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f8c680136d0 tx=0x7f8c6801b000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5c41891034167580 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:07:26.450 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.446+0000 7f8c567fc640 1 -- 172.21.15.42:0/4086681338 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c6800e7e0 con 0x55e2bebc4390 2024-08-30T09:07:26.450 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.446+0000 7f8c567fc640 1 -- 172.21.15.42:0/4086681338 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8c6800ee30 con 0x55e2bebc4390 2024-08-30T09:07:26.450 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.446+0000 7f8c567fc640 1 -- 172.21.15.42:0/4086681338 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c680113e0 con 0x55e2bebc4390 2024-08-30T09:07:26.454 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.448+0000 7f8c7cd8c740 1 -- 172.21.15.42:0/4086681338 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e2bebc4390 msgr2=0x55e2bef8ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:07:26.454 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.448+0000 7f8c7cd8c740 1 --2- 172.21.15.42:0/4086681338 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e2bebc4390 0x55e2bef8ad90 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f8c680136d0 tx=0x7f8c6801b000 comp rx=0 tx=0).stop 2024-08-30T09:07:26.454 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.448+0000 7f8c7cd8c740 1 -- 172.21.15.42:0/4086681338 shutdown_connections 2024-08-30T09:07:26.454 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.448+0000 7f8c7cd8c740 1 --2- 172.21.15.42:0/4086681338 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e2bebbc1b0 0x55e2bef8a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:26.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.448+0000 7f8c7cd8c740 1 --2- 172.21.15.42:0/4086681338 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e2bebc4390 0x55e2bef8ad90 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:26.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.448+0000 7f8c7cd8c740 1 --2- 172.21.15.42:0/4086681338 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e2bebe5070 0x55e2bef8a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:26.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.448+0000 7f8c7cd8c740 1 -- 172.21.15.42:0/4086681338 >> 172.21.15.42:0/4086681338 conn(0x55e2bebd9460 msgr2=0x55e2bef88200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:07:26.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.448+0000 7f8c7cd8c740 1 -- 172.21.15.42:0/4086681338 shutdown_connections 2024-08-30T09:07:26.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.448+0000 7f8c7cd8c740 1 -- 172.21.15.42:0/4086681338 wait complete. 2024-08-30T09:07:26.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.448+0000 7f8c7cd8c740 1 -- start start 2024-08-30T09:07:26.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.449+0000 7f8c7cd8c740 1 Processor -- start 2024-08-30T09:07:26.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.449+0000 7f8c7cd8c740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e2bebbc1b0 0x55e2bef95a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:07:26.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.449+0000 7f8c7cd8c740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e2bebc4390 0x55e2bef95fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:07:26.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.449+0000 7f8c7cd8c740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e2bebe5070 0x55e2befa1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:07:26.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.449+0000 7f8c7cd8c740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55e2bed53700 con 0x55e2bebbc1b0 2024-08-30T09:07:26.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.449+0000 7f8c7cd8c740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55e2bee77750 con 0x55e2bebe5070 2024-08-30T09:07:26.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.449+0000 7f8c7cd8c740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55e2bef98af0 con 0x55e2bebc4390 2024-08-30T09:07:26.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.449+0000 7f8c6f7fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e2bebc4390 0x55e2bef95fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:07:26.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.449+0000 7f8c6f7fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e2bebc4390 0x55e2bef95fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:39282/0 (socket says 172.21.15.42:39282) 2024-08-30T09:07:26.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.449+0000 7f8c6f7fe640 1 -- 172.21.15.42:0/3800375533 learned_addr learned my addr 172.21.15.42:0/3800375533 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:07:26.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.450+0000 7f8c6ffff640 1 --2- 172.21.15.42:0/3800375533 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e2bebbc1b0 0x55e2bef95a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:07:26.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.450+0000 7f8c7cd8c740 10 client.0 _ll_register_callbacks cb 0x55e2beea6e10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:07:26.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.450+0000 7f8c74db4640 1 --2- 172.21.15.42:0/3800375533 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e2bebe5070 0x55e2befa1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:07:26.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.450+0000 7f8c6f7fe640 1 -- 172.21.15.42:0/3800375533 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e2bebe5070 msgr2=0x55e2befa1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:07:26.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.450+0000 7f8c6f7fe640 1 --2- 172.21.15.42:0/3800375533 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e2bebe5070 0x55e2befa1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:26.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.450+0000 7f8c6f7fe640 1 -- 172.21.15.42:0/3800375533 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e2bebbc1b0 msgr2=0x55e2bef95a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:07:26.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.450+0000 7f8c6f7fe640 1 --2- 172.21.15.42:0/3800375533 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e2bebbc1b0 0x55e2bef95a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:26.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.450+0000 7f8c6f7fe640 1 -- 172.21.15.42:0/3800375533 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e2beeb0900 con 0x55e2bebc4390 2024-08-30T09:07:26.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.450+0000 7f8c6ffff640 1 --2- 172.21.15.42:0/3800375533 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e2bebbc1b0 0x55e2bef95a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-30T09:07:26.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.450+0000 7f8c6f7fe640 1 --2- 172.21.15.42:0/3800375533 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e2bebc4390 0x55e2bef95fd0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f8c640a8bd0 tx=0x7f8c640a8450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b8a176dc7e81fbd0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:07:26.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.450+0000 7f8c4f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-08-30T09:07:26.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.450+0000 7f8c4f7fe640 1 -- 172.21.15.42:0/3800375533 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c640acba0 con 0x55e2bebc4390 2024-08-30T09:07:26.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.451+0000 7f8c4f7fe640 1 -- 172.21.15.42:0/3800375533 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f8c640b0170 con 0x55e2bebc4390 2024-08-30T09:07:26.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.451+0000 7f8c7cd8c740 1 -- 172.21.15.42:0/3800375533 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8c6800ee30 con 0x55e2bebc4390 2024-08-30T09:07:26.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.451+0000 7f8c4f7fe640 1 -- 172.21.15.42:0/3800375533 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f8c640b04f0 con 0x55e2bebc4390 2024-08-30T09:07:26.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.451+0000 7f8c7cd8c740 10 client.26326 fetch_fsmap learned FSMap version 34 2024-08-30T09:07:26.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.451+0000 7f8c7cd8c740 1 -- 172.21.15.42:0/3800375533 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55e2befa4930 con 0x55e2bebc4390 2024-08-30T09:07:26.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.451+0000 7f8c4f7fe640 1 -- 172.21.15.42:0/3800375533 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8c640b4fa0 con 0x55e2bebc4390 2024-08-30T09:07:26.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.452+0000 7f8c4f7fe640 1 -- 172.21.15.42:0/3800375533 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 34) ==== 8366+0+0 (secure 0 0 0) 0x7f8c640a8510 con 0x55e2bebc4390 2024-08-30T09:07:26.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.452+0000 7f8c7cd8c740 10 client.26326 fetch_fsmap finished waiting for FSMap version 34 2024-08-30T09:07:26.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.452+0000 7f8c7cd8c740 10 client.26326 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-30T09:07:26.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.452+0000 7f8c7cd8c740 20 client.26326 populate_metadata read hostname 'smithi042' 2024-08-30T09:07:26.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.452+0000 7f8c7cd8c740 1 --2- 172.21.15.42:0/3800375533 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55e2beba39f0 0x55e2befa7d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:07:26.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.452+0000 7f8c7cd8c740 4 client.26326 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-30T09:07:26.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.452+0000 7f8c7cd8c740 1 -- 172.21.15.42:0/3800375533 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f8c6800ee30 con 0x55e2beba39f0 2024-08-30T09:07:26.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.452+0000 7f8c6ffff640 1 --2- 172.21.15.42:0/3800375533 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55e2beba39f0 0x55e2befa7d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:07:26.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.453+0000 7f8c6ffff640 1 --2- 172.21.15.42:0/3800375533 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55e2beba39f0 0x55e2befa7d50 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=67bb296d424e7327 server_cookie=45690a5c50ab6063 in_seq=0 out_seq=0 2024-08-30T09:07:26.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.453+0000 7f8c4f7fe640 10 client.26326 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:07:26.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.455+0000 7f8c4f7fe640 1 -- 172.21.15.42:0/3800375533 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f8c6800ee30 con 0x55e2beba39f0 2024-08-30T09:07:26.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.455+0000 7f8c4f7fe640 10 client.26326 handle_command_reply: tid=0 2024-08-30T09:07:26.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.455+0000 7f8c7cd8c740 1 client.26326 shutdown 2024-08-30T09:07:26.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.456+0000 7f8c7cd8c740 1 -- 172.21.15.42:0/3800375533 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e2bebc4390 msgr2=0x55e2bef95fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:07:26.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.456+0000 7f8c7cd8c740 1 --2- 172.21.15.42:0/3800375533 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e2bebc4390 0x55e2bef95fd0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f8c640a8bd0 tx=0x7f8c640a8450 comp rx=0 tx=0).stop 2024-08-30T09:07:26.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.456+0000 7f8c7cd8c740 1 -- 172.21.15.42:0/3800375533 shutdown_connections 2024-08-30T09:07:26.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.456+0000 7f8c7cd8c740 1 --2- 172.21.15.42:0/3800375533 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55e2beba39f0 0x55e2befa7d50 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:26.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.456+0000 7f8c4f7fe640 0 client.26326 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:07:26.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.456+0000 7f8c7cd8c740 1 --2- 172.21.15.42:0/3800375533 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e2bebc4390 0x55e2bef95fd0 secure :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f8c640a8bd0 tx=0x7f8c640a8450 comp rx=0 tx=0).stop 2024-08-30T09:07:26.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.456+0000 7f8c7cd8c740 1 --2- 172.21.15.42:0/3800375533 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e2bebe5070 0x55e2befa1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:26.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.456+0000 7f8c7cd8c740 1 --2- 172.21.15.42:0/3800375533 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e2bebbc1b0 0x55e2bef95a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:26.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.456+0000 7f8c7cd8c740 1 -- 172.21.15.42:0/3800375533 >> 172.21.15.42:0/3800375533 conn(0x55e2bebd9460 msgr2=0x55e2bef8d4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:07:26.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.457+0000 7f8c7cd8c740 1 -- 172.21.15.42:0/3800375533 shutdown_connections 2024-08-30T09:07:26.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.457+0000 7f8c7cd8c740 1 -- 172.21.15.42:0/3800375533 wait complete. 2024-08-30T09:07:26.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.457+0000 7f8c7cd8c740 20 client.26326 trim_cache size 0 max 16384 2024-08-30T09:07:26.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.474+0000 7f8c7cd8c740 1 Processor -- start 2024-08-30T09:07:26.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.474+0000 7f8c7cd8c740 1 -- start start 2024-08-30T09:07:26.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.474+0000 7f8c7cd8c740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e2beba39f0 0x55e2bef8db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:07:26.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.474+0000 7f8c7cd8c740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e2bebbc1b0 0x55e2bef92c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:07:26.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.474+0000 7f8c7cd8c740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e2bebe5070 0x55e2bef931c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:07:26.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.474+0000 7f8c6ffff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e2beba39f0 0x55e2bef8db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:07:26.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.474+0000 7f8c74db4640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e2bebe5070 0x55e2bef931c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:07:26.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.474+0000 7f8c6ffff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e2beba39f0 0x55e2bef8db20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:44322/0 (socket says 172.21.15.42:44322) 2024-08-30T09:07:26.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.474+0000 7f8c6ffff640 1 -- 172.21.15.42:0/2115058754 learned_addr learned my addr 172.21.15.42:0/2115058754 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:07:26.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.474+0000 7f8c7cd8c740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f8c701b96b0 con 0x55e2bebe5070 2024-08-30T09:07:26.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.475+0000 7f8c7cd8c740 1 -- 172.21.15.42:0/2115058754 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f8c701b6f20 con 0x55e2beba39f0 2024-08-30T09:07:26.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.475+0000 7f8c7cd8c740 1 -- 172.21.15.42:0/2115058754 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f8c701b4d70 con 0x55e2bebbc1b0 2024-08-30T09:07:26.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.475+0000 7f8c6f7fe640 1 --2- 172.21.15.42:0/2115058754 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e2bebbc1b0 0x55e2bef92c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:07:26.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.475+0000 7f8c6f7fe640 1 -- 172.21.15.42:0/2115058754 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e2beba39f0 msgr2=0x55e2bef8db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:07:26.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.475+0000 7f8c6f7fe640 1 --2- 172.21.15.42:0/2115058754 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e2beba39f0 0x55e2bef8db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:26.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.475+0000 7f8c6f7fe640 1 -- 172.21.15.42:0/2115058754 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e2bebe5070 msgr2=0x55e2bef931c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:07:26.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.475+0000 7f8c6f7fe640 1 --2- 172.21.15.42:0/2115058754 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e2bebe5070 0x55e2bef931c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:26.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.475+0000 7f8c6f7fe640 1 -- 172.21.15.42:0/2115058754 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e2bef8e060 con 0x55e2bebbc1b0 2024-08-30T09:07:26.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.475+0000 7f8c6f7fe640 1 --2- 172.21.15.42:0/2115058754 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e2bebbc1b0 0x55e2bef92c80 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f8c6400cfd0 tx=0x7f8c64000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4e5510cc2c3095ea server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:07:26.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.475+0000 7f8c55ffb640 1 -- 172.21.15.42:0/2115058754 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c640acba0 con 0x55e2bebbc1b0 2024-08-30T09:07:26.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.475+0000 7f8c55ffb640 1 -- 172.21.15.42:0/2115058754 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8c640b4b10 con 0x55e2bebbc1b0 2024-08-30T09:07:26.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.475+0000 7f8c55ffb640 1 -- 172.21.15.42:0/2115058754 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c640b0170 con 0x55e2bebbc1b0 2024-08-30T09:07:26.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.476+0000 7f8c7cd8c740 1 -- 172.21.15.42:0/2115058754 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e2bebbc1b0 msgr2=0x55e2bef92c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:07:26.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.476+0000 7f8c7cd8c740 1 --2- 172.21.15.42:0/2115058754 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e2bebbc1b0 0x55e2bef92c80 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f8c6400cfd0 tx=0x7f8c64000bc0 comp rx=0 tx=0).stop 2024-08-30T09:07:26.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.476+0000 7f8c7cd8c740 1 -- 172.21.15.42:0/2115058754 shutdown_connections 2024-08-30T09:07:26.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.476+0000 7f8c7cd8c740 1 --2- 172.21.15.42:0/2115058754 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e2bebbc1b0 0x55e2bef92c80 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:26.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.476+0000 7f8c7cd8c740 1 --2- 172.21.15.42:0/2115058754 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e2bebe5070 0x55e2bef931c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:26.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.476+0000 7f8c7cd8c740 1 --2- 172.21.15.42:0/2115058754 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e2beba39f0 0x55e2bef8db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:26.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.477+0000 7f8c7cd8c740 1 -- 172.21.15.42:0/2115058754 >> 172.21.15.42:0/2115058754 conn(0x55e2bebd9460 msgr2=0x55e2bef8d410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:07:26.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.477+0000 7f8c7cd8c740 1 -- 172.21.15.42:0/2115058754 shutdown_connections 2024-08-30T09:07:26.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.477+0000 7f8c7cd8c740 1 -- 172.21.15.42:0/2115058754 wait complete. 2024-08-30T09:07:26.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.477+0000 7f8c7cd8c740 1 -- start start 2024-08-30T09:07:26.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.477+0000 7f8c7cd8c740 1 Processor -- start 2024-08-30T09:07:26.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.478+0000 7f8c7cd8c740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e2beba39f0 0x55e2bef9a710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:07:26.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.478+0000 7f8c7cd8c740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e2bebbc1b0 0x55e2bef9acb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:07:26.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.478+0000 7f8c7cd8c740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e2bebe5070 0x55e2bf00b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:07:26.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.478+0000 7f8c7cd8c740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f8c701b53e0 con 0x55e2beba39f0 2024-08-30T09:07:26.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.478+0000 7f8c7cd8c740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f8c701b6850 con 0x55e2bebbc1b0 2024-08-30T09:07:26.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.478+0000 7f8c7cd8c740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f8c701b5ab0 con 0x55e2bebe5070 2024-08-30T09:07:26.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.478+0000 7f8c7cd8c740 10 client.0 _ll_register_callbacks cb 0x7f8c640a8330 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:07:26.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.478+0000 7f8c6ffff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e2beba39f0 0x55e2bef9a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:07:26.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.478+0000 7f8c6ffff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e2beba39f0 0x55e2bef9a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:50490/0 (socket says 172.21.15.42:50490) 2024-08-30T09:07:26.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.478+0000 7f8c6ffff640 1 -- 172.21.15.42:0/733357633 learned_addr learned my addr 172.21.15.42:0/733357633 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:07:26.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.478+0000 7f8c6f7fe640 1 --2- 172.21.15.42:0/733357633 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e2bebbc1b0 0x55e2bef9acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:07:26.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.478+0000 7f8c74db4640 1 --2- 172.21.15.42:0/733357633 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e2bebe5070 0x55e2bf00b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:07:26.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.478+0000 7f8c6f7fe640 1 -- 172.21.15.42:0/733357633 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e2bebe5070 msgr2=0x55e2bf00b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:07:26.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.478+0000 7f8c6f7fe640 1 --2- 172.21.15.42:0/733357633 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e2bebe5070 0x55e2bf00b7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:26.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.478+0000 7f8c6f7fe640 1 -- 172.21.15.42:0/733357633 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e2beba39f0 msgr2=0x55e2bef9a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:07:26.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.478+0000 7f8c6f7fe640 1 --2- 172.21.15.42:0/733357633 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e2beba39f0 0x55e2bef9a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:26.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.478+0000 7f8c6f7fe640 1 -- 172.21.15.42:0/733357633 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e2bf00be30 con 0x55e2bebbc1b0 2024-08-30T09:07:26.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.478+0000 7f8c74db4640 1 --2- 172.21.15.42:0/733357633 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e2bebe5070 0x55e2bf00b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-30T09:07:26.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.478+0000 7f8c6ffff640 1 --2- 172.21.15.42:0/733357633 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e2beba39f0 0x55e2bef9a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-30T09:07:26.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.478+0000 7f8c6f7fe640 1 --2- 172.21.15.42:0/733357633 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e2bebbc1b0 0x55e2bef9acb0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f8c640ac8a0 tx=0x7f8c640076c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e7713bf5586d735b server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:07:26.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.478+0000 7f8c4f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-30T09:07:26.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.479+0000 7f8c4f7fe640 1 -- 172.21.15.42:0/733357633 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c640b4fa0 con 0x55e2bebbc1b0 2024-08-30T09:07:26.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.479+0000 7f8c4f7fe640 1 -- 172.21.15.42:0/733357633 <== mon.2 v2:172.21.15.167:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f8c64018c50 con 0x55e2bebbc1b0 2024-08-30T09:07:26.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.479+0000 7f8c7cd8c740 1 -- 172.21.15.42:0/733357633 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55e2bef96490 con 0x55e2bebbc1b0 2024-08-30T09:07:26.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.479+0000 7f8c4f7fe640 1 -- 172.21.15.42:0/733357633 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f8c640acba0 con 0x55e2bebbc1b0 2024-08-30T09:07:26.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.479+0000 7f8c7cd8c740 10 client.26351 fetch_fsmap learned FSMap version 34 2024-08-30T09:07:26.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.479+0000 7f8c7cd8c740 1 -- 172.21.15.42:0/733357633 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55e2bf013170 con 0x55e2bebbc1b0 2024-08-30T09:07:26.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.479+0000 7f8c4f7fe640 1 -- 172.21.15.42:0/733357633 <== mon.2 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55e2bef96490 con 0x55e2bebbc1b0 2024-08-30T09:07:26.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.479+0000 7f8c4f7fe640 1 -- 172.21.15.42:0/733357633 <== mon.2 v2:172.21.15.167:3300/0 5 ==== fsmap(e 34) ==== 8366+0+0 (secure 0 0 0) 0x7f8c640b1680 con 0x55e2bebbc1b0 2024-08-30T09:07:26.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.479+0000 7f8c7cd8c740 10 client.26351 fetch_fsmap finished waiting for FSMap version 34 2024-08-30T09:07:26.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.479+0000 7f8c7cd8c740 10 client.26351 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-30T09:07:26.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.479+0000 7f8c7cd8c740 20 client.26351 populate_metadata read hostname 'smithi042' 2024-08-30T09:07:26.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.479+0000 7f8c7cd8c740 1 --2- 172.21.15.42:0/733357633 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55e2beb78da0 0x55e2bf0156b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:07:26.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.479+0000 7f8c7cd8c740 4 client.26351 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-30T09:07:26.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.480+0000 7f8c7cd8c740 1 -- 172.21.15.42:0/733357633 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e2bef035d0 con 0x55e2beb78da0 2024-08-30T09:07:26.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.480+0000 7f8c6ffff640 1 --2- 172.21.15.42:0/733357633 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55e2beb78da0 0x55e2bf0156b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:07:26.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.480+0000 7f8c4f7fe640 10 client.26351 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:07:26.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.480+0000 7f8c6ffff640 1 --2- 172.21.15.42:0/733357633 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55e2beb78da0 0x55e2bf0156b0 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8697c3e1111d17a4 server_cookie=6eb938f349b0cab5 in_seq=0 out_seq=0 2024-08-30T09:07:26.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.480+0000 7f8c4f7fe640 1 -- 172.21.15.42:0/733357633 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e2bef035d0 con 0x55e2beb78da0 2024-08-30T09:07:26.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.480+0000 7f8c4f7fe640 10 client.26351 handle_command_reply: tid=0 2024-08-30T09:07:26.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.480+0000 7f8c7cd8c740 1 client.26351 shutdown 2024-08-30T09:07:26.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.481+0000 7f8c7cd8c740 1 -- 172.21.15.42:0/733357633 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e2bebbc1b0 msgr2=0x55e2bef9acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:07:26.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.481+0000 7f8c7cd8c740 1 --2- 172.21.15.42:0/733357633 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e2bebbc1b0 0x55e2bef9acb0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f8c640ac8a0 tx=0x7f8c640076c0 comp rx=0 tx=0).stop 2024-08-30T09:07:26.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.481+0000 7f8c7cd8c740 1 -- 172.21.15.42:0/733357633 shutdown_connections 2024-08-30T09:07:26.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.481+0000 7f8c7cd8c740 1 --2- 172.21.15.42:0/733357633 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55e2beb78da0 0x55e2bf0156b0 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:26.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.481+0000 7f8c7cd8c740 1 --2- 172.21.15.42:0/733357633 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e2bebe5070 0x55e2bf00b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:26.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.481+0000 7f8c4f7fe640 0 client.26351 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:07:26.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.481+0000 7f8c7cd8c740 1 --2- 172.21.15.42:0/733357633 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e2bebbc1b0 0x55e2bef9acb0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:26.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.481+0000 7f8c7cd8c740 1 --2- 172.21.15.42:0/733357633 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e2beba39f0 0x55e2bef9a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:26.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.481+0000 7f8c7cd8c740 1 -- 172.21.15.42:0/733357633 >> 172.21.15.42:0/733357633 conn(0x55e2bebd9460 msgr2=0x55e2bef8b340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:07:26.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.481+0000 7f8c7cd8c740 1 -- 172.21.15.42:0/733357633 shutdown_connections 2024-08-30T09:07:26.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.481+0000 7f8c7cd8c740 1 -- 172.21.15.42:0/733357633 wait complete. 2024-08-30T09:07:26.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.481+0000 7f8c7cd8c740 20 client.26351 trim_cache size 0 max 16384 2024-08-30T09:07:26.489 INFO:teuthology.orchestra.run.smithi042.stdout:[] 2024-08-30T09:07:26.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.482+0000 7f8c76817640 1 -- 172.21.15.42:0/2645944807 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f8c580b4b60 msgr2=0x7f8c580b7070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:07:26.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.482+0000 7f8c76817640 1 --2- 172.21.15.42:0/2645944807 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f8c580b4b60 0x7f8c580b7070 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f8c68005fd0 tx=0x7f8c68010740 comp rx=0 tx=0).stop 2024-08-30T09:07:26.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.482+0000 7f8c76817640 1 -- 172.21.15.42:0/2645944807 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8c70104f50 msgr2=0x7f8c701a0e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:07:26.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.482+0000 7f8c76817640 1 --2- 172.21.15.42:0/2645944807 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8c70104f50 0x7f8c701a0e20 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f8c6400c8c0 tx=0x7f8c6400cd80 comp rx=0 tx=0).stop 2024-08-30T09:07:26.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.482+0000 7f8c76817640 1 -- 172.21.15.42:0/2645944807 shutdown_connections 2024-08-30T09:07:26.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.482+0000 7f8c76817640 1 --2- 172.21.15.42:0/2645944807 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f8c580b4b60 0x7f8c580b7070 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:26.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.482+0000 7f8c76817640 1 --2- 172.21.15.42:0/2645944807 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8c70104f50 0x7f8c701a0e20 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:26.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.482+0000 7f8c76817640 1 --2- 172.21.15.42:0/2645944807 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8c701058a0 0x7f8c701a60c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:26.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.482+0000 7f8c76817640 1 --2- 172.21.15.42:0/2645944807 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8c701028d0 0x7f8c701a0880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:26.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.482+0000 7f8c76817640 1 -- 172.21.15.42:0/2645944807 >> 172.21.15.42:0/2645944807 conn(0x7f8c70079740 msgr2=0x7f8c7006ba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:07:26.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.482+0000 7f8c76817640 1 -- 172.21.15.42:0/2645944807 shutdown_connections 2024-08-30T09:07:26.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:26.482+0000 7f8c76817640 1 -- 172.21.15.42:0/2645944807 wait complete. 2024-08-30T09:07:26.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:07:26 smithi042 ceph-mon[18652]: pgmap v573: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 6.5 MiB/s wr, 484 op/s 2024-08-30T09:07:26.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:07:26 smithi167 ceph-mon[22063]: pgmap v573: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 6.5 MiB/s wr, 484 op/s 2024-08-30T09:07:26.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:07:26 smithi190 ceph-mon[22001]: pgmap v573: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 6.5 MiB/s wr, 484 op/s 2024-08-30T09:07:27.917 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:27.914+0000 7f000da3f640 1 -- 172.21.15.42:0/2141855400 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0008106f70 msgr2=0x7f0008107470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:07:27.917 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:27.914+0000 7f000da3f640 1 --2- 172.21.15.42:0/2141855400 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0008106f70 0x7f0008107470 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f0000009c90 tx=0x7f000002a860 comp rx=0 tx=0).stop 2024-08-30T09:07:27.917 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:27.914+0000 7f000da3f640 1 -- 172.21.15.42:0/2141855400 shutdown_connections 2024-08-30T09:07:27.917 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:27.914+0000 7f000da3f640 1 --2- 172.21.15.42:0/2141855400 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f00081043b0 0x7f0008106830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:27.917 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:27.914+0000 7f000da3f640 1 --2- 172.21.15.42:0/2141855400 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0008106f70 0x7f0008107470 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:27.917 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:27.914+0000 7f000da3f640 1 --2- 172.21.15.42:0/2141855400 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f000806b4b0 0x7f0008103e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:27.918 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:27.915+0000 7f000da3f640 1 -- 172.21.15.42:0/2141855400 >> 172.21.15.42:0/2141855400 conn(0x7f00080fdc80 msgr2=0x7f0008100130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:07:27.918 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:27.915+0000 7f000da3f640 1 -- 172.21.15.42:0/2141855400 shutdown_connections 2024-08-30T09:07:27.918 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:27.915+0000 7f000da3f640 1 -- 172.21.15.42:0/2141855400 wait complete. 2024-08-30T09:07:27.918 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:27.915+0000 7f000da3f640 1 Processor -- start 2024-08-30T09:07:27.919 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:27.915+0000 7f000da3f640 1 -- start start 2024-08-30T09:07:27.919 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:27.916+0000 7f000da3f640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f000806b4b0 0x7f00081a0340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:07:27.919 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:27.916+0000 7f000da3f640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f00081043b0 0x7f00081a09b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:07:27.920 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:27.916+0000 7f000da3f640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0008106f70 0x7f00081a1560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:07:27.920 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:27.916+0000 7f000da3f640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f0008115720 con 0x7f0008106f70 2024-08-30T09:07:27.920 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:27.916+0000 7f000da3f640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f00081155a0 con 0x7f000806b4b0 2024-08-30T09:07:27.920 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:27.916+0000 7f000da3f640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f00081158a0 con 0x7f00081043b0 2024-08-30T09:07:27.920 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:27.916+0000 7f0006ffd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f000806b4b0 0x7f00081a0340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:07:27.920 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:27.916+0000 7f00077fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0008106f70 0x7f00081a1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:07:27.920 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:27.916+0000 7f0006ffd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f000806b4b0 0x7f00081a0340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:44346/0 (socket says 172.21.15.42:44346) 2024-08-30T09:07:27.920 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:27.916+0000 7f0006ffd640 1 -- 172.21.15.42:0/597197900 learned_addr learned my addr 172.21.15.42:0/597197900 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:07:27.920 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:27.916+0000 7efffffff640 1 --2- 172.21.15.42:0/597197900 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f00081043b0 0x7f00081a09b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:07:27.920 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:27.916+0000 7f0006ffd640 1 -- 172.21.15.42:0/597197900 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f00081043b0 msgr2=0x7f00081a09b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:07:27.920 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:27.916+0000 7f0006ffd640 1 --2- 172.21.15.42:0/597197900 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f00081043b0 0x7f00081a09b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:27.920 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:27.916+0000 7f0006ffd640 1 -- 172.21.15.42:0/597197900 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0008106f70 msgr2=0x7f00081a1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:07:27.921 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:27.916+0000 7f0006ffd640 1 --2- 172.21.15.42:0/597197900 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0008106f70 0x7f00081a1560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:27.921 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:27.916+0000 7f0006ffd640 1 -- 172.21.15.42:0/597197900 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00081a1d80 con 0x7f000806b4b0 2024-08-30T09:07:27.921 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:27.917+0000 7f0006ffd640 1 --2- 172.21.15.42:0/597197900 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f000806b4b0 0x7f00081a0340 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7efff800e9f0 tx=0x7efff800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:07:27.921 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:27.917+0000 7f000ca3d640 1 -- 172.21.15.42:0/597197900 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efff800cd80 con 0x7f000806b4b0 2024-08-30T09:07:27.921 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:27.917+0000 7f000ca3d640 1 -- 172.21.15.42:0/597197900 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efff80108e0 con 0x7f000806b4b0 2024-08-30T09:07:27.921 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:27.917+0000 7f000ca3d640 1 -- 172.21.15.42:0/597197900 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efff8018980 con 0x7f000806b4b0 2024-08-30T09:07:27.921 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:27.917+0000 7f000da3f640 1 -- 172.21.15.42:0/597197900 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00081a2070 con 0x7f000806b4b0 2024-08-30T09:07:27.921 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:27.917+0000 7f000da3f640 1 -- 172.21.15.42:0/597197900 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00081a8760 con 0x7f000806b4b0 2024-08-30T09:07:27.923 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:27.920+0000 7f0014a88740 1 Processor -- start 2024-08-30T09:07:27.924 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:27.922+0000 7f0014a88740 1 -- start start 2024-08-30T09:07:27.924 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:27.922+0000 7f000ca3d640 1 -- 172.21.15.42:0/597197900 <== mon.2 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7efff8010430 con 0x7f000806b4b0 2024-08-30T09:07:27.924 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:27.922+0000 7f0014a88740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d3634ff1b0 0x55d3638a5180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:07:27.924 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:27.922+0000 7f0014a88740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d3634d62f0 0x55d3638a57f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:07:27.924 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:27.922+0000 7f0014a88740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d3634de4d0 0x55d3638a6340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:07:27.924 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:27.922+0000 7f0014a88740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55d363818000 con 0x55d3634de4d0 2024-08-30T09:07:27.924 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:27.922+0000 7f0014a88740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55d363807590 con 0x55d3634d62f0 2024-08-30T09:07:27.925 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:27.922+0000 7f0014a88740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55d36380b540 con 0x55d3634ff1b0 2024-08-30T09:07:27.925 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:27.922+0000 7f00077fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d3634ff1b0 0x55d3638a5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:07:27.925 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:27.922+0000 7f00077fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d3634ff1b0 0x55d3638a5180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:39324/0 (socket says 172.21.15.42:39324) 2024-08-30T09:07:27.925 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:27.922+0000 7f00077fe640 1 -- 172.21.15.42:0/1160365834 learned_addr learned my addr 172.21.15.42:0/1160365834 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:07:27.925 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:27.922+0000 7f0006ffd640 1 --2- 172.21.15.42:0/1160365834 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d3634d62f0 0x55d3638a57f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:07:27.925 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:27.922+0000 7efffffff640 1 --2- 172.21.15.42:0/1160365834 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d3634de4d0 0x55d3638a6340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:07:27.926 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:27.923+0000 7f00077fe640 1 -- 172.21.15.42:0/1160365834 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d3634d62f0 msgr2=0x55d3638a57f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:07:27.926 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:27.923+0000 7f00077fe640 1 --2- 172.21.15.42:0/1160365834 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d3634d62f0 0x55d3638a57f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:27.926 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:27.923+0000 7f00077fe640 1 -- 172.21.15.42:0/1160365834 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d3634de4d0 msgr2=0x55d3638a6340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:07:27.926 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:27.923+0000 7f00077fe640 1 --2- 172.21.15.42:0/1160365834 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d3634de4d0 0x55d3638a6340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:27.927 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:27.923+0000 7f00077fe640 1 -- 172.21.15.42:0/1160365834 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d36356a790 con 0x55d3634ff1b0 2024-08-30T09:07:27.927 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:27.923+0000 7efffffff640 1 --2- 172.21.15.42:0/1160365834 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d3634de4d0 0x55d3638a6340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-30T09:07:27.927 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:27.923+0000 7f00077fe640 1 --2- 172.21.15.42:0/1160365834 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d3634ff1b0 0x55d3638a5180 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f0000009c60 tx=0x7f00000099b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e0a73233ff08d79b server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:07:27.927 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:27.923+0000 7efffdffb640 1 -- 172.21.15.42:0/1160365834 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0000031e30 con 0x55d3634ff1b0 2024-08-30T09:07:27.927 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:27.923+0000 7efffdffb640 1 -- 172.21.15.42:0/1160365834 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0000036030 con 0x55d3634ff1b0 2024-08-30T09:07:27.927 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:27.924+0000 7f000ca3d640 1 --2- 172.21.15.42:0/597197900 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7efff40b4bb0 0x7efff40b70c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:07:27.927 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:27.924+0000 7f000ca3d640 1 -- 172.21.15.42:0/597197900 <== mon.2 v2:172.21.15.167:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7efff80e5820 con 0x7f000806b4b0 2024-08-30T09:07:27.927 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:27.924+0000 7efffdffb640 1 -- 172.21.15.42:0/1160365834 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0000004030 con 0x55d3634ff1b0 2024-08-30T09:07:27.927 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:27.924+0000 7f0006ffd640 1 --2- 172.21.15.42:0/597197900 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7efff40b4bb0 0x7efff40b70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:07:27.927 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:27.925+0000 7f0014a88740 1 -- 172.21.15.42:0/1160365834 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d3634ff1b0 msgr2=0x55d3638a5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:07:27.927 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:27.925+0000 7f0014a88740 1 --2- 172.21.15.42:0/1160365834 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d3634ff1b0 0x55d3638a5180 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f0000009c60 tx=0x7f00000099b0 comp rx=0 tx=0).stop 2024-08-30T09:07:27.930 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:27.925+0000 7f0014a88740 1 -- 172.21.15.42:0/1160365834 shutdown_connections 2024-08-30T09:07:27.930 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:27.925+0000 7f0014a88740 1 --2- 172.21.15.42:0/1160365834 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d3634de4d0 0x55d3638a6340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:27.930 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:27.925+0000 7f0014a88740 1 --2- 172.21.15.42:0/1160365834 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d3634d62f0 0x55d3638a57f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:27.930 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:27.925+0000 7f0014a88740 1 --2- 172.21.15.42:0/1160365834 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d3634ff1b0 0x55d3638a5180 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:27.930 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:27.925+0000 7f0014a88740 1 -- 172.21.15.42:0/1160365834 >> 172.21.15.42:0/1160365834 conn(0x55d3634f3300 msgr2=0x55d3638a0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:07:27.930 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:27.925+0000 7f0006ffd640 1 --2- 172.21.15.42:0/597197900 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7efff40b4bb0 0x7efff40b70c0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7efff800e9c0 tx=0x7efff800cf20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:07:27.930 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:27.925+0000 7f0014a88740 1 -- 172.21.15.42:0/1160365834 shutdown_connections 2024-08-30T09:07:27.930 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:27.925+0000 7f0014a88740 1 -- 172.21.15.42:0/1160365834 wait complete. 2024-08-30T09:07:27.931 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:27.926+0000 7f0014a88740 1 -- start start 2024-08-30T09:07:27.931 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:27.926+0000 7f0014a88740 1 Processor -- start 2024-08-30T09:07:27.931 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:27.926+0000 7f0014a88740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d3634d62f0 0x55d3638aec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:07:27.931 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:27.926+0000 7f0014a88740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d3634de4d0 0x55d3638af2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:07:27.931 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:27.926+0000 7f0014a88740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d3634ff1b0 0x55d3638afe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:07:27.931 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:27.926+0000 7f0014a88740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55d363740ab0 con 0x55d3634ff1b0 2024-08-30T09:07:27.931 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:27.926+0000 7f0014a88740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55d36378e680 con 0x55d3634d62f0 2024-08-30T09:07:27.931 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:27.926+0000 7f0014a88740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55d3638a6ea0 con 0x55d3634de4d0 2024-08-30T09:07:27.931 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:27.926+0000 7f0014a88740 10 client.0 _ll_register_callbacks cb 0x55d363516a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:07:27.931 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:27.927+0000 7efffffff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d3634de4d0 0x55d3638af2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:07:27.931 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:27.927+0000 7efffffff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d3634de4d0 0x55d3638af2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:39598/0 (socket says 172.21.15.42:39598) 2024-08-30T09:07:27.931 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:27.927+0000 7efffffff640 1 -- 172.21.15.42:0/2759954434 learned_addr learned my addr 172.21.15.42:0/2759954434 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:07:27.931 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:27.927+0000 7f00077fe640 1 --2- 172.21.15.42:0/2759954434 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d3634d62f0 0x55d3638aec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:07:27.932 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:27.927+0000 7f00077fe640 1 --2- 172.21.15.42:0/2759954434 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d3634ff1b0 0x55d3638afe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:07:27.932 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:27.927+0000 7efffffff640 1 -- 172.21.15.42:0/2759954434 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d3634d62f0 msgr2=0x55d3638aec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:07:27.932 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:27.927+0000 7efffffff640 1 --2- 172.21.15.42:0/2759954434 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d3634d62f0 0x55d3638aec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:27.932 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:27.927+0000 7efffffff640 1 -- 172.21.15.42:0/2759954434 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d3634ff1b0 msgr2=0x55d3638afe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:07:27.932 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:27.927+0000 7efffffff640 1 --2- 172.21.15.42:0/2759954434 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d3634ff1b0 0x55d3638afe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:27.932 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:27.927+0000 7efffffff640 1 -- 172.21.15.42:0/2759954434 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d3635e3620 con 0x55d3634de4d0 2024-08-30T09:07:27.932 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:27.927+0000 7efffffff640 1 --2- 172.21.15.42:0/2759954434 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d3634de4d0 0x55d3638af2c0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7efff0012700 tx=0x7efff0012bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5dd0b0e668da7cf5 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:07:27.932 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:27.927+0000 7effe6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-08-30T09:07:27.932 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:27.927+0000 7effe6ffd640 1 -- 172.21.15.42:0/2759954434 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efff0004020 con 0x55d3634de4d0 2024-08-30T09:07:27.932 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:27.927+0000 7effe6ffd640 1 -- 172.21.15.42:0/2759954434 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7efff00041c0 con 0x55d3634de4d0 2024-08-30T09:07:27.932 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:27.927+0000 7f0014a88740 1 -- 172.21.15.42:0/2759954434 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0000036030 con 0x55d3634de4d0 2024-08-30T09:07:27.932 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:27.928+0000 7effe6ffd640 1 -- 172.21.15.42:0/2759954434 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7efff0025050 con 0x55d3634de4d0 2024-08-30T09:07:27.932 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:27.928+0000 7f0014a88740 10 client.26344 fetch_fsmap learned FSMap version 34 2024-08-30T09:07:27.933 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:27.928+0000 7f0014a88740 1 -- 172.21.15.42:0/2759954434 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55d3638b8e50 con 0x55d3634de4d0 2024-08-30T09:07:27.933 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:27.928+0000 7effe6ffd640 1 -- 172.21.15.42:0/2759954434 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0000036030 con 0x55d3634de4d0 2024-08-30T09:07:27.933 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:27.928+0000 7effe6ffd640 1 -- 172.21.15.42:0/2759954434 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 34) ==== 8366+0+0 (secure 0 0 0) 0x7efff0003920 con 0x55d3634de4d0 2024-08-30T09:07:27.933 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:27.928+0000 7f0014a88740 10 client.26344 fetch_fsmap finished waiting for FSMap version 34 2024-08-30T09:07:27.933 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:27.928+0000 7f0014a88740 10 client.26344 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-30T09:07:27.933 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:27.928+0000 7f0014a88740 20 client.26344 populate_metadata read hostname 'smithi042' 2024-08-30T09:07:27.933 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:27.928+0000 7f0014a88740 1 --2- 172.21.15.42:0/2759954434 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55d3634bdd80 0x55d3638bc600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:07:27.933 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:27.928+0000 7f0014a88740 4 client.26344 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-30T09:07:27.933 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:27.928+0000 7f0014a88740 1 -- 172.21.15.42:0/2759954434 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d363819f50 con 0x55d3634bdd80 2024-08-30T09:07:27.933 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:27.928+0000 7f00077fe640 1 --2- 172.21.15.42:0/2759954434 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55d3634bdd80 0x55d3638bc600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:07:27.934 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:27.929+0000 7effe6ffd640 10 client.26344 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:07:27.935 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:27.929+0000 7f00077fe640 1 --2- 172.21.15.42:0/2759954434 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55d3634bdd80 0x55d3638bc600 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=645278ce33aa04b2 server_cookie=5bda2ed1330821e0 in_seq=0 out_seq=0 2024-08-30T09:07:27.935 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:27.930+0000 7effe6ffd640 1 -- 172.21.15.42:0/2759954434 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d363819f50 con 0x55d3634bdd80 2024-08-30T09:07:27.935 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:27.930+0000 7effe6ffd640 10 client.26344 handle_command_reply: tid=0 2024-08-30T09:07:27.935 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:27.930+0000 7f0014a88740 1 client.26344 shutdown 2024-08-30T09:07:27.935 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:27.930+0000 7f0014a88740 1 -- 172.21.15.42:0/2759954434 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d3634de4d0 msgr2=0x55d3638af2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:07:27.935 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:27.930+0000 7f0014a88740 1 --2- 172.21.15.42:0/2759954434 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d3634de4d0 0x55d3638af2c0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7efff0012700 tx=0x7efff0012bc0 comp rx=0 tx=0).stop 2024-08-30T09:07:27.935 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:27.930+0000 7f0014a88740 1 -- 172.21.15.42:0/2759954434 shutdown_connections 2024-08-30T09:07:27.935 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:27.930+0000 7f0014a88740 1 --2- 172.21.15.42:0/2759954434 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55d3634bdd80 0x55d3638bc600 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:27.935 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:27.930+0000 7f0014a88740 1 --2- 172.21.15.42:0/2759954434 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d3634de4d0 0x55d3638af2c0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:27.935 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:27.930+0000 7f0014a88740 1 --2- 172.21.15.42:0/2759954434 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d3634ff1b0 0x55d3638afe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:27.935 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:27.930+0000 7f0014a88740 1 --2- 172.21.15.42:0/2759954434 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d3634d62f0 0x55d3638aec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:27.935 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:27.930+0000 7effe6ffd640 0 client.26344 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:07:27.935 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:27.930+0000 7f0014a88740 1 -- 172.21.15.42:0/2759954434 >> 172.21.15.42:0/2759954434 conn(0x55d3634f3300 msgr2=0x55d36389f170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:07:27.935 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:27.930+0000 7f0014a88740 1 -- 172.21.15.42:0/2759954434 shutdown_connections 2024-08-30T09:07:27.936 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:27.930+0000 7f0014a88740 1 -- 172.21.15.42:0/2759954434 wait complete. 2024-08-30T09:07:27.936 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:27.931+0000 7f0014a88740 20 client.26344 trim_cache size 0 max 16384 2024-08-30T09:07:27.963 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:27.957+0000 7f0014a88740 1 Processor -- start 2024-08-30T09:07:27.963 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:27.957+0000 7f0014a88740 1 -- start start 2024-08-30T09:07:27.963 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:27.957+0000 7f0014a88740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d3634d62f0 0x55d3638a4e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:07:27.964 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:27.957+0000 7f0014a88740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d3634de4d0 0x55d3638aec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:07:27.964 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:27.957+0000 7f0014a88740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d3634ff1b0 0x55d3638af140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:07:27.964 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:27.958+0000 7f00077fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d3634d62f0 0x55d3638a4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:07:27.964 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:27.958+0000 7f00077fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d3634d62f0 0x55d3638a4e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:39580/0 (socket says 172.21.15.42:39580) 2024-08-30T09:07:27.964 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:27.958+0000 7f00077fe640 1 -- 172.21.15.42:0/2966380225 learned_addr learned my addr 172.21.15.42:0/2966380225 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:07:27.964 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:27.958+0000 7f0014a88740 1 -- 172.21.15.42:0/2966380225 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f00081b5400 con 0x55d3634ff1b0 2024-08-30T09:07:27.964 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:27.958+0000 7f0014a88740 1 -- 172.21.15.42:0/2966380225 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f00081b2c70 con 0x55d3634d62f0 2024-08-30T09:07:27.964 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:27.958+0000 7f0014a88740 1 -- 172.21.15.42:0/2966380225 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f00081b0ac0 con 0x55d3634de4d0 2024-08-30T09:07:27.964 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:27.958+0000 7f00077fe640 1 --2- 172.21.15.42:0/2966380225 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d3634ff1b0 0x55d3638af140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:07:27.964 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:27.958+0000 7f00077fe640 1 -- 172.21.15.42:0/2966380225 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d3634de4d0 msgr2=0x55d3638aec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:07:27.964 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:27.958+0000 7f00077fe640 1 --2- 172.21.15.42:0/2966380225 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d3634de4d0 0x55d3638aec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:27.964 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:27.958+0000 7f00077fe640 1 -- 172.21.15.42:0/2966380225 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d3634ff1b0 msgr2=0x55d3638af140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:07:27.965 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:27.958+0000 7f00077fe640 1 --2- 172.21.15.42:0/2966380225 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d3634ff1b0 0x55d3638af140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:27.965 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:27.959+0000 7f00077fe640 1 -- 172.21.15.42:0/2966380225 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d3638acd00 con 0x55d3634d62f0 2024-08-30T09:07:27.965 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:27.959+0000 7f00077fe640 1 --2- 172.21.15.42:0/2966380225 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d3634d62f0 0x55d3638a4e40 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f0000009c90 tx=0x7f000002a860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b997cfcc553a822d server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:07:27.974 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:27.968+0000 7efffd7fa640 1 -- 172.21.15.42:0/2966380225 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0000031660 con 0x55d3634d62f0 2024-08-30T09:07:27.974 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:27.968+0000 7efffd7fa640 1 -- 172.21.15.42:0/2966380225 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0000036070 con 0x55d3634d62f0 2024-08-30T09:07:27.974 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:27.968+0000 7efffd7fa640 1 -- 172.21.15.42:0/2966380225 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0000030cf0 con 0x55d3634d62f0 2024-08-30T09:07:27.978 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:27.972+0000 7f0014a88740 1 -- 172.21.15.42:0/2966380225 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d3634d62f0 msgr2=0x55d3638a4e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:07:27.978 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:27.972+0000 7f0014a88740 1 --2- 172.21.15.42:0/2966380225 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d3634d62f0 0x55d3638a4e40 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f0000009c90 tx=0x7f000002a860 comp rx=0 tx=0).stop 2024-08-30T09:07:27.980 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:27.977+0000 7f0014a88740 1 -- 172.21.15.42:0/2966380225 shutdown_connections 2024-08-30T09:07:27.980 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:27.977+0000 7f0014a88740 1 --2- 172.21.15.42:0/2966380225 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d3634de4d0 0x55d3638aec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:27.980 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:27.977+0000 7f0014a88740 1 --2- 172.21.15.42:0/2966380225 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d3634ff1b0 0x55d3638af140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:27.981 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:27.977+0000 7f0014a88740 1 --2- 172.21.15.42:0/2966380225 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d3634d62f0 0x55d3638a4e40 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:27.981 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:27.977+0000 7f0014a88740 1 -- 172.21.15.42:0/2966380225 >> 172.21.15.42:0/2966380225 conn(0x55d3634f3300 msgr2=0x55d3638a9bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:07:27.987 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:27.981+0000 7f0014a88740 1 -- 172.21.15.42:0/2966380225 shutdown_connections 2024-08-30T09:07:27.988 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:27.981+0000 7f0014a88740 1 -- 172.21.15.42:0/2966380225 wait complete. 2024-08-30T09:07:27.988 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:27.981+0000 7f0014a88740 1 -- start start 2024-08-30T09:07:27.988 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:27.981+0000 7f0014a88740 1 Processor -- start 2024-08-30T09:07:27.988 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:27.982+0000 7f0014a88740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d3634d62f0 0x55d3638b68b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:07:27.988 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:27.982+0000 7f0014a88740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d3634de4d0 0x55d3638b5570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:07:27.988 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:27.982+0000 7f0014a88740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d3634ff1b0 0x55d3638b5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:07:27.988 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:27.982+0000 7f0014a88740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f00081b1130 con 0x55d3634de4d0 2024-08-30T09:07:27.988 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:27.982+0000 7f0014a88740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f00081b25a0 con 0x55d3634d62f0 2024-08-30T09:07:27.988 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:27.982+0000 7f0014a88740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f00081b1800 con 0x55d3634ff1b0 2024-08-30T09:07:27.988 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:27.982+0000 7f0014a88740 10 client.0 _ll_register_callbacks cb 0x7efff0009860 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:07:27.989 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:27.984+0000 7f00077fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d3634d62f0 0x55d3638b68b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:07:27.989 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:27.984+0000 7f00077fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d3634d62f0 0x55d3638b68b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:39596/0 (socket says 172.21.15.42:39596) 2024-08-30T09:07:27.989 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:27.984+0000 7f00077fe640 1 -- 172.21.15.42:0/853161306 learned_addr learned my addr 172.21.15.42:0/853161306 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:07:27.989 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:27.986+0000 7efffffff640 1 --2- 172.21.15.42:0/853161306 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d3634de4d0 0x55d3638b5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:07:27.989 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:27.986+0000 7efffffff640 1 -- 172.21.15.42:0/853161306 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d3634ff1b0 msgr2=0x55d3638b5ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:07:27.989 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:27.986+0000 7efffffff640 1 --2- 172.21.15.42:0/853161306 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d3634ff1b0 0x55d3638b5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:27.989 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:27.986+0000 7efffffff640 1 -- 172.21.15.42:0/853161306 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d3634d62f0 msgr2=0x55d3638b68b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:07:27.989 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:27.986+0000 7efffffff640 1 --2- 172.21.15.42:0/853161306 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d3634d62f0 0x55d3638b68b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:27.989 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:27.986+0000 7efffffff640 1 -- 172.21.15.42:0/853161306 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d3638b60f0 con 0x55d3634de4d0 2024-08-30T09:07:27.992 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:27.987+0000 7efffffff640 1 --2- 172.21.15.42:0/853161306 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d3634de4d0 0x55d3638b5570 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7efff0003d40 tx=0x7efff000f8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9addf74059daec5 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:07:27.993 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:27.987+0000 7effe6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-08-30T09:07:27.993 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:27.990+0000 7effe6ffd640 1 -- 172.21.15.42:0/853161306 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efff0022040 con 0x55d3634de4d0 2024-08-30T09:07:27.993 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:27.990+0000 7effe6ffd640 1 -- 172.21.15.42:0/853161306 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7efff000fbb0 con 0x55d3634de4d0 2024-08-30T09:07:27.993 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:27.990+0000 7f0014a88740 1 -- 172.21.15.42:0/853161306 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0000036070 con 0x55d3634de4d0 2024-08-30T09:07:28.000 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:27.995+0000 7effe6ffd640 1 -- 172.21.15.42:0/853161306 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7efff000fdd0 con 0x55d3634de4d0 2024-08-30T09:07:28.001 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:27.996+0000 7f0014a88740 10 client.16605 fetch_fsmap learned FSMap version 34 2024-08-30T09:07:28.001 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:27.996+0000 7f0014a88740 1 -- 172.21.15.42:0/853161306 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55d3638b4f30 con 0x55d3634de4d0 2024-08-30T09:07:28.006 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:28.005+0000 7effe6ffd640 1 -- 172.21.15.42:0/853161306 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efff0003aa0 con 0x55d3634de4d0 2024-08-30T09:07:28.007 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:28.005+0000 7effe6ffd640 1 -- 172.21.15.42:0/853161306 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 34) ==== 8366+0+0 (secure 0 0 0) 0x7efff001c730 con 0x55d3634de4d0 2024-08-30T09:07:28.008 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:28.006+0000 7f0014a88740 10 client.16605 fetch_fsmap finished waiting for FSMap version 34 2024-08-30T09:07:28.008 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:28.006+0000 7f0014a88740 10 client.16605 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-30T09:07:28.008 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:28.006+0000 7f0014a88740 20 client.16605 populate_metadata read hostname 'smithi042' 2024-08-30T09:07:28.008 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:28.006+0000 7f0014a88740 1 --2- 172.21.15.42:0/853161306 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55d3634931c0 0x55d36395c810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:07:28.008 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:28.006+0000 7f0014a88740 4 client.16605 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-30T09:07:28.008 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:28.006+0000 7f0014a88740 1 -- 172.21.15.42:0/853161306 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f0000036070 con 0x55d3634931c0 2024-08-30T09:07:28.017 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:28.015+0000 7f00077fe640 1 --2- 172.21.15.42:0/853161306 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55d3634931c0 0x55d36395c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:07:28.029 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:28.026+0000 7f00077fe640 1 --2- 172.21.15.42:0/853161306 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55d3634931c0 0x55d36395c810 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=506ecff6f474db80 server_cookie=41ffbc5c7150d0e3 in_seq=0 out_seq=0 2024-08-30T09:07:28.029 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:28.026+0000 7effe6ffd640 10 client.16605 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:07:28.113 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:28.111+0000 7effe6ffd640 1 -- 172.21.15.42:0/853161306 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f0000036070 con 0x55d3634931c0 2024-08-30T09:07:28.113 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:28.111+0000 7effe6ffd640 10 client.16605 handle_command_reply: tid=0 2024-08-30T09:07:28.113 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:28.111+0000 7f0014a88740 1 client.16605 shutdown 2024-08-30T09:07:28.115 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:28.112+0000 7f0014a88740 1 -- 172.21.15.42:0/853161306 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d3634de4d0 msgr2=0x55d3638b5570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:07:28.115 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:28.112+0000 7f0014a88740 1 --2- 172.21.15.42:0/853161306 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d3634de4d0 0x55d3638b5570 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7efff0003d40 tx=0x7efff000f8d0 comp rx=0 tx=0).stop 2024-08-30T09:07:28.115 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:28.112+0000 7f0014a88740 1 -- 172.21.15.42:0/853161306 shutdown_connections 2024-08-30T09:07:28.115 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:28.112+0000 7f0014a88740 1 --2- 172.21.15.42:0/853161306 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55d3634931c0 0x55d36395c810 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:28.115 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:28.112+0000 7f0014a88740 1 --2- 172.21.15.42:0/853161306 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d3634ff1b0 0x55d3638b5ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:28.115 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:28.112+0000 7f0014a88740 1 --2- 172.21.15.42:0/853161306 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d3634de4d0 0x55d3638b5570 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:28.115 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:28.112+0000 7f0014a88740 1 --2- 172.21.15.42:0/853161306 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d3634d62f0 0x55d3638b68b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:28.115 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:28.112+0000 7f0014a88740 1 -- 172.21.15.42:0/853161306 >> 172.21.15.42:0/853161306 conn(0x55d3634f3300 msgr2=0x55d36389f170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:07:28.115 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:28.112+0000 7effe6ffd640 0 client.16605 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:07:28.116 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:28.113+0000 7f0014a88740 1 -- 172.21.15.42:0/853161306 shutdown_connections 2024-08-30T09:07:28.116 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:28.113+0000 7f0014a88740 1 -- 172.21.15.42:0/853161306 wait complete. 2024-08-30T09:07:28.116 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:28.113+0000 7f0014a88740 20 client.16605 trim_cache size 0 max 16384 2024-08-30T09:07:28.116 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-08-30T09:07:28.116 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2024-08-30T09:07:28.116 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-08-30T09:07:28.116 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-08-30T09:07:28.117 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:28.114+0000 7f000da3f640 1 -- 172.21.15.42:0/597197900 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7efff40b4bb0 msgr2=0x7efff40b70c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:07:28.117 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:28.114+0000 7f000da3f640 1 --2- 172.21.15.42:0/597197900 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7efff40b4bb0 0x7efff40b70c0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7efff800e9c0 tx=0x7efff800cf20 comp rx=0 tx=0).stop 2024-08-30T09:07:28.117 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:28.114+0000 7f000da3f640 1 -- 172.21.15.42:0/597197900 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f000806b4b0 msgr2=0x7f00081a0340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:07:28.117 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:28.114+0000 7f000da3f640 1 --2- 172.21.15.42:0/597197900 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f000806b4b0 0x7f00081a0340 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7efff800e9f0 tx=0x7efff800eeb0 comp rx=0 tx=0).stop 2024-08-30T09:07:28.117 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:28.114+0000 7f000da3f640 1 -- 172.21.15.42:0/597197900 shutdown_connections 2024-08-30T09:07:28.117 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:28.114+0000 7f000da3f640 1 --2- 172.21.15.42:0/597197900 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7efff40b4bb0 0x7efff40b70c0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:28.117 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:28.114+0000 7f000da3f640 1 --2- 172.21.15.42:0/597197900 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f00081043b0 0x7f00081a09b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:28.117 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:28.114+0000 7f000da3f640 1 --2- 172.21.15.42:0/597197900 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0008106f70 0x7f00081a1560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:28.118 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:28.114+0000 7f000da3f640 1 --2- 172.21.15.42:0/597197900 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f000806b4b0 0x7f00081a0340 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:28.118 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:28.114+0000 7f000da3f640 1 -- 172.21.15.42:0/597197900 >> 172.21.15.42:0/597197900 conn(0x7f00080fdc80 msgr2=0x7f00081052a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:07:28.118 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:28.114+0000 7f000da3f640 1 -- 172.21.15.42:0/597197900 shutdown_connections 2024-08-30T09:07:28.118 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:28.114+0000 7f000da3f640 1 -- 172.21.15.42:0/597197900 wait complete. 2024-08-30T09:07:28.451 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-08-30T09:07:28.451 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-08-30T09:07:28.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:07:28 smithi167 ceph-mon[22063]: pgmap v574: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 6.7 MiB/s wr, 509 op/s 2024-08-30T09:07:28.884 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:07:28 smithi042 ceph-mon[18652]: pgmap v574: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 6.7 MiB/s wr, 509 op/s 2024-08-30T09:07:28.910 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T09:07:28.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:07:28 smithi190 ceph-mon[22001]: pgmap v574: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 6.7 MiB/s wr, 509 op/s 2024-08-30T09:07:29.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.600+0000 7f5c44c87640 1 -- 172.21.15.42:0/3960822516 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5c40102420 msgr2=0x7f5c40105f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:07:29.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.600+0000 7f5c44c87640 1 --2- 172.21.15.42:0/3960822516 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5c40102420 0x7f5c40105f80 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f5c38009be0 tx=0x7f5c3802a710 comp rx=0 tx=0).stop 2024-08-30T09:07:29.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.601+0000 7f5c44c87640 1 -- 172.21.15.42:0/3960822516 shutdown_connections 2024-08-30T09:07:29.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.601+0000 7f5c44c87640 1 --2- 172.21.15.42:0/3960822516 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5c40102420 0x7f5c40105f80 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:29.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.601+0000 7f5c44c87640 1 --2- 172.21.15.42:0/3960822516 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5c40101ad0 0x7f5c40101ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:29.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.601+0000 7f5c44c87640 1 --2- 172.21.15.42:0/3960822516 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5c401051e0 0x7f5c401055f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:29.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.601+0000 7f5c44c87640 1 -- 172.21.15.42:0/3960822516 >> 172.21.15.42:0/3960822516 conn(0x7f5c40079d30 msgr2=0x7f5c4007a140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:07:29.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.617+0000 7f5c44c87640 1 -- 172.21.15.42:0/3960822516 shutdown_connections 2024-08-30T09:07:29.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.617+0000 7f5c44c87640 1 -- 172.21.15.42:0/3960822516 wait complete. 2024-08-30T09:07:29.621 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.619+0000 7f5c44c87640 1 Processor -- start 2024-08-30T09:07:29.621 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.619+0000 7f5c44c87640 1 -- start start 2024-08-30T09:07:29.621 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.620+0000 7f5c44c87640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5c40101ad0 0x7f5c401a0890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:07:29.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.620+0000 7f5c44c87640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5c40102420 0x7f5c401a0e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:07:29.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.620+0000 7f5c44c87640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5c401051e0 0x7f5c401a60d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:07:29.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.620+0000 7f5c3dd9b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5c40102420 0x7f5c401a0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:07:29.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.620+0000 7f5c3dd9b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5c40102420 0x7f5c401a0e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:39638/0 (socket says 172.21.15.42:39638) 2024-08-30T09:07:29.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.621+0000 7f5c3dd9b640 1 -- 172.21.15.42:0/2823252888 learned_addr learned my addr 172.21.15.42:0/2823252888 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:07:29.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.621+0000 7f5c3ed9d640 1 --2- 172.21.15.42:0/2823252888 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5c401051e0 0x7f5c401a60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:07:29.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.621+0000 7f5c44c87640 1 -- 172.21.15.42:0/2823252888 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f5c40119ec0 con 0x7f5c40101ad0 2024-08-30T09:07:29.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.621+0000 7f5c44c87640 1 -- 172.21.15.42:0/2823252888 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f5c40119d40 con 0x7f5c401051e0 2024-08-30T09:07:29.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.622+0000 7f5c44c87640 1 -- 172.21.15.42:0/2823252888 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f5c4011a040 con 0x7f5c40102420 2024-08-30T09:07:29.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.623+0000 7f5c3e59c640 1 --2- 172.21.15.42:0/2823252888 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5c40101ad0 0x7f5c401a0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:07:29.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.626+0000 7f5c3dd9b640 1 -- 172.21.15.42:0/2823252888 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5c401051e0 msgr2=0x7f5c401a60d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:07:29.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.626+0000 7f5c3dd9b640 1 --2- 172.21.15.42:0/2823252888 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5c401051e0 0x7f5c401a60d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:29.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.626+0000 7f5c3dd9b640 1 -- 172.21.15.42:0/2823252888 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5c40101ad0 msgr2=0x7f5c401a0890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:07:29.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.626+0000 7f5c3dd9b640 1 --2- 172.21.15.42:0/2823252888 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5c40101ad0 0x7f5c401a0890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:29.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.626+0000 7f5c3dd9b640 1 -- 172.21.15.42:0/2823252888 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c401a6800 con 0x7f5c40102420 2024-08-30T09:07:29.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.626+0000 7f5c3dd9b640 1 --2- 172.21.15.42:0/2823252888 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5c40102420 0x7f5c401a0e30 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f5c3400e9f0 tx=0x7f5c3400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:07:29.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.626+0000 7f5c2f7fe640 1 -- 172.21.15.42:0/2823252888 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c3400ccc0 con 0x7f5c40102420 2024-08-30T09:07:29.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.627+0000 7f5c2f7fe640 1 -- 172.21.15.42:0/2823252888 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5c340108e0 con 0x7f5c40102420 2024-08-30T09:07:29.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.627+0000 7f5c44c87640 1 -- 172.21.15.42:0/2823252888 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c401a6af0 con 0x7f5c40102420 2024-08-30T09:07:29.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.627+0000 7f5c44c87640 1 -- 172.21.15.42:0/2823252888 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c401ad250 con 0x7f5c40102420 2024-08-30T09:07:29.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.631+0000 7f5c2f7fe640 1 -- 172.21.15.42:0/2823252888 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c34018880 con 0x7f5c40102420 2024-08-30T09:07:29.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.631+0000 7f5c4b1fc740 1 Processor -- start 2024-08-30T09:07:29.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.632+0000 7f5c4b1fc740 1 -- start start 2024-08-30T09:07:29.641 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.639+0000 7f5c2f7fe640 1 -- 172.21.15.42:0/2823252888 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7f5c34018a20 con 0x7f5c40102420 2024-08-30T09:07:29.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.641+0000 7f5c2f7fe640 1 --2- 172.21.15.42:0/2823252888 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f5c280b4c60 0x7f5c280b7170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:07:29.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.641+0000 7f5c2f7fe640 1 -- 172.21.15.42:0/2823252888 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f5c340e66a0 con 0x7f5c40102420 2024-08-30T09:07:29.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.641+0000 7f5c4b1fc740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55707cb6e350 0x55707cf14780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:07:29.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.641+0000 7f5c4b1fc740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55707cb45490 0x55707cf14cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:07:29.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.641+0000 7f5c4b1fc740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55707cb4d670 0x55707cf15200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:07:29.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.642+0000 7f5c4b1fc740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55707ce8a5f0 con 0x55707cb6e350 2024-08-30T09:07:29.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.642+0000 7f5c4b1fc740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55707ce79b80 con 0x55707cb4d670 2024-08-30T09:07:29.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.642+0000 7f5c4b1fc740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55707ce7db30 con 0x55707cb45490 2024-08-30T09:07:29.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.642+0000 7f5c3dd9b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55707cb45490 0x55707cf14cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:07:29.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.642+0000 7f5c3dd9b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55707cb45490 0x55707cf14cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:39652/0 (socket says 172.21.15.42:39652) 2024-08-30T09:07:29.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.642+0000 7f5c3dd9b640 1 -- 172.21.15.42:0/2129151843 learned_addr learned my addr 172.21.15.42:0/2129151843 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:07:29.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.642+0000 7f5c3ed9d640 1 --2- 172.21.15.42:0/2129151843 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55707cb4d670 0x55707cf15200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:07:29.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.642+0000 7f5c3ed9d640 1 --2- 172.21.15.42:0/2823252888 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f5c280b4c60 0x7f5c280b7170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:07:29.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.642+0000 7f5c3dd9b640 1 -- 172.21.15.42:0/2129151843 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55707cb4d670 msgr2=0x55707cf15200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:07:29.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.643+0000 7f5c3dd9b640 1 --2- 172.21.15.42:0/2129151843 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55707cb4d670 0x55707cf15200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:29.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.643+0000 7f5c3dd9b640 1 -- 172.21.15.42:0/2129151843 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55707cb6e350 msgr2=0x55707cf14780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:07:29.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.643+0000 7f5c3dd9b640 1 --2- 172.21.15.42:0/2129151843 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55707cb6e350 0x55707cf14780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:29.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.643+0000 7f5c3dd9b640 1 -- 172.21.15.42:0/2129151843 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55707cbd9930 con 0x55707cb45490 2024-08-30T09:07:29.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.643+0000 7f5c3dd9b640 1 --2- 172.21.15.42:0/2129151843 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55707cb45490 0x55707cf14cc0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f5c3400e9c0 tx=0x7f5c340acc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d4950398b3edbfc6 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:07:29.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.646+0000 7f5c2cff9640 1 -- 172.21.15.42:0/2129151843 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c340b1550 con 0x55707cb45490 2024-08-30T09:07:29.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.646+0000 7f5c3ed9d640 1 --2- 172.21.15.42:0/2823252888 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f5c280b4c60 0x7f5c280b7170 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f5c3802ad00 tx=0x7f5c38005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:07:29.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.646+0000 7f5c2cff9640 1 -- 172.21.15.42:0/2129151843 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5c340b6000 con 0x55707cb45490 2024-08-30T09:07:29.650 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.647+0000 7f5c4b1fc740 1 -- 172.21.15.42:0/2129151843 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55707cb45490 msgr2=0x55707cf14cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:07:29.650 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.647+0000 7f5c4b1fc740 1 --2- 172.21.15.42:0/2129151843 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55707cb45490 0x55707cf14cc0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f5c3400e9c0 tx=0x7f5c340acc90 comp rx=0 tx=0).stop 2024-08-30T09:07:29.650 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.647+0000 7f5c4b1fc740 1 -- 172.21.15.42:0/2129151843 shutdown_connections 2024-08-30T09:07:29.650 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.647+0000 7f5c4b1fc740 1 --2- 172.21.15.42:0/2129151843 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55707cb45490 0x55707cf14cc0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:29.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.647+0000 7f5c4b1fc740 1 --2- 172.21.15.42:0/2129151843 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55707cb4d670 0x55707cf15200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:29.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.648+0000 7f5c4b1fc740 1 --2- 172.21.15.42:0/2129151843 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55707cb6e350 0x55707cf14780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:29.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.648+0000 7f5c4b1fc740 1 -- 172.21.15.42:0/2129151843 >> 172.21.15.42:0/2129151843 conn(0x55707cb62740 msgr2=0x55707cf12670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:07:29.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.648+0000 7f5c4b1fc740 1 -- 172.21.15.42:0/2129151843 shutdown_connections 2024-08-30T09:07:29.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.649+0000 7f5c4b1fc740 1 -- 172.21.15.42:0/2129151843 wait complete. 2024-08-30T09:07:29.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.649+0000 7f5c4b1fc740 1 -- start start 2024-08-30T09:07:29.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.650+0000 7f5c4b1fc740 1 Processor -- start 2024-08-30T09:07:29.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.650+0000 7f5c4b1fc740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55707cb45490 0x55707cf1ff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:07:29.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.650+0000 7f5c4b1fc740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55707cb4d670 0x55707cf20470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:07:29.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.651+0000 7f5c4b1fc740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55707cb6e350 0x55707cf2c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:07:29.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.651+0000 7f5c3e59c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55707cb45490 0x55707cf1ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:07:29.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.651+0000 7f5c3e59c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55707cb45490 0x55707cf1ff30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:39648/0 (socket says 172.21.15.42:39648) 2024-08-30T09:07:29.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.651+0000 7f5c3e59c640 1 -- 172.21.15.42:0/4290852658 learned_addr learned my addr 172.21.15.42:0/4290852658 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:07:29.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.651+0000 7f5c4b1fc740 1 -- 172.21.15.42:0/4290852658 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55707ccdc890 con 0x55707cb6e350 2024-08-30T09:07:29.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.651+0000 7f5c4b1fc740 1 -- 172.21.15.42:0/4290852658 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55707ce00790 con 0x55707cb45490 2024-08-30T09:07:29.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.651+0000 7f5c4b1fc740 1 -- 172.21.15.42:0/4290852658 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55707cf23000 con 0x55707cb4d670 2024-08-30T09:07:29.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.651+0000 7f5c4b1fc740 10 client.0 _ll_register_callbacks cb 0x55707cb85ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:07:29.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.651+0000 7f5c3ed9d640 1 --2- 172.21.15.42:0/4290852658 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55707cb6e350 0x55707cf2c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:07:29.655 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.652+0000 7f5c3e59c640 1 -- 172.21.15.42:0/4290852658 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55707cb4d670 msgr2=0x55707cf20470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:07:29.655 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.652+0000 7f5c3e59c640 1 --2- 172.21.15.42:0/4290852658 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55707cb4d670 0x55707cf20470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:29.655 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.652+0000 7f5c3e59c640 1 -- 172.21.15.42:0/4290852658 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55707cb6e350 msgr2=0x55707cf2c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:07:29.655 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.652+0000 7f5c3e59c640 1 --2- 172.21.15.42:0/4290852658 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55707cb6e350 0x55707cf2c3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:29.656 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.652+0000 7f5c3e59c640 1 -- 172.21.15.42:0/4290852658 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55707ce37f30 con 0x55707cb45490 2024-08-30T09:07:29.656 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.652+0000 7f5c3ed9d640 1 --2- 172.21.15.42:0/4290852658 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55707cb6e350 0x55707cf2c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-30T09:07:29.656 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.652+0000 7f5c3e59c640 1 --2- 172.21.15.42:0/4290852658 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55707cb45490 0x55707cf1ff30 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f5c3000bed0 tx=0x7f5c300107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9e1294e93394f5af server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:07:29.657 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.653+0000 7f5c25ffb640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-30T09:07:29.657 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.653+0000 7f5c25ffb640 1 -- 172.21.15.42:0/4290852658 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c30011850 con 0x55707cb45490 2024-08-30T09:07:29.657 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.653+0000 7f5c4b1fc740 1 -- 172.21.15.42:0/4290852658 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5c340b6000 con 0x55707cb45490 2024-08-30T09:07:29.657 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.653+0000 7f5c25ffb640 1 -- 172.21.15.42:0/4290852658 <== mon.2 v2:172.21.15.167:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f5c300119f0 con 0x55707cb45490 2024-08-30T09:07:29.657 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.653+0000 7f5c25ffb640 1 -- 172.21.15.42:0/4290852658 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f5c30025050 con 0x55707cb45490 2024-08-30T09:07:29.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.659+0000 7f5c4b1fc740 10 client.26381 fetch_fsmap learned FSMap version 34 2024-08-30T09:07:29.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.659+0000 7f5c4b1fc740 1 -- 172.21.15.42:0/4290852658 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55707cf2eef0 con 0x55707cb45490 2024-08-30T09:07:29.666 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.662+0000 7f5c25ffb640 1 -- 172.21.15.42:0/4290852658 <== mon.2 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5c340b6000 con 0x55707cb45490 2024-08-30T09:07:29.666 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.662+0000 7f5c25ffb640 1 -- 172.21.15.42:0/4290852658 <== mon.2 v2:172.21.15.167:3300/0 5 ==== fsmap(e 34) ==== 8366+0+0 (secure 0 0 0) 0x7f5c3001a380 con 0x55707cb45490 2024-08-30T09:07:29.666 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.662+0000 7f5c4b1fc740 10 client.26381 fetch_fsmap finished waiting for FSMap version 34 2024-08-30T09:07:29.666 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.662+0000 7f5c4b1fc740 10 client.26381 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-30T09:07:29.666 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.662+0000 7f5c4b1fc740 20 client.26381 populate_metadata read hostname 'smithi042' 2024-08-30T09:07:29.666 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.662+0000 7f5c4b1fc740 1 --2- 172.21.15.42:0/4290852658 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55707cb2ccd0 0x55707cf32930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:07:29.667 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.662+0000 7f5c4b1fc740 4 client.26381 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-30T09:07:29.667 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.662+0000 7f5c4b1fc740 1 -- 172.21.15.42:0/4290852658 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55707ce8c540 con 0x55707cb2ccd0 2024-08-30T09:07:29.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.668+0000 7f5c3dd9b640 1 --2- 172.21.15.42:0/4290852658 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55707cb2ccd0 0x55707cf32930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:07:29.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.670+0000 7f5c3dd9b640 1 --2- 172.21.15.42:0/4290852658 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55707cb2ccd0 0x55707cf32930 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=875107bd146449fa server_cookie=dd6655acdd6fbe99 in_seq=0 out_seq=0 2024-08-30T09:07:29.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.671+0000 7f5c25ffb640 10 client.26381 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:07:29.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.673+0000 7f5c25ffb640 1 -- 172.21.15.42:0/4290852658 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55707ce8c540 con 0x55707cb2ccd0 2024-08-30T09:07:29.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.673+0000 7f5c25ffb640 10 client.26381 handle_command_reply: tid=0 2024-08-30T09:07:29.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.675+0000 7f5c4b1fc740 1 client.26381 shutdown 2024-08-30T09:07:29.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.676+0000 7f5c4b1fc740 1 -- 172.21.15.42:0/4290852658 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55707cb45490 msgr2=0x55707cf1ff30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:07:29.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.676+0000 7f5c4b1fc740 1 --2- 172.21.15.42:0/4290852658 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55707cb45490 0x55707cf1ff30 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f5c3000bed0 tx=0x7f5c300107a0 comp rx=0 tx=0).stop 2024-08-30T09:07:29.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.676+0000 7f5c4b1fc740 1 -- 172.21.15.42:0/4290852658 shutdown_connections 2024-08-30T09:07:29.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.676+0000 7f5c4b1fc740 1 --2- 172.21.15.42:0/4290852658 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55707cb2ccd0 0x55707cf32930 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:29.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.676+0000 7f5c4b1fc740 1 --2- 172.21.15.42:0/4290852658 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55707cb4d670 0x55707cf20470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:29.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.676+0000 7f5c4b1fc740 1 --2- 172.21.15.42:0/4290852658 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55707cb6e350 0x55707cf2c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:29.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.676+0000 7f5c4b1fc740 1 --2- 172.21.15.42:0/4290852658 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55707cb45490 0x55707cf1ff30 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:29.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.676+0000 7f5c4b1fc740 1 -- 172.21.15.42:0/4290852658 >> 172.21.15.42:0/4290852658 conn(0x55707cb62740 msgr2=0x55707cf17960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:07:29.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.676+0000 7f5c25ffb640 0 client.26381 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:07:29.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.677+0000 7f5c4b1fc740 1 -- 172.21.15.42:0/4290852658 shutdown_connections 2024-08-30T09:07:29.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.677+0000 7f5c4b1fc740 1 -- 172.21.15.42:0/4290852658 wait complete. 2024-08-30T09:07:29.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.678+0000 7f5c4b1fc740 20 client.26381 trim_cache size 0 max 16384 2024-08-30T09:07:29.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.710+0000 7f5c4b1fc740 1 Processor -- start 2024-08-30T09:07:29.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.710+0000 7f5c4b1fc740 1 -- start start 2024-08-30T09:07:29.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.710+0000 7f5c4b1fc740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55707cb45490 0x55707cf16170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:07:29.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.710+0000 7f5c4b1fc740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55707cb4d670 0x55707cf2c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:07:29.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.710+0000 7f5c4b1fc740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55707cb6e350 0x55707cf19fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:07:29.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.710+0000 7f5c4b1fc740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f5c20012610 con 0x55707cb4d670 2024-08-30T09:07:29.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.710+0000 7f5c4b1fc740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f5c2000fe80 con 0x55707cb6e350 2024-08-30T09:07:29.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.710+0000 7f5c4b1fc740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f5c2000dcd0 con 0x55707cb45490 2024-08-30T09:07:29.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.711+0000 7f5c3e59c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55707cb45490 0x55707cf16170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:07:29.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.711+0000 7f5c3e59c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55707cb45490 0x55707cf16170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:39676/0 (socket says 172.21.15.42:39676) 2024-08-30T09:07:29.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.711+0000 7f5c3e59c640 1 -- 172.21.15.42:0/1777786129 learned_addr learned my addr 172.21.15.42:0/1777786129 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:07:29.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.711+0000 7f5c3ed9d640 1 --2- 172.21.15.42:0/1777786129 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55707cb6e350 0x55707cf19fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:07:29.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.711+0000 7f5c3dd9b640 1 --2- 172.21.15.42:0/1777786129 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55707cb4d670 0x55707cf2c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:07:29.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.711+0000 7f5c3dd9b640 1 -- 172.21.15.42:0/1777786129 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55707cb45490 msgr2=0x55707cf16170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:07:29.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.711+0000 7f5c3dd9b640 1 --2- 172.21.15.42:0/1777786129 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55707cb45490 0x55707cf16170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:29.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.711+0000 7f5c3dd9b640 1 -- 172.21.15.42:0/1777786129 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55707cb6e350 msgr2=0x55707cf19fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:07:29.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.711+0000 7f5c3dd9b640 1 --2- 172.21.15.42:0/1777786129 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55707cb6e350 0x55707cf19fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:29.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.711+0000 7f5c3dd9b640 1 -- 172.21.15.42:0/1777786129 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55707cf2ed20 con 0x55707cb4d670 2024-08-30T09:07:29.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.712+0000 7f5c3dd9b640 1 --2- 172.21.15.42:0/1777786129 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55707cb4d670 0x55707cf2c2a0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x55707cf13400 tx=0x7f5c340acf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=120bf42b55672b3d server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:07:29.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.713+0000 7f5c27fff640 1 -- 172.21.15.42:0/1777786129 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c340e6370 con 0x55707cb4d670 2024-08-30T09:07:29.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.713+0000 7f5c27fff640 1 -- 172.21.15.42:0/1777786129 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5c340d3070 con 0x55707cb4d670 2024-08-30T09:07:29.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.713+0000 7f5c27fff640 1 -- 172.21.15.42:0/1777786129 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c34092e30 con 0x55707cb4d670 2024-08-30T09:07:29.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.714+0000 7f5c4b1fc740 1 -- 172.21.15.42:0/1777786129 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55707cb4d670 msgr2=0x55707cf2c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:07:29.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.714+0000 7f5c4b1fc740 1 --2- 172.21.15.42:0/1777786129 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55707cb4d670 0x55707cf2c2a0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x55707cf13400 tx=0x7f5c340acf10 comp rx=0 tx=0).stop 2024-08-30T09:07:29.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.714+0000 7f5c4b1fc740 1 -- 172.21.15.42:0/1777786129 shutdown_connections 2024-08-30T09:07:29.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.714+0000 7f5c4b1fc740 1 --2- 172.21.15.42:0/1777786129 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55707cb6e350 0x55707cf19fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:29.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.714+0000 7f5c4b1fc740 1 --2- 172.21.15.42:0/1777786129 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55707cb4d670 0x55707cf2c2a0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:29.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.714+0000 7f5c4b1fc740 1 --2- 172.21.15.42:0/1777786129 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55707cb45490 0x55707cf16170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:29.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.714+0000 7f5c4b1fc740 1 -- 172.21.15.42:0/1777786129 >> 172.21.15.42:0/1777786129 conn(0x55707cb62740 msgr2=0x55707cf19890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:07:29.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.714+0000 7f5c4b1fc740 1 -- 172.21.15.42:0/1777786129 shutdown_connections 2024-08-30T09:07:29.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.714+0000 7f5c4b1fc740 1 -- 172.21.15.42:0/1777786129 wait complete. 2024-08-30T09:07:29.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.715+0000 7f5c4b1fc740 1 -- start start 2024-08-30T09:07:29.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.715+0000 7f5c4b1fc740 1 Processor -- start 2024-08-30T09:07:29.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.716+0000 7f5c4b1fc740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55707cb45490 0x55707cf24980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:07:29.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.716+0000 7f5c4b1fc740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55707cb4d670 0x55707cf24ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:07:29.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.716+0000 7f5c3e59c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55707cb45490 0x55707cf24980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:07:29.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.716+0000 7f5c3e59c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55707cb45490 0x55707cf24980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:39664/0 (socket says 172.21.15.42:39664) 2024-08-30T09:07:29.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.716+0000 7f5c3dd9b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55707cb4d670 0x55707cf24ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:07:29.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.716+0000 7f5c3dd9b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55707cb4d670 0x55707cf24ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:39684/0 (socket says 172.21.15.42:39684) 2024-08-30T09:07:29.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.716+0000 7f5c3e59c640 1 -- 172.21.15.42:0/2664710857 learned_addr learned my addr 172.21.15.42:0/2664710857 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:07:29.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.716+0000 7f5c4b1fc740 1 --2- 172.21.15.42:0/2664710857 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55707cb6e350 0x55707cf9c330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:07:29.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.717+0000 7f5c4b1fc740 1 -- 172.21.15.42:0/2664710857 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f5c2000e340 con 0x55707cb6e350 2024-08-30T09:07:29.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.717+0000 7f5c4b1fc740 1 -- 172.21.15.42:0/2664710857 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f5c2000f7b0 con 0x55707cb45490 2024-08-30T09:07:29.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.717+0000 7f5c4b1fc740 1 -- 172.21.15.42:0/2664710857 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f5c2000ea10 con 0x55707cb4d670 2024-08-30T09:07:29.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.717+0000 7f5c3e59c640 1 -- 172.21.15.42:0/2664710857 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55707cb4d670 msgr2=0x55707cf24ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:07:29.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.717+0000 7f5c3e59c640 1 --2- 172.21.15.42:0/2664710857 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55707cb4d670 0x55707cf24ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:29.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.717+0000 7f5c3e59c640 1 -- 172.21.15.42:0/2664710857 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55707cb6e350 msgr2=0x55707cf9c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:07:29.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.717+0000 7f5c3e59c640 1 --2- 172.21.15.42:0/2664710857 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55707cb6e350 0x55707cf9c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:29.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.717+0000 7f5c3e59c640 1 -- 172.21.15.42:0/2664710857 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55707cf9c870 con 0x55707cb45490 2024-08-30T09:07:29.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.718+0000 7f5c4b1fc740 10 client.0 _ll_register_callbacks cb 0x7f5c3000b690 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:07:29.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.718+0000 7f5c3e59c640 1 --2- 172.21.15.42:0/2664710857 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55707cb45490 0x55707cf24980 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f5c30005d30 tx=0x7f5c30005990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=80156f54fe2cbad9 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:07:29.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.718+0000 7f5c4b1fc740 1 -- 172.21.15.42:0/2664710857 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5c340d3070 con 0x55707cb45490 2024-08-30T09:07:29.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.718+0000 7f5c25ffb640 10 client.26387 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-30T09:07:29.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.718+0000 7f5c25ffb640 1 -- 172.21.15.42:0/2664710857 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c3001a590 con 0x55707cb45490 2024-08-30T09:07:29.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.718+0000 7f5c25ffb640 1 -- 172.21.15.42:0/2664710857 <== mon.2 v2:172.21.15.167:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f5c3000ba20 con 0x55707cb45490 2024-08-30T09:07:29.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.719+0000 7f5c25ffb640 1 -- 172.21.15.42:0/2664710857 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f5c3003a050 con 0x55707cb45490 2024-08-30T09:07:29.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.719+0000 7f5c4b1fc740 10 client.26387 fetch_fsmap learned FSMap version 34 2024-08-30T09:07:29.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.719+0000 7f5c4b1fc740 1 -- 172.21.15.42:0/2664710857 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({fsmap=34}) -- 0x55707cf9d9a0 con 0x55707cb45490 2024-08-30T09:07:29.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.719+0000 7f5c25ffb640 1 -- 172.21.15.42:0/2664710857 <== mon.2 v2:172.21.15.167:3300/0 4 ==== fsmap(e 34) ==== 8366+0+0 (secure 0 0 0) 0x7f5c3001a970 con 0x55707cb45490 2024-08-30T09:07:29.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.719+0000 7f5c4b1fc740 10 client.26387 fetch_fsmap finished waiting for FSMap version 34 2024-08-30T09:07:29.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.719+0000 7f5c4b1fc740 10 client.26387 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-30T09:07:29.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.719+0000 7f5c4b1fc740 20 client.26387 populate_metadata read hostname 'smithi042' 2024-08-30T09:07:29.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.720+0000 7f5c4b1fc740 1 --2- 172.21.15.42:0/2664710857 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55707cb02080 0x55707cfa00a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:07:29.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.720+0000 7f5c4b1fc740 4 client.26387 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-08-30T09:07:29.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.720+0000 7f5c4b1fc740 1 -- 172.21.15.42:0/2664710857 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7f5c340d3070 con 0x55707cb02080 2024-08-30T09:07:29.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.720+0000 7f5c3dd9b640 1 --2- 172.21.15.42:0/2664710857 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55707cb02080 0x55707cfa00a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:07:29.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.721+0000 7f5c3dd9b640 1 --2- 172.21.15.42:0/2664710857 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55707cb02080 0x55707cfa00a0 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d43fcf2b01784da2 server_cookie=fa401e79a6251019 in_seq=0 out_seq=0 2024-08-30T09:07:29.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.721+0000 7f5c25ffb640 10 client.26387 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:07:29.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.721+0000 7f5c25ffb640 1 -- 172.21.15.42:0/2664710857 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7f5c340d3070 con 0x55707cb02080 2024-08-30T09:07:29.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.721+0000 7f5c25ffb640 10 client.26387 handle_command_reply: tid=0 2024-08-30T09:07:29.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.722+0000 7f5c4b1fc740 1 client.26387 shutdown 2024-08-30T09:07:29.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.722+0000 7f5c4b1fc740 1 -- 172.21.15.42:0/2664710857 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55707cb45490 msgr2=0x55707cf24980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:07:29.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.722+0000 7f5c4b1fc740 1 --2- 172.21.15.42:0/2664710857 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55707cb45490 0x55707cf24980 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f5c30005d30 tx=0x7f5c30005990 comp rx=0 tx=0).stop 2024-08-30T09:07:29.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.722+0000 7f5c4b1fc740 1 -- 172.21.15.42:0/2664710857 shutdown_connections 2024-08-30T09:07:29.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.722+0000 7f5c4b1fc740 1 --2- 172.21.15.42:0/2664710857 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55707cb02080 0x55707cfa00a0 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:29.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.722+0000 7f5c4b1fc740 1 --2- 172.21.15.42:0/2664710857 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55707cb4d670 0x55707cf24ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:29.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.722+0000 7f5c4b1fc740 1 --2- 172.21.15.42:0/2664710857 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55707cb6e350 0x55707cf9c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:29.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.722+0000 7f5c4b1fc740 1 --2- 172.21.15.42:0/2664710857 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55707cb45490 0x55707cf24980 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:29.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.722+0000 7f5c25ffb640 0 client.26387 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:07:29.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.723+0000 7f5c4b1fc740 1 -- 172.21.15.42:0/2664710857 >> 172.21.15.42:0/2664710857 conn(0x55707cb62740 msgr2=0x55707cf15f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:07:29.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.723+0000 7f5c4b1fc740 1 -- 172.21.15.42:0/2664710857 shutdown_connections 2024-08-30T09:07:29.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.723+0000 7f5c4b1fc740 1 -- 172.21.15.42:0/2664710857 wait complete. 2024-08-30T09:07:29.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.723+0000 7f5c4b1fc740 20 client.26387 trim_cache size 0 max 16384 2024-08-30T09:07:29.727 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-08-30T09:07:29.727 INFO:teuthology.orchestra.run.smithi042.stdout: "return_code": 0, 2024-08-30T09:07:29.727 INFO:teuthology.orchestra.run.smithi042.stdout: "scrub_tag": "598d8a75-f1cc-4388-80bc-a408bdb4eaac", 2024-08-30T09:07:29.727 INFO:teuthology.orchestra.run.smithi042.stdout: "mode": "asynchronous" 2024-08-30T09:07:29.727 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-08-30T09:07:29.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.724+0000 7f5c44c87640 1 -- 172.21.15.42:0/2823252888 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f5c280b4c60 msgr2=0x7f5c280b7170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:07:29.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.724+0000 7f5c44c87640 1 --2- 172.21.15.42:0/2823252888 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f5c280b4c60 0x7f5c280b7170 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f5c3802ad00 tx=0x7f5c38005fd0 comp rx=0 tx=0).stop 2024-08-30T09:07:29.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.724+0000 7f5c44c87640 1 -- 172.21.15.42:0/2823252888 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5c40102420 msgr2=0x7f5c401a0e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:07:29.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.724+0000 7f5c44c87640 1 --2- 172.21.15.42:0/2823252888 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5c40102420 0x7f5c401a0e30 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f5c3400e9f0 tx=0x7f5c3400eeb0 comp rx=0 tx=0).stop 2024-08-30T09:07:29.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.724+0000 7f5c44c87640 1 -- 172.21.15.42:0/2823252888 shutdown_connections 2024-08-30T09:07:29.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.724+0000 7f5c44c87640 1 --2- 172.21.15.42:0/2823252888 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f5c280b4c60 0x7f5c280b7170 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:29.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.724+0000 7f5c44c87640 1 --2- 172.21.15.42:0/2823252888 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5c40102420 0x7f5c401a0e30 secure :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f5c3400e9f0 tx=0x7f5c3400eeb0 comp rx=0 tx=0).stop 2024-08-30T09:07:29.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.724+0000 7f5c44c87640 1 --2- 172.21.15.42:0/2823252888 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5c401051e0 0x7f5c401a60d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:29.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.724+0000 7f5c44c87640 1 --2- 172.21.15.42:0/2823252888 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5c40101ad0 0x7f5c401a0890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:29.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.724+0000 7f5c44c87640 1 -- 172.21.15.42:0/2823252888 >> 172.21.15.42:0/2823252888 conn(0x7f5c40079d30 msgr2=0x7f5c40100a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:07:29.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.725+0000 7f5c44c87640 1 -- 172.21.15.42:0/2823252888 shutdown_connections 2024-08-30T09:07:29.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:29.725+0000 7f5c44c87640 1 -- 172.21.15.42:0/2823252888 wait complete. 2024-08-30T09:07:30.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:07:30 smithi167 ceph-mon[22063]: pgmap v575: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 8.6 MiB/s wr, 707 op/s 2024-08-30T09:07:30.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:07:30 smithi167 ceph-mon[22063]: scrub summary: active paths [/] 2024-08-30T09:07:30.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:07:30 smithi167 ceph-mon[22063]: scrub summary: active paths [/] 2024-08-30T09:07:30.828 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:07:30 smithi167 ceph-mon[22063]: scrub summary: idle+waiting paths [/] 2024-08-30T09:07:30.828 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:07:30 smithi167 ceph-mon[22063]: scrub summary: active paths [/] 2024-08-30T09:07:30.828 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:07:30 smithi167 ceph-mon[22063]: scrub summary: idle+waiting paths [/] 2024-08-30T09:07:30.828 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:07:30 smithi167 ceph-mon[22063]: scrub summary: active paths [/] 2024-08-30T09:07:30.828 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:07:30 smithi167 ceph-mon[22063]: scrub summary: idle+waiting paths [/] 2024-08-30T09:07:30.828 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:07:30 smithi167 ceph-mon[22063]: scrub summary: idle+waiting paths [/] 2024-08-30T09:07:30.828 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:07:30 smithi167 ceph-mon[22063]: scrub summary: active paths [/] 2024-08-30T09:07:30.828 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:07:30 smithi167 ceph-mon[22063]: scrub summary: active paths [/] 2024-08-30T09:07:30.828 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:07:30 smithi167 ceph-mon[22063]: scrub summary: idle+waiting paths [/] 2024-08-30T09:07:30.828 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:07:30 smithi167 ceph-mon[22063]: scrub summary: active paths [/] 2024-08-30T09:07:30.828 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:07:30 smithi167 ceph-mon[22063]: scrub summary: idle+waiting paths [/] 2024-08-30T09:07:30.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:07:30 smithi042 ceph-mon[18652]: pgmap v575: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 8.6 MiB/s wr, 707 op/s 2024-08-30T09:07:30.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:07:30 smithi042 ceph-mon[18652]: scrub summary: active paths [/] 2024-08-30T09:07:30.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:07:30 smithi042 ceph-mon[18652]: scrub summary: active paths [/] 2024-08-30T09:07:30.884 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:07:30 smithi042 ceph-mon[18652]: scrub summary: idle+waiting paths [/] 2024-08-30T09:07:30.884 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:07:30 smithi042 ceph-mon[18652]: scrub summary: active paths [/] 2024-08-30T09:07:30.884 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:07:30 smithi042 ceph-mon[18652]: scrub summary: idle+waiting paths [/] 2024-08-30T09:07:30.884 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:07:30 smithi042 ceph-mon[18652]: scrub summary: active paths [/] 2024-08-30T09:07:30.884 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:07:30 smithi042 ceph-mon[18652]: scrub summary: idle+waiting paths [/] 2024-08-30T09:07:30.884 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:07:30 smithi042 ceph-mon[18652]: scrub summary: idle+waiting paths [/] 2024-08-30T09:07:30.884 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:07:30 smithi042 ceph-mon[18652]: scrub summary: active paths [/] 2024-08-30T09:07:30.884 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:07:30 smithi042 ceph-mon[18652]: scrub summary: active paths [/] 2024-08-30T09:07:30.884 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:07:30 smithi042 ceph-mon[18652]: scrub summary: idle+waiting paths [/] 2024-08-30T09:07:30.884 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:07:30 smithi042 ceph-mon[18652]: scrub summary: active paths [/] 2024-08-30T09:07:30.884 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:07:30 smithi042 ceph-mon[18652]: scrub summary: idle+waiting paths [/] 2024-08-30T09:07:30.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:07:30 smithi190 ceph-mon[22001]: pgmap v575: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 8.6 MiB/s wr, 707 op/s 2024-08-30T09:07:30.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:07:30 smithi190 ceph-mon[22001]: scrub summary: active paths [/] 2024-08-30T09:07:30.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:07:30 smithi190 ceph-mon[22001]: scrub summary: active paths [/] 2024-08-30T09:07:30.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:07:30 smithi190 ceph-mon[22001]: scrub summary: idle+waiting paths [/] 2024-08-30T09:07:30.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:07:30 smithi190 ceph-mon[22001]: scrub summary: active paths [/] 2024-08-30T09:07:30.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:07:30 smithi190 ceph-mon[22001]: scrub summary: idle+waiting paths [/] 2024-08-30T09:07:30.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:07:30 smithi190 ceph-mon[22001]: scrub summary: active paths [/] 2024-08-30T09:07:30.937 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:07:30 smithi190 ceph-mon[22001]: scrub summary: idle+waiting paths [/] 2024-08-30T09:07:30.937 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:07:30 smithi190 ceph-mon[22001]: scrub summary: idle+waiting paths [/] 2024-08-30T09:07:30.937 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:07:30 smithi190 ceph-mon[22001]: scrub summary: active paths [/] 2024-08-30T09:07:30.937 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:07:30 smithi190 ceph-mon[22001]: scrub summary: active paths [/] 2024-08-30T09:07:30.937 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:07:30 smithi190 ceph-mon[22001]: scrub summary: idle+waiting paths [/] 2024-08-30T09:07:30.937 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:07:30 smithi190 ceph-mon[22001]: scrub summary: active paths [/] 2024-08-30T09:07:30.937 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:07:30 smithi190 ceph-mon[22001]: scrub summary: idle+waiting paths [/] 2024-08-30T09:07:30.953 INFO:teuthology.orchestra.run.smithi042.stderr:progress: 405.0 s, 504.4 tps, lat 199.674 ms stddev 235.028 2024-08-30T09:07:31.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:07:31 smithi167 ceph-mon[22063]: scrub queued for path: / 2024-08-30T09:07:31.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:07:31 smithi167 ceph-mon[22063]: scrub summary: idle+waiting paths [/] 2024-08-30T09:07:31.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:07:31 smithi167 ceph-mon[22063]: scrub summary: active paths [/] 2024-08-30T09:07:31.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:07:31 smithi167 ceph-mon[22063]: scrub summary: idle+waiting paths [/] 2024-08-30T09:07:31.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:07:31 smithi167 ceph-mon[22063]: scrub summary: idle+waiting paths [/] 2024-08-30T09:07:31.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:07:31 smithi042 ceph-mon[18652]: scrub queued for path: / 2024-08-30T09:07:31.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:07:31 smithi042 ceph-mon[18652]: scrub summary: idle+waiting paths [/] 2024-08-30T09:07:31.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:07:31 smithi042 ceph-mon[18652]: scrub summary: active paths [/] 2024-08-30T09:07:31.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:07:31 smithi042 ceph-mon[18652]: scrub summary: idle+waiting paths [/] 2024-08-30T09:07:31.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:07:31 smithi042 ceph-mon[18652]: scrub summary: idle+waiting paths [/] 2024-08-30T09:07:31.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:07:31 smithi190 ceph-mon[22001]: scrub queued for path: / 2024-08-30T09:07:31.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:07:31 smithi190 ceph-mon[22001]: scrub summary: idle+waiting paths [/] 2024-08-30T09:07:31.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:07:31 smithi190 ceph-mon[22001]: scrub summary: active paths [/] 2024-08-30T09:07:31.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:07:31 smithi190 ceph-mon[22001]: scrub summary: idle+waiting paths [/] 2024-08-30T09:07:31.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:07:31 smithi190 ceph-mon[22001]: scrub summary: idle+waiting paths [/] 2024-08-30T09:07:32.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:07:32 smithi167 ceph-mon[22063]: pgmap v576: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 5.8 MiB/s wr, 587 op/s 2024-08-30T09:07:32.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:07:32 smithi042 ceph-mon[18652]: pgmap v576: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 5.8 MiB/s wr, 587 op/s 2024-08-30T09:07:32.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:07:32 smithi190 ceph-mon[22001]: pgmap v576: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 5.8 MiB/s wr, 587 op/s 2024-08-30T09:07:34.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:07:34 smithi167 ceph-mon[22063]: pgmap v577: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 7.2 MiB/s wr, 769 op/s 2024-08-30T09:07:34.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:07:34 smithi042 ceph-mon[18652]: pgmap v577: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 7.2 MiB/s wr, 769 op/s 2024-08-30T09:07:34.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:07:34 smithi190 ceph-mon[22001]: pgmap v577: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 7.2 MiB/s wr, 769 op/s 2024-08-30T09:07:35.953 INFO:teuthology.orchestra.run.smithi042.stderr:progress: 410.0 s, 529.6 tps, lat 190.725 ms stddev 272.697 2024-08-30T09:07:36.005 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-30T09:07:36.538 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T09:07:36.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:07:36 smithi042 ceph-mon[18652]: pgmap v578: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 5.5 MiB/s wr, 632 op/s 2024-08-30T09:07:36.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:07:36 smithi190 ceph-mon[22001]: pgmap v578: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 5.5 MiB/s wr, 632 op/s 2024-08-30T09:07:37.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:07:36 smithi167 ceph-mon[22063]: pgmap v578: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 5.5 MiB/s wr, 632 op/s 2024-08-30T09:07:37.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.762+0000 7f69deb96640 1 -- 172.21.15.42:0/4118963114 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f69d8102880 msgr2=0x7f69d8102c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:07:37.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.762+0000 7f69deb96640 1 --2- 172.21.15.42:0/4118963114 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f69d8102880 0x7f69d8102c90 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f69cc009900 tx=0x7f69cc02a410 comp rx=0 tx=0).stop 2024-08-30T09:07:37.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.762+0000 7f69deb96640 1 -- 172.21.15.42:0/4118963114 shutdown_connections 2024-08-30T09:07:37.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.762+0000 7f69deb96640 1 --2- 172.21.15.42:0/4118963114 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f69d8108170 0x7f69d8108580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:37.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.762+0000 7f69deb96640 1 --2- 172.21.15.42:0/4118963114 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f69d8108ac0 0x7f69d810f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:37.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.762+0000 7f69deb96640 1 --2- 172.21.15.42:0/4118963114 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f69d8102880 0x7f69d8102c90 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:37.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.762+0000 7f69deb96640 1 -- 172.21.15.42:0/4118963114 >> 172.21.15.42:0/4118963114 conn(0x7f69d8079530 msgr2=0x7f69d8079940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:07:37.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.762+0000 7f69deb96640 1 -- 172.21.15.42:0/4118963114 shutdown_connections 2024-08-30T09:07:37.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.762+0000 7f69deb96640 1 -- 172.21.15.42:0/4118963114 wait complete. 2024-08-30T09:07:37.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.763+0000 7f69deb96640 1 Processor -- start 2024-08-30T09:07:37.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.763+0000 7f69deb96640 1 -- start start 2024-08-30T09:07:37.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.763+0000 7f69deb96640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f69d8102880 0x7f69d819a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:07:37.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.763+0000 7f69deb96640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f69d8108170 0x7f69d819a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:07:37.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.763+0000 7f69deb96640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f69d8108ac0 0x7f69d819fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:07:37.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.763+0000 7f69deb96640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f69d8113940 con 0x7f69d8108170 2024-08-30T09:07:37.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.763+0000 7f69deb96640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f69d81137c0 con 0x7f69d8108ac0 2024-08-30T09:07:37.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.763+0000 7f69deb96640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f69d8113ac0 con 0x7f69d8102880 2024-08-30T09:07:37.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.763+0000 7f69dc932640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f69d8102880 0x7f69d819a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:07:37.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.763+0000 7f69dc932640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f69d8102880 0x7f69d819a3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:39712/0 (socket says 172.21.15.42:39712) 2024-08-30T09:07:37.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.763+0000 7f69dc932640 1 -- 172.21.15.42:0/3984009491 learned_addr learned my addr 172.21.15.42:0/3984009491 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:07:37.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.763+0000 7f69dd133640 1 --2- 172.21.15.42:0/3984009491 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f69d8108ac0 0x7f69d819fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:07:37.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.763+0000 7f69d7fff640 1 --2- 172.21.15.42:0/3984009491 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f69d8108170 0x7f69d819a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:07:37.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.764+0000 7f69dc932640 1 -- 172.21.15.42:0/3984009491 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f69d8108ac0 msgr2=0x7f69d819fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:07:37.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.764+0000 7f69dc932640 1 --2- 172.21.15.42:0/3984009491 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f69d8108ac0 0x7f69d819fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:37.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.764+0000 7f69dc932640 1 -- 172.21.15.42:0/3984009491 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f69d8108170 msgr2=0x7f69d819a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:07:37.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.764+0000 7f69dc932640 1 --2- 172.21.15.42:0/3984009491 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f69d8108170 0x7f69d819a950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:37.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.764+0000 7f69dc932640 1 -- 172.21.15.42:0/3984009491 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69d81a0320 con 0x7f69d8102880 2024-08-30T09:07:37.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.764+0000 7f69dc932640 1 --2- 172.21.15.42:0/3984009491 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f69d8102880 0x7f69d819a3b0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f69cc006010 tx=0x7f69cc0042f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:07:37.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.764+0000 7f69d5ffb640 1 -- 172.21.15.42:0/3984009491 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69cc03f030 con 0x7f69d8102880 2024-08-30T09:07:37.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.764+0000 7f69deb96640 1 -- 172.21.15.42:0/3984009491 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69d81a05b0 con 0x7f69d8102880 2024-08-30T09:07:37.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.764+0000 7f69d5ffb640 1 -- 172.21.15.42:0/3984009491 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f69cc003800 con 0x7f69d8102880 2024-08-30T09:07:37.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.765+0000 7f69deb96640 1 -- 172.21.15.42:0/3984009491 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69d81a6d70 con 0x7f69d8102880 2024-08-30T09:07:37.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.765+0000 7f69d5ffb640 1 -- 172.21.15.42:0/3984009491 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69cc035070 con 0x7f69d8102880 2024-08-30T09:07:37.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.768+0000 7f69e510b740 1 Processor -- start 2024-08-30T09:07:37.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.768+0000 7f69d5ffb640 1 -- 172.21.15.42:0/3984009491 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7f69cc003cb0 con 0x7f69d8102880 2024-08-30T09:07:37.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.768+0000 7f69e510b740 1 -- start start 2024-08-30T09:07:37.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.769+0000 7f69e510b740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560e41967070 0x560e41d09d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:07:37.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.769+0000 7f69e510b740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560e4193e1b0 0x560e41d0a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:07:37.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.769+0000 7f69e510b740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560e41946390 0x560e41d1c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:07:37.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.769+0000 7f69e510b740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x560e41c83630 con 0x560e41967070 2024-08-30T09:07:37.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.769+0000 7f69e510b740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x560e41c72bc0 con 0x560e4193e1b0 2024-08-30T09:07:37.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.769+0000 7f69e510b740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x560e41c76b70 con 0x560e41946390 2024-08-30T09:07:37.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.769+0000 7f69dc932640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560e4193e1b0 0x560e41d0a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:07:37.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.769+0000 7f69d7fff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560e41946390 0x560e41d1c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:07:37.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.769+0000 7f69dc932640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560e4193e1b0 0x560e41d0a2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:39686/0 (socket says 172.21.15.42:39686) 2024-08-30T09:07:37.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.769+0000 7f69dc932640 1 -- 172.21.15.42:0/353276782 learned_addr learned my addr 172.21.15.42:0/353276782 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:07:37.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.769+0000 7f69dd133640 1 --2- 172.21.15.42:0/353276782 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560e41967070 0x560e41d09d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:07:37.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.769+0000 7f69dc932640 1 -- 172.21.15.42:0/353276782 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560e41946390 msgr2=0x560e41d1c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:07:37.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.769+0000 7f69dc932640 1 --2- 172.21.15.42:0/353276782 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560e41946390 0x560e41d1c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:37.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.769+0000 7f69dc932640 1 -- 172.21.15.42:0/353276782 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560e41967070 msgr2=0x560e41d09d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:07:37.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.769+0000 7f69dc932640 1 --2- 172.21.15.42:0/353276782 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560e41967070 0x560e41d09d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:37.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.769+0000 7f69dc932640 1 -- 172.21.15.42:0/353276782 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e419d2650 con 0x560e4193e1b0 2024-08-30T09:07:37.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.770+0000 7f69d5ffb640 1 --2- 172.21.15.42:0/3984009491 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f69c80b4bb0 0x7f69c80b70c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:07:37.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.770+0000 7f69d7fff640 1 --2- 172.21.15.42:0/353276782 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560e41946390 0x560e41d1c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-30T09:07:37.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.770+0000 7f69dc932640 1 --2- 172.21.15.42:0/353276782 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560e4193e1b0 0x560e41d0a2a0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f69cc109f00 tx=0x7f69cc10e450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fe1a814f83d8e643 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:07:37.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.770+0000 7f69d5ffb640 1 -- 172.21.15.42:0/3984009491 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f69cc105dd0 con 0x7f69d8102880 2024-08-30T09:07:37.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.770+0000 7f69dd133640 1 --2- 172.21.15.42:0/3984009491 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f69c80b4bb0 0x7f69c80b70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:07:37.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.770+0000 7f69c2ffd640 1 -- 172.21.15.42:0/353276782 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69cc049020 con 0x560e4193e1b0 2024-08-30T09:07:37.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.770+0000 7f69c2ffd640 1 -- 172.21.15.42:0/353276782 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f69cc0cbfd0 con 0x560e4193e1b0 2024-08-30T09:07:37.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.770+0000 7f69c2ffd640 1 -- 172.21.15.42:0/353276782 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69cc0c8380 con 0x560e4193e1b0 2024-08-30T09:07:37.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.771+0000 7f69e510b740 1 -- 172.21.15.42:0/353276782 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560e4193e1b0 msgr2=0x560e41d0a2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:07:37.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.771+0000 7f69dd133640 1 --2- 172.21.15.42:0/3984009491 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f69c80b4bb0 0x7f69c80b70c0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f69d000a400 tx=0x7f69d000d040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:07:37.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.771+0000 7f69e510b740 1 --2- 172.21.15.42:0/353276782 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560e4193e1b0 0x560e41d0a2a0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f69cc109f00 tx=0x7f69cc10e450 comp rx=0 tx=0).stop 2024-08-30T09:07:37.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.771+0000 7f69e510b740 1 -- 172.21.15.42:0/353276782 shutdown_connections 2024-08-30T09:07:37.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.771+0000 7f69e510b740 1 --2- 172.21.15.42:0/353276782 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560e41946390 0x560e41d1c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:37.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.771+0000 7f69e510b740 1 --2- 172.21.15.42:0/353276782 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560e4193e1b0 0x560e41d0a2a0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:37.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.771+0000 7f69e510b740 1 --2- 172.21.15.42:0/353276782 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560e41967070 0x560e41d09d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:37.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.771+0000 7f69e510b740 1 -- 172.21.15.42:0/353276782 >> 172.21.15.42:0/353276782 conn(0x560e4195b460 msgr2=0x560e41d0c240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:07:37.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.772+0000 7f69e510b740 1 -- 172.21.15.42:0/353276782 shutdown_connections 2024-08-30T09:07:37.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.772+0000 7f69e510b740 1 -- 172.21.15.42:0/353276782 wait complete. 2024-08-30T09:07:37.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.772+0000 7f69e510b740 1 -- start start 2024-08-30T09:07:37.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.772+0000 7f69e510b740 1 Processor -- start 2024-08-30T09:07:37.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.773+0000 7f69e510b740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560e4193e1b0 0x560e41d20750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:07:37.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.773+0000 7f69e510b740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560e41946390 0x560e41d20d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:07:37.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.773+0000 7f69e510b740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560e41967070 0x560e41d25d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:07:37.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.773+0000 7f69dc932640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560e4193e1b0 0x560e41d20750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:07:37.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.773+0000 7f69dc932640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560e4193e1b0 0x560e41d20750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:39732/0 (socket says 172.21.15.42:39732) 2024-08-30T09:07:37.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.773+0000 7f69dc932640 1 -- 172.21.15.42:0/1855995661 learned_addr learned my addr 172.21.15.42:0/1855995661 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:07:37.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.773+0000 7f69d7fff640 1 --2- 172.21.15.42:0/1855995661 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560e41946390 0x560e41d20d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:07:37.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.773+0000 7f69e510b740 1 -- 172.21.15.42:0/1855995661 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x560e41ad5700 con 0x560e41946390 2024-08-30T09:07:37.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.773+0000 7f69e510b740 1 -- 172.21.15.42:0/1855995661 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x560e41bf9700 con 0x560e41967070 2024-08-30T09:07:37.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.773+0000 7f69e510b740 1 -- 172.21.15.42:0/1855995661 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x560e41d1cf30 con 0x560e4193e1b0 2024-08-30T09:07:37.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.773+0000 7f69dd133640 1 --2- 172.21.15.42:0/1855995661 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560e41967070 0x560e41d25d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:07:37.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.773+0000 7f69e510b740 10 client.0 _ll_register_callbacks cb 0x560e41c28dc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:07:37.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.774+0000 7f69dc932640 1 -- 172.21.15.42:0/1855995661 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560e41967070 msgr2=0x560e41d25d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:07:37.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.774+0000 7f69dc932640 1 --2- 172.21.15.42:0/1855995661 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560e41967070 0x560e41d25d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:37.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.774+0000 7f69dc932640 1 -- 172.21.15.42:0/1855995661 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560e41946390 msgr2=0x560e41d20d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:07:37.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.774+0000 7f69dc932640 1 --2- 172.21.15.42:0/1855995661 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560e41946390 0x560e41d20d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:37.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.774+0000 7f69dc932640 1 -- 172.21.15.42:0/1855995661 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e41c328b0 con 0x560e4193e1b0 2024-08-30T09:07:37.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.774+0000 7f69dc932640 1 --2- 172.21.15.42:0/1855995661 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560e4193e1b0 0x560e41d20750 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f69cc0096c0 tx=0x7f69cc106050 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6278d4404b8bc7c0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:07:37.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.776+0000 7f69b7fff640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-08-30T09:07:37.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.776+0000 7f69b7fff640 1 -- 172.21.15.42:0/1855995661 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69cc049070 con 0x560e4193e1b0 2024-08-30T09:07:37.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.776+0000 7f69b7fff640 1 -- 172.21.15.42:0/1855995661 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f69cc1099b0 con 0x560e4193e1b0 2024-08-30T09:07:37.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.776+0000 7f69e510b740 1 -- 172.21.15.42:0/1855995661 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f69cc0cbfd0 con 0x560e4193e1b0 2024-08-30T09:07:37.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.776+0000 7f69b7fff640 1 -- 172.21.15.42:0/1855995661 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f69cc0cbc30 con 0x560e4193e1b0 2024-08-30T09:07:37.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.776+0000 7f69e510b740 10 client.26365 fetch_fsmap learned FSMap version 34 2024-08-30T09:07:37.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.776+0000 7f69e510b740 1 -- 172.21.15.42:0/1855995661 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x560e41d0fb50 con 0x560e4193e1b0 2024-08-30T09:07:37.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.776+0000 7f69b7fff640 1 -- 172.21.15.42:0/1855995661 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f69cc0f3120 con 0x560e4193e1b0 2024-08-30T09:07:37.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.777+0000 7f69b7fff640 1 -- 172.21.15.42:0/1855995661 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 34) ==== 8366+0+0 (secure 0 0 0) 0x7f69cc07c170 con 0x560e4193e1b0 2024-08-30T09:07:37.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.777+0000 7f69e510b740 10 client.26365 fetch_fsmap finished waiting for FSMap version 34 2024-08-30T09:07:37.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.777+0000 7f69e510b740 10 client.26365 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-30T09:07:37.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.777+0000 7f69e510b740 20 client.26365 populate_metadata read hostname 'smithi042' 2024-08-30T09:07:37.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.777+0000 7f69e510b740 1 --2- 172.21.15.42:0/1855995661 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x560e419259f0 0x560e41d13020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:07:37.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.777+0000 7f69e510b740 4 client.26365 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-30T09:07:37.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.777+0000 7f69e510b740 1 -- 172.21.15.42:0/1855995661 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f69cc0cbfd0 con 0x560e419259f0 2024-08-30T09:07:37.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.777+0000 7f69d7fff640 1 --2- 172.21.15.42:0/1855995661 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x560e419259f0 0x560e41d13020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:07:37.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.778+0000 7f69b7fff640 10 client.26365 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:07:37.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.778+0000 7f69d7fff640 1 --2- 172.21.15.42:0/1855995661 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x560e419259f0 0x560e41d13020 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f0bf5551fffb68c9 server_cookie=76666a7aa219389a in_seq=0 out_seq=0 2024-08-30T09:07:37.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.779+0000 7f69b7fff640 1 -- 172.21.15.42:0/1855995661 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f69cc0cbfd0 con 0x560e419259f0 2024-08-30T09:07:37.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.779+0000 7f69b7fff640 10 client.26365 handle_command_reply: tid=0 2024-08-30T09:07:37.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.779+0000 7f69e510b740 1 client.26365 shutdown 2024-08-30T09:07:37.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.779+0000 7f69e510b740 1 -- 172.21.15.42:0/1855995661 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560e4193e1b0 msgr2=0x560e41d20750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:07:37.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.779+0000 7f69e510b740 1 --2- 172.21.15.42:0/1855995661 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560e4193e1b0 0x560e41d20750 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f69cc0096c0 tx=0x7f69cc106050 comp rx=0 tx=0).stop 2024-08-30T09:07:37.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.779+0000 7f69e510b740 1 -- 172.21.15.42:0/1855995661 shutdown_connections 2024-08-30T09:07:37.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.779+0000 7f69e510b740 1 --2- 172.21.15.42:0/1855995661 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x560e419259f0 0x560e41d13020 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:37.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.779+0000 7f69e510b740 1 --2- 172.21.15.42:0/1855995661 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560e41967070 0x560e41d25d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:37.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.779+0000 7f69e510b740 1 --2- 172.21.15.42:0/1855995661 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560e41946390 0x560e41d20d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:37.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.779+0000 7f69e510b740 1 --2- 172.21.15.42:0/1855995661 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560e4193e1b0 0x560e41d20750 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:37.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.779+0000 7f69b7fff640 0 client.26365 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:07:37.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.779+0000 7f69e510b740 1 -- 172.21.15.42:0/1855995661 >> 172.21.15.42:0/1855995661 conn(0x560e4195b460 msgr2=0x560e41d09130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:07:37.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.779+0000 7f69e510b740 1 -- 172.21.15.42:0/1855995661 shutdown_connections 2024-08-30T09:07:37.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.779+0000 7f69e510b740 1 -- 172.21.15.42:0/1855995661 wait complete. 2024-08-30T09:07:37.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.779+0000 7f69e510b740 20 client.26365 trim_cache size 0 max 16384 2024-08-30T09:07:37.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.802+0000 7f69e510b740 1 Processor -- start 2024-08-30T09:07:37.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.804+0000 7f69e510b740 1 -- start start 2024-08-30T09:07:37.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.805+0000 7f69e510b740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560e4193e1b0 0x560e41d26460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:07:37.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.805+0000 7f69e510b740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560e41946390 0x560e41d0bed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:07:37.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.805+0000 7f69dc932640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560e4193e1b0 0x560e41d26460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:07:37.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.805+0000 7f69e510b740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560e41967070 0x560e41d0c410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:07:37.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.805+0000 7f69dc932640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560e4193e1b0 0x560e41d26460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:39558/0 (socket says 172.21.15.42:39558) 2024-08-30T09:07:37.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.805+0000 7f69d7fff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560e41946390 0x560e41d0bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:07:37.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.805+0000 7f69d7fff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560e41946390 0x560e41d0bed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:39746/0 (socket says 172.21.15.42:39746) 2024-08-30T09:07:37.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.805+0000 7f69e510b740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f69d81b11a0 con 0x560e4193e1b0 2024-08-30T09:07:37.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.805+0000 7f69dc932640 1 -- 172.21.15.42:0/1908805299 learned_addr learned my addr 172.21.15.42:0/1908805299 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:07:37.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.805+0000 7f69e510b740 1 -- 172.21.15.42:0/1908805299 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f69d81aea10 con 0x560e41967070 2024-08-30T09:07:37.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.806+0000 7f69e510b740 1 -- 172.21.15.42:0/1908805299 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f69d81ac860 con 0x560e41946390 2024-08-30T09:07:37.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.806+0000 7f69dd133640 1 --2- 172.21.15.42:0/1908805299 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560e41967070 0x560e41d0c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:07:37.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.806+0000 7f69dd133640 1 -- 172.21.15.42:0/1908805299 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560e41946390 msgr2=0x560e41d0bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:07:37.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.806+0000 7f69dd133640 1 --2- 172.21.15.42:0/1908805299 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560e41946390 0x560e41d0bed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:37.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.806+0000 7f69dd133640 1 -- 172.21.15.42:0/1908805299 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560e4193e1b0 msgr2=0x560e41d26460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:07:37.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.806+0000 7f69dd133640 1 --2- 172.21.15.42:0/1908805299 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560e4193e1b0 0x560e41d26460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:37.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.806+0000 7f69dd133640 1 -- 172.21.15.42:0/1908805299 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e41d1c1b0 con 0x560e41967070 2024-08-30T09:07:37.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.807+0000 7f69dd133640 1 --2- 172.21.15.42:0/1908805299 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560e41967070 0x560e41d0c410 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f69d00188e0 tx=0x7f69d0014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=65840844e86721ac server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:07:37.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.807+0000 7f69c27fc640 1 -- 172.21.15.42:0/1908805299 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69d0016bb0 con 0x560e41967070 2024-08-30T09:07:37.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.807+0000 7f69c27fc640 1 -- 172.21.15.42:0/1908805299 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f69d0029070 con 0x560e41967070 2024-08-30T09:07:37.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.807+0000 7f69c27fc640 1 -- 172.21.15.42:0/1908805299 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69d0017620 con 0x560e41967070 2024-08-30T09:07:37.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.808+0000 7f69e510b740 1 -- 172.21.15.42:0/1908805299 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560e41967070 msgr2=0x560e41d0c410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:07:37.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.808+0000 7f69e510b740 1 --2- 172.21.15.42:0/1908805299 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560e41967070 0x560e41d0c410 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f69d00188e0 tx=0x7f69d0014000 comp rx=0 tx=0).stop 2024-08-30T09:07:37.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.808+0000 7f69e510b740 1 -- 172.21.15.42:0/1908805299 shutdown_connections 2024-08-30T09:07:37.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.808+0000 7f69e510b740 1 --2- 172.21.15.42:0/1908805299 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560e41946390 0x560e41d0bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:37.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.808+0000 7f69e510b740 1 --2- 172.21.15.42:0/1908805299 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560e41967070 0x560e41d0c410 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:37.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.808+0000 7f69e510b740 1 --2- 172.21.15.42:0/1908805299 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560e4193e1b0 0x560e41d26460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:37.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.808+0000 7f69e510b740 1 -- 172.21.15.42:0/1908805299 >> 172.21.15.42:0/1908805299 conn(0x560e4195b460 msgr2=0x560e41d25d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:07:37.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.808+0000 7f69e510b740 1 -- 172.21.15.42:0/1908805299 shutdown_connections 2024-08-30T09:07:37.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.808+0000 7f69e510b740 1 -- 172.21.15.42:0/1908805299 wait complete. 2024-08-30T09:07:37.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.808+0000 7f69e510b740 1 -- start start 2024-08-30T09:07:37.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.809+0000 7f69e510b740 1 Processor -- start 2024-08-30T09:07:37.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.809+0000 7f69e510b740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560e4193e1b0 0x560e41d19b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:07:37.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.809+0000 7f69e510b740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560e41946390 0x560e41d17050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:07:37.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.809+0000 7f69e510b740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560e41967070 0x560e41d17590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:07:37.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.809+0000 7f69e510b740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f69d81aced0 con 0x560e41946390 2024-08-30T09:07:37.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.809+0000 7f69d7fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560e41946390 0x560e41d17050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:07:37.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.809+0000 7f69e510b740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f69d81ae340 con 0x560e41967070 2024-08-30T09:07:37.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.809+0000 7f69dc932640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560e4193e1b0 0x560e41d19b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:07:37.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.809+0000 7f69e510b740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f69d81ad5a0 con 0x560e4193e1b0 2024-08-30T09:07:37.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.809+0000 7f69dc932640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560e4193e1b0 0x560e41d19b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:39754/0 (socket says 172.21.15.42:39754) 2024-08-30T09:07:37.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.809+0000 7f69dc932640 1 -- 172.21.15.42:0/3107991007 learned_addr learned my addr 172.21.15.42:0/3107991007 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:07:37.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.809+0000 7f69e510b740 10 client.0 _ll_register_callbacks cb 0x7f69d0009990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:07:37.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.809+0000 7f69dd133640 1 --2- 172.21.15.42:0/3107991007 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560e41967070 0x560e41d17590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:07:37.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.810+0000 7f69d7fff640 1 -- 172.21.15.42:0/3107991007 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560e4193e1b0 msgr2=0x560e41d19b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:07:37.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.810+0000 7f69d7fff640 1 --2- 172.21.15.42:0/3107991007 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560e4193e1b0 0x560e41d19b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:37.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.810+0000 7f69d7fff640 1 -- 172.21.15.42:0/3107991007 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560e41967070 msgr2=0x560e41d17590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:07:37.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.810+0000 7f69d7fff640 1 --2- 172.21.15.42:0/3107991007 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560e41967070 0x560e41d17590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:37.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.810+0000 7f69d7fff640 1 -- 172.21.15.42:0/3107991007 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e41d17d00 con 0x560e41946390 2024-08-30T09:07:37.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.810+0000 7f69dc932640 1 --2- 172.21.15.42:0/3107991007 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560e4193e1b0 0x560e41d19b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-30T09:07:37.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.810+0000 7f69d7fff640 1 --2- 172.21.15.42:0/3107991007 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560e41946390 0x560e41d17050 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f69cc0095c0 tx=0x7f69c4016800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd9f5e7e5d6edbc3 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:07:37.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.810+0000 7f69b7fff640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-08-30T09:07:37.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.810+0000 7f69b7fff640 1 -- 172.21.15.42:0/3107991007 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69c400f040 con 0x560e41946390 2024-08-30T09:07:37.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.811+0000 7f69b7fff640 1 -- 172.21.15.42:0/3107991007 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f69c4005b90 con 0x560e41946390 2024-08-30T09:07:37.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.811+0000 7f69e510b740 1 -- 172.21.15.42:0/3107991007 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x560e41d0d0a0 con 0x560e41946390 2024-08-30T09:07:37.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.811+0000 7f69b7fff640 1 -- 172.21.15.42:0/3107991007 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f69c402e050 con 0x560e41946390 2024-08-30T09:07:37.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.811+0000 7f69e510b740 10 client.16632 fetch_fsmap learned FSMap version 34 2024-08-30T09:07:37.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.811+0000 7f69e510b740 1 -- 172.21.15.42:0/3107991007 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x560e41d95070 con 0x560e41946390 2024-08-30T09:07:37.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.811+0000 7f69b7fff640 1 -- 172.21.15.42:0/3107991007 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x560e41d0d0a0 con 0x560e41946390 2024-08-30T09:07:37.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.811+0000 7f69b7fff640 1 -- 172.21.15.42:0/3107991007 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 34) ==== 8366+0+0 (secure 0 0 0) 0x7f69c4023020 con 0x560e41946390 2024-08-30T09:07:37.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.812+0000 7f69e510b740 10 client.16632 fetch_fsmap finished waiting for FSMap version 34 2024-08-30T09:07:37.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.812+0000 7f69e510b740 10 client.16632 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-30T09:07:37.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.812+0000 7f69e510b740 20 client.16632 populate_metadata read hostname 'smithi042' 2024-08-30T09:07:37.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.812+0000 7f69e510b740 1 --2- 172.21.15.42:0/3107991007 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x560e418fada0 0x560e41d977e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:07:37.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.812+0000 7f69e510b740 4 client.16632 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-30T09:07:37.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.812+0000 7f69e510b740 1 -- 172.21.15.42:0/3107991007 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560e41c85580 con 0x560e418fada0 2024-08-30T09:07:37.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.812+0000 7f69dc932640 1 --2- 172.21.15.42:0/3107991007 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x560e418fada0 0x560e41d977e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:07:37.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.813+0000 7f69dc932640 1 --2- 172.21.15.42:0/3107991007 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x560e418fada0 0x560e41d977e0 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=13c026e609734665 server_cookie=452df61f843c912a in_seq=0 out_seq=0 2024-08-30T09:07:37.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.813+0000 7f69b7fff640 10 client.16632 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:07:37.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.813+0000 7f69b7fff640 1 -- 172.21.15.42:0/3107991007 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+2842 (crc 0 0 0) 0x560e41c85580 con 0x560e418fada0 2024-08-30T09:07:37.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.813+0000 7f69b7fff640 10 client.16632 handle_command_reply: tid=0 2024-08-30T09:07:37.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.813+0000 7f69e510b740 1 client.16632 shutdown 2024-08-30T09:07:37.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.813+0000 7f69e510b740 1 -- 172.21.15.42:0/3107991007 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560e41946390 msgr2=0x560e41d17050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:07:37.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.813+0000 7f69e510b740 1 --2- 172.21.15.42:0/3107991007 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560e41946390 0x560e41d17050 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f69cc0095c0 tx=0x7f69c4016800 comp rx=0 tx=0).stop 2024-08-30T09:07:37.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.814+0000 7f69e510b740 1 -- 172.21.15.42:0/3107991007 shutdown_connections 2024-08-30T09:07:37.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.814+0000 7f69e510b740 1 --2- 172.21.15.42:0/3107991007 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x560e418fada0 0x560e41d977e0 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:37.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.814+0000 7f69e510b740 1 --2- 172.21.15.42:0/3107991007 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560e41967070 0x560e41d17590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:37.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.814+0000 7f69e510b740 1 --2- 172.21.15.42:0/3107991007 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560e41946390 0x560e41d17050 secure :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f69cc0095c0 tx=0x7f69c4016800 comp rx=0 tx=0).stop 2024-08-30T09:07:37.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.814+0000 7f69e510b740 1 --2- 172.21.15.42:0/3107991007 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560e4193e1b0 0x560e41d19b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:37.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.814+0000 7f69e510b740 1 -- 172.21.15.42:0/3107991007 >> 172.21.15.42:0/3107991007 conn(0x560e4195b460 msgr2=0x560e41d222e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:07:37.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.814+0000 7f69b7fff640 0 client.16632 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:07:37.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.814+0000 7f69e510b740 1 -- 172.21.15.42:0/3107991007 shutdown_connections 2024-08-30T09:07:37.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.814+0000 7f69e510b740 1 -- 172.21.15.42:0/3107991007 wait complete. 2024-08-30T09:07:37.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.814+0000 7f69e510b740 20 client.16632 trim_cache size 0 max 16384 2024-08-30T09:07:37.819 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-08-30T09:07:37.819 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "IDLE (waiting for more scrubs)", 2024-08-30T09:07:37.819 INFO:teuthology.orchestra.run.smithi042.stdout: "scrubs": { 2024-08-30T09:07:37.819 INFO:teuthology.orchestra.run.smithi042.stdout: "598d8a75-f1cc-4388-80bc-a408bdb4eaac": { 2024-08-30T09:07:37.819 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:07:37.819 INFO:teuthology.orchestra.run.smithi042.stdout: "tag": "598d8a75-f1cc-4388-80bc-a408bdb4eaac", 2024-08-30T09:07:37.820 INFO:teuthology.orchestra.run.smithi042.stdout: "options": "recursive,force" 2024-08-30T09:07:37.820 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:07:37.820 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:07:37.820 INFO:teuthology.orchestra.run.smithi042.stdout: "scrub_stats": { 2024-08-30T09:07:37.820 INFO:teuthology.orchestra.run.smithi042.stdout: "598d8a75-f1cc-4388-80bc-a408bdb4eaac": { 2024-08-30T09:07:37.820 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:07:29.721122+0000", 2024-08-30T09:07:37.820 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:07:37.820 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:07:37.820 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:07:37.820 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:07:37.820 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:07:37.820 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:07:37.820 INFO:teuthology.orchestra.run.smithi042.stdout: "665ec740-7254-49cb-84ef-deff71c461b0": { 2024-08-30T09:07:37.820 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:06:37.144035+0000", 2024-08-30T09:07:37.820 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:07:37.821 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:07:37.821 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:07:37.821 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:07:37.821 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:07:37.821 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:07:37.821 INFO:teuthology.orchestra.run.smithi042.stdout: "1af4252a-4a99-47e9-bbf0-becf5e573dd5": { 2024-08-30T09:07:37.821 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:05:48.565881+0000", 2024-08-30T09:07:37.821 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:07:37.821 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:07:37.821 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:07:37.821 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:07:37.821 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:07:37.821 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:07:37.822 INFO:teuthology.orchestra.run.smithi042.stdout: "347d05ed-4bd3-4590-b099-314f719aab39": { 2024-08-30T09:07:37.822 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:05:00.439721+0000", 2024-08-30T09:07:37.822 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:07:37.822 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:07:37.822 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:07:37.822 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:07:37.822 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:07:37.822 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:07:37.822 INFO:teuthology.orchestra.run.smithi042.stdout: "88cd7478-2b20-47de-ba51-256479749232": { 2024-08-30T09:07:37.822 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:04:08.922625+0000", 2024-08-30T09:07:37.822 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:07:37.822 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:07:37.822 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:07:37.823 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:07:37.823 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:07:37.823 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:07:37.823 INFO:teuthology.orchestra.run.smithi042.stdout: "749f4867-0b70-49e5-af2d-8362a240ebc3": { 2024-08-30T09:07:37.823 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:03:16.386554+0000", 2024-08-30T09:07:37.823 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:07:37.823 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:07:37.823 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:07:37.823 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:07:37.823 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:07:37.823 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:07:37.823 INFO:teuthology.orchestra.run.smithi042.stdout: "85134e4b-ab4a-4885-b07e-a3ad01eaf13d": { 2024-08-30T09:07:37.823 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:01:27.738462+0000", 2024-08-30T09:07:37.823 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:07:37.824 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:07:37.824 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:07:37.824 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:07:37.824 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:07:37.824 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:07:37.824 INFO:teuthology.orchestra.run.smithi042.stdout: "c827ad7a-44ee-4e1d-9bb3-51fb7ab00c73": { 2024-08-30T09:07:37.824 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:02:21.686517+0000", 2024-08-30T09:07:37.824 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:07:37.824 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:07:37.824 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:07:37.824 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:07:37.824 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:07:37.824 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:07:37.824 INFO:teuthology.orchestra.run.smithi042.stdout: "5e2ed104-0625-478a-a78b-584e7f4c2817": { 2024-08-30T09:07:37.825 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:00:33.928400+0000", 2024-08-30T09:07:37.825 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:07:37.825 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:07:37.825 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:07:37.825 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:07:37.825 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:07:37.825 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:07:37.825 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:07:37.825 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-08-30T09:07:37.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.815+0000 7f69deb96640 1 -- 172.21.15.42:0/3984009491 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f69c80b4bb0 msgr2=0x7f69c80b70c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:07:37.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.815+0000 7f69deb96640 1 --2- 172.21.15.42:0/3984009491 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f69c80b4bb0 0x7f69c80b70c0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f69d000a400 tx=0x7f69d000d040 comp rx=0 tx=0).stop 2024-08-30T09:07:37.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.815+0000 7f69deb96640 1 -- 172.21.15.42:0/3984009491 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f69d8102880 msgr2=0x7f69d819a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:07:37.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.815+0000 7f69deb96640 1 --2- 172.21.15.42:0/3984009491 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f69d8102880 0x7f69d819a3b0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f69cc006010 tx=0x7f69cc0042f0 comp rx=0 tx=0).stop 2024-08-30T09:07:37.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.815+0000 7f69deb96640 1 -- 172.21.15.42:0/3984009491 shutdown_connections 2024-08-30T09:07:37.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.815+0000 7f69deb96640 1 --2- 172.21.15.42:0/3984009491 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f69c80b4bb0 0x7f69c80b70c0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:37.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.815+0000 7f69deb96640 1 --2- 172.21.15.42:0/3984009491 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f69d8108ac0 0x7f69d819fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:37.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.815+0000 7f69deb96640 1 --2- 172.21.15.42:0/3984009491 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f69d8108170 0x7f69d819a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:37.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.815+0000 7f69deb96640 1 --2- 172.21.15.42:0/3984009491 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f69d8102880 0x7f69d819a3b0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:37.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.815+0000 7f69deb96640 1 -- 172.21.15.42:0/3984009491 >> 172.21.15.42:0/3984009491 conn(0x7f69d8079530 msgr2=0x7f69d8106320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:07:37.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.815+0000 7f69deb96640 1 -- 172.21.15.42:0/3984009491 shutdown_connections 2024-08-30T09:07:37.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:07:37.815+0000 7f69deb96640 1 -- 172.21.15.42:0/3984009491 wait complete. 2024-08-30T09:07:38.582 INFO:tasks.cephfs.filesystem:scrub status for tag:598d8a75-f1cc-4388-80bc-a408bdb4eaac - {'path': '/', 'tag': '598d8a75-f1cc-4388-80bc-a408bdb4eaac', 'options': 'recursive,force'} 2024-08-30T09:07:38.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:07:38 smithi042 ceph-mon[18652]: pgmap v579: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 5.5 MiB/s wr, 646 op/s 2024-08-30T09:07:38.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:07:38 smithi190 ceph-mon[22001]: pgmap v579: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 5.5 MiB/s wr, 646 op/s 2024-08-30T09:07:39.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:07:38 smithi167 ceph-mon[22063]: pgmap v579: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 5.5 MiB/s wr, 646 op/s 2024-08-30T09:07:40.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:07:40 smithi042 ceph-mon[18652]: pgmap v580: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 6.3 MiB/s wr, 728 op/s 2024-08-30T09:07:40.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:07:40 smithi190 ceph-mon[22001]: pgmap v580: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 6.3 MiB/s wr, 728 op/s 2024-08-30T09:07:40.953 INFO:teuthology.orchestra.run.smithi042.stderr:progress: 415.0 s, 457.6 tps, lat 219.014 ms stddev 268.825 2024-08-30T09:07:41.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:07:40 smithi167 ceph-mon[22063]: pgmap v580: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 6.3 MiB/s wr, 728 op/s 2024-08-30T09:07:41.517 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.515+0000 7f55a898b640 1 -- 172.21.15.42:0/3855584658 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f55a41033c0 msgr2=0x7f55a41037d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:07:41.518 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.515+0000 7f55a898b640 1 --2- 172.21.15.42:0/3855584658 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f55a41033c0 0x7f55a41037d0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f5594009900 tx=0x7f559402a440 comp rx=0 tx=0).stop 2024-08-30T09:07:41.518 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.515+0000 7f55a898b640 1 -- 172.21.15.42:0/3855584658 shutdown_connections 2024-08-30T09:07:41.518 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.515+0000 7f55a898b640 1 --2- 172.21.15.42:0/3855584658 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f55a4103d60 0x7f55a410a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:41.518 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.515+0000 7f55a898b640 1 --2- 172.21.15.42:0/3855584658 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f55a41033c0 0x7f55a41037d0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:41.518 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.515+0000 7f55a898b640 1 --2- 172.21.15.42:0/3855584658 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f55a4102340 0x7f55a41027b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:41.518 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.515+0000 7f55a898b640 1 -- 172.21.15.42:0/3855584658 >> 172.21.15.42:0/3855584658 conn(0x7f55a40fdc80 msgr2=0x7f55a4100130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:07:41.519 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.516+0000 7f55a898b640 1 -- 172.21.15.42:0/3855584658 shutdown_connections 2024-08-30T09:07:41.519 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.516+0000 7f55a898b640 1 -- 172.21.15.42:0/3855584658 wait complete. 2024-08-30T09:07:41.519 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.517+0000 7f55a898b640 1 Processor -- start 2024-08-30T09:07:41.519 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.518+0000 7f55a898b640 1 -- start start 2024-08-30T09:07:41.522 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.518+0000 7f55a898b640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f55a4102340 0x7f55a41a0380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:07:41.522 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.518+0000 7f55a898b640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f55a41033c0 0x7f55a41a09f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:07:41.522 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.518+0000 7f55a898b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f55a4103d60 0x7f55a41a15a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:07:41.522 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.518+0000 7f55a898b640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f55a410f100 con 0x7f55a4102340 2024-08-30T09:07:41.522 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.518+0000 7f55a898b640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f55a410ef80 con 0x7f55a41033c0 2024-08-30T09:07:41.522 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.518+0000 7f55a898b640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f55a410f280 con 0x7f55a4103d60 2024-08-30T09:07:41.522 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.518+0000 7f55a259c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f55a4102340 0x7f55a41a0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:07:41.522 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.518+0000 7f559bfff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f55a41033c0 0x7f55a41a09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:07:41.522 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.518+0000 7f559bfff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f55a41033c0 0x7f55a41a09f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:38634/0 (socket says 172.21.15.42:38634) 2024-08-30T09:07:41.523 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.518+0000 7f559bfff640 1 -- 172.21.15.42:0/2047701578 learned_addr learned my addr 172.21.15.42:0/2047701578 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:07:41.523 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.518+0000 7f55a2d9d640 1 --2- 172.21.15.42:0/2047701578 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f55a4103d60 0x7f55a41a15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:07:41.523 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.519+0000 7f559bfff640 1 -- 172.21.15.42:0/2047701578 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f55a4103d60 msgr2=0x7f55a41a15a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:07:41.523 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.519+0000 7f559bfff640 1 --2- 172.21.15.42:0/2047701578 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f55a4103d60 0x7f55a41a15a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:41.523 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.519+0000 7f559bfff640 1 -- 172.21.15.42:0/2047701578 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f55a4102340 msgr2=0x7f55a41a0380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:07:41.523 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.519+0000 7f559bfff640 1 --2- 172.21.15.42:0/2047701578 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f55a4102340 0x7f55a41a0380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:41.523 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.519+0000 7f559bfff640 1 -- 172.21.15.42:0/2047701578 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55a41a1dc0 con 0x7f55a41033c0 2024-08-30T09:07:41.523 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.519+0000 7f559bfff640 1 --2- 172.21.15.42:0/2047701578 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f55a41033c0 0x7f55a41a09f0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f559402a960 tx=0x7f55940042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:07:41.523 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.519+0000 7f559b7fe640 1 -- 172.21.15.42:0/2047701578 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f559403f030 con 0x7f55a41033c0 2024-08-30T09:07:41.523 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.519+0000 7f55a898b640 1 -- 172.21.15.42:0/2047701578 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55a41a2050 con 0x7f55a41033c0 2024-08-30T09:07:41.524 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.519+0000 7f559b7fe640 1 -- 172.21.15.42:0/2047701578 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5594003cb0 con 0x7f55a41033c0 2024-08-30T09:07:41.524 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.519+0000 7f559b7fe640 1 -- 172.21.15.42:0/2047701578 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5594035070 con 0x7f55a41033c0 2024-08-30T09:07:41.524 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.519+0000 7f55a898b640 1 -- 172.21.15.42:0/2047701578 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55a41a87a0 con 0x7f55a41033c0 2024-08-30T09:07:41.526 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.522+0000 7f55af9d8740 1 Processor -- start 2024-08-30T09:07:41.526 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.522+0000 7f559b7fe640 1 -- 172.21.15.42:0/2047701578 <== mon.2 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7f5594042030 con 0x7f55a41033c0 2024-08-30T09:07:41.526 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.522+0000 7f55af9d8740 1 -- start start 2024-08-30T09:07:41.526 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.522+0000 7f55af9d8740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a8a567c1b0 0x55a8a5a22180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:07:41.526 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.522+0000 7f55af9d8740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a8a56532f0 0x55a8a5a227f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:07:41.526 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.522+0000 7f55af9d8740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a8a565b4d0 0x55a8a5a23340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:07:41.527 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.522+0000 7f55af9d8740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55a8a5995000 con 0x55a8a56532f0 2024-08-30T09:07:41.527 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.522+0000 7f55af9d8740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55a8a5984590 con 0x55a8a567c1b0 2024-08-30T09:07:41.527 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.522+0000 7f55af9d8740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55a8a5988540 con 0x55a8a565b4d0 2024-08-30T09:07:41.527 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.523+0000 7f55a259c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a8a56532f0 0x55a8a5a227f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:07:41.527 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.523+0000 7f559bfff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a8a565b4d0 0x55a8a5a23340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:07:41.527 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.523+0000 7f55a2d9d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a8a567c1b0 0x55a8a5a22180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:07:41.527 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.523+0000 7f55a259c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a8a56532f0 0x55a8a5a227f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34712/0 (socket says 172.21.15.42:34712) 2024-08-30T09:07:41.527 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.523+0000 7f55a2d9d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a8a567c1b0 0x55a8a5a22180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:38636/0 (socket says 172.21.15.42:38636) 2024-08-30T09:07:41.527 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.523+0000 7f55a2d9d640 1 -- 172.21.15.42:0/1926100765 learned_addr learned my addr 172.21.15.42:0/1926100765 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:07:41.527 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.523+0000 7f559b7fe640 1 --2- 172.21.15.42:0/2047701578 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f55900b4b30 0x7f55900b7040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:07:41.527 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.523+0000 7f559bfff640 1 -- 172.21.15.42:0/1926100765 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a8a567c1b0 msgr2=0x55a8a5a22180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:07:41.527 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.523+0000 7f559bfff640 1 --2- 172.21.15.42:0/1926100765 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a8a567c1b0 0x55a8a5a22180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:41.527 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.523+0000 7f559bfff640 1 -- 172.21.15.42:0/1926100765 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a8a56532f0 msgr2=0x55a8a5a227f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:07:41.528 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.523+0000 7f559bfff640 1 --2- 172.21.15.42:0/1926100765 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a8a56532f0 0x55a8a5a227f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:41.528 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.523+0000 7f559bfff640 1 -- 172.21.15.42:0/1926100765 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a8a56e7790 con 0x55a8a565b4d0 2024-08-30T09:07:41.528 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.524+0000 7f55a2d9d640 1 --2- 172.21.15.42:0/2047701578 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f55900b4b30 0x7f55900b7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:07:41.528 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.524+0000 7f559b7fe640 1 -- 172.21.15.42:0/2047701578 <== mon.2 v2:172.21.15.167:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f5594105a40 con 0x7f55a41033c0 2024-08-30T09:07:41.528 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.524+0000 7f559bfff640 1 --2- 172.21.15.42:0/1926100765 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a8a565b4d0 0x55a8a5a23340 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f5594103180 tx=0x7f5594090b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e1b1a43ea4692f35 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:07:41.528 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.525+0000 7f55a2d9d640 1 --2- 172.21.15.42:0/2047701578 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f55900b4b30 0x7f55900b7040 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f559c009fd0 tx=0x7f559c004b40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:07:41.530 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.525+0000 7f5598ff9640 1 -- 172.21.15.42:0/1926100765 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5594049070 con 0x55a8a565b4d0 2024-08-30T09:07:41.530 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.525+0000 7f5598ff9640 1 -- 172.21.15.42:0/1926100765 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5594003e60 con 0x55a8a565b4d0 2024-08-30T09:07:41.531 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.525+0000 7f5598ff9640 1 -- 172.21.15.42:0/1926100765 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55940962f0 con 0x55a8a565b4d0 2024-08-30T09:07:41.534 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.530+0000 7f55af9d8740 1 -- 172.21.15.42:0/1926100765 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a8a565b4d0 msgr2=0x55a8a5a23340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:07:41.534 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.530+0000 7f55af9d8740 1 --2- 172.21.15.42:0/1926100765 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a8a565b4d0 0x55a8a5a23340 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f5594103180 tx=0x7f5594090b90 comp rx=0 tx=0).stop 2024-08-30T09:07:41.538 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.536+0000 7f55af9d8740 1 -- 172.21.15.42:0/1926100765 shutdown_connections 2024-08-30T09:07:41.538 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.536+0000 7f55af9d8740 1 --2- 172.21.15.42:0/1926100765 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a8a565b4d0 0x55a8a5a23340 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:41.538 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.536+0000 7f55af9d8740 1 --2- 172.21.15.42:0/1926100765 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a8a56532f0 0x55a8a5a227f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:41.539 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.536+0000 7f55af9d8740 1 --2- 172.21.15.42:0/1926100765 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a8a567c1b0 0x55a8a5a22180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:41.539 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.536+0000 7f55af9d8740 1 -- 172.21.15.42:0/1926100765 >> 172.21.15.42:0/1926100765 conn(0x55a8a5670300 msgr2=0x55a8a5a1dc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:07:41.544 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.541+0000 7f55af9d8740 1 -- 172.21.15.42:0/1926100765 shutdown_connections 2024-08-30T09:07:41.545 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.542+0000 7f55af9d8740 1 -- 172.21.15.42:0/1926100765 wait complete. 2024-08-30T09:07:41.545 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.543+0000 7f55af9d8740 1 -- start start 2024-08-30T09:07:41.545 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.543+0000 7f55af9d8740 1 Processor -- start 2024-08-30T09:07:41.548 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.545+0000 7f55af9d8740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a8a56532f0 0x55a8a5a2b870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:07:41.548 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.545+0000 7f55af9d8740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a8a565b4d0 0x55a8a5a2bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:07:41.548 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.545+0000 7f55af9d8740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a8a567c1b0 0x55a8a5a2caa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:07:41.548 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.545+0000 7f55af9d8740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55a8a58bdab0 con 0x55a8a565b4d0 2024-08-30T09:07:41.548 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.545+0000 7f55af9d8740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55a8a590b680 con 0x55a8a56532f0 2024-08-30T09:07:41.548 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.545+0000 7f55af9d8740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55a8a5988540 con 0x55a8a567c1b0 2024-08-30T09:07:41.548 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.545+0000 7f55af9d8740 10 client.0 _ll_register_callbacks cb 0x55a8a5693a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:07:41.551 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.548+0000 7f55a2d9d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a8a567c1b0 0x55a8a5a2caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:07:41.552 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.548+0000 7f559bfff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a8a565b4d0 0x55a8a5a2bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:07:41.552 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.548+0000 7f559bfff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a8a565b4d0 0x55a8a5a2bf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34722/0 (socket says 172.21.15.42:34722) 2024-08-30T09:07:41.552 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.548+0000 7f559bfff640 1 -- 172.21.15.42:0/1302909124 learned_addr learned my addr 172.21.15.42:0/1302909124 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:07:41.552 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.550+0000 7f55a259c640 1 --2- 172.21.15.42:0/1302909124 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a8a56532f0 0x55a8a5a2b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:07:41.554 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.553+0000 7f55a259c640 1 -- 172.21.15.42:0/1302909124 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a8a567c1b0 msgr2=0x55a8a5a2caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:07:41.555 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.553+0000 7f55a259c640 1 --2- 172.21.15.42:0/1302909124 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a8a567c1b0 0x55a8a5a2caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:41.555 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.553+0000 7f55a259c640 1 -- 172.21.15.42:0/1302909124 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a8a565b4d0 msgr2=0x55a8a5a2bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:07:41.555 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.553+0000 7f55a259c640 1 --2- 172.21.15.42:0/1302909124 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a8a565b4d0 0x55a8a5a2bf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:41.555 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.553+0000 7f55a259c640 1 -- 172.21.15.42:0/1302909124 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a8a5760620 con 0x55a8a56532f0 2024-08-30T09:07:41.555 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.553+0000 7f559bfff640 1 --2- 172.21.15.42:0/1302909124 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a8a565b4d0 0x55a8a5a2bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-30T09:07:41.555 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.553+0000 7f55a2d9d640 1 --2- 172.21.15.42:0/1302909124 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a8a567c1b0 0x55a8a5a2caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-30T09:07:41.563 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.561+0000 7f55a259c640 1 --2- 172.21.15.42:0/1302909124 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a8a56532f0 0x55a8a5a2b870 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f558c012700 tx=0x7f558c012bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b272056a6dbdd484 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:07:41.563 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.561+0000 7f5589ffb640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-30T09:07:41.564 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.562+0000 7f5589ffb640 1 -- 172.21.15.42:0/1302909124 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f558c004030 con 0x55a8a56532f0 2024-08-30T09:07:41.564 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.562+0000 7f55af9d8740 1 -- 172.21.15.42:0/1302909124 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55a8a5996f50 con 0x55a8a56532f0 2024-08-30T09:07:41.564 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.562+0000 7f5589ffb640 1 -- 172.21.15.42:0/1302909124 <== mon.2 v2:172.21.15.167:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f558c0041d0 con 0x55a8a56532f0 2024-08-30T09:07:41.565 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.563+0000 7f5589ffb640 1 -- 172.21.15.42:0/1302909124 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f558c025050 con 0x55a8a56532f0 2024-08-30T09:07:41.572 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.570+0000 7f55af9d8740 10 client.26414 fetch_fsmap learned FSMap version 34 2024-08-30T09:07:41.572 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.570+0000 7f55af9d8740 1 -- 172.21.15.42:0/1302909124 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55a8a5a46b20 con 0x55a8a56532f0 2024-08-30T09:07:41.575 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.573+0000 7f5589ffb640 1 -- 172.21.15.42:0/1302909124 <== mon.2 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55a8a5996f50 con 0x55a8a56532f0 2024-08-30T09:07:41.577 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.575+0000 7f5589ffb640 1 -- 172.21.15.42:0/1302909124 <== mon.2 v2:172.21.15.167:3300/0 5 ==== fsmap(e 34) ==== 8366+0+0 (secure 0 0 0) 0x7f558c003a10 con 0x55a8a56532f0 2024-08-30T09:07:41.577 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.575+0000 7f55af9d8740 10 client.26414 fetch_fsmap finished waiting for FSMap version 34 2024-08-30T09:07:41.577 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.575+0000 7f55af9d8740 10 client.26414 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-30T09:07:41.577 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.575+0000 7f55af9d8740 20 client.26414 populate_metadata read hostname 'smithi042' 2024-08-30T09:07:41.577 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.575+0000 7f55af9d8740 1 --2- 172.21.15.42:0/1302909124 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55a8a563ad80 0x55a8a5a4a720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:07:41.577 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.575+0000 7f55af9d8740 4 client.26414 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-30T09:07:41.578 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.575+0000 7f55af9d8740 1 -- 172.21.15.42:0/1302909124 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a8a59929f0 con 0x55a8a563ad80 2024-08-30T09:07:41.580 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.577+0000 7f559bfff640 1 --2- 172.21.15.42:0/1302909124 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55a8a563ad80 0x55a8a5a4a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:07:41.583 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.581+0000 7f559bfff640 1 --2- 172.21.15.42:0/1302909124 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55a8a563ad80 0x55a8a5a4a720 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=53ad2f234f302731 server_cookie=fb8c62188ccab8be in_seq=0 out_seq=0 2024-08-30T09:07:41.583 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.581+0000 7f5589ffb640 10 client.26414 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:07:41.584 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.582+0000 7f5589ffb640 1 -- 172.21.15.42:0/1302909124 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a8a59929f0 con 0x55a8a563ad80 2024-08-30T09:07:41.584 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.582+0000 7f5589ffb640 10 client.26414 handle_command_reply: tid=0 2024-08-30T09:07:41.584 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.582+0000 7f55af9d8740 1 client.26414 shutdown 2024-08-30T09:07:41.586 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.584+0000 7f55af9d8740 1 -- 172.21.15.42:0/1302909124 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a8a56532f0 msgr2=0x55a8a5a2b870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:07:41.586 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.584+0000 7f55af9d8740 1 --2- 172.21.15.42:0/1302909124 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a8a56532f0 0x55a8a5a2b870 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f558c012700 tx=0x7f558c012bc0 comp rx=0 tx=0).stop 2024-08-30T09:07:41.590 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.587+0000 7f55af9d8740 1 -- 172.21.15.42:0/1302909124 shutdown_connections 2024-08-30T09:07:41.590 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.587+0000 7f55af9d8740 1 --2- 172.21.15.42:0/1302909124 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55a8a563ad80 0x55a8a5a4a720 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:41.590 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.587+0000 7f55af9d8740 1 --2- 172.21.15.42:0/1302909124 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a8a567c1b0 0x55a8a5a2caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:41.590 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.587+0000 7f55af9d8740 1 --2- 172.21.15.42:0/1302909124 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a8a565b4d0 0x55a8a5a2bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:41.590 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.587+0000 7f55af9d8740 1 --2- 172.21.15.42:0/1302909124 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a8a56532f0 0x55a8a5a2b870 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:41.591 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.587+0000 7f55af9d8740 1 -- 172.21.15.42:0/1302909124 >> 172.21.15.42:0/1302909124 conn(0x55a8a5670300 msgr2=0x55a8a5a1c170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:07:41.591 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.588+0000 7f5589ffb640 0 client.26414 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:07:41.591 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.588+0000 7f55af9d8740 1 -- 172.21.15.42:0/1302909124 shutdown_connections 2024-08-30T09:07:41.594 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.592+0000 7f55af9d8740 1 -- 172.21.15.42:0/1302909124 wait complete. 2024-08-30T09:07:41.594 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.592+0000 7f55af9d8740 20 client.26414 trim_cache size 0 max 16384 2024-08-30T09:07:41.632 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.629+0000 7f55af9d8740 1 Processor -- start 2024-08-30T09:07:41.632 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.630+0000 7f55af9d8740 1 -- start start 2024-08-30T09:07:41.633 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.630+0000 7f55af9d8740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a8a56532f0 0x55a8a5a4a690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:07:41.633 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.631+0000 7f55af9d8740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a8a565b4d0 0x55a8a5a1ca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:07:41.633 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.631+0000 7f55af9d8740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a8a567c1b0 0x55a8a5a1cf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:07:41.633 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.631+0000 7f55af9d8740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f55a41acb70 con 0x55a8a56532f0 2024-08-30T09:07:41.633 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.631+0000 7f55af9d8740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f55a41aa3e0 con 0x55a8a567c1b0 2024-08-30T09:07:41.633 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.631+0000 7f55af9d8740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f55a419c950 con 0x55a8a565b4d0 2024-08-30T09:07:41.634 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.631+0000 7f55a259c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a8a56532f0 0x55a8a5a4a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:07:41.634 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.631+0000 7f55a259c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a8a56532f0 0x55a8a5a4a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34724/0 (socket says 172.21.15.42:34724) 2024-08-30T09:07:41.634 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.631+0000 7f55a259c640 1 -- 172.21.15.42:0/287436625 learned_addr learned my addr 172.21.15.42:0/287436625 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:07:41.634 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.632+0000 7f55a259c640 1 -- 172.21.15.42:0/287436625 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a8a565b4d0 msgr2=0x55a8a5a1ca20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-30T09:07:41.634 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.632+0000 7f55a259c640 1 --2- 172.21.15.42:0/287436625 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a8a565b4d0 0x55a8a5a1ca20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:41.634 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.632+0000 7f55a259c640 1 -- 172.21.15.42:0/287436625 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a8a567c1b0 msgr2=0x55a8a5a1cf60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-30T09:07:41.635 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.632+0000 7f55a259c640 1 --2- 172.21.15.42:0/287436625 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a8a567c1b0 0x55a8a5a1cf60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:41.635 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.632+0000 7f55a259c640 1 -- 172.21.15.42:0/287436625 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a8a5a2bc90 con 0x55a8a56532f0 2024-08-30T09:07:41.635 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.632+0000 7f55a259c640 1 --2- 172.21.15.42:0/287436625 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a8a56532f0 0x55a8a5a4a690 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f558c010630 tx=0x7f558c010660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f5dd6d8961394be server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:07:41.635 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.633+0000 7f558bfff640 1 -- 172.21.15.42:0/287436625 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f558c022040 con 0x55a8a56532f0 2024-08-30T09:07:41.635 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.633+0000 7f558bfff640 1 -- 172.21.15.42:0/287436625 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f558c00fdd0 con 0x55a8a56532f0 2024-08-30T09:07:41.636 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.634+0000 7f55af9d8740 1 -- 172.21.15.42:0/287436625 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a8a56532f0 msgr2=0x55a8a5a4a690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:07:41.636 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.634+0000 7f55af9d8740 1 --2- 172.21.15.42:0/287436625 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a8a56532f0 0x55a8a5a4a690 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f558c010630 tx=0x7f558c010660 comp rx=0 tx=0).stop 2024-08-30T09:07:41.637 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.635+0000 7f55af9d8740 1 -- 172.21.15.42:0/287436625 shutdown_connections 2024-08-30T09:07:41.637 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.635+0000 7f55af9d8740 1 --2- 172.21.15.42:0/287436625 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a8a565b4d0 0x55a8a5a1ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:41.637 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.635+0000 7f55af9d8740 1 --2- 172.21.15.42:0/287436625 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a8a567c1b0 0x55a8a5a1cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:41.637 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.635+0000 7f55af9d8740 1 --2- 172.21.15.42:0/287436625 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a8a56532f0 0x55a8a5a4a690 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:41.637 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.635+0000 7f55af9d8740 1 -- 172.21.15.42:0/287436625 >> 172.21.15.42:0/287436625 conn(0x55a8a5670300 msgr2=0x55a8a5a18400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:07:41.644 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.643+0000 7f55af9d8740 1 -- 172.21.15.42:0/287436625 shutdown_connections 2024-08-30T09:07:41.649 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.647+0000 7f55af9d8740 1 -- 172.21.15.42:0/287436625 wait complete. 2024-08-30T09:07:41.649 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.647+0000 7f55af9d8740 1 -- start start 2024-08-30T09:07:41.649 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.647+0000 7f55af9d8740 1 Processor -- start 2024-08-30T09:07:41.650 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.648+0000 7f55af9d8740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a8a565b4d0 0x55a8a5a2fd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:07:41.650 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.648+0000 7f55af9d8740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a8a567c1b0 0x55a8a5a30420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:07:41.650 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.648+0000 7f55af9d8740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a8a56101c0 0x55a8a5a30f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:07:41.650 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.648+0000 7f55af9d8740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f55a41a8970 con 0x55a8a567c1b0 2024-08-30T09:07:41.650 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.648+0000 7f55af9d8740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f55a41a9d10 con 0x55a8a565b4d0 2024-08-30T09:07:41.650 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.648+0000 7f55af9d8740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f55a41a8f70 con 0x55a8a56101c0 2024-08-30T09:07:41.650 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.648+0000 7f55af9d8740 10 client.0 _ll_register_callbacks cb 0x7f558c0097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:07:41.653 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.651+0000 7f55a2d9d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a8a56101c0 0x55a8a5a30f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:07:41.653 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.651+0000 7f55a2d9d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a8a56101c0 0x55a8a5a30f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:33390/0 (socket says 172.21.15.42:33390) 2024-08-30T09:07:41.653 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.651+0000 7f55a2d9d640 1 -- 172.21.15.42:0/2287721713 learned_addr learned my addr 172.21.15.42:0/2287721713 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:07:41.653 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.651+0000 7f559bfff640 1 --2- 172.21.15.42:0/2287721713 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a8a567c1b0 0x55a8a5a30420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:07:41.653 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.651+0000 7f55a259c640 1 --2- 172.21.15.42:0/2287721713 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a8a565b4d0 0x55a8a5a2fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:07:41.656 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.653+0000 7f55a2d9d640 1 -- 172.21.15.42:0/2287721713 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a8a565b4d0 msgr2=0x55a8a5a2fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:07:41.656 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.653+0000 7f55a2d9d640 1 --2- 172.21.15.42:0/2287721713 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a8a565b4d0 0x55a8a5a2fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:41.656 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.654+0000 7f55a2d9d640 1 -- 172.21.15.42:0/2287721713 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a8a567c1b0 msgr2=0x55a8a5a30420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:07:41.657 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.654+0000 7f55a2d9d640 1 --2- 172.21.15.42:0/2287721713 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a8a567c1b0 0x55a8a5a30420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:41.657 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.654+0000 7f55a2d9d640 1 -- 172.21.15.42:0/2287721713 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a8a5a316f0 con 0x55a8a56101c0 2024-08-30T09:07:41.660 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.656+0000 7f55a2d9d640 1 --2- 172.21.15.42:0/2287721713 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a8a56101c0 0x55a8a5a30f90 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f559c0169a0 tx=0x7f559c00d040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8aa2350114052f09 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:07:41.660 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.658+0000 7f5589ffb640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-08-30T09:07:41.660 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.658+0000 7f5589ffb640 1 -- 172.21.15.42:0/2287721713 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f559c01c040 con 0x55a8a56101c0 2024-08-30T09:07:41.660 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.658+0000 7f5589ffb640 1 -- 172.21.15.42:0/2287721713 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f559c015c60 con 0x55a8a56101c0 2024-08-30T09:07:41.661 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.659+0000 7f55af9d8740 1 -- 172.21.15.42:0/2287721713 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55a8a5990130 con 0x55a8a56101c0 2024-08-30T09:07:41.662 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.660+0000 7f5589ffb640 1 -- 172.21.15.42:0/2287721713 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f559c015e60 con 0x55a8a56101c0 2024-08-30T09:07:41.663 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.661+0000 7f55af9d8740 10 client.26386 fetch_fsmap learned FSMap version 34 2024-08-30T09:07:41.663 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.661+0000 7f55af9d8740 1 -- 172.21.15.42:0/2287721713 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55a8a5a32b70 con 0x55a8a56101c0 2024-08-30T09:07:41.666 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.664+0000 7f5589ffb640 1 -- 172.21.15.42:0/2287721713 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55a8a5990130 con 0x55a8a56101c0 2024-08-30T09:07:41.667 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.665+0000 7f5589ffb640 1 -- 172.21.15.42:0/2287721713 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 34) ==== 8366+0+0 (secure 0 0 0) 0x7f559c033020 con 0x55a8a56101c0 2024-08-30T09:07:41.667 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.665+0000 7f55af9d8740 10 client.26386 fetch_fsmap finished waiting for FSMap version 34 2024-08-30T09:07:41.667 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.665+0000 7f55af9d8740 10 client.26386 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-30T09:07:41.667 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.665+0000 7f55af9d8740 20 client.26386 populate_metadata read hostname 'smithi042' 2024-08-30T09:07:41.667 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.665+0000 7f55af9d8740 1 --2- 172.21.15.42:0/2287721713 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55a8a5a33560 0x55a8a5ab7130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:07:41.667 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.665+0000 7f55af9d8740 4 client.26386 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-30T09:07:41.667 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.665+0000 7f55af9d8740 1 -- 172.21.15.42:0/2287721713 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a8a5907800 con 0x55a8a5a33560 2024-08-30T09:07:41.668 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.666+0000 7f55a259c640 1 --2- 172.21.15.42:0/2287721713 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55a8a5a33560 0x55a8a5ab7130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:07:41.669 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.667+0000 7f55a259c640 1 --2- 172.21.15.42:0/2287721713 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55a8a5a33560 0x55a8a5ab7130 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=22f1d76b5b24ad12 server_cookie=ad6f93643aaf85ff in_seq=0 out_seq=0 2024-08-30T09:07:41.674 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.669+0000 7f5589ffb640 10 client.26386 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:07:41.749 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.746+0000 7f5589ffb640 1 -- 172.21.15.42:0/2287721713 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a8a5907800 con 0x55a8a5a33560 2024-08-30T09:07:41.749 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.746+0000 7f5589ffb640 10 client.26386 handle_command_reply: tid=0 2024-08-30T09:07:41.749 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.747+0000 7f55af9d8740 1 client.26386 shutdown 2024-08-30T09:07:41.750 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.747+0000 7f55af9d8740 1 -- 172.21.15.42:0/2287721713 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a8a56101c0 msgr2=0x55a8a5a30f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:07:41.750 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.747+0000 7f55af9d8740 1 --2- 172.21.15.42:0/2287721713 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a8a56101c0 0x55a8a5a30f90 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f559c0169a0 tx=0x7f559c00d040 comp rx=0 tx=0).stop 2024-08-30T09:07:41.750 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.747+0000 7f55af9d8740 1 -- 172.21.15.42:0/2287721713 shutdown_connections 2024-08-30T09:07:41.750 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.747+0000 7f55af9d8740 1 --2- 172.21.15.42:0/2287721713 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55a8a5a33560 0x55a8a5ab7130 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:41.750 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.747+0000 7f55af9d8740 1 --2- 172.21.15.42:0/2287721713 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a8a56101c0 0x55a8a5a30f90 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:41.750 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.747+0000 7f55af9d8740 1 --2- 172.21.15.42:0/2287721713 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a8a567c1b0 0x55a8a5a30420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:41.750 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.747+0000 7f55af9d8740 1 --2- 172.21.15.42:0/2287721713 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a8a565b4d0 0x55a8a5a2fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:41.750 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.747+0000 7f55af9d8740 1 -- 172.21.15.42:0/2287721713 >> 172.21.15.42:0/2287721713 conn(0x55a8a5670300 msgr2=0x55a8a5a1c9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:07:41.750 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.747+0000 7f5589ffb640 0 client.26386 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:07:41.750 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.747+0000 7f55af9d8740 1 -- 172.21.15.42:0/2287721713 shutdown_connections 2024-08-30T09:07:41.750 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.747+0000 7f55af9d8740 1 -- 172.21.15.42:0/2287721713 wait complete. 2024-08-30T09:07:41.750 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.747+0000 7f55af9d8740 20 client.26386 trim_cache size 0 max 16384 2024-08-30T09:07:41.751 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-08-30T09:07:41.751 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2024-08-30T09:07:41.751 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-08-30T09:07:41.751 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-08-30T09:07:41.752 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.748+0000 7f55a898b640 1 -- 172.21.15.42:0/2047701578 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f55900b4b30 msgr2=0x7f55900b7040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:07:41.752 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.748+0000 7f55a898b640 1 --2- 172.21.15.42:0/2047701578 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f55900b4b30 0x7f55900b7040 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f559c009fd0 tx=0x7f559c004b40 comp rx=0 tx=0).stop 2024-08-30T09:07:41.752 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.748+0000 7f55a898b640 1 -- 172.21.15.42:0/2047701578 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f55a41033c0 msgr2=0x7f55a41a09f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:07:41.752 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.748+0000 7f55a898b640 1 --2- 172.21.15.42:0/2047701578 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f55a41033c0 0x7f55a41a09f0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f559402a960 tx=0x7f55940042f0 comp rx=0 tx=0).stop 2024-08-30T09:07:41.752 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.748+0000 7f55a898b640 1 -- 172.21.15.42:0/2047701578 shutdown_connections 2024-08-30T09:07:41.752 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.748+0000 7f55a898b640 1 --2- 172.21.15.42:0/2047701578 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f55900b4b30 0x7f55900b7040 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:41.753 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.748+0000 7f55a898b640 1 --2- 172.21.15.42:0/2047701578 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f55a4103d60 0x7f55a41a15a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:41.753 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.748+0000 7f55a898b640 1 --2- 172.21.15.42:0/2047701578 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f55a41033c0 0x7f55a41a09f0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:41.753 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.748+0000 7f55a898b640 1 --2- 172.21.15.42:0/2047701578 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f55a4102340 0x7f55a41a0380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:41.753 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.748+0000 7f55a898b640 1 -- 172.21.15.42:0/2047701578 >> 172.21.15.42:0/2047701578 conn(0x7f55a40fdc80 msgr2=0x7f55a41046b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:07:41.753 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.748+0000 7f55a898b640 1 -- 172.21.15.42:0/2047701578 shutdown_connections 2024-08-30T09:07:41.753 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:41.748+0000 7f55a898b640 1 -- 172.21.15.42:0/2047701578 wait complete. 2024-08-30T09:07:42.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:07:42 smithi042 ceph-mon[18652]: pgmap v581: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 4.5 MiB/s wr, 543 op/s 2024-08-30T09:07:42.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:07:42 smithi190 ceph-mon[22001]: pgmap v581: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 4.5 MiB/s wr, 543 op/s 2024-08-30T09:07:43.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:07:42 smithi167 ceph-mon[22063]: pgmap v581: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 4.5 MiB/s wr, 543 op/s 2024-08-30T09:07:43.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:07:43 smithi042 ceph-mon[18652]: scrub summary: idle 2024-08-30T09:07:43.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:07:43 smithi042 ceph-mon[18652]: scrub summary: idle 2024-08-30T09:07:43.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:07:43 smithi042 ceph-mon[18652]: scrub summary: idle 2024-08-30T09:07:43.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:07:43 smithi190 ceph-mon[22001]: scrub summary: idle 2024-08-30T09:07:43.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:07:43 smithi190 ceph-mon[22001]: scrub summary: idle 2024-08-30T09:07:43.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:07:43 smithi190 ceph-mon[22001]: scrub summary: idle 2024-08-30T09:07:44.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:07:43 smithi167 ceph-mon[22063]: scrub summary: idle 2024-08-30T09:07:44.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:07:43 smithi167 ceph-mon[22063]: scrub summary: idle 2024-08-30T09:07:44.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:07:43 smithi167 ceph-mon[22063]: scrub summary: idle 2024-08-30T09:07:44.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:07:44 smithi042 ceph-mon[18652]: pgmap v582: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 5.9 MiB/s wr, 656 op/s 2024-08-30T09:07:44.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:07:44 smithi190 ceph-mon[22001]: pgmap v582: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 5.9 MiB/s wr, 656 op/s 2024-08-30T09:07:45.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:07:44 smithi167 ceph-mon[22063]: pgmap v582: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 5.9 MiB/s wr, 656 op/s 2024-08-30T09:07:45.953 INFO:teuthology.orchestra.run.smithi042.stderr:progress: 420.0 s, 549.4 tps, lat 183.246 ms stddev 216.486 2024-08-30T09:07:46.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:07:46 smithi042 ceph-mon[18652]: pgmap v583: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 4.4 MiB/s wr, 465 op/s 2024-08-30T09:07:46.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:07:46 smithi190 ceph-mon[22001]: pgmap v583: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 4.4 MiB/s wr, 465 op/s 2024-08-30T09:07:47.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:07:46 smithi167 ceph-mon[22063]: pgmap v583: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 4.4 MiB/s wr, 465 op/s 2024-08-30T09:07:48.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:07:48 smithi042 ceph-mon[18652]: pgmap v584: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 4.4 MiB/s wr, 467 op/s 2024-08-30T09:07:48.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:07:48 smithi190 ceph-mon[22001]: pgmap v584: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 4.4 MiB/s wr, 467 op/s 2024-08-30T09:07:49.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:07:48 smithi167 ceph-mon[22063]: pgmap v584: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 4.4 MiB/s wr, 467 op/s 2024-08-30T09:07:50.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:07:50 smithi190 ceph-mon[22001]: pgmap v585: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 6.2 MiB/s wr, 602 op/s 2024-08-30T09:07:50.953 INFO:teuthology.orchestra.run.smithi042.stderr:progress: 425.0 s, 791.4 tps, lat 126.875 ms stddev 152.758 2024-08-30T09:07:51.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:07:50 smithi167 ceph-mon[22063]: pgmap v585: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 6.2 MiB/s wr, 602 op/s 2024-08-30T09:07:51.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:07:50 smithi042 ceph-mon[18652]: pgmap v585: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 6.2 MiB/s wr, 602 op/s 2024-08-30T09:07:52.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:07:52 smithi190 ceph-mon[22001]: pgmap v586: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 5.2 MiB/s wr, 495 op/s 2024-08-30T09:07:53.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:07:52 smithi167 ceph-mon[22063]: pgmap v586: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 5.2 MiB/s wr, 495 op/s 2024-08-30T09:07:53.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:07:52 smithi042 ceph-mon[18652]: pgmap v586: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 5.2 MiB/s wr, 495 op/s 2024-08-30T09:07:54.917 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:54.915+0000 7f5d0499e640 1 -- 172.21.15.42:0/3041673428 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5d0010cac0 msgr2=0x7f5d0010eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:07:54.917 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:54.915+0000 7f5d0499e640 1 --2- 172.21.15.42:0/3041673428 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5d0010cac0 0x7f5d0010eee0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f5cec009900 tx=0x7f5cec02a440 comp rx=0 tx=0).stop 2024-08-30T09:07:54.923 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:54.921+0000 7f5d0499e640 1 -- 172.21.15.42:0/3041673428 shutdown_connections 2024-08-30T09:07:54.923 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:54.921+0000 7f5d0499e640 1 --2- 172.21.15.42:0/3041673428 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5d0010f5d0 0x7f5d0010fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:54.923 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:54.921+0000 7f5d0499e640 1 --2- 172.21.15.42:0/3041673428 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5d0010cac0 0x7f5d0010eee0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:54.923 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:54.921+0000 7f5d0499e640 1 --2- 172.21.15.42:0/3041673428 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5d0010a160 0x7f5d0010c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:54.923 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:54.921+0000 7f5d0499e640 1 -- 172.21.15.42:0/3041673428 >> 172.21.15.42:0/3041673428 conn(0x7f5d000703e0 msgr2=0x7f5d00072890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:07:54.931 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:54.929+0000 7f5d0499e640 1 -- 172.21.15.42:0/3041673428 shutdown_connections 2024-08-30T09:07:54.933 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:54.931+0000 7f5d0499e640 1 -- 172.21.15.42:0/3041673428 wait complete. 2024-08-30T09:07:54.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:07:54 smithi190 ceph-mon[22001]: pgmap v587: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 6.0 MiB/s wr, 584 op/s 2024-08-30T09:07:54.938 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:54.932+0000 7f5d0499e640 1 Processor -- start 2024-08-30T09:07:54.939 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:54.935+0000 7f5d0499e640 1 -- start start 2024-08-30T09:07:54.939 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:54.936+0000 7f5d0499e640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5d0010a160 0x7f5d001a8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:07:54.939 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:54.936+0000 7f5d0499e640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5d0010cac0 0x7f5d001a9120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:07:54.939 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:54.936+0000 7f5d0499e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5d0010f5d0 0x7f5d001a9cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:07:54.939 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:54.936+0000 7f5d0499e640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f5d0011de30 con 0x7f5d0010cac0 2024-08-30T09:07:54.939 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:54.937+0000 7f5d0499e640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f5d0011dcb0 con 0x7f5d0010a160 2024-08-30T09:07:54.939 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:54.937+0000 7f5d0499e640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f5d0011dfb0 con 0x7f5d0010f5d0 2024-08-30T09:07:54.942 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:54.937+0000 7f5cfeffd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5d0010cac0 0x7f5d001a9120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:07:54.942 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:54.937+0000 7f5cfeffd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5d0010cac0 0x7f5d001a9120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:51560/0 (socket says 172.21.15.42:51560) 2024-08-30T09:07:54.942 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:54.937+0000 7f5cfeffd640 1 -- 172.21.15.42:0/2763326012 learned_addr learned my addr 172.21.15.42:0/2763326012 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:07:54.942 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:54.938+0000 7f5cfeffd640 1 -- 172.21.15.42:0/2763326012 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5d0010f5d0 msgr2=0x7f5d001a9cd0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-30T09:07:54.942 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:54.938+0000 7f5cfeffd640 1 --2- 172.21.15.42:0/2763326012 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5d0010f5d0 0x7f5d001a9cd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:54.942 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:54.938+0000 7f5cfeffd640 1 -- 172.21.15.42:0/2763326012 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5d0010a160 msgr2=0x7f5d001a8ab0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-30T09:07:54.942 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:54.938+0000 7f5cfeffd640 1 --2- 172.21.15.42:0/2763326012 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5d0010a160 0x7f5d001a8ab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:54.942 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:54.938+0000 7f5cfeffd640 1 -- 172.21.15.42:0/2763326012 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d001aa4f0 con 0x7f5d0010cac0 2024-08-30T09:07:54.942 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:54.938+0000 7f5cfeffd640 1 --2- 172.21.15.42:0/2763326012 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5d0010cac0 0x7f5d001a9120 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f5cec005ab0 tx=0x7f5cec003d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:07:54.943 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:54.940+0000 7f5cfcff9640 1 -- 172.21.15.42:0/2763326012 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cec03f070 con 0x7f5d0010cac0 2024-08-30T09:07:54.943 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:54.940+0000 7f5cfcff9640 1 -- 172.21.15.42:0/2763326012 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5cec03b460 con 0x7f5d0010cac0 2024-08-30T09:07:54.943 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:54.940+0000 7f5cfcff9640 1 -- 172.21.15.42:0/2763326012 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cec035030 con 0x7f5d0010cac0 2024-08-30T09:07:54.943 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:54.941+0000 7f5d0499e640 1 -- 172.21.15.42:0/2763326012 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d001aa780 con 0x7f5d0010cac0 2024-08-30T09:07:54.943 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:54.941+0000 7f5d0499e640 1 -- 172.21.15.42:0/2763326012 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d001b0d70 con 0x7f5d0010cac0 2024-08-30T09:07:54.954 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:54.950+0000 7f5d0cc42740 1 Processor -- start 2024-08-30T09:07:54.955 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:54.954+0000 7f5cfcff9640 1 -- 172.21.15.42:0/2763326012 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7f5cec042070 con 0x7f5d0010cac0 2024-08-30T09:07:54.958 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:54.957+0000 7f5d0cc42740 1 -- start start 2024-08-30T09:07:54.963 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:54.961+0000 7f5d0cc42740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56033c96f1b0 0x56033cd15160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:07:54.963 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:54.961+0000 7f5d0cc42740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56033c9462f0 0x56033cd157d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:07:54.964 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:54.961+0000 7f5d0cc42740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56033c94e4d0 0x56033cd16320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:07:54.964 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:54.961+0000 7f5d0cc42740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x56033cc88000 con 0x56033c96f1b0 2024-08-30T09:07:54.964 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:54.961+0000 7f5d0cc42740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x56033cc77590 con 0x56033c9462f0 2024-08-30T09:07:54.964 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:54.961+0000 7f5d0cc42740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x56033cc7b540 con 0x56033c94e4d0 2024-08-30T09:07:54.964 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:54.961+0000 7f5cfcff9640 1 --2- 172.21.15.42:0/2763326012 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f5ce40b4d40 0x7f5ce40b7250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:07:54.964 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:54.961+0000 7f5cfeffd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56033c94e4d0 0x56033cd16320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:07:54.964 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:54.961+0000 7f5cfffff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56033c96f1b0 0x56033cd15160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:07:54.965 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:54.962+0000 7f5cfcff9640 1 -- 172.21.15.42:0/2763326012 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f5cec105bd0 con 0x7f5d0010cac0 2024-08-30T09:07:54.965 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:54.962+0000 7f5cfeffd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56033c94e4d0 0x56033cd16320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:37388/0 (socket says 172.21.15.42:37388) 2024-08-30T09:07:54.965 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:54.962+0000 7f5cfeffd640 1 -- 172.21.15.42:0/4285334314 learned_addr learned my addr 172.21.15.42:0/4285334314 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:07:54.965 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:54.963+0000 7f5cfeffd640 1 -- 172.21.15.42:0/4285334314 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56033c9462f0 msgr2=0x56033cd157d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-30T09:07:54.965 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:54.963+0000 7f5cfeffd640 1 --2- 172.21.15.42:0/4285334314 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56033c9462f0 0x56033cd157d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:54.965 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:54.963+0000 7f5cfeffd640 1 -- 172.21.15.42:0/4285334314 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56033c96f1b0 msgr2=0x56033cd15160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:07:54.965 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:54.963+0000 7f5cfeffd640 1 --2- 172.21.15.42:0/4285334314 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56033c96f1b0 0x56033cd15160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:54.965 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:54.963+0000 7f5cfeffd640 1 -- 172.21.15.42:0/4285334314 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56033c9da790 con 0x56033c94e4d0 2024-08-30T09:07:54.968 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:54.964+0000 7f5cfeffd640 1 --2- 172.21.15.42:0/4285334314 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56033c94e4d0 0x56033cd16320 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f5cec009a20 tx=0x7f5cec0cbd20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=19deab20ebefb6ba server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:07:54.969 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:54.965+0000 7f5cf5ffb640 1 -- 172.21.15.42:0/4285334314 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cec049070 con 0x56033c94e4d0 2024-08-30T09:07:54.969 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:54.965+0000 7f5cf5ffb640 1 -- 172.21.15.42:0/4285334314 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5cec0cc210 con 0x56033c94e4d0 2024-08-30T09:07:54.969 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:54.965+0000 7f5cf5ffb640 1 -- 172.21.15.42:0/4285334314 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cec03b8d0 con 0x56033c94e4d0 2024-08-30T09:07:54.969 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:54.965+0000 7f5cfffff640 1 --2- 172.21.15.42:0/2763326012 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f5ce40b4d40 0x7f5ce40b7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:07:54.969 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:54.967+0000 7f5d0cc42740 1 -- 172.21.15.42:0/4285334314 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56033c94e4d0 msgr2=0x56033cd16320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:07:54.969 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:54.967+0000 7f5d0cc42740 1 --2- 172.21.15.42:0/4285334314 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56033c94e4d0 0x56033cd16320 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f5cec009a20 tx=0x7f5cec0cbd20 comp rx=0 tx=0).stop 2024-08-30T09:07:54.975 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:54.970+0000 7f5cfffff640 1 --2- 172.21.15.42:0/2763326012 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f5ce40b4d40 0x7f5ce40b7250 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f5cf800acb0 tx=0x7f5cf8006c30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:07:54.975 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:54.971+0000 7f5d0cc42740 1 -- 172.21.15.42:0/4285334314 shutdown_connections 2024-08-30T09:07:54.976 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:54.971+0000 7f5d0cc42740 1 --2- 172.21.15.42:0/4285334314 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56033c94e4d0 0x56033cd16320 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:54.976 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:54.971+0000 7f5d0cc42740 1 --2- 172.21.15.42:0/4285334314 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56033c9462f0 0x56033cd157d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:54.976 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:54.971+0000 7f5d0cc42740 1 --2- 172.21.15.42:0/4285334314 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56033c96f1b0 0x56033cd15160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:54.976 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:54.971+0000 7f5d0cc42740 1 -- 172.21.15.42:0/4285334314 >> 172.21.15.42:0/4285334314 conn(0x56033c963300 msgr2=0x56033cd10bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:07:54.976 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:54.972+0000 7f5d0cc42740 1 -- 172.21.15.42:0/4285334314 shutdown_connections 2024-08-30T09:07:54.976 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:54.973+0000 7f5d0cc42740 1 -- 172.21.15.42:0/4285334314 wait complete. 2024-08-30T09:07:54.976 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:54.973+0000 7f5d0cc42740 1 -- start start 2024-08-30T09:07:54.976 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:54.974+0000 7f5d0cc42740 1 Processor -- start 2024-08-30T09:07:54.977 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:54.974+0000 7f5d0cc42740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56033c9462f0 0x56033cd1e960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:07:54.977 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:54.974+0000 7f5d0cc42740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56033c94e4d0 0x56033cd1f020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:07:54.977 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:54.974+0000 7f5d0cc42740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56033c96f1b0 0x56033cd1fb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:07:54.977 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:54.974+0000 7f5d0cc42740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x56033cbb0ab0 con 0x56033c96f1b0 2024-08-30T09:07:54.977 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:54.974+0000 7f5d0cc42740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x56033cbfe680 con 0x56033c94e4d0 2024-08-30T09:07:54.977 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:54.974+0000 7f5d0cc42740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x56033cd16c00 con 0x56033c9462f0 2024-08-30T09:07:54.977 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:54.974+0000 7f5d0cc42740 10 client.0 _ll_register_callbacks cb 0x56033c986a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:07:54.977 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:54.974+0000 7f5cff7fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56033c9462f0 0x56033cd1e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:07:54.977 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:54.974+0000 7f5cff7fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56033c9462f0 0x56033cd1e960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:37392/0 (socket says 172.21.15.42:37392) 2024-08-30T09:07:54.978 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:54.974+0000 7f5cff7fe640 1 -- 172.21.15.42:0/45055081 learned_addr learned my addr 172.21.15.42:0/45055081 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:07:54.978 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:54.975+0000 7f5cfffff640 1 --2- 172.21.15.42:0/45055081 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56033c96f1b0 0x56033cd1fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:07:54.978 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:54.975+0000 7f5cff7fe640 1 -- 172.21.15.42:0/45055081 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56033c94e4d0 msgr2=0x56033cd1f020 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-30T09:07:54.978 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:54.975+0000 7f5cff7fe640 1 --2- 172.21.15.42:0/45055081 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56033c94e4d0 0x56033cd1f020 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:54.978 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:54.975+0000 7f5cff7fe640 1 -- 172.21.15.42:0/45055081 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56033c96f1b0 msgr2=0x56033cd1fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:07:54.978 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:54.975+0000 7f5cff7fe640 1 --2- 172.21.15.42:0/45055081 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56033c96f1b0 0x56033cd1fb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:54.978 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:54.975+0000 7f5cff7fe640 1 -- 172.21.15.42:0/45055081 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56033ca53620 con 0x56033c9462f0 2024-08-30T09:07:54.979 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:54.975+0000 7f5cff7fe640 1 --2- 172.21.15.42:0/45055081 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56033c9462f0 0x56033cd1e960 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f5cf0009e00 tx=0x7f5cf000b4b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4481b543ad413ce1 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:07:54.979 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:54.976+0000 7f5ce2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-08-30T09:07:54.979 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:54.976+0000 7f5ce2ffd640 1 -- 172.21.15.42:0/45055081 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cf0015020 con 0x56033c9462f0 2024-08-30T09:07:54.979 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:54.976+0000 7f5ce2ffd640 1 -- 172.21.15.42:0/45055081 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f5cf0011620 con 0x56033c9462f0 2024-08-30T09:07:54.979 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:54.976+0000 7f5d0cc42740 1 -- 172.21.15.42:0/45055081 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5cec0cc210 con 0x56033c9462f0 2024-08-30T09:07:54.979 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:54.976+0000 7f5ce2ffd640 1 -- 172.21.15.42:0/45055081 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f5cf0011820 con 0x56033c9462f0 2024-08-30T09:07:54.981 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:54.979+0000 7f5d0cc42740 10 client.26392 fetch_fsmap learned FSMap version 34 2024-08-30T09:07:54.981 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:54.979+0000 7f5d0cc42740 1 -- 172.21.15.42:0/45055081 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x56033cd28b70 con 0x56033c9462f0 2024-08-30T09:07:54.982 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:54.980+0000 7f5ce2ffd640 1 -- 172.21.15.42:0/45055081 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5cec0cc210 con 0x56033c9462f0 2024-08-30T09:07:54.982 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:54.980+0000 7f5ce2ffd640 1 -- 172.21.15.42:0/45055081 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 34) ==== 8366+0+0 (secure 0 0 0) 0x7f5cf0022020 con 0x56033c9462f0 2024-08-30T09:07:54.983 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:54.981+0000 7f5d0cc42740 10 client.26392 fetch_fsmap finished waiting for FSMap version 34 2024-08-30T09:07:54.983 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:54.981+0000 7f5d0cc42740 10 client.26392 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-30T09:07:54.983 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:54.981+0000 7f5d0cc42740 20 client.26392 populate_metadata read hostname 'smithi042' 2024-08-30T09:07:54.983 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:54.981+0000 7f5d0cc42740 1 --2- 172.21.15.42:0/45055081 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x56033c92dd80 0x56033cd2c420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:07:54.983 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:54.981+0000 7f5d0cc42740 4 client.26392 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-30T09:07:54.983 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:54.981+0000 7f5d0cc42740 1 -- 172.21.15.42:0/45055081 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56033cc89f50 con 0x56033c92dd80 2024-08-30T09:07:54.984 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:54.982+0000 7f5cfeffd640 1 --2- 172.21.15.42:0/45055081 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x56033c92dd80 0x56033cd2c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:07:54.985 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:54.983+0000 7f5cfeffd640 1 --2- 172.21.15.42:0/45055081 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x56033c92dd80 0x56033cd2c420 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4499b0be79dfdf88 server_cookie=9d02616a9a8c5687 in_seq=0 out_seq=0 2024-08-30T09:07:54.985 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:54.983+0000 7f5ce2ffd640 10 client.26392 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:07:54.987 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:54.985+0000 7f5ce2ffd640 1 -- 172.21.15.42:0/45055081 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56033cc89f50 con 0x56033c92dd80 2024-08-30T09:07:54.987 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:54.985+0000 7f5ce2ffd640 10 client.26392 handle_command_reply: tid=0 2024-08-30T09:07:54.987 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:54.985+0000 7f5d0cc42740 1 client.26392 shutdown 2024-08-30T09:07:54.987 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:54.985+0000 7f5d0cc42740 1 -- 172.21.15.42:0/45055081 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56033c9462f0 msgr2=0x56033cd1e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:07:54.988 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:54.985+0000 7f5d0cc42740 1 --2- 172.21.15.42:0/45055081 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56033c9462f0 0x56033cd1e960 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f5cf0009e00 tx=0x7f5cf000b4b0 comp rx=0 tx=0).stop 2024-08-30T09:07:54.988 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:54.985+0000 7f5d0cc42740 1 -- 172.21.15.42:0/45055081 shutdown_connections 2024-08-30T09:07:54.988 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:54.985+0000 7f5d0cc42740 1 --2- 172.21.15.42:0/45055081 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x56033c92dd80 0x56033cd2c420 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:54.988 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:54.985+0000 7f5d0cc42740 1 --2- 172.21.15.42:0/45055081 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56033c96f1b0 0x56033cd1fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:54.988 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:54.985+0000 7f5d0cc42740 1 --2- 172.21.15.42:0/45055081 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56033c94e4d0 0x56033cd1f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:54.988 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:54.985+0000 7f5d0cc42740 1 --2- 172.21.15.42:0/45055081 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56033c9462f0 0x56033cd1e960 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:54.988 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:54.985+0000 7f5ce2ffd640 0 client.26392 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:07:54.988 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:54.985+0000 7f5d0cc42740 1 -- 172.21.15.42:0/45055081 >> 172.21.15.42:0/45055081 conn(0x56033c963300 msgr2=0x56033cd0f150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:07:54.988 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:54.985+0000 7f5d0cc42740 1 -- 172.21.15.42:0/45055081 shutdown_connections 2024-08-30T09:07:54.988 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:54.985+0000 7f5d0cc42740 1 -- 172.21.15.42:0/45055081 wait complete. 2024-08-30T09:07:54.988 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:54.985+0000 7f5d0cc42740 20 client.26392 trim_cache size 0 max 16384 2024-08-30T09:07:55.014 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:55.012+0000 7f5d0cc42740 1 Processor -- start 2024-08-30T09:07:55.014 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:55.012+0000 7f5d0cc42740 1 -- start start 2024-08-30T09:07:55.015 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:55.013+0000 7f5d0cc42740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56033c9462f0 0x56033cd10c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:07:55.015 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:55.013+0000 7f5d0cc42740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56033c94e4d0 0x56033cd26a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:07:55.015 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:55.013+0000 7f5d0cc42740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56033c96f1b0 0x56033cd0f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:07:55.015 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:55.013+0000 7f5cfeffd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56033c94e4d0 0x56033cd26a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:07:55.016 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:55.013+0000 7f5cfeffd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56033c94e4d0 0x56033cd26a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:37400/0 (socket says 172.21.15.42:37400) 2024-08-30T09:07:55.016 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:55.013+0000 7f5cfeffd640 1 -- 172.21.15.42:0/3567933472 learned_addr learned my addr 172.21.15.42:0/3567933472 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:07:55.016 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:55.013+0000 7f5d0cc42740 1 -- 172.21.15.42:0/3567933472 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f5ce8012540 con 0x56033c96f1b0 2024-08-30T09:07:55.016 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:55.013+0000 7f5d0cc42740 1 -- 172.21.15.42:0/3567933472 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f5ce800fdb0 con 0x56033c9462f0 2024-08-30T09:07:55.016 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:55.013+0000 7f5d0cc42740 1 -- 172.21.15.42:0/3567933472 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f5ce800dc00 con 0x56033c94e4d0 2024-08-30T09:07:55.026 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:55.019+0000 7f5cfeffd640 1 -- 172.21.15.42:0/3567933472 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56033c9462f0 msgr2=0x56033cd10c40 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-30T09:07:55.026 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:55.019+0000 7f5cfeffd640 1 --2- 172.21.15.42:0/3567933472 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56033c9462f0 0x56033cd10c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:55.026 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:55.019+0000 7f5cfeffd640 1 -- 172.21.15.42:0/3567933472 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56033c96f1b0 msgr2=0x56033cd0f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:07:55.027 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:55.016+0000 7f5cfffff640 1 --2- 172.21.15.42:0/3567933472 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56033c96f1b0 0x56033cd0f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:07:55.034 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:55.027+0000 7f5cfeffd640 1 --2- 172.21.15.42:0/3567933472 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56033c96f1b0 0x56033cd0f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:55.034 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:55.027+0000 7f5cfeffd640 1 -- 172.21.15.42:0/3567933472 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56033cd16720 con 0x56033c94e4d0 2024-08-30T09:07:55.038 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:55.033+0000 7f5cfffff640 1 --2- 172.21.15.42:0/3567933472 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56033c96f1b0 0x56033cd0f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-30T09:07:55.038 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:55.033+0000 7f5cfeffd640 1 --2- 172.21.15.42:0/3567933472 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56033c94e4d0 0x56033cd26a00 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f5cec004110 tx=0x7f5cec03b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ad78a68c621cc93e server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:07:55.042 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:55.038+0000 7f5cf57fa640 1 -- 172.21.15.42:0/3567933472 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cec049040 con 0x56033c94e4d0 2024-08-30T09:07:55.042 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:55.038+0000 7f5cf57fa640 1 -- 172.21.15.42:0/3567933472 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5cec0cc210 con 0x56033c94e4d0 2024-08-30T09:07:55.042 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:55.038+0000 7f5cf57fa640 1 -- 172.21.15.42:0/3567933472 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cec03ba50 con 0x56033c94e4d0 2024-08-30T09:07:55.045 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:55.040+0000 7f5d0cc42740 1 -- 172.21.15.42:0/3567933472 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56033c94e4d0 msgr2=0x56033cd26a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:07:55.046 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:55.040+0000 7f5d0cc42740 1 --2- 172.21.15.42:0/3567933472 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56033c94e4d0 0x56033cd26a00 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f5cec004110 tx=0x7f5cec03b840 comp rx=0 tx=0).stop 2024-08-30T09:07:55.048 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:55.046+0000 7f5d0cc42740 1 -- 172.21.15.42:0/3567933472 shutdown_connections 2024-08-30T09:07:55.049 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:55.046+0000 7f5d0cc42740 1 --2- 172.21.15.42:0/3567933472 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56033c94e4d0 0x56033cd26a00 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:55.049 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:55.046+0000 7f5d0cc42740 1 --2- 172.21.15.42:0/3567933472 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56033c96f1b0 0x56033cd0f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:55.049 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:55.046+0000 7f5d0cc42740 1 --2- 172.21.15.42:0/3567933472 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56033c9462f0 0x56033cd10c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:55.049 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:55.046+0000 7f5d0cc42740 1 -- 172.21.15.42:0/3567933472 >> 172.21.15.42:0/3567933472 conn(0x56033c963300 msgr2=0x56033cd10530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:07:55.057 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:55.053+0000 7f5d0cc42740 1 -- 172.21.15.42:0/3567933472 shutdown_connections 2024-08-30T09:07:55.058 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:55.057+0000 7f5d0cc42740 1 -- 172.21.15.42:0/3567933472 wait complete. 2024-08-30T09:07:55.060 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:55.057+0000 7f5d0cc42740 1 -- start start 2024-08-30T09:07:55.060 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:55.057+0000 7f5d0cc42740 1 Processor -- start 2024-08-30T09:07:55.063 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:55.061+0000 7f5d0cc42740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56033c9462f0 0x56033cd26880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:07:55.063 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:55.061+0000 7f5d0cc42740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56033c94e4d0 0x56033cd25540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:07:55.063 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:55.061+0000 7f5d0cc42740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56033c96f1b0 0x56033cd25c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:07:55.063 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:55.061+0000 7f5d0cc42740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f5ce800e270 con 0x56033c94e4d0 2024-08-30T09:07:55.063 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:55.061+0000 7f5d0cc42740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f5ce800f6e0 con 0x56033c96f1b0 2024-08-30T09:07:55.063 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:55.061+0000 7f5d0cc42740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f5ce800e940 con 0x56033c9462f0 2024-08-30T09:07:55.063 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:55.061+0000 7f5d0cc42740 10 client.0 _ll_register_callbacks cb 0x7f5cf0006eb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:07:55.064 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:55.061+0000 7f5cff7fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56033c9462f0 0x56033cd26880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:07:55.064 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:55.061+0000 7f5cff7fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56033c9462f0 0x56033cd26880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:37404/0 (socket says 172.21.15.42:37404) 2024-08-30T09:07:55.064 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:55.061+0000 7f5cff7fe640 1 -- 172.21.15.42:0/3417330262 learned_addr learned my addr 172.21.15.42:0/3417330262 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:07:55.064 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:55.062+0000 7f5cfffff640 1 --2- 172.21.15.42:0/3417330262 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56033c96f1b0 0x56033cd25c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:07:55.064 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:55.062+0000 7f5cff7fe640 1 -- 172.21.15.42:0/3417330262 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56033c96f1b0 msgr2=0x56033cd25c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:07:55.064 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:55.062+0000 7f5cff7fe640 1 --2- 172.21.15.42:0/3417330262 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56033c96f1b0 0x56033cd25c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:55.065 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:55.062+0000 7f5cff7fe640 1 -- 172.21.15.42:0/3417330262 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56033c94e4d0 msgr2=0x56033cd25540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:07:55.065 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:55.062+0000 7f5cff7fe640 1 --2- 172.21.15.42:0/3417330262 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56033c94e4d0 0x56033cd25540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:55.065 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:55.062+0000 7f5cff7fe640 1 -- 172.21.15.42:0/3417330262 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56033cd24150 con 0x56033c9462f0 2024-08-30T09:07:55.065 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:55.063+0000 7f5cff7fe640 1 --2- 172.21.15.42:0/3417330262 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56033c9462f0 0x56033cd26880 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f5cf001ae80 tx=0x7f5cf00195b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6253d81cd3d4bad7 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:07:55.068 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:55.064+0000 7f5ce2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-08-30T09:07:55.068 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:55.064+0000 7f5ce2ffd640 1 -- 172.21.15.42:0/3417330262 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cf0027070 con 0x56033c9462f0 2024-08-30T09:07:55.068 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:55.064+0000 7f5ce2ffd640 1 -- 172.21.15.42:0/3417330262 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f5cf0011bc0 con 0x56033c9462f0 2024-08-30T09:07:55.068 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:55.064+0000 7f5d0cc42740 1 -- 172.21.15.42:0/3417330262 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5cec0cc210 con 0x56033c9462f0 2024-08-30T09:07:55.069 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:55.068+0000 7f5ce2ffd640 1 -- 172.21.15.42:0/3417330262 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f5cf0015040 con 0x56033c9462f0 2024-08-30T09:07:55.073 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:55.071+0000 7f5d0cc42740 10 client.26398 fetch_fsmap learned FSMap version 34 2024-08-30T09:07:55.074 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:55.071+0000 7f5d0cc42740 1 -- 172.21.15.42:0/3417330262 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x56033cd96f00 con 0x56033c9462f0 2024-08-30T09:07:55.074 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:55.072+0000 7f5ce2ffd640 1 -- 172.21.15.42:0/3417330262 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5cec0cc210 con 0x56033c9462f0 2024-08-30T09:07:55.074 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:55.072+0000 7f5ce2ffd640 1 -- 172.21.15.42:0/3417330262 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 34) ==== 8366+0+0 (secure 0 0 0) 0x7f5cf0022030 con 0x56033c9462f0 2024-08-30T09:07:55.074 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:55.072+0000 7f5d0cc42740 10 client.26398 fetch_fsmap finished waiting for FSMap version 34 2024-08-30T09:07:55.075 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:55.072+0000 7f5d0cc42740 10 client.26398 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-30T09:07:55.075 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:55.072+0000 7f5d0cc42740 20 client.26398 populate_metadata read hostname 'smithi042' 2024-08-30T09:07:55.075 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:55.072+0000 7f5d0cc42740 1 --2- 172.21.15.42:0/3417330262 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x56033c9031c0 0x56033cd99930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:07:55.075 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:55.072+0000 7f5d0cc42740 4 client.26398 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-30T09:07:55.075 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:55.072+0000 7f5d0cc42740 1 -- 172.21.15.42:0/3417330262 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56033cc859f0 con 0x56033c9031c0 2024-08-30T09:07:55.075 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:55.073+0000 7f5cfeffd640 1 --2- 172.21.15.42:0/3417330262 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x56033c9031c0 0x56033cd99930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:07:55.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:07:54 smithi167 ceph-mon[22063]: pgmap v587: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 6.0 MiB/s wr, 584 op/s 2024-08-30T09:07:55.078 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:55.074+0000 7f5cfeffd640 1 --2- 172.21.15.42:0/3417330262 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x56033c9031c0 0x56033cd99930 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f669da3965485004 server_cookie=3d1b6b2931c3f87a in_seq=0 out_seq=0 2024-08-30T09:07:55.078 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:55.074+0000 7f5ce2ffd640 10 client.26398 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:07:55.100 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:55.098+0000 7f5ce2ffd640 1 -- 172.21.15.42:0/3417330262 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56033cc859f0 con 0x56033c9031c0 2024-08-30T09:07:55.101 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:55.099+0000 7f5ce2ffd640 10 client.26398 handle_command_reply: tid=0 2024-08-30T09:07:55.101 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:55.099+0000 7f5d0cc42740 1 client.26398 shutdown 2024-08-30T09:07:55.101 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:55.099+0000 7f5d0cc42740 1 -- 172.21.15.42:0/3417330262 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56033c9462f0 msgr2=0x56033cd26880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:07:55.101 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:55.099+0000 7f5d0cc42740 1 --2- 172.21.15.42:0/3417330262 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56033c9462f0 0x56033cd26880 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f5cf001ae80 tx=0x7f5cf00195b0 comp rx=0 tx=0).stop 2024-08-30T09:07:55.105 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:55.103+0000 7f5d0cc42740 1 -- 172.21.15.42:0/3417330262 shutdown_connections 2024-08-30T09:07:55.105 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:55.103+0000 7f5d0cc42740 1 --2- 172.21.15.42:0/3417330262 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x56033c9031c0 0x56033cd99930 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:55.105 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:55.103+0000 7f5d0cc42740 1 --2- 172.21.15.42:0/3417330262 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56033c96f1b0 0x56033cd25c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:55.105 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:55.103+0000 7f5d0cc42740 1 --2- 172.21.15.42:0/3417330262 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56033c94e4d0 0x56033cd25540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:55.105 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:55.103+0000 7f5d0cc42740 1 --2- 172.21.15.42:0/3417330262 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56033c9462f0 0x56033cd26880 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:55.105 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:55.103+0000 7f5d0cc42740 1 -- 172.21.15.42:0/3417330262 >> 172.21.15.42:0/3417330262 conn(0x56033c963300 msgr2=0x56033cd0f150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:07:55.105 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:55.103+0000 7f5ce2ffd640 0 client.26398 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:07:55.105 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:55.104+0000 7f5d0cc42740 1 -- 172.21.15.42:0/3417330262 shutdown_connections 2024-08-30T09:07:55.106 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:55.104+0000 7f5d0cc42740 1 -- 172.21.15.42:0/3417330262 wait complete. 2024-08-30T09:07:55.106 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:55.104+0000 7f5d0cc42740 20 client.26398 trim_cache size 0 max 16384 2024-08-30T09:07:55.107 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-08-30T09:07:55.107 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2024-08-30T09:07:55.107 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-08-30T09:07:55.107 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-08-30T09:07:55.107 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:55.105+0000 7f5d0499e640 1 -- 172.21.15.42:0/2763326012 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f5ce40b4d40 msgr2=0x7f5ce40b7250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:07:55.107 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:55.105+0000 7f5d0499e640 1 --2- 172.21.15.42:0/2763326012 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f5ce40b4d40 0x7f5ce40b7250 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f5cf800acb0 tx=0x7f5cf8006c30 comp rx=0 tx=0).stop 2024-08-30T09:07:55.107 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:55.105+0000 7f5d0499e640 1 -- 172.21.15.42:0/2763326012 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5d0010cac0 msgr2=0x7f5d001a9120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:07:55.108 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:55.105+0000 7f5d0499e640 1 --2- 172.21.15.42:0/2763326012 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5d0010cac0 0x7f5d001a9120 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f5cec005ab0 tx=0x7f5cec003d80 comp rx=0 tx=0).stop 2024-08-30T09:07:55.108 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:55.106+0000 7f5d0499e640 1 -- 172.21.15.42:0/2763326012 shutdown_connections 2024-08-30T09:07:55.108 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:55.106+0000 7f5d0499e640 1 --2- 172.21.15.42:0/2763326012 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f5ce40b4d40 0x7f5ce40b7250 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:55.108 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:55.106+0000 7f5d0499e640 1 --2- 172.21.15.42:0/2763326012 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5d0010f5d0 0x7f5d001a9cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:55.108 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:55.106+0000 7f5d0499e640 1 --2- 172.21.15.42:0/2763326012 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5d0010cac0 0x7f5d001a9120 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:55.109 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:55.106+0000 7f5d0499e640 1 --2- 172.21.15.42:0/2763326012 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5d0010a160 0x7f5d001a8ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:07:55.109 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:55.106+0000 7f5d0499e640 1 -- 172.21.15.42:0/2763326012 >> 172.21.15.42:0/2763326012 conn(0x7f5d000703e0 msgr2=0x7f5d0010afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:07:55.109 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:55.106+0000 7f5d0499e640 1 -- 172.21.15.42:0/2763326012 shutdown_connections 2024-08-30T09:07:55.109 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:07:55.106+0000 7f5d0499e640 1 -- 172.21.15.42:0/2763326012 wait complete. 2024-08-30T09:07:55.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:07:54 smithi042 ceph-mon[18652]: pgmap v587: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 6.0 MiB/s wr, 584 op/s 2024-08-30T09:07:55.953 INFO:teuthology.orchestra.run.smithi042.stderr:progress: 430.0 s, 715.0 tps, lat 139.373 ms stddev 179.408 2024-08-30T09:07:56.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:07:56 smithi190 ceph-mon[22001]: pgmap v588: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.3 MiB/s wr, 493 op/s 2024-08-30T09:07:57.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:07:56 smithi167 ceph-mon[22063]: pgmap v588: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.3 MiB/s wr, 493 op/s 2024-08-30T09:07:57.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:07:56 smithi042 ceph-mon[18652]: pgmap v588: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.3 MiB/s wr, 493 op/s 2024-08-30T09:07:58.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:07:58 smithi190 ceph-mon[22001]: pgmap v589: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.3 MiB/s wr, 493 op/s 2024-08-30T09:07:59.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:07:58 smithi167 ceph-mon[22063]: pgmap v589: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.3 MiB/s wr, 493 op/s 2024-08-30T09:07:59.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:07:58 smithi042 ceph-mon[18652]: pgmap v589: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.3 MiB/s wr, 493 op/s 2024-08-30T09:08:00.935 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:08:00 smithi042 ceph-mon[18652]: pgmap v590: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 6.7 MiB/s wr, 625 op/s 2024-08-30T09:08:00.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:08:00 smithi190 ceph-mon[22001]: pgmap v590: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 6.7 MiB/s wr, 625 op/s 2024-08-30T09:08:00.953 INFO:teuthology.orchestra.run.smithi042.stderr:progress: 435.0 s, 589.4 tps, lat 167.872 ms stddev 174.834 2024-08-30T09:08:01.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:08:00 smithi167 ceph-mon[22063]: pgmap v590: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 6.7 MiB/s wr, 625 op/s 2024-08-30T09:08:02.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:08:02 smithi190 ceph-mon[22001]: pgmap v591: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 4.9 MiB/s wr, 489 op/s 2024-08-30T09:08:03.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:08:02 smithi167 ceph-mon[22063]: pgmap v591: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 4.9 MiB/s wr, 489 op/s 2024-08-30T09:08:03.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:08:02 smithi042 ceph-mon[18652]: pgmap v591: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 4.9 MiB/s wr, 489 op/s 2024-08-30T09:08:05.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:08:04 smithi167 ceph-mon[22063]: pgmap v592: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 7.2 MiB/s wr, 686 op/s 2024-08-30T09:08:05.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:08:04 smithi042 ceph-mon[18652]: pgmap v592: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 7.2 MiB/s wr, 686 op/s 2024-08-30T09:08:05.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:08:04 smithi190 ceph-mon[22001]: pgmap v592: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 7.2 MiB/s wr, 686 op/s 2024-08-30T09:08:05.953 INFO:teuthology.orchestra.run.smithi042.stderr:progress: 440.0 s, 442.8 tps, lat 222.528 ms stddev 289.319 2024-08-30T09:08:07.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:08:06 smithi167 ceph-mon[22063]: pgmap v593: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.5 MiB/s wr, 634 op/s 2024-08-30T09:08:07.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:08:06 smithi042 ceph-mon[18652]: pgmap v593: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.5 MiB/s wr, 634 op/s 2024-08-30T09:08:07.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:08:06 smithi190 ceph-mon[22001]: pgmap v593: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.5 MiB/s wr, 634 op/s 2024-08-30T09:08:08.290 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.284+0000 7f60e7169640 1 -- 172.21.15.42:0/1270757331 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f60e010cb50 msgr2=0x7f60e010f030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:08:08.290 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.284+0000 7f60e7169640 1 --2- 172.21.15.42:0/1270757331 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f60e010cb50 0x7f60e010f030 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f60d0009860 tx=0x7f60d002a360 comp rx=0 tx=0).stop 2024-08-30T09:08:08.290 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.286+0000 7f60e7169640 1 -- 172.21.15.42:0/1270757331 shutdown_connections 2024-08-30T09:08:08.291 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.286+0000 7f60e7169640 1 --2- 172.21.15.42:0/1270757331 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f60e010f770 0x7f60e011a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:08.291 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.286+0000 7f60e7169640 1 --2- 172.21.15.42:0/1270757331 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f60e010cb50 0x7f60e010f030 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:08.291 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.286+0000 7f60e7169640 1 --2- 172.21.15.42:0/1270757331 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f60e010a160 0x7f60e010c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:08.291 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.287+0000 7f60e7169640 1 -- 172.21.15.42:0/1270757331 >> 172.21.15.42:0/1270757331 conn(0x7f60e00705b0 msgr2=0x7f60e0072a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:08:08.293 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.290+0000 7f60e7169640 1 -- 172.21.15.42:0/1270757331 shutdown_connections 2024-08-30T09:08:08.296 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.294+0000 7f60e7169640 1 -- 172.21.15.42:0/1270757331 wait complete. 2024-08-30T09:08:08.301 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.294+0000 7f60e7169640 1 Processor -- start 2024-08-30T09:08:08.302 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.295+0000 7f60e7169640 1 -- start start 2024-08-30T09:08:08.302 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.295+0000 7f60e7169640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f60e010a160 0x7f60e01176c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:08:08.302 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.295+0000 7f60e7169640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f60e010cb50 0x7f60e0117d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:08:08.302 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.295+0000 7f60e7169640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f60e010f770 0x7f60e01188e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:08:08.302 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.295+0000 7f60e7169640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f60e011df50 con 0x7f60e010cb50 2024-08-30T09:08:08.302 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.295+0000 7f60e7169640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f60e011ddd0 con 0x7f60e010a160 2024-08-30T09:08:08.302 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.295+0000 7f60e7169640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f60e011e0d0 con 0x7f60e010f770 2024-08-30T09:08:08.302 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.295+0000 7f60e4f05640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f60e010a160 0x7f60e01176c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:08:08.302 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.295+0000 7f60e4f05640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f60e010a160 0x7f60e01176c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:59082/0 (socket says 172.21.15.42:59082) 2024-08-30T09:08:08.302 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.295+0000 7f60e4f05640 1 -- 172.21.15.42:0/2122408680 learned_addr learned my addr 172.21.15.42:0/2122408680 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:08:08.302 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.296+0000 7f60e4f05640 1 -- 172.21.15.42:0/2122408680 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f60e010f770 msgr2=0x7f60e01188e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-30T09:08:08.302 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.296+0000 7f60e4f05640 1 --2- 172.21.15.42:0/2122408680 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f60e010f770 0x7f60e01188e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:08.303 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.296+0000 7f60e4f05640 1 -- 172.21.15.42:0/2122408680 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f60e010cb50 msgr2=0x7f60e0117d30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-30T09:08:08.303 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.296+0000 7f60e4f05640 1 --2- 172.21.15.42:0/2122408680 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f60e010cb50 0x7f60e0117d30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:08.303 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.296+0000 7f60e4f05640 1 -- 172.21.15.42:0/2122408680 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60e0119100 con 0x7f60e010a160 2024-08-30T09:08:08.303 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.296+0000 7f60e4f05640 1 --2- 172.21.15.42:0/2122408680 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f60e010a160 0x7f60e01176c0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f60d400b450 tx=0x7f60d400b910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:08:08.303 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.297+0000 7f60ddffb640 1 -- 172.21.15.42:0/2122408680 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60d4015020 con 0x7f60e010a160 2024-08-30T09:08:08.303 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.297+0000 7f60e7169640 1 -- 172.21.15.42:0/2122408680 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60e01193f0 con 0x7f60e010a160 2024-08-30T09:08:08.303 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.297+0000 7f60e7169640 1 -- 172.21.15.42:0/2122408680 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60e0119a90 con 0x7f60e010a160 2024-08-30T09:08:08.303 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.297+0000 7f60ddffb640 1 -- 172.21.15.42:0/2122408680 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f60d4011d40 con 0x7f60e010a160 2024-08-30T09:08:08.303 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.297+0000 7f60ddffb640 1 -- 172.21.15.42:0/2122408680 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60d400f340 con 0x7f60e010a160 2024-08-30T09:08:08.306 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.301+0000 7f60ddffb640 1 -- 172.21.15.42:0/2122408680 <== mon.2 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7f60d4018020 con 0x7f60e010a160 2024-08-30T09:08:08.306 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.302+0000 7f60ddffb640 1 --2- 172.21.15.42:0/2122408680 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f60c80d6cd0 0x7f60c80d91e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:08:08.308 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.303+0000 7f60ddffb640 1 -- 172.21.15.42:0/2122408680 <== mon.2 v2:172.21.15.167:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f60d40e5e40 con 0x7f60e010a160 2024-08-30T09:08:08.308 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.303+0000 7f60ee0d7740 1 Processor -- start 2024-08-30T09:08:08.308 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.303+0000 7f60ee0d7740 1 -- start start 2024-08-30T09:08:08.308 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.303+0000 7f60ee0d7740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e87eba21b0 0x55e87ef47f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:08:08.308 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.303+0000 7f60ee0d7740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e87eb792f0 0x55e87ef485d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:08:08.308 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.303+0000 7f60ee0d7740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e87eb814d0 0x55e87ef49120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:08:08.308 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.303+0000 7f60ee0d7740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55e87eebb000 con 0x55e87eb814d0 2024-08-30T09:08:08.308 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.303+0000 7f60ee0d7740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55e87eeaa590 con 0x55e87eb792f0 2024-08-30T09:08:08.308 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.303+0000 7f60ee0d7740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55e87eeae540 con 0x55e87eba21b0 2024-08-30T09:08:08.308 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.303+0000 7f60e4f05640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e87eba21b0 0x55e87ef47f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:08:08.308 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.304+0000 7f60e4f05640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e87eba21b0 0x55e87ef47f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:41988/0 (socket says 172.21.15.42:41988) 2024-08-30T09:08:08.309 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.304+0000 7f60e4f05640 1 -- 172.21.15.42:0/1704025620 learned_addr learned my addr 172.21.15.42:0/1704025620 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:08:08.309 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.304+0000 7f60dffff640 1 --2- 172.21.15.42:0/2122408680 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f60c80d6cd0 0x7f60c80d91e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:08:08.311 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.305+0000 7f60e4f05640 1 -- 172.21.15.42:0/1704025620 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e87eb792f0 msgr2=0x55e87ef485d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-30T09:08:08.311 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.305+0000 7f60e4f05640 1 --2- 172.21.15.42:0/1704025620 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e87eb792f0 0x55e87ef485d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:08.312 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.305+0000 7f60e4f05640 1 -- 172.21.15.42:0/1704025620 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e87eb814d0 msgr2=0x55e87ef49120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:08:08.312 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.305+0000 7f60e4f05640 1 --2- 172.21.15.42:0/1704025620 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e87eb814d0 0x55e87ef49120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:08.312 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.305+0000 7f60e4f05640 1 -- 172.21.15.42:0/1704025620 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e87ec0d790 con 0x55e87eba21b0 2024-08-30T09:08:08.312 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.308+0000 7f60e4f05640 1 --2- 172.21.15.42:0/1704025620 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e87eba21b0 0x55e87ef47f60 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f60d40ac470 tx=0x7f60d40ac3e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=76de79cdb4a561b9 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:08:08.319 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.313+0000 7f60c6ffd640 1 -- 172.21.15.42:0/1704025620 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60d40b5150 con 0x55e87eba21b0 2024-08-30T09:08:08.319 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.313+0000 7f60c6ffd640 1 -- 172.21.15.42:0/1704025620 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f60d40b0d50 con 0x55e87eba21b0 2024-08-30T09:08:08.319 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.313+0000 7f60c6ffd640 1 -- 172.21.15.42:0/1704025620 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60d40ab740 con 0x55e87eba21b0 2024-08-30T09:08:08.319 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.316+0000 7f60ee0d7740 1 -- 172.21.15.42:0/1704025620 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e87eba21b0 msgr2=0x55e87ef47f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:08:08.319 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.316+0000 7f60ee0d7740 1 --2- 172.21.15.42:0/1704025620 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e87eba21b0 0x55e87ef47f60 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f60d40ac470 tx=0x7f60d40ac3e0 comp rx=0 tx=0).stop 2024-08-30T09:08:08.319 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.316+0000 7f60ee0d7740 1 -- 172.21.15.42:0/1704025620 shutdown_connections 2024-08-30T09:08:08.319 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.316+0000 7f60ee0d7740 1 --2- 172.21.15.42:0/1704025620 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e87eb814d0 0x55e87ef49120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:08.319 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.316+0000 7f60ee0d7740 1 --2- 172.21.15.42:0/1704025620 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e87eb792f0 0x55e87ef485d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:08.319 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.316+0000 7f60ee0d7740 1 --2- 172.21.15.42:0/1704025620 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e87eba21b0 0x55e87ef47f60 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:08.320 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.316+0000 7f60ee0d7740 1 -- 172.21.15.42:0/1704025620 >> 172.21.15.42:0/1704025620 conn(0x55e87eb96300 msgr2=0x55e87ef439f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:08:08.322 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.317+0000 7f60dffff640 1 --2- 172.21.15.42:0/2122408680 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f60c80d6cd0 0x7f60c80d91e0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f60d00095f0 tx=0x7f60d0000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:08:08.332 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.326+0000 7f60ee0d7740 1 -- 172.21.15.42:0/1704025620 shutdown_connections 2024-08-30T09:08:08.332 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.329+0000 7f60ee0d7740 1 -- 172.21.15.42:0/1704025620 wait complete. 2024-08-30T09:08:08.335 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.330+0000 7f60ee0d7740 1 -- start start 2024-08-30T09:08:08.335 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.330+0000 7f60ee0d7740 1 Processor -- start 2024-08-30T09:08:08.339 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.336+0000 7f60ee0d7740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e87eb792f0 0x55e87ef73b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:08:08.339 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.336+0000 7f60ee0d7740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e87eb814d0 0x55e87ef741c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:08:08.339 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.336+0000 7f60ee0d7740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e87eba21b0 0x55e87ef74d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:08:08.339 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.336+0000 7f60ee0d7740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55e87ede3ab0 con 0x55e87eb814d0 2024-08-30T09:08:08.339 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.336+0000 7f60ee0d7740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55e87ee31680 con 0x55e87eb792f0 2024-08-30T09:08:08.339 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.336+0000 7f60ee0d7740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55e87ef49c40 con 0x55e87eba21b0 2024-08-30T09:08:08.339 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.336+0000 7f60ee0d7740 10 client.0 _ll_register_callbacks cb 0x55e87ebb9a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:08:08.342 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.339+0000 7f60dffff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e87eb814d0 0x55e87ef741c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:08:08.342 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.339+0000 7f60dffff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e87eb814d0 0x55e87ef741c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38030/0 (socket says 172.21.15.42:38030) 2024-08-30T09:08:08.342 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.339+0000 7f60dffff640 1 -- 172.21.15.42:0/1826788417 learned_addr learned my addr 172.21.15.42:0/1826788417 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:08:08.342 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.340+0000 7f60e4f05640 1 --2- 172.21.15.42:0/1826788417 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e87eb792f0 0x55e87ef73b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:08:08.348 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.340+0000 7f60e5706640 1 --2- 172.21.15.42:0/1826788417 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e87eba21b0 0x55e87ef74d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:08:08.348 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.340+0000 7f60dffff640 1 -- 172.21.15.42:0/1826788417 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e87eba21b0 msgr2=0x55e87ef74d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:08:08.348 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.340+0000 7f60dffff640 1 --2- 172.21.15.42:0/1826788417 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e87eba21b0 0x55e87ef74d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:08.348 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.340+0000 7f60dffff640 1 -- 172.21.15.42:0/1826788417 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e87eb792f0 msgr2=0x55e87ef73b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:08:08.348 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.340+0000 7f60dffff640 1 --2- 172.21.15.42:0/1826788417 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e87eb792f0 0x55e87ef73b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:08.349 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.340+0000 7f60dffff640 1 -- 172.21.15.42:0/1826788417 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e87ec86620 con 0x55e87eb814d0 2024-08-30T09:08:08.349 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.341+0000 7f60dffff640 1 --2- 172.21.15.42:0/1826788417 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e87eb814d0 0x55e87ef741c0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f60d0034000 tx=0x7f60d0030340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6a167f522ddd877 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:08:08.349 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.341+0000 7f60bffff640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-08-30T09:08:08.349 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.341+0000 7f60bffff640 1 -- 172.21.15.42:0/1826788417 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60d004e060 con 0x55e87eb814d0 2024-08-30T09:08:08.349 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.341+0000 7f60ee0d7740 1 -- 172.21.15.42:0/1826788417 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f60d40b0d50 con 0x55e87eb814d0 2024-08-30T09:08:08.349 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.341+0000 7f60bffff640 1 -- 172.21.15.42:0/1826788417 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f60d0003800 con 0x55e87eb814d0 2024-08-30T09:08:08.349 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.342+0000 7f60bffff640 1 -- 172.21.15.42:0/1826788417 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f60d0003a00 con 0x55e87eb814d0 2024-08-30T09:08:08.349 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.342+0000 7f60ee0d7740 10 client.16665 fetch_fsmap learned FSMap version 34 2024-08-30T09:08:08.349 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.342+0000 7f60ee0d7740 1 -- 172.21.15.42:0/1826788417 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55e87ef7dd70 con 0x55e87eb814d0 2024-08-30T09:08:08.349 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.343+0000 7f60bffff640 1 -- 172.21.15.42:0/1826788417 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f60d40b0d50 con 0x55e87eb814d0 2024-08-30T09:08:08.349 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.343+0000 7f60bffff640 1 -- 172.21.15.42:0/1826788417 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 34) ==== 8366+0+0 (secure 0 0 0) 0x7f60d003e430 con 0x55e87eb814d0 2024-08-30T09:08:08.349 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.344+0000 7f60ee0d7740 10 client.16665 fetch_fsmap finished waiting for FSMap version 34 2024-08-30T09:08:08.349 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.344+0000 7f60ee0d7740 10 client.16665 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-30T09:08:08.350 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.344+0000 7f60ee0d7740 20 client.16665 populate_metadata read hostname 'smithi042' 2024-08-30T09:08:08.350 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.344+0000 7f60ee0d7740 1 --2- 172.21.15.42:0/1826788417 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55e87eb60d80 0x55e87ef81540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:08:08.350 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.344+0000 7f60ee0d7740 4 client.16665 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-30T09:08:08.350 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.344+0000 7f60ee0d7740 1 -- 172.21.15.42:0/1826788417 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e87eebcf50 con 0x55e87eb60d80 2024-08-30T09:08:08.350 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.348+0000 7f60e4f05640 1 --2- 172.21.15.42:0/1826788417 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55e87eb60d80 0x55e87ef81540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:08:08.358 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.355+0000 7f60e4f05640 1 --2- 172.21.15.42:0/1826788417 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55e87eb60d80 0x55e87ef81540 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=184ac3acc7be7194 server_cookie=31cc251cdc0147c3 in_seq=0 out_seq=0 2024-08-30T09:08:08.358 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.355+0000 7f60bffff640 10 client.16665 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:08:08.358 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.355+0000 7f60bffff640 1 -- 172.21.15.42:0/1826788417 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55e87eebcf50 con 0x55e87eb60d80 2024-08-30T09:08:08.358 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.355+0000 7f60bffff640 10 client.16665 handle_command_reply: tid=0 2024-08-30T09:08:08.358 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.356+0000 7f60ee0d7740 1 client.16665 shutdown 2024-08-30T09:08:08.360 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.357+0000 7f60ee0d7740 1 -- 172.21.15.42:0/1826788417 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e87eb814d0 msgr2=0x55e87ef741c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:08:08.360 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.357+0000 7f60ee0d7740 1 --2- 172.21.15.42:0/1826788417 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e87eb814d0 0x55e87ef741c0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f60d0034000 tx=0x7f60d0030340 comp rx=0 tx=0).stop 2024-08-30T09:08:08.360 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.357+0000 7f60ee0d7740 1 -- 172.21.15.42:0/1826788417 shutdown_connections 2024-08-30T09:08:08.360 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.357+0000 7f60ee0d7740 1 --2- 172.21.15.42:0/1826788417 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55e87eb60d80 0x55e87ef81540 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:08.360 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.357+0000 7f60ee0d7740 1 --2- 172.21.15.42:0/1826788417 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e87eba21b0 0x55e87ef74d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:08.360 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.357+0000 7f60ee0d7740 1 --2- 172.21.15.42:0/1826788417 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e87eb814d0 0x55e87ef741c0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:08.360 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.357+0000 7f60ee0d7740 1 --2- 172.21.15.42:0/1826788417 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e87eb792f0 0x55e87ef73b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:08.360 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.357+0000 7f60ee0d7740 1 -- 172.21.15.42:0/1826788417 >> 172.21.15.42:0/1826788417 conn(0x55e87eb96300 msgr2=0x55e87ef41c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:08:08.361 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.357+0000 7f60bffff640 0 client.16665 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:08:08.361 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.357+0000 7f60ee0d7740 1 -- 172.21.15.42:0/1826788417 shutdown_connections 2024-08-30T09:08:08.361 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.357+0000 7f60ee0d7740 1 -- 172.21.15.42:0/1826788417 wait complete. 2024-08-30T09:08:08.361 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.358+0000 7f60ee0d7740 20 client.16665 trim_cache size 0 max 16384 2024-08-30T09:08:08.378 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.375+0000 7f60ee0d7740 1 Processor -- start 2024-08-30T09:08:08.378 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.375+0000 7f60ee0d7740 1 -- start start 2024-08-30T09:08:08.378 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.375+0000 7f60ee0d7740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e87eb792f0 0x55e87ef81450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:08:08.378 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.375+0000 7f60ee0d7740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e87eb814d0 0x55e87ef72d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:08:08.378 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.375+0000 7f60ee0d7740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e87eba21b0 0x55e87ef4c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:08:08.378 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.375+0000 7f60ee0d7740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f60cc0124e0 con 0x55e87eb792f0 2024-08-30T09:08:08.378 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.375+0000 7f60ee0d7740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f60cc00fd50 con 0x55e87eb814d0 2024-08-30T09:08:08.379 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.375+0000 7f60ee0d7740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f60cc00dba0 con 0x55e87eba21b0 2024-08-30T09:08:08.379 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.375+0000 7f60e4f05640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e87eb792f0 0x55e87ef81450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:08:08.379 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.375+0000 7f60e4f05640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e87eb792f0 0x55e87ef81450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38038/0 (socket says 172.21.15.42:38038) 2024-08-30T09:08:08.379 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.375+0000 7f60e4f05640 1 -- 172.21.15.42:0/306282180 learned_addr learned my addr 172.21.15.42:0/306282180 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:08:08.379 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.375+0000 7f60dffff640 1 --2- 172.21.15.42:0/306282180 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e87eb814d0 0x55e87ef72d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:08:08.379 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.375+0000 7f60e5706640 1 --2- 172.21.15.42:0/306282180 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e87eba21b0 0x55e87ef4c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:08:08.379 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.376+0000 7f60e4f05640 1 -- 172.21.15.42:0/306282180 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e87eba21b0 msgr2=0x55e87ef4c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:08:08.379 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.376+0000 7f60e4f05640 1 --2- 172.21.15.42:0/306282180 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e87eba21b0 0x55e87ef4c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:08.379 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.376+0000 7f60e4f05640 1 -- 172.21.15.42:0/306282180 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e87eb814d0 msgr2=0x55e87ef72d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:08:08.379 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.376+0000 7f60e4f05640 1 --2- 172.21.15.42:0/306282180 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e87eb814d0 0x55e87ef72d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:08.379 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.376+0000 7f60e4f05640 1 -- 172.21.15.42:0/306282180 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e87ef44120 con 0x55e87eb792f0 2024-08-30T09:08:08.379 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.376+0000 7f60e4f05640 1 --2- 172.21.15.42:0/306282180 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e87eb792f0 0x55e87ef81450 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f60d400be10 tx=0x7f60d4010590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ec072c05e5c1b87 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:08:08.380 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.376+0000 7f60c67fc640 1 -- 172.21.15.42:0/306282180 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60d40d31d0 con 0x55e87eb792f0 2024-08-30T09:08:08.380 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.376+0000 7f60c67fc640 1 -- 172.21.15.42:0/306282180 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f60d40c6070 con 0x55e87eb792f0 2024-08-30T09:08:08.380 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.376+0000 7f60c67fc640 1 -- 172.21.15.42:0/306282180 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60d40ab9a0 con 0x55e87eb792f0 2024-08-30T09:08:08.380 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.377+0000 7f60ee0d7740 1 -- 172.21.15.42:0/306282180 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e87eb792f0 msgr2=0x55e87ef81450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:08:08.381 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.377+0000 7f60ee0d7740 1 --2- 172.21.15.42:0/306282180 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e87eb792f0 0x55e87ef81450 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f60d400be10 tx=0x7f60d4010590 comp rx=0 tx=0).stop 2024-08-30T09:08:08.381 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.378+0000 7f60ee0d7740 1 -- 172.21.15.42:0/306282180 shutdown_connections 2024-08-30T09:08:08.381 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.378+0000 7f60ee0d7740 1 --2- 172.21.15.42:0/306282180 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e87eba21b0 0x55e87ef4c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:08.381 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.378+0000 7f60ee0d7740 1 --2- 172.21.15.42:0/306282180 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e87eb814d0 0x55e87ef72d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:08.381 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.378+0000 7f60ee0d7740 1 --2- 172.21.15.42:0/306282180 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e87eb792f0 0x55e87ef81450 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:08.381 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.378+0000 7f60ee0d7740 1 -- 172.21.15.42:0/306282180 >> 172.21.15.42:0/306282180 conn(0x55e87eb96300 msgr2=0x55e87ef47c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:08:08.381 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.378+0000 7f60ee0d7740 1 -- 172.21.15.42:0/306282180 shutdown_connections 2024-08-30T09:08:08.381 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.378+0000 7f60ee0d7740 1 -- 172.21.15.42:0/306282180 wait complete. 2024-08-30T09:08:08.381 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.378+0000 7f60ee0d7740 1 -- start start 2024-08-30T09:08:08.381 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.379+0000 7f60ee0d7740 1 Processor -- start 2024-08-30T09:08:08.382 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.379+0000 7f60ee0d7740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e87eb792f0 0x55e87ef78e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:08:08.383 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.379+0000 7f60ee0d7740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e87eb814d0 0x55e87ef794e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:08:08.383 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.379+0000 7f60ee0d7740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e87eba21b0 0x55e87ef7a050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:08:08.383 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.379+0000 7f60ee0d7740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f60cc00e210 con 0x55e87eba21b0 2024-08-30T09:08:08.383 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.379+0000 7f60ee0d7740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f60cc00f680 con 0x55e87eb792f0 2024-08-30T09:08:08.383 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.379+0000 7f60ee0d7740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f60cc00e8e0 con 0x55e87eb814d0 2024-08-30T09:08:08.383 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.379+0000 7f60ee0d7740 10 client.0 _ll_register_callbacks cb 0x7f60d0035070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:08:08.383 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.379+0000 7f60e5706640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e87eba21b0 0x55e87ef7a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:08:08.383 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.379+0000 7f60e4f05640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e87eb792f0 0x55e87ef78e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:08:08.383 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.379+0000 7f60e5706640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e87eba21b0 0x55e87ef7a050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38040/0 (socket says 172.21.15.42:38040) 2024-08-30T09:08:08.383 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.379+0000 7f60e4f05640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e87eb792f0 0x55e87ef78e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:59124/0 (socket says 172.21.15.42:59124) 2024-08-30T09:08:08.383 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.379+0000 7f60e5706640 1 -- 172.21.15.42:0/1552679334 learned_addr learned my addr 172.21.15.42:0/1552679334 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:08:08.383 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.379+0000 7f60dffff640 1 --2- 172.21.15.42:0/1552679334 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e87eb814d0 0x55e87ef794e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:08:08.384 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.380+0000 7f60e5706640 1 -- 172.21.15.42:0/1552679334 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e87eb814d0 msgr2=0x55e87ef794e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:08:08.384 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.380+0000 7f60e5706640 1 --2- 172.21.15.42:0/1552679334 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e87eb814d0 0x55e87ef794e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:08.384 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.380+0000 7f60e5706640 1 -- 172.21.15.42:0/1552679334 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e87eb792f0 msgr2=0x55e87ef78e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:08:08.384 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.380+0000 7f60e5706640 1 --2- 172.21.15.42:0/1552679334 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e87eb792f0 0x55e87ef78e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:08.384 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.380+0000 7f60e5706640 1 -- 172.21.15.42:0/1552679334 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e87ef7a7b0 con 0x55e87eba21b0 2024-08-30T09:08:08.384 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.380+0000 7f60e5706640 1 --2- 172.21.15.42:0/1552679334 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e87eba21b0 0x55e87ef7a050 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f60d8010a30 tx=0x7f60d8010e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1e58af5a2793341 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:08:08.384 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.380+0000 7f60bffff640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-08-30T09:08:08.384 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.380+0000 7f60bffff640 1 -- 172.21.15.42:0/1552679334 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60d801e060 con 0x55e87eba21b0 2024-08-30T09:08:08.384 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.380+0000 7f60bffff640 1 -- 172.21.15.42:0/1552679334 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f60d800eba0 con 0x55e87eba21b0 2024-08-30T09:08:08.384 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.380+0000 7f60ee0d7740 1 -- 172.21.15.42:0/1552679334 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f60d40c6070 con 0x55e87eba21b0 2024-08-30T09:08:08.384 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.381+0000 7f60bffff640 1 -- 172.21.15.42:0/1552679334 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f60d8025050 con 0x55e87eba21b0 2024-08-30T09:08:08.384 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.381+0000 7f60ee0d7740 10 client.16671 fetch_fsmap learned FSMap version 34 2024-08-30T09:08:08.384 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.381+0000 7f60ee0d7740 1 -- 172.21.15.42:0/1552679334 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55e87ef7bba0 con 0x55e87eba21b0 2024-08-30T09:08:08.385 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.381+0000 7f60bffff640 1 -- 172.21.15.42:0/1552679334 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f60d40c6070 con 0x55e87eba21b0 2024-08-30T09:08:08.386 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.381+0000 7f60bffff640 1 -- 172.21.15.42:0/1552679334 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 34) ==== 8366+0+0 (secure 0 0 0) 0x7f60d800b040 con 0x55e87eba21b0 2024-08-30T09:08:08.386 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.381+0000 7f60ee0d7740 10 client.16671 fetch_fsmap finished waiting for FSMap version 34 2024-08-30T09:08:08.386 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.381+0000 7f60ee0d7740 10 client.16671 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-30T09:08:08.386 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.381+0000 7f60ee0d7740 20 client.16671 populate_metadata read hostname 'smithi042' 2024-08-30T09:08:08.386 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.381+0000 7f60ee0d7740 1 --2- 172.21.15.42:0/1552679334 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55e87eb361c0 0x55e87efee3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:08:08.386 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.381+0000 7f60ee0d7740 4 client.16671 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-30T09:08:08.386 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.381+0000 7f60ee0d7740 1 -- 172.21.15.42:0/1552679334 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e87eeb89f0 con 0x55e87eb361c0 2024-08-30T09:08:08.386 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.382+0000 7f60e4f05640 1 --2- 172.21.15.42:0/1552679334 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55e87eb361c0 0x55e87efee3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:08:08.386 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.382+0000 7f60e4f05640 1 --2- 172.21.15.42:0/1552679334 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55e87eb361c0 0x55e87efee3c0 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b6f2a96f0ee4480 server_cookie=4d0e2f5e085820cf in_seq=0 out_seq=0 2024-08-30T09:08:08.386 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.382+0000 7f60bffff640 10 client.16671 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:08:08.418 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.416+0000 7f60bffff640 1 -- 172.21.15.42:0/1552679334 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e87eeb89f0 con 0x55e87eb361c0 2024-08-30T09:08:08.418 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.416+0000 7f60bffff640 10 client.16671 handle_command_reply: tid=0 2024-08-30T09:08:08.419 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.416+0000 7f60ee0d7740 1 client.16671 shutdown 2024-08-30T09:08:08.419 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.416+0000 7f60ee0d7740 1 -- 172.21.15.42:0/1552679334 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e87eba21b0 msgr2=0x55e87ef7a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:08:08.419 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.416+0000 7f60ee0d7740 1 --2- 172.21.15.42:0/1552679334 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e87eba21b0 0x55e87ef7a050 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f60d8010a30 tx=0x7f60d8010e30 comp rx=0 tx=0).stop 2024-08-30T09:08:08.419 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.416+0000 7f60ee0d7740 1 -- 172.21.15.42:0/1552679334 shutdown_connections 2024-08-30T09:08:08.419 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.416+0000 7f60ee0d7740 1 --2- 172.21.15.42:0/1552679334 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55e87eb361c0 0x55e87efee3c0 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:08.419 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.416+0000 7f60ee0d7740 1 --2- 172.21.15.42:0/1552679334 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e87eb814d0 0x55e87ef794e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:08.419 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.416+0000 7f60ee0d7740 1 --2- 172.21.15.42:0/1552679334 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e87eba21b0 0x55e87ef7a050 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:08.419 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.416+0000 7f60ee0d7740 1 --2- 172.21.15.42:0/1552679334 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e87eb792f0 0x55e87ef78e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:08.419 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.416+0000 7f60bffff640 0 client.16671 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:08:08.419 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.416+0000 7f60ee0d7740 1 -- 172.21.15.42:0/1552679334 >> 172.21.15.42:0/1552679334 conn(0x55e87eb96300 msgr2=0x55e87ef81450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:08:08.419 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.416+0000 7f60ee0d7740 1 -- 172.21.15.42:0/1552679334 shutdown_connections 2024-08-30T09:08:08.419 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.416+0000 7f60ee0d7740 1 -- 172.21.15.42:0/1552679334 wait complete. 2024-08-30T09:08:08.420 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.416+0000 7f60ee0d7740 20 client.16671 trim_cache size 0 max 16384 2024-08-30T09:08:08.420 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-08-30T09:08:08.420 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2024-08-30T09:08:08.420 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-08-30T09:08:08.420 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-08-30T09:08:08.421 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.417+0000 7f60e7169640 1 -- 172.21.15.42:0/2122408680 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f60c80d6cd0 msgr2=0x7f60c80d91e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:08:08.421 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.417+0000 7f60e7169640 1 --2- 172.21.15.42:0/2122408680 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f60c80d6cd0 0x7f60c80d91e0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f60d00095f0 tx=0x7f60d0000bc0 comp rx=0 tx=0).stop 2024-08-30T09:08:08.421 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.417+0000 7f60e7169640 1 -- 172.21.15.42:0/2122408680 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f60e010a160 msgr2=0x7f60e01176c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:08:08.421 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.417+0000 7f60e7169640 1 --2- 172.21.15.42:0/2122408680 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f60e010a160 0x7f60e01176c0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f60d400b450 tx=0x7f60d400b910 comp rx=0 tx=0).stop 2024-08-30T09:08:08.421 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.417+0000 7f60e7169640 1 -- 172.21.15.42:0/2122408680 shutdown_connections 2024-08-30T09:08:08.421 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.417+0000 7f60e7169640 1 --2- 172.21.15.42:0/2122408680 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f60c80d6cd0 0x7f60c80d91e0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:08.422 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.417+0000 7f60e7169640 1 --2- 172.21.15.42:0/2122408680 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f60e010f770 0x7f60e01188e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:08.422 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.417+0000 7f60e7169640 1 --2- 172.21.15.42:0/2122408680 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f60e010cb50 0x7f60e0117d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:08.422 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.417+0000 7f60e7169640 1 --2- 172.21.15.42:0/2122408680 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f60e010a160 0x7f60e01176c0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:08.422 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.417+0000 7f60e7169640 1 -- 172.21.15.42:0/2122408680 >> 172.21.15.42:0/2122408680 conn(0x7f60e00705b0 msgr2=0x7f60e010af50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:08:08.422 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.417+0000 7f60e7169640 1 -- 172.21.15.42:0/2122408680 shutdown_connections 2024-08-30T09:08:08.422 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:08.417+0000 7f60e7169640 1 -- 172.21.15.42:0/2122408680 wait complete. 2024-08-30T09:08:08.583 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-30T09:08:08.888 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:08:08 smithi042 ceph-mon[18652]: pgmap v594: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 5.9 MiB/s wr, 612 op/s 2024-08-30T09:08:09.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:08:08 smithi167 ceph-mon[22063]: pgmap v594: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 5.9 MiB/s wr, 612 op/s 2024-08-30T09:08:09.126 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T09:08:09.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:08:08 smithi190 ceph-mon[22001]: pgmap v594: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 5.9 MiB/s wr, 612 op/s 2024-08-30T09:08:10.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.742+0000 7fa4448d1640 1 -- 172.21.15.42:0/2504400498 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa440111ce0 msgr2=0x7fa44010a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:08:10.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.742+0000 7fa4448d1640 1 --2- 172.21.15.42:0/2504400498 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa440111ce0 0x7fa44010a160 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fa438009900 tx=0x7fa43802a440 comp rx=0 tx=0).stop 2024-08-30T09:08:10.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.746+0000 7fa4448d1640 1 -- 172.21.15.42:0/2504400498 shutdown_connections 2024-08-30T09:08:10.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.746+0000 7fa4448d1640 1 --2- 172.21.15.42:0/2504400498 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa44011c9d0 0x7fa44011eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:10.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.746+0000 7fa4448d1640 1 --2- 172.21.15.42:0/2504400498 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa44010a780 0x7fa44010ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:10.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.746+0000 7fa4448d1640 1 --2- 172.21.15.42:0/2504400498 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa440111ce0 0x7fa44010a160 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:10.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.746+0000 7fa4448d1640 1 -- 172.21.15.42:0/2504400498 >> 172.21.15.42:0/2504400498 conn(0x7fa440070c60 msgr2=0x7fa440071070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:08:10.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.751+0000 7fa4448d1640 1 -- 172.21.15.42:0/2504400498 shutdown_connections 2024-08-30T09:08:10.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.752+0000 7fa4448d1640 1 -- 172.21.15.42:0/2504400498 wait complete. 2024-08-30T09:08:10.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.753+0000 7fa4448d1640 1 Processor -- start 2024-08-30T09:08:10.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.755+0000 7fa4448d1640 1 -- start start 2024-08-30T09:08:10.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.755+0000 7fa4448d1640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa44010a780 0x7fa4401a90b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:08:10.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.755+0000 7fa4448d1640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa440111ce0 0x7fa4401a9650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:08:10.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.755+0000 7fa4448d1640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa44011c9d0 0x7fa4401ae8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:08:10.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.755+0000 7fa4448d1640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fa440122700 con 0x7fa440111ce0 2024-08-30T09:08:10.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.755+0000 7fa4448d1640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa440122580 con 0x7fa44011c9d0 2024-08-30T09:08:10.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.755+0000 7fa4448d1640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fa440122880 con 0x7fa44010a780 2024-08-30T09:08:10.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.756+0000 7fa43ed9d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa44011c9d0 0x7fa4401ae8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:08:10.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.756+0000 7fa43ed9d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa44011c9d0 0x7fa4401ae8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:59154/0 (socket says 172.21.15.42:59154) 2024-08-30T09:08:10.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.756+0000 7fa43ed9d640 1 -- 172.21.15.42:0/1375760586 learned_addr learned my addr 172.21.15.42:0/1375760586 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:08:10.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.756+0000 7fa43e59c640 1 --2- 172.21.15.42:0/1375760586 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa44010a780 0x7fa4401a90b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:08:10.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.757+0000 7fa43ed9d640 1 -- 172.21.15.42:0/1375760586 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa44010a780 msgr2=0x7fa4401a90b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:08:10.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.757+0000 7fa43ed9d640 1 --2- 172.21.15.42:0/1375760586 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa44010a780 0x7fa4401a90b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:10.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.757+0000 7fa43ed9d640 1 -- 172.21.15.42:0/1375760586 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa440111ce0 msgr2=0x7fa4401a9650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:08:10.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.757+0000 7fa43ed9d640 1 --2- 172.21.15.42:0/1375760586 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa440111ce0 0x7fa4401a9650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:10.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.757+0000 7fa43ed9d640 1 -- 172.21.15.42:0/1375760586 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4401af020 con 0x7fa44011c9d0 2024-08-30T09:08:10.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.758+0000 7fa43ed9d640 1 --2- 172.21.15.42:0/1375760586 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa44011c9d0 0x7fa4401ae8f0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fa43000eaa0 tx=0x7fa43000ef60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:08:10.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.758+0000 7fa4377fe640 1 -- 172.21.15.42:0/1375760586 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa43000cdb0 con 0x7fa44011c9d0 2024-08-30T09:08:10.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.759+0000 7fa4377fe640 1 -- 172.21.15.42:0/1375760586 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa430005020 con 0x7fa44011c9d0 2024-08-30T09:08:10.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.759+0000 7fa4377fe640 1 -- 172.21.15.42:0/1375760586 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4300116d0 con 0x7fa44011c9d0 2024-08-30T09:08:10.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.759+0000 7fa4448d1640 1 -- 172.21.15.42:0/1375760586 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4401af310 con 0x7fa44011c9d0 2024-08-30T09:08:10.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.759+0000 7fa4448d1640 1 -- 172.21.15.42:0/1375760586 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4401b5a70 con 0x7fa44011c9d0 2024-08-30T09:08:10.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.766+0000 7fa44ae46740 1 Processor -- start 2024-08-30T09:08:10.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.770+0000 7fa4377fe640 1 -- 172.21.15.42:0/1375760586 <== mon.2 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7fa430011870 con 0x7fa44011c9d0 2024-08-30T09:08:10.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.770+0000 7fa44ae46740 1 -- start start 2024-08-30T09:08:10.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.770+0000 7fa44ae46740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b6ae915070 0x55b6aecba2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:08:10.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.770+0000 7fa44ae46740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b6ae8ec1b0 0x55b6aecba800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:08:10.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.770+0000 7fa44ae46740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b6ae8f4390 0x55b6aecbad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:08:10.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.770+0000 7fa44ae46740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55b6aec31630 con 0x55b6ae8f4390 2024-08-30T09:08:10.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.770+0000 7fa44ae46740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55b6aec20bc0 con 0x55b6ae915070 2024-08-30T09:08:10.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.771+0000 7fa44ae46740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55b6aec24b70 con 0x55b6ae8ec1b0 2024-08-30T09:08:10.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.772+0000 7fa4377fe640 1 --2- 172.21.15.42:0/1375760586 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fa42c0b4cc0 0x7fa42c0b71d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:08:10.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.772+0000 7fa43ed9d640 1 --2- 172.21.15.42:0/1375760586 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fa42c0b4cc0 0x7fa42c0b71d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:08:10.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.772+0000 7fa4377fe640 1 -- 172.21.15.42:0/1375760586 <== mon.2 v2:172.21.15.167:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7fa4300e5a10 con 0x7fa44011c9d0 2024-08-30T09:08:10.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.774+0000 7fa43ed9d640 1 --2- 172.21.15.42:0/1375760586 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fa42c0b4cc0 0x7fa42c0b71d0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fa4300a92c0 tx=0x7fa4300a7f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:08:10.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.774+0000 7fa43dd9b640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b6ae915070 0x55b6aecba2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:08:10.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.774+0000 7fa43dd9b640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b6ae915070 0x55b6aecba2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:59162/0 (socket says 172.21.15.42:59162) 2024-08-30T09:08:10.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.774+0000 7fa43dd9b640 1 -- 172.21.15.42:0/3512145517 learned_addr learned my addr 172.21.15.42:0/3512145517 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:08:10.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.774+0000 7fa43dd9b640 1 --2- 172.21.15.42:0/3512145517 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b6ae8f4390 0x55b6aecbad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:08:10.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.774+0000 7fa43e59c640 1 --2- 172.21.15.42:0/3512145517 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b6ae8ec1b0 0x55b6aecba800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:08:10.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.776+0000 7fa43e59c640 1 -- 172.21.15.42:0/3512145517 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b6ae915070 msgr2=0x55b6aecba2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:08:10.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.776+0000 7fa43e59c640 1 --2- 172.21.15.42:0/3512145517 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b6ae915070 0x55b6aecba2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:10.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.776+0000 7fa43e59c640 1 -- 172.21.15.42:0/3512145517 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b6ae8f4390 msgr2=0x55b6aecbad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:08:10.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.776+0000 7fa43e59c640 1 --2- 172.21.15.42:0/3512145517 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b6ae8f4390 0x55b6aecbad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:10.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.776+0000 7fa43e59c640 1 -- 172.21.15.42:0/3512145517 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6ae980650 con 0x55b6ae8ec1b0 2024-08-30T09:08:10.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.781+0000 7fa43e59c640 1 --2- 172.21.15.42:0/3512145517 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b6ae8ec1b0 0x55b6aecba800 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fa438009850 tx=0x7fa438009600 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e248e022b3c62bb8 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:08:10.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.781+0000 7fa434ff9640 1 -- 172.21.15.42:0/3512145517 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa438052070 con 0x55b6ae8ec1b0 2024-08-30T09:08:10.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.781+0000 7fa434ff9640 1 -- 172.21.15.42:0/3512145517 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa438004240 con 0x55b6ae8ec1b0 2024-08-30T09:08:10.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.781+0000 7fa434ff9640 1 -- 172.21.15.42:0/3512145517 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa438036040 con 0x55b6ae8ec1b0 2024-08-30T09:08:10.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.783+0000 7fa44ae46740 1 -- 172.21.15.42:0/3512145517 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b6ae8ec1b0 msgr2=0x55b6aecba800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:08:10.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.783+0000 7fa44ae46740 1 --2- 172.21.15.42:0/3512145517 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b6ae8ec1b0 0x55b6aecba800 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fa438009850 tx=0x7fa438009600 comp rx=0 tx=0).stop 2024-08-30T09:08:10.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.783+0000 7fa44ae46740 1 -- 172.21.15.42:0/3512145517 shutdown_connections 2024-08-30T09:08:10.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.783+0000 7fa44ae46740 1 --2- 172.21.15.42:0/3512145517 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b6ae8ec1b0 0x55b6aecba800 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:10.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.783+0000 7fa44ae46740 1 --2- 172.21.15.42:0/3512145517 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b6ae8f4390 0x55b6aecbad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:10.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.783+0000 7fa44ae46740 1 --2- 172.21.15.42:0/3512145517 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b6ae915070 0x55b6aecba2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:10.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.783+0000 7fa44ae46740 1 -- 172.21.15.42:0/3512145517 >> 172.21.15.42:0/3512145517 conn(0x55b6ae909460 msgr2=0x55b6aecb81b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:08:10.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.783+0000 7fa44ae46740 1 -- 172.21.15.42:0/3512145517 shutdown_connections 2024-08-30T09:08:10.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.784+0000 7fa44ae46740 1 -- 172.21.15.42:0/3512145517 wait complete. 2024-08-30T09:08:10.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.784+0000 7fa44ae46740 1 -- start start 2024-08-30T09:08:10.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.784+0000 7fa44ae46740 1 Processor -- start 2024-08-30T09:08:10.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.785+0000 7fa44ae46740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b6ae8ec1b0 0x55b6aecc5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:08:10.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.785+0000 7fa44ae46740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b6ae8f4390 0x55b6aecc6000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:08:10.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.785+0000 7fa44ae46740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b6ae915070 0x55b6aecd1d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:08:10.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.785+0000 7fa44ae46740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55b6aea83700 con 0x55b6ae8f4390 2024-08-30T09:08:10.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.785+0000 7fa44ae46740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55b6aeba7700 con 0x55b6ae8ec1b0 2024-08-30T09:08:10.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.785+0000 7fa44ae46740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55b6aecc8a40 con 0x55b6ae915070 2024-08-30T09:08:10.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.785+0000 7fa44ae46740 10 client.0 _ll_register_callbacks cb 0x55b6aebd6dc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:08:10.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.786+0000 7fa43dd9b640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b6ae8ec1b0 0x55b6aecc5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:08:10.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.786+0000 7fa43dd9b640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b6ae8ec1b0 0x55b6aecc5ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:59170/0 (socket says 172.21.15.42:59170) 2024-08-30T09:08:10.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.786+0000 7fa43dd9b640 1 -- 172.21.15.42:0/1362678706 learned_addr learned my addr 172.21.15.42:0/1362678706 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:08:10.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.786+0000 7fa43dd9b640 1 --2- 172.21.15.42:0/1362678706 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b6ae915070 0x55b6aecd1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:08:10.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.786+0000 7fa43dd9b640 1 -- 172.21.15.42:0/1362678706 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b6ae915070 msgr2=0x55b6aecd1d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:08:10.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.786+0000 7fa43dd9b640 1 --2- 172.21.15.42:0/1362678706 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b6ae915070 0x55b6aecd1d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:10.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.786+0000 7fa43dd9b640 1 -- 172.21.15.42:0/1362678706 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b6ae8f4390 msgr2=0x55b6aecc6000 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-30T09:08:10.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.786+0000 7fa43dd9b640 1 --2- 172.21.15.42:0/1362678706 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b6ae8f4390 0x55b6aecc6000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:10.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.786+0000 7fa43dd9b640 1 -- 172.21.15.42:0/1362678706 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b6aebe08b0 con 0x55b6ae8ec1b0 2024-08-30T09:08:10.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.788+0000 7fa43dd9b640 1 --2- 172.21.15.42:0/1362678706 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b6ae8ec1b0 0x55b6aecc5ac0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fa428011c50 tx=0x7fa428005a60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b7c977e03897521b server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:08:10.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.788+0000 7fa425ffb640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-30T09:08:10.797 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.795+0000 7fa425ffb640 1 -- 172.21.15.42:0/1362678706 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa428020420 con 0x55b6ae8ec1b0 2024-08-30T09:08:10.797 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.795+0000 7fa44ae46740 1 -- 172.21.15.42:0/1362678706 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa438004240 con 0x55b6ae8ec1b0 2024-08-30T09:08:10.797 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.795+0000 7fa425ffb640 1 -- 172.21.15.42:0/1362678706 <== mon.2 v2:172.21.15.167:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7fa428009050 con 0x55b6ae8ec1b0 2024-08-30T09:08:10.797 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.795+0000 7fa425ffb640 1 -- 172.21.15.42:0/1362678706 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fa428029050 con 0x55b6ae8ec1b0 2024-08-30T09:08:10.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.796+0000 7fa44ae46740 10 client.26444 fetch_fsmap learned FSMap version 34 2024-08-30T09:08:10.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.796+0000 7fa44ae46740 1 -- 172.21.15.42:0/1362678706 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55b6aecd4930 con 0x55b6ae8ec1b0 2024-08-30T09:08:10.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.801+0000 7fa425ffb640 1 -- 172.21.15.42:0/1362678706 <== mon.2 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa438004240 con 0x55b6ae8ec1b0 2024-08-30T09:08:10.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.801+0000 7fa425ffb640 1 -- 172.21.15.42:0/1362678706 <== mon.2 v2:172.21.15.167:3300/0 5 ==== fsmap(e 34) ==== 8366+0+0 (secure 0 0 0) 0x7fa42800f040 con 0x55b6ae8ec1b0 2024-08-30T09:08:10.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.803+0000 7fa44ae46740 10 client.26444 fetch_fsmap finished waiting for FSMap version 34 2024-08-30T09:08:10.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.803+0000 7fa44ae46740 10 client.26444 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-30T09:08:10.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.803+0000 7fa44ae46740 20 client.26444 populate_metadata read hostname 'smithi042' 2024-08-30T09:08:10.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.803+0000 7fa44ae46740 1 --2- 172.21.15.42:0/1362678706 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55b6ae8d39f0 0x55b6aecd7ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:08:10.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.803+0000 7fa44ae46740 4 client.26444 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-30T09:08:10.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.803+0000 7fa44ae46740 1 -- 172.21.15.42:0/1362678706 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b6aec33580 con 0x55b6ae8d39f0 2024-08-30T09:08:10.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.804+0000 7fa43e59c640 1 --2- 172.21.15.42:0/1362678706 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55b6ae8d39f0 0x55b6aecd7ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:08:10.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.804+0000 7fa43e59c640 1 --2- 172.21.15.42:0/1362678706 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55b6ae8d39f0 0x55b6aecd7ed0 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2df43fc8453e7a3f server_cookie=b0b3de389eb7e846 in_seq=0 out_seq=0 2024-08-30T09:08:10.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.804+0000 7fa425ffb640 10 client.26444 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:08:10.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.805+0000 7fa425ffb640 1 -- 172.21.15.42:0/1362678706 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b6aec33580 con 0x55b6ae8d39f0 2024-08-30T09:08:10.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.805+0000 7fa425ffb640 10 client.26444 handle_command_reply: tid=0 2024-08-30T09:08:10.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.805+0000 7fa44ae46740 1 client.26444 shutdown 2024-08-30T09:08:10.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.805+0000 7fa44ae46740 1 -- 172.21.15.42:0/1362678706 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b6ae8ec1b0 msgr2=0x55b6aecc5ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:08:10.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.805+0000 7fa44ae46740 1 --2- 172.21.15.42:0/1362678706 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b6ae8ec1b0 0x55b6aecc5ac0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fa428011c50 tx=0x7fa428005a60 comp rx=0 tx=0).stop 2024-08-30T09:08:10.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.805+0000 7fa44ae46740 1 -- 172.21.15.42:0/1362678706 shutdown_connections 2024-08-30T09:08:10.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.805+0000 7fa44ae46740 1 --2- 172.21.15.42:0/1362678706 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55b6ae8d39f0 0x55b6aecd7ed0 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:10.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.805+0000 7fa44ae46740 1 --2- 172.21.15.42:0/1362678706 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b6ae915070 0x55b6aecd1d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:10.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.805+0000 7fa44ae46740 1 --2- 172.21.15.42:0/1362678706 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b6ae8f4390 0x55b6aecc6000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:10.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.805+0000 7fa44ae46740 1 --2- 172.21.15.42:0/1362678706 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b6ae8ec1b0 0x55b6aecc5ac0 secure :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fa428011c50 tx=0x7fa428005a60 comp rx=0 tx=0).stop 2024-08-30T09:08:10.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.805+0000 7fa44ae46740 1 -- 172.21.15.42:0/1362678706 >> 172.21.15.42:0/1362678706 conn(0x55b6ae909460 msgr2=0x55b6aecbd450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:08:10.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.806+0000 7fa425ffb640 0 client.26444 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:08:10.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.806+0000 7fa44ae46740 1 -- 172.21.15.42:0/1362678706 shutdown_connections 2024-08-30T09:08:10.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.806+0000 7fa44ae46740 1 -- 172.21.15.42:0/1362678706 wait complete. 2024-08-30T09:08:10.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.806+0000 7fa44ae46740 20 client.26444 trim_cache size 0 max 16384 2024-08-30T09:08:10.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.833+0000 7fa44ae46740 1 Processor -- start 2024-08-30T09:08:10.835 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:08:10 smithi042 ceph-mon[18652]: pgmap v595: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.0 MiB/s wr, 726 op/s 2024-08-30T09:08:10.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.833+0000 7fa44ae46740 1 -- start start 2024-08-30T09:08:10.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.833+0000 7fa44ae46740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b6ae8f4390 0x55b6aecc5870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:08:10.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.833+0000 7fa44ae46740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b6ae915070 0x55b6aecc5db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:08:10.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.833+0000 7fa44ae46740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b6ae8a8da0 0x55b6aecbd3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:08:10.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.834+0000 7fa43dd9b640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b6ae8f4390 0x55b6aecc5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:08:10.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.834+0000 7fa43dd9b640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b6ae8f4390 0x55b6aecc5870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:59180/0 (socket says 172.21.15.42:59180) 2024-08-30T09:08:10.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.834+0000 7fa44ae46740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fa420012600 con 0x55b6ae915070 2024-08-30T09:08:10.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.834+0000 7fa44ae46740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa42000fe70 con 0x55b6ae8f4390 2024-08-30T09:08:10.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.834+0000 7fa44ae46740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fa42000dcc0 con 0x55b6ae8a8da0 2024-08-30T09:08:10.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.834+0000 7fa43dd9b640 1 -- 172.21.15.42:0/2971650115 learned_addr learned my addr 172.21.15.42:0/2971650115 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:08:10.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.834+0000 7fa43dd9b640 1 --2- 172.21.15.42:0/2971650115 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b6ae8a8da0 0x55b6aecbd3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:08:10.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.834+0000 7fa43e59c640 1 --2- 172.21.15.42:0/2971650115 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b6ae915070 0x55b6aecc5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:08:10.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.834+0000 7fa43dd9b640 1 -- 172.21.15.42:0/2971650115 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b6ae8a8da0 msgr2=0x55b6aecbd3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:08:10.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.834+0000 7fa43dd9b640 1 --2- 172.21.15.42:0/2971650115 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b6ae8a8da0 0x55b6aecbd3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:10.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.834+0000 7fa43dd9b640 1 -- 172.21.15.42:0/2971650115 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b6ae915070 msgr2=0x55b6aecc5db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:08:10.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.834+0000 7fa43dd9b640 1 --2- 172.21.15.42:0/2971650115 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b6ae915070 0x55b6aecc5db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:10.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.834+0000 7fa43dd9b640 1 -- 172.21.15.42:0/2971650115 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6aecc6710 con 0x55b6ae8f4390 2024-08-30T09:08:10.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.834+0000 7fa43e59c640 1 --2- 172.21.15.42:0/2971650115 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b6ae915070 0x55b6aecc5db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-30T09:08:10.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.835+0000 7fa43dd9b640 1 --2- 172.21.15.42:0/2971650115 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b6ae8f4390 0x55b6aecc5870 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fa428005cc0 tx=0x7fa42800ce50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e47b5d2aac596a49 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:08:10.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.835+0000 7fa427fff640 1 -- 172.21.15.42:0/2971650115 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4280115f0 con 0x55b6ae8f4390 2024-08-30T09:08:10.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.835+0000 7fa427fff640 1 -- 172.21.15.42:0/2971650115 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa428017070 con 0x55b6ae8f4390 2024-08-30T09:08:10.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.835+0000 7fa427fff640 1 -- 172.21.15.42:0/2971650115 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa428020420 con 0x55b6ae8f4390 2024-08-30T09:08:10.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.836+0000 7fa44ae46740 1 -- 172.21.15.42:0/2971650115 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b6ae8f4390 msgr2=0x55b6aecc5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:08:10.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.836+0000 7fa44ae46740 1 --2- 172.21.15.42:0/2971650115 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b6ae8f4390 0x55b6aecc5870 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fa428005cc0 tx=0x7fa42800ce50 comp rx=0 tx=0).stop 2024-08-30T09:08:10.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.836+0000 7fa44ae46740 1 -- 172.21.15.42:0/2971650115 shutdown_connections 2024-08-30T09:08:10.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.836+0000 7fa44ae46740 1 --2- 172.21.15.42:0/2971650115 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b6ae8a8da0 0x55b6aecbd3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:10.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.836+0000 7fa44ae46740 1 --2- 172.21.15.42:0/2971650115 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b6ae915070 0x55b6aecc5db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:10.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.836+0000 7fa44ae46740 1 --2- 172.21.15.42:0/2971650115 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b6ae8f4390 0x55b6aecc5870 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:10.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.836+0000 7fa44ae46740 1 -- 172.21.15.42:0/2971650115 >> 172.21.15.42:0/2971650115 conn(0x55b6ae909460 msgr2=0x55b6aecc4340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:08:10.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.836+0000 7fa44ae46740 1 -- 172.21.15.42:0/2971650115 shutdown_connections 2024-08-30T09:08:10.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.836+0000 7fa44ae46740 1 -- 172.21.15.42:0/2971650115 wait complete. 2024-08-30T09:08:10.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.837+0000 7fa44ae46740 1 -- start start 2024-08-30T09:08:10.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.837+0000 7fa44ae46740 1 Processor -- start 2024-08-30T09:08:10.843 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.838+0000 7fa44ae46740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b6ae8a8da0 0x55b6aecce490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:08:10.843 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.838+0000 7fa44ae46740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b6ae8f4390 0x55b6aecc9940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:08:10.843 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.838+0000 7fa43dd9b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b6ae8a8da0 0x55b6aecce490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:08:10.843 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.838+0000 7fa43dd9b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b6ae8a8da0 0x55b6aecce490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:42054/0 (socket says 172.21.15.42:42054) 2024-08-30T09:08:10.843 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.838+0000 7fa43dd9b640 1 -- 172.21.15.42:0/609507407 learned_addr learned my addr 172.21.15.42:0/609507407 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:08:10.843 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.838+0000 7fa44ae46740 1 --2- 172.21.15.42:0/609507407 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b6ae915070 0x55b6aecc9e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:08:10.843 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.838+0000 7fa44ae46740 1 -- 172.21.15.42:0/609507407 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fa42000e330 con 0x55b6ae8f4390 2024-08-30T09:08:10.843 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.838+0000 7fa43e59c640 1 --2- 172.21.15.42:0/609507407 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b6ae8f4390 0x55b6aecc9940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:08:10.843 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.838+0000 7fa44ae46740 1 -- 172.21.15.42:0/609507407 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa42000f7a0 con 0x55b6ae915070 2024-08-30T09:08:10.843 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.838+0000 7fa44ae46740 1 -- 172.21.15.42:0/609507407 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fa42000ea00 con 0x55b6ae8a8da0 2024-08-30T09:08:10.843 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.839+0000 7fa44ae46740 10 client.0 _ll_register_callbacks cb 0x7fa42800c610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:08:10.843 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.839+0000 7fa43dd9b640 1 --2- 172.21.15.42:0/609507407 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b6ae915070 0x55b6aecc9e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:08:10.843 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.839+0000 7fa43e59c640 1 -- 172.21.15.42:0/609507407 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b6ae8a8da0 msgr2=0x55b6aecce490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:08:10.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.839+0000 7fa43e59c640 1 --2- 172.21.15.42:0/609507407 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b6ae8a8da0 0x55b6aecce490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:10.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.839+0000 7fa43e59c640 1 -- 172.21.15.42:0/609507407 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b6ae915070 msgr2=0x55b6aecc9e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:08:10.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.839+0000 7fa43e59c640 1 --2- 172.21.15.42:0/609507407 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b6ae915070 0x55b6aecc9e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:10.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.839+0000 7fa43e59c640 1 -- 172.21.15.42:0/609507407 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b6aecca5c0 con 0x55b6ae8f4390 2024-08-30T09:08:10.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.839+0000 7fa43dd9b640 1 --2- 172.21.15.42:0/609507407 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b6ae915070 0x55b6aecc9e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-30T09:08:10.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.839+0000 7fa44ae46740 1 -- 172.21.15.42:0/609507407 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa428017070 con 0x55b6ae8f4390 2024-08-30T09:08:10.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.839+0000 7fa43e59c640 1 --2- 172.21.15.42:0/609507407 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b6ae8f4390 0x55b6aecc9940 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fa4380095a0 tx=0x7fa438031440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9e8e2f5c977f6f0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:08:10.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.839+0000 7fa425ffb640 10 client.16683 ms_handle_connect on v2:172.21.15.42:3300/0 2024-08-30T09:08:10.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.839+0000 7fa425ffb640 1 -- 172.21.15.42:0/609507407 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa438052040 con 0x55b6ae8f4390 2024-08-30T09:08:10.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.840+0000 7fa425ffb640 1 -- 172.21.15.42:0/609507407 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7fa4380055f0 con 0x55b6ae8f4390 2024-08-30T09:08:10.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.840+0000 7fa425ffb640 1 -- 172.21.15.42:0/609507407 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fa438036040 con 0x55b6ae8f4390 2024-08-30T09:08:10.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.840+0000 7fa44ae46740 10 client.16683 fetch_fsmap learned FSMap version 34 2024-08-30T09:08:10.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.840+0000 7fa44ae46740 1 -- 172.21.15.42:0/609507407 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({fsmap=34}) -- 0x55b6aed426a0 con 0x55b6ae8f4390 2024-08-30T09:08:10.847 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.840+0000 7fa425ffb640 1 -- 172.21.15.42:0/609507407 <== mon.0 v2:172.21.15.42:3300/0 4 ==== fsmap(e 34) ==== 8366+0+0 (secure 0 0 0) 0x7fa4380314e0 con 0x55b6ae8f4390 2024-08-30T09:08:10.847 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.840+0000 7fa44ae46740 10 client.16683 fetch_fsmap finished waiting for FSMap version 34 2024-08-30T09:08:10.847 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.840+0000 7fa44ae46740 10 client.16683 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-30T09:08:10.847 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.840+0000 7fa44ae46740 20 client.16683 populate_metadata read hostname 'smithi042' 2024-08-30T09:08:10.847 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.840+0000 7fa44ae46740 1 --2- 172.21.15.42:0/609507407 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55b6aed428d0 0x55b6aed44cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:08:10.847 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.840+0000 7fa44ae46740 4 client.16683 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-30T09:08:10.847 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.840+0000 7fa44ae46740 1 -- 172.21.15.42:0/609507407 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b6aec2f020 con 0x55b6aed428d0 2024-08-30T09:08:10.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.841+0000 7fa43dd9b640 1 --2- 172.21.15.42:0/609507407 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55b6aed428d0 0x55b6aed44cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:08:10.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.842+0000 7fa425ffb640 10 client.16683 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:08:10.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.842+0000 7fa43dd9b640 1 --2- 172.21.15.42:0/609507407 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55b6aed428d0 0x55b6aed44cf0 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ee2e6312962508e server_cookie=48ecca77400ebccc in_seq=0 out_seq=0 2024-08-30T09:08:10.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.842+0000 7fa425ffb640 1 -- 172.21.15.42:0/609507407 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+2646 (crc 0 0 0) 0x55b6aec2f020 con 0x55b6aed428d0 2024-08-30T09:08:10.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.842+0000 7fa425ffb640 10 client.16683 handle_command_reply: tid=0 2024-08-30T09:08:10.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.842+0000 7fa44ae46740 1 client.16683 shutdown 2024-08-30T09:08:10.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.843+0000 7fa44ae46740 1 -- 172.21.15.42:0/609507407 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b6ae8f4390 msgr2=0x55b6aecc9940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:08:10.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.843+0000 7fa44ae46740 1 --2- 172.21.15.42:0/609507407 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b6ae8f4390 0x55b6aecc9940 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fa4380095a0 tx=0x7fa438031440 comp rx=0 tx=0).stop 2024-08-30T09:08:10.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.843+0000 7fa44ae46740 1 -- 172.21.15.42:0/609507407 shutdown_connections 2024-08-30T09:08:10.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.843+0000 7fa44ae46740 1 --2- 172.21.15.42:0/609507407 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55b6aed428d0 0x55b6aed44cf0 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:10.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.843+0000 7fa44ae46740 1 --2- 172.21.15.42:0/609507407 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b6ae915070 0x55b6aecc9e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:10.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.843+0000 7fa44ae46740 1 --2- 172.21.15.42:0/609507407 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b6ae8f4390 0x55b6aecc9940 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:10.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.843+0000 7fa44ae46740 1 --2- 172.21.15.42:0/609507407 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b6ae8a8da0 0x55b6aecce490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:10.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.843+0000 7fa44ae46740 1 -- 172.21.15.42:0/609507407 >> 172.21.15.42:0/609507407 conn(0x55b6ae909460 msgr2=0x55b6aecc42d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:08:10.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.843+0000 7fa425ffb640 0 client.16683 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:08:10.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.843+0000 7fa44ae46740 1 -- 172.21.15.42:0/609507407 shutdown_connections 2024-08-30T09:08:10.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.843+0000 7fa44ae46740 1 -- 172.21.15.42:0/609507407 wait complete. 2024-08-30T09:08:10.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.843+0000 7fa44ae46740 20 client.16683 trim_cache size 0 max 16384 2024-08-30T09:08:10.849 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-08-30T09:08:10.849 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "no active scrubs running", 2024-08-30T09:08:10.849 INFO:teuthology.orchestra.run.smithi042.stdout: "scrubs": {}, 2024-08-30T09:08:10.849 INFO:teuthology.orchestra.run.smithi042.stdout: "scrub_stats": { 2024-08-30T09:08:10.849 INFO:teuthology.orchestra.run.smithi042.stdout: "598d8a75-f1cc-4388-80bc-a408bdb4eaac": { 2024-08-30T09:08:10.850 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:07:29.721122+0000", 2024-08-30T09:08:10.850 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:08:10.850 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:08:10.850 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:08:10.850 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:08:10.850 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:08:10.850 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:08:10.850 INFO:teuthology.orchestra.run.smithi042.stdout: "665ec740-7254-49cb-84ef-deff71c461b0": { 2024-08-30T09:08:10.850 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:06:37.144035+0000", 2024-08-30T09:08:10.850 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:08:10.850 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:08:10.850 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:08:10.850 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:08:10.850 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:08:10.851 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:08:10.851 INFO:teuthology.orchestra.run.smithi042.stdout: "1af4252a-4a99-47e9-bbf0-becf5e573dd5": { 2024-08-30T09:08:10.851 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:05:48.565881+0000", 2024-08-30T09:08:10.851 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:08:10.851 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:08:10.851 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:08:10.851 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:08:10.851 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:08:10.851 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:08:10.851 INFO:teuthology.orchestra.run.smithi042.stdout: "347d05ed-4bd3-4590-b099-314f719aab39": { 2024-08-30T09:08:10.851 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:05:00.439721+0000", 2024-08-30T09:08:10.851 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:08:10.851 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:08:10.851 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:08:10.852 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:08:10.852 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:08:10.852 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:08:10.852 INFO:teuthology.orchestra.run.smithi042.stdout: "88cd7478-2b20-47de-ba51-256479749232": { 2024-08-30T09:08:10.852 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:04:08.922625+0000", 2024-08-30T09:08:10.852 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:08:10.852 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:08:10.852 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:08:10.852 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:08:10.852 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:08:10.852 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:08:10.852 INFO:teuthology.orchestra.run.smithi042.stdout: "749f4867-0b70-49e5-af2d-8362a240ebc3": { 2024-08-30T09:08:10.852 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:03:16.386554+0000", 2024-08-30T09:08:10.853 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:08:10.853 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:08:10.853 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:08:10.853 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:08:10.853 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:08:10.853 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:08:10.853 INFO:teuthology.orchestra.run.smithi042.stdout: "85134e4b-ab4a-4885-b07e-a3ad01eaf13d": { 2024-08-30T09:08:10.853 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:01:27.738462+0000", 2024-08-30T09:08:10.853 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:08:10.853 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:08:10.853 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:08:10.853 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:08:10.853 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:08:10.853 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:08:10.854 INFO:teuthology.orchestra.run.smithi042.stdout: "c827ad7a-44ee-4e1d-9bb3-51fb7ab00c73": { 2024-08-30T09:08:10.854 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:02:21.686517+0000", 2024-08-30T09:08:10.854 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:08:10.854 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:08:10.854 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:08:10.854 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:08:10.854 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:08:10.854 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:08:10.854 INFO:teuthology.orchestra.run.smithi042.stdout: "5e2ed104-0625-478a-a78b-584e7f4c2817": { 2024-08-30T09:08:10.854 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:00:33.928400+0000", 2024-08-30T09:08:10.854 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:08:10.854 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:08:10.854 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:08:10.854 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:08:10.854 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:08:10.854 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:08:10.855 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:08:10.855 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-08-30T09:08:10.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.844+0000 7fa4448d1640 1 -- 172.21.15.42:0/1375760586 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fa42c0b4cc0 msgr2=0x7fa42c0b71d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:08:10.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.844+0000 7fa4448d1640 1 --2- 172.21.15.42:0/1375760586 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fa42c0b4cc0 0x7fa42c0b71d0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fa4300a92c0 tx=0x7fa4300a7f90 comp rx=0 tx=0).stop 2024-08-30T09:08:10.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.844+0000 7fa4448d1640 1 -- 172.21.15.42:0/1375760586 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa44011c9d0 msgr2=0x7fa4401ae8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:08:10.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.844+0000 7fa4448d1640 1 --2- 172.21.15.42:0/1375760586 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa44011c9d0 0x7fa4401ae8f0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fa43000eaa0 tx=0x7fa43000ef60 comp rx=0 tx=0).stop 2024-08-30T09:08:10.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.844+0000 7fa4448d1640 1 -- 172.21.15.42:0/1375760586 shutdown_connections 2024-08-30T09:08:10.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.844+0000 7fa4448d1640 1 --2- 172.21.15.42:0/1375760586 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fa42c0b4cc0 0x7fa42c0b71d0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:10.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.844+0000 7fa4448d1640 1 --2- 172.21.15.42:0/1375760586 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa44011c9d0 0x7fa4401ae8f0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:10.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.844+0000 7fa4448d1640 1 --2- 172.21.15.42:0/1375760586 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa440111ce0 0x7fa4401a9650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:10.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.844+0000 7fa4448d1640 1 --2- 172.21.15.42:0/1375760586 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa44010a780 0x7fa4401a90b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:10.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.844+0000 7fa4448d1640 1 -- 172.21.15.42:0/1375760586 >> 172.21.15.42:0/1375760586 conn(0x7fa440070c60 msgr2=0x7fa440071bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:08:10.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.845+0000 7fa4448d1640 1 -- 172.21.15.42:0/1375760586 shutdown_connections 2024-08-30T09:08:10.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:10.845+0000 7fa4448d1640 1 -- 172.21.15.42:0/1375760586 wait complete. 2024-08-30T09:08:10.953 INFO:teuthology.orchestra.run.smithi042.stderr:progress: 445.0 s, 396.4 tps, lat 254.676 ms stddev 333.469 2024-08-30T09:08:11.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:08:10 smithi167 ceph-mon[22063]: pgmap v595: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.0 MiB/s wr, 726 op/s 2024-08-30T09:08:11.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:08:10 smithi190 ceph-mon[22001]: pgmap v595: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.0 MiB/s wr, 726 op/s 2024-08-30T09:08:11.662 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-08-30T09:08:11.662 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph fs dump --format=json 2024-08-30T09:08:12.101 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T09:08:12.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:12.681+0000 7f958def9640 1 -- 172.21.15.42:0/1944235563 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9588110230 msgr2=0x7f958807b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:08:12.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:12.681+0000 7f958def9640 1 --2- 172.21.15.42:0/1944235563 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9588110230 0x7f958807b430 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f9580009ca0 tx=0x7f958002ed30 comp rx=0 tx=0).stop 2024-08-30T09:08:12.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:12.682+0000 7f958def9640 1 -- 172.21.15.42:0/1944235563 shutdown_connections 2024-08-30T09:08:12.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:12.682+0000 7f958def9640 1 --2- 172.21.15.42:0/1944235563 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9588110230 0x7f958807b430 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:12.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:12.682+0000 7f958def9640 1 --2- 172.21.15.42:0/1944235563 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f958810f8e0 0x7f958810fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:12.687 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:12.682+0000 7f958def9640 1 --2- 172.21.15.42:0/1944235563 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f958810b890 0x7f958810bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:12.687 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:12.682+0000 7f958def9640 1 -- 172.21.15.42:0/1944235563 >> 172.21.15.42:0/1944235563 conn(0x7f9588070c60 msgr2=0x7f9588071070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:08:12.687 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:12.682+0000 7f958def9640 1 -- 172.21.15.42:0/1944235563 shutdown_connections 2024-08-30T09:08:12.687 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:12.682+0000 7f958def9640 1 -- 172.21.15.42:0/1944235563 wait complete. 2024-08-30T09:08:12.687 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:12.685+0000 7f958def9640 1 Processor -- start 2024-08-30T09:08:12.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:12.687+0000 7f958def9640 1 -- start start 2024-08-30T09:08:12.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:12.687+0000 7f958def9640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f958810b890 0x7f9588080350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:08:12.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:12.687+0000 7f958def9640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f958810f8e0 0x7f9588080890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:08:12.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:12.687+0000 7f958def9640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9588110230 0x7f9588080dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:08:12.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:12.687+0000 7f958def9640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f958807ea60 con 0x7f958810b890 2024-08-30T09:08:12.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:12.687+0000 7f958def9640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f958807e8e0 con 0x7f958810f8e0 2024-08-30T09:08:12.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:12.687+0000 7f958def9640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f958807ebe0 con 0x7f9588110230 2024-08-30T09:08:12.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:12.688+0000 7f9587fff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9588110230 0x7f9588080dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:08:12.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:12.688+0000 7f9587fff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9588110230 0x7f9588080dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:42074/0 (socket says 172.21.15.42:42074) 2024-08-30T09:08:12.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:12.688+0000 7f9587fff640 1 -- 172.21.15.42:0/436387840 learned_addr learned my addr 172.21.15.42:0/436387840 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:08:12.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:12.688+0000 7f9587fff640 1 -- 172.21.15.42:0/436387840 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f958810f8e0 msgr2=0x7f9588080890 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-08-30T09:08:12.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:12.688+0000 7f9587fff640 1 --2- 172.21.15.42:0/436387840 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f958810f8e0 0x7f9588080890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:12.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:12.688+0000 7f9587fff640 1 -- 172.21.15.42:0/436387840 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f958810b890 msgr2=0x7f9588080350 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-30T09:08:12.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:12.688+0000 7f9587fff640 1 --2- 172.21.15.42:0/436387840 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f958810b890 0x7f9588080350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:12.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:12.688+0000 7f9587fff640 1 -- 172.21.15.42:0/436387840 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95881bd9c0 con 0x7f9588110230 2024-08-30T09:08:12.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:12.689+0000 7f9587fff640 1 --2- 172.21.15.42:0/436387840 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9588110230 0x7f9588080dd0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f9580035570 tx=0x7f958002fe20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:08:12.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:12.694+0000 7f9584ff9640 1 -- 172.21.15.42:0/436387840 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9580044020 con 0x7f9588110230 2024-08-30T09:08:12.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:12.694+0000 7f958def9640 1 -- 172.21.15.42:0/436387840 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95881bdc50 con 0x7f9588110230 2024-08-30T09:08:12.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:12.694+0000 7f958def9640 1 -- 172.21.15.42:0/436387840 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95881be130 con 0x7f9588110230 2024-08-30T09:08:12.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:12.694+0000 7f9584ff9640 1 -- 172.21.15.42:0/436387840 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9580004040 con 0x7f9588110230 2024-08-30T09:08:12.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:12.695+0000 7f9584ff9640 1 -- 172.21.15.42:0/436387840 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f958003c040 con 0x7f9588110230 2024-08-30T09:08:12.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:12.696+0000 7f958def9640 1 -- 172.21.15.42:0/436387840 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95740052c0 con 0x7f9588110230 2024-08-30T09:08:12.703 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:12.701+0000 7f9584ff9640 1 -- 172.21.15.42:0/436387840 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7f9580047020 con 0x7f9588110230 2024-08-30T09:08:12.706 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:12.704+0000 7f9584ff9640 1 --2- 172.21.15.42:0/436387840 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f95700b4d30 0x7f95700b7240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:08:12.706 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:12.704+0000 7f9584ff9640 1 -- 172.21.15.42:0/436387840 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f9580109e00 con 0x7f9588110230 2024-08-30T09:08:12.706 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:12.705+0000 7f9584ff9640 1 -- 172.21.15.42:0/436387840 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f958010a220 con 0x7f9588110230 2024-08-30T09:08:12.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:12.706+0000 7f95877fe640 1 --2- 172.21.15.42:0/436387840 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f95700b4d30 0x7f95700b7240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:08:12.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:12.707+0000 7f95877fe640 1 --2- 172.21.15.42:0/436387840 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f95700b4d30 0x7f95700b7240 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f9588085de0 tx=0x7f9578005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:08:13.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:08:12 smithi167 ceph-mon[22063]: pgmap v596: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 5.9 MiB/s wr, 605 op/s 2024-08-30T09:08:13.080 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-08-30T09:08:13.080 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:08:12 smithi042 ceph-mon[18652]: pgmap v596: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 5.9 MiB/s wr, 605 op/s 2024-08-30T09:08:13.081 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":34,"btime":"2024-08-30T09:06:48:392460+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14703,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/3188400000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":3188400000},{"type":"v1","addr":"172.21.15.42:6835","nonce":3188400000}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14715,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/3356537674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":3356537674},{"type":"v1","addr":"172.21.15.42:6837","nonce":3356537674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14766,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6839/2107785513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6838","nonce":2107785513},{"type":"v1","addr":"172.21.15.42:6839","nonce":2107785513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14778,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6841/2298438674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6840","nonce":2298438674},{"type":"v1","addr":"172.21.15.42:6841","nonce":2298438674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24553,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/543555570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":543555570},{"type":"v1","addr":"172.21.15.190:6833","nonce":543555570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24554,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/3907818941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":3907818941},{"type":"v1","addr":"172.21.15.167:6833","nonce":3907818941}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":34,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-30T08:58:03.055084+0000","modified":"2024-08-30T09:06:47.644639+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24629,"mds_1":24610,"mds_2":24599},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24572":{"gid":24572,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6835/234596845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":234596845},{"type":"v1","addr":"172.21.15.167:6835","nonce":234596845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24592":{"gid":24592,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6835/247905709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":247905709},{"type":"v1","addr":"172.21.15.190:6835","nonce":247905709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24598":{"gid":24598,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6837/863256349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6836","nonce":863256349},{"type":"v1","addr":"172.21.15.190:6837","nonce":863256349}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24599":{"gid":24599,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.167:6837/3011837856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":3011837856},{"type":"v1","addr":"172.21.15.167:6837","nonce":3011837856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24610":{"gid":24610,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.190:6839/719981110","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6838","nonce":719981110},{"type":"v1","addr":"172.21.15.190:6839","nonce":719981110}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24629":{"gid":24629,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.167:6839/3604645207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":3604645207},{"type":"v1","addr":"172.21.15.167:6839","nonce":3604645207}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24629,"qdb_cluster":[24629,24610,24599]},"id":1}]} 2024-08-30T09:08:13.081 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 34 2024-08-30T09:08:13.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:13.075+0000 7f958def9640 1 -- 172.21.15.42:0/436387840 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f95740055b0 con 0x7f9588110230 2024-08-30T09:08:13.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:13.077+0000 7f9584ff9640 1 -- 172.21.15.42:0/436387840 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11919 (secure 0 0 0) 0x7f95800d0bc0 con 0x7f9588110230 2024-08-30T09:08:13.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:13.085+0000 7f958def9640 1 -- 172.21.15.42:0/436387840 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f95700b4d30 msgr2=0x7f95700b7240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:08:13.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:13.085+0000 7f958def9640 1 --2- 172.21.15.42:0/436387840 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f95700b4d30 0x7f95700b7240 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f9588085de0 tx=0x7f9578005f90 comp rx=0 tx=0).stop 2024-08-30T09:08:13.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:13.085+0000 7f958def9640 1 -- 172.21.15.42:0/436387840 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9588110230 msgr2=0x7f9588080dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:08:13.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:13.085+0000 7f958def9640 1 --2- 172.21.15.42:0/436387840 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9588110230 0x7f9588080dd0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f9580035570 tx=0x7f958002fe20 comp rx=0 tx=0).stop 2024-08-30T09:08:13.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:13.087+0000 7f958def9640 1 -- 172.21.15.42:0/436387840 shutdown_connections 2024-08-30T09:08:13.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:13.087+0000 7f958def9640 1 --2- 172.21.15.42:0/436387840 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f95700b4d30 0x7f95700b7240 secure :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f9588085de0 tx=0x7f9578005f90 comp rx=0 tx=0).stop 2024-08-30T09:08:13.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:13.087+0000 7f958def9640 1 --2- 172.21.15.42:0/436387840 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9588110230 0x7f9588080dd0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:13.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:13.087+0000 7f958def9640 1 --2- 172.21.15.42:0/436387840 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f958810f8e0 0x7f9588080890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:13.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:13.087+0000 7f958def9640 1 --2- 172.21.15.42:0/436387840 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f958810b890 0x7f9588080350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:13.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:13.087+0000 7f958def9640 1 -- 172.21.15.42:0/436387840 >> 172.21.15.42:0/436387840 conn(0x7f9588070c60 msgr2=0x7f958810d680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:08:13.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:13.091+0000 7f958def9640 1 -- 172.21.15.42:0/436387840 shutdown_connections 2024-08-30T09:08:13.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:13.092+0000 7f958def9640 1 -- 172.21.15.42:0/436387840 wait complete. 2024-08-30T09:08:13.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:08:12 smithi190 ceph-mon[22001]: pgmap v596: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 5.9 MiB/s wr, 605 op/s 2024-08-30T09:08:14.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:08:13 smithi167 ceph-mon[22063]: from='client.? 172.21.15.42:0/436387840' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-30T09:08:14.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:08:13 smithi042 ceph-mon[18652]: from='client.? 172.21.15.42:0/436387840' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-30T09:08:14.188 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:08:13 smithi190 ceph-mon[22001]: from='client.? 172.21.15.42:0/436387840' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-30T09:08:14.374 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-08-30T09:08:14.860 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T09:08:15.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:08:14 smithi167 ceph-mon[22063]: pgmap v597: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.6 MiB/s wr, 665 op/s 2024-08-30T09:08:15.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:08:14 smithi042 ceph-mon[18652]: pgmap v597: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.6 MiB/s wr, 665 op/s 2024-08-30T09:08:15.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:08:14 smithi190 ceph-mon[22001]: pgmap v597: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.6 MiB/s wr, 665 op/s 2024-08-30T09:08:15.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.507+0000 7faf895ec640 1 -- 172.21.15.42:0/4751001 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faf84111ce0 msgr2=0x7faf8410a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:08:15.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.507+0000 7faf895ec640 1 --2- 172.21.15.42:0/4751001 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faf84111ce0 0x7faf8410a160 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7faf74009860 tx=0x7faf7402a380 comp rx=0 tx=0).stop 2024-08-30T09:08:15.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.510+0000 7faf895ec640 1 -- 172.21.15.42:0/4751001 shutdown_connections 2024-08-30T09:08:15.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.510+0000 7faf895ec640 1 --2- 172.21.15.42:0/4751001 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faf8410a780 0x7faf8410ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:15.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.510+0000 7faf895ec640 1 --2- 172.21.15.42:0/4751001 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faf8411c9d0 0x7faf8411eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:15.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.510+0000 7faf895ec640 1 --2- 172.21.15.42:0/4751001 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faf84111ce0 0x7faf8410a160 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:15.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.510+0000 7faf895ec640 1 -- 172.21.15.42:0/4751001 >> 172.21.15.42:0/4751001 conn(0x7faf84070c60 msgr2=0x7faf84071070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:08:15.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.513+0000 7faf895ec640 1 -- 172.21.15.42:0/4751001 shutdown_connections 2024-08-30T09:08:15.519 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.515+0000 7faf895ec640 1 -- 172.21.15.42:0/4751001 wait complete. 2024-08-30T09:08:15.519 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.516+0000 7faf895ec640 1 Processor -- start 2024-08-30T09:08:15.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.516+0000 7faf895ec640 1 -- start start 2024-08-30T09:08:15.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.516+0000 7faf895ec640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faf8410a780 0x7faf841a90a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:08:15.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.516+0000 7faf895ec640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faf84111ce0 0x7faf841a9640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:08:15.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.516+0000 7faf895ec640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faf8411c9d0 0x7faf841ae8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:08:15.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.516+0000 7faf895ec640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7faf84122690 con 0x7faf84111ce0 2024-08-30T09:08:15.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.516+0000 7faf895ec640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7faf84122510 con 0x7faf8410a780 2024-08-30T09:08:15.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.516+0000 7faf895ec640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7faf84122810 con 0x7faf8411c9d0 2024-08-30T09:08:15.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.519+0000 7faf827fc640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faf84111ce0 0x7faf841a9640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:08:15.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.519+0000 7faf837fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faf8411c9d0 0x7faf841ae8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:08:15.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.519+0000 7faf827fc640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faf84111ce0 0x7faf841a9640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38110/0 (socket says 172.21.15.42:38110) 2024-08-30T09:08:15.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.519+0000 7faf827fc640 1 -- 172.21.15.42:0/751705348 learned_addr learned my addr 172.21.15.42:0/751705348 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:08:15.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.519+0000 7faf82ffd640 1 --2- 172.21.15.42:0/751705348 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faf8410a780 0x7faf841a90a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:08:15.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.520+0000 7faf837fe640 1 -- 172.21.15.42:0/751705348 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faf8410a780 msgr2=0x7faf841a90a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:08:15.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.520+0000 7faf837fe640 1 --2- 172.21.15.42:0/751705348 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faf8410a780 0x7faf841a90a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:15.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.520+0000 7faf837fe640 1 -- 172.21.15.42:0/751705348 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faf84111ce0 msgr2=0x7faf841a9640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:08:15.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.520+0000 7faf837fe640 1 --2- 172.21.15.42:0/751705348 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faf84111ce0 0x7faf841a9640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:15.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.520+0000 7faf837fe640 1 -- 172.21.15.42:0/751705348 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf841af010 con 0x7faf8411c9d0 2024-08-30T09:08:15.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.520+0000 7faf837fe640 1 --2- 172.21.15.42:0/751705348 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faf8411c9d0 0x7faf841ae8e0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7faf7c009fd0 tx=0x7faf7c00ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:08:15.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.521+0000 7faf6bfff640 1 -- 172.21.15.42:0/751705348 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf7c00ce20 con 0x7faf8411c9d0 2024-08-30T09:08:15.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.521+0000 7faf895ec640 1 -- 172.21.15.42:0/751705348 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf841af300 con 0x7faf8411c9d0 2024-08-30T09:08:15.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.521+0000 7faf895ec640 1 -- 172.21.15.42:0/751705348 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf8407b3c0 con 0x7faf8411c9d0 2024-08-30T09:08:15.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.521+0000 7faf6bfff640 1 -- 172.21.15.42:0/751705348 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faf7c005020 con 0x7faf8411c9d0 2024-08-30T09:08:15.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.521+0000 7faf6bfff640 1 -- 172.21.15.42:0/751705348 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf7c010690 con 0x7faf8411c9d0 2024-08-30T09:08:15.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.530+0000 7faf6bfff640 1 -- 172.21.15.42:0/751705348 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7faf7c010830 con 0x7faf8411c9d0 2024-08-30T09:08:15.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.532+0000 7faf6bfff640 1 --2- 172.21.15.42:0/751705348 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7faf6c0b4cc0 0x7faf6c0b71d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:08:15.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.532+0000 7faf6bfff640 1 -- 172.21.15.42:0/751705348 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7faf7c0e6b00 con 0x7faf8411c9d0 2024-08-30T09:08:15.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.533+0000 7faf8fb61740 1 Processor -- start 2024-08-30T09:08:15.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.533+0000 7faf82ffd640 1 --2- 172.21.15.42:0/751705348 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7faf6c0b4cc0 0x7faf6c0b71d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:08:15.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.535+0000 7faf8fb61740 1 -- start start 2024-08-30T09:08:15.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.535+0000 7faf8fb61740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b9a1417070 0x55b9a17bc310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:08:15.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.535+0000 7faf8fb61740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b9a13ee1b0 0x55b9a17bc850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:08:15.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.535+0000 7faf8fb61740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b9a13f6390 0x55b9a17bcd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:08:15.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.535+0000 7faf8fb61740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55b9a1733680 con 0x55b9a13ee1b0 2024-08-30T09:08:15.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.535+0000 7faf8fb61740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55b9a1722c10 con 0x55b9a13f6390 2024-08-30T09:08:15.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.535+0000 7faf8fb61740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55b9a1726bc0 con 0x55b9a1417070 2024-08-30T09:08:15.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.536+0000 7faf837fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b9a13f6390 0x55b9a17bcd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:08:15.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.536+0000 7faf837fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b9a13f6390 0x55b9a17bcd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:59238/0 (socket says 172.21.15.42:59238) 2024-08-30T09:08:15.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.536+0000 7faf837fe640 1 -- 172.21.15.42:0/1607195528 learned_addr learned my addr 172.21.15.42:0/1607195528 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:08:15.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.536+0000 7faf82ffd640 1 --2- 172.21.15.42:0/751705348 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7faf6c0b4cc0 0x7faf6c0b71d0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7faf7402a970 tx=0x7faf7402a880 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:08:15.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.536+0000 7faf82ffd640 1 --2- 172.21.15.42:0/1607195528 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b9a1417070 0x55b9a17bc310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:08:15.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.536+0000 7faf837fe640 1 -- 172.21.15.42:0/1607195528 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b9a1417070 msgr2=0x55b9a17bc310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:08:15.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.536+0000 7faf837fe640 1 --2- 172.21.15.42:0/1607195528 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b9a1417070 0x55b9a17bc310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:15.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.536+0000 7faf837fe640 1 -- 172.21.15.42:0/1607195528 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b9a13ee1b0 msgr2=0x55b9a17bc850 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-30T09:08:15.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.536+0000 7faf837fe640 1 --2- 172.21.15.42:0/1607195528 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b9a13ee1b0 0x55b9a17bc850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:15.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.536+0000 7faf837fe640 1 -- 172.21.15.42:0/1607195528 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b9a1482650 con 0x55b9a13f6390 2024-08-30T09:08:15.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.537+0000 7faf837fe640 1 --2- 172.21.15.42:0/1607195528 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b9a13f6390 0x55b9a17bcd90 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7faf7c0ad130 tx=0x7faf7c0ad8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=54cabb59407a51f2 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:08:15.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.538+0000 7faf697fa640 1 -- 172.21.15.42:0/1607195528 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf7c0b64c0 con 0x55b9a13f6390 2024-08-30T09:08:15.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.538+0000 7faf697fa640 1 -- 172.21.15.42:0/1607195528 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faf7c0a9430 con 0x55b9a13f6390 2024-08-30T09:08:15.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.539+0000 7faf8fb61740 1 -- 172.21.15.42:0/1607195528 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b9a13f6390 msgr2=0x55b9a17bcd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:08:15.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.539+0000 7faf8fb61740 1 --2- 172.21.15.42:0/1607195528 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b9a13f6390 0x55b9a17bcd90 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7faf7c0ad130 tx=0x7faf7c0ad8a0 comp rx=0 tx=0).stop 2024-08-30T09:08:15.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.539+0000 7faf8fb61740 1 -- 172.21.15.42:0/1607195528 shutdown_connections 2024-08-30T09:08:15.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.539+0000 7faf8fb61740 1 --2- 172.21.15.42:0/1607195528 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b9a13f6390 0x55b9a17bcd90 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:15.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.539+0000 7faf8fb61740 1 --2- 172.21.15.42:0/1607195528 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b9a13ee1b0 0x55b9a17bc850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:15.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.539+0000 7faf8fb61740 1 --2- 172.21.15.42:0/1607195528 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b9a1417070 0x55b9a17bc310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:15.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.539+0000 7faf8fb61740 1 -- 172.21.15.42:0/1607195528 >> 172.21.15.42:0/1607195528 conn(0x55b9a140b460 msgr2=0x55b9a17ba200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:08:15.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.539+0000 7faf8fb61740 1 -- 172.21.15.42:0/1607195528 shutdown_connections 2024-08-30T09:08:15.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.539+0000 7faf8fb61740 1 -- 172.21.15.42:0/1607195528 wait complete. 2024-08-30T09:08:15.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.540+0000 7faf8fb61740 1 -- start start 2024-08-30T09:08:15.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.540+0000 7faf8fb61740 1 Processor -- start 2024-08-30T09:08:15.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.541+0000 7faf8fb61740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b9a13ee1b0 0x55b9a17c7aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:08:15.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.541+0000 7faf8fb61740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b9a13f6390 0x55b9a17c7fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:08:15.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.541+0000 7faf8fb61740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b9a1417070 0x55b9a17d3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:08:15.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.541+0000 7faf8fb61740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55b9a1585700 con 0x55b9a13ee1b0 2024-08-30T09:08:15.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.541+0000 7faf8fb61740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55b9a16a9750 con 0x55b9a13f6390 2024-08-30T09:08:15.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.541+0000 7faf8fb61740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55b9a17cab90 con 0x55b9a1417070 2024-08-30T09:08:15.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.541+0000 7faf8fb61740 10 client.0 _ll_register_callbacks cb 0x55b9a16d8e10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:08:15.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.542+0000 7faf827fc640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b9a13f6390 0x55b9a17c7fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:08:15.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.542+0000 7faf827fc640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b9a13f6390 0x55b9a17c7fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:59246/0 (socket says 172.21.15.42:59246) 2024-08-30T09:08:15.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.542+0000 7faf827fc640 1 -- 172.21.15.42:0/925085954 learned_addr learned my addr 172.21.15.42:0/925085954 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:08:15.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.542+0000 7faf837fe640 1 --2- 172.21.15.42:0/925085954 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b9a1417070 0x55b9a17d3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:08:15.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.543+0000 7faf827fc640 1 -- 172.21.15.42:0/925085954 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b9a1417070 msgr2=0x55b9a17d3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:08:15.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.543+0000 7faf827fc640 1 --2- 172.21.15.42:0/925085954 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b9a1417070 0x55b9a17d3f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:15.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.543+0000 7faf827fc640 1 -- 172.21.15.42:0/925085954 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b9a13ee1b0 msgr2=0x55b9a17c7aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:08:15.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.543+0000 7faf827fc640 1 --2- 172.21.15.42:0/925085954 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b9a13ee1b0 0x55b9a17c7aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:15.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.543+0000 7faf827fc640 1 -- 172.21.15.42:0/925085954 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b9a16e2900 con 0x55b9a13f6390 2024-08-30T09:08:15.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.543+0000 7faf827fc640 1 --2- 172.21.15.42:0/925085954 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b9a13f6390 0x55b9a17c7fe0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7faf7800de10 tx=0x7faf7800f730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c65fbe6dd622701f server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:08:15.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.543+0000 7faf66ff5640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-30T09:08:15.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.543+0000 7faf66ff5640 1 -- 172.21.15.42:0/925085954 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf78011850 con 0x55b9a13f6390 2024-08-30T09:08:15.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.543+0000 7faf8fb61740 1 -- 172.21.15.42:0/925085954 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faf7c0a9430 con 0x55b9a13f6390 2024-08-30T09:08:15.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.543+0000 7faf66ff5640 1 -- 172.21.15.42:0/925085954 <== mon.2 v2:172.21.15.167:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7faf780119f0 con 0x55b9a13f6390 2024-08-30T09:08:15.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.544+0000 7faf66ff5640 1 -- 172.21.15.42:0/925085954 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7faf78024050 con 0x55b9a13f6390 2024-08-30T09:08:15.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.546+0000 7faf8fb61740 10 client.26462 fetch_fsmap learned FSMap version 34 2024-08-30T09:08:15.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.546+0000 7faf8fb61740 1 -- 172.21.15.42:0/925085954 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55b9a17d6a10 con 0x55b9a13f6390 2024-08-30T09:08:15.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.547+0000 7faf66ff5640 1 -- 172.21.15.42:0/925085954 <== mon.2 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faf7c0a9430 con 0x55b9a13f6390 2024-08-30T09:08:15.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.547+0000 7faf66ff5640 1 -- 172.21.15.42:0/925085954 <== mon.2 v2:172.21.15.167:3300/0 5 ==== fsmap(e 34) ==== 8366+0+0 (secure 0 0 0) 0x7faf7801bce0 con 0x55b9a13f6390 2024-08-30T09:08:15.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.547+0000 7faf8fb61740 10 client.26462 fetch_fsmap finished waiting for FSMap version 34 2024-08-30T09:08:15.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.547+0000 7faf8fb61740 10 client.26462 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-08-30T09:08:15.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.547+0000 7faf8fb61740 20 client.26462 populate_metadata read hostname 'smithi042' 2024-08-30T09:08:15.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.547+0000 7faf8fb61740 1 --2- 172.21.15.42:0/925085954 >> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] conn(0x55b9a13d59f0 0x55b9a17da3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:08:15.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.547+0000 7faf8fb61740 4 client.26462 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-30T09:08:15.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.547+0000 7faf8fb61740 1 -- 172.21.15.42:0/925085954 --> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b9a17355d0 con 0x55b9a13d59f0 2024-08-30T09:08:15.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.548+0000 7faf82ffd640 1 --2- 172.21.15.42:0/925085954 >> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] conn(0x55b9a13d59f0 0x55b9a17da3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:08:15.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.548+0000 7faf82ffd640 1 --2- 172.21.15.42:0/925085954 >> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] conn(0x55b9a13d59f0 0x55b9a17da3f0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=79cc05c49bef3f20 server_cookie=da733af29c7cd3cf in_seq=0 out_seq=0 2024-08-30T09:08:15.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.548+0000 7faf66ff5640 10 client.26462 ms_handle_connect on v2:172.21.15.167:6836/3011837856 2024-08-30T09:08:15.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.551+0000 7faf66ff5640 1 -- 172.21.15.42:0/925085954 <== mds.2 v2:172.21.15.167:6836/3011837856 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b9a17355d0 con 0x55b9a13d59f0 2024-08-30T09:08:15.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.551+0000 7faf66ff5640 10 client.26462 handle_command_reply: tid=0 2024-08-30T09:08:15.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.551+0000 7faf8fb61740 1 client.26462 shutdown 2024-08-30T09:08:15.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.551+0000 7faf8fb61740 1 -- 172.21.15.42:0/925085954 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b9a13f6390 msgr2=0x55b9a17c7fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:08:15.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.551+0000 7faf8fb61740 1 --2- 172.21.15.42:0/925085954 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b9a13f6390 0x55b9a17c7fe0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7faf7800de10 tx=0x7faf7800f730 comp rx=0 tx=0).stop 2024-08-30T09:08:15.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.554+0000 7faf8fb61740 1 -- 172.21.15.42:0/925085954 shutdown_connections 2024-08-30T09:08:15.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.554+0000 7faf8fb61740 1 --2- 172.21.15.42:0/925085954 >> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] conn(0x55b9a13d59f0 0x55b9a17da3f0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:15.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.554+0000 7faf8fb61740 1 --2- 172.21.15.42:0/925085954 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b9a1417070 0x55b9a17d3f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:15.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.554+0000 7faf8fb61740 1 --2- 172.21.15.42:0/925085954 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b9a13f6390 0x55b9a17c7fe0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:15.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.554+0000 7faf8fb61740 1 --2- 172.21.15.42:0/925085954 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b9a13ee1b0 0x55b9a17c7aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:15.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.554+0000 7faf8fb61740 1 -- 172.21.15.42:0/925085954 >> 172.21.15.42:0/925085954 conn(0x55b9a140b460 msgr2=0x55b9a17bf4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:08:15.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.554+0000 7faf66ff5640 0 client.26462 ms_handle_reset on v2:172.21.15.167:6836/3011837856 2024-08-30T09:08:15.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.556+0000 7faf8fb61740 1 -- 172.21.15.42:0/925085954 shutdown_connections 2024-08-30T09:08:15.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.556+0000 7faf8fb61740 1 -- 172.21.15.42:0/925085954 wait complete. 2024-08-30T09:08:15.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.557+0000 7faf8fb61740 20 client.26462 trim_cache size 0 max 16384 2024-08-30T09:08:15.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.577+0000 7faf8fb61740 1 Processor -- start 2024-08-30T09:08:15.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.577+0000 7faf8fb61740 1 -- start start 2024-08-30T09:08:15.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.577+0000 7faf8fb61740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b9a13ee1b0 0x55b9a17c7f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:08:15.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.578+0000 7faf8fb61740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b9a13f6390 0x55b9a17d3e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:08:15.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.578+0000 7faf8fb61740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b9a1417070 0x55b9a17c4c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:08:15.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.578+0000 7faf8fb61740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7faf70012640 con 0x55b9a1417070 2024-08-30T09:08:15.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.578+0000 7faf8fb61740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7faf7000feb0 con 0x55b9a13ee1b0 2024-08-30T09:08:15.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.578+0000 7faf82ffd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b9a13ee1b0 0x55b9a17c7f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:08:15.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.578+0000 7faf82ffd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b9a13ee1b0 0x55b9a17c7f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:59252/0 (socket says 172.21.15.42:59252) 2024-08-30T09:08:15.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.578+0000 7faf82ffd640 1 -- 172.21.15.42:0/1710766559 learned_addr learned my addr 172.21.15.42:0/1710766559 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:08:15.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.578+0000 7faf827fc640 1 --2- 172.21.15.42:0/1710766559 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b9a13f6390 0x55b9a17d3e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:08:15.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.578+0000 7faf8fb61740 1 -- 172.21.15.42:0/1710766559 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7faf7000dd00 con 0x55b9a13f6390 2024-08-30T09:08:15.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.578+0000 7faf82ffd640 1 -- 172.21.15.42:0/1710766559 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b9a13f6390 msgr2=0x55b9a17d3e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:08:15.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.578+0000 7faf82ffd640 1 --2- 172.21.15.42:0/1710766559 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b9a13f6390 0x55b9a17d3e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:15.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.578+0000 7faf82ffd640 1 -- 172.21.15.42:0/1710766559 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b9a1417070 msgr2=0x55b9a17c4c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:08:15.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.578+0000 7faf82ffd640 1 --2- 172.21.15.42:0/1710766559 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b9a1417070 0x55b9a17c4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:15.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.578+0000 7faf82ffd640 1 -- 172.21.15.42:0/1710766559 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b9a17be600 con 0x55b9a13ee1b0 2024-08-30T09:08:15.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.578+0000 7faf82ffd640 1 --2- 172.21.15.42:0/1710766559 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b9a13ee1b0 0x55b9a17c7f60 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7faf74009780 tx=0x7faf7402a380 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f49a99d9516e9269 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:08:15.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.579+0000 7faf68ff9640 1 -- 172.21.15.42:0/1710766559 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf74061070 con 0x55b9a13ee1b0 2024-08-30T09:08:15.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.579+0000 7faf68ff9640 1 -- 172.21.15.42:0/1710766559 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faf74056040 con 0x55b9a13ee1b0 2024-08-30T09:08:15.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.579+0000 7faf68ff9640 1 -- 172.21.15.42:0/1710766559 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf74037040 con 0x55b9a13ee1b0 2024-08-30T09:08:15.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.580+0000 7faf8fb61740 1 -- 172.21.15.42:0/1710766559 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b9a13ee1b0 msgr2=0x55b9a17c7f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:08:15.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.580+0000 7faf8fb61740 1 --2- 172.21.15.42:0/1710766559 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b9a13ee1b0 0x55b9a17c7f60 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7faf74009780 tx=0x7faf7402a380 comp rx=0 tx=0).stop 2024-08-30T09:08:15.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.580+0000 7faf8fb61740 1 -- 172.21.15.42:0/1710766559 shutdown_connections 2024-08-30T09:08:15.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.580+0000 7faf8fb61740 1 --2- 172.21.15.42:0/1710766559 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b9a13f6390 0x55b9a17d3e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:15.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.580+0000 7faf8fb61740 1 --2- 172.21.15.42:0/1710766559 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b9a1417070 0x55b9a17c4c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:15.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.580+0000 7faf8fb61740 1 --2- 172.21.15.42:0/1710766559 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b9a13ee1b0 0x55b9a17c7f60 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:15.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.580+0000 7faf8fb61740 1 -- 172.21.15.42:0/1710766559 >> 172.21.15.42:0/1710766559 conn(0x55b9a140b460 msgr2=0x55b9a17c7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:08:15.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.580+0000 7faf8fb61740 1 -- 172.21.15.42:0/1710766559 shutdown_connections 2024-08-30T09:08:15.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.580+0000 7faf8fb61740 1 -- 172.21.15.42:0/1710766559 wait complete. 2024-08-30T09:08:15.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.580+0000 7faf8fb61740 1 -- start start 2024-08-30T09:08:15.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.580+0000 7faf8fb61740 1 Processor -- start 2024-08-30T09:08:15.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.581+0000 7faf8fb61740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b9a13ee1b0 0x55b9a17cbe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:08:15.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.581+0000 7faf8fb61740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b9a13f6390 0x55b9a17cc390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:08:15.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.581+0000 7faf8fb61740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b9a1417070 0x55b9a17cc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:08:15.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.581+0000 7faf8fb61740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7faf7000e370 con 0x55b9a13f6390 2024-08-30T09:08:15.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.581+0000 7faf8fb61740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7faf7000f7e0 con 0x55b9a1417070 2024-08-30T09:08:15.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.581+0000 7faf8fb61740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7faf7000ea40 con 0x55b9a13ee1b0 2024-08-30T09:08:15.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.581+0000 7faf8fb61740 10 client.0 _ll_register_callbacks cb 0x7faf7800d3c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:08:15.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.581+0000 7faf827fc640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b9a13f6390 0x55b9a17cc390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:08:15.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.581+0000 7faf827fc640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b9a13f6390 0x55b9a17cc390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38138/0 (socket says 172.21.15.42:38138) 2024-08-30T09:08:15.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.581+0000 7faf827fc640 1 -- 172.21.15.42:0/982753155 learned_addr learned my addr 172.21.15.42:0/982753155 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:08:15.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.581+0000 7faf82ffd640 1 --2- 172.21.15.42:0/982753155 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b9a13ee1b0 0x55b9a17cbe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:08:15.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.581+0000 7faf837fe640 1 --2- 172.21.15.42:0/982753155 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b9a1417070 0x55b9a17cc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:08:15.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.581+0000 7faf82ffd640 1 -- 172.21.15.42:0/982753155 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b9a1417070 msgr2=0x55b9a17cc8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:08:15.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.581+0000 7faf82ffd640 1 --2- 172.21.15.42:0/982753155 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b9a1417070 0x55b9a17cc8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:15.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.581+0000 7faf82ffd640 1 -- 172.21.15.42:0/982753155 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b9a13f6390 msgr2=0x55b9a17cc390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:08:15.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.581+0000 7faf82ffd640 1 --2- 172.21.15.42:0/982753155 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b9a13f6390 0x55b9a17cc390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:15.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.581+0000 7faf82ffd640 1 -- 172.21.15.42:0/982753155 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b9a1843df0 con 0x55b9a13ee1b0 2024-08-30T09:08:15.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.581+0000 7faf827fc640 1 --2- 172.21.15.42:0/982753155 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b9a13f6390 0x55b9a17cc390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-30T09:08:15.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.581+0000 7faf82ffd640 1 --2- 172.21.15.42:0/982753155 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b9a13ee1b0 0x55b9a17cbe50 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7faf74009830 tx=0x7faf740056f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d556114db5bf55de server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:08:15.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.581+0000 7faf66ff5640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-08-30T09:08:15.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.582+0000 7faf66ff5640 1 -- 172.21.15.42:0/982753155 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf7406b070 con 0x55b9a13ee1b0 2024-08-30T09:08:15.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.582+0000 7faf66ff5640 1 -- 172.21.15.42:0/982753155 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7faf74004c90 con 0x55b9a13ee1b0 2024-08-30T09:08:15.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.582+0000 7faf8fb61740 1 -- 172.21.15.42:0/982753155 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faf74056040 con 0x55b9a13ee1b0 2024-08-30T09:08:15.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.582+0000 7faf66ff5640 1 -- 172.21.15.42:0/982753155 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7faf74037040 con 0x55b9a13ee1b0 2024-08-30T09:08:15.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.582+0000 7faf8fb61740 10 client.26452 fetch_fsmap learned FSMap version 34 2024-08-30T09:08:15.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.582+0000 7faf8fb61740 1 -- 172.21.15.42:0/982753155 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55b9a1844fb0 con 0x55b9a13ee1b0 2024-08-30T09:08:15.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.582+0000 7faf66ff5640 1 -- 172.21.15.42:0/982753155 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faf74056040 con 0x55b9a13ee1b0 2024-08-30T09:08:15.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.583+0000 7faf66ff5640 1 -- 172.21.15.42:0/982753155 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 34) ==== 8366+0+0 (secure 0 0 0) 0x7faf7405b040 con 0x55b9a13ee1b0 2024-08-30T09:08:15.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.584+0000 7faf8fb61740 10 client.26452 fetch_fsmap finished waiting for FSMap version 34 2024-08-30T09:08:15.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.584+0000 7faf8fb61740 10 client.26452 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-08-30T09:08:15.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.584+0000 7faf8fb61740 20 client.26452 populate_metadata read hostname 'smithi042' 2024-08-30T09:08:15.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.585+0000 7faf8fb61740 1 --2- 172.21.15.42:0/982753155 >> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] conn(0x55b9a13aada0 0x55b9a18476e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:08:15.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.585+0000 7faf8fb61740 4 client.26452 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-30T09:08:15.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.585+0000 7faf8fb61740 1 -- 172.21.15.42:0/982753155 --> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b9a1731070 con 0x55b9a13aada0 2024-08-30T09:08:15.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.585+0000 7faf827fc640 1 --2- 172.21.15.42:0/982753155 >> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] conn(0x55b9a13aada0 0x55b9a18476e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:08:15.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.585+0000 7faf827fc640 1 --2- 172.21.15.42:0/982753155 >> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] conn(0x55b9a13aada0 0x55b9a18476e0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b657bbd11a7b05a4 server_cookie=e3a3802777dafd8e in_seq=0 out_seq=0 2024-08-30T09:08:15.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.585+0000 7faf66ff5640 10 client.26452 ms_handle_connect on v2:172.21.15.167:6836/3011837856 2024-08-30T09:08:15.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.586+0000 7faf66ff5640 1 -- 172.21.15.42:0/982753155 <== mds.2 v2:172.21.15.167:6836/3011837856 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b9a1731070 con 0x55b9a13aada0 2024-08-30T09:08:15.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.586+0000 7faf66ff5640 10 client.26452 handle_command_reply: tid=0 2024-08-30T09:08:15.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.586+0000 7faf8fb61740 1 client.26452 shutdown 2024-08-30T09:08:15.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.586+0000 7faf8fb61740 1 -- 172.21.15.42:0/982753155 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b9a13ee1b0 msgr2=0x55b9a17cbe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:08:15.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.586+0000 7faf8fb61740 1 --2- 172.21.15.42:0/982753155 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b9a13ee1b0 0x55b9a17cbe50 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7faf74009830 tx=0x7faf740056f0 comp rx=0 tx=0).stop 2024-08-30T09:08:15.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.586+0000 7faf8fb61740 1 -- 172.21.15.42:0/982753155 shutdown_connections 2024-08-30T09:08:15.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.586+0000 7faf8fb61740 1 --2- 172.21.15.42:0/982753155 >> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] conn(0x55b9a13aada0 0x55b9a18476e0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:15.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.586+0000 7faf8fb61740 1 --2- 172.21.15.42:0/982753155 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b9a1417070 0x55b9a17cc8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:15.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.586+0000 7faf66ff5640 0 client.26452 ms_handle_reset on v2:172.21.15.167:6836/3011837856 2024-08-30T09:08:15.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.586+0000 7faf8fb61740 1 --2- 172.21.15.42:0/982753155 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b9a13f6390 0x55b9a17cc390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:15.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.586+0000 7faf8fb61740 1 --2- 172.21.15.42:0/982753155 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b9a13ee1b0 0x55b9a17cbe50 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:15.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.586+0000 7faf8fb61740 1 -- 172.21.15.42:0/982753155 >> 172.21.15.42:0/982753155 conn(0x55b9a140b460 msgr2=0x55b9a17bd340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:08:15.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.586+0000 7faf8fb61740 1 -- 172.21.15.42:0/982753155 shutdown_connections 2024-08-30T09:08:15.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.586+0000 7faf8fb61740 1 -- 172.21.15.42:0/982753155 wait complete. 2024-08-30T09:08:15.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.586+0000 7faf8fb61740 20 client.26452 trim_cache size 0 max 16384 2024-08-30T09:08:15.593 INFO:teuthology.orchestra.run.smithi042.stdout:[] 2024-08-30T09:08:15.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.587+0000 7faf895ec640 1 -- 172.21.15.42:0/751705348 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7faf6c0b4cc0 msgr2=0x7faf6c0b71d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:08:15.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.587+0000 7faf895ec640 1 --2- 172.21.15.42:0/751705348 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7faf6c0b4cc0 0x7faf6c0b71d0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7faf7402a970 tx=0x7faf7402a880 comp rx=0 tx=0).stop 2024-08-30T09:08:15.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.587+0000 7faf895ec640 1 -- 172.21.15.42:0/751705348 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faf8411c9d0 msgr2=0x7faf841ae8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:08:15.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.587+0000 7faf895ec640 1 --2- 172.21.15.42:0/751705348 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faf8411c9d0 0x7faf841ae8e0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7faf7c009fd0 tx=0x7faf7c00ef10 comp rx=0 tx=0).stop 2024-08-30T09:08:15.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.587+0000 7faf895ec640 1 -- 172.21.15.42:0/751705348 shutdown_connections 2024-08-30T09:08:15.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.587+0000 7faf895ec640 1 --2- 172.21.15.42:0/751705348 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7faf6c0b4cc0 0x7faf6c0b71d0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:15.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.587+0000 7faf895ec640 1 --2- 172.21.15.42:0/751705348 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faf8411c9d0 0x7faf841ae8e0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:15.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.587+0000 7faf895ec640 1 --2- 172.21.15.42:0/751705348 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faf84111ce0 0x7faf841a9640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:15.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.587+0000 7faf895ec640 1 --2- 172.21.15.42:0/751705348 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faf8410a780 0x7faf841a90a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:15.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.587+0000 7faf895ec640 1 -- 172.21.15.42:0/751705348 >> 172.21.15.42:0/751705348 conn(0x7faf84070c60 msgr2=0x7faf8411e230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:08:15.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.587+0000 7faf895ec640 1 -- 172.21.15.42:0/751705348 shutdown_connections 2024-08-30T09:08:15.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:15.587+0000 7faf895ec640 1 -- 172.21.15.42:0/751705348 wait complete. 2024-08-30T09:08:15.953 INFO:teuthology.orchestra.run.smithi042.stderr:progress: 450.0 s, 440.6 tps, lat 225.238 ms stddev 270.459 2024-08-30T09:08:16.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:08:15 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-30T09:08:16.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:08:15 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-30T09:08:16.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:08:15 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-30T09:08:16.466 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-08-30T09:08:16.838 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:08:16 smithi042 ceph-mon[18652]: pgmap v598: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 4.4 MiB/s wr, 472 op/s 2024-08-30T09:08:16.948 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T09:08:17.104 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:08:16 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T09:08:17.104 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:08:16 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-30T09:08:17.105 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:08:16 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T09:08:17.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:08:16 smithi190 ceph-mon[22001]: pgmap v598: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 4.4 MiB/s wr, 472 op/s 2024-08-30T09:08:17.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:08:16 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T09:08:17.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:08:16 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-30T09:08:17.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:08:16 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T09:08:17.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:08:16 smithi167 ceph-mon[22063]: pgmap v598: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 4.4 MiB/s wr, 472 op/s 2024-08-30T09:08:17.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:08:16 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T09:08:17.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:08:16 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-30T09:08:17.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:08:16 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T09:08:18.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.557+0000 7f600b46c640 1 -- 172.21.15.42:0/3772557209 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f600410dbb0 msgr2=0x7f60041100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:08:18.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.557+0000 7f600b46c640 1 --2- 172.21.15.42:0/3772557209 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f600410dbb0 0x7f60041100a0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f5ffc009ca0 tx=0x7f5ffc02a8c0 comp rx=0 tx=0).stop 2024-08-30T09:08:18.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.557+0000 7f600b46c640 1 -- 172.21.15.42:0/3772557209 shutdown_connections 2024-08-30T09:08:18.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.557+0000 7f600b46c640 1 --2- 172.21.15.42:0/3772557209 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f60041050a0 0x7f600410d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:18.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.557+0000 7f600b46c640 1 --2- 172.21.15.42:0/3772557209 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f600410dbb0 0x7f60041100a0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:18.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.557+0000 7f600b46c640 1 --2- 172.21.15.42:0/3772557209 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6004104750 0x7f6004104b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:18.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.557+0000 7f600b46c640 1 -- 172.21.15.42:0/3772557209 >> 172.21.15.42:0/3772557209 conn(0x7f60040794d0 msgr2=0x7f60040798e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:08:18.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.557+0000 7f600b46c640 1 -- 172.21.15.42:0/3772557209 shutdown_connections 2024-08-30T09:08:18.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.557+0000 7f600b46c640 1 -- 172.21.15.42:0/3772557209 wait complete. 2024-08-30T09:08:18.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.558+0000 7f600b46c640 1 Processor -- start 2024-08-30T09:08:18.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.558+0000 7f600b46c640 1 -- start start 2024-08-30T09:08:18.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.558+0000 7f600b46c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6004104750 0x7f600419a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:08:18.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.558+0000 7f600b46c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f60041050a0 0x7f600419a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:08:18.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.558+0000 7f600b46c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f600410dbb0 0x7f600419fb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:08:18.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.558+0000 7f600b46c640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f6004113810 con 0x7f60041050a0 2024-08-30T09:08:18.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.558+0000 7f600b46c640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f6004113690 con 0x7f6004104750 2024-08-30T09:08:18.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.558+0000 7f600b46c640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f6004113990 con 0x7f600410dbb0 2024-08-30T09:08:18.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.558+0000 7f6008a07640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f60041050a0 0x7f600419a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:08:18.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.558+0000 7f6008a07640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f60041050a0 0x7f600419a8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38154/0 (socket says 172.21.15.42:38154) 2024-08-30T09:08:18.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.558+0000 7f6008a07640 1 -- 172.21.15.42:0/3637875314 learned_addr learned my addr 172.21.15.42:0/3637875314 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:08:18.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.558+0000 7f6009208640 1 --2- 172.21.15.42:0/3637875314 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6004104750 0x7f600419a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:08:18.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.558+0000 7f6008a07640 1 -- 172.21.15.42:0/3637875314 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f600410dbb0 msgr2=0x7f600419fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:08:18.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.558+0000 7f6008a07640 1 --2- 172.21.15.42:0/3637875314 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f600410dbb0 0x7f600419fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:18.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.558+0000 7f6008a07640 1 -- 172.21.15.42:0/3637875314 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6004104750 msgr2=0x7f600419a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:08:18.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.558+0000 7f6008a07640 1 --2- 172.21.15.42:0/3637875314 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6004104750 0x7f600419a320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:18.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.558+0000 7f6008a07640 1 -- 172.21.15.42:0/3637875314 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60041a0290 con 0x7f60041050a0 2024-08-30T09:08:18.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.559+0000 7f6008a07640 1 --2- 172.21.15.42:0/3637875314 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f60041050a0 0x7f600419a8c0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f5ff800d740 tx=0x7f5ff800dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:08:18.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.559+0000 7f5ff67fc640 1 -- 172.21.15.42:0/3637875314 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ff8017070 con 0x7f60041050a0 2024-08-30T09:08:18.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.559+0000 7f600b46c640 1 -- 172.21.15.42:0/3637875314 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60041a0580 con 0x7f60041050a0 2024-08-30T09:08:18.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.559+0000 7f5ff67fc640 1 -- 172.21.15.42:0/3637875314 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ff800de50 con 0x7f60041050a0 2024-08-30T09:08:18.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.559+0000 7f5ff67fc640 1 -- 172.21.15.42:0/3637875314 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ff800f620 con 0x7f60041050a0 2024-08-30T09:08:18.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.559+0000 7f600b46c640 1 -- 172.21.15.42:0/3637875314 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60041a6ce0 con 0x7f60041050a0 2024-08-30T09:08:18.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.561+0000 7f60119e1740 1 Processor -- start 2024-08-30T09:08:18.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.562+0000 7f60119e1740 1 -- start start 2024-08-30T09:08:18.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.562+0000 7f5ff67fc640 1 -- 172.21.15.42:0/3637875314 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7f5ff801a070 con 0x7f60041050a0 2024-08-30T09:08:18.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.562+0000 7f60119e1740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559e9a4af070 0x559e9a854350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:08:18.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.562+0000 7f60119e1740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559e9a4861b0 0x559e9a854890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:08:18.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.562+0000 7f60119e1740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559e9a48e390 0x559e9a854dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:08:18.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.562+0000 7f60119e1740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x559e9a7cb680 con 0x559e9a4861b0 2024-08-30T09:08:18.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.562+0000 7f60119e1740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x559e9a7bac10 con 0x559e9a4af070 2024-08-30T09:08:18.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.562+0000 7f60119e1740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x559e9a7bebc0 con 0x559e9a48e390 2024-08-30T09:08:18.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.562+0000 7f6009208640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559e9a4861b0 0x559e9a854890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:08:18.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.562+0000 7f6009208640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559e9a4861b0 0x559e9a854890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38170/0 (socket says 172.21.15.42:38170) 2024-08-30T09:08:18.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.562+0000 7f6009208640 1 -- 172.21.15.42:0/3702293902 learned_addr learned my addr 172.21.15.42:0/3702293902 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:08:18.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.562+0000 7f6008a07640 1 --2- 172.21.15.42:0/3702293902 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559e9a48e390 0x559e9a854dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:08:18.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.562+0000 7f6009a09640 1 --2- 172.21.15.42:0/3702293902 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559e9a4af070 0x559e9a854350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:08:18.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.562+0000 7f6009208640 1 -- 172.21.15.42:0/3702293902 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559e9a48e390 msgr2=0x559e9a854dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:08:18.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.562+0000 7f6009208640 1 --2- 172.21.15.42:0/3702293902 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559e9a48e390 0x559e9a854dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:18.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.562+0000 7f6009208640 1 -- 172.21.15.42:0/3702293902 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559e9a4af070 msgr2=0x559e9a854350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:08:18.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.562+0000 7f6009208640 1 --2- 172.21.15.42:0/3702293902 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559e9a4af070 0x559e9a854350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:18.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.562+0000 7f6009208640 1 -- 172.21.15.42:0/3702293902 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e9a51a650 con 0x559e9a4861b0 2024-08-30T09:08:18.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.563+0000 7f6009208640 1 --2- 172.21.15.42:0/3702293902 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559e9a4861b0 0x559e9a854890 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f6000009c20 tx=0x7f60000117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1779e238282e5b server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:08:18.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.563+0000 7f5ff67fc640 1 --2- 172.21.15.42:0/3637875314 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f5ff00b8f40 0x7f5ff00bb450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:08:18.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.563+0000 7f6009a09640 1 --2- 172.21.15.42:0/3637875314 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f5ff00b8f40 0x7f5ff00bb450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:08:18.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.563+0000 7f5fef7fe640 1 -- 172.21.15.42:0/3702293902 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6000012450 con 0x559e9a4861b0 2024-08-30T09:08:18.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.563+0000 7f5ff67fc640 1 -- 172.21.15.42:0/3637875314 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f5ff80e4ef0 con 0x7f60041050a0 2024-08-30T09:08:18.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.563+0000 7f5fef7fe640 1 -- 172.21.15.42:0/3702293902 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6000012aa0 con 0x559e9a4861b0 2024-08-30T09:08:18.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.563+0000 7f5fef7fe640 1 -- 172.21.15.42:0/3702293902 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f600000fa30 con 0x559e9a4861b0 2024-08-30T09:08:18.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.564+0000 7f60119e1740 1 -- 172.21.15.42:0/3702293902 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559e9a4861b0 msgr2=0x559e9a854890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:08:18.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.564+0000 7f6009a09640 1 --2- 172.21.15.42:0/3637875314 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f5ff00b8f40 0x7f5ff00bb450 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f5ffc0094e0 tx=0x7f5ffc0093f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:08:18.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.564+0000 7f60119e1740 1 --2- 172.21.15.42:0/3702293902 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559e9a4861b0 0x559e9a854890 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f6000009c20 tx=0x7f60000117b0 comp rx=0 tx=0).stop 2024-08-30T09:08:18.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.564+0000 7f60119e1740 1 -- 172.21.15.42:0/3702293902 shutdown_connections 2024-08-30T09:08:18.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.564+0000 7f60119e1740 1 --2- 172.21.15.42:0/3702293902 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559e9a48e390 0x559e9a854dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:18.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.564+0000 7f60119e1740 1 --2- 172.21.15.42:0/3702293902 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559e9a4861b0 0x559e9a854890 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:18.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.564+0000 7f60119e1740 1 --2- 172.21.15.42:0/3702293902 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559e9a4af070 0x559e9a854350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:18.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.564+0000 7f60119e1740 1 -- 172.21.15.42:0/3702293902 >> 172.21.15.42:0/3702293902 conn(0x559e9a4a3460 msgr2=0x559e9a852240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:08:18.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.564+0000 7f60119e1740 1 -- 172.21.15.42:0/3702293902 shutdown_connections 2024-08-30T09:08:18.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.564+0000 7f60119e1740 1 -- 172.21.15.42:0/3702293902 wait complete. 2024-08-30T09:08:18.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.564+0000 7f60119e1740 1 -- start start 2024-08-30T09:08:18.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.564+0000 7f60119e1740 1 Processor -- start 2024-08-30T09:08:18.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.565+0000 7f60119e1740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559e9a4861b0 0x559e9a85fa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:08:18.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.565+0000 7f60119e1740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559e9a48e390 0x559e9a85ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:08:18.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.565+0000 7f60119e1740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559e9a4af070 0x559e9a86be50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:08:18.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.565+0000 7f60119e1740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x559e9a61d700 con 0x559e9a4af070 2024-08-30T09:08:18.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.565+0000 7f60119e1740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x559e9a741750 con 0x559e9a48e390 2024-08-30T09:08:18.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.565+0000 7f60119e1740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x559e9a862b30 con 0x559e9a4861b0 2024-08-30T09:08:18.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.565+0000 7f60119e1740 10 client.0 _ll_register_callbacks cb 0x559e9a770e10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:08:18.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.565+0000 7f6009a09640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559e9a4af070 0x559e9a86be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:08:18.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.565+0000 7f6009a09640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559e9a4af070 0x559e9a86be50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38186/0 (socket says 172.21.15.42:38186) 2024-08-30T09:08:18.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.565+0000 7f6009a09640 1 -- 172.21.15.42:0/2893384055 learned_addr learned my addr 172.21.15.42:0/2893384055 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:08:18.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.565+0000 7f6009208640 1 --2- 172.21.15.42:0/2893384055 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559e9a4861b0 0x559e9a85fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:08:18.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.565+0000 7f6008a07640 1 --2- 172.21.15.42:0/2893384055 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559e9a48e390 0x559e9a85ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:08:18.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.565+0000 7f6009a09640 1 -- 172.21.15.42:0/2893384055 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559e9a4861b0 msgr2=0x559e9a85fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:08:18.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.565+0000 7f6009a09640 1 --2- 172.21.15.42:0/2893384055 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559e9a4861b0 0x559e9a85fa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:18.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.565+0000 7f6009a09640 1 -- 172.21.15.42:0/2893384055 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559e9a48e390 msgr2=0x559e9a85ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:08:18.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.565+0000 7f6009a09640 1 --2- 172.21.15.42:0/2893384055 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559e9a48e390 0x559e9a85ffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:18.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.565+0000 7f6009a09640 1 -- 172.21.15.42:0/2893384055 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e9a77a900 con 0x559e9a4af070 2024-08-30T09:08:18.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.565+0000 7f6008a07640 1 --2- 172.21.15.42:0/2893384055 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559e9a48e390 0x559e9a85ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-30T09:08:18.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.566+0000 7f6009a09640 1 --2- 172.21.15.42:0/2893384055 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559e9a4af070 0x559e9a86be50 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f5ffc031700 tx=0x7f5ffc0095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55264517326c5d3c server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:08:18.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.566+0000 7f5fecff9640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-08-30T09:08:18.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.566+0000 7f5fecff9640 1 -- 172.21.15.42:0/2893384055 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ffc04d070 con 0x559e9a4af070 2024-08-30T09:08:18.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.566+0000 7f5fecff9640 1 -- 172.21.15.42:0/2893384055 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f5ffc0045c0 con 0x559e9a4af070 2024-08-30T09:08:18.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.566+0000 7f60119e1740 1 -- 172.21.15.42:0/2893384055 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6000012aa0 con 0x559e9a4af070 2024-08-30T09:08:18.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.566+0000 7f5fecff9640 1 -- 172.21.15.42:0/2893384055 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f5ffc0047c0 con 0x559e9a4af070 2024-08-30T09:08:18.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.566+0000 7f60119e1740 10 client.16704 fetch_fsmap learned FSMap version 34 2024-08-30T09:08:18.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.566+0000 7f60119e1740 1 -- 172.21.15.42:0/2893384055 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x559e9a86e870 con 0x559e9a4af070 2024-08-30T09:08:18.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.566+0000 7f5fecff9640 1 -- 172.21.15.42:0/2893384055 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6000012aa0 con 0x559e9a4af070 2024-08-30T09:08:18.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.566+0000 7f5fecff9640 1 -- 172.21.15.42:0/2893384055 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 34) ==== 8366+0+0 (secure 0 0 0) 0x7f5ffc042040 con 0x559e9a4af070 2024-08-30T09:08:18.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.566+0000 7f60119e1740 10 client.16704 fetch_fsmap finished waiting for FSMap version 34 2024-08-30T09:08:18.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.566+0000 7f60119e1740 10 client.16704 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-08-30T09:08:18.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.566+0000 7f60119e1740 20 client.16704 populate_metadata read hostname 'smithi042' 2024-08-30T09:08:18.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.566+0000 7f60119e1740 1 --2- 172.21.15.42:0/2893384055 >> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] conn(0x559e9a46d9f0 0x559e9a871e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:08:18.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.566+0000 7f60119e1740 4 client.16704 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-30T09:08:18.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.566+0000 7f60119e1740 1 -- 172.21.15.42:0/2893384055 --> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559e9a7cd5d0 con 0x559e9a46d9f0 2024-08-30T09:08:18.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.567+0000 7f6009208640 1 --2- 172.21.15.42:0/2893384055 >> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] conn(0x559e9a46d9f0 0x559e9a871e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:08:18.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.567+0000 7f5fecff9640 10 client.16704 ms_handle_connect on v2:172.21.15.190:6838/719981110 2024-08-30T09:08:18.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.567+0000 7f6009208640 1 --2- 172.21.15.42:0/2893384055 >> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] conn(0x559e9a46d9f0 0x559e9a871e20 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=461584f0c540cd81 server_cookie=5ea2d901301fe69b in_seq=0 out_seq=0 2024-08-30T09:08:18.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.568+0000 7f5fecff9640 1 -- 172.21.15.42:0/2893384055 <== mds.1 v2:172.21.15.190:6838/719981110 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x559e9a7cd5d0 con 0x559e9a46d9f0 2024-08-30T09:08:18.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.568+0000 7f5fecff9640 10 client.16704 handle_command_reply: tid=0 2024-08-30T09:08:18.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.568+0000 7f60119e1740 1 client.16704 shutdown 2024-08-30T09:08:18.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.569+0000 7f60119e1740 1 -- 172.21.15.42:0/2893384055 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559e9a4af070 msgr2=0x559e9a86be50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:08:18.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.569+0000 7f60119e1740 1 --2- 172.21.15.42:0/2893384055 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559e9a4af070 0x559e9a86be50 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f5ffc031700 tx=0x7f5ffc0095c0 comp rx=0 tx=0).stop 2024-08-30T09:08:18.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.569+0000 7f60119e1740 1 -- 172.21.15.42:0/2893384055 shutdown_connections 2024-08-30T09:08:18.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.569+0000 7f60119e1740 1 --2- 172.21.15.42:0/2893384055 >> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] conn(0x559e9a46d9f0 0x559e9a871e20 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:18.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.569+0000 7f60119e1740 1 --2- 172.21.15.42:0/2893384055 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559e9a4af070 0x559e9a86be50 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:18.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.569+0000 7f60119e1740 1 --2- 172.21.15.42:0/2893384055 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559e9a48e390 0x559e9a85ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:18.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.569+0000 7f60119e1740 1 --2- 172.21.15.42:0/2893384055 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559e9a4861b0 0x559e9a85fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:18.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.569+0000 7f60119e1740 1 -- 172.21.15.42:0/2893384055 >> 172.21.15.42:0/2893384055 conn(0x559e9a4a3460 msgr2=0x559e9a8574e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:08:18.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.569+0000 7f5fecff9640 0 client.16704 ms_handle_reset on v2:172.21.15.190:6838/719981110 2024-08-30T09:08:18.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.569+0000 7f60119e1740 1 -- 172.21.15.42:0/2893384055 shutdown_connections 2024-08-30T09:08:18.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.569+0000 7f60119e1740 1 -- 172.21.15.42:0/2893384055 wait complete. 2024-08-30T09:08:18.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.569+0000 7f60119e1740 20 client.16704 trim_cache size 0 max 16384 2024-08-30T09:08:18.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.582+0000 7f60119e1740 1 Processor -- start 2024-08-30T09:08:18.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.582+0000 7f60119e1740 1 -- start start 2024-08-30T09:08:18.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.582+0000 7f60119e1740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559e9a4861b0 0x559e9a857b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:08:18.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.582+0000 7f60119e1740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559e9a48e390 0x559e9a85cc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:08:18.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.582+0000 7f60119e1740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559e9a4af070 0x559e9a85d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:08:18.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.582+0000 7f60119e1740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f60041b1160 con 0x559e9a48e390 2024-08-30T09:08:18.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.582+0000 7f60119e1740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f60041ae9d0 con 0x559e9a4861b0 2024-08-30T09:08:18.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.582+0000 7f60119e1740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f60041ac820 con 0x559e9a4af070 2024-08-30T09:08:18.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.583+0000 7f6008a07640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559e9a48e390 0x559e9a85cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:08:18.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.583+0000 7f6008a07640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559e9a48e390 0x559e9a85cc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38198/0 (socket says 172.21.15.42:38198) 2024-08-30T09:08:18.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.583+0000 7f6008a07640 1 -- 172.21.15.42:0/776450723 learned_addr learned my addr 172.21.15.42:0/776450723 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:08:18.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.583+0000 7f6009208640 1 --2- 172.21.15.42:0/776450723 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559e9a4861b0 0x559e9a857b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:08:18.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.583+0000 7f6009a09640 1 --2- 172.21.15.42:0/776450723 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559e9a4af070 0x559e9a85d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:08:18.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.583+0000 7f6008a07640 1 -- 172.21.15.42:0/776450723 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559e9a4af070 msgr2=0x559e9a85d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:08:18.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.583+0000 7f6008a07640 1 --2- 172.21.15.42:0/776450723 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559e9a4af070 0x559e9a85d1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:18.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.583+0000 7f6008a07640 1 -- 172.21.15.42:0/776450723 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559e9a4861b0 msgr2=0x559e9a857b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:08:18.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.583+0000 7f6008a07640 1 --2- 172.21.15.42:0/776450723 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559e9a4861b0 0x559e9a857b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:18.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.583+0000 7f6008a07640 1 -- 172.21.15.42:0/776450723 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e9a860d30 con 0x559e9a48e390 2024-08-30T09:08:18.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.583+0000 7f6008a07640 1 --2- 172.21.15.42:0/776450723 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559e9a48e390 0x559e9a85cc60 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f5ff80e2600 tx=0x7f5ff8010c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a904702120ac0719 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:08:18.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.583+0000 7f5feeffd640 1 -- 172.21.15.42:0/776450723 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ff8021070 con 0x559e9a48e390 2024-08-30T09:08:18.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.583+0000 7f5feeffd640 1 -- 172.21.15.42:0/776450723 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ff80b4040 con 0x559e9a48e390 2024-08-30T09:08:18.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.583+0000 7f5feeffd640 1 -- 172.21.15.42:0/776450723 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ff80a7820 con 0x559e9a48e390 2024-08-30T09:08:18.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.584+0000 7f60119e1740 1 -- 172.21.15.42:0/776450723 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559e9a48e390 msgr2=0x559e9a85cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:08:18.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.584+0000 7f60119e1740 1 --2- 172.21.15.42:0/776450723 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559e9a48e390 0x559e9a85cc60 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f5ff80e2600 tx=0x7f5ff8010c70 comp rx=0 tx=0).stop 2024-08-30T09:08:18.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.584+0000 7f60119e1740 1 -- 172.21.15.42:0/776450723 shutdown_connections 2024-08-30T09:08:18.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.584+0000 7f60119e1740 1 --2- 172.21.15.42:0/776450723 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559e9a4af070 0x559e9a85d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:18.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.584+0000 7f60119e1740 1 --2- 172.21.15.42:0/776450723 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559e9a48e390 0x559e9a85cc60 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:18.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.584+0000 7f60119e1740 1 --2- 172.21.15.42:0/776450723 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559e9a4861b0 0x559e9a857b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:18.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.584+0000 7f60119e1740 1 -- 172.21.15.42:0/776450723 >> 172.21.15.42:0/776450723 conn(0x559e9a4a3460 msgr2=0x559e9a857450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:08:18.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.584+0000 7f60119e1740 1 -- 172.21.15.42:0/776450723 shutdown_connections 2024-08-30T09:08:18.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.584+0000 7f60119e1740 1 -- 172.21.15.42:0/776450723 wait complete. 2024-08-30T09:08:18.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.584+0000 7f60119e1740 1 -- start start 2024-08-30T09:08:18.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.584+0000 7f60119e1740 1 Processor -- start 2024-08-30T09:08:18.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.585+0000 7f60119e1740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559e9a4861b0 0x559e9a868370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:08:18.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.585+0000 7f60119e1740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559e9a48e390 0x559e9a864830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:08:18.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.585+0000 7f60119e1740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559e9a4af070 0x559e9a864d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:08:18.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.585+0000 7f60119e1740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f60041ace90 con 0x559e9a48e390 2024-08-30T09:08:18.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.585+0000 7f60119e1740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f60041ae300 con 0x559e9a4af070 2024-08-30T09:08:18.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.585+0000 7f60119e1740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f60041ad560 con 0x559e9a4861b0 2024-08-30T09:08:18.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.585+0000 7f60119e1740 10 client.0 _ll_register_callbacks cb 0x7f6000005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:08:18.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.585+0000 7f6008a07640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559e9a48e390 0x559e9a864830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:08:18.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.585+0000 7f6008a07640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559e9a48e390 0x559e9a864830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38200/0 (socket says 172.21.15.42:38200) 2024-08-30T09:08:18.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.585+0000 7f6008a07640 1 -- 172.21.15.42:0/473842526 learned_addr learned my addr 172.21.15.42:0/473842526 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:08:18.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.585+0000 7f6009208640 1 --2- 172.21.15.42:0/473842526 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559e9a4861b0 0x559e9a868370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:08:18.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.585+0000 7f6009a09640 1 --2- 172.21.15.42:0/473842526 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559e9a4af070 0x559e9a864d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:08:18.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.585+0000 7f6008a07640 1 -- 172.21.15.42:0/473842526 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559e9a4861b0 msgr2=0x559e9a868370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:08:18.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.585+0000 7f6008a07640 1 --2- 172.21.15.42:0/473842526 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559e9a4861b0 0x559e9a868370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:18.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.585+0000 7f6008a07640 1 -- 172.21.15.42:0/473842526 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559e9a4af070 msgr2=0x559e9a864d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:08:18.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.585+0000 7f6008a07640 1 --2- 172.21.15.42:0/473842526 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559e9a4af070 0x559e9a864d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:18.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.585+0000 7f6008a07640 1 -- 172.21.15.42:0/473842526 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e9a8652e0 con 0x559e9a48e390 2024-08-30T09:08:18.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.585+0000 7f6008a07640 1 --2- 172.21.15.42:0/473842526 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559e9a48e390 0x559e9a864830 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f5ff80e52a0 tx=0x7f5ff8009e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4d724843de2d06e server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:08:18.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.585+0000 7f5fecff9640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-08-30T09:08:18.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.586+0000 7f5fecff9640 1 -- 172.21.15.42:0/473842526 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ff8021070 con 0x559e9a48e390 2024-08-30T09:08:18.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.586+0000 7f5fecff9640 1 -- 172.21.15.42:0/473842526 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f5ff80d21c0 con 0x559e9a48e390 2024-08-30T09:08:18.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.586+0000 7f60119e1740 1 -- 172.21.15.42:0/473842526 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5ff80b4040 con 0x559e9a48e390 2024-08-30T09:08:18.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.586+0000 7f5fecff9640 1 -- 172.21.15.42:0/473842526 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f5ff80ab0a0 con 0x559e9a48e390 2024-08-30T09:08:18.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.586+0000 7f60119e1740 10 client.16710 fetch_fsmap learned FSMap version 34 2024-08-30T09:08:18.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.586+0000 7f60119e1740 1 -- 172.21.15.42:0/473842526 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x559e9a8dd550 con 0x559e9a48e390 2024-08-30T09:08:18.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.586+0000 7f5fecff9640 1 -- 172.21.15.42:0/473842526 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ff80b4040 con 0x559e9a48e390 2024-08-30T09:08:18.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.586+0000 7f5fecff9640 1 -- 172.21.15.42:0/473842526 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 34) ==== 8366+0+0 (secure 0 0 0) 0x7f5ff80ce0b0 con 0x559e9a48e390 2024-08-30T09:08:18.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.586+0000 7f60119e1740 10 client.16710 fetch_fsmap finished waiting for FSMap version 34 2024-08-30T09:08:18.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.586+0000 7f60119e1740 10 client.16710 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-08-30T09:08:18.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.586+0000 7f60119e1740 20 client.16710 populate_metadata read hostname 'smithi042' 2024-08-30T09:08:18.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.586+0000 7f60119e1740 1 --2- 172.21.15.42:0/473842526 >> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] conn(0x559e9a442da0 0x559e9a8dfc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:08:18.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.586+0000 7f60119e1740 4 client.16710 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-30T09:08:18.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.586+0000 7f60119e1740 1 -- 172.21.15.42:0/473842526 --> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559e9a7c9070 con 0x559e9a442da0 2024-08-30T09:08:18.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.587+0000 7f6009208640 1 --2- 172.21.15.42:0/473842526 >> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] conn(0x559e9a442da0 0x559e9a8dfc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:08:18.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.587+0000 7f5fecff9640 10 client.16710 ms_handle_connect on v2:172.21.15.190:6838/719981110 2024-08-30T09:08:18.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.587+0000 7f6009208640 1 --2- 172.21.15.42:0/473842526 >> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] conn(0x559e9a442da0 0x559e9a8dfc60 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9e86cc61cd8fa87b server_cookie=dadabd51321edf0a in_seq=0 out_seq=0 2024-08-30T09:08:18.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.587+0000 7f5fecff9640 1 -- 172.21.15.42:0/473842526 <== mds.1 v2:172.21.15.190:6838/719981110 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559e9a7c9070 con 0x559e9a442da0 2024-08-30T09:08:18.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.587+0000 7f5fecff9640 10 client.16710 handle_command_reply: tid=0 2024-08-30T09:08:18.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.587+0000 7f60119e1740 1 client.16710 shutdown 2024-08-30T09:08:18.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.587+0000 7f60119e1740 1 -- 172.21.15.42:0/473842526 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559e9a48e390 msgr2=0x559e9a864830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:08:18.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.587+0000 7f60119e1740 1 --2- 172.21.15.42:0/473842526 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559e9a48e390 0x559e9a864830 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f5ff80e52a0 tx=0x7f5ff8009e40 comp rx=0 tx=0).stop 2024-08-30T09:08:18.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.587+0000 7f60119e1740 1 -- 172.21.15.42:0/473842526 shutdown_connections 2024-08-30T09:08:18.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.587+0000 7f60119e1740 1 --2- 172.21.15.42:0/473842526 >> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] conn(0x559e9a442da0 0x559e9a8dfc60 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:18.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.587+0000 7f60119e1740 1 --2- 172.21.15.42:0/473842526 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559e9a4af070 0x559e9a864d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:18.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.587+0000 7f60119e1740 1 --2- 172.21.15.42:0/473842526 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559e9a48e390 0x559e9a864830 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:18.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.587+0000 7f60119e1740 1 --2- 172.21.15.42:0/473842526 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559e9a4861b0 0x559e9a868370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:18.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.587+0000 7f5fecff9640 0 client.16710 ms_handle_reset on v2:172.21.15.190:6838/719981110 2024-08-30T09:08:18.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.587+0000 7f60119e1740 1 -- 172.21.15.42:0/473842526 >> 172.21.15.42:0/473842526 conn(0x559e9a4a3460 msgr2=0x559e9a855380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:08:18.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.588+0000 7f60119e1740 1 -- 172.21.15.42:0/473842526 shutdown_connections 2024-08-30T09:08:18.029 INFO:teuthology.orchestra.run.smithi042.stdout:[] 2024-08-30T09:08:18.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.588+0000 7f60119e1740 1 -- 172.21.15.42:0/473842526 wait complete. 2024-08-30T09:08:18.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.588+0000 7f60119e1740 20 client.16710 trim_cache size 0 max 16384 2024-08-30T09:08:18.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.588+0000 7f600b46c640 1 -- 172.21.15.42:0/3637875314 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f5ff00b8f40 msgr2=0x7f5ff00bb450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:08:18.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.588+0000 7f600b46c640 1 --2- 172.21.15.42:0/3637875314 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f5ff00b8f40 0x7f5ff00bb450 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f5ffc0094e0 tx=0x7f5ffc0093f0 comp rx=0 tx=0).stop 2024-08-30T09:08:18.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.588+0000 7f600b46c640 1 -- 172.21.15.42:0/3637875314 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f60041050a0 msgr2=0x7f600419a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:08:18.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.588+0000 7f600b46c640 1 --2- 172.21.15.42:0/3637875314 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f60041050a0 0x7f600419a8c0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f5ff800d740 tx=0x7f5ff800dc00 comp rx=0 tx=0).stop 2024-08-30T09:08:18.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.588+0000 7f600b46c640 1 -- 172.21.15.42:0/3637875314 shutdown_connections 2024-08-30T09:08:18.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.588+0000 7f600b46c640 1 --2- 172.21.15.42:0/3637875314 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f5ff00b8f40 0x7f5ff00bb450 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:18.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.588+0000 7f600b46c640 1 --2- 172.21.15.42:0/3637875314 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f600410dbb0 0x7f600419fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:18.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.588+0000 7f600b46c640 1 --2- 172.21.15.42:0/3637875314 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f60041050a0 0x7f600419a8c0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:18.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.588+0000 7f600b46c640 1 --2- 172.21.15.42:0/3637875314 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6004104750 0x7f600419a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:18.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.588+0000 7f600b46c640 1 -- 172.21.15.42:0/3637875314 >> 172.21.15.42:0/3637875314 conn(0x7f60040794d0 msgr2=0x7f60040ff570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:08:18.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.588+0000 7f600b46c640 1 -- 172.21.15.42:0/3637875314 shutdown_connections 2024-08-30T09:08:18.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:17.588+0000 7f600b46c640 1 -- 172.21.15.42:0/3637875314 wait complete. 2024-08-30T09:08:19.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:08:18 smithi042 ceph-mon[18652]: pgmap v599: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 3.9 MiB/s wr, 401 op/s 2024-08-30T09:08:19.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:08:18 smithi190 ceph-mon[22001]: pgmap v599: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 3.9 MiB/s wr, 401 op/s 2024-08-30T09:08:19.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:08:18 smithi167 ceph-mon[22063]: pgmap v599: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 3.9 MiB/s wr, 401 op/s 2024-08-30T09:08:20.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:08:19 smithi042 ceph-mon[18652]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-30T09:08:20.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:08:19 smithi190 ceph-mon[22001]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-30T09:08:20.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:08:19 smithi167 ceph-mon[22063]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-30T09:08:20.645 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-08-30T09:08:20.953 INFO:teuthology.orchestra.run.smithi042.stderr:progress: 455.0 s, 529.6 tps, lat 189.521 ms stddev 263.586 2024-08-30T09:08:21.124 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T09:08:21.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:08:20 smithi042 ceph-mon[18652]: pgmap v600: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.2 MiB/s wr, 522 op/s 2024-08-30T09:08:21.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:08:20 smithi190 ceph-mon[22001]: pgmap v600: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.2 MiB/s wr, 522 op/s 2024-08-30T09:08:21.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:08:20 smithi167 ceph-mon[22063]: pgmap v600: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.2 MiB/s wr, 522 op/s 2024-08-30T09:08:21.821 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:21.819+0000 7fae9c88d640 1 -- 172.21.15.42:0/3757423149 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fae98075a90 msgr2=0x7fae98075ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:08:21.821 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:21.819+0000 7fae9c88d640 1 --2- 172.21.15.42:0/3757423149 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fae98075a90 0x7fae98075ea0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fae90009ca0 tx=0x7fae9002a8c0 comp rx=0 tx=0).stop 2024-08-30T09:08:21.822 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:21.819+0000 7fae9c88d640 1 -- 172.21.15.42:0/3757423149 shutdown_connections 2024-08-30T09:08:21.822 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:21.819+0000 7fae9c88d640 1 --2- 172.21.15.42:0/3757423149 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fae9810fbe0 0x7fae98112000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:21.822 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:21.819+0000 7fae9c88d640 1 --2- 172.21.15.42:0/3757423149 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fae98075a90 0x7fae98075ea0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:21.822 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:21.819+0000 7fae9c88d640 1 --2- 172.21.15.42:0/3757423149 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fae98076e90 0x7fae98075550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:21.822 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:21.819+0000 7fae9c88d640 1 -- 172.21.15.42:0/3757423149 >> 172.21.15.42:0/3757423149 conn(0x7fae980fedc0 msgr2=0x7fae98101250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:08:21.822 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:21.819+0000 7fae9c88d640 1 -- 172.21.15.42:0/3757423149 shutdown_connections 2024-08-30T09:08:21.822 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:21.819+0000 7fae9c88d640 1 -- 172.21.15.42:0/3757423149 wait complete. 2024-08-30T09:08:21.824 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:21.820+0000 7fae9c88d640 1 Processor -- start 2024-08-30T09:08:21.824 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:21.820+0000 7fae9c88d640 1 -- start start 2024-08-30T09:08:21.824 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:21.820+0000 7fae9c88d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fae98075a90 0x7fae981a0140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:08:21.824 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:21.820+0000 7fae9c88d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fae98076e90 0x7fae981a07b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:08:21.824 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:21.820+0000 7fae9c88d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fae9810fbe0 0x7fae981a1360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:08:21.824 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:21.820+0000 7fae9c88d640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fae981156b0 con 0x7fae98075a90 2024-08-30T09:08:21.825 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:21.820+0000 7fae9c88d640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fae98115530 con 0x7fae98076e90 2024-08-30T09:08:21.825 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:21.820+0000 7fae9c88d640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fae98115830 con 0x7fae9810fbe0 2024-08-30T09:08:21.825 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:21.821+0000 7fae9659c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fae98075a90 0x7fae981a0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:08:21.825 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:21.821+0000 7fae9659c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fae98075a90 0x7fae981a0140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:50836/0 (socket says 172.21.15.42:50836) 2024-08-30T09:08:21.825 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:21.821+0000 7fae9659c640 1 -- 172.21.15.42:0/1839116542 learned_addr learned my addr 172.21.15.42:0/1839116542 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:08:21.825 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:21.821+0000 7fae96d9d640 1 --2- 172.21.15.42:0/1839116542 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fae9810fbe0 0x7fae981a1360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:08:21.825 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:21.821+0000 7fae95d9b640 1 --2- 172.21.15.42:0/1839116542 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fae98076e90 0x7fae981a07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:08:21.825 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:21.821+0000 7fae9659c640 1 -- 172.21.15.42:0/1839116542 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fae9810fbe0 msgr2=0x7fae981a1360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:08:21.825 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:21.821+0000 7fae9659c640 1 --2- 172.21.15.42:0/1839116542 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fae9810fbe0 0x7fae981a1360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:21.825 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:21.821+0000 7fae9659c640 1 -- 172.21.15.42:0/1839116542 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fae98076e90 msgr2=0x7fae981a07b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:08:21.825 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:21.821+0000 7fae9659c640 1 --2- 172.21.15.42:0/1839116542 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fae98076e90 0x7fae981a07b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:21.825 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:21.821+0000 7fae9659c640 1 -- 172.21.15.42:0/1839116542 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae981a1b80 con 0x7fae98075a90 2024-08-30T09:08:21.825 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:21.821+0000 7fae9659c640 1 --2- 172.21.15.42:0/1839116542 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fae98075a90 0x7fae981a0140 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fae8c00e9f0 tx=0x7fae8c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:08:21.825 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:21.821+0000 7fae877fe640 1 -- 172.21.15.42:0/1839116542 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae8c00cd80 con 0x7fae98075a90 2024-08-30T09:08:21.825 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:21.821+0000 7fae877fe640 1 -- 172.21.15.42:0/1839116542 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae8c0054d0 con 0x7fae98075a90 2024-08-30T09:08:21.826 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:21.821+0000 7fae9c88d640 1 -- 172.21.15.42:0/1839116542 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae981a1e70 con 0x7fae98075a90 2024-08-30T09:08:21.826 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:21.821+0000 7fae877fe640 1 -- 172.21.15.42:0/1839116542 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae8c010690 con 0x7fae98075a90 2024-08-30T09:08:21.826 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:21.822+0000 7fae9c88d640 1 -- 172.21.15.42:0/1839116542 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae981a8560 con 0x7fae98075a90 2024-08-30T09:08:21.827 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:21.825+0000 7fae877fe640 1 -- 172.21.15.42:0/1839116542 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7fae8c004e90 con 0x7fae98075a90 2024-08-30T09:08:21.828 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:21.826+0000 7fae877fe640 1 --2- 172.21.15.42:0/1839116542 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fae800b4b30 0x7fae800b7040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:08:21.829 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:21.826+0000 7fae877fe640 1 -- 172.21.15.42:0/1839116542 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7fae8c0e5910 con 0x7fae98075a90 2024-08-30T09:08:21.829 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:21.826+0000 7fae95d9b640 1 --2- 172.21.15.42:0/1839116542 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fae800b4b30 0x7fae800b7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:08:21.829 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:21.827+0000 7fae95d9b640 1 --2- 172.21.15.42:0/1839116542 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fae800b4b30 0x7fae800b7040 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fae900094e0 tx=0x7fae90000f40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:08:21.863 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:21.861+0000 7faea37fd740 1 Processor -- start 2024-08-30T09:08:21.867 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:21.865+0000 7faea37fd740 1 -- start start 2024-08-30T09:08:21.867 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:21.865+0000 7faea37fd740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cd690ec1b0 0x55cd69492180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:08:21.867 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:21.865+0000 7faea37fd740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cd690c32f0 0x55cd694927f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:08:21.869 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:21.866+0000 7faea37fd740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cd690cb4d0 0x55cd69493340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:08:21.869 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:21.866+0000 7faea37fd740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55cd69405000 con 0x55cd690cb4d0 2024-08-30T09:08:21.869 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:21.866+0000 7faea37fd740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55cd693f4590 con 0x55cd690ec1b0 2024-08-30T09:08:21.870 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:21.866+0000 7faea37fd740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55cd693f8540 con 0x55cd690c32f0 2024-08-30T09:08:21.870 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:21.866+0000 7fae95d9b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cd690c32f0 0x55cd694927f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:08:21.870 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:21.866+0000 7fae95d9b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cd690c32f0 0x55cd694927f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:60894/0 (socket says 172.21.15.42:60894) 2024-08-30T09:08:21.870 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:21.866+0000 7fae95d9b640 1 -- 172.21.15.42:0/1340112334 learned_addr learned my addr 172.21.15.42:0/1340112334 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:08:21.870 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:21.866+0000 7fae9659c640 1 --2- 172.21.15.42:0/1340112334 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cd690ec1b0 0x55cd69492180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:08:21.870 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:21.866+0000 7fae95d9b640 1 -- 172.21.15.42:0/1340112334 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cd690ec1b0 msgr2=0x55cd69492180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:08:21.870 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:21.867+0000 7fae95d9b640 1 --2- 172.21.15.42:0/1340112334 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cd690ec1b0 0x55cd69492180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:21.870 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:21.867+0000 7fae95d9b640 1 -- 172.21.15.42:0/1340112334 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cd690cb4d0 msgr2=0x55cd69493340 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-30T09:08:21.870 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:21.867+0000 7fae95d9b640 1 --2- 172.21.15.42:0/1340112334 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cd690cb4d0 0x55cd69493340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:21.870 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:21.867+0000 7fae95d9b640 1 -- 172.21.15.42:0/1340112334 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cd69157790 con 0x55cd690c32f0 2024-08-30T09:08:21.870 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:21.867+0000 7fae95d9b640 1 --2- 172.21.15.42:0/1340112334 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cd690c32f0 0x55cd694927f0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fae90031d60 tx=0x7fae9003f000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=925581da0e2c78a9 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:08:21.870 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:21.867+0000 7fae9659c640 1 --2- 172.21.15.42:0/1340112334 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cd690ec1b0 0x55cd69492180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-30T09:08:21.879 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:21.877+0000 7fae84ff9640 1 -- 172.21.15.42:0/1340112334 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae90003d80 con 0x55cd690c32f0 2024-08-30T09:08:21.879 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:21.877+0000 7fae84ff9640 1 -- 172.21.15.42:0/1340112334 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae9004f070 con 0x55cd690c32f0 2024-08-30T09:08:21.879 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:21.877+0000 7fae84ff9640 1 -- 172.21.15.42:0/1340112334 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae9002f950 con 0x55cd690c32f0 2024-08-30T09:08:21.880 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:21.878+0000 7faea37fd740 1 -- 172.21.15.42:0/1340112334 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cd690c32f0 msgr2=0x55cd694927f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:08:21.880 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:21.878+0000 7faea37fd740 1 --2- 172.21.15.42:0/1340112334 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cd690c32f0 0x55cd694927f0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fae90031d60 tx=0x7fae9003f000 comp rx=0 tx=0).stop 2024-08-30T09:08:21.886 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:21.884+0000 7faea37fd740 1 -- 172.21.15.42:0/1340112334 shutdown_connections 2024-08-30T09:08:21.886 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:21.884+0000 7faea37fd740 1 --2- 172.21.15.42:0/1340112334 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cd690c32f0 0x55cd694927f0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:21.886 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:21.884+0000 7faea37fd740 1 --2- 172.21.15.42:0/1340112334 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cd690cb4d0 0x55cd69493340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:21.886 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:21.884+0000 7faea37fd740 1 --2- 172.21.15.42:0/1340112334 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cd690ec1b0 0x55cd69492180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:21.886 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:21.884+0000 7faea37fd740 1 -- 172.21.15.42:0/1340112334 >> 172.21.15.42:0/1340112334 conn(0x55cd690e0300 msgr2=0x55cd6948dc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:08:21.903 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:21.901+0000 7faea37fd740 1 -- 172.21.15.42:0/1340112334 shutdown_connections 2024-08-30T09:08:21.903 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:21.901+0000 7faea37fd740 1 -- 172.21.15.42:0/1340112334 wait complete. 2024-08-30T09:08:21.904 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:21.902+0000 7faea37fd740 1 -- start start 2024-08-30T09:08:21.904 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:21.902+0000 7faea37fd740 1 Processor -- start 2024-08-30T09:08:21.911 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:21.908+0000 7faea37fd740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cd690c32f0 0x55cd6949bae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:08:21.911 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:21.908+0000 7faea37fd740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cd690cb4d0 0x55cd6949c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:08:21.911 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:21.908+0000 7faea37fd740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cd690ec1b0 0x55cd6949cd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:08:21.911 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:21.908+0000 7faea37fd740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55cd6932dab0 con 0x55cd690c32f0 2024-08-30T09:08:21.911 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:21.908+0000 7faea37fd740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55cd6937b680 con 0x55cd690ec1b0 2024-08-30T09:08:21.911 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:21.908+0000 7faea37fd740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55cd69493c80 con 0x55cd690cb4d0 2024-08-30T09:08:21.911 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:21.908+0000 7fae95d9b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cd690cb4d0 0x55cd6949c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:08:21.911 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:21.908+0000 7fae95d9b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cd690cb4d0 0x55cd6949c1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:60900/0 (socket says 172.21.15.42:60900) 2024-08-30T09:08:21.912 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:21.908+0000 7fae95d9b640 1 -- 172.21.15.42:0/1332236004 learned_addr learned my addr 172.21.15.42:0/1332236004 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:08:21.912 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:21.909+0000 7faea37fd740 10 client.0 _ll_register_callbacks cb 0x55cd69103a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:08:21.915 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:21.913+0000 7fae95d9b640 1 -- 172.21.15.42:0/1332236004 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cd690ec1b0 msgr2=0x55cd6949cd10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-30T09:08:21.916 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:21.913+0000 7fae95d9b640 1 --2- 172.21.15.42:0/1332236004 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cd690ec1b0 0x55cd6949cd10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:21.916 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:21.913+0000 7fae95d9b640 1 -- 172.21.15.42:0/1332236004 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cd690c32f0 msgr2=0x55cd6949bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:08:21.916 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:21.913+0000 7fae95d9b640 1 --2- 172.21.15.42:0/1332236004 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cd690c32f0 0x55cd6949bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:21.916 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:21.913+0000 7fae95d9b640 1 -- 172.21.15.42:0/1332236004 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cd691d0620 con 0x55cd690cb4d0 2024-08-30T09:08:21.916 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:21.913+0000 7fae95d9b640 1 --2- 172.21.15.42:0/1332236004 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cd690cb4d0 0x55cd6949c1a0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fae9002f930 tx=0x7fae9002f960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c89f9e8eac4a1ef8 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:08:21.918 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:21.916+0000 7fae7dffb640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-08-30T09:08:21.921 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:21.919+0000 7fae7dffb640 1 -- 172.21.15.42:0/1332236004 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae90004450 con 0x55cd690cb4d0 2024-08-30T09:08:21.921 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:21.919+0000 7fae7dffb640 1 -- 172.21.15.42:0/1332236004 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7fae900045f0 con 0x55cd690cb4d0 2024-08-30T09:08:21.921 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:21.920+0000 7faea37fd740 1 -- 172.21.15.42:0/1332236004 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae9004f070 con 0x55cd690cb4d0 2024-08-30T09:08:21.924 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:21.922+0000 7fae7dffb640 1 -- 172.21.15.42:0/1332236004 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fae90031e50 con 0x55cd690cb4d0 2024-08-30T09:08:21.924 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:21.922+0000 7faea37fd740 10 client.26479 fetch_fsmap learned FSMap version 35 2024-08-30T09:08:21.924 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:21.922+0000 7faea37fd740 1 -- 172.21.15.42:0/1332236004 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55cd694a5c40 con 0x55cd690cb4d0 2024-08-30T09:08:21.925 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:21.923+0000 7fae7dffb640 1 -- 172.21.15.42:0/1332236004 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae9004f070 con 0x55cd690cb4d0 2024-08-30T09:08:21.925 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:21.923+0000 7fae7dffb640 1 -- 172.21.15.42:0/1332236004 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 35) ==== 8362+0+0 (secure 0 0 0) 0x7fae90043040 con 0x55cd690cb4d0 2024-08-30T09:08:21.925 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:21.923+0000 7faea37fd740 10 client.26479 fetch_fsmap finished waiting for FSMap version 35 2024-08-30T09:08:21.925 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:21.923+0000 7faea37fd740 10 client.26479 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-30T09:08:21.926 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:21.923+0000 7faea37fd740 20 client.26479 populate_metadata read hostname 'smithi042' 2024-08-30T09:08:21.926 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:21.923+0000 7faea37fd740 1 --2- 172.21.15.42:0/1332236004 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55cd690aad80 0x55cd694a9450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:08:21.926 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:21.923+0000 7faea37fd740 4 client.26479 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-30T09:08:21.926 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:21.923+0000 7faea37fd740 1 -- 172.21.15.42:0/1332236004 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cd69406f50 con 0x55cd690aad80 2024-08-30T09:08:21.928 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:21.926+0000 7fae9659c640 1 --2- 172.21.15.42:0/1332236004 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55cd690aad80 0x55cd694a9450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:08:21.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:21.931+0000 7f24e359e640 1 -- 172.21.15.42:0/4050965382 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f24e410d990 msgr2=0x7f24e410dda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:08:21.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:21.931+0000 7f24e359e640 1 --2- 172.21.15.42:0/4050965382 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f24e410d990 0x7f24e410dda0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f24dc01acb0 tx=0x7f24dc03b920 comp rx=0 tx=0).stop 2024-08-30T09:08:21.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:21.935+0000 7f24e359e640 1 -- 172.21.15.42:0/4050965382 shutdown_connections 2024-08-30T09:08:21.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:21.935+0000 7f24e359e640 1 --2- 172.21.15.42:0/4050965382 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f24e411c940 0x7f24e411ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:21.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:21.935+0000 7f24e359e640 1 --2- 172.21.15.42:0/4050965382 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f24e410d990 0x7f24e410dda0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:21.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:21.935+0000 7f24e359e640 1 --2- 172.21.15.42:0/4050965382 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f24e4111a70 0x7f24e410d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:21.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:21.935+0000 7f24e359e640 1 -- 172.21.15.42:0/4050965382 >> 172.21.15.42:0/4050965382 conn(0x7f24e406e790 msgr2=0x7f24e406ebd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:08:21.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:21.937+0000 7f24e359e640 1 -- 172.21.15.42:0/4050965382 shutdown_connections 2024-08-30T09:08:21.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:21.937+0000 7f24e359e640 1 -- 172.21.15.42:0/4050965382 wait complete. 2024-08-30T09:08:21.943 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:21.938+0000 7fae7dffb640 10 client.26479 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:08:21.943 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:21.938+0000 7fae9659c640 1 --2- 172.21.15.42:0/1332236004 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55cd690aad80 0x55cd694a9450 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=16ad8ba6dca12c30 server_cookie=fc5a29679544050d in_seq=0 out_seq=0 2024-08-30T09:08:21.947 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:21.942+0000 7fae7dffb640 1 -- 172.21.15.42:0/1332236004 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55cd69406f50 con 0x55cd690aad80 2024-08-30T09:08:21.947 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:21.942+0000 7fae7dffb640 10 client.26479 handle_command_reply: tid=0 2024-08-30T09:08:21.947 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:21.942+0000 7faea37fd740 1 client.26479 shutdown 2024-08-30T09:08:21.947 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:21.943+0000 7faea37fd740 1 -- 172.21.15.42:0/1332236004 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cd690cb4d0 msgr2=0x55cd6949c1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:08:21.947 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:21.943+0000 7faea37fd740 1 --2- 172.21.15.42:0/1332236004 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cd690cb4d0 0x55cd6949c1a0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fae9002f930 tx=0x7fae9002f960 comp rx=0 tx=0).stop 2024-08-30T09:08:21.947 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:21.944+0000 7faea37fd740 1 -- 172.21.15.42:0/1332236004 shutdown_connections 2024-08-30T09:08:21.947 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:21.944+0000 7faea37fd740 1 --2- 172.21.15.42:0/1332236004 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55cd690aad80 0x55cd694a9450 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:21.947 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:21.944+0000 7faea37fd740 1 --2- 172.21.15.42:0/1332236004 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cd690cb4d0 0x55cd6949c1a0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:21.947 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:21.944+0000 7faea37fd740 1 --2- 172.21.15.42:0/1332236004 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cd690ec1b0 0x55cd6949cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:21.947 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:21.944+0000 7fae7dffb640 0 client.26479 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:08:21.947 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:21.944+0000 7faea37fd740 1 --2- 172.21.15.42:0/1332236004 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cd690c32f0 0x55cd6949bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:21.947 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:21.944+0000 7faea37fd740 1 -- 172.21.15.42:0/1332236004 >> 172.21.15.42:0/1332236004 conn(0x55cd690e0300 msgr2=0x55cd6948c170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:08:21.949 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:21.938+0000 7f24e359e640 1 Processor -- start 2024-08-30T09:08:21.949 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:21.938+0000 7f24e359e640 1 -- start start 2024-08-30T09:08:21.949 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:21.938+0000 7f24e359e640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f24e410d990 0x7f24e411c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:08:21.949 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:21.938+0000 7f24e359e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f24e4111a70 0x7f24e41183e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:08:21.949 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:21.938+0000 7f24e359e640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f24e411c940 0x7f24e4118920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:08:21.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:21.938+0000 7f24e359e640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f24e4114990 con 0x7f24e410d990 2024-08-30T09:08:21.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:21.938+0000 7f24e359e640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f24e4114810 con 0x7f24e411c940 2024-08-30T09:08:21.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:21.938+0000 7f24e359e640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f24e4114b10 con 0x7f24e4111a70 2024-08-30T09:08:21.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:21.939+0000 7f24e2d9d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f24e411c940 0x7f24e4118920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:08:21.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:21.939+0000 7f24e1d9b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f24e4111a70 0x7f24e41183e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:08:21.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:21.939+0000 7f24e1d9b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f24e4111a70 0x7f24e41183e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:60906/0 (socket says 172.21.15.42:60906) 2024-08-30T09:08:21.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:21.939+0000 7f24e1d9b640 1 -- 172.21.15.42:0/3435484190 learned_addr learned my addr 172.21.15.42:0/3435484190 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:08:21.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:21.939+0000 7f24e259c640 1 --2- 172.21.15.42:0/3435484190 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f24e410d990 0x7f24e411c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:08:21.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:21.940+0000 7f24e2d9d640 1 -- 172.21.15.42:0/3435484190 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f24e4111a70 msgr2=0x7f24e41183e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:08:21.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:21.940+0000 7f24e2d9d640 1 --2- 172.21.15.42:0/3435484190 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f24e4111a70 0x7f24e41183e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:21.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:21.940+0000 7f24e2d9d640 1 -- 172.21.15.42:0/3435484190 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f24e410d990 msgr2=0x7f24e411c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:08:21.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:21.940+0000 7f24e2d9d640 1 --2- 172.21.15.42:0/3435484190 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f24e410d990 0x7f24e411c100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:21.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:21.940+0000 7f24e2d9d640 1 -- 172.21.15.42:0/3435484190 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24e41190f0 con 0x7f24e411c940 2024-08-30T09:08:21.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:21.940+0000 7f24e2d9d640 1 --2- 172.21.15.42:0/3435484190 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f24e411c940 0x7f24e4118920 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f24d800d680 tx=0x7f24d800db40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:08:21.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:21.941+0000 7f24d37fe640 1 -- 172.21.15.42:0/3435484190 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24d8017070 con 0x7f24e411c940 2024-08-30T09:08:21.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:21.941+0000 7f24d37fe640 1 -- 172.21.15.42:0/3435484190 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f24d8009820 con 0x7f24e411c940 2024-08-30T09:08:21.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:21.941+0000 7f24d37fe640 1 -- 172.21.15.42:0/3435484190 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24d8004ce0 con 0x7f24e411c940 2024-08-30T09:08:21.951 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:21.947+0000 7faea37fd740 1 -- 172.21.15.42:0/1332236004 shutdown_connections 2024-08-30T09:08:21.951 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:21.947+0000 7faea37fd740 1 -- 172.21.15.42:0/1332236004 wait complete. 2024-08-30T09:08:21.951 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:21.947+0000 7faea37fd740 20 client.26479 trim_cache size 0 max 16384 2024-08-30T09:08:21.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:21.945+0000 7f24e359e640 1 -- 172.21.15.42:0/3435484190 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24e4075390 con 0x7f24e411c940 2024-08-30T09:08:21.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:21.945+0000 7f24e359e640 1 -- 172.21.15.42:0/3435484190 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24e4075850 con 0x7f24e411c940 2024-08-30T09:08:21.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:21.949+0000 7f24ef06a740 1 Processor -- start 2024-08-30T09:08:21.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:21.949+0000 7f24ef06a740 1 -- start start 2024-08-30T09:08:21.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:21.949+0000 7f24ef06a740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56012befa070 0x56012c2a2170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:08:21.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:21.950+0000 7f24ef06a740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56012bed11b0 0x56012c29f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:08:21.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:21.950+0000 7f24ef06a740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56012bed9390 0x56012c29fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:08:21.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:21.950+0000 7f24ef06a740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x56012c216680 con 0x56012befa070 2024-08-30T09:08:21.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:21.950+0000 7f24ef06a740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x56012c205c10 con 0x56012bed9390 2024-08-30T09:08:21.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:21.950+0000 7f24ef06a740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x56012c209bc0 con 0x56012bed11b0 2024-08-30T09:08:21.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:21.950+0000 7f24e1d9b640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56012bed9390 0x56012c29fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:08:21.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:21.950+0000 7f24e1d9b640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56012bed9390 0x56012c29fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:52414/0 (socket says 172.21.15.42:52414) 2024-08-30T09:08:21.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:21.950+0000 7f24e1d9b640 1 -- 172.21.15.42:0/237238209 learned_addr learned my addr 172.21.15.42:0/237238209 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:08:21.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:21.950+0000 7f24e1d9b640 1 --2- 172.21.15.42:0/237238209 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56012befa070 0x56012c2a2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:08:21.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:21.950+0000 7f24e259c640 1 --2- 172.21.15.42:0/237238209 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56012bed11b0 0x56012c29f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:08:21.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:21.950+0000 7f24e1d9b640 1 -- 172.21.15.42:0/237238209 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56012bed11b0 msgr2=0x56012c29f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:08:21.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:21.950+0000 7f24e1d9b640 1 --2- 172.21.15.42:0/237238209 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56012bed11b0 0x56012c29f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:21.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:21.950+0000 7f24e1d9b640 1 -- 172.21.15.42:0/237238209 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56012befa070 msgr2=0x56012c2a2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:08:21.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:21.950+0000 7f24e1d9b640 1 --2- 172.21.15.42:0/237238209 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56012befa070 0x56012c2a2170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:21.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:21.950+0000 7f24e1d9b640 1 -- 172.21.15.42:0/237238209 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56012bf65650 con 0x56012bed9390 2024-08-30T09:08:21.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:21.951+0000 7f24e1d9b640 1 --2- 172.21.15.42:0/237238209 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56012bed9390 0x56012c29fd20 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f24dc041040 tx=0x7f24dc01a5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=90af7ad01e945014 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:08:21.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:21.954+0000 7f24d37fe640 1 -- 172.21.15.42:0/3435484190 <== mon.2 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7f24d80099d0 con 0x7f24e411c940 2024-08-30T09:08:21.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:21.955+0000 7f24e359e640 1 -- 172.21.15.42:0/237238209 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24dc017070 con 0x56012bed9390 2024-08-30T09:08:21.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:21.955+0000 7f24e359e640 1 -- 172.21.15.42:0/237238209 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f24dc003970 con 0x56012bed9390 2024-08-30T09:08:21.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:21.955+0000 7f24d37fe640 1 --2- 172.21.15.42:0/3435484190 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f24cc0b4d30 0x7f24cc0b7240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:08:21.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:21.955+0000 7f24d37fe640 1 -- 172.21.15.42:0/3435484190 <== mon.2 v2:172.21.15.167:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f24d80e4f60 con 0x7f24e411c940 2024-08-30T09:08:21.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:21.956+0000 7f24ef06a740 1 -- 172.21.15.42:0/237238209 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56012bed9390 msgr2=0x56012c29fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:08:21.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:21.956+0000 7f24ef06a740 1 --2- 172.21.15.42:0/237238209 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56012bed9390 0x56012c29fd20 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f24dc041040 tx=0x7f24dc01a5d0 comp rx=0 tx=0).stop 2024-08-30T09:08:21.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:21.959+0000 7f24e259c640 1 --2- 172.21.15.42:0/3435484190 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f24cc0b4d30 0x7f24cc0b7240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:08:21.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:21.961+0000 7f24ef06a740 1 -- 172.21.15.42:0/237238209 shutdown_connections 2024-08-30T09:08:21.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:21.961+0000 7f24ef06a740 1 --2- 172.21.15.42:0/237238209 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56012bed11b0 0x56012c29f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:21.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:21.961+0000 7f24ef06a740 1 --2- 172.21.15.42:0/237238209 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56012bed9390 0x56012c29fd20 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:21.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:21.961+0000 7f24ef06a740 1 --2- 172.21.15.42:0/237238209 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56012befa070 0x56012c2a2170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:21.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:21.961+0000 7f24ef06a740 1 -- 172.21.15.42:0/237238209 >> 172.21.15.42:0/237238209 conn(0x56012beee460 msgr2=0x56012c29d670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:08:21.969 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:21.967+0000 7f24ef06a740 1 -- 172.21.15.42:0/237238209 shutdown_connections 2024-08-30T09:08:21.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:21.968+0000 7f24e259c640 1 --2- 172.21.15.42:0/3435484190 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f24cc0b4d30 0x7f24cc0b7240 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f24e411be50 tx=0x7f24d4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:08:21.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:21.970+0000 7f24ef06a740 1 -- 172.21.15.42:0/237238209 wait complete. 2024-08-30T09:08:21.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:21.970+0000 7f24ef06a740 1 -- start start 2024-08-30T09:08:21.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:21.970+0000 7f24ef06a740 1 Processor -- start 2024-08-30T09:08:21.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:21.974+0000 7f24ef06a740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56012bed11b0 0x56012c2aae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:08:21.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:21.974+0000 7f24ef06a740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56012bed9390 0x56012c2ab390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:08:21.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:21.974+0000 7f24ef06a740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56012befa070 0x56012c2d8f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:08:21.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:21.974+0000 7f24ef06a740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x56012c068700 con 0x56012befa070 2024-08-30T09:08:21.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:21.974+0000 7f24ef06a740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x56012c18c750 con 0x56012bed9390 2024-08-30T09:08:21.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:21.974+0000 7f24ef06a740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x56012c2cfbd0 con 0x56012bed11b0 2024-08-30T09:08:21.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:21.974+0000 7f24ef06a740 10 client.0 _ll_register_callbacks cb 0x56012c1bbe10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:08:21.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:21.976+0000 7f24e259c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56012bed11b0 0x56012c2aae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:08:21.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:21.976+0000 7f24e259c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56012bed11b0 0x56012c2aae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:60924/0 (socket says 172.21.15.42:60924) 2024-08-30T09:08:21.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:21.976+0000 7f24e259c640 1 -- 172.21.15.42:0/1692534525 learned_addr learned my addr 172.21.15.42:0/1692534525 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:08:21.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:21.977+0000 7f24e1d9b640 1 --2- 172.21.15.42:0/1692534525 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56012bed9390 0x56012c2ab390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:08:21.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:21.977+0000 7f24e259c640 1 -- 172.21.15.42:0/1692534525 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56012bed9390 msgr2=0x56012c2ab390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:08:21.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:21.977+0000 7f24e259c640 1 --2- 172.21.15.42:0/1692534525 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56012bed9390 0x56012c2ab390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:21.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:21.977+0000 7f24e259c640 1 -- 172.21.15.42:0/1692534525 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56012befa070 msgr2=0x56012c2d8f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:08:21.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:21.977+0000 7f24e259c640 1 --2- 172.21.15.42:0/1692534525 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56012befa070 0x56012c2d8f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:21.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:21.977+0000 7f24e259c640 1 -- 172.21.15.42:0/1692534525 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56012c1c5900 con 0x56012bed11b0 2024-08-30T09:08:21.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:21.980+0000 7f24e259c640 1 --2- 172.21.15.42:0/1692534525 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56012bed11b0 0x56012c2aae50 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f24d40097f0 tx=0x7f24d4004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=422a511ac7bd4ba0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:08:21.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:21.980+0000 7f24c9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-08-30T09:08:21.986 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:21.984+0000 7f24c9ffb640 1 -- 172.21.15.42:0/1692534525 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24d4050060 con 0x56012bed11b0 2024-08-30T09:08:21.986 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:21.984+0000 7f24ef06a740 1 -- 172.21.15.42:0/1692534525 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f24dc003970 con 0x56012bed11b0 2024-08-30T09:08:21.987 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:21.984+0000 7f24c9ffb640 1 -- 172.21.15.42:0/1692534525 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f24d403a030 con 0x56012bed11b0 2024-08-30T09:08:21.987 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:21.984+0000 7f24c9ffb640 1 -- 172.21.15.42:0/1692534525 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f24d4057050 con 0x56012bed11b0 2024-08-30T09:08:21.987 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:21.985+0000 7f24ef06a740 10 client.26491 fetch_fsmap learned FSMap version 35 2024-08-30T09:08:21.987 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:21.985+0000 7f24ef06a740 1 -- 172.21.15.42:0/1692534525 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x56012c2dba60 con 0x56012bed11b0 2024-08-30T09:08:21.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:21.989+0000 7f24c9ffb640 1 -- 172.21.15.42:0/1692534525 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f24dc003970 con 0x56012bed11b0 2024-08-30T09:08:21.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:21.989+0000 7f24c9ffb640 1 -- 172.21.15.42:0/1692534525 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 35) ==== 8362+0+0 (secure 0 0 0) 0x7f24d403e3c0 con 0x56012bed11b0 2024-08-30T09:08:21.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:21.989+0000 7f24ef06a740 10 client.26491 fetch_fsmap finished waiting for FSMap version 35 2024-08-30T09:08:21.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:21.989+0000 7f24ef06a740 10 client.26491 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-30T09:08:21.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:21.989+0000 7f24ef06a740 20 client.26491 populate_metadata read hostname 'smithi042' 2024-08-30T09:08:21.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:21.989+0000 7f24ef06a740 1 --2- 172.21.15.42:0/1692534525 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x56012beb89f0 0x56012c2df4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:08:21.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:21.989+0000 7f24ef06a740 4 client.26491 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-30T09:08:21.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:21.989+0000 7f24ef06a740 1 -- 172.21.15.42:0/1692534525 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56012c2185d0 con 0x56012beb89f0 2024-08-30T09:08:21.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:21.989+0000 7f24e1d9b640 1 --2- 172.21.15.42:0/1692534525 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x56012beb89f0 0x56012c2df4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:08:21.993 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:21.991+0000 7faea37fd740 1 Processor -- start 2024-08-30T09:08:21.993 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:21.992+0000 7faea37fd740 1 -- start start 2024-08-30T09:08:21.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:21.992+0000 7f24e1d9b640 1 --2- 172.21.15.42:0/1692534525 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x56012beb89f0 0x56012c2df4c0 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d5c8f1a31fbfade5 server_cookie=6836ec8e3aeebe30 in_seq=0 out_seq=0 2024-08-30T09:08:21.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:21.992+0000 7f24c9ffb640 10 client.26491 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:08:21.995 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:21.992+0000 7faea37fd740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cd690c32f0 0x55cd694930e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:08:21.995 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:21.992+0000 7faea37fd740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cd690cb4d0 0x55cd694a3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:08:21.995 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:21.992+0000 7faea37fd740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cd690ec1b0 0x55cd69491e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:08:21.995 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:21.992+0000 7faea37fd740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fae981b5220 con 0x55cd690c32f0 2024-08-30T09:08:21.995 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:21.992+0000 7faea37fd740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fae981b2ac0 con 0x55cd690ec1b0 2024-08-30T09:08:21.995 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:21.992+0000 7faea37fd740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fae981b0910 con 0x55cd690cb4d0 2024-08-30T09:08:21.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:21.993+0000 7f24c9ffb640 1 -- 172.21.15.42:0/1692534525 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56012c2185d0 con 0x56012beb89f0 2024-08-30T09:08:21.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:21.993+0000 7f24c9ffb640 10 client.26491 handle_command_reply: tid=0 2024-08-30T09:08:21.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:21.993+0000 7f24ef06a740 1 client.26491 shutdown 2024-08-30T09:08:21.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:21.994+0000 7f24ef06a740 1 -- 172.21.15.42:0/1692534525 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56012bed11b0 msgr2=0x56012c2aae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:08:21.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:21.994+0000 7f24ef06a740 1 --2- 172.21.15.42:0/1692534525 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56012bed11b0 0x56012c2aae50 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f24d40097f0 tx=0x7f24d4004730 comp rx=0 tx=0).stop 2024-08-30T09:08:21.998 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:21.993+0000 7fae95d9b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cd690cb4d0 0x55cd694a3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:08:21.998 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:21.993+0000 7fae9659c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cd690c32f0 0x55cd694930e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:08:21.998 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:21.993+0000 7fae95d9b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cd690cb4d0 0x55cd694a3bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:60934/0 (socket says 172.21.15.42:60934) 2024-08-30T09:08:21.998 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:21.993+0000 7fae95d9b640 1 -- 172.21.15.42:0/946179290 learned_addr learned my addr 172.21.15.42:0/946179290 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:08:21.998 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:21.994+0000 7fae96d9d640 1 --2- 172.21.15.42:0/946179290 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cd690ec1b0 0x55cd69491e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:08:21.999 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:21.997+0000 7fae95d9b640 1 -- 172.21.15.42:0/946179290 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cd690ec1b0 msgr2=0x55cd69491e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:08:21.999 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:21.997+0000 7fae95d9b640 1 --2- 172.21.15.42:0/946179290 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cd690ec1b0 0x55cd69491e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:21.999 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:21.997+0000 7fae95d9b640 1 -- 172.21.15.42:0/946179290 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cd690c32f0 msgr2=0x55cd694930e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:08:21.999 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:21.997+0000 7fae95d9b640 1 --2- 172.21.15.42:0/946179290 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cd690c32f0 0x55cd694930e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:21.999 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:21.997+0000 7fae95d9b640 1 -- 172.21.15.42:0/946179290 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cd69499cf0 con 0x55cd690cb4d0 2024-08-30T09:08:22.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:21.997+0000 7f24ef06a740 1 -- 172.21.15.42:0/1692534525 shutdown_connections 2024-08-30T09:08:22.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:21.997+0000 7f24ef06a740 1 --2- 172.21.15.42:0/1692534525 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x56012beb89f0 0x56012c2df4c0 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:22.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:21.997+0000 7f24ef06a740 1 --2- 172.21.15.42:0/1692534525 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56012befa070 0x56012c2d8f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:22.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:21.997+0000 7f24ef06a740 1 --2- 172.21.15.42:0/1692534525 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56012bed9390 0x56012c2ab390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:22.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:21.997+0000 7f24ef06a740 1 --2- 172.21.15.42:0/1692534525 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56012bed11b0 0x56012c2aae50 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:22.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:21.997+0000 7f24ef06a740 1 -- 172.21.15.42:0/1692534525 >> 172.21.15.42:0/1692534525 conn(0x56012beee460 msgr2=0x56012c2a1170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:08:22.001 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:21.997+0000 7fae95d9b640 1 --2- 172.21.15.42:0/946179290 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cd690cb4d0 0x55cd694a3bd0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fae90041040 tx=0x7fae9002f530 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ef20274a136a00cf server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:08:22.001 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:21.999+0000 7fae7ffff640 1 -- 172.21.15.42:0/946179290 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae9006a030 con 0x55cd690cb4d0 2024-08-30T09:08:22.001 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:21.999+0000 7fae7ffff640 1 -- 172.21.15.42:0/946179290 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae9004f040 con 0x55cd690cb4d0 2024-08-30T09:08:22.001 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:21.999+0000 7fae7ffff640 1 -- 172.21.15.42:0/946179290 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae90004650 con 0x55cd690cb4d0 2024-08-30T09:08:22.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:21.997+0000 7f24c9ffb640 0 client.26491 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:08:22.002 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:22.000+0000 7faea37fd740 1 -- 172.21.15.42:0/946179290 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cd690cb4d0 msgr2=0x55cd694a3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:08:22.002 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:22.000+0000 7faea37fd740 1 --2- 172.21.15.42:0/946179290 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cd690cb4d0 0x55cd694a3bd0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fae90041040 tx=0x7fae9002f530 comp rx=0 tx=0).stop 2024-08-30T09:08:22.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:21.999+0000 7f24ef06a740 1 -- 172.21.15.42:0/1692534525 shutdown_connections 2024-08-30T09:08:22.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:21.999+0000 7f24ef06a740 1 -- 172.21.15.42:0/1692534525 wait complete. 2024-08-30T09:08:22.003 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:22.000+0000 7faea37fd740 1 -- 172.21.15.42:0/946179290 shutdown_connections 2024-08-30T09:08:22.003 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:22.000+0000 7faea37fd740 1 --2- 172.21.15.42:0/946179290 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cd690cb4d0 0x55cd694a3bd0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:22.003 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:22.000+0000 7faea37fd740 1 --2- 172.21.15.42:0/946179290 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cd690ec1b0 0x55cd69491e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:22.003 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:22.000+0000 7faea37fd740 1 --2- 172.21.15.42:0/946179290 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cd690c32f0 0x55cd694930e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:22.003 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:22.000+0000 7faea37fd740 1 -- 172.21.15.42:0/946179290 >> 172.21.15.42:0/946179290 conn(0x55cd690e0300 msgr2=0x55cd69488400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:08:22.003 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:22.000+0000 7faea37fd740 1 -- 172.21.15.42:0/946179290 shutdown_connections 2024-08-30T09:08:22.003 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:22.000+0000 7faea37fd740 1 -- 172.21.15.42:0/946179290 wait complete. 2024-08-30T09:08:22.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:22.000+0000 7f24ef06a740 20 client.26491 trim_cache size 0 max 16384 2024-08-30T09:08:22.005 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:22.001+0000 7faea37fd740 1 -- start start 2024-08-30T09:08:22.006 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:22.001+0000 7faea37fd740 1 Processor -- start 2024-08-30T09:08:22.006 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:22.002+0000 7faea37fd740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cd690c32f0 0x55cd694a2b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:08:22.006 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:22.002+0000 7faea37fd740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cd690cb4d0 0x55cd694a3060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:08:22.006 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:22.002+0000 7faea37fd740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cd690ec1b0 0x55cd694a3770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:08:22.006 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:22.002+0000 7faea37fd740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fae981b0f80 con 0x55cd690c32f0 2024-08-30T09:08:22.006 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:22.002+0000 7faea37fd740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fae981b23f0 con 0x55cd690cb4d0 2024-08-30T09:08:22.006 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:22.002+0000 7faea37fd740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fae981b1650 con 0x55cd690ec1b0 2024-08-30T09:08:22.006 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:22.002+0000 7faea37fd740 10 client.0 _ll_register_callbacks cb 0x7fae90009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:08:22.006 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:22.003+0000 7fae9659c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cd690c32f0 0x55cd694a2b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:08:22.006 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:22.003+0000 7fae9659c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cd690c32f0 0x55cd694a2b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:50916/0 (socket says 172.21.15.42:50916) 2024-08-30T09:08:22.006 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:22.003+0000 7fae9659c640 1 -- 172.21.15.42:0/3165970467 learned_addr learned my addr 172.21.15.42:0/3165970467 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:08:22.006 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:22.003+0000 7fae96d9d640 1 --2- 172.21.15.42:0/3165970467 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cd690ec1b0 0x55cd694a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:08:22.006 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:22.003+0000 7fae9659c640 1 -- 172.21.15.42:0/3165970467 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cd690ec1b0 msgr2=0x55cd694a3770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:08:22.006 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:22.003+0000 7fae9659c640 1 --2- 172.21.15.42:0/3165970467 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cd690ec1b0 0x55cd694a3770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:22.007 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:22.003+0000 7fae9659c640 1 -- 172.21.15.42:0/3165970467 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cd690cb4d0 msgr2=0x55cd694a3060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:08:22.007 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:22.003+0000 7fae95d9b640 1 --2- 172.21.15.42:0/3165970467 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cd690cb4d0 0x55cd694a3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:08:22.007 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:22.004+0000 7fae9659c640 1 --2- 172.21.15.42:0/3165970467 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cd690cb4d0 0x55cd694a3060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:22.007 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:22.004+0000 7fae9659c640 1 -- 172.21.15.42:0/3165970467 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cd694a3f00 con 0x55cd690c32f0 2024-08-30T09:08:22.007 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:22.004+0000 7fae9659c640 1 --2- 172.21.15.42:0/3165970467 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cd690c32f0 0x55cd694a2b20 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fae8c0e3080 tx=0x7fae8c0abb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0757cfca0ee18ae server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:08:22.007 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:22.004+0000 7fae7dffb640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-08-30T09:08:22.008 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:22.006+0000 7fae7dffb640 1 -- 172.21.15.42:0/3165970467 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae8c0abfb0 con 0x55cd690c32f0 2024-08-30T09:08:22.008 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:22.006+0000 7faea37fd740 1 -- 172.21.15.42:0/3165970467 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae9004f040 con 0x55cd690c32f0 2024-08-30T09:08:22.008 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:22.006+0000 7fae7dffb640 1 -- 172.21.15.42:0/3165970467 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7fae8c0046c0 con 0x55cd690c32f0 2024-08-30T09:08:22.009 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:22.007+0000 7fae7dffb640 1 -- 172.21.15.42:0/3165970467 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fae8c0b0820 con 0x55cd690c32f0 2024-08-30T09:08:22.009 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:22.007+0000 7faea37fd740 10 client.16731 fetch_fsmap learned FSMap version 35 2024-08-30T09:08:22.009 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:22.007+0000 7faea37fd740 1 -- 172.21.15.42:0/3165970467 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55cd69512fa0 con 0x55cd690c32f0 2024-08-30T09:08:22.012 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:22.011+0000 7fae7dffb640 1 -- 172.21.15.42:0/3165970467 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae9004f040 con 0x55cd690c32f0 2024-08-30T09:08:22.013 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:22.011+0000 7fae7dffb640 1 -- 172.21.15.42:0/3165970467 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 35) ==== 8362+0+0 (secure 0 0 0) 0x7fae8c0ac210 con 0x55cd690c32f0 2024-08-30T09:08:22.016 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:22.014+0000 7faea37fd740 10 client.16731 fetch_fsmap finished waiting for FSMap version 35 2024-08-30T09:08:22.016 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:22.014+0000 7faea37fd740 10 client.16731 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-30T09:08:22.016 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:22.014+0000 7faea37fd740 20 client.16731 populate_metadata read hostname 'smithi042' 2024-08-30T09:08:22.016 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:22.014+0000 7faea37fd740 1 --2- 172.21.15.42:0/3165970467 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55cd690801c0 0x55cd69515920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:08:22.016 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:22.014+0000 7faea37fd740 4 client.16731 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-30T09:08:22.016 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:22.014+0000 7faea37fd740 1 -- 172.21.15.42:0/3165970467 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55cd694029f0 con 0x55cd690801c0 2024-08-30T09:08:22.019 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:22.017+0000 7fae95d9b640 1 --2- 172.21.15.42:0/3165970467 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55cd690801c0 0x55cd69515920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:08:22.025 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:22.023+0000 7fae95d9b640 1 --2- 172.21.15.42:0/3165970467 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55cd690801c0 0x55cd69515920 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ab2e7cbc4546ff1f server_cookie=9789d92557f95c53 in_seq=0 out_seq=0 2024-08-30T09:08:22.025 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:22.023+0000 7fae7dffb640 10 client.16731 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:08:22.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:22.029+0000 7f24ef06a740 1 Processor -- start 2024-08-30T09:08:22.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:22.033+0000 7f24ef06a740 1 -- start start 2024-08-30T09:08:22.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:22.033+0000 7f24ef06a740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56012bed11b0 0x56012c2a96d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:08:22.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:22.033+0000 7f24ef06a740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56012bed9390 0x56012c2a9c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:08:22.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:22.034+0000 7f24ef06a740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56012befa070 0x56012c29f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:08:22.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:22.034+0000 7f24ef06a740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f24e4081fd0 con 0x56012bed9390 2024-08-30T09:08:22.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:22.034+0000 7f24ef06a740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f24e407f870 con 0x56012befa070 2024-08-30T09:08:22.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:22.034+0000 7f24ef06a740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f24e410dde0 con 0x56012bed11b0 2024-08-30T09:08:22.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:22.035+0000 7f24e259c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56012bed11b0 0x56012c2a96d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:08:22.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:22.035+0000 7f24e1d9b640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56012bed9390 0x56012c2a9c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:08:22.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:22.035+0000 7f24e1d9b640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56012bed9390 0x56012c2a9c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:50926/0 (socket says 172.21.15.42:50926) 2024-08-30T09:08:22.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:22.035+0000 7f24e1d9b640 1 -- 172.21.15.42:0/3387938666 learned_addr learned my addr 172.21.15.42:0/3387938666 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:08:22.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:22.036+0000 7f24e2d9d640 1 --2- 172.21.15.42:0/3387938666 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56012befa070 0x56012c29f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:08:22.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:22.037+0000 7f24e2d9d640 1 -- 172.21.15.42:0/3387938666 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56012bed11b0 msgr2=0x56012c2a96d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:08:22.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:22.037+0000 7f24e2d9d640 1 --2- 172.21.15.42:0/3387938666 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56012bed11b0 0x56012c2a96d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:22.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:22.037+0000 7f24e2d9d640 1 -- 172.21.15.42:0/3387938666 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56012bed9390 msgr2=0x56012c2a9c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:08:22.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:22.037+0000 7f24e2d9d640 1 --2- 172.21.15.42:0/3387938666 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56012bed9390 0x56012c2a9c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:22.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:22.037+0000 7f24e2d9d640 1 -- 172.21.15.42:0/3387938666 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56012beb8c20 con 0x56012befa070 2024-08-30T09:08:22.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:22.037+0000 7f24e259c640 1 --2- 172.21.15.42:0/3387938666 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56012bed11b0 0x56012c2a96d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-30T09:08:22.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:22.037+0000 7f24e2d9d640 1 --2- 172.21.15.42:0/3387938666 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56012befa070 0x56012c29f6b0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f24d8009610 tx=0x7f24d80b43c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ee44bae08b82acef server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:08:22.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:22.038+0000 7f24cbfff640 1 -- 172.21.15.42:0/3387938666 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24d80ab620 con 0x56012befa070 2024-08-30T09:08:22.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:22.038+0000 7f24cbfff640 1 -- 172.21.15.42:0/3387938666 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f24d80afe90 con 0x56012befa070 2024-08-30T09:08:22.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:22.038+0000 7f24cbfff640 1 -- 172.21.15.42:0/3387938666 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24d80a6f90 con 0x56012befa070 2024-08-30T09:08:22.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:22.039+0000 7f24ef06a740 1 -- 172.21.15.42:0/3387938666 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56012befa070 msgr2=0x56012c29f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:08:22.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:22.039+0000 7f24ef06a740 1 --2- 172.21.15.42:0/3387938666 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56012befa070 0x56012c29f6b0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f24d8009610 tx=0x7f24d80b43c0 comp rx=0 tx=0).stop 2024-08-30T09:08:22.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:22.039+0000 7f24ef06a740 1 -- 172.21.15.42:0/3387938666 shutdown_connections 2024-08-30T09:08:22.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:22.039+0000 7f24ef06a740 1 --2- 172.21.15.42:0/3387938666 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56012befa070 0x56012c29f6b0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:22.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:22.039+0000 7f24ef06a740 1 --2- 172.21.15.42:0/3387938666 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56012bed9390 0x56012c2a9c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:22.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:22.039+0000 7f24ef06a740 1 --2- 172.21.15.42:0/3387938666 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56012bed11b0 0x56012c2a96d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:22.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:22.039+0000 7f24ef06a740 1 -- 172.21.15.42:0/3387938666 >> 172.21.15.42:0/3387938666 conn(0x56012beee460 msgr2=0x56012c2a5f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:08:22.044 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:22.042+0000 7fae7dffb640 1 -- 172.21.15.42:0/3165970467 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55cd694029f0 con 0x55cd690801c0 2024-08-30T09:08:22.044 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:22.042+0000 7fae7dffb640 10 client.16731 handle_command_reply: tid=0 2024-08-30T09:08:22.045 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:22.043+0000 7faea37fd740 1 client.16731 shutdown 2024-08-30T09:08:22.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:22.043+0000 7f24ef06a740 1 -- 172.21.15.42:0/3387938666 shutdown_connections 2024-08-30T09:08:22.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:22.043+0000 7f24ef06a740 1 -- 172.21.15.42:0/3387938666 wait complete. 2024-08-30T09:08:22.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:22.044+0000 7f24ef06a740 1 -- start start 2024-08-30T09:08:22.046 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:22.044+0000 7faea37fd740 1 -- 172.21.15.42:0/3165970467 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cd690c32f0 msgr2=0x55cd694a2b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:08:22.046 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:22.044+0000 7faea37fd740 1 --2- 172.21.15.42:0/3165970467 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cd690c32f0 0x55cd694a2b20 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fae8c0e3080 tx=0x7fae8c0abb80 comp rx=0 tx=0).stop 2024-08-30T09:08:22.046 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:22.044+0000 7faea37fd740 1 -- 172.21.15.42:0/3165970467 shutdown_connections 2024-08-30T09:08:22.047 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:22.044+0000 7faea37fd740 1 --2- 172.21.15.42:0/3165970467 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55cd690801c0 0x55cd69515920 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:22.047 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:22.044+0000 7faea37fd740 1 --2- 172.21.15.42:0/3165970467 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cd690ec1b0 0x55cd694a3770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:22.047 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:22.044+0000 7faea37fd740 1 --2- 172.21.15.42:0/3165970467 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cd690cb4d0 0x55cd694a3060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:22.047 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:22.044+0000 7faea37fd740 1 --2- 172.21.15.42:0/3165970467 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cd690c32f0 0x55cd694a2b20 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:22.047 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:22.044+0000 7faea37fd740 1 -- 172.21.15.42:0/3165970467 >> 172.21.15.42:0/3165970467 conn(0x55cd690e0300 msgr2=0x55cd69488400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:08:22.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:22.044+0000 7f24ef06a740 1 Processor -- start 2024-08-30T09:08:22.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:22.045+0000 7f24ef06a740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56012bed11b0 0x56012c2d5570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:08:22.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:22.045+0000 7f24ef06a740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56012bed9390 0x56012c2d0a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:08:22.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:22.045+0000 7f24ef06a740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56012befa070 0x56012c2d0f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:08:22.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:22.045+0000 7f24ef06a740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f24e407dd30 con 0x56012bed11b0 2024-08-30T09:08:22.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:22.045+0000 7f24ef06a740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f24e407f1a0 con 0x56012bed9390 2024-08-30T09:08:22.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:22.045+0000 7f24ef06a740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f24e407e400 con 0x56012befa070 2024-08-30T09:08:22.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:22.045+0000 7f24ef06a740 10 client.0 _ll_register_callbacks cb 0x7f24d4009660 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:08:22.048 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:22.046+0000 7fae7dffb640 0 client.16731 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:08:22.048 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:22.046+0000 7faea37fd740 1 -- 172.21.15.42:0/3165970467 shutdown_connections 2024-08-30T09:08:22.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:22.046+0000 7f24e1d9b640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56012bed9390 0x56012c2d0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:08:22.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:22.046+0000 7f24e259c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56012bed11b0 0x56012c2d5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:08:22.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:22.046+0000 7f24e259c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56012bed11b0 0x56012c2d5570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:50940/0 (socket says 172.21.15.42:50940) 2024-08-30T09:08:22.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:22.046+0000 7f24e259c640 1 -- 172.21.15.42:0/1680829696 learned_addr learned my addr 172.21.15.42:0/1680829696 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:08:22.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:22.046+0000 7f24e2d9d640 1 --2- 172.21.15.42:0/1680829696 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56012befa070 0x56012c2d0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:08:22.049 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:22.046+0000 7faea37fd740 1 -- 172.21.15.42:0/3165970467 wait complete. 2024-08-30T09:08:22.050 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:22.047+0000 7faea37fd740 20 client.16731 trim_cache size 0 max 16384 2024-08-30T09:08:22.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:22.046+0000 7f24e259c640 1 -- 172.21.15.42:0/1680829696 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56012befa070 msgr2=0x56012c2d0f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:08:22.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:22.046+0000 7f24e259c640 1 --2- 172.21.15.42:0/1680829696 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56012befa070 0x56012c2d0f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:22.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:22.046+0000 7f24e259c640 1 -- 172.21.15.42:0/1680829696 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56012bed9390 msgr2=0x56012c2d0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:08:22.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:22.046+0000 7f24e259c640 1 --2- 172.21.15.42:0/1680829696 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56012bed9390 0x56012c2d0a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:22.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:22.046+0000 7f24e259c640 1 -- 172.21.15.42:0/1680829696 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56012c2d15a0 con 0x56012bed11b0 2024-08-30T09:08:22.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:22.047+0000 7f24e259c640 1 --2- 172.21.15.42:0/1680829696 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56012bed11b0 0x56012c2d5570 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f24d40097f0 tx=0x7f24d403eec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f9bf61e92077473 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:08:22.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:22.047+0000 7f24c9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-08-30T09:08:22.051 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-08-30T09:08:22.051 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2024-08-30T09:08:22.051 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-08-30T09:08:22.051 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-08-30T09:08:22.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:22.047+0000 7f24c9ffb640 1 -- 172.21.15.42:0/1680829696 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24d4061020 con 0x56012bed11b0 2024-08-30T09:08:22.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:22.048+0000 7f24ef06a740 1 -- 172.21.15.42:0/1680829696 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f24d80afe90 con 0x56012bed11b0 2024-08-30T09:08:22.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:22.048+0000 7f24c9ffb640 1 -- 172.21.15.42:0/1680829696 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f24d403e3a0 con 0x56012bed11b0 2024-08-30T09:08:22.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:22.048+0000 7f24c9ffb640 1 -- 172.21.15.42:0/1680829696 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f24d406c050 con 0x56012bed11b0 2024-08-30T09:08:22.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:22.048+0000 7f24ef06a740 10 client.16737 fetch_fsmap learned FSMap version 35 2024-08-30T09:08:22.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:22.048+0000 7f24ef06a740 1 -- 172.21.15.42:0/1680829696 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x56012c349cb0 con 0x56012bed11b0 2024-08-30T09:08:22.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:22.049+0000 7f24c9ffb640 1 -- 172.21.15.42:0/1680829696 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f24d80afe90 con 0x56012bed11b0 2024-08-30T09:08:22.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:22.049+0000 7f24c9ffb640 1 -- 172.21.15.42:0/1680829696 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 35) ==== 8362+0+0 (secure 0 0 0) 0x7f24d4005240 con 0x56012bed11b0 2024-08-30T09:08:22.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:22.049+0000 7f24ef06a740 10 client.16737 fetch_fsmap finished waiting for FSMap version 35 2024-08-30T09:08:22.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:22.049+0000 7f24ef06a740 10 client.16737 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-30T09:08:22.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:22.049+0000 7f24ef06a740 20 client.16737 populate_metadata read hostname 'smithi042' 2024-08-30T09:08:22.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:22.049+0000 7f24ef06a740 1 --2- 172.21.15.42:0/1680829696 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x56012be8dda0 0x56012c34c470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:08:22.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:22.049+0000 7f24ef06a740 4 client.16737 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-30T09:08:22.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:22.049+0000 7f24ef06a740 1 -- 172.21.15.42:0/1680829696 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56012c214070 con 0x56012be8dda0 2024-08-30T09:08:22.054 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:22.050+0000 7fae9c88d640 1 -- 172.21.15.42:0/1839116542 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fae800b4b30 msgr2=0x7fae800b7040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:08:22.054 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:22.050+0000 7fae9c88d640 1 --2- 172.21.15.42:0/1839116542 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fae800b4b30 0x7fae800b7040 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fae900094e0 tx=0x7fae90000f40 comp rx=0 tx=0).stop 2024-08-30T09:08:22.054 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:22.050+0000 7fae9c88d640 1 -- 172.21.15.42:0/1839116542 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fae98075a90 msgr2=0x7fae981a0140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:08:22.054 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:22.050+0000 7fae9c88d640 1 --2- 172.21.15.42:0/1839116542 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fae98075a90 0x7fae981a0140 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fae8c00e9f0 tx=0x7fae8c00edf0 comp rx=0 tx=0).stop 2024-08-30T09:08:22.054 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:22.050+0000 7fae9c88d640 1 -- 172.21.15.42:0/1839116542 shutdown_connections 2024-08-30T09:08:22.054 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:22.050+0000 7fae9c88d640 1 --2- 172.21.15.42:0/1839116542 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fae800b4b30 0x7fae800b7040 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:22.054 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:22.050+0000 7fae9c88d640 1 --2- 172.21.15.42:0/1839116542 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fae9810fbe0 0x7fae981a1360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:22.055 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:22.050+0000 7fae9c88d640 1 --2- 172.21.15.42:0/1839116542 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fae98076e90 0x7fae981a07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:22.055 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:22.050+0000 7fae9c88d640 1 --2- 172.21.15.42:0/1839116542 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fae98075a90 0x7fae981a0140 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:22.055 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:22.050+0000 7fae9c88d640 1 -- 172.21.15.42:0/1839116542 >> 172.21.15.42:0/1839116542 conn(0x7fae980fedc0 msgr2=0x7fae98101250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:08:22.055 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:22.050+0000 7fae9c88d640 1 -- 172.21.15.42:0/1839116542 shutdown_connections 2024-08-30T09:08:22.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:22.050+0000 7f24e1d9b640 1 --2- 172.21.15.42:0/1680829696 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x56012be8dda0 0x56012c34c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:08:22.056 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:22.050+0000 7fae9c88d640 1 -- 172.21.15.42:0/1839116542 wait complete. 2024-08-30T09:08:22.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:22.050+0000 7f24c9ffb640 10 client.16737 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:08:22.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:22.051+0000 7f24e1d9b640 1 --2- 172.21.15.42:0/1680829696 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x56012be8dda0 0x56012c34c470 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bba643bad2bb12ba server_cookie=609076545650db93 in_seq=0 out_seq=0 2024-08-30T09:08:22.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:22.052+0000 7f24c9ffb640 1 -- 172.21.15.42:0/1680829696 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56012c214070 con 0x56012be8dda0 2024-08-30T09:08:22.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:22.052+0000 7f24c9ffb640 10 client.16737 handle_command_reply: tid=0 2024-08-30T09:08:22.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:22.052+0000 7f24ef06a740 1 client.16737 shutdown 2024-08-30T09:08:22.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:22.053+0000 7f24ef06a740 1 -- 172.21.15.42:0/1680829696 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56012bed11b0 msgr2=0x56012c2d5570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:08:22.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:22.053+0000 7f24ef06a740 1 --2- 172.21.15.42:0/1680829696 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56012bed11b0 0x56012c2d5570 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f24d40097f0 tx=0x7f24d403eec0 comp rx=0 tx=0).stop 2024-08-30T09:08:22.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:22.053+0000 7f24ef06a740 1 -- 172.21.15.42:0/1680829696 shutdown_connections 2024-08-30T09:08:22.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:22.053+0000 7f24ef06a740 1 --2- 172.21.15.42:0/1680829696 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x56012be8dda0 0x56012c34c470 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:22.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:22.053+0000 7f24ef06a740 1 --2- 172.21.15.42:0/1680829696 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56012befa070 0x56012c2d0f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:22.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:22.053+0000 7f24ef06a740 1 --2- 172.21.15.42:0/1680829696 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56012bed9390 0x56012c2d0a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:22.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:22.053+0000 7f24ef06a740 1 --2- 172.21.15.42:0/1680829696 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56012bed11b0 0x56012c2d5570 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:22.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:22.053+0000 7f24ef06a740 1 -- 172.21.15.42:0/1680829696 >> 172.21.15.42:0/1680829696 conn(0x56012beee460 msgr2=0x56012c2aac00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:08:22.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:22.053+0000 7f24c9ffb640 0 client.16737 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:08:22.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:22.053+0000 7f24ef06a740 1 -- 172.21.15.42:0/1680829696 shutdown_connections 2024-08-30T09:08:22.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:22.053+0000 7f24ef06a740 1 -- 172.21.15.42:0/1680829696 wait complete. 2024-08-30T09:08:22.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:22.054+0000 7f24ef06a740 20 client.16737 trim_cache size 0 max 16384 2024-08-30T09:08:22.058 INFO:teuthology.orchestra.run.smithi042.stdout:[] 2024-08-30T09:08:22.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:22.054+0000 7f24d17fa640 1 -- 172.21.15.42:0/3435484190 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f24cc0b4d30 msgr2=0x7f24cc0b7240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:08:22.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:22.054+0000 7f24d17fa640 1 --2- 172.21.15.42:0/3435484190 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f24cc0b4d30 0x7f24cc0b7240 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f24e411be50 tx=0x7f24d4005f90 comp rx=0 tx=0).stop 2024-08-30T09:08:22.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:22.054+0000 7f24d17fa640 1 -- 172.21.15.42:0/3435484190 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f24e411c940 msgr2=0x7f24e4118920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:08:22.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:22.054+0000 7f24d17fa640 1 --2- 172.21.15.42:0/3435484190 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f24e411c940 0x7f24e4118920 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f24d800d680 tx=0x7f24d800db40 comp rx=0 tx=0).stop 2024-08-30T09:08:22.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:22.055+0000 7f24d17fa640 1 -- 172.21.15.42:0/3435484190 shutdown_connections 2024-08-30T09:08:22.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:22.055+0000 7f24d17fa640 1 --2- 172.21.15.42:0/3435484190 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f24cc0b4d30 0x7f24cc0b7240 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:22.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:22.055+0000 7f24d17fa640 1 --2- 172.21.15.42:0/3435484190 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f24e4111a70 0x7f24e41183e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:22.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:22.055+0000 7f24d17fa640 1 --2- 172.21.15.42:0/3435484190 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f24e411c940 0x7f24e4118920 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:22.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:22.055+0000 7f24d17fa640 1 --2- 172.21.15.42:0/3435484190 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f24e410d990 0x7f24e411c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:22.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:22.055+0000 7f24d17fa640 1 -- 172.21.15.42:0/3435484190 >> 172.21.15.42:0/3435484190 conn(0x7f24e406e790 msgr2=0x7f24e411dea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:08:22.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:22.056+0000 7f24d17fa640 1 -- 172.21.15.42:0/3435484190 shutdown_connections 2024-08-30T09:08:22.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:22.056+0000 7f24d17fa640 1 -- 172.21.15.42:0/3435484190 wait complete. 2024-08-30T09:08:23.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:08:22 smithi042 ceph-mon[18652]: pgmap v601: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 4.1 MiB/s wr, 417 op/s 2024-08-30T09:08:23.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:08:22 smithi190 ceph-mon[22001]: pgmap v601: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 4.1 MiB/s wr, 417 op/s 2024-08-30T09:08:23.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:08:22 smithi167 ceph-mon[22063]: pgmap v601: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 4.1 MiB/s wr, 417 op/s 2024-08-30T09:08:23.846 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-08-30T09:08:23.847 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-08-30T09:08:24.262 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T09:08:24.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.854+0000 7fa5ca124640 1 -- 172.21.15.42:0/2155896828 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa5c410a780 msgr2=0x7fa5c410ab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:08:24.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.854+0000 7fa5ca124640 1 --2- 172.21.15.42:0/2155896828 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa5c410a780 0x7fa5c410ab90 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fa5bc009ca0 tx=0x7fa5bc02a8f0 comp rx=0 tx=0).stop 2024-08-30T09:08:24.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.857+0000 7fa5ca124640 1 -- 172.21.15.42:0/2155896828 shutdown_connections 2024-08-30T09:08:24.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.857+0000 7fa5ca124640 1 --2- 172.21.15.42:0/2155896828 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa5c410a780 0x7fa5c410ab90 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:24.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.857+0000 7fa5ca124640 1 --2- 172.21.15.42:0/2155896828 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa5c411c9d0 0x7fa5c411eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:24.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.857+0000 7fa5ca124640 1 --2- 172.21.15.42:0/2155896828 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa5c4111ce0 0x7fa5c410a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:24.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.857+0000 7fa5ca124640 1 -- 172.21.15.42:0/2155896828 >> 172.21.15.42:0/2155896828 conn(0x7fa5c4070c60 msgr2=0x7fa5c4071070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:08:24.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.860+0000 7fa5ca124640 1 -- 172.21.15.42:0/2155896828 shutdown_connections 2024-08-30T09:08:24.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.861+0000 7fa5ca124640 1 -- 172.21.15.42:0/2155896828 wait complete. 2024-08-30T09:08:24.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.862+0000 7fa5ca124640 1 Processor -- start 2024-08-30T09:08:24.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.865+0000 7fa5ca124640 1 -- start start 2024-08-30T09:08:24.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.865+0000 7fa5ca124640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa5c410a780 0x7fa5c41a91b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:08:24.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.865+0000 7fa5ca124640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa5c4111ce0 0x7fa5c41a9750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:08:24.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.865+0000 7fa5ca124640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa5c411c9d0 0x7fa5c41ae9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:08:24.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.865+0000 7fa5ca124640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fa5c4122700 con 0x7fa5c411c9d0 2024-08-30T09:08:24.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.865+0000 7fa5ca124640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa5c4122580 con 0x7fa5c4111ce0 2024-08-30T09:08:24.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.865+0000 7fa5ca124640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fa5c4122880 con 0x7fa5c410a780 2024-08-30T09:08:24.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.867+0000 7fa5c3fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa5c411c9d0 0x7fa5c41ae9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:08:24.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.867+0000 7fa5c3fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa5c411c9d0 0x7fa5c41ae9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:50952/0 (socket says 172.21.15.42:50952) 2024-08-30T09:08:24.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.867+0000 7fa5c3fff640 1 -- 172.21.15.42:0/2305053481 learned_addr learned my addr 172.21.15.42:0/2305053481 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:08:24.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.868+0000 7fa5c3fff640 1 -- 172.21.15.42:0/2305053481 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa5c410a780 msgr2=0x7fa5c41a91b0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-08-30T09:08:24.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.868+0000 7fa5c3fff640 1 --2- 172.21.15.42:0/2305053481 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa5c410a780 0x7fa5c41a91b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:24.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.868+0000 7fa5c3fff640 1 -- 172.21.15.42:0/2305053481 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa5c4111ce0 msgr2=0x7fa5c41a9750 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-30T09:08:24.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.868+0000 7fa5c3fff640 1 --2- 172.21.15.42:0/2305053481 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa5c4111ce0 0x7fa5c41a9750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:24.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.868+0000 7fa5c3fff640 1 -- 172.21.15.42:0/2305053481 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5c41af0f0 con 0x7fa5c411c9d0 2024-08-30T09:08:24.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.869+0000 7fa5c3fff640 1 --2- 172.21.15.42:0/2305053481 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa5c411c9d0 0x7fa5c41ae9f0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fa5b800b3f0 tx=0x7fa5b800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:08:24.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.870+0000 7fa5c0ff9640 1 -- 172.21.15.42:0/2305053481 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5b8015020 con 0x7fa5c411c9d0 2024-08-30T09:08:24.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.870+0000 7fa5ca124640 1 -- 172.21.15.42:0/2305053481 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5c41af3e0 con 0x7fa5c411c9d0 2024-08-30T09:08:24.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.870+0000 7fa5ca124640 1 -- 172.21.15.42:0/2305053481 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5c4112120 con 0x7fa5c411c9d0 2024-08-30T09:08:24.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.871+0000 7fa5c0ff9640 1 -- 172.21.15.42:0/2305053481 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa5b8011ce0 con 0x7fa5c411c9d0 2024-08-30T09:08:24.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.871+0000 7fa5c0ff9640 1 -- 172.21.15.42:0/2305053481 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5b8010e00 con 0x7fa5c411c9d0 2024-08-30T09:08:24.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.876+0000 7fa5d0699740 1 Processor -- start 2024-08-30T09:08:24.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.878+0000 7fa5d0699740 1 -- start start 2024-08-30T09:08:24.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.878+0000 7fa5d0699740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e3c7117350 0x55e3c74bd780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:08:24.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.878+0000 7fa5d0699740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e3c70ee490 0x55e3c74bdcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:08:24.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.878+0000 7fa5d0699740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e3c70f6670 0x55e3c74be200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:08:24.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.878+0000 7fa5d0699740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55e3c74335f0 con 0x55e3c70f6670 2024-08-30T09:08:24.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.878+0000 7fa5d0699740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55e3c7422b80 con 0x55e3c70ee490 2024-08-30T09:08:24.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.878+0000 7fa5d0699740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55e3c7426b30 con 0x55e3c7117350 2024-08-30T09:08:24.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.879+0000 7fa5c37fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e3c70ee490 0x55e3c74bdcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:08:24.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.879+0000 7fa5c2ffd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e3c7117350 0x55e3c74bd780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:08:24.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.879+0000 7fa5c2ffd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e3c7117350 0x55e3c74bd780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:60996/0 (socket says 172.21.15.42:60996) 2024-08-30T09:08:24.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.879+0000 7fa5c2ffd640 1 -- 172.21.15.42:0/4241938506 learned_addr learned my addr 172.21.15.42:0/4241938506 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:08:24.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.879+0000 7fa5c0ff9640 1 -- 172.21.15.42:0/2305053481 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7fa5b8018020 con 0x7fa5c411c9d0 2024-08-30T09:08:24.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.880+0000 7fa5c2ffd640 1 --2- 172.21.15.42:0/4241938506 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e3c70f6670 0x55e3c74be200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:08:24.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.880+0000 7fa5c37fe640 1 -- 172.21.15.42:0/4241938506 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e3c7117350 msgr2=0x55e3c74bd780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:08:24.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.880+0000 7fa5c37fe640 1 --2- 172.21.15.42:0/4241938506 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e3c7117350 0x55e3c74bd780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:24.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.880+0000 7fa5c37fe640 1 -- 172.21.15.42:0/4241938506 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e3c70f6670 msgr2=0x55e3c74be200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:08:24.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.880+0000 7fa5c37fe640 1 --2- 172.21.15.42:0/4241938506 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e3c70f6670 0x55e3c74be200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:24.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.880+0000 7fa5c37fe640 1 -- 172.21.15.42:0/4241938506 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e3c7182930 con 0x55e3c70ee490 2024-08-30T09:08:24.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.880+0000 7fa5c2ffd640 1 --2- 172.21.15.42:0/4241938506 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e3c70f6670 0x55e3c74be200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-30T09:08:24.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.880+0000 7fa5c37fe640 1 --2- 172.21.15.42:0/4241938506 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e3c70ee490 0x55e3c74bdcc0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fa5b4009ad0 tx=0x7fa5b4009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f2f3e64a095c3abc server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:08:24.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.881+0000 7fa5c0ff9640 1 --2- 172.21.15.42:0/2305053481 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fa5ac0b4d00 0x7fa5ac0b7210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:08:24.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.881+0000 7fa5c0ff9640 1 -- 172.21.15.42:0/2305053481 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7fa5b80e5bb0 con 0x7fa5c411c9d0 2024-08-30T09:08:24.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.882+0000 7fa5c2ffd640 1 --2- 172.21.15.42:0/2305053481 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fa5ac0b4d00 0x7fa5ac0b7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:08:24.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.883+0000 7fa5a9ffb640 1 -- 172.21.15.42:0/4241938506 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5b400ed10 con 0x55e3c70ee490 2024-08-30T09:08:24.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.883+0000 7fa5a9ffb640 1 -- 172.21.15.42:0/4241938506 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa5b4005020 con 0x55e3c70ee490 2024-08-30T09:08:24.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.883+0000 7fa5a9ffb640 1 -- 172.21.15.42:0/4241938506 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5b4004a00 con 0x55e3c70ee490 2024-08-30T09:08:24.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.884+0000 7fa5d0699740 1 -- 172.21.15.42:0/4241938506 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e3c70ee490 msgr2=0x55e3c74bdcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:08:24.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.884+0000 7fa5d0699740 1 --2- 172.21.15.42:0/4241938506 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e3c70ee490 0x55e3c74bdcc0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fa5b4009ad0 tx=0x7fa5b4009f90 comp rx=0 tx=0).stop 2024-08-30T09:08:24.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.884+0000 7fa5d0699740 1 -- 172.21.15.42:0/4241938506 shutdown_connections 2024-08-30T09:08:24.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.884+0000 7fa5d0699740 1 --2- 172.21.15.42:0/4241938506 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e3c70f6670 0x55e3c74be200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:24.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.884+0000 7fa5d0699740 1 --2- 172.21.15.42:0/4241938506 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e3c70ee490 0x55e3c74bdcc0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:24.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.884+0000 7fa5d0699740 1 --2- 172.21.15.42:0/4241938506 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e3c7117350 0x55e3c74bd780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:24.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.884+0000 7fa5d0699740 1 -- 172.21.15.42:0/4241938506 >> 172.21.15.42:0/4241938506 conn(0x55e3c710b740 msgr2=0x55e3c74bb670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:08:24.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.885+0000 7fa5c2ffd640 1 --2- 172.21.15.42:0/2305053481 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fa5ac0b4d00 0x7fa5ac0b7210 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fa5bc031b90 tx=0x7fa5bc000f40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:08:24.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.885+0000 7fa5d0699740 1 -- 172.21.15.42:0/4241938506 shutdown_connections 2024-08-30T09:08:24.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.885+0000 7fa5d0699740 1 -- 172.21.15.42:0/4241938506 wait complete. 2024-08-30T09:08:24.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.885+0000 7fa5d0699740 1 -- start start 2024-08-30T09:08:24.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.886+0000 7fa5d0699740 1 Processor -- start 2024-08-30T09:08:24.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.886+0000 7fa5d0699740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e3c70ee490 0x55e3c74c8f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:08:24.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.886+0000 7fa5d0699740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e3c70f6670 0x55e3c74c9470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:08:24.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.886+0000 7fa5d0699740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e3c7117350 0x55e3c74d52e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:08:24.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.886+0000 7fa5d0699740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55e3c7285890 con 0x55e3c7117350 2024-08-30T09:08:24.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.886+0000 7fa5d0699740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55e3c73a9790 con 0x55e3c70f6670 2024-08-30T09:08:24.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.886+0000 7fa5d0699740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55e3c74cbf60 con 0x55e3c70ee490 2024-08-30T09:08:24.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.886+0000 7fa5d0699740 10 client.0 _ll_register_callbacks cb 0x55e3c712eba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:08:24.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.887+0000 7fa5c3fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e3c7117350 0x55e3c74d52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:08:24.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.887+0000 7fa5c3fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e3c7117350 0x55e3c74d52e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:50962/0 (socket says 172.21.15.42:50962) 2024-08-30T09:08:24.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.887+0000 7fa5c3fff640 1 -- 172.21.15.42:0/3296874019 learned_addr learned my addr 172.21.15.42:0/3296874019 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:08:24.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.887+0000 7fa5c3fff640 1 -- 172.21.15.42:0/3296874019 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e3c70ee490 msgr2=0x55e3c74c8f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:08:24.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.887+0000 7fa5c3fff640 1 --2- 172.21.15.42:0/3296874019 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e3c70ee490 0x55e3c74c8f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:24.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.887+0000 7fa5c3fff640 1 -- 172.21.15.42:0/3296874019 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e3c70f6670 msgr2=0x55e3c74c9470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:08:24.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.887+0000 7fa5c3fff640 1 --2- 172.21.15.42:0/3296874019 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e3c70f6670 0x55e3c74c9470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:24.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.887+0000 7fa5c3fff640 1 -- 172.21.15.42:0/3296874019 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e3c73e0f30 con 0x55e3c7117350 2024-08-30T09:08:24.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.888+0000 7fa5a2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-08-30T09:08:24.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.888+0000 7fa5c3fff640 1 --2- 172.21.15.42:0/3296874019 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e3c7117350 0x55e3c74d52e0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fa5b800b3c0 tx=0x7fa5b80ac8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=495671236ee33096 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:08:24.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.888+0000 7fa5a2ffd640 1 -- 172.21.15.42:0/3296874019 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5b80b0a60 con 0x55e3c7117350 2024-08-30T09:08:24.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.888+0000 7fa5d0699740 1 -- 172.21.15.42:0/3296874019 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa5b4005020 con 0x55e3c7117350 2024-08-30T09:08:24.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.888+0000 7fa5a2ffd640 1 -- 172.21.15.42:0/3296874019 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7fa5b80a87e0 con 0x55e3c7117350 2024-08-30T09:08:24.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.889+0000 7fa5a2ffd640 1 -- 172.21.15.42:0/3296874019 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fa5b80b0a60 con 0x55e3c7117350 2024-08-30T09:08:24.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.889+0000 7fa5d0699740 10 client.16746 fetch_fsmap learned FSMap version 35 2024-08-30T09:08:24.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.889+0000 7fa5d0699740 1 -- 172.21.15.42:0/3296874019 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55e3c74d7dd0 con 0x55e3c7117350 2024-08-30T09:08:24.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.889+0000 7fa5a2ffd640 1 -- 172.21.15.42:0/3296874019 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa5b4005020 con 0x55e3c7117350 2024-08-30T09:08:24.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.890+0000 7fa5a2ffd640 1 -- 172.21.15.42:0/3296874019 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 35) ==== 8362+0+0 (secure 0 0 0) 0x7fa5b80a7bd0 con 0x55e3c7117350 2024-08-30T09:08:24.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.890+0000 7fa5d0699740 10 client.16746 fetch_fsmap finished waiting for FSMap version 35 2024-08-30T09:08:24.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.890+0000 7fa5d0699740 10 client.16746 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-30T09:08:24.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.890+0000 7fa5d0699740 20 client.16746 populate_metadata read hostname 'smithi042' 2024-08-30T09:08:24.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.890+0000 7fa5d0699740 1 --2- 172.21.15.42:0/3296874019 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55e3c70d5cd0 0x55e3c74db7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:08:24.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.891+0000 7fa5d0699740 4 client.16746 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-30T09:08:24.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.891+0000 7fa5d0699740 1 -- 172.21.15.42:0/3296874019 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e3c7435540 con 0x55e3c70d5cd0 2024-08-30T09:08:24.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.891+0000 7fa5c37fe640 1 --2- 172.21.15.42:0/3296874019 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55e3c70d5cd0 0x55e3c74db7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:08:24.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.891+0000 7fa5a2ffd640 10 client.16746 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:08:24.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.891+0000 7fa5c37fe640 1 --2- 172.21.15.42:0/3296874019 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55e3c70d5cd0 0x55e3c74db7c0 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=79e3b0e3bf57932e server_cookie=3ac2d7013058aa82 in_seq=0 out_seq=0 2024-08-30T09:08:24.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.892+0000 7fa5a2ffd640 1 -- 172.21.15.42:0/3296874019 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55e3c7435540 con 0x55e3c70d5cd0 2024-08-30T09:08:24.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.892+0000 7fa5a2ffd640 10 client.16746 handle_command_reply: tid=0 2024-08-30T09:08:24.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.892+0000 7fa5d0699740 1 client.16746 shutdown 2024-08-30T09:08:24.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.893+0000 7fa5d0699740 1 -- 172.21.15.42:0/3296874019 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e3c7117350 msgr2=0x55e3c74d52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:08:24.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.893+0000 7fa5d0699740 1 --2- 172.21.15.42:0/3296874019 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e3c7117350 0x55e3c74d52e0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fa5b800b3c0 tx=0x7fa5b80ac8f0 comp rx=0 tx=0).stop 2024-08-30T09:08:24.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.893+0000 7fa5d0699740 1 -- 172.21.15.42:0/3296874019 shutdown_connections 2024-08-30T09:08:24.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.893+0000 7fa5d0699740 1 --2- 172.21.15.42:0/3296874019 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55e3c70d5cd0 0x55e3c74db7c0 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:24.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.893+0000 7fa5a2ffd640 0 client.16746 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:08:24.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.893+0000 7fa5d0699740 1 --2- 172.21.15.42:0/3296874019 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e3c7117350 0x55e3c74d52e0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:24.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.893+0000 7fa5d0699740 1 --2- 172.21.15.42:0/3296874019 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e3c70f6670 0x55e3c74c9470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:24.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.893+0000 7fa5d0699740 1 --2- 172.21.15.42:0/3296874019 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e3c70ee490 0x55e3c74c8f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:24.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.893+0000 7fa5d0699740 1 -- 172.21.15.42:0/3296874019 >> 172.21.15.42:0/3296874019 conn(0x55e3c710b740 msgr2=0x55e3c74c0910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:08:24.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.893+0000 7fa5d0699740 1 -- 172.21.15.42:0/3296874019 shutdown_connections 2024-08-30T09:08:24.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.893+0000 7fa5d0699740 1 -- 172.21.15.42:0/3296874019 wait complete. 2024-08-30T09:08:24.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.893+0000 7fa5d0699740 20 client.16746 trim_cache size 0 max 16384 2024-08-30T09:08:24.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.915+0000 7fa5d0699740 1 Processor -- start 2024-08-30T09:08:24.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.916+0000 7fa5d0699740 1 -- start start 2024-08-30T09:08:24.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.916+0000 7fa5d0699740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e3c70d5cd0 0x55e3c74c0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:08:24.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.916+0000 7fa5d0699740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e3c70ee490 0x55e3c74c6120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:08:24.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.916+0000 7fa5d0699740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e3c70f6670 0x55e3c74c6660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:08:24.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.916+0000 7fa5d0699740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fa5b0012720 con 0x55e3c70f6670 2024-08-30T09:08:24.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.916+0000 7fa5d0699740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa5b000ffc0 con 0x55e3c70d5cd0 2024-08-30T09:08:24.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.916+0000 7fa5d0699740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fa5b000ddb0 con 0x55e3c70ee490 2024-08-30T09:08:24.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.916+0000 7fa5c37fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e3c70d5cd0 0x55e3c74c0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:08:24.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.916+0000 7fa5c3fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e3c70f6670 0x55e3c74c6660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:08:24.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.916+0000 7fa5c37fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e3c70d5cd0 0x55e3c74c0f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:52482/0 (socket says 172.21.15.42:52482) 2024-08-30T09:08:24.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.916+0000 7fa5c37fe640 1 -- 172.21.15.42:0/3976535104 learned_addr learned my addr 172.21.15.42:0/3976535104 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:08:24.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.916+0000 7fa5c2ffd640 1 --2- 172.21.15.42:0/3976535104 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e3c70ee490 0x55e3c74c6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:08:24.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.917+0000 7fa5c37fe640 1 -- 172.21.15.42:0/3976535104 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e3c70ee490 msgr2=0x55e3c74c6120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:08:24.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.917+0000 7fa5c37fe640 1 --2- 172.21.15.42:0/3976535104 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e3c70ee490 0x55e3c74c6120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:24.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.917+0000 7fa5c37fe640 1 -- 172.21.15.42:0/3976535104 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e3c70f6670 msgr2=0x55e3c74c6660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:08:24.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.917+0000 7fa5c37fe640 1 --2- 172.21.15.42:0/3976535104 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e3c70f6670 0x55e3c74c6660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:24.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.917+0000 7fa5c37fe640 1 -- 172.21.15.42:0/3976535104 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e3c74dbd90 con 0x55e3c70d5cd0 2024-08-30T09:08:24.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.917+0000 7fa5c37fe640 1 --2- 172.21.15.42:0/3976535104 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e3c70d5cd0 0x55e3c74c0f90 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fa5b400e7e0 tx=0x7fa5b40096e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=149dd2c8387bde42 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:08:24.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.917+0000 7fa5a97fa640 1 -- 172.21.15.42:0/3976535104 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5b4009750 con 0x55e3c70d5cd0 2024-08-30T09:08:24.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.917+0000 7fa5a97fa640 1 -- 172.21.15.42:0/3976535104 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55e3c7435540 con 0x55e3c70d5cd0 2024-08-30T09:08:24.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.917+0000 7fa5a97fa640 1 -- 172.21.15.42:0/3976535104 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5b4004b50 con 0x55e3c70d5cd0 2024-08-30T09:08:24.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.918+0000 7fa5d0699740 1 -- 172.21.15.42:0/3976535104 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e3c70d5cd0 msgr2=0x55e3c74c0f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:08:24.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.918+0000 7fa5d0699740 1 --2- 172.21.15.42:0/3976535104 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e3c70d5cd0 0x55e3c74c0f90 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fa5b400e7e0 tx=0x7fa5b40096e0 comp rx=0 tx=0).stop 2024-08-30T09:08:24.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.918+0000 7fa5d0699740 1 -- 172.21.15.42:0/3976535104 shutdown_connections 2024-08-30T09:08:24.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.918+0000 7fa5d0699740 1 --2- 172.21.15.42:0/3976535104 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e3c70ee490 0x55e3c74c6120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:24.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.918+0000 7fa5d0699740 1 --2- 172.21.15.42:0/3976535104 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e3c70f6670 0x55e3c74c6660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:24.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.918+0000 7fa5d0699740 1 --2- 172.21.15.42:0/3976535104 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e3c70d5cd0 0x55e3c74c0f90 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:24.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.918+0000 7fa5d0699740 1 -- 172.21.15.42:0/3976535104 >> 172.21.15.42:0/3976535104 conn(0x55e3c710b740 msgr2=0x55e3c74c0880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:08:24.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.918+0000 7fa5d0699740 1 -- 172.21.15.42:0/3976535104 shutdown_connections 2024-08-30T09:08:24.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.919+0000 7fa5d0699740 1 -- 172.21.15.42:0/3976535104 wait complete. 2024-08-30T09:08:24.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.919+0000 7fa5d0699740 1 -- start start 2024-08-30T09:08:24.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.919+0000 7fa5d0699740 1 Processor -- start 2024-08-30T09:08:24.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.920+0000 7fa5d0699740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e3c70d5cd0 0x55e3c74d16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:08:24.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.920+0000 7fa5d0699740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e3c70ee490 0x55e3c74cdba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:08:24.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.920+0000 7fa5d0699740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e3c70f6670 0x55e3c74ce0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:08:24.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.920+0000 7fa5d0699740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fa5b000e480 con 0x55e3c70f6670 2024-08-30T09:08:24.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.920+0000 7fa5d0699740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa5b000f8f0 con 0x55e3c70d5cd0 2024-08-30T09:08:24.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.920+0000 7fa5d0699740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fa5b000eb50 con 0x55e3c70ee490 2024-08-30T09:08:24.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.920+0000 7fa5c37fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e3c70d5cd0 0x55e3c74d16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:08:24.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.920+0000 7fa5c37fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e3c70d5cd0 0x55e3c74d16e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:52490/0 (socket says 172.21.15.42:52490) 2024-08-30T09:08:24.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.920+0000 7fa5c37fe640 1 -- 172.21.15.42:0/404892386 learned_addr learned my addr 172.21.15.42:0/404892386 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:08:24.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.920+0000 7fa5c2ffd640 1 --2- 172.21.15.42:0/404892386 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e3c70ee490 0x55e3c74cdba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:08:24.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.920+0000 7fa5c3fff640 1 --2- 172.21.15.42:0/404892386 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e3c70f6670 0x55e3c74ce0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:08:24.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.920+0000 7fa5d0699740 10 client.0 _ll_register_callbacks cb 0x7fa5b8012f20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:08:24.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.920+0000 7fa5c37fe640 1 -- 172.21.15.42:0/404892386 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e3c70ee490 msgr2=0x55e3c74cdba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:08:24.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.920+0000 7fa5c37fe640 1 --2- 172.21.15.42:0/404892386 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e3c70ee490 0x55e3c74cdba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:24.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.920+0000 7fa5c37fe640 1 -- 172.21.15.42:0/404892386 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e3c70f6670 msgr2=0x55e3c74ce0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:08:24.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.920+0000 7fa5c37fe640 1 --2- 172.21.15.42:0/404892386 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e3c70f6670 0x55e3c74ce0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:24.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.920+0000 7fa5c37fe640 1 -- 172.21.15.42:0/404892386 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e3c74ce620 con 0x55e3c70d5cd0 2024-08-30T09:08:24.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.920+0000 7fa5c37fe640 1 --2- 172.21.15.42:0/404892386 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e3c70d5cd0 0x55e3c74d16e0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fa5b40092e0 tx=0x7fa5b40097c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=720577c341df6637 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:08:24.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.920+0000 7fa5a2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-30T09:08:24.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.921+0000 7fa5a2ffd640 1 -- 172.21.15.42:0/404892386 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5b4004ce0 con 0x55e3c70d5cd0 2024-08-30T09:08:24.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.921+0000 7fa5a2ffd640 1 -- 172.21.15.42:0/404892386 <== mon.2 v2:172.21.15.167:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7fa5b4005040 con 0x55e3c70d5cd0 2024-08-30T09:08:24.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.921+0000 7fa5d0699740 1 -- 172.21.15.42:0/404892386 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55e3c7435540 con 0x55e3c70d5cd0 2024-08-30T09:08:24.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.921+0000 7fa5a2ffd640 1 -- 172.21.15.42:0/404892386 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fa5b403c050 con 0x55e3c70d5cd0 2024-08-30T09:08:24.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.921+0000 7fa5d0699740 10 client.26525 fetch_fsmap learned FSMap version 35 2024-08-30T09:08:24.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.921+0000 7fa5d0699740 1 -- 172.21.15.42:0/404892386 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55e3c7546df0 con 0x55e3c70d5cd0 2024-08-30T09:08:24.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.921+0000 7fa5a2ffd640 1 -- 172.21.15.42:0/404892386 <== mon.2 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55e3c7435540 con 0x55e3c70d5cd0 2024-08-30T09:08:24.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.921+0000 7fa5a2ffd640 1 -- 172.21.15.42:0/404892386 <== mon.2 v2:172.21.15.167:3300/0 5 ==== fsmap(e 35) ==== 8362+0+0 (secure 0 0 0) 0x7fa5b4033070 con 0x55e3c70d5cd0 2024-08-30T09:08:24.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.921+0000 7fa5d0699740 10 client.26525 fetch_fsmap finished waiting for FSMap version 35 2024-08-30T09:08:24.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.921+0000 7fa5d0699740 10 client.26525 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-30T09:08:24.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.921+0000 7fa5d0699740 20 client.26525 populate_metadata read hostname 'smithi042' 2024-08-30T09:08:24.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.921+0000 7fa5d0699740 1 --2- 172.21.15.42:0/404892386 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55e3c7117350 0x55e3c75492c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:08:24.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.921+0000 7fa5d0699740 4 client.26525 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-08-30T09:08:24.930 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:08:24 smithi042 ceph-mon[18652]: pgmap v602: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 4.8 MiB/s wr, 507 op/s 2024-08-30T09:08:24.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.921+0000 7fa5d0699740 1 -- 172.21.15.42:0/404892386 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55e3c7430fe0 con 0x55e3c7117350 2024-08-30T09:08:24.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.922+0000 7fa5c2ffd640 1 --2- 172.21.15.42:0/404892386 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55e3c7117350 0x55e3c75492c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:08:24.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.922+0000 7fa5a2ffd640 10 client.26525 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:08:24.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.922+0000 7fa5c2ffd640 1 --2- 172.21.15.42:0/404892386 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55e3c7117350 0x55e3c75492c0 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a3a04403cf24288a server_cookie=86cde245524f0cc3 in_seq=0 out_seq=0 2024-08-30T09:08:24.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.923+0000 7fa5a2ffd640 1 -- 172.21.15.42:0/404892386 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55e3c7430fe0 con 0x55e3c7117350 2024-08-30T09:08:24.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.923+0000 7fa5a2ffd640 10 client.26525 handle_command_reply: tid=0 2024-08-30T09:08:24.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.924+0000 7fa5d0699740 1 client.26525 shutdown 2024-08-30T09:08:24.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.925+0000 7fa5d0699740 1 -- 172.21.15.42:0/404892386 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e3c70d5cd0 msgr2=0x55e3c74d16e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:08:24.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.925+0000 7fa5d0699740 1 --2- 172.21.15.42:0/404892386 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e3c70d5cd0 0x55e3c74d16e0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fa5b40092e0 tx=0x7fa5b40097c0 comp rx=0 tx=0).stop 2024-08-30T09:08:24.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.925+0000 7fa5d0699740 1 -- 172.21.15.42:0/404892386 shutdown_connections 2024-08-30T09:08:24.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.925+0000 7fa5d0699740 1 --2- 172.21.15.42:0/404892386 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55e3c7117350 0x55e3c75492c0 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:24.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.925+0000 7fa5d0699740 1 --2- 172.21.15.42:0/404892386 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e3c70ee490 0x55e3c74cdba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:24.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.925+0000 7fa5d0699740 1 --2- 172.21.15.42:0/404892386 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e3c70f6670 0x55e3c74ce0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:24.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.925+0000 7fa5d0699740 1 --2- 172.21.15.42:0/404892386 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e3c70d5cd0 0x55e3c74d16e0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:24.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.925+0000 7fa5d0699740 1 -- 172.21.15.42:0/404892386 >> 172.21.15.42:0/404892386 conn(0x55e3c710b740 msgr2=0x55e3c74be760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:08:24.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.925+0000 7fa5a2ffd640 0 client.26525 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:08:24.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.925+0000 7fa5d0699740 1 -- 172.21.15.42:0/404892386 shutdown_connections 2024-08-30T09:08:24.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.925+0000 7fa5d0699740 1 -- 172.21.15.42:0/404892386 wait complete. 2024-08-30T09:08:24.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.925+0000 7fa5d0699740 20 client.26525 trim_cache size 0 max 16384 2024-08-30T09:08:24.934 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-08-30T09:08:24.934 INFO:teuthology.orchestra.run.smithi042.stdout: "return_code": 0, 2024-08-30T09:08:24.934 INFO:teuthology.orchestra.run.smithi042.stdout: "scrub_tag": "99a29ec1-9455-4ae4-854a-398b32896675", 2024-08-30T09:08:24.934 INFO:teuthology.orchestra.run.smithi042.stdout: "mode": "asynchronous" 2024-08-30T09:08:24.934 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-08-30T09:08:24.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.926+0000 7fa5ca124640 1 -- 172.21.15.42:0/2305053481 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fa5ac0b4d00 msgr2=0x7fa5ac0b7210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:08:24.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.926+0000 7fa5ca124640 1 --2- 172.21.15.42:0/2305053481 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fa5ac0b4d00 0x7fa5ac0b7210 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fa5bc031b90 tx=0x7fa5bc000f40 comp rx=0 tx=0).stop 2024-08-30T09:08:24.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.926+0000 7fa5ca124640 1 -- 172.21.15.42:0/2305053481 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa5c411c9d0 msgr2=0x7fa5c41ae9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:08:24.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.926+0000 7fa5ca124640 1 --2- 172.21.15.42:0/2305053481 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa5c411c9d0 0x7fa5c41ae9f0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fa5b800b3f0 tx=0x7fa5b800b8b0 comp rx=0 tx=0).stop 2024-08-30T09:08:24.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.926+0000 7fa5ca124640 1 -- 172.21.15.42:0/2305053481 shutdown_connections 2024-08-30T09:08:24.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.926+0000 7fa5ca124640 1 --2- 172.21.15.42:0/2305053481 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fa5ac0b4d00 0x7fa5ac0b7210 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:24.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.926+0000 7fa5ca124640 1 --2- 172.21.15.42:0/2305053481 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa5c411c9d0 0x7fa5c41ae9f0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:24.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.926+0000 7fa5ca124640 1 --2- 172.21.15.42:0/2305053481 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa5c4111ce0 0x7fa5c41a9750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:24.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.926+0000 7fa5ca124640 1 --2- 172.21.15.42:0/2305053481 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa5c410a780 0x7fa5c41a91b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:24.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.926+0000 7fa5ca124640 1 -- 172.21.15.42:0/2305053481 >> 172.21.15.42:0/2305053481 conn(0x7fa5c4070c60 msgr2=0x7fa5c411e150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:08:24.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.926+0000 7fa5ca124640 1 -- 172.21.15.42:0/2305053481 shutdown_connections 2024-08-30T09:08:24.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:24.927+0000 7fa5ca124640 1 -- 172.21.15.42:0/2305053481 wait complete. 2024-08-30T09:08:25.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:08:24 smithi190 ceph-mon[22001]: pgmap v602: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 4.8 MiB/s wr, 507 op/s 2024-08-30T09:08:25.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:08:24 smithi167 ceph-mon[22063]: pgmap v602: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 4.8 MiB/s wr, 507 op/s 2024-08-30T09:08:25.327 INFO:journalctl@ceph.mds.b.smithi167.stdout:Aug 30 09:08:24 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mds-b[52473]: 2024-08-30T09:08:24.956+0000 7f9d3fee4640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000000f [...2,head] /volumes/qa/sv_1/d3168875-d2e5-410d-8754-72d29955777b/postgres/data/pg_subtrans/ auth{1=1} v2239 ap=2 f(v9 m2024-08-30T09:07:25.092709+0000 1=1+0) n(v22 rc2024-08-30T09:07:25.092709+0000 b212992 2=1+1) (inest sync->mix g=1) caps={15402=pAsLsXsFs/pAsLsXsFsxcral@24} mcw={1=AsLsXsFs} | dirtyscattered=0 request=0 lock=0 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x555a32f6d080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x1000000000f:[<0x10000000006/pg_subtrans v284>,<0x10000000005/data v125>,<0x100000001fb/postgres v31>,<0x100000001fa/d3168875-d2e5-410d-8754-72d29955777b v45>,<0x10000000001/sv_1 v63>,<0x10000000000/qa v49>,<0x1/volumes v44>]//[]","memoryvalue":"(2)0x1000000000f:[<0x10000000006/pg_subtrans v2239>,<0x10000000005/data v695>,<0x100000001fb/postgres v414>,<0x100000001fa/d3168875-d2e5-410d-8754-72d29955777b v433>,<0x10000000001/sv_1 v448>,<0x10000000000/qa v429>,<0x1/volumes v395>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-08-30T09:07:25.039710+0000 9=9+0)","ondisk_value.rstat":"n(v0 rc2024-08-30T09:07:25.092709+0000 b212992 2=1+1)","memory_value.dirstat":"f(v9 m2024-08-30T09:07:25.092709+0000 1=1+0)","memory_value.rstat":"n(v22 rc2024-08-30T09:07:25.092709+0000 b212992 2=1+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-08-30T09:08:25.328 INFO:journalctl@ceph.mds.b.smithi167.stdout:Aug 30 09:08:24 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mds-b[52473]: 2024-08-30T09:08:24.961+0000 7f9d3fee4640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000004a7 [...2,head] /volumes/qa/sv_1/d3168875-d2e5-410d-8754-72d29955777b/postgres/data/base/13494/ auth{1=1,2=1} fragtree_t(*^3) v4486 ap=2 f(v2 m2024-08-30T09:07:23.934719+0000 305=305+0) n(v39 rc2024-08-30T09:08:17.512267+0000 b554037807 306=305+1) (inest sync->mix g=1,2) caps={15402=pAsLsXsFs/pAsLsXsFsxcral@45} mcw={2=AsLsXsFs} | ptrwaiter=0 dirtyscattered=0 request=0 lock=0 importingcaps=0 dirfrag=8 caps=1 stickydirs=0 exportingcaps=0 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 tempexporting=0 scrubqueue=0 0x555a33734580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x100000004a7:[<0x10000000014/13494 v3287>,<0x10000000006/base v934>,<0x10000000005/data v125>,<0x100000001fb/postgres v31>,<0x100000001fa/d3168875-d2e5-410d-8754-72d29955777b v45>,<0x10000000001/sv_1 v63>,<0x10000000000/qa v49>,<0x1/volumes v44>]//[]","memoryvalue":"(2)0x100000004a7:[<0x10000000014/13494 v4486>,<0x10000000006/base v2326>,<0x10000000005/data v695>,<0x100000001fb/postgres v414>,<0x100000001fa/d3168875-d2e5-410d-8754-72d29955777b v433>,<0x10000000001/sv_1 v448>,<0x10000000000/qa v429>,<0x1/volumes v395>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-08-30T09:07:23.934719+0000 308=308+0)","ondisk_value.rstat":"n(v0 rc2024-08-30T09:08:17.512267+0000 b554037807 306=305+1)","memory_value.dirstat":"f(v2 m2024-08-30T09:07:23.934719+0000 305=305+0)","memory_value.rstat":"n(v39 rc2024-08-30T09:08:17.512267+0000 b554037807 306=305+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-08-30T09:08:25.773 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-30T09:08:25.953 INFO:teuthology.orchestra.run.smithi042.stderr:progress: 460.0 s, 490.8 tps, lat 206.420 ms stddev 281.724 2024-08-30T09:08:26.135 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:08:25 smithi042 ceph-mon[18652]: scrub queued for path: / 2024-08-30T09:08:26.135 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:08:25 smithi042 ceph-mon[18652]: scrub summary: idle+waiting paths [/] 2024-08-30T09:08:26.135 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:08:25 smithi042 ceph-mon[18652]: scrub summary: active paths [/] 2024-08-30T09:08:26.135 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:08:25 smithi042 ceph-mon[18652]: scrub summary: active paths [/] 2024-08-30T09:08:26.135 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:08:25 smithi042 ceph-mon[18652]: scrub summary: active paths [/] 2024-08-30T09:08:26.136 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:08:25 smithi042 ceph-mon[18652]: scrub summary: idle+waiting paths [/] 2024-08-30T09:08:26.136 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:08:25 smithi042 ceph-mon[18652]: scrub summary: active paths [/] 2024-08-30T09:08:26.136 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:08:25 smithi042 ceph-mon[18652]: scrub summary: idle+waiting paths [/] 2024-08-30T09:08:26.136 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:08:25 smithi042 ceph-mon[18652]: scrub summary: active paths [/] 2024-08-30T09:08:26.136 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:08:25 smithi042 ceph-mon[18652]: scrub summary: idle+waiting paths [/] 2024-08-30T09:08:26.136 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:08:25 smithi042 ceph-mon[18652]: scrub summary: idle+waiting paths [/] 2024-08-30T09:08:26.136 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:08:25 smithi042 ceph-mon[18652]: scrub summary: active paths [/] 2024-08-30T09:08:26.136 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:08:25 smithi042 ceph-mon[18652]: scrub summary: active paths [/] 2024-08-30T09:08:26.136 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:08:25 smithi042 ceph-mon[18652]: scrub summary: idle+waiting paths [/] 2024-08-30T09:08:26.136 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:08:25 smithi042 ceph-mon[18652]: scrub summary: active paths [/] 2024-08-30T09:08:26.136 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:08:25 smithi042 ceph-mon[18652]: Scrub error on inode 0x1000000000f (/volumes/qa/sv_1/d3168875-d2e5-410d-8754-72d29955777b/postgres/data/pg_subtrans) see mds.b log and `damage ls` output for details 2024-08-30T09:08:26.136 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:08:25 smithi042 ceph-mon[18652]: Scrub error on inode 0x100000004a7 (/volumes/qa/sv_1/d3168875-d2e5-410d-8754-72d29955777b/postgres/data/base/13494) see mds.b log and `damage ls` output for details 2024-08-30T09:08:26.136 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:08:25 smithi042 ceph-mon[18652]: scrub summary: idle+waiting paths [/] 2024-08-30T09:08:26.137 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:08:25 smithi042 ceph-mon[18652]: scrub summary: idle+waiting paths [/] 2024-08-30T09:08:26.137 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:08:25 smithi042 ceph-mon[18652]: scrub summary: idle+waiting paths [/] 2024-08-30T09:08:26.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:08:25 smithi190 ceph-mon[22001]: scrub queued for path: / 2024-08-30T09:08:26.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:08:25 smithi190 ceph-mon[22001]: scrub summary: idle+waiting paths [/] 2024-08-30T09:08:26.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:08:25 smithi190 ceph-mon[22001]: scrub summary: active paths [/] 2024-08-30T09:08:26.187 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:08:25 smithi190 ceph-mon[22001]: scrub summary: active paths [/] 2024-08-30T09:08:26.187 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:08:25 smithi190 ceph-mon[22001]: scrub summary: active paths [/] 2024-08-30T09:08:26.187 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:08:25 smithi190 ceph-mon[22001]: scrub summary: idle+waiting paths [/] 2024-08-30T09:08:26.187 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:08:25 smithi190 ceph-mon[22001]: scrub summary: active paths [/] 2024-08-30T09:08:26.187 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:08:25 smithi190 ceph-mon[22001]: scrub summary: idle+waiting paths [/] 2024-08-30T09:08:26.187 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:08:25 smithi190 ceph-mon[22001]: scrub summary: active paths [/] 2024-08-30T09:08:26.187 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:08:25 smithi190 ceph-mon[22001]: scrub summary: idle+waiting paths [/] 2024-08-30T09:08:26.187 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:08:25 smithi190 ceph-mon[22001]: scrub summary: idle+waiting paths [/] 2024-08-30T09:08:26.187 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:08:25 smithi190 ceph-mon[22001]: scrub summary: active paths [/] 2024-08-30T09:08:26.187 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:08:25 smithi190 ceph-mon[22001]: scrub summary: active paths [/] 2024-08-30T09:08:26.187 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:08:25 smithi190 ceph-mon[22001]: scrub summary: idle+waiting paths [/] 2024-08-30T09:08:26.187 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:08:25 smithi190 ceph-mon[22001]: scrub summary: active paths [/] 2024-08-30T09:08:26.187 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:08:25 smithi190 ceph-mon[22001]: Scrub error on inode 0x1000000000f (/volumes/qa/sv_1/d3168875-d2e5-410d-8754-72d29955777b/postgres/data/pg_subtrans) see mds.b log and `damage ls` output for details 2024-08-30T09:08:26.187 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:08:25 smithi190 ceph-mon[22001]: Scrub error on inode 0x100000004a7 (/volumes/qa/sv_1/d3168875-d2e5-410d-8754-72d29955777b/postgres/data/base/13494) see mds.b log and `damage ls` output for details 2024-08-30T09:08:26.187 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:08:25 smithi190 ceph-mon[22001]: scrub summary: idle+waiting paths [/] 2024-08-30T09:08:26.188 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:08:25 smithi190 ceph-mon[22001]: scrub summary: idle+waiting paths [/] 2024-08-30T09:08:26.188 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:08:25 smithi190 ceph-mon[22001]: scrub summary: idle+waiting paths [/] 2024-08-30T09:08:26.322 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T09:08:26.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:08:25 smithi167 ceph-mon[22063]: scrub queued for path: / 2024-08-30T09:08:26.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:08:25 smithi167 ceph-mon[22063]: scrub summary: idle+waiting paths [/] 2024-08-30T09:08:26.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:08:25 smithi167 ceph-mon[22063]: scrub summary: active paths [/] 2024-08-30T09:08:26.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:08:25 smithi167 ceph-mon[22063]: scrub summary: active paths [/] 2024-08-30T09:08:26.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:08:25 smithi167 ceph-mon[22063]: scrub summary: active paths [/] 2024-08-30T09:08:26.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:08:25 smithi167 ceph-mon[22063]: scrub summary: idle+waiting paths [/] 2024-08-30T09:08:26.328 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:08:25 smithi167 ceph-mon[22063]: scrub summary: active paths [/] 2024-08-30T09:08:26.328 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:08:25 smithi167 ceph-mon[22063]: scrub summary: idle+waiting paths [/] 2024-08-30T09:08:26.328 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:08:25 smithi167 ceph-mon[22063]: scrub summary: active paths [/] 2024-08-30T09:08:26.328 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:08:25 smithi167 ceph-mon[22063]: scrub summary: idle+waiting paths [/] 2024-08-30T09:08:26.328 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:08:25 smithi167 ceph-mon[22063]: scrub summary: idle+waiting paths [/] 2024-08-30T09:08:26.328 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:08:25 smithi167 ceph-mon[22063]: scrub summary: active paths [/] 2024-08-30T09:08:26.328 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:08:25 smithi167 ceph-mon[22063]: scrub summary: active paths [/] 2024-08-30T09:08:26.328 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:08:25 smithi167 ceph-mon[22063]: scrub summary: idle+waiting paths [/] 2024-08-30T09:08:26.328 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:08:25 smithi167 ceph-mon[22063]: scrub summary: active paths [/] 2024-08-30T09:08:26.328 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:08:25 smithi167 ceph-mon[22063]: Scrub error on inode 0x1000000000f (/volumes/qa/sv_1/d3168875-d2e5-410d-8754-72d29955777b/postgres/data/pg_subtrans) see mds.b log and `damage ls` output for details 2024-08-30T09:08:26.328 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:08:25 smithi167 ceph-mon[22063]: Scrub error on inode 0x100000004a7 (/volumes/qa/sv_1/d3168875-d2e5-410d-8754-72d29955777b/postgres/data/base/13494) see mds.b log and `damage ls` output for details 2024-08-30T09:08:26.328 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:08:25 smithi167 ceph-mon[22063]: scrub summary: idle+waiting paths [/] 2024-08-30T09:08:26.328 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:08:25 smithi167 ceph-mon[22063]: scrub summary: idle+waiting paths [/] 2024-08-30T09:08:26.328 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:08:25 smithi167 ceph-mon[22063]: scrub summary: idle+waiting paths [/] 2024-08-30T09:08:27.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:08:26 smithi042 ceph-mon[18652]: pgmap v603: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 4.3 MiB/s wr, 454 op/s 2024-08-30T09:08:27.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:08:26 smithi190 ceph-mon[22001]: pgmap v603: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 4.3 MiB/s wr, 454 op/s 2024-08-30T09:08:27.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:08:26 smithi167 ceph-mon[22063]: pgmap v603: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 4.3 MiB/s wr, 454 op/s 2024-08-30T09:08:27.983 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-08-30T09:08:27.983 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "IDLE (waiting for more scrubs)", 2024-08-30T09:08:27.983 INFO:teuthology.orchestra.run.smithi042.stdout: "scrubs": { 2024-08-30T09:08:27.983 INFO:teuthology.orchestra.run.smithi042.stdout: "99a29ec1-9455-4ae4-854a-398b32896675": { 2024-08-30T09:08:27.983 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:08:27.983 INFO:teuthology.orchestra.run.smithi042.stdout: "tag": "99a29ec1-9455-4ae4-854a-398b32896675", 2024-08-30T09:08:27.983 INFO:teuthology.orchestra.run.smithi042.stdout: "options": "recursive,force" 2024-08-30T09:08:27.983 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:08:27.984 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:08:27.984 INFO:teuthology.orchestra.run.smithi042.stdout: "scrub_stats": { 2024-08-30T09:08:27.984 INFO:teuthology.orchestra.run.smithi042.stdout: "99a29ec1-9455-4ae4-854a-398b32896675": { 2024-08-30T09:08:27.984 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:08:24.923176+0000", 2024-08-30T09:08:27.984 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:08:27.984 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:08:27.984 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:08:27.984 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:08:27.984 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:08:27.984 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:08:27.984 INFO:teuthology.orchestra.run.smithi042.stdout: "598d8a75-f1cc-4388-80bc-a408bdb4eaac": { 2024-08-30T09:08:27.984 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:07:29.721122+0000", 2024-08-30T09:08:27.984 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:08:27.985 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:08:27.985 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:08:27.985 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:08:27.985 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:08:27.985 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:08:27.985 INFO:teuthology.orchestra.run.smithi042.stdout: "665ec740-7254-49cb-84ef-deff71c461b0": { 2024-08-30T09:08:27.985 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:06:37.144035+0000", 2024-08-30T09:08:27.985 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:08:27.985 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:08:27.985 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:08:27.985 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:08:27.985 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:08:27.985 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:08:27.985 INFO:teuthology.orchestra.run.smithi042.stdout: "1af4252a-4a99-47e9-bbf0-becf5e573dd5": { 2024-08-30T09:08:27.986 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:05:48.565881+0000", 2024-08-30T09:08:27.986 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:08:27.986 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:08:27.986 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:08:27.986 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:08:27.986 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:08:27.986 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:08:27.986 INFO:teuthology.orchestra.run.smithi042.stdout: "347d05ed-4bd3-4590-b099-314f719aab39": { 2024-08-30T09:08:27.986 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:05:00.439721+0000", 2024-08-30T09:08:27.986 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:08:27.986 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:08:27.986 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:08:27.986 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:08:27.986 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:08:27.987 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:08:27.987 INFO:teuthology.orchestra.run.smithi042.stdout: "88cd7478-2b20-47de-ba51-256479749232": { 2024-08-30T09:08:27.987 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:04:08.922625+0000", 2024-08-30T09:08:27.987 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:08:27.987 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:08:27.987 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:08:27.987 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:08:27.987 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:08:27.987 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:08:27.987 INFO:teuthology.orchestra.run.smithi042.stdout: "749f4867-0b70-49e5-af2d-8362a240ebc3": { 2024-08-30T09:08:27.987 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:03:16.386554+0000", 2024-08-30T09:08:27.987 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:08:27.987 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:08:27.987 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:08:27.988 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:08:27.988 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:08:27.988 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:08:27.988 INFO:teuthology.orchestra.run.smithi042.stdout: "85134e4b-ab4a-4885-b07e-a3ad01eaf13d": { 2024-08-30T09:08:27.988 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:01:27.738462+0000", 2024-08-30T09:08:27.988 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:08:27.988 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:08:27.988 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:08:27.988 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:08:27.988 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:08:27.988 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:08:27.988 INFO:teuthology.orchestra.run.smithi042.stdout: "c827ad7a-44ee-4e1d-9bb3-51fb7ab00c73": { 2024-08-30T09:08:27.988 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:02:21.686517+0000", 2024-08-30T09:08:27.988 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:08:27.989 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:08:27.989 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:08:27.989 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:08:27.989 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:08:27.989 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:08:27.989 INFO:teuthology.orchestra.run.smithi042.stdout: "5e2ed104-0625-478a-a78b-584e7f4c2817": { 2024-08-30T09:08:27.989 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:00:33.928400+0000", 2024-08-30T09:08:27.989 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:08:27.989 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:08:27.989 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:08:27.989 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:08:27.989 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:08:27.989 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:08:27.989 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:08:27.990 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-08-30T09:08:27.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.891+0000 7fc956f70640 1 -- 172.21.15.42:0/2872009939 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc950114200 msgr2=0x7fc9501166d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:08:27.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.891+0000 7fc956f70640 1 --2- 172.21.15.42:0/2872009939 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc950114200 0x7fc9501166d0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fc944009900 tx=0x7fc94402a3f0 comp rx=0 tx=0).stop 2024-08-30T09:08:27.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.891+0000 7fc956f70640 1 -- 172.21.15.42:0/2872009939 shutdown_connections 2024-08-30T09:08:27.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.891+0000 7fc956f70640 1 --2- 172.21.15.42:0/2872009939 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc950114200 0x7fc9501166d0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:27.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.891+0000 7fc956f70640 1 --2- 172.21.15.42:0/2872009939 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc950075b70 0x7fc950075f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:27.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.891+0000 7fc956f70640 1 --2- 172.21.15.42:0/2872009939 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc950077120 0x7fc950075550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:27.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.891+0000 7fc956f70640 1 -- 172.21.15.42:0/2872009939 >> 172.21.15.42:0/2872009939 conn(0x7fc9500ff380 msgr2=0x7fc9501017d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:08:27.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.891+0000 7fc956f70640 1 -- 172.21.15.42:0/2872009939 shutdown_connections 2024-08-30T09:08:27.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.891+0000 7fc956f70640 1 -- 172.21.15.42:0/2872009939 wait complete. 2024-08-30T09:08:27.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.892+0000 7fc956f70640 1 Processor -- start 2024-08-30T09:08:27.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.892+0000 7fc956f70640 1 -- start start 2024-08-30T09:08:27.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.893+0000 7fc956f70640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc950075b70 0x7fc9501a07c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:08:27.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.893+0000 7fc956f70640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc950077120 0x7fc9501a0d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:08:27.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.893+0000 7fc956f70640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc950114200 0x7fc9501a6000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:08:27.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.893+0000 7fc956f70640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fc950119e90 con 0x7fc950075b70 2024-08-30T09:08:27.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.893+0000 7fc956f70640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc950119d10 con 0x7fc950114200 2024-08-30T09:08:27.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.893+0000 7fc956f70640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fc95011a010 con 0x7fc950077120 2024-08-30T09:08:27.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.893+0000 7fc954d0c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc950075b70 0x7fc9501a07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:08:27.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.893+0000 7fc954d0c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc950075b70 0x7fc9501a07c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:50994/0 (socket says 172.21.15.42:50994) 2024-08-30T09:08:27.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.893+0000 7fc954d0c640 1 -- 172.21.15.42:0/2085964462 learned_addr learned my addr 172.21.15.42:0/2085964462 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:08:27.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.893+0000 7fc94ffff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc950077120 0x7fc9501a0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:08:27.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.893+0000 7fc95550d640 1 --2- 172.21.15.42:0/2085964462 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc950114200 0x7fc9501a6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:08:27.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.893+0000 7fc94ffff640 1 -- 172.21.15.42:0/2085964462 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc950114200 msgr2=0x7fc9501a6000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:08:27.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.893+0000 7fc94ffff640 1 --2- 172.21.15.42:0/2085964462 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc950114200 0x7fc9501a6000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:27.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.893+0000 7fc94ffff640 1 -- 172.21.15.42:0/2085964462 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc950075b70 msgr2=0x7fc9501a07c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:08:27.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.893+0000 7fc94ffff640 1 --2- 172.21.15.42:0/2085964462 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc950075b70 0x7fc9501a07c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:27.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.893+0000 7fc94ffff640 1 -- 172.21.15.42:0/2085964462 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9501a6730 con 0x7fc950077120 2024-08-30T09:08:27.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.893+0000 7fc954d0c640 1 --2- 172.21.15.42:0/2085964462 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc950075b70 0x7fc9501a07c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-30T09:08:27.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.893+0000 7fc95550d640 1 --2- 172.21.15.42:0/2085964462 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc950114200 0x7fc9501a6000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-30T09:08:27.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.894+0000 7fc94ffff640 1 --2- 172.21.15.42:0/2085964462 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc950077120 0x7fc9501a0d60 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fc94000e9c0 tx=0x7fc94000ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:08:27.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.894+0000 7fc94dffb640 1 -- 172.21.15.42:0/2085964462 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc94000ccc0 con 0x7fc950077120 2024-08-30T09:08:27.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.894+0000 7fc94dffb640 1 -- 172.21.15.42:0/2085964462 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc940005020 con 0x7fc950077120 2024-08-30T09:08:27.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.894+0000 7fc956f70640 1 -- 172.21.15.42:0/2085964462 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9501a6a20 con 0x7fc950077120 2024-08-30T09:08:27.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.894+0000 7fc956f70640 1 -- 172.21.15.42:0/2085964462 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9501ad180 con 0x7fc950077120 2024-08-30T09:08:27.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.897+0000 7fc94dffb640 1 -- 172.21.15.42:0/2085964462 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc940004a00 con 0x7fc950077120 2024-08-30T09:08:27.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.900+0000 7fc95d4e5740 1 Processor -- start 2024-08-30T09:08:27.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.900+0000 7fc94dffb640 1 -- 172.21.15.42:0/2085964462 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7fc940004ba0 con 0x7fc950077120 2024-08-30T09:08:27.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.900+0000 7fc95d4e5740 1 -- start start 2024-08-30T09:08:27.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.901+0000 7fc95d4e5740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556fd4ec6070 0x556fd526b300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:08:27.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.901+0000 7fc95d4e5740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556fd4e9d1b0 0x556fd526b840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:08:27.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.901+0000 7fc95d4e5740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556fd4ea5390 0x556fd526bd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:08:27.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.901+0000 7fc95d4e5740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x556fd51e2630 con 0x556fd4ec6070 2024-08-30T09:08:27.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.901+0000 7fc95d4e5740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x556fd51d1bc0 con 0x556fd4ea5390 2024-08-30T09:08:27.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.901+0000 7fc95d4e5740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x556fd51d5b70 con 0x556fd4e9d1b0 2024-08-30T09:08:27.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.901+0000 7fc94ffff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556fd4ea5390 0x556fd526bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:08:27.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.901+0000 7fc94ffff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556fd4ea5390 0x556fd526bd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:52504/0 (socket says 172.21.15.42:52504) 2024-08-30T09:08:27.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.901+0000 7fc94ffff640 1 -- 172.21.15.42:0/133174079 learned_addr learned my addr 172.21.15.42:0/133174079 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:08:27.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.901+0000 7fc95550d640 1 --2- 172.21.15.42:0/133174079 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556fd4ec6070 0x556fd526b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:08:27.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.901+0000 7fc94ffff640 1 -- 172.21.15.42:0/133174079 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556fd4e9d1b0 msgr2=0x556fd526b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:08:27.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.901+0000 7fc94ffff640 1 --2- 172.21.15.42:0/133174079 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556fd4e9d1b0 0x556fd526b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:27.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.901+0000 7fc94ffff640 1 -- 172.21.15.42:0/133174079 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556fd4ec6070 msgr2=0x556fd526b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:08:27.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.901+0000 7fc94ffff640 1 --2- 172.21.15.42:0/133174079 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556fd4ec6070 0x556fd526b300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:27.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.901+0000 7fc94ffff640 1 -- 172.21.15.42:0/133174079 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556fd4f31650 con 0x556fd4ea5390 2024-08-30T09:08:27.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.902+0000 7fc94ffff640 1 --2- 172.21.15.42:0/133174079 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556fd4ea5390 0x556fd526bd80 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fc9400eabc0 tx=0x7fc9400ec610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a5ede263b19f12a7 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:08:27.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.902+0000 7fc936ffd640 1 -- 172.21.15.42:0/133174079 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9400f4020 con 0x556fd4ea5390 2024-08-30T09:08:27.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.902+0000 7fc936ffd640 1 -- 172.21.15.42:0/133174079 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc9400ece40 con 0x556fd4ea5390 2024-08-30T09:08:27.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.902+0000 7fc936ffd640 1 -- 172.21.15.42:0/133174079 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9400e4e60 con 0x556fd4ea5390 2024-08-30T09:08:27.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.903+0000 7fc94dffb640 1 --2- 172.21.15.42:0/2085964462 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fc9380b4c30 0x7fc9380b7140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:08:27.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.903+0000 7fc95d4e5740 1 -- 172.21.15.42:0/133174079 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556fd4ea5390 msgr2=0x556fd526bd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:08:27.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.903+0000 7fc95d4e5740 1 --2- 172.21.15.42:0/133174079 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556fd4ea5390 0x556fd526bd80 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fc9400eabc0 tx=0x7fc9400ec610 comp rx=0 tx=0).stop 2024-08-30T09:08:27.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.904+0000 7fc94dffb640 1 -- 172.21.15.42:0/2085964462 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7fc9400e4ab0 con 0x7fc950077120 2024-08-30T09:08:27.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.904+0000 7fc95550d640 1 --2- 172.21.15.42:0/2085964462 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fc9380b4c30 0x7fc9380b7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:08:27.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.904+0000 7fc95d4e5740 1 -- 172.21.15.42:0/133174079 shutdown_connections 2024-08-30T09:08:27.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.904+0000 7fc95d4e5740 1 --2- 172.21.15.42:0/133174079 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556fd4e9d1b0 0x556fd526b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:27.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.904+0000 7fc95d4e5740 1 --2- 172.21.15.42:0/133174079 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556fd4ea5390 0x556fd526bd80 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:27.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.904+0000 7fc95d4e5740 1 --2- 172.21.15.42:0/133174079 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556fd4ec6070 0x556fd526b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:27.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.904+0000 7fc95d4e5740 1 -- 172.21.15.42:0/133174079 >> 172.21.15.42:0/133174079 conn(0x556fd4eba460 msgr2=0x556fd52691f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:08:27.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.905+0000 7fc95d4e5740 1 -- 172.21.15.42:0/133174079 shutdown_connections 2024-08-30T09:08:27.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.905+0000 7fc95d4e5740 1 -- 172.21.15.42:0/133174079 wait complete. 2024-08-30T09:08:27.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.905+0000 7fc95550d640 1 --2- 172.21.15.42:0/2085964462 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fc9380b4c30 0x7fc9380b7140 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fc944006010 tx=0x7fc944005a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:08:27.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.905+0000 7fc95d4e5740 1 -- start start 2024-08-30T09:08:27.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.905+0000 7fc95d4e5740 1 Processor -- start 2024-08-30T09:08:27.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.906+0000 7fc95d4e5740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556fd4e9d1b0 0x556fd5276b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:08:27.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.906+0000 7fc95d4e5740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556fd4ea5390 0x556fd5277070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:08:27.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.906+0000 7fc95d4e5740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556fd4ec6070 0x556fd5293e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:08:27.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.906+0000 7fc95d4e5740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x556fd5034700 con 0x556fd4e9d1b0 2024-08-30T09:08:27.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.906+0000 7fc95d4e5740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x556fd5158700 con 0x556fd4ec6070 2024-08-30T09:08:27.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.906+0000 7fc95d4e5740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x556fd528aaf0 con 0x556fd4ea5390 2024-08-30T09:08:27.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.906+0000 7fc95d4e5740 10 client.0 _ll_register_callbacks cb 0x556fd5187dc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:08:27.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.906+0000 7fc954d0c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556fd4e9d1b0 0x556fd5276b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:08:27.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.906+0000 7fc954d0c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556fd4e9d1b0 0x556fd5276b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:51018/0 (socket says 172.21.15.42:51018) 2024-08-30T09:08:27.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.906+0000 7fc954d0c640 1 -- 172.21.15.42:0/2919742752 learned_addr learned my addr 172.21.15.42:0/2919742752 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:08:27.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.906+0000 7fc95550d640 1 --2- 172.21.15.42:0/2919742752 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556fd4ec6070 0x556fd5293e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:08:27.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.907+0000 7fc954d0c640 1 -- 172.21.15.42:0/2919742752 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556fd4ea5390 msgr2=0x556fd5277070 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-30T09:08:27.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.907+0000 7fc954d0c640 1 --2- 172.21.15.42:0/2919742752 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556fd4ea5390 0x556fd5277070 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:27.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.907+0000 7fc954d0c640 1 -- 172.21.15.42:0/2919742752 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556fd4ec6070 msgr2=0x556fd5293e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:08:27.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.907+0000 7fc954d0c640 1 --2- 172.21.15.42:0/2919742752 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556fd4ec6070 0x556fd5293e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:27.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.907+0000 7fc954d0c640 1 -- 172.21.15.42:0/2919742752 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556fd51918b0 con 0x556fd4e9d1b0 2024-08-30T09:08:27.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.907+0000 7fc95550d640 1 --2- 172.21.15.42:0/2919742752 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556fd4ec6070 0x556fd5293e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-30T09:08:27.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.907+0000 7fc954d0c640 1 --2- 172.21.15.42:0/2919742752 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556fd4e9d1b0 0x556fd5276b30 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fc948004a80 tx=0x7fc94800f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa3569662863bdeb server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:08:27.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.907+0000 7fc92ffff640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-08-30T09:08:27.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.907+0000 7fc92ffff640 1 -- 172.21.15.42:0/2919742752 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc94801e060 con 0x556fd4e9d1b0 2024-08-30T09:08:27.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.907+0000 7fc92ffff640 1 -- 172.21.15.42:0/2919742752 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7fc948003b70 con 0x556fd4e9d1b0 2024-08-30T09:08:27.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.907+0000 7fc95d4e5740 1 -- 172.21.15.42:0/2919742752 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x556fd51e4580 con 0x556fd4e9d1b0 2024-08-30T09:08:27.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.908+0000 7fc92ffff640 1 -- 172.21.15.42:0/2919742752 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fc948025050 con 0x556fd4e9d1b0 2024-08-30T09:08:27.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.908+0000 7fc95d4e5740 10 client.16764 fetch_fsmap learned FSMap version 35 2024-08-30T09:08:27.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.908+0000 7fc95d4e5740 1 -- 172.21.15.42:0/2919742752 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x556fd5296a40 con 0x556fd4e9d1b0 2024-08-30T09:08:27.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.908+0000 7fc92ffff640 1 -- 172.21.15.42:0/2919742752 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x556fd51e4580 con 0x556fd4e9d1b0 2024-08-30T09:08:27.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.908+0000 7fc92ffff640 1 -- 172.21.15.42:0/2919742752 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 35) ==== 8362+0+0 (secure 0 0 0) 0x7fc94800e8a0 con 0x556fd4e9d1b0 2024-08-30T09:08:27.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.908+0000 7fc95d4e5740 10 client.16764 fetch_fsmap finished waiting for FSMap version 35 2024-08-30T09:08:27.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.909+0000 7fc95d4e5740 10 client.16764 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-30T09:08:27.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.909+0000 7fc95d4e5740 20 client.16764 populate_metadata read hostname 'smithi042' 2024-08-30T09:08:27.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.909+0000 7fc95d4e5740 1 --2- 172.21.15.42:0/2919742752 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x556fd4e849f0 0x556fd5299ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:08:27.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.909+0000 7fc95d4e5740 4 client.16764 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-30T09:08:27.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.909+0000 7fc95d4e5740 1 -- 172.21.15.42:0/2919742752 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556fd51e0020 con 0x556fd4e849f0 2024-08-30T09:08:27.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.909+0000 7fc94ffff640 1 --2- 172.21.15.42:0/2919742752 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x556fd4e849f0 0x556fd5299ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:08:27.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.910+0000 7fc94ffff640 1 --2- 172.21.15.42:0/2919742752 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x556fd4e849f0 0x556fd5299ff0 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e01f77ade35a18ae server_cookie=703cdf8b17fe7a8d in_seq=0 out_seq=0 2024-08-30T09:08:27.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.910+0000 7fc92ffff640 10 client.16764 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:08:27.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.911+0000 7fc92ffff640 1 -- 172.21.15.42:0/2919742752 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x556fd51e0020 con 0x556fd4e849f0 2024-08-30T09:08:27.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.911+0000 7fc92ffff640 10 client.16764 handle_command_reply: tid=0 2024-08-30T09:08:27.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.911+0000 7fc95d4e5740 1 client.16764 shutdown 2024-08-30T09:08:27.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.912+0000 7fc95d4e5740 1 -- 172.21.15.42:0/2919742752 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556fd4e9d1b0 msgr2=0x556fd5276b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:08:27.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.912+0000 7fc95d4e5740 1 --2- 172.21.15.42:0/2919742752 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556fd4e9d1b0 0x556fd5276b30 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fc948004a80 tx=0x7fc94800f790 comp rx=0 tx=0).stop 2024-08-30T09:08:27.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.912+0000 7fc95d4e5740 1 -- 172.21.15.42:0/2919742752 shutdown_connections 2024-08-30T09:08:27.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.912+0000 7fc95d4e5740 1 --2- 172.21.15.42:0/2919742752 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x556fd4e849f0 0x556fd5299ff0 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:27.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.912+0000 7fc92ffff640 0 client.16764 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:08:27.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.912+0000 7fc95d4e5740 1 --2- 172.21.15.42:0/2919742752 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556fd4ea5390 0x556fd5277070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:27.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.912+0000 7fc95d4e5740 1 --2- 172.21.15.42:0/2919742752 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556fd4ec6070 0x556fd5293e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:27.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.912+0000 7fc95d4e5740 1 --2- 172.21.15.42:0/2919742752 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556fd4e9d1b0 0x556fd5276b30 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:27.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.912+0000 7fc95d4e5740 1 -- 172.21.15.42:0/2919742752 >> 172.21.15.42:0/2919742752 conn(0x556fd4eba460 msgr2=0x556fd526e490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:08:27.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.912+0000 7fc95d4e5740 1 -- 172.21.15.42:0/2919742752 shutdown_connections 2024-08-30T09:08:27.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.912+0000 7fc95d4e5740 1 -- 172.21.15.42:0/2919742752 wait complete. 2024-08-30T09:08:27.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.912+0000 7fc95d4e5740 20 client.16764 trim_cache size 0 max 16384 2024-08-30T09:08:27.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.938+0000 7fc95d4e5740 1 Processor -- start 2024-08-30T09:08:27.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.938+0000 7fc95d4e5740 1 -- start start 2024-08-30T09:08:27.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.938+0000 7fc95d4e5740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556fd4e849f0 0x556fd5275a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:08:27.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.938+0000 7fc95d4e5740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556fd4e9d1b0 0x556fd5269220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:08:27.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.938+0000 7fc95d4e5740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556fd4ea5390 0x556fd5293d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:08:27.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.938+0000 7fc95d4e5740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fc93c0124a0 con 0x556fd4e849f0 2024-08-30T09:08:27.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.939+0000 7fc95d4e5740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc93c00fd40 con 0x556fd4ea5390 2024-08-30T09:08:28.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.939+0000 7fc95d4e5740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fc93c00db90 con 0x556fd4e9d1b0 2024-08-30T09:08:28.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.939+0000 7fc954d0c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556fd4e849f0 0x556fd5275a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:08:28.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.939+0000 7fc954d0c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556fd4e849f0 0x556fd5275a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:51034/0 (socket says 172.21.15.42:51034) 2024-08-30T09:08:28.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.939+0000 7fc954d0c640 1 -- 172.21.15.42:0/1159457469 learned_addr learned my addr 172.21.15.42:0/1159457469 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:08:28.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.939+0000 7fc94ffff640 1 --2- 172.21.15.42:0/1159457469 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556fd4e9d1b0 0x556fd5269220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:08:28.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.939+0000 7fc95550d640 1 --2- 172.21.15.42:0/1159457469 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556fd4ea5390 0x556fd5293d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:08:28.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.939+0000 7fc954d0c640 1 -- 172.21.15.42:0/1159457469 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556fd4e9d1b0 msgr2=0x556fd5269220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:08:28.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.939+0000 7fc954d0c640 1 --2- 172.21.15.42:0/1159457469 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556fd4e9d1b0 0x556fd5269220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:28.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.939+0000 7fc954d0c640 1 -- 172.21.15.42:0/1159457469 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556fd4ea5390 msgr2=0x556fd5293d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:08:28.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.939+0000 7fc954d0c640 1 --2- 172.21.15.42:0/1159457469 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556fd4ea5390 0x556fd5293d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:28.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.939+0000 7fc954d0c640 1 -- 172.21.15.42:0/1159457469 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556fd5296870 con 0x556fd4e849f0 2024-08-30T09:08:28.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.940+0000 7fc954d0c640 1 --2- 172.21.15.42:0/1159457469 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556fd4e849f0 0x556fd5275a20 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fc94800ecf0 tx=0x7fc94800d4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4d169956d92cc61 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:08:28.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.940+0000 7fc9367fc640 1 -- 172.21.15.42:0/1159457469 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc94802d070 con 0x556fd4e849f0 2024-08-30T09:08:28.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.940+0000 7fc9367fc640 1 -- 172.21.15.42:0/1159457469 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc948003b70 con 0x556fd4e849f0 2024-08-30T09:08:28.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.940+0000 7fc9367fc640 1 -- 172.21.15.42:0/1159457469 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc948004220 con 0x556fd4e849f0 2024-08-30T09:08:28.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.941+0000 7fc95d4e5740 1 -- 172.21.15.42:0/1159457469 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556fd4e849f0 msgr2=0x556fd5275a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:08:28.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.941+0000 7fc95d4e5740 1 --2- 172.21.15.42:0/1159457469 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556fd4e849f0 0x556fd5275a20 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fc94800ecf0 tx=0x7fc94800d4d0 comp rx=0 tx=0).stop 2024-08-30T09:08:28.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.941+0000 7fc95d4e5740 1 -- 172.21.15.42:0/1159457469 shutdown_connections 2024-08-30T09:08:28.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.941+0000 7fc95d4e5740 1 --2- 172.21.15.42:0/1159457469 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556fd4e9d1b0 0x556fd5269220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:28.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.941+0000 7fc95d4e5740 1 --2- 172.21.15.42:0/1159457469 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556fd4ea5390 0x556fd5293d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:28.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.941+0000 7fc95d4e5740 1 --2- 172.21.15.42:0/1159457469 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556fd4e849f0 0x556fd5275a20 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:28.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.941+0000 7fc95d4e5740 1 -- 172.21.15.42:0/1159457469 >> 172.21.15.42:0/1159457469 conn(0x556fd4eba460 msgr2=0x556fd5268b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:08:28.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.941+0000 7fc95d4e5740 1 -- 172.21.15.42:0/1159457469 shutdown_connections 2024-08-30T09:08:28.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.941+0000 7fc95d4e5740 1 -- 172.21.15.42:0/1159457469 wait complete. 2024-08-30T09:08:28.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.942+0000 7fc95d4e5740 1 -- start start 2024-08-30T09:08:28.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.942+0000 7fc95d4e5740 1 Processor -- start 2024-08-30T09:08:28.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.942+0000 7fc95d4e5740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556fd4e849f0 0x556fd52902e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:08:28.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.942+0000 7fc95d4e5740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556fd4e9d1b0 0x556fd528b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:08:28.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.942+0000 7fc95d4e5740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556fd4ea5390 0x556fd528bd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:08:28.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.943+0000 7fc95d4e5740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fc93c00e200 con 0x556fd4e9d1b0 2024-08-30T09:08:28.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.943+0000 7fc95d4e5740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc93c00f670 con 0x556fd4ea5390 2024-08-30T09:08:28.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.943+0000 7fc95d4e5740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fc93c00e8d0 con 0x556fd4e849f0 2024-08-30T09:08:28.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.943+0000 7fc954d0c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556fd4e849f0 0x556fd52902e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:08:28.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.943+0000 7fc954d0c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556fd4e849f0 0x556fd52902e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:32840/0 (socket says 172.21.15.42:32840) 2024-08-30T09:08:28.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.943+0000 7fc954d0c640 1 -- 172.21.15.42:0/1839766936 learned_addr learned my addr 172.21.15.42:0/1839766936 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:08:28.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.943+0000 7fc95550d640 1 --2- 172.21.15.42:0/1839766936 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556fd4ea5390 0x556fd528bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:08:28.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.943+0000 7fc94ffff640 1 --2- 172.21.15.42:0/1839766936 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556fd4e9d1b0 0x556fd528b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:08:28.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.943+0000 7fc95550d640 1 -- 172.21.15.42:0/1839766936 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556fd4e849f0 msgr2=0x556fd52902e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:08:28.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.943+0000 7fc95550d640 1 --2- 172.21.15.42:0/1839766936 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556fd4e849f0 0x556fd52902e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:28.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.943+0000 7fc95550d640 1 -- 172.21.15.42:0/1839766936 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556fd4e9d1b0 msgr2=0x556fd528b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:08:28.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.943+0000 7fc95550d640 1 --2- 172.21.15.42:0/1839766936 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556fd4e9d1b0 0x556fd528b7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:28.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.943+0000 7fc95550d640 1 -- 172.21.15.42:0/1839766936 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556fd528c260 con 0x556fd4ea5390 2024-08-30T09:08:28.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.943+0000 7fc95d4e5740 10 client.0 _ll_register_callbacks cb 0x7fc948011a60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:08:28.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.943+0000 7fc95d4e5740 1 -- 172.21.15.42:0/1839766936 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc948003b70 con 0x556fd4ea5390 2024-08-30T09:08:28.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.944+0000 7fc95550d640 1 --2- 172.21.15.42:0/1839766936 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556fd4ea5390 0x556fd528bd20 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fc94402a910 tx=0x7fc944009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a0bb27b33b6cc4cd server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:08:28.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.944+0000 7fc92ffff640 10 client.26543 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-30T09:08:28.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.944+0000 7fc92ffff640 1 -- 172.21.15.42:0/1839766936 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc944035040 con 0x556fd4ea5390 2024-08-30T09:08:28.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.944+0000 7fc92ffff640 1 -- 172.21.15.42:0/1839766936 <== mon.2 v2:172.21.15.167:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7fc944031dc0 con 0x556fd4ea5390 2024-08-30T09:08:28.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.944+0000 7fc92ffff640 1 -- 172.21.15.42:0/1839766936 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fc94403d450 con 0x556fd4ea5390 2024-08-30T09:08:28.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.944+0000 7fc95d4e5740 10 client.26543 fetch_fsmap learned FSMap version 35 2024-08-30T09:08:28.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.944+0000 7fc95d4e5740 1 -- 172.21.15.42:0/1839766936 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({fsmap=35}) -- 0x556fd5304580 con 0x556fd4ea5390 2024-08-30T09:08:28.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.945+0000 7fc92ffff640 1 -- 172.21.15.42:0/1839766936 <== mon.2 v2:172.21.15.167:3300/0 4 ==== fsmap(e 35) ==== 8362+0+0 (secure 0 0 0) 0x7fc94403d5f0 con 0x556fd4ea5390 2024-08-30T09:08:28.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.945+0000 7fc95d4e5740 10 client.26543 fetch_fsmap finished waiting for FSMap version 35 2024-08-30T09:08:28.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.945+0000 7fc95d4e5740 10 client.26543 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-30T09:08:28.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.945+0000 7fc95d4e5740 20 client.26543 populate_metadata read hostname 'smithi042' 2024-08-30T09:08:28.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.945+0000 7fc95d4e5740 1 --2- 172.21.15.42:0/1839766936 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x556fd4ec6070 0x556fd5306cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:08:28.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.945+0000 7fc95d4e5740 4 client.26543 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-30T09:08:28.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.945+0000 7fc95d4e5740 1 -- 172.21.15.42:0/1839766936 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "scrub status"}) -- 0x556fd51dd760 con 0x556fd4ec6070 2024-08-30T09:08:28.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.945+0000 7fc954d0c640 1 --2- 172.21.15.42:0/1839766936 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x556fd4ec6070 0x556fd5306cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:08:28.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.946+0000 7fc954d0c640 1 --2- 172.21.15.42:0/1839766936 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x556fd4ec6070 0x556fd5306cb0 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=82b7b4e1caabff4f server_cookie=5e991b52313e84fb in_seq=0 out_seq=0 2024-08-30T09:08:28.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.946+0000 7fc92ffff640 10 client.26543 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:08:28.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.946+0000 7fc92ffff640 1 -- 172.21.15.42:0/1839766936 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3126 (crc 0 0 0) 0x556fd51dd760 con 0x556fd4ec6070 2024-08-30T09:08:28.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.946+0000 7fc92ffff640 10 client.26543 handle_command_reply: tid=0 2024-08-30T09:08:28.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.946+0000 7fc95d4e5740 1 client.26543 shutdown 2024-08-30T09:08:28.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.946+0000 7fc95d4e5740 1 -- 172.21.15.42:0/1839766936 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556fd4ea5390 msgr2=0x556fd528bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:08:28.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.946+0000 7fc95d4e5740 1 --2- 172.21.15.42:0/1839766936 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556fd4ea5390 0x556fd528bd20 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fc94402a910 tx=0x7fc944009260 comp rx=0 tx=0).stop 2024-08-30T09:08:28.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.946+0000 7fc95d4e5740 1 -- 172.21.15.42:0/1839766936 shutdown_connections 2024-08-30T09:08:28.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.946+0000 7fc95d4e5740 1 --2- 172.21.15.42:0/1839766936 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x556fd4ec6070 0x556fd5306cb0 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:28.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.946+0000 7fc95d4e5740 1 --2- 172.21.15.42:0/1839766936 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556fd4ea5390 0x556fd528bd20 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:28.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.946+0000 7fc95d4e5740 1 --2- 172.21.15.42:0/1839766936 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556fd4e9d1b0 0x556fd528b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:28.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.946+0000 7fc95d4e5740 1 --2- 172.21.15.42:0/1839766936 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556fd4e849f0 0x556fd52902e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:28.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.946+0000 7fc92ffff640 0 client.26543 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:08:28.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.946+0000 7fc95d4e5740 1 -- 172.21.15.42:0/1839766936 >> 172.21.15.42:0/1839766936 conn(0x556fd4eba460 msgr2=0x556fd526e400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:08:28.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.947+0000 7fc95d4e5740 1 -- 172.21.15.42:0/1839766936 shutdown_connections 2024-08-30T09:08:28.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.947+0000 7fc95d4e5740 1 -- 172.21.15.42:0/1839766936 wait complete. 2024-08-30T09:08:28.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.947+0000 7fc95d4e5740 20 client.26543 trim_cache size 0 max 16384 2024-08-30T09:08:28.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.949+0000 7fc956f70640 1 -- 172.21.15.42:0/2085964462 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fc9380b4c30 msgr2=0x7fc9380b7140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:08:28.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.949+0000 7fc956f70640 1 --2- 172.21.15.42:0/2085964462 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fc9380b4c30 0x7fc9380b7140 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fc944006010 tx=0x7fc944005a40 comp rx=0 tx=0).stop 2024-08-30T09:08:28.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.949+0000 7fc956f70640 1 -- 172.21.15.42:0/2085964462 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc950077120 msgr2=0x7fc9501a0d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:08:28.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.949+0000 7fc956f70640 1 --2- 172.21.15.42:0/2085964462 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc950077120 0x7fc9501a0d60 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fc94000e9c0 tx=0x7fc94000ee80 comp rx=0 tx=0).stop 2024-08-30T09:08:28.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.949+0000 7fc956f70640 1 -- 172.21.15.42:0/2085964462 shutdown_connections 2024-08-30T09:08:28.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.949+0000 7fc956f70640 1 --2- 172.21.15.42:0/2085964462 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fc9380b4c30 0x7fc9380b7140 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:28.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.949+0000 7fc956f70640 1 --2- 172.21.15.42:0/2085964462 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc950077120 0x7fc9501a0d60 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:28.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.949+0000 7fc956f70640 1 --2- 172.21.15.42:0/2085964462 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc950114200 0x7fc9501a6000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:28.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.949+0000 7fc956f70640 1 --2- 172.21.15.42:0/2085964462 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc950075b70 0x7fc9501a07c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:28.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.949+0000 7fc956f70640 1 -- 172.21.15.42:0/2085964462 >> 172.21.15.42:0/2085964462 conn(0x7fc9500ff380 msgr2=0x7fc950110100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:08:28.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.950+0000 7fc956f70640 1 -- 172.21.15.42:0/2085964462 shutdown_connections 2024-08-30T09:08:28.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:08:26.950+0000 7fc956f70640 1 -- 172.21.15.42:0/2085964462 wait complete. 2024-08-30T09:08:28.664 INFO:tasks.cephfs.filesystem:scrub status for tag:99a29ec1-9455-4ae4-854a-398b32896675 - {'path': '/', 'tag': '99a29ec1-9455-4ae4-854a-398b32896675', 'options': 'recursive,force'} 2024-08-30T09:08:29.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:08:28 smithi190 ceph-mon[22001]: pgmap v604: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 4.3 MiB/s wr, 470 op/s 2024-08-30T09:08:29.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:08:28 smithi190 ceph-mon[22001]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-30T09:08:29.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:08:28 smithi167 ceph-mon[22063]: pgmap v604: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 4.3 MiB/s wr, 470 op/s 2024-08-30T09:08:29.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:08:28 smithi167 ceph-mon[22063]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-30T09:08:29.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:08:28 smithi042 ceph-mon[18652]: pgmap v604: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 4.3 MiB/s wr, 470 op/s 2024-08-30T09:08:29.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:08:28 smithi042 ceph-mon[18652]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-30T09:08:30.953 INFO:teuthology.orchestra.run.smithi042.stderr:progress: 465.0 s, 486.8 tps, lat 204.971 ms stddev 250.674 2024-08-30T09:08:31.185 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:08:30 smithi042 ceph-mon[18652]: pgmap v605: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 5.2 MiB/s wr, 627 op/s 2024-08-30T09:08:31.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:08:30 smithi190 ceph-mon[22001]: pgmap v605: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 5.2 MiB/s wr, 627 op/s 2024-08-30T09:08:31.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:08:30 smithi167 ceph-mon[22063]: pgmap v605: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 5.2 MiB/s wr, 627 op/s 2024-08-30T09:08:32.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:08:31 smithi190 ceph-mon[22001]: pgmap v606: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 4.1 MiB/s wr, 516 op/s 2024-08-30T09:08:32.199 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:08:31 smithi042 ceph-mon[18652]: pgmap v606: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 4.1 MiB/s wr, 516 op/s 2024-08-30T09:08:32.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:08:31 smithi167 ceph-mon[22063]: pgmap v606: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 4.1 MiB/s wr, 516 op/s 2024-08-30T09:08:34.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:08:34 smithi167 ceph-mon[22063]: pgmap v607: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 5.0 MiB/s wr, 611 op/s 2024-08-30T09:08:34.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:08:34 smithi042 ceph-mon[18652]: pgmap v607: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 5.0 MiB/s wr, 611 op/s 2024-08-30T09:08:34.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:08:34 smithi190 ceph-mon[22001]: pgmap v607: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 5.0 MiB/s wr, 611 op/s 2024-08-30T09:08:35.228 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.225+0000 7f23cc770640 1 -- 172.21.15.42:0/1135068990 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f23c4103d60 msgr2=0x7f23c406a980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:08:35.229 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.225+0000 7f23cc770640 1 --2- 172.21.15.42:0/1135068990 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f23c4103d60 0x7f23c406a980 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f23b8009850 tx=0x7f23b802a360 comp rx=0 tx=0).stop 2024-08-30T09:08:35.233 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.228+0000 7f23cc770640 1 -- 172.21.15.42:0/1135068990 shutdown_connections 2024-08-30T09:08:35.233 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.228+0000 7f23cc770640 1 --2- 172.21.15.42:0/1135068990 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f23c4103d60 0x7f23c406a980 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:35.233 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.228+0000 7f23cc770640 1 --2- 172.21.15.42:0/1135068990 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f23c41033c0 0x7f23c41037d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:35.233 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.228+0000 7f23cc770640 1 --2- 172.21.15.42:0/1135068990 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f23c4102340 0x7f23c41027b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:35.233 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.228+0000 7f23cc770640 1 -- 172.21.15.42:0/1135068990 >> 172.21.15.42:0/1135068990 conn(0x7f23c40fdc80 msgr2=0x7f23c4100130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:08:35.234 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.231+0000 7f23cc770640 1 -- 172.21.15.42:0/1135068990 shutdown_connections 2024-08-30T09:08:35.234 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.232+0000 7f23cc770640 1 -- 172.21.15.42:0/1135068990 wait complete. 2024-08-30T09:08:35.236 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.233+0000 7f23cc770640 1 Processor -- start 2024-08-30T09:08:35.236 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.233+0000 7f23cc770640 1 -- start start 2024-08-30T09:08:35.236 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.233+0000 7f23cc770640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f23c4102340 0x7f23c410a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:08:35.236 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.233+0000 7f23cc770640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f23c41033c0 0x7f23c4110690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:08:35.236 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.233+0000 7f23cc770640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f23c4103d60 0x7f23c410abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:08:35.236 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.233+0000 7f23cc770640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f23c406d050 con 0x7f23c4102340 2024-08-30T09:08:35.236 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.233+0000 7f23cc770640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f23c406ced0 con 0x7f23c4103d60 2024-08-30T09:08:35.236 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.233+0000 7f23cc770640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f23c406d1d0 con 0x7f23c41033c0 2024-08-30T09:08:35.237 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.233+0000 7f23c9d0b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f23c41033c0 0x7f23c4110690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:08:35.237 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.233+0000 7f23cad0d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f23c4103d60 0x7f23c410abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:08:35.237 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.234+0000 7f23c9d0b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f23c41033c0 0x7f23c4110690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:46642/0 (socket says 172.21.15.42:46642) 2024-08-30T09:08:35.237 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.234+0000 7f23c9d0b640 1 -- 172.21.15.42:0/840375434 learned_addr learned my addr 172.21.15.42:0/840375434 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:08:35.237 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.234+0000 7f23c9d0b640 1 -- 172.21.15.42:0/840375434 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f23c4103d60 msgr2=0x7f23c410abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:08:35.237 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.234+0000 7f23c9d0b640 1 --2- 172.21.15.42:0/840375434 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f23c4103d60 0x7f23c410abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:35.237 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.234+0000 7f23c9d0b640 1 -- 172.21.15.42:0/840375434 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f23c4102340 msgr2=0x7f23c410a460 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-30T09:08:35.237 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.234+0000 7f23c9d0b640 1 --2- 172.21.15.42:0/840375434 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f23c4102340 0x7f23c410a460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:35.237 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.234+0000 7f23c9d0b640 1 -- 172.21.15.42:0/840375434 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23c410b860 con 0x7f23c41033c0 2024-08-30T09:08:35.241 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.235+0000 7f23c9d0b640 1 --2- 172.21.15.42:0/840375434 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f23c41033c0 0x7f23c4110690 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f23c000e9c0 tx=0x7f23c000ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:08:35.242 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.236+0000 7f23b77fe640 1 -- 172.21.15.42:0/840375434 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23c000ccc0 con 0x7f23c41033c0 2024-08-30T09:08:35.242 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.237+0000 7f23b77fe640 1 -- 172.21.15.42:0/840375434 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f23c00054d0 con 0x7f23c41033c0 2024-08-30T09:08:35.242 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.237+0000 7f23b77fe640 1 -- 172.21.15.42:0/840375434 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23c0010430 con 0x7f23c41033c0 2024-08-30T09:08:35.242 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.238+0000 7f23cc770640 1 -- 172.21.15.42:0/840375434 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23c410baf0 con 0x7f23c41033c0 2024-08-30T09:08:35.242 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.238+0000 7f23cc770640 1 -- 172.21.15.42:0/840375434 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23c410bfc0 con 0x7f23c41033c0 2024-08-30T09:08:35.247 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.245+0000 7f23b77fe640 1 -- 172.21.15.42:0/840375434 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7f23c0004f80 con 0x7f23c41033c0 2024-08-30T09:08:35.251 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.249+0000 7f23b77fe640 1 --2- 172.21.15.42:0/840375434 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f23b00b4d30 0x7f23b00b7240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:08:35.252 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.250+0000 7f23b77fe640 1 -- 172.21.15.42:0/840375434 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f23c00e5800 con 0x7f23c41033c0 2024-08-30T09:08:35.252 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.250+0000 7f23d36d9740 1 Processor -- start 2024-08-30T09:08:35.252 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.250+0000 7f23ca50c640 1 --2- 172.21.15.42:0/840375434 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f23b00b4d30 0x7f23b00b7240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:08:35.253 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.251+0000 7f23d36d9740 1 -- start start 2024-08-30T09:08:35.253 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.251+0000 7f23d36d9740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5632efb5f1b0 0x5632eff05180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:08:35.253 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.251+0000 7f23d36d9740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5632efb362f0 0x5632eff057f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:08:35.253 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.251+0000 7f23d36d9740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5632efb3e4d0 0x5632eff06340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:08:35.253 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.251+0000 7f23d36d9740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x5632efe78000 con 0x5632efb5f1b0 2024-08-30T09:08:35.253 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.251+0000 7f23d36d9740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5632efe67590 con 0x5632efb362f0 2024-08-30T09:08:35.253 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.251+0000 7f23d36d9740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5632efe6b540 con 0x5632efb3e4d0 2024-08-30T09:08:35.254 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.251+0000 7f23ca50c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5632efb5f1b0 0x5632eff05180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:08:35.254 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.251+0000 7f23ca50c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5632efb5f1b0 0x5632eff05180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:57334/0 (socket says 172.21.15.42:57334) 2024-08-30T09:08:35.254 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.251+0000 7f23ca50c640 1 -- 172.21.15.42:0/3249740509 learned_addr learned my addr 172.21.15.42:0/3249740509 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:08:35.254 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.252+0000 7f23ca50c640 1 --2- 172.21.15.42:0/840375434 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f23b00b4d30 0x7f23b00b7240 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f23bc00ac20 tx=0x7f23bc00c800 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:08:35.254 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.252+0000 7f23ca50c640 1 -- 172.21.15.42:0/3249740509 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5632efb3e4d0 msgr2=0x5632eff06340 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-30T09:08:35.254 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.252+0000 7f23ca50c640 1 --2- 172.21.15.42:0/3249740509 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5632efb3e4d0 0x5632eff06340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:35.254 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.252+0000 7f23ca50c640 1 -- 172.21.15.42:0/3249740509 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5632efb362f0 msgr2=0x5632eff057f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:08:35.254 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.252+0000 7f23ca50c640 1 --2- 172.21.15.42:0/3249740509 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5632efb362f0 0x5632eff057f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:35.254 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.252+0000 7f23ca50c640 1 -- 172.21.15.42:0/3249740509 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5632efbca790 con 0x5632efb5f1b0 2024-08-30T09:08:35.257 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.253+0000 7f23ca50c640 1 --2- 172.21.15.42:0/3249740509 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5632efb5f1b0 0x5632eff05180 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f23bc0186b0 tx=0x7f23bc017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6124a801d6950e1 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:08:35.257 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.254+0000 7f23b4ff9640 1 -- 172.21.15.42:0/3249740509 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23bc012040 con 0x5632efb5f1b0 2024-08-30T09:08:35.258 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.254+0000 7f23b4ff9640 1 -- 172.21.15.42:0/3249740509 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f23bc014050 con 0x5632efb5f1b0 2024-08-30T09:08:35.258 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.254+0000 7f23b4ff9640 1 -- 172.21.15.42:0/3249740509 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23bc017900 con 0x5632efb5f1b0 2024-08-30T09:08:35.258 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.255+0000 7f23d36d9740 1 -- 172.21.15.42:0/3249740509 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5632efb5f1b0 msgr2=0x5632eff05180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:08:35.258 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.255+0000 7f23d36d9740 1 --2- 172.21.15.42:0/3249740509 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5632efb5f1b0 0x5632eff05180 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f23bc0186b0 tx=0x7f23bc017000 comp rx=0 tx=0).stop 2024-08-30T09:08:35.258 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.255+0000 7f23d36d9740 1 -- 172.21.15.42:0/3249740509 shutdown_connections 2024-08-30T09:08:35.258 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.255+0000 7f23d36d9740 1 --2- 172.21.15.42:0/3249740509 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5632efb3e4d0 0x5632eff06340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:35.258 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.255+0000 7f23d36d9740 1 --2- 172.21.15.42:0/3249740509 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5632efb362f0 0x5632eff057f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:35.258 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.255+0000 7f23d36d9740 1 --2- 172.21.15.42:0/3249740509 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5632efb5f1b0 0x5632eff05180 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:35.258 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.255+0000 7f23d36d9740 1 -- 172.21.15.42:0/3249740509 >> 172.21.15.42:0/3249740509 conn(0x5632efb53300 msgr2=0x5632eff00c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:08:35.265 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.263+0000 7f23d36d9740 1 -- 172.21.15.42:0/3249740509 shutdown_connections 2024-08-30T09:08:35.265 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.263+0000 7f23d36d9740 1 -- 172.21.15.42:0/3249740509 wait complete. 2024-08-30T09:08:35.265 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.263+0000 7f23d36d9740 1 -- start start 2024-08-30T09:08:35.265 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.263+0000 7f23d36d9740 1 Processor -- start 2024-08-30T09:08:35.266 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.264+0000 7f23d36d9740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5632efb362f0 0x5632eff0ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:08:35.266 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.264+0000 7f23d36d9740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5632efb3e4d0 0x5632eff0f380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:08:35.267 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.264+0000 7f23d36d9740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5632efb5f1b0 0x5632eff0fef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:08:35.267 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.264+0000 7f23d36d9740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x5632efda0ab0 con 0x5632efb5f1b0 2024-08-30T09:08:35.267 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.264+0000 7f23d36d9740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5632efdee680 con 0x5632efb3e4d0 2024-08-30T09:08:35.267 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.265+0000 7f23d36d9740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5632eff06e60 con 0x5632efb362f0 2024-08-30T09:08:35.267 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.265+0000 7f23d36d9740 10 client.0 _ll_register_callbacks cb 0x5632efb76a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:08:35.273 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.266+0000 7f23ca50c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5632efb362f0 0x5632eff0ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:08:35.273 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.266+0000 7f23ca50c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5632efb362f0 0x5632eff0ecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:46658/0 (socket says 172.21.15.42:46658) 2024-08-30T09:08:35.273 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.266+0000 7f23ca50c640 1 -- 172.21.15.42:0/2332983953 learned_addr learned my addr 172.21.15.42:0/2332983953 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:08:35.273 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.266+0000 7f23c9d0b640 1 --2- 172.21.15.42:0/2332983953 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5632efb3e4d0 0x5632eff0f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:08:35.273 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.266+0000 7f23ca50c640 1 -- 172.21.15.42:0/2332983953 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5632efb3e4d0 msgr2=0x5632eff0f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:08:35.273 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.266+0000 7f23ca50c640 1 --2- 172.21.15.42:0/2332983953 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5632efb3e4d0 0x5632eff0f380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:35.273 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.266+0000 7f23ca50c640 1 -- 172.21.15.42:0/2332983953 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5632efb5f1b0 msgr2=0x5632eff0fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:08:35.273 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.266+0000 7f23ca50c640 1 --2- 172.21.15.42:0/2332983953 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5632efb5f1b0 0x5632eff0fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:35.273 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.266+0000 7f23ca50c640 1 -- 172.21.15.42:0/2332983953 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5632efc43620 con 0x5632efb362f0 2024-08-30T09:08:35.273 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.269+0000 7f23ca50c640 1 --2- 172.21.15.42:0/2332983953 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5632efb362f0 0x5632eff0ecc0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f23bc018680 tx=0x7f23bc021670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6012f4e978cfb124 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:08:35.273 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.271+0000 7f23adffb640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-08-30T09:08:35.274 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.271+0000 7f23adffb640 1 -- 172.21.15.42:0/2332983953 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23bc012040 con 0x5632efb362f0 2024-08-30T09:08:35.274 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.271+0000 7f23adffb640 1 -- 172.21.15.42:0/2332983953 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f23bc017960 con 0x5632efb362f0 2024-08-30T09:08:35.274 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.271+0000 7f23d36d9740 1 -- 172.21.15.42:0/2332983953 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f23bc014050 con 0x5632efb362f0 2024-08-30T09:08:35.281 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.276+0000 7f23adffb640 1 -- 172.21.15.42:0/2332983953 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f23bc021860 con 0x5632efb362f0 2024-08-30T09:08:35.281 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.279+0000 7f23d36d9740 10 client.26533 fetch_fsmap learned FSMap version 36 2024-08-30T09:08:35.281 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.279+0000 7f23d36d9740 1 -- 172.21.15.42:0/2332983953 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5632eff18e40 con 0x5632efb362f0 2024-08-30T09:08:35.286 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.284+0000 7f23adffb640 1 -- 172.21.15.42:0/2332983953 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f23bc014050 con 0x5632efb362f0 2024-08-30T09:08:35.286 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.284+0000 7f23adffb640 1 -- 172.21.15.42:0/2332983953 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 36) ==== 8366+0+0 (secure 0 0 0) 0x7f23bc033030 con 0x5632efb362f0 2024-08-30T09:08:35.286 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.284+0000 7f23d36d9740 10 client.26533 fetch_fsmap finished waiting for FSMap version 36 2024-08-30T09:08:35.286 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.284+0000 7f23d36d9740 10 client.26533 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-30T09:08:35.286 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.284+0000 7f23d36d9740 20 client.26533 populate_metadata read hostname 'smithi042' 2024-08-30T09:08:35.287 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.284+0000 7f23d36d9740 1 --2- 172.21.15.42:0/2332983953 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x5632efb1dd80 0x5632eff1c330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:08:35.287 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.284+0000 7f23d36d9740 4 client.26533 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-30T09:08:35.287 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.284+0000 7f23d36d9740 1 -- 172.21.15.42:0/2332983953 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5632efe79f50 con 0x5632efb1dd80 2024-08-30T09:08:35.287 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.286+0000 7f23c9d0b640 1 --2- 172.21.15.42:0/2332983953 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x5632efb1dd80 0x5632eff1c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:08:35.293 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.291+0000 7f23c9d0b640 1 --2- 172.21.15.42:0/2332983953 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x5632efb1dd80 0x5632eff1c330 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a795db9aa0b44340 server_cookie=cf0703bceef4307b in_seq=0 out_seq=0 2024-08-30T09:08:35.293 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.292+0000 7f23adffb640 10 client.26533 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:08:35.297 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.295+0000 7f23adffb640 1 -- 172.21.15.42:0/2332983953 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5632efe79f50 con 0x5632efb1dd80 2024-08-30T09:08:35.298 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.295+0000 7f23adffb640 10 client.26533 handle_command_reply: tid=0 2024-08-30T09:08:35.298 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.296+0000 7f23d36d9740 1 client.26533 shutdown 2024-08-30T09:08:35.301 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.298+0000 7f23d36d9740 1 -- 172.21.15.42:0/2332983953 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5632efb362f0 msgr2=0x5632eff0ecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:08:35.301 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.298+0000 7f23d36d9740 1 --2- 172.21.15.42:0/2332983953 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5632efb362f0 0x5632eff0ecc0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f23bc018680 tx=0x7f23bc021670 comp rx=0 tx=0).stop 2024-08-30T09:08:35.301 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.299+0000 7f23d36d9740 1 -- 172.21.15.42:0/2332983953 shutdown_connections 2024-08-30T09:08:35.301 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.299+0000 7f23d36d9740 1 --2- 172.21.15.42:0/2332983953 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x5632efb1dd80 0x5632eff1c330 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:35.301 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.299+0000 7f23d36d9740 1 --2- 172.21.15.42:0/2332983953 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5632efb5f1b0 0x5632eff0fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:35.301 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.299+0000 7f23d36d9740 1 --2- 172.21.15.42:0/2332983953 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5632efb3e4d0 0x5632eff0f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:35.301 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.299+0000 7f23d36d9740 1 --2- 172.21.15.42:0/2332983953 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5632efb362f0 0x5632eff0ecc0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:35.301 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.299+0000 7f23d36d9740 1 -- 172.21.15.42:0/2332983953 >> 172.21.15.42:0/2332983953 conn(0x5632efb53300 msgr2=0x5632efeff170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:08:35.301 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.299+0000 7f23adffb640 0 client.26533 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:08:35.305 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.303+0000 7f23d36d9740 1 -- 172.21.15.42:0/2332983953 shutdown_connections 2024-08-30T09:08:35.305 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.303+0000 7f23d36d9740 1 -- 172.21.15.42:0/2332983953 wait complete. 2024-08-30T09:08:35.305 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.303+0000 7f23d36d9740 20 client.26533 trim_cache size 0 max 16384 2024-08-30T09:08:35.326 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.322+0000 7f23d36d9740 1 Processor -- start 2024-08-30T09:08:35.326 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.323+0000 7f23d36d9740 1 -- start start 2024-08-30T09:08:35.326 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.323+0000 7f23d36d9740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5632efb362f0 0x5632eff1c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:08:35.326 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.323+0000 7f23d36d9740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5632efb3e4d0 0x5632efeff170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:08:35.326 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.323+0000 7f23d36d9740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5632efb5f1b0 0x5632eff0dee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:08:35.326 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.323+0000 7f23d36d9740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f23c41c6560 con 0x5632efb362f0 2024-08-30T09:08:35.326 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.323+0000 7f23ca50c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5632efb362f0 0x5632eff1c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:08:35.327 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.323+0000 7f23ca50c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5632efb362f0 0x5632eff1c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:57350/0 (socket says 172.21.15.42:57350) 2024-08-30T09:08:35.327 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.323+0000 7f23ca50c640 1 -- 172.21.15.42:0/2085628846 learned_addr learned my addr 172.21.15.42:0/2085628846 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:08:35.327 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.323+0000 7f23d36d9740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f23c41c3e00 con 0x5632efb3e4d0 2024-08-30T09:08:35.327 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.323+0000 7f23d36d9740 1 -- 172.21.15.42:0/2085628846 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f23c41af110 con 0x5632efb5f1b0 2024-08-30T09:08:35.327 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.323+0000 7f23c9d0b640 1 --2- 172.21.15.42:0/2085628846 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5632efb3e4d0 0x5632efeff170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:08:35.327 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.323+0000 7f23cad0d640 1 --2- 172.21.15.42:0/2085628846 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5632efb5f1b0 0x5632eff0dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:08:35.327 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.324+0000 7f23ca50c640 1 -- 172.21.15.42:0/2085628846 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5632efb5f1b0 msgr2=0x5632eff0dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:08:35.327 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.324+0000 7f23ca50c640 1 --2- 172.21.15.42:0/2085628846 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5632efb5f1b0 0x5632eff0dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:35.327 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.324+0000 7f23ca50c640 1 -- 172.21.15.42:0/2085628846 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5632efb3e4d0 msgr2=0x5632efeff170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:08:35.327 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.324+0000 7f23ca50c640 1 --2- 172.21.15.42:0/2085628846 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5632efb3e4d0 0x5632efeff170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:35.327 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.324+0000 7f23ca50c640 1 -- 172.21.15.42:0/2085628846 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5632efb1dfb0 con 0x5632efb362f0 2024-08-30T09:08:35.327 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.324+0000 7f23ca50c640 1 --2- 172.21.15.42:0/2085628846 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5632efb362f0 0x5632eff1c2c0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f23bc0186b0 tx=0x7f23bc00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b24d6f1caa4077d2 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:08:35.328 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.324+0000 7f23affff640 1 -- 172.21.15.42:0/2085628846 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23bc010070 con 0x5632efb362f0 2024-08-30T09:08:35.328 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.324+0000 7f23affff640 1 -- 172.21.15.42:0/2085628846 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f23bc00ce50 con 0x5632efb362f0 2024-08-30T09:08:35.328 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.324+0000 7f23affff640 1 -- 172.21.15.42:0/2085628846 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23bc012070 con 0x5632efb362f0 2024-08-30T09:08:35.332 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.325+0000 7f23d36d9740 1 -- 172.21.15.42:0/2085628846 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5632efb362f0 msgr2=0x5632eff1c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:08:35.332 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.325+0000 7f23d36d9740 1 --2- 172.21.15.42:0/2085628846 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5632efb362f0 0x5632eff1c2c0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f23bc0186b0 tx=0x7f23bc00cde0 comp rx=0 tx=0).stop 2024-08-30T09:08:35.332 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.325+0000 7f23d36d9740 1 -- 172.21.15.42:0/2085628846 shutdown_connections 2024-08-30T09:08:35.332 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.325+0000 7f23d36d9740 1 --2- 172.21.15.42:0/2085628846 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5632efb5f1b0 0x5632eff0dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:35.332 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.325+0000 7f23d36d9740 1 --2- 172.21.15.42:0/2085628846 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5632efb3e4d0 0x5632efeff170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:35.332 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.325+0000 7f23d36d9740 1 --2- 172.21.15.42:0/2085628846 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5632efb362f0 0x5632eff1c2c0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:35.332 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.325+0000 7f23d36d9740 1 -- 172.21.15.42:0/2085628846 >> 172.21.15.42:0/2085628846 conn(0x5632efb53300 msgr2=0x5632eff10520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:08:35.332 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.325+0000 7f23d36d9740 1 -- 172.21.15.42:0/2085628846 shutdown_connections 2024-08-30T09:08:35.333 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.325+0000 7f23d36d9740 1 -- 172.21.15.42:0/2085628846 wait complete. 2024-08-30T09:08:35.333 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.326+0000 7f23d36d9740 1 -- start start 2024-08-30T09:08:35.333 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.326+0000 7f23d36d9740 1 Processor -- start 2024-08-30T09:08:35.333 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.326+0000 7f23d36d9740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5632efb362f0 0x5632eff13790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:08:35.333 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.326+0000 7f23d36d9740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5632efb3e4d0 0x5632eff13e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:08:35.333 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.326+0000 7f23d36d9740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5632efb5f1b0 0x5632eff149c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:08:35.333 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.326+0000 7f23d36d9740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f23c41c22c0 con 0x5632efb5f1b0 2024-08-30T09:08:35.333 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.326+0000 7f23d36d9740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f23c41c3730 con 0x5632efb362f0 2024-08-30T09:08:35.333 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.326+0000 7f23d36d9740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f23c41c2990 con 0x5632efb3e4d0 2024-08-30T09:08:35.333 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.326+0000 7f23ca50c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5632efb362f0 0x5632eff13790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:08:35.333 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.326+0000 7f23ca50c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5632efb362f0 0x5632eff13790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:40424/0 (socket says 172.21.15.42:40424) 2024-08-30T09:08:35.333 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.326+0000 7f23ca50c640 1 -- 172.21.15.42:0/1347445928 learned_addr learned my addr 172.21.15.42:0/1347445928 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:08:35.334 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.326+0000 7f23cad0d640 1 --2- 172.21.15.42:0/1347445928 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5632efb5f1b0 0x5632eff149c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:08:35.334 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.327+0000 7f23c9d0b640 1 --2- 172.21.15.42:0/1347445928 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5632efb3e4d0 0x5632eff13e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:08:35.334 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.327+0000 7f23d36d9740 10 client.0 _ll_register_callbacks cb 0x7f23bc0185d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:08:35.334 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.327+0000 7f23cad0d640 1 -- 172.21.15.42:0/1347445928 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5632efb3e4d0 msgr2=0x5632eff13e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:08:35.334 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.327+0000 7f23cad0d640 1 --2- 172.21.15.42:0/1347445928 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5632efb3e4d0 0x5632eff13e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:35.334 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.327+0000 7f23cad0d640 1 -- 172.21.15.42:0/1347445928 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5632efb362f0 msgr2=0x5632eff13790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:08:35.334 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.327+0000 7f23cad0d640 1 --2- 172.21.15.42:0/1347445928 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5632efb362f0 0x5632eff13790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:35.334 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.327+0000 7f23cad0d640 1 -- 172.21.15.42:0/1347445928 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5632eff15120 con 0x5632efb5f1b0 2024-08-30T09:08:35.334 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.327+0000 7f23ca50c640 1 --2- 172.21.15.42:0/1347445928 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5632efb362f0 0x5632eff13790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-30T09:08:35.334 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.327+0000 7f23cad0d640 1 --2- 172.21.15.42:0/1347445928 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5632efb5f1b0 0x5632eff149c0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f23b8030460 tx=0x7f23b802ae00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a6cb3a5635a4d28 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:08:35.334 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.327+0000 7f23adffb640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-08-30T09:08:35.334 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.327+0000 7f23adffb640 1 -- 172.21.15.42:0/1347445928 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23b8037040 con 0x5632efb5f1b0 2024-08-30T09:08:35.334 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.327+0000 7f23adffb640 1 -- 172.21.15.42:0/1347445928 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f23b8004040 con 0x5632efb5f1b0 2024-08-30T09:08:35.334 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.327+0000 7f23d36d9740 1 -- 172.21.15.42:0/1347445928 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f23bc00ce50 con 0x5632efb5f1b0 2024-08-30T09:08:35.335 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.328+0000 7f23adffb640 1 -- 172.21.15.42:0/1347445928 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f23b8004260 con 0x5632efb5f1b0 2024-08-30T09:08:35.335 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.328+0000 7f23d36d9740 10 client.16779 fetch_fsmap learned FSMap version 36 2024-08-30T09:08:35.335 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.328+0000 7f23d36d9740 1 -- 172.21.15.42:0/1347445928 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5632eff16440 con 0x5632efb5f1b0 2024-08-30T09:08:35.335 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.328+0000 7f23adffb640 1 -- 172.21.15.42:0/1347445928 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f23bc00ce50 con 0x5632efb5f1b0 2024-08-30T09:08:35.335 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.328+0000 7f23adffb640 1 -- 172.21.15.42:0/1347445928 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 36) ==== 8366+0+0 (secure 0 0 0) 0x7f23b8004810 con 0x5632efb5f1b0 2024-08-30T09:08:35.335 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.328+0000 7f23d36d9740 10 client.16779 fetch_fsmap finished waiting for FSMap version 36 2024-08-30T09:08:35.335 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.328+0000 7f23d36d9740 10 client.16779 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-30T09:08:35.335 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.328+0000 7f23d36d9740 20 client.16779 populate_metadata read hostname 'smithi042' 2024-08-30T09:08:35.335 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.328+0000 7f23d36d9740 1 --2- 172.21.15.42:0/1347445928 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x5632efaf31c0 0x5632eff88d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:08:35.335 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.328+0000 7f23d36d9740 4 client.16779 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-30T09:08:35.335 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.328+0000 7f23d36d9740 1 -- 172.21.15.42:0/1347445928 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5632efe759f0 con 0x5632efaf31c0 2024-08-30T09:08:35.335 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.329+0000 7f23ca50c640 1 --2- 172.21.15.42:0/1347445928 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x5632efaf31c0 0x5632eff88d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:08:35.335 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.329+0000 7f23ca50c640 1 --2- 172.21.15.42:0/1347445928 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x5632efaf31c0 0x5632eff88d50 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ccedfb10844766f7 server_cookie=bf94ddd68e8ff6db in_seq=0 out_seq=0 2024-08-30T09:08:35.336 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.329+0000 7f23adffb640 10 client.16779 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:08:35.382 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.379+0000 7f23adffb640 1 -- 172.21.15.42:0/1347445928 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5632efe759f0 con 0x5632efaf31c0 2024-08-30T09:08:35.382 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.379+0000 7f23adffb640 10 client.16779 handle_command_reply: tid=0 2024-08-30T09:08:35.382 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.379+0000 7f23d36d9740 1 client.16779 shutdown 2024-08-30T09:08:35.382 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.379+0000 7f23d36d9740 1 -- 172.21.15.42:0/1347445928 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5632efb5f1b0 msgr2=0x5632eff149c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:08:35.382 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.379+0000 7f23d36d9740 1 --2- 172.21.15.42:0/1347445928 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5632efb5f1b0 0x5632eff149c0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f23b8030460 tx=0x7f23b802ae00 comp rx=0 tx=0).stop 2024-08-30T09:08:35.382 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.379+0000 7f23d36d9740 1 -- 172.21.15.42:0/1347445928 shutdown_connections 2024-08-30T09:08:35.382 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.379+0000 7f23d36d9740 1 --2- 172.21.15.42:0/1347445928 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x5632efaf31c0 0x5632eff88d50 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:35.382 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.379+0000 7f23adffb640 0 client.16779 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:08:35.382 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.379+0000 7f23d36d9740 1 --2- 172.21.15.42:0/1347445928 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5632efb3e4d0 0x5632eff13e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:35.382 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.379+0000 7f23d36d9740 1 --2- 172.21.15.42:0/1347445928 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5632efb5f1b0 0x5632eff149c0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:35.382 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.379+0000 7f23d36d9740 1 --2- 172.21.15.42:0/1347445928 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5632efb362f0 0x5632eff13790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:35.383 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.379+0000 7f23d36d9740 1 -- 172.21.15.42:0/1347445928 >> 172.21.15.42:0/1347445928 conn(0x5632efb53300 msgr2=0x5632eff0dee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:08:35.383 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.379+0000 7f23d36d9740 1 -- 172.21.15.42:0/1347445928 shutdown_connections 2024-08-30T09:08:35.383 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.379+0000 7f23d36d9740 1 -- 172.21.15.42:0/1347445928 wait complete. 2024-08-30T09:08:35.383 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.380+0000 7f23d36d9740 20 client.16779 trim_cache size 0 max 16384 2024-08-30T09:08:35.383 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-08-30T09:08:35.383 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2024-08-30T09:08:35.383 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-08-30T09:08:35.383 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-08-30T09:08:35.384 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.380+0000 7f23cc770640 1 -- 172.21.15.42:0/840375434 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f23b00b4d30 msgr2=0x7f23b00b7240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:08:35.384 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.380+0000 7f23cc770640 1 --2- 172.21.15.42:0/840375434 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f23b00b4d30 0x7f23b00b7240 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f23bc00ac20 tx=0x7f23bc00c800 comp rx=0 tx=0).stop 2024-08-30T09:08:35.384 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.380+0000 7f23cc770640 1 -- 172.21.15.42:0/840375434 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f23c41033c0 msgr2=0x7f23c4110690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:08:35.384 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.380+0000 7f23cc770640 1 --2- 172.21.15.42:0/840375434 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f23c41033c0 0x7f23c4110690 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f23c000e9c0 tx=0x7f23c000ee80 comp rx=0 tx=0).stop 2024-08-30T09:08:35.384 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.380+0000 7f23cc770640 1 -- 172.21.15.42:0/840375434 shutdown_connections 2024-08-30T09:08:35.385 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.380+0000 7f23cc770640 1 --2- 172.21.15.42:0/840375434 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f23b00b4d30 0x7f23b00b7240 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:35.385 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.380+0000 7f23cc770640 1 --2- 172.21.15.42:0/840375434 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f23c41033c0 0x7f23c4110690 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:35.385 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.380+0000 7f23cc770640 1 --2- 172.21.15.42:0/840375434 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f23c4103d60 0x7f23c410abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:35.385 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.380+0000 7f23cc770640 1 --2- 172.21.15.42:0/840375434 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f23c4102340 0x7f23c410a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:35.385 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.380+0000 7f23cc770640 1 -- 172.21.15.42:0/840375434 >> 172.21.15.42:0/840375434 conn(0x7f23c40fdc80 msgr2=0x7f23c4100100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:08:35.385 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.381+0000 7f23cc770640 1 -- 172.21.15.42:0/840375434 shutdown_connections 2024-08-30T09:08:35.385 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:35.381+0000 7f23cc770640 1 -- 172.21.15.42:0/840375434 wait complete. 2024-08-30T09:08:35.953 INFO:teuthology.orchestra.run.smithi042.stderr:progress: 470.0 s, 510.0 tps, lat 199.747 ms stddev 264.854 2024-08-30T09:08:36.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:08:36 smithi167 ceph-mon[22063]: pgmap v608: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 4.2 MiB/s wr, 524 op/s 2024-08-30T09:08:36.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:08:36 smithi042 ceph-mon[18652]: pgmap v608: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 4.2 MiB/s wr, 524 op/s 2024-08-30T09:08:36.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:08:36 smithi190 ceph-mon[22001]: pgmap v608: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 4.2 MiB/s wr, 524 op/s 2024-08-30T09:08:38.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:08:38 smithi167 ceph-mon[22063]: pgmap v609: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 4.1 MiB/s wr, 526 op/s 2024-08-30T09:08:38.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:08:38 smithi167 ceph-mon[22063]: scrub summary: idle 2024-08-30T09:08:38.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:08:38 smithi167 ceph-mon[22063]: scrub summary: idle 2024-08-30T09:08:38.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:08:38 smithi042 ceph-mon[18652]: pgmap v609: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 4.1 MiB/s wr, 526 op/s 2024-08-30T09:08:38.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:08:38 smithi042 ceph-mon[18652]: scrub summary: idle 2024-08-30T09:08:38.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:08:38 smithi042 ceph-mon[18652]: scrub summary: idle 2024-08-30T09:08:38.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:08:38 smithi190 ceph-mon[22001]: pgmap v609: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 4.1 MiB/s wr, 526 op/s 2024-08-30T09:08:38.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:08:38 smithi190 ceph-mon[22001]: scrub summary: idle 2024-08-30T09:08:38.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:08:38 smithi190 ceph-mon[22001]: scrub summary: idle 2024-08-30T09:08:39.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:08:39 smithi167 ceph-mon[22063]: scrub summary: idle 2024-08-30T09:08:39.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:08:39 smithi042 ceph-mon[18652]: scrub summary: idle 2024-08-30T09:08:39.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:08:39 smithi190 ceph-mon[22001]: scrub summary: idle 2024-08-30T09:08:40.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:08:40 smithi167 ceph-mon[22063]: pgmap v610: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 5.0 MiB/s wr, 588 op/s 2024-08-30T09:08:40.587 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:08:40 smithi190 ceph-mon[22001]: pgmap v610: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 5.0 MiB/s wr, 588 op/s 2024-08-30T09:08:40.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:08:40 smithi042 ceph-mon[18652]: pgmap v610: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 5.0 MiB/s wr, 588 op/s 2024-08-30T09:08:40.953 INFO:teuthology.orchestra.run.smithi042.stderr:progress: 475.0 s, 844.4 tps, lat 119.414 ms stddev 146.573 2024-08-30T09:08:42.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:08:42 smithi167 ceph-mon[22063]: pgmap v611: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 4.4 MiB/s wr, 470 op/s 2024-08-30T09:08:42.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:08:42 smithi042 ceph-mon[18652]: pgmap v611: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 4.4 MiB/s wr, 470 op/s 2024-08-30T09:08:42.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:08:42 smithi190 ceph-mon[22001]: pgmap v611: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 4.4 MiB/s wr, 470 op/s 2024-08-30T09:08:44.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:08:44 smithi167 ceph-mon[22063]: pgmap v612: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 5.5 MiB/s wr, 569 op/s 2024-08-30T09:08:44.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:08:44 smithi042 ceph-mon[18652]: pgmap v612: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 5.5 MiB/s wr, 569 op/s 2024-08-30T09:08:44.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:08:44 smithi190 ceph-mon[22001]: pgmap v612: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 5.5 MiB/s wr, 569 op/s 2024-08-30T09:08:45.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:08:45 smithi167 ceph-mon[22063]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-30T09:08:45.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:08:45 smithi042 ceph-mon[18652]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-30T09:08:45.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:08:45 smithi190 ceph-mon[22001]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-30T09:08:45.953 INFO:teuthology.orchestra.run.smithi042.stderr:progress: 480.0 s, 567.4 tps, lat 174.448 ms stddev 211.401 2024-08-30T09:08:46.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:08:46 smithi167 ceph-mon[22063]: pgmap v613: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 4.6 MiB/s wr, 483 op/s 2024-08-30T09:08:46.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:08:46 smithi042 ceph-mon[18652]: pgmap v613: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 4.6 MiB/s wr, 483 op/s 2024-08-30T09:08:46.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:08:46 smithi190 ceph-mon[22001]: pgmap v613: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 4.6 MiB/s wr, 483 op/s 2024-08-30T09:08:48.535 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.533+0000 7fa4152d3640 1 -- 172.21.15.42:0/1637254947 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa400040770 con 0x7fa4101033c0 2024-08-30T09:08:48.537 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.534+0000 7fa418539640 1 -- 172.21.15.42:0/1637254947 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa4101033c0 msgr2=0x7fa4101037d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:08:48.537 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.534+0000 7fa418539640 1 --2- 172.21.15.42:0/1637254947 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa4101033c0 0x7fa4101037d0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fa40001a910 tx=0x7fa40003b470 comp rx=0 tx=0).stop 2024-08-30T09:08:48.537 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.534+0000 7fa418539640 1 -- 172.21.15.42:0/1637254947 shutdown_connections 2024-08-30T09:08:48.537 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.534+0000 7fa418539640 1 --2- 172.21.15.42:0/1637254947 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa410103d60 0x7fa41010a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:48.537 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.534+0000 7fa418539640 1 --2- 172.21.15.42:0/1637254947 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa4101033c0 0x7fa4101037d0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:48.538 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.534+0000 7fa418539640 1 --2- 172.21.15.42:0/1637254947 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa410102340 0x7fa4101027b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:48.538 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.535+0000 7fa418539640 1 -- 172.21.15.42:0/1637254947 >> 172.21.15.42:0/1637254947 conn(0x7fa4100fdc80 msgr2=0x7fa410100130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:08:48.546 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.545+0000 7fa418539640 1 -- 172.21.15.42:0/1637254947 shutdown_connections 2024-08-30T09:08:48.549 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.546+0000 7fa418539640 1 -- 172.21.15.42:0/1637254947 wait complete. 2024-08-30T09:08:48.549 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.547+0000 7fa418539640 1 Processor -- start 2024-08-30T09:08:48.555 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.551+0000 7fa418539640 1 -- start start 2024-08-30T09:08:48.555 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.551+0000 7fa418539640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa410102340 0x7fa410109eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:08:48.555 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.551+0000 7fa418539640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa4101033c0 0x7fa41010a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:08:48.555 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.551+0000 7fa418539640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa410103d60 0x7fa41010b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:08:48.555 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.551+0000 7fa418539640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fa410072a90 con 0x7fa410102340 2024-08-30T09:08:48.555 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.551+0000 7fa418539640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa410072910 con 0x7fa4101033c0 2024-08-30T09:08:48.555 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.551+0000 7fa418539640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fa410072c10 con 0x7fa410103d60 2024-08-30T09:08:48.555 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.552+0000 7fa415ad4640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa4101033c0 0x7fa41010a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:08:48.555 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.552+0000 7fa415ad4640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa4101033c0 0x7fa41010a520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:58596/0 (socket says 172.21.15.42:58596) 2024-08-30T09:08:48.556 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.552+0000 7fa415ad4640 1 -- 172.21.15.42:0/878894658 learned_addr learned my addr 172.21.15.42:0/878894658 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:08:48.556 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.552+0000 7fa415ad4640 1 -- 172.21.15.42:0/878894658 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa410103d60 msgr2=0x7fa41010b990 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-30T09:08:48.556 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.552+0000 7fa415ad4640 1 --2- 172.21.15.42:0/878894658 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa410103d60 0x7fa41010b990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:48.556 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.552+0000 7fa415ad4640 1 -- 172.21.15.42:0/878894658 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa410102340 msgr2=0x7fa410109eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:08:48.556 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.552+0000 7fa415ad4640 1 --2- 172.21.15.42:0/878894658 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa410102340 0x7fa410109eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:48.556 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.552+0000 7fa415ad4640 1 -- 172.21.15.42:0/878894658 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa41010c1b0 con 0x7fa4101033c0 2024-08-30T09:08:48.559 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.557+0000 7fa415ad4640 1 --2- 172.21.15.42:0/878894658 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa4101033c0 0x7fa41010a520 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fa400000c00 tx=0x7fa400004b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:08:48.565 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.563+0000 7fa4077fe640 1 -- 172.21.15.42:0/878894658 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa40004f070 con 0x7fa4101033c0 2024-08-30T09:08:48.566 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.563+0000 7fa4077fe640 1 -- 172.21.15.42:0/878894658 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa400040c50 con 0x7fa4101033c0 2024-08-30T09:08:48.566 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.563+0000 7fa4077fe640 1 -- 172.21.15.42:0/878894658 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa400046030 con 0x7fa4101033c0 2024-08-30T09:08:48.567 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.565+0000 7fa418539640 1 -- 172.21.15.42:0/878894658 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa41010c440 con 0x7fa4101033c0 2024-08-30T09:08:48.568 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.565+0000 7fa418539640 1 -- 172.21.15.42:0/878894658 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa410112a10 con 0x7fa4101033c0 2024-08-30T09:08:48.572 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.569+0000 7fa41f4a8740 1 Processor -- start 2024-08-30T09:08:48.576 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.574+0000 7fa4077fe640 1 -- 172.21.15.42:0/878894658 <== mon.2 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7fa400052070 con 0x7fa4101033c0 2024-08-30T09:08:48.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:08:48 smithi167 ceph-mon[22063]: pgmap v614: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 4.6 MiB/s wr, 487 op/s 2024-08-30T09:08:48.583 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.581+0000 7fa41f4a8740 1 -- start start 2024-08-30T09:08:48.583 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.581+0000 7fa41f4a8740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d64cf8c1b0 0x55d64d332180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:08:48.584 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.582+0000 7fa41f4a8740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d64cf632f0 0x55d64d3327f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:08:48.585 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.582+0000 7fa41f4a8740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d64cf6b4d0 0x55d64d333340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:08:48.585 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.582+0000 7fa41f4a8740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55d64d2a5000 con 0x55d64cf6b4d0 2024-08-30T09:08:48.585 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.582+0000 7fa41f4a8740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55d64d294590 con 0x55d64cf632f0 2024-08-30T09:08:48.585 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.582+0000 7fa41f4a8740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55d64d298540 con 0x55d64cf8c1b0 2024-08-30T09:08:48.588 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.583+0000 7fa415ad4640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d64cf6b4d0 0x55d64d333340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:08:48.589 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.583+0000 7fa415ad4640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d64cf6b4d0 0x55d64d333340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:58362/0 (socket says 172.21.15.42:58362) 2024-08-30T09:08:48.589 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.583+0000 7fa415ad4640 1 -- 172.21.15.42:0/2764573992 learned_addr learned my addr 172.21.15.42:0/2764573992 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:08:48.589 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.583+0000 7fa4077fe640 1 --2- 172.21.15.42:0/878894658 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fa3fc0b4d40 0x7fa3fc0b7250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:08:48.589 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.583+0000 7fa415ad4640 1 -- 172.21.15.42:0/2764573992 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d64cf8c1b0 msgr2=0x55d64d332180 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-30T09:08:48.589 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.583+0000 7fa415ad4640 1 --2- 172.21.15.42:0/2764573992 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d64cf8c1b0 0x55d64d332180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:48.589 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.583+0000 7fa415ad4640 1 -- 172.21.15.42:0/2764573992 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d64cf632f0 msgr2=0x55d64d3327f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-30T09:08:48.589 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.583+0000 7fa415ad4640 1 --2- 172.21.15.42:0/2764573992 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d64cf632f0 0x55d64d3327f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:48.589 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.583+0000 7fa415ad4640 1 -- 172.21.15.42:0/2764573992 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d64cff7790 con 0x55d64cf6b4d0 2024-08-30T09:08:48.589 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.584+0000 7fa415ad4640 1 --2- 172.21.15.42:0/2764573992 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d64cf6b4d0 0x55d64d333340 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fa400109c80 tx=0x7fa4000d8c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c262452df9d1e12d server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:08:48.589 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.584+0000 7fa4077fe640 1 -- 172.21.15.42:0/878894658 <== mon.2 v2:172.21.15.167:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7fa400116840 con 0x7fa4101033c0 2024-08-30T09:08:48.589 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.585+0000 7fa404bfe640 1 -- 172.21.15.42:0/2764573992 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4000dcce0 con 0x55d64cf6b4d0 2024-08-30T09:08:48.589 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.585+0000 7fa404bfe640 1 -- 172.21.15.42:0/2764573992 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa4000dca20 con 0x55d64cf6b4d0 2024-08-30T09:08:48.590 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.586+0000 7fa416ad6640 1 --2- 172.21.15.42:0/878894658 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fa3fc0b4d40 0x7fa3fc0b7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:08:48.590 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.586+0000 7fa41f4a8740 1 -- 172.21.15.42:0/2764573992 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d64cf6b4d0 msgr2=0x55d64d333340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:08:48.590 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.586+0000 7fa41f4a8740 1 --2- 172.21.15.42:0/2764573992 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d64cf6b4d0 0x55d64d333340 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fa400109c80 tx=0x7fa4000d8c00 comp rx=0 tx=0).stop 2024-08-30T09:08:48.597 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.593+0000 7fa41f4a8740 1 -- 172.21.15.42:0/2764573992 shutdown_connections 2024-08-30T09:08:48.597 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.593+0000 7fa41f4a8740 1 --2- 172.21.15.42:0/2764573992 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d64cf6b4d0 0x55d64d333340 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:48.598 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.593+0000 7fa41f4a8740 1 --2- 172.21.15.42:0/2764573992 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d64cf632f0 0x55d64d3327f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:48.598 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.593+0000 7fa41f4a8740 1 --2- 172.21.15.42:0/2764573992 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d64cf8c1b0 0x55d64d332180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:48.598 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.593+0000 7fa41f4a8740 1 -- 172.21.15.42:0/2764573992 >> 172.21.15.42:0/2764573992 conn(0x55d64cf80300 msgr2=0x55d64d32dc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:08:48.599 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.597+0000 7fa41f4a8740 1 -- 172.21.15.42:0/2764573992 shutdown_connections 2024-08-30T09:08:48.603 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.598+0000 7fa416ad6640 1 --2- 172.21.15.42:0/878894658 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fa3fc0b4d40 0x7fa3fc0b7250 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fa40800aa00 tx=0x7fa4080091d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:08:48.603 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.601+0000 7fa41f4a8740 1 -- 172.21.15.42:0/2764573992 wait complete. 2024-08-30T09:08:48.603 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.601+0000 7fa41f4a8740 1 -- start start 2024-08-30T09:08:48.607 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.605+0000 7fa41f4a8740 1 Processor -- start 2024-08-30T09:08:48.607 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.605+0000 7fa41f4a8740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d64cf632f0 0x55d64d33b880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:08:48.608 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.605+0000 7fa41f4a8740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d64cf6b4d0 0x55d64d33bf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:08:48.608 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.605+0000 7fa41f4a8740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d64cf8c1b0 0x55d64d33cab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:08:48.608 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.605+0000 7fa41f4a8740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55d64d1cdab0 con 0x55d64cf8c1b0 2024-08-30T09:08:48.608 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.605+0000 7fa41f4a8740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55d64d21b680 con 0x55d64cf6b4d0 2024-08-30T09:08:48.608 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.605+0000 7fa41f4a8740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55d64d333c20 con 0x55d64cf632f0 2024-08-30T09:08:48.608 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.605+0000 7fa41f4a8740 10 client.0 _ll_register_callbacks cb 0x55d64cfa3a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:08:48.608 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.606+0000 7fa416ad6640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d64cf8c1b0 0x55d64d33cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:08:48.608 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.606+0000 7fa416ad6640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d64cf8c1b0 0x55d64d33cab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:58368/0 (socket says 172.21.15.42:58368) 2024-08-30T09:08:48.608 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.606+0000 7fa416ad6640 1 -- 172.21.15.42:0/306594079 learned_addr learned my addr 172.21.15.42:0/306594079 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:08:48.609 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.606+0000 7fa4162d5640 1 --2- 172.21.15.42:0/306594079 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d64cf632f0 0x55d64d33b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:08:48.610 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.607+0000 7fa415ad4640 1 --2- 172.21.15.42:0/306594079 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d64cf6b4d0 0x55d64d33bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:08:48.610 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.608+0000 7fa4162d5640 1 -- 172.21.15.42:0/306594079 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d64cf6b4d0 msgr2=0x55d64d33bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:08:48.610 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.608+0000 7fa4162d5640 1 --2- 172.21.15.42:0/306594079 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d64cf6b4d0 0x55d64d33bf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:48.610 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.608+0000 7fa4162d5640 1 -- 172.21.15.42:0/306594079 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d64cf8c1b0 msgr2=0x55d64d33cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:08:48.611 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.608+0000 7fa4162d5640 1 --2- 172.21.15.42:0/306594079 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d64cf8c1b0 0x55d64d33cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:48.611 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.608+0000 7fa4162d5640 1 -- 172.21.15.42:0/306594079 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d64d070620 con 0x55d64cf632f0 2024-08-30T09:08:48.611 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.608+0000 7fa415ad4640 1 --2- 172.21.15.42:0/306594079 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d64cf6b4d0 0x55d64d33bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-30T09:08:48.611 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.609+0000 7fa4162d5640 1 --2- 172.21.15.42:0/306594079 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d64cf632f0 0x55d64d33b880 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fa40c009db0 tx=0x7fa40c00b4b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4751bd3b71777574 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:08:48.611 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.610+0000 7fa3f9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-08-30T09:08:48.612 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.610+0000 7fa3f9ffb640 1 -- 172.21.15.42:0/306594079 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa40c015020 con 0x55d64cf632f0 2024-08-30T09:08:48.612 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.610+0000 7fa3f9ffb640 1 -- 172.21.15.42:0/306594079 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7fa40c011420 con 0x55d64cf632f0 2024-08-30T09:08:48.614 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.613+0000 7fa41f4a8740 1 -- 172.21.15.42:0/306594079 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa4000dca20 con 0x55d64cf632f0 2024-08-30T09:08:48.620 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.618+0000 7fa3f9ffb640 1 -- 172.21.15.42:0/306594079 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fa40c011680 con 0x55d64cf632f0 2024-08-30T09:08:48.624 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.623+0000 7fa41f4a8740 10 client.26542 fetch_fsmap learned FSMap version 37 2024-08-30T09:08:48.625 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.623+0000 7fa41f4a8740 1 -- 172.21.15.42:0/306594079 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55d64d345b20 con 0x55d64cf632f0 2024-08-30T09:08:48.625 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.623+0000 7fa3f9ffb640 1 -- 172.21.15.42:0/306594079 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa4000dca20 con 0x55d64cf632f0 2024-08-30T09:08:48.625 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.623+0000 7fa3f9ffb640 1 -- 172.21.15.42:0/306594079 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7fa40c019a00 con 0x55d64cf632f0 2024-08-30T09:08:48.626 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.624+0000 7fa41f4a8740 10 client.26542 fetch_fsmap finished waiting for FSMap version 37 2024-08-30T09:08:48.626 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.624+0000 7fa41f4a8740 10 client.26542 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-30T09:08:48.626 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.624+0000 7fa41f4a8740 20 client.26542 populate_metadata read hostname 'smithi042' 2024-08-30T09:08:48.626 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.624+0000 7fa41f4a8740 1 --2- 172.21.15.42:0/306594079 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55d64cf4ad80 0x55d64d3492e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:08:48.626 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.624+0000 7fa41f4a8740 4 client.26542 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-30T09:08:48.626 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.624+0000 7fa41f4a8740 1 -- 172.21.15.42:0/306594079 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d64d2a6f50 con 0x55d64cf4ad80 2024-08-30T09:08:48.627 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.625+0000 7fa415ad4640 1 --2- 172.21.15.42:0/306594079 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55d64cf4ad80 0x55d64d3492e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:08:48.629 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.627+0000 7fa415ad4640 1 --2- 172.21.15.42:0/306594079 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55d64cf4ad80 0x55d64d3492e0 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b6208f194ea631e1 server_cookie=728ec28f2ad52816 in_seq=0 out_seq=0 2024-08-30T09:08:48.629 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.627+0000 7fa3f9ffb640 10 client.26542 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:08:48.631 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.629+0000 7fa3f9ffb640 1 -- 172.21.15.42:0/306594079 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d64d2a6f50 con 0x55d64cf4ad80 2024-08-30T09:08:48.631 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.629+0000 7fa3f9ffb640 10 client.26542 handle_command_reply: tid=0 2024-08-30T09:08:48.631 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.629+0000 7fa41f4a8740 1 client.26542 shutdown 2024-08-30T09:08:48.631 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.629+0000 7fa41f4a8740 1 -- 172.21.15.42:0/306594079 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d64cf632f0 msgr2=0x55d64d33b880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:08:48.631 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.629+0000 7fa41f4a8740 1 --2- 172.21.15.42:0/306594079 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d64cf632f0 0x55d64d33b880 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fa40c009db0 tx=0x7fa40c00b4b0 comp rx=0 tx=0).stop 2024-08-30T09:08:48.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:08:48 smithi042 ceph-mon[18652]: pgmap v614: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 4.6 MiB/s wr, 487 op/s 2024-08-30T09:08:48.635 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.632+0000 7fa41f4a8740 1 -- 172.21.15.42:0/306594079 shutdown_connections 2024-08-30T09:08:48.635 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.632+0000 7fa41f4a8740 1 --2- 172.21.15.42:0/306594079 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55d64cf4ad80 0x55d64d3492e0 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:48.635 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.632+0000 7fa41f4a8740 1 --2- 172.21.15.42:0/306594079 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d64cf8c1b0 0x55d64d33cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:48.635 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.632+0000 7fa41f4a8740 1 --2- 172.21.15.42:0/306594079 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d64cf6b4d0 0x55d64d33bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:48.635 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.632+0000 7fa41f4a8740 1 --2- 172.21.15.42:0/306594079 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d64cf632f0 0x55d64d33b880 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:48.635 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.632+0000 7fa41f4a8740 1 -- 172.21.15.42:0/306594079 >> 172.21.15.42:0/306594079 conn(0x55d64cf80300 msgr2=0x55d64d32c170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:08:48.636 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.632+0000 7fa3f9ffb640 0 client.26542 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:08:48.637 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.635+0000 7fa41f4a8740 1 -- 172.21.15.42:0/306594079 shutdown_connections 2024-08-30T09:08:48.637 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.635+0000 7fa41f4a8740 1 -- 172.21.15.42:0/306594079 wait complete. 2024-08-30T09:08:48.639 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.637+0000 7fa41f4a8740 20 client.26542 trim_cache size 0 max 16384 2024-08-30T09:08:48.659 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.657+0000 7fa41f4a8740 1 Processor -- start 2024-08-30T09:08:48.659 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.657+0000 7fa41f4a8740 1 -- start start 2024-08-30T09:08:48.661 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.658+0000 7fa41f4a8740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d64cf632f0 0x55d64d328440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:08:48.661 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.658+0000 7fa41f4a8740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d64cf6b4d0 0x55d64d343990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:08:48.661 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.658+0000 7fa41f4a8740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d64cf8c1b0 0x55d64d332160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:08:48.661 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.658+0000 7fa41f4a8740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fa4101b5580 con 0x55d64cf8c1b0 2024-08-30T09:08:48.662 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.658+0000 7fa4162d5640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d64cf632f0 0x55d64d328440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:08:48.662 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.658+0000 7fa4162d5640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d64cf632f0 0x55d64d328440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:33436/0 (socket says 172.21.15.42:33436) 2024-08-30T09:08:48.662 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.658+0000 7fa4162d5640 1 -- 172.21.15.42:0/2721742178 learned_addr learned my addr 172.21.15.42:0/2721742178 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:08:48.662 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.658+0000 7fa41f4a8740 1 -- 172.21.15.42:0/2721742178 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa4101b2e50 con 0x55d64cf6b4d0 2024-08-30T09:08:48.662 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.658+0000 7fa415ad4640 1 --2- 172.21.15.42:0/2721742178 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d64cf6b4d0 0x55d64d343990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:08:48.662 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.658+0000 7fa416ad6640 1 --2- 172.21.15.42:0/2721742178 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d64cf8c1b0 0x55d64d332160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:08:48.662 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.658+0000 7fa41f4a8740 1 -- 172.21.15.42:0/2721742178 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fa4101b0ca0 con 0x55d64cf632f0 2024-08-30T09:08:48.662 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.659+0000 7fa415ad4640 1 -- 172.21.15.42:0/2721742178 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d64cf632f0 msgr2=0x55d64d328440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:08:48.662 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.659+0000 7fa415ad4640 1 --2- 172.21.15.42:0/2721742178 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d64cf632f0 0x55d64d328440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:48.662 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.659+0000 7fa415ad4640 1 -- 172.21.15.42:0/2721742178 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d64cf8c1b0 msgr2=0x55d64d332160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:08:48.662 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.659+0000 7fa415ad4640 1 --2- 172.21.15.42:0/2721742178 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d64cf8c1b0 0x55d64d332160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:48.662 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.659+0000 7fa415ad4640 1 -- 172.21.15.42:0/2721742178 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d64d33ce20 con 0x55d64cf6b4d0 2024-08-30T09:08:48.662 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.659+0000 7fa416ad6640 1 --2- 172.21.15.42:0/2721742178 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d64cf8c1b0 0x55d64d332160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-30T09:08:48.662 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.659+0000 7fa415ad4640 1 --2- 172.21.15.42:0/2721742178 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d64cf6b4d0 0x55d64d343990 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fa400117000 tx=0x7fa4000d8b10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bfe0ce27807ce193 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:08:48.663 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.659+0000 7fa3fbfff640 1 -- 172.21.15.42:0/2721742178 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4000dc860 con 0x55d64cf6b4d0 2024-08-30T09:08:48.663 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.659+0000 7fa3fbfff640 1 -- 172.21.15.42:0/2721742178 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa4000dd610 con 0x55d64cf6b4d0 2024-08-30T09:08:48.663 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.659+0000 7fa3fbfff640 1 -- 172.21.15.42:0/2721742178 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa400042cd0 con 0x55d64cf6b4d0 2024-08-30T09:08:48.666 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.660+0000 7fa41f4a8740 1 -- 172.21.15.42:0/2721742178 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d64cf6b4d0 msgr2=0x55d64d343990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:08:48.666 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.660+0000 7fa41f4a8740 1 --2- 172.21.15.42:0/2721742178 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d64cf6b4d0 0x55d64d343990 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fa400117000 tx=0x7fa4000d8b10 comp rx=0 tx=0).stop 2024-08-30T09:08:48.666 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.660+0000 7fa41f4a8740 1 -- 172.21.15.42:0/2721742178 shutdown_connections 2024-08-30T09:08:48.666 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.660+0000 7fa41f4a8740 1 --2- 172.21.15.42:0/2721742178 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d64cf8c1b0 0x55d64d332160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:48.666 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.660+0000 7fa41f4a8740 1 --2- 172.21.15.42:0/2721742178 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d64cf6b4d0 0x55d64d343990 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:48.666 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.660+0000 7fa41f4a8740 1 --2- 172.21.15.42:0/2721742178 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d64cf632f0 0x55d64d328440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:48.666 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.660+0000 7fa41f4a8740 1 -- 172.21.15.42:0/2721742178 >> 172.21.15.42:0/2721742178 conn(0x55d64cf80300 msgr2=0x55d64d3369c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:08:48.667 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.660+0000 7fa41f4a8740 1 -- 172.21.15.42:0/2721742178 shutdown_connections 2024-08-30T09:08:48.667 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.660+0000 7fa41f4a8740 1 -- 172.21.15.42:0/2721742178 wait complete. 2024-08-30T09:08:48.667 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.661+0000 7fa41f4a8740 1 -- start start 2024-08-30T09:08:48.667 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.661+0000 7fa41f4a8740 1 Processor -- start 2024-08-30T09:08:48.667 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.661+0000 7fa41f4a8740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d64cf632f0 0x55d64d343220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:08:48.667 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.661+0000 7fa41f4a8740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d64cf6b4d0 0x55d64d3438e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:08:48.667 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.661+0000 7fa41f4a8740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d64cf8c1b0 0x55d64d341ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:08:48.667 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.661+0000 7fa41f4a8740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fa4101b1310 con 0x55d64cf6b4d0 2024-08-30T09:08:48.667 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.661+0000 7fa41f4a8740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa4101b2780 con 0x55d64cf8c1b0 2024-08-30T09:08:48.667 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.661+0000 7fa41f4a8740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fa4101b19e0 con 0x55d64cf632f0 2024-08-30T09:08:48.667 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.661+0000 7fa415ad4640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d64cf6b4d0 0x55d64d3438e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:08:48.667 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.661+0000 7fa41f4a8740 10 client.0 _ll_register_callbacks cb 0x7fa40c006e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:08:48.667 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.661+0000 7fa415ad4640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d64cf6b4d0 0x55d64d3438e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:58386/0 (socket says 172.21.15.42:58386) 2024-08-30T09:08:48.667 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.661+0000 7fa415ad4640 1 -- 172.21.15.42:0/278597305 learned_addr learned my addr 172.21.15.42:0/278597305 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:08:48.667 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.662+0000 7fa4162d5640 1 --2- 172.21.15.42:0/278597305 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d64cf632f0 0x55d64d343220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:08:48.668 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.662+0000 7fa416ad6640 1 --2- 172.21.15.42:0/278597305 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d64cf8c1b0 0x55d64d341ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:08:48.668 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.662+0000 7fa4162d5640 1 -- 172.21.15.42:0/278597305 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d64cf8c1b0 msgr2=0x55d64d341ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:08:48.668 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.662+0000 7fa4162d5640 1 --2- 172.21.15.42:0/278597305 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d64cf8c1b0 0x55d64d341ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:48.668 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.662+0000 7fa4162d5640 1 -- 172.21.15.42:0/278597305 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d64cf6b4d0 msgr2=0x55d64d3438e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:08:48.668 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.662+0000 7fa4162d5640 1 --2- 172.21.15.42:0/278597305 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d64cf6b4d0 0x55d64d3438e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:48.668 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.662+0000 7fa4162d5640 1 -- 172.21.15.42:0/278597305 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d64d342760 con 0x55d64cf632f0 2024-08-30T09:08:48.668 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.662+0000 7fa415ad4640 1 --2- 172.21.15.42:0/278597305 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d64cf6b4d0 0x55d64d3438e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-30T09:08:48.668 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.662+0000 7fa4162d5640 1 --2- 172.21.15.42:0/278597305 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d64cf632f0 0x55d64d343220 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fa40c009a10 tx=0x7fa40c011a50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4507c2056c88337f server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:08:48.668 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.662+0000 7fa3f9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-08-30T09:08:48.668 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.662+0000 7fa3f9ffb640 1 -- 172.21.15.42:0/278597305 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa40c015040 con 0x55d64cf632f0 2024-08-30T09:08:48.668 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.662+0000 7fa41f4a8740 1 -- 172.21.15.42:0/278597305 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa4000dd610 con 0x55d64cf632f0 2024-08-30T09:08:48.668 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.662+0000 7fa3f9ffb640 1 -- 172.21.15.42:0/278597305 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7fa40c0193a0 con 0x55d64cf632f0 2024-08-30T09:08:48.669 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.663+0000 7fa3f9ffb640 1 -- 172.21.15.42:0/278597305 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fa40c032050 con 0x55d64cf632f0 2024-08-30T09:08:48.669 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.663+0000 7fa41f4a8740 10 client.26548 fetch_fsmap learned FSMap version 37 2024-08-30T09:08:48.669 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.663+0000 7fa41f4a8740 1 -- 172.21.15.42:0/278597305 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55d64d341660 con 0x55d64cf632f0 2024-08-30T09:08:48.669 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.663+0000 7fa3f9ffb640 1 -- 172.21.15.42:0/278597305 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa4000dd610 con 0x55d64cf632f0 2024-08-30T09:08:48.669 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.663+0000 7fa3f9ffb640 1 -- 172.21.15.42:0/278597305 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7fa40c022660 con 0x55d64cf632f0 2024-08-30T09:08:48.669 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.663+0000 7fa41f4a8740 10 client.26548 fetch_fsmap finished waiting for FSMap version 37 2024-08-30T09:08:48.669 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.663+0000 7fa41f4a8740 10 client.26548 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-30T09:08:48.669 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.663+0000 7fa41f4a8740 20 client.26548 populate_metadata read hostname 'smithi042' 2024-08-30T09:08:48.669 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.663+0000 7fa41f4a8740 1 --2- 172.21.15.42:0/278597305 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55d64cf201c0 0x55d64d3b6330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:08:48.669 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.663+0000 7fa41f4a8740 4 client.26548 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-30T09:08:48.669 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.663+0000 7fa41f4a8740 1 -- 172.21.15.42:0/278597305 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d64d2a29f0 con 0x55d64cf201c0 2024-08-30T09:08:48.669 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.663+0000 7fa415ad4640 1 --2- 172.21.15.42:0/278597305 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55d64cf201c0 0x55d64d3b6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:08:48.669 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.664+0000 7fa415ad4640 1 --2- 172.21.15.42:0/278597305 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55d64cf201c0 0x55d64d3b6330 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e63dfdf346e27f8c server_cookie=939c0a498460f8b3 in_seq=0 out_seq=0 2024-08-30T09:08:48.669 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.664+0000 7fa3f9ffb640 10 client.26548 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:08:48.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:08:48 smithi190 ceph-mon[22001]: pgmap v614: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 4.6 MiB/s wr, 487 op/s 2024-08-30T09:08:48.843 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.841+0000 7fa3f9ffb640 1 -- 172.21.15.42:0/278597305 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d64d2a29f0 con 0x55d64cf201c0 2024-08-30T09:08:48.843 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.841+0000 7fa3f9ffb640 10 client.26548 handle_command_reply: tid=0 2024-08-30T09:08:48.844 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.841+0000 7fa41f4a8740 1 client.26548 shutdown 2024-08-30T09:08:48.844 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.841+0000 7fa41f4a8740 1 -- 172.21.15.42:0/278597305 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d64cf632f0 msgr2=0x55d64d343220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:08:48.844 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.841+0000 7fa41f4a8740 1 --2- 172.21.15.42:0/278597305 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d64cf632f0 0x55d64d343220 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fa40c009a10 tx=0x7fa40c011a50 comp rx=0 tx=0).stop 2024-08-30T09:08:48.844 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.841+0000 7fa41f4a8740 1 -- 172.21.15.42:0/278597305 shutdown_connections 2024-08-30T09:08:48.844 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.841+0000 7fa41f4a8740 1 --2- 172.21.15.42:0/278597305 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55d64cf201c0 0x55d64d3b6330 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:48.844 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.841+0000 7fa41f4a8740 1 --2- 172.21.15.42:0/278597305 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d64cf8c1b0 0x55d64d341ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:48.844 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.841+0000 7fa3f9ffb640 0 client.26548 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:08:48.844 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.841+0000 7fa41f4a8740 1 --2- 172.21.15.42:0/278597305 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d64cf6b4d0 0x55d64d3438e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:48.844 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.841+0000 7fa41f4a8740 1 --2- 172.21.15.42:0/278597305 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d64cf632f0 0x55d64d343220 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:48.844 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.841+0000 7fa41f4a8740 1 -- 172.21.15.42:0/278597305 >> 172.21.15.42:0/278597305 conn(0x55d64cf80300 msgr2=0x55d64d3369c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:08:48.844 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.841+0000 7fa41f4a8740 1 -- 172.21.15.42:0/278597305 shutdown_connections 2024-08-30T09:08:48.844 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.841+0000 7fa41f4a8740 1 -- 172.21.15.42:0/278597305 wait complete. 2024-08-30T09:08:48.845 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.841+0000 7fa41f4a8740 20 client.26548 trim_cache size 0 max 16384 2024-08-30T09:08:48.845 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-08-30T09:08:48.845 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2024-08-30T09:08:48.845 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-08-30T09:08:48.845 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-08-30T09:08:48.845 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.842+0000 7fa418539640 1 -- 172.21.15.42:0/878894658 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fa3fc0b4d40 msgr2=0x7fa3fc0b7250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:08:48.845 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.843+0000 7fa418539640 1 --2- 172.21.15.42:0/878894658 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fa3fc0b4d40 0x7fa3fc0b7250 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fa40800aa00 tx=0x7fa4080091d0 comp rx=0 tx=0).stop 2024-08-30T09:08:48.845 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.843+0000 7fa418539640 1 -- 172.21.15.42:0/878894658 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa4101033c0 msgr2=0x7fa41010a520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:08:48.846 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.843+0000 7fa418539640 1 --2- 172.21.15.42:0/878894658 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa4101033c0 0x7fa41010a520 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fa400000c00 tx=0x7fa400004b90 comp rx=0 tx=0).stop 2024-08-30T09:08:48.847 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.846+0000 7fa418539640 1 -- 172.21.15.42:0/878894658 shutdown_connections 2024-08-30T09:08:48.848 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.846+0000 7fa418539640 1 --2- 172.21.15.42:0/878894658 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fa3fc0b4d40 0x7fa3fc0b7250 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:48.848 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.846+0000 7fa418539640 1 --2- 172.21.15.42:0/878894658 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa410103d60 0x7fa41010b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:48.848 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.846+0000 7fa418539640 1 --2- 172.21.15.42:0/878894658 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa4101033c0 0x7fa41010a520 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:48.848 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.846+0000 7fa418539640 1 --2- 172.21.15.42:0/878894658 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa410102340 0x7fa410109eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:08:48.848 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.846+0000 7fa418539640 1 -- 172.21.15.42:0/878894658 >> 172.21.15.42:0/878894658 conn(0x7fa4100fdc80 msgr2=0x7fa4100fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:08:48.853 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.852+0000 7fa418539640 1 -- 172.21.15.42:0/878894658 shutdown_connections 2024-08-30T09:08:48.856 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:08:48.852+0000 7fa418539640 1 -- 172.21.15.42:0/878894658 wait complete. 2024-08-30T09:08:50.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:08:50 smithi167 ceph-mon[22063]: pgmap v615: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 5.9 MiB/s wr, 610 op/s 2024-08-30T09:08:50.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:08:50 smithi042 ceph-mon[18652]: pgmap v615: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 5.9 MiB/s wr, 610 op/s 2024-08-30T09:08:50.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:08:50 smithi190 ceph-mon[22001]: pgmap v615: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 5.9 MiB/s wr, 610 op/s 2024-08-30T09:08:50.953 INFO:teuthology.orchestra.run.smithi042.stderr:progress: 485.0 s, 415.4 tps, lat 229.939 ms stddev 305.841 2024-08-30T09:08:52.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:08:52 smithi167 ceph-mon[22063]: pgmap v616: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.1 MiB/s wr, 559 op/s 2024-08-30T09:08:52.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:08:52 smithi042 ceph-mon[18652]: pgmap v616: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.1 MiB/s wr, 559 op/s 2024-08-30T09:08:52.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:08:52 smithi190 ceph-mon[22001]: pgmap v616: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.1 MiB/s wr, 559 op/s 2024-08-30T09:08:54.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:08:54 smithi167 ceph-mon[22063]: pgmap v617: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 6.3 MiB/s wr, 696 op/s 2024-08-30T09:08:54.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:08:54 smithi042 ceph-mon[18652]: pgmap v617: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 6.3 MiB/s wr, 696 op/s 2024-08-30T09:08:54.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:08:54 smithi190 ceph-mon[22001]: pgmap v617: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 6.3 MiB/s wr, 696 op/s 2024-08-30T09:08:55.953 INFO:teuthology.orchestra.run.smithi042.stderr:progress: 490.0 s, 703.6 tps, lat 147.520 ms stddev 193.263 2024-08-30T09:08:56.634 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:08:56 smithi042 ceph-mon[18652]: pgmap v618: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 5.0 MiB/s wr, 593 op/s 2024-08-30T09:08:56.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:08:56 smithi190 ceph-mon[22001]: pgmap v618: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 5.0 MiB/s wr, 593 op/s 2024-08-30T09:08:56.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:08:56 smithi167 ceph-mon[22063]: pgmap v618: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 5.0 MiB/s wr, 593 op/s 2024-08-30T09:08:58.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:08:58 smithi042 ceph-mon[18652]: pgmap v619: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 5.3 MiB/s wr, 611 op/s 2024-08-30T09:08:58.665 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-30T09:08:58.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:08:58 smithi190 ceph-mon[22001]: pgmap v619: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 5.3 MiB/s wr, 611 op/s 2024-08-30T09:08:58.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:08:58 smithi167 ceph-mon[22063]: pgmap v619: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 5.3 MiB/s wr, 611 op/s 2024-08-30T09:08:59.653 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T09:09:00.607 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:09:00 smithi042 ceph-mon[18652]: pgmap v620: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 6.2 MiB/s wr, 680 op/s 2024-08-30T09:09:00.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:09:00 smithi190 ceph-mon[22001]: pgmap v620: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 6.2 MiB/s wr, 680 op/s 2024-08-30T09:09:00.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.761+0000 7fe43e3ba640 1 -- 172.21.15.42:0/1313309933 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe438111ce0 msgr2=0x7fe43810a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:09:00.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.761+0000 7fe43e3ba640 1 --2- 172.21.15.42:0/1313309933 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe438111ce0 0x7fe43810a160 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fe42801a910 tx=0x7fe42803b470 comp rx=0 tx=0).stop 2024-08-30T09:09:00.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.762+0000 7fe43e3ba640 1 -- 172.21.15.42:0/1313309933 shutdown_connections 2024-08-30T09:09:00.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.762+0000 7fe43e3ba640 1 --2- 172.21.15.42:0/1313309933 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe43810a780 0x7fe43810ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:00.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.762+0000 7fe43e3ba640 1 --2- 172.21.15.42:0/1313309933 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe43811c9d0 0x7fe43811eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:00.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.762+0000 7fe43e3ba640 1 --2- 172.21.15.42:0/1313309933 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe438111ce0 0x7fe43810a160 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:00.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.762+0000 7fe43e3ba640 1 -- 172.21.15.42:0/1313309933 >> 172.21.15.42:0/1313309933 conn(0x7fe438070c60 msgr2=0x7fe438071070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:09:00.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.766+0000 7fe43e3ba640 1 -- 172.21.15.42:0/1313309933 shutdown_connections 2024-08-30T09:09:00.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.766+0000 7fe43e3ba640 1 -- 172.21.15.42:0/1313309933 wait complete. 2024-08-30T09:09:00.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.767+0000 7fe43e3ba640 1 Processor -- start 2024-08-30T09:09:00.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.768+0000 7fe43e3ba640 1 -- start start 2024-08-30T09:09:00.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.768+0000 7fe43e3ba640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe43810a780 0x7fe43811c180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:00.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.768+0000 7fe43e3ba640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe438111ce0 0x7fe438118460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:00.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.768+0000 7fe43e3ba640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe43811c9d0 0x7fe438118a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:00.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.768+0000 7fe43e3ba640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fe4381149b0 con 0x7fe438111ce0 2024-08-30T09:09:00.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.768+0000 7fe43e3ba640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe438114830 con 0x7fe43811c9d0 2024-08-30T09:09:00.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.768+0000 7fe43e3ba640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fe438114b30 con 0x7fe43810a780 2024-08-30T09:09:00.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.771+0000 7fe437fff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe43810a780 0x7fe43811c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:00.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.771+0000 7fe4377fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe438111ce0 0x7fe438118460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:00.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.771+0000 7fe4377fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe438111ce0 0x7fe438118460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:53922/0 (socket says 172.21.15.42:53922) 2024-08-30T09:09:00.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.771+0000 7fe4377fe640 1 -- 172.21.15.42:0/1668949669 learned_addr learned my addr 172.21.15.42:0/1668949669 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:09:00.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.772+0000 7fe437fff640 1 -- 172.21.15.42:0/1668949669 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe43811c9d0 msgr2=0x7fe438118a20 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-08-30T09:09:00.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.772+0000 7fe437fff640 1 --2- 172.21.15.42:0/1668949669 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe43811c9d0 0x7fe438118a20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:00.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.772+0000 7fe437fff640 1 -- 172.21.15.42:0/1668949669 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe438111ce0 msgr2=0x7fe438118460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:09:00.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.772+0000 7fe437fff640 1 --2- 172.21.15.42:0/1668949669 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe438111ce0 0x7fe438118460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:00.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.772+0000 7fe437fff640 1 -- 172.21.15.42:0/1668949669 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe438119240 con 0x7fe43810a780 2024-08-30T09:09:00.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.777+0000 7fe437fff640 1 --2- 172.21.15.42:0/1668949669 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe43810a780 0x7fe43811c180 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fe42803ba60 tx=0x7fe4280056d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:09:00.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.780+0000 7fe4357fa640 1 -- 172.21.15.42:0/1668949669 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe428051070 con 0x7fe43810a780 2024-08-30T09:09:00.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.780+0000 7fe43e3ba640 1 -- 172.21.15.42:0/1668949669 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe43807d440 con 0x7fe43810a780 2024-08-30T09:09:00.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.780+0000 7fe43e3ba640 1 -- 172.21.15.42:0/1668949669 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe43807d980 con 0x7fe43810a780 2024-08-30T09:09:00.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.780+0000 7fe4357fa640 1 -- 172.21.15.42:0/1668949669 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe428040cd0 con 0x7fe43810a780 2024-08-30T09:09:00.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.780+0000 7fe4357fa640 1 -- 172.21.15.42:0/1668949669 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe428042ba0 con 0x7fe43810a780 2024-08-30T09:09:00.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.787+0000 7fe4357fa640 1 -- 172.21.15.42:0/1668949669 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7fe428054070 con 0x7fe43810a780 2024-08-30T09:09:00.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.789+0000 7fe4357fa640 1 --2- 172.21.15.42:0/1668949669 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fe4200b4d30 0x7fe4200b7240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:00.796 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.789+0000 7fe4357fa640 1 -- 172.21.15.42:0/1668949669 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7fe428116a00 con 0x7fe43810a780 2024-08-30T09:09:00.796 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.791+0000 7fe44492f740 1 Processor -- start 2024-08-30T09:09:00.797 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.794+0000 7fe4377fe640 1 --2- 172.21.15.42:0/1668949669 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fe4200b4d30 0x7fe4200b7240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:00.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.795+0000 7fe44492f740 1 -- start start 2024-08-30T09:09:00.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.796+0000 7fe44492f740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c11f0c0070 0x55c11f4652c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:00.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.796+0000 7fe44492f740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c11f0971b0 0x55c11f465800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:00.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.796+0000 7fe44492f740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c11f09f390 0x55c11f465d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:00.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.796+0000 7fe44492f740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55c11f3dc630 con 0x55c11f0971b0 2024-08-30T09:09:00.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.796+0000 7fe44492f740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55c11f3cbbc0 con 0x55c11f09f390 2024-08-30T09:09:00.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.796+0000 7fe44492f740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55c11f3cfb70 con 0x55c11f0c0070 2024-08-30T09:09:00.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.797+0000 7fe43c957640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c11f09f390 0x55c11f465d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:00.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.798+0000 7fe43c957640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c11f09f390 0x55c11f465d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:42840/0 (socket says 172.21.15.42:42840) 2024-08-30T09:09:00.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.798+0000 7fe43c957640 1 -- 172.21.15.42:0/1854457508 learned_addr learned my addr 172.21.15.42:0/1854457508 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:09:00.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.801+0000 7fe437fff640 1 --2- 172.21.15.42:0/1854457508 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c11f0c0070 0x55c11f4652c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:00.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.802+0000 7fe43c957640 1 -- 172.21.15.42:0/1854457508 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c11f0c0070 msgr2=0x55c11f4652c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:09:00.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.802+0000 7fe43c957640 1 --2- 172.21.15.42:0/1854457508 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c11f0c0070 0x55c11f4652c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:00.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.802+0000 7fe43c957640 1 -- 172.21.15.42:0/1854457508 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c11f0971b0 msgr2=0x55c11f465800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:09:00.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.802+0000 7fe43c957640 1 --2- 172.21.15.42:0/1854457508 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c11f0971b0 0x55c11f465800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:00.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.802+0000 7fe43c957640 1 -- 172.21.15.42:0/1854457508 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c11f12b650 con 0x55c11f09f390 2024-08-30T09:09:00.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.804+0000 7fe43c957640 1 --2- 172.21.15.42:0/1854457508 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c11f09f390 0x55c11f465d40 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fe42c009bd0 tx=0x7fe42c00e720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1695b4cc22caffc9 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:09:00.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.805+0000 7fe41e7fc640 1 -- 172.21.15.42:0/1854457508 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe42c005020 con 0x55c11f09f390 2024-08-30T09:09:00.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.805+0000 7fe41e7fc640 1 -- 172.21.15.42:0/1854457508 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe42c0047a0 con 0x55c11f09f390 2024-08-30T09:09:00.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.805+0000 7fe41e7fc640 1 -- 172.21.15.42:0/1854457508 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe42c010690 con 0x55c11f09f390 2024-08-30T09:09:00.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.807+0000 7fe44492f740 1 -- 172.21.15.42:0/1854457508 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c11f09f390 msgr2=0x55c11f465d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:09:00.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.807+0000 7fe44492f740 1 --2- 172.21.15.42:0/1854457508 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c11f09f390 0x55c11f465d40 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fe42c009bd0 tx=0x7fe42c00e720 comp rx=0 tx=0).stop 2024-08-30T09:09:00.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.807+0000 7fe44492f740 1 -- 172.21.15.42:0/1854457508 shutdown_connections 2024-08-30T09:09:00.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.807+0000 7fe44492f740 1 --2- 172.21.15.42:0/1854457508 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c11f09f390 0x55c11f465d40 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:00.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.807+0000 7fe44492f740 1 --2- 172.21.15.42:0/1854457508 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c11f0971b0 0x55c11f465800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:00.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.807+0000 7fe44492f740 1 --2- 172.21.15.42:0/1854457508 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c11f0c0070 0x55c11f4652c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:00.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.807+0000 7fe44492f740 1 -- 172.21.15.42:0/1854457508 >> 172.21.15.42:0/1854457508 conn(0x55c11f0b4460 msgr2=0x55c11f4631b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:09:00.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.813+0000 7fe44492f740 1 -- 172.21.15.42:0/1854457508 shutdown_connections 2024-08-30T09:09:00.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.817+0000 7fe4377fe640 1 --2- 172.21.15.42:0/1668949669 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fe4200b4d30 0x7fe4200b7240 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fe438071340 tx=0x7fe4300107a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:09:00.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.817+0000 7fe44492f740 1 -- 172.21.15.42:0/1854457508 wait complete. 2024-08-30T09:09:00.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.817+0000 7fe44492f740 1 -- start start 2024-08-30T09:09:00.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.817+0000 7fe44492f740 1 Processor -- start 2024-08-30T09:09:00.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.818+0000 7fe44492f740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c11f09f390 0x55c11f470a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:00.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.818+0000 7fe44492f740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c11f0c0070 0x55c11f470fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:00.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.818+0000 7fe44492f740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c11f07e9f0 0x55c11f47ce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:00.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.818+0000 7fe44492f740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55c11f22e700 con 0x55c11f0c0070 2024-08-30T09:09:00.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.818+0000 7fe44492f740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55c11f352700 con 0x55c11f07e9f0 2024-08-30T09:09:00.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.818+0000 7fe44492f740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55c11f473aa0 con 0x55c11f09f390 2024-08-30T09:09:00.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.818+0000 7fe44492f740 10 client.0 _ll_register_callbacks cb 0x55c11f381dc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:09:00.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.819+0000 7fe43c957640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c11f07e9f0 0x55c11f47ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:00.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.819+0000 7fe43c957640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c11f07e9f0 0x55c11f47ce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:42842/0 (socket says 172.21.15.42:42842) 2024-08-30T09:09:00.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.819+0000 7fe43c957640 1 -- 172.21.15.42:0/2830657822 learned_addr learned my addr 172.21.15.42:0/2830657822 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:09:00.826 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:09:00 smithi167 ceph-mon[22063]: pgmap v620: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 6.2 MiB/s wr, 680 op/s 2024-08-30T09:09:00.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.824+0000 7fe43c957640 1 -- 172.21.15.42:0/2830657822 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c11f09f390 msgr2=0x55c11f470a60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-30T09:09:00.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.824+0000 7fe43c957640 1 --2- 172.21.15.42:0/2830657822 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c11f09f390 0x55c11f470a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:00.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.824+0000 7fe43c957640 1 -- 172.21.15.42:0/2830657822 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c11f0c0070 msgr2=0x55c11f470fa0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-30T09:09:00.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.824+0000 7fe43c957640 1 --2- 172.21.15.42:0/2830657822 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c11f0c0070 0x55c11f470fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:00.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.824+0000 7fe43c957640 1 -- 172.21.15.42:0/2830657822 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c11f38b8b0 con 0x55c11f07e9f0 2024-08-30T09:09:00.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.829+0000 7fe43c957640 1 --2- 172.21.15.42:0/2830657822 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c11f07e9f0 0x55c11f47ce20 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fe42c004ee0 tx=0x7fe42c0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e9281271e7096c40 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:09:00.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.830+0000 7fe4177fe640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-30T09:09:00.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.834+0000 7fe4177fe640 1 -- 172.21.15.42:0/2830657822 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe42c01d040 con 0x55c11f07e9f0 2024-08-30T09:09:00.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.834+0000 7fe4177fe640 1 -- 172.21.15.42:0/2830657822 <== mon.2 v2:172.21.15.167:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7fe42c005020 con 0x55c11f07e9f0 2024-08-30T09:09:00.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.834+0000 7fe44492f740 1 -- 172.21.15.42:0/2830657822 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe42c0047a0 con 0x55c11f07e9f0 2024-08-30T09:09:00.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.835+0000 7fe4177fe640 1 -- 172.21.15.42:0/2830657822 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fe42c005240 con 0x55c11f07e9f0 2024-08-30T09:09:00.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.840+0000 7fe44492f740 10 client.26576 fetch_fsmap learned FSMap version 37 2024-08-30T09:09:00.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.840+0000 7fe44492f740 1 -- 172.21.15.42:0/2830657822 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55c11f47f8c0 con 0x55c11f07e9f0 2024-08-30T09:09:00.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.842+0000 7fe4177fe640 1 -- 172.21.15.42:0/2830657822 <== mon.2 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe42c0047a0 con 0x55c11f07e9f0 2024-08-30T09:09:00.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.842+0000 7fe4177fe640 1 -- 172.21.15.42:0/2830657822 <== mon.2 v2:172.21.15.167:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7fe42c018c70 con 0x55c11f07e9f0 2024-08-30T09:09:00.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.842+0000 7fe44492f740 10 client.26576 fetch_fsmap finished waiting for FSMap version 37 2024-08-30T09:09:00.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.842+0000 7fe44492f740 10 client.26576 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-30T09:09:00.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.842+0000 7fe44492f740 20 client.26576 populate_metadata read hostname 'smithi042' 2024-08-30T09:09:00.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.842+0000 7fe44492f740 1 --2- 172.21.15.42:0/2830657822 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55c11f053da0 0x55c11f482cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:00.846 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.842+0000 7fe44492f740 4 client.26576 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-30T09:09:00.846 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.842+0000 7fe44492f740 1 -- 172.21.15.42:0/2830657822 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c11f3de580 con 0x55c11f053da0 2024-08-30T09:09:00.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.846+0000 7fe437fff640 1 --2- 172.21.15.42:0/2830657822 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55c11f053da0 0x55c11f482cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:00.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.855+0000 7fe437fff640 1 --2- 172.21.15.42:0/2830657822 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55c11f053da0 0x55c11f482cd0 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=42ebafd2731a93f1 server_cookie=62c44f6701625654 in_seq=0 out_seq=0 2024-08-30T09:09:00.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.856+0000 7fe4177fe640 10 client.26576 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:09:00.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.861+0000 7fe4177fe640 1 -- 172.21.15.42:0/2830657822 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55c11f3de580 con 0x55c11f053da0 2024-08-30T09:09:00.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.861+0000 7fe4177fe640 10 client.26576 handle_command_reply: tid=0 2024-08-30T09:09:00.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.861+0000 7fe44492f740 1 client.26576 shutdown 2024-08-30T09:09:00.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.863+0000 7fe44492f740 1 -- 172.21.15.42:0/2830657822 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c11f07e9f0 msgr2=0x55c11f47ce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:09:00.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.863+0000 7fe44492f740 1 --2- 172.21.15.42:0/2830657822 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c11f07e9f0 0x55c11f47ce20 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fe42c004ee0 tx=0x7fe42c0046c0 comp rx=0 tx=0).stop 2024-08-30T09:09:00.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.865+0000 7fe44492f740 1 -- 172.21.15.42:0/2830657822 shutdown_connections 2024-08-30T09:09:00.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.865+0000 7fe44492f740 1 --2- 172.21.15.42:0/2830657822 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55c11f053da0 0x55c11f482cd0 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:00.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.865+0000 7fe44492f740 1 --2- 172.21.15.42:0/2830657822 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c11f07e9f0 0x55c11f47ce20 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:00.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.865+0000 7fe44492f740 1 --2- 172.21.15.42:0/2830657822 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c11f0c0070 0x55c11f470fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:00.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.865+0000 7fe44492f740 1 --2- 172.21.15.42:0/2830657822 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c11f09f390 0x55c11f470a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:00.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.865+0000 7fe44492f740 1 -- 172.21.15.42:0/2830657822 >> 172.21.15.42:0/2830657822 conn(0x55c11f0b4460 msgr2=0x55c11f464270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:09:00.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.865+0000 7fe4177fe640 0 client.26576 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:09:00.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.866+0000 7fe44492f740 1 -- 172.21.15.42:0/2830657822 shutdown_connections 2024-08-30T09:09:00.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.868+0000 7fe44492f740 1 -- 172.21.15.42:0/2830657822 wait complete. 2024-08-30T09:09:00.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.869+0000 7fe44492f740 20 client.26576 trim_cache size 0 max 16384 2024-08-30T09:09:00.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.907+0000 7fe44492f740 1 Processor -- start 2024-08-30T09:09:00.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.909+0000 7fe44492f740 1 -- start start 2024-08-30T09:09:00.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.909+0000 7fe44492f740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c11f07e9f0 0x55c11f4631c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:00.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.909+0000 7fe44492f740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c11f09f390 0x55c11f47cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:00.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.909+0000 7fe44492f740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c11f0c0070 0x55c11f4652a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:00.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.909+0000 7fe44492f740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fe4240125b0 con 0x55c11f09f390 2024-08-30T09:09:00.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.909+0000 7fe44492f740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe42400fe80 con 0x55c11f07e9f0 2024-08-30T09:09:00.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.909+0000 7fe44492f740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fe42400dcd0 con 0x55c11f0c0070 2024-08-30T09:09:00.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.910+0000 7fe437fff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c11f07e9f0 0x55c11f4631c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:00.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.910+0000 7fe437fff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c11f07e9f0 0x55c11f4631c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:42856/0 (socket says 172.21.15.42:42856) 2024-08-30T09:09:00.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.910+0000 7fe437fff640 1 -- 172.21.15.42:0/3768874384 learned_addr learned my addr 172.21.15.42:0/3768874384 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:09:00.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.910+0000 7fe4377fe640 1 --2- 172.21.15.42:0/3768874384 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c11f09f390 0x55c11f47cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:00.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.911+0000 7fe437fff640 1 -- 172.21.15.42:0/3768874384 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c11f0c0070 msgr2=0x55c11f4652a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:09:00.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.911+0000 7fe437fff640 1 --2- 172.21.15.42:0/3768874384 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c11f0c0070 0x55c11f4652a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:00.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.911+0000 7fe437fff640 1 -- 172.21.15.42:0/3768874384 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c11f09f390 msgr2=0x55c11f47cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:09:00.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.911+0000 7fe437fff640 1 --2- 172.21.15.42:0/3768874384 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c11f09f390 0x55c11f47cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:00.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.911+0000 7fe437fff640 1 -- 172.21.15.42:0/3768874384 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c11f46ef60 con 0x55c11f07e9f0 2024-08-30T09:09:00.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.912+0000 7fe437fff640 1 --2- 172.21.15.42:0/3768874384 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c11f07e9f0 0x55c11f4631c0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fe42801a2b0 tx=0x7fe428059000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=597b0981d36fa7f6 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:09:00.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.913+0000 7fe41dffb640 1 -- 172.21.15.42:0/3768874384 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe42808a070 con 0x55c11f07e9f0 2024-08-30T09:09:00.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.913+0000 7fe41dffb640 1 -- 172.21.15.42:0/3768874384 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe4280dd740 con 0x55c11f07e9f0 2024-08-30T09:09:00.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.913+0000 7fe41dffb640 1 -- 172.21.15.42:0/3768874384 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe42805b040 con 0x55c11f07e9f0 2024-08-30T09:09:00.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.914+0000 7fe44492f740 1 -- 172.21.15.42:0/3768874384 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c11f07e9f0 msgr2=0x55c11f4631c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:09:00.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.914+0000 7fe44492f740 1 --2- 172.21.15.42:0/3768874384 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c11f07e9f0 0x55c11f4631c0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fe42801a2b0 tx=0x7fe428059000 comp rx=0 tx=0).stop 2024-08-30T09:09:00.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.915+0000 7fe44492f740 1 -- 172.21.15.42:0/3768874384 shutdown_connections 2024-08-30T09:09:00.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.915+0000 7fe44492f740 1 --2- 172.21.15.42:0/3768874384 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c11f0c0070 0x55c11f4652a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:00.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.915+0000 7fe44492f740 1 --2- 172.21.15.42:0/3768874384 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c11f09f390 0x55c11f47cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:00.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.915+0000 7fe44492f740 1 --2- 172.21.15.42:0/3768874384 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c11f07e9f0 0x55c11f4631c0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:00.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.915+0000 7fe44492f740 1 -- 172.21.15.42:0/3768874384 >> 172.21.15.42:0/3768874384 conn(0x55c11f0b4460 msgr2=0x55c11f462ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:09:00.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.915+0000 7fe44492f740 1 -- 172.21.15.42:0/3768874384 shutdown_connections 2024-08-30T09:09:00.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.915+0000 7fe44492f740 1 -- 172.21.15.42:0/3768874384 wait complete. 2024-08-30T09:09:00.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.916+0000 7fe44492f740 1 -- start start 2024-08-30T09:09:00.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.916+0000 7fe44492f740 1 Processor -- start 2024-08-30T09:09:00.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.917+0000 7fe44492f740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c11f07e9f0 0x55c11f478f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:00.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.917+0000 7fe44492f740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c11f09f390 0x55c11f4794c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:00.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.917+0000 7fe44492f740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c11f0c0070 0x55c11f4873c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:00.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.917+0000 7fe4377fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c11f09f390 0x55c11f4794c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:00.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.917+0000 7fe4377fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c11f09f390 0x55c11f4794c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:42868/0 (socket says 172.21.15.42:42868) 2024-08-30T09:09:00.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.917+0000 7fe437fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c11f07e9f0 0x55c11f478f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:00.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.917+0000 7fe4377fe640 1 -- 172.21.15.42:0/360054660 learned_addr learned my addr 172.21.15.42:0/360054660 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:09:00.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.918+0000 7fe43c957640 1 --2- 172.21.15.42:0/360054660 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c11f0c0070 0x55c11f4873c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:00.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.918+0000 7fe44492f740 1 -- 172.21.15.42:0/360054660 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fe42400e340 con 0x55c11f07e9f0 2024-08-30T09:09:00.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.918+0000 7fe44492f740 1 -- 172.21.15.42:0/360054660 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe42400f7b0 con 0x55c11f09f390 2024-08-30T09:09:00.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.918+0000 7fe44492f740 1 -- 172.21.15.42:0/360054660 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fe42400ea10 con 0x55c11f0c0070 2024-08-30T09:09:00.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.918+0000 7fe44492f740 10 client.0 _ll_register_callbacks cb 0x7fe42c009aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:09:00.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.918+0000 7fe4377fe640 1 -- 172.21.15.42:0/360054660 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c11f0c0070 msgr2=0x55c11f4873c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:09:00.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.918+0000 7fe4377fe640 1 --2- 172.21.15.42:0/360054660 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c11f0c0070 0x55c11f4873c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:00.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.918+0000 7fe4377fe640 1 -- 172.21.15.42:0/360054660 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c11f07e9f0 msgr2=0x55c11f478f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:09:00.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.918+0000 7fe4377fe640 1 --2- 172.21.15.42:0/360054660 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c11f07e9f0 0x55c11f478f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:00.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.918+0000 7fe4377fe640 1 -- 172.21.15.42:0/360054660 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c11f4879b0 con 0x55c11f09f390 2024-08-30T09:09:00.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.918+0000 7fe43c957640 1 --2- 172.21.15.42:0/360054660 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c11f0c0070 0x55c11f4873c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-30T09:09:00.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.919+0000 7fe4377fe640 1 --2- 172.21.15.42:0/360054660 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c11f09f390 0x55c11f4794c0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fe430005f40 tx=0x7fe430004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d692b28197c34a85 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:09:00.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.919+0000 7fe4177fe640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-30T09:09:00.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.919+0000 7fe4177fe640 1 -- 172.21.15.42:0/360054660 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe430025070 con 0x55c11f09f390 2024-08-30T09:09:00.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.919+0000 7fe4177fe640 1 -- 172.21.15.42:0/360054660 <== mon.2 v2:172.21.15.167:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7fe430021630 con 0x55c11f09f390 2024-08-30T09:09:00.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.919+0000 7fe44492f740 1 -- 172.21.15.42:0/360054660 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe4280dd740 con 0x55c11f09f390 2024-08-30T09:09:00.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.920+0000 7fe4177fe640 1 -- 172.21.15.42:0/360054660 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fe43002f050 con 0x55c11f09f390 2024-08-30T09:09:00.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.920+0000 7fe44492f740 10 client.26582 fetch_fsmap learned FSMap version 37 2024-08-30T09:09:00.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.920+0000 7fe44492f740 1 -- 172.21.15.42:0/360054660 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55c11f488c40 con 0x55c11f09f390 2024-08-30T09:09:00.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.920+0000 7fe4177fe640 1 -- 172.21.15.42:0/360054660 <== mon.2 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe430017040 con 0x55c11f09f390 2024-08-30T09:09:00.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.920+0000 7fe4177fe640 1 -- 172.21.15.42:0/360054660 <== mon.2 v2:172.21.15.167:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7fe43002a740 con 0x55c11f09f390 2024-08-30T09:09:00.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.920+0000 7fe44492f740 10 client.26582 fetch_fsmap finished waiting for FSMap version 37 2024-08-30T09:09:00.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.920+0000 7fe44492f740 10 client.26582 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-30T09:09:00.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.920+0000 7fe44492f740 20 client.26582 populate_metadata read hostname 'smithi042' 2024-08-30T09:09:00.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.920+0000 7fe44492f740 1 --2- 172.21.15.42:0/360054660 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55c11f4ee7c0 0x55c11f4f0be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:00.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.920+0000 7fe44492f740 4 client.26582 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-30T09:09:00.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.920+0000 7fe44492f740 1 -- 172.21.15.42:0/360054660 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7fe4280dd740 con 0x55c11f4ee7c0 2024-08-30T09:09:00.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.921+0000 7fe437fff640 1 --2- 172.21.15.42:0/360054660 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55c11f4ee7c0 0x55c11f4f0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:00.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.921+0000 7fe437fff640 1 --2- 172.21.15.42:0/360054660 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55c11f4ee7c0 0x55c11f4f0be0 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=333bdc76f11f5740 server_cookie=aa44e4fd791aa59f in_seq=0 out_seq=0 2024-08-30T09:09:00.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.921+0000 7fe4177fe640 10 client.26582 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:09:00.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.922+0000 7fe4177fe640 1 -- 172.21.15.42:0/360054660 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+2930 (crc 0 0 0) 0x7fe4280dd740 con 0x55c11f4ee7c0 2024-08-30T09:09:00.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.922+0000 7fe4177fe640 10 client.26582 handle_command_reply: tid=0 2024-08-30T09:09:00.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.922+0000 7fe44492f740 1 client.26582 shutdown 2024-08-30T09:09:00.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.922+0000 7fe44492f740 1 -- 172.21.15.42:0/360054660 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c11f09f390 msgr2=0x55c11f4794c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:09:00.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.922+0000 7fe44492f740 1 --2- 172.21.15.42:0/360054660 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c11f09f390 0x55c11f4794c0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fe430005f40 tx=0x7fe430004ab0 comp rx=0 tx=0).stop 2024-08-30T09:09:00.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.922+0000 7fe44492f740 1 -- 172.21.15.42:0/360054660 shutdown_connections 2024-08-30T09:09:00.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.922+0000 7fe44492f740 1 --2- 172.21.15.42:0/360054660 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55c11f4ee7c0 0x55c11f4f0be0 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:00.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.922+0000 7fe4177fe640 0 client.26582 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:09:00.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.922+0000 7fe44492f740 1 --2- 172.21.15.42:0/360054660 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c11f0c0070 0x55c11f4873c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:00.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.922+0000 7fe44492f740 1 --2- 172.21.15.42:0/360054660 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c11f09f390 0x55c11f4794c0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:00.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.922+0000 7fe44492f740 1 --2- 172.21.15.42:0/360054660 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c11f07e9f0 0x55c11f478f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:00.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.922+0000 7fe44492f740 1 -- 172.21.15.42:0/360054660 >> 172.21.15.42:0/360054660 conn(0x55c11f0b4460 msgr2=0x55c11f4662f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:09:00.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.922+0000 7fe44492f740 1 -- 172.21.15.42:0/360054660 shutdown_connections 2024-08-30T09:09:00.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.922+0000 7fe44492f740 1 -- 172.21.15.42:0/360054660 wait complete. 2024-08-30T09:09:00.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.923+0000 7fe44492f740 20 client.26582 trim_cache size 0 max 16384 2024-08-30T09:09:00.928 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-08-30T09:09:00.928 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "no active scrubs running", 2024-08-30T09:09:00.929 INFO:teuthology.orchestra.run.smithi042.stdout: "scrubs": {}, 2024-08-30T09:09:00.929 INFO:teuthology.orchestra.run.smithi042.stdout: "scrub_stats": { 2024-08-30T09:09:00.929 INFO:teuthology.orchestra.run.smithi042.stdout: "99a29ec1-9455-4ae4-854a-398b32896675": { 2024-08-30T09:09:00.929 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:08:24.923176+0000", 2024-08-30T09:09:00.929 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:09:00.929 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:09:00.929 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:09:00.929 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:09:00.929 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:09:00.929 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:09:00.929 INFO:teuthology.orchestra.run.smithi042.stdout: "598d8a75-f1cc-4388-80bc-a408bdb4eaac": { 2024-08-30T09:09:00.929 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:07:29.721122+0000", 2024-08-30T09:09:00.929 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:09:00.929 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:09:00.929 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:09:00.930 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:09:00.930 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:09:00.930 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:09:00.930 INFO:teuthology.orchestra.run.smithi042.stdout: "665ec740-7254-49cb-84ef-deff71c461b0": { 2024-08-30T09:09:00.930 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:06:37.144035+0000", 2024-08-30T09:09:00.930 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:09:00.930 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:09:00.930 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:09:00.930 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:09:00.930 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:09:00.930 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:09:00.930 INFO:teuthology.orchestra.run.smithi042.stdout: "1af4252a-4a99-47e9-bbf0-becf5e573dd5": { 2024-08-30T09:09:00.930 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:05:48.565881+0000", 2024-08-30T09:09:00.930 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:09:00.930 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:09:00.931 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:09:00.931 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:09:00.931 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:09:00.931 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:09:00.931 INFO:teuthology.orchestra.run.smithi042.stdout: "347d05ed-4bd3-4590-b099-314f719aab39": { 2024-08-30T09:09:00.931 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:05:00.439721+0000", 2024-08-30T09:09:00.931 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:09:00.931 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:09:00.931 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:09:00.931 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:09:00.931 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:09:00.931 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:09:00.931 INFO:teuthology.orchestra.run.smithi042.stdout: "88cd7478-2b20-47de-ba51-256479749232": { 2024-08-30T09:09:00.931 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:04:08.922625+0000", 2024-08-30T09:09:00.931 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:09:00.932 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:09:00.932 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:09:00.932 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:09:00.932 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:09:00.932 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:09:00.932 INFO:teuthology.orchestra.run.smithi042.stdout: "749f4867-0b70-49e5-af2d-8362a240ebc3": { 2024-08-30T09:09:00.932 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:03:16.386554+0000", 2024-08-30T09:09:00.932 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:09:00.932 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:09:00.932 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:09:00.932 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:09:00.932 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:09:00.932 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:09:00.933 INFO:teuthology.orchestra.run.smithi042.stdout: "85134e4b-ab4a-4885-b07e-a3ad01eaf13d": { 2024-08-30T09:09:00.933 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:01:27.738462+0000", 2024-08-30T09:09:00.933 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:09:00.933 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:09:00.933 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:09:00.933 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:09:00.933 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:09:00.933 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:09:00.933 INFO:teuthology.orchestra.run.smithi042.stdout: "c827ad7a-44ee-4e1d-9bb3-51fb7ab00c73": { 2024-08-30T09:09:00.933 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:02:21.686517+0000", 2024-08-30T09:09:00.933 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:09:00.933 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:09:00.933 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:09:00.933 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:09:00.933 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:09:00.934 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:09:00.934 INFO:teuthology.orchestra.run.smithi042.stdout: "5e2ed104-0625-478a-a78b-584e7f4c2817": { 2024-08-30T09:09:00.934 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:00:33.928400+0000", 2024-08-30T09:09:00.934 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:09:00.934 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:09:00.934 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:09:00.934 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:09:00.934 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:09:00.934 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:09:00.934 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:09:00.934 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-08-30T09:09:00.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.924+0000 7fe43e3ba640 1 -- 172.21.15.42:0/1668949669 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fe4200b4d30 msgr2=0x7fe4200b7240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:09:00.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.924+0000 7fe43e3ba640 1 --2- 172.21.15.42:0/1668949669 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fe4200b4d30 0x7fe4200b7240 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fe438071340 tx=0x7fe4300107a0 comp rx=0 tx=0).stop 2024-08-30T09:09:00.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.924+0000 7fe43e3ba640 1 -- 172.21.15.42:0/1668949669 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe43810a780 msgr2=0x7fe43811c180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:09:00.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.924+0000 7fe43e3ba640 1 --2- 172.21.15.42:0/1668949669 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe43810a780 0x7fe43811c180 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fe42803ba60 tx=0x7fe4280056d0 comp rx=0 tx=0).stop 2024-08-30T09:09:00.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.924+0000 7fe43e3ba640 1 -- 172.21.15.42:0/1668949669 shutdown_connections 2024-08-30T09:09:00.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.924+0000 7fe43e3ba640 1 --2- 172.21.15.42:0/1668949669 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fe4200b4d30 0x7fe4200b7240 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:00.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.924+0000 7fe43e3ba640 1 --2- 172.21.15.42:0/1668949669 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe43811c9d0 0x7fe438118a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:00.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.924+0000 7fe43e3ba640 1 --2- 172.21.15.42:0/1668949669 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe438111ce0 0x7fe438118460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:00.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.924+0000 7fe43e3ba640 1 --2- 172.21.15.42:0/1668949669 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe43810a780 0x7fe43811c180 secure :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fe42803ba60 tx=0x7fe4280056d0 comp rx=0 tx=0).stop 2024-08-30T09:09:00.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.925+0000 7fe43e3ba640 1 -- 172.21.15.42:0/1668949669 >> 172.21.15.42:0/1668949669 conn(0x7fe438070c60 msgr2=0x7fe43810b7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:09:00.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.925+0000 7fe43e3ba640 1 -- 172.21.15.42:0/1668949669 shutdown_connections 2024-08-30T09:09:00.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:00.925+0000 7fe43e3ba640 1 -- 172.21.15.42:0/1668949669 wait complete. 2024-08-30T09:09:00.953 INFO:teuthology.orchestra.run.smithi042.stderr:progress: 495.0 s, 610.4 tps, lat 161.725 ms stddev 176.684 2024-08-30T09:09:01.752 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-08-30T09:09:01.752 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph fs dump --format=json 2024-08-30T09:09:02.058 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.053+0000 7f893cd1a640 1 -- 172.21.15.42:0/3825361894 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f893810f5d0 msgr2=0x7f893810fad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:09:02.058 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.053+0000 7f893cd1a640 1 --2- 172.21.15.42:0/3825361894 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f893810f5d0 0x7f893810fad0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f8930009be0 tx=0x7f893002a6f0 comp rx=0 tx=0).stop 2024-08-30T09:09:02.058 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.053+0000 7f893cd1a640 1 -- 172.21.15.42:0/3825361894 shutdown_connections 2024-08-30T09:09:02.058 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.053+0000 7f893cd1a640 1 --2- 172.21.15.42:0/3825361894 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f893810f5d0 0x7f893810fad0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:02.058 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.053+0000 7f893cd1a640 1 --2- 172.21.15.42:0/3825361894 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f893810cac0 0x7f893810eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:02.059 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.053+0000 7f893cd1a640 1 --2- 172.21.15.42:0/3825361894 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f893810a160 0x7f893810c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:02.059 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.053+0000 7f893cd1a640 1 -- 172.21.15.42:0/3825361894 >> 172.21.15.42:0/3825361894 conn(0x7f8938070410 msgr2=0x7f89380728c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:09:02.067 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.062+0000 7f893cd1a640 1 -- 172.21.15.42:0/3825361894 shutdown_connections 2024-08-30T09:09:02.068 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.064+0000 7f893cd1a640 1 -- 172.21.15.42:0/3825361894 wait complete. 2024-08-30T09:09:02.073 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.067+0000 7f893cd1a640 1 Processor -- start 2024-08-30T09:09:02.073 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.067+0000 7f893cd1a640 1 -- start start 2024-08-30T09:09:02.073 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.067+0000 7f893cd1a640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f893810a160 0x7f89381a8970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:02.073 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.067+0000 7f893cd1a640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f893810cac0 0x7f89381a8fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:02.073 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.067+0000 7f893cd1a640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f893810f5d0 0x7f89381a9b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:02.073 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.067+0000 7f893cd1a640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f893811dd20 con 0x7f893810cac0 2024-08-30T09:09:02.073 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.067+0000 7f893cd1a640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f893811dba0 con 0x7f893810f5d0 2024-08-30T09:09:02.073 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.067+0000 7f893cd1a640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f893811dea0 con 0x7f893810a160 2024-08-30T09:09:02.073 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.068+0000 7f893659c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f893810a160 0x7f89381a8970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:02.073 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.068+0000 7f893659c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f893810a160 0x7f89381a8970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:35698/0 (socket says 172.21.15.42:35698) 2024-08-30T09:09:02.073 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.068+0000 7f893659c640 1 -- 172.21.15.42:0/3019472078 learned_addr learned my addr 172.21.15.42:0/3019472078 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:09:02.074 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.069+0000 7f8936d9d640 1 --2- 172.21.15.42:0/3019472078 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f893810f5d0 0x7f89381a9b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:02.076 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.070+0000 7f893659c640 1 -- 172.21.15.42:0/3019472078 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f893810f5d0 msgr2=0x7f89381a9b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:09:02.076 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.070+0000 7f893659c640 1 --2- 172.21.15.42:0/3019472078 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f893810f5d0 0x7f89381a9b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:02.076 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.070+0000 7f893659c640 1 -- 172.21.15.42:0/3019472078 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f893810cac0 msgr2=0x7f89381a8fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:09:02.076 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.070+0000 7f893659c640 1 --2- 172.21.15.42:0/3019472078 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f893810cac0 0x7f89381a8fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:02.076 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.071+0000 7f893659c640 1 -- 172.21.15.42:0/3019472078 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89381aa3b0 con 0x7f893810a160 2024-08-30T09:09:02.079 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.073+0000 7f893659c640 1 --2- 172.21.15.42:0/3019472078 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f893810a160 0x7f89381a8970 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f892800e9c0 tx=0x7f892800ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:09:02.086 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.077+0000 7f89277fe640 1 -- 172.21.15.42:0/3019472078 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f892800cc40 con 0x7f893810a160 2024-08-30T09:09:02.086 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.077+0000 7f893cd1a640 1 -- 172.21.15.42:0/3019472078 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89381aa6a0 con 0x7f893810a160 2024-08-30T09:09:02.086 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.077+0000 7f893cd1a640 1 -- 172.21.15.42:0/3019472078 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f893807b5a0 con 0x7f893810a160 2024-08-30T09:09:02.086 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.077+0000 7f89277fe640 1 -- 172.21.15.42:0/3019472078 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f892800cde0 con 0x7f893810a160 2024-08-30T09:09:02.086 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.077+0000 7f89277fe640 1 -- 172.21.15.42:0/3019472078 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8928010430 con 0x7f893810a160 2024-08-30T09:09:02.086 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.082+0000 7f8943c7d740 1 Processor -- start 2024-08-30T09:09:02.086 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.082+0000 7f8943c7d740 1 -- start start 2024-08-30T09:09:02.092 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.082+0000 7f8943c7d740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562cfedc81b0 0x562cff16e160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:02.092 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.082+0000 7f8943c7d740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562cfed9f2f0 0x562cff16e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:02.092 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.082+0000 7f8943c7d740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562cfeda74d0 0x562cff16f320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:02.092 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.082+0000 7f8943c7d740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x562cff0e1000 con 0x562cfedc81b0 2024-08-30T09:09:02.092 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.082+0000 7f8943c7d740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x562cff0d0590 con 0x562cfed9f2f0 2024-08-30T09:09:02.092 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.082+0000 7f8943c7d740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x562cff0d4540 con 0x562cfeda74d0 2024-08-30T09:09:02.092 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.082+0000 7f8935d9b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562cfeda74d0 0x562cff16f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:02.092 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.082+0000 7f8935d9b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562cfeda74d0 0x562cff16f320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:35710/0 (socket says 172.21.15.42:35710) 2024-08-30T09:09:02.092 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.082+0000 7f8935d9b640 1 -- 172.21.15.42:0/892157912 learned_addr learned my addr 172.21.15.42:0/892157912 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:09:02.092 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.083+0000 7f8935d9b640 1 -- 172.21.15.42:0/892157912 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562cfed9f2f0 msgr2=0x562cff16e7d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-30T09:09:02.092 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.083+0000 7f8935d9b640 1 --2- 172.21.15.42:0/892157912 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562cfed9f2f0 0x562cff16e7d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:02.093 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.083+0000 7f8935d9b640 1 -- 172.21.15.42:0/892157912 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562cfedc81b0 msgr2=0x562cff16e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:09:02.093 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.083+0000 7f8935d9b640 1 --2- 172.21.15.42:0/892157912 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562cfedc81b0 0x562cff16e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:02.093 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.083+0000 7f8935d9b640 1 -- 172.21.15.42:0/892157912 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562cfee33790 con 0x562cfeda74d0 2024-08-30T09:09:02.093 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.084+0000 7f8935d9b640 1 --2- 172.21.15.42:0/892157912 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562cfeda74d0 0x562cff16f320 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f892c009c40 tx=0x7f892c00e720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ea5e5edf89d78da5 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:09:02.097 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.094+0000 7f89277fe640 1 -- 172.21.15.42:0/3019472078 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7f8928004ee0 con 0x7f893810a160 2024-08-30T09:09:02.102 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.095+0000 7f89277fe640 1 --2- 172.21.15.42:0/3019472078 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f89200b4d40 0x7f89200b7250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:02.102 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.095+0000 7f89277fe640 1 -- 172.21.15.42:0/3019472078 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f89280e5640 con 0x7f893810a160 2024-08-30T09:09:02.102 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.096+0000 7f8924ff9640 1 -- 172.21.15.42:0/892157912 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f892c005020 con 0x562cfeda74d0 2024-08-30T09:09:02.102 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.096+0000 7f8924ff9640 1 -- 172.21.15.42:0/892157912 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f892c0047a0 con 0x562cfeda74d0 2024-08-30T09:09:02.102 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.096+0000 7f8924ff9640 1 -- 172.21.15.42:0/892157912 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f892c010720 con 0x562cfeda74d0 2024-08-30T09:09:02.102 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.098+0000 7f8943c7d740 1 -- 172.21.15.42:0/892157912 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562cfeda74d0 msgr2=0x562cff16f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:09:02.102 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.098+0000 7f8943c7d740 1 --2- 172.21.15.42:0/892157912 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562cfeda74d0 0x562cff16f320 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f892c009c40 tx=0x7f892c00e720 comp rx=0 tx=0).stop 2024-08-30T09:09:02.102 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.098+0000 7f8943c7d740 1 -- 172.21.15.42:0/892157912 shutdown_connections 2024-08-30T09:09:02.102 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.098+0000 7f8943c7d740 1 --2- 172.21.15.42:0/892157912 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562cfeda74d0 0x562cff16f320 secure :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f892c009c40 tx=0x7f892c00e720 comp rx=0 tx=0).stop 2024-08-30T09:09:02.102 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.098+0000 7f8943c7d740 1 --2- 172.21.15.42:0/892157912 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562cfed9f2f0 0x562cff16e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:02.103 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.098+0000 7f8943c7d740 1 --2- 172.21.15.42:0/892157912 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562cfedc81b0 0x562cff16e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:02.103 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.098+0000 7f8943c7d740 1 -- 172.21.15.42:0/892157912 >> 172.21.15.42:0/892157912 conn(0x562cfedbc300 msgr2=0x562cff169bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:09:02.104 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.101+0000 7f8936d9d640 1 --2- 172.21.15.42:0/3019472078 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f89200b4d40 0x7f89200b7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:02.107 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.104+0000 7f8936d9d640 1 --2- 172.21.15.42:0/3019472078 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f89200b4d40 0x7f89200b7250 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f89300099a0 tx=0x7f8930004a60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:09:02.107 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.104+0000 7f8943c7d740 1 -- 172.21.15.42:0/892157912 shutdown_connections 2024-08-30T09:09:02.115 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.108+0000 7f8943c7d740 1 -- 172.21.15.42:0/892157912 wait complete. 2024-08-30T09:09:02.115 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.108+0000 7f8943c7d740 1 -- start start 2024-08-30T09:09:02.116 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.108+0000 7f8943c7d740 1 Processor -- start 2024-08-30T09:09:02.116 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.109+0000 7f8943c7d740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562cfed9f2f0 0x562cff177e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:02.116 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.109+0000 7f8943c7d740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562cfedc81b0 0x562cff1784d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:02.116 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.109+0000 7f8943c7d740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562cfed86d80 0x562cff178fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:02.116 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.109+0000 7f8943c7d740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x562cff009ab0 con 0x562cfedc81b0 2024-08-30T09:09:02.116 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.109+0000 7f8943c7d740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x562cff057680 con 0x562cfed86d80 2024-08-30T09:09:02.116 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.109+0000 7f8943c7d740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x562cff16fe40 con 0x562cfed9f2f0 2024-08-30T09:09:02.116 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.109+0000 7f8943c7d740 10 client.0 _ll_register_callbacks cb 0x562cfeddfa00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:09:02.116 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.109+0000 7f8936d9d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562cfed86d80 0x562cff178fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:02.116 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.109+0000 7f8936d9d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562cfed86d80 0x562cff178fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:42900/0 (socket says 172.21.15.42:42900) 2024-08-30T09:09:02.116 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.109+0000 7f8936d9d640 1 -- 172.21.15.42:0/2173202277 learned_addr learned my addr 172.21.15.42:0/2173202277 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:09:02.116 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.109+0000 7f893659c640 1 --2- 172.21.15.42:0/2173202277 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562cfed9f2f0 0x562cff177e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:02.116 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.110+0000 7f893659c640 1 -- 172.21.15.42:0/2173202277 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562cfed86d80 msgr2=0x562cff178fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:09:02.116 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.110+0000 7f893659c640 1 --2- 172.21.15.42:0/2173202277 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562cfed86d80 0x562cff178fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:02.117 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.110+0000 7f893659c640 1 -- 172.21.15.42:0/2173202277 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562cfedc81b0 msgr2=0x562cff1784d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-30T09:09:02.117 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.110+0000 7f893659c640 1 --2- 172.21.15.42:0/2173202277 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562cfedc81b0 0x562cff1784d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:02.117 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.110+0000 7f893659c640 1 -- 172.21.15.42:0/2173202277 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562cfeeac620 con 0x562cfed9f2f0 2024-08-30T09:09:02.117 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.110+0000 7f893659c640 1 --2- 172.21.15.42:0/2173202277 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562cfed9f2f0 0x562cff177e10 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f892800c6a0 tx=0x7f89280ab850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1bdadb908b9ec77e server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:09:02.117 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.110+0000 7f891dffb640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-08-30T09:09:02.117 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.111+0000 7f891dffb640 1 -- 172.21.15.42:0/2173202277 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89280d2020 con 0x562cfed9f2f0 2024-08-30T09:09:02.117 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.111+0000 7f891dffb640 1 -- 172.21.15.42:0/2173202277 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f89280e5320 con 0x562cfed9f2f0 2024-08-30T09:09:02.117 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.111+0000 7f8943c7d740 1 -- 172.21.15.42:0/2173202277 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f892c0047a0 con 0x562cfed9f2f0 2024-08-30T09:09:02.117 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.111+0000 7f891dffb640 1 -- 172.21.15.42:0/2173202277 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f89280c6050 con 0x562cfed9f2f0 2024-08-30T09:09:02.119 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.112+0000 7f8943c7d740 10 client.26569 fetch_fsmap learned FSMap version 37 2024-08-30T09:09:02.119 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.112+0000 7f8943c7d740 1 -- 172.21.15.42:0/2173202277 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x562cff181f90 con 0x562cfed9f2f0 2024-08-30T09:09:02.122 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.117+0000 7f891dffb640 1 -- 172.21.15.42:0/2173202277 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f892c0047a0 con 0x562cfed9f2f0 2024-08-30T09:09:02.123 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.117+0000 7f891dffb640 1 -- 172.21.15.42:0/2173202277 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7f89280abee0 con 0x562cfed9f2f0 2024-08-30T09:09:02.124 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.121+0000 7f8943c7d740 10 client.26569 fetch_fsmap finished waiting for FSMap version 37 2024-08-30T09:09:02.124 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.121+0000 7f8943c7d740 10 client.26569 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-30T09:09:02.124 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.121+0000 7f8943c7d740 20 client.26569 populate_metadata read hostname 'smithi042' 2024-08-30T09:09:02.124 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.121+0000 7f8943c7d740 1 --2- 172.21.15.42:0/2173202277 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x562cfed5c1c0 0x562cff185710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:02.124 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.121+0000 7f8943c7d740 4 client.26569 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-30T09:09:02.125 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.121+0000 7f8943c7d740 1 -- 172.21.15.42:0/2173202277 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562cff0e2f50 con 0x562cfed5c1c0 2024-08-30T09:09:02.125 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.121+0000 7f8935d9b640 1 --2- 172.21.15.42:0/2173202277 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x562cfed5c1c0 0x562cff185710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:02.129 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.126+0000 7f8935d9b640 1 --2- 172.21.15.42:0/2173202277 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x562cfed5c1c0 0x562cff185710 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=388a62b0288195d1 server_cookie=22fda073666be16e in_seq=0 out_seq=0 2024-08-30T09:09:02.129 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.126+0000 7f891dffb640 10 client.26569 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:09:02.132 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.129+0000 7f891dffb640 1 -- 172.21.15.42:0/2173202277 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x562cff0e2f50 con 0x562cfed5c1c0 2024-08-30T09:09:02.132 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.129+0000 7f891dffb640 10 client.26569 handle_command_reply: tid=0 2024-08-30T09:09:02.133 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.129+0000 7f8943c7d740 1 client.26569 shutdown 2024-08-30T09:09:02.138 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.133+0000 7f8943c7d740 1 -- 172.21.15.42:0/2173202277 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562cfed9f2f0 msgr2=0x562cff177e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:09:02.138 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.133+0000 7f8943c7d740 1 --2- 172.21.15.42:0/2173202277 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562cfed9f2f0 0x562cff177e10 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f892800c6a0 tx=0x7f89280ab850 comp rx=0 tx=0).stop 2024-08-30T09:09:02.143 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.139+0000 7f8943c7d740 1 -- 172.21.15.42:0/2173202277 shutdown_connections 2024-08-30T09:09:02.143 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.139+0000 7f8943c7d740 1 --2- 172.21.15.42:0/2173202277 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x562cfed5c1c0 0x562cff185710 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:02.143 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.139+0000 7f8943c7d740 1 --2- 172.21.15.42:0/2173202277 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562cfed86d80 0x562cff178fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:02.143 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.139+0000 7f8943c7d740 1 --2- 172.21.15.42:0/2173202277 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562cfedc81b0 0x562cff1784d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:02.143 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.139+0000 7f8943c7d740 1 --2- 172.21.15.42:0/2173202277 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562cfed9f2f0 0x562cff177e10 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:02.143 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.140+0000 7f8943c7d740 1 -- 172.21.15.42:0/2173202277 >> 172.21.15.42:0/2173202277 conn(0x562cfedbc300 msgr2=0x562cff168150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:09:02.143 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.140+0000 7f891dffb640 0 client.26569 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:09:02.147 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.144+0000 7f8943c7d740 1 -- 172.21.15.42:0/2173202277 shutdown_connections 2024-08-30T09:09:02.150 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.146+0000 7f8943c7d740 1 -- 172.21.15.42:0/2173202277 wait complete. 2024-08-30T09:09:02.150 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.147+0000 7f8943c7d740 20 client.26569 trim_cache size 0 max 16384 2024-08-30T09:09:02.210 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.207+0000 7f8943c7d740 1 Processor -- start 2024-08-30T09:09:02.214 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.211+0000 7f8943c7d740 1 -- start start 2024-08-30T09:09:02.214 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.211+0000 7f8943c7d740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562cfed86d80 0x562cff185650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:02.214 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.211+0000 7f8943c7d740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562cfed9f2f0 0x562cff172b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:02.214 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.211+0000 7f8943c7d740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562cfedc81b0 0x562cff1730d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:02.214 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.211+0000 7f8943c7d740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f8918012510 con 0x562cfed9f2f0 2024-08-30T09:09:02.214 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.211+0000 7f8943c7d740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f891800fde0 con 0x562cfedc81b0 2024-08-30T09:09:02.215 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.211+0000 7f8943c7d740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f891800dc30 con 0x562cfed86d80 2024-08-30T09:09:02.217 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.214+0000 7f893659c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562cfed86d80 0x562cff185650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:02.217 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.214+0000 7f893659c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562cfed86d80 0x562cff185650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:35726/0 (socket says 172.21.15.42:35726) 2024-08-30T09:09:02.217 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.214+0000 7f893659c640 1 -- 172.21.15.42:0/3856117576 learned_addr learned my addr 172.21.15.42:0/3856117576 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:09:02.217 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.215+0000 7f8936d9d640 1 --2- 172.21.15.42:0/3856117576 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562cfedc81b0 0x562cff1730d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:02.227 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.217+0000 7f893659c640 1 -- 172.21.15.42:0/3856117576 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562cfedc81b0 msgr2=0x562cff1730d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:09:02.227 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.217+0000 7f893659c640 1 --2- 172.21.15.42:0/3856117576 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562cfedc81b0 0x562cff1730d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:02.227 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.217+0000 7f893659c640 1 -- 172.21.15.42:0/3856117576 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562cfed9f2f0 msgr2=0x562cff172b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:09:02.227 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.217+0000 7f893659c640 1 --2- 172.21.15.42:0/3856117576 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562cfed9f2f0 0x562cff172b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:02.227 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.217+0000 7f893659c640 1 -- 172.21.15.42:0/3856117576 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562cff168860 con 0x562cfed86d80 2024-08-30T09:09:02.227 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.223+0000 7f893659c640 1 --2- 172.21.15.42:0/3856117576 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562cfed86d80 0x562cff185650 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f8928000c00 tx=0x7f89280a7c70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e9746127095ecbe0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:09:02.227 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.224+0000 7f891ffff640 1 -- 172.21.15.42:0/3856117576 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89280d2040 con 0x562cfed86d80 2024-08-30T09:09:02.227 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.224+0000 7f891ffff640 1 -- 172.21.15.42:0/3856117576 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f89280cb020 con 0x562cfed86d80 2024-08-30T09:09:02.228 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.224+0000 7f891ffff640 1 -- 172.21.15.42:0/3856117576 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89280b4fa0 con 0x562cfed86d80 2024-08-30T09:09:02.228 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.225+0000 7f8943c7d740 1 -- 172.21.15.42:0/3856117576 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562cfed86d80 msgr2=0x562cff185650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:09:02.228 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.225+0000 7f8943c7d740 1 --2- 172.21.15.42:0/3856117576 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562cfed86d80 0x562cff185650 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f8928000c00 tx=0x7f89280a7c70 comp rx=0 tx=0).stop 2024-08-30T09:09:02.231 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.228+0000 7f8943c7d740 1 -- 172.21.15.42:0/3856117576 shutdown_connections 2024-08-30T09:09:02.231 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.228+0000 7f8943c7d740 1 --2- 172.21.15.42:0/3856117576 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562cfedc81b0 0x562cff1730d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:02.231 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.228+0000 7f8943c7d740 1 --2- 172.21.15.42:0/3856117576 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562cfed9f2f0 0x562cff172b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:02.231 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.228+0000 7f8943c7d740 1 --2- 172.21.15.42:0/3856117576 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562cfed86d80 0x562cff185650 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:02.231 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.228+0000 7f8943c7d740 1 -- 172.21.15.42:0/3856117576 >> 172.21.15.42:0/3856117576 conn(0x562cfedbc300 msgr2=0x562cff168150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:09:02.237 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.235+0000 7f8943c7d740 1 -- 172.21.15.42:0/3856117576 shutdown_connections 2024-08-30T09:09:02.239 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.236+0000 7f8943c7d740 1 -- 172.21.15.42:0/3856117576 wait complete. 2024-08-30T09:09:02.239 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.236+0000 7f8943c7d740 1 -- start start 2024-08-30T09:09:02.242 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.236+0000 7f8943c7d740 1 Processor -- start 2024-08-30T09:09:02.243 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.240+0000 7f8943c7d740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562cfed86d80 0x562cff17f800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:02.244 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.240+0000 7f8943c7d740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562cfed9f2f0 0x562cff17fec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:02.244 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.240+0000 7f8943c7d740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562cfedc81b0 0x562cff17e650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:02.244 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.240+0000 7f8943c7d740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f891800e2a0 con 0x562cfedc81b0 2024-08-30T09:09:02.244 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.240+0000 7f8943c7d740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f891800f710 con 0x562cfed9f2f0 2024-08-30T09:09:02.244 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.240+0000 7f8943c7d740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f891800e970 con 0x562cfed86d80 2024-08-30T09:09:02.244 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.240+0000 7f8943c7d740 10 client.0 _ll_register_callbacks cb 0x7f89280a7b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:09:02.244 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.241+0000 7f893659c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562cfed86d80 0x562cff17f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:02.244 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.241+0000 7f8935d9b640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562cfed9f2f0 0x562cff17fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:02.244 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.241+0000 7f8935d9b640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562cfed9f2f0 0x562cff17fec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:42918/0 (socket says 172.21.15.42:42918) 2024-08-30T09:09:02.244 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.241+0000 7f8935d9b640 1 -- 172.21.15.42:0/313904722 learned_addr learned my addr 172.21.15.42:0/313904722 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:09:02.248 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.245+0000 7f893659c640 1 -- 172.21.15.42:0/313904722 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562cfed9f2f0 msgr2=0x562cff17fec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:09:02.248 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.245+0000 7f893659c640 1 --2- 172.21.15.42:0/313904722 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562cfed9f2f0 0x562cff17fec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:02.248 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.245+0000 7f893659c640 1 -- 172.21.15.42:0/313904722 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562cfedc81b0 msgr2=0x562cff17e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:09:02.248 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.245+0000 7f893659c640 1 --2- 172.21.15.42:0/313904722 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562cfedc81b0 0x562cff17e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:02.248 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.245+0000 7f893659c640 1 -- 172.21.15.42:0/313904722 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562cff17ede0 con 0x562cfed86d80 2024-08-30T09:09:02.248 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.245+0000 7f893659c640 1 --2- 172.21.15.42:0/313904722 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562cfed86d80 0x562cff17f800 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f89280abbc0 tx=0x7f89280b4ed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d5d4c57ab0927262 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:09:02.250 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.247+0000 7f891dffb640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-08-30T09:09:02.250 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.247+0000 7f891dffb640 1 -- 172.21.15.42:0/313904722 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89280d2040 con 0x562cfed86d80 2024-08-30T09:09:02.250 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.247+0000 7f891dffb640 1 -- 172.21.15.42:0/313904722 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f89280a8270 con 0x562cfed86d80 2024-08-30T09:09:02.252 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.249+0000 7f8943c7d740 1 -- 172.21.15.42:0/313904722 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f89280cb020 con 0x562cfed86d80 2024-08-30T09:09:02.255 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.253+0000 7f891dffb640 1 -- 172.21.15.42:0/313904722 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f89280ab3f0 con 0x562cfed86d80 2024-08-30T09:09:02.258 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.256+0000 7f8943c7d740 10 client.26575 fetch_fsmap learned FSMap version 37 2024-08-30T09:09:02.259 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.256+0000 7f8943c7d740 1 -- 172.21.15.42:0/313904722 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x562cff17dc20 con 0x562cfed86d80 2024-08-30T09:09:02.264 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.261+0000 7f891dffb640 1 -- 172.21.15.42:0/313904722 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f89280cb020 con 0x562cfed86d80 2024-08-30T09:09:02.264 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.261+0000 7f891dffb640 1 -- 172.21.15.42:0/313904722 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7f8928085ce0 con 0x562cfed86d80 2024-08-30T09:09:02.265 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.262+0000 7f8943c7d740 10 client.26575 fetch_fsmap finished waiting for FSMap version 37 2024-08-30T09:09:02.265 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.262+0000 7f8943c7d740 10 client.26575 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-30T09:09:02.265 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.262+0000 7f8943c7d740 20 client.26575 populate_metadata read hostname 'smithi042' 2024-08-30T09:09:02.265 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.262+0000 7f8943c7d740 1 --2- 172.21.15.42:0/313904722 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x562cff1f0460 0x562cff1f2880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:02.265 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.262+0000 7f8943c7d740 4 client.26575 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-30T09:09:02.265 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.262+0000 7f8943c7d740 1 -- 172.21.15.42:0/313904722 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562cff0de9f0 con 0x562cff1f0460 2024-08-30T09:09:02.265 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.262+0000 7f8935d9b640 1 --2- 172.21.15.42:0/313904722 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x562cff1f0460 0x562cff1f2880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:02.270 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.268+0000 7f8935d9b640 1 --2- 172.21.15.42:0/313904722 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x562cff1f0460 0x562cff1f2880 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=46efcfe8b8b76211 server_cookie=1e276085ddc6d951 in_seq=0 out_seq=0 2024-08-30T09:09:02.272 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.269+0000 7f891dffb640 10 client.26575 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:09:02.326 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T09:09:02.353 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.350+0000 7f891dffb640 1 -- 172.21.15.42:0/313904722 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562cff0de9f0 con 0x562cff1f0460 2024-08-30T09:09:02.353 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.350+0000 7f891dffb640 10 client.26575 handle_command_reply: tid=0 2024-08-30T09:09:02.353 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.350+0000 7f8943c7d740 1 client.26575 shutdown 2024-08-30T09:09:02.354 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.350+0000 7f8943c7d740 1 -- 172.21.15.42:0/313904722 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562cfed86d80 msgr2=0x562cff17f800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:09:02.354 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.350+0000 7f8943c7d740 1 --2- 172.21.15.42:0/313904722 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562cfed86d80 0x562cff17f800 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f89280abbc0 tx=0x7f89280b4ed0 comp rx=0 tx=0).stop 2024-08-30T09:09:02.354 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.351+0000 7f8943c7d740 1 -- 172.21.15.42:0/313904722 shutdown_connections 2024-08-30T09:09:02.354 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.351+0000 7f8943c7d740 1 --2- 172.21.15.42:0/313904722 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x562cff1f0460 0x562cff1f2880 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:02.355 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.351+0000 7f8943c7d740 1 --2- 172.21.15.42:0/313904722 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562cfedc81b0 0x562cff17e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:02.355 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.351+0000 7f8943c7d740 1 --2- 172.21.15.42:0/313904722 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562cfed9f2f0 0x562cff17fec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:02.355 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.351+0000 7f8943c7d740 1 --2- 172.21.15.42:0/313904722 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562cfed86d80 0x562cff17f800 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:02.355 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.351+0000 7f8943c7d740 1 -- 172.21.15.42:0/313904722 >> 172.21.15.42:0/313904722 conn(0x562cfedbc300 msgr2=0x562cff168150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:09:02.355 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.351+0000 7f891dffb640 0 client.26575 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:09:02.355 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.351+0000 7f8943c7d740 1 -- 172.21.15.42:0/313904722 shutdown_connections 2024-08-30T09:09:02.355 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.351+0000 7f8943c7d740 1 -- 172.21.15.42:0/313904722 wait complete. 2024-08-30T09:09:02.355 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.351+0000 7f8943c7d740 20 client.26575 trim_cache size 0 max 16384 2024-08-30T09:09:02.355 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-08-30T09:09:02.356 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2024-08-30T09:09:02.356 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-08-30T09:09:02.356 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-08-30T09:09:02.357 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.352+0000 7f893cd1a640 1 -- 172.21.15.42:0/3019472078 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f89200b4d40 msgr2=0x7f89200b7250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:09:02.357 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.352+0000 7f893cd1a640 1 --2- 172.21.15.42:0/3019472078 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f89200b4d40 0x7f89200b7250 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f89300099a0 tx=0x7f8930004a60 comp rx=0 tx=0).stop 2024-08-30T09:09:02.357 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.353+0000 7f893cd1a640 1 -- 172.21.15.42:0/3019472078 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f893810a160 msgr2=0x7f89381a8970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:09:02.357 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.353+0000 7f893cd1a640 1 --2- 172.21.15.42:0/3019472078 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f893810a160 0x7f89381a8970 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f892800e9c0 tx=0x7f892800ee80 comp rx=0 tx=0).stop 2024-08-30T09:09:02.358 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.354+0000 7f893cd1a640 1 -- 172.21.15.42:0/3019472078 shutdown_connections 2024-08-30T09:09:02.358 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.354+0000 7f893cd1a640 1 --2- 172.21.15.42:0/3019472078 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f89200b4d40 0x7f89200b7250 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:02.358 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.354+0000 7f893cd1a640 1 --2- 172.21.15.42:0/3019472078 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f893810f5d0 0x7f89381a9b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:02.358 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.354+0000 7f893cd1a640 1 --2- 172.21.15.42:0/3019472078 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f893810cac0 0x7f89381a8fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:02.358 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.354+0000 7f893cd1a640 1 --2- 172.21.15.42:0/3019472078 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f893810a160 0x7f89381a8970 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:02.358 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.354+0000 7f893cd1a640 1 -- 172.21.15.42:0/3019472078 >> 172.21.15.42:0/3019472078 conn(0x7f8938070410 msgr2=0x7f893810aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:09:02.358 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.354+0000 7f893cd1a640 1 -- 172.21.15.42:0/3019472078 shutdown_connections 2024-08-30T09:09:02.358 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:02.354+0000 7f893cd1a640 1 -- 172.21.15.42:0/3019472078 wait complete. 2024-08-30T09:09:02.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:09:02 smithi042 ceph-mon[18652]: pgmap v621: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.0 MiB/s wr, 551 op/s 2024-08-30T09:09:02.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:09:02 smithi190 ceph-mon[22001]: pgmap v621: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.0 MiB/s wr, 551 op/s 2024-08-30T09:09:02.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:09:02 smithi167 ceph-mon[22063]: pgmap v621: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.0 MiB/s wr, 551 op/s 2024-08-30T09:09:02.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:02.990+0000 7f41dda82640 1 -- 172.21.15.42:0/736865925 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f41d8108140 msgr2=0x7f41d8108550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:09:02.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:02.990+0000 7f41dda82640 1 --2- 172.21.15.42:0/736865925 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f41d8108140 0x7f41d8108550 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f41cc00fb10 tx=0x7f41cc02e570 comp rx=0 tx=0).stop 2024-08-30T09:09:02.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:02.990+0000 7f41dda82640 1 -- 172.21.15.42:0/736865925 shutdown_connections 2024-08-30T09:09:02.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:02.990+0000 7f41dda82640 1 --2- 172.21.15.42:0/736865925 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f41d8108a90 0x7f41d810f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:02.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:02.990+0000 7f41dda82640 1 --2- 172.21.15.42:0/736865925 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f41d8108140 0x7f41d8108550 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:02.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:02.990+0000 7f41dda82640 1 --2- 172.21.15.42:0/736865925 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f41d8102800 0x7f41d8102c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:02.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:02.990+0000 7f41dda82640 1 -- 172.21.15.42:0/736865925 >> 172.21.15.42:0/736865925 conn(0x7f41d80794d0 msgr2=0x7f41d80798e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:09:02.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:02.990+0000 7f41dda82640 1 -- 172.21.15.42:0/736865925 shutdown_connections 2024-08-30T09:09:02.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:02.991+0000 7f41dda82640 1 -- 172.21.15.42:0/736865925 wait complete. 2024-08-30T09:09:02.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:02.992+0000 7f41dda82640 1 Processor -- start 2024-08-30T09:09:02.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:02.992+0000 7f41dda82640 1 -- start start 2024-08-30T09:09:02.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:02.992+0000 7f41dda82640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f41d8102800 0x7f41d81980f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:02.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:02.993+0000 7f41dda82640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f41d8108140 0x7f41d8198690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:02.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:02.993+0000 7f41dda82640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f41d8108a90 0x7f41d819d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:02.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:02.993+0000 7f41dda82640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f41d8111640 con 0x7f41d8102800 2024-08-30T09:09:02.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:02.993+0000 7f41dda82640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f41d81114c0 con 0x7f41d8108a90 2024-08-30T09:09:02.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:02.993+0000 7f41dda82640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f41d81117c0 con 0x7f41d8108140 2024-08-30T09:09:02.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:02.993+0000 7f41d6ffd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f41d8108140 0x7f41d8198690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:02.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:02.993+0000 7f41d77fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f41d8102800 0x7f41d81980f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:02.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:02.993+0000 7f41d77fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f41d8102800 0x7f41d81980f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:54002/0 (socket says 172.21.15.42:54002) 2024-08-30T09:09:02.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:02.993+0000 7f41d77fe640 1 -- 172.21.15.42:0/2497428029 learned_addr learned my addr 172.21.15.42:0/2497428029 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:09:02.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:02.993+0000 7f41d7fff640 1 --2- 172.21.15.42:0/2497428029 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f41d8108a90 0x7f41d819d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:02.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:02.993+0000 7f41d7fff640 1 -- 172.21.15.42:0/2497428029 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f41d8108140 msgr2=0x7f41d8198690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:09:02.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:02.993+0000 7f41d7fff640 1 --2- 172.21.15.42:0/2497428029 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f41d8108140 0x7f41d8198690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:02.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:02.993+0000 7f41d7fff640 1 -- 172.21.15.42:0/2497428029 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f41d8102800 msgr2=0x7f41d81980f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:09:02.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:02.993+0000 7f41d7fff640 1 --2- 172.21.15.42:0/2497428029 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f41d8102800 0x7f41d81980f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:02.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:02.993+0000 7f41d7fff640 1 -- 172.21.15.42:0/2497428029 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41d819e060 con 0x7f41d8108a90 2024-08-30T09:09:02.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:02.993+0000 7f41d77fe640 1 --2- 172.21.15.42:0/2497428029 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f41d8102800 0x7f41d81980f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-30T09:09:02.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:02.994+0000 7f41d7fff640 1 --2- 172.21.15.42:0/2497428029 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f41d8108a90 0x7f41d819d930 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f41d000ec50 tx=0x7f41d000c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:09:02.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:02.994+0000 7f41d4ff9640 1 -- 172.21.15.42:0/2497428029 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41d0003b70 con 0x7f41d8108a90 2024-08-30T09:09:02.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:02.994+0000 7f41dda82640 1 -- 172.21.15.42:0/2497428029 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41d819e350 con 0x7f41d8108a90 2024-08-30T09:09:02.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:02.994+0000 7f41d4ff9640 1 -- 172.21.15.42:0/2497428029 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f41d0003d10 con 0x7f41d8108a90 2024-08-30T09:09:03.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:02.995+0000 7f41d4ff9640 1 -- 172.21.15.42:0/2497428029 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41d0010830 con 0x7f41d8108a90 2024-08-30T09:09:03.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:02.995+0000 7f41dda82640 1 -- 172.21.15.42:0/2497428029 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41d81a4ab0 con 0x7f41d8108a90 2024-08-30T09:09:03.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:02.996+0000 7f41dda82640 1 -- 172.21.15.42:0/2497428029 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41d8102c10 con 0x7f41d8108a90 2024-08-30T09:09:03.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:02.999+0000 7f41d4ff9640 1 -- 172.21.15.42:0/2497428029 <== mon.2 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7f41d0003ec0 con 0x7f41d8108a90 2024-08-30T09:09:03.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:03.000+0000 7f41d4ff9640 1 --2- 172.21.15.42:0/2497428029 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f41c00b4bb0 0x7f41c00b70c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:03.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:03.001+0000 7f41d4ff9640 1 -- 172.21.15.42:0/2497428029 <== mon.2 v2:172.21.15.167:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f41d00e5910 con 0x7f41d8108a90 2024-08-30T09:09:03.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:03.001+0000 7f41d77fe640 1 --2- 172.21.15.42:0/2497428029 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f41c00b4bb0 0x7f41c00b70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:03.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:03.001+0000 7f41d77fe640 1 --2- 172.21.15.42:0/2497428029 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f41c00b4bb0 0x7f41c00b70c0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f41c800bfc0 tx=0x7f41c800d040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:09:03.012 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:03.009+0000 7f41d4ff9640 1 -- 172.21.15.42:0/2497428029 <== mon.2 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f41d00ac690 con 0x7f41d8108a90 2024-08-30T09:09:03.425 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:03.421+0000 7f41dda82640 1 -- 172.21.15.42:0/2497428029 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f41d810a8c0 con 0x7f41d8108a90 2024-08-30T09:09:03.425 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:03.422+0000 7f41d4ff9640 1 -- 172.21.15.42:0/2497428029 <== mon.2 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+11918 (secure 0 0 0) 0x7f41d00abf90 con 0x7f41d8108a90 2024-08-30T09:09:03.426 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-08-30T09:09:03.426 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":37,"btime":"2024-08-30T09:08:44:258036+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14703,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/3188400000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":3188400000},{"type":"v1","addr":"172.21.15.42:6835","nonce":3188400000}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14715,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/3356537674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":3356537674},{"type":"v1","addr":"172.21.15.42:6837","nonce":3356537674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14766,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6839/2107785513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6838","nonce":2107785513},{"type":"v1","addr":"172.21.15.42:6839","nonce":2107785513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14778,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6841/2298438674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6840","nonce":2298438674},{"type":"v1","addr":"172.21.15.42:6841","nonce":2298438674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24553,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/543555570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":543555570},{"type":"v1","addr":"172.21.15.190:6833","nonce":543555570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24554,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/3907818941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":3907818941},{"type":"v1","addr":"172.21.15.167:6833","nonce":3907818941}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":37,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-30T08:58:03.055084+0000","modified":"2024-08-30T09:08:43.330738+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24629,"mds_1":24610,"mds_2":24599},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24572":{"gid":24572,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6835/234596845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":234596845},{"type":"v1","addr":"172.21.15.167:6835","nonce":234596845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24592":{"gid":24592,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6835/247905709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":247905709},{"type":"v1","addr":"172.21.15.190:6835","nonce":247905709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24598":{"gid":24598,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6837/863256349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6836","nonce":863256349},{"type":"v1","addr":"172.21.15.190:6837","nonce":863256349}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24599":{"gid":24599,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.167:6837/3011837856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":3011837856},{"type":"v1","addr":"172.21.15.167:6837","nonce":3011837856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24610":{"gid":24610,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.190:6839/719981110","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6838","nonce":719981110},{"type":"v1","addr":"172.21.15.190:6839","nonce":719981110}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24629":{"gid":24629,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.167:6839/3604645207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":3604645207},{"type":"v1","addr":"172.21.15.167:6839","nonce":3604645207}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24629,"qdb_cluster":[24599,24610,24629]},"id":1}]} 2024-08-30T09:09:03.426 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 37 2024-08-30T09:09:03.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:03.434+0000 7f41dda82640 1 -- 172.21.15.42:0/2497428029 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f41c00b4bb0 msgr2=0x7f41c00b70c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:09:03.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:03.434+0000 7f41dda82640 1 --2- 172.21.15.42:0/2497428029 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f41c00b4bb0 0x7f41c00b70c0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f41c800bfc0 tx=0x7f41c800d040 comp rx=0 tx=0).stop 2024-08-30T09:09:03.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:03.434+0000 7f41dda82640 1 -- 172.21.15.42:0/2497428029 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f41d8108a90 msgr2=0x7f41d819d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:09:03.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:03.434+0000 7f41dda82640 1 --2- 172.21.15.42:0/2497428029 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f41d8108a90 0x7f41d819d930 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f41d000ec50 tx=0x7f41d000c6a0 comp rx=0 tx=0).stop 2024-08-30T09:09:03.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:03.434+0000 7f41dda82640 1 -- 172.21.15.42:0/2497428029 shutdown_connections 2024-08-30T09:09:03.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:03.434+0000 7f41dda82640 1 --2- 172.21.15.42:0/2497428029 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f41c00b4bb0 0x7f41c00b70c0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:03.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:03.434+0000 7f41dda82640 1 --2- 172.21.15.42:0/2497428029 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f41d8108140 0x7f41d8198690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:03.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:03.434+0000 7f41dda82640 1 --2- 172.21.15.42:0/2497428029 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f41d8108a90 0x7f41d819d930 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:03.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:03.434+0000 7f41dda82640 1 --2- 172.21.15.42:0/2497428029 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f41d8102800 0x7f41d81980f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:03.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:03.434+0000 7f41dda82640 1 -- 172.21.15.42:0/2497428029 >> 172.21.15.42:0/2497428029 conn(0x7f41d80794d0 msgr2=0x7f41d81042b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:09:03.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:03.434+0000 7f41dda82640 1 -- 172.21.15.42:0/2497428029 shutdown_connections 2024-08-30T09:09:03.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:03.434+0000 7f41dda82640 1 -- 172.21.15.42:0/2497428029 wait complete. 2024-08-30T09:09:04.160 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-08-30T09:09:04.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:09:04 smithi042 ceph-mon[18652]: pgmap v622: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 5.9 MiB/s wr, 638 op/s 2024-08-30T09:09:04.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:09:04 smithi042 ceph-mon[18652]: from='client.? 172.21.15.42:0/2497428029' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-30T09:09:04.642 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T09:09:04.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:09:04 smithi190 ceph-mon[22001]: pgmap v622: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 5.9 MiB/s wr, 638 op/s 2024-08-30T09:09:04.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:09:04 smithi190 ceph-mon[22001]: from='client.? 172.21.15.42:0/2497428029' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-30T09:09:04.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:09:04 smithi167 ceph-mon[22063]: pgmap v622: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 5.9 MiB/s wr, 638 op/s 2024-08-30T09:09:04.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:09:04 smithi167 ceph-mon[22063]: from='client.? 172.21.15.42:0/2497428029' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-30T09:09:05.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.398+0000 7f3020209640 1 -- 172.21.15.42:0/3793067576 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f301810a780 msgr2=0x7f301810ab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:09:05.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.398+0000 7f3020209640 1 --2- 172.21.15.42:0/3793067576 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f301810a780 0x7f301810ab90 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f301401a910 tx=0x7f301403b450 comp rx=0 tx=0).stop 2024-08-30T09:09:05.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.398+0000 7f3020209640 1 -- 172.21.15.42:0/3793067576 shutdown_connections 2024-08-30T09:09:05.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.398+0000 7f3020209640 1 --2- 172.21.15.42:0/3793067576 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f301811c9d0 0x7f301811eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:05.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.398+0000 7f3020209640 1 --2- 172.21.15.42:0/3793067576 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f301810a780 0x7f301810ab90 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:05.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.398+0000 7f3020209640 1 --2- 172.21.15.42:0/3793067576 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3018111ce0 0x7f301810a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:05.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.398+0000 7f3020209640 1 -- 172.21.15.42:0/3793067576 >> 172.21.15.42:0/3793067576 conn(0x7f3018070c60 msgr2=0x7f3018071070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:09:05.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.404+0000 7f3020209640 1 -- 172.21.15.42:0/3793067576 shutdown_connections 2024-08-30T09:09:05.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.404+0000 7f3020209640 1 -- 172.21.15.42:0/3793067576 wait complete. 2024-08-30T09:09:05.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.405+0000 7f3020209640 1 Processor -- start 2024-08-30T09:09:05.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.405+0000 7f3020209640 1 -- start start 2024-08-30T09:09:05.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.405+0000 7f3020209640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f301810a780 0x7f301811c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:05.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.405+0000 7f3020209640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3018111ce0 0x7f3018118480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:05.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.405+0000 7f3020209640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f301811c9d0 0x7f30181189e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:05.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.405+0000 7f3020209640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f3018114970 con 0x7f3018111ce0 2024-08-30T09:09:05.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.405+0000 7f3020209640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f30181147f0 con 0x7f301810a780 2024-08-30T09:09:05.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.405+0000 7f3020209640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f3018114af0 con 0x7f301811c9d0 2024-08-30T09:09:05.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.406+0000 7f301e7a6640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f301811c9d0 0x7f30181189e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:05.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.406+0000 7f301e7a6640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f301811c9d0 0x7f30181189e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:35780/0 (socket says 172.21.15.42:35780) 2024-08-30T09:09:05.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.406+0000 7f301e7a6640 1 -- 172.21.15.42:0/1249355373 learned_addr learned my addr 172.21.15.42:0/1249355373 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:09:05.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.406+0000 7f301e7a6640 1 -- 172.21.15.42:0/1249355373 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f301810a780 msgr2=0x7f301811c1a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-30T09:09:05.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.406+0000 7f301e7a6640 1 --2- 172.21.15.42:0/1249355373 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f301810a780 0x7f301811c1a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:05.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.406+0000 7f301e7a6640 1 -- 172.21.15.42:0/1249355373 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3018111ce0 msgr2=0x7f3018118480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:09:05.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.406+0000 7f301e7a6640 1 --2- 172.21.15.42:0/1249355373 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3018111ce0 0x7f3018118480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:05.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.406+0000 7f301e7a6640 1 -- 172.21.15.42:0/1249355373 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3018119200 con 0x7f301811c9d0 2024-08-30T09:09:05.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.406+0000 7f301e7a6640 1 --2- 172.21.15.42:0/1249355373 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f301811c9d0 0x7f30181189e0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f301000eb10 tx=0x7f301000c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:09:05.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.407+0000 7f300effd640 1 -- 172.21.15.42:0/1249355373 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3010004b20 con 0x7f301811c9d0 2024-08-30T09:09:05.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.407+0000 7f3020209640 1 -- 172.21.15.42:0/1249355373 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f301807b420 con 0x7f301811c9d0 2024-08-30T09:09:05.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.407+0000 7f3020209640 1 -- 172.21.15.42:0/1249355373 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f301807b960 con 0x7f301811c9d0 2024-08-30T09:09:05.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.407+0000 7f300effd640 1 -- 172.21.15.42:0/1249355373 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3010004cc0 con 0x7f301811c9d0 2024-08-30T09:09:05.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.408+0000 7f300effd640 1 -- 172.21.15.42:0/1249355373 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3010010720 con 0x7f301811c9d0 2024-08-30T09:09:05.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.412+0000 7f302677e740 1 Processor -- start 2024-08-30T09:09:05.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.414+0000 7f300effd640 1 -- 172.21.15.42:0/1249355373 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7f30100108c0 con 0x7f301811c9d0 2024-08-30T09:09:05.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.415+0000 7f300effd640 1 --2- 172.21.15.42:0/1249355373 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f30040b4d40 0x7f30040b7250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:05.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.416+0000 7f300effd640 1 -- 172.21.15.42:0/1249355373 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f30100e5940 con 0x7f301811c9d0 2024-08-30T09:09:05.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.416+0000 7f302677e740 1 -- start start 2024-08-30T09:09:05.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.417+0000 7f301d7a4640 1 --2- 172.21.15.42:0/1249355373 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f30040b4d40 0x7f30040b7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:05.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.417+0000 7f302677e740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a8fa67d070 0x55a8faa22350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:05.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.417+0000 7f302677e740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a8fa6541b0 0x55a8faa22890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:05.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.417+0000 7f302677e740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a8fa65c390 0x55a8faa22dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:05.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.417+0000 7f302677e740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55a8fa999680 con 0x55a8fa6541b0 2024-08-30T09:09:05.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.417+0000 7f302677e740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55a8fa988c10 con 0x55a8fa65c390 2024-08-30T09:09:05.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.418+0000 7f301e7a6640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a8fa65c390 0x55a8faa22dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:05.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.418+0000 7f301d7a4640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a8fa6541b0 0x55a8faa22890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:05.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.418+0000 7f301e7a6640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a8fa65c390 0x55a8faa22dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:42956/0 (socket says 172.21.15.42:42956) 2024-08-30T09:09:05.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.418+0000 7f301d7a4640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a8fa6541b0 0x55a8faa22890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:54014/0 (socket says 172.21.15.42:54014) 2024-08-30T09:09:05.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.418+0000 7f301e7a6640 1 -- 172.21.15.42:0/1340728370 learned_addr learned my addr 172.21.15.42:0/1340728370 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:09:05.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.418+0000 7f301dfa5640 1 --2- 172.21.15.42:0/1340728370 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a8fa67d070 0x55a8faa22350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:05.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.418+0000 7f302677e740 1 -- 172.21.15.42:0/1340728370 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55a8fa98cbc0 con 0x55a8fa67d070 2024-08-30T09:09:05.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.418+0000 7f301dfa5640 1 -- 172.21.15.42:0/1340728370 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a8fa65c390 msgr2=0x55a8faa22dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:09:05.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.418+0000 7f301dfa5640 1 --2- 172.21.15.42:0/1340728370 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a8fa65c390 0x55a8faa22dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:05.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.418+0000 7f301dfa5640 1 -- 172.21.15.42:0/1340728370 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a8fa6541b0 msgr2=0x55a8faa22890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:09:05.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.418+0000 7f301dfa5640 1 --2- 172.21.15.42:0/1340728370 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a8fa6541b0 0x55a8faa22890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:05.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.418+0000 7f301dfa5640 1 -- 172.21.15.42:0/1340728370 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a8fa6e8650 con 0x55a8fa67d070 2024-08-30T09:09:05.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.418+0000 7f301d7a4640 1 --2- 172.21.15.42:0/1340728370 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a8fa6541b0 0x55a8faa22890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-30T09:09:05.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.418+0000 7f301d7a4640 1 --2- 172.21.15.42:0/1249355373 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f30040b4d40 0x7f30040b7250 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f301401a860 tx=0x7f30140047b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:09:05.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.418+0000 7f301dfa5640 1 --2- 172.21.15.42:0/1340728370 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a8fa67d070 0x55a8faa22350 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f3008009aa0 tx=0x7f3008009f60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6cc3eedc33df5767 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:09:05.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.419+0000 7f3003fff640 1 -- 172.21.15.42:0/1340728370 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f300800ece0 con 0x55a8fa67d070 2024-08-30T09:09:05.424 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.419+0000 7f3003fff640 1 -- 172.21.15.42:0/1340728370 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f300800ee80 con 0x55a8fa67d070 2024-08-30T09:09:05.424 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.419+0000 7f3003fff640 1 -- 172.21.15.42:0/1340728370 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30080049e0 con 0x55a8fa67d070 2024-08-30T09:09:05.425 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.420+0000 7f302677e740 1 -- 172.21.15.42:0/1340728370 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a8fa67d070 msgr2=0x55a8faa22350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:09:05.425 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.420+0000 7f302677e740 1 --2- 172.21.15.42:0/1340728370 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a8fa67d070 0x55a8faa22350 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f3008009aa0 tx=0x7f3008009f60 comp rx=0 tx=0).stop 2024-08-30T09:09:05.425 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.420+0000 7f302677e740 1 -- 172.21.15.42:0/1340728370 shutdown_connections 2024-08-30T09:09:05.425 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.421+0000 7f302677e740 1 --2- 172.21.15.42:0/1340728370 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a8fa65c390 0x55a8faa22dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:05.425 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.421+0000 7f302677e740 1 --2- 172.21.15.42:0/1340728370 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a8fa6541b0 0x55a8faa22890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:05.425 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.421+0000 7f302677e740 1 --2- 172.21.15.42:0/1340728370 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a8fa67d070 0x55a8faa22350 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:05.425 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.421+0000 7f302677e740 1 -- 172.21.15.42:0/1340728370 >> 172.21.15.42:0/1340728370 conn(0x55a8fa671460 msgr2=0x55a8faa20240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:09:05.425 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.421+0000 7f302677e740 1 -- 172.21.15.42:0/1340728370 shutdown_connections 2024-08-30T09:09:05.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.421+0000 7f302677e740 1 -- 172.21.15.42:0/1340728370 wait complete. 2024-08-30T09:09:05.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.422+0000 7f302677e740 1 -- start start 2024-08-30T09:09:05.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.422+0000 7f302677e740 1 Processor -- start 2024-08-30T09:09:05.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.423+0000 7f302677e740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a8fa6541b0 0x55a8faa2da40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:05.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.423+0000 7f302677e740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a8fa65c390 0x55a8faa2df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:05.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.424+0000 7f302677e740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a8fa67d070 0x55a8faa39f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:05.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.424+0000 7f301d7a4640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a8fa65c390 0x55a8faa2df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:05.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.424+0000 7f301d7a4640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a8fa65c390 0x55a8faa2df80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:42970/0 (socket says 172.21.15.42:42970) 2024-08-30T09:09:05.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.424+0000 7f301dfa5640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a8fa6541b0 0x55a8faa2da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:05.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.424+0000 7f301dfa5640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a8fa6541b0 0x55a8faa2da40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:35802/0 (socket says 172.21.15.42:35802) 2024-08-30T09:09:05.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.424+0000 7f301e7a6640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a8fa67d070 0x55a8faa39f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:05.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.424+0000 7f301d7a4640 1 -- 172.21.15.42:0/1306566072 learned_addr learned my addr 172.21.15.42:0/1306566072 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:09:05.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.424+0000 7f302677e740 1 -- 172.21.15.42:0/1306566072 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55a8fa7eb700 con 0x55a8fa67d070 2024-08-30T09:09:05.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.425+0000 7f302677e740 1 -- 172.21.15.42:0/1306566072 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55a8fa90f750 con 0x55a8fa65c390 2024-08-30T09:09:05.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.425+0000 7f302677e740 1 -- 172.21.15.42:0/1306566072 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55a8faa30bd0 con 0x55a8fa6541b0 2024-08-30T09:09:05.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.425+0000 7f302677e740 10 client.0 _ll_register_callbacks cb 0x55a8fa93ee10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:09:05.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.425+0000 7f301d7a4640 1 -- 172.21.15.42:0/1306566072 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a8fa6541b0 msgr2=0x55a8faa2da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:09:05.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.425+0000 7f301d7a4640 1 --2- 172.21.15.42:0/1306566072 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a8fa6541b0 0x55a8faa2da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:05.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.425+0000 7f301d7a4640 1 -- 172.21.15.42:0/1306566072 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a8fa67d070 msgr2=0x55a8faa39f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:09:05.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.425+0000 7f301d7a4640 1 --2- 172.21.15.42:0/1306566072 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a8fa67d070 0x55a8faa39f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:05.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.425+0000 7f301d7a4640 1 -- 172.21.15.42:0/1306566072 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a8fa948900 con 0x55a8fa65c390 2024-08-30T09:09:05.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.426+0000 7f301d7a4640 1 --2- 172.21.15.42:0/1306566072 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a8fa65c390 0x55a8faa2df80 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f3014044000 tx=0x7f3014000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2e184d8842b3b3e4 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:09:05.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.426+0000 7f30017fa640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-30T09:09:05.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.426+0000 7f30017fa640 1 -- 172.21.15.42:0/1306566072 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f301404f060 con 0x55a8fa65c390 2024-08-30T09:09:05.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.426+0000 7f302677e740 1 -- 172.21.15.42:0/1306566072 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55a8fa99b5d0 con 0x55a8fa65c390 2024-08-30T09:09:05.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.426+0000 7f30017fa640 1 -- 172.21.15.42:0/1306566072 <== mon.2 v2:172.21.15.167:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f3014005510 con 0x55a8fa65c390 2024-08-30T09:09:05.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.427+0000 7f30017fa640 1 -- 172.21.15.42:0/1306566072 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f3014004020 con 0x55a8fa65c390 2024-08-30T09:09:05.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.427+0000 7f302677e740 10 client.26612 fetch_fsmap learned FSMap version 37 2024-08-30T09:09:05.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.427+0000 7f302677e740 1 -- 172.21.15.42:0/1306566072 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55a8faa3c940 con 0x55a8fa65c390 2024-08-30T09:09:05.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.427+0000 7f30017fa640 1 -- 172.21.15.42:0/1306566072 <== mon.2 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3014018070 con 0x55a8fa65c390 2024-08-30T09:09:05.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.427+0000 7f30017fa640 1 -- 172.21.15.42:0/1306566072 <== mon.2 v2:172.21.15.167:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7f3014040510 con 0x55a8fa65c390 2024-08-30T09:09:05.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.427+0000 7f302677e740 10 client.26612 fetch_fsmap finished waiting for FSMap version 37 2024-08-30T09:09:05.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.427+0000 7f302677e740 10 client.26612 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-08-30T09:09:05.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.427+0000 7f302677e740 20 client.26612 populate_metadata read hostname 'smithi042' 2024-08-30T09:09:05.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.427+0000 7f302677e740 1 --2- 172.21.15.42:0/1306566072 >> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] conn(0x55a8fa63b9f0 0x55a8faa3fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:05.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.427+0000 7f302677e740 4 client.26612 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-30T09:09:05.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.427+0000 7f302677e740 1 -- 172.21.15.42:0/1306566072 --> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a8fa99b5d0 con 0x55a8fa63b9f0 2024-08-30T09:09:05.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.428+0000 7f301dfa5640 1 --2- 172.21.15.42:0/1306566072 >> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] conn(0x55a8fa63b9f0 0x55a8faa3fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:05.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.428+0000 7f30017fa640 10 client.26612 ms_handle_connect on v2:172.21.15.167:6836/3011837856 2024-08-30T09:09:05.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.429+0000 7f301dfa5640 1 --2- 172.21.15.42:0/1306566072 >> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] conn(0x55a8fa63b9f0 0x55a8faa3fe10 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d9cf7eba23d78152 server_cookie=fbe229678a93c2ec in_seq=0 out_seq=0 2024-08-30T09:09:05.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.430+0000 7f30017fa640 1 -- 172.21.15.42:0/1306566072 <== mds.2 v2:172.21.15.167:6836/3011837856 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a8fa99b5d0 con 0x55a8fa63b9f0 2024-08-30T09:09:05.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.430+0000 7f30017fa640 10 client.26612 handle_command_reply: tid=0 2024-08-30T09:09:05.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.430+0000 7f302677e740 1 client.26612 shutdown 2024-08-30T09:09:05.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.430+0000 7f302677e740 1 -- 172.21.15.42:0/1306566072 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a8fa65c390 msgr2=0x55a8faa2df80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:09:05.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.430+0000 7f302677e740 1 --2- 172.21.15.42:0/1306566072 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a8fa65c390 0x55a8faa2df80 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f3014044000 tx=0x7f3014000bc0 comp rx=0 tx=0).stop 2024-08-30T09:09:05.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.430+0000 7f302677e740 1 -- 172.21.15.42:0/1306566072 shutdown_connections 2024-08-30T09:09:05.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.430+0000 7f302677e740 1 --2- 172.21.15.42:0/1306566072 >> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] conn(0x55a8fa63b9f0 0x55a8faa3fe10 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:05.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.430+0000 7f302677e740 1 --2- 172.21.15.42:0/1306566072 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a8fa67d070 0x55a8faa39f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:05.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.430+0000 7f302677e740 1 --2- 172.21.15.42:0/1306566072 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a8fa65c390 0x55a8faa2df80 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:05.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.430+0000 7f302677e740 1 --2- 172.21.15.42:0/1306566072 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a8fa6541b0 0x55a8faa2da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:05.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.430+0000 7f302677e740 1 -- 172.21.15.42:0/1306566072 >> 172.21.15.42:0/1306566072 conn(0x55a8fa671460 msgr2=0x55a8faa23d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:09:05.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.430+0000 7f30017fa640 0 client.26612 ms_handle_reset on v2:172.21.15.167:6836/3011837856 2024-08-30T09:09:05.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.430+0000 7f302677e740 1 -- 172.21.15.42:0/1306566072 shutdown_connections 2024-08-30T09:09:05.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.430+0000 7f302677e740 1 -- 172.21.15.42:0/1306566072 wait complete. 2024-08-30T09:09:05.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.430+0000 7f302677e740 20 client.26612 trim_cache size 0 max 16384 2024-08-30T09:09:05.452 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.447+0000 7f302677e740 1 Processor -- start 2024-08-30T09:09:05.452 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.447+0000 7f302677e740 1 -- start start 2024-08-30T09:09:05.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.448+0000 7f302677e740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a8fa6541b0 0x55a8faa2d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:05.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.448+0000 7f302677e740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a8fa65c390 0x55a8faa2dd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:05.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.448+0000 7f302677e740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a8fa67d070 0x55a8faa2ac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:05.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.448+0000 7f302677e740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f2ffc012540 con 0x55a8fa67d070 2024-08-30T09:09:05.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.448+0000 7f302677e740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f2ffc00fe10 con 0x55a8fa6541b0 2024-08-30T09:09:05.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.448+0000 7f302677e740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f2ffc00dc60 con 0x55a8fa65c390 2024-08-30T09:09:05.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.448+0000 7f301dfa5640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a8fa6541b0 0x55a8faa2d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:05.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.448+0000 7f301e7a6640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a8fa67d070 0x55a8faa2ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:05.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.448+0000 7f301e7a6640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a8fa67d070 0x55a8faa2ac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:54024/0 (socket says 172.21.15.42:54024) 2024-08-30T09:09:05.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.448+0000 7f301e7a6640 1 -- 172.21.15.42:0/481432492 learned_addr learned my addr 172.21.15.42:0/481432492 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:09:05.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.448+0000 7f301d7a4640 1 --2- 172.21.15.42:0/481432492 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a8fa65c390 0x55a8faa2dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:05.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.449+0000 7f301e7a6640 1 -- 172.21.15.42:0/481432492 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a8fa65c390 msgr2=0x55a8faa2dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:09:05.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.449+0000 7f301e7a6640 1 --2- 172.21.15.42:0/481432492 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a8fa65c390 0x55a8faa2dd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:05.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.449+0000 7f301e7a6640 1 -- 172.21.15.42:0/481432492 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a8fa6541b0 msgr2=0x55a8faa2d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:09:05.454 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.449+0000 7f301e7a6640 1 --2- 172.21.15.42:0/481432492 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a8fa6541b0 0x55a8faa2d7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:05.454 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.449+0000 7f301e7a6640 1 -- 172.21.15.42:0/481432492 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a8faa24be0 con 0x55a8fa67d070 2024-08-30T09:09:05.454 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.449+0000 7f301e7a6640 1 --2- 172.21.15.42:0/481432492 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a8fa67d070 0x55a8faa2ac30 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f30100a8000 tx=0x7f30100ab770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3da31415446b913d server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:09:05.454 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.450+0000 7f30037fe640 1 -- 172.21.15.42:0/481432492 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30100c6070 con 0x55a8fa67d070 2024-08-30T09:09:05.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.450+0000 7f30037fe640 1 -- 172.21.15.42:0/481432492 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f30100abf80 con 0x55a8fa67d070 2024-08-30T09:09:05.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.450+0000 7f30037fe640 1 -- 172.21.15.42:0/481432492 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3010097360 con 0x55a8fa67d070 2024-08-30T09:09:05.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.452+0000 7f302677e740 1 -- 172.21.15.42:0/481432492 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a8fa67d070 msgr2=0x55a8faa2ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:09:05.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.452+0000 7f302677e740 1 --2- 172.21.15.42:0/481432492 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a8fa67d070 0x55a8faa2ac30 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f30100a8000 tx=0x7f30100ab770 comp rx=0 tx=0).stop 2024-08-30T09:09:05.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.452+0000 7f302677e740 1 -- 172.21.15.42:0/481432492 shutdown_connections 2024-08-30T09:09:05.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.452+0000 7f302677e740 1 --2- 172.21.15.42:0/481432492 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a8fa65c390 0x55a8faa2dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:05.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.452+0000 7f302677e740 1 --2- 172.21.15.42:0/481432492 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a8fa67d070 0x55a8faa2ac30 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:05.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.452+0000 7f302677e740 1 --2- 172.21.15.42:0/481432492 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a8fa6541b0 0x55a8faa2d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:05.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.452+0000 7f302677e740 1 -- 172.21.15.42:0/481432492 >> 172.21.15.42:0/481432492 conn(0x55a8fa671460 msgr2=0x55a8faa2c2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:09:05.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.452+0000 7f302677e740 1 -- 172.21.15.42:0/481432492 shutdown_connections 2024-08-30T09:09:05.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.452+0000 7f302677e740 1 -- 172.21.15.42:0/481432492 wait complete. 2024-08-30T09:09:05.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.452+0000 7f302677e740 1 -- start start 2024-08-30T09:09:05.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.452+0000 7f302677e740 1 Processor -- start 2024-08-30T09:09:05.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.453+0000 7f302677e740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a8fa6541b0 0x55a8faa32700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:05.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.453+0000 7f302677e740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a8fa65c390 0x55a8faa32ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:05.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.453+0000 7f302677e740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a8fa67d070 0x55a8faaa37a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:05.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.453+0000 7f302677e740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f2ffc00e2d0 con 0x55a8fa67d070 2024-08-30T09:09:05.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.453+0000 7f302677e740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f2ffc00f740 con 0x55a8fa65c390 2024-08-30T09:09:05.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.453+0000 7f302677e740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f2ffc00e9a0 con 0x55a8fa6541b0 2024-08-30T09:09:05.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.453+0000 7f302677e740 10 client.0 _ll_register_callbacks cb 0x7f30100093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:09:05.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.453+0000 7f301dfa5640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a8fa6541b0 0x55a8faa32700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:05.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.453+0000 7f301dfa5640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a8fa6541b0 0x55a8faa32700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:35820/0 (socket says 172.21.15.42:35820) 2024-08-30T09:09:05.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.453+0000 7f301dfa5640 1 -- 172.21.15.42:0/4173807346 learned_addr learned my addr 172.21.15.42:0/4173807346 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:09:05.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.453+0000 7f301e7a6640 1 --2- 172.21.15.42:0/4173807346 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a8fa67d070 0x55a8faaa37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:05.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.453+0000 7f301d7a4640 1 --2- 172.21.15.42:0/4173807346 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a8fa65c390 0x55a8faa32ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:05.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.454+0000 7f301e7a6640 1 -- 172.21.15.42:0/4173807346 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a8fa6541b0 msgr2=0x55a8faa32700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:09:05.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.454+0000 7f301e7a6640 1 --2- 172.21.15.42:0/4173807346 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a8fa6541b0 0x55a8faa32700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:05.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.454+0000 7f301e7a6640 1 -- 172.21.15.42:0/4173807346 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a8fa65c390 msgr2=0x55a8faa32ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:09:05.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.454+0000 7f301e7a6640 1 --2- 172.21.15.42:0/4173807346 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a8fa65c390 0x55a8faa32ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:05.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.454+0000 7f301e7a6640 1 -- 172.21.15.42:0/4173807346 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a8faaa3e10 con 0x55a8fa67d070 2024-08-30T09:09:05.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.454+0000 7f301e7a6640 1 --2- 172.21.15.42:0/4173807346 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a8fa67d070 0x55a8faaa37a0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f3010004f90 tx=0x7f30100ac740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=947b14df6744caf4 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:09:05.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.454+0000 7f30017fa640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-08-30T09:09:05.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.454+0000 7f30017fa640 1 -- 172.21.15.42:0/4173807346 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30100c6050 con 0x55a8fa67d070 2024-08-30T09:09:05.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.454+0000 7f30017fa640 1 -- 172.21.15.42:0/4173807346 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f30100cd300 con 0x55a8fa67d070 2024-08-30T09:09:05.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.454+0000 7f302677e740 1 -- 172.21.15.42:0/4173807346 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f30100abf80 con 0x55a8fa67d070 2024-08-30T09:09:05.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.454+0000 7f30017fa640 1 -- 172.21.15.42:0/4173807346 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f30100ba050 con 0x55a8fa67d070 2024-08-30T09:09:05.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.454+0000 7f302677e740 10 client.16830 fetch_fsmap learned FSMap version 37 2024-08-30T09:09:05.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.454+0000 7f302677e740 1 -- 172.21.15.42:0/4173807346 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55a8faaab0e0 con 0x55a8fa67d070 2024-08-30T09:09:05.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.455+0000 7f30017fa640 1 -- 172.21.15.42:0/4173807346 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f30100abf80 con 0x55a8fa67d070 2024-08-30T09:09:05.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.455+0000 7f30017fa640 1 -- 172.21.15.42:0/4173807346 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7f30100b4e10 con 0x55a8fa67d070 2024-08-30T09:09:05.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.455+0000 7f302677e740 10 client.16830 fetch_fsmap finished waiting for FSMap version 37 2024-08-30T09:09:05.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.455+0000 7f302677e740 10 client.16830 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-08-30T09:09:05.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.455+0000 7f302677e740 20 client.16830 populate_metadata read hostname 'smithi042' 2024-08-30T09:09:05.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.455+0000 7f302677e740 1 --2- 172.21.15.42:0/4173807346 >> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] conn(0x55a8fa610da0 0x55a8faaad820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:05.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.455+0000 7f302677e740 4 client.16830 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-30T09:09:05.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.455+0000 7f302677e740 1 -- 172.21.15.42:0/4173807346 --> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a8fa997070 con 0x55a8fa610da0 2024-08-30T09:09:05.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.455+0000 7f301dfa5640 1 --2- 172.21.15.42:0/4173807346 >> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] conn(0x55a8fa610da0 0x55a8faaad820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:05.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.456+0000 7f301dfa5640 1 --2- 172.21.15.42:0/4173807346 >> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] conn(0x55a8fa610da0 0x55a8faaad820 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3638265867db19ff server_cookie=f1c869eaf225a913 in_seq=0 out_seq=0 2024-08-30T09:09:05.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.456+0000 7f30017fa640 10 client.16830 ms_handle_connect on v2:172.21.15.167:6836/3011837856 2024-08-30T09:09:05.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.456+0000 7f30017fa640 1 -- 172.21.15.42:0/4173807346 <== mds.2 v2:172.21.15.167:6836/3011837856 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a8fa997070 con 0x55a8fa610da0 2024-08-30T09:09:05.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.456+0000 7f30017fa640 10 client.16830 handle_command_reply: tid=0 2024-08-30T09:09:05.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.456+0000 7f302677e740 1 client.16830 shutdown 2024-08-30T09:09:05.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.456+0000 7f302677e740 1 -- 172.21.15.42:0/4173807346 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a8fa67d070 msgr2=0x55a8faaa37a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:09:05.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.456+0000 7f302677e740 1 --2- 172.21.15.42:0/4173807346 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a8fa67d070 0x55a8faaa37a0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f3010004f90 tx=0x7f30100ac740 comp rx=0 tx=0).stop 2024-08-30T09:09:05.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.456+0000 7f302677e740 1 -- 172.21.15.42:0/4173807346 shutdown_connections 2024-08-30T09:09:05.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.456+0000 7f302677e740 1 --2- 172.21.15.42:0/4173807346 >> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] conn(0x55a8fa610da0 0x55a8faaad820 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:05.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.456+0000 7f302677e740 1 --2- 172.21.15.42:0/4173807346 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a8fa67d070 0x55a8faaa37a0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:05.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.456+0000 7f302677e740 1 --2- 172.21.15.42:0/4173807346 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a8fa65c390 0x55a8faa32ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:05.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.456+0000 7f302677e740 1 --2- 172.21.15.42:0/4173807346 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a8fa6541b0 0x55a8faa32700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:05.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.456+0000 7f30017fa640 0 client.16830 ms_handle_reset on v2:172.21.15.167:6836/3011837856 2024-08-30T09:09:05.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.456+0000 7f302677e740 1 -- 172.21.15.42:0/4173807346 >> 172.21.15.42:0/4173807346 conn(0x55a8fa671460 msgr2=0x55a8faa2c010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:09:05.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.456+0000 7f302677e740 1 -- 172.21.15.42:0/4173807346 shutdown_connections 2024-08-30T09:09:05.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.456+0000 7f302677e740 1 -- 172.21.15.42:0/4173807346 wait complete. 2024-08-30T09:09:05.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.456+0000 7f302677e740 20 client.16830 trim_cache size 0 max 16384 2024-08-30T09:09:05.463 INFO:teuthology.orchestra.run.smithi042.stdout:[] 2024-08-30T09:09:05.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.457+0000 7f3020209640 1 -- 172.21.15.42:0/1249355373 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f30040b4d40 msgr2=0x7f30040b7250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:09:05.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.457+0000 7f3020209640 1 --2- 172.21.15.42:0/1249355373 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f30040b4d40 0x7f30040b7250 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f301401a860 tx=0x7f30140047b0 comp rx=0 tx=0).stop 2024-08-30T09:09:05.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.457+0000 7f3020209640 1 -- 172.21.15.42:0/1249355373 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f301811c9d0 msgr2=0x7f30181189e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:09:05.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.457+0000 7f3020209640 1 --2- 172.21.15.42:0/1249355373 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f301811c9d0 0x7f30181189e0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f301000eb10 tx=0x7f301000c2d0 comp rx=0 tx=0).stop 2024-08-30T09:09:05.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.457+0000 7f3020209640 1 -- 172.21.15.42:0/1249355373 shutdown_connections 2024-08-30T09:09:05.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.457+0000 7f3020209640 1 --2- 172.21.15.42:0/1249355373 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f30040b4d40 0x7f30040b7250 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:05.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.457+0000 7f3020209640 1 --2- 172.21.15.42:0/1249355373 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f301811c9d0 0x7f30181189e0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:05.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.457+0000 7f3020209640 1 --2- 172.21.15.42:0/1249355373 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3018111ce0 0x7f3018118480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:05.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.457+0000 7f3020209640 1 --2- 172.21.15.42:0/1249355373 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f301810a780 0x7f301811c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:05.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.457+0000 7f3020209640 1 -- 172.21.15.42:0/1249355373 >> 172.21.15.42:0/1249355373 conn(0x7f3018070c60 msgr2=0x7f3018073bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:09:05.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.457+0000 7f3020209640 1 -- 172.21.15.42:0/1249355373 shutdown_connections 2024-08-30T09:09:05.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:05.458+0000 7f3020209640 1 -- 172.21.15.42:0/1249355373 wait complete. 2024-08-30T09:09:05.953 INFO:teuthology.orchestra.run.smithi042.stderr:progress: 500.0 s, 543.0 tps, lat 184.826 ms stddev 239.614 2024-08-30T09:09:06.673 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-08-30T09:09:06.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:09:06 smithi042 ceph-mon[18652]: pgmap v623: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.6 MiB/s wr, 491 op/s 2024-08-30T09:09:07.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:09:06 smithi167 ceph-mon[22063]: pgmap v623: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.6 MiB/s wr, 491 op/s 2024-08-30T09:09:07.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:09:06 smithi190 ceph-mon[22001]: pgmap v623: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.6 MiB/s wr, 491 op/s 2024-08-30T09:09:07.231 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T09:09:07.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:07.853+0000 7f9040938640 1 -- 172.21.15.42:0/711236930 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f903c10b890 msgr2=0x7f903c10bca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:09:07.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:07.853+0000 7f9040938640 1 --2- 172.21.15.42:0/711236930 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f903c10b890 0x7f903c10bca0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f902c00b880 tx=0x7f902c005f90 comp rx=0 tx=0).stop 2024-08-30T09:09:07.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:07.854+0000 7f9040938640 1 -- 172.21.15.42:0/711236930 shutdown_connections 2024-08-30T09:09:07.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:07.854+0000 7f9040938640 1 --2- 172.21.15.42:0/711236930 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f903c110230 0x7f903c07b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:07.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:07.854+0000 7f9040938640 1 --2- 172.21.15.42:0/711236930 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f903c10f8e0 0x7f903c10fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:07.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:07.854+0000 7f9040938640 1 --2- 172.21.15.42:0/711236930 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f903c10b890 0x7f903c10bca0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:07.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:07.854+0000 7f9040938640 1 -- 172.21.15.42:0/711236930 >> 172.21.15.42:0/711236930 conn(0x7f903c070c60 msgr2=0x7f903c071070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:09:07.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:07.855+0000 7f9040938640 1 -- 172.21.15.42:0/711236930 shutdown_connections 2024-08-30T09:09:07.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:07.858+0000 7f9040938640 1 -- 172.21.15.42:0/711236930 wait complete. 2024-08-30T09:09:07.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:07.859+0000 7f9040938640 1 Processor -- start 2024-08-30T09:09:07.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:07.859+0000 7f9040938640 1 -- start start 2024-08-30T09:09:07.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:07.859+0000 7f9040938640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f903c10b890 0x7f903c080590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:07.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:07.859+0000 7f9040938640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f903c110230 0x7f903c080ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:07.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:07.859+0000 7f9040938640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f903c0810e0 0x7f903c138a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:07.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:07.859+0000 7f9040938640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f903c07e940 con 0x7f903c0810e0 2024-08-30T09:09:07.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:07.859+0000 7f9040938640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f903c07e7c0 con 0x7f903c110230 2024-08-30T09:09:07.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:07.859+0000 7f9040938640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f903c07eac0 con 0x7f903c10b890 2024-08-30T09:09:07.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:07.860+0000 7f9039d9b640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f903c110230 0x7f903c080ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:07.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:07.860+0000 7f903ad9d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f903c0810e0 0x7f903c138a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:07.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:07.861+0000 7f903ad9d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f903c0810e0 0x7f903c138a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:54058/0 (socket says 172.21.15.42:54058) 2024-08-30T09:09:07.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:07.861+0000 7f903ad9d640 1 -- 172.21.15.42:0/1511291826 learned_addr learned my addr 172.21.15.42:0/1511291826 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:09:07.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:07.861+0000 7f903ad9d640 1 -- 172.21.15.42:0/1511291826 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f903c10b890 msgr2=0x7f903c080590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-30T09:09:07.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:07.861+0000 7f903ad9d640 1 --2- 172.21.15.42:0/1511291826 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f903c10b890 0x7f903c080590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:07.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:07.861+0000 7f903ad9d640 1 -- 172.21.15.42:0/1511291826 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f903c110230 msgr2=0x7f903c080ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:09:07.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:07.861+0000 7f903ad9d640 1 --2- 172.21.15.42:0/1511291826 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f903c110230 0x7f903c080ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:07.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:07.861+0000 7f903ad9d640 1 -- 172.21.15.42:0/1511291826 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f903c139260 con 0x7f903c0810e0 2024-08-30T09:09:07.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:07.861+0000 7f9039d9b640 1 --2- 172.21.15.42:0/1511291826 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f903c110230 0x7f903c080ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-30T09:09:07.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:07.862+0000 7f903ad9d640 1 --2- 172.21.15.42:0/1511291826 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f903c0810e0 0x7f903c138a40 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f903400bcc0 tx=0x7f9034008600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:09:07.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:07.867+0000 7f90337fe640 1 -- 172.21.15.42:0/1511291826 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9034004b20 con 0x7f903c0810e0 2024-08-30T09:09:07.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:07.867+0000 7f9040938640 1 -- 172.21.15.42:0/1511291826 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f903c139550 con 0x7f903c0810e0 2024-08-30T09:09:07.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:07.868+0000 7f9040938640 1 -- 172.21.15.42:0/1511291826 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f903c139a90 con 0x7f903c0810e0 2024-08-30T09:09:07.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:07.868+0000 7f90337fe640 1 -- 172.21.15.42:0/1511291826 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9034004cc0 con 0x7f903c0810e0 2024-08-30T09:09:07.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:07.868+0000 7f90337fe640 1 -- 172.21.15.42:0/1511291826 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9034015420 con 0x7f903c0810e0 2024-08-30T09:09:07.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:07.876+0000 7f9046ead740 1 Processor -- start 2024-08-30T09:09:07.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:07.878+0000 7f90337fe640 1 -- 172.21.15.42:0/1511291826 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7f903401c020 con 0x7f903c0810e0 2024-08-30T09:09:07.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:07.880+0000 7f90337fe640 1 --2- 172.21.15.42:0/1511291826 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f90280b4d00 0x7f90280b7210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:07.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:07.880+0000 7f90337fe640 1 -- 172.21.15.42:0/1511291826 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f90340e4e00 con 0x7f903c0810e0 2024-08-30T09:09:07.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:07.880+0000 7f9039d9b640 1 --2- 172.21.15.42:0/1511291826 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f90280b4d00 0x7f90280b7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:07.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:07.881+0000 7f9039d9b640 1 --2- 172.21.15.42:0/1511291826 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f90280b4d00 0x7f90280b7210 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f903c071200 tx=0x7f902400c040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:09:07.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:07.882+0000 7f9046ead740 1 -- start start 2024-08-30T09:09:07.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:07.882+0000 7f9046ead740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d9f9e07070 0x55d9fa1ac310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:07.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:07.882+0000 7f9046ead740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d9f9dde1b0 0x55d9fa1ac850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:07.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:07.882+0000 7f9046ead740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d9f9de6390 0x55d9fa1acd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:07.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:07.882+0000 7f9046ead740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55d9fa123680 con 0x55d9f9de6390 2024-08-30T09:09:07.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:07.882+0000 7f9046ead740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55d9fa112c10 con 0x55d9f9dde1b0 2024-08-30T09:09:07.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:07.882+0000 7f9046ead740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55d9fa116bc0 con 0x55d9f9e07070 2024-08-30T09:09:07.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:07.883+0000 7f903ad9d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d9f9de6390 0x55d9fa1acd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:07.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:07.883+0000 7f903ad9d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d9f9de6390 0x55d9fa1acd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:54070/0 (socket says 172.21.15.42:54070) 2024-08-30T09:09:07.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:07.883+0000 7f903ad9d640 1 -- 172.21.15.42:0/3253764871 learned_addr learned my addr 172.21.15.42:0/3253764871 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:09:07.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:07.884+0000 7f9039d9b640 1 --2- 172.21.15.42:0/3253764871 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d9f9dde1b0 0x55d9fa1ac850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:07.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:07.884+0000 7f9039d9b640 1 -- 172.21.15.42:0/3253764871 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d9f9e07070 msgr2=0x55d9fa1ac310 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-30T09:09:07.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:07.884+0000 7f9039d9b640 1 --2- 172.21.15.42:0/3253764871 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d9f9e07070 0x55d9fa1ac310 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:07.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:07.884+0000 7f9039d9b640 1 -- 172.21.15.42:0/3253764871 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d9f9de6390 msgr2=0x55d9fa1acd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:09:07.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:07.884+0000 7f9039d9b640 1 --2- 172.21.15.42:0/3253764871 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d9f9de6390 0x55d9fa1acd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:07.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:07.884+0000 7f9039d9b640 1 -- 172.21.15.42:0/3253764871 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d9f9e72650 con 0x55d9f9dde1b0 2024-08-30T09:09:07.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:07.884+0000 7f9039d9b640 1 --2- 172.21.15.42:0/3253764871 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d9f9dde1b0 0x55d9fa1ac850 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f903c06c4f0 tx=0x7f9024013000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9db7be9e1daaa2a4 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:09:07.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:07.885+0000 7f9030ff9640 1 -- 172.21.15.42:0/3253764871 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f902401d040 con 0x55d9f9dde1b0 2024-08-30T09:09:07.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:07.886+0000 7f9030ff9640 1 -- 172.21.15.42:0/3253764871 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9024026070 con 0x55d9f9dde1b0 2024-08-30T09:09:07.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:07.887+0000 7f9046ead740 1 -- 172.21.15.42:0/3253764871 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d9f9dde1b0 msgr2=0x55d9fa1ac850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:09:07.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:07.887+0000 7f9046ead740 1 --2- 172.21.15.42:0/3253764871 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d9f9dde1b0 0x55d9fa1ac850 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f903c06c4f0 tx=0x7f9024013000 comp rx=0 tx=0).stop 2024-08-30T09:09:07.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:07.887+0000 7f9046ead740 1 -- 172.21.15.42:0/3253764871 shutdown_connections 2024-08-30T09:09:07.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:07.887+0000 7f9046ead740 1 --2- 172.21.15.42:0/3253764871 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d9f9de6390 0x55d9fa1acd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:07.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:07.887+0000 7f9046ead740 1 --2- 172.21.15.42:0/3253764871 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d9f9dde1b0 0x55d9fa1ac850 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:07.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:07.887+0000 7f9046ead740 1 --2- 172.21.15.42:0/3253764871 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d9f9e07070 0x55d9fa1ac310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:07.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:07.887+0000 7f9046ead740 1 -- 172.21.15.42:0/3253764871 >> 172.21.15.42:0/3253764871 conn(0x55d9f9dfb460 msgr2=0x55d9fa1aa200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:09:07.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:07.888+0000 7f9046ead740 1 -- 172.21.15.42:0/3253764871 shutdown_connections 2024-08-30T09:09:07.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:07.888+0000 7f9046ead740 1 -- 172.21.15.42:0/3253764871 wait complete. 2024-08-30T09:09:07.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:07.889+0000 7f9046ead740 1 -- start start 2024-08-30T09:09:07.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:07.889+0000 7f9046ead740 1 Processor -- start 2024-08-30T09:09:07.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:07.889+0000 7f9046ead740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d9f9dde1b0 0x55d9fa1b7980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:07.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:07.889+0000 7f9046ead740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d9f9de6390 0x55d9fa1b7ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:07.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:07.889+0000 7f9046ead740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d9f9e07070 0x55d9fa1c3f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:07.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:07.889+0000 7f9046ead740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55d9f9f75700 con 0x55d9f9e07070 2024-08-30T09:09:07.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:07.889+0000 7f9046ead740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55d9fa099750 con 0x55d9f9de6390 2024-08-30T09:09:07.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:07.889+0000 7f9046ead740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55d9fa1bab40 con 0x55d9f9dde1b0 2024-08-30T09:09:07.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:07.889+0000 7f9046ead740 10 client.0 _ll_register_callbacks cb 0x55d9fa0c8e10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:09:07.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:07.890+0000 7f903a59c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d9f9dde1b0 0x55d9fa1b7980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:07.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:07.890+0000 7f903a59c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d9f9dde1b0 0x55d9fa1b7980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:35854/0 (socket says 172.21.15.42:35854) 2024-08-30T09:09:07.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:07.890+0000 7f903a59c640 1 -- 172.21.15.42:0/2550405454 learned_addr learned my addr 172.21.15.42:0/2550405454 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:09:07.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:07.890+0000 7f9039d9b640 1 --2- 172.21.15.42:0/2550405454 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d9f9de6390 0x55d9fa1b7ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:07.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:07.892+0000 7f903a59c640 1 -- 172.21.15.42:0/2550405454 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d9f9de6390 msgr2=0x55d9fa1b7ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:09:07.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:07.892+0000 7f903a59c640 1 --2- 172.21.15.42:0/2550405454 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d9f9de6390 0x55d9fa1b7ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:07.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:07.892+0000 7f903a59c640 1 -- 172.21.15.42:0/2550405454 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d9f9e07070 msgr2=0x55d9fa1c3f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:09:07.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:07.892+0000 7f903a59c640 1 --2- 172.21.15.42:0/2550405454 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d9f9e07070 0x55d9fa1c3f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:07.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:07.892+0000 7f903a59c640 1 -- 172.21.15.42:0/2550405454 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d9fa0d2900 con 0x55d9f9dde1b0 2024-08-30T09:09:07.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:07.894+0000 7f903a59c640 1 --2- 172.21.15.42:0/2550405454 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d9f9dde1b0 0x55d9fa1b7980 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f902c00a620 tx=0x7f902c00a650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4c47b410bddfd31c server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:09:07.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:07.894+0000 7f9021ffb640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-08-30T09:09:07.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:07.897+0000 7f9021ffb640 1 -- 172.21.15.42:0/2550405454 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f902c03f020 con 0x55d9f9dde1b0 2024-08-30T09:09:07.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:07.898+0000 7f9021ffb640 1 -- 172.21.15.42:0/2550405454 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f902c004250 con 0x55d9f9dde1b0 2024-08-30T09:09:07.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:07.899+0000 7f9046ead740 1 -- 172.21.15.42:0/2550405454 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9024026070 con 0x55d9f9dde1b0 2024-08-30T09:09:07.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:07.900+0000 7f9021ffb640 1 -- 172.21.15.42:0/2550405454 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f902c00b460 con 0x55d9f9dde1b0 2024-08-30T09:09:07.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:07.905+0000 7f9046ead740 10 client.26605 fetch_fsmap learned FSMap version 37 2024-08-30T09:09:07.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:07.905+0000 7f9046ead740 1 -- 172.21.15.42:0/2550405454 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55d9fa1c6920 con 0x55d9f9dde1b0 2024-08-30T09:09:07.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:07.908+0000 7f9021ffb640 1 -- 172.21.15.42:0/2550405454 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9024026070 con 0x55d9f9dde1b0 2024-08-30T09:09:07.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:07.909+0000 7f9021ffb640 1 -- 172.21.15.42:0/2550405454 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7f902c02ac10 con 0x55d9f9dde1b0 2024-08-30T09:09:07.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:07.915+0000 7f9046ead740 10 client.26605 fetch_fsmap finished waiting for FSMap version 37 2024-08-30T09:09:07.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:07.915+0000 7f9046ead740 10 client.26605 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-08-30T09:09:07.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:07.915+0000 7f9046ead740 20 client.26605 populate_metadata read hostname 'smithi042' 2024-08-30T09:09:07.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:07.915+0000 7f9046ead740 1 --2- 172.21.15.42:0/2550405454 >> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] conn(0x55d9f9dc59f0 0x55d9fa1c9e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:07.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:07.915+0000 7f9046ead740 4 client.26605 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-30T09:09:07.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:07.915+0000 7f9046ead740 1 -- 172.21.15.42:0/2550405454 --> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d9fa1255d0 con 0x55d9f9dc59f0 2024-08-30T09:09:07.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:07.918+0000 7f9039d9b640 1 --2- 172.21.15.42:0/2550405454 >> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] conn(0x55d9f9dc59f0 0x55d9fa1c9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:07.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:07.926+0000 7f9039d9b640 1 --2- 172.21.15.42:0/2550405454 >> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] conn(0x55d9f9dc59f0 0x55d9fa1c9e20 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=23786738dda3bd0 server_cookie=a8d43692e547b9dc in_seq=0 out_seq=0 2024-08-30T09:09:07.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:07.926+0000 7f9021ffb640 10 client.26605 ms_handle_connect on v2:172.21.15.190:6838/719981110 2024-08-30T09:09:07.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:07.928+0000 7f9021ffb640 1 -- 172.21.15.42:0/2550405454 <== mds.1 v2:172.21.15.190:6838/719981110 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d9fa1255d0 con 0x55d9f9dc59f0 2024-08-30T09:09:07.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:07.928+0000 7f9021ffb640 10 client.26605 handle_command_reply: tid=0 2024-08-30T09:09:07.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:07.929+0000 7f9046ead740 1 client.26605 shutdown 2024-08-30T09:09:07.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:07.930+0000 7f9046ead740 1 -- 172.21.15.42:0/2550405454 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d9f9dde1b0 msgr2=0x55d9fa1b7980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:09:07.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:07.930+0000 7f9046ead740 1 --2- 172.21.15.42:0/2550405454 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d9f9dde1b0 0x55d9fa1b7980 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f902c00a620 tx=0x7f902c00a650 comp rx=0 tx=0).stop 2024-08-30T09:09:07.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:07.934+0000 7f9046ead740 1 -- 172.21.15.42:0/2550405454 shutdown_connections 2024-08-30T09:09:07.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:07.934+0000 7f9046ead740 1 --2- 172.21.15.42:0/2550405454 >> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] conn(0x55d9f9dc59f0 0x55d9fa1c9e20 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:07.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:07.934+0000 7f9046ead740 1 --2- 172.21.15.42:0/2550405454 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d9f9e07070 0x55d9fa1c3f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:07.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:07.934+0000 7f9046ead740 1 --2- 172.21.15.42:0/2550405454 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d9f9de6390 0x55d9fa1b7ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:07.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:07.934+0000 7f9046ead740 1 --2- 172.21.15.42:0/2550405454 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d9f9dde1b0 0x55d9fa1b7980 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:07.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:07.934+0000 7f9046ead740 1 -- 172.21.15.42:0/2550405454 >> 172.21.15.42:0/2550405454 conn(0x55d9f9dfb460 msgr2=0x55d9fa1af4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:09:07.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:07.935+0000 7f9021ffb640 0 client.26605 ms_handle_reset on v2:172.21.15.190:6838/719981110 2024-08-30T09:09:07.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:07.938+0000 7f9046ead740 1 -- 172.21.15.42:0/2550405454 shutdown_connections 2024-08-30T09:09:07.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:07.939+0000 7f9046ead740 1 -- 172.21.15.42:0/2550405454 wait complete. 2024-08-30T09:09:07.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:07.942+0000 7f9046ead740 20 client.26605 trim_cache size 0 max 16384 2024-08-30T09:09:07.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:07.973+0000 7f9046ead740 1 Processor -- start 2024-08-30T09:09:07.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:07.973+0000 7f9046ead740 1 -- start start 2024-08-30T09:09:07.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:07.973+0000 7f9046ead740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d9f9dde1b0 0x55d9fa1ad340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:07.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:07.974+0000 7f9046ead740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d9f9de6390 0x55d9fa1a97d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:07.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:07.974+0000 7f903a59c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d9f9dde1b0 0x55d9fa1ad340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:07.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:07.974+0000 7f903a59c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d9f9dde1b0 0x55d9fa1ad340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:43112/0 (socket says 172.21.15.42:43112) 2024-08-30T09:09:07.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:07.974+0000 7f9046ead740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d9f9e07070 0x55d9fa1a9d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:07.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:07.974+0000 7f903a59c640 1 -- 172.21.15.42:0/2618137954 learned_addr learned my addr 172.21.15.42:0/2618137954 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:09:07.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:07.974+0000 7f9046ead740 1 -- 172.21.15.42:0/2618137954 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f901c012450 con 0x55d9f9de6390 2024-08-30T09:09:07.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:07.974+0000 7f9046ead740 1 -- 172.21.15.42:0/2618137954 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f901c00fd20 con 0x55d9f9e07070 2024-08-30T09:09:07.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:07.974+0000 7f903ad9d640 1 --2- 172.21.15.42:0/2618137954 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d9f9e07070 0x55d9fa1a9d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:07.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:07.975+0000 7f9046ead740 1 -- 172.21.15.42:0/2618137954 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f901c00dc20 con 0x55d9f9dde1b0 2024-08-30T09:09:07.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:07.976+0000 7f903a59c640 1 -- 172.21.15.42:0/2618137954 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d9f9e07070 msgr2=0x55d9fa1a9d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:09:07.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:07.976+0000 7f903a59c640 1 --2- 172.21.15.42:0/2618137954 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d9f9e07070 0x55d9fa1a9d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:07.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:07.976+0000 7f903a59c640 1 -- 172.21.15.42:0/2618137954 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d9f9de6390 msgr2=0x55d9fa1a97d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:09:07.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:07.976+0000 7f903a59c640 1 --2- 172.21.15.42:0/2618137954 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d9f9de6390 0x55d9fa1a97d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:07.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:07.976+0000 7f903a59c640 1 -- 172.21.15.42:0/2618137954 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d9fa1af170 con 0x55d9f9dde1b0 2024-08-30T09:09:07.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:07.979+0000 7f903a59c640 1 --2- 172.21.15.42:0/2618137954 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d9f9dde1b0 0x55d9fa1ad340 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f902c02a3d0 tx=0x7f902c00b340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=367f349458adeba0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:09:07.988 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:07.985+0000 7f9023fff640 1 -- 172.21.15.42:0/2618137954 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f902c03f040 con 0x55d9f9dde1b0 2024-08-30T09:09:07.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:07.985+0000 7f9023fff640 1 -- 172.21.15.42:0/2618137954 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f902c005bd0 con 0x55d9f9dde1b0 2024-08-30T09:09:07.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:07.985+0000 7f9023fff640 1 -- 172.21.15.42:0/2618137954 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f902c02a7f0 con 0x55d9f9dde1b0 2024-08-30T09:09:07.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:07.987+0000 7f9046ead740 1 -- 172.21.15.42:0/2618137954 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d9f9dde1b0 msgr2=0x55d9fa1ad340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:09:07.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:07.987+0000 7f9046ead740 1 --2- 172.21.15.42:0/2618137954 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d9f9dde1b0 0x55d9fa1ad340 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f902c02a3d0 tx=0x7f902c00b340 comp rx=0 tx=0).stop 2024-08-30T09:09:07.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:07.992+0000 7f9046ead740 1 -- 172.21.15.42:0/2618137954 shutdown_connections 2024-08-30T09:09:07.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:07.992+0000 7f9046ead740 1 --2- 172.21.15.42:0/2618137954 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d9f9e07070 0x55d9fa1a9d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:07.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:07.992+0000 7f9046ead740 1 --2- 172.21.15.42:0/2618137954 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d9f9de6390 0x55d9fa1a97d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:07.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:07.992+0000 7f9046ead740 1 --2- 172.21.15.42:0/2618137954 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d9f9dde1b0 0x55d9fa1ad340 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:07.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:07.992+0000 7f9046ead740 1 -- 172.21.15.42:0/2618137954 >> 172.21.15.42:0/2618137954 conn(0x55d9f9dfb460 msgr2=0x55d9fa1b9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:09:07.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:07.995+0000 7f9046ead740 1 -- 172.21.15.42:0/2618137954 shutdown_connections 2024-08-30T09:09:08.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:07.998+0000 7f9046ead740 1 -- 172.21.15.42:0/2618137954 wait complete. 2024-08-30T09:09:08.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:07.998+0000 7f9046ead740 1 -- start start 2024-08-30T09:09:08.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:07.999+0000 7f9046ead740 1 Processor -- start 2024-08-30T09:09:08.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:07.999+0000 7f9046ead740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d9f9dde1b0 0x55d9fa1bc050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:08.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:08.000+0000 7f9046ead740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d9f9de6390 0x55d9fa1bc590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:08.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:08.000+0000 7f903a59c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d9f9dde1b0 0x55d9fa1bc050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:08.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:08.000+0000 7f903a59c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d9f9dde1b0 0x55d9fa1bc050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:58414/0 (socket says 172.21.15.42:58414) 2024-08-30T09:09:08.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:08.000+0000 7f903a59c640 1 -- 172.21.15.42:0/2038252441 learned_addr learned my addr 172.21.15.42:0/2038252441 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:09:08.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:08.000+0000 7f9046ead740 1 --2- 172.21.15.42:0/2038252441 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d9f9e07070 0x55d9fa1bcad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:08.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:08.000+0000 7f9046ead740 1 -- 172.21.15.42:0/2038252441 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f901c00e1e0 con 0x55d9f9de6390 2024-08-30T09:09:08.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:08.000+0000 7f9046ead740 1 -- 172.21.15.42:0/2038252441 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f901c00f650 con 0x55d9f9dde1b0 2024-08-30T09:09:08.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:08.000+0000 7f903ad9d640 1 --2- 172.21.15.42:0/2038252441 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d9f9e07070 0x55d9fa1bcad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:08.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:08.001+0000 7f9046ead740 1 -- 172.21.15.42:0/2038252441 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f901c00e8b0 con 0x55d9f9e07070 2024-08-30T09:09:08.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:08.001+0000 7f9046ead740 10 client.0 _ll_register_callbacks cb 0x7f903400d7b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:09:08.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:08.001+0000 7f903ad9d640 1 -- 172.21.15.42:0/2038252441 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d9f9dde1b0 msgr2=0x55d9fa1bc050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:09:08.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:08.001+0000 7f903ad9d640 1 --2- 172.21.15.42:0/2038252441 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d9f9dde1b0 0x55d9fa1bc050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:08.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:08.001+0000 7f903ad9d640 1 -- 172.21.15.42:0/2038252441 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d9f9de6390 msgr2=0x55d9fa1bc590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:09:08.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:08.001+0000 7f903ad9d640 1 --2- 172.21.15.42:0/2038252441 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d9f9de6390 0x55d9fa1bc590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:08.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:08.001+0000 7f903ad9d640 1 -- 172.21.15.42:0/2038252441 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d9fa233a20 con 0x55d9f9e07070 2024-08-30T09:09:08.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:08.007+0000 7f903ad9d640 1 --2- 172.21.15.42:0/2038252441 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d9f9e07070 0x55d9fa1bcad0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f90340ab3d0 tx=0x7f903400d9a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4f75c98cbe308037 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:09:08.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:08.007+0000 7f9021ffb640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-08-30T09:09:08.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:08.007+0000 7f9021ffb640 1 -- 172.21.15.42:0/2038252441 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9034096070 con 0x55d9f9e07070 2024-08-30T09:09:08.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:08.007+0000 7f9021ffb640 1 -- 172.21.15.42:0/2038252441 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f90340ccbb0 con 0x55d9f9e07070 2024-08-30T09:09:08.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:08.009+0000 7f9046ead740 1 -- 172.21.15.42:0/2038252441 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f902c005bd0 con 0x55d9f9e07070 2024-08-30T09:09:08.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:08.013+0000 7f9021ffb640 1 -- 172.21.15.42:0/2038252441 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f90340aabb0 con 0x55d9f9e07070 2024-08-30T09:09:08.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:08.018+0000 7f9046ead740 10 client.26611 fetch_fsmap learned FSMap version 37 2024-08-30T09:09:08.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:08.018+0000 7f9046ead740 1 -- 172.21.15.42:0/2038252441 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55d9fa234cc0 con 0x55d9f9e07070 2024-08-30T09:09:08.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:08.021+0000 7f9021ffb640 1 -- 172.21.15.42:0/2038252441 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f902c005bd0 con 0x55d9f9e07070 2024-08-30T09:09:08.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:08.021+0000 7f9021ffb640 1 -- 172.21.15.42:0/2038252441 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7f90340a5b60 con 0x55d9f9e07070 2024-08-30T09:09:08.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:08.021+0000 7f9046ead740 10 client.26611 fetch_fsmap finished waiting for FSMap version 37 2024-08-30T09:09:08.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:08.021+0000 7f9046ead740 10 client.26611 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-08-30T09:09:08.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:08.021+0000 7f9046ead740 20 client.26611 populate_metadata read hostname 'smithi042' 2024-08-30T09:09:08.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:08.021+0000 7f9046ead740 1 --2- 172.21.15.42:0/2038252441 >> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] conn(0x55d9f9d9ada0 0x55d9fa2374b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:08.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:08.021+0000 7f9046ead740 4 client.26611 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-30T09:09:08.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:08.021+0000 7f9046ead740 1 -- 172.21.15.42:0/2038252441 --> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d9fa121070 con 0x55d9f9d9ada0 2024-08-30T09:09:08.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:08.022+0000 7f903a59c640 1 --2- 172.21.15.42:0/2038252441 >> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] conn(0x55d9f9d9ada0 0x55d9fa2374b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:08.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:08.023+0000 7f903a59c640 1 --2- 172.21.15.42:0/2038252441 >> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] conn(0x55d9f9d9ada0 0x55d9fa2374b0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=72c9e8fb8097c930 server_cookie=d612264baadb620b in_seq=0 out_seq=0 2024-08-30T09:09:08.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:08.023+0000 7f9021ffb640 10 client.26611 ms_handle_connect on v2:172.21.15.190:6838/719981110 2024-08-30T09:09:08.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:08.028+0000 7f9021ffb640 1 -- 172.21.15.42:0/2038252441 <== mds.1 v2:172.21.15.190:6838/719981110 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d9fa121070 con 0x55d9f9d9ada0 2024-08-30T09:09:08.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:08.028+0000 7f9021ffb640 10 client.26611 handle_command_reply: tid=0 2024-08-30T09:09:08.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:08.028+0000 7f9046ead740 1 client.26611 shutdown 2024-08-30T09:09:08.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:08.033+0000 7f9046ead740 1 -- 172.21.15.42:0/2038252441 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d9f9e07070 msgr2=0x55d9fa1bcad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:09:08.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:08.033+0000 7f9046ead740 1 --2- 172.21.15.42:0/2038252441 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d9f9e07070 0x55d9fa1bcad0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f90340ab3d0 tx=0x7f903400d9a0 comp rx=0 tx=0).stop 2024-08-30T09:09:08.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:08.033+0000 7f9046ead740 1 -- 172.21.15.42:0/2038252441 shutdown_connections 2024-08-30T09:09:08.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:08.033+0000 7f9046ead740 1 --2- 172.21.15.42:0/2038252441 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d9f9e07070 0x55d9fa1bcad0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:08.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:08.033+0000 7f9046ead740 1 --2- 172.21.15.42:0/2038252441 >> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] conn(0x55d9f9d9ada0 0x55d9fa2374b0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:08.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:08.033+0000 7f9046ead740 1 --2- 172.21.15.42:0/2038252441 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d9f9de6390 0x55d9fa1bc590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:08.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:08.033+0000 7f9046ead740 1 --2- 172.21.15.42:0/2038252441 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d9f9dde1b0 0x55d9fa1bc050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:08.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:08.033+0000 7f9046ead740 1 -- 172.21.15.42:0/2038252441 >> 172.21.15.42:0/2038252441 conn(0x55d9f9dfb460 msgr2=0x55d9fa1b8c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:09:08.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:08.033+0000 7f9021ffb640 0 client.26611 ms_handle_reset on v2:172.21.15.190:6838/719981110 2024-08-30T09:09:08.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:08.034+0000 7f9046ead740 1 -- 172.21.15.42:0/2038252441 shutdown_connections 2024-08-30T09:09:08.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:08.034+0000 7f9046ead740 1 -- 172.21.15.42:0/2038252441 wait complete. 2024-08-30T09:09:08.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:08.034+0000 7f9046ead740 20 client.26611 trim_cache size 0 max 16384 2024-08-30T09:09:08.038 INFO:teuthology.orchestra.run.smithi042.stdout:[] 2024-08-30T09:09:08.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:08.034+0000 7f9040938640 1 -- 172.21.15.42:0/1511291826 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f90280b4d00 msgr2=0x7f90280b7210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:09:08.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:08.034+0000 7f9040938640 1 --2- 172.21.15.42:0/1511291826 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f90280b4d00 0x7f90280b7210 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f903c071200 tx=0x7f902400c040 comp rx=0 tx=0).stop 2024-08-30T09:09:08.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:08.035+0000 7f9040938640 1 -- 172.21.15.42:0/1511291826 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f903c0810e0 msgr2=0x7f903c138a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:09:08.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:08.035+0000 7f9040938640 1 --2- 172.21.15.42:0/1511291826 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f903c0810e0 0x7f903c138a40 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f903400bcc0 tx=0x7f9034008600 comp rx=0 tx=0).stop 2024-08-30T09:09:08.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:08.035+0000 7f9040938640 1 -- 172.21.15.42:0/1511291826 shutdown_connections 2024-08-30T09:09:08.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:08.035+0000 7f9040938640 1 --2- 172.21.15.42:0/1511291826 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f90280b4d00 0x7f90280b7210 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:08.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:08.035+0000 7f9040938640 1 --2- 172.21.15.42:0/1511291826 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f903c0810e0 0x7f903c138a40 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:08.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:08.035+0000 7f9040938640 1 --2- 172.21.15.42:0/1511291826 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f903c110230 0x7f903c080ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:08.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:08.035+0000 7f9040938640 1 --2- 172.21.15.42:0/1511291826 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f903c10b890 0x7f903c080590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:08.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:08.035+0000 7f9040938640 1 -- 172.21.15.42:0/1511291826 >> 172.21.15.42:0/1511291826 conn(0x7f903c070c60 msgr2=0x7f903c10d960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:09:08.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:08.036+0000 7f9040938640 1 -- 172.21.15.42:0/1511291826 shutdown_connections 2024-08-30T09:09:08.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:08.036+0000 7f9040938640 1 -- 172.21.15.42:0/1511291826 wait complete. 2024-08-30T09:09:09.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:09:08 smithi167 ceph-mon[22063]: pgmap v624: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 4.9 MiB/s wr, 513 op/s 2024-08-30T09:09:09.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:09:08 smithi042 ceph-mon[18652]: pgmap v624: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 4.9 MiB/s wr, 513 op/s 2024-08-30T09:09:09.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:09:08 smithi190 ceph-mon[22001]: pgmap v624: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 4.9 MiB/s wr, 513 op/s 2024-08-30T09:09:10.953 INFO:teuthology.orchestra.run.smithi042.stderr:progress: 505.0 s, 468.0 tps, lat 214.861 ms stddev 309.268 2024-08-30T09:09:10.980 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-08-30T09:09:11.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:09:10 smithi167 ceph-mon[22063]: pgmap v625: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 5.3 MiB/s wr, 567 op/s 2024-08-30T09:09:11.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:09:10 smithi042 ceph-mon[18652]: pgmap v625: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 5.3 MiB/s wr, 567 op/s 2024-08-30T09:09:11.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:09:10 smithi190 ceph-mon[22001]: pgmap v625: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 5.3 MiB/s wr, 567 op/s 2024-08-30T09:09:11.506 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T09:09:13.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:09:12 smithi167 ceph-mon[22063]: pgmap v626: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 4.6 MiB/s wr, 514 op/s 2024-08-30T09:09:13.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:09:12 smithi042 ceph-mon[18652]: pgmap v626: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 4.6 MiB/s wr, 514 op/s 2024-08-30T09:09:13.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:09:12 smithi190 ceph-mon[22001]: pgmap v626: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 4.6 MiB/s wr, 514 op/s 2024-08-30T09:09:13.734 INFO:teuthology.orchestra.run.smithi042.stdout:[] 2024-08-30T09:09:13.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.764+0000 7f3cf7581640 1 -- 172.21.15.42:0/1705652669 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3cf0108a90 msgr2=0x7f3cf010f2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:09:13.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.764+0000 7f3cf7581640 1 --2- 172.21.15.42:0/1705652669 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3cf0108a90 0x7f3cf010f2c0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f3ce8009ca0 tx=0x7f3ce802a8c0 comp rx=0 tx=0).stop 2024-08-30T09:09:13.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.764+0000 7f3cf7581640 1 -- 172.21.15.42:0/1705652669 shutdown_connections 2024-08-30T09:09:13.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.764+0000 7f3cf7581640 1 --2- 172.21.15.42:0/1705652669 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3cf0108a90 0x7f3cf010f2c0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:13.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.764+0000 7f3cf7581640 1 --2- 172.21.15.42:0/1705652669 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3cf0108140 0x7f3cf0108550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:13.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.764+0000 7f3cf7581640 1 --2- 172.21.15.42:0/1705652669 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3cf0102800 0x7f3cf0102c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:13.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.764+0000 7f3cf7581640 1 -- 172.21.15.42:0/1705652669 >> 172.21.15.42:0/1705652669 conn(0x7f3cf00794d0 msgr2=0x7f3cf00798e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:09:13.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.764+0000 7f3cf7581640 1 -- 172.21.15.42:0/1705652669 shutdown_connections 2024-08-30T09:09:13.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.764+0000 7f3cf7581640 1 -- 172.21.15.42:0/1705652669 wait complete. 2024-08-30T09:09:13.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.765+0000 7f3cf7581640 1 Processor -- start 2024-08-30T09:09:13.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.765+0000 7f3cf7581640 1 -- start start 2024-08-30T09:09:13.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.765+0000 7f3cf7581640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3cf0102800 0x7f3cf00ff410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:13.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.765+0000 7f3cf7581640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3cf0108140 0x7f3cf00ff950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:13.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.765+0000 7f3cf7581640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3cf0108a90 0x7f3cf00ffe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:13.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.765+0000 7f3cf7581640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f3cf0112710 con 0x7f3cf0108140 2024-08-30T09:09:13.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.765+0000 7f3cf7581640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f3cf0112590 con 0x7f3cf0108a90 2024-08-30T09:09:13.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.765+0000 7f3cf7581640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f3cf0112890 con 0x7f3cf0102800 2024-08-30T09:09:13.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.765+0000 7f3cf4b1c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3cf0108140 0x7f3cf00ff950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:13.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.765+0000 7f3cf4b1c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3cf0108140 0x7f3cf00ff950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34972/0 (socket says 172.21.15.42:34972) 2024-08-30T09:09:13.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.765+0000 7f3cf4b1c640 1 -- 172.21.15.42:0/2054089892 learned_addr learned my addr 172.21.15.42:0/2054089892 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:09:13.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.766+0000 7f3cf531d640 1 --2- 172.21.15.42:0/2054089892 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3cf0102800 0x7f3cf00ff410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:13.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.766+0000 7f3cf5b1e640 1 --2- 172.21.15.42:0/2054089892 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3cf0108a90 0x7f3cf00ffe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:13.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.766+0000 7f3cf4b1c640 1 -- 172.21.15.42:0/2054089892 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3cf0102800 msgr2=0x7f3cf00ff410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:09:13.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.766+0000 7f3cf4b1c640 1 --2- 172.21.15.42:0/2054089892 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3cf0102800 0x7f3cf00ff410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:13.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.766+0000 7f3cf4b1c640 1 -- 172.21.15.42:0/2054089892 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3cf0108a90 msgr2=0x7f3cf00ffe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:09:13.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.766+0000 7f3cf4b1c640 1 --2- 172.21.15.42:0/2054089892 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3cf0108a90 0x7f3cf00ffe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:13.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.766+0000 7f3cf4b1c640 1 -- 172.21.15.42:0/2054089892 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cf01a83e0 con 0x7f3cf0108140 2024-08-30T09:09:13.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.766+0000 7f3cf5b1e640 1 --2- 172.21.15.42:0/2054089892 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3cf0108a90 0x7f3cf00ffe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-30T09:09:13.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.766+0000 7f3cf531d640 1 --2- 172.21.15.42:0/2054089892 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3cf0102800 0x7f3cf00ff410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-30T09:09:13.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.766+0000 7f3cf4b1c640 1 --2- 172.21.15.42:0/2054089892 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3cf0108140 0x7f3cf00ff950 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f3ce000c970 tx=0x7f3ce000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:09:13.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.766+0000 7f3ce67fc640 1 -- 172.21.15.42:0/2054089892 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ce0014070 con 0x7f3cf0108140 2024-08-30T09:09:13.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.766+0000 7f3ce67fc640 1 -- 172.21.15.42:0/2054089892 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ce0004440 con 0x7f3cf0108140 2024-08-30T09:09:13.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.766+0000 7f3cf7581640 1 -- 172.21.15.42:0/2054089892 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3cf01a86d0 con 0x7f3cf0108140 2024-08-30T09:09:13.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.766+0000 7f3ce67fc640 1 -- 172.21.15.42:0/2054089892 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ce0003e50 con 0x7f3cf0108140 2024-08-30T09:09:13.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.766+0000 7f3cf7581640 1 -- 172.21.15.42:0/2054089892 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3cf01a8c10 con 0x7f3cf0108140 2024-08-30T09:09:13.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.769+0000 7f3cfdaf6740 1 Processor -- start 2024-08-30T09:09:13.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.770+0000 7f3cfdaf6740 1 -- start start 2024-08-30T09:09:13.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.770+0000 7f3cfdaf6740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55683ce3d070 0x55683d1eab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:13.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.770+0000 7f3cfdaf6740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55683ce141b0 0x55683d1eb060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:13.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.770+0000 7f3cfdaf6740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55683ce1c390 0x55683d1eb5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:13.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.770+0000 7f3cfdaf6740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55683d159680 con 0x55683ce3d070 2024-08-30T09:09:13.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.770+0000 7f3cfdaf6740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55683d148c10 con 0x55683ce1c390 2024-08-30T09:09:13.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.770+0000 7f3cfdaf6740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55683d14cbc0 con 0x55683ce141b0 2024-08-30T09:09:13.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.770+0000 7f3ce67fc640 1 -- 172.21.15.42:0/2054089892 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7f3ce0017070 con 0x7f3cf0108140 2024-08-30T09:09:13.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.770+0000 7f3cf531d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55683ce141b0 0x55683d1eb060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:13.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.770+0000 7f3cf531d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55683ce141b0 0x55683d1eb060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:43132/0 (socket says 172.21.15.42:43132) 2024-08-30T09:09:13.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.770+0000 7f3cf531d640 1 -- 172.21.15.42:0/82661475 learned_addr learned my addr 172.21.15.42:0/82661475 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:09:13.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.772+0000 7f3cf4b1c640 1 --2- 172.21.15.42:0/82661475 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55683ce1c390 0x55683d1eb5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:13.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.772+0000 7f3ce67fc640 1 --2- 172.21.15.42:0/2054089892 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f3cdc0b4d50 0x7f3cdc0b7260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:13.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.772+0000 7f3cf5b1e640 1 --2- 172.21.15.42:0/2054089892 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f3cdc0b4d50 0x7f3cdc0b7260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:13.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.772+0000 7f3ce67fc640 1 -- 172.21.15.42:0/2054089892 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f3ce00e5090 con 0x7f3cf0108140 2024-08-30T09:09:13.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.772+0000 7f3cf531d640 1 -- 172.21.15.42:0/82661475 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55683ce1c390 msgr2=0x55683d1eb5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:09:13.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.772+0000 7f3cf531d640 1 --2- 172.21.15.42:0/82661475 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55683ce1c390 0x55683d1eb5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:13.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.772+0000 7f3cf531d640 1 -- 172.21.15.42:0/82661475 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55683ce3d070 msgr2=0x55683d1eab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:09:13.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.772+0000 7f3cf531d640 1 --2- 172.21.15.42:0/82661475 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55683ce3d070 0x55683d1eab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:13.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.772+0000 7f3cf531d640 1 -- 172.21.15.42:0/82661475 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55683cea8650 con 0x55683ce141b0 2024-08-30T09:09:13.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.774+0000 7f3cf5b1e640 1 --2- 172.21.15.42:0/2054089892 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f3cdc0b4d50 0x7f3cdc0b7260 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f3ce8005e30 tx=0x7f3ce8005d80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:09:13.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.774+0000 7f3cf531d640 1 --2- 172.21.15.42:0/82661475 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55683ce141b0 0x55683d1eb060 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f3cec033500 tx=0x7f3cec0339c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ffcabb2cae65fa86 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:09:13.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.774+0000 7f3cdb7fe640 1 -- 172.21.15.42:0/82661475 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cec004020 con 0x55683ce141b0 2024-08-30T09:09:13.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.775+0000 7f3cdb7fe640 1 -- 172.21.15.42:0/82661475 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3cec0041c0 con 0x55683ce141b0 2024-08-30T09:09:13.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.775+0000 7f3cdb7fe640 1 -- 172.21.15.42:0/82661475 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cec0317f0 con 0x55683ce141b0 2024-08-30T09:09:13.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.776+0000 7f3cfdaf6740 1 -- 172.21.15.42:0/82661475 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55683ce141b0 msgr2=0x55683d1eb060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:09:13.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.776+0000 7f3cfdaf6740 1 --2- 172.21.15.42:0/82661475 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55683ce141b0 0x55683d1eb060 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f3cec033500 tx=0x7f3cec0339c0 comp rx=0 tx=0).stop 2024-08-30T09:09:13.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.776+0000 7f3cfdaf6740 1 -- 172.21.15.42:0/82661475 shutdown_connections 2024-08-30T09:09:13.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.776+0000 7f3cfdaf6740 1 --2- 172.21.15.42:0/82661475 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55683ce141b0 0x55683d1eb060 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:13.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.776+0000 7f3cfdaf6740 1 --2- 172.21.15.42:0/82661475 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55683ce1c390 0x55683d1eb5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:13.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.776+0000 7f3cfdaf6740 1 --2- 172.21.15.42:0/82661475 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55683ce3d070 0x55683d1eab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:13.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.776+0000 7f3cfdaf6740 1 -- 172.21.15.42:0/82661475 >> 172.21.15.42:0/82661475 conn(0x55683ce31460 msgr2=0x55683d1e8a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:09:13.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.776+0000 7f3cfdaf6740 1 -- 172.21.15.42:0/82661475 shutdown_connections 2024-08-30T09:09:13.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.776+0000 7f3cfdaf6740 1 -- 172.21.15.42:0/82661475 wait complete. 2024-08-30T09:09:13.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.777+0000 7f3cfdaf6740 1 -- start start 2024-08-30T09:09:13.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.777+0000 7f3cfdaf6740 1 Processor -- start 2024-08-30T09:09:13.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.777+0000 7f3cfdaf6740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55683ce141b0 0x55683d1ee2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:13.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.777+0000 7f3cfdaf6740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55683ce1c390 0x55683d1ee8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:13.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.777+0000 7f3cfdaf6740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55683ce3d070 0x55683d1f38f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:13.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.777+0000 7f3cfdaf6740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55683cfab700 con 0x55683ce3d070 2024-08-30T09:09:13.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.777+0000 7f3cfdaf6740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55683d0cf750 con 0x55683ce1c390 2024-08-30T09:09:13.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.777+0000 7f3cfdaf6740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55683d1e7e50 con 0x55683ce141b0 2024-08-30T09:09:13.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.777+0000 7f3cfdaf6740 10 client.0 _ll_register_callbacks cb 0x55683d0fee10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:09:13.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.777+0000 7f3cf5b1e640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55683ce3d070 0x55683d1f38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:13.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.778+0000 7f3cf5b1e640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55683ce3d070 0x55683d1f38f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34986/0 (socket says 172.21.15.42:34986) 2024-08-30T09:09:13.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.778+0000 7f3cf5b1e640 1 -- 172.21.15.42:0/3639740749 learned_addr learned my addr 172.21.15.42:0/3639740749 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:09:13.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.778+0000 7f3cf4b1c640 1 --2- 172.21.15.42:0/3639740749 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55683ce1c390 0x55683d1ee8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:13.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.778+0000 7f3cf531d640 1 --2- 172.21.15.42:0/3639740749 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55683ce141b0 0x55683d1ee2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:13.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.778+0000 7f3cf5b1e640 1 -- 172.21.15.42:0/3639740749 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55683ce141b0 msgr2=0x55683d1ee2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:09:13.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.778+0000 7f3cf5b1e640 1 --2- 172.21.15.42:0/3639740749 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55683ce141b0 0x55683d1ee2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:13.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.778+0000 7f3cf5b1e640 1 -- 172.21.15.42:0/3639740749 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55683ce1c390 msgr2=0x55683d1ee8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:09:13.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.778+0000 7f3cf5b1e640 1 --2- 172.21.15.42:0/3639740749 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55683ce1c390 0x55683d1ee8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:13.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.778+0000 7f3cf5b1e640 1 -- 172.21.15.42:0/3639740749 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55683d108900 con 0x55683ce3d070 2024-08-30T09:09:13.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.778+0000 7f3cf531d640 1 --2- 172.21.15.42:0/3639740749 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55683ce141b0 0x55683d1ee2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-30T09:09:13.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.778+0000 7f3cf5b1e640 1 --2- 172.21.15.42:0/3639740749 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55683ce3d070 0x55683d1f38f0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f3ce8009a60 tx=0x7f3ce8005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b5b1c3207c520c2 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:09:13.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.778+0000 7f3cd8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-08-30T09:09:13.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.778+0000 7f3cd8ff9640 1 -- 172.21.15.42:0/3639740749 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ce8003d80 con 0x55683ce3d070 2024-08-30T09:09:13.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.778+0000 7f3cd8ff9640 1 -- 172.21.15.42:0/3639740749 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f3ce8004040 con 0x55683ce3d070 2024-08-30T09:09:13.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.778+0000 7f3cfdaf6740 1 -- 172.21.15.42:0/3639740749 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3cec0041c0 con 0x55683ce3d070 2024-08-30T09:09:13.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.778+0000 7f3cd8ff9640 1 -- 172.21.15.42:0/3639740749 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f3ce8004240 con 0x55683ce3d070 2024-08-30T09:09:13.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.779+0000 7f3cfdaf6740 10 client.16848 fetch_fsmap learned FSMap version 37 2024-08-30T09:09:13.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.779+0000 7f3cfdaf6740 1 -- 172.21.15.42:0/3639740749 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55683d1fc840 con 0x55683ce3d070 2024-08-30T09:09:13.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.779+0000 7f3cd8ff9640 1 -- 172.21.15.42:0/3639740749 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3cec0041c0 con 0x55683ce3d070 2024-08-30T09:09:13.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.779+0000 7f3cd8ff9640 1 -- 172.21.15.42:0/3639740749 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7f3ce802fb50 con 0x55683ce3d070 2024-08-30T09:09:13.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.779+0000 7f3cfdaf6740 10 client.16848 fetch_fsmap finished waiting for FSMap version 37 2024-08-30T09:09:13.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.779+0000 7f3cfdaf6740 10 client.16848 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-30T09:09:13.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.779+0000 7f3cfdaf6740 20 client.16848 populate_metadata read hostname 'smithi042' 2024-08-30T09:09:13.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.779+0000 7f3cfdaf6740 1 --2- 172.21.15.42:0/3639740749 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55683cdfb9f0 0x55683d1ffd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:13.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.779+0000 7f3cfdaf6740 4 client.16848 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-30T09:09:13.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.779+0000 7f3cfdaf6740 1 -- 172.21.15.42:0/3639740749 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55683d15b5d0 con 0x55683cdfb9f0 2024-08-30T09:09:13.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.779+0000 7f3cf531d640 1 --2- 172.21.15.42:0/3639740749 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55683cdfb9f0 0x55683d1ffd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:13.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.780+0000 7f3cd8ff9640 10 client.16848 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:09:13.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.780+0000 7f3cf531d640 1 --2- 172.21.15.42:0/3639740749 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55683cdfb9f0 0x55683d1ffd70 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aff35774621a654 server_cookie=cb297d64f52c4f14 in_seq=0 out_seq=0 2024-08-30T09:09:13.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.781+0000 7f3cd8ff9640 1 -- 172.21.15.42:0/3639740749 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55683d15b5d0 con 0x55683cdfb9f0 2024-08-30T09:09:13.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.781+0000 7f3cd8ff9640 10 client.16848 handle_command_reply: tid=0 2024-08-30T09:09:13.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.781+0000 7f3cfdaf6740 1 client.16848 shutdown 2024-08-30T09:09:13.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.781+0000 7f3cfdaf6740 1 -- 172.21.15.42:0/3639740749 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55683ce3d070 msgr2=0x55683d1f38f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:09:13.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.781+0000 7f3cfdaf6740 1 --2- 172.21.15.42:0/3639740749 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55683ce3d070 0x55683d1f38f0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f3ce8009a60 tx=0x7f3ce8005fd0 comp rx=0 tx=0).stop 2024-08-30T09:09:13.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.781+0000 7f3cfdaf6740 1 -- 172.21.15.42:0/3639740749 shutdown_connections 2024-08-30T09:09:13.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.781+0000 7f3cfdaf6740 1 --2- 172.21.15.42:0/3639740749 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55683cdfb9f0 0x55683d1ffd70 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:13.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.781+0000 7f3cfdaf6740 1 --2- 172.21.15.42:0/3639740749 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55683ce3d070 0x55683d1f38f0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:13.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.781+0000 7f3cd8ff9640 0 client.16848 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:09:13.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.781+0000 7f3cfdaf6740 1 --2- 172.21.15.42:0/3639740749 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55683ce1c390 0x55683d1ee8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:13.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.781+0000 7f3cfdaf6740 1 --2- 172.21.15.42:0/3639740749 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55683ce141b0 0x55683d1ee2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:13.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.781+0000 7f3cfdaf6740 1 -- 172.21.15.42:0/3639740749 >> 172.21.15.42:0/3639740749 conn(0x55683ce31460 msgr2=0x55683d1ec510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:09:13.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.781+0000 7f3cfdaf6740 1 -- 172.21.15.42:0/3639740749 shutdown_connections 2024-08-30T09:09:13.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.781+0000 7f3cfdaf6740 1 -- 172.21.15.42:0/3639740749 wait complete. 2024-08-30T09:09:13.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.781+0000 7f3cfdaf6740 20 client.16848 trim_cache size 0 max 16384 2024-08-30T09:09:13.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.794+0000 7f3cfdaf6740 1 Processor -- start 2024-08-30T09:09:13.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.795+0000 7f3cfdaf6740 1 -- start start 2024-08-30T09:09:13.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.795+0000 7f3cfdaf6740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55683ce141b0 0x55683d1f1700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:13.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.795+0000 7f3cfdaf6740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55683ce1c390 0x55683d1f1c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:13.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.795+0000 7f3cfdaf6740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55683ce3d070 0x55683d1f2180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:13.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.795+0000 7f3cfdaf6740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f3cf01b0ec0 con 0x55683ce1c390 2024-08-30T09:09:13.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.795+0000 7f3cfdaf6740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f3cf01ae790 con 0x55683ce141b0 2024-08-30T09:09:13.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.795+0000 7f3cfdaf6740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f3cf01ac5e0 con 0x55683ce3d070 2024-08-30T09:09:13.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.795+0000 7f3cf4b1c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55683ce1c390 0x55683d1f1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:13.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.795+0000 7f3cf4b1c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55683ce1c390 0x55683d1f1c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34994/0 (socket says 172.21.15.42:34994) 2024-08-30T09:09:13.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.795+0000 7f3cf4b1c640 1 -- 172.21.15.42:0/2335862199 learned_addr learned my addr 172.21.15.42:0/2335862199 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:09:13.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.795+0000 7f3cf5b1e640 1 --2- 172.21.15.42:0/2335862199 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55683ce3d070 0x55683d1f2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:13.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.795+0000 7f3cf531d640 1 --2- 172.21.15.42:0/2335862199 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55683ce141b0 0x55683d1f1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:13.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.795+0000 7f3cf4b1c640 1 -- 172.21.15.42:0/2335862199 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55683ce3d070 msgr2=0x55683d1f2180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:09:13.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.795+0000 7f3cf4b1c640 1 --2- 172.21.15.42:0/2335862199 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55683ce3d070 0x55683d1f2180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:13.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.795+0000 7f3cf4b1c640 1 -- 172.21.15.42:0/2335862199 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55683ce141b0 msgr2=0x55683d1f1700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:09:13.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.795+0000 7f3cf4b1c640 1 --2- 172.21.15.42:0/2335862199 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55683ce141b0 0x55683d1f1700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:13.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.795+0000 7f3cf4b1c640 1 -- 172.21.15.42:0/2335862199 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55683d1eec40 con 0x55683ce1c390 2024-08-30T09:09:13.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.795+0000 7f3cf4b1c640 1 --2- 172.21.15.42:0/2335862199 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55683ce1c390 0x55683d1f1c40 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f3ce00e2740 tx=0x7f3ce00057d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a465edd3ae43823 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:09:13.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.796+0000 7f3cdaffd640 1 -- 172.21.15.42:0/2335862199 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ce00b4330 con 0x55683ce1c390 2024-08-30T09:09:13.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.796+0000 7f3cdaffd640 1 -- 172.21.15.42:0/2335862199 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ce00ab1e0 con 0x55683ce1c390 2024-08-30T09:09:13.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.796+0000 7f3cdaffd640 1 -- 172.21.15.42:0/2335862199 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ce00a7960 con 0x55683ce1c390 2024-08-30T09:09:13.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.796+0000 7f3cfdaf6740 1 -- 172.21.15.42:0/2335862199 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55683ce1c390 msgr2=0x55683d1f1c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:09:13.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.796+0000 7f3cfdaf6740 1 --2- 172.21.15.42:0/2335862199 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55683ce1c390 0x55683d1f1c40 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f3ce00e2740 tx=0x7f3ce00057d0 comp rx=0 tx=0).stop 2024-08-30T09:09:13.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.796+0000 7f3cfdaf6740 1 -- 172.21.15.42:0/2335862199 shutdown_connections 2024-08-30T09:09:13.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.796+0000 7f3cfdaf6740 1 --2- 172.21.15.42:0/2335862199 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55683ce3d070 0x55683d1f2180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:13.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.796+0000 7f3cfdaf6740 1 --2- 172.21.15.42:0/2335862199 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55683ce1c390 0x55683d1f1c40 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:13.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.796+0000 7f3cfdaf6740 1 --2- 172.21.15.42:0/2335862199 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55683ce141b0 0x55683d1f1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:13.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.796+0000 7f3cfdaf6740 1 -- 172.21.15.42:0/2335862199 >> 172.21.15.42:0/2335862199 conn(0x55683ce31460 msgr2=0x55683d1e67a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:09:13.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.796+0000 7f3cfdaf6740 1 -- 172.21.15.42:0/2335862199 shutdown_connections 2024-08-30T09:09:13.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.796+0000 7f3cfdaf6740 1 -- 172.21.15.42:0/2335862199 wait complete. 2024-08-30T09:09:13.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.797+0000 7f3cfdaf6740 1 -- start start 2024-08-30T09:09:13.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.797+0000 7f3cfdaf6740 1 Processor -- start 2024-08-30T09:09:13.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.797+0000 7f3cfdaf6740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55683ce141b0 0x55683d1ec500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:13.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.797+0000 7f3cfdaf6740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55683ce1c390 0x55683d1ea9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:13.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.797+0000 7f3cfdaf6740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55683ce3d070 0x55683d1eaf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:13.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.797+0000 7f3cfdaf6740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f3cf01acc50 con 0x55683ce141b0 2024-08-30T09:09:13.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.797+0000 7f3cfdaf6740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f3cf01ae0c0 con 0x55683ce3d070 2024-08-30T09:09:13.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.797+0000 7f3cfdaf6740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f3cf01ad320 con 0x55683ce1c390 2024-08-30T09:09:13.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.797+0000 7f3cfdaf6740 10 client.0 _ll_register_callbacks cb 0x7f3cec005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:09:13.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.797+0000 7f3cf531d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55683ce141b0 0x55683d1ec500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:13.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.797+0000 7f3cf531d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55683ce141b0 0x55683d1ec500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34996/0 (socket says 172.21.15.42:34996) 2024-08-30T09:09:13.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.797+0000 7f3cf531d640 1 -- 172.21.15.42:0/2554123777 learned_addr learned my addr 172.21.15.42:0/2554123777 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:09:13.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.798+0000 7f3cf4b1c640 1 --2- 172.21.15.42:0/2554123777 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55683ce1c390 0x55683d1ea9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:13.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.798+0000 7f3cf5b1e640 1 --2- 172.21.15.42:0/2554123777 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55683ce3d070 0x55683d1eaf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:13.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.798+0000 7f3cf531d640 1 -- 172.21.15.42:0/2554123777 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55683ce1c390 msgr2=0x55683d1ea9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:09:13.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.798+0000 7f3cf531d640 1 --2- 172.21.15.42:0/2554123777 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55683ce1c390 0x55683d1ea9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:13.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.798+0000 7f3cf531d640 1 -- 172.21.15.42:0/2554123777 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55683ce3d070 msgr2=0x55683d1eaf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:09:13.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.798+0000 7f3cf531d640 1 --2- 172.21.15.42:0/2554123777 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55683ce3d070 0x55683d1eaf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:13.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.798+0000 7f3cf531d640 1 -- 172.21.15.42:0/2554123777 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55683d1eb490 con 0x55683ce141b0 2024-08-30T09:09:13.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.798+0000 7f3cf531d640 1 --2- 172.21.15.42:0/2554123777 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55683ce141b0 0x55683d1ec500 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f3cec003e50 tx=0x7f3cec004530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ef2d05f32e1e6 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:09:13.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.798+0000 7f3cd8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-08-30T09:09:13.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.798+0000 7f3cd8ff9640 1 -- 172.21.15.42:0/2554123777 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cec055060 con 0x55683ce141b0 2024-08-30T09:09:13.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.798+0000 7f3cd8ff9640 1 -- 172.21.15.42:0/2554123777 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f3cec005250 con 0x55683ce141b0 2024-08-30T09:09:13.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.798+0000 7f3cfdaf6740 1 -- 172.21.15.42:0/2554123777 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3ce00ab1e0 con 0x55683ce141b0 2024-08-30T09:09:13.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.798+0000 7f3cd8ff9640 1 -- 172.21.15.42:0/2554123777 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f3cec05c050 con 0x55683ce141b0 2024-08-30T09:09:13.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.798+0000 7f3cfdaf6740 10 client.16854 fetch_fsmap learned FSMap version 37 2024-08-30T09:09:13.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.798+0000 7f3cfdaf6740 1 -- 172.21.15.42:0/2554123777 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55683d26b450 con 0x55683ce141b0 2024-08-30T09:09:13.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.799+0000 7f3cd8ff9640 1 -- 172.21.15.42:0/2554123777 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ce00ab1e0 con 0x55683ce141b0 2024-08-30T09:09:13.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.799+0000 7f3cd8ff9640 1 -- 172.21.15.42:0/2554123777 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7f3cec004680 con 0x55683ce141b0 2024-08-30T09:09:13.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.799+0000 7f3cfdaf6740 10 client.16854 fetch_fsmap finished waiting for FSMap version 37 2024-08-30T09:09:13.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.799+0000 7f3cfdaf6740 10 client.16854 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-30T09:09:13.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.799+0000 7f3cfdaf6740 20 client.16854 populate_metadata read hostname 'smithi042' 2024-08-30T09:09:13.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.799+0000 7f3cfdaf6740 1 --2- 172.21.15.42:0/2554123777 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55683cdd0da0 0x55683d26daf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:13.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.799+0000 7f3cfdaf6740 4 client.16854 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-30T09:09:13.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.799+0000 7f3cfdaf6740 1 -- 172.21.15.42:0/2554123777 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55683d157070 con 0x55683cdd0da0 2024-08-30T09:09:13.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.799+0000 7f3cf4b1c640 1 --2- 172.21.15.42:0/2554123777 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55683cdd0da0 0x55683d26daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:13.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.800+0000 7f3cf4b1c640 1 --2- 172.21.15.42:0/2554123777 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55683cdd0da0 0x55683d26daf0 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f765cb789aa0ba6 server_cookie=693f69975f5ec07f in_seq=0 out_seq=0 2024-08-30T09:09:13.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.800+0000 7f3cd8ff9640 10 client.16854 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:09:13.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.800+0000 7f3cd8ff9640 1 -- 172.21.15.42:0/2554123777 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55683d157070 con 0x55683cdd0da0 2024-08-30T09:09:13.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.800+0000 7f3cd8ff9640 10 client.16854 handle_command_reply: tid=0 2024-08-30T09:09:13.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.800+0000 7f3cfdaf6740 1 client.16854 shutdown 2024-08-30T09:09:13.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.800+0000 7f3cfdaf6740 1 -- 172.21.15.42:0/2554123777 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55683ce141b0 msgr2=0x55683d1ec500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:09:13.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.800+0000 7f3cfdaf6740 1 --2- 172.21.15.42:0/2554123777 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55683ce141b0 0x55683d1ec500 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f3cec003e50 tx=0x7f3cec004530 comp rx=0 tx=0).stop 2024-08-30T09:09:13.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.800+0000 7f3cfdaf6740 1 -- 172.21.15.42:0/2554123777 shutdown_connections 2024-08-30T09:09:13.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.800+0000 7f3cfdaf6740 1 --2- 172.21.15.42:0/2554123777 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55683cdd0da0 0x55683d26daf0 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:13.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.800+0000 7f3cfdaf6740 1 --2- 172.21.15.42:0/2554123777 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55683ce1c390 0x55683d1ea9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:13.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.800+0000 7f3cfdaf6740 1 --2- 172.21.15.42:0/2554123777 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55683ce3d070 0x55683d1eaf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:13.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.800+0000 7f3cd8ff9640 0 client.16854 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:09:13.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.800+0000 7f3cfdaf6740 1 --2- 172.21.15.42:0/2554123777 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55683ce141b0 0x55683d1ec500 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:13.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.800+0000 7f3cfdaf6740 1 -- 172.21.15.42:0/2554123777 >> 172.21.15.42:0/2554123777 conn(0x55683ce31460 msgr2=0x55683d1e63a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:09:13.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.800+0000 7f3cfdaf6740 1 -- 172.21.15.42:0/2554123777 shutdown_connections 2024-08-30T09:09:13.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.800+0000 7f3cfdaf6740 1 -- 172.21.15.42:0/2554123777 wait complete. 2024-08-30T09:09:13.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.800+0000 7f3cfdaf6740 20 client.16854 trim_cache size 0 max 16384 2024-08-30T09:09:13.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.801+0000 7f3cf7581640 1 -- 172.21.15.42:0/2054089892 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f3cdc0b4d50 msgr2=0x7f3cdc0b7260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:09:13.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.801+0000 7f3cf7581640 1 --2- 172.21.15.42:0/2054089892 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f3cdc0b4d50 0x7f3cdc0b7260 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f3ce8005e30 tx=0x7f3ce8005d80 comp rx=0 tx=0).stop 2024-08-30T09:09:13.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.801+0000 7f3cf7581640 1 -- 172.21.15.42:0/2054089892 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3cf0108140 msgr2=0x7f3cf00ff950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:09:13.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.801+0000 7f3cf7581640 1 --2- 172.21.15.42:0/2054089892 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3cf0108140 0x7f3cf00ff950 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f3ce000c970 tx=0x7f3ce000ce30 comp rx=0 tx=0).stop 2024-08-30T09:09:13.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.801+0000 7f3cf7581640 1 -- 172.21.15.42:0/2054089892 shutdown_connections 2024-08-30T09:09:13.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.801+0000 7f3cf7581640 1 --2- 172.21.15.42:0/2054089892 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f3cdc0b4d50 0x7f3cdc0b7260 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:13.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.801+0000 7f3cf7581640 1 --2- 172.21.15.42:0/2054089892 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3cf0108a90 0x7f3cf00ffe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:13.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.801+0000 7f3cf7581640 1 --2- 172.21.15.42:0/2054089892 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3cf0108140 0x7f3cf00ff950 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:13.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.801+0000 7f3cf7581640 1 --2- 172.21.15.42:0/2054089892 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3cf0102800 0x7f3cf00ff410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:13.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.801+0000 7f3cf7581640 1 -- 172.21.15.42:0/2054089892 >> 172.21.15.42:0/2054089892 conn(0x7f3cf00794d0 msgr2=0x7f3cf006af30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:09:13.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.801+0000 7f3cf7581640 1 -- 172.21.15.42:0/2054089892 shutdown_connections 2024-08-30T09:09:13.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:12.801+0000 7f3cf7581640 1 -- 172.21.15.42:0/2054089892 wait complete. 2024-08-30T09:09:15.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:09:14 smithi167 ceph-mon[22063]: pgmap v627: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 5.9 MiB/s wr, 642 op/s 2024-08-30T09:09:15.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:09:14 smithi042 ceph-mon[18652]: pgmap v627: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 5.9 MiB/s wr, 642 op/s 2024-08-30T09:09:15.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:09:14 smithi190 ceph-mon[22001]: pgmap v627: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 5.9 MiB/s wr, 642 op/s 2024-08-30T09:09:15.529 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.526+0000 7ff29f8be640 1 -- 172.21.15.42:0/232549572 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff298103d60 msgr2=0x7ff29810a700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:09:15.529 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.526+0000 7ff29f8be640 1 --2- 172.21.15.42:0/232549572 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff298103d60 0x7ff29810a700 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7ff290009c90 tx=0x7ff29002a8a0 comp rx=0 tx=0).stop 2024-08-30T09:09:15.532 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.528+0000 7ff29f8be640 1 -- 172.21.15.42:0/232549572 shutdown_connections 2024-08-30T09:09:15.532 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.528+0000 7ff29f8be640 1 --2- 172.21.15.42:0/232549572 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff298103d60 0x7ff29810a700 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:15.532 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.528+0000 7ff29f8be640 1 --2- 172.21.15.42:0/232549572 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff2981033c0 0x7ff2981037d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:15.532 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.529+0000 7ff29f8be640 1 --2- 172.21.15.42:0/232549572 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff298102340 0x7ff2981027b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:15.532 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.529+0000 7ff29f8be640 1 -- 172.21.15.42:0/232549572 >> 172.21.15.42:0/232549572 conn(0x7ff2980fdc80 msgr2=0x7ff298100130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:09:15.537 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.535+0000 7ff29f8be640 1 -- 172.21.15.42:0/232549572 shutdown_connections 2024-08-30T09:09:15.538 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.536+0000 7ff29f8be640 1 -- 172.21.15.42:0/232549572 wait complete. 2024-08-30T09:09:15.540 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.537+0000 7ff29f8be640 1 Processor -- start 2024-08-30T09:09:15.540 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.537+0000 7ff29f8be640 1 -- start start 2024-08-30T09:09:15.541 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.538+0000 7ff29f8be640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff298102340 0x7ff298199dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:15.541 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.538+0000 7ff29f8be640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff2981033c0 0x7ff29819a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:15.541 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.538+0000 7ff29f8be640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff298103d60 0x7ff29819af80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:15.542 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.538+0000 7ff29f8be640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7ff29810f150 con 0x7ff298103d60 2024-08-30T09:09:15.542 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.538+0000 7ff29f8be640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff29810efd0 con 0x7ff2981033c0 2024-08-30T09:09:15.542 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.538+0000 7ff29f8be640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7ff29810f2d0 con 0x7ff298102340 2024-08-30T09:09:15.542 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.539+0000 7ff29de5b640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff298103d60 0x7ff29819af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:15.543 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.539+0000 7ff29de5b640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff298103d60 0x7ff29819af80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35008/0 (socket says 172.21.15.42:35008) 2024-08-30T09:09:15.543 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.539+0000 7ff29de5b640 1 -- 172.21.15.42:0/1120496319 learned_addr learned my addr 172.21.15.42:0/1120496319 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:09:15.543 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.540+0000 7ff29de5b640 1 -- 172.21.15.42:0/1120496319 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff298102340 msgr2=0x7ff298199dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:09:15.543 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.540+0000 7ff29de5b640 1 --2- 172.21.15.42:0/1120496319 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff298102340 0x7ff298199dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:15.543 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.540+0000 7ff29de5b640 1 -- 172.21.15.42:0/1120496319 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff2981033c0 msgr2=0x7ff29819a440 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-30T09:09:15.543 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.540+0000 7ff29de5b640 1 --2- 172.21.15.42:0/1120496319 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff2981033c0 0x7ff29819a440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:15.543 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.540+0000 7ff29de5b640 1 -- 172.21.15.42:0/1120496319 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff29819b7a0 con 0x7ff298103d60 2024-08-30T09:09:15.544 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.541+0000 7ff29de5b640 1 --2- 172.21.15.42:0/1120496319 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff298103d60 0x7ff29819af80 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7ff290003d30 tx=0x7ff290003d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:09:15.545 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.542+0000 7ff28e7fc640 1 -- 172.21.15.42:0/1120496319 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff290004c90 con 0x7ff298103d60 2024-08-30T09:09:15.545 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.542+0000 7ff29f8be640 1 -- 172.21.15.42:0/1120496319 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff29819ba30 con 0x7ff298103d60 2024-08-30T09:09:15.546 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.543+0000 7ff29f8be640 1 -- 172.21.15.42:0/1120496319 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2981a2180 con 0x7ff298103d60 2024-08-30T09:09:15.549 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.546+0000 7ff2a6823740 1 Processor -- start 2024-08-30T09:09:15.550 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.547+0000 7ff2a6823740 1 -- start start 2024-08-30T09:09:15.550 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.547+0000 7ff2a6823740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c2904401b0 0x55c2907e6460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:15.550 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.547+0000 7ff2a6823740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c2904172f0 0x55c2907e6ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:15.550 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.547+0000 7ff2a6823740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c29041f4d0 0x55c2907e7620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:15.550 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.547+0000 7ff2a6823740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55c290759000 con 0x55c29041f4d0 2024-08-30T09:09:15.550 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.547+0000 7ff2a6823740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55c290748590 con 0x55c2904172f0 2024-08-30T09:09:15.550 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.547+0000 7ff2a6823740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55c29074c540 con 0x55c2904401b0 2024-08-30T09:09:15.551 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.547+0000 7ff28e7fc640 1 -- 172.21.15.42:0/1120496319 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff290004e30 con 0x7ff298103d60 2024-08-30T09:09:15.551 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.547+0000 7ff28e7fc640 1 -- 172.21.15.42:0/1120496319 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff290030ad0 con 0x7ff298103d60 2024-08-30T09:09:15.552 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.549+0000 7ff29ce59640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c2904401b0 0x55c2907e6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:15.552 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.549+0000 7ff29ce59640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c2904401b0 0x55c2907e6460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:43196/0 (socket says 172.21.15.42:43196) 2024-08-30T09:09:15.552 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.549+0000 7ff29ce59640 1 -- 172.21.15.42:0/2567806485 learned_addr learned my addr 172.21.15.42:0/2567806485 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:09:15.554 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.551+0000 7ff29ce59640 1 --2- 172.21.15.42:0/2567806485 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c29041f4d0 0x55c2907e7620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:15.554 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.551+0000 7ff29ce59640 1 -- 172.21.15.42:0/2567806485 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c2904172f0 msgr2=0x55c2907e6ad0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-30T09:09:15.554 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.551+0000 7ff29d65a640 1 --2- 172.21.15.42:0/2567806485 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c2904172f0 0x55c2907e6ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:15.554 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.551+0000 7ff29ce59640 1 --2- 172.21.15.42:0/2567806485 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c2904172f0 0x55c2907e6ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:15.554 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.551+0000 7ff29ce59640 1 -- 172.21.15.42:0/2567806485 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c29041f4d0 msgr2=0x55c2907e7620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:09:15.554 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.551+0000 7ff29ce59640 1 --2- 172.21.15.42:0/2567806485 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c29041f4d0 0x55c2907e7620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:15.554 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.551+0000 7ff29ce59640 1 -- 172.21.15.42:0/2567806485 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c2904ab790 con 0x55c2904401b0 2024-08-30T09:09:15.555 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.552+0000 7ff28e7fc640 1 -- 172.21.15.42:0/1120496319 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7ff290030d50 con 0x7ff298103d60 2024-08-30T09:09:15.556 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.554+0000 7ff29ce59640 1 --2- 172.21.15.42:0/2567806485 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c2904401b0 0x55c2907e6460 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7ff28800bc30 tx=0x7ff28800b2c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=baca27def0d97040 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:09:15.557 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.554+0000 7ff28e7fc640 1 --2- 172.21.15.42:0/1120496319 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7ff2840b4d40 0x7ff2840b7250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:15.557 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.554+0000 7ff28e7fc640 1 -- 172.21.15.42:0/1120496319 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7ff290106570 con 0x7ff298103d60 2024-08-30T09:09:15.559 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.556+0000 7ff282ffd640 1 -- 172.21.15.42:0/2567806485 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff28801b070 con 0x55c2904401b0 2024-08-30T09:09:15.559 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.556+0000 7ff282ffd640 1 -- 172.21.15.42:0/2567806485 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2880060d0 con 0x55c2904401b0 2024-08-30T09:09:15.559 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.556+0000 7ff282ffd640 1 -- 172.21.15.42:0/2567806485 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff288004da0 con 0x55c2904401b0 2024-08-30T09:09:15.561 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.558+0000 7ff2a6823740 1 -- 172.21.15.42:0/2567806485 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c2904401b0 msgr2=0x55c2907e6460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:09:15.561 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.558+0000 7ff2a6823740 1 --2- 172.21.15.42:0/2567806485 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c2904401b0 0x55c2907e6460 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7ff28800bc30 tx=0x7ff28800b2c0 comp rx=0 tx=0).stop 2024-08-30T09:09:15.563 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.559+0000 7ff2a6823740 1 -- 172.21.15.42:0/2567806485 shutdown_connections 2024-08-30T09:09:15.563 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.559+0000 7ff2a6823740 1 --2- 172.21.15.42:0/2567806485 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c29041f4d0 0x55c2907e7620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:15.563 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.559+0000 7ff2a6823740 1 --2- 172.21.15.42:0/2567806485 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c2904172f0 0x55c2907e6ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:15.563 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.559+0000 7ff2a6823740 1 --2- 172.21.15.42:0/2567806485 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c2904401b0 0x55c2907e6460 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:15.563 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.559+0000 7ff2a6823740 1 -- 172.21.15.42:0/2567806485 >> 172.21.15.42:0/2567806485 conn(0x55c290434300 msgr2=0x55c2907e1f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:09:15.564 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.562+0000 7ff29d65a640 1 --2- 172.21.15.42:0/1120496319 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7ff2840b4d40 0x7ff2840b7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:15.567 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.564+0000 7ff2a6823740 1 -- 172.21.15.42:0/2567806485 shutdown_connections 2024-08-30T09:09:15.569 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.565+0000 7ff2a6823740 1 -- 172.21.15.42:0/2567806485 wait complete. 2024-08-30T09:09:15.569 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.566+0000 7ff2a6823740 1 -- start start 2024-08-30T09:09:15.569 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.566+0000 7ff2a6823740 1 Processor -- start 2024-08-30T09:09:15.573 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.567+0000 7ff2a6823740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c2904172f0 0x55c2907efcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:15.573 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.567+0000 7ff2a6823740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c29041f4d0 0x55c2907f0390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:15.573 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.567+0000 7ff2a6823740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c2904401b0 0x55c2907f7ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:15.573 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.567+0000 7ff2a6823740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55c290681ab0 con 0x55c2904172f0 2024-08-30T09:09:15.573 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.567+0000 7ff2a6823740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55c2906cf680 con 0x55c2904401b0 2024-08-30T09:09:15.573 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.567+0000 7ff2a6823740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55c2907e80b0 con 0x55c29041f4d0 2024-08-30T09:09:15.573 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.567+0000 7ff2a6823740 10 client.0 _ll_register_callbacks cb 0x55c290457a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:09:15.573 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.568+0000 7ff29ce59640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c29041f4d0 0x55c2907f0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:15.573 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.568+0000 7ff29ce59640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c29041f4d0 0x55c2907f0390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:43212/0 (socket says 172.21.15.42:43212) 2024-08-30T09:09:15.573 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.568+0000 7ff29ce59640 1 -- 172.21.15.42:0/1883058699 learned_addr learned my addr 172.21.15.42:0/1883058699 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:09:15.573 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.568+0000 7ff29de5b640 1 --2- 172.21.15.42:0/1883058699 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c2904401b0 0x55c2907f7ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:15.577 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.571+0000 7ff29de5b640 1 -- 172.21.15.42:0/1883058699 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c29041f4d0 msgr2=0x55c2907f0390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:09:15.578 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.571+0000 7ff29de5b640 1 --2- 172.21.15.42:0/1883058699 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c29041f4d0 0x55c2907f0390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:15.578 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.571+0000 7ff29de5b640 1 -- 172.21.15.42:0/1883058699 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c2904172f0 msgr2=0x55c2907efcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:09:15.578 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.571+0000 7ff29de5b640 1 --2- 172.21.15.42:0/1883058699 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c2904172f0 0x55c2907efcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:15.578 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.571+0000 7ff29de5b640 1 -- 172.21.15.42:0/1883058699 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c290524620 con 0x55c2904401b0 2024-08-30T09:09:15.578 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.573+0000 7ff29de5b640 1 --2- 172.21.15.42:0/1883058699 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c2904401b0 0x55c2907f7ee0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7ff290009c60 tx=0x7ff2900cc780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a0a20c4d5b25a737 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:09:15.578 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.574+0000 7ff280ff9640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-30T09:09:15.582 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.579+0000 7ff280ff9640 1 -- 172.21.15.42:0/1883058699 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2900d1070 con 0x55c2904401b0 2024-08-30T09:09:15.582 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.579+0000 7ff29d65a640 1 --2- 172.21.15.42:0/1120496319 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7ff2840b4d40 0x7ff2840b7250 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7ff294005fd0 tx=0x7ff294005c40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:09:15.582 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.579+0000 7ff2a6823740 1 -- 172.21.15.42:0/1883058699 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff2880060d0 con 0x55c2904401b0 2024-08-30T09:09:15.582 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.579+0000 7ff280ff9640 1 -- 172.21.15.42:0/1883058699 <== mon.2 v2:172.21.15.167:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7ff2900c9110 con 0x55c2904401b0 2024-08-30T09:09:15.583 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.580+0000 7ff280ff9640 1 -- 172.21.15.42:0/1883058699 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7ff2900d1210 con 0x55c2904401b0 2024-08-30T09:09:15.583 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.580+0000 7ff2a6823740 10 client.26651 fetch_fsmap learned FSMap version 37 2024-08-30T09:09:15.583 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.580+0000 7ff2a6823740 1 -- 172.21.15.42:0/1883058699 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55c29080aeb0 con 0x55c2904401b0 2024-08-30T09:09:15.586 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.583+0000 7ff280ff9640 1 -- 172.21.15.42:0/1883058699 <== mon.2 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2880060d0 con 0x55c2904401b0 2024-08-30T09:09:15.586 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.583+0000 7ff280ff9640 1 -- 172.21.15.42:0/1883058699 <== mon.2 v2:172.21.15.167:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7ff290030070 con 0x55c2904401b0 2024-08-30T09:09:15.586 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.583+0000 7ff2a6823740 10 client.26651 fetch_fsmap finished waiting for FSMap version 37 2024-08-30T09:09:15.586 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.583+0000 7ff2a6823740 10 client.26651 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-30T09:09:15.586 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.583+0000 7ff2a6823740 20 client.26651 populate_metadata read hostname 'smithi042' 2024-08-30T09:09:15.586 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.583+0000 7ff2a6823740 1 --2- 172.21.15.42:0/1883058699 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55c2903fed80 0x55c29080e780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:15.586 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.583+0000 7ff2a6823740 4 client.26651 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-30T09:09:15.586 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.583+0000 7ff2a6823740 1 -- 172.21.15.42:0/1883058699 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c29075af50 con 0x55c2903fed80 2024-08-30T09:09:15.594 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.592+0000 7ff29d65a640 1 --2- 172.21.15.42:0/1883058699 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55c2903fed80 0x55c29080e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:15.605 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.603+0000 7ff29d65a640 1 --2- 172.21.15.42:0/1883058699 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55c2903fed80 0x55c29080e780 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cbd4d500d4897d8 server_cookie=de50917ffdc20387 in_seq=0 out_seq=0 2024-08-30T09:09:15.606 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.604+0000 7ff280ff9640 10 client.26651 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:09:15.609 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.606+0000 7ff280ff9640 1 -- 172.21.15.42:0/1883058699 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55c29075af50 con 0x55c2903fed80 2024-08-30T09:09:15.609 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.607+0000 7ff280ff9640 10 client.26651 handle_command_reply: tid=0 2024-08-30T09:09:15.610 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.607+0000 7ff2a6823740 1 client.26651 shutdown 2024-08-30T09:09:15.614 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.610+0000 7ff2a6823740 1 -- 172.21.15.42:0/1883058699 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c2904401b0 msgr2=0x55c2907f7ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:09:15.614 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.610+0000 7ff2a6823740 1 --2- 172.21.15.42:0/1883058699 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c2904401b0 0x55c2907f7ee0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7ff290009c60 tx=0x7ff2900cc780 comp rx=0 tx=0).stop 2024-08-30T09:09:15.615 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.612+0000 7ff2a6823740 1 -- 172.21.15.42:0/1883058699 shutdown_connections 2024-08-30T09:09:15.615 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.612+0000 7ff2a6823740 1 --2- 172.21.15.42:0/1883058699 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55c2903fed80 0x55c29080e780 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:15.615 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.612+0000 7ff2a6823740 1 --2- 172.21.15.42:0/1883058699 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c29041f4d0 0x55c2907f0390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:15.615 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.612+0000 7ff2a6823740 1 --2- 172.21.15.42:0/1883058699 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c2904401b0 0x55c2907f7ee0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:15.615 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.612+0000 7ff2a6823740 1 --2- 172.21.15.42:0/1883058699 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c2904172f0 0x55c2907efcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:15.615 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.612+0000 7ff2a6823740 1 -- 172.21.15.42:0/1883058699 >> 172.21.15.42:0/1883058699 conn(0x55c290434300 msgr2=0x55c2907e05a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:09:15.615 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.612+0000 7ff280ff9640 0 client.26651 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:09:15.616 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.613+0000 7ff2a6823740 1 -- 172.21.15.42:0/1883058699 shutdown_connections 2024-08-30T09:09:15.617 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.614+0000 7ff2a6823740 1 -- 172.21.15.42:0/1883058699 wait complete. 2024-08-30T09:09:15.617 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.615+0000 7ff2a6823740 20 client.26651 trim_cache size 0 max 16384 2024-08-30T09:09:15.651 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.648+0000 7ff2a6823740 1 Processor -- start 2024-08-30T09:09:15.651 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.648+0000 7ff2a6823740 1 -- start start 2024-08-30T09:09:15.651 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.648+0000 7ff2a6823740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c2904172f0 0x55c2907e1870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:15.651 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.648+0000 7ff2a6823740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c29041f4d0 0x55c2907f7d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:15.651 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.648+0000 7ff2a6823740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c2904401b0 0x55c2907e6090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:15.652 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.648+0000 7ff2a6823740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7ff27c012510 con 0x55c2904172f0 2024-08-30T09:09:15.652 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.648+0000 7ff2a6823740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff27c00fde0 con 0x55c29041f4d0 2024-08-30T09:09:15.652 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.648+0000 7ff2a6823740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7ff27c00dc30 con 0x55c2904401b0 2024-08-30T09:09:15.652 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.649+0000 7ff29ce59640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c29041f4d0 0x55c2907f7d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:15.652 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.649+0000 7ff29de5b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c2904401b0 0x55c2907e6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:15.652 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.649+0000 7ff29de5b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c2904401b0 0x55c2907e6090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:43226/0 (socket says 172.21.15.42:43226) 2024-08-30T09:09:15.652 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.649+0000 7ff29ce59640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c29041f4d0 0x55c2907f7d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:58522/0 (socket says 172.21.15.42:58522) 2024-08-30T09:09:15.652 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.649+0000 7ff29de5b640 1 -- 172.21.15.42:0/1338979947 learned_addr learned my addr 172.21.15.42:0/1338979947 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:09:15.653 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.649+0000 7ff29ce59640 1 -- 172.21.15.42:0/1338979947 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c2904401b0 msgr2=0x55c2907e6090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:09:15.653 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.649+0000 7ff29ce59640 1 --2- 172.21.15.42:0/1338979947 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c2904401b0 0x55c2907e6090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:15.653 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.649+0000 7ff29ce59640 1 -- 172.21.15.42:0/1338979947 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c2904172f0 msgr2=0x55c2907e1870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:09:15.653 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.649+0000 7ff29ce59640 1 --2- 172.21.15.42:0/1338979947 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c2904172f0 0x55c2907e1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:15.653 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.650+0000 7ff29ce59640 1 -- 172.21.15.42:0/1338979947 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c2903fefb0 con 0x55c29041f4d0 2024-08-30T09:09:15.653 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.650+0000 7ff29ce59640 1 --2- 172.21.15.42:0/1338979947 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c29041f4d0 0x55c2907f7d00 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x55c2907dc970 tx=0x7ff28800b3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9d96d0f13a866780 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:09:15.655 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.652+0000 7ff2827fc640 1 -- 172.21.15.42:0/1338979947 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff28801b040 con 0x55c29041f4d0 2024-08-30T09:09:15.655 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.652+0000 7ff2827fc640 1 -- 172.21.15.42:0/1338979947 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff28800b040 con 0x55c29041f4d0 2024-08-30T09:09:15.657 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.654+0000 7ff2a6823740 1 -- 172.21.15.42:0/1338979947 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c29041f4d0 msgr2=0x55c2907f7d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:09:15.657 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.654+0000 7ff2a6823740 1 --2- 172.21.15.42:0/1338979947 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c29041f4d0 0x55c2907f7d00 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x55c2907dc970 tx=0x7ff28800b3b0 comp rx=0 tx=0).stop 2024-08-30T09:09:15.659 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.657+0000 7ff2a6823740 1 -- 172.21.15.42:0/1338979947 shutdown_connections 2024-08-30T09:09:15.660 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.657+0000 7ff2a6823740 1 --2- 172.21.15.42:0/1338979947 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c2904401b0 0x55c2907e6090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:15.660 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.657+0000 7ff2a6823740 1 --2- 172.21.15.42:0/1338979947 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c29041f4d0 0x55c2907f7d00 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:15.660 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.657+0000 7ff2a6823740 1 --2- 172.21.15.42:0/1338979947 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c2904172f0 0x55c2907e1870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:15.660 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.657+0000 7ff2a6823740 1 -- 172.21.15.42:0/1338979947 >> 172.21.15.42:0/1338979947 conn(0x55c290434300 msgr2=0x55c2907f8440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:09:15.664 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.662+0000 7ff2a6823740 1 -- 172.21.15.42:0/1338979947 shutdown_connections 2024-08-30T09:09:15.669 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.666+0000 7ff2a6823740 1 -- 172.21.15.42:0/1338979947 wait complete. 2024-08-30T09:09:15.669 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.666+0000 7ff2a6823740 1 -- start start 2024-08-30T09:09:15.669 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.666+0000 7ff2a6823740 1 Processor -- start 2024-08-30T09:09:15.670 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.667+0000 7ff2a6823740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c2904172f0 0x55c2907f5110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:15.670 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.667+0000 7ff2a6823740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c29041f4d0 0x55c2907f57d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:15.670 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.667+0000 7ff2a6823740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c2904401b0 0x55c2907f62d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:15.670 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.667+0000 7ff2a6823740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7ff27c00e2a0 con 0x55c2904172f0 2024-08-30T09:09:15.670 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.667+0000 7ff2a6823740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff27c00f710 con 0x55c29041f4d0 2024-08-30T09:09:15.671 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.667+0000 7ff2a6823740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7ff27c00e970 con 0x55c2904401b0 2024-08-30T09:09:15.671 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.667+0000 7ff2a6823740 10 client.0 _ll_register_callbacks cb 0x7ff2900c89e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:09:15.671 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.667+0000 7ff29ce59640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c29041f4d0 0x55c2907f57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:15.671 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.667+0000 7ff29ce59640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c29041f4d0 0x55c2907f57d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:58536/0 (socket says 172.21.15.42:58536) 2024-08-30T09:09:15.671 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.667+0000 7ff29ce59640 1 -- 172.21.15.42:0/3767572964 learned_addr learned my addr 172.21.15.42:0/3767572964 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:09:15.671 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.667+0000 7ff29de5b640 1 --2- 172.21.15.42:0/3767572964 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c2904401b0 0x55c2907f62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:15.671 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.668+0000 7ff29ce59640 1 -- 172.21.15.42:0/3767572964 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c2904401b0 msgr2=0x55c2907f62d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:09:15.671 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.668+0000 7ff29ce59640 1 --2- 172.21.15.42:0/3767572964 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c2904401b0 0x55c2907f62d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:15.672 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.668+0000 7ff29ce59640 1 -- 172.21.15.42:0/3767572964 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c2904172f0 msgr2=0x55c2907f5110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:09:15.672 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.668+0000 7ff29ce59640 1 --2- 172.21.15.42:0/3767572964 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c2904172f0 0x55c2907f5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:15.672 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.668+0000 7ff29ce59640 1 -- 172.21.15.42:0/3767572964 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c2907f6a30 con 0x55c29041f4d0 2024-08-30T09:09:15.675 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.672+0000 7ff29ce59640 1 --2- 172.21.15.42:0/3767572964 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c29041f4d0 0x55c2907f57d0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7ff28800bb50 tx=0x7ff288006540 comp rx=0 tx=0).ready entity=mon.2 client_cookie=14e0d27c94b2e48e server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:09:15.675 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.672+0000 7ff277fff640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-30T09:09:15.677 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.674+0000 7ff277fff640 1 -- 172.21.15.42:0/3767572964 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff28801b070 con 0x55c29041f4d0 2024-08-30T09:09:15.678 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.675+0000 7ff2a6823740 1 -- 172.21.15.42:0/3767572964 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff28800b040 con 0x55c29041f4d0 2024-08-30T09:09:15.678 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.675+0000 7ff277fff640 1 -- 172.21.15.42:0/3767572964 <== mon.2 v2:172.21.15.167:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7ff2880066e0 con 0x55c29041f4d0 2024-08-30T09:09:15.678 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.676+0000 7ff277fff640 1 -- 172.21.15.42:0/3767572964 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7ff288036050 con 0x55c29041f4d0 2024-08-30T09:09:15.682 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.679+0000 7ff2a6823740 10 client.26657 fetch_fsmap learned FSMap version 37 2024-08-30T09:09:15.682 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.679+0000 7ff2a6823740 1 -- 172.21.15.42:0/3767572964 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55c2907f7d20 con 0x55c29041f4d0 2024-08-30T09:09:15.687 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.684+0000 7ff277fff640 1 -- 172.21.15.42:0/3767572964 <== mon.2 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff28800b040 con 0x55c29041f4d0 2024-08-30T09:09:15.688 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.686+0000 7ff277fff640 1 -- 172.21.15.42:0/3767572964 <== mon.2 v2:172.21.15.167:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7ff2880284a0 con 0x55c29041f4d0 2024-08-30T09:09:15.689 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.686+0000 7ff2a6823740 10 client.26657 fetch_fsmap finished waiting for FSMap version 37 2024-08-30T09:09:15.689 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.686+0000 7ff2a6823740 10 client.26657 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-30T09:09:15.689 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.686+0000 7ff2a6823740 20 client.26657 populate_metadata read hostname 'smithi042' 2024-08-30T09:09:15.689 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.686+0000 7ff2a6823740 1 --2- 172.21.15.42:0/3767572964 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55c2903d41c0 0x55c29088c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:15.689 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.686+0000 7ff2a6823740 4 client.26657 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-30T09:09:15.689 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.686+0000 7ff2a6823740 1 -- 172.21.15.42:0/3767572964 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c2907569f0 con 0x55c2903d41c0 2024-08-30T09:09:15.693 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.690+0000 7ff29d65a640 1 --2- 172.21.15.42:0/3767572964 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55c2903d41c0 0x55c29088c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:15.710 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.707+0000 7ff29d65a640 1 --2- 172.21.15.42:0/3767572964 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55c2903d41c0 0x55c29088c350 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a94610fdc8ffe03 server_cookie=a21f26424d978890 in_seq=0 out_seq=0 2024-08-30T09:09:15.712 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.707+0000 7ff277fff640 10 client.26657 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:09:15.821 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.817+0000 7ff277fff640 1 -- 172.21.15.42:0/3767572964 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c2907569f0 con 0x55c2903d41c0 2024-08-30T09:09:15.821 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.817+0000 7ff277fff640 10 client.26657 handle_command_reply: tid=0 2024-08-30T09:09:15.821 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.817+0000 7ff2a6823740 1 client.26657 shutdown 2024-08-30T09:09:15.821 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.817+0000 7ff2a6823740 1 -- 172.21.15.42:0/3767572964 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c29041f4d0 msgr2=0x55c2907f57d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:09:15.821 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.817+0000 7ff2a6823740 1 --2- 172.21.15.42:0/3767572964 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c29041f4d0 0x55c2907f57d0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7ff28800bb50 tx=0x7ff288006540 comp rx=0 tx=0).stop 2024-08-30T09:09:15.821 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.818+0000 7ff2a6823740 1 -- 172.21.15.42:0/3767572964 shutdown_connections 2024-08-30T09:09:15.821 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.818+0000 7ff2a6823740 1 --2- 172.21.15.42:0/3767572964 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55c2903d41c0 0x55c29088c350 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:15.821 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.818+0000 7ff2a6823740 1 --2- 172.21.15.42:0/3767572964 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c2904401b0 0x55c2907f62d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:15.822 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.818+0000 7ff2a6823740 1 --2- 172.21.15.42:0/3767572964 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c29041f4d0 0x55c2907f57d0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:15.822 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.818+0000 7ff2a6823740 1 --2- 172.21.15.42:0/3767572964 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c2904172f0 0x55c2907f5110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:15.822 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.818+0000 7ff2a6823740 1 -- 172.21.15.42:0/3767572964 >> 172.21.15.42:0/3767572964 conn(0x55c290434300 msgr2=0x55c2907f8440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:09:15.822 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.818+0000 7ff277fff640 0 client.26657 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:09:15.822 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.818+0000 7ff2a6823740 1 -- 172.21.15.42:0/3767572964 shutdown_connections 2024-08-30T09:09:15.822 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.818+0000 7ff2a6823740 1 -- 172.21.15.42:0/3767572964 wait complete. 2024-08-30T09:09:15.822 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.818+0000 7ff2a6823740 20 client.26657 trim_cache size 0 max 16384 2024-08-30T09:09:15.822 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-08-30T09:09:15.822 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2024-08-30T09:09:15.822 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-08-30T09:09:15.823 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-08-30T09:09:15.823 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.819+0000 7ff283fff640 1 -- 172.21.15.42:0/1120496319 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7ff2840b4d40 msgr2=0x7ff2840b7250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:09:15.823 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.819+0000 7ff283fff640 1 --2- 172.21.15.42:0/1120496319 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7ff2840b4d40 0x7ff2840b7250 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7ff294005fd0 tx=0x7ff294005c40 comp rx=0 tx=0).stop 2024-08-30T09:09:15.823 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.819+0000 7ff283fff640 1 -- 172.21.15.42:0/1120496319 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff298103d60 msgr2=0x7ff29819af80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:09:15.823 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.819+0000 7ff283fff640 1 --2- 172.21.15.42:0/1120496319 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff298103d60 0x7ff29819af80 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7ff290003d30 tx=0x7ff290003d60 comp rx=0 tx=0).stop 2024-08-30T09:09:15.824 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.819+0000 7ff283fff640 1 -- 172.21.15.42:0/1120496319 shutdown_connections 2024-08-30T09:09:15.824 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.819+0000 7ff283fff640 1 --2- 172.21.15.42:0/1120496319 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7ff2840b4d40 0x7ff2840b7250 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:15.824 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.819+0000 7ff283fff640 1 --2- 172.21.15.42:0/1120496319 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff298103d60 0x7ff29819af80 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:15.824 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.819+0000 7ff283fff640 1 --2- 172.21.15.42:0/1120496319 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff2981033c0 0x7ff29819a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:15.824 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.819+0000 7ff283fff640 1 --2- 172.21.15.42:0/1120496319 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff298102340 0x7ff298199dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:15.824 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.819+0000 7ff283fff640 1 -- 172.21.15.42:0/1120496319 >> 172.21.15.42:0/1120496319 conn(0x7ff2980fdc80 msgr2=0x7ff2981066c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:09:15.824 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.819+0000 7ff283fff640 1 -- 172.21.15.42:0/1120496319 shutdown_connections 2024-08-30T09:09:15.824 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:15.819+0000 7ff283fff640 1 -- 172.21.15.42:0/1120496319 wait complete. 2024-08-30T09:09:15.824 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-08-30T09:09:15.824 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-08-30T09:09:15.953 INFO:teuthology.orchestra.run.smithi042.stderr:progress: 510.0 s, 515.2 tps, lat 189.085 ms stddev 241.965 2024-08-30T09:09:16.343 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T09:09:17.121 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:09:16 smithi042 ceph-mon[18652]: pgmap v628: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 4.7 MiB/s wr, 529 op/s 2024-08-30T09:09:17.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:09:16 smithi190 ceph-mon[22001]: pgmap v628: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 4.7 MiB/s wr, 529 op/s 2024-08-30T09:09:17.290 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.286+0000 7fd876940640 1 -- 172.21.15.42:0/3566266857 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd870108ac0 msgr2=0x7fd87010f2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:09:17.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.286+0000 7fd876940640 1 --2- 172.21.15.42:0/3566266857 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd870108ac0 0x7fd87010f2f0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fd868009ca0 tx=0x7fd86802a8f0 comp rx=0 tx=0).stop 2024-08-30T09:09:17.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.287+0000 7fd876940640 1 -- 172.21.15.42:0/3566266857 shutdown_connections 2024-08-30T09:09:17.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.287+0000 7fd876940640 1 --2- 172.21.15.42:0/3566266857 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd870108170 0x7fd870108580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:17.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.287+0000 7fd876940640 1 --2- 172.21.15.42:0/3566266857 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd870108ac0 0x7fd87010f2f0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:17.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.287+0000 7fd876940640 1 --2- 172.21.15.42:0/3566266857 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd870102880 0x7fd870102c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:17.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.287+0000 7fd876940640 1 -- 172.21.15.42:0/3566266857 >> 172.21.15.42:0/3566266857 conn(0x7fd870079530 msgr2=0x7fd870079940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:09:17.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.291+0000 7fd876940640 1 -- 172.21.15.42:0/3566266857 shutdown_connections 2024-08-30T09:09:17.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.291+0000 7fd876940640 1 -- 172.21.15.42:0/3566266857 wait complete. 2024-08-30T09:09:17.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.292+0000 7fd876940640 1 Processor -- start 2024-08-30T09:09:17.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.293+0000 7fd876940640 1 -- start start 2024-08-30T09:09:17.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.293+0000 7fd876940640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd870102880 0x7fd87010bd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:17.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.296+0000 7fd876940640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd870108170 0x7fd87010c240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:17.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.296+0000 7fd876940640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd870108ac0 0x7fd8701ac400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:17.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.296+0000 7fd86ffff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd870102880 0x7fd87010bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:17.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.296+0000 7fd86ffff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd870102880 0x7fd87010bd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:43248/0 (socket says 172.21.15.42:43248) 2024-08-30T09:09:17.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.296+0000 7fd86f7fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd870108170 0x7fd87010c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:17.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.296+0000 7fd86ffff640 1 -- 172.21.15.42:0/897279249 learned_addr learned my addr 172.21.15.42:0/897279249 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:09:17.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.296+0000 7fd876940640 1 -- 172.21.15.42:0/897279249 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fd8701116d0 con 0x7fd870108ac0 2024-08-30T09:09:17.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.296+0000 7fd876940640 1 -- 172.21.15.42:0/897279249 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd870111550 con 0x7fd870108170 2024-08-30T09:09:17.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.296+0000 7fd876940640 1 -- 172.21.15.42:0/897279249 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fd870111850 con 0x7fd870102880 2024-08-30T09:09:17.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.297+0000 7fd874edd640 1 --2- 172.21.15.42:0/897279249 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd870108ac0 0x7fd8701ac400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:17.308 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.298+0000 7fd86ffff640 1 -- 172.21.15.42:0/897279249 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd870108170 msgr2=0x7fd87010c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:09:17.308 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.298+0000 7fd86ffff640 1 --2- 172.21.15.42:0/897279249 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd870108170 0x7fd87010c240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:17.308 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.298+0000 7fd86ffff640 1 -- 172.21.15.42:0/897279249 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd870108ac0 msgr2=0x7fd8701ac400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:09:17.308 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.298+0000 7fd86ffff640 1 --2- 172.21.15.42:0/897279249 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd870108ac0 0x7fd8701ac400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:17.308 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.298+0000 7fd86ffff640 1 -- 172.21.15.42:0/897279249 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8701ac940 con 0x7fd870102880 2024-08-30T09:09:17.308 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.303+0000 7fd86ffff640 1 --2- 172.21.15.42:0/897279249 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd870102880 0x7fd87010bd00 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fd86000b4a0 tx=0x7fd86000b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:09:17.308 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.304+0000 7fd86d7fa640 1 -- 172.21.15.42:0/897279249 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd860014070 con 0x7fd870102880 2024-08-30T09:09:17.308 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.304+0000 7fd86d7fa640 1 -- 172.21.15.42:0/897279249 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd860010ac0 con 0x7fd870102880 2024-08-30T09:09:17.308 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.304+0000 7fd86d7fa640 1 -- 172.21.15.42:0/897279249 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd860017c60 con 0x7fd870102880 2024-08-30T09:09:17.308 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.305+0000 7fd876940640 1 -- 172.21.15.42:0/897279249 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8701acc30 con 0x7fd870102880 2024-08-30T09:09:17.308 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.305+0000 7fd876940640 1 -- 172.21.15.42:0/897279249 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8701ad120 con 0x7fd870102880 2024-08-30T09:09:17.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.312+0000 7fd87ceb5740 1 Processor -- start 2024-08-30T09:09:17.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.312+0000 7fd87ceb5740 1 -- start start 2024-08-30T09:09:17.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.312+0000 7fd87ceb5740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5626c1a0c350 0x5626c1db2780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:17.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.312+0000 7fd87ceb5740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5626c19e3490 0x5626c1db2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:17.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.312+0000 7fd87ceb5740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5626c19eb670 0x5626c1db3200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:17.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.312+0000 7fd87ceb5740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x5626c1d285f0 con 0x5626c1a0c350 2024-08-30T09:09:17.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.312+0000 7fd87ceb5740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5626c1d17b80 con 0x5626c19e3490 2024-08-30T09:09:17.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.312+0000 7fd87ceb5740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5626c1d1bb30 con 0x5626c19eb670 2024-08-30T09:09:17.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.313+0000 7fd86d7fa640 1 -- 172.21.15.42:0/897279249 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7fd860017070 con 0x7fd870102880 2024-08-30T09:09:17.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.313+0000 7fd86f7fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5626c19eb670 0x5626c1db3200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:17.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.313+0000 7fd86f7fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5626c19eb670 0x5626c1db3200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:43260/0 (socket says 172.21.15.42:43260) 2024-08-30T09:09:17.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.313+0000 7fd86f7fe640 1 -- 172.21.15.42:0/1584031271 learned_addr learned my addr 172.21.15.42:0/1584031271 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:09:17.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.313+0000 7fd86ffff640 1 --2- 172.21.15.42:0/1584031271 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5626c19e3490 0x5626c1db2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:17.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.315+0000 7fd874edd640 1 --2- 172.21.15.42:0/1584031271 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5626c1a0c350 0x5626c1db2780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:17.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.315+0000 7fd86f7fe640 1 -- 172.21.15.42:0/1584031271 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5626c19e3490 msgr2=0x5626c1db2cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:09:17.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.315+0000 7fd86f7fe640 1 --2- 172.21.15.42:0/1584031271 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5626c19e3490 0x5626c1db2cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:17.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.315+0000 7fd86f7fe640 1 -- 172.21.15.42:0/1584031271 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5626c1a0c350 msgr2=0x5626c1db2780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:09:17.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.315+0000 7fd86f7fe640 1 --2- 172.21.15.42:0/1584031271 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5626c1a0c350 0x5626c1db2780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:17.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.315+0000 7fd86f7fe640 1 -- 172.21.15.42:0/1584031271 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5626c1a77930 con 0x5626c19eb670 2024-08-30T09:09:17.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.315+0000 7fd86f7fe640 1 --2- 172.21.15.42:0/1584031271 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5626c19eb670 0x5626c1db3200 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fd86400bc30 tx=0x7fd8640107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=874bbd4cddc533ef server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:09:17.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.319+0000 7fd874edd640 1 --2- 172.21.15.42:0/1584031271 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5626c1a0c350 0x5626c1db2780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-30T09:09:17.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.319+0000 7fd8567fc640 1 -- 172.21.15.42:0/1584031271 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd86401c480 con 0x5626c19eb670 2024-08-30T09:09:17.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.319+0000 7fd8567fc640 1 -- 172.21.15.42:0/1584031271 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd86401cad0 con 0x5626c19eb670 2024-08-30T09:09:17.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.320+0000 7fd87ceb5740 1 -- 172.21.15.42:0/1584031271 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5626c19eb670 msgr2=0x5626c1db3200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:09:17.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.320+0000 7fd87ceb5740 1 --2- 172.21.15.42:0/1584031271 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5626c19eb670 0x5626c1db3200 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fd86400bc30 tx=0x7fd8640107a0 comp rx=0 tx=0).stop 2024-08-30T09:09:17.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.321+0000 7fd86d7fa640 1 --2- 172.21.15.42:0/897279249 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fd8580b4d40 0x7fd8580b7250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:17.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.321+0000 7fd86d7fa640 1 -- 172.21.15.42:0/897279249 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7fd8600e46c0 con 0x7fd870102880 2024-08-30T09:09:17.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.322+0000 7fd87ceb5740 1 -- 172.21.15.42:0/1584031271 shutdown_connections 2024-08-30T09:09:17.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.322+0000 7fd87ceb5740 1 --2- 172.21.15.42:0/1584031271 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5626c19eb670 0x5626c1db3200 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:17.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.322+0000 7fd87ceb5740 1 --2- 172.21.15.42:0/1584031271 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5626c19e3490 0x5626c1db2cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:17.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.322+0000 7fd87ceb5740 1 --2- 172.21.15.42:0/1584031271 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5626c1a0c350 0x5626c1db2780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:17.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.322+0000 7fd87ceb5740 1 -- 172.21.15.42:0/1584031271 >> 172.21.15.42:0/1584031271 conn(0x5626c1a00740 msgr2=0x5626c1db0670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:09:17.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.322+0000 7fd87ceb5740 1 -- 172.21.15.42:0/1584031271 shutdown_connections 2024-08-30T09:09:17.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.322+0000 7fd87ceb5740 1 -- 172.21.15.42:0/1584031271 wait complete. 2024-08-30T09:09:17.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.323+0000 7fd87ceb5740 1 -- start start 2024-08-30T09:09:17.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.323+0000 7fd87ceb5740 1 Processor -- start 2024-08-30T09:09:17.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:09:16 smithi167 ceph-mon[22063]: pgmap v628: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 4.7 MiB/s wr, 529 op/s 2024-08-30T09:09:17.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.326+0000 7fd874edd640 1 --2- 172.21.15.42:0/897279249 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fd8580b4d40 0x7fd8580b7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:17.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.326+0000 7fd87ceb5740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5626c19e3490 0x5626c1dbdf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:17.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.326+0000 7fd87ceb5740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5626c19eb670 0x5626c1dbe490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:17.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.326+0000 7fd87ceb5740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5626c1a0c350 0x5626c1ddb300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:17.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.326+0000 7fd87ceb5740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x5626c1b7a890 con 0x5626c1a0c350 2024-08-30T09:09:17.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.326+0000 7fd87ceb5740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5626c1c9e790 con 0x5626c19eb670 2024-08-30T09:09:17.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.326+0000 7fd87ceb5740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5626c1dd1fc0 con 0x5626c19e3490 2024-08-30T09:09:17.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.326+0000 7fd87ceb5740 10 client.0 _ll_register_callbacks cb 0x5626c1a23ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:09:17.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.328+0000 7fd86ffff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5626c19e3490 0x5626c1dbdf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:17.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.328+0000 7fd86ffff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5626c19e3490 0x5626c1dbdf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:43262/0 (socket says 172.21.15.42:43262) 2024-08-30T09:09:17.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.328+0000 7fd86ffff640 1 -- 172.21.15.42:0/2866696909 learned_addr learned my addr 172.21.15.42:0/2866696909 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:09:17.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.330+0000 7fd86f7fe640 1 --2- 172.21.15.42:0/2866696909 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5626c19eb670 0x5626c1dbe490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:17.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.330+0000 7fd86ffff640 1 -- 172.21.15.42:0/2866696909 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5626c19eb670 msgr2=0x5626c1dbe490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:09:17.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.330+0000 7fd86ffff640 1 --2- 172.21.15.42:0/2866696909 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5626c19eb670 0x5626c1dbe490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:17.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.330+0000 7fd86ffff640 1 -- 172.21.15.42:0/2866696909 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5626c1a0c350 msgr2=0x5626c1ddb300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:09:17.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.330+0000 7fd86ffff640 1 --2- 172.21.15.42:0/2866696909 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5626c1a0c350 0x5626c1ddb300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:17.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.330+0000 7fd86ffff640 1 -- 172.21.15.42:0/2866696909 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5626c1cd5f30 con 0x5626c19e3490 2024-08-30T09:09:17.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.333+0000 7fd86ffff640 1 --2- 172.21.15.42:0/2866696909 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5626c19e3490 0x5626c1dbdf50 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fd8600a7f70 tx=0x7fd8600e7f60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2485162628ce81be server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:09:17.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.335+0000 7fd84f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-08-30T09:09:17.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.335+0000 7fd84f7fe640 1 -- 172.21.15.42:0/2866696909 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd86001e070 con 0x5626c19e3490 2024-08-30T09:09:17.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.335+0000 7fd84f7fe640 1 -- 172.21.15.42:0/2866696909 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7fd860017e00 con 0x5626c19e3490 2024-08-30T09:09:17.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.338+0000 7fd87ceb5740 1 -- 172.21.15.42:0/2866696909 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd86401cad0 con 0x5626c19e3490 2024-08-30T09:09:17.345 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.342+0000 7fd84f7fe640 1 -- 172.21.15.42:0/2866696909 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fd8600aa810 con 0x5626c19e3490 2024-08-30T09:09:17.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.343+0000 7fd87ceb5740 10 client.26656 fetch_fsmap learned FSMap version 37 2024-08-30T09:09:17.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.343+0000 7fd87ceb5740 1 -- 172.21.15.42:0/2866696909 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5626c1ddde00 con 0x5626c19e3490 2024-08-30T09:09:17.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.344+0000 7fd874edd640 1 --2- 172.21.15.42:0/897279249 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fd8580b4d40 0x7fd8580b7250 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fd868009a60 tx=0x7fd868009810 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:09:17.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.346+0000 7fd84f7fe640 1 -- 172.21.15.42:0/2866696909 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd86401cad0 con 0x5626c19e3490 2024-08-30T09:09:17.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.346+0000 7fd84f7fe640 1 -- 172.21.15.42:0/2866696909 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7fd8600e4070 con 0x5626c19e3490 2024-08-30T09:09:17.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.346+0000 7fd87ceb5740 10 client.26656 fetch_fsmap finished waiting for FSMap version 37 2024-08-30T09:09:17.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.346+0000 7fd87ceb5740 10 client.26656 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-30T09:09:17.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.346+0000 7fd87ceb5740 20 client.26656 populate_metadata read hostname 'smithi042' 2024-08-30T09:09:17.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.346+0000 7fd87ceb5740 1 --2- 172.21.15.42:0/2866696909 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x5626c19cacd0 0x5626c1de1740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:17.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.346+0000 7fd87ceb5740 4 client.26656 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-30T09:09:17.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.346+0000 7fd87ceb5740 1 -- 172.21.15.42:0/2866696909 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5626c1d2a540 con 0x5626c19cacd0 2024-08-30T09:09:17.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.346+0000 7fd86f7fe640 1 --2- 172.21.15.42:0/2866696909 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x5626c19cacd0 0x5626c1de1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:17.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.347+0000 7fd84f7fe640 10 client.26656 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:09:17.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.347+0000 7fd86f7fe640 1 --2- 172.21.15.42:0/2866696909 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x5626c19cacd0 0x5626c1de1740 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8cd02bc43a944ea3 server_cookie=b0a8797b46e01299 in_seq=0 out_seq=0 2024-08-30T09:09:17.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.353+0000 7fd84f7fe640 1 -- 172.21.15.42:0/2866696909 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5626c1d2a540 con 0x5626c19cacd0 2024-08-30T09:09:17.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.353+0000 7fd84f7fe640 10 client.26656 handle_command_reply: tid=0 2024-08-30T09:09:17.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.353+0000 7fd87ceb5740 1 client.26656 shutdown 2024-08-30T09:09:17.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.357+0000 7fd87ceb5740 1 -- 172.21.15.42:0/2866696909 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5626c19e3490 msgr2=0x5626c1dbdf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:09:17.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.357+0000 7fd87ceb5740 1 --2- 172.21.15.42:0/2866696909 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5626c19e3490 0x5626c1dbdf50 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fd8600a7f70 tx=0x7fd8600e7f60 comp rx=0 tx=0).stop 2024-08-30T09:09:17.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.358+0000 7fd87ceb5740 1 -- 172.21.15.42:0/2866696909 shutdown_connections 2024-08-30T09:09:17.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.358+0000 7fd87ceb5740 1 --2- 172.21.15.42:0/2866696909 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x5626c19cacd0 0x5626c1de1740 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:17.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.358+0000 7fd87ceb5740 1 --2- 172.21.15.42:0/2866696909 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5626c1a0c350 0x5626c1ddb300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:17.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.358+0000 7fd87ceb5740 1 --2- 172.21.15.42:0/2866696909 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5626c19eb670 0x5626c1dbe490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:17.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.358+0000 7fd87ceb5740 1 --2- 172.21.15.42:0/2866696909 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5626c19e3490 0x5626c1dbdf50 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:17.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.358+0000 7fd87ceb5740 1 -- 172.21.15.42:0/2866696909 >> 172.21.15.42:0/2866696909 conn(0x5626c1a00740 msgr2=0x5626c1db5960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:09:17.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.358+0000 7fd84f7fe640 0 client.26656 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:09:17.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.358+0000 7fd87ceb5740 1 -- 172.21.15.42:0/2866696909 shutdown_connections 2024-08-30T09:09:17.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.359+0000 7fd87ceb5740 1 -- 172.21.15.42:0/2866696909 wait complete. 2024-08-30T09:09:17.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.364+0000 7fd87ceb5740 20 client.26656 trim_cache size 0 max 16384 2024-08-30T09:09:17.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.402+0000 7fd87ceb5740 1 Processor -- start 2024-08-30T09:09:17.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.404+0000 7fd87ceb5740 1 -- start start 2024-08-30T09:09:17.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.404+0000 7fd87ceb5740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5626c19e3490 0x5626c1db9730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:17.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.404+0000 7fd87ceb5740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5626c19eb670 0x5626c1db5f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:17.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.404+0000 7fd87ceb5740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5626c1a0c350 0x5626c1dbb140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:17.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.404+0000 7fd87ceb5740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fd8701b1090 con 0x5626c19eb670 2024-08-30T09:09:17.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.404+0000 7fd87ceb5740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd8701ae960 con 0x5626c19e3490 2024-08-30T09:09:17.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.404+0000 7fd87ceb5740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fd870074790 con 0x5626c1a0c350 2024-08-30T09:09:17.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.404+0000 7fd86f7fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5626c19eb670 0x5626c1db5f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:17.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.405+0000 7fd86f7fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5626c19eb670 0x5626c1db5f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35098/0 (socket says 172.21.15.42:35098) 2024-08-30T09:09:17.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.405+0000 7fd86f7fe640 1 -- 172.21.15.42:0/1093780508 learned_addr learned my addr 172.21.15.42:0/1093780508 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:09:17.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.405+0000 7fd874edd640 1 --2- 172.21.15.42:0/1093780508 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5626c1a0c350 0x5626c1dbb140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:17.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.406+0000 7fd86ffff640 1 --2- 172.21.15.42:0/1093780508 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5626c19e3490 0x5626c1db9730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:17.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.406+0000 7fd874edd640 1 -- 172.21.15.42:0/1093780508 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5626c19e3490 msgr2=0x5626c1db9730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:09:17.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.406+0000 7fd874edd640 1 --2- 172.21.15.42:0/1093780508 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5626c19e3490 0x5626c1db9730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:17.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.406+0000 7fd874edd640 1 -- 172.21.15.42:0/1093780508 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5626c19eb670 msgr2=0x5626c1db5f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:09:17.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.406+0000 7fd874edd640 1 --2- 172.21.15.42:0/1093780508 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5626c19eb670 0x5626c1db5f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:17.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.406+0000 7fd874edd640 1 -- 172.21.15.42:0/1093780508 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5626c1dddc10 con 0x5626c1a0c350 2024-08-30T09:09:17.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.409+0000 7fd874edd640 1 --2- 172.21.15.42:0/1093780508 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5626c1a0c350 0x5626c1dbb140 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fd868006010 tx=0x7fd868040000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1e612dd7828ce79b server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:09:17.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.414+0000 7fd855ffb640 1 -- 172.21.15.42:0/1093780508 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd868030d40 con 0x5626c1a0c350 2024-08-30T09:09:17.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.414+0000 7fd855ffb640 1 -- 172.21.15.42:0/1093780508 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd868041030 con 0x5626c1a0c350 2024-08-30T09:09:17.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.414+0000 7fd855ffb640 1 -- 172.21.15.42:0/1093780508 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8680496a0 con 0x5626c1a0c350 2024-08-30T09:09:17.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.416+0000 7fd87ceb5740 1 -- 172.21.15.42:0/1093780508 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5626c1a0c350 msgr2=0x5626c1dbb140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:09:17.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.416+0000 7fd87ceb5740 1 --2- 172.21.15.42:0/1093780508 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5626c1a0c350 0x5626c1dbb140 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fd868006010 tx=0x7fd868040000 comp rx=0 tx=0).stop 2024-08-30T09:09:17.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.417+0000 7fd87ceb5740 1 -- 172.21.15.42:0/1093780508 shutdown_connections 2024-08-30T09:09:17.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.417+0000 7fd87ceb5740 1 --2- 172.21.15.42:0/1093780508 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5626c1a0c350 0x5626c1dbb140 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:17.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.417+0000 7fd87ceb5740 1 --2- 172.21.15.42:0/1093780508 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5626c19eb670 0x5626c1db5f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:17.421 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.417+0000 7fd87ceb5740 1 --2- 172.21.15.42:0/1093780508 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5626c19e3490 0x5626c1db9730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:17.421 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.417+0000 7fd87ceb5740 1 -- 172.21.15.42:0/1093780508 >> 172.21.15.42:0/1093780508 conn(0x5626c1a00740 msgr2=0x5626c1db5880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:09:17.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.418+0000 7fd87ceb5740 1 -- 172.21.15.42:0/1093780508 shutdown_connections 2024-08-30T09:09:17.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.420+0000 7fd87ceb5740 1 -- 172.21.15.42:0/1093780508 wait complete. 2024-08-30T09:09:17.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.420+0000 7fd87ceb5740 1 -- start start 2024-08-30T09:09:17.424 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.421+0000 7fd87ceb5740 1 Processor -- start 2024-08-30T09:09:17.425 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.421+0000 7fd87ceb5740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5626c19e3490 0x5626c1dd3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:17.425 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.422+0000 7fd87ceb5740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5626c19eb670 0x5626c1dd37c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:17.425 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.422+0000 7fd86ffff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5626c19e3490 0x5626c1dd3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:17.425 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.422+0000 7fd86ffff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5626c19e3490 0x5626c1dd3280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:58592/0 (socket says 172.21.15.42:58592) 2024-08-30T09:09:17.425 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.422+0000 7fd86ffff640 1 -- 172.21.15.42:0/864108337 learned_addr learned my addr 172.21.15.42:0/864108337 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:09:17.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.422+0000 7fd87ceb5740 1 --2- 172.21.15.42:0/864108337 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5626c1a0c350 0x5626c1dd3d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:17.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.422+0000 7fd87ceb5740 1 -- 172.21.15.42:0/864108337 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fd870074e00 con 0x5626c19eb670 2024-08-30T09:09:17.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.423+0000 7fd874edd640 1 --2- 172.21.15.42:0/864108337 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5626c1a0c350 0x5626c1dd3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:17.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.423+0000 7fd86f7fe640 1 --2- 172.21.15.42:0/864108337 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5626c19eb670 0x5626c1dd37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:17.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.422+0000 7fd87ceb5740 1 -- 172.21.15.42:0/864108337 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd8701ae290 con 0x5626c19e3490 2024-08-30T09:09:17.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.423+0000 7fd87ceb5740 1 -- 172.21.15.42:0/864108337 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fd8701ad4f0 con 0x5626c1a0c350 2024-08-30T09:09:17.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.423+0000 7fd86ffff640 1 -- 172.21.15.42:0/864108337 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5626c1a0c350 msgr2=0x5626c1dd3d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:09:17.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.423+0000 7fd86ffff640 1 --2- 172.21.15.42:0/864108337 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5626c1a0c350 0x5626c1dd3d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:17.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.423+0000 7fd86ffff640 1 -- 172.21.15.42:0/864108337 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5626c19eb670 msgr2=0x5626c1dd37c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:09:17.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.423+0000 7fd86ffff640 1 --2- 172.21.15.42:0/864108337 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5626c19eb670 0x5626c1dd37c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:17.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.423+0000 7fd86ffff640 1 -- 172.21.15.42:0/864108337 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5626c1e4b190 con 0x5626c19e3490 2024-08-30T09:09:17.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.424+0000 7fd87ceb5740 10 client.0 _ll_register_callbacks cb 0x7fd86000b3f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:09:17.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.424+0000 7fd86ffff640 1 --2- 172.21.15.42:0/864108337 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5626c19e3490 0x5626c1dd3280 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fd8600e1e20 tx=0x7fd8600176c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=adbf95f4431ff3f1 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:09:17.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.424+0000 7fd84f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-30T09:09:17.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.424+0000 7fd84f7fe640 1 -- 172.21.15.42:0/864108337 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd86001e070 con 0x5626c19e3490 2024-08-30T09:09:17.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.424+0000 7fd87ceb5740 1 -- 172.21.15.42:0/864108337 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd868041030 con 0x5626c19e3490 2024-08-30T09:09:17.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.424+0000 7fd84f7fe640 1 -- 172.21.15.42:0/864108337 <== mon.2 v2:172.21.15.167:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7fd8600a72f0 con 0x5626c19e3490 2024-08-30T09:09:17.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.424+0000 7fd84f7fe640 1 -- 172.21.15.42:0/864108337 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fd8600ab400 con 0x5626c19e3490 2024-08-30T09:09:17.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.425+0000 7fd87ceb5740 10 client.26675 fetch_fsmap learned FSMap version 37 2024-08-30T09:09:17.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.425+0000 7fd87ceb5740 1 -- 172.21.15.42:0/864108337 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({fsmap=37}) -- 0x5626c1e4c340 con 0x5626c19e3490 2024-08-30T09:09:17.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.425+0000 7fd84f7fe640 1 -- 172.21.15.42:0/864108337 <== mon.2 v2:172.21.15.167:3300/0 4 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7fd8600e4040 con 0x5626c19e3490 2024-08-30T09:09:17.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.425+0000 7fd87ceb5740 10 client.26675 fetch_fsmap finished waiting for FSMap version 37 2024-08-30T09:09:17.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.425+0000 7fd87ceb5740 10 client.26675 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-30T09:09:17.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.425+0000 7fd87ceb5740 20 client.26675 populate_metadata read hostname 'smithi042' 2024-08-30T09:09:17.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.425+0000 7fd87ceb5740 1 --2- 172.21.15.42:0/864108337 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x5626c19a0080 0x5626c1e4eac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:17.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.425+0000 7fd87ceb5740 4 client.26675 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-08-30T09:09:17.430 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.425+0000 7fd87ceb5740 1 -- 172.21.15.42:0/864108337 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7fd868041030 con 0x5626c19a0080 2024-08-30T09:09:17.430 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.426+0000 7fd86f7fe640 1 --2- 172.21.15.42:0/864108337 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x5626c19a0080 0x5626c1e4eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:17.430 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.426+0000 7fd86f7fe640 1 --2- 172.21.15.42:0/864108337 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x5626c19a0080 0x5626c1e4eac0 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5622c845328ad38b server_cookie=a011019af8bf3789 in_seq=0 out_seq=0 2024-08-30T09:09:17.430 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.427+0000 7fd84f7fe640 10 client.26675 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:09:17.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.428+0000 7fd84f7fe640 1 -- 172.21.15.42:0/864108337 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7fd868041030 con 0x5626c19a0080 2024-08-30T09:09:17.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.428+0000 7fd84f7fe640 10 client.26675 handle_command_reply: tid=0 2024-08-30T09:09:17.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.428+0000 7fd87ceb5740 1 client.26675 shutdown 2024-08-30T09:09:17.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.428+0000 7fd87ceb5740 1 -- 172.21.15.42:0/864108337 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5626c19e3490 msgr2=0x5626c1dd3280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:09:17.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.428+0000 7fd87ceb5740 1 --2- 172.21.15.42:0/864108337 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5626c19e3490 0x5626c1dd3280 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fd8600e1e20 tx=0x7fd8600176c0 comp rx=0 tx=0).stop 2024-08-30T09:09:17.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.429+0000 7fd87ceb5740 1 -- 172.21.15.42:0/864108337 shutdown_connections 2024-08-30T09:09:17.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.429+0000 7fd87ceb5740 1 --2- 172.21.15.42:0/864108337 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x5626c19a0080 0x5626c1e4eac0 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:17.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.429+0000 7fd84f7fe640 0 client.26675 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:09:17.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.429+0000 7fd87ceb5740 1 --2- 172.21.15.42:0/864108337 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5626c1a0c350 0x5626c1dd3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:17.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.429+0000 7fd87ceb5740 1 --2- 172.21.15.42:0/864108337 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5626c19eb670 0x5626c1dd37c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:17.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.429+0000 7fd87ceb5740 1 --2- 172.21.15.42:0/864108337 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5626c19e3490 0x5626c1dd3280 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:17.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.429+0000 7fd87ceb5740 1 -- 172.21.15.42:0/864108337 >> 172.21.15.42:0/864108337 conn(0x5626c1a00740 msgr2=0x5626c1db37b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:09:17.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.429+0000 7fd87ceb5740 1 -- 172.21.15.42:0/864108337 shutdown_connections 2024-08-30T09:09:17.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.429+0000 7fd87ceb5740 1 -- 172.21.15.42:0/864108337 wait complete. 2024-08-30T09:09:17.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.429+0000 7fd87ceb5740 20 client.26675 trim_cache size 0 max 16384 2024-08-30T09:09:17.433 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-08-30T09:09:17.433 INFO:teuthology.orchestra.run.smithi042.stdout: "return_code": 0, 2024-08-30T09:09:17.433 INFO:teuthology.orchestra.run.smithi042.stdout: "scrub_tag": "37f2f28f-496f-41a6-b4cf-8846fb3f43ab", 2024-08-30T09:09:17.434 INFO:teuthology.orchestra.run.smithi042.stdout: "mode": "asynchronous" 2024-08-30T09:09:17.434 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-08-30T09:09:17.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.430+0000 7fd876940640 1 -- 172.21.15.42:0/897279249 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fd8580b4d40 msgr2=0x7fd8580b7250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:09:17.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.430+0000 7fd876940640 1 --2- 172.21.15.42:0/897279249 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fd8580b4d40 0x7fd8580b7250 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fd868009a60 tx=0x7fd868009810 comp rx=0 tx=0).stop 2024-08-30T09:09:17.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.430+0000 7fd876940640 1 -- 172.21.15.42:0/897279249 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd870102880 msgr2=0x7fd87010bd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:09:17.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.430+0000 7fd876940640 1 --2- 172.21.15.42:0/897279249 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd870102880 0x7fd87010bd00 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fd86000b4a0 tx=0x7fd86000b960 comp rx=0 tx=0).stop 2024-08-30T09:09:17.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.430+0000 7fd876940640 1 -- 172.21.15.42:0/897279249 shutdown_connections 2024-08-30T09:09:17.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.431+0000 7fd876940640 1 --2- 172.21.15.42:0/897279249 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fd8580b4d40 0x7fd8580b7250 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:17.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.431+0000 7fd876940640 1 --2- 172.21.15.42:0/897279249 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd870108ac0 0x7fd8701ac400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:17.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.431+0000 7fd876940640 1 --2- 172.21.15.42:0/897279249 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd870108170 0x7fd87010c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:17.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.431+0000 7fd876940640 1 --2- 172.21.15.42:0/897279249 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd870102880 0x7fd87010bd00 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:17.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.431+0000 7fd876940640 1 -- 172.21.15.42:0/897279249 >> 172.21.15.42:0/897279249 conn(0x7fd870079530 msgr2=0x7fd8701061c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:09:17.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.431+0000 7fd876940640 1 -- 172.21.15.42:0/897279249 shutdown_connections 2024-08-30T09:09:17.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:17.431+0000 7fd876940640 1 -- 172.21.15.42:0/897279249 wait complete. 2024-08-30T09:09:17.827 INFO:journalctl@ceph.mds.b.smithi167.stdout:Aug 30 09:09:17 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mds-b[52473]: 2024-08-30T09:09:17.469+0000 7f9d3fee4640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000000f [...2,head] /volumes/qa/sv_1/d3168875-d2e5-410d-8754-72d29955777b/postgres/data/pg_subtrans/ auth{1=1} v2355 ap=2 f(v9 m2024-08-30T09:07:25.092709+0000 1=1+0) n(v23 rc2024-08-30T09:07:25.092709+0000 b212992 2=1+1) (inest sync->mix g=1) caps={15402=pAsLsXsFs/pAsLsXsFsxcral@24} mcw={1=AsLsXsFs} | dirtyscattered=0 request=0 lock=0 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x555a32f6d080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x1000000000f:[<0x10000000006/pg_subtrans v284>,<0x10000000005/data v125>,<0x100000001fb/postgres v31>,<0x100000001fa/d3168875-d2e5-410d-8754-72d29955777b v45>,<0x10000000001/sv_1 v63>,<0x10000000000/qa v49>,<0x1/volumes v44>]//[]","memoryvalue":"(2)0x1000000000f:[<0x10000000006/pg_subtrans v2355>,<0x10000000005/data v744>,<0x100000001fb/postgres v450>,<0x100000001fa/d3168875-d2e5-410d-8754-72d29955777b v470>,<0x10000000001/sv_1 v493>,<0x10000000000/qa v474>,<0x1/volumes v436>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-08-30T09:07:25.039710+0000 9=9+0)","ondisk_value.rstat":"n(v0 rc2024-08-30T09:07:25.092709+0000 b212992 2=1+1)","memory_value.dirstat":"f(v9 m2024-08-30T09:07:25.092709+0000 1=1+0)","memory_value.rstat":"n(v23 rc2024-08-30T09:07:25.092709+0000 b212992 2=1+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-08-30T09:09:18.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:09:17 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-30T09:09:18.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:09:17 smithi042 ceph-mon[18652]: scrub queued for path: / 2024-08-30T09:09:18.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:09:17 smithi042 ceph-mon[18652]: scrub summary: idle+waiting paths [/] 2024-08-30T09:09:18.134 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:09:17 smithi042 ceph-mon[18652]: scrub summary: active paths [/] 2024-08-30T09:09:18.134 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:09:17 smithi042 ceph-mon[18652]: scrub summary: active paths [/] 2024-08-30T09:09:18.134 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:09:17 smithi042 ceph-mon[18652]: scrub summary: idle+waiting paths [/] 2024-08-30T09:09:18.134 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:09:17 smithi042 ceph-mon[18652]: scrub summary: active paths [/] 2024-08-30T09:09:18.134 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:09:17 smithi042 ceph-mon[18652]: scrub summary: idle+waiting paths [/] 2024-08-30T09:09:18.134 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:09:17 smithi042 ceph-mon[18652]: scrub summary: active paths [/] 2024-08-30T09:09:18.134 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:09:17 smithi042 ceph-mon[18652]: scrub summary: idle+waiting paths [/] 2024-08-30T09:09:18.134 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:09:17 smithi042 ceph-mon[18652]: scrub summary: active paths [/] 2024-08-30T09:09:18.134 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:09:17 smithi042 ceph-mon[18652]: Scrub error on inode 0x1000000000f (/volumes/qa/sv_1/d3168875-d2e5-410d-8754-72d29955777b/postgres/data/pg_subtrans) see mds.b log and `damage ls` output for details 2024-08-30T09:09:18.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:09:17 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-30T09:09:18.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:09:17 smithi190 ceph-mon[22001]: scrub queued for path: / 2024-08-30T09:09:18.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:09:17 smithi190 ceph-mon[22001]: scrub summary: idle+waiting paths [/] 2024-08-30T09:09:18.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:09:17 smithi190 ceph-mon[22001]: scrub summary: active paths [/] 2024-08-30T09:09:18.187 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:09:17 smithi190 ceph-mon[22001]: scrub summary: active paths [/] 2024-08-30T09:09:18.187 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:09:17 smithi190 ceph-mon[22001]: scrub summary: idle+waiting paths [/] 2024-08-30T09:09:18.187 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:09:17 smithi190 ceph-mon[22001]: scrub summary: active paths [/] 2024-08-30T09:09:18.187 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:09:17 smithi190 ceph-mon[22001]: scrub summary: idle+waiting paths [/] 2024-08-30T09:09:18.187 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:09:17 smithi190 ceph-mon[22001]: scrub summary: active paths [/] 2024-08-30T09:09:18.187 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:09:17 smithi190 ceph-mon[22001]: scrub summary: idle+waiting paths [/] 2024-08-30T09:09:18.187 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:09:17 smithi190 ceph-mon[22001]: scrub summary: active paths [/] 2024-08-30T09:09:18.187 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:09:17 smithi190 ceph-mon[22001]: Scrub error on inode 0x1000000000f (/volumes/qa/sv_1/d3168875-d2e5-410d-8754-72d29955777b/postgres/data/pg_subtrans) see mds.b log and `damage ls` output for details 2024-08-30T09:09:18.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:09:17 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-30T09:09:18.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:09:17 smithi167 ceph-mon[22063]: scrub queued for path: / 2024-08-30T09:09:18.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:09:17 smithi167 ceph-mon[22063]: scrub summary: idle+waiting paths [/] 2024-08-30T09:09:18.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:09:17 smithi167 ceph-mon[22063]: scrub summary: active paths [/] 2024-08-30T09:09:18.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:09:17 smithi167 ceph-mon[22063]: scrub summary: active paths [/] 2024-08-30T09:09:18.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:09:17 smithi167 ceph-mon[22063]: scrub summary: idle+waiting paths [/] 2024-08-30T09:09:18.328 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:09:17 smithi167 ceph-mon[22063]: scrub summary: active paths [/] 2024-08-30T09:09:18.328 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:09:17 smithi167 ceph-mon[22063]: scrub summary: idle+waiting paths [/] 2024-08-30T09:09:18.328 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:09:17 smithi167 ceph-mon[22063]: scrub summary: active paths [/] 2024-08-30T09:09:18.328 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:09:17 smithi167 ceph-mon[22063]: scrub summary: idle+waiting paths [/] 2024-08-30T09:09:18.328 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:09:17 smithi167 ceph-mon[22063]: scrub summary: active paths [/] 2024-08-30T09:09:18.328 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:09:17 smithi167 ceph-mon[22063]: Scrub error on inode 0x1000000000f (/volumes/qa/sv_1/d3168875-d2e5-410d-8754-72d29955777b/postgres/data/pg_subtrans) see mds.b log and `damage ls` output for details 2024-08-30T09:09:18.770 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-30T09:09:19.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:09:18 smithi042 ceph-mon[18652]: pgmap v629: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 5.2 MiB/s wr, 566 op/s 2024-08-30T09:09:19.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:09:18 smithi042 ceph-mon[18652]: scrub summary: active paths [/] 2024-08-30T09:09:19.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:09:18 smithi042 ceph-mon[18652]: scrub summary: idle+waiting paths [/] 2024-08-30T09:09:19.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:09:18 smithi042 ceph-mon[18652]: scrub summary: active paths [/] 2024-08-30T09:09:19.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:09:18 smithi042 ceph-mon[18652]: scrub summary: idle+waiting paths [/] 2024-08-30T09:09:19.134 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:09:18 smithi042 ceph-mon[18652]: scrub summary: active paths [/] 2024-08-30T09:09:19.134 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:09:18 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T09:09:19.134 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:09:18 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-30T09:09:19.134 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:09:18 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T09:09:19.134 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:09:18 smithi042 ceph-mon[18652]: scrub summary: idle+waiting paths [/] 2024-08-30T09:09:19.134 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:09:18 smithi042 ceph-mon[18652]: scrub summary: idle+waiting paths [/] 2024-08-30T09:09:19.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:09:18 smithi190 ceph-mon[22001]: pgmap v629: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 5.2 MiB/s wr, 566 op/s 2024-08-30T09:09:19.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:09:18 smithi190 ceph-mon[22001]: scrub summary: active paths [/] 2024-08-30T09:09:19.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:09:18 smithi190 ceph-mon[22001]: scrub summary: idle+waiting paths [/] 2024-08-30T09:09:19.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:09:18 smithi190 ceph-mon[22001]: scrub summary: active paths [/] 2024-08-30T09:09:19.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:09:18 smithi190 ceph-mon[22001]: scrub summary: idle+waiting paths [/] 2024-08-30T09:09:19.187 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:09:18 smithi190 ceph-mon[22001]: scrub summary: active paths [/] 2024-08-30T09:09:19.187 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:09:18 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T09:09:19.187 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:09:18 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-30T09:09:19.187 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:09:18 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T09:09:19.187 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:09:18 smithi190 ceph-mon[22001]: scrub summary: idle+waiting paths [/] 2024-08-30T09:09:19.187 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:09:18 smithi190 ceph-mon[22001]: scrub summary: idle+waiting paths [/] 2024-08-30T09:09:19.321 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T09:09:19.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:09:18 smithi167 ceph-mon[22063]: pgmap v629: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 5.2 MiB/s wr, 566 op/s 2024-08-30T09:09:19.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:09:18 smithi167 ceph-mon[22063]: scrub summary: active paths [/] 2024-08-30T09:09:19.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:09:18 smithi167 ceph-mon[22063]: scrub summary: idle+waiting paths [/] 2024-08-30T09:09:19.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:09:18 smithi167 ceph-mon[22063]: scrub summary: active paths [/] 2024-08-30T09:09:19.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:09:18 smithi167 ceph-mon[22063]: scrub summary: idle+waiting paths [/] 2024-08-30T09:09:19.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:09:18 smithi167 ceph-mon[22063]: scrub summary: active paths [/] 2024-08-30T09:09:19.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:09:18 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T09:09:19.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:09:18 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-30T09:09:19.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:09:18 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T09:09:19.328 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:09:18 smithi167 ceph-mon[22063]: scrub summary: idle+waiting paths [/] 2024-08-30T09:09:19.328 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:09:18 smithi167 ceph-mon[22063]: scrub summary: idle+waiting paths [/] 2024-08-30T09:09:19.949 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:19.944+0000 7f1257fff640 1 -- 172.21.15.42:0/2879350620 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f125811c940 msgr2=0x7f125811ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:09:19.949 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:19.944+0000 7f1257fff640 1 --2- 172.21.15.42:0/2879350620 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f125811c940 0x7f125811ed60 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f1248009900 tx=0x7f124802a410 comp rx=0 tx=0).stop 2024-08-30T09:09:19.949 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:19.946+0000 7f1257fff640 1 -- 172.21.15.42:0/2879350620 shutdown_connections 2024-08-30T09:09:19.949 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:19.946+0000 7f1257fff640 1 --2- 172.21.15.42:0/2879350620 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f125811c940 0x7f125811ed60 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:19.949 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:19.946+0000 7f1257fff640 1 --2- 172.21.15.42:0/2879350620 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f125810d990 0x7f125810dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:19.949 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:19.946+0000 7f1257fff640 1 --2- 172.21.15.42:0/2879350620 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1258111a70 0x7f125810d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:19.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:19.946+0000 7f1257fff640 1 -- 172.21.15.42:0/2879350620 >> 172.21.15.42:0/2879350620 conn(0x7f125806e790 msgr2=0x7f125806ebd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:09:19.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:19.947+0000 7f1257fff640 1 -- 172.21.15.42:0/2879350620 shutdown_connections 2024-08-30T09:09:19.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:19.947+0000 7f1257fff640 1 -- 172.21.15.42:0/2879350620 wait complete. 2024-08-30T09:09:19.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:19.948+0000 7f1257fff640 1 Processor -- start 2024-08-30T09:09:19.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:19.949+0000 7f1257fff640 1 -- start start 2024-08-30T09:09:19.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:19.949+0000 7f1257fff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f125810d990 0x7f125811c170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:19.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:19.949+0000 7f1257fff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1258111a70 0x7f1258115440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:19.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:19.950+0000 7f1257fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f125811c940 0x7f1258115980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:19.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:19.950+0000 7f1257fff640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f12581224c0 con 0x7f125811c940 2024-08-30T09:09:19.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:19.950+0000 7f1257fff640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f1258122340 con 0x7f1258111a70 2024-08-30T09:09:19.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:19.950+0000 7f1257fff640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f1258122640 con 0x7f125810d990 2024-08-30T09:09:19.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:19.950+0000 7f12567fc640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1258111a70 0x7f1258115440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:19.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:19.950+0000 7f1256ffd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f125810d990 0x7f125811c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:19.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:19.950+0000 7f1256ffd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f125810d990 0x7f125811c170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:55096/0 (socket says 172.21.15.42:55096) 2024-08-30T09:09:19.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:19.950+0000 7f1256ffd640 1 -- 172.21.15.42:0/2966781866 learned_addr learned my addr 172.21.15.42:0/2966781866 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:09:19.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:19.951+0000 7f12577fe640 1 --2- 172.21.15.42:0/2966781866 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f125811c940 0x7f1258115980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:19.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:19.951+0000 7f1256ffd640 1 -- 172.21.15.42:0/2966781866 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1258111a70 msgr2=0x7f1258115440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:09:19.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:19.951+0000 7f1256ffd640 1 --2- 172.21.15.42:0/2966781866 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1258111a70 0x7f1258115440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:19.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:19.951+0000 7f1256ffd640 1 -- 172.21.15.42:0/2966781866 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f125811c940 msgr2=0x7f1258115980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:09:19.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:19.951+0000 7f1256ffd640 1 --2- 172.21.15.42:0/2966781866 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f125811c940 0x7f1258115980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:19.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:19.951+0000 7f1256ffd640 1 -- 172.21.15.42:0/2966781866 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12581161a0 con 0x7f125810d990 2024-08-30T09:09:19.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:19.954+0000 7f1256ffd640 1 --2- 172.21.15.42:0/2966781866 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f125810d990 0x7f125811c170 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f124c00e9c0 tx=0x7f124c00ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:09:19.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:19.955+0000 7f123ffff640 1 -- 172.21.15.42:0/2966781866 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f124c00cd80 con 0x7f125810d990 2024-08-30T09:09:19.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:19.955+0000 7f123ffff640 1 -- 172.21.15.42:0/2966781866 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f124c0035f0 con 0x7f125810d990 2024-08-30T09:09:19.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:19.955+0000 7f123ffff640 1 -- 172.21.15.42:0/2966781866 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f124c010690 con 0x7f125810d990 2024-08-30T09:09:19.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:19.955+0000 7f1257fff640 1 -- 172.21.15.42:0/2966781866 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f125811ae40 con 0x7f125810d990 2024-08-30T09:09:19.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:19.955+0000 7f1257fff640 1 -- 172.21.15.42:0/2966781866 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f125811b350 con 0x7f125810d990 2024-08-30T09:09:19.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:19.959+0000 7f1263963740 1 Processor -- start 2024-08-30T09:09:19.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:19.959+0000 7f1263963740 1 -- start start 2024-08-30T09:09:19.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:19.960+0000 7f1263963740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557413d24070 0x5574140cc160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:19.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:19.960+0000 7f1263963740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557413cfb1b0 0x5574140c9660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:19.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:19.960+0000 7f1263963740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557413d03390 0x5574140c9da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:19.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:19.960+0000 7f1263963740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x557414040630 con 0x557413d24070 2024-08-30T09:09:19.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:19.960+0000 7f1263963740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55741402fbc0 con 0x557413d03390 2024-08-30T09:09:19.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:19.960+0000 7f1263963740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x557414033b70 con 0x557413cfb1b0 2024-08-30T09:09:19.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:19.960+0000 7f12567fc640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557413d03390 0x5574140c9da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:19.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:19.960+0000 7f12567fc640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557413d03390 0x5574140c9da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:46100/0 (socket says 172.21.15.42:46100) 2024-08-30T09:09:19.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:19.960+0000 7f12567fc640 1 -- 172.21.15.42:0/2906766201 learned_addr learned my addr 172.21.15.42:0/2906766201 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:09:19.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:19.960+0000 7f12577fe640 1 --2- 172.21.15.42:0/2906766201 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557413d24070 0x5574140cc160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:19.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:19.962+0000 7f123ffff640 1 -- 172.21.15.42:0/2966781866 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7f124c0037a0 con 0x7f125810d990 2024-08-30T09:09:19.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:19.963+0000 7f123ffff640 1 --2- 172.21.15.42:0/2966781866 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f12400b4d40 0x7f12400b7250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:19.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:19.963+0000 7f123ffff640 1 -- 172.21.15.42:0/2966781866 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f124c0e58b0 con 0x7f125810d990 2024-08-30T09:09:19.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:19.964+0000 7f1256ffd640 1 --2- 172.21.15.42:0/2906766201 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557413cfb1b0 0x5574140c9660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:19.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:19.964+0000 7f12567fc640 1 -- 172.21.15.42:0/2906766201 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557413cfb1b0 msgr2=0x5574140c9660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:09:19.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:19.964+0000 7f12567fc640 1 --2- 172.21.15.42:0/2906766201 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557413cfb1b0 0x5574140c9660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:19.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:19.964+0000 7f12567fc640 1 -- 172.21.15.42:0/2906766201 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557413d24070 msgr2=0x5574140cc160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:09:19.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:19.964+0000 7f12567fc640 1 --2- 172.21.15.42:0/2906766201 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557413d24070 0x5574140cc160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:19.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:19.964+0000 7f12567fc640 1 -- 172.21.15.42:0/2906766201 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557413d8f650 con 0x557413d03390 2024-08-30T09:09:19.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:19.965+0000 7f12577fe640 1 --2- 172.21.15.42:0/2966781866 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f12400b4d40 0x7f12400b7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:19.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:19.965+0000 7f12577fe640 1 --2- 172.21.15.42:0/2966781866 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f12400b4d40 0x7f12400b7250 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f1248031440 tx=0x7f1248009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:09:19.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:19.975+0000 7f12567fc640 1 --2- 172.21.15.42:0/2906766201 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557413d03390 0x5574140c9da0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f1250009fa0 tx=0x7f125000a890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f3a2f5d4d0eada9e server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:09:19.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:19.980+0000 7f1257fff640 1 -- 172.21.15.42:0/2906766201 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12500357a0 con 0x557413d03390 2024-08-30T09:09:19.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:19.980+0000 7f1257fff640 1 -- 172.21.15.42:0/2906766201 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f125003b070 con 0x557413d03390 2024-08-30T09:09:19.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:19.980+0000 7f1257fff640 1 -- 172.21.15.42:0/2906766201 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1250036d10 con 0x557413d03390 2024-08-30T09:09:19.988 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:19.983+0000 7f1263963740 1 -- 172.21.15.42:0/2906766201 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557413d03390 msgr2=0x5574140c9da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:09:19.988 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:19.983+0000 7f1263963740 1 --2- 172.21.15.42:0/2906766201 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557413d03390 0x5574140c9da0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f1250009fa0 tx=0x7f125000a890 comp rx=0 tx=0).stop 2024-08-30T09:09:19.988 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:19.985+0000 7f1263963740 1 -- 172.21.15.42:0/2906766201 shutdown_connections 2024-08-30T09:09:19.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:19.985+0000 7f1263963740 1 --2- 172.21.15.42:0/2906766201 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557413cfb1b0 0x5574140c9660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:19.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:19.985+0000 7f1263963740 1 --2- 172.21.15.42:0/2906766201 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557413d03390 0x5574140c9da0 secure :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f1250009fa0 tx=0x7f125000a890 comp rx=0 tx=0).stop 2024-08-30T09:09:19.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:19.985+0000 7f1263963740 1 --2- 172.21.15.42:0/2906766201 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557413d24070 0x5574140cc160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:19.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:19.985+0000 7f1263963740 1 -- 172.21.15.42:0/2906766201 >> 172.21.15.42:0/2906766201 conn(0x557413d18460 msgr2=0x5574140c7550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:09:19.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:19.991+0000 7f1263963740 1 -- 172.21.15.42:0/2906766201 shutdown_connections 2024-08-30T09:09:19.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:19.992+0000 7f1263963740 1 -- 172.21.15.42:0/2906766201 wait complete. 2024-08-30T09:09:19.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:19.993+0000 7f1263963740 1 -- start start 2024-08-30T09:09:19.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:19.993+0000 7f1263963740 1 Processor -- start 2024-08-30T09:09:19.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:19.994+0000 7f1263963740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557413cfb1b0 0x5574140d4e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:19.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:19.994+0000 7f1263963740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557413d24070 0x5574140d53d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:19.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:19.994+0000 7f1263963740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557413ce29f0 0x5574140e0f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:19.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:19.995+0000 7f12567fc640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557413d24070 0x5574140d53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:19.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:19.995+0000 7f12567fc640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557413d24070 0x5574140d53d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:55112/0 (socket says 172.21.15.42:55112) 2024-08-30T09:09:19.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:19.995+0000 7f12567fc640 1 -- 172.21.15.42:0/2783535013 learned_addr learned my addr 172.21.15.42:0/2783535013 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:09:19.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:19.995+0000 7f1256ffd640 1 --2- 172.21.15.42:0/2783535013 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557413cfb1b0 0x5574140d4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:19.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:19.995+0000 7f12577fe640 1 --2- 172.21.15.42:0/2783535013 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557413ce29f0 0x5574140e0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:19.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:19.995+0000 7f1263963740 1 -- 172.21.15.42:0/2783535013 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x557413e92700 con 0x557413ce29f0 2024-08-30T09:09:19.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:19.996+0000 7f1263963740 1 -- 172.21.15.42:0/2783535013 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x557413fb6700 con 0x557413cfb1b0 2024-08-30T09:09:19.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:19.996+0000 7f1263963740 1 -- 172.21.15.42:0/2783535013 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5574140d7c20 con 0x557413d24070 2024-08-30T09:09:19.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:19.996+0000 7f1263963740 10 client.0 _ll_register_callbacks cb 0x557413fe5dc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:09:20.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:19.996+0000 7f1256ffd640 1 -- 172.21.15.42:0/2783535013 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557413d24070 msgr2=0x5574140d53d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:09:20.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:19.996+0000 7f1256ffd640 1 --2- 172.21.15.42:0/2783535013 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557413d24070 0x5574140d53d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:20.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:19.996+0000 7f1256ffd640 1 -- 172.21.15.42:0/2783535013 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557413ce29f0 msgr2=0x5574140e0f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:09:20.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:19.996+0000 7f1256ffd640 1 --2- 172.21.15.42:0/2783535013 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557413ce29f0 0x5574140e0f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:20.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:19.996+0000 7f1256ffd640 1 -- 172.21.15.42:0/2783535013 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557413fef8b0 con 0x557413cfb1b0 2024-08-30T09:09:20.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:19.997+0000 7f123aff5640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-30T09:09:20.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:19.997+0000 7f1256ffd640 1 --2- 172.21.15.42:0/2783535013 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557413cfb1b0 0x5574140d4e50 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f124c00c6a0 tx=0x7f124c0aba00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dba7951abfc15c9b server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:09:20.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:19.997+0000 7f1263963740 1 -- 172.21.15.42:0/2783535013 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f125003b070 con 0x557413cfb1b0 2024-08-30T09:09:20.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:19.997+0000 7f123aff5640 1 -- 172.21.15.42:0/2783535013 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f124c0b0760 con 0x557413cfb1b0 2024-08-30T09:09:20.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:19.997+0000 7f123aff5640 1 -- 172.21.15.42:0/2783535013 <== mon.2 v2:172.21.15.167:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f124c0a84e0 con 0x557413cfb1b0 2024-08-30T09:09:20.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:19.998+0000 7f12577fe640 1 --2- 172.21.15.42:0/2783535013 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557413ce29f0 0x5574140e0f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-30T09:09:20.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:20.000+0000 7f123aff5640 1 -- 172.21.15.42:0/2783535013 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f124c0b0760 con 0x557413cfb1b0 2024-08-30T09:09:20.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:20.001+0000 7f1263963740 10 client.26687 fetch_fsmap learned FSMap version 37 2024-08-30T09:09:20.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:20.001+0000 7f1263963740 1 -- 172.21.15.42:0/2783535013 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({fsmap=37}) -- 0x5574140e39b0 con 0x557413cfb1b0 2024-08-30T09:09:20.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:20.004+0000 7f123aff5640 1 -- 172.21.15.42:0/2783535013 <== mon.2 v2:172.21.15.167:3300/0 4 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7f124c0b4d80 con 0x557413cfb1b0 2024-08-30T09:09:20.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:20.004+0000 7f1263963740 10 client.26687 fetch_fsmap finished waiting for FSMap version 37 2024-08-30T09:09:20.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:20.004+0000 7f1263963740 10 client.26687 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-30T09:09:20.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:20.004+0000 7f1263963740 20 client.26687 populate_metadata read hostname 'smithi042' 2024-08-30T09:09:20.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:20.004+0000 7f1263963740 1 --2- 172.21.15.42:0/2783535013 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x557413cb7da0 0x5574140e72c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:20.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:20.004+0000 7f1263963740 4 client.26687 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-30T09:09:20.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:20.004+0000 7f1263963740 1 -- 172.21.15.42:0/2783535013 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557414042580 con 0x557413cb7da0 2024-08-30T09:09:20.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:20.006+0000 7f12567fc640 1 --2- 172.21.15.42:0/2783535013 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x557413cb7da0 0x5574140e72c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:20.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:20.013+0000 7f12567fc640 1 --2- 172.21.15.42:0/2783535013 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x557413cb7da0 0x5574140e72c0 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b23d58a6ea36017 server_cookie=acc0cf0399ad5fe7 in_seq=0 out_seq=0 2024-08-30T09:09:20.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:20.013+0000 7f123aff5640 10 client.26687 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:09:20.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:20.015+0000 7f123aff5640 1 -- 172.21.15.42:0/2783535013 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x557414042580 con 0x557413cb7da0 2024-08-30T09:09:20.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:20.015+0000 7f123aff5640 10 client.26687 handle_command_reply: tid=0 2024-08-30T09:09:20.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:20.017+0000 7f1263963740 1 client.26687 shutdown 2024-08-30T09:09:20.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:20.019+0000 7f1263963740 1 -- 172.21.15.42:0/2783535013 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557413cfb1b0 msgr2=0x5574140d4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:09:20.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:20.019+0000 7f1263963740 1 --2- 172.21.15.42:0/2783535013 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557413cfb1b0 0x5574140d4e50 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f124c00c6a0 tx=0x7f124c0aba00 comp rx=0 tx=0).stop 2024-08-30T09:09:20.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:20.021+0000 7f1263963740 1 -- 172.21.15.42:0/2783535013 shutdown_connections 2024-08-30T09:09:20.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:20.021+0000 7f1263963740 1 --2- 172.21.15.42:0/2783535013 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x557413cb7da0 0x5574140e72c0 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:20.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:20.021+0000 7f123aff5640 0 client.26687 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:09:20.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:20.021+0000 7f1263963740 1 --2- 172.21.15.42:0/2783535013 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557413d24070 0x5574140d53d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:20.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:20.022+0000 7f1263963740 1 --2- 172.21.15.42:0/2783535013 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557413ce29f0 0x5574140e0f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:20.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:20.022+0000 7f1263963740 1 --2- 172.21.15.42:0/2783535013 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557413cfb1b0 0x5574140d4e50 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:20.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:20.022+0000 7f1263963740 1 -- 172.21.15.42:0/2783535013 >> 172.21.15.42:0/2783535013 conn(0x557413d18460 msgr2=0x5574140cb030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:09:20.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:20.022+0000 7f1263963740 1 -- 172.21.15.42:0/2783535013 shutdown_connections 2024-08-30T09:09:20.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:20.022+0000 7f1263963740 1 -- 172.21.15.42:0/2783535013 wait complete. 2024-08-30T09:09:20.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:20.022+0000 7f1263963740 20 client.26687 trim_cache size 0 max 16384 2024-08-30T09:09:20.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:20.060+0000 7f1263963740 1 Processor -- start 2024-08-30T09:09:20.064 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:09:19 smithi042 ceph-mon[18652]: scrub summary: idle+waiting paths [/] 2024-08-30T09:09:20.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:20.061+0000 7f1263963740 1 -- start start 2024-08-30T09:09:20.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:20.061+0000 7f1263963740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557413cb7da0 0x5574140d0890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:20.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:20.061+0000 7f1263963740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557413ce29f0 0x5574140c8c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:20.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:20.061+0000 7f1263963740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557413cfb1b0 0x5574140e0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:20.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:20.062+0000 7f1263963740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f12581b3710 con 0x557413cb7da0 2024-08-30T09:09:20.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:20.062+0000 7f1263963740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f12581b1020 con 0x557413cfb1b0 2024-08-30T09:09:20.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:20.062+0000 7f1263963740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f1258116740 con 0x557413ce29f0 2024-08-30T09:09:20.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:20.062+0000 7f12567fc640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557413ce29f0 0x5574140c8c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:20.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:20.062+0000 7f12567fc640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557413ce29f0 0x5574140c8c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:55126/0 (socket says 172.21.15.42:55126) 2024-08-30T09:09:20.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:20.062+0000 7f12567fc640 1 -- 172.21.15.42:0/58760446 learned_addr learned my addr 172.21.15.42:0/58760446 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:09:20.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:20.062+0000 7f12577fe640 1 --2- 172.21.15.42:0/58760446 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557413cfb1b0 0x5574140e0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:20.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:20.064+0000 7f1256ffd640 1 --2- 172.21.15.42:0/58760446 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557413cb7da0 0x5574140d0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:20.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:20.064+0000 7f1256ffd640 1 -- 172.21.15.42:0/58760446 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557413ce29f0 msgr2=0x5574140c8c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:09:20.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:20.064+0000 7f1256ffd640 1 --2- 172.21.15.42:0/58760446 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557413ce29f0 0x5574140c8c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:20.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:20.064+0000 7f1256ffd640 1 -- 172.21.15.42:0/58760446 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557413cfb1b0 msgr2=0x5574140e0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:09:20.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:20.064+0000 7f1256ffd640 1 --2- 172.21.15.42:0/58760446 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557413cfb1b0 0x5574140e0e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:20.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:20.064+0000 7f1256ffd640 1 -- 172.21.15.42:0/58760446 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5574140c6130 con 0x557413cb7da0 2024-08-30T09:09:20.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:20.067+0000 7f1256ffd640 1 --2- 172.21.15.42:0/58760446 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557413cb7da0 0x5574140d0890 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f124c0aa660 tx=0x7f124c0b0000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ec22ce7811d3baf server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:09:20.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:20.069+0000 7f123cff9640 1 -- 172.21.15.42:0/58760446 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f124c0ca020 con 0x557413cb7da0 2024-08-30T09:09:20.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:20.070+0000 7f123cff9640 1 -- 172.21.15.42:0/58760446 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f125003b070 con 0x557413cb7da0 2024-08-30T09:09:20.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:20.070+0000 7f123cff9640 1 -- 172.21.15.42:0/58760446 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f124c0b0760 con 0x557413cb7da0 2024-08-30T09:09:20.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:20.071+0000 7f1263963740 1 -- 172.21.15.42:0/58760446 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557413cb7da0 msgr2=0x5574140d0890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:09:20.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:20.071+0000 7f1263963740 1 --2- 172.21.15.42:0/58760446 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557413cb7da0 0x5574140d0890 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f124c0aa660 tx=0x7f124c0b0000 comp rx=0 tx=0).stop 2024-08-30T09:09:20.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:20.076+0000 7f1263963740 1 -- 172.21.15.42:0/58760446 shutdown_connections 2024-08-30T09:09:20.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:20.076+0000 7f1263963740 1 --2- 172.21.15.42:0/58760446 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557413ce29f0 0x5574140c8c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:20.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:20.076+0000 7f1263963740 1 --2- 172.21.15.42:0/58760446 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557413cfb1b0 0x5574140e0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:20.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:20.076+0000 7f1263963740 1 --2- 172.21.15.42:0/58760446 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557413cb7da0 0x5574140d0890 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:20.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:20.076+0000 7f1263963740 1 -- 172.21.15.42:0/58760446 >> 172.21.15.42:0/58760446 conn(0x557413d18460 msgr2=0x5574140d0180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:09:20.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:20.084+0000 7f1263963740 1 -- 172.21.15.42:0/58760446 shutdown_connections 2024-08-30T09:09:20.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:20.084+0000 7f1263963740 1 -- 172.21.15.42:0/58760446 wait complete. 2024-08-30T09:09:20.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:20.084+0000 7f1263963740 1 -- start start 2024-08-30T09:09:20.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:20.084+0000 7f1263963740 1 Processor -- start 2024-08-30T09:09:20.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:20.085+0000 7f1263963740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557413cb7da0 0x5574140d28e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:20.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:20.085+0000 7f1263963740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557413ce29f0 0x5574140dce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:20.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:20.085+0000 7f1263963740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557413cfb1b0 0x5574140dd3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:20.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:20.085+0000 7f1263963740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f12581af4e0 con 0x557413ce29f0 2024-08-30T09:09:20.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:20.085+0000 7f1263963740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f12581b0950 con 0x557413cfb1b0 2024-08-30T09:09:20.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:20.085+0000 7f1263963740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f12581afbb0 con 0x557413cb7da0 2024-08-30T09:09:20.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:20.085+0000 7f1263963740 10 client.0 _ll_register_callbacks cb 0x7f124c0046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:09:20.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:20.085+0000 7f12577fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557413cfb1b0 0x5574140dd3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:20.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:20.085+0000 7f12577fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557413cfb1b0 0x5574140dd3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:46114/0 (socket says 172.21.15.42:46114) 2024-08-30T09:09:20.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:20.086+0000 7f12577fe640 1 -- 172.21.15.42:0/2477507528 learned_addr learned my addr 172.21.15.42:0/2477507528 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:09:20.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:20.086+0000 7f1256ffd640 1 --2- 172.21.15.42:0/2477507528 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557413cb7da0 0x5574140d28e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:20.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:20.087+0000 7f1256ffd640 1 -- 172.21.15.42:0/2477507528 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557413cfb1b0 msgr2=0x5574140dd3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:09:20.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:20.087+0000 7f1256ffd640 1 --2- 172.21.15.42:0/2477507528 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557413cfb1b0 0x5574140dd3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:20.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:20.087+0000 7f1256ffd640 1 -- 172.21.15.42:0/2477507528 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557413ce29f0 msgr2=0x5574140dce70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-30T09:09:20.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:20.087+0000 7f1256ffd640 1 --2- 172.21.15.42:0/2477507528 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557413ce29f0 0x5574140dce70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:20.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:20.087+0000 7f1256ffd640 1 -- 172.21.15.42:0/2477507528 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5574140ddb20 con 0x557413cb7da0 2024-08-30T09:09:20.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:20.089+0000 7f1256ffd640 1 --2- 172.21.15.42:0/2477507528 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557413cb7da0 0x5574140d28e0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f124c00cfd0 tx=0x7f124c0a7db0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e32f9e5f09d21fa2 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:09:20.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:20.089+0000 7f123aff5640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-08-30T09:09:20.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:20.089+0000 7f123aff5640 1 -- 172.21.15.42:0/2477507528 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f124c0ca050 con 0x557413cb7da0 2024-08-30T09:09:20.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:20.089+0000 7f1263963740 1 -- 172.21.15.42:0/2477507528 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f125003b070 con 0x557413cb7da0 2024-08-30T09:09:20.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:20.090+0000 7f123aff5640 1 -- 172.21.15.42:0/2477507528 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f124c0cd360 con 0x557413cb7da0 2024-08-30T09:09:20.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:20.090+0000 7f123aff5640 1 -- 172.21.15.42:0/2477507528 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f124c0b5210 con 0x557413cb7da0 2024-08-30T09:09:20.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:20.090+0000 7f1263963740 10 client.26680 fetch_fsmap learned FSMap version 37 2024-08-30T09:09:20.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:20.090+0000 7f1263963740 1 -- 172.21.15.42:0/2477507528 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5574140ddb20 con 0x557413cb7da0 2024-08-30T09:09:20.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:20.098+0000 7f123aff5640 1 -- 172.21.15.42:0/2477507528 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f125003b070 con 0x557413cb7da0 2024-08-30T09:09:20.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:20.098+0000 7f123aff5640 1 -- 172.21.15.42:0/2477507528 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7f124c0b4d80 con 0x557413cb7da0 2024-08-30T09:09:20.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:20.098+0000 7f1263963740 10 client.26680 fetch_fsmap finished waiting for FSMap version 37 2024-08-30T09:09:20.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:20.098+0000 7f1263963740 10 client.26680 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-30T09:09:20.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:20.098+0000 7f1263963740 20 client.26680 populate_metadata read hostname 'smithi042' 2024-08-30T09:09:20.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:20.098+0000 7f1263963740 1 --2- 172.21.15.42:0/2477507528 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x557413d24070 0x5574141550b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:20.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:20.098+0000 7f1263963740 4 client.26680 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-30T09:09:20.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:20.098+0000 7f1263963740 1 -- 172.21.15.42:0/2477507528 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55741403e020 con 0x557413d24070 2024-08-30T09:09:20.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:20.098+0000 7f12567fc640 1 --2- 172.21.15.42:0/2477507528 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x557413d24070 0x5574141550b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:20.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:20.100+0000 7f12567fc640 1 --2- 172.21.15.42:0/2477507528 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x557413d24070 0x5574141550b0 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d9b1542cab58cbcc server_cookie=cb6d878b5a08f1fb in_seq=0 out_seq=0 2024-08-30T09:09:20.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:20.101+0000 7f123aff5640 10 client.26680 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:09:20.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:20.101+0000 7f123aff5640 1 -- 172.21.15.42:0/2477507528 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3410 (crc 0 0 0) 0x55741403e020 con 0x557413d24070 2024-08-30T09:09:20.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:20.101+0000 7f123aff5640 10 client.26680 handle_command_reply: tid=0 2024-08-30T09:09:20.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:20.101+0000 7f1263963740 1 client.26680 shutdown 2024-08-30T09:09:20.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:20.102+0000 7f1263963740 1 -- 172.21.15.42:0/2477507528 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557413cb7da0 msgr2=0x5574140d28e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:09:20.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:20.102+0000 7f1263963740 1 --2- 172.21.15.42:0/2477507528 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557413cb7da0 0x5574140d28e0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f124c00cfd0 tx=0x7f124c0a7db0 comp rx=0 tx=0).stop 2024-08-30T09:09:20.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:20.103+0000 7f1263963740 1 -- 172.21.15.42:0/2477507528 shutdown_connections 2024-08-30T09:09:20.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:20.103+0000 7f1263963740 1 --2- 172.21.15.42:0/2477507528 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x557413d24070 0x5574141550b0 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:20.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:20.103+0000 7f1263963740 1 --2- 172.21.15.42:0/2477507528 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557413cfb1b0 0x5574140dd3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:20.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:20.103+0000 7f1263963740 1 --2- 172.21.15.42:0/2477507528 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557413ce29f0 0x5574140dce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:20.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:20.103+0000 7f1263963740 1 --2- 172.21.15.42:0/2477507528 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557413cb7da0 0x5574140d28e0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:20.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:20.103+0000 7f1263963740 1 -- 172.21.15.42:0/2477507528 >> 172.21.15.42:0/2477507528 conn(0x557413d18460 msgr2=0x5574140c6e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:09:20.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:20.103+0000 7f123aff5640 0 client.26680 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:09:20.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:20.104+0000 7f1263963740 1 -- 172.21.15.42:0/2477507528 shutdown_connections 2024-08-30T09:09:20.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:20.104+0000 7f1263963740 1 -- 172.21.15.42:0/2477507528 wait complete. 2024-08-30T09:09:20.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:20.105+0000 7f1263963740 20 client.26680 trim_cache size 0 max 16384 2024-08-30T09:09:20.112 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-08-30T09:09:20.112 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "IDLE (waiting for more scrubs)", 2024-08-30T09:09:20.113 INFO:teuthology.orchestra.run.smithi042.stdout: "scrubs": { 2024-08-30T09:09:20.113 INFO:teuthology.orchestra.run.smithi042.stdout: "37f2f28f-496f-41a6-b4cf-8846fb3f43ab": { 2024-08-30T09:09:20.113 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:09:20.113 INFO:teuthology.orchestra.run.smithi042.stdout: "tag": "37f2f28f-496f-41a6-b4cf-8846fb3f43ab", 2024-08-30T09:09:20.113 INFO:teuthology.orchestra.run.smithi042.stdout: "options": "recursive,force" 2024-08-30T09:09:20.113 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:09:20.113 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:09:20.113 INFO:teuthology.orchestra.run.smithi042.stdout: "scrub_stats": { 2024-08-30T09:09:20.113 INFO:teuthology.orchestra.run.smithi042.stdout: "37f2f28f-496f-41a6-b4cf-8846fb3f43ab": { 2024-08-30T09:09:20.113 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:09:17.428286+0000", 2024-08-30T09:09:20.113 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:09:20.113 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:09:20.113 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:09:20.114 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:09:20.114 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:09:20.114 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:09:20.114 INFO:teuthology.orchestra.run.smithi042.stdout: "99a29ec1-9455-4ae4-854a-398b32896675": { 2024-08-30T09:09:20.114 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:08:24.923176+0000", 2024-08-30T09:09:20.114 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:09:20.114 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:09:20.114 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:09:20.114 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:09:20.114 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:09:20.114 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:09:20.114 INFO:teuthology.orchestra.run.smithi042.stdout: "598d8a75-f1cc-4388-80bc-a408bdb4eaac": { 2024-08-30T09:09:20.114 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:07:29.721122+0000", 2024-08-30T09:09:20.114 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:09:20.115 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:09:20.115 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:09:20.115 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:09:20.115 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:09:20.115 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:09:20.115 INFO:teuthology.orchestra.run.smithi042.stdout: "665ec740-7254-49cb-84ef-deff71c461b0": { 2024-08-30T09:09:20.115 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:06:37.144035+0000", 2024-08-30T09:09:20.115 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:09:20.115 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:09:20.115 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:09:20.115 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:09:20.115 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:09:20.115 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:09:20.115 INFO:teuthology.orchestra.run.smithi042.stdout: "1af4252a-4a99-47e9-bbf0-becf5e573dd5": { 2024-08-30T09:09:20.116 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:05:48.565881+0000", 2024-08-30T09:09:20.116 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:09:20.116 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:09:20.116 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:09:20.116 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:09:20.116 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:09:20.116 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:09:20.116 INFO:teuthology.orchestra.run.smithi042.stdout: "347d05ed-4bd3-4590-b099-314f719aab39": { 2024-08-30T09:09:20.116 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:05:00.439721+0000", 2024-08-30T09:09:20.116 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:09:20.116 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:09:20.116 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:09:20.116 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:09:20.117 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:09:20.117 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:09:20.117 INFO:teuthology.orchestra.run.smithi042.stdout: "88cd7478-2b20-47de-ba51-256479749232": { 2024-08-30T09:09:20.117 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:04:08.922625+0000", 2024-08-30T09:09:20.117 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:09:20.117 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:09:20.117 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:09:20.117 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:09:20.117 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:09:20.117 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:09:20.117 INFO:teuthology.orchestra.run.smithi042.stdout: "749f4867-0b70-49e5-af2d-8362a240ebc3": { 2024-08-30T09:09:20.117 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:03:16.386554+0000", 2024-08-30T09:09:20.117 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:09:20.118 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:09:20.118 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:09:20.118 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:09:20.118 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:09:20.118 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:09:20.118 INFO:teuthology.orchestra.run.smithi042.stdout: "85134e4b-ab4a-4885-b07e-a3ad01eaf13d": { 2024-08-30T09:09:20.118 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:01:27.738462+0000", 2024-08-30T09:09:20.118 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:09:20.118 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:09:20.118 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:09:20.118 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:09:20.118 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:09:20.118 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:09:20.119 INFO:teuthology.orchestra.run.smithi042.stdout: "c827ad7a-44ee-4e1d-9bb3-51fb7ab00c73": { 2024-08-30T09:09:20.119 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:02:21.686517+0000", 2024-08-30T09:09:20.119 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:09:20.119 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:09:20.119 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:09:20.119 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:09:20.119 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:09:20.119 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:09:20.119 INFO:teuthology.orchestra.run.smithi042.stdout: "5e2ed104-0625-478a-a78b-584e7f4c2817": { 2024-08-30T09:09:20.119 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:00:33.928400+0000", 2024-08-30T09:09:20.119 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:09:20.119 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:09:20.119 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:09:20.119 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:09:20.119 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:09:20.120 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:09:20.120 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:09:20.120 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-08-30T09:09:20.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:20.109+0000 7f123dffb640 1 -- 172.21.15.42:0/2966781866 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f12400b4d40 msgr2=0x7f12400b7250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:09:20.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:20.109+0000 7f123dffb640 1 --2- 172.21.15.42:0/2966781866 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f12400b4d40 0x7f12400b7250 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f1248031440 tx=0x7f1248009260 comp rx=0 tx=0).stop 2024-08-30T09:09:20.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:20.109+0000 7f123dffb640 1 -- 172.21.15.42:0/2966781866 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f125810d990 msgr2=0x7f125811c170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:09:20.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:20.109+0000 7f123dffb640 1 --2- 172.21.15.42:0/2966781866 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f125810d990 0x7f125811c170 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f124c00e9c0 tx=0x7f124c00ee80 comp rx=0 tx=0).stop 2024-08-30T09:09:20.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:20.110+0000 7f123dffb640 1 -- 172.21.15.42:0/2966781866 shutdown_connections 2024-08-30T09:09:20.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:20.110+0000 7f123dffb640 1 --2- 172.21.15.42:0/2966781866 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f12400b4d40 0x7f12400b7250 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:20.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:20.110+0000 7f123dffb640 1 --2- 172.21.15.42:0/2966781866 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f125811c940 0x7f1258115980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:20.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:20.110+0000 7f123dffb640 1 --2- 172.21.15.42:0/2966781866 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1258111a70 0x7f1258115440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:20.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:20.110+0000 7f123dffb640 1 --2- 172.21.15.42:0/2966781866 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f125810d990 0x7f125811c170 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:20.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:20.110+0000 7f123dffb640 1 -- 172.21.15.42:0/2966781866 >> 172.21.15.42:0/2966781866 conn(0x7f125806e790 msgr2=0x7f1258110720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:09:20.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:20.110+0000 7f123dffb640 1 -- 172.21.15.42:0/2966781866 shutdown_connections 2024-08-30T09:09:20.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:20.110+0000 7f123dffb640 1 -- 172.21.15.42:0/2966781866 wait complete. 2024-08-30T09:09:20.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:09:19 smithi190 ceph-mon[22001]: scrub summary: idle+waiting paths [/] 2024-08-30T09:09:20.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:09:19 smithi167 ceph-mon[22063]: scrub summary: idle+waiting paths [/] 2024-08-30T09:09:20.954 INFO:teuthology.orchestra.run.smithi042.stderr:progress: 515.0 s, 554.2 tps, lat 180.012 ms stddev 216.553 2024-08-30T09:09:21.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:09:20 smithi042 ceph-mon[18652]: pgmap v630: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 5.3 MiB/s wr, 618 op/s 2024-08-30T09:09:21.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:09:20 smithi190 ceph-mon[22001]: pgmap v630: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 5.3 MiB/s wr, 618 op/s 2024-08-30T09:09:21.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:09:20 smithi167 ceph-mon[22063]: pgmap v630: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 5.3 MiB/s wr, 618 op/s 2024-08-30T09:09:21.758 INFO:tasks.cephfs.filesystem:scrub status for tag:37f2f28f-496f-41a6-b4cf-8846fb3f43ab - {'path': '/', 'tag': '37f2f28f-496f-41a6-b4cf-8846fb3f43ab', 'options': 'recursive,force'} 2024-08-30T09:09:23.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:09:22 smithi042 ceph-mon[18652]: pgmap v631: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 5.0 MiB/s wr, 578 op/s 2024-08-30T09:09:23.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:09:22 smithi190 ceph-mon[22001]: pgmap v631: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 5.0 MiB/s wr, 578 op/s 2024-08-30T09:09:23.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:09:22 smithi167 ceph-mon[22063]: pgmap v631: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 5.0 MiB/s wr, 578 op/s 2024-08-30T09:09:25.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:09:24 smithi042 ceph-mon[18652]: pgmap v632: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 5.3 MiB/s wr, 660 op/s 2024-08-30T09:09:25.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:09:24 smithi190 ceph-mon[22001]: pgmap v632: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 5.3 MiB/s wr, 660 op/s 2024-08-30T09:09:25.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:09:24 smithi167 ceph-mon[22063]: pgmap v632: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 5.3 MiB/s wr, 660 op/s 2024-08-30T09:09:25.953 INFO:teuthology.orchestra.run.smithi042.stderr:progress: 520.0 s, 520.0 tps, lat 196.229 ms stddev 217.992 2024-08-30T09:09:27.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:09:26 smithi042 ceph-mon[18652]: pgmap v633: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 4.2 MiB/s wr, 567 op/s 2024-08-30T09:09:27.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:09:26 smithi190 ceph-mon[22001]: pgmap v633: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 4.2 MiB/s wr, 567 op/s 2024-08-30T09:09:27.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:09:26 smithi167 ceph-mon[22063]: pgmap v633: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 4.2 MiB/s wr, 567 op/s 2024-08-30T09:09:28.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:09:27 smithi190 ceph-mon[22001]: scrub summary: idle 2024-08-30T09:09:28.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:09:27 smithi190 ceph-mon[22001]: scrub summary: idle 2024-08-30T09:09:28.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:09:27 smithi167 ceph-mon[22063]: scrub summary: idle 2024-08-30T09:09:28.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:09:27 smithi167 ceph-mon[22063]: scrub summary: idle 2024-08-30T09:09:28.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:09:27 smithi042 ceph-mon[18652]: scrub summary: idle 2024-08-30T09:09:28.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:09:27 smithi042 ceph-mon[18652]: scrub summary: idle 2024-08-30T09:09:28.993 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:28.987+0000 7f1d2d169640 1 -- 172.21.15.42:0/39101943 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1d2806c2d0 msgr2=0x7f1d2810c690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:09:28.993 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:28.987+0000 7f1d2d169640 1 --2- 172.21.15.42:0/39101943 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1d2806c2d0 0x7f1d2810c690 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f1d1c009900 tx=0x7f1d1c02a440 comp rx=0 tx=0).stop 2024-08-30T09:09:28.993 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:28.989+0000 7f1d2d169640 1 -- 172.21.15.42:0/39101943 shutdown_connections 2024-08-30T09:09:28.993 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:28.989+0000 7f1d2d169640 1 --2- 172.21.15.42:0/39101943 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1d2810f790 0x7f1d2810fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:28.994 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:28.989+0000 7f1d2d169640 1 --2- 172.21.15.42:0/39101943 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1d2810cbd0 0x7f1d2810f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:28.994 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:28.989+0000 7f1d2d169640 1 --2- 172.21.15.42:0/39101943 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1d2806c2d0 0x7f1d2810c690 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:28.994 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:28.989+0000 7f1d2d169640 1 -- 172.21.15.42:0/39101943 >> 172.21.15.42:0/39101943 conn(0x7f1d280716c0 msgr2=0x7f1d28073b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:09:28.998 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:28.995+0000 7f1d2d169640 1 -- 172.21.15.42:0/39101943 shutdown_connections 2024-08-30T09:09:29.003 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:28.999+0000 7f1d2d169640 1 -- 172.21.15.42:0/39101943 wait complete. 2024-08-30T09:09:29.003 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.000+0000 7f1d2d169640 1 Processor -- start 2024-08-30T09:09:29.003 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.000+0000 7f1d2d169640 1 -- start start 2024-08-30T09:09:29.003 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.000+0000 7f1d2d169640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1d2806c2d0 0x7f1d2811a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:29.003 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.000+0000 7f1d2d169640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1d2810cbd0 0x7f1d28119260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:29.003 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.000+0000 7f1d2d169640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1d2810f790 0x7f1d281197a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:29.003 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.000+0000 7f1d2d169640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f1d28111560 con 0x7f1d2810cbd0 2024-08-30T09:09:29.004 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.000+0000 7f1d2d169640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f1d281113e0 con 0x7f1d2806c2d0 2024-08-30T09:09:29.004 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.000+0000 7f1d2d169640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f1d281116e0 con 0x7f1d2810f790 2024-08-30T09:09:29.006 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.002+0000 7f1d26d9d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1d2806c2d0 0x7f1d2811a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:29.006 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.002+0000 7f1d26d9d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1d2806c2d0 0x7f1d2811a9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:55676/0 (socket says 172.21.15.42:55676) 2024-08-30T09:09:29.006 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.002+0000 7f1d26d9d640 1 -- 172.21.15.42:0/2046454615 learned_addr learned my addr 172.21.15.42:0/2046454615 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:09:29.006 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.002+0000 7f1d2759e640 1 --2- 172.21.15.42:0/2046454615 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1d2810f790 0x7f1d281197a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:29.006 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.003+0000 7f1d26d9d640 1 -- 172.21.15.42:0/2046454615 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1d2810f790 msgr2=0x7f1d281197a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:09:29.006 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.003+0000 7f1d26d9d640 1 --2- 172.21.15.42:0/2046454615 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1d2810f790 0x7f1d281197a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:29.006 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.003+0000 7f1d26d9d640 1 -- 172.21.15.42:0/2046454615 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1d2810cbd0 msgr2=0x7f1d28119260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:09:29.006 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.003+0000 7f1d26d9d640 1 --2- 172.21.15.42:0/2046454615 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1d2810cbd0 0x7f1d28119260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:29.006 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.003+0000 7f1d26d9d640 1 -- 172.21.15.42:0/2046454615 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d28119d10 con 0x7f1d2806c2d0 2024-08-30T09:09:29.009 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.005+0000 7f1d26d9d640 1 --2- 172.21.15.42:0/2046454615 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1d2806c2d0 0x7f1d2811a9a0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f1d1c02aa30 tx=0x7f1d1c003b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:09:29.011 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.008+0000 7f1d0ffff640 1 -- 172.21.15.42:0/2046454615 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d1c03f070 con 0x7f1d2806c2d0 2024-08-30T09:09:29.011 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.009+0000 7f1d0ffff640 1 -- 172.21.15.42:0/2046454615 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d1c031bb0 con 0x7f1d2806c2d0 2024-08-30T09:09:29.011 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.009+0000 7f1d0ffff640 1 -- 172.21.15.42:0/2046454615 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d1c035030 con 0x7f1d2806c2d0 2024-08-30T09:09:29.011 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.009+0000 7f1d2d169640 1 -- 172.21.15.42:0/2046454615 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d28119fa0 con 0x7f1d2806c2d0 2024-08-30T09:09:29.012 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.009+0000 7f1d2d169640 1 -- 172.21.15.42:0/2046454615 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d2807b5a0 con 0x7f1d2806c2d0 2024-08-30T09:09:29.023 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.013+0000 7f1d340d6740 1 Processor -- start 2024-08-30T09:09:29.032 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.027+0000 7f1d340d6740 1 -- start start 2024-08-30T09:09:29.032 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.027+0000 7f1d340d6740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561ea1bba1b0 0x561ea1f5ff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:29.032 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.027+0000 7f1d340d6740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561ea1b912f0 0x561ea1f605d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:29.032 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.027+0000 7f1d340d6740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561ea1b994d0 0x561ea1f61120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:29.033 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.027+0000 7f1d340d6740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x561ea1ed3000 con 0x561ea1bba1b0 2024-08-30T09:09:29.033 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.027+0000 7f1d340d6740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x561ea1ec2590 con 0x561ea1b994d0 2024-08-30T09:09:29.033 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.027+0000 7f1d340d6740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x561ea1ec6540 con 0x561ea1b912f0 2024-08-30T09:09:29.033 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.028+0000 7f1d2759e640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561ea1bba1b0 0x561ea1f5ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:29.033 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.029+0000 7f1d2759e640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561ea1bba1b0 0x561ea1f5ff60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52240/0 (socket says 172.21.15.42:52240) 2024-08-30T09:09:29.033 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.029+0000 7f1d2759e640 1 -- 172.21.15.42:0/3085417227 learned_addr learned my addr 172.21.15.42:0/3085417227 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:09:29.033 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.029+0000 7f1d2659c640 1 --2- 172.21.15.42:0/3085417227 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561ea1b994d0 0x561ea1f61120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:29.033 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.029+0000 7f1d2659c640 1 -- 172.21.15.42:0/3085417227 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561ea1b912f0 msgr2=0x561ea1f605d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-30T09:09:29.033 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.029+0000 7f1d2659c640 1 --2- 172.21.15.42:0/3085417227 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561ea1b912f0 0x561ea1f605d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:29.033 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.029+0000 7f1d2659c640 1 -- 172.21.15.42:0/3085417227 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561ea1bba1b0 msgr2=0x561ea1f5ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:09:29.033 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.029+0000 7f1d2659c640 1 --2- 172.21.15.42:0/3085417227 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561ea1bba1b0 0x561ea1f5ff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:29.033 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.029+0000 7f1d2659c640 1 -- 172.21.15.42:0/3085417227 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561ea1c25790 con 0x561ea1b994d0 2024-08-30T09:09:29.033 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.030+0000 7f1d2659c640 1 --2- 172.21.15.42:0/3085417227 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561ea1b994d0 0x561ea1f61120 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f1d20009a60 tx=0x7f1d20009f20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=110562702cee177c server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:09:29.034 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.031+0000 7f1d0ffff640 1 -- 172.21.15.42:0/2046454615 <== mon.2 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7f1d1c042070 con 0x7f1d2806c2d0 2024-08-30T09:09:29.034 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.032+0000 7f1d0ffff640 1 --2- 172.21.15.42:0/2046454615 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f1d100b4d40 0x7f1d100b7250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:29.035 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.032+0000 7f1d0ffff640 1 -- 172.21.15.42:0/2046454615 <== mon.2 v2:172.21.15.167:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f1d1c105d30 con 0x7f1d2806c2d0 2024-08-30T09:09:29.035 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.032+0000 7f1d0d7fa640 1 -- 172.21.15.42:0/3085417227 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d2000ed70 con 0x561ea1b994d0 2024-08-30T09:09:29.036 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.032+0000 7f1d0d7fa640 1 -- 172.21.15.42:0/3085417227 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d200108e0 con 0x561ea1b994d0 2024-08-30T09:09:29.036 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.034+0000 7f1d340d6740 1 -- 172.21.15.42:0/3085417227 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561ea1b994d0 msgr2=0x561ea1f61120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:09:29.036 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.034+0000 7f1d340d6740 1 --2- 172.21.15.42:0/3085417227 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561ea1b994d0 0x561ea1f61120 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f1d20009a60 tx=0x7f1d20009f20 comp rx=0 tx=0).stop 2024-08-30T09:09:29.039 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.037+0000 7f1d2759e640 1 --2- 172.21.15.42:0/2046454615 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f1d100b4d40 0x7f1d100b7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:29.039 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.037+0000 7f1d340d6740 1 -- 172.21.15.42:0/3085417227 shutdown_connections 2024-08-30T09:09:29.039 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.037+0000 7f1d340d6740 1 --2- 172.21.15.42:0/3085417227 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561ea1b912f0 0x561ea1f605d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:29.039 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.037+0000 7f1d340d6740 1 --2- 172.21.15.42:0/3085417227 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561ea1b994d0 0x561ea1f61120 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:29.039 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.037+0000 7f1d340d6740 1 --2- 172.21.15.42:0/3085417227 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561ea1bba1b0 0x561ea1f5ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:29.039 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.037+0000 7f1d340d6740 1 -- 172.21.15.42:0/3085417227 >> 172.21.15.42:0/3085417227 conn(0x561ea1bae300 msgr2=0x561ea1f5b9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:09:29.045 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.041+0000 7f1d340d6740 1 -- 172.21.15.42:0/3085417227 shutdown_connections 2024-08-30T09:09:29.045 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.043+0000 7f1d2759e640 1 --2- 172.21.15.42:0/2046454615 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f1d100b4d40 0x7f1d100b7250 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f1d18005c40 tx=0x7f1d18009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:09:29.048 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.044+0000 7f1d340d6740 1 -- 172.21.15.42:0/3085417227 wait complete. 2024-08-30T09:09:29.048 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.045+0000 7f1d340d6740 1 -- start start 2024-08-30T09:09:29.048 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.045+0000 7f1d340d6740 1 Processor -- start 2024-08-30T09:09:29.048 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.045+0000 7f1d340d6740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561ea1b912f0 0x561ea1f69cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:29.048 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.045+0000 7f1d340d6740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561ea1b994d0 0x561ea1f6a380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:29.048 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.046+0000 7f1d340d6740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561ea1bba1b0 0x561ea1f6aef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:29.048 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.046+0000 7f1d340d6740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x561ea1dfbab0 con 0x561ea1b994d0 2024-08-30T09:09:29.049 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.046+0000 7f1d340d6740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x561ea1e49680 con 0x561ea1b912f0 2024-08-30T09:09:29.049 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.046+0000 7f1d340d6740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x561ea1f61c90 con 0x561ea1bba1b0 2024-08-30T09:09:29.049 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.046+0000 7f1d340d6740 10 client.0 _ll_register_callbacks cb 0x561ea1bd1a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:09:29.049 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.046+0000 7f1d2759e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561ea1bba1b0 0x561ea1f6aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:29.050 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.046+0000 7f1d26d9d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561ea1b912f0 0x561ea1f69cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:29.050 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.046+0000 7f1d2759e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561ea1bba1b0 0x561ea1f6aef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:42392/0 (socket says 172.21.15.42:42392) 2024-08-30T09:09:29.050 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.046+0000 7f1d2759e640 1 -- 172.21.15.42:0/767484365 learned_addr learned my addr 172.21.15.42:0/767484365 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:09:29.050 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.046+0000 7f1d26d9d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561ea1b912f0 0x561ea1f69cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:55704/0 (socket says 172.21.15.42:55704) 2024-08-30T09:09:29.053 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.051+0000 7f1d2659c640 1 --2- 172.21.15.42:0/767484365 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561ea1b994d0 0x561ea1f6a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:29.056 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.053+0000 7f1d26d9d640 1 -- 172.21.15.42:0/767484365 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561ea1bba1b0 msgr2=0x561ea1f6aef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:09:29.056 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.053+0000 7f1d26d9d640 1 --2- 172.21.15.42:0/767484365 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561ea1bba1b0 0x561ea1f6aef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:29.056 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.053+0000 7f1d26d9d640 1 -- 172.21.15.42:0/767484365 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561ea1b994d0 msgr2=0x561ea1f6a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:09:29.056 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.053+0000 7f1d26d9d640 1 --2- 172.21.15.42:0/767484365 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561ea1b994d0 0x561ea1f6a380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:29.056 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.053+0000 7f1d26d9d640 1 -- 172.21.15.42:0/767484365 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561ea1c9e620 con 0x561ea1b912f0 2024-08-30T09:09:29.060 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.055+0000 7f1d2759e640 1 --2- 172.21.15.42:0/767484365 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561ea1bba1b0 0x561ea1f6aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-30T09:09:29.060 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.055+0000 7f1d26d9d640 1 --2- 172.21.15.42:0/767484365 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561ea1b912f0 0x561ea1f69cc0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f1d1c0c83f0 tx=0x7f1d1c03b6b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=96f118cd8164ba46 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:09:29.060 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.055+0000 7f1d0aff5640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-30T09:09:29.060 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.057+0000 7f1d0aff5640 1 -- 172.21.15.42:0/767484365 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d1c049070 con 0x561ea1b912f0 2024-08-30T09:09:29.060 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.057+0000 7f1d340d6740 1 -- 172.21.15.42:0/767484365 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x561ea1ed4f50 con 0x561ea1b912f0 2024-08-30T09:09:29.060 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.057+0000 7f1d0aff5640 1 -- 172.21.15.42:0/767484365 <== mon.2 v2:172.21.15.167:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f1d1c0ef0b0 con 0x561ea1b912f0 2024-08-30T09:09:29.066 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.062+0000 7f1d0aff5640 1 -- 172.21.15.42:0/767484365 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f1d1c0cc370 con 0x561ea1b912f0 2024-08-30T09:09:29.066 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.064+0000 7f1d340d6740 10 client.26705 fetch_fsmap learned FSMap version 37 2024-08-30T09:09:29.066 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.064+0000 7f1d340d6740 1 -- 172.21.15.42:0/767484365 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x561ea1f73ed0 con 0x561ea1b912f0 2024-08-30T09:09:29.069 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.065+0000 7f1d0aff5640 1 -- 172.21.15.42:0/767484365 <== mon.2 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x561ea1ed4f50 con 0x561ea1b912f0 2024-08-30T09:09:29.069 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.065+0000 7f1d0aff5640 1 -- 172.21.15.42:0/767484365 <== mon.2 v2:172.21.15.167:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7f1d1c0c8600 con 0x561ea1b912f0 2024-08-30T09:09:29.069 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.065+0000 7f1d340d6740 10 client.26705 fetch_fsmap finished waiting for FSMap version 37 2024-08-30T09:09:29.070 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.065+0000 7f1d340d6740 10 client.26705 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-30T09:09:29.070 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.065+0000 7f1d340d6740 20 client.26705 populate_metadata read hostname 'smithi042' 2024-08-30T09:09:29.070 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.065+0000 7f1d340d6740 1 --2- 172.21.15.42:0/767484365 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x561ea1b78d80 0x561ea1f77840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:29.070 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.065+0000 7f1d340d6740 4 client.26705 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-30T09:09:29.070 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.065+0000 7f1d340d6740 1 -- 172.21.15.42:0/767484365 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561ea1ed09f0 con 0x561ea1b78d80 2024-08-30T09:09:29.070 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.067+0000 7f1d2659c640 1 --2- 172.21.15.42:0/767484365 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x561ea1b78d80 0x561ea1f77840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:29.072 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.068+0000 7f1d2659c640 1 --2- 172.21.15.42:0/767484365 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x561ea1b78d80 0x561ea1f77840 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=66fbfddcb7c0e91e server_cookie=38ee8e7487db88df in_seq=0 out_seq=0 2024-08-30T09:09:29.072 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.068+0000 7f1d0aff5640 10 client.26705 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:09:29.072 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.069+0000 7f1d0aff5640 1 -- 172.21.15.42:0/767484365 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x561ea1ed09f0 con 0x561ea1b78d80 2024-08-30T09:09:29.072 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.069+0000 7f1d0aff5640 10 client.26705 handle_command_reply: tid=0 2024-08-30T09:09:29.072 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.069+0000 7f1d340d6740 1 client.26705 shutdown 2024-08-30T09:09:29.073 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.069+0000 7f1d340d6740 1 -- 172.21.15.42:0/767484365 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561ea1b912f0 msgr2=0x561ea1f69cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:09:29.073 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.069+0000 7f1d340d6740 1 --2- 172.21.15.42:0/767484365 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561ea1b912f0 0x561ea1f69cc0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f1d1c0c83f0 tx=0x7f1d1c03b6b0 comp rx=0 tx=0).stop 2024-08-30T09:09:29.074 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.071+0000 7f1d340d6740 1 -- 172.21.15.42:0/767484365 shutdown_connections 2024-08-30T09:09:29.074 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.071+0000 7f1d340d6740 1 --2- 172.21.15.42:0/767484365 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x561ea1b78d80 0x561ea1f77840 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:29.074 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.071+0000 7f1d340d6740 1 --2- 172.21.15.42:0/767484365 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561ea1bba1b0 0x561ea1f6aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:29.074 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.071+0000 7f1d340d6740 1 --2- 172.21.15.42:0/767484365 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561ea1b994d0 0x561ea1f6a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:29.074 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.071+0000 7f1d340d6740 1 --2- 172.21.15.42:0/767484365 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561ea1b912f0 0x561ea1f69cc0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:29.074 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.071+0000 7f1d340d6740 1 -- 172.21.15.42:0/767484365 >> 172.21.15.42:0/767484365 conn(0x561ea1bae300 msgr2=0x561ea1f59f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:09:29.074 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.071+0000 7f1d0aff5640 0 client.26705 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:09:29.074 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.072+0000 7f1d340d6740 1 -- 172.21.15.42:0/767484365 shutdown_connections 2024-08-30T09:09:29.074 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.072+0000 7f1d340d6740 1 -- 172.21.15.42:0/767484365 wait complete. 2024-08-30T09:09:29.075 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.073+0000 7f1d340d6740 20 client.26705 trim_cache size 0 max 16384 2024-08-30T09:09:29.103 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.102+0000 7f1d340d6740 1 Processor -- start 2024-08-30T09:09:29.105 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.102+0000 7f1d340d6740 1 -- start start 2024-08-30T09:09:29.105 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.103+0000 7f1d340d6740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561ea1b912f0 0x561ea1f60df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:29.105 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.103+0000 7f1d340d6740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561ea1b994d0 0x561ea1f59eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:29.105 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.103+0000 7f1d340d6740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561ea1bba1b0 0x561ea1f5b330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:29.105 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.103+0000 7f1d340d6740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f1d281d73f0 con 0x561ea1b912f0 2024-08-30T09:09:29.105 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.103+0000 7f1d340d6740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f1d28085d50 con 0x561ea1bba1b0 2024-08-30T09:09:29.106 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.103+0000 7f1d340d6740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f1d28083c00 con 0x561ea1b994d0 2024-08-30T09:09:29.106 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.103+0000 7f1d2759e640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561ea1bba1b0 0x561ea1f5b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:29.107 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.103+0000 7f1d2759e640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561ea1bba1b0 0x561ea1f5b330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:55710/0 (socket says 172.21.15.42:55710) 2024-08-30T09:09:29.107 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.103+0000 7f1d2759e640 1 -- 172.21.15.42:0/2828238060 learned_addr learned my addr 172.21.15.42:0/2828238060 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:09:29.107 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.103+0000 7f1d2659c640 1 --2- 172.21.15.42:0/2828238060 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561ea1b994d0 0x561ea1f59eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:29.107 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.104+0000 7f1d26d9d640 1 --2- 172.21.15.42:0/2828238060 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561ea1b912f0 0x561ea1f60df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:29.107 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.104+0000 7f1d2759e640 1 -- 172.21.15.42:0/2828238060 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561ea1b994d0 msgr2=0x561ea1f59eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:09:29.107 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.104+0000 7f1d2759e640 1 --2- 172.21.15.42:0/2828238060 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561ea1b994d0 0x561ea1f59eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:29.107 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.104+0000 7f1d2759e640 1 -- 172.21.15.42:0/2828238060 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561ea1b912f0 msgr2=0x561ea1f60df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:09:29.107 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.104+0000 7f1d2759e640 1 --2- 172.21.15.42:0/2828238060 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561ea1b912f0 0x561ea1f60df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:29.107 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.104+0000 7f1d2759e640 1 -- 172.21.15.42:0/2828238060 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561ea1f6b450 con 0x561ea1bba1b0 2024-08-30T09:09:29.107 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.104+0000 7f1d2759e640 1 --2- 172.21.15.42:0/2828238060 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561ea1bba1b0 0x561ea1f5b330 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f1d180128f0 tx=0x7f1d18014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=265459393e0aa017 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:09:29.108 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.104+0000 7f1d0cff9640 1 -- 172.21.15.42:0/2828238060 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d18020bb0 con 0x561ea1bba1b0 2024-08-30T09:09:29.108 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.104+0000 7f1d0cff9640 1 -- 172.21.15.42:0/2828238060 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d18020d50 con 0x561ea1bba1b0 2024-08-30T09:09:29.108 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.104+0000 7f1d0cff9640 1 -- 172.21.15.42:0/2828238060 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d1801b600 con 0x561ea1bba1b0 2024-08-30T09:09:29.111 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.105+0000 7f1d340d6740 1 -- 172.21.15.42:0/2828238060 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561ea1bba1b0 msgr2=0x561ea1f5b330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:09:29.112 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.105+0000 7f1d340d6740 1 --2- 172.21.15.42:0/2828238060 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561ea1bba1b0 0x561ea1f5b330 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f1d180128f0 tx=0x7f1d18014000 comp rx=0 tx=0).stop 2024-08-30T09:09:29.112 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.105+0000 7f1d340d6740 1 -- 172.21.15.42:0/2828238060 shutdown_connections 2024-08-30T09:09:29.112 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.105+0000 7f1d340d6740 1 --2- 172.21.15.42:0/2828238060 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561ea1b994d0 0x561ea1f59eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:29.112 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.105+0000 7f1d340d6740 1 --2- 172.21.15.42:0/2828238060 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561ea1bba1b0 0x561ea1f5b330 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:29.112 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.105+0000 7f1d340d6740 1 --2- 172.21.15.42:0/2828238060 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561ea1b912f0 0x561ea1f60df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:29.112 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.105+0000 7f1d340d6740 1 -- 172.21.15.42:0/2828238060 >> 172.21.15.42:0/2828238060 conn(0x561ea1bae300 msgr2=0x561ea1f777b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:09:29.112 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.105+0000 7f1d340d6740 1 -- 172.21.15.42:0/2828238060 shutdown_connections 2024-08-30T09:09:29.112 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.105+0000 7f1d340d6740 1 -- 172.21.15.42:0/2828238060 wait complete. 2024-08-30T09:09:29.112 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.106+0000 7f1d340d6740 1 -- start start 2024-08-30T09:09:29.112 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.106+0000 7f1d340d6740 1 Processor -- start 2024-08-30T09:09:29.112 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.106+0000 7f1d340d6740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561ea1b912f0 0x561ea1f6bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:29.112 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.106+0000 7f1d340d6740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561ea1b994d0 0x561ea1f6c690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:29.113 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.106+0000 7f1d340d6740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561ea1bba1b0 0x561ea1f6d140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:29.113 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.106+0000 7f1d340d6740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f1d28084250 con 0x561ea1b994d0 2024-08-30T09:09:29.113 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.106+0000 7f1d340d6740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f1d28085680 con 0x561ea1bba1b0 2024-08-30T09:09:29.113 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.106+0000 7f1d340d6740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f1d280848e0 con 0x561ea1b912f0 2024-08-30T09:09:29.113 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.106+0000 7f1d340d6740 10 client.0 _ll_register_callbacks cb 0x7f1d1c0cc210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:09:29.113 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.106+0000 7f1d26d9d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561ea1b912f0 0x561ea1f6bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:29.113 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.106+0000 7f1d26d9d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561ea1b912f0 0x561ea1f6bfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:42412/0 (socket says 172.21.15.42:42412) 2024-08-30T09:09:29.113 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.106+0000 7f1d26d9d640 1 -- 172.21.15.42:0/1379716427 learned_addr learned my addr 172.21.15.42:0/1379716427 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:09:29.113 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.107+0000 7f1d2659c640 1 --2- 172.21.15.42:0/1379716427 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561ea1b994d0 0x561ea1f6c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:29.113 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.107+0000 7f1d2759e640 1 --2- 172.21.15.42:0/1379716427 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561ea1bba1b0 0x561ea1f6d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:29.113 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.107+0000 7f1d26d9d640 1 -- 172.21.15.42:0/1379716427 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561ea1bba1b0 msgr2=0x561ea1f6d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:09:29.113 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.107+0000 7f1d26d9d640 1 --2- 172.21.15.42:0/1379716427 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561ea1bba1b0 0x561ea1f6d140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:29.114 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.107+0000 7f1d26d9d640 1 -- 172.21.15.42:0/1379716427 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561ea1b994d0 msgr2=0x561ea1f6c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:09:29.114 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.107+0000 7f1d26d9d640 1 --2- 172.21.15.42:0/1379716427 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561ea1b994d0 0x561ea1f6c690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:29.114 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.107+0000 7f1d26d9d640 1 -- 172.21.15.42:0/1379716427 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561ea1f6d8b0 con 0x561ea1b912f0 2024-08-30T09:09:29.114 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.107+0000 7f1d26d9d640 1 --2- 172.21.15.42:0/1379716427 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561ea1b912f0 0x561ea1f6bfd0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f1d1c005ab0 tx=0x7f1d1c0cb620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2f939fcf4b741495 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:09:29.114 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.107+0000 7f1d0aff5640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-08-30T09:09:29.114 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.107+0000 7f1d0aff5640 1 -- 172.21.15.42:0/1379716427 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d1c049070 con 0x561ea1b912f0 2024-08-30T09:09:29.114 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.107+0000 7f1d340d6740 1 -- 172.21.15.42:0/1379716427 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1d18020d50 con 0x561ea1b912f0 2024-08-30T09:09:29.114 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.107+0000 7f1d0aff5640 1 -- 172.21.15.42:0/1379716427 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f1d1c0cb710 con 0x561ea1b912f0 2024-08-30T09:09:29.114 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.107+0000 7f1d0aff5640 1 -- 172.21.15.42:0/1379716427 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f1d1c0e9050 con 0x561ea1b912f0 2024-08-30T09:09:29.114 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.107+0000 7f1d340d6740 10 client.26692 fetch_fsmap learned FSMap version 37 2024-08-30T09:09:29.114 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.107+0000 7f1d340d6740 1 -- 172.21.15.42:0/1379716427 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x561ea1fe0cc0 con 0x561ea1b912f0 2024-08-30T09:09:29.114 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.108+0000 7f1d0aff5640 1 -- 172.21.15.42:0/1379716427 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d18020d50 con 0x561ea1b912f0 2024-08-30T09:09:29.114 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.108+0000 7f1d0aff5640 1 -- 172.21.15.42:0/1379716427 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7f1d1c02ac40 con 0x561ea1b912f0 2024-08-30T09:09:29.114 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.108+0000 7f1d340d6740 10 client.26692 fetch_fsmap finished waiting for FSMap version 37 2024-08-30T09:09:29.115 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.108+0000 7f1d340d6740 10 client.26692 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-30T09:09:29.115 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.108+0000 7f1d340d6740 20 client.26692 populate_metadata read hostname 'smithi042' 2024-08-30T09:09:29.115 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.108+0000 7f1d340d6740 1 --2- 172.21.15.42:0/1379716427 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x561ea1b4e1c0 0x561ea1fe3520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:29.115 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.108+0000 7f1d340d6740 4 client.26692 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-30T09:09:29.115 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.108+0000 7f1d340d6740 1 -- 172.21.15.42:0/1379716427 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561ea1ece130 con 0x561ea1b4e1c0 2024-08-30T09:09:29.115 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.108+0000 7f1d2659c640 1 --2- 172.21.15.42:0/1379716427 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x561ea1b4e1c0 0x561ea1fe3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:29.115 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.108+0000 7f1d0aff5640 10 client.26692 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:09:29.115 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.108+0000 7f1d2659c640 1 --2- 172.21.15.42:0/1379716427 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x561ea1b4e1c0 0x561ea1fe3520 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6c03c9a9a94bbe89 server_cookie=9af0d05e043ebfa5 in_seq=0 out_seq=0 2024-08-30T09:09:29.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:09:28 smithi190 ceph-mon[22001]: pgmap v634: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 4.4 MiB/s wr, 587 op/s 2024-08-30T09:09:29.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:09:28 smithi190 ceph-mon[22001]: scrub summary: idle 2024-08-30T09:09:29.267 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.264+0000 7f1d0aff5640 1 -- 172.21.15.42:0/1379716427 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561ea1ece130 con 0x561ea1b4e1c0 2024-08-30T09:09:29.267 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.264+0000 7f1d0aff5640 10 client.26692 handle_command_reply: tid=0 2024-08-30T09:09:29.268 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.264+0000 7f1d340d6740 1 client.26692 shutdown 2024-08-30T09:09:29.268 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.264+0000 7f1d340d6740 1 -- 172.21.15.42:0/1379716427 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561ea1b912f0 msgr2=0x561ea1f6bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:09:29.268 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.264+0000 7f1d340d6740 1 --2- 172.21.15.42:0/1379716427 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561ea1b912f0 0x561ea1f6bfd0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f1d1c005ab0 tx=0x7f1d1c0cb620 comp rx=0 tx=0).stop 2024-08-30T09:09:29.268 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.265+0000 7f1d340d6740 1 -- 172.21.15.42:0/1379716427 shutdown_connections 2024-08-30T09:09:29.268 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.265+0000 7f1d340d6740 1 --2- 172.21.15.42:0/1379716427 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x561ea1b4e1c0 0x561ea1fe3520 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:29.268 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.265+0000 7f1d340d6740 1 --2- 172.21.15.42:0/1379716427 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561ea1bba1b0 0x561ea1f6d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:29.268 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.265+0000 7f1d0aff5640 0 client.26692 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:09:29.268 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.265+0000 7f1d340d6740 1 --2- 172.21.15.42:0/1379716427 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561ea1b994d0 0x561ea1f6c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:29.268 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.265+0000 7f1d340d6740 1 --2- 172.21.15.42:0/1379716427 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561ea1b912f0 0x561ea1f6bfd0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:29.268 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.265+0000 7f1d340d6740 1 -- 172.21.15.42:0/1379716427 >> 172.21.15.42:0/1379716427 conn(0x561ea1bae300 msgr2=0x561ea1f60df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:09:29.268 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.265+0000 7f1d340d6740 1 -- 172.21.15.42:0/1379716427 shutdown_connections 2024-08-30T09:09:29.269 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.265+0000 7f1d340d6740 1 -- 172.21.15.42:0/1379716427 wait complete. 2024-08-30T09:09:29.269 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.265+0000 7f1d340d6740 20 client.26692 trim_cache size 0 max 16384 2024-08-30T09:09:29.269 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-08-30T09:09:29.269 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2024-08-30T09:09:29.269 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-08-30T09:09:29.269 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-08-30T09:09:29.270 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.266+0000 7f1d2d169640 1 -- 172.21.15.42:0/2046454615 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f1d100b4d40 msgr2=0x7f1d100b7250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:09:29.270 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.266+0000 7f1d2d169640 1 --2- 172.21.15.42:0/2046454615 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f1d100b4d40 0x7f1d100b7250 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f1d18005c40 tx=0x7f1d18009500 comp rx=0 tx=0).stop 2024-08-30T09:09:29.270 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.266+0000 7f1d2d169640 1 -- 172.21.15.42:0/2046454615 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1d2806c2d0 msgr2=0x7f1d2811a9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:09:29.270 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.266+0000 7f1d2d169640 1 --2- 172.21.15.42:0/2046454615 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1d2806c2d0 0x7f1d2811a9a0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f1d1c02aa30 tx=0x7f1d1c003b90 comp rx=0 tx=0).stop 2024-08-30T09:09:29.270 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.266+0000 7f1d2d169640 1 -- 172.21.15.42:0/2046454615 shutdown_connections 2024-08-30T09:09:29.270 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.266+0000 7f1d2d169640 1 --2- 172.21.15.42:0/2046454615 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f1d100b4d40 0x7f1d100b7250 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:29.270 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.266+0000 7f1d2d169640 1 --2- 172.21.15.42:0/2046454615 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1d2810f790 0x7f1d281197a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:29.270 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.266+0000 7f1d2d169640 1 --2- 172.21.15.42:0/2046454615 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1d2810cbd0 0x7f1d28119260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:29.270 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.266+0000 7f1d2d169640 1 --2- 172.21.15.42:0/2046454615 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1d2806c2d0 0x7f1d2811a9a0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:29.270 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.266+0000 7f1d2d169640 1 -- 172.21.15.42:0/2046454615 >> 172.21.15.42:0/2046454615 conn(0x7f1d280716c0 msgr2=0x7f1d28073ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:09:29.270 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.267+0000 7f1d2d169640 1 -- 172.21.15.42:0/2046454615 shutdown_connections 2024-08-30T09:09:29.271 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:29.267+0000 7f1d2d169640 1 -- 172.21.15.42:0/2046454615 wait complete. 2024-08-30T09:09:29.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:09:28 smithi167 ceph-mon[22063]: pgmap v634: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 4.4 MiB/s wr, 587 op/s 2024-08-30T09:09:29.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:09:28 smithi167 ceph-mon[22063]: scrub summary: idle 2024-08-30T09:09:29.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:09:28 smithi042 ceph-mon[18652]: pgmap v634: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 4.4 MiB/s wr, 587 op/s 2024-08-30T09:09:29.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:09:28 smithi042 ceph-mon[18652]: scrub summary: idle 2024-08-30T09:09:30.953 INFO:teuthology.orchestra.run.smithi042.stderr:progress: 525.0 s, 548.8 tps, lat 183.632 ms stddev 206.498 2024-08-30T09:09:31.185 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:09:30 smithi042 ceph-mon[18652]: pgmap v635: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 4.7 MiB/s wr, 620 op/s 2024-08-30T09:09:31.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:09:30 smithi190 ceph-mon[22001]: pgmap v635: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 4.7 MiB/s wr, 620 op/s 2024-08-30T09:09:31.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:09:30 smithi167 ceph-mon[22063]: pgmap v635: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 4.7 MiB/s wr, 620 op/s 2024-08-30T09:09:33.052 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:09:32 smithi042 ceph-mon[18652]: pgmap v636: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 4.4 MiB/s wr, 545 op/s 2024-08-30T09:09:33.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:09:32 smithi190 ceph-mon[22001]: pgmap v636: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 4.4 MiB/s wr, 545 op/s 2024-08-30T09:09:33.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:09:32 smithi167 ceph-mon[22063]: pgmap v636: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 4.4 MiB/s wr, 545 op/s 2024-08-30T09:09:35.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:09:34 smithi190 ceph-mon[22001]: pgmap v637: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 4.5 MiB/s wr, 572 op/s 2024-08-30T09:09:35.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:09:34 smithi167 ceph-mon[22063]: pgmap v637: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 4.5 MiB/s wr, 572 op/s 2024-08-30T09:09:35.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:09:34 smithi042 ceph-mon[18652]: pgmap v637: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 4.5 MiB/s wr, 572 op/s 2024-08-30T09:09:35.953 INFO:teuthology.orchestra.run.smithi042.stderr:progress: 530.0 s, 559.4 tps, lat 177.625 ms stddev 211.389 2024-08-30T09:09:37.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:09:36 smithi190 ceph-mon[22001]: pgmap v638: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 4.5 MiB/s wr, 514 op/s 2024-08-30T09:09:37.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:09:36 smithi167 ceph-mon[22063]: pgmap v638: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 4.5 MiB/s wr, 514 op/s 2024-08-30T09:09:37.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:09:36 smithi042 ceph-mon[18652]: pgmap v638: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 4.5 MiB/s wr, 514 op/s 2024-08-30T09:09:38.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:09:37 smithi167 ceph-mon[22063]: pgmap v639: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 4.7 MiB/s wr, 513 op/s 2024-08-30T09:09:38.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:09:37 smithi042 ceph-mon[18652]: pgmap v639: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 4.7 MiB/s wr, 513 op/s 2024-08-30T09:09:38.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:09:37 smithi190 ceph-mon[22001]: pgmap v639: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 4.7 MiB/s wr, 513 op/s 2024-08-30T09:09:40.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:09:40 smithi167 ceph-mon[22063]: pgmap v640: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 5.0 MiB/s wr, 559 op/s 2024-08-30T09:09:40.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:09:40 smithi042 ceph-mon[18652]: pgmap v640: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 5.0 MiB/s wr, 559 op/s 2024-08-30T09:09:40.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:09:40 smithi190 ceph-mon[22001]: pgmap v640: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 5.0 MiB/s wr, 559 op/s 2024-08-30T09:09:40.953 INFO:teuthology.orchestra.run.smithi042.stderr:progress: 535.0 s, 557.8 tps, lat 179.116 ms stddev 216.032 2024-08-30T09:09:42.390 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.386+0000 7ff1a15a1640 1 -- 172.21.15.42:0/2137673105 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff19c1068c0 msgr2=0x7ff19c108ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:09:42.391 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.386+0000 7ff1a15a1640 1 --2- 172.21.15.42:0/2137673105 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff19c1068c0 0x7ff19c108ce0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7ff190009900 tx=0x7ff19002a410 comp rx=0 tx=0).stop 2024-08-30T09:09:42.391 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.386+0000 7ff1a15a1640 1 -- 172.21.15.42:0/2137673105 shutdown_connections 2024-08-30T09:09:42.391 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.386+0000 7ff1a15a1640 1 --2- 172.21.15.42:0/2137673105 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff19c109220 0x7ff19c10b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:42.391 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.386+0000 7ff1a15a1640 1 --2- 172.21.15.42:0/2137673105 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff19c1068c0 0x7ff19c108ce0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:42.391 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.386+0000 7ff1a15a1640 1 --2- 172.21.15.42:0/2137673105 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff19c101830 0x7ff19c101c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:42.391 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.386+0000 7ff1a15a1640 1 -- 172.21.15.42:0/2137673105 >> 172.21.15.42:0/2137673105 conn(0x7ff19c0fd430 msgr2=0x7ff19c0ff8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:09:42.391 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.387+0000 7ff1a15a1640 1 -- 172.21.15.42:0/2137673105 shutdown_connections 2024-08-30T09:09:42.391 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.387+0000 7ff1a15a1640 1 -- 172.21.15.42:0/2137673105 wait complete. 2024-08-30T09:09:42.391 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.387+0000 7ff1a15a1640 1 Processor -- start 2024-08-30T09:09:42.391 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.387+0000 7ff1a15a1640 1 -- start start 2024-08-30T09:09:42.391 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.388+0000 7ff1a15a1640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff19c101830 0x7ff19c103f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:42.391 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.388+0000 7ff1a15a1640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff19c1068c0 0x7ff19c1027c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:42.392 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.388+0000 7ff1a15a1640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff19c109220 0x7ff19c102d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:42.392 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.388+0000 7ff1a15a1640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7ff19c10f0d0 con 0x7ff19c1068c0 2024-08-30T09:09:42.392 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.388+0000 7ff1a15a1640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff19c10ef50 con 0x7ff19c109220 2024-08-30T09:09:42.392 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.388+0000 7ff1a15a1640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7ff19c10f250 con 0x7ff19c101830 2024-08-30T09:09:42.392 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.388+0000 7ff19a7fc640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff19c1068c0 0x7ff19c1027c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:42.392 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.388+0000 7ff19a7fc640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff19c1068c0 0x7ff19c1027c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:50208/0 (socket says 172.21.15.42:50208) 2024-08-30T09:09:42.392 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.388+0000 7ff19a7fc640 1 -- 172.21.15.42:0/3736560242 learned_addr learned my addr 172.21.15.42:0/3736560242 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:09:42.392 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.388+0000 7ff19affd640 1 --2- 172.21.15.42:0/3736560242 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff19c101830 0x7ff19c103f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:42.392 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.388+0000 7ff19b7fe640 1 --2- 172.21.15.42:0/3736560242 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff19c109220 0x7ff19c102d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:42.392 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.388+0000 7ff19a7fc640 1 -- 172.21.15.42:0/3736560242 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff19c101830 msgr2=0x7ff19c103f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:09:42.392 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.388+0000 7ff19a7fc640 1 --2- 172.21.15.42:0/3736560242 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff19c101830 0x7ff19c103f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:42.392 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.388+0000 7ff19a7fc640 1 -- 172.21.15.42:0/3736560242 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff19c109220 msgr2=0x7ff19c102d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:09:42.392 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.388+0000 7ff19a7fc640 1 --2- 172.21.15.42:0/3736560242 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff19c109220 0x7ff19c102d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:42.393 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.388+0000 7ff19a7fc640 1 -- 172.21.15.42:0/3736560242 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff19c103430 con 0x7ff19c1068c0 2024-08-30T09:09:42.393 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.388+0000 7ff19a7fc640 1 --2- 172.21.15.42:0/3736560242 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff19c1068c0 0x7ff19c1027c0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7ff190006010 tx=0x7ff190004340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:09:42.393 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.388+0000 7ff183fff640 1 -- 172.21.15.42:0/3736560242 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff19003f030 con 0x7ff19c1068c0 2024-08-30T09:09:42.393 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.388+0000 7ff183fff640 1 -- 172.21.15.42:0/3736560242 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff190003cb0 con 0x7ff19c1068c0 2024-08-30T09:09:42.393 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.388+0000 7ff1a15a1640 1 -- 172.21.15.42:0/3736560242 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff19c1a3d60 con 0x7ff19c1068c0 2024-08-30T09:09:42.393 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.388+0000 7ff183fff640 1 -- 172.21.15.42:0/3736560242 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff190035070 con 0x7ff19c1068c0 2024-08-30T09:09:42.393 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.389+0000 7ff1a15a1640 1 -- 172.21.15.42:0/3736560242 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff19c1a4160 con 0x7ff19c1068c0 2024-08-30T09:09:42.397 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.391+0000 7ff1a85e5740 1 Processor -- start 2024-08-30T09:09:42.397 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.391+0000 7ff1a85e5740 1 -- start start 2024-08-30T09:09:42.397 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.391+0000 7ff1a85e5740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bb85e2d1b0 0x55bb861d2f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:42.397 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.391+0000 7ff1a85e5740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bb85e042f0 0x55bb861d35d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:42.398 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.391+0000 7ff1a85e5740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bb85e0c4d0 0x55bb861d4120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:42.398 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.391+0000 7ff1a85e5740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55bb86146000 con 0x55bb85e2d1b0 2024-08-30T09:09:42.398 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.391+0000 7ff1a85e5740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55bb86135590 con 0x55bb85e042f0 2024-08-30T09:09:42.398 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.391+0000 7ff1a85e5740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55bb86139540 con 0x55bb85e0c4d0 2024-08-30T09:09:42.398 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.391+0000 7ff183fff640 1 -- 172.21.15.42:0/3736560242 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7ff190042030 con 0x7ff19c1068c0 2024-08-30T09:09:42.398 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.391+0000 7ff19b7fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bb85e2d1b0 0x55bb861d2f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:42.398 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.391+0000 7ff19b7fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bb85e2d1b0 0x55bb861d2f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:50218/0 (socket says 172.21.15.42:50218) 2024-08-30T09:09:42.398 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.391+0000 7ff19b7fe640 1 -- 172.21.15.42:0/23210799 learned_addr learned my addr 172.21.15.42:0/23210799 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:09:42.398 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.391+0000 7ff19affd640 1 --2- 172.21.15.42:0/23210799 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bb85e042f0 0x55bb861d35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:42.398 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.392+0000 7ff19a7fc640 1 --2- 172.21.15.42:0/23210799 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bb85e0c4d0 0x55bb861d4120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:42.398 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.392+0000 7ff19b7fe640 1 -- 172.21.15.42:0/23210799 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bb85e0c4d0 msgr2=0x55bb861d4120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:09:42.398 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.392+0000 7ff19b7fe640 1 --2- 172.21.15.42:0/23210799 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bb85e0c4d0 0x55bb861d4120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:42.399 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.392+0000 7ff19b7fe640 1 -- 172.21.15.42:0/23210799 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bb85e042f0 msgr2=0x55bb861d35d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:09:42.399 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.392+0000 7ff19b7fe640 1 --2- 172.21.15.42:0/23210799 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bb85e042f0 0x55bb861d35d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:42.399 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.392+0000 7ff19b7fe640 1 -- 172.21.15.42:0/23210799 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb85e98790 con 0x55bb85e2d1b0 2024-08-30T09:09:42.399 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.392+0000 7ff19b7fe640 1 --2- 172.21.15.42:0/23210799 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bb85e2d1b0 0x55bb861d2f60 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7ff194009ee0 tx=0x7ff194010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf6cde93ab613664 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:09:42.399 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.392+0000 7ff183fff640 1 --2- 172.21.15.42:0/3736560242 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7ff1840b4b30 0x7ff1840b7040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:42.399 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.392+0000 7ff1817fa640 1 -- 172.21.15.42:0/23210799 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff194016070 con 0x55bb85e2d1b0 2024-08-30T09:09:42.399 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.392+0000 7ff1817fa640 1 -- 172.21.15.42:0/23210799 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff194004d20 con 0x55bb85e2d1b0 2024-08-30T09:09:42.399 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.392+0000 7ff1817fa640 1 -- 172.21.15.42:0/23210799 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff194012740 con 0x55bb85e2d1b0 2024-08-30T09:09:42.399 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.392+0000 7ff19affd640 1 --2- 172.21.15.42:0/3736560242 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7ff1840b4b30 0x7ff1840b7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:42.399 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.392+0000 7ff183fff640 1 -- 172.21.15.42:0/3736560242 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7ff190105b90 con 0x7ff19c1068c0 2024-08-30T09:09:42.399 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.393+0000 7ff1a85e5740 1 -- 172.21.15.42:0/23210799 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bb85e2d1b0 msgr2=0x55bb861d2f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:09:42.399 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.393+0000 7ff19affd640 1 --2- 172.21.15.42:0/3736560242 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7ff1840b4b30 0x7ff1840b7040 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7ff18c004610 tx=0x7ff18c00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:09:42.399 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.393+0000 7ff1a85e5740 1 --2- 172.21.15.42:0/23210799 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bb85e2d1b0 0x55bb861d2f60 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7ff194009ee0 tx=0x7ff194010740 comp rx=0 tx=0).stop 2024-08-30T09:09:42.399 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.393+0000 7ff1a85e5740 1 -- 172.21.15.42:0/23210799 shutdown_connections 2024-08-30T09:09:42.400 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.393+0000 7ff1a85e5740 1 --2- 172.21.15.42:0/23210799 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bb85e0c4d0 0x55bb861d4120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:42.400 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.393+0000 7ff1a85e5740 1 --2- 172.21.15.42:0/23210799 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bb85e042f0 0x55bb861d35d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:42.400 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.393+0000 7ff1a85e5740 1 --2- 172.21.15.42:0/23210799 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bb85e2d1b0 0x55bb861d2f60 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:42.400 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.393+0000 7ff1a85e5740 1 -- 172.21.15.42:0/23210799 >> 172.21.15.42:0/23210799 conn(0x55bb85e21300 msgr2=0x55bb861ce9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:09:42.400 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.393+0000 7ff1a85e5740 1 -- 172.21.15.42:0/23210799 shutdown_connections 2024-08-30T09:09:42.400 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.393+0000 7ff1a85e5740 1 -- 172.21.15.42:0/23210799 wait complete. 2024-08-30T09:09:42.400 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.393+0000 7ff1a85e5740 1 -- start start 2024-08-30T09:09:42.400 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.394+0000 7ff1a85e5740 1 Processor -- start 2024-08-30T09:09:42.400 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.394+0000 7ff1a85e5740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bb85e042f0 0x55bb861dca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:42.400 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.394+0000 7ff1a85e5740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bb85e0c4d0 0x55bb861dd100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:42.400 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.394+0000 7ff1a85e5740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bb85e2d1b0 0x55bb861ddc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:42.400 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.394+0000 7ff1a85e5740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55bb8606eab0 con 0x55bb85e2d1b0 2024-08-30T09:09:42.400 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.394+0000 7ff1a85e5740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55bb860bc680 con 0x55bb85e0c4d0 2024-08-30T09:09:42.401 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.394+0000 7ff1a85e5740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55bb861d4c40 con 0x55bb85e042f0 2024-08-30T09:09:42.401 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.394+0000 7ff1a85e5740 10 client.0 _ll_register_callbacks cb 0x55bb85e44a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:09:42.401 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.394+0000 7ff19affd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bb85e042f0 0x55bb861dca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:42.401 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.394+0000 7ff19affd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bb85e042f0 0x55bb861dca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:38356/0 (socket says 172.21.15.42:38356) 2024-08-30T09:09:42.401 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.394+0000 7ff19affd640 1 -- 172.21.15.42:0/720233924 learned_addr learned my addr 172.21.15.42:0/720233924 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:09:42.401 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.394+0000 7ff19b7fe640 1 --2- 172.21.15.42:0/720233924 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bb85e2d1b0 0x55bb861ddc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:42.401 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.394+0000 7ff19a7fc640 1 --2- 172.21.15.42:0/720233924 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bb85e0c4d0 0x55bb861dd100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:42.401 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.394+0000 7ff19affd640 1 -- 172.21.15.42:0/720233924 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bb85e0c4d0 msgr2=0x55bb861dd100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:09:42.401 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.394+0000 7ff19affd640 1 --2- 172.21.15.42:0/720233924 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bb85e0c4d0 0x55bb861dd100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:42.401 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.394+0000 7ff19affd640 1 -- 172.21.15.42:0/720233924 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bb85e2d1b0 msgr2=0x55bb861ddc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:09:42.401 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.394+0000 7ff19affd640 1 --2- 172.21.15.42:0/720233924 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bb85e2d1b0 0x55bb861ddc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:42.401 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.394+0000 7ff19affd640 1 -- 172.21.15.42:0/720233924 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb85f11620 con 0x55bb85e042f0 2024-08-30T09:09:42.401 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.394+0000 7ff19b7fe640 1 --2- 172.21.15.42:0/720233924 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bb85e2d1b0 0x55bb861ddc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-30T09:09:42.401 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.395+0000 7ff19affd640 1 --2- 172.21.15.42:0/720233924 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bb85e042f0 0x55bb861dca40 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7ff18c003ce0 tx=0x7ff18c004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=93f50b26298c0e4b server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:09:42.402 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.395+0000 7ff17eff5640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-08-30T09:09:42.402 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.395+0000 7ff17eff5640 1 -- 172.21.15.42:0/720233924 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff18c02c060 con 0x55bb85e042f0 2024-08-30T09:09:42.402 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.395+0000 7ff1a85e5740 1 -- 172.21.15.42:0/720233924 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff194004d20 con 0x55bb85e042f0 2024-08-30T09:09:42.403 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.395+0000 7ff17eff5640 1 -- 172.21.15.42:0/720233924 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7ff18c017030 con 0x55bb85e042f0 2024-08-30T09:09:42.403 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.395+0000 7ff17eff5640 1 -- 172.21.15.42:0/720233924 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7ff18c033050 con 0x55bb85e042f0 2024-08-30T09:09:42.403 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.395+0000 7ff1a85e5740 10 client.26704 fetch_fsmap learned FSMap version 37 2024-08-30T09:09:42.403 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.395+0000 7ff1a85e5740 1 -- 172.21.15.42:0/720233924 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55bb861e6ce0 con 0x55bb85e042f0 2024-08-30T09:09:42.403 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.395+0000 7ff17eff5640 1 -- 172.21.15.42:0/720233924 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff194004d20 con 0x55bb85e042f0 2024-08-30T09:09:42.403 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.397+0000 7ff17eff5640 1 -- 172.21.15.42:0/720233924 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7ff18c011440 con 0x55bb85e042f0 2024-08-30T09:09:42.403 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.397+0000 7ff1a85e5740 10 client.26704 fetch_fsmap finished waiting for FSMap version 37 2024-08-30T09:09:42.403 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.397+0000 7ff1a85e5740 10 client.26704 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-30T09:09:42.403 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.397+0000 7ff1a85e5740 20 client.26704 populate_metadata read hostname 'smithi042' 2024-08-30T09:09:42.403 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.397+0000 7ff1a85e5740 1 --2- 172.21.15.42:0/720233924 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55bb85debd80 0x55bb861ea910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:42.403 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.397+0000 7ff1a85e5740 4 client.26704 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-30T09:09:42.403 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.397+0000 7ff1a85e5740 1 -- 172.21.15.42:0/720233924 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bb86147f50 con 0x55bb85debd80 2024-08-30T09:09:42.403 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.398+0000 7ff19a7fc640 1 --2- 172.21.15.42:0/720233924 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55bb85debd80 0x55bb861ea910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:42.404 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.399+0000 7ff19a7fc640 1 --2- 172.21.15.42:0/720233924 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55bb85debd80 0x55bb861ea910 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f5c4547a821fbc3 server_cookie=1ce79672dceeb58b in_seq=0 out_seq=0 2024-08-30T09:09:42.404 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.399+0000 7ff17eff5640 10 client.26704 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:09:42.404 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.400+0000 7ff17eff5640 1 -- 172.21.15.42:0/720233924 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55bb86147f50 con 0x55bb85debd80 2024-08-30T09:09:42.404 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.400+0000 7ff17eff5640 10 client.26704 handle_command_reply: tid=0 2024-08-30T09:09:42.404 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.400+0000 7ff1a85e5740 1 client.26704 shutdown 2024-08-30T09:09:42.404 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.401+0000 7ff1a85e5740 1 -- 172.21.15.42:0/720233924 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bb85e042f0 msgr2=0x55bb861dca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:09:42.404 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.401+0000 7ff1a85e5740 1 --2- 172.21.15.42:0/720233924 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bb85e042f0 0x55bb861dca40 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7ff18c003ce0 tx=0x7ff18c004730 comp rx=0 tx=0).stop 2024-08-30T09:09:42.404 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.401+0000 7ff1a85e5740 1 -- 172.21.15.42:0/720233924 shutdown_connections 2024-08-30T09:09:42.405 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.401+0000 7ff1a85e5740 1 --2- 172.21.15.42:0/720233924 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55bb85debd80 0x55bb861ea910 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:42.405 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.401+0000 7ff1a85e5740 1 --2- 172.21.15.42:0/720233924 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bb85e2d1b0 0x55bb861ddc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:42.405 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.401+0000 7ff1a85e5740 1 --2- 172.21.15.42:0/720233924 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bb85e0c4d0 0x55bb861dd100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:42.405 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.401+0000 7ff1a85e5740 1 --2- 172.21.15.42:0/720233924 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bb85e042f0 0x55bb861dca40 secure :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7ff18c003ce0 tx=0x7ff18c004730 comp rx=0 tx=0).stop 2024-08-30T09:09:42.405 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.401+0000 7ff1a85e5740 1 -- 172.21.15.42:0/720233924 >> 172.21.15.42:0/720233924 conn(0x55bb85e21300 msgr2=0x55bb861ccc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:09:42.405 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.401+0000 7ff17eff5640 0 client.26704 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:09:42.408 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.406+0000 7ff1a85e5740 1 -- 172.21.15.42:0/720233924 shutdown_connections 2024-08-30T09:09:42.411 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.410+0000 7ff1a85e5740 1 -- 172.21.15.42:0/720233924 wait complete. 2024-08-30T09:09:42.412 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.410+0000 7ff1a85e5740 20 client.26704 trim_cache size 0 max 16384 2024-08-30T09:09:42.440 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.438+0000 7ff1a85e5740 1 Processor -- start 2024-08-30T09:09:42.440 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.438+0000 7ff1a85e5740 1 -- start start 2024-08-30T09:09:42.440 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.438+0000 7ff1a85e5740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bb85e0c4d0 0x55bb861dbc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:42.440 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.438+0000 7ff1a85e5740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bb85e2d1b0 0x55bb861e4b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:42.441 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.438+0000 7ff1a85e5740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bb85dc11c0 0x55bb861d2b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:42.441 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.438+0000 7ff1a85e5740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7ff19c1b0e80 con 0x55bb85e2d1b0 2024-08-30T09:09:42.441 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.438+0000 7ff1a85e5740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff19c1ae750 con 0x55bb85e0c4d0 2024-08-30T09:09:42.441 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.438+0000 7ff1a85e5740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7ff19c1ac5a0 con 0x55bb85dc11c0 2024-08-30T09:09:42.441 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.438+0000 7ff19a7fc640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bb85e2d1b0 0x55bb861e4b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:42.441 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.438+0000 7ff19affd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bb85e0c4d0 0x55bb861dbc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:42.441 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.438+0000 7ff19a7fc640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bb85e2d1b0 0x55bb861e4b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:50226/0 (socket says 172.21.15.42:50226) 2024-08-30T09:09:42.441 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.438+0000 7ff19a7fc640 1 -- 172.21.15.42:0/3578096761 learned_addr learned my addr 172.21.15.42:0/3578096761 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:09:42.442 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.439+0000 7ff19b7fe640 1 --2- 172.21.15.42:0/3578096761 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bb85dc11c0 0x55bb861d2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:42.442 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.439+0000 7ff19a7fc640 1 -- 172.21.15.42:0/3578096761 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bb85dc11c0 msgr2=0x55bb861d2b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:09:42.442 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.439+0000 7ff19a7fc640 1 --2- 172.21.15.42:0/3578096761 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bb85dc11c0 0x55bb861d2b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:42.442 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.439+0000 7ff19a7fc640 1 -- 172.21.15.42:0/3578096761 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bb85e0c4d0 msgr2=0x55bb861dbc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:09:42.442 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.439+0000 7ff19a7fc640 1 --2- 172.21.15.42:0/3578096761 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bb85e0c4d0 0x55bb861dbc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:42.442 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.439+0000 7ff19a7fc640 1 -- 172.21.15.42:0/3578096761 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb861ddfe0 con 0x55bb85e2d1b0 2024-08-30T09:09:42.442 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.439+0000 7ff19a7fc640 1 --2- 172.21.15.42:0/3578096761 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bb85e2d1b0 0x55bb861e4b50 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7ff1900c94a0 tx=0x7ff1900ef040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b65e83d714ce79f server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:09:42.442 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.440+0000 7ff180ff9640 1 -- 172.21.15.42:0/3578096761 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff190049040 con 0x55bb85e2d1b0 2024-08-30T09:09:42.442 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.440+0000 7ff180ff9640 1 -- 172.21.15.42:0/3578096761 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff19010a040 con 0x55bb85e2d1b0 2024-08-30T09:09:42.443 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.440+0000 7ff180ff9640 1 -- 172.21.15.42:0/3578096761 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff19003b460 con 0x55bb85e2d1b0 2024-08-30T09:09:42.444 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.441+0000 7ff1a85e5740 1 -- 172.21.15.42:0/3578096761 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bb85e2d1b0 msgr2=0x55bb861e4b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:09:42.444 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.441+0000 7ff1a85e5740 1 --2- 172.21.15.42:0/3578096761 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bb85e2d1b0 0x55bb861e4b50 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7ff1900c94a0 tx=0x7ff1900ef040 comp rx=0 tx=0).stop 2024-08-30T09:09:42.456 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.454+0000 7ff1a85e5740 1 -- 172.21.15.42:0/3578096761 shutdown_connections 2024-08-30T09:09:42.456 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.454+0000 7ff1a85e5740 1 --2- 172.21.15.42:0/3578096761 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bb85dc11c0 0x55bb861d2b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:42.457 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.454+0000 7ff1a85e5740 1 --2- 172.21.15.42:0/3578096761 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bb85e2d1b0 0x55bb861e4b50 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:42.457 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.454+0000 7ff1a85e5740 1 --2- 172.21.15.42:0/3578096761 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bb85e0c4d0 0x55bb861dbc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:42.457 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.454+0000 7ff1a85e5740 1 -- 172.21.15.42:0/3578096761 >> 172.21.15.42:0/3578096761 conn(0x55bb85e21300 msgr2=0x55bb861d3df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:09:42.466 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.464+0000 7ff1a85e5740 1 -- 172.21.15.42:0/3578096761 shutdown_connections 2024-08-30T09:09:42.471 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.468+0000 7ff1a85e5740 1 -- 172.21.15.42:0/3578096761 wait complete. 2024-08-30T09:09:42.471 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.468+0000 7ff1a85e5740 1 -- start start 2024-08-30T09:09:42.471 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.468+0000 7ff1a85e5740 1 Processor -- start 2024-08-30T09:09:42.471 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.468+0000 7ff1a85e5740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bb85dc11c0 0x55bb861e1410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:42.471 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.469+0000 7ff1a85e5740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bb85e0c4d0 0x55bb861e1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:42.471 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.469+0000 7ff1a85e5740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bb85e2d1b0 0x55bb861e2640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:42.471 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.469+0000 7ff1a85e5740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7ff19c1acc10 con 0x55bb85dc11c0 2024-08-30T09:09:42.471 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.469+0000 7ff1a85e5740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff19c1ae080 con 0x55bb85e0c4d0 2024-08-30T09:09:42.471 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.469+0000 7ff1a85e5740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7ff19c1ad2e0 con 0x55bb85e2d1b0 2024-08-30T09:09:42.471 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.469+0000 7ff1a85e5740 10 client.0 _ll_register_callbacks cb 0x7ff18c005c50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:09:42.472 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.469+0000 7ff19a7fc640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bb85e0c4d0 0x55bb861e1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:42.472 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.469+0000 7ff19a7fc640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bb85e0c4d0 0x55bb861e1ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:52048/0 (socket says 172.21.15.42:52048) 2024-08-30T09:09:42.472 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.469+0000 7ff19a7fc640 1 -- 172.21.15.42:0/3912779748 learned_addr learned my addr 172.21.15.42:0/3912779748 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:09:42.472 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.470+0000 7ff19a7fc640 1 -- 172.21.15.42:0/3912779748 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bb85e2d1b0 msgr2=0x55bb861e2640 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-30T09:09:42.472 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.470+0000 7ff19a7fc640 1 --2- 172.21.15.42:0/3912779748 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bb85e2d1b0 0x55bb861e2640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:42.472 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.470+0000 7ff19a7fc640 1 -- 172.21.15.42:0/3912779748 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bb85dc11c0 msgr2=0x55bb861e1410 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-30T09:09:42.472 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.470+0000 7ff19a7fc640 1 --2- 172.21.15.42:0/3912779748 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bb85dc11c0 0x55bb861e1410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:42.472 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.470+0000 7ff19a7fc640 1 -- 172.21.15.42:0/3912779748 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb861e2da0 con 0x55bb85e0c4d0 2024-08-30T09:09:42.475 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.473+0000 7ff19a7fc640 1 --2- 172.21.15.42:0/3912779748 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bb85e0c4d0 0x55bb861e1ad0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7ff1900c82b0 tx=0x7ff190005d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6b55237c97c6add4 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:09:42.478 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.474+0000 7ff17eff5640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-30T09:09:42.482 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.477+0000 7ff17eff5640 1 -- 172.21.15.42:0/3912779748 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff190049070 con 0x55bb85e0c4d0 2024-08-30T09:09:42.482 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.477+0000 7ff1a85e5740 1 -- 172.21.15.42:0/3912779748 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff19010a040 con 0x55bb85e0c4d0 2024-08-30T09:09:42.483 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.477+0000 7ff17eff5640 1 -- 172.21.15.42:0/3912779748 <== mon.2 v2:172.21.15.167:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7ff1900c84c0 con 0x55bb85e0c4d0 2024-08-30T09:09:42.483 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.478+0000 7ff17eff5640 1 -- 172.21.15.42:0/3912779748 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7ff1900cc230 con 0x55bb85e0c4d0 2024-08-30T09:09:42.486 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.483+0000 7ff1a85e5740 10 client.26729 fetch_fsmap learned FSMap version 37 2024-08-30T09:09:42.487 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.483+0000 7ff1a85e5740 1 -- 172.21.15.42:0/3912779748 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55bb861e4090 con 0x55bb85e0c4d0 2024-08-30T09:09:42.487 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.485+0000 7ff17eff5640 1 -- 172.21.15.42:0/3912779748 <== mon.2 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff19010a040 con 0x55bb85e0c4d0 2024-08-30T09:09:42.487 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.485+0000 7ff17eff5640 1 -- 172.21.15.42:0/3912779748 <== mon.2 v2:172.21.15.167:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7ff1900b1080 con 0x55bb85e0c4d0 2024-08-30T09:09:42.490 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.488+0000 7ff1a85e5740 10 client.26729 fetch_fsmap finished waiting for FSMap version 37 2024-08-30T09:09:42.490 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.488+0000 7ff1a85e5740 10 client.26729 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-30T09:09:42.490 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.488+0000 7ff1a85e5740 20 client.26729 populate_metadata read hostname 'smithi042' 2024-08-30T09:09:42.491 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.488+0000 7ff1a85e5740 1 --2- 172.21.15.42:0/3912779748 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55bb861e4a00 0x55bb862572e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:42.491 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.488+0000 7ff1a85e5740 4 client.26729 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-30T09:09:42.491 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.488+0000 7ff1a85e5740 1 -- 172.21.15.42:0/3912779748 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55bb861439f0 con 0x55bb861e4a00 2024-08-30T09:09:42.494 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.491+0000 7ff19affd640 1 --2- 172.21.15.42:0/3912779748 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55bb861e4a00 0x55bb862572e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:42.504 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.502+0000 7ff19affd640 1 --2- 172.21.15.42:0/3912779748 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55bb861e4a00 0x55bb862572e0 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=48acf0ad01276a27 server_cookie=cbcbccd932bfa342 in_seq=0 out_seq=0 2024-08-30T09:09:42.504 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.503+0000 7ff17eff5640 10 client.26729 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:09:42.574 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.572+0000 7ff17eff5640 1 -- 172.21.15.42:0/3912779748 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55bb861439f0 con 0x55bb861e4a00 2024-08-30T09:09:42.575 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.572+0000 7ff17eff5640 10 client.26729 handle_command_reply: tid=0 2024-08-30T09:09:42.575 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.572+0000 7ff1a85e5740 1 client.26729 shutdown 2024-08-30T09:09:42.576 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.573+0000 7ff1a85e5740 1 -- 172.21.15.42:0/3912779748 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bb85e0c4d0 msgr2=0x55bb861e1ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:09:42.576 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.573+0000 7ff1a85e5740 1 --2- 172.21.15.42:0/3912779748 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bb85e0c4d0 0x55bb861e1ad0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7ff1900c82b0 tx=0x7ff190005d50 comp rx=0 tx=0).stop 2024-08-30T09:09:42.576 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.573+0000 7ff1a85e5740 1 -- 172.21.15.42:0/3912779748 shutdown_connections 2024-08-30T09:09:42.576 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.573+0000 7ff1a85e5740 1 --2- 172.21.15.42:0/3912779748 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55bb861e4a00 0x55bb862572e0 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:42.576 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.573+0000 7ff1a85e5740 1 --2- 172.21.15.42:0/3912779748 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bb85e2d1b0 0x55bb861e2640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:42.576 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.573+0000 7ff1a85e5740 1 --2- 172.21.15.42:0/3912779748 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bb85e0c4d0 0x55bb861e1ad0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:42.576 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.573+0000 7ff1a85e5740 1 --2- 172.21.15.42:0/3912779748 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bb85dc11c0 0x55bb861e1410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:42.576 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.573+0000 7ff1a85e5740 1 -- 172.21.15.42:0/3912779748 >> 172.21.15.42:0/3912779748 conn(0x55bb85e21300 msgr2=0x55bb861d3df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:09:42.576 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.573+0000 7ff17eff5640 0 client.26729 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:09:42.576 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.573+0000 7ff1a85e5740 1 -- 172.21.15.42:0/3912779748 shutdown_connections 2024-08-30T09:09:42.576 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.574+0000 7ff1a85e5740 1 -- 172.21.15.42:0/3912779748 wait complete. 2024-08-30T09:09:42.576 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.574+0000 7ff1a85e5740 20 client.26729 trim_cache size 0 max 16384 2024-08-30T09:09:42.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:09:42 smithi167 ceph-mon[22063]: pgmap v641: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 4.3 MiB/s wr, 490 op/s 2024-08-30T09:09:42.577 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-08-30T09:09:42.577 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2024-08-30T09:09:42.577 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-08-30T09:09:42.577 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-08-30T09:09:42.578 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.575+0000 7ff1a15a1640 1 -- 172.21.15.42:0/3736560242 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7ff1840b4b30 msgr2=0x7ff1840b7040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:09:42.578 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.575+0000 7ff1a15a1640 1 --2- 172.21.15.42:0/3736560242 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7ff1840b4b30 0x7ff1840b7040 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7ff18c004610 tx=0x7ff18c00b390 comp rx=0 tx=0).stop 2024-08-30T09:09:42.578 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.575+0000 7ff1a15a1640 1 -- 172.21.15.42:0/3736560242 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff19c1068c0 msgr2=0x7ff19c1027c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:09:42.578 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.575+0000 7ff1a15a1640 1 --2- 172.21.15.42:0/3736560242 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff19c1068c0 0x7ff19c1027c0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7ff190006010 tx=0x7ff190004340 comp rx=0 tx=0).stop 2024-08-30T09:09:42.578 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.575+0000 7ff1a15a1640 1 -- 172.21.15.42:0/3736560242 shutdown_connections 2024-08-30T09:09:42.578 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.575+0000 7ff1a15a1640 1 --2- 172.21.15.42:0/3736560242 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7ff1840b4b30 0x7ff1840b7040 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:42.578 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.575+0000 7ff1a15a1640 1 --2- 172.21.15.42:0/3736560242 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff19c109220 0x7ff19c102d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:42.579 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.575+0000 7ff1a15a1640 1 --2- 172.21.15.42:0/3736560242 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff19c1068c0 0x7ff19c1027c0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:42.579 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.575+0000 7ff1a15a1640 1 --2- 172.21.15.42:0/3736560242 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff19c101830 0x7ff19c103f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:42.579 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.575+0000 7ff1a15a1640 1 -- 172.21.15.42:0/3736560242 >> 172.21.15.42:0/3736560242 conn(0x7ff19c0fd430 msgr2=0x7ff19c107790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:09:42.579 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.575+0000 7ff1a15a1640 1 -- 172.21.15.42:0/3736560242 shutdown_connections 2024-08-30T09:09:42.579 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:42.575+0000 7ff1a15a1640 1 -- 172.21.15.42:0/3736560242 wait complete. 2024-08-30T09:09:42.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:09:42 smithi042 ceph-mon[18652]: pgmap v641: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 4.3 MiB/s wr, 490 op/s 2024-08-30T09:09:42.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:09:42 smithi190 ceph-mon[22001]: pgmap v641: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 4.3 MiB/s wr, 490 op/s 2024-08-30T09:09:44.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:09:44 smithi167 ceph-mon[22063]: pgmap v642: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 5.1 MiB/s wr, 578 op/s 2024-08-30T09:09:44.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:09:44 smithi042 ceph-mon[18652]: pgmap v642: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 5.1 MiB/s wr, 578 op/s 2024-08-30T09:09:44.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:09:44 smithi190 ceph-mon[22001]: pgmap v642: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 5.1 MiB/s wr, 578 op/s 2024-08-30T09:09:45.953 INFO:teuthology.orchestra.run.smithi042.stderr:progress: 540.0 s, 439.8 tps, lat 230.941 ms stddev 303.440 2024-08-30T09:09:46.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:09:46 smithi167 ceph-mon[22063]: pgmap v643: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 4.7 MiB/s wr, 528 op/s 2024-08-30T09:09:46.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:09:46 smithi042 ceph-mon[18652]: pgmap v643: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 4.7 MiB/s wr, 528 op/s 2024-08-30T09:09:46.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:09:46 smithi190 ceph-mon[22001]: pgmap v643: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 4.7 MiB/s wr, 528 op/s 2024-08-30T09:09:47.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:09:47 smithi167 ceph-mon[22063]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-30T09:09:47.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:09:47 smithi042 ceph-mon[18652]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-30T09:09:47.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:09:47 smithi190 ceph-mon[22001]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-30T09:09:48.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:09:48 smithi167 ceph-mon[22063]: pgmap v644: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 4.4 MiB/s wr, 491 op/s 2024-08-30T09:09:48.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:09:48 smithi042 ceph-mon[18652]: pgmap v644: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 4.4 MiB/s wr, 491 op/s 2024-08-30T09:09:48.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:09:48 smithi190 ceph-mon[22001]: pgmap v644: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 4.4 MiB/s wr, 491 op/s 2024-08-30T09:09:50.953 INFO:teuthology.orchestra.run.smithi042.stderr:progress: 545.0 s, 451.6 tps, lat 217.583 ms stddev 269.444 2024-08-30T09:09:51.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:09:50 smithi167 ceph-mon[22063]: pgmap v645: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 4.5 MiB/s wr, 529 op/s 2024-08-30T09:09:51.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:09:50 smithi042 ceph-mon[18652]: pgmap v645: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 4.5 MiB/s wr, 529 op/s 2024-08-30T09:09:51.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:09:50 smithi190 ceph-mon[22001]: pgmap v645: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 4.5 MiB/s wr, 529 op/s 2024-08-30T09:09:51.760 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-30T09:09:52.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:09:51 smithi042 ceph-mon[18652]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-30T09:09:52.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:09:51 smithi190 ceph-mon[22001]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-30T09:09:52.303 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T09:09:52.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:09:51 smithi167 ceph-mon[22063]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-30T09:09:52.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:52.924+0000 7fb3424f7640 1 -- 172.21.15.42:0/3274969295 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb33c10a3f0 msgr2=0x7fb33c10a800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:09:52.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:52.924+0000 7fb3424f7640 1 --2- 172.21.15.42:0/3274969295 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb33c10a3f0 0x7fb33c10a800 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fb32c008570 tx=0x7fb32c02a1f0 comp rx=0 tx=0).stop 2024-08-30T09:09:52.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:52.931+0000 7fb3424f7640 1 -- 172.21.15.42:0/3274969295 shutdown_connections 2024-08-30T09:09:52.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:52.931+0000 7fb3424f7640 1 --2- 172.21.15.42:0/3274969295 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb33c1103e0 0x7fb33c116c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:52.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:52.931+0000 7fb3424f7640 1 --2- 172.21.15.42:0/3274969295 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb33c10fa90 0x7fb33c10fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:52.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:52.931+0000 7fb3424f7640 1 --2- 172.21.15.42:0/3274969295 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb33c10a3f0 0x7fb33c10a800 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:52.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:52.931+0000 7fb3424f7640 1 -- 172.21.15.42:0/3274969295 >> 172.21.15.42:0/3274969295 conn(0x7fb33c06ebf0 msgr2=0x7fb33c06f030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:09:52.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:52.932+0000 7fb3424f7640 1 -- 172.21.15.42:0/3274969295 shutdown_connections 2024-08-30T09:09:52.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:52.934+0000 7fb3424f7640 1 -- 172.21.15.42:0/3274969295 wait complete. 2024-08-30T09:09:52.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:52.935+0000 7fb3424f7640 1 Processor -- start 2024-08-30T09:09:52.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:52.935+0000 7fb3424f7640 1 -- start start 2024-08-30T09:09:52.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:52.936+0000 7fb3424f7640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb33c10a3f0 0x7fb33c1b1760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:52.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:52.936+0000 7fb3424f7640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb33c10fa90 0x7fb33c1b1d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:52.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:52.936+0000 7fb3424f7640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb33c1103e0 0x7fb33c1b6fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:52.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:52.936+0000 7fb3424f7640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fb33c119d00 con 0x7fb33c1103e0 2024-08-30T09:09:52.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:52.936+0000 7fb3424f7640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb33c119b80 con 0x7fb33c10a3f0 2024-08-30T09:09:52.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:52.936+0000 7fb3424f7640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fb33c119e80 con 0x7fb33c10fa90 2024-08-30T09:09:52.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:52.936+0000 7fb341cf6640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb33c1103e0 0x7fb33c1b6fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:52.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:52.936+0000 7fb341cf6640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb33c1103e0 0x7fb33c1b6fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:46180/0 (socket says 172.21.15.42:46180) 2024-08-30T09:09:52.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:52.936+0000 7fb341cf6640 1 -- 172.21.15.42:0/3803757475 learned_addr learned my addr 172.21.15.42:0/3803757475 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:09:52.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:52.937+0000 7fb341cf6640 1 -- 172.21.15.42:0/3803757475 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb33c10fa90 msgr2=0x7fb33c1b1d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:09:52.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:52.937+0000 7fb340cf4640 1 --2- 172.21.15.42:0/3803757475 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb33c10fa90 0x7fb33c1b1d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:52.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:52.937+0000 7fb341cf6640 1 --2- 172.21.15.42:0/3803757475 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb33c10fa90 0x7fb33c1b1d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:52.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:52.937+0000 7fb341cf6640 1 -- 172.21.15.42:0/3803757475 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb33c10a3f0 msgr2=0x7fb33c1b1760 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-30T09:09:52.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:52.937+0000 7fb341cf6640 1 --2- 172.21.15.42:0/3803757475 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb33c10a3f0 0x7fb33c1b1760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:52.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:52.937+0000 7fb341cf6640 1 -- 172.21.15.42:0/3803757475 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb33c1b76d0 con 0x7fb33c1103e0 2024-08-30T09:09:52.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:52.938+0000 7fb341cf6640 1 --2- 172.21.15.42:0/3803757475 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb33c1103e0 0x7fb33c1b6fa0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fb338005fd0 tx=0x7fb33800ded0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:09:52.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:52.939+0000 7fb3327fc640 1 -- 172.21.15.42:0/3803757475 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb33800cbe0 con 0x7fb33c1103e0 2024-08-30T09:09:52.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:52.940+0000 7fb3424f7640 1 -- 172.21.15.42:0/3803757475 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb33c1b79c0 con 0x7fb33c1103e0 2024-08-30T09:09:52.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:52.940+0000 7fb3424f7640 1 -- 172.21.15.42:0/3803757475 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb33c1a5200 con 0x7fb33c1103e0 2024-08-30T09:09:52.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:52.940+0000 7fb3327fc640 1 -- 172.21.15.42:0/3803757475 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb33800cd80 con 0x7fb33c1103e0 2024-08-30T09:09:52.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:52.940+0000 7fb3327fc640 1 -- 172.21.15.42:0/3803757475 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb338015410 con 0x7fb33c1103e0 2024-08-30T09:09:52.949 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:52.947+0000 7fb349cce740 1 Processor -- start 2024-08-30T09:09:52.949 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:52.947+0000 7fb349cce740 1 -- start start 2024-08-30T09:09:52.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:52.948+0000 7fb349cce740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5652418f7070 0x565241c9c460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:52.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:52.948+0000 7fb349cce740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5652418ce1b0 0x565241c9c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:52.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:52.948+0000 7fb3327fc640 1 -- 172.21.15.42:0/3803757475 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7fb33801c020 con 0x7fb33c1103e0 2024-08-30T09:09:52.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:52.948+0000 7fb340cf4640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5652418f7070 0x565241c9c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:52.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:52.948+0000 7fb340cf4640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5652418f7070 0x565241c9c460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:45218/0 (socket says 172.21.15.42:45218) 2024-08-30T09:09:52.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:52.948+0000 7fb340cf4640 1 -- 172.21.15.42:0/2398354342 learned_addr learned my addr 172.21.15.42:0/2398354342 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:09:52.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:52.949+0000 7fb349cce740 1 --2- 172.21.15.42:0/2398354342 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5652418d6390 0x565241c9cee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:52.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:52.949+0000 7fb349cce740 1 -- 172.21.15.42:0/2398354342 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x565241c13630 con 0x5652418ce1b0 2024-08-30T09:09:52.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:52.949+0000 7fb349cce740 1 -- 172.21.15.42:0/2398354342 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x565241c02bc0 con 0x5652418f7070 2024-08-30T09:09:52.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:52.949+0000 7fb349cce740 1 -- 172.21.15.42:0/2398354342 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x565241c06b70 con 0x5652418d6390 2024-08-30T09:09:52.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:52.949+0000 7fb3327fc640 1 --2- 172.21.15.42:0/3803757475 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fb3280b4ce0 0x7fb3280b71f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:52.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:52.949+0000 7fb3414f5640 1 --2- 172.21.15.42:0/2398354342 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5652418ce1b0 0x565241c9c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:52.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:52.950+0000 7fb3327fc640 1 -- 172.21.15.42:0/3803757475 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7fb3380e4e80 con 0x7fb33c1103e0 2024-08-30T09:09:52.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:52.950+0000 7fb341cf6640 1 --2- 172.21.15.42:0/3803757475 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fb3280b4ce0 0x7fb3280b71f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:52.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:52.950+0000 7fb3414f5640 1 -- 172.21.15.42:0/2398354342 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5652418d6390 msgr2=0x565241c9cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:09:52.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:52.950+0000 7fb3414f5640 1 --2- 172.21.15.42:0/2398354342 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5652418d6390 0x565241c9cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:52.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:52.950+0000 7fb3414f5640 1 -- 172.21.15.42:0/2398354342 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5652418f7070 msgr2=0x565241c9c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:09:52.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:52.950+0000 7fb3414f5640 1 --2- 172.21.15.42:0/2398354342 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5652418f7070 0x565241c9c460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:52.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:52.950+0000 7fb3414f5640 1 -- 172.21.15.42:0/2398354342 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565241962650 con 0x5652418ce1b0 2024-08-30T09:09:52.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:52.951+0000 7fb341cf6640 1 --2- 172.21.15.42:0/3803757475 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fb3280b4ce0 0x7fb3280b71f0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fb33800c8e0 tx=0x7fb33800c850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:09:52.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:52.952+0000 7fb3414f5640 1 --2- 172.21.15.42:0/2398354342 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5652418ce1b0 0x565241c9c9a0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fb32c02a7e0 tx=0x7fb32c005e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32d14d84cb2c2edf server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:09:52.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:52.952+0000 7fb3277fe640 1 -- 172.21.15.42:0/2398354342 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb32c009e00 con 0x5652418ce1b0 2024-08-30T09:09:52.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:52.952+0000 7fb3277fe640 1 -- 172.21.15.42:0/2398354342 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb32c040070 con 0x5652418ce1b0 2024-08-30T09:09:52.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:52.953+0000 7fb3277fe640 1 -- 172.21.15.42:0/2398354342 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb32c03b480 con 0x5652418ce1b0 2024-08-30T09:09:52.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:52.953+0000 7fb349cce740 1 -- 172.21.15.42:0/2398354342 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5652418ce1b0 msgr2=0x565241c9c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:09:52.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:52.953+0000 7fb349cce740 1 --2- 172.21.15.42:0/2398354342 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5652418ce1b0 0x565241c9c9a0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fb32c02a7e0 tx=0x7fb32c005e70 comp rx=0 tx=0).stop 2024-08-30T09:09:52.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:52.954+0000 7fb349cce740 1 -- 172.21.15.42:0/2398354342 shutdown_connections 2024-08-30T09:09:52.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:52.954+0000 7fb349cce740 1 --2- 172.21.15.42:0/2398354342 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5652418d6390 0x565241c9cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:52.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:52.954+0000 7fb349cce740 1 --2- 172.21.15.42:0/2398354342 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5652418ce1b0 0x565241c9c9a0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:52.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:52.954+0000 7fb349cce740 1 --2- 172.21.15.42:0/2398354342 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5652418f7070 0x565241c9c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:52.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:52.954+0000 7fb349cce740 1 -- 172.21.15.42:0/2398354342 >> 172.21.15.42:0/2398354342 conn(0x5652418eb460 msgr2=0x565241c9a350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:09:52.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:52.956+0000 7fb349cce740 1 -- 172.21.15.42:0/2398354342 shutdown_connections 2024-08-30T09:09:52.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:52.956+0000 7fb349cce740 1 -- 172.21.15.42:0/2398354342 wait complete. 2024-08-30T09:09:52.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:52.956+0000 7fb349cce740 1 -- start start 2024-08-30T09:09:52.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:52.956+0000 7fb349cce740 1 Processor -- start 2024-08-30T09:09:52.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:52.958+0000 7fb349cce740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5652418ce1b0 0x565241ca7b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:52.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:52.958+0000 7fb349cce740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5652418d6390 0x565241ca80a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:52.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:52.958+0000 7fb349cce740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5652418f7070 0x565241cb4000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:52.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:52.958+0000 7fb349cce740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x565241a65700 con 0x5652418f7070 2024-08-30T09:09:52.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:52.958+0000 7fb349cce740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x565241b89700 con 0x5652418ce1b0 2024-08-30T09:09:52.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:52.958+0000 7fb349cce740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x565241caac30 con 0x5652418d6390 2024-08-30T09:09:52.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:52.958+0000 7fb349cce740 10 client.0 _ll_register_callbacks cb 0x565241bb8dc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:09:52.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:52.958+0000 7fb340cf4640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5652418ce1b0 0x565241ca7b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:52.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:52.958+0000 7fb340cf4640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5652418ce1b0 0x565241ca7b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:45234/0 (socket says 172.21.15.42:45234) 2024-08-30T09:09:52.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:52.958+0000 7fb340cf4640 1 -- 172.21.15.42:0/1602650733 learned_addr learned my addr 172.21.15.42:0/1602650733 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:09:52.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:52.958+0000 7fb340cf4640 1 --2- 172.21.15.42:0/1602650733 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5652418f7070 0x565241cb4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:52.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:52.958+0000 7fb3414f5640 1 --2- 172.21.15.42:0/1602650733 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5652418d6390 0x565241ca80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:52.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:52.958+0000 7fb340cf4640 1 -- 172.21.15.42:0/1602650733 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5652418d6390 msgr2=0x565241ca80a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:09:52.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:52.958+0000 7fb340cf4640 1 --2- 172.21.15.42:0/1602650733 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5652418d6390 0x565241ca80a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:52.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:52.958+0000 7fb340cf4640 1 -- 172.21.15.42:0/1602650733 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5652418f7070 msgr2=0x565241cb4000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:09:52.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:52.958+0000 7fb340cf4640 1 --2- 172.21.15.42:0/1602650733 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5652418f7070 0x565241cb4000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:52.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:52.958+0000 7fb340cf4640 1 -- 172.21.15.42:0/1602650733 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565241bc28b0 con 0x5652418ce1b0 2024-08-30T09:09:52.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:52.959+0000 7fb340cf4640 1 --2- 172.21.15.42:0/1602650733 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5652418ce1b0 0x565241ca7b60 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fb334014fd0 tx=0x7fb334007af0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c4f95f238bc8054c server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:09:52.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:52.959+0000 7fb324ff9640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-30T09:09:52.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:52.959+0000 7fb324ff9640 1 -- 172.21.15.42:0/1602650733 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb33400a070 con 0x5652418ce1b0 2024-08-30T09:09:52.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:52.959+0000 7fb324ff9640 1 -- 172.21.15.42:0/1602650733 <== mon.2 v2:172.21.15.167:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7fb3340169b0 con 0x5652418ce1b0 2024-08-30T09:09:52.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:52.959+0000 7fb349cce740 1 -- 172.21.15.42:0/1602650733 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb32c040070 con 0x5652418ce1b0 2024-08-30T09:09:52.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:52.959+0000 7fb324ff9640 1 -- 172.21.15.42:0/1602650733 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fb334016bb0 con 0x5652418ce1b0 2024-08-30T09:09:52.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:52.959+0000 7fb349cce740 10 client.26735 fetch_fsmap learned FSMap version 39 2024-08-30T09:09:52.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:52.960+0000 7fb349cce740 1 -- 172.21.15.42:0/1602650733 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x565241cb6a90 con 0x5652418ce1b0 2024-08-30T09:09:52.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:52.960+0000 7fb324ff9640 1 -- 172.21.15.42:0/1602650733 <== mon.2 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb32c040070 con 0x5652418ce1b0 2024-08-30T09:09:52.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:52.960+0000 7fb324ff9640 1 -- 172.21.15.42:0/1602650733 <== mon.2 v2:172.21.15.167:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7fb334029020 con 0x5652418ce1b0 2024-08-30T09:09:52.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:52.960+0000 7fb349cce740 10 client.26735 fetch_fsmap finished waiting for FSMap version 39 2024-08-30T09:09:52.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:52.961+0000 7fb349cce740 10 client.26735 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-30T09:09:52.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:52.961+0000 7fb349cce740 20 client.26735 populate_metadata read hostname 'smithi042' 2024-08-30T09:09:52.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:52.961+0000 7fb349cce740 1 --2- 172.21.15.42:0/1602650733 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x5652418b59f0 0x565241cb9fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:52.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:52.961+0000 7fb349cce740 4 client.26735 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-30T09:09:52.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:52.961+0000 7fb349cce740 1 -- 172.21.15.42:0/1602650733 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x565241c15580 con 0x5652418b59f0 2024-08-30T09:09:52.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:52.963+0000 7fb3414f5640 1 --2- 172.21.15.42:0/1602650733 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x5652418b59f0 0x565241cb9fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:52.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:52.963+0000 7fb3414f5640 1 --2- 172.21.15.42:0/1602650733 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x5652418b59f0 0x565241cb9fe0 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f66d162e8de603d9 server_cookie=3a151a66969cd401 in_seq=0 out_seq=0 2024-08-30T09:09:52.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:52.963+0000 7fb324ff9640 10 client.26735 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:09:52.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:52.964+0000 7fb324ff9640 1 -- 172.21.15.42:0/1602650733 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x565241c15580 con 0x5652418b59f0 2024-08-30T09:09:52.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:52.964+0000 7fb324ff9640 10 client.26735 handle_command_reply: tid=0 2024-08-30T09:09:52.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:52.964+0000 7fb349cce740 1 client.26735 shutdown 2024-08-30T09:09:52.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:52.965+0000 7fb349cce740 1 -- 172.21.15.42:0/1602650733 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5652418ce1b0 msgr2=0x565241ca7b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:09:52.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:52.965+0000 7fb349cce740 1 --2- 172.21.15.42:0/1602650733 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5652418ce1b0 0x565241ca7b60 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fb334014fd0 tx=0x7fb334007af0 comp rx=0 tx=0).stop 2024-08-30T09:09:52.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:52.965+0000 7fb349cce740 1 -- 172.21.15.42:0/1602650733 shutdown_connections 2024-08-30T09:09:52.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:52.965+0000 7fb349cce740 1 --2- 172.21.15.42:0/1602650733 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x5652418b59f0 0x565241cb9fe0 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:52.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:52.966+0000 7fb324ff9640 0 client.26735 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:09:52.969 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:52.966+0000 7fb349cce740 1 --2- 172.21.15.42:0/1602650733 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5652418d6390 0x565241ca80a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:52.969 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:52.966+0000 7fb349cce740 1 --2- 172.21.15.42:0/1602650733 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5652418f7070 0x565241cb4000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:52.969 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:52.966+0000 7fb349cce740 1 --2- 172.21.15.42:0/1602650733 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5652418ce1b0 0x565241ca7b60 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:52.969 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:52.966+0000 7fb349cce740 1 -- 172.21.15.42:0/1602650733 >> 172.21.15.42:0/1602650733 conn(0x5652418eb460 msgr2=0x565241c9f5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:09:52.969 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:52.966+0000 7fb349cce740 1 -- 172.21.15.42:0/1602650733 shutdown_connections 2024-08-30T09:09:52.969 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:52.967+0000 7fb349cce740 1 -- 172.21.15.42:0/1602650733 wait complete. 2024-08-30T09:09:52.969 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:52.967+0000 7fb349cce740 20 client.26735 trim_cache size 0 max 16384 2024-08-30T09:09:52.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:52.991+0000 7fb349cce740 1 Processor -- start 2024-08-30T09:09:52.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:52.991+0000 7fb349cce740 1 -- start start 2024-08-30T09:09:52.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:52.991+0000 7fb349cce740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5652418b59f0 0x565241ca7f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:52.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:52.992+0000 7fb349cce740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5652418ce1b0 0x565241ca9a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:52.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:52.992+0000 7fb349cce740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5652418d6390 0x565241c9a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:52.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:52.992+0000 7fb349cce740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fb320012580 con 0x5652418d6390 2024-08-30T09:09:52.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:52.992+0000 7fb349cce740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb32000fdf0 con 0x5652418ce1b0 2024-08-30T09:09:52.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:52.992+0000 7fb349cce740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fb32000dc40 con 0x5652418b59f0 2024-08-30T09:09:52.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:52.992+0000 7fb340cf4640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5652418b59f0 0x565241ca7f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:52.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:52.992+0000 7fb3414f5640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5652418ce1b0 0x565241ca9a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:52.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:52.992+0000 7fb340cf4640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5652418b59f0 0x565241ca7f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:41680/0 (socket says 172.21.15.42:41680) 2024-08-30T09:09:52.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:52.992+0000 7fb340cf4640 1 -- 172.21.15.42:0/2545485205 learned_addr learned my addr 172.21.15.42:0/2545485205 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:09:52.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:52.992+0000 7fb340cf4640 1 --2- 172.21.15.42:0/2545485205 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5652418d6390 0x565241c9a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:52.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:52.993+0000 7fb340cf4640 1 -- 172.21.15.42:0/2545485205 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5652418ce1b0 msgr2=0x565241ca9a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:09:52.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:52.993+0000 7fb340cf4640 1 --2- 172.21.15.42:0/2545485205 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5652418ce1b0 0x565241ca9a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:52.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:52.993+0000 7fb340cf4640 1 -- 172.21.15.42:0/2545485205 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5652418d6390 msgr2=0x565241c9a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:09:52.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:52.993+0000 7fb340cf4640 1 --2- 172.21.15.42:0/2545485205 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5652418d6390 0x565241c9a0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:52.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:52.993+0000 7fb340cf4640 1 -- 172.21.15.42:0/2545485205 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565241c9f2c0 con 0x5652418b59f0 2024-08-30T09:09:52.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:52.993+0000 7fb3414f5640 1 --2- 172.21.15.42:0/2545485205 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5652418ce1b0 0x565241ca9a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-30T09:09:52.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:52.993+0000 7fb340cf4640 1 --2- 172.21.15.42:0/2545485205 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5652418b59f0 0x565241ca7f80 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fb334007940 tx=0x7fb33401a040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d1eeedcb72c34727 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:09:52.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:52.994+0000 7fb326ffd640 1 -- 172.21.15.42:0/2545485205 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb33400a070 con 0x5652418b59f0 2024-08-30T09:09:52.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:52.994+0000 7fb326ffd640 1 -- 172.21.15.42:0/2545485205 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb33400c040 con 0x5652418b59f0 2024-08-30T09:09:52.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:52.994+0000 7fb326ffd640 1 -- 172.21.15.42:0/2545485205 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb334018070 con 0x5652418b59f0 2024-08-30T09:09:52.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:52.995+0000 7fb349cce740 1 -- 172.21.15.42:0/2545485205 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5652418b59f0 msgr2=0x565241ca7f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:09:52.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:52.995+0000 7fb349cce740 1 --2- 172.21.15.42:0/2545485205 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5652418b59f0 0x565241ca7f80 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fb334007940 tx=0x7fb33401a040 comp rx=0 tx=0).stop 2024-08-30T09:09:52.998 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:09:52 smithi042 ceph-mon[18652]: pgmap v646: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 3.9 MiB/s wr, 455 op/s 2024-08-30T09:09:52.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:52.995+0000 7fb349cce740 1 -- 172.21.15.42:0/2545485205 shutdown_connections 2024-08-30T09:09:52.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:52.995+0000 7fb349cce740 1 --2- 172.21.15.42:0/2545485205 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5652418d6390 0x565241c9a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:52.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:52.995+0000 7fb349cce740 1 --2- 172.21.15.42:0/2545485205 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5652418ce1b0 0x565241ca9a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:52.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:52.995+0000 7fb349cce740 1 --2- 172.21.15.42:0/2545485205 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5652418b59f0 0x565241ca7f80 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:52.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:52.995+0000 7fb349cce740 1 -- 172.21.15.42:0/2545485205 >> 172.21.15.42:0/2545485205 conn(0x5652418eb460 msgr2=0x565241c999b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:09:52.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:52.996+0000 7fb349cce740 1 -- 172.21.15.42:0/2545485205 shutdown_connections 2024-08-30T09:09:52.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:52.996+0000 7fb349cce740 1 -- 172.21.15.42:0/2545485205 wait complete. 2024-08-30T09:09:52.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:52.996+0000 7fb349cce740 1 -- start start 2024-08-30T09:09:53.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:52.997+0000 7fb349cce740 1 Processor -- start 2024-08-30T09:09:53.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:52.997+0000 7fb349cce740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5652418b59f0 0x565241caff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:53.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:52.998+0000 7fb349cce740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5652418ce1b0 0x565241cb0480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:53.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:52.998+0000 7fb349cce740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5652418d6390 0x565241caaef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:53.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:52.998+0000 7fb349cce740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fb32000e2b0 con 0x5652418ce1b0 2024-08-30T09:09:53.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:52.998+0000 7fb340cf4640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5652418b59f0 0x565241caff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:53.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:52.998+0000 7fb349cce740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb32000f720 con 0x5652418d6390 2024-08-30T09:09:53.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:52.998+0000 7fb340cf4640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5652418b59f0 0x565241caff40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:41688/0 (socket says 172.21.15.42:41688) 2024-08-30T09:09:53.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:52.998+0000 7fb340cf4640 1 -- 172.21.15.42:0/2225197500 learned_addr learned my addr 172.21.15.42:0/2225197500 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:09:53.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:52.998+0000 7fb349cce740 1 -- 172.21.15.42:0/2225197500 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fb32000e980 con 0x5652418b59f0 2024-08-30T09:09:53.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:52.998+0000 7fb349cce740 10 client.0 _ll_register_callbacks cb 0x7fb334007e40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:09:53.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:52.998+0000 7fb3414f5640 1 --2- 172.21.15.42:0/2225197500 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5652418ce1b0 0x565241cb0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:53.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:52.999+0000 7fb340cf4640 1 --2- 172.21.15.42:0/2225197500 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5652418d6390 0x565241caaef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:53.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:52.999+0000 7fb340cf4640 1 -- 172.21.15.42:0/2225197500 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5652418d6390 msgr2=0x565241caaef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:09:53.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:52.999+0000 7fb340cf4640 1 --2- 172.21.15.42:0/2225197500 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5652418d6390 0x565241caaef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:53.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:52.999+0000 7fb340cf4640 1 -- 172.21.15.42:0/2225197500 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5652418ce1b0 msgr2=0x565241cb0480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:09:53.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:52.999+0000 7fb340cf4640 1 --2- 172.21.15.42:0/2225197500 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5652418ce1b0 0x565241cb0480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:53.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:52.999+0000 7fb340cf4640 1 -- 172.21.15.42:0/2225197500 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565241ca2790 con 0x5652418b59f0 2024-08-30T09:09:53.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:53.000+0000 7fb340cf4640 1 --2- 172.21.15.42:0/2225197500 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5652418b59f0 0x565241caff40 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fb334008010 tx=0x7fb334000f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4393b9f9a0dfba89 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:09:53.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:53.000+0000 7fb324ff9640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-08-30T09:09:53.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:53.000+0000 7fb324ff9640 1 -- 172.21.15.42:0/2225197500 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb33400a040 con 0x5652418b59f0 2024-08-30T09:09:53.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:53.001+0000 7fb324ff9640 1 -- 172.21.15.42:0/2225197500 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7fb334003a30 con 0x5652418b59f0 2024-08-30T09:09:53.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:53.003+0000 7fb349cce740 1 -- 172.21.15.42:0/2225197500 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb33400c040 con 0x5652418b59f0 2024-08-30T09:09:53.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:53.003+0000 7fb324ff9640 1 -- 172.21.15.42:0/2225197500 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fb334003c30 con 0x5652418b59f0 2024-08-30T09:09:53.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:53.003+0000 7fb349cce740 10 client.26716 fetch_fsmap learned FSMap version 39 2024-08-30T09:09:53.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:53.003+0000 7fb349cce740 1 -- 172.21.15.42:0/2225197500 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x565241cb68c0 con 0x5652418b59f0 2024-08-30T09:09:53.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:53.004+0000 7fb324ff9640 1 -- 172.21.15.42:0/2225197500 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb33400c040 con 0x5652418b59f0 2024-08-30T09:09:53.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:53.004+0000 7fb324ff9640 1 -- 172.21.15.42:0/2225197500 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7fb334036040 con 0x5652418b59f0 2024-08-30T09:09:53.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:53.004+0000 7fb349cce740 10 client.26716 fetch_fsmap finished waiting for FSMap version 39 2024-08-30T09:09:53.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:53.004+0000 7fb349cce740 10 client.26716 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-30T09:09:53.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:53.004+0000 7fb349cce740 20 client.26716 populate_metadata read hostname 'smithi042' 2024-08-30T09:09:53.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:53.004+0000 7fb349cce740 1 --2- 172.21.15.42:0/2225197500 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x5652418f7070 0x565241d27130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:53.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:53.004+0000 7fb349cce740 4 client.26716 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-30T09:09:53.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:53.004+0000 7fb349cce740 1 -- 172.21.15.42:0/2225197500 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "scrub status"}) -- 0x565241c11020 con 0x5652418f7070 2024-08-30T09:09:53.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:53.004+0000 7fb3414f5640 1 --2- 172.21.15.42:0/2225197500 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x5652418f7070 0x565241d27130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:53.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:53.005+0000 7fb324ff9640 10 client.26716 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:09:53.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:53.005+0000 7fb3414f5640 1 --2- 172.21.15.42:0/2225197500 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x5652418f7070 0x565241d27130 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d54eafc4ecb4093c server_cookie=5809823e5066c0e in_seq=0 out_seq=0 2024-08-30T09:09:53.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:53.005+0000 7fb324ff9640 1 -- 172.21.15.42:0/2225197500 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3214 (crc 0 0 0) 0x565241c11020 con 0x5652418f7070 2024-08-30T09:09:53.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:53.005+0000 7fb324ff9640 10 client.26716 handle_command_reply: tid=0 2024-08-30T09:09:53.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:53.005+0000 7fb349cce740 1 client.26716 shutdown 2024-08-30T09:09:53.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:53.005+0000 7fb349cce740 1 -- 172.21.15.42:0/2225197500 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5652418b59f0 msgr2=0x565241caff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:09:53.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:53.005+0000 7fb349cce740 1 --2- 172.21.15.42:0/2225197500 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5652418b59f0 0x565241caff40 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fb334008010 tx=0x7fb334000f40 comp rx=0 tx=0).stop 2024-08-30T09:09:53.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:53.005+0000 7fb349cce740 1 -- 172.21.15.42:0/2225197500 shutdown_connections 2024-08-30T09:09:53.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:53.005+0000 7fb349cce740 1 --2- 172.21.15.42:0/2225197500 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x5652418f7070 0x565241d27130 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:53.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:53.005+0000 7fb349cce740 1 --2- 172.21.15.42:0/2225197500 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5652418d6390 0x565241caaef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:53.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:53.005+0000 7fb349cce740 1 --2- 172.21.15.42:0/2225197500 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5652418ce1b0 0x565241cb0480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:53.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:53.005+0000 7fb349cce740 1 --2- 172.21.15.42:0/2225197500 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5652418b59f0 0x565241caff40 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:53.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:53.005+0000 7fb349cce740 1 -- 172.21.15.42:0/2225197500 >> 172.21.15.42:0/2225197500 conn(0x5652418eb460 msgr2=0x565241c9a0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:09:53.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:53.005+0000 7fb324ff9640 0 client.26716 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:09:53.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:53.006+0000 7fb349cce740 1 -- 172.21.15.42:0/2225197500 shutdown_connections 2024-08-30T09:09:53.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:53.006+0000 7fb349cce740 1 -- 172.21.15.42:0/2225197500 wait complete. 2024-08-30T09:09:53.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:53.006+0000 7fb349cce740 20 client.26716 trim_cache size 0 max 16384 2024-08-30T09:09:53.010 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-08-30T09:09:53.010 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "no active scrubs running", 2024-08-30T09:09:53.010 INFO:teuthology.orchestra.run.smithi042.stdout: "scrubs": {}, 2024-08-30T09:09:53.010 INFO:teuthology.orchestra.run.smithi042.stdout: "scrub_stats": { 2024-08-30T09:09:53.010 INFO:teuthology.orchestra.run.smithi042.stdout: "37f2f28f-496f-41a6-b4cf-8846fb3f43ab": { 2024-08-30T09:09:53.010 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:09:17.428286+0000", 2024-08-30T09:09:53.011 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:09:53.011 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:09:53.011 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:09:53.011 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:09:53.011 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:09:53.011 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:09:53.011 INFO:teuthology.orchestra.run.smithi042.stdout: "99a29ec1-9455-4ae4-854a-398b32896675": { 2024-08-30T09:09:53.011 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:08:24.923176+0000", 2024-08-30T09:09:53.011 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:09:53.011 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:09:53.011 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:09:53.011 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:09:53.011 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:09:53.012 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:09:53.012 INFO:teuthology.orchestra.run.smithi042.stdout: "598d8a75-f1cc-4388-80bc-a408bdb4eaac": { 2024-08-30T09:09:53.012 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:07:29.721122+0000", 2024-08-30T09:09:53.012 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:09:53.012 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:09:53.012 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:09:53.012 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:09:53.012 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:09:53.012 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:09:53.012 INFO:teuthology.orchestra.run.smithi042.stdout: "665ec740-7254-49cb-84ef-deff71c461b0": { 2024-08-30T09:09:53.012 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:06:37.144035+0000", 2024-08-30T09:09:53.012 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:09:53.012 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:09:53.012 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:09:53.013 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:09:53.013 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:09:53.013 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:09:53.013 INFO:teuthology.orchestra.run.smithi042.stdout: "1af4252a-4a99-47e9-bbf0-becf5e573dd5": { 2024-08-30T09:09:53.013 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:05:48.565881+0000", 2024-08-30T09:09:53.013 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:09:53.013 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:09:53.013 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:09:53.013 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:09:53.013 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:09:53.013 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:09:53.013 INFO:teuthology.orchestra.run.smithi042.stdout: "347d05ed-4bd3-4590-b099-314f719aab39": { 2024-08-30T09:09:53.014 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:05:00.439721+0000", 2024-08-30T09:09:53.014 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:09:53.014 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:09:53.014 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:09:53.014 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:09:53.014 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:09:53.014 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:09:53.014 INFO:teuthology.orchestra.run.smithi042.stdout: "88cd7478-2b20-47de-ba51-256479749232": { 2024-08-30T09:09:53.014 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:04:08.922625+0000", 2024-08-30T09:09:53.014 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:09:53.014 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:09:53.015 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:09:53.015 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:09:53.015 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:09:53.015 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:09:53.015 INFO:teuthology.orchestra.run.smithi042.stdout: "749f4867-0b70-49e5-af2d-8362a240ebc3": { 2024-08-30T09:09:53.015 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:03:16.386554+0000", 2024-08-30T09:09:53.015 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:09:53.015 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:09:53.015 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:09:53.015 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:09:53.015 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:09:53.015 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:09:53.015 INFO:teuthology.orchestra.run.smithi042.stdout: "85134e4b-ab4a-4885-b07e-a3ad01eaf13d": { 2024-08-30T09:09:53.015 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:01:27.738462+0000", 2024-08-30T09:09:53.016 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:09:53.016 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:09:53.016 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:09:53.016 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:09:53.016 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:09:53.016 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:09:53.016 INFO:teuthology.orchestra.run.smithi042.stdout: "c827ad7a-44ee-4e1d-9bb3-51fb7ab00c73": { 2024-08-30T09:09:53.016 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:02:21.686517+0000", 2024-08-30T09:09:53.016 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:09:53.016 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:09:53.016 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:09:53.016 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:09:53.016 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:09:53.016 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:09:53.016 INFO:teuthology.orchestra.run.smithi042.stdout: "5e2ed104-0625-478a-a78b-584e7f4c2817": { 2024-08-30T09:09:53.017 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:00:33.928400+0000", 2024-08-30T09:09:53.017 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:09:53.017 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:09:53.017 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:09:53.017 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:09:53.017 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:09:53.017 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:09:53.017 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:09:53.017 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-08-30T09:09:53.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:53.007+0000 7fb3424f7640 1 -- 172.21.15.42:0/3803757475 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fb3280b4ce0 msgr2=0x7fb3280b71f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:09:53.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:53.007+0000 7fb3424f7640 1 --2- 172.21.15.42:0/3803757475 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fb3280b4ce0 0x7fb3280b71f0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fb33800c8e0 tx=0x7fb33800c850 comp rx=0 tx=0).stop 2024-08-30T09:09:53.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:53.007+0000 7fb3424f7640 1 -- 172.21.15.42:0/3803757475 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb33c1103e0 msgr2=0x7fb33c1b6fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:09:53.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:53.007+0000 7fb3424f7640 1 --2- 172.21.15.42:0/3803757475 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb33c1103e0 0x7fb33c1b6fa0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fb338005fd0 tx=0x7fb33800ded0 comp rx=0 tx=0).stop 2024-08-30T09:09:53.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:53.007+0000 7fb3424f7640 1 -- 172.21.15.42:0/3803757475 shutdown_connections 2024-08-30T09:09:53.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:53.007+0000 7fb3424f7640 1 --2- 172.21.15.42:0/3803757475 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fb3280b4ce0 0x7fb3280b71f0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:53.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:53.007+0000 7fb3424f7640 1 --2- 172.21.15.42:0/3803757475 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb33c10fa90 0x7fb33c1b1d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:53.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:53.007+0000 7fb3424f7640 1 --2- 172.21.15.42:0/3803757475 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb33c1103e0 0x7fb33c1b6fa0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:53.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:53.007+0000 7fb3424f7640 1 --2- 172.21.15.42:0/3803757475 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb33c10a3f0 0x7fb33c1b1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:53.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:53.007+0000 7fb3424f7640 1 -- 172.21.15.42:0/3803757475 >> 172.21.15.42:0/3803757475 conn(0x7fb33c06ebf0 msgr2=0x7fb33c10dbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:09:53.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:53.007+0000 7fb3424f7640 1 -- 172.21.15.42:0/3803757475 shutdown_connections 2024-08-30T09:09:53.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:53.007+0000 7fb3424f7640 1 -- 172.21.15.42:0/3803757475 wait complete. 2024-08-30T09:09:53.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:09:52 smithi190 ceph-mon[22001]: pgmap v646: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 3.9 MiB/s wr, 455 op/s 2024-08-30T09:09:53.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:09:52 smithi167 ceph-mon[22063]: pgmap v646: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 3.9 MiB/s wr, 455 op/s 2024-08-30T09:09:53.879 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-08-30T09:09:53.880 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph fs dump --format=json 2024-08-30T09:09:54.385 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T09:09:55.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:55.000+0000 7f470ac16640 1 -- 172.21.15.42:0/3836725333 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f46fc095020 msgr2=0x7f46fc095430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:09:55.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:55.000+0000 7f470ac16640 1 --2- 172.21.15.42:0/3836725333 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f46fc095020 0x7f46fc095430 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f46f8009900 tx=0x7f46f802a410 comp rx=0 tx=0).stop 2024-08-30T09:09:55.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:55.001+0000 7f470ac16640 1 -- 172.21.15.42:0/3836725333 shutdown_connections 2024-08-30T09:09:55.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:55.001+0000 7f470ac16640 1 --2- 172.21.15.42:0/3836725333 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f46fc09a960 0x7f46fc09ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:55.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:55.001+0000 7f470ac16640 1 --2- 172.21.15.42:0/3836725333 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f46fc09b2b0 0x7f46fc0a1ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:55.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:55.001+0000 7f470ac16640 1 --2- 172.21.15.42:0/3836725333 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f46fc095020 0x7f46fc095430 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:55.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:55.001+0000 7f470ac16640 1 -- 172.21.15.42:0/3836725333 >> 172.21.15.42:0/3836725333 conn(0x7f46fc00cd70 msgr2=0x7f46fc00d180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:09:55.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:55.001+0000 7f470ac16640 1 -- 172.21.15.42:0/3836725333 shutdown_connections 2024-08-30T09:09:55.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:55.001+0000 7f470ac16640 1 -- 172.21.15.42:0/3836725333 wait complete. 2024-08-30T09:09:55.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:55.002+0000 7f470ac16640 1 Processor -- start 2024-08-30T09:09:55.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:55.002+0000 7f470ac16640 1 -- start start 2024-08-30T09:09:55.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:55.002+0000 7f470ac16640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f46fc095020 0x7f46fc12ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:55.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:55.002+0000 7f470ac16640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f46fc09a960 0x7f46fc12cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:55.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:55.002+0000 7f470ac16640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f46fc09b2b0 0x7f46fc132270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:55.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:55.002+0000 7f470ac16640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f46fc0a60c0 con 0x7f46fc09a960 2024-08-30T09:09:55.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:55.002+0000 7f470ac16640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f46fc0a5f40 con 0x7f46fc09b2b0 2024-08-30T09:09:55.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:55.002+0000 7f470ac16640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f46fc0a6240 con 0x7f46fc095020 2024-08-30T09:09:55.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:55.002+0000 7f47091b3640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f46fc09b2b0 0x7f46fc132270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:55.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:55.002+0000 7f47089b2640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f46fc095020 0x7f46fc12ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:55.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:55.002+0000 7f47091b3640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f46fc09b2b0 0x7f46fc132270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:45256/0 (socket says 172.21.15.42:45256) 2024-08-30T09:09:55.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:55.002+0000 7f47091b3640 1 -- 172.21.15.42:0/4089024704 learned_addr learned my addr 172.21.15.42:0/4089024704 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:09:55.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:55.003+0000 7f4703fff640 1 --2- 172.21.15.42:0/4089024704 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f46fc09a960 0x7f46fc12cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:55.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:55.003+0000 7f47091b3640 1 -- 172.21.15.42:0/4089024704 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f46fc095020 msgr2=0x7f46fc12ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:09:55.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:55.003+0000 7f47091b3640 1 --2- 172.21.15.42:0/4089024704 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f46fc095020 0x7f46fc12ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:55.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:55.003+0000 7f47091b3640 1 -- 172.21.15.42:0/4089024704 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f46fc09a960 msgr2=0x7f46fc12cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:09:55.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:55.003+0000 7f47091b3640 1 --2- 172.21.15.42:0/4089024704 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f46fc09a960 0x7f46fc12cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:55.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:55.003+0000 7f47091b3640 1 -- 172.21.15.42:0/4089024704 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46fc1329a0 con 0x7f46fc09b2b0 2024-08-30T09:09:55.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:55.003+0000 7f47091b3640 1 --2- 172.21.15.42:0/4089024704 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f46fc09b2b0 0x7f46fc132270 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f4704000b60 tx=0x7f4704079970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:09:55.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:55.003+0000 7f4701ffb640 1 -- 172.21.15.42:0/4089024704 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4704077580 con 0x7f46fc09b2b0 2024-08-30T09:09:55.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:55.003+0000 7f4701ffb640 1 -- 172.21.15.42:0/4089024704 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4704077bd0 con 0x7f46fc09b2b0 2024-08-30T09:09:55.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:55.003+0000 7f470ac16640 1 -- 172.21.15.42:0/4089024704 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46fc132c90 con 0x7f46fc09b2b0 2024-08-30T09:09:55.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:55.003+0000 7f4701ffb640 1 -- 172.21.15.42:0/4089024704 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f470406bb80 con 0x7f46fc09b2b0 2024-08-30T09:09:55.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:55.004+0000 7f470ac16640 1 -- 172.21.15.42:0/4089024704 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46fc1393f0 con 0x7f46fc09b2b0 2024-08-30T09:09:55.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:55.005+0000 7f470ac16640 1 -- 172.21.15.42:0/4089024704 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46fc095430 con 0x7f46fc09b2b0 2024-08-30T09:09:55.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:55.007+0000 7f4701ffb640 1 -- 172.21.15.42:0/4089024704 <== mon.2 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7f470408c060 con 0x7f46fc09b2b0 2024-08-30T09:09:55.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:55.008+0000 7f4701ffb640 1 --2- 172.21.15.42:0/4089024704 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f46f40b4b30 0x7f46f40b7040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:55.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:55.008+0000 7f4701ffb640 1 -- 172.21.15.42:0/4089024704 <== mon.2 v2:172.21.15.167:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f4704152a50 con 0x7f46fc09b2b0 2024-08-30T09:09:55.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:55.008+0000 7f47089b2640 1 --2- 172.21.15.42:0/4089024704 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f46f40b4b30 0x7f46f40b7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:55.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:55.011+0000 7f47089b2640 1 --2- 172.21.15.42:0/4089024704 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f46f40b4b30 0x7f46f40b7040 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f46f8005ab0 tx=0x7f46f8005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:09:55.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:55.012+0000 7f4701ffb640 1 -- 172.21.15.42:0/4089024704 <== mon.2 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f4704080040 con 0x7f46fc09b2b0 2024-08-30T09:09:55.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:09:54 smithi042 ceph-mon[18652]: pgmap v647: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 253 KiB/s rd, 4.6 MiB/s wr, 521 op/s 2024-08-30T09:09:55.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:09:54 smithi190 ceph-mon[22001]: pgmap v647: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 253 KiB/s rd, 4.6 MiB/s wr, 521 op/s 2024-08-30T09:09:55.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:09:54 smithi167 ceph-mon[22063]: pgmap v647: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 253 KiB/s rd, 4.6 MiB/s wr, 521 op/s 2024-08-30T09:09:55.393 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:55.391+0000 7f470ac16640 1 -- 172.21.15.42:0/4089024704 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f46fc12db60 con 0x7f46fc09b2b0 2024-08-30T09:09:55.394 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:55.392+0000 7f4701ffb640 1 -- 172.21.15.42:0/4089024704 <== mon.2 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+11921 (secure 0 0 0) 0x7f47041197f0 con 0x7f46fc09b2b0 2024-08-30T09:09:55.394 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-08-30T09:09:55.394 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":39,"btime":"2024-08-30T09:09:50:819266+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14703,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/3188400000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":3188400000},{"type":"v1","addr":"172.21.15.42:6835","nonce":3188400000}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14715,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/3356537674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":3356537674},{"type":"v1","addr":"172.21.15.42:6837","nonce":3356537674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14766,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6839/2107785513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6838","nonce":2107785513},{"type":"v1","addr":"172.21.15.42:6839","nonce":2107785513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14778,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6841/2298438674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6840","nonce":2298438674},{"type":"v1","addr":"172.21.15.42:6841","nonce":2298438674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24553,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/543555570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":543555570},{"type":"v1","addr":"172.21.15.190:6833","nonce":543555570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24554,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/3907818941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":3907818941},{"type":"v1","addr":"172.21.15.167:6833","nonce":3907818941}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":39,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-30T08:58:03.055084+0000","modified":"2024-08-30T09:09:50.673616+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24629,"mds_1":24610,"mds_2":24599},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24572":{"gid":24572,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6835/234596845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":234596845},{"type":"v1","addr":"172.21.15.167:6835","nonce":234596845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24592":{"gid":24592,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6835/247905709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":247905709},{"type":"v1","addr":"172.21.15.190:6835","nonce":247905709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24598":{"gid":24598,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6837/863256349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6836","nonce":863256349},{"type":"v1","addr":"172.21.15.190:6837","nonce":863256349}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24599":{"gid":24599,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.167:6837/3011837856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":3011837856},{"type":"v1","addr":"172.21.15.167:6837","nonce":3011837856}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24610":{"gid":24610,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.190:6839/719981110","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6838","nonce":719981110},{"type":"v1","addr":"172.21.15.190:6839","nonce":719981110}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24629":{"gid":24629,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.167:6839/3604645207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":3604645207},{"type":"v1","addr":"172.21.15.167:6839","nonce":3604645207}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24629,"qdb_cluster":[24599,24610,24629]},"id":1}]} 2024-08-30T09:09:55.394 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 39 2024-08-30T09:09:55.397 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:55.394+0000 7f470ac16640 1 -- 172.21.15.42:0/4089024704 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f46f40b4b30 msgr2=0x7f46f40b7040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:09:55.397 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:55.394+0000 7f470ac16640 1 --2- 172.21.15.42:0/4089024704 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f46f40b4b30 0x7f46f40b7040 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f46f8005ab0 tx=0x7f46f8005a20 comp rx=0 tx=0).stop 2024-08-30T09:09:55.397 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:55.394+0000 7f470ac16640 1 -- 172.21.15.42:0/4089024704 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f46fc09b2b0 msgr2=0x7f46fc132270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:09:55.397 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:55.394+0000 7f470ac16640 1 --2- 172.21.15.42:0/4089024704 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f46fc09b2b0 0x7f46fc132270 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f4704000b60 tx=0x7f4704079970 comp rx=0 tx=0).stop 2024-08-30T09:09:55.397 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:55.394+0000 7f470ac16640 1 -- 172.21.15.42:0/4089024704 shutdown_connections 2024-08-30T09:09:55.397 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:55.394+0000 7f470ac16640 1 --2- 172.21.15.42:0/4089024704 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f46f40b4b30 0x7f46f40b7040 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:55.397 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:55.394+0000 7f470ac16640 1 --2- 172.21.15.42:0/4089024704 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f46fc09b2b0 0x7f46fc132270 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:55.397 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:55.394+0000 7f470ac16640 1 --2- 172.21.15.42:0/4089024704 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f46fc09a960 0x7f46fc12cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:55.397 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:55.394+0000 7f470ac16640 1 --2- 172.21.15.42:0/4089024704 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f46fc095020 0x7f46fc12ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:55.397 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:55.394+0000 7f470ac16640 1 -- 172.21.15.42:0/4089024704 >> 172.21.15.42:0/4089024704 conn(0x7f46fc00cd70 msgr2=0x7f46fc0989f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:09:55.397 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:55.394+0000 7f470ac16640 1 -- 172.21.15.42:0/4089024704 shutdown_connections 2024-08-30T09:09:55.397 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:55.395+0000 7f470ac16640 1 -- 172.21.15.42:0/4089024704 wait complete. 2024-08-30T09:09:55.757 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.753+0000 7faed9044640 1 -- 172.21.15.42:0/2095581659 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faed4101af0 msgr2=0x7faed4101f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:09:55.757 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.753+0000 7faed9044640 1 --2- 172.21.15.42:0/2095581659 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faed4101af0 0x7faed4101f60 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7faecc00b9f0 tx=0x7faecc02c0f0 comp rx=0 tx=0).stop 2024-08-30T09:09:55.761 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.757+0000 7faed9044640 1 -- 172.21.15.42:0/2095581659 shutdown_connections 2024-08-30T09:09:55.761 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.757+0000 7faed9044640 1 --2- 172.21.15.42:0/2095581659 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faed4103510 0x7faed4109eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:55.761 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.757+0000 7faed9044640 1 --2- 172.21.15.42:0/2095581659 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faed4102b70 0x7faed4102f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:55.761 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.757+0000 7faed9044640 1 --2- 172.21.15.42:0/2095581659 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faed4101af0 0x7faed4101f60 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:55.761 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.757+0000 7faed9044640 1 -- 172.21.15.42:0/2095581659 >> 172.21.15.42:0/2095581659 conn(0x7faed40fd430 msgr2=0x7faed40ff8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:09:55.766 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.764+0000 7faed9044640 1 -- 172.21.15.42:0/2095581659 shutdown_connections 2024-08-30T09:09:55.767 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.765+0000 7faed9044640 1 -- 172.21.15.42:0/2095581659 wait complete. 2024-08-30T09:09:55.769 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.766+0000 7faed9044640 1 Processor -- start 2024-08-30T09:09:55.770 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.767+0000 7faed9044640 1 -- start start 2024-08-30T09:09:55.770 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.768+0000 7faed9044640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faed4101af0 0x7faed4197b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:55.771 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.768+0000 7faed9044640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faed4102b70 0x7faed41981d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:55.771 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.768+0000 7faed9044640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faed4103510 0x7faed4198d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:55.771 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.768+0000 7faed2d9d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faed4101af0 0x7faed4197b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:55.771 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.768+0000 7faed2d9d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faed4101af0 0x7faed4197b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:41718/0 (socket says 172.21.15.42:41718) 2024-08-30T09:09:55.771 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.769+0000 7faed2d9d640 1 -- 172.21.15.42:0/2412669124 learned_addr learned my addr 172.21.15.42:0/2412669124 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:09:55.772 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.769+0000 7faed9044640 1 -- 172.21.15.42:0/2412669124 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7faed410cf30 con 0x7faed4103510 2024-08-30T09:09:55.772 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.769+0000 7faed9044640 1 -- 172.21.15.42:0/2412669124 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7faed410cdb0 con 0x7faed4102b70 2024-08-30T09:09:55.772 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.769+0000 7faed9044640 1 -- 172.21.15.42:0/2412669124 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7faed410d0b0 con 0x7faed4101af0 2024-08-30T09:09:55.772 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.769+0000 7faed259c640 1 --2- 172.21.15.42:0/2412669124 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faed4102b70 0x7faed41981d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:55.773 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.769+0000 7faed2d9d640 1 -- 172.21.15.42:0/2412669124 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faed4102b70 msgr2=0x7faed41981d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:09:55.773 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.769+0000 7faed2d9d640 1 --2- 172.21.15.42:0/2412669124 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faed4102b70 0x7faed41981d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:55.773 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.770+0000 7faed2d9d640 1 -- 172.21.15.42:0/2412669124 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faed4103510 msgr2=0x7faed4198d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:09:55.773 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.770+0000 7faed2d9d640 1 --2- 172.21.15.42:0/2412669124 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faed4103510 0x7faed4198d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:55.773 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.770+0000 7faed2d9d640 1 -- 172.21.15.42:0/2412669124 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faed41995a0 con 0x7faed4101af0 2024-08-30T09:09:55.773 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.770+0000 7faed2d9d640 1 --2- 172.21.15.42:0/2412669124 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faed4101af0 0x7faed4197b60 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7faecc009570 tx=0x7faecc004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:09:55.775 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.770+0000 7faecbfff640 1 -- 172.21.15.42:0/2412669124 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faecc042070 con 0x7faed4101af0 2024-08-30T09:09:55.775 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.770+0000 7faecbfff640 1 -- 172.21.15.42:0/2412669124 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faecc009980 con 0x7faed4101af0 2024-08-30T09:09:55.775 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.770+0000 7faecbfff640 1 -- 172.21.15.42:0/2412669124 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faecc039070 con 0x7faed4101af0 2024-08-30T09:09:55.775 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.770+0000 7faed9044640 1 -- 172.21.15.42:0/2412669124 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faed4199830 con 0x7faed4101af0 2024-08-30T09:09:55.775 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.771+0000 7faed9044640 1 -- 172.21.15.42:0/2412669124 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faed4199d70 con 0x7faed4101af0 2024-08-30T09:09:55.780 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.778+0000 7faecbfff640 1 -- 172.21.15.42:0/2412669124 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7faecc045070 con 0x7faed4101af0 2024-08-30T09:09:55.782 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.780+0000 7faecbfff640 1 --2- 172.21.15.42:0/2412669124 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7faec00b4c70 0x7faec00b7180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:55.782 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.780+0000 7faed259c640 1 --2- 172.21.15.42:0/2412669124 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7faec00b4c70 0x7faec00b7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:55.782 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.780+0000 7faecbfff640 1 -- 172.21.15.42:0/2412669124 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7faecc105ed0 con 0x7faed4101af0 2024-08-30T09:09:55.789 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.784+0000 7faee008f740 1 Processor -- start 2024-08-30T09:09:55.789 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.784+0000 7faee008f740 1 -- start start 2024-08-30T09:09:55.789 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.785+0000 7faee008f740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55feed7e31b0 0x55feedb891d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:55.789 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.785+0000 7faee008f740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55feed7ba2f0 0x55feedb89840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:55.789 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.785+0000 7faee008f740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55feed7c24d0 0x55feedb8a390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:55.789 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.785+0000 7faee008f740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55feedafc000 con 0x55feed7ba2f0 2024-08-30T09:09:55.789 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.785+0000 7faee008f740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55feedaeb590 con 0x55feed7c24d0 2024-08-30T09:09:55.789 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.785+0000 7faee008f740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55feedaef540 con 0x55feed7e31b0 2024-08-30T09:09:55.789 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.785+0000 7faed259c640 1 --2- 172.21.15.42:0/2412669124 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7faec00b4c70 0x7faec00b7180 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7faebc00a530 tx=0x7faebc00a4c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:09:55.789 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.786+0000 7faed259c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55feed7ba2f0 0x55feedb89840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:55.789 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.786+0000 7faed259c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55feed7ba2f0 0x55feedb89840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:46266/0 (socket says 172.21.15.42:46266) 2024-08-30T09:09:55.789 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.786+0000 7faed259c640 1 -- 172.21.15.42:0/3259694506 learned_addr learned my addr 172.21.15.42:0/3259694506 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:09:55.789 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.786+0000 7faed2d9d640 1 --2- 172.21.15.42:0/3259694506 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55feed7e31b0 0x55feedb891d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:55.792 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.786+0000 7faed259c640 1 -- 172.21.15.42:0/3259694506 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55feed7e31b0 msgr2=0x55feedb891d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:09:55.792 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.786+0000 7faed259c640 1 --2- 172.21.15.42:0/3259694506 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55feed7e31b0 0x55feedb891d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:55.792 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.786+0000 7faed259c640 1 -- 172.21.15.42:0/3259694506 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55feed7c24d0 msgr2=0x55feedb8a390 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-30T09:09:55.792 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.786+0000 7faed259c640 1 --2- 172.21.15.42:0/3259694506 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55feed7c24d0 0x55feedb8a390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:55.794 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.786+0000 7faed259c640 1 -- 172.21.15.42:0/3259694506 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55feed84e790 con 0x55feed7ba2f0 2024-08-30T09:09:55.794 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.787+0000 7faed259c640 1 --2- 172.21.15.42:0/3259694506 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55feed7ba2f0 0x55feedb89840 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7faebc003bc0 tx=0x7faebc014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36d74e5387521a30 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:09:55.795 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.788+0000 7faec97fa640 1 -- 172.21.15.42:0/3259694506 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faebc00eb50 con 0x55feed7ba2f0 2024-08-30T09:09:55.795 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.788+0000 7faec97fa640 1 -- 172.21.15.42:0/3259694506 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faebc00ecf0 con 0x55feed7ba2f0 2024-08-30T09:09:55.795 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.788+0000 7faec97fa640 1 -- 172.21.15.42:0/3259694506 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faebc004de0 con 0x55feed7ba2f0 2024-08-30T09:09:55.795 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.790+0000 7faee008f740 1 -- 172.21.15.42:0/3259694506 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55feed7ba2f0 msgr2=0x55feedb89840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:09:55.795 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.790+0000 7faee008f740 1 --2- 172.21.15.42:0/3259694506 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55feed7ba2f0 0x55feedb89840 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7faebc003bc0 tx=0x7faebc014000 comp rx=0 tx=0).stop 2024-08-30T09:09:55.795 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.793+0000 7faee008f740 1 -- 172.21.15.42:0/3259694506 shutdown_connections 2024-08-30T09:09:55.795 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.793+0000 7faee008f740 1 --2- 172.21.15.42:0/3259694506 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55feed7c24d0 0x55feedb8a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:55.796 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.793+0000 7faee008f740 1 --2- 172.21.15.42:0/3259694506 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55feed7ba2f0 0x55feedb89840 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:55.799 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.793+0000 7faee008f740 1 --2- 172.21.15.42:0/3259694506 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55feed7e31b0 0x55feedb891d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:55.799 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.793+0000 7faee008f740 1 -- 172.21.15.42:0/3259694506 >> 172.21.15.42:0/3259694506 conn(0x55feed7d7300 msgr2=0x55feedb84c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:09:55.799 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.793+0000 7faee008f740 1 -- 172.21.15.42:0/3259694506 shutdown_connections 2024-08-30T09:09:55.799 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.793+0000 7faee008f740 1 -- 172.21.15.42:0/3259694506 wait complete. 2024-08-30T09:09:55.799 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.793+0000 7faee008f740 1 -- start start 2024-08-30T09:09:55.799 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.794+0000 7faee008f740 1 Processor -- start 2024-08-30T09:09:55.800 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.794+0000 7faee008f740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55feed7ba2f0 0x55feedba3cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:55.800 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.794+0000 7faee008f740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55feed7c24d0 0x55feedba4370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:55.800 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.794+0000 7faee008f740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55feed7e31b0 0x55feedba4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:55.800 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.794+0000 7faed2d9d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55feed7ba2f0 0x55feedba3cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:55.800 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.794+0000 7faee008f740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55feeda24ab0 con 0x55feed7c24d0 2024-08-30T09:09:55.800 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.794+0000 7faee008f740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55feeda72680 con 0x55feed7e31b0 2024-08-30T09:09:55.800 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.794+0000 7faee008f740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55feedb8aeb0 con 0x55feed7ba2f0 2024-08-30T09:09:55.800 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.794+0000 7faed2d9d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55feed7ba2f0 0x55feedba3cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:41738/0 (socket says 172.21.15.42:41738) 2024-08-30T09:09:55.800 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.794+0000 7faed2d9d640 1 -- 172.21.15.42:0/367099351 learned_addr learned my addr 172.21.15.42:0/367099351 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:09:55.800 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.795+0000 7faee008f740 10 client.0 _ll_register_callbacks cb 0x55feed7faa00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:09:55.800 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.795+0000 7faed259c640 1 --2- 172.21.15.42:0/367099351 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55feed7c24d0 0x55feedba4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:55.801 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.795+0000 7faed359e640 1 --2- 172.21.15.42:0/367099351 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55feed7e31b0 0x55feedba4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:55.801 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.795+0000 7faed2d9d640 1 -- 172.21.15.42:0/367099351 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55feed7e31b0 msgr2=0x55feedba4ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:09:55.801 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.795+0000 7faed2d9d640 1 --2- 172.21.15.42:0/367099351 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55feed7e31b0 0x55feedba4ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:55.801 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.795+0000 7faed2d9d640 1 -- 172.21.15.42:0/367099351 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55feed7c24d0 msgr2=0x55feedba4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:09:55.801 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.795+0000 7faed2d9d640 1 --2- 172.21.15.42:0/367099351 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55feed7c24d0 0x55feedba4370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:55.801 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.795+0000 7faed2d9d640 1 -- 172.21.15.42:0/367099351 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55feed8c7620 con 0x55feed7ba2f0 2024-08-30T09:09:55.801 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.795+0000 7faed2d9d640 1 --2- 172.21.15.42:0/367099351 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55feed7ba2f0 0x55feedba3cb0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7faecc02c660 tx=0x7faecc02cf30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4c7edf2188dbf93f server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:09:55.801 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.795+0000 7faeba7fc640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-08-30T09:09:55.801 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.796+0000 7faeba7fc640 1 -- 172.21.15.42:0/367099351 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faecc0ccc10 con 0x55feed7ba2f0 2024-08-30T09:09:55.801 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.796+0000 7faee008f740 1 -- 172.21.15.42:0/367099351 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faebc00ecf0 con 0x55feed7ba2f0 2024-08-30T09:09:55.801 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.796+0000 7faeba7fc640 1 -- 172.21.15.42:0/367099351 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7faecc0ef2d0 con 0x55feed7ba2f0 2024-08-30T09:09:55.801 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.796+0000 7faeba7fc640 1 -- 172.21.15.42:0/367099351 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7faecc0cc510 con 0x55feed7ba2f0 2024-08-30T09:09:55.801 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.796+0000 7faee008f740 10 client.26731 fetch_fsmap learned FSMap version 39 2024-08-30T09:09:55.802 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.796+0000 7faee008f740 1 -- 172.21.15.42:0/367099351 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55feedbade30 con 0x55feed7ba2f0 2024-08-30T09:09:55.802 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.796+0000 7faeba7fc640 1 -- 172.21.15.42:0/367099351 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faecc0d0d80 con 0x55feed7ba2f0 2024-08-30T09:09:55.802 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.796+0000 7faeba7fc640 1 -- 172.21.15.42:0/367099351 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7faecc0d53a0 con 0x55feed7ba2f0 2024-08-30T09:09:55.802 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.797+0000 7faee008f740 10 client.26731 fetch_fsmap finished waiting for FSMap version 39 2024-08-30T09:09:55.802 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.797+0000 7faee008f740 10 client.26731 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-30T09:09:55.802 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.797+0000 7faee008f740 20 client.26731 populate_metadata read hostname 'smithi042' 2024-08-30T09:09:55.802 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.797+0000 7faee008f740 1 --2- 172.21.15.42:0/367099351 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55feed7a1d80 0x55feedbb1710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:55.802 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.797+0000 7faee008f740 4 client.26731 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-30T09:09:55.802 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.797+0000 7faee008f740 1 -- 172.21.15.42:0/367099351 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7faebc00ecf0 con 0x55feed7a1d80 2024-08-30T09:09:55.802 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.797+0000 7faed259c640 1 --2- 172.21.15.42:0/367099351 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55feed7a1d80 0x55feedbb1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:55.802 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.797+0000 7faeba7fc640 10 client.26731 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:09:55.802 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.797+0000 7faed259c640 1 --2- 172.21.15.42:0/367099351 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55feed7a1d80 0x55feedbb1710 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=50e62ff82d171f44 server_cookie=e9e17231789e18d9 in_seq=0 out_seq=0 2024-08-30T09:09:55.804 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.798+0000 7faeba7fc640 1 -- 172.21.15.42:0/367099351 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7faebc00ecf0 con 0x55feed7a1d80 2024-08-30T09:09:55.804 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.798+0000 7faeba7fc640 10 client.26731 handle_command_reply: tid=0 2024-08-30T09:09:55.804 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.798+0000 7faee008f740 1 client.26731 shutdown 2024-08-30T09:09:55.804 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.799+0000 7faee008f740 1 -- 172.21.15.42:0/367099351 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55feed7ba2f0 msgr2=0x55feedba3cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:09:55.804 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.799+0000 7faee008f740 1 --2- 172.21.15.42:0/367099351 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55feed7ba2f0 0x55feedba3cb0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7faecc02c660 tx=0x7faecc02cf30 comp rx=0 tx=0).stop 2024-08-30T09:09:55.804 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.799+0000 7faee008f740 1 -- 172.21.15.42:0/367099351 shutdown_connections 2024-08-30T09:09:55.804 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.799+0000 7faee008f740 1 --2- 172.21.15.42:0/367099351 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55feed7a1d80 0x55feedbb1710 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:55.804 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.799+0000 7faee008f740 1 --2- 172.21.15.42:0/367099351 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55feed7e31b0 0x55feedba4ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:55.804 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.799+0000 7faee008f740 1 --2- 172.21.15.42:0/367099351 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55feed7c24d0 0x55feedba4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:55.804 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.799+0000 7faeba7fc640 0 client.26731 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:09:55.804 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.799+0000 7faee008f740 1 --2- 172.21.15.42:0/367099351 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55feed7ba2f0 0x55feedba3cb0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:55.804 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.799+0000 7faee008f740 1 -- 172.21.15.42:0/367099351 >> 172.21.15.42:0/367099351 conn(0x55feed7d7300 msgr2=0x55feedb831c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:09:55.805 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.799+0000 7faee008f740 1 -- 172.21.15.42:0/367099351 shutdown_connections 2024-08-30T09:09:55.805 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.799+0000 7faee008f740 1 -- 172.21.15.42:0/367099351 wait complete. 2024-08-30T09:09:55.805 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.799+0000 7faee008f740 20 client.26731 trim_cache size 0 max 16384 2024-08-30T09:09:55.820 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.818+0000 7faee008f740 1 Processor -- start 2024-08-30T09:09:55.820 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.818+0000 7faee008f740 1 -- start start 2024-08-30T09:09:55.821 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.818+0000 7faee008f740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55feed7ba2f0 0x55feedb845a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:55.821 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.819+0000 7faee008f740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55feed7c24d0 0x55feedba23e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:55.821 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.819+0000 7faee008f740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55feed7e31b0 0x55feedba2920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:55.821 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.819+0000 7faee008f740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7faeb4012400 con 0x55feed7ba2f0 2024-08-30T09:09:55.821 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.819+0000 7faee008f740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7faeb400fc70 con 0x55feed7e31b0 2024-08-30T09:09:55.821 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.819+0000 7faed2d9d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55feed7ba2f0 0x55feedb845a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:55.821 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.819+0000 7faee008f740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7faeb400dac0 con 0x55feed7c24d0 2024-08-30T09:09:55.821 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.819+0000 7faed2d9d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55feed7ba2f0 0x55feedb845a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:46282/0 (socket says 172.21.15.42:46282) 2024-08-30T09:09:55.822 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.819+0000 7faed2d9d640 1 -- 172.21.15.42:0/1838913150 learned_addr learned my addr 172.21.15.42:0/1838913150 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:09:55.822 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.819+0000 7faed259c640 1 --2- 172.21.15.42:0/1838913150 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55feed7c24d0 0x55feedba23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:55.822 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.819+0000 7faed359e640 1 --2- 172.21.15.42:0/1838913150 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55feed7e31b0 0x55feedba2920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:55.822 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.819+0000 7faed2d9d640 1 -- 172.21.15.42:0/1838913150 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55feed7c24d0 msgr2=0x55feedba23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:09:55.822 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.819+0000 7faed2d9d640 1 --2- 172.21.15.42:0/1838913150 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55feed7c24d0 0x55feedba23e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:55.822 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.819+0000 7faed2d9d640 1 -- 172.21.15.42:0/1838913150 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55feed7e31b0 msgr2=0x55feedba2920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:09:55.823 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.819+0000 7faed2d9d640 1 --2- 172.21.15.42:0/1838913150 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55feed7e31b0 0x55feedba2920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:55.823 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.819+0000 7faed2d9d640 1 -- 172.21.15.42:0/1838913150 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55feedbac180 con 0x55feed7ba2f0 2024-08-30T09:09:55.823 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.820+0000 7faed2d9d640 1 --2- 172.21.15.42:0/1838913150 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55feed7ba2f0 0x55feedb845a0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7faecc0c9780 tx=0x7faecc02cec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ab7683a535bd688 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:09:55.823 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.820+0000 7faec8ff9640 1 -- 172.21.15.42:0/1838913150 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faecc0ccc10 con 0x55feed7ba2f0 2024-08-30T09:09:55.823 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.820+0000 7faec8ff9640 1 -- 172.21.15.42:0/1838913150 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faecc0d0d80 con 0x55feed7ba2f0 2024-08-30T09:09:55.823 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.820+0000 7faec8ff9640 1 -- 172.21.15.42:0/1838913150 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faecc0cc770 con 0x55feed7ba2f0 2024-08-30T09:09:55.824 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.821+0000 7faee008f740 1 -- 172.21.15.42:0/1838913150 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55feed7ba2f0 msgr2=0x55feedb845a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:09:55.824 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.821+0000 7faee008f740 1 --2- 172.21.15.42:0/1838913150 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55feed7ba2f0 0x55feedb845a0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7faecc0c9780 tx=0x7faecc02cec0 comp rx=0 tx=0).stop 2024-08-30T09:09:55.824 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.821+0000 7faee008f740 1 -- 172.21.15.42:0/1838913150 shutdown_connections 2024-08-30T09:09:55.824 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.821+0000 7faee008f740 1 --2- 172.21.15.42:0/1838913150 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55feed7c24d0 0x55feedba23e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:55.824 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.821+0000 7faee008f740 1 --2- 172.21.15.42:0/1838913150 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55feed7e31b0 0x55feedba2920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:55.824 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.821+0000 7faee008f740 1 --2- 172.21.15.42:0/1838913150 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55feed7ba2f0 0x55feedb845a0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:55.824 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.821+0000 7faee008f740 1 -- 172.21.15.42:0/1838913150 >> 172.21.15.42:0/1838913150 conn(0x55feed7d7300 msgr2=0x55feedb88eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:09:55.824 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.822+0000 7faee008f740 1 -- 172.21.15.42:0/1838913150 shutdown_connections 2024-08-30T09:09:55.825 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.822+0000 7faee008f740 1 -- 172.21.15.42:0/1838913150 wait complete. 2024-08-30T09:09:55.825 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.822+0000 7faee008f740 1 -- start start 2024-08-30T09:09:55.825 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.822+0000 7faee008f740 1 Processor -- start 2024-08-30T09:09:55.827 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.822+0000 7faee008f740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55feed7ba2f0 0x55feedbabba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:55.828 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.822+0000 7faee008f740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55feed7c24d0 0x55feedbaa860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:55.828 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.822+0000 7faee008f740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55feed7e31b0 0x55feedbaaeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:55.828 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.823+0000 7faee008f740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7faeb400e130 con 0x55feed7e31b0 2024-08-30T09:09:55.828 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.823+0000 7faee008f740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7faeb400f5a0 con 0x55feed7c24d0 2024-08-30T09:09:55.828 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.823+0000 7faee008f740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7faeb400e800 con 0x55feed7ba2f0 2024-08-30T09:09:55.828 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.823+0000 7faed259c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55feed7c24d0 0x55feedbaa860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:55.828 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.823+0000 7faed2d9d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55feed7ba2f0 0x55feedbabba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:55.828 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.823+0000 7faed259c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55feed7c24d0 0x55feedbaa860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:45308/0 (socket says 172.21.15.42:45308) 2024-08-30T09:09:55.828 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.823+0000 7faed259c640 1 -- 172.21.15.42:0/4043228364 learned_addr learned my addr 172.21.15.42:0/4043228364 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:09:55.828 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.823+0000 7faed2d9d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55feed7ba2f0 0x55feedbabba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:41756/0 (socket says 172.21.15.42:41756) 2024-08-30T09:09:55.828 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.823+0000 7faed359e640 1 --2- 172.21.15.42:0/4043228364 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55feed7e31b0 0x55feedbaaeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:55.828 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.823+0000 7faee008f740 10 client.0 _ll_register_callbacks cb 0x7faecc0cbc80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:09:55.829 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.823+0000 7faed259c640 1 -- 172.21.15.42:0/4043228364 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55feed7ba2f0 msgr2=0x55feedbabba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:09:55.829 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.823+0000 7faed259c640 1 --2- 172.21.15.42:0/4043228364 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55feed7ba2f0 0x55feedbabba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:55.829 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.823+0000 7faed259c640 1 -- 172.21.15.42:0/4043228364 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55feed7e31b0 msgr2=0x55feedbaaeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:09:55.829 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.823+0000 7faed259c640 1 --2- 172.21.15.42:0/4043228364 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55feed7e31b0 0x55feedbaaeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:55.829 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.823+0000 7faed259c640 1 -- 172.21.15.42:0/4043228364 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55feedbab620 con 0x55feed7c24d0 2024-08-30T09:09:55.829 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.823+0000 7faed2d9d640 1 --2- 172.21.15.42:0/4043228364 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55feed7ba2f0 0x55feedbabba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-30T09:09:55.829 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.823+0000 7faed259c640 1 --2- 172.21.15.42:0/4043228364 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55feed7c24d0 0x55feedbaa860 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7faebc004030 tx=0x7faebc00ed40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=82430edaf7fead77 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:09:55.829 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.823+0000 7faeba7fc640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-30T09:09:55.829 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.824+0000 7faeba7fc640 1 -- 172.21.15.42:0/4043228364 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faebc005450 con 0x55feed7c24d0 2024-08-30T09:09:55.829 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.824+0000 7faee008f740 1 -- 172.21.15.42:0/4043228364 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faecc0d0d80 con 0x55feed7c24d0 2024-08-30T09:09:55.829 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.824+0000 7faeba7fc640 1 -- 172.21.15.42:0/4043228364 <== mon.2 v2:172.21.15.167:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7faebc004630 con 0x55feed7c24d0 2024-08-30T09:09:55.829 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.824+0000 7faeba7fc640 1 -- 172.21.15.42:0/4043228364 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7faebc004830 con 0x55feed7c24d0 2024-08-30T09:09:55.829 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.824+0000 7faee008f740 10 client.26762 fetch_fsmap learned FSMap version 39 2024-08-30T09:09:55.830 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.824+0000 7faee008f740 1 -- 172.21.15.42:0/4043228364 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55feedc1c130 con 0x55feed7c24d0 2024-08-30T09:09:55.830 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.824+0000 7faeba7fc640 1 -- 172.21.15.42:0/4043228364 <== mon.2 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faecc0d0d80 con 0x55feed7c24d0 2024-08-30T09:09:55.830 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.825+0000 7faeba7fc640 1 -- 172.21.15.42:0/4043228364 <== mon.2 v2:172.21.15.167:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7faebc040030 con 0x55feed7c24d0 2024-08-30T09:09:55.830 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.825+0000 7faee008f740 10 client.26762 fetch_fsmap finished waiting for FSMap version 39 2024-08-30T09:09:55.830 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.825+0000 7faee008f740 10 client.26762 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-30T09:09:55.830 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.825+0000 7faee008f740 20 client.26762 populate_metadata read hostname 'smithi042' 2024-08-30T09:09:55.830 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.825+0000 7faee008f740 1 --2- 172.21.15.42:0/4043228364 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55feed7771c0 0x55feedc1ea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:55.830 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.825+0000 7faee008f740 4 client.26762 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-30T09:09:55.830 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.825+0000 7faee008f740 1 -- 172.21.15.42:0/4043228364 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55feedafdf50 con 0x55feed7771c0 2024-08-30T09:09:55.830 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.825+0000 7faed2d9d640 1 --2- 172.21.15.42:0/4043228364 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55feed7771c0 0x55feedc1ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:55.830 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.825+0000 7faeba7fc640 10 client.26762 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:09:55.830 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.825+0000 7faed2d9d640 1 --2- 172.21.15.42:0/4043228364 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55feed7771c0 0x55feedc1ea60 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=465a12cbeda921bb server_cookie=2ee0766ce97cb5e4 in_seq=0 out_seq=0 2024-08-30T09:09:55.841 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.838+0000 7faeba7fc640 1 -- 172.21.15.42:0/4043228364 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55feedafdf50 con 0x55feed7771c0 2024-08-30T09:09:55.842 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.839+0000 7faeba7fc640 10 client.26762 handle_command_reply: tid=0 2024-08-30T09:09:55.842 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.839+0000 7faee008f740 1 client.26762 shutdown 2024-08-30T09:09:55.842 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.839+0000 7faee008f740 1 -- 172.21.15.42:0/4043228364 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55feed7c24d0 msgr2=0x55feedbaa860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:09:55.842 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.839+0000 7faee008f740 1 --2- 172.21.15.42:0/4043228364 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55feed7c24d0 0x55feedbaa860 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7faebc004030 tx=0x7faebc00ed40 comp rx=0 tx=0).stop 2024-08-30T09:09:55.842 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.839+0000 7faee008f740 1 -- 172.21.15.42:0/4043228364 shutdown_connections 2024-08-30T09:09:55.842 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.839+0000 7faee008f740 1 --2- 172.21.15.42:0/4043228364 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55feed7771c0 0x55feedc1ea60 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:55.842 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.839+0000 7faeba7fc640 0 client.26762 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:09:55.842 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.839+0000 7faee008f740 1 --2- 172.21.15.42:0/4043228364 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55feed7e31b0 0x55feedbaaeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:55.842 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.839+0000 7faee008f740 1 --2- 172.21.15.42:0/4043228364 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55feed7c24d0 0x55feedbaa860 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:55.842 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.839+0000 7faee008f740 1 --2- 172.21.15.42:0/4043228364 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55feed7ba2f0 0x55feedbabba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:55.842 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.839+0000 7faee008f740 1 -- 172.21.15.42:0/4043228364 >> 172.21.15.42:0/4043228364 conn(0x55feed7d7300 msgr2=0x55feedb8dbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:09:55.842 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.839+0000 7faee008f740 1 -- 172.21.15.42:0/4043228364 shutdown_connections 2024-08-30T09:09:55.842 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.839+0000 7faee008f740 1 -- 172.21.15.42:0/4043228364 wait complete. 2024-08-30T09:09:55.843 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.839+0000 7faee008f740 20 client.26762 trim_cache size 0 max 16384 2024-08-30T09:09:55.843 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-08-30T09:09:55.843 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2024-08-30T09:09:55.843 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-08-30T09:09:55.843 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-08-30T09:09:55.844 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.840+0000 7faed9044640 1 -- 172.21.15.42:0/2412669124 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7faec00b4c70 msgr2=0x7faec00b7180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:09:55.844 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.840+0000 7faed9044640 1 --2- 172.21.15.42:0/2412669124 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7faec00b4c70 0x7faec00b7180 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7faebc00a530 tx=0x7faebc00a4c0 comp rx=0 tx=0).stop 2024-08-30T09:09:55.844 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.840+0000 7faed9044640 1 -- 172.21.15.42:0/2412669124 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faed4101af0 msgr2=0x7faed4197b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:09:55.844 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.840+0000 7faed9044640 1 --2- 172.21.15.42:0/2412669124 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faed4101af0 0x7faed4197b60 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7faecc009570 tx=0x7faecc004270 comp rx=0 tx=0).stop 2024-08-30T09:09:55.844 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.840+0000 7faed9044640 1 -- 172.21.15.42:0/2412669124 shutdown_connections 2024-08-30T09:09:55.844 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.840+0000 7faed9044640 1 --2- 172.21.15.42:0/2412669124 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7faec00b4c70 0x7faec00b7180 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:55.844 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.840+0000 7faed9044640 1 --2- 172.21.15.42:0/2412669124 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faed4103510 0x7faed4198d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:55.844 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.840+0000 7faed9044640 1 --2- 172.21.15.42:0/2412669124 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faed4102b70 0x7faed41981d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:55.844 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.840+0000 7faed9044640 1 --2- 172.21.15.42:0/2412669124 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faed4101af0 0x7faed4197b60 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:55.845 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.840+0000 7faed9044640 1 -- 172.21.15.42:0/2412669124 >> 172.21.15.42:0/2412669124 conn(0x7faed40fd430 msgr2=0x7faed4105e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:09:55.845 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.840+0000 7faed9044640 1 -- 172.21.15.42:0/2412669124 shutdown_connections 2024-08-30T09:09:55.845 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:09:55.840+0000 7faed9044640 1 -- 172.21.15.42:0/2412669124 wait complete. 2024-08-30T09:09:55.953 INFO:teuthology.orchestra.run.smithi042.stderr:progress: 550.0 s, 433.6 tps, lat 226.852 ms stddev 285.725 2024-08-30T09:09:56.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:09:55 smithi042 ceph-mon[18652]: from='client.? 172.21.15.42:0/4089024704' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-30T09:09:56.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:09:55 smithi190 ceph-mon[22001]: from='client.? 172.21.15.42:0/4089024704' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-30T09:09:56.292 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-08-30T09:09:56.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:09:55 smithi167 ceph-mon[22063]: from='client.? 172.21.15.42:0/4089024704' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-30T09:09:56.775 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T09:09:57.108 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:09:56 smithi042 ceph-mon[18652]: pgmap v648: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 3.8 MiB/s wr, 452 op/s 2024-08-30T09:09:57.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:09:56 smithi190 ceph-mon[22001]: pgmap v648: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 3.8 MiB/s wr, 452 op/s 2024-08-30T09:09:57.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:09:56 smithi167 ceph-mon[22063]: pgmap v648: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 3.8 MiB/s wr, 452 op/s 2024-08-30T09:09:57.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.471+0000 7f48e9b91640 1 -- 172.21.15.42:0/966241872 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f48e410a780 msgr2=0x7f48e410ab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:09:57.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.471+0000 7f48e9b91640 1 --2- 172.21.15.42:0/966241872 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f48e410a780 0x7f48e410ab90 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f48dc01acb0 tx=0x7f48dc03b900 comp rx=0 tx=0).stop 2024-08-30T09:09:57.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.471+0000 7f48e9b91640 1 -- 172.21.15.42:0/966241872 shutdown_connections 2024-08-30T09:09:57.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.471+0000 7f48e9b91640 1 --2- 172.21.15.42:0/966241872 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f48e411c9d0 0x7f48e411eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:57.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.471+0000 7f48e9b91640 1 --2- 172.21.15.42:0/966241872 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f48e410a780 0x7f48e410ab90 secure :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f48dc01acb0 tx=0x7f48dc03b900 comp rx=0 tx=0).stop 2024-08-30T09:09:57.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.471+0000 7f48e9b91640 1 --2- 172.21.15.42:0/966241872 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f48e4111ce0 0x7f48e410a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:57.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.471+0000 7f48e9b91640 1 -- 172.21.15.42:0/966241872 >> 172.21.15.42:0/966241872 conn(0x7f48e4070c60 msgr2=0x7f48e4071070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:09:57.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.473+0000 7f48e9b91640 1 -- 172.21.15.42:0/966241872 shutdown_connections 2024-08-30T09:09:57.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.477+0000 7f48e9b91640 1 -- 172.21.15.42:0/966241872 wait complete. 2024-08-30T09:09:57.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.478+0000 7f48e9b91640 1 Processor -- start 2024-08-30T09:09:57.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.478+0000 7f48e9b91640 1 -- start start 2024-08-30T09:09:57.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.479+0000 7f48e9b91640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f48e4111ce0 0x7f48e411c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:57.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.479+0000 7f48e9b91640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f48e411c9d0 0x7f48e41185d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:57.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.479+0000 7f48e9b91640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f48e4118c50 0x7f48e41acb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:57.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.479+0000 7f48e9b91640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f48e4114a10 con 0x7f48e411c9d0 2024-08-30T09:09:57.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.479+0000 7f48e9b91640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f48e4114890 con 0x7f48e4111ce0 2024-08-30T09:09:57.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.479+0000 7f48e9b91640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f48e4114b90 con 0x7f48e4118c50 2024-08-30T09:09:57.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.482+0000 7f48e3fff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f48e4118c50 0x7f48e41acb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:57.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.483+0000 7f48e3fff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f48e4118c50 0x7f48e41acb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:41772/0 (socket says 172.21.15.42:41772) 2024-08-30T09:09:57.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.483+0000 7f48e3fff640 1 -- 172.21.15.42:0/3600926840 learned_addr learned my addr 172.21.15.42:0/3600926840 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:09:57.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.483+0000 7f48e3fff640 1 -- 172.21.15.42:0/3600926840 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f48e4111ce0 msgr2=0x7f48e411c3b0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-08-30T09:09:57.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.483+0000 7f48e3fff640 1 --2- 172.21.15.42:0/3600926840 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f48e4111ce0 0x7f48e411c3b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:57.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.483+0000 7f48e3fff640 1 -- 172.21.15.42:0/3600926840 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f48e411c9d0 msgr2=0x7f48e41185d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:09:57.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.483+0000 7f48e3fff640 1 --2- 172.21.15.42:0/3600926840 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f48e411c9d0 0x7f48e41185d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:57.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.484+0000 7f48e3fff640 1 -- 172.21.15.42:0/3600926840 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48e41ad0b0 con 0x7f48e4118c50 2024-08-30T09:09:57.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.484+0000 7f48e3fff640 1 --2- 172.21.15.42:0/3600926840 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f48e4118c50 0x7f48e41acb70 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f48d400d750 tx=0x7f48d400dc10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:09:57.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.484+0000 7f48e0ff9640 1 -- 172.21.15.42:0/3600926840 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48d4017070 con 0x7f48e4118c50 2024-08-30T09:09:57.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.484+0000 7f48e0ff9640 1 -- 172.21.15.42:0/3600926840 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f48d40054d0 con 0x7f48e4118c50 2024-08-30T09:09:57.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.484+0000 7f48e9b91640 1 -- 172.21.15.42:0/3600926840 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48e41ad280 con 0x7f48e4118c50 2024-08-30T09:09:57.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.485+0000 7f48e9b91640 1 -- 172.21.15.42:0/3600926840 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48e41ad730 con 0x7f48e4118c50 2024-08-30T09:09:57.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.488+0000 7f48f0106740 1 Processor -- start 2024-08-30T09:09:57.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.488+0000 7f48f0106740 1 -- start start 2024-08-30T09:09:57.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.489+0000 7f48f0106740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c153f0f070 0x55c1542b4310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:57.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.489+0000 7f48f0106740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c153ee61b0 0x55c1542b4850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:57.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.489+0000 7f48f0106740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c153eee390 0x55c1542b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:57.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.489+0000 7f48f0106740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55c15422b680 con 0x55c153f0f070 2024-08-30T09:09:57.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.489+0000 7f48f0106740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55c15421ac10 con 0x55c153eee390 2024-08-30T09:09:57.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.489+0000 7f48f0106740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55c15421ebc0 con 0x55c153ee61b0 2024-08-30T09:09:57.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.489+0000 7f48e0ff9640 1 -- 172.21.15.42:0/3600926840 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48d4004c60 con 0x7f48e4118c50 2024-08-30T09:09:57.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.490+0000 7f48e37fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c153ee61b0 0x55c1542b4850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:57.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.490+0000 7f48e37fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c153ee61b0 0x55c1542b4850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:41782/0 (socket says 172.21.15.42:41782) 2024-08-30T09:09:57.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.490+0000 7f48e37fe640 1 -- 172.21.15.42:0/3934662892 learned_addr learned my addr 172.21.15.42:0/3934662892 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:09:57.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.492+0000 7f48e37fe640 1 -- 172.21.15.42:0/3934662892 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c153eee390 msgr2=0x55c1542b4d90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-30T09:09:57.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.492+0000 7f48e37fe640 1 --2- 172.21.15.42:0/3934662892 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c153eee390 0x55c1542b4d90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:57.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.492+0000 7f48e37fe640 1 -- 172.21.15.42:0/3934662892 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c153f0f070 msgr2=0x55c1542b4310 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-30T09:09:57.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.492+0000 7f48e37fe640 1 --2- 172.21.15.42:0/3934662892 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c153f0f070 0x55c1542b4310 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:57.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.492+0000 7f48e37fe640 1 -- 172.21.15.42:0/3934662892 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c153f7a650 con 0x55c153ee61b0 2024-08-30T09:09:57.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.493+0000 7f48e37fe640 1 --2- 172.21.15.42:0/3934662892 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c153ee61b0 0x55c1542b4850 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f48d8009740 tx=0x7f48d8009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=808e5646ff4d94ef server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:09:57.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.497+0000 7f48c9ffb640 1 -- 172.21.15.42:0/3934662892 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48d8015020 con 0x55c153ee61b0 2024-08-30T09:09:57.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.497+0000 7f48c9ffb640 1 -- 172.21.15.42:0/3934662892 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f48d8009e50 con 0x55c153ee61b0 2024-08-30T09:09:57.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.498+0000 7f48f0106740 1 -- 172.21.15.42:0/3934662892 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c153ee61b0 msgr2=0x55c1542b4850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:09:57.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.498+0000 7f48f0106740 1 --2- 172.21.15.42:0/3934662892 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c153ee61b0 0x55c1542b4850 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f48d8009740 tx=0x7f48d8009c00 comp rx=0 tx=0).stop 2024-08-30T09:09:57.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.506+0000 7f48f0106740 1 -- 172.21.15.42:0/3934662892 shutdown_connections 2024-08-30T09:09:57.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.509+0000 7f48e0ff9640 1 -- 172.21.15.42:0/3600926840 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7f48d4009bd0 con 0x7f48e4118c50 2024-08-30T09:09:57.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.509+0000 7f48f0106740 1 --2- 172.21.15.42:0/3934662892 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c153ee61b0 0x55c1542b4850 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:57.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.509+0000 7f48f0106740 1 --2- 172.21.15.42:0/3934662892 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c153eee390 0x55c1542b4d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:57.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.509+0000 7f48f0106740 1 --2- 172.21.15.42:0/3934662892 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c153f0f070 0x55c1542b4310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:57.519 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.509+0000 7f48f0106740 1 -- 172.21.15.42:0/3934662892 >> 172.21.15.42:0/3934662892 conn(0x55c153f03460 msgr2=0x55c1542b2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:09:57.519 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.510+0000 7f48e0ff9640 1 --2- 172.21.15.42:0/3600926840 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f48cc0b4c00 0x7f48cc0b7110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:57.519 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.510+0000 7f48e0ff9640 1 -- 172.21.15.42:0/3600926840 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f48d40e4d00 con 0x7f48e4118c50 2024-08-30T09:09:57.519 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.511+0000 7f48f0106740 1 -- 172.21.15.42:0/3934662892 shutdown_connections 2024-08-30T09:09:57.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.513+0000 7f48f0106740 1 -- 172.21.15.42:0/3934662892 wait complete. 2024-08-30T09:09:57.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.514+0000 7f48e37fe640 1 --2- 172.21.15.42:0/3600926840 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f48cc0b4c00 0x7f48cc0b7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:57.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.514+0000 7f48f0106740 1 -- start start 2024-08-30T09:09:57.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.514+0000 7f48f0106740 1 Processor -- start 2024-08-30T09:09:57.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.515+0000 7f48e37fe640 1 --2- 172.21.15.42:0/3600926840 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f48cc0b4c00 0x7f48cc0b7110 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f48d8011900 tx=0x7f48d8011a00 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:09:57.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.515+0000 7f48f0106740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c153ee61b0 0x55c1542bf9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:57.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.515+0000 7f48f0106740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c153eee390 0x55c1542bff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:57.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.515+0000 7f48f0106740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c153f0f070 0x55c1542cbf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:57.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.515+0000 7f48f0106740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55c15407d700 con 0x55c153ee61b0 2024-08-30T09:09:57.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.516+0000 7f48f0106740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55c1541a1750 con 0x55c153f0f070 2024-08-30T09:09:57.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.516+0000 7f48f0106740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55c1542c2b40 con 0x55c153eee390 2024-08-30T09:09:57.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.516+0000 7f48e2ffd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c153eee390 0x55c1542bff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:57.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.516+0000 7f48e2ffd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c153eee390 0x55c1542bff10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:41790/0 (socket says 172.21.15.42:41790) 2024-08-30T09:09:57.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.516+0000 7f48e2ffd640 1 -- 172.21.15.42:0/319580084 learned_addr learned my addr 172.21.15.42:0/319580084 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:09:57.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.516+0000 7f48f0106740 10 client.0 _ll_register_callbacks cb 0x55c1541d0e10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:09:57.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.516+0000 7f48e3fff640 1 --2- 172.21.15.42:0/319580084 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c153f0f070 0x55c1542cbf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:57.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.516+0000 7f48e2ffd640 1 -- 172.21.15.42:0/319580084 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c153f0f070 msgr2=0x55c1542cbf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:09:57.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.516+0000 7f48e2ffd640 1 --2- 172.21.15.42:0/319580084 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c153f0f070 0x55c1542cbf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:57.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.516+0000 7f48e2ffd640 1 -- 172.21.15.42:0/319580084 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c153ee61b0 msgr2=0x55c1542bf9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:09:57.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.516+0000 7f48e2ffd640 1 --2- 172.21.15.42:0/319580084 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c153ee61b0 0x55c1542bf9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:57.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.516+0000 7f48e2ffd640 1 -- 172.21.15.42:0/319580084 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c1541da900 con 0x55c153eee390 2024-08-30T09:09:57.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.516+0000 7f48e3fff640 1 --2- 172.21.15.42:0/319580084 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c153f0f070 0x55c1542cbf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-30T09:09:57.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.517+0000 7f48e2ffd640 1 --2- 172.21.15.42:0/319580084 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c153eee390 0x55c1542bff10 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f48dc03be70 tx=0x7f48dc01a160 comp rx=0 tx=0).ready entity=mon.1 client_cookie=75f370d62b1e22bd server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:09:57.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.518+0000 7f48c2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-08-30T09:09:57.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.518+0000 7f48c2ffd640 1 -- 172.21.15.42:0/319580084 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48dc045040 con 0x55c153eee390 2024-08-30T09:09:57.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.518+0000 7f48c2ffd640 1 -- 172.21.15.42:0/319580084 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f48dc040800 con 0x55c153eee390 2024-08-30T09:09:57.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.518+0000 7f48f0106740 1 -- 172.21.15.42:0/319580084 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f48d8009e50 con 0x55c153eee390 2024-08-30T09:09:57.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.520+0000 7f48c2ffd640 1 -- 172.21.15.42:0/319580084 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f48dc040a00 con 0x55c153eee390 2024-08-30T09:09:57.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.521+0000 7f48f0106740 10 client.26749 fetch_fsmap learned FSMap version 39 2024-08-30T09:09:57.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.521+0000 7f48f0106740 1 -- 172.21.15.42:0/319580084 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55c1542ce8b0 con 0x55c153eee390 2024-08-30T09:09:57.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.522+0000 7f48c2ffd640 1 -- 172.21.15.42:0/319580084 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f48dc041600 con 0x55c153eee390 2024-08-30T09:09:57.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.522+0000 7f48c2ffd640 1 -- 172.21.15.42:0/319580084 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7f48dc004040 con 0x55c153eee390 2024-08-30T09:09:57.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.522+0000 7f48f0106740 10 client.26749 fetch_fsmap finished waiting for FSMap version 39 2024-08-30T09:09:57.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.522+0000 7f48f0106740 10 client.26749 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-08-30T09:09:57.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.522+0000 7f48f0106740 20 client.26749 populate_metadata read hostname 'smithi042' 2024-08-30T09:09:57.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.522+0000 7f48f0106740 1 --2- 172.21.15.42:0/319580084 >> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] conn(0x55c153ecd9f0 0x55c1542d1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:57.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.522+0000 7f48f0106740 4 client.26749 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-30T09:09:57.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.522+0000 7f48f0106740 1 -- 172.21.15.42:0/319580084 --> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f48d8009e50 con 0x55c153ecd9f0 2024-08-30T09:09:57.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.523+0000 7f48e37fe640 1 --2- 172.21.15.42:0/319580084 >> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] conn(0x55c153ecd9f0 0x55c1542d1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:57.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.524+0000 7f48c2ffd640 10 client.26749 ms_handle_connect on v2:172.21.15.167:6836/3011837856 2024-08-30T09:09:57.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.524+0000 7f48e37fe640 1 --2- 172.21.15.42:0/319580084 >> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] conn(0x55c153ecd9f0 0x55c1542d1f00 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9b05ecf2d7f941cd server_cookie=b559993ec554506c in_seq=0 out_seq=0 2024-08-30T09:09:57.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.525+0000 7f48c2ffd640 1 -- 172.21.15.42:0/319580084 <== mds.2 v2:172.21.15.167:6836/3011837856 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f48d8009e50 con 0x55c153ecd9f0 2024-08-30T09:09:57.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.525+0000 7f48c2ffd640 10 client.26749 handle_command_reply: tid=0 2024-08-30T09:09:57.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.525+0000 7f48f0106740 1 client.26749 shutdown 2024-08-30T09:09:57.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.525+0000 7f48f0106740 1 -- 172.21.15.42:0/319580084 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c153eee390 msgr2=0x55c1542bff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:09:57.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.525+0000 7f48f0106740 1 --2- 172.21.15.42:0/319580084 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c153eee390 0x55c1542bff10 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f48dc03be70 tx=0x7f48dc01a160 comp rx=0 tx=0).stop 2024-08-30T09:09:57.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.525+0000 7f48f0106740 1 -- 172.21.15.42:0/319580084 shutdown_connections 2024-08-30T09:09:57.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.525+0000 7f48f0106740 1 --2- 172.21.15.42:0/319580084 >> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] conn(0x55c153ecd9f0 0x55c1542d1f00 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:57.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.526+0000 7f48f0106740 1 --2- 172.21.15.42:0/319580084 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c153eee390 0x55c1542bff10 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:57.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.526+0000 7f48f0106740 1 --2- 172.21.15.42:0/319580084 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c153f0f070 0x55c1542cbf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:57.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.526+0000 7f48f0106740 1 --2- 172.21.15.42:0/319580084 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c153ee61b0 0x55c1542bf9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:57.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.526+0000 7f48c2ffd640 0 client.26749 ms_handle_reset on v2:172.21.15.167:6836/3011837856 2024-08-30T09:09:57.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.526+0000 7f48f0106740 1 -- 172.21.15.42:0/319580084 >> 172.21.15.42:0/319580084 conn(0x55c153f03460 msgr2=0x55c1542b74f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:09:57.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.526+0000 7f48f0106740 1 -- 172.21.15.42:0/319580084 shutdown_connections 2024-08-30T09:09:57.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.526+0000 7f48f0106740 1 -- 172.21.15.42:0/319580084 wait complete. 2024-08-30T09:09:57.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.526+0000 7f48f0106740 20 client.26749 trim_cache size 0 max 16384 2024-08-30T09:09:57.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.548+0000 7f48f0106740 1 Processor -- start 2024-08-30T09:09:57.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.548+0000 7f48f0106740 1 -- start start 2024-08-30T09:09:57.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.549+0000 7f48f0106740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c153ecd9f0 0x55c1542be900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:57.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.549+0000 7f48f0106740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c153ee61b0 0x55c1542b5a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:57.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.549+0000 7f48f0106740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c153eee390 0x55c1542b17d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:57.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.549+0000 7f48f0106740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f48d00125f0 con 0x55c153eee390 2024-08-30T09:09:57.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.549+0000 7f48f0106740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f48d000fe60 con 0x55c153ee61b0 2024-08-30T09:09:57.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.549+0000 7f48f0106740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f48d000dcb0 con 0x55c153ecd9f0 2024-08-30T09:09:57.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.549+0000 7f48e37fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c153ecd9f0 0x55c1542be900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:57.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.549+0000 7f48e37fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c153ecd9f0 0x55c1542be900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:41792/0 (socket says 172.21.15.42:41792) 2024-08-30T09:09:57.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.549+0000 7f48e37fe640 1 -- 172.21.15.42:0/3874713023 learned_addr learned my addr 172.21.15.42:0/3874713023 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:09:57.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.549+0000 7f48e3fff640 1 --2- 172.21.15.42:0/3874713023 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c153eee390 0x55c1542b17d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:57.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.549+0000 7f48e2ffd640 1 --2- 172.21.15.42:0/3874713023 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c153ee61b0 0x55c1542b5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:57.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.549+0000 7f48e3fff640 1 -- 172.21.15.42:0/3874713023 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c153ecd9f0 msgr2=0x55c1542be900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:09:57.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.549+0000 7f48e3fff640 1 --2- 172.21.15.42:0/3874713023 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c153ecd9f0 0x55c1542be900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:57.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.549+0000 7f48e3fff640 1 -- 172.21.15.42:0/3874713023 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c153ee61b0 msgr2=0x55c1542b5a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:09:57.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.549+0000 7f48e3fff640 1 --2- 172.21.15.42:0/3874713023 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c153ee61b0 0x55c1542b5a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:57.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.549+0000 7f48e3fff640 1 -- 172.21.15.42:0/3874713023 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c1542bdf10 con 0x55c153eee390 2024-08-30T09:09:57.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.549+0000 7f48e37fe640 1 --2- 172.21.15.42:0/3874713023 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c153ecd9f0 0x55c1542be900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-30T09:09:57.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.550+0000 7f48e3fff640 1 --2- 172.21.15.42:0/3874713023 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c153eee390 0x55c1542b17d0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f48d40e2410 tx=0x7f48d40aba40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70198df1fb07992f server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:09:57.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.550+0000 7f48c97fa640 1 -- 172.21.15.42:0/3874713023 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48d4020040 con 0x55c153eee390 2024-08-30T09:09:57.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.550+0000 7f48c97fa640 1 -- 172.21.15.42:0/3874713023 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f48d40b4660 con 0x55c153eee390 2024-08-30T09:09:57.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.550+0000 7f48c97fa640 1 -- 172.21.15.42:0/3874713023 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48d4092af0 con 0x55c153eee390 2024-08-30T09:09:57.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.551+0000 7f48f0106740 1 -- 172.21.15.42:0/3874713023 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c153eee390 msgr2=0x55c1542b17d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:09:57.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.551+0000 7f48f0106740 1 --2- 172.21.15.42:0/3874713023 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c153eee390 0x55c1542b17d0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f48d40e2410 tx=0x7f48d40aba40 comp rx=0 tx=0).stop 2024-08-30T09:09:57.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.552+0000 7f48f0106740 1 -- 172.21.15.42:0/3874713023 shutdown_connections 2024-08-30T09:09:57.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.552+0000 7f48f0106740 1 --2- 172.21.15.42:0/3874713023 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c153eee390 0x55c1542b17d0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:57.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.552+0000 7f48f0106740 1 --2- 172.21.15.42:0/3874713023 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c153ee61b0 0x55c1542b5a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:57.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.552+0000 7f48f0106740 1 --2- 172.21.15.42:0/3874713023 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c153ecd9f0 0x55c1542be900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:57.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.552+0000 7f48f0106740 1 -- 172.21.15.42:0/3874713023 >> 172.21.15.42:0/3874713023 conn(0x55c153f03460 msgr2=0x55c1542b5340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:09:57.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.552+0000 7f48f0106740 1 -- 172.21.15.42:0/3874713023 shutdown_connections 2024-08-30T09:09:57.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.552+0000 7f48f0106740 1 -- 172.21.15.42:0/3874713023 wait complete. 2024-08-30T09:09:57.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.552+0000 7f48f0106740 1 -- start start 2024-08-30T09:09:57.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.553+0000 7f48f0106740 1 Processor -- start 2024-08-30T09:09:57.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.553+0000 7f48f0106740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c153ecd9f0 0x55c1542c4620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:57.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.553+0000 7f48f0106740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c153ee61b0 0x55c1542c4b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:57.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.553+0000 7f48e37fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c153ecd9f0 0x55c1542c4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:57.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.553+0000 7f48f0106740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c153eee390 0x55c15433b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:57.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.553+0000 7f48e37fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c153ecd9f0 0x55c1542c4620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:41808/0 (socket says 172.21.15.42:41808) 2024-08-30T09:09:57.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.553+0000 7f48e37fe640 1 -- 172.21.15.42:0/1815799752 learned_addr learned my addr 172.21.15.42:0/1815799752 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:09:57.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.553+0000 7f48e2ffd640 1 --2- 172.21.15.42:0/1815799752 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c153ee61b0 0x55c1542c4b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:57.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.553+0000 7f48f0106740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f48d000e320 con 0x55c153ee61b0 2024-08-30T09:09:57.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.553+0000 7f48f0106740 1 -- 172.21.15.42:0/1815799752 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f48d000f790 con 0x55c153eee390 2024-08-30T09:09:57.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.553+0000 7f48f0106740 1 -- 172.21.15.42:0/1815799752 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f48d000e9f0 con 0x55c153ecd9f0 2024-08-30T09:09:57.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.553+0000 7f48f0106740 10 client.0 _ll_register_callbacks cb 0x7f48dc01ab30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:09:57.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.554+0000 7f48e3fff640 1 --2- 172.21.15.42:0/1815799752 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c153eee390 0x55c15433b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:57.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.554+0000 7f48e2ffd640 1 -- 172.21.15.42:0/1815799752 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c153ecd9f0 msgr2=0x55c1542c4620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:09:57.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.554+0000 7f48e2ffd640 1 --2- 172.21.15.42:0/1815799752 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c153ecd9f0 0x55c1542c4620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:57.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.554+0000 7f48e2ffd640 1 -- 172.21.15.42:0/1815799752 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c153eee390 msgr2=0x55c15433b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:09:57.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.554+0000 7f48e2ffd640 1 --2- 172.21.15.42:0/1815799752 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c153eee390 0x55c15433b8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:57.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.554+0000 7f48e2ffd640 1 -- 172.21.15.42:0/1815799752 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c15433beb0 con 0x55c153ee61b0 2024-08-30T09:09:57.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.554+0000 7f48e3fff640 1 --2- 172.21.15.42:0/1815799752 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c153eee390 0x55c15433b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-30T09:09:57.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.554+0000 7f48e37fe640 1 --2- 172.21.15.42:0/1815799752 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c153ecd9f0 0x55c1542c4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-30T09:09:57.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.554+0000 7f48e2ffd640 1 --2- 172.21.15.42:0/1815799752 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c153ee61b0 0x55c1542c4b60 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f48dc01ac00 tx=0x7f48dc03bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc6ddc7a2f973962 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:09:57.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.554+0000 7f48c2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-08-30T09:09:57.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.554+0000 7f48c2ffd640 1 -- 172.21.15.42:0/1815799752 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48dc068020 con 0x55c153ee61b0 2024-08-30T09:09:57.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.554+0000 7f48c2ffd640 1 -- 172.21.15.42:0/1815799752 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f48dc040800 con 0x55c153ee61b0 2024-08-30T09:09:57.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.554+0000 7f48f0106740 1 -- 172.21.15.42:0/1815799752 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f48d40b4660 con 0x55c153ee61b0 2024-08-30T09:09:57.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.555+0000 7f48c2ffd640 1 -- 172.21.15.42:0/1815799752 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f48dc045070 con 0x55c153ee61b0 2024-08-30T09:09:57.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.555+0000 7f48f0106740 10 client.16956 fetch_fsmap learned FSMap version 39 2024-08-30T09:09:57.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.555+0000 7f48f0106740 1 -- 172.21.15.42:0/1815799752 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55c15433d0d0 con 0x55c153ee61b0 2024-08-30T09:09:57.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.555+0000 7f48c2ffd640 1 -- 172.21.15.42:0/1815799752 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f48dc004210 con 0x55c153ee61b0 2024-08-30T09:09:57.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.555+0000 7f48c2ffd640 1 -- 172.21.15.42:0/1815799752 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7f48dc056480 con 0x55c153ee61b0 2024-08-30T09:09:57.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.555+0000 7f48f0106740 10 client.16956 fetch_fsmap finished waiting for FSMap version 39 2024-08-30T09:09:57.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.555+0000 7f48f0106740 10 client.16956 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-08-30T09:09:57.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.555+0000 7f48f0106740 20 client.16956 populate_metadata read hostname 'smithi042' 2024-08-30T09:09:57.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.555+0000 7f48f0106740 1 --2- 172.21.15.42:0/1815799752 >> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] conn(0x55c153f0f070 0x55c15433f970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:57.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.555+0000 7f48f0106740 4 client.16956 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-30T09:09:57.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.555+0000 7f48f0106740 1 -- 172.21.15.42:0/1815799752 --> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f48d40b4660 con 0x55c153f0f070 2024-08-30T09:09:57.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.556+0000 7f48e37fe640 1 --2- 172.21.15.42:0/1815799752 >> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] conn(0x55c153f0f070 0x55c15433f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:57.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.556+0000 7f48c2ffd640 10 client.16956 ms_handle_connect on v2:172.21.15.167:6836/3011837856 2024-08-30T09:09:57.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.556+0000 7f48e37fe640 1 --2- 172.21.15.42:0/1815799752 >> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] conn(0x55c153f0f070 0x55c15433f970 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d759b0dfa5732e23 server_cookie=e3d85bb4137b59e1 in_seq=0 out_seq=0 2024-08-30T09:09:57.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.556+0000 7f48c2ffd640 1 -- 172.21.15.42:0/1815799752 <== mds.2 v2:172.21.15.167:6836/3011837856 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f48d40b4660 con 0x55c153f0f070 2024-08-30T09:09:57.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.556+0000 7f48c2ffd640 10 client.16956 handle_command_reply: tid=0 2024-08-30T09:09:57.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.556+0000 7f48f0106740 1 client.16956 shutdown 2024-08-30T09:09:57.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.557+0000 7f48f0106740 1 -- 172.21.15.42:0/1815799752 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c153ee61b0 msgr2=0x55c1542c4b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:09:57.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.557+0000 7f48f0106740 1 --2- 172.21.15.42:0/1815799752 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c153ee61b0 0x55c1542c4b60 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f48dc01ac00 tx=0x7f48dc03bf80 comp rx=0 tx=0).stop 2024-08-30T09:09:57.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.557+0000 7f48f0106740 1 -- 172.21.15.42:0/1815799752 shutdown_connections 2024-08-30T09:09:57.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.557+0000 7f48f0106740 1 --2- 172.21.15.42:0/1815799752 >> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] conn(0x55c153f0f070 0x55c15433f970 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:57.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.557+0000 7f48c2ffd640 0 client.16956 ms_handle_reset on v2:172.21.15.167:6836/3011837856 2024-08-30T09:09:57.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.557+0000 7f48f0106740 1 --2- 172.21.15.42:0/1815799752 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c153eee390 0x55c15433b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:57.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.557+0000 7f48f0106740 1 --2- 172.21.15.42:0/1815799752 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c153ee61b0 0x55c1542c4b60 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:57.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.557+0000 7f48f0106740 1 --2- 172.21.15.42:0/1815799752 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c153ecd9f0 0x55c1542c4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:57.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.557+0000 7f48f0106740 1 -- 172.21.15.42:0/1815799752 >> 172.21.15.42:0/1815799752 conn(0x55c153f03460 msgr2=0x55c1542c0d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:09:57.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.557+0000 7f48f0106740 1 -- 172.21.15.42:0/1815799752 shutdown_connections 2024-08-30T09:09:57.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.557+0000 7f48f0106740 1 -- 172.21.15.42:0/1815799752 wait complete. 2024-08-30T09:09:57.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.557+0000 7f48f0106740 20 client.16956 trim_cache size 0 max 16384 2024-08-30T09:09:57.566 INFO:teuthology.orchestra.run.smithi042.stdout:[] 2024-08-30T09:09:57.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.558+0000 7f48e9b91640 1 -- 172.21.15.42:0/3600926840 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f48cc0b4c00 msgr2=0x7f48cc0b7110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:09:57.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.558+0000 7f48e9b91640 1 --2- 172.21.15.42:0/3600926840 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f48cc0b4c00 0x7f48cc0b7110 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f48d8011900 tx=0x7f48d8011a00 comp rx=0 tx=0).stop 2024-08-30T09:09:57.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.558+0000 7f48e9b91640 1 -- 172.21.15.42:0/3600926840 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f48e4118c50 msgr2=0x7f48e41acb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:09:57.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.558+0000 7f48e9b91640 1 --2- 172.21.15.42:0/3600926840 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f48e4118c50 0x7f48e41acb70 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f48d400d750 tx=0x7f48d400dc10 comp rx=0 tx=0).stop 2024-08-30T09:09:57.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.558+0000 7f48e9b91640 1 -- 172.21.15.42:0/3600926840 shutdown_connections 2024-08-30T09:09:57.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.558+0000 7f48e9b91640 1 --2- 172.21.15.42:0/3600926840 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f48cc0b4c00 0x7f48cc0b7110 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:57.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.558+0000 7f48e9b91640 1 --2- 172.21.15.42:0/3600926840 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f48e4118c50 0x7f48e41acb70 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:57.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.558+0000 7f48e9b91640 1 --2- 172.21.15.42:0/3600926840 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f48e411c9d0 0x7f48e41185d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:57.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.558+0000 7f48e9b91640 1 --2- 172.21.15.42:0/3600926840 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f48e4111ce0 0x7f48e411c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:57.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.558+0000 7f48e9b91640 1 -- 172.21.15.42:0/3600926840 >> 172.21.15.42:0/3600926840 conn(0x7f48e4070c60 msgr2=0x7f48e4073bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:09:57.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.559+0000 7f48e9b91640 1 -- 172.21.15.42:0/3600926840 shutdown_connections 2024-08-30T09:09:57.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:57.559+0000 7f48e9b91640 1 -- 172.21.15.42:0/3600926840 wait complete. 2024-08-30T09:09:58.310 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-08-30T09:09:58.905 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T09:09:59.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:09:58 smithi042 ceph-mon[18652]: pgmap v649: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 3.7 MiB/s wr, 428 op/s 2024-08-30T09:09:59.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:09:58 smithi190 ceph-mon[22001]: pgmap v649: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 3.7 MiB/s wr, 428 op/s 2024-08-30T09:09:59.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:09:58 smithi167 ceph-mon[22063]: pgmap v649: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 3.7 MiB/s wr, 428 op/s 2024-08-30T09:09:59.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:59.875+0000 7f73ef0d7640 1 -- 172.21.15.42:0/3075759520 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f73e8111ce0 msgr2=0x7f73e810a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:09:59.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:59.875+0000 7f73ef0d7640 1 --2- 172.21.15.42:0/3075759520 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f73e8111ce0 0x7f73e810a160 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f73e0009ca0 tx=0x7f73e002a8f0 comp rx=0 tx=0).stop 2024-08-30T09:09:59.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:59.876+0000 7f73ef0d7640 1 -- 172.21.15.42:0/3075759520 shutdown_connections 2024-08-30T09:09:59.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:59.876+0000 7f73ef0d7640 1 --2- 172.21.15.42:0/3075759520 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f73e810a780 0x7f73e810ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:59.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:59.876+0000 7f73ef0d7640 1 --2- 172.21.15.42:0/3075759520 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f73e811c9d0 0x7f73e811eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:59.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:59.876+0000 7f73ef0d7640 1 --2- 172.21.15.42:0/3075759520 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f73e8111ce0 0x7f73e810a160 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:59.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:59.876+0000 7f73ef0d7640 1 -- 172.21.15.42:0/3075759520 >> 172.21.15.42:0/3075759520 conn(0x7f73e8070c60 msgr2=0x7f73e8071070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:09:59.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:59.881+0000 7f73ef0d7640 1 -- 172.21.15.42:0/3075759520 shutdown_connections 2024-08-30T09:09:59.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:59.883+0000 7f73ef0d7640 1 -- 172.21.15.42:0/3075759520 wait complete. 2024-08-30T09:09:59.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:59.884+0000 7f73ef0d7640 1 Processor -- start 2024-08-30T09:09:59.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:59.884+0000 7f73ef0d7640 1 -- start start 2024-08-30T09:09:59.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:59.885+0000 7f73ef0d7640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f73e810a780 0x7f73e81a90b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:59.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:59.885+0000 7f73ef0d7640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f73e8111ce0 0x7f73e81a9650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:59.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:59.885+0000 7f73ef0d7640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f73e811c9d0 0x7f73e81ae8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:59.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:59.885+0000 7f73ef0d7640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f73e8122700 con 0x7f73e811c9d0 2024-08-30T09:09:59.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:59.885+0000 7f73ef0d7640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f73e8122580 con 0x7f73e8111ce0 2024-08-30T09:09:59.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:59.885+0000 7f73ef0d7640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f73e8122880 con 0x7f73e810a780 2024-08-30T09:09:59.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:59.887+0000 7f73ed674640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f73e811c9d0 0x7f73e81ae8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:59.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:59.887+0000 7f73ed674640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f73e811c9d0 0x7f73e81ae8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:57700/0 (socket says 172.21.15.42:57700) 2024-08-30T09:09:59.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:59.887+0000 7f73ed674640 1 -- 172.21.15.42:0/3800962118 learned_addr learned my addr 172.21.15.42:0/3800962118 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:09:59.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:59.888+0000 7f73ed674640 1 -- 172.21.15.42:0/3800962118 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f73e810a780 msgr2=0x7f73e81a90b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:09:59.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:59.888+0000 7f73ed674640 1 --2- 172.21.15.42:0/3800962118 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f73e810a780 0x7f73e81a90b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:59.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:59.888+0000 7f73ed674640 1 -- 172.21.15.42:0/3800962118 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f73e8111ce0 msgr2=0x7f73e81a9650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:09:59.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:59.888+0000 7f73ed674640 1 --2- 172.21.15.42:0/3800962118 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f73e8111ce0 0x7f73e81a9650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:59.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:59.888+0000 7f73ed674640 1 -- 172.21.15.42:0/3800962118 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73e81aeff0 con 0x7f73e811c9d0 2024-08-30T09:09:59.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:59.889+0000 7f73ed674640 1 --2- 172.21.15.42:0/3800962118 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f73e811c9d0 0x7f73e81ae8f0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f73dc00e970 tx=0x7f73dc00ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:09:59.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:59.890+0000 7f73e5ffb640 1 -- 172.21.15.42:0/3800962118 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73dc00ccc0 con 0x7f73e811c9d0 2024-08-30T09:09:59.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:59.890+0000 7f73ef0d7640 1 -- 172.21.15.42:0/3800962118 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73e81af2e0 con 0x7f73e811c9d0 2024-08-30T09:09:59.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:59.890+0000 7f73ef0d7640 1 -- 172.21.15.42:0/3800962118 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73e8112120 con 0x7f73e811c9d0 2024-08-30T09:09:59.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:59.894+0000 7f73f564c740 1 Processor -- start 2024-08-30T09:09:59.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:59.895+0000 7f73f564c740 1 -- start start 2024-08-30T09:09:59.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:59.895+0000 7f73f564c740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a678261070 0x55a678606310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:59.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:59.895+0000 7f73f564c740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a6782381b0 0x55a678606850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:59.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:59.895+0000 7f73f564c740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a678240390 0x55a678606d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:59.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:59.895+0000 7f73f564c740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55a67857d680 con 0x55a678240390 2024-08-30T09:09:59.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:59.895+0000 7f73f564c740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55a67856cc10 con 0x55a678261070 2024-08-30T09:09:59.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:59.895+0000 7f73f564c740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55a678570bc0 con 0x55a6782381b0 2024-08-30T09:09:59.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:59.895+0000 7f73ece73640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a6782381b0 0x55a678606850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:59.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:59.895+0000 7f73ece73640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a6782381b0 0x55a678606850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:43378/0 (socket says 172.21.15.42:43378) 2024-08-30T09:09:59.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:59.895+0000 7f73ece73640 1 -- 172.21.15.42:0/2320714090 learned_addr learned my addr 172.21.15.42:0/2320714090 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:09:59.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:59.896+0000 7f73e7fff640 1 --2- 172.21.15.42:0/2320714090 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a678261070 0x55a678606310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:59.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:59.896+0000 7f73e7fff640 1 --2- 172.21.15.42:0/2320714090 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a678240390 0x55a678606d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:59.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:59.899+0000 7f73e5ffb640 1 -- 172.21.15.42:0/3800962118 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f73dc00ce60 con 0x7f73e811c9d0 2024-08-30T09:09:59.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:59.899+0000 7f73e5ffb640 1 -- 172.21.15.42:0/3800962118 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73dc010430 con 0x7f73e811c9d0 2024-08-30T09:09:59.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:59.902+0000 7f73ece73640 1 -- 172.21.15.42:0/2320714090 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a678261070 msgr2=0x55a678606310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:09:59.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:59.902+0000 7f73ece73640 1 --2- 172.21.15.42:0/2320714090 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a678261070 0x55a678606310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:59.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:59.902+0000 7f73ece73640 1 -- 172.21.15.42:0/2320714090 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a678240390 msgr2=0x55a678606d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:09:59.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:59.902+0000 7f73ece73640 1 --2- 172.21.15.42:0/2320714090 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a678240390 0x55a678606d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:59.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:59.902+0000 7f73ece73640 1 -- 172.21.15.42:0/2320714090 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a6782cc650 con 0x55a6782381b0 2024-08-30T09:09:59.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:59.904+0000 7f73e5ffb640 1 -- 172.21.15.42:0/3800962118 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7f73dc005020 con 0x7f73e811c9d0 2024-08-30T09:09:59.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:59.905+0000 7f73e5ffb640 1 --2- 172.21.15.42:0/3800962118 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f73d00b4d40 0x7f73d00b7250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:59.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:59.905+0000 7f73e5ffb640 1 -- 172.21.15.42:0/3800962118 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f73dc0e56a0 con 0x7f73e811c9d0 2024-08-30T09:09:59.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:59.906+0000 7f73ece73640 1 --2- 172.21.15.42:0/2320714090 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a6782381b0 0x55a678606850 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f73e002aee0 tx=0x7f73e00099a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=497f8ae282f15ef6 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:09:59.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:59.908+0000 7f73e7fff640 1 --2- 172.21.15.42:0/3800962118 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f73d00b4d40 0x7f73d00b7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:59.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:59.908+0000 7f73ceffd640 1 -- 172.21.15.42:0/2320714090 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73e0031ca0 con 0x55a6782381b0 2024-08-30T09:09:59.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:59.908+0000 7f73ceffd640 1 -- 172.21.15.42:0/2320714090 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f73e0036040 con 0x55a6782381b0 2024-08-30T09:09:59.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:59.908+0000 7f73ceffd640 1 -- 172.21.15.42:0/2320714090 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73e0004350 con 0x55a6782381b0 2024-08-30T09:09:59.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:59.909+0000 7f73f564c740 1 -- 172.21.15.42:0/2320714090 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a6782381b0 msgr2=0x55a678606850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:09:59.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:59.909+0000 7f73f564c740 1 --2- 172.21.15.42:0/2320714090 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a6782381b0 0x55a678606850 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f73e002aee0 tx=0x7f73e00099a0 comp rx=0 tx=0).stop 2024-08-30T09:09:59.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:59.914+0000 7f73e7fff640 1 --2- 172.21.15.42:0/3800962118 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f73d00b4d40 0x7f73d00b7250 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f73e811bfc0 tx=0x7f73d800b040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:09:59.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:59.917+0000 7f73f564c740 1 -- 172.21.15.42:0/2320714090 shutdown_connections 2024-08-30T09:09:59.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:59.917+0000 7f73f564c740 1 --2- 172.21.15.42:0/2320714090 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a6782381b0 0x55a678606850 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:59.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:59.917+0000 7f73f564c740 1 --2- 172.21.15.42:0/2320714090 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a678240390 0x55a678606d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:59.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:59.917+0000 7f73f564c740 1 --2- 172.21.15.42:0/2320714090 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a678261070 0x55a678606310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:59.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:59.917+0000 7f73f564c740 1 -- 172.21.15.42:0/2320714090 >> 172.21.15.42:0/2320714090 conn(0x55a678255460 msgr2=0x55a678604200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:09:59.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:59.921+0000 7f73f564c740 1 -- 172.21.15.42:0/2320714090 shutdown_connections 2024-08-30T09:09:59.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:59.921+0000 7f73f564c740 1 -- 172.21.15.42:0/2320714090 wait complete. 2024-08-30T09:09:59.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:59.922+0000 7f73f564c740 1 -- start start 2024-08-30T09:09:59.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:59.922+0000 7f73f564c740 1 Processor -- start 2024-08-30T09:09:59.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:59.923+0000 7f73f564c740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a6782381b0 0x55a678611ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:59.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:59.923+0000 7f73f564c740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a678240390 0x55a678612010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:59.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:59.923+0000 7f73f564c740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a678261070 0x55a67861de10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:59.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:59.923+0000 7f73f564c740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55a6783cf700 con 0x55a678261070 2024-08-30T09:09:59.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:59.923+0000 7f73f564c740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55a6784f3750 con 0x55a678240390 2024-08-30T09:09:59.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:59.923+0000 7f73f564c740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55a678614a90 con 0x55a6782381b0 2024-08-30T09:09:59.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:59.923+0000 7f73f564c740 10 client.0 _ll_register_callbacks cb 0x55a678522e10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:09:59.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:59.924+0000 7f73ed674640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a678261070 0x55a67861de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:59.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:59.924+0000 7f73e7fff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a678240390 0x55a678612010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:59.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:59.924+0000 7f73ed674640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a678261070 0x55a67861de10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:57710/0 (socket says 172.21.15.42:57710) 2024-08-30T09:09:59.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:59.924+0000 7f73ed674640 1 -- 172.21.15.42:0/2926855946 learned_addr learned my addr 172.21.15.42:0/2926855946 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:09:59.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:59.924+0000 7f73ed674640 1 -- 172.21.15.42:0/2926855946 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a6782381b0 msgr2=0x55a678611ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:09:59.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:59.924+0000 7f73ed674640 1 --2- 172.21.15.42:0/2926855946 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a6782381b0 0x55a678611ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:59.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:59.924+0000 7f73ed674640 1 -- 172.21.15.42:0/2926855946 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a678240390 msgr2=0x55a678612010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:09:59.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:59.924+0000 7f73ed674640 1 --2- 172.21.15.42:0/2926855946 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a678240390 0x55a678612010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:59.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:59.924+0000 7f73ed674640 1 -- 172.21.15.42:0/2926855946 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a67852c900 con 0x55a678261070 2024-08-30T09:09:59.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:59.924+0000 7f73ed674640 1 --2- 172.21.15.42:0/2926855946 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a678261070 0x55a67861de10 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f73dc0ab9e0 tx=0x7f73dc0ac3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91e4a4dfac78684c server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:09:59.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:59.926+0000 7f73c7fff640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-08-30T09:09:59.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:59.926+0000 7f73c7fff640 1 -- 172.21.15.42:0/2926855946 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73dc0d2020 con 0x55a678261070 2024-08-30T09:09:59.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:59.926+0000 7f73c7fff640 1 -- 172.21.15.42:0/2926855946 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f73dc0b01b0 con 0x55a678261070 2024-08-30T09:09:59.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:59.927+0000 7f73f564c740 1 -- 172.21.15.42:0/2926855946 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f73e0036040 con 0x55a678261070 2024-08-30T09:09:59.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:59.929+0000 7f73c7fff640 1 -- 172.21.15.42:0/2926855946 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f73dc0b5000 con 0x55a678261070 2024-08-30T09:09:59.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:59.932+0000 7f73f564c740 10 client.16968 fetch_fsmap learned FSMap version 39 2024-08-30T09:09:59.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:59.932+0000 7f73f564c740 1 -- 172.21.15.42:0/2926855946 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55a678620970 con 0x55a678261070 2024-08-30T09:09:59.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:59.935+0000 7f73c7fff640 1 -- 172.21.15.42:0/2926855946 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f73e0036040 con 0x55a678261070 2024-08-30T09:09:59.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:59.935+0000 7f73c7fff640 1 -- 172.21.15.42:0/2926855946 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7f73dc0046c0 con 0x55a678261070 2024-08-30T09:09:59.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:59.935+0000 7f73f564c740 10 client.16968 fetch_fsmap finished waiting for FSMap version 39 2024-08-30T09:09:59.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:59.935+0000 7f73f564c740 10 client.16968 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-08-30T09:09:59.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:59.935+0000 7f73f564c740 20 client.16968 populate_metadata read hostname 'smithi042' 2024-08-30T09:09:59.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:59.935+0000 7f73f564c740 1 --2- 172.21.15.42:0/2926855946 >> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] conn(0x55a67821f9f0 0x55a6786243b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:59.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:59.935+0000 7f73f564c740 4 client.16968 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-30T09:09:59.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:59.935+0000 7f73f564c740 1 -- 172.21.15.42:0/2926855946 --> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a67857f5d0 con 0x55a67821f9f0 2024-08-30T09:09:59.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:59.935+0000 7f73ece73640 1 --2- 172.21.15.42:0/2926855946 >> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] conn(0x55a67821f9f0 0x55a6786243b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:59.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:59.936+0000 7f73c7fff640 10 client.16968 ms_handle_connect on v2:172.21.15.190:6838/719981110 2024-08-30T09:09:59.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:59.937+0000 7f73ece73640 1 --2- 172.21.15.42:0/2926855946 >> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] conn(0x55a67821f9f0 0x55a6786243b0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2f4ee1738a2bf9b6 server_cookie=4cc8b26a655167a5 in_seq=0 out_seq=0 2024-08-30T09:09:59.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:59.940+0000 7f73c7fff640 1 -- 172.21.15.42:0/2926855946 <== mds.1 v2:172.21.15.190:6838/719981110 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a67857f5d0 con 0x55a67821f9f0 2024-08-30T09:09:59.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:59.940+0000 7f73c7fff640 10 client.16968 handle_command_reply: tid=0 2024-08-30T09:09:59.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:59.940+0000 7f73f564c740 1 client.16968 shutdown 2024-08-30T09:09:59.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:59.941+0000 7f73f564c740 1 -- 172.21.15.42:0/2926855946 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a678261070 msgr2=0x55a67861de10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:09:59.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:59.941+0000 7f73f564c740 1 --2- 172.21.15.42:0/2926855946 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a678261070 0x55a67861de10 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f73dc0ab9e0 tx=0x7f73dc0ac3e0 comp rx=0 tx=0).stop 2024-08-30T09:09:59.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:59.942+0000 7f73f564c740 1 -- 172.21.15.42:0/2926855946 shutdown_connections 2024-08-30T09:09:59.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:59.942+0000 7f73f564c740 1 --2- 172.21.15.42:0/2926855946 >> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] conn(0x55a67821f9f0 0x55a6786243b0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:59.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:59.942+0000 7f73f564c740 1 --2- 172.21.15.42:0/2926855946 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a678261070 0x55a67861de10 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:59.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:59.942+0000 7f73f564c740 1 --2- 172.21.15.42:0/2926855946 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a678240390 0x55a678612010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:59.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:59.942+0000 7f73f564c740 1 --2- 172.21.15.42:0/2926855946 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a6782381b0 0x55a678611ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:59.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:59.942+0000 7f73f564c740 1 -- 172.21.15.42:0/2926855946 >> 172.21.15.42:0/2926855946 conn(0x55a678255460 msgr2=0x55a6786094a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:09:59.947 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:59.944+0000 7f73c7fff640 0 client.16968 ms_handle_reset on v2:172.21.15.190:6838/719981110 2024-08-30T09:09:59.947 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:59.945+0000 7f73f564c740 1 -- 172.21.15.42:0/2926855946 shutdown_connections 2024-08-30T09:09:59.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:59.948+0000 7f73f564c740 1 -- 172.21.15.42:0/2926855946 wait complete. 2024-08-30T09:09:59.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:59.950+0000 7f73f564c740 20 client.16968 trim_cache size 0 max 16384 2024-08-30T09:09:59.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:59.981+0000 7f73f564c740 1 Processor -- start 2024-08-30T09:09:59.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:59.981+0000 7f73f564c740 1 -- start start 2024-08-30T09:09:59.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:59.982+0000 7f73f564c740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a6782381b0 0x55a678611f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:59.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:59.982+0000 7f73f564c740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a678240390 0x55a678609410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:59.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:59.982+0000 7f73f564c740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a678261070 0x55a678609950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:59.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:59.982+0000 7f73f564c740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f73d4012510 con 0x55a6782381b0 2024-08-30T09:09:59.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:59.982+0000 7f73e7fff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a678240390 0x55a678609410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:59.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:59.982+0000 7f73e7fff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a678240390 0x55a678609410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:60754/0 (socket says 172.21.15.42:60754) 2024-08-30T09:09:59.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:59.982+0000 7f73e7fff640 1 -- 172.21.15.42:0/3354684803 learned_addr learned my addr 172.21.15.42:0/3354684803 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:09:59.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:59.982+0000 7f73f564c740 1 -- 172.21.15.42:0/3354684803 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f73d400fd80 con 0x55a678240390 2024-08-30T09:09:59.986 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:59.982+0000 7f73f564c740 1 -- 172.21.15.42:0/3354684803 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f73d400dbd0 con 0x55a678261070 2024-08-30T09:09:59.986 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:59.982+0000 7f73ed674640 1 --2- 172.21.15.42:0/3354684803 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a678261070 0x55a678609950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:59.986 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:59.983+0000 7f73ece73640 1 --2- 172.21.15.42:0/3354684803 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a6782381b0 0x55a678611f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:09:59.986 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:59.983+0000 7f73ed674640 1 -- 172.21.15.42:0/3354684803 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a678240390 msgr2=0x55a678609410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:09:59.986 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:59.983+0000 7f73ed674640 1 --2- 172.21.15.42:0/3354684803 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a678240390 0x55a678609410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:59.986 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:59.983+0000 7f73ed674640 1 -- 172.21.15.42:0/3354684803 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a6782381b0 msgr2=0x55a678611f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:09:59.987 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:59.983+0000 7f73ed674640 1 --2- 172.21.15.42:0/3354684803 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a6782381b0 0x55a678611f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:59.987 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:59.983+0000 7f73ed674640 1 -- 172.21.15.42:0/3354684803 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a67860c700 con 0x55a678261070 2024-08-30T09:09:59.987 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:59.983+0000 7f73ed674640 1 --2- 172.21.15.42:0/3354684803 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a678261070 0x55a678609950 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f73dc0ac590 tx=0x7f73dc004b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6322a76a63b15092 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:09:59.987 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:59.983+0000 7f73ce7fc640 1 -- 172.21.15.42:0/3354684803 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73dc0d2040 con 0x55a678261070 2024-08-30T09:09:59.987 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:59.984+0000 7f73ce7fc640 1 -- 172.21.15.42:0/3354684803 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f73dc0ca070 con 0x55a678261070 2024-08-30T09:09:59.987 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:59.984+0000 7f73ce7fc640 1 -- 172.21.15.42:0/3354684803 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73dc0b01b0 con 0x55a678261070 2024-08-30T09:09:59.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:59.985+0000 7f73f564c740 1 -- 172.21.15.42:0/3354684803 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a678261070 msgr2=0x55a678609950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:09:59.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:59.985+0000 7f73f564c740 1 --2- 172.21.15.42:0/3354684803 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a678261070 0x55a678609950 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f73dc0ac590 tx=0x7f73dc004b60 comp rx=0 tx=0).stop 2024-08-30T09:09:59.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:59.988+0000 7f73f564c740 1 -- 172.21.15.42:0/3354684803 shutdown_connections 2024-08-30T09:09:59.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:59.988+0000 7f73f564c740 1 --2- 172.21.15.42:0/3354684803 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a678261070 0x55a678609950 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:59.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:59.988+0000 7f73f564c740 1 --2- 172.21.15.42:0/3354684803 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a678240390 0x55a678609410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:59.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:59.988+0000 7f73f564c740 1 --2- 172.21.15.42:0/3354684803 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a6782381b0 0x55a678611f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:09:59.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:59.988+0000 7f73f564c740 1 -- 172.21.15.42:0/3354684803 >> 172.21.15.42:0/3354684803 conn(0x55a678255460 msgr2=0x55a678611880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:09:59.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:59.993+0000 7f73f564c740 1 -- 172.21.15.42:0/3354684803 shutdown_connections 2024-08-30T09:09:59.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:59.993+0000 7f73f564c740 1 -- 172.21.15.42:0/3354684803 wait complete. 2024-08-30T09:09:59.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:59.994+0000 7f73f564c740 1 -- start start 2024-08-30T09:09:59.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:59.994+0000 7f73f564c740 1 Processor -- start 2024-08-30T09:09:59.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:59.995+0000 7f73f564c740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a6782381b0 0x55a67861a610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:59.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:59.995+0000 7f73f564c740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a678240390 0x55a678615ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:59.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:59.995+0000 7f73f564c740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a678261070 0x55a6786160c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:09:59.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:59.995+0000 7f73f564c740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f73d400e240 con 0x55a6782381b0 2024-08-30T09:09:59.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:59.995+0000 7f73f564c740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f73d400f6b0 con 0x55a678261070 2024-08-30T09:09:59.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:59.995+0000 7f73f564c740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f73d400e910 con 0x55a678240390 2024-08-30T09:09:59.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:59.995+0000 7f73f564c740 10 client.0 _ll_register_callbacks cb 0x7f73dc009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:10:00.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:59.996+0000 7f73ed674640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a678261070 0x55a6786160c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:10:00.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:59.996+0000 7f73e7fff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a678240390 0x55a678615ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:10:00.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:59.996+0000 7f73ece73640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a6782381b0 0x55a67861a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:10:00.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:59.996+0000 7f73ece73640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a6782381b0 0x55a67861a610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:57722/0 (socket says 172.21.15.42:57722) 2024-08-30T09:10:00.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:59.996+0000 7f73ece73640 1 -- 172.21.15.42:0/261859928 learned_addr learned my addr 172.21.15.42:0/261859928 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:10:00.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:59.997+0000 7f73ece73640 1 -- 172.21.15.42:0/261859928 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a678240390 msgr2=0x55a678615ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:10:00.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:59.997+0000 7f73ece73640 1 --2- 172.21.15.42:0/261859928 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a678240390 0x55a678615ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:00.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:59.997+0000 7f73ece73640 1 -- 172.21.15.42:0/261859928 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a678261070 msgr2=0x55a6786160c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:10:00.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:59.997+0000 7f73ece73640 1 --2- 172.21.15.42:0/261859928 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a678261070 0x55a6786160c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:00.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:59.997+0000 7f73ece73640 1 -- 172.21.15.42:0/261859928 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a6786168a0 con 0x55a6782381b0 2024-08-30T09:10:00.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:59.998+0000 7f73ece73640 1 --2- 172.21.15.42:0/261859928 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a6782381b0 0x55a67861a610 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f73e0003f60 tx=0x7f73e0003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2b421ded569abf5 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:10:00.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:59.998+0000 7f73c7fff640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-08-30T09:10:00.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:59.999+0000 7f73c7fff640 1 -- 172.21.15.42:0/261859928 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73e0009540 con 0x55a6782381b0 2024-08-30T09:10:00.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:59.999+0000 7f73c7fff640 1 -- 172.21.15.42:0/261859928 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f73e0031480 con 0x55a6782381b0 2024-08-30T09:10:00.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:09:59.999+0000 7f73f564c740 1 -- 172.21.15.42:0/261859928 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f73dc0ca070 con 0x55a6782381b0 2024-08-30T09:10:00.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:00.000+0000 7f73c7fff640 1 -- 172.21.15.42:0/261859928 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f73e0031cb0 con 0x55a6782381b0 2024-08-30T09:10:00.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:00.000+0000 7f73f564c740 10 client.16974 fetch_fsmap learned FSMap version 39 2024-08-30T09:10:00.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:00.000+0000 7f73f564c740 1 -- 172.21.15.42:0/261859928 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55a67868ee30 con 0x55a6782381b0 2024-08-30T09:10:00.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:00.001+0000 7f73c7fff640 1 -- 172.21.15.42:0/261859928 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f73e0036040 con 0x55a6782381b0 2024-08-30T09:10:00.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:00.001+0000 7f73c7fff640 1 -- 172.21.15.42:0/261859928 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7f73e004f070 con 0x55a6782381b0 2024-08-30T09:10:00.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:00.001+0000 7f73f564c740 10 client.16974 fetch_fsmap finished waiting for FSMap version 39 2024-08-30T09:10:00.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:00.001+0000 7f73f564c740 10 client.16974 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-08-30T09:10:00.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:00.001+0000 7f73f564c740 20 client.16974 populate_metadata read hostname 'smithi042' 2024-08-30T09:10:00.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:00.001+0000 7f73f564c740 1 --2- 172.21.15.42:0/261859928 >> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] conn(0x55a6781f4da0 0x55a678691180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:00.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:00.001+0000 7f73f564c740 4 client.16974 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-30T09:10:00.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:00.001+0000 7f73f564c740 1 -- 172.21.15.42:0/261859928 --> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f73dc0ca070 con 0x55a6781f4da0 2024-08-30T09:10:00.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:00.003+0000 7f73e7fff640 1 --2- 172.21.15.42:0/261859928 >> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] conn(0x55a6781f4da0 0x55a678691180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:10:00.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:00.007+0000 7f73e7fff640 1 --2- 172.21.15.42:0/261859928 >> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] conn(0x55a6781f4da0 0x55a678691180 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a62b7ffa4b12a3e7 server_cookie=97231c4727a129be in_seq=0 out_seq=0 2024-08-30T09:10:00.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:00.007+0000 7f73c7fff640 10 client.16974 ms_handle_connect on v2:172.21.15.190:6838/719981110 2024-08-30T09:10:00.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:00.007+0000 7f73c7fff640 1 -- 172.21.15.42:0/261859928 <== mds.1 v2:172.21.15.190:6838/719981110 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f73dc0ca070 con 0x55a6781f4da0 2024-08-30T09:10:00.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:00.007+0000 7f73c7fff640 10 client.16974 handle_command_reply: tid=0 2024-08-30T09:10:00.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:00.009+0000 7f73f564c740 1 client.16974 shutdown 2024-08-30T09:10:00.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:00.016+0000 7f73f564c740 1 -- 172.21.15.42:0/261859928 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a6782381b0 msgr2=0x55a67861a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:10:00.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:00.016+0000 7f73f564c740 1 --2- 172.21.15.42:0/261859928 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a6782381b0 0x55a67861a610 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f73e0003f60 tx=0x7f73e0003ba0 comp rx=0 tx=0).stop 2024-08-30T09:10:00.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:00.016+0000 7f73f564c740 1 -- 172.21.15.42:0/261859928 shutdown_connections 2024-08-30T09:10:00.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:00.016+0000 7f73f564c740 1 --2- 172.21.15.42:0/261859928 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a678240390 0x55a678615ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:00.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:00.017+0000 7f73f564c740 1 --2- 172.21.15.42:0/261859928 >> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] conn(0x55a6781f4da0 0x55a678691180 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:00.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:00.017+0000 7f73f564c740 1 --2- 172.21.15.42:0/261859928 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a678261070 0x55a6786160c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:00.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:00.017+0000 7f73f564c740 1 --2- 172.21.15.42:0/261859928 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a6782381b0 0x55a67861a610 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:00.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:00.017+0000 7f73f564c740 1 -- 172.21.15.42:0/261859928 >> 172.21.15.42:0/261859928 conn(0x55a678255460 msgr2=0x55a678609410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:10:00.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:00.017+0000 7f73c7fff640 0 client.16974 ms_handle_reset on v2:172.21.15.190:6838/719981110 2024-08-30T09:10:00.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:00.023+0000 7f73f564c740 1 -- 172.21.15.42:0/261859928 shutdown_connections 2024-08-30T09:10:00.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:00.023+0000 7f73f564c740 1 -- 172.21.15.42:0/261859928 wait complete. 2024-08-30T09:10:00.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:00.023+0000 7f73f564c740 20 client.16974 trim_cache size 0 max 16384 2024-08-30T09:10:00.025 INFO:teuthology.orchestra.run.smithi042.stdout:[] 2024-08-30T09:10:00.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:00.024+0000 7f73ef0d7640 1 -- 172.21.15.42:0/3800962118 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f73d00b4d40 msgr2=0x7f73d00b7250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:10:00.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:00.024+0000 7f73ef0d7640 1 --2- 172.21.15.42:0/3800962118 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f73d00b4d40 0x7f73d00b7250 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f73e811bfc0 tx=0x7f73d800b040 comp rx=0 tx=0).stop 2024-08-30T09:10:00.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:00.024+0000 7f73ef0d7640 1 -- 172.21.15.42:0/3800962118 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f73e811c9d0 msgr2=0x7f73e81ae8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:10:00.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:00.024+0000 7f73ef0d7640 1 --2- 172.21.15.42:0/3800962118 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f73e811c9d0 0x7f73e81ae8f0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f73dc00e970 tx=0x7f73dc00ed70 comp rx=0 tx=0).stop 2024-08-30T09:10:00.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:00.024+0000 7f73ef0d7640 1 -- 172.21.15.42:0/3800962118 shutdown_connections 2024-08-30T09:10:00.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:00.024+0000 7f73ef0d7640 1 --2- 172.21.15.42:0/3800962118 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f73d00b4d40 0x7f73d00b7250 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:00.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:00.024+0000 7f73ef0d7640 1 --2- 172.21.15.42:0/3800962118 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f73e811c9d0 0x7f73e81ae8f0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:00.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:00.024+0000 7f73ef0d7640 1 --2- 172.21.15.42:0/3800962118 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f73e8111ce0 0x7f73e81a9650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:00.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:00.024+0000 7f73ef0d7640 1 --2- 172.21.15.42:0/3800962118 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f73e810a780 0x7f73e81a90b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:00.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:00.024+0000 7f73ef0d7640 1 -- 172.21.15.42:0/3800962118 >> 172.21.15.42:0/3800962118 conn(0x7f73e8070c60 msgr2=0x7f73e811e220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:10:00.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:00.025+0000 7f73ef0d7640 1 -- 172.21.15.42:0/3800962118 shutdown_connections 2024-08-30T09:10:00.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:00.025+0000 7f73ef0d7640 1 -- 172.21.15.42:0/3800962118 wait complete. 2024-08-30T09:10:00.903 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-08-30T09:10:00.953 INFO:teuthology.orchestra.run.smithi042.stderr:progress: 555.0 s, 520.2 tps, lat 197.165 ms stddev 230.090 2024-08-30T09:10:01.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:10:00 smithi042 ceph-mon[18652]: pgmap v650: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 4.1 MiB/s wr, 493 op/s 2024-08-30T09:10:01.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:10:00 smithi042 ceph-mon[18652]: overall HEALTH_OK 2024-08-30T09:10:01.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:10:00 smithi190 ceph-mon[22001]: pgmap v650: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 4.1 MiB/s wr, 493 op/s 2024-08-30T09:10:01.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:10:00 smithi190 ceph-mon[22001]: overall HEALTH_OK 2024-08-30T09:10:01.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:10:00 smithi167 ceph-mon[22063]: pgmap v650: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 4.1 MiB/s wr, 493 op/s 2024-08-30T09:10:01.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:10:00 smithi167 ceph-mon[22063]: overall HEALTH_OK 2024-08-30T09:10:01.442 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T09:10:02.135 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.131+0000 7f1964cc5640 1 -- 172.21.15.42:0/2575637287 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1960102540 msgr2=0x7f1960102950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:10:02.135 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.131+0000 7f1964cc5640 1 --2- 172.21.15.42:0/2575637287 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1960102540 0x7f1960102950 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f1950009890 tx=0x7f195002a390 comp rx=0 tx=0).stop 2024-08-30T09:10:02.135 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.132+0000 7f1964cc5640 1 -- 172.21.15.42:0/2575637287 shutdown_connections 2024-08-30T09:10:02.135 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.132+0000 7f1964cc5640 1 --2- 172.21.15.42:0/2575637287 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1960107980 0x7f1960107e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:02.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.132+0000 7f1964cc5640 1 --2- 172.21.15.42:0/2575637287 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1960102e90 0x7f19601073c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:02.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.132+0000 7f1964cc5640 1 --2- 172.21.15.42:0/2575637287 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1960102540 0x7f1960102950 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:02.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.132+0000 7f1964cc5640 1 -- 172.21.15.42:0/2575637287 >> 172.21.15.42:0/2575637287 conn(0x7f19600794d0 msgr2=0x7f19600798e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:10:02.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.136+0000 7f1964cc5640 1 -- 172.21.15.42:0/2575637287 shutdown_connections 2024-08-30T09:10:02.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.139+0000 7f1964cc5640 1 -- 172.21.15.42:0/2575637287 wait complete. 2024-08-30T09:10:02.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.140+0000 7f1964cc5640 1 Processor -- start 2024-08-30T09:10:02.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.144+0000 7f1964cc5640 1 -- start start 2024-08-30T09:10:02.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.145+0000 7f1964cc5640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1960102540 0x7f1960074db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:02.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.145+0000 7f1964cc5640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1960102e90 0x7f196006f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:02.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.145+0000 7f1964cc5640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1960107980 0x7f196006f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:02.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.145+0000 7f1964cc5640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f196006d440 con 0x7f1960102e90 2024-08-30T09:10:02.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.145+0000 7f1964cc5640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f196006d2c0 con 0x7f1960102540 2024-08-30T09:10:02.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.145+0000 7f1964cc5640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f196006d5c0 con 0x7f1960107980 2024-08-30T09:10:02.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.147+0000 7f195e59c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1960102540 0x7f1960074db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:10:02.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.147+0000 7f195e59c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1960102540 0x7f1960074db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:60774/0 (socket says 172.21.15.42:60774) 2024-08-30T09:10:02.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.147+0000 7f195e59c640 1 -- 172.21.15.42:0/3842960059 learned_addr learned my addr 172.21.15.42:0/3842960059 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:10:02.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.147+0000 7f195e59c640 1 -- 172.21.15.42:0/3842960059 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1960107980 msgr2=0x7f196006f7c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-30T09:10:02.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.147+0000 7f195e59c640 1 --2- 172.21.15.42:0/3842960059 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1960107980 0x7f196006f7c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:02.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.147+0000 7f195e59c640 1 -- 172.21.15.42:0/3842960059 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1960102e90 msgr2=0x7f196006f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:10:02.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.147+0000 7f195e59c640 1 --2- 172.21.15.42:0/3842960059 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1960102e90 0x7f196006f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:02.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.147+0000 7f195e59c640 1 -- 172.21.15.42:0/3842960059 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1960108990 con 0x7f1960102540 2024-08-30T09:10:02.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.148+0000 7f195e59c640 1 --2- 172.21.15.42:0/3842960059 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1960102540 0x7f1960074db0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f1950035040 tx=0x7f1950030d60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:10:02.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.148+0000 7f194f7fe640 1 -- 172.21.15.42:0/3842960059 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1950040020 con 0x7f1960102540 2024-08-30T09:10:02.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.148+0000 7f194f7fe640 1 -- 172.21.15.42:0/3842960059 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1950038040 con 0x7f1960102540 2024-08-30T09:10:02.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.148+0000 7f194f7fe640 1 -- 172.21.15.42:0/3842960059 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1950037070 con 0x7f1960102540 2024-08-30T09:10:02.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.148+0000 7f1964cc5640 1 -- 172.21.15.42:0/3842960059 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1960108c20 con 0x7f1960102540 2024-08-30T09:10:02.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.149+0000 7f1964cc5640 1 -- 172.21.15.42:0/3842960059 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19601090b0 con 0x7f1960102540 2024-08-30T09:10:02.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.152+0000 7f196b23a740 1 Processor -- start 2024-08-30T09:10:02.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.153+0000 7f196b23a740 1 -- start start 2024-08-30T09:10:02.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.153+0000 7f196b23a740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a1b0601070 0x55a1b09a6350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:02.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.153+0000 7f196b23a740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a1b05d81b0 0x55a1b09a6890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:02.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.153+0000 7f196b23a740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a1b05e0390 0x55a1b09a6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:02.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.153+0000 7f196b23a740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55a1b091d680 con 0x55a1b0601070 2024-08-30T09:10:02.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.153+0000 7f196b23a740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55a1b090cc10 con 0x55a1b05e0390 2024-08-30T09:10:02.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.153+0000 7f196b23a740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55a1b0910bc0 con 0x55a1b05d81b0 2024-08-30T09:10:02.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.156+0000 7f195dd9b640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a1b05e0390 0x55a1b09a6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:10:02.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.156+0000 7f195dd9b640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a1b05e0390 0x55a1b09a6dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:60776/0 (socket says 172.21.15.42:60776) 2024-08-30T09:10:02.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.156+0000 7f195dd9b640 1 -- 172.21.15.42:0/4000022259 learned_addr learned my addr 172.21.15.42:0/4000022259 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:10:02.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.156+0000 7f195ed9d640 1 --2- 172.21.15.42:0/4000022259 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a1b0601070 0x55a1b09a6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:10:02.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.156+0000 7f195e59c640 1 --2- 172.21.15.42:0/4000022259 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a1b05d81b0 0x55a1b09a6890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:10:02.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.161+0000 7f194f7fe640 1 -- 172.21.15.42:0/3842960059 <== mon.2 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7f1950043020 con 0x7f1960102540 2024-08-30T09:10:02.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.161+0000 7f195e59c640 1 -- 172.21.15.42:0/4000022259 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a1b05e0390 msgr2=0x55a1b09a6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:10:02.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.161+0000 7f195e59c640 1 --2- 172.21.15.42:0/4000022259 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a1b05e0390 0x55a1b09a6dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:02.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.161+0000 7f195e59c640 1 -- 172.21.15.42:0/4000022259 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a1b0601070 msgr2=0x55a1b09a6350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:10:02.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.161+0000 7f195e59c640 1 --2- 172.21.15.42:0/4000022259 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a1b0601070 0x55a1b09a6350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:02.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.161+0000 7f195e59c640 1 -- 172.21.15.42:0/4000022259 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a1b066c650 con 0x55a1b05d81b0 2024-08-30T09:10:02.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.162+0000 7f194f7fe640 1 --2- 172.21.15.42:0/3842960059 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f19480b4d40 0x7f19480b7250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:02.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.163+0000 7f194f7fe640 1 -- 172.21.15.42:0/3842960059 <== mon.2 v2:172.21.15.167:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f195010a2b0 con 0x7f1960102540 2024-08-30T09:10:02.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.165+0000 7f195e59c640 1 --2- 172.21.15.42:0/4000022259 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a1b05d81b0 0x55a1b09a6890 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f1950107960 tx=0x7f195010ccd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=294967247f01740 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:10:02.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.165+0000 7f194cff9640 1 -- 172.21.15.42:0/4000022259 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19500cc720 con 0x55a1b05d81b0 2024-08-30T09:10:02.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.165+0000 7f194cff9640 1 -- 172.21.15.42:0/4000022259 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f19500d0400 con 0x55a1b05d81b0 2024-08-30T09:10:02.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.166+0000 7f195ed9d640 1 --2- 172.21.15.42:0/3842960059 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f19480b4d40 0x7f19480b7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:10:02.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.167+0000 7f196b23a740 1 -- 172.21.15.42:0/4000022259 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a1b05d81b0 msgr2=0x55a1b09a6890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:10:02.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.167+0000 7f196b23a740 1 --2- 172.21.15.42:0/4000022259 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a1b05d81b0 0x55a1b09a6890 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f1950107960 tx=0x7f195010ccd0 comp rx=0 tx=0).stop 2024-08-30T09:10:02.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.174+0000 7f195ed9d640 1 --2- 172.21.15.42:0/3842960059 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f19480b4d40 0x7f19480b7250 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f1958005fd0 tx=0x7f1958010040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:10:02.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.174+0000 7f196b23a740 1 -- 172.21.15.42:0/4000022259 shutdown_connections 2024-08-30T09:10:02.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.174+0000 7f196b23a740 1 --2- 172.21.15.42:0/4000022259 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a1b05d81b0 0x55a1b09a6890 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:02.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.174+0000 7f196b23a740 1 --2- 172.21.15.42:0/4000022259 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a1b05e0390 0x55a1b09a6dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:02.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.174+0000 7f196b23a740 1 --2- 172.21.15.42:0/4000022259 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a1b0601070 0x55a1b09a6350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:02.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.174+0000 7f196b23a740 1 -- 172.21.15.42:0/4000022259 >> 172.21.15.42:0/4000022259 conn(0x55a1b05f5460 msgr2=0x55a1b09a4240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:10:02.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.174+0000 7f196b23a740 1 -- 172.21.15.42:0/4000022259 shutdown_connections 2024-08-30T09:10:02.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.176+0000 7f196b23a740 1 -- 172.21.15.42:0/4000022259 wait complete. 2024-08-30T09:10:02.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.177+0000 7f196b23a740 1 -- start start 2024-08-30T09:10:02.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.177+0000 7f196b23a740 1 Processor -- start 2024-08-30T09:10:02.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.178+0000 7f196b23a740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a1b05d81b0 0x55a1b09b1b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:02.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.178+0000 7f196b23a740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a1b05e0390 0x55a1b09b20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:02.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.178+0000 7f196b23a740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a1b0601070 0x55a1b09bdc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:02.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.178+0000 7f196b23a740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55a1b076f700 con 0x55a1b05e0390 2024-08-30T09:10:02.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.178+0000 7f196b23a740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55a1b0893750 con 0x55a1b05d81b0 2024-08-30T09:10:02.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.178+0000 7f196b23a740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55a1b09b48f0 con 0x55a1b0601070 2024-08-30T09:10:02.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.178+0000 7f196b23a740 10 client.0 _ll_register_callbacks cb 0x55a1b08c2e10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:10:02.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.179+0000 7f195dd9b640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a1b05e0390 0x55a1b09b20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:10:02.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.179+0000 7f195dd9b640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a1b05e0390 0x55a1b09b20f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:57770/0 (socket says 172.21.15.42:57770) 2024-08-30T09:10:02.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.179+0000 7f195ed9d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a1b0601070 0x55a1b09bdc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:10:02.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.179+0000 7f195dd9b640 1 -- 172.21.15.42:0/998407715 learned_addr learned my addr 172.21.15.42:0/998407715 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:10:02.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.179+0000 7f195e59c640 1 --2- 172.21.15.42:0/998407715 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a1b05d81b0 0x55a1b09b1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:10:02.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.179+0000 7f195ed9d640 1 -- 172.21.15.42:0/998407715 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a1b05d81b0 msgr2=0x55a1b09b1b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:10:02.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.179+0000 7f195ed9d640 1 --2- 172.21.15.42:0/998407715 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a1b05d81b0 0x55a1b09b1b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:02.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.179+0000 7f195ed9d640 1 -- 172.21.15.42:0/998407715 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a1b05e0390 msgr2=0x55a1b09b20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:10:02.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.179+0000 7f195ed9d640 1 --2- 172.21.15.42:0/998407715 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a1b05e0390 0x55a1b09b20f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:02.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.180+0000 7f195ed9d640 1 -- 172.21.15.42:0/998407715 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a1b08cc900 con 0x55a1b0601070 2024-08-30T09:10:02.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.180+0000 7f195ed9d640 1 --2- 172.21.15.42:0/998407715 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a1b0601070 0x55a1b09bdc80 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f1958003f50 tx=0x7f1958004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=db3464ef2ccf2d12 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:10:02.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.181+0000 7f1945ffb640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-08-30T09:10:02.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.181+0000 7f1945ffb640 1 -- 172.21.15.42:0/998407715 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1958029060 con 0x55a1b0601070 2024-08-30T09:10:02.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.181+0000 7f196b23a740 1 -- 172.21.15.42:0/998407715 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55a1b091f5d0 con 0x55a1b0601070 2024-08-30T09:10:02.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.181+0000 7f195e59c640 1 --2- 172.21.15.42:0/998407715 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a1b05d81b0 0x55a1b09b1b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-30T09:10:02.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.181+0000 7f1945ffb640 1 -- 172.21.15.42:0/998407715 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f195801b6d0 con 0x55a1b0601070 2024-08-30T09:10:02.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.181+0000 7f1945ffb640 1 -- 172.21.15.42:0/998407715 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f1958030050 con 0x55a1b0601070 2024-08-30T09:10:02.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.182+0000 7f195dd9b640 1 --2- 172.21.15.42:0/998407715 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a1b05e0390 0x55a1b09b20f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-30T09:10:02.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.183+0000 7f196b23a740 10 client.26773 fetch_fsmap learned FSMap version 39 2024-08-30T09:10:02.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.183+0000 7f196b23a740 1 -- 172.21.15.42:0/998407715 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55a1b09c0930 con 0x55a1b0601070 2024-08-30T09:10:02.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.183+0000 7f1945ffb640 1 -- 172.21.15.42:0/998407715 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55a1b091f5d0 con 0x55a1b0601070 2024-08-30T09:10:02.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.185+0000 7f1945ffb640 1 -- 172.21.15.42:0/998407715 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7f195800b040 con 0x55a1b0601070 2024-08-30T09:10:02.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.185+0000 7f196b23a740 10 client.26773 fetch_fsmap finished waiting for FSMap version 39 2024-08-30T09:10:02.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.185+0000 7f196b23a740 10 client.26773 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-30T09:10:02.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.185+0000 7f196b23a740 20 client.26773 populate_metadata read hostname 'smithi042' 2024-08-30T09:10:02.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.185+0000 7f196b23a740 1 --2- 172.21.15.42:0/998407715 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55a1b05bf9f0 0x55a1b09c4360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:02.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.185+0000 7f196b23a740 4 client.26773 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-30T09:10:02.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.185+0000 7f196b23a740 1 -- 172.21.15.42:0/998407715 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a1b091b070 con 0x55a1b05bf9f0 2024-08-30T09:10:02.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.186+0000 7f195e59c640 1 --2- 172.21.15.42:0/998407715 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55a1b05bf9f0 0x55a1b09c4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:10:02.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.187+0000 7f195e59c640 1 --2- 172.21.15.42:0/998407715 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55a1b05bf9f0 0x55a1b09c4360 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d519cf5dcea764 server_cookie=97e3e84287bd9211 in_seq=0 out_seq=0 2024-08-30T09:10:02.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.189+0000 7f1945ffb640 10 client.26773 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:10:02.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.189+0000 7f1945ffb640 1 -- 172.21.15.42:0/998407715 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a1b091b070 con 0x55a1b05bf9f0 2024-08-30T09:10:02.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.189+0000 7f1945ffb640 10 client.26773 handle_command_reply: tid=0 2024-08-30T09:10:02.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.189+0000 7f196b23a740 1 client.26773 shutdown 2024-08-30T09:10:02.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.194+0000 7f196b23a740 1 -- 172.21.15.42:0/998407715 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a1b0601070 msgr2=0x55a1b09bdc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:10:02.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.194+0000 7f196b23a740 1 --2- 172.21.15.42:0/998407715 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a1b0601070 0x55a1b09bdc80 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f1958003f50 tx=0x7f1958004a40 comp rx=0 tx=0).stop 2024-08-30T09:10:02.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.194+0000 7f196b23a740 1 -- 172.21.15.42:0/998407715 shutdown_connections 2024-08-30T09:10:02.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.194+0000 7f196b23a740 1 --2- 172.21.15.42:0/998407715 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55a1b05bf9f0 0x55a1b09c4360 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:02.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.194+0000 7f196b23a740 1 --2- 172.21.15.42:0/998407715 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a1b0601070 0x55a1b09bdc80 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:02.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.194+0000 7f196b23a740 1 --2- 172.21.15.42:0/998407715 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a1b05e0390 0x55a1b09b20f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:02.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.194+0000 7f196b23a740 1 --2- 172.21.15.42:0/998407715 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a1b05d81b0 0x55a1b09b1b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:02.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.194+0000 7f196b23a740 1 -- 172.21.15.42:0/998407715 >> 172.21.15.42:0/998407715 conn(0x55a1b05f5460 msgr2=0x55a1b09a9530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:10:02.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.195+0000 7f1945ffb640 0 client.26773 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:10:02.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.197+0000 7f196b23a740 1 -- 172.21.15.42:0/998407715 shutdown_connections 2024-08-30T09:10:02.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.198+0000 7f196b23a740 1 -- 172.21.15.42:0/998407715 wait complete. 2024-08-30T09:10:02.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.199+0000 7f196b23a740 20 client.26773 trim_cache size 0 max 16384 2024-08-30T09:10:02.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.230+0000 7f196b23a740 1 Processor -- start 2024-08-30T09:10:02.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.231+0000 7f196b23a740 1 -- start start 2024-08-30T09:10:02.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.231+0000 7f196b23a740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a1b05d81b0 0x55a1b09bdb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:02.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.232+0000 7f196b23a740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a1b05e0390 0x55a1b09be0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:02.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.232+0000 7f196b23a740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a1b0601070 0x55a1b09a3b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:02.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.232+0000 7f196b23a740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f1940012460 con 0x55a1b0601070 2024-08-30T09:10:02.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.232+0000 7f196b23a740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f194000fcd0 con 0x55a1b05e0390 2024-08-30T09:10:02.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.232+0000 7f196b23a740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f194000dbd0 con 0x55a1b05d81b0 2024-08-30T09:10:02.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.233+0000 7f195dd9b640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a1b05e0390 0x55a1b09be0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:10:02.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.233+0000 7f195dd9b640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a1b05e0390 0x55a1b09be0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:60786/0 (socket says 172.21.15.42:60786) 2024-08-30T09:10:02.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.233+0000 7f195dd9b640 1 -- 172.21.15.42:0/3952448114 learned_addr learned my addr 172.21.15.42:0/3952448114 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:10:02.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.235+0000 7f195e59c640 1 --2- 172.21.15.42:0/3952448114 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a1b05d81b0 0x55a1b09bdb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:10:02.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.235+0000 7f195dd9b640 1 -- 172.21.15.42:0/3952448114 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a1b05d81b0 msgr2=0x55a1b09bdb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:10:02.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.235+0000 7f195dd9b640 1 --2- 172.21.15.42:0/3952448114 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a1b05d81b0 0x55a1b09bdb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:02.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.235+0000 7f195dd9b640 1 -- 172.21.15.42:0/3952448114 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a1b0601070 msgr2=0x55a1b09a3b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:10:02.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.235+0000 7f195dd9b640 1 --2- 172.21.15.42:0/3952448114 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a1b0601070 0x55a1b09a3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:02.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.235+0000 7f195dd9b640 1 -- 172.21.15.42:0/3952448114 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a1b09ad670 con 0x55a1b05e0390 2024-08-30T09:10:02.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.236+0000 7f195dd9b640 1 --2- 172.21.15.42:0/3952448114 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a1b05e0390 0x55a1b09be0c0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f195400bb80 tx=0x7f1954012820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=47c61e4a20c5ef9c server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:10:02.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.236+0000 7f1947fff640 1 -- 172.21.15.42:0/3952448114 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1954005020 con 0x55a1b05e0390 2024-08-30T09:10:02.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.236+0000 7f1947fff640 1 -- 172.21.15.42:0/3952448114 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f19540047a0 con 0x55a1b05e0390 2024-08-30T09:10:02.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.236+0000 7f1947fff640 1 -- 172.21.15.42:0/3952448114 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f195401c6f0 con 0x55a1b05e0390 2024-08-30T09:10:02.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.237+0000 7f196b23a740 1 -- 172.21.15.42:0/3952448114 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a1b05e0390 msgr2=0x55a1b09be0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:10:02.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.238+0000 7f196b23a740 1 --2- 172.21.15.42:0/3952448114 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a1b05e0390 0x55a1b09be0c0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f195400bb80 tx=0x7f1954012820 comp rx=0 tx=0).stop 2024-08-30T09:10:02.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.238+0000 7f196b23a740 1 -- 172.21.15.42:0/3952448114 shutdown_connections 2024-08-30T09:10:02.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.238+0000 7f196b23a740 1 --2- 172.21.15.42:0/3952448114 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a1b0601070 0x55a1b09a3b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:02.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.238+0000 7f196b23a740 1 --2- 172.21.15.42:0/3952448114 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a1b05e0390 0x55a1b09be0c0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:02.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.238+0000 7f196b23a740 1 --2- 172.21.15.42:0/3952448114 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a1b05d81b0 0x55a1b09bdb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:02.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.238+0000 7f196b23a740 1 -- 172.21.15.42:0/3952448114 >> 172.21.15.42:0/3952448114 conn(0x55a1b05f5460 msgr2=0x55a1b09b03f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:10:02.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.238+0000 7f196b23a740 1 -- 172.21.15.42:0/3952448114 shutdown_connections 2024-08-30T09:10:02.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.239+0000 7f196b23a740 1 -- 172.21.15.42:0/3952448114 wait complete. 2024-08-30T09:10:02.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.239+0000 7f196b23a740 1 -- start start 2024-08-30T09:10:02.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.239+0000 7f196b23a740 1 Processor -- start 2024-08-30T09:10:02.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.240+0000 7f196b23a740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a1b05d81b0 0x55a1b09ba290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:02.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.240+0000 7f196b23a740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a1b05e0390 0x55a1b09b5790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:02.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.240+0000 7f196b23a740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a1b0601070 0x55a1b09b5cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:02.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.240+0000 7f196b23a740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f194000e190 con 0x55a1b05e0390 2024-08-30T09:10:02.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.240+0000 7f196b23a740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f194000f600 con 0x55a1b0601070 2024-08-30T09:10:02.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.240+0000 7f196b23a740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f194000e860 con 0x55a1b05d81b0 2024-08-30T09:10:02.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.240+0000 7f196b23a740 10 client.0 _ll_register_callbacks cb 0x7f1958003c50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:10:02.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.240+0000 7f195e59c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a1b05d81b0 0x55a1b09ba290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:10:02.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.240+0000 7f195e59c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a1b05d81b0 0x55a1b09ba290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:43448/0 (socket says 172.21.15.42:43448) 2024-08-30T09:10:02.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.240+0000 7f195e59c640 1 -- 172.21.15.42:0/1322205970 learned_addr learned my addr 172.21.15.42:0/1322205970 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:10:02.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.240+0000 7f195dd9b640 1 --2- 172.21.15.42:0/1322205970 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a1b05e0390 0x55a1b09b5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:10:02.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.240+0000 7f195ed9d640 1 --2- 172.21.15.42:0/1322205970 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a1b0601070 0x55a1b09b5cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:10:02.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.240+0000 7f195e59c640 1 -- 172.21.15.42:0/1322205970 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a1b0601070 msgr2=0x55a1b09b5cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:10:02.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.241+0000 7f195e59c640 1 --2- 172.21.15.42:0/1322205970 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a1b0601070 0x55a1b09b5cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:02.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.241+0000 7f195e59c640 1 -- 172.21.15.42:0/1322205970 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a1b05e0390 msgr2=0x55a1b09b5790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:10:02.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.241+0000 7f195e59c640 1 --2- 172.21.15.42:0/1322205970 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a1b05e0390 0x55a1b09b5790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:02.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.241+0000 7f195e59c640 1 -- 172.21.15.42:0/1322205970 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a1b09b6340 con 0x55a1b05d81b0 2024-08-30T09:10:02.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.241+0000 7f195dd9b640 1 --2- 172.21.15.42:0/1322205970 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a1b05e0390 0x55a1b09b5790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-30T09:10:02.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.241+0000 7f195e59c640 1 --2- 172.21.15.42:0/1322205970 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a1b05d81b0 0x55a1b09ba290 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f19500099b0 tx=0x7f19500097e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a3c8d4f61d0a264d server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:10:02.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.241+0000 7f1945ffb640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-08-30T09:10:02.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.241+0000 7f1945ffb640 1 -- 172.21.15.42:0/1322205970 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19500d0240 con 0x55a1b05d81b0 2024-08-30T09:10:02.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.242+0000 7f1945ffb640 1 -- 172.21.15.42:0/1322205970 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f19500313a0 con 0x55a1b05d81b0 2024-08-30T09:10:02.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.242+0000 7f196b23a740 1 -- 172.21.15.42:0/1322205970 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f19540047a0 con 0x55a1b05d81b0 2024-08-30T09:10:02.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.243+0000 7f1945ffb640 1 -- 172.21.15.42:0/1322205970 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f19500d06c0 con 0x55a1b05d81b0 2024-08-30T09:10:02.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.244+0000 7f196b23a740 10 client.26779 fetch_fsmap learned FSMap version 39 2024-08-30T09:10:02.246 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.244+0000 7f196b23a740 1 -- 172.21.15.42:0/1322205970 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55a1b0a2eb20 con 0x55a1b05d81b0 2024-08-30T09:10:02.246 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.244+0000 7f1945ffb640 1 -- 172.21.15.42:0/1322205970 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f19540047a0 con 0x55a1b05d81b0 2024-08-30T09:10:02.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.244+0000 7f1945ffb640 1 -- 172.21.15.42:0/1322205970 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7f19500d0b50 con 0x55a1b05d81b0 2024-08-30T09:10:02.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.246+0000 7f196b23a740 10 client.26779 fetch_fsmap finished waiting for FSMap version 39 2024-08-30T09:10:02.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.246+0000 7f196b23a740 10 client.26779 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-30T09:10:02.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.246+0000 7f196b23a740 20 client.26779 populate_metadata read hostname 'smithi042' 2024-08-30T09:10:02.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.246+0000 7f196b23a740 1 --2- 172.21.15.42:0/1322205970 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55a1b0594da0 0x55a1b0a31170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:02.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.246+0000 7f196b23a740 4 client.26779 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-30T09:10:02.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.246+0000 7f196b23a740 1 -- 172.21.15.42:0/1322205970 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a1b09187b0 con 0x55a1b0594da0 2024-08-30T09:10:02.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.246+0000 7f195dd9b640 1 --2- 172.21.15.42:0/1322205970 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55a1b0594da0 0x55a1b0a31170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:10:02.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.248+0000 7f195dd9b640 1 --2- 172.21.15.42:0/1322205970 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55a1b0594da0 0x55a1b0a31170 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=405845cdc53d9e80 server_cookie=fdea571a505393c0 in_seq=0 out_seq=0 2024-08-30T09:10:02.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.249+0000 7f1945ffb640 10 client.26779 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:10:02.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.250+0000 7f1945ffb640 1 -- 172.21.15.42:0/1322205970 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a1b09187b0 con 0x55a1b0594da0 2024-08-30T09:10:02.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.250+0000 7f1945ffb640 10 client.26779 handle_command_reply: tid=0 2024-08-30T09:10:02.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.250+0000 7f196b23a740 1 client.26779 shutdown 2024-08-30T09:10:02.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.250+0000 7f196b23a740 1 -- 172.21.15.42:0/1322205970 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a1b05d81b0 msgr2=0x55a1b09ba290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:10:02.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.250+0000 7f196b23a740 1 --2- 172.21.15.42:0/1322205970 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a1b05d81b0 0x55a1b09ba290 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f19500099b0 tx=0x7f19500097e0 comp rx=0 tx=0).stop 2024-08-30T09:10:02.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.250+0000 7f196b23a740 1 -- 172.21.15.42:0/1322205970 shutdown_connections 2024-08-30T09:10:02.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.250+0000 7f196b23a740 1 --2- 172.21.15.42:0/1322205970 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55a1b0594da0 0x55a1b0a31170 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:02.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.250+0000 7f196b23a740 1 --2- 172.21.15.42:0/1322205970 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a1b0601070 0x55a1b09b5cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:02.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.250+0000 7f196b23a740 1 --2- 172.21.15.42:0/1322205970 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a1b05e0390 0x55a1b09b5790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:02.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.250+0000 7f196b23a740 1 --2- 172.21.15.42:0/1322205970 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a1b05d81b0 0x55a1b09ba290 secure :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f19500099b0 tx=0x7f19500097e0 comp rx=0 tx=0).stop 2024-08-30T09:10:02.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.250+0000 7f196b23a740 1 -- 172.21.15.42:0/1322205970 >> 172.21.15.42:0/1322205970 conn(0x55a1b05f5460 msgr2=0x55a1b09a3b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:10:02.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.250+0000 7f1945ffb640 0 client.26779 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:10:02.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.250+0000 7f196b23a740 1 -- 172.21.15.42:0/1322205970 shutdown_connections 2024-08-30T09:10:02.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.250+0000 7f196b23a740 1 -- 172.21.15.42:0/1322205970 wait complete. 2024-08-30T09:10:02.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.250+0000 7f196b23a740 20 client.26779 trim_cache size 0 max 16384 2024-08-30T09:10:02.255 INFO:teuthology.orchestra.run.smithi042.stdout:[] 2024-08-30T09:10:02.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.253+0000 7f1964cc5640 1 -- 172.21.15.42:0/3842960059 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f19480b4d40 msgr2=0x7f19480b7250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:10:02.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.253+0000 7f1964cc5640 1 --2- 172.21.15.42:0/3842960059 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f19480b4d40 0x7f19480b7250 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f1958005fd0 tx=0x7f1958010040 comp rx=0 tx=0).stop 2024-08-30T09:10:02.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.253+0000 7f1964cc5640 1 -- 172.21.15.42:0/3842960059 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1960102540 msgr2=0x7f1960074db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:10:02.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.253+0000 7f1964cc5640 1 --2- 172.21.15.42:0/3842960059 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1960102540 0x7f1960074db0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f1950035040 tx=0x7f1950030d60 comp rx=0 tx=0).stop 2024-08-30T09:10:02.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.254+0000 7f1964cc5640 1 -- 172.21.15.42:0/3842960059 shutdown_connections 2024-08-30T09:10:02.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.254+0000 7f1964cc5640 1 --2- 172.21.15.42:0/3842960059 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f19480b4d40 0x7f19480b7250 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:02.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.254+0000 7f1964cc5640 1 --2- 172.21.15.42:0/3842960059 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1960107980 0x7f196006f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:02.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.254+0000 7f1964cc5640 1 --2- 172.21.15.42:0/3842960059 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1960102e90 0x7f196006f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:02.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.254+0000 7f1964cc5640 1 --2- 172.21.15.42:0/3842960059 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1960102540 0x7f1960074db0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:02.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.254+0000 7f1964cc5640 1 -- 172.21.15.42:0/3842960059 >> 172.21.15.42:0/3842960059 conn(0x7f19600794d0 msgr2=0x7f1960105390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:10:02.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.254+0000 7f1964cc5640 1 -- 172.21.15.42:0/3842960059 shutdown_connections 2024-08-30T09:10:02.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:02.254+0000 7f1964cc5640 1 -- 172.21.15.42:0/3842960059 wait complete. 2024-08-30T09:10:03.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:10:02 smithi190 ceph-mon[22001]: pgmap v651: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 3.5 MiB/s wr, 405 op/s 2024-08-30T09:10:03.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:10:02 smithi167 ceph-mon[22063]: pgmap v651: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 3.5 MiB/s wr, 405 op/s 2024-08-30T09:10:03.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:10:02 smithi042 ceph-mon[18652]: pgmap v651: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 3.5 MiB/s wr, 405 op/s 2024-08-30T09:10:05.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:10:04 smithi042 ceph-mon[18652]: pgmap v652: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 4.4 MiB/s wr, 500 op/s 2024-08-30T09:10:05.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:10:04 smithi190 ceph-mon[22001]: pgmap v652: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 4.4 MiB/s wr, 500 op/s 2024-08-30T09:10:05.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:10:04 smithi167 ceph-mon[22063]: pgmap v652: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 4.4 MiB/s wr, 500 op/s 2024-08-30T09:10:05.953 INFO:teuthology.orchestra.run.smithi042.stderr:progress: 560.0 s, 500.8 tps, lat 200.842 ms stddev 228.751 2024-08-30T09:10:06.368 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-08-30T09:10:06.368 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-08-30T09:10:06.915 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T09:10:07.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:10:06 smithi190 ceph-mon[22001]: pgmap v653: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 4.3 MiB/s wr, 462 op/s 2024-08-30T09:10:07.216 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:10:06 smithi042 ceph-mon[18652]: pgmap v653: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 4.3 MiB/s wr, 462 op/s 2024-08-30T09:10:07.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:10:06 smithi167 ceph-mon[22063]: pgmap v653: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 4.3 MiB/s wr, 462 op/s 2024-08-30T09:10:07.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.542+0000 7f2b30467640 1 -- 172.21.15.42:0/3191165999 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2b28077120 msgr2=0x7f2b28075550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:10:07.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.542+0000 7f2b30467640 1 --2- 172.21.15.42:0/3191165999 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2b28077120 0x7f2b28075550 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f2b24009860 tx=0x7f2b2402a380 comp rx=0 tx=0).stop 2024-08-30T09:10:07.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.543+0000 7f2b30467640 1 -- 172.21.15.42:0/3191165999 shutdown_connections 2024-08-30T09:10:07.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.543+0000 7f2b30467640 1 --2- 172.21.15.42:0/3191165999 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2b28075b70 0x7f2b28075f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:07.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.543+0000 7f2b30467640 1 --2- 172.21.15.42:0/3191165999 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2b281141f0 0x7f2b281166c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:07.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.543+0000 7f2b30467640 1 --2- 172.21.15.42:0/3191165999 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2b28077120 0x7f2b28075550 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:07.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.543+0000 7f2b30467640 1 -- 172.21.15.42:0/3191165999 >> 172.21.15.42:0/3191165999 conn(0x7f2b280ff3b0 msgr2=0x7f2b28101800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:10:07.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.543+0000 7f2b30467640 1 -- 172.21.15.42:0/3191165999 shutdown_connections 2024-08-30T09:10:07.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.548+0000 7f2b30467640 1 -- 172.21.15.42:0/3191165999 wait complete. 2024-08-30T09:10:07.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.549+0000 7f2b30467640 1 Processor -- start 2024-08-30T09:10:07.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.549+0000 7f2b30467640 1 -- start start 2024-08-30T09:10:07.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.549+0000 7f2b30467640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2b28075b70 0x7f2b28074a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:07.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.549+0000 7f2b30467640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2b28077120 0x7f2b28071d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:07.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.549+0000 7f2b30467640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2b281141f0 0x7f2b28113a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:07.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.549+0000 7f2b30467640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f2b2810a270 con 0x7f2b281141f0 2024-08-30T09:10:07.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.549+0000 7f2b30467640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f2b2810a0f0 con 0x7f2b28075b70 2024-08-30T09:10:07.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.549+0000 7f2b30467640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f2b2810a3f0 con 0x7f2b28077120 2024-08-30T09:10:07.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.549+0000 7f2b2ea04640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2b281141f0 0x7f2b28113a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:10:07.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.549+0000 7f2b2ea04640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2b281141f0 0x7f2b28113a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:57786/0 (socket says 172.21.15.42:57786) 2024-08-30T09:10:07.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.549+0000 7f2b2ea04640 1 -- 172.21.15.42:0/3308352185 learned_addr learned my addr 172.21.15.42:0/3308352185 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:10:07.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.550+0000 7f2b2e203640 1 --2- 172.21.15.42:0/3308352185 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2b28075b70 0x7f2b28074a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:10:07.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.550+0000 7f2b2ea04640 1 -- 172.21.15.42:0/3308352185 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2b28077120 msgr2=0x7f2b28071d30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-30T09:10:07.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.550+0000 7f2b2ea04640 1 --2- 172.21.15.42:0/3308352185 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2b28077120 0x7f2b28071d30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:07.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.550+0000 7f2b2ea04640 1 -- 172.21.15.42:0/3308352185 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2b28075b70 msgr2=0x7f2b28074a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:10:07.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.550+0000 7f2b2ea04640 1 --2- 172.21.15.42:0/3308352185 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2b28075b70 0x7f2b28074a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:07.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.550+0000 7f2b2ea04640 1 -- 172.21.15.42:0/3308352185 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b28072270 con 0x7f2b281141f0 2024-08-30T09:10:07.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.550+0000 7f2b2ea04640 1 --2- 172.21.15.42:0/3308352185 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2b281141f0 0x7f2b28113a90 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f2b2001fd90 tx=0x7f2b2001d750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:10:07.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.551+0000 7f2b1f7fe640 1 -- 172.21.15.42:0/3308352185 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b2001de60 con 0x7f2b281141f0 2024-08-30T09:10:07.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.551+0000 7f2b1f7fe640 1 -- 172.21.15.42:0/3308352185 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2b20003970 con 0x7f2b281141f0 2024-08-30T09:10:07.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.551+0000 7f2b1f7fe640 1 -- 172.21.15.42:0/3308352185 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b20004380 con 0x7f2b281141f0 2024-08-30T09:10:07.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.551+0000 7f2b30467640 1 -- 172.21.15.42:0/3308352185 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b28072500 con 0x7f2b281141f0 2024-08-30T09:10:07.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.551+0000 7f2b30467640 1 -- 172.21.15.42:0/3308352185 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b280729f0 con 0x7f2b281141f0 2024-08-30T09:10:07.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.555+0000 7f2b369dc740 1 Processor -- start 2024-08-30T09:10:07.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.556+0000 7f2b1f7fe640 1 -- 172.21.15.42:0/3308352185 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7f2b2002e020 con 0x7f2b281141f0 2024-08-30T09:10:07.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.557+0000 7f2b1f7fe640 1 --2- 172.21.15.42:0/3308352185 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f2b140b4d30 0x7f2b140b7240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:07.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.557+0000 7f2b1f7fe640 1 -- 172.21.15.42:0/3308352185 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f2b200f5f50 con 0x7f2b281141f0 2024-08-30T09:10:07.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.558+0000 7f2b2da02640 1 --2- 172.21.15.42:0/3308352185 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f2b140b4d30 0x7f2b140b7240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:10:07.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.558+0000 7f2b369dc740 1 -- start start 2024-08-30T09:10:07.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.558+0000 7f2b369dc740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558b53dff350 0x558b541a5780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:07.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.558+0000 7f2b369dc740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558b53dd6490 0x558b541a5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:07.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.558+0000 7f2b369dc740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558b53dde670 0x558b541a6200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:07.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.558+0000 7f2b369dc740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x558b5411b5f0 con 0x558b53dd6490 2024-08-30T09:10:07.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.558+0000 7f2b369dc740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x558b5410ab80 con 0x558b53dde670 2024-08-30T09:10:07.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.558+0000 7f2b369dc740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x558b5410eb30 con 0x558b53dff350 2024-08-30T09:10:07.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.560+0000 7f2b2ea04640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558b53dde670 0x558b541a6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:10:07.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.560+0000 7f2b2e203640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558b53dff350 0x558b541a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:10:07.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.560+0000 7f2b2e203640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558b53dff350 0x558b541a5780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:43464/0 (socket says 172.21.15.42:43464) 2024-08-30T09:10:07.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.560+0000 7f2b2e203640 1 -- 172.21.15.42:0/2579520987 learned_addr learned my addr 172.21.15.42:0/2579520987 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:10:07.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.560+0000 7f2b2e203640 1 -- 172.21.15.42:0/2579520987 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558b53dde670 msgr2=0x558b541a6200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:10:07.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.560+0000 7f2b2e203640 1 --2- 172.21.15.42:0/2579520987 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558b53dde670 0x558b541a6200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:07.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.560+0000 7f2b2e203640 1 -- 172.21.15.42:0/2579520987 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558b53dd6490 msgr2=0x558b541a5cc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-30T09:10:07.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.560+0000 7f2b2e203640 1 --2- 172.21.15.42:0/2579520987 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558b53dd6490 0x558b541a5cc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:07.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.560+0000 7f2b2e203640 1 -- 172.21.15.42:0/2579520987 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b53e6a930 con 0x558b53dff350 2024-08-30T09:10:07.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.561+0000 7f2b2e203640 1 --2- 172.21.15.42:0/2579520987 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558b53dff350 0x558b541a5780 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f2b24000c00 tx=0x7f2b24009490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=305706e201ae5836 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:10:07.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.562+0000 7f2b1cd9a640 1 -- 172.21.15.42:0/2579520987 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b24040030 con 0x558b53dff350 2024-08-30T09:10:07.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.562+0000 7f2b1cd9a640 1 -- 172.21.15.42:0/2579520987 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2b24038040 con 0x558b53dff350 2024-08-30T09:10:07.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.562+0000 7f2b1cd9a640 1 -- 172.21.15.42:0/2579520987 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b24037070 con 0x558b53dff350 2024-08-30T09:10:07.570 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.567+0000 7f2b369dc740 1 -- 172.21.15.42:0/2579520987 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558b53dff350 msgr2=0x558b541a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:10:07.570 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.567+0000 7f2b369dc740 1 --2- 172.21.15.42:0/2579520987 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558b53dff350 0x558b541a5780 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f2b24000c00 tx=0x7f2b24009490 comp rx=0 tx=0).stop 2024-08-30T09:10:07.570 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.567+0000 7f2b369dc740 1 -- 172.21.15.42:0/2579520987 shutdown_connections 2024-08-30T09:10:07.570 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.567+0000 7f2b369dc740 1 --2- 172.21.15.42:0/2579520987 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558b53dde670 0x558b541a6200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:07.570 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.567+0000 7f2b369dc740 1 --2- 172.21.15.42:0/2579520987 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558b53dd6490 0x558b541a5cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:07.570 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.567+0000 7f2b369dc740 1 --2- 172.21.15.42:0/2579520987 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558b53dff350 0x558b541a5780 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:07.570 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.567+0000 7f2b369dc740 1 -- 172.21.15.42:0/2579520987 >> 172.21.15.42:0/2579520987 conn(0x558b53df3740 msgr2=0x558b541a3670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:10:07.570 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.568+0000 7f2b2da02640 1 --2- 172.21.15.42:0/3308352185 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f2b140b4d30 0x7f2b140b7240 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f2b28115e40 tx=0x7f2b18005b80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:10:07.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.568+0000 7f2b369dc740 1 -- 172.21.15.42:0/2579520987 shutdown_connections 2024-08-30T09:10:07.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.568+0000 7f2b369dc740 1 -- 172.21.15.42:0/2579520987 wait complete. 2024-08-30T09:10:07.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.569+0000 7f2b369dc740 1 -- start start 2024-08-30T09:10:07.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.569+0000 7f2b369dc740 1 Processor -- start 2024-08-30T09:10:07.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.570+0000 7f2b369dc740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558b53dd6490 0x558b541b0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:07.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.570+0000 7f2b369dc740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558b53dde670 0x558b541b1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:07.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.570+0000 7f2b369dc740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558b53dff350 0x558b541bd1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:07.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.570+0000 7f2b369dc740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x558b53f6d890 con 0x558b53dff350 2024-08-30T09:10:07.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.570+0000 7f2b369dc740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x558b54091790 con 0x558b53dd6490 2024-08-30T09:10:07.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.570+0000 7f2b369dc740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x558b541b3f00 con 0x558b53dde670 2024-08-30T09:10:07.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.570+0000 7f2b369dc740 10 client.0 _ll_register_callbacks cb 0x558b53e16ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:10:07.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.570+0000 7f2b2da02640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558b53dde670 0x558b541b1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:10:07.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.570+0000 7f2b2da02640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558b53dde670 0x558b541b1510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:43470/0 (socket says 172.21.15.42:43470) 2024-08-30T09:10:07.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.570+0000 7f2b2da02640 1 -- 172.21.15.42:0/2007750969 learned_addr learned my addr 172.21.15.42:0/2007750969 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:10:07.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.570+0000 7f2b2e203640 1 --2- 172.21.15.42:0/2007750969 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558b53dd6490 0x558b541b0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:10:07.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.570+0000 7f2b2ea04640 1 --2- 172.21.15.42:0/2007750969 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558b53dff350 0x558b541bd1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:10:07.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.570+0000 7f2b2e203640 1 -- 172.21.15.42:0/2007750969 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558b53dde670 msgr2=0x558b541b1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:10:07.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.570+0000 7f2b2e203640 1 --2- 172.21.15.42:0/2007750969 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558b53dde670 0x558b541b1510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:07.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.570+0000 7f2b2e203640 1 -- 172.21.15.42:0/2007750969 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558b53dff350 msgr2=0x558b541bd1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:10:07.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.570+0000 7f2b2e203640 1 --2- 172.21.15.42:0/2007750969 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558b53dff350 0x558b541bd1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:07.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.570+0000 7f2b2e203640 1 -- 172.21.15.42:0/2007750969 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558b540c8f30 con 0x558b53dd6490 2024-08-30T09:10:07.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.571+0000 7f2b2e203640 1 --2- 172.21.15.42:0/2007750969 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558b53dd6490 0x558b541b0f90 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f2b24030460 tx=0x7f2b240045b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1176c85254d872ea server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:10:07.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.571+0000 7f2b11ffb640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-30T09:10:07.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.571+0000 7f2b11ffb640 1 -- 172.21.15.42:0/2007750969 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b2404b020 con 0x558b53dd6490 2024-08-30T09:10:07.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.571+0000 7f2b11ffb640 1 -- 172.21.15.42:0/2007750969 <== mon.2 v2:172.21.15.167:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f2b240035f0 con 0x558b53dd6490 2024-08-30T09:10:07.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.571+0000 7f2b369dc740 1 -- 172.21.15.42:0/2007750969 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2b24038040 con 0x558b53dd6490 2024-08-30T09:10:07.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.571+0000 7f2b11ffb640 1 -- 172.21.15.42:0/2007750969 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f2b24056050 con 0x558b53dd6490 2024-08-30T09:10:07.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.571+0000 7f2b369dc740 10 client.26810 fetch_fsmap learned FSMap version 39 2024-08-30T09:10:07.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.571+0000 7f2b369dc740 1 -- 172.21.15.42:0/2007750969 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x558b541bfd60 con 0x558b53dd6490 2024-08-30T09:10:07.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.572+0000 7f2b11ffb640 1 -- 172.21.15.42:0/2007750969 <== mon.2 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2b24038040 con 0x558b53dd6490 2024-08-30T09:10:07.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.572+0000 7f2b11ffb640 1 -- 172.21.15.42:0/2007750969 <== mon.2 v2:172.21.15.167:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7f2b240325c0 con 0x558b53dd6490 2024-08-30T09:10:07.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.572+0000 7f2b369dc740 10 client.26810 fetch_fsmap finished waiting for FSMap version 39 2024-08-30T09:10:07.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.573+0000 7f2b369dc740 10 client.26810 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-30T09:10:07.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.573+0000 7f2b369dc740 20 client.26810 populate_metadata read hostname 'smithi042' 2024-08-30T09:10:07.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.573+0000 7f2b369dc740 1 --2- 172.21.15.42:0/2007750969 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x558b53dbdcd0 0x558b541c30c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:07.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.573+0000 7f2b369dc740 4 client.26810 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-30T09:10:07.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.573+0000 7f2b369dc740 1 -- 172.21.15.42:0/2007750969 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558b5411d540 con 0x558b53dbdcd0 2024-08-30T09:10:07.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.573+0000 7f2b2da02640 1 --2- 172.21.15.42:0/2007750969 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x558b53dbdcd0 0x558b541c30c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:10:07.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.573+0000 7f2b11ffb640 10 client.26810 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:10:07.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.573+0000 7f2b2da02640 1 --2- 172.21.15.42:0/2007750969 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x558b53dbdcd0 0x558b541c30c0 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=109dd31490a2d41f server_cookie=75e977600d9462f1 in_seq=0 out_seq=0 2024-08-30T09:10:07.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.574+0000 7f2b11ffb640 1 -- 172.21.15.42:0/2007750969 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x558b5411d540 con 0x558b53dbdcd0 2024-08-30T09:10:07.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.574+0000 7f2b11ffb640 10 client.26810 handle_command_reply: tid=0 2024-08-30T09:10:07.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.574+0000 7f2b369dc740 1 client.26810 shutdown 2024-08-30T09:10:07.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.575+0000 7f2b369dc740 1 -- 172.21.15.42:0/2007750969 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558b53dd6490 msgr2=0x558b541b0f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:10:07.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.575+0000 7f2b369dc740 1 --2- 172.21.15.42:0/2007750969 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558b53dd6490 0x558b541b0f90 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f2b24030460 tx=0x7f2b240045b0 comp rx=0 tx=0).stop 2024-08-30T09:10:07.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.575+0000 7f2b369dc740 1 -- 172.21.15.42:0/2007750969 shutdown_connections 2024-08-30T09:10:07.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.575+0000 7f2b369dc740 1 --2- 172.21.15.42:0/2007750969 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x558b53dbdcd0 0x558b541c30c0 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:07.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.575+0000 7f2b369dc740 1 --2- 172.21.15.42:0/2007750969 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558b53dde670 0x558b541b1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:07.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.575+0000 7f2b369dc740 1 --2- 172.21.15.42:0/2007750969 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558b53dff350 0x558b541bd1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:07.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.575+0000 7f2b369dc740 1 --2- 172.21.15.42:0/2007750969 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558b53dd6490 0x558b541b0f90 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:07.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.575+0000 7f2b11ffb640 0 client.26810 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:10:07.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.575+0000 7f2b369dc740 1 -- 172.21.15.42:0/2007750969 >> 172.21.15.42:0/2007750969 conn(0x558b53df3740 msgr2=0x558b541a8910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:10:07.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.575+0000 7f2b369dc740 1 -- 172.21.15.42:0/2007750969 shutdown_connections 2024-08-30T09:10:07.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.575+0000 7f2b369dc740 1 -- 172.21.15.42:0/2007750969 wait complete. 2024-08-30T09:10:07.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.576+0000 7f2b369dc740 20 client.26810 trim_cache size 0 max 16384 2024-08-30T09:10:07.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.599+0000 7f2b369dc740 1 Processor -- start 2024-08-30T09:10:07.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.599+0000 7f2b369dc740 1 -- start start 2024-08-30T09:10:07.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.599+0000 7f2b369dc740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558b53dd6490 0x558b541a8ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:07.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.599+0000 7f2b369dc740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558b53dde670 0x558b541a6ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:07.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.599+0000 7f2b369dc740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558b53dff350 0x558b541bd160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:07.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.599+0000 7f2b369dc740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f2b281b9790 con 0x558b53dd6490 2024-08-30T09:10:07.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.599+0000 7f2b369dc740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f2b281b7000 con 0x558b53dff350 2024-08-30T09:10:07.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.599+0000 7f2b369dc740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f2b281b4e50 con 0x558b53dde670 2024-08-30T09:10:07.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.599+0000 7f2b2e203640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558b53dd6490 0x558b541a8ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:10:07.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.599+0000 7f2b2da02640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558b53dde670 0x558b541a6ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:10:07.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.599+0000 7f2b2e203640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558b53dd6490 0x558b541a8ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:57806/0 (socket says 172.21.15.42:57806) 2024-08-30T09:10:07.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.599+0000 7f2b2e203640 1 -- 172.21.15.42:0/515741097 learned_addr learned my addr 172.21.15.42:0/515741097 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:10:07.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.600+0000 7f2b2ea04640 1 --2- 172.21.15.42:0/515741097 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558b53dff350 0x558b541bd160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:10:07.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.601+0000 7f2b2ea04640 1 -- 172.21.15.42:0/515741097 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558b53dde670 msgr2=0x558b541a6ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:10:07.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.601+0000 7f2b2ea04640 1 --2- 172.21.15.42:0/515741097 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558b53dde670 0x558b541a6ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:07.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.601+0000 7f2b2ea04640 1 -- 172.21.15.42:0/515741097 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558b53dd6490 msgr2=0x558b541a8ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:10:07.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.601+0000 7f2b2ea04640 1 --2- 172.21.15.42:0/515741097 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558b53dd6490 0x558b541a8ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:07.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.601+0000 7f2b2ea04640 1 -- 172.21.15.42:0/515741097 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b541b2030 con 0x558b53dff350 2024-08-30T09:10:07.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.601+0000 7f2b2ea04640 1 --2- 172.21.15.42:0/515741097 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558b53dff350 0x558b541bd160 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f2b200b9800 tx=0x7f2b200df040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=29d7ac9975ab1f17 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:10:07.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.602+0000 7f2b13fff640 1 -- 172.21.15.42:0/515741097 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b2002b040 con 0x558b53dff350 2024-08-30T09:10:07.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.602+0000 7f2b13fff640 1 -- 172.21.15.42:0/515741097 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2b200c5040 con 0x558b53dff350 2024-08-30T09:10:07.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.602+0000 7f2b13fff640 1 -- 172.21.15.42:0/515741097 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b200b8820 con 0x558b53dff350 2024-08-30T09:10:07.605 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.603+0000 7f2b369dc740 1 -- 172.21.15.42:0/515741097 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558b53dff350 msgr2=0x558b541bd160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:10:07.605 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.603+0000 7f2b369dc740 1 --2- 172.21.15.42:0/515741097 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558b53dff350 0x558b541bd160 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f2b200b9800 tx=0x7f2b200df040 comp rx=0 tx=0).stop 2024-08-30T09:10:07.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.603+0000 7f2b369dc740 1 -- 172.21.15.42:0/515741097 shutdown_connections 2024-08-30T09:10:07.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.603+0000 7f2b369dc740 1 --2- 172.21.15.42:0/515741097 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558b53dde670 0x558b541a6ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:07.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.603+0000 7f2b369dc740 1 --2- 172.21.15.42:0/515741097 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558b53dff350 0x558b541bd160 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:07.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.603+0000 7f2b369dc740 1 --2- 172.21.15.42:0/515741097 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558b53dd6490 0x558b541a8ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:07.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.603+0000 7f2b369dc740 1 -- 172.21.15.42:0/515741097 >> 172.21.15.42:0/515741097 conn(0x558b53df3740 msgr2=0x558b541a67b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:10:07.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.603+0000 7f2b369dc740 1 -- 172.21.15.42:0/515741097 shutdown_connections 2024-08-30T09:10:07.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.603+0000 7f2b369dc740 1 -- 172.21.15.42:0/515741097 wait complete. 2024-08-30T09:10:07.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.604+0000 7f2b369dc740 1 -- start start 2024-08-30T09:10:07.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.604+0000 7f2b369dc740 1 Processor -- start 2024-08-30T09:10:07.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.604+0000 7f2b369dc740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558b53dd6490 0x558b542219f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:07.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.604+0000 7f2b369dc740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558b53dde670 0x558b54221f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:07.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.604+0000 7f2b369dc740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558b53dff350 0x558b54223a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:07.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.604+0000 7f2b369dc740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f2b281b54c0 con 0x558b53dff350 2024-08-30T09:10:07.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.605+0000 7f2b369dc740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f2b281b6930 con 0x558b53dde670 2024-08-30T09:10:07.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.605+0000 7f2b369dc740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f2b281b5b90 con 0x558b53dd6490 2024-08-30T09:10:07.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.605+0000 7f2b2e203640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558b53dd6490 0x558b542219f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:10:07.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.605+0000 7f2b369dc740 10 client.0 _ll_register_callbacks cb 0x7f2b20003ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:10:07.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.605+0000 7f2b2e203640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558b53dd6490 0x558b542219f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:43494/0 (socket says 172.21.15.42:43494) 2024-08-30T09:10:07.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.605+0000 7f2b2e203640 1 -- 172.21.15.42:0/979781568 learned_addr learned my addr 172.21.15.42:0/979781568 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:10:07.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.605+0000 7f2b2da02640 1 --2- 172.21.15.42:0/979781568 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558b53dde670 0x558b54221f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:10:07.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.605+0000 7f2b2ea04640 1 --2- 172.21.15.42:0/979781568 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558b53dff350 0x558b54223a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:10:07.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.605+0000 7f2b2e203640 1 -- 172.21.15.42:0/979781568 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558b53dde670 msgr2=0x558b54221f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:10:07.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.605+0000 7f2b2e203640 1 --2- 172.21.15.42:0/979781568 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558b53dde670 0x558b54221f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:07.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.605+0000 7f2b2e203640 1 -- 172.21.15.42:0/979781568 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558b53dff350 msgr2=0x558b54223a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:10:07.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.605+0000 7f2b2e203640 1 --2- 172.21.15.42:0/979781568 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558b53dff350 0x558b54223a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:07.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.605+0000 7f2b2e203640 1 -- 172.21.15.42:0/979781568 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558b542240f0 con 0x558b53dd6490 2024-08-30T09:10:07.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.605+0000 7f2b2e203640 1 --2- 172.21.15.42:0/979781568 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558b53dd6490 0x558b542219f0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f2b240329b0 tx=0x7f2b240041e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1aaa2a2648c23fa2 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:10:07.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.605+0000 7f2b11ffb640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-08-30T09:10:07.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.606+0000 7f2b11ffb640 1 -- 172.21.15.42:0/979781568 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b24040070 con 0x558b53dd6490 2024-08-30T09:10:07.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.606+0000 7f2b11ffb640 1 -- 172.21.15.42:0/979781568 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f2b24032440 con 0x558b53dd6490 2024-08-30T09:10:07.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.606+0000 7f2b369dc740 1 -- 172.21.15.42:0/979781568 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2b200c5040 con 0x558b53dd6490 2024-08-30T09:10:07.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.606+0000 7f2b11ffb640 1 -- 172.21.15.42:0/979781568 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f2b2404b040 con 0x558b53dd6490 2024-08-30T09:10:07.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.606+0000 7f2b369dc740 10 client.26791 fetch_fsmap learned FSMap version 39 2024-08-30T09:10:07.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.606+0000 7f2b369dc740 1 -- 172.21.15.42:0/979781568 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x558b5422e760 con 0x558b53dd6490 2024-08-30T09:10:07.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.606+0000 7f2b11ffb640 1 -- 172.21.15.42:0/979781568 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2b200c5040 con 0x558b53dd6490 2024-08-30T09:10:07.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.606+0000 7f2b11ffb640 1 -- 172.21.15.42:0/979781568 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7f2b240046c0 con 0x558b53dd6490 2024-08-30T09:10:07.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.606+0000 7f2b369dc740 10 client.26791 fetch_fsmap finished waiting for FSMap version 39 2024-08-30T09:10:07.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.606+0000 7f2b369dc740 10 client.26791 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-30T09:10:07.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.606+0000 7f2b369dc740 20 client.26791 populate_metadata read hostname 'smithi042' 2024-08-30T09:10:07.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.606+0000 7f2b369dc740 1 --2- 172.21.15.42:0/979781568 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x558b53d93080 0x558b54230f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:07.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.606+0000 7f2b369dc740 4 client.26791 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-08-30T09:10:07.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.606+0000 7f2b369dc740 1 -- 172.21.15.42:0/979781568 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x558b54118fe0 con 0x558b53d93080 2024-08-30T09:10:07.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.607+0000 7f2b2da02640 1 --2- 172.21.15.42:0/979781568 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x558b53d93080 0x558b54230f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:10:07.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.607+0000 7f2b11ffb640 10 client.26791 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:10:07.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.607+0000 7f2b2da02640 1 --2- 172.21.15.42:0/979781568 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x558b53d93080 0x558b54230f00 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=23726355c7e6e3e server_cookie=ef9e2d0434c5d714 in_seq=0 out_seq=0 2024-08-30T09:10:07.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.608+0000 7f2b11ffb640 1 -- 172.21.15.42:0/979781568 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x558b54118fe0 con 0x558b53d93080 2024-08-30T09:10:07.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.608+0000 7f2b11ffb640 10 client.26791 handle_command_reply: tid=0 2024-08-30T09:10:07.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.609+0000 7f2b369dc740 1 client.26791 shutdown 2024-08-30T09:10:07.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.609+0000 7f2b369dc740 1 -- 172.21.15.42:0/979781568 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558b53dd6490 msgr2=0x558b542219f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:10:07.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.609+0000 7f2b369dc740 1 --2- 172.21.15.42:0/979781568 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558b53dd6490 0x558b542219f0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f2b240329b0 tx=0x7f2b240041e0 comp rx=0 tx=0).stop 2024-08-30T09:10:07.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.609+0000 7f2b369dc740 1 -- 172.21.15.42:0/979781568 shutdown_connections 2024-08-30T09:10:07.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.609+0000 7f2b369dc740 1 --2- 172.21.15.42:0/979781568 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x558b53d93080 0x558b54230f00 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:07.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.609+0000 7f2b369dc740 1 --2- 172.21.15.42:0/979781568 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558b53dff350 0x558b54223a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:07.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.609+0000 7f2b369dc740 1 --2- 172.21.15.42:0/979781568 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558b53dde670 0x558b54221f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:07.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.609+0000 7f2b369dc740 1 --2- 172.21.15.42:0/979781568 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558b53dd6490 0x558b542219f0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:07.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.609+0000 7f2b369dc740 1 -- 172.21.15.42:0/979781568 >> 172.21.15.42:0/979781568 conn(0x558b53df3740 msgr2=0x558b541afba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:10:07.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.609+0000 7f2b11ffb640 0 client.26791 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:10:07.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.609+0000 7f2b369dc740 1 -- 172.21.15.42:0/979781568 shutdown_connections 2024-08-30T09:10:07.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.609+0000 7f2b369dc740 1 -- 172.21.15.42:0/979781568 wait complete. 2024-08-30T09:10:07.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.609+0000 7f2b369dc740 20 client.26791 trim_cache size 0 max 16384 2024-08-30T09:10:07.615 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-08-30T09:10:07.615 INFO:teuthology.orchestra.run.smithi042.stdout: "return_code": 0, 2024-08-30T09:10:07.615 INFO:teuthology.orchestra.run.smithi042.stdout: "scrub_tag": "a7210f9a-046b-4188-953c-819b9f1863a9", 2024-08-30T09:10:07.615 INFO:teuthology.orchestra.run.smithi042.stdout: "mode": "asynchronous" 2024-08-30T09:10:07.615 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-08-30T09:10:07.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.610+0000 7f2b30467640 1 -- 172.21.15.42:0/3308352185 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f2b140b4d30 msgr2=0x7f2b140b7240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:10:07.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.610+0000 7f2b30467640 1 --2- 172.21.15.42:0/3308352185 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f2b140b4d30 0x7f2b140b7240 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f2b28115e40 tx=0x7f2b18005b80 comp rx=0 tx=0).stop 2024-08-30T09:10:07.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.610+0000 7f2b30467640 1 -- 172.21.15.42:0/3308352185 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2b281141f0 msgr2=0x7f2b28113a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:10:07.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.610+0000 7f2b30467640 1 --2- 172.21.15.42:0/3308352185 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2b281141f0 0x7f2b28113a90 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f2b2001fd90 tx=0x7f2b2001d750 comp rx=0 tx=0).stop 2024-08-30T09:10:07.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.610+0000 7f2b30467640 1 -- 172.21.15.42:0/3308352185 shutdown_connections 2024-08-30T09:10:07.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.610+0000 7f2b30467640 1 --2- 172.21.15.42:0/3308352185 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f2b140b4d30 0x7f2b140b7240 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:07.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.610+0000 7f2b30467640 1 --2- 172.21.15.42:0/3308352185 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2b28077120 0x7f2b28071d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:07.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.610+0000 7f2b30467640 1 --2- 172.21.15.42:0/3308352185 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2b281141f0 0x7f2b28113a90 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:07.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.610+0000 7f2b30467640 1 --2- 172.21.15.42:0/3308352185 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2b28075b70 0x7f2b28074a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:07.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.610+0000 7f2b30467640 1 -- 172.21.15.42:0/3308352185 >> 172.21.15.42:0/3308352185 conn(0x7f2b280ff3b0 msgr2=0x7f2b28103350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:10:07.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.610+0000 7f2b30467640 1 -- 172.21.15.42:0/3308352185 shutdown_connections 2024-08-30T09:10:07.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:07.610+0000 7f2b30467640 1 -- 172.21.15.42:0/3308352185 wait complete. 2024-08-30T09:10:08.204 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:10:07 smithi042 ceph-mon[18652]: scrub queued for path: / 2024-08-30T09:10:08.205 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:10:07 smithi042 ceph-mon[18652]: scrub summary: idle+waiting paths [/] 2024-08-30T09:10:08.205 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:10:07 smithi042 ceph-mon[18652]: scrub summary: active paths [/] 2024-08-30T09:10:08.205 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:10:07 smithi042 ceph-mon[18652]: scrub summary: active paths [/] 2024-08-30T09:10:08.205 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:10:07 smithi042 ceph-mon[18652]: scrub summary: idle+waiting paths [/] 2024-08-30T09:10:08.205 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:10:07 smithi042 ceph-mon[18652]: scrub summary: active paths [/] 2024-08-30T09:10:08.205 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:10:07 smithi042 ceph-mon[18652]: scrub summary: idle+waiting paths [/] 2024-08-30T09:10:08.205 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:10:07 smithi042 ceph-mon[18652]: scrub summary: active paths [/] 2024-08-30T09:10:08.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:10:07 smithi167 ceph-mon[22063]: scrub queued for path: / 2024-08-30T09:10:08.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:10:07 smithi167 ceph-mon[22063]: scrub summary: idle+waiting paths [/] 2024-08-30T09:10:08.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:10:07 smithi167 ceph-mon[22063]: scrub summary: active paths [/] 2024-08-30T09:10:08.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:10:07 smithi167 ceph-mon[22063]: scrub summary: active paths [/] 2024-08-30T09:10:08.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:10:07 smithi167 ceph-mon[22063]: scrub summary: idle+waiting paths [/] 2024-08-30T09:10:08.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:10:07 smithi167 ceph-mon[22063]: scrub summary: active paths [/] 2024-08-30T09:10:08.328 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:10:07 smithi167 ceph-mon[22063]: scrub summary: idle+waiting paths [/] 2024-08-30T09:10:08.328 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:10:07 smithi167 ceph-mon[22063]: scrub summary: active paths [/] 2024-08-30T09:10:08.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:10:07 smithi190 ceph-mon[22001]: scrub queued for path: / 2024-08-30T09:10:08.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:10:07 smithi190 ceph-mon[22001]: scrub summary: idle+waiting paths [/] 2024-08-30T09:10:08.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:10:07 smithi190 ceph-mon[22001]: scrub summary: active paths [/] 2024-08-30T09:10:08.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:10:07 smithi190 ceph-mon[22001]: scrub summary: active paths [/] 2024-08-30T09:10:08.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:10:07 smithi190 ceph-mon[22001]: scrub summary: idle+waiting paths [/] 2024-08-30T09:10:08.437 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:10:07 smithi190 ceph-mon[22001]: scrub summary: active paths [/] 2024-08-30T09:10:08.437 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:10:07 smithi190 ceph-mon[22001]: scrub summary: idle+waiting paths [/] 2024-08-30T09:10:08.437 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:10:07 smithi190 ceph-mon[22001]: scrub summary: active paths [/] 2024-08-30T09:10:08.592 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-30T09:10:09.022 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.019+0000 7f970551e640 1 -- 172.21.15.42:0/3021899262 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f970010cac0 msgr2=0x7f970010eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:10:09.022 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.019+0000 7f970551e640 1 --2- 172.21.15.42:0/3021899262 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f970010cac0 0x7f970010eee0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f96f0009900 tx=0x7f96f002a400 comp rx=0 tx=0).stop 2024-08-30T09:10:09.030 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.027+0000 7f970551e640 1 -- 172.21.15.42:0/3021899262 shutdown_connections 2024-08-30T09:10:09.030 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.027+0000 7f970551e640 1 --2- 172.21.15.42:0/3021899262 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f970010f5d0 0x7f970010fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:09.030 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.027+0000 7f970551e640 1 --2- 172.21.15.42:0/3021899262 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f970010cac0 0x7f970010eee0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:09.030 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.027+0000 7f970551e640 1 --2- 172.21.15.42:0/3021899262 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f970010a160 0x7f970010c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:09.030 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.027+0000 7f970551e640 1 -- 172.21.15.42:0/3021899262 >> 172.21.15.42:0/3021899262 conn(0x7f97000703e0 msgr2=0x7f9700072890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:10:09.034 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.033+0000 7f970551e640 1 -- 172.21.15.42:0/3021899262 shutdown_connections 2024-08-30T09:10:09.039 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.038+0000 7f970551e640 1 -- 172.21.15.42:0/3021899262 wait complete. 2024-08-30T09:10:09.041 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.039+0000 7f970551e640 1 Processor -- start 2024-08-30T09:10:09.041 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.039+0000 7f970551e640 1 -- start start 2024-08-30T09:10:09.042 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.040+0000 7f970551e640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f970010a160 0x7f97001a8940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:09.042 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.040+0000 7f970551e640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f970010cac0 0x7f97001a8fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:09.042 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.040+0000 7f970551e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f970010f5d0 0x7f97001a9b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:09.042 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.040+0000 7f970551e640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f970011dd50 con 0x7f970010cac0 2024-08-30T09:10:09.042 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.040+0000 7f970551e640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f970011dbd0 con 0x7f970010a160 2024-08-30T09:10:09.043 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.040+0000 7f970551e640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f970011ded0 con 0x7f970010f5d0 2024-08-30T09:10:09.043 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.040+0000 7f9704d1d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f970010f5d0 0x7f97001a9b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:10:09.043 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.040+0000 7f9704d1d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f970010f5d0 0x7f97001a9b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:48670/0 (socket says 172.21.15.42:48670) 2024-08-30T09:10:09.043 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.040+0000 7f9704d1d640 1 -- 172.21.15.42:0/775250684 learned_addr learned my addr 172.21.15.42:0/775250684 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:10:09.043 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.041+0000 7f96fffff640 1 --2- 172.21.15.42:0/775250684 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f970010a160 0x7f97001a8940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:10:09.048 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.045+0000 7f96fffff640 1 -- 172.21.15.42:0/775250684 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f970010f5d0 msgr2=0x7f97001a9b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:10:09.048 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.045+0000 7f96fffff640 1 --2- 172.21.15.42:0/775250684 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f970010f5d0 0x7f97001a9b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:09.048 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.045+0000 7f96fffff640 1 -- 172.21.15.42:0/775250684 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f970010cac0 msgr2=0x7f97001a8fb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-30T09:10:09.048 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.045+0000 7f96fffff640 1 --2- 172.21.15.42:0/775250684 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f970010cac0 0x7f97001a8fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:09.048 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.045+0000 7f96fffff640 1 -- 172.21.15.42:0/775250684 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97001aa380 con 0x7f970010a160 2024-08-30T09:10:09.048 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.046+0000 7f9704d1d640 1 --2- 172.21.15.42:0/775250684 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f970010f5d0 0x7f97001a9b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-30T09:10:09.053 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.051+0000 7f96fffff640 1 --2- 172.21.15.42:0/775250684 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f970010a160 0x7f97001a8940 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f96f400c970 tx=0x7f96f400ce30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:10:09.055 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.053+0000 7f96fd7fa640 1 -- 172.21.15.42:0/775250684 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96f4007bb0 con 0x7f970010a160 2024-08-30T09:10:09.055 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.053+0000 7f96fd7fa640 1 -- 172.21.15.42:0/775250684 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f96f4007d50 con 0x7f970010a160 2024-08-30T09:10:09.055 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.053+0000 7f96fd7fa640 1 -- 172.21.15.42:0/775250684 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96f4004df0 con 0x7f970010a160 2024-08-30T09:10:09.059 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.054+0000 7f970551e640 1 -- 172.21.15.42:0/775250684 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97001aa670 con 0x7f970010a160 2024-08-30T09:10:09.060 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.054+0000 7f970551e640 1 -- 172.21.15.42:0/775250684 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97001b0d60 con 0x7f970010a160 2024-08-30T09:10:09.064 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.059+0000 7f970d7cd740 1 Processor -- start 2024-08-30T09:10:09.065 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.061+0000 7f970d7cd740 1 -- start start 2024-08-30T09:10:09.065 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.061+0000 7f970d7cd740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563d0c4511b0 0x563d0c7f86e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:09.065 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.061+0000 7f970d7cd740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563d0c4282f0 0x563d0c7f8d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:09.065 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.061+0000 7f970d7cd740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563d0c4304d0 0x563d0c7f71e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:09.065 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.061+0000 7f970d7cd740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x563d0c76a000 con 0x563d0c4511b0 2024-08-30T09:10:09.065 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.061+0000 7f970d7cd740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x563d0c759590 con 0x563d0c4304d0 2024-08-30T09:10:09.065 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.061+0000 7f970d7cd740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x563d0c75d540 con 0x563d0c4282f0 2024-08-30T09:10:09.065 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.062+0000 7f9704d1d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563d0c4511b0 0x563d0c7f86e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:10:09.065 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.062+0000 7f9704d1d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563d0c4511b0 0x563d0c7f86e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:39410/0 (socket says 172.21.15.42:39410) 2024-08-30T09:10:09.065 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.062+0000 7f9704d1d640 1 -- 172.21.15.42:0/1874476572 learned_addr learned my addr 172.21.15.42:0/1874476572 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:10:09.065 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.062+0000 7f96fd7fa640 1 -- 172.21.15.42:0/775250684 <== mon.2 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7f96f4004fb0 con 0x7f970010a160 2024-08-30T09:10:09.066 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.063+0000 7f9704d1d640 1 -- 172.21.15.42:0/1874476572 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563d0c4282f0 msgr2=0x563d0c7f8d50 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-30T09:10:09.066 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.063+0000 7f9704d1d640 1 --2- 172.21.15.42:0/1874476572 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563d0c4282f0 0x563d0c7f8d50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:09.066 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.063+0000 7f9704d1d640 1 -- 172.21.15.42:0/1874476572 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563d0c4304d0 msgr2=0x563d0c7f71e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-30T09:10:09.066 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.063+0000 7f9704d1d640 1 --2- 172.21.15.42:0/1874476572 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563d0c4304d0 0x563d0c7f71e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:09.066 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.063+0000 7f9704d1d640 1 -- 172.21.15.42:0/1874476572 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d0c4bc790 con 0x563d0c4511b0 2024-08-30T09:10:09.066 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.063+0000 7f9704d1d640 1 --2- 172.21.15.42:0/1874476572 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563d0c4511b0 0x563d0c7f86e0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f96f800f710 tx=0x7f96f800fbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=293c7062d8fb9191 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:10:09.066 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.064+0000 7f96fd7fa640 1 --2- 172.21.15.42:0/775250684 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f96e80b4d40 0x7f96e80b7250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:09.067 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.064+0000 7f96fd7fa640 1 -- 172.21.15.42:0/775250684 <== mon.2 v2:172.21.15.167:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f96f40e50d0 con 0x7f970010a160 2024-08-30T09:10:09.067 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.064+0000 7f96e67fc640 1 -- 172.21.15.42:0/1874476572 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96f8003b70 con 0x563d0c4511b0 2024-08-30T09:10:09.067 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.064+0000 7f96e67fc640 1 -- 172.21.15.42:0/1874476572 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f96f800ddd0 con 0x563d0c4511b0 2024-08-30T09:10:09.067 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.064+0000 7f96e67fc640 1 -- 172.21.15.42:0/1874476572 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96f8010430 con 0x563d0c4511b0 2024-08-30T09:10:09.071 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.069+0000 7f970d7cd740 1 -- 172.21.15.42:0/1874476572 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563d0c4511b0 msgr2=0x563d0c7f86e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:10:09.071 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.069+0000 7f970d7cd740 1 --2- 172.21.15.42:0/1874476572 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563d0c4511b0 0x563d0c7f86e0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f96f800f710 tx=0x7f96f800fbd0 comp rx=0 tx=0).stop 2024-08-30T09:10:09.072 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.070+0000 7f96fffff640 1 --2- 172.21.15.42:0/775250684 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f96e80b4d40 0x7f96e80b7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:10:09.075 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.074+0000 7f970d7cd740 1 -- 172.21.15.42:0/1874476572 shutdown_connections 2024-08-30T09:10:09.076 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.074+0000 7f970d7cd740 1 --2- 172.21.15.42:0/1874476572 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563d0c4282f0 0x563d0c7f8d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:09.076 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.074+0000 7f970d7cd740 1 --2- 172.21.15.42:0/1874476572 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563d0c4304d0 0x563d0c7f71e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:09.076 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.074+0000 7f970d7cd740 1 --2- 172.21.15.42:0/1874476572 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563d0c4511b0 0x563d0c7f86e0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:09.076 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.074+0000 7f970d7cd740 1 -- 172.21.15.42:0/1874476572 >> 172.21.15.42:0/1874476572 conn(0x563d0c445300 msgr2=0x563d0c7f2e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:10:09.080 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.078+0000 7f96fffff640 1 --2- 172.21.15.42:0/775250684 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f96e80b4d40 0x7f96e80b7250 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f96f40ab410 tx=0x7f96f400cf60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:10:09.082 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.080+0000 7f970d7cd740 1 -- 172.21.15.42:0/1874476572 shutdown_connections 2024-08-30T09:10:09.086 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.084+0000 7f970d7cd740 1 -- 172.21.15.42:0/1874476572 wait complete. 2024-08-30T09:10:09.086 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.084+0000 7f970d7cd740 1 -- start start 2024-08-30T09:10:09.086 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.084+0000 7f970d7cd740 1 Processor -- start 2024-08-30T09:10:09.087 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.085+0000 7f970d7cd740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563d0c4282f0 0x563d0c811e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:09.088 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.085+0000 7f970d7cd740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563d0c4304d0 0x563d0c8124c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:09.088 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.085+0000 7f970d7cd740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563d0c4511b0 0x563d0c813030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:09.088 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.085+0000 7f970d7cd740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x563d0c692ab0 con 0x563d0c4511b0 2024-08-30T09:10:09.088 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.085+0000 7f970d7cd740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x563d0c6e0680 con 0x563d0c4304d0 2024-08-30T09:10:09.088 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.085+0000 7f970d7cd740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x563d0c7f7d00 con 0x563d0c4282f0 2024-08-30T09:10:09.088 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.085+0000 7f970d7cd740 10 client.0 _ll_register_callbacks cb 0x563d0c468a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:10:09.088 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.085+0000 7f9704d1d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563d0c4282f0 0x563d0c811e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:10:09.088 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.085+0000 7f9704d1d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563d0c4282f0 0x563d0c811e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:48682/0 (socket says 172.21.15.42:48682) 2024-08-30T09:10:09.088 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.085+0000 7f9704d1d640 1 -- 172.21.15.42:0/3959668107 learned_addr learned my addr 172.21.15.42:0/3959668107 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:10:09.088 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.086+0000 7f9704d1d640 1 --2- 172.21.15.42:0/3959668107 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563d0c4511b0 0x563d0c813030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:10:09.089 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.087+0000 7f9704d1d640 1 -- 172.21.15.42:0/3959668107 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563d0c4304d0 msgr2=0x563d0c8124c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-30T09:10:09.089 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.087+0000 7f9704d1d640 1 --2- 172.21.15.42:0/3959668107 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563d0c4304d0 0x563d0c8124c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:09.089 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.087+0000 7f9704d1d640 1 -- 172.21.15.42:0/3959668107 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563d0c4511b0 msgr2=0x563d0c813030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:10:09.089 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.087+0000 7f9704d1d640 1 --2- 172.21.15.42:0/3959668107 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563d0c4511b0 0x563d0c813030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:09.089 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.087+0000 7f9704d1d640 1 -- 172.21.15.42:0/3959668107 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d0c535620 con 0x563d0c4282f0 2024-08-30T09:10:09.093 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.091+0000 7f9704d1d640 1 --2- 172.21.15.42:0/3959668107 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563d0c4282f0 0x563d0c811e00 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f96f8007fd0 tx=0x7f96f8007f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=365dfe5e50569de2 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:10:09.096 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.092+0000 7f96df7fe640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-08-30T09:10:09.098 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.096+0000 7f96df7fe640 1 -- 172.21.15.42:0/3959668107 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96f802b060 con 0x563d0c4282f0 2024-08-30T09:10:09.098 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.096+0000 7f970d7cd740 1 -- 172.21.15.42:0/3959668107 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f96f800ddd0 con 0x563d0c4282f0 2024-08-30T09:10:09.098 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.096+0000 7f96df7fe640 1 -- 172.21.15.42:0/3959668107 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f96f8010e00 con 0x563d0c4282f0 2024-08-30T09:10:09.103 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.102+0000 7f96df7fe640 1 -- 172.21.15.42:0/3959668107 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f96f8003970 con 0x563d0c4282f0 2024-08-30T09:10:09.106 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.102+0000 7f970d7cd740 10 client.26797 fetch_fsmap learned FSMap version 39 2024-08-30T09:10:09.106 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.102+0000 7f970d7cd740 1 -- 172.21.15.42:0/3959668107 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x563d0c81bfb0 con 0x563d0c4282f0 2024-08-30T09:10:09.106 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.103+0000 7f96df7fe640 1 -- 172.21.15.42:0/3959668107 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f96f800ddd0 con 0x563d0c4282f0 2024-08-30T09:10:09.106 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.104+0000 7f96df7fe640 1 -- 172.21.15.42:0/3959668107 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7f96f8018820 con 0x563d0c4282f0 2024-08-30T09:10:09.107 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.105+0000 7f970d7cd740 10 client.26797 fetch_fsmap finished waiting for FSMap version 39 2024-08-30T09:10:09.107 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.105+0000 7f970d7cd740 10 client.26797 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-30T09:10:09.107 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.105+0000 7f970d7cd740 20 client.26797 populate_metadata read hostname 'smithi042' 2024-08-30T09:10:09.107 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.105+0000 7f970d7cd740 1 --2- 172.21.15.42:0/3959668107 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x563d0c40fd80 0x563d0c81faf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:09.107 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.105+0000 7f970d7cd740 4 client.26797 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-30T09:10:09.107 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.105+0000 7f970d7cd740 1 -- 172.21.15.42:0/3959668107 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563d0c76bf50 con 0x563d0c40fd80 2024-08-30T09:10:09.108 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.105+0000 7f96ff7fe640 1 --2- 172.21.15.42:0/3959668107 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x563d0c40fd80 0x563d0c81faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:10:09.108 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.106+0000 7f96ff7fe640 1 --2- 172.21.15.42:0/3959668107 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x563d0c40fd80 0x563d0c81faf0 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa16ddf1fe32d09e server_cookie=1a2f888c649dbaa0 in_seq=0 out_seq=0 2024-08-30T09:10:09.108 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.106+0000 7f96df7fe640 10 client.26797 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:10:09.109 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.107+0000 7f96df7fe640 1 -- 172.21.15.42:0/3959668107 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x563d0c76bf50 con 0x563d0c40fd80 2024-08-30T09:10:09.109 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.107+0000 7f96df7fe640 10 client.26797 handle_command_reply: tid=0 2024-08-30T09:10:09.109 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.107+0000 7f970d7cd740 1 client.26797 shutdown 2024-08-30T09:10:09.110 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.108+0000 7f970d7cd740 1 -- 172.21.15.42:0/3959668107 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563d0c4282f0 msgr2=0x563d0c811e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:10:09.110 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.108+0000 7f970d7cd740 1 --2- 172.21.15.42:0/3959668107 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563d0c4282f0 0x563d0c811e00 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f96f8007fd0 tx=0x7f96f8007f40 comp rx=0 tx=0).stop 2024-08-30T09:10:09.110 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.108+0000 7f970d7cd740 1 -- 172.21.15.42:0/3959668107 shutdown_connections 2024-08-30T09:10:09.110 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.108+0000 7f970d7cd740 1 --2- 172.21.15.42:0/3959668107 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x563d0c40fd80 0x563d0c81faf0 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:09.110 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.108+0000 7f970d7cd740 1 --2- 172.21.15.42:0/3959668107 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563d0c4511b0 0x563d0c813030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:09.110 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.108+0000 7f970d7cd740 1 --2- 172.21.15.42:0/3959668107 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563d0c4304d0 0x563d0c8124c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:09.110 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.108+0000 7f970d7cd740 1 --2- 172.21.15.42:0/3959668107 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563d0c4282f0 0x563d0c811e00 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:09.110 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.108+0000 7f970d7cd740 1 -- 172.21.15.42:0/3959668107 >> 172.21.15.42:0/3959668107 conn(0x563d0c445300 msgr2=0x563d0c7f1370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:10:09.113 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.108+0000 7f96df7fe640 0 client.26797 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:10:09.117 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.115+0000 7f970d7cd740 1 -- 172.21.15.42:0/3959668107 shutdown_connections 2024-08-30T09:10:09.118 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.117+0000 7f970d7cd740 1 -- 172.21.15.42:0/3959668107 wait complete. 2024-08-30T09:10:09.124 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.123+0000 7f970d7cd740 20 client.26797 trim_cache size 0 max 16384 2024-08-30T09:10:09.140 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T09:10:09.157 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.154+0000 7f970d7cd740 1 Processor -- start 2024-08-30T09:10:09.157 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.154+0000 7f970d7cd740 1 -- start start 2024-08-30T09:10:09.157 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.154+0000 7f970d7cd740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563d0c4282f0 0x563d0c811710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:09.157 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.154+0000 7f970d7cd740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563d0c4304d0 0x563d0c7f4370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:09.157 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.154+0000 7f970d7cd740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563d0c4511b0 0x563d0c7f8290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:09.157 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.154+0000 7f970d7cd740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f96ec0125c0 con 0x563d0c4511b0 2024-08-30T09:10:09.157 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.154+0000 7f970d7cd740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f96ec00fe30 con 0x563d0c4282f0 2024-08-30T09:10:09.157 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.154+0000 7f970d7cd740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f96ec00dc80 con 0x563d0c4304d0 2024-08-30T09:10:09.157 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.154+0000 7f96ff7fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563d0c4304d0 0x563d0c7f4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:10:09.157 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.154+0000 7f9704d1d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563d0c4511b0 0x563d0c7f8290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:10:09.157 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.154+0000 7f96ff7fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563d0c4304d0 0x563d0c7f4370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:48686/0 (socket says 172.21.15.42:48686) 2024-08-30T09:10:09.157 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.154+0000 7f96ff7fe640 1 -- 172.21.15.42:0/1234854781 learned_addr learned my addr 172.21.15.42:0/1234854781 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:10:09.158 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.154+0000 7f9704d1d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563d0c4511b0 0x563d0c7f8290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:39434/0 (socket says 172.21.15.42:39434) 2024-08-30T09:10:09.158 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.154+0000 7f9704d1d640 1 --2- 172.21.15.42:0/1234854781 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563d0c4282f0 0x563d0c811710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:10:09.158 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.155+0000 7f96ff7fe640 1 -- 172.21.15.42:0/1234854781 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563d0c4282f0 msgr2=0x563d0c811710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:10:09.158 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.155+0000 7f96ff7fe640 1 --2- 172.21.15.42:0/1234854781 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563d0c4282f0 0x563d0c811710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:09.158 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.155+0000 7f96ff7fe640 1 -- 172.21.15.42:0/1234854781 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563d0c4511b0 msgr2=0x563d0c7f8290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:10:09.158 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.155+0000 7f96ff7fe640 1 --2- 172.21.15.42:0/1234854781 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563d0c4511b0 0x563d0c7f8290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:09.158 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.155+0000 7f96ff7fe640 1 -- 172.21.15.42:0/1234854781 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d0c40ffb0 con 0x563d0c4304d0 2024-08-30T09:10:09.158 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.155+0000 7f96ff7fe640 1 --2- 172.21.15.42:0/1234854781 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563d0c4304d0 0x563d0c7f4370 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f96f0006010 tx=0x7f96f0031400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5771916fe441baab server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:10:09.159 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.155+0000 7f96e5ffb640 1 -- 172.21.15.42:0/1234854781 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96f0035040 con 0x563d0c4304d0 2024-08-30T09:10:09.159 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.156+0000 7f96e5ffb640 1 -- 172.21.15.42:0/1234854781 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f96f0047070 con 0x563d0c4304d0 2024-08-30T09:10:09.159 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.156+0000 7f96e5ffb640 1 -- 172.21.15.42:0/1234854781 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96f0003db0 con 0x563d0c4304d0 2024-08-30T09:10:09.161 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.159+0000 7f970d7cd740 1 -- 172.21.15.42:0/1234854781 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563d0c4304d0 msgr2=0x563d0c7f4370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:10:09.161 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.159+0000 7f970d7cd740 1 --2- 172.21.15.42:0/1234854781 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563d0c4304d0 0x563d0c7f4370 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f96f0006010 tx=0x7f96f0031400 comp rx=0 tx=0).stop 2024-08-30T09:10:09.161 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.159+0000 7f970d7cd740 1 -- 172.21.15.42:0/1234854781 shutdown_connections 2024-08-30T09:10:09.162 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.159+0000 7f970d7cd740 1 --2- 172.21.15.42:0/1234854781 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563d0c4304d0 0x563d0c7f4370 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:09.162 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.159+0000 7f970d7cd740 1 --2- 172.21.15.42:0/1234854781 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563d0c4511b0 0x563d0c7f8290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:09.162 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.159+0000 7f970d7cd740 1 --2- 172.21.15.42:0/1234854781 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563d0c4282f0 0x563d0c811710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:09.162 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.159+0000 7f970d7cd740 1 -- 172.21.15.42:0/1234854781 >> 172.21.15.42:0/1234854781 conn(0x563d0c445300 msgr2=0x563d0c811000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:10:09.162 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.159+0000 7f970d7cd740 1 -- 172.21.15.42:0/1234854781 shutdown_connections 2024-08-30T09:10:09.162 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.159+0000 7f970d7cd740 1 -- 172.21.15.42:0/1234854781 wait complete. 2024-08-30T09:10:09.162 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.159+0000 7f970d7cd740 1 -- start start 2024-08-30T09:10:09.164 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.160+0000 7f970d7cd740 1 Processor -- start 2024-08-30T09:10:09.164 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.160+0000 7f970d7cd740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563d0c4282f0 0x563d0c8172d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:09.164 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.160+0000 7f970d7cd740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563d0c4304d0 0x563d0c817990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:09.164 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.160+0000 7f970d7cd740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563d0c4511b0 0x563d0c8184b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:09.164 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.160+0000 7f970d7cd740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f96ec00e2f0 con 0x563d0c4282f0 2024-08-30T09:10:09.165 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.160+0000 7f970d7cd740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f96ec00f760 con 0x563d0c4511b0 2024-08-30T09:10:09.165 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.160+0000 7f970d7cd740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f96ec00e9c0 con 0x563d0c4304d0 2024-08-30T09:10:09.165 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.161+0000 7f96ff7fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563d0c4304d0 0x563d0c817990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:10:09.165 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.161+0000 7f970d7cd740 10 client.0 _ll_register_callbacks cb 0x7f96f800f5e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:10:09.165 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.161+0000 7f96ff7fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563d0c4304d0 0x563d0c817990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:48696/0 (socket says 172.21.15.42:48696) 2024-08-30T09:10:09.165 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.161+0000 7f96ff7fe640 1 -- 172.21.15.42:0/3467634290 learned_addr learned my addr 172.21.15.42:0/3467634290 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:10:09.165 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.161+0000 7f9704d1d640 1 --2- 172.21.15.42:0/3467634290 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563d0c4282f0 0x563d0c8172d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:10:09.165 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.161+0000 7f9704d1d640 1 --2- 172.21.15.42:0/3467634290 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563d0c4511b0 0x563d0c8184b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:10:09.165 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.161+0000 7f96ff7fe640 1 -- 172.21.15.42:0/3467634290 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563d0c4511b0 msgr2=0x563d0c8184b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:10:09.165 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.161+0000 7f96ff7fe640 1 --2- 172.21.15.42:0/3467634290 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563d0c4511b0 0x563d0c8184b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:09.165 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.161+0000 7f96ff7fe640 1 -- 172.21.15.42:0/3467634290 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563d0c4282f0 msgr2=0x563d0c8172d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:10:09.165 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.161+0000 7f96ff7fe640 1 --2- 172.21.15.42:0/3467634290 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563d0c4282f0 0x563d0c8172d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:09.165 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.161+0000 7f96ff7fe640 1 -- 172.21.15.42:0/3467634290 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d0c818c10 con 0x563d0c4304d0 2024-08-30T09:10:09.165 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.161+0000 7f96ff7fe640 1 --2- 172.21.15.42:0/3467634290 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563d0c4304d0 0x563d0c817990 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f96f00096b0 tx=0x7f96f002a900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ba663ba432153771 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:10:09.166 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.161+0000 7f96df7fe640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-08-30T09:10:09.166 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.162+0000 7f96df7fe640 1 -- 172.21.15.42:0/3467634290 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96f004f070 con 0x563d0c4304d0 2024-08-30T09:10:09.166 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.162+0000 7f970d7cd740 1 -- 172.21.15.42:0/3467634290 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f96f0047070 con 0x563d0c4304d0 2024-08-30T09:10:09.166 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.162+0000 7f96df7fe640 1 -- 172.21.15.42:0/3467634290 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f96f0003b30 con 0x563d0c4304d0 2024-08-30T09:10:09.166 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.162+0000 7f96df7fe640 1 -- 172.21.15.42:0/3467634290 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f96f0035040 con 0x563d0c4304d0 2024-08-30T09:10:09.166 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.162+0000 7f970d7cd740 10 client.26803 fetch_fsmap learned FSMap version 39 2024-08-30T09:10:09.166 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.162+0000 7f970d7cd740 1 -- 172.21.15.42:0/3467634290 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x563d0c819f30 con 0x563d0c4304d0 2024-08-30T09:10:09.166 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.162+0000 7f96df7fe640 1 -- 172.21.15.42:0/3467634290 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f96f0047070 con 0x563d0c4304d0 2024-08-30T09:10:09.166 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.162+0000 7f96df7fe640 1 -- 172.21.15.42:0/3467634290 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7f96f0004b00 con 0x563d0c4304d0 2024-08-30T09:10:09.166 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.162+0000 7f970d7cd740 10 client.26803 fetch_fsmap finished waiting for FSMap version 39 2024-08-30T09:10:09.166 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.163+0000 7f970d7cd740 10 client.26803 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-30T09:10:09.166 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.163+0000 7f970d7cd740 20 client.26803 populate_metadata read hostname 'smithi042' 2024-08-30T09:10:09.166 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.163+0000 7f970d7cd740 1 --2- 172.21.15.42:0/3467634290 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x563d0c3e51c0 0x563d0c88c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:09.167 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.163+0000 7f970d7cd740 4 client.26803 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-30T09:10:09.167 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.163+0000 7f970d7cd740 1 -- 172.21.15.42:0/3467634290 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563d0c7679f0 con 0x563d0c3e51c0 2024-08-30T09:10:09.167 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.163+0000 7f9704d1d640 1 --2- 172.21.15.42:0/3467634290 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x563d0c3e51c0 0x563d0c88c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:10:09.167 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.163+0000 7f96df7fe640 10 client.26803 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:10:09.167 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.163+0000 7f9704d1d640 1 --2- 172.21.15.42:0/3467634290 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x563d0c3e51c0 0x563d0c88c9c0 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b058b50a5e515c9 server_cookie=b29db53ad5565876 in_seq=0 out_seq=0 2024-08-30T09:10:09.178 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.176+0000 7f96df7fe640 1 -- 172.21.15.42:0/3467634290 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563d0c7679f0 con 0x563d0c3e51c0 2024-08-30T09:10:09.178 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.176+0000 7f96df7fe640 10 client.26803 handle_command_reply: tid=0 2024-08-30T09:10:09.178 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.176+0000 7f970d7cd740 1 client.26803 shutdown 2024-08-30T09:10:09.180 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.177+0000 7f970d7cd740 1 -- 172.21.15.42:0/3467634290 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563d0c4304d0 msgr2=0x563d0c817990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:10:09.180 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.177+0000 7f970d7cd740 1 --2- 172.21.15.42:0/3467634290 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563d0c4304d0 0x563d0c817990 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f96f00096b0 tx=0x7f96f002a900 comp rx=0 tx=0).stop 2024-08-30T09:10:09.180 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.178+0000 7f970d7cd740 1 -- 172.21.15.42:0/3467634290 shutdown_connections 2024-08-30T09:10:09.180 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.178+0000 7f970d7cd740 1 --2- 172.21.15.42:0/3467634290 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x563d0c3e51c0 0x563d0c88c9c0 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:09.180 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.178+0000 7f970d7cd740 1 --2- 172.21.15.42:0/3467634290 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563d0c4304d0 0x563d0c817990 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:09.180 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.178+0000 7f970d7cd740 1 --2- 172.21.15.42:0/3467634290 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563d0c4511b0 0x563d0c8184b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:09.180 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.178+0000 7f970d7cd740 1 --2- 172.21.15.42:0/3467634290 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563d0c4282f0 0x563d0c8172d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:09.180 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.178+0000 7f96df7fe640 0 client.26803 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:10:09.180 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.178+0000 7f970d7cd740 1 -- 172.21.15.42:0/3467634290 >> 172.21.15.42:0/3467634290 conn(0x563d0c445300 msgr2=0x563d0c7f8290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:10:09.181 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.178+0000 7f970d7cd740 1 -- 172.21.15.42:0/3467634290 shutdown_connections 2024-08-30T09:10:09.181 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.179+0000 7f970d7cd740 1 -- 172.21.15.42:0/3467634290 wait complete. 2024-08-30T09:10:09.181 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.179+0000 7f970d7cd740 20 client.26803 trim_cache size 0 max 16384 2024-08-30T09:10:09.181 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-08-30T09:10:09.182 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2024-08-30T09:10:09.182 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-08-30T09:10:09.182 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-08-30T09:10:09.183 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.180+0000 7f970551e640 1 -- 172.21.15.42:0/775250684 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f96e80b4d40 msgr2=0x7f96e80b7250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:10:09.183 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.180+0000 7f970551e640 1 --2- 172.21.15.42:0/775250684 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f96e80b4d40 0x7f96e80b7250 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f96f40ab410 tx=0x7f96f400cf60 comp rx=0 tx=0).stop 2024-08-30T09:10:09.183 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.180+0000 7f970551e640 1 -- 172.21.15.42:0/775250684 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f970010a160 msgr2=0x7f97001a8940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:10:09.183 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.180+0000 7f970551e640 1 --2- 172.21.15.42:0/775250684 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f970010a160 0x7f97001a8940 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f96f400c970 tx=0x7f96f400ce30 comp rx=0 tx=0).stop 2024-08-30T09:10:09.183 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.180+0000 7f970551e640 1 -- 172.21.15.42:0/775250684 shutdown_connections 2024-08-30T09:10:09.183 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.180+0000 7f970551e640 1 --2- 172.21.15.42:0/775250684 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f96e80b4d40 0x7f96e80b7250 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:09.183 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.180+0000 7f970551e640 1 --2- 172.21.15.42:0/775250684 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f970010f5d0 0x7f97001a9b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:09.183 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.180+0000 7f970551e640 1 --2- 172.21.15.42:0/775250684 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f970010cac0 0x7f97001a8fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:09.183 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.180+0000 7f970551e640 1 --2- 172.21.15.42:0/775250684 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f970010a160 0x7f97001a8940 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:09.183 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.180+0000 7f970551e640 1 -- 172.21.15.42:0/775250684 >> 172.21.15.42:0/775250684 conn(0x7f97000703e0 msgr2=0x7f970010afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:10:09.183 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.180+0000 7f970551e640 1 -- 172.21.15.42:0/775250684 shutdown_connections 2024-08-30T09:10:09.184 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:09.180+0000 7f970551e640 1 -- 172.21.15.42:0/775250684 wait complete. 2024-08-30T09:10:09.273 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:10:08 smithi042 ceph-mon[18652]: pgmap v654: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 4.3 MiB/s wr, 439 op/s 2024-08-30T09:10:09.274 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:10:08 smithi042 ceph-mon[18652]: scrub summary: active paths [/] 2024-08-30T09:10:09.274 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:10:08 smithi042 ceph-mon[18652]: scrub summary: idle+waiting paths [/] 2024-08-30T09:10:09.274 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:10:08 smithi042 ceph-mon[18652]: scrub summary: active paths [/] 2024-08-30T09:10:09.274 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:10:08 smithi042 ceph-mon[18652]: scrub summary: idle+waiting paths [/] 2024-08-30T09:10:09.274 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:10:08 smithi042 ceph-mon[18652]: scrub summary: active paths [/] 2024-08-30T09:10:09.274 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:10:08 smithi042 ceph-mon[18652]: scrub summary: idle+waiting paths [/] 2024-08-30T09:10:09.274 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:10:08 smithi042 ceph-mon[18652]: scrub summary: idle+waiting paths [/] 2024-08-30T09:10:09.274 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:10:08 smithi042 ceph-mon[18652]: scrub summary: idle+waiting paths [/] 2024-08-30T09:10:09.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:10:08 smithi167 ceph-mon[22063]: pgmap v654: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 4.3 MiB/s wr, 439 op/s 2024-08-30T09:10:09.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:10:08 smithi167 ceph-mon[22063]: scrub summary: active paths [/] 2024-08-30T09:10:09.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:10:08 smithi167 ceph-mon[22063]: scrub summary: idle+waiting paths [/] 2024-08-30T09:10:09.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:10:08 smithi167 ceph-mon[22063]: scrub summary: active paths [/] 2024-08-30T09:10:09.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:10:08 smithi167 ceph-mon[22063]: scrub summary: idle+waiting paths [/] 2024-08-30T09:10:09.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:10:08 smithi167 ceph-mon[22063]: scrub summary: active paths [/] 2024-08-30T09:10:09.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:10:08 smithi167 ceph-mon[22063]: scrub summary: idle+waiting paths [/] 2024-08-30T09:10:09.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:10:08 smithi167 ceph-mon[22063]: scrub summary: idle+waiting paths [/] 2024-08-30T09:10:09.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:10:08 smithi167 ceph-mon[22063]: scrub summary: idle+waiting paths [/] 2024-08-30T09:10:09.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:10:08 smithi190 ceph-mon[22001]: pgmap v654: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 4.3 MiB/s wr, 439 op/s 2024-08-30T09:10:09.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:10:08 smithi190 ceph-mon[22001]: scrub summary: active paths [/] 2024-08-30T09:10:09.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:10:08 smithi190 ceph-mon[22001]: scrub summary: idle+waiting paths [/] 2024-08-30T09:10:09.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:10:08 smithi190 ceph-mon[22001]: scrub summary: active paths [/] 2024-08-30T09:10:09.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:10:08 smithi190 ceph-mon[22001]: scrub summary: idle+waiting paths [/] 2024-08-30T09:10:09.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:10:08 smithi190 ceph-mon[22001]: scrub summary: active paths [/] 2024-08-30T09:10:09.437 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:10:08 smithi190 ceph-mon[22001]: scrub summary: idle+waiting paths [/] 2024-08-30T09:10:09.437 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:10:08 smithi190 ceph-mon[22001]: scrub summary: idle+waiting paths [/] 2024-08-30T09:10:09.437 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:10:08 smithi190 ceph-mon[22001]: scrub summary: idle+waiting paths [/] 2024-08-30T09:10:10.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:10:09 smithi167 ceph-mon[22063]: pgmap v655: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 5.4 MiB/s wr, 569 op/s 2024-08-30T09:10:10.337 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:10:09 smithi190 ceph-mon[22001]: pgmap v655: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 5.4 MiB/s wr, 569 op/s 2024-08-30T09:10:10.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:10:09 smithi042 ceph-mon[18652]: pgmap v655: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 5.4 MiB/s wr, 569 op/s 2024-08-30T09:10:10.953 INFO:teuthology.orchestra.run.smithi042.stderr:progress: 565.0 s, 474.2 tps, lat 198.886 ms stddev 228.892 2024-08-30T09:10:12.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:10:12 smithi167 ceph-mon[22063]: pgmap v656: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 4.4 MiB/s wr, 464 op/s 2024-08-30T09:10:12.621 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:10:12 smithi042 ceph-mon[18652]: pgmap v656: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 4.4 MiB/s wr, 464 op/s 2024-08-30T09:10:12.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:10:12 smithi190 ceph-mon[22001]: pgmap v656: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 4.4 MiB/s wr, 464 op/s 2024-08-30T09:10:14.493 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-08-30T09:10:14.494 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "IDLE (waiting for more scrubs)", 2024-08-30T09:10:14.494 INFO:teuthology.orchestra.run.smithi042.stdout: "scrubs": { 2024-08-30T09:10:14.494 INFO:teuthology.orchestra.run.smithi042.stdout: "a7210f9a-046b-4188-953c-819b9f1863a9": { 2024-08-30T09:10:14.494 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:10:14.494 INFO:teuthology.orchestra.run.smithi042.stdout: "tag": "a7210f9a-046b-4188-953c-819b9f1863a9", 2024-08-30T09:10:14.494 INFO:teuthology.orchestra.run.smithi042.stdout: "options": "recursive,force" 2024-08-30T09:10:14.494 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:10:14.494 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:10:14.494 INFO:teuthology.orchestra.run.smithi042.stdout: "scrub_stats": { 2024-08-30T09:10:14.494 INFO:teuthology.orchestra.run.smithi042.stdout: "37f2f28f-496f-41a6-b4cf-8846fb3f43ab": { 2024-08-30T09:10:14.494 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:09:17.428286+0000", 2024-08-30T09:10:14.494 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:10:14.494 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:10:14.495 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:10:14.495 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:10:14.495 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:10:14.495 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:10:14.495 INFO:teuthology.orchestra.run.smithi042.stdout: "99a29ec1-9455-4ae4-854a-398b32896675": { 2024-08-30T09:10:14.495 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:08:24.923176+0000", 2024-08-30T09:10:14.495 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:10:14.495 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:10:14.495 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:10:14.495 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:10:14.495 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:10:14.495 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:10:14.495 INFO:teuthology.orchestra.run.smithi042.stdout: "a7210f9a-046b-4188-953c-819b9f1863a9": { 2024-08-30T09:10:14.495 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:10:07.608430+0000", 2024-08-30T09:10:14.496 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:10:14.496 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:10:14.496 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:10:14.496 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:10:14.496 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:10:14.496 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:10:14.496 INFO:teuthology.orchestra.run.smithi042.stdout: "598d8a75-f1cc-4388-80bc-a408bdb4eaac": { 2024-08-30T09:10:14.496 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:07:29.721122+0000", 2024-08-30T09:10:14.496 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:10:14.496 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:10:14.496 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:10:14.496 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:10:14.496 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:10:14.497 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:10:14.497 INFO:teuthology.orchestra.run.smithi042.stdout: "665ec740-7254-49cb-84ef-deff71c461b0": { 2024-08-30T09:10:14.497 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:06:37.144035+0000", 2024-08-30T09:10:14.497 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:10:14.497 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:10:14.497 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:10:14.497 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:10:14.497 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:10:14.497 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:10:14.497 INFO:teuthology.orchestra.run.smithi042.stdout: "1af4252a-4a99-47e9-bbf0-becf5e573dd5": { 2024-08-30T09:10:14.497 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:05:48.565881+0000", 2024-08-30T09:10:14.497 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:10:14.497 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:10:14.498 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:10:14.498 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:10:14.498 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:10:14.498 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:10:14.498 INFO:teuthology.orchestra.run.smithi042.stdout: "347d05ed-4bd3-4590-b099-314f719aab39": { 2024-08-30T09:10:14.498 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:05:00.439721+0000", 2024-08-30T09:10:14.498 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:10:14.498 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:10:14.498 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:10:14.498 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:10:14.498 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:10:14.498 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:10:14.498 INFO:teuthology.orchestra.run.smithi042.stdout: "88cd7478-2b20-47de-ba51-256479749232": { 2024-08-30T09:10:14.498 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:04:08.922625+0000", 2024-08-30T09:10:14.499 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:10:14.499 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:10:14.499 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:10:14.499 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:10:14.499 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:10:14.499 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:10:14.499 INFO:teuthology.orchestra.run.smithi042.stdout: "749f4867-0b70-49e5-af2d-8362a240ebc3": { 2024-08-30T09:10:14.499 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:03:16.386554+0000", 2024-08-30T09:10:14.499 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:10:14.499 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:10:14.499 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:10:14.499 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:10:14.500 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:10:14.500 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:10:14.500 INFO:teuthology.orchestra.run.smithi042.stdout: "85134e4b-ab4a-4885-b07e-a3ad01eaf13d": { 2024-08-30T09:10:14.500 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:01:27.738462+0000", 2024-08-30T09:10:14.500 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:10:14.500 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:10:14.500 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:10:14.500 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:10:14.500 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:10:14.500 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:10:14.500 INFO:teuthology.orchestra.run.smithi042.stdout: "c827ad7a-44ee-4e1d-9bb3-51fb7ab00c73": { 2024-08-30T09:10:14.501 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:02:21.686517+0000", 2024-08-30T09:10:14.501 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:10:14.501 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:10:14.501 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:10:14.501 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:10:14.501 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:10:14.501 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:10:14.501 INFO:teuthology.orchestra.run.smithi042.stdout: "5e2ed104-0625-478a-a78b-584e7f4c2817": { 2024-08-30T09:10:14.501 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:00:33.928400+0000", 2024-08-30T09:10:14.501 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:10:14.501 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:10:14.501 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:10:14.501 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:10:14.502 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:10:14.502 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:10:14.502 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:10:14.502 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-08-30T09:10:14.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.690+0000 7f7c4d210640 1 -- 172.21.15.42:0/3541600897 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7c48109bb0 msgr2=0x7f7c481103e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:10:14.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.690+0000 7f7c4d210640 1 --2- 172.21.15.42:0/3541600897 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7c48109bb0 0x7f7c481103e0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f7c40009be0 tx=0x7f7c4002a6c0 comp rx=0 tx=0).stop 2024-08-30T09:10:14.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.690+0000 7f7c4d210640 1 -- 172.21.15.42:0/3541600897 shutdown_connections 2024-08-30T09:10:14.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.691+0000 7f7c4d210640 1 --2- 172.21.15.42:0/3541600897 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7c48109bb0 0x7f7c481103e0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:14.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.691+0000 7f7c4d210640 1 --2- 172.21.15.42:0/3541600897 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7c48109260 0x7f7c48109670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:14.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.691+0000 7f7c4d210640 1 --2- 172.21.15.42:0/3541600897 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7c48103920 0x7f7c48103d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:14.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.691+0000 7f7c4d210640 1 -- 172.21.15.42:0/3541600897 >> 172.21.15.42:0/3541600897 conn(0x7f7c480ff3b0 msgr2=0x7f7c48101800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:10:14.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.691+0000 7f7c4d210640 1 -- 172.21.15.42:0/3541600897 shutdown_connections 2024-08-30T09:10:14.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.691+0000 7f7c4d210640 1 -- 172.21.15.42:0/3541600897 wait complete. 2024-08-30T09:10:14.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.691+0000 7f7c4d210640 1 Processor -- start 2024-08-30T09:10:14.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.692+0000 7f7c4d210640 1 -- start start 2024-08-30T09:10:14.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.692+0000 7f7c4d210640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7c48103920 0x7f7c4819a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:14.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.692+0000 7f7c4d210640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7c48109260 0x7f7c4819a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:14.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.692+0000 7f7c4d210640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7c48109bb0 0x7f7c4819fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:14.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.692+0000 7f7c4d210640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f7c48113870 con 0x7f7c48103920 2024-08-30T09:10:14.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.692+0000 7f7c4d210640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f7c481136f0 con 0x7f7c48109260 2024-08-30T09:10:14.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.692+0000 7f7c4d210640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f7c481139f0 con 0x7f7c48109bb0 2024-08-30T09:10:14.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.692+0000 7f7c46d9d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7c48103920 0x7f7c4819a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:10:14.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.693+0000 7f7c46d9d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7c48103920 0x7f7c4819a1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:39462/0 (socket says 172.21.15.42:39462) 2024-08-30T09:10:14.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.693+0000 7f7c46d9d640 1 -- 172.21.15.42:0/108900750 learned_addr learned my addr 172.21.15.42:0/108900750 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:10:14.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.693+0000 7f7c4759e640 1 --2- 172.21.15.42:0/108900750 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7c48109bb0 0x7f7c4819fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:10:14.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.693+0000 7f7c3e59c640 1 --2- 172.21.15.42:0/108900750 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7c48109260 0x7f7c4819a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:10:14.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.693+0000 7f7c4759e640 1 -- 172.21.15.42:0/108900750 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7c48109260 msgr2=0x7f7c4819a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:10:14.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.693+0000 7f7c4759e640 1 --2- 172.21.15.42:0/108900750 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7c48109260 0x7f7c4819a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:14.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.693+0000 7f7c4759e640 1 -- 172.21.15.42:0/108900750 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7c48103920 msgr2=0x7f7c4819a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:10:14.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.693+0000 7f7c4759e640 1 --2- 172.21.15.42:0/108900750 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7c48103920 0x7f7c4819a1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:14.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.693+0000 7f7c4759e640 1 -- 172.21.15.42:0/108900750 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c481a0130 con 0x7f7c48109bb0 2024-08-30T09:10:14.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.693+0000 7f7c46d9d640 1 --2- 172.21.15.42:0/108900750 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7c48103920 0x7f7c4819a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-30T09:10:14.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.693+0000 7f7c4759e640 1 --2- 172.21.15.42:0/108900750 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7c48109bb0 0x7f7c4819fa00 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f7c40009b30 tx=0x7f7c40003b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:10:14.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.693+0000 7f7c44d99640 1 -- 172.21.15.42:0/108900750 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c40041020 con 0x7f7c48109bb0 2024-08-30T09:10:14.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.694+0000 7f7c4d210640 1 -- 172.21.15.42:0/108900750 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c481a03c0 con 0x7f7c48109bb0 2024-08-30T09:10:14.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.694+0000 7f7c44d99640 1 -- 172.21.15.42:0/108900750 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7c400318f0 con 0x7f7c48109bb0 2024-08-30T09:10:14.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.694+0000 7f7c44d99640 1 -- 172.21.15.42:0/108900750 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c40036040 con 0x7f7c48109bb0 2024-08-30T09:10:14.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.694+0000 7f7c4d210640 1 -- 172.21.15.42:0/108900750 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c481a6b80 con 0x7f7c48109bb0 2024-08-30T09:10:14.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.698+0000 7f7c44d99640 1 -- 172.21.15.42:0/108900750 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7f7c40044020 con 0x7f7c48109bb0 2024-08-30T09:10:14.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.699+0000 7f7c44d99640 1 --2- 172.21.15.42:0/108900750 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f7c280b4b30 0x7f7c280b7040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:14.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.699+0000 7f7c53785740 1 Processor -- start 2024-08-30T09:10:14.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.699+0000 7f7c53785740 1 -- start start 2024-08-30T09:10:14.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.699+0000 7f7c44d99640 1 -- 172.21.15.42:0/108900750 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f7c40105f80 con 0x7f7c48109bb0 2024-08-30T09:10:14.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.699+0000 7f7c46d9d640 1 --2- 172.21.15.42:0/108900750 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f7c280b4b30 0x7f7c280b7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:10:14.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.699+0000 7f7c53785740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f5757a2070 0x55f575b47300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:14.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.699+0000 7f7c53785740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f5757791b0 0x55f575b47840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:14.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.700+0000 7f7c46d9d640 1 --2- 172.21.15.42:0/108900750 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f7c280b4b30 0x7f7c280b7040 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f7c38011500 tx=0x7f7c38005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:10:14.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.700+0000 7f7c53785740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f575781390 0x55f575b47d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:14.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.700+0000 7f7c53785740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55f575abe630 con 0x55f5757791b0 2024-08-30T09:10:14.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.700+0000 7f7c53785740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55f575aadbc0 con 0x55f575781390 2024-08-30T09:10:14.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.700+0000 7f7c53785740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55f575ab1b70 con 0x55f5757a2070 2024-08-30T09:10:14.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.700+0000 7f7c46d9d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f5757a2070 0x55f575b47300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:10:14.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.700+0000 7f7c46d9d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f5757a2070 0x55f575b47300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:48728/0 (socket says 172.21.15.42:48728) 2024-08-30T09:10:14.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.700+0000 7f7c46d9d640 1 -- 172.21.15.42:0/2383422235 learned_addr learned my addr 172.21.15.42:0/2383422235 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:10:14.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.700+0000 7f7c3e59c640 1 --2- 172.21.15.42:0/2383422235 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f5757791b0 0x55f575b47840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:10:14.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.700+0000 7f7c4759e640 1 --2- 172.21.15.42:0/2383422235 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f575781390 0x55f575b47d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:10:14.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.700+0000 7f7c46d9d640 1 -- 172.21.15.42:0/2383422235 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f575781390 msgr2=0x55f575b47d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:10:14.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.700+0000 7f7c46d9d640 1 --2- 172.21.15.42:0/2383422235 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f575781390 0x55f575b47d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:14.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.700+0000 7f7c46d9d640 1 -- 172.21.15.42:0/2383422235 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f5757791b0 msgr2=0x55f575b47840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:10:14.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.700+0000 7f7c46d9d640 1 --2- 172.21.15.42:0/2383422235 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f5757791b0 0x55f575b47840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:14.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.700+0000 7f7c46d9d640 1 -- 172.21.15.42:0/2383422235 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f57580d650 con 0x55f5757a2070 2024-08-30T09:10:14.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.700+0000 7f7c4759e640 1 --2- 172.21.15.42:0/2383422235 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f575781390 0x55f575b47d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-30T09:10:14.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.700+0000 7f7c46d9d640 1 --2- 172.21.15.42:0/2383422235 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f5757a2070 0x55f575b47300 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f7c3800f770 tx=0x7f7c38014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=953e8d55a669ffd8 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:10:14.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.701+0000 7f7c3d59a640 1 -- 172.21.15.42:0/2383422235 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c3801c880 con 0x55f5757a2070 2024-08-30T09:10:14.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.701+0000 7f7c3d59a640 1 -- 172.21.15.42:0/2383422235 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7c38025390 con 0x55f5757a2070 2024-08-30T09:10:14.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.701+0000 7f7c3d59a640 1 -- 172.21.15.42:0/2383422235 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c38025e60 con 0x55f5757a2070 2024-08-30T09:10:14.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.702+0000 7f7c53785740 1 -- 172.21.15.42:0/2383422235 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f5757a2070 msgr2=0x55f575b47300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:10:14.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.702+0000 7f7c53785740 1 --2- 172.21.15.42:0/2383422235 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f5757a2070 0x55f575b47300 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f7c3800f770 tx=0x7f7c38014000 comp rx=0 tx=0).stop 2024-08-30T09:10:14.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.702+0000 7f7c53785740 1 -- 172.21.15.42:0/2383422235 shutdown_connections 2024-08-30T09:10:14.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.702+0000 7f7c53785740 1 --2- 172.21.15.42:0/2383422235 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f575781390 0x55f575b47d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:14.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.702+0000 7f7c53785740 1 --2- 172.21.15.42:0/2383422235 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f5757791b0 0x55f575b47840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:14.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.702+0000 7f7c53785740 1 --2- 172.21.15.42:0/2383422235 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f5757a2070 0x55f575b47300 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:14.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.702+0000 7f7c53785740 1 -- 172.21.15.42:0/2383422235 >> 172.21.15.42:0/2383422235 conn(0x55f575796460 msgr2=0x55f575b451f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:10:14.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.702+0000 7f7c53785740 1 -- 172.21.15.42:0/2383422235 shutdown_connections 2024-08-30T09:10:14.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.702+0000 7f7c53785740 1 -- 172.21.15.42:0/2383422235 wait complete. 2024-08-30T09:10:14.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.702+0000 7f7c53785740 1 -- start start 2024-08-30T09:10:14.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.702+0000 7f7c53785740 1 Processor -- start 2024-08-30T09:10:14.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.703+0000 7f7c53785740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f5757791b0 0x55f575b56e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:14.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.703+0000 7f7c53785740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f575781390 0x55f575b573a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:14.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.703+0000 7f7c53785740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f5757a2070 0x55f575b631a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:14.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.703+0000 7f7c53785740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55f575910700 con 0x55f5757a2070 2024-08-30T09:10:14.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.703+0000 7f7c53785740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55f575a34700 con 0x55f575781390 2024-08-30T09:10:14.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.703+0000 7f7c53785740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55f575b59e40 con 0x55f5757791b0 2024-08-30T09:10:14.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.703+0000 7f7c53785740 10 client.0 _ll_register_callbacks cb 0x55f575a63dc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:10:14.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.703+0000 7f7c46d9d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f5757791b0 0x55f575b56e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:10:14.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.703+0000 7f7c46d9d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f5757791b0 0x55f575b56e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:48730/0 (socket says 172.21.15.42:48730) 2024-08-30T09:10:14.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.703+0000 7f7c46d9d640 1 -- 172.21.15.42:0/2679904937 learned_addr learned my addr 172.21.15.42:0/2679904937 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:10:14.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.703+0000 7f7c4759e640 1 --2- 172.21.15.42:0/2679904937 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f5757a2070 0x55f575b631a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:10:14.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.703+0000 7f7c3e59c640 1 --2- 172.21.15.42:0/2679904937 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f575781390 0x55f575b573a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:10:14.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.703+0000 7f7c46d9d640 1 -- 172.21.15.42:0/2679904937 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f575781390 msgr2=0x55f575b573a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:10:14.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.703+0000 7f7c46d9d640 1 --2- 172.21.15.42:0/2679904937 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f575781390 0x55f575b573a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:14.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.703+0000 7f7c46d9d640 1 -- 172.21.15.42:0/2679904937 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f5757a2070 msgr2=0x55f575b631a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:10:14.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.703+0000 7f7c46d9d640 1 --2- 172.21.15.42:0/2679904937 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f5757a2070 0x55f575b631a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:14.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.703+0000 7f7c46d9d640 1 -- 172.21.15.42:0/2679904937 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f575a6d8b0 con 0x55f5757791b0 2024-08-30T09:10:14.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.704+0000 7f7c46d9d640 1 --2- 172.21.15.42:0/2679904937 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f5757791b0 0x55f575b56e60 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f7c38025ce0 tx=0x7f7c38025ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=15894c56fbc0ede4 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:10:14.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.704+0000 7f7c367fc640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-08-30T09:10:14.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.704+0000 7f7c367fc640 1 -- 172.21.15.42:0/2679904937 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c38033060 con 0x55f5757791b0 2024-08-30T09:10:14.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.704+0000 7f7c53785740 1 -- 172.21.15.42:0/2679904937 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7c38025390 con 0x55f5757791b0 2024-08-30T09:10:14.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.704+0000 7f7c367fc640 1 -- 172.21.15.42:0/2679904937 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f7c38005260 con 0x55f5757791b0 2024-08-30T09:10:14.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.705+0000 7f7c367fc640 1 -- 172.21.15.42:0/2679904937 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f7c38005460 con 0x55f5757791b0 2024-08-30T09:10:14.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.706+0000 7f7c53785740 10 client.26815 fetch_fsmap learned FSMap version 39 2024-08-30T09:10:14.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.706+0000 7f7c53785740 1 -- 172.21.15.42:0/2679904937 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55f575b65b10 con 0x55f5757791b0 2024-08-30T09:10:14.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.707+0000 7f7c367fc640 1 -- 172.21.15.42:0/2679904937 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7c38025390 con 0x55f5757791b0 2024-08-30T09:10:14.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.707+0000 7f7c367fc640 1 -- 172.21.15.42:0/2679904937 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7f7c3801c9f0 con 0x55f5757791b0 2024-08-30T09:10:14.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.707+0000 7f7c53785740 10 client.26815 fetch_fsmap finished waiting for FSMap version 39 2024-08-30T09:10:14.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.707+0000 7f7c53785740 10 client.26815 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-30T09:10:14.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.707+0000 7f7c53785740 20 client.26815 populate_metadata read hostname 'smithi042' 2024-08-30T09:10:14.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.707+0000 7f7c53785740 1 --2- 172.21.15.42:0/2679904937 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55f5757609f0 0x55f575b68e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:14.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.707+0000 7f7c53785740 4 client.26815 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-30T09:10:14.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.707+0000 7f7c53785740 1 -- 172.21.15.42:0/2679904937 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f575ac0580 con 0x55f5757609f0 2024-08-30T09:10:14.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.708+0000 7f7c3e59c640 1 --2- 172.21.15.42:0/2679904937 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55f5757609f0 0x55f575b68e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:10:14.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.708+0000 7f7c367fc640 10 client.26815 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:10:14.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.708+0000 7f7c3e59c640 1 --2- 172.21.15.42:0/2679904937 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55f5757609f0 0x55f575b68e70 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e4c4b3a4c8f1c54 server_cookie=d4748cfc8dd960e7 in_seq=0 out_seq=0 2024-08-30T09:10:14.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.709+0000 7f7c367fc640 1 -- 172.21.15.42:0/2679904937 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f575ac0580 con 0x55f5757609f0 2024-08-30T09:10:14.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.709+0000 7f7c367fc640 10 client.26815 handle_command_reply: tid=0 2024-08-30T09:10:14.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.709+0000 7f7c53785740 1 client.26815 shutdown 2024-08-30T09:10:14.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.709+0000 7f7c53785740 1 -- 172.21.15.42:0/2679904937 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f5757791b0 msgr2=0x55f575b56e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:10:14.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.709+0000 7f7c53785740 1 --2- 172.21.15.42:0/2679904937 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f5757791b0 0x55f575b56e60 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f7c38025ce0 tx=0x7f7c38025ad0 comp rx=0 tx=0).stop 2024-08-30T09:10:14.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.709+0000 7f7c53785740 1 -- 172.21.15.42:0/2679904937 shutdown_connections 2024-08-30T09:10:14.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.709+0000 7f7c53785740 1 --2- 172.21.15.42:0/2679904937 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55f5757609f0 0x55f575b68e70 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:14.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.709+0000 7f7c53785740 1 --2- 172.21.15.42:0/2679904937 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f5757a2070 0x55f575b631a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:14.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.709+0000 7f7c53785740 1 --2- 172.21.15.42:0/2679904937 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f575781390 0x55f575b573a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:14.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.709+0000 7f7c53785740 1 --2- 172.21.15.42:0/2679904937 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f5757791b0 0x55f575b56e60 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:14.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.709+0000 7f7c53785740 1 -- 172.21.15.42:0/2679904937 >> 172.21.15.42:0/2679904937 conn(0x55f575796460 msgr2=0x55f575b48cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:10:14.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.710+0000 7f7c367fc640 0 client.26815 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:10:14.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.710+0000 7f7c53785740 1 -- 172.21.15.42:0/2679904937 shutdown_connections 2024-08-30T09:10:14.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.710+0000 7f7c53785740 1 -- 172.21.15.42:0/2679904937 wait complete. 2024-08-30T09:10:14.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.710+0000 7f7c53785740 20 client.26815 trim_cache size 0 max 16384 2024-08-30T09:10:14.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.735+0000 7f7c53785740 1 Processor -- start 2024-08-30T09:10:14.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.735+0000 7f7c53785740 1 -- start start 2024-08-30T09:10:14.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.735+0000 7f7c53785740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f5757791b0 0x55f575b556e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:14.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.735+0000 7f7c53785740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f575781390 0x55f575b55c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:14.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.735+0000 7f7c53785740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f5757a2070 0x55f575b54050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:14.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.735+0000 7f7c53785740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f7c481b0f40 con 0x55f575781390 2024-08-30T09:10:14.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.735+0000 7f7c53785740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f7c481ae7b0 con 0x55f5757791b0 2024-08-30T09:10:14.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.735+0000 7f7c53785740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f7c481ac600 con 0x55f5757a2070 2024-08-30T09:10:14.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.736+0000 7f7c46d9d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f5757791b0 0x55f575b556e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:10:14.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.736+0000 7f7c46d9d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f5757791b0 0x55f575b556e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:36720/0 (socket says 172.21.15.42:36720) 2024-08-30T09:10:14.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.736+0000 7f7c46d9d640 1 -- 172.21.15.42:0/2281243278 learned_addr learned my addr 172.21.15.42:0/2281243278 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:10:14.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.736+0000 7f7c3e59c640 1 --2- 172.21.15.42:0/2281243278 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f575781390 0x55f575b55c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:10:14.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.736+0000 7f7c4759e640 1 --2- 172.21.15.42:0/2281243278 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f5757a2070 0x55f575b54050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:10:14.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.736+0000 7f7c46d9d640 1 -- 172.21.15.42:0/2281243278 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f5757a2070 msgr2=0x55f575b54050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:10:14.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.736+0000 7f7c46d9d640 1 --2- 172.21.15.42:0/2281243278 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f5757a2070 0x55f575b54050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:14.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.736+0000 7f7c46d9d640 1 -- 172.21.15.42:0/2281243278 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f575781390 msgr2=0x55f575b55c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:10:14.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.736+0000 7f7c46d9d640 1 --2- 172.21.15.42:0/2281243278 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f575781390 0x55f575b55c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:14.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.736+0000 7f7c46d9d640 1 -- 172.21.15.42:0/2281243278 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f575b55360 con 0x55f5757791b0 2024-08-30T09:10:14.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.736+0000 7f7c46d9d640 1 --2- 172.21.15.42:0/2281243278 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f5757791b0 0x55f575b556e0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f7c38000c00 tx=0x7f7c3800f710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=afcd1ae74610fef0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:10:14.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.737+0000 7f7c3cd99640 1 -- 172.21.15.42:0/2281243278 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c38033070 con 0x55f5757791b0 2024-08-30T09:10:14.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.737+0000 7f7c3cd99640 1 -- 172.21.15.42:0/2281243278 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7c38025390 con 0x55f5757791b0 2024-08-30T09:10:14.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.737+0000 7f7c3cd99640 1 -- 172.21.15.42:0/2281243278 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c38026910 con 0x55f5757791b0 2024-08-30T09:10:14.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.738+0000 7f7c53785740 1 -- 172.21.15.42:0/2281243278 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f5757791b0 msgr2=0x55f575b556e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:10:14.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.738+0000 7f7c53785740 1 --2- 172.21.15.42:0/2281243278 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f5757791b0 0x55f575b556e0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f7c38000c00 tx=0x7f7c3800f710 comp rx=0 tx=0).stop 2024-08-30T09:10:14.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.738+0000 7f7c53785740 1 -- 172.21.15.42:0/2281243278 shutdown_connections 2024-08-30T09:10:14.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.738+0000 7f7c53785740 1 --2- 172.21.15.42:0/2281243278 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f5757a2070 0x55f575b54050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:14.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.738+0000 7f7c53785740 1 --2- 172.21.15.42:0/2281243278 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f575781390 0x55f575b55c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:14.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.738+0000 7f7c53785740 1 --2- 172.21.15.42:0/2281243278 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f5757791b0 0x55f575b556e0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:14.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.738+0000 7f7c53785740 1 -- 172.21.15.42:0/2281243278 >> 172.21.15.42:0/2281243278 conn(0x55f575796460 msgr2=0x55f575b51fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:10:14.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.738+0000 7f7c53785740 1 -- 172.21.15.42:0/2281243278 shutdown_connections 2024-08-30T09:10:14.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.738+0000 7f7c53785740 1 -- 172.21.15.42:0/2281243278 wait complete. 2024-08-30T09:10:14.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.739+0000 7f7c53785740 1 -- start start 2024-08-30T09:10:14.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.739+0000 7f7c53785740 1 Processor -- start 2024-08-30T09:10:14.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.739+0000 7f7c53785740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f5757791b0 0x55f575b509a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:14.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.739+0000 7f7c53785740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f575781390 0x55f575b50ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:14.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.739+0000 7f7c53785740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f5757a2070 0x55f575b51420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:14.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.739+0000 7f7c53785740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f7c481acc70 con 0x55f5757791b0 2024-08-30T09:10:14.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.739+0000 7f7c53785740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f7c481ae0e0 con 0x55f575781390 2024-08-30T09:10:14.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.739+0000 7f7c53785740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f7c481ad340 con 0x55f5757a2070 2024-08-30T09:10:14.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.739+0000 7f7c53785740 10 client.0 _ll_register_callbacks cb 0x7f7c30009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:10:14.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.739+0000 7f7c46d9d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f5757791b0 0x55f575b509a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:10:14.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.739+0000 7f7c46d9d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f5757791b0 0x55f575b509a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:39506/0 (socket says 172.21.15.42:39506) 2024-08-30T09:10:14.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.739+0000 7f7c46d9d640 1 -- 172.21.15.42:0/277088431 learned_addr learned my addr 172.21.15.42:0/277088431 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:10:14.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.739+0000 7f7c4759e640 1 --2- 172.21.15.42:0/277088431 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f5757a2070 0x55f575b51420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:10:14.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.739+0000 7f7c3e59c640 1 --2- 172.21.15.42:0/277088431 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f575781390 0x55f575b50ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:10:14.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.740+0000 7f7c46d9d640 1 -- 172.21.15.42:0/277088431 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f5757a2070 msgr2=0x55f575b51420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:10:14.519 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.740+0000 7f7c46d9d640 1 --2- 172.21.15.42:0/277088431 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f5757a2070 0x55f575b51420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:14.519 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.740+0000 7f7c46d9d640 1 -- 172.21.15.42:0/277088431 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f575781390 msgr2=0x55f575b50ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:10:14.519 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.740+0000 7f7c46d9d640 1 --2- 172.21.15.42:0/277088431 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f575781390 0x55f575b50ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:14.519 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.740+0000 7f7c46d9d640 1 -- 172.21.15.42:0/277088431 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f575b6b550 con 0x55f5757791b0 2024-08-30T09:10:14.519 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.740+0000 7f7c46d9d640 1 --2- 172.21.15.42:0/277088431 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f5757791b0 0x55f575b509a0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f7c380263f0 tx=0x7f7c3801cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc74dde4e757b568 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:10:14.519 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.740+0000 7f7c367fc640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-08-30T09:10:14.519 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.740+0000 7f7c367fc640 1 -- 172.21.15.42:0/277088431 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c38033070 con 0x55f5757791b0 2024-08-30T09:10:14.519 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.740+0000 7f7c367fc640 1 -- 172.21.15.42:0/277088431 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f7c38005260 con 0x55f5757791b0 2024-08-30T09:10:14.519 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.740+0000 7f7c53785740 1 -- 172.21.15.42:0/277088431 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7c38025390 con 0x55f5757791b0 2024-08-30T09:10:14.519 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.740+0000 7f7c367fc640 1 -- 172.21.15.42:0/277088431 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f7c3804b050 con 0x55f5757791b0 2024-08-30T09:10:14.519 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.740+0000 7f7c53785740 10 client.17034 fetch_fsmap learned FSMap version 39 2024-08-30T09:10:14.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.740+0000 7f7c53785740 1 -- 172.21.15.42:0/277088431 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55f575bd8010 con 0x55f5757791b0 2024-08-30T09:10:14.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.741+0000 7f7c367fc640 1 -- 172.21.15.42:0/277088431 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7c38025390 con 0x55f5757791b0 2024-08-30T09:10:14.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.741+0000 7f7c367fc640 1 -- 172.21.15.42:0/277088431 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7f7c38046030 con 0x55f5757791b0 2024-08-30T09:10:14.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.741+0000 7f7c53785740 10 client.17034 fetch_fsmap finished waiting for FSMap version 39 2024-08-30T09:10:14.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.741+0000 7f7c53785740 10 client.17034 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-30T09:10:14.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.741+0000 7f7c53785740 20 client.17034 populate_metadata read hostname 'smithi042' 2024-08-30T09:10:14.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.741+0000 7f7c53785740 1 --2- 172.21.15.42:0/277088431 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55f575735da0 0x55f575bda710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:14.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.741+0000 7f7c53785740 4 client.17034 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-30T09:10:14.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.741+0000 7f7c53785740 1 -- 172.21.15.42:0/277088431 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f575abc020 con 0x55f575735da0 2024-08-30T09:10:14.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.741+0000 7f7c3e59c640 1 --2- 172.21.15.42:0/277088431 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55f575735da0 0x55f575bda710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:10:14.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.742+0000 7f7c367fc640 10 client.17034 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:10:14.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.742+0000 7f7c3e59c640 1 --2- 172.21.15.42:0/277088431 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55f575735da0 0x55f575bda710 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b2a151162f26913 server_cookie=bc93fe7fbe05dfa0 in_seq=0 out_seq=0 2024-08-30T09:10:14.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.742+0000 7f7c367fc640 1 -- 172.21.15.42:0/277088431 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3694 (crc 0 0 0) 0x55f575abc020 con 0x55f575735da0 2024-08-30T09:10:14.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.742+0000 7f7c367fc640 10 client.17034 handle_command_reply: tid=0 2024-08-30T09:10:14.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.742+0000 7f7c53785740 1 client.17034 shutdown 2024-08-30T09:10:14.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.742+0000 7f7c53785740 1 -- 172.21.15.42:0/277088431 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f5757791b0 msgr2=0x55f575b509a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:10:14.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.742+0000 7f7c53785740 1 --2- 172.21.15.42:0/277088431 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f5757791b0 0x55f575b509a0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f7c380263f0 tx=0x7f7c3801cd20 comp rx=0 tx=0).stop 2024-08-30T09:10:14.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.742+0000 7f7c53785740 1 -- 172.21.15.42:0/277088431 shutdown_connections 2024-08-30T09:10:14.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.742+0000 7f7c53785740 1 --2- 172.21.15.42:0/277088431 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55f575735da0 0x55f575bda710 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:14.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.742+0000 7f7c367fc640 0 client.17034 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:10:14.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.742+0000 7f7c53785740 1 --2- 172.21.15.42:0/277088431 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f5757a2070 0x55f575b51420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:14.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.742+0000 7f7c53785740 1 --2- 172.21.15.42:0/277088431 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f575781390 0x55f575b50ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:14.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.742+0000 7f7c53785740 1 --2- 172.21.15.42:0/277088431 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f5757791b0 0x55f575b509a0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:14.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.742+0000 7f7c53785740 1 -- 172.21.15.42:0/277088431 >> 172.21.15.42:0/277088431 conn(0x55f575796460 msgr2=0x55f575b56d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:10:14.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.743+0000 7f7c53785740 1 -- 172.21.15.42:0/277088431 shutdown_connections 2024-08-30T09:10:14.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.743+0000 7f7c53785740 1 -- 172.21.15.42:0/277088431 wait complete. 2024-08-30T09:10:14.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.744+0000 7f7c53785740 20 client.17034 trim_cache size 0 max 16384 2024-08-30T09:10:14.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.745+0000 7f7c4d210640 1 -- 172.21.15.42:0/108900750 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f7c280b4b30 msgr2=0x7f7c280b7040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:10:14.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.745+0000 7f7c4d210640 1 --2- 172.21.15.42:0/108900750 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f7c280b4b30 0x7f7c280b7040 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f7c38011500 tx=0x7f7c38005f90 comp rx=0 tx=0).stop 2024-08-30T09:10:14.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.745+0000 7f7c4d210640 1 -- 172.21.15.42:0/108900750 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7c48109bb0 msgr2=0x7f7c4819fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:10:14.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.745+0000 7f7c4d210640 1 --2- 172.21.15.42:0/108900750 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7c48109bb0 0x7f7c4819fa00 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f7c40009b30 tx=0x7f7c40003b80 comp rx=0 tx=0).stop 2024-08-30T09:10:14.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.745+0000 7f7c4d210640 1 -- 172.21.15.42:0/108900750 shutdown_connections 2024-08-30T09:10:14.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.745+0000 7f7c4d210640 1 --2- 172.21.15.42:0/108900750 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f7c280b4b30 0x7f7c280b7040 secure :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f7c38011500 tx=0x7f7c38005f90 comp rx=0 tx=0).stop 2024-08-30T09:10:14.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.745+0000 7f7c4d210640 1 --2- 172.21.15.42:0/108900750 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7c48109bb0 0x7f7c4819fa00 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:14.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.745+0000 7f7c4d210640 1 --2- 172.21.15.42:0/108900750 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7c48109260 0x7f7c4819a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:14.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.745+0000 7f7c4d210640 1 --2- 172.21.15.42:0/108900750 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7c48103920 0x7f7c4819a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:14.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.745+0000 7f7c4d210640 1 -- 172.21.15.42:0/108900750 >> 172.21.15.42:0/108900750 conn(0x7f7c480ff3b0 msgr2=0x7f7c48106810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:10:14.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.745+0000 7f7c4d210640 1 -- 172.21.15.42:0/108900750 shutdown_connections 2024-08-30T09:10:14.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:13.745+0000 7f7c4d210640 1 -- 172.21.15.42:0/108900750 wait complete. 2024-08-30T09:10:14.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:10:14 smithi167 ceph-mon[22063]: pgmap v657: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 5.4 MiB/s wr, 633 op/s 2024-08-30T09:10:14.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:10:14 smithi042 ceph-mon[18652]: pgmap v657: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 5.4 MiB/s wr, 633 op/s 2024-08-30T09:10:14.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:10:14 smithi190 ceph-mon[22001]: pgmap v657: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 5.4 MiB/s wr, 633 op/s 2024-08-30T09:10:15.634 INFO:tasks.cephfs.filesystem:scrub status for tag:a7210f9a-046b-4188-953c-819b9f1863a9 - {'path': '/', 'tag': 'a7210f9a-046b-4188-953c-819b9f1863a9', 'options': 'recursive,force'} 2024-08-30T09:10:15.953 INFO:teuthology.orchestra.run.smithi042.stderr:progress: 570.0 s, 500.6 tps, lat 212.210 ms stddev 253.045 2024-08-30T09:10:16.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:10:16 smithi167 ceph-mon[22063]: pgmap v658: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 4.6 MiB/s wr, 556 op/s 2024-08-30T09:10:16.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:10:16 smithi042 ceph-mon[18652]: pgmap v658: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 4.6 MiB/s wr, 556 op/s 2024-08-30T09:10:16.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:10:16 smithi190 ceph-mon[22001]: pgmap v658: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 4.6 MiB/s wr, 556 op/s 2024-08-30T09:10:18.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:10:18 smithi167 ceph-mon[22063]: pgmap v659: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 3.9 MiB/s wr, 519 op/s 2024-08-30T09:10:18.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:10:18 smithi167 ceph-mon[22063]: scrub summary: idle 2024-08-30T09:10:18.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:10:18 smithi167 ceph-mon[22063]: scrub summary: idle 2024-08-30T09:10:18.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:10:18 smithi042 ceph-mon[18652]: pgmap v659: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 3.9 MiB/s wr, 519 op/s 2024-08-30T09:10:18.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:10:18 smithi042 ceph-mon[18652]: scrub summary: idle 2024-08-30T09:10:18.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:10:18 smithi042 ceph-mon[18652]: scrub summary: idle 2024-08-30T09:10:18.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:10:18 smithi190 ceph-mon[22001]: pgmap v659: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 3.9 MiB/s wr, 519 op/s 2024-08-30T09:10:18.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:10:18 smithi190 ceph-mon[22001]: scrub summary: idle 2024-08-30T09:10:18.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:10:18 smithi190 ceph-mon[22001]: scrub summary: idle 2024-08-30T09:10:19.490 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:10:19 smithi167 ceph-mon[22063]: scrub summary: idle 2024-08-30T09:10:19.490 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:10:19 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-30T09:10:19.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:10:19 smithi042 ceph-mon[18652]: scrub summary: idle 2024-08-30T09:10:19.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:10:19 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-30T09:10:19.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:10:19 smithi190 ceph-mon[22001]: scrub summary: idle 2024-08-30T09:10:19.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:10:19 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-30T09:10:20.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:10:20 smithi167 ceph-mon[22063]: pgmap v660: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 4.8 MiB/s wr, 627 op/s 2024-08-30T09:10:20.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:10:20 smithi042 ceph-mon[18652]: pgmap v660: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 4.8 MiB/s wr, 627 op/s 2024-08-30T09:10:20.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:10:20 smithi190 ceph-mon[22001]: pgmap v660: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 4.8 MiB/s wr, 627 op/s 2024-08-30T09:10:20.953 INFO:teuthology.orchestra.run.smithi042.stderr:progress: 575.0 s, 451.6 tps, lat 213.636 ms stddev 238.816 2024-08-30T09:10:22.383 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.381+0000 7f3333fff640 1 -- 172.21.15.42:0/2911096753 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3334030c00 con 0x7f333c07a9a0 2024-08-30T09:10:22.384 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.381+0000 7f33435f8640 1 -- 172.21.15.42:0/2911096753 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f333c07a9a0 msgr2=0x7f333c07ceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:10:22.384 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.381+0000 7f33435f8640 1 --2- 172.21.15.42:0/2911096753 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f333c07a9a0 0x7f333c07ceb0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f3334009ca0 tx=0x7f333402a8b0 comp rx=0 tx=0).stop 2024-08-30T09:10:22.384 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.381+0000 7f33435f8640 1 -- 172.21.15.42:0/2911096753 shutdown_connections 2024-08-30T09:10:22.384 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.381+0000 7f33435f8640 1 --2- 172.21.15.42:0/2911096753 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f333c07a9a0 0x7f333c07ceb0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:22.384 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.381+0000 7f33435f8640 1 --2- 172.21.15.42:0/2911096753 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f333c078040 0x7f333c07a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:22.384 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.381+0000 7f33435f8640 1 --2- 172.21.15.42:0/2911096753 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f333c075280 0x7f333c0756f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:22.384 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.381+0000 7f33435f8640 1 -- 172.21.15.42:0/2911096753 >> 172.21.15.42:0/2911096753 conn(0x7f333c0705b0 msgr2=0x7f333c072a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:10:22.384 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.381+0000 7f33435f8640 1 -- 172.21.15.42:0/2911096753 shutdown_connections 2024-08-30T09:10:22.384 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.381+0000 7f33435f8640 1 -- 172.21.15.42:0/2911096753 wait complete. 2024-08-30T09:10:22.385 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.382+0000 7f33435f8640 1 Processor -- start 2024-08-30T09:10:22.387 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.385+0000 7f33435f8640 1 -- start start 2024-08-30T09:10:22.387 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.385+0000 7f33435f8640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f333c075280 0x7f333c1b9a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:22.387 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.385+0000 7f33435f8640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f333c078040 0x7f333c1ba100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:22.387 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.385+0000 7f33435f8640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f333c07a9a0 0x7f333c1bacb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:22.388 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.385+0000 7f33435f8640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f333c080680 con 0x7f333c075280 2024-08-30T09:10:22.388 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.385+0000 7f33435f8640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f333c080500 con 0x7f333c07a9a0 2024-08-30T09:10:22.388 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.385+0000 7f33435f8640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f333c080800 con 0x7f333c078040 2024-08-30T09:10:22.388 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.385+0000 7f3341b95640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f333c07a9a0 0x7f333c1bacb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:10:22.388 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.385+0000 7f3341b95640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f333c07a9a0 0x7f333c1bacb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:59200/0 (socket says 172.21.15.42:59200) 2024-08-30T09:10:22.388 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.385+0000 7f3341b95640 1 -- 172.21.15.42:0/983221375 learned_addr learned my addr 172.21.15.42:0/983221375 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:10:22.388 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.385+0000 7f3340b93640 1 --2- 172.21.15.42:0/983221375 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f333c078040 0x7f333c1ba100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:10:22.389 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.386+0000 7f3341b95640 1 -- 172.21.15.42:0/983221375 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f333c078040 msgr2=0x7f333c1ba100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:10:22.389 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.386+0000 7f3341b95640 1 --2- 172.21.15.42:0/983221375 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f333c078040 0x7f333c1ba100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:22.389 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.386+0000 7f3341b95640 1 -- 172.21.15.42:0/983221375 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f333c075280 msgr2=0x7f333c1b9a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:10:22.389 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.386+0000 7f3341b95640 1 --2- 172.21.15.42:0/983221375 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f333c075280 0x7f333c1b9a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:22.389 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.386+0000 7f3341b95640 1 -- 172.21.15.42:0/983221375 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f333c1bb4d0 con 0x7f333c07a9a0 2024-08-30T09:10:22.389 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.386+0000 7f3341b95640 1 --2- 172.21.15.42:0/983221375 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f333c07a9a0 0x7f333c1bacb0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f33340307a0 tx=0x7f3334003b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:10:22.390 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.387+0000 7f33327fc640 1 -- 172.21.15.42:0/983221375 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3334004240 con 0x7f333c07a9a0 2024-08-30T09:10:22.390 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.387+0000 7f33327fc640 1 -- 172.21.15.42:0/983221375 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3334040070 con 0x7f333c07a9a0 2024-08-30T09:10:22.390 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.387+0000 7f33327fc640 1 -- 172.21.15.42:0/983221375 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3334045bc0 con 0x7f333c07a9a0 2024-08-30T09:10:22.390 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.388+0000 7f33435f8640 1 -- 172.21.15.42:0/983221375 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f333c1bb760 con 0x7f333c07a9a0 2024-08-30T09:10:22.390 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.388+0000 7f33435f8640 1 -- 172.21.15.42:0/983221375 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f333c12c350 con 0x7f333c07a9a0 2024-08-30T09:10:22.396 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.394+0000 7f334a639740 1 Processor -- start 2024-08-30T09:10:22.396 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.394+0000 7f334a639740 1 -- start start 2024-08-30T09:10:22.396 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.394+0000 7f334a639740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ccccf8d1b0 0x55cccd333180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:22.397 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.394+0000 7f334a639740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ccccf642f0 0x55cccd3337f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:22.397 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.395+0000 7f334a639740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ccccf6c4d0 0x55cccd334340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:22.397 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.395+0000 7f334a639740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55cccd2a6000 con 0x55ccccf642f0 2024-08-30T09:10:22.397 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.395+0000 7f334a639740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55cccd295590 con 0x55ccccf8d1b0 2024-08-30T09:10:22.397 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.395+0000 7f334a639740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55cccd299540 con 0x55ccccf6c4d0 2024-08-30T09:10:22.397 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.395+0000 7f3340b93640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ccccf8d1b0 0x55cccd333180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:10:22.397 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.395+0000 7f3340b93640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ccccf8d1b0 0x55cccd333180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:59208/0 (socket says 172.21.15.42:59208) 2024-08-30T09:10:22.398 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.395+0000 7f3340b93640 1 -- 172.21.15.42:0/2186181748 learned_addr learned my addr 172.21.15.42:0/2186181748 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:10:22.398 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.395+0000 7f3340b93640 1 --2- 172.21.15.42:0/2186181748 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ccccf6c4d0 0x55cccd334340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:10:22.398 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.395+0000 7f3341394640 1 --2- 172.21.15.42:0/2186181748 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ccccf642f0 0x55cccd3337f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:10:22.398 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.396+0000 7f3340b93640 1 -- 172.21.15.42:0/2186181748 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ccccf6c4d0 msgr2=0x55cccd334340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:10:22.398 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.396+0000 7f3340b93640 1 --2- 172.21.15.42:0/2186181748 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ccccf6c4d0 0x55cccd334340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:22.398 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.396+0000 7f3340b93640 1 -- 172.21.15.42:0/2186181748 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ccccf642f0 msgr2=0x55cccd3337f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:10:22.398 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.396+0000 7f3340b93640 1 --2- 172.21.15.42:0/2186181748 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ccccf642f0 0x55cccd3337f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:22.398 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.396+0000 7f3340b93640 1 -- 172.21.15.42:0/2186181748 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ccccff8790 con 0x55ccccf8d1b0 2024-08-30T09:10:22.399 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.396+0000 7f3340b93640 1 --2- 172.21.15.42:0/2186181748 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ccccf8d1b0 0x55cccd333180 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f332c012820 tx=0x7f332c00b7e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5a9d66153cbde643 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:10:22.399 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.397+0000 7f33277fe640 1 -- 172.21.15.42:0/2186181748 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f332c012da0 con 0x55ccccf8d1b0 2024-08-30T09:10:22.399 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.398+0000 7f33277fe640 1 -- 172.21.15.42:0/2186181748 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f332c010c20 con 0x55ccccf8d1b0 2024-08-30T09:10:22.401 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.399+0000 7f334a639740 1 -- 172.21.15.42:0/2186181748 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ccccf8d1b0 msgr2=0x55cccd333180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:10:22.401 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.399+0000 7f334a639740 1 --2- 172.21.15.42:0/2186181748 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ccccf8d1b0 0x55cccd333180 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f332c012820 tx=0x7f332c00b7e0 comp rx=0 tx=0).stop 2024-08-30T09:10:22.403 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.401+0000 7f33327fc640 1 -- 172.21.15.42:0/983221375 <== mon.2 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7f3334045470 con 0x7f333c07a9a0 2024-08-30T09:10:22.403 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.402+0000 7f334a639740 1 -- 172.21.15.42:0/2186181748 shutdown_connections 2024-08-30T09:10:22.404 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.402+0000 7f334a639740 1 --2- 172.21.15.42:0/2186181748 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ccccf6c4d0 0x55cccd334340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:22.404 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.402+0000 7f334a639740 1 --2- 172.21.15.42:0/2186181748 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ccccf642f0 0x55cccd3337f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:22.404 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.402+0000 7f334a639740 1 --2- 172.21.15.42:0/2186181748 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ccccf8d1b0 0x55cccd333180 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:22.404 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.402+0000 7f334a639740 1 -- 172.21.15.42:0/2186181748 >> 172.21.15.42:0/2186181748 conn(0x55ccccf81300 msgr2=0x55cccd32ec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:10:22.404 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.402+0000 7f334a639740 1 -- 172.21.15.42:0/2186181748 shutdown_connections 2024-08-30T09:10:22.404 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.402+0000 7f334a639740 1 -- 172.21.15.42:0/2186181748 wait complete. 2024-08-30T09:10:22.405 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.403+0000 7f334a639740 1 -- start start 2024-08-30T09:10:22.405 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.403+0000 7f334a639740 1 Processor -- start 2024-08-30T09:10:22.406 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.404+0000 7f334a639740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ccccf642f0 0x55cccd33cca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:22.407 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.404+0000 7f334a639740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ccccf6c4d0 0x55cccd33d360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:22.407 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.404+0000 7f334a639740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ccccf8d1b0 0x55cccd33ded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:22.407 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.404+0000 7f334a639740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55cccd1ceab0 con 0x55ccccf6c4d0 2024-08-30T09:10:22.407 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.404+0000 7f334a639740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55cccd21c680 con 0x55ccccf8d1b0 2024-08-30T09:10:22.407 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.404+0000 7f334a639740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55cccd334eb0 con 0x55ccccf642f0 2024-08-30T09:10:22.407 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.404+0000 7f334a639740 10 client.0 _ll_register_callbacks cb 0x55ccccfa4a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:10:22.407 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.404+0000 7f3340b93640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ccccf8d1b0 0x55cccd33ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:10:22.407 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.404+0000 7f3340b93640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ccccf8d1b0 0x55cccd33ded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:59216/0 (socket says 172.21.15.42:59216) 2024-08-30T09:10:22.407 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.404+0000 7f3340b93640 1 -- 172.21.15.42:0/1686212070 learned_addr learned my addr 172.21.15.42:0/1686212070 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:10:22.407 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.404+0000 7f3340b93640 1 --2- 172.21.15.42:0/1686212070 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ccccf642f0 0x55cccd33cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:10:22.410 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.405+0000 7f3341394640 1 --2- 172.21.15.42:0/1686212070 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ccccf6c4d0 0x55cccd33d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:10:22.411 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.405+0000 7f3340b93640 1 -- 172.21.15.42:0/1686212070 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ccccf642f0 msgr2=0x55cccd33cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:10:22.411 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.405+0000 7f3340b93640 1 --2- 172.21.15.42:0/1686212070 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ccccf642f0 0x55cccd33cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:22.411 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.405+0000 7f3340b93640 1 -- 172.21.15.42:0/1686212070 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ccccf6c4d0 msgr2=0x55cccd33d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:10:22.411 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.405+0000 7f3340b93640 1 --2- 172.21.15.42:0/1686212070 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ccccf6c4d0 0x55cccd33d360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:22.411 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.405+0000 7f3340b93640 1 -- 172.21.15.42:0/1686212070 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cccd071620 con 0x55ccccf8d1b0 2024-08-30T09:10:22.411 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.405+0000 7f3340b93640 1 --2- 172.21.15.42:0/1686212070 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ccccf8d1b0 0x55cccd33ded0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f332c00b450 tx=0x7f332c014040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a7cc717e9f2f4360 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:10:22.411 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.408+0000 7f3324ff9640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-30T09:10:22.411 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.408+0000 7f3324ff9640 1 -- 172.21.15.42:0/1686212070 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f332c024040 con 0x55ccccf8d1b0 2024-08-30T09:10:22.411 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.408+0000 7f3324ff9640 1 -- 172.21.15.42:0/1686212070 <== mon.2 v2:172.21.15.167:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f332c010770 con 0x55ccccf8d1b0 2024-08-30T09:10:22.411 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.408+0000 7f334a639740 1 -- 172.21.15.42:0/1686212070 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f332c010c20 con 0x55ccccf8d1b0 2024-08-30T09:10:22.414 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.412+0000 7f33327fc640 1 --2- 172.21.15.42:0/983221375 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f33280b4bb0 0x7f33280b70c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:22.414 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.412+0000 7f33327fc640 1 -- 172.21.15.42:0/983221375 <== mon.2 v2:172.21.15.167:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f33341065c0 con 0x7f333c07a9a0 2024-08-30T09:10:22.414 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.413+0000 7f3324ff9640 1 -- 172.21.15.42:0/1686212070 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f332c010970 con 0x55ccccf8d1b0 2024-08-30T09:10:22.418 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.417+0000 7f334a639740 10 client.26852 fetch_fsmap learned FSMap version 39 2024-08-30T09:10:22.419 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.417+0000 7f334a639740 1 -- 172.21.15.42:0/1686212070 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55cccd346e20 con 0x55ccccf8d1b0 2024-08-30T09:10:22.421 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.419+0000 7f3341394640 1 --2- 172.21.15.42:0/983221375 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f33280b4bb0 0x7f33280b70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:10:22.421 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.419+0000 7f3324ff9640 1 -- 172.21.15.42:0/1686212070 <== mon.2 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f332c010c20 con 0x55ccccf8d1b0 2024-08-30T09:10:22.421 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.419+0000 7f3324ff9640 1 -- 172.21.15.42:0/1686212070 <== mon.2 v2:172.21.15.167:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7f332c02a040 con 0x55ccccf8d1b0 2024-08-30T09:10:22.427 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.425+0000 7f334a639740 10 client.26852 fetch_fsmap finished waiting for FSMap version 39 2024-08-30T09:10:22.427 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.425+0000 7f334a639740 10 client.26852 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-30T09:10:22.427 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.425+0000 7f334a639740 20 client.26852 populate_metadata read hostname 'smithi042' 2024-08-30T09:10:22.427 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.425+0000 7f334a639740 1 --2- 172.21.15.42:0/1686212070 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55ccccf4bd80 0x55cccd34a290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:22.427 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.425+0000 7f334a639740 4 client.26852 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-30T09:10:22.427 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.425+0000 7f334a639740 1 -- 172.21.15.42:0/1686212070 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cccd2a7f50 con 0x55ccccf4bd80 2024-08-30T09:10:22.429 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.425+0000 7f3340b93640 1 --2- 172.21.15.42:0/1686212070 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55ccccf4bd80 0x55cccd34a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:10:22.430 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.428+0000 7f3340b93640 1 --2- 172.21.15.42:0/1686212070 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55ccccf4bd80 0x55cccd34a290 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eba3839bc9a0614a server_cookie=c74d06fe06db5c58 in_seq=0 out_seq=0 2024-08-30T09:10:22.430 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.428+0000 7f3324ff9640 10 client.26852 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:10:22.431 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.429+0000 7f3324ff9640 1 -- 172.21.15.42:0/1686212070 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55cccd2a7f50 con 0x55ccccf4bd80 2024-08-30T09:10:22.431 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.429+0000 7f3324ff9640 10 client.26852 handle_command_reply: tid=0 2024-08-30T09:10:22.432 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.430+0000 7f334a639740 1 client.26852 shutdown 2024-08-30T09:10:22.434 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.431+0000 7f3341394640 1 --2- 172.21.15.42:0/983221375 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f33280b4bb0 0x7f33280b70c0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f3338009d10 tx=0x7f33380097d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:10:22.434 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.432+0000 7f334a639740 1 -- 172.21.15.42:0/1686212070 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ccccf8d1b0 msgr2=0x55cccd33ded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:10:22.434 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.432+0000 7f334a639740 1 --2- 172.21.15.42:0/1686212070 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ccccf8d1b0 0x55cccd33ded0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f332c00b450 tx=0x7f332c014040 comp rx=0 tx=0).stop 2024-08-30T09:10:22.434 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.432+0000 7f334a639740 1 -- 172.21.15.42:0/1686212070 shutdown_connections 2024-08-30T09:10:22.434 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.432+0000 7f334a639740 1 --2- 172.21.15.42:0/1686212070 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55ccccf4bd80 0x55cccd34a290 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:22.434 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.432+0000 7f334a639740 1 --2- 172.21.15.42:0/1686212070 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ccccf8d1b0 0x55cccd33ded0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:22.434 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.432+0000 7f334a639740 1 --2- 172.21.15.42:0/1686212070 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ccccf6c4d0 0x55cccd33d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:22.434 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.432+0000 7f334a639740 1 --2- 172.21.15.42:0/1686212070 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ccccf642f0 0x55cccd33cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:22.434 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.432+0000 7f334a639740 1 -- 172.21.15.42:0/1686212070 >> 172.21.15.42:0/1686212070 conn(0x55ccccf81300 msgr2=0x55cccd32d170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:10:22.434 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.432+0000 7f3324ff9640 0 client.26852 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:10:22.435 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.433+0000 7f334a639740 1 -- 172.21.15.42:0/1686212070 shutdown_connections 2024-08-30T09:10:22.435 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.433+0000 7f334a639740 1 -- 172.21.15.42:0/1686212070 wait complete. 2024-08-30T09:10:22.437 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.436+0000 7f334a639740 20 client.26852 trim_cache size 0 max 16384 2024-08-30T09:10:22.468 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.466+0000 7f334a639740 1 Processor -- start 2024-08-30T09:10:22.468 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.466+0000 7f334a639740 1 -- start start 2024-08-30T09:10:22.469 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.467+0000 7f334a639740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ccccf81300 0x55cccd33c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:22.469 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.467+0000 7f334a639740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ccccf642f0 0x55cccd32e180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:22.469 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.467+0000 7f334a639740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ccccf6c4d0 0x55cccd32e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:22.469 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.467+0000 7f334a639740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f333c1d73b0 con 0x55ccccf81300 2024-08-30T09:10:22.469 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.467+0000 7f334a639740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f333c1d4c20 con 0x55ccccf642f0 2024-08-30T09:10:22.470 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.467+0000 7f334a639740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f333c1347c0 con 0x55ccccf6c4d0 2024-08-30T09:10:22.470 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.467+0000 7f3341394640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ccccf81300 0x55cccd33c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:10:22.470 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.467+0000 7f3341394640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ccccf81300 0x55cccd33c4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:44984/0 (socket says 172.21.15.42:44984) 2024-08-30T09:10:22.470 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.467+0000 7f3341394640 1 -- 172.21.15.42:0/375139532 learned_addr learned my addr 172.21.15.42:0/375139532 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:10:22.470 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.467+0000 7f3341b95640 1 --2- 172.21.15.42:0/375139532 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ccccf6c4d0 0x55cccd32e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:10:22.470 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.468+0000 7f3341b95640 1 -- 172.21.15.42:0/375139532 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ccccf642f0 msgr2=0x55cccd32e180 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-30T09:10:22.470 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.468+0000 7f3341b95640 1 --2- 172.21.15.42:0/375139532 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ccccf642f0 0x55cccd32e180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:22.470 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.468+0000 7f3341b95640 1 -- 172.21.15.42:0/375139532 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ccccf81300 msgr2=0x55cccd33c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:10:22.471 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.468+0000 7f3341b95640 1 --2- 172.21.15.42:0/375139532 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ccccf81300 0x55cccd33c4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:22.471 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.468+0000 7f3341b95640 1 -- 172.21.15.42:0/375139532 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cccd328980 con 0x55ccccf6c4d0 2024-08-30T09:10:22.471 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.469+0000 7f3341b95640 1 --2- 172.21.15.42:0/375139532 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ccccf6c4d0 0x55cccd32e6c0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f33340ccb90 tx=0x7f33340cc370 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fd33b1a224db5655 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:10:22.472 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.470+0000 7f3326ffd640 1 -- 172.21.15.42:0/375139532 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33340ccc00 con 0x55ccccf6c4d0 2024-08-30T09:10:22.472 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.470+0000 7f3326ffd640 1 -- 172.21.15.42:0/375139532 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3334036030 con 0x55ccccf6c4d0 2024-08-30T09:10:22.472 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.470+0000 7f3326ffd640 1 -- 172.21.15.42:0/375139532 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3334106270 con 0x55ccccf6c4d0 2024-08-30T09:10:22.473 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.471+0000 7f334a639740 1 -- 172.21.15.42:0/375139532 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ccccf6c4d0 msgr2=0x55cccd32e6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:10:22.473 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.471+0000 7f334a639740 1 --2- 172.21.15.42:0/375139532 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ccccf6c4d0 0x55cccd32e6c0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f33340ccb90 tx=0x7f33340cc370 comp rx=0 tx=0).stop 2024-08-30T09:10:22.474 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.471+0000 7f334a639740 1 -- 172.21.15.42:0/375139532 shutdown_connections 2024-08-30T09:10:22.474 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.471+0000 7f334a639740 1 --2- 172.21.15.42:0/375139532 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ccccf6c4d0 0x55cccd32e6c0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:22.474 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.471+0000 7f334a639740 1 --2- 172.21.15.42:0/375139532 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ccccf642f0 0x55cccd32e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:22.474 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.471+0000 7f334a639740 1 --2- 172.21.15.42:0/375139532 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ccccf81300 0x55cccd33c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:22.474 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.471+0000 7f334a639740 1 -- 172.21.15.42:0/375139532 >> 172.21.15.42:0/375139532 conn(0x55cccd333a50 msgr2=0x55cccd33bd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:10:22.474 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.471+0000 7f334a639740 1 -- 172.21.15.42:0/375139532 shutdown_connections 2024-08-30T09:10:22.474 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.471+0000 7f334a639740 1 -- 172.21.15.42:0/375139532 wait complete. 2024-08-30T09:10:22.474 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.472+0000 7f334a639740 1 -- start start 2024-08-30T09:10:22.474 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.472+0000 7f334a639740 1 Processor -- start 2024-08-30T09:10:22.477 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.472+0000 7f334a639740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ccccf642f0 0x55cccd343800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:22.477 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.472+0000 7f334a639740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ccccf6c4d0 0x55cccd343d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:22.477 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.472+0000 7f334a639740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ccccf81300 0x55cccd3423c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:22.477 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.472+0000 7f334a639740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f333c1d3100 con 0x55ccccf81300 2024-08-30T09:10:22.477 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.472+0000 7f334a639740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f333c1d4550 con 0x55ccccf6c4d0 2024-08-30T09:10:22.477 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.472+0000 7f334a639740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f333c1d37b0 con 0x55ccccf642f0 2024-08-30T09:10:22.478 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.472+0000 7f334a639740 10 client.0 _ll_register_callbacks cb 0x7f332c00bef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:10:22.478 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.473+0000 7f3341b95640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ccccf81300 0x55cccd3423c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:10:22.478 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.473+0000 7f3340b93640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ccccf6c4d0 0x55cccd343d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:10:22.478 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.473+0000 7f3340b93640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ccccf6c4d0 0x55cccd343d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:59234/0 (socket says 172.21.15.42:59234) 2024-08-30T09:10:22.478 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.473+0000 7f3340b93640 1 -- 172.21.15.42:0/358473792 learned_addr learned my addr 172.21.15.42:0/358473792 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:10:22.478 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.473+0000 7f3341394640 1 --2- 172.21.15.42:0/358473792 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ccccf642f0 0x55cccd343800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:10:22.478 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.473+0000 7f3341b95640 1 -- 172.21.15.42:0/358473792 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ccccf642f0 msgr2=0x55cccd343800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:10:22.478 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.473+0000 7f3341b95640 1 --2- 172.21.15.42:0/358473792 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ccccf642f0 0x55cccd343800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:22.478 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.473+0000 7f3341b95640 1 -- 172.21.15.42:0/358473792 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ccccf6c4d0 msgr2=0x55cccd343d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:10:22.478 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.473+0000 7f3341b95640 1 --2- 172.21.15.42:0/358473792 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ccccf6c4d0 0x55cccd343d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:22.478 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.473+0000 7f3341b95640 1 -- 172.21.15.42:0/358473792 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cccd344560 con 0x55ccccf81300 2024-08-30T09:10:22.478 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.473+0000 7f3341394640 1 --2- 172.21.15.42:0/358473792 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ccccf642f0 0x55cccd343800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-30T09:10:22.479 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.473+0000 7f3341b95640 1 --2- 172.21.15.42:0/358473792 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ccccf81300 0x55cccd3423c0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x55cccd1cec00 tx=0x7f33340c8400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11b4fa4d3a6656b8 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:10:22.479 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.473+0000 7f3324ff9640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-08-30T09:10:22.479 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.474+0000 7f3324ff9640 1 -- 172.21.15.42:0/358473792 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33340ccc00 con 0x55ccccf81300 2024-08-30T09:10:22.479 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.474+0000 7f334a639740 1 -- 172.21.15.42:0/358473792 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3334036030 con 0x55ccccf81300 2024-08-30T09:10:22.479 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.474+0000 7f3324ff9640 1 -- 172.21.15.42:0/358473792 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f3334106270 con 0x55ccccf81300 2024-08-30T09:10:22.479 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.474+0000 7f3324ff9640 1 -- 172.21.15.42:0/358473792 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f33340d5f20 con 0x55ccccf81300 2024-08-30T09:10:22.479 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.474+0000 7f334a639740 10 client.17049 fetch_fsmap learned FSMap version 39 2024-08-30T09:10:22.479 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.474+0000 7f334a639740 1 -- 172.21.15.42:0/358473792 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55cccd3b51d0 con 0x55ccccf81300 2024-08-30T09:10:22.479 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.474+0000 7f3324ff9640 1 -- 172.21.15.42:0/358473792 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3334036030 con 0x55ccccf81300 2024-08-30T09:10:22.479 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.475+0000 7f3324ff9640 1 -- 172.21.15.42:0/358473792 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7f33340efce0 con 0x55ccccf81300 2024-08-30T09:10:22.479 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.475+0000 7f334a639740 10 client.17049 fetch_fsmap finished waiting for FSMap version 39 2024-08-30T09:10:22.479 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.475+0000 7f334a639740 10 client.17049 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-30T09:10:22.479 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.475+0000 7f334a639740 20 client.17049 populate_metadata read hostname 'smithi042' 2024-08-30T09:10:22.480 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.475+0000 7f334a639740 1 --2- 172.21.15.42:0/358473792 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55ccccf8d1b0 0x55cccd3b7950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:22.480 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.475+0000 7f334a639740 4 client.17049 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-30T09:10:22.480 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.475+0000 7f334a639740 1 -- 172.21.15.42:0/358473792 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55cccd2a39f0 con 0x55ccccf8d1b0 2024-08-30T09:10:22.480 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.475+0000 7f3341394640 1 --2- 172.21.15.42:0/358473792 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55ccccf8d1b0 0x55cccd3b7950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:10:22.480 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.475+0000 7f3324ff9640 10 client.17049 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:10:22.480 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.475+0000 7f3341394640 1 --2- 172.21.15.42:0/358473792 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55ccccf8d1b0 0x55cccd3b7950 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=94bd79ccf41b805f server_cookie=52e20975202ef66f in_seq=0 out_seq=0 2024-08-30T09:10:22.516 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.513+0000 7f3324ff9640 1 -- 172.21.15.42:0/358473792 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55cccd2a39f0 con 0x55ccccf8d1b0 2024-08-30T09:10:22.517 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.513+0000 7f3324ff9640 10 client.17049 handle_command_reply: tid=0 2024-08-30T09:10:22.517 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.513+0000 7f334a639740 1 client.17049 shutdown 2024-08-30T09:10:22.517 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.514+0000 7f334a639740 1 -- 172.21.15.42:0/358473792 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ccccf81300 msgr2=0x55cccd3423c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:10:22.517 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.514+0000 7f334a639740 1 --2- 172.21.15.42:0/358473792 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ccccf81300 0x55cccd3423c0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x55cccd1cec00 tx=0x7f33340c8400 comp rx=0 tx=0).stop 2024-08-30T09:10:22.517 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.514+0000 7f334a639740 1 -- 172.21.15.42:0/358473792 shutdown_connections 2024-08-30T09:10:22.517 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.514+0000 7f334a639740 1 --2- 172.21.15.42:0/358473792 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55ccccf8d1b0 0x55cccd3b7950 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:22.517 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.514+0000 7f334a639740 1 --2- 172.21.15.42:0/358473792 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ccccf81300 0x55cccd3423c0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:22.517 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.514+0000 7f3324ff9640 0 client.17049 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:10:22.517 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.514+0000 7f334a639740 1 --2- 172.21.15.42:0/358473792 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ccccf6c4d0 0x55cccd343d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:22.517 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.514+0000 7f334a639740 1 --2- 172.21.15.42:0/358473792 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ccccf642f0 0x55cccd343800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:22.517 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.514+0000 7f334a639740 1 -- 172.21.15.42:0/358473792 >> 172.21.15.42:0/358473792 conn(0x55cccd333a50 msgr2=0x55cccd33bd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:10:22.518 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.514+0000 7f334a639740 1 -- 172.21.15.42:0/358473792 shutdown_connections 2024-08-30T09:10:22.518 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.514+0000 7f334a639740 1 -- 172.21.15.42:0/358473792 wait complete. 2024-08-30T09:10:22.518 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.514+0000 7f334a639740 20 client.17049 trim_cache size 0 max 16384 2024-08-30T09:10:22.518 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-08-30T09:10:22.518 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2024-08-30T09:10:22.518 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-08-30T09:10:22.518 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-08-30T09:10:22.519 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.515+0000 7f33435f8640 1 -- 172.21.15.42:0/983221375 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f33280b4bb0 msgr2=0x7f33280b70c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:10:22.519 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.515+0000 7f33435f8640 1 --2- 172.21.15.42:0/983221375 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f33280b4bb0 0x7f33280b70c0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f3338009d10 tx=0x7f33380097d0 comp rx=0 tx=0).stop 2024-08-30T09:10:22.519 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.515+0000 7f33435f8640 1 -- 172.21.15.42:0/983221375 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f333c07a9a0 msgr2=0x7f333c1bacb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:10:22.519 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.515+0000 7f33435f8640 1 --2- 172.21.15.42:0/983221375 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f333c07a9a0 0x7f333c1bacb0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f33340307a0 tx=0x7f3334003b80 comp rx=0 tx=0).stop 2024-08-30T09:10:22.519 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.515+0000 7f33435f8640 1 -- 172.21.15.42:0/983221375 shutdown_connections 2024-08-30T09:10:22.520 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.515+0000 7f33435f8640 1 --2- 172.21.15.42:0/983221375 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f33280b4bb0 0x7f33280b70c0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:22.520 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.515+0000 7f33435f8640 1 --2- 172.21.15.42:0/983221375 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f333c078040 0x7f333c1ba100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:22.520 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.515+0000 7f33435f8640 1 --2- 172.21.15.42:0/983221375 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f333c07a9a0 0x7f333c1bacb0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:22.520 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.515+0000 7f33435f8640 1 --2- 172.21.15.42:0/983221375 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f333c075280 0x7f333c1b9a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:22.520 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.515+0000 7f33435f8640 1 -- 172.21.15.42:0/983221375 >> 172.21.15.42:0/983221375 conn(0x7f333c0705b0 msgr2=0x7f333c078e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:10:22.520 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.515+0000 7f33435f8640 1 -- 172.21.15.42:0/983221375 shutdown_connections 2024-08-30T09:10:22.520 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:22.515+0000 7f33435f8640 1 -- 172.21.15.42:0/983221375 wait complete. 2024-08-30T09:10:23.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:10:22 smithi042 ceph-mon[18652]: pgmap v661: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 3.6 MiB/s wr, 476 op/s 2024-08-30T09:10:23.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:10:22 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T09:10:23.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:10:22 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T09:10:23.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:10:22 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T09:10:23.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:10:22 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T09:10:23.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:10:22 smithi190 ceph-mon[22001]: pgmap v661: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 3.6 MiB/s wr, 476 op/s 2024-08-30T09:10:23.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:10:22 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T09:10:23.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:10:22 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T09:10:23.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:10:22 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T09:10:23.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:10:22 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T09:10:23.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:10:22 smithi167 ceph-mon[22063]: pgmap v661: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 3.6 MiB/s wr, 476 op/s 2024-08-30T09:10:23.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:10:22 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T09:10:23.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:10:22 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T09:10:23.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:10:22 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T09:10:23.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:10:22 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T09:10:25.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:10:24 smithi190 ceph-mon[22001]: pgmap v662: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 4.5 MiB/s wr, 581 op/s 2024-08-30T09:10:25.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:10:24 smithi167 ceph-mon[22063]: pgmap v662: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 4.5 MiB/s wr, 581 op/s 2024-08-30T09:10:25.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:10:24 smithi042 ceph-mon[18652]: pgmap v662: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 4.5 MiB/s wr, 581 op/s 2024-08-30T09:10:25.953 INFO:teuthology.orchestra.run.smithi042.stderr:progress: 580.0 s, 482.0 tps, lat 210.200 ms stddev 230.178 2024-08-30T09:10:26.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:10:26 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T09:10:26.437 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:10:26 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T09:10:26.437 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:10:26 smithi190 ceph-mon[22001]: pgmap v663: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 3.9 MiB/s wr, 448 op/s 2024-08-30T09:10:26.437 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:10:26 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T09:10:26.437 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:10:26 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-30T09:10:26.437 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:10:26 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T09:10:26.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:10:26 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T09:10:26.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:10:26 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T09:10:26.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:10:26 smithi167 ceph-mon[22063]: pgmap v663: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 3.9 MiB/s wr, 448 op/s 2024-08-30T09:10:26.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:10:26 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T09:10:26.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:10:26 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-30T09:10:26.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:10:26 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T09:10:26.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:10:26 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T09:10:26.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:10:26 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T09:10:26.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:10:26 smithi042 ceph-mon[18652]: pgmap v663: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 3.9 MiB/s wr, 448 op/s 2024-08-30T09:10:26.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:10:26 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T09:10:26.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:10:26 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-30T09:10:26.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:10:26 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T09:10:28.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:10:28 smithi167 ceph-mon[22063]: pgmap v664: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 3.8 MiB/s wr, 435 op/s 2024-08-30T09:10:28.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:10:28 smithi042 ceph-mon[18652]: pgmap v664: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 3.8 MiB/s wr, 435 op/s 2024-08-30T09:10:28.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:10:28 smithi190 ceph-mon[22001]: pgmap v664: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 3.8 MiB/s wr, 435 op/s 2024-08-30T09:10:30.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:10:30 smithi167 ceph-mon[22063]: pgmap v665: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 4.8 MiB/s wr, 540 op/s 2024-08-30T09:10:30.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:10:30 smithi042 ceph-mon[18652]: pgmap v665: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 4.8 MiB/s wr, 540 op/s 2024-08-30T09:10:30.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:10:30 smithi190 ceph-mon[22001]: pgmap v665: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 4.8 MiB/s wr, 540 op/s 2024-08-30T09:10:30.953 INFO:teuthology.orchestra.run.smithi042.stderr:progress: 585.0 s, 565.6 tps, lat 174.635 ms stddev 193.942 2024-08-30T09:10:32.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:10:32 smithi167 ceph-mon[22063]: pgmap v666: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 3.5 MiB/s wr, 398 op/s 2024-08-30T09:10:32.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:10:32 smithi042 ceph-mon[18652]: pgmap v666: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 3.5 MiB/s wr, 398 op/s 2024-08-30T09:10:32.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:10:32 smithi190 ceph-mon[22001]: pgmap v666: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 3.5 MiB/s wr, 398 op/s 2024-08-30T09:10:34.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:10:34 smithi167 ceph-mon[22063]: pgmap v667: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.8 MiB/s wr, 548 op/s 2024-08-30T09:10:34.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:10:34 smithi042 ceph-mon[18652]: pgmap v667: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.8 MiB/s wr, 548 op/s 2024-08-30T09:10:34.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:10:34 smithi190 ceph-mon[22001]: pgmap v667: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.8 MiB/s wr, 548 op/s 2024-08-30T09:10:35.691 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:35.685+0000 7fd16cd70640 1 -- 172.21.15.42:0/3781346261 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd168102350 msgr2=0x7fd1681027c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:10:35.691 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:35.685+0000 7fd16cd70640 1 --2- 172.21.15.42:0/3781346261 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd168102350 0x7fd1681027c0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fd15801acb0 tx=0x7fd15803b920 comp rx=0 tx=0).stop 2024-08-30T09:10:35.691 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:35.687+0000 7fd16cd70640 1 -- 172.21.15.42:0/3781346261 shutdown_connections 2024-08-30T09:10:35.691 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:35.687+0000 7fd16cd70640 1 --2- 172.21.15.42:0/3781346261 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd168103d70 0x7fd16810a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:35.691 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:35.687+0000 7fd16cd70640 1 --2- 172.21.15.42:0/3781346261 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd1681033d0 0x7fd1681037e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:35.691 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:35.687+0000 7fd16cd70640 1 --2- 172.21.15.42:0/3781346261 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd168102350 0x7fd1681027c0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:35.692 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:35.687+0000 7fd16cd70640 1 -- 172.21.15.42:0/3781346261 >> 172.21.15.42:0/3781346261 conn(0x7fd1680fdcd0 msgr2=0x7fd168100140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:10:35.692 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:35.690+0000 7fd16cd70640 1 -- 172.21.15.42:0/3781346261 shutdown_connections 2024-08-30T09:10:35.694 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:35.692+0000 7fd16cd70640 1 -- 172.21.15.42:0/3781346261 wait complete. 2024-08-30T09:10:35.695 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:35.694+0000 7fd16cd70640 1 Processor -- start 2024-08-30T09:10:35.696 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:35.694+0000 7fd16cd70640 1 -- start start 2024-08-30T09:10:35.696 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:35.694+0000 7fd16cd70640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd168102350 0x7fd168074830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:35.696 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:35.694+0000 7fd16cd70640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd1681033d0 0x7fd1680730f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:35.697 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:35.694+0000 7fd16cd70640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd168103d70 0x7fd168073630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:35.697 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:35.694+0000 7fd16cd70640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fd16806d460 con 0x7fd168103d70 2024-08-30T09:10:35.697 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:35.694+0000 7fd16cd70640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd16806d2e0 con 0x7fd1681033d0 2024-08-30T09:10:35.697 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:35.694+0000 7fd16cd70640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fd16806d5e0 con 0x7fd168102350 2024-08-30T09:10:35.698 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:35.695+0000 7fd166d9d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd168103d70 0x7fd168073630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:10:35.698 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:35.695+0000 7fd166d9d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd168103d70 0x7fd168073630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:57736/0 (socket says 172.21.15.42:57736) 2024-08-30T09:10:35.699 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:35.695+0000 7fd166d9d640 1 -- 172.21.15.42:0/3879698388 learned_addr learned my addr 172.21.15.42:0/3879698388 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:10:35.700 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:35.698+0000 7fd165d9b640 1 --2- 172.21.15.42:0/3879698388 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd1681033d0 0x7fd1680730f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:10:35.700 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:35.698+0000 7fd16659c640 1 --2- 172.21.15.42:0/3879698388 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd168102350 0x7fd168074830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:10:35.702 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:35.700+0000 7fd166d9d640 1 -- 172.21.15.42:0/3879698388 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd168102350 msgr2=0x7fd168074830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:10:35.702 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:35.700+0000 7fd166d9d640 1 --2- 172.21.15.42:0/3879698388 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd168102350 0x7fd168074830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:35.702 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:35.700+0000 7fd166d9d640 1 -- 172.21.15.42:0/3879698388 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd1681033d0 msgr2=0x7fd1680730f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:10:35.703 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:35.700+0000 7fd166d9d640 1 --2- 172.21.15.42:0/3879698388 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd1681033d0 0x7fd1680730f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:35.703 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:35.700+0000 7fd166d9d640 1 -- 172.21.15.42:0/3879698388 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd168073ba0 con 0x7fd168103d70 2024-08-30T09:10:35.706 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:35.704+0000 7fd166d9d640 1 --2- 172.21.15.42:0/3879698388 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd168103d70 0x7fd168073630 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fd16000c970 tx=0x7fd16000cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:10:35.708 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:35.706+0000 7fd1577fe640 1 -- 172.21.15.42:0/3879698388 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd16000ae60 con 0x7fd168103d70 2024-08-30T09:10:35.709 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:35.706+0000 7fd16cd70640 1 -- 172.21.15.42:0/3879698388 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd168073e90 con 0x7fd168103d70 2024-08-30T09:10:35.709 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:35.706+0000 7fd16cd70640 1 -- 172.21.15.42:0/3879698388 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1681a8710 con 0x7fd168103d70 2024-08-30T09:10:35.715 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:35.712+0000 7fd173cdb740 1 Processor -- start 2024-08-30T09:10:35.715 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:35.712+0000 7fd173cdb740 1 -- start start 2024-08-30T09:10:35.715 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:35.713+0000 7fd173cdb740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555c43b9d1b0 0x555c43f432a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:35.716 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:35.713+0000 7fd173cdb740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555c43b742f0 0x555c43f43910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:35.716 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:35.713+0000 7fd173cdb740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555c43b7c4d0 0x555c43f44460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:35.716 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:35.713+0000 7fd173cdb740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x555c43eb6000 con 0x555c43b9d1b0 2024-08-30T09:10:35.716 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:35.713+0000 7fd173cdb740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x555c43ea5590 con 0x555c43b7c4d0 2024-08-30T09:10:35.716 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:35.713+0000 7fd173cdb740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x555c43ea9540 con 0x555c43b742f0 2024-08-30T09:10:35.716 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:35.713+0000 7fd1577fe640 1 -- 172.21.15.42:0/3879698388 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd160005020 con 0x7fd168103d70 2024-08-30T09:10:35.716 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:35.713+0000 7fd1577fe640 1 -- 172.21.15.42:0/3879698388 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd160004d00 con 0x7fd168103d70 2024-08-30T09:10:35.716 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:35.713+0000 7fd1577fe640 1 -- 172.21.15.42:0/3879698388 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7fd16000f450 con 0x7fd168103d70 2024-08-30T09:10:35.716 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:35.713+0000 7fd16659c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555c43b742f0 0x555c43f43910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:10:35.716 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:35.713+0000 7fd16659c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555c43b742f0 0x555c43f43910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:34186/0 (socket says 172.21.15.42:34186) 2024-08-30T09:10:35.716 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:35.713+0000 7fd16659c640 1 -- 172.21.15.42:0/3574620631 learned_addr learned my addr 172.21.15.42:0/3574620631 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:10:35.716 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:35.713+0000 7fd165d9b640 1 --2- 172.21.15.42:0/3574620631 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555c43b7c4d0 0x555c43f44460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:10:35.717 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:35.713+0000 7fd165d9b640 1 --2- 172.21.15.42:0/3574620631 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555c43b9d1b0 0x555c43f432a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:10:35.717 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:35.714+0000 7fd16659c640 1 -- 172.21.15.42:0/3574620631 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555c43b7c4d0 msgr2=0x555c43f44460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:10:35.717 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:35.714+0000 7fd16659c640 1 --2- 172.21.15.42:0/3574620631 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555c43b7c4d0 0x555c43f44460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:35.717 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:35.714+0000 7fd16659c640 1 -- 172.21.15.42:0/3574620631 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555c43b9d1b0 msgr2=0x555c43f432a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:10:35.717 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:35.714+0000 7fd16659c640 1 --2- 172.21.15.42:0/3574620631 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555c43b9d1b0 0x555c43f432a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:35.717 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:35.715+0000 7fd16659c640 1 -- 172.21.15.42:0/3574620631 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555c43c08790 con 0x555c43b742f0 2024-08-30T09:10:35.717 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:35.715+0000 7fd165d9b640 1 --2- 172.21.15.42:0/3574620631 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555c43b9d1b0 0x555c43f432a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-30T09:10:35.720 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:35.718+0000 7fd16659c640 1 --2- 172.21.15.42:0/3574620631 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555c43b742f0 0x555c43f43910 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fd15803be90 tx=0x7fd158040440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a9a2c223d26703f4 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:10:35.726 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:35.722+0000 7fd1577fe640 1 --2- 172.21.15.42:0/3879698388 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fd1500b4d70 0x7fd1500b7280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:35.726 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:35.722+0000 7fd1577fe640 1 -- 172.21.15.42:0/3879698388 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7fd1600e6070 con 0x7fd168103d70 2024-08-30T09:10:35.727 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:35.723+0000 7fd14ffff640 1 -- 172.21.15.42:0/3574620631 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd158045070 con 0x555c43b742f0 2024-08-30T09:10:35.727 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:35.723+0000 7fd14ffff640 1 -- 172.21.15.42:0/3574620631 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd158040900 con 0x555c43b742f0 2024-08-30T09:10:35.727 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:35.724+0000 7fd173cdb740 1 -- 172.21.15.42:0/3574620631 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555c43b742f0 msgr2=0x555c43f43910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:10:35.727 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:35.724+0000 7fd173cdb740 1 --2- 172.21.15.42:0/3574620631 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555c43b742f0 0x555c43f43910 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fd15803be90 tx=0x7fd158040440 comp rx=0 tx=0).stop 2024-08-30T09:10:35.730 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:35.729+0000 7fd173cdb740 1 -- 172.21.15.42:0/3574620631 shutdown_connections 2024-08-30T09:10:35.731 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:35.729+0000 7fd173cdb740 1 --2- 172.21.15.42:0/3574620631 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555c43b742f0 0x555c43f43910 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:35.731 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:35.729+0000 7fd173cdb740 1 --2- 172.21.15.42:0/3574620631 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555c43b7c4d0 0x555c43f44460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:35.731 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:35.729+0000 7fd173cdb740 1 --2- 172.21.15.42:0/3574620631 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555c43b9d1b0 0x555c43f432a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:35.731 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:35.729+0000 7fd173cdb740 1 -- 172.21.15.42:0/3574620631 >> 172.21.15.42:0/3574620631 conn(0x555c43b91300 msgr2=0x555c43f3ed30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:10:35.732 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:35.729+0000 7fd165d9b640 1 --2- 172.21.15.42:0/3879698388 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fd1500b4d70 0x7fd1500b7280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:10:35.732 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:35.730+0000 7fd173cdb740 1 -- 172.21.15.42:0/3574620631 shutdown_connections 2024-08-30T09:10:35.736 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:35.734+0000 7fd173cdb740 1 -- 172.21.15.42:0/3574620631 wait complete. 2024-08-30T09:10:35.737 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:35.734+0000 7fd173cdb740 1 -- start start 2024-08-30T09:10:35.737 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:35.734+0000 7fd173cdb740 1 Processor -- start 2024-08-30T09:10:35.737 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:35.735+0000 7fd173cdb740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555c43b742f0 0x555c43f4c990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:35.737 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:35.735+0000 7fd173cdb740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555c43b7c4d0 0x555c43f4d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:35.737 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:35.735+0000 7fd173cdb740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555c43b9d1b0 0x555c43f4dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:35.737 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:35.735+0000 7fd173cdb740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x555c43ddeab0 con 0x555c43b9d1b0 2024-08-30T09:10:35.737 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:35.735+0000 7fd173cdb740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x555c43e2c680 con 0x555c43b7c4d0 2024-08-30T09:10:35.738 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:35.735+0000 7fd173cdb740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x555c43f44ee0 con 0x555c43b742f0 2024-08-30T09:10:35.738 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:35.735+0000 7fd173cdb740 10 client.0 _ll_register_callbacks cb 0x555c43bb4a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:10:35.738 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:35.736+0000 7fd166d9d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555c43b9d1b0 0x555c43f4dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:10:35.739 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:35.736+0000 7fd166d9d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555c43b9d1b0 0x555c43f4dbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:57742/0 (socket says 172.21.15.42:57742) 2024-08-30T09:10:35.739 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:35.736+0000 7fd166d9d640 1 -- 172.21.15.42:0/2791963764 learned_addr learned my addr 172.21.15.42:0/2791963764 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:10:35.742 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:35.738+0000 7fd16659c640 1 --2- 172.21.15.42:0/2791963764 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555c43b742f0 0x555c43f4c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:10:35.742 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:35.740+0000 7fd165d9b640 1 --2- 172.21.15.42:0/2791963764 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555c43b7c4d0 0x555c43f4d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:10:35.742 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:35.740+0000 7fd165d9b640 1 --2- 172.21.15.42:0/3879698388 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fd1500b4d70 0x7fd1500b7280 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fd15c006aa0 tx=0x7fd15c009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:10:35.743 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:35.740+0000 7fd16659c640 1 -- 172.21.15.42:0/2791963764 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555c43b7c4d0 msgr2=0x555c43f4d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:10:35.743 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:35.740+0000 7fd16659c640 1 --2- 172.21.15.42:0/2791963764 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555c43b7c4d0 0x555c43f4d050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:35.743 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:35.740+0000 7fd16659c640 1 -- 172.21.15.42:0/2791963764 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555c43b9d1b0 msgr2=0x555c43f4dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:10:35.743 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:35.740+0000 7fd16659c640 1 --2- 172.21.15.42:0/2791963764 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555c43b9d1b0 0x555c43f4dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:35.743 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:35.740+0000 7fd16659c640 1 -- 172.21.15.42:0/2791963764 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555c43c81620 con 0x555c43b742f0 2024-08-30T09:10:35.743 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:35.741+0000 7fd16659c640 1 --2- 172.21.15.42:0/2791963764 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555c43b742f0 0x555c43f4c990 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fd15803bf10 tx=0x7fd158003c60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=edce6b8a5eb6bb9a server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:10:35.743 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:35.741+0000 7fd14dffb640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-08-30T09:10:35.748 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:35.746+0000 7fd14dffb640 1 -- 172.21.15.42:0/2791963764 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd158045040 con 0x555c43b742f0 2024-08-30T09:10:35.748 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:35.746+0000 7fd173cdb740 1 -- 172.21.15.42:0/2791963764 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd158040900 con 0x555c43b742f0 2024-08-30T09:10:35.748 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:35.746+0000 7fd14dffb640 1 -- 172.21.15.42:0/2791963764 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7fd158058040 con 0x555c43b742f0 2024-08-30T09:10:35.753 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:35.751+0000 7fd14dffb640 1 -- 172.21.15.42:0/2791963764 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fd158040cc0 con 0x555c43b742f0 2024-08-30T09:10:35.753 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:35.751+0000 7fd173cdb740 10 client.26848 fetch_fsmap learned FSMap version 39 2024-08-30T09:10:35.753 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:35.751+0000 7fd173cdb740 1 -- 172.21.15.42:0/2791963764 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x555c43f56a80 con 0x555c43b742f0 2024-08-30T09:10:35.754 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:35.752+0000 7fd14dffb640 1 -- 172.21.15.42:0/2791963764 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd158040900 con 0x555c43b742f0 2024-08-30T09:10:35.754 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:35.752+0000 7fd14dffb640 1 -- 172.21.15.42:0/2791963764 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7fd158051070 con 0x555c43b742f0 2024-08-30T09:10:35.754 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:35.752+0000 7fd173cdb740 10 client.26848 fetch_fsmap finished waiting for FSMap version 39 2024-08-30T09:10:35.754 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:35.752+0000 7fd173cdb740 10 client.26848 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-30T09:10:35.754 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:35.752+0000 7fd173cdb740 20 client.26848 populate_metadata read hostname 'smithi042' 2024-08-30T09:10:35.754 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:35.752+0000 7fd173cdb740 1 --2- 172.21.15.42:0/2791963764 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x555c43b5bd80 0x555c43f5a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:35.754 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:35.752+0000 7fd173cdb740 4 client.26848 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-30T09:10:35.754 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:35.752+0000 7fd173cdb740 1 -- 172.21.15.42:0/2791963764 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555c43eb7f50 con 0x555c43b5bd80 2024-08-30T09:10:35.755 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:35.753+0000 7fd165d9b640 1 --2- 172.21.15.42:0/2791963764 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x555c43b5bd80 0x555c43f5a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:10:35.756 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:35.754+0000 7fd165d9b640 1 --2- 172.21.15.42:0/2791963764 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x555c43b5bd80 0x555c43f5a350 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b24d695027472535 server_cookie=82a7700a72bdd273 in_seq=0 out_seq=0 2024-08-30T09:10:35.756 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:35.754+0000 7fd14dffb640 10 client.26848 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:10:35.757 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:35.755+0000 7fd14dffb640 1 -- 172.21.15.42:0/2791963764 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x555c43eb7f50 con 0x555c43b5bd80 2024-08-30T09:10:35.757 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:35.755+0000 7fd14dffb640 10 client.26848 handle_command_reply: tid=0 2024-08-30T09:10:35.757 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:35.755+0000 7fd173cdb740 1 client.26848 shutdown 2024-08-30T09:10:35.762 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:35.760+0000 7fd173cdb740 1 -- 172.21.15.42:0/2791963764 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555c43b742f0 msgr2=0x555c43f4c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:10:35.762 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:35.760+0000 7fd173cdb740 1 --2- 172.21.15.42:0/2791963764 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555c43b742f0 0x555c43f4c990 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fd15803bf10 tx=0x7fd158003c60 comp rx=0 tx=0).stop 2024-08-30T09:10:35.774 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:35.770+0000 7fd173cdb740 1 -- 172.21.15.42:0/2791963764 shutdown_connections 2024-08-30T09:10:35.774 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:35.770+0000 7fd173cdb740 1 --2- 172.21.15.42:0/2791963764 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x555c43b5bd80 0x555c43f5a350 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:35.774 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:35.770+0000 7fd173cdb740 1 --2- 172.21.15.42:0/2791963764 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555c43b9d1b0 0x555c43f4dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:35.774 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:35.770+0000 7fd173cdb740 1 --2- 172.21.15.42:0/2791963764 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555c43b7c4d0 0x555c43f4d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:35.774 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:35.770+0000 7fd173cdb740 1 --2- 172.21.15.42:0/2791963764 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555c43b742f0 0x555c43f4c990 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:35.774 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:35.770+0000 7fd173cdb740 1 -- 172.21.15.42:0/2791963764 >> 172.21.15.42:0/2791963764 conn(0x555c43b91300 msgr2=0x555c43f3d330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:10:35.775 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:35.770+0000 7fd14dffb640 0 client.26848 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:10:35.775 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:35.770+0000 7fd173cdb740 1 -- 172.21.15.42:0/2791963764 shutdown_connections 2024-08-30T09:10:35.775 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:35.770+0000 7fd173cdb740 1 -- 172.21.15.42:0/2791963764 wait complete. 2024-08-30T09:10:35.775 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:35.770+0000 7fd173cdb740 20 client.26848 trim_cache size 0 max 16384 2024-08-30T09:10:35.808 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:35.805+0000 7fd173cdb740 1 Processor -- start 2024-08-30T09:10:35.808 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:35.805+0000 7fd173cdb740 1 -- start start 2024-08-30T09:10:35.808 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:35.805+0000 7fd173cdb740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555c43b742f0 0x555c43f4a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:35.808 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:35.805+0000 7fd173cdb740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555c43b7c4d0 0x555c43f5eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:35.808 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:35.805+0000 7fd173cdb740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555c43b9d1b0 0x555c43f5f040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:35.809 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:35.805+0000 7fd173cdb740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fd148012600 con 0x555c43b742f0 2024-08-30T09:10:35.809 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:35.805+0000 7fd173cdb740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd14800fe70 con 0x555c43b7c4d0 2024-08-30T09:10:35.809 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:35.805+0000 7fd173cdb740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fd14800dcc0 con 0x555c43b9d1b0 2024-08-30T09:10:35.809 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:35.806+0000 7fd16659c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555c43b742f0 0x555c43f4a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:10:35.809 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:35.806+0000 7fd16659c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555c43b742f0 0x555c43f4a270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:57756/0 (socket says 172.21.15.42:57756) 2024-08-30T09:10:35.809 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:35.806+0000 7fd16659c640 1 -- 172.21.15.42:0/2379314709 learned_addr learned my addr 172.21.15.42:0/2379314709 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:10:35.809 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:35.806+0000 7fd166d9d640 1 --2- 172.21.15.42:0/2379314709 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555c43b9d1b0 0x555c43f5f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:10:35.809 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:35.806+0000 7fd16659c640 1 -- 172.21.15.42:0/2379314709 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555c43b9d1b0 msgr2=0x555c43f5f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:10:35.809 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:35.806+0000 7fd16659c640 1 --2- 172.21.15.42:0/2379314709 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555c43b9d1b0 0x555c43f5f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:35.809 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:35.806+0000 7fd16659c640 1 -- 172.21.15.42:0/2379314709 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555c43b7c4d0 msgr2=0x555c43f5eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:10:35.809 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:35.806+0000 7fd165d9b640 1 --2- 172.21.15.42:0/2379314709 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555c43b7c4d0 0x555c43f5eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:10:35.809 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:35.806+0000 7fd16659c640 1 --2- 172.21.15.42:0/2379314709 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555c43b7c4d0 0x555c43f5eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:35.809 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:35.806+0000 7fd16659c640 1 -- 172.21.15.42:0/2379314709 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555c43f433f0 con 0x555c43b742f0 2024-08-30T09:10:35.810 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:35.806+0000 7fd16659c640 1 --2- 172.21.15.42:0/2379314709 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555c43b742f0 0x555c43f4a270 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fd158000f80 tx=0x7fd158004920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca4e8c25ac6f1358 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:10:35.810 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:35.806+0000 7fd14f7fe640 1 -- 172.21.15.42:0/2379314709 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd15806c070 con 0x555c43b742f0 2024-08-30T09:10:35.810 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:35.806+0000 7fd14f7fe640 1 -- 172.21.15.42:0/2379314709 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd158060070 con 0x555c43b742f0 2024-08-30T09:10:35.810 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:35.806+0000 7fd14f7fe640 1 -- 172.21.15.42:0/2379314709 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd158045070 con 0x555c43b742f0 2024-08-30T09:10:35.814 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:35.807+0000 7fd173cdb740 1 -- 172.21.15.42:0/2379314709 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555c43b742f0 msgr2=0x555c43f4a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:10:35.814 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:35.807+0000 7fd173cdb740 1 --2- 172.21.15.42:0/2379314709 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555c43b742f0 0x555c43f4a270 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fd158000f80 tx=0x7fd158004920 comp rx=0 tx=0).stop 2024-08-30T09:10:35.814 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:35.807+0000 7fd173cdb740 1 -- 172.21.15.42:0/2379314709 shutdown_connections 2024-08-30T09:10:35.814 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:35.807+0000 7fd173cdb740 1 --2- 172.21.15.42:0/2379314709 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555c43b9d1b0 0x555c43f5f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:35.814 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:35.807+0000 7fd173cdb740 1 --2- 172.21.15.42:0/2379314709 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555c43b7c4d0 0x555c43f5eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:35.814 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:35.807+0000 7fd173cdb740 1 --2- 172.21.15.42:0/2379314709 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555c43b742f0 0x555c43f4a270 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:35.815 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:35.807+0000 7fd173cdb740 1 -- 172.21.15.42:0/2379314709 >> 172.21.15.42:0/2379314709 conn(0x555c43b91300 msgr2=0x555c43f3db00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:10:35.815 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:35.807+0000 7fd173cdb740 1 -- 172.21.15.42:0/2379314709 shutdown_connections 2024-08-30T09:10:35.815 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:35.807+0000 7fd173cdb740 1 -- 172.21.15.42:0/2379314709 wait complete. 2024-08-30T09:10:35.815 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:35.808+0000 7fd173cdb740 1 -- start start 2024-08-30T09:10:35.815 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:35.808+0000 7fd173cdb740 1 Processor -- start 2024-08-30T09:10:35.815 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:35.808+0000 7fd173cdb740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555c43b742f0 0x555c43f539f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:35.815 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:35.808+0000 7fd173cdb740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555c43b7c4d0 0x555c43f540b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:35.815 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:35.808+0000 7fd173cdb740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555c43b9d1b0 0x555c43fbce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:35.815 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:35.808+0000 7fd173cdb740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fd14800e330 con 0x555c43b742f0 2024-08-30T09:10:35.815 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:35.808+0000 7fd173cdb740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd14800f7a0 con 0x555c43b9d1b0 2024-08-30T09:10:35.815 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:35.808+0000 7fd16659c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555c43b742f0 0x555c43f539f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:10:35.815 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:35.808+0000 7fd173cdb740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fd14800ea00 con 0x555c43b7c4d0 2024-08-30T09:10:35.815 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:35.808+0000 7fd16659c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555c43b742f0 0x555c43f539f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:57772/0 (socket says 172.21.15.42:57772) 2024-08-30T09:10:35.815 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:35.808+0000 7fd16659c640 1 -- 172.21.15.42:0/2164855656 learned_addr learned my addr 172.21.15.42:0/2164855656 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:10:35.815 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:35.809+0000 7fd173cdb740 10 client.0 _ll_register_callbacks cb 0x7fd158040370 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:10:35.816 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:35.809+0000 7fd166d9d640 1 --2- 172.21.15.42:0/2164855656 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555c43b9d1b0 0x555c43fbce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:10:35.816 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:35.809+0000 7fd165d9b640 1 --2- 172.21.15.42:0/2164855656 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555c43b7c4d0 0x555c43f540b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:10:35.816 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:35.809+0000 7fd16659c640 1 -- 172.21.15.42:0/2164855656 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555c43b7c4d0 msgr2=0x555c43f540b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:10:35.816 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:35.809+0000 7fd16659c640 1 --2- 172.21.15.42:0/2164855656 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555c43b7c4d0 0x555c43f540b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:35.816 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:35.809+0000 7fd16659c640 1 -- 172.21.15.42:0/2164855656 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555c43b9d1b0 msgr2=0x555c43fbce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:10:35.816 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:35.809+0000 7fd16659c640 1 --2- 172.21.15.42:0/2164855656 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555c43b9d1b0 0x555c43fbce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:35.816 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:35.809+0000 7fd16659c640 1 -- 172.21.15.42:0/2164855656 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555c43fbd5c0 con 0x555c43b742f0 2024-08-30T09:10:35.816 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:35.809+0000 7fd166d9d640 1 --2- 172.21.15.42:0/2164855656 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555c43b9d1b0 0x555c43fbce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-30T09:10:35.816 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:35.809+0000 7fd165d9b640 1 --2- 172.21.15.42:0/2164855656 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555c43b7c4d0 0x555c43f540b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-30T09:10:35.816 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:35.809+0000 7fd16659c640 1 --2- 172.21.15.42:0/2164855656 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555c43b742f0 0x555c43f539f0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fd1580538c0 tx=0x7fd15801a480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc04b64f62aac33e server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:10:35.816 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:35.809+0000 7fd14d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-08-30T09:10:35.816 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:35.809+0000 7fd14d7fa640 1 -- 172.21.15.42:0/2164855656 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd15806c070 con 0x555c43b742f0 2024-08-30T09:10:35.816 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:35.809+0000 7fd14d7fa640 1 -- 172.21.15.42:0/2164855656 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7fd158003f50 con 0x555c43b742f0 2024-08-30T09:10:35.817 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:35.809+0000 7fd173cdb740 1 -- 172.21.15.42:0/2164855656 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd158060070 con 0x555c43b742f0 2024-08-30T09:10:35.817 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:35.809+0000 7fd14d7fa640 1 -- 172.21.15.42:0/2164855656 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fd158079050 con 0x555c43b742f0 2024-08-30T09:10:35.817 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:35.809+0000 7fd173cdb740 10 client.17061 fetch_fsmap learned FSMap version 39 2024-08-30T09:10:35.817 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:35.809+0000 7fd173cdb740 1 -- 172.21.15.42:0/2164855656 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x555c43fc48d0 con 0x555c43b742f0 2024-08-30T09:10:35.817 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:35.810+0000 7fd14d7fa640 1 -- 172.21.15.42:0/2164855656 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd158005c70 con 0x555c43b742f0 2024-08-30T09:10:35.817 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:35.810+0000 7fd14d7fa640 1 -- 172.21.15.42:0/2164855656 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7fd158051040 con 0x555c43b742f0 2024-08-30T09:10:35.817 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:35.810+0000 7fd173cdb740 10 client.17061 fetch_fsmap finished waiting for FSMap version 39 2024-08-30T09:10:35.817 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:35.810+0000 7fd173cdb740 10 client.17061 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-30T09:10:35.817 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:35.810+0000 7fd173cdb740 20 client.17061 populate_metadata read hostname 'smithi042' 2024-08-30T09:10:35.817 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:35.810+0000 7fd173cdb740 1 --2- 172.21.15.42:0/2164855656 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x555c43b311c0 0x555c43fc7200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:35.817 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:35.810+0000 7fd173cdb740 4 client.17061 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-30T09:10:35.817 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:35.810+0000 7fd173cdb740 1 -- 172.21.15.42:0/2164855656 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7fd158060070 con 0x555c43b311c0 2024-08-30T09:10:35.817 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:35.810+0000 7fd165d9b640 1 --2- 172.21.15.42:0/2164855656 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x555c43b311c0 0x555c43fc7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:10:35.817 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:35.811+0000 7fd165d9b640 1 --2- 172.21.15.42:0/2164855656 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x555c43b311c0 0x555c43fc7200 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3828a2a586fa713e server_cookie=b969d4c6c07eb27 in_seq=0 out_seq=0 2024-08-30T09:10:35.818 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:35.811+0000 7fd14d7fa640 10 client.17061 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:10:35.953 INFO:teuthology.orchestra.run.smithi042.stderr:progress: 590.0 s, 535.2 tps, lat 187.441 ms stddev 237.640 2024-08-30T09:10:36.148 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:36.145+0000 7fd14d7fa640 1 -- 172.21.15.42:0/2164855656 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7fd158060070 con 0x555c43b311c0 2024-08-30T09:10:36.148 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:36.145+0000 7fd14d7fa640 10 client.17061 handle_command_reply: tid=0 2024-08-30T09:10:36.148 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:36.145+0000 7fd173cdb740 1 client.17061 shutdown 2024-08-30T09:10:36.148 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:36.145+0000 7fd173cdb740 1 -- 172.21.15.42:0/2164855656 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555c43b742f0 msgr2=0x555c43f539f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:10:36.148 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:36.145+0000 7fd173cdb740 1 --2- 172.21.15.42:0/2164855656 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555c43b742f0 0x555c43f539f0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fd1580538c0 tx=0x7fd15801a480 comp rx=0 tx=0).stop 2024-08-30T09:10:36.148 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:36.145+0000 7fd173cdb740 1 -- 172.21.15.42:0/2164855656 shutdown_connections 2024-08-30T09:10:36.149 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:36.145+0000 7fd173cdb740 1 --2- 172.21.15.42:0/2164855656 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x555c43b311c0 0x555c43fc7200 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:36.149 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:36.145+0000 7fd14d7fa640 0 client.17061 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:10:36.149 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:36.146+0000 7fd173cdb740 1 --2- 172.21.15.42:0/2164855656 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555c43b7c4d0 0x555c43f540b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:36.149 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:36.146+0000 7fd173cdb740 1 --2- 172.21.15.42:0/2164855656 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555c43b9d1b0 0x555c43fbce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:36.149 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:36.146+0000 7fd173cdb740 1 --2- 172.21.15.42:0/2164855656 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555c43b742f0 0x555c43f539f0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:36.149 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:36.146+0000 7fd173cdb740 1 -- 172.21.15.42:0/2164855656 >> 172.21.15.42:0/2164855656 conn(0x555c43b91300 msgr2=0x555c43f49260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:10:36.149 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:36.146+0000 7fd173cdb740 1 -- 172.21.15.42:0/2164855656 shutdown_connections 2024-08-30T09:10:36.149 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:36.146+0000 7fd173cdb740 1 -- 172.21.15.42:0/2164855656 wait complete. 2024-08-30T09:10:36.149 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:36.146+0000 7fd173cdb740 20 client.17061 trim_cache size 0 max 16384 2024-08-30T09:10:36.149 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-08-30T09:10:36.150 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2024-08-30T09:10:36.150 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-08-30T09:10:36.150 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-08-30T09:10:36.151 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:36.147+0000 7fd1557fa640 1 -- 172.21.15.42:0/3879698388 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fd1500b4d70 msgr2=0x7fd1500b7280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:10:36.151 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:36.147+0000 7fd1557fa640 1 --2- 172.21.15.42:0/3879698388 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fd1500b4d70 0x7fd1500b7280 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fd15c006aa0 tx=0x7fd15c009110 comp rx=0 tx=0).stop 2024-08-30T09:10:36.151 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:36.147+0000 7fd1557fa640 1 -- 172.21.15.42:0/3879698388 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd168103d70 msgr2=0x7fd168073630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:10:36.151 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:36.147+0000 7fd1557fa640 1 --2- 172.21.15.42:0/3879698388 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd168103d70 0x7fd168073630 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fd16000c970 tx=0x7fd16000cd70 comp rx=0 tx=0).stop 2024-08-30T09:10:36.151 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:36.147+0000 7fd1557fa640 1 -- 172.21.15.42:0/3879698388 shutdown_connections 2024-08-30T09:10:36.151 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:36.147+0000 7fd1557fa640 1 --2- 172.21.15.42:0/3879698388 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fd1500b4d70 0x7fd1500b7280 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:36.151 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:36.147+0000 7fd1557fa640 1 --2- 172.21.15.42:0/3879698388 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd168103d70 0x7fd168073630 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:36.151 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:36.147+0000 7fd1557fa640 1 --2- 172.21.15.42:0/3879698388 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd1681033d0 0x7fd1680730f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:36.151 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:36.147+0000 7fd1557fa640 1 --2- 172.21.15.42:0/3879698388 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd168102350 0x7fd168074830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:36.151 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:36.147+0000 7fd1557fa640 1 -- 172.21.15.42:0/3879698388 >> 172.21.15.42:0/3879698388 conn(0x7fd1680fdcd0 msgr2=0x7fd1681066f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:10:36.151 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:36.147+0000 7fd1557fa640 1 -- 172.21.15.42:0/3879698388 shutdown_connections 2024-08-30T09:10:36.151 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:36.147+0000 7fd1557fa640 1 -- 172.21.15.42:0/3879698388 wait complete. 2024-08-30T09:10:36.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:10:36 smithi167 ceph-mon[22063]: pgmap v668: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 4.3 MiB/s wr, 484 op/s 2024-08-30T09:10:36.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:10:36 smithi042 ceph-mon[18652]: pgmap v668: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 4.3 MiB/s wr, 484 op/s 2024-08-30T09:10:36.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:10:36 smithi190 ceph-mon[22001]: pgmap v668: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 4.3 MiB/s wr, 484 op/s 2024-08-30T09:10:38.578 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:10:38 smithi167 ceph-mon[22063]: pgmap v669: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 3.9 MiB/s wr, 460 op/s 2024-08-30T09:10:38.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:10:38 smithi042 ceph-mon[18652]: pgmap v669: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 3.9 MiB/s wr, 460 op/s 2024-08-30T09:10:38.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:10:38 smithi190 ceph-mon[22001]: pgmap v669: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 3.9 MiB/s wr, 460 op/s 2024-08-30T09:10:40.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:10:40 smithi167 ceph-mon[22063]: pgmap v670: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.0 MiB/s wr, 581 op/s 2024-08-30T09:10:40.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:10:40 smithi042 ceph-mon[18652]: pgmap v670: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.0 MiB/s wr, 581 op/s 2024-08-30T09:10:40.694 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:10:40 smithi190 ceph-mon[22001]: pgmap v670: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.0 MiB/s wr, 581 op/s 2024-08-30T09:10:40.953 INFO:teuthology.orchestra.run.smithi042.stderr:progress: 595.0 s, 603.0 tps, lat 163.965 ms stddev 187.860 2024-08-30T09:10:42.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:10:42 smithi167 ceph-mon[22063]: pgmap v671: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 3.9 MiB/s wr, 451 op/s 2024-08-30T09:10:42.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:10:42 smithi042 ceph-mon[18652]: pgmap v671: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 3.9 MiB/s wr, 451 op/s 2024-08-30T09:10:42.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:10:42 smithi190 ceph-mon[22001]: pgmap v671: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 3.9 MiB/s wr, 451 op/s 2024-08-30T09:10:44.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:10:44 smithi167 ceph-mon[22063]: pgmap v672: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.9 MiB/s wr, 612 op/s 2024-08-30T09:10:44.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:10:44 smithi042 ceph-mon[18652]: pgmap v672: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.9 MiB/s wr, 612 op/s 2024-08-30T09:10:44.664 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:10:44 smithi190 ceph-mon[22001]: pgmap v672: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.9 MiB/s wr, 612 op/s 2024-08-30T09:10:45.635 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-30T09:10:45.954 INFO:teuthology.orchestra.run.smithi042.stderr:progress: 600.0 s, 487.8 tps, lat 204.361 ms stddev 259.475 2024-08-30T09:10:46.230 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T09:10:46.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:10:46 smithi167 ceph-mon[22063]: pgmap v673: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.6 MiB/s wr, 506 op/s 2024-08-30T09:10:46.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:10:46 smithi042 ceph-mon[18652]: pgmap v673: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.6 MiB/s wr, 506 op/s 2024-08-30T09:10:46.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:10:46 smithi190 ceph-mon[22001]: pgmap v673: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.6 MiB/s wr, 506 op/s 2024-08-30T09:10:47.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.005+0000 7fb01150e640 1 -- 172.21.15.42:0/2799547020 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb00c1101e0 msgr2=0x7fb00c116a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:10:47.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.005+0000 7fb01150e640 1 --2- 172.21.15.42:0/2799547020 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb00c1101e0 0x7fb00c116a10 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7faffc009900 tx=0x7faffc02a440 comp rx=0 tx=0).stop 2024-08-30T09:10:47.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.005+0000 7fb01150e640 1 -- 172.21.15.42:0/2799547020 shutdown_connections 2024-08-30T09:10:47.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.005+0000 7fb01150e640 1 --2- 172.21.15.42:0/2799547020 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb00c1101e0 0x7fb00c116a10 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:47.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.005+0000 7fb01150e640 1 --2- 172.21.15.42:0/2799547020 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb00c10f890 0x7fb00c10fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:47.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.005+0000 7fb01150e640 1 --2- 172.21.15.42:0/2799547020 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb00c109f50 0x7fb00c10a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:47.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.005+0000 7fb01150e640 1 -- 172.21.15.42:0/2799547020 >> 172.21.15.42:0/2799547020 conn(0x7fb00c077e80 msgr2=0x7fb00c107f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:10:47.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.012+0000 7fb01150e640 1 -- 172.21.15.42:0/2799547020 shutdown_connections 2024-08-30T09:10:47.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.013+0000 7fb01150e640 1 -- 172.21.15.42:0/2799547020 wait complete. 2024-08-30T09:10:47.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.014+0000 7fb01150e640 1 Processor -- start 2024-08-30T09:10:47.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.015+0000 7fb01150e640 1 -- start start 2024-08-30T09:10:47.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.015+0000 7fb01150e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb00c109f50 0x7fb00c074d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:47.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.015+0000 7fb01150e640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb00c10f890 0x7fb00c06f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:47.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.015+0000 7fb01150e640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb00c1101e0 0x7fb00c06f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:47.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.015+0000 7fb01150e640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fb00c06d480 con 0x7fb00c1101e0 2024-08-30T09:10:47.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.015+0000 7fb01150e640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb00c06d300 con 0x7fb00c10f890 2024-08-30T09:10:47.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.015+0000 7fb01150e640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fb00c06d600 con 0x7fb00c109f50 2024-08-30T09:10:47.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.016+0000 7fb00b7fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb00c1101e0 0x7fb00c06f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:10:47.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.016+0000 7fb00a7fc640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb00c10f890 0x7fb00c06f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:10:47.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.016+0000 7fb00a7fc640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb00c10f890 0x7fb00c06f020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:41972/0 (socket says 172.21.15.42:41972) 2024-08-30T09:10:47.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.016+0000 7fb00a7fc640 1 -- 172.21.15.42:0/2689871758 learned_addr learned my addr 172.21.15.42:0/2689871758 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:10:47.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.016+0000 7fb00a7fc640 1 -- 172.21.15.42:0/2689871758 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb00c109f50 msgr2=0x7fb00c074d60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-30T09:10:47.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.016+0000 7fb00a7fc640 1 --2- 172.21.15.42:0/2689871758 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb00c109f50 0x7fb00c074d60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:47.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.016+0000 7fb00a7fc640 1 -- 172.21.15.42:0/2689871758 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb00c1101e0 msgr2=0x7fb00c06f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:10:47.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.016+0000 7fb00a7fc640 1 --2- 172.21.15.42:0/2689871758 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb00c1101e0 0x7fb00c06f720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:47.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.016+0000 7fb00a7fc640 1 -- 172.21.15.42:0/2689871758 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb00c1b53d0 con 0x7fb00c10f890 2024-08-30T09:10:47.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.017+0000 7fb00a7fc640 1 --2- 172.21.15.42:0/2689871758 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb00c10f890 0x7fb00c06f020 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fb00400b990 tx=0x7fb00400be50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:10:47.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.018+0000 7faff3fff640 1 -- 172.21.15.42:0/2689871758 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb004019070 con 0x7fb00c10f890 2024-08-30T09:10:47.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.019+0000 7fb01150e640 1 -- 172.21.15.42:0/2689871758 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb00c1b5600 con 0x7fb00c10f890 2024-08-30T09:10:47.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.019+0000 7fb01150e640 1 -- 172.21.15.42:0/2689871758 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb00c1b5ab0 con 0x7fb00c10f890 2024-08-30T09:10:47.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.020+0000 7faff3fff640 1 -- 172.21.15.42:0/2689871758 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb00400cba0 con 0x7fb00c10f890 2024-08-30T09:10:47.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.020+0000 7faff3fff640 1 -- 172.21.15.42:0/2689871758 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0040125a0 con 0x7fb00c10f890 2024-08-30T09:10:47.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.022+0000 7fb017a83740 1 Processor -- start 2024-08-30T09:10:47.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.034+0000 7fb017a83740 1 -- start start 2024-08-30T09:10:47.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.034+0000 7faff3fff640 1 -- 172.21.15.42:0/2689871758 <== mon.2 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7fb00401c070 con 0x7fb00c10f890 2024-08-30T09:10:47.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.034+0000 7fb017a83740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561f83d2f070 0x561f840d42d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:47.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.034+0000 7fb017a83740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561f83d061b0 0x561f840d4810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:47.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.034+0000 7fb017a83740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561f83d0e390 0x561f840d4d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:47.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.034+0000 7fb017a83740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x561f8404b630 con 0x561f83d0e390 2024-08-30T09:10:47.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.034+0000 7fb017a83740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x561f8403abc0 con 0x561f83d2f070 2024-08-30T09:10:47.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.034+0000 7fb017a83740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x561f8403eb70 con 0x561f83d061b0 2024-08-30T09:10:47.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.035+0000 7fb00a7fc640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561f83d0e390 0x561f840d4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:10:47.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.035+0000 7fb00a7fc640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561f83d0e390 0x561f840d4d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41642/0 (socket says 172.21.15.42:41642) 2024-08-30T09:10:47.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.035+0000 7fb00a7fc640 1 -- 172.21.15.42:0/3449060020 learned_addr learned my addr 172.21.15.42:0/3449060020 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:10:47.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.035+0000 7fb00a7fc640 1 -- 172.21.15.42:0/3449060020 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561f83d061b0 msgr2=0x561f840d4810 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-30T09:10:47.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.035+0000 7fb00a7fc640 1 --2- 172.21.15.42:0/3449060020 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561f83d061b0 0x561f840d4810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:47.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.035+0000 7fb00a7fc640 1 -- 172.21.15.42:0/3449060020 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561f83d2f070 msgr2=0x561f840d42d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-30T09:10:47.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.036+0000 7fb00a7fc640 1 --2- 172.21.15.42:0/3449060020 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561f83d2f070 0x561f840d42d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:47.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.036+0000 7fb00a7fc640 1 -- 172.21.15.42:0/3449060020 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561f83d9a650 con 0x561f83d0e390 2024-08-30T09:10:47.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.036+0000 7fb00a7fc640 1 --2- 172.21.15.42:0/3449060020 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561f83d0e390 0x561f840d4d50 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fb0040eab70 tx=0x7fb0040eac60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=264fa62155099322 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:10:47.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.036+0000 7faff3fff640 1 --2- 172.21.15.42:0/2689871758 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7faff40b4d40 0x7faff40b7250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:47.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.036+0000 7faff3fff640 1 -- 172.21.15.42:0/2689871758 <== mon.2 v2:172.21.15.167:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7fb0040e4950 con 0x7fb00c10f890 2024-08-30T09:10:47.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.038+0000 7faff17fa640 1 -- 172.21.15.42:0/3449060020 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0040a6e50 con 0x561f83d0e390 2024-08-30T09:10:47.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.039+0000 7faff17fa640 1 -- 172.21.15.42:0/3449060020 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb0040f1070 con 0x561f83d0e390 2024-08-30T09:10:47.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.039+0000 7fb00b7fe640 1 --2- 172.21.15.42:0/2689871758 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7faff40b4d40 0x7faff40b7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:10:47.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.040+0000 7fb017a83740 1 -- 172.21.15.42:0/3449060020 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561f83d0e390 msgr2=0x561f840d4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:10:47.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.040+0000 7fb017a83740 1 --2- 172.21.15.42:0/3449060020 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561f83d0e390 0x561f840d4d50 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fb0040eab70 tx=0x7fb0040eac60 comp rx=0 tx=0).stop 2024-08-30T09:10:47.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.040+0000 7fb017a83740 1 -- 172.21.15.42:0/3449060020 shutdown_connections 2024-08-30T09:10:47.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.040+0000 7fb017a83740 1 --2- 172.21.15.42:0/3449060020 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561f83d061b0 0x561f840d4810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:47.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.040+0000 7fb017a83740 1 --2- 172.21.15.42:0/3449060020 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561f83d0e390 0x561f840d4d50 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:47.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.040+0000 7fb017a83740 1 --2- 172.21.15.42:0/3449060020 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561f83d2f070 0x561f840d42d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:47.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.040+0000 7fb017a83740 1 -- 172.21.15.42:0/3449060020 >> 172.21.15.42:0/3449060020 conn(0x561f83d23460 msgr2=0x561f840d21c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:10:47.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.041+0000 7fb017a83740 1 -- 172.21.15.42:0/3449060020 shutdown_connections 2024-08-30T09:10:47.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.041+0000 7fb017a83740 1 -- 172.21.15.42:0/3449060020 wait complete. 2024-08-30T09:10:47.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.042+0000 7fb017a83740 1 -- start start 2024-08-30T09:10:47.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.042+0000 7fb017a83740 1 Processor -- start 2024-08-30T09:10:47.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.042+0000 7fb00b7fe640 1 --2- 172.21.15.42:0/2689871758 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7faff40b4d40 0x7faff40b7250 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7faffc0096c0 tx=0x7faffc005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:10:47.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.042+0000 7fb017a83740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561f83d061b0 0x561f840df9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:47.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.042+0000 7fb017a83740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561f83d0e390 0x561f840dff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:47.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.042+0000 7fb017a83740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561f83d2f070 0x561f840ebe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:47.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.042+0000 7fb017a83740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x561f83e9d700 con 0x561f83d061b0 2024-08-30T09:10:47.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.042+0000 7fb017a83740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x561f83fc1700 con 0x561f83d0e390 2024-08-30T09:10:47.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.043+0000 7fb017a83740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x561f840e2b00 con 0x561f83d2f070 2024-08-30T09:10:47.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.043+0000 7fb017a83740 10 client.0 _ll_register_callbacks cb 0x561f83ff0dc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:10:47.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.043+0000 7fb00a7fc640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561f83d0e390 0x561f840dff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:10:47.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.043+0000 7fb00a7fc640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561f83d0e390 0x561f840dff00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:41984/0 (socket says 172.21.15.42:41984) 2024-08-30T09:10:47.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.043+0000 7fb00a7fc640 1 -- 172.21.15.42:0/4135751600 learned_addr learned my addr 172.21.15.42:0/4135751600 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:10:47.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.043+0000 7fb00b7fe640 1 --2- 172.21.15.42:0/4135751600 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561f83d2f070 0x561f840ebe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:10:47.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.044+0000 7fb00a7fc640 1 -- 172.21.15.42:0/4135751600 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561f83d2f070 msgr2=0x561f840ebe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:10:47.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.044+0000 7fb00a7fc640 1 --2- 172.21.15.42:0/4135751600 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561f83d2f070 0x561f840ebe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:47.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.044+0000 7fb00a7fc640 1 -- 172.21.15.42:0/4135751600 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561f83d061b0 msgr2=0x561f840df9c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-30T09:10:47.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.044+0000 7fb00a7fc640 1 --2- 172.21.15.42:0/4135751600 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561f83d061b0 0x561f840df9c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:47.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.044+0000 7fb00a7fc640 1 -- 172.21.15.42:0/4135751600 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561f83ffa8b0 con 0x561f83d0e390 2024-08-30T09:10:47.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.044+0000 7fb00a7fc640 1 --2- 172.21.15.42:0/4135751600 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561f83d0e390 0x561f840dff00 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fb004016000 tx=0x7fb0040ab7a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=66e2381b801bfc3a server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:10:47.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.044+0000 7fafeeff5640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-30T09:10:47.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.044+0000 7fafeeff5640 1 -- 172.21.15.42:0/4135751600 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0040aa8e0 con 0x561f83d0e390 2024-08-30T09:10:47.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.044+0000 7fb017a83740 1 -- 172.21.15.42:0/4135751600 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb0040f1070 con 0x561f83d0e390 2024-08-30T09:10:47.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.044+0000 7fafeeff5640 1 -- 172.21.15.42:0/4135751600 <== mon.2 v2:172.21.15.167:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7fb0040ab400 con 0x561f83d0e390 2024-08-30T09:10:47.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.045+0000 7fafeeff5640 1 -- 172.21.15.42:0/4135751600 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fb0040aad60 con 0x561f83d0e390 2024-08-30T09:10:47.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.045+0000 7fb017a83740 10 client.26867 fetch_fsmap learned FSMap version 39 2024-08-30T09:10:47.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.045+0000 7fb017a83740 1 -- 172.21.15.42:0/4135751600 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x561f840ff8d0 con 0x561f83d0e390 2024-08-30T09:10:47.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.046+0000 7fafeeff5640 1 -- 172.21.15.42:0/4135751600 <== mon.2 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb0040f1070 con 0x561f83d0e390 2024-08-30T09:10:47.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.047+0000 7fafeeff5640 1 -- 172.21.15.42:0/4135751600 <== mon.2 v2:172.21.15.167:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7fb0040961a0 con 0x561f83d0e390 2024-08-30T09:10:47.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.047+0000 7fb017a83740 10 client.26867 fetch_fsmap finished waiting for FSMap version 39 2024-08-30T09:10:47.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.047+0000 7fb017a83740 10 client.26867 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-30T09:10:47.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.047+0000 7fb017a83740 20 client.26867 populate_metadata read hostname 'smithi042' 2024-08-30T09:10:47.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.047+0000 7fb017a83740 1 --2- 172.21.15.42:0/4135751600 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x561f83ced9f0 0x561f84102e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:47.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.048+0000 7fb017a83740 4 client.26867 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-30T09:10:47.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.048+0000 7fb017a83740 1 -- 172.21.15.42:0/4135751600 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561f8404d580 con 0x561f83ced9f0 2024-08-30T09:10:47.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.048+0000 7fb00affd640 1 --2- 172.21.15.42:0/4135751600 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x561f83ced9f0 0x561f84102e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:10:47.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.050+0000 7fb00affd640 1 --2- 172.21.15.42:0/4135751600 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x561f83ced9f0 0x561f84102e30 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=64ff99d3fd9ba320 server_cookie=5f27a936cd0f771a in_seq=0 out_seq=0 2024-08-30T09:10:47.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.050+0000 7fafeeff5640 10 client.26867 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:10:47.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.050+0000 7fafeeff5640 1 -- 172.21.15.42:0/4135751600 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x561f8404d580 con 0x561f83ced9f0 2024-08-30T09:10:47.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.050+0000 7fafeeff5640 10 client.26867 handle_command_reply: tid=0 2024-08-30T09:10:47.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.051+0000 7fb017a83740 1 client.26867 shutdown 2024-08-30T09:10:47.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.051+0000 7fb017a83740 1 -- 172.21.15.42:0/4135751600 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561f83d0e390 msgr2=0x561f840dff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:10:47.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.051+0000 7fb017a83740 1 --2- 172.21.15.42:0/4135751600 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561f83d0e390 0x561f840dff00 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fb004016000 tx=0x7fb0040ab7a0 comp rx=0 tx=0).stop 2024-08-30T09:10:47.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.052+0000 7fb017a83740 1 -- 172.21.15.42:0/4135751600 shutdown_connections 2024-08-30T09:10:47.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.052+0000 7fb017a83740 1 --2- 172.21.15.42:0/4135751600 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x561f83ced9f0 0x561f84102e30 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:47.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.052+0000 7fb017a83740 1 --2- 172.21.15.42:0/4135751600 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561f83d2f070 0x561f840ebe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:47.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.052+0000 7fb017a83740 1 --2- 172.21.15.42:0/4135751600 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561f83d0e390 0x561f840dff00 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:47.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.052+0000 7fb017a83740 1 --2- 172.21.15.42:0/4135751600 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561f83d061b0 0x561f840df9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:47.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.052+0000 7fb017a83740 1 -- 172.21.15.42:0/4135751600 >> 172.21.15.42:0/4135751600 conn(0x561f83d23460 msgr2=0x561f840d7460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:10:47.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.052+0000 7fafeeff5640 0 client.26867 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:10:47.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.052+0000 7fb017a83740 1 -- 172.21.15.42:0/4135751600 shutdown_connections 2024-08-30T09:10:47.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.052+0000 7fb017a83740 1 -- 172.21.15.42:0/4135751600 wait complete. 2024-08-30T09:10:47.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.054+0000 7fb017a83740 20 client.26867 trim_cache size 0 max 16384 2024-08-30T09:10:47.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.088+0000 7fb017a83740 1 Processor -- start 2024-08-30T09:10:47.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.088+0000 7fb017a83740 1 -- start start 2024-08-30T09:10:47.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.089+0000 7fb017a83740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561f83d061b0 0x561f840d5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:47.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.089+0000 7fb017a83740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561f83d0e390 0x561f840e15f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:47.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.089+0000 7fb017a83740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561f83d2f070 0x561f840dcbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:47.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.089+0000 7fb017a83740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fb00c1b9880 con 0x561f83d2f070 2024-08-30T09:10:47.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.089+0000 7fb017a83740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb00c1b70f0 con 0x561f83d0e390 2024-08-30T09:10:47.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.089+0000 7fb00b7fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561f83d2f070 0x561f840dcbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:10:47.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.090+0000 7fb017a83740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fb00c0833f0 con 0x561f83d061b0 2024-08-30T09:10:47.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.090+0000 7fb00affd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561f83d061b0 0x561f840d5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:10:47.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.090+0000 7fb00b7fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561f83d2f070 0x561f840dcbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41646/0 (socket says 172.21.15.42:41646) 2024-08-30T09:10:47.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.090+0000 7fb00b7fe640 1 -- 172.21.15.42:0/1938199240 learned_addr learned my addr 172.21.15.42:0/1938199240 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:10:47.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.090+0000 7fb00a7fc640 1 --2- 172.21.15.42:0/1938199240 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561f83d0e390 0x561f840e15f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:10:47.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.093+0000 7fb00a7fc640 1 -- 172.21.15.42:0/1938199240 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561f83d061b0 msgr2=0x561f840d5cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:10:47.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.093+0000 7fb00a7fc640 1 --2- 172.21.15.42:0/1938199240 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561f83d061b0 0x561f840d5cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:47.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.093+0000 7fb00a7fc640 1 -- 172.21.15.42:0/1938199240 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561f83d2f070 msgr2=0x561f840dcbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:10:47.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.093+0000 7fb00a7fc640 1 --2- 172.21.15.42:0/1938199240 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561f83d2f070 0x561f840dcbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:47.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.093+0000 7fb00a7fc640 1 -- 172.21.15.42:0/1938199240 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561f83cedc20 con 0x561f83d0e390 2024-08-30T09:10:47.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.098+0000 7fb00a7fc640 1 --2- 172.21.15.42:0/1938199240 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561f83d0e390 0x561f840e15f0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fb0040af700 tx=0x7fb0040eac60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b3c06225e7d8968e server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:10:47.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.102+0000 7faff0ff9640 1 -- 172.21.15.42:0/1938199240 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0040d1070 con 0x561f83d0e390 2024-08-30T09:10:47.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.102+0000 7faff0ff9640 1 -- 172.21.15.42:0/1938199240 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb0040f1030 con 0x561f83d0e390 2024-08-30T09:10:47.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.102+0000 7faff0ff9640 1 -- 172.21.15.42:0/1938199240 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0040aaf90 con 0x561f83d0e390 2024-08-30T09:10:47.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.105+0000 7fb017a83740 1 -- 172.21.15.42:0/1938199240 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561f83d0e390 msgr2=0x561f840e15f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:10:47.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.105+0000 7fb017a83740 1 --2- 172.21.15.42:0/1938199240 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561f83d0e390 0x561f840e15f0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fb0040af700 tx=0x7fb0040eac60 comp rx=0 tx=0).stop 2024-08-30T09:10:47.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.107+0000 7fb017a83740 1 -- 172.21.15.42:0/1938199240 shutdown_connections 2024-08-30T09:10:47.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.107+0000 7fb017a83740 1 --2- 172.21.15.42:0/1938199240 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561f83d2f070 0x561f840dcbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:47.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.107+0000 7fb017a83740 1 --2- 172.21.15.42:0/1938199240 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561f83d0e390 0x561f840e15f0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:47.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.107+0000 7fb017a83740 1 --2- 172.21.15.42:0/1938199240 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561f83d061b0 0x561f840d5cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:47.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.107+0000 7fb017a83740 1 -- 172.21.15.42:0/1938199240 >> 172.21.15.42:0/1938199240 conn(0x561f83d23460 msgr2=0x561f840e0ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:10:47.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.114+0000 7fb017a83740 1 -- 172.21.15.42:0/1938199240 shutdown_connections 2024-08-30T09:10:47.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.114+0000 7fb017a83740 1 -- 172.21.15.42:0/1938199240 wait complete. 2024-08-30T09:10:47.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.115+0000 7fb017a83740 1 -- start start 2024-08-30T09:10:47.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.115+0000 7fb017a83740 1 Processor -- start 2024-08-30T09:10:47.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.115+0000 7fb017a83740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561f83d061b0 0x561f840e45f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:47.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.116+0000 7fb017a83740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561f83d0e390 0x561f840e4b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:47.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.116+0000 7fb017a83740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561f83d2f070 0x561f8416c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:47.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.116+0000 7fb017a83740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fb00c0839b0 con 0x561f83d061b0 2024-08-30T09:10:47.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.116+0000 7fb017a83740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb00c1b6a20 con 0x561f83d2f070 2024-08-30T09:10:47.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.116+0000 7fb017a83740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fb00c1b5dc0 con 0x561f83d0e390 2024-08-30T09:10:47.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.116+0000 7fb017a83740 10 client.0 _ll_register_callbacks cb 0x7fb0040ea9c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:10:47.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.116+0000 7fb00affd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561f83d061b0 0x561f840e45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:10:47.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.116+0000 7fb00affd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561f83d061b0 0x561f840e45f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41658/0 (socket says 172.21.15.42:41658) 2024-08-30T09:10:47.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.116+0000 7fb00affd640 1 -- 172.21.15.42:0/3841260569 learned_addr learned my addr 172.21.15.42:0/3841260569 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:10:47.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.119+0000 7fb00b7fe640 1 --2- 172.21.15.42:0/3841260569 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561f83d2f070 0x561f8416c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:10:47.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.119+0000 7fb00a7fc640 1 --2- 172.21.15.42:0/3841260569 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561f83d0e390 0x561f840e4b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:10:47.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.121+0000 7fb00affd640 1 -- 172.21.15.42:0/3841260569 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561f83d0e390 msgr2=0x561f840e4b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:10:47.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.121+0000 7fb00affd640 1 --2- 172.21.15.42:0/3841260569 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561f83d0e390 0x561f840e4b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:47.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.121+0000 7fb00affd640 1 -- 172.21.15.42:0/3841260569 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561f83d2f070 msgr2=0x561f8416c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:10:47.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.121+0000 7fb00affd640 1 --2- 172.21.15.42:0/3841260569 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561f83d2f070 0x561f8416c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:47.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.121+0000 7fb00affd640 1 -- 172.21.15.42:0/3841260569 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561f8416cd80 con 0x561f83d061b0 2024-08-30T09:10:47.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.122+0000 7fb00affd640 1 --2- 172.21.15.42:0/3841260569 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561f83d061b0 0x561f840e45f0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x561f840d2f50 tx=0x7fb000006f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b65ac28327c84cc1 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:10:47.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.122+0000 7fafeeff5640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-08-30T09:10:47.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.123+0000 7fafeeff5640 1 -- 172.21.15.42:0/3841260569 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb00001f070 con 0x561f83d061b0 2024-08-30T09:10:47.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.123+0000 7fb017a83740 1 -- 172.21.15.42:0/3841260569 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb0040f1030 con 0x561f83d061b0 2024-08-30T09:10:47.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.123+0000 7fafeeff5640 1 -- 172.21.15.42:0/3841260569 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7fb000016d40 con 0x561f83d061b0 2024-08-30T09:10:47.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.124+0000 7fafeeff5640 1 -- 172.21.15.42:0/3841260569 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fb00002b050 con 0x561f83d061b0 2024-08-30T09:10:47.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.131+0000 7fb017a83740 10 client.17070 fetch_fsmap learned FSMap version 39 2024-08-30T09:10:47.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.131+0000 7fb017a83740 1 -- 172.21.15.42:0/3841260569 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x561f8416e0a0 con 0x561f83d061b0 2024-08-30T09:10:47.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.135+0000 7fafeeff5640 1 -- 172.21.15.42:0/3841260569 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb0040f1030 con 0x561f83d061b0 2024-08-30T09:10:47.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.135+0000 7fafeeff5640 1 -- 172.21.15.42:0/3841260569 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7fb000015690 con 0x561f83d061b0 2024-08-30T09:10:47.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.135+0000 7fb017a83740 10 client.17070 fetch_fsmap finished waiting for FSMap version 39 2024-08-30T09:10:47.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.135+0000 7fb017a83740 10 client.17070 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-30T09:10:47.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.135+0000 7fb017a83740 20 client.17070 populate_metadata read hostname 'smithi042' 2024-08-30T09:10:47.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.135+0000 7fb017a83740 1 --2- 172.21.15.42:0/3841260569 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x561f83cc2da0 0x561f84170920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:47.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.135+0000 7fb017a83740 4 client.17070 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-30T09:10:47.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.135+0000 7fb017a83740 1 -- 172.21.15.42:0/3841260569 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "scrub status"}) -- 0x561f84049020 con 0x561f83cc2da0 2024-08-30T09:10:47.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.140+0000 7fb00a7fc640 1 --2- 172.21.15.42:0/3841260569 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x561f83cc2da0 0x561f84170920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:10:47.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.144+0000 7fb00a7fc640 1 --2- 172.21.15.42:0/3841260569 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x561f83cc2da0 0x561f84170920 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=53dfa6ff311c60fd server_cookie=adf60769aeebe0a0 in_seq=0 out_seq=0 2024-08-30T09:10:47.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.144+0000 7fafeeff5640 10 client.17070 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:10:47.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.144+0000 7fafeeff5640 1 -- 172.21.15.42:0/3841260569 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3498 (crc 0 0 0) 0x561f84049020 con 0x561f83cc2da0 2024-08-30T09:10:47.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.144+0000 7fafeeff5640 10 client.17070 handle_command_reply: tid=0 2024-08-30T09:10:47.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.144+0000 7fb017a83740 1 client.17070 shutdown 2024-08-30T09:10:47.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.151+0000 7fb017a83740 1 -- 172.21.15.42:0/3841260569 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561f83d061b0 msgr2=0x561f840e45f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:10:47.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.151+0000 7fb017a83740 1 --2- 172.21.15.42:0/3841260569 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561f83d061b0 0x561f840e45f0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x561f840d2f50 tx=0x7fb000006f80 comp rx=0 tx=0).stop 2024-08-30T09:10:47.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.154+0000 7fb017a83740 1 -- 172.21.15.42:0/3841260569 shutdown_connections 2024-08-30T09:10:47.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.154+0000 7fb017a83740 1 --2- 172.21.15.42:0/3841260569 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x561f83cc2da0 0x561f84170920 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:47.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.154+0000 7fb017a83740 1 --2- 172.21.15.42:0/3841260569 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561f83d0e390 0x561f840e4b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:47.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.154+0000 7fb017a83740 1 --2- 172.21.15.42:0/3841260569 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561f83d2f070 0x561f8416c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:47.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.154+0000 7fb017a83740 1 --2- 172.21.15.42:0/3841260569 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561f83d061b0 0x561f840e45f0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:47.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.154+0000 7fb017a83740 1 -- 172.21.15.42:0/3841260569 >> 172.21.15.42:0/3841260569 conn(0x561f83d23460 msgr2=0x561f840d5a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:10:47.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.154+0000 7fafeeff5640 0 client.17070 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:10:47.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.157+0000 7fb017a83740 1 -- 172.21.15.42:0/3841260569 shutdown_connections 2024-08-30T09:10:47.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.157+0000 7fb017a83740 1 -- 172.21.15.42:0/3841260569 wait complete. 2024-08-30T09:10:47.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.157+0000 7fb017a83740 20 client.17070 trim_cache size 0 max 16384 2024-08-30T09:10:47.159 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-08-30T09:10:47.160 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "no active scrubs running", 2024-08-30T09:10:47.160 INFO:teuthology.orchestra.run.smithi042.stdout: "scrubs": {}, 2024-08-30T09:10:47.160 INFO:teuthology.orchestra.run.smithi042.stdout: "scrub_stats": { 2024-08-30T09:10:47.160 INFO:teuthology.orchestra.run.smithi042.stdout: "37f2f28f-496f-41a6-b4cf-8846fb3f43ab": { 2024-08-30T09:10:47.160 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:09:17.428286+0000", 2024-08-30T09:10:47.160 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:10:47.160 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:10:47.160 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:10:47.160 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:10:47.160 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:10:47.160 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:10:47.160 INFO:teuthology.orchestra.run.smithi042.stdout: "99a29ec1-9455-4ae4-854a-398b32896675": { 2024-08-30T09:10:47.160 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:08:24.923176+0000", 2024-08-30T09:10:47.160 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:10:47.161 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:10:47.161 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:10:47.161 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:10:47.161 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:10:47.161 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:10:47.161 INFO:teuthology.orchestra.run.smithi042.stdout: "a7210f9a-046b-4188-953c-819b9f1863a9": { 2024-08-30T09:10:47.161 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:10:07.608430+0000", 2024-08-30T09:10:47.161 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:10:47.161 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:10:47.161 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:10:47.161 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:10:47.161 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:10:47.161 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:10:47.161 INFO:teuthology.orchestra.run.smithi042.stdout: "598d8a75-f1cc-4388-80bc-a408bdb4eaac": { 2024-08-30T09:10:47.161 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:07:29.721122+0000", 2024-08-30T09:10:47.162 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:10:47.162 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:10:47.162 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:10:47.162 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:10:47.162 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:10:47.162 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:10:47.162 INFO:teuthology.orchestra.run.smithi042.stdout: "665ec740-7254-49cb-84ef-deff71c461b0": { 2024-08-30T09:10:47.162 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:06:37.144035+0000", 2024-08-30T09:10:47.162 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:10:47.162 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:10:47.162 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:10:47.162 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:10:47.162 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:10:47.162 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:10:47.162 INFO:teuthology.orchestra.run.smithi042.stdout: "1af4252a-4a99-47e9-bbf0-becf5e573dd5": { 2024-08-30T09:10:47.163 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:05:48.565881+0000", 2024-08-30T09:10:47.163 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:10:47.163 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:10:47.163 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:10:47.163 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:10:47.163 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:10:47.163 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:10:47.163 INFO:teuthology.orchestra.run.smithi042.stdout: "347d05ed-4bd3-4590-b099-314f719aab39": { 2024-08-30T09:10:47.163 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:05:00.439721+0000", 2024-08-30T09:10:47.163 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:10:47.163 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:10:47.163 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:10:47.163 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:10:47.163 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:10:47.163 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:10:47.164 INFO:teuthology.orchestra.run.smithi042.stdout: "88cd7478-2b20-47de-ba51-256479749232": { 2024-08-30T09:10:47.164 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:04:08.922625+0000", 2024-08-30T09:10:47.164 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:10:47.164 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:10:47.164 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:10:47.164 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:10:47.164 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:10:47.164 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:10:47.164 INFO:teuthology.orchestra.run.smithi042.stdout: "749f4867-0b70-49e5-af2d-8362a240ebc3": { 2024-08-30T09:10:47.164 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:03:16.386554+0000", 2024-08-30T09:10:47.164 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:10:47.164 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:10:47.164 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:10:47.165 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:10:47.165 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:10:47.165 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:10:47.165 INFO:teuthology.orchestra.run.smithi042.stdout: "85134e4b-ab4a-4885-b07e-a3ad01eaf13d": { 2024-08-30T09:10:47.165 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:01:27.738462+0000", 2024-08-30T09:10:47.165 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:10:47.165 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:10:47.165 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:10:47.165 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:10:47.165 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:10:47.165 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:10:47.165 INFO:teuthology.orchestra.run.smithi042.stdout: "c827ad7a-44ee-4e1d-9bb3-51fb7ab00c73": { 2024-08-30T09:10:47.165 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:02:21.686517+0000", 2024-08-30T09:10:47.166 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:10:47.166 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:10:47.166 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:10:47.166 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:10:47.166 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:10:47.166 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:10:47.166 INFO:teuthology.orchestra.run.smithi042.stdout: "5e2ed104-0625-478a-a78b-584e7f4c2817": { 2024-08-30T09:10:47.166 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:00:33.928400+0000", 2024-08-30T09:10:47.166 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:10:47.166 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:10:47.166 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:10:47.166 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:10:47.166 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:10:47.166 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:10:47.166 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:10:47.167 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-08-30T09:10:47.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.162+0000 7fb01150e640 1 -- 172.21.15.42:0/2689871758 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7faff40b4d40 msgr2=0x7faff40b7250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:10:47.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.162+0000 7fb01150e640 1 --2- 172.21.15.42:0/2689871758 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7faff40b4d40 0x7faff40b7250 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7faffc0096c0 tx=0x7faffc005fd0 comp rx=0 tx=0).stop 2024-08-30T09:10:47.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.162+0000 7fb01150e640 1 -- 172.21.15.42:0/2689871758 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb00c10f890 msgr2=0x7fb00c06f020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:10:47.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.162+0000 7fb01150e640 1 --2- 172.21.15.42:0/2689871758 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb00c10f890 0x7fb00c06f020 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fb00400b990 tx=0x7fb00400be50 comp rx=0 tx=0).stop 2024-08-30T09:10:47.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.162+0000 7fb01150e640 1 -- 172.21.15.42:0/2689871758 shutdown_connections 2024-08-30T09:10:47.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.162+0000 7fb01150e640 1 --2- 172.21.15.42:0/2689871758 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7faff40b4d40 0x7faff40b7250 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:47.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.162+0000 7fb01150e640 1 --2- 172.21.15.42:0/2689871758 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb00c1101e0 0x7fb00c06f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:47.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.162+0000 7fb01150e640 1 --2- 172.21.15.42:0/2689871758 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb00c10f890 0x7fb00c06f020 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:47.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.162+0000 7fb01150e640 1 --2- 172.21.15.42:0/2689871758 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb00c109f50 0x7fb00c074d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:47.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.162+0000 7fb01150e640 1 -- 172.21.15.42:0/2689871758 >> 172.21.15.42:0/2689871758 conn(0x7fb00c077e80 msgr2=0x7fb00c10de50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:10:47.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.162+0000 7fb01150e640 1 -- 172.21.15.42:0/2689871758 shutdown_connections 2024-08-30T09:10:47.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:47.162+0000 7fb01150e640 1 -- 172.21.15.42:0/2689871758 wait complete. 2024-08-30T09:10:47.991 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-08-30T09:10:47.991 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph fs dump --format=json 2024-08-30T09:10:48.331 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:10:48 smithi042 ceph-mon[18652]: pgmap v674: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.5 MiB/s wr, 497 op/s 2024-08-30T09:10:48.445 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T09:10:48.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:10:48 smithi190 ceph-mon[22001]: pgmap v674: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.5 MiB/s wr, 497 op/s 2024-08-30T09:10:48.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:10:48 smithi167 ceph-mon[22063]: pgmap v674: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.5 MiB/s wr, 497 op/s 2024-08-30T09:10:49.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:49.078+0000 7fe706ba1640 1 -- 172.21.15.42:0/786063311 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe70010ac80 msgr2=0x7fe700115d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:10:49.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:49.078+0000 7fe6ff7fe640 1 -- 172.21.15.42:0/786063311 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6f8036070 con 0x7fe70010ac80 2024-08-30T09:10:49.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:49.078+0000 7fe706ba1640 1 --2- 172.21.15.42:0/786063311 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe70010ac80 0x7fe700115d70 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fe6f8009be0 tx=0x7fe6f802a6d0 comp rx=0 tx=0).stop 2024-08-30T09:10:49.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:49.079+0000 7fe706ba1640 1 -- 172.21.15.42:0/786063311 shutdown_connections 2024-08-30T09:10:49.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:49.079+0000 7fe706ba1640 1 --2- 172.21.15.42:0/786063311 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe70010ac80 0x7fe700115d70 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:49.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:49.079+0000 7fe706ba1640 1 --2- 172.21.15.42:0/786063311 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe7001163e0 0x7fe7001188f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:49.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:49.079+0000 7fe706ba1640 1 --2- 172.21.15.42:0/786063311 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe70010a330 0x7fe70010a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:49.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:49.079+0000 7fe706ba1640 1 -- 172.21.15.42:0/786063311 >> 172.21.15.42:0/786063311 conn(0x7fe700070c60 msgr2=0x7fe700071070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:10:49.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:49.083+0000 7fe706ba1640 1 -- 172.21.15.42:0/786063311 shutdown_connections 2024-08-30T09:10:49.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:49.084+0000 7fe706ba1640 1 -- 172.21.15.42:0/786063311 wait complete. 2024-08-30T09:10:49.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:49.085+0000 7fe706ba1640 1 Processor -- start 2024-08-30T09:10:49.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:49.087+0000 7fe706ba1640 1 -- start start 2024-08-30T09:10:49.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:49.087+0000 7fe706ba1640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe70010a330 0x7fe70011bc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:49.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:49.087+0000 7fe706ba1640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe70010ac80 0x7fe70011c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:49.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:49.087+0000 7fe706ba1640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe7001163e0 0x7fe700121480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:49.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:49.087+0000 7fe706ba1640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fe700119e60 con 0x7fe70010a330 2024-08-30T09:10:49.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:49.087+0000 7fe706ba1640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe700119ce0 con 0x7fe7001163e0 2024-08-30T09:10:49.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:49.087+0000 7fe706ba1640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fe700119fe0 con 0x7fe70010ac80 2024-08-30T09:10:49.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:49.087+0000 7fe6fffff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe70010ac80 0x7fe70011c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:10:49.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:49.088+0000 7fe6fffff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe70010ac80 0x7fe70011c1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:40594/0 (socket says 172.21.15.42:40594) 2024-08-30T09:10:49.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:49.088+0000 7fe6fffff640 1 -- 172.21.15.42:0/3378854446 learned_addr learned my addr 172.21.15.42:0/3378854446 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:10:49.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:49.088+0000 7fe70513e640 1 --2- 172.21.15.42:0/3378854446 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe7001163e0 0x7fe700121480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:10:49.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:49.088+0000 7fe70493d640 1 --2- 172.21.15.42:0/3378854446 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe70010a330 0x7fe70011bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:10:49.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:49.088+0000 7fe70513e640 1 -- 172.21.15.42:0/3378854446 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe70010ac80 msgr2=0x7fe70011c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:10:49.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:49.088+0000 7fe70513e640 1 --2- 172.21.15.42:0/3378854446 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe70010ac80 0x7fe70011c1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:49.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:49.088+0000 7fe70513e640 1 -- 172.21.15.42:0/3378854446 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe70010a330 msgr2=0x7fe70011bc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:10:49.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:49.088+0000 7fe70513e640 1 --2- 172.21.15.42:0/3378854446 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe70010a330 0x7fe70011bc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:49.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:49.089+0000 7fe70513e640 1 -- 172.21.15.42:0/3378854446 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe700121bb0 con 0x7fe7001163e0 2024-08-30T09:10:49.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:49.089+0000 7fe70493d640 1 --2- 172.21.15.42:0/3378854446 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe70010a330 0x7fe70011bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-30T09:10:49.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:49.089+0000 7fe70513e640 1 --2- 172.21.15.42:0/3378854446 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe7001163e0 0x7fe700121480 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fe6f400d680 tx=0x7fe6f400db40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:10:49.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:49.090+0000 7fe6fdffb640 1 -- 172.21.15.42:0/3378854446 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6f4017070 con 0x7fe7001163e0 2024-08-30T09:10:49.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:49.091+0000 7fe6fdffb640 1 -- 172.21.15.42:0/3378854446 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe6f4009820 con 0x7fe7001163e0 2024-08-30T09:10:49.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:49.091+0000 7fe6fdffb640 1 -- 172.21.15.42:0/3378854446 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6f4004c40 con 0x7fe7001163e0 2024-08-30T09:10:49.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:49.092+0000 7fe706ba1640 1 -- 172.21.15.42:0/3378854446 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe700121e40 con 0x7fe7001163e0 2024-08-30T09:10:49.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:49.092+0000 7fe706ba1640 1 -- 172.21.15.42:0/3378854446 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe700124600 con 0x7fe7001163e0 2024-08-30T09:10:49.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:49.097+0000 7fe706ba1640 1 -- 172.21.15.42:0/3378854446 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe70010b380 con 0x7fe7001163e0 2024-08-30T09:10:49.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:49.100+0000 7fe6fdffb640 1 -- 172.21.15.42:0/3378854446 <== mon.2 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7fe6f4005220 con 0x7fe7001163e0 2024-08-30T09:10:49.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:49.102+0000 7fe6fdffb640 1 --2- 172.21.15.42:0/3378854446 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fe6e80b4d30 0x7fe6e80b7240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:49.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:49.102+0000 7fe6fdffb640 1 -- 172.21.15.42:0/3378854446 <== mon.2 v2:172.21.15.167:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7fe6f40e4ff0 con 0x7fe7001163e0 2024-08-30T09:10:49.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:49.105+0000 7fe70493d640 1 --2- 172.21.15.42:0/3378854446 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fe6e80b4d30 0x7fe6e80b7240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:10:49.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:49.106+0000 7fe70493d640 1 --2- 172.21.15.42:0/3378854446 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fe6e80b4d30 0x7fe6e80b7240 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fe700122510 tx=0x7fe6f0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:10:49.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:49.107+0000 7fe6fdffb640 1 -- 172.21.15.42:0/3378854446 <== mon.2 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7fe6f4020020 con 0x7fe7001163e0 2024-08-30T09:10:49.312 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.309+0000 7f2464494640 1 -- 172.21.15.42:0/536992827 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f245c103510 msgr2=0x7f245c109eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:10:49.312 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.309+0000 7f2464494640 1 --2- 172.21.15.42:0/536992827 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f245c103510 0x7f245c109eb0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f2454009be0 tx=0x7f245402a6d0 comp rx=0 tx=0).stop 2024-08-30T09:10:49.312 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.309+0000 7f2464494640 1 -- 172.21.15.42:0/536992827 shutdown_connections 2024-08-30T09:10:49.312 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.309+0000 7f2464494640 1 --2- 172.21.15.42:0/536992827 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f245c103510 0x7f245c109eb0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:49.312 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.310+0000 7f2464494640 1 --2- 172.21.15.42:0/536992827 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f245c102b70 0x7f245c102f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:49.312 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.310+0000 7f2464494640 1 --2- 172.21.15.42:0/536992827 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f245c101af0 0x7f245c101f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:49.312 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.310+0000 7f2464494640 1 -- 172.21.15.42:0/536992827 >> 172.21.15.42:0/536992827 conn(0x7f245c0fd430 msgr2=0x7f245c0ff8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:10:49.313 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.310+0000 7f2464494640 1 -- 172.21.15.42:0/536992827 shutdown_connections 2024-08-30T09:10:49.313 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.310+0000 7f2464494640 1 -- 172.21.15.42:0/536992827 wait complete. 2024-08-30T09:10:49.313 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.311+0000 7f2464494640 1 Processor -- start 2024-08-30T09:10:49.314 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.312+0000 7f2464494640 1 -- start start 2024-08-30T09:10:49.315 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.312+0000 7f2464494640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f245c101af0 0x7f245c1978b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:49.315 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.312+0000 7f2464494640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f245c102b70 0x7f245c197f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:49.315 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.312+0000 7f2464494640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f245c103510 0x7f245c198ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:49.315 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.313+0000 7f2461a2f640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f245c102b70 0x7f245c197f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:10:49.315 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.313+0000 7f2461a2f640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f245c102b70 0x7f245c197f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:58518/0 (socket says 172.21.15.42:58518) 2024-08-30T09:10:49.315 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.313+0000 7f2462230640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f245c101af0 0x7f245c1978b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:10:49.315 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.313+0000 7f2461a2f640 1 -- 172.21.15.42:0/1707122322 learned_addr learned my addr 172.21.15.42:0/1707122322 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:10:49.316 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.313+0000 7f2464494640 1 -- 172.21.15.42:0/1707122322 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f245c10cee0 con 0x7f245c101af0 2024-08-30T09:10:49.316 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.313+0000 7f2464494640 1 -- 172.21.15.42:0/1707122322 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f245c10cd60 con 0x7f245c102b70 2024-08-30T09:10:49.316 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.313+0000 7f2464494640 1 -- 172.21.15.42:0/1707122322 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f245c10d060 con 0x7f245c103510 2024-08-30T09:10:49.316 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.313+0000 7f2462a31640 1 --2- 172.21.15.42:0/1707122322 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f245c103510 0x7f245c198ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:10:49.317 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.314+0000 7f2462a31640 1 -- 172.21.15.42:0/1707122322 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f245c102b70 msgr2=0x7f245c197f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:10:49.317 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.314+0000 7f2462a31640 1 --2- 172.21.15.42:0/1707122322 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f245c102b70 0x7f245c197f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:49.317 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.314+0000 7f2462a31640 1 -- 172.21.15.42:0/1707122322 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f245c101af0 msgr2=0x7f245c1978b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:10:49.317 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.314+0000 7f2462a31640 1 --2- 172.21.15.42:0/1707122322 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f245c101af0 0x7f245c1978b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:49.317 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.314+0000 7f2462a31640 1 -- 172.21.15.42:0/1707122322 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f245c1992f0 con 0x7f245c103510 2024-08-30T09:10:49.317 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.314+0000 7f2462a31640 1 --2- 172.21.15.42:0/1707122322 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f245c103510 0x7f245c198ad0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f2454009b30 tx=0x7f2454003ed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:10:49.317 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.314+0000 7f24537fe640 1 -- 172.21.15.42:0/1707122322 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2454041030 con 0x7f245c103510 2024-08-30T09:10:49.317 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.314+0000 7f2464494640 1 -- 172.21.15.42:0/1707122322 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f245c199580 con 0x7f245c103510 2024-08-30T09:10:49.317 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.314+0000 7f2464494640 1 -- 172.21.15.42:0/1707122322 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f245c19fcd0 con 0x7f245c103510 2024-08-30T09:10:49.317 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.315+0000 7f24537fe640 1 -- 172.21.15.42:0/1707122322 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2454030970 con 0x7f245c103510 2024-08-30T09:10:49.318 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.315+0000 7f24537fe640 1 -- 172.21.15.42:0/1707122322 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2454036040 con 0x7f245c103510 2024-08-30T09:10:49.322 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.320+0000 7f246b405740 1 Processor -- start 2024-08-30T09:10:49.322 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.320+0000 7f24537fe640 1 -- 172.21.15.42:0/1707122322 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7f2454044070 con 0x7f245c103510 2024-08-30T09:10:49.322 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.321+0000 7f246b405740 1 -- start start 2024-08-30T09:10:49.323 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.321+0000 7f246b405740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56485ea011b0 0x56485eda7180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:49.324 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.321+0000 7f246b405740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56485e9d82f0 0x56485eda77f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:49.324 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.321+0000 7f246b405740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56485e9e04d0 0x56485eda8340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:49.324 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.321+0000 7f246b405740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x56485ed1a000 con 0x56485ea011b0 2024-08-30T09:10:49.324 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.321+0000 7f246b405740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x56485ed09590 con 0x56485e9e04d0 2024-08-30T09:10:49.324 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.321+0000 7f246b405740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x56485ed0d540 con 0x56485e9d82f0 2024-08-30T09:10:49.324 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.322+0000 7f2461a2f640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56485ea011b0 0x56485eda7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:10:49.324 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.322+0000 7f2461a2f640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56485e9e04d0 0x56485eda8340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:10:49.324 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.322+0000 7f2461a2f640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56485e9e04d0 0x56485eda8340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:58528/0 (socket says 172.21.15.42:58528) 2024-08-30T09:10:49.325 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.322+0000 7f2461a2f640 1 -- 172.21.15.42:0/2360563808 learned_addr learned my addr 172.21.15.42:0/2360563808 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:10:49.325 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.322+0000 7f2461a2f640 1 -- 172.21.15.42:0/2360563808 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56485e9d82f0 msgr2=0x56485eda77f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:10:49.325 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.322+0000 7f2461a2f640 1 --2- 172.21.15.42:0/2360563808 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56485e9d82f0 0x56485eda77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:49.326 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.322+0000 7f2461a2f640 1 -- 172.21.15.42:0/2360563808 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56485ea011b0 msgr2=0x56485eda7180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:10:49.326 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.322+0000 7f2461a2f640 1 --2- 172.21.15.42:0/2360563808 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56485ea011b0 0x56485eda7180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:49.326 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.322+0000 7f2461a2f640 1 -- 172.21.15.42:0/2360563808 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56485ea6c790 con 0x56485e9e04d0 2024-08-30T09:10:49.326 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.322+0000 7f2461a2f640 1 --2- 172.21.15.42:0/2360563808 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56485e9e04d0 0x56485eda8340 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f244c00da30 tx=0x7f244c005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=afaed80c411b58dd server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:10:49.326 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.323+0000 7f24537fe640 1 --2- 172.21.15.42:0/1707122322 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f24480b4c30 0x7f24480b7140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:49.326 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.323+0000 7f24537fe640 1 -- 172.21.15.42:0/1707122322 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f2454105fb0 con 0x7f245c103510 2024-08-30T09:10:49.327 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.323+0000 7f2450bfe640 1 -- 172.21.15.42:0/2360563808 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f244c01f040 con 0x56485e9e04d0 2024-08-30T09:10:49.327 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.323+0000 7f2450bfe640 1 -- 172.21.15.42:0/2360563808 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f244c0054d0 con 0x56485e9e04d0 2024-08-30T09:10:49.327 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.323+0000 7f2450bfe640 1 -- 172.21.15.42:0/2360563808 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f244c004b90 con 0x56485e9e04d0 2024-08-30T09:10:49.328 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.324+0000 7f2462230640 1 --2- 172.21.15.42:0/1707122322 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f24480b4c30 0x7f24480b7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:10:49.328 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.324+0000 7f246b405740 1 -- 172.21.15.42:0/2360563808 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56485e9e04d0 msgr2=0x56485eda8340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:10:49.328 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.324+0000 7f246b405740 1 --2- 172.21.15.42:0/2360563808 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56485e9e04d0 0x56485eda8340 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f244c00da30 tx=0x7f244c005f90 comp rx=0 tx=0).stop 2024-08-30T09:10:49.328 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.325+0000 7f2462230640 1 --2- 172.21.15.42:0/1707122322 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f24480b4c30 0x7f24480b7140 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f2458005fd0 tx=0x7f2458009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:10:49.328 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.326+0000 7f246b405740 1 -- 172.21.15.42:0/2360563808 shutdown_connections 2024-08-30T09:10:49.329 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.326+0000 7f246b405740 1 --2- 172.21.15.42:0/2360563808 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56485e9d82f0 0x56485eda77f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:49.329 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.326+0000 7f246b405740 1 --2- 172.21.15.42:0/2360563808 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56485e9e04d0 0x56485eda8340 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:49.329 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.326+0000 7f246b405740 1 --2- 172.21.15.42:0/2360563808 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56485ea011b0 0x56485eda7180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:49.329 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.326+0000 7f246b405740 1 -- 172.21.15.42:0/2360563808 >> 172.21.15.42:0/2360563808 conn(0x56485e9f5300 msgr2=0x56485eda2c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:10:49.331 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.330+0000 7f246b405740 1 -- 172.21.15.42:0/2360563808 shutdown_connections 2024-08-30T09:10:49.332 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.330+0000 7f246b405740 1 -- 172.21.15.42:0/2360563808 wait complete. 2024-08-30T09:10:49.332 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.330+0000 7f246b405740 1 -- start start 2024-08-30T09:10:49.333 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.331+0000 7f246b405740 1 Processor -- start 2024-08-30T09:10:49.333 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.331+0000 7f246b405740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56485e9d82f0 0x56485edb0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:49.334 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.332+0000 7f246b405740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56485e9e04d0 0x56485edb13a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:49.334 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.332+0000 7f2462230640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56485e9d82f0 0x56485edb0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:10:49.334 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.332+0000 7f2462230640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56485e9d82f0 0x56485edb0ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:40638/0 (socket says 172.21.15.42:40638) 2024-08-30T09:10:49.334 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.332+0000 7f2462230640 1 -- 172.21.15.42:0/268453773 learned_addr learned my addr 172.21.15.42:0/268453773 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:10:49.334 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.332+0000 7f246b405740 1 --2- 172.21.15.42:0/268453773 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56485ea011b0 0x56485edb1f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:49.335 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.332+0000 7f246b405740 1 -- 172.21.15.42:0/268453773 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x56485ec42ab0 con 0x56485e9e04d0 2024-08-30T09:10:49.335 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.332+0000 7f246b405740 1 -- 172.21.15.42:0/268453773 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x56485ec90680 con 0x56485ea011b0 2024-08-30T09:10:49.335 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.332+0000 7f2462a31640 1 --2- 172.21.15.42:0/268453773 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56485ea011b0 0x56485edb1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:10:49.335 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.332+0000 7f246b405740 1 -- 172.21.15.42:0/268453773 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x56485eda8e60 con 0x56485e9d82f0 2024-08-30T09:10:49.335 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.333+0000 7f246b405740 10 client.0 _ll_register_callbacks cb 0x56485ea18a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:10:49.335 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.333+0000 7f2462230640 1 -- 172.21.15.42:0/268453773 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56485ea011b0 msgr2=0x56485edb1f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:10:49.336 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.333+0000 7f2462230640 1 --2- 172.21.15.42:0/268453773 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56485ea011b0 0x56485edb1f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:49.336 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.333+0000 7f2462230640 1 -- 172.21.15.42:0/268453773 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56485e9e04d0 msgr2=0x56485edb13a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:10:49.336 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.333+0000 7f2462230640 1 --2- 172.21.15.42:0/268453773 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56485e9e04d0 0x56485edb13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:49.336 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.333+0000 7f2462230640 1 -- 172.21.15.42:0/268453773 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56485eae5620 con 0x56485e9d82f0 2024-08-30T09:10:49.336 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.333+0000 7f246b405740 1 -- 172.21.15.42:0/268453773 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f244c0054d0 con 0x56485e9d82f0 2024-08-30T09:10:49.339 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.337+0000 7f2462230640 1 --2- 172.21.15.42:0/268453773 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56485e9d82f0 0x56485edb0ce0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f2458018780 tx=0x7f2458004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=566fb938a81514bd server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:10:49.343 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.339+0000 7f2445ffb640 10 client.26878 ms_handle_connect on v2:172.21.15.190:3300/0 2024-08-30T09:10:49.343 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.342+0000 7f2445ffb640 1 -- 172.21.15.42:0/268453773 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2458026070 con 0x56485e9d82f0 2024-08-30T09:10:49.348 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.346+0000 7f2445ffb640 1 -- 172.21.15.42:0/268453773 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f2458004220 con 0x56485e9d82f0 2024-08-30T09:10:49.348 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.346+0000 7f2445ffb640 1 -- 172.21.15.42:0/268453773 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f2458004460 con 0x56485e9d82f0 2024-08-30T09:10:49.349 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.347+0000 7f246b405740 10 client.26878 fetch_fsmap learned FSMap version 39 2024-08-30T09:10:49.349 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.347+0000 7f246b405740 1 -- 172.21.15.42:0/268453773 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({fsmap=39}) -- 0x56485edbae40 con 0x56485e9d82f0 2024-08-30T09:10:49.355 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.352+0000 7f2445ffb640 1 -- 172.21.15.42:0/268453773 <== mon.1 v2:172.21.15.190:3300/0 4 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7f2458021070 con 0x56485e9d82f0 2024-08-30T09:10:49.355 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.353+0000 7f246b405740 10 client.26878 fetch_fsmap finished waiting for FSMap version 39 2024-08-30T09:10:49.355 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.353+0000 7f246b405740 10 client.26878 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-30T09:10:49.355 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.353+0000 7f246b405740 20 client.26878 populate_metadata read hostname 'smithi042' 2024-08-30T09:10:49.355 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.353+0000 7f246b405740 1 --2- 172.21.15.42:0/268453773 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x56485e9bfd80 0x56485edbe680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:49.355 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.353+0000 7f246b405740 4 client.26878 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-30T09:10:49.355 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.353+0000 7f246b405740 1 -- 172.21.15.42:0/268453773 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56485ed1bf50 con 0x56485e9bfd80 2024-08-30T09:10:49.355 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.353+0000 7f2461a2f640 1 --2- 172.21.15.42:0/268453773 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x56485e9bfd80 0x56485edbe680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:10:49.363 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.361+0000 7f2461a2f640 1 --2- 172.21.15.42:0/268453773 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x56485e9bfd80 0x56485edbe680 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=97f3007ec581891e server_cookie=e5428405ceccb763 in_seq=0 out_seq=0 2024-08-30T09:10:49.365 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.364+0000 7f2445ffb640 10 client.26878 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:10:49.370 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.368+0000 7f2445ffb640 1 -- 172.21.15.42:0/268453773 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56485ed1bf50 con 0x56485e9bfd80 2024-08-30T09:10:49.370 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.368+0000 7f2445ffb640 10 client.26878 handle_command_reply: tid=0 2024-08-30T09:10:49.374 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.372+0000 7f246b405740 1 client.26878 shutdown 2024-08-30T09:10:49.378 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.375+0000 7f246b405740 1 -- 172.21.15.42:0/268453773 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56485e9d82f0 msgr2=0x56485edb0ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:10:49.378 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.375+0000 7f246b405740 1 --2- 172.21.15.42:0/268453773 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56485e9d82f0 0x56485edb0ce0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f2458018780 tx=0x7f2458004730 comp rx=0 tx=0).stop 2024-08-30T09:10:49.378 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.375+0000 7f246b405740 1 -- 172.21.15.42:0/268453773 shutdown_connections 2024-08-30T09:10:49.378 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.375+0000 7f246b405740 1 --2- 172.21.15.42:0/268453773 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x56485e9bfd80 0x56485edbe680 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:49.378 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.375+0000 7f246b405740 1 --2- 172.21.15.42:0/268453773 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56485ea011b0 0x56485edb1f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:49.378 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.375+0000 7f246b405740 1 --2- 172.21.15.42:0/268453773 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56485e9e04d0 0x56485edb13a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:49.379 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.375+0000 7f246b405740 1 --2- 172.21.15.42:0/268453773 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56485e9d82f0 0x56485edb0ce0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:49.379 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.375+0000 7f246b405740 1 -- 172.21.15.42:0/268453773 >> 172.21.15.42:0/268453773 conn(0x56485e9f5300 msgr2=0x56485eda1170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:10:49.379 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.376+0000 7f2445ffb640 0 client.26878 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:10:49.382 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.379+0000 7f246b405740 1 -- 172.21.15.42:0/268453773 shutdown_connections 2024-08-30T09:10:49.382 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.379+0000 7f246b405740 1 -- 172.21.15.42:0/268453773 wait complete. 2024-08-30T09:10:49.384 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.382+0000 7f246b405740 20 client.26878 trim_cache size 0 max 16384 2024-08-30T09:10:49.436 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.434+0000 7f246b405740 1 Processor -- start 2024-08-30T09:10:49.436 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.434+0000 7f246b405740 1 -- start start 2024-08-30T09:10:49.436 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.434+0000 7f246b405740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56485e9d82f0 0x56485ed9d400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:49.436 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.434+0000 7f246b405740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56485e9e04d0 0x56485eda6e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:49.436 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.434+0000 7f246b405740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56485ea011b0 0x56485edaaff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:49.437 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.434+0000 7f246b405740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f24400124e0 con 0x56485e9d82f0 2024-08-30T09:10:49.437 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.434+0000 7f246b405740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f244000fd50 con 0x56485e9e04d0 2024-08-30T09:10:49.437 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.434+0000 7f246b405740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f244000dba0 con 0x56485ea011b0 2024-08-30T09:10:49.437 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.435+0000 7f2462230640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56485e9d82f0 0x56485ed9d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:10:49.437 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.435+0000 7f2462a31640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56485ea011b0 0x56485edaaff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:10:49.437 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.435+0000 7f2462a31640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56485ea011b0 0x56485edaaff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:40650/0 (socket says 172.21.15.42:40650) 2024-08-30T09:10:49.438 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.435+0000 7f2462230640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56485e9d82f0 0x56485ed9d400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:46134/0 (socket says 172.21.15.42:46134) 2024-08-30T09:10:49.438 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.435+0000 7f2462a31640 1 -- 172.21.15.42:0/2428646539 learned_addr learned my addr 172.21.15.42:0/2428646539 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:10:49.438 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.435+0000 7f2461a2f640 1 --2- 172.21.15.42:0/2428646539 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56485e9e04d0 0x56485eda6e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:10:49.441 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.439+0000 7f2462a31640 1 -- 172.21.15.42:0/2428646539 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56485e9e04d0 msgr2=0x56485eda6e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:10:49.441 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.439+0000 7f2462a31640 1 --2- 172.21.15.42:0/2428646539 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56485e9e04d0 0x56485eda6e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:49.442 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.439+0000 7f2462a31640 1 -- 172.21.15.42:0/2428646539 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56485e9d82f0 msgr2=0x56485ed9d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:10:49.442 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.439+0000 7f2462a31640 1 --2- 172.21.15.42:0/2428646539 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56485e9d82f0 0x56485ed9d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:49.442 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.439+0000 7f2462a31640 1 -- 172.21.15.42:0/2428646539 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56485e9bffb0 con 0x56485ea011b0 2024-08-30T09:10:49.443 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.441+0000 7f2462a31640 1 --2- 172.21.15.42:0/2428646539 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56485ea011b0 0x56485edaaff0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f24540c8670 tx=0x7f24540cc100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9d448fec0b7a0cba server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:10:49.443 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.442+0000 7f2447fff640 1 -- 172.21.15.42:0/2428646539 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24540cc5f0 con 0x56485ea011b0 2024-08-30T09:10:49.444 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.442+0000 7f2447fff640 1 -- 172.21.15.42:0/2428646539 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f24540d0e60 con 0x56485ea011b0 2024-08-30T09:10:49.445 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.443+0000 7f246b405740 1 -- 172.21.15.42:0/2428646539 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56485ea011b0 msgr2=0x56485edaaff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:10:49.445 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.443+0000 7f246b405740 1 --2- 172.21.15.42:0/2428646539 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56485ea011b0 0x56485edaaff0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f24540c8670 tx=0x7f24540cc100 comp rx=0 tx=0).stop 2024-08-30T09:10:49.445 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.443+0000 7f246b405740 1 -- 172.21.15.42:0/2428646539 shutdown_connections 2024-08-30T09:10:49.445 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.443+0000 7f246b405740 1 --2- 172.21.15.42:0/2428646539 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56485ea011b0 0x56485edaaff0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:49.445 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.443+0000 7f246b405740 1 --2- 172.21.15.42:0/2428646539 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56485e9e04d0 0x56485eda6e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:49.446 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.443+0000 7f246b405740 1 --2- 172.21.15.42:0/2428646539 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56485e9d82f0 0x56485ed9d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:49.446 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.443+0000 7f246b405740 1 -- 172.21.15.42:0/2428646539 >> 172.21.15.42:0/2428646539 conn(0x56485e9f5300 msgr2=0x56485edbe590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:10:49.446 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.444+0000 7f246b405740 1 -- 172.21.15.42:0/2428646539 shutdown_connections 2024-08-30T09:10:49.447 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.445+0000 7f246b405740 1 -- 172.21.15.42:0/2428646539 wait complete. 2024-08-30T09:10:49.447 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.445+0000 7f246b405740 1 -- start start 2024-08-30T09:10:49.448 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.446+0000 7f246b405740 1 Processor -- start 2024-08-30T09:10:49.448 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.446+0000 7f246b405740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56485e9d82f0 0x56485edb4480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:49.448 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.446+0000 7f246b405740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56485e9e04d0 0x56485edb4b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:49.448 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.446+0000 7f246b405740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56485e9951c0 0x56485edb8650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:49.448 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.446+0000 7f246b405740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f244000e210 con 0x56485e9d82f0 2024-08-30T09:10:49.448 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.446+0000 7f246b405740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f244000f680 con 0x56485e9951c0 2024-08-30T09:10:49.449 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.446+0000 7f246b405740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f244000e8e0 con 0x56485e9e04d0 2024-08-30T09:10:49.449 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.446+0000 7f246b405740 10 client.0 _ll_register_callbacks cb 0x7f245800b920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:10:49.449 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.446+0000 7f2462230640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56485e9d82f0 0x56485edb4480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:10:49.449 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.446+0000 7f2462230640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56485e9d82f0 0x56485edb4480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:46150/0 (socket says 172.21.15.42:46150) 2024-08-30T09:10:49.449 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.446+0000 7f2462230640 1 -- 172.21.15.42:0/3526652989 learned_addr learned my addr 172.21.15.42:0/3526652989 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:10:49.449 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.447+0000 7f2462230640 1 -- 172.21.15.42:0/3526652989 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56485e9e04d0 msgr2=0x56485edb4b40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-30T09:10:49.450 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.447+0000 7f2462230640 1 --2- 172.21.15.42:0/3526652989 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56485e9e04d0 0x56485edb4b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:49.450 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.447+0000 7f2462230640 1 -- 172.21.15.42:0/3526652989 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56485e9951c0 msgr2=0x56485edb8650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:10:49.450 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.447+0000 7f2462230640 1 --2- 172.21.15.42:0/3526652989 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56485e9951c0 0x56485edb8650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:49.450 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.447+0000 7f2462230640 1 -- 172.21.15.42:0/3526652989 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56485edb8db0 con 0x56485e9d82f0 2024-08-30T09:10:49.450 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.448+0000 7f2462230640 1 --2- 172.21.15.42:0/3526652989 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56485e9d82f0 0x56485edb4480 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f245800b9f0 tx=0x7f2458017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1f797e2c44e266c server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:10:49.450 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.448+0000 7f2445ffb640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-08-30T09:10:49.451 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.448+0000 7f2445ffb640 1 -- 172.21.15.42:0/3526652989 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2458026030 con 0x56485e9d82f0 2024-08-30T09:10:49.451 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.448+0000 7f246b405740 1 -- 172.21.15.42:0/3526652989 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56485eda1710 con 0x56485e9d82f0 2024-08-30T09:10:49.451 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.448+0000 7f2445ffb640 1 -- 172.21.15.42:0/3526652989 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f2458005250 con 0x56485e9d82f0 2024-08-30T09:10:49.451 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.449+0000 7f2445ffb640 1 -- 172.21.15.42:0/3526652989 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f2458047050 con 0x56485e9d82f0 2024-08-30T09:10:49.451 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.449+0000 7f246b405740 10 client.17088 fetch_fsmap learned FSMap version 39 2024-08-30T09:10:49.451 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.449+0000 7f246b405740 1 -- 172.21.15.42:0/3526652989 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x56485edb6320 con 0x56485e9d82f0 2024-08-30T09:10:49.452 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.450+0000 7f2445ffb640 1 -- 172.21.15.42:0/3526652989 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x56485eda1710 con 0x56485e9d82f0 2024-08-30T09:10:49.452 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.450+0000 7f2445ffb640 1 -- 172.21.15.42:0/3526652989 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7f2458033430 con 0x56485e9d82f0 2024-08-30T09:10:49.452 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.450+0000 7f246b405740 10 client.17088 fetch_fsmap finished waiting for FSMap version 39 2024-08-30T09:10:49.453 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.450+0000 7f246b405740 10 client.17088 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-30T09:10:49.453 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.450+0000 7f246b405740 20 client.17088 populate_metadata read hostname 'smithi042' 2024-08-30T09:10:49.453 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.450+0000 7f246b405740 1 --2- 172.21.15.42:0/3526652989 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x56485ee3a900 0x56485ee3cd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:49.453 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.450+0000 7f246b405740 4 client.17088 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-30T09:10:49.453 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.450+0000 7f246b405740 1 -- 172.21.15.42:0/3526652989 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56485ed179f0 con 0x56485ee3a900 2024-08-30T09:10:49.453 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.452+0000 7f2461a2f640 1 --2- 172.21.15.42:0/3526652989 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x56485ee3a900 0x56485ee3cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:10:49.455 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.453+0000 7f2461a2f640 1 --2- 172.21.15.42:0/3526652989 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x56485ee3a900 0x56485ee3cd70 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=116ce79b43ffd8fa server_cookie=77312f081f4a72d1 in_seq=0 out_seq=0 2024-08-30T09:10:49.455 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.453+0000 7f2445ffb640 10 client.17088 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:10:49.471 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.468+0000 7f2445ffb640 1 -- 172.21.15.42:0/3526652989 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56485ed179f0 con 0x56485ee3a900 2024-08-30T09:10:49.471 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.468+0000 7f2445ffb640 10 client.17088 handle_command_reply: tid=0 2024-08-30T09:10:49.471 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.468+0000 7f246b405740 1 client.17088 shutdown 2024-08-30T09:10:49.471 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.469+0000 7f246b405740 1 -- 172.21.15.42:0/3526652989 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56485e9d82f0 msgr2=0x56485edb4480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:10:49.471 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.469+0000 7f246b405740 1 --2- 172.21.15.42:0/3526652989 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56485e9d82f0 0x56485edb4480 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f245800b9f0 tx=0x7f2458017000 comp rx=0 tx=0).stop 2024-08-30T09:10:49.471 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.469+0000 7f246b405740 1 -- 172.21.15.42:0/3526652989 shutdown_connections 2024-08-30T09:10:49.471 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.469+0000 7f246b405740 1 --2- 172.21.15.42:0/3526652989 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x56485ee3a900 0x56485ee3cd70 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:49.471 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.469+0000 7f246b405740 1 --2- 172.21.15.42:0/3526652989 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56485e9e04d0 0x56485edb4b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:49.472 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.469+0000 7f246b405740 1 --2- 172.21.15.42:0/3526652989 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56485e9951c0 0x56485edb8650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:49.472 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.469+0000 7f246b405740 1 --2- 172.21.15.42:0/3526652989 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56485e9d82f0 0x56485edb4480 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:49.472 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.469+0000 7f2445ffb640 0 client.17088 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:10:49.472 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.469+0000 7f246b405740 1 -- 172.21.15.42:0/3526652989 >> 172.21.15.42:0/3526652989 conn(0x56485e9f5300 msgr2=0x56485edbe590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:10:49.472 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.469+0000 7f246b405740 1 -- 172.21.15.42:0/3526652989 shutdown_connections 2024-08-30T09:10:49.472 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.469+0000 7f246b405740 1 -- 172.21.15.42:0/3526652989 wait complete. 2024-08-30T09:10:49.472 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.469+0000 7f246b405740 20 client.17088 trim_cache size 0 max 16384 2024-08-30T09:10:49.473 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-08-30T09:10:49.473 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2024-08-30T09:10:49.473 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-08-30T09:10:49.473 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-08-30T09:10:49.474 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.470+0000 7f2464494640 1 -- 172.21.15.42:0/1707122322 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f24480b4c30 msgr2=0x7f24480b7140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:10:49.474 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.470+0000 7f2464494640 1 --2- 172.21.15.42:0/1707122322 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f24480b4c30 0x7f24480b7140 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f2458005fd0 tx=0x7f2458009260 comp rx=0 tx=0).stop 2024-08-30T09:10:49.474 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.470+0000 7f2464494640 1 -- 172.21.15.42:0/1707122322 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f245c103510 msgr2=0x7f245c198ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:10:49.474 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.470+0000 7f2464494640 1 --2- 172.21.15.42:0/1707122322 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f245c103510 0x7f245c198ad0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f2454009b30 tx=0x7f2454003ed0 comp rx=0 tx=0).stop 2024-08-30T09:10:49.474 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.470+0000 7f2464494640 1 -- 172.21.15.42:0/1707122322 shutdown_connections 2024-08-30T09:10:49.474 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.470+0000 7f2464494640 1 --2- 172.21.15.42:0/1707122322 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f24480b4c30 0x7f24480b7140 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:49.475 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.470+0000 7f2464494640 1 --2- 172.21.15.42:0/1707122322 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f245c103510 0x7f245c198ad0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:49.475 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.470+0000 7f2464494640 1 --2- 172.21.15.42:0/1707122322 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f245c102b70 0x7f245c197f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:49.475 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.470+0000 7f2464494640 1 --2- 172.21.15.42:0/1707122322 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f245c101af0 0x7f245c1978b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:49.475 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.470+0000 7f2464494640 1 -- 172.21.15.42:0/1707122322 >> 172.21.15.42:0/1707122322 conn(0x7f245c0fd430 msgr2=0x7f245c105e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:10:49.475 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.471+0000 7f2464494640 1 -- 172.21.15.42:0/1707122322 shutdown_connections 2024-08-30T09:10:49.475 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:10:49.471+0000 7f2464494640 1 -- 172.21.15.42:0/1707122322 wait complete. 2024-08-30T09:10:49.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:49.514+0000 7fe706ba1640 1 -- 172.21.15.42:0/3378854446 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe700111430 con 0x7fe7001163e0 2024-08-30T09:10:49.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:49.515+0000 7fe6fdffb640 1 -- 172.21.15.42:0/3378854446 <== mon.2 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+11921 (secure 0 0 0) 0x7fe6f40abdb0 con 0x7fe7001163e0 2024-08-30T09:10:49.519 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-08-30T09:10:49.519 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":39,"btime":"2024-08-30T09:09:50:819266+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14703,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/3188400000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":3188400000},{"type":"v1","addr":"172.21.15.42:6835","nonce":3188400000}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14715,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/3356537674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":3356537674},{"type":"v1","addr":"172.21.15.42:6837","nonce":3356537674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14766,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6839/2107785513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6838","nonce":2107785513},{"type":"v1","addr":"172.21.15.42:6839","nonce":2107785513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14778,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6841/2298438674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6840","nonce":2298438674},{"type":"v1","addr":"172.21.15.42:6841","nonce":2298438674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24553,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/543555570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":543555570},{"type":"v1","addr":"172.21.15.190:6833","nonce":543555570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24554,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/3907818941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":3907818941},{"type":"v1","addr":"172.21.15.167:6833","nonce":3907818941}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":39,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-30T08:58:03.055084+0000","modified":"2024-08-30T09:09:50.673616+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24629,"mds_1":24610,"mds_2":24599},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24572":{"gid":24572,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6835/234596845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":234596845},{"type":"v1","addr":"172.21.15.167:6835","nonce":234596845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24592":{"gid":24592,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6835/247905709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":247905709},{"type":"v1","addr":"172.21.15.190:6835","nonce":247905709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24598":{"gid":24598,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6837/863256349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6836","nonce":863256349},{"type":"v1","addr":"172.21.15.190:6837","nonce":863256349}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24599":{"gid":24599,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.167:6837/3011837856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":3011837856},{"type":"v1","addr":"172.21.15.167:6837","nonce":3011837856}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24610":{"gid":24610,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.190:6839/719981110","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6838","nonce":719981110},{"type":"v1","addr":"172.21.15.190:6839","nonce":719981110}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24629":{"gid":24629,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.167:6839/3604645207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":3604645207},{"type":"v1","addr":"172.21.15.167:6839","nonce":3604645207}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24629,"qdb_cluster":[24599,24610,24629]},"id":1}]} 2024-08-30T09:10:49.519 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 39 2024-08-30T09:10:49.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:49.519+0000 7fe706ba1640 1 -- 172.21.15.42:0/3378854446 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fe6e80b4d30 msgr2=0x7fe6e80b7240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:10:49.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:49.519+0000 7fe706ba1640 1 --2- 172.21.15.42:0/3378854446 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fe6e80b4d30 0x7fe6e80b7240 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fe700122510 tx=0x7fe6f0005f90 comp rx=0 tx=0).stop 2024-08-30T09:10:49.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:49.519+0000 7fe706ba1640 1 -- 172.21.15.42:0/3378854446 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe7001163e0 msgr2=0x7fe700121480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:10:49.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:49.519+0000 7fe706ba1640 1 --2- 172.21.15.42:0/3378854446 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe7001163e0 0x7fe700121480 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fe6f400d680 tx=0x7fe6f400db40 comp rx=0 tx=0).stop 2024-08-30T09:10:49.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:49.519+0000 7fe706ba1640 1 -- 172.21.15.42:0/3378854446 shutdown_connections 2024-08-30T09:10:49.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:49.519+0000 7fe706ba1640 1 --2- 172.21.15.42:0/3378854446 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fe6e80b4d30 0x7fe6e80b7240 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:49.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:49.519+0000 7fe706ba1640 1 --2- 172.21.15.42:0/3378854446 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe70010ac80 0x7fe70011c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:49.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:49.519+0000 7fe706ba1640 1 --2- 172.21.15.42:0/3378854446 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe7001163e0 0x7fe700121480 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:49.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:49.519+0000 7fe706ba1640 1 --2- 172.21.15.42:0/3378854446 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe70010a330 0x7fe70011bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:49.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:49.519+0000 7fe706ba1640 1 -- 172.21.15.42:0/3378854446 >> 172.21.15.42:0/3378854446 conn(0x7fe700070c60 msgr2=0x7fe700071ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:10:49.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:49.519+0000 7fe706ba1640 1 -- 172.21.15.42:0/3378854446 shutdown_connections 2024-08-30T09:10:49.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:49.519+0000 7fe706ba1640 1 -- 172.21.15.42:0/3378854446 wait complete. 2024-08-30T09:10:50.388 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-08-30T09:10:50.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:10:50 smithi042 ceph-mon[18652]: pgmap v675: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 7.2 MiB/s wr, 610 op/s 2024-08-30T09:10:50.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:10:50 smithi042 ceph-mon[18652]: from='client.? 172.21.15.42:0/3378854446' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-30T09:10:50.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:10:50 smithi190 ceph-mon[22001]: pgmap v675: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 7.2 MiB/s wr, 610 op/s 2024-08-30T09:10:50.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:10:50 smithi190 ceph-mon[22001]: from='client.? 172.21.15.42:0/3378854446' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-30T09:10:50.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:10:50 smithi167 ceph-mon[22063]: pgmap v675: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 7.2 MiB/s wr, 610 op/s 2024-08-30T09:10:50.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:10:50 smithi167 ceph-mon[22063]: from='client.? 172.21.15.42:0/3378854446' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-30T09:10:50.955 INFO:teuthology.orchestra.run.smithi042.stderr:progress: 605.0 s, 465.6 tps, lat 215.847 ms stddev 244.765 2024-08-30T09:10:50.959 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T09:10:51.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.549+0000 7fc56ba25640 1 -- 172.21.15.42:0/3501851249 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc564107990 msgr2=0x7fc564107e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:10:51.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.549+0000 7fc56ba25640 1 --2- 172.21.15.42:0/3501851249 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc564107990 0x7fc564107e70 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fc55c009cd0 tx=0x7fc55c02a8e0 comp rx=0 tx=0).stop 2024-08-30T09:10:51.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.550+0000 7fc56ba25640 1 -- 172.21.15.42:0/3501851249 shutdown_connections 2024-08-30T09:10:51.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.550+0000 7fc56ba25640 1 --2- 172.21.15.42:0/3501851249 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc564107990 0x7fc564107e70 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:51.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.550+0000 7fc56ba25640 1 --2- 172.21.15.42:0/3501851249 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc564102ea0 0x7fc5641073d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:51.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.550+0000 7fc56ba25640 1 --2- 172.21.15.42:0/3501851249 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc564102550 0x7fc564102960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:51.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.550+0000 7fc56ba25640 1 -- 172.21.15.42:0/3501851249 >> 172.21.15.42:0/3501851249 conn(0x7fc5640794d0 msgr2=0x7fc5640798e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:10:51.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.550+0000 7fc56ba25640 1 -- 172.21.15.42:0/3501851249 shutdown_connections 2024-08-30T09:10:51.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.550+0000 7fc56ba25640 1 -- 172.21.15.42:0/3501851249 wait complete. 2024-08-30T09:10:51.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.551+0000 7fc56ba25640 1 Processor -- start 2024-08-30T09:10:51.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.551+0000 7fc56ba25640 1 -- start start 2024-08-30T09:10:51.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.552+0000 7fc56ba25640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc564102550 0x7fc564116a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:51.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.552+0000 7fc56ba25640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc564102ea0 0x7fc56410fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:51.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.552+0000 7fc56ba25640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc564107990 0x7fc564110290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:51.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.552+0000 7fc56ba25640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fc564119ea0 con 0x7fc564102ea0 2024-08-30T09:10:51.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.552+0000 7fc56ba25640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc564119d20 con 0x7fc564107990 2024-08-30T09:10:51.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.552+0000 7fc56ba25640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fc56411a020 con 0x7fc564102550 2024-08-30T09:10:51.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.552+0000 7fc568fc0640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc564102ea0 0x7fc56410fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:10:51.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.552+0000 7fc568fc0640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc564102ea0 0x7fc56410fd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:46178/0 (socket says 172.21.15.42:46178) 2024-08-30T09:10:51.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.552+0000 7fc568fc0640 1 -- 172.21.15.42:0/2555590108 learned_addr learned my addr 172.21.15.42:0/2555590108 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:10:51.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.552+0000 7fc5697c1640 1 --2- 172.21.15.42:0/2555590108 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc564102550 0x7fc564116a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:10:51.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.552+0000 7fc569fc2640 1 --2- 172.21.15.42:0/2555590108 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc564107990 0x7fc564110290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:10:51.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.552+0000 7fc568fc0640 1 -- 172.21.15.42:0/2555590108 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc564102550 msgr2=0x7fc564116a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:10:51.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.552+0000 7fc568fc0640 1 --2- 172.21.15.42:0/2555590108 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc564102550 0x7fc564116a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:51.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.552+0000 7fc568fc0640 1 -- 172.21.15.42:0/2555590108 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc564107990 msgr2=0x7fc564110290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:10:51.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.552+0000 7fc568fc0640 1 --2- 172.21.15.42:0/2555590108 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc564107990 0x7fc564110290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:51.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.552+0000 7fc568fc0640 1 -- 172.21.15.42:0/2555590108 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc564110ab0 con 0x7fc564102ea0 2024-08-30T09:10:51.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.552+0000 7fc569fc2640 1 --2- 172.21.15.42:0/2555590108 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc564107990 0x7fc564110290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-30T09:10:51.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.552+0000 7fc568fc0640 1 --2- 172.21.15.42:0/2555590108 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc564102ea0 0x7fc56410fd10 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fc55400e9f0 tx=0x7fc55400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:10:51.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.553+0000 7fc55a7fc640 1 -- 172.21.15.42:0/2555590108 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc55400ccc0 con 0x7fc564102ea0 2024-08-30T09:10:51.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.553+0000 7fc56ba25640 1 -- 172.21.15.42:0/2555590108 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc564115770 con 0x7fc564102ea0 2024-08-30T09:10:51.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.553+0000 7fc56ba25640 1 -- 172.21.15.42:0/2555590108 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc564115c40 con 0x7fc564102ea0 2024-08-30T09:10:51.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.553+0000 7fc55a7fc640 1 -- 172.21.15.42:0/2555590108 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc5540108e0 con 0x7fc564102ea0 2024-08-30T09:10:51.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.554+0000 7fc55a7fc640 1 -- 172.21.15.42:0/2555590108 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5540187a0 con 0x7fc564102ea0 2024-08-30T09:10:51.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.556+0000 7fc571f9a740 1 Processor -- start 2024-08-30T09:10:51.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.556+0000 7fc571f9a740 1 -- start start 2024-08-30T09:10:51.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.556+0000 7fc571f9a740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5586e84d6070 0x5586e887b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:51.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.556+0000 7fc571f9a740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5586e84ad1b0 0x5586e887b850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:51.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.556+0000 7fc571f9a740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5586e84b5390 0x5586e887bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:51.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.556+0000 7fc571f9a740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x5586e87f2680 con 0x5586e84ad1b0 2024-08-30T09:10:51.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.556+0000 7fc571f9a740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5586e87e1c10 con 0x5586e84b5390 2024-08-30T09:10:51.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.556+0000 7fc571f9a740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5586e87e5bc0 con 0x5586e84d6070 2024-08-30T09:10:51.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.556+0000 7fc55a7fc640 1 -- 172.21.15.42:0/2555590108 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7fc554010430 con 0x7fc564102ea0 2024-08-30T09:10:51.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.557+0000 7fc569fc2640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5586e84d6070 0x5586e887b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:10:51.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.557+0000 7fc569fc2640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5586e84d6070 0x5586e887b310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:40674/0 (socket says 172.21.15.42:40674) 2024-08-30T09:10:51.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.557+0000 7fc569fc2640 1 -- 172.21.15.42:0/3273675340 learned_addr learned my addr 172.21.15.42:0/3273675340 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:10:51.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.557+0000 7fc568fc0640 1 --2- 172.21.15.42:0/3273675340 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5586e84b5390 0x5586e887bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:10:51.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.557+0000 7fc569fc2640 1 -- 172.21.15.42:0/3273675340 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5586e84b5390 msgr2=0x5586e887bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:10:51.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.557+0000 7fc569fc2640 1 --2- 172.21.15.42:0/3273675340 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5586e84b5390 0x5586e887bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:51.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.557+0000 7fc569fc2640 1 -- 172.21.15.42:0/3273675340 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5586e84ad1b0 msgr2=0x5586e887b850 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-30T09:10:51.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.558+0000 7fc569fc2640 1 --2- 172.21.15.42:0/3273675340 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5586e84ad1b0 0x5586e887b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:51.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.558+0000 7fc569fc2640 1 -- 172.21.15.42:0/3273675340 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5586e8541650 con 0x5586e84d6070 2024-08-30T09:10:51.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.558+0000 7fc55a7fc640 1 --2- 172.21.15.42:0/2555590108 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fc5500b4c30 0x7fc5500b7140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:51.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.558+0000 7fc5697c1640 1 --2- 172.21.15.42:0/2555590108 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fc5500b4c30 0x7fc5500b7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:10:51.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.559+0000 7fc55a7fc640 1 -- 172.21.15.42:0/2555590108 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7fc5540e57a0 con 0x7fc564102ea0 2024-08-30T09:10:51.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.559+0000 7fc569fc2640 1 --2- 172.21.15.42:0/3273675340 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5586e84d6070 0x5586e887b310 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fc55c009cd0 tx=0x7fc55c031430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=18dde1d9bab6322d server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:10:51.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.559+0000 7fc5697c1640 1 --2- 172.21.15.42:0/2555590108 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fc5500b4c30 0x7fc5500b7140 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fc560009710 tx=0x7fc5600096a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:10:51.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.559+0000 7fc54f7fe640 1 -- 172.21.15.42:0/3273675340 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc55c031850 con 0x5586e84d6070 2024-08-30T09:10:51.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.559+0000 7fc54f7fe640 1 -- 172.21.15.42:0/3273675340 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc55c036040 con 0x5586e84d6070 2024-08-30T09:10:51.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.559+0000 7fc54f7fe640 1 -- 172.21.15.42:0/3273675340 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc55c004630 con 0x5586e84d6070 2024-08-30T09:10:51.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.560+0000 7fc571f9a740 1 -- 172.21.15.42:0/3273675340 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5586e84d6070 msgr2=0x5586e887b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:10:51.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.560+0000 7fc571f9a740 1 --2- 172.21.15.42:0/3273675340 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5586e84d6070 0x5586e887b310 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fc55c009cd0 tx=0x7fc55c031430 comp rx=0 tx=0).stop 2024-08-30T09:10:51.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.560+0000 7fc571f9a740 1 -- 172.21.15.42:0/3273675340 shutdown_connections 2024-08-30T09:10:51.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.560+0000 7fc571f9a740 1 --2- 172.21.15.42:0/3273675340 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5586e84b5390 0x5586e887bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:51.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.560+0000 7fc571f9a740 1 --2- 172.21.15.42:0/3273675340 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5586e84ad1b0 0x5586e887b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:51.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.560+0000 7fc571f9a740 1 --2- 172.21.15.42:0/3273675340 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5586e84d6070 0x5586e887b310 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:51.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.560+0000 7fc571f9a740 1 -- 172.21.15.42:0/3273675340 >> 172.21.15.42:0/3273675340 conn(0x5586e84ca460 msgr2=0x5586e8879200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:10:51.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.561+0000 7fc571f9a740 1 -- 172.21.15.42:0/3273675340 shutdown_connections 2024-08-30T09:10:51.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.561+0000 7fc571f9a740 1 -- 172.21.15.42:0/3273675340 wait complete. 2024-08-30T09:10:51.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.561+0000 7fc571f9a740 1 -- start start 2024-08-30T09:10:51.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.561+0000 7fc571f9a740 1 Processor -- start 2024-08-30T09:10:51.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.562+0000 7fc571f9a740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5586e84ad1b0 0x5586e8886b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:51.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.562+0000 7fc571f9a740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5586e84b5390 0x5586e8887050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:51.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.562+0000 7fc571f9a740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5586e84d6070 0x5586e8892e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:51.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.562+0000 7fc568fc0640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5586e84b5390 0x5586e8887050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:10:51.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.562+0000 7fc568fc0640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5586e84b5390 0x5586e8887050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:46204/0 (socket says 172.21.15.42:46204) 2024-08-30T09:10:51.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.562+0000 7fc568fc0640 1 -- 172.21.15.42:0/1093075662 learned_addr learned my addr 172.21.15.42:0/1093075662 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:10:51.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.562+0000 7fc5697c1640 1 --2- 172.21.15.42:0/1093075662 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5586e84ad1b0 0x5586e8886b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:10:51.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.562+0000 7fc571f9a740 1 -- 172.21.15.42:0/1093075662 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x5586e8644700 con 0x5586e84b5390 2024-08-30T09:10:51.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.562+0000 7fc571f9a740 1 -- 172.21.15.42:0/1093075662 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5586e8768750 con 0x5586e84d6070 2024-08-30T09:10:51.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.562+0000 7fc571f9a740 1 -- 172.21.15.42:0/1093075662 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5586e8889a90 con 0x5586e84ad1b0 2024-08-30T09:10:51.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.562+0000 7fc569fc2640 1 --2- 172.21.15.42:0/1093075662 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5586e84d6070 0x5586e8892e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:10:51.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.562+0000 7fc571f9a740 10 client.0 _ll_register_callbacks cb 0x5586e8797e10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:10:51.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.562+0000 7fc5697c1640 1 -- 172.21.15.42:0/1093075662 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5586e84d6070 msgr2=0x5586e8892e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:10:51.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.562+0000 7fc5697c1640 1 --2- 172.21.15.42:0/1093075662 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5586e84d6070 0x5586e8892e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:51.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.562+0000 7fc5697c1640 1 -- 172.21.15.42:0/1093075662 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5586e84b5390 msgr2=0x5586e8887050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:10:51.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.562+0000 7fc5697c1640 1 --2- 172.21.15.42:0/1093075662 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5586e84b5390 0x5586e8887050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:51.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.562+0000 7fc5697c1640 1 -- 172.21.15.42:0/1093075662 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5586e87a1900 con 0x5586e84ad1b0 2024-08-30T09:10:51.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.563+0000 7fc5697c1640 1 --2- 172.21.15.42:0/1093075662 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5586e84ad1b0 0x5586e8886b10 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fc560004700 tx=0x7fc560005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=17c679a7efa07c9a server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:10:51.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.563+0000 7fc54cff9640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-08-30T09:10:51.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.563+0000 7fc54cff9640 1 -- 172.21.15.42:0/1093075662 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc56002b060 con 0x5586e84ad1b0 2024-08-30T09:10:51.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.563+0000 7fc54cff9640 1 -- 172.21.15.42:0/1093075662 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7fc56001a790 con 0x5586e84ad1b0 2024-08-30T09:10:51.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.563+0000 7fc571f9a740 1 -- 172.21.15.42:0/1093075662 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc55c036040 con 0x5586e84ad1b0 2024-08-30T09:10:51.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.563+0000 7fc54cff9640 1 -- 172.21.15.42:0/1093075662 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fc560032050 con 0x5586e84ad1b0 2024-08-30T09:10:51.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.563+0000 7fc571f9a740 10 client.26893 fetch_fsmap learned FSMap version 39 2024-08-30T09:10:51.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.563+0000 7fc571f9a740 1 -- 172.21.15.42:0/1093075662 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5586e8895990 con 0x5586e84ad1b0 2024-08-30T09:10:51.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.563+0000 7fc54cff9640 1 -- 172.21.15.42:0/1093075662 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc55c036040 con 0x5586e84ad1b0 2024-08-30T09:10:51.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.563+0000 7fc54cff9640 1 -- 172.21.15.42:0/1093075662 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7fc56000fa90 con 0x5586e84ad1b0 2024-08-30T09:10:51.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.563+0000 7fc571f9a740 10 client.26893 fetch_fsmap finished waiting for FSMap version 39 2024-08-30T09:10:51.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.563+0000 7fc571f9a740 10 client.26893 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-08-30T09:10:51.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.564+0000 7fc571f9a740 20 client.26893 populate_metadata read hostname 'smithi042' 2024-08-30T09:10:51.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.564+0000 7fc571f9a740 1 --2- 172.21.15.42:0/1093075662 >> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] conn(0x5586e84949f0 0x5586e8898f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:51.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.564+0000 7fc571f9a740 4 client.26893 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-30T09:10:51.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.564+0000 7fc571f9a740 1 -- 172.21.15.42:0/1093075662 --> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5586e87f45d0 con 0x5586e84949f0 2024-08-30T09:10:51.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.564+0000 7fc568fc0640 1 --2- 172.21.15.42:0/1093075662 >> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] conn(0x5586e84949f0 0x5586e8898f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:10:51.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.564+0000 7fc568fc0640 1 --2- 172.21.15.42:0/1093075662 >> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] conn(0x5586e84949f0 0x5586e8898f30 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b39fc3202d0a8cb0 server_cookie=cdca1a0367beb317 in_seq=0 out_seq=0 2024-08-30T09:10:51.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.564+0000 7fc54cff9640 10 client.26893 ms_handle_connect on v2:172.21.15.167:6836/3011837856 2024-08-30T09:10:51.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.565+0000 7fc54cff9640 1 -- 172.21.15.42:0/1093075662 <== mds.2 v2:172.21.15.167:6836/3011837856 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5586e87f45d0 con 0x5586e84949f0 2024-08-30T09:10:51.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.565+0000 7fc54cff9640 10 client.26893 handle_command_reply: tid=0 2024-08-30T09:10:51.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.565+0000 7fc571f9a740 1 client.26893 shutdown 2024-08-30T09:10:51.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.565+0000 7fc571f9a740 1 -- 172.21.15.42:0/1093075662 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5586e84ad1b0 msgr2=0x5586e8886b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:10:51.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.565+0000 7fc571f9a740 1 --2- 172.21.15.42:0/1093075662 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5586e84ad1b0 0x5586e8886b10 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fc560004700 tx=0x7fc560005f90 comp rx=0 tx=0).stop 2024-08-30T09:10:51.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.565+0000 7fc571f9a740 1 -- 172.21.15.42:0/1093075662 shutdown_connections 2024-08-30T09:10:51.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.565+0000 7fc571f9a740 1 --2- 172.21.15.42:0/1093075662 >> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] conn(0x5586e84949f0 0x5586e8898f30 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:51.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.565+0000 7fc54cff9640 0 client.26893 ms_handle_reset on v2:172.21.15.167:6836/3011837856 2024-08-30T09:10:51.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.565+0000 7fc571f9a740 1 --2- 172.21.15.42:0/1093075662 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5586e84d6070 0x5586e8892e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:51.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.565+0000 7fc571f9a740 1 --2- 172.21.15.42:0/1093075662 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5586e84b5390 0x5586e8887050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:51.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.565+0000 7fc571f9a740 1 --2- 172.21.15.42:0/1093075662 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5586e84ad1b0 0x5586e8886b10 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:51.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.565+0000 7fc571f9a740 1 -- 172.21.15.42:0/1093075662 >> 172.21.15.42:0/1093075662 conn(0x5586e84ca460 msgr2=0x5586e887e4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:10:51.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.566+0000 7fc571f9a740 1 -- 172.21.15.42:0/1093075662 shutdown_connections 2024-08-30T09:10:51.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.566+0000 7fc571f9a740 1 -- 172.21.15.42:0/1093075662 wait complete. 2024-08-30T09:10:51.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.566+0000 7fc571f9a740 20 client.26893 trim_cache size 0 max 16384 2024-08-30T09:10:51.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.585+0000 7fc571f9a740 1 Processor -- start 2024-08-30T09:10:51.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.586+0000 7fc571f9a740 1 -- start start 2024-08-30T09:10:51.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.586+0000 7fc571f9a740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5586e84ad1b0 0x5586e8879230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:51.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.586+0000 7fc571f9a740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5586e84b5390 0x5586e8892d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:51.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.586+0000 7fc571f9a740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5586e84d6070 0x5586e8893250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:51.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.586+0000 7fc571f9a740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fc5480124e0 con 0x5586e84b5390 2024-08-30T09:10:51.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.586+0000 7fc571f9a740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc54800fd50 con 0x5586e84ad1b0 2024-08-30T09:10:51.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.586+0000 7fc571f9a740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fc54800dba0 con 0x5586e84d6070 2024-08-30T09:10:51.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.587+0000 7fc568fc0640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5586e84b5390 0x5586e8892d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:10:51.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.587+0000 7fc568fc0640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5586e84b5390 0x5586e8892d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:46214/0 (socket says 172.21.15.42:46214) 2024-08-30T09:10:51.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.587+0000 7fc568fc0640 1 -- 172.21.15.42:0/934111172 learned_addr learned my addr 172.21.15.42:0/934111172 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:10:51.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.587+0000 7fc5697c1640 1 --2- 172.21.15.42:0/934111172 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5586e84ad1b0 0x5586e8879230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:10:51.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.587+0000 7fc569fc2640 1 --2- 172.21.15.42:0/934111172 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5586e84d6070 0x5586e8893250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:10:51.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.587+0000 7fc568fc0640 1 -- 172.21.15.42:0/934111172 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5586e84d6070 msgr2=0x5586e8893250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:10:51.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.587+0000 7fc568fc0640 1 --2- 172.21.15.42:0/934111172 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5586e84d6070 0x5586e8893250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:51.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.587+0000 7fc568fc0640 1 -- 172.21.15.42:0/934111172 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5586e84ad1b0 msgr2=0x5586e8879230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:10:51.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.587+0000 7fc568fc0640 1 --2- 172.21.15.42:0/934111172 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5586e84ad1b0 0x5586e8879230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:51.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.587+0000 7fc568fc0640 1 -- 172.21.15.42:0/934111172 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5586e88957c0 con 0x5586e84b5390 2024-08-30T09:10:51.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.588+0000 7fc568fc0640 1 --2- 172.21.15.42:0/934111172 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5586e84b5390 0x5586e8892d10 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fc55400cf10 tx=0x7fc5540ea040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b784a2a86b9607a4 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:10:51.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.588+0000 7fc54effd640 1 -- 172.21.15.42:0/934111172 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5540af070 con 0x5586e84b5390 2024-08-30T09:10:51.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.588+0000 7fc54effd640 1 -- 172.21.15.42:0/934111172 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc554010720 con 0x5586e84b5390 2024-08-30T09:10:51.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.588+0000 7fc54effd640 1 -- 172.21.15.42:0/934111172 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5540b5100 con 0x5586e84b5390 2024-08-30T09:10:51.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.589+0000 7fc571f9a740 1 -- 172.21.15.42:0/934111172 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5586e84b5390 msgr2=0x5586e8892d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:10:51.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.589+0000 7fc571f9a740 1 --2- 172.21.15.42:0/934111172 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5586e84b5390 0x5586e8892d10 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fc55400cf10 tx=0x7fc5540ea040 comp rx=0 tx=0).stop 2024-08-30T09:10:51.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.589+0000 7fc571f9a740 1 -- 172.21.15.42:0/934111172 shutdown_connections 2024-08-30T09:10:51.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.589+0000 7fc571f9a740 1 --2- 172.21.15.42:0/934111172 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5586e84d6070 0x5586e8893250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:51.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.589+0000 7fc571f9a740 1 --2- 172.21.15.42:0/934111172 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5586e84b5390 0x5586e8892d10 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:51.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.589+0000 7fc571f9a740 1 --2- 172.21.15.42:0/934111172 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5586e84ad1b0 0x5586e8879230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:51.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.589+0000 7fc571f9a740 1 -- 172.21.15.42:0/934111172 >> 172.21.15.42:0/934111172 conn(0x5586e84ca460 msgr2=0x5586e8878b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:10:51.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.589+0000 7fc571f9a740 1 -- 172.21.15.42:0/934111172 shutdown_connections 2024-08-30T09:10:51.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.589+0000 7fc571f9a740 1 -- 172.21.15.42:0/934111172 wait complete. 2024-08-30T09:10:51.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.590+0000 7fc571f9a740 1 -- start start 2024-08-30T09:10:51.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.590+0000 7fc571f9a740 1 Processor -- start 2024-08-30T09:10:51.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.590+0000 7fc571f9a740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5586e84ad1b0 0x5586e888afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:51.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.591+0000 7fc571f9a740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5586e84b5390 0x5586e888b510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:51.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.591+0000 7fc571f9a740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5586e84d6070 0x5586e8902930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:51.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.591+0000 7fc568fc0640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5586e84b5390 0x5586e888b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:10:51.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.591+0000 7fc568fc0640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5586e84b5390 0x5586e888b510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:58620/0 (socket says 172.21.15.42:58620) 2024-08-30T09:10:51.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.591+0000 7fc569fc2640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5586e84d6070 0x5586e8902930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:10:51.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.591+0000 7fc571f9a740 1 -- 172.21.15.42:0/1323514688 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fc54800e210 con 0x5586e84d6070 2024-08-30T09:10:51.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.591+0000 7fc571f9a740 1 -- 172.21.15.42:0/1323514688 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc54800f680 con 0x5586e84b5390 2024-08-30T09:10:51.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.591+0000 7fc571f9a740 1 -- 172.21.15.42:0/1323514688 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fc54800e8e0 con 0x5586e84ad1b0 2024-08-30T09:10:51.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.591+0000 7fc568fc0640 1 -- 172.21.15.42:0/1323514688 learned_addr learned my addr 172.21.15.42:0/1323514688 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:10:51.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.591+0000 7fc5697c1640 1 --2- 172.21.15.42:0/1323514688 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5586e84ad1b0 0x5586e888afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:10:51.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.591+0000 7fc571f9a740 10 client.0 _ll_register_callbacks cb 0x7fc560009a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:10:51.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.591+0000 7fc568fc0640 1 -- 172.21.15.42:0/1323514688 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5586e84ad1b0 msgr2=0x5586e888afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:10:51.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.591+0000 7fc568fc0640 1 --2- 172.21.15.42:0/1323514688 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5586e84ad1b0 0x5586e888afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:51.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.591+0000 7fc568fc0640 1 -- 172.21.15.42:0/1323514688 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5586e84d6070 msgr2=0x5586e8902930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:10:51.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.591+0000 7fc568fc0640 1 --2- 172.21.15.42:0/1323514688 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5586e84d6070 0x5586e8902930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:51.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.591+0000 7fc568fc0640 1 -- 172.21.15.42:0/1323514688 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5586e888bd20 con 0x5586e84b5390 2024-08-30T09:10:51.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.591+0000 7fc568fc0640 1 --2- 172.21.15.42:0/1323514688 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5586e84b5390 0x5586e888b510 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fc554093010 tx=0x7fc5540abde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=685fd33b34cbf35c server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:10:51.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.592+0000 7fc54cff9640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-30T09:10:51.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.592+0000 7fc54cff9640 1 -- 172.21.15.42:0/1323514688 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5540af070 con 0x5586e84b5390 2024-08-30T09:10:51.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.592+0000 7fc54cff9640 1 -- 172.21.15.42:0/1323514688 <== mon.2 v2:172.21.15.167:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7fc5540a8070 con 0x5586e84b5390 2024-08-30T09:10:51.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.592+0000 7fc571f9a740 1 -- 172.21.15.42:0/1323514688 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5586e87f0070 con 0x5586e84b5390 2024-08-30T09:10:51.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.592+0000 7fc54cff9640 1 -- 172.21.15.42:0/1323514688 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fc5540ac4e0 con 0x5586e84b5390 2024-08-30T09:10:51.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.592+0000 7fc571f9a740 10 client.26912 fetch_fsmap learned FSMap version 39 2024-08-30T09:10:51.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.592+0000 7fc571f9a740 1 -- 172.21.15.42:0/1323514688 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5586e8903db0 con 0x5586e84b5390 2024-08-30T09:10:51.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.592+0000 7fc54cff9640 1 -- 172.21.15.42:0/1323514688 <== mon.2 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc5540b5100 con 0x5586e84b5390 2024-08-30T09:10:51.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.593+0000 7fc54cff9640 1 -- 172.21.15.42:0/1323514688 <== mon.2 v2:172.21.15.167:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7fc554093140 con 0x5586e84b5390 2024-08-30T09:10:51.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.593+0000 7fc571f9a740 10 client.26912 fetch_fsmap finished waiting for FSMap version 39 2024-08-30T09:10:51.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.593+0000 7fc571f9a740 10 client.26912 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-08-30T09:10:51.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.593+0000 7fc571f9a740 20 client.26912 populate_metadata read hostname 'smithi042' 2024-08-30T09:10:51.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.593+0000 7fc571f9a740 1 --2- 172.21.15.42:0/1323514688 >> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] conn(0x5586e8469da0 0x5586e8906510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:51.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.593+0000 7fc571f9a740 4 client.26912 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-30T09:10:51.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.593+0000 7fc571f9a740 1 -- 172.21.15.42:0/1323514688 --> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5586e87f0070 con 0x5586e8469da0 2024-08-30T09:10:51.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.594+0000 7fc5697c1640 1 --2- 172.21.15.42:0/1323514688 >> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] conn(0x5586e8469da0 0x5586e8906510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:10:51.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.594+0000 7fc54cff9640 10 client.26912 ms_handle_connect on v2:172.21.15.167:6836/3011837856 2024-08-30T09:10:51.703 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.595+0000 7fc5697c1640 1 --2- 172.21.15.42:0/1323514688 >> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] conn(0x5586e8469da0 0x5586e8906510 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4f5fcbe677093c25 server_cookie=e160278ada0b06d9 in_seq=0 out_seq=0 2024-08-30T09:10:51.703 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.595+0000 7fc54cff9640 1 -- 172.21.15.42:0/1323514688 <== mds.2 v2:172.21.15.167:6836/3011837856 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5586e87f0070 con 0x5586e8469da0 2024-08-30T09:10:51.703 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.595+0000 7fc54cff9640 10 client.26912 handle_command_reply: tid=0 2024-08-30T09:10:51.703 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.595+0000 7fc571f9a740 1 client.26912 shutdown 2024-08-30T09:10:51.703 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.595+0000 7fc571f9a740 1 -- 172.21.15.42:0/1323514688 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5586e84b5390 msgr2=0x5586e888b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:10:51.703 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.595+0000 7fc571f9a740 1 --2- 172.21.15.42:0/1323514688 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5586e84b5390 0x5586e888b510 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fc554093010 tx=0x7fc5540abde0 comp rx=0 tx=0).stop 2024-08-30T09:10:51.703 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.595+0000 7fc571f9a740 1 -- 172.21.15.42:0/1323514688 shutdown_connections 2024-08-30T09:10:51.703 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.596+0000 7fc571f9a740 1 --2- 172.21.15.42:0/1323514688 >> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] conn(0x5586e8469da0 0x5586e8906510 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:51.703 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.596+0000 7fc571f9a740 1 --2- 172.21.15.42:0/1323514688 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5586e84d6070 0x5586e8902930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:51.703 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.596+0000 7fc571f9a740 1 --2- 172.21.15.42:0/1323514688 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5586e84b5390 0x5586e888b510 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:51.703 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.596+0000 7fc571f9a740 1 --2- 172.21.15.42:0/1323514688 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5586e84ad1b0 0x5586e888afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:51.703 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.596+0000 7fc571f9a740 1 -- 172.21.15.42:0/1323514688 >> 172.21.15.42:0/1323514688 conn(0x5586e84ca460 msgr2=0x5586e887e410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:10:51.703 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.596+0000 7fc54cff9640 0 client.26912 ms_handle_reset on v2:172.21.15.167:6836/3011837856 2024-08-30T09:10:51.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.596+0000 7fc571f9a740 1 -- 172.21.15.42:0/1323514688 shutdown_connections 2024-08-30T09:10:51.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.596+0000 7fc571f9a740 1 -- 172.21.15.42:0/1323514688 wait complete. 2024-08-30T09:10:51.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.596+0000 7fc571f9a740 20 client.26912 trim_cache size 0 max 16384 2024-08-30T09:10:51.704 INFO:teuthology.orchestra.run.smithi042.stdout:[] 2024-08-30T09:10:51.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.597+0000 7fc56ba25640 1 -- 172.21.15.42:0/2555590108 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fc5500b4c30 msgr2=0x7fc5500b7140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:10:51.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.597+0000 7fc56ba25640 1 --2- 172.21.15.42:0/2555590108 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fc5500b4c30 0x7fc5500b7140 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fc560009710 tx=0x7fc5600096a0 comp rx=0 tx=0).stop 2024-08-30T09:10:51.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.597+0000 7fc56ba25640 1 -- 172.21.15.42:0/2555590108 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc564102ea0 msgr2=0x7fc56410fd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:10:51.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.597+0000 7fc56ba25640 1 --2- 172.21.15.42:0/2555590108 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc564102ea0 0x7fc56410fd10 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fc55400e9f0 tx=0x7fc55400eeb0 comp rx=0 tx=0).stop 2024-08-30T09:10:51.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.597+0000 7fc56ba25640 1 -- 172.21.15.42:0/2555590108 shutdown_connections 2024-08-30T09:10:51.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.597+0000 7fc56ba25640 1 --2- 172.21.15.42:0/2555590108 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fc5500b4c30 0x7fc5500b7140 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:51.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.597+0000 7fc56ba25640 1 --2- 172.21.15.42:0/2555590108 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc564107990 0x7fc564110290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:51.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.597+0000 7fc56ba25640 1 --2- 172.21.15.42:0/2555590108 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc564102ea0 0x7fc56410fd10 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:51.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.597+0000 7fc56ba25640 1 --2- 172.21.15.42:0/2555590108 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc564102550 0x7fc564116a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:51.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.597+0000 7fc56ba25640 1 -- 172.21.15.42:0/2555590108 >> 172.21.15.42:0/2555590108 conn(0x7fc5640794d0 msgr2=0x7fc5641053a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:10:51.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.597+0000 7fc56ba25640 1 -- 172.21.15.42:0/2555590108 shutdown_connections 2024-08-30T09:10:51.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:51.597+0000 7fc56ba25640 1 -- 172.21.15.42:0/2555590108 wait complete. 2024-08-30T09:10:52.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:10:52 smithi042 ceph-mon[18652]: pgmap v676: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.2 MiB/s wr, 464 op/s 2024-08-30T09:10:52.450 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-08-30T09:10:52.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:10:52 smithi167 ceph-mon[22063]: pgmap v676: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.2 MiB/s wr, 464 op/s 2024-08-30T09:10:52.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:10:52 smithi190 ceph-mon[22001]: pgmap v676: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.2 MiB/s wr, 464 op/s 2024-08-30T09:10:52.972 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T09:10:53.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.633+0000 7f854ffff640 1 -- 172.21.15.42:0/627326625 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f855010fa90 msgr2=0x7f855010fea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:10:53.637 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.633+0000 7f854ffff640 1 --2- 172.21.15.42:0/627326625 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f855010fa90 0x7f855010fea0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f85400092e0 tx=0x7f854002b4e0 comp rx=0 tx=0).stop 2024-08-30T09:10:53.637 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.634+0000 7f854ffff640 1 -- 172.21.15.42:0/627326625 shutdown_connections 2024-08-30T09:10:53.637 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.634+0000 7f854ffff640 1 --2- 172.21.15.42:0/627326625 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f855010fa90 0x7f855010fea0 secure :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f85400092e0 tx=0x7f854002b4e0 comp rx=0 tx=0).stop 2024-08-30T09:10:53.637 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.634+0000 7f854ffff640 1 --2- 172.21.15.42:0/627326625 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f85501103e0 0x7f8550116c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:53.637 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.634+0000 7f854ffff640 1 --2- 172.21.15.42:0/627326625 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f855010a3f0 0x7f855010a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:53.637 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.634+0000 7f854ffff640 1 -- 172.21.15.42:0/627326625 >> 172.21.15.42:0/627326625 conn(0x7f855006ebf0 msgr2=0x7f855006f030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:10:53.637 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.636+0000 7f854ffff640 1 -- 172.21.15.42:0/627326625 shutdown_connections 2024-08-30T09:10:53.641 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.640+0000 7f854ffff640 1 -- 172.21.15.42:0/627326625 wait complete. 2024-08-30T09:10:53.647 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.645+0000 7f854ffff640 1 Processor -- start 2024-08-30T09:10:53.647 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.646+0000 7f854ffff640 1 -- start start 2024-08-30T09:10:53.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.646+0000 7f854ffff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f855010a3f0 0x7f85501a0800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:53.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.646+0000 7f854ffff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f85501103e0 0x7f85501a0da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:53.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.647+0000 7f854ffff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f85501a5f80 0x7f85501a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:53.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.647+0000 7f854ffff640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f8550119c70 con 0x7f855010a3f0 2024-08-30T09:10:53.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.647+0000 7f854ffff640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f8550119af0 con 0x7f85501a5f80 2024-08-30T09:10:53.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.647+0000 7f854ffff640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f8550119df0 con 0x7f85501103e0 2024-08-30T09:10:53.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.650+0000 7f854effd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f855010a3f0 0x7f85501a0800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:10:53.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.650+0000 7f854effd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f855010a3f0 0x7f85501a0800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:46234/0 (socket says 172.21.15.42:46234) 2024-08-30T09:10:53.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.650+0000 7f854effd640 1 -- 172.21.15.42:0/3874489764 learned_addr learned my addr 172.21.15.42:0/3874489764 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:10:53.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.651+0000 7f854effd640 1 -- 172.21.15.42:0/3874489764 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f85501103e0 msgr2=0x7f85501a0da0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-30T09:10:53.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.651+0000 7f854effd640 1 --2- 172.21.15.42:0/3874489764 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f85501103e0 0x7f85501a0da0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:53.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.651+0000 7f854effd640 1 -- 172.21.15.42:0/3874489764 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f85501a5f80 msgr2=0x7f85501a6390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:10:53.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.651+0000 7f854effd640 1 --2- 172.21.15.42:0/3874489764 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f85501a5f80 0x7f85501a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:53.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.651+0000 7f854effd640 1 -- 172.21.15.42:0/3874489764 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85501a6b60 con 0x7f855010a3f0 2024-08-30T09:10:53.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.652+0000 7f854effd640 1 --2- 172.21.15.42:0/3874489764 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f855010a3f0 0x7f85501a0800 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f853800e9c0 tx=0x7f853800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:10:53.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.652+0000 7f854cff9640 1 -- 172.21.15.42:0/3874489764 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f853800ccc0 con 0x7f855010a3f0 2024-08-30T09:10:53.655 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.653+0000 7f854ffff640 1 -- 172.21.15.42:0/3874489764 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85501ad100 con 0x7f855010a3f0 2024-08-30T09:10:53.655 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.653+0000 7f854ffff640 1 -- 172.21.15.42:0/3874489764 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85501ad4f0 con 0x7f855010a3f0 2024-08-30T09:10:53.655 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.653+0000 7f854cff9640 1 -- 172.21.15.42:0/3874489764 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f85380054d0 con 0x7f855010a3f0 2024-08-30T09:10:53.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.653+0000 7f854cff9640 1 -- 172.21.15.42:0/3874489764 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8538010430 con 0x7f855010a3f0 2024-08-30T09:10:53.660 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.658+0000 7f855b975740 1 Processor -- start 2024-08-30T09:10:53.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.661+0000 7f855b975740 1 -- start start 2024-08-30T09:10:53.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.661+0000 7f855b975740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a779a19070 0x55a779dc1020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:53.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.661+0000 7f855b975740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a7799f01b0 0x55a779dc1560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:53.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.661+0000 7f855b975740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a7799f8390 0x55a779dd4a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:53.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.661+0000 7f855b975740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55a779d35680 con 0x55a7799f8390 2024-08-30T09:10:53.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.661+0000 7f855b975740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55a779d24c10 con 0x55a7799f01b0 2024-08-30T09:10:53.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.661+0000 7f855b975740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55a779d28bc0 con 0x55a779a19070 2024-08-30T09:10:53.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.662+0000 7f8547fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a7799f8390 0x55a779dd4a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:10:53.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.662+0000 7f8547fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a7799f8390 0x55a779dd4a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:46244/0 (socket says 172.21.15.42:46244) 2024-08-30T09:10:53.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.662+0000 7f8547fff640 1 -- 172.21.15.42:0/2908916140 learned_addr learned my addr 172.21.15.42:0/2908916140 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:10:53.667 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.662+0000 7f854f7fe640 1 --2- 172.21.15.42:0/2908916140 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a779a19070 0x55a779dc1020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:10:53.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.666+0000 7f8547fff640 1 -- 172.21.15.42:0/2908916140 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a779a19070 msgr2=0x55a779dc1020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:10:53.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.670+0000 7f8547fff640 1 --2- 172.21.15.42:0/2908916140 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a779a19070 0x55a779dc1020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:53.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.670+0000 7f8547fff640 1 -- 172.21.15.42:0/2908916140 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a7799f01b0 msgr2=0x55a779dc1560 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-30T09:10:53.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.670+0000 7f8547fff640 1 --2- 172.21.15.42:0/2908916140 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a7799f01b0 0x55a779dc1560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:53.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.670+0000 7f8547fff640 1 -- 172.21.15.42:0/2908916140 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a779a84650 con 0x55a7799f8390 2024-08-30T09:10:53.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.678+0000 7f8547fff640 1 --2- 172.21.15.42:0/2908916140 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a7799f8390 0x55a779dd4a50 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f8540000c00 tx=0x7f8540005d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65f1cd863412ab8c server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:10:53.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.678+0000 7f85457fa640 1 -- 172.21.15.42:0/2908916140 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8540031620 con 0x55a7799f8390 2024-08-30T09:10:53.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.678+0000 7f85457fa640 1 -- 172.21.15.42:0/2908916140 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8540031980 con 0x55a7799f8390 2024-08-30T09:10:53.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.679+0000 7f854cff9640 1 -- 172.21.15.42:0/3874489764 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7f85380046c0 con 0x7f855010a3f0 2024-08-30T09:10:53.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.679+0000 7f855b975740 1 -- 172.21.15.42:0/2908916140 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a7799f8390 msgr2=0x55a779dd4a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:10:53.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.679+0000 7f855b975740 1 --2- 172.21.15.42:0/2908916140 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a7799f8390 0x55a779dd4a50 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f8540000c00 tx=0x7f8540005d00 comp rx=0 tx=0).stop 2024-08-30T09:10:53.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.680+0000 7f855b975740 1 -- 172.21.15.42:0/2908916140 shutdown_connections 2024-08-30T09:10:53.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.680+0000 7f855b975740 1 --2- 172.21.15.42:0/2908916140 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a7799f8390 0x55a779dd4a50 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:53.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.680+0000 7f855b975740 1 --2- 172.21.15.42:0/2908916140 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a7799f01b0 0x55a779dc1560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:53.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.680+0000 7f855b975740 1 --2- 172.21.15.42:0/2908916140 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a779a19070 0x55a779dc1020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:53.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.680+0000 7f855b975740 1 -- 172.21.15.42:0/2908916140 >> 172.21.15.42:0/2908916140 conn(0x55a779a0d460 msgr2=0x55a779dbbcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:10:53.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.680+0000 7f855b975740 1 -- 172.21.15.42:0/2908916140 shutdown_connections 2024-08-30T09:10:53.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.680+0000 7f855b975740 1 -- 172.21.15.42:0/2908916140 wait complete. 2024-08-30T09:10:53.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.684+0000 7f855b975740 1 -- start start 2024-08-30T09:10:53.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.684+0000 7f855b975740 1 Processor -- start 2024-08-30T09:10:53.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.685+0000 7f855b975740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a7799f01b0 0x55a779dca170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:53.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.685+0000 7f855b975740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a779a19070 0x55a779dd4900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:53.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.685+0000 7f855b975740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a7799d79f0 0x55a779dca7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:53.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.685+0000 7f855b975740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55a779b87700 con 0x55a7799d79f0 2024-08-30T09:10:53.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.685+0000 7f855b975740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55a779cab750 con 0x55a779a19070 2024-08-30T09:10:53.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.685+0000 7f855b975740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55a779dc25c0 con 0x55a7799f01b0 2024-08-30T09:10:53.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.685+0000 7f855b975740 10 client.0 _ll_register_callbacks cb 0x55a779cdae10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:10:53.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.685+0000 7f854cff9640 1 --2- 172.21.15.42:0/3874489764 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f853c0b4bc0 0x7f853c0b70d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:53.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.685+0000 7f8547fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a7799d79f0 0x55a779dca7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:10:53.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.685+0000 7f854effd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a779a19070 0x55a779dd4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:10:53.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.685+0000 7f854effd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a779a19070 0x55a779dd4900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:58628/0 (socket says 172.21.15.42:58628) 2024-08-30T09:10:53.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.685+0000 7f854effd640 1 -- 172.21.15.42:0/4013545052 learned_addr learned my addr 172.21.15.42:0/4013545052 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:10:53.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.685+0000 7f854cff9640 1 -- 172.21.15.42:0/3874489764 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f85380e55c0 con 0x7f855010a3f0 2024-08-30T09:10:53.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.686+0000 7f8547fff640 1 -- 172.21.15.42:0/4013545052 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a7799f01b0 msgr2=0x55a779dca170 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-30T09:10:53.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.686+0000 7f8547fff640 1 --2- 172.21.15.42:0/4013545052 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a7799f01b0 0x55a779dca170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:53.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.686+0000 7f8547fff640 1 -- 172.21.15.42:0/4013545052 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a779a19070 msgr2=0x55a779dd4900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:10:53.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.686+0000 7f8547fff640 1 --2- 172.21.15.42:0/4013545052 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a779a19070 0x55a779dd4900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:53.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.686+0000 7f8547fff640 1 -- 172.21.15.42:0/4013545052 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a779ce4900 con 0x55a7799d79f0 2024-08-30T09:10:53.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.686+0000 7f8547fff640 1 --2- 172.21.15.42:0/4013545052 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a7799d79f0 0x55a779dca7b0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f8540003650 tx=0x7f8540031650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3742a27b12a215d server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:10:53.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.686+0000 7f85367fc640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-08-30T09:10:53.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.688+0000 7f854f7fe640 1 --2- 172.21.15.42:0/3874489764 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f853c0b4bc0 0x7f853c0b70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:10:53.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.693+0000 7f85367fc640 1 -- 172.21.15.42:0/4013545052 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8540049070 con 0x55a7799d79f0 2024-08-30T09:10:53.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.693+0000 7f854f7fe640 1 --2- 172.21.15.42:0/3874489764 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f853c0b4bc0 0x7f853c0b70d0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f8548009270 tx=0x7f8548009ac0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:10:53.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.693+0000 7f855b975740 1 -- 172.21.15.42:0/4013545052 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55a779d375d0 con 0x55a7799d79f0 2024-08-30T09:10:53.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.693+0000 7f85367fc640 1 -- 172.21.15.42:0/4013545052 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f8540004430 con 0x55a7799d79f0 2024-08-30T09:10:53.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.699+0000 7f85367fc640 1 -- 172.21.15.42:0/4013545052 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f8540009ce0 con 0x55a7799d79f0 2024-08-30T09:10:53.703 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.701+0000 7f855b975740 10 client.17115 fetch_fsmap learned FSMap version 40 2024-08-30T09:10:53.703 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.701+0000 7f855b975740 1 -- 172.21.15.42:0/4013545052 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55a779dd87e0 con 0x55a7799d79f0 2024-08-30T09:10:53.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.705+0000 7f85367fc640 1 -- 172.21.15.42:0/4013545052 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55a779d375d0 con 0x55a7799d79f0 2024-08-30T09:10:53.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.705+0000 7f85367fc640 1 -- 172.21.15.42:0/4013545052 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 40) ==== 8366+0+0 (secure 0 0 0) 0x7f85400447c0 con 0x55a7799d79f0 2024-08-30T09:10:53.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.705+0000 7f855b975740 10 client.17115 fetch_fsmap finished waiting for FSMap version 40 2024-08-30T09:10:53.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.705+0000 7f855b975740 10 client.17115 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-08-30T09:10:53.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.705+0000 7f855b975740 20 client.17115 populate_metadata read hostname 'smithi042' 2024-08-30T09:10:53.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.705+0000 7f855b975740 1 --2- 172.21.15.42:0/4013545052 >> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] conn(0x55a7799acda0 0x55a779ddbd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:53.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.705+0000 7f855b975740 4 client.17115 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-30T09:10:53.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.705+0000 7f855b975740 1 -- 172.21.15.42:0/4013545052 --> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a779d33070 con 0x55a7799acda0 2024-08-30T09:10:53.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.706+0000 7f854effd640 1 --2- 172.21.15.42:0/4013545052 >> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] conn(0x55a7799acda0 0x55a779ddbd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:10:53.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.706+0000 7f854effd640 1 --2- 172.21.15.42:0/4013545052 >> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] conn(0x55a7799acda0 0x55a779ddbd50 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=609ad021a542f985 server_cookie=ff95950e6c8759fe in_seq=0 out_seq=0 2024-08-30T09:10:53.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.706+0000 7f85367fc640 10 client.17115 ms_handle_connect on v2:172.21.15.190:6838/719981110 2024-08-30T09:10:53.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.709+0000 7f85367fc640 1 -- 172.21.15.42:0/4013545052 <== mds.1 v2:172.21.15.190:6838/719981110 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a779d33070 con 0x55a7799acda0 2024-08-30T09:10:53.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.709+0000 7f85367fc640 10 client.17115 handle_command_reply: tid=0 2024-08-30T09:10:53.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.709+0000 7f855b975740 1 client.17115 shutdown 2024-08-30T09:10:53.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.710+0000 7f855b975740 1 -- 172.21.15.42:0/4013545052 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a7799d79f0 msgr2=0x55a779dca7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:10:53.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.710+0000 7f855b975740 1 --2- 172.21.15.42:0/4013545052 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a7799d79f0 0x55a779dca7b0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f8540003650 tx=0x7f8540031650 comp rx=0 tx=0).stop 2024-08-30T09:10:53.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.710+0000 7f855b975740 1 -- 172.21.15.42:0/4013545052 shutdown_connections 2024-08-30T09:10:53.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.710+0000 7f855b975740 1 --2- 172.21.15.42:0/4013545052 >> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] conn(0x55a7799acda0 0x55a779ddbd50 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:53.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.710+0000 7f855b975740 1 --2- 172.21.15.42:0/4013545052 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a7799d79f0 0x55a779dca7b0 secure :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f8540003650 tx=0x7f8540031650 comp rx=0 tx=0).stop 2024-08-30T09:10:53.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.710+0000 7f855b975740 1 --2- 172.21.15.42:0/4013545052 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a779a19070 0x55a779dd4900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:53.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.710+0000 7f855b975740 1 --2- 172.21.15.42:0/4013545052 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a7799f01b0 0x55a779dca170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:53.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.710+0000 7f855b975740 1 -- 172.21.15.42:0/4013545052 >> 172.21.15.42:0/4013545052 conn(0x55a779a0d460 msgr2=0x55a779dbbca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:10:53.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.710+0000 7f85367fc640 0 client.17115 ms_handle_reset on v2:172.21.15.190:6838/719981110 2024-08-30T09:10:53.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.716+0000 7f855b975740 1 -- 172.21.15.42:0/4013545052 shutdown_connections 2024-08-30T09:10:53.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.717+0000 7f855b975740 1 -- 172.21.15.42:0/4013545052 wait complete. 2024-08-30T09:10:53.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.718+0000 7f855b975740 20 client.17115 trim_cache size 0 max 16384 2024-08-30T09:10:53.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.744+0000 7f855b975740 1 Processor -- start 2024-08-30T09:10:53.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.744+0000 7f855b975740 1 -- start start 2024-08-30T09:10:53.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.745+0000 7f855b975740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a7799f01b0 0x55a779dbb860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:53.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.745+0000 7f855b975740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a779a19070 0x55a779dbbda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:53.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.745+0000 7f855b975740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a779dc50b0 0x55a779dc54c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:53.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.745+0000 7f855b975740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f8530012610 con 0x55a779a19070 2024-08-30T09:10:53.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.745+0000 7f855b975740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f853000fe80 con 0x55a7799f01b0 2024-08-30T09:10:53.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.745+0000 7f855b975740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f853000dcd0 con 0x55a779dc50b0 2024-08-30T09:10:53.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.745+0000 7f854effd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a7799f01b0 0x55a779dbb860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:10:53.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.745+0000 7f854effd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a7799f01b0 0x55a779dbb860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:58634/0 (socket says 172.21.15.42:58634) 2024-08-30T09:10:53.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.745+0000 7f854effd640 1 -- 172.21.15.42:0/3922725833 learned_addr learned my addr 172.21.15.42:0/3922725833 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:10:53.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.745+0000 7f854f7fe640 1 --2- 172.21.15.42:0/3922725833 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a779dc50b0 0x55a779dc54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:10:53.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.746+0000 7f854effd640 1 -- 172.21.15.42:0/3922725833 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a779dc50b0 msgr2=0x55a779dc54c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:10:53.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.746+0000 7f854effd640 1 --2- 172.21.15.42:0/3922725833 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a779dc50b0 0x55a779dc54c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:53.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.746+0000 7f854effd640 1 -- 172.21.15.42:0/3922725833 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a779a19070 msgr2=0x55a779dbbda0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-30T09:10:53.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.746+0000 7f854effd640 1 --2- 172.21.15.42:0/3922725833 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a779a19070 0x55a779dbbda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:53.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.746+0000 7f854effd640 1 -- 172.21.15.42:0/3922725833 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a779dc5a00 con 0x55a7799f01b0 2024-08-30T09:10:53.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.749+0000 7f854effd640 1 --2- 172.21.15.42:0/3922725833 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a7799f01b0 0x55a779dbb860 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f85380049d0 tx=0x7f85380a8250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=74304e04cde4dbdf server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:10:53.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.750+0000 7f8544ff9640 1 -- 172.21.15.42:0/3922725833 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85380ab710 con 0x55a7799f01b0 2024-08-30T09:10:53.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.750+0000 7f8544ff9640 1 -- 172.21.15.42:0/3922725833 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f85380ac300 con 0x55a7799f01b0 2024-08-30T09:10:53.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.751+0000 7f855b975740 1 -- 172.21.15.42:0/3922725833 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a7799f01b0 msgr2=0x55a779dbb860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:10:53.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.751+0000 7f855b975740 1 --2- 172.21.15.42:0/3922725833 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a7799f01b0 0x55a779dbb860 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f85380049d0 tx=0x7f85380a8250 comp rx=0 tx=0).stop 2024-08-30T09:10:53.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.751+0000 7f855b975740 1 -- 172.21.15.42:0/3922725833 shutdown_connections 2024-08-30T09:10:53.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.751+0000 7f855b975740 1 --2- 172.21.15.42:0/3922725833 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a779dc50b0 0x55a779dc54c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:53.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.751+0000 7f855b975740 1 --2- 172.21.15.42:0/3922725833 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a779a19070 0x55a779dbbda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:53.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.751+0000 7f855b975740 1 --2- 172.21.15.42:0/3922725833 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a7799f01b0 0x55a779dbb860 secure :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f85380049d0 tx=0x7f85380a8250 comp rx=0 tx=0).stop 2024-08-30T09:10:53.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.751+0000 7f855b975740 1 -- 172.21.15.42:0/3922725833 >> 172.21.15.42:0/3922725833 conn(0x55a779a0d460 msgr2=0x55a779dc0fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:10:53.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.756+0000 7f855b975740 1 -- 172.21.15.42:0/3922725833 shutdown_connections 2024-08-30T09:10:53.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.759+0000 7f855b975740 1 -- 172.21.15.42:0/3922725833 wait complete. 2024-08-30T09:10:53.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.759+0000 7f855b975740 1 -- start start 2024-08-30T09:10:53.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.759+0000 7f855b975740 1 Processor -- start 2024-08-30T09:10:53.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.760+0000 7f855b975740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a779a19070 0x55a779de1890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:53.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.760+0000 7f855b975740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a779dc50b0 0x55a779de1e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:53.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.760+0000 7f855b975740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a779ddc3a0 0x55a779ddc830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:53.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.760+0000 7f855b975740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f853000e340 con 0x55a779ddc3a0 2024-08-30T09:10:53.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.760+0000 7f855b975740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f853000f7b0 con 0x55a779dc50b0 2024-08-30T09:10:53.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.760+0000 7f855b975740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f853000ea10 con 0x55a779a19070 2024-08-30T09:10:53.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.760+0000 7f855b975740 10 client.0 _ll_register_callbacks cb 0x7f85480093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:10:53.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.762+0000 7f8547fff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a779dc50b0 0x55a779de1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:10:53.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.762+0000 7f8547fff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a779dc50b0 0x55a779de1e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:58642/0 (socket says 172.21.15.42:58642) 2024-08-30T09:10:53.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.762+0000 7f8547fff640 1 -- 172.21.15.42:0/1272738992 learned_addr learned my addr 172.21.15.42:0/1272738992 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:10:53.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.765+0000 7f8547fff640 1 -- 172.21.15.42:0/1272738992 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a779a19070 msgr2=0x55a779de1890 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-30T09:10:53.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.765+0000 7f8547fff640 1 --2- 172.21.15.42:0/1272738992 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a779a19070 0x55a779de1890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:53.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.765+0000 7f8547fff640 1 -- 172.21.15.42:0/1272738992 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a779ddc3a0 msgr2=0x55a779ddc830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:10:53.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.765+0000 7f8547fff640 1 --2- 172.21.15.42:0/1272738992 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a779ddc3a0 0x55a779ddc830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:53.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.765+0000 7f8547fff640 1 -- 172.21.15.42:0/1272738992 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a779ddcfc0 con 0x55a779dc50b0 2024-08-30T09:10:53.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.771+0000 7f8547fff640 1 --2- 172.21.15.42:0/1272738992 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a779dc50b0 0x55a779de1e30 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f854002ba00 tx=0x7f8540004440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3986dc97a3de4a3f server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:10:53.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.771+0000 7f85367fc640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-30T09:10:53.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.772+0000 7f85367fc640 1 -- 172.21.15.42:0/1272738992 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8540049040 con 0x55a779dc50b0 2024-08-30T09:10:53.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.772+0000 7f855b975740 1 -- 172.21.15.42:0/1272738992 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f85380ac300 con 0x55a779dc50b0 2024-08-30T09:10:53.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.772+0000 7f85367fc640 1 -- 172.21.15.42:0/1272738992 <== mon.2 v2:172.21.15.167:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f8540044860 con 0x55a779dc50b0 2024-08-30T09:10:53.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.776+0000 7f85367fc640 1 -- 172.21.15.42:0/1272738992 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f8540005850 con 0x55a779dc50b0 2024-08-30T09:10:53.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.780+0000 7f855b975740 10 client.26924 fetch_fsmap learned FSMap version 40 2024-08-30T09:10:53.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.780+0000 7f855b975740 1 -- 172.21.15.42:0/1272738992 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55a779e691b0 con 0x55a779dc50b0 2024-08-30T09:10:53.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.781+0000 7f85367fc640 1 -- 172.21.15.42:0/1272738992 <== mon.2 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f85380ac300 con 0x55a779dc50b0 2024-08-30T09:10:53.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.781+0000 7f85367fc640 1 -- 172.21.15.42:0/1272738992 <== mon.2 v2:172.21.15.167:3300/0 5 ==== fsmap(e 40) ==== 8366+0+0 (secure 0 0 0) 0x7f8540005040 con 0x55a779dc50b0 2024-08-30T09:10:53.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.782+0000 7f855b975740 10 client.26924 fetch_fsmap finished waiting for FSMap version 40 2024-08-30T09:10:53.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.782+0000 7f855b975740 10 client.26924 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-08-30T09:10:53.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.782+0000 7f855b975740 20 client.26924 populate_metadata read hostname 'smithi042' 2024-08-30T09:10:53.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.782+0000 7f855b975740 1 --2- 172.21.15.42:0/1272738992 >> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] conn(0x55a779e69810 0x55a779e6bc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:53.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.782+0000 7f855b975740 4 client.26924 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-30T09:10:53.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.782+0000 7f855b975740 1 -- 172.21.15.42:0/1272738992 --> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a779d307b0 con 0x55a779e69810 2024-08-30T09:10:53.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.784+0000 7f854effd640 1 --2- 172.21.15.42:0/1272738992 >> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] conn(0x55a779e69810 0x55a779e6bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:10:53.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.793+0000 7f85367fc640 10 client.26924 ms_handle_connect on v2:172.21.15.190:6838/719981110 2024-08-30T09:10:53.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.793+0000 7f854effd640 1 --2- 172.21.15.42:0/1272738992 >> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] conn(0x55a779e69810 0x55a779e6bc30 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6086a2d44c3f3889 server_cookie=5885bd078664dc11 in_seq=0 out_seq=0 2024-08-30T09:10:53.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.793+0000 7f85367fc640 1 -- 172.21.15.42:0/1272738992 <== mds.1 v2:172.21.15.190:6838/719981110 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a779d307b0 con 0x55a779e69810 2024-08-30T09:10:53.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.793+0000 7f85367fc640 10 client.26924 handle_command_reply: tid=0 2024-08-30T09:10:53.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.793+0000 7f855b975740 1 client.26924 shutdown 2024-08-30T09:10:53.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.797+0000 7f855b975740 1 -- 172.21.15.42:0/1272738992 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a779dc50b0 msgr2=0x55a779de1e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:10:53.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.797+0000 7f855b975740 1 --2- 172.21.15.42:0/1272738992 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a779dc50b0 0x55a779de1e30 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f854002ba00 tx=0x7f8540004440 comp rx=0 tx=0).stop 2024-08-30T09:10:53.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.797+0000 7f855b975740 1 -- 172.21.15.42:0/1272738992 shutdown_connections 2024-08-30T09:10:53.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.797+0000 7f855b975740 1 --2- 172.21.15.42:0/1272738992 >> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] conn(0x55a779e69810 0x55a779e6bc30 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:53.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.797+0000 7f855b975740 1 --2- 172.21.15.42:0/1272738992 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a779ddc3a0 0x55a779ddc830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:53.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.797+0000 7f855b975740 1 --2- 172.21.15.42:0/1272738992 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a779dc50b0 0x55a779de1e30 secure :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f854002ba00 tx=0x7f8540004440 comp rx=0 tx=0).stop 2024-08-30T09:10:53.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.797+0000 7f855b975740 1 --2- 172.21.15.42:0/1272738992 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a779a19070 0x55a779de1890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:53.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.797+0000 7f855b975740 1 -- 172.21.15.42:0/1272738992 >> 172.21.15.42:0/1272738992 conn(0x55a779a0d460 msgr2=0x55a779dcd580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:10:53.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.797+0000 7f85367fc640 0 client.26924 ms_handle_reset on v2:172.21.15.190:6838/719981110 2024-08-30T09:10:53.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.797+0000 7f855b975740 1 -- 172.21.15.42:0/1272738992 shutdown_connections 2024-08-30T09:10:53.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.797+0000 7f855b975740 1 -- 172.21.15.42:0/1272738992 wait complete. 2024-08-30T09:10:53.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.797+0000 7f855b975740 20 client.26924 trim_cache size 0 max 16384 2024-08-30T09:10:53.800 INFO:teuthology.orchestra.run.smithi042.stdout:[] 2024-08-30T09:10:53.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.798+0000 7f854ffff640 1 -- 172.21.15.42:0/3874489764 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f853c0b4bc0 msgr2=0x7f853c0b70d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:10:53.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.798+0000 7f854ffff640 1 --2- 172.21.15.42:0/3874489764 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f853c0b4bc0 0x7f853c0b70d0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f8548009270 tx=0x7f8548009ac0 comp rx=0 tx=0).stop 2024-08-30T09:10:53.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.798+0000 7f854ffff640 1 -- 172.21.15.42:0/3874489764 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f855010a3f0 msgr2=0x7f85501a0800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:10:53.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.798+0000 7f854ffff640 1 --2- 172.21.15.42:0/3874489764 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f855010a3f0 0x7f85501a0800 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f853800e9c0 tx=0x7f853800ee80 comp rx=0 tx=0).stop 2024-08-30T09:10:53.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.798+0000 7f854ffff640 1 -- 172.21.15.42:0/3874489764 shutdown_connections 2024-08-30T09:10:53.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.798+0000 7f854ffff640 1 --2- 172.21.15.42:0/3874489764 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f853c0b4bc0 0x7f853c0b70d0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:53.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.798+0000 7f854ffff640 1 --2- 172.21.15.42:0/3874489764 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f85501103e0 0x7f85501a0da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:53.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.798+0000 7f854ffff640 1 --2- 172.21.15.42:0/3874489764 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f85501a5f80 0x7f85501a6390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:53.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.798+0000 7f854ffff640 1 --2- 172.21.15.42:0/3874489764 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f855010a3f0 0x7f85501a0800 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:53.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.798+0000 7f854ffff640 1 -- 172.21.15.42:0/3874489764 >> 172.21.15.42:0/3874489764 conn(0x7f855006ebf0 msgr2=0x7f855010dbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:10:53.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.799+0000 7f854ffff640 1 -- 172.21.15.42:0/3874489764 shutdown_connections 2024-08-30T09:10:53.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:53.799+0000 7f854ffff640 1 -- 172.21.15.42:0/3874489764 wait complete. 2024-08-30T09:10:54.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:10:54 smithi042 ceph-mon[18652]: pgmap v677: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 8.2 MiB/s wr, 594 op/s 2024-08-30T09:10:54.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:10:54 smithi042 ceph-mon[18652]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-30T09:10:54.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:10:54 smithi190 ceph-mon[22001]: pgmap v677: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 8.2 MiB/s wr, 594 op/s 2024-08-30T09:10:54.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:10:54 smithi190 ceph-mon[22001]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-30T09:10:54.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:10:54 smithi167 ceph-mon[22063]: pgmap v677: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 8.2 MiB/s wr, 594 op/s 2024-08-30T09:10:54.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:10:54 smithi167 ceph-mon[22063]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-30T09:10:55.953 INFO:teuthology.orchestra.run.smithi042.stderr:progress: 610.0 s, 483.0 tps, lat 204.561 ms stddev 267.088 2024-08-30T09:10:56.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:10:56 smithi042 ceph-mon[18652]: pgmap v678: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 7.6 MiB/s wr, 497 op/s 2024-08-30T09:10:56.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:10:56 smithi190 ceph-mon[22001]: pgmap v678: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 7.6 MiB/s wr, 497 op/s 2024-08-30T09:10:56.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:10:56 smithi167 ceph-mon[22063]: pgmap v678: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 7.6 MiB/s wr, 497 op/s 2024-08-30T09:10:58.078 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-08-30T09:10:58.511 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T09:10:58.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:10:58 smithi042 ceph-mon[18652]: pgmap v679: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 7.3 MiB/s wr, 502 op/s 2024-08-30T09:10:58.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:10:58 smithi190 ceph-mon[22001]: pgmap v679: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 7.3 MiB/s wr, 502 op/s 2024-08-30T09:10:58.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:10:58 smithi167 ceph-mon[22063]: pgmap v679: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 7.3 MiB/s wr, 502 op/s 2024-08-30T09:10:59.485 INFO:teuthology.orchestra.run.smithi042.stdout:[] 2024-08-30T09:10:59.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.070+0000 7f431ee5e640 1 -- 172.21.15.42:0/3922159938 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4318109bc0 msgr2=0x7f43181103f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:10:59.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.070+0000 7f431ee5e640 1 --2- 172.21.15.42:0/3922159938 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4318109bc0 0x7f43181103f0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f4310009ca0 tx=0x7f431002a8c0 comp rx=0 tx=0).stop 2024-08-30T09:10:59.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.070+0000 7f431ee5e640 1 -- 172.21.15.42:0/3922159938 shutdown_connections 2024-08-30T09:10:59.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.070+0000 7f431ee5e640 1 --2- 172.21.15.42:0/3922159938 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4318109270 0x7f4318109680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:59.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.070+0000 7f431ee5e640 1 --2- 172.21.15.42:0/3922159938 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4318109bc0 0x7f43181103f0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:59.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.070+0000 7f431ee5e640 1 --2- 172.21.15.42:0/3922159938 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4318103930 0x7f4318103d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:59.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.070+0000 7f431ee5e640 1 -- 172.21.15.42:0/3922159938 >> 172.21.15.42:0/3922159938 conn(0x7f43180ff3a0 msgr2=0x7f43181017f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:10:59.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.070+0000 7f431ee5e640 1 -- 172.21.15.42:0/3922159938 shutdown_connections 2024-08-30T09:10:59.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.070+0000 7f431ee5e640 1 -- 172.21.15.42:0/3922159938 wait complete. 2024-08-30T09:10:59.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.071+0000 7f431ee5e640 1 Processor -- start 2024-08-30T09:10:59.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.077+0000 7f431ee5e640 1 -- start start 2024-08-30T09:10:59.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.077+0000 7f431ee5e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4318103930 0x7f431819a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:59.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.077+0000 7f431ee5e640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4318109270 0x7f431819a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:59.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.077+0000 7f431ee5e640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4318109bc0 0x7f431819fa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:59.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.077+0000 7f431ee5e640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f4318113880 con 0x7f4318109bc0 2024-08-30T09:10:59.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.077+0000 7f431ee5e640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4318113700 con 0x7f4318109270 2024-08-30T09:10:59.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.077+0000 7f431ee5e640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f4318113a00 con 0x7f4318103930 2024-08-30T09:10:59.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.079+0000 7f431cbfa640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4318103930 0x7f431819a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:10:59.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.079+0000 7f431cbfa640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4318103930 0x7f431819a1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:47830/0 (socket says 172.21.15.42:47830) 2024-08-30T09:10:59.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.079+0000 7f431cbfa640 1 -- 172.21.15.42:0/829198374 learned_addr learned my addr 172.21.15.42:0/829198374 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:10:59.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.079+0000 7f4317fff640 1 --2- 172.21.15.42:0/829198374 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4318109270 0x7f431819a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:10:59.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.079+0000 7f431cbfa640 1 -- 172.21.15.42:0/829198374 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4318109270 msgr2=0x7f431819a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:10:59.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.079+0000 7f431cbfa640 1 --2- 172.21.15.42:0/829198374 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4318109270 0x7f431819a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:59.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.079+0000 7f431cbfa640 1 -- 172.21.15.42:0/829198374 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4318109bc0 msgr2=0x7f431819fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:10:59.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.079+0000 7f431cbfa640 1 --2- 172.21.15.42:0/829198374 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4318109bc0 0x7f431819fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:59.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.080+0000 7f431cbfa640 1 -- 172.21.15.42:0/829198374 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43181a0110 con 0x7f4318103930 2024-08-30T09:10:59.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.080+0000 7f431cbfa640 1 --2- 172.21.15.42:0/829198374 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4318103930 0x7f431819a1d0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f430c00ea20 tx=0x7f430c00eee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:10:59.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.083+0000 7f4315ffb640 1 -- 172.21.15.42:0/829198374 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f430c00cd80 con 0x7f4318103930 2024-08-30T09:10:59.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.083+0000 7f4315ffb640 1 -- 172.21.15.42:0/829198374 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f430c0108e0 con 0x7f4318103930 2024-08-30T09:10:59.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.083+0000 7f4315ffb640 1 -- 172.21.15.42:0/829198374 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f430c0188b0 con 0x7f4318103930 2024-08-30T09:10:59.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.085+0000 7f431ee5e640 1 -- 172.21.15.42:0/829198374 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43181a0400 con 0x7f4318103930 2024-08-30T09:10:59.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.085+0000 7f431ee5e640 1 -- 172.21.15.42:0/829198374 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f431806cb80 con 0x7f4318103930 2024-08-30T09:10:59.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.089+0000 7f43253d3740 1 Processor -- start 2024-08-30T09:10:59.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.104+0000 7f43253d3740 1 -- start start 2024-08-30T09:10:59.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.104+0000 7f4315ffb640 1 -- 172.21.15.42:0/829198374 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7f430c010430 con 0x7f4318103930 2024-08-30T09:10:59.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.109+0000 7f4315ffb640 1 --2- 172.21.15.42:0/829198374 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f43080b4d00 0x7f43080b7210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:59.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.109+0000 7f4315ffb640 1 -- 172.21.15.42:0/829198374 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f430c0e5b20 con 0x7f4318103930 2024-08-30T09:10:59.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.110+0000 7f43253d3740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e2aeb13070 0x55e2aeeb5d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:59.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.110+0000 7f43253d3740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e2aeaea1b0 0x55e2aeeb62f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:59.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.110+0000 7f43253d3740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e2aeaf2390 0x55e2aeec86c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:59.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.110+0000 7f43253d3740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55e2aee2f680 con 0x55e2aeb13070 2024-08-30T09:10:59.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.110+0000 7f43253d3740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55e2aee1ec10 con 0x55e2aeaf2390 2024-08-30T09:10:59.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.110+0000 7f43253d3740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55e2aee22bc0 con 0x55e2aeaea1b0 2024-08-30T09:10:59.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.112+0000 7f431d3fb640 1 --2- 172.21.15.42:0/829198374 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f43080b4d00 0x7f43080b7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:10:59.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.112+0000 7f431cbfa640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e2aeb13070 0x55e2aeeb5d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:10:59.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.112+0000 7f431cbfa640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e2aeb13070 0x55e2aeeb5d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42596/0 (socket says 172.21.15.42:42596) 2024-08-30T09:10:59.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.112+0000 7f431cbfa640 1 -- 172.21.15.42:0/166802611 learned_addr learned my addr 172.21.15.42:0/166802611 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:10:59.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.113+0000 7f4317fff640 1 --2- 172.21.15.42:0/166802611 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e2aeaea1b0 0x55e2aeeb62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:10:59.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.114+0000 7f431d3fb640 1 --2- 172.21.15.42:0/166802611 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e2aeaf2390 0x55e2aeec86c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:10:59.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.116+0000 7f431cbfa640 1 -- 172.21.15.42:0/166802611 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e2aeaea1b0 msgr2=0x55e2aeeb62f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:10:59.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.116+0000 7f431d3fb640 1 --2- 172.21.15.42:0/829198374 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f43080b4d00 0x7f43080b7210 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f431002ade0 tx=0x7f43100094a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:10:59.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.116+0000 7f431cbfa640 1 --2- 172.21.15.42:0/166802611 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e2aeaea1b0 0x55e2aeeb62f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:59.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.116+0000 7f431cbfa640 1 -- 172.21.15.42:0/166802611 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e2aeaf2390 msgr2=0x55e2aeec86c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:10:59.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.116+0000 7f431cbfa640 1 --2- 172.21.15.42:0/166802611 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e2aeaf2390 0x55e2aeec86c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:59.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.116+0000 7f431cbfa640 1 -- 172.21.15.42:0/166802611 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e2aeb7e650 con 0x55e2aeb13070 2024-08-30T09:10:59.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.121+0000 7f431cbfa640 1 --2- 172.21.15.42:0/166802611 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e2aeb13070 0x55e2aeeb5d70 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f430c0ac0f0 tx=0x7f430c010860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10e1f8b783981e83 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:10:59.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.128+0000 7f4302ffd640 1 -- 172.21.15.42:0/166802611 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f430c0ac160 con 0x55e2aeb13070 2024-08-30T09:10:59.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.128+0000 7f4302ffd640 1 -- 172.21.15.42:0/166802611 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f430c0b09d0 con 0x55e2aeb13070 2024-08-30T09:10:59.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.128+0000 7f4302ffd640 1 -- 172.21.15.42:0/166802611 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f430c0b5250 con 0x55e2aeb13070 2024-08-30T09:10:59.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.130+0000 7f43253d3740 1 -- 172.21.15.42:0/166802611 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e2aeb13070 msgr2=0x55e2aeeb5d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:10:59.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.130+0000 7f43253d3740 1 --2- 172.21.15.42:0/166802611 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e2aeb13070 0x55e2aeeb5d70 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f430c0ac0f0 tx=0x7f430c010860 comp rx=0 tx=0).stop 2024-08-30T09:10:59.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.130+0000 7f43253d3740 1 -- 172.21.15.42:0/166802611 shutdown_connections 2024-08-30T09:10:59.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.130+0000 7f43253d3740 1 --2- 172.21.15.42:0/166802611 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e2aeaea1b0 0x55e2aeeb62f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:59.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.130+0000 7f43253d3740 1 --2- 172.21.15.42:0/166802611 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e2aeaf2390 0x55e2aeec86c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:59.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.130+0000 7f43253d3740 1 --2- 172.21.15.42:0/166802611 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e2aeb13070 0x55e2aeeb5d70 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:59.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.130+0000 7f43253d3740 1 -- 172.21.15.42:0/166802611 >> 172.21.15.42:0/166802611 conn(0x55e2aeb07460 msgr2=0x55e2aeeb8290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:10:59.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.134+0000 7f43253d3740 1 -- 172.21.15.42:0/166802611 shutdown_connections 2024-08-30T09:10:59.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.137+0000 7f43253d3740 1 -- 172.21.15.42:0/166802611 wait complete. 2024-08-30T09:10:59.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.137+0000 7f43253d3740 1 -- start start 2024-08-30T09:10:59.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.137+0000 7f43253d3740 1 Processor -- start 2024-08-30T09:10:59.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.138+0000 7f43253d3740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e2aeaea1b0 0x55e2aeeccae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:59.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.138+0000 7f43253d3740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e2aeaf2390 0x55e2aeecd0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:59.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.139+0000 7f43253d3740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e2aeb13070 0x55e2aeed20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:59.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.139+0000 7f43253d3740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55e2aec81700 con 0x55e2aeaf2390 2024-08-30T09:10:59.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.139+0000 7f43253d3740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55e2aeda5750 con 0x55e2aeb13070 2024-08-30T09:10:59.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.139+0000 7f43253d3740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55e2aeec9160 con 0x55e2aeaea1b0 2024-08-30T09:10:59.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.139+0000 7f43253d3740 10 client.0 _ll_register_callbacks cb 0x55e2aedd4e10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:10:59.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.139+0000 7f431d3fb640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e2aeb13070 0x55e2aeed20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:10:59.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.139+0000 7f431d3fb640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e2aeb13070 0x55e2aeed20e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:36362/0 (socket says 172.21.15.42:36362) 2024-08-30T09:10:59.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.139+0000 7f431d3fb640 1 -- 172.21.15.42:0/774837507 learned_addr learned my addr 172.21.15.42:0/774837507 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:10:59.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.141+0000 7f4317fff640 1 --2- 172.21.15.42:0/774837507 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e2aeaf2390 0x55e2aeecd0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:10:59.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.141+0000 7f431cbfa640 1 --2- 172.21.15.42:0/774837507 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e2aeaea1b0 0x55e2aeeccae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:10:59.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.143+0000 7f431d3fb640 1 -- 172.21.15.42:0/774837507 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e2aeaea1b0 msgr2=0x55e2aeeccae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:10:59.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.143+0000 7f431d3fb640 1 --2- 172.21.15.42:0/774837507 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e2aeaea1b0 0x55e2aeeccae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:59.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.143+0000 7f431d3fb640 1 -- 172.21.15.42:0/774837507 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e2aeaf2390 msgr2=0x55e2aeecd0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:10:59.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.143+0000 7f431d3fb640 1 --2- 172.21.15.42:0/774837507 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e2aeaf2390 0x55e2aeecd0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:59.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.143+0000 7f431d3fb640 1 -- 172.21.15.42:0/774837507 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e2aedde900 con 0x55e2aeb13070 2024-08-30T09:10:59.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.152+0000 7f431d3fb640 1 --2- 172.21.15.42:0/774837507 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e2aeb13070 0x55e2aeed20e0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f4310031680 tx=0x7f4310005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ce7df822a666c5c5 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:10:59.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.152+0000 7f42f7fff640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-30T09:10:59.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.154+0000 7f42f7fff640 1 -- 172.21.15.42:0/774837507 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4310004450 con 0x55e2aeb13070 2024-08-30T09:10:59.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.155+0000 7f42f7fff640 1 -- 172.21.15.42:0/774837507 <== mon.2 v2:172.21.15.167:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f43100045f0 con 0x55e2aeb13070 2024-08-30T09:10:59.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.155+0000 7f43253d3740 1 -- 172.21.15.42:0/774837507 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f430c0b09d0 con 0x55e2aeb13070 2024-08-30T09:10:59.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.155+0000 7f42f7fff640 1 -- 172.21.15.42:0/774837507 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f431005a050 con 0x55e2aeb13070 2024-08-30T09:10:59.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.158+0000 7f43253d3740 10 client.26933 fetch_fsmap learned FSMap version 41 2024-08-30T09:10:59.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.158+0000 7f43253d3740 1 -- 172.21.15.42:0/774837507 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55e2aeedb030 con 0x55e2aeb13070 2024-08-30T09:10:59.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.159+0000 7f42f7fff640 1 -- 172.21.15.42:0/774837507 <== mon.2 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4310051070 con 0x55e2aeb13070 2024-08-30T09:10:59.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.159+0000 7f42f7fff640 1 -- 172.21.15.42:0/774837507 <== mon.2 v2:172.21.15.167:3300/0 5 ==== fsmap(e 41) ==== 8362+0+0 (secure 0 0 0) 0x7f4310030760 con 0x55e2aeb13070 2024-08-30T09:10:59.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.159+0000 7f43253d3740 10 client.26933 fetch_fsmap finished waiting for FSMap version 41 2024-08-30T09:10:59.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.159+0000 7f43253d3740 10 client.26933 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-30T09:10:59.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.159+0000 7f43253d3740 20 client.26933 populate_metadata read hostname 'smithi042' 2024-08-30T09:10:59.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.159+0000 7f43253d3740 1 --2- 172.21.15.42:0/774837507 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55e2aead19f0 0x55e2aeede4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:59.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.159+0000 7f43253d3740 4 client.26933 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-30T09:10:59.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.159+0000 7f43253d3740 1 -- 172.21.15.42:0/774837507 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f430c0b09d0 con 0x55e2aead19f0 2024-08-30T09:10:59.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.160+0000 7f431cbfa640 1 --2- 172.21.15.42:0/774837507 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55e2aead19f0 0x55e2aeede4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:10:59.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.163+0000 7f431cbfa640 1 --2- 172.21.15.42:0/774837507 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55e2aead19f0 0x55e2aeede4d0 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3af1285e715ec693 server_cookie=bedd5a6ef145c20e in_seq=0 out_seq=0 2024-08-30T09:10:59.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.163+0000 7f42f7fff640 10 client.26933 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:10:59.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.165+0000 7f42f7fff640 1 -- 172.21.15.42:0/774837507 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f430c0b09d0 con 0x55e2aead19f0 2024-08-30T09:10:59.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.166+0000 7f42f7fff640 10 client.26933 handle_command_reply: tid=0 2024-08-30T09:10:59.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.166+0000 7f43253d3740 1 client.26933 shutdown 2024-08-30T09:10:59.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.166+0000 7f43253d3740 1 -- 172.21.15.42:0/774837507 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e2aeb13070 msgr2=0x55e2aeed20e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:10:59.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.166+0000 7f43253d3740 1 --2- 172.21.15.42:0/774837507 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e2aeb13070 0x55e2aeed20e0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f4310031680 tx=0x7f4310005fd0 comp rx=0 tx=0).stop 2024-08-30T09:10:59.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.170+0000 7f43253d3740 1 -- 172.21.15.42:0/774837507 shutdown_connections 2024-08-30T09:10:59.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.170+0000 7f43253d3740 1 --2- 172.21.15.42:0/774837507 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55e2aead19f0 0x55e2aeede4d0 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:59.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.170+0000 7f43253d3740 1 --2- 172.21.15.42:0/774837507 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e2aeb13070 0x55e2aeed20e0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:59.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.170+0000 7f43253d3740 1 --2- 172.21.15.42:0/774837507 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e2aeaf2390 0x55e2aeecd0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:59.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.170+0000 7f43253d3740 1 --2- 172.21.15.42:0/774837507 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e2aeaea1b0 0x55e2aeeccae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:59.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.170+0000 7f43253d3740 1 -- 172.21.15.42:0/774837507 >> 172.21.15.42:0/774837507 conn(0x55e2aeb07460 msgr2=0x55e2aeeb7240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:10:59.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.170+0000 7f42f7fff640 0 client.26933 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:10:59.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.176+0000 7f43253d3740 1 -- 172.21.15.42:0/774837507 shutdown_connections 2024-08-30T09:10:59.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.176+0000 7f43253d3740 1 -- 172.21.15.42:0/774837507 wait complete. 2024-08-30T09:10:59.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.177+0000 7f43253d3740 20 client.26933 trim_cache size 0 max 16384 2024-08-30T09:10:59.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.205+0000 7f43253d3740 1 Processor -- start 2024-08-30T09:10:59.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.206+0000 7f43253d3740 1 -- start start 2024-08-30T09:10:59.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.207+0000 7f43253d3740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e2aeaea1b0 0x55e2aeeca3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:59.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.207+0000 7f43253d3740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e2aeaf2390 0x55e2aeeb7f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:59.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.207+0000 7f43253d3740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e2aeb13070 0x55e2aeeb5d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:59.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.207+0000 7f43253d3740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f43181b9970 con 0x55e2aeaea1b0 2024-08-30T09:10:59.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.207+0000 7f43253d3740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4318076bd0 con 0x55e2aeb13070 2024-08-30T09:10:59.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.207+0000 7f43253d3740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f4318074a20 con 0x55e2aeaf2390 2024-08-30T09:10:59.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.207+0000 7f431d3fb640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e2aeb13070 0x55e2aeeb5d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:10:59.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.207+0000 7f431d3fb640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e2aeb13070 0x55e2aeeb5d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:36366/0 (socket says 172.21.15.42:36366) 2024-08-30T09:10:59.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.207+0000 7f431d3fb640 1 -- 172.21.15.42:0/2845594966 learned_addr learned my addr 172.21.15.42:0/2845594966 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:10:59.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.207+0000 7f4317fff640 1 --2- 172.21.15.42:0/2845594966 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e2aeaf2390 0x55e2aeeb7f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:10:59.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.207+0000 7f431cbfa640 1 --2- 172.21.15.42:0/2845594966 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e2aeaea1b0 0x55e2aeeca3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:10:59.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.208+0000 7f431d3fb640 1 -- 172.21.15.42:0/2845594966 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e2aeaf2390 msgr2=0x55e2aeeb7f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:10:59.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.208+0000 7f431d3fb640 1 --2- 172.21.15.42:0/2845594966 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e2aeaf2390 0x55e2aeeb7f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:59.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.208+0000 7f431d3fb640 1 -- 172.21.15.42:0/2845594966 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e2aeaea1b0 msgr2=0x55e2aeeca3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:10:59.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.208+0000 7f431d3fb640 1 --2- 172.21.15.42:0/2845594966 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e2aeaea1b0 0x55e2aeeca3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:59.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.208+0000 7f431d3fb640 1 -- 172.21.15.42:0/2845594966 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e2aeec79b0 con 0x55e2aeb13070 2024-08-30T09:10:59.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.208+0000 7f4317fff640 1 --2- 172.21.15.42:0/2845594966 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e2aeaf2390 0x55e2aeeb7f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-30T09:10:59.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.209+0000 7f431d3fb640 1 --2- 172.21.15.42:0/2845594966 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e2aeb13070 0x55e2aeeb5d10 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f4310032fd0 tx=0x7f4310033000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=67df1efab35ad6b3 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:10:59.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.209+0000 7f43027fc640 1 -- 172.21.15.42:0/2845594966 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43100305c0 con 0x55e2aeb13070 2024-08-30T09:10:59.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.209+0000 7f43027fc640 1 -- 172.21.15.42:0/2845594966 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4310051040 con 0x55e2aeb13070 2024-08-30T09:10:59.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.210+0000 7f43027fc640 1 -- 172.21.15.42:0/2845594966 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4310048870 con 0x55e2aeb13070 2024-08-30T09:10:59.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.210+0000 7f43253d3740 1 -- 172.21.15.42:0/2845594966 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e2aeb13070 msgr2=0x55e2aeeb5d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:10:59.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.210+0000 7f43253d3740 1 --2- 172.21.15.42:0/2845594966 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e2aeb13070 0x55e2aeeb5d10 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f4310032fd0 tx=0x7f4310033000 comp rx=0 tx=0).stop 2024-08-30T09:10:59.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.210+0000 7f43253d3740 1 -- 172.21.15.42:0/2845594966 shutdown_connections 2024-08-30T09:10:59.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.210+0000 7f43253d3740 1 --2- 172.21.15.42:0/2845594966 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e2aeaf2390 0x55e2aeeb7f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:59.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.210+0000 7f43253d3740 1 --2- 172.21.15.42:0/2845594966 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e2aeb13070 0x55e2aeeb5d10 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:59.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.210+0000 7f43253d3740 1 --2- 172.21.15.42:0/2845594966 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e2aeaea1b0 0x55e2aeeca3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:59.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.210+0000 7f43253d3740 1 -- 172.21.15.42:0/2845594966 >> 172.21.15.42:0/2845594966 conn(0x55e2aeb07460 msgr2=0x55e2aeec6610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:10:59.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.211+0000 7f43253d3740 1 -- 172.21.15.42:0/2845594966 shutdown_connections 2024-08-30T09:10:59.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.211+0000 7f43253d3740 1 -- 172.21.15.42:0/2845594966 wait complete. 2024-08-30T09:10:59.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.211+0000 7f43253d3740 1 -- start start 2024-08-30T09:10:59.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.212+0000 7f43253d3740 1 Processor -- start 2024-08-30T09:10:59.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.213+0000 7f43253d3740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e2aeaea1b0 0x55e2aeed1630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:59.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.213+0000 7f43253d3740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e2aeaf2390 0x55e2aeed0080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:59.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.213+0000 7f43253d3740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e2aeb13070 0x55e2aeed05c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:59.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.213+0000 7f43253d3740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f4318075090 con 0x55e2aeaf2390 2024-08-30T09:10:59.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.213+0000 7f43253d3740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4318076500 con 0x55e2aeaea1b0 2024-08-30T09:10:59.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.213+0000 7f43253d3740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f4318075760 con 0x55e2aeb13070 2024-08-30T09:10:59.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.213+0000 7f43253d3740 10 client.0 _ll_register_callbacks cb 0x7f4310009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:10:59.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.213+0000 7f4317fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e2aeaf2390 0x55e2aeed0080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:10:59.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.213+0000 7f4317fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e2aeaf2390 0x55e2aeed0080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42616/0 (socket says 172.21.15.42:42616) 2024-08-30T09:10:59.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.213+0000 7f4317fff640 1 -- 172.21.15.42:0/329361725 learned_addr learned my addr 172.21.15.42:0/329361725 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:10:59.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.214+0000 7f431cbfa640 1 --2- 172.21.15.42:0/329361725 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e2aeaea1b0 0x55e2aeed1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:10:59.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.214+0000 7f431d3fb640 1 --2- 172.21.15.42:0/329361725 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e2aeb13070 0x55e2aeed05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:10:59.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.214+0000 7f4317fff640 1 -- 172.21.15.42:0/329361725 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e2aeb13070 msgr2=0x55e2aeed05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:10:59.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.214+0000 7f4317fff640 1 --2- 172.21.15.42:0/329361725 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e2aeb13070 0x55e2aeed05c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:59.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.214+0000 7f4317fff640 1 -- 172.21.15.42:0/329361725 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e2aeaea1b0 msgr2=0x55e2aeed1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:10:59.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.214+0000 7f4317fff640 1 --2- 172.21.15.42:0/329361725 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e2aeaea1b0 0x55e2aeed1630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:59.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.214+0000 7f4317fff640 1 -- 172.21.15.42:0/329361725 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e2aeed9100 con 0x55e2aeaf2390 2024-08-30T09:10:59.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.215+0000 7f4317fff640 1 --2- 172.21.15.42:0/329361725 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e2aeaf2390 0x55e2aeed0080 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f431806b8d0 tx=0x7f43040127c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b67cb73691e3dd5 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:10:59.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.215+0000 7f42f7fff640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-08-30T09:10:59.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.215+0000 7f42f7fff640 1 -- 172.21.15.42:0/329361725 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4304020060 con 0x55e2aeaf2390 2024-08-30T09:10:59.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.215+0000 7f43253d3740 1 -- 172.21.15.42:0/329361725 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4310051040 con 0x55e2aeaf2390 2024-08-30T09:10:59.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.215+0000 7f42f7fff640 1 -- 172.21.15.42:0/329361725 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f43040106a0 con 0x55e2aeaf2390 2024-08-30T09:10:59.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.216+0000 7f42f7fff640 1 -- 172.21.15.42:0/329361725 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f43040108c0 con 0x55e2aeaf2390 2024-08-30T09:10:59.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.216+0000 7f43253d3740 10 client.17130 fetch_fsmap learned FSMap version 41 2024-08-30T09:10:59.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.216+0000 7f43253d3740 1 -- 172.21.15.42:0/329361725 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55e2aeebc620 con 0x55e2aeaf2390 2024-08-30T09:10:59.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.216+0000 7f42f7fff640 1 -- 172.21.15.42:0/329361725 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4310051040 con 0x55e2aeaf2390 2024-08-30T09:10:59.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.217+0000 7f42f7fff640 1 -- 172.21.15.42:0/329361725 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 41) ==== 8362+0+0 (secure 0 0 0) 0x7f4304025ca0 con 0x55e2aeaf2390 2024-08-30T09:10:59.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.218+0000 7f43253d3740 10 client.17130 fetch_fsmap finished waiting for FSMap version 41 2024-08-30T09:10:59.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.218+0000 7f43253d3740 10 client.17130 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-30T09:10:59.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.218+0000 7f43253d3740 20 client.17130 populate_metadata read hostname 'smithi042' 2024-08-30T09:10:59.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.218+0000 7f43253d3740 1 --2- 172.21.15.42:0/329361725 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55e2aeaa6da0 0x55e2aeebedf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:10:59.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.218+0000 7f43253d3740 4 client.17130 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-30T09:10:59.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.218+0000 7f43253d3740 1 -- 172.21.15.42:0/329361725 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e2aee315d0 con 0x55e2aeaa6da0 2024-08-30T09:10:59.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.218+0000 7f431cbfa640 1 --2- 172.21.15.42:0/329361725 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55e2aeaa6da0 0x55e2aeebedf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:10:59.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.219+0000 7f431cbfa640 1 --2- 172.21.15.42:0/329361725 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55e2aeaa6da0 0x55e2aeebedf0 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=965955962ca126c5 server_cookie=29c2ddd1c9d70131 in_seq=0 out_seq=0 2024-08-30T09:10:59.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.220+0000 7f42f7fff640 10 client.17130 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:10:59.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.220+0000 7f42f7fff640 1 -- 172.21.15.42:0/329361725 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e2aee315d0 con 0x55e2aeaa6da0 2024-08-30T09:10:59.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.220+0000 7f42f7fff640 10 client.17130 handle_command_reply: tid=0 2024-08-30T09:10:59.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.220+0000 7f43253d3740 1 client.17130 shutdown 2024-08-30T09:10:59.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.220+0000 7f43253d3740 1 -- 172.21.15.42:0/329361725 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e2aeaf2390 msgr2=0x55e2aeed0080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:10:59.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.220+0000 7f43253d3740 1 --2- 172.21.15.42:0/329361725 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e2aeaf2390 0x55e2aeed0080 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f431806b8d0 tx=0x7f43040127c0 comp rx=0 tx=0).stop 2024-08-30T09:10:59.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.220+0000 7f43253d3740 1 -- 172.21.15.42:0/329361725 shutdown_connections 2024-08-30T09:10:59.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.220+0000 7f43253d3740 1 --2- 172.21.15.42:0/329361725 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55e2aeaa6da0 0x55e2aeebedf0 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:59.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.220+0000 7f43253d3740 1 --2- 172.21.15.42:0/329361725 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e2aeb13070 0x55e2aeed05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:59.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.220+0000 7f43253d3740 1 --2- 172.21.15.42:0/329361725 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e2aeaf2390 0x55e2aeed0080 secure :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f431806b8d0 tx=0x7f43040127c0 comp rx=0 tx=0).stop 2024-08-30T09:10:59.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.220+0000 7f43253d3740 1 --2- 172.21.15.42:0/329361725 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e2aeaea1b0 0x55e2aeed1630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:59.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.220+0000 7f43253d3740 1 -- 172.21.15.42:0/329361725 >> 172.21.15.42:0/329361725 conn(0x55e2aeb07460 msgr2=0x55e2aeecb0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:10:59.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.220+0000 7f42f7fff640 0 client.17130 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:10:59.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.221+0000 7f43253d3740 1 -- 172.21.15.42:0/329361725 shutdown_connections 2024-08-30T09:10:59.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.221+0000 7f43253d3740 1 -- 172.21.15.42:0/329361725 wait complete. 2024-08-30T09:10:59.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.221+0000 7f43253d3740 20 client.17130 trim_cache size 0 max 16384 2024-08-30T09:10:59.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.222+0000 7f431ee5e640 1 -- 172.21.15.42:0/829198374 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f43080b4d00 msgr2=0x7f43080b7210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:10:59.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.222+0000 7f431ee5e640 1 --2- 172.21.15.42:0/829198374 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f43080b4d00 0x7f43080b7210 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f431002ade0 tx=0x7f43100094a0 comp rx=0 tx=0).stop 2024-08-30T09:10:59.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.222+0000 7f431ee5e640 1 -- 172.21.15.42:0/829198374 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4318103930 msgr2=0x7f431819a1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:10:59.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.222+0000 7f431ee5e640 1 --2- 172.21.15.42:0/829198374 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4318103930 0x7f431819a1d0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f430c00ea20 tx=0x7f430c00eee0 comp rx=0 tx=0).stop 2024-08-30T09:10:59.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.223+0000 7f431ee5e640 1 -- 172.21.15.42:0/829198374 shutdown_connections 2024-08-30T09:10:59.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.223+0000 7f431ee5e640 1 --2- 172.21.15.42:0/829198374 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f43080b4d00 0x7f43080b7210 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:59.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.223+0000 7f431ee5e640 1 --2- 172.21.15.42:0/829198374 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4318109bc0 0x7f431819fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:59.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.223+0000 7f431ee5e640 1 --2- 172.21.15.42:0/829198374 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4318109270 0x7f431819a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:59.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.223+0000 7f431ee5e640 1 --2- 172.21.15.42:0/829198374 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4318103930 0x7f431819a1d0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:10:59.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.223+0000 7f431ee5e640 1 -- 172.21.15.42:0/829198374 >> 172.21.15.42:0/829198374 conn(0x7f43180ff3a0 msgr2=0x7f43181017c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:10:59.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.223+0000 7f431ee5e640 1 -- 172.21.15.42:0/829198374 shutdown_connections 2024-08-30T09:10:59.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:10:59.223+0000 7f431ee5e640 1 -- 172.21.15.42:0/829198374 wait complete. 2024-08-30T09:10:59.502 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:10:59 smithi042 ceph-mon[18652]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-30T09:10:59.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:10:59 smithi190 ceph-mon[22001]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-30T09:10:59.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:10:59 smithi167 ceph-mon[22063]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-30T09:11:00.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:11:00 smithi042 ceph-mon[18652]: pgmap v680: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 8.9 MiB/s wr, 572 op/s 2024-08-30T09:11:00.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:11:00 smithi190 ceph-mon[22001]: pgmap v680: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 8.9 MiB/s wr, 572 op/s 2024-08-30T09:11:00.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:11:00 smithi167 ceph-mon[22063]: pgmap v680: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 8.9 MiB/s wr, 572 op/s 2024-08-30T09:11:00.953 INFO:teuthology.orchestra.run.smithi042.stderr:progress: 615.0 s, 510.8 tps, lat 199.228 ms stddev 236.059 2024-08-30T09:11:01.865 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-08-30T09:11:01.865 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-08-30T09:11:02.450 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T09:11:02.646 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.642+0000 7f3a114d6640 1 -- 172.21.15.42:0/1603796846 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3a0c10ab40 msgr2=0x7f3a0c10b010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:11:02.646 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.642+0000 7f3a114d6640 1 --2- 172.21.15.42:0/1603796846 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3a0c10ab40 0x7f3a0c10b010 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f3a04009900 tx=0x7f3a0402a420 comp rx=0 tx=0).stop 2024-08-30T09:11:02.658 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.651+0000 7f3a114d6640 1 -- 172.21.15.42:0/1603796846 shutdown_connections 2024-08-30T09:11:02.658 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.651+0000 7f3a114d6640 1 --2- 172.21.15.42:0/1603796846 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3a0c10ab40 0x7f3a0c10b010 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:02.658 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.651+0000 7f3a114d6640 1 --2- 172.21.15.42:0/1603796846 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3a0c111d30 0x7f3a0c114150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:02.658 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.651+0000 7f3a114d6640 1 --2- 172.21.15.42:0/1603796846 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3a0c10a160 0x7f3a0c10a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:02.658 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.651+0000 7f3a114d6640 1 -- 172.21.15.42:0/1603796846 >> 172.21.15.42:0/1603796846 conn(0x7f3a0c0705b0 msgr2=0x7f3a0c072a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:11:02.658 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.656+0000 7f3a114d6640 1 -- 172.21.15.42:0/1603796846 shutdown_connections 2024-08-30T09:11:02.660 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.657+0000 7f3a114d6640 1 -- 172.21.15.42:0/1603796846 wait complete. 2024-08-30T09:11:02.660 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.658+0000 7f3a114d6640 1 Processor -- start 2024-08-30T09:11:02.663 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.661+0000 7f3a114d6640 1 -- start start 2024-08-30T09:11:02.663 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.661+0000 7f3a114d6640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3a0c10a160 0x7f3a0c11caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:02.664 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.661+0000 7f3a114d6640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3a0c10ab40 0x7f3a0c11b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:02.664 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.661+0000 7f3a114d6640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3a0c111d30 0x7f3a0c11b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:02.664 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.661+0000 7f3a114d6640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f3a0c115690 con 0x7f3a0c10a160 2024-08-30T09:11:02.664 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.661+0000 7f3a114d6640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f3a0c115510 con 0x7f3a0c111d30 2024-08-30T09:11:02.664 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.661+0000 7f3a114d6640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f3a0c115810 con 0x7f3a0c10ab40 2024-08-30T09:11:02.667 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.664+0000 7f3a0b7fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3a0c111d30 0x7f3a0c11b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:02.668 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.664+0000 7f3a0b7fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3a0c111d30 0x7f3a0c11b8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:36386/0 (socket says 172.21.15.42:36386) 2024-08-30T09:11:02.668 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.664+0000 7f3a0b7fe640 1 -- 172.21.15.42:0/527313641 learned_addr learned my addr 172.21.15.42:0/527313641 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:11:02.671 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.667+0000 7f3a0affd640 1 --2- 172.21.15.42:0/527313641 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3a0c10a160 0x7f3a0c11caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:02.671 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.667+0000 7f3a0a7fc640 1 --2- 172.21.15.42:0/527313641 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3a0c10ab40 0x7f3a0c11b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:02.671 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.668+0000 7f3a0b7fe640 1 -- 172.21.15.42:0/527313641 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3a0c10ab40 msgr2=0x7f3a0c11b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:11:02.671 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.668+0000 7f3a0b7fe640 1 --2- 172.21.15.42:0/527313641 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3a0c10ab40 0x7f3a0c11b360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:02.671 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.668+0000 7f3a0b7fe640 1 -- 172.21.15.42:0/527313641 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3a0c10a160 msgr2=0x7f3a0c11caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:11:02.671 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.668+0000 7f3a0b7fe640 1 --2- 172.21.15.42:0/527313641 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3a0c10a160 0x7f3a0c11caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:02.671 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.668+0000 7f3a0b7fe640 1 -- 172.21.15.42:0/527313641 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a0c11be60 con 0x7f3a0c111d30 2024-08-30T09:11:02.672 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.670+0000 7f3a0b7fe640 1 --2- 172.21.15.42:0/527313641 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3a0c111d30 0x7f3a0c11b8a0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f39f400eb10 tx=0x7f39f400c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:11:02.675 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.673+0000 7f3a03fff640 1 -- 172.21.15.42:0/527313641 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39f4004b20 con 0x7f3a0c111d30 2024-08-30T09:11:02.675 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.673+0000 7f3a03fff640 1 -- 172.21.15.42:0/527313641 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f39f4004cc0 con 0x7f3a0c111d30 2024-08-30T09:11:02.675 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.673+0000 7f3a03fff640 1 -- 172.21.15.42:0/527313641 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39f4010690 con 0x7f3a0c111d30 2024-08-30T09:11:02.675 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.673+0000 7f3a114d6640 1 -- 172.21.15.42:0/527313641 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a0c11c150 con 0x7f3a0c111d30 2024-08-30T09:11:02.675 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.673+0000 7f3a114d6640 1 -- 172.21.15.42:0/527313641 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a0c1b9630 con 0x7f3a0c111d30 2024-08-30T09:11:02.682 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.678+0000 7f3a18436740 1 Processor -- start 2024-08-30T09:11:02.682 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.680+0000 7f3a18436740 1 -- start start 2024-08-30T09:11:02.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:11:02 smithi190 ceph-mon[22001]: pgmap v681: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 7.0 MiB/s wr, 432 op/s 2024-08-30T09:11:02.690 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.687+0000 7f3a18436740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561bb4f491b0 0x561bb52ef180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:02.691 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.687+0000 7f3a18436740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561bb4f202f0 0x561bb52ef7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:02.691 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.687+0000 7f3a18436740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561bb4f284d0 0x561bb52f0340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:02.691 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.687+0000 7f3a18436740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x561bb5262000 con 0x561bb4f491b0 2024-08-30T09:11:02.691 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.687+0000 7f3a18436740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x561bb5251590 con 0x561bb4f202f0 2024-08-30T09:11:02.691 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.687+0000 7f3a18436740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x561bb5255540 con 0x561bb4f284d0 2024-08-30T09:11:02.691 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.689+0000 7f3a0a7fc640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561bb4f284d0 0x561bb52f0340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:02.691 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.689+0000 7f3a0a7fc640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561bb4f491b0 0x561bb52ef180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:02.691 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.689+0000 7f3a0a7fc640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561bb4f491b0 0x561bb52ef180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42636/0 (socket says 172.21.15.42:42636) 2024-08-30T09:11:02.691 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.689+0000 7f3a0a7fc640 1 -- 172.21.15.42:0/4061590612 learned_addr learned my addr 172.21.15.42:0/4061590612 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:11:02.691 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.689+0000 7f3a0affd640 1 --2- 172.21.15.42:0/4061590612 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561bb4f202f0 0x561bb52ef7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:02.692 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.689+0000 7f3a03fff640 1 -- 172.21.15.42:0/527313641 <== mon.2 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7f39f4005000 con 0x7f3a0c111d30 2024-08-30T09:11:02.692 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.689+0000 7f3a0affd640 1 -- 172.21.15.42:0/4061590612 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561bb4f284d0 msgr2=0x561bb52f0340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:11:02.692 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.689+0000 7f3a0affd640 1 --2- 172.21.15.42:0/4061590612 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561bb4f284d0 0x561bb52f0340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:02.692 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.690+0000 7f3a0affd640 1 -- 172.21.15.42:0/4061590612 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561bb4f491b0 msgr2=0x561bb52ef180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:11:02.692 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.690+0000 7f3a0affd640 1 --2- 172.21.15.42:0/4061590612 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561bb4f491b0 0x561bb52ef180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:02.692 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.690+0000 7f3a0affd640 1 -- 172.21.15.42:0/4061590612 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561bb4fb4790 con 0x561bb4f202f0 2024-08-30T09:11:02.692 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.690+0000 7f3a0affd640 1 --2- 172.21.15.42:0/4061590612 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561bb4f202f0 0x561bb52ef7f0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f39fc00bbe0 tx=0x7f39fc00f730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fe1bc6782a9adf1f server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:11:02.693 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.691+0000 7f3a03fff640 1 --2- 172.21.15.42:0/527313641 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f39f80b4d40 0x7f39f80b7250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:02.693 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.692+0000 7f3a03fff640 1 -- 172.21.15.42:0/527313641 <== mon.2 v2:172.21.15.167:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f39f40e5a70 con 0x7f3a0c111d30 2024-08-30T09:11:02.694 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.690+0000 7f3a017fa640 1 -- 172.21.15.42:0/4061590612 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39fc015070 con 0x561bb4f202f0 2024-08-30T09:11:02.694 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.692+0000 7f3a017fa640 1 -- 172.21.15.42:0/4061590612 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f39fc0035f0 con 0x561bb4f202f0 2024-08-30T09:11:02.694 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.692+0000 7f3a017fa640 1 -- 172.21.15.42:0/4061590612 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39fc0116a0 con 0x561bb4f202f0 2024-08-30T09:11:02.698 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.696+0000 7f3a0a7fc640 1 --2- 172.21.15.42:0/527313641 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f39f80b4d40 0x7f39f80b7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:02.700 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.699+0000 7f3a18436740 1 -- 172.21.15.42:0/4061590612 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561bb4f202f0 msgr2=0x561bb52ef7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:11:02.701 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.699+0000 7f3a18436740 1 --2- 172.21.15.42:0/4061590612 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561bb4f202f0 0x561bb52ef7f0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f39fc00bbe0 tx=0x7f39fc00f730 comp rx=0 tx=0).stop 2024-08-30T09:11:02.703 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.702+0000 7f3a18436740 1 -- 172.21.15.42:0/4061590612 shutdown_connections 2024-08-30T09:11:02.703 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.702+0000 7f3a18436740 1 --2- 172.21.15.42:0/4061590612 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561bb4f284d0 0x561bb52f0340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:02.704 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.702+0000 7f3a18436740 1 --2- 172.21.15.42:0/4061590612 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561bb4f202f0 0x561bb52ef7f0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:02.704 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.702+0000 7f3a18436740 1 --2- 172.21.15.42:0/4061590612 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561bb4f491b0 0x561bb52ef180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:02.704 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.702+0000 7f3a18436740 1 -- 172.21.15.42:0/4061590612 >> 172.21.15.42:0/4061590612 conn(0x561bb4f3d300 msgr2=0x561bb52eac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:11:02.707 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.705+0000 7f3a0a7fc640 1 --2- 172.21.15.42:0/527313641 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f39f80b4d40 0x7f39f80b7250 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f3a04005f70 tx=0x7f3a0402a420 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:11:02.710 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.707+0000 7f3a18436740 1 -- 172.21.15.42:0/4061590612 shutdown_connections 2024-08-30T09:11:02.712 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.708+0000 7f3a18436740 1 -- 172.21.15.42:0/4061590612 wait complete. 2024-08-30T09:11:02.712 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.708+0000 7f3a18436740 1 -- start start 2024-08-30T09:11:02.713 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.708+0000 7f3a18436740 1 Processor -- start 2024-08-30T09:11:02.713 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.709+0000 7f3a18436740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561bb4f202f0 0x561bb52f8cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:02.713 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.709+0000 7f3a18436740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561bb4f284d0 0x561bb52f9370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:02.713 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.709+0000 7f3a18436740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561bb4f491b0 0x561bb52f9ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:02.713 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.709+0000 7f3a18436740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x561bb518aab0 con 0x561bb4f284d0 2024-08-30T09:11:02.713 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.709+0000 7f3a18436740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x561bb51d8680 con 0x561bb4f202f0 2024-08-30T09:11:02.713 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.709+0000 7f3a18436740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x561bb52f0e60 con 0x561bb4f491b0 2024-08-30T09:11:02.713 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.709+0000 7f3a18436740 10 client.0 _ll_register_callbacks cb 0x561bb4f60a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:11:02.713 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.709+0000 7f3a0affd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561bb4f202f0 0x561bb52f8cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:02.713 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.709+0000 7f3a0affd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561bb4f202f0 0x561bb52f8cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:36394/0 (socket says 172.21.15.42:36394) 2024-08-30T09:11:02.713 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.709+0000 7f3a0affd640 1 -- 172.21.15.42:0/4256658125 learned_addr learned my addr 172.21.15.42:0/4256658125 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:11:02.713 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.709+0000 7f3a0b7fe640 1 --2- 172.21.15.42:0/4256658125 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561bb4f491b0 0x561bb52f9ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:02.714 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.710+0000 7f3a0affd640 1 -- 172.21.15.42:0/4256658125 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561bb4f491b0 msgr2=0x561bb52f9ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:11:02.714 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.710+0000 7f3a0affd640 1 --2- 172.21.15.42:0/4256658125 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561bb4f491b0 0x561bb52f9ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:02.714 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.710+0000 7f3a0affd640 1 -- 172.21.15.42:0/4256658125 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561bb4f284d0 msgr2=0x561bb52f9370 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-30T09:11:02.714 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.710+0000 7f3a0affd640 1 --2- 172.21.15.42:0/4256658125 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561bb4f284d0 0x561bb52f9370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:02.714 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.710+0000 7f3a0affd640 1 -- 172.21.15.42:0/4256658125 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561bb502d620 con 0x561bb4f202f0 2024-08-30T09:11:02.714 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.710+0000 7f3a0affd640 1 --2- 172.21.15.42:0/4256658125 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561bb4f202f0 0x561bb52f8cb0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f39fc00bf00 tx=0x7f39fc00bf30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=93520d79129f32a2 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:11:02.714 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.710+0000 7f39f27fc640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-30T09:11:02.715 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.711+0000 7f39f27fc640 1 -- 172.21.15.42:0/4256658125 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39fc015070 con 0x561bb4f202f0 2024-08-30T09:11:02.715 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.711+0000 7f3a18436740 1 -- 172.21.15.42:0/4256658125 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f39fc0035f0 con 0x561bb4f202f0 2024-08-30T09:11:02.715 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.711+0000 7f39f27fc640 1 -- 172.21.15.42:0/4256658125 <== mon.2 v2:172.21.15.167:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f39fc004200 con 0x561bb4f202f0 2024-08-30T09:11:02.715 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.711+0000 7f39f27fc640 1 -- 172.21.15.42:0/4256658125 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f39fc02e050 con 0x561bb4f202f0 2024-08-30T09:11:02.715 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.712+0000 7f3a18436740 10 client.26951 fetch_fsmap learned FSMap version 41 2024-08-30T09:11:02.715 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.712+0000 7f3a18436740 1 -- 172.21.15.42:0/4256658125 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x561bb5324e70 con 0x561bb4f202f0 2024-08-30T09:11:02.719 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.716+0000 7f39f27fc640 1 -- 172.21.15.42:0/4256658125 <== mon.2 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f39fc0035f0 con 0x561bb4f202f0 2024-08-30T09:11:02.719 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.716+0000 7f39f27fc640 1 -- 172.21.15.42:0/4256658125 <== mon.2 v2:172.21.15.167:3300/0 5 ==== fsmap(e 41) ==== 8362+0+0 (secure 0 0 0) 0x7f39fc0049a0 con 0x561bb4f202f0 2024-08-30T09:11:02.720 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.716+0000 7f3a18436740 10 client.26951 fetch_fsmap finished waiting for FSMap version 41 2024-08-30T09:11:02.720 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.716+0000 7f3a18436740 10 client.26951 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-30T09:11:02.720 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.716+0000 7f3a18436740 20 client.26951 populate_metadata read hostname 'smithi042' 2024-08-30T09:11:02.720 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.716+0000 7f3a18436740 1 --2- 172.21.15.42:0/4256658125 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x561bb4f07d80 0x561bb5328a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:02.720 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.716+0000 7f3a18436740 4 client.26951 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-30T09:11:02.720 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.716+0000 7f3a18436740 1 -- 172.21.15.42:0/4256658125 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561bb5263f50 con 0x561bb4f07d80 2024-08-30T09:11:02.723 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.722+0000 7f3a0a7fc640 1 --2- 172.21.15.42:0/4256658125 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x561bb4f07d80 0x561bb5328a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:02.730 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.728+0000 7f3a0a7fc640 1 --2- 172.21.15.42:0/4256658125 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x561bb4f07d80 0x561bb5328a20 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=93fbb76b95a16ae9 server_cookie=743f08c6dbb81e02 in_seq=0 out_seq=0 2024-08-30T09:11:02.730 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.728+0000 7f39f27fc640 10 client.26951 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:11:02.733 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.731+0000 7f39f27fc640 1 -- 172.21.15.42:0/4256658125 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x561bb5263f50 con 0x561bb4f07d80 2024-08-30T09:11:02.733 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.732+0000 7f39f27fc640 10 client.26951 handle_command_reply: tid=0 2024-08-30T09:11:02.737 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.732+0000 7f3a18436740 1 client.26951 shutdown 2024-08-30T09:11:02.744 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.741+0000 7f3a18436740 1 -- 172.21.15.42:0/4256658125 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561bb4f202f0 msgr2=0x561bb52f8cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:11:02.744 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.741+0000 7f3a18436740 1 --2- 172.21.15.42:0/4256658125 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561bb4f202f0 0x561bb52f8cb0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f39fc00bf00 tx=0x7f39fc00bf30 comp rx=0 tx=0).stop 2024-08-30T09:11:02.747 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.745+0000 7f3a18436740 1 -- 172.21.15.42:0/4256658125 shutdown_connections 2024-08-30T09:11:02.747 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.745+0000 7f3a18436740 1 --2- 172.21.15.42:0/4256658125 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x561bb4f07d80 0x561bb5328a20 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:02.748 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.745+0000 7f3a18436740 1 --2- 172.21.15.42:0/4256658125 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561bb4f491b0 0x561bb52f9ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:02.748 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.745+0000 7f3a18436740 1 --2- 172.21.15.42:0/4256658125 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561bb4f284d0 0x561bb52f9370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:02.748 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.745+0000 7f3a18436740 1 --2- 172.21.15.42:0/4256658125 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561bb4f202f0 0x561bb52f8cb0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:02.748 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.745+0000 7f3a18436740 1 -- 172.21.15.42:0/4256658125 >> 172.21.15.42:0/4256658125 conn(0x561bb4f3d300 msgr2=0x561bb52e9170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:11:02.748 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.746+0000 7f39f27fc640 0 client.26951 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:11:02.749 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.747+0000 7f3a18436740 1 -- 172.21.15.42:0/4256658125 shutdown_connections 2024-08-30T09:11:02.749 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.747+0000 7f3a18436740 1 -- 172.21.15.42:0/4256658125 wait complete. 2024-08-30T09:11:02.749 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.747+0000 7f3a18436740 20 client.26951 trim_cache size 0 max 16384 2024-08-30T09:11:02.779 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:11:02 smithi042 ceph-mon[18652]: pgmap v681: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 7.0 MiB/s wr, 432 op/s 2024-08-30T09:11:02.796 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.794+0000 7f3a18436740 1 Processor -- start 2024-08-30T09:11:02.796 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.794+0000 7f3a18436740 1 -- start start 2024-08-30T09:11:02.797 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.795+0000 7f3a18436740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561bb4f202f0 0x561bb52f7380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:02.798 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.796+0000 7f3a18436740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561bb4f284d0 0x561bb5300dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:02.798 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.796+0000 7f3a18436740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561bb4f491b0 0x561bb52f78c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:02.798 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.796+0000 7f3a0affd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561bb4f202f0 0x561bb52f7380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:02.798 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.796+0000 7f3a0affd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561bb4f202f0 0x561bb52f7380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:47900/0 (socket says 172.21.15.42:47900) 2024-08-30T09:11:02.798 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.796+0000 7f3a0affd640 1 -- 172.21.15.42:0/2437272461 learned_addr learned my addr 172.21.15.42:0/2437272461 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:11:02.798 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.796+0000 7f3a0b7fe640 1 --2- 172.21.15.42:0/2437272461 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561bb4f491b0 0x561bb52f78c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:02.799 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.796+0000 7f3a18436740 1 -- 172.21.15.42:0/2437272461 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f39ec012610 con 0x561bb4f284d0 2024-08-30T09:11:02.799 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.796+0000 7f3a18436740 1 -- 172.21.15.42:0/2437272461 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f39ec00fe80 con 0x561bb4f491b0 2024-08-30T09:11:02.799 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.796+0000 7f3a18436740 1 -- 172.21.15.42:0/2437272461 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f39ec00dcd0 con 0x561bb4f202f0 2024-08-30T09:11:02.800 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.798+0000 7f3a0affd640 1 -- 172.21.15.42:0/2437272461 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561bb4f491b0 msgr2=0x561bb52f78c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:11:02.800 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.798+0000 7f3a0affd640 1 --2- 172.21.15.42:0/2437272461 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561bb4f491b0 0x561bb52f78c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:02.800 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.798+0000 7f3a0affd640 1 -- 172.21.15.42:0/2437272461 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561bb4f284d0 msgr2=0x561bb5300dc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-30T09:11:02.800 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.798+0000 7f3a0affd640 1 --2- 172.21.15.42:0/2437272461 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561bb4f284d0 0x561bb5300dc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:02.800 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.798+0000 7f3a0affd640 1 -- 172.21.15.42:0/2437272461 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561bb52effc0 con 0x561bb4f202f0 2024-08-30T09:11:02.800 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.798+0000 7f3a0affd640 1 --2- 172.21.15.42:0/2437272461 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561bb4f202f0 0x561bb52f7380 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f39fc00b970 tx=0x7f39fc005bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c6c364d77118c92a server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:11:02.801 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.798+0000 7f3a00ff9640 1 -- 172.21.15.42:0/2437272461 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39fc036020 con 0x561bb4f202f0 2024-08-30T09:11:02.802 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.799+0000 7f3a00ff9640 1 -- 172.21.15.42:0/2437272461 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f39fc0035f0 con 0x561bb4f202f0 2024-08-30T09:11:02.802 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.799+0000 7f3a00ff9640 1 -- 172.21.15.42:0/2437272461 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39fc015070 con 0x561bb4f202f0 2024-08-30T09:11:02.803 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.801+0000 7f3a18436740 1 -- 172.21.15.42:0/2437272461 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561bb4f202f0 msgr2=0x561bb52f7380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:11:02.803 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.801+0000 7f3a18436740 1 --2- 172.21.15.42:0/2437272461 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561bb4f202f0 0x561bb52f7380 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f39fc00b970 tx=0x7f39fc005bd0 comp rx=0 tx=0).stop 2024-08-30T09:11:02.809 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.807+0000 7f3a18436740 1 -- 172.21.15.42:0/2437272461 shutdown_connections 2024-08-30T09:11:02.809 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.807+0000 7f3a18436740 1 --2- 172.21.15.42:0/2437272461 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561bb4f491b0 0x561bb52f78c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:02.809 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.807+0000 7f3a18436740 1 --2- 172.21.15.42:0/2437272461 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561bb4f284d0 0x561bb5300dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:02.809 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.807+0000 7f3a18436740 1 --2- 172.21.15.42:0/2437272461 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561bb4f202f0 0x561bb52f7380 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:02.809 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.807+0000 7f3a18436740 1 -- 172.21.15.42:0/2437272461 >> 172.21.15.42:0/2437272461 conn(0x561bb4f3d300 msgr2=0x561bb52ea550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:11:02.816 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.814+0000 7f3a18436740 1 -- 172.21.15.42:0/2437272461 shutdown_connections 2024-08-30T09:11:02.818 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.817+0000 7f3a18436740 1 -- 172.21.15.42:0/2437272461 wait complete. 2024-08-30T09:11:02.819 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.817+0000 7f3a18436740 1 -- start start 2024-08-30T09:11:02.819 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.817+0000 7f3a18436740 1 Processor -- start 2024-08-30T09:11:02.823 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.819+0000 7f3a18436740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561bb4f202f0 0x561bb52fd620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:02.823 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.819+0000 7f3a18436740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561bb4f284d0 0x561bb52fdce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:02.824 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.819+0000 7f3a18436740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561bb4f491b0 0x561bb52fe850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:02.824 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.819+0000 7f3a18436740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f39ec00e340 con 0x561bb4f491b0 2024-08-30T09:11:02.824 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.819+0000 7f3a18436740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f39ec00f7b0 con 0x561bb4f284d0 2024-08-30T09:11:02.824 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.819+0000 7f3a18436740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f39ec00ea10 con 0x561bb4f202f0 2024-08-30T09:11:02.824 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.819+0000 7f3a18436740 10 client.0 _ll_register_callbacks cb 0x7f39fc00ba30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:11:02.824 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.821+0000 7f3a0b7fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561bb4f491b0 0x561bb52fe850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:02.824 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.821+0000 7f3a0b7fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561bb4f491b0 0x561bb52fe850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42650/0 (socket says 172.21.15.42:42650) 2024-08-30T09:11:02.824 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.821+0000 7f3a0b7fe640 1 -- 172.21.15.42:0/2322077078 learned_addr learned my addr 172.21.15.42:0/2322077078 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:11:02.825 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.822+0000 7f3a0a7fc640 1 --2- 172.21.15.42:0/2322077078 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561bb4f284d0 0x561bb52fdce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:02.825 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.822+0000 7f3a0affd640 1 --2- 172.21.15.42:0/2322077078 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561bb4f202f0 0x561bb52fd620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:02.825 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.823+0000 7f3a0a7fc640 1 -- 172.21.15.42:0/2322077078 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561bb4f202f0 msgr2=0x561bb52fd620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:11:02.825 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.823+0000 7f3a0a7fc640 1 --2- 172.21.15.42:0/2322077078 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561bb4f202f0 0x561bb52fd620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:02.825 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.823+0000 7f3a0a7fc640 1 -- 172.21.15.42:0/2322077078 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561bb4f491b0 msgr2=0x561bb52fe850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:11:02.825 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.823+0000 7f3a0a7fc640 1 --2- 172.21.15.42:0/2322077078 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561bb4f491b0 0x561bb52fe850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:02.826 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.823+0000 7f3a0a7fc640 1 -- 172.21.15.42:0/2322077078 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561bb52fefb0 con 0x561bb4f284d0 2024-08-30T09:11:02.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:11:02 smithi167 ceph-mon[22063]: pgmap v681: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 7.0 MiB/s wr, 432 op/s 2024-08-30T09:11:02.829 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.827+0000 7f3a0a7fc640 1 --2- 172.21.15.42:0/2322077078 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561bb4f284d0 0x561bb52fdce0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f3a04031440 tx=0x7f3a040314a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f1f3d23db5140898 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:11:02.829 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.827+0000 7f39f27fc640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-30T09:11:02.829 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.827+0000 7f39f27fc640 1 -- 172.21.15.42:0/2322077078 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a04035040 con 0x561bb4f284d0 2024-08-30T09:11:02.829 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.827+0000 7f39f27fc640 1 -- 172.21.15.42:0/2322077078 <== mon.2 v2:172.21.15.167:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f3a04003dd0 con 0x561bb4f284d0 2024-08-30T09:11:02.829 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.827+0000 7f3a18436740 1 -- 172.21.15.42:0/2322077078 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f39fc0035f0 con 0x561bb4f284d0 2024-08-30T09:11:02.830 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.828+0000 7f39f27fc640 1 -- 172.21.15.42:0/2322077078 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f3a040325b0 con 0x561bb4f284d0 2024-08-30T09:11:02.830 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.829+0000 7f3a18436740 10 client.26957 fetch_fsmap learned FSMap version 41 2024-08-30T09:11:02.831 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.829+0000 7f3a18436740 1 -- 172.21.15.42:0/2322077078 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x561bb5300370 con 0x561bb4f284d0 2024-08-30T09:11:02.833 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.831+0000 7f39f27fc640 1 -- 172.21.15.42:0/2322077078 <== mon.2 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f39fc0035f0 con 0x561bb4f284d0 2024-08-30T09:11:02.833 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.831+0000 7f39f27fc640 1 -- 172.21.15.42:0/2322077078 <== mon.2 v2:172.21.15.167:3300/0 5 ==== fsmap(e 41) ==== 8362+0+0 (secure 0 0 0) 0x7f3a04059070 con 0x561bb4f284d0 2024-08-30T09:11:02.833 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.831+0000 7f3a18436740 10 client.26957 fetch_fsmap finished waiting for FSMap version 41 2024-08-30T09:11:02.834 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.831+0000 7f3a18436740 10 client.26957 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-30T09:11:02.834 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.831+0000 7f3a18436740 20 client.26957 populate_metadata read hostname 'smithi042' 2024-08-30T09:11:02.834 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.831+0000 7f3a18436740 1 --2- 172.21.15.42:0/2322077078 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x561bb4edd1c0 0x561bb5300cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:02.834 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.831+0000 7f3a18436740 4 client.26957 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-30T09:11:02.834 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.831+0000 7f3a18436740 1 -- 172.21.15.42:0/2322077078 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561bb525f9f0 con 0x561bb4edd1c0 2024-08-30T09:11:02.834 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.832+0000 7f3a0affd640 1 --2- 172.21.15.42:0/2322077078 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x561bb4edd1c0 0x561bb5300cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:02.842 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.837+0000 7f3a0affd640 1 --2- 172.21.15.42:0/2322077078 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x561bb4edd1c0 0x561bb5300cb0 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d33af431ba03959d server_cookie=49caa44586a45ee1 in_seq=0 out_seq=0 2024-08-30T09:11:02.842 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.838+0000 7f39f27fc640 10 client.26957 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:11:02.885 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.883+0000 7f39f27fc640 1 -- 172.21.15.42:0/2322077078 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561bb525f9f0 con 0x561bb4edd1c0 2024-08-30T09:11:02.885 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.883+0000 7f39f27fc640 10 client.26957 handle_command_reply: tid=0 2024-08-30T09:11:02.885 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.883+0000 7f3a18436740 1 client.26957 shutdown 2024-08-30T09:11:02.885 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.883+0000 7f3a18436740 1 -- 172.21.15.42:0/2322077078 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561bb4f284d0 msgr2=0x561bb52fdce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:11:02.885 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.883+0000 7f3a18436740 1 --2- 172.21.15.42:0/2322077078 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561bb4f284d0 0x561bb52fdce0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f3a04031440 tx=0x7f3a040314a0 comp rx=0 tx=0).stop 2024-08-30T09:11:02.889 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.887+0000 7f3a18436740 1 -- 172.21.15.42:0/2322077078 shutdown_connections 2024-08-30T09:11:02.889 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.887+0000 7f3a18436740 1 --2- 172.21.15.42:0/2322077078 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x561bb4edd1c0 0x561bb5300cb0 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:02.889 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.887+0000 7f3a18436740 1 --2- 172.21.15.42:0/2322077078 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561bb4f491b0 0x561bb52fe850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:02.889 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.887+0000 7f3a18436740 1 --2- 172.21.15.42:0/2322077078 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561bb4f284d0 0x561bb52fdce0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:02.889 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.887+0000 7f3a18436740 1 --2- 172.21.15.42:0/2322077078 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561bb4f202f0 0x561bb52fd620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:02.889 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.887+0000 7f3a18436740 1 -- 172.21.15.42:0/2322077078 >> 172.21.15.42:0/2322077078 conn(0x561bb4f3d300 msgr2=0x561bb52f7380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:11:02.889 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.887+0000 7f39f27fc640 0 client.26957 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:11:02.890 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.888+0000 7f3a18436740 1 -- 172.21.15.42:0/2322077078 shutdown_connections 2024-08-30T09:11:02.890 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.888+0000 7f3a18436740 1 -- 172.21.15.42:0/2322077078 wait complete. 2024-08-30T09:11:02.890 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.888+0000 7f3a18436740 20 client.26957 trim_cache size 0 max 16384 2024-08-30T09:11:02.890 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-08-30T09:11:02.891 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2024-08-30T09:11:02.891 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-08-30T09:11:02.891 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-08-30T09:11:02.892 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.889+0000 7f3a114d6640 1 -- 172.21.15.42:0/527313641 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f39f80b4d40 msgr2=0x7f39f80b7250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:11:02.892 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.889+0000 7f3a114d6640 1 --2- 172.21.15.42:0/527313641 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f39f80b4d40 0x7f39f80b7250 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f3a04005f70 tx=0x7f3a0402a420 comp rx=0 tx=0).stop 2024-08-30T09:11:02.892 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.889+0000 7f3a114d6640 1 -- 172.21.15.42:0/527313641 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3a0c111d30 msgr2=0x7f3a0c11b8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:11:02.892 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.889+0000 7f3a114d6640 1 --2- 172.21.15.42:0/527313641 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3a0c111d30 0x7f3a0c11b8a0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f39f400eb10 tx=0x7f39f400c2d0 comp rx=0 tx=0).stop 2024-08-30T09:11:02.892 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.889+0000 7f3a114d6640 1 -- 172.21.15.42:0/527313641 shutdown_connections 2024-08-30T09:11:02.892 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.889+0000 7f3a114d6640 1 --2- 172.21.15.42:0/527313641 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f39f80b4d40 0x7f39f80b7250 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:02.892 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.889+0000 7f3a114d6640 1 --2- 172.21.15.42:0/527313641 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3a0c10ab40 0x7f3a0c11b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:02.892 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.889+0000 7f3a114d6640 1 --2- 172.21.15.42:0/527313641 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3a0c111d30 0x7f3a0c11b8a0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:02.892 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.889+0000 7f3a114d6640 1 --2- 172.21.15.42:0/527313641 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3a0c10a160 0x7f3a0c11caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:02.892 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.889+0000 7f3a114d6640 1 -- 172.21.15.42:0/527313641 >> 172.21.15.42:0/527313641 conn(0x7f3a0c0705b0 msgr2=0x7f3a0c10dd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:11:02.892 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.889+0000 7f3a114d6640 1 -- 172.21.15.42:0/527313641 shutdown_connections 2024-08-30T09:11:02.892 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:02.889+0000 7f3a114d6640 1 -- 172.21.15.42:0/527313641 wait complete. 2024-08-30T09:11:03.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.161+0000 7fbf10b48640 1 -- 172.21.15.42:0/1627481707 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbf0c107480 msgr2=0x7fbf0c107890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:11:03.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.161+0000 7fbf10b48640 1 --2- 172.21.15.42:0/1627481707 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbf0c107480 0x7fbf0c107890 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fbf04009be0 tx=0x7fbf0402a710 comp rx=0 tx=0).stop 2024-08-30T09:11:03.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.166+0000 7fbf10b48640 1 -- 172.21.15.42:0/1627481707 shutdown_connections 2024-08-30T09:11:03.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.166+0000 7fbf10b48640 1 --2- 172.21.15.42:0/1627481707 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbf0c107480 0x7fbf0c107890 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:03.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.166+0000 7fbf10b48640 1 --2- 172.21.15.42:0/1627481707 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbf0c107dd0 0x7fbf0c10e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:03.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.166+0000 7fbf10b48640 1 --2- 172.21.15.42:0/1627481707 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbf0c101b40 0x7fbf0c101f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:03.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.166+0000 7fbf10b48640 1 -- 172.21.15.42:0/1627481707 >> 172.21.15.42:0/1627481707 conn(0x7fbf0c0775a0 msgr2=0x7fbf0c0779e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:11:03.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.173+0000 7fbf10b48640 1 -- 172.21.15.42:0/1627481707 shutdown_connections 2024-08-30T09:11:03.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.173+0000 7fbf10b48640 1 -- 172.21.15.42:0/1627481707 wait complete. 2024-08-30T09:11:03.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.174+0000 7fbf10b48640 1 Processor -- start 2024-08-30T09:11:03.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.175+0000 7fbf10b48640 1 -- start start 2024-08-30T09:11:03.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.175+0000 7fbf10b48640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbf0c101b40 0x7fbf0c197f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:03.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.175+0000 7fbf10b48640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbf0c107480 0x7fbf0c1984c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:03.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.175+0000 7fbf10b48640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbf0c107dd0 0x7fbf0c198a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:03.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.175+0000 7fbf0a59c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbf0c101b40 0x7fbf0c197f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:03.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.175+0000 7fbf0a59c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbf0c101b40 0x7fbf0c197f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:47920/0 (socket says 172.21.15.42:47920) 2024-08-30T09:11:03.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.175+0000 7fbf09d9b640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbf0c107480 0x7fbf0c1984c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:03.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.176+0000 7fbf09d9b640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbf0c107480 0x7fbf0c1984c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:36432/0 (socket says 172.21.15.42:36432) 2024-08-30T09:11:03.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.176+0000 7fbf0a59c640 1 -- 172.21.15.42:0/537252809 learned_addr learned my addr 172.21.15.42:0/537252809 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:11:03.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.176+0000 7fbf10b48640 1 -- 172.21.15.42:0/537252809 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fbf0c111ae0 con 0x7fbf0c107dd0 2024-08-30T09:11:03.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.176+0000 7fbf10b48640 1 -- 172.21.15.42:0/537252809 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fbf0c111960 con 0x7fbf0c107480 2024-08-30T09:11:03.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.176+0000 7fbf10b48640 1 -- 172.21.15.42:0/537252809 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fbf0c111c60 con 0x7fbf0c101b40 2024-08-30T09:11:03.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.180+0000 7fbf0ad9d640 1 --2- 172.21.15.42:0/537252809 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbf0c107dd0 0x7fbf0c198a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:03.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.180+0000 7fbf09d9b640 1 -- 172.21.15.42:0/537252809 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbf0c101b40 msgr2=0x7fbf0c197f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:11:03.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.180+0000 7fbf09d9b640 1 --2- 172.21.15.42:0/537252809 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbf0c101b40 0x7fbf0c197f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:03.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.180+0000 7fbf09d9b640 1 -- 172.21.15.42:0/537252809 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbf0c107dd0 msgr2=0x7fbf0c198a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:11:03.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.180+0000 7fbf09d9b640 1 --2- 172.21.15.42:0/537252809 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbf0c107dd0 0x7fbf0c198a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:03.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.180+0000 7fbf09d9b640 1 -- 172.21.15.42:0/537252809 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf0c1ac770 con 0x7fbf0c107480 2024-08-30T09:11:03.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.180+0000 7fbf0a59c640 1 --2- 172.21.15.42:0/537252809 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbf0c101b40 0x7fbf0c197f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-30T09:11:03.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.181+0000 7fbf09d9b640 1 --2- 172.21.15.42:0/537252809 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbf0c107480 0x7fbf0c1984c0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fbf04009b30 tx=0x7fbf04030dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:11:03.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.186+0000 7fbefb7fe640 1 -- 172.21.15.42:0/537252809 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf04041020 con 0x7fbf0c107480 2024-08-30T09:11:03.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.186+0000 7fbefb7fe640 1 -- 172.21.15.42:0/537252809 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf040318f0 con 0x7fbf0c107480 2024-08-30T09:11:03.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.186+0000 7fbf10b48640 1 -- 172.21.15.42:0/537252809 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf0c1aca00 con 0x7fbf0c107480 2024-08-30T09:11:03.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.186+0000 7fbefb7fe640 1 -- 172.21.15.42:0/537252809 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf04036040 con 0x7fbf0c107480 2024-08-30T09:11:03.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.187+0000 7fbf10b48640 1 -- 172.21.15.42:0/537252809 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf0c1aceb0 con 0x7fbf0c107480 2024-08-30T09:11:03.195 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.193+0000 7fbf170bd740 1 Processor -- start 2024-08-30T09:11:03.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.195+0000 7fbefb7fe640 1 -- 172.21.15.42:0/537252809 <== mon.2 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7fbf04044020 con 0x7fbf0c107480 2024-08-30T09:11:03.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.196+0000 7fbf170bd740 1 -- start start 2024-08-30T09:11:03.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.196+0000 7fbf170bd740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ac6602b350 0x55ac663d1780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:03.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.196+0000 7fbf170bd740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ac66002490 0x55ac663d1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:03.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.196+0000 7fbf170bd740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ac6600a670 0x55ac663d2200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:03.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.196+0000 7fbf170bd740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55ac663475f0 con 0x55ac6600a670 2024-08-30T09:11:03.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.196+0000 7fbf170bd740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55ac66336b80 con 0x55ac66002490 2024-08-30T09:11:03.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.196+0000 7fbf170bd740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55ac6633ab30 con 0x55ac6602b350 2024-08-30T09:11:03.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.197+0000 7fbefb7fe640 1 --2- 172.21.15.42:0/537252809 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fbef40b4d40 0x7fbef40b7250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:03.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.198+0000 7fbf09d9b640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ac6600a670 0x55ac663d2200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:03.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.198+0000 7fbf09d9b640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ac6600a670 0x55ac663d2200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42680/0 (socket says 172.21.15.42:42680) 2024-08-30T09:11:03.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.198+0000 7fbf09d9b640 1 -- 172.21.15.42:0/3925185598 learned_addr learned my addr 172.21.15.42:0/3925185598 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:11:03.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.198+0000 7fbefb7fe640 1 -- 172.21.15.42:0/537252809 <== mon.2 v2:172.21.15.167:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7fbf04105e80 con 0x7fbf0c107480 2024-08-30T09:11:03.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.202+0000 7fbf0ad9d640 1 --2- 172.21.15.42:0/3925185598 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ac6602b350 0x55ac663d1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:03.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.202+0000 7fbf0a59c640 1 --2- 172.21.15.42:0/3925185598 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ac66002490 0x55ac663d1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:03.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.202+0000 7fbf0ad9d640 1 --2- 172.21.15.42:0/537252809 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fbef40b4d40 0x7fbef40b7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:03.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.203+0000 7fbf09d9b640 1 -- 172.21.15.42:0/3925185598 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ac6602b350 msgr2=0x55ac663d1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:11:03.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.203+0000 7fbf09d9b640 1 --2- 172.21.15.42:0/3925185598 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ac6602b350 0x55ac663d1780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:03.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.203+0000 7fbf09d9b640 1 -- 172.21.15.42:0/3925185598 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ac66002490 msgr2=0x55ac663d1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:11:03.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.203+0000 7fbf09d9b640 1 --2- 172.21.15.42:0/3925185598 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ac66002490 0x55ac663d1cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:03.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.203+0000 7fbf09d9b640 1 -- 172.21.15.42:0/3925185598 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac66096930 con 0x55ac6600a670 2024-08-30T09:11:03.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.209+0000 7fbf0ad9d640 1 --2- 172.21.15.42:0/537252809 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fbef40b4d40 0x7fbef40b7250 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fbf00009510 tx=0x7fbf00009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:11:03.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.210+0000 7fbf09d9b640 1 --2- 172.21.15.42:0/3925185598 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ac6600a670 0x55ac663d2200 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fbf041035e0 tx=0x7fbf04005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43fcddcd8c99923d server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:11:03.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.214+0000 7fbef8ff9640 1 -- 172.21.15.42:0/3925185598 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf040d5120 con 0x55ac6600a670 2024-08-30T09:11:03.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.214+0000 7fbef8ff9640 1 -- 172.21.15.42:0/3925185598 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf040cbfd0 con 0x55ac6600a670 2024-08-30T09:11:03.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.214+0000 7fbef8ff9640 1 -- 172.21.15.42:0/3925185598 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf040046f0 con 0x55ac6600a670 2024-08-30T09:11:03.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.216+0000 7fbf170bd740 1 -- 172.21.15.42:0/3925185598 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ac6600a670 msgr2=0x55ac663d2200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:11:03.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.216+0000 7fbf170bd740 1 --2- 172.21.15.42:0/3925185598 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ac6600a670 0x55ac663d2200 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fbf041035e0 tx=0x7fbf04005fd0 comp rx=0 tx=0).stop 2024-08-30T09:11:03.221 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.219+0000 7fbf170bd740 1 -- 172.21.15.42:0/3925185598 shutdown_connections 2024-08-30T09:11:03.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.219+0000 7fbf170bd740 1 --2- 172.21.15.42:0/3925185598 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ac6600a670 0x55ac663d2200 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:03.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.219+0000 7fbf170bd740 1 --2- 172.21.15.42:0/3925185598 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ac66002490 0x55ac663d1cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:03.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.219+0000 7fbf170bd740 1 --2- 172.21.15.42:0/3925185598 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ac6602b350 0x55ac663d1780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:03.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.219+0000 7fbf170bd740 1 -- 172.21.15.42:0/3925185598 >> 172.21.15.42:0/3925185598 conn(0x55ac6601f740 msgr2=0x55ac663cf670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:11:03.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.225+0000 7fbf170bd740 1 -- 172.21.15.42:0/3925185598 shutdown_connections 2024-08-30T09:11:03.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.228+0000 7fbf170bd740 1 -- 172.21.15.42:0/3925185598 wait complete. 2024-08-30T09:11:03.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.229+0000 7fbf170bd740 1 -- start start 2024-08-30T09:11:03.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.229+0000 7fbf170bd740 1 Processor -- start 2024-08-30T09:11:03.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.229+0000 7fbf170bd740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ac66002490 0x55ac663dcf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:03.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.229+0000 7fbf170bd740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ac6600a670 0x55ac663dd500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:03.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.229+0000 7fbf170bd740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ac6602b350 0x55ac663f9f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:03.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.229+0000 7fbf170bd740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55ac66199890 con 0x55ac6600a670 2024-08-30T09:11:03.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.229+0000 7fbf170bd740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55ac662bd790 con 0x55ac6602b350 2024-08-30T09:11:03.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.229+0000 7fbf170bd740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55ac663f0d30 con 0x55ac66002490 2024-08-30T09:11:03.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.230+0000 7fbf170bd740 10 client.0 _ll_register_callbacks cb 0x55ac66042ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:11:03.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.230+0000 7fbf0ad9d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ac6602b350 0x55ac663f9f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:03.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.230+0000 7fbf0ad9d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ac6602b350 0x55ac663f9f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:36450/0 (socket says 172.21.15.42:36450) 2024-08-30T09:11:03.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.230+0000 7fbf0ad9d640 1 -- 172.21.15.42:0/2545365568 learned_addr learned my addr 172.21.15.42:0/2545365568 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:11:03.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.230+0000 7fbf09d9b640 1 --2- 172.21.15.42:0/2545365568 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ac6600a670 0x55ac663dd500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:03.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.231+0000 7fbf0a59c640 1 --2- 172.21.15.42:0/2545365568 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ac66002490 0x55ac663dcf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:03.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.232+0000 7fbf09d9b640 1 -- 172.21.15.42:0/2545365568 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ac66002490 msgr2=0x55ac663dcf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:11:03.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.232+0000 7fbf09d9b640 1 --2- 172.21.15.42:0/2545365568 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ac66002490 0x55ac663dcf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:03.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.232+0000 7fbf09d9b640 1 -- 172.21.15.42:0/2545365568 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ac6602b350 msgr2=0x55ac663f9f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:11:03.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.232+0000 7fbf09d9b640 1 --2- 172.21.15.42:0/2545365568 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ac6602b350 0x55ac663f9f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:03.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.232+0000 7fbf09d9b640 1 -- 172.21.15.42:0/2545365568 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac662f4f30 con 0x55ac6600a670 2024-08-30T09:11:03.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.234+0000 7fbf0a59c640 1 --2- 172.21.15.42:0/2545365568 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ac66002490 0x55ac663dcf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-30T09:11:03.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.239+0000 7fbf09d9b640 1 --2- 172.21.15.42:0/2545365568 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ac6600a670 0x55ac663dd500 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fbf040cc450 tx=0x7fbf0402ac80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f38029de787cc1b9 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:11:03.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.239+0000 7fbef1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-08-30T09:11:03.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.241+0000 7fbef1ffb640 1 -- 172.21.15.42:0/2545365568 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf040f3070 con 0x55ac6600a670 2024-08-30T09:11:03.246 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.241+0000 7fbf170bd740 1 -- 172.21.15.42:0/2545365568 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbf040cbfd0 con 0x55ac6600a670 2024-08-30T09:11:03.246 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.241+0000 7fbef1ffb640 1 -- 172.21.15.42:0/2545365568 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7fbf040ef0f0 con 0x55ac6600a670 2024-08-30T09:11:03.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.247+0000 7fbef1ffb640 1 -- 172.21.15.42:0/2545365568 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fbf040d5120 con 0x55ac6600a670 2024-08-30T09:11:03.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.248+0000 7fbf170bd740 10 client.17154 fetch_fsmap learned FSMap version 41 2024-08-30T09:11:03.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.248+0000 7fbf170bd740 1 -- 172.21.15.42:0/2545365568 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55ac6640db10 con 0x55ac6600a670 2024-08-30T09:11:03.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.252+0000 7fbef1ffb640 1 -- 172.21.15.42:0/2545365568 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf040cbfd0 con 0x55ac6600a670 2024-08-30T09:11:03.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.252+0000 7fbef1ffb640 1 -- 172.21.15.42:0/2545365568 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 41) ==== 8362+0+0 (secure 0 0 0) 0x7fbf0407c080 con 0x55ac6600a670 2024-08-30T09:11:03.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.253+0000 7fbf170bd740 10 client.17154 fetch_fsmap finished waiting for FSMap version 41 2024-08-30T09:11:03.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.253+0000 7fbf170bd740 10 client.17154 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-30T09:11:03.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.253+0000 7fbf170bd740 20 client.17154 populate_metadata read hostname 'smithi042' 2024-08-30T09:11:03.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.253+0000 7fbf170bd740 1 --2- 172.21.15.42:0/2545365568 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55ac65fe9cd0 0x55ac66410e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:03.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.253+0000 7fbf170bd740 4 client.17154 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-30T09:11:03.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.253+0000 7fbf170bd740 1 -- 172.21.15.42:0/2545365568 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ac66349540 con 0x55ac65fe9cd0 2024-08-30T09:11:03.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.255+0000 7fbf0a59c640 1 --2- 172.21.15.42:0/2545365568 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55ac65fe9cd0 0x55ac66410e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:03.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.256+0000 7fbf0a59c640 1 --2- 172.21.15.42:0/2545365568 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55ac65fe9cd0 0x55ac66410e70 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ef4e90c7e0cff96b server_cookie=6b56596fb6bb9f13 in_seq=0 out_seq=0 2024-08-30T09:11:03.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.258+0000 7fbef1ffb640 10 client.17154 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:11:03.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.258+0000 7fbef1ffb640 1 -- 172.21.15.42:0/2545365568 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55ac66349540 con 0x55ac65fe9cd0 2024-08-30T09:11:03.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.258+0000 7fbef1ffb640 10 client.17154 handle_command_reply: tid=0 2024-08-30T09:11:03.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.258+0000 7fbf170bd740 1 client.17154 shutdown 2024-08-30T09:11:03.266 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.263+0000 7fbf170bd740 1 -- 172.21.15.42:0/2545365568 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ac6600a670 msgr2=0x55ac663dd500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:11:03.266 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.263+0000 7fbf170bd740 1 --2- 172.21.15.42:0/2545365568 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ac6600a670 0x55ac663dd500 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fbf040cc450 tx=0x7fbf0402ac80 comp rx=0 tx=0).stop 2024-08-30T09:11:03.266 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.263+0000 7fbf170bd740 1 -- 172.21.15.42:0/2545365568 shutdown_connections 2024-08-30T09:11:03.266 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.263+0000 7fbf170bd740 1 --2- 172.21.15.42:0/2545365568 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55ac65fe9cd0 0x55ac66410e70 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:03.266 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.263+0000 7fbf170bd740 1 --2- 172.21.15.42:0/2545365568 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ac6602b350 0x55ac663f9f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:03.266 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.263+0000 7fbf170bd740 1 --2- 172.21.15.42:0/2545365568 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ac6600a670 0x55ac663dd500 secure :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fbf040cc450 tx=0x7fbf0402ac80 comp rx=0 tx=0).stop 2024-08-30T09:11:03.266 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.263+0000 7fbf170bd740 1 --2- 172.21.15.42:0/2545365568 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ac66002490 0x55ac663dcf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:03.266 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.263+0000 7fbf170bd740 1 -- 172.21.15.42:0/2545365568 >> 172.21.15.42:0/2545365568 conn(0x55ac6601f740 msgr2=0x55ac663d4910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:11:03.266 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.263+0000 7fbef1ffb640 0 client.17154 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:11:03.266 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.264+0000 7fbf170bd740 1 -- 172.21.15.42:0/2545365568 shutdown_connections 2024-08-30T09:11:03.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.268+0000 7fbf170bd740 1 -- 172.21.15.42:0/2545365568 wait complete. 2024-08-30T09:11:03.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.268+0000 7fbf170bd740 20 client.17154 trim_cache size 0 max 16384 2024-08-30T09:11:03.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.305+0000 7fbf170bd740 1 Processor -- start 2024-08-30T09:11:03.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.305+0000 7fbf170bd740 1 -- start start 2024-08-30T09:11:03.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.305+0000 7fbf170bd740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ac66002490 0x55ac663d8730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:03.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.305+0000 7fbf170bd740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ac6602b350 0x55ac663d2ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:03.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.305+0000 7fbf170bd740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ac65fbf080 0x55ac663d9160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:03.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.305+0000 7fbf170bd740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fbf0c1a3e50 con 0x55ac66002490 2024-08-30T09:11:03.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.305+0000 7fbf170bd740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fbf0c1a1700 con 0x55ac65fbf080 2024-08-30T09:11:03.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.305+0000 7fbf170bd740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fbf0c074a40 con 0x55ac6602b350 2024-08-30T09:11:03.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.306+0000 7fbf0ad9d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ac65fbf080 0x55ac663d9160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:03.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.306+0000 7fbf0ad9d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ac65fbf080 0x55ac663d9160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:36462/0 (socket says 172.21.15.42:36462) 2024-08-30T09:11:03.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.306+0000 7fbf0ad9d640 1 -- 172.21.15.42:0/2293386225 learned_addr learned my addr 172.21.15.42:0/2293386225 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:11:03.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.307+0000 7fbf0a59c640 1 --2- 172.21.15.42:0/2293386225 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ac66002490 0x55ac663d8730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:03.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.309+0000 7fbf0ad9d640 1 -- 172.21.15.42:0/2293386225 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ac6602b350 msgr2=0x55ac663d2ec0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-30T09:11:03.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.309+0000 7fbf0ad9d640 1 --2- 172.21.15.42:0/2293386225 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ac6602b350 0x55ac663d2ec0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:03.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.309+0000 7fbf0ad9d640 1 -- 172.21.15.42:0/2293386225 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ac66002490 msgr2=0x55ac663d8730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:11:03.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.309+0000 7fbf0ad9d640 1 --2- 172.21.15.42:0/2293386225 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ac66002490 0x55ac663d8730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:03.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.309+0000 7fbf0ad9d640 1 -- 172.21.15.42:0/2293386225 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac663dbee0 con 0x55ac65fbf080 2024-08-30T09:11:03.313 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.310+0000 7fbf0ad9d640 1 --2- 172.21.15.42:0/2293386225 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ac65fbf080 0x55ac663d9160 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fbf00012f90 tx=0x7fbf00008000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=be38eb1d1cc6df57 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:11:03.313 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.310+0000 7fbef3fff640 1 -- 172.21.15.42:0/2293386225 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf0000dcb0 con 0x55ac65fbf080 2024-08-30T09:11:03.313 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.310+0000 7fbef3fff640 1 -- 172.21.15.42:0/2293386225 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf0000de50 con 0x55ac65fbf080 2024-08-30T09:11:03.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.311+0000 7fbf170bd740 1 -- 172.21.15.42:0/2293386225 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ac65fbf080 msgr2=0x55ac663d9160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:11:03.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.311+0000 7fbf170bd740 1 --2- 172.21.15.42:0/2293386225 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ac65fbf080 0x55ac663d9160 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fbf00012f90 tx=0x7fbf00008000 comp rx=0 tx=0).stop 2024-08-30T09:11:03.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.312+0000 7fbf170bd740 1 -- 172.21.15.42:0/2293386225 shutdown_connections 2024-08-30T09:11:03.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.312+0000 7fbf170bd740 1 --2- 172.21.15.42:0/2293386225 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ac6602b350 0x55ac663d2ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:03.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.312+0000 7fbf170bd740 1 --2- 172.21.15.42:0/2293386225 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ac65fbf080 0x55ac663d9160 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:03.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.312+0000 7fbf170bd740 1 --2- 172.21.15.42:0/2293386225 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ac66002490 0x55ac663d8730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:03.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.312+0000 7fbf170bd740 1 -- 172.21.15.42:0/2293386225 >> 172.21.15.42:0/2293386225 conn(0x55ac6601f740 msgr2=0x55ac663d27b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:11:03.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.321+0000 7fbf170bd740 1 -- 172.21.15.42:0/2293386225 shutdown_connections 2024-08-30T09:11:03.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.324+0000 7fbf170bd740 1 -- 172.21.15.42:0/2293386225 wait complete. 2024-08-30T09:11:03.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.324+0000 7fbf170bd740 1 -- start start 2024-08-30T09:11:03.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.325+0000 7fbf170bd740 1 Processor -- start 2024-08-30T09:11:03.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.328+0000 7fbf170bd740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ac65fbf080 0x55ac66414640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:03.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.328+0000 7fbf170bd740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ac6602b350 0x55ac66412b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:03.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.328+0000 7fbf170bd740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ac66413060 0x55ac66413470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:03.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.328+0000 7fbf170bd740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fbf0c0750b0 con 0x55ac65fbf080 2024-08-30T09:11:03.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.328+0000 7fbf170bd740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fbf0c1a1030 con 0x55ac6602b350 2024-08-30T09:11:03.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.328+0000 7fbf170bd740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fbf0c1a0290 con 0x55ac66413060 2024-08-30T09:11:03.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.328+0000 7fbf170bd740 10 client.0 _ll_register_callbacks cb 0x7fbf00009b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:11:03.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.330+0000 7fbf0ad9d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ac66413060 0x55ac66413470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:03.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.330+0000 7fbf09d9b640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ac6602b350 0x55ac66412b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:03.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.330+0000 7fbf0ad9d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ac66413060 0x55ac66413470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:47968/0 (socket says 172.21.15.42:47968) 2024-08-30T09:11:03.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.330+0000 7fbf0ad9d640 1 -- 172.21.15.42:0/1083290463 learned_addr learned my addr 172.21.15.42:0/1083290463 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:11:03.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.331+0000 7fbf0a59c640 1 --2- 172.21.15.42:0/1083290463 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ac65fbf080 0x55ac66414640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:03.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.332+0000 7fbf09d9b640 1 -- 172.21.15.42:0/1083290463 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ac66413060 msgr2=0x55ac66413470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:11:03.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.332+0000 7fbf09d9b640 1 --2- 172.21.15.42:0/1083290463 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ac66413060 0x55ac66413470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:03.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.332+0000 7fbf09d9b640 1 -- 172.21.15.42:0/1083290463 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ac65fbf080 msgr2=0x55ac66414640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:11:03.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.332+0000 7fbf09d9b640 1 --2- 172.21.15.42:0/1083290463 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ac65fbf080 0x55ac66414640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:03.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.332+0000 7fbf09d9b640 1 -- 172.21.15.42:0/1083290463 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac6647b680 con 0x55ac6602b350 2024-08-30T09:11:03.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.338+0000 7fbf09d9b640 1 --2- 172.21.15.42:0/1083290463 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ac6602b350 0x55ac66412b20 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fbf0400b0b0 tx=0x7fbf040ef260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8075f57a7eb8f439 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:11:03.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.338+0000 7fbef1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-30T09:11:03.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.339+0000 7fbef1ffb640 1 -- 172.21.15.42:0/1083290463 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf040f3040 con 0x55ac6602b350 2024-08-30T09:11:03.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.339+0000 7fbef1ffb640 1 -- 172.21.15.42:0/1083290463 <== mon.2 v2:172.21.15.167:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7fbf040d5560 con 0x55ac6602b350 2024-08-30T09:11:03.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.340+0000 7fbf170bd740 1 -- 172.21.15.42:0/1083290463 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbf0000de50 con 0x55ac6602b350 2024-08-30T09:11:03.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.340+0000 7fbef1ffb640 1 -- 172.21.15.42:0/1083290463 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fbf040cbe10 con 0x55ac6602b350 2024-08-30T09:11:03.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.345+0000 7fbf170bd740 10 client.26975 fetch_fsmap learned FSMap version 41 2024-08-30T09:11:03.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.345+0000 7fbf170bd740 1 -- 172.21.15.42:0/1083290463 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55ac6647b680 con 0x55ac6602b350 2024-08-30T09:11:03.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.352+0000 7fbef1ffb640 1 -- 172.21.15.42:0/1083290463 <== mon.2 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf0000de50 con 0x55ac6602b350 2024-08-30T09:11:03.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.353+0000 7fbef1ffb640 1 -- 172.21.15.42:0/1083290463 <== mon.2 v2:172.21.15.167:3300/0 5 ==== fsmap(e 41) ==== 8362+0+0 (secure 0 0 0) 0x7fbf040cb720 con 0x55ac6602b350 2024-08-30T09:11:03.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.359+0000 7fbf170bd740 10 client.26975 fetch_fsmap finished waiting for FSMap version 41 2024-08-30T09:11:03.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.359+0000 7fbf170bd740 10 client.26975 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-30T09:11:03.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.359+0000 7fbf170bd740 20 client.26975 populate_metadata read hostname 'smithi042' 2024-08-30T09:11:03.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.359+0000 7fbf170bd740 1 --2- 172.21.15.42:0/1083290463 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55ac6647caf0 0x55ac6647ef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:03.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.359+0000 7fbf170bd740 4 client.26975 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-08-30T09:11:03.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.359+0000 7fbf170bd740 1 -- 172.21.15.42:0/1083290463 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55ac663dd890 con 0x55ac6647caf0 2024-08-30T09:11:03.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.362+0000 7fbf0a59c640 1 --2- 172.21.15.42:0/1083290463 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55ac6647caf0 0x55ac6647ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:03.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.363+0000 7fbf0a59c640 1 --2- 172.21.15.42:0/1083290463 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55ac6647caf0 0x55ac6647ef10 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e65363e6c03d9285 server_cookie=41e3331bced522a1 in_seq=0 out_seq=0 2024-08-30T09:11:03.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.363+0000 7fbef1ffb640 10 client.26975 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:11:03.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.364+0000 7fbef1ffb640 1 -- 172.21.15.42:0/1083290463 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55ac663dd890 con 0x55ac6647caf0 2024-08-30T09:11:03.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.364+0000 7fbef1ffb640 10 client.26975 handle_command_reply: tid=0 2024-08-30T09:11:03.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.365+0000 7fbf170bd740 1 client.26975 shutdown 2024-08-30T09:11:03.368 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.365+0000 7fbf170bd740 1 -- 172.21.15.42:0/1083290463 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ac6602b350 msgr2=0x55ac66412b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:11:03.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.365+0000 7fbf170bd740 1 --2- 172.21.15.42:0/1083290463 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ac6602b350 0x55ac66412b20 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fbf0400b0b0 tx=0x7fbf040ef260 comp rx=0 tx=0).stop 2024-08-30T09:11:03.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.365+0000 7fbf170bd740 1 -- 172.21.15.42:0/1083290463 shutdown_connections 2024-08-30T09:11:03.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.365+0000 7fbf170bd740 1 --2- 172.21.15.42:0/1083290463 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55ac6647caf0 0x55ac6647ef10 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:03.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.365+0000 7fbf170bd740 1 --2- 172.21.15.42:0/1083290463 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ac66413060 0x55ac66413470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:03.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.365+0000 7fbf170bd740 1 --2- 172.21.15.42:0/1083290463 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ac6602b350 0x55ac66412b20 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:03.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.365+0000 7fbf170bd740 1 --2- 172.21.15.42:0/1083290463 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ac65fbf080 0x55ac66414640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:03.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.365+0000 7fbef1ffb640 0 client.26975 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:11:03.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.365+0000 7fbf170bd740 1 -- 172.21.15.42:0/1083290463 >> 172.21.15.42:0/1083290463 conn(0x55ac6601f740 msgr2=0x55ac663ded60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:11:03.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.369+0000 7fbf170bd740 1 -- 172.21.15.42:0/1083290463 shutdown_connections 2024-08-30T09:11:03.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.369+0000 7fbf170bd740 1 -- 172.21.15.42:0/1083290463 wait complete. 2024-08-30T09:11:03.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.369+0000 7fbf170bd740 20 client.26975 trim_cache size 0 max 16384 2024-08-30T09:11:03.371 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-08-30T09:11:03.371 INFO:teuthology.orchestra.run.smithi042.stdout: "return_code": 0, 2024-08-30T09:11:03.371 INFO:teuthology.orchestra.run.smithi042.stdout: "scrub_tag": "b62c4ffc-9387-4677-b9ba-1e94ba308f17", 2024-08-30T09:11:03.371 INFO:teuthology.orchestra.run.smithi042.stdout: "mode": "asynchronous" 2024-08-30T09:11:03.372 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-08-30T09:11:03.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.370+0000 7fbf10b48640 1 -- 172.21.15.42:0/537252809 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fbef40b4d40 msgr2=0x7fbef40b7250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:11:03.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.370+0000 7fbf10b48640 1 --2- 172.21.15.42:0/537252809 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fbef40b4d40 0x7fbef40b7250 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fbf00009510 tx=0x7fbf00009260 comp rx=0 tx=0).stop 2024-08-30T09:11:03.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.370+0000 7fbf10b48640 1 -- 172.21.15.42:0/537252809 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbf0c107480 msgr2=0x7fbf0c1984c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:11:03.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.370+0000 7fbf10b48640 1 --2- 172.21.15.42:0/537252809 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbf0c107480 0x7fbf0c1984c0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fbf04009b30 tx=0x7fbf04030dc0 comp rx=0 tx=0).stop 2024-08-30T09:11:03.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.371+0000 7fbf10b48640 1 -- 172.21.15.42:0/537252809 shutdown_connections 2024-08-30T09:11:03.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.371+0000 7fbf10b48640 1 --2- 172.21.15.42:0/537252809 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fbef40b4d40 0x7fbef40b7250 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:03.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.371+0000 7fbf10b48640 1 --2- 172.21.15.42:0/537252809 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbf0c107dd0 0x7fbf0c198a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:03.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.371+0000 7fbf10b48640 1 --2- 172.21.15.42:0/537252809 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbf0c107480 0x7fbf0c1984c0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:03.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.371+0000 7fbf10b48640 1 --2- 172.21.15.42:0/537252809 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbf0c101b40 0x7fbf0c197f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:03.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.371+0000 7fbf10b48640 1 -- 172.21.15.42:0/537252809 >> 172.21.15.42:0/537252809 conn(0x7fbf0c0775a0 msgr2=0x7fbf0c103310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:11:03.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.373+0000 7fbf10b48640 1 -- 172.21.15.42:0/537252809 shutdown_connections 2024-08-30T09:11:03.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:03.373+0000 7fbf10b48640 1 -- 172.21.15.42:0/537252809 wait complete. 2024-08-30T09:11:04.135 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-30T09:11:04.588 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T09:11:04.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:11:04 smithi190 ceph-mon[22001]: pgmap v682: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 8.4 MiB/s wr, 576 op/s 2024-08-30T09:11:04.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:11:04 smithi190 ceph-mon[22001]: scrub queued for path: / 2024-08-30T09:11:04.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:11:04 smithi190 ceph-mon[22001]: scrub summary: idle+waiting paths [/] 2024-08-30T09:11:04.687 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:11:04 smithi190 ceph-mon[22001]: scrub summary: active paths [/] 2024-08-30T09:11:04.687 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:11:04 smithi190 ceph-mon[22001]: scrub summary: active paths [/] 2024-08-30T09:11:04.743 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:11:04 smithi042 ceph-mon[18652]: pgmap v682: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 8.4 MiB/s wr, 576 op/s 2024-08-30T09:11:04.743 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:11:04 smithi042 ceph-mon[18652]: scrub queued for path: / 2024-08-30T09:11:04.743 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:11:04 smithi042 ceph-mon[18652]: scrub summary: idle+waiting paths [/] 2024-08-30T09:11:04.743 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:11:04 smithi042 ceph-mon[18652]: scrub summary: active paths [/] 2024-08-30T09:11:04.743 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:11:04 smithi042 ceph-mon[18652]: scrub summary: active paths [/] 2024-08-30T09:11:04.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:11:04 smithi167 ceph-mon[22063]: pgmap v682: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 8.4 MiB/s wr, 576 op/s 2024-08-30T09:11:04.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:11:04 smithi167 ceph-mon[22063]: scrub queued for path: / 2024-08-30T09:11:04.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:11:04 smithi167 ceph-mon[22063]: scrub summary: idle+waiting paths [/] 2024-08-30T09:11:04.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:11:04 smithi167 ceph-mon[22063]: scrub summary: active paths [/] 2024-08-30T09:11:04.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:11:04 smithi167 ceph-mon[22063]: scrub summary: active paths [/] 2024-08-30T09:11:05.441 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.437+0000 7f2cabfff640 1 -- 172.21.15.42:0/3439838184 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2cac10d990 msgr2=0x7f2cac10dda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:11:05.441 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.437+0000 7f2cabfff640 1 --2- 172.21.15.42:0/3439838184 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2cac10d990 0x7f2cac10dda0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f2ca4009ca0 tx=0x7f2ca402a8f0 comp rx=0 tx=0).stop 2024-08-30T09:11:05.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.439+0000 7f2cabfff640 1 -- 172.21.15.42:0/3439838184 shutdown_connections 2024-08-30T09:11:05.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.439+0000 7f2cabfff640 1 --2- 172.21.15.42:0/3439838184 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2cac11c940 0x7f2cac11ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:05.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.439+0000 7f2cabfff640 1 --2- 172.21.15.42:0/3439838184 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2cac10d990 0x7f2cac10dda0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:05.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.439+0000 7f2cabfff640 1 --2- 172.21.15.42:0/3439838184 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2cac111a70 0x7f2cac10d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:05.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.439+0000 7f2cabfff640 1 -- 172.21.15.42:0/3439838184 >> 172.21.15.42:0/3439838184 conn(0x7f2cac06e790 msgr2=0x7f2cac06ebd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:11:05.445 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.441+0000 7f2cabfff640 1 -- 172.21.15.42:0/3439838184 shutdown_connections 2024-08-30T09:11:05.446 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.443+0000 7f2cabfff640 1 -- 172.21.15.42:0/3439838184 wait complete. 2024-08-30T09:11:05.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.444+0000 7f2cabfff640 1 Processor -- start 2024-08-30T09:11:05.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.446+0000 7f2cabfff640 1 -- start start 2024-08-30T09:11:05.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.447+0000 7f2cabfff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2cac10d990 0x7f2cac1a8f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:05.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.447+0000 7f2cabfff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2cac111a70 0x7f2cac1a94a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:05.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.447+0000 7f2cabfff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2cac11c940 0x7f2cac1ae740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:05.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.447+0000 7f2caaffd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2cac10d990 0x7f2cac1a8f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:05.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.447+0000 7f2caaffd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2cac10d990 0x7f2cac1a8f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:36480/0 (socket says 172.21.15.42:36480) 2024-08-30T09:11:05.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.447+0000 7f2caaffd640 1 -- 172.21.15.42:0/639255858 learned_addr learned my addr 172.21.15.42:0/639255858 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:11:05.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.447+0000 7f2cabfff640 1 -- 172.21.15.42:0/639255858 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f2cac122510 con 0x7f2cac11c940 2024-08-30T09:11:05.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.447+0000 7f2cabfff640 1 -- 172.21.15.42:0/639255858 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f2cac122390 con 0x7f2cac10d990 2024-08-30T09:11:05.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.447+0000 7f2cabfff640 1 -- 172.21.15.42:0/639255858 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f2cac122690 con 0x7f2cac111a70 2024-08-30T09:11:05.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.452+0000 7f2caa7fc640 1 --2- 172.21.15.42:0/639255858 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2cac111a70 0x7f2cac1a94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:05.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.453+0000 7f2caaffd640 1 -- 172.21.15.42:0/639255858 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2cac111a70 msgr2=0x7f2cac1a94a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:11:05.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.453+0000 7f2caaffd640 1 --2- 172.21.15.42:0/639255858 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2cac111a70 0x7f2cac1a94a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:05.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.453+0000 7f2caaffd640 1 -- 172.21.15.42:0/639255858 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2cac11c940 msgr2=0x7f2cac1ae740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:11:05.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.453+0000 7f2caaffd640 1 --2- 172.21.15.42:0/639255858 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2cac11c940 0x7f2cac1ae740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:05.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.453+0000 7f2caaffd640 1 -- 172.21.15.42:0/639255858 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cac1aee70 con 0x7f2cac10d990 2024-08-30T09:11:05.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.461+0000 7f2caaffd640 1 --2- 172.21.15.42:0/639255858 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2cac10d990 0x7f2cac1a8f00 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f2c9c00d770 tx=0x7f2c9c00dc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:11:05.465 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.462+0000 7f2cb0825640 1 -- 172.21.15.42:0/639255858 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c9c017070 con 0x7f2cac10d990 2024-08-30T09:11:05.465 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.462+0000 7f2cb0825640 1 -- 172.21.15.42:0/639255858 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c9c009c30 con 0x7f2cac10d990 2024-08-30T09:11:05.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.464+0000 7f2cb0825640 1 -- 172.21.15.42:0/639255858 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c9c003cc0 con 0x7f2cac10d990 2024-08-30T09:11:05.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.464+0000 7f2cabfff640 1 -- 172.21.15.42:0/639255858 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2cac1af160 con 0x7f2cac10d990 2024-08-30T09:11:05.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.464+0000 7f2cabfff640 1 -- 172.21.15.42:0/639255858 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2cac1b58c0 con 0x7f2cac10d990 2024-08-30T09:11:05.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.471+0000 7f2cb7d9c740 1 Processor -- start 2024-08-30T09:11:05.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.471+0000 7f2cb7d9c740 1 -- start start 2024-08-30T09:11:05.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.471+0000 7f2cb7d9c740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5577642e5070 0x55776468a460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:05.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.471+0000 7f2cb7d9c740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5577642bc1b0 0x55776468a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:05.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.471+0000 7f2cb7d9c740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5577642c4390 0x55776468aee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:05.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.471+0000 7f2cb7d9c740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x557764601630 con 0x5577642c4390 2024-08-30T09:11:05.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.471+0000 7f2cb7d9c740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5577645f0bc0 con 0x5577642bc1b0 2024-08-30T09:11:05.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.471+0000 7f2cb7d9c740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5577645f4b70 con 0x5577642e5070 2024-08-30T09:11:05.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.472+0000 7f2caa7fc640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5577642c4390 0x55776468aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:05.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.472+0000 7f2caa7fc640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5577642c4390 0x55776468aee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42740/0 (socket says 172.21.15.42:42740) 2024-08-30T09:11:05.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.472+0000 7f2cab7fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5577642e5070 0x55776468a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:05.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.472+0000 7f2cab7fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5577642e5070 0x55776468a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:47996/0 (socket says 172.21.15.42:47996) 2024-08-30T09:11:05.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.472+0000 7f2caa7fc640 1 -- 172.21.15.42:0/2328429633 learned_addr learned my addr 172.21.15.42:0/2328429633 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:11:05.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.473+0000 7f2cab7fe640 1 -- 172.21.15.42:0/2328429633 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5577642bc1b0 msgr2=0x55776468a9a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-30T09:11:05.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.473+0000 7f2cab7fe640 1 --2- 172.21.15.42:0/2328429633 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5577642bc1b0 0x55776468a9a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:05.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.473+0000 7f2cab7fe640 1 -- 172.21.15.42:0/2328429633 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5577642c4390 msgr2=0x55776468aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:11:05.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.473+0000 7f2cab7fe640 1 --2- 172.21.15.42:0/2328429633 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5577642c4390 0x55776468aee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:05.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.473+0000 7f2cab7fe640 1 -- 172.21.15.42:0/2328429633 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557764350650 con 0x5577642e5070 2024-08-30T09:11:05.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.474+0000 7f2caa7fc640 1 --2- 172.21.15.42:0/2328429633 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5577642c4390 0x55776468aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-30T09:11:05.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.475+0000 7f2cb0825640 1 -- 172.21.15.42:0/639255858 <== mon.2 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7f2c9c003e60 con 0x7f2cac10d990 2024-08-30T09:11:05.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.476+0000 7f2cb0825640 1 --2- 172.21.15.42:0/639255858 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f2c940b4ae0 0x7f2c940b6ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:05.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.476+0000 7f2cab7fe640 1 --2- 172.21.15.42:0/2328429633 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5577642e5070 0x55776468a460 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f2ca00096a0 tx=0x7f2ca0009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=33ca15326143b085 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:11:05.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.476+0000 7f2cb0825640 1 -- 172.21.15.42:0/639255858 <== mon.2 v2:172.21.15.167:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f2c9c0a7ae0 con 0x7f2cac10d990 2024-08-30T09:11:05.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.477+0000 7f2c91ffb640 1 -- 172.21.15.42:0/2328429633 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ca0015020 con 0x5577642e5070 2024-08-30T09:11:05.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.477+0000 7f2c91ffb640 1 -- 172.21.15.42:0/2328429633 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2ca000bd10 con 0x5577642e5070 2024-08-30T09:11:05.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.477+0000 7f2c91ffb640 1 -- 172.21.15.42:0/2328429633 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ca0010380 con 0x5577642e5070 2024-08-30T09:11:05.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.479+0000 7f2cb7d9c740 1 -- 172.21.15.42:0/2328429633 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5577642e5070 msgr2=0x55776468a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:11:05.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.479+0000 7f2cb7d9c740 1 --2- 172.21.15.42:0/2328429633 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5577642e5070 0x55776468a460 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f2ca00096a0 tx=0x7f2ca0009b60 comp rx=0 tx=0).stop 2024-08-30T09:11:05.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.488+0000 7f2cb7d9c740 1 -- 172.21.15.42:0/2328429633 shutdown_connections 2024-08-30T09:11:05.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.488+0000 7f2cb7d9c740 1 --2- 172.21.15.42:0/2328429633 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5577642c4390 0x55776468aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:05.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.488+0000 7f2cb7d9c740 1 --2- 172.21.15.42:0/2328429633 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5577642bc1b0 0x55776468a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:05.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.488+0000 7f2cb7d9c740 1 --2- 172.21.15.42:0/2328429633 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5577642e5070 0x55776468a460 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:05.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.488+0000 7f2cb7d9c740 1 -- 172.21.15.42:0/2328429633 >> 172.21.15.42:0/2328429633 conn(0x5577642d9460 msgr2=0x557764688350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:11:05.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.492+0000 7f2caaffd640 1 --2- 172.21.15.42:0/639255858 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f2c940b4ae0 0x7f2c940b6ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:05.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.496+0000 7f2cb7d9c740 1 -- 172.21.15.42:0/2328429633 shutdown_connections 2024-08-30T09:11:05.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.499+0000 7f2cb7d9c740 1 -- 172.21.15.42:0/2328429633 wait complete. 2024-08-30T09:11:05.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.499+0000 7f2cb7d9c740 1 -- start start 2024-08-30T09:11:05.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.500+0000 7f2cb7d9c740 1 Processor -- start 2024-08-30T09:11:05.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.502+0000 7f2cb7d9c740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5577642bc1b0 0x5577646a6c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:05.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.502+0000 7f2cb7d9c740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5577642c4390 0x5577646a7140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:05.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.502+0000 7f2cb7d9c740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5577642e5070 0x5577646b2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:05.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.502+0000 7f2cb7d9c740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x557764453700 con 0x5577642c4390 2024-08-30T09:11:05.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.502+0000 7f2cb7d9c740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x557764577700 con 0x5577642e5070 2024-08-30T09:11:05.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.502+0000 7f2cb7d9c740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5577646a9ba0 con 0x5577642bc1b0 2024-08-30T09:11:05.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.502+0000 7f2cb7d9c740 10 client.0 _ll_register_callbacks cb 0x5577645a6dc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:11:05.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.504+0000 7f2caa7fc640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5577642c4390 0x5577646a7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:05.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.504+0000 7f2caa7fc640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5577642c4390 0x5577646a7140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42748/0 (socket says 172.21.15.42:42748) 2024-08-30T09:11:05.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.504+0000 7f2caa7fc640 1 -- 172.21.15.42:0/3163143481 learned_addr learned my addr 172.21.15.42:0/3163143481 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:11:05.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.505+0000 7f2cab7fe640 1 --2- 172.21.15.42:0/3163143481 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5577642bc1b0 0x5577646a6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:05.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.505+0000 7f2cab7fe640 1 --2- 172.21.15.42:0/3163143481 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5577642e5070 0x5577646b2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:05.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.508+0000 7f2caaffd640 1 --2- 172.21.15.42:0/639255858 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f2c940b4ae0 0x7f2c940b6ff0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f2c9c0a8730 tx=0x7f2c9c0b0e80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:11:05.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.508+0000 7f2cab7fe640 1 -- 172.21.15.42:0/3163143481 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5577642e5070 msgr2=0x5577646b2ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:11:05.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.508+0000 7f2cab7fe640 1 --2- 172.21.15.42:0/3163143481 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5577642e5070 0x5577646b2ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:05.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.508+0000 7f2cab7fe640 1 -- 172.21.15.42:0/3163143481 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5577642c4390 msgr2=0x5577646a7140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:11:05.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.508+0000 7f2cab7fe640 1 --2- 172.21.15.42:0/3163143481 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5577642c4390 0x5577646a7140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:05.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.508+0000 7f2cab7fe640 1 -- 172.21.15.42:0/3163143481 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5577645b08b0 con 0x5577642bc1b0 2024-08-30T09:11:05.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.510+0000 7f2cab7fe640 1 --2- 172.21.15.42:0/3163143481 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5577642bc1b0 0x5577646a6c00 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f2ca0016000 tx=0x7f2ca0011b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b7d8abf784a425d0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:11:05.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.510+0000 7f2c8f7f6640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-08-30T09:11:05.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.512+0000 7f2c8f7f6640 1 -- 172.21.15.42:0/3163143481 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ca0015030 con 0x5577642bc1b0 2024-08-30T09:11:05.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.512+0000 7f2cb7d9c740 1 -- 172.21.15.42:0/3163143481 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2ca000bd10 con 0x5577642bc1b0 2024-08-30T09:11:05.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.512+0000 7f2c8f7f6640 1 -- 172.21.15.42:0/3163143481 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f2ca000b8a0 con 0x5577642bc1b0 2024-08-30T09:11:05.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.513+0000 7f2c8f7f6640 1 -- 172.21.15.42:0/3163143481 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f2ca002e050 con 0x5577642bc1b0 2024-08-30T09:11:05.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.516+0000 7f2cb7d9c740 10 client.26956 fetch_fsmap learned FSMap version 41 2024-08-30T09:11:05.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.516+0000 7f2cb7d9c740 1 -- 172.21.15.42:0/3163143481 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5577646b5990 con 0x5577642bc1b0 2024-08-30T09:11:05.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.526+0000 7f2c8f7f6640 1 -- 172.21.15.42:0/3163143481 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2ca000bd10 con 0x5577642bc1b0 2024-08-30T09:11:05.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.526+0000 7f2c8f7f6640 1 -- 172.21.15.42:0/3163143481 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 41) ==== 8362+0+0 (secure 0 0 0) 0x7f2ca001a420 con 0x5577642bc1b0 2024-08-30T09:11:05.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.530+0000 7f2cb7d9c740 10 client.26956 fetch_fsmap finished waiting for FSMap version 41 2024-08-30T09:11:05.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.530+0000 7f2cb7d9c740 10 client.26956 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-30T09:11:05.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.530+0000 7f2cb7d9c740 20 client.26956 populate_metadata read hostname 'smithi042' 2024-08-30T09:11:05.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.530+0000 7f2cb7d9c740 1 --2- 172.21.15.42:0/3163143481 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x5577642a39f0 0x5577646b8d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:05.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.530+0000 7f2cb7d9c740 4 client.26956 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-30T09:11:05.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.530+0000 7f2cb7d9c740 1 -- 172.21.15.42:0/3163143481 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557764603580 con 0x5577642a39f0 2024-08-30T09:11:05.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.533+0000 7f2caa7fc640 1 --2- 172.21.15.42:0/3163143481 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x5577642a39f0 0x5577646b8d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:05.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.540+0000 7f2caa7fc640 1 --2- 172.21.15.42:0/3163143481 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x5577642a39f0 0x5577646b8d40 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=264d27cf8dad5b11 server_cookie=8bcd981f9e48728d in_seq=0 out_seq=0 2024-08-30T09:11:05.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.541+0000 7f2c8f7f6640 10 client.26956 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:11:05.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.544+0000 7f2c8f7f6640 1 -- 172.21.15.42:0/3163143481 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x557764603580 con 0x5577642a39f0 2024-08-30T09:11:05.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.544+0000 7f2c8f7f6640 10 client.26956 handle_command_reply: tid=0 2024-08-30T09:11:05.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.544+0000 7f2cb7d9c740 1 client.26956 shutdown 2024-08-30T09:11:05.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.552+0000 7f2cb7d9c740 1 -- 172.21.15.42:0/3163143481 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5577642bc1b0 msgr2=0x5577646a6c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:11:05.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.552+0000 7f2cb7d9c740 1 --2- 172.21.15.42:0/3163143481 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5577642bc1b0 0x5577646a6c00 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f2ca0016000 tx=0x7f2ca0011b80 comp rx=0 tx=0).stop 2024-08-30T09:11:05.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.556+0000 7f2cb7d9c740 1 -- 172.21.15.42:0/3163143481 shutdown_connections 2024-08-30T09:11:05.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.556+0000 7f2cb7d9c740 1 --2- 172.21.15.42:0/3163143481 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x5577642a39f0 0x5577646b8d40 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:05.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.556+0000 7f2cb7d9c740 1 --2- 172.21.15.42:0/3163143481 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5577642e5070 0x5577646b2ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:05.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.556+0000 7f2cb7d9c740 1 --2- 172.21.15.42:0/3163143481 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5577642c4390 0x5577646a7140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:05.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.556+0000 7f2cb7d9c740 1 --2- 172.21.15.42:0/3163143481 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5577642bc1b0 0x5577646a6c00 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:05.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.556+0000 7f2c8f7f6640 0 client.26956 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:11:05.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.556+0000 7f2cb7d9c740 1 -- 172.21.15.42:0/3163143481 >> 172.21.15.42:0/3163143481 conn(0x5577642d9460 msgr2=0x55776468be50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:11:05.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.556+0000 7f2cb7d9c740 1 -- 172.21.15.42:0/3163143481 shutdown_connections 2024-08-30T09:11:05.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.556+0000 7f2cb7d9c740 1 -- 172.21.15.42:0/3163143481 wait complete. 2024-08-30T09:11:05.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.556+0000 7f2cb7d9c740 20 client.26956 trim_cache size 0 max 16384 2024-08-30T09:11:05.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.595+0000 7f2cb7d9c740 1 Processor -- start 2024-08-30T09:11:05.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.595+0000 7f2cb7d9c740 1 -- start start 2024-08-30T09:11:05.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.595+0000 7f2cb7d9c740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5577642bc1b0 0x557764688380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:05.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.595+0000 7f2cb7d9c740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5577642c4390 0x5577646b2dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:05.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.595+0000 7f2cb7d9c740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5577642e5070 0x5577646be3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:05.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.595+0000 7f2cb7d9c740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f2cac1b8ea0 con 0x5577642e5070 2024-08-30T09:11:05.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.595+0000 7f2cb7d9c740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f2cac1b9570 con 0x5577642bc1b0 2024-08-30T09:11:05.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.595+0000 7f2cb7d9c740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f2cac1b6de0 con 0x5577642c4390 2024-08-30T09:11:05.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.596+0000 7f2caa7fc640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5577642c4390 0x5577646b2dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:05.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.596+0000 7f2caa7fc640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5577642c4390 0x5577646b2dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:48002/0 (socket says 172.21.15.42:48002) 2024-08-30T09:11:05.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.596+0000 7f2caa7fc640 1 -- 172.21.15.42:0/107895530 learned_addr learned my addr 172.21.15.42:0/107895530 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:11:05.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.596+0000 7f2cab7fe640 1 --2- 172.21.15.42:0/107895530 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5577642bc1b0 0x557764688380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:05.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.596+0000 7f2cab7fe640 1 --2- 172.21.15.42:0/107895530 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5577642e5070 0x5577646be3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:05.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.596+0000 7f2cab7fe640 1 -- 172.21.15.42:0/107895530 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5577642c4390 msgr2=0x5577646b2dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:11:05.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.596+0000 7f2cab7fe640 1 --2- 172.21.15.42:0/107895530 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5577642c4390 0x5577646b2dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:05.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.596+0000 7f2cab7fe640 1 -- 172.21.15.42:0/107895530 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5577642e5070 msgr2=0x5577646be3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:11:05.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.596+0000 7f2cab7fe640 1 --2- 172.21.15.42:0/107895530 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5577642e5070 0x5577646be3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:05.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.596+0000 7f2cab7fe640 1 -- 172.21.15.42:0/107895530 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55776468b250 con 0x5577642bc1b0 2024-08-30T09:11:05.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.598+0000 7f2cab7fe640 1 --2- 172.21.15.42:0/107895530 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5577642bc1b0 0x557764688380 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f2ca0006ee0 tx=0x7f2ca0009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7cb8590462d8d0fa server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:11:05.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.598+0000 7f2c917fa640 1 -- 172.21.15.42:0/107895530 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ca0015040 con 0x5577642bc1b0 2024-08-30T09:11:05.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.598+0000 7f2c917fa640 1 -- 172.21.15.42:0/107895530 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2ca0020030 con 0x5577642bc1b0 2024-08-30T09:11:05.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.598+0000 7f2c917fa640 1 -- 172.21.15.42:0/107895530 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ca001ac40 con 0x5577642bc1b0 2024-08-30T09:11:05.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.601+0000 7f2cb7d9c740 1 -- 172.21.15.42:0/107895530 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5577642bc1b0 msgr2=0x557764688380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:11:05.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.601+0000 7f2cb7d9c740 1 --2- 172.21.15.42:0/107895530 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5577642bc1b0 0x557764688380 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f2ca0006ee0 tx=0x7f2ca0009b60 comp rx=0 tx=0).stop 2024-08-30T09:11:05.605 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:11:05 smithi042 ceph-mon[18652]: scrub summary: active paths [/] 2024-08-30T09:11:05.605 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:11:05 smithi042 ceph-mon[18652]: scrub summary: idle+waiting paths [/] 2024-08-30T09:11:05.605 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:11:05 smithi042 ceph-mon[18652]: scrub summary: active paths [/] 2024-08-30T09:11:05.605 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:11:05 smithi042 ceph-mon[18652]: scrub summary: idle+waiting paths [/] 2024-08-30T09:11:05.605 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:11:05 smithi042 ceph-mon[18652]: scrub summary: idle+waiting paths [/] 2024-08-30T09:11:05.605 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:11:05 smithi042 ceph-mon[18652]: scrub summary: idle+waiting paths [/] 2024-08-30T09:11:05.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.601+0000 7f2cb7d9c740 1 -- 172.21.15.42:0/107895530 shutdown_connections 2024-08-30T09:11:05.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.601+0000 7f2cb7d9c740 1 --2- 172.21.15.42:0/107895530 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5577642c4390 0x5577646b2dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:05.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.601+0000 7f2cb7d9c740 1 --2- 172.21.15.42:0/107895530 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5577642e5070 0x5577646be3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:05.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.601+0000 7f2cb7d9c740 1 --2- 172.21.15.42:0/107895530 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5577642bc1b0 0x557764688380 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:05.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.601+0000 7f2cb7d9c740 1 -- 172.21.15.42:0/107895530 >> 172.21.15.42:0/107895530 conn(0x5577642d9460 msgr2=0x557764687c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:11:05.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.602+0000 7f2cb7d9c740 1 -- 172.21.15.42:0/107895530 shutdown_connections 2024-08-30T09:11:05.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.602+0000 7f2cb7d9c740 1 -- 172.21.15.42:0/107895530 wait complete. 2024-08-30T09:11:05.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.603+0000 7f2cb7d9c740 1 -- start start 2024-08-30T09:11:05.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.603+0000 7f2cb7d9c740 1 Processor -- start 2024-08-30T09:11:05.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.604+0000 7f2cb7d9c740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5577642bc1b0 0x5577646ab1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:05.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.604+0000 7f2cb7d9c740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5577642c4390 0x5577646ab720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:05.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.604+0000 7f2cb7d9c740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5577642e5070 0x5577647225d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:05.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.604+0000 7f2cb7d9c740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f2cac11a130 con 0x5577642c4390 2024-08-30T09:11:05.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.604+0000 7f2cb7d9c740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f2cac1b6710 con 0x5577642bc1b0 2024-08-30T09:11:05.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.604+0000 7f2cb7d9c740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f2cac1b5a90 con 0x5577642e5070 2024-08-30T09:11:05.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.604+0000 7f2cb7d9c740 10 client.0 _ll_register_callbacks cb 0x7f2ca4009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:11:05.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.604+0000 7f2cab7fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5577642bc1b0 0x5577646ab1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:05.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.604+0000 7f2cab7fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5577642bc1b0 0x5577646ab1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:36514/0 (socket says 172.21.15.42:36514) 2024-08-30T09:11:05.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.604+0000 7f2cab7fe640 1 -- 172.21.15.42:0/2685682569 learned_addr learned my addr 172.21.15.42:0/2685682569 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:11:05.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.604+0000 7f2cab7fe640 1 --2- 172.21.15.42:0/2685682569 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5577642e5070 0x5577647225d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:05.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.605+0000 7f2cab7fe640 1 -- 172.21.15.42:0/2685682569 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5577642e5070 msgr2=0x5577647225d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:11:05.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.605+0000 7f2cab7fe640 1 --2- 172.21.15.42:0/2685682569 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5577642e5070 0x5577647225d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:05.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.605+0000 7f2cab7fe640 1 -- 172.21.15.42:0/2685682569 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5577642c4390 msgr2=0x5577646ab720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:11:05.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.605+0000 7f2cab7fe640 1 --2- 172.21.15.42:0/2685682569 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5577642c4390 0x5577646ab720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:05.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.605+0000 7f2cab7fe640 1 -- 172.21.15.42:0/2685682569 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557764722b10 con 0x5577642bc1b0 2024-08-30T09:11:05.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.605+0000 7f2c8f7f6640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-30T09:11:05.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.605+0000 7f2cab7fe640 1 --2- 172.21.15.42:0/2685682569 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5577642bc1b0 0x5577646ab1e0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x5577646afd80 tx=0x7f2ca0008040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2cf565ab11f695d8 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:11:05.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.605+0000 7f2c8f7f6640 1 -- 172.21.15.42:0/2685682569 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ca0015070 con 0x5577642bc1b0 2024-08-30T09:11:05.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.605+0000 7f2cb7d9c740 1 -- 172.21.15.42:0/2685682569 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2ca0020030 con 0x5577642bc1b0 2024-08-30T09:11:05.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.605+0000 7f2c8f7f6640 1 -- 172.21.15.42:0/2685682569 <== mon.2 v2:172.21.15.167:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f2ca0005a00 con 0x5577642bc1b0 2024-08-30T09:11:05.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.607+0000 7f2c8f7f6640 1 -- 172.21.15.42:0/2685682569 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f2ca0015070 con 0x5577642bc1b0 2024-08-30T09:11:05.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.607+0000 7f2cb7d9c740 10 client.26990 fetch_fsmap learned FSMap version 41 2024-08-30T09:11:05.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.607+0000 7f2cb7d9c740 1 -- 172.21.15.42:0/2685682569 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x557764723bf0 con 0x5577642bc1b0 2024-08-30T09:11:05.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.607+0000 7f2c8f7f6640 1 -- 172.21.15.42:0/2685682569 <== mon.2 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2ca0020030 con 0x5577642bc1b0 2024-08-30T09:11:05.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.608+0000 7f2c8f7f6640 1 -- 172.21.15.42:0/2685682569 <== mon.2 v2:172.21.15.167:3300/0 5 ==== fsmap(e 41) ==== 8362+0+0 (secure 0 0 0) 0x7f2ca0004020 con 0x5577642bc1b0 2024-08-30T09:11:05.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.608+0000 7f2cb7d9c740 10 client.26990 fetch_fsmap finished waiting for FSMap version 41 2024-08-30T09:11:05.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.608+0000 7f2cb7d9c740 10 client.26990 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-30T09:11:05.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.608+0000 7f2cb7d9c740 20 client.26990 populate_metadata read hostname 'smithi042' 2024-08-30T09:11:05.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.608+0000 7f2cb7d9c740 1 --2- 172.21.15.42:0/2685682569 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x557764278da0 0x557764726310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:05.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.608+0000 7f2cb7d9c740 4 client.26990 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-30T09:11:05.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.608+0000 7f2cb7d9c740 1 -- 172.21.15.42:0/2685682569 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5577645ff020 con 0x557764278da0 2024-08-30T09:11:05.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.609+0000 7f2caa7fc640 1 --2- 172.21.15.42:0/2685682569 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x557764278da0 0x557764726310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:05.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.616+0000 7f2c8f7f6640 10 client.26990 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:11:05.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.616+0000 7f2caa7fc640 1 --2- 172.21.15.42:0/2685682569 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x557764278da0 0x557764726310 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e57495a77c4bb66b server_cookie=ae2548796adfdf71 in_seq=0 out_seq=0 2024-08-30T09:11:05.621 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.618+0000 7f2c8f7f6640 1 -- 172.21.15.42:0/2685682569 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3978 (crc 0 0 0) 0x5577645ff020 con 0x557764278da0 2024-08-30T09:11:05.621 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.618+0000 7f2c8f7f6640 10 client.26990 handle_command_reply: tid=0 2024-08-30T09:11:05.621 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.618+0000 7f2cb7d9c740 1 client.26990 shutdown 2024-08-30T09:11:05.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.620+0000 7f2cb7d9c740 1 -- 172.21.15.42:0/2685682569 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5577642bc1b0 msgr2=0x5577646ab1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:11:05.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.620+0000 7f2cb7d9c740 1 --2- 172.21.15.42:0/2685682569 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5577642bc1b0 0x5577646ab1e0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x5577646afd80 tx=0x7f2ca0008040 comp rx=0 tx=0).stop 2024-08-30T09:11:05.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.620+0000 7f2cb7d9c740 1 -- 172.21.15.42:0/2685682569 shutdown_connections 2024-08-30T09:11:05.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.620+0000 7f2cb7d9c740 1 --2- 172.21.15.42:0/2685682569 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x557764278da0 0x557764726310 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:05.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.620+0000 7f2cb7d9c740 1 --2- 172.21.15.42:0/2685682569 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5577642e5070 0x5577647225d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:05.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.620+0000 7f2cb7d9c740 1 --2- 172.21.15.42:0/2685682569 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5577642c4390 0x5577646ab720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:05.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.620+0000 7f2cb7d9c740 1 --2- 172.21.15.42:0/2685682569 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5577642bc1b0 0x5577646ab1e0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:05.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.620+0000 7f2cb7d9c740 1 -- 172.21.15.42:0/2685682569 >> 172.21.15.42:0/2685682569 conn(0x5577642d9460 msgr2=0x55776468b490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:11:05.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.620+0000 7f2c8f7f6640 0 client.26990 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:11:05.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.620+0000 7f2cb7d9c740 1 -- 172.21.15.42:0/2685682569 shutdown_connections 2024-08-30T09:11:05.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.620+0000 7f2cb7d9c740 1 -- 172.21.15.42:0/2685682569 wait complete. 2024-08-30T09:11:05.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.620+0000 7f2cb7d9c740 20 client.26990 trim_cache size 0 max 16384 2024-08-30T09:11:05.625 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-08-30T09:11:05.625 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "IDLE (waiting for more scrubs)", 2024-08-30T09:11:05.625 INFO:teuthology.orchestra.run.smithi042.stdout: "scrubs": { 2024-08-30T09:11:05.625 INFO:teuthology.orchestra.run.smithi042.stdout: "b62c4ffc-9387-4677-b9ba-1e94ba308f17": { 2024-08-30T09:11:05.625 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:11:05.625 INFO:teuthology.orchestra.run.smithi042.stdout: "tag": "b62c4ffc-9387-4677-b9ba-1e94ba308f17", 2024-08-30T09:11:05.625 INFO:teuthology.orchestra.run.smithi042.stdout: "options": "recursive,force" 2024-08-30T09:11:05.625 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:11:05.625 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:11:05.626 INFO:teuthology.orchestra.run.smithi042.stdout: "scrub_stats": { 2024-08-30T09:11:05.626 INFO:teuthology.orchestra.run.smithi042.stdout: "37f2f28f-496f-41a6-b4cf-8846fb3f43ab": { 2024-08-30T09:11:05.626 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:09:17.428286+0000", 2024-08-30T09:11:05.626 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:11:05.626 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:11:05.626 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:11:05.626 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:11:05.626 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:11:05.626 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:11:05.626 INFO:teuthology.orchestra.run.smithi042.stdout: "99a29ec1-9455-4ae4-854a-398b32896675": { 2024-08-30T09:11:05.626 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:08:24.923176+0000", 2024-08-30T09:11:05.626 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:11:05.626 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:11:05.627 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:11:05.627 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:11:05.627 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:11:05.627 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:11:05.627 INFO:teuthology.orchestra.run.smithi042.stdout: "a7210f9a-046b-4188-953c-819b9f1863a9": { 2024-08-30T09:11:05.627 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:10:07.608430+0000", 2024-08-30T09:11:05.627 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:11:05.627 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:11:05.627 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:11:05.627 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:11:05.627 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:11:05.627 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:11:05.627 INFO:teuthology.orchestra.run.smithi042.stdout: "598d8a75-f1cc-4388-80bc-a408bdb4eaac": { 2024-08-30T09:11:05.627 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:07:29.721122+0000", 2024-08-30T09:11:05.627 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:11:05.628 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:11:05.628 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:11:05.628 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:11:05.628 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:11:05.628 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:11:05.628 INFO:teuthology.orchestra.run.smithi042.stdout: "665ec740-7254-49cb-84ef-deff71c461b0": { 2024-08-30T09:11:05.628 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:06:37.144035+0000", 2024-08-30T09:11:05.628 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:11:05.628 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:11:05.628 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:11:05.628 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:11:05.628 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:11:05.629 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:11:05.629 INFO:teuthology.orchestra.run.smithi042.stdout: "1af4252a-4a99-47e9-bbf0-becf5e573dd5": { 2024-08-30T09:11:05.629 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:05:48.565881+0000", 2024-08-30T09:11:05.629 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:11:05.629 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:11:05.629 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:11:05.629 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:11:05.629 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:11:05.629 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:11:05.629 INFO:teuthology.orchestra.run.smithi042.stdout: "347d05ed-4bd3-4590-b099-314f719aab39": { 2024-08-30T09:11:05.629 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:05:00.439721+0000", 2024-08-30T09:11:05.629 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:11:05.629 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:11:05.630 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:11:05.630 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:11:05.630 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:11:05.630 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:11:05.630 INFO:teuthology.orchestra.run.smithi042.stdout: "b62c4ffc-9387-4677-b9ba-1e94ba308f17": { 2024-08-30T09:11:05.630 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:11:03.363478+0000", 2024-08-30T09:11:05.630 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:11:05.630 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:11:05.630 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:11:05.630 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:11:05.630 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:11:05.630 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:11:05.630 INFO:teuthology.orchestra.run.smithi042.stdout: "88cd7478-2b20-47de-ba51-256479749232": { 2024-08-30T09:11:05.630 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:04:08.922625+0000", 2024-08-30T09:11:05.631 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:11:05.631 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:11:05.631 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:11:05.631 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:11:05.631 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:11:05.631 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:11:05.631 INFO:teuthology.orchestra.run.smithi042.stdout: "749f4867-0b70-49e5-af2d-8362a240ebc3": { 2024-08-30T09:11:05.631 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:03:16.386554+0000", 2024-08-30T09:11:05.631 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:11:05.631 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:11:05.631 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:11:05.631 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:11:05.631 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:11:05.632 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:11:05.632 INFO:teuthology.orchestra.run.smithi042.stdout: "85134e4b-ab4a-4885-b07e-a3ad01eaf13d": { 2024-08-30T09:11:05.632 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:01:27.738462+0000", 2024-08-30T09:11:05.632 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:11:05.632 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:11:05.632 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:11:05.632 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:11:05.632 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:11:05.632 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:11:05.632 INFO:teuthology.orchestra.run.smithi042.stdout: "c827ad7a-44ee-4e1d-9bb3-51fb7ab00c73": { 2024-08-30T09:11:05.632 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:02:21.686517+0000", 2024-08-30T09:11:05.632 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:11:05.632 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:11:05.633 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:11:05.633 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:11:05.633 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:11:05.633 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:11:05.633 INFO:teuthology.orchestra.run.smithi042.stdout: "5e2ed104-0625-478a-a78b-584e7f4c2817": { 2024-08-30T09:11:05.633 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:00:33.928400+0000", 2024-08-30T09:11:05.633 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:11:05.633 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:11:05.633 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:11:05.633 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:11:05.633 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:11:05.633 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:11:05.633 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:11:05.634 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-08-30T09:11:05.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.621+0000 7f2cabfff640 1 -- 172.21.15.42:0/639255858 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f2c940b4ae0 msgr2=0x7f2c940b6ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:11:05.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.621+0000 7f2cabfff640 1 --2- 172.21.15.42:0/639255858 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f2c940b4ae0 0x7f2c940b6ff0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f2c9c0a8730 tx=0x7f2c9c0b0e80 comp rx=0 tx=0).stop 2024-08-30T09:11:05.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.621+0000 7f2cabfff640 1 -- 172.21.15.42:0/639255858 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2cac10d990 msgr2=0x7f2cac1a8f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:11:05.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.621+0000 7f2cabfff640 1 --2- 172.21.15.42:0/639255858 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2cac10d990 0x7f2cac1a8f00 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f2c9c00d770 tx=0x7f2c9c00dc30 comp rx=0 tx=0).stop 2024-08-30T09:11:05.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.621+0000 7f2cabfff640 1 -- 172.21.15.42:0/639255858 shutdown_connections 2024-08-30T09:11:05.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.621+0000 7f2cabfff640 1 --2- 172.21.15.42:0/639255858 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f2c940b4ae0 0x7f2c940b6ff0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:05.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.621+0000 7f2cabfff640 1 --2- 172.21.15.42:0/639255858 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2cac111a70 0x7f2cac1a94a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:05.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.621+0000 7f2cabfff640 1 --2- 172.21.15.42:0/639255858 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2cac11c940 0x7f2cac1ae740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:05.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.621+0000 7f2cabfff640 1 --2- 172.21.15.42:0/639255858 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2cac10d990 0x7f2cac1a8f00 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:05.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.621+0000 7f2cabfff640 1 -- 172.21.15.42:0/639255858 >> 172.21.15.42:0/639255858 conn(0x7f2cac06e790 msgr2=0x7f2cac110720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:11:05.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.621+0000 7f2cabfff640 1 -- 172.21.15.42:0/639255858 shutdown_connections 2024-08-30T09:11:05.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:05.621+0000 7f2cabfff640 1 -- 172.21.15.42:0/639255858 wait complete. 2024-08-30T09:11:05.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:11:05 smithi190 ceph-mon[22001]: scrub summary: active paths [/] 2024-08-30T09:11:05.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:11:05 smithi190 ceph-mon[22001]: scrub summary: idle+waiting paths [/] 2024-08-30T09:11:05.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:11:05 smithi190 ceph-mon[22001]: scrub summary: active paths [/] 2024-08-30T09:11:05.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:11:05 smithi190 ceph-mon[22001]: scrub summary: idle+waiting paths [/] 2024-08-30T09:11:05.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:11:05 smithi190 ceph-mon[22001]: scrub summary: idle+waiting paths [/] 2024-08-30T09:11:05.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:11:05 smithi190 ceph-mon[22001]: scrub summary: idle+waiting paths [/] 2024-08-30T09:11:05.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:11:05 smithi167 ceph-mon[22063]: scrub summary: active paths [/] 2024-08-30T09:11:05.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:11:05 smithi167 ceph-mon[22063]: scrub summary: idle+waiting paths [/] 2024-08-30T09:11:05.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:11:05 smithi167 ceph-mon[22063]: scrub summary: active paths [/] 2024-08-30T09:11:05.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:11:05 smithi167 ceph-mon[22063]: scrub summary: idle+waiting paths [/] 2024-08-30T09:11:05.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:11:05 smithi167 ceph-mon[22063]: scrub summary: idle+waiting paths [/] 2024-08-30T09:11:05.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:11:05 smithi167 ceph-mon[22063]: scrub summary: idle+waiting paths [/] 2024-08-30T09:11:05.953 INFO:teuthology.orchestra.run.smithi042.stderr:progress: 620.0 s, 471.4 tps, lat 203.990 ms stddev 257.222 2024-08-30T09:11:06.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:11:06 smithi190 ceph-mon[22001]: pgmap v683: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 7.3 MiB/s wr, 473 op/s 2024-08-30T09:11:06.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:11:06 smithi167 ceph-mon[22063]: pgmap v683: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 7.3 MiB/s wr, 473 op/s 2024-08-30T09:11:06.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:11:06 smithi042 ceph-mon[18652]: pgmap v683: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 7.3 MiB/s wr, 473 op/s 2024-08-30T09:11:07.411 INFO:tasks.cephfs.filesystem:scrub status for tag:b62c4ffc-9387-4677-b9ba-1e94ba308f17 - {'path': '/', 'tag': 'b62c4ffc-9387-4677-b9ba-1e94ba308f17', 'options': 'recursive,force'} 2024-08-30T09:11:08.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:11:08 smithi190 ceph-mon[22001]: pgmap v684: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 7.0 MiB/s wr, 491 op/s 2024-08-30T09:11:08.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:11:08 smithi167 ceph-mon[22063]: pgmap v684: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 7.0 MiB/s wr, 491 op/s 2024-08-30T09:11:08.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:11:08 smithi042 ceph-mon[18652]: pgmap v684: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 7.0 MiB/s wr, 491 op/s 2024-08-30T09:11:10.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:11:10 smithi190 ceph-mon[22001]: pgmap v685: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 7.6 MiB/s wr, 586 op/s 2024-08-30T09:11:10.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:11:10 smithi167 ceph-mon[22063]: pgmap v685: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 7.6 MiB/s wr, 586 op/s 2024-08-30T09:11:10.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:11:10 smithi042 ceph-mon[18652]: pgmap v685: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 7.6 MiB/s wr, 586 op/s 2024-08-30T09:11:10.953 INFO:teuthology.orchestra.run.smithi042.stderr:progress: 625.0 s, 514.8 tps, lat 199.772 ms stddev 243.318 2024-08-30T09:11:12.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:11:12 smithi190 ceph-mon[22001]: pgmap v686: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 5.7 MiB/s wr, 479 op/s 2024-08-30T09:11:12.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:11:12 smithi167 ceph-mon[22063]: pgmap v686: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 5.7 MiB/s wr, 479 op/s 2024-08-30T09:11:12.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:11:12 smithi042 ceph-mon[18652]: pgmap v686: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 5.7 MiB/s wr, 479 op/s 2024-08-30T09:11:13.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:11:13 smithi167 ceph-mon[22063]: scrub summary: idle 2024-08-30T09:11:13.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:11:13 smithi167 ceph-mon[22063]: scrub summary: idle 2024-08-30T09:11:13.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:11:13 smithi042 ceph-mon[18652]: scrub summary: idle 2024-08-30T09:11:13.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:11:13 smithi042 ceph-mon[18652]: scrub summary: idle 2024-08-30T09:11:13.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:11:13 smithi190 ceph-mon[22001]: scrub summary: idle 2024-08-30T09:11:13.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:11:13 smithi190 ceph-mon[22001]: scrub summary: idle 2024-08-30T09:11:14.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:11:14 smithi167 ceph-mon[22063]: scrub summary: idle 2024-08-30T09:11:14.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:11:14 smithi167 ceph-mon[22063]: pgmap v687: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 8.3 MiB/s wr, 624 op/s 2024-08-30T09:11:14.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:11:14 smithi042 ceph-mon[18652]: scrub summary: idle 2024-08-30T09:11:14.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:11:14 smithi042 ceph-mon[18652]: pgmap v687: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 8.3 MiB/s wr, 624 op/s 2024-08-30T09:11:14.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:11:14 smithi190 ceph-mon[22001]: scrub summary: idle 2024-08-30T09:11:14.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:11:14 smithi190 ceph-mon[22001]: pgmap v687: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 8.3 MiB/s wr, 624 op/s 2024-08-30T09:11:15.953 INFO:teuthology.orchestra.run.smithi042.stderr:progress: 630.0 s, 490.6 tps, lat 206.994 ms stddev 246.958 2024-08-30T09:11:16.066 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.063+0000 7fadb10de640 1 -- 172.21.15.42:0/3310719478 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fadac1044e0 msgr2=0x7fadac1048f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:11:16.066 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.063+0000 7fadb10de640 1 --2- 172.21.15.42:0/3310719478 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fadac1044e0 0x7fadac1048f0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fada0009900 tx=0x7fada002a440 comp rx=0 tx=0).stop 2024-08-30T09:11:16.071 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.064+0000 7fadb10de640 1 -- 172.21.15.42:0/3310719478 shutdown_connections 2024-08-30T09:11:16.071 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.064+0000 7fadb10de640 1 --2- 172.21.15.42:0/3310719478 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fadac1044e0 0x7fadac1048f0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:16.071 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.064+0000 7fadb10de640 1 --2- 172.21.15.42:0/3310719478 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fadac104e80 0x7fadac10b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:16.071 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.064+0000 7fadb10de640 1 --2- 172.21.15.42:0/3310719478 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fadac103460 0x7fadac1038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:16.071 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.064+0000 7fadb10de640 1 -- 172.21.15.42:0/3310719478 >> 172.21.15.42:0/3310719478 conn(0x7fadac0fed80 msgr2=0x7fadac101250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:11:16.073 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.068+0000 7fadb10de640 1 -- 172.21.15.42:0/3310719478 shutdown_connections 2024-08-30T09:11:16.073 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.069+0000 7fadb10de640 1 -- 172.21.15.42:0/3310719478 wait complete. 2024-08-30T09:11:16.073 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.070+0000 7fadb10de640 1 Processor -- start 2024-08-30T09:11:16.077 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.070+0000 7fadb10de640 1 -- start start 2024-08-30T09:11:16.077 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.070+0000 7fadb10de640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fadac103460 0x7fadac114210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:16.077 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.070+0000 7fadb10de640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fadac1044e0 0x7fadac114880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:16.077 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.070+0000 7fadb10de640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fadac104e80 0x7fadac112cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:16.078 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.071+0000 7fadb10de640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fadac10cec0 con 0x7fadac104e80 2024-08-30T09:11:16.078 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.071+0000 7fadb10de640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fadac10cd40 con 0x7fadac1044e0 2024-08-30T09:11:16.078 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.071+0000 7fadb10de640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fadac10d040 con 0x7fadac103460 2024-08-30T09:11:16.078 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.072+0000 7fadab59e640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fadac104e80 0x7fadac112cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:16.078 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.072+0000 7fadab59e640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fadac104e80 0x7fadac112cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43018/0 (socket says 172.21.15.42:43018) 2024-08-30T09:11:16.078 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.072+0000 7fadab59e640 1 -- 172.21.15.42:0/773849130 learned_addr learned my addr 172.21.15.42:0/773849130 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:11:16.080 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.075+0000 7fadaa59c640 1 --2- 172.21.15.42:0/773849130 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fadac1044e0 0x7fadac114880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:16.080 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.076+0000 7fadaad9d640 1 --2- 172.21.15.42:0/773849130 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fadac103460 0x7fadac114210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:16.084 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.079+0000 7fadab59e640 1 -- 172.21.15.42:0/773849130 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fadac103460 msgr2=0x7fadac114210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:11:16.084 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.079+0000 7fadab59e640 1 --2- 172.21.15.42:0/773849130 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fadac103460 0x7fadac114210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:16.084 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.079+0000 7fadab59e640 1 -- 172.21.15.42:0/773849130 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fadac1044e0 msgr2=0x7fadac114880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:11:16.084 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.079+0000 7fadab59e640 1 --2- 172.21.15.42:0/773849130 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fadac1044e0 0x7fadac114880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:16.084 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.079+0000 7fadab59e640 1 -- 172.21.15.42:0/773849130 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadac1134d0 con 0x7fadac104e80 2024-08-30T09:11:16.093 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.090+0000 7fadab59e640 1 --2- 172.21.15.42:0/773849130 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fadac104e80 0x7fadac112cb0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fada400eb10 tx=0x7fada400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:11:16.097 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.092+0000 7fad93fff640 1 -- 172.21.15.42:0/773849130 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fada4004b20 con 0x7fadac104e80 2024-08-30T09:11:16.098 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.093+0000 7fad93fff640 1 -- 172.21.15.42:0/773849130 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fada4004cc0 con 0x7fadac104e80 2024-08-30T09:11:16.098 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.093+0000 7fad93fff640 1 -- 172.21.15.42:0/773849130 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fada4010670 con 0x7fadac104e80 2024-08-30T09:11:16.098 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.093+0000 7fadb10de640 1 -- 172.21.15.42:0/773849130 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fadac1137c0 con 0x7fadac104e80 2024-08-30T09:11:16.098 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.093+0000 7fadb10de640 1 -- 172.21.15.42:0/773849130 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fadac1b0dc0 con 0x7fadac104e80 2024-08-30T09:11:16.102 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.098+0000 7fad93fff640 1 -- 172.21.15.42:0/773849130 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7fada4005020 con 0x7fadac104e80 2024-08-30T09:11:16.102 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.100+0000 7fadb8051740 1 Processor -- start 2024-08-30T09:11:16.103 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.100+0000 7fad93fff640 1 --2- 172.21.15.42:0/773849130 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fad940b4cc0 0x7fad940b71d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:16.103 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.100+0000 7fadb8051740 1 -- start start 2024-08-30T09:11:16.103 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.101+0000 7fad93fff640 1 -- 172.21.15.42:0/773849130 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7fada40e5a50 con 0x7fadac104e80 2024-08-30T09:11:16.104 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.101+0000 7fadb8051740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ba2f9c31b0 0x55ba2fd69180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:16.105 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.101+0000 7fadb8051740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ba2f99a2f0 0x55ba2fd697f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:16.105 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.101+0000 7fadb8051740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ba2f9a24d0 0x55ba2fd6a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:16.105 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.101+0000 7fadb8051740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55ba2fcdc000 con 0x55ba2f9c31b0 2024-08-30T09:11:16.105 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.101+0000 7fadb8051740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55ba2fccb590 con 0x55ba2f9a24d0 2024-08-30T09:11:16.105 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.102+0000 7fadaad9d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ba2f9c31b0 0x55ba2fd69180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:16.105 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.102+0000 7fadaad9d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ba2f9c31b0 0x55ba2fd69180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43032/0 (socket says 172.21.15.42:43032) 2024-08-30T09:11:16.105 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.102+0000 7fadaad9d640 1 -- 172.21.15.42:0/3358002820 learned_addr learned my addr 172.21.15.42:0/3358002820 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:11:16.105 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.102+0000 7fadb8051740 1 -- 172.21.15.42:0/3358002820 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55ba2fccf540 con 0x55ba2f99a2f0 2024-08-30T09:11:16.105 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.102+0000 7fadab59e640 1 --2- 172.21.15.42:0/3358002820 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ba2f9a24d0 0x55ba2fd6a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:16.106 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.102+0000 7fadab59e640 1 -- 172.21.15.42:0/3358002820 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ba2f99a2f0 msgr2=0x55ba2fd697f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-30T09:11:16.106 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.102+0000 7fadab59e640 1 --2- 172.21.15.42:0/3358002820 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ba2f99a2f0 0x55ba2fd697f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:16.106 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.102+0000 7fadab59e640 1 -- 172.21.15.42:0/3358002820 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ba2f9c31b0 msgr2=0x55ba2fd69180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:11:16.106 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.102+0000 7fadab59e640 1 --2- 172.21.15.42:0/3358002820 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ba2f9c31b0 0x55ba2fd69180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:16.106 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.102+0000 7fadab59e640 1 -- 172.21.15.42:0/3358002820 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba2fa2e790 con 0x55ba2f9a24d0 2024-08-30T09:11:16.106 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.103+0000 7fadab59e640 1 --2- 172.21.15.42:0/3358002820 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ba2f9a24d0 0x55ba2fd6a340 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fada40a9360 tx=0x7fada40aba40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=646aefb8bf966414 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:11:16.107 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.103+0000 7fad917fa640 1 -- 172.21.15.42:0/3358002820 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fada40ac7f0 con 0x55ba2f9a24d0 2024-08-30T09:11:16.107 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.103+0000 7fad917fa640 1 -- 172.21.15.42:0/3358002820 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fada40ac0f0 con 0x55ba2f9a24d0 2024-08-30T09:11:16.107 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.104+0000 7fadb8051740 1 -- 172.21.15.42:0/3358002820 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ba2f9a24d0 msgr2=0x55ba2fd6a340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:11:16.107 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.104+0000 7fad917fa640 1 -- 172.21.15.42:0/3358002820 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fada40ac5b0 con 0x55ba2f9a24d0 2024-08-30T09:11:16.108 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.104+0000 7fadb8051740 1 --2- 172.21.15.42:0/3358002820 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ba2f9a24d0 0x55ba2fd6a340 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fada40a9360 tx=0x7fada40aba40 comp rx=0 tx=0).stop 2024-08-30T09:11:16.108 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.105+0000 7fadb8051740 1 -- 172.21.15.42:0/3358002820 shutdown_connections 2024-08-30T09:11:16.108 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.105+0000 7fadb8051740 1 --2- 172.21.15.42:0/3358002820 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ba2f99a2f0 0x55ba2fd697f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:16.108 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.105+0000 7fadb8051740 1 --2- 172.21.15.42:0/3358002820 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ba2f9a24d0 0x55ba2fd6a340 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:16.108 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.105+0000 7fadb8051740 1 --2- 172.21.15.42:0/3358002820 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ba2f9c31b0 0x55ba2fd69180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:16.108 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.105+0000 7fadb8051740 1 -- 172.21.15.42:0/3358002820 >> 172.21.15.42:0/3358002820 conn(0x55ba2f9b7300 msgr2=0x55ba2fd64c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:11:16.109 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.106+0000 7fadaa59c640 1 --2- 172.21.15.42:0/773849130 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fad940b4cc0 0x7fad940b71d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:16.110 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.107+0000 7fadb8051740 1 -- 172.21.15.42:0/3358002820 shutdown_connections 2024-08-30T09:11:16.110 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.107+0000 7fadb8051740 1 -- 172.21.15.42:0/3358002820 wait complete. 2024-08-30T09:11:16.111 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.107+0000 7fadaa59c640 1 --2- 172.21.15.42:0/773849130 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fad940b4cc0 0x7fad940b71d0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fada002aa30 tx=0x7fada0000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:11:16.111 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.108+0000 7fadb8051740 1 -- start start 2024-08-30T09:11:16.111 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.108+0000 7fadb8051740 1 Processor -- start 2024-08-30T09:11:16.112 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.109+0000 7fadb8051740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ba2f99a2f0 0x55ba2fd72c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:16.112 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.109+0000 7fadb8051740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ba2f9a24d0 0x55ba2fd73310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:16.112 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.109+0000 7fadb8051740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ba2f9c31b0 0x55ba2fd73e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:16.112 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.109+0000 7fadb8051740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55ba2fc04ab0 con 0x55ba2f99a2f0 2024-08-30T09:11:16.112 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.109+0000 7fadaad9d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ba2f99a2f0 0x55ba2fd72c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:16.113 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.109+0000 7fadaad9d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ba2f99a2f0 0x55ba2fd72c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43048/0 (socket says 172.21.15.42:43048) 2024-08-30T09:11:16.113 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.109+0000 7fadaad9d640 1 -- 172.21.15.42:0/234930302 learned_addr learned my addr 172.21.15.42:0/234930302 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:11:16.113 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.109+0000 7fadaa59c640 1 --2- 172.21.15.42:0/234930302 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ba2f9a24d0 0x55ba2fd73310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:16.113 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.109+0000 7fadab59e640 1 --2- 172.21.15.42:0/234930302 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ba2f9c31b0 0x55ba2fd73e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:16.113 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.109+0000 7fadb8051740 1 -- 172.21.15.42:0/234930302 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55ba2fc52680 con 0x55ba2f9a24d0 2024-08-30T09:11:16.113 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.109+0000 7fadb8051740 1 -- 172.21.15.42:0/234930302 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55ba2fd6aeb0 con 0x55ba2f9c31b0 2024-08-30T09:11:16.113 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.109+0000 7fadb8051740 10 client.0 _ll_register_callbacks cb 0x55ba2f9daa00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:11:16.115 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.112+0000 7fadaa59c640 1 -- 172.21.15.42:0/234930302 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ba2f9c31b0 msgr2=0x55ba2fd73e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:11:16.115 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.112+0000 7fadaa59c640 1 --2- 172.21.15.42:0/234930302 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ba2f9c31b0 0x55ba2fd73e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:16.116 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.112+0000 7fadaa59c640 1 -- 172.21.15.42:0/234930302 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ba2f99a2f0 msgr2=0x55ba2fd72c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:11:16.116 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.112+0000 7fadaa59c640 1 --2- 172.21.15.42:0/234930302 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ba2f99a2f0 0x55ba2fd72c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:16.116 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.112+0000 7fadaa59c640 1 -- 172.21.15.42:0/234930302 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ba2faa7620 con 0x55ba2f9a24d0 2024-08-30T09:11:16.116 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.113+0000 7fadaa59c640 1 --2- 172.21.15.42:0/234930302 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ba2f9a24d0 0x55ba2fd73310 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fada0009230 tx=0x7fada002a440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=adfae0edaba14ccd server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:11:16.116 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.113+0000 7fad8eff5640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-30T09:11:16.116 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.113+0000 7fad8eff5640 1 -- 172.21.15.42:0/234930302 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fada0031640 con 0x55ba2f9a24d0 2024-08-30T09:11:16.116 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.113+0000 7fad8eff5640 1 -- 172.21.15.42:0/234930302 <== mon.2 v2:172.21.15.167:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7fada00317e0 con 0x55ba2f9a24d0 2024-08-30T09:11:16.117 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.113+0000 7fadb8051740 1 -- 172.21.15.42:0/234930302 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fada40ac0f0 con 0x55ba2f9a24d0 2024-08-30T09:11:16.117 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.114+0000 7fad8eff5640 1 -- 172.21.15.42:0/234930302 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fada00319e0 con 0x55ba2f9a24d0 2024-08-30T09:11:16.117 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.115+0000 7fadb8051740 10 client.27002 fetch_fsmap learned FSMap version 41 2024-08-30T09:11:16.117 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.115+0000 7fadb8051740 1 -- 172.21.15.42:0/234930302 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55ba2fd7ce40 con 0x55ba2f9a24d0 2024-08-30T09:11:16.118 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.115+0000 7fad8eff5640 1 -- 172.21.15.42:0/234930302 <== mon.2 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fada40ac0f0 con 0x55ba2f9a24d0 2024-08-30T09:11:16.118 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.115+0000 7fad8eff5640 1 -- 172.21.15.42:0/234930302 <== mon.2 v2:172.21.15.167:3300/0 5 ==== fsmap(e 41) ==== 8362+0+0 (secure 0 0 0) 0x7fada004acb0 con 0x55ba2f9a24d0 2024-08-30T09:11:16.118 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.115+0000 7fadb8051740 10 client.27002 fetch_fsmap finished waiting for FSMap version 41 2024-08-30T09:11:16.118 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.115+0000 7fadb8051740 10 client.27002 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-30T09:11:16.119 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.115+0000 7fadb8051740 20 client.27002 populate_metadata read hostname 'smithi042' 2024-08-30T09:11:16.119 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.115+0000 7fadb8051740 1 --2- 172.21.15.42:0/234930302 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55ba2f981d80 0x55ba2fd80510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:16.119 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.115+0000 7fadb8051740 4 client.27002 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-30T09:11:16.119 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.115+0000 7fadb8051740 1 -- 172.21.15.42:0/234930302 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ba2fcddf50 con 0x55ba2f981d80 2024-08-30T09:11:16.119 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.116+0000 7fadaad9d640 1 --2- 172.21.15.42:0/234930302 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55ba2f981d80 0x55ba2fd80510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:16.120 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.116+0000 7fad8eff5640 10 client.27002 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:11:16.120 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.116+0000 7fadaad9d640 1 --2- 172.21.15.42:0/234930302 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55ba2f981d80 0x55ba2fd80510 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f9ec79a240a86b8 server_cookie=cf9b1f30e3d07c9b in_seq=0 out_seq=0 2024-08-30T09:11:16.120 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.117+0000 7fad8eff5640 1 -- 172.21.15.42:0/234930302 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55ba2fcddf50 con 0x55ba2f981d80 2024-08-30T09:11:16.120 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.117+0000 7fad8eff5640 10 client.27002 handle_command_reply: tid=0 2024-08-30T09:11:16.120 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.117+0000 7fadb8051740 1 client.27002 shutdown 2024-08-30T09:11:16.121 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.117+0000 7fadb8051740 1 -- 172.21.15.42:0/234930302 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ba2f9a24d0 msgr2=0x55ba2fd73310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:11:16.121 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.117+0000 7fadb8051740 1 --2- 172.21.15.42:0/234930302 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ba2f9a24d0 0x55ba2fd73310 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fada0009230 tx=0x7fada002a440 comp rx=0 tx=0).stop 2024-08-30T09:11:16.121 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.117+0000 7fadb8051740 1 -- 172.21.15.42:0/234930302 shutdown_connections 2024-08-30T09:11:16.121 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.117+0000 7fadb8051740 1 --2- 172.21.15.42:0/234930302 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55ba2f981d80 0x55ba2fd80510 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:16.121 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.117+0000 7fadb8051740 1 --2- 172.21.15.42:0/234930302 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ba2f9c31b0 0x55ba2fd73e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:16.121 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.117+0000 7fadb8051740 1 --2- 172.21.15.42:0/234930302 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ba2f9a24d0 0x55ba2fd73310 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:16.121 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.117+0000 7fadb8051740 1 --2- 172.21.15.42:0/234930302 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ba2f99a2f0 0x55ba2fd72c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:16.121 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.117+0000 7fadb8051740 1 -- 172.21.15.42:0/234930302 >> 172.21.15.42:0/234930302 conn(0x55ba2f9b7300 msgr2=0x55ba2fd63170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:11:16.121 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.117+0000 7fad8eff5640 0 client.27002 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:11:16.121 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.117+0000 7fadb8051740 1 -- 172.21.15.42:0/234930302 shutdown_connections 2024-08-30T09:11:16.121 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.117+0000 7fadb8051740 1 -- 172.21.15.42:0/234930302 wait complete. 2024-08-30T09:11:16.121 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.117+0000 7fadb8051740 20 client.27002 trim_cache size 0 max 16384 2024-08-30T09:11:16.143 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.139+0000 7fadb8051740 1 Processor -- start 2024-08-30T09:11:16.143 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.139+0000 7fadb8051740 1 -- start start 2024-08-30T09:11:16.143 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.140+0000 7fadb8051740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ba2f99a2f0 0x55ba2fd68d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:16.143 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.140+0000 7fadb8051740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ba2f9a24d0 0x55ba2fd71380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:16.143 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.140+0000 7fadb8051740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ba2f9c31b0 0x55ba2fd718c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:16.143 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.140+0000 7fadb8051740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fadac1bdc80 con 0x55ba2f99a2f0 2024-08-30T09:11:16.143 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.140+0000 7fadb8051740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fadac1bb4f0 con 0x55ba2f9a24d0 2024-08-30T09:11:16.143 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.140+0000 7fadb8051740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fadac1b9340 con 0x55ba2f9c31b0 2024-08-30T09:11:16.144 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.140+0000 7fadaad9d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ba2f99a2f0 0x55ba2fd68d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:16.145 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.140+0000 7fadaa59c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ba2f9a24d0 0x55ba2fd71380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:16.145 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.140+0000 7fadab59e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ba2f9c31b0 0x55ba2fd718c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:16.145 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.140+0000 7fadaa59c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ba2f9a24d0 0x55ba2fd71380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:50188/0 (socket says 172.21.15.42:50188) 2024-08-30T09:11:16.145 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.140+0000 7fadaa59c640 1 -- 172.21.15.42:0/1676070485 learned_addr learned my addr 172.21.15.42:0/1676070485 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:11:16.145 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.140+0000 7fadaad9d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ba2f99a2f0 0x55ba2fd68d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43050/0 (socket says 172.21.15.42:43050) 2024-08-30T09:11:16.145 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.141+0000 7fadaa59c640 1 -- 172.21.15.42:0/1676070485 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ba2f9c31b0 msgr2=0x55ba2fd718c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:11:16.145 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.141+0000 7fadaa59c640 1 --2- 172.21.15.42:0/1676070485 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ba2f9c31b0 0x55ba2fd718c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:16.145 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.141+0000 7fadaa59c640 1 -- 172.21.15.42:0/1676070485 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ba2f99a2f0 msgr2=0x55ba2fd68d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:11:16.145 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.141+0000 7fadaa59c640 1 --2- 172.21.15.42:0/1676070485 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ba2f99a2f0 0x55ba2fd68d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:16.145 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.141+0000 7fadaa59c640 1 -- 172.21.15.42:0/1676070485 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba2fd71110 con 0x55ba2f9a24d0 2024-08-30T09:11:16.145 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.141+0000 7fadaad9d640 1 --2- 172.21.15.42:0/1676070485 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ba2f99a2f0 0x55ba2fd68d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-30T09:11:16.146 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.141+0000 7fadaa59c640 1 --2- 172.21.15.42:0/1676070485 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ba2f9a24d0 0x55ba2fd71380 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fada002aa00 tx=0x7fada005c8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=254e96017f7aca80 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:11:16.146 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.141+0000 7fad90ff9640 1 -- 172.21.15.42:0/1676070485 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fada0061070 con 0x55ba2f9a24d0 2024-08-30T09:11:16.146 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.141+0000 7fad90ff9640 1 -- 172.21.15.42:0/1676070485 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fada002ae40 con 0x55ba2f9a24d0 2024-08-30T09:11:16.146 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.141+0000 7fad90ff9640 1 -- 172.21.15.42:0/1676070485 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fada0031830 con 0x55ba2f9a24d0 2024-08-30T09:11:16.149 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.142+0000 7fadb8051740 1 -- 172.21.15.42:0/1676070485 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ba2f9a24d0 msgr2=0x55ba2fd71380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:11:16.149 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.143+0000 7fadb8051740 1 --2- 172.21.15.42:0/1676070485 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ba2f9a24d0 0x55ba2fd71380 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fada002aa00 tx=0x7fada005c8f0 comp rx=0 tx=0).stop 2024-08-30T09:11:16.149 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.143+0000 7fadb8051740 1 -- 172.21.15.42:0/1676070485 shutdown_connections 2024-08-30T09:11:16.149 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.143+0000 7fadb8051740 1 --2- 172.21.15.42:0/1676070485 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ba2f9c31b0 0x55ba2fd718c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:16.149 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.143+0000 7fadb8051740 1 --2- 172.21.15.42:0/1676070485 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ba2f9a24d0 0x55ba2fd71380 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:16.149 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.143+0000 7fadb8051740 1 --2- 172.21.15.42:0/1676070485 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ba2f99a2f0 0x55ba2fd68d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:16.149 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.143+0000 7fadb8051740 1 -- 172.21.15.42:0/1676070485 >> 172.21.15.42:0/1676070485 conn(0x55ba2f9b7300 msgr2=0x55ba2fd64550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:11:16.149 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.143+0000 7fadb8051740 1 -- 172.21.15.42:0/1676070485 shutdown_connections 2024-08-30T09:11:16.149 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.143+0000 7fadb8051740 1 -- 172.21.15.42:0/1676070485 wait complete. 2024-08-30T09:11:16.149 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.143+0000 7fadb8051740 1 -- start start 2024-08-30T09:11:16.149 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.143+0000 7fadb8051740 1 Processor -- start 2024-08-30T09:11:16.150 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.144+0000 7fadb8051740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ba2f99a2f0 0x55ba2fd796b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:16.150 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.144+0000 7fadb8051740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ba2f9a24d0 0x55ba2fd79bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:16.150 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.144+0000 7fadb8051740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ba2f9c31b0 0x55ba2fd7a130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:16.150 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.144+0000 7fadb8051740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fadac1b99b0 con 0x55ba2f9a24d0 2024-08-30T09:11:16.150 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.144+0000 7fadb8051740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fadac1bae20 con 0x55ba2f99a2f0 2024-08-30T09:11:16.150 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.144+0000 7fadb8051740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fadac1ba080 con 0x55ba2f9c31b0 2024-08-30T09:11:16.150 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.144+0000 7fadb8051740 10 client.0 _ll_register_callbacks cb 0x7fada0003670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:11:16.150 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.144+0000 7fadaa59c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ba2f9a24d0 0x55ba2fd79bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:16.150 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.144+0000 7fadaa59c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ba2f9a24d0 0x55ba2fd79bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43064/0 (socket says 172.21.15.42:43064) 2024-08-30T09:11:16.150 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.144+0000 7fadaa59c640 1 -- 172.21.15.42:0/1182776777 learned_addr learned my addr 172.21.15.42:0/1182776777 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:11:16.150 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.144+0000 7fadaad9d640 1 --2- 172.21.15.42:0/1182776777 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ba2f99a2f0 0x55ba2fd796b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:16.150 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.144+0000 7fadab59e640 1 --2- 172.21.15.42:0/1182776777 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ba2f9c31b0 0x55ba2fd7a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:16.151 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.144+0000 7fadaa59c640 1 -- 172.21.15.42:0/1182776777 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ba2f9c31b0 msgr2=0x55ba2fd7a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:11:16.151 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.144+0000 7fadaa59c640 1 --2- 172.21.15.42:0/1182776777 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ba2f9c31b0 0x55ba2fd7a130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:16.151 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.144+0000 7fadaa59c640 1 -- 172.21.15.42:0/1182776777 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ba2f99a2f0 msgr2=0x55ba2fd796b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:11:16.151 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.144+0000 7fadaa59c640 1 --2- 172.21.15.42:0/1182776777 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ba2f99a2f0 0x55ba2fd796b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:16.151 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.144+0000 7fadaa59c640 1 -- 172.21.15.42:0/1182776777 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ba2fd78420 con 0x55ba2f9a24d0 2024-08-30T09:11:16.151 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.144+0000 7fadab59e640 1 --2- 172.21.15.42:0/1182776777 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ba2f9c31b0 0x55ba2fd7a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-30T09:11:16.151 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.145+0000 7fadaad9d640 1 --2- 172.21.15.42:0/1182776777 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ba2f99a2f0 0x55ba2fd796b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-30T09:11:16.151 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.145+0000 7fadaa59c640 1 --2- 172.21.15.42:0/1182776777 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ba2f9a24d0 0x55ba2fd79bf0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fada0033040 tx=0x7fada0031860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18f4062795b0592a server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:11:16.151 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.145+0000 7fad8eff5640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-08-30T09:11:16.151 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.145+0000 7fad8eff5640 1 -- 172.21.15.42:0/1182776777 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fada0061040 con 0x55ba2f9a24d0 2024-08-30T09:11:16.151 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.145+0000 7fad8eff5640 1 -- 172.21.15.42:0/1182776777 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7fada005cc90 con 0x55ba2f9a24d0 2024-08-30T09:11:16.151 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.145+0000 7fadb8051740 1 -- 172.21.15.42:0/1182776777 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fada002ae40 con 0x55ba2f9a24d0 2024-08-30T09:11:16.151 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.145+0000 7fad8eff5640 1 -- 172.21.15.42:0/1182776777 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fada007a050 con 0x55ba2f9a24d0 2024-08-30T09:11:16.152 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.145+0000 7fadb8051740 10 client.17190 fetch_fsmap learned FSMap version 41 2024-08-30T09:11:16.152 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.145+0000 7fadb8051740 1 -- 172.21.15.42:0/1182776777 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55ba2fdeb2f0 con 0x55ba2f9a24d0 2024-08-30T09:11:16.152 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.146+0000 7fad8eff5640 1 -- 172.21.15.42:0/1182776777 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fada005c710 con 0x55ba2f9a24d0 2024-08-30T09:11:16.153 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.146+0000 7fad8eff5640 1 -- 172.21.15.42:0/1182776777 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 41) ==== 8362+0+0 (secure 0 0 0) 0x7fada0066460 con 0x55ba2f9a24d0 2024-08-30T09:11:16.153 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.147+0000 7fadb8051740 10 client.17190 fetch_fsmap finished waiting for FSMap version 41 2024-08-30T09:11:16.153 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.147+0000 7fadb8051740 10 client.17190 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-30T09:11:16.153 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.147+0000 7fadb8051740 20 client.17190 populate_metadata read hostname 'smithi042' 2024-08-30T09:11:16.153 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.147+0000 7fadb8051740 1 --2- 172.21.15.42:0/1182776777 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55ba2f9571c0 0x55ba2fdedb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:16.153 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.147+0000 7fadb8051740 4 client.17190 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-30T09:11:16.153 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.147+0000 7fadaad9d640 1 --2- 172.21.15.42:0/1182776777 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55ba2f9571c0 0x55ba2fdedb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:16.153 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.147+0000 7fadb8051740 1 -- 172.21.15.42:0/1182776777 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7fada002ae40 con 0x55ba2f9571c0 2024-08-30T09:11:16.154 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.148+0000 7fadaad9d640 1 --2- 172.21.15.42:0/1182776777 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55ba2f9571c0 0x55ba2fdedb80 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=100ddbdfbb853f39 server_cookie=dfedfc4782e73b00 in_seq=0 out_seq=0 2024-08-30T09:11:16.154 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.148+0000 7fad8eff5640 10 client.17190 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:11:16.166 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.162+0000 7fad8eff5640 1 -- 172.21.15.42:0/1182776777 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7fada002ae40 con 0x55ba2f9571c0 2024-08-30T09:11:16.166 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.162+0000 7fad8eff5640 10 client.17190 handle_command_reply: tid=0 2024-08-30T09:11:16.166 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.162+0000 7fadb8051740 1 client.17190 shutdown 2024-08-30T09:11:16.166 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.162+0000 7fadb8051740 1 -- 172.21.15.42:0/1182776777 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ba2f9a24d0 msgr2=0x55ba2fd79bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:11:16.166 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.162+0000 7fadb8051740 1 --2- 172.21.15.42:0/1182776777 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ba2f9a24d0 0x55ba2fd79bf0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fada0033040 tx=0x7fada0031860 comp rx=0 tx=0).stop 2024-08-30T09:11:16.166 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.163+0000 7fadb8051740 1 -- 172.21.15.42:0/1182776777 shutdown_connections 2024-08-30T09:11:16.166 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.163+0000 7fadb8051740 1 --2- 172.21.15.42:0/1182776777 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55ba2f9571c0 0x55ba2fdedb80 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:16.166 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.163+0000 7fadb8051740 1 --2- 172.21.15.42:0/1182776777 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ba2f9c31b0 0x55ba2fd7a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:16.166 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.163+0000 7fadb8051740 1 --2- 172.21.15.42:0/1182776777 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ba2f9a24d0 0x55ba2fd79bf0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:16.166 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.163+0000 7fadb8051740 1 --2- 172.21.15.42:0/1182776777 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ba2f99a2f0 0x55ba2fd796b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:16.166 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.163+0000 7fadb8051740 1 -- 172.21.15.42:0/1182776777 >> 172.21.15.42:0/1182776777 conn(0x55ba2f9b7300 msgr2=0x55ba2fd71380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:11:16.167 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.163+0000 7fad8eff5640 0 client.17190 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:11:16.167 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.163+0000 7fadb8051740 1 -- 172.21.15.42:0/1182776777 shutdown_connections 2024-08-30T09:11:16.167 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.163+0000 7fadb8051740 1 -- 172.21.15.42:0/1182776777 wait complete. 2024-08-30T09:11:16.167 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.163+0000 7fadb8051740 20 client.17190 trim_cache size 0 max 16384 2024-08-30T09:11:16.167 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-08-30T09:11:16.167 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2024-08-30T09:11:16.167 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-08-30T09:11:16.167 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-08-30T09:11:16.168 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.163+0000 7fadb10de640 1 -- 172.21.15.42:0/773849130 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fad940b4cc0 msgr2=0x7fad940b71d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:11:16.168 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.163+0000 7fadb10de640 1 --2- 172.21.15.42:0/773849130 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fad940b4cc0 0x7fad940b71d0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fada002aa30 tx=0x7fada0000bc0 comp rx=0 tx=0).stop 2024-08-30T09:11:16.168 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.163+0000 7fadb10de640 1 -- 172.21.15.42:0/773849130 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fadac104e80 msgr2=0x7fadac112cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:11:16.168 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.164+0000 7fadb10de640 1 --2- 172.21.15.42:0/773849130 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fadac104e80 0x7fadac112cb0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fada400eb10 tx=0x7fada400c2d0 comp rx=0 tx=0).stop 2024-08-30T09:11:16.168 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.164+0000 7fadb10de640 1 -- 172.21.15.42:0/773849130 shutdown_connections 2024-08-30T09:11:16.168 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.164+0000 7fadb10de640 1 --2- 172.21.15.42:0/773849130 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fad940b4cc0 0x7fad940b71d0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:16.169 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.164+0000 7fadb10de640 1 --2- 172.21.15.42:0/773849130 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fadac104e80 0x7fadac112cb0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:16.169 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.164+0000 7fadb10de640 1 --2- 172.21.15.42:0/773849130 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fadac1044e0 0x7fadac114880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:16.169 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.164+0000 7fadb10de640 1 --2- 172.21.15.42:0/773849130 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fadac103460 0x7fadac114210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:16.169 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.164+0000 7fadb10de640 1 -- 172.21.15.42:0/773849130 >> 172.21.15.42:0/773849130 conn(0x7fadac0fed80 msgr2=0x7fadac107800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:11:16.169 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.164+0000 7fadb10de640 1 -- 172.21.15.42:0/773849130 shutdown_connections 2024-08-30T09:11:16.169 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:16.164+0000 7fadb10de640 1 -- 172.21.15.42:0/773849130 wait complete. 2024-08-30T09:11:16.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:11:16 smithi167 ceph-mon[22063]: pgmap v688: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 6.8 MiB/s wr, 512 op/s 2024-08-30T09:11:16.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:11:16 smithi042 ceph-mon[18652]: pgmap v688: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 6.8 MiB/s wr, 512 op/s 2024-08-30T09:11:16.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:11:16 smithi190 ceph-mon[22001]: pgmap v688: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 6.8 MiB/s wr, 512 op/s 2024-08-30T09:11:18.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:11:18 smithi167 ceph-mon[22063]: pgmap v689: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 6.3 MiB/s wr, 532 op/s 2024-08-30T09:11:18.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:11:18 smithi042 ceph-mon[18652]: pgmap v689: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 6.3 MiB/s wr, 532 op/s 2024-08-30T09:11:18.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:11:18 smithi190 ceph-mon[22001]: pgmap v689: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 6.3 MiB/s wr, 532 op/s 2024-08-30T09:11:20.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:11:20 smithi167 ceph-mon[22063]: pgmap v690: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 7.2 MiB/s wr, 559 op/s 2024-08-30T09:11:20.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:11:20 smithi042 ceph-mon[18652]: pgmap v690: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 7.2 MiB/s wr, 559 op/s 2024-08-30T09:11:20.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:11:20 smithi190 ceph-mon[22001]: pgmap v690: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 7.2 MiB/s wr, 559 op/s 2024-08-30T09:11:20.953 INFO:teuthology.orchestra.run.smithi042.stderr:progress: 635.0 s, 625.2 tps, lat 164.483 ms stddev 199.830 2024-08-30T09:11:22.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:11:22 smithi167 ceph-mon[22063]: pgmap v691: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 5.9 MiB/s wr, 416 op/s 2024-08-30T09:11:22.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:11:22 smithi042 ceph-mon[18652]: pgmap v691: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 5.9 MiB/s wr, 416 op/s 2024-08-30T09:11:22.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:11:22 smithi190 ceph-mon[22001]: pgmap v691: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 5.9 MiB/s wr, 416 op/s 2024-08-30T09:11:24.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:11:24 smithi167 ceph-mon[22063]: pgmap v692: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.2 MiB/s wr, 563 op/s 2024-08-30T09:11:24.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:11:24 smithi042 ceph-mon[18652]: pgmap v692: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.2 MiB/s wr, 563 op/s 2024-08-30T09:11:24.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:11:24 smithi190 ceph-mon[22001]: pgmap v692: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.2 MiB/s wr, 563 op/s 2024-08-30T09:11:25.953 INFO:teuthology.orchestra.run.smithi042.stderr:progress: 640.0 s, 521.2 tps, lat 186.363 ms stddev 224.997 2024-08-30T09:11:26.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:11:26 smithi167 ceph-mon[22063]: pgmap v693: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 6.1 MiB/s wr, 462 op/s 2024-08-30T09:11:26.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:11:26 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-30T09:11:26.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:11:26 smithi042 ceph-mon[18652]: pgmap v693: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 6.1 MiB/s wr, 462 op/s 2024-08-30T09:11:26.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:11:26 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-30T09:11:26.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:11:26 smithi190 ceph-mon[22001]: pgmap v693: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 6.1 MiB/s wr, 462 op/s 2024-08-30T09:11:26.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:11:26 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-30T09:11:27.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:11:27 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T09:11:27.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:11:27 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T09:11:27.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:11:27 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T09:11:27.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:11:27 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-30T09:11:27.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:11:27 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T09:11:27.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:11:27 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T09:11:27.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:11:27 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T09:11:27.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:11:27 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T09:11:27.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:11:27 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-30T09:11:27.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:11:27 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T09:11:27.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:11:27 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T09:11:27.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:11:27 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T09:11:27.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:11:27 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T09:11:27.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:11:27 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-30T09:11:27.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:11:27 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T09:11:28.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:11:28 smithi167 ceph-mon[22063]: pgmap v694: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 6.2 MiB/s wr, 473 op/s 2024-08-30T09:11:28.884 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:11:28 smithi042 ceph-mon[18652]: pgmap v694: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 6.2 MiB/s wr, 473 op/s 2024-08-30T09:11:28.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:11:28 smithi190 ceph-mon[22001]: pgmap v694: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 6.2 MiB/s wr, 473 op/s 2024-08-30T09:11:29.352 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.347+0000 7fe24ce3f640 1 -- 172.21.15.42:0/3180502349 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe24810ab40 msgr2=0x7fe24810b010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:11:29.352 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.347+0000 7fe24ce3f640 1 --2- 172.21.15.42:0/3180502349 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe24810ab40 0x7fe24810b010 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fe238009900 tx=0x7fe23802e830 comp rx=0 tx=0).stop 2024-08-30T09:11:29.354 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.350+0000 7fe24ce3f640 1 -- 172.21.15.42:0/3180502349 shutdown_connections 2024-08-30T09:11:29.354 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.350+0000 7fe24ce3f640 1 --2- 172.21.15.42:0/3180502349 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe248111d30 0x7fe248114150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:29.354 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.350+0000 7fe24ce3f640 1 --2- 172.21.15.42:0/3180502349 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe24810ab40 0x7fe24810b010 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:29.354 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.350+0000 7fe24ce3f640 1 --2- 172.21.15.42:0/3180502349 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe24810a160 0x7fe24810a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:29.354 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.350+0000 7fe24ce3f640 1 -- 172.21.15.42:0/3180502349 >> 172.21.15.42:0/3180502349 conn(0x7fe2480705b0 msgr2=0x7fe248072a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:11:29.358 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.355+0000 7fe24ce3f640 1 -- 172.21.15.42:0/3180502349 shutdown_connections 2024-08-30T09:11:29.363 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.359+0000 7fe24ce3f640 1 -- 172.21.15.42:0/3180502349 wait complete. 2024-08-30T09:11:29.363 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.360+0000 7fe24ce3f640 1 Processor -- start 2024-08-30T09:11:29.366 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.363+0000 7fe24ce3f640 1 -- start start 2024-08-30T09:11:29.367 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.363+0000 7fe24ce3f640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe24810a160 0x7fe2481a02a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:29.367 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.363+0000 7fe24ce3f640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe24810ab40 0x7fe2481a0910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:29.367 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.363+0000 7fe24ce3f640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe248111d30 0x7fe2481a14c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:29.367 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.363+0000 7fe24ce3f640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fe248115710 con 0x7fe24810ab40 2024-08-30T09:11:29.367 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.363+0000 7fe24ce3f640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe248115590 con 0x7fe248111d30 2024-08-30T09:11:29.367 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.363+0000 7fe24ce3f640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fe248115890 con 0x7fe24810a160 2024-08-30T09:11:29.369 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.366+0000 7fe245d9b640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe24810ab40 0x7fe2481a0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:29.369 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.366+0000 7fe245d9b640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe24810ab40 0x7fe2481a0910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:59656/0 (socket says 172.21.15.42:59656) 2024-08-30T09:11:29.369 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.366+0000 7fe245d9b640 1 -- 172.21.15.42:0/1817944717 learned_addr learned my addr 172.21.15.42:0/1817944717 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:11:29.370 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.367+0000 7fe24659c640 1 --2- 172.21.15.42:0/1817944717 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe24810a160 0x7fe2481a02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:29.371 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.368+0000 7fe246d9d640 1 --2- 172.21.15.42:0/1817944717 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe248111d30 0x7fe2481a14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:29.373 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.369+0000 7fe245d9b640 1 -- 172.21.15.42:0/1817944717 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe24810a160 msgr2=0x7fe2481a02a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:11:29.373 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.370+0000 7fe245d9b640 1 --2- 172.21.15.42:0/1817944717 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe24810a160 0x7fe2481a02a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:29.373 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.370+0000 7fe245d9b640 1 -- 172.21.15.42:0/1817944717 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe248111d30 msgr2=0x7fe2481a14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:11:29.373 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.370+0000 7fe245d9b640 1 --2- 172.21.15.42:0/1817944717 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe248111d30 0x7fe2481a14c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:29.373 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.370+0000 7fe245d9b640 1 -- 172.21.15.42:0/1817944717 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2481a1ce0 con 0x7fe24810ab40 2024-08-30T09:11:29.375 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.373+0000 7fe245d9b640 1 --2- 172.21.15.42:0/1817944717 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe24810ab40 0x7fe2481a0910 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fe238036520 tx=0x7fe2380043d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:11:29.384 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.379+0000 7fe23f7fe640 1 -- 172.21.15.42:0/1817944717 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe238042020 con 0x7fe24810ab40 2024-08-30T09:11:29.384 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.379+0000 7fe23f7fe640 1 -- 172.21.15.42:0/1817944717 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe238003ca0 con 0x7fe24810ab40 2024-08-30T09:11:29.384 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.379+0000 7fe23f7fe640 1 -- 172.21.15.42:0/1817944717 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe23803a070 con 0x7fe24810ab40 2024-08-30T09:11:29.384 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.381+0000 7fe24ce3f640 1 -- 172.21.15.42:0/1817944717 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2481a1f70 con 0x7fe24810ab40 2024-08-30T09:11:29.384 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.381+0000 7fe24ce3f640 1 -- 172.21.15.42:0/1817944717 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2481a24b0 con 0x7fe24810ab40 2024-08-30T09:11:29.388 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.385+0000 7fe253e89740 1 Processor -- start 2024-08-30T09:11:29.390 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.386+0000 7fe253e89740 1 -- start start 2024-08-30T09:11:29.390 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.386+0000 7fe253e89740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557af8e111b0 0x557af91b7180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:29.390 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.386+0000 7fe253e89740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557af8de82f0 0x557af91b77f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:29.390 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.386+0000 7fe253e89740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557af8df04d0 0x557af91b8340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:29.390 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.386+0000 7fe253e89740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x557af912a000 con 0x557af8e111b0 2024-08-30T09:11:29.390 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.386+0000 7fe253e89740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x557af9119590 con 0x557af8de82f0 2024-08-30T09:11:29.390 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.386+0000 7fe253e89740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x557af911d540 con 0x557af8df04d0 2024-08-30T09:11:29.395 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.388+0000 7fe23f7fe640 1 -- 172.21.15.42:0/1817944717 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7fe238045020 con 0x7fe24810ab40 2024-08-30T09:11:29.395 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.389+0000 7fe24659c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557af8de82f0 0x557af91b77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:29.395 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.390+0000 7fe23f7fe640 1 --2- 172.21.15.42:0/1817944717 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fe2340b4d30 0x7fe2340b7240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:29.395 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.390+0000 7fe23f7fe640 1 -- 172.21.15.42:0/1817944717 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7fe238106200 con 0x7fe24810ab40 2024-08-30T09:11:29.395 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.392+0000 7fe245d9b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557af8df04d0 0x557af91b8340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:29.395 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.392+0000 7fe245d9b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557af8df04d0 0x557af91b8340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:56880/0 (socket says 172.21.15.42:56880) 2024-08-30T09:11:29.396 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.393+0000 7fe24659c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557af8de82f0 0x557af91b77f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:42486/0 (socket says 172.21.15.42:42486) 2024-08-30T09:11:29.396 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.393+0000 7fe24659c640 1 -- 172.21.15.42:0/4255908701 learned_addr learned my addr 172.21.15.42:0/4255908701 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:11:29.399 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.396+0000 7fe246d9d640 1 --2- 172.21.15.42:0/4255908701 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557af8e111b0 0x557af91b7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:29.399 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.396+0000 7fe246d9d640 1 --2- 172.21.15.42:0/1817944717 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fe2340b4d30 0x7fe2340b7240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:29.399 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.396+0000 7fe245d9b640 1 -- 172.21.15.42:0/4255908701 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557af8de82f0 msgr2=0x557af91b77f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:11:29.399 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.396+0000 7fe245d9b640 1 --2- 172.21.15.42:0/4255908701 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557af8de82f0 0x557af91b77f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:29.399 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.396+0000 7fe245d9b640 1 -- 172.21.15.42:0/4255908701 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557af8e111b0 msgr2=0x557af91b7180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:11:29.400 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.396+0000 7fe245d9b640 1 --2- 172.21.15.42:0/4255908701 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557af8e111b0 0x557af91b7180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:29.400 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.396+0000 7fe245d9b640 1 -- 172.21.15.42:0/4255908701 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557af8e7c790 con 0x557af8df04d0 2024-08-30T09:11:29.405 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.401+0000 7fe245d9b640 1 --2- 172.21.15.42:0/4255908701 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557af8df04d0 0x557af91b8340 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fe238000c00 tx=0x7fe238009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=20a416cfe71b6b42 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:11:29.409 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.405+0000 7fe246d9d640 1 --2- 172.21.15.42:0/1817944717 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fe2340b4d30 0x7fe2340b7240 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fe230009fd0 tx=0x7fe230009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:11:29.409 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.406+0000 7fe23cff9640 1 -- 172.21.15.42:0/4255908701 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2380cc350 con 0x557af8df04d0 2024-08-30T09:11:29.409 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.406+0000 7fe23cff9640 1 -- 172.21.15.42:0/4255908701 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe2380ccf40 con 0x557af8df04d0 2024-08-30T09:11:29.410 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.407+0000 7fe253e89740 1 -- 172.21.15.42:0/4255908701 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557af8df04d0 msgr2=0x557af91b8340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:11:29.410 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.407+0000 7fe253e89740 1 --2- 172.21.15.42:0/4255908701 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557af8df04d0 0x557af91b8340 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fe238000c00 tx=0x7fe238009260 comp rx=0 tx=0).stop 2024-08-30T09:11:29.417 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.414+0000 7fe253e89740 1 -- 172.21.15.42:0/4255908701 shutdown_connections 2024-08-30T09:11:29.417 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.414+0000 7fe253e89740 1 --2- 172.21.15.42:0/4255908701 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557af8df04d0 0x557af91b8340 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:29.418 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.414+0000 7fe253e89740 1 --2- 172.21.15.42:0/4255908701 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557af8de82f0 0x557af91b77f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:29.418 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.414+0000 7fe253e89740 1 --2- 172.21.15.42:0/4255908701 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557af8e111b0 0x557af91b7180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:29.418 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.414+0000 7fe253e89740 1 -- 172.21.15.42:0/4255908701 >> 172.21.15.42:0/4255908701 conn(0x557af8e05300 msgr2=0x557af91b2c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:11:29.422 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.418+0000 7fe253e89740 1 -- 172.21.15.42:0/4255908701 shutdown_connections 2024-08-30T09:11:29.422 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.418+0000 7fe253e89740 1 -- 172.21.15.42:0/4255908701 wait complete. 2024-08-30T09:11:29.422 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.419+0000 7fe253e89740 1 -- start start 2024-08-30T09:11:29.422 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.419+0000 7fe253e89740 1 Processor -- start 2024-08-30T09:11:29.423 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.419+0000 7fe253e89740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557af8de82f0 0x557af91c0770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:29.423 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.419+0000 7fe253e89740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557af8df04d0 0x557af91c0e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:29.423 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.419+0000 7fe253e89740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557af8e111b0 0x557af91c19a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:29.423 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.419+0000 7fe253e89740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x557af9052ab0 con 0x557af8df04d0 2024-08-30T09:11:29.423 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.419+0000 7fe253e89740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x557af90a0680 con 0x557af8de82f0 2024-08-30T09:11:29.423 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.419+0000 7fe253e89740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x557af911d540 con 0x557af8e111b0 2024-08-30T09:11:29.423 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.419+0000 7fe253e89740 10 client.0 _ll_register_callbacks cb 0x557af8e28a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:11:29.423 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.420+0000 7fe24659c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557af8de82f0 0x557af91c0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:29.423 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.420+0000 7fe24659c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557af8de82f0 0x557af91c0770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:42492/0 (socket says 172.21.15.42:42492) 2024-08-30T09:11:29.423 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.420+0000 7fe24659c640 1 -- 172.21.15.42:0/2591814670 learned_addr learned my addr 172.21.15.42:0/2591814670 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:11:29.423 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.420+0000 7fe246d9d640 1 --2- 172.21.15.42:0/2591814670 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557af8e111b0 0x557af91c19a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:29.424 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.421+0000 7fe24659c640 1 -- 172.21.15.42:0/2591814670 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557af8e111b0 msgr2=0x557af91c19a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:11:29.424 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.421+0000 7fe24659c640 1 --2- 172.21.15.42:0/2591814670 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557af8e111b0 0x557af91c19a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:29.425 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.421+0000 7fe24659c640 1 -- 172.21.15.42:0/2591814670 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557af8df04d0 msgr2=0x557af91c0e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:11:29.425 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.421+0000 7fe24659c640 1 --2- 172.21.15.42:0/2591814670 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557af8df04d0 0x557af91c0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:29.425 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.421+0000 7fe24659c640 1 -- 172.21.15.42:0/2591814670 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557af8ef5620 con 0x557af8de82f0 2024-08-30T09:11:29.425 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.421+0000 7fe24659c640 1 --2- 172.21.15.42:0/2591814670 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557af8de82f0 0x557af91c0770 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fe240005bc0 tx=0x7fe240013b70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=81c1dbfa359cd4e8 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:11:29.425 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.421+0000 7fe22dffb640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-30T09:11:29.425 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.421+0000 7fe22dffb640 1 -- 172.21.15.42:0/2591814670 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe240011dd0 con 0x557af8de82f0 2024-08-30T09:11:29.425 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.421+0000 7fe22dffb640 1 -- 172.21.15.42:0/2591814670 <== mon.2 v2:172.21.15.167:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7fe240004020 con 0x557af8de82f0 2024-08-30T09:11:29.425 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.421+0000 7fe253e89740 1 -- 172.21.15.42:0/2591814670 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe2380ccf40 con 0x557af8de82f0 2024-08-30T09:11:29.425 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.422+0000 7fe22dffb640 1 -- 172.21.15.42:0/2591814670 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fe240004220 con 0x557af8de82f0 2024-08-30T09:11:29.425 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.423+0000 7fe253e89740 10 client.27020 fetch_fsmap learned FSMap version 41 2024-08-30T09:11:29.426 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.423+0000 7fe253e89740 1 -- 172.21.15.42:0/2591814670 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x557af91caa10 con 0x557af8de82f0 2024-08-30T09:11:29.429 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.426+0000 7fe22dffb640 1 -- 172.21.15.42:0/2591814670 <== mon.2 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe2380ccf40 con 0x557af8de82f0 2024-08-30T09:11:29.429 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.426+0000 7fe22dffb640 1 -- 172.21.15.42:0/2591814670 <== mon.2 v2:172.21.15.167:3300/0 5 ==== fsmap(e 41) ==== 8362+0+0 (secure 0 0 0) 0x7fe24001a070 con 0x557af8de82f0 2024-08-30T09:11:29.429 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.426+0000 7fe253e89740 10 client.27020 fetch_fsmap finished waiting for FSMap version 41 2024-08-30T09:11:29.429 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.426+0000 7fe253e89740 10 client.27020 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-30T09:11:29.429 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.426+0000 7fe253e89740 20 client.27020 populate_metadata read hostname 'smithi042' 2024-08-30T09:11:29.429 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.426+0000 7fe253e89740 1 --2- 172.21.15.42:0/2591814670 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x557af8dcfd80 0x557af91ce250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:29.429 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.426+0000 7fe253e89740 4 client.27020 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-30T09:11:29.429 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.426+0000 7fe253e89740 1 -- 172.21.15.42:0/2591814670 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557af912bf50 con 0x557af8dcfd80 2024-08-30T09:11:29.432 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.430+0000 7fe245d9b640 1 --2- 172.21.15.42:0/2591814670 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x557af8dcfd80 0x557af91ce250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:29.438 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.433+0000 7fe245d9b640 1 --2- 172.21.15.42:0/2591814670 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x557af8dcfd80 0x557af91ce250 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=220d4856a3dc147 server_cookie=5a7d6b1f56137906 in_seq=0 out_seq=0 2024-08-30T09:11:29.438 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.434+0000 7fe22dffb640 10 client.27020 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:11:29.439 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.435+0000 7fe22dffb640 1 -- 172.21.15.42:0/2591814670 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x557af912bf50 con 0x557af8dcfd80 2024-08-30T09:11:29.439 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.435+0000 7fe22dffb640 10 client.27020 handle_command_reply: tid=0 2024-08-30T09:11:29.439 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.436+0000 7fe253e89740 1 client.27020 shutdown 2024-08-30T09:11:29.450 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.446+0000 7fe253e89740 1 -- 172.21.15.42:0/2591814670 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557af8de82f0 msgr2=0x557af91c0770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:11:29.450 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.446+0000 7fe253e89740 1 --2- 172.21.15.42:0/2591814670 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557af8de82f0 0x557af91c0770 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fe240005bc0 tx=0x7fe240013b70 comp rx=0 tx=0).stop 2024-08-30T09:11:29.451 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.448+0000 7fe253e89740 1 -- 172.21.15.42:0/2591814670 shutdown_connections 2024-08-30T09:11:29.451 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.448+0000 7fe253e89740 1 --2- 172.21.15.42:0/2591814670 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x557af8dcfd80 0x557af91ce250 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:29.451 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.448+0000 7fe253e89740 1 --2- 172.21.15.42:0/2591814670 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557af8e111b0 0x557af91c19a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:29.451 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.448+0000 7fe253e89740 1 --2- 172.21.15.42:0/2591814670 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557af8df04d0 0x557af91c0e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:29.451 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.448+0000 7fe253e89740 1 --2- 172.21.15.42:0/2591814670 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557af8de82f0 0x557af91c0770 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:29.451 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.448+0000 7fe253e89740 1 -- 172.21.15.42:0/2591814670 >> 172.21.15.42:0/2591814670 conn(0x557af8e05300 msgr2=0x557af91b1170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:11:29.451 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.448+0000 7fe22dffb640 0 client.27020 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:11:29.455 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.448+0000 7fe253e89740 1 -- 172.21.15.42:0/2591814670 shutdown_connections 2024-08-30T09:11:29.455 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.448+0000 7fe253e89740 1 -- 172.21.15.42:0/2591814670 wait complete. 2024-08-30T09:11:29.455 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.448+0000 7fe253e89740 20 client.27020 trim_cache size 0 max 16384 2024-08-30T09:11:29.489 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.482+0000 7fe253e89740 1 Processor -- start 2024-08-30T09:11:29.489 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.482+0000 7fe253e89740 1 -- start start 2024-08-30T09:11:29.489 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.483+0000 7fe253e89740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557af8de82f0 0x557af91b1880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:29.489 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.483+0000 7fe253e89740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557af8df04d0 0x557af91c8880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:29.489 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.483+0000 7fe253e89740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557af8e111b0 0x557af91bc660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:29.489 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.483+0000 7fe253e89740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fe2481c6310 con 0x557af8de82f0 2024-08-30T09:11:29.489 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.483+0000 7fe253e89740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe2481c3b80 con 0x557af8e111b0 2024-08-30T09:11:29.489 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.483+0000 7fe253e89740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fe2481c19d0 con 0x557af8df04d0 2024-08-30T09:11:29.489 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.483+0000 7fe246d9d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557af8e111b0 0x557af91bc660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:29.489 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.483+0000 7fe245d9b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557af8df04d0 0x557af91c8880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:29.489 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.483+0000 7fe24659c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557af8de82f0 0x557af91b1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:29.489 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.483+0000 7fe246d9d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557af8e111b0 0x557af91bc660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:42502/0 (socket says 172.21.15.42:42502) 2024-08-30T09:11:29.490 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.483+0000 7fe245d9b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557af8df04d0 0x557af91c8880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:56894/0 (socket says 172.21.15.42:56894) 2024-08-30T09:11:29.490 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.483+0000 7fe246d9d640 1 -- 172.21.15.42:0/1086468809 learned_addr learned my addr 172.21.15.42:0/1086468809 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:11:29.490 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.484+0000 7fe245d9b640 1 -- 172.21.15.42:0/1086468809 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557af8e111b0 msgr2=0x557af91bc660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:11:29.490 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.484+0000 7fe245d9b640 1 --2- 172.21.15.42:0/1086468809 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557af8e111b0 0x557af91bc660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:29.490 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.484+0000 7fe245d9b640 1 -- 172.21.15.42:0/1086468809 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557af8de82f0 msgr2=0x557af91b1880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:11:29.490 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.484+0000 7fe245d9b640 1 --2- 172.21.15.42:0/1086468809 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557af8de82f0 0x557af91b1880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:29.490 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.484+0000 7fe245d9b640 1 -- 172.21.15.42:0/1086468809 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557af91ca830 con 0x557af8df04d0 2024-08-30T09:11:29.490 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.484+0000 7fe245d9b640 1 --2- 172.21.15.42:0/1086468809 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557af8df04d0 0x557af91c8880 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fe2380cc120 tx=0x7fe238036420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f43802eec1b4f2bb server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:11:29.490 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.486+0000 7fe22ffff640 1 -- 172.21.15.42:0/1086468809 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2380cc840 con 0x557af8df04d0 2024-08-30T09:11:29.490 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.486+0000 7fe22ffff640 1 -- 172.21.15.42:0/1086468809 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe2380d10b0 con 0x557af8df04d0 2024-08-30T09:11:29.490 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.486+0000 7fe22ffff640 1 -- 172.21.15.42:0/1086468809 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2380ca9f0 con 0x557af8df04d0 2024-08-30T09:11:29.491 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.488+0000 7fe253e89740 1 -- 172.21.15.42:0/1086468809 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557af8df04d0 msgr2=0x557af91c8880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:11:29.491 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.488+0000 7fe253e89740 1 --2- 172.21.15.42:0/1086468809 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557af8df04d0 0x557af91c8880 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fe2380cc120 tx=0x7fe238036420 comp rx=0 tx=0).stop 2024-08-30T09:11:29.491 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.488+0000 7fe253e89740 1 -- 172.21.15.42:0/1086468809 shutdown_connections 2024-08-30T09:11:29.492 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.489+0000 7fe253e89740 1 --2- 172.21.15.42:0/1086468809 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557af8df04d0 0x557af91c8880 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:29.492 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.489+0000 7fe253e89740 1 --2- 172.21.15.42:0/1086468809 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557af8e111b0 0x557af91bc660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:29.492 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.489+0000 7fe253e89740 1 --2- 172.21.15.42:0/1086468809 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557af8de82f0 0x557af91b1880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:29.492 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.489+0000 7fe253e89740 1 -- 172.21.15.42:0/1086468809 >> 172.21.15.42:0/1086468809 conn(0x557af8e05300 msgr2=0x557af91b1170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:11:29.493 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.491+0000 7fe253e89740 1 -- 172.21.15.42:0/1086468809 shutdown_connections 2024-08-30T09:11:29.493 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.491+0000 7fe253e89740 1 -- 172.21.15.42:0/1086468809 wait complete. 2024-08-30T09:11:29.494 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.491+0000 7fe253e89740 1 -- start start 2024-08-30T09:11:29.494 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.492+0000 7fe253e89740 1 Processor -- start 2024-08-30T09:11:29.495 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.492+0000 7fe253e89740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557af8de82f0 0x557af91c7430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:29.495 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.492+0000 7fe253e89740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557af8df04d0 0x557af91c7970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:29.495 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.493+0000 7fe253e89740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557af8e111b0 0x557af9236c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:29.496 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.493+0000 7fe245d9b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557af8df04d0 0x557af91c7970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:29.496 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.493+0000 7fe245d9b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557af8df04d0 0x557af91c7970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:56908/0 (socket says 172.21.15.42:56908) 2024-08-30T09:11:29.496 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.493+0000 7fe246d9d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557af8e111b0 0x557af9236c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:29.496 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.493+0000 7fe246d9d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557af8e111b0 0x557af9236c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:42510/0 (socket says 172.21.15.42:42510) 2024-08-30T09:11:29.496 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.493+0000 7fe245d9b640 1 -- 172.21.15.42:0/1554120382 learned_addr learned my addr 172.21.15.42:0/1554120382 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:11:29.497 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.493+0000 7fe253e89740 1 -- 172.21.15.42:0/1554120382 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fe2481c2040 con 0x557af8de82f0 2024-08-30T09:11:29.497 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.493+0000 7fe24659c640 1 --2- 172.21.15.42:0/1554120382 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557af8de82f0 0x557af91c7430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:29.497 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.493+0000 7fe253e89740 1 -- 172.21.15.42:0/1554120382 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe2481c34b0 con 0x557af8e111b0 2024-08-30T09:11:29.497 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.493+0000 7fe253e89740 1 -- 172.21.15.42:0/1554120382 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fe2481c2710 con 0x557af8df04d0 2024-08-30T09:11:29.497 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.494+0000 7fe253e89740 10 client.0 _ll_register_callbacks cb 0x7fe240009de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:11:29.498 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.494+0000 7fe24659c640 1 -- 172.21.15.42:0/1554120382 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557af8df04d0 msgr2=0x557af91c7970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:11:29.498 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.494+0000 7fe24659c640 1 --2- 172.21.15.42:0/1554120382 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557af8df04d0 0x557af91c7970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:29.498 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.494+0000 7fe24659c640 1 -- 172.21.15.42:0/1554120382 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557af8e111b0 msgr2=0x557af9236c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:11:29.498 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.494+0000 7fe24659c640 1 --2- 172.21.15.42:0/1554120382 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557af8e111b0 0x557af9236c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:29.498 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.494+0000 7fe24659c640 1 -- 172.21.15.42:0/1554120382 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557af91c8190 con 0x557af8de82f0 2024-08-30T09:11:29.498 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.494+0000 7fe24659c640 1 --2- 172.21.15.42:0/1554120382 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557af8de82f0 0x557af91c7430 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fe24001b830 tx=0x7fe24001b530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fe563931a45b2b2 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:11:29.498 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.495+0000 7fe22dffb640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-08-30T09:11:29.498 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.495+0000 7fe22dffb640 1 -- 172.21.15.42:0/1554120382 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe24002d070 con 0x557af8de82f0 2024-08-30T09:11:29.498 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.495+0000 7fe22dffb640 1 -- 172.21.15.42:0/1554120382 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7fe24001cdd0 con 0x557af8de82f0 2024-08-30T09:11:29.499 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.495+0000 7fe253e89740 1 -- 172.21.15.42:0/1554120382 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe2380d10b0 con 0x557af8de82f0 2024-08-30T09:11:29.499 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.496+0000 7fe22dffb640 1 -- 172.21.15.42:0/1554120382 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fe24001b860 con 0x557af8de82f0 2024-08-30T09:11:29.504 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.499+0000 7fe253e89740 10 client.17205 fetch_fsmap learned FSMap version 41 2024-08-30T09:11:29.504 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.499+0000 7fe253e89740 1 -- 172.21.15.42:0/1554120382 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x557af9238180 con 0x557af8de82f0 2024-08-30T09:11:29.508 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.505+0000 7fe22dffb640 1 -- 172.21.15.42:0/1554120382 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe2400054f0 con 0x557af8de82f0 2024-08-30T09:11:29.508 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.505+0000 7fe22dffb640 1 -- 172.21.15.42:0/1554120382 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 41) ==== 8362+0+0 (secure 0 0 0) 0x7fe24001a040 con 0x557af8de82f0 2024-08-30T09:11:29.513 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.510+0000 7fe253e89740 10 client.17205 fetch_fsmap finished waiting for FSMap version 41 2024-08-30T09:11:29.513 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.510+0000 7fe253e89740 10 client.17205 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-30T09:11:29.513 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.510+0000 7fe253e89740 20 client.17205 populate_metadata read hostname 'smithi042' 2024-08-30T09:11:29.513 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.510+0000 7fe253e89740 1 --2- 172.21.15.42:0/1554120382 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x557af8da51c0 0x557af923ab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:29.513 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.510+0000 7fe253e89740 4 client.17205 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-30T09:11:29.513 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.510+0000 7fe253e89740 1 -- 172.21.15.42:0/1554120382 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7fe2380d10b0 con 0x557af8da51c0 2024-08-30T09:11:29.517 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.514+0000 7fe245d9b640 1 --2- 172.21.15.42:0/1554120382 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x557af8da51c0 0x557af923ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:29.523 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.521+0000 7fe245d9b640 1 --2- 172.21.15.42:0/1554120382 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x557af8da51c0 0x557af923ab20 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=54bb843c72e1518 server_cookie=219cbb3f6b2002fc in_seq=0 out_seq=0 2024-08-30T09:11:29.526 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.522+0000 7fe22dffb640 10 client.17205 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:11:29.888 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.885+0000 7fe22dffb640 1 -- 172.21.15.42:0/1554120382 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7fe2380d10b0 con 0x557af8da51c0 2024-08-30T09:11:29.889 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.885+0000 7fe22dffb640 10 client.17205 handle_command_reply: tid=0 2024-08-30T09:11:29.889 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.885+0000 7fe253e89740 1 client.17205 shutdown 2024-08-30T09:11:29.889 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.885+0000 7fe253e89740 1 -- 172.21.15.42:0/1554120382 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557af8de82f0 msgr2=0x557af91c7430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:11:29.889 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.886+0000 7fe253e89740 1 --2- 172.21.15.42:0/1554120382 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557af8de82f0 0x557af91c7430 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fe24001b830 tx=0x7fe24001b530 comp rx=0 tx=0).stop 2024-08-30T09:11:29.889 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.886+0000 7fe253e89740 1 -- 172.21.15.42:0/1554120382 shutdown_connections 2024-08-30T09:11:29.889 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.886+0000 7fe253e89740 1 --2- 172.21.15.42:0/1554120382 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x557af8da51c0 0x557af923ab20 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:29.890 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.886+0000 7fe253e89740 1 --2- 172.21.15.42:0/1554120382 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557af8df04d0 0x557af91c7970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:29.890 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.886+0000 7fe253e89740 1 --2- 172.21.15.42:0/1554120382 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557af8e111b0 0x557af9236c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:29.890 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.886+0000 7fe253e89740 1 --2- 172.21.15.42:0/1554120382 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557af8de82f0 0x557af91c7430 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:29.890 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.886+0000 7fe253e89740 1 -- 172.21.15.42:0/1554120382 >> 172.21.15.42:0/1554120382 conn(0x557af8e05300 msgr2=0x557af91bb0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:11:29.890 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.886+0000 7fe22dffb640 0 client.17205 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:11:29.890 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.887+0000 7fe253e89740 1 -- 172.21.15.42:0/1554120382 shutdown_connections 2024-08-30T09:11:29.890 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.887+0000 7fe253e89740 1 -- 172.21.15.42:0/1554120382 wait complete. 2024-08-30T09:11:29.890 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.887+0000 7fe253e89740 20 client.17205 trim_cache size 0 max 16384 2024-08-30T09:11:29.891 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-08-30T09:11:29.891 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2024-08-30T09:11:29.891 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-08-30T09:11:29.891 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-08-30T09:11:29.892 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.888+0000 7fe24ce3f640 1 -- 172.21.15.42:0/1817944717 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fe2340b4d30 msgr2=0x7fe2340b7240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:11:29.892 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.888+0000 7fe24ce3f640 1 --2- 172.21.15.42:0/1817944717 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fe2340b4d30 0x7fe2340b7240 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fe230009fd0 tx=0x7fe230009500 comp rx=0 tx=0).stop 2024-08-30T09:11:29.892 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.888+0000 7fe24ce3f640 1 -- 172.21.15.42:0/1817944717 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe24810ab40 msgr2=0x7fe2481a0910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:11:29.892 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.888+0000 7fe24ce3f640 1 --2- 172.21.15.42:0/1817944717 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe24810ab40 0x7fe2481a0910 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fe238036520 tx=0x7fe2380043d0 comp rx=0 tx=0).stop 2024-08-30T09:11:29.892 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.888+0000 7fe24ce3f640 1 -- 172.21.15.42:0/1817944717 shutdown_connections 2024-08-30T09:11:29.892 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.888+0000 7fe24ce3f640 1 --2- 172.21.15.42:0/1817944717 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fe2340b4d30 0x7fe2340b7240 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:29.893 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.888+0000 7fe24ce3f640 1 --2- 172.21.15.42:0/1817944717 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe248111d30 0x7fe2481a14c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:29.893 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.888+0000 7fe24ce3f640 1 --2- 172.21.15.42:0/1817944717 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe24810ab40 0x7fe2481a0910 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:29.893 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.888+0000 7fe24ce3f640 1 --2- 172.21.15.42:0/1817944717 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe24810a160 0x7fe2481a02a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:29.893 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.888+0000 7fe24ce3f640 1 -- 172.21.15.42:0/1817944717 >> 172.21.15.42:0/1817944717 conn(0x7fe2480705b0 msgr2=0x7fe2480709c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:11:29.893 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.888+0000 7fe24ce3f640 1 -- 172.21.15.42:0/1817944717 shutdown_connections 2024-08-30T09:11:29.893 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:29.888+0000 7fe24ce3f640 1 -- 172.21.15.42:0/1817944717 wait complete. 2024-08-30T09:11:30.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:11:30 smithi042 ceph-mon[18652]: pgmap v695: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 8.2 MiB/s wr, 552 op/s 2024-08-30T09:11:30.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:11:30 smithi190 ceph-mon[22001]: pgmap v695: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 8.2 MiB/s wr, 552 op/s 2024-08-30T09:11:30.953 INFO:teuthology.orchestra.run.smithi042.stderr:progress: 645.0 s, 499.0 tps, lat 205.329 ms stddev 236.541 2024-08-30T09:11:31.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:11:30 smithi167 ceph-mon[22063]: pgmap v695: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 8.2 MiB/s wr, 552 op/s 2024-08-30T09:11:32.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:11:32 smithi042 ceph-mon[18652]: pgmap v696: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6.3 MiB/s wr, 449 op/s 2024-08-30T09:11:32.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:11:32 smithi190 ceph-mon[22001]: pgmap v696: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6.3 MiB/s wr, 449 op/s 2024-08-30T09:11:33.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:11:32 smithi167 ceph-mon[22063]: pgmap v696: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6.3 MiB/s wr, 449 op/s 2024-08-30T09:11:34.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:11:34 smithi042 ceph-mon[18652]: pgmap v697: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 8.3 MiB/s wr, 604 op/s 2024-08-30T09:11:34.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:11:34 smithi190 ceph-mon[22001]: pgmap v697: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 8.3 MiB/s wr, 604 op/s 2024-08-30T09:11:35.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:11:34 smithi167 ceph-mon[22063]: pgmap v697: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 8.3 MiB/s wr, 604 op/s 2024-08-30T09:11:35.954 INFO:teuthology.orchestra.run.smithi042.stderr:progress: 650.0 s, 450.6 tps, lat 215.182 ms stddev 234.271 2024-08-30T09:11:36.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:11:36 smithi042 ceph-mon[18652]: pgmap v698: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 6.8 MiB/s wr, 507 op/s 2024-08-30T09:11:36.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:11:36 smithi190 ceph-mon[22001]: pgmap v698: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 6.8 MiB/s wr, 507 op/s 2024-08-30T09:11:37.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:11:36 smithi167 ceph-mon[22063]: pgmap v698: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 6.8 MiB/s wr, 507 op/s 2024-08-30T09:11:37.412 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-30T09:11:37.855 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T09:11:38.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.558+0000 7f8a7e0a1640 1 -- 172.21.15.42:0/3009128391 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8a781101c0 msgr2=0x7f8a781169f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:11:38.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.558+0000 7f8a7e0a1640 1 --2- 172.21.15.42:0/3009128391 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8a781101c0 0x7f8a781169f0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f8a70009900 tx=0x7f8a7002a420 comp rx=0 tx=0).stop 2024-08-30T09:11:38.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.558+0000 7f8a7e0a1640 1 -- 172.21.15.42:0/3009128391 shutdown_connections 2024-08-30T09:11:38.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.558+0000 7f8a7e0a1640 1 --2- 172.21.15.42:0/3009128391 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8a7810f870 0x7f8a7810fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:38.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.558+0000 7f8a7e0a1640 1 --2- 172.21.15.42:0/3009128391 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8a781101c0 0x7f8a781169f0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:38.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.558+0000 7f8a7e0a1640 1 --2- 172.21.15.42:0/3009128391 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8a78109f30 0x7f8a7810a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:38.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.558+0000 7f8a7e0a1640 1 -- 172.21.15.42:0/3009128391 >> 172.21.15.42:0/3009128391 conn(0x7f8a78077e90 msgr2=0x7f8a78107f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:11:38.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.561+0000 7f8a7e0a1640 1 -- 172.21.15.42:0/3009128391 shutdown_connections 2024-08-30T09:11:38.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.561+0000 7f8a7e0a1640 1 -- 172.21.15.42:0/3009128391 wait complete. 2024-08-30T09:11:38.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.562+0000 7f8a7e0a1640 1 Processor -- start 2024-08-30T09:11:38.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.562+0000 7f8a7e0a1640 1 -- start start 2024-08-30T09:11:38.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.562+0000 7f8a7e0a1640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8a78109f30 0x7f8a78074cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:38.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.562+0000 7f8a7e0a1640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8a7810f870 0x7f8a7806ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:38.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.562+0000 7f8a7e0a1640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8a781101c0 0x7f8a7806f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:38.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.562+0000 7f8a7e0a1640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f8a7806d480 con 0x7f8a7810f870 2024-08-30T09:11:38.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.562+0000 7f8a7e0a1640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f8a7806d300 con 0x7f8a78109f30 2024-08-30T09:11:38.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.562+0000 7f8a7e0a1640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f8a7806d600 con 0x7f8a781101c0 2024-08-30T09:11:38.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.563+0000 7f8a777fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8a78109f30 0x7f8a78074cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:38.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.563+0000 7f8a777fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8a78109f30 0x7f8a78074cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:45388/0 (socket says 172.21.15.42:45388) 2024-08-30T09:11:38.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.563+0000 7f8a777fe640 1 -- 172.21.15.42:0/3780702573 learned_addr learned my addr 172.21.15.42:0/3780702573 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:11:38.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.563+0000 7f8a6effd640 1 --2- 172.21.15.42:0/3780702573 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8a7810f870 0x7f8a7806ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:38.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.563+0000 7f8a777fe640 1 -- 172.21.15.42:0/3780702573 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8a781101c0 msgr2=0x7f8a7806f580 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-08-30T09:11:38.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.564+0000 7f8a777fe640 1 --2- 172.21.15.42:0/3780702573 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8a781101c0 0x7f8a7806f580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:38.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.564+0000 7f8a777fe640 1 -- 172.21.15.42:0/3780702573 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8a7810f870 msgr2=0x7f8a7806ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:11:38.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.564+0000 7f8a777fe640 1 --2- 172.21.15.42:0/3780702573 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8a7810f870 0x7f8a7806ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:38.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.564+0000 7f8a777fe640 1 -- 172.21.15.42:0/3780702573 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a7806fda0 con 0x7f8a78109f30 2024-08-30T09:11:38.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.564+0000 7f8a6effd640 1 --2- 172.21.15.42:0/3780702573 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8a7810f870 0x7f8a7806ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-30T09:11:38.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.565+0000 7f8a777fe640 1 --2- 172.21.15.42:0/3780702573 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8a78109f30 0x7f8a78074cb0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f8a6800cce0 tx=0x7f8a6800a620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:11:38.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.569+0000 7f8a757fa640 1 -- 172.21.15.42:0/3780702573 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a68004b20 con 0x7f8a78109f30 2024-08-30T09:11:38.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.569+0000 7f8a7e0a1640 1 -- 172.21.15.42:0/3780702573 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a78081b00 con 0x7f8a78109f30 2024-08-30T09:11:38.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.569+0000 7f8a7e0a1640 1 -- 172.21.15.42:0/3780702573 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a78082040 con 0x7f8a78109f30 2024-08-30T09:11:38.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.573+0000 7f8a84616740 1 Processor -- start 2024-08-30T09:11:38.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.573+0000 7f8a84616740 1 -- start start 2024-08-30T09:11:38.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.573+0000 7f8a84616740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5565887ee070 0x556588b932c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:38.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.573+0000 7f8a84616740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5565887c51b0 0x556588b93800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:38.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.573+0000 7f8a84616740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5565887cd390 0x556588b93d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:38.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.573+0000 7f8a84616740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x556588b0a630 con 0x5565887c51b0 2024-08-30T09:11:38.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.573+0000 7f8a84616740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x556588af9bc0 con 0x5565887cd390 2024-08-30T09:11:38.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.573+0000 7f8a84616740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x556588afdb70 con 0x5565887ee070 2024-08-30T09:11:38.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.574+0000 7f8a757fa640 1 -- 172.21.15.42:0/3780702573 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a68004cc0 con 0x7f8a78109f30 2024-08-30T09:11:38.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.574+0000 7f8a757fa640 1 -- 172.21.15.42:0/3780702573 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a6800f690 con 0x7f8a78109f30 2024-08-30T09:11:38.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.574+0000 7f8a6effd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5565887cd390 0x556588b93d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:38.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.574+0000 7f8a6effd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5565887cd390 0x556588b93d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:45392/0 (socket says 172.21.15.42:45392) 2024-08-30T09:11:38.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.574+0000 7f8a6effd640 1 -- 172.21.15.42:0/3652061680 learned_addr learned my addr 172.21.15.42:0/3652061680 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:11:38.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.575+0000 7f8a77fff640 1 --2- 172.21.15.42:0/3652061680 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5565887ee070 0x556588b932c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:38.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.578+0000 7f8a777fe640 1 --2- 172.21.15.42:0/3652061680 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5565887c51b0 0x556588b93800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:38.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.579+0000 7f8a6effd640 1 -- 172.21.15.42:0/3652061680 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5565887ee070 msgr2=0x556588b932c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:11:38.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.579+0000 7f8a6effd640 1 --2- 172.21.15.42:0/3652061680 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5565887ee070 0x556588b932c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:38.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.579+0000 7f8a6effd640 1 -- 172.21.15.42:0/3652061680 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5565887c51b0 msgr2=0x556588b93800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:11:38.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.579+0000 7f8a6effd640 1 --2- 172.21.15.42:0/3652061680 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5565887c51b0 0x556588b93800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:38.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.579+0000 7f8a6effd640 1 -- 172.21.15.42:0/3652061680 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556588859650 con 0x5565887cd390 2024-08-30T09:11:38.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.580+0000 7f8a757fa640 1 -- 172.21.15.42:0/3780702573 <== mon.2 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7f8a6800f8f0 con 0x7f8a78109f30 2024-08-30T09:11:38.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.581+0000 7f8a6effd640 1 --2- 172.21.15.42:0/3652061680 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5565887cd390 0x556588b93d40 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f8a6000aa30 tx=0x7f8a6000aef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=276e00002fccabbe server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:11:38.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.581+0000 7f8a6dffb640 1 -- 172.21.15.42:0/3652061680 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a60015070 con 0x5565887cd390 2024-08-30T09:11:38.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.581+0000 7f8a6dffb640 1 -- 172.21.15.42:0/3652061680 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a6000ccc0 con 0x5565887cd390 2024-08-30T09:11:38.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.581+0000 7f8a6dffb640 1 -- 172.21.15.42:0/3652061680 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a60018650 con 0x5565887cd390 2024-08-30T09:11:38.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.581+0000 7f8a757fa640 1 --2- 172.21.15.42:0/3780702573 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f8a640b4d40 0x7f8a640b7250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:38.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.582+0000 7f8a757fa640 1 -- 172.21.15.42:0/3780702573 <== mon.2 v2:172.21.15.167:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f8a680e57c0 con 0x7f8a78109f30 2024-08-30T09:11:38.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.583+0000 7f8a84616740 1 -- 172.21.15.42:0/3652061680 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5565887cd390 msgr2=0x556588b93d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:11:38.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.583+0000 7f8a84616740 1 --2- 172.21.15.42:0/3652061680 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5565887cd390 0x556588b93d40 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f8a6000aa30 tx=0x7f8a6000aef0 comp rx=0 tx=0).stop 2024-08-30T09:11:38.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.584+0000 7f8a84616740 1 -- 172.21.15.42:0/3652061680 shutdown_connections 2024-08-30T09:11:38.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.584+0000 7f8a84616740 1 --2- 172.21.15.42:0/3652061680 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5565887cd390 0x556588b93d40 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:38.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.584+0000 7f8a84616740 1 --2- 172.21.15.42:0/3652061680 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5565887c51b0 0x556588b93800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:38.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.584+0000 7f8a84616740 1 --2- 172.21.15.42:0/3652061680 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5565887ee070 0x556588b932c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:38.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.584+0000 7f8a84616740 1 -- 172.21.15.42:0/3652061680 >> 172.21.15.42:0/3652061680 conn(0x5565887e2460 msgr2=0x556588b911b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:11:38.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.588+0000 7f8a77fff640 1 --2- 172.21.15.42:0/3780702573 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f8a640b4d40 0x7f8a640b7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:38.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.588+0000 7f8a84616740 1 -- 172.21.15.42:0/3652061680 shutdown_connections 2024-08-30T09:11:38.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.590+0000 7f8a77fff640 1 --2- 172.21.15.42:0/3780702573 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f8a640b4d40 0x7f8a640b7250 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f8a70006010 tx=0x7f8a70031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:11:38.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.591+0000 7f8a84616740 1 -- 172.21.15.42:0/3652061680 wait complete. 2024-08-30T09:11:38.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.591+0000 7f8a84616740 1 -- start start 2024-08-30T09:11:38.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.592+0000 7f8a84616740 1 Processor -- start 2024-08-30T09:11:38.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.592+0000 7f8a84616740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5565887c51b0 0x556588b9ea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:38.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.593+0000 7f8a84616740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5565887cd390 0x556588b9efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:38.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.593+0000 7f8a84616740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5565887ee070 0x556588baae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:38.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.593+0000 7f8a84616740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55658895c700 con 0x5565887ee070 2024-08-30T09:11:38.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.593+0000 7f8a84616740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x556588a80700 con 0x5565887cd390 2024-08-30T09:11:38.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.593+0000 7f8a84616740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x556588ba1aa0 con 0x5565887c51b0 2024-08-30T09:11:38.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.593+0000 7f8a84616740 10 client.0 _ll_register_callbacks cb 0x556588aafdc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:11:38.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.596+0000 7f8a77fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5565887ee070 0x556588baae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:38.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.596+0000 7f8a77fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5565887ee070 0x556588baae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:48340/0 (socket says 172.21.15.42:48340) 2024-08-30T09:11:38.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.596+0000 7f8a77fff640 1 -- 172.21.15.42:0/2273013749 learned_addr learned my addr 172.21.15.42:0/2273013749 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:11:38.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.597+0000 7f8a777fe640 1 --2- 172.21.15.42:0/2273013749 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5565887c51b0 0x556588b9ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:38.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.597+0000 7f8a6effd640 1 --2- 172.21.15.42:0/2273013749 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5565887cd390 0x556588b9efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:38.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.597+0000 7f8a77fff640 1 -- 172.21.15.42:0/2273013749 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5565887c51b0 msgr2=0x556588b9ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:11:38.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.597+0000 7f8a77fff640 1 --2- 172.21.15.42:0/2273013749 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5565887c51b0 0x556588b9ea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:38.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.597+0000 7f8a77fff640 1 -- 172.21.15.42:0/2273013749 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5565887cd390 msgr2=0x556588b9efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:11:38.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.597+0000 7f8a77fff640 1 --2- 172.21.15.42:0/2273013749 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5565887cd390 0x556588b9efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:38.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.597+0000 7f8a77fff640 1 -- 172.21.15.42:0/2273013749 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556588ab98b0 con 0x5565887ee070 2024-08-30T09:11:38.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.599+0000 7f8a77fff640 1 --2- 172.21.15.42:0/2273013749 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5565887ee070 0x556588baae20 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f8a70033000 tx=0x7f8a70009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7720d5dbbcef2809 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:11:38.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.599+0000 7f8a56ffd640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-08-30T09:11:38.605 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.603+0000 7f8a56ffd640 1 -- 172.21.15.42:0/2273013749 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a70042070 con 0x5565887ee070 2024-08-30T09:11:38.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.605+0000 7f8a84616740 1 -- 172.21.15.42:0/2273013749 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a6000ccc0 con 0x5565887ee070 2024-08-30T09:11:38.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.605+0000 7f8a56ffd640 1 -- 172.21.15.42:0/2273013749 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f8a700037f0 con 0x5565887ee070 2024-08-30T09:11:38.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.607+0000 7f8a56ffd640 1 -- 172.21.15.42:0/2273013749 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f8a70057050 con 0x5565887ee070 2024-08-30T09:11:38.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.607+0000 7f8a84616740 10 client.17214 fetch_fsmap learned FSMap version 41 2024-08-30T09:11:38.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.607+0000 7f8a84616740 1 -- 172.21.15.42:0/2273013749 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x556588bad940 con 0x5565887ee070 2024-08-30T09:11:38.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.607+0000 7f8a56ffd640 1 -- 172.21.15.42:0/2273013749 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a6000ccc0 con 0x5565887ee070 2024-08-30T09:11:38.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.607+0000 7f8a56ffd640 1 -- 172.21.15.42:0/2273013749 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 41) ==== 8362+0+0 (secure 0 0 0) 0x7f8a7003a040 con 0x5565887ee070 2024-08-30T09:11:38.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.608+0000 7f8a84616740 10 client.17214 fetch_fsmap finished waiting for FSMap version 41 2024-08-30T09:11:38.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.608+0000 7f8a84616740 10 client.17214 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-30T09:11:38.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.608+0000 7f8a84616740 20 client.17214 populate_metadata read hostname 'smithi042' 2024-08-30T09:11:38.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.608+0000 7f8a84616740 1 --2- 172.21.15.42:0/2273013749 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x5565887ac9f0 0x556588bb1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:38.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.608+0000 7f8a84616740 4 client.17214 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-30T09:11:38.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.608+0000 7f8a84616740 1 -- 172.21.15.42:0/2273013749 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556588b0c580 con 0x5565887ac9f0 2024-08-30T09:11:38.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.608+0000 7f8a777fe640 1 --2- 172.21.15.42:0/2273013749 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x5565887ac9f0 0x556588bb1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:38.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.609+0000 7f8a56ffd640 10 client.17214 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:11:38.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.609+0000 7f8a777fe640 1 --2- 172.21.15.42:0/2273013749 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x5565887ac9f0 0x556588bb1310 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6dadaec814a6968f server_cookie=ffcf811fa1128bc9 in_seq=0 out_seq=0 2024-08-30T09:11:38.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.609+0000 7f8a56ffd640 1 -- 172.21.15.42:0/2273013749 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x556588b0c580 con 0x5565887ac9f0 2024-08-30T09:11:38.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.609+0000 7f8a56ffd640 10 client.17214 handle_command_reply: tid=0 2024-08-30T09:11:38.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.610+0000 7f8a84616740 1 client.17214 shutdown 2024-08-30T09:11:38.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.610+0000 7f8a84616740 1 -- 172.21.15.42:0/2273013749 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5565887ee070 msgr2=0x556588baae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:11:38.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.610+0000 7f8a84616740 1 --2- 172.21.15.42:0/2273013749 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5565887ee070 0x556588baae20 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f8a70033000 tx=0x7f8a70009260 comp rx=0 tx=0).stop 2024-08-30T09:11:38.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.611+0000 7f8a84616740 1 -- 172.21.15.42:0/2273013749 shutdown_connections 2024-08-30T09:11:38.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.611+0000 7f8a84616740 1 --2- 172.21.15.42:0/2273013749 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x5565887ac9f0 0x556588bb1310 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:38.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.611+0000 7f8a84616740 1 --2- 172.21.15.42:0/2273013749 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5565887ee070 0x556588baae20 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:38.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.611+0000 7f8a56ffd640 0 client.17214 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:11:38.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.611+0000 7f8a84616740 1 --2- 172.21.15.42:0/2273013749 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5565887cd390 0x556588b9efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:38.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.611+0000 7f8a84616740 1 --2- 172.21.15.42:0/2273013749 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5565887c51b0 0x556588b9ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:38.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.611+0000 7f8a84616740 1 -- 172.21.15.42:0/2273013749 >> 172.21.15.42:0/2273013749 conn(0x5565887e2460 msgr2=0x556588b96450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:11:38.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.611+0000 7f8a84616740 1 -- 172.21.15.42:0/2273013749 shutdown_connections 2024-08-30T09:11:38.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.611+0000 7f8a84616740 1 -- 172.21.15.42:0/2273013749 wait complete. 2024-08-30T09:11:38.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.611+0000 7f8a84616740 20 client.17214 trim_cache size 0 max 16384 2024-08-30T09:11:38.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:11:38 smithi042 ceph-mon[18652]: pgmap v699: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 6.2 MiB/s wr, 472 op/s 2024-08-30T09:11:38.638 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.634+0000 7f8a84616740 1 Processor -- start 2024-08-30T09:11:38.638 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.635+0000 7f8a84616740 1 -- start start 2024-08-30T09:11:38.638 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.635+0000 7f8a84616740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5565887ac9f0 0x556588baad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:38.638 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.635+0000 7f8a84616740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5565887c51b0 0x556588b9e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:38.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.635+0000 7f8a777fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5565887ac9f0 0x556588baad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:38.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.635+0000 7f8a777fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5565887ac9f0 0x556588baad20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:42170/0 (socket says 172.21.15.42:42170) 2024-08-30T09:11:38.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.635+0000 7f8a84616740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5565887cd390 0x556588b9ef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:38.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.635+0000 7f8a777fe640 1 -- 172.21.15.42:0/1263602732 learned_addr learned my addr 172.21.15.42:0/1263602732 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:11:38.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.635+0000 7f8a84616740 1 -- 172.21.15.42:0/1263602732 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f8a58012520 con 0x5565887cd390 2024-08-30T09:11:38.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.635+0000 7f8a84616740 1 -- 172.21.15.42:0/1263602732 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f8a5800fd90 con 0x5565887c51b0 2024-08-30T09:11:38.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.635+0000 7f8a84616740 1 -- 172.21.15.42:0/1263602732 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f8a5800dbe0 con 0x5565887ac9f0 2024-08-30T09:11:38.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.635+0000 7f8a6effd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5565887c51b0 0x556588b9e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:38.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.635+0000 7f8a77fff640 1 --2- 172.21.15.42:0/1263602732 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5565887cd390 0x556588b9ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:38.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.636+0000 7f8a777fe640 1 -- 172.21.15.42:0/1263602732 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5565887c51b0 msgr2=0x556588b9e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:11:38.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.636+0000 7f8a777fe640 1 --2- 172.21.15.42:0/1263602732 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5565887c51b0 0x556588b9e9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:38.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.636+0000 7f8a777fe640 1 -- 172.21.15.42:0/1263602732 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5565887cd390 msgr2=0x556588b9ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:11:38.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.636+0000 7f8a777fe640 1 --2- 172.21.15.42:0/1263602732 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5565887cd390 0x556588b9ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:38.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.636+0000 7f8a777fe640 1 -- 172.21.15.42:0/1263602732 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556588b958d0 con 0x5565887ac9f0 2024-08-30T09:11:38.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.636+0000 7f8a6effd640 1 --2- 172.21.15.42:0/1263602732 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5565887c51b0 0x556588b9e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-30T09:11:38.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.636+0000 7f8a777fe640 1 --2- 172.21.15.42:0/1263602732 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5565887ac9f0 0x556588baad20 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f8a6800ab20 tx=0x7f8a68000f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6d4bdd086a0af650 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:11:38.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.636+0000 7f8a6d7fa640 1 -- 172.21.15.42:0/1263602732 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a6801d040 con 0x5565887ac9f0 2024-08-30T09:11:38.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.636+0000 7f8a6d7fa640 1 -- 172.21.15.42:0/1263602732 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a680ab910 con 0x5565887ac9f0 2024-08-30T09:11:38.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.636+0000 7f8a6d7fa640 1 -- 172.21.15.42:0/1263602732 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a680971c0 con 0x5565887ac9f0 2024-08-30T09:11:38.641 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.637+0000 7f8a84616740 1 -- 172.21.15.42:0/1263602732 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5565887ac9f0 msgr2=0x556588baad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:11:38.641 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.637+0000 7f8a84616740 1 --2- 172.21.15.42:0/1263602732 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5565887ac9f0 0x556588baad20 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f8a6800ab20 tx=0x7f8a68000f40 comp rx=0 tx=0).stop 2024-08-30T09:11:38.641 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.637+0000 7f8a84616740 1 -- 172.21.15.42:0/1263602732 shutdown_connections 2024-08-30T09:11:38.641 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.637+0000 7f8a84616740 1 --2- 172.21.15.42:0/1263602732 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5565887cd390 0x556588b9ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:38.641 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.637+0000 7f8a84616740 1 --2- 172.21.15.42:0/1263602732 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5565887c51b0 0x556588b9e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:38.641 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.637+0000 7f8a84616740 1 --2- 172.21.15.42:0/1263602732 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5565887ac9f0 0x556588baad20 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:38.641 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.637+0000 7f8a84616740 1 -- 172.21.15.42:0/1263602732 >> 172.21.15.42:0/1263602732 conn(0x5565887e2460 msgr2=0x556588b8fb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:11:38.641 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.637+0000 7f8a84616740 1 -- 172.21.15.42:0/1263602732 shutdown_connections 2024-08-30T09:11:38.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.637+0000 7f8a84616740 1 -- 172.21.15.42:0/1263602732 wait complete. 2024-08-30T09:11:38.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.638+0000 7f8a84616740 1 -- start start 2024-08-30T09:11:38.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.638+0000 7f8a84616740 1 Processor -- start 2024-08-30T09:11:38.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.638+0000 7f8a84616740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5565887ac9f0 0x556588ba6d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:38.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.638+0000 7f8a84616740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5565887c51b0 0x556588ba7360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:38.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.638+0000 7f8a84616740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5565887cd390 0x556588ba3d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:38.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.639+0000 7f8a84616740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f8a5800e250 con 0x5565887c51b0 2024-08-30T09:11:38.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.639+0000 7f8a84616740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f8a5800f6c0 con 0x5565887cd390 2024-08-30T09:11:38.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.639+0000 7f8a84616740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f8a5800e920 con 0x5565887ac9f0 2024-08-30T09:11:38.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.639+0000 7f8a84616740 10 client.0 _ll_register_callbacks cb 0x7f8a70009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:11:38.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.639+0000 7f8a77fff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5565887cd390 0x556588ba3d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:38.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.639+0000 7f8a77fff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5565887cd390 0x556588ba3d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:45416/0 (socket says 172.21.15.42:45416) 2024-08-30T09:11:38.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.639+0000 7f8a77fff640 1 -- 172.21.15.42:0/1031708988 learned_addr learned my addr 172.21.15.42:0/1031708988 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:11:38.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.639+0000 7f8a777fe640 1 --2- 172.21.15.42:0/1031708988 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5565887ac9f0 0x556588ba6d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:38.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.639+0000 7f8a6effd640 1 --2- 172.21.15.42:0/1031708988 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5565887c51b0 0x556588ba7360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:38.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.639+0000 7f8a777fe640 1 -- 172.21.15.42:0/1031708988 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5565887cd390 msgr2=0x556588ba3d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:11:38.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.639+0000 7f8a777fe640 1 --2- 172.21.15.42:0/1031708988 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5565887cd390 0x556588ba3d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:38.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.639+0000 7f8a777fe640 1 -- 172.21.15.42:0/1031708988 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5565887c51b0 msgr2=0x556588ba7360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:11:38.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.639+0000 7f8a777fe640 1 --2- 172.21.15.42:0/1031708988 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5565887c51b0 0x556588ba7360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:38.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.639+0000 7f8a777fe640 1 -- 172.21.15.42:0/1031708988 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556588b9f450 con 0x5565887ac9f0 2024-08-30T09:11:38.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.640+0000 7f8a777fe640 1 --2- 172.21.15.42:0/1031708988 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5565887ac9f0 0x556588ba6d80 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f8a680b52e0 tx=0x7f8a680b5120 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f7b081496b31cbf3 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:11:38.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.640+0000 7f8a56ffd640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-08-30T09:11:38.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.640+0000 7f8a56ffd640 1 -- 172.21.15.42:0/1031708988 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a680d2020 con 0x5565887ac9f0 2024-08-30T09:11:38.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.640+0000 7f8a56ffd640 1 -- 172.21.15.42:0/1031708988 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f8a680a8090 con 0x5565887ac9f0 2024-08-30T09:11:38.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.640+0000 7f8a84616740 1 -- 172.21.15.42:0/1031708988 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a680ab910 con 0x5565887ac9f0 2024-08-30T09:11:38.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.640+0000 7f8a56ffd640 1 -- 172.21.15.42:0/1031708988 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f8a6801d040 con 0x5565887ac9f0 2024-08-30T09:11:38.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.640+0000 7f8a84616740 10 client.26998 fetch_fsmap learned FSMap version 41 2024-08-30T09:11:38.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.640+0000 7f8a84616740 1 -- 172.21.15.42:0/1031708988 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x556588c1c2f0 con 0x5565887ac9f0 2024-08-30T09:11:38.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.641+0000 7f8a56ffd640 1 -- 172.21.15.42:0/1031708988 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a680ab910 con 0x5565887ac9f0 2024-08-30T09:11:38.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.641+0000 7f8a56ffd640 1 -- 172.21.15.42:0/1031708988 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 41) ==== 8362+0+0 (secure 0 0 0) 0x7f8a680b4c90 con 0x5565887ac9f0 2024-08-30T09:11:38.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.641+0000 7f8a84616740 10 client.26998 fetch_fsmap finished waiting for FSMap version 41 2024-08-30T09:11:38.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.641+0000 7f8a84616740 10 client.26998 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-30T09:11:38.647 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.643+0000 7f8a84616740 20 client.26998 populate_metadata read hostname 'smithi042' 2024-08-30T09:11:38.647 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.643+0000 7f8a84616740 1 --2- 172.21.15.42:0/1031708988 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x5565887ee070 0x556588c1e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:38.647 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.643+0000 7f8a84616740 4 client.26998 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-30T09:11:38.647 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.643+0000 7f8a84616740 1 -- 172.21.15.42:0/1031708988 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "scrub status"}) -- 0x556588b08020 con 0x5565887ee070 2024-08-30T09:11:38.647 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.643+0000 7f8a6effd640 1 --2- 172.21.15.42:0/1031708988 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x5565887ee070 0x556588c1e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:38.647 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.644+0000 7f8a56ffd640 10 client.26998 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:11:38.647 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.644+0000 7f8a6effd640 1 --2- 172.21.15.42:0/1031708988 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x5565887ee070 0x556588c1e9a0 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e99075426fcbcf7 server_cookie=24c2dfc158d519cb in_seq=0 out_seq=0 2024-08-30T09:11:38.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.645+0000 7f8a56ffd640 1 -- 172.21.15.42:0/1031708988 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3782 (crc 0 0 0) 0x556588b08020 con 0x5565887ee070 2024-08-30T09:11:38.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.645+0000 7f8a56ffd640 10 client.26998 handle_command_reply: tid=0 2024-08-30T09:11:38.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.645+0000 7f8a84616740 1 client.26998 shutdown 2024-08-30T09:11:38.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.648+0000 7f8a84616740 1 -- 172.21.15.42:0/1031708988 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5565887ac9f0 msgr2=0x556588ba6d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:11:38.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.648+0000 7f8a84616740 1 --2- 172.21.15.42:0/1031708988 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5565887ac9f0 0x556588ba6d80 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f8a680b52e0 tx=0x7f8a680b5120 comp rx=0 tx=0).stop 2024-08-30T09:11:38.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.648+0000 7f8a84616740 1 -- 172.21.15.42:0/1031708988 shutdown_connections 2024-08-30T09:11:38.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.648+0000 7f8a84616740 1 --2- 172.21.15.42:0/1031708988 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x5565887ee070 0x556588c1e9a0 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:38.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.648+0000 7f8a84616740 1 --2- 172.21.15.42:0/1031708988 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5565887cd390 0x556588ba3d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:38.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.648+0000 7f8a84616740 1 --2- 172.21.15.42:0/1031708988 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5565887c51b0 0x556588ba7360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:38.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.649+0000 7f8a84616740 1 --2- 172.21.15.42:0/1031708988 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5565887ac9f0 0x556588ba6d80 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:38.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.649+0000 7f8a56ffd640 0 client.26998 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:11:38.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.649+0000 7f8a84616740 1 -- 172.21.15.42:0/1031708988 >> 172.21.15.42:0/1031708988 conn(0x5565887e2460 msgr2=0x556588b9e9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:11:38.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.649+0000 7f8a84616740 1 -- 172.21.15.42:0/1031708988 shutdown_connections 2024-08-30T09:11:38.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.649+0000 7f8a84616740 1 -- 172.21.15.42:0/1031708988 wait complete. 2024-08-30T09:11:38.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.649+0000 7f8a84616740 20 client.26998 trim_cache size 0 max 16384 2024-08-30T09:11:38.653 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-08-30T09:11:38.653 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "no active scrubs running", 2024-08-30T09:11:38.653 INFO:teuthology.orchestra.run.smithi042.stdout: "scrubs": {}, 2024-08-30T09:11:38.653 INFO:teuthology.orchestra.run.smithi042.stdout: "scrub_stats": { 2024-08-30T09:11:38.653 INFO:teuthology.orchestra.run.smithi042.stdout: "37f2f28f-496f-41a6-b4cf-8846fb3f43ab": { 2024-08-30T09:11:38.654 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:09:17.428286+0000", 2024-08-30T09:11:38.654 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:11:38.654 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:11:38.654 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:11:38.654 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:11:38.654 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:11:38.654 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:11:38.654 INFO:teuthology.orchestra.run.smithi042.stdout: "99a29ec1-9455-4ae4-854a-398b32896675": { 2024-08-30T09:11:38.654 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:08:24.923176+0000", 2024-08-30T09:11:38.654 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:11:38.654 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:11:38.654 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:11:38.655 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:11:38.655 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:11:38.655 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:11:38.655 INFO:teuthology.orchestra.run.smithi042.stdout: "a7210f9a-046b-4188-953c-819b9f1863a9": { 2024-08-30T09:11:38.655 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:10:07.608430+0000", 2024-08-30T09:11:38.655 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:11:38.655 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:11:38.655 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:11:38.655 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:11:38.655 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:11:38.655 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:11:38.655 INFO:teuthology.orchestra.run.smithi042.stdout: "598d8a75-f1cc-4388-80bc-a408bdb4eaac": { 2024-08-30T09:11:38.655 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:07:29.721122+0000", 2024-08-30T09:11:38.655 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:11:38.655 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:11:38.656 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:11:38.656 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:11:38.656 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:11:38.656 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:11:38.656 INFO:teuthology.orchestra.run.smithi042.stdout: "665ec740-7254-49cb-84ef-deff71c461b0": { 2024-08-30T09:11:38.656 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:06:37.144035+0000", 2024-08-30T09:11:38.656 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:11:38.656 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:11:38.656 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:11:38.656 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:11:38.656 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:11:38.656 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:11:38.656 INFO:teuthology.orchestra.run.smithi042.stdout: "1af4252a-4a99-47e9-bbf0-becf5e573dd5": { 2024-08-30T09:11:38.657 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:05:48.565881+0000", 2024-08-30T09:11:38.657 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:11:38.657 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:11:38.657 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:11:38.657 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:11:38.657 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:11:38.657 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:11:38.657 INFO:teuthology.orchestra.run.smithi042.stdout: "347d05ed-4bd3-4590-b099-314f719aab39": { 2024-08-30T09:11:38.657 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:05:00.439721+0000", 2024-08-30T09:11:38.657 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:11:38.657 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:11:38.657 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:11:38.657 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:11:38.657 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:11:38.657 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:11:38.658 INFO:teuthology.orchestra.run.smithi042.stdout: "b62c4ffc-9387-4677-b9ba-1e94ba308f17": { 2024-08-30T09:11:38.658 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:11:03.363478+0000", 2024-08-30T09:11:38.658 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:11:38.658 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:11:38.658 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:11:38.658 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:11:38.658 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:11:38.658 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:11:38.658 INFO:teuthology.orchestra.run.smithi042.stdout: "88cd7478-2b20-47de-ba51-256479749232": { 2024-08-30T09:11:38.658 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:04:08.922625+0000", 2024-08-30T09:11:38.658 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:11:38.658 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:11:38.658 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:11:38.658 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:11:38.658 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:11:38.659 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:11:38.659 INFO:teuthology.orchestra.run.smithi042.stdout: "749f4867-0b70-49e5-af2d-8362a240ebc3": { 2024-08-30T09:11:38.659 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:03:16.386554+0000", 2024-08-30T09:11:38.659 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:11:38.659 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:11:38.659 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:11:38.659 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:11:38.659 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:11:38.659 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:11:38.659 INFO:teuthology.orchestra.run.smithi042.stdout: "85134e4b-ab4a-4885-b07e-a3ad01eaf13d": { 2024-08-30T09:11:38.659 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:01:27.738462+0000", 2024-08-30T09:11:38.659 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:11:38.659 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:11:38.659 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:11:38.660 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:11:38.660 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:11:38.660 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:11:38.660 INFO:teuthology.orchestra.run.smithi042.stdout: "c827ad7a-44ee-4e1d-9bb3-51fb7ab00c73": { 2024-08-30T09:11:38.660 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:02:21.686517+0000", 2024-08-30T09:11:38.660 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:11:38.660 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:11:38.660 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:11:38.660 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:11:38.660 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:11:38.660 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:11:38.660 INFO:teuthology.orchestra.run.smithi042.stdout: "5e2ed104-0625-478a-a78b-584e7f4c2817": { 2024-08-30T09:11:38.660 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:00:33.928400+0000", 2024-08-30T09:11:38.660 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:11:38.660 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:11:38.661 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:11:38.661 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:11:38.661 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:11:38.661 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:11:38.661 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:11:38.661 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-08-30T09:11:38.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.651+0000 7f8a7e0a1640 1 -- 172.21.15.42:0/3780702573 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f8a640b4d40 msgr2=0x7f8a640b7250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:11:38.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.651+0000 7f8a7e0a1640 1 --2- 172.21.15.42:0/3780702573 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f8a640b4d40 0x7f8a640b7250 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f8a70006010 tx=0x7f8a70031400 comp rx=0 tx=0).stop 2024-08-30T09:11:38.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.651+0000 7f8a7e0a1640 1 -- 172.21.15.42:0/3780702573 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8a78109f30 msgr2=0x7f8a78074cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:11:38.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.651+0000 7f8a7e0a1640 1 --2- 172.21.15.42:0/3780702573 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8a78109f30 0x7f8a78074cb0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f8a6800cce0 tx=0x7f8a6800a620 comp rx=0 tx=0).stop 2024-08-30T09:11:38.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.651+0000 7f8a7e0a1640 1 -- 172.21.15.42:0/3780702573 shutdown_connections 2024-08-30T09:11:38.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.651+0000 7f8a7e0a1640 1 --2- 172.21.15.42:0/3780702573 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f8a640b4d40 0x7f8a640b7250 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:38.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.651+0000 7f8a7e0a1640 1 --2- 172.21.15.42:0/3780702573 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8a781101c0 0x7f8a7806f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:38.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.651+0000 7f8a7e0a1640 1 --2- 172.21.15.42:0/3780702573 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8a7810f870 0x7f8a7806ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:38.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.651+0000 7f8a7e0a1640 1 --2- 172.21.15.42:0/3780702573 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8a78109f30 0x7f8a78074cb0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:38.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.651+0000 7f8a7e0a1640 1 -- 172.21.15.42:0/3780702573 >> 172.21.15.42:0/3780702573 conn(0x7f8a78077e90 msgr2=0x7f8a7810d940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:11:38.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.652+0000 7f8a7e0a1640 1 -- 172.21.15.42:0/3780702573 shutdown_connections 2024-08-30T09:11:38.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:38.652+0000 7f8a7e0a1640 1 -- 172.21.15.42:0/3780702573 wait complete. 2024-08-30T09:11:38.878 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:11:38 smithi167 ceph-mon[22063]: pgmap v699: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 6.2 MiB/s wr, 472 op/s 2024-08-30T09:11:38.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:11:38 smithi190 ceph-mon[22001]: pgmap v699: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 6.2 MiB/s wr, 472 op/s 2024-08-30T09:11:39.490 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-08-30T09:11:39.490 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph fs dump --format=json 2024-08-30T09:11:39.943 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T09:11:40.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:40.597+0000 7f96fa533640 1 -- 172.21.15.42:0/1214628413 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f96f4111ce0 msgr2=0x7f96f410a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:11:40.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:40.597+0000 7f96fa533640 1 --2- 172.21.15.42:0/1214628413 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f96f4111ce0 0x7f96f410a160 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f96e4009860 tx=0x7f96e402a380 comp rx=0 tx=0).stop 2024-08-30T09:11:40.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:40.601+0000 7f96fa533640 1 -- 172.21.15.42:0/1214628413 shutdown_connections 2024-08-30T09:11:40.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:40.601+0000 7f96fa533640 1 --2- 172.21.15.42:0/1214628413 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f96f411c9d0 0x7f96f411eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:40.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:40.601+0000 7f96fa533640 1 --2- 172.21.15.42:0/1214628413 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f96f410a780 0x7f96f410ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:40.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:40.601+0000 7f96fa533640 1 --2- 172.21.15.42:0/1214628413 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f96f4111ce0 0x7f96f410a160 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:40.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:40.601+0000 7f96fa533640 1 -- 172.21.15.42:0/1214628413 >> 172.21.15.42:0/1214628413 conn(0x7f96f4070c60 msgr2=0x7f96f4071070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:11:40.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:40.601+0000 7f96fa533640 1 -- 172.21.15.42:0/1214628413 shutdown_connections 2024-08-30T09:11:40.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:40.601+0000 7f96fa533640 1 -- 172.21.15.42:0/1214628413 wait complete. 2024-08-30T09:11:40.605 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:40.602+0000 7f96fa533640 1 Processor -- start 2024-08-30T09:11:40.605 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:40.602+0000 7f96fa533640 1 -- start start 2024-08-30T09:11:40.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:40.603+0000 7f96fa533640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f96f410a780 0x7f96f411c200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:40.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:40.603+0000 7f96fa533640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f96f4111ce0 0x7f96f41184e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:40.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:40.603+0000 7f96fa533640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f96f411c9d0 0x7f96f4118aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:40.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:40.603+0000 7f96fa533640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f96f41149b0 con 0x7f96f411c9d0 2024-08-30T09:11:40.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:40.603+0000 7f96f37fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f96f4111ce0 0x7f96f41184e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:40.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:40.603+0000 7f96fa533640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f96f4114830 con 0x7f96f4111ce0 2024-08-30T09:11:40.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:40.603+0000 7f96f3fff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f96f410a780 0x7f96f411c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:40.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:40.603+0000 7f96f37fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f96f4111ce0 0x7f96f41184e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:45426/0 (socket says 172.21.15.42:45426) 2024-08-30T09:11:40.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:40.603+0000 7f96f37fe640 1 -- 172.21.15.42:0/183160855 learned_addr learned my addr 172.21.15.42:0/183160855 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:11:40.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:40.603+0000 7f96fa533640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f96f4114b30 con 0x7f96f410a780 2024-08-30T09:11:40.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:40.608+0000 7f96f8ad0640 1 --2- 172.21.15.42:0/183160855 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f96f411c9d0 0x7f96f4118aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:40.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:40.610+0000 7f96f8ad0640 1 -- 172.21.15.42:0/183160855 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f96f410a780 msgr2=0x7f96f411c200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:11:40.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:40.610+0000 7f96f8ad0640 1 --2- 172.21.15.42:0/183160855 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f96f410a780 0x7f96f411c200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:40.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:40.610+0000 7f96f8ad0640 1 -- 172.21.15.42:0/183160855 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f96f4111ce0 msgr2=0x7f96f41184e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:11:40.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:40.610+0000 7f96f8ad0640 1 --2- 172.21.15.42:0/183160855 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f96f4111ce0 0x7f96f41184e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:40.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:40.610+0000 7f96f8ad0640 1 -- 172.21.15.42:0/183160855 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96f41192c0 con 0x7f96f411c9d0 2024-08-30T09:11:40.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:40.611+0000 7f96f8ad0640 1 --2- 172.21.15.42:0/183160855 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f96f411c9d0 0x7f96f4118aa0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f96ec01fb30 tx=0x7f96ec01d380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:11:40.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:40.613+0000 7f96f17fa640 1 -- 172.21.15.42:0/183160855 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96ec01ddc0 con 0x7f96f411c9d0 2024-08-30T09:11:40.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:40.613+0000 7f96f17fa640 1 -- 172.21.15.42:0/183160855 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f96ec003970 con 0x7f96f411c9d0 2024-08-30T09:11:40.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:40.613+0000 7f96f17fa640 1 -- 172.21.15.42:0/183160855 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96ec004510 con 0x7f96f411c9d0 2024-08-30T09:11:40.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:40.614+0000 7f96fa533640 1 -- 172.21.15.42:0/183160855 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96f41acc30 con 0x7f96f411c9d0 2024-08-30T09:11:40.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:40.615+0000 7f96fa533640 1 -- 172.21.15.42:0/183160855 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96f41ad120 con 0x7f96f411c9d0 2024-08-30T09:11:40.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:40.618+0000 7f96fa533640 1 -- 172.21.15.42:0/183160855 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96f410ab90 con 0x7f96f411c9d0 2024-08-30T09:11:40.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:40.619+0000 7f96f17fa640 1 -- 172.21.15.42:0/183160855 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7f96ec02e020 con 0x7f96f411c9d0 2024-08-30T09:11:40.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:40.621+0000 7f96f17fa640 1 --2- 172.21.15.42:0/183160855 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f96dc0b4cc0 0x7f96dc0b71d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:40.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:40.621+0000 7f96f17fa640 1 -- 172.21.15.42:0/183160855 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f96ec0f6960 con 0x7f96f411c9d0 2024-08-30T09:11:40.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:40.624+0000 7f96f3fff640 1 --2- 172.21.15.42:0/183160855 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f96dc0b4cc0 0x7f96dc0b71d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:40.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:40.631+0000 7f96f17fa640 1 -- 172.21.15.42:0/183160855 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f96ec065130 con 0x7f96f411c9d0 2024-08-30T09:11:40.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:40.633+0000 7f96f3fff640 1 --2- 172.21.15.42:0/183160855 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f96dc0b4cc0 0x7f96dc0b71d0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f96e4009830 tx=0x7f96e402a880 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:11:40.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:11:40 smithi042 ceph-mon[18652]: pgmap v700: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.8 MiB/s wr, 566 op/s 2024-08-30T09:11:40.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:11:40 smithi190 ceph-mon[22001]: pgmap v700: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.8 MiB/s wr, 566 op/s 2024-08-30T09:11:40.953 INFO:teuthology.orchestra.run.smithi042.stderr:progress: 655.0 s, 481.2 tps, lat 204.068 ms stddev 235.986 2024-08-30T09:11:41.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:41.043+0000 7f96fa533640 1 -- 172.21.15.42:0/183160855 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f96f41ad3b0 con 0x7f96f411c9d0 2024-08-30T09:11:41.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:41.046+0000 7f96f17fa640 1 -- 172.21.15.42:0/183160855 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) ==== 94+0+11918 (secure 0 0 0) 0x7f96ec0bda50 con 0x7f96f411c9d0 2024-08-30T09:11:41.049 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-08-30T09:11:41.049 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":41,"btime":"2024-08-30T09:10:58:348262+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14703,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/3188400000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":3188400000},{"type":"v1","addr":"172.21.15.42:6835","nonce":3188400000}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14715,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/3356537674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":3356537674},{"type":"v1","addr":"172.21.15.42:6837","nonce":3356537674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14766,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6839/2107785513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6838","nonce":2107785513},{"type":"v1","addr":"172.21.15.42:6839","nonce":2107785513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14778,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6841/2298438674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6840","nonce":2298438674},{"type":"v1","addr":"172.21.15.42:6841","nonce":2298438674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24553,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/543555570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":543555570},{"type":"v1","addr":"172.21.15.190:6833","nonce":543555570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24554,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/3907818941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":3907818941},{"type":"v1","addr":"172.21.15.167:6833","nonce":3907818941}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":41,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-30T08:58:03.055084+0000","modified":"2024-08-30T09:10:57.650059+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24629,"mds_1":24610,"mds_2":24599},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24572":{"gid":24572,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6835/234596845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":234596845},{"type":"v1","addr":"172.21.15.167:6835","nonce":234596845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24592":{"gid":24592,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6835/247905709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":247905709},{"type":"v1","addr":"172.21.15.190:6835","nonce":247905709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24598":{"gid":24598,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6837/863256349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6836","nonce":863256349},{"type":"v1","addr":"172.21.15.190:6837","nonce":863256349}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24599":{"gid":24599,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.167:6837/3011837856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":3011837856},{"type":"v1","addr":"172.21.15.167:6837","nonce":3011837856}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24610":{"gid":24610,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.190:6839/719981110","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6838","nonce":719981110},{"type":"v1","addr":"172.21.15.190:6839","nonce":719981110}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24629":{"gid":24629,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.167:6839/3604645207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":3604645207},{"type":"v1","addr":"172.21.15.167:6839","nonce":3604645207}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24629,"qdb_cluster":[24599,24610,24629]},"id":1}]} 2024-08-30T09:11:41.049 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 41 2024-08-30T09:11:41.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:41.051+0000 7f96fa533640 1 -- 172.21.15.42:0/183160855 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f96dc0b4cc0 msgr2=0x7f96dc0b71d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:11:41.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:41.051+0000 7f96fa533640 1 --2- 172.21.15.42:0/183160855 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f96dc0b4cc0 0x7f96dc0b71d0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f96e4009830 tx=0x7f96e402a880 comp rx=0 tx=0).stop 2024-08-30T09:11:41.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:41.051+0000 7f96fa533640 1 -- 172.21.15.42:0/183160855 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f96f411c9d0 msgr2=0x7f96f4118aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:11:41.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:41.051+0000 7f96fa533640 1 --2- 172.21.15.42:0/183160855 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f96f411c9d0 0x7f96f4118aa0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f96ec01fb30 tx=0x7f96ec01d380 comp rx=0 tx=0).stop 2024-08-30T09:11:41.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:41.056+0000 7f96fa533640 1 -- 172.21.15.42:0/183160855 shutdown_connections 2024-08-30T09:11:41.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:41.056+0000 7f96fa533640 1 --2- 172.21.15.42:0/183160855 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f96dc0b4cc0 0x7f96dc0b71d0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:41.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:41.056+0000 7f96fa533640 1 --2- 172.21.15.42:0/183160855 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f96f411c9d0 0x7f96f4118aa0 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:41.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:41.056+0000 7f96fa533640 1 --2- 172.21.15.42:0/183160855 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f96f4111ce0 0x7f96f41184e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:41.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:41.056+0000 7f96fa533640 1 --2- 172.21.15.42:0/183160855 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f96f410a780 0x7f96f411c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:41.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:41.056+0000 7f96fa533640 1 -- 172.21.15.42:0/183160855 >> 172.21.15.42:0/183160855 conn(0x7f96f4070c60 msgr2=0x7f96f4073bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:11:41.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:41.061+0000 7f96fa533640 1 -- 172.21.15.42:0/183160855 shutdown_connections 2024-08-30T09:11:41.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:41.064+0000 7f96fa533640 1 -- 172.21.15.42:0/183160855 wait complete. 2024-08-30T09:11:41.076 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:11:40 smithi167 ceph-mon[22063]: pgmap v700: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.8 MiB/s wr, 566 op/s 2024-08-30T09:11:41.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:11:41 smithi042 ceph-mon[18652]: from='client.? 172.21.15.42:0/183160855' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-30T09:11:41.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:11:41 smithi190 ceph-mon[22001]: from='client.? 172.21.15.42:0/183160855' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-30T09:11:42.010 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-08-30T09:11:42.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:11:41 smithi167 ceph-mon[22063]: from='client.? 172.21.15.42:0/183160855' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-30T09:11:42.462 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T09:11:42.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:11:42 smithi042 ceph-mon[18652]: pgmap v701: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 5.4 MiB/s wr, 437 op/s 2024-08-30T09:11:42.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:11:42 smithi190 ceph-mon[22001]: pgmap v701: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 5.4 MiB/s wr, 437 op/s 2024-08-30T09:11:43.046 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.044+0000 7f4e58c94640 1 -- 172.21.15.42:0/3159414472 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4e54103510 msgr2=0x7f4e54109eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:11:43.046 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.044+0000 7f4e58c94640 1 --2- 172.21.15.42:0/3159414472 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4e54103510 0x7f4e54109eb0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f4e4c009f40 tx=0x7f4e4c02b820 comp rx=0 tx=0).stop 2024-08-30T09:11:43.047 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.044+0000 7f4e58c94640 1 -- 172.21.15.42:0/3159414472 shutdown_connections 2024-08-30T09:11:43.047 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.044+0000 7f4e58c94640 1 --2- 172.21.15.42:0/3159414472 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4e54102b70 0x7f4e54102f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:43.047 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.044+0000 7f4e58c94640 1 --2- 172.21.15.42:0/3159414472 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4e54103510 0x7f4e54109eb0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:43.047 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.044+0000 7f4e58c94640 1 --2- 172.21.15.42:0/3159414472 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4e54101af0 0x7f4e54101f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:43.047 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.044+0000 7f4e58c94640 1 -- 172.21.15.42:0/3159414472 >> 172.21.15.42:0/3159414472 conn(0x7f4e540fd430 msgr2=0x7f4e540ff8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:11:43.047 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.044+0000 7f4e58c94640 1 -- 172.21.15.42:0/3159414472 shutdown_connections 2024-08-30T09:11:43.047 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.044+0000 7f4e58c94640 1 -- 172.21.15.42:0/3159414472 wait complete. 2024-08-30T09:11:43.048 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.045+0000 7f4e58c94640 1 Processor -- start 2024-08-30T09:11:43.048 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.045+0000 7f4e58c94640 1 -- start start 2024-08-30T09:11:43.049 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.045+0000 7f4e58c94640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4e54101af0 0x7f4e541a0310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:43.049 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.045+0000 7f4e58c94640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4e54102b70 0x7f4e541a0980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:43.049 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.045+0000 7f4e58c94640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4e54103510 0x7f4e541a1530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:43.049 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.045+0000 7f4e58c94640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f4e5410cee0 con 0x7f4e54103510 2024-08-30T09:11:43.049 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.045+0000 7f4e58c94640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4e5410cd60 con 0x7f4e54102b70 2024-08-30T09:11:43.049 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.045+0000 7f4e58c94640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f4e5410d060 con 0x7f4e54101af0 2024-08-30T09:11:43.049 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.046+0000 7f4e51d9b640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4e54102b70 0x7f4e541a0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:43.049 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.046+0000 7f4e51d9b640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4e54102b70 0x7f4e541a0980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:45436/0 (socket says 172.21.15.42:45436) 2024-08-30T09:11:43.049 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.046+0000 7f4e51d9b640 1 -- 172.21.15.42:0/1040963234 learned_addr learned my addr 172.21.15.42:0/1040963234 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:11:43.049 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.046+0000 7f4e5259c640 1 --2- 172.21.15.42:0/1040963234 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4e54101af0 0x7f4e541a0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:43.050 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.046+0000 7f4e52d9d640 1 --2- 172.21.15.42:0/1040963234 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4e54103510 0x7f4e541a1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:43.050 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.046+0000 7f4e51d9b640 1 -- 172.21.15.42:0/1040963234 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4e54101af0 msgr2=0x7f4e541a0310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:11:43.050 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.046+0000 7f4e51d9b640 1 --2- 172.21.15.42:0/1040963234 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4e54101af0 0x7f4e541a0310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:43.050 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.046+0000 7f4e51d9b640 1 -- 172.21.15.42:0/1040963234 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4e54103510 msgr2=0x7f4e541a1530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:11:43.050 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.046+0000 7f4e51d9b640 1 --2- 172.21.15.42:0/1040963234 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4e54103510 0x7f4e541a1530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:43.050 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.046+0000 7f4e51d9b640 1 -- 172.21.15.42:0/1040963234 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e541a1cc0 con 0x7f4e54102b70 2024-08-30T09:11:43.050 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.046+0000 7f4e51d9b640 1 --2- 172.21.15.42:0/1040963234 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4e54102b70 0x7f4e541a0980 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f4e4800c920 tx=0x7f4e4800cde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:11:43.063 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.061+0000 7f4e437fe640 1 -- 172.21.15.42:0/1040963234 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e48007c70 con 0x7f4e54102b70 2024-08-30T09:11:43.063 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.061+0000 7f4e437fe640 1 -- 172.21.15.42:0/1040963234 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4e48007e10 con 0x7f4e54102b70 2024-08-30T09:11:43.065 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.062+0000 7f4e437fe640 1 -- 172.21.15.42:0/1040963234 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e48018820 con 0x7f4e54102b70 2024-08-30T09:11:43.066 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.064+0000 7f4e58c94640 1 -- 172.21.15.42:0/1040963234 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e541a1fb0 con 0x7f4e54102b70 2024-08-30T09:11:43.066 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.064+0000 7f4e58c94640 1 -- 172.21.15.42:0/1040963234 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e541a8550 con 0x7f4e54102b70 2024-08-30T09:11:43.071 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.069+0000 7f4e5fc00740 1 Processor -- start 2024-08-30T09:11:43.075 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.073+0000 7f4e5fc00740 1 -- start start 2024-08-30T09:11:43.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:11:42 smithi167 ceph-mon[22063]: pgmap v701: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 5.4 MiB/s wr, 437 op/s 2024-08-30T09:11:43.078 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.073+0000 7f4e5fc00740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559cb24b81b0 0x559cb285df60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:43.078 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.073+0000 7f4e5fc00740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559cb248f2f0 0x559cb285e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:43.078 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.073+0000 7f4e5fc00740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559cb24974d0 0x559cb285f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:43.078 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.073+0000 7f4e5fc00740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x559cb27d1000 con 0x559cb24b81b0 2024-08-30T09:11:43.078 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.073+0000 7f4e5fc00740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x559cb27c0590 con 0x559cb24974d0 2024-08-30T09:11:43.078 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.073+0000 7f4e5fc00740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x559cb27c4540 con 0x559cb248f2f0 2024-08-30T09:11:43.078 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.074+0000 7f4e5259c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559cb248f2f0 0x559cb285e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:43.078 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.075+0000 7f4e5259c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559cb248f2f0 0x559cb285e5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:42238/0 (socket says 172.21.15.42:42238) 2024-08-30T09:11:43.078 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.075+0000 7f4e5259c640 1 -- 172.21.15.42:0/1777066703 learned_addr learned my addr 172.21.15.42:0/1777066703 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:11:43.079 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.076+0000 7f4e51d9b640 1 --2- 172.21.15.42:0/1777066703 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559cb24974d0 0x559cb285f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:43.079 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.076+0000 7f4e437fe640 1 -- 172.21.15.42:0/1040963234 <== mon.2 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7f4e480189c0 con 0x7f4e54102b70 2024-08-30T09:11:43.079 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.076+0000 7f4e51d9b640 1 -- 172.21.15.42:0/1777066703 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559cb248f2f0 msgr2=0x559cb285e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:11:43.079 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.076+0000 7f4e51d9b640 1 --2- 172.21.15.42:0/1777066703 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559cb248f2f0 0x559cb285e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:43.079 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.076+0000 7f4e51d9b640 1 -- 172.21.15.42:0/1777066703 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559cb24b81b0 msgr2=0x559cb285df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:11:43.079 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.076+0000 7f4e51d9b640 1 --2- 172.21.15.42:0/1777066703 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559cb24b81b0 0x559cb285df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:43.080 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.076+0000 7f4e51d9b640 1 -- 172.21.15.42:0/1777066703 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559cb2523790 con 0x559cb24974d0 2024-08-30T09:11:43.080 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.077+0000 7f4e51d9b640 1 --2- 172.21.15.42:0/1777066703 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559cb24974d0 0x559cb285f120 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f4e480e2620 tx=0x7f4e480ed570 comp rx=0 tx=0).ready entity=mon.2 client_cookie=513555fb446be719 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:11:43.080 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.078+0000 7f4e437fe640 1 --2- 172.21.15.42:0/1040963234 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f4e3c0b4bd0 0x7f4e3c0b70e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:43.080 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.079+0000 7f4e40ff9640 1 -- 172.21.15.42:0/1777066703 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e480f8070 con 0x559cb24974d0 2024-08-30T09:11:43.080 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.079+0000 7f4e40ff9640 1 -- 172.21.15.42:0/1777066703 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4e480f3ab0 con 0x559cb24974d0 2024-08-30T09:11:43.081 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.079+0000 7f4e437fe640 1 -- 172.21.15.42:0/1040963234 <== mon.2 v2:172.21.15.167:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f4e480e4f70 con 0x7f4e54102b70 2024-08-30T09:11:43.085 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.080+0000 7f4e5fc00740 1 -- 172.21.15.42:0/1777066703 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559cb24974d0 msgr2=0x559cb285f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:11:43.085 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.080+0000 7f4e5fc00740 1 --2- 172.21.15.42:0/1777066703 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559cb24974d0 0x559cb285f120 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f4e480e2620 tx=0x7f4e480ed570 comp rx=0 tx=0).stop 2024-08-30T09:11:43.085 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.083+0000 7f4e52d9d640 1 --2- 172.21.15.42:0/1040963234 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f4e3c0b4bd0 0x7f4e3c0b70e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:43.086 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.085+0000 7f4e5fc00740 1 -- 172.21.15.42:0/1777066703 shutdown_connections 2024-08-30T09:11:43.086 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.085+0000 7f4e5fc00740 1 --2- 172.21.15.42:0/1777066703 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559cb248f2f0 0x559cb285e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:43.087 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.085+0000 7f4e5fc00740 1 --2- 172.21.15.42:0/1777066703 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559cb24974d0 0x559cb285f120 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:43.087 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.085+0000 7f4e5fc00740 1 --2- 172.21.15.42:0/1777066703 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559cb24b81b0 0x559cb285df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:43.087 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.085+0000 7f4e5fc00740 1 -- 172.21.15.42:0/1777066703 >> 172.21.15.42:0/1777066703 conn(0x559cb24ac300 msgr2=0x559cb28599f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:11:43.091 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.089+0000 7f4e5fc00740 1 -- 172.21.15.42:0/1777066703 shutdown_connections 2024-08-30T09:11:43.091 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.089+0000 7f4e52d9d640 1 --2- 172.21.15.42:0/1040963234 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f4e3c0b4bd0 0x7f4e3c0b70e0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f4e4c02be10 tx=0x7f4e4c0061f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:11:43.097 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.095+0000 7f4e5fc00740 1 -- 172.21.15.42:0/1777066703 wait complete. 2024-08-30T09:11:43.097 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.095+0000 7f4e5fc00740 1 -- start start 2024-08-30T09:11:43.097 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.095+0000 7f4e5fc00740 1 Processor -- start 2024-08-30T09:11:43.098 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.096+0000 7f4e5fc00740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559cb248f2f0 0x559cb2867ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:43.098 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.096+0000 7f4e5fc00740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559cb24974d0 0x559cb28681a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:43.098 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.096+0000 7f4e5fc00740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559cb24b81b0 0x559cb2868d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:43.098 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.096+0000 7f4e5fc00740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x559cb26f9ab0 con 0x559cb24974d0 2024-08-30T09:11:43.098 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.096+0000 7f4e5fc00740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x559cb2747680 con 0x559cb248f2f0 2024-08-30T09:11:43.098 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.096+0000 7f4e5fc00740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x559cb285fac0 con 0x559cb24b81b0 2024-08-30T09:11:43.098 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.096+0000 7f4e5fc00740 10 client.0 _ll_register_callbacks cb 0x559cb24cfa00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:11:43.102 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.100+0000 7f4e52d9d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559cb24b81b0 0x559cb2868d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:43.102 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.100+0000 7f4e52d9d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559cb24b81b0 0x559cb2868d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:42244/0 (socket says 172.21.15.42:42244) 2024-08-30T09:11:43.102 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.100+0000 7f4e52d9d640 1 -- 172.21.15.42:0/271285606 learned_addr learned my addr 172.21.15.42:0/271285606 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:11:43.102 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.100+0000 7f4e51d9b640 1 --2- 172.21.15.42:0/271285606 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559cb24974d0 0x559cb28681a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:43.102 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.101+0000 7f4e5259c640 1 --2- 172.21.15.42:0/271285606 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559cb248f2f0 0x559cb2867ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:43.103 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.101+0000 7f4e52d9d640 1 -- 172.21.15.42:0/271285606 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559cb248f2f0 msgr2=0x559cb2867ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:11:43.103 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.101+0000 7f4e52d9d640 1 --2- 172.21.15.42:0/271285606 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559cb248f2f0 0x559cb2867ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:43.103 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.101+0000 7f4e52d9d640 1 -- 172.21.15.42:0/271285606 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559cb24974d0 msgr2=0x559cb28681a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:11:43.103 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.101+0000 7f4e52d9d640 1 --2- 172.21.15.42:0/271285606 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559cb24974d0 0x559cb28681a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:43.103 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.101+0000 7f4e52d9d640 1 -- 172.21.15.42:0/271285606 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559cb259c620 con 0x559cb24b81b0 2024-08-30T09:11:43.104 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.102+0000 7f4e52d9d640 1 --2- 172.21.15.42:0/271285606 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559cb24b81b0 0x559cb2868d10 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f4e4c0062a0 tx=0x7f4e4c003f50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=57e82424b39ac277 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:11:43.107 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.105+0000 7f4e39ffb640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-08-30T09:11:43.107 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.105+0000 7f4e39ffb640 1 -- 172.21.15.42:0/271285606 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e4c04e060 con 0x559cb24b81b0 2024-08-30T09:11:43.107 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.105+0000 7f4e39ffb640 1 -- 172.21.15.42:0/271285606 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f4e4c030040 con 0x559cb24b81b0 2024-08-30T09:11:43.107 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.105+0000 7f4e5fc00740 1 -- 172.21.15.42:0/271285606 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x559cb27d2f50 con 0x559cb24b81b0 2024-08-30T09:11:43.110 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.109+0000 7f4e39ffb640 1 -- 172.21.15.42:0/271285606 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f4e4c038040 con 0x559cb24b81b0 2024-08-30T09:11:43.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.105+0000 7fa0ffd1a640 1 -- 172.21.15.42:0/1266285119 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa0f810a780 msgr2=0x7fa0f810ab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:11:43.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.105+0000 7fa0ffd1a640 1 --2- 172.21.15.42:0/1266285119 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa0f810a780 0x7fa0f810ab90 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fa0f0009ca0 tx=0x7fa0f002a8f0 comp rx=0 tx=0).stop 2024-08-30T09:11:43.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.107+0000 7fa0ffd1a640 1 -- 172.21.15.42:0/1266285119 shutdown_connections 2024-08-30T09:11:43.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.107+0000 7fa0ffd1a640 1 --2- 172.21.15.42:0/1266285119 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa0f811c9d0 0x7fa0f811eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:43.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.107+0000 7fa0ffd1a640 1 --2- 172.21.15.42:0/1266285119 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa0f810a780 0x7fa0f810ab90 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:43.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.107+0000 7fa0ffd1a640 1 --2- 172.21.15.42:0/1266285119 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa0f8111ce0 0x7fa0f810a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:43.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.107+0000 7fa0ffd1a640 1 -- 172.21.15.42:0/1266285119 >> 172.21.15.42:0/1266285119 conn(0x7fa0f8070c60 msgr2=0x7fa0f8071070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:11:43.112 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.111+0000 7f4e5fc00740 10 client.27016 fetch_fsmap learned FSMap version 41 2024-08-30T09:11:43.112 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.111+0000 7f4e5fc00740 1 -- 172.21.15.42:0/271285606 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x559cb2882dc0 con 0x559cb24b81b0 2024-08-30T09:11:43.116 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.114+0000 7f4e39ffb640 1 -- 172.21.15.42:0/271285606 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x559cb27d2f50 con 0x559cb24b81b0 2024-08-30T09:11:43.116 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.114+0000 7f4e39ffb640 1 -- 172.21.15.42:0/271285606 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 41) ==== 8362+0+0 (secure 0 0 0) 0x7f4e4c0499f0 con 0x559cb24b81b0 2024-08-30T09:11:43.117 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.115+0000 7f4e5fc00740 10 client.27016 fetch_fsmap finished waiting for FSMap version 41 2024-08-30T09:11:43.117 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.115+0000 7f4e5fc00740 10 client.27016 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-30T09:11:43.117 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.115+0000 7f4e5fc00740 20 client.27016 populate_metadata read hostname 'smithi042' 2024-08-30T09:11:43.117 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.115+0000 7f4e5fc00740 1 --2- 172.21.15.42:0/271285606 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x559cb2476d80 0x559cb2886990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:43.117 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.115+0000 7f4e5fc00740 4 client.27016 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-30T09:11:43.117 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.115+0000 7f4e5fc00740 1 -- 172.21.15.42:0/271285606 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559cb27ce9f0 con 0x559cb2476d80 2024-08-30T09:11:43.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.111+0000 7fa0ffd1a640 1 -- 172.21.15.42:0/1266285119 shutdown_connections 2024-08-30T09:11:43.119 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.117+0000 7f4e5259c640 1 --2- 172.21.15.42:0/271285606 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x559cb2476d80 0x559cb2886990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:43.123 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.121+0000 7f4e5259c640 1 --2- 172.21.15.42:0/271285606 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x559cb2476d80 0x559cb2886990 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b28e51a2f0ecc676 server_cookie=e6faeb7cab0c868d in_seq=0 out_seq=0 2024-08-30T09:11:43.125 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.124+0000 7f4e39ffb640 10 client.27016 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:11:43.126 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.124+0000 7f4e39ffb640 1 -- 172.21.15.42:0/271285606 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x559cb27ce9f0 con 0x559cb2476d80 2024-08-30T09:11:43.126 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.124+0000 7f4e39ffb640 10 client.27016 handle_command_reply: tid=0 2024-08-30T09:11:43.126 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.124+0000 7f4e5fc00740 1 client.27016 shutdown 2024-08-30T09:11:43.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.118+0000 7fa0ffd1a640 1 -- 172.21.15.42:0/1266285119 wait complete. 2024-08-30T09:11:43.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.119+0000 7fa0ffd1a640 1 Processor -- start 2024-08-30T09:11:43.134 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.126+0000 7fa0ffd1a640 1 -- start start 2024-08-30T09:11:43.134 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.126+0000 7fa0ffd1a640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa0f810a780 0x7fa0f811a190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:43.134 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.126+0000 7fa0ffd1a640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa0f811c9d0 0x7fa0f8113460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:43.134 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.126+0000 7fa0ffd1a640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa0f81139a0 0x7fa0f8113df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:43.134 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.126+0000 7fa0ffd1a640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fa0f8122700 con 0x7fa0f81139a0 2024-08-30T09:11:43.134 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.126+0000 7fa0ffd1a640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa0f8122580 con 0x7fa0f810a780 2024-08-30T09:11:43.134 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.126+0000 7fa0ffd1a640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fa0f8122880 con 0x7fa0f811c9d0 2024-08-30T09:11:43.134 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.131+0000 7fa0fd2b5640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa0f811c9d0 0x7fa0f8113460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:43.135 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.132+0000 7fa0fe2b7640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa0f81139a0 0x7fa0f8113df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:43.135 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.132+0000 7fa0fe2b7640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa0f81139a0 0x7fa0f8113df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:48420/0 (socket says 172.21.15.42:48420) 2024-08-30T09:11:43.135 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.132+0000 7fa0fe2b7640 1 -- 172.21.15.42:0/3863621708 learned_addr learned my addr 172.21.15.42:0/3863621708 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:11:43.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.133+0000 7fa0fe2b7640 1 -- 172.21.15.42:0/3863621708 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa0f811c9d0 msgr2=0x7fa0f8113460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:11:43.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.133+0000 7fa0fe2b7640 1 --2- 172.21.15.42:0/3863621708 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa0f811c9d0 0x7fa0f8113460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:43.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.133+0000 7fa0fe2b7640 1 -- 172.21.15.42:0/3863621708 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa0f810a780 msgr2=0x7fa0f811a190 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-30T09:11:43.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.133+0000 7fa0fe2b7640 1 --2- 172.21.15.42:0/3863621708 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa0f810a780 0x7fa0f811a190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:43.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.133+0000 7fa0fe2b7640 1 -- 172.21.15.42:0/3863621708 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0f8118fa0 con 0x7fa0f81139a0 2024-08-30T09:11:43.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.134+0000 7fa0fe2b7640 1 --2- 172.21.15.42:0/3863621708 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa0f81139a0 0x7fa0f8113df0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fa0e800d750 tx=0x7fa0e800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:11:43.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.134+0000 7fa0eeffd640 1 -- 172.21.15.42:0/3863621708 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0e8017070 con 0x7fa0f81139a0 2024-08-30T09:11:43.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.134+0000 7fa0eeffd640 1 -- 172.21.15.42:0/3863621708 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa0e80054d0 con 0x7fa0f81139a0 2024-08-30T09:11:43.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.134+0000 7fa0eeffd640 1 -- 172.21.15.42:0/3863621708 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0e8004c60 con 0x7fa0f81139a0 2024-08-30T09:11:43.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.135+0000 7fa0ffd1a640 1 -- 172.21.15.42:0/3863621708 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0f8119290 con 0x7fa0f81139a0 2024-08-30T09:11:43.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.135+0000 7fa0ffd1a640 1 -- 172.21.15.42:0/3863621708 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0f8119780 con 0x7fa0f81139a0 2024-08-30T09:11:43.143 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.142+0000 7f4e5fc00740 1 -- 172.21.15.42:0/271285606 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559cb24b81b0 msgr2=0x559cb2868d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:11:43.144 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.142+0000 7f4e5fc00740 1 --2- 172.21.15.42:0/271285606 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559cb24b81b0 0x559cb2868d10 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f4e4c0062a0 tx=0x7f4e4c003f50 comp rx=0 tx=0).stop 2024-08-30T09:11:43.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.141+0000 7fa0eeffd640 1 -- 172.21.15.42:0/3863621708 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7fa0e8010460 con 0x7fa0f81139a0 2024-08-30T09:11:43.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.142+0000 7fa0eeffd640 1 --2- 172.21.15.42:0/3863621708 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fa0e40b4d30 0x7fa0e40b7240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:43.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.142+0000 7fa0eeffd640 1 -- 172.21.15.42:0/3863621708 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7fa0e80e5e30 con 0x7fa0f81139a0 2024-08-30T09:11:43.148 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.146+0000 7f4e5fc00740 1 -- 172.21.15.42:0/271285606 shutdown_connections 2024-08-30T09:11:43.148 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.146+0000 7f4e5fc00740 1 --2- 172.21.15.42:0/271285606 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x559cb2476d80 0x559cb2886990 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:43.148 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.146+0000 7f4e5fc00740 1 --2- 172.21.15.42:0/271285606 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559cb24b81b0 0x559cb2868d10 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:43.148 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.146+0000 7f4e5fc00740 1 --2- 172.21.15.42:0/271285606 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559cb24974d0 0x559cb28681a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:43.148 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.146+0000 7f4e5fc00740 1 --2- 172.21.15.42:0/271285606 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559cb248f2f0 0x559cb2867ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:43.148 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.146+0000 7f4e5fc00740 1 -- 172.21.15.42:0/271285606 >> 172.21.15.42:0/271285606 conn(0x559cb24ac300 msgr2=0x559cb2857bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:11:43.148 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.146+0000 7f4e39ffb640 0 client.27016 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:11:43.152 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.150+0000 7f4e5fc00740 1 -- 172.21.15.42:0/271285606 shutdown_connections 2024-08-30T09:11:43.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.146+0000 7fa10628f740 1 Processor -- start 2024-08-30T09:11:43.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.146+0000 7fa10628f740 1 -- start start 2024-08-30T09:11:43.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.147+0000 7fa10628f740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563c7f7d9070 0x563c7fb7e310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:43.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.147+0000 7fa10628f740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563c7f7b01b0 0x563c7fb7e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:43.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.147+0000 7fa10628f740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563c7f7b8390 0x563c7fb7ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:43.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.147+0000 7fa10628f740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x563c7faf5680 con 0x563c7f7b01b0 2024-08-30T09:11:43.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.147+0000 7fa10628f740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x563c7fae4c10 con 0x563c7f7b8390 2024-08-30T09:11:43.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.147+0000 7fa10628f740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x563c7fae8bc0 con 0x563c7f7d9070 2024-08-30T09:11:43.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.147+0000 7fa0fdab6640 1 --2- 172.21.15.42:0/3863621708 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fa0e40b4d30 0x7fa0e40b7240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:43.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.148+0000 7fa0fe2b7640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563c7f7b8390 0x563c7fb7ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:43.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.148+0000 7fa0fe2b7640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563c7f7b8390 0x563c7fb7ed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:45474/0 (socket says 172.21.15.42:45474) 2024-08-30T09:11:43.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.148+0000 7fa0fe2b7640 1 -- 172.21.15.42:0/916642079 learned_addr learned my addr 172.21.15.42:0/916642079 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:11:43.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.152+0000 7fa0fd2b5640 1 --2- 172.21.15.42:0/916642079 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563c7f7b01b0 0x563c7fb7e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:43.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.152+0000 7fa0fe2b7640 1 -- 172.21.15.42:0/916642079 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563c7f7d9070 msgr2=0x563c7fb7e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:11:43.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.152+0000 7fa0fdab6640 1 --2- 172.21.15.42:0/916642079 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563c7f7d9070 0x563c7fb7e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:43.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.152+0000 7fa0fe2b7640 1 --2- 172.21.15.42:0/916642079 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563c7f7d9070 0x563c7fb7e310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:43.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.152+0000 7fa0fe2b7640 1 -- 172.21.15.42:0/916642079 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563c7f7b01b0 msgr2=0x563c7fb7e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:11:43.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.152+0000 7fa0fe2b7640 1 --2- 172.21.15.42:0/916642079 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563c7f7b01b0 0x563c7fb7e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:43.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.152+0000 7fa0fe2b7640 1 -- 172.21.15.42:0/916642079 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c7f844650 con 0x563c7f7b8390 2024-08-30T09:11:43.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.152+0000 7fa0fd2b5640 1 --2- 172.21.15.42:0/916642079 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563c7f7b01b0 0x563c7fb7e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-30T09:11:43.157 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.155+0000 7f4e5fc00740 1 -- 172.21.15.42:0/271285606 wait complete. 2024-08-30T09:11:43.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.155+0000 7fa0fe2b7640 1 --2- 172.21.15.42:0/916642079 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563c7f7b8390 0x563c7fb7ed90 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fa0e8009610 tx=0x7fa0e8005680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b8ba64a2c7634acb server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:11:43.159 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.157+0000 7f4e5fc00740 20 client.27016 trim_cache size 0 max 16384 2024-08-30T09:11:43.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.158+0000 7fa0e3fff640 1 -- 172.21.15.42:0/916642079 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0e8020040 con 0x563c7f7b8390 2024-08-30T09:11:43.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.158+0000 7fa0e3fff640 1 -- 172.21.15.42:0/916642079 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa0e80ac470 con 0x563c7f7b8390 2024-08-30T09:11:43.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.158+0000 7fa0e3fff640 1 -- 172.21.15.42:0/916642079 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0e80cdbe0 con 0x563c7f7b8390 2024-08-30T09:11:43.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.160+0000 7fa10628f740 1 -- 172.21.15.42:0/916642079 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563c7f7b8390 msgr2=0x563c7fb7ed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:11:43.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.160+0000 7fa10628f740 1 --2- 172.21.15.42:0/916642079 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563c7f7b8390 0x563c7fb7ed90 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fa0e8009610 tx=0x7fa0e8005680 comp rx=0 tx=0).stop 2024-08-30T09:11:43.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.165+0000 7fa10628f740 1 -- 172.21.15.42:0/916642079 shutdown_connections 2024-08-30T09:11:43.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.165+0000 7fa10628f740 1 --2- 172.21.15.42:0/916642079 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563c7f7b8390 0x563c7fb7ed90 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:43.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.165+0000 7fa10628f740 1 --2- 172.21.15.42:0/916642079 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563c7f7b01b0 0x563c7fb7e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:43.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.165+0000 7fa10628f740 1 --2- 172.21.15.42:0/916642079 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563c7f7d9070 0x563c7fb7e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:43.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.165+0000 7fa10628f740 1 -- 172.21.15.42:0/916642079 >> 172.21.15.42:0/916642079 conn(0x563c7f7cd460 msgr2=0x563c7fb7c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:11:43.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.168+0000 7fa0fdab6640 1 --2- 172.21.15.42:0/3863621708 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fa0e40b4d30 0x7fa0e40b7240 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fa0f80751f0 tx=0x7fa0f400b350 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:11:43.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.171+0000 7fa10628f740 1 -- 172.21.15.42:0/916642079 shutdown_connections 2024-08-30T09:11:43.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.173+0000 7fa10628f740 1 -- 172.21.15.42:0/916642079 wait complete. 2024-08-30T09:11:43.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.173+0000 7fa10628f740 1 -- start start 2024-08-30T09:11:43.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.173+0000 7fa10628f740 1 Processor -- start 2024-08-30T09:11:43.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.174+0000 7fa10628f740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563c7f7b01b0 0x563c7fb89a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:43.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.174+0000 7fa10628f740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563c7f7b8390 0x563c7fb89fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:43.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.174+0000 7fa10628f740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563c7f7d9070 0x563c7fb95de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:43.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.174+0000 7fa10628f740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x563c7f947700 con 0x563c7f7d9070 2024-08-30T09:11:43.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.174+0000 7fa10628f740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x563c7fa6b750 con 0x563c7f7b8390 2024-08-30T09:11:43.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.174+0000 7fa10628f740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x563c7fb8ca90 con 0x563c7f7b01b0 2024-08-30T09:11:43.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.174+0000 7fa10628f740 10 client.0 _ll_register_callbacks cb 0x563c7fa9ae10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:11:43.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.174+0000 7fa0fd2b5640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563c7f7b8390 0x563c7fb89fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:43.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.174+0000 7fa0fd2b5640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563c7f7b8390 0x563c7fb89fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:45478/0 (socket says 172.21.15.42:45478) 2024-08-30T09:11:43.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.174+0000 7fa0fd2b5640 1 -- 172.21.15.42:0/3286371492 learned_addr learned my addr 172.21.15.42:0/3286371492 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:11:43.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.175+0000 7fa0fd2b5640 1 -- 172.21.15.42:0/3286371492 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563c7f7b01b0 msgr2=0x563c7fb89a90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-30T09:11:43.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.175+0000 7fa0fd2b5640 1 --2- 172.21.15.42:0/3286371492 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563c7f7b01b0 0x563c7fb89a90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:43.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.175+0000 7fa0fd2b5640 1 -- 172.21.15.42:0/3286371492 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563c7f7d9070 msgr2=0x563c7fb95de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:11:43.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.175+0000 7fa0fd2b5640 1 --2- 172.21.15.42:0/3286371492 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563c7f7d9070 0x563c7fb95de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:43.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.175+0000 7fa0fd2b5640 1 -- 172.21.15.42:0/3286371492 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c7faa4900 con 0x563c7f7b8390 2024-08-30T09:11:43.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.175+0000 7fa0fd2b5640 1 --2- 172.21.15.42:0/3286371492 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563c7f7b8390 0x563c7fb89fd0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fa0f0006010 tx=0x7fa0f0031f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1a81a8ef82edc092 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:11:43.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.177+0000 7fa0e17fa640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-30T09:11:43.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.177+0000 7fa0e17fa640 1 -- 172.21.15.42:0/3286371492 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0f0031740 con 0x563c7f7b8390 2024-08-30T09:11:43.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.178+0000 7fa10628f740 1 -- 172.21.15.42:0/3286371492 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa0e80ac470 con 0x563c7f7b8390 2024-08-30T09:11:43.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.178+0000 7fa0e17fa640 1 -- 172.21.15.42:0/3286371492 <== mon.2 v2:172.21.15.167:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7fa0f0003970 con 0x563c7f7b8390 2024-08-30T09:11:43.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.178+0000 7fa0e17fa640 1 -- 172.21.15.42:0/3286371492 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fa0f0003b70 con 0x563c7f7b8390 2024-08-30T09:11:43.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.178+0000 7fa10628f740 10 client.27065 fetch_fsmap learned FSMap version 41 2024-08-30T09:11:43.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.178+0000 7fa10628f740 1 -- 172.21.15.42:0/3286371492 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x563c7fb98800 con 0x563c7f7b8390 2024-08-30T09:11:43.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.181+0000 7fa0e17fa640 1 -- 172.21.15.42:0/3286371492 <== mon.2 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa0e80ac470 con 0x563c7f7b8390 2024-08-30T09:11:43.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.182+0000 7fa0e17fa640 1 -- 172.21.15.42:0/3286371492 <== mon.2 v2:172.21.15.167:3300/0 5 ==== fsmap(e 41) ==== 8362+0+0 (secure 0 0 0) 0x7fa0f002f6a0 con 0x563c7f7b8390 2024-08-30T09:11:43.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.183+0000 7fa10628f740 10 client.27065 fetch_fsmap finished waiting for FSMap version 41 2024-08-30T09:11:43.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.183+0000 7fa10628f740 10 client.27065 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-08-30T09:11:43.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.183+0000 7fa10628f740 20 client.27065 populate_metadata read hostname 'smithi042' 2024-08-30T09:11:43.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.183+0000 7fa10628f740 1 --2- 172.21.15.42:0/3286371492 >> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] conn(0x563c7f7979f0 0x563c7fb9bd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:43.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.183+0000 7fa10628f740 4 client.27065 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-30T09:11:43.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.183+0000 7fa10628f740 1 -- 172.21.15.42:0/3286371492 --> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563c7faf75d0 con 0x563c7f7979f0 2024-08-30T09:11:43.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.183+0000 7fa0fdab6640 1 --2- 172.21.15.42:0/3286371492 >> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] conn(0x563c7f7979f0 0x563c7fb9bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:43.191 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.189+0000 7f4e5fc00740 1 Processor -- start 2024-08-30T09:11:43.191 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.189+0000 7f4e5fc00740 1 -- start start 2024-08-30T09:11:43.192 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.190+0000 7f4e5fc00740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559cb248f2f0 0x559cb2866070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:43.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.190+0000 7fa0fdab6640 1 --2- 172.21.15.42:0/3286371492 >> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] conn(0x563c7f7979f0 0x563c7fb9bd40 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=db7e7556289ca024 server_cookie=9751ce889a57f3b7 in_seq=0 out_seq=0 2024-08-30T09:11:43.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.190+0000 7fa0e17fa640 10 client.27065 ms_handle_connect on v2:172.21.15.167:6836/3011837856 2024-08-30T09:11:43.193 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.190+0000 7f4e5fc00740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559cb24974d0 0x559cb28665b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:43.193 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.190+0000 7f4e5259c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559cb248f2f0 0x559cb2866070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:43.193 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.190+0000 7f4e5259c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559cb248f2f0 0x559cb2866070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:42276/0 (socket says 172.21.15.42:42276) 2024-08-30T09:11:43.193 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.190+0000 7f4e51d9b640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559cb24974d0 0x559cb28665b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:43.193 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.190+0000 7f4e51d9b640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559cb24974d0 0x559cb28665b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:48438/0 (socket says 172.21.15.42:48438) 2024-08-30T09:11:43.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.190+0000 7fa0e17fa640 1 -- 172.21.15.42:0/3286371492 <== mds.2 v2:172.21.15.167:6836/3011837856 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x563c7faf75d0 con 0x563c7f7979f0 2024-08-30T09:11:43.194 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.191+0000 7f4e5259c640 1 -- 172.21.15.42:0/2469671335 learned_addr learned my addr 172.21.15.42:0/2469671335 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:11:43.194 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.191+0000 7f4e5fc00740 1 --2- 172.21.15.42:0/2469671335 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559cb24b81b0 0x559cb2866ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:43.194 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.191+0000 7f4e5fc00740 1 -- 172.21.15.42:0/2469671335 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f4e541aca00 con 0x559cb24974d0 2024-08-30T09:11:43.194 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.191+0000 7f4e52d9d640 1 --2- 172.21.15.42:0/2469671335 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559cb24b81b0 0x559cb2866ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:43.194 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.191+0000 7f4e5fc00740 1 -- 172.21.15.42:0/2469671335 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4e541aa270 con 0x559cb24b81b0 2024-08-30T09:11:43.194 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.191+0000 7f4e5fc00740 1 -- 172.21.15.42:0/2469671335 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f4e54074cf0 con 0x559cb248f2f0 2024-08-30T09:11:43.195 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.192+0000 7f4e5259c640 1 -- 172.21.15.42:0/2469671335 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559cb24b81b0 msgr2=0x559cb2866ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:11:43.195 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.192+0000 7f4e5259c640 1 --2- 172.21.15.42:0/2469671335 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559cb24b81b0 0x559cb2866ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:43.195 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.192+0000 7f4e5259c640 1 -- 172.21.15.42:0/2469671335 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559cb24974d0 msgr2=0x559cb28665b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:11:43.195 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.192+0000 7f4e5259c640 1 --2- 172.21.15.42:0/2469671335 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559cb24974d0 0x559cb28665b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:43.195 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.192+0000 7f4e5259c640 1 -- 172.21.15.42:0/2469671335 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559cb28684d0 con 0x559cb248f2f0 2024-08-30T09:11:43.195 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.193+0000 7f4e5259c640 1 --2- 172.21.15.42:0/2469671335 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559cb248f2f0 0x559cb2866070 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x559cb2887130 tx=0x7f4e44005880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5bab231b1271e56 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:11:43.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.191+0000 7fa0e17fa640 10 client.27065 handle_command_reply: tid=0 2024-08-30T09:11:43.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.191+0000 7fa10628f740 1 client.27065 shutdown 2024-08-30T09:11:43.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.195+0000 7fa10628f740 1 -- 172.21.15.42:0/3286371492 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563c7f7b8390 msgr2=0x563c7fb89fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:11:43.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.195+0000 7fa10628f740 1 --2- 172.21.15.42:0/3286371492 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563c7f7b8390 0x563c7fb89fd0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fa0f0006010 tx=0x7fa0f0031f40 comp rx=0 tx=0).stop 2024-08-30T09:11:43.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.195+0000 7fa10628f740 1 -- 172.21.15.42:0/3286371492 shutdown_connections 2024-08-30T09:11:43.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.195+0000 7fa10628f740 1 --2- 172.21.15.42:0/3286371492 >> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] conn(0x563c7f7979f0 0x563c7fb9bd40 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:43.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.195+0000 7fa10628f740 1 --2- 172.21.15.42:0/3286371492 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563c7f7d9070 0x563c7fb95de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:43.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.195+0000 7fa10628f740 1 --2- 172.21.15.42:0/3286371492 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563c7f7b8390 0x563c7fb89fd0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:43.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.195+0000 7fa10628f740 1 --2- 172.21.15.42:0/3286371492 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563c7f7b01b0 0x563c7fb89a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:43.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.195+0000 7fa10628f740 1 -- 172.21.15.42:0/3286371492 >> 172.21.15.42:0/3286371492 conn(0x563c7f7cd460 msgr2=0x563c7fb814a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:11:43.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.195+0000 7fa0e17fa640 0 client.27065 ms_handle_reset on v2:172.21.15.167:6836/3011837856 2024-08-30T09:11:43.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.195+0000 7fa10628f740 1 -- 172.21.15.42:0/3286371492 shutdown_connections 2024-08-30T09:11:43.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.195+0000 7fa10628f740 1 -- 172.21.15.42:0/3286371492 wait complete. 2024-08-30T09:11:43.199 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.196+0000 7f4e3bfff640 1 -- 172.21.15.42:0/2469671335 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e44014e30 con 0x559cb248f2f0 2024-08-30T09:11:43.199 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.197+0000 7f4e3bfff640 1 -- 172.21.15.42:0/2469671335 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4e44014110 con 0x559cb248f2f0 2024-08-30T09:11:43.199 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.197+0000 7f4e3bfff640 1 -- 172.21.15.42:0/2469671335 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e44003db0 con 0x559cb248f2f0 2024-08-30T09:11:43.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.196+0000 7fa10628f740 20 client.27065 trim_cache size 0 max 16384 2024-08-30T09:11:43.201 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.199+0000 7f4e5fc00740 1 -- 172.21.15.42:0/2469671335 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559cb248f2f0 msgr2=0x559cb2866070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:11:43.201 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.199+0000 7f4e5fc00740 1 --2- 172.21.15.42:0/2469671335 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559cb248f2f0 0x559cb2866070 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x559cb2887130 tx=0x7f4e44005880 comp rx=0 tx=0).stop 2024-08-30T09:11:43.202 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.199+0000 7f4e5fc00740 1 -- 172.21.15.42:0/2469671335 shutdown_connections 2024-08-30T09:11:43.202 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.199+0000 7f4e5fc00740 1 --2- 172.21.15.42:0/2469671335 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559cb24b81b0 0x559cb2866ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:43.202 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.199+0000 7f4e5fc00740 1 --2- 172.21.15.42:0/2469671335 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559cb24974d0 0x559cb28665b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:43.202 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.199+0000 7f4e5fc00740 1 --2- 172.21.15.42:0/2469671335 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559cb248f2f0 0x559cb2866070 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:43.202 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.199+0000 7f4e5fc00740 1 -- 172.21.15.42:0/2469671335 >> 172.21.15.42:0/2469671335 conn(0x559cb24ac300 msgr2=0x559cb2862910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:11:43.202 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.199+0000 7f4e5fc00740 1 -- 172.21.15.42:0/2469671335 shutdown_connections 2024-08-30T09:11:43.202 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.200+0000 7f4e5fc00740 1 -- 172.21.15.42:0/2469671335 wait complete. 2024-08-30T09:11:43.202 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.200+0000 7f4e5fc00740 1 -- start start 2024-08-30T09:11:43.203 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.200+0000 7f4e5fc00740 1 Processor -- start 2024-08-30T09:11:43.203 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.201+0000 7f4e5fc00740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559cb248f2f0 0x559cb286a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:43.203 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.201+0000 7f4e5fc00740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559cb24974d0 0x559cb286b060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:43.204 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.202+0000 7f4e5259c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559cb248f2f0 0x559cb286a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:43.204 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.202+0000 7f4e5259c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559cb248f2f0 0x559cb286a9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:45486/0 (socket says 172.21.15.42:45486) 2024-08-30T09:11:43.204 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.202+0000 7f4e5259c640 1 -- 172.21.15.42:0/1045511650 learned_addr learned my addr 172.21.15.42:0/1045511650 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:11:43.204 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.202+0000 7f4e5fc00740 1 --2- 172.21.15.42:0/1045511650 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559cb24b81b0 0x559cb286bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:43.204 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.202+0000 7f4e5fc00740 1 -- 172.21.15.42:0/1045511650 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f4e541a8840 con 0x559cb24974d0 2024-08-30T09:11:43.205 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.202+0000 7f4e5fc00740 1 -- 172.21.15.42:0/1045511650 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4e541a9ba0 con 0x559cb248f2f0 2024-08-30T09:11:43.205 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.202+0000 7f4e52d9d640 1 --2- 172.21.15.42:0/1045511650 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559cb24b81b0 0x559cb286bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:43.205 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.202+0000 7f4e5fc00740 1 -- 172.21.15.42:0/1045511650 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f4e541a8e00 con 0x559cb24b81b0 2024-08-30T09:11:43.205 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.203+0000 7f4e5259c640 1 -- 172.21.15.42:0/1045511650 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559cb24b81b0 msgr2=0x559cb286bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:11:43.206 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.203+0000 7f4e5259c640 1 --2- 172.21.15.42:0/1045511650 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559cb24b81b0 0x559cb286bbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:43.206 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.203+0000 7f4e5259c640 1 -- 172.21.15.42:0/1045511650 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559cb24974d0 msgr2=0x559cb286b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:11:43.206 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.203+0000 7f4e5259c640 1 --2- 172.21.15.42:0/1045511650 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559cb24974d0 0x559cb286b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:43.206 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.203+0000 7f4e5259c640 1 -- 172.21.15.42:0/1045511650 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559cb286c330 con 0x559cb248f2f0 2024-08-30T09:11:43.206 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.203+0000 7f4e5fc00740 10 client.0 _ll_register_callbacks cb 0x7f4e4c036070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:11:43.206 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.203+0000 7f4e5259c640 1 --2- 172.21.15.42:0/1045511650 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559cb248f2f0 0x559cb286a9a0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f4e44005780 tx=0x7f4e440035f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1c06f095271e6a1e server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:11:43.206 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.204+0000 7f4e5fc00740 1 -- 172.21.15.42:0/1045511650 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4e44014110 con 0x559cb248f2f0 2024-08-30T09:11:43.207 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.205+0000 7f4e39ffb640 10 client.27071 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-30T09:11:43.207 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.205+0000 7f4e39ffb640 1 -- 172.21.15.42:0/1045511650 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e44033060 con 0x559cb248f2f0 2024-08-30T09:11:43.207 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.205+0000 7f4e39ffb640 1 -- 172.21.15.42:0/1045511650 <== mon.2 v2:172.21.15.167:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f4e44003790 con 0x559cb248f2f0 2024-08-30T09:11:43.207 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.205+0000 7f4e39ffb640 1 -- 172.21.15.42:0/1045511650 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f4e4403a050 con 0x559cb248f2f0 2024-08-30T09:11:43.207 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.205+0000 7f4e5fc00740 10 client.27071 fetch_fsmap learned FSMap version 41 2024-08-30T09:11:43.207 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.205+0000 7f4e5fc00740 1 -- 172.21.15.42:0/1045511650 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({fsmap=41}) -- 0x559cb286f530 con 0x559cb248f2f0 2024-08-30T09:11:43.208 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.206+0000 7f4e39ffb640 1 -- 172.21.15.42:0/1045511650 <== mon.2 v2:172.21.15.167:3300/0 4 ==== fsmap(e 41) ==== 8362+0+0 (secure 0 0 0) 0x7f4e44003b30 con 0x559cb248f2f0 2024-08-30T09:11:43.208 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.206+0000 7f4e5fc00740 10 client.27071 fetch_fsmap finished waiting for FSMap version 41 2024-08-30T09:11:43.208 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.206+0000 7f4e5fc00740 10 client.27071 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-30T09:11:43.208 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.206+0000 7f4e5fc00740 20 client.27071 populate_metadata read hostname 'smithi042' 2024-08-30T09:11:43.208 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.206+0000 7f4e5fc00740 1 --2- 172.21.15.42:0/1045511650 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x559cb244c1c0 0x559cb28f34b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:43.208 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.206+0000 7f4e5fc00740 4 client.27071 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-30T09:11:43.208 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.206+0000 7f4e5fc00740 1 -- 172.21.15.42:0/1045511650 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f4e44014110 con 0x559cb244c1c0 2024-08-30T09:11:43.211 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.209+0000 7f4e51d9b640 1 --2- 172.21.15.42:0/1045511650 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x559cb244c1c0 0x559cb28f34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:43.216 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.215+0000 7f4e39ffb640 10 client.27071 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:11:43.217 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.215+0000 7f4e51d9b640 1 --2- 172.21.15.42:0/1045511650 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x559cb244c1c0 0x559cb28f34b0 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6c9e112648791ea4 server_cookie=e9629c72ec93d3c6 in_seq=0 out_seq=0 2024-08-30T09:11:43.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.224+0000 7fa10628f740 1 Processor -- start 2024-08-30T09:11:43.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.224+0000 7fa10628f740 1 -- start start 2024-08-30T09:11:43.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.225+0000 7fa10628f740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563c7f7b01b0 0x563c7fb89f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:43.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.225+0000 7fa10628f740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563c7f7b8390 0x563c7fb81410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:43.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.225+0000 7fa10628f740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563c7f7d9070 0x563c7fb81950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:43.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.225+0000 7fa10628f740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fa0dc0123f0 con 0x563c7f7b01b0 2024-08-30T09:11:43.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.225+0000 7fa10628f740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa0dc00fc60 con 0x563c7f7d9070 2024-08-30T09:11:43.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.225+0000 7fa10628f740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fa0dc00db60 con 0x563c7f7b8390 2024-08-30T09:11:43.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.225+0000 7fa0fd2b5640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563c7f7b8390 0x563c7fb81410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:43.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.225+0000 7fa0fd2b5640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563c7f7b8390 0x563c7fb81410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:42282/0 (socket says 172.21.15.42:42282) 2024-08-30T09:11:43.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.225+0000 7fa0fd2b5640 1 -- 172.21.15.42:0/1618286821 learned_addr learned my addr 172.21.15.42:0/1618286821 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:11:43.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.225+0000 7fa0fe2b7640 1 --2- 172.21.15.42:0/1618286821 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563c7f7d9070 0x563c7fb81950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:43.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.226+0000 7fa0fd2b5640 1 -- 172.21.15.42:0/1618286821 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563c7f7d9070 msgr2=0x563c7fb81950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:11:43.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.226+0000 7fa0fd2b5640 1 --2- 172.21.15.42:0/1618286821 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563c7f7d9070 0x563c7fb81950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:43.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.226+0000 7fa0fd2b5640 1 -- 172.21.15.42:0/1618286821 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563c7f7b01b0 msgr2=0x563c7fb89f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:11:43.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.226+0000 7fa0fd2b5640 1 --2- 172.21.15.42:0/1618286821 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563c7f7b01b0 0x563c7fb89f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:43.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.226+0000 7fa0fd2b5640 1 -- 172.21.15.42:0/1618286821 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c7fb846c0 con 0x563c7f7b8390 2024-08-30T09:11:43.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.226+0000 7fa0fd2b5640 1 --2- 172.21.15.42:0/1618286821 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563c7f7b8390 0x563c7fb81410 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fa0f0004af0 tx=0x7fa0f00095c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1a9623d603bd3ac8 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:11:43.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.226+0000 7fa0e37fe640 1 -- 172.21.15.42:0/1618286821 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0f0003970 con 0x563c7f7b8390 2024-08-30T09:11:43.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.226+0000 7fa0e37fe640 1 -- 172.21.15.42:0/1618286821 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa0f0036070 con 0x563c7f7b8390 2024-08-30T09:11:43.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.226+0000 7fa0e37fe640 1 -- 172.21.15.42:0/1618286821 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0f0030920 con 0x563c7f7b8390 2024-08-30T09:11:43.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.228+0000 7fa10628f740 1 -- 172.21.15.42:0/1618286821 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563c7f7b8390 msgr2=0x563c7fb81410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:11:43.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.228+0000 7fa10628f740 1 --2- 172.21.15.42:0/1618286821 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563c7f7b8390 0x563c7fb81410 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fa0f0004af0 tx=0x7fa0f00095c0 comp rx=0 tx=0).stop 2024-08-30T09:11:43.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.228+0000 7fa10628f740 1 -- 172.21.15.42:0/1618286821 shutdown_connections 2024-08-30T09:11:43.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.228+0000 7fa10628f740 1 --2- 172.21.15.42:0/1618286821 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563c7f7b8390 0x563c7fb81410 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:43.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.228+0000 7fa10628f740 1 --2- 172.21.15.42:0/1618286821 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563c7f7d9070 0x563c7fb81950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:43.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.228+0000 7fa10628f740 1 --2- 172.21.15.42:0/1618286821 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563c7f7b01b0 0x563c7fb89f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:43.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.228+0000 7fa10628f740 1 -- 172.21.15.42:0/1618286821 >> 172.21.15.42:0/1618286821 conn(0x563c7f7cd460 msgr2=0x563c7fb89840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:11:43.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.228+0000 7fa10628f740 1 -- 172.21.15.42:0/1618286821 shutdown_connections 2024-08-30T09:11:43.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.228+0000 7fa10628f740 1 -- 172.21.15.42:0/1618286821 wait complete. 2024-08-30T09:11:43.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.229+0000 7fa10628f740 1 -- start start 2024-08-30T09:11:43.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.229+0000 7fa10628f740 1 Processor -- start 2024-08-30T09:11:43.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.230+0000 7fa10628f740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563c7f7b01b0 0x563c7fb8ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:43.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.230+0000 7fa10628f740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563c7f7b8390 0x563c7fb8e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:43.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.230+0000 7fa0fdab6640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563c7f7b01b0 0x563c7fb8ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:43.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.231+0000 7fa0fdab6640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563c7f7b01b0 0x563c7fb8ddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:45508/0 (socket says 172.21.15.42:45508) 2024-08-30T09:11:43.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.231+0000 7fa0fdab6640 1 -- 172.21.15.42:0/969806268 learned_addr learned my addr 172.21.15.42:0/969806268 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:11:43.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.231+0000 7fa0fd2b5640 1 --2- 172.21.15.42:0/969806268 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563c7f7b8390 0x563c7fb8e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:43.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.231+0000 7fa10628f740 1 --2- 172.21.15.42:0/969806268 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563c7f7d9070 0x563c7fb8e830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:43.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.231+0000 7fa10628f740 1 -- 172.21.15.42:0/969806268 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fa0dc00e120 con 0x563c7f7d9070 2024-08-30T09:11:43.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.231+0000 7fa10628f740 1 -- 172.21.15.42:0/969806268 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa0dc00f590 con 0x563c7f7b01b0 2024-08-30T09:11:43.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.231+0000 7fa10628f740 1 -- 172.21.15.42:0/969806268 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fa0dc00e7f0 con 0x563c7f7b8390 2024-08-30T09:11:43.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.231+0000 7fa10628f740 10 client.0 _ll_register_callbacks cb 0x7fa0f0009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:11:43.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.232+0000 7fa0fe2b7640 1 --2- 172.21.15.42:0/969806268 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563c7f7d9070 0x563c7fb8e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:43.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.232+0000 7fa0fe2b7640 1 -- 172.21.15.42:0/969806268 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563c7f7b8390 msgr2=0x563c7fb8e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:11:43.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.232+0000 7fa0fe2b7640 1 --2- 172.21.15.42:0/969806268 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563c7f7b8390 0x563c7fb8e2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:43.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.232+0000 7fa0fe2b7640 1 -- 172.21.15.42:0/969806268 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563c7f7b01b0 msgr2=0x563c7fb8ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:11:43.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.232+0000 7fa0fe2b7640 1 --2- 172.21.15.42:0/969806268 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563c7f7b01b0 0x563c7fb8ddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:43.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.232+0000 7fa0fe2b7640 1 -- 172.21.15.42:0/969806268 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c7fc057f0 con 0x563c7f7d9070 2024-08-30T09:11:43.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.233+0000 7fa0fe2b7640 1 --2- 172.21.15.42:0/969806268 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563c7f7d9070 0x563c7fb8e830 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fa0e80ac400 tx=0x7fa0e80048d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ea0e1f0ee206634 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:11:43.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.233+0000 7fa0e17fa640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-08-30T09:11:43.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.233+0000 7fa0e17fa640 1 -- 172.21.15.42:0/969806268 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0e8020040 con 0x563c7f7d9070 2024-08-30T09:11:43.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.233+0000 7fa0fdab6640 1 --2- 172.21.15.42:0/969806268 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563c7f7b01b0 0x563c7fb8ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-30T09:11:43.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.234+0000 7fa0e17fa640 1 -- 172.21.15.42:0/969806268 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7fa0e80cf040 con 0x563c7f7d9070 2024-08-30T09:11:43.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.234+0000 7fa0fd2b5640 1 --2- 172.21.15.42:0/969806268 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563c7f7b8390 0x563c7fb8e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-30T09:11:43.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.234+0000 7fa10628f740 1 -- 172.21.15.42:0/969806268 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa0f0036070 con 0x563c7f7d9070 2024-08-30T09:11:43.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.234+0000 7fa0e17fa640 1 -- 172.21.15.42:0/969806268 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fa0e80b0ce0 con 0x563c7f7d9070 2024-08-30T09:11:43.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.235+0000 7fa10628f740 10 client.17247 fetch_fsmap learned FSMap version 41 2024-08-30T09:11:43.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.235+0000 7fa10628f740 1 -- 172.21.15.42:0/969806268 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x563c7fc06a50 con 0x563c7f7d9070 2024-08-30T09:11:43.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.236+0000 7fa0e17fa640 1 -- 172.21.15.42:0/969806268 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa0e80ac470 con 0x563c7f7d9070 2024-08-30T09:11:43.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.236+0000 7fa0e17fa640 1 -- 172.21.15.42:0/969806268 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 41) ==== 8362+0+0 (secure 0 0 0) 0x7fa0e80cdbe0 con 0x563c7f7d9070 2024-08-30T09:11:43.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.238+0000 7fa10628f740 10 client.17247 fetch_fsmap finished waiting for FSMap version 41 2024-08-30T09:11:43.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.238+0000 7fa10628f740 10 client.17247 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-08-30T09:11:43.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.238+0000 7fa10628f740 20 client.17247 populate_metadata read hostname 'smithi042' 2024-08-30T09:11:43.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.238+0000 7fa10628f740 1 --2- 172.21.15.42:0/969806268 >> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] conn(0x563c7f76cda0 0x563c7fc08db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:43.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.238+0000 7fa10628f740 4 client.17247 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-30T09:11:43.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.238+0000 7fa10628f740 1 -- 172.21.15.42:0/969806268 --> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fa0f0036070 con 0x563c7f76cda0 2024-08-30T09:11:43.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.239+0000 7fa0fdab6640 1 --2- 172.21.15.42:0/969806268 >> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] conn(0x563c7f76cda0 0x563c7fc08db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:43.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.241+0000 7fa0fdab6640 1 --2- 172.21.15.42:0/969806268 >> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] conn(0x563c7f76cda0 0x563c7fc08db0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=158ff06663a048e6 server_cookie=db9b330de6005dcc in_seq=0 out_seq=0 2024-08-30T09:11:43.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.241+0000 7fa0e17fa640 10 client.17247 ms_handle_connect on v2:172.21.15.167:6836/3011837856 2024-08-30T09:11:43.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.242+0000 7fa0e17fa640 1 -- 172.21.15.42:0/969806268 <== mds.2 v2:172.21.15.167:6836/3011837856 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fa0f0036070 con 0x563c7f76cda0 2024-08-30T09:11:43.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.242+0000 7fa0e17fa640 10 client.17247 handle_command_reply: tid=0 2024-08-30T09:11:43.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.243+0000 7fa10628f740 1 client.17247 shutdown 2024-08-30T09:11:43.246 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.244+0000 7fa10628f740 1 -- 172.21.15.42:0/969806268 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563c7f7d9070 msgr2=0x563c7fb8e830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:11:43.246 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.244+0000 7fa10628f740 1 --2- 172.21.15.42:0/969806268 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563c7f7d9070 0x563c7fb8e830 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fa0e80ac400 tx=0x7fa0e80048d0 comp rx=0 tx=0).stop 2024-08-30T09:11:43.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.245+0000 7fa10628f740 1 -- 172.21.15.42:0/969806268 shutdown_connections 2024-08-30T09:11:43.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.245+0000 7fa10628f740 1 --2- 172.21.15.42:0/969806268 >> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] conn(0x563c7f76cda0 0x563c7fc08db0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:43.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.245+0000 7fa10628f740 1 --2- 172.21.15.42:0/969806268 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563c7f7b8390 0x563c7fb8e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:43.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.245+0000 7fa10628f740 1 --2- 172.21.15.42:0/969806268 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563c7f7d9070 0x563c7fb8e830 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:43.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.245+0000 7fa0e17fa640 0 client.17247 ms_handle_reset on v2:172.21.15.167:6836/3011837856 2024-08-30T09:11:43.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.245+0000 7fa10628f740 1 --2- 172.21.15.42:0/969806268 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563c7f7b01b0 0x563c7fb8ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:43.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.245+0000 7fa10628f740 1 -- 172.21.15.42:0/969806268 >> 172.21.15.42:0/969806268 conn(0x563c7f7cd460 msgr2=0x563c7fb87ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:11:43.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.247+0000 7fa10628f740 1 -- 172.21.15.42:0/969806268 shutdown_connections 2024-08-30T09:11:43.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.247+0000 7fa10628f740 1 -- 172.21.15.42:0/969806268 wait complete. 2024-08-30T09:11:43.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.248+0000 7fa10628f740 20 client.17247 trim_cache size 0 max 16384 2024-08-30T09:11:43.253 INFO:teuthology.orchestra.run.smithi042.stdout:[] 2024-08-30T09:11:43.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.255+0000 7fa0ffd1a640 1 -- 172.21.15.42:0/3863621708 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fa0e40b4d30 msgr2=0x7fa0e40b7240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:11:43.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.255+0000 7fa0ffd1a640 1 --2- 172.21.15.42:0/3863621708 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fa0e40b4d30 0x7fa0e40b7240 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fa0f80751f0 tx=0x7fa0f400b350 comp rx=0 tx=0).stop 2024-08-30T09:11:43.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.255+0000 7fa0ffd1a640 1 -- 172.21.15.42:0/3863621708 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa0f81139a0 msgr2=0x7fa0f8113df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:11:43.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.255+0000 7fa0ffd1a640 1 --2- 172.21.15.42:0/3863621708 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa0f81139a0 0x7fa0f8113df0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fa0e800d750 tx=0x7fa0e800dc10 comp rx=0 tx=0).stop 2024-08-30T09:11:43.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.255+0000 7fa0ffd1a640 1 -- 172.21.15.42:0/3863621708 shutdown_connections 2024-08-30T09:11:43.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.255+0000 7fa0ffd1a640 1 --2- 172.21.15.42:0/3863621708 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fa0e40b4d30 0x7fa0e40b7240 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:43.258 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.255+0000 7fa0ffd1a640 1 --2- 172.21.15.42:0/3863621708 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa0f811c9d0 0x7fa0f8113460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:43.258 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.255+0000 7fa0ffd1a640 1 --2- 172.21.15.42:0/3863621708 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa0f81139a0 0x7fa0f8113df0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:43.258 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.255+0000 7fa0ffd1a640 1 --2- 172.21.15.42:0/3863621708 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa0f810a780 0x7fa0f811a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:43.258 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.255+0000 7fa0ffd1a640 1 -- 172.21.15.42:0/3863621708 >> 172.21.15.42:0/3863621708 conn(0x7fa0f8070c60 msgr2=0x7fa0f811e1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:11:43.258 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.255+0000 7fa0ffd1a640 1 -- 172.21.15.42:0/3863621708 shutdown_connections 2024-08-30T09:11:43.258 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:43.255+0000 7fa0ffd1a640 1 -- 172.21.15.42:0/3863621708 wait complete. 2024-08-30T09:11:43.268 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.265+0000 7f4e39ffb640 1 -- 172.21.15.42:0/1045511650 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f4e44014110 con 0x559cb244c1c0 2024-08-30T09:11:43.268 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.265+0000 7f4e39ffb640 10 client.27071 handle_command_reply: tid=0 2024-08-30T09:11:43.268 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.265+0000 7f4e5fc00740 1 client.27071 shutdown 2024-08-30T09:11:43.272 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.270+0000 7f4e5fc00740 1 -- 172.21.15.42:0/1045511650 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559cb248f2f0 msgr2=0x559cb286a9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:11:43.272 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.270+0000 7f4e5fc00740 1 --2- 172.21.15.42:0/1045511650 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559cb248f2f0 0x559cb286a9a0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f4e44005780 tx=0x7f4e440035f0 comp rx=0 tx=0).stop 2024-08-30T09:11:43.275 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.273+0000 7f4e5fc00740 1 -- 172.21.15.42:0/1045511650 shutdown_connections 2024-08-30T09:11:43.275 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.273+0000 7f4e5fc00740 1 --2- 172.21.15.42:0/1045511650 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x559cb244c1c0 0x559cb28f34b0 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:43.275 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.273+0000 7f4e5fc00740 1 --2- 172.21.15.42:0/1045511650 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559cb24b81b0 0x559cb286bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:43.275 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.273+0000 7f4e5fc00740 1 --2- 172.21.15.42:0/1045511650 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559cb24974d0 0x559cb286b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:43.276 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.273+0000 7f4e5fc00740 1 --2- 172.21.15.42:0/1045511650 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559cb248f2f0 0x559cb286a9a0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:43.276 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.273+0000 7f4e5fc00740 1 -- 172.21.15.42:0/1045511650 >> 172.21.15.42:0/1045511650 conn(0x559cb24ac300 msgr2=0x559cb2865ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:11:43.276 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.273+0000 7f4e39ffb640 0 client.27071 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:11:43.278 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.277+0000 7f4e5fc00740 1 -- 172.21.15.42:0/1045511650 shutdown_connections 2024-08-30T09:11:43.281 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.278+0000 7f4e5fc00740 1 -- 172.21.15.42:0/1045511650 wait complete. 2024-08-30T09:11:43.281 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.279+0000 7f4e5fc00740 20 client.27071 trim_cache size 0 max 16384 2024-08-30T09:11:43.281 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-08-30T09:11:43.281 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2024-08-30T09:11:43.281 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-08-30T09:11:43.281 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-08-30T09:11:43.283 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.281+0000 7f4e58c94640 1 -- 172.21.15.42:0/1040963234 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f4e3c0b4bd0 msgr2=0x7f4e3c0b70e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:11:43.283 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.281+0000 7f4e58c94640 1 --2- 172.21.15.42:0/1040963234 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f4e3c0b4bd0 0x7f4e3c0b70e0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f4e4c02be10 tx=0x7f4e4c0061f0 comp rx=0 tx=0).stop 2024-08-30T09:11:43.283 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.281+0000 7f4e58c94640 1 -- 172.21.15.42:0/1040963234 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4e54102b70 msgr2=0x7f4e541a0980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:11:43.284 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.281+0000 7f4e58c94640 1 --2- 172.21.15.42:0/1040963234 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4e54102b70 0x7f4e541a0980 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f4e4800c920 tx=0x7f4e4800cde0 comp rx=0 tx=0).stop 2024-08-30T09:11:43.284 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.281+0000 7f4e58c94640 1 -- 172.21.15.42:0/1040963234 shutdown_connections 2024-08-30T09:11:43.284 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.281+0000 7f4e58c94640 1 --2- 172.21.15.42:0/1040963234 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f4e3c0b4bd0 0x7f4e3c0b70e0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:43.284 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.281+0000 7f4e58c94640 1 --2- 172.21.15.42:0/1040963234 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4e54103510 0x7f4e541a1530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:43.284 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.281+0000 7f4e58c94640 1 --2- 172.21.15.42:0/1040963234 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4e54102b70 0x7f4e541a0980 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:43.284 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.281+0000 7f4e58c94640 1 --2- 172.21.15.42:0/1040963234 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4e54101af0 0x7f4e541a0310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:43.284 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.281+0000 7f4e58c94640 1 -- 172.21.15.42:0/1040963234 >> 172.21.15.42:0/1040963234 conn(0x7f4e540fd430 msgr2=0x7f4e54194570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:11:43.284 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.282+0000 7f4e58c94640 1 -- 172.21.15.42:0/1040963234 shutdown_connections 2024-08-30T09:11:43.288 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:43.286+0000 7f4e58c94640 1 -- 172.21.15.42:0/1040963234 wait complete. 2024-08-30T09:11:44.188 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-08-30T09:11:44.656 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T09:11:44.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:11:44 smithi042 ceph-mon[18652]: pgmap v702: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 6.4 MiB/s wr, 539 op/s 2024-08-30T09:11:44.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:11:44 smithi190 ceph-mon[22001]: pgmap v702: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 6.4 MiB/s wr, 539 op/s 2024-08-30T09:11:45.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:11:44 smithi167 ceph-mon[22063]: pgmap v702: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 6.4 MiB/s wr, 539 op/s 2024-08-30T09:11:45.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.275+0000 7f348908f640 1 -- 172.21.15.42:0/1120052149 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f347c040c10 con 0x7f3484102380 2024-08-30T09:11:45.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.276+0000 7f348b093640 1 -- 172.21.15.42:0/1120052149 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3484102380 msgr2=0x7f3484102790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:11:45.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.276+0000 7f348b093640 1 --2- 172.21.15.42:0/1120052149 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3484102380 0x7f3484102790 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f347c01ace0 tx=0x7f347c03b970 comp rx=0 tx=0).stop 2024-08-30T09:11:45.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.276+0000 7f348b093640 1 -- 172.21.15.42:0/1120052149 shutdown_connections 2024-08-30T09:11:45.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.276+0000 7f348b093640 1 --2- 172.21.15.42:0/1120052149 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f34841077c0 0x7f3484107ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:45.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.276+0000 7f348b093640 1 --2- 172.21.15.42:0/1120052149 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3484102cd0 0x7f3484107200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:45.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.276+0000 7f348b093640 1 --2- 172.21.15.42:0/1120052149 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3484102380 0x7f3484102790 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:45.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.276+0000 7f348b093640 1 -- 172.21.15.42:0/1120052149 >> 172.21.15.42:0/1120052149 conn(0x7f34840fe050 msgr2=0x7f34841004a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:11:45.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.281+0000 7f348b093640 1 -- 172.21.15.42:0/1120052149 shutdown_connections 2024-08-30T09:11:45.290 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.281+0000 7f348b093640 1 -- 172.21.15.42:0/1120052149 wait complete. 2024-08-30T09:11:45.290 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.282+0000 7f348b093640 1 Processor -- start 2024-08-30T09:11:45.290 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.287+0000 7f348b093640 1 -- start start 2024-08-30T09:11:45.290 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.287+0000 7f348b093640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3484102380 0x7f3484074190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:45.290 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.287+0000 7f348b093640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3484102cd0 0x7f3484074710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:45.290 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.287+0000 7f348b093640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f34841077c0 0x7f348410ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:45.290 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.287+0000 7f348b093640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f348410b140 con 0x7f34841077c0 2024-08-30T09:11:45.290 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.287+0000 7f348b093640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f348410afc0 con 0x7f3484102cd0 2024-08-30T09:11:45.290 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.287+0000 7f348b093640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f348410b2c0 con 0x7f3484102380 2024-08-30T09:11:45.290 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.288+0000 7f348a892640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f34841077c0 0x7f348410ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:45.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.288+0000 7f348a091640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3484102380 0x7f3484074190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:45.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.288+0000 7f348a892640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f34841077c0 0x7f348410ee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:48488/0 (socket says 172.21.15.42:48488) 2024-08-30T09:11:45.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.288+0000 7f348a892640 1 -- 172.21.15.42:0/209412248 learned_addr learned my addr 172.21.15.42:0/209412248 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:11:45.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.288+0000 7f348a892640 1 -- 172.21.15.42:0/209412248 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3484102380 msgr2=0x7f3484074190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:11:45.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.288+0000 7f348a892640 1 --2- 172.21.15.42:0/209412248 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3484102380 0x7f3484074190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:45.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.288+0000 7f348a892640 1 -- 172.21.15.42:0/209412248 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3484102cd0 msgr2=0x7f3484074710 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-08-30T09:11:45.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.288+0000 7f348a892640 1 --2- 172.21.15.42:0/209412248 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3484102cd0 0x7f3484074710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:45.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.289+0000 7f348a892640 1 -- 172.21.15.42:0/209412248 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f348410f4e0 con 0x7f34841077c0 2024-08-30T09:11:45.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.289+0000 7f348a892640 1 --2- 172.21.15.42:0/209412248 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f34841077c0 0x7f348410ee40 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f3480009fd0 tx=0x7f348000b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:11:45.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.289+0000 7f347b7fe640 1 -- 172.21.15.42:0/209412248 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3480011830 con 0x7f34841077c0 2024-08-30T09:11:45.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.289+0000 7f348b093640 1 -- 172.21.15.42:0/209412248 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f348410f7d0 con 0x7f34841077c0 2024-08-30T09:11:45.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.289+0000 7f348b093640 1 -- 172.21.15.42:0/209412248 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3484115100 con 0x7f34841077c0 2024-08-30T09:11:45.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.290+0000 7f347b7fe640 1 -- 172.21.15.42:0/209412248 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3480016070 con 0x7f34841077c0 2024-08-30T09:11:45.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.290+0000 7f347b7fe640 1 -- 172.21.15.42:0/209412248 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3480010e50 con 0x7f34841077c0 2024-08-30T09:11:45.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.293+0000 7f349286a740 1 Processor -- start 2024-08-30T09:11:45.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.295+0000 7f347b7fe640 1 -- 172.21.15.42:0/209412248 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7f348001b4a0 con 0x7f34841077c0 2024-08-30T09:11:45.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.296+0000 7f347b7fe640 1 --2- 172.21.15.42:0/209412248 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f34700b4d50 0x7f34700b7260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:45.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.297+0000 7f347b7fe640 1 -- 172.21.15.42:0/209412248 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f34800a86a0 con 0x7f34841077c0 2024-08-30T09:11:45.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.297+0000 7f349286a740 1 -- start start 2024-08-30T09:11:45.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.298+0000 7f349286a740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f001a9d070 0x55f001e424b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:45.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.298+0000 7f349286a740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f001a741b0 0x55f001e429f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:45.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.298+0000 7f349286a740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f001a7c390 0x55f001e42f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:45.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.298+0000 7f349286a740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55f001db9680 con 0x55f001a741b0 2024-08-30T09:11:45.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.298+0000 7f349286a740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55f001da8c10 con 0x55f001a9d070 2024-08-30T09:11:45.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.298+0000 7f349286a740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55f001dacbc0 con 0x55f001a7c390 2024-08-30T09:11:45.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.298+0000 7f348a892640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f001a7c390 0x55f001e42f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:45.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.298+0000 7f348a892640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f001a7c390 0x55f001e42f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:42320/0 (socket says 172.21.15.42:42320) 2024-08-30T09:11:45.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.298+0000 7f348a892640 1 -- 172.21.15.42:0/3082481566 learned_addr learned my addr 172.21.15.42:0/3082481566 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:11:45.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.298+0000 7f348a091640 1 --2- 172.21.15.42:0/3082481566 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f001a9d070 0x55f001e424b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:45.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.298+0000 7f348a892640 1 -- 172.21.15.42:0/3082481566 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f001a9d070 msgr2=0x55f001e424b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:11:45.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.298+0000 7f348a892640 1 --2- 172.21.15.42:0/3082481566 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f001a9d070 0x55f001e424b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:45.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.298+0000 7f348a892640 1 -- 172.21.15.42:0/3082481566 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f001a741b0 msgr2=0x55f001e429f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-30T09:11:45.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.298+0000 7f348a892640 1 --2- 172.21.15.42:0/3082481566 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f001a741b0 0x55f001e429f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:45.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.298+0000 7f348a892640 1 -- 172.21.15.42:0/3082481566 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f001b08650 con 0x55f001a7c390 2024-08-30T09:11:45.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.299+0000 7f348a892640 1 --2- 172.21.15.42:0/3082481566 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f001a7c390 0x55f001e42f30 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f34800a9320 tx=0x7f34800b3c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e992fa313a8726e3 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:11:45.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.299+0000 7f3478d9a640 1 -- 172.21.15.42:0/3082481566 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34800a88a0 con 0x55f001a7c390 2024-08-30T09:11:45.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.299+0000 7f3478d9a640 1 -- 172.21.15.42:0/3082481566 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f34800ac7b0 con 0x55f001a7c390 2024-08-30T09:11:45.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.300+0000 7f3478d9a640 1 -- 172.21.15.42:0/3082481566 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34800a8a40 con 0x55f001a7c390 2024-08-30T09:11:45.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.300+0000 7f349286a740 1 -- 172.21.15.42:0/3082481566 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f001a7c390 msgr2=0x55f001e42f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:11:45.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.300+0000 7f349286a740 1 --2- 172.21.15.42:0/3082481566 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f001a7c390 0x55f001e42f30 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f34800a9320 tx=0x7f34800b3c30 comp rx=0 tx=0).stop 2024-08-30T09:11:45.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.300+0000 7f349286a740 1 -- 172.21.15.42:0/3082481566 shutdown_connections 2024-08-30T09:11:45.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.300+0000 7f349286a740 1 --2- 172.21.15.42:0/3082481566 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f001a7c390 0x55f001e42f30 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:45.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.300+0000 7f349286a740 1 --2- 172.21.15.42:0/3082481566 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f001a741b0 0x55f001e429f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:45.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.300+0000 7f349286a740 1 --2- 172.21.15.42:0/3082481566 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f001a9d070 0x55f001e424b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:45.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.300+0000 7f349286a740 1 -- 172.21.15.42:0/3082481566 >> 172.21.15.42:0/3082481566 conn(0x55f001a91460 msgr2=0x55f001e403a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:11:45.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.302+0000 7f3489890640 1 --2- 172.21.15.42:0/209412248 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f34700b4d50 0x7f34700b7260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:45.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.302+0000 7f349286a740 1 -- 172.21.15.42:0/3082481566 shutdown_connections 2024-08-30T09:11:45.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.302+0000 7f349286a740 1 -- 172.21.15.42:0/3082481566 wait complete. 2024-08-30T09:11:45.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.303+0000 7f349286a740 1 -- start start 2024-08-30T09:11:45.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.303+0000 7f349286a740 1 Processor -- start 2024-08-30T09:11:45.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.303+0000 7f349286a740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f001a741b0 0x55f001e4dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:45.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.303+0000 7f349286a740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f001a7c390 0x55f001e4e120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:45.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.303+0000 7f349286a740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f001a9d070 0x55f001e59f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:45.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.303+0000 7f349286a740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55f001c0b700 con 0x55f001a7c390 2024-08-30T09:11:45.308 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.303+0000 7f349286a740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55f001d2f750 con 0x55f001a9d070 2024-08-30T09:11:45.308 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.303+0000 7f349286a740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55f001e50bf0 con 0x55f001a741b0 2024-08-30T09:11:45.308 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.303+0000 7f349286a740 10 client.0 _ll_register_callbacks cb 0x55f001d5ee10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:11:45.308 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.304+0000 7f348a892640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f001a9d070 0x55f001e59f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:45.308 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.304+0000 7f348a892640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f001a9d070 0x55f001e59f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:45550/0 (socket says 172.21.15.42:45550) 2024-08-30T09:11:45.308 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.304+0000 7f348a892640 1 -- 172.21.15.42:0/3801940956 learned_addr learned my addr 172.21.15.42:0/3801940956 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:11:45.308 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.304+0000 7f348a091640 1 --2- 172.21.15.42:0/3801940956 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f001a741b0 0x55f001e4dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:45.308 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.304+0000 7f348a892640 1 -- 172.21.15.42:0/3801940956 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f001a741b0 msgr2=0x55f001e4dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:11:45.308 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.304+0000 7f348a892640 1 --2- 172.21.15.42:0/3801940956 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f001a741b0 0x55f001e4dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:45.308 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.304+0000 7f348a892640 1 -- 172.21.15.42:0/3801940956 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f001a7c390 msgr2=0x55f001e4e120 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-30T09:11:45.308 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.304+0000 7f348a892640 1 --2- 172.21.15.42:0/3801940956 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f001a7c390 0x55f001e4e120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:45.308 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.304+0000 7f348a892640 1 -- 172.21.15.42:0/3801940956 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f001d68900 con 0x55f001a9d070 2024-08-30T09:11:45.308 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.305+0000 7f348a892640 1 --2- 172.21.15.42:0/3801940956 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f001a9d070 0x55f001e59f90 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f34800a9120 tx=0x7f34800a7cc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a518c6df66f9081 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:11:45.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.305+0000 7f346dffb640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-30T09:11:45.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.307+0000 7f346dffb640 1 -- 172.21.15.42:0/3801940956 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34800a8c10 con 0x55f001a9d070 2024-08-30T09:11:45.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.307+0000 7f349286a740 1 -- 172.21.15.42:0/3801940956 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f34800ac7b0 con 0x55f001a9d070 2024-08-30T09:11:45.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.307+0000 7f346dffb640 1 -- 172.21.15.42:0/3801940956 <== mon.2 v2:172.21.15.167:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f34800ab310 con 0x55f001a9d070 2024-08-30T09:11:45.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.308+0000 7f346dffb640 1 -- 172.21.15.42:0/3801940956 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f34800a8db0 con 0x55f001a9d070 2024-08-30T09:11:45.313 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.312+0000 7f349286a740 10 client.27080 fetch_fsmap learned FSMap version 41 2024-08-30T09:11:45.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.312+0000 7f349286a740 1 -- 172.21.15.42:0/3801940956 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55f001e5c970 con 0x55f001a9d070 2024-08-30T09:11:45.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.313+0000 7f346dffb640 1 -- 172.21.15.42:0/3801940956 <== mon.2 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f34800ac7b0 con 0x55f001a9d070 2024-08-30T09:11:45.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.314+0000 7f346dffb640 1 -- 172.21.15.42:0/3801940956 <== mon.2 v2:172.21.15.167:3300/0 5 ==== fsmap(e 41) ==== 8362+0+0 (secure 0 0 0) 0x7f348000eb70 con 0x55f001a9d070 2024-08-30T09:11:45.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.314+0000 7f349286a740 10 client.27080 fetch_fsmap finished waiting for FSMap version 41 2024-08-30T09:11:45.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.314+0000 7f349286a740 10 client.27080 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-08-30T09:11:45.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.315+0000 7f349286a740 20 client.27080 populate_metadata read hostname 'smithi042' 2024-08-30T09:11:45.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.315+0000 7f349286a740 1 --2- 172.21.15.42:0/3801940956 >> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] conn(0x55f001a5b9f0 0x55f001e5fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:45.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.315+0000 7f349286a740 4 client.27080 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-30T09:11:45.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.315+0000 7f349286a740 1 -- 172.21.15.42:0/3801940956 --> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f001dbb5d0 con 0x55f001a5b9f0 2024-08-30T09:11:45.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.316+0000 7f348a091640 1 --2- 172.21.15.42:0/3801940956 >> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] conn(0x55f001a5b9f0 0x55f001e5fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:45.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.319+0000 7f3489890640 1 --2- 172.21.15.42:0/209412248 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f34700b4d50 0x7f34700b7260 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x55f001e41130 tx=0x7f3474006b70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:11:45.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.324+0000 7f348a091640 1 --2- 172.21.15.42:0/3801940956 >> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] conn(0x55f001a5b9f0 0x55f001e5fcd0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b245f0af52642549 server_cookie=190d73922007bb33 in_seq=0 out_seq=0 2024-08-30T09:11:45.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.324+0000 7f346dffb640 10 client.27080 ms_handle_connect on v2:172.21.15.190:6838/719981110 2024-08-30T09:11:45.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.326+0000 7f346dffb640 1 -- 172.21.15.42:0/3801940956 <== mds.1 v2:172.21.15.190:6838/719981110 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f001dbb5d0 con 0x55f001a5b9f0 2024-08-30T09:11:45.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.326+0000 7f346dffb640 10 client.27080 handle_command_reply: tid=0 2024-08-30T09:11:45.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.326+0000 7f349286a740 1 client.27080 shutdown 2024-08-30T09:11:45.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.326+0000 7f349286a740 1 -- 172.21.15.42:0/3801940956 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f001a9d070 msgr2=0x55f001e59f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:11:45.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.326+0000 7f349286a740 1 --2- 172.21.15.42:0/3801940956 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f001a9d070 0x55f001e59f90 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f34800a9120 tx=0x7f34800a7cc0 comp rx=0 tx=0).stop 2024-08-30T09:11:45.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.328+0000 7f349286a740 1 -- 172.21.15.42:0/3801940956 shutdown_connections 2024-08-30T09:11:45.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.328+0000 7f349286a740 1 --2- 172.21.15.42:0/3801940956 >> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] conn(0x55f001a5b9f0 0x55f001e5fcd0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:45.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.328+0000 7f349286a740 1 --2- 172.21.15.42:0/3801940956 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f001a9d070 0x55f001e59f90 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:45.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.328+0000 7f349286a740 1 --2- 172.21.15.42:0/3801940956 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f001a7c390 0x55f001e4e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:45.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.328+0000 7f349286a740 1 --2- 172.21.15.42:0/3801940956 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f001a741b0 0x55f001e4dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:45.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.328+0000 7f349286a740 1 -- 172.21.15.42:0/3801940956 >> 172.21.15.42:0/3801940956 conn(0x55f001a91460 msgr2=0x55f001e45640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:11:45.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.328+0000 7f346dffb640 0 client.27080 ms_handle_reset on v2:172.21.15.190:6838/719981110 2024-08-30T09:11:45.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.328+0000 7f349286a740 1 -- 172.21.15.42:0/3801940956 shutdown_connections 2024-08-30T09:11:45.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.328+0000 7f349286a740 1 -- 172.21.15.42:0/3801940956 wait complete. 2024-08-30T09:11:45.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.328+0000 7f349286a740 20 client.27080 trim_cache size 0 max 16384 2024-08-30T09:11:45.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.359+0000 7f349286a740 1 Processor -- start 2024-08-30T09:11:45.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.360+0000 7f349286a740 1 -- start start 2024-08-30T09:11:45.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.360+0000 7f349286a740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f001a741b0 0x55f001e403d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:45.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.361+0000 7f349286a740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f001a7c390 0x55f001e59f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:45.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.361+0000 7f348a091640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f001a741b0 0x55f001e403d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:45.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.361+0000 7f348a091640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f001a741b0 0x55f001e403d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:42342/0 (socket says 172.21.15.42:42342) 2024-08-30T09:11:45.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.361+0000 7f348a091640 1 -- 172.21.15.42:0/4155783506 learned_addr learned my addr 172.21.15.42:0/4155783506 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:11:45.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.361+0000 7f349286a740 1 --2- 172.21.15.42:0/4155783506 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f001a9d070 0x55f001e4add0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:45.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.361+0000 7f349286a740 1 -- 172.21.15.42:0/4155783506 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f34680123c0 con 0x55f001a7c390 2024-08-30T09:11:45.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.362+0000 7f348a892640 1 --2- 172.21.15.42:0/4155783506 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f001a9d070 0x55f001e4add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:45.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.362+0000 7f349286a740 1 -- 172.21.15.42:0/4155783506 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f346800fc30 con 0x55f001a9d070 2024-08-30T09:11:45.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.362+0000 7f349286a740 1 -- 172.21.15.42:0/4155783506 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f346800db30 con 0x55f001a741b0 2024-08-30T09:11:45.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.363+0000 7f3489890640 1 --2- 172.21.15.42:0/4155783506 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f001a7c390 0x55f001e59f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:45.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.363+0000 7f348a892640 1 -- 172.21.15.42:0/4155783506 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f001a741b0 msgr2=0x55f001e403d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:11:45.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.363+0000 7f348a892640 1 --2- 172.21.15.42:0/4155783506 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f001a741b0 0x55f001e403d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:45.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.363+0000 7f348a892640 1 -- 172.21.15.42:0/4155783506 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f001a7c390 msgr2=0x55f001e59f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:11:45.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.363+0000 7f348a892640 1 --2- 172.21.15.42:0/4155783506 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f001a7c390 0x55f001e59f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:45.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.363+0000 7f348a892640 1 -- 172.21.15.42:0/4155783506 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f001e4eea0 con 0x55f001a9d070 2024-08-30T09:11:45.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.363+0000 7f3489890640 1 --2- 172.21.15.42:0/4155783506 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f001a7c390 0x55f001e59f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-30T09:11:45.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.364+0000 7f348a892640 1 --2- 172.21.15.42:0/4155783506 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f001a9d070 0x55f001e4add0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f348000ef70 tx=0x7f34800046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=20dec0ae7cfda2f5 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:11:45.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.364+0000 7f346ffff640 1 -- 172.21.15.42:0/4155783506 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34800a8f80 con 0x55f001a9d070 2024-08-30T09:11:45.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.364+0000 7f346ffff640 1 -- 172.21.15.42:0/4155783506 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f34800a7fa0 con 0x55f001a9d070 2024-08-30T09:11:45.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.366+0000 7f349286a740 1 -- 172.21.15.42:0/4155783506 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f001a9d070 msgr2=0x55f001e4add0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:11:45.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.366+0000 7f349286a740 1 --2- 172.21.15.42:0/4155783506 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f001a9d070 0x55f001e4add0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f348000ef70 tx=0x7f34800046c0 comp rx=0 tx=0).stop 2024-08-30T09:11:45.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.366+0000 7f349286a740 1 -- 172.21.15.42:0/4155783506 shutdown_connections 2024-08-30T09:11:45.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.366+0000 7f349286a740 1 --2- 172.21.15.42:0/4155783506 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f001a9d070 0x55f001e4add0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:45.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.366+0000 7f349286a740 1 --2- 172.21.15.42:0/4155783506 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f001a7c390 0x55f001e59f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:45.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.366+0000 7f349286a740 1 --2- 172.21.15.42:0/4155783506 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f001a741b0 0x55f001e403d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:45.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.366+0000 7f349286a740 1 -- 172.21.15.42:0/4155783506 >> 172.21.15.42:0/4155783506 conn(0x55f001a91460 msgr2=0x55f001e3fcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:11:45.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.367+0000 7f349286a740 1 -- 172.21.15.42:0/4155783506 shutdown_connections 2024-08-30T09:11:45.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.367+0000 7f349286a740 1 -- 172.21.15.42:0/4155783506 wait complete. 2024-08-30T09:11:45.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.367+0000 7f349286a740 1 -- start start 2024-08-30T09:11:45.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.367+0000 7f349286a740 1 Processor -- start 2024-08-30T09:11:45.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.368+0000 7f349286a740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f001a741b0 0x55f001e5f960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:45.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.368+0000 7f349286a740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f001a7c390 0x55f001e5ff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:45.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.368+0000 7f349286a740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f001a9d070 0x55f001e63e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:45.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.368+0000 7f349286a740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f346800e0f0 con 0x55f001a9d070 2024-08-30T09:11:45.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.368+0000 7f349286a740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f346800f560 con 0x55f001a741b0 2024-08-30T09:11:45.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.368+0000 7f349286a740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f346800e7c0 con 0x55f001a7c390 2024-08-30T09:11:45.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.368+0000 7f349286a740 10 client.0 _ll_register_callbacks cb 0x7f34800710f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:11:45.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.368+0000 7f348a091640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f001a741b0 0x55f001e5f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:45.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.369+0000 7f3489890640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f001a7c390 0x55f001e5ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:45.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.369+0000 7f3489890640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f001a7c390 0x55f001e5ff40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:42348/0 (socket says 172.21.15.42:42348) 2024-08-30T09:11:45.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.369+0000 7f3489890640 1 -- 172.21.15.42:0/3628935925 learned_addr learned my addr 172.21.15.42:0/3628935925 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:11:45.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.369+0000 7f348a091640 1 -- 172.21.15.42:0/3628935925 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f001a7c390 msgr2=0x55f001e5ff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:11:45.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.369+0000 7f348a091640 1 --2- 172.21.15.42:0/3628935925 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f001a7c390 0x55f001e5ff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:45.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.369+0000 7f348a091640 1 -- 172.21.15.42:0/3628935925 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f001a9d070 msgr2=0x55f001e63e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:11:45.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.369+0000 7f348a091640 1 --2- 172.21.15.42:0/3628935925 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f001a9d070 0x55f001e63e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:45.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.369+0000 7f348a091640 1 -- 172.21.15.42:0/3628935925 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f001e644a0 con 0x55f001a741b0 2024-08-30T09:11:45.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.372+0000 7f348a091640 1 --2- 172.21.15.42:0/3628935925 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f001a741b0 0x55f001e5f960 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f347c000f80 tx=0x7f347c003b00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fffb23c601eac397 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:11:45.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.372+0000 7f346dffb640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-30T09:11:45.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.372+0000 7f346dffb640 1 -- 172.21.15.42:0/3628935925 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f347c062060 con 0x55f001a741b0 2024-08-30T09:11:45.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.372+0000 7f346dffb640 1 -- 172.21.15.42:0/3628935925 <== mon.2 v2:172.21.15.167:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f347c019040 con 0x55f001a741b0 2024-08-30T09:11:45.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.372+0000 7f349286a740 1 -- 172.21.15.42:0/3628935925 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55f001db7070 con 0x55f001a741b0 2024-08-30T09:11:45.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.374+0000 7f346dffb640 1 -- 172.21.15.42:0/3628935925 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f347c045040 con 0x55f001a741b0 2024-08-30T09:11:45.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.376+0000 7f349286a740 10 client.27086 fetch_fsmap learned FSMap version 41 2024-08-30T09:11:45.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.376+0000 7f349286a740 1 -- 172.21.15.42:0/3628935925 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55f001ec9e90 con 0x55f001a741b0 2024-08-30T09:11:45.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.379+0000 7f346dffb640 1 -- 172.21.15.42:0/3628935925 <== mon.2 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55f001db7070 con 0x55f001a741b0 2024-08-30T09:11:45.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.379+0000 7f346dffb640 1 -- 172.21.15.42:0/3628935925 <== mon.2 v2:172.21.15.167:3300/0 5 ==== fsmap(e 41) ==== 8362+0+0 (secure 0 0 0) 0x7f347c041a80 con 0x55f001a741b0 2024-08-30T09:11:45.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.379+0000 7f349286a740 10 client.27086 fetch_fsmap finished waiting for FSMap version 41 2024-08-30T09:11:45.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.379+0000 7f349286a740 10 client.27086 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-08-30T09:11:45.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.379+0000 7f349286a740 20 client.27086 populate_metadata read hostname 'smithi042' 2024-08-30T09:11:45.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.379+0000 7f349286a740 1 --2- 172.21.15.42:0/3628935925 >> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] conn(0x55f001a30da0 0x55f001ecc5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:45.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.379+0000 7f349286a740 4 client.27086 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-30T09:11:45.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.379+0000 7f349286a740 1 -- 172.21.15.42:0/3628935925 --> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f001db47b0 con 0x55f001a30da0 2024-08-30T09:11:45.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.379+0000 7f3489890640 1 --2- 172.21.15.42:0/3628935925 >> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] conn(0x55f001a30da0 0x55f001ecc5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:45.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.380+0000 7f346dffb640 10 client.27086 ms_handle_connect on v2:172.21.15.190:6838/719981110 2024-08-30T09:11:45.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.381+0000 7f3489890640 1 --2- 172.21.15.42:0/3628935925 >> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] conn(0x55f001a30da0 0x55f001ecc5e0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ee198733639435c1 server_cookie=d710c5523fd5ae71 in_seq=0 out_seq=0 2024-08-30T09:11:45.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.385+0000 7f346dffb640 1 -- 172.21.15.42:0/3628935925 <== mds.1 v2:172.21.15.190:6838/719981110 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f001db47b0 con 0x55f001a30da0 2024-08-30T09:11:45.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.385+0000 7f346dffb640 10 client.27086 handle_command_reply: tid=0 2024-08-30T09:11:45.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.385+0000 7f349286a740 1 client.27086 shutdown 2024-08-30T09:11:45.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.385+0000 7f349286a740 1 -- 172.21.15.42:0/3628935925 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f001a741b0 msgr2=0x55f001e5f960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:11:45.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.385+0000 7f349286a740 1 --2- 172.21.15.42:0/3628935925 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f001a741b0 0x55f001e5f960 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f347c000f80 tx=0x7f347c003b00 comp rx=0 tx=0).stop 2024-08-30T09:11:45.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.385+0000 7f349286a740 1 -- 172.21.15.42:0/3628935925 shutdown_connections 2024-08-30T09:11:45.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.385+0000 7f349286a740 1 --2- 172.21.15.42:0/3628935925 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f001a7c390 0x55f001e5ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:45.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.385+0000 7f349286a740 1 --2- 172.21.15.42:0/3628935925 >> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] conn(0x55f001a30da0 0x55f001ecc5e0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:45.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.385+0000 7f349286a740 1 --2- 172.21.15.42:0/3628935925 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f001a9d070 0x55f001e63e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:45.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.385+0000 7f349286a740 1 --2- 172.21.15.42:0/3628935925 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f001a741b0 0x55f001e5f960 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:45.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.385+0000 7f346dffb640 0 client.27086 ms_handle_reset on v2:172.21.15.190:6838/719981110 2024-08-30T09:11:45.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.385+0000 7f349286a740 1 -- 172.21.15.42:0/3628935925 >> 172.21.15.42:0/3628935925 conn(0x55f001a91460 msgr2=0x55f001e3e810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:11:45.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.386+0000 7f349286a740 1 -- 172.21.15.42:0/3628935925 shutdown_connections 2024-08-30T09:11:45.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.386+0000 7f349286a740 1 -- 172.21.15.42:0/3628935925 wait complete. 2024-08-30T09:11:45.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.386+0000 7f349286a740 20 client.27086 trim_cache size 0 max 16384 2024-08-30T09:11:45.390 INFO:teuthology.orchestra.run.smithi042.stdout:[] 2024-08-30T09:11:45.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.387+0000 7f348b093640 1 -- 172.21.15.42:0/209412248 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f34700b4d50 msgr2=0x7f34700b7260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:11:45.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.387+0000 7f348b093640 1 --2- 172.21.15.42:0/209412248 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f34700b4d50 0x7f34700b7260 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x55f001e41130 tx=0x7f3474006b70 comp rx=0 tx=0).stop 2024-08-30T09:11:45.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.387+0000 7f348b093640 1 -- 172.21.15.42:0/209412248 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f34841077c0 msgr2=0x7f348410ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:11:45.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.387+0000 7f348b093640 1 --2- 172.21.15.42:0/209412248 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f34841077c0 0x7f348410ee40 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f3480009fd0 tx=0x7f348000b8e0 comp rx=0 tx=0).stop 2024-08-30T09:11:45.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.387+0000 7f348b093640 1 -- 172.21.15.42:0/209412248 shutdown_connections 2024-08-30T09:11:45.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.387+0000 7f348b093640 1 --2- 172.21.15.42:0/209412248 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f34700b4d50 0x7f34700b7260 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:45.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.387+0000 7f348b093640 1 --2- 172.21.15.42:0/209412248 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f34841077c0 0x7f348410ee40 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:45.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.387+0000 7f348b093640 1 --2- 172.21.15.42:0/209412248 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3484102cd0 0x7f3484074710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:45.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.387+0000 7f348b093640 1 --2- 172.21.15.42:0/209412248 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3484102380 0x7f3484074190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:45.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.387+0000 7f348b093640 1 -- 172.21.15.42:0/209412248 >> 172.21.15.42:0/209412248 conn(0x7f34840fe050 msgr2=0x7f34841051d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:11:45.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.387+0000 7f348b093640 1 -- 172.21.15.42:0/209412248 shutdown_connections 2024-08-30T09:11:45.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:45.387+0000 7f348b093640 1 -- 172.21.15.42:0/209412248 wait complete. 2024-08-30T09:11:45.953 INFO:teuthology.orchestra.run.smithi042.stderr:progress: 660.0 s, 415.4 tps, lat 241.677 ms stddev 296.778 2024-08-30T09:11:46.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:11:46 smithi042 ceph-mon[18652]: pgmap v703: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.6 MiB/s wr, 443 op/s 2024-08-30T09:11:46.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:11:46 smithi190 ceph-mon[22001]: pgmap v703: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.6 MiB/s wr, 443 op/s 2024-08-30T09:11:47.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:11:46 smithi167 ceph-mon[22063]: pgmap v703: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.6 MiB/s wr, 443 op/s 2024-08-30T09:11:48.864 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:11:48 smithi042 ceph-mon[18652]: pgmap v704: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 5.0 MiB/s wr, 401 op/s 2024-08-30T09:11:48.917 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-08-30T09:11:48.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:11:48 smithi190 ceph-mon[22001]: pgmap v704: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 5.0 MiB/s wr, 401 op/s 2024-08-30T09:11:49.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:11:48 smithi167 ceph-mon[22063]: pgmap v704: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 5.0 MiB/s wr, 401 op/s 2024-08-30T09:11:49.404 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T09:11:50.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.064+0000 7fcb7f300640 1 -- 172.21.15.42:0/3330756212 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcb78111a70 msgr2=0x7fcb7810d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:11:50.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.064+0000 7fcb7f300640 1 --2- 172.21.15.42:0/3330756212 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcb78111a70 0x7fcb7810d450 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fcb74009900 tx=0x7fcb7402a440 comp rx=0 tx=0).stop 2024-08-30T09:11:50.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.069+0000 7fcb7f300640 1 -- 172.21.15.42:0/3330756212 shutdown_connections 2024-08-30T09:11:50.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.069+0000 7fcb7f300640 1 --2- 172.21.15.42:0/3330756212 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcb7811c940 0x7fcb7811ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:50.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.069+0000 7fcb7f300640 1 --2- 172.21.15.42:0/3330756212 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcb7810d990 0x7fcb7810dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:50.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.069+0000 7fcb7f300640 1 --2- 172.21.15.42:0/3330756212 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcb78111a70 0x7fcb7810d450 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:50.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.069+0000 7fcb7f300640 1 -- 172.21.15.42:0/3330756212 >> 172.21.15.42:0/3330756212 conn(0x7fcb7806e790 msgr2=0x7fcb7806ebd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:11:50.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.073+0000 7fcb7f300640 1 -- 172.21.15.42:0/3330756212 shutdown_connections 2024-08-30T09:11:50.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.076+0000 7fcb7f300640 1 -- 172.21.15.42:0/3330756212 wait complete. 2024-08-30T09:11:50.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.077+0000 7fcb7f300640 1 Processor -- start 2024-08-30T09:11:50.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.078+0000 7fcb7f300640 1 -- start start 2024-08-30T09:11:50.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.079+0000 7fcb7f300640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcb7810d990 0x7fcb7811c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:50.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.079+0000 7fcb7f300640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcb78111a70 0x7fcb78116550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:50.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.079+0000 7fcb7f300640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcb7811c940 0x7fcb78116bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:50.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.080+0000 7fcb7f300640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fcb78112a20 con 0x7fcb7810d990 2024-08-30T09:11:50.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.080+0000 7fcb7f300640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fcb781128a0 con 0x7fcb7811c940 2024-08-30T09:11:50.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.080+0000 7fcb7f300640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fcb78112ba0 con 0x7fcb78111a70 2024-08-30T09:11:50.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.081+0000 7fcb7dafd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcb78111a70 0x7fcb78116550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:50.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.082+0000 7fcb7e2fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcb7810d990 0x7fcb7811c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:50.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.082+0000 7fcb7eaff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcb7811c940 0x7fcb78116bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:50.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.084+0000 7fcb7e2fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcb7810d990 0x7fcb7811c290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:58350/0 (socket says 172.21.15.42:58350) 2024-08-30T09:11:50.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.084+0000 7fcb7e2fe640 1 -- 172.21.15.42:0/4176515816 learned_addr learned my addr 172.21.15.42:0/4176515816 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:11:50.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.085+0000 7fcb7e2fe640 1 -- 172.21.15.42:0/4176515816 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcb78111a70 msgr2=0x7fcb78116550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:11:50.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.085+0000 7fcb7e2fe640 1 --2- 172.21.15.42:0/4176515816 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcb78111a70 0x7fcb78116550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:50.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.085+0000 7fcb7e2fe640 1 -- 172.21.15.42:0/4176515816 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcb7811c940 msgr2=0x7fcb78116bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:11:50.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.085+0000 7fcb7e2fe640 1 --2- 172.21.15.42:0/4176515816 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcb7811c940 0x7fcb78116bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:50.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.085+0000 7fcb7e2fe640 1 -- 172.21.15.42:0/4176515816 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb781bd960 con 0x7fcb7810d990 2024-08-30T09:11:50.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.085+0000 7fcb7e2fe640 1 --2- 172.21.15.42:0/4176515816 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcb7810d990 0x7fcb7811c290 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fcb74009850 tx=0x7fcb74003e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:11:50.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.086+0000 7fcb6f7fe640 1 -- 172.21.15.42:0/4176515816 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb74040030 con 0x7fcb7810d990 2024-08-30T09:11:50.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.086+0000 7fcb6f7fe640 1 -- 172.21.15.42:0/4176515816 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb7403c8f0 con 0x7fcb7810d990 2024-08-30T09:11:50.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.087+0000 7fcb6f7fe640 1 -- 172.21.15.42:0/4176515816 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb740319a0 con 0x7fcb7810d990 2024-08-30T09:11:50.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.088+0000 7fcb7f300640 1 -- 172.21.15.42:0/4176515816 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb781bdb30 con 0x7fcb7810d990 2024-08-30T09:11:50.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.088+0000 7fcb7f300640 1 -- 172.21.15.42:0/4176515816 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb781bdf50 con 0x7fcb7810d990 2024-08-30T09:11:50.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.097+0000 7fcb86ad7740 1 Processor -- start 2024-08-30T09:11:50.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.102+0000 7fcb6f7fe640 1 -- 172.21.15.42:0/4176515816 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7fcb74043030 con 0x7fcb7810d990 2024-08-30T09:11:50.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.108+0000 7fcb86ad7740 1 -- start start 2024-08-30T09:11:50.114 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.110+0000 7fcb86ad7740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5558848ad070 0x555884c524f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:50.114 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.110+0000 7fcb86ad7740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5558848841b0 0x555884c52a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:50.114 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.110+0000 7fcb86ad7740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55588488c390 0x555884c52f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:50.114 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.110+0000 7fcb86ad7740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x555884bc9680 con 0x5558848ad070 2024-08-30T09:11:50.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.110+0000 7fcb86ad7740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x555884bb8c10 con 0x55588488c390 2024-08-30T09:11:50.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.110+0000 7fcb86ad7740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x555884bbcbc0 con 0x5558848841b0 2024-08-30T09:11:50.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.110+0000 7fcb7eaff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5558848ad070 0x555884c524f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:50.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.110+0000 7fcb7eaff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5558848ad070 0x555884c524f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:58366/0 (socket says 172.21.15.42:58366) 2024-08-30T09:11:50.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.110+0000 7fcb7eaff640 1 -- 172.21.15.42:0/565062058 learned_addr learned my addr 172.21.15.42:0/565062058 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:11:50.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.110+0000 7fcb6f7fe640 1 --2- 172.21.15.42:0/4176515816 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fcb640b4b90 0x7fcb640b70a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:50.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.111+0000 7fcb6f7fe640 1 -- 172.21.15.42:0/4176515816 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7fcb74106180 con 0x7fcb7810d990 2024-08-30T09:11:50.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.111+0000 7fcb7eaff640 1 -- 172.21.15.42:0/565062058 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5558848841b0 msgr2=0x555884c52a30 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-30T09:11:50.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.111+0000 7fcb7eaff640 1 --2- 172.21.15.42:0/565062058 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5558848841b0 0x555884c52a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:50.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.111+0000 7fcb7eaff640 1 -- 172.21.15.42:0/565062058 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55588488c390 msgr2=0x555884c52f70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-30T09:11:50.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.111+0000 7fcb7eaff640 1 --2- 172.21.15.42:0/565062058 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55588488c390 0x555884c52f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:50.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.111+0000 7fcb7eaff640 1 -- 172.21.15.42:0/565062058 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555884918650 con 0x5558848ad070 2024-08-30T09:11:50.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.111+0000 7fcb7eaff640 1 --2- 172.21.15.42:0/565062058 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5558848ad070 0x555884c524f0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fcb6800aca0 tx=0x7fcb680117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de59479d7577c097 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:11:50.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.111+0000 7fcb6cff9640 1 -- 172.21.15.42:0/565062058 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb6801f070 con 0x5558848ad070 2024-08-30T09:11:50.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.112+0000 7fcb6cff9640 1 -- 172.21.15.42:0/565062058 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb68012b00 con 0x5558848ad070 2024-08-30T09:11:50.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.113+0000 7fcb86ad7740 1 -- 172.21.15.42:0/565062058 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5558848ad070 msgr2=0x555884c524f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:11:50.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.113+0000 7fcb6cff9640 1 -- 172.21.15.42:0/565062058 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb6801f070 con 0x5558848ad070 2024-08-30T09:11:50.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.113+0000 7fcb86ad7740 1 --2- 172.21.15.42:0/565062058 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5558848ad070 0x555884c524f0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fcb6800aca0 tx=0x7fcb680117b0 comp rx=0 tx=0).stop 2024-08-30T09:11:50.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.113+0000 7fcb7eaff640 1 --2- 172.21.15.42:0/4176515816 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fcb640b4b90 0x7fcb640b70a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:50.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.118+0000 7fcb7eaff640 1 --2- 172.21.15.42:0/4176515816 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fcb640b4b90 0x7fcb640b70a0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fcb6800aa30 tx=0x7fcb6800a6e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:11:50.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.119+0000 7fcb86ad7740 1 -- 172.21.15.42:0/565062058 shutdown_connections 2024-08-30T09:11:50.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.119+0000 7fcb86ad7740 1 --2- 172.21.15.42:0/565062058 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5558848841b0 0x555884c52a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:50.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.119+0000 7fcb86ad7740 1 --2- 172.21.15.42:0/565062058 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55588488c390 0x555884c52f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:50.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.119+0000 7fcb86ad7740 1 --2- 172.21.15.42:0/565062058 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5558848ad070 0x555884c524f0 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:50.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.119+0000 7fcb86ad7740 1 -- 172.21.15.42:0/565062058 >> 172.21.15.42:0/565062058 conn(0x5558848a1460 msgr2=0x555884c503e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:11:50.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.122+0000 7fcb86ad7740 1 -- 172.21.15.42:0/565062058 shutdown_connections 2024-08-30T09:11:50.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.123+0000 7fcb86ad7740 1 -- 172.21.15.42:0/565062058 wait complete. 2024-08-30T09:11:50.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.123+0000 7fcb86ad7740 1 -- start start 2024-08-30T09:11:50.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.123+0000 7fcb86ad7740 1 Processor -- start 2024-08-30T09:11:50.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.124+0000 7fcb86ad7740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5558848841b0 0x555884c5dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:50.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.124+0000 7fcb86ad7740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55588488c390 0x555884c5e120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:50.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.124+0000 7fcb86ad7740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5558848ad070 0x555884c6a080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:50.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.124+0000 7fcb86ad7740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x555884a1b700 con 0x5558848ad070 2024-08-30T09:11:50.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.124+0000 7fcb86ad7740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x555884b3f750 con 0x5558848841b0 2024-08-30T09:11:50.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.124+0000 7fcb86ad7740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x555884c60d20 con 0x55588488c390 2024-08-30T09:11:50.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.124+0000 7fcb86ad7740 10 client.0 _ll_register_callbacks cb 0x555884b6ee10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:11:50.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.125+0000 7fcb7e2fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5558848841b0 0x555884c5dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:50.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.125+0000 7fcb7e2fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5558848841b0 0x555884c5dbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:38118/0 (socket says 172.21.15.42:38118) 2024-08-30T09:11:50.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.126+0000 7fcb7e2fe640 1 -- 172.21.15.42:0/1373687804 learned_addr learned my addr 172.21.15.42:0/1373687804 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:11:50.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.126+0000 7fcb7dafd640 1 --2- 172.21.15.42:0/1373687804 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55588488c390 0x555884c5e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:50.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.126+0000 7fcb7eaff640 1 --2- 172.21.15.42:0/1373687804 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5558848ad070 0x555884c6a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:50.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.127+0000 7fcb7dafd640 1 -- 172.21.15.42:0/1373687804 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5558848841b0 msgr2=0x555884c5dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:11:50.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.127+0000 7fcb7dafd640 1 --2- 172.21.15.42:0/1373687804 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5558848841b0 0x555884c5dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:50.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.127+0000 7fcb7dafd640 1 -- 172.21.15.42:0/1373687804 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5558848ad070 msgr2=0x555884c6a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:11:50.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.127+0000 7fcb7dafd640 1 --2- 172.21.15.42:0/1373687804 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5558848ad070 0x555884c6a080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:50.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.127+0000 7fcb7dafd640 1 -- 172.21.15.42:0/1373687804 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555884b78900 con 0x55588488c390 2024-08-30T09:11:50.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.127+0000 7fcb7eaff640 1 --2- 172.21.15.42:0/1373687804 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5558848ad070 0x555884c6a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-30T09:11:50.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.135+0000 7fcb7dafd640 1 --2- 172.21.15.42:0/1373687804 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55588488c390 0x555884c5e120 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fcb7806c420 tx=0x7fcb7000c8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f72d45141d730375 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:11:50.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.135+0000 7fcb61ffb640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-08-30T09:11:50.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.139+0000 7fcb61ffb640 1 -- 172.21.15.42:0/1373687804 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb70015020 con 0x55588488c390 2024-08-30T09:11:50.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.139+0000 7fcb61ffb640 1 -- 172.21.15.42:0/1373687804 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7fcb70011840 con 0x55588488c390 2024-08-30T09:11:50.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.140+0000 7fcb86ad7740 1 -- 172.21.15.42:0/1373687804 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcb68012b00 con 0x55588488c390 2024-08-30T09:11:50.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.145+0000 7fcb61ffb640 1 -- 172.21.15.42:0/1373687804 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fcb70011a40 con 0x55588488c390 2024-08-30T09:11:50.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.145+0000 7fcb86ad7740 10 client.27046 fetch_fsmap learned FSMap version 41 2024-08-30T09:11:50.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.145+0000 7fcb86ad7740 1 -- 172.21.15.42:0/1373687804 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x555884c6cb30 con 0x55588488c390 2024-08-30T09:11:50.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.149+0000 7fcb61ffb640 1 -- 172.21.15.42:0/1373687804 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb68012b00 con 0x55588488c390 2024-08-30T09:11:50.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.149+0000 7fcb61ffb640 1 -- 172.21.15.42:0/1373687804 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 41) ==== 8362+0+0 (secure 0 0 0) 0x7fcb70022020 con 0x55588488c390 2024-08-30T09:11:50.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.149+0000 7fcb86ad7740 10 client.27046 fetch_fsmap finished waiting for FSMap version 41 2024-08-30T09:11:50.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.149+0000 7fcb86ad7740 10 client.27046 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-30T09:11:50.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.149+0000 7fcb86ad7740 20 client.27046 populate_metadata read hostname 'smithi042' 2024-08-30T09:11:50.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.149+0000 7fcb86ad7740 1 --2- 172.21.15.42:0/1373687804 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55588486b9f0 0x555884c70490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:50.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.149+0000 7fcb86ad7740 4 client.27046 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-30T09:11:50.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.149+0000 7fcb86ad7740 1 -- 172.21.15.42:0/1373687804 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555884bcb5d0 con 0x55588486b9f0 2024-08-30T09:11:50.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.158+0000 7fcb7e2fe640 1 --2- 172.21.15.42:0/1373687804 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55588486b9f0 0x555884c70490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:50.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.169+0000 7fcb7e2fe640 1 --2- 172.21.15.42:0/1373687804 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55588486b9f0 0x555884c70490 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=99be102b33395852 server_cookie=1033f3a344b16a09 in_seq=0 out_seq=0 2024-08-30T09:11:50.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.169+0000 7fcb61ffb640 10 client.27046 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:11:50.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.170+0000 7fcb61ffb640 1 -- 172.21.15.42:0/1373687804 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x555884bcb5d0 con 0x55588486b9f0 2024-08-30T09:11:50.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.170+0000 7fcb61ffb640 10 client.27046 handle_command_reply: tid=0 2024-08-30T09:11:50.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.170+0000 7fcb86ad7740 1 client.27046 shutdown 2024-08-30T09:11:50.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.171+0000 7fcb86ad7740 1 -- 172.21.15.42:0/1373687804 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55588488c390 msgr2=0x555884c5e120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:11:50.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.171+0000 7fcb86ad7740 1 --2- 172.21.15.42:0/1373687804 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55588488c390 0x555884c5e120 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fcb7806c420 tx=0x7fcb7000c8f0 comp rx=0 tx=0).stop 2024-08-30T09:11:50.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.171+0000 7fcb86ad7740 1 -- 172.21.15.42:0/1373687804 shutdown_connections 2024-08-30T09:11:50.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.171+0000 7fcb86ad7740 1 --2- 172.21.15.42:0/1373687804 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55588486b9f0 0x555884c70490 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:50.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.171+0000 7fcb86ad7740 1 --2- 172.21.15.42:0/1373687804 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55588488c390 0x555884c5e120 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:50.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.171+0000 7fcb86ad7740 1 --2- 172.21.15.42:0/1373687804 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5558848ad070 0x555884c6a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:50.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.171+0000 7fcb86ad7740 1 --2- 172.21.15.42:0/1373687804 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5558848841b0 0x555884c5dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:50.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.171+0000 7fcb86ad7740 1 -- 172.21.15.42:0/1373687804 >> 172.21.15.42:0/1373687804 conn(0x5558848a1460 msgr2=0x555884c556d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:11:50.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.171+0000 7fcb61ffb640 0 client.27046 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:11:50.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.171+0000 7fcb86ad7740 1 -- 172.21.15.42:0/1373687804 shutdown_connections 2024-08-30T09:11:50.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.171+0000 7fcb86ad7740 1 -- 172.21.15.42:0/1373687804 wait complete. 2024-08-30T09:11:50.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.171+0000 7fcb86ad7740 20 client.27046 trim_cache size 0 max 16384 2024-08-30T09:11:50.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.196+0000 7fcb86ad7740 1 Processor -- start 2024-08-30T09:11:50.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.196+0000 7fcb86ad7740 1 -- start start 2024-08-30T09:11:50.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.196+0000 7fcb86ad7740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5558848841b0 0x555884c5b790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:50.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.196+0000 7fcb86ad7740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55588488c390 0x555884c70640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:50.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.196+0000 7fcb86ad7740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5558848ad070 0x555884c5d990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:50.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.196+0000 7fcb86ad7740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fcb781ca4c0 con 0x5558848841b0 2024-08-30T09:11:50.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.196+0000 7fcb7e2fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5558848841b0 0x555884c5b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:50.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.197+0000 7fcb7e2fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5558848841b0 0x555884c5b790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:58388/0 (socket says 172.21.15.42:58388) 2024-08-30T09:11:50.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.197+0000 7fcb7e2fe640 1 -- 172.21.15.42:0/2631949542 learned_addr learned my addr 172.21.15.42:0/2631949542 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:11:50.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.197+0000 7fcb7eaff640 1 --2- 172.21.15.42:0/2631949542 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5558848ad070 0x555884c5d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:50.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.197+0000 7fcb86ad7740 1 -- 172.21.15.42:0/2631949542 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fcb781c7d30 con 0x5558848ad070 2024-08-30T09:11:50.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.197+0000 7fcb86ad7740 1 -- 172.21.15.42:0/2631949542 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fcb781c5b80 con 0x55588488c390 2024-08-30T09:11:50.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.197+0000 7fcb7dafd640 1 --2- 172.21.15.42:0/2631949542 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55588488c390 0x555884c70640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:50.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.197+0000 7fcb7eaff640 1 -- 172.21.15.42:0/2631949542 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55588488c390 msgr2=0x555884c70640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:11:50.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.197+0000 7fcb7eaff640 1 --2- 172.21.15.42:0/2631949542 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55588488c390 0x555884c70640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:50.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.197+0000 7fcb7eaff640 1 -- 172.21.15.42:0/2631949542 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5558848841b0 msgr2=0x555884c5b790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:11:50.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.197+0000 7fcb7eaff640 1 --2- 172.21.15.42:0/2631949542 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5558848841b0 0x555884c5b790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:50.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.198+0000 7fcb7eaff640 1 -- 172.21.15.42:0/2631949542 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555884c54d30 con 0x5558848ad070 2024-08-30T09:11:50.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.198+0000 7fcb7dafd640 1 --2- 172.21.15.42:0/2631949542 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55588488c390 0x555884c70640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-30T09:11:50.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.198+0000 7fcb7e2fe640 1 --2- 172.21.15.42:0/2631949542 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5558848841b0 0x555884c5b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-30T09:11:50.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.198+0000 7fcb7eaff640 1 --2- 172.21.15.42:0/2631949542 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5558848ad070 0x555884c5d990 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fcb6800aca0 tx=0x7fcb68024000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=826822ee7bfc75c0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:11:50.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.198+0000 7fcb63fff640 1 -- 172.21.15.42:0/2631949542 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb6801f040 con 0x5558848ad070 2024-08-30T09:11:50.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.199+0000 7fcb63fff640 1 -- 172.21.15.42:0/2631949542 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb68034070 con 0x5558848ad070 2024-08-30T09:11:50.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.199+0000 7fcb63fff640 1 -- 172.21.15.42:0/2631949542 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb68004d20 con 0x5558848ad070 2024-08-30T09:11:50.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.200+0000 7fcb86ad7740 1 -- 172.21.15.42:0/2631949542 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5558848ad070 msgr2=0x555884c5d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:11:50.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.200+0000 7fcb86ad7740 1 --2- 172.21.15.42:0/2631949542 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5558848ad070 0x555884c5d990 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fcb6800aca0 tx=0x7fcb68024000 comp rx=0 tx=0).stop 2024-08-30T09:11:50.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.200+0000 7fcb86ad7740 1 -- 172.21.15.42:0/2631949542 shutdown_connections 2024-08-30T09:11:50.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.200+0000 7fcb86ad7740 1 --2- 172.21.15.42:0/2631949542 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55588488c390 0x555884c70640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:50.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.200+0000 7fcb86ad7740 1 --2- 172.21.15.42:0/2631949542 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5558848ad070 0x555884c5d990 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:50.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.200+0000 7fcb86ad7740 1 --2- 172.21.15.42:0/2631949542 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5558848841b0 0x555884c5b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:50.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.200+0000 7fcb86ad7740 1 -- 172.21.15.42:0/2631949542 >> 172.21.15.42:0/2631949542 conn(0x5558848a1460 msgr2=0x555884c5c460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:11:50.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.200+0000 7fcb86ad7740 1 -- 172.21.15.42:0/2631949542 shutdown_connections 2024-08-30T09:11:50.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.200+0000 7fcb86ad7740 1 -- 172.21.15.42:0/2631949542 wait complete. 2024-08-30T09:11:50.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.201+0000 7fcb86ad7740 1 -- start start 2024-08-30T09:11:50.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.201+0000 7fcb86ad7740 1 Processor -- start 2024-08-30T09:11:50.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.202+0000 7fcb86ad7740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5558848841b0 0x555884c75c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:50.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.202+0000 7fcb86ad7740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55588488c390 0x555884c76210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:50.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.202+0000 7fcb7e2fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5558848841b0 0x555884c75c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:50.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.202+0000 7fcb7e2fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5558848841b0 0x555884c75c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:38128/0 (socket says 172.21.15.42:38128) 2024-08-30T09:11:50.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.202+0000 7fcb7e2fe640 1 -- 172.21.15.42:0/2995187536 learned_addr learned my addr 172.21.15.42:0/2995187536 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:11:50.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.202+0000 7fcb86ad7740 1 --2- 172.21.15.42:0/2995187536 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5558848ad070 0x555884c65fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:50.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.202+0000 7fcb86ad7740 1 -- 172.21.15.42:0/2995187536 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fcb781c61f0 con 0x5558848ad070 2024-08-30T09:11:50.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.203+0000 7fcb86ad7740 1 -- 172.21.15.42:0/2995187536 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fcb781c7660 con 0x5558848841b0 2024-08-30T09:11:50.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.203+0000 7fcb86ad7740 1 -- 172.21.15.42:0/2995187536 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fcb781c68c0 con 0x55588488c390 2024-08-30T09:11:50.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.203+0000 7fcb7dafd640 1 --2- 172.21.15.42:0/2995187536 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55588488c390 0x555884c76210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:50.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.203+0000 7fcb7e2fe640 1 -- 172.21.15.42:0/2995187536 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55588488c390 msgr2=0x555884c76210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:11:50.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.203+0000 7fcb7e2fe640 1 --2- 172.21.15.42:0/2995187536 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55588488c390 0x555884c76210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:50.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.203+0000 7fcb7e2fe640 1 -- 172.21.15.42:0/2995187536 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5558848ad070 msgr2=0x555884c65fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:11:50.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.203+0000 7fcb7e2fe640 1 --2- 172.21.15.42:0/2995187536 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5558848ad070 0x555884c65fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:50.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.203+0000 7fcb7e2fe640 1 -- 172.21.15.42:0/2995187536 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555884c66520 con 0x5558848841b0 2024-08-30T09:11:50.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.203+0000 7fcb86ad7740 10 client.0 _ll_register_callbacks cb 0x7fcb70008040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:11:50.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.203+0000 7fcb86ad7740 1 -- 172.21.15.42:0/2995187536 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcb68034070 con 0x5558848841b0 2024-08-30T09:11:50.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.203+0000 7fcb7e2fe640 1 --2- 172.21.15.42:0/2995187536 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5558848841b0 0x555884c75c90 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fcb740c87b0 tx=0x7fcb74103760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b0c3a7bf3767fcb7 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:11:50.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.204+0000 7fcb61ffb640 10 client.27098 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-30T09:11:50.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.204+0000 7fcb61ffb640 1 -- 172.21.15.42:0/2995187536 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb740cc7c0 con 0x5558848841b0 2024-08-30T09:11:50.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.204+0000 7fcb61ffb640 1 -- 172.21.15.42:0/2995187536 <== mon.2 v2:172.21.15.167:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7fcb740c8d20 con 0x5558848841b0 2024-08-30T09:11:50.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.204+0000 7fcb61ffb640 1 -- 172.21.15.42:0/2995187536 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fcb740d1030 con 0x5558848841b0 2024-08-30T09:11:50.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.204+0000 7fcb86ad7740 10 client.27098 fetch_fsmap learned FSMap version 41 2024-08-30T09:11:50.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.204+0000 7fcb86ad7740 1 -- 172.21.15.42:0/2995187536 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({fsmap=41}) -- 0x555884c6fd50 con 0x5558848841b0 2024-08-30T09:11:50.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.206+0000 7fcb61ffb640 1 -- 172.21.15.42:0/2995187536 <== mon.2 v2:172.21.15.167:3300/0 4 ==== fsmap(e 41) ==== 8362+0+0 (secure 0 0 0) 0x7fcb740cca20 con 0x5558848841b0 2024-08-30T09:11:50.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.206+0000 7fcb86ad7740 10 client.27098 fetch_fsmap finished waiting for FSMap version 41 2024-08-30T09:11:50.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.206+0000 7fcb86ad7740 10 client.27098 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-30T09:11:50.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.206+0000 7fcb86ad7740 20 client.27098 populate_metadata read hostname 'smithi042' 2024-08-30T09:11:50.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.206+0000 7fcb86ad7740 1 --2- 172.21.15.42:0/2995187536 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x555884840da0 0x555884cdcea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:50.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.206+0000 7fcb86ad7740 4 client.27098 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-30T09:11:50.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.206+0000 7fcb86ad7740 1 -- 172.21.15.42:0/2995187536 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555884bc7070 con 0x555884840da0 2024-08-30T09:11:50.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.206+0000 7fcb7dafd640 1 --2- 172.21.15.42:0/2995187536 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x555884840da0 0x555884cdcea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:50.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.206+0000 7fcb61ffb640 10 client.27098 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:11:50.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.206+0000 7fcb7dafd640 1 --2- 172.21.15.42:0/2995187536 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x555884840da0 0x555884cdcea0 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dfcbd8eb80f0ee45 server_cookie=1891d9d43ca10fc8 in_seq=0 out_seq=0 2024-08-30T09:11:50.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.207+0000 7fcb61ffb640 1 -- 172.21.15.42:0/2995187536 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555884bc7070 con 0x555884840da0 2024-08-30T09:11:50.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.207+0000 7fcb61ffb640 10 client.27098 handle_command_reply: tid=0 2024-08-30T09:11:50.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.207+0000 7fcb86ad7740 1 client.27098 shutdown 2024-08-30T09:11:50.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.207+0000 7fcb86ad7740 1 -- 172.21.15.42:0/2995187536 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5558848841b0 msgr2=0x555884c75c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:11:50.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.207+0000 7fcb86ad7740 1 --2- 172.21.15.42:0/2995187536 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5558848841b0 0x555884c75c90 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fcb740c87b0 tx=0x7fcb74103760 comp rx=0 tx=0).stop 2024-08-30T09:11:50.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.207+0000 7fcb86ad7740 1 -- 172.21.15.42:0/2995187536 shutdown_connections 2024-08-30T09:11:50.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.207+0000 7fcb86ad7740 1 --2- 172.21.15.42:0/2995187536 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x555884840da0 0x555884cdcea0 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:50.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.207+0000 7fcb86ad7740 1 --2- 172.21.15.42:0/2995187536 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55588488c390 0x555884c76210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:50.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.207+0000 7fcb86ad7740 1 --2- 172.21.15.42:0/2995187536 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5558848ad070 0x555884c65fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:50.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.207+0000 7fcb86ad7740 1 --2- 172.21.15.42:0/2995187536 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5558848841b0 0x555884c75c90 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:50.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.207+0000 7fcb86ad7740 1 -- 172.21.15.42:0/2995187536 >> 172.21.15.42:0/2995187536 conn(0x5558848a1460 msgr2=0x555884c4fce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:11:50.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.207+0000 7fcb61ffb640 0 client.27098 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:11:50.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.207+0000 7fcb86ad7740 1 -- 172.21.15.42:0/2995187536 shutdown_connections 2024-08-30T09:11:50.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.207+0000 7fcb86ad7740 1 -- 172.21.15.42:0/2995187536 wait complete. 2024-08-30T09:11:50.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.207+0000 7fcb86ad7740 20 client.27098 trim_cache size 0 max 16384 2024-08-30T09:11:50.212 INFO:teuthology.orchestra.run.smithi042.stdout:[] 2024-08-30T09:11:50.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.208+0000 7fcb7f300640 1 -- 172.21.15.42:0/4176515816 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fcb640b4b90 msgr2=0x7fcb640b70a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:11:50.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.208+0000 7fcb7f300640 1 --2- 172.21.15.42:0/4176515816 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fcb640b4b90 0x7fcb640b70a0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fcb6800aa30 tx=0x7fcb6800a6e0 comp rx=0 tx=0).stop 2024-08-30T09:11:50.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.208+0000 7fcb7f300640 1 -- 172.21.15.42:0/4176515816 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcb7810d990 msgr2=0x7fcb7811c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:11:50.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.208+0000 7fcb7f300640 1 --2- 172.21.15.42:0/4176515816 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcb7810d990 0x7fcb7811c290 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fcb74009850 tx=0x7fcb74003e90 comp rx=0 tx=0).stop 2024-08-30T09:11:50.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.208+0000 7fcb7f300640 1 -- 172.21.15.42:0/4176515816 shutdown_connections 2024-08-30T09:11:50.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.208+0000 7fcb7f300640 1 --2- 172.21.15.42:0/4176515816 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fcb640b4b90 0x7fcb640b70a0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:50.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.208+0000 7fcb7f300640 1 --2- 172.21.15.42:0/4176515816 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcb78111a70 0x7fcb78116550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:50.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.208+0000 7fcb7f300640 1 --2- 172.21.15.42:0/4176515816 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcb7811c940 0x7fcb78116bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:50.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.208+0000 7fcb7f300640 1 --2- 172.21.15.42:0/4176515816 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcb7810d990 0x7fcb7811c290 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:50.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.208+0000 7fcb7f300640 1 -- 172.21.15.42:0/4176515816 >> 172.21.15.42:0/4176515816 conn(0x7fcb7806e790 msgr2=0x7fcb7810f5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:11:50.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.208+0000 7fcb7f300640 1 -- 172.21.15.42:0/4176515816 shutdown_connections 2024-08-30T09:11:50.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:50.208+0000 7fcb7f300640 1 -- 172.21.15.42:0/4176515816 wait complete. 2024-08-30T09:11:50.953 INFO:teuthology.orchestra.run.smithi042.stderr:progress: 665.0 s, 436.8 tps, lat 232.415 ms stddev 284.621 2024-08-30T09:11:51.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:11:50 smithi042 ceph-mon[18652]: pgmap v705: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 6.7 MiB/s wr, 526 op/s 2024-08-30T09:11:51.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:11:50 smithi190 ceph-mon[22001]: pgmap v705: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 6.7 MiB/s wr, 526 op/s 2024-08-30T09:11:51.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:11:50 smithi167 ceph-mon[22063]: pgmap v705: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 6.7 MiB/s wr, 526 op/s 2024-08-30T09:11:51.919 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-08-30T09:11:51.919 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-08-30T09:11:52.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:11:51 smithi190 ceph-mon[22001]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-30T09:11:52.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:11:51 smithi167 ceph-mon[22063]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-30T09:11:52.370 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T09:11:52.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:11:51 smithi042 ceph-mon[18652]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-30T09:11:53.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:11:52 smithi190 ceph-mon[22001]: pgmap v706: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 4.8 MiB/s wr, 386 op/s 2024-08-30T09:11:53.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:11:52 smithi167 ceph-mon[22063]: pgmap v706: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 4.8 MiB/s wr, 386 op/s 2024-08-30T09:11:53.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:11:52 smithi042 ceph-mon[18652]: pgmap v706: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 4.8 MiB/s wr, 386 op/s 2024-08-30T09:11:54.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.729+0000 7f7bc340d640 1 -- 172.21.15.42:0/279874061 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7bbc103920 msgr2=0x7f7bbc103d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:11:54.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.729+0000 7f7bc340d640 1 --2- 172.21.15.42:0/279874061 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7bbc103920 0x7f7bbc103d30 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f7bac009900 tx=0x7f7bac02a440 comp rx=0 tx=0).stop 2024-08-30T09:11:54.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.730+0000 7f7bc340d640 1 -- 172.21.15.42:0/279874061 shutdown_connections 2024-08-30T09:11:54.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.730+0000 7f7bc340d640 1 --2- 172.21.15.42:0/279874061 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7bbc109260 0x7f7bbc109670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:54.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.731+0000 7f7bc340d640 1 --2- 172.21.15.42:0/279874061 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7bbc109bb0 0x7f7bbc1103e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:54.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.731+0000 7f7bc340d640 1 --2- 172.21.15.42:0/279874061 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7bbc103920 0x7f7bbc103d30 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:54.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.731+0000 7f7bc340d640 1 -- 172.21.15.42:0/279874061 >> 172.21.15.42:0/279874061 conn(0x7f7bbc0ff3b0 msgr2=0x7f7bbc101800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:11:54.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.738+0000 7f7bc340d640 1 -- 172.21.15.42:0/279874061 shutdown_connections 2024-08-30T09:11:54.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.739+0000 7f7bc340d640 1 -- 172.21.15.42:0/279874061 wait complete. 2024-08-30T09:11:54.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.740+0000 7f7bc340d640 1 Processor -- start 2024-08-30T09:11:54.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.740+0000 7f7bc340d640 1 -- start start 2024-08-30T09:11:54.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.741+0000 7f7bc340d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7bbc103920 0x7f7bbc19a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:54.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.741+0000 7f7bc340d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7bbc109260 0x7f7bbc19a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:54.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.741+0000 7f7bc340d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7bbc109bb0 0x7f7bbc19fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:54.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.741+0000 7f7bc340d640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f7bbc1139a0 con 0x7f7bbc103920 2024-08-30T09:11:54.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.741+0000 7f7bc340d640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f7bbc113820 con 0x7f7bbc109bb0 2024-08-30T09:11:54.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.743+0000 7f7bc19aa640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7bbc109bb0 0x7f7bbc19fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:54.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.744+0000 7f7bc19aa640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7bbc109bb0 0x7f7bbc19fc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:38140/0 (socket says 172.21.15.42:38140) 2024-08-30T09:11:54.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.744+0000 7f7bc19aa640 1 -- 172.21.15.42:0/3959001230 learned_addr learned my addr 172.21.15.42:0/3959001230 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:11:54.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.744+0000 7f7bc340d640 1 -- 172.21.15.42:0/3959001230 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f7bbc113b20 con 0x7f7bbc109260 2024-08-30T09:11:54.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.746+0000 7f7bc09a8640 1 --2- 172.21.15.42:0/3959001230 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7bbc109260 0x7f7bbc19a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:54.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.746+0000 7f7bc19aa640 1 -- 172.21.15.42:0/3959001230 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7bbc109260 msgr2=0x7f7bbc19a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:11:54.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.746+0000 7f7bc19aa640 1 --2- 172.21.15.42:0/3959001230 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7bbc109260 0x7f7bbc19a970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:54.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.746+0000 7f7bc19aa640 1 -- 172.21.15.42:0/3959001230 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7bbc103920 msgr2=0x7f7bbc19a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:11:54.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.746+0000 7f7bc19aa640 1 --2- 172.21.15.42:0/3959001230 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7bbc103920 0x7f7bbc19a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:54.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.746+0000 7f7bc19aa640 1 -- 172.21.15.42:0/3959001230 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bbc1a0340 con 0x7f7bbc109bb0 2024-08-30T09:11:54.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.747+0000 7f7bc19aa640 1 --2- 172.21.15.42:0/3959001230 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7bbc109bb0 0x7f7bbc19fc10 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f7bb400ff40 tx=0x7f7bb400ff70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:11:54.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.748+0000 7f7bb27fc640 1 -- 172.21.15.42:0/3959001230 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bb400d0d0 con 0x7f7bbc109bb0 2024-08-30T09:11:54.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.749+0000 7f7bb27fc640 1 -- 172.21.15.42:0/3959001230 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7bb400b760 con 0x7f7bbc109bb0 2024-08-30T09:11:54.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.749+0000 7f7bb27fc640 1 -- 172.21.15.42:0/3959001230 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bb4018770 con 0x7f7bbc109bb0 2024-08-30T09:11:54.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.749+0000 7f7bc340d640 1 -- 172.21.15.42:0/3959001230 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7bbc1a05d0 con 0x7f7bbc109bb0 2024-08-30T09:11:54.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.749+0000 7f7bc340d640 1 -- 172.21.15.42:0/3959001230 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7bbc1a0a50 con 0x7f7bbc109bb0 2024-08-30T09:11:54.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.756+0000 7f7bb27fc640 1 -- 172.21.15.42:0/3959001230 <== mon.2 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7f7bb4009030 con 0x7f7bbc109bb0 2024-08-30T09:11:54.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.757+0000 7f7bc9982740 1 Processor -- start 2024-08-30T09:11:54.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.757+0000 7f7bc9982740 1 -- start start 2024-08-30T09:11:54.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.758+0000 7f7bc9982740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e7ffb5d350 0x55e7fff03780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:54.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.758+0000 7f7bc9982740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e7ffb34490 0x55e7fff03cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:54.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.758+0000 7f7bc9982740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e7ffb3c670 0x55e7fff04200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:54.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.758+0000 7f7bc9982740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55e7ffe795f0 con 0x55e7ffb34490 2024-08-30T09:11:54.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.758+0000 7f7bc9982740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55e7ffe68b80 con 0x55e7ffb5d350 2024-08-30T09:11:54.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.758+0000 7f7bc9982740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55e7ffe6cb30 con 0x55e7ffb3c670 2024-08-30T09:11:54.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.758+0000 7f7bb27fc640 1 --2- 172.21.15.42:0/3959001230 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f7ba80b4cc0 0x7f7ba80b71d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:54.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.759+0000 7f7bc11a9640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e7ffb34490 0x55e7fff03cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:54.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.759+0000 7f7bc11a9640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e7ffb34490 0x55e7fff03cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:58440/0 (socket says 172.21.15.42:58440) 2024-08-30T09:11:54.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.759+0000 7f7bc11a9640 1 -- 172.21.15.42:0/3072477362 learned_addr learned my addr 172.21.15.42:0/3072477362 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:11:54.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.759+0000 7f7bb27fc640 1 -- 172.21.15.42:0/3959001230 <== mon.2 v2:172.21.15.167:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f7bb40e54d0 con 0x7f7bbc109bb0 2024-08-30T09:11:54.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.759+0000 7f7bc11a9640 1 -- 172.21.15.42:0/3072477362 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e7ffb3c670 msgr2=0x55e7fff04200 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-30T09:11:54.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.759+0000 7f7bc11a9640 1 --2- 172.21.15.42:0/3072477362 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e7ffb3c670 0x55e7fff04200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:54.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.759+0000 7f7bc11a9640 1 -- 172.21.15.42:0/3072477362 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e7ffb5d350 msgr2=0x55e7fff03780 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-30T09:11:54.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.759+0000 7f7bc11a9640 1 --2- 172.21.15.42:0/3072477362 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e7ffb5d350 0x55e7fff03780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:54.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.760+0000 7f7bc11a9640 1 -- 172.21.15.42:0/3072477362 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e7ffbc8930 con 0x55e7ffb34490 2024-08-30T09:11:54.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.760+0000 7f7bc19aa640 1 --2- 172.21.15.42:0/3959001230 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f7ba80b4cc0 0x7f7ba80b71d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:54.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.760+0000 7f7bc11a9640 1 --2- 172.21.15.42:0/3072477362 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e7ffb34490 0x55e7fff03cc0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f7bac009230 tx=0x7f7bac000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=540c8c52b771efc server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:11:54.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.760+0000 7f7ba77fe640 1 -- 172.21.15.42:0/3072477362 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bac03e030 con 0x55e7ffb34490 2024-08-30T09:11:54.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.761+0000 7f7ba77fe640 1 -- 172.21.15.42:0/3072477362 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7bac004280 con 0x55e7ffb34490 2024-08-30T09:11:54.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.761+0000 7f7ba77fe640 1 -- 172.21.15.42:0/3072477362 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bac03e030 con 0x55e7ffb34490 2024-08-30T09:11:54.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.762+0000 7f7bc9982740 1 -- 172.21.15.42:0/3072477362 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e7ffb34490 msgr2=0x55e7fff03cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:11:54.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.762+0000 7f7bc9982740 1 --2- 172.21.15.42:0/3072477362 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e7ffb34490 0x55e7fff03cc0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f7bac009230 tx=0x7f7bac000bc0 comp rx=0 tx=0).stop 2024-08-30T09:11:54.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.762+0000 7f7bc19aa640 1 --2- 172.21.15.42:0/3959001230 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f7ba80b4cc0 0x7f7ba80b71d0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f7bb40abaa0 tx=0x7f7bb4012040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:11:54.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.763+0000 7f7bc9982740 1 -- 172.21.15.42:0/3072477362 shutdown_connections 2024-08-30T09:11:54.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.763+0000 7f7bc9982740 1 --2- 172.21.15.42:0/3072477362 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e7ffb3c670 0x55e7fff04200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:54.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.763+0000 7f7bc9982740 1 --2- 172.21.15.42:0/3072477362 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e7ffb34490 0x55e7fff03cc0 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:54.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.763+0000 7f7bc9982740 1 --2- 172.21.15.42:0/3072477362 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e7ffb5d350 0x55e7fff03780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:54.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.763+0000 7f7bc9982740 1 -- 172.21.15.42:0/3072477362 >> 172.21.15.42:0/3072477362 conn(0x55e7ffb51740 msgr2=0x55e7fff01670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:11:54.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.763+0000 7f7bc9982740 1 -- 172.21.15.42:0/3072477362 shutdown_connections 2024-08-30T09:11:54.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.764+0000 7f7bc9982740 1 -- 172.21.15.42:0/3072477362 wait complete. 2024-08-30T09:11:54.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.764+0000 7f7bc9982740 1 -- start start 2024-08-30T09:11:54.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.764+0000 7f7bc9982740 1 Processor -- start 2024-08-30T09:11:54.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.765+0000 7f7bc9982740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e7ffb34490 0x55e7fff0ee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:54.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.765+0000 7f7bc9982740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e7ffb3c670 0x55e7fff0f380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:54.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.765+0000 7f7bc9982740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e7ffb5d350 0x55e7fff1b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:54.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.765+0000 7f7bc9982740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55e7ffccb890 con 0x55e7ffb34490 2024-08-30T09:11:54.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.765+0000 7f7bc9982740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55e7ffdef790 con 0x55e7ffb5d350 2024-08-30T09:11:54.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.765+0000 7f7bc9982740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55e7fff11f50 con 0x55e7ffb3c670 2024-08-30T09:11:54.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.765+0000 7f7bc9982740 10 client.0 _ll_register_callbacks cb 0x55e7ffb74ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:11:54.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.766+0000 7f7bc11a9640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e7ffb3c670 0x55e7fff0f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:54.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.766+0000 7f7bc11a9640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e7ffb3c670 0x55e7fff0f380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:52720/0 (socket says 172.21.15.42:52720) 2024-08-30T09:11:54.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.766+0000 7f7bc11a9640 1 -- 172.21.15.42:0/3306577427 learned_addr learned my addr 172.21.15.42:0/3306577427 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:11:54.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.767+0000 7f7bc11a9640 1 -- 172.21.15.42:0/3306577427 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e7ffb5d350 msgr2=0x55e7fff1b2d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-30T09:11:54.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.767+0000 7f7bc11a9640 1 --2- 172.21.15.42:0/3306577427 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e7ffb5d350 0x55e7fff1b2d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:54.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.767+0000 7f7bc11a9640 1 -- 172.21.15.42:0/3306577427 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e7ffb34490 msgr2=0x55e7fff0ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:11:54.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.767+0000 7f7bc11a9640 1 --2- 172.21.15.42:0/3306577427 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e7ffb34490 0x55e7fff0ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:54.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.767+0000 7f7bc11a9640 1 -- 172.21.15.42:0/3306577427 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e7ffe26f30 con 0x55e7ffb3c670 2024-08-30T09:11:54.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.768+0000 7f7bc11a9640 1 --2- 172.21.15.42:0/3306577427 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e7ffb3c670 0x55e7fff0f380 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f7bac02a9b0 tx=0x7f7bac0038a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=955c7d3680d17a01 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:11:54.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.769+0000 7f7ba4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-08-30T09:11:54.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.769+0000 7f7ba4ff9640 1 -- 172.21.15.42:0/3306577427 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bac03e030 con 0x55e7ffb3c670 2024-08-30T09:11:54.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.769+0000 7f7ba4ff9640 1 -- 172.21.15.42:0/3306577427 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f7bac003ca0 con 0x55e7ffb3c670 2024-08-30T09:11:54.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.769+0000 7f7bc9982740 1 -- 172.21.15.42:0/3306577427 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7bac004280 con 0x55e7ffb3c670 2024-08-30T09:11:54.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.770+0000 7f7ba4ff9640 1 -- 172.21.15.42:0/3306577427 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f7bac035040 con 0x55e7ffb3c670 2024-08-30T09:11:54.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.770+0000 7f7bc9982740 10 client.27055 fetch_fsmap learned FSMap version 42 2024-08-30T09:11:54.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.770+0000 7f7bc9982740 1 -- 172.21.15.42:0/3306577427 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55e7fff2eca0 con 0x55e7ffb3c670 2024-08-30T09:11:54.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.772+0000 7f7ba4ff9640 1 -- 172.21.15.42:0/3306577427 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7bac004280 con 0x55e7ffb3c670 2024-08-30T09:11:54.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.772+0000 7f7ba4ff9640 1 -- 172.21.15.42:0/3306577427 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 42) ==== 8358+0+0 (secure 0 0 0) 0x7f7bac054780 con 0x55e7ffb3c670 2024-08-30T09:11:54.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.772+0000 7f7bc9982740 10 client.27055 fetch_fsmap finished waiting for FSMap version 42 2024-08-30T09:11:54.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.772+0000 7f7bc9982740 10 client.27055 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-30T09:11:54.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.772+0000 7f7bc9982740 20 client.27055 populate_metadata read hostname 'smithi042' 2024-08-30T09:11:54.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.772+0000 7f7bc9982740 1 --2- 172.21.15.42:0/3306577427 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55e7ffb1bcd0 0x55e7fff32000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:54.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.772+0000 7f7bc9982740 4 client.27055 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-30T09:11:54.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.772+0000 7f7bc9982740 1 -- 172.21.15.42:0/3306577427 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e7ffe7b540 con 0x55e7ffb1bcd0 2024-08-30T09:11:54.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.773+0000 7f7bc09a8640 1 --2- 172.21.15.42:0/3306577427 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55e7ffb1bcd0 0x55e7fff32000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:54.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.773+0000 7f7ba4ff9640 10 client.27055 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:11:54.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.774+0000 7f7bc09a8640 1 --2- 172.21.15.42:0/3306577427 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55e7ffb1bcd0 0x55e7fff32000 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7be229d48ac99c3e server_cookie=7bd0669c6ec185fc in_seq=0 out_seq=0 2024-08-30T09:11:54.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.775+0000 7f7ba4ff9640 1 -- 172.21.15.42:0/3306577427 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55e7ffe7b540 con 0x55e7ffb1bcd0 2024-08-30T09:11:54.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.775+0000 7f7ba4ff9640 10 client.27055 handle_command_reply: tid=0 2024-08-30T09:11:54.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.775+0000 7f7bc9982740 1 client.27055 shutdown 2024-08-30T09:11:54.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.776+0000 7f7bc9982740 1 -- 172.21.15.42:0/3306577427 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e7ffb3c670 msgr2=0x55e7fff0f380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:11:54.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.776+0000 7f7bc9982740 1 --2- 172.21.15.42:0/3306577427 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e7ffb3c670 0x55e7fff0f380 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f7bac02a9b0 tx=0x7f7bac0038a0 comp rx=0 tx=0).stop 2024-08-30T09:11:54.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.776+0000 7f7bc9982740 1 -- 172.21.15.42:0/3306577427 shutdown_connections 2024-08-30T09:11:54.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.776+0000 7f7bc9982740 1 --2- 172.21.15.42:0/3306577427 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55e7ffb1bcd0 0x55e7fff32000 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:54.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.776+0000 7f7bc9982740 1 --2- 172.21.15.42:0/3306577427 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e7ffb3c670 0x55e7fff0f380 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:54.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.776+0000 7f7bc9982740 1 --2- 172.21.15.42:0/3306577427 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e7ffb5d350 0x55e7fff1b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:54.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.776+0000 7f7bc9982740 1 --2- 172.21.15.42:0/3306577427 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e7ffb34490 0x55e7fff0ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:54.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.776+0000 7f7bc9982740 1 -- 172.21.15.42:0/3306577427 >> 172.21.15.42:0/3306577427 conn(0x55e7ffb51740 msgr2=0x55e7fff06910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:11:54.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.776+0000 7f7ba4ff9640 0 client.27055 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:11:54.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.777+0000 7f7bc9982740 1 -- 172.21.15.42:0/3306577427 shutdown_connections 2024-08-30T09:11:54.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.777+0000 7f7bc9982740 1 -- 172.21.15.42:0/3306577427 wait complete. 2024-08-30T09:11:54.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.777+0000 7f7bc9982740 20 client.27055 trim_cache size 0 max 16384 2024-08-30T09:11:54.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.815+0000 7f7bc9982740 1 Processor -- start 2024-08-30T09:11:54.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.817+0000 7f7bc9982740 1 -- start start 2024-08-30T09:11:54.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.817+0000 7f7bc9982740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e7ffb34490 0x55e7fff04ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:54.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.817+0000 7f7bc9982740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e7ffb3c670 0x55e7fff00ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:54.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.817+0000 7f7bc9982740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e7ffb5d350 0x55e7fff011e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:54.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.817+0000 7f7bc9982740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f7ba0012500 con 0x55e7ffb3c670 2024-08-30T09:11:54.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.817+0000 7f7bc9982740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f7ba000fda0 con 0x55e7ffb5d350 2024-08-30T09:11:54.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.817+0000 7f7bc9982740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f7ba000dbf0 con 0x55e7ffb34490 2024-08-30T09:11:54.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.818+0000 7f7bc09a8640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e7ffb34490 0x55e7fff04ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:54.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.818+0000 7f7bc09a8640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e7ffb34490 0x55e7fff04ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:52724/0 (socket says 172.21.15.42:52724) 2024-08-30T09:11:54.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.818+0000 7f7bc09a8640 1 -- 172.21.15.42:0/3773298700 learned_addr learned my addr 172.21.15.42:0/3773298700 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:11:54.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.819+0000 7f7bc09a8640 1 --2- 172.21.15.42:0/3773298700 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e7ffb5d350 0x55e7fff011e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:54.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.821+0000 7f7bc11a9640 1 --2- 172.21.15.42:0/3773298700 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e7ffb3c670 0x55e7fff00ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:54.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.822+0000 7f7bc11a9640 1 -- 172.21.15.42:0/3773298700 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e7ffb34490 msgr2=0x55e7fff04ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:11:54.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.822+0000 7f7bc11a9640 1 --2- 172.21.15.42:0/3773298700 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e7ffb34490 0x55e7fff04ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:54.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.822+0000 7f7bc11a9640 1 -- 172.21.15.42:0/3773298700 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e7ffb5d350 msgr2=0x55e7fff011e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:11:54.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.822+0000 7f7bc11a9640 1 --2- 172.21.15.42:0/3773298700 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e7ffb5d350 0x55e7fff011e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:54.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.822+0000 7f7bc11a9640 1 -- 172.21.15.42:0/3773298700 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e7fff05ca0 con 0x55e7ffb3c670 2024-08-30T09:11:54.829 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.828+0000 7f7bc11a9640 1 --2- 172.21.15.42:0/3773298700 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e7ffb3c670 0x55e7fff00ca0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f7bac054c20 tx=0x7f7bac000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9af57450e57b7ce8 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:11:54.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.837+0000 7f7ba6ffd640 1 -- 172.21.15.42:0/3773298700 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bac03e040 con 0x55e7ffb3c670 2024-08-30T09:11:54.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.837+0000 7f7ba6ffd640 1 -- 172.21.15.42:0/3773298700 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7bac058020 con 0x55e7ffb3c670 2024-08-30T09:11:54.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.838+0000 7f7bc9982740 1 -- 172.21.15.42:0/3773298700 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e7ffb3c670 msgr2=0x55e7fff00ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:11:54.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.838+0000 7f7bc9982740 1 --2- 172.21.15.42:0/3773298700 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e7ffb3c670 0x55e7fff00ca0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f7bac054c20 tx=0x7f7bac000bc0 comp rx=0 tx=0).stop 2024-08-30T09:11:54.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.842+0000 7f7bc9982740 1 -- 172.21.15.42:0/3773298700 shutdown_connections 2024-08-30T09:11:54.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.842+0000 7f7bc9982740 1 --2- 172.21.15.42:0/3773298700 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e7ffb5d350 0x55e7fff011e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:54.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.842+0000 7f7bc9982740 1 --2- 172.21.15.42:0/3773298700 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e7ffb3c670 0x55e7fff00ca0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:54.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.842+0000 7f7bc9982740 1 --2- 172.21.15.42:0/3773298700 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e7ffb34490 0x55e7fff04ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:54.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.842+0000 7f7bc9982740 1 -- 172.21.15.42:0/3773298700 >> 172.21.15.42:0/3773298700 conn(0x55e7ffb51740 msgr2=0x55e7fff047b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:11:54.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.846+0000 7f7bc9982740 1 -- 172.21.15.42:0/3773298700 shutdown_connections 2024-08-30T09:11:54.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.846+0000 7f7bc9982740 1 -- 172.21.15.42:0/3773298700 wait complete. 2024-08-30T09:11:54.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.847+0000 7f7bc9982740 1 -- start start 2024-08-30T09:11:54.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.847+0000 7f7bc9982740 1 Processor -- start 2024-08-30T09:11:54.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.848+0000 7f7bc9982740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e7ffb34490 0x55e7fff34060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:54.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.848+0000 7f7bc9982740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e7ffb3c670 0x55e7fff345a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:54.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.848+0000 7f7bc9982740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e7ffb5d350 0x55e7fff34ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:54.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.848+0000 7f7bc9982740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f7ba000e260 con 0x55e7ffb3c670 2024-08-30T09:11:54.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.848+0000 7f7bc9982740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f7ba000f6d0 con 0x55e7ffb34490 2024-08-30T09:11:54.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.848+0000 7f7bc9982740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f7ba000e930 con 0x55e7ffb5d350 2024-08-30T09:11:54.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.848+0000 7f7bc9982740 10 client.0 _ll_register_callbacks cb 0x7f7bac0040d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:11:54.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.849+0000 7f7bc11a9640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e7ffb3c670 0x55e7fff345a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:54.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.849+0000 7f7bc11a9640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e7ffb3c670 0x55e7fff345a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:58472/0 (socket says 172.21.15.42:58472) 2024-08-30T09:11:54.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.849+0000 7f7bc11a9640 1 -- 172.21.15.42:0/1289690257 learned_addr learned my addr 172.21.15.42:0/1289690257 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:11:54.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.855+0000 7f7bc09a8640 1 --2- 172.21.15.42:0/1289690257 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e7ffb34490 0x55e7fff34060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:54.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.855+0000 7f7bc09a8640 1 --2- 172.21.15.42:0/1289690257 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e7ffb5d350 0x55e7fff34ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:54.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.863+0000 7f7bc11a9640 1 -- 172.21.15.42:0/1289690257 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e7ffb5d350 msgr2=0x55e7fff34ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:11:54.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.863+0000 7f7bc11a9640 1 --2- 172.21.15.42:0/1289690257 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e7ffb5d350 0x55e7fff34ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:54.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.863+0000 7f7bc11a9640 1 -- 172.21.15.42:0/1289690257 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e7ffb34490 msgr2=0x55e7fff34060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:11:54.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.863+0000 7f7bc11a9640 1 --2- 172.21.15.42:0/1289690257 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e7ffb34490 0x55e7fff34060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:54.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.863+0000 7f7bc11a9640 1 -- 172.21.15.42:0/1289690257 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e7fff9c7c0 con 0x55e7ffb3c670 2024-08-30T09:11:54.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.865+0000 7f7bc11a9640 1 --2- 172.21.15.42:0/1289690257 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e7ffb3c670 0x55e7fff345a0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f7bac004650 tx=0x7f7bac009640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eebbd4cd20343146 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:11:54.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.865+0000 7f7ba4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-08-30T09:11:54.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.867+0000 7f7ba4ff9640 1 -- 172.21.15.42:0/1289690257 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bac061060 con 0x55e7ffb3c670 2024-08-30T09:11:54.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.867+0000 7f7ba4ff9640 1 -- 172.21.15.42:0/1289690257 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f7bac054620 con 0x55e7ffb3c670 2024-08-30T09:11:54.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.868+0000 7f7bc9982740 1 -- 172.21.15.42:0/1289690257 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55e7ffe76fe0 con 0x55e7ffb3c670 2024-08-30T09:11:54.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.869+0000 7f7ba4ff9640 1 -- 172.21.15.42:0/1289690257 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f7bac054820 con 0x55e7ffb3c670 2024-08-30T09:11:54.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.869+0000 7f7bc9982740 10 client.17274 fetch_fsmap learned FSMap version 42 2024-08-30T09:11:54.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.869+0000 7f7bc9982740 1 -- 172.21.15.42:0/1289690257 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55e7fff9db10 con 0x55e7ffb3c670 2024-08-30T09:11:54.872 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.870+0000 7f7ba4ff9640 1 -- 172.21.15.42:0/1289690257 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55e7ffe76fe0 con 0x55e7ffb3c670 2024-08-30T09:11:54.872 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.870+0000 7f7ba4ff9640 1 -- 172.21.15.42:0/1289690257 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 42) ==== 8358+0+0 (secure 0 0 0) 0x7f7bac064050 con 0x55e7ffb3c670 2024-08-30T09:11:54.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.870+0000 7f7bc9982740 10 client.17274 fetch_fsmap finished waiting for FSMap version 42 2024-08-30T09:11:54.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.870+0000 7f7bc9982740 10 client.17274 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-30T09:11:54.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.870+0000 7f7bc9982740 20 client.17274 populate_metadata read hostname 'smithi042' 2024-08-30T09:11:54.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.870+0000 7f7bc9982740 1 --2- 172.21.15.42:0/1289690257 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55e7ffaf1080 0x55e7fffa0180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:54.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.870+0000 7f7bc9982740 4 client.17274 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-08-30T09:11:54.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.870+0000 7f7bc9982740 1 -- 172.21.15.42:0/1289690257 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55e7ffe74720 con 0x55e7ffaf1080 2024-08-30T09:11:54.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.872+0000 7f7bc09a8640 1 --2- 172.21.15.42:0/1289690257 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55e7ffaf1080 0x55e7fffa0180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:54.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.878+0000 7f7ba4ff9640 10 client.17274 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:11:54.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.878+0000 7f7bc09a8640 1 --2- 172.21.15.42:0/1289690257 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55e7ffaf1080 0x55e7fffa0180 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=994ab2b5090cb01a server_cookie=51d0c8e0fd268e8f in_seq=0 out_seq=0 2024-08-30T09:11:54.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.884+0000 7f7ba4ff9640 1 -- 172.21.15.42:0/1289690257 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55e7ffe74720 con 0x55e7ffaf1080 2024-08-30T09:11:54.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.884+0000 7f7ba4ff9640 10 client.17274 handle_command_reply: tid=0 2024-08-30T09:11:54.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.885+0000 7f7bc9982740 1 client.17274 shutdown 2024-08-30T09:11:54.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.889+0000 7f7bc9982740 1 -- 172.21.15.42:0/1289690257 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e7ffb3c670 msgr2=0x55e7fff345a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:11:54.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.889+0000 7f7bc9982740 1 --2- 172.21.15.42:0/1289690257 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e7ffb3c670 0x55e7fff345a0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f7bac004650 tx=0x7f7bac009640 comp rx=0 tx=0).stop 2024-08-30T09:11:54.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.889+0000 7f7bc9982740 1 -- 172.21.15.42:0/1289690257 shutdown_connections 2024-08-30T09:11:54.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.889+0000 7f7bc9982740 1 --2- 172.21.15.42:0/1289690257 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55e7ffaf1080 0x55e7fffa0180 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:54.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.889+0000 7f7bc9982740 1 --2- 172.21.15.42:0/1289690257 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e7ffb5d350 0x55e7fff34ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:54.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.889+0000 7f7bc9982740 1 --2- 172.21.15.42:0/1289690257 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e7ffb3c670 0x55e7fff345a0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:54.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.889+0000 7f7bc9982740 1 --2- 172.21.15.42:0/1289690257 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e7ffb34490 0x55e7fff34060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:54.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.889+0000 7f7ba4ff9640 0 client.17274 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:11:54.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.889+0000 7f7bc9982740 1 -- 172.21.15.42:0/1289690257 >> 172.21.15.42:0/1289690257 conn(0x55e7ffb51740 msgr2=0x55e7fff102d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:11:54.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.889+0000 7f7bc9982740 1 -- 172.21.15.42:0/1289690257 shutdown_connections 2024-08-30T09:11:54.893 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-08-30T09:11:54.893 INFO:teuthology.orchestra.run.smithi042.stdout: "return_code": 0, 2024-08-30T09:11:54.893 INFO:teuthology.orchestra.run.smithi042.stdout: "scrub_tag": "ff464e7f-123b-4bc6-bc7e-aced018168de", 2024-08-30T09:11:54.893 INFO:teuthology.orchestra.run.smithi042.stdout: "mode": "asynchronous" 2024-08-30T09:11:54.893 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-08-30T09:11:54.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.889+0000 7f7bc9982740 1 -- 172.21.15.42:0/1289690257 wait complete. 2024-08-30T09:11:54.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.890+0000 7f7bc9982740 20 client.17274 trim_cache size 0 max 16384 2024-08-30T09:11:54.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.890+0000 7f7bc340d640 1 -- 172.21.15.42:0/3959001230 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f7ba80b4cc0 msgr2=0x7f7ba80b71d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:11:54.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.890+0000 7f7bc340d640 1 --2- 172.21.15.42:0/3959001230 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f7ba80b4cc0 0x7f7ba80b71d0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f7bb40abaa0 tx=0x7f7bb4012040 comp rx=0 tx=0).stop 2024-08-30T09:11:54.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.890+0000 7f7bc340d640 1 -- 172.21.15.42:0/3959001230 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7bbc109bb0 msgr2=0x7f7bbc19fc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:11:54.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.890+0000 7f7bc340d640 1 --2- 172.21.15.42:0/3959001230 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7bbc109bb0 0x7f7bbc19fc10 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f7bb400ff40 tx=0x7f7bb400ff70 comp rx=0 tx=0).stop 2024-08-30T09:11:54.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.891+0000 7f7bc340d640 1 -- 172.21.15.42:0/3959001230 shutdown_connections 2024-08-30T09:11:54.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.891+0000 7f7bc340d640 1 --2- 172.21.15.42:0/3959001230 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f7ba80b4cc0 0x7f7ba80b71d0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:54.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.891+0000 7f7bc340d640 1 --2- 172.21.15.42:0/3959001230 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7bbc109260 0x7f7bbc19a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:54.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.891+0000 7f7bc340d640 1 --2- 172.21.15.42:0/3959001230 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7bbc109bb0 0x7f7bbc19fc10 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:54.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.891+0000 7f7bc340d640 1 --2- 172.21.15.42:0/3959001230 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7bbc103920 0x7f7bbc19a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:54.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.891+0000 7f7bc340d640 1 -- 172.21.15.42:0/3959001230 >> 172.21.15.42:0/3959001230 conn(0x7f7bbc0ff3b0 msgr2=0x7f7bbc1017d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:11:54.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.891+0000 7f7bc340d640 1 -- 172.21.15.42:0/3959001230 shutdown_connections 2024-08-30T09:11:54.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:54.891+0000 7f7bc340d640 1 -- 172.21.15.42:0/3959001230 wait complete. 2024-08-30T09:11:55.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:11:54 smithi042 ceph-mon[18652]: pgmap v707: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 6.6 MiB/s wr, 510 op/s 2024-08-30T09:11:55.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:11:54 smithi190 ceph-mon[22001]: pgmap v707: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 6.6 MiB/s wr, 510 op/s 2024-08-30T09:11:55.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:11:54 smithi167 ceph-mon[22063]: pgmap v707: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 6.6 MiB/s wr, 510 op/s 2024-08-30T09:11:55.327 INFO:journalctl@ceph.mds.b.smithi167.stdout:Aug 30 09:11:54 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mds-b[52473]: 2024-08-30T09:11:54.941+0000 7f9d3fee4640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000000f [...2,head] /volumes/qa/sv_1/d3168875-d2e5-410d-8754-72d29955777b/postgres/data/pg_subtrans/ auth{1=1} v2497 ap=2 f(v11 m2024-08-30T09:10:39.595179+0000 3=3+0) n(v31 rc2024-08-30T09:10:39.598179+0000 b638976 4=3+1) (inest sync->mix g=1) caps={15402=pAsLsXsFs/pAsLsXsFsxcral@28} mcw={1=AsLsXsFs} | dirtyscattered=0 request=0 lock=0 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x555a32f6d080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x1000000000f:[<0x10000000006/pg_subtrans v284>,<0x10000000005/data v125>,<0x100000001fb/postgres v31>,<0x100000001fa/d3168875-d2e5-410d-8754-72d29955777b v45>,<0x10000000001/sv_1 v63>,<0x10000000000/qa v49>,<0x1/volumes v44>]//[]","memoryvalue":"(2)0x1000000000f:[<0x10000000006/pg_subtrans v2497>,<0x10000000005/data v905>,<0x100000001fb/postgres v560>,<0x100000001fa/d3168875-d2e5-410d-8754-72d29955777b v583>,<0x10000000001/sv_1 v604>,<0x10000000000/qa v579>,<0x1/volumes v545>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-08-30T09:09:44.086585+0000 2=2+0)","ondisk_value.rstat":"n(v0 rc2024-08-30T09:10:39.598179+0000 b638976 4=3+1)","memory_value.dirstat":"f(v11 m2024-08-30T09:10:39.595179+0000 3=3+0)","memory_value.rstat":"n(v31 rc2024-08-30T09:10:39.598179+0000 b638976 4=3+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-08-30T09:11:55.602 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-30T09:11:55.953 INFO:teuthology.orchestra.run.smithi042.stderr:progress: 670.0 s, 559.6 tps, lat 181.828 ms stddev 211.874 2024-08-30T09:11:55.996 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T09:11:56.184 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:11:55 smithi042 ceph-mon[18652]: scrub queued for path: / 2024-08-30T09:11:56.184 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:11:55 smithi042 ceph-mon[18652]: scrub summary: idle+waiting paths [/] 2024-08-30T09:11:56.184 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:11:55 smithi042 ceph-mon[18652]: scrub summary: active paths [/] 2024-08-30T09:11:56.184 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:11:55 smithi042 ceph-mon[18652]: scrub summary: active paths [/] 2024-08-30T09:11:56.184 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:11:55 smithi042 ceph-mon[18652]: scrub summary: active paths [/] 2024-08-30T09:11:56.184 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:11:55 smithi042 ceph-mon[18652]: scrub summary: idle+waiting paths [/] 2024-08-30T09:11:56.184 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:11:55 smithi042 ceph-mon[18652]: scrub summary: active paths [/] 2024-08-30T09:11:56.184 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:11:55 smithi042 ceph-mon[18652]: scrub summary: idle+waiting paths [/] 2024-08-30T09:11:56.184 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:11:55 smithi042 ceph-mon[18652]: scrub summary: active paths [/] 2024-08-30T09:11:56.184 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:11:55 smithi042 ceph-mon[18652]: Scrub error on inode 0x1000000000f (/volumes/qa/sv_1/d3168875-d2e5-410d-8754-72d29955777b/postgres/data/pg_subtrans) see mds.b log and `damage ls` output for details 2024-08-30T09:11:56.184 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:11:55 smithi042 ceph-mon[18652]: scrub summary: idle+waiting paths [/] 2024-08-30T09:11:56.184 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:11:55 smithi042 ceph-mon[18652]: scrub summary: idle+waiting paths [/] 2024-08-30T09:11:56.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:11:55 smithi190 ceph-mon[22001]: scrub queued for path: / 2024-08-30T09:11:56.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:11:55 smithi190 ceph-mon[22001]: scrub summary: idle+waiting paths [/] 2024-08-30T09:11:56.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:11:55 smithi190 ceph-mon[22001]: scrub summary: active paths [/] 2024-08-30T09:11:56.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:11:55 smithi190 ceph-mon[22001]: scrub summary: active paths [/] 2024-08-30T09:11:56.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:11:55 smithi190 ceph-mon[22001]: scrub summary: active paths [/] 2024-08-30T09:11:56.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:11:55 smithi190 ceph-mon[22001]: scrub summary: idle+waiting paths [/] 2024-08-30T09:11:56.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:11:55 smithi190 ceph-mon[22001]: scrub summary: active paths [/] 2024-08-30T09:11:56.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:11:55 smithi190 ceph-mon[22001]: scrub summary: idle+waiting paths [/] 2024-08-30T09:11:56.187 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:11:55 smithi190 ceph-mon[22001]: scrub summary: active paths [/] 2024-08-30T09:11:56.187 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:11:55 smithi190 ceph-mon[22001]: Scrub error on inode 0x1000000000f (/volumes/qa/sv_1/d3168875-d2e5-410d-8754-72d29955777b/postgres/data/pg_subtrans) see mds.b log and `damage ls` output for details 2024-08-30T09:11:56.187 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:11:55 smithi190 ceph-mon[22001]: scrub summary: idle+waiting paths [/] 2024-08-30T09:11:56.187 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:11:55 smithi190 ceph-mon[22001]: scrub summary: idle+waiting paths [/] 2024-08-30T09:11:56.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:11:55 smithi167 ceph-mon[22063]: scrub queued for path: / 2024-08-30T09:11:56.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:11:55 smithi167 ceph-mon[22063]: scrub summary: idle+waiting paths [/] 2024-08-30T09:11:56.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:11:55 smithi167 ceph-mon[22063]: scrub summary: active paths [/] 2024-08-30T09:11:56.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:11:55 smithi167 ceph-mon[22063]: scrub summary: active paths [/] 2024-08-30T09:11:56.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:11:55 smithi167 ceph-mon[22063]: scrub summary: active paths [/] 2024-08-30T09:11:56.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:11:55 smithi167 ceph-mon[22063]: scrub summary: idle+waiting paths [/] 2024-08-30T09:11:56.328 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:11:55 smithi167 ceph-mon[22063]: scrub summary: active paths [/] 2024-08-30T09:11:56.328 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:11:55 smithi167 ceph-mon[22063]: scrub summary: idle+waiting paths [/] 2024-08-30T09:11:56.328 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:11:55 smithi167 ceph-mon[22063]: scrub summary: active paths [/] 2024-08-30T09:11:56.328 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:11:55 smithi167 ceph-mon[22063]: Scrub error on inode 0x1000000000f (/volumes/qa/sv_1/d3168875-d2e5-410d-8754-72d29955777b/postgres/data/pg_subtrans) see mds.b log and `damage ls` output for details 2024-08-30T09:11:56.328 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:11:55 smithi167 ceph-mon[22063]: scrub summary: idle+waiting paths [/] 2024-08-30T09:11:56.328 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:11:55 smithi167 ceph-mon[22063]: scrub summary: idle+waiting paths [/] 2024-08-30T09:11:56.459 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.456+0000 7fd318139640 1 -- 172.21.15.42:0/177533236 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd310106660 msgr2=0x7fd310108a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:11:56.459 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.456+0000 7fd318139640 1 --2- 172.21.15.42:0/177533236 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd310106660 0x7fd310108a80 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fd300009900 tx=0x7fd30002a400 comp rx=0 tx=0).stop 2024-08-30T09:11:56.459 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.456+0000 7fd318139640 1 -- 172.21.15.42:0/177533236 shutdown_connections 2024-08-30T09:11:56.459 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.457+0000 7fd318139640 1 --2- 172.21.15.42:0/177533236 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd310109170 0x7fd31010b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:56.459 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.457+0000 7fd318139640 1 --2- 172.21.15.42:0/177533236 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd310106660 0x7fd310108a80 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:56.459 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.457+0000 7fd318139640 1 --2- 172.21.15.42:0/177533236 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd31006b410 0x7fd310106120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:56.459 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.457+0000 7fd318139640 1 -- 172.21.15.42:0/177533236 >> 172.21.15.42:0/177533236 conn(0x7fd3100fdc80 msgr2=0x7fd310100130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:11:56.459 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.457+0000 7fd318139640 1 -- 172.21.15.42:0/177533236 shutdown_connections 2024-08-30T09:11:56.459 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.457+0000 7fd318139640 1 -- 172.21.15.42:0/177533236 wait complete. 2024-08-30T09:11:56.460 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.458+0000 7fd318139640 1 Processor -- start 2024-08-30T09:11:56.460 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.458+0000 7fd318139640 1 -- start start 2024-08-30T09:11:56.460 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.458+0000 7fd318139640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd31006b410 0x7fd31019bd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:56.460 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.458+0000 7fd318139640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd310106660 0x7fd31019c400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:56.460 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.458+0000 7fd318139640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd310109170 0x7fd31019cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:56.462 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.459+0000 7fd318139640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fd310111360 con 0x7fd310109170 2024-08-30T09:11:56.462 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.459+0000 7fd318139640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd3101111e0 con 0x7fd31006b410 2024-08-30T09:11:56.462 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.459+0000 7fd318139640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fd3101114e0 con 0x7fd310106660 2024-08-30T09:11:56.462 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.459+0000 7fd315ed5640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd31006b410 0x7fd31019bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:56.462 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.459+0000 7fd3156d4640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd310106660 0x7fd31019c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:56.462 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.459+0000 7fd315ed5640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd31006b410 0x7fd31019bd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:38174/0 (socket says 172.21.15.42:38174) 2024-08-30T09:11:56.462 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.459+0000 7fd315ed5640 1 -- 172.21.15.42:0/2598627578 learned_addr learned my addr 172.21.15.42:0/2598627578 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:11:56.462 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.459+0000 7fd3166d6640 1 --2- 172.21.15.42:0/2598627578 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd310109170 0x7fd31019cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:56.462 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.459+0000 7fd315ed5640 1 -- 172.21.15.42:0/2598627578 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd310106660 msgr2=0x7fd31019c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:11:56.463 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.459+0000 7fd315ed5640 1 --2- 172.21.15.42:0/2598627578 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd310106660 0x7fd31019c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:56.463 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.459+0000 7fd315ed5640 1 -- 172.21.15.42:0/2598627578 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd310109170 msgr2=0x7fd31019cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:11:56.463 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.459+0000 7fd315ed5640 1 --2- 172.21.15.42:0/2598627578 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd310109170 0x7fd31019cfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:56.463 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.459+0000 7fd315ed5640 1 -- 172.21.15.42:0/2598627578 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd31019d7d0 con 0x7fd31006b410 2024-08-30T09:11:56.463 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.459+0000 7fd3166d6640 1 --2- 172.21.15.42:0/2598627578 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd310109170 0x7fd31019cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-30T09:11:56.463 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.459+0000 7fd315ed5640 1 --2- 172.21.15.42:0/2598627578 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd31006b410 0x7fd31019bd90 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fd30c00b4a0 tx=0x7fd30c00b960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:11:56.463 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.459+0000 7fd306ffd640 1 -- 172.21.15.42:0/2598627578 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd30c014070 con 0x7fd31006b410 2024-08-30T09:11:56.463 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.460+0000 7fd306ffd640 1 -- 172.21.15.42:0/2598627578 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd30c010ac0 con 0x7fd31006b410 2024-08-30T09:11:56.463 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.460+0000 7fd318139640 1 -- 172.21.15.42:0/2598627578 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd31019dac0 con 0x7fd31006b410 2024-08-30T09:11:56.463 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.460+0000 7fd306ffd640 1 -- 172.21.15.42:0/2598627578 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd30c017c30 con 0x7fd31006b410 2024-08-30T09:11:56.463 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.460+0000 7fd318139640 1 -- 172.21.15.42:0/2598627578 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3101a41b0 con 0x7fd31006b410 2024-08-30T09:11:56.466 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.463+0000 7fd306ffd640 1 -- 172.21.15.42:0/2598627578 <== mon.2 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7fd30c017070 con 0x7fd31006b410 2024-08-30T09:11:56.466 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.464+0000 7fd31f0a6740 1 Processor -- start 2024-08-30T09:11:56.467 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.465+0000 7fd306ffd640 1 --2- 172.21.15.42:0/2598627578 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fd2fc0b4bb0 0x7fd2fc0b70c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:56.467 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.465+0000 7fd306ffd640 1 -- 172.21.15.42:0/2598627578 <== mon.2 v2:172.21.15.167:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7fd30c0e4980 con 0x7fd31006b410 2024-08-30T09:11:56.468 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.466+0000 7fd31f0a6740 1 -- start start 2024-08-30T09:11:56.468 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.466+0000 7fd3166d6640 1 --2- 172.21.15.42:0/2598627578 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fd2fc0b4bb0 0x7fd2fc0b70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:56.468 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.466+0000 7fd31f0a6740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f97a23e1b0 0x55f97a5e3f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:56.468 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.466+0000 7fd31f0a6740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f97a2152f0 0x55f97a5e45d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:56.468 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.466+0000 7fd31f0a6740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f97a21d4d0 0x55f97a5e5120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:56.468 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.466+0000 7fd31f0a6740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55f97a557000 con 0x55f97a21d4d0 2024-08-30T09:11:56.468 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.466+0000 7fd31f0a6740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55f97a546590 con 0x55f97a23e1b0 2024-08-30T09:11:56.468 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.466+0000 7fd31f0a6740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55f97a54a540 con 0x55f97a2152f0 2024-08-30T09:11:56.469 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.466+0000 7fd315ed5640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f97a23e1b0 0x55f97a5e3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:56.469 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.466+0000 7fd315ed5640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f97a23e1b0 0x55f97a5e3f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:38186/0 (socket says 172.21.15.42:38186) 2024-08-30T09:11:56.469 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.466+0000 7fd315ed5640 1 -- 172.21.15.42:0/1139200403 learned_addr learned my addr 172.21.15.42:0/1139200403 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:11:56.469 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.466+0000 7fd3156d4640 1 --2- 172.21.15.42:0/1139200403 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f97a2152f0 0x55f97a5e45d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:56.469 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.467+0000 7fd3166d6640 1 --2- 172.21.15.42:0/1139200403 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f97a21d4d0 0x55f97a5e5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:56.469 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.467+0000 7fd315ed5640 1 -- 172.21.15.42:0/1139200403 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f97a2152f0 msgr2=0x55f97a5e45d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:11:56.469 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.467+0000 7fd315ed5640 1 --2- 172.21.15.42:0/1139200403 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f97a2152f0 0x55f97a5e45d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:56.469 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.467+0000 7fd315ed5640 1 -- 172.21.15.42:0/1139200403 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f97a21d4d0 msgr2=0x55f97a5e5120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:11:56.470 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.467+0000 7fd315ed5640 1 --2- 172.21.15.42:0/1139200403 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f97a21d4d0 0x55f97a5e5120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:56.470 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.467+0000 7fd315ed5640 1 -- 172.21.15.42:0/1139200403 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f97a2a9790 con 0x55f97a23e1b0 2024-08-30T09:11:56.470 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.467+0000 7fd3166d6640 1 --2- 172.21.15.42:0/1139200403 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f97a21d4d0 0x55f97a5e5120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-30T09:11:56.470 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.467+0000 7fd315ed5640 1 --2- 172.21.15.42:0/1139200403 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f97a23e1b0 0x55f97a5e3f60 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fd30c0a8340 tx=0x7fd30c0047d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fae300e02f14f5e1 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:11:56.470 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.467+0000 7fd2fbfff640 1 -- 172.21.15.42:0/1139200403 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd30c01e070 con 0x55f97a23e1b0 2024-08-30T09:11:56.470 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.468+0000 7fd2fbfff640 1 -- 172.21.15.42:0/1139200403 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd30c0b4340 con 0x55f97a23e1b0 2024-08-30T09:11:56.470 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.468+0000 7fd2fbfff640 1 -- 172.21.15.42:0/1139200403 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd30c0b4110 con 0x55f97a23e1b0 2024-08-30T09:11:56.471 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.469+0000 7fd31f0a6740 1 -- 172.21.15.42:0/1139200403 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f97a23e1b0 msgr2=0x55f97a5e3f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:11:56.471 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.469+0000 7fd31f0a6740 1 --2- 172.21.15.42:0/1139200403 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f97a23e1b0 0x55f97a5e3f60 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fd30c0a8340 tx=0x7fd30c0047d0 comp rx=0 tx=0).stop 2024-08-30T09:11:56.471 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.469+0000 7fd3166d6640 1 --2- 172.21.15.42:0/2598627578 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fd2fc0b4bb0 0x7fd2fc0b70c0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fd308004fd0 tx=0x7fd308009a70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:11:56.471 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.469+0000 7fd31f0a6740 1 -- 172.21.15.42:0/1139200403 shutdown_connections 2024-08-30T09:11:56.471 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.469+0000 7fd31f0a6740 1 --2- 172.21.15.42:0/1139200403 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f97a2152f0 0x55f97a5e45d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:56.472 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.469+0000 7fd31f0a6740 1 --2- 172.21.15.42:0/1139200403 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f97a21d4d0 0x55f97a5e5120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:56.472 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.469+0000 7fd31f0a6740 1 --2- 172.21.15.42:0/1139200403 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f97a23e1b0 0x55f97a5e3f60 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:56.472 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.469+0000 7fd31f0a6740 1 -- 172.21.15.42:0/1139200403 >> 172.21.15.42:0/1139200403 conn(0x55f97a232300 msgr2=0x55f97a5df9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:11:56.472 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.469+0000 7fd31f0a6740 1 -- 172.21.15.42:0/1139200403 shutdown_connections 2024-08-30T09:11:56.472 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.469+0000 7fd31f0a6740 1 -- 172.21.15.42:0/1139200403 wait complete. 2024-08-30T09:11:56.474 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.470+0000 7fd31f0a6740 1 -- start start 2024-08-30T09:11:56.474 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.470+0000 7fd31f0a6740 1 Processor -- start 2024-08-30T09:11:56.474 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.470+0000 7fd31f0a6740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f97a2152f0 0x55f97a5fea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:56.474 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.470+0000 7fd31f0a6740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f97a21d4d0 0x55f97a5ff0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:56.474 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.470+0000 7fd31f0a6740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f97a23e1b0 0x55f97a5ffc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:56.474 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.470+0000 7fd31f0a6740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55f97a47fab0 con 0x55f97a2152f0 2024-08-30T09:11:56.474 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.470+0000 7fd31f0a6740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55f97a4cd680 con 0x55f97a23e1b0 2024-08-30T09:11:56.475 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.470+0000 7fd31f0a6740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55f97a5e5c40 con 0x55f97a21d4d0 2024-08-30T09:11:56.475 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.470+0000 7fd31f0a6740 10 client.0 _ll_register_callbacks cb 0x55f97a255a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:11:56.475 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.471+0000 7fd3166d6640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f97a23e1b0 0x55f97a5ffc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:56.475 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.471+0000 7fd3166d6640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f97a23e1b0 0x55f97a5ffc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:38190/0 (socket says 172.21.15.42:38190) 2024-08-30T09:11:56.475 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.471+0000 7fd3166d6640 1 -- 172.21.15.42:0/615706606 learned_addr learned my addr 172.21.15.42:0/615706606 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:11:56.475 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.471+0000 7fd3156d4640 1 --2- 172.21.15.42:0/615706606 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f97a21d4d0 0x55f97a5ff0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:56.475 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.471+0000 7fd315ed5640 1 --2- 172.21.15.42:0/615706606 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f97a2152f0 0x55f97a5fea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:56.475 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.471+0000 7fd3166d6640 1 -- 172.21.15.42:0/615706606 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f97a21d4d0 msgr2=0x55f97a5ff0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:11:56.475 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.471+0000 7fd3166d6640 1 --2- 172.21.15.42:0/615706606 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f97a21d4d0 0x55f97a5ff0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:56.475 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.471+0000 7fd3166d6640 1 -- 172.21.15.42:0/615706606 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f97a2152f0 msgr2=0x55f97a5fea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:11:56.475 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.471+0000 7fd3166d6640 1 --2- 172.21.15.42:0/615706606 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f97a2152f0 0x55f97a5fea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:56.475 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.471+0000 7fd3166d6640 1 -- 172.21.15.42:0/615706606 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f97a322620 con 0x55f97a23e1b0 2024-08-30T09:11:56.475 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.471+0000 7fd3166d6640 1 --2- 172.21.15.42:0/615706606 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f97a23e1b0 0x55f97a5ffc50 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fd308005bd0 tx=0x7fd308004a90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9ec3ad000619b20b server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:11:56.475 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.471+0000 7fd2f97fa640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-30T09:11:56.476 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.472+0000 7fd2f97fa640 1 -- 172.21.15.42:0/615706606 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd30801e8a0 con 0x55f97a23e1b0 2024-08-30T09:11:56.476 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.472+0000 7fd2f97fa640 1 -- 172.21.15.42:0/615706606 <== mon.2 v2:172.21.15.167:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7fd30801ea40 con 0x55f97a23e1b0 2024-08-30T09:11:56.476 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.472+0000 7fd31f0a6740 1 -- 172.21.15.42:0/615706606 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd30c0b4340 con 0x55f97a23e1b0 2024-08-30T09:11:56.476 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.472+0000 7fd2f97fa640 1 -- 172.21.15.42:0/615706606 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fd30802e050 con 0x55f97a23e1b0 2024-08-30T09:11:56.476 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.472+0000 7fd31f0a6740 10 client.27119 fetch_fsmap learned FSMap version 42 2024-08-30T09:11:56.476 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.472+0000 7fd31f0a6740 1 -- 172.21.15.42:0/615706606 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55f97a608be0 con 0x55f97a23e1b0 2024-08-30T09:11:56.478 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.472+0000 7fd2f97fa640 1 -- 172.21.15.42:0/615706606 <== mon.2 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd30c0b4340 con 0x55f97a23e1b0 2024-08-30T09:11:56.478 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.472+0000 7fd2f97fa640 1 -- 172.21.15.42:0/615706606 <== mon.2 v2:172.21.15.167:3300/0 5 ==== fsmap(e 42) ==== 8358+0+0 (secure 0 0 0) 0x7fd30801d040 con 0x55f97a23e1b0 2024-08-30T09:11:56.478 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.473+0000 7fd31f0a6740 10 client.27119 fetch_fsmap finished waiting for FSMap version 42 2024-08-30T09:11:56.478 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.473+0000 7fd31f0a6740 10 client.27119 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-30T09:11:56.478 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.473+0000 7fd31f0a6740 20 client.27119 populate_metadata read hostname 'smithi042' 2024-08-30T09:11:56.478 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.473+0000 7fd31f0a6740 1 --2- 172.21.15.42:0/615706606 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55f97a1fcd80 0x55f97a60c880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:56.478 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.473+0000 7fd31f0a6740 4 client.27119 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-30T09:11:56.478 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.473+0000 7fd31f0a6740 1 -- 172.21.15.42:0/615706606 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f97a558f50 con 0x55f97a1fcd80 2024-08-30T09:11:56.478 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.473+0000 7fd315ed5640 1 --2- 172.21.15.42:0/615706606 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55f97a1fcd80 0x55f97a60c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:56.478 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.474+0000 7fd2f97fa640 10 client.27119 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:11:56.478 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.474+0000 7fd315ed5640 1 --2- 172.21.15.42:0/615706606 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55f97a1fcd80 0x55f97a60c880 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=64526c822d5b92ba server_cookie=76a9fa33c82cb0ff in_seq=0 out_seq=0 2024-08-30T09:11:56.478 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.474+0000 7fd2f97fa640 1 -- 172.21.15.42:0/615706606 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f97a558f50 con 0x55f97a1fcd80 2024-08-30T09:11:56.479 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.474+0000 7fd2f97fa640 10 client.27119 handle_command_reply: tid=0 2024-08-30T09:11:56.479 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.474+0000 7fd31f0a6740 1 client.27119 shutdown 2024-08-30T09:11:56.479 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.475+0000 7fd31f0a6740 1 -- 172.21.15.42:0/615706606 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f97a23e1b0 msgr2=0x55f97a5ffc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:11:56.479 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.475+0000 7fd31f0a6740 1 --2- 172.21.15.42:0/615706606 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f97a23e1b0 0x55f97a5ffc50 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fd308005bd0 tx=0x7fd308004a90 comp rx=0 tx=0).stop 2024-08-30T09:11:56.479 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.475+0000 7fd31f0a6740 1 -- 172.21.15.42:0/615706606 shutdown_connections 2024-08-30T09:11:56.479 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.475+0000 7fd31f0a6740 1 --2- 172.21.15.42:0/615706606 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55f97a1fcd80 0x55f97a60c880 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:56.479 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.475+0000 7fd31f0a6740 1 --2- 172.21.15.42:0/615706606 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f97a21d4d0 0x55f97a5ff0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:56.479 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.475+0000 7fd31f0a6740 1 --2- 172.21.15.42:0/615706606 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f97a23e1b0 0x55f97a5ffc50 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:56.479 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.475+0000 7fd31f0a6740 1 --2- 172.21.15.42:0/615706606 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f97a2152f0 0x55f97a5fea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:56.479 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.475+0000 7fd31f0a6740 1 -- 172.21.15.42:0/615706606 >> 172.21.15.42:0/615706606 conn(0x55f97a232300 msgr2=0x55f97a5ddc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:11:56.479 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.475+0000 7fd2f97fa640 0 client.27119 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:11:56.479 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.475+0000 7fd31f0a6740 1 -- 172.21.15.42:0/615706606 shutdown_connections 2024-08-30T09:11:56.479 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.475+0000 7fd31f0a6740 1 -- 172.21.15.42:0/615706606 wait complete. 2024-08-30T09:11:56.479 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.475+0000 7fd31f0a6740 20 client.27119 trim_cache size 0 max 16384 2024-08-30T09:11:56.499 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.497+0000 7fd31f0a6740 1 Processor -- start 2024-08-30T09:11:56.499 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.497+0000 7fd31f0a6740 1 -- start start 2024-08-30T09:11:56.499 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.497+0000 7fd31f0a6740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f97a2152f0 0x55f97a60c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:56.500 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.498+0000 7fd31f0a6740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f97a21d4d0 0x55f97a5e4da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:56.500 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.498+0000 7fd31f0a6740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f97a23e1b0 0x55f97a5e0510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:56.500 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.498+0000 7fd31f0a6740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fd3101a8800 con 0x55f97a2152f0 2024-08-30T09:11:56.501 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.498+0000 7fd31f0a6740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd3101a60a0 con 0x55f97a23e1b0 2024-08-30T09:11:56.501 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.498+0000 7fd31f0a6740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fd310074f00 con 0x55f97a21d4d0 2024-08-30T09:11:56.501 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.498+0000 7fd315ed5640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f97a2152f0 0x55f97a60c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:56.501 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.498+0000 7fd315ed5640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f97a2152f0 0x55f97a60c7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:58508/0 (socket says 172.21.15.42:58508) 2024-08-30T09:11:56.501 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.498+0000 7fd315ed5640 1 -- 172.21.15.42:0/1392933182 learned_addr learned my addr 172.21.15.42:0/1392933182 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:11:56.501 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.498+0000 7fd3166d6640 1 --2- 172.21.15.42:0/1392933182 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f97a23e1b0 0x55f97a5e0510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:56.501 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.498+0000 7fd3156d4640 1 --2- 172.21.15.42:0/1392933182 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f97a21d4d0 0x55f97a5e4da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:56.501 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.498+0000 7fd315ed5640 1 -- 172.21.15.42:0/1392933182 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f97a21d4d0 msgr2=0x55f97a5e4da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:11:56.501 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.498+0000 7fd315ed5640 1 --2- 172.21.15.42:0/1392933182 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f97a21d4d0 0x55f97a5e4da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:56.501 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.498+0000 7fd315ed5640 1 -- 172.21.15.42:0/1392933182 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f97a23e1b0 msgr2=0x55f97a5e0510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:11:56.501 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.498+0000 7fd315ed5640 1 --2- 172.21.15.42:0/1392933182 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f97a23e1b0 0x55f97a5e0510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:56.501 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.498+0000 7fd315ed5640 1 -- 172.21.15.42:0/1392933182 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f97a5e0120 con 0x55f97a2152f0 2024-08-30T09:11:56.501 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.498+0000 7fd315ed5640 1 --2- 172.21.15.42:0/1392933182 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f97a2152f0 0x55f97a60c7d0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fd30c00be60 tx=0x7fd30c012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fe00395f8977fc4 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:11:56.501 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.499+0000 7fd2fb7fe640 1 -- 172.21.15.42:0/1392933182 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd30c08f070 con 0x55f97a2152f0 2024-08-30T09:11:56.501 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.499+0000 7fd2fb7fe640 1 -- 172.21.15.42:0/1392933182 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd30c0b4340 con 0x55f97a2152f0 2024-08-30T09:11:56.502 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.499+0000 7fd2fb7fe640 1 -- 172.21.15.42:0/1392933182 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd30c01e040 con 0x55f97a2152f0 2024-08-30T09:11:56.502 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.500+0000 7fd31f0a6740 1 -- 172.21.15.42:0/1392933182 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f97a2152f0 msgr2=0x55f97a60c7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:11:56.502 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.500+0000 7fd31f0a6740 1 --2- 172.21.15.42:0/1392933182 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f97a2152f0 0x55f97a60c7d0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fd30c00be60 tx=0x7fd30c012000 comp rx=0 tx=0).stop 2024-08-30T09:11:56.502 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.500+0000 7fd31f0a6740 1 -- 172.21.15.42:0/1392933182 shutdown_connections 2024-08-30T09:11:56.502 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.500+0000 7fd31f0a6740 1 --2- 172.21.15.42:0/1392933182 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f97a21d4d0 0x55f97a5e4da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:56.503 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.500+0000 7fd31f0a6740 1 --2- 172.21.15.42:0/1392933182 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f97a23e1b0 0x55f97a5e0510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:56.503 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.500+0000 7fd31f0a6740 1 --2- 172.21.15.42:0/1392933182 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f97a2152f0 0x55f97a60c7d0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:56.503 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.500+0000 7fd31f0a6740 1 -- 172.21.15.42:0/1392933182 >> 172.21.15.42:0/1392933182 conn(0x55f97a232300 msgr2=0x55f97a5e3b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:11:56.503 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.501+0000 7fd31f0a6740 1 -- 172.21.15.42:0/1392933182 shutdown_connections 2024-08-30T09:11:56.503 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.502+0000 7fd31f0a6740 1 -- 172.21.15.42:0/1392933182 wait complete. 2024-08-30T09:11:56.504 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.502+0000 7fd31f0a6740 1 -- start start 2024-08-30T09:11:56.504 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.503+0000 7fd31f0a6740 1 Processor -- start 2024-08-30T09:11:56.505 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.503+0000 7fd31f0a6740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f97a2152f0 0x55f97a6065d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:56.506 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.504+0000 7fd31f0a6740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f97a21d4d0 0x55f97a602510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:56.506 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.504+0000 7fd31f0a6740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f97a23e1b0 0x55f97a602a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:56.506 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.504+0000 7fd31f0a6740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fd3101a4560 con 0x55f97a2152f0 2024-08-30T09:11:56.506 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.504+0000 7fd3156d4640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f97a21d4d0 0x55f97a602510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:56.506 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.504+0000 7fd3156d4640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f97a21d4d0 0x55f97a602510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:38212/0 (socket says 172.21.15.42:38212) 2024-08-30T09:11:56.506 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.504+0000 7fd3156d4640 1 -- 172.21.15.42:0/2313438821 learned_addr learned my addr 172.21.15.42:0/2313438821 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:11:56.506 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.504+0000 7fd3166d6640 1 --2- 172.21.15.42:0/2313438821 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f97a23e1b0 0x55f97a602a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:56.506 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.504+0000 7fd31f0a6740 1 -- 172.21.15.42:0/2313438821 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd3101a59d0 con 0x55f97a21d4d0 2024-08-30T09:11:56.506 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.504+0000 7fd31f0a6740 1 -- 172.21.15.42:0/2313438821 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fd3101a4c30 con 0x55f97a23e1b0 2024-08-30T09:11:56.507 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.505+0000 7fd31f0a6740 10 client.0 _ll_register_callbacks cb 0x7fd3080098e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:11:56.507 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.505+0000 7fd3156d4640 1 -- 172.21.15.42:0/2313438821 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f97a23e1b0 msgr2=0x55f97a602a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:11:56.507 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.505+0000 7fd3156d4640 1 --2- 172.21.15.42:0/2313438821 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f97a23e1b0 0x55f97a602a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:56.507 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.505+0000 7fd3156d4640 1 -- 172.21.15.42:0/2313438821 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f97a2152f0 msgr2=0x55f97a6065d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:11:56.507 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.505+0000 7fd3156d4640 1 --2- 172.21.15.42:0/2313438821 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f97a2152f0 0x55f97a6065d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:56.507 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.505+0000 7fd3156d4640 1 -- 172.21.15.42:0/2313438821 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f97a602f90 con 0x55f97a21d4d0 2024-08-30T09:11:56.508 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.505+0000 7fd3156d4640 1 --2- 172.21.15.42:0/2313438821 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f97a21d4d0 0x55f97a602510 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fd300009900 tx=0x7fd3000048f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=180cc203faf87b7a server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:11:56.508 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.505+0000 7fd2f97fa640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-30T09:11:56.508 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.505+0000 7fd31f0a6740 1 -- 172.21.15.42:0/2313438821 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd30c0b4340 con 0x55f97a21d4d0 2024-08-30T09:11:56.508 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.506+0000 7fd2f97fa640 1 -- 172.21.15.42:0/2313438821 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd300035040 con 0x55f97a21d4d0 2024-08-30T09:11:56.508 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.506+0000 7fd2f97fa640 1 -- 172.21.15.42:0/2313438821 <== mon.2 v2:172.21.15.167:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7fd300004030 con 0x55f97a21d4d0 2024-08-30T09:11:56.508 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.506+0000 7fd2f97fa640 1 -- 172.21.15.42:0/2313438821 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fd300005620 con 0x55f97a21d4d0 2024-08-30T09:11:56.509 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.507+0000 7fd31f0a6740 10 client.27125 fetch_fsmap learned FSMap version 42 2024-08-30T09:11:56.509 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.507+0000 7fd31f0a6740 1 -- 172.21.15.42:0/2313438821 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({fsmap=42}) -- 0x55f97a677280 con 0x55f97a21d4d0 2024-08-30T09:11:56.510 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.508+0000 7fd2f97fa640 1 -- 172.21.15.42:0/2313438821 <== mon.2 v2:172.21.15.167:3300/0 4 ==== fsmap(e 42) ==== 8358+0+0 (secure 0 0 0) 0x7fd300004230 con 0x55f97a21d4d0 2024-08-30T09:11:56.510 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.508+0000 7fd31f0a6740 10 client.27125 fetch_fsmap finished waiting for FSMap version 42 2024-08-30T09:11:56.510 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.508+0000 7fd31f0a6740 10 client.27125 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-30T09:11:56.510 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.508+0000 7fd31f0a6740 20 client.27125 populate_metadata read hostname 'smithi042' 2024-08-30T09:11:56.510 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.508+0000 7fd31f0a6740 1 --2- 172.21.15.42:0/2313438821 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55f97a1d21c0 0x55f97a679aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:56.511 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.508+0000 7fd31f0a6740 4 client.27125 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-30T09:11:56.511 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.508+0000 7fd31f0a6740 1 -- 172.21.15.42:0/2313438821 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f97a5549f0 con 0x55f97a1d21c0 2024-08-30T09:11:56.511 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.509+0000 7fd315ed5640 1 --2- 172.21.15.42:0/2313438821 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55f97a1d21c0 0x55f97a679aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:56.511 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.510+0000 7fd315ed5640 1 --2- 172.21.15.42:0/2313438821 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55f97a1d21c0 0x55f97a679aa0 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2c6a349aaf9ef504 server_cookie=b0d0f9180be20b7f in_seq=0 out_seq=0 2024-08-30T09:11:56.511 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.510+0000 7fd2f97fa640 10 client.27125 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:11:56.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.678+0000 7f1738075640 1 -- 172.21.15.42:0/1976482877 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f17301023d0 msgr2=0x7f1730105f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:11:56.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.678+0000 7f1738075640 1 --2- 172.21.15.42:0/1976482877 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f17301023d0 0x7f1730105f30 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f172801acb0 tx=0x7f172803b900 comp rx=0 tx=0).stop 2024-08-30T09:11:56.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.682+0000 7f1738075640 1 -- 172.21.15.42:0/1976482877 shutdown_connections 2024-08-30T09:11:56.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.682+0000 7f1738075640 1 --2- 172.21.15.42:0/1976482877 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f17301023d0 0x7f1730105f30 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:56.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.682+0000 7f1738075640 1 --2- 172.21.15.42:0/1976482877 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1730101a80 0x7f1730101e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:56.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.682+0000 7f1738075640 1 --2- 172.21.15.42:0/1976482877 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1730105190 0x7f17301055a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:56.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.682+0000 7f1738075640 1 -- 172.21.15.42:0/1976482877 >> 172.21.15.42:0/1976482877 conn(0x7f1730077a00 msgr2=0x7f1730077e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:11:56.685 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.683+0000 7f1738075640 1 -- 172.21.15.42:0/1976482877 shutdown_connections 2024-08-30T09:11:56.687 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.685+0000 7f1738075640 1 -- 172.21.15.42:0/1976482877 wait complete. 2024-08-30T09:11:56.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.686+0000 7f1738075640 1 Processor -- start 2024-08-30T09:11:56.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.687+0000 7f1738075640 1 -- start start 2024-08-30T09:11:56.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.688+0000 7f1738075640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1730101a80 0x7f17300740b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:56.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.688+0000 7f1738075640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f17301023d0 0x7f1730074650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:56.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.688+0000 7f1738075640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1730105190 0x7f17301a6000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:56.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.688+0000 7f1738075640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f173010b5e0 con 0x7f1730101a80 2024-08-30T09:11:56.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.688+0000 7f1738075640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f173010b460 con 0x7f17301023d0 2024-08-30T09:11:56.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.688+0000 7f1738075640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f173010b760 con 0x7f1730105190 2024-08-30T09:11:56.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.688+0000 7f1736612640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1730105190 0x7f17301a6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:56.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.688+0000 7f1736612640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1730105190 0x7f17301a6000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:52812/0 (socket says 172.21.15.42:52812) 2024-08-30T09:11:56.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.688+0000 7f1736612640 1 -- 172.21.15.42:0/2534541155 learned_addr learned my addr 172.21.15.42:0/2534541155 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:11:56.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.689+0000 7f1735610640 1 --2- 172.21.15.42:0/2534541155 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f17301023d0 0x7f1730074650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:56.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.692+0000 7f1735e11640 1 --2- 172.21.15.42:0/2534541155 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1730101a80 0x7f17300740b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:56.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.692+0000 7f1735610640 1 -- 172.21.15.42:0/2534541155 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1730105190 msgr2=0x7f17301a6000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:11:56.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.692+0000 7f1735610640 1 --2- 172.21.15.42:0/2534541155 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1730105190 0x7f17301a6000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:56.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.692+0000 7f1735610640 1 -- 172.21.15.42:0/2534541155 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1730101a80 msgr2=0x7f17300740b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:11:56.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.692+0000 7f1735610640 1 --2- 172.21.15.42:0/2534541155 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1730101a80 0x7f17300740b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:56.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.692+0000 7f1735610640 1 -- 172.21.15.42:0/2534541155 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17301a6730 con 0x7f17301023d0 2024-08-30T09:11:56.695 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.692+0000 7fd2f97fa640 1 -- 172.21.15.42:0/2313438821 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f97a5549f0 con 0x55f97a1d21c0 2024-08-30T09:11:56.695 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.692+0000 7fd2f97fa640 10 client.27125 handle_command_reply: tid=0 2024-08-30T09:11:56.695 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.693+0000 7fd31f0a6740 1 client.27125 shutdown 2024-08-30T09:11:56.695 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.693+0000 7fd31f0a6740 1 -- 172.21.15.42:0/2313438821 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f97a21d4d0 msgr2=0x55f97a602510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:11:56.695 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.693+0000 7fd31f0a6740 1 --2- 172.21.15.42:0/2313438821 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f97a21d4d0 0x55f97a602510 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fd300009900 tx=0x7fd3000048f0 comp rx=0 tx=0).stop 2024-08-30T09:11:56.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.693+0000 7f1735610640 1 --2- 172.21.15.42:0/2534541155 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f17301023d0 0x7f1730074650 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f172000d6a0 tx=0x7f172000db60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:11:56.697 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.693+0000 7fd31f0a6740 1 -- 172.21.15.42:0/2313438821 shutdown_connections 2024-08-30T09:11:56.697 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.693+0000 7fd31f0a6740 1 --2- 172.21.15.42:0/2313438821 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55f97a1d21c0 0x55f97a679aa0 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:56.697 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.693+0000 7fd31f0a6740 1 --2- 172.21.15.42:0/2313438821 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f97a23e1b0 0x55f97a602a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:56.697 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.693+0000 7fd2f97fa640 0 client.27125 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:11:56.697 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.694+0000 7fd31f0a6740 1 --2- 172.21.15.42:0/2313438821 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f97a21d4d0 0x55f97a602510 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:56.697 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.694+0000 7fd31f0a6740 1 --2- 172.21.15.42:0/2313438821 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f97a2152f0 0x55f97a6065d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:56.697 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.694+0000 7fd31f0a6740 1 -- 172.21.15.42:0/2313438821 >> 172.21.15.42:0/2313438821 conn(0x55f97a232300 msgr2=0x55f97a5e4da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:11:56.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.693+0000 7f1726ffd640 1 -- 172.21.15.42:0/2534541155 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1720017070 con 0x7f17301023d0 2024-08-30T09:11:56.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.693+0000 7f1738075640 1 -- 172.21.15.42:0/2534541155 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17301a6a20 con 0x7f17301023d0 2024-08-30T09:11:56.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.694+0000 7f1726ffd640 1 -- 172.21.15.42:0/2534541155 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1720009820 con 0x7f17301023d0 2024-08-30T09:11:56.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.694+0000 7f1738075640 1 -- 172.21.15.42:0/2534541155 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17301ad180 con 0x7f17301023d0 2024-08-30T09:11:56.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.694+0000 7f1726ffd640 1 -- 172.21.15.42:0/2534541155 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1720004bc0 con 0x7f17301023d0 2024-08-30T09:11:56.698 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.694+0000 7fd31f0a6740 1 -- 172.21.15.42:0/2313438821 shutdown_connections 2024-08-30T09:11:56.698 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.694+0000 7fd31f0a6740 1 -- 172.21.15.42:0/2313438821 wait complete. 2024-08-30T09:11:56.698 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.694+0000 7fd31f0a6740 20 client.27125 trim_cache size 0 max 16384 2024-08-30T09:11:56.699 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-08-30T09:11:56.699 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2024-08-30T09:11:56.699 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-08-30T09:11:56.699 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-08-30T09:11:56.700 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.697+0000 7fd318139640 1 -- 172.21.15.42:0/2598627578 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fd2fc0b4bb0 msgr2=0x7fd2fc0b70c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:11:56.700 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.697+0000 7fd318139640 1 --2- 172.21.15.42:0/2598627578 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fd2fc0b4bb0 0x7fd2fc0b70c0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fd308004fd0 tx=0x7fd308009a70 comp rx=0 tx=0).stop 2024-08-30T09:11:56.700 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.697+0000 7fd318139640 1 -- 172.21.15.42:0/2598627578 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd31006b410 msgr2=0x7fd31019bd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:11:56.700 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.697+0000 7fd318139640 1 --2- 172.21.15.42:0/2598627578 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd31006b410 0x7fd31019bd90 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fd30c00b4a0 tx=0x7fd30c00b960 comp rx=0 tx=0).stop 2024-08-30T09:11:56.700 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.697+0000 7fd318139640 1 -- 172.21.15.42:0/2598627578 shutdown_connections 2024-08-30T09:11:56.700 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.697+0000 7fd318139640 1 --2- 172.21.15.42:0/2598627578 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fd2fc0b4bb0 0x7fd2fc0b70c0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:56.700 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.697+0000 7fd318139640 1 --2- 172.21.15.42:0/2598627578 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd310106660 0x7fd31019c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:56.700 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.697+0000 7fd318139640 1 --2- 172.21.15.42:0/2598627578 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd310109170 0x7fd31019cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:56.700 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.697+0000 7fd318139640 1 --2- 172.21.15.42:0/2598627578 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd31006b410 0x7fd31019bd90 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:56.700 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.697+0000 7fd318139640 1 -- 172.21.15.42:0/2598627578 >> 172.21.15.42:0/2598627578 conn(0x7fd3100fdc80 msgr2=0x7fd31010a030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:11:56.701 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.697+0000 7fd318139640 1 -- 172.21.15.42:0/2598627578 shutdown_connections 2024-08-30T09:11:56.701 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:11:56.697+0000 7fd318139640 1 -- 172.21.15.42:0/2598627578 wait complete. 2024-08-30T09:11:56.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.701+0000 7f173e5ea740 1 Processor -- start 2024-08-30T09:11:56.703 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.702+0000 7f173e5ea740 1 -- start start 2024-08-30T09:11:56.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.707+0000 7f173e5ea740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fa20654070 0x55fa209f92c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:56.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.707+0000 7f173e5ea740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fa2062b1b0 0x55fa209f9800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:56.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.707+0000 7f173e5ea740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fa20633390 0x55fa209f9d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:56.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.707+0000 7f173e5ea740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55fa20970630 con 0x55fa20654070 2024-08-30T09:11:56.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.707+0000 7f173e5ea740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55fa2095fbc0 con 0x55fa20633390 2024-08-30T09:11:56.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.707+0000 7f173e5ea740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55fa20963b70 con 0x55fa2062b1b0 2024-08-30T09:11:56.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.708+0000 7f1736612640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fa20654070 0x55fa209f92c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:56.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.708+0000 7f1736612640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fa20654070 0x55fa209f92c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:58556/0 (socket says 172.21.15.42:58556) 2024-08-30T09:11:56.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.708+0000 7f1736612640 1 -- 172.21.15.42:0/2868935131 learned_addr learned my addr 172.21.15.42:0/2868935131 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:11:56.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.708+0000 7f1735e11640 1 --2- 172.21.15.42:0/2868935131 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fa2062b1b0 0x55fa209f9800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:56.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.710+0000 7f1736612640 1 -- 172.21.15.42:0/2868935131 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fa2062b1b0 msgr2=0x55fa209f9800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:11:56.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.710+0000 7f1736612640 1 --2- 172.21.15.42:0/2868935131 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fa2062b1b0 0x55fa209f9800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:56.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.710+0000 7f1736612640 1 -- 172.21.15.42:0/2868935131 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fa20633390 msgr2=0x55fa209f9d40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-30T09:11:56.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.710+0000 7f1736612640 1 --2- 172.21.15.42:0/2868935131 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fa20633390 0x55fa209f9d40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:56.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.710+0000 7f1736612640 1 -- 172.21.15.42:0/2868935131 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa206bf650 con 0x55fa20654070 2024-08-30T09:11:56.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.711+0000 7f1736612640 1 --2- 172.21.15.42:0/2868935131 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fa20654070 0x55fa209f92c0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f17280406c0 tx=0x7f17280406f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11e734137c7b5c27 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:11:56.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.711+0000 7f171bfff640 1 -- 172.21.15.42:0/2868935131 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1728045030 con 0x55fa20654070 2024-08-30T09:11:56.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.711+0000 7f171bfff640 1 -- 172.21.15.42:0/2868935131 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1728041ab0 con 0x55fa20654070 2024-08-30T09:11:56.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.712+0000 7f171bfff640 1 -- 172.21.15.42:0/2868935131 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1728040a40 con 0x55fa20654070 2024-08-30T09:11:56.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.713+0000 7f1726ffd640 1 -- 172.21.15.42:0/2534541155 <== mon.2 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7f1720005220 con 0x7f17301023d0 2024-08-30T09:11:56.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.713+0000 7f173e5ea740 1 -- 172.21.15.42:0/2868935131 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fa20654070 msgr2=0x55fa209f92c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:11:56.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.713+0000 7f173e5ea740 1 --2- 172.21.15.42:0/2868935131 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fa20654070 0x55fa209f92c0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f17280406c0 tx=0x7f17280406f0 comp rx=0 tx=0).stop 2024-08-30T09:11:56.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.714+0000 7f173e5ea740 1 -- 172.21.15.42:0/2868935131 shutdown_connections 2024-08-30T09:11:56.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.714+0000 7f173e5ea740 1 --2- 172.21.15.42:0/2868935131 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fa2062b1b0 0x55fa209f9800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:56.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.714+0000 7f173e5ea740 1 --2- 172.21.15.42:0/2868935131 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fa20633390 0x55fa209f9d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:56.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.714+0000 7f173e5ea740 1 --2- 172.21.15.42:0/2868935131 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fa20654070 0x55fa209f92c0 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:56.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.714+0000 7f173e5ea740 1 -- 172.21.15.42:0/2868935131 >> 172.21.15.42:0/2868935131 conn(0x55fa20648460 msgr2=0x55fa209f71b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:11:56.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.714+0000 7f173e5ea740 1 -- 172.21.15.42:0/2868935131 shutdown_connections 2024-08-30T09:11:56.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.714+0000 7f173e5ea740 1 -- 172.21.15.42:0/2868935131 wait complete. 2024-08-30T09:11:56.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.714+0000 7f1726ffd640 1 --2- 172.21.15.42:0/2534541155 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f171c0b4b80 0x7f171c0b7090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:56.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.714+0000 7f173e5ea740 1 -- start start 2024-08-30T09:11:56.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.714+0000 7f1726ffd640 1 -- 172.21.15.42:0/2534541155 <== mon.2 v2:172.21.15.167:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f17200e4d80 con 0x7f17301023d0 2024-08-30T09:11:56.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.715+0000 7f173e5ea740 1 Processor -- start 2024-08-30T09:11:56.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.716+0000 7f173e5ea740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fa2062b1b0 0x55fa20a04b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:56.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.716+0000 7f173e5ea740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fa20633390 0x55fa20a050a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:56.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.716+0000 7f173e5ea740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fa20654070 0x55fa20a10dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:56.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.716+0000 7f173e5ea740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55fa207c2700 con 0x55fa20633390 2024-08-30T09:11:56.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.716+0000 7f173e5ea740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55fa208e6700 con 0x55fa2062b1b0 2024-08-30T09:11:56.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.716+0000 7f173e5ea740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55fa20a07a40 con 0x55fa20654070 2024-08-30T09:11:56.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.716+0000 7f1736612640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fa20654070 0x55fa20a10dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:56.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.716+0000 7f1736612640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fa20654070 0x55fa20a10dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:52836/0 (socket says 172.21.15.42:52836) 2024-08-30T09:11:56.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.716+0000 7f1736612640 1 -- 172.21.15.42:0/704845254 learned_addr learned my addr 172.21.15.42:0/704845254 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:11:56.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.716+0000 7f173e5ea740 10 client.0 _ll_register_callbacks cb 0x55fa20915dc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:11:56.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.720+0000 7f1735610640 1 --2- 172.21.15.42:0/704845254 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fa20633390 0x55fa20a050a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:56.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.721+0000 7f1735e11640 1 --2- 172.21.15.42:0/704845254 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fa2062b1b0 0x55fa20a04b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:56.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.721+0000 7f1735e11640 1 --2- 172.21.15.42:0/2534541155 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f171c0b4b80 0x7f171c0b7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:56.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.724+0000 7f1736612640 1 -- 172.21.15.42:0/704845254 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fa2062b1b0 msgr2=0x55fa20a04b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:11:56.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.724+0000 7f1736612640 1 --2- 172.21.15.42:0/704845254 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fa2062b1b0 0x55fa20a04b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:56.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.724+0000 7f1736612640 1 -- 172.21.15.42:0/704845254 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fa20633390 msgr2=0x55fa20a050a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:11:56.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.724+0000 7f1736612640 1 --2- 172.21.15.42:0/704845254 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fa20633390 0x55fa20a050a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:56.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.724+0000 7f1736612640 1 -- 172.21.15.42:0/704845254 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa2091f8b0 con 0x55fa20654070 2024-08-30T09:11:56.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.725+0000 7f1735610640 1 --2- 172.21.15.42:0/704845254 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fa20633390 0x55fa20a050a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-30T09:11:56.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.728+0000 7f1736612640 1 --2- 172.21.15.42:0/704845254 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fa20654070 0x55fa20a10dc0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f172803bec0 tx=0x7f1728040a00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cb5d406e00f04ae3 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:11:56.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.730+0000 7f17197fa640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-08-30T09:11:56.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.730+0000 7f1735e11640 1 --2- 172.21.15.42:0/2534541155 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f171c0b4b80 0x7f171c0b7090 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x55fa209f7f40 tx=0x7f172c005d40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:11:56.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.736+0000 7f17197fa640 1 -- 172.21.15.42:0/704845254 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1728045040 con 0x55fa20654070 2024-08-30T09:11:56.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.736+0000 7f17197fa640 1 -- 172.21.15.42:0/704845254 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f1728051070 con 0x55fa20654070 2024-08-30T09:11:56.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.737+0000 7f173e5ea740 1 -- 172.21.15.42:0/704845254 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1728041ab0 con 0x55fa20654070 2024-08-30T09:11:56.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.739+0000 7f17197fa640 1 -- 172.21.15.42:0/704845254 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f1728064050 con 0x55fa20654070 2024-08-30T09:11:56.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.740+0000 7f173e5ea740 10 client.27091 fetch_fsmap learned FSMap version 42 2024-08-30T09:11:56.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.740+0000 7f173e5ea740 1 -- 172.21.15.42:0/704845254 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55fa20a13910 con 0x55fa20654070 2024-08-30T09:11:56.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.745+0000 7f17197fa640 1 -- 172.21.15.42:0/704845254 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f172805a070 con 0x55fa20654070 2024-08-30T09:11:56.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.745+0000 7f17197fa640 1 -- 172.21.15.42:0/704845254 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 42) ==== 8358+0+0 (secure 0 0 0) 0x7f1728004560 con 0x55fa20654070 2024-08-30T09:11:56.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.745+0000 7f173e5ea740 10 client.27091 fetch_fsmap finished waiting for FSMap version 42 2024-08-30T09:11:56.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.745+0000 7f173e5ea740 10 client.27091 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-30T09:11:56.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.745+0000 7f173e5ea740 20 client.27091 populate_metadata read hostname 'smithi042' 2024-08-30T09:11:56.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.745+0000 7f173e5ea740 1 --2- 172.21.15.42:0/704845254 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55fa206129f0 0x55fa20a16c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:56.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.746+0000 7f173e5ea740 4 client.27091 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-30T09:11:56.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.746+0000 7f173e5ea740 1 -- 172.21.15.42:0/704845254 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f1728041ab0 con 0x55fa206129f0 2024-08-30T09:11:56.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.747+0000 7f1735e11640 1 --2- 172.21.15.42:0/704845254 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55fa206129f0 0x55fa20a16c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:56.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.754+0000 7f17197fa640 10 client.27091 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:11:56.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.754+0000 7f1735e11640 1 --2- 172.21.15.42:0/704845254 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55fa206129f0 0x55fa20a16c70 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=499132a110e25799 server_cookie=a1206d15ab343b2e in_seq=0 out_seq=0 2024-08-30T09:11:56.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.757+0000 7f17197fa640 1 -- 172.21.15.42:0/704845254 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f1728041ab0 con 0x55fa206129f0 2024-08-30T09:11:56.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.757+0000 7f17197fa640 10 client.27091 handle_command_reply: tid=0 2024-08-30T09:11:56.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.757+0000 7f173e5ea740 1 client.27091 shutdown 2024-08-30T09:11:56.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.759+0000 7f173e5ea740 1 -- 172.21.15.42:0/704845254 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fa20654070 msgr2=0x55fa20a10dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:11:56.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.759+0000 7f173e5ea740 1 --2- 172.21.15.42:0/704845254 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fa20654070 0x55fa20a10dc0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f172803bec0 tx=0x7f1728040a00 comp rx=0 tx=0).stop 2024-08-30T09:11:56.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.759+0000 7f173e5ea740 1 -- 172.21.15.42:0/704845254 shutdown_connections 2024-08-30T09:11:56.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.759+0000 7f173e5ea740 1 --2- 172.21.15.42:0/704845254 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55fa206129f0 0x55fa20a16c70 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:56.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.759+0000 7f173e5ea740 1 --2- 172.21.15.42:0/704845254 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fa20654070 0x55fa20a10dc0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:56.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.759+0000 7f173e5ea740 1 --2- 172.21.15.42:0/704845254 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fa20633390 0x55fa20a050a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:56.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.759+0000 7f173e5ea740 1 --2- 172.21.15.42:0/704845254 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fa2062b1b0 0x55fa20a04b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:56.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.759+0000 7f173e5ea740 1 -- 172.21.15.42:0/704845254 >> 172.21.15.42:0/704845254 conn(0x55fa20648460 msgr2=0x55fa209fc450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:11:56.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.759+0000 7f17197fa640 0 client.27091 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:11:56.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.763+0000 7f173e5ea740 1 -- 172.21.15.42:0/704845254 shutdown_connections 2024-08-30T09:11:56.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.763+0000 7f173e5ea740 1 -- 172.21.15.42:0/704845254 wait complete. 2024-08-30T09:11:56.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.763+0000 7f173e5ea740 20 client.27091 trim_cache size 0 max 16384 2024-08-30T09:11:56.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.801+0000 7f173e5ea740 1 Processor -- start 2024-08-30T09:11:56.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.802+0000 7f173e5ea740 1 -- start start 2024-08-30T09:11:56.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.803+0000 7f173e5ea740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fa2062b1b0 0x55fa20a002d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:56.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.803+0000 7f173e5ea740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fa20633390 0x55fa20a033a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:56.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.803+0000 7f173e5ea740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fa20654070 0x55fa209fcad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:56.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.803+0000 7f173e5ea740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f17140125d0 con 0x55fa2062b1b0 2024-08-30T09:11:56.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.803+0000 7f173e5ea740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f171400fe70 con 0x55fa20633390 2024-08-30T09:11:56.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.803+0000 7f173e5ea740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f171400dcc0 con 0x55fa20654070 2024-08-30T09:11:56.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.804+0000 7f1735e11640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fa2062b1b0 0x55fa20a002d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:56.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.804+0000 7f1735e11640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fa2062b1b0 0x55fa20a002d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:58574/0 (socket says 172.21.15.42:58574) 2024-08-30T09:11:56.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.804+0000 7f1735e11640 1 -- 172.21.15.42:0/3581649521 learned_addr learned my addr 172.21.15.42:0/3581649521 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:11:56.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.805+0000 7f1735610640 1 --2- 172.21.15.42:0/3581649521 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fa20633390 0x55fa20a033a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:56.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.806+0000 7f1735610640 1 -- 172.21.15.42:0/3581649521 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fa20654070 msgr2=0x55fa209fcad0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-30T09:11:56.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.806+0000 7f1735610640 1 --2- 172.21.15.42:0/3581649521 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fa20654070 0x55fa209fcad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:56.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.806+0000 7f1735610640 1 -- 172.21.15.42:0/3581649521 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fa2062b1b0 msgr2=0x55fa20a002d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:11:56.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.806+0000 7f1735610640 1 --2- 172.21.15.42:0/3581649521 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fa2062b1b0 0x55fa20a002d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:56.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.806+0000 7f1735610640 1 -- 172.21.15.42:0/3581649521 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa209fb5b0 con 0x55fa20633390 2024-08-30T09:11:56.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.807+0000 7f1735610640 1 --2- 172.21.15.42:0/3581649521 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fa20633390 0x55fa20a033a0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f17200e2490 tx=0x7f17200057f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2915f45c9c64a5ae server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:11:56.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.808+0000 7f171b7fe640 1 -- 172.21.15.42:0/3581649521 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1720020040 con 0x55fa20633390 2024-08-30T09:11:56.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.809+0000 7f171b7fe640 1 -- 172.21.15.42:0/3581649521 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f17200ab3c0 con 0x55fa20633390 2024-08-30T09:11:56.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.809+0000 7f171b7fe640 1 -- 172.21.15.42:0/3581649521 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17200ce1d0 con 0x55fa20633390 2024-08-30T09:11:56.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.810+0000 7f173e5ea740 1 -- 172.21.15.42:0/3581649521 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fa20633390 msgr2=0x55fa20a033a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:11:56.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.810+0000 7f173e5ea740 1 --2- 172.21.15.42:0/3581649521 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fa20633390 0x55fa20a033a0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f17200e2490 tx=0x7f17200057f0 comp rx=0 tx=0).stop 2024-08-30T09:11:56.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.813+0000 7f173e5ea740 1 -- 172.21.15.42:0/3581649521 shutdown_connections 2024-08-30T09:11:56.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.813+0000 7f173e5ea740 1 --2- 172.21.15.42:0/3581649521 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fa20654070 0x55fa209fcad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:56.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.813+0000 7f173e5ea740 1 --2- 172.21.15.42:0/3581649521 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fa20633390 0x55fa20a033a0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:56.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.813+0000 7f173e5ea740 1 --2- 172.21.15.42:0/3581649521 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fa2062b1b0 0x55fa20a002d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:56.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.813+0000 7f173e5ea740 1 -- 172.21.15.42:0/3581649521 >> 172.21.15.42:0/3581649521 conn(0x55fa20648460 msgr2=0x55fa209fc3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:11:56.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.817+0000 7f173e5ea740 1 -- 172.21.15.42:0/3581649521 shutdown_connections 2024-08-30T09:11:56.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.820+0000 7f173e5ea740 1 -- 172.21.15.42:0/3581649521 wait complete. 2024-08-30T09:11:56.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.821+0000 7f173e5ea740 1 -- start start 2024-08-30T09:11:56.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.821+0000 7f173e5ea740 1 Processor -- start 2024-08-30T09:11:56.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.827+0000 7f173e5ea740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fa2062b1b0 0x55fa20a18470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:56.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.828+0000 7f173e5ea740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fa20633390 0x55fa20a189b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:56.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.828+0000 7f173e5ea740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fa20654070 0x55fa20a803f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:56.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.828+0000 7f1735e11640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fa2062b1b0 0x55fa20a18470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:56.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.828+0000 7f1735e11640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fa2062b1b0 0x55fa20a18470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:38256/0 (socket says 172.21.15.42:38256) 2024-08-30T09:11:56.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.828+0000 7f1735e11640 1 -- 172.21.15.42:0/2657784954 learned_addr learned my addr 172.21.15.42:0/2657784954 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:11:56.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.828+0000 7f173e5ea740 1 -- 172.21.15.42:0/2657784954 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f171400e330 con 0x55fa20633390 2024-08-30T09:11:56.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.828+0000 7f173e5ea740 1 -- 172.21.15.42:0/2657784954 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f171400f7a0 con 0x55fa2062b1b0 2024-08-30T09:11:56.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.828+0000 7f173e5ea740 1 -- 172.21.15.42:0/2657784954 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f171400ea00 con 0x55fa20654070 2024-08-30T09:11:56.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.828+0000 7f1736612640 1 --2- 172.21.15.42:0/2657784954 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fa20654070 0x55fa20a803f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:56.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.828+0000 7f1735e11640 1 -- 172.21.15.42:0/2657784954 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fa20654070 msgr2=0x55fa20a803f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:11:56.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.829+0000 7f1735e11640 1 --2- 172.21.15.42:0/2657784954 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fa20654070 0x55fa20a803f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:56.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.829+0000 7f1735e11640 1 -- 172.21.15.42:0/2657784954 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fa20633390 msgr2=0x55fa20a189b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-30T09:11:56.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.829+0000 7f1735e11640 1 --2- 172.21.15.42:0/2657784954 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fa20633390 0x55fa20a189b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:56.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.829+0000 7f1735e11640 1 -- 172.21.15.42:0/2657784954 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa20a809e0 con 0x55fa2062b1b0 2024-08-30T09:11:56.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.829+0000 7f1735e11640 1 --2- 172.21.15.42:0/2657784954 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fa2062b1b0 0x55fa20a18470 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f172c010cd0 tx=0x7f172c005c50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dfa608959c66b342 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:11:56.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.829+0000 7f173e5ea740 10 client.0 _ll_register_callbacks cb 0x7f17280405e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:11:56.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.829+0000 7f173e5ea740 1 -- 172.21.15.42:0/2657784954 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f17200ab3c0 con 0x55fa2062b1b0 2024-08-30T09:11:56.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.831+0000 7f17197fa640 10 client.27140 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-30T09:11:56.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.831+0000 7f17197fa640 1 -- 172.21.15.42:0/2657784954 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f172c004410 con 0x55fa2062b1b0 2024-08-30T09:11:56.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.831+0000 7f17197fa640 1 -- 172.21.15.42:0/2657784954 <== mon.2 v2:172.21.15.167:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f172c0035f0 con 0x55fa2062b1b0 2024-08-30T09:11:56.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.833+0000 7f17197fa640 1 -- 172.21.15.42:0/2657784954 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f172c0037f0 con 0x55fa2062b1b0 2024-08-30T09:11:56.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.836+0000 7f173e5ea740 10 client.27140 fetch_fsmap learned FSMap version 42 2024-08-30T09:11:56.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.836+0000 7f173e5ea740 1 -- 172.21.15.42:0/2657784954 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({fsmap=42}) -- 0x55fa20a81b30 con 0x55fa2062b1b0 2024-08-30T09:11:56.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.837+0000 7f17197fa640 1 -- 172.21.15.42:0/2657784954 <== mon.2 v2:172.21.15.167:3300/0 4 ==== fsmap(e 42) ==== 8358+0+0 (secure 0 0 0) 0x7f172c02a070 con 0x55fa2062b1b0 2024-08-30T09:11:56.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.837+0000 7f173e5ea740 10 client.27140 fetch_fsmap finished waiting for FSMap version 42 2024-08-30T09:11:56.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.837+0000 7f173e5ea740 10 client.27140 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-30T09:11:56.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.837+0000 7f173e5ea740 20 client.27140 populate_metadata read hostname 'smithi042' 2024-08-30T09:11:56.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.837+0000 7f173e5ea740 1 --2- 172.21.15.42:0/2657784954 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55fa205e7da0 0x55fa20a841b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:11:56.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.837+0000 7f173e5ea740 4 client.27140 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-30T09:11:56.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.837+0000 7f173e5ea740 1 -- 172.21.15.42:0/2657784954 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f17200ab3c0 con 0x55fa205e7da0 2024-08-30T09:11:56.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.839+0000 7f1735610640 1 --2- 172.21.15.42:0/2657784954 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55fa205e7da0 0x55fa20a841b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:11:56.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.846+0000 7f17197fa640 10 client.27140 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:11:56.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.846+0000 7f1735610640 1 --2- 172.21.15.42:0/2657784954 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55fa205e7da0 0x55fa20a841b0 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=31ace164d3508c73 server_cookie=e4c3a45b3dabd018 in_seq=0 out_seq=0 2024-08-30T09:11:56.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.847+0000 7f17197fa640 1 -- 172.21.15.42:0/2657784954 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+4262 (crc 0 0 0) 0x7f17200ab3c0 con 0x55fa205e7da0 2024-08-30T09:11:56.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.847+0000 7f17197fa640 10 client.27140 handle_command_reply: tid=0 2024-08-30T09:11:56.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.847+0000 7f173e5ea740 1 client.27140 shutdown 2024-08-30T09:11:56.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.847+0000 7f173e5ea740 1 -- 172.21.15.42:0/2657784954 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fa2062b1b0 msgr2=0x55fa20a18470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:11:56.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.847+0000 7f173e5ea740 1 --2- 172.21.15.42:0/2657784954 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fa2062b1b0 0x55fa20a18470 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f172c010cd0 tx=0x7f172c005c50 comp rx=0 tx=0).stop 2024-08-30T09:11:56.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.847+0000 7f173e5ea740 1 -- 172.21.15.42:0/2657784954 shutdown_connections 2024-08-30T09:11:56.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.847+0000 7f173e5ea740 1 --2- 172.21.15.42:0/2657784954 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55fa205e7da0 0x55fa20a841b0 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:56.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.847+0000 7f173e5ea740 1 --2- 172.21.15.42:0/2657784954 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fa20654070 0x55fa20a803f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:56.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.847+0000 7f173e5ea740 1 --2- 172.21.15.42:0/2657784954 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fa20633390 0x55fa20a189b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:56.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.847+0000 7f173e5ea740 1 --2- 172.21.15.42:0/2657784954 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fa2062b1b0 0x55fa20a18470 secure :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f172c010cd0 tx=0x7f172c005c50 comp rx=0 tx=0).stop 2024-08-30T09:11:56.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.847+0000 7f173e5ea740 1 -- 172.21.15.42:0/2657784954 >> 172.21.15.42:0/2657784954 conn(0x55fa20648460 msgr2=0x55fa209fc3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:11:56.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.847+0000 7f17197fa640 0 client.27140 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:11:56.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.849+0000 7f173e5ea740 1 -- 172.21.15.42:0/2657784954 shutdown_connections 2024-08-30T09:11:56.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.849+0000 7f173e5ea740 1 -- 172.21.15.42:0/2657784954 wait complete. 2024-08-30T09:11:56.853 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.850+0000 7f173e5ea740 20 client.27140 trim_cache size 0 max 16384 2024-08-30T09:11:56.857 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-08-30T09:11:56.857 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "IDLE (waiting for more scrubs)", 2024-08-30T09:11:56.857 INFO:teuthology.orchestra.run.smithi042.stdout: "scrubs": { 2024-08-30T09:11:56.857 INFO:teuthology.orchestra.run.smithi042.stdout: "ff464e7f-123b-4bc6-bc7e-aced018168de": { 2024-08-30T09:11:56.857 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:11:56.857 INFO:teuthology.orchestra.run.smithi042.stdout: "tag": "ff464e7f-123b-4bc6-bc7e-aced018168de", 2024-08-30T09:11:56.857 INFO:teuthology.orchestra.run.smithi042.stdout: "options": "recursive,force" 2024-08-30T09:11:56.857 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:11:56.857 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:11:56.857 INFO:teuthology.orchestra.run.smithi042.stdout: "scrub_stats": { 2024-08-30T09:11:56.858 INFO:teuthology.orchestra.run.smithi042.stdout: "5e2ed104-0625-478a-a78b-584e7f4c2817": { 2024-08-30T09:11:56.858 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:00:33.928400+0000", 2024-08-30T09:11:56.858 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:11:56.858 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:11:56.858 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:11:56.858 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:11:56.858 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:11:56.858 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:11:56.858 INFO:teuthology.orchestra.run.smithi042.stdout: "c827ad7a-44ee-4e1d-9bb3-51fb7ab00c73": { 2024-08-30T09:11:56.858 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:02:21.686517+0000", 2024-08-30T09:11:56.858 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:11:56.858 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:11:56.858 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:11:56.858 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:11:56.858 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:11:56.859 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:11:56.859 INFO:teuthology.orchestra.run.smithi042.stdout: "85134e4b-ab4a-4885-b07e-a3ad01eaf13d": { 2024-08-30T09:11:56.859 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:01:27.738462+0000", 2024-08-30T09:11:56.859 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:11:56.859 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:11:56.859 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:11:56.859 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:11:56.859 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:11:56.859 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:11:56.859 INFO:teuthology.orchestra.run.smithi042.stdout: "749f4867-0b70-49e5-af2d-8362a240ebc3": { 2024-08-30T09:11:56.859 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:03:16.386554+0000", 2024-08-30T09:11:56.859 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:11:56.859 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:11:56.859 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:11:56.859 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:11:56.860 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:11:56.860 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:11:56.860 INFO:teuthology.orchestra.run.smithi042.stdout: "88cd7478-2b20-47de-ba51-256479749232": { 2024-08-30T09:11:56.860 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:04:08.922625+0000", 2024-08-30T09:11:56.860 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:11:56.860 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:11:56.860 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:11:56.860 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:11:56.860 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:11:56.860 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:11:56.860 INFO:teuthology.orchestra.run.smithi042.stdout: "347d05ed-4bd3-4590-b099-314f719aab39": { 2024-08-30T09:11:56.860 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:05:00.439721+0000", 2024-08-30T09:11:56.860 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:11:56.860 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:11:56.860 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:11:56.861 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:11:56.861 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:11:56.861 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:11:56.861 INFO:teuthology.orchestra.run.smithi042.stdout: "1af4252a-4a99-47e9-bbf0-becf5e573dd5": { 2024-08-30T09:11:56.861 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:05:48.565881+0000", 2024-08-30T09:11:56.861 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:11:56.861 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:11:56.861 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:11:56.861 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:11:56.861 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:11:56.861 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:11:56.861 INFO:teuthology.orchestra.run.smithi042.stdout: "665ec740-7254-49cb-84ef-deff71c461b0": { 2024-08-30T09:11:56.861 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:06:37.144035+0000", 2024-08-30T09:11:56.861 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:11:56.862 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:11:56.862 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:11:56.862 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:11:56.862 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:11:56.862 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:11:56.862 INFO:teuthology.orchestra.run.smithi042.stdout: "598d8a75-f1cc-4388-80bc-a408bdb4eaac": { 2024-08-30T09:11:56.862 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:07:29.721122+0000", 2024-08-30T09:11:56.862 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:11:56.862 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:11:56.862 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:11:56.862 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:11:56.862 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:11:56.862 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:11:56.862 INFO:teuthology.orchestra.run.smithi042.stdout: "b62c4ffc-9387-4677-b9ba-1e94ba308f17": { 2024-08-30T09:11:56.862 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:11:03.363478+0000", 2024-08-30T09:11:56.862 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:11:56.863 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:11:56.863 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:11:56.863 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:11:56.863 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:11:56.863 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:11:56.863 INFO:teuthology.orchestra.run.smithi042.stdout: "a7210f9a-046b-4188-953c-819b9f1863a9": { 2024-08-30T09:11:56.863 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:10:07.608430+0000", 2024-08-30T09:11:56.863 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:11:56.863 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:11:56.863 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:11:56.863 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:11:56.863 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:11:56.863 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:11:56.863 INFO:teuthology.orchestra.run.smithi042.stdout: "99a29ec1-9455-4ae4-854a-398b32896675": { 2024-08-30T09:11:56.864 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:08:24.923176+0000", 2024-08-30T09:11:56.864 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:11:56.864 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:11:56.864 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:11:56.864 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:11:56.864 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:11:56.864 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:11:56.864 INFO:teuthology.orchestra.run.smithi042.stdout: "ff464e7f-123b-4bc6-bc7e-aced018168de": { 2024-08-30T09:11:56.864 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:11:54.879598+0000", 2024-08-30T09:11:56.864 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:11:56.864 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:11:56.864 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:11:56.864 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:11:56.864 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:11:56.864 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:11:56.865 INFO:teuthology.orchestra.run.smithi042.stdout: "37f2f28f-496f-41a6-b4cf-8846fb3f43ab": { 2024-08-30T09:11:56.865 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:09:17.428286+0000", 2024-08-30T09:11:56.865 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:11:56.865 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:11:56.865 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:11:56.865 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:11:56.865 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:11:56.865 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:11:56.865 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:11:56.865 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-08-30T09:11:56.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.855+0000 7f1738075640 1 -- 172.21.15.42:0/2534541155 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f171c0b4b80 msgr2=0x7f171c0b7090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:11:56.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.855+0000 7f1738075640 1 --2- 172.21.15.42:0/2534541155 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f171c0b4b80 0x7f171c0b7090 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x55fa209f7f40 tx=0x7f172c005d40 comp rx=0 tx=0).stop 2024-08-30T09:11:56.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.856+0000 7f1738075640 1 -- 172.21.15.42:0/2534541155 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f17301023d0 msgr2=0x7f1730074650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:11:56.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.856+0000 7f1738075640 1 --2- 172.21.15.42:0/2534541155 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f17301023d0 0x7f1730074650 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f172000d6a0 tx=0x7f172000db60 comp rx=0 tx=0).stop 2024-08-30T09:11:56.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.856+0000 7f1738075640 1 -- 172.21.15.42:0/2534541155 shutdown_connections 2024-08-30T09:11:56.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.856+0000 7f1738075640 1 --2- 172.21.15.42:0/2534541155 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f171c0b4b80 0x7f171c0b7090 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:56.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.856+0000 7f1738075640 1 --2- 172.21.15.42:0/2534541155 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1730105190 0x7f17301a6000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:56.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.856+0000 7f1738075640 1 --2- 172.21.15.42:0/2534541155 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f17301023d0 0x7f1730074650 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:56.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.856+0000 7f1738075640 1 --2- 172.21.15.42:0/2534541155 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1730101a80 0x7f17300740b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:11:56.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.856+0000 7f1738075640 1 -- 172.21.15.42:0/2534541155 >> 172.21.15.42:0/2534541155 conn(0x7f1730077a00 msgr2=0x7f1730100a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:11:56.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.856+0000 7f1738075640 1 -- 172.21.15.42:0/2534541155 shutdown_connections 2024-08-30T09:11:56.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:11:56.856+0000 7f1738075640 1 -- 172.21.15.42:0/2534541155 wait complete. 2024-08-30T09:11:57.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:11:56 smithi042 ceph-mon[18652]: pgmap v708: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.0 MiB/s wr, 457 op/s 2024-08-30T09:11:57.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:11:56 smithi042 ceph-mon[18652]: scrub summary: idle+waiting paths [/] 2024-08-30T09:11:57.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:11:56 smithi190 ceph-mon[22001]: pgmap v708: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.0 MiB/s wr, 457 op/s 2024-08-30T09:11:57.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:11:56 smithi190 ceph-mon[22001]: scrub summary: idle+waiting paths [/] 2024-08-30T09:11:57.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:11:56 smithi167 ceph-mon[22063]: pgmap v708: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.0 MiB/s wr, 457 op/s 2024-08-30T09:11:57.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:11:56 smithi167 ceph-mon[22063]: scrub summary: idle+waiting paths [/] 2024-08-30T09:11:57.640 INFO:tasks.cephfs.filesystem:scrub status for tag:ff464e7f-123b-4bc6-bc7e-aced018168de - {'path': '/', 'tag': 'ff464e7f-123b-4bc6-bc7e-aced018168de', 'options': 'recursive,force'} 2024-08-30T09:11:59.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:11:58 smithi167 ceph-mon[22063]: pgmap v709: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 4.8 MiB/s wr, 418 op/s 2024-08-30T09:11:59.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:11:58 smithi042 ceph-mon[18652]: pgmap v709: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 4.8 MiB/s wr, 418 op/s 2024-08-30T09:11:59.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:11:58 smithi190 ceph-mon[22001]: pgmap v709: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 4.8 MiB/s wr, 418 op/s 2024-08-30T09:12:00.953 INFO:teuthology.orchestra.run.smithi042.stderr:progress: 675.0 s, 550.0 tps, lat 184.706 ms stddev 216.006 2024-08-30T09:12:01.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:12:00 smithi167 ceph-mon[22063]: pgmap v710: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 6.8 MiB/s wr, 638 op/s 2024-08-30T09:12:01.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:12:00 smithi042 ceph-mon[18652]: pgmap v710: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 6.8 MiB/s wr, 638 op/s 2024-08-30T09:12:01.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:12:00 smithi190 ceph-mon[22001]: pgmap v710: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 6.8 MiB/s wr, 638 op/s 2024-08-30T09:12:03.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:12:02 smithi167 ceph-mon[22063]: pgmap v711: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 5.3 MiB/s wr, 514 op/s 2024-08-30T09:12:03.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:12:02 smithi167 ceph-mon[22063]: scrub summary: idle 2024-08-30T09:12:03.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:12:02 smithi167 ceph-mon[22063]: scrub summary: idle 2024-08-30T09:12:03.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:12:02 smithi042 ceph-mon[18652]: pgmap v711: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 5.3 MiB/s wr, 514 op/s 2024-08-30T09:12:03.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:12:02 smithi042 ceph-mon[18652]: scrub summary: idle 2024-08-30T09:12:03.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:12:02 smithi042 ceph-mon[18652]: scrub summary: idle 2024-08-30T09:12:03.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:12:02 smithi190 ceph-mon[22001]: pgmap v711: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 5.3 MiB/s wr, 514 op/s 2024-08-30T09:12:03.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:12:02 smithi190 ceph-mon[22001]: scrub summary: idle 2024-08-30T09:12:03.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:12:02 smithi190 ceph-mon[22001]: scrub summary: idle 2024-08-30T09:12:04.101 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:12:03 smithi042 ceph-mon[18652]: scrub summary: idle 2024-08-30T09:12:04.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:12:03 smithi167 ceph-mon[22063]: scrub summary: idle 2024-08-30T09:12:04.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:12:03 smithi190 ceph-mon[22001]: scrub summary: idle 2024-08-30T09:12:05.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:12:04 smithi167 ceph-mon[22063]: pgmap v712: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 7.3 MiB/s wr, 642 op/s 2024-08-30T09:12:05.345 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:12:04 smithi042 ceph-mon[18652]: pgmap v712: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 7.3 MiB/s wr, 642 op/s 2024-08-30T09:12:05.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:12:04 smithi190 ceph-mon[22001]: pgmap v712: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 7.3 MiB/s wr, 642 op/s 2024-08-30T09:12:05.953 INFO:teuthology.orchestra.run.smithi042.stderr:progress: 680.0 s, 578.4 tps, lat 169.232 ms stddev 190.738 2024-08-30T09:12:07.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:12:06 smithi167 ceph-mon[22063]: pgmap v713: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 6.0 MiB/s wr, 577 op/s 2024-08-30T09:12:07.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:12:06 smithi042 ceph-mon[18652]: pgmap v713: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 6.0 MiB/s wr, 577 op/s 2024-08-30T09:12:07.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:12:06 smithi190 ceph-mon[22001]: pgmap v713: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 6.0 MiB/s wr, 577 op/s 2024-08-30T09:12:09.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:12:08 smithi167 ceph-mon[22063]: pgmap v714: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 5.7 MiB/s wr, 534 op/s 2024-08-30T09:12:09.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:12:08 smithi042 ceph-mon[18652]: pgmap v714: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 5.7 MiB/s wr, 534 op/s 2024-08-30T09:12:09.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:12:08 smithi190 ceph-mon[22001]: pgmap v714: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 5.7 MiB/s wr, 534 op/s 2024-08-30T09:12:09.886 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:09.884+0000 7fea5017b640 1 -- 172.21.15.42:0/2413361956 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fea48103d60 msgr2=0x7fea4810a700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:12:09.886 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:09.884+0000 7fea5017b640 1 --2- 172.21.15.42:0/2413361956 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fea48103d60 0x7fea4810a700 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fea40009910 tx=0x7fea4002a420 comp rx=0 tx=0).stop 2024-08-30T09:12:09.888 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:09.886+0000 7fea5017b640 1 -- 172.21.15.42:0/2413361956 shutdown_connections 2024-08-30T09:12:09.888 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:09.886+0000 7fea5017b640 1 --2- 172.21.15.42:0/2413361956 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fea481033c0 0x7fea481037d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:09.888 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:09.886+0000 7fea5017b640 1 --2- 172.21.15.42:0/2413361956 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fea48103d60 0x7fea4810a700 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:09.888 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:09.886+0000 7fea5017b640 1 --2- 172.21.15.42:0/2413361956 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fea48102340 0x7fea481027b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:09.888 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:09.886+0000 7fea5017b640 1 -- 172.21.15.42:0/2413361956 >> 172.21.15.42:0/2413361956 conn(0x7fea480fdc80 msgr2=0x7fea48100130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:12:09.894 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:09.892+0000 7fea5017b640 1 -- 172.21.15.42:0/2413361956 shutdown_connections 2024-08-30T09:12:09.899 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:09.896+0000 7fea5017b640 1 -- 172.21.15.42:0/2413361956 wait complete. 2024-08-30T09:12:09.900 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:09.897+0000 7fea5017b640 1 Processor -- start 2024-08-30T09:12:09.905 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:09.899+0000 7fea5017b640 1 -- start start 2024-08-30T09:12:09.905 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:09.899+0000 7fea5017b640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fea48102340 0x7fea48197b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:12:09.905 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:09.899+0000 7fea5017b640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fea481033c0 0x7fea481981b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:12:09.906 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:09.899+0000 7fea5017b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fea48103d60 0x7fea48198cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:12:09.906 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:09.899+0000 7fea5017b640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fea481112f0 con 0x7fea481033c0 2024-08-30T09:12:09.906 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:09.899+0000 7fea5017b640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fea4810cd60 con 0x7fea48102340 2024-08-30T09:12:09.906 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:09.899+0000 7fea5017b640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fea48111470 con 0x7fea48103d60 2024-08-30T09:12:09.906 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:09.902+0000 7fea4d716640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fea481033c0 0x7fea481981b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:12:09.906 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:09.902+0000 7fea4d716640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fea481033c0 0x7fea481981b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:55296/0 (socket says 172.21.15.42:55296) 2024-08-30T09:12:09.906 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:09.902+0000 7fea4d716640 1 -- 172.21.15.42:0/3871877953 learned_addr learned my addr 172.21.15.42:0/3871877953 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:12:09.906 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:09.903+0000 7fea4df17640 1 --2- 172.21.15.42:0/3871877953 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fea48102340 0x7fea48197b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:12:09.913 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:09.904+0000 7fea4e718640 1 --2- 172.21.15.42:0/3871877953 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fea48103d60 0x7fea48198cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:12:09.913 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:09.907+0000 7fea4e718640 1 -- 172.21.15.42:0/3871877953 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fea48102340 msgr2=0x7fea48197b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:12:09.913 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:09.907+0000 7fea4e718640 1 --2- 172.21.15.42:0/3871877953 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fea48102340 0x7fea48197b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:09.913 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:09.907+0000 7fea4e718640 1 -- 172.21.15.42:0/3871877953 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fea481033c0 msgr2=0x7fea481981b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:12:09.913 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:09.907+0000 7fea4e718640 1 --2- 172.21.15.42:0/3871877953 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fea481033c0 0x7fea481981b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:09.913 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:09.907+0000 7fea4e718640 1 -- 172.21.15.42:0/3871877953 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea48199510 con 0x7fea48103d60 2024-08-30T09:12:09.918 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:09.911+0000 7fea4e718640 1 --2- 172.21.15.42:0/3871877953 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fea48103d60 0x7fea48198cf0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fea40009230 tx=0x7fea40030de0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:12:09.918 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:09.915+0000 7fea46ffd640 1 -- 172.21.15.42:0/3871877953 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea40004450 con 0x7fea48103d60 2024-08-30T09:12:09.918 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:09.915+0000 7fea5017b640 1 -- 172.21.15.42:0/3871877953 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea481997a0 con 0x7fea48103d60 2024-08-30T09:12:09.918 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:09.915+0000 7fea5017b640 1 -- 172.21.15.42:0/3871877953 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea48199ab0 con 0x7fea48103d60 2024-08-30T09:12:09.918 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:09.915+0000 7fea46ffd640 1 -- 172.21.15.42:0/3871877953 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea400318b0 con 0x7fea48103d60 2024-08-30T09:12:09.918 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:09.915+0000 7fea46ffd640 1 -- 172.21.15.42:0/3871877953 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea40003db0 con 0x7fea48103d60 2024-08-30T09:12:09.932 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:09.922+0000 7fea571c2740 1 Processor -- start 2024-08-30T09:12:09.941 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:09.933+0000 7fea571c2740 1 -- start start 2024-08-30T09:12:09.941 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:09.933+0000 7fea571c2740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5563603ea1b0 0x556360790180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:12:09.941 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:09.933+0000 7fea571c2740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5563603c12f0 0x5563607907f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:12:09.942 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:09.933+0000 7fea571c2740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5563603c94d0 0x556360791340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:12:09.942 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:09.933+0000 7fea571c2740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x556360703000 con 0x5563603c94d0 2024-08-30T09:12:09.942 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:09.933+0000 7fea571c2740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5563606f2590 con 0x5563603ea1b0 2024-08-30T09:12:09.942 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:09.933+0000 7fea571c2740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5563606f6540 con 0x5563603c12f0 2024-08-30T09:12:09.942 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:09.934+0000 7fea46ffd640 1 -- 172.21.15.42:0/3871877953 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7fea40043020 con 0x7fea48103d60 2024-08-30T09:12:09.942 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:09.935+0000 7fea46ffd640 1 --2- 172.21.15.42:0/3871877953 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fea300b4d40 0x7fea300b7250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:12:09.942 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:09.936+0000 7fea46ffd640 1 -- 172.21.15.42:0/3871877953 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7fea40106880 con 0x7fea48103d60 2024-08-30T09:12:09.942 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:09.937+0000 7fea4df17640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5563603c12f0 0x5563607907f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:12:09.944 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:09.940+0000 7fea4d716640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5563603ea1b0 0x556360790180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:12:09.944 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:09.941+0000 7fea4df17640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5563603c12f0 0x5563607907f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:47340/0 (socket says 172.21.15.42:47340) 2024-08-30T09:12:09.944 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:09.941+0000 7fea4df17640 1 -- 172.21.15.42:0/4001138844 learned_addr learned my addr 172.21.15.42:0/4001138844 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:12:09.944 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:09.942+0000 7fea4d716640 1 --2- 172.21.15.42:0/4001138844 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5563603c94d0 0x556360791340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:12:09.946 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:09.943+0000 7fea4e718640 1 --2- 172.21.15.42:0/3871877953 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fea300b4d40 0x7fea300b7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:12:09.948 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:09.945+0000 7fea4e718640 1 --2- 172.21.15.42:0/3871877953 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fea300b4d40 0x7fea300b7250 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fea40005f60 tx=0x7fea40009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:12:09.948 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:09.945+0000 7fea4df17640 1 -- 172.21.15.42:0/4001138844 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5563603ea1b0 msgr2=0x556360790180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:12:09.949 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:09.945+0000 7fea4df17640 1 --2- 172.21.15.42:0/4001138844 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5563603ea1b0 0x556360790180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:09.949 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:09.945+0000 7fea4df17640 1 -- 172.21.15.42:0/4001138844 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5563603c94d0 msgr2=0x556360791340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:12:09.949 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:09.945+0000 7fea4df17640 1 --2- 172.21.15.42:0/4001138844 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5563603c94d0 0x556360791340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:09.949 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:09.945+0000 7fea4df17640 1 -- 172.21.15.42:0/4001138844 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556360455790 con 0x5563603c12f0 2024-08-30T09:12:09.949 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:09.946+0000 7fea4df17640 1 --2- 172.21.15.42:0/4001138844 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5563603c12f0 0x5563607907f0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fea380114f0 tx=0x7fea380119b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c880288889ccef1c server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:12:09.949 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:09.946+0000 7fea2ffff640 1 -- 172.21.15.42:0/4001138844 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea3801e070 con 0x5563603c12f0 2024-08-30T09:12:09.949 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:09.946+0000 7fea2ffff640 1 -- 172.21.15.42:0/4001138844 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea3800fad0 con 0x5563603c12f0 2024-08-30T09:12:09.949 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:09.946+0000 7fea2ffff640 1 -- 172.21.15.42:0/4001138844 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea38019b00 con 0x5563603c12f0 2024-08-30T09:12:09.951 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:09.949+0000 7fea571c2740 1 -- 172.21.15.42:0/4001138844 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5563603c12f0 msgr2=0x5563607907f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:12:09.951 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:09.949+0000 7fea571c2740 1 --2- 172.21.15.42:0/4001138844 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5563603c12f0 0x5563607907f0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fea380114f0 tx=0x7fea380119b0 comp rx=0 tx=0).stop 2024-08-30T09:12:09.953 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:09.951+0000 7fea571c2740 1 -- 172.21.15.42:0/4001138844 shutdown_connections 2024-08-30T09:12:09.953 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:09.951+0000 7fea571c2740 1 --2- 172.21.15.42:0/4001138844 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5563603c12f0 0x5563607907f0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:09.953 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:09.951+0000 7fea571c2740 1 --2- 172.21.15.42:0/4001138844 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5563603c94d0 0x556360791340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:09.953 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:09.951+0000 7fea571c2740 1 --2- 172.21.15.42:0/4001138844 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5563603ea1b0 0x556360790180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:09.953 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:09.951+0000 7fea571c2740 1 -- 172.21.15.42:0/4001138844 >> 172.21.15.42:0/4001138844 conn(0x5563603de300 msgr2=0x55636078bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:12:09.961 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:09.959+0000 7fea571c2740 1 -- 172.21.15.42:0/4001138844 shutdown_connections 2024-08-30T09:12:09.963 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:09.961+0000 7fea571c2740 1 -- 172.21.15.42:0/4001138844 wait complete. 2024-08-30T09:12:09.963 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:09.961+0000 7fea571c2740 1 -- start start 2024-08-30T09:12:09.965 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:09.961+0000 7fea571c2740 1 Processor -- start 2024-08-30T09:12:09.967 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:09.965+0000 7fea571c2740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5563603c12f0 0x556360799c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:12:09.968 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:09.965+0000 7fea571c2740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5563603c94d0 0x55636079a320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:12:09.968 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:09.965+0000 7fea571c2740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5563603ea1b0 0x55636079ae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:12:09.968 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:09.965+0000 7fea571c2740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55636062bab0 con 0x5563603c12f0 2024-08-30T09:12:09.968 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:09.965+0000 7fea571c2740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x556360679680 con 0x5563603ea1b0 2024-08-30T09:12:09.968 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:09.965+0000 7fea571c2740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x556360791e60 con 0x5563603c94d0 2024-08-30T09:12:09.968 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:09.965+0000 7fea571c2740 10 client.0 _ll_register_callbacks cb 0x556360401a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:12:09.968 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:09.966+0000 7fea4df17640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5563603c94d0 0x55636079a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:12:09.968 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:09.966+0000 7fea4df17640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5563603c94d0 0x55636079a320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:47348/0 (socket says 172.21.15.42:47348) 2024-08-30T09:12:09.968 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:09.966+0000 7fea4df17640 1 -- 172.21.15.42:0/2792628373 learned_addr learned my addr 172.21.15.42:0/2792628373 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:12:09.969 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:09.966+0000 7fea4d716640 1 --2- 172.21.15.42:0/2792628373 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5563603c12f0 0x556360799c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:12:09.969 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:09.966+0000 7fea4d716640 1 --2- 172.21.15.42:0/2792628373 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5563603ea1b0 0x55636079ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:12:09.969 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:09.967+0000 7fea4df17640 1 -- 172.21.15.42:0/2792628373 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5563603ea1b0 msgr2=0x55636079ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:12:09.969 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:09.967+0000 7fea4df17640 1 --2- 172.21.15.42:0/2792628373 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5563603ea1b0 0x55636079ae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:09.969 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:09.967+0000 7fea4df17640 1 -- 172.21.15.42:0/2792628373 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5563603c12f0 msgr2=0x556360799c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:12:09.969 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:09.967+0000 7fea4df17640 1 --2- 172.21.15.42:0/2792628373 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5563603c12f0 0x556360799c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:09.969 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:09.967+0000 7fea4df17640 1 -- 172.21.15.42:0/2792628373 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5563604ce620 con 0x5563603c94d0 2024-08-30T09:12:09.970 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:09.968+0000 7fea4df17640 1 --2- 172.21.15.42:0/2792628373 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5563603c94d0 0x55636079a320 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fea38009de0 tx=0x7fea3800f850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dfdff319d02609a5 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:12:09.970 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:09.968+0000 7fea2d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-08-30T09:12:09.970 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:09.968+0000 7fea2d7fa640 1 -- 172.21.15.42:0/2792628373 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea3801e040 con 0x5563603c94d0 2024-08-30T09:12:09.970 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:09.968+0000 7fea571c2740 1 -- 172.21.15.42:0/2792628373 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea3800fad0 con 0x5563603c94d0 2024-08-30T09:12:09.970 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:09.968+0000 7fea2d7fa640 1 -- 172.21.15.42:0/2792628373 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7fea38016050 con 0x5563603c94d0 2024-08-30T09:12:09.971 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:09.969+0000 7fea2d7fa640 1 -- 172.21.15.42:0/2792628373 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fea38031050 con 0x5563603c94d0 2024-08-30T09:12:09.971 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:09.969+0000 7fea571c2740 10 client.27103 fetch_fsmap learned FSMap version 42 2024-08-30T09:12:09.971 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:09.969+0000 7fea571c2740 1 -- 172.21.15.42:0/2792628373 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x5563607a3e10 con 0x5563603c94d0 2024-08-30T09:12:09.971 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:09.970+0000 7fea2d7fa640 1 -- 172.21.15.42:0/2792628373 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea3800fad0 con 0x5563603c94d0 2024-08-30T09:12:09.973 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:09.970+0000 7fea2d7fa640 1 -- 172.21.15.42:0/2792628373 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 42) ==== 8358+0+0 (secure 0 0 0) 0x7fea38021030 con 0x5563603c94d0 2024-08-30T09:12:09.973 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:09.970+0000 7fea571c2740 10 client.27103 fetch_fsmap finished waiting for FSMap version 42 2024-08-30T09:12:09.973 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:09.970+0000 7fea571c2740 10 client.27103 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-30T09:12:09.973 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:09.970+0000 7fea571c2740 20 client.27103 populate_metadata read hostname 'smithi042' 2024-08-30T09:12:09.973 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:09.970+0000 7fea571c2740 1 --2- 172.21.15.42:0/2792628373 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x5563603a8d80 0x5563607a7920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:12:09.973 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:09.970+0000 7fea571c2740 4 client.27103 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-30T09:12:09.973 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:09.970+0000 7fea571c2740 1 -- 172.21.15.42:0/2792628373 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556360704f50 con 0x5563603a8d80 2024-08-30T09:12:09.974 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:09.971+0000 7fea4d716640 1 --2- 172.21.15.42:0/2792628373 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x5563603a8d80 0x5563607a7920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:12:09.974 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:09.972+0000 7fea4d716640 1 --2- 172.21.15.42:0/2792628373 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x5563603a8d80 0x5563607a7920 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3684eb6b46e4cacc server_cookie=d833beab798a4e55 in_seq=0 out_seq=0 2024-08-30T09:12:09.974 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:09.972+0000 7fea2d7fa640 10 client.27103 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:12:09.975 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:09.973+0000 7fea2d7fa640 1 -- 172.21.15.42:0/2792628373 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x556360704f50 con 0x5563603a8d80 2024-08-30T09:12:09.975 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:09.973+0000 7fea2d7fa640 10 client.27103 handle_command_reply: tid=0 2024-08-30T09:12:09.975 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:09.973+0000 7fea571c2740 1 client.27103 shutdown 2024-08-30T09:12:09.976 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:09.973+0000 7fea571c2740 1 -- 172.21.15.42:0/2792628373 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5563603c94d0 msgr2=0x55636079a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:12:09.976 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:09.973+0000 7fea571c2740 1 --2- 172.21.15.42:0/2792628373 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5563603c94d0 0x55636079a320 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fea38009de0 tx=0x7fea3800f850 comp rx=0 tx=0).stop 2024-08-30T09:12:09.976 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:09.974+0000 7fea571c2740 1 -- 172.21.15.42:0/2792628373 shutdown_connections 2024-08-30T09:12:09.976 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:09.974+0000 7fea571c2740 1 --2- 172.21.15.42:0/2792628373 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x5563603a8d80 0x5563607a7920 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:09.976 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:09.974+0000 7fea571c2740 1 --2- 172.21.15.42:0/2792628373 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5563603c94d0 0x55636079a320 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:09.976 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:09.974+0000 7fea571c2740 1 --2- 172.21.15.42:0/2792628373 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5563603ea1b0 0x55636079ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:09.976 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:09.974+0000 7fea571c2740 1 --2- 172.21.15.42:0/2792628373 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5563603c12f0 0x556360799c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:09.976 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:09.974+0000 7fea571c2740 1 -- 172.21.15.42:0/2792628373 >> 172.21.15.42:0/2792628373 conn(0x5563603de300 msgr2=0x55636078a170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:12:09.976 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:09.974+0000 7fea2d7fa640 0 client.27103 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:12:09.976 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:09.974+0000 7fea571c2740 1 -- 172.21.15.42:0/2792628373 shutdown_connections 2024-08-30T09:12:09.977 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:09.974+0000 7fea571c2740 1 -- 172.21.15.42:0/2792628373 wait complete. 2024-08-30T09:12:09.977 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:09.974+0000 7fea571c2740 20 client.27103 trim_cache size 0 max 16384 2024-08-30T09:12:09.994 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:09.991+0000 7fea571c2740 1 Processor -- start 2024-08-30T09:12:09.994 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:09.991+0000 7fea571c2740 1 -- start start 2024-08-30T09:12:09.994 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:09.991+0000 7fea571c2740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5563603c12f0 0x55636079ae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:12:09.994 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:09.991+0000 7fea571c2740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5563603c94d0 0x55636079b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:12:09.994 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:09.991+0000 7fea571c2740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5563603ea1b0 0x556360799c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:12:09.994 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:09.991+0000 7fea571c2740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fea340124b0 con 0x5563603c94d0 2024-08-30T09:12:09.994 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:09.991+0000 7fea571c2740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fea3400fd50 con 0x5563603ea1b0 2024-08-30T09:12:09.994 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:09.991+0000 7fea571c2740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fea3400dba0 con 0x5563603c12f0 2024-08-30T09:12:09.994 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:09.991+0000 7fea4df17640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5563603c94d0 0x55636079b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:12:09.994 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:09.991+0000 7fea4df17640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5563603c94d0 0x55636079b3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:55332/0 (socket says 172.21.15.42:55332) 2024-08-30T09:12:09.994 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:09.991+0000 7fea4df17640 1 -- 172.21.15.42:0/2234340322 learned_addr learned my addr 172.21.15.42:0/2234340322 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:12:09.994 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:09.992+0000 7fea4df17640 1 -- 172.21.15.42:0/2234340322 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5563603c12f0 msgr2=0x55636079ae90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-30T09:12:09.994 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:09.992+0000 7fea4df17640 1 --2- 172.21.15.42:0/2234340322 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5563603c12f0 0x55636079ae90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:09.995 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:09.992+0000 7fea4df17640 1 -- 172.21.15.42:0/2234340322 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5563603ea1b0 msgr2=0x556360799c20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-30T09:12:09.995 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:09.992+0000 7fea4df17640 1 --2- 172.21.15.42:0/2234340322 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5563603ea1b0 0x556360799c20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:09.995 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:09.992+0000 7fea4df17640 1 -- 172.21.15.42:0/2234340322 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556360791f70 con 0x5563603c94d0 2024-08-30T09:12:09.995 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:09.992+0000 7fea4df17640 1 --2- 172.21.15.42:0/2234340322 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5563603c94d0 0x55636079b3d0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fea38006010 tx=0x7fea38005cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23bd508cb4f1d612 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:12:09.995 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:09.992+0000 7fea2f7fe640 1 -- 172.21.15.42:0/2234340322 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea3801e070 con 0x5563603c94d0 2024-08-30T09:12:09.995 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:09.992+0000 7fea2f7fe640 1 -- 172.21.15.42:0/2234340322 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea380051d0 con 0x5563603c94d0 2024-08-30T09:12:09.995 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:09.992+0000 7fea2f7fe640 1 -- 172.21.15.42:0/2234340322 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea380058b0 con 0x5563603c94d0 2024-08-30T09:12:09.996 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:09.993+0000 7fea571c2740 1 -- 172.21.15.42:0/2234340322 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5563603c94d0 msgr2=0x55636079b3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:12:09.996 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:09.993+0000 7fea571c2740 1 --2- 172.21.15.42:0/2234340322 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5563603c94d0 0x55636079b3d0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fea38006010 tx=0x7fea38005cf0 comp rx=0 tx=0).stop 2024-08-30T09:12:09.996 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:09.995+0000 7fea571c2740 1 -- 172.21.15.42:0/2234340322 shutdown_connections 2024-08-30T09:12:09.997 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:09.995+0000 7fea571c2740 1 --2- 172.21.15.42:0/2234340322 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5563603ea1b0 0x556360799c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:09.997 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:09.995+0000 7fea571c2740 1 --2- 172.21.15.42:0/2234340322 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5563603c94d0 0x55636079b3d0 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:09.997 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:09.995+0000 7fea571c2740 1 --2- 172.21.15.42:0/2234340322 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5563603c12f0 0x55636079ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:09.997 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:09.995+0000 7fea571c2740 1 -- 172.21.15.42:0/2234340322 >> 172.21.15.42:0/2234340322 conn(0x5563603de300 msgr2=0x556360798380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:12:09.997 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:09.996+0000 7fea571c2740 1 -- 172.21.15.42:0/2234340322 shutdown_connections 2024-08-30T09:12:09.997 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:09.996+0000 7fea571c2740 1 -- 172.21.15.42:0/2234340322 wait complete. 2024-08-30T09:12:09.998 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:09.996+0000 7fea571c2740 1 -- start start 2024-08-30T09:12:09.998 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:09.996+0000 7fea571c2740 1 Processor -- start 2024-08-30T09:12:09.999 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:09.997+0000 7fea571c2740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5563603c12f0 0x55636079e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:12:09.999 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:09.997+0000 7fea571c2740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5563603c94d0 0x55636079e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:12:09.999 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:09.997+0000 7fea571c2740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5563603ea1b0 0x55636079f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:12:09.999 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:09.997+0000 7fea571c2740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fea3400e210 con 0x5563603c94d0 2024-08-30T09:12:09.999 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:09.997+0000 7fea571c2740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fea3400f680 con 0x5563603ea1b0 2024-08-30T09:12:09.999 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:09.997+0000 7fea571c2740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fea3400e8e0 con 0x5563603c12f0 2024-08-30T09:12:09.999 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:09.997+0000 7fea571c2740 10 client.0 _ll_register_callbacks cb 0x7fea3801c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:12:09.999 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:09.997+0000 7fea4d716640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5563603ea1b0 0x55636079f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:12:09.999 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:09.997+0000 7fea4d716640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5563603ea1b0 0x55636079f3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:50644/0 (socket says 172.21.15.42:50644) 2024-08-30T09:12:10.000 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:09.997+0000 7fea4d716640 1 -- 172.21.15.42:0/4292523165 learned_addr learned my addr 172.21.15.42:0/4292523165 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:12:10.000 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:09.997+0000 7fea4df17640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5563603c94d0 0x55636079e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:12:10.000 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:09.997+0000 7fea4d716640 1 --2- 172.21.15.42:0/4292523165 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5563603c12f0 0x55636079e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:12:10.001 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:09.997+0000 7fea4df17640 1 -- 172.21.15.42:0/4292523165 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5563603c12f0 msgr2=0x55636079e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:12:10.002 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:09.997+0000 7fea4df17640 1 --2- 172.21.15.42:0/4292523165 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5563603c12f0 0x55636079e1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:10.002 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:09.997+0000 7fea4df17640 1 -- 172.21.15.42:0/4292523165 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5563603ea1b0 msgr2=0x55636079f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:12:10.002 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:09.997+0000 7fea4df17640 1 --2- 172.21.15.42:0/4292523165 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5563603ea1b0 0x55636079f3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:10.002 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:09.997+0000 7fea4df17640 1 -- 172.21.15.42:0/4292523165 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55636079fb00 con 0x5563603c94d0 2024-08-30T09:12:10.002 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:09.997+0000 7fea4d716640 1 --2- 172.21.15.42:0/4292523165 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5563603ea1b0 0x55636079f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-30T09:12:10.002 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:09.998+0000 7fea4df17640 1 --2- 172.21.15.42:0/4292523165 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5563603c94d0 0x55636079e8a0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fea38006010 tx=0x7fea3800f900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d818b7e31d52285 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:12:10.002 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:09.998+0000 7fea2d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-08-30T09:12:10.002 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:09.998+0000 7fea2d7fa640 1 -- 172.21.15.42:0/4292523165 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea3801e040 con 0x5563603c94d0 2024-08-30T09:12:10.002 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:09.998+0000 7fea2d7fa640 1 -- 172.21.15.42:0/4292523165 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7fea38003970 con 0x5563603c94d0 2024-08-30T09:12:10.002 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:09.998+0000 7fea571c2740 1 -- 172.21.15.42:0/4292523165 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea380051d0 con 0x5563603c94d0 2024-08-30T09:12:10.002 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:09.998+0000 7fea2d7fa640 1 -- 172.21.15.42:0/4292523165 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fea38003b70 con 0x5563603c94d0 2024-08-30T09:12:10.002 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:09.998+0000 7fea571c2740 10 client.17316 fetch_fsmap learned FSMap version 42 2024-08-30T09:12:10.003 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:09.999+0000 7fea571c2740 1 -- 172.21.15.42:0/4292523165 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x5563607a0df0 con 0x5563603c94d0 2024-08-30T09:12:10.003 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:09.999+0000 7fea2d7fa640 1 -- 172.21.15.42:0/4292523165 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea380051d0 con 0x5563603c94d0 2024-08-30T09:12:10.003 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:09.999+0000 7fea2d7fa640 1 -- 172.21.15.42:0/4292523165 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 42) ==== 8358+0+0 (secure 0 0 0) 0x7fea38021040 con 0x5563603c94d0 2024-08-30T09:12:10.003 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:09.999+0000 7fea571c2740 10 client.17316 fetch_fsmap finished waiting for FSMap version 42 2024-08-30T09:12:10.003 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:09.999+0000 7fea571c2740 10 client.17316 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-30T09:12:10.003 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:09.999+0000 7fea571c2740 20 client.17316 populate_metadata read hostname 'smithi042' 2024-08-30T09:12:10.003 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:09.999+0000 7fea571c2740 1 --2- 172.21.15.42:0/4292523165 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55636037e1c0 0x5563607a1840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:12:10.003 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:09.999+0000 7fea571c2740 4 client.17316 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-30T09:12:10.003 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:09.999+0000 7fea571c2740 1 -- 172.21.15.42:0/4292523165 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5563607009f0 con 0x55636037e1c0 2024-08-30T09:12:10.003 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:09.999+0000 7fea4d716640 1 --2- 172.21.15.42:0/4292523165 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55636037e1c0 0x5563607a1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:12:10.003 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:10.000+0000 7fea2d7fa640 10 client.17316 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:12:10.004 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:10.000+0000 7fea4d716640 1 --2- 172.21.15.42:0/4292523165 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55636037e1c0 0x5563607a1840 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=df6fb65bc09f4046 server_cookie=6b83c68d82b383ee in_seq=0 out_seq=0 2024-08-30T09:12:10.015 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:10.012+0000 7fea2d7fa640 1 -- 172.21.15.42:0/4292523165 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5563607009f0 con 0x55636037e1c0 2024-08-30T09:12:10.015 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:10.013+0000 7fea2d7fa640 10 client.17316 handle_command_reply: tid=0 2024-08-30T09:12:10.016 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:10.014+0000 7fea571c2740 1 client.17316 shutdown 2024-08-30T09:12:10.017 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:10.014+0000 7fea571c2740 1 -- 172.21.15.42:0/4292523165 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5563603c94d0 msgr2=0x55636079e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:12:10.017 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:10.014+0000 7fea571c2740 1 --2- 172.21.15.42:0/4292523165 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5563603c94d0 0x55636079e8a0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fea38006010 tx=0x7fea3800f900 comp rx=0 tx=0).stop 2024-08-30T09:12:10.017 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:10.015+0000 7fea571c2740 1 -- 172.21.15.42:0/4292523165 shutdown_connections 2024-08-30T09:12:10.017 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:10.015+0000 7fea571c2740 1 --2- 172.21.15.42:0/4292523165 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55636037e1c0 0x5563607a1840 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:10.017 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:10.015+0000 7fea2d7fa640 0 client.17316 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:12:10.017 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:10.015+0000 7fea571c2740 1 --2- 172.21.15.42:0/4292523165 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5563603ea1b0 0x55636079f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:10.017 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:10.015+0000 7fea571c2740 1 --2- 172.21.15.42:0/4292523165 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5563603c94d0 0x55636079e8a0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:10.017 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:10.015+0000 7fea571c2740 1 --2- 172.21.15.42:0/4292523165 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5563603c12f0 0x55636079e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:10.017 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:10.015+0000 7fea571c2740 1 -- 172.21.15.42:0/4292523165 >> 172.21.15.42:0/4292523165 conn(0x5563603de300 msgr2=0x556360799c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:12:10.018 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-08-30T09:12:10.018 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2024-08-30T09:12:10.018 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-08-30T09:12:10.018 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-08-30T09:12:10.019 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:10.015+0000 7fea571c2740 1 -- 172.21.15.42:0/4292523165 shutdown_connections 2024-08-30T09:12:10.019 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:10.015+0000 7fea571c2740 1 -- 172.21.15.42:0/4292523165 wait complete. 2024-08-30T09:12:10.019 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:10.015+0000 7fea571c2740 20 client.17316 trim_cache size 0 max 16384 2024-08-30T09:12:10.019 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:10.016+0000 7fea5017b640 1 -- 172.21.15.42:0/3871877953 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fea300b4d40 msgr2=0x7fea300b7250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:12:10.020 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:10.016+0000 7fea5017b640 1 --2- 172.21.15.42:0/3871877953 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fea300b4d40 0x7fea300b7250 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fea40005f60 tx=0x7fea40009260 comp rx=0 tx=0).stop 2024-08-30T09:12:10.020 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:10.016+0000 7fea5017b640 1 -- 172.21.15.42:0/3871877953 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fea48103d60 msgr2=0x7fea48198cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:12:10.020 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:10.016+0000 7fea5017b640 1 --2- 172.21.15.42:0/3871877953 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fea48103d60 0x7fea48198cf0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fea40009230 tx=0x7fea40030de0 comp rx=0 tx=0).stop 2024-08-30T09:12:10.020 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:10.017+0000 7fea5017b640 1 -- 172.21.15.42:0/3871877953 shutdown_connections 2024-08-30T09:12:10.020 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:10.017+0000 7fea5017b640 1 --2- 172.21.15.42:0/3871877953 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fea300b4d40 0x7fea300b7250 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:10.020 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:10.017+0000 7fea5017b640 1 --2- 172.21.15.42:0/3871877953 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fea48103d60 0x7fea48198cf0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:10.020 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:10.017+0000 7fea5017b640 1 --2- 172.21.15.42:0/3871877953 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fea481033c0 0x7fea481981b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:10.020 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:10.017+0000 7fea5017b640 1 --2- 172.21.15.42:0/3871877953 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fea48102340 0x7fea48197b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:10.020 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:10.017+0000 7fea5017b640 1 -- 172.21.15.42:0/3871877953 >> 172.21.15.42:0/3871877953 conn(0x7fea480fdc80 msgr2=0x7fea4810eef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:12:10.020 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:10.017+0000 7fea5017b640 1 -- 172.21.15.42:0/3871877953 shutdown_connections 2024-08-30T09:12:10.020 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:10.017+0000 7fea5017b640 1 -- 172.21.15.42:0/3871877953 wait complete. 2024-08-30T09:12:10.953 INFO:teuthology.orchestra.run.smithi042.stderr:progress: 685.0 s, 472.8 tps, lat 208.995 ms stddev 244.591 2024-08-30T09:12:11.238 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:12:10 smithi190 ceph-mon[22001]: pgmap v715: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 7.5 MiB/s wr, 658 op/s 2024-08-30T09:12:11.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:12:10 smithi167 ceph-mon[22063]: pgmap v715: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 7.5 MiB/s wr, 658 op/s 2024-08-30T09:12:11.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:12:10 smithi042 ceph-mon[18652]: pgmap v715: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 7.5 MiB/s wr, 658 op/s 2024-08-30T09:12:12.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:12:11 smithi167 ceph-mon[22063]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-30T09:12:12.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:12:11 smithi042 ceph-mon[18652]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-30T09:12:12.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:12:11 smithi190 ceph-mon[22001]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-30T09:12:13.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:12:12 smithi167 ceph-mon[22063]: pgmap v716: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.5 MiB/s wr, 430 op/s 2024-08-30T09:12:13.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:12:12 smithi042 ceph-mon[18652]: pgmap v716: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.5 MiB/s wr, 430 op/s 2024-08-30T09:12:13.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:12:12 smithi190 ceph-mon[22001]: pgmap v716: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.5 MiB/s wr, 430 op/s 2024-08-30T09:12:15.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:12:14 smithi167 ceph-mon[22063]: pgmap v717: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.3 MiB/s wr, 548 op/s 2024-08-30T09:12:15.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:12:14 smithi042 ceph-mon[18652]: pgmap v717: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.3 MiB/s wr, 548 op/s 2024-08-30T09:12:15.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:12:15 smithi190 ceph-mon[22001]: pgmap v717: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.3 MiB/s wr, 548 op/s 2024-08-30T09:12:15.953 INFO:teuthology.orchestra.run.smithi042.stderr:progress: 690.0 s, 525.4 tps, lat 195.537 ms stddev 216.975 2024-08-30T09:12:17.328 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:12:17 smithi167 ceph-mon[22063]: pgmap v718: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 5.3 MiB/s wr, 465 op/s 2024-08-30T09:12:17.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:12:17 smithi042 ceph-mon[18652]: pgmap v718: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 5.3 MiB/s wr, 465 op/s 2024-08-30T09:12:17.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:12:17 smithi190 ceph-mon[22001]: pgmap v718: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 5.3 MiB/s wr, 465 op/s 2024-08-30T09:12:19.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:12:19 smithi167 ceph-mon[22063]: pgmap v719: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 5.0 MiB/s wr, 440 op/s 2024-08-30T09:12:19.384 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:12:19 smithi042 ceph-mon[18652]: pgmap v719: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 5.0 MiB/s wr, 440 op/s 2024-08-30T09:12:19.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:12:19 smithi190 ceph-mon[22001]: pgmap v719: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 5.0 MiB/s wr, 440 op/s 2024-08-30T09:12:20.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:12:20 smithi167 ceph-mon[22063]: pgmap v720: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 6.8 MiB/s wr, 582 op/s 2024-08-30T09:12:20.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:12:20 smithi042 ceph-mon[18652]: pgmap v720: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 6.8 MiB/s wr, 582 op/s 2024-08-30T09:12:20.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:12:20 smithi190 ceph-mon[22001]: pgmap v720: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 6.8 MiB/s wr, 582 op/s 2024-08-30T09:12:20.953 INFO:teuthology.orchestra.run.smithi042.stderr:progress: 695.0 s, 523.4 tps, lat 189.288 ms stddev 205.052 2024-08-30T09:12:22.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:12:22 smithi042 ceph-mon[18652]: pgmap v721: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 5.1 MiB/s wr, 458 op/s 2024-08-30T09:12:22.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:12:22 smithi190 ceph-mon[22001]: pgmap v721: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 5.1 MiB/s wr, 458 op/s 2024-08-30T09:12:22.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:12:22 smithi167 ceph-mon[22063]: pgmap v721: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 5.1 MiB/s wr, 458 op/s 2024-08-30T09:12:23.178 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.176+0000 7fcc9cad5640 1 -- 172.21.15.42:0/637427551 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcc98103d60 msgr2=0x7fcc9810a700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:12:23.178 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.176+0000 7fcc9cad5640 1 --2- 172.21.15.42:0/637427551 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcc98103d60 0x7fcc9810a700 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fcc8c00d920 tx=0x7fcc8c02e5c0 comp rx=0 tx=0).stop 2024-08-30T09:12:23.179 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.176+0000 7fcc9cad5640 1 -- 172.21.15.42:0/637427551 shutdown_connections 2024-08-30T09:12:23.179 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.176+0000 7fcc9cad5640 1 --2- 172.21.15.42:0/637427551 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcc981033c0 0x7fcc981037d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:23.179 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.176+0000 7fcc9cad5640 1 --2- 172.21.15.42:0/637427551 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcc98103d60 0x7fcc9810a700 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:23.179 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.176+0000 7fcc9cad5640 1 --2- 172.21.15.42:0/637427551 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcc98102340 0x7fcc981027b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:23.179 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.176+0000 7fcc9cad5640 1 -- 172.21.15.42:0/637427551 >> 172.21.15.42:0/637427551 conn(0x7fcc980fdc80 msgr2=0x7fcc98100130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:12:23.179 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.177+0000 7fcc9cad5640 1 -- 172.21.15.42:0/637427551 shutdown_connections 2024-08-30T09:12:23.179 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.177+0000 7fcc9cad5640 1 -- 172.21.15.42:0/637427551 wait complete. 2024-08-30T09:12:23.180 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.178+0000 7fcc9cad5640 1 Processor -- start 2024-08-30T09:12:23.180 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.178+0000 7fcc9cad5640 1 -- start start 2024-08-30T09:12:23.181 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.178+0000 7fcc9cad5640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcc98102340 0x7fcc981a0480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:12:23.181 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.178+0000 7fcc9cad5640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcc981033c0 0x7fcc981a0af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:12:23.181 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.178+0000 7fcc9cad5640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcc98103d60 0x7fcc981a16a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:12:23.181 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.178+0000 7fcc9cad5640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fcc9810cfe0 con 0x7fcc98103d60 2024-08-30T09:12:23.181 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.178+0000 7fcc9cad5640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fcc9810ce60 con 0x7fcc98102340 2024-08-30T09:12:23.181 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.178+0000 7fcc9cad5640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fcc9810d160 con 0x7fcc981033c0 2024-08-30T09:12:23.181 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.179+0000 7fcc9659c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcc98102340 0x7fcc981a0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:12:23.181 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.179+0000 7fcc96d9d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcc98103d60 0x7fcc981a16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:12:23.181 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.179+0000 7fcc9659c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcc98102340 0x7fcc981a0480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:41110/0 (socket says 172.21.15.42:41110) 2024-08-30T09:12:23.181 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.179+0000 7fcc9659c640 1 -- 172.21.15.42:0/1573758518 learned_addr learned my addr 172.21.15.42:0/1573758518 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:12:23.181 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.179+0000 7fcc95d9b640 1 --2- 172.21.15.42:0/1573758518 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcc981033c0 0x7fcc981a0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:12:23.182 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.179+0000 7fcc9659c640 1 -- 172.21.15.42:0/1573758518 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcc981033c0 msgr2=0x7fcc981a0af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:12:23.182 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.179+0000 7fcc9659c640 1 --2- 172.21.15.42:0/1573758518 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcc981033c0 0x7fcc981a0af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:23.182 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.179+0000 7fcc9659c640 1 -- 172.21.15.42:0/1573758518 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcc98103d60 msgr2=0x7fcc981a16a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:12:23.182 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.179+0000 7fcc9659c640 1 --2- 172.21.15.42:0/1573758518 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcc98103d60 0x7fcc981a16a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:23.182 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.179+0000 7fcc9659c640 1 -- 172.21.15.42:0/1573758518 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc981a1ec0 con 0x7fcc98102340 2024-08-30T09:12:23.182 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.179+0000 7fcc9659c640 1 --2- 172.21.15.42:0/1573758518 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcc98102340 0x7fcc981a0480 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fcc9000ba50 tx=0x7fcc9000bf10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:12:23.183 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.180+0000 7fcc877fe640 1 -- 172.21.15.42:0/1573758518 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc90019070 con 0x7fcc98102340 2024-08-30T09:12:23.183 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.180+0000 7fcc877fe640 1 -- 172.21.15.42:0/1573758518 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcc9000cbb0 con 0x7fcc98102340 2024-08-30T09:12:23.184 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.180+0000 7fcc877fe640 1 -- 172.21.15.42:0/1573758518 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc900075d0 con 0x7fcc98102340 2024-08-30T09:12:23.184 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.181+0000 7fcc9cad5640 1 -- 172.21.15.42:0/1573758518 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc981a21b0 con 0x7fcc98102340 2024-08-30T09:12:23.184 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.182+0000 7fcc9cad5640 1 -- 172.21.15.42:0/1573758518 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc98192920 con 0x7fcc98102340 2024-08-30T09:12:23.193 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.185+0000 7fcca3a3e740 1 Processor -- start 2024-08-30T09:12:23.193 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.191+0000 7fcca3a3e740 1 -- start start 2024-08-30T09:12:23.193 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.191+0000 7fcca3a3e740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56126b8f81b0 0x56126bc9df60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:12:23.193 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.191+0000 7fcca3a3e740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56126b8cf2f0 0x56126bc9e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:12:23.193 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.191+0000 7fcca3a3e740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56126b8d74d0 0x56126bc9f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:12:23.193 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.191+0000 7fcca3a3e740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x56126bc11000 con 0x56126b8d74d0 2024-08-30T09:12:23.193 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.191+0000 7fcca3a3e740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x56126bc00590 con 0x56126b8f81b0 2024-08-30T09:12:23.193 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.191+0000 7fcca3a3e740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x56126bc04540 con 0x56126b8cf2f0 2024-08-30T09:12:23.194 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.191+0000 7fcc95d9b640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56126b8d74d0 0x56126bc9f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:12:23.194 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.191+0000 7fcc95d9b640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56126b8d74d0 0x56126bc9f120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38510/0 (socket says 172.21.15.42:38510) 2024-08-30T09:12:23.195 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.191+0000 7fcc95d9b640 1 -- 172.21.15.42:0/2893209431 learned_addr learned my addr 172.21.15.42:0/2893209431 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:12:23.195 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.191+0000 7fcc96d9d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56126b8f81b0 0x56126bc9df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:12:23.195 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.192+0000 7fcc877fe640 1 -- 172.21.15.42:0/1573758518 <== mon.2 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7fcc90014440 con 0x7fcc98102340 2024-08-30T09:12:23.195 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.192+0000 7fcc95d9b640 1 -- 172.21.15.42:0/2893209431 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56126b8cf2f0 msgr2=0x56126bc9e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:12:23.195 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.192+0000 7fcc95d9b640 1 --2- 172.21.15.42:0/2893209431 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56126b8cf2f0 0x56126bc9e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:23.195 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.192+0000 7fcc95d9b640 1 -- 172.21.15.42:0/2893209431 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56126b8f81b0 msgr2=0x56126bc9df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:12:23.195 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.192+0000 7fcc95d9b640 1 --2- 172.21.15.42:0/2893209431 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56126b8f81b0 0x56126bc9df60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:23.195 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.192+0000 7fcc95d9b640 1 -- 172.21.15.42:0/2893209431 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56126b963790 con 0x56126b8d74d0 2024-08-30T09:12:23.195 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.192+0000 7fcc95d9b640 1 --2- 172.21.15.42:0/2893209431 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56126b8d74d0 0x56126bc9f120 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fcc88009a40 tx=0x7fcc88009f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b82285946458647e server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:12:23.195 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.193+0000 7fcc84ff9640 1 -- 172.21.15.42:0/2893209431 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc88017070 con 0x56126b8d74d0 2024-08-30T09:12:23.195 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.193+0000 7fcc84ff9640 1 -- 172.21.15.42:0/2893209431 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcc88013470 con 0x56126b8d74d0 2024-08-30T09:12:23.195 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.193+0000 7fcc84ff9640 1 -- 172.21.15.42:0/2893209431 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc88012630 con 0x56126b8d74d0 2024-08-30T09:12:23.196 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.193+0000 7fcc877fe640 1 --2- 172.21.15.42:0/1573758518 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fcc800b4cc0 0x7fcc800b71d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:12:23.197 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.194+0000 7fcca3a3e740 1 -- 172.21.15.42:0/2893209431 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56126b8d74d0 msgr2=0x56126bc9f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:12:23.197 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.194+0000 7fcca3a3e740 1 --2- 172.21.15.42:0/2893209431 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56126b8d74d0 0x56126bc9f120 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fcc88009a40 tx=0x7fcc88009f00 comp rx=0 tx=0).stop 2024-08-30T09:12:23.197 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.194+0000 7fcc96d9d640 1 --2- 172.21.15.42:0/1573758518 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fcc800b4cc0 0x7fcc800b71d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:12:23.197 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.194+0000 7fcc877fe640 1 -- 172.21.15.42:0/1573758518 <== mon.2 v2:172.21.15.167:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7fcc900e48a0 con 0x7fcc98102340 2024-08-30T09:12:23.197 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.194+0000 7fcca3a3e740 1 -- 172.21.15.42:0/2893209431 shutdown_connections 2024-08-30T09:12:23.197 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.194+0000 7fcca3a3e740 1 --2- 172.21.15.42:0/2893209431 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56126b8cf2f0 0x56126bc9e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:23.197 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.194+0000 7fcca3a3e740 1 --2- 172.21.15.42:0/2893209431 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56126b8d74d0 0x56126bc9f120 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:23.197 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.194+0000 7fcca3a3e740 1 --2- 172.21.15.42:0/2893209431 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56126b8f81b0 0x56126bc9df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:23.197 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.194+0000 7fcca3a3e740 1 -- 172.21.15.42:0/2893209431 >> 172.21.15.42:0/2893209431 conn(0x56126b8ec300 msgr2=0x56126bc999f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:12:23.197 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.194+0000 7fcca3a3e740 1 -- 172.21.15.42:0/2893209431 shutdown_connections 2024-08-30T09:12:23.197 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.194+0000 7fcca3a3e740 1 -- 172.21.15.42:0/2893209431 wait complete. 2024-08-30T09:12:23.197 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.194+0000 7fcc96d9d640 1 --2- 172.21.15.42:0/1573758518 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fcc800b4cc0 0x7fcc800b71d0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fcc8c02ebb0 tx=0x7fcc8c003d60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:12:23.197 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.195+0000 7fcca3a3e740 1 -- start start 2024-08-30T09:12:23.198 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.195+0000 7fcca3a3e740 1 Processor -- start 2024-08-30T09:12:23.200 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.197+0000 7fcca3a3e740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56126b8cf2f0 0x56126bca7ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:12:23.200 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.197+0000 7fcca3a3e740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56126b8d74d0 0x56126bca8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:12:23.200 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.197+0000 7fcca3a3e740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56126b8f81b0 0x56126bca8d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:12:23.200 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.197+0000 7fcca3a3e740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x56126bb39ab0 con 0x56126b8f81b0 2024-08-30T09:12:23.200 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.197+0000 7fcca3a3e740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x56126bb87680 con 0x56126b8cf2f0 2024-08-30T09:12:23.201 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.197+0000 7fcca3a3e740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x56126bc9fc40 con 0x56126b8d74d0 2024-08-30T09:12:23.201 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.197+0000 7fcca3a3e740 10 client.0 _ll_register_callbacks cb 0x56126b90fa00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:12:23.201 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.197+0000 7fcc95d9b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56126b8d74d0 0x56126bca8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:12:23.201 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.197+0000 7fcc95d9b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56126b8d74d0 0x56126bca8190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:59002/0 (socket says 172.21.15.42:59002) 2024-08-30T09:12:23.201 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.197+0000 7fcc95d9b640 1 -- 172.21.15.42:0/2049523874 learned_addr learned my addr 172.21.15.42:0/2049523874 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:12:23.201 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.197+0000 7fcc9659c640 1 --2- 172.21.15.42:0/2049523874 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56126b8cf2f0 0x56126bca7ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:12:23.201 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.198+0000 7fcc95d9b640 1 -- 172.21.15.42:0/2049523874 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56126b8cf2f0 msgr2=0x56126bca7ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:12:23.201 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.198+0000 7fcc95d9b640 1 --2- 172.21.15.42:0/2049523874 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56126b8cf2f0 0x56126bca7ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:23.201 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.198+0000 7fcc95d9b640 1 -- 172.21.15.42:0/2049523874 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56126b8f81b0 msgr2=0x56126bca8d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:12:23.201 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.198+0000 7fcc95d9b640 1 --2- 172.21.15.42:0/2049523874 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56126b8f81b0 0x56126bca8d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:23.201 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.198+0000 7fcc95d9b640 1 -- 172.21.15.42:0/2049523874 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56126b9dc620 con 0x56126b8d74d0 2024-08-30T09:12:23.201 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.198+0000 7fcc9659c640 1 --2- 172.21.15.42:0/2049523874 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56126b8cf2f0 0x56126bca7ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-30T09:12:23.201 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.198+0000 7fcc95d9b640 1 --2- 172.21.15.42:0/2049523874 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56126b8d74d0 0x56126bca8190 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fcc88009380 tx=0x7fcc88013850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=51c69b7bc4fac053 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:12:23.201 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.198+0000 7fcc7dffb640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-08-30T09:12:23.201 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.198+0000 7fcc7dffb640 1 -- 172.21.15.42:0/2049523874 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc88017050 con 0x56126b8d74d0 2024-08-30T09:12:23.202 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.198+0000 7fcc7dffb640 1 -- 172.21.15.42:0/2049523874 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7fcc8801c650 con 0x56126b8d74d0 2024-08-30T09:12:23.202 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.198+0000 7fcca3a3e740 1 -- 172.21.15.42:0/2049523874 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcc88013470 con 0x56126b8d74d0 2024-08-30T09:12:23.202 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.198+0000 7fcc7dffb640 1 -- 172.21.15.42:0/2049523874 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fcc8802f050 con 0x56126b8d74d0 2024-08-30T09:12:23.202 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.198+0000 7fcca3a3e740 10 client.27112 fetch_fsmap learned FSMap version 43 2024-08-30T09:12:23.202 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.198+0000 7fcca3a3e740 1 -- 172.21.15.42:0/2049523874 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x56126bcb1c50 con 0x56126b8d74d0 2024-08-30T09:12:23.204 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.199+0000 7fcc7dffb640 1 -- 172.21.15.42:0/2049523874 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcc88013470 con 0x56126b8d74d0 2024-08-30T09:12:23.204 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.200+0000 7fcc7dffb640 1 -- 172.21.15.42:0/2049523874 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7fcc88026850 con 0x56126b8d74d0 2024-08-30T09:12:23.204 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.200+0000 7fcca3a3e740 10 client.27112 fetch_fsmap finished waiting for FSMap version 43 2024-08-30T09:12:23.204 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.200+0000 7fcca3a3e740 10 client.27112 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-30T09:12:23.204 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.200+0000 7fcca3a3e740 20 client.27112 populate_metadata read hostname 'smithi042' 2024-08-30T09:12:23.204 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.200+0000 7fcca3a3e740 1 --2- 172.21.15.42:0/2049523874 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x56126b8b6d80 0x56126bcb57d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:12:23.204 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.200+0000 7fcca3a3e740 4 client.27112 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-30T09:12:23.204 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.200+0000 7fcca3a3e740 1 -- 172.21.15.42:0/2049523874 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56126bc12f50 con 0x56126b8b6d80 2024-08-30T09:12:23.204 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.200+0000 7fcc9659c640 1 --2- 172.21.15.42:0/2049523874 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x56126b8b6d80 0x56126bcb57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:12:23.204 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.200+0000 7fcc7dffb640 10 client.27112 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:12:23.204 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.200+0000 7fcc9659c640 1 --2- 172.21.15.42:0/2049523874 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x56126b8b6d80 0x56126bcb57d0 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=432d54d61d85535d server_cookie=ae492c8c52f58614 in_seq=0 out_seq=0 2024-08-30T09:12:23.204 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.201+0000 7fcc7dffb640 1 -- 172.21.15.42:0/2049523874 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56126bc12f50 con 0x56126b8b6d80 2024-08-30T09:12:23.205 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.201+0000 7fcc7dffb640 10 client.27112 handle_command_reply: tid=0 2024-08-30T09:12:23.205 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.201+0000 7fcca3a3e740 1 client.27112 shutdown 2024-08-30T09:12:23.205 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.201+0000 7fcca3a3e740 1 -- 172.21.15.42:0/2049523874 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56126b8d74d0 msgr2=0x56126bca8190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:12:23.205 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.201+0000 7fcca3a3e740 1 --2- 172.21.15.42:0/2049523874 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56126b8d74d0 0x56126bca8190 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fcc88009380 tx=0x7fcc88013850 comp rx=0 tx=0).stop 2024-08-30T09:12:23.205 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.201+0000 7fcca3a3e740 1 -- 172.21.15.42:0/2049523874 shutdown_connections 2024-08-30T09:12:23.205 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.201+0000 7fcca3a3e740 1 --2- 172.21.15.42:0/2049523874 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x56126b8b6d80 0x56126bcb57d0 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:23.205 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.201+0000 7fcca3a3e740 1 --2- 172.21.15.42:0/2049523874 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56126b8d74d0 0x56126bca8190 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:23.205 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.201+0000 7fcc7dffb640 0 client.27112 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:12:23.205 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.202+0000 7fcca3a3e740 1 --2- 172.21.15.42:0/2049523874 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56126b8f81b0 0x56126bca8d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:23.205 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.202+0000 7fcca3a3e740 1 --2- 172.21.15.42:0/2049523874 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56126b8cf2f0 0x56126bca7ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:23.205 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.202+0000 7fcca3a3e740 1 -- 172.21.15.42:0/2049523874 >> 172.21.15.42:0/2049523874 conn(0x56126b8ec300 msgr2=0x56126bc97c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:12:23.205 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.202+0000 7fcca3a3e740 1 -- 172.21.15.42:0/2049523874 shutdown_connections 2024-08-30T09:12:23.205 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.202+0000 7fcca3a3e740 1 -- 172.21.15.42:0/2049523874 wait complete. 2024-08-30T09:12:23.206 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.202+0000 7fcca3a3e740 20 client.27112 trim_cache size 0 max 16384 2024-08-30T09:12:23.224 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.222+0000 7fcca3a3e740 1 Processor -- start 2024-08-30T09:12:23.224 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.222+0000 7fcca3a3e740 1 -- start start 2024-08-30T09:12:23.226 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.223+0000 7fcca3a3e740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56126b8b6d80 0x56126bc99330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:12:23.226 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.223+0000 7fcca3a3e740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56126b8cf2f0 0x56126bcafbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:12:23.226 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.223+0000 7fcca3a3e740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56126b8d74d0 0x56126bc99870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:12:23.226 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.223+0000 7fcca3a3e740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fcc981bdc90 con 0x56126b8cf2f0 2024-08-30T09:12:23.226 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.223+0000 7fcca3a3e740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fcc981bb560 con 0x56126b8b6d80 2024-08-30T09:12:23.226 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.223+0000 7fcc9659c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56126b8b6d80 0x56126bc99330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:12:23.226 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.223+0000 7fcc9659c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56126b8b6d80 0x56126bc99330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:41130/0 (socket says 172.21.15.42:41130) 2024-08-30T09:12:23.226 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.223+0000 7fcc9659c640 1 -- 172.21.15.42:0/2219627133 learned_addr learned my addr 172.21.15.42:0/2219627133 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:12:23.226 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.223+0000 7fcc95d9b640 1 --2- 172.21.15.42:0/2219627133 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56126b8cf2f0 0x56126bcafbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:12:23.227 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.223+0000 7fcc96d9d640 1 --2- 172.21.15.42:0/2219627133 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56126b8d74d0 0x56126bc99870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:12:23.227 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.223+0000 7fcca3a3e740 1 -- 172.21.15.42:0/2219627133 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fcc9819abb0 con 0x56126b8d74d0 2024-08-30T09:12:23.227 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.224+0000 7fcc9659c640 1 -- 172.21.15.42:0/2219627133 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56126b8d74d0 msgr2=0x56126bc99870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:12:23.227 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.224+0000 7fcc9659c640 1 --2- 172.21.15.42:0/2219627133 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56126b8d74d0 0x56126bc99870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:23.227 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.224+0000 7fcc9659c640 1 -- 172.21.15.42:0/2219627133 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56126b8cf2f0 msgr2=0x56126bcafbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:12:23.227 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.224+0000 7fcc9659c640 1 --2- 172.21.15.42:0/2219627133 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56126b8cf2f0 0x56126bcafbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:23.227 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.224+0000 7fcc9659c640 1 -- 172.21.15.42:0/2219627133 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56126bca9070 con 0x56126b8b6d80 2024-08-30T09:12:23.227 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.224+0000 7fcc9659c640 1 --2- 172.21.15.42:0/2219627133 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56126b8b6d80 0x56126bc99330 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fcc900a70f0 tx=0x7fcc900e4040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=61c86fa6d514262c server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:12:23.227 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.224+0000 7fcc7ffff640 1 -- 172.21.15.42:0/2219627133 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc900e2120 con 0x56126b8b6d80 2024-08-30T09:12:23.227 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.224+0000 7fcc7ffff640 1 -- 172.21.15.42:0/2219627133 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcc9000cd60 con 0x56126b8b6d80 2024-08-30T09:12:23.227 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.224+0000 7fcc7ffff640 1 -- 172.21.15.42:0/2219627133 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc900b40b0 con 0x56126b8b6d80 2024-08-30T09:12:23.229 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.225+0000 7fcca3a3e740 1 -- 172.21.15.42:0/2219627133 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56126b8b6d80 msgr2=0x56126bc99330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:12:23.229 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.225+0000 7fcca3a3e740 1 --2- 172.21.15.42:0/2219627133 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56126b8b6d80 0x56126bc99330 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fcc900a70f0 tx=0x7fcc900e4040 comp rx=0 tx=0).stop 2024-08-30T09:12:23.229 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.225+0000 7fcca3a3e740 1 -- 172.21.15.42:0/2219627133 shutdown_connections 2024-08-30T09:12:23.229 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.225+0000 7fcca3a3e740 1 --2- 172.21.15.42:0/2219627133 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56126b8d74d0 0x56126bc99870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:23.229 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.225+0000 7fcca3a3e740 1 --2- 172.21.15.42:0/2219627133 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56126b8cf2f0 0x56126bcafbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:23.229 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.225+0000 7fcca3a3e740 1 --2- 172.21.15.42:0/2219627133 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56126b8b6d80 0x56126bc99330 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:23.229 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.225+0000 7fcca3a3e740 1 -- 172.21.15.42:0/2219627133 >> 172.21.15.42:0/2219627133 conn(0x56126b8ec300 msgr2=0x56126bc9dc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:12:23.229 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.225+0000 7fcca3a3e740 1 -- 172.21.15.42:0/2219627133 shutdown_connections 2024-08-30T09:12:23.229 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.225+0000 7fcca3a3e740 1 -- 172.21.15.42:0/2219627133 wait complete. 2024-08-30T09:12:23.230 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.225+0000 7fcca3a3e740 1 -- start start 2024-08-30T09:12:23.230 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.226+0000 7fcca3a3e740 1 Processor -- start 2024-08-30T09:12:23.230 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.226+0000 7fcca3a3e740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56126b8b6d80 0x56126bcace20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:12:23.230 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.226+0000 7fcca3a3e740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56126b8cf2f0 0x56126bcad4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:12:23.230 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.226+0000 7fcca3a3e740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56126b8d74d0 0x56126bcae050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:12:23.230 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.226+0000 7fcca3a3e740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fcc981b9a20 con 0x56126b8cf2f0 2024-08-30T09:12:23.230 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.226+0000 7fcca3a3e740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fcc981bae90 con 0x56126b8b6d80 2024-08-30T09:12:23.230 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.226+0000 7fcca3a3e740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fcc981ba0f0 con 0x56126b8d74d0 2024-08-30T09:12:23.230 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.226+0000 7fcc9659c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56126b8b6d80 0x56126bcace20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:12:23.230 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.226+0000 7fcc9659c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56126b8b6d80 0x56126bcace20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:41132/0 (socket says 172.21.15.42:41132) 2024-08-30T09:12:23.230 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.226+0000 7fcc9659c640 1 -- 172.21.15.42:0/3541679516 learned_addr learned my addr 172.21.15.42:0/3541679516 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:12:23.230 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.226+0000 7fcca3a3e740 10 client.0 _ll_register_callbacks cb 0x7fcc8c005ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:12:23.230 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.226+0000 7fcc96d9d640 1 --2- 172.21.15.42:0/3541679516 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56126b8d74d0 0x56126bcae050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:12:23.231 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.226+0000 7fcc95d9b640 1 --2- 172.21.15.42:0/3541679516 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56126b8cf2f0 0x56126bcad4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:12:23.231 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.226+0000 7fcc96d9d640 1 -- 172.21.15.42:0/3541679516 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56126b8b6d80 msgr2=0x56126bcace20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:12:23.231 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.226+0000 7fcc96d9d640 1 --2- 172.21.15.42:0/3541679516 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56126b8b6d80 0x56126bcace20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:23.231 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.226+0000 7fcc96d9d640 1 -- 172.21.15.42:0/3541679516 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56126b8cf2f0 msgr2=0x56126bcad4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:12:23.231 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.226+0000 7fcc96d9d640 1 --2- 172.21.15.42:0/3541679516 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56126b8cf2f0 0x56126bcad4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:23.231 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.226+0000 7fcc96d9d640 1 -- 172.21.15.42:0/3541679516 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56126bcae7b0 con 0x56126b8d74d0 2024-08-30T09:12:23.231 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.227+0000 7fcc95d9b640 1 --2- 172.21.15.42:0/3541679516 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56126b8cf2f0 0x56126bcad4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-30T09:12:23.233 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.229+0000 7fcc96d9d640 1 --2- 172.21.15.42:0/3541679516 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56126b8d74d0 0x56126bcae050 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fcc8c000e60 tx=0x7fcc8c02e5c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c4a215f9481c1f29 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:12:23.233 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.229+0000 7fcc7dffb640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-08-30T09:12:23.234 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.229+0000 7fcc7dffb640 1 -- 172.21.15.42:0/3541679516 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc8c003e90 con 0x56126b8d74d0 2024-08-30T09:12:23.234 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.229+0000 7fcc7dffb640 1 -- 172.21.15.42:0/3541679516 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7fcc8c004030 con 0x56126b8d74d0 2024-08-30T09:12:23.234 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.229+0000 7fcca3a3e740 1 -- 172.21.15.42:0/3541679516 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcc9000cd60 con 0x56126b8d74d0 2024-08-30T09:12:23.234 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.229+0000 7fcc7dffb640 1 -- 172.21.15.42:0/3541679516 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fcc8c004230 con 0x56126b8d74d0 2024-08-30T09:12:23.234 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.229+0000 7fcca3a3e740 10 client.27118 fetch_fsmap learned FSMap version 43 2024-08-30T09:12:23.234 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.230+0000 7fcca3a3e740 1 -- 172.21.15.42:0/3541679516 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x56126bcafb20 con 0x56126b8d74d0 2024-08-30T09:12:23.234 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.230+0000 7fcc7dffb640 1 -- 172.21.15.42:0/3541679516 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcc8c052070 con 0x56126b8d74d0 2024-08-30T09:12:23.234 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.230+0000 7fcc7dffb640 1 -- 172.21.15.42:0/3541679516 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7fcc8c004830 con 0x56126b8d74d0 2024-08-30T09:12:23.234 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.230+0000 7fcca3a3e740 10 client.27118 fetch_fsmap finished waiting for FSMap version 43 2024-08-30T09:12:23.234 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.230+0000 7fcca3a3e740 10 client.27118 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-30T09:12:23.234 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.230+0000 7fcca3a3e740 20 client.27118 populate_metadata read hostname 'smithi042' 2024-08-30T09:12:23.234 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.230+0000 7fcca3a3e740 1 --2- 172.21.15.42:0/3541679516 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x56126b8f81b0 0x56126bd225a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:12:23.234 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.230+0000 7fcca3a3e740 4 client.27118 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-30T09:12:23.235 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.230+0000 7fcca3a3e740 1 -- 172.21.15.42:0/3541679516 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7fcc9000cd60 con 0x56126b8f81b0 2024-08-30T09:12:23.235 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.230+0000 7fcc9659c640 1 --2- 172.21.15.42:0/3541679516 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x56126b8f81b0 0x56126bd225a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:12:23.235 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.231+0000 7fcc9659c640 1 --2- 172.21.15.42:0/3541679516 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x56126b8f81b0 0x56126bd225a0 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c30c3af623b845ab server_cookie=34f455dfd6932c5e in_seq=0 out_seq=0 2024-08-30T09:12:23.235 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.231+0000 7fcc7dffb640 10 client.27118 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:12:23.273 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.271+0000 7fcc7dffb640 1 -- 172.21.15.42:0/3541679516 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7fcc9000cd60 con 0x56126b8f81b0 2024-08-30T09:12:23.273 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.271+0000 7fcc7dffb640 10 client.27118 handle_command_reply: tid=0 2024-08-30T09:12:23.275 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.273+0000 7fcca3a3e740 1 client.27118 shutdown 2024-08-30T09:12:23.286 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.284+0000 7fcca3a3e740 1 -- 172.21.15.42:0/3541679516 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56126b8d74d0 msgr2=0x56126bcae050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:12:23.287 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.284+0000 7fcca3a3e740 1 --2- 172.21.15.42:0/3541679516 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56126b8d74d0 0x56126bcae050 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fcc8c000e60 tx=0x7fcc8c02e5c0 comp rx=0 tx=0).stop 2024-08-30T09:12:23.290 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.289+0000 7fcca3a3e740 1 -- 172.21.15.42:0/3541679516 shutdown_connections 2024-08-30T09:12:23.290 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.289+0000 7fcca3a3e740 1 --2- 172.21.15.42:0/3541679516 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x56126b8f81b0 0x56126bd225a0 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:23.291 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.289+0000 7fcca3a3e740 1 --2- 172.21.15.42:0/3541679516 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56126b8d74d0 0x56126bcae050 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:23.291 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.289+0000 7fcca3a3e740 1 --2- 172.21.15.42:0/3541679516 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56126b8cf2f0 0x56126bcad4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:23.291 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.289+0000 7fcca3a3e740 1 --2- 172.21.15.42:0/3541679516 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56126b8b6d80 0x56126bcace20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:23.291 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.289+0000 7fcca3a3e740 1 -- 172.21.15.42:0/3541679516 >> 172.21.15.42:0/3541679516 conn(0x56126b8ec300 msgr2=0x56126bca29d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:12:23.291 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.289+0000 7fcc7dffb640 0 client.27118 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:12:23.297 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.295+0000 7fcca3a3e740 1 -- 172.21.15.42:0/3541679516 shutdown_connections 2024-08-30T09:12:23.301 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.299+0000 7fcca3a3e740 1 -- 172.21.15.42:0/3541679516 wait complete. 2024-08-30T09:12:23.303 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.301+0000 7fcca3a3e740 20 client.27118 trim_cache size 0 max 16384 2024-08-30T09:12:23.307 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-08-30T09:12:23.307 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2024-08-30T09:12:23.308 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-08-30T09:12:23.308 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-08-30T09:12:23.316 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.312+0000 7fcc9cad5640 1 -- 172.21.15.42:0/1573758518 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fcc800b4cc0 msgr2=0x7fcc800b71d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:12:23.316 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.312+0000 7fcc9cad5640 1 --2- 172.21.15.42:0/1573758518 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fcc800b4cc0 0x7fcc800b71d0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fcc8c02ebb0 tx=0x7fcc8c003d60 comp rx=0 tx=0).stop 2024-08-30T09:12:23.316 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.312+0000 7fcc9cad5640 1 -- 172.21.15.42:0/1573758518 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcc98102340 msgr2=0x7fcc981a0480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:12:23.316 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.312+0000 7fcc9cad5640 1 --2- 172.21.15.42:0/1573758518 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcc98102340 0x7fcc981a0480 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fcc9000ba50 tx=0x7fcc9000bf10 comp rx=0 tx=0).stop 2024-08-30T09:12:23.317 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.314+0000 7fcc9cad5640 1 -- 172.21.15.42:0/1573758518 shutdown_connections 2024-08-30T09:12:23.317 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.314+0000 7fcc9cad5640 1 --2- 172.21.15.42:0/1573758518 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fcc800b4cc0 0x7fcc800b71d0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:23.317 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.314+0000 7fcc9cad5640 1 --2- 172.21.15.42:0/1573758518 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcc981033c0 0x7fcc981a0af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:23.317 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.314+0000 7fcc9cad5640 1 --2- 172.21.15.42:0/1573758518 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcc98103d60 0x7fcc981a16a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:23.317 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.314+0000 7fcc9cad5640 1 --2- 172.21.15.42:0/1573758518 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcc98102340 0x7fcc981a0480 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:23.317 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.314+0000 7fcc9cad5640 1 -- 172.21.15.42:0/1573758518 >> 172.21.15.42:0/1573758518 conn(0x7fcc980fdc80 msgr2=0x7fcc981066e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:12:23.321 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.319+0000 7fcc9cad5640 1 -- 172.21.15.42:0/1573758518 shutdown_connections 2024-08-30T09:12:23.323 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:23.322+0000 7fcc9cad5640 1 -- 172.21.15.42:0/1573758518 wait complete. 2024-08-30T09:12:24.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:12:24 smithi042 ceph-mon[18652]: pgmap v722: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 6.7 MiB/s wr, 595 op/s 2024-08-30T09:12:24.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:12:24 smithi190 ceph-mon[22001]: pgmap v722: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 6.7 MiB/s wr, 595 op/s 2024-08-30T09:12:24.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:12:24 smithi167 ceph-mon[22063]: pgmap v722: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 6.7 MiB/s wr, 595 op/s 2024-08-30T09:12:25.953 INFO:teuthology.orchestra.run.smithi042.stderr:progress: 700.0 s, 530.0 tps, lat 192.339 ms stddev 198.651 2024-08-30T09:12:26.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:12:26 smithi042 ceph-mon[18652]: pgmap v723: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 5.7 MiB/s wr, 499 op/s 2024-08-30T09:12:26.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:12:26 smithi190 ceph-mon[22001]: pgmap v723: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 5.7 MiB/s wr, 499 op/s 2024-08-30T09:12:26.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:12:26 smithi167 ceph-mon[22063]: pgmap v723: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 5.7 MiB/s wr, 499 op/s 2024-08-30T09:12:27.640 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-30T09:12:28.229 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T09:12:28.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:12:28 smithi042 ceph-mon[18652]: pgmap v724: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 5.1 MiB/s wr, 483 op/s 2024-08-30T09:12:28.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:12:28 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-30T09:12:28.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:12:28 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T09:12:28.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:12:28 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T09:12:28.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:12:28 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T09:12:28.634 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:12:28 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T09:12:28.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:12:28 smithi190 ceph-mon[22001]: pgmap v724: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 5.1 MiB/s wr, 483 op/s 2024-08-30T09:12:28.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:12:28 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-30T09:12:28.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:12:28 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T09:12:28.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:12:28 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T09:12:28.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:12:28 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T09:12:28.687 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:12:28 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T09:12:28.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:12:28 smithi167 ceph-mon[22063]: pgmap v724: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 5.1 MiB/s wr, 483 op/s 2024-08-30T09:12:28.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:12:28 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-30T09:12:28.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:12:28 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T09:12:28.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:12:28 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T09:12:28.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:12:28 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T09:12:28.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:12:28 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T09:12:29.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.858+0000 7f9e1f2c7640 1 -- 172.21.15.42:0/3197481470 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9e18108180 msgr2=0x7f9e18108590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:12:29.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.858+0000 7f9e1f2c7640 1 --2- 172.21.15.42:0/3197481470 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9e18108180 0x7f9e18108590 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f9e08009900 tx=0x7f9e0802a3f0 comp rx=0 tx=0).stop 2024-08-30T09:12:29.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.858+0000 7f9e1f2c7640 1 -- 172.21.15.42:0/3197481470 shutdown_connections 2024-08-30T09:12:29.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.858+0000 7f9e1f2c7640 1 --2- 172.21.15.42:0/3197481470 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9e18108ad0 0x7f9e1810f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:29.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.858+0000 7f9e1f2c7640 1 --2- 172.21.15.42:0/3197481470 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9e18108180 0x7f9e18108590 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:29.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.858+0000 7f9e1f2c7640 1 --2- 172.21.15.42:0/3197481470 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9e18102840 0x7f9e18102c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:29.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.858+0000 7f9e1f2c7640 1 -- 172.21.15.42:0/3197481470 >> 172.21.15.42:0/3197481470 conn(0x7f9e180794f0 msgr2=0x7f9e18079900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:12:29.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.858+0000 7f9e1f2c7640 1 -- 172.21.15.42:0/3197481470 shutdown_connections 2024-08-30T09:12:29.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.858+0000 7f9e1f2c7640 1 -- 172.21.15.42:0/3197481470 wait complete. 2024-08-30T09:12:29.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.859+0000 7f9e1f2c7640 1 Processor -- start 2024-08-30T09:12:29.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.859+0000 7f9e1f2c7640 1 -- start start 2024-08-30T09:12:29.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.859+0000 7f9e1f2c7640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9e18102840 0x7f9e1810b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:12:29.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.859+0000 7f9e1f2c7640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9e18108180 0x7f9e1810bd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:12:29.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.859+0000 7f9e1f2c7640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9e18108ad0 0x7f9e1810c260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:12:29.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.860+0000 7f9e1f2c7640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f9e18111680 con 0x7f9e18102840 2024-08-30T09:12:29.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.860+0000 7f9e1f2c7640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f9e18111500 con 0x7f9e18108180 2024-08-30T09:12:29.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.860+0000 7f9e1f2c7640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f9e18111800 con 0x7f9e18108ad0 2024-08-30T09:12:29.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.860+0000 7f9e1d864640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9e18108ad0 0x7f9e1810c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:12:29.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.860+0000 7f9e1d864640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9e18108ad0 0x7f9e1810c260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:32982/0 (socket says 172.21.15.42:32982) 2024-08-30T09:12:29.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.860+0000 7f9e1d864640 1 -- 172.21.15.42:0/2234112981 learned_addr learned my addr 172.21.15.42:0/2234112981 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:12:29.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.860+0000 7f9e1c862640 1 --2- 172.21.15.42:0/2234112981 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9e18108180 0x7f9e1810bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:12:29.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.860+0000 7f9e1d063640 1 --2- 172.21.15.42:0/2234112981 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9e18102840 0x7f9e1810b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:12:29.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.860+0000 7f9e1d864640 1 -- 172.21.15.42:0/2234112981 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9e18108180 msgr2=0x7f9e1810bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:12:29.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.860+0000 7f9e1d864640 1 --2- 172.21.15.42:0/2234112981 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9e18108180 0x7f9e1810bd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:29.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.860+0000 7f9e1d864640 1 -- 172.21.15.42:0/2234112981 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9e18102840 msgr2=0x7f9e1810b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:12:29.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.860+0000 7f9e1d864640 1 --2- 172.21.15.42:0/2234112981 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9e18102840 0x7f9e1810b7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:29.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.860+0000 7f9e1d864640 1 -- 172.21.15.42:0/2234112981 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e18117310 con 0x7f9e18108ad0 2024-08-30T09:12:29.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.861+0000 7f9e1d063640 1 --2- 172.21.15.42:0/2234112981 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9e18102840 0x7f9e1810b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-30T09:12:29.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.861+0000 7f9e1d864640 1 --2- 172.21.15.42:0/2234112981 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9e18108ad0 0x7f9e1810c260 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f9e1000cce0 tx=0x7f9e1000a620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:12:29.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.861+0000 7f9e0e7fc640 1 -- 172.21.15.42:0/2234112981 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e1000f450 con 0x7f9e18108ad0 2024-08-30T09:12:29.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.861+0000 7f9e0e7fc640 1 -- 172.21.15.42:0/2234112981 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9e1000faa0 con 0x7f9e18108ad0 2024-08-30T09:12:29.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.861+0000 7f9e1f2c7640 1 -- 172.21.15.42:0/2234112981 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e18117600 con 0x7f9e18108ad0 2024-08-30T09:12:29.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.861+0000 7f9e0e7fc640 1 -- 172.21.15.42:0/2234112981 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e10018b70 con 0x7f9e18108ad0 2024-08-30T09:12:29.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.862+0000 7f9e1f2c7640 1 -- 172.21.15.42:0/2234112981 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e18117b10 con 0x7f9e18108ad0 2024-08-30T09:12:29.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.865+0000 7f9e0e7fc640 1 -- 172.21.15.42:0/2234112981 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7f9e10018d10 con 0x7f9e18108ad0 2024-08-30T09:12:29.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.866+0000 7f9e0e7fc640 1 --2- 172.21.15.42:0/2234112981 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f9e040b4bb0 0x7f9e040b70c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:12:29.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.867+0000 7f9e0e7fc640 1 -- 172.21.15.42:0/2234112981 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f9e100e58a0 con 0x7f9e18108ad0 2024-08-30T09:12:29.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.867+0000 7f9e1d063640 1 --2- 172.21.15.42:0/2234112981 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f9e040b4bb0 0x7f9e040b70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:12:29.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.867+0000 7f9e2583c740 1 Processor -- start 2024-08-30T09:12:29.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.868+0000 7f9e2583c740 1 -- start start 2024-08-30T09:12:29.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.868+0000 7f9e1d063640 1 --2- 172.21.15.42:0/2234112981 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f9e040b4bb0 0x7f9e040b70c0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f9e14005fd0 tx=0x7f9e14005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:12:29.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.868+0000 7f9e2583c740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5587b9b72070 0x5587b9f172c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:12:29.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.868+0000 7f9e2583c740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5587b9b491b0 0x5587b9f17800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:12:29.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.868+0000 7f9e2583c740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5587b9b51390 0x5587b9f17d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:12:29.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.868+0000 7f9e2583c740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x5587b9e8e630 con 0x5587b9b72070 2024-08-30T09:12:29.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.868+0000 7f9e2583c740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5587b9e7dbc0 con 0x5587b9b51390 2024-08-30T09:12:29.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.868+0000 7f9e2583c740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5587b9e81b70 con 0x5587b9b491b0 2024-08-30T09:12:29.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.868+0000 7f9e1c862640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5587b9b491b0 0x5587b9f17800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:12:29.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.868+0000 7f9e1c862640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5587b9b491b0 0x5587b9f17800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:32992/0 (socket says 172.21.15.42:32992) 2024-08-30T09:12:29.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.868+0000 7f9e1c862640 1 -- 172.21.15.42:0/1008460427 learned_addr learned my addr 172.21.15.42:0/1008460427 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:12:29.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.869+0000 7f9e1d864640 1 --2- 172.21.15.42:0/1008460427 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5587b9b51390 0x5587b9f17d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:12:29.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.870+0000 7f9e1d063640 1 --2- 172.21.15.42:0/1008460427 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5587b9b72070 0x5587b9f172c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:12:29.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.870+0000 7f9e1c862640 1 -- 172.21.15.42:0/1008460427 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5587b9b51390 msgr2=0x5587b9f17d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:12:29.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.870+0000 7f9e1c862640 1 --2- 172.21.15.42:0/1008460427 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5587b9b51390 0x5587b9f17d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:29.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.870+0000 7f9e1c862640 1 -- 172.21.15.42:0/1008460427 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5587b9b72070 msgr2=0x5587b9f172c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:12:29.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.870+0000 7f9e1c862640 1 --2- 172.21.15.42:0/1008460427 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5587b9b72070 0x5587b9f172c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:29.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.870+0000 7f9e1c862640 1 -- 172.21.15.42:0/1008460427 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5587b9bdd650 con 0x5587b9b491b0 2024-08-30T09:12:29.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.871+0000 7f9e1c862640 1 --2- 172.21.15.42:0/1008460427 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5587b9b491b0 0x5587b9f17800 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f9e08005ab0 tx=0x7f9e08004390 comp rx=0 tx=0).ready entity=mon.1 client_cookie=18538a49c7ea31ff server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:12:29.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.871+0000 7f9e037fe640 1 -- 172.21.15.42:0/1008460427 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e08035040 con 0x5587b9b491b0 2024-08-30T09:12:29.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.871+0000 7f9e037fe640 1 -- 172.21.15.42:0/1008460427 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9e08004750 con 0x5587b9b491b0 2024-08-30T09:12:29.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.871+0000 7f9e037fe640 1 -- 172.21.15.42:0/1008460427 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e08003850 con 0x5587b9b491b0 2024-08-30T09:12:29.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.872+0000 7f9e2583c740 1 -- 172.21.15.42:0/1008460427 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5587b9b491b0 msgr2=0x5587b9f17800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:12:29.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.872+0000 7f9e2583c740 1 --2- 172.21.15.42:0/1008460427 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5587b9b491b0 0x5587b9f17800 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f9e08005ab0 tx=0x7f9e08004390 comp rx=0 tx=0).stop 2024-08-30T09:12:29.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.872+0000 7f9e2583c740 1 -- 172.21.15.42:0/1008460427 shutdown_connections 2024-08-30T09:12:29.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.872+0000 7f9e2583c740 1 --2- 172.21.15.42:0/1008460427 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5587b9b491b0 0x5587b9f17800 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:29.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.872+0000 7f9e2583c740 1 --2- 172.21.15.42:0/1008460427 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5587b9b51390 0x5587b9f17d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:29.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.872+0000 7f9e2583c740 1 --2- 172.21.15.42:0/1008460427 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5587b9b72070 0x5587b9f172c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:29.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.872+0000 7f9e2583c740 1 -- 172.21.15.42:0/1008460427 >> 172.21.15.42:0/1008460427 conn(0x5587b9b66460 msgr2=0x5587b9f151b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:12:29.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.873+0000 7f9e2583c740 1 -- 172.21.15.42:0/1008460427 shutdown_connections 2024-08-30T09:12:29.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.873+0000 7f9e2583c740 1 -- 172.21.15.42:0/1008460427 wait complete. 2024-08-30T09:12:29.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.873+0000 7f9e2583c740 1 -- start start 2024-08-30T09:12:29.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.873+0000 7f9e2583c740 1 Processor -- start 2024-08-30T09:12:29.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.874+0000 7f9e2583c740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5587b9b491b0 0x5587b9f22a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:12:29.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.874+0000 7f9e2583c740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5587b9b51390 0x5587b9f22fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:12:29.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.874+0000 7f9e2583c740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5587b9b72070 0x5587b9f2ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:12:29.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.874+0000 7f9e1c862640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5587b9b51390 0x5587b9f22fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:12:29.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.875+0000 7f9e1c862640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5587b9b51390 0x5587b9f22fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:42436/0 (socket says 172.21.15.42:42436) 2024-08-30T09:12:29.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.875+0000 7f9e1c862640 1 -- 172.21.15.42:0/710052268 learned_addr learned my addr 172.21.15.42:0/710052268 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:12:29.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.875+0000 7f9e1d063640 1 --2- 172.21.15.42:0/710052268 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5587b9b491b0 0x5587b9f22a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:12:29.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.875+0000 7f9e2583c740 1 -- 172.21.15.42:0/710052268 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x5587b9ce0700 con 0x5587b9b491b0 2024-08-30T09:12:29.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.875+0000 7f9e2583c740 1 -- 172.21.15.42:0/710052268 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5587b9e04700 con 0x5587b9b51390 2024-08-30T09:12:29.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.875+0000 7f9e2583c740 1 -- 172.21.15.42:0/710052268 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5587b9f25a40 con 0x5587b9b72070 2024-08-30T09:12:29.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.875+0000 7f9e1d864640 1 --2- 172.21.15.42:0/710052268 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5587b9b72070 0x5587b9f2ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:12:29.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.875+0000 7f9e2583c740 10 client.0 _ll_register_callbacks cb 0x5587b9e33dc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:12:29.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.875+0000 7f9e1d864640 1 -- 172.21.15.42:0/710052268 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5587b9b51390 msgr2=0x5587b9f22fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:12:29.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.875+0000 7f9e1d864640 1 --2- 172.21.15.42:0/710052268 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5587b9b51390 0x5587b9f22fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:29.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.875+0000 7f9e1d864640 1 -- 172.21.15.42:0/710052268 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5587b9b491b0 msgr2=0x5587b9f22a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:12:29.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.875+0000 7f9e1d864640 1 --2- 172.21.15.42:0/710052268 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5587b9b491b0 0x5587b9f22a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:29.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.875+0000 7f9e1d864640 1 -- 172.21.15.42:0/710052268 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5587b9e3d8b0 con 0x5587b9b72070 2024-08-30T09:12:29.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.875+0000 7f9e1c862640 1 --2- 172.21.15.42:0/710052268 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5587b9b51390 0x5587b9f22fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-30T09:12:29.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.875+0000 7f9e1d063640 1 --2- 172.21.15.42:0/710052268 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5587b9b491b0 0x5587b9f22a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-30T09:12:29.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.876+0000 7f9e1d864640 1 --2- 172.21.15.42:0/710052268 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5587b9b72070 0x5587b9f2ed60 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f9e1000ccb0 tx=0x7f9e100ab770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f943fd0f5829e6cd server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:12:29.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.876+0000 7f9e00ff9640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-08-30T09:12:29.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.876+0000 7f9e00ff9640 1 -- 172.21.15.42:0/710052268 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e100b07b0 con 0x5587b9b72070 2024-08-30T09:12:29.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.876+0000 7f9e00ff9640 1 -- 172.21.15.42:0/710052268 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f9e100971b0 con 0x5587b9b72070 2024-08-30T09:12:29.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.876+0000 7f9e2583c740 1 -- 172.21.15.42:0/710052268 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9e08004750 con 0x5587b9b72070 2024-08-30T09:12:29.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.876+0000 7f9e00ff9640 1 -- 172.21.15.42:0/710052268 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f9e100b5260 con 0x5587b9b72070 2024-08-30T09:12:29.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.876+0000 7f9e2583c740 10 client.27130 fetch_fsmap learned FSMap version 43 2024-08-30T09:12:29.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.876+0000 7f9e2583c740 1 -- 172.21.15.42:0/710052268 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x5587b9f318b0 con 0x5587b9b72070 2024-08-30T09:12:29.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.876+0000 7f9e00ff9640 1 -- 172.21.15.42:0/710052268 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9e100b4dd0 con 0x5587b9b72070 2024-08-30T09:12:29.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.877+0000 7f9e00ff9640 1 -- 172.21.15.42:0/710052268 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f9e100ab1a0 con 0x5587b9b72070 2024-08-30T09:12:29.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.877+0000 7f9e2583c740 10 client.27130 fetch_fsmap finished waiting for FSMap version 43 2024-08-30T09:12:29.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.877+0000 7f9e2583c740 10 client.27130 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-30T09:12:29.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.877+0000 7f9e2583c740 20 client.27130 populate_metadata read hostname 'smithi042' 2024-08-30T09:12:29.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.877+0000 7f9e2583c740 1 --2- 172.21.15.42:0/710052268 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x5587b9b309f0 0x5587b9f35250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:12:29.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.877+0000 7f9e2583c740 4 client.27130 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-30T09:12:29.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.877+0000 7f9e2583c740 1 -- 172.21.15.42:0/710052268 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f9e08004750 con 0x5587b9b309f0 2024-08-30T09:12:29.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.877+0000 7f9e1d063640 1 --2- 172.21.15.42:0/710052268 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x5587b9b309f0 0x5587b9f35250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:12:29.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.877+0000 7f9e00ff9640 10 client.27130 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:12:29.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.877+0000 7f9e1d063640 1 --2- 172.21.15.42:0/710052268 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x5587b9b309f0 0x5587b9f35250 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5725be4c014b45f5 server_cookie=78aa34f37a4dace7 in_seq=0 out_seq=0 2024-08-30T09:12:29.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.878+0000 7f9e00ff9640 1 -- 172.21.15.42:0/710052268 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f9e08004750 con 0x5587b9b309f0 2024-08-30T09:12:29.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.878+0000 7f9e00ff9640 10 client.27130 handle_command_reply: tid=0 2024-08-30T09:12:29.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.878+0000 7f9e2583c740 1 client.27130 shutdown 2024-08-30T09:12:29.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.879+0000 7f9e2583c740 1 -- 172.21.15.42:0/710052268 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5587b9b72070 msgr2=0x5587b9f2ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:12:29.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.879+0000 7f9e2583c740 1 --2- 172.21.15.42:0/710052268 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5587b9b72070 0x5587b9f2ed60 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f9e1000ccb0 tx=0x7f9e100ab770 comp rx=0 tx=0).stop 2024-08-30T09:12:29.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.879+0000 7f9e2583c740 1 -- 172.21.15.42:0/710052268 shutdown_connections 2024-08-30T09:12:29.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.879+0000 7f9e2583c740 1 --2- 172.21.15.42:0/710052268 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x5587b9b309f0 0x5587b9f35250 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:29.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.879+0000 7f9e2583c740 1 --2- 172.21.15.42:0/710052268 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5587b9b72070 0x5587b9f2ed60 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:29.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.879+0000 7f9e2583c740 1 --2- 172.21.15.42:0/710052268 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5587b9b51390 0x5587b9f22fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:29.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.879+0000 7f9e2583c740 1 --2- 172.21.15.42:0/710052268 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5587b9b491b0 0x5587b9f22a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:29.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.879+0000 7f9e2583c740 1 -- 172.21.15.42:0/710052268 >> 172.21.15.42:0/710052268 conn(0x5587b9b66460 msgr2=0x5587b9f1a4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:12:29.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.879+0000 7f9e00ff9640 0 client.27130 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:12:29.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.879+0000 7f9e2583c740 1 -- 172.21.15.42:0/710052268 shutdown_connections 2024-08-30T09:12:29.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.879+0000 7f9e2583c740 1 -- 172.21.15.42:0/710052268 wait complete. 2024-08-30T09:12:29.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.879+0000 7f9e2583c740 20 client.27130 trim_cache size 0 max 16384 2024-08-30T09:12:29.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.910+0000 7f9e2583c740 1 Processor -- start 2024-08-30T09:12:29.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.910+0000 7f9e2583c740 1 -- start start 2024-08-30T09:12:29.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.911+0000 7f9e2583c740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5587b9b491b0 0x5587b9f212e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:12:29.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.911+0000 7f9e2583c740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5587b9b51390 0x5587b9f21820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:12:29.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.911+0000 7f9e2583c740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5587b9b72070 0x5587b9f1a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:12:29.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.911+0000 7f9e2583c740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f9e181c1df0 con 0x5587b9b51390 2024-08-30T09:12:29.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.911+0000 7f9e1c862640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5587b9b51390 0x5587b9f21820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:12:29.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.911+0000 7f9e1d063640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5587b9b491b0 0x5587b9f212e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:12:29.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.911+0000 7f9e1d063640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5587b9b491b0 0x5587b9f212e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:42448/0 (socket says 172.21.15.42:42448) 2024-08-30T09:12:29.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.911+0000 7f9e1d063640 1 -- 172.21.15.42:0/2633337742 learned_addr learned my addr 172.21.15.42:0/2633337742 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:12:29.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.911+0000 7f9e1c862640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5587b9b51390 0x5587b9f21820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41374/0 (socket says 172.21.15.42:41374) 2024-08-30T09:12:29.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.911+0000 7f9e1d864640 1 --2- 172.21.15.42:0/2633337742 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5587b9b72070 0x5587b9f1a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:12:29.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.911+0000 7f9e2583c740 1 -- 172.21.15.42:0/2633337742 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f9e181bf6c0 con 0x5587b9b491b0 2024-08-30T09:12:29.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.911+0000 7f9e2583c740 1 -- 172.21.15.42:0/2633337742 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f9e1811f7a0 con 0x5587b9b72070 2024-08-30T09:12:29.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.912+0000 7f9e1c862640 1 -- 172.21.15.42:0/2633337742 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5587b9b72070 msgr2=0x5587b9f1a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:12:29.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.912+0000 7f9e1c862640 1 --2- 172.21.15.42:0/2633337742 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5587b9b72070 0x5587b9f1a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:29.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.912+0000 7f9e1c862640 1 -- 172.21.15.42:0/2633337742 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5587b9b491b0 msgr2=0x5587b9f212e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:12:29.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.912+0000 7f9e1c862640 1 --2- 172.21.15.42:0/2633337742 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5587b9b491b0 0x5587b9f212e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:29.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.912+0000 7f9e1c862640 1 -- 172.21.15.42:0/2633337742 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5587b9f20f60 con 0x5587b9b51390 2024-08-30T09:12:29.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.912+0000 7f9e1d063640 1 --2- 172.21.15.42:0/2633337742 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5587b9b491b0 0x5587b9f212e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-30T09:12:29.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.912+0000 7f9e1d864640 1 --2- 172.21.15.42:0/2633337742 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5587b9b72070 0x5587b9f1a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-30T09:12:29.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.912+0000 7f9e1c862640 1 --2- 172.21.15.42:0/2633337742 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5587b9b51390 0x5587b9f21820 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f9e08009900 tx=0x7f9e08000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e1a0eee692eb9bf server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:12:29.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.912+0000 7f9e02ffd640 1 -- 172.21.15.42:0/2633337742 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e08035070 con 0x5587b9b51390 2024-08-30T09:12:29.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.913+0000 7f9e02ffd640 1 -- 172.21.15.42:0/2633337742 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9e08004750 con 0x5587b9b51390 2024-08-30T09:12:29.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.913+0000 7f9e02ffd640 1 -- 172.21.15.42:0/2633337742 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e080316a0 con 0x5587b9b51390 2024-08-30T09:12:29.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.913+0000 7f9e2583c740 1 -- 172.21.15.42:0/2633337742 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5587b9b51390 msgr2=0x5587b9f21820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:12:29.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.913+0000 7f9e2583c740 1 --2- 172.21.15.42:0/2633337742 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5587b9b51390 0x5587b9f21820 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f9e08009900 tx=0x7f9e08000bc0 comp rx=0 tx=0).stop 2024-08-30T09:12:29.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.914+0000 7f9e2583c740 1 -- 172.21.15.42:0/2633337742 shutdown_connections 2024-08-30T09:12:29.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.914+0000 7f9e2583c740 1 --2- 172.21.15.42:0/2633337742 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5587b9b72070 0x5587b9f1a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:29.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.914+0000 7f9e2583c740 1 --2- 172.21.15.42:0/2633337742 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5587b9b51390 0x5587b9f21820 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:29.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.914+0000 7f9e2583c740 1 --2- 172.21.15.42:0/2633337742 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5587b9b491b0 0x5587b9f212e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:29.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.914+0000 7f9e2583c740 1 -- 172.21.15.42:0/2633337742 >> 172.21.15.42:0/2633337742 conn(0x5587b9b66460 msgr2=0x5587b9f1dba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:12:29.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.914+0000 7f9e2583c740 1 -- 172.21.15.42:0/2633337742 shutdown_connections 2024-08-30T09:12:29.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.914+0000 7f9e2583c740 1 -- 172.21.15.42:0/2633337742 wait complete. 2024-08-30T09:12:29.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.914+0000 7f9e2583c740 1 -- start start 2024-08-30T09:12:29.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.914+0000 7f9e2583c740 1 Processor -- start 2024-08-30T09:12:29.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.915+0000 7f9e2583c740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5587b9b491b0 0x5587b9f266b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:12:29.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.915+0000 7f9e2583c740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5587b9b51390 0x5587b9f26c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:12:29.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.915+0000 7f9e1d063640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5587b9b491b0 0x5587b9f266b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:12:29.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.915+0000 7f9e2583c740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5587b9b72070 0x5587b9f97c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:12:29.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.915+0000 7f9e2583c740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f9e181bdb80 con 0x5587b9b491b0 2024-08-30T09:12:29.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.915+0000 7f9e2583c740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f9e181beff0 con 0x5587b9b51390 2024-08-30T09:12:29.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.915+0000 7f9e2583c740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f9e181be250 con 0x5587b9b72070 2024-08-30T09:12:29.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.915+0000 7f9e2583c740 10 client.0 _ll_register_callbacks cb 0x7f9e100abd10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:12:29.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.915+0000 7f9e1c862640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5587b9b51390 0x5587b9f26c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:12:29.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.916+0000 7f9e1c862640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5587b9b51390 0x5587b9f26c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:42456/0 (socket says 172.21.15.42:42456) 2024-08-30T09:12:29.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.916+0000 7f9e1c862640 1 -- 172.21.15.42:0/3036062505 learned_addr learned my addr 172.21.15.42:0/3036062505 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:12:29.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.916+0000 7f9e1d864640 1 --2- 172.21.15.42:0/3036062505 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5587b9b72070 0x5587b9f97c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:12:29.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.916+0000 7f9e1c862640 1 -- 172.21.15.42:0/3036062505 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5587b9b72070 msgr2=0x5587b9f97c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:12:29.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.916+0000 7f9e1c862640 1 --2- 172.21.15.42:0/3036062505 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5587b9b72070 0x5587b9f97c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:29.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.916+0000 7f9e1c862640 1 -- 172.21.15.42:0/3036062505 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5587b9b491b0 msgr2=0x5587b9f266b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:12:29.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.916+0000 7f9e1c862640 1 --2- 172.21.15.42:0/3036062505 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5587b9b491b0 0x5587b9f266b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:29.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.916+0000 7f9e1c862640 1 -- 172.21.15.42:0/3036062505 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5587b9f982f0 con 0x5587b9b51390 2024-08-30T09:12:29.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.916+0000 7f9e1c862640 1 --2- 172.21.15.42:0/3036062505 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5587b9b51390 0x5587b9f26c90 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f9e0802a430 tx=0x7f9e080054c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=73531dda5e4f2f33 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:12:29.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.916+0000 7f9e00ff9640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-30T09:12:29.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.917+0000 7f9e00ff9640 1 -- 172.21.15.42:0/3036062505 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e08053020 con 0x5587b9b51390 2024-08-30T09:12:29.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.917+0000 7f9e2583c740 1 -- 172.21.15.42:0/3036062505 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9e08004750 con 0x5587b9b51390 2024-08-30T09:12:29.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.917+0000 7f9e00ff9640 1 -- 172.21.15.42:0/3036062505 <== mon.2 v2:172.21.15.167:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f9e08004b10 con 0x5587b9b51390 2024-08-30T09:12:29.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.917+0000 7f9e00ff9640 1 -- 172.21.15.42:0/3036062505 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f9e0805e050 con 0x5587b9b51390 2024-08-30T09:12:29.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.917+0000 7f9e2583c740 10 client.27182 fetch_fsmap learned FSMap version 43 2024-08-30T09:12:29.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.917+0000 7f9e2583c740 1 -- 172.21.15.42:0/3036062505 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x5587b9f9f6f0 con 0x5587b9b51390 2024-08-30T09:12:29.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.917+0000 7f9e00ff9640 1 -- 172.21.15.42:0/3036062505 <== mon.2 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9e08004750 con 0x5587b9b51390 2024-08-30T09:12:29.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.917+0000 7f9e00ff9640 1 -- 172.21.15.42:0/3036062505 <== mon.2 v2:172.21.15.167:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f9e0803c440 con 0x5587b9b51390 2024-08-30T09:12:29.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.917+0000 7f9e2583c740 10 client.27182 fetch_fsmap finished waiting for FSMap version 43 2024-08-30T09:12:29.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.917+0000 7f9e2583c740 10 client.27182 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-30T09:12:29.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.917+0000 7f9e2583c740 20 client.27182 populate_metadata read hostname 'smithi042' 2024-08-30T09:12:29.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.917+0000 7f9e2583c740 1 --2- 172.21.15.42:0/3036062505 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x5587b9b05da0 0x5587b9fa1920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:12:29.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.917+0000 7f9e2583c740 4 client.27182 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-30T09:12:29.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.917+0000 7f9e2583c740 1 -- 172.21.15.42:0/3036062505 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5587b9e90580 con 0x5587b9b05da0 2024-08-30T09:12:29.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.918+0000 7f9e1d063640 1 --2- 172.21.15.42:0/3036062505 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x5587b9b05da0 0x5587b9fa1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:12:29.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.918+0000 7f9e00ff9640 10 client.27182 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:12:29.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.918+0000 7f9e1d063640 1 --2- 172.21.15.42:0/3036062505 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x5587b9b05da0 0x5587b9fa1920 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4a740547e43eb1c8 server_cookie=9ae2fa836bd01eee in_seq=0 out_seq=0 2024-08-30T09:12:29.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.919+0000 7f9e00ff9640 1 -- 172.21.15.42:0/3036062505 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+4066 (crc 0 0 0) 0x5587b9e90580 con 0x5587b9b05da0 2024-08-30T09:12:29.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.919+0000 7f9e00ff9640 10 client.27182 handle_command_reply: tid=0 2024-08-30T09:12:29.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.919+0000 7f9e2583c740 1 client.27182 shutdown 2024-08-30T09:12:29.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.919+0000 7f9e2583c740 1 -- 172.21.15.42:0/3036062505 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5587b9b51390 msgr2=0x5587b9f26c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:12:29.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.919+0000 7f9e2583c740 1 --2- 172.21.15.42:0/3036062505 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5587b9b51390 0x5587b9f26c90 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f9e0802a430 tx=0x7f9e080054c0 comp rx=0 tx=0).stop 2024-08-30T09:12:29.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.919+0000 7f9e2583c740 1 -- 172.21.15.42:0/3036062505 shutdown_connections 2024-08-30T09:12:29.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.919+0000 7f9e2583c740 1 --2- 172.21.15.42:0/3036062505 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x5587b9b05da0 0x5587b9fa1920 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:29.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.919+0000 7f9e2583c740 1 --2- 172.21.15.42:0/3036062505 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5587b9b72070 0x5587b9f97c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:29.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.919+0000 7f9e2583c740 1 --2- 172.21.15.42:0/3036062505 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5587b9b51390 0x5587b9f26c90 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:29.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.919+0000 7f9e00ff9640 0 client.27182 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:12:29.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.919+0000 7f9e2583c740 1 --2- 172.21.15.42:0/3036062505 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5587b9b491b0 0x5587b9f266b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:29.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.919+0000 7f9e2583c740 1 -- 172.21.15.42:0/3036062505 >> 172.21.15.42:0/3036062505 conn(0x5587b9b66460 msgr2=0x5587b9f1dba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:12:29.159 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-08-30T09:12:29.159 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "no active scrubs running", 2024-08-30T09:12:29.159 INFO:teuthology.orchestra.run.smithi042.stdout: "scrubs": {}, 2024-08-30T09:12:29.159 INFO:teuthology.orchestra.run.smithi042.stdout: "scrub_stats": { 2024-08-30T09:12:29.159 INFO:teuthology.orchestra.run.smithi042.stdout: "5e2ed104-0625-478a-a78b-584e7f4c2817": { 2024-08-30T09:12:29.160 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:00:33.928400+0000", 2024-08-30T09:12:29.160 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:12:29.160 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:12:29.160 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:12:29.160 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:12:29.160 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:12:29.160 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:12:29.160 INFO:teuthology.orchestra.run.smithi042.stdout: "c827ad7a-44ee-4e1d-9bb3-51fb7ab00c73": { 2024-08-30T09:12:29.160 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:02:21.686517+0000", 2024-08-30T09:12:29.160 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:12:29.160 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:12:29.160 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:12:29.160 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:12:29.161 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:12:29.161 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:12:29.161 INFO:teuthology.orchestra.run.smithi042.stdout: "85134e4b-ab4a-4885-b07e-a3ad01eaf13d": { 2024-08-30T09:12:29.161 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:01:27.738462+0000", 2024-08-30T09:12:29.161 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:12:29.161 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:12:29.161 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:12:29.161 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:12:29.161 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:12:29.161 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:12:29.161 INFO:teuthology.orchestra.run.smithi042.stdout: "749f4867-0b70-49e5-af2d-8362a240ebc3": { 2024-08-30T09:12:29.161 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:03:16.386554+0000", 2024-08-30T09:12:29.161 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:12:29.161 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:12:29.162 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:12:29.162 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:12:29.162 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:12:29.162 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:12:29.162 INFO:teuthology.orchestra.run.smithi042.stdout: "88cd7478-2b20-47de-ba51-256479749232": { 2024-08-30T09:12:29.162 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:04:08.922625+0000", 2024-08-30T09:12:29.162 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:12:29.162 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:12:29.162 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:12:29.162 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:12:29.162 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:12:29.162 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:12:29.162 INFO:teuthology.orchestra.run.smithi042.stdout: "347d05ed-4bd3-4590-b099-314f719aab39": { 2024-08-30T09:12:29.162 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:05:00.439721+0000", 2024-08-30T09:12:29.162 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:12:29.163 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:12:29.163 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:12:29.163 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:12:29.163 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:12:29.163 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:12:29.163 INFO:teuthology.orchestra.run.smithi042.stdout: "1af4252a-4a99-47e9-bbf0-becf5e573dd5": { 2024-08-30T09:12:29.163 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:05:48.565881+0000", 2024-08-30T09:12:29.163 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:12:29.163 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:12:29.163 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:12:29.163 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:12:29.163 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:12:29.163 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:12:29.163 INFO:teuthology.orchestra.run.smithi042.stdout: "665ec740-7254-49cb-84ef-deff71c461b0": { 2024-08-30T09:12:29.164 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:06:37.144035+0000", 2024-08-30T09:12:29.164 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:12:29.164 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:12:29.164 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:12:29.164 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:12:29.164 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:12:29.164 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:12:29.164 INFO:teuthology.orchestra.run.smithi042.stdout: "598d8a75-f1cc-4388-80bc-a408bdb4eaac": { 2024-08-30T09:12:29.164 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:07:29.721122+0000", 2024-08-30T09:12:29.164 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:12:29.164 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:12:29.164 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:12:29.164 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:12:29.164 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:12:29.165 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:12:29.165 INFO:teuthology.orchestra.run.smithi042.stdout: "b62c4ffc-9387-4677-b9ba-1e94ba308f17": { 2024-08-30T09:12:29.165 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:11:03.363478+0000", 2024-08-30T09:12:29.165 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:12:29.165 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:12:29.165 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:12:29.165 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:12:29.165 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:12:29.165 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:12:29.165 INFO:teuthology.orchestra.run.smithi042.stdout: "a7210f9a-046b-4188-953c-819b9f1863a9": { 2024-08-30T09:12:29.165 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:10:07.608430+0000", 2024-08-30T09:12:29.165 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:12:29.165 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:12:29.165 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:12:29.166 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:12:29.166 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:12:29.166 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:12:29.166 INFO:teuthology.orchestra.run.smithi042.stdout: "99a29ec1-9455-4ae4-854a-398b32896675": { 2024-08-30T09:12:29.166 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:08:24.923176+0000", 2024-08-30T09:12:29.166 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:12:29.166 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:12:29.166 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:12:29.166 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:12:29.166 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:12:29.166 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:12:29.166 INFO:teuthology.orchestra.run.smithi042.stdout: "ff464e7f-123b-4bc6-bc7e-aced018168de": { 2024-08-30T09:12:29.166 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:11:54.879598+0000", 2024-08-30T09:12:29.167 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:12:29.167 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:12:29.167 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:12:29.167 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:12:29.167 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:12:29.167 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:12:29.167 INFO:teuthology.orchestra.run.smithi042.stdout: "37f2f28f-496f-41a6-b4cf-8846fb3f43ab": { 2024-08-30T09:12:29.167 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:09:17.428286+0000", 2024-08-30T09:12:29.167 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:12:29.167 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:12:29.167 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:12:29.167 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:12:29.167 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:12:29.168 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:12:29.168 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:12:29.168 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-08-30T09:12:29.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.919+0000 7f9e2583c740 1 -- 172.21.15.42:0/3036062505 shutdown_connections 2024-08-30T09:12:29.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.919+0000 7f9e2583c740 1 -- 172.21.15.42:0/3036062505 wait complete. 2024-08-30T09:12:29.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.919+0000 7f9e2583c740 20 client.27182 trim_cache size 0 max 16384 2024-08-30T09:12:29.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.921+0000 7f9e1f2c7640 1 -- 172.21.15.42:0/2234112981 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f9e040b4bb0 msgr2=0x7f9e040b70c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:12:29.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.921+0000 7f9e1f2c7640 1 --2- 172.21.15.42:0/2234112981 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f9e040b4bb0 0x7f9e040b70c0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f9e14005fd0 tx=0x7f9e14005850 comp rx=0 tx=0).stop 2024-08-30T09:12:29.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.921+0000 7f9e1f2c7640 1 -- 172.21.15.42:0/2234112981 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9e18108ad0 msgr2=0x7f9e1810c260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:12:29.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.921+0000 7f9e1f2c7640 1 --2- 172.21.15.42:0/2234112981 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9e18108ad0 0x7f9e1810c260 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f9e1000cce0 tx=0x7f9e1000a620 comp rx=0 tx=0).stop 2024-08-30T09:12:29.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.921+0000 7f9e1f2c7640 1 -- 172.21.15.42:0/2234112981 shutdown_connections 2024-08-30T09:12:29.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.921+0000 7f9e1f2c7640 1 --2- 172.21.15.42:0/2234112981 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f9e040b4bb0 0x7f9e040b70c0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:29.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.921+0000 7f9e1f2c7640 1 --2- 172.21.15.42:0/2234112981 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9e18108ad0 0x7f9e1810c260 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:29.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.921+0000 7f9e1f2c7640 1 --2- 172.21.15.42:0/2234112981 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9e18108180 0x7f9e1810bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:29.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.921+0000 7f9e1f2c7640 1 --2- 172.21.15.42:0/2234112981 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9e18102840 0x7f9e1810b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:29.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.921+0000 7f9e1f2c7640 1 -- 172.21.15.42:0/2234112981 >> 172.21.15.42:0/2234112981 conn(0x7f9e180794f0 msgr2=0x7f9e18106270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:12:29.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.921+0000 7f9e1f2c7640 1 -- 172.21.15.42:0/2234112981 shutdown_connections 2024-08-30T09:12:29.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:28.921+0000 7f9e1f2c7640 1 -- 172.21.15.42:0/2234112981 wait complete. 2024-08-30T09:12:29.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:12:29 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T09:12:29.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:12:29 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-30T09:12:29.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:12:29 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T09:12:29.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:12:29 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T09:12:29.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:12:29 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-30T09:12:29.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:12:29 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T09:12:29.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:12:29 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T09:12:29.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:12:29 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-30T09:12:29.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:12:29 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T09:12:30.825 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:12:30 smithi042 ceph-mon[18652]: pgmap v725: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 6.7 MiB/s wr, 593 op/s 2024-08-30T09:12:30.953 INFO:teuthology.orchestra.run.smithi042.stderr:progress: 705.0 s, 443.6 tps, lat 215.121 ms stddev 260.581 2024-08-30T09:12:30.990 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-08-30T09:12:30.990 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph fs dump --format=json 2024-08-30T09:12:31.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:12:30 smithi167 ceph-mon[22063]: pgmap v725: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 6.7 MiB/s wr, 593 op/s 2024-08-30T09:12:31.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:12:30 smithi190 ceph-mon[22001]: pgmap v725: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 6.7 MiB/s wr, 593 op/s 2024-08-30T09:12:31.428 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T09:12:32.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:32.550+0000 7f98d9e7a640 1 -- 172.21.15.42:0/2279462719 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f98d410aab0 msgr2=0x7f98d4118160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:12:32.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:32.550+0000 7f98d9e7a640 1 --2- 172.21.15.42:0/2279462719 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f98d410aab0 0x7f98d4118160 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f98cc009860 tx=0x7f98cc032b90 comp rx=0 tx=0).stop 2024-08-30T09:12:32.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:32.555+0000 7f98d9e7a640 1 -- 172.21.15.42:0/2279462719 shutdown_connections 2024-08-30T09:12:32.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:32.555+0000 7f98d9e7a640 1 --2- 172.21.15.42:0/2279462719 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f98d410aab0 0x7f98d4118160 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:32.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:32.555+0000 7f98d9e7a640 1 --2- 172.21.15.42:0/2279462719 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f98d41186a0 0x7f98d411aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:32.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:32.555+0000 7f98d9e7a640 1 --2- 172.21.15.42:0/2279462719 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f98d410a160 0x7f98d410a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:32.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:32.555+0000 7f98d9e7a640 1 -- 172.21.15.42:0/2279462719 >> 172.21.15.42:0/2279462719 conn(0x7f98d4070c60 msgr2=0x7f98d4071070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:12:32.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:32.555+0000 7f98d9e7a640 1 -- 172.21.15.42:0/2279462719 shutdown_connections 2024-08-30T09:12:32.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:32.555+0000 7f98d9e7a640 1 -- 172.21.15.42:0/2279462719 wait complete. 2024-08-30T09:12:32.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:32.556+0000 7f98d9e7a640 1 Processor -- start 2024-08-30T09:12:32.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:32.556+0000 7f98d9e7a640 1 -- start start 2024-08-30T09:12:32.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:32.557+0000 7f98d9e7a640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f98d410a160 0x7f98d41a0920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:12:32.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:32.557+0000 7f98d9e7a640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f98d410aab0 0x7f98d41a0ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:12:32.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:32.557+0000 7f98d9e7a640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f98d41186a0 0x7f98d41a6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:12:32.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:32.557+0000 7f98d9e7a640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f98d41105a0 con 0x7f98d410a160 2024-08-30T09:12:32.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:32.557+0000 7f98d9e7a640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f98d4110420 con 0x7f98d41186a0 2024-08-30T09:12:32.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:32.557+0000 7f98d9e7a640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f98d4110720 con 0x7f98d410aab0 2024-08-30T09:12:32.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:32.558+0000 7f98d3fff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f98d41186a0 0x7f98d41a6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:12:32.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:32.558+0000 7f98d3fff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f98d41186a0 0x7f98d41a6160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:42470/0 (socket says 172.21.15.42:42470) 2024-08-30T09:12:32.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:32.558+0000 7f98d37fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f98d410a160 0x7f98d41a0920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:12:32.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:32.558+0000 7f98d37fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f98d410a160 0x7f98d41a0920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41406/0 (socket says 172.21.15.42:41406) 2024-08-30T09:12:32.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:32.558+0000 7f98d37fe640 1 -- 172.21.15.42:0/1429168128 learned_addr learned my addr 172.21.15.42:0/1429168128 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:12:32.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:32.558+0000 7f98d2ffd640 1 --2- 172.21.15.42:0/1429168128 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f98d410aab0 0x7f98d41a0ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:12:32.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:32.558+0000 7f98d37fe640 1 -- 172.21.15.42:0/1429168128 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f98d410aab0 msgr2=0x7f98d41a0ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:12:32.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:32.558+0000 7f98d37fe640 1 --2- 172.21.15.42:0/1429168128 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f98d410aab0 0x7f98d41a0ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:32.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:32.558+0000 7f98d37fe640 1 -- 172.21.15.42:0/1429168128 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f98d41186a0 msgr2=0x7f98d41a6160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:12:32.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:32.558+0000 7f98d37fe640 1 --2- 172.21.15.42:0/1429168128 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f98d41186a0 0x7f98d41a6160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:32.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:32.558+0000 7f98d37fe640 1 -- 172.21.15.42:0/1429168128 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98d41a6890 con 0x7f98d410a160 2024-08-30T09:12:32.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:32.559+0000 7f98d37fe640 1 --2- 172.21.15.42:0/1429168128 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f98d410a160 0x7f98d41a0920 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f98bc00efb0 tx=0x7f98bc00c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:12:32.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:32.560+0000 7f98d0ff9640 1 -- 172.21.15.42:0/1429168128 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98bc01b070 con 0x7f98d410a160 2024-08-30T09:12:32.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:32.560+0000 7f98d0ff9640 1 -- 172.21.15.42:0/1429168128 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f98bc00ba20 con 0x7f98d410a160 2024-08-30T09:12:32.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:32.560+0000 7f98d0ff9640 1 -- 172.21.15.42:0/1429168128 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98bc016b30 con 0x7f98d410a160 2024-08-30T09:12:32.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:32.560+0000 7f98d9e7a640 1 -- 172.21.15.42:0/1429168128 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98d41a6b80 con 0x7f98d410a160 2024-08-30T09:12:32.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:32.560+0000 7f98d9e7a640 1 -- 172.21.15.42:0/1429168128 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98d41ad2e0 con 0x7f98d410a160 2024-08-30T09:12:32.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:32.561+0000 7f98d9e7a640 1 -- 172.21.15.42:0/1429168128 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98d410b380 con 0x7f98d410a160 2024-08-30T09:12:32.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:32.568+0000 7f98d0ff9640 1 -- 172.21.15.42:0/1429168128 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7f98bc01e070 con 0x7f98d410a160 2024-08-30T09:12:32.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:32.569+0000 7f98d0ff9640 1 --2- 172.21.15.42:0/1429168128 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f98b40b4d40 0x7f98b40b7250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:12:32.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:32.569+0000 7f98d0ff9640 1 -- 172.21.15.42:0/1429168128 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f98bc0e5930 con 0x7f98d410a160 2024-08-30T09:12:32.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:32.572+0000 7f98d2ffd640 1 --2- 172.21.15.42:0/1429168128 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f98b40b4d40 0x7f98b40b7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:12:32.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:32.573+0000 7f98d2ffd640 1 --2- 172.21.15.42:0/1429168128 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f98b40b4d40 0x7f98b40b7250 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f98cc033100 tx=0x7f98cc033090 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:12:32.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:32.580+0000 7f98d0ff9640 1 -- 172.21.15.42:0/1429168128 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f98bc0ac0a0 con 0x7f98d410a160 2024-08-30T09:12:32.961 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-08-30T09:12:32.961 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":43,"btime":"2024-08-30T09:12:10:979656+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14703,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/3188400000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":3188400000},{"type":"v1","addr":"172.21.15.42:6835","nonce":3188400000}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14715,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/3356537674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":3356537674},{"type":"v1","addr":"172.21.15.42:6837","nonce":3356537674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14766,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6839/2107785513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6838","nonce":2107785513},{"type":"v1","addr":"172.21.15.42:6839","nonce":2107785513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14778,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6841/2298438674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6840","nonce":2298438674},{"type":"v1","addr":"172.21.15.42:6841","nonce":2298438674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24553,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/543555570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":543555570},{"type":"v1","addr":"172.21.15.190:6833","nonce":543555570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24554,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/3907818941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":3907818941},{"type":"v1","addr":"172.21.15.167:6833","nonce":3907818941}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":43,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-30T08:58:03.055084+0000","modified":"2024-08-30T09:12:10.677077+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24629,"mds_1":24610,"mds_2":24599},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24572":{"gid":24572,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6835/234596845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":234596845},{"type":"v1","addr":"172.21.15.167:6835","nonce":234596845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24592":{"gid":24592,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6835/247905709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":247905709},{"type":"v1","addr":"172.21.15.190:6835","nonce":247905709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24598":{"gid":24598,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6837/863256349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6836","nonce":863256349},{"type":"v1","addr":"172.21.15.190:6837","nonce":863256349}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24599":{"gid":24599,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.167:6837/3011837856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":3011837856},{"type":"v1","addr":"172.21.15.167:6837","nonce":3011837856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24610":{"gid":24610,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.190:6839/719981110","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6838","nonce":719981110},{"type":"v1","addr":"172.21.15.190:6839","nonce":719981110}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24629":{"gid":24629,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.167:6839/3604645207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":3604645207},{"type":"v1","addr":"172.21.15.167:6839","nonce":3604645207}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24629,"qdb_cluster":[24599,24610,24629]},"id":1}]} 2024-08-30T09:12:32.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:32.957+0000 7f98d9e7a640 1 -- 172.21.15.42:0/1429168128 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f98d410a570 con 0x7f98d410a160 2024-08-30T09:12:32.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:32.959+0000 7f98d0ff9640 1 -- 172.21.15.42:0/1429168128 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) ==== 94+0+11915 (secure 0 0 0) 0x7f98bc0b0860 con 0x7f98d410a160 2024-08-30T09:12:32.962 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 43 2024-08-30T09:12:32.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:32.968+0000 7f98d9e7a640 1 -- 172.21.15.42:0/1429168128 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f98b40b4d40 msgr2=0x7f98b40b7250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:12:32.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:32.968+0000 7f98d9e7a640 1 --2- 172.21.15.42:0/1429168128 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f98b40b4d40 0x7f98b40b7250 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f98cc033100 tx=0x7f98cc033090 comp rx=0 tx=0).stop 2024-08-30T09:12:32.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:32.968+0000 7f98d9e7a640 1 -- 172.21.15.42:0/1429168128 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f98d410a160 msgr2=0x7f98d41a0920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:12:32.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:32.968+0000 7f98d9e7a640 1 --2- 172.21.15.42:0/1429168128 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f98d410a160 0x7f98d41a0920 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f98bc00efb0 tx=0x7f98bc00c800 comp rx=0 tx=0).stop 2024-08-30T09:12:32.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:32.970+0000 7f98d9e7a640 1 -- 172.21.15.42:0/1429168128 shutdown_connections 2024-08-30T09:12:32.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:32.970+0000 7f98d9e7a640 1 --2- 172.21.15.42:0/1429168128 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f98b40b4d40 0x7f98b40b7250 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:32.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:32.970+0000 7f98d9e7a640 1 --2- 172.21.15.42:0/1429168128 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f98d410aab0 0x7f98d41a0ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:32.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:32.970+0000 7f98d9e7a640 1 --2- 172.21.15.42:0/1429168128 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f98d41186a0 0x7f98d41a6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:32.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:32.970+0000 7f98d9e7a640 1 --2- 172.21.15.42:0/1429168128 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f98d410a160 0x7f98d41a0920 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:32.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:32.970+0000 7f98d9e7a640 1 -- 172.21.15.42:0/1429168128 >> 172.21.15.42:0/1429168128 conn(0x7f98d4070c60 msgr2=0x7f98d4118d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:12:32.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:32.973+0000 7f98d9e7a640 1 -- 172.21.15.42:0/1429168128 shutdown_connections 2024-08-30T09:12:32.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:32.977+0000 7f98d9e7a640 1 -- 172.21.15.42:0/1429168128 wait complete. 2024-08-30T09:12:32.997 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:12:32 smithi042 ceph-mon[18652]: pgmap v726: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 4.9 MiB/s wr, 454 op/s 2024-08-30T09:12:33.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:12:32 smithi167 ceph-mon[22063]: pgmap v726: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 4.9 MiB/s wr, 454 op/s 2024-08-30T09:12:33.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:12:32 smithi190 ceph-mon[22001]: pgmap v726: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 4.9 MiB/s wr, 454 op/s 2024-08-30T09:12:33.884 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:12:33 smithi042 ceph-mon[18652]: from='client.? 172.21.15.42:0/1429168128' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-30T09:12:33.887 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-08-30T09:12:34.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:12:33 smithi167 ceph-mon[22063]: from='client.? 172.21.15.42:0/1429168128' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-30T09:12:34.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:12:33 smithi190 ceph-mon[22001]: from='client.? 172.21.15.42:0/1429168128' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-30T09:12:34.292 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T09:12:34.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.954+0000 7fe353fff640 1 -- 172.21.15.42:0/3950987603 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe34c0318a0 con 0x7fe35c10ad30 2024-08-30T09:12:34.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.955+0000 7fe362230640 1 -- 172.21.15.42:0/3950987603 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe35c10ad30 msgr2=0x7fe35c115d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:12:34.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.955+0000 7fe362230640 1 --2- 172.21.15.42:0/3950987603 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe35c10ad30 0x7fe35c115d70 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fe34c009900 tx=0x7fe34c02a3f0 comp rx=0 tx=0).stop 2024-08-30T09:12:34.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.955+0000 7fe362230640 1 -- 172.21.15.42:0/3950987603 shutdown_connections 2024-08-30T09:12:34.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.955+0000 7fe362230640 1 --2- 172.21.15.42:0/3950987603 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe35c10ad30 0x7fe35c115d70 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:34.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.955+0000 7fe362230640 1 --2- 172.21.15.42:0/3950987603 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe35c1162b0 0x7fe35c118730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:34.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.955+0000 7fe362230640 1 --2- 172.21.15.42:0/3950987603 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe35c10a3e0 0x7fe35c10a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:34.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.955+0000 7fe362230640 1 -- 172.21.15.42:0/3950987603 >> 172.21.15.42:0/3950987603 conn(0x7fe35c06e790 msgr2=0x7fe35c06ebd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:12:34.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.955+0000 7fe362230640 1 -- 172.21.15.42:0/3950987603 shutdown_connections 2024-08-30T09:12:34.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.955+0000 7fe362230640 1 -- 172.21.15.42:0/3950987603 wait complete. 2024-08-30T09:12:34.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.956+0000 7fe362230640 1 Processor -- start 2024-08-30T09:12:34.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.956+0000 7fe362230640 1 -- start start 2024-08-30T09:12:34.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.956+0000 7fe362230640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe35c10a3e0 0x7fe35c1a2900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:12:34.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.956+0000 7fe362230640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe35c10ad30 0x7fe35c1a2ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:12:34.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.956+0000 7fe362230640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe35c1162b0 0x7fe35c1a8140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:12:34.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.956+0000 7fe362230640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fe35c11be90 con 0x7fe35c1162b0 2024-08-30T09:12:34.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.956+0000 7fe362230640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe35c11bd10 con 0x7fe35c10a3e0 2024-08-30T09:12:34.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.957+0000 7fe362230640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fe35c11c010 con 0x7fe35c10ad30 2024-08-30T09:12:34.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.957+0000 7fe36122e640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe35c10a3e0 0x7fe35c1a2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:12:34.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.957+0000 7fe36122e640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe35c10a3e0 0x7fe35c1a2900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:42492/0 (socket says 172.21.15.42:42492) 2024-08-30T09:12:34.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.957+0000 7fe36122e640 1 -- 172.21.15.42:0/1225499620 learned_addr learned my addr 172.21.15.42:0/1225499620 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:12:34.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.957+0000 7fe361a2f640 1 --2- 172.21.15.42:0/1225499620 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe35c1162b0 0x7fe35c1a8140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:12:34.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.957+0000 7fe360a2d640 1 --2- 172.21.15.42:0/1225499620 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe35c10ad30 0x7fe35c1a2ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:12:34.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.957+0000 7fe36122e640 1 -- 172.21.15.42:0/1225499620 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe35c10ad30 msgr2=0x7fe35c1a2ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:12:34.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.957+0000 7fe36122e640 1 --2- 172.21.15.42:0/1225499620 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe35c10ad30 0x7fe35c1a2ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:34.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.957+0000 7fe36122e640 1 -- 172.21.15.42:0/1225499620 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe35c1162b0 msgr2=0x7fe35c1a8140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:12:34.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.957+0000 7fe36122e640 1 --2- 172.21.15.42:0/1225499620 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe35c1162b0 0x7fe35c1a8140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:34.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.957+0000 7fe36122e640 1 -- 172.21.15.42:0/1225499620 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe35c1a8870 con 0x7fe35c10a3e0 2024-08-30T09:12:34.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.957+0000 7fe361a2f640 1 --2- 172.21.15.42:0/1225499620 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe35c1162b0 0x7fe35c1a8140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-30T09:12:34.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.958+0000 7fe36122e640 1 --2- 172.21.15.42:0/1225499620 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe35c10a3e0 0x7fe35c1a2900 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fe35800ebc0 tx=0x7fe35800c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:12:34.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.958+0000 7fe3527fc640 1 -- 172.21.15.42:0/1225499620 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe358010430 con 0x7fe35c10a3e0 2024-08-30T09:12:34.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.958+0000 7fe362230640 1 -- 172.21.15.42:0/1225499620 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe35c1a8b60 con 0x7fe35c10a3e0 2024-08-30T09:12:34.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.958+0000 7fe3527fc640 1 -- 172.21.15.42:0/1225499620 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe358010a80 con 0x7fe35c10a3e0 2024-08-30T09:12:34.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.958+0000 7fe362230640 1 -- 172.21.15.42:0/1225499620 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe35c10bb00 con 0x7fe35c10a3e0 2024-08-30T09:12:34.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.958+0000 7fe3527fc640 1 -- 172.21.15.42:0/1225499620 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe358018a50 con 0x7fe35c10a3e0 2024-08-30T09:12:34.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.962+0000 7fe369a07740 1 Processor -- start 2024-08-30T09:12:34.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.962+0000 7fe369a07740 1 -- start start 2024-08-30T09:12:34.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.962+0000 7fe369a07740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564ad225e070 0x564ad260bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:12:34.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.962+0000 7fe369a07740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564ad22351b0 0x564ad260c200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:12:34.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.962+0000 7fe369a07740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564ad223d390 0x564ad260c740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:12:34.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.962+0000 7fe369a07740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x564ad257a680 con 0x564ad225e070 2024-08-30T09:12:34.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.962+0000 7fe369a07740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x564ad2569c10 con 0x564ad223d390 2024-08-30T09:12:34.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.962+0000 7fe369a07740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x564ad256dbc0 con 0x564ad22351b0 2024-08-30T09:12:34.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.963+0000 7fe361a2f640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564ad225e070 0x564ad260bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:12:34.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.963+0000 7fe36122e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564ad22351b0 0x564ad260c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:12:34.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.963+0000 7fe361a2f640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564ad225e070 0x564ad260bcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41430/0 (socket says 172.21.15.42:41430) 2024-08-30T09:12:34.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.963+0000 7fe361a2f640 1 -- 172.21.15.42:0/1044645170 learned_addr learned my addr 172.21.15.42:0/1044645170 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:12:34.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.963+0000 7fe36122e640 1 -- 172.21.15.42:0/1044645170 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564ad223d390 msgr2=0x564ad260c740 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-30T09:12:34.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.963+0000 7fe36122e640 1 --2- 172.21.15.42:0/1044645170 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564ad223d390 0x564ad260c740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:34.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.963+0000 7fe36122e640 1 -- 172.21.15.42:0/1044645170 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564ad225e070 msgr2=0x564ad260bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:12:34.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.963+0000 7fe36122e640 1 --2- 172.21.15.42:0/1044645170 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564ad225e070 0x564ad260bcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:34.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.963+0000 7fe36122e640 1 -- 172.21.15.42:0/1044645170 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ad22c9650 con 0x564ad22351b0 2024-08-30T09:12:34.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.963+0000 7fe3527fc640 1 -- 172.21.15.42:0/1225499620 <== mon.2 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7fe358018bf0 con 0x7fe35c10a3e0 2024-08-30T09:12:34.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.964+0000 7fe36122e640 1 --2- 172.21.15.42:0/1044645170 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564ad22351b0 0x564ad260c200 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fe3580e3290 tx=0x7fe3580ee670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=18a0b05b58b227dd server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:12:34.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.964+0000 7fe3477fe640 1 -- 172.21.15.42:0/1044645170 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3580f5070 con 0x564ad22351b0 2024-08-30T09:12:34.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.964+0000 7fe3477fe640 1 -- 172.21.15.42:0/1044645170 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe3580f1ad0 con 0x564ad22351b0 2024-08-30T09:12:34.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.964+0000 7fe3527fc640 1 --2- 172.21.15.42:0/1225499620 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fe3480b4c70 0x7fe3480b7180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:12:34.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.965+0000 7fe3527fc640 1 -- 172.21.15.42:0/1225499620 <== mon.2 v2:172.21.15.167:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7fe3580e5be0 con 0x7fe35c10a3e0 2024-08-30T09:12:34.969 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.965+0000 7fe361a2f640 1 --2- 172.21.15.42:0/1225499620 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fe3480b4c70 0x7fe3480b7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:12:34.969 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.966+0000 7fe3477fe640 1 -- 172.21.15.42:0/1044645170 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3580f0750 con 0x564ad22351b0 2024-08-30T09:12:34.969 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.966+0000 7fe369a07740 1 -- 172.21.15.42:0/1044645170 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564ad22351b0 msgr2=0x564ad260c200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:12:34.969 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.966+0000 7fe369a07740 1 --2- 172.21.15.42:0/1044645170 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564ad22351b0 0x564ad260c200 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fe3580e3290 tx=0x7fe3580ee670 comp rx=0 tx=0).stop 2024-08-30T09:12:34.969 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.966+0000 7fe369a07740 1 -- 172.21.15.42:0/1044645170 shutdown_connections 2024-08-30T09:12:34.969 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.966+0000 7fe369a07740 1 --2- 172.21.15.42:0/1044645170 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564ad22351b0 0x564ad260c200 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:34.969 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.966+0000 7fe369a07740 1 --2- 172.21.15.42:0/1044645170 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564ad223d390 0x564ad260c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:34.969 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.966+0000 7fe369a07740 1 --2- 172.21.15.42:0/1044645170 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564ad225e070 0x564ad260bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:34.969 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.966+0000 7fe369a07740 1 -- 172.21.15.42:0/1044645170 >> 172.21.15.42:0/1044645170 conn(0x564ad2252460 msgr2=0x564ad2609bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:12:34.969 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.966+0000 7fe369a07740 1 -- 172.21.15.42:0/1044645170 shutdown_connections 2024-08-30T09:12:34.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.967+0000 7fe361a2f640 1 --2- 172.21.15.42:0/1225499620 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fe3480b4c70 0x7fe3480b7180 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fe3540126c0 tx=0x7fe354005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:12:34.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.967+0000 7fe369a07740 1 -- 172.21.15.42:0/1044645170 wait complete. 2024-08-30T09:12:34.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.967+0000 7fe369a07740 1 -- start start 2024-08-30T09:12:34.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.968+0000 7fe369a07740 1 Processor -- start 2024-08-30T09:12:34.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.968+0000 7fe369a07740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564ad22351b0 0x564ad2605f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:12:34.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.968+0000 7fe369a07740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564ad223d390 0x564ad2606440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:12:34.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.968+0000 7fe369a07740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564ad225e070 0x564ad26069a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:12:34.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.968+0000 7fe369a07740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x564ad23cc700 con 0x564ad223d390 2024-08-30T09:12:34.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.968+0000 7fe369a07740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x564ad24f0750 con 0x564ad22351b0 2024-08-30T09:12:34.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.968+0000 7fe369a07740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x564ad2609090 con 0x564ad225e070 2024-08-30T09:12:34.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.968+0000 7fe369a07740 10 client.0 _ll_register_callbacks cb 0x564ad251fe10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:12:34.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.969+0000 7fe360a2d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564ad223d390 0x564ad2606440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:12:34.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.969+0000 7fe36122e640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564ad22351b0 0x564ad2605f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:12:34.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.969+0000 7fe360a2d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564ad223d390 0x564ad2606440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41442/0 (socket says 172.21.15.42:41442) 2024-08-30T09:12:34.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.969+0000 7fe360a2d640 1 -- 172.21.15.42:0/3508419093 learned_addr learned my addr 172.21.15.42:0/3508419093 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:12:34.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.969+0000 7fe36122e640 1 --2- 172.21.15.42:0/3508419093 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564ad22351b0 0x564ad2605f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:42508/0 (socket says 172.21.15.42:42508) 2024-08-30T09:12:34.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.969+0000 7fe361a2f640 1 --2- 172.21.15.42:0/3508419093 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564ad225e070 0x564ad26069a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:12:34.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.969+0000 7fe36122e640 1 -- 172.21.15.42:0/3508419093 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564ad225e070 msgr2=0x564ad26069a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:12:34.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.969+0000 7fe36122e640 1 --2- 172.21.15.42:0/3508419093 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564ad225e070 0x564ad26069a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:34.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.969+0000 7fe36122e640 1 -- 172.21.15.42:0/3508419093 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564ad223d390 msgr2=0x564ad2606440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:12:34.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.969+0000 7fe36122e640 1 --2- 172.21.15.42:0/3508419093 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564ad223d390 0x564ad2606440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:34.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.969+0000 7fe36122e640 1 -- 172.21.15.42:0/3508419093 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ad2529900 con 0x564ad22351b0 2024-08-30T09:12:34.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.969+0000 7fe361a2f640 1 --2- 172.21.15.42:0/3508419093 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564ad225e070 0x564ad26069a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-30T09:12:34.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.969+0000 7fe36122e640 1 --2- 172.21.15.42:0/3508419093 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564ad22351b0 0x564ad2605f00 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fe3580a9490 tx=0x7fe35800cf10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8b44980fdba67f06 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:12:34.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.969+0000 7fe344ff9640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-30T09:12:34.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.969+0000 7fe344ff9640 1 -- 172.21.15.42:0/3508419093 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3580f5070 con 0x564ad22351b0 2024-08-30T09:12:34.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.970+0000 7fe369a07740 1 -- 172.21.15.42:0/3508419093 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe3580f1ad0 con 0x564ad22351b0 2024-08-30T09:12:34.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.970+0000 7fe344ff9640 1 -- 172.21.15.42:0/3508419093 <== mon.2 v2:172.21.15.167:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7fe3580ac690 con 0x564ad22351b0 2024-08-30T09:12:34.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.970+0000 7fe344ff9640 1 -- 172.21.15.42:0/3508419093 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fe3580a80e0 con 0x564ad22351b0 2024-08-30T09:12:34.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.970+0000 7fe369a07740 10 client.27194 fetch_fsmap learned FSMap version 43 2024-08-30T09:12:34.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.970+0000 7fe369a07740 1 -- 172.21.15.42:0/3508419093 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x564ad262e9f0 con 0x564ad22351b0 2024-08-30T09:12:34.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.970+0000 7fe344ff9640 1 -- 172.21.15.42:0/3508419093 <== mon.2 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe3580f1ad0 con 0x564ad22351b0 2024-08-30T09:12:34.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.970+0000 7fe344ff9640 1 -- 172.21.15.42:0/3508419093 <== mon.2 v2:172.21.15.167:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7fe3580b50b0 con 0x564ad22351b0 2024-08-30T09:12:34.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.970+0000 7fe369a07740 10 client.27194 fetch_fsmap finished waiting for FSMap version 43 2024-08-30T09:12:34.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.970+0000 7fe369a07740 10 client.27194 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-08-30T09:12:34.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.970+0000 7fe369a07740 20 client.27194 populate_metadata read hostname 'smithi042' 2024-08-30T09:12:34.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.970+0000 7fe369a07740 1 --2- 172.21.15.42:0/3508419093 >> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] conn(0x564ad221c9f0 0x564ad2631d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:12:34.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.970+0000 7fe369a07740 4 client.27194 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-30T09:12:34.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.970+0000 7fe369a07740 1 -- 172.21.15.42:0/3508419093 --> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564ad257c5d0 con 0x564ad221c9f0 2024-08-30T09:12:34.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.972+0000 7fe360a2d640 1 --2- 172.21.15.42:0/3508419093 >> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] conn(0x564ad221c9f0 0x564ad2631d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:12:34.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.972+0000 7fe344ff9640 10 client.27194 ms_handle_connect on v2:172.21.15.167:6836/3011837856 2024-08-30T09:12:34.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.972+0000 7fe360a2d640 1 --2- 172.21.15.42:0/3508419093 >> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] conn(0x564ad221c9f0 0x564ad2631d50 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=da20b5684a4c59f1 server_cookie=b56395d4e47815d2 in_seq=0 out_seq=0 2024-08-30T09:12:34.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.973+0000 7fe344ff9640 1 -- 172.21.15.42:0/3508419093 <== mds.2 v2:172.21.15.167:6836/3011837856 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x564ad257c5d0 con 0x564ad221c9f0 2024-08-30T09:12:34.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.973+0000 7fe344ff9640 10 client.27194 handle_command_reply: tid=0 2024-08-30T09:12:34.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.973+0000 7fe369a07740 1 client.27194 shutdown 2024-08-30T09:12:34.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.973+0000 7fe369a07740 1 -- 172.21.15.42:0/3508419093 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564ad22351b0 msgr2=0x564ad2605f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:12:34.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.973+0000 7fe369a07740 1 --2- 172.21.15.42:0/3508419093 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564ad22351b0 0x564ad2605f00 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fe3580a9490 tx=0x7fe35800cf10 comp rx=0 tx=0).stop 2024-08-30T09:12:34.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.973+0000 7fe369a07740 1 -- 172.21.15.42:0/3508419093 shutdown_connections 2024-08-30T09:12:34.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.973+0000 7fe369a07740 1 --2- 172.21.15.42:0/3508419093 >> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] conn(0x564ad221c9f0 0x564ad2631d50 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:34.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.973+0000 7fe344ff9640 0 client.27194 ms_handle_reset on v2:172.21.15.167:6836/3011837856 2024-08-30T09:12:34.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.973+0000 7fe369a07740 1 --2- 172.21.15.42:0/3508419093 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564ad225e070 0x564ad26069a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:34.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.973+0000 7fe369a07740 1 --2- 172.21.15.42:0/3508419093 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564ad223d390 0x564ad2606440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:34.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.973+0000 7fe369a07740 1 --2- 172.21.15.42:0/3508419093 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564ad22351b0 0x564ad2605f00 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:34.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.973+0000 7fe369a07740 1 -- 172.21.15.42:0/3508419093 >> 172.21.15.42:0/3508419093 conn(0x564ad2252460 msgr2=0x564ad260d6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:12:34.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.974+0000 7fe369a07740 1 -- 172.21.15.42:0/3508419093 shutdown_connections 2024-08-30T09:12:34.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.974+0000 7fe369a07740 1 -- 172.21.15.42:0/3508419093 wait complete. 2024-08-30T09:12:34.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.974+0000 7fe369a07740 20 client.27194 trim_cache size 0 max 16384 2024-08-30T09:12:34.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.991+0000 7fe369a07740 1 Processor -- start 2024-08-30T09:12:34.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.991+0000 7fe369a07740 1 -- start start 2024-08-30T09:12:34.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.991+0000 7fe369a07740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564ad22351b0 0x564ad2607ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:12:34.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.991+0000 7fe369a07740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564ad223d390 0x564ad26030f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:12:34.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.991+0000 7fe369a07740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564ad225e070 0x564ad2603630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:12:34.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.991+0000 7fe369a07740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fe35c1b9700 con 0x564ad22351b0 2024-08-30T09:12:34.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.991+0000 7fe369a07740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe35c1b6fd0 con 0x564ad225e070 2024-08-30T09:12:34.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.991+0000 7fe369a07740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fe35c1b4e20 con 0x564ad223d390 2024-08-30T09:12:34.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.991+0000 7fe36122e640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564ad22351b0 0x564ad2607ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:12:34.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.991+0000 7fe361a2f640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564ad225e070 0x564ad2603630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:12:34.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.991+0000 7fe361a2f640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564ad225e070 0x564ad2603630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:42510/0 (socket says 172.21.15.42:42510) 2024-08-30T09:12:34.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.991+0000 7fe361a2f640 1 -- 172.21.15.42:0/154159454 learned_addr learned my addr 172.21.15.42:0/154159454 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:12:34.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.991+0000 7fe360a2d640 1 --2- 172.21.15.42:0/154159454 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564ad223d390 0x564ad26030f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:12:34.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.992+0000 7fe361a2f640 1 -- 172.21.15.42:0/154159454 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564ad223d390 msgr2=0x564ad26030f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:12:34.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.992+0000 7fe361a2f640 1 --2- 172.21.15.42:0/154159454 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564ad223d390 0x564ad26030f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:34.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.992+0000 7fe361a2f640 1 -- 172.21.15.42:0/154159454 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564ad22351b0 msgr2=0x564ad2607ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:12:34.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.992+0000 7fe361a2f640 1 --2- 172.21.15.42:0/154159454 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564ad22351b0 0x564ad2607ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:34.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.992+0000 7fe361a2f640 1 -- 172.21.15.42:0/154159454 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ad26071c0 con 0x564ad225e070 2024-08-30T09:12:34.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.992+0000 7fe36122e640 1 --2- 172.21.15.42:0/154159454 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564ad22351b0 0x564ad2607ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-30T09:12:34.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.992+0000 7fe361a2f640 1 --2- 172.21.15.42:0/154159454 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564ad225e070 0x564ad2603630 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fe354012f50 tx=0x7fe35401a000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=19a9bf01c7cf988e server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:12:34.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.992+0000 7fe346ffd640 1 -- 172.21.15.42:0/154159454 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe354004050 con 0x564ad225e070 2024-08-30T09:12:34.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.992+0000 7fe346ffd640 1 -- 172.21.15.42:0/154159454 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe3540041f0 con 0x564ad225e070 2024-08-30T09:12:34.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.992+0000 7fe346ffd640 1 -- 172.21.15.42:0/154159454 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe354003b30 con 0x564ad225e070 2024-08-30T09:12:34.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.993+0000 7fe369a07740 1 -- 172.21.15.42:0/154159454 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564ad225e070 msgr2=0x564ad2603630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:12:34.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.993+0000 7fe369a07740 1 --2- 172.21.15.42:0/154159454 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564ad225e070 0x564ad2603630 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fe354012f50 tx=0x7fe35401a000 comp rx=0 tx=0).stop 2024-08-30T09:12:34.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.993+0000 7fe369a07740 1 -- 172.21.15.42:0/154159454 shutdown_connections 2024-08-30T09:12:34.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.993+0000 7fe369a07740 1 --2- 172.21.15.42:0/154159454 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564ad223d390 0x564ad26030f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:34.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.993+0000 7fe369a07740 1 --2- 172.21.15.42:0/154159454 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564ad225e070 0x564ad2603630 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:34.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.993+0000 7fe369a07740 1 --2- 172.21.15.42:0/154159454 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564ad22351b0 0x564ad2607ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:34.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.993+0000 7fe369a07740 1 -- 172.21.15.42:0/154159454 >> 172.21.15.42:0/154159454 conn(0x564ad2252460 msgr2=0x564ad262ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:12:34.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.993+0000 7fe369a07740 1 -- 172.21.15.42:0/154159454 shutdown_connections 2024-08-30T09:12:34.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.994+0000 7fe369a07740 1 -- 172.21.15.42:0/154159454 wait complete. 2024-08-30T09:12:34.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.994+0000 7fe369a07740 1 -- start start 2024-08-30T09:12:34.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.994+0000 7fe369a07740 1 Processor -- start 2024-08-30T09:12:34.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.995+0000 7fe369a07740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564ad22351b0 0x564ad2633030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:12:34.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.995+0000 7fe369a07740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564ad223d390 0x564ad2633570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:12:34.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.995+0000 7fe360a2d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564ad223d390 0x564ad2633570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:12:35.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.995+0000 7fe36122e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564ad22351b0 0x564ad2633030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:12:35.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.995+0000 7fe360a2d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564ad223d390 0x564ad2633570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41456/0 (socket says 172.21.15.42:41456) 2024-08-30T09:12:35.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.995+0000 7fe369a07740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564ad225e070 0x564ad269b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:12:35.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.995+0000 7fe36122e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564ad22351b0 0x564ad2633030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:33102/0 (socket says 172.21.15.42:33102) 2024-08-30T09:12:35.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.995+0000 7fe360a2d640 1 -- 172.21.15.42:0/854418809 learned_addr learned my addr 172.21.15.42:0/854418809 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:12:35.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.995+0000 7fe369a07740 1 -- 172.21.15.42:0/854418809 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fe35c1b5490 con 0x564ad223d390 2024-08-30T09:12:35.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.995+0000 7fe369a07740 1 -- 172.21.15.42:0/854418809 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe35c1b6900 con 0x564ad225e070 2024-08-30T09:12:35.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.995+0000 7fe369a07740 1 -- 172.21.15.42:0/854418809 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fe35c1b5b60 con 0x564ad22351b0 2024-08-30T09:12:35.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.995+0000 7fe369a07740 10 client.0 _ll_register_callbacks cb 0x7fe354009570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:12:35.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.995+0000 7fe361a2f640 1 --2- 172.21.15.42:0/854418809 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564ad225e070 0x564ad269b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:12:35.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.995+0000 7fe36122e640 1 -- 172.21.15.42:0/854418809 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564ad225e070 msgr2=0x564ad269b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:12:35.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.995+0000 7fe36122e640 1 --2- 172.21.15.42:0/854418809 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564ad225e070 0x564ad269b4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:35.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.996+0000 7fe36122e640 1 -- 172.21.15.42:0/854418809 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564ad223d390 msgr2=0x564ad2633570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:12:35.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.996+0000 7fe36122e640 1 --2- 172.21.15.42:0/854418809 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564ad223d390 0x564ad2633570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:35.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.996+0000 7fe36122e640 1 -- 172.21.15.42:0/854418809 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ad2633e10 con 0x564ad22351b0 2024-08-30T09:12:35.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.996+0000 7fe36122e640 1 --2- 172.21.15.42:0/854418809 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564ad22351b0 0x564ad2633030 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fe3580a82a0 tx=0x7fe358009200 comp rx=0 tx=0).ready entity=mon.1 client_cookie=480ae0bf6428465a server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:12:35.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.996+0000 7fe344ff9640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-08-30T09:12:35.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.996+0000 7fe344ff9640 1 -- 172.21.15.42:0/854418809 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3580f5040 con 0x564ad22351b0 2024-08-30T09:12:35.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.996+0000 7fe344ff9640 1 -- 172.21.15.42:0/854418809 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7fe3580105d0 con 0x564ad22351b0 2024-08-30T09:12:35.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.996+0000 7fe369a07740 1 -- 172.21.15.42:0/854418809 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe3540041f0 con 0x564ad22351b0 2024-08-30T09:12:35.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.996+0000 7fe344ff9640 1 -- 172.21.15.42:0/854418809 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fe3580ab990 con 0x564ad22351b0 2024-08-30T09:12:35.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.996+0000 7fe369a07740 10 client.27160 fetch_fsmap learned FSMap version 43 2024-08-30T09:12:35.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.996+0000 7fe369a07740 1 -- 172.21.15.42:0/854418809 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x564ad269c9f0 con 0x564ad22351b0 2024-08-30T09:12:35.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.997+0000 7fe344ff9640 1 -- 172.21.15.42:0/854418809 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe3580abd30 con 0x564ad22351b0 2024-08-30T09:12:35.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.997+0000 7fe344ff9640 1 -- 172.21.15.42:0/854418809 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7fe3580ab480 con 0x564ad22351b0 2024-08-30T09:12:35.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.997+0000 7fe369a07740 10 client.27160 fetch_fsmap finished waiting for FSMap version 43 2024-08-30T09:12:35.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.997+0000 7fe369a07740 10 client.27160 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-08-30T09:12:35.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.997+0000 7fe369a07740 20 client.27160 populate_metadata read hostname 'smithi042' 2024-08-30T09:12:35.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.997+0000 7fe369a07740 1 --2- 172.21.15.42:0/854418809 >> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] conn(0x564ad21f1da0 0x564ad269ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:12:35.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.997+0000 7fe369a07740 4 client.27160 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-30T09:12:35.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.997+0000 7fe369a07740 1 -- 172.21.15.42:0/854418809 --> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fe3540041f0 con 0x564ad21f1da0 2024-08-30T09:12:35.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.997+0000 7fe360a2d640 1 --2- 172.21.15.42:0/854418809 >> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] conn(0x564ad21f1da0 0x564ad269ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:12:35.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.998+0000 7fe344ff9640 10 client.27160 ms_handle_connect on v2:172.21.15.167:6836/3011837856 2024-08-30T09:12:35.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.998+0000 7fe360a2d640 1 --2- 172.21.15.42:0/854418809 >> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] conn(0x564ad21f1da0 0x564ad269ee80 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ee60d291f3c15adb server_cookie=bfdc92f454f86a2b in_seq=0 out_seq=0 2024-08-30T09:12:35.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.998+0000 7fe344ff9640 1 -- 172.21.15.42:0/854418809 <== mds.2 v2:172.21.15.167:6836/3011837856 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fe3540041f0 con 0x564ad21f1da0 2024-08-30T09:12:35.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.998+0000 7fe344ff9640 10 client.27160 handle_command_reply: tid=0 2024-08-30T09:12:35.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.998+0000 7fe369a07740 1 client.27160 shutdown 2024-08-30T09:12:35.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.999+0000 7fe369a07740 1 -- 172.21.15.42:0/854418809 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564ad22351b0 msgr2=0x564ad2633030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:12:35.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.999+0000 7fe369a07740 1 --2- 172.21.15.42:0/854418809 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564ad22351b0 0x564ad2633030 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fe3580a82a0 tx=0x7fe358009200 comp rx=0 tx=0).stop 2024-08-30T09:12:35.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.999+0000 7fe369a07740 1 -- 172.21.15.42:0/854418809 shutdown_connections 2024-08-30T09:12:35.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.999+0000 7fe369a07740 1 --2- 172.21.15.42:0/854418809 >> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] conn(0x564ad21f1da0 0x564ad269ee80 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:35.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.999+0000 7fe369a07740 1 --2- 172.21.15.42:0/854418809 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564ad225e070 0x564ad269b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:35.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.999+0000 7fe369a07740 1 --2- 172.21.15.42:0/854418809 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564ad223d390 0x564ad2633570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:35.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.999+0000 7fe369a07740 1 --2- 172.21.15.42:0/854418809 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564ad22351b0 0x564ad2633030 secure :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fe3580a82a0 tx=0x7fe358009200 comp rx=0 tx=0).stop 2024-08-30T09:12:35.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.999+0000 7fe369a07740 1 -- 172.21.15.42:0/854418809 >> 172.21.15.42:0/854418809 conn(0x564ad2252460 msgr2=0x564ad262ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:12:35.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.999+0000 7fe344ff9640 0 client.27160 ms_handle_reset on v2:172.21.15.167:6836/3011837856 2024-08-30T09:12:35.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.999+0000 7fe369a07740 1 -- 172.21.15.42:0/854418809 shutdown_connections 2024-08-30T09:12:35.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.999+0000 7fe369a07740 1 -- 172.21.15.42:0/854418809 wait complete. 2024-08-30T09:12:35.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:34.999+0000 7fe369a07740 20 client.27160 trim_cache size 0 max 16384 2024-08-30T09:12:35.005 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:12:34 smithi042 ceph-mon[18652]: pgmap v727: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 6.3 MiB/s wr, 553 op/s 2024-08-30T09:12:35.005 INFO:teuthology.orchestra.run.smithi042.stdout:[] 2024-08-30T09:12:35.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:35.000+0000 7fe362230640 1 -- 172.21.15.42:0/1225499620 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fe3480b4c70 msgr2=0x7fe3480b7180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:12:35.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:35.000+0000 7fe362230640 1 --2- 172.21.15.42:0/1225499620 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fe3480b4c70 0x7fe3480b7180 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fe3540126c0 tx=0x7fe354005f90 comp rx=0 tx=0).stop 2024-08-30T09:12:35.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:35.000+0000 7fe362230640 1 -- 172.21.15.42:0/1225499620 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe35c10a3e0 msgr2=0x7fe35c1a2900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:12:35.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:35.000+0000 7fe362230640 1 --2- 172.21.15.42:0/1225499620 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe35c10a3e0 0x7fe35c1a2900 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fe35800ebc0 tx=0x7fe35800c2d0 comp rx=0 tx=0).stop 2024-08-30T09:12:35.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:35.000+0000 7fe362230640 1 -- 172.21.15.42:0/1225499620 shutdown_connections 2024-08-30T09:12:35.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:35.000+0000 7fe362230640 1 --2- 172.21.15.42:0/1225499620 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fe3480b4c70 0x7fe3480b7180 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:35.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:35.000+0000 7fe362230640 1 --2- 172.21.15.42:0/1225499620 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe35c10ad30 0x7fe35c1a2ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:35.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:35.000+0000 7fe362230640 1 --2- 172.21.15.42:0/1225499620 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe35c1162b0 0x7fe35c1a8140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:35.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:35.000+0000 7fe362230640 1 --2- 172.21.15.42:0/1225499620 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe35c10a3e0 0x7fe35c1a2900 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:35.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:35.000+0000 7fe362230640 1 -- 172.21.15.42:0/1225499620 >> 172.21.15.42:0/1225499620 conn(0x7fe35c06e790 msgr2=0x7fe35c072ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:12:35.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:35.000+0000 7fe362230640 1 -- 172.21.15.42:0/1225499620 shutdown_connections 2024-08-30T09:12:35.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:35.001+0000 7fe362230640 1 -- 172.21.15.42:0/1225499620 wait complete. 2024-08-30T09:12:35.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:12:34 smithi167 ceph-mon[22063]: pgmap v727: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 6.3 MiB/s wr, 553 op/s 2024-08-30T09:12:35.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:12:34 smithi190 ceph-mon[22001]: pgmap v727: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 6.3 MiB/s wr, 553 op/s 2024-08-30T09:12:35.953 INFO:teuthology.orchestra.run.smithi042.stderr:progress: 710.0 s, 463.6 tps, lat 220.662 ms stddev 274.307 2024-08-30T09:12:36.026 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-08-30T09:12:36.553 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T09:12:36.561 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.559+0000 7f4571607640 1 -- 172.21.15.42:0/2853086236 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f456c10a160 msgr2=0x7f456c10c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:12:36.561 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.559+0000 7f4571607640 1 --2- 172.21.15.42:0/2853086236 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f456c10a160 0x7f456c10c580 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f455c009900 tx=0x7f455c02a440 comp rx=0 tx=0).stop 2024-08-30T09:12:36.566 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.560+0000 7f4571607640 1 -- 172.21.15.42:0/2853086236 shutdown_connections 2024-08-30T09:12:36.566 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.560+0000 7f4571607640 1 --2- 172.21.15.42:0/2853086236 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f456c10f770 0x7f456c11a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:36.566 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.560+0000 7f4571607640 1 --2- 172.21.15.42:0/2853086236 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f456c10cb50 0x7f456c10f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:36.566 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.560+0000 7f4571607640 1 --2- 172.21.15.42:0/2853086236 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f456c10a160 0x7f456c10c580 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:36.566 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.560+0000 7f4571607640 1 -- 172.21.15.42:0/2853086236 >> 172.21.15.42:0/2853086236 conn(0x7f456c0705b0 msgr2=0x7f456c072a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:12:36.566 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.564+0000 7f4571607640 1 -- 172.21.15.42:0/2853086236 shutdown_connections 2024-08-30T09:12:36.568 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.566+0000 7f4571607640 1 -- 172.21.15.42:0/2853086236 wait complete. 2024-08-30T09:12:36.569 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.567+0000 7f4571607640 1 Processor -- start 2024-08-30T09:12:36.573 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.571+0000 7f4571607640 1 -- start start 2024-08-30T09:12:36.573 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.571+0000 7f4571607640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f456c10a160 0x7f456c1a8b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:12:36.573 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.571+0000 7f4571607640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f456c10cb50 0x7f456c1a91d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:12:36.573 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.571+0000 7f4571607640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f456c10f770 0x7f456c1a9d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:12:36.573 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.571+0000 7f4571607640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f456c11df30 con 0x7f456c10f770 2024-08-30T09:12:36.573 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.571+0000 7f4571607640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f456c11ddb0 con 0x7f456c10a160 2024-08-30T09:12:36.574 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.571+0000 7f4571607640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f456c11e0b0 con 0x7f456c10cb50 2024-08-30T09:12:36.576 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.573+0000 7f456b7fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f456c10f770 0x7f456c1a9d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:12:36.576 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.573+0000 7f456b7fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f456c10f770 0x7f456c1a9d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41464/0 (socket says 172.21.15.42:41464) 2024-08-30T09:12:36.576 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.573+0000 7f456b7fe640 1 -- 172.21.15.42:0/2248289509 learned_addr learned my addr 172.21.15.42:0/2248289509 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:12:36.577 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.574+0000 7f456a7fc640 1 --2- 172.21.15.42:0/2248289509 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f456c10cb50 0x7f456c1a91d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:12:36.577 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.574+0000 7f456affd640 1 --2- 172.21.15.42:0/2248289509 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f456c10a160 0x7f456c1a8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:12:36.580 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.576+0000 7f456a7fc640 1 -- 172.21.15.42:0/2248289509 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f456c10a160 msgr2=0x7f456c1a8b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:12:36.580 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.576+0000 7f456a7fc640 1 --2- 172.21.15.42:0/2248289509 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f456c10a160 0x7f456c1a8b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:36.580 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.576+0000 7f456a7fc640 1 -- 172.21.15.42:0/2248289509 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f456c10f770 msgr2=0x7f456c1a9d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:12:36.581 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.576+0000 7f456a7fc640 1 --2- 172.21.15.42:0/2248289509 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f456c10f770 0x7f456c1a9d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:36.581 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.576+0000 7f456a7fc640 1 -- 172.21.15.42:0/2248289509 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f456c1aa5a0 con 0x7f456c10cb50 2024-08-30T09:12:36.585 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.580+0000 7f456a7fc640 1 --2- 172.21.15.42:0/2248289509 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f456c10cb50 0x7f456c1a91d0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f456000c970 tx=0x7f456000ce30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:12:36.590 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.589+0000 7f4553fff640 1 -- 172.21.15.42:0/2248289509 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4560007bb0 con 0x7f456c10cb50 2024-08-30T09:12:36.591 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.589+0000 7f4553fff640 1 -- 172.21.15.42:0/2248289509 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4560007d50 con 0x7f456c10cb50 2024-08-30T09:12:36.591 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.589+0000 7f4553fff640 1 -- 172.21.15.42:0/2248289509 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4560004df0 con 0x7f456c10cb50 2024-08-30T09:12:36.591 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.589+0000 7f4571607640 1 -- 172.21.15.42:0/2248289509 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f456c1aa890 con 0x7f456c10cb50 2024-08-30T09:12:36.591 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.589+0000 7f4571607640 1 -- 172.21.15.42:0/2248289509 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f456c112300 con 0x7f456c10cb50 2024-08-30T09:12:36.604 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.595+0000 7f457864f740 1 Processor -- start 2024-08-30T09:12:36.604 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.602+0000 7f4553fff640 1 -- 172.21.15.42:0/2248289509 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7f4560004fb0 con 0x7f456c10cb50 2024-08-30T09:12:36.605 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.603+0000 7f457864f740 1 -- start start 2024-08-30T09:12:36.605 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.603+0000 7f457864f740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556a282831b0 0x556a28628f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:12:36.605 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.603+0000 7f457864f740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556a2825a2f0 0x556a286295d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:12:36.606 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.603+0000 7f457864f740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556a282624d0 0x556a2862a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:12:36.606 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.603+0000 7f457864f740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x556a2859c000 con 0x556a2825a2f0 2024-08-30T09:12:36.606 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.603+0000 7f457864f740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x556a2858b590 con 0x556a282831b0 2024-08-30T09:12:36.606 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.603+0000 7f457864f740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x556a2858f540 con 0x556a282624d0 2024-08-30T09:12:36.606 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.603+0000 7f456b7fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556a282831b0 0x556a28628f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:12:36.606 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.604+0000 7f456b7fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556a282831b0 0x556a28628f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:42556/0 (socket says 172.21.15.42:42556) 2024-08-30T09:12:36.606 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.604+0000 7f456b7fe640 1 -- 172.21.15.42:0/21191160 learned_addr learned my addr 172.21.15.42:0/21191160 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:12:36.607 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.604+0000 7f456b7fe640 1 -- 172.21.15.42:0/21191160 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556a282624d0 msgr2=0x556a2862a120 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-30T09:12:36.607 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.604+0000 7f456b7fe640 1 --2- 172.21.15.42:0/21191160 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556a282624d0 0x556a2862a120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:36.607 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.604+0000 7f456b7fe640 1 -- 172.21.15.42:0/21191160 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556a2825a2f0 msgr2=0x556a286295d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-30T09:12:36.607 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.604+0000 7f456b7fe640 1 --2- 172.21.15.42:0/21191160 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556a2825a2f0 0x556a286295d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:36.607 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.604+0000 7f456b7fe640 1 -- 172.21.15.42:0/21191160 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556a282ee790 con 0x556a282831b0 2024-08-30T09:12:36.607 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.605+0000 7f456b7fe640 1 --2- 172.21.15.42:0/21191160 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556a282831b0 0x556a28628f60 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f456400e1c0 tx=0x7f4564009710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=553e7a5b1cad4d57 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:12:36.608 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.605+0000 7f4553fff640 1 --2- 172.21.15.42:0/2248289509 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f45540b4d40 0x7f45540b7250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:12:36.608 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.605+0000 7f4553fff640 1 -- 172.21.15.42:0/2248289509 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f45600e5100 con 0x7f456c10cb50 2024-08-30T09:12:36.608 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.606+0000 7f45517fa640 1 -- 172.21.15.42:0/21191160 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4564018020 con 0x556a282831b0 2024-08-30T09:12:36.608 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.606+0000 7f45517fa640 1 -- 172.21.15.42:0/21191160 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4564014b30 con 0x556a282831b0 2024-08-30T09:12:36.609 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.607+0000 7f457864f740 1 -- 172.21.15.42:0/21191160 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556a282831b0 msgr2=0x556a28628f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:12:36.609 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.607+0000 7f457864f740 1 --2- 172.21.15.42:0/21191160 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556a282831b0 0x556a28628f60 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f456400e1c0 tx=0x7f4564009710 comp rx=0 tx=0).stop 2024-08-30T09:12:36.611 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.610+0000 7f457864f740 1 -- 172.21.15.42:0/21191160 shutdown_connections 2024-08-30T09:12:36.612 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.610+0000 7f457864f740 1 --2- 172.21.15.42:0/21191160 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556a282624d0 0x556a2862a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:36.612 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.610+0000 7f457864f740 1 --2- 172.21.15.42:0/21191160 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556a2825a2f0 0x556a286295d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:36.612 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.610+0000 7f457864f740 1 --2- 172.21.15.42:0/21191160 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556a282831b0 0x556a28628f60 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:36.612 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.610+0000 7f457864f740 1 -- 172.21.15.42:0/21191160 >> 172.21.15.42:0/21191160 conn(0x556a28277300 msgr2=0x556a286249f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:12:36.619 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.617+0000 7f456affd640 1 --2- 172.21.15.42:0/2248289509 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f45540b4d40 0x7f45540b7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:12:36.620 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.618+0000 7f457864f740 1 -- 172.21.15.42:0/21191160 shutdown_connections 2024-08-30T09:12:36.623 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.621+0000 7f457864f740 1 -- 172.21.15.42:0/21191160 wait complete. 2024-08-30T09:12:36.623 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.621+0000 7f457864f740 1 -- start start 2024-08-30T09:12:36.623 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.621+0000 7f457864f740 1 Processor -- start 2024-08-30T09:12:36.624 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.622+0000 7f457864f740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556a2825a2f0 0x556a286329e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:12:36.624 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.622+0000 7f457864f740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556a282624d0 0x556a286330a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:12:36.624 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.622+0000 7f457864f740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556a282831b0 0x556a28633c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:12:36.624 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.622+0000 7f457864f740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x556a284c4ab0 con 0x556a282624d0 2024-08-30T09:12:36.624 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.622+0000 7f457864f740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x556a28512680 con 0x556a2825a2f0 2024-08-30T09:12:36.624 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.622+0000 7f457864f740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x556a2862ac90 con 0x556a282831b0 2024-08-30T09:12:36.624 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.622+0000 7f457864f740 10 client.0 _ll_register_callbacks cb 0x556a2829aa00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:12:36.626 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.624+0000 7f456b7fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556a282831b0 0x556a28633c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:12:36.626 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.624+0000 7f456a7fc640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556a282624d0 0x556a286330a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:12:36.626 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.624+0000 7f456a7fc640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556a282624d0 0x556a286330a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41470/0 (socket says 172.21.15.42:41470) 2024-08-30T09:12:36.626 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.624+0000 7f456b7fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556a282831b0 0x556a28633c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:33144/0 (socket says 172.21.15.42:33144) 2024-08-30T09:12:36.626 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.624+0000 7f456a7fc640 1 -- 172.21.15.42:0/3272509913 learned_addr learned my addr 172.21.15.42:0/3272509913 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:12:36.627 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.625+0000 7f456b7fe640 1 -- 172.21.15.42:0/3272509913 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556a2825a2f0 msgr2=0x556a286329e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-30T09:12:36.627 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.625+0000 7f456b7fe640 1 --2- 172.21.15.42:0/3272509913 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556a2825a2f0 0x556a286329e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:36.627 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.625+0000 7f456b7fe640 1 -- 172.21.15.42:0/3272509913 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556a282624d0 msgr2=0x556a286330a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:12:36.627 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.625+0000 7f456b7fe640 1 --2- 172.21.15.42:0/3272509913 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556a282624d0 0x556a286330a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:36.627 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.625+0000 7f456b7fe640 1 -- 172.21.15.42:0/3272509913 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556a28367620 con 0x556a282831b0 2024-08-30T09:12:36.628 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.625+0000 7f456b7fe640 1 --2- 172.21.15.42:0/3272509913 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556a282831b0 0x556a28633c10 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f4564004b90 tx=0x7f45640146b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=29db060cad68d358 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:12:36.628 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.626+0000 7f454eff5640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-08-30T09:12:36.628 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.626+0000 7f454eff5640 1 -- 172.21.15.42:0/3272509913 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4564029650 con 0x556a282831b0 2024-08-30T09:12:36.628 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.626+0000 7f457864f740 1 -- 172.21.15.42:0/3272509913 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4564014b30 con 0x556a282831b0 2024-08-30T09:12:36.628 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.626+0000 7f454eff5640 1 -- 172.21.15.42:0/3272509913 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f45640297f0 con 0x556a282831b0 2024-08-30T09:12:36.628 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.626+0000 7f454eff5640 1 -- 172.21.15.42:0/3272509913 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f4564032050 con 0x556a282831b0 2024-08-30T09:12:36.630 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.629+0000 7f457864f740 10 client.27169 fetch_fsmap learned FSMap version 43 2024-08-30T09:12:36.631 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.629+0000 7f457864f740 1 -- 172.21.15.42:0/3272509913 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x556a2863cbb0 con 0x556a282831b0 2024-08-30T09:12:36.633 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.631+0000 7f454eff5640 1 -- 172.21.15.42:0/3272509913 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4564014b30 con 0x556a282831b0 2024-08-30T09:12:36.633 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.631+0000 7f454eff5640 1 -- 172.21.15.42:0/3272509913 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f4564014ce0 con 0x556a282831b0 2024-08-30T09:12:36.633 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.631+0000 7f457864f740 10 client.27169 fetch_fsmap finished waiting for FSMap version 43 2024-08-30T09:12:36.633 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.631+0000 7f457864f740 10 client.27169 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-30T09:12:36.633 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.631+0000 7f457864f740 20 client.27169 populate_metadata read hostname 'smithi042' 2024-08-30T09:12:36.633 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.631+0000 7f457864f740 1 --2- 172.21.15.42:0/3272509913 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x556a28241d80 0x556a28640590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:12:36.633 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.631+0000 7f457864f740 4 client.27169 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-30T09:12:36.633 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.631+0000 7f457864f740 1 -- 172.21.15.42:0/3272509913 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556a2859df50 con 0x556a28241d80 2024-08-30T09:12:36.641 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.639+0000 7f456affd640 1 --2- 172.21.15.42:0/3272509913 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x556a28241d80 0x556a28640590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:12:36.641 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.639+0000 7f456affd640 1 --2- 172.21.15.42:0/2248289509 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f45540b4d40 0x7f45540b7250 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f455c006010 tx=0x7f455c000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:12:36.656 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.655+0000 7f456affd640 1 --2- 172.21.15.42:0/3272509913 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x556a28241d80 0x556a28640590 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f1caf1c13e63b097 server_cookie=f6428df73de67ec8 in_seq=0 out_seq=0 2024-08-30T09:12:36.657 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.656+0000 7f454eff5640 10 client.27169 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:12:36.659 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.657+0000 7f454eff5640 1 -- 172.21.15.42:0/3272509913 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x556a2859df50 con 0x556a28241d80 2024-08-30T09:12:36.659 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.657+0000 7f454eff5640 10 client.27169 handle_command_reply: tid=0 2024-08-30T09:12:36.659 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.657+0000 7f457864f740 1 client.27169 shutdown 2024-08-30T09:12:36.664 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.662+0000 7f457864f740 1 -- 172.21.15.42:0/3272509913 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556a282831b0 msgr2=0x556a28633c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:12:36.664 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.662+0000 7f457864f740 1 --2- 172.21.15.42:0/3272509913 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556a282831b0 0x556a28633c10 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f4564004b90 tx=0x7f45640146b0 comp rx=0 tx=0).stop 2024-08-30T09:12:36.665 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.663+0000 7f457864f740 1 -- 172.21.15.42:0/3272509913 shutdown_connections 2024-08-30T09:12:36.665 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.663+0000 7f457864f740 1 --2- 172.21.15.42:0/3272509913 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x556a28241d80 0x556a28640590 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:36.666 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.664+0000 7f457864f740 1 --2- 172.21.15.42:0/3272509913 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556a282831b0 0x556a28633c10 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:36.666 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.664+0000 7f457864f740 1 --2- 172.21.15.42:0/3272509913 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556a282624d0 0x556a286330a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:36.666 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.664+0000 7f457864f740 1 --2- 172.21.15.42:0/3272509913 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556a2825a2f0 0x556a286329e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:36.666 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.664+0000 7f457864f740 1 -- 172.21.15.42:0/3272509913 >> 172.21.15.42:0/3272509913 conn(0x556a28277300 msgr2=0x556a28622bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:12:36.668 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.667+0000 7f454eff5640 0 client.27169 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:12:36.672 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.670+0000 7f457864f740 1 -- 172.21.15.42:0/3272509913 shutdown_connections 2024-08-30T09:12:36.672 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.670+0000 7f457864f740 1 -- 172.21.15.42:0/3272509913 wait complete. 2024-08-30T09:12:36.672 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.670+0000 7f457864f740 20 client.27169 trim_cache size 0 max 16384 2024-08-30T09:12:36.716 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.714+0000 7f457864f740 1 Processor -- start 2024-08-30T09:12:36.718 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.716+0000 7f457864f740 1 -- start start 2024-08-30T09:12:36.718 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.716+0000 7f457864f740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556a2825a2f0 0x556a28623b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:12:36.718 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.717+0000 7f457864f740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556a282624d0 0x556a28631c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:12:36.719 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.717+0000 7f457864f740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556a282831b0 0x556a28632140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:12:36.719 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.717+0000 7f456a7fc640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556a282624d0 0x556a28631c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:12:36.720 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.717+0000 7f456a7fc640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556a282624d0 0x556a28631c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:33152/0 (socket says 172.21.15.42:33152) 2024-08-30T09:12:36.720 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.717+0000 7f456b7fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556a282831b0 0x556a28632140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:12:36.720 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.717+0000 7f456b7fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556a282831b0 0x556a28632140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:42572/0 (socket says 172.21.15.42:42572) 2024-08-30T09:12:36.720 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.717+0000 7f456a7fc640 1 -- 172.21.15.42:0/2479584545 learned_addr learned my addr 172.21.15.42:0/2479584545 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:12:36.720 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.717+0000 7f457864f740 1 -- 172.21.15.42:0/2479584545 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f456c1c6590 con 0x556a2825a2f0 2024-08-30T09:12:36.720 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.717+0000 7f457864f740 1 -- 172.21.15.42:0/2479584545 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f456c1c3e60 con 0x556a282831b0 2024-08-30T09:12:36.720 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.717+0000 7f457864f740 1 -- 172.21.15.42:0/2479584545 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f456c1c1db0 con 0x556a282624d0 2024-08-30T09:12:36.722 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.721+0000 7f456affd640 1 --2- 172.21.15.42:0/2479584545 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556a2825a2f0 0x556a28623b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:12:36.726 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.724+0000 7f456b7fe640 1 -- 172.21.15.42:0/2479584545 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556a282624d0 msgr2=0x556a28631c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:12:36.726 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.724+0000 7f456b7fe640 1 --2- 172.21.15.42:0/2479584545 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556a282624d0 0x556a28631c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:36.726 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.724+0000 7f456b7fe640 1 -- 172.21.15.42:0/2479584545 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556a2825a2f0 msgr2=0x556a28623b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:12:36.726 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.724+0000 7f456b7fe640 1 --2- 172.21.15.42:0/2479584545 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556a2825a2f0 0x556a28623b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:36.726 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.724+0000 7f456b7fe640 1 -- 172.21.15.42:0/2479584545 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556a28625120 con 0x556a282831b0 2024-08-30T09:12:36.726 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.724+0000 7f456affd640 1 --2- 172.21.15.42:0/2479584545 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556a2825a2f0 0x556a28623b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-30T09:12:36.729 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.725+0000 7f456b7fe640 1 --2- 172.21.15.42:0/2479584545 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556a282831b0 0x556a28632140 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f4564002230 tx=0x7f4564009710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=513e49b55bba88e6 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:12:36.731 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.728+0000 7f4550ff9640 1 -- 172.21.15.42:0/2479584545 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4564018070 con 0x556a282831b0 2024-08-30T09:12:36.731 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.728+0000 7f4550ff9640 1 -- 172.21.15.42:0/2479584545 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4564009a30 con 0x556a282831b0 2024-08-30T09:12:36.731 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.729+0000 7f457864f740 1 -- 172.21.15.42:0/2479584545 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556a282831b0 msgr2=0x556a28632140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:12:36.731 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.729+0000 7f457864f740 1 --2- 172.21.15.42:0/2479584545 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556a282831b0 0x556a28632140 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f4564002230 tx=0x7f4564009710 comp rx=0 tx=0).stop 2024-08-30T09:12:36.736 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.733+0000 7f457864f740 1 -- 172.21.15.42:0/2479584545 shutdown_connections 2024-08-30T09:12:36.736 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.733+0000 7f457864f740 1 --2- 172.21.15.42:0/2479584545 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556a282624d0 0x556a28631c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:36.737 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.733+0000 7f457864f740 1 --2- 172.21.15.42:0/2479584545 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556a282831b0 0x556a28632140 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:36.737 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.733+0000 7f457864f740 1 --2- 172.21.15.42:0/2479584545 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556a2825a2f0 0x556a28623b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:36.737 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.733+0000 7f457864f740 1 -- 172.21.15.42:0/2479584545 >> 172.21.15.42:0/2479584545 conn(0x556a28277300 msgr2=0x556a2862d740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:12:36.743 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.739+0000 7f457864f740 1 -- 172.21.15.42:0/2479584545 shutdown_connections 2024-08-30T09:12:36.743 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.741+0000 7f457864f740 1 -- 172.21.15.42:0/2479584545 wait complete. 2024-08-30T09:12:36.743 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.742+0000 7f457864f740 1 -- start start 2024-08-30T09:12:36.744 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.742+0000 7f457864f740 1 Processor -- start 2024-08-30T09:12:36.744 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.742+0000 7f457864f740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556a2825a2f0 0x556a28635930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:12:36.744 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.742+0000 7f457864f740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556a282624d0 0x556a28635ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:12:36.745 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.742+0000 7f457864f740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556a282831b0 0x556a28636af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:12:36.745 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.742+0000 7f457864f740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f456c1c2320 con 0x556a282624d0 2024-08-30T09:12:36.745 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.742+0000 7f457864f740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f456c1c3790 con 0x556a2825a2f0 2024-08-30T09:12:36.745 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.742+0000 7f457864f740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f456c1c29f0 con 0x556a282831b0 2024-08-30T09:12:36.745 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.742+0000 7f457864f740 10 client.0 _ll_register_callbacks cb 0x7f456400ef90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:12:36.746 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.743+0000 7f456affd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556a2825a2f0 0x556a28635930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:12:36.746 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.743+0000 7f456b7fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556a282831b0 0x556a28636af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:12:36.746 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.743+0000 7f456b7fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556a282831b0 0x556a28636af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:33158/0 (socket says 172.21.15.42:33158) 2024-08-30T09:12:36.746 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.743+0000 7f456b7fe640 1 -- 172.21.15.42:0/1659912885 learned_addr learned my addr 172.21.15.42:0/1659912885 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:12:36.746 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.743+0000 7f456a7fc640 1 --2- 172.21.15.42:0/1659912885 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556a282624d0 0x556a28635ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:12:36.747 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.745+0000 7f456a7fc640 1 -- 172.21.15.42:0/1659912885 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556a282831b0 msgr2=0x556a28636af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:12:36.747 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.745+0000 7f456a7fc640 1 --2- 172.21.15.42:0/1659912885 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556a282831b0 0x556a28636af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:36.747 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.745+0000 7f456a7fc640 1 -- 172.21.15.42:0/1659912885 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556a2825a2f0 msgr2=0x556a28635930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:12:36.747 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.745+0000 7f456a7fc640 1 --2- 172.21.15.42:0/1659912885 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556a2825a2f0 0x556a28635930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:36.747 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.745+0000 7f456a7fc640 1 -- 172.21.15.42:0/1659912885 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556a28637250 con 0x556a282624d0 2024-08-30T09:12:36.748 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.745+0000 7f456a7fc640 1 --2- 172.21.15.42:0/1659912885 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556a282624d0 0x556a28635ff0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f45600a77c0 tx=0x7f45600abc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee07fc09644b7d8b server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:12:36.749 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.747+0000 7f454eff5640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-08-30T09:12:36.749 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.747+0000 7f454eff5640 1 -- 172.21.15.42:0/1659912885 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f456001d040 con 0x556a282624d0 2024-08-30T09:12:36.749 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.747+0000 7f457864f740 1 -- 172.21.15.42:0/1659912885 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4564009a30 con 0x556a282624d0 2024-08-30T09:12:36.750 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.747+0000 7f454eff5640 1 -- 172.21.15.42:0/1659912885 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f45600054b0 con 0x556a282624d0 2024-08-30T09:12:36.753 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.751+0000 7f454eff5640 1 -- 172.21.15.42:0/1659912885 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f45600a5e60 con 0x556a282624d0 2024-08-30T09:12:36.755 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.753+0000 7f457864f740 10 client.17379 fetch_fsmap learned FSMap version 43 2024-08-30T09:12:36.756 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.753+0000 7f457864f740 1 -- 172.21.15.42:0/1659912885 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x556a2863a620 con 0x556a282624d0 2024-08-30T09:12:36.767 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.765+0000 7f454eff5640 1 -- 172.21.15.42:0/1659912885 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4564009a30 con 0x556a282624d0 2024-08-30T09:12:36.770 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.766+0000 7f454eff5640 1 -- 172.21.15.42:0/1659912885 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f45600d21b0 con 0x556a282624d0 2024-08-30T09:12:36.771 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.767+0000 7f457864f740 10 client.17379 fetch_fsmap finished waiting for FSMap version 43 2024-08-30T09:12:36.771 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.767+0000 7f457864f740 10 client.17379 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-30T09:12:36.771 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.767+0000 7f457864f740 20 client.17379 populate_metadata read hostname 'smithi042' 2024-08-30T09:12:36.771 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.767+0000 7f457864f740 1 --2- 172.21.15.42:0/1659912885 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x556a282171c0 0x556a286ad460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:12:36.771 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.767+0000 7f457864f740 4 client.17379 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-30T09:12:36.771 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.767+0000 7f457864f740 1 -- 172.21.15.42:0/1659912885 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556a2862ae90 con 0x556a282171c0 2024-08-30T09:12:36.773 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.770+0000 7f456affd640 1 --2- 172.21.15.42:0/1659912885 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x556a282171c0 0x556a286ad460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:12:36.785 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.782+0000 7f456affd640 1 --2- 172.21.15.42:0/1659912885 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x556a282171c0 0x556a286ad460 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f03e4d8d53b82c8 server_cookie=fbea1a599d2bc24 in_seq=0 out_seq=0 2024-08-30T09:12:36.785 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.783+0000 7f454eff5640 10 client.17379 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:12:36.898 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.895+0000 7f454eff5640 1 -- 172.21.15.42:0/1659912885 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556a2862ae90 con 0x556a282171c0 2024-08-30T09:12:36.898 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.895+0000 7f454eff5640 10 client.17379 handle_command_reply: tid=0 2024-08-30T09:12:36.898 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.895+0000 7f457864f740 1 client.17379 shutdown 2024-08-30T09:12:36.900 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.898+0000 7f457864f740 1 -- 172.21.15.42:0/1659912885 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556a282624d0 msgr2=0x556a28635ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:12:36.900 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.898+0000 7f457864f740 1 --2- 172.21.15.42:0/1659912885 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556a282624d0 0x556a28635ff0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f45600a77c0 tx=0x7f45600abc90 comp rx=0 tx=0).stop 2024-08-30T09:12:36.904 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.902+0000 7f457864f740 1 -- 172.21.15.42:0/1659912885 shutdown_connections 2024-08-30T09:12:36.904 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.902+0000 7f457864f740 1 --2- 172.21.15.42:0/1659912885 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x556a282171c0 0x556a286ad460 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:36.904 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.902+0000 7f457864f740 1 --2- 172.21.15.42:0/1659912885 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556a282831b0 0x556a28636af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:36.904 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.902+0000 7f457864f740 1 --2- 172.21.15.42:0/1659912885 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556a282624d0 0x556a28635ff0 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:36.904 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.902+0000 7f457864f740 1 --2- 172.21.15.42:0/1659912885 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556a2825a2f0 0x556a28635930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:36.904 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.902+0000 7f457864f740 1 -- 172.21.15.42:0/1659912885 >> 172.21.15.42:0/1659912885 conn(0x556a28277300 msgr2=0x556a28631c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:12:36.904 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.902+0000 7f454eff5640 0 client.17379 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:12:36.906 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.904+0000 7f457864f740 1 -- 172.21.15.42:0/1659912885 shutdown_connections 2024-08-30T09:12:36.906 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.904+0000 7f457864f740 1 -- 172.21.15.42:0/1659912885 wait complete. 2024-08-30T09:12:36.906 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.904+0000 7f457864f740 20 client.17379 trim_cache size 0 max 16384 2024-08-30T09:12:36.909 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-08-30T09:12:36.909 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2024-08-30T09:12:36.909 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-08-30T09:12:36.909 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-08-30T09:12:36.911 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.909+0000 7f4571607640 1 -- 172.21.15.42:0/2248289509 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f45540b4d40 msgr2=0x7f45540b7250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:12:36.911 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.909+0000 7f4571607640 1 --2- 172.21.15.42:0/2248289509 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f45540b4d40 0x7f45540b7250 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f455c006010 tx=0x7f455c000bc0 comp rx=0 tx=0).stop 2024-08-30T09:12:36.911 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.909+0000 7f4571607640 1 -- 172.21.15.42:0/2248289509 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f456c10cb50 msgr2=0x7f456c1a91d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:12:36.911 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.909+0000 7f4571607640 1 --2- 172.21.15.42:0/2248289509 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f456c10cb50 0x7f456c1a91d0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f456000c970 tx=0x7f456000ce30 comp rx=0 tx=0).stop 2024-08-30T09:12:36.911 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.909+0000 7f4571607640 1 -- 172.21.15.42:0/2248289509 shutdown_connections 2024-08-30T09:12:36.911 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.909+0000 7f4571607640 1 --2- 172.21.15.42:0/2248289509 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f45540b4d40 0x7f45540b7250 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:36.911 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.909+0000 7f4571607640 1 --2- 172.21.15.42:0/2248289509 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f456c10cb50 0x7f456c1a91d0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:36.911 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.909+0000 7f4571607640 1 --2- 172.21.15.42:0/2248289509 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f456c10f770 0x7f456c1a9d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:36.911 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.909+0000 7f4571607640 1 --2- 172.21.15.42:0/2248289509 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f456c10a160 0x7f456c1a8b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:36.911 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.909+0000 7f4571607640 1 -- 172.21.15.42:0/2248289509 >> 172.21.15.42:0/2248289509 conn(0x7f456c0705b0 msgr2=0x7f456c10af30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:12:36.914 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.911+0000 7f4571607640 1 -- 172.21.15.42:0/2248289509 shutdown_connections 2024-08-30T09:12:36.914 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:36.911+0000 7f4571607640 1 -- 172.21.15.42:0/2248289509 wait complete. 2024-08-30T09:12:36.988 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:12:36 smithi042 ceph-mon[18652]: pgmap v728: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 5.0 MiB/s wr, 439 op/s 2024-08-30T09:12:37.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:12:36 smithi167 ceph-mon[22063]: pgmap v728: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 5.0 MiB/s wr, 439 op/s 2024-08-30T09:12:37.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:12:36 smithi190 ceph-mon[22001]: pgmap v728: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 5.0 MiB/s wr, 439 op/s 2024-08-30T09:12:37.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.267+0000 7fa8f98c0640 1 -- 172.21.15.42:0/838952876 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa8f4109a10 msgr2=0x7fa8f4110240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:12:37.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.267+0000 7fa8f98c0640 1 --2- 172.21.15.42:0/838952876 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa8f4109a10 0x7fa8f4110240 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fa8ec009ca0 tx=0x7fa8ec02a8b0 comp rx=0 tx=0).stop 2024-08-30T09:12:37.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.267+0000 7fa8f98c0640 1 -- 172.21.15.42:0/838952876 shutdown_connections 2024-08-30T09:12:37.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.267+0000 7fa8f98c0640 1 --2- 172.21.15.42:0/838952876 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa8f4109a10 0x7fa8f4110240 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:37.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.267+0000 7fa8f98c0640 1 --2- 172.21.15.42:0/838952876 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa8f41090c0 0x7fa8f41094d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:37.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.267+0000 7fa8f98c0640 1 --2- 172.21.15.42:0/838952876 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa8f4103780 0x7fa8f4103b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:37.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.267+0000 7fa8f98c0640 1 -- 172.21.15.42:0/838952876 >> 172.21.15.42:0/838952876 conn(0x7fa8f40ff1d0 msgr2=0x7fa8f4101620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:12:37.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.267+0000 7fa8f98c0640 1 -- 172.21.15.42:0/838952876 shutdown_connections 2024-08-30T09:12:37.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.267+0000 7fa8f98c0640 1 -- 172.21.15.42:0/838952876 wait complete. 2024-08-30T09:12:37.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.269+0000 7fa8f98c0640 1 Processor -- start 2024-08-30T09:12:37.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.269+0000 7fa8f98c0640 1 -- start start 2024-08-30T09:12:37.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.269+0000 7fa8f98c0640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa8f4103780 0x7fa8f4197ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:12:37.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.269+0000 7fa8f98c0640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa8f41090c0 0x7fa8f4198280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:12:37.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.269+0000 7fa8f98c0640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa8f4109a10 0x7fa8f419d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:12:37.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.269+0000 7fa8f98c0640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fa8f41114e0 con 0x7fa8f4109a10 2024-08-30T09:12:37.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.269+0000 7fa8f98c0640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa8f4111360 con 0x7fa8f41090c0 2024-08-30T09:12:37.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.269+0000 7fa8f98c0640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fa8f4111660 con 0x7fa8f4103780 2024-08-30T09:12:37.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.270+0000 7fa8f3fff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa8f41090c0 0x7fa8f4198280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:12:37.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.270+0000 7fa8f88be640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa8f4103780 0x7fa8f4197ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:12:37.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.270+0000 7fa8f3fff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa8f41090c0 0x7fa8f4198280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:42594/0 (socket says 172.21.15.42:42594) 2024-08-30T09:12:37.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.270+0000 7fa8f3fff640 1 -- 172.21.15.42:0/1857082052 learned_addr learned my addr 172.21.15.42:0/1857082052 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:12:37.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.271+0000 7fa8f3fff640 1 -- 172.21.15.42:0/1857082052 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa8f4103780 msgr2=0x7fa8f4197ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:12:37.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.271+0000 7fa8f3fff640 1 --2- 172.21.15.42:0/1857082052 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa8f4103780 0x7fa8f4197ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:37.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.271+0000 7fa8f3fff640 1 -- 172.21.15.42:0/1857082052 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa8f4109a10 msgr2=0x7fa8f419d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:12:37.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.271+0000 7fa8f3fff640 1 --2- 172.21.15.42:0/1857082052 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa8f4109a10 0x7fa8f419d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:37.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.271+0000 7fa8f3fff640 1 -- 172.21.15.42:0/1857082052 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8f419dbc0 con 0x7fa8f41090c0 2024-08-30T09:12:37.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.272+0000 7fa8f3fff640 1 --2- 172.21.15.42:0/1857082052 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa8f41090c0 0x7fa8f4198280 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fa8e000d740 tx=0x7fa8e000dc00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:12:37.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.273+0000 7fa8f1ffb640 1 -- 172.21.15.42:0/1857082052 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8e0017070 con 0x7fa8f41090c0 2024-08-30T09:12:37.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.274+0000 7fa8f1ffb640 1 -- 172.21.15.42:0/1857082052 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa8e000de50 con 0x7fa8f41090c0 2024-08-30T09:12:37.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.274+0000 7fa8f1ffb640 1 -- 172.21.15.42:0/1857082052 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8e000f620 con 0x7fa8f41090c0 2024-08-30T09:12:37.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.275+0000 7fa8f98c0640 1 -- 172.21.15.42:0/1857082052 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8f419deb0 con 0x7fa8f41090c0 2024-08-30T09:12:37.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.275+0000 7fa8f98c0640 1 -- 172.21.15.42:0/1857082052 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8f419e370 con 0x7fa8f41090c0 2024-08-30T09:12:37.285 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.282+0000 7fa8f1ffb640 1 -- 172.21.15.42:0/1857082052 <== mon.2 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7fa8e001a070 con 0x7fa8f41090c0 2024-08-30T09:12:37.285 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.283+0000 7fa8f1ffb640 1 --2- 172.21.15.42:0/1857082052 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fa8e40b4d40 0x7fa8e40b7250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:12:37.285 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.283+0000 7fa8f1ffb640 1 -- 172.21.15.42:0/1857082052 <== mon.2 v2:172.21.15.167:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7fa8e00e4ad0 con 0x7fa8f41090c0 2024-08-30T09:12:37.290 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.288+0000 7fa8f88be640 1 --2- 172.21.15.42:0/1857082052 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fa8e40b4d40 0x7fa8e40b7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:12:37.290 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.288+0000 7fa901097740 1 Processor -- start 2024-08-30T09:12:37.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.290+0000 7fa8f88be640 1 --2- 172.21.15.42:0/1857082052 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fa8e40b4d40 0x7fa8e40b7250 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fa8e8009bd0 tx=0x7fa8e8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:12:37.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.301+0000 7fa901097740 1 -- start start 2024-08-30T09:12:37.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.302+0000 7fa901097740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b9ee5de070 0x55b9ee9834b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:12:37.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.302+0000 7fa901097740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b9ee5b51b0 0x55b9ee9839f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:12:37.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.302+0000 7fa901097740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b9ee5bd390 0x55b9ee983f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:12:37.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.302+0000 7fa901097740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55b9ee8fa680 con 0x55b9ee5de070 2024-08-30T09:12:37.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.302+0000 7fa901097740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55b9ee8e9c10 con 0x55b9ee5bd390 2024-08-30T09:12:37.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.302+0000 7fa901097740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55b9ee8edbc0 con 0x55b9ee5b51b0 2024-08-30T09:12:37.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.303+0000 7fa8f88be640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b9ee5de070 0x55b9ee9834b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:12:37.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.303+0000 7fa8f88be640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b9ee5de070 0x55b9ee9834b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41514/0 (socket says 172.21.15.42:41514) 2024-08-30T09:12:37.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.303+0000 7fa8f88be640 1 -- 172.21.15.42:0/2119470712 learned_addr learned my addr 172.21.15.42:0/2119470712 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:12:37.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.304+0000 7fa8f88be640 1 -- 172.21.15.42:0/2119470712 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b9ee5b51b0 msgr2=0x55b9ee9839f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-30T09:12:37.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.304+0000 7fa8f88be640 1 --2- 172.21.15.42:0/2119470712 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b9ee5b51b0 0x55b9ee9839f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:37.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.304+0000 7fa8f88be640 1 -- 172.21.15.42:0/2119470712 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b9ee5bd390 msgr2=0x55b9ee983f30 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-30T09:12:37.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.304+0000 7fa8f88be640 1 --2- 172.21.15.42:0/2119470712 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b9ee5bd390 0x55b9ee983f30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:37.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.304+0000 7fa8f88be640 1 -- 172.21.15.42:0/2119470712 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b9ee649650 con 0x55b9ee5de070 2024-08-30T09:12:37.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.304+0000 7fa8f88be640 1 --2- 172.21.15.42:0/2119470712 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b9ee5de070 0x55b9ee9834b0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fa8e800e630 tx=0x7fa8e8014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bb231ab30ed7c23 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:12:37.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.308+0000 7fa8deffd640 1 -- 172.21.15.42:0/2119470712 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8e8004020 con 0x55b9ee5de070 2024-08-30T09:12:37.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.308+0000 7fa8deffd640 1 -- 172.21.15.42:0/2119470712 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa8e80041c0 con 0x55b9ee5de070 2024-08-30T09:12:37.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.308+0000 7fa8deffd640 1 -- 172.21.15.42:0/2119470712 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8e8026760 con 0x55b9ee5de070 2024-08-30T09:12:37.313 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.310+0000 7fa901097740 1 -- 172.21.15.42:0/2119470712 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b9ee5de070 msgr2=0x55b9ee9834b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:12:37.313 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.310+0000 7fa901097740 1 --2- 172.21.15.42:0/2119470712 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b9ee5de070 0x55b9ee9834b0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fa8e800e630 tx=0x7fa8e8014000 comp rx=0 tx=0).stop 2024-08-30T09:12:37.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.313+0000 7fa901097740 1 -- 172.21.15.42:0/2119470712 shutdown_connections 2024-08-30T09:12:37.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.313+0000 7fa901097740 1 --2- 172.21.15.42:0/2119470712 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b9ee5b51b0 0x55b9ee9839f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:37.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.313+0000 7fa901097740 1 --2- 172.21.15.42:0/2119470712 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b9ee5bd390 0x55b9ee983f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:37.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.313+0000 7fa901097740 1 --2- 172.21.15.42:0/2119470712 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b9ee5de070 0x55b9ee9834b0 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:37.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.313+0000 7fa901097740 1 -- 172.21.15.42:0/2119470712 >> 172.21.15.42:0/2119470712 conn(0x55b9ee5d2460 msgr2=0x55b9ee9813a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:12:37.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.319+0000 7fa901097740 1 -- 172.21.15.42:0/2119470712 shutdown_connections 2024-08-30T09:12:37.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.323+0000 7fa901097740 1 -- 172.21.15.42:0/2119470712 wait complete. 2024-08-30T09:12:37.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.323+0000 7fa901097740 1 -- start start 2024-08-30T09:12:37.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.323+0000 7fa901097740 1 Processor -- start 2024-08-30T09:12:37.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.324+0000 7fa901097740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b9ee5b51b0 0x55b9ee98ef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:12:37.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.324+0000 7fa901097740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b9ee5bd390 0x55b9ee98f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:12:37.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.324+0000 7fa901097740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b9ee5de070 0x55b9ee9abf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:12:37.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.324+0000 7fa901097740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55b9ee74c700 con 0x55b9ee5bd390 2024-08-30T09:12:37.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.324+0000 7fa901097740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55b9ee870750 con 0x55b9ee5de070 2024-08-30T09:12:37.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.324+0000 7fa901097740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55b9ee991c90 con 0x55b9ee5b51b0 2024-08-30T09:12:37.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.324+0000 7fa901097740 10 client.0 _ll_register_callbacks cb 0x55b9ee89fe10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:12:37.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.324+0000 7fa8f90bf640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b9ee5de070 0x55b9ee9abf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:12:37.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.324+0000 7fa8f90bf640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b9ee5de070 0x55b9ee9abf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:42612/0 (socket says 172.21.15.42:42612) 2024-08-30T09:12:37.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.324+0000 7fa8f90bf640 1 -- 172.21.15.42:0/3533087270 learned_addr learned my addr 172.21.15.42:0/3533087270 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:12:37.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.328+0000 7fa8f88be640 1 --2- 172.21.15.42:0/3533087270 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b9ee5b51b0 0x55b9ee98ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:12:37.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.331+0000 7fa8f90bf640 1 -- 172.21.15.42:0/3533087270 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b9ee5b51b0 msgr2=0x55b9ee98ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:12:37.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.331+0000 7fa8f90bf640 1 --2- 172.21.15.42:0/3533087270 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b9ee5b51b0 0x55b9ee98ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:37.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.331+0000 7fa8f90bf640 1 -- 172.21.15.42:0/3533087270 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b9ee5bd390 msgr2=0x55b9ee98f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:12:37.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.331+0000 7fa8f90bf640 1 --2- 172.21.15.42:0/3533087270 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b9ee5bd390 0x55b9ee98f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:37.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.331+0000 7fa8f90bf640 1 -- 172.21.15.42:0/3533087270 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b9ee8a9900 con 0x55b9ee5de070 2024-08-30T09:12:37.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.340+0000 7fa8f90bf640 1 --2- 172.21.15.42:0/3533087270 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b9ee5de070 0x55b9ee9abf90 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fa8ec004af0 tx=0x7fa8ec030800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c50e1f92304030d2 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:12:37.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.341+0000 7fa8d3fff640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-30T09:12:37.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.346+0000 7fa8d3fff640 1 -- 172.21.15.42:0/3533087270 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8ec003970 con 0x55b9ee5de070 2024-08-30T09:12:37.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.346+0000 7fa901097740 1 -- 172.21.15.42:0/3533087270 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa8e80041c0 con 0x55b9ee5de070 2024-08-30T09:12:37.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.346+0000 7fa8d3fff640 1 -- 172.21.15.42:0/3533087270 <== mon.2 v2:172.21.15.167:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7fa8ec003b10 con 0x55b9ee5de070 2024-08-30T09:12:37.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.350+0000 7fa8d3fff640 1 -- 172.21.15.42:0/3533087270 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fa8ec003d10 con 0x55b9ee5de070 2024-08-30T09:12:37.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.353+0000 7fa901097740 10 client.27218 fetch_fsmap learned FSMap version 43 2024-08-30T09:12:37.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.353+0000 7fa901097740 1 -- 172.21.15.42:0/3533087270 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55b9ee9aeba0 con 0x55b9ee5de070 2024-08-30T09:12:37.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.353+0000 7fa8d3fff640 1 -- 172.21.15.42:0/3533087270 <== mon.2 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa8e80041c0 con 0x55b9ee5de070 2024-08-30T09:12:37.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.353+0000 7fa8d3fff640 1 -- 172.21.15.42:0/3533087270 <== mon.2 v2:172.21.15.167:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7fa8ec031680 con 0x55b9ee5de070 2024-08-30T09:12:37.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.356+0000 7fa901097740 10 client.27218 fetch_fsmap finished waiting for FSMap version 43 2024-08-30T09:12:37.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.357+0000 7fa901097740 10 client.27218 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-08-30T09:12:37.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.357+0000 7fa901097740 20 client.27218 populate_metadata read hostname 'smithi042' 2024-08-30T09:12:37.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.357+0000 7fa901097740 1 --2- 172.21.15.42:0/3533087270 >> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] conn(0x55b9ee59c9f0 0x55b9ee9b2030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:12:37.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.357+0000 7fa901097740 4 client.27218 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-30T09:12:37.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.357+0000 7fa901097740 1 -- 172.21.15.42:0/3533087270 --> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b9ee8fc5d0 con 0x55b9ee59c9f0 2024-08-30T09:12:37.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.359+0000 7fa8f88be640 1 --2- 172.21.15.42:0/3533087270 >> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] conn(0x55b9ee59c9f0 0x55b9ee9b2030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:12:37.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.372+0000 7fa8f88be640 1 --2- 172.21.15.42:0/3533087270 >> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] conn(0x55b9ee59c9f0 0x55b9ee9b2030 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=32ac99a5f8851343 server_cookie=55dbc190c5948595 in_seq=0 out_seq=0 2024-08-30T09:12:37.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.372+0000 7fa8d3fff640 10 client.27218 ms_handle_connect on v2:172.21.15.190:6838/719981110 2024-08-30T09:12:37.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.373+0000 7fa8d3fff640 1 -- 172.21.15.42:0/3533087270 <== mds.1 v2:172.21.15.190:6838/719981110 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b9ee8fc5d0 con 0x55b9ee59c9f0 2024-08-30T09:12:37.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.373+0000 7fa8d3fff640 10 client.27218 handle_command_reply: tid=0 2024-08-30T09:12:37.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.373+0000 7fa901097740 1 client.27218 shutdown 2024-08-30T09:12:37.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.376+0000 7fa901097740 1 -- 172.21.15.42:0/3533087270 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b9ee5de070 msgr2=0x55b9ee9abf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:12:37.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.376+0000 7fa901097740 1 --2- 172.21.15.42:0/3533087270 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b9ee5de070 0x55b9ee9abf90 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fa8ec004af0 tx=0x7fa8ec030800 comp rx=0 tx=0).stop 2024-08-30T09:12:37.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.377+0000 7fa901097740 1 -- 172.21.15.42:0/3533087270 shutdown_connections 2024-08-30T09:12:37.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.377+0000 7fa901097740 1 --2- 172.21.15.42:0/3533087270 >> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] conn(0x55b9ee59c9f0 0x55b9ee9b2030 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:37.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.377+0000 7fa901097740 1 --2- 172.21.15.42:0/3533087270 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b9ee5de070 0x55b9ee9abf90 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:37.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.377+0000 7fa901097740 1 --2- 172.21.15.42:0/3533087270 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b9ee5bd390 0x55b9ee98f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:37.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.377+0000 7fa901097740 1 --2- 172.21.15.42:0/3533087270 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b9ee5b51b0 0x55b9ee98ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:37.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.377+0000 7fa901097740 1 -- 172.21.15.42:0/3533087270 >> 172.21.15.42:0/3533087270 conn(0x55b9ee5d2460 msgr2=0x55b9ee986690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:12:37.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.377+0000 7fa8d3fff640 0 client.27218 ms_handle_reset on v2:172.21.15.190:6838/719981110 2024-08-30T09:12:37.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.382+0000 7fa901097740 1 -- 172.21.15.42:0/3533087270 shutdown_connections 2024-08-30T09:12:37.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.384+0000 7fa901097740 1 -- 172.21.15.42:0/3533087270 wait complete. 2024-08-30T09:12:37.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.386+0000 7fa901097740 20 client.27218 trim_cache size 0 max 16384 2024-08-30T09:12:37.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.412+0000 7fa901097740 1 Processor -- start 2024-08-30T09:12:37.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.413+0000 7fa901097740 1 -- start start 2024-08-30T09:12:37.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.413+0000 7fa901097740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b9ee5b51b0 0x55b9ee9895d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:12:37.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.413+0000 7fa901097740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b9ee5bd390 0x55b9ee9814a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:12:37.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.413+0000 7fa901097740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b9ee5de070 0x55b9ee9abf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:12:37.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.413+0000 7fa901097740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fa8f41a87a0 con 0x55b9ee5de070 2024-08-30T09:12:37.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.413+0000 7fa901097740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa8f41a6070 con 0x55b9ee5b51b0 2024-08-30T09:12:37.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.413+0000 7fa901097740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fa8f4074800 con 0x55b9ee5bd390 2024-08-30T09:12:37.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.414+0000 7fa8f90bf640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b9ee5de070 0x55b9ee9abf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:12:37.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.414+0000 7fa8f90bf640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b9ee5de070 0x55b9ee9abf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41528/0 (socket says 172.21.15.42:41528) 2024-08-30T09:12:37.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.414+0000 7fa8f90bf640 1 -- 172.21.15.42:0/3178940901 learned_addr learned my addr 172.21.15.42:0/3178940901 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:12:37.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.414+0000 7fa8f90bf640 1 -- 172.21.15.42:0/3178940901 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b9ee5bd390 msgr2=0x55b9ee9814a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-30T09:12:37.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.414+0000 7fa8f90bf640 1 --2- 172.21.15.42:0/3178940901 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b9ee5bd390 0x55b9ee9814a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:37.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.414+0000 7fa8f90bf640 1 -- 172.21.15.42:0/3178940901 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b9ee5b51b0 msgr2=0x55b9ee9895d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-30T09:12:37.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.414+0000 7fa8f90bf640 1 --2- 172.21.15.42:0/3178940901 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b9ee5b51b0 0x55b9ee9895d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:37.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.414+0000 7fa8f90bf640 1 -- 172.21.15.42:0/3178940901 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b9ee991730 con 0x55b9ee5de070 2024-08-30T09:12:37.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.415+0000 7fa8f90bf640 1 --2- 172.21.15.42:0/3178940901 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b9ee5de070 0x55b9ee9abf10 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fa8ec04e4b0 tx=0x7fa8ec030fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f8beb2b97cbdd9 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:12:37.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.417+0000 7fa8de7fc640 1 -- 172.21.15.42:0/3178940901 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8ec030b00 con 0x55b9ee5de070 2024-08-30T09:12:37.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.417+0000 7fa8de7fc640 1 -- 172.21.15.42:0/3178940901 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa8ec053020 con 0x55b9ee5de070 2024-08-30T09:12:37.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.417+0000 7fa8de7fc640 1 -- 172.21.15.42:0/3178940901 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8ec04ea00 con 0x55b9ee5de070 2024-08-30T09:12:37.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.418+0000 7fa901097740 1 -- 172.21.15.42:0/3178940901 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b9ee5de070 msgr2=0x55b9ee9abf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:12:37.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.418+0000 7fa901097740 1 --2- 172.21.15.42:0/3178940901 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b9ee5de070 0x55b9ee9abf10 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fa8ec04e4b0 tx=0x7fa8ec030fd0 comp rx=0 tx=0).stop 2024-08-30T09:12:37.421 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.419+0000 7fa901097740 1 -- 172.21.15.42:0/3178940901 shutdown_connections 2024-08-30T09:12:37.421 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.419+0000 7fa901097740 1 --2- 172.21.15.42:0/3178940901 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b9ee5bd390 0x55b9ee9814a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:37.421 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.419+0000 7fa901097740 1 --2- 172.21.15.42:0/3178940901 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b9ee5de070 0x55b9ee9abf10 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:37.421 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.419+0000 7fa901097740 1 --2- 172.21.15.42:0/3178940901 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b9ee5b51b0 0x55b9ee9895d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:37.421 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.419+0000 7fa901097740 1 -- 172.21.15.42:0/3178940901 >> 172.21.15.42:0/3178940901 conn(0x55b9ee5d2460 msgr2=0x55b9ee980d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:12:37.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.424+0000 7fa901097740 1 -- 172.21.15.42:0/3178940901 shutdown_connections 2024-08-30T09:12:37.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.424+0000 7fa901097740 1 -- 172.21.15.42:0/3178940901 wait complete. 2024-08-30T09:12:37.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.425+0000 7fa901097740 1 -- start start 2024-08-30T09:12:37.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.425+0000 7fa901097740 1 Processor -- start 2024-08-30T09:12:37.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.426+0000 7fa901097740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b9ee5b51b0 0x55b9ee9a3f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:12:37.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.426+0000 7fa901097740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b9ee5bd390 0x55b9ee9a4440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:12:37.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.426+0000 7fa8f88be640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b9ee5b51b0 0x55b9ee9a3f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:12:37.430 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.426+0000 7fa8f88be640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b9ee5b51b0 0x55b9ee9a3f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:33212/0 (socket says 172.21.15.42:33212) 2024-08-30T09:12:37.430 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.427+0000 7fa8f88be640 1 -- 172.21.15.42:0/3065351835 learned_addr learned my addr 172.21.15.42:0/3065351835 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:12:37.430 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.427+0000 7fa901097740 1 --2- 172.21.15.42:0/3065351835 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b9ee5de070 0x55b9ee9a4980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:12:37.430 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.427+0000 7fa901097740 1 -- 172.21.15.42:0/3065351835 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fa8f4074e70 con 0x55b9ee5bd390 2024-08-30T09:12:37.430 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.427+0000 7fa8f90bf640 1 --2- 172.21.15.42:0/3065351835 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b9ee5de070 0x55b9ee9a4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:12:37.430 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.427+0000 7fa901097740 1 -- 172.21.15.42:0/3065351835 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa8f41a59a0 con 0x55b9ee5de070 2024-08-30T09:12:37.430 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.427+0000 7fa8f3fff640 1 --2- 172.21.15.42:0/3065351835 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b9ee5bd390 0x55b9ee9a4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:12:37.430 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.428+0000 7fa901097740 1 -- 172.21.15.42:0/3065351835 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fa8f41a4c00 con 0x55b9ee5b51b0 2024-08-30T09:12:37.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.428+0000 7fa901097740 10 client.0 _ll_register_callbacks cb 0x7fa8e8005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:12:37.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.428+0000 7fa8f88be640 1 -- 172.21.15.42:0/3065351835 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b9ee5de070 msgr2=0x55b9ee9a4980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:12:37.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.428+0000 7fa8f88be640 1 --2- 172.21.15.42:0/3065351835 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b9ee5de070 0x55b9ee9a4980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:37.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.428+0000 7fa8f88be640 1 -- 172.21.15.42:0/3065351835 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b9ee5bd390 msgr2=0x55b9ee9a4440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:12:37.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.428+0000 7fa8f88be640 1 --2- 172.21.15.42:0/3065351835 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b9ee5bd390 0x55b9ee9a4440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:37.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.428+0000 7fa8f88be640 1 -- 172.21.15.42:0/3065351835 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b9eea1ba30 con 0x55b9ee5b51b0 2024-08-30T09:12:37.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.430+0000 7fa8f88be640 1 --2- 172.21.15.42:0/3065351835 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b9ee5b51b0 0x55b9ee9a3f00 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fa8e800e5d0 tx=0x7fa8e8005720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5c8856c203cfbf97 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:12:37.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.430+0000 7fa8d3fff640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-08-30T09:12:37.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.430+0000 7fa8d3fff640 1 -- 172.21.15.42:0/3065351835 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8e8004040 con 0x55b9ee5b51b0 2024-08-30T09:12:37.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.430+0000 7fa901097740 1 -- 172.21.15.42:0/3065351835 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa8ec053020 con 0x55b9ee5b51b0 2024-08-30T09:12:37.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.430+0000 7fa8d3fff640 1 -- 172.21.15.42:0/3065351835 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7fa8e80041e0 con 0x55b9ee5b51b0 2024-08-30T09:12:37.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.431+0000 7fa8d3fff640 1 -- 172.21.15.42:0/3065351835 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fa8e8042050 con 0x55b9ee5b51b0 2024-08-30T09:12:37.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.431+0000 7fa901097740 10 client.27187 fetch_fsmap learned FSMap version 43 2024-08-30T09:12:37.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.431+0000 7fa901097740 1 -- 172.21.15.42:0/3065351835 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({fsmap=43}) -- 0x55b9eea1cc00 con 0x55b9ee5b51b0 2024-08-30T09:12:37.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.432+0000 7fa8d3fff640 1 -- 172.21.15.42:0/3065351835 <== mon.1 v2:172.21.15.190:3300/0 4 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7fa8e8026480 con 0x55b9ee5b51b0 2024-08-30T09:12:37.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.432+0000 7fa901097740 10 client.27187 fetch_fsmap finished waiting for FSMap version 43 2024-08-30T09:12:37.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.432+0000 7fa901097740 10 client.27187 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-08-30T09:12:37.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.432+0000 7fa901097740 20 client.27187 populate_metadata read hostname 'smithi042' 2024-08-30T09:12:37.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.432+0000 7fa901097740 1 --2- 172.21.15.42:0/3065351835 >> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] conn(0x55b9ee571da0 0x55b9eea1ee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:12:37.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.432+0000 7fa901097740 4 client.27187 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-30T09:12:37.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.432+0000 7fa901097740 1 -- 172.21.15.42:0/3065351835 --> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fa8ec053020 con 0x55b9ee571da0 2024-08-30T09:12:37.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.433+0000 7fa8f3fff640 1 --2- 172.21.15.42:0/3065351835 >> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] conn(0x55b9ee571da0 0x55b9eea1ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:12:37.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.433+0000 7fa8f3fff640 1 --2- 172.21.15.42:0/3065351835 >> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] conn(0x55b9ee571da0 0x55b9eea1ee90 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8f3af3a511ba3b28 server_cookie=ee4adee231b5d9e4 in_seq=0 out_seq=0 2024-08-30T09:12:37.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.433+0000 7fa8d3fff640 10 client.27187 ms_handle_connect on v2:172.21.15.190:6838/719981110 2024-08-30T09:12:37.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.433+0000 7fa8d3fff640 1 -- 172.21.15.42:0/3065351835 <== mds.1 v2:172.21.15.190:6838/719981110 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fa8ec053020 con 0x55b9ee571da0 2024-08-30T09:12:37.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.433+0000 7fa8d3fff640 10 client.27187 handle_command_reply: tid=0 2024-08-30T09:12:37.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.433+0000 7fa901097740 1 client.27187 shutdown 2024-08-30T09:12:37.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.434+0000 7fa901097740 1 -- 172.21.15.42:0/3065351835 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b9ee5b51b0 msgr2=0x55b9ee9a3f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:12:37.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.434+0000 7fa901097740 1 --2- 172.21.15.42:0/3065351835 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b9ee5b51b0 0x55b9ee9a3f00 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fa8e800e5d0 tx=0x7fa8e8005720 comp rx=0 tx=0).stop 2024-08-30T09:12:37.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.434+0000 7fa901097740 1 -- 172.21.15.42:0/3065351835 shutdown_connections 2024-08-30T09:12:37.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.434+0000 7fa901097740 1 --2- 172.21.15.42:0/3065351835 >> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] conn(0x55b9ee571da0 0x55b9eea1ee90 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:37.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.434+0000 7fa8d3fff640 0 client.27187 ms_handle_reset on v2:172.21.15.190:6838/719981110 2024-08-30T09:12:37.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.434+0000 7fa901097740 1 --2- 172.21.15.42:0/3065351835 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b9ee5de070 0x55b9ee9a4980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:37.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.434+0000 7fa901097740 1 --2- 172.21.15.42:0/3065351835 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b9ee5bd390 0x55b9ee9a4440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:37.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.434+0000 7fa901097740 1 --2- 172.21.15.42:0/3065351835 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b9ee5b51b0 0x55b9ee9a3f00 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:37.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.434+0000 7fa901097740 1 -- 172.21.15.42:0/3065351835 >> 172.21.15.42:0/3065351835 conn(0x55b9ee5d2460 msgr2=0x55b9ee9844e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:12:37.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.434+0000 7fa901097740 1 -- 172.21.15.42:0/3065351835 shutdown_connections 2024-08-30T09:12:37.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.435+0000 7fa901097740 1 -- 172.21.15.42:0/3065351835 wait complete. 2024-08-30T09:12:37.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.435+0000 7fa901097740 20 client.27187 trim_cache size 0 max 16384 2024-08-30T09:12:37.438 INFO:teuthology.orchestra.run.smithi042.stdout:[] 2024-08-30T09:12:37.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.436+0000 7fa8f98c0640 1 -- 172.21.15.42:0/1857082052 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fa8e40b4d40 msgr2=0x7fa8e40b7250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:12:37.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.436+0000 7fa8f98c0640 1 --2- 172.21.15.42:0/1857082052 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fa8e40b4d40 0x7fa8e40b7250 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fa8e8009bd0 tx=0x7fa8e8005f90 comp rx=0 tx=0).stop 2024-08-30T09:12:37.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.436+0000 7fa8f98c0640 1 -- 172.21.15.42:0/1857082052 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa8f41090c0 msgr2=0x7fa8f4198280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:12:37.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.436+0000 7fa8f98c0640 1 --2- 172.21.15.42:0/1857082052 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa8f41090c0 0x7fa8f4198280 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fa8e000d740 tx=0x7fa8e000dc00 comp rx=0 tx=0).stop 2024-08-30T09:12:37.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.436+0000 7fa8f98c0640 1 -- 172.21.15.42:0/1857082052 shutdown_connections 2024-08-30T09:12:37.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.436+0000 7fa8f98c0640 1 --2- 172.21.15.42:0/1857082052 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fa8e40b4d40 0x7fa8e40b7250 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:37.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.436+0000 7fa8f98c0640 1 --2- 172.21.15.42:0/1857082052 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa8f4109a10 0x7fa8f419d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:37.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.436+0000 7fa8f98c0640 1 --2- 172.21.15.42:0/1857082052 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa8f41090c0 0x7fa8f4198280 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:37.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.436+0000 7fa8f98c0640 1 --2- 172.21.15.42:0/1857082052 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa8f4103780 0x7fa8f4197ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:37.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.436+0000 7fa8f98c0640 1 -- 172.21.15.42:0/1857082052 >> 172.21.15.42:0/1857082052 conn(0x7fa8f40ff1d0 msgr2=0x7fa8f41015f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:12:37.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.436+0000 7fa8f98c0640 1 -- 172.21.15.42:0/1857082052 shutdown_connections 2024-08-30T09:12:37.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:37.437+0000 7fa8f98c0640 1 -- 172.21.15.42:0/1857082052 wait complete. 2024-08-30T09:12:38.884 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-08-30T09:12:39.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:12:38 smithi167 ceph-mon[22063]: pgmap v729: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 4.9 MiB/s wr, 431 op/s 2024-08-30T09:12:39.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:12:38 smithi042 ceph-mon[18652]: pgmap v729: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 4.9 MiB/s wr, 431 op/s 2024-08-30T09:12:39.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:12:38 smithi190 ceph-mon[22001]: pgmap v729: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 4.9 MiB/s wr, 431 op/s 2024-08-30T09:12:39.430 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T09:12:40.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.079+0000 7ff50fe38640 1 -- 172.21.15.42:0/2694005663 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff50810a780 msgr2=0x7ff50810ab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:12:40.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.079+0000 7ff50fe38640 1 --2- 172.21.15.42:0/2694005663 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff50810a780 0x7ff50810ab90 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7ff500009ca0 tx=0x7ff50002a8f0 comp rx=0 tx=0).stop 2024-08-30T09:12:40.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.081+0000 7ff50fe38640 1 -- 172.21.15.42:0/2694005663 shutdown_connections 2024-08-30T09:12:40.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.081+0000 7ff50fe38640 1 --2- 172.21.15.42:0/2694005663 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff50811c9d0 0x7ff50811eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:40.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.081+0000 7ff50fe38640 1 --2- 172.21.15.42:0/2694005663 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff50810a780 0x7ff50810ab90 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:40.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.081+0000 7ff50fe38640 1 --2- 172.21.15.42:0/2694005663 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff508111ce0 0x7ff50810a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:40.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.081+0000 7ff50fe38640 1 -- 172.21.15.42:0/2694005663 >> 172.21.15.42:0/2694005663 conn(0x7ff508070c60 msgr2=0x7ff508071070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:12:40.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.085+0000 7ff50fe38640 1 -- 172.21.15.42:0/2694005663 shutdown_connections 2024-08-30T09:12:40.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.085+0000 7ff50fe38640 1 -- 172.21.15.42:0/2694005663 wait complete. 2024-08-30T09:12:40.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.086+0000 7ff50fe38640 1 Processor -- start 2024-08-30T09:12:40.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.094+0000 7ff50fe38640 1 -- start start 2024-08-30T09:12:40.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.094+0000 7ff50fe38640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff50810a780 0x7ff5081a8e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:12:40.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.094+0000 7ff50fe38640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff508111ce0 0x7ff5081a9410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:12:40.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.094+0000 7ff50fe38640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff50811c9d0 0x7ff5081ae6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:12:40.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.094+0000 7ff50fe38640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7ff508122660 con 0x7ff50810a780 2024-08-30T09:12:40.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.094+0000 7ff50fe38640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff5081224e0 con 0x7ff50811c9d0 2024-08-30T09:12:40.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.094+0000 7ff50fe38640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7ff5081227e0 con 0x7ff508111ce0 2024-08-30T09:12:40.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.096+0000 7ff50d3d3640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff508111ce0 0x7ff5081a9410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:12:40.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.097+0000 7ff50e3d5640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff50811c9d0 0x7ff5081ae6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:12:40.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.098+0000 7ff50e3d5640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff50811c9d0 0x7ff5081ae6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:35688/0 (socket says 172.21.15.42:35688) 2024-08-30T09:12:40.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.098+0000 7ff50e3d5640 1 -- 172.21.15.42:0/1545207124 learned_addr learned my addr 172.21.15.42:0/1545207124 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:12:40.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.100+0000 7ff50dbd4640 1 --2- 172.21.15.42:0/1545207124 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff50810a780 0x7ff5081a8e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:12:40.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.102+0000 7ff50d3d3640 1 -- 172.21.15.42:0/1545207124 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff50811c9d0 msgr2=0x7ff5081ae6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:12:40.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.102+0000 7ff50d3d3640 1 --2- 172.21.15.42:0/1545207124 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff50811c9d0 0x7ff5081ae6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:40.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.103+0000 7ff50d3d3640 1 -- 172.21.15.42:0/1545207124 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff50810a780 msgr2=0x7ff5081a8e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:12:40.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.103+0000 7ff50d3d3640 1 --2- 172.21.15.42:0/1545207124 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff50810a780 0x7ff5081a8e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:40.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.103+0000 7ff50d3d3640 1 -- 172.21.15.42:0/1545207124 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5081aedb0 con 0x7ff508111ce0 2024-08-30T09:12:40.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.107+0000 7ff50d3d3640 1 --2- 172.21.15.42:0/1545207124 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff508111ce0 0x7ff5081a9410 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7ff50806c710 tx=0x7ff500030820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:12:40.114 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.112+0000 7ff4feffd640 1 -- 172.21.15.42:0/1545207124 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff500030e20 con 0x7ff508111ce0 2024-08-30T09:12:40.114 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.112+0000 7ff50fe38640 1 -- 172.21.15.42:0/1545207124 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5081af040 con 0x7ff508111ce0 2024-08-30T09:12:40.114 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.112+0000 7ff50fe38640 1 -- 172.21.15.42:0/1545207124 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff508112120 con 0x7ff508111ce0 2024-08-30T09:12:40.114 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.112+0000 7ff4feffd640 1 -- 172.21.15.42:0/1545207124 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff50003f070 con 0x7ff508111ce0 2024-08-30T09:12:40.114 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.112+0000 7ff4feffd640 1 -- 172.21.15.42:0/1545207124 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff50003b6f0 con 0x7ff508111ce0 2024-08-30T09:12:40.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.117+0000 7ff5163ad740 1 Processor -- start 2024-08-30T09:12:40.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.119+0000 7ff5163ad740 1 -- start start 2024-08-30T09:12:40.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.120+0000 7ff4feffd640 1 -- 172.21.15.42:0/1545207124 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7ff50003b890 con 0x7ff508111ce0 2024-08-30T09:12:40.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.120+0000 7ff5163ad740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564b3333f070 0x564b336e4350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:12:40.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.120+0000 7ff5163ad740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564b333161b0 0x564b336e4890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:12:40.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.120+0000 7ff5163ad740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564b3331e390 0x564b336e4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:12:40.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.120+0000 7ff50e3d5640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564b3333f070 0x564b336e4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:12:40.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.120+0000 7ff50e3d5640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564b3333f070 0x564b336e4350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:35700/0 (socket says 172.21.15.42:35700) 2024-08-30T09:12:40.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.120+0000 7ff5163ad740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x564b3365b680 con 0x564b333161b0 2024-08-30T09:12:40.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.120+0000 7ff5163ad740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x564b3364ac10 con 0x564b3333f070 2024-08-30T09:12:40.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.120+0000 7ff5163ad740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x564b3364ebc0 con 0x564b3331e390 2024-08-30T09:12:40.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.120+0000 7ff50e3d5640 1 -- 172.21.15.42:0/308641981 learned_addr learned my addr 172.21.15.42:0/308641981 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:12:40.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.121+0000 7ff50d3d3640 1 --2- 172.21.15.42:0/308641981 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564b3331e390 0x564b336e4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:12:40.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.121+0000 7ff50dbd4640 1 --2- 172.21.15.42:0/308641981 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564b333161b0 0x564b336e4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:12:40.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.121+0000 7ff4feffd640 1 --2- 172.21.15.42:0/1545207124 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7ff4f40b4c70 0x7ff4f40b7180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:12:40.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.122+0000 7ff4feffd640 1 -- 172.21.15.42:0/1545207124 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7ff500106cd0 con 0x7ff508111ce0 2024-08-30T09:12:40.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.122+0000 7ff50d3d3640 1 -- 172.21.15.42:0/308641981 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564b3333f070 msgr2=0x564b336e4350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:12:40.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.122+0000 7ff50d3d3640 1 --2- 172.21.15.42:0/308641981 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564b3333f070 0x564b336e4350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:40.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.122+0000 7ff50d3d3640 1 -- 172.21.15.42:0/308641981 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564b333161b0 msgr2=0x564b336e4890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:12:40.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.122+0000 7ff50d3d3640 1 --2- 172.21.15.42:0/308641981 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564b333161b0 0x564b336e4890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:40.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.122+0000 7ff50d3d3640 1 -- 172.21.15.42:0/308641981 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564b333aa650 con 0x564b3331e390 2024-08-30T09:12:40.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.122+0000 7ff50d3d3640 1 --2- 172.21.15.42:0/308641981 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564b3331e390 0x564b336e4dd0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7ff5001043b0 tx=0x7ff500005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=642a8a36b907863e server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:12:40.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.125+0000 7ff4f3fff640 1 -- 172.21.15.42:0/308641981 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5000ccc60 con 0x564b3331e390 2024-08-30T09:12:40.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.125+0000 7ff4f3fff640 1 -- 172.21.15.42:0/308641981 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff500036030 con 0x564b3331e390 2024-08-30T09:12:40.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.125+0000 7ff4f3fff640 1 -- 172.21.15.42:0/308641981 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5000cca80 con 0x564b3331e390 2024-08-30T09:12:40.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.127+0000 7ff5163ad740 1 -- 172.21.15.42:0/308641981 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564b3331e390 msgr2=0x564b336e4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:12:40.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.127+0000 7ff5163ad740 1 --2- 172.21.15.42:0/308641981 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564b3331e390 0x564b336e4dd0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7ff5001043b0 tx=0x7ff500005fd0 comp rx=0 tx=0).stop 2024-08-30T09:12:40.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.129+0000 7ff50e3d5640 1 --2- 172.21.15.42:0/1545207124 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7ff4f40b4c70 0x7ff4f40b7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:12:40.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.129+0000 7ff5163ad740 1 -- 172.21.15.42:0/308641981 shutdown_connections 2024-08-30T09:12:40.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.129+0000 7ff5163ad740 1 --2- 172.21.15.42:0/308641981 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564b3331e390 0x564b336e4dd0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:40.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.129+0000 7ff5163ad740 1 --2- 172.21.15.42:0/308641981 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564b333161b0 0x564b336e4890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:40.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.129+0000 7ff5163ad740 1 --2- 172.21.15.42:0/308641981 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564b3333f070 0x564b336e4350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:40.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.129+0000 7ff5163ad740 1 -- 172.21.15.42:0/308641981 >> 172.21.15.42:0/308641981 conn(0x564b33333460 msgr2=0x564b336e2240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:12:40.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.132+0000 7ff5163ad740 1 -- 172.21.15.42:0/308641981 shutdown_connections 2024-08-30T09:12:40.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.134+0000 7ff5163ad740 1 -- 172.21.15.42:0/308641981 wait complete. 2024-08-30T09:12:40.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.134+0000 7ff5163ad740 1 -- start start 2024-08-30T09:12:40.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.134+0000 7ff5163ad740 1 Processor -- start 2024-08-30T09:12:40.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.136+0000 7ff50e3d5640 1 --2- 172.21.15.42:0/1545207124 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7ff4f40b4c70 0x7ff4f40b7180 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7ff50400bd80 tx=0x7ff5040046c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:12:40.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.137+0000 7ff5163ad740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564b333161b0 0x564b336efac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:12:40.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.138+0000 7ff5163ad740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564b3331e390 0x564b336f0000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:12:40.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.138+0000 7ff5163ad740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564b3333f070 0x564b336fbb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:12:40.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.138+0000 7ff50d3d3640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564b3331e390 0x564b336f0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:12:40.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.138+0000 7ff50d3d3640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564b3331e390 0x564b336f0000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:35714/0 (socket says 172.21.15.42:35714) 2024-08-30T09:12:40.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.138+0000 7ff50d3d3640 1 -- 172.21.15.42:0/2115717395 learned_addr learned my addr 172.21.15.42:0/2115717395 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:12:40.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.138+0000 7ff5163ad740 1 -- 172.21.15.42:0/2115717395 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x564b334ad700 con 0x564b333161b0 2024-08-30T09:12:40.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.138+0000 7ff5163ad740 1 -- 172.21.15.42:0/2115717395 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x564b335d1750 con 0x564b3331e390 2024-08-30T09:12:40.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.138+0000 7ff5163ad740 1 -- 172.21.15.42:0/2115717395 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x564b336f28f0 con 0x564b3333f070 2024-08-30T09:12:40.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.139+0000 7ff5163ad740 10 client.0 _ll_register_callbacks cb 0x564b33600e10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:12:40.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.139+0000 7ff50d3d3640 1 -- 172.21.15.42:0/2115717395 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564b3333f070 msgr2=0x564b336fbb50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-30T09:12:40.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.139+0000 7ff50d3d3640 1 --2- 172.21.15.42:0/2115717395 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564b3333f070 0x564b336fbb50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:40.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.139+0000 7ff50d3d3640 1 -- 172.21.15.42:0/2115717395 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564b333161b0 msgr2=0x564b336efac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:12:40.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.139+0000 7ff50d3d3640 1 --2- 172.21.15.42:0/2115717395 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564b333161b0 0x564b336efac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:40.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.139+0000 7ff50d3d3640 1 -- 172.21.15.42:0/2115717395 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564b3360a900 con 0x564b3331e390 2024-08-30T09:12:40.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.139+0000 7ff50d3d3640 1 --2- 172.21.15.42:0/2115717395 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564b3331e390 0x564b336f0000 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7ff500104380 tx=0x7ff500004d70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4a3531f5abf2642e server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:12:40.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.139+0000 7ff5163ad740 1 -- 172.21.15.42:0/2115717395 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff500036030 con 0x564b3331e390 2024-08-30T09:12:40.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.140+0000 7ff4f17fa640 10 client.27233 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-30T09:12:40.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.140+0000 7ff4f17fa640 1 -- 172.21.15.42:0/2115717395 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff50010a070 con 0x564b3331e390 2024-08-30T09:12:40.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.140+0000 7ff4f17fa640 1 -- 172.21.15.42:0/2115717395 <== mon.2 v2:172.21.15.167:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7ff5000cc780 con 0x564b3331e390 2024-08-30T09:12:40.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.140+0000 7ff4f17fa640 1 -- 172.21.15.42:0/2115717395 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7ff5000cca80 con 0x564b3331e390 2024-08-30T09:12:40.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.140+0000 7ff5163ad740 10 client.27233 fetch_fsmap learned FSMap version 43 2024-08-30T09:12:40.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.140+0000 7ff5163ad740 1 -- 172.21.15.42:0/2115717395 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({fsmap=43}) -- 0x564b336fe4e0 con 0x564b3331e390 2024-08-30T09:12:40.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.147+0000 7ff4f17fa640 1 -- 172.21.15.42:0/2115717395 <== mon.2 v2:172.21.15.167:3300/0 4 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7ff50004c070 con 0x564b3331e390 2024-08-30T09:12:40.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.147+0000 7ff5163ad740 10 client.27233 fetch_fsmap finished waiting for FSMap version 43 2024-08-30T09:12:40.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.147+0000 7ff5163ad740 10 client.27233 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-30T09:12:40.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.147+0000 7ff5163ad740 20 client.27233 populate_metadata read hostname 'smithi042' 2024-08-30T09:12:40.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.147+0000 7ff5163ad740 1 --2- 172.21.15.42:0/2115717395 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x564b332fd9f0 0x564b33701970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:12:40.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.147+0000 7ff5163ad740 4 client.27233 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-30T09:12:40.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.147+0000 7ff5163ad740 1 -- 172.21.15.42:0/2115717395 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7ff500036030 con 0x564b332fd9f0 2024-08-30T09:12:40.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.147+0000 7ff50dbd4640 1 --2- 172.21.15.42:0/2115717395 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x564b332fd9f0 0x564b33701970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:12:40.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.148+0000 7ff50dbd4640 1 --2- 172.21.15.42:0/2115717395 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x564b332fd9f0 0x564b33701970 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=78c8d304ae0a5296 server_cookie=6c21ec049d96c15 in_seq=0 out_seq=0 2024-08-30T09:12:40.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.148+0000 7ff4f17fa640 10 client.27233 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:12:40.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.149+0000 7ff4f17fa640 1 -- 172.21.15.42:0/2115717395 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7ff500036030 con 0x564b332fd9f0 2024-08-30T09:12:40.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.149+0000 7ff4f17fa640 10 client.27233 handle_command_reply: tid=0 2024-08-30T09:12:40.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.149+0000 7ff5163ad740 1 client.27233 shutdown 2024-08-30T09:12:40.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.154+0000 7ff5163ad740 1 -- 172.21.15.42:0/2115717395 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564b3331e390 msgr2=0x564b336f0000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:12:40.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.154+0000 7ff5163ad740 1 --2- 172.21.15.42:0/2115717395 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564b3331e390 0x564b336f0000 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7ff500104380 tx=0x7ff500004d70 comp rx=0 tx=0).stop 2024-08-30T09:12:40.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.156+0000 7ff5163ad740 1 -- 172.21.15.42:0/2115717395 shutdown_connections 2024-08-30T09:12:40.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.156+0000 7ff5163ad740 1 --2- 172.21.15.42:0/2115717395 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x564b332fd9f0 0x564b33701970 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:40.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.156+0000 7ff5163ad740 1 --2- 172.21.15.42:0/2115717395 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564b3333f070 0x564b336fbb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:40.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.156+0000 7ff5163ad740 1 --2- 172.21.15.42:0/2115717395 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564b3331e390 0x564b336f0000 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:40.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.156+0000 7ff5163ad740 1 --2- 172.21.15.42:0/2115717395 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564b333161b0 0x564b336efac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:40.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.156+0000 7ff5163ad740 1 -- 172.21.15.42:0/2115717395 >> 172.21.15.42:0/2115717395 conn(0x564b33333460 msgr2=0x564b336e74e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:12:40.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.157+0000 7ff4f17fa640 0 client.27233 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:12:40.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.157+0000 7ff5163ad740 1 -- 172.21.15.42:0/2115717395 shutdown_connections 2024-08-30T09:12:40.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.157+0000 7ff5163ad740 1 -- 172.21.15.42:0/2115717395 wait complete. 2024-08-30T09:12:40.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.159+0000 7ff5163ad740 20 client.27233 trim_cache size 0 max 16384 2024-08-30T09:12:40.195 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.192+0000 7ff5163ad740 1 Processor -- start 2024-08-30T09:12:40.195 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.193+0000 7ff5163ad740 1 -- start start 2024-08-30T09:12:40.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.193+0000 7ff5163ad740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564b333161b0 0x564b336e5a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:12:40.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.193+0000 7ff5163ad740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564b3331e390 0x564b336eccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:12:40.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.194+0000 7ff5163ad740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564b3333f070 0x564b336ed1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:12:40.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.194+0000 7ff5163ad740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7ff4ec012540 con 0x564b333161b0 2024-08-30T09:12:40.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.194+0000 7ff5163ad740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff4ec00fe10 con 0x564b3333f070 2024-08-30T09:12:40.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.194+0000 7ff50e3d5640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564b3333f070 0x564b336ed1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:12:40.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.194+0000 7ff50e3d5640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564b3333f070 0x564b336ed1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:35730/0 (socket says 172.21.15.42:35730) 2024-08-30T09:12:40.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.194+0000 7ff50e3d5640 1 -- 172.21.15.42:0/1263861891 learned_addr learned my addr 172.21.15.42:0/1263861891 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:12:40.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.194+0000 7ff5163ad740 1 -- 172.21.15.42:0/1263861891 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7ff4ec00dc60 con 0x564b3331e390 2024-08-30T09:12:40.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.195+0000 7ff50dbd4640 1 --2- 172.21.15.42:0/1263861891 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564b333161b0 0x564b336e5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:12:40.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.195+0000 7ff50e3d5640 1 -- 172.21.15.42:0/1263861891 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564b3331e390 msgr2=0x564b336eccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:12:40.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.195+0000 7ff50e3d5640 1 --2- 172.21.15.42:0/1263861891 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564b3331e390 0x564b336eccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:40.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.195+0000 7ff50e3d5640 1 -- 172.21.15.42:0/1263861891 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564b333161b0 msgr2=0x564b336e5a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:12:40.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.195+0000 7ff50e3d5640 1 --2- 172.21.15.42:0/1263861891 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564b333161b0 0x564b336e5a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:40.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.195+0000 7ff50e3d5640 1 -- 172.21.15.42:0/1263861891 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564b336f2200 con 0x564b3333f070 2024-08-30T09:12:40.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.196+0000 7ff50e3d5640 1 --2- 172.21.15.42:0/1263861891 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564b3333f070 0x564b336ed1f0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7ff504010740 tx=0x7ff504015000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d7a8505477c5e3a0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:12:40.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.196+0000 7ff4f37fe640 1 -- 172.21.15.42:0/1263861891 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff50401e6e0 con 0x564b3333f070 2024-08-30T09:12:40.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.196+0000 7ff4f37fe640 1 -- 172.21.15.42:0/1263861891 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff504022030 con 0x564b3333f070 2024-08-30T09:12:40.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.197+0000 7ff5163ad740 1 -- 172.21.15.42:0/1263861891 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564b3333f070 msgr2=0x564b336ed1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:12:40.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.197+0000 7ff5163ad740 1 --2- 172.21.15.42:0/1263861891 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564b3333f070 0x564b336ed1f0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7ff504010740 tx=0x7ff504015000 comp rx=0 tx=0).stop 2024-08-30T09:12:40.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.202+0000 7ff5163ad740 1 -- 172.21.15.42:0/1263861891 shutdown_connections 2024-08-30T09:12:40.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.202+0000 7ff5163ad740 1 --2- 172.21.15.42:0/1263861891 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564b3331e390 0x564b336eccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:40.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.202+0000 7ff5163ad740 1 --2- 172.21.15.42:0/1263861891 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564b3333f070 0x564b336ed1f0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:40.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.202+0000 7ff5163ad740 1 --2- 172.21.15.42:0/1263861891 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564b333161b0 0x564b336e5a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:40.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.202+0000 7ff5163ad740 1 -- 172.21.15.42:0/1263861891 >> 172.21.15.42:0/1263861891 conn(0x564b33333460 msgr2=0x564b336e5380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:12:40.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.207+0000 7ff5163ad740 1 -- 172.21.15.42:0/1263861891 shutdown_connections 2024-08-30T09:12:40.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.207+0000 7ff5163ad740 1 -- 172.21.15.42:0/1263861891 wait complete. 2024-08-30T09:12:40.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.207+0000 7ff5163ad740 1 -- start start 2024-08-30T09:12:40.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.207+0000 7ff5163ad740 1 Processor -- start 2024-08-30T09:12:40.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.208+0000 7ff5163ad740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564b333161b0 0x564b336f5c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:12:40.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.208+0000 7ff5163ad740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564b3331e390 0x564b336f61c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:12:40.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.208+0000 7ff5163ad740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564b3333f070 0x564b3376b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:12:40.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.208+0000 7ff5163ad740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7ff4ec00e2d0 con 0x564b3333f070 2024-08-30T09:12:40.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.208+0000 7ff5163ad740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff4ec00f740 con 0x564b3331e390 2024-08-30T09:12:40.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.208+0000 7ff5163ad740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7ff4ec00e9a0 con 0x564b333161b0 2024-08-30T09:12:40.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.208+0000 7ff5163ad740 10 client.0 _ll_register_callbacks cb 0x7ff500009900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:12:40.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.211+0000 7ff50e3d5640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564b3333f070 0x564b3376b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:12:40.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.211+0000 7ff50e3d5640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564b3333f070 0x564b3376b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37596/0 (socket says 172.21.15.42:37596) 2024-08-30T09:12:40.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.211+0000 7ff50e3d5640 1 -- 172.21.15.42:0/2201168881 learned_addr learned my addr 172.21.15.42:0/2201168881 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:12:40.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.211+0000 7ff50dbd4640 1 --2- 172.21.15.42:0/2201168881 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564b333161b0 0x564b336f5c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:12:40.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.211+0000 7ff50d3d3640 1 --2- 172.21.15.42:0/2201168881 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564b3331e390 0x564b336f61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:12:40.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.212+0000 7ff50d3d3640 1 -- 172.21.15.42:0/2201168881 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564b333161b0 msgr2=0x564b336f5c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:12:40.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.212+0000 7ff50d3d3640 1 --2- 172.21.15.42:0/2201168881 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564b333161b0 0x564b336f5c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:40.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.212+0000 7ff50d3d3640 1 -- 172.21.15.42:0/2201168881 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564b3333f070 msgr2=0x564b3376b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:12:40.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.212+0000 7ff50d3d3640 1 --2- 172.21.15.42:0/2201168881 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564b3333f070 0x564b3376b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:40.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.212+0000 7ff50d3d3640 1 -- 172.21.15.42:0/2201168881 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564b336f69b0 con 0x564b3331e390 2024-08-30T09:12:40.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.220+0000 7ff50d3d3640 1 --2- 172.21.15.42:0/2201168881 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564b3331e390 0x564b336f61c0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7ff5000cd010 tx=0x7ff500005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d42868988137afb1 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:12:40.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.220+0000 7ff4f17fa640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-30T09:12:40.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.220+0000 7ff4f17fa640 1 -- 172.21.15.42:0/2201168881 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff50010a040 con 0x564b3331e390 2024-08-30T09:12:40.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.220+0000 7ff5163ad740 1 -- 172.21.15.42:0/2201168881 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff504022030 con 0x564b3331e390 2024-08-30T09:12:40.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.220+0000 7ff4f17fa640 1 -- 172.21.15.42:0/2201168881 <== mon.2 v2:172.21.15.167:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7ff50010a3b0 con 0x564b3331e390 2024-08-30T09:12:40.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.221+0000 7ff4f17fa640 1 -- 172.21.15.42:0/2201168881 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7ff5000ccc60 con 0x564b3331e390 2024-08-30T09:12:40.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.221+0000 7ff5163ad740 10 client.27239 fetch_fsmap learned FSMap version 43 2024-08-30T09:12:40.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.221+0000 7ff5163ad740 1 -- 172.21.15.42:0/2201168881 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x564b336f69b0 con 0x564b3331e390 2024-08-30T09:12:40.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.225+0000 7ff4f17fa640 1 -- 172.21.15.42:0/2201168881 <== mon.2 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff504022030 con 0x564b3331e390 2024-08-30T09:12:40.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.225+0000 7ff4f17fa640 1 -- 172.21.15.42:0/2201168881 <== mon.2 v2:172.21.15.167:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7ff50004c070 con 0x564b3331e390 2024-08-30T09:12:40.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.225+0000 7ff5163ad740 10 client.27239 fetch_fsmap finished waiting for FSMap version 43 2024-08-30T09:12:40.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.225+0000 7ff5163ad740 10 client.27239 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-30T09:12:40.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.225+0000 7ff5163ad740 20 client.27239 populate_metadata read hostname 'smithi042' 2024-08-30T09:12:40.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.225+0000 7ff5163ad740 1 --2- 172.21.15.42:0/2201168881 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x564b332d2da0 0x564b3376eca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:12:40.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.225+0000 7ff5163ad740 4 client.27239 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-30T09:12:40.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.225+0000 7ff5163ad740 1 -- 172.21.15.42:0/2201168881 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564b3365d5d0 con 0x564b332d2da0 2024-08-30T09:12:40.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.225+0000 7ff50dbd4640 1 --2- 172.21.15.42:0/2201168881 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x564b332d2da0 0x564b3376eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:12:40.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.227+0000 7ff50dbd4640 1 --2- 172.21.15.42:0/2201168881 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x564b332d2da0 0x564b3376eca0 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f38a6e0c3c62430 server_cookie=57ec30e2474b69c2 in_seq=0 out_seq=0 2024-08-30T09:12:40.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.227+0000 7ff4f17fa640 10 client.27239 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:12:40.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.228+0000 7ff4f17fa640 1 -- 172.21.15.42:0/2201168881 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564b3365d5d0 con 0x564b332d2da0 2024-08-30T09:12:40.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.228+0000 7ff4f17fa640 10 client.27239 handle_command_reply: tid=0 2024-08-30T09:12:40.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.228+0000 7ff5163ad740 1 client.27239 shutdown 2024-08-30T09:12:40.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.230+0000 7ff5163ad740 1 -- 172.21.15.42:0/2201168881 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564b3331e390 msgr2=0x564b336f61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:12:40.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.230+0000 7ff5163ad740 1 --2- 172.21.15.42:0/2201168881 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564b3331e390 0x564b336f61c0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7ff5000cd010 tx=0x7ff500005fd0 comp rx=0 tx=0).stop 2024-08-30T09:12:40.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.238+0000 7ff5163ad740 1 -- 172.21.15.42:0/2201168881 shutdown_connections 2024-08-30T09:12:40.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.238+0000 7ff5163ad740 1 --2- 172.21.15.42:0/2201168881 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x564b332d2da0 0x564b3376eca0 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:40.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.238+0000 7ff5163ad740 1 --2- 172.21.15.42:0/2201168881 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564b3333f070 0x564b3376b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:40.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.238+0000 7ff5163ad740 1 --2- 172.21.15.42:0/2201168881 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564b3331e390 0x564b336f61c0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:40.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.238+0000 7ff5163ad740 1 --2- 172.21.15.42:0/2201168881 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564b333161b0 0x564b336f5c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:40.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.238+0000 7ff5163ad740 1 -- 172.21.15.42:0/2201168881 >> 172.21.15.42:0/2201168881 conn(0x564b33333460 msgr2=0x564b336e5d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:12:40.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.239+0000 7ff4f17fa640 0 client.27239 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:12:40.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.243+0000 7ff5163ad740 1 -- 172.21.15.42:0/2201168881 shutdown_connections 2024-08-30T09:12:40.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.243+0000 7ff5163ad740 1 -- 172.21.15.42:0/2201168881 wait complete. 2024-08-30T09:12:40.246 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.243+0000 7ff5163ad740 20 client.27239 trim_cache size 0 max 16384 2024-08-30T09:12:40.246 INFO:teuthology.orchestra.run.smithi042.stdout:[] 2024-08-30T09:12:40.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.244+0000 7ff50fe38640 1 -- 172.21.15.42:0/1545207124 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7ff4f40b4c70 msgr2=0x7ff4f40b7180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:12:40.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.244+0000 7ff50fe38640 1 --2- 172.21.15.42:0/1545207124 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7ff4f40b4c70 0x7ff4f40b7180 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7ff50400bd80 tx=0x7ff5040046c0 comp rx=0 tx=0).stop 2024-08-30T09:12:40.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.245+0000 7ff50fe38640 1 -- 172.21.15.42:0/1545207124 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff508111ce0 msgr2=0x7ff5081a9410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:12:40.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.245+0000 7ff50fe38640 1 --2- 172.21.15.42:0/1545207124 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff508111ce0 0x7ff5081a9410 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7ff50806c710 tx=0x7ff500030820 comp rx=0 tx=0).stop 2024-08-30T09:12:40.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.245+0000 7ff50fe38640 1 -- 172.21.15.42:0/1545207124 shutdown_connections 2024-08-30T09:12:40.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.245+0000 7ff50fe38640 1 --2- 172.21.15.42:0/1545207124 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7ff4f40b4c70 0x7ff4f40b7180 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:40.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.245+0000 7ff50fe38640 1 --2- 172.21.15.42:0/1545207124 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff508111ce0 0x7ff5081a9410 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:40.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.245+0000 7ff50fe38640 1 --2- 172.21.15.42:0/1545207124 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff50811c9d0 0x7ff5081ae6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:40.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.245+0000 7ff50fe38640 1 --2- 172.21.15.42:0/1545207124 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff50810a780 0x7ff5081a8e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:40.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.245+0000 7ff50fe38640 1 -- 172.21.15.42:0/1545207124 >> 172.21.15.42:0/1545207124 conn(0x7ff508070c60 msgr2=0x7ff50811cea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:12:40.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.249+0000 7ff50fe38640 1 -- 172.21.15.42:0/1545207124 shutdown_connections 2024-08-30T09:12:40.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:40.249+0000 7ff50fe38640 1 -- 172.21.15.42:0/1545207124 wait complete. 2024-08-30T09:12:40.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:12:40 smithi042 ceph-mon[18652]: pgmap v730: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 6.1 MiB/s wr, 526 op/s 2024-08-30T09:12:40.953 INFO:teuthology.orchestra.run.smithi042.stderr:progress: 715.0 s, 470.2 tps, lat 208.026 ms stddev 277.280 2024-08-30T09:12:41.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:12:40 smithi167 ceph-mon[22063]: pgmap v730: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 6.1 MiB/s wr, 526 op/s 2024-08-30T09:12:41.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:12:40 smithi190 ceph-mon[22001]: pgmap v730: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 6.1 MiB/s wr, 526 op/s 2024-08-30T09:12:42.006 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-08-30T09:12:42.006 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-08-30T09:12:42.600 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T09:12:43.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:12:42 smithi167 ceph-mon[22063]: pgmap v731: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 4.4 MiB/s wr, 406 op/s 2024-08-30T09:12:43.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:12:42 smithi042 ceph-mon[18652]: pgmap v731: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 4.4 MiB/s wr, 406 op/s 2024-08-30T09:12:43.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:12:42 smithi190 ceph-mon[22001]: pgmap v731: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 4.4 MiB/s wr, 406 op/s 2024-08-30T09:12:45.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:12:44 smithi167 ceph-mon[22063]: pgmap v732: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.0 MiB/s wr, 528 op/s 2024-08-30T09:12:45.078 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:12:44 smithi042 ceph-mon[18652]: pgmap v732: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.0 MiB/s wr, 528 op/s 2024-08-30T09:12:45.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:12:44 smithi190 ceph-mon[22001]: pgmap v732: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.0 MiB/s wr, 528 op/s 2024-08-30T09:12:45.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.258+0000 7f22d0846640 1 -- 172.21.15.42:0/2298998119 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f22cc102f20 msgr2=0x7f22cc103330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:12:45.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.258+0000 7f22d0846640 1 --2- 172.21.15.42:0/2298998119 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f22cc102f20 0x7f22cc103330 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f22bc009900 tx=0x7f22bc02a410 comp rx=0 tx=0).stop 2024-08-30T09:12:45.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.258+0000 7f22d0846640 1 -- 172.21.15.42:0/2298998119 shutdown_connections 2024-08-30T09:12:45.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.258+0000 7f22d0846640 1 --2- 172.21.15.42:0/2298998119 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f22cc108aa0 0x7f22cc10f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:45.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.258+0000 7f22d0846640 1 --2- 172.21.15.42:0/2298998119 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f22cc108150 0x7f22cc108560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:45.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.258+0000 7f22d0846640 1 --2- 172.21.15.42:0/2298998119 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f22cc102f20 0x7f22cc103330 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:45.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.258+0000 7f22d0846640 1 -- 172.21.15.42:0/2298998119 >> 172.21.15.42:0/2298998119 conn(0x7f22cc077a00 msgr2=0x7f22cc077e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:12:45.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.259+0000 7f22d0846640 1 -- 172.21.15.42:0/2298998119 shutdown_connections 2024-08-30T09:12:45.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.259+0000 7f22d0846640 1 -- 172.21.15.42:0/2298998119 wait complete. 2024-08-30T09:12:45.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.260+0000 7f22d0846640 1 Processor -- start 2024-08-30T09:12:45.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.260+0000 7f22d0846640 1 -- start start 2024-08-30T09:12:45.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.260+0000 7f22d0846640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f22cc102f20 0x7f22cc19c450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:12:45.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.260+0000 7f22d0846640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f22cc108150 0x7f22cc19c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:12:45.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.260+0000 7f22d0846640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f22cc108aa0 0x7f22cc1a1c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:12:45.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.260+0000 7f22d0846640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f22cc112750 con 0x7f22cc108aa0 2024-08-30T09:12:45.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.260+0000 7f22d0846640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f22cc1125d0 con 0x7f22cc102f20 2024-08-30T09:12:45.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.260+0000 7f22d0846640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f22cc1128d0 con 0x7f22cc108150 2024-08-30T09:12:45.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.260+0000 7f22c9d9b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f22cc108150 0x7f22cc19c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:12:45.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.260+0000 7f22cad9d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f22cc108aa0 0x7f22cc1a1c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:12:45.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.260+0000 7f22c9d9b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f22cc108150 0x7f22cc19c9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:46524/0 (socket says 172.21.15.42:46524) 2024-08-30T09:12:45.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.260+0000 7f22c9d9b640 1 -- 172.21.15.42:0/260085615 learned_addr learned my addr 172.21.15.42:0/260085615 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:12:45.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.260+0000 7f22ca59c640 1 --2- 172.21.15.42:0/260085615 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f22cc102f20 0x7f22cc19c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:12:45.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.261+0000 7f22cad9d640 1 -- 172.21.15.42:0/260085615 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f22cc108150 msgr2=0x7f22cc19c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:12:45.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.261+0000 7f22cad9d640 1 --2- 172.21.15.42:0/260085615 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f22cc108150 0x7f22cc19c9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:45.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.261+0000 7f22cad9d640 1 -- 172.21.15.42:0/260085615 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f22cc102f20 msgr2=0x7f22cc19c450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:12:45.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.261+0000 7f22cad9d640 1 --2- 172.21.15.42:0/260085615 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f22cc102f20 0x7f22cc19c450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:45.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.261+0000 7f22cad9d640 1 -- 172.21.15.42:0/260085615 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22cc1a23c0 con 0x7f22cc108aa0 2024-08-30T09:12:45.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.261+0000 7f22ca59c640 1 --2- 172.21.15.42:0/260085615 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f22cc102f20 0x7f22cc19c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-30T09:12:45.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.261+0000 7f22cad9d640 1 --2- 172.21.15.42:0/260085615 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f22cc108aa0 0x7f22cc1a1c90 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f22c4009fd0 tx=0x7f22c400ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:12:45.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.261+0000 7f22bb7fe640 1 -- 172.21.15.42:0/260085615 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22c400ce20 con 0x7f22cc108aa0 2024-08-30T09:12:45.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.261+0000 7f22bb7fe640 1 -- 172.21.15.42:0/260085615 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f22c4005020 con 0x7f22cc108aa0 2024-08-30T09:12:45.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.261+0000 7f22d0846640 1 -- 172.21.15.42:0/260085615 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22cc1a26b0 con 0x7f22cc108aa0 2024-08-30T09:12:45.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.262+0000 7f22bb7fe640 1 -- 172.21.15.42:0/260085615 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22c4010690 con 0x7f22cc108aa0 2024-08-30T09:12:45.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.262+0000 7f22d0846640 1 -- 172.21.15.42:0/260085615 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22cc1a8e10 con 0x7f22cc108aa0 2024-08-30T09:12:45.266 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.264+0000 7f22bb7fe640 1 -- 172.21.15.42:0/260085615 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7f22c40054d0 con 0x7f22cc108aa0 2024-08-30T09:12:45.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.265+0000 7f22bb7fe640 1 --2- 172.21.15.42:0/260085615 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f22b40b4c30 0x7f22b40b7140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:12:45.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.265+0000 7f22d6dbb740 1 Processor -- start 2024-08-30T09:12:45.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.265+0000 7f22ca59c640 1 --2- 172.21.15.42:0/260085615 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f22b40b4c30 0x7f22b40b7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:12:45.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.265+0000 7f22d6dbb740 1 -- start start 2024-08-30T09:12:45.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.265+0000 7f22bb7fe640 1 -- 172.21.15.42:0/260085615 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f22c40e5bb0 con 0x7f22cc108aa0 2024-08-30T09:12:45.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.266+0000 7f22d6dbb740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56338a9d6350 0x56338ad7c780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:12:45.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.266+0000 7f22d6dbb740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56338a9ad490 0x56338ad7ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:12:45.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.266+0000 7f22d6dbb740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56338a9b5670 0x56338ad7d200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:12:45.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.266+0000 7f22d6dbb740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x56338acf25f0 con 0x56338a9d6350 2024-08-30T09:12:45.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.266+0000 7f22d6dbb740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x56338ace1b80 con 0x56338a9ad490 2024-08-30T09:12:45.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.266+0000 7f22d6dbb740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x56338ace5b30 con 0x56338a9b5670 2024-08-30T09:12:45.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.266+0000 7f22ca59c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56338a9d6350 0x56338ad7c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:12:45.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.266+0000 7f22ca59c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56338a9d6350 0x56338ad7c780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37636/0 (socket says 172.21.15.42:37636) 2024-08-30T09:12:45.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.266+0000 7f22ca59c640 1 -- 172.21.15.42:0/1405983121 learned_addr learned my addr 172.21.15.42:0/1405983121 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:12:45.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.266+0000 7f22cad9d640 1 --2- 172.21.15.42:0/1405983121 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56338a9b5670 0x56338ad7d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:12:45.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.266+0000 7f22ca59c640 1 --2- 172.21.15.42:0/260085615 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f22b40b4c30 0x7f22b40b7140 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f22bc000c00 tx=0x7f22bc005d50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:12:45.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.266+0000 7f22c9d9b640 1 --2- 172.21.15.42:0/1405983121 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56338a9ad490 0x56338ad7ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:12:45.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.266+0000 7f22ca59c640 1 -- 172.21.15.42:0/1405983121 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56338a9b5670 msgr2=0x56338ad7d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:12:45.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.266+0000 7f22ca59c640 1 --2- 172.21.15.42:0/1405983121 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56338a9b5670 0x56338ad7d200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:45.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.266+0000 7f22ca59c640 1 -- 172.21.15.42:0/1405983121 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56338a9ad490 msgr2=0x56338ad7ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:12:45.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.266+0000 7f22ca59c640 1 --2- 172.21.15.42:0/1405983121 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56338a9ad490 0x56338ad7ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:45.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.266+0000 7f22ca59c640 1 -- 172.21.15.42:0/1405983121 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56338aa41930 con 0x56338a9d6350 2024-08-30T09:12:45.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.266+0000 7f22c9d9b640 1 --2- 172.21.15.42:0/1405983121 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56338a9ad490 0x56338ad7ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-30T09:12:45.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.266+0000 7f22ca59c640 1 --2- 172.21.15.42:0/1405983121 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56338a9d6350 0x56338ad7c780 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f22bc009900 tx=0x7f22bc03f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec2d8cdf9444db80 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:12:45.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.267+0000 7f22b8ff9640 1 -- 172.21.15.42:0/1405983121 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22bc035020 con 0x56338a9d6350 2024-08-30T09:12:45.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.267+0000 7f22b8ff9640 1 -- 172.21.15.42:0/1405983121 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f22bc004c10 con 0x56338a9d6350 2024-08-30T09:12:45.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.267+0000 7f22b8ff9640 1 -- 172.21.15.42:0/1405983121 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22bc031e60 con 0x56338a9d6350 2024-08-30T09:12:45.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.267+0000 7f22d6dbb740 1 -- 172.21.15.42:0/1405983121 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56338a9d6350 msgr2=0x56338ad7c780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:12:45.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.268+0000 7f22d6dbb740 1 --2- 172.21.15.42:0/1405983121 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56338a9d6350 0x56338ad7c780 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f22bc009900 tx=0x7f22bc03f000 comp rx=0 tx=0).stop 2024-08-30T09:12:45.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.268+0000 7f22d6dbb740 1 -- 172.21.15.42:0/1405983121 shutdown_connections 2024-08-30T09:12:45.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.268+0000 7f22d6dbb740 1 --2- 172.21.15.42:0/1405983121 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56338a9b5670 0x56338ad7d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:45.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.268+0000 7f22d6dbb740 1 --2- 172.21.15.42:0/1405983121 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56338a9ad490 0x56338ad7ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:45.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.268+0000 7f22d6dbb740 1 --2- 172.21.15.42:0/1405983121 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56338a9d6350 0x56338ad7c780 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:45.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.268+0000 7f22d6dbb740 1 -- 172.21.15.42:0/1405983121 >> 172.21.15.42:0/1405983121 conn(0x56338a9ca740 msgr2=0x56338ad7a670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:12:45.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.268+0000 7f22d6dbb740 1 -- 172.21.15.42:0/1405983121 shutdown_connections 2024-08-30T09:12:45.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.268+0000 7f22d6dbb740 1 -- 172.21.15.42:0/1405983121 wait complete. 2024-08-30T09:12:45.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.268+0000 7f22d6dbb740 1 -- start start 2024-08-30T09:12:45.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.268+0000 7f22d6dbb740 1 Processor -- start 2024-08-30T09:12:45.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.269+0000 7f22d6dbb740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56338a9ad490 0x56338ad87ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:12:45.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.269+0000 7f22d6dbb740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56338a9b5670 0x56338ad88400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:12:45.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.269+0000 7f22d6dbb740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56338a9d6350 0x56338ad94280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:12:45.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.269+0000 7f22d6dbb740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x56338ab44890 con 0x56338a9ad490 2024-08-30T09:12:45.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.269+0000 7f22d6dbb740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x56338ac68790 con 0x56338a9d6350 2024-08-30T09:12:45.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.269+0000 7f22d6dbb740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x56338ad8af00 con 0x56338a9b5670 2024-08-30T09:12:45.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.269+0000 7f22ca59c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56338a9ad490 0x56338ad87ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:12:45.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.269+0000 7f22ca59c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56338a9ad490 0x56338ad87ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37652/0 (socket says 172.21.15.42:37652) 2024-08-30T09:12:45.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.269+0000 7f22ca59c640 1 -- 172.21.15.42:0/1923366398 learned_addr learned my addr 172.21.15.42:0/1923366398 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:12:45.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.269+0000 7f22d6dbb740 10 client.0 _ll_register_callbacks cb 0x56338a9edba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:12:45.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.269+0000 7f22c9d9b640 1 --2- 172.21.15.42:0/1923366398 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56338a9b5670 0x56338ad88400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:12:45.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.269+0000 7f22cad9d640 1 --2- 172.21.15.42:0/1923366398 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56338a9d6350 0x56338ad94280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:12:45.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.269+0000 7f22c9d9b640 1 -- 172.21.15.42:0/1923366398 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56338a9d6350 msgr2=0x56338ad94280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:12:45.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.269+0000 7f22c9d9b640 1 --2- 172.21.15.42:0/1923366398 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56338a9d6350 0x56338ad94280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:45.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.269+0000 7f22c9d9b640 1 -- 172.21.15.42:0/1923366398 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56338a9ad490 msgr2=0x56338ad87ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:12:45.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.269+0000 7f22c9d9b640 1 --2- 172.21.15.42:0/1923366398 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56338a9ad490 0x56338ad87ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:45.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.269+0000 7f22c9d9b640 1 -- 172.21.15.42:0/1923366398 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56338ac9ff30 con 0x56338a9b5670 2024-08-30T09:12:45.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.270+0000 7f22c9d9b640 1 --2- 172.21.15.42:0/1923366398 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56338a9b5670 0x56338ad88400 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f22cc107b30 tx=0x7f22c000be60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=88b76367dce30a32 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:12:45.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.270+0000 7f22b1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-08-30T09:12:45.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.270+0000 7f22ca59c640 1 --2- 172.21.15.42:0/1923366398 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56338a9ad490 0x56338ad87ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-30T09:12:45.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.270+0000 7f22b1ffb640 1 -- 172.21.15.42:0/1923366398 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22c000ec10 con 0x56338a9b5670 2024-08-30T09:12:45.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.270+0000 7f22b1ffb640 1 -- 172.21.15.42:0/1923366398 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f22c000edb0 con 0x56338a9b5670 2024-08-30T09:12:45.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.270+0000 7f22d6dbb740 1 -- 172.21.15.42:0/1923366398 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f22bc004c10 con 0x56338a9b5670 2024-08-30T09:12:45.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.270+0000 7f22b1ffb640 1 -- 172.21.15.42:0/1923366398 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f22c002c050 con 0x56338a9b5670 2024-08-30T09:12:45.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.270+0000 7f22d6dbb740 10 client.27211 fetch_fsmap learned FSMap version 43 2024-08-30T09:12:45.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.270+0000 7f22d6dbb740 1 -- 172.21.15.42:0/1923366398 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x56338ad96d30 con 0x56338a9b5670 2024-08-30T09:12:45.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.271+0000 7f22b1ffb640 1 -- 172.21.15.42:0/1923366398 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f22bc004c10 con 0x56338a9b5670 2024-08-30T09:12:45.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.271+0000 7f22b1ffb640 1 -- 172.21.15.42:0/1923366398 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f22c0021400 con 0x56338a9b5670 2024-08-30T09:12:45.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.271+0000 7f22d6dbb740 10 client.27211 fetch_fsmap finished waiting for FSMap version 43 2024-08-30T09:12:45.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.271+0000 7f22d6dbb740 10 client.27211 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-30T09:12:45.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.271+0000 7f22d6dbb740 20 client.27211 populate_metadata read hostname 'smithi042' 2024-08-30T09:12:45.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.271+0000 7f22d6dbb740 1 --2- 172.21.15.42:0/1923366398 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x56338a994cd0 0x56338ad9a390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:12:45.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.271+0000 7f22d6dbb740 4 client.27211 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-30T09:12:45.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.271+0000 7f22d6dbb740 1 -- 172.21.15.42:0/1923366398 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56338acf4540 con 0x56338a994cd0 2024-08-30T09:12:45.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.271+0000 7f22ca59c640 1 --2- 172.21.15.42:0/1923366398 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x56338a994cd0 0x56338ad9a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:12:45.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.272+0000 7f22b1ffb640 10 client.27211 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:12:45.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.272+0000 7f22ca59c640 1 --2- 172.21.15.42:0/1923366398 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x56338a994cd0 0x56338ad9a390 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1aa5b18afdbfd02a server_cookie=601636e72489141 in_seq=0 out_seq=0 2024-08-30T09:12:45.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.272+0000 7f22b1ffb640 1 -- 172.21.15.42:0/1923366398 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56338acf4540 con 0x56338a994cd0 2024-08-30T09:12:45.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.272+0000 7f22b1ffb640 10 client.27211 handle_command_reply: tid=0 2024-08-30T09:12:45.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.272+0000 7f22d6dbb740 1 client.27211 shutdown 2024-08-30T09:12:45.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.273+0000 7f22d6dbb740 1 -- 172.21.15.42:0/1923366398 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56338a9b5670 msgr2=0x56338ad88400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:12:45.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.273+0000 7f22d6dbb740 1 --2- 172.21.15.42:0/1923366398 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56338a9b5670 0x56338ad88400 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f22cc107b30 tx=0x7f22c000be60 comp rx=0 tx=0).stop 2024-08-30T09:12:45.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.273+0000 7f22d6dbb740 1 -- 172.21.15.42:0/1923366398 shutdown_connections 2024-08-30T09:12:45.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.273+0000 7f22d6dbb740 1 --2- 172.21.15.42:0/1923366398 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x56338a994cd0 0x56338ad9a390 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:45.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.273+0000 7f22d6dbb740 1 --2- 172.21.15.42:0/1923366398 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56338a9b5670 0x56338ad88400 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:45.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.273+0000 7f22b1ffb640 0 client.27211 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:12:45.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.273+0000 7f22d6dbb740 1 --2- 172.21.15.42:0/1923366398 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56338a9d6350 0x56338ad94280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:45.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.273+0000 7f22d6dbb740 1 --2- 172.21.15.42:0/1923366398 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56338a9ad490 0x56338ad87ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:45.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.273+0000 7f22d6dbb740 1 -- 172.21.15.42:0/1923366398 >> 172.21.15.42:0/1923366398 conn(0x56338a9ca740 msgr2=0x56338ad7f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:12:45.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.273+0000 7f22d6dbb740 1 -- 172.21.15.42:0/1923366398 shutdown_connections 2024-08-30T09:12:45.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.273+0000 7f22d6dbb740 1 -- 172.21.15.42:0/1923366398 wait complete. 2024-08-30T09:12:45.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.273+0000 7f22d6dbb740 20 client.27211 trim_cache size 0 max 16384 2024-08-30T09:12:45.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.298+0000 7f22d6dbb740 1 Processor -- start 2024-08-30T09:12:45.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.298+0000 7f22d6dbb740 1 -- start start 2024-08-30T09:12:45.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.299+0000 7f22d6dbb740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56338a9ad490 0x56338ad7dec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:12:45.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.299+0000 7f22d6dbb740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56338a9b5670 0x56338ad79ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:12:45.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.299+0000 7f22d6dbb740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56338a9d6350 0x56338ad7a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:12:45.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.299+0000 7f22d6dbb740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f22cc1b0f30 con 0x56338a9d6350 2024-08-30T09:12:45.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.299+0000 7f22d6dbb740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f22cc1ae800 con 0x56338a9ad490 2024-08-30T09:12:45.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.299+0000 7f22c9d9b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56338a9b5670 0x56338ad79ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:12:45.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.299+0000 7f22ca59c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56338a9ad490 0x56338ad7dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:12:45.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.299+0000 7f22ca59c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56338a9ad490 0x56338ad7dec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:35794/0 (socket says 172.21.15.42:35794) 2024-08-30T09:12:45.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.299+0000 7f22ca59c640 1 -- 172.21.15.42:0/1225468542 learned_addr learned my addr 172.21.15.42:0/1225468542 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:12:45.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.299+0000 7f22cad9d640 1 --2- 172.21.15.42:0/1225468542 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56338a9d6350 0x56338ad7a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:12:45.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.300+0000 7f22d6dbb740 1 -- 172.21.15.42:0/1225468542 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f22cc1ac690 con 0x56338a9b5670 2024-08-30T09:12:45.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.300+0000 7f22c9d9b640 1 -- 172.21.15.42:0/1225468542 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56338a9ad490 msgr2=0x56338ad7dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:12:45.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.300+0000 7f22c9d9b640 1 --2- 172.21.15.42:0/1225468542 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56338a9ad490 0x56338ad7dec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:45.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.300+0000 7f22c9d9b640 1 -- 172.21.15.42:0/1225468542 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56338a9d6350 msgr2=0x56338ad7a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:12:45.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.300+0000 7f22c9d9b640 1 --2- 172.21.15.42:0/1225468542 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56338a9d6350 0x56338ad7a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:45.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.300+0000 7f22c9d9b640 1 -- 172.21.15.42:0/1225468542 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56338ad89180 con 0x56338a9b5670 2024-08-30T09:12:45.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.301+0000 7f22c9d9b640 1 --2- 172.21.15.42:0/1225468542 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56338a9b5670 0x56338ad79ca0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f22cc107b30 tx=0x7f22c0021cf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2060399d194ca2e4 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:12:45.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.301+0000 7f22b3fff640 1 -- 172.21.15.42:0/1225468542 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22c000ee20 con 0x56338a9b5670 2024-08-30T09:12:45.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.301+0000 7f22b3fff640 1 -- 172.21.15.42:0/1225468542 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f22c0035070 con 0x56338a9b5670 2024-08-30T09:12:45.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.301+0000 7f22b3fff640 1 -- 172.21.15.42:0/1225468542 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22c00035f0 con 0x56338a9b5670 2024-08-30T09:12:45.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.302+0000 7f22d6dbb740 1 -- 172.21.15.42:0/1225468542 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56338a9b5670 msgr2=0x56338ad79ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:12:45.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.302+0000 7f22d6dbb740 1 --2- 172.21.15.42:0/1225468542 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56338a9b5670 0x56338ad79ca0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f22cc107b30 tx=0x7f22c0021cf0 comp rx=0 tx=0).stop 2024-08-30T09:12:45.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.303+0000 7f22d6dbb740 1 -- 172.21.15.42:0/1225468542 shutdown_connections 2024-08-30T09:12:45.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.303+0000 7f22d6dbb740 1 --2- 172.21.15.42:0/1225468542 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56338a9b5670 0x56338ad79ca0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:45.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.303+0000 7f22d6dbb740 1 --2- 172.21.15.42:0/1225468542 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56338a9d6350 0x56338ad7a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:45.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.303+0000 7f22d6dbb740 1 --2- 172.21.15.42:0/1225468542 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56338a9ad490 0x56338ad7dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:45.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.303+0000 7f22d6dbb740 1 -- 172.21.15.42:0/1225468542 >> 172.21.15.42:0/1225468542 conn(0x56338a9ca740 msgr2=0x56338ad7d7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:12:45.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.303+0000 7f22d6dbb740 1 -- 172.21.15.42:0/1225468542 shutdown_connections 2024-08-30T09:12:45.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.303+0000 7f22d6dbb740 1 -- 172.21.15.42:0/1225468542 wait complete. 2024-08-30T09:12:45.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.303+0000 7f22d6dbb740 1 -- start start 2024-08-30T09:12:45.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.303+0000 7f22d6dbb740 1 Processor -- start 2024-08-30T09:12:45.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.304+0000 7f22d6dbb740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56338a9ad490 0x56338ad8c470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:12:45.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.304+0000 7f22d6dbb740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56338a9b5670 0x56338ad8c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:12:45.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.304+0000 7f22d6dbb740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56338a9d6350 0x56338ae03d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:12:45.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.304+0000 7f22d6dbb740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f22cc1acce0 con 0x56338a9d6350 2024-08-30T09:12:45.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.304+0000 7f22d6dbb740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f22cc1ae130 con 0x56338a9b5670 2024-08-30T09:12:45.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.304+0000 7f22d6dbb740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f22cc1ad390 con 0x56338a9ad490 2024-08-30T09:12:45.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.304+0000 7f22d6dbb740 10 client.0 _ll_register_callbacks cb 0x7f22c0018be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:12:45.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.304+0000 7f22cad9d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56338a9d6350 0x56338ae03d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:12:45.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.304+0000 7f22ca59c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56338a9ad490 0x56338ad8c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:12:45.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.304+0000 7f22cad9d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56338a9d6350 0x56338ae03d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37660/0 (socket says 172.21.15.42:37660) 2024-08-30T09:12:45.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.304+0000 7f22ca59c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56338a9ad490 0x56338ad8c470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:46540/0 (socket says 172.21.15.42:46540) 2024-08-30T09:12:45.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.304+0000 7f22cad9d640 1 -- 172.21.15.42:0/2113458230 learned_addr learned my addr 172.21.15.42:0/2113458230 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:12:45.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.304+0000 7f22c9d9b640 1 --2- 172.21.15.42:0/2113458230 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56338a9b5670 0x56338ad8c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:12:45.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.304+0000 7f22ca59c640 1 -- 172.21.15.42:0/2113458230 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56338a9b5670 msgr2=0x56338ad8c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:12:45.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.304+0000 7f22ca59c640 1 --2- 172.21.15.42:0/2113458230 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56338a9b5670 0x56338ad8c9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:45.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.304+0000 7f22ca59c640 1 -- 172.21.15.42:0/2113458230 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56338a9d6350 msgr2=0x56338ae03d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:12:45.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.304+0000 7f22ca59c640 1 --2- 172.21.15.42:0/2113458230 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56338a9d6350 0x56338ae03d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:45.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.304+0000 7f22ca59c640 1 -- 172.21.15.42:0/2113458230 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56338ad8d1a0 con 0x56338a9ad490 2024-08-30T09:12:45.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.304+0000 7f22cad9d640 1 --2- 172.21.15.42:0/2113458230 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56338a9d6350 0x56338ae03d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-30T09:12:45.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.304+0000 7f22ca59c640 1 --2- 172.21.15.42:0/2113458230 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56338a9ad490 0x56338ad8c470 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f22bc006010 tx=0x7f22bc005a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5bbe2c1fd58e85f4 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:12:45.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.304+0000 7f22b1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-08-30T09:12:45.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.305+0000 7f22b1ffb640 1 -- 172.21.15.42:0/2113458230 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22bc05f030 con 0x56338a9ad490 2024-08-30T09:12:45.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.305+0000 7f22d6dbb740 1 -- 172.21.15.42:0/2113458230 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f22c0035070 con 0x56338a9ad490 2024-08-30T09:12:45.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.305+0000 7f22b1ffb640 1 -- 172.21.15.42:0/2113458230 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f22bc0319e0 con 0x56338a9ad490 2024-08-30T09:12:45.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.305+0000 7f22b1ffb640 1 -- 172.21.15.42:0/2113458230 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f22bc06c050 con 0x56338a9ad490 2024-08-30T09:12:45.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.305+0000 7f22d6dbb740 10 client.27217 fetch_fsmap learned FSMap version 43 2024-08-30T09:12:45.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.305+0000 7f22d6dbb740 1 -- 172.21.15.42:0/2113458230 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x56338ae05280 con 0x56338a9ad490 2024-08-30T09:12:45.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.305+0000 7f22b1ffb640 1 -- 172.21.15.42:0/2113458230 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f22c0035070 con 0x56338a9ad490 2024-08-30T09:12:45.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.305+0000 7f22b1ffb640 1 -- 172.21.15.42:0/2113458230 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f22bc03a040 con 0x56338a9ad490 2024-08-30T09:12:45.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.305+0000 7f22d6dbb740 10 client.27217 fetch_fsmap finished waiting for FSMap version 43 2024-08-30T09:12:45.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.305+0000 7f22d6dbb740 10 client.27217 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-30T09:12:45.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.305+0000 7f22d6dbb740 20 client.27217 populate_metadata read hostname 'smithi042' 2024-08-30T09:12:45.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.305+0000 7f22d6dbb740 1 --2- 172.21.15.42:0/2113458230 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x56338a96a080 0x56338ae07620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:12:45.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.305+0000 7f22d6dbb740 4 client.27217 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-08-30T09:12:45.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.305+0000 7f22d6dbb740 1 -- 172.21.15.42:0/2113458230 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x56338aceffe0 con 0x56338a96a080 2024-08-30T09:12:45.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.306+0000 7f22c9d9b640 1 --2- 172.21.15.42:0/2113458230 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x56338a96a080 0x56338ae07620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:12:45.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.306+0000 7f22b1ffb640 10 client.27217 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:12:45.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.306+0000 7f22c9d9b640 1 --2- 172.21.15.42:0/2113458230 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x56338a96a080 0x56338ae07620 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=250857e27962f842 server_cookie=3ede85c08ac53094 in_seq=0 out_seq=0 2024-08-30T09:12:45.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.308+0000 7f22b1ffb640 1 -- 172.21.15.42:0/2113458230 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56338aceffe0 con 0x56338a96a080 2024-08-30T09:12:45.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.308+0000 7f22b1ffb640 10 client.27217 handle_command_reply: tid=0 2024-08-30T09:12:45.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.308+0000 7f22d6dbb740 1 client.27217 shutdown 2024-08-30T09:12:45.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.308+0000 7f22d6dbb740 1 -- 172.21.15.42:0/2113458230 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56338a9ad490 msgr2=0x56338ad8c470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:12:45.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.308+0000 7f22d6dbb740 1 --2- 172.21.15.42:0/2113458230 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56338a9ad490 0x56338ad8c470 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f22bc006010 tx=0x7f22bc005a20 comp rx=0 tx=0).stop 2024-08-30T09:12:45.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.308+0000 7f22d6dbb740 1 -- 172.21.15.42:0/2113458230 shutdown_connections 2024-08-30T09:12:45.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.308+0000 7f22d6dbb740 1 --2- 172.21.15.42:0/2113458230 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x56338a96a080 0x56338ae07620 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:45.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.308+0000 7f22d6dbb740 1 --2- 172.21.15.42:0/2113458230 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56338a9d6350 0x56338ae03d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:45.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.308+0000 7f22d6dbb740 1 --2- 172.21.15.42:0/2113458230 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56338a9b5670 0x56338ad8c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:45.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.308+0000 7f22d6dbb740 1 --2- 172.21.15.42:0/2113458230 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56338a9ad490 0x56338ad8c470 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:45.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.308+0000 7f22d6dbb740 1 -- 172.21.15.42:0/2113458230 >> 172.21.15.42:0/2113458230 conn(0x56338a9ca740 msgr2=0x56338ad89350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:12:45.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.308+0000 7f22b1ffb640 0 client.27217 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:12:45.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.309+0000 7f22d6dbb740 1 -- 172.21.15.42:0/2113458230 shutdown_connections 2024-08-30T09:12:45.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.309+0000 7f22d6dbb740 1 -- 172.21.15.42:0/2113458230 wait complete. 2024-08-30T09:12:45.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.309+0000 7f22d6dbb740 20 client.27217 trim_cache size 0 max 16384 2024-08-30T09:12:45.315 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-08-30T09:12:45.315 INFO:teuthology.orchestra.run.smithi042.stdout: "return_code": 0, 2024-08-30T09:12:45.315 INFO:teuthology.orchestra.run.smithi042.stdout: "scrub_tag": "0cbf734e-f988-4b21-acac-57b55e3bde62", 2024-08-30T09:12:45.315 INFO:teuthology.orchestra.run.smithi042.stdout: "mode": "asynchronous" 2024-08-30T09:12:45.315 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-08-30T09:12:45.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.309+0000 7f22d0846640 1 -- 172.21.15.42:0/260085615 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f22b40b4c30 msgr2=0x7f22b40b7140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:12:45.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.309+0000 7f22d0846640 1 --2- 172.21.15.42:0/260085615 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f22b40b4c30 0x7f22b40b7140 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f22bc000c00 tx=0x7f22bc005d50 comp rx=0 tx=0).stop 2024-08-30T09:12:45.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.309+0000 7f22d0846640 1 -- 172.21.15.42:0/260085615 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f22cc108aa0 msgr2=0x7f22cc1a1c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:12:45.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.309+0000 7f22d0846640 1 --2- 172.21.15.42:0/260085615 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f22cc108aa0 0x7f22cc1a1c90 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f22c4009fd0 tx=0x7f22c400ef10 comp rx=0 tx=0).stop 2024-08-30T09:12:45.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.309+0000 7f22d0846640 1 -- 172.21.15.42:0/260085615 shutdown_connections 2024-08-30T09:12:45.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.309+0000 7f22d0846640 1 --2- 172.21.15.42:0/260085615 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f22b40b4c30 0x7f22b40b7140 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:45.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.309+0000 7f22d0846640 1 --2- 172.21.15.42:0/260085615 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f22cc108150 0x7f22cc19c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:45.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.309+0000 7f22d0846640 1 --2- 172.21.15.42:0/260085615 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f22cc108aa0 0x7f22cc1a1c90 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:45.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.309+0000 7f22d0846640 1 --2- 172.21.15.42:0/260085615 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f22cc102f20 0x7f22cc19c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:45.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.309+0000 7f22d0846640 1 -- 172.21.15.42:0/260085615 >> 172.21.15.42:0/260085615 conn(0x7f22cc077a00 msgr2=0x7f22cc106060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:12:45.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.309+0000 7f22d0846640 1 -- 172.21.15.42:0/260085615 shutdown_connections 2024-08-30T09:12:45.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:45.310+0000 7f22d0846640 1 -- 172.21.15.42:0/260085615 wait complete. 2024-08-30T09:12:45.800 INFO:journalctl@ceph.mds.b.smithi167.stdout:Aug 30 09:12:45 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mds-b[52473]: 2024-08-30T09:12:45.346+0000 7f9d3fee4640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000000f [...2,head] /volumes/qa/sv_1/d3168875-d2e5-410d-8754-72d29955777b/postgres/data/pg_subtrans/ auth{1=1} v2497 ap=2 f(v11 m2024-08-30T09:10:39.595179+0000 3=3+0) n(v31 rc2024-08-30T09:10:39.598179+0000 b638976 4=3+1) (inest sync->mix g=1) caps={15402=pAsLsXsFs/pAsLsXsFsxcral@28} mcw={1=AsLsXsFs} | dirtyscattered=0 request=0 lock=0 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x555a32f6d080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x1000000000f:[<0x10000000006/pg_subtrans v284>,<0x10000000005/data v125>,<0x100000001fb/postgres v31>,<0x100000001fa/d3168875-d2e5-410d-8754-72d29955777b v45>,<0x10000000001/sv_1 v63>,<0x10000000000/qa v49>,<0x1/volumes v44>]//[]","memoryvalue":"(2)0x1000000000f:[<0x10000000006/pg_subtrans v2497>,<0x10000000005/data v935>,<0x100000001fb/postgres v585>,<0x100000001fa/d3168875-d2e5-410d-8754-72d29955777b v610>,<0x10000000001/sv_1 v633>,<0x10000000000/qa v608>,<0x1/volumes v571>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-08-30T09:09:44.086585+0000 2=2+0)","ondisk_value.rstat":"n(v0 rc2024-08-30T09:10:39.598179+0000 b638976 4=3+1)","memory_value.dirstat":"f(v11 m2024-08-30T09:10:39.595179+0000 3=3+0)","memory_value.rstat":"n(v31 rc2024-08-30T09:10:39.598179+0000 b638976 4=3+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-08-30T09:12:45.953 INFO:teuthology.orchestra.run.smithi042.stderr:progress: 720.0 s, 443.4 tps, lat 228.078 ms stddev 295.955 2024-08-30T09:12:45.979 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:12:45 smithi042 ceph-mon[18652]: scrub summary: active paths [/] 2024-08-30T09:12:45.979 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:12:45 smithi042 ceph-mon[18652]: scrub summary: idle+waiting paths [/] 2024-08-30T09:12:45.979 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:12:45 smithi042 ceph-mon[18652]: scrub summary: active paths [/] 2024-08-30T09:12:45.979 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:12:45 smithi042 ceph-mon[18652]: scrub summary: idle+waiting paths [/] 2024-08-30T09:12:45.979 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:12:45 smithi042 ceph-mon[18652]: scrub summary: active paths [/] 2024-08-30T09:12:46.029 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-30T09:12:46.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:12:45 smithi167 ceph-mon[22063]: scrub summary: active paths [/] 2024-08-30T09:12:46.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:12:45 smithi167 ceph-mon[22063]: scrub summary: idle+waiting paths [/] 2024-08-30T09:12:46.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:12:45 smithi167 ceph-mon[22063]: scrub summary: active paths [/] 2024-08-30T09:12:46.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:12:45 smithi167 ceph-mon[22063]: scrub summary: idle+waiting paths [/] 2024-08-30T09:12:46.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:12:45 smithi167 ceph-mon[22063]: scrub summary: active paths [/] 2024-08-30T09:12:46.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:12:45 smithi190 ceph-mon[22001]: scrub summary: active paths [/] 2024-08-30T09:12:46.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:12:45 smithi190 ceph-mon[22001]: scrub summary: idle+waiting paths [/] 2024-08-30T09:12:46.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:12:45 smithi190 ceph-mon[22001]: scrub summary: active paths [/] 2024-08-30T09:12:46.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:12:45 smithi190 ceph-mon[22001]: scrub summary: idle+waiting paths [/] 2024-08-30T09:12:46.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:12:45 smithi190 ceph-mon[22001]: scrub summary: active paths [/] 2024-08-30T09:12:46.447 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T09:12:46.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:12:46 smithi042 ceph-mon[18652]: pgmap v733: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.6 MiB/s wr, 434 op/s 2024-08-30T09:12:46.884 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:12:46 smithi042 ceph-mon[18652]: scrub queued for path: / 2024-08-30T09:12:46.884 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:12:46 smithi042 ceph-mon[18652]: scrub summary: idle+waiting paths [/] 2024-08-30T09:12:46.884 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:12:46 smithi042 ceph-mon[18652]: scrub summary: active paths [/] 2024-08-30T09:12:46.884 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:12:46 smithi042 ceph-mon[18652]: scrub summary: active paths [/] 2024-08-30T09:12:46.884 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:12:46 smithi042 ceph-mon[18652]: scrub summary: idle+waiting paths [/] 2024-08-30T09:12:46.884 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:12:46 smithi042 ceph-mon[18652]: scrub summary: active paths [/] 2024-08-30T09:12:46.884 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:12:46 smithi042 ceph-mon[18652]: scrub summary: idle+waiting paths [/] 2024-08-30T09:12:46.884 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:12:46 smithi042 ceph-mon[18652]: scrub summary: active paths [/] 2024-08-30T09:12:46.884 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:12:46 smithi042 ceph-mon[18652]: Scrub error on inode 0x1000000000f (/volumes/qa/sv_1/d3168875-d2e5-410d-8754-72d29955777b/postgres/data/pg_subtrans) see mds.b log and `damage ls` output for details 2024-08-30T09:12:46.884 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:12:46 smithi042 ceph-mon[18652]: scrub summary: idle+waiting paths [/] 2024-08-30T09:12:46.884 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:12:46 smithi042 ceph-mon[18652]: scrub summary: idle+waiting paths [/] 2024-08-30T09:12:46.885 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:12:46 smithi042 ceph-mon[18652]: scrub summary: idle+waiting paths [/] 2024-08-30T09:12:47.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:12:46 smithi167 ceph-mon[22063]: pgmap v733: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.6 MiB/s wr, 434 op/s 2024-08-30T09:12:47.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:12:46 smithi167 ceph-mon[22063]: scrub queued for path: / 2024-08-30T09:12:47.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:12:46 smithi167 ceph-mon[22063]: scrub summary: idle+waiting paths [/] 2024-08-30T09:12:47.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:12:46 smithi167 ceph-mon[22063]: scrub summary: active paths [/] 2024-08-30T09:12:47.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:12:46 smithi167 ceph-mon[22063]: scrub summary: active paths [/] 2024-08-30T09:12:47.078 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:12:46 smithi167 ceph-mon[22063]: scrub summary: idle+waiting paths [/] 2024-08-30T09:12:47.078 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:12:46 smithi167 ceph-mon[22063]: scrub summary: active paths [/] 2024-08-30T09:12:47.078 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:12:46 smithi167 ceph-mon[22063]: scrub summary: idle+waiting paths [/] 2024-08-30T09:12:47.078 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:12:46 smithi167 ceph-mon[22063]: scrub summary: active paths [/] 2024-08-30T09:12:47.078 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:12:46 smithi167 ceph-mon[22063]: Scrub error on inode 0x1000000000f (/volumes/qa/sv_1/d3168875-d2e5-410d-8754-72d29955777b/postgres/data/pg_subtrans) see mds.b log and `damage ls` output for details 2024-08-30T09:12:47.078 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:12:46 smithi167 ceph-mon[22063]: scrub summary: idle+waiting paths [/] 2024-08-30T09:12:47.078 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:12:46 smithi167 ceph-mon[22063]: scrub summary: idle+waiting paths [/] 2024-08-30T09:12:47.078 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:12:46 smithi167 ceph-mon[22063]: scrub summary: idle+waiting paths [/] 2024-08-30T09:12:47.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:12:46 smithi190 ceph-mon[22001]: pgmap v733: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.6 MiB/s wr, 434 op/s 2024-08-30T09:12:47.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:12:46 smithi190 ceph-mon[22001]: scrub queued for path: / 2024-08-30T09:12:47.187 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:12:46 smithi190 ceph-mon[22001]: scrub summary: idle+waiting paths [/] 2024-08-30T09:12:47.187 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:12:46 smithi190 ceph-mon[22001]: scrub summary: active paths [/] 2024-08-30T09:12:47.187 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:12:46 smithi190 ceph-mon[22001]: scrub summary: active paths [/] 2024-08-30T09:12:47.187 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:12:46 smithi190 ceph-mon[22001]: scrub summary: idle+waiting paths [/] 2024-08-30T09:12:47.187 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:12:46 smithi190 ceph-mon[22001]: scrub summary: active paths [/] 2024-08-30T09:12:47.187 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:12:46 smithi190 ceph-mon[22001]: scrub summary: idle+waiting paths [/] 2024-08-30T09:12:47.187 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:12:46 smithi190 ceph-mon[22001]: scrub summary: active paths [/] 2024-08-30T09:12:47.187 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:12:46 smithi190 ceph-mon[22001]: Scrub error on inode 0x1000000000f (/volumes/qa/sv_1/d3168875-d2e5-410d-8754-72d29955777b/postgres/data/pg_subtrans) see mds.b log and `damage ls` output for details 2024-08-30T09:12:47.187 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:12:46 smithi190 ceph-mon[22001]: scrub summary: idle+waiting paths [/] 2024-08-30T09:12:47.187 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:12:46 smithi190 ceph-mon[22001]: scrub summary: idle+waiting paths [/] 2024-08-30T09:12:47.187 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:12:46 smithi190 ceph-mon[22001]: scrub summary: idle+waiting paths [/] 2024-08-30T09:12:47.618 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-08-30T09:12:47.619 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "IDLE (waiting for more scrubs)", 2024-08-30T09:12:47.619 INFO:teuthology.orchestra.run.smithi042.stdout: "scrubs": { 2024-08-30T09:12:47.619 INFO:teuthology.orchestra.run.smithi042.stdout: "0cbf734e-f988-4b21-acac-57b55e3bde62": { 2024-08-30T09:12:47.619 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:12:47.619 INFO:teuthology.orchestra.run.smithi042.stdout: "tag": "0cbf734e-f988-4b21-acac-57b55e3bde62", 2024-08-30T09:12:47.619 INFO:teuthology.orchestra.run.smithi042.stdout: "options": "recursive,force" 2024-08-30T09:12:47.619 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:12:47.619 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:12:47.619 INFO:teuthology.orchestra.run.smithi042.stdout: "scrub_stats": { 2024-08-30T09:12:47.619 INFO:teuthology.orchestra.run.smithi042.stdout: "0cbf734e-f988-4b21-acac-57b55e3bde62": { 2024-08-30T09:12:47.619 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:12:45.306738+0000", 2024-08-30T09:12:47.619 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:12:47.619 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:12:47.619 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:12:47.619 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:12:47.620 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:12:47.620 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:12:47.620 INFO:teuthology.orchestra.run.smithi042.stdout: "5e2ed104-0625-478a-a78b-584e7f4c2817": { 2024-08-30T09:12:47.620 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:00:33.928400+0000", 2024-08-30T09:12:47.620 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:12:47.620 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:12:47.620 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:12:47.620 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:12:47.620 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:12:47.620 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:12:47.620 INFO:teuthology.orchestra.run.smithi042.stdout: "c827ad7a-44ee-4e1d-9bb3-51fb7ab00c73": { 2024-08-30T09:12:47.620 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:02:21.686517+0000", 2024-08-30T09:12:47.620 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:12:47.621 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:12:47.621 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:12:47.621 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:12:47.621 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:12:47.621 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:12:47.621 INFO:teuthology.orchestra.run.smithi042.stdout: "85134e4b-ab4a-4885-b07e-a3ad01eaf13d": { 2024-08-30T09:12:47.621 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:01:27.738462+0000", 2024-08-30T09:12:47.621 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:12:47.621 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:12:47.621 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:12:47.621 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:12:47.621 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:12:47.621 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:12:47.621 INFO:teuthology.orchestra.run.smithi042.stdout: "749f4867-0b70-49e5-af2d-8362a240ebc3": { 2024-08-30T09:12:47.621 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:03:16.386554+0000", 2024-08-30T09:12:47.622 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:12:47.622 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:12:47.622 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:12:47.622 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:12:47.622 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:12:47.622 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:12:47.622 INFO:teuthology.orchestra.run.smithi042.stdout: "88cd7478-2b20-47de-ba51-256479749232": { 2024-08-30T09:12:47.622 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:04:08.922625+0000", 2024-08-30T09:12:47.622 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:12:47.622 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:12:47.622 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:12:47.622 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:12:47.622 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:12:47.622 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:12:47.622 INFO:teuthology.orchestra.run.smithi042.stdout: "347d05ed-4bd3-4590-b099-314f719aab39": { 2024-08-30T09:12:47.623 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:05:00.439721+0000", 2024-08-30T09:12:47.623 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:12:47.623 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:12:47.623 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:12:47.623 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:12:47.623 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:12:47.623 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:12:47.623 INFO:teuthology.orchestra.run.smithi042.stdout: "1af4252a-4a99-47e9-bbf0-becf5e573dd5": { 2024-08-30T09:12:47.623 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:05:48.565881+0000", 2024-08-30T09:12:47.623 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:12:47.623 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:12:47.623 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:12:47.623 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:12:47.623 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:12:47.623 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:12:47.624 INFO:teuthology.orchestra.run.smithi042.stdout: "665ec740-7254-49cb-84ef-deff71c461b0": { 2024-08-30T09:12:47.624 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:06:37.144035+0000", 2024-08-30T09:12:47.624 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:12:47.624 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:12:47.624 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:12:47.624 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:12:47.624 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:12:47.624 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:12:47.624 INFO:teuthology.orchestra.run.smithi042.stdout: "598d8a75-f1cc-4388-80bc-a408bdb4eaac": { 2024-08-30T09:12:47.624 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:07:29.721122+0000", 2024-08-30T09:12:47.624 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:12:47.624 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:12:47.624 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:12:47.624 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:12:47.625 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:12:47.625 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:12:47.625 INFO:teuthology.orchestra.run.smithi042.stdout: "b62c4ffc-9387-4677-b9ba-1e94ba308f17": { 2024-08-30T09:12:47.625 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:11:03.363478+0000", 2024-08-30T09:12:47.625 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:12:47.625 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:12:47.625 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:12:47.625 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:12:47.625 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:12:47.625 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:12:47.625 INFO:teuthology.orchestra.run.smithi042.stdout: "a7210f9a-046b-4188-953c-819b9f1863a9": { 2024-08-30T09:12:47.625 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:10:07.608430+0000", 2024-08-30T09:12:47.625 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:12:47.625 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:12:47.626 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:12:47.626 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:12:47.626 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:12:47.626 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:12:47.626 INFO:teuthology.orchestra.run.smithi042.stdout: "99a29ec1-9455-4ae4-854a-398b32896675": { 2024-08-30T09:12:47.626 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:08:24.923176+0000", 2024-08-30T09:12:47.626 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:12:47.626 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:12:47.626 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:12:47.626 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:12:47.626 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:12:47.626 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:12:47.626 INFO:teuthology.orchestra.run.smithi042.stdout: "ff464e7f-123b-4bc6-bc7e-aced018168de": { 2024-08-30T09:12:47.626 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:11:54.879598+0000", 2024-08-30T09:12:47.626 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:12:47.627 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:12:47.627 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:12:47.627 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:12:47.627 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:12:47.627 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:12:47.627 INFO:teuthology.orchestra.run.smithi042.stdout: "37f2f28f-496f-41a6-b4cf-8846fb3f43ab": { 2024-08-30T09:12:47.627 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:09:17.428286+0000", 2024-08-30T09:12:47.627 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:12:47.627 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:12:47.627 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:12:47.627 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:12:47.627 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:12:47.627 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:12:47.627 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:12:47.628 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-08-30T09:12:47.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.051+0000 7f7114798640 1 -- 172.21.15.42:0/3485641249 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f710c103900 msgr2=0x7f710c103d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:12:47.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.051+0000 7f7114798640 1 --2- 172.21.15.42:0/3485641249 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f710c103900 0x7f710c103d10 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f7108009900 tx=0x7f710802a410 comp rx=0 tx=0).stop 2024-08-30T09:12:47.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.051+0000 7f7114798640 1 -- 172.21.15.42:0/3485641249 shutdown_connections 2024-08-30T09:12:47.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.051+0000 7f7114798640 1 --2- 172.21.15.42:0/3485641249 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f710c109b90 0x7f710c1103c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:47.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.051+0000 7f7114798640 1 --2- 172.21.15.42:0/3485641249 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f710c109240 0x7f710c109650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:47.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.051+0000 7f7114798640 1 --2- 172.21.15.42:0/3485641249 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f710c103900 0x7f710c103d10 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:47.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.052+0000 7f7114798640 1 -- 172.21.15.42:0/3485641249 >> 172.21.15.42:0/3485641249 conn(0x7f710c07b5f0 msgr2=0x7f710c07ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:12:47.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.052+0000 7f7114798640 1 -- 172.21.15.42:0/3485641249 shutdown_connections 2024-08-30T09:12:47.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.052+0000 7f7114798640 1 -- 172.21.15.42:0/3485641249 wait complete. 2024-08-30T09:12:47.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.052+0000 7f7114798640 1 Processor -- start 2024-08-30T09:12:47.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.052+0000 7f7114798640 1 -- start start 2024-08-30T09:12:47.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.052+0000 7f7114798640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f710c103900 0x7f710c19a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:12:47.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.052+0000 7f7114798640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f710c109240 0x7f710c19a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:12:47.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.052+0000 7f7114798640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f710c109b90 0x7f710c19fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:12:47.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.053+0000 7f7114798640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f710c113850 con 0x7f710c109b90 2024-08-30T09:12:47.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.053+0000 7f7114798640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f710c1136d0 con 0x7f710c109240 2024-08-30T09:12:47.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.053+0000 7f7114798640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f710c1139d0 con 0x7f710c103900 2024-08-30T09:12:47.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.053+0000 7f7112d35640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f710c109b90 0x7f710c19fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:12:47.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.053+0000 7f7112d35640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f710c109b90 0x7f710c19fa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37676/0 (socket says 172.21.15.42:37676) 2024-08-30T09:12:47.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.053+0000 7f7112d35640 1 -- 172.21.15.42:0/2066095011 learned_addr learned my addr 172.21.15.42:0/2066095011 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:12:47.630 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.053+0000 7f7111d33640 1 --2- 172.21.15.42:0/2066095011 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f710c109240 0x7f710c19a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:12:47.630 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.053+0000 7f7112534640 1 --2- 172.21.15.42:0/2066095011 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f710c103900 0x7f710c19a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:12:47.630 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.053+0000 7f7112d35640 1 -- 172.21.15.42:0/2066095011 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f710c103900 msgr2=0x7f710c19a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:12:47.630 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.053+0000 7f7112d35640 1 --2- 172.21.15.42:0/2066095011 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f710c103900 0x7f710c19a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:47.630 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.053+0000 7f7112d35640 1 -- 172.21.15.42:0/2066095011 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f710c109240 msgr2=0x7f710c19a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:12:47.630 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.053+0000 7f7112d35640 1 --2- 172.21.15.42:0/2066095011 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f710c109240 0x7f710c19a7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:47.630 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.053+0000 7f7112d35640 1 -- 172.21.15.42:0/2066095011 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f710c1a01b0 con 0x7f710c109b90 2024-08-30T09:12:47.630 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.053+0000 7f7112534640 1 --2- 172.21.15.42:0/2066095011 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f710c103900 0x7f710c19a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-30T09:12:47.630 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.053+0000 7f7111d33640 1 --2- 172.21.15.42:0/2066095011 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f710c109240 0x7f710c19a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-30T09:12:47.630 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.053+0000 7f7112d35640 1 --2- 172.21.15.42:0/2066095011 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f710c109b90 0x7f710c19fa80 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f710400eaa0 tx=0x7f710400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:12:47.630 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.053+0000 7f71037fe640 1 -- 172.21.15.42:0/2066095011 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f710400cdb0 con 0x7f710c109b90 2024-08-30T09:12:47.630 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.053+0000 7f71037fe640 1 -- 172.21.15.42:0/2066095011 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7104005020 con 0x7f710c109b90 2024-08-30T09:12:47.630 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.053+0000 7f7114798640 1 -- 172.21.15.42:0/2066095011 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f710c1a04a0 con 0x7f710c109b90 2024-08-30T09:12:47.630 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.053+0000 7f71037fe640 1 -- 172.21.15.42:0/2066095011 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71040116d0 con 0x7f710c109b90 2024-08-30T09:12:47.630 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.053+0000 7f7114798640 1 -- 172.21.15.42:0/2066095011 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f710c1a6c00 con 0x7f710c109b90 2024-08-30T09:12:47.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.056+0000 7f71037fe640 1 -- 172.21.15.42:0/2066095011 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7f71040051d0 con 0x7f710c109b90 2024-08-30T09:12:47.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.056+0000 7f711ad0d740 1 Processor -- start 2024-08-30T09:12:47.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.057+0000 7f711ad0d740 1 -- start start 2024-08-30T09:12:47.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.057+0000 7f711ad0d740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c5557ea070 0x55c555b8f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:12:47.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.057+0000 7f711ad0d740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c5557c11b0 0x55c555b8f800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:12:47.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.057+0000 7f711ad0d740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c5557c9390 0x55c555b8fd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:12:47.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.057+0000 7f711ad0d740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55c555b06630 con 0x55c5557c9390 2024-08-30T09:12:47.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.057+0000 7f711ad0d740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55c555af5bc0 con 0x55c5557ea070 2024-08-30T09:12:47.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.057+0000 7f711ad0d740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55c555af9b70 con 0x55c5557c11b0 2024-08-30T09:12:47.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.057+0000 7f7111d33640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c5557c9390 0x55c555b8fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:12:47.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.057+0000 7f7111d33640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c5557c9390 0x55c555b8fd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37692/0 (socket says 172.21.15.42:37692) 2024-08-30T09:12:47.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.057+0000 7f7111d33640 1 -- 172.21.15.42:0/714560487 learned_addr learned my addr 172.21.15.42:0/714560487 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:12:47.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.057+0000 7f7112534640 1 --2- 172.21.15.42:0/714560487 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c5557c11b0 0x55c555b8f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:12:47.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.057+0000 7f7111d33640 1 --2- 172.21.15.42:0/714560487 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c5557ea070 0x55c555b8f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:12:47.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.058+0000 7f7111d33640 1 -- 172.21.15.42:0/714560487 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c5557c11b0 msgr2=0x55c555b8f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:12:47.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.058+0000 7f7111d33640 1 --2- 172.21.15.42:0/714560487 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c5557c11b0 0x55c555b8f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:47.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.058+0000 7f7111d33640 1 -- 172.21.15.42:0/714560487 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c5557ea070 msgr2=0x55c555b8f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:12:47.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.058+0000 7f7111d33640 1 --2- 172.21.15.42:0/714560487 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c5557ea070 0x55c555b8f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:47.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.058+0000 7f71037fe640 1 --2- 172.21.15.42:0/2066095011 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f70f80b4bb0 0x7f70f80b70c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:12:47.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.058+0000 7f7111d33640 1 -- 172.21.15.42:0/714560487 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c555855650 con 0x55c5557c9390 2024-08-30T09:12:47.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.058+0000 7f7112d35640 1 --2- 172.21.15.42:0/2066095011 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f70f80b4bb0 0x7f70f80b70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:12:47.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.058+0000 7f7111d33640 1 --2- 172.21.15.42:0/714560487 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c5557c9390 0x55c555b8fd40 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f70fc0127c0 tx=0x7f70fc0096c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b02638c87a20978 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:12:47.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.058+0000 7f71037fe640 1 -- 172.21.15.42:0/2066095011 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f71040e5720 con 0x7f710c109b90 2024-08-30T09:12:47.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.058+0000 7f7100ff9640 1 -- 172.21.15.42:0/714560487 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70fc012d40 con 0x55c5557c9390 2024-08-30T09:12:47.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.058+0000 7f7100ff9640 1 -- 172.21.15.42:0/714560487 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f70fc010c80 con 0x55c5557c9390 2024-08-30T09:12:47.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.058+0000 7f7100ff9640 1 -- 172.21.15.42:0/714560487 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70fc01fba0 con 0x55c5557c9390 2024-08-30T09:12:47.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.059+0000 7f711ad0d740 1 -- 172.21.15.42:0/714560487 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c5557c9390 msgr2=0x55c555b8fd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:12:47.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.059+0000 7f7112d35640 1 --2- 172.21.15.42:0/2066095011 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f70f80b4bb0 0x7f70f80b70c0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f71040abac0 tx=0x7f71040dbfc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:12:47.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.059+0000 7f711ad0d740 1 --2- 172.21.15.42:0/714560487 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c5557c9390 0x55c555b8fd40 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f70fc0127c0 tx=0x7f70fc0096c0 comp rx=0 tx=0).stop 2024-08-30T09:12:47.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.059+0000 7f711ad0d740 1 -- 172.21.15.42:0/714560487 shutdown_connections 2024-08-30T09:12:47.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.059+0000 7f711ad0d740 1 --2- 172.21.15.42:0/714560487 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c5557c11b0 0x55c555b8f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:47.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.059+0000 7f711ad0d740 1 --2- 172.21.15.42:0/714560487 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c5557c9390 0x55c555b8fd40 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:47.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.059+0000 7f711ad0d740 1 --2- 172.21.15.42:0/714560487 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c5557ea070 0x55c555b8f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:47.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.059+0000 7f711ad0d740 1 -- 172.21.15.42:0/714560487 >> 172.21.15.42:0/714560487 conn(0x55c5557de460 msgr2=0x55c555b8d1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:12:47.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.059+0000 7f711ad0d740 1 -- 172.21.15.42:0/714560487 shutdown_connections 2024-08-30T09:12:47.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.059+0000 7f711ad0d740 1 -- 172.21.15.42:0/714560487 wait complete. 2024-08-30T09:12:47.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.060+0000 7f711ad0d740 1 -- start start 2024-08-30T09:12:47.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.060+0000 7f711ad0d740 1 Processor -- start 2024-08-30T09:12:47.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.060+0000 7f711ad0d740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c5557c11b0 0x55c555b9aa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:12:47.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.060+0000 7f711ad0d740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c5557c9390 0x55c555b9afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:12:47.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.060+0000 7f711ad0d740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c5557ea070 0x55c555ba6d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:12:47.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.060+0000 7f711ad0d740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55c555958700 con 0x55c5557c9390 2024-08-30T09:12:47.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.060+0000 7f711ad0d740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55c555a7c700 con 0x55c5557c11b0 2024-08-30T09:12:47.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.060+0000 7f711ad0d740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55c555b9daa0 con 0x55c5557ea070 2024-08-30T09:12:47.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.060+0000 7f711ad0d740 10 client.0 _ll_register_callbacks cb 0x55c555aabdc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:12:47.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.060+0000 7f7112534640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c5557c9390 0x55c555b9afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:12:47.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.060+0000 7f7112534640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c5557c9390 0x55c555b9afd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37708/0 (socket says 172.21.15.42:37708) 2024-08-30T09:12:47.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.060+0000 7f7112534640 1 -- 172.21.15.42:0/3165053574 learned_addr learned my addr 172.21.15.42:0/3165053574 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:12:47.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.060+0000 7f7111d33640 1 --2- 172.21.15.42:0/3165053574 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c5557c11b0 0x55c555b9aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:12:47.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.060+0000 7f7111d33640 1 --2- 172.21.15.42:0/3165053574 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c5557ea070 0x55c555ba6d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:12:47.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.061+0000 7f7112534640 1 -- 172.21.15.42:0/3165053574 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c5557ea070 msgr2=0x55c555ba6d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:12:47.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.061+0000 7f7112534640 1 --2- 172.21.15.42:0/3165053574 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c5557ea070 0x55c555ba6d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:47.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.061+0000 7f7112534640 1 -- 172.21.15.42:0/3165053574 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c5557c11b0 msgr2=0x55c555b9aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:12:47.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.061+0000 7f7112534640 1 --2- 172.21.15.42:0/3165053574 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c5557c11b0 0x55c555b9aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:47.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.061+0000 7f7112534640 1 -- 172.21.15.42:0/3165053574 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c555ab58b0 con 0x55c5557c9390 2024-08-30T09:12:47.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.061+0000 7f7111d33640 1 --2- 172.21.15.42:0/3165053574 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c5557ea070 0x55c555ba6d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-30T09:12:47.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.061+0000 7f7112534640 1 --2- 172.21.15.42:0/3165053574 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c5557c9390 0x55c555b9afd0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f710802a930 tx=0x7f71080316d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=927321e5bfa2e040 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:12:47.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.061+0000 7f70f5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-08-30T09:12:47.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.061+0000 7f70f5ffb640 1 -- 172.21.15.42:0/3165053574 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7108035040 con 0x55c5557c9390 2024-08-30T09:12:47.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.061+0000 7f711ad0d740 1 -- 172.21.15.42:0/3165053574 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f70fc010c80 con 0x55c5557c9390 2024-08-30T09:12:47.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.061+0000 7f70f5ffb640 1 -- 172.21.15.42:0/3165053574 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f7108004240 con 0x55c5557c9390 2024-08-30T09:12:47.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.061+0000 7f70f5ffb640 1 -- 172.21.15.42:0/3165053574 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f710804e050 con 0x55c5557c9390 2024-08-30T09:12:47.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.061+0000 7f711ad0d740 10 client.17436 fetch_fsmap learned FSMap version 43 2024-08-30T09:12:47.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.061+0000 7f711ad0d740 1 -- 172.21.15.42:0/3165053574 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55c555ba9920 con 0x55c5557c9390 2024-08-30T09:12:47.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.061+0000 7f70f5ffb640 1 -- 172.21.15.42:0/3165053574 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f70fc010c80 con 0x55c5557c9390 2024-08-30T09:12:47.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.062+0000 7f70f5ffb640 1 -- 172.21.15.42:0/3165053574 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f7108004960 con 0x55c5557c9390 2024-08-30T09:12:47.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.062+0000 7f711ad0d740 10 client.17436 fetch_fsmap finished waiting for FSMap version 43 2024-08-30T09:12:47.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.062+0000 7f711ad0d740 10 client.17436 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-30T09:12:47.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.062+0000 7f711ad0d740 20 client.17436 populate_metadata read hostname 'smithi042' 2024-08-30T09:12:47.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.062+0000 7f711ad0d740 1 --2- 172.21.15.42:0/3165053574 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55c5557a89f0 0x55c555bacda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:12:47.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.062+0000 7f711ad0d740 4 client.17436 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-30T09:12:47.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.062+0000 7f711ad0d740 1 -- 172.21.15.42:0/3165053574 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c555b08580 con 0x55c5557a89f0 2024-08-30T09:12:47.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.062+0000 7f7111d33640 1 --2- 172.21.15.42:0/3165053574 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55c5557a89f0 0x55c555bacda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:12:47.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.062+0000 7f7111d33640 1 --2- 172.21.15.42:0/3165053574 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55c5557a89f0 0x55c555bacda0 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9f55aa25e4d70ff7 server_cookie=59acd4c9dcda42f9 in_seq=0 out_seq=0 2024-08-30T09:12:47.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.062+0000 7f70f5ffb640 10 client.17436 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:12:47.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.063+0000 7f70f5ffb640 1 -- 172.21.15.42:0/3165053574 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55c555b08580 con 0x55c5557a89f0 2024-08-30T09:12:47.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.063+0000 7f70f5ffb640 10 client.17436 handle_command_reply: tid=0 2024-08-30T09:12:47.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.063+0000 7f711ad0d740 1 client.17436 shutdown 2024-08-30T09:12:47.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.064+0000 7f711ad0d740 1 -- 172.21.15.42:0/3165053574 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c5557c9390 msgr2=0x55c555b9afd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:12:47.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.064+0000 7f711ad0d740 1 --2- 172.21.15.42:0/3165053574 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c5557c9390 0x55c555b9afd0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f710802a930 tx=0x7f71080316d0 comp rx=0 tx=0).stop 2024-08-30T09:12:47.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.064+0000 7f711ad0d740 1 -- 172.21.15.42:0/3165053574 shutdown_connections 2024-08-30T09:12:47.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.064+0000 7f711ad0d740 1 --2- 172.21.15.42:0/3165053574 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55c5557a89f0 0x55c555bacda0 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:47.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.064+0000 7f70f5ffb640 0 client.17436 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:12:47.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.064+0000 7f711ad0d740 1 --2- 172.21.15.42:0/3165053574 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c5557ea070 0x55c555ba6d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:47.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.064+0000 7f711ad0d740 1 --2- 172.21.15.42:0/3165053574 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c5557c9390 0x55c555b9afd0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:47.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.064+0000 7f711ad0d740 1 --2- 172.21.15.42:0/3165053574 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c5557c11b0 0x55c555b9aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:47.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.064+0000 7f711ad0d740 1 -- 172.21.15.42:0/3165053574 >> 172.21.15.42:0/3165053574 conn(0x55c5557de460 msgr2=0x55c555b924a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:12:47.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.064+0000 7f711ad0d740 1 -- 172.21.15.42:0/3165053574 shutdown_connections 2024-08-30T09:12:47.637 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.064+0000 7f711ad0d740 1 -- 172.21.15.42:0/3165053574 wait complete. 2024-08-30T09:12:47.637 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.064+0000 7f711ad0d740 20 client.17436 trim_cache size 0 max 16384 2024-08-30T09:12:47.637 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.082+0000 7f711ad0d740 1 Processor -- start 2024-08-30T09:12:47.637 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.082+0000 7f711ad0d740 1 -- start start 2024-08-30T09:12:47.637 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.082+0000 7f711ad0d740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c5557c11b0 0x55c555b90cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:12:47.637 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.082+0000 7f711ad0d740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c5557c9390 0x55c555b9c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:12:47.637 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.082+0000 7f711ad0d740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c5557ea070 0x55c555ba6c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:12:47.637 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.082+0000 7f711ad0d740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f710c1b3130 con 0x55c5557c9390 2024-08-30T09:12:47.637 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.082+0000 7f711ad0d740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f710c1b0a00 con 0x55c5557ea070 2024-08-30T09:12:47.637 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.082+0000 7f711ad0d740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f710c1ae850 con 0x55c5557c11b0 2024-08-30T09:12:47.637 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.082+0000 7f7112534640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c5557c9390 0x55c555b9c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:12:47.637 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.082+0000 7f7112534640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c5557c9390 0x55c555b9c6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37724/0 (socket says 172.21.15.42:37724) 2024-08-30T09:12:47.637 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.082+0000 7f7112534640 1 -- 172.21.15.42:0/3225453575 learned_addr learned my addr 172.21.15.42:0/3225453575 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:12:47.637 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.082+0000 7f7111d33640 1 --2- 172.21.15.42:0/3225453575 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c5557c11b0 0x55c555b90cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:12:47.638 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.082+0000 7f7111d33640 1 --2- 172.21.15.42:0/3225453575 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c5557ea070 0x55c555ba6c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:12:47.638 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.082+0000 7f7112534640 1 -- 172.21.15.42:0/3225453575 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c5557c11b0 msgr2=0x55c555b90cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:12:47.638 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.082+0000 7f7112534640 1 --2- 172.21.15.42:0/3225453575 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c5557c11b0 0x55c555b90cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:47.638 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.082+0000 7f7112534640 1 -- 172.21.15.42:0/3225453575 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c5557ea070 msgr2=0x55c555ba6c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:12:47.638 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.082+0000 7f7112534640 1 --2- 172.21.15.42:0/3225453575 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c5557ea070 0x55c555ba6c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:47.638 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.082+0000 7f7112534640 1 -- 172.21.15.42:0/3225453575 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c555b915b0 con 0x55c5557c9390 2024-08-30T09:12:47.638 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.082+0000 7f7111d33640 1 --2- 172.21.15.42:0/3225453575 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c5557ea070 0x55c555ba6c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-30T09:12:47.638 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.083+0000 7f7112534640 1 --2- 172.21.15.42:0/3225453575 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c5557c9390 0x55c555b9c6c0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f710802a450 tx=0x7f7108001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44b7c35de6189e9c server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:12:47.638 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.083+0000 7f70f7fff640 1 -- 172.21.15.42:0/3225453575 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7108035070 con 0x55c5557c9390 2024-08-30T09:12:47.638 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.083+0000 7f70f7fff640 1 -- 172.21.15.42:0/3225453575 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7108057070 con 0x55c5557c9390 2024-08-30T09:12:47.638 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.083+0000 7f70f7fff640 1 -- 172.21.15.42:0/3225453575 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71080055c0 con 0x55c5557c9390 2024-08-30T09:12:47.638 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.083+0000 7f711ad0d740 1 -- 172.21.15.42:0/3225453575 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c5557c9390 msgr2=0x55c555b9c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:12:47.638 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.084+0000 7f711ad0d740 1 --2- 172.21.15.42:0/3225453575 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c5557c9390 0x55c555b9c6c0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f710802a450 tx=0x7f7108001e50 comp rx=0 tx=0).stop 2024-08-30T09:12:47.638 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.084+0000 7f711ad0d740 1 -- 172.21.15.42:0/3225453575 shutdown_connections 2024-08-30T09:12:47.638 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.084+0000 7f711ad0d740 1 --2- 172.21.15.42:0/3225453575 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c5557ea070 0x55c555ba6c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:47.638 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.084+0000 7f711ad0d740 1 --2- 172.21.15.42:0/3225453575 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c5557c9390 0x55c555b9c6c0 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:47.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.084+0000 7f711ad0d740 1 --2- 172.21.15.42:0/3225453575 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c5557c11b0 0x55c555b90cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:47.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.084+0000 7f711ad0d740 1 -- 172.21.15.42:0/3225453575 >> 172.21.15.42:0/3225453575 conn(0x55c5557de460 msgr2=0x55c555b9bfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:12:47.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.084+0000 7f711ad0d740 1 -- 172.21.15.42:0/3225453575 shutdown_connections 2024-08-30T09:12:47.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.084+0000 7f711ad0d740 1 -- 172.21.15.42:0/3225453575 wait complete. 2024-08-30T09:12:47.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.084+0000 7f711ad0d740 1 -- start start 2024-08-30T09:12:47.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.084+0000 7f711ad0d740 1 Processor -- start 2024-08-30T09:12:47.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.085+0000 7f711ad0d740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c5557c11b0 0x55c555ba14c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:12:47.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.085+0000 7f711ad0d740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c5557c9390 0x55c555ba1a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:12:47.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.085+0000 7f711ad0d740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c5557ea070 0x55c555c16770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:12:47.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.085+0000 7f711ad0d740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f710c1aeec0 con 0x55c5557c11b0 2024-08-30T09:12:47.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.085+0000 7f711ad0d740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f710c1b0330 con 0x55c5557c9390 2024-08-30T09:12:47.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.085+0000 7f711ad0d740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f710c1af590 con 0x55c5557ea070 2024-08-30T09:12:47.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.085+0000 7f711ad0d740 10 client.0 _ll_register_callbacks cb 0x7f7108009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:12:47.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.085+0000 7f7111d33640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c5557c11b0 0x55c555ba14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:12:47.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.085+0000 7f7111d33640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c5557c11b0 0x55c555ba14c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37738/0 (socket says 172.21.15.42:37738) 2024-08-30T09:12:47.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.085+0000 7f7111d33640 1 -- 172.21.15.42:0/3213023819 learned_addr learned my addr 172.21.15.42:0/3213023819 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:12:47.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.085+0000 7f7112534640 1 --2- 172.21.15.42:0/3213023819 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c5557c9390 0x55c555ba1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:12:47.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.085+0000 7f7111d33640 1 -- 172.21.15.42:0/3213023819 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c5557ea070 msgr2=0x55c555c16770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:12:47.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.085+0000 7f7111d33640 1 --2- 172.21.15.42:0/3213023819 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c5557ea070 0x55c555c16770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:47.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.085+0000 7f7111d33640 1 -- 172.21.15.42:0/3213023819 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c5557c9390 msgr2=0x55c555ba1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:12:47.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.085+0000 7f7111d33640 1 --2- 172.21.15.42:0/3213023819 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c5557c9390 0x55c555ba1a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:47.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.085+0000 7f7111d33640 1 -- 172.21.15.42:0/3213023819 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c555c16d00 con 0x55c5557c11b0 2024-08-30T09:12:47.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.085+0000 7f7111d33640 1 --2- 172.21.15.42:0/3213023819 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c5557c11b0 0x55c555ba14c0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f70fc014010 tx=0x7f70fc01f760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57b08cfd312fda53 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:12:47.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.085+0000 7f70f5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-08-30T09:12:47.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.085+0000 7f70f5ffb640 1 -- 172.21.15.42:0/3213023819 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70fc0099f0 con 0x55c5557c11b0 2024-08-30T09:12:47.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.085+0000 7f70f5ffb640 1 -- 172.21.15.42:0/3213023819 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f70fc02df90 con 0x55c5557c11b0 2024-08-30T09:12:47.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.085+0000 7f711ad0d740 1 -- 172.21.15.42:0/3213023819 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7108057070 con 0x55c5557c11b0 2024-08-30T09:12:47.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.085+0000 7f70f5ffb640 1 -- 172.21.15.42:0/3213023819 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f70fc03a050 con 0x55c5557c11b0 2024-08-30T09:12:47.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.086+0000 7f711ad0d740 10 client.17442 fetch_fsmap learned FSMap version 43 2024-08-30T09:12:47.641 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.086+0000 7f711ad0d740 1 -- 172.21.15.42:0/3213023819 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55c555c17f00 con 0x55c5557c11b0 2024-08-30T09:12:47.641 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.086+0000 7f70f5ffb640 1 -- 172.21.15.42:0/3213023819 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7108057070 con 0x55c5557c11b0 2024-08-30T09:12:47.641 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.086+0000 7f70f5ffb640 1 -- 172.21.15.42:0/3213023819 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f70fc02b030 con 0x55c5557c11b0 2024-08-30T09:12:47.641 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.086+0000 7f711ad0d740 10 client.17442 fetch_fsmap finished waiting for FSMap version 43 2024-08-30T09:12:47.641 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.086+0000 7f711ad0d740 10 client.17442 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-30T09:12:47.641 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.086+0000 7f711ad0d740 20 client.17442 populate_metadata read hostname 'smithi042' 2024-08-30T09:12:47.641 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.086+0000 7f711ad0d740 1 --2- 172.21.15.42:0/3213023819 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55c55577dda0 0x55c555c1a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:12:47.641 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.086+0000 7f711ad0d740 4 client.17442 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-30T09:12:47.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.086+0000 7f711ad0d740 1 -- 172.21.15.42:0/3213023819 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c555b04020 con 0x55c55577dda0 2024-08-30T09:12:47.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.086+0000 7f7112534640 1 --2- 172.21.15.42:0/3213023819 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55c55577dda0 0x55c555c1a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:12:47.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.087+0000 7f7112534640 1 --2- 172.21.15.42:0/3213023819 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55c55577dda0 0x55c555c1a3e0 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a439b3faffca2d33 server_cookie=40fbeb5bb2446c8 in_seq=0 out_seq=0 2024-08-30T09:12:47.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.087+0000 7f70f5ffb640 10 client.17442 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:12:47.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.087+0000 7f70f5ffb640 1 -- 172.21.15.42:0/3213023819 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+4546 (crc 0 0 0) 0x55c555b04020 con 0x55c55577dda0 2024-08-30T09:12:47.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.087+0000 7f70f5ffb640 10 client.17442 handle_command_reply: tid=0 2024-08-30T09:12:47.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.087+0000 7f711ad0d740 1 client.17442 shutdown 2024-08-30T09:12:47.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.087+0000 7f711ad0d740 1 -- 172.21.15.42:0/3213023819 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c5557c11b0 msgr2=0x55c555ba14c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:12:47.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.087+0000 7f711ad0d740 1 --2- 172.21.15.42:0/3213023819 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c5557c11b0 0x55c555ba14c0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f70fc014010 tx=0x7f70fc01f760 comp rx=0 tx=0).stop 2024-08-30T09:12:47.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.087+0000 7f711ad0d740 1 -- 172.21.15.42:0/3213023819 shutdown_connections 2024-08-30T09:12:47.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.087+0000 7f711ad0d740 1 --2- 172.21.15.42:0/3213023819 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55c55577dda0 0x55c555c1a3e0 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:47.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.087+0000 7f70f5ffb640 0 client.17442 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:12:47.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.087+0000 7f711ad0d740 1 --2- 172.21.15.42:0/3213023819 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c5557ea070 0x55c555c16770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:47.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.087+0000 7f711ad0d740 1 --2- 172.21.15.42:0/3213023819 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c5557c9390 0x55c555ba1a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:47.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.087+0000 7f711ad0d740 1 --2- 172.21.15.42:0/3213023819 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c5557c11b0 0x55c555ba14c0 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:47.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.087+0000 7f711ad0d740 1 -- 172.21.15.42:0/3213023819 >> 172.21.15.42:0/3213023819 conn(0x55c5557de460 msgr2=0x55c555b90a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:12:47.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.088+0000 7f711ad0d740 1 -- 172.21.15.42:0/3213023819 shutdown_connections 2024-08-30T09:12:47.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.088+0000 7f711ad0d740 1 -- 172.21.15.42:0/3213023819 wait complete. 2024-08-30T09:12:47.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.088+0000 7f711ad0d740 20 client.17442 trim_cache size 0 max 16384 2024-08-30T09:12:47.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.088+0000 7f7114798640 1 -- 172.21.15.42:0/2066095011 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f70f80b4bb0 msgr2=0x7f70f80b70c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:12:47.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.088+0000 7f7114798640 1 --2- 172.21.15.42:0/2066095011 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f70f80b4bb0 0x7f70f80b70c0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f71040abac0 tx=0x7f71040dbfc0 comp rx=0 tx=0).stop 2024-08-30T09:12:47.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.088+0000 7f7114798640 1 -- 172.21.15.42:0/2066095011 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f710c109b90 msgr2=0x7f710c19fa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:12:47.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.088+0000 7f7114798640 1 --2- 172.21.15.42:0/2066095011 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f710c109b90 0x7f710c19fa80 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f710400eaa0 tx=0x7f710400ef60 comp rx=0 tx=0).stop 2024-08-30T09:12:47.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.088+0000 7f7114798640 1 -- 172.21.15.42:0/2066095011 shutdown_connections 2024-08-30T09:12:47.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.088+0000 7f7114798640 1 --2- 172.21.15.42:0/2066095011 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f70f80b4bb0 0x7f70f80b70c0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:47.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.088+0000 7f7114798640 1 --2- 172.21.15.42:0/2066095011 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f710c109b90 0x7f710c19fa80 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:47.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.088+0000 7f7114798640 1 --2- 172.21.15.42:0/2066095011 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f710c109240 0x7f710c19a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:47.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.088+0000 7f7114798640 1 --2- 172.21.15.42:0/2066095011 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f710c103900 0x7f710c19a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:47.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.088+0000 7f7114798640 1 -- 172.21.15.42:0/2066095011 >> 172.21.15.42:0/2066095011 conn(0x7f710c07b5f0 msgr2=0x7f710c1072d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:12:47.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.088+0000 7f7114798640 1 -- 172.21.15.42:0/2066095011 shutdown_connections 2024-08-30T09:12:47.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:12:47.088+0000 7f7114798640 1 -- 172.21.15.42:0/2066095011 wait complete. 2024-08-30T09:12:49.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:12:48 smithi167 ceph-mon[22063]: pgmap v734: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 4.4 MiB/s wr, 413 op/s 2024-08-30T09:12:49.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:12:48 smithi042 ceph-mon[18652]: pgmap v734: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 4.4 MiB/s wr, 413 op/s 2024-08-30T09:12:49.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:12:48 smithi190 ceph-mon[22001]: pgmap v734: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 4.4 MiB/s wr, 413 op/s 2024-08-30T09:12:50.086 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:50.084+0000 7f358bd57640 1 -- 172.21.15.42:0/511856213 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3584102340 msgr2=0x7f35841027b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:12:50.087 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:50.084+0000 7f358bd57640 1 --2- 172.21.15.42:0/511856213 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3584102340 0x7f35841027b0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f3574009900 tx=0x7f357402a410 comp rx=0 tx=0).stop 2024-08-30T09:12:50.087 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:50.084+0000 7f358bd57640 1 -- 172.21.15.42:0/511856213 shutdown_connections 2024-08-30T09:12:50.087 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:50.084+0000 7f358bd57640 1 --2- 172.21.15.42:0/511856213 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3584103d60 0x7f358410a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:50.087 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:50.084+0000 7f358bd57640 1 --2- 172.21.15.42:0/511856213 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f35841033c0 0x7f35841037d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:50.087 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:50.084+0000 7f358bd57640 1 --2- 172.21.15.42:0/511856213 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3584102340 0x7f35841027b0 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:50.087 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:50.084+0000 7f358bd57640 1 -- 172.21.15.42:0/511856213 >> 172.21.15.42:0/511856213 conn(0x7f35840fdc80 msgr2=0x7f3584100130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:12:50.087 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:50.084+0000 7f358bd57640 1 -- 172.21.15.42:0/511856213 shutdown_connections 2024-08-30T09:12:50.087 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:50.084+0000 7f358bd57640 1 -- 172.21.15.42:0/511856213 wait complete. 2024-08-30T09:12:50.090 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:50.085+0000 7f358bd57640 1 Processor -- start 2024-08-30T09:12:50.090 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:50.085+0000 7f358bd57640 1 -- start start 2024-08-30T09:12:50.090 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:50.086+0000 7f358bd57640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3584102340 0x7f3584198c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:12:50.090 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:50.086+0000 7f358bd57640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f35841033c0 0x7f3584199290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:12:50.090 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:50.086+0000 7f358bd57640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3584103d60 0x7f3584199e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:12:50.090 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:50.086+0000 7f358bd57640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f358410dfc0 con 0x7f35841033c0 2024-08-30T09:12:50.090 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:50.086+0000 7f358bd57640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f358410de40 con 0x7f3584103d60 2024-08-30T09:12:50.090 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:50.086+0000 7f358bd57640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f358410e140 con 0x7f3584102340 2024-08-30T09:12:50.090 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:50.086+0000 7f35892f2640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f35841033c0 0x7f3584199290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:12:50.090 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:50.086+0000 7f35892f2640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f35841033c0 0x7f3584199290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:59552/0 (socket says 172.21.15.42:59552) 2024-08-30T09:12:50.090 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:50.086+0000 7f35892f2640 1 -- 172.21.15.42:0/446166839 learned_addr learned my addr 172.21.15.42:0/446166839 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:12:50.091 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:50.086+0000 7f358a2f4640 1 --2- 172.21.15.42:0/446166839 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3584103d60 0x7f3584199e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:12:50.091 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:50.086+0000 7f35892f2640 1 -- 172.21.15.42:0/446166839 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3584102340 msgr2=0x7f3584198c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:12:50.091 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:50.086+0000 7f35892f2640 1 --2- 172.21.15.42:0/446166839 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3584102340 0x7f3584198c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:50.091 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:50.086+0000 7f35892f2640 1 -- 172.21.15.42:0/446166839 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3584103d60 msgr2=0x7f3584199e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:12:50.091 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:50.086+0000 7f35892f2640 1 --2- 172.21.15.42:0/446166839 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3584103d60 0x7f3584199e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:50.091 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:50.086+0000 7f35892f2640 1 -- 172.21.15.42:0/446166839 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f358419a5d0 con 0x7f35841033c0 2024-08-30T09:12:50.091 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:50.086+0000 7f358a2f4640 1 --2- 172.21.15.42:0/446166839 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3584103d60 0x7f3584199e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-30T09:12:50.091 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:50.086+0000 7f35892f2640 1 --2- 172.21.15.42:0/446166839 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f35841033c0 0x7f3584199290 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f357c00ebc0 tx=0x7f357c00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:12:50.091 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:50.087+0000 7f357affd640 1 -- 172.21.15.42:0/446166839 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f357c010630 con 0x7f35841033c0 2024-08-30T09:12:50.091 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:50.087+0000 7f358bd57640 1 -- 172.21.15.42:0/446166839 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f358419a8c0 con 0x7f35841033c0 2024-08-30T09:12:50.091 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:50.087+0000 7f357affd640 1 -- 172.21.15.42:0/446166839 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f357c010c80 con 0x7f35841033c0 2024-08-30T09:12:50.091 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:50.087+0000 7f357affd640 1 -- 172.21.15.42:0/446166839 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f357c018b60 con 0x7f35841033c0 2024-08-30T09:12:50.092 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:50.087+0000 7f358bd57640 1 -- 172.21.15.42:0/446166839 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f358406ab60 con 0x7f35841033c0 2024-08-30T09:12:50.093 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:50.090+0000 7f3592cc7740 1 Processor -- start 2024-08-30T09:12:50.093 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:50.090+0000 7f357affd640 1 -- 172.21.15.42:0/446166839 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7f357c018d00 con 0x7f35841033c0 2024-08-30T09:12:50.094 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:50.090+0000 7f3592cc7740 1 -- start start 2024-08-30T09:12:50.094 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:50.090+0000 7f3592cc7740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562c8e5011b0 0x562c8e8a7180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:12:50.094 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:50.090+0000 7f3592cc7740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562c8e4d82f0 0x562c8e8a77f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:12:50.094 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:50.090+0000 7f3592cc7740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562c8e4e04d0 0x562c8e8a8340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:12:50.094 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:50.090+0000 7f3592cc7740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x562c8e81a000 con 0x562c8e4d82f0 2024-08-30T09:12:50.094 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:50.090+0000 7f3592cc7740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x562c8e809590 con 0x562c8e4e04d0 2024-08-30T09:12:50.094 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:50.090+0000 7f3592cc7740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x562c8e80d540 con 0x562c8e5011b0 2024-08-30T09:12:50.094 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:50.090+0000 7f3589af3640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562c8e4d82f0 0x562c8e8a77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:12:50.095 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:50.090+0000 7f3589af3640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562c8e4d82f0 0x562c8e8a77f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:59562/0 (socket says 172.21.15.42:59562) 2024-08-30T09:12:50.095 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:50.090+0000 7f3589af3640 1 -- 172.21.15.42:0/2951515155 learned_addr learned my addr 172.21.15.42:0/2951515155 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:12:50.095 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:50.091+0000 7f358a2f4640 1 --2- 172.21.15.42:0/2951515155 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562c8e5011b0 0x562c8e8a7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:12:50.095 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:50.091+0000 7f35892f2640 1 --2- 172.21.15.42:0/2951515155 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562c8e4e04d0 0x562c8e8a8340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:12:50.095 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:50.091+0000 7f357affd640 1 --2- 172.21.15.42:0/446166839 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f35700b6dc0 0x7f35700b92d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:12:50.095 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:50.091+0000 7f3589af3640 1 -- 172.21.15.42:0/2951515155 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562c8e5011b0 msgr2=0x562c8e8a7180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:12:50.095 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:50.091+0000 7f3589af3640 1 --2- 172.21.15.42:0/2951515155 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562c8e5011b0 0x562c8e8a7180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:50.095 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:50.091+0000 7f3589af3640 1 -- 172.21.15.42:0/2951515155 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562c8e4e04d0 msgr2=0x562c8e8a8340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:12:50.095 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:50.091+0000 7f3589af3640 1 --2- 172.21.15.42:0/2951515155 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562c8e4e04d0 0x562c8e8a8340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:50.095 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:50.091+0000 7f3589af3640 1 -- 172.21.15.42:0/2951515155 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c8e56c790 con 0x562c8e4d82f0 2024-08-30T09:12:50.095 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:50.091+0000 7f358a2f4640 1 --2- 172.21.15.42:0/2951515155 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562c8e5011b0 0x562c8e8a7180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-30T09:12:50.095 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:50.091+0000 7f358a2f4640 1 --2- 172.21.15.42:0/446166839 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f35700b6dc0 0x7f35700b92d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:12:50.096 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:50.091+0000 7f357affd640 1 -- 172.21.15.42:0/446166839 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f357c0e5bc0 con 0x7f35841033c0 2024-08-30T09:12:50.096 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:50.091+0000 7f3589af3640 1 --2- 172.21.15.42:0/2951515155 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562c8e4d82f0 0x562c8e8a77f0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f3574031500 tx=0x7f3574000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a1a9ada48eeff80 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:12:50.096 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:50.091+0000 7f356ffff640 1 -- 172.21.15.42:0/2951515155 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f357403e070 con 0x562c8e4d82f0 2024-08-30T09:12:50.096 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:50.092+0000 7f356ffff640 1 -- 172.21.15.42:0/2951515155 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3574004050 con 0x562c8e4d82f0 2024-08-30T09:12:50.096 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:50.092+0000 7f356ffff640 1 -- 172.21.15.42:0/2951515155 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3574035040 con 0x562c8e4d82f0 2024-08-30T09:12:50.099 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:50.092+0000 7f3592cc7740 1 -- 172.21.15.42:0/2951515155 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562c8e4d82f0 msgr2=0x562c8e8a77f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:12:50.099 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:50.092+0000 7f3592cc7740 1 --2- 172.21.15.42:0/2951515155 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562c8e4d82f0 0x562c8e8a77f0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f3574031500 tx=0x7f3574000bc0 comp rx=0 tx=0).stop 2024-08-30T09:12:50.099 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:50.092+0000 7f358a2f4640 1 --2- 172.21.15.42:0/446166839 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f35700b6dc0 0x7f35700b92d0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f35800051e0 tx=0x7f3580014040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:12:50.100 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:50.092+0000 7f3592cc7740 1 -- 172.21.15.42:0/2951515155 shutdown_connections 2024-08-30T09:12:50.100 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:50.092+0000 7f3592cc7740 1 --2- 172.21.15.42:0/2951515155 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562c8e4e04d0 0x562c8e8a8340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:50.100 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:50.092+0000 7f3592cc7740 1 --2- 172.21.15.42:0/2951515155 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562c8e4d82f0 0x562c8e8a77f0 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:50.100 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:50.092+0000 7f3592cc7740 1 --2- 172.21.15.42:0/2951515155 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562c8e5011b0 0x562c8e8a7180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:50.100 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:50.092+0000 7f3592cc7740 1 -- 172.21.15.42:0/2951515155 >> 172.21.15.42:0/2951515155 conn(0x562c8e4f5300 msgr2=0x562c8e8a2c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:12:50.100 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:50.093+0000 7f3592cc7740 1 -- 172.21.15.42:0/2951515155 shutdown_connections 2024-08-30T09:12:50.100 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:50.093+0000 7f3592cc7740 1 -- 172.21.15.42:0/2951515155 wait complete. 2024-08-30T09:12:50.100 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:50.093+0000 7f3592cc7740 1 -- start start 2024-08-30T09:12:50.100 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:50.093+0000 7f3592cc7740 1 Processor -- start 2024-08-30T09:12:50.100 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:50.093+0000 7f3592cc7740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562c8e4d82f0 0x562c8e8b0b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:12:50.100 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:50.093+0000 7f3592cc7740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562c8e4e04d0 0x562c8e8b1230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:12:50.101 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:50.093+0000 7f3592cc7740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562c8e5011b0 0x562c8e8b1da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:12:50.101 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:50.094+0000 7f3592cc7740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x562c8e742ab0 con 0x562c8e4e04d0 2024-08-30T09:12:50.101 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:50.094+0000 7f3592cc7740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x562c8e790680 con 0x562c8e5011b0 2024-08-30T09:12:50.101 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:50.094+0000 7f3592cc7740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x562c8e8a8e00 con 0x562c8e4d82f0 2024-08-30T09:12:50.101 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:50.094+0000 7f35892f2640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562c8e4e04d0 0x562c8e8b1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:12:50.101 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:50.094+0000 7f35892f2640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562c8e4e04d0 0x562c8e8b1230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:59578/0 (socket says 172.21.15.42:59578) 2024-08-30T09:12:50.101 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:50.094+0000 7f35892f2640 1 -- 172.21.15.42:0/2364906099 learned_addr learned my addr 172.21.15.42:0/2364906099 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:12:50.101 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:50.094+0000 7f3592cc7740 10 client.0 _ll_register_callbacks cb 0x562c8e518a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:12:50.101 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:50.094+0000 7f358a2f4640 1 --2- 172.21.15.42:0/2364906099 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562c8e5011b0 0x562c8e8b1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:12:50.101 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:50.094+0000 7f3589af3640 1 --2- 172.21.15.42:0/2364906099 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562c8e4d82f0 0x562c8e8b0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:12:50.101 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:50.094+0000 7f35892f2640 1 -- 172.21.15.42:0/2364906099 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562c8e4d82f0 msgr2=0x562c8e8b0b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:12:50.101 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:50.094+0000 7f35892f2640 1 --2- 172.21.15.42:0/2364906099 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562c8e4d82f0 0x562c8e8b0b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:50.101 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:50.094+0000 7f35892f2640 1 -- 172.21.15.42:0/2364906099 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562c8e5011b0 msgr2=0x562c8e8b1da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:12:50.102 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:50.094+0000 7f35892f2640 1 --2- 172.21.15.42:0/2364906099 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562c8e5011b0 0x562c8e8b1da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:50.102 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:50.094+0000 7f35892f2640 1 -- 172.21.15.42:0/2364906099 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c8e5e5620 con 0x562c8e4e04d0 2024-08-30T09:12:50.102 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:50.094+0000 7f3589af3640 1 --2- 172.21.15.42:0/2364906099 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562c8e4d82f0 0x562c8e8b0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-30T09:12:50.102 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:50.094+0000 7f35892f2640 1 --2- 172.21.15.42:0/2364906099 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562c8e4e04d0 0x562c8e8b1230 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f357c00cba0 tx=0x7f357c009200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bec492063b8e5a40 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:12:50.102 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:50.094+0000 7f356d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-08-30T09:12:50.102 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:50.094+0000 7f356d7fa640 1 -- 172.21.15.42:0/2364906099 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f357c0ab9d0 con 0x562c8e4e04d0 2024-08-30T09:12:50.102 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:50.094+0000 7f356d7fa640 1 -- 172.21.15.42:0/2364906099 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f357c0109d0 con 0x562c8e4e04d0 2024-08-30T09:12:50.102 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:50.094+0000 7f3592cc7740 1 -- 172.21.15.42:0/2364906099 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3574004050 con 0x562c8e4e04d0 2024-08-30T09:12:50.102 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:50.094+0000 7f356d7fa640 1 -- 172.21.15.42:0/2364906099 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f357c018420 con 0x562c8e4e04d0 2024-08-30T09:12:50.102 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:50.095+0000 7f3592cc7740 10 client.17454 fetch_fsmap learned FSMap version 43 2024-08-30T09:12:50.102 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:50.095+0000 7f3592cc7740 1 -- 172.21.15.42:0/2364906099 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x562c8e8badb0 con 0x562c8e4e04d0 2024-08-30T09:12:50.102 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:50.095+0000 7f356d7fa640 1 -- 172.21.15.42:0/2364906099 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3574004050 con 0x562c8e4e04d0 2024-08-30T09:12:50.103 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:50.095+0000 7f356d7fa640 1 -- 172.21.15.42:0/2364906099 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f357c0a7bd0 con 0x562c8e4e04d0 2024-08-30T09:12:50.103 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:50.095+0000 7f3592cc7740 10 client.17454 fetch_fsmap finished waiting for FSMap version 43 2024-08-30T09:12:50.103 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:50.095+0000 7f3592cc7740 10 client.17454 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-30T09:12:50.103 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:50.095+0000 7f3592cc7740 20 client.17454 populate_metadata read hostname 'smithi042' 2024-08-30T09:12:50.103 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:50.095+0000 7f3592cc7740 1 --2- 172.21.15.42:0/2364906099 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x562c8e4bfd80 0x562c8e8be4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:12:50.103 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:50.095+0000 7f3592cc7740 4 client.17454 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-30T09:12:50.103 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:50.095+0000 7f3592cc7740 1 -- 172.21.15.42:0/2364906099 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562c8e81bf50 con 0x562c8e4bfd80 2024-08-30T09:12:50.104 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:50.098+0000 7f3589af3640 1 --2- 172.21.15.42:0/2364906099 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x562c8e4bfd80 0x562c8e8be4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:12:50.104 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:50.099+0000 7f3589af3640 1 --2- 172.21.15.42:0/2364906099 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x562c8e4bfd80 0x562c8e8be4d0 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=42f5c6a99fb574c7 server_cookie=dca69346858718b3 in_seq=0 out_seq=0 2024-08-30T09:12:50.104 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:50.099+0000 7f356d7fa640 10 client.17454 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:12:50.105 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:50.100+0000 7f356d7fa640 1 -- 172.21.15.42:0/2364906099 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x562c8e81bf50 con 0x562c8e4bfd80 2024-08-30T09:12:50.105 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:50.100+0000 7f356d7fa640 10 client.17454 handle_command_reply: tid=0 2024-08-30T09:12:50.105 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:50.100+0000 7f3592cc7740 1 client.17454 shutdown 2024-08-30T09:12:50.105 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:50.100+0000 7f3592cc7740 1 -- 172.21.15.42:0/2364906099 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562c8e4e04d0 msgr2=0x562c8e8b1230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:12:50.105 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:50.100+0000 7f3592cc7740 1 --2- 172.21.15.42:0/2364906099 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562c8e4e04d0 0x562c8e8b1230 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f357c00cba0 tx=0x7f357c009200 comp rx=0 tx=0).stop 2024-08-30T09:12:50.105 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:50.100+0000 7f3592cc7740 1 -- 172.21.15.42:0/2364906099 shutdown_connections 2024-08-30T09:12:50.105 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:50.100+0000 7f3592cc7740 1 --2- 172.21.15.42:0/2364906099 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x562c8e4bfd80 0x562c8e8be4d0 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:50.105 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:50.100+0000 7f3592cc7740 1 --2- 172.21.15.42:0/2364906099 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562c8e5011b0 0x562c8e8b1da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:50.105 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:50.100+0000 7f3592cc7740 1 --2- 172.21.15.42:0/2364906099 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562c8e4e04d0 0x562c8e8b1230 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:50.105 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:50.100+0000 7f356d7fa640 0 client.17454 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:12:50.105 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:50.100+0000 7f3592cc7740 1 --2- 172.21.15.42:0/2364906099 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562c8e4d82f0 0x562c8e8b0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:50.105 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:50.100+0000 7f3592cc7740 1 -- 172.21.15.42:0/2364906099 >> 172.21.15.42:0/2364906099 conn(0x562c8e4f5300 msgr2=0x562c8e8a1170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:12:50.106 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:50.100+0000 7f3592cc7740 1 -- 172.21.15.42:0/2364906099 shutdown_connections 2024-08-30T09:12:50.106 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:50.100+0000 7f3592cc7740 1 -- 172.21.15.42:0/2364906099 wait complete. 2024-08-30T09:12:50.106 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:50.100+0000 7f3592cc7740 20 client.17454 trim_cache size 0 max 16384 2024-08-30T09:12:50.118 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:50.115+0000 7f3592cc7740 1 Processor -- start 2024-08-30T09:12:50.118 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:50.115+0000 7f3592cc7740 1 -- start start 2024-08-30T09:12:50.119 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:50.115+0000 7f3592cc7740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562c8e4d82f0 0x562c8e8a3150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:12:50.119 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:50.115+0000 7f3592cc7740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562c8e4e04d0 0x562c8e8a3690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:12:50.119 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:50.115+0000 7f3592cc7740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562c8e5011b0 0x562c8e8a3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:12:50.119 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:50.115+0000 7f3592cc7740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f35841ac990 con 0x562c8e4e04d0 2024-08-30T09:12:50.119 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:50.115+0000 7f3592cc7740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f35841aa260 con 0x562c8e5011b0 2024-08-30T09:12:50.119 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:50.115+0000 7f3592cc7740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f35841a80b0 con 0x562c8e4d82f0 2024-08-30T09:12:50.119 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:50.116+0000 7f35892f2640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562c8e4e04d0 0x562c8e8a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:12:50.119 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:50.116+0000 7f35892f2640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562c8e4e04d0 0x562c8e8a3690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:59580/0 (socket says 172.21.15.42:59580) 2024-08-30T09:12:50.119 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:50.116+0000 7f35892f2640 1 -- 172.21.15.42:0/1254636411 learned_addr learned my addr 172.21.15.42:0/1254636411 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:12:50.119 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:50.116+0000 7f358a2f4640 1 --2- 172.21.15.42:0/1254636411 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562c8e5011b0 0x562c8e8a3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:12:50.119 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:50.116+0000 7f35892f2640 1 -- 172.21.15.42:0/1254636411 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562c8e4d82f0 msgr2=0x562c8e8a3150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:12:50.119 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:50.116+0000 7f3589af3640 1 --2- 172.21.15.42:0/1254636411 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562c8e4d82f0 0x562c8e8a3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:12:50.119 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:50.116+0000 7f35892f2640 1 --2- 172.21.15.42:0/1254636411 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562c8e4d82f0 0x562c8e8a3150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:50.120 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:50.116+0000 7f35892f2640 1 -- 172.21.15.42:0/1254636411 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562c8e5011b0 msgr2=0x562c8e8a3bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:12:50.120 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:50.116+0000 7f35892f2640 1 --2- 172.21.15.42:0/1254636411 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562c8e5011b0 0x562c8e8a3bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:50.120 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:50.116+0000 7f35892f2640 1 -- 172.21.15.42:0/1254636411 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c8e8a7fc0 con 0x562c8e4e04d0 2024-08-30T09:12:50.120 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:50.116+0000 7f3589af3640 1 --2- 172.21.15.42:0/1254636411 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562c8e4d82f0 0x562c8e8a3150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-30T09:12:50.120 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:50.116+0000 7f35892f2640 1 --2- 172.21.15.42:0/1254636411 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562c8e4e04d0 0x562c8e8a3690 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f357c00eb90 tx=0x7f357c000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5f1492cd03499f3 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:12:50.120 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:50.116+0000 7f356f7fe640 1 -- 172.21.15.42:0/1254636411 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f357c0abd70 con 0x562c8e4e04d0 2024-08-30T09:12:50.120 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:50.116+0000 7f356f7fe640 1 -- 172.21.15.42:0/1254636411 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f357c0ac960 con 0x562c8e4e04d0 2024-08-30T09:12:50.120 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:50.116+0000 7f356f7fe640 1 -- 172.21.15.42:0/1254636411 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f357c0c1e40 con 0x562c8e4e04d0 2024-08-30T09:12:50.124 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:50.117+0000 7f3592cc7740 1 -- 172.21.15.42:0/1254636411 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562c8e4e04d0 msgr2=0x562c8e8a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:12:50.124 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:50.117+0000 7f3592cc7740 1 --2- 172.21.15.42:0/1254636411 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562c8e4e04d0 0x562c8e8a3690 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f357c00eb90 tx=0x7f357c000f40 comp rx=0 tx=0).stop 2024-08-30T09:12:50.124 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:50.117+0000 7f3592cc7740 1 -- 172.21.15.42:0/1254636411 shutdown_connections 2024-08-30T09:12:50.124 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:50.117+0000 7f3592cc7740 1 --2- 172.21.15.42:0/1254636411 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562c8e5011b0 0x562c8e8a3bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:50.124 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:50.117+0000 7f3592cc7740 1 --2- 172.21.15.42:0/1254636411 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562c8e4e04d0 0x562c8e8a3690 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:50.124 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:50.117+0000 7f3592cc7740 1 --2- 172.21.15.42:0/1254636411 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562c8e4d82f0 0x562c8e8a3150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:50.124 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:50.117+0000 7f3592cc7740 1 -- 172.21.15.42:0/1254636411 >> 172.21.15.42:0/1254636411 conn(0x562c8e4f5300 msgr2=0x562c8e8abbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:12:50.124 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:50.117+0000 7f3592cc7740 1 -- 172.21.15.42:0/1254636411 shutdown_connections 2024-08-30T09:12:50.124 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:50.117+0000 7f3592cc7740 1 -- 172.21.15.42:0/1254636411 wait complete. 2024-08-30T09:12:50.125 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:50.117+0000 7f3592cc7740 1 -- start start 2024-08-30T09:12:50.125 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:50.118+0000 7f3592cc7740 1 Processor -- start 2024-08-30T09:12:50.125 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:50.118+0000 7f3592cc7740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562c8e4d82f0 0x562c8e8b8490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:12:50.125 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:50.118+0000 7f3592cc7740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562c8e4e04d0 0x562c8e8b7150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:12:50.125 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:50.118+0000 7f3592cc7740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562c8e5011b0 0x562c8e8b76f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:12:50.125 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:50.118+0000 7f3592cc7740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f35841a8720 con 0x562c8e5011b0 2024-08-30T09:12:50.125 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:50.118+0000 7f3592cc7740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f35841a9b90 con 0x562c8e4e04d0 2024-08-30T09:12:50.125 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:50.118+0000 7f3592cc7740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f35841a8df0 con 0x562c8e4d82f0 2024-08-30T09:12:50.125 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:50.118+0000 7f3592cc7740 10 client.0 _ll_register_callbacks cb 0x7f357c0ac0c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:12:50.125 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:50.118+0000 7f358a2f4640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562c8e5011b0 0x562c8e8b76f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:12:50.125 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:50.118+0000 7f358a2f4640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562c8e5011b0 0x562c8e8b76f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:59594/0 (socket says 172.21.15.42:59594) 2024-08-30T09:12:50.125 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:50.118+0000 7f358a2f4640 1 -- 172.21.15.42:0/44697165 learned_addr learned my addr 172.21.15.42:0/44697165 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:12:50.125 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:50.118+0000 7f35892f2640 1 --2- 172.21.15.42:0/44697165 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562c8e4e04d0 0x562c8e8b7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:12:50.126 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:50.118+0000 7f3589af3640 1 --2- 172.21.15.42:0/44697165 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562c8e4d82f0 0x562c8e8b8490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:12:50.126 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:50.118+0000 7f358a2f4640 1 -- 172.21.15.42:0/44697165 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562c8e4d82f0 msgr2=0x562c8e8b8490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:12:50.126 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:50.118+0000 7f358a2f4640 1 --2- 172.21.15.42:0/44697165 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562c8e4d82f0 0x562c8e8b8490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:50.126 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:50.118+0000 7f358a2f4640 1 -- 172.21.15.42:0/44697165 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562c8e4e04d0 msgr2=0x562c8e8b7150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:12:50.126 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:50.118+0000 7f358a2f4640 1 --2- 172.21.15.42:0/44697165 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562c8e4e04d0 0x562c8e8b7150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:50.126 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:50.118+0000 7f358a2f4640 1 -- 172.21.15.42:0/44697165 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c8e8b7e80 con 0x562c8e5011b0 2024-08-30T09:12:50.126 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:50.118+0000 7f35892f2640 1 --2- 172.21.15.42:0/44697165 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562c8e4e04d0 0x562c8e8b7150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-30T09:12:50.126 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:50.119+0000 7f358a2f4640 1 --2- 172.21.15.42:0/44697165 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562c8e5011b0 0x562c8e8b76f0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f358001be70 tx=0x7f3580005f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b55ffce08c30b4c server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:12:50.126 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:50.119+0000 7f356d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-08-30T09:12:50.126 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:50.119+0000 7f356d7fa640 1 -- 172.21.15.42:0/44697165 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35800287f0 con 0x562c8e5011b0 2024-08-30T09:12:50.126 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:50.119+0000 7f356d7fa640 1 -- 172.21.15.42:0/44697165 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f3580028990 con 0x562c8e5011b0 2024-08-30T09:12:50.126 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:50.119+0000 7f3592cc7740 1 -- 172.21.15.42:0/44697165 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f357c0ac960 con 0x562c8e5011b0 2024-08-30T09:12:50.126 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:50.119+0000 7f356d7fa640 1 -- 172.21.15.42:0/44697165 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f3580037050 con 0x562c8e5011b0 2024-08-30T09:12:50.127 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:50.119+0000 7f3592cc7740 10 client.17460 fetch_fsmap learned FSMap version 43 2024-08-30T09:12:50.127 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:50.119+0000 7f3592cc7740 1 -- 172.21.15.42:0/44697165 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x562c8e928e90 con 0x562c8e5011b0 2024-08-30T09:12:50.127 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:50.119+0000 7f356d7fa640 1 -- 172.21.15.42:0/44697165 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f357c0ac960 con 0x562c8e5011b0 2024-08-30T09:12:50.127 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:50.119+0000 7f356d7fa640 1 -- 172.21.15.42:0/44697165 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f3580029a90 con 0x562c8e5011b0 2024-08-30T09:12:50.127 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:50.119+0000 7f3592cc7740 10 client.17460 fetch_fsmap finished waiting for FSMap version 43 2024-08-30T09:12:50.127 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:50.119+0000 7f3592cc7740 10 client.17460 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-30T09:12:50.127 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:50.119+0000 7f3592cc7740 20 client.17460 populate_metadata read hostname 'smithi042' 2024-08-30T09:12:50.127 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:50.119+0000 7f3592cc7740 1 --2- 172.21.15.42:0/44697165 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x562c8e4951c0 0x562c8e92b930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:12:50.127 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:50.119+0000 7f3592cc7740 4 client.17460 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-30T09:12:50.127 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:50.119+0000 7f3592cc7740 1 -- 172.21.15.42:0/44697165 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562c8e8179f0 con 0x562c8e4951c0 2024-08-30T09:12:50.127 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:50.122+0000 7f3589af3640 1 --2- 172.21.15.42:0/44697165 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x562c8e4951c0 0x562c8e92b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:12:50.128 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:50.122+0000 7f3589af3640 1 --2- 172.21.15.42:0/44697165 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x562c8e4951c0 0x562c8e92b930 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b018c2d392e3b6b server_cookie=f5dba38e76330104 in_seq=0 out_seq=0 2024-08-30T09:12:50.128 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:50.122+0000 7f356d7fa640 10 client.17460 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:12:50.944 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:12:50 smithi042 ceph-mon[18652]: pgmap v735: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 6.5 MiB/s wr, 670 op/s 2024-08-30T09:12:50.953 INFO:teuthology.orchestra.run.smithi042.stderr:progress: 725.0 s, 300.2 tps, lat 279.756 ms stddev 365.344 2024-08-30T09:12:50.993 INFO:tasks.cephfs.filesystem:scrub status for tag:0cbf734e-f988-4b21-acac-57b55e3bde62 - {'path': '/', 'tag': '0cbf734e-f988-4b21-acac-57b55e3bde62', 'options': 'recursive,force'} 2024-08-30T09:12:51.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:12:50 smithi167 ceph-mon[22063]: pgmap v735: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 6.5 MiB/s wr, 670 op/s 2024-08-30T09:12:51.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:12:50 smithi190 ceph-mon[22001]: pgmap v735: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 6.5 MiB/s wr, 670 op/s 2024-08-30T09:12:51.478 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:51.475+0000 7f356d7fa640 1 -- 172.21.15.42:0/44697165 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562c8e8179f0 con 0x562c8e4951c0 2024-08-30T09:12:51.478 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:51.475+0000 7f356d7fa640 10 client.17460 handle_command_reply: tid=0 2024-08-30T09:12:51.478 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:51.476+0000 7f3592cc7740 1 client.17460 shutdown 2024-08-30T09:12:51.478 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:51.476+0000 7f3592cc7740 1 -- 172.21.15.42:0/44697165 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562c8e5011b0 msgr2=0x562c8e8b76f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:12:51.478 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:51.476+0000 7f3592cc7740 1 --2- 172.21.15.42:0/44697165 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562c8e5011b0 0x562c8e8b76f0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f358001be70 tx=0x7f3580005f50 comp rx=0 tx=0).stop 2024-08-30T09:12:51.479 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:51.476+0000 7f3592cc7740 1 -- 172.21.15.42:0/44697165 shutdown_connections 2024-08-30T09:12:51.479 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:51.476+0000 7f3592cc7740 1 --2- 172.21.15.42:0/44697165 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x562c8e4951c0 0x562c8e92b930 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:51.480 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:51.477+0000 7f356d7fa640 0 client.17460 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:12:51.480 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:51.477+0000 7f3592cc7740 1 --2- 172.21.15.42:0/44697165 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562c8e5011b0 0x562c8e8b76f0 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:51.480 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:51.477+0000 7f3592cc7740 1 --2- 172.21.15.42:0/44697165 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562c8e4e04d0 0x562c8e8b7150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:51.480 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:51.477+0000 7f3592cc7740 1 --2- 172.21.15.42:0/44697165 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562c8e4d82f0 0x562c8e8b8490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:51.480 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:51.477+0000 7f3592cc7740 1 -- 172.21.15.42:0/44697165 >> 172.21.15.42:0/44697165 conn(0x562c8e4f5300 msgr2=0x562c8e8af100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:12:51.480 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:51.477+0000 7f3592cc7740 1 -- 172.21.15.42:0/44697165 shutdown_connections 2024-08-30T09:12:51.480 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:51.477+0000 7f3592cc7740 1 -- 172.21.15.42:0/44697165 wait complete. 2024-08-30T09:12:51.480 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:51.477+0000 7f3592cc7740 20 client.17460 trim_cache size 0 max 16384 2024-08-30T09:12:51.481 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-08-30T09:12:51.481 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2024-08-30T09:12:51.481 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-08-30T09:12:51.481 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-08-30T09:12:51.481 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:51.478+0000 7f358bd57640 1 -- 172.21.15.42:0/446166839 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f35700b6dc0 msgr2=0x7f35700b92d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:12:51.481 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:51.478+0000 7f358bd57640 1 --2- 172.21.15.42:0/446166839 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f35700b6dc0 0x7f35700b92d0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f35800051e0 tx=0x7f3580014040 comp rx=0 tx=0).stop 2024-08-30T09:12:51.482 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:51.478+0000 7f358bd57640 1 -- 172.21.15.42:0/446166839 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f35841033c0 msgr2=0x7f3584199290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:12:51.482 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:51.478+0000 7f358bd57640 1 --2- 172.21.15.42:0/446166839 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f35841033c0 0x7f3584199290 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f357c00ebc0 tx=0x7f357c00c6a0 comp rx=0 tx=0).stop 2024-08-30T09:12:51.482 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:51.478+0000 7f358bd57640 1 -- 172.21.15.42:0/446166839 shutdown_connections 2024-08-30T09:12:51.482 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:51.478+0000 7f358bd57640 1 --2- 172.21.15.42:0/446166839 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f35700b6dc0 0x7f35700b92d0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:51.482 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:51.478+0000 7f358bd57640 1 --2- 172.21.15.42:0/446166839 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3584103d60 0x7f3584199e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:51.482 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:51.478+0000 7f358bd57640 1 --2- 172.21.15.42:0/446166839 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f35841033c0 0x7f3584199290 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:51.482 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:51.478+0000 7f358bd57640 1 --2- 172.21.15.42:0/446166839 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3584102340 0x7f3584198c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:12:51.482 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:51.478+0000 7f358bd57640 1 -- 172.21.15.42:0/446166839 >> 172.21.15.42:0/446166839 conn(0x7f35840fdc80 msgr2=0x7f35841066e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:12:51.482 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:51.479+0000 7f358bd57640 1 -- 172.21.15.42:0/446166839 shutdown_connections 2024-08-30T09:12:51.482 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:12:51.479+0000 7f358bd57640 1 -- 172.21.15.42:0/446166839 wait complete. 2024-08-30T09:12:53.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:12:52 smithi167 ceph-mon[22063]: pgmap v736: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 6.2 MiB/s wr, 629 op/s 2024-08-30T09:12:53.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:12:52 smithi167 ceph-mon[22063]: scrub summary: idle 2024-08-30T09:12:53.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:12:52 smithi042 ceph-mon[18652]: pgmap v736: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 6.2 MiB/s wr, 629 op/s 2024-08-30T09:12:53.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:12:52 smithi042 ceph-mon[18652]: scrub summary: idle 2024-08-30T09:12:53.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:12:52 smithi190 ceph-mon[22001]: pgmap v736: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 6.2 MiB/s wr, 629 op/s 2024-08-30T09:12:53.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:12:52 smithi190 ceph-mon[22001]: scrub summary: idle 2024-08-30T09:12:54.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:12:53 smithi042 ceph-mon[18652]: scrub summary: idle 2024-08-30T09:12:54.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:12:53 smithi042 ceph-mon[18652]: scrub summary: idle 2024-08-30T09:12:54.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:12:53 smithi190 ceph-mon[22001]: scrub summary: idle 2024-08-30T09:12:54.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:12:53 smithi190 ceph-mon[22001]: scrub summary: idle 2024-08-30T09:12:54.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:12:53 smithi167 ceph-mon[22063]: scrub summary: idle 2024-08-30T09:12:54.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:12:53 smithi167 ceph-mon[22063]: scrub summary: idle 2024-08-30T09:12:55.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:12:54 smithi167 ceph-mon[22063]: pgmap v737: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 12 MiB/s wr, 1.04k op/s 2024-08-30T09:12:55.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:12:54 smithi042 ceph-mon[18652]: pgmap v737: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 12 MiB/s wr, 1.04k op/s 2024-08-30T09:12:55.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:12:54 smithi190 ceph-mon[22001]: pgmap v737: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 12 MiB/s wr, 1.04k op/s 2024-08-30T09:12:55.953 INFO:teuthology.orchestra.run.smithi042.stderr:progress: 730.0 s, 481.8 tps, lat 243.675 ms stddev 387.071 2024-08-30T09:12:57.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:12:56 smithi042 ceph-mon[18652]: pgmap v738: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 12 MiB/s wr, 1.01k op/s 2024-08-30T09:12:57.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:12:56 smithi190 ceph-mon[22001]: pgmap v738: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 12 MiB/s wr, 1.01k op/s 2024-08-30T09:12:57.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:12:56 smithi167 ceph-mon[22063]: pgmap v738: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 12 MiB/s wr, 1.01k op/s 2024-08-30T09:12:59.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:12:58 smithi042 ceph-mon[18652]: pgmap v739: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 12 MiB/s wr, 993 op/s 2024-08-30T09:12:59.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:12:58 smithi190 ceph-mon[22001]: pgmap v739: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 12 MiB/s wr, 993 op/s 2024-08-30T09:12:59.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:12:58 smithi167 ceph-mon[22063]: pgmap v739: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 12 MiB/s wr, 993 op/s 2024-08-30T09:13:00.953 INFO:teuthology.orchestra.run.smithi042.stderr:progress: 735.0 s, 417.0 tps, lat 215.338 ms stddev 248.753 2024-08-30T09:13:01.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:13:00 smithi042 ceph-mon[18652]: pgmap v740: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 13 MiB/s wr, 1.15k op/s 2024-08-30T09:13:01.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:13:00 smithi190 ceph-mon[22001]: pgmap v740: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 13 MiB/s wr, 1.15k op/s 2024-08-30T09:13:01.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:13:00 smithi167 ceph-mon[22063]: pgmap v740: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 13 MiB/s wr, 1.15k op/s 2024-08-30T09:13:03.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:13:02 smithi042 ceph-mon[18652]: pgmap v741: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 12 MiB/s wr, 916 op/s 2024-08-30T09:13:03.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:13:02 smithi190 ceph-mon[22001]: pgmap v741: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 12 MiB/s wr, 916 op/s 2024-08-30T09:13:03.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:13:02 smithi167 ceph-mon[22063]: pgmap v741: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 12 MiB/s wr, 916 op/s 2024-08-30T09:13:04.651 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.649+0000 7f2c22f90640 1 -- 172.21.15.42:0/2592623734 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2c1c103510 msgr2=0x7f2c1c109eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:13:04.651 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.649+0000 7f2c22f90640 1 --2- 172.21.15.42:0/2592623734 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2c1c103510 0x7f2c1c109eb0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f2c0400aa10 tx=0x7f2c0402b740 comp rx=0 tx=0).stop 2024-08-30T09:13:04.657 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.655+0000 7f2c22f90640 1 -- 172.21.15.42:0/2592623734 shutdown_connections 2024-08-30T09:13:04.657 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.655+0000 7f2c22f90640 1 --2- 172.21.15.42:0/2592623734 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2c1c102b70 0x7f2c1c102f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:04.657 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.655+0000 7f2c22f90640 1 --2- 172.21.15.42:0/2592623734 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2c1c103510 0x7f2c1c109eb0 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:04.657 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.655+0000 7f2c22f90640 1 --2- 172.21.15.42:0/2592623734 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2c1c101af0 0x7f2c1c101f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:04.657 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.655+0000 7f2c22f90640 1 -- 172.21.15.42:0/2592623734 >> 172.21.15.42:0/2592623734 conn(0x7f2c1c0fd430 msgr2=0x7f2c1c0ff8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:13:04.658 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.656+0000 7f2c22f90640 1 -- 172.21.15.42:0/2592623734 shutdown_connections 2024-08-30T09:13:04.659 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.657+0000 7f2c22f90640 1 -- 172.21.15.42:0/2592623734 wait complete. 2024-08-30T09:13:04.660 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.659+0000 7f2c22f90640 1 Processor -- start 2024-08-30T09:13:04.661 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.659+0000 7f2c22f90640 1 -- start start 2024-08-30T09:13:04.662 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.659+0000 7f2c22f90640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2c1c101af0 0x7f2c1c1a03a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:04.662 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.659+0000 7f2c22f90640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2c1c102b70 0x7f2c1c1a0a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:04.662 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.660+0000 7f2c22f90640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2c1c103510 0x7f2c1c1a1550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:04.662 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.660+0000 7f2c20d2c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2c1c101af0 0x7f2c1c1a03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:13:04.662 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.660+0000 7f2c20d2c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2c1c101af0 0x7f2c1c1a03a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:34000/0 (socket says 172.21.15.42:34000) 2024-08-30T09:13:04.662 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.660+0000 7f2c20d2c640 1 -- 172.21.15.42:0/3945225608 learned_addr learned my addr 172.21.15.42:0/3945225608 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:13:04.662 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.660+0000 7f2c22f90640 1 -- 172.21.15.42:0/3945225608 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f2c1c10c9c0 con 0x7f2c1c102b70 2024-08-30T09:13:04.662 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.660+0000 7f2c22f90640 1 -- 172.21.15.42:0/3945225608 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f2c1c10c840 con 0x7f2c1c103510 2024-08-30T09:13:04.662 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.660+0000 7f2c2152d640 1 --2- 172.21.15.42:0/3945225608 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2c1c103510 0x7f2c1c1a1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:13:04.663 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.660+0000 7f2c22f90640 1 -- 172.21.15.42:0/3945225608 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f2c1c10cb40 con 0x7f2c1c101af0 2024-08-30T09:13:04.667 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.663+0000 7f2c2152d640 1 -- 172.21.15.42:0/3945225608 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2c1c101af0 msgr2=0x7f2c1c1a03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:13:04.667 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.663+0000 7f2c2152d640 1 --2- 172.21.15.42:0/3945225608 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2c1c101af0 0x7f2c1c1a03a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:04.667 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.663+0000 7f2c2152d640 1 -- 172.21.15.42:0/3945225608 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2c1c102b70 msgr2=0x7f2c1c1a0a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:13:04.667 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.663+0000 7f2c2152d640 1 --2- 172.21.15.42:0/3945225608 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2c1c102b70 0x7f2c1c1a0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:04.667 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.663+0000 7f2c2152d640 1 -- 172.21.15.42:0/3945225608 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c1c1a1d70 con 0x7f2c1c103510 2024-08-30T09:13:04.667 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.665+0000 7f2c2152d640 1 --2- 172.21.15.42:0/3945225608 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2c1c103510 0x7f2c1c1a1550 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f2c0400a340 tx=0x7f2c04030d90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:13:04.668 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.665+0000 7f2c19ffb640 1 -- 172.21.15.42:0/3945225608 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c0403f070 con 0x7f2c1c103510 2024-08-30T09:13:04.668 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.665+0000 7f2c19ffb640 1 -- 172.21.15.42:0/3945225608 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c04003eb0 con 0x7f2c1c103510 2024-08-30T09:13:04.668 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.665+0000 7f2c19ffb640 1 -- 172.21.15.42:0/3945225608 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c04036040 con 0x7f2c1c103510 2024-08-30T09:13:04.668 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.666+0000 7f2c22f90640 1 -- 172.21.15.42:0/3945225608 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c1c1a2000 con 0x7f2c1c103510 2024-08-30T09:13:04.669 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.666+0000 7f2c22f90640 1 -- 172.21.15.42:0/3945225608 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c1c1a85f0 con 0x7f2c1c103510 2024-08-30T09:13:04.671 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.669+0000 7f2c29fdc740 1 Processor -- start 2024-08-30T09:13:04.672 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.669+0000 7f2c29fdc740 1 -- start start 2024-08-30T09:13:04.672 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.670+0000 7f2c29fdc740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e8908541b0 0x55e890bfa180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:04.672 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.670+0000 7f2c29fdc740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e89082b2f0 0x55e890bfa7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:04.672 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.670+0000 7f2c29fdc740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e8908334d0 0x55e890bfb340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:04.672 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.670+0000 7f2c29fdc740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55e890b6d000 con 0x55e8908334d0 2024-08-30T09:13:04.672 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.670+0000 7f2c29fdc740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55e890b5c590 con 0x55e89082b2f0 2024-08-30T09:13:04.672 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.670+0000 7f2c29fdc740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55e890b60540 con 0x55e8908541b0 2024-08-30T09:13:04.673 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.670+0000 7f2c20d2c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e89082b2f0 0x55e890bfa7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:13:04.673 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.670+0000 7f2c20d2c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e89082b2f0 0x55e890bfa7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:40320/0 (socket says 172.21.15.42:40320) 2024-08-30T09:13:04.673 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.670+0000 7f2c20d2c640 1 -- 172.21.15.42:0/3786735579 learned_addr learned my addr 172.21.15.42:0/3786735579 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:13:04.674 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.670+0000 7f2c20d2c640 1 -- 172.21.15.42:0/3786735579 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e8908541b0 msgr2=0x55e890bfa180 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-30T09:13:04.674 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.671+0000 7f2c20d2c640 1 --2- 172.21.15.42:0/3786735579 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e8908541b0 0x55e890bfa180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:04.674 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.671+0000 7f2c20d2c640 1 -- 172.21.15.42:0/3786735579 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e8908334d0 msgr2=0x55e890bfb340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:13:04.674 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.671+0000 7f2c1bfff640 1 --2- 172.21.15.42:0/3786735579 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e8908334d0 0x55e890bfb340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:13:04.674 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.671+0000 7f2c20d2c640 1 --2- 172.21.15.42:0/3786735579 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e8908334d0 0x55e890bfb340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:04.674 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.671+0000 7f2c20d2c640 1 -- 172.21.15.42:0/3786735579 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e8908bf790 con 0x55e89082b2f0 2024-08-30T09:13:04.674 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.671+0000 7f2c1bfff640 1 --2- 172.21.15.42:0/3786735579 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e8908334d0 0x55e890bfb340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-30T09:13:04.674 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.671+0000 7f2c20d2c640 1 --2- 172.21.15.42:0/3786735579 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e89082b2f0 0x55e890bfa7f0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f2c0c007f20 tx=0x7f2c0c00e7e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e85d0ebf4975c0b1 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:13:04.674 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.671+0000 7f2c12ffd640 1 -- 172.21.15.42:0/3786735579 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c0c014070 con 0x55e89082b2f0 2024-08-30T09:13:04.674 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.671+0000 7f2c12ffd640 1 -- 172.21.15.42:0/3786735579 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c0c010cf0 con 0x55e89082b2f0 2024-08-30T09:13:04.674 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.671+0000 7f2c12ffd640 1 -- 172.21.15.42:0/3786735579 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c0c019db0 con 0x55e89082b2f0 2024-08-30T09:13:04.675 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.673+0000 7f2c29fdc740 1 -- 172.21.15.42:0/3786735579 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e89082b2f0 msgr2=0x55e890bfa7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:13:04.675 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.673+0000 7f2c29fdc740 1 --2- 172.21.15.42:0/3786735579 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e89082b2f0 0x55e890bfa7f0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f2c0c007f20 tx=0x7f2c0c00e7e0 comp rx=0 tx=0).stop 2024-08-30T09:13:04.675 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.673+0000 7f2c19ffb640 1 -- 172.21.15.42:0/3945225608 <== mon.2 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7f2c04042070 con 0x7f2c1c103510 2024-08-30T09:13:04.675 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.673+0000 7f2c29fdc740 1 -- 172.21.15.42:0/3786735579 shutdown_connections 2024-08-30T09:13:04.675 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.673+0000 7f2c29fdc740 1 --2- 172.21.15.42:0/3786735579 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e8908334d0 0x55e890bfb340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:04.675 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.673+0000 7f2c29fdc740 1 --2- 172.21.15.42:0/3786735579 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e89082b2f0 0x55e890bfa7f0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:04.675 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.673+0000 7f2c29fdc740 1 --2- 172.21.15.42:0/3786735579 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e8908541b0 0x55e890bfa180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:04.675 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.673+0000 7f2c29fdc740 1 -- 172.21.15.42:0/3786735579 >> 172.21.15.42:0/3786735579 conn(0x55e890848300 msgr2=0x55e890bf5c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:13:04.676 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.673+0000 7f2c29fdc740 1 -- 172.21.15.42:0/3786735579 shutdown_connections 2024-08-30T09:13:04.676 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.673+0000 7f2c29fdc740 1 -- 172.21.15.42:0/3786735579 wait complete. 2024-08-30T09:13:04.676 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.674+0000 7f2c29fdc740 1 -- start start 2024-08-30T09:13:04.676 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.674+0000 7f2c29fdc740 1 Processor -- start 2024-08-30T09:13:04.678 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.676+0000 7f2c19ffb640 1 --2- 172.21.15.42:0/3945225608 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f2c080b4bd0 0x7f2c080b70e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:04.678 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.676+0000 7f2c19ffb640 1 -- 172.21.15.42:0/3945225608 <== mon.2 v2:172.21.15.167:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f2c04105bd0 con 0x7f2c1c103510 2024-08-30T09:13:04.679 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.677+0000 7f2c1bfff640 1 --2- 172.21.15.42:0/3945225608 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f2c080b4bd0 0x7f2c080b70e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:13:04.679 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.677+0000 7f2c29fdc740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e89082b2f0 0x55e890c14d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:04.679 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.677+0000 7f2c29fdc740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e8908334d0 0x55e890c153c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:04.679 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.677+0000 7f2c29fdc740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e8908541b0 0x55e890c15f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:04.680 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.677+0000 7f2c29fdc740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55e890a95ab0 con 0x55e89082b2f0 2024-08-30T09:13:04.680 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.677+0000 7f2c29fdc740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55e890ae3680 con 0x55e8908334d0 2024-08-30T09:13:04.680 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.677+0000 7f2c29fdc740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55e890bfbe60 con 0x55e8908541b0 2024-08-30T09:13:04.680 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.677+0000 7f2c20d2c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e89082b2f0 0x55e890c14d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:13:04.680 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.678+0000 7f2c20d2c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e89082b2f0 0x55e890c14d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:44554/0 (socket says 172.21.15.42:44554) 2024-08-30T09:13:04.680 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.678+0000 7f2c20d2c640 1 -- 172.21.15.42:0/4232421269 learned_addr learned my addr 172.21.15.42:0/4232421269 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:13:04.680 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.678+0000 7f2c29fdc740 10 client.0 _ll_register_callbacks cb 0x55e89086ba00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:13:04.681 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.678+0000 7f2c1bfff640 1 --2- 172.21.15.42:0/4232421269 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e8908334d0 0x55e890c153c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:13:04.681 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.678+0000 7f2c20d2c640 1 -- 172.21.15.42:0/4232421269 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e8908541b0 msgr2=0x55e890c15f30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-30T09:13:04.681 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.678+0000 7f2c20d2c640 1 --2- 172.21.15.42:0/4232421269 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e8908541b0 0x55e890c15f30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:04.681 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.678+0000 7f2c20d2c640 1 -- 172.21.15.42:0/4232421269 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e8908334d0 msgr2=0x55e890c153c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:13:04.681 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.678+0000 7f2c20d2c640 1 --2- 172.21.15.42:0/4232421269 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e8908334d0 0x55e890c153c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:04.681 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.678+0000 7f2c20d2c640 1 -- 172.21.15.42:0/4232421269 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e890938620 con 0x55e89082b2f0 2024-08-30T09:13:04.681 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.678+0000 7f2c1bfff640 1 --2- 172.21.15.42:0/3945225608 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f2c080b4bd0 0x7f2c080b70e0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f2c14001b80 tx=0x7f2c14014280 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:13:04.681 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.679+0000 7f2c20d2c640 1 --2- 172.21.15.42:0/4232421269 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e89082b2f0 0x55e890c14d00 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f2c0c0196b0 tx=0x7f2c0c01afa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10c2ab44f5ea23ca server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:13:04.681 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.679+0000 7f2bfbfff640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-08-30T09:13:04.682 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.679+0000 7f2bfbfff640 1 -- 172.21.15.42:0/4232421269 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c0c028020 con 0x55e89082b2f0 2024-08-30T09:13:04.682 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.679+0000 7f2c29fdc740 1 -- 172.21.15.42:0/4232421269 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2c0c010cf0 con 0x55e89082b2f0 2024-08-30T09:13:04.682 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.679+0000 7f2bfbfff640 1 -- 172.21.15.42:0/4232421269 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f2c0c003970 con 0x55e89082b2f0 2024-08-30T09:13:04.682 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.680+0000 7f2bfbfff640 1 -- 172.21.15.42:0/4232421269 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f2c0c030050 con 0x55e89082b2f0 2024-08-30T09:13:04.682 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.680+0000 7f2c29fdc740 10 client.17466 fetch_fsmap learned FSMap version 44 2024-08-30T09:13:04.682 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.680+0000 7f2c29fdc740 1 -- 172.21.15.42:0/4232421269 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55e890c1edd0 con 0x55e89082b2f0 2024-08-30T09:13:04.683 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.681+0000 7f2bfbfff640 1 -- 172.21.15.42:0/4232421269 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c0c010cf0 con 0x55e89082b2f0 2024-08-30T09:13:04.684 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.681+0000 7f2bfbfff640 1 -- 172.21.15.42:0/4232421269 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 44) ==== 8358+0+0 (secure 0 0 0) 0x7f2c0c004230 con 0x55e89082b2f0 2024-08-30T09:13:04.684 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.681+0000 7f2c29fdc740 10 client.17466 fetch_fsmap finished waiting for FSMap version 44 2024-08-30T09:13:04.684 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.681+0000 7f2c29fdc740 10 client.17466 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-30T09:13:04.684 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.681+0000 7f2c29fdc740 20 client.17466 populate_metadata read hostname 'smithi042' 2024-08-30T09:13:04.684 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.681+0000 7f2c29fdc740 1 --2- 172.21.15.42:0/4232421269 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55e890812d80 0x55e890c221f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:04.684 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.681+0000 7f2c29fdc740 4 client.17466 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-30T09:13:04.684 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.681+0000 7f2c29fdc740 1 -- 172.21.15.42:0/4232421269 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e890b6ef50 con 0x55e890812d80 2024-08-30T09:13:04.684 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.681+0000 7f2c1bfff640 1 --2- 172.21.15.42:0/4232421269 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55e890812d80 0x55e890c221f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:13:04.684 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.682+0000 7f2bfbfff640 10 client.17466 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:13:04.684 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.682+0000 7f2c1bfff640 1 --2- 172.21.15.42:0/4232421269 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55e890812d80 0x55e890c221f0 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a9f9e4f6027d509 server_cookie=61a69f24fd9b9c1b in_seq=0 out_seq=0 2024-08-30T09:13:04.685 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.682+0000 7f2bfbfff640 1 -- 172.21.15.42:0/4232421269 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55e890b6ef50 con 0x55e890812d80 2024-08-30T09:13:04.685 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.682+0000 7f2bfbfff640 10 client.17466 handle_command_reply: tid=0 2024-08-30T09:13:04.685 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.682+0000 7f2c29fdc740 1 client.17466 shutdown 2024-08-30T09:13:04.685 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.683+0000 7f2c29fdc740 1 -- 172.21.15.42:0/4232421269 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e89082b2f0 msgr2=0x55e890c14d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:13:04.686 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.683+0000 7f2c29fdc740 1 --2- 172.21.15.42:0/4232421269 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e89082b2f0 0x55e890c14d00 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f2c0c0196b0 tx=0x7f2c0c01afa0 comp rx=0 tx=0).stop 2024-08-30T09:13:04.686 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.683+0000 7f2c29fdc740 1 -- 172.21.15.42:0/4232421269 shutdown_connections 2024-08-30T09:13:04.686 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.683+0000 7f2c29fdc740 1 --2- 172.21.15.42:0/4232421269 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55e890812d80 0x55e890c221f0 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:04.686 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.683+0000 7f2c29fdc740 1 --2- 172.21.15.42:0/4232421269 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e8908541b0 0x55e890c15f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:04.686 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.683+0000 7f2c29fdc740 1 --2- 172.21.15.42:0/4232421269 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e8908334d0 0x55e890c153c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:04.686 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.683+0000 7f2c29fdc740 1 --2- 172.21.15.42:0/4232421269 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e89082b2f0 0x55e890c14d00 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:04.686 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.683+0000 7f2c29fdc740 1 -- 172.21.15.42:0/4232421269 >> 172.21.15.42:0/4232421269 conn(0x55e890848300 msgr2=0x55e890bf4170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:13:04.686 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.683+0000 7f2bfbfff640 0 client.17466 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:13:04.686 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.683+0000 7f2c29fdc740 1 -- 172.21.15.42:0/4232421269 shutdown_connections 2024-08-30T09:13:04.686 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.683+0000 7f2c29fdc740 1 -- 172.21.15.42:0/4232421269 wait complete. 2024-08-30T09:13:04.686 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.683+0000 7f2c29fdc740 20 client.17466 trim_cache size 0 max 16384 2024-08-30T09:13:04.702 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.699+0000 7f2c29fdc740 1 Processor -- start 2024-08-30T09:13:04.702 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.699+0000 7f2c29fdc740 1 -- start start 2024-08-30T09:13:04.702 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.699+0000 7f2c29fdc740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e89082b2f0 0x55e890bf7060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:04.702 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.699+0000 7f2c29fdc740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e8908334d0 0x55e890c1ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:04.702 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.699+0000 7f2c29fdc740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e8908541b0 0x55e890bf4980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:04.702 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.700+0000 7f2c29fdc740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f2c1c1acb00 con 0x55e89082b2f0 2024-08-30T09:13:04.702 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.700+0000 7f2c29fdc740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f2c1c1aa3a0 con 0x55e8908541b0 2024-08-30T09:13:04.702 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.700+0000 7f2c29fdc740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f2c1c074d60 con 0x55e8908334d0 2024-08-30T09:13:04.702 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.700+0000 7f2c2152d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e8908541b0 0x55e890bf4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:13:04.702 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.700+0000 7f2c20d2c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e89082b2f0 0x55e890bf7060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:13:04.703 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.700+0000 7f2c2152d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e8908541b0 0x55e890bf4980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:40334/0 (socket says 172.21.15.42:40334) 2024-08-30T09:13:04.703 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.700+0000 7f2c2152d640 1 -- 172.21.15.42:0/1203863298 learned_addr learned my addr 172.21.15.42:0/1203863298 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:13:04.703 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.700+0000 7f2c20d2c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e89082b2f0 0x55e890bf7060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:44558/0 (socket says 172.21.15.42:44558) 2024-08-30T09:13:04.703 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.700+0000 7f2c1bfff640 1 --2- 172.21.15.42:0/1203863298 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e8908334d0 0x55e890c1ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:13:04.703 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.700+0000 7f2c2152d640 1 -- 172.21.15.42:0/1203863298 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e8908334d0 msgr2=0x55e890c1ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:13:04.703 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.700+0000 7f2c2152d640 1 --2- 172.21.15.42:0/1203863298 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e8908334d0 0x55e890c1ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:04.703 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.700+0000 7f2c2152d640 1 -- 172.21.15.42:0/1203863298 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e89082b2f0 msgr2=0x55e890bf7060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:13:04.703 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.700+0000 7f2c2152d640 1 --2- 172.21.15.42:0/1203863298 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e89082b2f0 0x55e890bf7060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:04.703 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.700+0000 7f2c2152d640 1 -- 172.21.15.42:0/1203863298 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e890bfbd40 con 0x55e8908541b0 2024-08-30T09:13:04.705 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.700+0000 7f2c2152d640 1 --2- 172.21.15.42:0/1203863298 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e8908541b0 0x55e890bf4980 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f2c040c8200 tx=0x7f2c0402b740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=63bd1efd1b881aaf server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:13:04.705 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.701+0000 7f2c127fc640 1 -- 172.21.15.42:0/1203863298 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c04030e80 con 0x55e8908541b0 2024-08-30T09:13:04.705 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.701+0000 7f2c127fc640 1 -- 172.21.15.42:0/1203863298 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c040cbd20 con 0x55e8908541b0 2024-08-30T09:13:04.705 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.701+0000 7f2c127fc640 1 -- 172.21.15.42:0/1203863298 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c040c8410 con 0x55e8908541b0 2024-08-30T09:13:04.705 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.701+0000 7f2c29fdc740 1 -- 172.21.15.42:0/1203863298 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e8908541b0 msgr2=0x55e890bf4980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:13:04.705 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.701+0000 7f2c29fdc740 1 --2- 172.21.15.42:0/1203863298 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e8908541b0 0x55e890bf4980 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f2c040c8200 tx=0x7f2c0402b740 comp rx=0 tx=0).stop 2024-08-30T09:13:04.706 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.701+0000 7f2c29fdc740 1 -- 172.21.15.42:0/1203863298 shutdown_connections 2024-08-30T09:13:04.706 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.701+0000 7f2c29fdc740 1 --2- 172.21.15.42:0/1203863298 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e8908334d0 0x55e890c1ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:04.706 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.701+0000 7f2c29fdc740 1 --2- 172.21.15.42:0/1203863298 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e8908541b0 0x55e890bf4980 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:04.706 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.702+0000 7f2c29fdc740 1 --2- 172.21.15.42:0/1203863298 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e89082b2f0 0x55e890bf7060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:04.706 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.702+0000 7f2c29fdc740 1 -- 172.21.15.42:0/1203863298 >> 172.21.15.42:0/1203863298 conn(0x55e890848300 msgr2=0x55e890bfebf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:13:04.706 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.702+0000 7f2c29fdc740 1 -- 172.21.15.42:0/1203863298 shutdown_connections 2024-08-30T09:13:04.706 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.702+0000 7f2c29fdc740 1 -- 172.21.15.42:0/1203863298 wait complete. 2024-08-30T09:13:04.706 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.702+0000 7f2c29fdc740 1 -- start start 2024-08-30T09:13:04.706 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.702+0000 7f2c29fdc740 1 Processor -- start 2024-08-30T09:13:04.706 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.703+0000 7f2c29fdc740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e89082b2f0 0x55e890c19e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:04.706 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.703+0000 7f2c29fdc740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e8908334d0 0x55e890c1a510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:04.706 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.703+0000 7f2c29fdc740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e8908541b0 0x55e890c1b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:04.706 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.703+0000 7f2c29fdc740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f2c1c1a88e0 con 0x55e8908334d0 2024-08-30T09:13:04.707 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.703+0000 7f2c29fdc740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f2c1c1a9cd0 con 0x55e8908541b0 2024-08-30T09:13:04.707 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.703+0000 7f2c29fdc740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f2c1c1a8f30 con 0x55e89082b2f0 2024-08-30T09:13:04.707 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.703+0000 7f2c29fdc740 10 client.0 _ll_register_callbacks cb 0x7f2c0c007da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:13:04.709 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.703+0000 7f2c2152d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e8908541b0 0x55e890c1b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:13:04.709 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.703+0000 7f2c2152d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e8908541b0 0x55e890c1b080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:40344/0 (socket says 172.21.15.42:40344) 2024-08-30T09:13:04.709 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.703+0000 7f2c2152d640 1 -- 172.21.15.42:0/1353845351 learned_addr learned my addr 172.21.15.42:0/1353845351 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:13:04.709 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.704+0000 7f2c20d2c640 1 --2- 172.21.15.42:0/1353845351 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e89082b2f0 0x55e890c19e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:13:04.709 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.704+0000 7f2c2152d640 1 -- 172.21.15.42:0/1353845351 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e89082b2f0 msgr2=0x55e890c19e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:13:04.709 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.704+0000 7f2c2152d640 1 --2- 172.21.15.42:0/1353845351 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e89082b2f0 0x55e890c19e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:04.709 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.704+0000 7f2c2152d640 1 -- 172.21.15.42:0/1353845351 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e8908334d0 msgr2=0x55e890c1a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:13:04.709 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.704+0000 7f2c2152d640 1 --2- 172.21.15.42:0/1353845351 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e8908334d0 0x55e890c1a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:04.709 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.704+0000 7f2c2152d640 1 -- 172.21.15.42:0/1353845351 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e890c1b7e0 con 0x55e8908541b0 2024-08-30T09:13:04.709 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.704+0000 7f2c20d2c640 1 --2- 172.21.15.42:0/1353845351 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e89082b2f0 0x55e890c19e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-30T09:13:04.709 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.704+0000 7f2c2152d640 1 --2- 172.21.15.42:0/1353845351 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e8908541b0 0x55e890c1b080 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f2c040c8640 tx=0x7f2c040c8670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=79f980b7eeff228c server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:13:04.709 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.705+0000 7f2bfbfff640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-30T09:13:04.710 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.705+0000 7f2bfbfff640 1 -- 172.21.15.42:0/1353845351 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c040d0a80 con 0x55e8908541b0 2024-08-30T09:13:04.710 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.705+0000 7f2bfbfff640 1 -- 172.21.15.42:0/1353845351 <== mon.2 v2:172.21.15.167:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f2c040d3e00 con 0x55e8908541b0 2024-08-30T09:13:04.710 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.705+0000 7f2c29fdc740 1 -- 172.21.15.42:0/1353845351 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2c040cbd20 con 0x55e8908541b0 2024-08-30T09:13:04.710 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.705+0000 7f2bfbfff640 1 -- 172.21.15.42:0/1353845351 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f2c040d50a0 con 0x55e8908541b0 2024-08-30T09:13:04.710 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.705+0000 7f2c29fdc740 10 client.27302 fetch_fsmap learned FSMap version 44 2024-08-30T09:13:04.710 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.705+0000 7f2c29fdc740 1 -- 172.21.15.42:0/1353845351 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55e890c1cb00 con 0x55e8908541b0 2024-08-30T09:13:04.710 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.705+0000 7f2bfbfff640 1 -- 172.21.15.42:0/1353845351 <== mon.2 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c040cbd20 con 0x55e8908541b0 2024-08-30T09:13:04.710 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.706+0000 7f2bfbfff640 1 -- 172.21.15.42:0/1353845351 <== mon.2 v2:172.21.15.167:3300/0 5 ==== fsmap(e 44) ==== 8358+0+0 (secure 0 0 0) 0x7f2c040f0070 con 0x55e8908541b0 2024-08-30T09:13:04.710 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.706+0000 7f2c29fdc740 10 client.27302 fetch_fsmap finished waiting for FSMap version 44 2024-08-30T09:13:04.710 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.706+0000 7f2c29fdc740 10 client.27302 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-30T09:13:04.710 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.706+0000 7f2c29fdc740 20 client.27302 populate_metadata read hostname 'smithi042' 2024-08-30T09:13:04.710 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.706+0000 7f2c29fdc740 1 --2- 172.21.15.42:0/1353845351 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55e8907e81c0 0x55e890c8ec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:04.711 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.706+0000 7f2c29fdc740 4 client.27302 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-30T09:13:04.711 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.706+0000 7f2c29fdc740 1 -- 172.21.15.42:0/1353845351 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e890b6a9f0 con 0x55e8907e81c0 2024-08-30T09:13:04.711 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.706+0000 7f2c20d2c640 1 --2- 172.21.15.42:0/1353845351 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55e8907e81c0 0x55e890c8ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:13:04.711 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.706+0000 7f2bfbfff640 10 client.27302 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:13:04.711 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.706+0000 7f2c20d2c640 1 --2- 172.21.15.42:0/1353845351 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55e8907e81c0 0x55e890c8ec60 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ff3b788135def1e0 server_cookie=5e3bbf42d025f9a0 in_seq=0 out_seq=0 2024-08-30T09:13:04.721 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.719+0000 7f2bfbfff640 1 -- 172.21.15.42:0/1353845351 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e890b6a9f0 con 0x55e8907e81c0 2024-08-30T09:13:04.721 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.719+0000 7f2bfbfff640 10 client.27302 handle_command_reply: tid=0 2024-08-30T09:13:04.721 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.719+0000 7f2c29fdc740 1 client.27302 shutdown 2024-08-30T09:13:04.721 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.719+0000 7f2c29fdc740 1 -- 172.21.15.42:0/1353845351 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e8908541b0 msgr2=0x55e890c1b080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:13:04.721 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.719+0000 7f2c29fdc740 1 --2- 172.21.15.42:0/1353845351 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e8908541b0 0x55e890c1b080 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f2c040c8640 tx=0x7f2c040c8670 comp rx=0 tx=0).stop 2024-08-30T09:13:04.722 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.720+0000 7f2c29fdc740 1 -- 172.21.15.42:0/1353845351 shutdown_connections 2024-08-30T09:13:04.722 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.720+0000 7f2c29fdc740 1 --2- 172.21.15.42:0/1353845351 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55e8907e81c0 0x55e890c8ec60 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:04.722 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.720+0000 7f2c29fdc740 1 --2- 172.21.15.42:0/1353845351 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e8908541b0 0x55e890c1b080 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:04.722 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.720+0000 7f2bfbfff640 0 client.27302 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:13:04.722 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.720+0000 7f2c29fdc740 1 --2- 172.21.15.42:0/1353845351 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e8908334d0 0x55e890c1a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:04.722 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.720+0000 7f2c29fdc740 1 --2- 172.21.15.42:0/1353845351 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e89082b2f0 0x55e890c19e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:04.722 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.720+0000 7f2c29fdc740 1 -- 172.21.15.42:0/1353845351 >> 172.21.15.42:0/1353845351 conn(0x55e890848300 msgr2=0x55e890c13300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:13:04.722 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.720+0000 7f2c29fdc740 1 -- 172.21.15.42:0/1353845351 shutdown_connections 2024-08-30T09:13:04.722 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.720+0000 7f2c29fdc740 1 -- 172.21.15.42:0/1353845351 wait complete. 2024-08-30T09:13:04.723 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.720+0000 7f2c29fdc740 20 client.27302 trim_cache size 0 max 16384 2024-08-30T09:13:04.723 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-08-30T09:13:04.723 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2024-08-30T09:13:04.723 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-08-30T09:13:04.723 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-08-30T09:13:04.724 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.721+0000 7f2c22f90640 1 -- 172.21.15.42:0/3945225608 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f2c080b4bd0 msgr2=0x7f2c080b70e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:13:04.724 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.721+0000 7f2c22f90640 1 --2- 172.21.15.42:0/3945225608 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f2c080b4bd0 0x7f2c080b70e0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f2c14001b80 tx=0x7f2c14014280 comp rx=0 tx=0).stop 2024-08-30T09:13:04.724 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.721+0000 7f2c22f90640 1 -- 172.21.15.42:0/3945225608 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2c1c103510 msgr2=0x7f2c1c1a1550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:13:04.724 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.721+0000 7f2c22f90640 1 --2- 172.21.15.42:0/3945225608 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2c1c103510 0x7f2c1c1a1550 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f2c0400a340 tx=0x7f2c04030d90 comp rx=0 tx=0).stop 2024-08-30T09:13:04.725 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.721+0000 7f2c22f90640 1 -- 172.21.15.42:0/3945225608 shutdown_connections 2024-08-30T09:13:04.725 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.721+0000 7f2c22f90640 1 --2- 172.21.15.42:0/3945225608 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f2c080b4bd0 0x7f2c080b70e0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:04.725 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.721+0000 7f2c22f90640 1 --2- 172.21.15.42:0/3945225608 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2c1c103510 0x7f2c1c1a1550 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:04.725 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.721+0000 7f2c22f90640 1 --2- 172.21.15.42:0/3945225608 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2c1c102b70 0x7f2c1c1a0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:04.725 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.721+0000 7f2c22f90640 1 --2- 172.21.15.42:0/3945225608 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2c1c101af0 0x7f2c1c1a03a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:04.725 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.721+0000 7f2c22f90640 1 -- 172.21.15.42:0/3945225608 >> 172.21.15.42:0/3945225608 conn(0x7f2c1c0fd430 msgr2=0x7f2c1c10f980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:13:04.725 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.722+0000 7f2c22f90640 1 -- 172.21.15.42:0/3945225608 shutdown_connections 2024-08-30T09:13:04.725 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:04.722+0000 7f2c22f90640 1 -- 172.21.15.42:0/3945225608 wait complete. 2024-08-30T09:13:05.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:13:04 smithi042 ceph-mon[18652]: pgmap v742: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 17 MiB/s wr, 1.30k op/s 2024-08-30T09:13:05.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:13:04 smithi042 ceph-mon[18652]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-30T09:13:05.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:13:04 smithi190 ceph-mon[22001]: pgmap v742: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 17 MiB/s wr, 1.30k op/s 2024-08-30T09:13:05.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:13:04 smithi190 ceph-mon[22001]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-30T09:13:05.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:13:04 smithi167 ceph-mon[22063]: pgmap v742: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 17 MiB/s wr, 1.30k op/s 2024-08-30T09:13:05.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:13:04 smithi167 ceph-mon[22063]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-30T09:13:05.953 INFO:teuthology.orchestra.run.smithi042.stderr:progress: 740.0 s, 392.8 tps, lat 273.876 ms stddev 478.981 2024-08-30T09:13:07.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:13:06 smithi042 ceph-mon[18652]: pgmap v743: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 13 MiB/s wr, 1.01k op/s 2024-08-30T09:13:07.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:13:06 smithi190 ceph-mon[22001]: pgmap v743: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 13 MiB/s wr, 1.01k op/s 2024-08-30T09:13:07.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:13:06 smithi167 ceph-mon[22063]: pgmap v743: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 13 MiB/s wr, 1.01k op/s 2024-08-30T09:13:09.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:13:08 smithi042 ceph-mon[18652]: pgmap v744: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 12 MiB/s wr, 930 op/s 2024-08-30T09:13:09.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:13:08 smithi190 ceph-mon[22001]: pgmap v744: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 12 MiB/s wr, 930 op/s 2024-08-30T09:13:09.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:13:08 smithi167 ceph-mon[22063]: pgmap v744: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 12 MiB/s wr, 930 op/s 2024-08-30T09:13:10.953 INFO:teuthology.orchestra.run.smithi042.stderr:progress: 745.0 s, 471.8 tps, lat 214.095 ms stddev 401.248 2024-08-30T09:13:11.128 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:13:10 smithi167 ceph-mon[22063]: pgmap v745: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 13 MiB/s wr, 1.07k op/s 2024-08-30T09:13:11.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:13:10 smithi042 ceph-mon[18652]: pgmap v745: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 13 MiB/s wr, 1.07k op/s 2024-08-30T09:13:11.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:13:10 smithi190 ceph-mon[22001]: pgmap v745: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 13 MiB/s wr, 1.07k op/s 2024-08-30T09:13:13.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:13:12 smithi042 ceph-mon[18652]: pgmap v746: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 12 MiB/s wr, 918 op/s 2024-08-30T09:13:13.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:13:12 smithi190 ceph-mon[22001]: pgmap v746: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 12 MiB/s wr, 918 op/s 2024-08-30T09:13:13.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:13:12 smithi167 ceph-mon[22063]: pgmap v746: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 12 MiB/s wr, 918 op/s 2024-08-30T09:13:15.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:13:14 smithi190 ceph-mon[22001]: pgmap v747: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 13 MiB/s wr, 1.01k op/s 2024-08-30T09:13:15.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:13:14 smithi190 ceph-mon[22001]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-30T09:13:15.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:13:14 smithi167 ceph-mon[22063]: pgmap v747: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 13 MiB/s wr, 1.01k op/s 2024-08-30T09:13:15.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:13:14 smithi167 ceph-mon[22063]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-30T09:13:15.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:13:14 smithi042 ceph-mon[18652]: pgmap v747: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 13 MiB/s wr, 1.01k op/s 2024-08-30T09:13:15.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:13:14 smithi042 ceph-mon[18652]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-30T09:13:15.953 INFO:teuthology.orchestra.run.smithi042.stderr:progress: 750.0 s, 488.4 tps, lat 209.709 ms stddev 231.983 2024-08-30T09:13:17.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:13:16 smithi190 ceph-mon[22001]: pgmap v748: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 6.5 MiB/s wr, 575 op/s 2024-08-30T09:13:17.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:13:16 smithi167 ceph-mon[22063]: pgmap v748: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 6.5 MiB/s wr, 575 op/s 2024-08-30T09:13:17.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:13:16 smithi042 ceph-mon[18652]: pgmap v748: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 6.5 MiB/s wr, 575 op/s 2024-08-30T09:13:17.857 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.855+0000 7f93434c5640 1 -- 172.21.15.42:0/383652503 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f933c102350 msgr2=0x7f933c1027c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:13:17.857 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.855+0000 7f93434c5640 1 --2- 172.21.15.42:0/383652503 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f933c102350 0x7f933c1027c0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f9338009900 tx=0x7f933802a410 comp rx=0 tx=0).stop 2024-08-30T09:13:17.857 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.855+0000 7f93434c5640 1 -- 172.21.15.42:0/383652503 shutdown_connections 2024-08-30T09:13:17.858 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.855+0000 7f93434c5640 1 --2- 172.21.15.42:0/383652503 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f933c1033d0 0x7f933c1037e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:17.858 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.855+0000 7f93434c5640 1 --2- 172.21.15.42:0/383652503 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f933c103d70 0x7f933c10a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:17.858 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.855+0000 7f93434c5640 1 --2- 172.21.15.42:0/383652503 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f933c102350 0x7f933c1027c0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:17.858 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.855+0000 7f93434c5640 1 -- 172.21.15.42:0/383652503 >> 172.21.15.42:0/383652503 conn(0x7f933c0fdcd0 msgr2=0x7f933c100140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:13:17.858 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.855+0000 7f93434c5640 1 -- 172.21.15.42:0/383652503 shutdown_connections 2024-08-30T09:13:17.858 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.855+0000 7f93434c5640 1 -- 172.21.15.42:0/383652503 wait complete. 2024-08-30T09:13:17.859 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.856+0000 7f93434c5640 1 Processor -- start 2024-08-30T09:13:17.860 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.856+0000 7f93434c5640 1 -- start start 2024-08-30T09:13:17.860 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.856+0000 7f93434c5640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f933c102350 0x7f933c197af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:17.860 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.856+0000 7f93434c5640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f933c1033d0 0x7f933c198160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:17.860 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.856+0000 7f93434c5640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f933c103d70 0x7f933c198d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:17.860 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.856+0000 7f93434c5640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f933c10ced0 con 0x7f933c1033d0 2024-08-30T09:13:17.860 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.856+0000 7f93434c5640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f933c10cd50 con 0x7f933c103d70 2024-08-30T09:13:17.860 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.856+0000 7f93434c5640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f933c10d050 con 0x7f933c102350 2024-08-30T09:13:17.860 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.856+0000 7f9341a62640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f933c103d70 0x7f933c198d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:13:17.860 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.856+0000 7f9341a62640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f933c103d70 0x7f933c198d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:51744/0 (socket says 172.21.15.42:51744) 2024-08-30T09:13:17.860 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.856+0000 7f9341a62640 1 -- 172.21.15.42:0/729710775 learned_addr learned my addr 172.21.15.42:0/729710775 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:13:17.860 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.857+0000 7f9341261640 1 --2- 172.21.15.42:0/729710775 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f933c102350 0x7f933c197af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:13:17.860 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.857+0000 7f9340a60640 1 --2- 172.21.15.42:0/729710775 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f933c1033d0 0x7f933c198160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:13:17.861 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.857+0000 7f9341a62640 1 -- 172.21.15.42:0/729710775 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f933c102350 msgr2=0x7f933c197af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:13:17.861 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.857+0000 7f9341a62640 1 --2- 172.21.15.42:0/729710775 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f933c102350 0x7f933c197af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:17.861 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.857+0000 7f9341a62640 1 -- 172.21.15.42:0/729710775 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f933c1033d0 msgr2=0x7f933c198160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:13:17.861 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.857+0000 7f9341a62640 1 --2- 172.21.15.42:0/729710775 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f933c1033d0 0x7f933c198160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:17.861 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.857+0000 7f9341a62640 1 -- 172.21.15.42:0/729710775 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f933c199530 con 0x7f933c103d70 2024-08-30T09:13:17.861 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.857+0000 7f9341a62640 1 --2- 172.21.15.42:0/729710775 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f933c103d70 0x7f933c198d10 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f9334009e20 tx=0x7f933400e490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:13:17.861 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.857+0000 7f93327fc640 1 -- 172.21.15.42:0/729710775 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9334004b20 con 0x7f933c103d70 2024-08-30T09:13:17.861 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.858+0000 7f93327fc640 1 -- 172.21.15.42:0/729710775 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9334004cc0 con 0x7f933c103d70 2024-08-30T09:13:17.861 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.858+0000 7f93327fc640 1 -- 172.21.15.42:0/729710775 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9334005860 con 0x7f933c103d70 2024-08-30T09:13:17.861 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.858+0000 7f93434c5640 1 -- 172.21.15.42:0/729710775 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f933c199820 con 0x7f933c103d70 2024-08-30T09:13:17.861 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.858+0000 7f93434c5640 1 -- 172.21.15.42:0/729710775 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f933c1a4320 con 0x7f933c103d70 2024-08-30T09:13:17.863 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.861+0000 7f934a433740 1 Processor -- start 2024-08-30T09:13:17.863 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.861+0000 7f93327fc640 1 -- 172.21.15.42:0/729710775 <== mon.2 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7f933401b020 con 0x7f933c103d70 2024-08-30T09:13:17.863 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.862+0000 7f934a433740 1 -- start start 2024-08-30T09:13:17.865 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.862+0000 7f934a433740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56278e9711b0 0x56278ed16f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:17.865 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.862+0000 7f934a433740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56278e9482f0 0x56278ed175d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:17.866 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.862+0000 7f934a433740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56278e9504d0 0x56278ed18120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:17.866 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.862+0000 7f934a433740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x56278ec8a000 con 0x56278e9482f0 2024-08-30T09:13:17.866 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.862+0000 7f934a433740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x56278ec79590 con 0x56278e9504d0 2024-08-30T09:13:17.866 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.862+0000 7f934a433740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x56278ec7d540 con 0x56278e9711b0 2024-08-30T09:13:17.866 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.862+0000 7f9340a60640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56278e9504d0 0x56278ed18120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:13:17.866 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.862+0000 7f9340a60640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56278e9504d0 0x56278ed18120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:51754/0 (socket says 172.21.15.42:51754) 2024-08-30T09:13:17.866 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.862+0000 7f9340a60640 1 -- 172.21.15.42:0/2328841812 learned_addr learned my addr 172.21.15.42:0/2328841812 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:13:17.866 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.862+0000 7f9340a60640 1 --2- 172.21.15.42:0/2328841812 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56278e9711b0 0x56278ed16f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:13:17.866 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.863+0000 7f9341261640 1 --2- 172.21.15.42:0/2328841812 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56278e9482f0 0x56278ed175d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:13:17.866 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.863+0000 7f93327fc640 1 --2- 172.21.15.42:0/729710775 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f93280b4bb0 0x7f93280b70c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:17.866 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.863+0000 7f9340a60640 1 -- 172.21.15.42:0/2328841812 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56278e9711b0 msgr2=0x56278ed16f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:13:17.866 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.863+0000 7f9340a60640 1 --2- 172.21.15.42:0/2328841812 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56278e9711b0 0x56278ed16f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:17.867 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.863+0000 7f9340a60640 1 -- 172.21.15.42:0/2328841812 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56278e9482f0 msgr2=0x56278ed175d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:13:17.867 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.863+0000 7f9340a60640 1 --2- 172.21.15.42:0/2328841812 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56278e9482f0 0x56278ed175d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:17.867 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.863+0000 7f9340a60640 1 -- 172.21.15.42:0/2328841812 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56278e9dc790 con 0x56278e9504d0 2024-08-30T09:13:17.867 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.863+0000 7f93327fc640 1 -- 172.21.15.42:0/729710775 <== mon.2 v2:172.21.15.167:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f93340e4d90 con 0x7f933c103d70 2024-08-30T09:13:17.867 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.864+0000 7f9340a60640 1 --2- 172.21.15.42:0/2328841812 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56278e9504d0 0x56278ed18120 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f932c0093e0 tx=0x7f932c009370 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cef5170f3da8bc1c server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:13:17.868 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.864+0000 7f9341a62640 1 --2- 172.21.15.42:0/729710775 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f93280b4bb0 0x7f93280b70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:13:17.868 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.864+0000 7f93277fe640 1 -- 172.21.15.42:0/2328841812 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f932c004250 con 0x56278e9504d0 2024-08-30T09:13:17.868 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.864+0000 7f93277fe640 1 -- 172.21.15.42:0/2328841812 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f932c0043f0 con 0x56278e9504d0 2024-08-30T09:13:17.868 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.864+0000 7f93277fe640 1 -- 172.21.15.42:0/2328841812 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f932c00e6e0 con 0x56278e9504d0 2024-08-30T09:13:17.868 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.865+0000 7f934a433740 1 -- 172.21.15.42:0/2328841812 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56278e9504d0 msgr2=0x56278ed18120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:13:17.869 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.865+0000 7f934a433740 1 --2- 172.21.15.42:0/2328841812 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56278e9504d0 0x56278ed18120 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f932c0093e0 tx=0x7f932c009370 comp rx=0 tx=0).stop 2024-08-30T09:13:17.869 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.865+0000 7f9341a62640 1 --2- 172.21.15.42:0/729710775 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f93280b4bb0 0x7f93280b70c0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f93340a74b0 tx=0x7f9334009f10 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:13:17.869 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.866+0000 7f934a433740 1 -- 172.21.15.42:0/2328841812 shutdown_connections 2024-08-30T09:13:17.869 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.866+0000 7f934a433740 1 --2- 172.21.15.42:0/2328841812 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56278e9504d0 0x56278ed18120 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:17.869 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.866+0000 7f934a433740 1 --2- 172.21.15.42:0/2328841812 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56278e9482f0 0x56278ed175d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:17.869 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.866+0000 7f934a433740 1 --2- 172.21.15.42:0/2328841812 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56278e9711b0 0x56278ed16f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:17.869 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.866+0000 7f934a433740 1 -- 172.21.15.42:0/2328841812 >> 172.21.15.42:0/2328841812 conn(0x56278e965300 msgr2=0x56278ed129f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:13:17.869 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.866+0000 7f934a433740 1 -- 172.21.15.42:0/2328841812 shutdown_connections 2024-08-30T09:13:17.869 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.866+0000 7f934a433740 1 -- 172.21.15.42:0/2328841812 wait complete. 2024-08-30T09:13:17.869 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.866+0000 7f934a433740 1 -- start start 2024-08-30T09:13:17.869 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.866+0000 7f934a433740 1 Processor -- start 2024-08-30T09:13:17.872 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.867+0000 7f934a433740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56278e9482f0 0x56278ed20ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:17.872 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.867+0000 7f934a433740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56278e9504d0 0x56278ed21190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:17.872 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.867+0000 7f934a433740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56278e9711b0 0x56278ed21c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:17.872 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.867+0000 7f9340a60640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56278e9482f0 0x56278ed20ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:13:17.873 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.867+0000 7f9340a60640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56278e9482f0 0x56278ed20ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:52500/0 (socket says 172.21.15.42:52500) 2024-08-30T09:13:17.873 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.867+0000 7f9340a60640 1 -- 172.21.15.42:0/2120684675 learned_addr learned my addr 172.21.15.42:0/2120684675 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:13:17.873 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.867+0000 7f934a433740 1 -- 172.21.15.42:0/2120684675 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x56278ebb2ab0 con 0x56278e9711b0 2024-08-30T09:13:17.873 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.867+0000 7f934a433740 1 -- 172.21.15.42:0/2120684675 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x56278ec00680 con 0x56278e9504d0 2024-08-30T09:13:17.873 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.867+0000 7f934a433740 1 -- 172.21.15.42:0/2120684675 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x56278ed18c40 con 0x56278e9482f0 2024-08-30T09:13:17.873 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.867+0000 7f9341261640 1 --2- 172.21.15.42:0/2120684675 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56278e9504d0 0x56278ed21190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:13:17.873 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.867+0000 7f9340a60640 1 --2- 172.21.15.42:0/2120684675 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56278e9711b0 0x56278ed21c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:13:17.873 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.867+0000 7f934a433740 10 client.0 _ll_register_callbacks cb 0x56278e988a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:13:17.873 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.868+0000 7f9341261640 1 -- 172.21.15.42:0/2120684675 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56278e9482f0 msgr2=0x56278ed20ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:13:17.873 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.868+0000 7f9341261640 1 --2- 172.21.15.42:0/2120684675 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56278e9482f0 0x56278ed20ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:17.873 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.868+0000 7f9341261640 1 -- 172.21.15.42:0/2120684675 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56278e9711b0 msgr2=0x56278ed21c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:13:17.873 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.868+0000 7f9341261640 1 --2- 172.21.15.42:0/2120684675 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56278e9711b0 0x56278ed21c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:17.873 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.868+0000 7f9341261640 1 -- 172.21.15.42:0/2120684675 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56278ea55620 con 0x56278e9504d0 2024-08-30T09:13:17.873 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.868+0000 7f9341261640 1 --2- 172.21.15.42:0/2120684675 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56278e9504d0 0x56278ed21190 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f93380057c0 tx=0x7f9338005f30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f4386769f53cef49 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:13:17.874 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.868+0000 7f9324ff9640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-30T09:13:17.874 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.868+0000 7f9324ff9640 1 -- 172.21.15.42:0/2120684675 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9338035050 con 0x56278e9504d0 2024-08-30T09:13:17.874 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.868+0000 7f9324ff9640 1 -- 172.21.15.42:0/2120684675 <== mon.2 v2:172.21.15.167:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f9338031890 con 0x56278e9504d0 2024-08-30T09:13:17.874 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.868+0000 7f934a433740 1 -- 172.21.15.42:0/2120684675 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f932c0043f0 con 0x56278e9504d0 2024-08-30T09:13:17.874 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.868+0000 7f9324ff9640 1 -- 172.21.15.42:0/2120684675 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f9338031a90 con 0x56278e9504d0 2024-08-30T09:13:17.874 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.868+0000 7f934a433740 10 client.27314 fetch_fsmap learned FSMap version 45 2024-08-30T09:13:17.874 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.868+0000 7f934a433740 1 -- 172.21.15.42:0/2120684675 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x56278ed2ac10 con 0x56278e9504d0 2024-08-30T09:13:17.874 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.869+0000 7f9324ff9640 1 -- 172.21.15.42:0/2120684675 <== mon.2 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f932c0043f0 con 0x56278e9504d0 2024-08-30T09:13:17.874 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.869+0000 7f9324ff9640 1 -- 172.21.15.42:0/2120684675 <== mon.2 v2:172.21.15.167:3300/0 5 ==== fsmap(e 45) ==== 8362+0+0 (secure 0 0 0) 0x7f933803b430 con 0x56278e9504d0 2024-08-30T09:13:17.874 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.869+0000 7f934a433740 10 client.27314 fetch_fsmap finished waiting for FSMap version 45 2024-08-30T09:13:17.874 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.869+0000 7f934a433740 10 client.27314 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-30T09:13:17.874 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.869+0000 7f934a433740 20 client.27314 populate_metadata read hostname 'smithi042' 2024-08-30T09:13:17.874 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.869+0000 7f934a433740 1 --2- 172.21.15.42:0/2120684675 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x56278e92fd80 0x56278ed2e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:17.874 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.869+0000 7f934a433740 4 client.27314 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-30T09:13:17.875 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.869+0000 7f934a433740 1 -- 172.21.15.42:0/2120684675 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56278ec8bf50 con 0x56278e92fd80 2024-08-30T09:13:17.875 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.870+0000 7f9340a60640 1 --2- 172.21.15.42:0/2120684675 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x56278e92fd80 0x56278ed2e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:13:17.875 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.870+0000 7f9340a60640 1 --2- 172.21.15.42:0/2120684675 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x56278e92fd80 0x56278ed2e3a0 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c64fef88bab172dd server_cookie=327acb9dedb16cc0 in_seq=0 out_seq=0 2024-08-30T09:13:17.875 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.870+0000 7f9324ff9640 10 client.27314 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:13:17.876 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.871+0000 7f9324ff9640 1 -- 172.21.15.42:0/2120684675 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56278ec8bf50 con 0x56278e92fd80 2024-08-30T09:13:17.876 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.871+0000 7f9324ff9640 10 client.27314 handle_command_reply: tid=0 2024-08-30T09:13:17.876 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.871+0000 7f934a433740 1 client.27314 shutdown 2024-08-30T09:13:17.876 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.871+0000 7f934a433740 1 -- 172.21.15.42:0/2120684675 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56278e9504d0 msgr2=0x56278ed21190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:13:17.876 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.871+0000 7f934a433740 1 --2- 172.21.15.42:0/2120684675 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56278e9504d0 0x56278ed21190 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f93380057c0 tx=0x7f9338005f30 comp rx=0 tx=0).stop 2024-08-30T09:13:17.876 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.871+0000 7f934a433740 1 -- 172.21.15.42:0/2120684675 shutdown_connections 2024-08-30T09:13:17.876 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.871+0000 7f934a433740 1 --2- 172.21.15.42:0/2120684675 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x56278e92fd80 0x56278ed2e3a0 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:17.876 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.871+0000 7f934a433740 1 --2- 172.21.15.42:0/2120684675 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56278e9711b0 0x56278ed21c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:17.876 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.871+0000 7f9324ff9640 0 client.27314 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:13:17.876 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.871+0000 7f934a433740 1 --2- 172.21.15.42:0/2120684675 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56278e9504d0 0x56278ed21190 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:17.877 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.871+0000 7f934a433740 1 --2- 172.21.15.42:0/2120684675 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56278e9482f0 0x56278ed20ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:17.877 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.871+0000 7f934a433740 1 -- 172.21.15.42:0/2120684675 >> 172.21.15.42:0/2120684675 conn(0x56278e965300 msgr2=0x56278ed135f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:13:17.877 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.871+0000 7f934a433740 1 -- 172.21.15.42:0/2120684675 shutdown_connections 2024-08-30T09:13:17.877 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.871+0000 7f934a433740 1 -- 172.21.15.42:0/2120684675 wait complete. 2024-08-30T09:13:17.877 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.871+0000 7f934a433740 20 client.27314 trim_cache size 0 max 16384 2024-08-30T09:13:17.896 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.893+0000 7f934a433740 1 Processor -- start 2024-08-30T09:13:17.896 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.893+0000 7f934a433740 1 -- start start 2024-08-30T09:13:17.896 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.893+0000 7f934a433740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56278e92fd80 0x56278ed1bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:17.896 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.893+0000 7f934a433740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56278e9482f0 0x56278ed211e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:17.896 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.893+0000 7f934a433740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56278e9504d0 0x56278ed12330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:17.896 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.893+0000 7f934a433740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f933c1acc10 con 0x56278e92fd80 2024-08-30T09:13:17.897 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.893+0000 7f934a433740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f933c1aa480 con 0x56278e9482f0 2024-08-30T09:13:17.897 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.893+0000 7f934a433740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f933c1a82d0 con 0x56278e9504d0 2024-08-30T09:13:17.897 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.893+0000 7f9340a60640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56278e92fd80 0x56278ed1bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:13:17.897 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.893+0000 7f9340a60640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56278e92fd80 0x56278ed1bee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:59074/0 (socket says 172.21.15.42:59074) 2024-08-30T09:13:17.897 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.893+0000 7f9340a60640 1 -- 172.21.15.42:0/1800035988 learned_addr learned my addr 172.21.15.42:0/1800035988 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:13:17.897 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.893+0000 7f9341261640 1 --2- 172.21.15.42:0/1800035988 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56278e9482f0 0x56278ed211e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:13:17.897 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.893+0000 7f9340a60640 1 --2- 172.21.15.42:0/1800035988 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56278e9504d0 0x56278ed12330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:13:17.897 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.893+0000 7f9341261640 1 -- 172.21.15.42:0/1800035988 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56278e9504d0 msgr2=0x56278ed12330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:13:17.897 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.893+0000 7f9341261640 1 --2- 172.21.15.42:0/1800035988 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56278e9504d0 0x56278ed12330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:17.897 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.893+0000 7f9341261640 1 -- 172.21.15.42:0/1800035988 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56278e92fd80 msgr2=0x56278ed1bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:13:17.897 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.893+0000 7f9341261640 1 --2- 172.21.15.42:0/1800035988 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56278e92fd80 0x56278ed1bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:17.897 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.893+0000 7f9341261640 1 -- 172.21.15.42:0/1800035988 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56278ed28f30 con 0x56278e9482f0 2024-08-30T09:13:17.897 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.893+0000 7f9340a60640 1 --2- 172.21.15.42:0/1800035988 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56278e92fd80 0x56278ed1bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-30T09:13:17.897 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.894+0000 7f9341261640 1 --2- 172.21.15.42:0/1800035988 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56278e9482f0 0x56278ed211e0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f9338006010 tx=0x7f933802a410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2d21f1996a721477 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:13:17.898 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.894+0000 7f9326ffd640 1 -- 172.21.15.42:0/1800035988 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9338057070 con 0x56278e9482f0 2024-08-30T09:13:17.898 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.894+0000 7f9326ffd640 1 -- 172.21.15.42:0/1800035988 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f933803ba70 con 0x56278e9482f0 2024-08-30T09:13:17.898 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.894+0000 7f9326ffd640 1 -- 172.21.15.42:0/1800035988 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9338035040 con 0x56278e9482f0 2024-08-30T09:13:17.901 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.895+0000 7f934a433740 1 -- 172.21.15.42:0/1800035988 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56278e9482f0 msgr2=0x56278ed211e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:13:17.901 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.895+0000 7f934a433740 1 --2- 172.21.15.42:0/1800035988 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56278e9482f0 0x56278ed211e0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f9338006010 tx=0x7f933802a410 comp rx=0 tx=0).stop 2024-08-30T09:13:17.901 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.895+0000 7f934a433740 1 -- 172.21.15.42:0/1800035988 shutdown_connections 2024-08-30T09:13:17.901 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.895+0000 7f934a433740 1 --2- 172.21.15.42:0/1800035988 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56278e9504d0 0x56278ed12330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:17.901 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.895+0000 7f934a433740 1 --2- 172.21.15.42:0/1800035988 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56278e9482f0 0x56278ed211e0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:17.901 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.895+0000 7f934a433740 1 --2- 172.21.15.42:0/1800035988 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56278e92fd80 0x56278ed1bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:17.901 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.895+0000 7f934a433740 1 -- 172.21.15.42:0/1800035988 >> 172.21.15.42:0/1800035988 conn(0x56278e965300 msgr2=0x56278ed20ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:13:17.901 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.895+0000 7f934a433740 1 -- 172.21.15.42:0/1800035988 shutdown_connections 2024-08-30T09:13:17.901 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.895+0000 7f934a433740 1 -- 172.21.15.42:0/1800035988 wait complete. 2024-08-30T09:13:17.901 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.895+0000 7f934a433740 1 -- start start 2024-08-30T09:13:17.901 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.895+0000 7f934a433740 1 Processor -- start 2024-08-30T09:13:17.902 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.896+0000 7f934a433740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56278e92fd80 0x56278ed23a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:17.902 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.896+0000 7f934a433740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56278e9482f0 0x56278ed24130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:17.902 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.896+0000 7f934a433740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56278e9504d0 0x56278ed24ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:17.902 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.896+0000 7f934a433740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f933c1a8940 con 0x56278e92fd80 2024-08-30T09:13:17.902 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.896+0000 7f934a433740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f933c1a9db0 con 0x56278e9504d0 2024-08-30T09:13:17.902 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.896+0000 7f934a433740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f933c1a9010 con 0x56278e9482f0 2024-08-30T09:13:17.902 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.896+0000 7f9340a60640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56278e92fd80 0x56278ed23a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:13:17.902 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.896+0000 7f934a433740 10 client.0 _ll_register_callbacks cb 0x7f932c005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:13:17.902 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.896+0000 7f9340a60640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56278e92fd80 0x56278ed23a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:59080/0 (socket says 172.21.15.42:59080) 2024-08-30T09:13:17.902 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.896+0000 7f9340a60640 1 -- 172.21.15.42:0/2928601662 learned_addr learned my addr 172.21.15.42:0/2928601662 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:13:17.902 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.896+0000 7f9341261640 1 --2- 172.21.15.42:0/2928601662 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56278e9482f0 0x56278ed24130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:13:17.902 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.896+0000 7f9340a60640 1 --2- 172.21.15.42:0/2928601662 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56278e9504d0 0x56278ed24ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:13:17.902 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.896+0000 7f9340a60640 1 -- 172.21.15.42:0/2928601662 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56278e9482f0 msgr2=0x56278ed24130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:13:17.903 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.896+0000 7f9340a60640 1 --2- 172.21.15.42:0/2928601662 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56278e9482f0 0x56278ed24130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:17.903 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.896+0000 7f9340a60640 1 -- 172.21.15.42:0/2928601662 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56278e9504d0 msgr2=0x56278ed24ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:13:17.903 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.896+0000 7f9340a60640 1 --2- 172.21.15.42:0/2928601662 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56278e9504d0 0x56278ed24ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:17.903 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.896+0000 7f9340a60640 1 -- 172.21.15.42:0/2928601662 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56278ed25400 con 0x56278e92fd80 2024-08-30T09:13:17.903 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.897+0000 7f9340a60640 1 --2- 172.21.15.42:0/2928601662 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56278e92fd80 0x56278ed23a70 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f932c011ca0 tx=0x7f932c009fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=410caf024b39e5e6 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:13:17.903 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.897+0000 7f9324ff9640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-08-30T09:13:17.903 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.897+0000 7f9324ff9640 1 -- 172.21.15.42:0/2928601662 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f932c04e060 con 0x56278e92fd80 2024-08-30T09:13:17.903 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.897+0000 7f9324ff9640 1 -- 172.21.15.42:0/2928601662 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f932c020410 con 0x56278e92fd80 2024-08-30T09:13:17.903 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.897+0000 7f934a433740 1 -- 172.21.15.42:0/2928601662 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f933803ba70 con 0x56278e92fd80 2024-08-30T09:13:17.903 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.898+0000 7f9324ff9640 1 -- 172.21.15.42:0/2928601662 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f932c020610 con 0x56278e92fd80 2024-08-30T09:13:17.903 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.898+0000 7f934a433740 10 client.17487 fetch_fsmap learned FSMap version 45 2024-08-30T09:13:17.903 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.898+0000 7f934a433740 1 -- 172.21.15.42:0/2928601662 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x56278ed28750 con 0x56278e92fd80 2024-08-30T09:13:17.903 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.898+0000 7f9324ff9640 1 -- 172.21.15.42:0/2928601662 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f933803ba70 con 0x56278e92fd80 2024-08-30T09:13:17.903 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.899+0000 7f9324ff9640 1 -- 172.21.15.42:0/2928601662 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 45) ==== 8362+0+0 (secure 0 0 0) 0x7f932c03f040 con 0x56278e92fd80 2024-08-30T09:13:17.904 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.899+0000 7f934a433740 10 client.17487 fetch_fsmap finished waiting for FSMap version 45 2024-08-30T09:13:17.904 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.899+0000 7f934a433740 10 client.17487 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-30T09:13:17.904 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.899+0000 7f934a433740 20 client.17487 populate_metadata read hostname 'smithi042' 2024-08-30T09:13:17.904 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.899+0000 7f934a433740 1 --2- 172.21.15.42:0/2928601662 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x56278e9711b0 0x56278ed9ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:17.904 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.899+0000 7f934a433740 4 client.17487 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-30T09:13:17.904 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.899+0000 7f934a433740 1 -- 172.21.15.42:0/2928601662 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56278ec879f0 con 0x56278e9711b0 2024-08-30T09:13:17.904 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.899+0000 7f9341261640 1 --2- 172.21.15.42:0/2928601662 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x56278e9711b0 0x56278ed9ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:13:17.904 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.899+0000 7f9324ff9640 10 client.17487 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:13:17.905 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.899+0000 7f9341261640 1 --2- 172.21.15.42:0/2928601662 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x56278e9711b0 0x56278ed9ad70 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=43c32946f48a8fa1 server_cookie=1fb2f6373db9f4b6 in_seq=0 out_seq=0 2024-08-30T09:13:17.932 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.930+0000 7f9324ff9640 1 -- 172.21.15.42:0/2928601662 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56278ec879f0 con 0x56278e9711b0 2024-08-30T09:13:17.932 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.930+0000 7f9324ff9640 10 client.17487 handle_command_reply: tid=0 2024-08-30T09:13:17.933 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.930+0000 7f934a433740 1 client.17487 shutdown 2024-08-30T09:13:17.933 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.930+0000 7f934a433740 1 -- 172.21.15.42:0/2928601662 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56278e92fd80 msgr2=0x56278ed23a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:13:17.933 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.930+0000 7f934a433740 1 --2- 172.21.15.42:0/2928601662 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56278e92fd80 0x56278ed23a70 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f932c011ca0 tx=0x7f932c009fb0 comp rx=0 tx=0).stop 2024-08-30T09:13:17.933 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.930+0000 7f934a433740 1 -- 172.21.15.42:0/2928601662 shutdown_connections 2024-08-30T09:13:17.933 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.930+0000 7f934a433740 1 --2- 172.21.15.42:0/2928601662 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x56278e9711b0 0x56278ed9ad70 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:17.933 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.930+0000 7f934a433740 1 --2- 172.21.15.42:0/2928601662 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56278e9482f0 0x56278ed24130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:17.933 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.930+0000 7f934a433740 1 --2- 172.21.15.42:0/2928601662 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56278e9504d0 0x56278ed24ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:17.933 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.930+0000 7f934a433740 1 --2- 172.21.15.42:0/2928601662 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56278e92fd80 0x56278ed23a70 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:17.933 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.930+0000 7f934a433740 1 -- 172.21.15.42:0/2928601662 >> 172.21.15.42:0/2928601662 conn(0x56278e965300 msgr2=0x56278ed12330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:13:17.933 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.930+0000 7f9324ff9640 0 client.17487 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:13:17.933 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.930+0000 7f934a433740 1 -- 172.21.15.42:0/2928601662 shutdown_connections 2024-08-30T09:13:17.933 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.931+0000 7f934a433740 1 -- 172.21.15.42:0/2928601662 wait complete. 2024-08-30T09:13:17.933 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.931+0000 7f934a433740 20 client.17487 trim_cache size 0 max 16384 2024-08-30T09:13:17.934 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-08-30T09:13:17.934 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2024-08-30T09:13:17.934 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-08-30T09:13:17.934 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-08-30T09:13:17.935 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.931+0000 7f93434c5640 1 -- 172.21.15.42:0/729710775 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f93280b4bb0 msgr2=0x7f93280b70c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:13:17.935 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.931+0000 7f93434c5640 1 --2- 172.21.15.42:0/729710775 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f93280b4bb0 0x7f93280b70c0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f93340a74b0 tx=0x7f9334009f10 comp rx=0 tx=0).stop 2024-08-30T09:13:17.935 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.931+0000 7f93434c5640 1 -- 172.21.15.42:0/729710775 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f933c103d70 msgr2=0x7f933c198d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:13:17.935 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.931+0000 7f93434c5640 1 --2- 172.21.15.42:0/729710775 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f933c103d70 0x7f933c198d10 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f9334009e20 tx=0x7f933400e490 comp rx=0 tx=0).stop 2024-08-30T09:13:17.935 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.931+0000 7f93434c5640 1 -- 172.21.15.42:0/729710775 shutdown_connections 2024-08-30T09:13:17.935 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.931+0000 7f93434c5640 1 --2- 172.21.15.42:0/729710775 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f93280b4bb0 0x7f93280b70c0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:17.935 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.931+0000 7f93434c5640 1 --2- 172.21.15.42:0/729710775 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f933c103d70 0x7f933c198d10 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:17.935 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.931+0000 7f93434c5640 1 --2- 172.21.15.42:0/729710775 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f933c1033d0 0x7f933c198160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:17.935 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.931+0000 7f93434c5640 1 --2- 172.21.15.42:0/729710775 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f933c102350 0x7f933c197af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:17.935 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.931+0000 7f93434c5640 1 -- 172.21.15.42:0/729710775 >> 172.21.15.42:0/729710775 conn(0x7f933c0fdcd0 msgr2=0x7f933c1066f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:13:17.935 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.931+0000 7f93434c5640 1 -- 172.21.15.42:0/729710775 shutdown_connections 2024-08-30T09:13:17.935 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:17.931+0000 7f93434c5640 1 -- 172.21.15.42:0/729710775 wait complete. 2024-08-30T09:13:19.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:13:18 smithi190 ceph-mon[22001]: pgmap v749: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 4.5 MiB/s wr, 427 op/s 2024-08-30T09:13:19.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:13:18 smithi167 ceph-mon[22063]: pgmap v749: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 4.5 MiB/s wr, 427 op/s 2024-08-30T09:13:19.382 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:13:18 smithi042 ceph-mon[18652]: pgmap v749: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 4.5 MiB/s wr, 427 op/s 2024-08-30T09:13:20.953 INFO:teuthology.orchestra.run.smithi042.stderr:progress: 755.0 s, 467.8 tps, lat 208.412 ms stddev 249.726 2024-08-30T09:13:20.994 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-30T09:13:21.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:13:20 smithi190 ceph-mon[22001]: pgmap v750: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 5.6 MiB/s wr, 556 op/s 2024-08-30T09:13:21.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:13:20 smithi167 ceph-mon[22063]: pgmap v750: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 5.6 MiB/s wr, 556 op/s 2024-08-30T09:13:21.385 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:13:20 smithi042 ceph-mon[18652]: pgmap v750: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 5.6 MiB/s wr, 556 op/s 2024-08-30T09:13:21.513 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T09:13:22.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.134+0000 7f46d0d2a640 1 -- 172.21.15.42:0/1163470728 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f46c40a50f0 msgr2=0x7f46c40a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:13:22.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.134+0000 7f46d0d2a640 1 --2- 172.21.15.42:0/1163470728 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f46c40a50f0 0x7f46c40a5500 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f46c0009900 tx=0x7f46c002a440 comp rx=0 tx=0).stop 2024-08-30T09:13:22.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.134+0000 7f46d0d2a640 1 -- 172.21.15.42:0/1163470728 shutdown_connections 2024-08-30T09:13:22.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.134+0000 7f46d0d2a640 1 --2- 172.21.15.42:0/1163470728 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f46c40ab1b0 0x7f46c40ab9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:22.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.134+0000 7f46d0d2a640 1 --2- 172.21.15.42:0/1163470728 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f46c40aa860 0x7f46c40aac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:22.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.134+0000 7f46d0d2a640 1 --2- 172.21.15.42:0/1163470728 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f46c40a50f0 0x7f46c40a5500 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:22.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.134+0000 7f46d0d2a640 1 -- 172.21.15.42:0/1163470728 >> 172.21.15.42:0/1163470728 conn(0x7f46c401cc70 msgr2=0x7f46c401d080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:13:22.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.136+0000 7f46d0d2a640 1 -- 172.21.15.42:0/1163470728 shutdown_connections 2024-08-30T09:13:22.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.136+0000 7f46d0d2a640 1 -- 172.21.15.42:0/1163470728 wait complete. 2024-08-30T09:13:22.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.137+0000 7f46d0d2a640 1 Processor -- start 2024-08-30T09:13:22.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.138+0000 7f46d0d2a640 1 -- start start 2024-08-30T09:13:22.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.138+0000 7f46d0d2a640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f46c40a50f0 0x7f46c40bae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:22.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.138+0000 7f46d0d2a640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f46c40aa860 0x7f46c40b4110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:22.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.138+0000 7f46d0d2a640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f46c40ab1b0 0x7f46c40b4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:22.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.138+0000 7f46d0d2a640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f46c40bd770 con 0x7f46c40a50f0 2024-08-30T09:13:22.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.138+0000 7f46d0d2a640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f46c40bd5f0 con 0x7f46c40ab1b0 2024-08-30T09:13:22.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.138+0000 7f46d0d2a640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f46c40bd8f0 con 0x7f46c40aa860 2024-08-30T09:13:22.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.139+0000 7f46cb7fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f46c40a50f0 0x7f46c40bae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:13:22.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.139+0000 7f46cb7fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f46c40a50f0 0x7f46c40bae40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60286/0 (socket says 172.21.15.42:60286) 2024-08-30T09:13:22.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.139+0000 7f46cb7fe640 1 -- 172.21.15.42:0/4259042745 learned_addr learned my addr 172.21.15.42:0/4259042745 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:13:22.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.139+0000 7f46cbfff640 1 --2- 172.21.15.42:0/4259042745 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f46c40ab1b0 0x7f46c40b4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:13:22.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.139+0000 7f46cb7fe640 1 -- 172.21.15.42:0/4259042745 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f46c40aa860 msgr2=0x7f46c40b4110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:13:22.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.139+0000 7f46cb7fe640 1 --2- 172.21.15.42:0/4259042745 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f46c40aa860 0x7f46c40b4110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:22.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.139+0000 7f46cb7fe640 1 -- 172.21.15.42:0/4259042745 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f46c40ab1b0 msgr2=0x7f46c40b4690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:13:22.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.139+0000 7f46cb7fe640 1 --2- 172.21.15.42:0/4259042745 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f46c40ab1b0 0x7f46c40b4690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:22.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.139+0000 7f46cb7fe640 1 -- 172.21.15.42:0/4259042745 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46c40b4eb0 con 0x7f46c40a50f0 2024-08-30T09:13:22.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.141+0000 7f46cb7fe640 1 --2- 172.21.15.42:0/4259042745 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f46c40a50f0 0x7f46c40bae40 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f46c0006010 tx=0x7f46c0005720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:13:22.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.142+0000 7f46c8ff9640 1 -- 172.21.15.42:0/4259042745 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46c003f030 con 0x7f46c40a50f0 2024-08-30T09:13:22.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.142+0000 7f46c8ff9640 1 -- 172.21.15.42:0/4259042745 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f46c0004e80 con 0x7f46c40a50f0 2024-08-30T09:13:22.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.142+0000 7f46c8ff9640 1 -- 172.21.15.42:0/4259042745 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46c0035070 con 0x7f46c40a50f0 2024-08-30T09:13:22.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.143+0000 7f46d0d2a640 1 -- 172.21.15.42:0/4259042745 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46c40b9b10 con 0x7f46c40a50f0 2024-08-30T09:13:22.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.143+0000 7f46d0d2a640 1 -- 172.21.15.42:0/4259042745 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46c40ba020 con 0x7f46c40a50f0 2024-08-30T09:13:22.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.148+0000 7f46d8501740 1 Processor -- start 2024-08-30T09:13:22.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.149+0000 7f46d8501740 1 -- start start 2024-08-30T09:13:22.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.149+0000 7f46d8501740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a391784070 0x55a391b294a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:22.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.149+0000 7f46d8501740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a39175b1b0 0x55a391b299e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:22.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.149+0000 7f46d8501740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a391763390 0x55a391b29f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:22.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.149+0000 7f46d8501740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55a391aa0630 con 0x55a391763390 2024-08-30T09:13:22.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.149+0000 7f46d8501740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55a391a8fbc0 con 0x55a39175b1b0 2024-08-30T09:13:22.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.149+0000 7f46d8501740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55a391a93b70 con 0x55a391784070 2024-08-30T09:13:22.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.149+0000 7f46c8ff9640 1 -- 172.21.15.42:0/4259042745 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7f46c0042030 con 0x7f46c40a50f0 2024-08-30T09:13:22.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.149+0000 7f46cb7fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a39175b1b0 0x55a391b299e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:13:22.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.149+0000 7f46cb7fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a39175b1b0 0x55a391b299e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:54898/0 (socket says 172.21.15.42:54898) 2024-08-30T09:13:22.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.149+0000 7f46cb7fe640 1 -- 172.21.15.42:0/1158674088 learned_addr learned my addr 172.21.15.42:0/1158674088 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:13:22.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.152+0000 7f46cbfff640 1 --2- 172.21.15.42:0/1158674088 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a391784070 0x55a391b294a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:13:22.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.153+0000 7f46c8ff9640 1 --2- 172.21.15.42:0/4259042745 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f46bc0b4d30 0x7f46bc0b7240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:22.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.151+0000 7f46caffd640 1 --2- 172.21.15.42:0/1158674088 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a391763390 0x55a391b29f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:13:22.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.154+0000 7f46cbfff640 1 --2- 172.21.15.42:0/4259042745 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f46bc0b4d30 0x7f46bc0b7240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:13:22.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.154+0000 7f46c8ff9640 1 -- 172.21.15.42:0/4259042745 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f46c0105c30 con 0x7f46c40a50f0 2024-08-30T09:13:22.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.154+0000 7f46cbfff640 1 -- 172.21.15.42:0/1158674088 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a39175b1b0 msgr2=0x55a391b299e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:13:22.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.154+0000 7f46cbfff640 1 --2- 172.21.15.42:0/1158674088 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a39175b1b0 0x55a391b299e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:22.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.154+0000 7f46cbfff640 1 -- 172.21.15.42:0/1158674088 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a391763390 msgr2=0x55a391b29f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:13:22.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.154+0000 7f46cbfff640 1 --2- 172.21.15.42:0/1158674088 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a391763390 0x55a391b29f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:22.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.154+0000 7f46cbfff640 1 -- 172.21.15.42:0/1158674088 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a3917ef650 con 0x55a391784070 2024-08-30T09:13:22.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.157+0000 7f46cbfff640 1 --2- 172.21.15.42:0/1158674088 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a391784070 0x55a391b294a0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f46cc052600 tx=0x7f46cc06dd70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=506dc07ce93af43b server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:13:22.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.159+0000 7f46b5ffb640 1 -- 172.21.15.42:0/1158674088 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46cc076050 con 0x55a391784070 2024-08-30T09:13:22.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.159+0000 7f46b5ffb640 1 -- 172.21.15.42:0/1158674088 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f46cc077040 con 0x55a391784070 2024-08-30T09:13:22.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.160+0000 7f46d8501740 1 -- 172.21.15.42:0/1158674088 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a391784070 msgr2=0x55a391b294a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:13:22.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.160+0000 7f46d8501740 1 --2- 172.21.15.42:0/1158674088 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a391784070 0x55a391b294a0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f46cc052600 tx=0x7f46cc06dd70 comp rx=0 tx=0).stop 2024-08-30T09:13:22.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.161+0000 7f46cbfff640 1 --2- 172.21.15.42:0/4259042745 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f46bc0b4d30 0x7f46bc0b7240 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f46cc068ae0 tx=0x7f46cc082000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:13:22.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.162+0000 7f46d8501740 1 -- 172.21.15.42:0/1158674088 shutdown_connections 2024-08-30T09:13:22.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.162+0000 7f46d8501740 1 --2- 172.21.15.42:0/1158674088 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a391763390 0x55a391b29f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:22.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.162+0000 7f46d8501740 1 --2- 172.21.15.42:0/1158674088 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a39175b1b0 0x55a391b299e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:22.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.162+0000 7f46d8501740 1 --2- 172.21.15.42:0/1158674088 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a391784070 0x55a391b294a0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:22.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.162+0000 7f46d8501740 1 -- 172.21.15.42:0/1158674088 >> 172.21.15.42:0/1158674088 conn(0x55a391778460 msgr2=0x55a391b27390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:13:22.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.165+0000 7f46d8501740 1 -- 172.21.15.42:0/1158674088 shutdown_connections 2024-08-30T09:13:22.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.168+0000 7f46d8501740 1 -- 172.21.15.42:0/1158674088 wait complete. 2024-08-30T09:13:22.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.168+0000 7f46d8501740 1 -- start start 2024-08-30T09:13:22.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.168+0000 7f46d8501740 1 Processor -- start 2024-08-30T09:13:22.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.169+0000 7f46d8501740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a39175b1b0 0x55a391b34bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:22.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.169+0000 7f46d8501740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a391763390 0x55a391b35100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:22.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.169+0000 7f46d8501740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a391784070 0x55a391b40f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:22.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.169+0000 7f46d8501740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55a3918f2700 con 0x55a39175b1b0 2024-08-30T09:13:22.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.169+0000 7f46d8501740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55a391a16700 con 0x55a391784070 2024-08-30T09:13:22.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.169+0000 7f46d8501740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55a391b37c20 con 0x55a391763390 2024-08-30T09:13:22.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.169+0000 7f46d8501740 10 client.0 _ll_register_callbacks cb 0x55a391a45dc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:13:22.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.170+0000 7f46cbfff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a391784070 0x55a391b40f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:13:22.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.170+0000 7f46cbfff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a391784070 0x55a391b40f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:54904/0 (socket says 172.21.15.42:54904) 2024-08-30T09:13:22.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.170+0000 7f46cbfff640 1 -- 172.21.15.42:0/104462958 learned_addr learned my addr 172.21.15.42:0/104462958 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:13:22.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.175+0000 7f46caffd640 1 --2- 172.21.15.42:0/104462958 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a391763390 0x55a391b35100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:13:22.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.175+0000 7f46cbfff640 1 -- 172.21.15.42:0/104462958 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a391763390 msgr2=0x55a391b35100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:13:22.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.175+0000 7f46cbfff640 1 --2- 172.21.15.42:0/104462958 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a391763390 0x55a391b35100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:22.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.175+0000 7f46cb7fe640 1 --2- 172.21.15.42:0/104462958 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a39175b1b0 0x55a391b34bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:13:22.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.175+0000 7f46cbfff640 1 -- 172.21.15.42:0/104462958 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a39175b1b0 msgr2=0x55a391b34bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:13:22.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.175+0000 7f46cbfff640 1 --2- 172.21.15.42:0/104462958 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a39175b1b0 0x55a391b34bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:22.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.175+0000 7f46cbfff640 1 -- 172.21.15.42:0/104462958 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a391a4f8b0 con 0x55a391784070 2024-08-30T09:13:22.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.175+0000 7f46cb7fe640 1 --2- 172.21.15.42:0/104462958 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a39175b1b0 0x55a391b34bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-30T09:13:22.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.182+0000 7f46cbfff640 1 --2- 172.21.15.42:0/104462958 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a391784070 0x55a391b40f60 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f46cc066a10 tx=0x7f46cc051c70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d8dfa7b9237e9c1 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:13:22.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.182+0000 7f46aaffd640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-30T09:13:22.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.186+0000 7f46aaffd640 1 -- 172.21.15.42:0/104462958 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46cc094060 con 0x55a391784070 2024-08-30T09:13:22.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.186+0000 7f46aaffd640 1 -- 172.21.15.42:0/104462958 <== mon.2 v2:172.21.15.167:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f46cc07fd80 con 0x55a391784070 2024-08-30T09:13:22.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.186+0000 7f46d8501740 1 -- 172.21.15.42:0/104462958 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55a391aa2580 con 0x55a391784070 2024-08-30T09:13:22.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.187+0000 7f46aaffd640 1 -- 172.21.15.42:0/104462958 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f46cc06d070 con 0x55a391784070 2024-08-30T09:13:22.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.188+0000 7f46d8501740 10 client.27329 fetch_fsmap learned FSMap version 45 2024-08-30T09:13:22.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.188+0000 7f46d8501740 1 -- 172.21.15.42:0/104462958 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55a391b43ac0 con 0x55a391784070 2024-08-30T09:13:22.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.192+0000 7f46aaffd640 1 -- 172.21.15.42:0/104462958 <== mon.2 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55a391aa2580 con 0x55a391784070 2024-08-30T09:13:22.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.192+0000 7f46aaffd640 1 -- 172.21.15.42:0/104462958 <== mon.2 v2:172.21.15.167:3300/0 5 ==== fsmap(e 45) ==== 8362+0+0 (secure 0 0 0) 0x7f46cc0635a0 con 0x55a391784070 2024-08-30T09:13:22.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.193+0000 7f46d8501740 10 client.27329 fetch_fsmap finished waiting for FSMap version 45 2024-08-30T09:13:22.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.193+0000 7f46d8501740 10 client.27329 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-30T09:13:22.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.193+0000 7f46d8501740 20 client.27329 populate_metadata read hostname 'smithi042' 2024-08-30T09:13:22.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.193+0000 7f46d8501740 1 --2- 172.21.15.42:0/104462958 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55a3917429f0 0x55a391b46f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:22.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.193+0000 7f46d8501740 4 client.27329 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-30T09:13:22.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.193+0000 7f46d8501740 1 -- 172.21.15.42:0/104462958 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a391a9e020 con 0x55a3917429f0 2024-08-30T09:13:22.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.195+0000 7f46cb7fe640 1 --2- 172.21.15.42:0/104462958 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55a3917429f0 0x55a391b46f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:13:22.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.199+0000 7f46cb7fe640 1 --2- 172.21.15.42:0/104462958 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55a3917429f0 0x55a391b46f30 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5615cf940b881bae server_cookie=d87e732b497ffba2 in_seq=0 out_seq=0 2024-08-30T09:13:22.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.200+0000 7f46aaffd640 10 client.27329 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:13:22.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.203+0000 7f46aaffd640 1 -- 172.21.15.42:0/104462958 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a391a9e020 con 0x55a3917429f0 2024-08-30T09:13:22.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.203+0000 7f46aaffd640 10 client.27329 handle_command_reply: tid=0 2024-08-30T09:13:22.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.203+0000 7f46d8501740 1 client.27329 shutdown 2024-08-30T09:13:22.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.206+0000 7f46d8501740 1 -- 172.21.15.42:0/104462958 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a391784070 msgr2=0x55a391b40f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:13:22.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.206+0000 7f46d8501740 1 --2- 172.21.15.42:0/104462958 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a391784070 0x55a391b40f60 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f46cc066a10 tx=0x7f46cc051c70 comp rx=0 tx=0).stop 2024-08-30T09:13:22.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.207+0000 7f46d8501740 1 -- 172.21.15.42:0/104462958 shutdown_connections 2024-08-30T09:13:22.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.207+0000 7f46d8501740 1 --2- 172.21.15.42:0/104462958 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55a3917429f0 0x55a391b46f30 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:22.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.207+0000 7f46d8501740 1 --2- 172.21.15.42:0/104462958 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a391763390 0x55a391b35100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:22.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.207+0000 7f46d8501740 1 --2- 172.21.15.42:0/104462958 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a391784070 0x55a391b40f60 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:22.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.207+0000 7f46d8501740 1 --2- 172.21.15.42:0/104462958 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a39175b1b0 0x55a391b34bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:22.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.207+0000 7f46d8501740 1 -- 172.21.15.42:0/104462958 >> 172.21.15.42:0/104462958 conn(0x55a391778460 msgr2=0x55a391b2c630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:13:22.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.207+0000 7f46aaffd640 0 client.27329 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:13:22.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.214+0000 7f46d8501740 1 -- 172.21.15.42:0/104462958 shutdown_connections 2024-08-30T09:13:22.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.215+0000 7f46d8501740 1 -- 172.21.15.42:0/104462958 wait complete. 2024-08-30T09:13:22.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.218+0000 7f46d8501740 20 client.27329 trim_cache size 0 max 16384 2024-08-30T09:13:22.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.258+0000 7f46d8501740 1 Processor -- start 2024-08-30T09:13:22.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.258+0000 7f46d8501740 1 -- start start 2024-08-30T09:13:22.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.258+0000 7f46d8501740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a39175b1b0 0x55a391b27400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:22.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.259+0000 7f46d8501740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a391763390 0x55a391b40e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:22.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.259+0000 7f46d8501740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a391784070 0x55a391b31db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:22.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.259+0000 7f46d8501740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f46c414e9b0 con 0x55a391784070 2024-08-30T09:13:22.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.259+0000 7f46d8501740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f46c414c260 con 0x55a39175b1b0 2024-08-30T09:13:22.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.259+0000 7f46d8501740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f46c40aac70 con 0x55a391763390 2024-08-30T09:13:22.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.259+0000 7f46cb7fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a39175b1b0 0x55a391b27400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:13:22.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.259+0000 7f46cb7fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a39175b1b0 0x55a391b27400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:54920/0 (socket says 172.21.15.42:54920) 2024-08-30T09:13:22.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.259+0000 7f46cb7fe640 1 -- 172.21.15.42:0/2607403856 learned_addr learned my addr 172.21.15.42:0/2607403856 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:13:22.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.259+0000 7f46cbfff640 1 --2- 172.21.15.42:0/2607403856 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a391784070 0x55a391b31db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:13:22.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.259+0000 7f46cb7fe640 1 -- 172.21.15.42:0/2607403856 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a391763390 msgr2=0x55a391b40e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:13:22.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.259+0000 7f46cb7fe640 1 --2- 172.21.15.42:0/2607403856 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a391763390 0x55a391b40e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:22.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.259+0000 7f46cb7fe640 1 -- 172.21.15.42:0/2607403856 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a391784070 msgr2=0x55a391b31db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:13:22.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.259+0000 7f46cb7fe640 1 --2- 172.21.15.42:0/2607403856 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a391784070 0x55a391b31db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:22.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.259+0000 7f46cb7fe640 1 -- 172.21.15.42:0/2607403856 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a391b33b20 con 0x55a39175b1b0 2024-08-30T09:13:22.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.260+0000 7f46cb7fe640 1 --2- 172.21.15.42:0/2607403856 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a39175b1b0 0x55a391b27400 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f46c00096c0 tx=0x7f46c003ba40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6c2d4abef2590f6b server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:13:22.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.260+0000 7f46b57fa640 1 -- 172.21.15.42:0/2607403856 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46c00d3e60 con 0x55a39175b1b0 2024-08-30T09:13:22.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.260+0000 7f46b57fa640 1 -- 172.21.15.42:0/2607403856 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f46c00ef070 con 0x55a39175b1b0 2024-08-30T09:13:22.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.260+0000 7f46b57fa640 1 -- 172.21.15.42:0/2607403856 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46c0049040 con 0x55a39175b1b0 2024-08-30T09:13:22.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.261+0000 7f46d8501740 1 -- 172.21.15.42:0/2607403856 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a39175b1b0 msgr2=0x55a391b27400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:13:22.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.261+0000 7f46d8501740 1 --2- 172.21.15.42:0/2607403856 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a39175b1b0 0x55a391b27400 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f46c00096c0 tx=0x7f46c003ba40 comp rx=0 tx=0).stop 2024-08-30T09:13:22.266 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.262+0000 7f46d8501740 1 -- 172.21.15.42:0/2607403856 shutdown_connections 2024-08-30T09:13:22.266 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.262+0000 7f46d8501740 1 --2- 172.21.15.42:0/2607403856 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a391763390 0x55a391b40e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:22.266 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.262+0000 7f46d8501740 1 --2- 172.21.15.42:0/2607403856 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a391784070 0x55a391b31db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:22.266 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.262+0000 7f46d8501740 1 --2- 172.21.15.42:0/2607403856 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a39175b1b0 0x55a391b27400 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:22.266 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.262+0000 7f46d8501740 1 -- 172.21.15.42:0/2607403856 >> 172.21.15.42:0/2607403856 conn(0x55a391778460 msgr2=0x55a391b26cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:13:22.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.267+0000 7f46d8501740 1 -- 172.21.15.42:0/2607403856 shutdown_connections 2024-08-30T09:13:22.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.272+0000 7f46d8501740 1 -- 172.21.15.42:0/2607403856 wait complete. 2024-08-30T09:13:22.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.272+0000 7f46d8501740 1 -- start start 2024-08-30T09:13:22.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.272+0000 7f46d8501740 1 Processor -- start 2024-08-30T09:13:22.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.273+0000 7f46d8501740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a39175b1b0 0x55a391b3d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:22.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.273+0000 7f46d8501740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a391763390 0x55a391b318b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:22.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.273+0000 7f46d8501740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a391784070 0x55a391b31df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:22.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.273+0000 7f46d8501740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f46c414a770 con 0x55a39175b1b0 2024-08-30T09:13:22.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.273+0000 7f46d8501740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f46c414bb90 con 0x55a391784070 2024-08-30T09:13:22.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.273+0000 7f46d8501740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f46c414adf0 con 0x55a391763390 2024-08-30T09:13:22.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.273+0000 7f46d8501740 10 client.0 _ll_register_callbacks cb 0x7f46cc087e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:13:22.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.273+0000 7f46cb7fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a39175b1b0 0x55a391b3d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:13:22.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.273+0000 7f46cb7fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a39175b1b0 0x55a391b3d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60328/0 (socket says 172.21.15.42:60328) 2024-08-30T09:13:22.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.273+0000 7f46cb7fe640 1 -- 172.21.15.42:0/3089447411 learned_addr learned my addr 172.21.15.42:0/3089447411 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:13:22.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.276+0000 7f46caffd640 1 --2- 172.21.15.42:0/3089447411 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a391763390 0x55a391b318b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:13:22.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.277+0000 7f46cbfff640 1 --2- 172.21.15.42:0/3089447411 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a391784070 0x55a391b31df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:13:22.285 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.282+0000 7f46cb7fe640 1 -- 172.21.15.42:0/3089447411 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a391763390 msgr2=0x55a391b318b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:13:22.285 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.282+0000 7f46cb7fe640 1 --2- 172.21.15.42:0/3089447411 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a391763390 0x55a391b318b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:22.285 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.282+0000 7f46cb7fe640 1 -- 172.21.15.42:0/3089447411 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a391784070 msgr2=0x55a391b31df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:13:22.285 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.282+0000 7f46cb7fe640 1 --2- 172.21.15.42:0/3089447411 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a391784070 0x55a391b31df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:22.285 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.282+0000 7f46cb7fe640 1 -- 172.21.15.42:0/3089447411 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a391b323b0 con 0x55a39175b1b0 2024-08-30T09:13:22.285 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.282+0000 7f46cb7fe640 1 --2- 172.21.15.42:0/3089447411 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a39175b1b0 0x55a391b3d4c0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f46c0000c00 tx=0x7f46c00cc360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d618792a691731e2 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:13:22.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.282+0000 7f46aaffd640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-08-30T09:13:22.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.283+0000 7f46aaffd640 1 -- 172.21.15.42:0/3089447411 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46c0049070 con 0x55a39175b1b0 2024-08-30T09:13:22.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.283+0000 7f46aaffd640 1 -- 172.21.15.42:0/3089447411 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f46c0103450 con 0x55a39175b1b0 2024-08-30T09:13:22.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.283+0000 7f46d8501740 1 -- 172.21.15.42:0/3089447411 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55a391a9b760 con 0x55a39175b1b0 2024-08-30T09:13:22.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.284+0000 7f46aaffd640 1 -- 172.21.15.42:0/3089447411 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f46c00d3e60 con 0x55a39175b1b0 2024-08-30T09:13:22.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.284+0000 7f46d8501740 10 client.17502 fetch_fsmap learned FSMap version 45 2024-08-30T09:13:22.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.284+0000 7f46d8501740 1 -- 172.21.15.42:0/3089447411 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55a391b3aca0 con 0x55a39175b1b0 2024-08-30T09:13:22.298 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.295+0000 7f46aaffd640 1 -- 172.21.15.42:0/3089447411 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55a391a9b760 con 0x55a39175b1b0 2024-08-30T09:13:22.298 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.295+0000 7f46aaffd640 1 -- 172.21.15.42:0/3089447411 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 45) ==== 8362+0+0 (secure 0 0 0) 0x7f46c003b6a0 con 0x55a39175b1b0 2024-08-30T09:13:22.298 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.295+0000 7f46d8501740 10 client.17502 fetch_fsmap finished waiting for FSMap version 45 2024-08-30T09:13:22.298 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.295+0000 7f46d8501740 10 client.17502 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-30T09:13:22.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.295+0000 7f46d8501740 20 client.17502 populate_metadata read hostname 'smithi042' 2024-08-30T09:13:22.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.295+0000 7f46d8501740 1 --2- 172.21.15.42:0/3089447411 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55a391717da0 0x55a391bb3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:22.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.295+0000 7f46d8501740 4 client.17502 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-30T09:13:22.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.295+0000 7f46d8501740 1 -- 172.21.15.42:0/3089447411 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a391a12a50 con 0x55a391717da0 2024-08-30T09:13:22.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.302+0000 7f46caffd640 1 --2- 172.21.15.42:0/3089447411 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55a391717da0 0x55a391bb3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:13:22.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.309+0000 7f46caffd640 1 --2- 172.21.15.42:0/3089447411 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55a391717da0 0x55a391bb3d40 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=67019995980f979e server_cookie=a3d5399e7821788f in_seq=0 out_seq=0 2024-08-30T09:13:22.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.309+0000 7f46aaffd640 10 client.17502 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:13:22.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.309+0000 7f46aaffd640 1 -- 172.21.15.42:0/3089447411 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+4350 (crc 0 0 0) 0x55a391a12a50 con 0x55a391717da0 2024-08-30T09:13:22.313 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.309+0000 7f46aaffd640 10 client.17502 handle_command_reply: tid=0 2024-08-30T09:13:22.313 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.309+0000 7f46d8501740 1 client.17502 shutdown 2024-08-30T09:13:22.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.311+0000 7f46d8501740 1 -- 172.21.15.42:0/3089447411 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a39175b1b0 msgr2=0x55a391b3d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:13:22.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.311+0000 7f46d8501740 1 --2- 172.21.15.42:0/3089447411 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a39175b1b0 0x55a391b3d4c0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f46c0000c00 tx=0x7f46c00cc360 comp rx=0 tx=0).stop 2024-08-30T09:13:22.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.313+0000 7f46d8501740 1 -- 172.21.15.42:0/3089447411 shutdown_connections 2024-08-30T09:13:22.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.313+0000 7f46d8501740 1 --2- 172.21.15.42:0/3089447411 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55a391717da0 0x55a391bb3d40 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:22.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.313+0000 7f46d8501740 1 --2- 172.21.15.42:0/3089447411 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a391763390 0x55a391b318b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:22.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.313+0000 7f46d8501740 1 --2- 172.21.15.42:0/3089447411 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a391784070 0x55a391b31df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:22.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.313+0000 7f46d8501740 1 --2- 172.21.15.42:0/3089447411 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a39175b1b0 0x55a391b3d4c0 secure :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f46c0000c00 tx=0x7f46c00cc360 comp rx=0 tx=0).stop 2024-08-30T09:13:22.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.313+0000 7f46aaffd640 0 client.17502 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:13:22.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.313+0000 7f46d8501740 1 -- 172.21.15.42:0/3089447411 >> 172.21.15.42:0/3089447411 conn(0x55a391778460 msgr2=0x55a391b25800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:13:22.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.314+0000 7f46d8501740 1 -- 172.21.15.42:0/3089447411 shutdown_connections 2024-08-30T09:13:22.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.314+0000 7f46d8501740 1 -- 172.21.15.42:0/3089447411 wait complete. 2024-08-30T09:13:22.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.314+0000 7f46d8501740 20 client.17502 trim_cache size 0 max 16384 2024-08-30T09:13:22.319 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-08-30T09:13:22.319 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "no active scrubs running", 2024-08-30T09:13:22.319 INFO:teuthology.orchestra.run.smithi042.stdout: "scrubs": {}, 2024-08-30T09:13:22.319 INFO:teuthology.orchestra.run.smithi042.stdout: "scrub_stats": { 2024-08-30T09:13:22.319 INFO:teuthology.orchestra.run.smithi042.stdout: "0cbf734e-f988-4b21-acac-57b55e3bde62": { 2024-08-30T09:13:22.319 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:12:45.306738+0000", 2024-08-30T09:13:22.319 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:13:22.319 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:13:22.319 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:13:22.319 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:13:22.320 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:13:22.320 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:13:22.320 INFO:teuthology.orchestra.run.smithi042.stdout: "5e2ed104-0625-478a-a78b-584e7f4c2817": { 2024-08-30T09:13:22.320 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:00:33.928400+0000", 2024-08-30T09:13:22.320 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:13:22.320 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:13:22.320 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:13:22.320 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:13:22.320 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:13:22.320 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:13:22.320 INFO:teuthology.orchestra.run.smithi042.stdout: "c827ad7a-44ee-4e1d-9bb3-51fb7ab00c73": { 2024-08-30T09:13:22.320 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:02:21.686517+0000", 2024-08-30T09:13:22.320 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:13:22.320 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:13:22.321 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:13:22.321 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:13:22.321 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:13:22.321 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:13:22.321 INFO:teuthology.orchestra.run.smithi042.stdout: "85134e4b-ab4a-4885-b07e-a3ad01eaf13d": { 2024-08-30T09:13:22.321 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:01:27.738462+0000", 2024-08-30T09:13:22.321 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:13:22.321 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:13:22.321 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:13:22.321 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:13:22.321 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:13:22.321 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:13:22.321 INFO:teuthology.orchestra.run.smithi042.stdout: "749f4867-0b70-49e5-af2d-8362a240ebc3": { 2024-08-30T09:13:22.322 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:03:16.386554+0000", 2024-08-30T09:13:22.322 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:13:22.322 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:13:22.322 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:13:22.322 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:13:22.322 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:13:22.322 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:13:22.322 INFO:teuthology.orchestra.run.smithi042.stdout: "88cd7478-2b20-47de-ba51-256479749232": { 2024-08-30T09:13:22.322 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:04:08.922625+0000", 2024-08-30T09:13:22.322 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:13:22.322 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:13:22.322 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:13:22.322 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:13:22.322 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:13:22.323 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:13:22.323 INFO:teuthology.orchestra.run.smithi042.stdout: "347d05ed-4bd3-4590-b099-314f719aab39": { 2024-08-30T09:13:22.323 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:05:00.439721+0000", 2024-08-30T09:13:22.323 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:13:22.323 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:13:22.323 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:13:22.323 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:13:22.323 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:13:22.323 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:13:22.323 INFO:teuthology.orchestra.run.smithi042.stdout: "1af4252a-4a99-47e9-bbf0-becf5e573dd5": { 2024-08-30T09:13:22.323 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:05:48.565881+0000", 2024-08-30T09:13:22.323 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:13:22.323 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:13:22.324 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:13:22.324 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:13:22.324 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:13:22.324 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:13:22.324 INFO:teuthology.orchestra.run.smithi042.stdout: "665ec740-7254-49cb-84ef-deff71c461b0": { 2024-08-30T09:13:22.324 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:06:37.144035+0000", 2024-08-30T09:13:22.324 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:13:22.324 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:13:22.324 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:13:22.324 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:13:22.324 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:13:22.324 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:13:22.325 INFO:teuthology.orchestra.run.smithi042.stdout: "598d8a75-f1cc-4388-80bc-a408bdb4eaac": { 2024-08-30T09:13:22.325 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:07:29.721122+0000", 2024-08-30T09:13:22.325 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:13:22.325 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:13:22.325 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:13:22.325 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:13:22.325 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:13:22.325 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:13:22.325 INFO:teuthology.orchestra.run.smithi042.stdout: "b62c4ffc-9387-4677-b9ba-1e94ba308f17": { 2024-08-30T09:13:22.325 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:11:03.363478+0000", 2024-08-30T09:13:22.325 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:13:22.325 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:13:22.325 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:13:22.326 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:13:22.326 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:13:22.326 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:13:22.326 INFO:teuthology.orchestra.run.smithi042.stdout: "a7210f9a-046b-4188-953c-819b9f1863a9": { 2024-08-30T09:13:22.326 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:10:07.608430+0000", 2024-08-30T09:13:22.326 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:13:22.326 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:13:22.326 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:13:22.326 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:13:22.326 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:13:22.326 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:13:22.326 INFO:teuthology.orchestra.run.smithi042.stdout: "99a29ec1-9455-4ae4-854a-398b32896675": { 2024-08-30T09:13:22.327 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:08:24.923176+0000", 2024-08-30T09:13:22.327 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:13:22.327 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:13:22.327 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:13:22.327 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:13:22.327 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:13:22.327 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:13:22.327 INFO:teuthology.orchestra.run.smithi042.stdout: "ff464e7f-123b-4bc6-bc7e-aced018168de": { 2024-08-30T09:13:22.327 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:11:54.879598+0000", 2024-08-30T09:13:22.327 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:13:22.327 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:13:22.327 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:13:22.328 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:13:22.328 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:13:22.328 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:13:22.328 INFO:teuthology.orchestra.run.smithi042.stdout: "37f2f28f-496f-41a6-b4cf-8846fb3f43ab": { 2024-08-30T09:13:22.328 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:09:17.428286+0000", 2024-08-30T09:13:22.328 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:13:22.328 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:13:22.328 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:13:22.328 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:13:22.328 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:13:22.329 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:13:22.329 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:13:22.329 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-08-30T09:13:22.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.315+0000 7f46d0d2a640 1 -- 172.21.15.42:0/4259042745 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f46bc0b4d30 msgr2=0x7f46bc0b7240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:13:22.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.315+0000 7f46d0d2a640 1 --2- 172.21.15.42:0/4259042745 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f46bc0b4d30 0x7f46bc0b7240 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f46cc068ae0 tx=0x7f46cc082000 comp rx=0 tx=0).stop 2024-08-30T09:13:22.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.315+0000 7f46d0d2a640 1 -- 172.21.15.42:0/4259042745 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f46c40a50f0 msgr2=0x7f46c40bae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:13:22.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.315+0000 7f46d0d2a640 1 --2- 172.21.15.42:0/4259042745 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f46c40a50f0 0x7f46c40bae40 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f46c0006010 tx=0x7f46c0005720 comp rx=0 tx=0).stop 2024-08-30T09:13:22.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.315+0000 7f46d0d2a640 1 -- 172.21.15.42:0/4259042745 shutdown_connections 2024-08-30T09:13:22.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.315+0000 7f46d0d2a640 1 --2- 172.21.15.42:0/4259042745 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f46bc0b4d30 0x7f46bc0b7240 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:22.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.315+0000 7f46d0d2a640 1 --2- 172.21.15.42:0/4259042745 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f46c40aa860 0x7f46c40b4110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:22.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.315+0000 7f46d0d2a640 1 --2- 172.21.15.42:0/4259042745 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f46c40ab1b0 0x7f46c40b4690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:22.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.315+0000 7f46d0d2a640 1 --2- 172.21.15.42:0/4259042745 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f46c40a50f0 0x7f46c40bae40 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:22.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.315+0000 7f46d0d2a640 1 -- 172.21.15.42:0/4259042745 >> 172.21.15.42:0/4259042745 conn(0x7f46c401cc70 msgr2=0x7f46c40a39c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:13:22.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.315+0000 7f46d0d2a640 1 -- 172.21.15.42:0/4259042745 shutdown_connections 2024-08-30T09:13:22.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:22.315+0000 7f46d0d2a640 1 -- 172.21.15.42:0/4259042745 wait complete. 2024-08-30T09:13:23.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:13:22 smithi042 ceph-mon[18652]: pgmap v751: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 4.4 MiB/s wr, 425 op/s 2024-08-30T09:13:23.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:13:22 smithi190 ceph-mon[22001]: pgmap v751: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 4.4 MiB/s wr, 425 op/s 2024-08-30T09:13:23.217 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-08-30T09:13:23.218 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph fs dump --format=json 2024-08-30T09:13:23.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:13:22 smithi167 ceph-mon[22063]: pgmap v751: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 4.4 MiB/s wr, 425 op/s 2024-08-30T09:13:23.729 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T09:13:25.136 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:13:24 smithi042 ceph-mon[18652]: pgmap v752: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 5.7 MiB/s wr, 550 op/s 2024-08-30T09:13:25.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:13:24 smithi167 ceph-mon[22063]: pgmap v752: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 5.7 MiB/s wr, 550 op/s 2024-08-30T09:13:25.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:13:24 smithi190 ceph-mon[22001]: pgmap v752: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 5.7 MiB/s wr, 550 op/s 2024-08-30T09:13:25.953 INFO:teuthology.orchestra.run.smithi042.stderr:progress: 760.0 s, 411.0 tps, lat 238.584 ms stddev 279.711 2024-08-30T09:13:26.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:25.040+0000 7f762ad61640 1 -- 172.21.15.42:0/3148152198 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7624111ce0 msgr2=0x7f762410a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:13:26.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:25.040+0000 7f762ad61640 1 --2- 172.21.15.42:0/3148152198 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7624111ce0 0x7f762410a160 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f7618009900 tx=0x7f761802a440 comp rx=0 tx=0).stop 2024-08-30T09:13:26.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:25.041+0000 7f762ad61640 1 -- 172.21.15.42:0/3148152198 shutdown_connections 2024-08-30T09:13:26.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:25.041+0000 7f762ad61640 1 --2- 172.21.15.42:0/3148152198 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f762411c9d0 0x7f762411eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:26.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:25.041+0000 7f762ad61640 1 --2- 172.21.15.42:0/3148152198 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f762410a780 0x7f762410ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:26.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:25.041+0000 7f762ad61640 1 --2- 172.21.15.42:0/3148152198 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7624111ce0 0x7f762410a160 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:26.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:25.041+0000 7f762ad61640 1 -- 172.21.15.42:0/3148152198 >> 172.21.15.42:0/3148152198 conn(0x7f7624070c60 msgr2=0x7f7624071070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:13:26.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:25.041+0000 7f762ad61640 1 -- 172.21.15.42:0/3148152198 shutdown_connections 2024-08-30T09:13:26.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:25.041+0000 7f762ad61640 1 -- 172.21.15.42:0/3148152198 wait complete. 2024-08-30T09:13:26.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:25.042+0000 7f762ad61640 1 Processor -- start 2024-08-30T09:13:26.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:25.042+0000 7f762ad61640 1 -- start start 2024-08-30T09:13:26.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:25.042+0000 7f762ad61640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f762410a780 0x7f76241a90a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:26.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:25.042+0000 7f762ad61640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7624111ce0 0x7f76241a9640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:26.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:25.042+0000 7f762ad61640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f762411c9d0 0x7f76241ae8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:26.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:25.042+0000 7f762ad61640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f76241226b0 con 0x7f7624111ce0 2024-08-30T09:13:26.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:25.042+0000 7f762ad61640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f7624122530 con 0x7f762411c9d0 2024-08-30T09:13:26.252 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-08-30T09:13:26.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:25.042+0000 7f762ad61640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f7624122830 con 0x7f762410a780 2024-08-30T09:13:26.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:25.042+0000 7f7628afd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f762410a780 0x7f76241a90a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:13:26.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:25.042+0000 7f7623fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7624111ce0 0x7f76241a9640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:13:26.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:25.042+0000 7f7628afd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f762410a780 0x7f76241a90a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:40334/0 (socket says 172.21.15.42:40334) 2024-08-30T09:13:26.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:25.042+0000 7f7628afd640 1 -- 172.21.15.42:0/3386868397 learned_addr learned my addr 172.21.15.42:0/3386868397 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:13:26.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:25.043+0000 7f76292fe640 1 --2- 172.21.15.42:0/3386868397 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f762411c9d0 0x7f76241ae8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:13:26.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:25.043+0000 7f7623fff640 1 -- 172.21.15.42:0/3386868397 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f762410a780 msgr2=0x7f76241a90a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:13:26.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:25.043+0000 7f7623fff640 1 --2- 172.21.15.42:0/3386868397 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f762410a780 0x7f76241a90a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:26.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:25.043+0000 7f7623fff640 1 -- 172.21.15.42:0/3386868397 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f762411c9d0 msgr2=0x7f76241ae8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:13:26.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:25.043+0000 7f7623fff640 1 --2- 172.21.15.42:0/3386868397 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f762411c9d0 0x7f76241ae8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:26.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:25.043+0000 7f7623fff640 1 -- 172.21.15.42:0/3386868397 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76241af010 con 0x7f7624111ce0 2024-08-30T09:13:26.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:25.043+0000 7f7628afd640 1 --2- 172.21.15.42:0/3386868397 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f762410a780 0x7f76241a90a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-30T09:13:26.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:25.043+0000 7f7623fff640 1 --2- 172.21.15.42:0/3386868397 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7624111ce0 0x7f76241a9640 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f762406c480 tx=0x7f761c00be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:13:26.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:25.043+0000 7f7621ffb640 1 -- 172.21.15.42:0/3386868397 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f761c019070 con 0x7f7624111ce0 2024-08-30T09:13:26.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:25.043+0000 7f762ad61640 1 -- 172.21.15.42:0/3386868397 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76241af300 con 0x7f7624111ce0 2024-08-30T09:13:26.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:25.043+0000 7f7621ffb640 1 -- 172.21.15.42:0/3386868397 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f761c0148f0 con 0x7f7624111ce0 2024-08-30T09:13:26.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:25.043+0000 7f7621ffb640 1 -- 172.21.15.42:0/3386868397 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f761c012880 con 0x7f7624111ce0 2024-08-30T09:13:26.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:25.044+0000 7f762ad61640 1 -- 172.21.15.42:0/3386868397 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76241b5ae0 con 0x7f7624111ce0 2024-08-30T09:13:26.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:25.045+0000 7f760b7fe640 1 -- 172.21.15.42:0/3386868397 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7604005290 con 0x7f7624111ce0 2024-08-30T09:13:26.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:25.046+0000 7f7621ffb640 1 -- 172.21.15.42:0/3386868397 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7f761c01c070 con 0x7f7624111ce0 2024-08-30T09:13:26.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:25.047+0000 7f7621ffb640 1 --2- 172.21.15.42:0/3386868397 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f760c0b4c30 0x7f760c0b7140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:26.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:25.047+0000 7f7621ffb640 1 -- 172.21.15.42:0/3386868397 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f761c0e51e0 con 0x7f7624111ce0 2024-08-30T09:13:26.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:25.047+0000 7f7628afd640 1 --2- 172.21.15.42:0/3386868397 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f760c0b4c30 0x7f760c0b7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:13:26.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:25.048+0000 7f7628afd640 1 --2- 172.21.15.42:0/3386868397 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f760c0b4c30 0x7f760c0b7140 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f7618005de0 tx=0x7f7618005d10 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:13:26.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:25.051+0000 7f7621ffb640 1 -- 172.21.15.42:0/3386868397 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f761c0ab8d0 con 0x7f7624111ce0 2024-08-30T09:13:26.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:25.408+0000 7f760b7fe640 1 -- 172.21.15.42:0/3386868397 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7604005580 con 0x7f7624111ce0 2024-08-30T09:13:26.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:25.409+0000 7f7621ffb640 1 -- 172.21.15.42:0/3386868397 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) ==== 94+0+11918 (secure 0 0 0) 0x7f761c0b0090 con 0x7f7624111ce0 2024-08-30T09:13:26.254 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 45 2024-08-30T09:13:26.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:25.411+0000 7f760b7fe640 1 -- 172.21.15.42:0/3386868397 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f760c0b4c30 msgr2=0x7f760c0b7140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:13:26.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:25.411+0000 7f760b7fe640 1 --2- 172.21.15.42:0/3386868397 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f760c0b4c30 0x7f760c0b7140 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f7618005de0 tx=0x7f7618005d10 comp rx=0 tx=0).stop 2024-08-30T09:13:26.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:25.411+0000 7f760b7fe640 1 -- 172.21.15.42:0/3386868397 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7624111ce0 msgr2=0x7f76241a9640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:13:26.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:25.411+0000 7f760b7fe640 1 --2- 172.21.15.42:0/3386868397 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7624111ce0 0x7f76241a9640 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f762406c480 tx=0x7f761c00be00 comp rx=0 tx=0).stop 2024-08-30T09:13:26.255 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":45,"btime":"2024-08-30T09:13:13:870384+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14703,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/3188400000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":3188400000},{"type":"v1","addr":"172.21.15.42:6835","nonce":3188400000}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14715,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/3356537674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":3356537674},{"type":"v1","addr":"172.21.15.42:6837","nonce":3356537674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14766,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6839/2107785513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6838","nonce":2107785513},{"type":"v1","addr":"172.21.15.42:6839","nonce":2107785513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14778,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6841/2298438674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6840","nonce":2298438674},{"type":"v1","addr":"172.21.15.42:6841","nonce":2298438674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24553,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/543555570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":543555570},{"type":"v1","addr":"172.21.15.190:6833","nonce":543555570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24554,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/3907818941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":3907818941},{"type":"v1","addr":"172.21.15.167:6833","nonce":3907818941}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":45,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-30T08:58:03.055084+0000","modified":"2024-08-30T09:13:13.341590+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24629,"mds_1":24610,"mds_2":24599},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24572":{"gid":24572,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6835/234596845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":234596845},{"type":"v1","addr":"172.21.15.167:6835","nonce":234596845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24592":{"gid":24592,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6835/247905709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":247905709},{"type":"v1","addr":"172.21.15.190:6835","nonce":247905709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24598":{"gid":24598,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6837/863256349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6836","nonce":863256349},{"type":"v1","addr":"172.21.15.190:6837","nonce":863256349}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24599":{"gid":24599,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.167:6837/3011837856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":3011837856},{"type":"v1","addr":"172.21.15.167:6837","nonce":3011837856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24610":{"gid":24610,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.190:6839/719981110","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6838","nonce":719981110},{"type":"v1","addr":"172.21.15.190:6839","nonce":719981110}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24629":{"gid":24629,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.167:6839/3604645207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":3604645207},{"type":"v1","addr":"172.21.15.167:6839","nonce":3604645207}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24629,"qdb_cluster":[24599,24610,24629]},"id":1}]} 2024-08-30T09:13:26.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:25.411+0000 7f760b7fe640 1 -- 172.21.15.42:0/3386868397 shutdown_connections 2024-08-30T09:13:26.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:25.411+0000 7f760b7fe640 1 --2- 172.21.15.42:0/3386868397 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f760c0b4c30 0x7f760c0b7140 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:26.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:25.411+0000 7f760b7fe640 1 --2- 172.21.15.42:0/3386868397 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f762411c9d0 0x7f76241ae8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:26.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:25.411+0000 7f760b7fe640 1 --2- 172.21.15.42:0/3386868397 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7624111ce0 0x7f76241a9640 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:26.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:25.411+0000 7f760b7fe640 1 --2- 172.21.15.42:0/3386868397 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f762410a780 0x7f76241a90a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:26.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:25.411+0000 7f760b7fe640 1 -- 172.21.15.42:0/3386868397 >> 172.21.15.42:0/3386868397 conn(0x7f7624070c60 msgr2=0x7f762411e230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:13:26.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:25.411+0000 7f760b7fe640 1 -- 172.21.15.42:0/3386868397 shutdown_connections 2024-08-30T09:13:26.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:25.411+0000 7f760b7fe640 1 -- 172.21.15.42:0/3386868397 wait complete. 2024-08-30T09:13:26.256 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:13:25 smithi042 ceph-mon[18652]: from='client.? 172.21.15.42:0/3386868397' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-30T09:13:26.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:13:25 smithi167 ceph-mon[22063]: from='client.? 172.21.15.42:0/3386868397' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-30T09:13:26.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:13:25 smithi190 ceph-mon[22001]: from='client.? 172.21.15.42:0/3386868397' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-30T09:13:27.306 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:13:26 smithi042 ceph-mon[18652]: pgmap v753: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 4.5 MiB/s wr, 445 op/s 2024-08-30T09:13:27.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:13:26 smithi167 ceph-mon[22063]: pgmap v753: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 4.5 MiB/s wr, 445 op/s 2024-08-30T09:13:27.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:13:26 smithi190 ceph-mon[22001]: pgmap v753: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 4.5 MiB/s wr, 445 op/s 2024-08-30T09:13:27.752 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-08-30T09:13:28.297 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T09:13:28.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:28.915+0000 7f9e26dbd640 1 -- 172.21.15.42:0/1500223487 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9e20102800 msgr2=0x7f9e20102c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:13:28.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:28.915+0000 7f9e26dbd640 1 --2- 172.21.15.42:0/1500223487 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9e20102800 0x7f9e20102c10 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f9e14009900 tx=0x7f9e1402a410 comp rx=0 tx=0).stop 2024-08-30T09:13:28.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:28.915+0000 7f9e26dbd640 1 -- 172.21.15.42:0/1500223487 shutdown_connections 2024-08-30T09:13:28.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:28.915+0000 7f9e26dbd640 1 --2- 172.21.15.42:0/1500223487 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9e20108140 0x7f9e20108550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:28.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:28.915+0000 7f9e26dbd640 1 --2- 172.21.15.42:0/1500223487 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9e20108a90 0x7f9e2010f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:28.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:28.915+0000 7f9e26dbd640 1 --2- 172.21.15.42:0/1500223487 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9e20102800 0x7f9e20102c10 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:28.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:28.915+0000 7f9e26dbd640 1 -- 172.21.15.42:0/1500223487 >> 172.21.15.42:0/1500223487 conn(0x7f9e200794d0 msgr2=0x7f9e200798e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:13:28.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:28.915+0000 7f9e26dbd640 1 -- 172.21.15.42:0/1500223487 shutdown_connections 2024-08-30T09:13:28.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:28.915+0000 7f9e26dbd640 1 -- 172.21.15.42:0/1500223487 wait complete. 2024-08-30T09:13:28.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:28.917+0000 7f9e26dbd640 1 Processor -- start 2024-08-30T09:13:28.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:28.917+0000 7f9e26dbd640 1 -- start start 2024-08-30T09:13:28.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:28.917+0000 7f9e26dbd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9e20102800 0x7f9e20199120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:28.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:28.918+0000 7f9e26dbd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9e20108140 0x7f9e201996c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:28.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:28.918+0000 7f9e26dbd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9e20108a90 0x7f9e2019e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:28.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:28.918+0000 7f9e24b59640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9e20102800 0x7f9e20199120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:13:28.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:28.918+0000 7f9e24b59640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9e20102800 0x7f9e20199120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:57530/0 (socket says 172.21.15.42:57530) 2024-08-30T09:13:28.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:28.918+0000 7f9e26dbd640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f9e20112710 con 0x7f9e20108140 2024-08-30T09:13:28.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:28.918+0000 7f9e26dbd640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f9e20112590 con 0x7f9e20108a90 2024-08-30T09:13:28.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:28.918+0000 7f9e26dbd640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f9e20112890 con 0x7f9e20102800 2024-08-30T09:13:28.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:28.918+0000 7f9e24b59640 1 -- 172.21.15.42:0/1461828064 learned_addr learned my addr 172.21.15.42:0/1461828064 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:13:28.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:28.918+0000 7f9e2535a640 1 --2- 172.21.15.42:0/1461828064 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9e20108a90 0x7f9e2019e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:13:28.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:28.921+0000 7f9e2535a640 1 -- 172.21.15.42:0/1461828064 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9e20102800 msgr2=0x7f9e20199120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:13:28.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:28.921+0000 7f9e2535a640 1 --2- 172.21.15.42:0/1461828064 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9e20102800 0x7f9e20199120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:28.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:28.921+0000 7f9e2535a640 1 -- 172.21.15.42:0/1461828064 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9e20108140 msgr2=0x7f9e201996c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:13:28.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:28.921+0000 7f9e2535a640 1 --2- 172.21.15.42:0/1461828064 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9e20108140 0x7f9e201996c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:28.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:28.921+0000 7f9e2535a640 1 -- 172.21.15.42:0/1461828064 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e2019f090 con 0x7f9e20108a90 2024-08-30T09:13:28.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:28.927+0000 7f9e2535a640 1 --2- 172.21.15.42:0/1461828064 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9e20108a90 0x7f9e2019e960 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f9e18009fd0 tx=0x7f9e1800ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:13:28.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:28.934+0000 7f9e1dffb640 1 -- 172.21.15.42:0/1461828064 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e1800ce20 con 0x7f9e20108a90 2024-08-30T09:13:28.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:28.934+0000 7f9e1dffb640 1 -- 172.21.15.42:0/1461828064 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9e18005020 con 0x7f9e20108a90 2024-08-30T09:13:28.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:28.934+0000 7f9e1dffb640 1 -- 172.21.15.42:0/1461828064 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e18010690 con 0x7f9e20108a90 2024-08-30T09:13:28.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:28.935+0000 7f9e26dbd640 1 -- 172.21.15.42:0/1461828064 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e2019f380 con 0x7f9e20108a90 2024-08-30T09:13:28.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:28.936+0000 7f9e26dbd640 1 -- 172.21.15.42:0/1461828064 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e2006ac00 con 0x7f9e20108a90 2024-08-30T09:13:28.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:28.942+0000 7f9e2d332740 1 Processor -- start 2024-08-30T09:13:28.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:28.947+0000 7f9e1dffb640 1 -- 172.21.15.42:0/1461828064 <== mon.2 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7f9e18004a40 con 0x7f9e20108a90 2024-08-30T09:13:28.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:28.948+0000 7f9e1dffb640 1 --2- 172.21.15.42:0/1461828064 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f9e100b4c80 0x7f9e100b7190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:28.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:28.948+0000 7f9e1dffb640 1 -- 172.21.15.42:0/1461828064 <== mon.2 v2:172.21.15.167:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f9e180e5a60 con 0x7f9e20108a90 2024-08-30T09:13:28.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:28.950+0000 7f9e2d332740 1 -- start start 2024-08-30T09:13:28.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:28.950+0000 7f9e2d332740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556f15367070 0x556f15714b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:28.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:28.950+0000 7f9e2d332740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556f1533e1b0 0x556f15715060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:28.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:28.950+0000 7f9e2d332740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556f15346390 0x556f157155a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:28.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:28.950+0000 7f9e2d332740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x556f15683680 con 0x556f1533e1b0 2024-08-30T09:13:28.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:28.950+0000 7f9e2d332740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x556f15672c10 con 0x556f15346390 2024-08-30T09:13:28.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:28.950+0000 7f9e2d332740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x556f15676bc0 con 0x556f15367070 2024-08-30T09:13:28.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:28.952+0000 7f9e24b59640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556f15367070 0x556f15714b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:13:28.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:28.952+0000 7f9e24b59640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556f15367070 0x556f15714b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:57536/0 (socket says 172.21.15.42:57536) 2024-08-30T09:13:28.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:28.952+0000 7f9e24b59640 1 -- 172.21.15.42:0/452528052 learned_addr learned my addr 172.21.15.42:0/452528052 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:13:28.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:28.952+0000 7f9e1ffff640 1 --2- 172.21.15.42:0/452528052 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556f1533e1b0 0x556f15715060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:13:28.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:28.954+0000 7f9e24b59640 1 -- 172.21.15.42:0/452528052 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556f15346390 msgr2=0x556f157155a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-30T09:13:28.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:28.954+0000 7f9e24b59640 1 --2- 172.21.15.42:0/452528052 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556f15346390 0x556f157155a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:28.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:28.954+0000 7f9e24b59640 1 -- 172.21.15.42:0/452528052 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556f1533e1b0 msgr2=0x556f15715060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:13:28.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:28.954+0000 7f9e24b59640 1 --2- 172.21.15.42:0/452528052 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556f1533e1b0 0x556f15715060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:28.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:28.954+0000 7f9e24b59640 1 -- 172.21.15.42:0/452528052 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f153d2650 con 0x556f15367070 2024-08-30T09:13:28.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:28.952+0000 7f9e1ffff640 1 --2- 172.21.15.42:0/1461828064 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f9e100b4c80 0x7f9e100b7190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:13:28.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:28.955+0000 7f9e24b59640 1 --2- 172.21.15.42:0/452528052 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556f15367070 0x556f15714b20 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f9e14009a20 tx=0x7f9e14032bb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=af01e5390edb2e2e server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:13:28.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:28.964+0000 7f9e0affd640 1 -- 172.21.15.42:0/452528052 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e1403f070 con 0x556f15367070 2024-08-30T09:13:28.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:28.964+0000 7f9e0affd640 1 -- 172.21.15.42:0/452528052 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9e14004030 con 0x556f15367070 2024-08-30T09:13:28.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:28.964+0000 7f9e0affd640 1 -- 172.21.15.42:0/452528052 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e14035070 con 0x556f15367070 2024-08-30T09:13:28.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:28.967+0000 7f9e2d332740 1 -- 172.21.15.42:0/452528052 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556f15367070 msgr2=0x556f15714b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:13:28.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:28.967+0000 7f9e2d332740 1 --2- 172.21.15.42:0/452528052 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556f15367070 0x556f15714b20 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f9e14009a20 tx=0x7f9e14032bb0 comp rx=0 tx=0).stop 2024-08-30T09:13:28.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:28.971+0000 7f9e1ffff640 1 --2- 172.21.15.42:0/1461828064 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f9e100b4c80 0x7f9e100b7190 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x556f157137a0 tx=0x7f9e0c005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:13:28.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:28.971+0000 7f9e2d332740 1 -- 172.21.15.42:0/452528052 shutdown_connections 2024-08-30T09:13:28.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:28.971+0000 7f9e2d332740 1 --2- 172.21.15.42:0/452528052 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556f15346390 0x556f157155a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:28.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:28.971+0000 7f9e2d332740 1 --2- 172.21.15.42:0/452528052 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556f1533e1b0 0x556f15715060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:28.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:28.971+0000 7f9e2d332740 1 --2- 172.21.15.42:0/452528052 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556f15367070 0x556f15714b20 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:28.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:28.971+0000 7f9e2d332740 1 -- 172.21.15.42:0/452528052 >> 172.21.15.42:0/452528052 conn(0x556f1535b460 msgr2=0x556f15712a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:13:28.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:28.975+0000 7f9e2d332740 1 -- 172.21.15.42:0/452528052 shutdown_connections 2024-08-30T09:13:28.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:28.976+0000 7f9e2d332740 1 -- 172.21.15.42:0/452528052 wait complete. 2024-08-30T09:13:28.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:28.977+0000 7f9e2d332740 1 -- start start 2024-08-30T09:13:28.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:28.977+0000 7f9e2d332740 1 Processor -- start 2024-08-30T09:13:28.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:28.977+0000 7f9e2d332740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556f1533e1b0 0x556f157202a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:28.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:28.977+0000 7f9e2d332740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556f15346390 0x556f157207e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:28.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:28.977+0000 7f9e2d332740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556f15367070 0x556f1572c620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:28.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:28.977+0000 7f9e2d332740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x556f154d5700 con 0x556f15346390 2024-08-30T09:13:28.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:28.977+0000 7f9e2d332740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x556f155f9750 con 0x556f1533e1b0 2024-08-30T09:13:28.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:28.977+0000 7f9e2d332740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x556f157232a0 con 0x556f15367070 2024-08-30T09:13:28.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:28.977+0000 7f9e2d332740 10 client.0 _ll_register_callbacks cb 0x556f15628e10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:13:28.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:28.980+0000 7f9e24b59640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556f1533e1b0 0x556f157202a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:13:28.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:28.980+0000 7f9e24b59640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556f1533e1b0 0x556f157202a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:50004/0 (socket says 172.21.15.42:50004) 2024-08-30T09:13:28.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:28.980+0000 7f9e24b59640 1 -- 172.21.15.42:0/48404511 learned_addr learned my addr 172.21.15.42:0/48404511 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:13:28.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:28.981+0000 7f9e1ffff640 1 --2- 172.21.15.42:0/48404511 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556f15346390 0x556f157207e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:13:28.987 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:28.982+0000 7f9e24b59640 1 -- 172.21.15.42:0/48404511 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556f15367070 msgr2=0x556f1572c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:13:28.988 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:28.982+0000 7f9e24b59640 1 --2- 172.21.15.42:0/48404511 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556f15367070 0x556f1572c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:28.988 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:28.982+0000 7f9e24b59640 1 -- 172.21.15.42:0/48404511 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556f15346390 msgr2=0x556f157207e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:13:28.988 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:28.982+0000 7f9e24b59640 1 --2- 172.21.15.42:0/48404511 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556f15346390 0x556f157207e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:28.988 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:28.982+0000 7f9e24b59640 1 -- 172.21.15.42:0/48404511 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556f15632900 con 0x556f1533e1b0 2024-08-30T09:13:28.988 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:28.984+0000 7f9e24b59640 1 --2- 172.21.15.42:0/48404511 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556f1533e1b0 0x556f157202a0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f9e14031440 tx=0x7f9e14005150 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f9500492037bda12 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:13:28.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:28.985+0000 7f9dfffff640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-30T09:13:28.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:28.987+0000 7f9dfffff640 1 -- 172.21.15.42:0/48404511 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e1403f070 con 0x556f1533e1b0 2024-08-30T09:13:28.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:28.987+0000 7f9dfffff640 1 -- 172.21.15.42:0/48404511 <== mon.2 v2:172.21.15.167:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f9e140316b0 con 0x556f1533e1b0 2024-08-30T09:13:28.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:28.990+0000 7f9e2d332740 1 -- 172.21.15.42:0/48404511 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9e14004030 con 0x556f1533e1b0 2024-08-30T09:13:28.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:28.993+0000 7f9dfffff640 1 -- 172.21.15.42:0/48404511 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f9e14035040 con 0x556f1533e1b0 2024-08-30T09:13:28.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:28.993+0000 7f9e2d332740 10 client.27350 fetch_fsmap learned FSMap version 45 2024-08-30T09:13:28.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:28.993+0000 7f9e2d332740 1 -- 172.21.15.42:0/48404511 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x556f1572f0a0 con 0x556f1533e1b0 2024-08-30T09:13:28.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:28.994+0000 7f9dfffff640 1 -- 172.21.15.42:0/48404511 <== mon.2 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9e14004030 con 0x556f1533e1b0 2024-08-30T09:13:28.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:28.995+0000 7f9dfffff640 1 -- 172.21.15.42:0/48404511 <== mon.2 v2:172.21.15.167:3300/0 5 ==== fsmap(e 45) ==== 8362+0+0 (secure 0 0 0) 0x7f9e140035f0 con 0x556f1533e1b0 2024-08-30T09:13:29.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:28.997+0000 7f9e2d332740 10 client.27350 fetch_fsmap finished waiting for FSMap version 45 2024-08-30T09:13:29.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:28.997+0000 7f9e2d332740 10 client.27350 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-08-30T09:13:29.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:28.997+0000 7f9e2d332740 20 client.27350 populate_metadata read hostname 'smithi042' 2024-08-30T09:13:29.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:28.997+0000 7f9e2d332740 1 --2- 172.21.15.42:0/48404511 >> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] conn(0x556f153259f0 0x556f15732400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:29.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:28.997+0000 7f9e2d332740 4 client.27350 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-30T09:13:29.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:28.997+0000 7f9e2d332740 1 -- 172.21.15.42:0/48404511 --> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556f156855d0 con 0x556f153259f0 2024-08-30T09:13:29.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:28.999+0000 7f9e1ffff640 1 --2- 172.21.15.42:0/48404511 >> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] conn(0x556f153259f0 0x556f15732400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:13:29.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:29.003+0000 7f9e1ffff640 1 --2- 172.21.15.42:0/48404511 >> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] conn(0x556f153259f0 0x556f15732400 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7d1075f893224d82 server_cookie=6553c94caa3c05cb in_seq=0 out_seq=0 2024-08-30T09:13:29.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:29.004+0000 7f9dfffff640 10 client.27350 ms_handle_connect on v2:172.21.15.167:6836/3011837856 2024-08-30T09:13:29.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:29.006+0000 7f9dfffff640 1 -- 172.21.15.42:0/48404511 <== mds.2 v2:172.21.15.167:6836/3011837856 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x556f156855d0 con 0x556f153259f0 2024-08-30T09:13:29.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:29.006+0000 7f9dfffff640 10 client.27350 handle_command_reply: tid=0 2024-08-30T09:13:29.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:29.006+0000 7f9e2d332740 1 client.27350 shutdown 2024-08-30T09:13:29.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:29.006+0000 7f9e2d332740 1 -- 172.21.15.42:0/48404511 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556f1533e1b0 msgr2=0x556f157202a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:13:29.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:29.006+0000 7f9e2d332740 1 --2- 172.21.15.42:0/48404511 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556f1533e1b0 0x556f157202a0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f9e14031440 tx=0x7f9e14005150 comp rx=0 tx=0).stop 2024-08-30T09:13:29.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:29.007+0000 7f9e2d332740 1 -- 172.21.15.42:0/48404511 shutdown_connections 2024-08-30T09:13:29.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:29.007+0000 7f9e2d332740 1 --2- 172.21.15.42:0/48404511 >> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] conn(0x556f153259f0 0x556f15732400 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:29.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:29.007+0000 7f9e2d332740 1 --2- 172.21.15.42:0/48404511 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556f15367070 0x556f1572c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:29.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:29.007+0000 7f9e2d332740 1 --2- 172.21.15.42:0/48404511 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556f15346390 0x556f157207e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:29.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:29.007+0000 7f9e2d332740 1 --2- 172.21.15.42:0/48404511 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556f1533e1b0 0x556f157202a0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:29.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:29.007+0000 7f9e2d332740 1 -- 172.21.15.42:0/48404511 >> 172.21.15.42:0/48404511 conn(0x556f1535b460 msgr2=0x556f15717cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:13:29.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:29.007+0000 7f9dfffff640 0 client.27350 ms_handle_reset on v2:172.21.15.167:6836/3011837856 2024-08-30T09:13:29.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:29.011+0000 7f9e2d332740 1 -- 172.21.15.42:0/48404511 shutdown_connections 2024-08-30T09:13:29.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:29.012+0000 7f9e2d332740 1 -- 172.21.15.42:0/48404511 wait complete. 2024-08-30T09:13:29.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:29.012+0000 7f9e2d332740 20 client.27350 trim_cache size 0 max 16384 2024-08-30T09:13:29.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:29.041+0000 7f9e2d332740 1 Processor -- start 2024-08-30T09:13:29.044 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:13:28 smithi042 ceph-mon[18652]: pgmap v754: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 4.3 MiB/s wr, 417 op/s 2024-08-30T09:13:29.044 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:13:28 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-30T09:13:29.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:29.042+0000 7f9e2d332740 1 -- start start 2024-08-30T09:13:29.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:29.042+0000 7f9e2d332740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556f1533e1b0 0x556f1572c590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:29.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:29.042+0000 7f9e2d332740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556f15346390 0x556f15716210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:29.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:29.042+0000 7f9e24b59640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556f1533e1b0 0x556f1572c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:13:29.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:29.043+0000 7f9e24b59640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556f1533e1b0 0x556f1572c590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:50008/0 (socket says 172.21.15.42:50008) 2024-08-30T09:13:29.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:29.043+0000 7f9e24b59640 1 -- 172.21.15.42:0/1328513679 learned_addr learned my addr 172.21.15.42:0/1328513679 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:13:29.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:29.043+0000 7f9e2d332740 1 --2- 172.21.15.42:0/1328513679 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556f15367070 0x556f1571d490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:29.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:29.043+0000 7f9e2d332740 1 -- 172.21.15.42:0/1328513679 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f9e040125a0 con 0x556f15367070 2024-08-30T09:13:29.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:29.043+0000 7f9e2d332740 1 -- 172.21.15.42:0/1328513679 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f9e0400fe10 con 0x556f1533e1b0 2024-08-30T09:13:29.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:29.043+0000 7f9e2d332740 1 -- 172.21.15.42:0/1328513679 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f9e0400dc60 con 0x556f15346390 2024-08-30T09:13:29.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:29.043+0000 7f9e1ffff640 1 --2- 172.21.15.42:0/1328513679 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556f15346390 0x556f15716210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:13:29.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:29.044+0000 7f9e24b59640 1 -- 172.21.15.42:0/1328513679 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556f15346390 msgr2=0x556f15716210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:13:29.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:29.044+0000 7f9e24b59640 1 --2- 172.21.15.42:0/1328513679 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556f15346390 0x556f15716210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:29.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:29.044+0000 7f9e24b59640 1 -- 172.21.15.42:0/1328513679 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556f15367070 msgr2=0x556f1571d490 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-30T09:13:29.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:29.044+0000 7f9e24b59640 1 --2- 172.21.15.42:0/1328513679 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556f15367070 0x556f1571d490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:29.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:29.044+0000 7f9e24b59640 1 -- 172.21.15.42:0/1328513679 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f157137d0 con 0x556f1533e1b0 2024-08-30T09:13:29.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:29.044+0000 7f9e24b59640 1 --2- 172.21.15.42:0/1328513679 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556f1533e1b0 0x556f1572c590 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f9e14009a20 tx=0x7f9e14005700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ce907439213e27d9 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:13:29.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:29.044+0000 7f9e0a7fc640 1 -- 172.21.15.42:0/1328513679 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e1403f040 con 0x556f1533e1b0 2024-08-30T09:13:29.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:29.045+0000 7f9e0a7fc640 1 -- 172.21.15.42:0/1328513679 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9e14005d50 con 0x556f1533e1b0 2024-08-30T09:13:29.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:29.045+0000 7f9e0a7fc640 1 -- 172.21.15.42:0/1328513679 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e14031910 con 0x556f1533e1b0 2024-08-30T09:13:29.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:29.046+0000 7f9e2d332740 1 -- 172.21.15.42:0/1328513679 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556f1533e1b0 msgr2=0x556f1572c590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:13:29.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:29.046+0000 7f9e2d332740 1 --2- 172.21.15.42:0/1328513679 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556f1533e1b0 0x556f1572c590 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f9e14009a20 tx=0x7f9e14005700 comp rx=0 tx=0).stop 2024-08-30T09:13:29.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:29.046+0000 7f9e2d332740 1 -- 172.21.15.42:0/1328513679 shutdown_connections 2024-08-30T09:13:29.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:29.046+0000 7f9e2d332740 1 --2- 172.21.15.42:0/1328513679 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556f15346390 0x556f15716210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:29.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:29.046+0000 7f9e2d332740 1 --2- 172.21.15.42:0/1328513679 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556f15367070 0x556f1571d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:29.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:29.046+0000 7f9e2d332740 1 --2- 172.21.15.42:0/1328513679 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556f1533e1b0 0x556f1572c590 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:29.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:29.046+0000 7f9e2d332740 1 -- 172.21.15.42:0/1328513679 >> 172.21.15.42:0/1328513679 conn(0x556f1535b460 msgr2=0x556f15715b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:13:29.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:29.046+0000 7f9e2d332740 1 -- 172.21.15.42:0/1328513679 shutdown_connections 2024-08-30T09:13:29.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:29.046+0000 7f9e2d332740 1 -- 172.21.15.42:0/1328513679 wait complete. 2024-08-30T09:13:29.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:29.047+0000 7f9e2d332740 1 -- start start 2024-08-30T09:13:29.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:29.047+0000 7f9e2d332740 1 Processor -- start 2024-08-30T09:13:29.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:29.048+0000 7f9e2d332740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556f1533e1b0 0x556f15733750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:29.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:29.048+0000 7f9e2d332740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556f15346390 0x556f15733c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:29.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:29.048+0000 7f9e2d332740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556f15367070 0x556f1579bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:29.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:29.048+0000 7f9e2d332740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f9e0400e2d0 con 0x556f15346390 2024-08-30T09:13:29.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:29.048+0000 7f9e2d332740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f9e0400f740 con 0x556f1533e1b0 2024-08-30T09:13:29.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:29.048+0000 7f9e2d332740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f9e0400e9a0 con 0x556f15367070 2024-08-30T09:13:29.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:29.048+0000 7f9e2d332740 10 client.0 _ll_register_callbacks cb 0x7f9e0c005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:13:29.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:29.049+0000 7f9e2535a640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556f15367070 0x556f1579bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:13:29.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:29.049+0000 7f9e2535a640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556f15367070 0x556f1579bb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:57558/0 (socket says 172.21.15.42:57558) 2024-08-30T09:13:29.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:29.049+0000 7f9e2535a640 1 -- 172.21.15.42:0/1944011146 learned_addr learned my addr 172.21.15.42:0/1944011146 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:13:29.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:29.049+0000 7f9e24b59640 1 --2- 172.21.15.42:0/1944011146 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556f1533e1b0 0x556f15733750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:13:29.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:29.049+0000 7f9e2535a640 1 -- 172.21.15.42:0/1944011146 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556f1533e1b0 msgr2=0x556f15733750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:13:29.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:29.049+0000 7f9e2535a640 1 --2- 172.21.15.42:0/1944011146 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556f1533e1b0 0x556f15733750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:29.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:29.049+0000 7f9e2535a640 1 -- 172.21.15.42:0/1944011146 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556f15346390 msgr2=0x556f15733c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:13:29.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:29.049+0000 7f9e2535a640 1 --2- 172.21.15.42:0/1944011146 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556f15346390 0x556f15733c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:29.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:29.049+0000 7f9e2535a640 1 -- 172.21.15.42:0/1944011146 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556f15734560 con 0x556f15367070 2024-08-30T09:13:29.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:29.050+0000 7f9e2535a640 1 --2- 172.21.15.42:0/1944011146 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556f15367070 0x556f1579bb80 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f9e180e31d0 tx=0x7f9e180abcd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=279ea73b237033fe server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:13:29.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:29.050+0000 7f9dfffff640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-08-30T09:13:29.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:29.050+0000 7f9dfffff640 1 -- 172.21.15.42:0/1944011146 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e180b5420 con 0x556f15367070 2024-08-30T09:13:29.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:29.050+0000 7f9dfffff640 1 -- 172.21.15.42:0/1944011146 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f9e18004d30 con 0x556f15367070 2024-08-30T09:13:29.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:29.050+0000 7f9e2d332740 1 -- 172.21.15.42:0/1944011146 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9e14005d50 con 0x556f15367070 2024-08-30T09:13:29.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:29.051+0000 7f9dfffff640 1 -- 172.21.15.42:0/1944011146 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f9e180ac360 con 0x556f15367070 2024-08-30T09:13:29.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:29.051+0000 7f9e2d332740 10 client.27292 fetch_fsmap learned FSMap version 45 2024-08-30T09:13:29.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:29.051+0000 7f9e2d332740 1 -- 172.21.15.42:0/1944011146 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x556f1579d1b0 con 0x556f15367070 2024-08-30T09:13:29.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:29.052+0000 7f9dfffff640 1 -- 172.21.15.42:0/1944011146 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9e14005d50 con 0x556f15367070 2024-08-30T09:13:29.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:29.052+0000 7f9dfffff640 1 -- 172.21.15.42:0/1944011146 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 45) ==== 8362+0+0 (secure 0 0 0) 0x7f9e180971a0 con 0x556f15367070 2024-08-30T09:13:29.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:29.052+0000 7f9e2d332740 10 client.27292 fetch_fsmap finished waiting for FSMap version 45 2024-08-30T09:13:29.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:29.052+0000 7f9e2d332740 10 client.27292 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-08-30T09:13:29.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:29.053+0000 7f9e2d332740 20 client.27292 populate_metadata read hostname 'smithi042' 2024-08-30T09:13:29.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:29.053+0000 7f9e2d332740 1 --2- 172.21.15.42:0/1944011146 >> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] conn(0x556f152fada0 0x556f1579f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:29.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:29.053+0000 7f9e2d332740 4 client.27292 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-30T09:13:29.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:29.053+0000 7f9e2d332740 1 -- 172.21.15.42:0/1944011146 --> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556f15681070 con 0x556f152fada0 2024-08-30T09:13:29.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:29.053+0000 7f9e24b59640 1 --2- 172.21.15.42:0/1944011146 >> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] conn(0x556f152fada0 0x556f1579f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:13:29.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:29.054+0000 7f9dfffff640 10 client.27292 ms_handle_connect on v2:172.21.15.167:6836/3011837856 2024-08-30T09:13:29.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:29.054+0000 7f9e24b59640 1 --2- 172.21.15.42:0/1944011146 >> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] conn(0x556f152fada0 0x556f1579f9c0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5af2f1311ba770b3 server_cookie=834ff58cdfc63259 in_seq=0 out_seq=0 2024-08-30T09:13:29.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:29.054+0000 7f9dfffff640 1 -- 172.21.15.42:0/1944011146 <== mds.2 v2:172.21.15.167:6836/3011837856 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556f15681070 con 0x556f152fada0 2024-08-30T09:13:29.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:29.054+0000 7f9dfffff640 10 client.27292 handle_command_reply: tid=0 2024-08-30T09:13:29.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:29.054+0000 7f9e2d332740 1 client.27292 shutdown 2024-08-30T09:13:29.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:29.057+0000 7f9e2d332740 1 -- 172.21.15.42:0/1944011146 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556f15367070 msgr2=0x556f1579bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:13:29.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:29.057+0000 7f9e2d332740 1 --2- 172.21.15.42:0/1944011146 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556f15367070 0x556f1579bb80 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f9e180e31d0 tx=0x7f9e180abcd0 comp rx=0 tx=0).stop 2024-08-30T09:13:29.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:29.057+0000 7f9e2d332740 1 -- 172.21.15.42:0/1944011146 shutdown_connections 2024-08-30T09:13:29.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:29.057+0000 7f9e2d332740 1 --2- 172.21.15.42:0/1944011146 >> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] conn(0x556f152fada0 0x556f1579f9c0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:29.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:29.057+0000 7f9e2d332740 1 --2- 172.21.15.42:0/1944011146 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556f15367070 0x556f1579bb80 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:29.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:29.057+0000 7f9e2d332740 1 --2- 172.21.15.42:0/1944011146 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556f15346390 0x556f15733c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:29.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:29.057+0000 7f9e2d332740 1 --2- 172.21.15.42:0/1944011146 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556f1533e1b0 0x556f15733750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:29.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:29.057+0000 7f9e2d332740 1 -- 172.21.15.42:0/1944011146 >> 172.21.15.42:0/1944011146 conn(0x556f1535b460 msgr2=0x556f157164c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:13:29.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:29.058+0000 7f9dfffff640 0 client.27292 ms_handle_reset on v2:172.21.15.167:6836/3011837856 2024-08-30T09:13:29.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:29.058+0000 7f9e2d332740 1 -- 172.21.15.42:0/1944011146 shutdown_connections 2024-08-30T09:13:29.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:29.058+0000 7f9e2d332740 1 -- 172.21.15.42:0/1944011146 wait complete. 2024-08-30T09:13:29.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:29.058+0000 7f9e2d332740 20 client.27292 trim_cache size 0 max 16384 2024-08-30T09:13:29.062 INFO:teuthology.orchestra.run.smithi042.stdout:[] 2024-08-30T09:13:29.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:29.060+0000 7f9e26dbd640 1 -- 172.21.15.42:0/1461828064 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f9e100b4c80 msgr2=0x7f9e100b7190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:13:29.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:29.060+0000 7f9e26dbd640 1 --2- 172.21.15.42:0/1461828064 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f9e100b4c80 0x7f9e100b7190 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x556f157137a0 tx=0x7f9e0c005850 comp rx=0 tx=0).stop 2024-08-30T09:13:29.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:29.060+0000 7f9e26dbd640 1 -- 172.21.15.42:0/1461828064 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9e20108a90 msgr2=0x7f9e2019e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:13:29.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:29.060+0000 7f9e26dbd640 1 --2- 172.21.15.42:0/1461828064 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9e20108a90 0x7f9e2019e960 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f9e18009fd0 tx=0x7f9e1800ef10 comp rx=0 tx=0).stop 2024-08-30T09:13:29.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:29.060+0000 7f9e26dbd640 1 -- 172.21.15.42:0/1461828064 shutdown_connections 2024-08-30T09:13:29.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:29.060+0000 7f9e26dbd640 1 --2- 172.21.15.42:0/1461828064 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f9e100b4c80 0x7f9e100b7190 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:29.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:29.060+0000 7f9e26dbd640 1 --2- 172.21.15.42:0/1461828064 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9e20108a90 0x7f9e2019e960 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:29.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:29.060+0000 7f9e26dbd640 1 --2- 172.21.15.42:0/1461828064 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9e20108140 0x7f9e201996c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:29.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:29.060+0000 7f9e26dbd640 1 --2- 172.21.15.42:0/1461828064 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9e20102800 0x7f9e20199120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:29.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:29.060+0000 7f9e26dbd640 1 -- 172.21.15.42:0/1461828064 >> 172.21.15.42:0/1461828064 conn(0x7f9e200794d0 msgr2=0x7f9e20101170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:13:29.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:29.062+0000 7f9e26dbd640 1 -- 172.21.15.42:0/1461828064 shutdown_connections 2024-08-30T09:13:29.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:29.062+0000 7f9e26dbd640 1 -- 172.21.15.42:0/1461828064 wait complete. 2024-08-30T09:13:29.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:13:28 smithi167 ceph-mon[22063]: pgmap v754: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 4.3 MiB/s wr, 417 op/s 2024-08-30T09:13:29.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:13:28 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-30T09:13:29.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:13:28 smithi190 ceph-mon[22001]: pgmap v754: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 4.3 MiB/s wr, 417 op/s 2024-08-30T09:13:29.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:13:28 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-30T09:13:29.804 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-08-30T09:13:30.136 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:13:29 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T09:13:30.136 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:13:29 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-30T09:13:30.136 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:13:29 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T09:13:30.238 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T09:13:30.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:13:29 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T09:13:30.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:13:29 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-30T09:13:30.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:13:29 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T09:13:30.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:13:29 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T09:13:30.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:13:29 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-30T09:13:30.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:13:29 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T09:13:30.953 INFO:teuthology.orchestra.run.smithi042.stderr:progress: 765.0 s, 428.2 tps, lat 237.289 ms stddev 281.002 2024-08-30T09:13:30.990 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:13:30 smithi042 ceph-mon[18652]: pgmap v755: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 5.4 MiB/s wr, 528 op/s 2024-08-30T09:13:31.121 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.117+0000 7f9074f9d640 1 -- 172.21.15.42:0/1338947747 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f907010aab0 msgr2=0x7f90701114a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:13:31.121 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.118+0000 7f9074f9d640 1 --2- 172.21.15.42:0/1338947747 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f907010aab0 0x7f90701114a0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f9060009900 tx=0x7f906002a410 comp rx=0 tx=0).stop 2024-08-30T09:13:31.121 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.118+0000 7f9074f9d640 1 -- 172.21.15.42:0/1338947747 shutdown_connections 2024-08-30T09:13:31.122 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.118+0000 7f9074f9d640 1 --2- 172.21.15.42:0/1338947747 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f907010aab0 0x7f90701114a0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:31.122 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.118+0000 7f9074f9d640 1 --2- 172.21.15.42:0/1338947747 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f90701119e0 0x7f9070113e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:31.122 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.118+0000 7f9074f9d640 1 --2- 172.21.15.42:0/1338947747 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f907010a160 0x7f907010a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:31.122 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.118+0000 7f9074f9d640 1 -- 172.21.15.42:0/1338947747 >> 172.21.15.42:0/1338947747 conn(0x7f90700703e0 msgr2=0x7f9070072890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:13:31.122 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.118+0000 7f9074f9d640 1 -- 172.21.15.42:0/1338947747 shutdown_connections 2024-08-30T09:13:31.122 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.118+0000 7f9074f9d640 1 -- 172.21.15.42:0/1338947747 wait complete. 2024-08-30T09:13:31.122 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.119+0000 7f9074f9d640 1 Processor -- start 2024-08-30T09:13:31.123 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.120+0000 7f9074f9d640 1 -- start start 2024-08-30T09:13:31.124 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.120+0000 7f9074f9d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f907010a160 0x7f90701a22f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:31.124 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.120+0000 7f9074f9d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f907010aab0 0x7f90701a2960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:31.124 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.121+0000 7f9074f9d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f90701119e0 0x7f90701a3510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:31.124 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.121+0000 7f906f7fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f907010a160 0x7f90701a22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:13:31.125 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.121+0000 7f906f7fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f907010a160 0x7f90701a22f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:59744/0 (socket says 172.21.15.42:59744) 2024-08-30T09:13:31.125 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.121+0000 7f906effd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f907010aab0 0x7f90701a2960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:13:31.125 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.121+0000 7f906effd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f907010aab0 0x7f90701a2960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:57578/0 (socket says 172.21.15.42:57578) 2024-08-30T09:13:31.125 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.121+0000 7f9074f9d640 1 -- 172.21.15.42:0/3092739382 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f90701176c0 con 0x7f907010a160 2024-08-30T09:13:31.125 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.121+0000 7f906f7fe640 1 -- 172.21.15.42:0/3092739382 learned_addr learned my addr 172.21.15.42:0/3092739382 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:13:31.125 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.121+0000 7f9074f9d640 1 -- 172.21.15.42:0/3092739382 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f9070117540 con 0x7f90701119e0 2024-08-30T09:13:31.125 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.121+0000 7f9074f9d640 1 -- 172.21.15.42:0/3092739382 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f9070117840 con 0x7f907010aab0 2024-08-30T09:13:31.125 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.121+0000 7f906ffff640 1 --2- 172.21.15.42:0/3092739382 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f90701119e0 0x7f90701a3510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:13:31.125 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.121+0000 7f906effd640 1 -- 172.21.15.42:0/3092739382 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f90701119e0 msgr2=0x7f90701a3510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:13:31.125 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.121+0000 7f906effd640 1 --2- 172.21.15.42:0/3092739382 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f90701119e0 0x7f90701a3510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:31.125 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.121+0000 7f906effd640 1 -- 172.21.15.42:0/3092739382 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f907010a160 msgr2=0x7f90701a22f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:13:31.125 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.121+0000 7f906effd640 1 --2- 172.21.15.42:0/3092739382 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f907010a160 0x7f90701a22f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:31.125 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.121+0000 7f906effd640 1 -- 172.21.15.42:0/3092739382 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90701a3d30 con 0x7f907010aab0 2024-08-30T09:13:31.126 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.122+0000 7f906effd640 1 --2- 172.21.15.42:0/3092739382 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f907010aab0 0x7f90701a2960 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f9060006010 tx=0x7f9060004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:13:31.126 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.122+0000 7f906cff9640 1 -- 172.21.15.42:0/3092739382 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f906003f070 con 0x7f907010aab0 2024-08-30T09:13:31.126 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.122+0000 7f906cff9640 1 -- 172.21.15.42:0/3092739382 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9060003ca0 con 0x7f907010aab0 2024-08-30T09:13:31.126 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.122+0000 7f9074f9d640 1 -- 172.21.15.42:0/3092739382 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90701a3fc0 con 0x7f907010aab0 2024-08-30T09:13:31.126 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.122+0000 7f906cff9640 1 -- 172.21.15.42:0/3092739382 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9060035070 con 0x7f907010aab0 2024-08-30T09:13:31.127 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.122+0000 7f9074f9d640 1 -- 172.21.15.42:0/3092739382 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90701aa530 con 0x7f907010aab0 2024-08-30T09:13:31.127 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.125+0000 7f907d244740 1 Processor -- start 2024-08-30T09:13:31.129 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.125+0000 7f907d244740 1 -- start start 2024-08-30T09:13:31.130 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.125+0000 7f907d244740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562a3359f1b0 0x562a33949790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:31.130 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.125+0000 7f907d244740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562a335762f0 0x562a33949e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:31.130 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.125+0000 7f907d244740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562a3357e4d0 0x562a3394a950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:31.130 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.125+0000 7f907d244740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x562a338b8000 con 0x562a335762f0 2024-08-30T09:13:31.130 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.125+0000 7f907d244740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x562a338a7590 con 0x562a3359f1b0 2024-08-30T09:13:31.130 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.125+0000 7f907d244740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x562a338ab540 con 0x562a3357e4d0 2024-08-30T09:13:31.130 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.125+0000 7f906ffff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562a3359f1b0 0x562a33949790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:13:31.130 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.125+0000 7f906f7fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562a335762f0 0x562a33949e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:13:31.130 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.126+0000 7f906ffff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562a3359f1b0 0x562a33949790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:50044/0 (socket says 172.21.15.42:50044) 2024-08-30T09:13:31.130 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.126+0000 7f906ffff640 1 -- 172.21.15.42:0/2169807743 learned_addr learned my addr 172.21.15.42:0/2169807743 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:13:31.130 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.126+0000 7f906cff9640 1 -- 172.21.15.42:0/3092739382 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7f906003b4f0 con 0x7f907010aab0 2024-08-30T09:13:31.130 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.126+0000 7f906effd640 1 --2- 172.21.15.42:0/2169807743 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562a3357e4d0 0x562a3394a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:13:31.131 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.126+0000 7f906ffff640 1 -- 172.21.15.42:0/2169807743 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562a3357e4d0 msgr2=0x562a3394a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:13:31.131 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.126+0000 7f906ffff640 1 --2- 172.21.15.42:0/2169807743 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562a3357e4d0 0x562a3394a950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:31.131 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.126+0000 7f906ffff640 1 -- 172.21.15.42:0/2169807743 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562a335762f0 msgr2=0x562a33949e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:13:31.131 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.126+0000 7f906ffff640 1 --2- 172.21.15.42:0/2169807743 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562a335762f0 0x562a33949e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:31.131 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.126+0000 7f906ffff640 1 -- 172.21.15.42:0/2169807743 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a3360a790 con 0x562a3359f1b0 2024-08-30T09:13:31.131 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.126+0000 7f906f7fe640 1 --2- 172.21.15.42:0/2169807743 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562a335762f0 0x562a33949e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-30T09:13:31.131 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.126+0000 7f906effd640 1 --2- 172.21.15.42:0/2169807743 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562a3357e4d0 0x562a3394a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-30T09:13:31.131 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.126+0000 7f906ffff640 1 --2- 172.21.15.42:0/2169807743 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562a3359f1b0 0x562a33949790 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f9068009fd0 tx=0x7f90680118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5d70cdecbbfd33f4 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:13:31.131 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.126+0000 7f9055ffb640 1 -- 172.21.15.42:0/2169807743 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9068004d20 con 0x562a3359f1b0 2024-08-30T09:13:31.131 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.126+0000 7f906cff9640 1 --2- 172.21.15.42:0/3092739382 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f90580b4b30 0x7f90580b7040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:31.131 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.126+0000 7f9055ffb640 1 -- 172.21.15.42:0/2169807743 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9068004050 con 0x562a3359f1b0 2024-08-30T09:13:31.131 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.126+0000 7f9055ffb640 1 -- 172.21.15.42:0/2169807743 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9068005260 con 0x562a3359f1b0 2024-08-30T09:13:31.131 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.127+0000 7f906f7fe640 1 --2- 172.21.15.42:0/3092739382 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f90580b4b30 0x7f90580b7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:13:31.132 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.127+0000 7f906cff9640 1 -- 172.21.15.42:0/3092739382 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f9060105920 con 0x7f907010aab0 2024-08-30T09:13:31.135 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.127+0000 7f907d244740 1 -- 172.21.15.42:0/2169807743 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562a3359f1b0 msgr2=0x562a33949790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:13:31.135 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.127+0000 7f906f7fe640 1 --2- 172.21.15.42:0/3092739382 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f90580b4b30 0x7f90580b7040 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f90640058c0 tx=0x7f906400a380 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:13:31.135 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.127+0000 7f907d244740 1 --2- 172.21.15.42:0/2169807743 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562a3359f1b0 0x562a33949790 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f9068009fd0 tx=0x7f90680118d0 comp rx=0 tx=0).stop 2024-08-30T09:13:31.135 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.127+0000 7f907d244740 1 -- 172.21.15.42:0/2169807743 shutdown_connections 2024-08-30T09:13:31.135 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.127+0000 7f907d244740 1 --2- 172.21.15.42:0/2169807743 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562a3357e4d0 0x562a3394a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:31.135 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.127+0000 7f907d244740 1 --2- 172.21.15.42:0/2169807743 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562a335762f0 0x562a33949e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:31.136 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.127+0000 7f907d244740 1 --2- 172.21.15.42:0/2169807743 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562a3359f1b0 0x562a33949790 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:31.136 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.127+0000 7f907d244740 1 -- 172.21.15.42:0/2169807743 >> 172.21.15.42:0/2169807743 conn(0x562a33593300 msgr2=0x562a33940e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:13:31.136 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.128+0000 7f907d244740 1 -- 172.21.15.42:0/2169807743 shutdown_connections 2024-08-30T09:13:31.136 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.128+0000 7f907d244740 1 -- 172.21.15.42:0/2169807743 wait complete. 2024-08-30T09:13:31.136 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.128+0000 7f907d244740 1 -- start start 2024-08-30T09:13:31.136 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.128+0000 7f907d244740 1 Processor -- start 2024-08-30T09:13:31.136 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.128+0000 7f907d244740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562a335762f0 0x562a33953330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:31.136 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.128+0000 7f907d244740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562a3357e4d0 0x562a339539f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:31.136 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.128+0000 7f907d244740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562a3359f1b0 0x562a339544f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:31.136 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.128+0000 7f907d244740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x562a337e0ab0 con 0x562a3357e4d0 2024-08-30T09:13:31.136 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.128+0000 7f907d244740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x562a3382e680 con 0x562a3359f1b0 2024-08-30T09:13:31.136 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.128+0000 7f907d244740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x562a3394b470 con 0x562a335762f0 2024-08-30T09:13:31.136 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.128+0000 7f907d244740 10 client.0 _ll_register_callbacks cb 0x562a335b6a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:13:31.137 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.128+0000 7f906effd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562a3357e4d0 0x562a339539f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:13:31.137 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.129+0000 7f906effd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562a3357e4d0 0x562a339539f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:59760/0 (socket says 172.21.15.42:59760) 2024-08-30T09:13:31.137 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.129+0000 7f906effd640 1 -- 172.21.15.42:0/1062795541 learned_addr learned my addr 172.21.15.42:0/1062795541 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:13:31.137 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.129+0000 7f906f7fe640 1 --2- 172.21.15.42:0/1062795541 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562a335762f0 0x562a33953330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:13:31.137 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.129+0000 7f906ffff640 1 --2- 172.21.15.42:0/1062795541 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562a3359f1b0 0x562a339544f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:13:31.137 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.129+0000 7f906effd640 1 -- 172.21.15.42:0/1062795541 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562a335762f0 msgr2=0x562a33953330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:13:31.137 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.129+0000 7f906effd640 1 --2- 172.21.15.42:0/1062795541 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562a335762f0 0x562a33953330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:31.137 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.129+0000 7f906effd640 1 -- 172.21.15.42:0/1062795541 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562a3359f1b0 msgr2=0x562a339544f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:13:31.137 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.129+0000 7f906effd640 1 --2- 172.21.15.42:0/1062795541 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562a3359f1b0 0x562a339544f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:31.137 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.129+0000 7f906effd640 1 -- 172.21.15.42:0/1062795541 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562a33683620 con 0x562a3357e4d0 2024-08-30T09:13:31.137 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.129+0000 7f906effd640 1 --2- 172.21.15.42:0/1062795541 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562a3357e4d0 0x562a339539f0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f90600cbfa0 tx=0x7f9060000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=347080e23a79e210 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:13:31.137 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.129+0000 7f904effd640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-08-30T09:13:31.137 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.129+0000 7f904effd640 1 -- 172.21.15.42:0/1062795541 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9060048070 con 0x562a3357e4d0 2024-08-30T09:13:31.137 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.129+0000 7f904effd640 1 -- 172.21.15.42:0/1062795541 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f90600cc480 con 0x562a3357e4d0 2024-08-30T09:13:31.138 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.129+0000 7f907d244740 1 -- 172.21.15.42:0/1062795541 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9068004050 con 0x562a3357e4d0 2024-08-30T09:13:31.138 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.129+0000 7f904effd640 1 -- 172.21.15.42:0/1062795541 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f9060003e50 con 0x562a3357e4d0 2024-08-30T09:13:31.138 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.129+0000 7f907d244740 10 client.17523 fetch_fsmap learned FSMap version 45 2024-08-30T09:13:31.138 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.129+0000 7f907d244740 1 -- 172.21.15.42:0/1062795541 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x562a3395d3b0 con 0x562a3357e4d0 2024-08-30T09:13:31.138 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.129+0000 7f904effd640 1 -- 172.21.15.42:0/1062795541 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9068004050 con 0x562a3357e4d0 2024-08-30T09:13:31.138 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.130+0000 7f904effd640 1 -- 172.21.15.42:0/1062795541 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 45) ==== 8362+0+0 (secure 0 0 0) 0x7f90600cf070 con 0x562a3357e4d0 2024-08-30T09:13:31.138 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.130+0000 7f907d244740 10 client.17523 fetch_fsmap finished waiting for FSMap version 45 2024-08-30T09:13:31.138 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.130+0000 7f907d244740 10 client.17523 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-30T09:13:31.138 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.130+0000 7f907d244740 20 client.17523 populate_metadata read hostname 'smithi042' 2024-08-30T09:13:31.138 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.130+0000 7f907d244740 1 --2- 172.21.15.42:0/1062795541 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x562a3355dd80 0x562a33960b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:31.138 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.130+0000 7f907d244740 4 client.17523 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-30T09:13:31.138 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.130+0000 7f907d244740 1 -- 172.21.15.42:0/1062795541 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562a338b9f50 con 0x562a3355dd80 2024-08-30T09:13:31.138 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.130+0000 7f906f7fe640 1 --2- 172.21.15.42:0/1062795541 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x562a3355dd80 0x562a33960b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:13:31.138 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.130+0000 7f904effd640 10 client.17523 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:13:31.138 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.130+0000 7f906f7fe640 1 --2- 172.21.15.42:0/1062795541 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x562a3355dd80 0x562a33960b50 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3bc111fb1203a5cb server_cookie=78c25ff55f4c2194 in_seq=0 out_seq=0 2024-08-30T09:13:31.139 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.131+0000 7f904effd640 1 -- 172.21.15.42:0/1062795541 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x562a338b9f50 con 0x562a3355dd80 2024-08-30T09:13:31.139 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.131+0000 7f904effd640 10 client.17523 handle_command_reply: tid=0 2024-08-30T09:13:31.139 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.131+0000 7f907d244740 1 client.17523 shutdown 2024-08-30T09:13:31.139 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.131+0000 7f907d244740 1 -- 172.21.15.42:0/1062795541 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562a3357e4d0 msgr2=0x562a339539f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:13:31.139 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.131+0000 7f907d244740 1 --2- 172.21.15.42:0/1062795541 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562a3357e4d0 0x562a339539f0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f90600cbfa0 tx=0x7f9060000bc0 comp rx=0 tx=0).stop 2024-08-30T09:13:31.139 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.131+0000 7f907d244740 1 -- 172.21.15.42:0/1062795541 shutdown_connections 2024-08-30T09:13:31.139 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.131+0000 7f907d244740 1 --2- 172.21.15.42:0/1062795541 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x562a3355dd80 0x562a33960b50 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:31.139 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.131+0000 7f907d244740 1 --2- 172.21.15.42:0/1062795541 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562a3359f1b0 0x562a339544f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:31.139 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.131+0000 7f907d244740 1 --2- 172.21.15.42:0/1062795541 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562a3357e4d0 0x562a339539f0 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:31.139 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.131+0000 7f904effd640 0 client.17523 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:13:31.139 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.131+0000 7f907d244740 1 --2- 172.21.15.42:0/1062795541 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562a335762f0 0x562a33953330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:31.140 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.131+0000 7f907d244740 1 -- 172.21.15.42:0/1062795541 >> 172.21.15.42:0/1062795541 conn(0x562a33593300 msgr2=0x562a3393f370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:13:31.140 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.132+0000 7f907d244740 1 -- 172.21.15.42:0/1062795541 shutdown_connections 2024-08-30T09:13:31.140 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.132+0000 7f907d244740 1 -- 172.21.15.42:0/1062795541 wait complete. 2024-08-30T09:13:31.140 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.132+0000 7f907d244740 20 client.17523 trim_cache size 0 max 16384 2024-08-30T09:13:31.154 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.150+0000 7f907d244740 1 Processor -- start 2024-08-30T09:13:31.155 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.150+0000 7f907d244740 1 -- start start 2024-08-30T09:13:31.155 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.150+0000 7f907d244740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562a335762f0 0x562a3394a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:31.155 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.150+0000 7f907d244740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562a3357e4d0 0x562a33940750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:31.155 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.150+0000 7f907d244740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562a3359f1b0 0x562a3393fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:31.155 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.150+0000 7f907d244740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f90701b51a0 con 0x562a335762f0 2024-08-30T09:13:31.155 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.150+0000 7f907d244740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f90701b2a10 con 0x562a3359f1b0 2024-08-30T09:13:31.155 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.150+0000 7f907d244740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f90701b0860 con 0x562a3357e4d0 2024-08-30T09:13:31.155 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.150+0000 7f906f7fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562a335762f0 0x562a3394a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:13:31.155 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.150+0000 7f906f7fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562a335762f0 0x562a3394a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:59812/0 (socket says 172.21.15.42:59812) 2024-08-30T09:13:31.155 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.150+0000 7f906f7fe640 1 -- 172.21.15.42:0/3449449601 learned_addr learned my addr 172.21.15.42:0/3449449601 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:13:31.155 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.151+0000 7f906effd640 1 --2- 172.21.15.42:0/3449449601 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562a3357e4d0 0x562a33940750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:13:31.155 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.151+0000 7f906ffff640 1 --2- 172.21.15.42:0/3449449601 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562a3359f1b0 0x562a3393fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:13:31.155 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.151+0000 7f906f7fe640 1 -- 172.21.15.42:0/3449449601 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562a3357e4d0 msgr2=0x562a33940750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:13:31.155 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.151+0000 7f906f7fe640 1 --2- 172.21.15.42:0/3449449601 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562a3357e4d0 0x562a33940750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:31.156 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.151+0000 7f906f7fe640 1 -- 172.21.15.42:0/3449449601 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562a3359f1b0 msgr2=0x562a3393fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:13:31.156 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.151+0000 7f906f7fe640 1 --2- 172.21.15.42:0/3449449601 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562a3359f1b0 0x562a3393fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:31.156 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.151+0000 7f906f7fe640 1 -- 172.21.15.42:0/3449449601 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a3393f160 con 0x562a335762f0 2024-08-30T09:13:31.156 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.151+0000 7f906effd640 1 --2- 172.21.15.42:0/3449449601 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562a3357e4d0 0x562a33940750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-30T09:13:31.156 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.151+0000 7f906f7fe640 1 --2- 172.21.15.42:0/3449449601 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562a335762f0 0x562a3394a5d0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f906400afd0 tx=0x7f9064017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25a0dd3031b029c5 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:13:31.156 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.151+0000 7f90557fa640 1 -- 172.21.15.42:0/3449449601 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f906401bdf0 con 0x562a335762f0 2024-08-30T09:13:31.156 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.151+0000 7f90557fa640 1 -- 172.21.15.42:0/3449449601 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f90640054d0 con 0x562a335762f0 2024-08-30T09:13:31.156 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.151+0000 7f90557fa640 1 -- 172.21.15.42:0/3449449601 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9064028670 con 0x562a335762f0 2024-08-30T09:13:31.159 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.152+0000 7f907d244740 1 -- 172.21.15.42:0/3449449601 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562a335762f0 msgr2=0x562a3394a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:13:31.159 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.152+0000 7f907d244740 1 --2- 172.21.15.42:0/3449449601 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562a335762f0 0x562a3394a5d0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f906400afd0 tx=0x7f9064017000 comp rx=0 tx=0).stop 2024-08-30T09:13:31.159 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.152+0000 7f907d244740 1 -- 172.21.15.42:0/3449449601 shutdown_connections 2024-08-30T09:13:31.160 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.152+0000 7f907d244740 1 --2- 172.21.15.42:0/3449449601 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562a3357e4d0 0x562a33940750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:31.160 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.152+0000 7f907d244740 1 --2- 172.21.15.42:0/3449449601 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562a3359f1b0 0x562a3393fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:31.160 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.152+0000 7f907d244740 1 --2- 172.21.15.42:0/3449449601 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562a335762f0 0x562a3394a5d0 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:31.160 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.152+0000 7f907d244740 1 -- 172.21.15.42:0/3449449601 >> 172.21.15.42:0/3449449601 conn(0x562a33593300 msgr2=0x562a3393b6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:13:31.160 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.153+0000 7f907d244740 1 -- 172.21.15.42:0/3449449601 shutdown_connections 2024-08-30T09:13:31.160 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.153+0000 7f907d244740 1 -- 172.21.15.42:0/3449449601 wait complete. 2024-08-30T09:13:31.160 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.153+0000 7f907d244740 1 -- start start 2024-08-30T09:13:31.160 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.153+0000 7f907d244740 1 Processor -- start 2024-08-30T09:13:31.160 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.153+0000 7f907d244740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562a335762f0 0x562a339492a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:31.160 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.153+0000 7f907d244740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562a3357e4d0 0x562a33949940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:31.160 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.153+0000 7f907d244740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562a3359f1b0 0x562a33955230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:31.160 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.154+0000 7f907d244740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f90701b0ed0 con 0x562a335762f0 2024-08-30T09:13:31.160 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.154+0000 7f907d244740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f90701b2340 con 0x562a3359f1b0 2024-08-30T09:13:31.161 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.154+0000 7f907d244740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f90701b15a0 con 0x562a3357e4d0 2024-08-30T09:13:31.161 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.154+0000 7f907d244740 10 client.0 _ll_register_callbacks cb 0x7f9060009290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:13:31.161 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.154+0000 7f906effd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562a3357e4d0 0x562a33949940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:13:31.161 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.154+0000 7f906f7fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562a335762f0 0x562a339492a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:13:31.161 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.154+0000 7f906effd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562a3357e4d0 0x562a33949940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:57638/0 (socket says 172.21.15.42:57638) 2024-08-30T09:13:31.161 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.154+0000 7f906effd640 1 -- 172.21.15.42:0/1771789937 learned_addr learned my addr 172.21.15.42:0/1771789937 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:13:31.161 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.154+0000 7f906ffff640 1 --2- 172.21.15.42:0/1771789937 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562a3359f1b0 0x562a33955230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:13:31.161 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.154+0000 7f906f7fe640 1 -- 172.21.15.42:0/1771789937 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562a3357e4d0 msgr2=0x562a33949940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:13:31.161 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.154+0000 7f906f7fe640 1 --2- 172.21.15.42:0/1771789937 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562a3357e4d0 0x562a33949940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:31.161 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.154+0000 7f906f7fe640 1 -- 172.21.15.42:0/1771789937 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562a3359f1b0 msgr2=0x562a33955230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:13:31.161 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.154+0000 7f906f7fe640 1 --2- 172.21.15.42:0/1771789937 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562a3359f1b0 0x562a33955230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:31.161 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.154+0000 7f906f7fe640 1 -- 172.21.15.42:0/1771789937 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562a339559c0 con 0x562a335762f0 2024-08-30T09:13:31.161 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.154+0000 7f906effd640 1 --2- 172.21.15.42:0/1771789937 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562a3357e4d0 0x562a33949940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-30T09:13:31.161 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.154+0000 7f906f7fe640 1 --2- 172.21.15.42:0/1771789937 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562a335762f0 0x562a339492a0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x562a33953870 tx=0x7f90640057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=356bbc84929d05e5 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:13:31.162 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.154+0000 7f904effd640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-08-30T09:13:31.162 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.154+0000 7f904effd640 1 -- 172.21.15.42:0/1771789937 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90640046c0 con 0x562a335762f0 2024-08-30T09:13:31.162 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.154+0000 7f904effd640 1 -- 172.21.15.42:0/1771789937 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f9064004860 con 0x562a335762f0 2024-08-30T09:13:31.162 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.154+0000 7f907d244740 1 -- 172.21.15.42:0/1771789937 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f90640054d0 con 0x562a335762f0 2024-08-30T09:13:31.162 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.155+0000 7f904effd640 1 -- 172.21.15.42:0/1771789937 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f9064045050 con 0x562a335762f0 2024-08-30T09:13:31.162 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.155+0000 7f907d244740 10 client.17541 fetch_fsmap learned FSMap version 45 2024-08-30T09:13:31.162 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.155+0000 7f907d244740 1 -- 172.21.15.42:0/1771789937 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x562a3395ac90 con 0x562a335762f0 2024-08-30T09:13:31.162 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.155+0000 7f904effd640 1 -- 172.21.15.42:0/1771789937 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f90640054d0 con 0x562a335762f0 2024-08-30T09:13:31.162 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.155+0000 7f904effd640 1 -- 172.21.15.42:0/1771789937 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 45) ==== 8362+0+0 (secure 0 0 0) 0x7f906402d070 con 0x562a335762f0 2024-08-30T09:13:31.162 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.155+0000 7f907d244740 10 client.17541 fetch_fsmap finished waiting for FSMap version 45 2024-08-30T09:13:31.162 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.155+0000 7f907d244740 10 client.17541 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-30T09:13:31.162 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.155+0000 7f907d244740 20 client.17541 populate_metadata read hostname 'smithi042' 2024-08-30T09:13:31.162 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.155+0000 7f907d244740 1 --2- 172.21.15.42:0/1771789937 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x562a335331c0 0x562a339d1f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:31.162 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.155+0000 7f907d244740 4 client.17541 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-30T09:13:31.163 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.155+0000 7f907d244740 1 -- 172.21.15.42:0/1771789937 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562a338b59f0 con 0x562a335331c0 2024-08-30T09:13:31.163 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.155+0000 7f906effd640 1 --2- 172.21.15.42:0/1771789937 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x562a335331c0 0x562a339d1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:13:31.163 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.156+0000 7f904effd640 10 client.17541 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:13:31.163 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.156+0000 7f906effd640 1 --2- 172.21.15.42:0/1771789937 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x562a335331c0 0x562a339d1f60 crc :-1 s=READY pgs=262 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=49ad37926f835a05 server_cookie=2ac50a28b1285860 in_seq=0 out_seq=0 2024-08-30T09:13:31.228 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.224+0000 7f904effd640 1 -- 172.21.15.42:0/1771789937 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562a338b59f0 con 0x562a335331c0 2024-08-30T09:13:31.228 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.224+0000 7f904effd640 10 client.17541 handle_command_reply: tid=0 2024-08-30T09:13:31.229 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.224+0000 7f907d244740 1 client.17541 shutdown 2024-08-30T09:13:31.229 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.224+0000 7f907d244740 1 -- 172.21.15.42:0/1771789937 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562a335762f0 msgr2=0x562a339492a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:13:31.229 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.224+0000 7f907d244740 1 --2- 172.21.15.42:0/1771789937 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562a335762f0 0x562a339492a0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x562a33953870 tx=0x7f90640057c0 comp rx=0 tx=0).stop 2024-08-30T09:13:31.229 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.224+0000 7f907d244740 1 -- 172.21.15.42:0/1771789937 shutdown_connections 2024-08-30T09:13:31.229 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.224+0000 7f907d244740 1 --2- 172.21.15.42:0/1771789937 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x562a335331c0 0x562a339d1f60 crc :-1 s=READY pgs=262 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:31.229 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.224+0000 7f907d244740 1 --2- 172.21.15.42:0/1771789937 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562a3357e4d0 0x562a33949940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:31.229 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.224+0000 7f907d244740 1 --2- 172.21.15.42:0/1771789937 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562a3359f1b0 0x562a33955230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:31.229 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.224+0000 7f907d244740 1 --2- 172.21.15.42:0/1771789937 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562a335762f0 0x562a339492a0 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:31.229 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.224+0000 7f907d244740 1 -- 172.21.15.42:0/1771789937 >> 172.21.15.42:0/1771789937 conn(0x562a33593300 msgr2=0x562a3394a5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:13:31.229 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.224+0000 7f904effd640 0 client.17541 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:13:31.229 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.224+0000 7f907d244740 1 -- 172.21.15.42:0/1771789937 shutdown_connections 2024-08-30T09:13:31.229 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.224+0000 7f907d244740 1 -- 172.21.15.42:0/1771789937 wait complete. 2024-08-30T09:13:31.230 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.224+0000 7f907d244740 20 client.17541 trim_cache size 0 max 16384 2024-08-30T09:13:31.230 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-08-30T09:13:31.230 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2024-08-30T09:13:31.230 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-08-30T09:13:31.230 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-08-30T09:13:31.232 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.229+0000 7f9074f9d640 1 -- 172.21.15.42:0/3092739382 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f90580b4b30 msgr2=0x7f90580b7040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:13:31.232 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.229+0000 7f9074f9d640 1 --2- 172.21.15.42:0/3092739382 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f90580b4b30 0x7f90580b7040 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f90640058c0 tx=0x7f906400a380 comp rx=0 tx=0).stop 2024-08-30T09:13:31.232 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.229+0000 7f9074f9d640 1 -- 172.21.15.42:0/3092739382 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f907010aab0 msgr2=0x7f90701a2960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:13:31.232 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.229+0000 7f9074f9d640 1 --2- 172.21.15.42:0/3092739382 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f907010aab0 0x7f90701a2960 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f9060006010 tx=0x7f9060004270 comp rx=0 tx=0).stop 2024-08-30T09:13:31.235 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.231+0000 7f9074f9d640 1 -- 172.21.15.42:0/3092739382 shutdown_connections 2024-08-30T09:13:31.235 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.231+0000 7f9074f9d640 1 --2- 172.21.15.42:0/3092739382 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f90580b4b30 0x7f90580b7040 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:31.235 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.231+0000 7f9074f9d640 1 --2- 172.21.15.42:0/3092739382 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f907010aab0 0x7f90701a2960 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:31.236 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.231+0000 7f9074f9d640 1 --2- 172.21.15.42:0/3092739382 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f90701119e0 0x7f90701a3510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:31.236 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.231+0000 7f9074f9d640 1 --2- 172.21.15.42:0/3092739382 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f907010a160 0x7f90701a22f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:31.236 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.231+0000 7f9074f9d640 1 -- 172.21.15.42:0/3092739382 >> 172.21.15.42:0/3092739382 conn(0x7f90700703e0 msgr2=0x7f907010d480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:13:31.236 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.231+0000 7f9074f9d640 1 -- 172.21.15.42:0/3092739382 shutdown_connections 2024-08-30T09:13:31.236 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:31.231+0000 7f9074f9d640 1 -- 172.21.15.42:0/3092739382 wait complete. 2024-08-30T09:13:31.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:13:30 smithi167 ceph-mon[22063]: pgmap v755: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 5.4 MiB/s wr, 528 op/s 2024-08-30T09:13:31.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:13:30 smithi190 ceph-mon[22001]: pgmap v755: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 5.4 MiB/s wr, 528 op/s 2024-08-30T09:13:32.419 INFO:teuthology.orchestra.run.smithi042.stdout:[] 2024-08-30T09:13:32.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.140+0000 7fccaaf8b640 1 -- 172.21.15.42:0/3363389743 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcca4102540 msgr2=0x7fcca4102950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:13:32.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.140+0000 7fccaaf8b640 1 --2- 172.21.15.42:0/3363389743 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcca4102540 0x7fcca4102950 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fcc94009890 tx=0x7fcc9402a380 comp rx=0 tx=0).stop 2024-08-30T09:13:32.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.140+0000 7fccaaf8b640 1 -- 172.21.15.42:0/3363389743 shutdown_connections 2024-08-30T09:13:32.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.140+0000 7fccaaf8b640 1 --2- 172.21.15.42:0/3363389743 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcca4107980 0x7fcca4107e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:32.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.140+0000 7fccaaf8b640 1 --2- 172.21.15.42:0/3363389743 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcca4102e90 0x7fcca41073c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:32.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.140+0000 7fccaaf8b640 1 --2- 172.21.15.42:0/3363389743 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcca4102540 0x7fcca4102950 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:32.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.140+0000 7fccaaf8b640 1 -- 172.21.15.42:0/3363389743 >> 172.21.15.42:0/3363389743 conn(0x7fcca40794d0 msgr2=0x7fcca40798e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:13:32.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.140+0000 7fccaaf8b640 1 -- 172.21.15.42:0/3363389743 shutdown_connections 2024-08-30T09:13:32.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.140+0000 7fccaaf8b640 1 -- 172.21.15.42:0/3363389743 wait complete. 2024-08-30T09:13:32.421 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.140+0000 7fccaaf8b640 1 Processor -- start 2024-08-30T09:13:32.421 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.140+0000 7fccaaf8b640 1 -- start start 2024-08-30T09:13:32.421 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.141+0000 7fccaaf8b640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcca4102540 0x7fcca41a0890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:32.421 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.141+0000 7fccaaf8b640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcca4102e90 0x7fcca41a0e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:32.421 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.141+0000 7fccaaf8b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcca4107980 0x7fcca41a60d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:32.421 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.141+0000 7fccaaf8b640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fcca4119e20 con 0x7fcca4102e90 2024-08-30T09:13:32.421 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.141+0000 7fccaaf8b640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fcca4119ca0 con 0x7fcca4102540 2024-08-30T09:13:32.421 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.141+0000 7fccaaf8b640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fcca4119fa0 con 0x7fcca4107980 2024-08-30T09:13:32.421 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.141+0000 7fcca3fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcca4102e90 0x7fcca41a0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:13:32.421 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.141+0000 7fcca3fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcca4102e90 0x7fcca41a0e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:59788/0 (socket says 172.21.15.42:59788) 2024-08-30T09:13:32.421 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.141+0000 7fcca3fff640 1 -- 172.21.15.42:0/199512344 learned_addr learned my addr 172.21.15.42:0/199512344 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:13:32.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.141+0000 7fcca8d27640 1 --2- 172.21.15.42:0/199512344 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcca4102540 0x7fcca41a0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:13:32.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.141+0000 7fcca9528640 1 --2- 172.21.15.42:0/199512344 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcca4107980 0x7fcca41a60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:13:32.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.141+0000 7fcca3fff640 1 -- 172.21.15.42:0/199512344 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcca4107980 msgr2=0x7fcca41a60d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:13:32.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.141+0000 7fcca3fff640 1 --2- 172.21.15.42:0/199512344 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcca4107980 0x7fcca41a60d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:32.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.141+0000 7fcca3fff640 1 -- 172.21.15.42:0/199512344 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcca4102540 msgr2=0x7fcca41a0890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:13:32.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.141+0000 7fcca3fff640 1 --2- 172.21.15.42:0/199512344 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcca4102540 0x7fcca41a0890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:32.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.141+0000 7fcca3fff640 1 -- 172.21.15.42:0/199512344 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcca41a6800 con 0x7fcca4102e90 2024-08-30T09:13:32.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.141+0000 7fcca9528640 1 --2- 172.21.15.42:0/199512344 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcca4107980 0x7fcca41a60d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-30T09:13:32.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.141+0000 7fcca3fff640 1 --2- 172.21.15.42:0/199512344 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcca4102e90 0x7fcca41a0e30 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fcc8c00efc0 tx=0x7fcc8c00c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:13:32.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.142+0000 7fcca1ffb640 1 -- 172.21.15.42:0/199512344 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc8c004b20 con 0x7fcca4102e90 2024-08-30T09:13:32.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.142+0000 7fccaaf8b640 1 -- 172.21.15.42:0/199512344 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcca41a6af0 con 0x7fcca4102e90 2024-08-30T09:13:32.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.142+0000 7fcca1ffb640 1 -- 172.21.15.42:0/199512344 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcc8c004cc0 con 0x7fcca4102e90 2024-08-30T09:13:32.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.142+0000 7fcca1ffb640 1 -- 172.21.15.42:0/199512344 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc8c005480 con 0x7fcca4102e90 2024-08-30T09:13:32.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.142+0000 7fccaaf8b640 1 -- 172.21.15.42:0/199512344 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcca41ad250 con 0x7fcca4102e90 2024-08-30T09:13:32.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.144+0000 7fcca1ffb640 1 -- 172.21.15.42:0/199512344 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7fcc8c005620 con 0x7fcca4102e90 2024-08-30T09:13:32.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.144+0000 7fccb1500740 1 Processor -- start 2024-08-30T09:13:32.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.145+0000 7fccb1500740 1 -- start start 2024-08-30T09:13:32.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.145+0000 7fccb1500740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b60f00b070 0x55b60f3b0350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:32.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.145+0000 7fccb1500740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b60efe21b0 0x55b60f3b0890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:32.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.145+0000 7fccb1500740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b60efea390 0x55b60f3b0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:32.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.145+0000 7fccb1500740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55b60f327680 con 0x55b60efe21b0 2024-08-30T09:13:32.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.145+0000 7fccb1500740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55b60f316c10 con 0x55b60f00b070 2024-08-30T09:13:32.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.145+0000 7fccb1500740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55b60f31abc0 con 0x55b60efea390 2024-08-30T09:13:32.424 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.145+0000 7fcca8d27640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b60efe21b0 0x55b60f3b0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:13:32.424 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.145+0000 7fcca8d27640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b60efe21b0 0x55b60f3b0890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:59802/0 (socket says 172.21.15.42:59802) 2024-08-30T09:13:32.424 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.145+0000 7fcca8d27640 1 -- 172.21.15.42:0/1025458228 learned_addr learned my addr 172.21.15.42:0/1025458228 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:13:32.424 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.145+0000 7fcca1ffb640 1 --2- 172.21.15.42:0/199512344 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fcc900b4b30 0x7fcc900b7040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:32.424 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.145+0000 7fcca3fff640 1 --2- 172.21.15.42:0/1025458228 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b60efea390 0x55b60f3b0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:13:32.424 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.145+0000 7fcca9528640 1 --2- 172.21.15.42:0/1025458228 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b60f00b070 0x55b60f3b0350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:13:32.424 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.146+0000 7fcca8d27640 1 -- 172.21.15.42:0/1025458228 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b60efea390 msgr2=0x55b60f3b0dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:13:32.424 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.146+0000 7fcca8d27640 1 --2- 172.21.15.42:0/1025458228 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b60efea390 0x55b60f3b0dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:32.424 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.146+0000 7fcca8d27640 1 -- 172.21.15.42:0/1025458228 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b60f00b070 msgr2=0x55b60f3b0350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:13:32.424 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.146+0000 7fcca8d27640 1 --2- 172.21.15.42:0/1025458228 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b60f00b070 0x55b60f3b0350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:32.425 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.146+0000 7fcca8d27640 1 -- 172.21.15.42:0/1025458228 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b60f076650 con 0x55b60efe21b0 2024-08-30T09:13:32.425 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.146+0000 7fcca8d27640 1 --2- 172.21.15.42:0/1025458228 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b60efe21b0 0x55b60f3b0890 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fcc94009890 tx=0x7fcc94009430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f57e194ffd6b0f13 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:13:32.425 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.146+0000 7fcca9528640 1 --2- 172.21.15.42:0/199512344 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fcc900b4b30 0x7fcc900b7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:13:32.425 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.146+0000 7fcc9affd640 1 -- 172.21.15.42:0/1025458228 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc94037040 con 0x55b60efe21b0 2024-08-30T09:13:32.425 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.146+0000 7fcc9affd640 1 -- 172.21.15.42:0/1025458228 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcc94004030 con 0x55b60efe21b0 2024-08-30T09:13:32.425 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.146+0000 7fcc9affd640 1 -- 172.21.15.42:0/1025458228 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc940035f0 con 0x55b60efe21b0 2024-08-30T09:13:32.425 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.146+0000 7fcca1ffb640 1 -- 172.21.15.42:0/199512344 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7fcc8c0e56c0 con 0x7fcca4102e90 2024-08-30T09:13:32.425 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.147+0000 7fccb1500740 1 -- 172.21.15.42:0/1025458228 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b60efe21b0 msgr2=0x55b60f3b0890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:13:32.425 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.147+0000 7fccb1500740 1 --2- 172.21.15.42:0/1025458228 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b60efe21b0 0x55b60f3b0890 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fcc94009890 tx=0x7fcc94009430 comp rx=0 tx=0).stop 2024-08-30T09:13:32.425 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.147+0000 7fcca9528640 1 --2- 172.21.15.42:0/199512344 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fcc900b4b30 0x7fcc900b7040 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fcc9c014690 tx=0x7fcc9c00b770 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:13:32.425 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.147+0000 7fccb1500740 1 -- 172.21.15.42:0/1025458228 shutdown_connections 2024-08-30T09:13:32.425 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.147+0000 7fccb1500740 1 --2- 172.21.15.42:0/1025458228 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b60efea390 0x55b60f3b0dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:32.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.147+0000 7fccb1500740 1 --2- 172.21.15.42:0/1025458228 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b60efe21b0 0x55b60f3b0890 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:32.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.147+0000 7fccb1500740 1 --2- 172.21.15.42:0/1025458228 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b60f00b070 0x55b60f3b0350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:32.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.147+0000 7fccb1500740 1 -- 172.21.15.42:0/1025458228 >> 172.21.15.42:0/1025458228 conn(0x55b60efff460 msgr2=0x55b60f3ae240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:13:32.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.147+0000 7fccb1500740 1 -- 172.21.15.42:0/1025458228 shutdown_connections 2024-08-30T09:13:32.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.147+0000 7fccb1500740 1 -- 172.21.15.42:0/1025458228 wait complete. 2024-08-30T09:13:32.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.147+0000 7fccb1500740 1 -- start start 2024-08-30T09:13:32.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.147+0000 7fccb1500740 1 Processor -- start 2024-08-30T09:13:32.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.148+0000 7fccb1500740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b60efe21b0 0x55b60f3bbab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:32.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.148+0000 7fccb1500740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b60efea390 0x55b60f3bbff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:32.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.148+0000 7fccb1500740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b60f00b070 0x55b60f3c7df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:32.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.148+0000 7fccb1500740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55b60f179700 con 0x55b60efe21b0 2024-08-30T09:13:32.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.148+0000 7fccb1500740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55b60f29d750 con 0x55b60efea390 2024-08-30T09:13:32.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.148+0000 7fccb1500740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55b60f3bead0 con 0x55b60f00b070 2024-08-30T09:13:32.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.148+0000 7fcca8d27640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b60efe21b0 0x55b60f3bbab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:13:32.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.148+0000 7fccb1500740 10 client.0 _ll_register_callbacks cb 0x55b60f2cce10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:13:32.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.148+0000 7fcca8d27640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b60efe21b0 0x55b60f3bbab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:59810/0 (socket says 172.21.15.42:59810) 2024-08-30T09:13:32.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.148+0000 7fcca8d27640 1 -- 172.21.15.42:0/3073925310 learned_addr learned my addr 172.21.15.42:0/3073925310 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:13:32.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.148+0000 7fcca3fff640 1 --2- 172.21.15.42:0/3073925310 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b60efea390 0x55b60f3bbff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:13:32.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.148+0000 7fcca9528640 1 --2- 172.21.15.42:0/3073925310 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b60f00b070 0x55b60f3c7df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:13:32.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.148+0000 7fcca8d27640 1 -- 172.21.15.42:0/3073925310 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b60f00b070 msgr2=0x55b60f3c7df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:13:32.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.148+0000 7fcca8d27640 1 --2- 172.21.15.42:0/3073925310 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b60f00b070 0x55b60f3c7df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:32.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.148+0000 7fcca8d27640 1 -- 172.21.15.42:0/3073925310 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b60efea390 msgr2=0x55b60f3bbff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:13:32.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.148+0000 7fcca8d27640 1 --2- 172.21.15.42:0/3073925310 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b60efea390 0x55b60f3bbff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:32.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.148+0000 7fcca8d27640 1 -- 172.21.15.42:0/3073925310 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b60f2d6900 con 0x55b60efe21b0 2024-08-30T09:13:32.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.148+0000 7fcca9528640 1 --2- 172.21.15.42:0/3073925310 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b60f00b070 0x55b60f3c7df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-30T09:13:32.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.149+0000 7fcca8d27640 1 --2- 172.21.15.42:0/3073925310 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b60efe21b0 0x55b60f3bbab0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fcc94035040 tx=0x7fcc9402a380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93d854d5eff46b0e server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:13:32.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.149+0000 7fcc83fff640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-08-30T09:13:32.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.149+0000 7fcc83fff640 1 -- 172.21.15.42:0/3073925310 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc9404d070 con 0x55b60efe21b0 2024-08-30T09:13:32.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.149+0000 7fccb1500740 1 -- 172.21.15.42:0/3073925310 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcc94004030 con 0x55b60efe21b0 2024-08-30T09:13:32.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.149+0000 7fcc83fff640 1 -- 172.21.15.42:0/3073925310 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7fcc940043e0 con 0x55b60efe21b0 2024-08-30T09:13:32.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.149+0000 7fcc83fff640 1 -- 172.21.15.42:0/3073925310 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fcc94059050 con 0x55b60efe21b0 2024-08-30T09:13:32.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.149+0000 7fccb1500740 10 client.17535 fetch_fsmap learned FSMap version 45 2024-08-30T09:13:32.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.149+0000 7fccb1500740 1 -- 172.21.15.42:0/3073925310 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55b60f3ca870 con 0x55b60efe21b0 2024-08-30T09:13:32.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.149+0000 7fcc83fff640 1 -- 172.21.15.42:0/3073925310 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcc94004030 con 0x55b60efe21b0 2024-08-30T09:13:32.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.149+0000 7fcc83fff640 1 -- 172.21.15.42:0/3073925310 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 45) ==== 8362+0+0 (secure 0 0 0) 0x7fcc94003cb0 con 0x55b60efe21b0 2024-08-30T09:13:32.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.149+0000 7fccb1500740 10 client.17535 fetch_fsmap finished waiting for FSMap version 45 2024-08-30T09:13:32.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.149+0000 7fccb1500740 10 client.17535 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-08-30T09:13:32.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.149+0000 7fccb1500740 20 client.17535 populate_metadata read hostname 'smithi042' 2024-08-30T09:13:32.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.149+0000 7fccb1500740 1 --2- 172.21.15.42:0/3073925310 >> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] conn(0x55b60efc99f0 0x55b60f3cdbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:32.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.149+0000 7fccb1500740 4 client.17535 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-30T09:13:32.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.149+0000 7fccb1500740 1 -- 172.21.15.42:0/3073925310 --> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b60f3295d0 con 0x55b60efc99f0 2024-08-30T09:13:32.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.150+0000 7fcca3fff640 1 --2- 172.21.15.42:0/3073925310 >> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] conn(0x55b60efc99f0 0x55b60f3cdbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:13:32.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.150+0000 7fcca3fff640 1 --2- 172.21.15.42:0/3073925310 >> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] conn(0x55b60efc99f0 0x55b60f3cdbd0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1eb09cd23f4d6d7b server_cookie=6ea082c21d50cbcc in_seq=0 out_seq=0 2024-08-30T09:13:32.430 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.150+0000 7fcc83fff640 10 client.17535 ms_handle_connect on v2:172.21.15.190:6838/719981110 2024-08-30T09:13:32.430 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.151+0000 7fcc83fff640 1 -- 172.21.15.42:0/3073925310 <== mds.1 v2:172.21.15.190:6838/719981110 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b60f3295d0 con 0x55b60efc99f0 2024-08-30T09:13:32.430 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.151+0000 7fcc83fff640 10 client.17535 handle_command_reply: tid=0 2024-08-30T09:13:32.430 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.151+0000 7fccb1500740 1 client.17535 shutdown 2024-08-30T09:13:32.430 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.151+0000 7fccb1500740 1 -- 172.21.15.42:0/3073925310 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b60efe21b0 msgr2=0x55b60f3bbab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:13:32.430 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.151+0000 7fccb1500740 1 --2- 172.21.15.42:0/3073925310 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b60efe21b0 0x55b60f3bbab0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fcc94035040 tx=0x7fcc9402a380 comp rx=0 tx=0).stop 2024-08-30T09:13:32.430 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.151+0000 7fccb1500740 1 -- 172.21.15.42:0/3073925310 shutdown_connections 2024-08-30T09:13:32.430 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.151+0000 7fccb1500740 1 --2- 172.21.15.42:0/3073925310 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b60f00b070 0x55b60f3c7df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:32.430 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.151+0000 7fccb1500740 1 --2- 172.21.15.42:0/3073925310 >> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] conn(0x55b60efc99f0 0x55b60f3cdbd0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:32.430 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.151+0000 7fccb1500740 1 --2- 172.21.15.42:0/3073925310 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b60efea390 0x55b60f3bbff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:32.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.151+0000 7fccb1500740 1 --2- 172.21.15.42:0/3073925310 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b60efe21b0 0x55b60f3bbab0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:32.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.151+0000 7fcc83fff640 0 client.17535 ms_handle_reset on v2:172.21.15.190:6838/719981110 2024-08-30T09:13:32.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.151+0000 7fccb1500740 1 -- 172.21.15.42:0/3073925310 >> 172.21.15.42:0/3073925310 conn(0x55b60efff460 msgr2=0x55b60f3b34e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:13:32.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.152+0000 7fccb1500740 1 -- 172.21.15.42:0/3073925310 shutdown_connections 2024-08-30T09:13:32.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.152+0000 7fccb1500740 1 -- 172.21.15.42:0/3073925310 wait complete. 2024-08-30T09:13:32.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.152+0000 7fccb1500740 20 client.17535 trim_cache size 0 max 16384 2024-08-30T09:13:32.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.165+0000 7fccb1500740 1 Processor -- start 2024-08-30T09:13:32.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.165+0000 7fccb1500740 1 -- start start 2024-08-30T09:13:32.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.166+0000 7fccb1500740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b60efe21b0 0x55b60f3b1a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:32.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.166+0000 7fccb1500740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b60efea390 0x55b60f3b8ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:32.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.166+0000 7fccb1500740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b60f00b070 0x55b60f3b91e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:32.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.166+0000 7fccb1500740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fcca41b9880 con 0x55b60efe21b0 2024-08-30T09:13:32.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.166+0000 7fccb1500740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fcca41b70f0 con 0x55b60efea390 2024-08-30T09:13:32.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.166+0000 7fccb1500740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fcca41b4f40 con 0x55b60f00b070 2024-08-30T09:13:32.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.166+0000 7fcca8d27640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b60efe21b0 0x55b60f3b1a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:13:32.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.166+0000 7fcca8d27640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b60efe21b0 0x55b60f3b1a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:59820/0 (socket says 172.21.15.42:59820) 2024-08-30T09:13:32.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.166+0000 7fcca8d27640 1 -- 172.21.15.42:0/2001320685 learned_addr learned my addr 172.21.15.42:0/2001320685 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:13:32.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.166+0000 7fcca3fff640 1 --2- 172.21.15.42:0/2001320685 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b60efea390 0x55b60f3b8ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:13:32.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.166+0000 7fcca9528640 1 --2- 172.21.15.42:0/2001320685 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b60f00b070 0x55b60f3b91e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:13:32.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.166+0000 7fcca8d27640 1 -- 172.21.15.42:0/2001320685 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b60f00b070 msgr2=0x55b60f3b91e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:13:32.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.166+0000 7fcca8d27640 1 --2- 172.21.15.42:0/2001320685 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b60f00b070 0x55b60f3b91e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:32.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.166+0000 7fcca8d27640 1 -- 172.21.15.42:0/2001320685 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b60efea390 msgr2=0x55b60f3b8ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:13:32.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.166+0000 7fcca8d27640 1 --2- 172.21.15.42:0/2001320685 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b60efea390 0x55b60f3b8ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:32.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.166+0000 7fcca8d27640 1 -- 172.21.15.42:0/2001320685 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b60f3be660 con 0x55b60efe21b0 2024-08-30T09:13:32.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.166+0000 7fcca8d27640 1 --2- 172.21.15.42:0/2001320685 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b60efe21b0 0x55b60f3b1a90 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fcc9402a8a0 tx=0x7fcc9403d560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20bbbe5208a99e94 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:13:32.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.167+0000 7fcc9a7fc640 1 -- 172.21.15.42:0/2001320685 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc94037030 con 0x55b60efe21b0 2024-08-30T09:13:32.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.167+0000 7fcc9a7fc640 1 -- 172.21.15.42:0/2001320685 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcc940308c0 con 0x55b60efe21b0 2024-08-30T09:13:32.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.167+0000 7fcc9a7fc640 1 -- 172.21.15.42:0/2001320685 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc9404d040 con 0x55b60efe21b0 2024-08-30T09:13:32.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.167+0000 7fccb1500740 1 -- 172.21.15.42:0/2001320685 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b60efe21b0 msgr2=0x55b60f3b1a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:13:32.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.168+0000 7fccb1500740 1 --2- 172.21.15.42:0/2001320685 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b60efe21b0 0x55b60f3b1a90 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fcc9402a8a0 tx=0x7fcc9403d560 comp rx=0 tx=0).stop 2024-08-30T09:13:32.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.168+0000 7fccb1500740 1 -- 172.21.15.42:0/2001320685 shutdown_connections 2024-08-30T09:13:32.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.168+0000 7fccb1500740 1 --2- 172.21.15.42:0/2001320685 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b60f00b070 0x55b60f3b91e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:32.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.168+0000 7fccb1500740 1 --2- 172.21.15.42:0/2001320685 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b60efea390 0x55b60f3b8ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:32.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.168+0000 7fccb1500740 1 --2- 172.21.15.42:0/2001320685 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b60efe21b0 0x55b60f3b1a90 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:32.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.168+0000 7fccb1500740 1 -- 172.21.15.42:0/2001320685 >> 172.21.15.42:0/2001320685 conn(0x55b60efff460 msgr2=0x55b60f3b1380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:13:32.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.168+0000 7fccb1500740 1 -- 172.21.15.42:0/2001320685 shutdown_connections 2024-08-30T09:13:32.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.168+0000 7fccb1500740 1 -- 172.21.15.42:0/2001320685 wait complete. 2024-08-30T09:13:32.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.168+0000 7fccb1500740 1 -- start start 2024-08-30T09:13:32.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.168+0000 7fccb1500740 1 Processor -- start 2024-08-30T09:13:32.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.169+0000 7fccb1500740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b60efe21b0 0x55b60f42c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:32.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.169+0000 7fccb1500740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b60efea390 0x55b60f42c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:32.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.169+0000 7fccb1500740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b60f00b070 0x55b60f42cd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:32.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.169+0000 7fccb1500740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fcca41b55b0 con 0x55b60efea390 2024-08-30T09:13:32.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.169+0000 7fccb1500740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fcca41b6a20 con 0x55b60efe21b0 2024-08-30T09:13:32.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.169+0000 7fccb1500740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fcca41b5c80 con 0x55b60f00b070 2024-08-30T09:13:32.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.169+0000 7fcca3fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b60efea390 0x55b60f42c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:13:32.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.169+0000 7fcca3fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b60efea390 0x55b60f42c820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:59836/0 (socket says 172.21.15.42:59836) 2024-08-30T09:13:32.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.169+0000 7fcca3fff640 1 -- 172.21.15.42:0/2286079684 learned_addr learned my addr 172.21.15.42:0/2286079684 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:13:32.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.169+0000 7fcca8d27640 1 --2- 172.21.15.42:0/2286079684 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b60efe21b0 0x55b60f42c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:13:32.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.169+0000 7fccb1500740 10 client.0 _ll_register_callbacks cb 0x7fcc8c0a8210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:13:32.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.169+0000 7fcca9528640 1 --2- 172.21.15.42:0/2286079684 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b60f00b070 0x55b60f42cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:13:32.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.169+0000 7fcca3fff640 1 -- 172.21.15.42:0/2286079684 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b60f00b070 msgr2=0x55b60f42cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:13:32.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.169+0000 7fcca3fff640 1 --2- 172.21.15.42:0/2286079684 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b60f00b070 0x55b60f42cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:32.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.169+0000 7fcca3fff640 1 -- 172.21.15.42:0/2286079684 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b60efe21b0 msgr2=0x55b60f42c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:13:32.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.170+0000 7fcca3fff640 1 --2- 172.21.15.42:0/2286079684 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b60efe21b0 0x55b60f42c2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:32.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.170+0000 7fcca3fff640 1 -- 172.21.15.42:0/2286079684 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b60f437490 con 0x55b60efea390 2024-08-30T09:13:32.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.170+0000 7fcca8d27640 1 --2- 172.21.15.42:0/2286079684 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b60efe21b0 0x55b60f42c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-30T09:13:32.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.170+0000 7fcca3fff640 1 --2- 172.21.15.42:0/2286079684 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b60efea390 0x55b60f42c820 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fcc8c00bef0 tx=0x7fcc8c093040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fee16c3a078f0ffe server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:13:32.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.170+0000 7fcc83fff640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-08-30T09:13:32.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.170+0000 7fcc83fff640 1 -- 172.21.15.42:0/2286079684 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc8c0ab4d0 con 0x55b60efea390 2024-08-30T09:13:32.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.170+0000 7fcc83fff640 1 -- 172.21.15.42:0/2286079684 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7fcc8c0930b0 con 0x55b60efea390 2024-08-30T09:13:32.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.170+0000 7fccb1500740 1 -- 172.21.15.42:0/2286079684 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55b60f325070 con 0x55b60efea390 2024-08-30T09:13:32.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.170+0000 7fcc83fff640 1 -- 172.21.15.42:0/2286079684 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fcc8c0ab870 con 0x55b60efea390 2024-08-30T09:13:32.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.170+0000 7fccb1500740 10 client.17547 fetch_fsmap learned FSMap version 45 2024-08-30T09:13:32.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.170+0000 7fccb1500740 1 -- 172.21.15.42:0/2286079684 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55b60f438710 con 0x55b60efea390 2024-08-30T09:13:32.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.170+0000 7fcc83fff640 1 -- 172.21.15.42:0/2286079684 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55b60f325070 con 0x55b60efea390 2024-08-30T09:13:32.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.170+0000 7fcc83fff640 1 -- 172.21.15.42:0/2286079684 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 45) ==== 8362+0+0 (secure 0 0 0) 0x7fcc8c0e4040 con 0x55b60efea390 2024-08-30T09:13:32.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.170+0000 7fccb1500740 10 client.17547 fetch_fsmap finished waiting for FSMap version 45 2024-08-30T09:13:32.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.170+0000 7fccb1500740 10 client.17547 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-08-30T09:13:32.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.170+0000 7fccb1500740 20 client.17547 populate_metadata read hostname 'smithi042' 2024-08-30T09:13:32.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.171+0000 7fccb1500740 1 --2- 172.21.15.42:0/2286079684 >> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] conn(0x55b60ef9eda0 0x55b60f43af10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:32.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.171+0000 7fccb1500740 4 client.17547 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-30T09:13:32.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.171+0000 7fccb1500740 1 -- 172.21.15.42:0/2286079684 --> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b60f3227b0 con 0x55b60ef9eda0 2024-08-30T09:13:32.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.171+0000 7fcca8d27640 1 --2- 172.21.15.42:0/2286079684 >> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] conn(0x55b60ef9eda0 0x55b60f43af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:13:32.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.171+0000 7fcc83fff640 10 client.17547 ms_handle_connect on v2:172.21.15.190:6838/719981110 2024-08-30T09:13:32.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.171+0000 7fcca8d27640 1 --2- 172.21.15.42:0/2286079684 >> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] conn(0x55b60ef9eda0 0x55b60f43af10 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=83dc3135cad6ecc3 server_cookie=dbf97f7bc25ebff9 in_seq=0 out_seq=0 2024-08-30T09:13:32.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.171+0000 7fcc83fff640 1 -- 172.21.15.42:0/2286079684 <== mds.1 v2:172.21.15.190:6838/719981110 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b60f3227b0 con 0x55b60ef9eda0 2024-08-30T09:13:32.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.171+0000 7fcc83fff640 10 client.17547 handle_command_reply: tid=0 2024-08-30T09:13:32.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.171+0000 7fccb1500740 1 client.17547 shutdown 2024-08-30T09:13:32.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.171+0000 7fccb1500740 1 -- 172.21.15.42:0/2286079684 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b60efea390 msgr2=0x55b60f42c820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:13:32.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.171+0000 7fccb1500740 1 --2- 172.21.15.42:0/2286079684 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b60efea390 0x55b60f42c820 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fcc8c00bef0 tx=0x7fcc8c093040 comp rx=0 tx=0).stop 2024-08-30T09:13:32.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.172+0000 7fccb1500740 1 -- 172.21.15.42:0/2286079684 shutdown_connections 2024-08-30T09:13:32.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.172+0000 7fccb1500740 1 --2- 172.21.15.42:0/2286079684 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b60f00b070 0x55b60f42cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:32.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.172+0000 7fccb1500740 1 --2- 172.21.15.42:0/2286079684 >> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] conn(0x55b60ef9eda0 0x55b60f43af10 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:32.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.172+0000 7fccb1500740 1 --2- 172.21.15.42:0/2286079684 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b60efea390 0x55b60f42c820 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:32.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.172+0000 7fcc83fff640 0 client.17547 ms_handle_reset on v2:172.21.15.190:6838/719981110 2024-08-30T09:13:32.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.172+0000 7fccb1500740 1 --2- 172.21.15.42:0/2286079684 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b60efe21b0 0x55b60f42c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:32.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.172+0000 7fccb1500740 1 -- 172.21.15.42:0/2286079684 >> 172.21.15.42:0/2286079684 conn(0x55b60efff460 msgr2=0x55b60f3bcd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:13:32.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.172+0000 7fccb1500740 1 -- 172.21.15.42:0/2286079684 shutdown_connections 2024-08-30T09:13:32.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.172+0000 7fccb1500740 1 -- 172.21.15.42:0/2286079684 wait complete. 2024-08-30T09:13:32.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.172+0000 7fccb1500740 20 client.17547 trim_cache size 0 max 16384 2024-08-30T09:13:32.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.172+0000 7fccaaf8b640 1 -- 172.21.15.42:0/199512344 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fcc900b4b30 msgr2=0x7fcc900b7040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:13:32.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.172+0000 7fccaaf8b640 1 --2- 172.21.15.42:0/199512344 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fcc900b4b30 0x7fcc900b7040 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fcc9c014690 tx=0x7fcc9c00b770 comp rx=0 tx=0).stop 2024-08-30T09:13:32.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.172+0000 7fccaaf8b640 1 -- 172.21.15.42:0/199512344 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcca4102e90 msgr2=0x7fcca41a0e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:13:32.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.172+0000 7fccaaf8b640 1 --2- 172.21.15.42:0/199512344 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcca4102e90 0x7fcca41a0e30 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fcc8c00efc0 tx=0x7fcc8c00c800 comp rx=0 tx=0).stop 2024-08-30T09:13:32.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.172+0000 7fccaaf8b640 1 -- 172.21.15.42:0/199512344 shutdown_connections 2024-08-30T09:13:32.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.172+0000 7fccaaf8b640 1 --2- 172.21.15.42:0/199512344 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fcc900b4b30 0x7fcc900b7040 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:32.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.173+0000 7fccaaf8b640 1 --2- 172.21.15.42:0/199512344 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcca4107980 0x7fcca41a60d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:32.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.173+0000 7fccaaf8b640 1 --2- 172.21.15.42:0/199512344 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcca4102e90 0x7fcca41a0e30 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:32.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.173+0000 7fccaaf8b640 1 --2- 172.21.15.42:0/199512344 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcca4102540 0x7fcca41a0890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:32.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.173+0000 7fccaaf8b640 1 -- 172.21.15.42:0/199512344 >> 172.21.15.42:0/199512344 conn(0x7fcca40794d0 msgr2=0x7fcca4105390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:13:32.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.173+0000 7fccaaf8b640 1 -- 172.21.15.42:0/199512344 shutdown_connections 2024-08-30T09:13:32.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:31.173+0000 7fccaaf8b640 1 -- 172.21.15.42:0/199512344 wait complete. 2024-08-30T09:13:33.275 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:13:32 smithi042 ceph-mon[18652]: pgmap v756: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 4.3 MiB/s wr, 389 op/s 2024-08-30T09:13:33.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:13:32 smithi167 ceph-mon[22063]: pgmap v756: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 4.3 MiB/s wr, 389 op/s 2024-08-30T09:13:33.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:13:32 smithi190 ceph-mon[22001]: pgmap v756: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 4.3 MiB/s wr, 389 op/s 2024-08-30T09:13:34.762 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-08-30T09:13:35.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:13:34 smithi042 ceph-mon[18652]: pgmap v757: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 4.7 MiB/s wr, 475 op/s 2024-08-30T09:13:35.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:13:34 smithi167 ceph-mon[22063]: pgmap v757: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 4.7 MiB/s wr, 475 op/s 2024-08-30T09:13:35.337 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T09:13:35.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:13:34 smithi190 ceph-mon[22001]: pgmap v757: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 4.7 MiB/s wr, 475 op/s 2024-08-30T09:13:35.954 INFO:teuthology.orchestra.run.smithi042.stderr:progress: 770.0 s, 483.2 tps, lat 212.621 ms stddev 261.585 2024-08-30T09:13:36.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.043+0000 7f8a96585640 1 -- 172.21.15.42:0/3490426184 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8a9010a780 msgr2=0x7f8a9010ab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:13:36.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.043+0000 7f8a96585640 1 --2- 172.21.15.42:0/3490426184 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8a9010a780 0x7f8a9010ab90 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f8a88009ca0 tx=0x7f8a8802a8f0 comp rx=0 tx=0).stop 2024-08-30T09:13:36.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.044+0000 7f8a96585640 1 -- 172.21.15.42:0/3490426184 shutdown_connections 2024-08-30T09:13:36.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.044+0000 7f8a96585640 1 --2- 172.21.15.42:0/3490426184 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8a9010a780 0x7f8a9010ab90 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:36.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.044+0000 7f8a96585640 1 --2- 172.21.15.42:0/3490426184 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8a9011c9d0 0x7f8a9011eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:36.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.044+0000 7f8a96585640 1 --2- 172.21.15.42:0/3490426184 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8a90111ce0 0x7f8a9010a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:36.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.044+0000 7f8a96585640 1 -- 172.21.15.42:0/3490426184 >> 172.21.15.42:0/3490426184 conn(0x7f8a90070c60 msgr2=0x7f8a90071070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:13:36.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.045+0000 7f8a96585640 1 -- 172.21.15.42:0/3490426184 shutdown_connections 2024-08-30T09:13:36.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.045+0000 7f8a96585640 1 -- 172.21.15.42:0/3490426184 wait complete. 2024-08-30T09:13:36.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.046+0000 7f8a96585640 1 Processor -- start 2024-08-30T09:13:36.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.046+0000 7f8a96585640 1 -- start start 2024-08-30T09:13:36.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.046+0000 7f8a96585640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8a9010a780 0x7f8a9011a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:36.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.046+0000 7f8a96585640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8a90111ce0 0x7f8a90113480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:36.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.046+0000 7f8a96585640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8a9011c9d0 0x7f8a90113a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:36.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.046+0000 7f8a96585640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f8a90122700 con 0x7f8a9010a780 2024-08-30T09:13:36.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.046+0000 7f8a96585640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f8a90122580 con 0x7f8a9011c9d0 2024-08-30T09:13:36.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.046+0000 7f8a96585640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f8a90122880 con 0x7f8a90111ce0 2024-08-30T09:13:36.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.047+0000 7f8a8ffff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8a9010a780 0x7f8a9011a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:13:36.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.048+0000 7f8a8ffff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8a9010a780 0x7f8a9011a1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:59842/0 (socket says 172.21.15.42:59842) 2024-08-30T09:13:36.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.048+0000 7f8a8ffff640 1 -- 172.21.15.42:0/3393754821 learned_addr learned my addr 172.21.15.42:0/3393754821 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:13:36.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.048+0000 7f8a8f7fe640 1 --2- 172.21.15.42:0/3393754821 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8a90111ce0 0x7f8a90113480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:13:36.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.049+0000 7f8a8ffff640 1 -- 172.21.15.42:0/3393754821 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8a90111ce0 msgr2=0x7f8a90113480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:13:36.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.049+0000 7f8a8ffff640 1 --2- 172.21.15.42:0/3393754821 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8a90111ce0 0x7f8a90113480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:36.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.049+0000 7f8a8ffff640 1 -- 172.21.15.42:0/3393754821 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8a9011c9d0 msgr2=0x7f8a90113a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:13:36.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.049+0000 7f8a94b22640 1 --2- 172.21.15.42:0/3393754821 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8a9011c9d0 0x7f8a90113a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:13:36.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.049+0000 7f8a8ffff640 1 --2- 172.21.15.42:0/3393754821 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8a9011c9d0 0x7f8a90113a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:36.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.049+0000 7f8a8ffff640 1 -- 172.21.15.42:0/3393754821 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a90114240 con 0x7f8a9010a780 2024-08-30T09:13:36.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.049+0000 7f8a94b22640 1 --2- 172.21.15.42:0/3393754821 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8a9011c9d0 0x7f8a90113a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-30T09:13:36.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.053+0000 7f8a8ffff640 1 --2- 172.21.15.42:0/3393754821 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8a9010a780 0x7f8a9011a1b0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f8a8400b450 tx=0x7f8a8400b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:13:36.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.054+0000 7f8a8d7fa640 1 -- 172.21.15.42:0/3393754821 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a84015020 con 0x7f8a9010a780 2024-08-30T09:13:36.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.054+0000 7f8a8d7fa640 1 -- 172.21.15.42:0/3393754821 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a84011d40 con 0x7f8a9010a780 2024-08-30T09:13:36.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.054+0000 7f8a96585640 1 -- 172.21.15.42:0/3393754821 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a90118ee0 con 0x7f8a9010a780 2024-08-30T09:13:36.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.054+0000 7f8a96585640 1 -- 172.21.15.42:0/3393754821 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a901193a0 con 0x7f8a9010a780 2024-08-30T09:13:36.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.057+0000 7f8a9cafa740 1 Processor -- start 2024-08-30T09:13:36.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.058+0000 7f8a8d7fa640 1 -- 172.21.15.42:0/3393754821 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a84010e50 con 0x7f8a9010a780 2024-08-30T09:13:36.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.059+0000 7f8a9cafa740 1 -- start start 2024-08-30T09:13:36.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.059+0000 7f8a9cafa740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55627d9af070 0x55627dd54310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:36.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.059+0000 7f8a9cafa740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55627d9861b0 0x55627dd54850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:36.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.059+0000 7f8a9cafa740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55627d98e390 0x55627dd54d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:36.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.059+0000 7f8a9cafa740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55627dccb680 con 0x55627d9861b0 2024-08-30T09:13:36.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.059+0000 7f8a9cafa740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55627dcbac10 con 0x55627d98e390 2024-08-30T09:13:36.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.059+0000 7f8a9cafa740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55627dcbebc0 con 0x55627d9af070 2024-08-30T09:13:36.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.059+0000 7f8a8ffff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55627d9861b0 0x55627dd54850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:13:36.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.059+0000 7f8a8ffff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55627d9861b0 0x55627dd54850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:59852/0 (socket says 172.21.15.42:59852) 2024-08-30T09:13:36.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.059+0000 7f8a8ffff640 1 -- 172.21.15.42:0/3735762645 learned_addr learned my addr 172.21.15.42:0/3735762645 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:13:36.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.060+0000 7f8a8f7fe640 1 --2- 172.21.15.42:0/3735762645 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55627d98e390 0x55627dd54d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:13:36.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.060+0000 7f8a94b22640 1 --2- 172.21.15.42:0/3735762645 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55627d9af070 0x55627dd54310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:13:36.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.063+0000 7f8a8ffff640 1 -- 172.21.15.42:0/3735762645 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55627d9af070 msgr2=0x55627dd54310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:13:36.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.063+0000 7f8a8ffff640 1 --2- 172.21.15.42:0/3735762645 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55627d9af070 0x55627dd54310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:36.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.063+0000 7f8a8ffff640 1 -- 172.21.15.42:0/3735762645 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55627d98e390 msgr2=0x55627dd54d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:13:36.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.063+0000 7f8a8ffff640 1 --2- 172.21.15.42:0/3735762645 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55627d98e390 0x55627dd54d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:36.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.063+0000 7f8a8ffff640 1 -- 172.21.15.42:0/3735762645 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55627da1a650 con 0x55627d9861b0 2024-08-30T09:13:36.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.065+0000 7f8a8d7fa640 1 -- 172.21.15.42:0/3393754821 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7f8a84018020 con 0x7f8a9010a780 2024-08-30T09:13:36.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.067+0000 7f8a8d7fa640 1 --2- 172.21.15.42:0/3393754821 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f8a780b4e60 0x7f8a780b7370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:36.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.067+0000 7f8a8d7fa640 1 -- 172.21.15.42:0/3393754821 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f8a8402a050 con 0x7f8a9010a780 2024-08-30T09:13:36.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.067+0000 7f8a8ffff640 1 --2- 172.21.15.42:0/3735762645 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55627d9861b0 0x55627dd54850 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f8a84010910 tx=0x7f8a840107f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6d4de27473a3d3e server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:13:36.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.068+0000 7f8a94b22640 1 --2- 172.21.15.42:0/3393754821 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f8a780b4e60 0x7f8a780b7370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:13:36.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.071+0000 7f8a767fc640 1 -- 172.21.15.42:0/3735762645 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a840acc30 con 0x55627d9861b0 2024-08-30T09:13:36.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.072+0000 7f8a767fc640 1 -- 172.21.15.42:0/3735762645 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a840ac970 con 0x55627d9861b0 2024-08-30T09:13:36.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.073+0000 7f8a9cafa740 1 -- 172.21.15.42:0/3735762645 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55627d9861b0 msgr2=0x55627dd54850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:13:36.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.073+0000 7f8a9cafa740 1 --2- 172.21.15.42:0/3735762645 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55627d9861b0 0x55627dd54850 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f8a84010910 tx=0x7f8a840107f0 comp rx=0 tx=0).stop 2024-08-30T09:13:36.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.076+0000 7f8a94b22640 1 --2- 172.21.15.42:0/3393754821 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f8a780b4e60 0x7f8a780b7370 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f8a80005d00 tx=0x7f8a8000c2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:13:36.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.078+0000 7f8a9cafa740 1 -- 172.21.15.42:0/3735762645 shutdown_connections 2024-08-30T09:13:36.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.078+0000 7f8a9cafa740 1 --2- 172.21.15.42:0/3735762645 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55627d98e390 0x55627dd54d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:36.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.078+0000 7f8a9cafa740 1 --2- 172.21.15.42:0/3735762645 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55627d9861b0 0x55627dd54850 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:36.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.078+0000 7f8a9cafa740 1 --2- 172.21.15.42:0/3735762645 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55627d9af070 0x55627dd54310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:36.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.078+0000 7f8a9cafa740 1 -- 172.21.15.42:0/3735762645 >> 172.21.15.42:0/3735762645 conn(0x55627d9a3460 msgr2=0x55627dd52200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:13:36.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.085+0000 7f8a9cafa740 1 -- 172.21.15.42:0/3735762645 shutdown_connections 2024-08-30T09:13:36.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.089+0000 7f8a9cafa740 1 -- 172.21.15.42:0/3735762645 wait complete. 2024-08-30T09:13:36.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.089+0000 7f8a9cafa740 1 -- start start 2024-08-30T09:13:36.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.090+0000 7f8a9cafa740 1 Processor -- start 2024-08-30T09:13:36.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.091+0000 7f8a9cafa740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55627d9861b0 0x55627dd5fab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:36.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.091+0000 7f8a9cafa740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55627d98e390 0x55627dd5fff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:36.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.091+0000 7f8a9cafa740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55627d9af070 0x55627dd6bde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:36.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.092+0000 7f8a9cafa740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55627db1d700 con 0x55627d9861b0 2024-08-30T09:13:36.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.092+0000 7f8a94b22640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55627d9af070 0x55627dd6bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:13:36.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.092+0000 7f8a94b22640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55627d9af070 0x55627dd6bde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:50170/0 (socket says 172.21.15.42:50170) 2024-08-30T09:13:36.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.092+0000 7f8a94b22640 1 -- 172.21.15.42:0/946868546 learned_addr learned my addr 172.21.15.42:0/946868546 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:13:36.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.092+0000 7f8a9cafa740 1 -- 172.21.15.42:0/946868546 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55627dc41750 con 0x55627d9af070 2024-08-30T09:13:36.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.092+0000 7f8a9cafa740 1 -- 172.21.15.42:0/946868546 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55627dd62af0 con 0x55627d98e390 2024-08-30T09:13:36.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.093+0000 7f8a9cafa740 10 client.0 _ll_register_callbacks cb 0x55627dc70e10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:13:36.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.093+0000 7f8a94b22640 1 -- 172.21.15.42:0/946868546 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55627d98e390 msgr2=0x55627dd5fff0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-30T09:13:36.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.093+0000 7f8a94b22640 1 --2- 172.21.15.42:0/946868546 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55627d98e390 0x55627dd5fff0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:36.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.093+0000 7f8a94b22640 1 -- 172.21.15.42:0/946868546 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55627d9861b0 msgr2=0x55627dd5fab0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-30T09:13:36.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.093+0000 7f8a94b22640 1 --2- 172.21.15.42:0/946868546 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55627d9861b0 0x55627dd5fab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:36.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.093+0000 7f8a94b22640 1 -- 172.21.15.42:0/946868546 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55627dc7a900 con 0x55627d9af070 2024-08-30T09:13:36.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.094+0000 7f8a94b22640 1 --2- 172.21.15.42:0/946868546 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55627d9af070 0x55627dd6bde0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f8a800058a0 tx=0x7f8a80004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=99002df8d1961d76 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:13:36.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.094+0000 7f8a9cafa740 1 -- 172.21.15.42:0/946868546 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55627dccd5d0 con 0x55627d9af070 2024-08-30T09:13:36.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.095+0000 7f8a6f7fe640 10 client.27401 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-30T09:13:36.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.095+0000 7f8a6f7fe640 1 -- 172.21.15.42:0/946868546 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a80023070 con 0x55627d9af070 2024-08-30T09:13:36.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.095+0000 7f8a6f7fe640 1 -- 172.21.15.42:0/946868546 <== mon.2 v2:172.21.15.167:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f8a8001f860 con 0x55627d9af070 2024-08-30T09:13:36.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.095+0000 7f8a6f7fe640 1 -- 172.21.15.42:0/946868546 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f8a8002f050 con 0x55627d9af070 2024-08-30T09:13:36.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.100+0000 7f8a9cafa740 10 client.27401 fetch_fsmap learned FSMap version 45 2024-08-30T09:13:36.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.100+0000 7f8a9cafa740 1 -- 172.21.15.42:0/946868546 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({fsmap=45}) -- 0x55627dd6e9e0 con 0x55627d9af070 2024-08-30T09:13:36.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.107+0000 7f8a6f7fe640 1 -- 172.21.15.42:0/946868546 <== mon.2 v2:172.21.15.167:3300/0 4 ==== fsmap(e 45) ==== 8362+0+0 (secure 0 0 0) 0x7f8a8000f040 con 0x55627d9af070 2024-08-30T09:13:36.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.107+0000 7f8a9cafa740 10 client.27401 fetch_fsmap finished waiting for FSMap version 45 2024-08-30T09:13:36.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.107+0000 7f8a9cafa740 10 client.27401 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-30T09:13:36.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.107+0000 7f8a9cafa740 20 client.27401 populate_metadata read hostname 'smithi042' 2024-08-30T09:13:36.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.107+0000 7f8a9cafa740 1 --2- 172.21.15.42:0/946868546 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55627d96d9f0 0x55627dd72290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:36.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.107+0000 7f8a9cafa740 4 client.27401 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-30T09:13:36.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.107+0000 7f8a9cafa740 1 -- 172.21.15.42:0/946868546 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55627dcc9070 con 0x55627d96d9f0 2024-08-30T09:13:36.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.109+0000 7f8a8ffff640 1 --2- 172.21.15.42:0/946868546 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55627d96d9f0 0x55627dd72290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:13:36.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.126+0000 7f8a8ffff640 1 --2- 172.21.15.42:0/946868546 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55627d96d9f0 0x55627dd72290 crc :-1 s=READY pgs=263 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2c63686d332ca756 server_cookie=17d52641f6312f2d in_seq=0 out_seq=0 2024-08-30T09:13:36.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.126+0000 7f8a6f7fe640 10 client.27401 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:13:36.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.127+0000 7f8a6f7fe640 1 -- 172.21.15.42:0/946868546 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55627dcc9070 con 0x55627d96d9f0 2024-08-30T09:13:36.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.127+0000 7f8a6f7fe640 10 client.27401 handle_command_reply: tid=0 2024-08-30T09:13:36.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.127+0000 7f8a9cafa740 1 client.27401 shutdown 2024-08-30T09:13:36.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.128+0000 7f8a9cafa740 1 -- 172.21.15.42:0/946868546 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55627d9af070 msgr2=0x55627dd6bde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:13:36.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.128+0000 7f8a9cafa740 1 --2- 172.21.15.42:0/946868546 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55627d9af070 0x55627dd6bde0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f8a800058a0 tx=0x7f8a80004730 comp rx=0 tx=0).stop 2024-08-30T09:13:36.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.129+0000 7f8a9cafa740 1 -- 172.21.15.42:0/946868546 shutdown_connections 2024-08-30T09:13:36.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.129+0000 7f8a9cafa740 1 --2- 172.21.15.42:0/946868546 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55627d96d9f0 0x55627dd72290 crc :-1 s=READY pgs=263 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:36.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.129+0000 7f8a9cafa740 1 --2- 172.21.15.42:0/946868546 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55627d98e390 0x55627dd5fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:36.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.129+0000 7f8a9cafa740 1 --2- 172.21.15.42:0/946868546 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55627d9af070 0x55627dd6bde0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:36.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.129+0000 7f8a9cafa740 1 --2- 172.21.15.42:0/946868546 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55627d9861b0 0x55627dd5fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:36.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.129+0000 7f8a9cafa740 1 -- 172.21.15.42:0/946868546 >> 172.21.15.42:0/946868546 conn(0x55627d9a3460 msgr2=0x55627dd574a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:13:36.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.129+0000 7f8a6f7fe640 0 client.27401 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:13:36.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.129+0000 7f8a9cafa740 1 -- 172.21.15.42:0/946868546 shutdown_connections 2024-08-30T09:13:36.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.129+0000 7f8a9cafa740 1 -- 172.21.15.42:0/946868546 wait complete. 2024-08-30T09:13:36.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.129+0000 7f8a9cafa740 20 client.27401 trim_cache size 0 max 16384 2024-08-30T09:13:36.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.155+0000 7f8a9cafa740 1 Processor -- start 2024-08-30T09:13:36.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.156+0000 7f8a9cafa740 1 -- start start 2024-08-30T09:13:36.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.156+0000 7f8a9cafa740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55627d9861b0 0x55627dd4d230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:36.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.156+0000 7f8a9cafa740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55627d98e390 0x55627dd57410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:36.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.156+0000 7f8a9cafa740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55627d9af070 0x55627dd57950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:36.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.156+0000 7f8a9cafa740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f8a901ca9b0 con 0x55627d98e390 2024-08-30T09:13:36.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.157+0000 7f8a9cafa740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f8a901c8220 con 0x55627d9861b0 2024-08-30T09:13:36.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.157+0000 7f8a9cafa740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f8a901c6070 con 0x55627d9af070 2024-08-30T09:13:36.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.157+0000 7f8a8ffff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55627d9861b0 0x55627dd4d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:13:36.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.157+0000 7f8a8f7fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55627d98e390 0x55627dd57410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:13:36.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.157+0000 7f8a8f7fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55627d98e390 0x55627dd57410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:59862/0 (socket says 172.21.15.42:59862) 2024-08-30T09:13:36.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.157+0000 7f8a8f7fe640 1 -- 172.21.15.42:0/1677316348 learned_addr learned my addr 172.21.15.42:0/1677316348 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:13:36.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.158+0000 7f8a94b22640 1 --2- 172.21.15.42:0/1677316348 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55627d9af070 0x55627dd57950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:13:36.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.161+0000 7f8a8f7fe640 1 -- 172.21.15.42:0/1677316348 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55627d9af070 msgr2=0x55627dd57950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:13:36.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.161+0000 7f8a8f7fe640 1 --2- 172.21.15.42:0/1677316348 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55627d9af070 0x55627dd57950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:36.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.161+0000 7f8a8f7fe640 1 -- 172.21.15.42:0/1677316348 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55627d9861b0 msgr2=0x55627dd4d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:13:36.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.161+0000 7f8a8f7fe640 1 --2- 172.21.15.42:0/1677316348 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55627d9861b0 0x55627dd4d230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:36.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.161+0000 7f8a8f7fe640 1 -- 172.21.15.42:0/1677316348 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55627dd62650 con 0x55627d98e390 2024-08-30T09:13:36.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.166+0000 7f8a8f7fe640 1 --2- 172.21.15.42:0/1677316348 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55627d98e390 0x55627dd57410 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f8a88006010 tx=0x7f8a88009650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a0d54eada51367e server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:13:36.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.167+0000 7f8a75ffb640 1 -- 172.21.15.42:0/1677316348 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a88031a90 con 0x55627d98e390 2024-08-30T09:13:36.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.167+0000 7f8a75ffb640 1 -- 172.21.15.42:0/1677316348 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a88036050 con 0x55627d98e390 2024-08-30T09:13:36.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.167+0000 7f8a75ffb640 1 -- 172.21.15.42:0/1677316348 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a88004040 con 0x55627d98e390 2024-08-30T09:13:36.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.170+0000 7f8a9cafa740 1 -- 172.21.15.42:0/1677316348 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55627d98e390 msgr2=0x55627dd57410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:13:36.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.170+0000 7f8a9cafa740 1 --2- 172.21.15.42:0/1677316348 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55627d98e390 0x55627dd57410 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f8a88006010 tx=0x7f8a88009650 comp rx=0 tx=0).stop 2024-08-30T09:13:36.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.176+0000 7f8a9cafa740 1 -- 172.21.15.42:0/1677316348 shutdown_connections 2024-08-30T09:13:36.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.176+0000 7f8a9cafa740 1 --2- 172.21.15.42:0/1677316348 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55627d9af070 0x55627dd57950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:36.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.176+0000 7f8a9cafa740 1 --2- 172.21.15.42:0/1677316348 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55627d98e390 0x55627dd57410 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:36.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.176+0000 7f8a9cafa740 1 --2- 172.21.15.42:0/1677316348 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55627d9861b0 0x55627dd4d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:36.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.176+0000 7f8a9cafa740 1 -- 172.21.15.42:0/1677316348 >> 172.21.15.42:0/1677316348 conn(0x55627d9a3460 msgr2=0x55627dd5e330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:13:36.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.181+0000 7f8a9cafa740 1 -- 172.21.15.42:0/1677316348 shutdown_connections 2024-08-30T09:13:36.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.181+0000 7f8a9cafa740 1 -- 172.21.15.42:0/1677316348 wait complete. 2024-08-30T09:13:36.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.181+0000 7f8a9cafa740 1 -- start start 2024-08-30T09:13:36.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.182+0000 7f8a9cafa740 1 Processor -- start 2024-08-30T09:13:36.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.183+0000 7f8a9cafa740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55627d9861b0 0x55627dd686b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:36.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.183+0000 7f8a9cafa740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55627d98e390 0x55627dd63b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:36.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.183+0000 7f8a9cafa740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55627d9af070 0x55627dd64250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:36.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.183+0000 7f8a9cafa740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f8a901c66e0 con 0x55627d98e390 2024-08-30T09:13:36.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.183+0000 7f8a9cafa740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f8a901c7b50 con 0x55627d9861b0 2024-08-30T09:13:36.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.183+0000 7f8a94b22640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55627d9af070 0x55627dd64250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:13:36.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.183+0000 7f8a8ffff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55627d9861b0 0x55627dd686b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:13:36.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.183+0000 7f8a94b22640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55627d9af070 0x55627dd64250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:57690/0 (socket says 172.21.15.42:57690) 2024-08-30T09:13:36.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.183+0000 7f8a8ffff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55627d9861b0 0x55627dd686b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:50176/0 (socket says 172.21.15.42:50176) 2024-08-30T09:13:36.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.183+0000 7f8a94b22640 1 -- 172.21.15.42:0/4289562640 learned_addr learned my addr 172.21.15.42:0/4289562640 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:13:36.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.183+0000 7f8a9cafa740 1 -- 172.21.15.42:0/4289562640 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f8a901c6db0 con 0x55627d9af070 2024-08-30T09:13:36.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.184+0000 7f8a9cafa740 10 client.0 _ll_register_callbacks cb 0x7f8a80016070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:13:36.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.184+0000 7f8a8ffff640 1 -- 172.21.15.42:0/4289562640 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55627d9af070 msgr2=0x55627dd64250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:13:36.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.184+0000 7f8a8ffff640 1 --2- 172.21.15.42:0/4289562640 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55627d9af070 0x55627dd64250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:36.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.184+0000 7f8a8ffff640 1 -- 172.21.15.42:0/4289562640 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55627d98e390 msgr2=0x55627dd63b60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-30T09:13:36.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.184+0000 7f8a8ffff640 1 --2- 172.21.15.42:0/4289562640 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55627d98e390 0x55627dd63b60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:36.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.184+0000 7f8a8ffff640 1 -- 172.21.15.42:0/4289562640 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55627dddbc90 con 0x55627d9861b0 2024-08-30T09:13:36.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.188+0000 7f8a8ffff640 1 --2- 172.21.15.42:0/4289562640 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55627d9861b0 0x55627dd686b0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f8a840acdf0 tx=0x7f8a840092a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=10c1a60f6ff441d2 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:13:36.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.190+0000 7f8a6f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-30T09:13:36.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.190+0000 7f8a6f7fe640 1 -- 172.21.15.42:0/4289562640 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a840ac7b0 con 0x55627d9861b0 2024-08-30T09:13:36.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.190+0000 7f8a9cafa740 1 -- 172.21.15.42:0/4289562640 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a88036050 con 0x55627d9861b0 2024-08-30T09:13:36.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.190+0000 7f8a6f7fe640 1 -- 172.21.15.42:0/4289562640 <== mon.2 v2:172.21.15.167:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f8a840a9450 con 0x55627d9861b0 2024-08-30T09:13:36.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.196+0000 7f8a6f7fe640 1 -- 172.21.15.42:0/4289562640 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f8a840b16d0 con 0x55627d9861b0 2024-08-30T09:13:36.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.200+0000 7f8a9cafa740 10 client.27407 fetch_fsmap learned FSMap version 45 2024-08-30T09:13:36.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.200+0000 7f8a9cafa740 1 -- 172.21.15.42:0/4289562640 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55627dddcd50 con 0x55627d9861b0 2024-08-30T09:13:36.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.205+0000 7f8a6f7fe640 1 -- 172.21.15.42:0/4289562640 <== mon.2 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a88036050 con 0x55627d9861b0 2024-08-30T09:13:36.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.205+0000 7f8a6f7fe640 1 -- 172.21.15.42:0/4289562640 <== mon.2 v2:172.21.15.167:3300/0 5 ==== fsmap(e 45) ==== 8362+0+0 (secure 0 0 0) 0x7f8a84022060 con 0x55627d9861b0 2024-08-30T09:13:36.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.208+0000 7f8a9cafa740 10 client.27407 fetch_fsmap finished waiting for FSMap version 45 2024-08-30T09:13:36.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.208+0000 7f8a9cafa740 10 client.27407 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-30T09:13:36.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.208+0000 7f8a9cafa740 20 client.27407 populate_metadata read hostname 'smithi042' 2024-08-30T09:13:36.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.208+0000 7f8a9cafa740 1 --2- 172.21.15.42:0/4289562640 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55627d942da0 0x55627ddded60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:36.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.208+0000 7f8a9cafa740 4 client.27407 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-30T09:13:36.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.208+0000 7f8a9cafa740 1 -- 172.21.15.42:0/4289562640 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55627dcc67b0 con 0x55627d942da0 2024-08-30T09:13:36.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.211+0000 7f8a8f7fe640 1 --2- 172.21.15.42:0/4289562640 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55627d942da0 0x55627ddded60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:13:36.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.219+0000 7f8a8f7fe640 1 --2- 172.21.15.42:0/4289562640 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55627d942da0 0x55627ddded60 crc :-1 s=READY pgs=264 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2541caf36fdfee67 server_cookie=84edabf78bb5aff0 in_seq=0 out_seq=0 2024-08-30T09:13:36.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.219+0000 7f8a6f7fe640 10 client.27407 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:13:36.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.220+0000 7f8a6f7fe640 1 -- 172.21.15.42:0/4289562640 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55627dcc67b0 con 0x55627d942da0 2024-08-30T09:13:36.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.220+0000 7f8a6f7fe640 10 client.27407 handle_command_reply: tid=0 2024-08-30T09:13:36.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.221+0000 7f8a9cafa740 1 client.27407 shutdown 2024-08-30T09:13:36.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.229+0000 7f8a9cafa740 1 -- 172.21.15.42:0/4289562640 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55627d9861b0 msgr2=0x55627dd686b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:13:36.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.229+0000 7f8a9cafa740 1 --2- 172.21.15.42:0/4289562640 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55627d9861b0 0x55627dd686b0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f8a840acdf0 tx=0x7f8a840092a0 comp rx=0 tx=0).stop 2024-08-30T09:13:36.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.229+0000 7f8a9cafa740 1 -- 172.21.15.42:0/4289562640 shutdown_connections 2024-08-30T09:13:36.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.229+0000 7f8a9cafa740 1 --2- 172.21.15.42:0/4289562640 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55627d942da0 0x55627ddded60 crc :-1 s=READY pgs=264 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:36.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.229+0000 7f8a9cafa740 1 --2- 172.21.15.42:0/4289562640 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55627d9af070 0x55627dd64250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:36.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.229+0000 7f8a9cafa740 1 --2- 172.21.15.42:0/4289562640 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55627d98e390 0x55627dd63b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:36.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.229+0000 7f8a9cafa740 1 --2- 172.21.15.42:0/4289562640 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55627d9861b0 0x55627dd686b0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:36.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.229+0000 7f8a6f7fe640 0 client.27407 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:13:36.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.229+0000 7f8a9cafa740 1 -- 172.21.15.42:0/4289562640 >> 172.21.15.42:0/4289562640 conn(0x55627d9a3460 msgr2=0x55627dd55320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:13:36.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.229+0000 7f8a9cafa740 1 -- 172.21.15.42:0/4289562640 shutdown_connections 2024-08-30T09:13:36.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.229+0000 7f8a9cafa740 1 -- 172.21.15.42:0/4289562640 wait complete. 2024-08-30T09:13:36.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.229+0000 7f8a9cafa740 20 client.27407 trim_cache size 0 max 16384 2024-08-30T09:13:36.234 INFO:teuthology.orchestra.run.smithi042.stdout:[] 2024-08-30T09:13:36.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.234+0000 7f8a96585640 1 -- 172.21.15.42:0/3393754821 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f8a780b4e60 msgr2=0x7f8a780b7370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:13:36.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.234+0000 7f8a96585640 1 --2- 172.21.15.42:0/3393754821 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f8a780b4e60 0x7f8a780b7370 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f8a80005d00 tx=0x7f8a8000c2d0 comp rx=0 tx=0).stop 2024-08-30T09:13:36.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.234+0000 7f8a96585640 1 -- 172.21.15.42:0/3393754821 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8a9010a780 msgr2=0x7f8a9011a1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:13:36.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.234+0000 7f8a96585640 1 --2- 172.21.15.42:0/3393754821 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8a9010a780 0x7f8a9011a1b0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f8a8400b450 tx=0x7f8a8400b910 comp rx=0 tx=0).stop 2024-08-30T09:13:36.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.234+0000 7f8a96585640 1 -- 172.21.15.42:0/3393754821 shutdown_connections 2024-08-30T09:13:36.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.234+0000 7f8a96585640 1 --2- 172.21.15.42:0/3393754821 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f8a780b4e60 0x7f8a780b7370 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:36.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.234+0000 7f8a96585640 1 --2- 172.21.15.42:0/3393754821 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8a90111ce0 0x7f8a90113480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:36.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.234+0000 7f8a96585640 1 --2- 172.21.15.42:0/3393754821 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8a9011c9d0 0x7f8a90113a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:36.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.234+0000 7f8a96585640 1 --2- 172.21.15.42:0/3393754821 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8a9010a780 0x7f8a9011a1b0 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:36.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.234+0000 7f8a96585640 1 -- 172.21.15.42:0/3393754821 >> 172.21.15.42:0/3393754821 conn(0x7f8a90070c60 msgr2=0x7f8a9011e150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:13:36.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.235+0000 7f8a96585640 1 -- 172.21.15.42:0/3393754821 shutdown_connections 2024-08-30T09:13:36.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:36.235+0000 7f8a96585640 1 -- 172.21.15.42:0/3393754821 wait complete. 2024-08-30T09:13:37.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:13:36 smithi167 ceph-mon[22063]: pgmap v758: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 3.8 MiB/s wr, 388 op/s 2024-08-30T09:13:37.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:13:36 smithi042 ceph-mon[18652]: pgmap v758: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 3.8 MiB/s wr, 388 op/s 2024-08-30T09:13:37.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:13:36 smithi190 ceph-mon[22001]: pgmap v758: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 3.8 MiB/s wr, 388 op/s 2024-08-30T09:13:38.142 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-08-30T09:13:38.142 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-08-30T09:13:38.743 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T09:13:39.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:13:39 smithi042 ceph-mon[18652]: pgmap v759: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 3.8 MiB/s wr, 358 op/s 2024-08-30T09:13:39.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:13:39 smithi167 ceph-mon[22063]: pgmap v759: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 3.8 MiB/s wr, 358 op/s 2024-08-30T09:13:39.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:13:39 smithi190 ceph-mon[22001]: pgmap v759: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 3.8 MiB/s wr, 358 op/s 2024-08-30T09:13:39.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.327+0000 7fa38fe21640 1 -- 172.21.15.42:0/1529848496 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa38810a780 msgr2=0x7fa38810ab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:13:39.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.327+0000 7fa38fe21640 1 --2- 172.21.15.42:0/1529848496 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa38810a780 0x7fa38810ab90 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fa378009900 tx=0x7fa37802a440 comp rx=0 tx=0).stop 2024-08-30T09:13:39.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.328+0000 7fa38fe21640 1 -- 172.21.15.42:0/1529848496 shutdown_connections 2024-08-30T09:13:39.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.328+0000 7fa38fe21640 1 --2- 172.21.15.42:0/1529848496 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa38811c9d0 0x7fa38811eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:39.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.328+0000 7fa38fe21640 1 --2- 172.21.15.42:0/1529848496 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa38810a780 0x7fa38810ab90 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:39.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.328+0000 7fa38fe21640 1 --2- 172.21.15.42:0/1529848496 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa388111ce0 0x7fa38810a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:39.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.328+0000 7fa38fe21640 1 -- 172.21.15.42:0/1529848496 >> 172.21.15.42:0/1529848496 conn(0x7fa388070c60 msgr2=0x7fa388071070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:13:39.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.328+0000 7fa38fe21640 1 -- 172.21.15.42:0/1529848496 shutdown_connections 2024-08-30T09:13:39.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.332+0000 7fa38fe21640 1 -- 172.21.15.42:0/1529848496 wait complete. 2024-08-30T09:13:39.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.333+0000 7fa38fe21640 1 Processor -- start 2024-08-30T09:13:39.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.333+0000 7fa38fe21640 1 -- start start 2024-08-30T09:13:39.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.333+0000 7fa38fe21640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa388111ce0 0x7fa388113650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:39.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.333+0000 7fa38fe21640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa38811c9d0 0x7fa388113b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:39.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.333+0000 7fa38fe21640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa3881140d0 0x7fa388118ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:39.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.333+0000 7fa38fe21640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fa388122610 con 0x7fa38811c9d0 2024-08-30T09:13:39.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.333+0000 7fa38fe21640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa388122490 con 0x7fa3881140d0 2024-08-30T09:13:39.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.333+0000 7fa38fe21640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fa388122790 con 0x7fa388111ce0 2024-08-30T09:13:39.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.334+0000 7fa38e3be640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa3881140d0 0x7fa388118ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:13:39.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.334+0000 7fa38e3be640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa3881140d0 0x7fa388118ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:57164/0 (socket says 172.21.15.42:57164) 2024-08-30T09:13:39.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.334+0000 7fa38e3be640 1 -- 172.21.15.42:0/2596835641 learned_addr learned my addr 172.21.15.42:0/2596835641 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:13:39.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.334+0000 7fa38dbbd640 1 --2- 172.21.15.42:0/2596835641 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa388111ce0 0x7fa388113650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:13:39.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.337+0000 7fa38d3bc640 1 --2- 172.21.15.42:0/2596835641 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa38811c9d0 0x7fa388113b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:13:39.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.337+0000 7fa38e3be640 1 -- 172.21.15.42:0/2596835641 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa388111ce0 msgr2=0x7fa388113650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:13:39.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.337+0000 7fa38e3be640 1 --2- 172.21.15.42:0/2596835641 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa388111ce0 0x7fa388113650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:39.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.337+0000 7fa38e3be640 1 -- 172.21.15.42:0/2596835641 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa38811c9d0 msgr2=0x7fa388113b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:13:39.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.337+0000 7fa38e3be640 1 --2- 172.21.15.42:0/2596835641 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa38811c9d0 0x7fa388113b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:39.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.337+0000 7fa38e3be640 1 -- 172.21.15.42:0/2596835641 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa388119530 con 0x7fa3881140d0 2024-08-30T09:13:39.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.342+0000 7fa38e3be640 1 --2- 172.21.15.42:0/2596835641 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa3881140d0 0x7fa388118ff0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fa38000efd0 tx=0x7fa38000c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:13:39.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.344+0000 7fa37effd640 1 -- 172.21.15.42:0/2596835641 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa380003b70 con 0x7fa3881140d0 2024-08-30T09:13:39.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.345+0000 7fa37effd640 1 -- 172.21.15.42:0/2596835641 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa380003d10 con 0x7fa3881140d0 2024-08-30T09:13:39.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.345+0000 7fa37effd640 1 -- 172.21.15.42:0/2596835641 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3800108a0 con 0x7fa3881140d0 2024-08-30T09:13:39.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.345+0000 7fa38fe21640 1 -- 172.21.15.42:0/2596835641 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa388119820 con 0x7fa3881140d0 2024-08-30T09:13:39.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.345+0000 7fa38fe21640 1 -- 172.21.15.42:0/2596835641 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa388119ce0 con 0x7fa3881140d0 2024-08-30T09:13:39.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.349+0000 7fa396396740 1 Processor -- start 2024-08-30T09:13:39.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.350+0000 7fa37effd640 1 -- 172.21.15.42:0/2596835641 <== mon.2 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7fa380004020 con 0x7fa3881140d0 2024-08-30T09:13:39.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.350+0000 7fa396396740 1 -- start start 2024-08-30T09:13:39.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.350+0000 7fa396396740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d8cd547350 0x55d8cd8ed780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:39.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.350+0000 7fa396396740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d8cd51e490 0x55d8cd8edcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:39.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.350+0000 7fa396396740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d8cd526670 0x55d8cd8ee200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:39.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.350+0000 7fa396396740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55d8cd8635f0 con 0x55d8cd51e490 2024-08-30T09:13:39.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.350+0000 7fa396396740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55d8cd852b80 con 0x55d8cd547350 2024-08-30T09:13:39.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.350+0000 7fa396396740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55d8cd856b30 con 0x55d8cd526670 2024-08-30T09:13:39.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.350+0000 7fa38dbbd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d8cd51e490 0x55d8cd8edcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:13:39.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.350+0000 7fa37effd640 1 --2- 172.21.15.42:0/2596835641 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fa3740b4cc0 0x7fa3740b71d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:39.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.350+0000 7fa38dbbd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d8cd51e490 0x55d8cd8edcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:39120/0 (socket says 172.21.15.42:39120) 2024-08-30T09:13:39.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.350+0000 7fa38dbbd640 1 -- 172.21.15.42:0/2410107241 learned_addr learned my addr 172.21.15.42:0/2410107241 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:13:39.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.351+0000 7fa38d3bc640 1 --2- 172.21.15.42:0/2410107241 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d8cd547350 0x55d8cd8ed780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:13:39.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.351+0000 7fa37effd640 1 -- 172.21.15.42:0/2596835641 <== mon.2 v2:172.21.15.167:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7fa3800e5800 con 0x7fa3881140d0 2024-08-30T09:13:39.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.351+0000 7fa38d3bc640 1 --2- 172.21.15.42:0/2410107241 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d8cd526670 0x55d8cd8ee200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:13:39.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.351+0000 7fa38e3be640 1 --2- 172.21.15.42:0/2596835641 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fa3740b4cc0 0x7fa3740b71d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:13:39.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.351+0000 7fa38dbbd640 1 -- 172.21.15.42:0/2410107241 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d8cd526670 msgr2=0x55d8cd8ee200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:13:39.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.351+0000 7fa38dbbd640 1 --2- 172.21.15.42:0/2410107241 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d8cd526670 0x55d8cd8ee200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:39.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.351+0000 7fa38dbbd640 1 -- 172.21.15.42:0/2410107241 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d8cd547350 msgr2=0x55d8cd8ed780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:13:39.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.351+0000 7fa38dbbd640 1 --2- 172.21.15.42:0/2410107241 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d8cd547350 0x55d8cd8ed780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:39.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.351+0000 7fa38dbbd640 1 -- 172.21.15.42:0/2410107241 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d8cd5b2930 con 0x55d8cd51e490 2024-08-30T09:13:39.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.351+0000 7fa38dbbd640 1 --2- 172.21.15.42:0/2410107241 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d8cd51e490 0x55d8cd8edcc0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fa38400bcd0 tx=0x7fa3840107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e685012ab2245ee server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:13:39.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.351+0000 7fa38e3be640 1 --2- 172.21.15.42:0/2596835641 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fa3740b4cc0 0x7fa3740b71d0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fa3800e2fc0 tx=0x7fa380009f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:13:39.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.352+0000 7fa373fff640 1 -- 172.21.15.42:0/2410107241 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa384005020 con 0x55d8cd51e490 2024-08-30T09:13:39.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.352+0000 7fa373fff640 1 -- 172.21.15.42:0/2410107241 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa3840047a0 con 0x55d8cd51e490 2024-08-30T09:13:39.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.352+0000 7fa373fff640 1 -- 172.21.15.42:0/2410107241 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa384011640 con 0x55d8cd51e490 2024-08-30T09:13:39.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.353+0000 7fa396396740 1 -- 172.21.15.42:0/2410107241 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d8cd51e490 msgr2=0x55d8cd8edcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:13:39.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.353+0000 7fa396396740 1 --2- 172.21.15.42:0/2410107241 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d8cd51e490 0x55d8cd8edcc0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fa38400bcd0 tx=0x7fa3840107a0 comp rx=0 tx=0).stop 2024-08-30T09:13:39.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.353+0000 7fa396396740 1 -- 172.21.15.42:0/2410107241 shutdown_connections 2024-08-30T09:13:39.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.353+0000 7fa396396740 1 --2- 172.21.15.42:0/2410107241 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d8cd526670 0x55d8cd8ee200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:39.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.353+0000 7fa396396740 1 --2- 172.21.15.42:0/2410107241 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d8cd51e490 0x55d8cd8edcc0 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:39.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.353+0000 7fa396396740 1 --2- 172.21.15.42:0/2410107241 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d8cd547350 0x55d8cd8ed780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:39.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.353+0000 7fa396396740 1 -- 172.21.15.42:0/2410107241 >> 172.21.15.42:0/2410107241 conn(0x55d8cd53b740 msgr2=0x55d8cd8eb670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:13:39.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.353+0000 7fa396396740 1 -- 172.21.15.42:0/2410107241 shutdown_connections 2024-08-30T09:13:39.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.353+0000 7fa396396740 1 -- 172.21.15.42:0/2410107241 wait complete. 2024-08-30T09:13:39.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.353+0000 7fa396396740 1 -- start start 2024-08-30T09:13:39.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.354+0000 7fa396396740 1 Processor -- start 2024-08-30T09:13:39.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.354+0000 7fa396396740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d8cd51e490 0x55d8cd8f8f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:39.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.354+0000 7fa396396740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d8cd526670 0x55d8cd8f9490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:39.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.354+0000 7fa396396740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d8cd547350 0x55d8cd905280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:39.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.354+0000 7fa38d3bc640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d8cd51e490 0x55d8cd8f8f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:13:39.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.354+0000 7fa38d3bc640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d8cd51e490 0x55d8cd8f8f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:39130/0 (socket says 172.21.15.42:39130) 2024-08-30T09:13:39.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.355+0000 7fa396396740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55d8cd6b5890 con 0x55d8cd51e490 2024-08-30T09:13:39.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.355+0000 7fa396396740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55d8cd7d9790 con 0x55d8cd526670 2024-08-30T09:13:39.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.355+0000 7fa396396740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55d8cd8fbf60 con 0x55d8cd547350 2024-08-30T09:13:39.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.355+0000 7fa396396740 10 client.0 _ll_register_callbacks cb 0x55d8cd55eba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:13:39.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.355+0000 7fa38d3bc640 1 -- 172.21.15.42:0/1823593617 learned_addr learned my addr 172.21.15.42:0/1823593617 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:13:39.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.355+0000 7fa38dbbd640 1 --2- 172.21.15.42:0/1823593617 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d8cd526670 0x55d8cd8f9490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:13:39.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.355+0000 7fa38d3bc640 1 --2- 172.21.15.42:0/1823593617 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d8cd547350 0x55d8cd905280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:13:39.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.355+0000 7fa38d3bc640 1 -- 172.21.15.42:0/1823593617 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d8cd547350 msgr2=0x55d8cd905280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:13:39.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.355+0000 7fa38d3bc640 1 --2- 172.21.15.42:0/1823593617 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d8cd547350 0x55d8cd905280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:39.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.355+0000 7fa38d3bc640 1 -- 172.21.15.42:0/1823593617 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d8cd526670 msgr2=0x55d8cd8f9490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:13:39.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.355+0000 7fa38d3bc640 1 --2- 172.21.15.42:0/1823593617 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d8cd526670 0x55d8cd8f9490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:39.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.355+0000 7fa38d3bc640 1 -- 172.21.15.42:0/1823593617 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d8cd810f30 con 0x55d8cd51e490 2024-08-30T09:13:39.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.355+0000 7fa38d3bc640 1 --2- 172.21.15.42:0/1823593617 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d8cd51e490 0x55d8cd8f8f50 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fa378009230 tx=0x7fa37802a440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33479a63f6b484ce server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:13:39.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.355+0000 7fa3717fa640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-08-30T09:13:39.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.356+0000 7fa3717fa640 1 -- 172.21.15.42:0/1823593617 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa378005410 con 0x55d8cd51e490 2024-08-30T09:13:39.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.356+0000 7fa3717fa640 1 -- 172.21.15.42:0/1823593617 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7fa3780055b0 con 0x55d8cd51e490 2024-08-30T09:13:39.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.356+0000 7fa396396740 1 -- 172.21.15.42:0/1823593617 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa3840047a0 con 0x55d8cd51e490 2024-08-30T09:13:39.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.356+0000 7fa3717fa640 1 -- 172.21.15.42:0/1823593617 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fa378054050 con 0x55d8cd51e490 2024-08-30T09:13:39.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.356+0000 7fa396396740 10 client.17568 fetch_fsmap learned FSMap version 45 2024-08-30T09:13:39.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.356+0000 7fa396396740 1 -- 172.21.15.42:0/1823593617 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55d8cd907d60 con 0x55d8cd51e490 2024-08-30T09:13:39.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.356+0000 7fa3717fa640 1 -- 172.21.15.42:0/1823593617 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa3840047a0 con 0x55d8cd51e490 2024-08-30T09:13:39.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.356+0000 7fa3717fa640 1 -- 172.21.15.42:0/1823593617 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 45) ==== 8362+0+0 (secure 0 0 0) 0x7fa378040720 con 0x55d8cd51e490 2024-08-30T09:13:39.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.356+0000 7fa396396740 10 client.17568 fetch_fsmap finished waiting for FSMap version 45 2024-08-30T09:13:39.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.356+0000 7fa396396740 10 client.17568 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-30T09:13:39.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.356+0000 7fa396396740 20 client.17568 populate_metadata read hostname 'smithi042' 2024-08-30T09:13:39.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.356+0000 7fa396396740 1 --2- 172.21.15.42:0/1823593617 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55d8cd505cd0 0x55d8cd90b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:39.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.356+0000 7fa396396740 4 client.17568 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-30T09:13:39.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.356+0000 7fa396396740 1 -- 172.21.15.42:0/1823593617 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d8cd865540 con 0x55d8cd505cd0 2024-08-30T09:13:39.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.357+0000 7fa38dbbd640 1 --2- 172.21.15.42:0/1823593617 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55d8cd505cd0 0x55d8cd90b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:13:39.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.357+0000 7fa38dbbd640 1 --2- 172.21.15.42:0/1823593617 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55d8cd505cd0 0x55d8cd90b280 crc :-1 s=READY pgs=265 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4292b279cfbd0e81 server_cookie=11e16c0c0573b376 in_seq=0 out_seq=0 2024-08-30T09:13:39.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.357+0000 7fa3717fa640 10 client.17568 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:13:39.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.358+0000 7fa3717fa640 1 -- 172.21.15.42:0/1823593617 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d8cd865540 con 0x55d8cd505cd0 2024-08-30T09:13:39.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.358+0000 7fa3717fa640 10 client.17568 handle_command_reply: tid=0 2024-08-30T09:13:39.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.358+0000 7fa396396740 1 client.17568 shutdown 2024-08-30T09:13:39.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.358+0000 7fa396396740 1 -- 172.21.15.42:0/1823593617 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d8cd51e490 msgr2=0x55d8cd8f8f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:13:39.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.358+0000 7fa396396740 1 --2- 172.21.15.42:0/1823593617 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d8cd51e490 0x55d8cd8f8f50 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fa378009230 tx=0x7fa37802a440 comp rx=0 tx=0).stop 2024-08-30T09:13:39.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.359+0000 7fa396396740 1 -- 172.21.15.42:0/1823593617 shutdown_connections 2024-08-30T09:13:39.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.359+0000 7fa396396740 1 --2- 172.21.15.42:0/1823593617 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55d8cd505cd0 0x55d8cd90b280 crc :-1 s=READY pgs=265 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:39.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.359+0000 7fa396396740 1 --2- 172.21.15.42:0/1823593617 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d8cd547350 0x55d8cd905280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:39.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.359+0000 7fa396396740 1 --2- 172.21.15.42:0/1823593617 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d8cd526670 0x55d8cd8f9490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:39.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.359+0000 7fa396396740 1 --2- 172.21.15.42:0/1823593617 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d8cd51e490 0x55d8cd8f8f50 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:39.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.359+0000 7fa396396740 1 -- 172.21.15.42:0/1823593617 >> 172.21.15.42:0/1823593617 conn(0x55d8cd53b740 msgr2=0x55d8cd8f0910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:13:39.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.359+0000 7fa3717fa640 0 client.17568 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:13:39.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.359+0000 7fa396396740 1 -- 172.21.15.42:0/1823593617 shutdown_connections 2024-08-30T09:13:39.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.359+0000 7fa396396740 1 -- 172.21.15.42:0/1823593617 wait complete. 2024-08-30T09:13:39.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.359+0000 7fa396396740 20 client.17568 trim_cache size 0 max 16384 2024-08-30T09:13:39.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.381+0000 7fa396396740 1 Processor -- start 2024-08-30T09:13:39.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.381+0000 7fa396396740 1 -- start start 2024-08-30T09:13:39.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.381+0000 7fa396396740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d8cd51e490 0x55d8cd8ef170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:39.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.381+0000 7fa396396740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d8cd526670 0x55d8cd8f6140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:39.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.381+0000 7fa396396740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d8cd547350 0x55d8cd8f6680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:39.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.381+0000 7fa396396740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fa3881db6d0 con 0x55d8cd51e490 2024-08-30T09:13:39.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.381+0000 7fa396396740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa3881dbe00 con 0x55d8cd547350 2024-08-30T09:13:39.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.381+0000 7fa396396740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fa3881b7660 con 0x55d8cd526670 2024-08-30T09:13:39.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.381+0000 7fa38d3bc640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d8cd51e490 0x55d8cd8ef170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:13:39.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.381+0000 7fa38d3bc640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d8cd51e490 0x55d8cd8ef170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:39146/0 (socket says 172.21.15.42:39146) 2024-08-30T09:13:39.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.381+0000 7fa38d3bc640 1 -- 172.21.15.42:0/31955381 learned_addr learned my addr 172.21.15.42:0/31955381 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:13:39.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.381+0000 7fa38dbbd640 1 --2- 172.21.15.42:0/31955381 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d8cd526670 0x55d8cd8f6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:13:39.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.381+0000 7fa38d3bc640 1 --2- 172.21.15.42:0/31955381 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d8cd547350 0x55d8cd8f6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:13:39.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.382+0000 7fa38d3bc640 1 -- 172.21.15.42:0/31955381 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d8cd526670 msgr2=0x55d8cd8f6140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:13:39.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.382+0000 7fa38d3bc640 1 --2- 172.21.15.42:0/31955381 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d8cd526670 0x55d8cd8f6140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:39.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.382+0000 7fa38d3bc640 1 -- 172.21.15.42:0/31955381 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d8cd547350 msgr2=0x55d8cd8f6680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:13:39.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.382+0000 7fa38d3bc640 1 --2- 172.21.15.42:0/31955381 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d8cd547350 0x55d8cd8f6680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:39.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.382+0000 7fa38d3bc640 1 -- 172.21.15.42:0/31955381 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d8cd8fa210 con 0x55d8cd51e490 2024-08-30T09:13:39.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.382+0000 7fa38d3bc640 1 --2- 172.21.15.42:0/31955381 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d8cd51e490 0x55d8cd8ef170 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fa3780043d0 tx=0x7fa378004cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63881a806ae8d756 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:13:39.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.382+0000 7fa3737fe640 1 -- 172.21.15.42:0/31955381 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa378005940 con 0x55d8cd51e490 2024-08-30T09:13:39.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.382+0000 7fa3737fe640 1 -- 172.21.15.42:0/31955381 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa37802ae40 con 0x55d8cd51e490 2024-08-30T09:13:39.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.382+0000 7fa3737fe640 1 -- 172.21.15.42:0/31955381 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa378003a60 con 0x55d8cd51e490 2024-08-30T09:13:39.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.383+0000 7fa396396740 1 -- 172.21.15.42:0/31955381 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d8cd51e490 msgr2=0x55d8cd8ef170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:13:39.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.383+0000 7fa396396740 1 --2- 172.21.15.42:0/31955381 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d8cd51e490 0x55d8cd8ef170 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fa3780043d0 tx=0x7fa378004cb0 comp rx=0 tx=0).stop 2024-08-30T09:13:39.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.383+0000 7fa396396740 1 -- 172.21.15.42:0/31955381 shutdown_connections 2024-08-30T09:13:39.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.383+0000 7fa396396740 1 --2- 172.21.15.42:0/31955381 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d8cd526670 0x55d8cd8f6140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:39.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.383+0000 7fa396396740 1 --2- 172.21.15.42:0/31955381 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d8cd547350 0x55d8cd8f6680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:39.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.383+0000 7fa396396740 1 --2- 172.21.15.42:0/31955381 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d8cd51e490 0x55d8cd8ef170 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:39.792 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.383+0000 7fa396396740 1 -- 172.21.15.42:0/31955381 >> 172.21.15.42:0/31955381 conn(0x55d8cd53b740 msgr2=0x55d8cd8f0ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:13:39.792 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.383+0000 7fa396396740 1 -- 172.21.15.42:0/31955381 shutdown_connections 2024-08-30T09:13:39.792 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.383+0000 7fa396396740 1 -- 172.21.15.42:0/31955381 wait complete. 2024-08-30T09:13:39.792 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.384+0000 7fa396396740 1 -- start start 2024-08-30T09:13:39.792 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.384+0000 7fa396396740 1 Processor -- start 2024-08-30T09:13:39.792 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.384+0000 7fa396396740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d8cd51e490 0x55d8cd8ff6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:39.792 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.384+0000 7fa396396740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d8cd526670 0x55d8cd8ffc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:39.792 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.384+0000 7fa396396740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d8cd547350 0x55d8cd974ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:39.792 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.384+0000 7fa396396740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fa3881b5b60 con 0x55d8cd547350 2024-08-30T09:13:39.792 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.384+0000 7fa396396740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa3881b6f90 con 0x55d8cd526670 2024-08-30T09:13:39.792 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.384+0000 7fa396396740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fa3881b61f0 con 0x55d8cd51e490 2024-08-30T09:13:39.792 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.384+0000 7fa38d3bc640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d8cd547350 0x55d8cd974ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:13:39.792 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.384+0000 7fa396396740 10 client.0 _ll_register_callbacks cb 0x7fa378004490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:13:39.792 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.384+0000 7fa38d3bc640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d8cd547350 0x55d8cd974ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:39162/0 (socket says 172.21.15.42:39162) 2024-08-30T09:13:39.793 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.384+0000 7fa38dbbd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d8cd526670 0x55d8cd8ffc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:13:39.793 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.384+0000 7fa38d3bc640 1 -- 172.21.15.42:0/286690904 learned_addr learned my addr 172.21.15.42:0/286690904 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:13:39.793 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.384+0000 7fa38d3bc640 1 --2- 172.21.15.42:0/286690904 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d8cd51e490 0x55d8cd8ff6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:13:39.793 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.385+0000 7fa38d3bc640 1 -- 172.21.15.42:0/286690904 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d8cd51e490 msgr2=0x55d8cd8ff6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:13:39.793 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.385+0000 7fa38d3bc640 1 --2- 172.21.15.42:0/286690904 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d8cd51e490 0x55d8cd8ff6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:39.793 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.385+0000 7fa38d3bc640 1 -- 172.21.15.42:0/286690904 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d8cd526670 msgr2=0x55d8cd8ffc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:13:39.793 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.385+0000 7fa38d3bc640 1 --2- 172.21.15.42:0/286690904 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d8cd526670 0x55d8cd8ffc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:39.793 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.385+0000 7fa38d3bc640 1 -- 172.21.15.42:0/286690904 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d8cd9751e0 con 0x55d8cd547350 2024-08-30T09:13:39.793 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.385+0000 7fa38dbbd640 1 --2- 172.21.15.42:0/286690904 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d8cd526670 0x55d8cd8ffc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-30T09:13:39.793 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.385+0000 7fa38d3bc640 1 --2- 172.21.15.42:0/286690904 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d8cd547350 0x55d8cd974ca0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fa378004ba0 tx=0x7fa37805c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8df8f0956d1468e3 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:13:39.793 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.385+0000 7fa3717fa640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-08-30T09:13:39.793 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.385+0000 7fa3717fa640 1 -- 172.21.15.42:0/286690904 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa378060040 con 0x55d8cd547350 2024-08-30T09:13:39.793 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.385+0000 7fa3717fa640 1 -- 172.21.15.42:0/286690904 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7fa378040450 con 0x55d8cd547350 2024-08-30T09:13:39.793 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.385+0000 7fa396396740 1 -- 172.21.15.42:0/286690904 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa37802ae40 con 0x55d8cd547350 2024-08-30T09:13:39.794 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.385+0000 7fa3717fa640 1 -- 172.21.15.42:0/286690904 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fa378040650 con 0x55d8cd547350 2024-08-30T09:13:39.794 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.385+0000 7fa396396740 10 client.17574 fetch_fsmap learned FSMap version 45 2024-08-30T09:13:39.794 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.385+0000 7fa396396740 1 -- 172.21.15.42:0/286690904 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55d8cd976380 con 0x55d8cd547350 2024-08-30T09:13:39.794 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.386+0000 7fa3717fa640 1 -- 172.21.15.42:0/286690904 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa37802ae40 con 0x55d8cd547350 2024-08-30T09:13:39.794 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.386+0000 7fa3717fa640 1 -- 172.21.15.42:0/286690904 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 45) ==== 8362+0+0 (secure 0 0 0) 0x7fa378066070 con 0x55d8cd547350 2024-08-30T09:13:39.794 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.386+0000 7fa396396740 10 client.17574 fetch_fsmap finished waiting for FSMap version 45 2024-08-30T09:13:39.794 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.386+0000 7fa396396740 10 client.17574 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-30T09:13:39.794 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.386+0000 7fa396396740 20 client.17574 populate_metadata read hostname 'smithi042' 2024-08-30T09:13:39.794 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.386+0000 7fa396396740 1 --2- 172.21.15.42:0/286690904 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55d8cd4db080 0x55d8cd978a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:39.794 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.386+0000 7fa396396740 4 client.17574 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-08-30T09:13:39.794 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.386+0000 7fa396396740 1 -- 172.21.15.42:0/286690904 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55d8cd860fe0 con 0x55d8cd4db080 2024-08-30T09:13:39.794 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.386+0000 7fa38dbbd640 1 --2- 172.21.15.42:0/286690904 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55d8cd4db080 0x55d8cd978a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:13:39.794 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.387+0000 7fa3717fa640 10 client.17574 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:13:39.794 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.387+0000 7fa38dbbd640 1 --2- 172.21.15.42:0/286690904 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55d8cd4db080 0x55d8cd978a20 crc :-1 s=READY pgs=266 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=309cedcfdb0ee34a server_cookie=171f360ea7afa6b5 in_seq=0 out_seq=0 2024-08-30T09:13:39.794 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.388+0000 7fa3717fa640 1 -- 172.21.15.42:0/286690904 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55d8cd860fe0 con 0x55d8cd4db080 2024-08-30T09:13:39.794 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.388+0000 7fa3717fa640 10 client.17574 handle_command_reply: tid=0 2024-08-30T09:13:39.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.388+0000 7fa396396740 1 client.17574 shutdown 2024-08-30T09:13:39.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.388+0000 7fa396396740 1 -- 172.21.15.42:0/286690904 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d8cd547350 msgr2=0x55d8cd974ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:13:39.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.388+0000 7fa396396740 1 --2- 172.21.15.42:0/286690904 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d8cd547350 0x55d8cd974ca0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fa378004ba0 tx=0x7fa37805c040 comp rx=0 tx=0).stop 2024-08-30T09:13:39.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.388+0000 7fa396396740 1 -- 172.21.15.42:0/286690904 shutdown_connections 2024-08-30T09:13:39.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.388+0000 7fa396396740 1 --2- 172.21.15.42:0/286690904 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55d8cd4db080 0x55d8cd978a20 crc :-1 s=READY pgs=266 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:39.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.388+0000 7fa396396740 1 --2- 172.21.15.42:0/286690904 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d8cd547350 0x55d8cd974ca0 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:39.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.388+0000 7fa396396740 1 --2- 172.21.15.42:0/286690904 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d8cd526670 0x55d8cd8ffc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:39.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.388+0000 7fa3717fa640 0 client.17574 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:13:39.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.388+0000 7fa396396740 1 --2- 172.21.15.42:0/286690904 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d8cd51e490 0x55d8cd8ff6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:39.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.388+0000 7fa396396740 1 -- 172.21.15.42:0/286690904 >> 172.21.15.42:0/286690904 conn(0x55d8cd53b740 msgr2=0x55d8cd8fa3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:13:39.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.388+0000 7fa396396740 1 -- 172.21.15.42:0/286690904 shutdown_connections 2024-08-30T09:13:39.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.389+0000 7fa396396740 1 -- 172.21.15.42:0/286690904 wait complete. 2024-08-30T09:13:39.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.389+0000 7fa396396740 20 client.17574 trim_cache size 0 max 16384 2024-08-30T09:13:39.796 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-08-30T09:13:39.796 INFO:teuthology.orchestra.run.smithi042.stdout: "return_code": 0, 2024-08-30T09:13:39.796 INFO:teuthology.orchestra.run.smithi042.stdout: "scrub_tag": "03521407-a747-49b9-a415-6233551c26fb", 2024-08-30T09:13:39.796 INFO:teuthology.orchestra.run.smithi042.stdout: "mode": "asynchronous" 2024-08-30T09:13:39.796 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-08-30T09:13:39.797 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.395+0000 7fa38fe21640 1 -- 172.21.15.42:0/2596835641 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fa3740b4cc0 msgr2=0x7fa3740b71d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:13:39.797 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.395+0000 7fa38fe21640 1 --2- 172.21.15.42:0/2596835641 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fa3740b4cc0 0x7fa3740b71d0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fa3800e2fc0 tx=0x7fa380009f90 comp rx=0 tx=0).stop 2024-08-30T09:13:39.797 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.395+0000 7fa38fe21640 1 -- 172.21.15.42:0/2596835641 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa3881140d0 msgr2=0x7fa388118ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:13:39.797 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.395+0000 7fa38fe21640 1 --2- 172.21.15.42:0/2596835641 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa3881140d0 0x7fa388118ff0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fa38000efd0 tx=0x7fa38000c6a0 comp rx=0 tx=0).stop 2024-08-30T09:13:39.797 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.396+0000 7fa38fe21640 1 -- 172.21.15.42:0/2596835641 shutdown_connections 2024-08-30T09:13:39.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.396+0000 7fa38fe21640 1 --2- 172.21.15.42:0/2596835641 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fa3740b4cc0 0x7fa3740b71d0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:39.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.396+0000 7fa38fe21640 1 --2- 172.21.15.42:0/2596835641 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa3881140d0 0x7fa388118ff0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:39.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.396+0000 7fa38fe21640 1 --2- 172.21.15.42:0/2596835641 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa38811c9d0 0x7fa388113b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:39.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.396+0000 7fa38fe21640 1 --2- 172.21.15.42:0/2596835641 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa388111ce0 0x7fa388113650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:39.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.396+0000 7fa38fe21640 1 -- 172.21.15.42:0/2596835641 >> 172.21.15.42:0/2596835641 conn(0x7fa388070c60 msgr2=0x7fa388071bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:13:39.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.399+0000 7fa38fe21640 1 -- 172.21.15.42:0/2596835641 shutdown_connections 2024-08-30T09:13:39.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:39.400+0000 7fa38fe21640 1 -- 172.21.15.42:0/2596835641 wait complete. 2024-08-30T09:13:40.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:13:40 smithi042 ceph-mon[18652]: scrub queued for path: / 2024-08-30T09:13:40.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:13:40 smithi042 ceph-mon[18652]: scrub summary: idle+waiting paths [/] 2024-08-30T09:13:40.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:13:40 smithi042 ceph-mon[18652]: scrub summary: active paths [/] 2024-08-30T09:13:40.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:13:40 smithi042 ceph-mon[18652]: scrub summary: active paths [/] 2024-08-30T09:13:40.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:13:40 smithi042 ceph-mon[18652]: scrub summary: idle+waiting paths [/] 2024-08-30T09:13:40.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:13:40 smithi042 ceph-mon[18652]: scrub summary: active paths [/] 2024-08-30T09:13:40.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:13:40 smithi167 ceph-mon[22063]: scrub queued for path: / 2024-08-30T09:13:40.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:13:40 smithi167 ceph-mon[22063]: scrub summary: idle+waiting paths [/] 2024-08-30T09:13:40.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:13:40 smithi167 ceph-mon[22063]: scrub summary: active paths [/] 2024-08-30T09:13:40.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:13:40 smithi167 ceph-mon[22063]: scrub summary: active paths [/] 2024-08-30T09:13:40.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:13:40 smithi167 ceph-mon[22063]: scrub summary: idle+waiting paths [/] 2024-08-30T09:13:40.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:13:40 smithi167 ceph-mon[22063]: scrub summary: active paths [/] 2024-08-30T09:13:40.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:13:40 smithi190 ceph-mon[22001]: scrub queued for path: / 2024-08-30T09:13:40.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:13:40 smithi190 ceph-mon[22001]: scrub summary: idle+waiting paths [/] 2024-08-30T09:13:40.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:13:40 smithi190 ceph-mon[22001]: scrub summary: active paths [/] 2024-08-30T09:13:40.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:13:40 smithi190 ceph-mon[22001]: scrub summary: active paths [/] 2024-08-30T09:13:40.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:13:40 smithi190 ceph-mon[22001]: scrub summary: idle+waiting paths [/] 2024-08-30T09:13:40.437 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:13:40 smithi190 ceph-mon[22001]: scrub summary: active paths [/] 2024-08-30T09:13:40.953 INFO:teuthology.orchestra.run.smithi042.stderr:progress: 775.0 s, 438.2 tps, lat 224.926 ms stddev 267.528 2024-08-30T09:13:41.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:13:41 smithi167 ceph-mon[22063]: pgmap v760: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 5.0 MiB/s wr, 497 op/s 2024-08-30T09:13:41.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:13:41 smithi167 ceph-mon[22063]: scrub summary: active paths [/] 2024-08-30T09:13:41.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:13:41 smithi167 ceph-mon[22063]: scrub summary: idle+waiting paths [/] 2024-08-30T09:13:41.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:13:41 smithi167 ceph-mon[22063]: scrub summary: idle+waiting paths [/] 2024-08-30T09:13:41.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:13:41 smithi167 ceph-mon[22063]: scrub summary: idle+waiting paths [/] 2024-08-30T09:13:41.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:13:41 smithi042 ceph-mon[18652]: pgmap v760: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 5.0 MiB/s wr, 497 op/s 2024-08-30T09:13:41.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:13:41 smithi042 ceph-mon[18652]: scrub summary: active paths [/] 2024-08-30T09:13:41.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:13:41 smithi042 ceph-mon[18652]: scrub summary: idle+waiting paths [/] 2024-08-30T09:13:41.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:13:41 smithi042 ceph-mon[18652]: scrub summary: idle+waiting paths [/] 2024-08-30T09:13:41.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:13:41 smithi042 ceph-mon[18652]: scrub summary: idle+waiting paths [/] 2024-08-30T09:13:41.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:13:41 smithi190 ceph-mon[22001]: pgmap v760: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 5.0 MiB/s wr, 497 op/s 2024-08-30T09:13:41.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:13:41 smithi190 ceph-mon[22001]: scrub summary: active paths [/] 2024-08-30T09:13:41.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:13:41 smithi190 ceph-mon[22001]: scrub summary: idle+waiting paths [/] 2024-08-30T09:13:41.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:13:41 smithi190 ceph-mon[22001]: scrub summary: idle+waiting paths [/] 2024-08-30T09:13:41.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:13:41 smithi190 ceph-mon[22001]: scrub summary: idle+waiting paths [/] 2024-08-30T09:13:41.510 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-30T09:13:42.036 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T09:13:42.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.639+0000 7f6003373640 1 -- 172.21.15.42:0/2816207260 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5ffc10a3e0 msgr2=0x7f5ffc10a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:13:42.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.639+0000 7f6003373640 1 --2- 172.21.15.42:0/2816207260 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5ffc10a3e0 0x7f5ffc10a7f0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f5ff800fb10 tx=0x7f5ff802e5b0 comp rx=0 tx=0).stop 2024-08-30T09:13:42.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.639+0000 7f6003373640 1 -- 172.21.15.42:0/2816207260 shutdown_connections 2024-08-30T09:13:42.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.639+0000 7f6003373640 1 --2- 172.21.15.42:0/2816207260 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5ffc1162b0 0x7f5ffc118730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:42.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.639+0000 7f6003373640 1 --2- 172.21.15.42:0/2816207260 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5ffc10ad30 0x7f5ffc115d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:42.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.639+0000 7f6003373640 1 --2- 172.21.15.42:0/2816207260 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5ffc10a3e0 0x7f5ffc10a7f0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:42.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.639+0000 7f6003373640 1 -- 172.21.15.42:0/2816207260 >> 172.21.15.42:0/2816207260 conn(0x7f5ffc06e790 msgr2=0x7f5ffc06ebd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:13:42.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.639+0000 7f6003373640 1 -- 172.21.15.42:0/2816207260 shutdown_connections 2024-08-30T09:13:42.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.639+0000 7f6003373640 1 -- 172.21.15.42:0/2816207260 wait complete. 2024-08-30T09:13:42.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.640+0000 7f6003373640 1 Processor -- start 2024-08-30T09:13:42.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.640+0000 7f6003373640 1 -- start start 2024-08-30T09:13:42.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.640+0000 7f6003373640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5ffc10a3e0 0x7f5ffc121460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:42.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.640+0000 7f6003373640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5ffc10ad30 0x7f5ffc11a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:42.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.641+0000 7f6002371640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5ffc10a3e0 0x7f5ffc121460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:13:42.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.641+0000 7f6002371640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5ffc10a3e0 0x7f5ffc121460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:57190/0 (socket says 172.21.15.42:57190) 2024-08-30T09:13:42.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.641+0000 7f6003373640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5ffc1162b0 0x7f5ffc11ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:42.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.641+0000 7f6001b70640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5ffc10ad30 0x7f5ffc11a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:13:42.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.641+0000 7f6001b70640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5ffc10ad30 0x7f5ffc11a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:36472/0 (socket says 172.21.15.42:36472) 2024-08-30T09:13:42.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.641+0000 7f6002371640 1 -- 172.21.15.42:0/3182178206 learned_addr learned my addr 172.21.15.42:0/3182178206 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:13:42.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.641+0000 7f6003373640 1 -- 172.21.15.42:0/3182178206 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f5ffc122500 con 0x7f5ffc1162b0 2024-08-30T09:13:42.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.641+0000 7f6003373640 1 -- 172.21.15.42:0/3182178206 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f5ffc122380 con 0x7f5ffc10a3e0 2024-08-30T09:13:42.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.641+0000 7f6003373640 1 -- 172.21.15.42:0/3182178206 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f5ffc122680 con 0x7f5ffc10ad30 2024-08-30T09:13:42.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.641+0000 7f6002b72640 1 --2- 172.21.15.42:0/3182178206 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5ffc1162b0 0x7f5ffc11ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:13:42.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.646+0000 7f6002371640 1 -- 172.21.15.42:0/3182178206 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5ffc10ad30 msgr2=0x7f5ffc11a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:13:42.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.646+0000 7f6002371640 1 --2- 172.21.15.42:0/3182178206 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5ffc10ad30 0x7f5ffc11a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:42.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.646+0000 7f6002371640 1 -- 172.21.15.42:0/3182178206 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5ffc1162b0 msgr2=0x7f5ffc11ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:13:42.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.646+0000 7f6002371640 1 --2- 172.21.15.42:0/3182178206 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5ffc1162b0 0x7f5ffc11ac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:42.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.646+0000 7f6002371640 1 -- 172.21.15.42:0/3182178206 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ffc11b2b0 con 0x7f5ffc10a3e0 2024-08-30T09:13:42.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.647+0000 7f6002371640 1 --2- 172.21.15.42:0/3182178206 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5ffc10a3e0 0x7f5ffc121460 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f5ff802eba0 tx=0x7f5ff800e660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:13:42.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.649+0000 7f5ff37fe640 1 -- 172.21.15.42:0/3182178206 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ff800f460 con 0x7f5ffc10a3e0 2024-08-30T09:13:42.655 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.649+0000 7f5ff37fe640 1 -- 172.21.15.42:0/3182178206 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ff800e960 con 0x7f5ffc10a3e0 2024-08-30T09:13:42.655 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.649+0000 7f5ff37fe640 1 -- 172.21.15.42:0/3182178206 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ff8004fd0 con 0x7f5ffc10a3e0 2024-08-30T09:13:42.655 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.650+0000 7f6003373640 1 -- 172.21.15.42:0/3182178206 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ffc11b540 con 0x7f5ffc10a3e0 2024-08-30T09:13:42.655 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.650+0000 7f6003373640 1 -- 172.21.15.42:0/3182178206 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ffc120300 con 0x7f5ffc10a3e0 2024-08-30T09:13:42.661 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.655+0000 7f5ff37fe640 1 -- 172.21.15.42:0/3182178206 <== mon.2 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7f5ff8005170 con 0x7f5ffc10a3e0 2024-08-30T09:13:42.661 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.656+0000 7f5ff37fe640 1 --2- 172.21.15.42:0/3182178206 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f5fe80b4cc0 0x7f5fe80b71d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:42.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.657+0000 7f5ff37fe640 1 -- 172.21.15.42:0/3182178206 <== mon.2 v2:172.21.15.167:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f5ff81057f0 con 0x7f5ffc10a3e0 2024-08-30T09:13:42.666 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.659+0000 7f600ab4a740 1 Processor -- start 2024-08-30T09:13:42.667 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.660+0000 7f600ab4a740 1 -- start start 2024-08-30T09:13:42.667 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.660+0000 7f600ab4a740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cfb11d7070 0x55cfb157c460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:42.667 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.660+0000 7f600ab4a740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cfb11ae1b0 0x55cfb157c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:42.667 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.660+0000 7f600ab4a740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cfb11b6390 0x55cfb157cee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:42.667 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.660+0000 7f600ab4a740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55cfb14f3630 con 0x55cfb11ae1b0 2024-08-30T09:13:42.667 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.660+0000 7f600ab4a740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55cfb14e2bc0 con 0x55cfb11d7070 2024-08-30T09:13:42.667 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.660+0000 7f600ab4a740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55cfb14e6b70 con 0x55cfb11b6390 2024-08-30T09:13:42.667 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.660+0000 7f6002b72640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cfb11b6390 0x55cfb157cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:13:42.668 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.660+0000 7f6001b70640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cfb11ae1b0 0x55cfb157c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:13:42.668 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.660+0000 7f6002b72640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cfb11b6390 0x55cfb157cee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:36476/0 (socket says 172.21.15.42:36476) 2024-08-30T09:13:42.668 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.660+0000 7f6002b72640 1 -- 172.21.15.42:0/1862216250 learned_addr learned my addr 172.21.15.42:0/1862216250 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:13:42.668 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.660+0000 7f6002371640 1 --2- 172.21.15.42:0/1862216250 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cfb11d7070 0x55cfb157c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:13:42.668 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.661+0000 7f6002b72640 1 -- 172.21.15.42:0/1862216250 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cfb11d7070 msgr2=0x55cfb157c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:13:42.668 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.661+0000 7f6002b72640 1 --2- 172.21.15.42:0/1862216250 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cfb11d7070 0x55cfb157c460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:42.668 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.661+0000 7f6002b72640 1 -- 172.21.15.42:0/1862216250 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cfb11ae1b0 msgr2=0x55cfb157c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:13:42.668 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.661+0000 7f6002b72640 1 --2- 172.21.15.42:0/1862216250 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cfb11ae1b0 0x55cfb157c9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:42.668 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.661+0000 7f6002b72640 1 -- 172.21.15.42:0/1862216250 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cfb1242650 con 0x55cfb11b6390 2024-08-30T09:13:42.668 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.662+0000 7f6001b70640 1 --2- 172.21.15.42:0/3182178206 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f5fe80b4cc0 0x7f5fe80b71d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:13:42.668 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.663+0000 7f6002b72640 1 --2- 172.21.15.42:0/1862216250 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cfb11b6390 0x55cfb157cee0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f5ff4009fd0 tx=0x7f5ff40338e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3d1bca5462927be8 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:13:42.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.663+0000 7f5ff0ff9640 1 -- 172.21.15.42:0/1862216250 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ff4004d20 con 0x55cfb11b6390 2024-08-30T09:13:42.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.663+0000 7f5ff0ff9640 1 -- 172.21.15.42:0/1862216250 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ff4004050 con 0x55cfb11b6390 2024-08-30T09:13:42.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.663+0000 7f5ff0ff9640 1 -- 172.21.15.42:0/1862216250 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ff4005280 con 0x55cfb11b6390 2024-08-30T09:13:42.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.664+0000 7f600ab4a740 1 -- 172.21.15.42:0/1862216250 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cfb11b6390 msgr2=0x55cfb157cee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:13:42.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.664+0000 7f600ab4a740 1 --2- 172.21.15.42:0/1862216250 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cfb11b6390 0x55cfb157cee0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f5ff4009fd0 tx=0x7f5ff40338e0 comp rx=0 tx=0).stop 2024-08-30T09:13:42.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.664+0000 7f6001b70640 1 --2- 172.21.15.42:0/3182178206 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f5fe80b4cc0 0x7f5fe80b71d0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x55cfb157b0e0 tx=0x7f5fec0046e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:13:42.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.664+0000 7f600ab4a740 1 -- 172.21.15.42:0/1862216250 shutdown_connections 2024-08-30T09:13:42.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.664+0000 7f600ab4a740 1 --2- 172.21.15.42:0/1862216250 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cfb11b6390 0x55cfb157cee0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:42.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.664+0000 7f600ab4a740 1 --2- 172.21.15.42:0/1862216250 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cfb11ae1b0 0x55cfb157c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:42.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.664+0000 7f600ab4a740 1 --2- 172.21.15.42:0/1862216250 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cfb11d7070 0x55cfb157c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:42.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.664+0000 7f600ab4a740 1 -- 172.21.15.42:0/1862216250 >> 172.21.15.42:0/1862216250 conn(0x55cfb11cb460 msgr2=0x55cfb157a350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:13:42.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.665+0000 7f600ab4a740 1 -- 172.21.15.42:0/1862216250 shutdown_connections 2024-08-30T09:13:42.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.665+0000 7f600ab4a740 1 -- 172.21.15.42:0/1862216250 wait complete. 2024-08-30T09:13:42.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.665+0000 7f600ab4a740 1 -- start start 2024-08-30T09:13:42.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.666+0000 7f600ab4a740 1 Processor -- start 2024-08-30T09:13:42.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.666+0000 7f600ab4a740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cfb11ae1b0 0x55cfb1587ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:42.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.667+0000 7f600ab4a740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cfb11b6390 0x55cfb15880e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:42.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.667+0000 7f6002371640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cfb11ae1b0 0x55cfb1587ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:13:42.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.667+0000 7f6002371640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cfb11ae1b0 0x55cfb1587ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:57216/0 (socket says 172.21.15.42:57216) 2024-08-30T09:13:42.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.667+0000 7f6002371640 1 -- 172.21.15.42:0/2146629036 learned_addr learned my addr 172.21.15.42:0/2146629036 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:13:42.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.667+0000 7f600ab4a740 1 --2- 172.21.15.42:0/2146629036 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cfb11d7070 0x55cfb1593fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:42.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.667+0000 7f6001b70640 1 --2- 172.21.15.42:0/2146629036 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cfb11b6390 0x55cfb15880e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:13:42.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.667+0000 7f600ab4a740 1 -- 172.21.15.42:0/2146629036 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55cfb1345700 con 0x55cfb11b6390 2024-08-30T09:13:42.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.667+0000 7f6002b72640 1 --2- 172.21.15.42:0/2146629036 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cfb11d7070 0x55cfb1593fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:13:42.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.667+0000 7f600ab4a740 1 -- 172.21.15.42:0/2146629036 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55cfb1469700 con 0x55cfb11ae1b0 2024-08-30T09:13:42.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.667+0000 7f600ab4a740 1 -- 172.21.15.42:0/2146629036 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55cfb158ac90 con 0x55cfb11d7070 2024-08-30T09:13:42.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.668+0000 7f600ab4a740 10 client.0 _ll_register_callbacks cb 0x55cfb1498dc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:13:42.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.668+0000 7f6002371640 1 -- 172.21.15.42:0/2146629036 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cfb11d7070 msgr2=0x55cfb1593fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:13:42.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.668+0000 7f6002371640 1 --2- 172.21.15.42:0/2146629036 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cfb11d7070 0x55cfb1593fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:42.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.668+0000 7f6002371640 1 -- 172.21.15.42:0/2146629036 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cfb11b6390 msgr2=0x55cfb15880e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:13:42.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.668+0000 7f6002371640 1 --2- 172.21.15.42:0/2146629036 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cfb11b6390 0x55cfb15880e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:42.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.668+0000 7f6002371640 1 -- 172.21.15.42:0/2146629036 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cfb14a28b0 con 0x55cfb11ae1b0 2024-08-30T09:13:42.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.668+0000 7f6002371640 1 --2- 172.21.15.42:0/2146629036 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cfb11ae1b0 0x55cfb1587ba0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f5ff81032d0 tx=0x7f5ff8005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f24839f97e8f3947 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:13:42.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.669+0000 7f5fe5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-30T09:13:42.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.669+0000 7f5fe5ffb640 1 -- 172.21.15.42:0/2146629036 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ff80f2020 con 0x55cfb11ae1b0 2024-08-30T09:13:42.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.669+0000 7f5fe5ffb640 1 -- 172.21.15.42:0/2146629036 <== mon.2 v2:172.21.15.167:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f5ff80b71b0 con 0x55cfb11ae1b0 2024-08-30T09:13:42.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.669+0000 7f600ab4a740 1 -- 172.21.15.42:0/2146629036 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5ff4004050 con 0x55cfb11ae1b0 2024-08-30T09:13:42.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.669+0000 7f5fe5ffb640 1 -- 172.21.15.42:0/2146629036 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f5ff80cbf20 con 0x55cfb11ae1b0 2024-08-30T09:13:42.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.669+0000 7f600ab4a740 10 client.27434 fetch_fsmap learned FSMap version 45 2024-08-30T09:13:42.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.669+0000 7f600ab4a740 1 -- 172.21.15.42:0/2146629036 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55cfb1596a30 con 0x55cfb11ae1b0 2024-08-30T09:13:42.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.669+0000 7f5fe5ffb640 1 -- 172.21.15.42:0/2146629036 <== mon.2 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ff4004050 con 0x55cfb11ae1b0 2024-08-30T09:13:42.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.669+0000 7f5fe5ffb640 1 -- 172.21.15.42:0/2146629036 <== mon.2 v2:172.21.15.167:3300/0 5 ==== fsmap(e 45) ==== 8362+0+0 (secure 0 0 0) 0x7f5ff80c7890 con 0x55cfb11ae1b0 2024-08-30T09:13:42.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.669+0000 7f600ab4a740 10 client.27434 fetch_fsmap finished waiting for FSMap version 45 2024-08-30T09:13:42.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.669+0000 7f600ab4a740 10 client.27434 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-30T09:13:42.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.669+0000 7f600ab4a740 20 client.27434 populate_metadata read hostname 'smithi042' 2024-08-30T09:13:42.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.670+0000 7f600ab4a740 1 --2- 172.21.15.42:0/2146629036 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55cfb11959f0 0x55cfb1599f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:42.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.670+0000 7f600ab4a740 4 client.27434 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-30T09:13:42.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.670+0000 7f600ab4a740 1 -- 172.21.15.42:0/2146629036 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cfb14f5580 con 0x55cfb11959f0 2024-08-30T09:13:42.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.670+0000 7f6001b70640 1 --2- 172.21.15.42:0/2146629036 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55cfb11959f0 0x55cfb1599f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:13:42.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.670+0000 7f5fe5ffb640 10 client.27434 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:13:42.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.670+0000 7f6001b70640 1 --2- 172.21.15.42:0/2146629036 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55cfb11959f0 0x55cfb1599f00 crc :-1 s=READY pgs=267 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=865f627026f419c0 server_cookie=18545386af1e94a2 in_seq=0 out_seq=0 2024-08-30T09:13:42.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.671+0000 7f5fe5ffb640 1 -- 172.21.15.42:0/2146629036 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55cfb14f5580 con 0x55cfb11959f0 2024-08-30T09:13:42.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.671+0000 7f5fe5ffb640 10 client.27434 handle_command_reply: tid=0 2024-08-30T09:13:42.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.671+0000 7f600ab4a740 1 client.27434 shutdown 2024-08-30T09:13:42.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.671+0000 7f600ab4a740 1 -- 172.21.15.42:0/2146629036 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cfb11ae1b0 msgr2=0x55cfb1587ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:13:42.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.671+0000 7f600ab4a740 1 --2- 172.21.15.42:0/2146629036 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cfb11ae1b0 0x55cfb1587ba0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f5ff81032d0 tx=0x7f5ff8005fd0 comp rx=0 tx=0).stop 2024-08-30T09:13:42.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.671+0000 7f600ab4a740 1 -- 172.21.15.42:0/2146629036 shutdown_connections 2024-08-30T09:13:42.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.671+0000 7f600ab4a740 1 --2- 172.21.15.42:0/2146629036 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55cfb11959f0 0x55cfb1599f00 crc :-1 s=READY pgs=267 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:42.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.671+0000 7f600ab4a740 1 --2- 172.21.15.42:0/2146629036 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cfb11d7070 0x55cfb1593fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:42.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.671+0000 7f600ab4a740 1 --2- 172.21.15.42:0/2146629036 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cfb11b6390 0x55cfb15880e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:42.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.671+0000 7f5fe5ffb640 0 client.27434 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:13:42.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.671+0000 7f600ab4a740 1 --2- 172.21.15.42:0/2146629036 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cfb11ae1b0 0x55cfb1587ba0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:42.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.671+0000 7f600ab4a740 1 -- 172.21.15.42:0/2146629036 >> 172.21.15.42:0/2146629036 conn(0x55cfb11cb460 msgr2=0x55cfb157f5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:13:42.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.672+0000 7f600ab4a740 1 -- 172.21.15.42:0/2146629036 shutdown_connections 2024-08-30T09:13:42.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.672+0000 7f600ab4a740 1 -- 172.21.15.42:0/2146629036 wait complete. 2024-08-30T09:13:42.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.672+0000 7f600ab4a740 20 client.27434 trim_cache size 0 max 16384 2024-08-30T09:13:42.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.696+0000 7f600ab4a740 1 Processor -- start 2024-08-30T09:13:42.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.696+0000 7f600ab4a740 1 -- start start 2024-08-30T09:13:42.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.697+0000 7f600ab4a740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cfb11cb460 0x55cfb157a080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:42.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.697+0000 7f600ab4a740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cfb11959f0 0x55cfb1584d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:42.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.697+0000 7f6002371640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cfb11cb460 0x55cfb157a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:13:42.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.697+0000 7f6002371640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cfb11cb460 0x55cfb157a080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:39200/0 (socket says 172.21.15.42:39200) 2024-08-30T09:13:42.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.697+0000 7f600ab4a740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cfb11ae1b0 0x55cfb15852d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:42.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.698+0000 7f6001b70640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cfb11959f0 0x55cfb1584d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:13:42.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.698+0000 7f6002371640 1 -- 172.21.15.42:0/3864883504 learned_addr learned my addr 172.21.15.42:0/3864883504 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:13:42.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.698+0000 7f600ab4a740 1 -- 172.21.15.42:0/3864883504 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f5ffc080e70 con 0x55cfb11cb460 2024-08-30T09:13:42.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.698+0000 7f6002b72640 1 --2- 172.21.15.42:0/3864883504 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cfb11ae1b0 0x55cfb15852d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:13:42.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.698+0000 7f600ab4a740 1 -- 172.21.15.42:0/3864883504 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f5ffc07e6e0 con 0x55cfb11959f0 2024-08-30T09:13:42.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.698+0000 7f600ab4a740 1 -- 172.21.15.42:0/3864883504 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f5ffc11ba30 con 0x55cfb11ae1b0 2024-08-30T09:13:42.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.698+0000 7f6001b70640 1 -- 172.21.15.42:0/3864883504 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cfb11ae1b0 msgr2=0x55cfb15852d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:13:42.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.698+0000 7f6001b70640 1 --2- 172.21.15.42:0/3864883504 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cfb11ae1b0 0x55cfb15852d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:42.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.698+0000 7f6001b70640 1 -- 172.21.15.42:0/3864883504 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cfb11cb460 msgr2=0x55cfb157a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:13:42.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.698+0000 7f6001b70640 1 --2- 172.21.15.42:0/3864883504 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cfb11cb460 0x55cfb157a080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:42.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.698+0000 7f6001b70640 1 -- 172.21.15.42:0/3864883504 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cfb1586b00 con 0x55cfb11959f0 2024-08-30T09:13:42.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.698+0000 7f6001b70640 1 --2- 172.21.15.42:0/3864883504 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cfb11959f0 0x55cfb1584d90 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x55cfb159a600 tx=0x7f5fec017000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e6a3c55b79589daa server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:13:42.703 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.699+0000 7f5fe7fff640 1 -- 172.21.15.42:0/3864883504 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fec010e40 con 0x55cfb11959f0 2024-08-30T09:13:42.703 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.699+0000 7f5fe7fff640 1 -- 172.21.15.42:0/3864883504 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5fec02c030 con 0x55cfb11959f0 2024-08-30T09:13:42.703 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.699+0000 7f5fe7fff640 1 -- 172.21.15.42:0/3864883504 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fec030740 con 0x55cfb11959f0 2024-08-30T09:13:42.703 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.700+0000 7f600ab4a740 1 -- 172.21.15.42:0/3864883504 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cfb11959f0 msgr2=0x55cfb1584d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:13:42.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.700+0000 7f600ab4a740 1 --2- 172.21.15.42:0/3864883504 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cfb11959f0 0x55cfb1584d90 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x55cfb159a600 tx=0x7f5fec017000 comp rx=0 tx=0).stop 2024-08-30T09:13:42.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.700+0000 7f600ab4a740 1 -- 172.21.15.42:0/3864883504 shutdown_connections 2024-08-30T09:13:42.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.700+0000 7f600ab4a740 1 --2- 172.21.15.42:0/3864883504 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cfb11ae1b0 0x55cfb15852d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:42.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.700+0000 7f600ab4a740 1 --2- 172.21.15.42:0/3864883504 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cfb11959f0 0x55cfb1584d90 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:42.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.700+0000 7f600ab4a740 1 --2- 172.21.15.42:0/3864883504 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cfb11cb460 0x55cfb157a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:42.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.700+0000 7f600ab4a740 1 -- 172.21.15.42:0/3864883504 >> 172.21.15.42:0/3864883504 conn(0x55cfb159a1f0 msgr2=0x55cfb1579970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:13:42.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.701+0000 7f600ab4a740 1 -- 172.21.15.42:0/3864883504 shutdown_connections 2024-08-30T09:13:42.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.701+0000 7f600ab4a740 1 -- 172.21.15.42:0/3864883504 wait complete. 2024-08-30T09:13:42.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.701+0000 7f600ab4a740 1 -- start start 2024-08-30T09:13:42.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.702+0000 7f600ab4a740 1 Processor -- start 2024-08-30T09:13:42.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.702+0000 7f600ab4a740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cfb11959f0 0x55cfb15907c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:42.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.702+0000 7f600ab4a740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cfb11ae1b0 0x55cfb158af00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:42.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.702+0000 7f6002371640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cfb11959f0 0x55cfb15907c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:13:42.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.702+0000 7f6002371640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cfb11959f0 0x55cfb15907c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:57244/0 (socket says 172.21.15.42:57244) 2024-08-30T09:13:42.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.703+0000 7f6002371640 1 -- 172.21.15.42:0/518287477 learned_addr learned my addr 172.21.15.42:0/518287477 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:13:42.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.703+0000 7f600ab4a740 1 --2- 172.21.15.42:0/518287477 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cfb11cb460 0x55cfb158b500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:42.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.703+0000 7f6001b70640 1 --2- 172.21.15.42:0/518287477 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cfb11ae1b0 0x55cfb158af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:13:42.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.703+0000 7f600ab4a740 1 -- 172.21.15.42:0/518287477 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f5ffc07cba0 con 0x55cfb11ae1b0 2024-08-30T09:13:42.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.703+0000 7f600ab4a740 1 -- 172.21.15.42:0/518287477 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f5ffc07e010 con 0x55cfb11959f0 2024-08-30T09:13:42.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.703+0000 7f600ab4a740 1 -- 172.21.15.42:0/518287477 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f5ffc07d270 con 0x55cfb11cb460 2024-08-30T09:13:42.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.703+0000 7f600ab4a740 10 client.0 _ll_register_callbacks cb 0x7f5ff800f830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:13:42.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.703+0000 7f6002b72640 1 --2- 172.21.15.42:0/518287477 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cfb11cb460 0x55cfb158b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:13:42.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.703+0000 7f6002371640 1 -- 172.21.15.42:0/518287477 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cfb11cb460 msgr2=0x55cfb158b500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:13:42.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.703+0000 7f6002371640 1 --2- 172.21.15.42:0/518287477 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cfb11cb460 0x55cfb158b500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:42.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.703+0000 7f6002371640 1 -- 172.21.15.42:0/518287477 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cfb11ae1b0 msgr2=0x55cfb158af00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:13:42.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.703+0000 7f6002371640 1 --2- 172.21.15.42:0/518287477 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cfb11ae1b0 0x55cfb158af00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:42.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.703+0000 7f6002371640 1 -- 172.21.15.42:0/518287477 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cfb158bc60 con 0x55cfb11959f0 2024-08-30T09:13:42.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.703+0000 7f6002371640 1 --2- 172.21.15.42:0/518287477 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cfb11959f0 0x55cfb15907c0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f5ff80c84c0 tx=0x7f5ff80d0610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=652642d8d145cc14 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:13:42.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.703+0000 7f5fe5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-30T09:13:42.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.703+0000 7f5fe5ffb640 1 -- 172.21.15.42:0/518287477 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ff80f2040 con 0x55cfb11959f0 2024-08-30T09:13:42.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.704+0000 7f5fe5ffb640 1 -- 172.21.15.42:0/518287477 <== mon.2 v2:172.21.15.167:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f5ff80cabf0 con 0x55cfb11959f0 2024-08-30T09:13:42.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.704+0000 7f600ab4a740 1 -- 172.21.15.42:0/518287477 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5fec02c030 con 0x55cfb11959f0 2024-08-30T09:13:42.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.704+0000 7f5fe5ffb640 1 -- 172.21.15.42:0/518287477 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f5ff80cb690 con 0x55cfb11959f0 2024-08-30T09:13:42.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.704+0000 7f600ab4a740 10 client.27440 fetch_fsmap learned FSMap version 45 2024-08-30T09:13:42.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.704+0000 7f600ab4a740 1 -- 172.21.15.42:0/518287477 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55cfb1605880 con 0x55cfb11959f0 2024-08-30T09:13:42.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.704+0000 7f5fe5ffb640 1 -- 172.21.15.42:0/518287477 <== mon.2 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5fec02c030 con 0x55cfb11959f0 2024-08-30T09:13:42.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.705+0000 7f5fe5ffb640 1 -- 172.21.15.42:0/518287477 <== mon.2 v2:172.21.15.167:3300/0 5 ==== fsmap(e 45) ==== 8362+0+0 (secure 0 0 0) 0x7f5ff80ed110 con 0x55cfb11959f0 2024-08-30T09:13:42.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.705+0000 7f600ab4a740 10 client.27440 fetch_fsmap finished waiting for FSMap version 45 2024-08-30T09:13:42.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.705+0000 7f600ab4a740 10 client.27440 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-30T09:13:42.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.705+0000 7f600ab4a740 20 client.27440 populate_metadata read hostname 'smithi042' 2024-08-30T09:13:42.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.705+0000 7f600ab4a740 1 --2- 172.21.15.42:0/518287477 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55cfb11b6390 0x55cfb1607e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:42.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.705+0000 7f600ab4a740 4 client.27440 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-30T09:13:42.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.705+0000 7f600ab4a740 1 -- 172.21.15.42:0/518287477 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55cfb14f1020 con 0x55cfb11b6390 2024-08-30T09:13:42.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.705+0000 7f6001b70640 1 --2- 172.21.15.42:0/518287477 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55cfb11b6390 0x55cfb1607e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:13:42.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.706+0000 7f6001b70640 1 --2- 172.21.15.42:0/518287477 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55cfb11b6390 0x55cfb1607e70 crc :-1 s=READY pgs=268 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ac5fad80c9974bd server_cookie=5fd87fe87e0a5b9e in_seq=0 out_seq=0 2024-08-30T09:13:42.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.706+0000 7f5fe5ffb640 10 client.27440 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:13:42.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.706+0000 7f5fe5ffb640 1 -- 172.21.15.42:0/518287477 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+4830 (crc 0 0 0) 0x55cfb14f1020 con 0x55cfb11b6390 2024-08-30T09:13:42.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.706+0000 7f5fe5ffb640 10 client.27440 handle_command_reply: tid=0 2024-08-30T09:13:42.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.706+0000 7f600ab4a740 1 client.27440 shutdown 2024-08-30T09:13:42.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.706+0000 7f600ab4a740 1 -- 172.21.15.42:0/518287477 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cfb11959f0 msgr2=0x55cfb15907c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:13:42.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.706+0000 7f600ab4a740 1 --2- 172.21.15.42:0/518287477 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cfb11959f0 0x55cfb15907c0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f5ff80c84c0 tx=0x7f5ff80d0610 comp rx=0 tx=0).stop 2024-08-30T09:13:42.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.706+0000 7f600ab4a740 1 -- 172.21.15.42:0/518287477 shutdown_connections 2024-08-30T09:13:42.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.706+0000 7f600ab4a740 1 --2- 172.21.15.42:0/518287477 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55cfb11b6390 0x55cfb1607e70 crc :-1 s=READY pgs=268 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:42.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.706+0000 7f600ab4a740 1 --2- 172.21.15.42:0/518287477 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cfb11cb460 0x55cfb158b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:42.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.706+0000 7f600ab4a740 1 --2- 172.21.15.42:0/518287477 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cfb11ae1b0 0x55cfb158af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:42.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.706+0000 7f600ab4a740 1 --2- 172.21.15.42:0/518287477 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cfb11959f0 0x55cfb15907c0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:42.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.706+0000 7f600ab4a740 1 -- 172.21.15.42:0/518287477 >> 172.21.15.42:0/518287477 conn(0x55cfb159a1f0 msgr2=0x55cfb1589050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:13:42.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.706+0000 7f5fe5ffb640 0 client.27440 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:13:42.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.706+0000 7f600ab4a740 1 -- 172.21.15.42:0/518287477 shutdown_connections 2024-08-30T09:13:42.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.706+0000 7f600ab4a740 1 -- 172.21.15.42:0/518287477 wait complete. 2024-08-30T09:13:42.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.707+0000 7f600ab4a740 20 client.27440 trim_cache size 0 max 16384 2024-08-30T09:13:42.719 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-08-30T09:13:42.719 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "IDLE (waiting for more scrubs)", 2024-08-30T09:13:42.719 INFO:teuthology.orchestra.run.smithi042.stdout: "scrubs": { 2024-08-30T09:13:42.719 INFO:teuthology.orchestra.run.smithi042.stdout: "03521407-a747-49b9-a415-6233551c26fb": { 2024-08-30T09:13:42.719 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:13:42.719 INFO:teuthology.orchestra.run.smithi042.stdout: "tag": "03521407-a747-49b9-a415-6233551c26fb", 2024-08-30T09:13:42.719 INFO:teuthology.orchestra.run.smithi042.stdout: "options": "recursive,force" 2024-08-30T09:13:42.719 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:13:42.719 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:13:42.719 INFO:teuthology.orchestra.run.smithi042.stdout: "scrub_stats": { 2024-08-30T09:13:42.719 INFO:teuthology.orchestra.run.smithi042.stdout: "03521407-a747-49b9-a415-6233551c26fb": { 2024-08-30T09:13:42.719 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:13:39.387817+0000", 2024-08-30T09:13:42.719 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:13:42.720 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:13:42.720 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:13:42.720 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:13:42.720 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:13:42.720 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:13:42.720 INFO:teuthology.orchestra.run.smithi042.stdout: "0cbf734e-f988-4b21-acac-57b55e3bde62": { 2024-08-30T09:13:42.720 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:12:45.306738+0000", 2024-08-30T09:13:42.720 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:13:42.720 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:13:42.720 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:13:42.720 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:13:42.720 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:13:42.721 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:13:42.721 INFO:teuthology.orchestra.run.smithi042.stdout: "5e2ed104-0625-478a-a78b-584e7f4c2817": { 2024-08-30T09:13:42.721 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:00:33.928400+0000", 2024-08-30T09:13:42.721 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:13:42.721 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:13:42.721 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:13:42.721 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:13:42.721 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:13:42.721 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:13:42.721 INFO:teuthology.orchestra.run.smithi042.stdout: "c827ad7a-44ee-4e1d-9bb3-51fb7ab00c73": { 2024-08-30T09:13:42.721 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:02:21.686517+0000", 2024-08-30T09:13:42.721 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:13:42.721 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:13:42.722 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:13:42.722 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:13:42.722 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:13:42.722 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:13:42.722 INFO:teuthology.orchestra.run.smithi042.stdout: "85134e4b-ab4a-4885-b07e-a3ad01eaf13d": { 2024-08-30T09:13:42.722 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:01:27.738462+0000", 2024-08-30T09:13:42.722 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:13:42.722 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:13:42.722 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:13:42.722 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:13:42.722 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:13:42.722 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:13:42.722 INFO:teuthology.orchestra.run.smithi042.stdout: "749f4867-0b70-49e5-af2d-8362a240ebc3": { 2024-08-30T09:13:42.722 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:03:16.386554+0000", 2024-08-30T09:13:42.723 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:13:42.723 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:13:42.723 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:13:42.723 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:13:42.723 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:13:42.723 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:13:42.723 INFO:teuthology.orchestra.run.smithi042.stdout: "88cd7478-2b20-47de-ba51-256479749232": { 2024-08-30T09:13:42.723 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:04:08.922625+0000", 2024-08-30T09:13:42.723 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:13:42.723 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:13:42.723 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:13:42.723 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:13:42.723 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:13:42.724 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:13:42.724 INFO:teuthology.orchestra.run.smithi042.stdout: "347d05ed-4bd3-4590-b099-314f719aab39": { 2024-08-30T09:13:42.724 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:05:00.439721+0000", 2024-08-30T09:13:42.724 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:13:42.724 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:13:42.724 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:13:42.724 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:13:42.724 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:13:42.724 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:13:42.724 INFO:teuthology.orchestra.run.smithi042.stdout: "1af4252a-4a99-47e9-bbf0-becf5e573dd5": { 2024-08-30T09:13:42.724 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:05:48.565881+0000", 2024-08-30T09:13:42.724 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:13:42.724 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:13:42.725 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:13:42.725 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:13:42.725 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:13:42.725 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:13:42.725 INFO:teuthology.orchestra.run.smithi042.stdout: "665ec740-7254-49cb-84ef-deff71c461b0": { 2024-08-30T09:13:42.725 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:06:37.144035+0000", 2024-08-30T09:13:42.725 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:13:42.725 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:13:42.725 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:13:42.725 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:13:42.725 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:13:42.725 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:13:42.725 INFO:teuthology.orchestra.run.smithi042.stdout: "598d8a75-f1cc-4388-80bc-a408bdb4eaac": { 2024-08-30T09:13:42.726 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:07:29.721122+0000", 2024-08-30T09:13:42.726 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:13:42.726 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:13:42.726 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:13:42.726 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:13:42.726 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:13:42.726 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:13:42.726 INFO:teuthology.orchestra.run.smithi042.stdout: "b62c4ffc-9387-4677-b9ba-1e94ba308f17": { 2024-08-30T09:13:42.726 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:11:03.363478+0000", 2024-08-30T09:13:42.726 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:13:42.726 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:13:42.726 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:13:42.726 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:13:42.726 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:13:42.726 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:13:42.726 INFO:teuthology.orchestra.run.smithi042.stdout: "a7210f9a-046b-4188-953c-819b9f1863a9": { 2024-08-30T09:13:42.727 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:10:07.608430+0000", 2024-08-30T09:13:42.727 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:13:42.727 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:13:42.727 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:13:42.727 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:13:42.727 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:13:42.727 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:13:42.727 INFO:teuthology.orchestra.run.smithi042.stdout: "99a29ec1-9455-4ae4-854a-398b32896675": { 2024-08-30T09:13:42.727 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:08:24.923176+0000", 2024-08-30T09:13:42.727 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:13:42.727 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:13:42.727 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:13:42.727 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:13:42.727 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:13:42.728 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:13:42.728 INFO:teuthology.orchestra.run.smithi042.stdout: "ff464e7f-123b-4bc6-bc7e-aced018168de": { 2024-08-30T09:13:42.728 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:11:54.879598+0000", 2024-08-30T09:13:42.728 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:13:42.728 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:13:42.728 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:13:42.728 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:13:42.728 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:13:42.728 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:13:42.728 INFO:teuthology.orchestra.run.smithi042.stdout: "37f2f28f-496f-41a6-b4cf-8846fb3f43ab": { 2024-08-30T09:13:42.728 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:09:17.428286+0000", 2024-08-30T09:13:42.728 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:13:42.728 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:13:42.729 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:13:42.729 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:13:42.729 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:13:42.729 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:13:42.729 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:13:42.729 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-08-30T09:13:42.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.707+0000 7f6003373640 1 -- 172.21.15.42:0/3182178206 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f5fe80b4cc0 msgr2=0x7f5fe80b71d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:13:42.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.707+0000 7f6003373640 1 --2- 172.21.15.42:0/3182178206 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f5fe80b4cc0 0x7f5fe80b71d0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x55cfb157b0e0 tx=0x7f5fec0046e0 comp rx=0 tx=0).stop 2024-08-30T09:13:42.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.707+0000 7f6003373640 1 -- 172.21.15.42:0/3182178206 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5ffc10a3e0 msgr2=0x7f5ffc121460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:13:42.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.707+0000 7f6003373640 1 --2- 172.21.15.42:0/3182178206 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5ffc10a3e0 0x7f5ffc121460 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f5ff802eba0 tx=0x7f5ff800e660 comp rx=0 tx=0).stop 2024-08-30T09:13:42.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.707+0000 7f6003373640 1 -- 172.21.15.42:0/3182178206 shutdown_connections 2024-08-30T09:13:42.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.707+0000 7f6003373640 1 --2- 172.21.15.42:0/3182178206 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f5fe80b4cc0 0x7f5fe80b71d0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:42.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.707+0000 7f6003373640 1 --2- 172.21.15.42:0/3182178206 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5ffc10ad30 0x7f5ffc11a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:42.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.707+0000 7f6003373640 1 --2- 172.21.15.42:0/3182178206 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5ffc1162b0 0x7f5ffc11ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:42.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.707+0000 7f6003373640 1 --2- 172.21.15.42:0/3182178206 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5ffc10a3e0 0x7f5ffc121460 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:42.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.707+0000 7f6003373640 1 -- 172.21.15.42:0/3182178206 >> 172.21.15.42:0/3182178206 conn(0x7f5ffc06e790 msgr2=0x7f5ffc10dd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:13:42.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.707+0000 7f6003373640 1 -- 172.21.15.42:0/3182178206 shutdown_connections 2024-08-30T09:13:42.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:13:42.707+0000 7f6003373640 1 -- 172.21.15.42:0/3182178206 wait complete. 2024-08-30T09:13:43.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:13:43 smithi167 ceph-mon[22063]: pgmap v761: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 4.0 MiB/s wr, 394 op/s 2024-08-30T09:13:43.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:13:43 smithi042 ceph-mon[18652]: pgmap v761: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 4.0 MiB/s wr, 394 op/s 2024-08-30T09:13:43.424 INFO:tasks.cephfs.filesystem:scrub status for tag:03521407-a747-49b9-a415-6233551c26fb - {'path': '/', 'tag': '03521407-a747-49b9-a415-6233551c26fb', 'options': 'recursive,force'} 2024-08-30T09:13:43.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:13:43 smithi190 ceph-mon[22001]: pgmap v761: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 4.0 MiB/s wr, 394 op/s 2024-08-30T09:13:44.380 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.374+0000 7fdd2a379640 1 -- 172.21.15.42:0/3216868339 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdd241092e0 msgr2=0x7fdd2410b700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:13:44.380 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.374+0000 7fdd2a379640 1 --2- 172.21.15.42:0/3216868339 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdd241092e0 0x7fdd2410b700 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fdd1c009ca0 tx=0x7fdd1c02a8c0 comp rx=0 tx=0).stop 2024-08-30T09:13:44.380 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.374+0000 7fdd2a379640 1 -- 172.21.15.42:0/3216868339 shutdown_connections 2024-08-30T09:13:44.381 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.374+0000 7fdd2a379640 1 --2- 172.21.15.42:0/3216868339 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdd241092e0 0x7fdd2410b700 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:44.381 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.374+0000 7fdd2a379640 1 --2- 172.21.15.42:0/3216868339 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdd24106980 0x7fdd24108da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:44.381 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.374+0000 7fdd2a379640 1 --2- 172.21.15.42:0/3216868339 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdd24101830 0x7fdd24101c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:44.381 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.374+0000 7fdd2a379640 1 -- 172.21.15.42:0/3216868339 >> 172.21.15.42:0/3216868339 conn(0x7fdd240fd430 msgr2=0x7fdd240ff8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:13:44.381 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.374+0000 7fdd2a379640 1 -- 172.21.15.42:0/3216868339 shutdown_connections 2024-08-30T09:13:44.381 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.374+0000 7fdd2a379640 1 -- 172.21.15.42:0/3216868339 wait complete. 2024-08-30T09:13:44.381 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.375+0000 7fdd2a379640 1 Processor -- start 2024-08-30T09:13:44.381 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.375+0000 7fdd2a379640 1 -- start start 2024-08-30T09:13:44.381 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.375+0000 7fdd2a379640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdd24101830 0x7fdd24103fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:44.381 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.375+0000 7fdd2a379640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdd24106980 0x7fdd24102880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:44.382 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.375+0000 7fdd2a379640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdd241092e0 0x7fdd24102dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:44.382 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.375+0000 7fdd2a379640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fdd2410f110 con 0x7fdd241092e0 2024-08-30T09:13:44.382 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.375+0000 7fdd2a379640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fdd2410ef90 con 0x7fdd24101830 2024-08-30T09:13:44.382 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.375+0000 7fdd2a379640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fdd2410f290 con 0x7fdd24106980 2024-08-30T09:13:44.382 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.375+0000 7fdd28916640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdd241092e0 0x7fdd24102dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:13:44.382 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.375+0000 7fdd28916640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdd241092e0 0x7fdd24102dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:39234/0 (socket says 172.21.15.42:39234) 2024-08-30T09:13:44.382 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.375+0000 7fdd28916640 1 -- 172.21.15.42:0/3708006518 learned_addr learned my addr 172.21.15.42:0/3708006518 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:13:44.382 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.375+0000 7fdd23fff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdd24101830 0x7fdd24103fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:13:44.383 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.375+0000 7fdd237fe640 1 --2- 172.21.15.42:0/3708006518 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdd24106980 0x7fdd24102880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:13:44.383 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.376+0000 7fdd23fff640 1 -- 172.21.15.42:0/3708006518 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdd24106980 msgr2=0x7fdd24102880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:13:44.383 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.376+0000 7fdd23fff640 1 --2- 172.21.15.42:0/3708006518 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdd24106980 0x7fdd24102880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:44.383 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.376+0000 7fdd23fff640 1 -- 172.21.15.42:0/3708006518 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdd241092e0 msgr2=0x7fdd24102dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:13:44.383 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.376+0000 7fdd23fff640 1 --2- 172.21.15.42:0/3708006518 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdd241092e0 0x7fdd24102dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:44.383 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.376+0000 7fdd23fff640 1 -- 172.21.15.42:0/3708006518 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd241034f0 con 0x7fdd24101830 2024-08-30T09:13:44.383 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.376+0000 7fdd28916640 1 --2- 172.21.15.42:0/3708006518 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdd241092e0 0x7fdd24102dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-30T09:13:44.383 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.376+0000 7fdd23fff640 1 --2- 172.21.15.42:0/3708006518 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdd24101830 0x7fdd24103fc0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fdd1400e9f0 tx=0x7fdd1400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:13:44.384 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.376+0000 7fdd217fa640 1 -- 172.21.15.42:0/3708006518 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd1400ccc0 con 0x7fdd24101830 2024-08-30T09:13:44.384 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.376+0000 7fdd217fa640 1 -- 172.21.15.42:0/3708006518 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd14005020 con 0x7fdd24101830 2024-08-30T09:13:44.384 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.376+0000 7fdd2a379640 1 -- 172.21.15.42:0/3708006518 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd241a3d30 con 0x7fdd24101830 2024-08-30T09:13:44.384 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.376+0000 7fdd217fa640 1 -- 172.21.15.42:0/3708006518 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd14004a00 con 0x7fdd24101830 2024-08-30T09:13:44.384 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.376+0000 7fdd2a379640 1 -- 172.21.15.42:0/3708006518 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd241a41a0 con 0x7fdd24101830 2024-08-30T09:13:44.392 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.380+0000 7fdd313c4740 1 Processor -- start 2024-08-30T09:13:44.392 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.380+0000 7fdd313c4740 1 -- start start 2024-08-30T09:13:44.392 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.381+0000 7fdd313c4740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559dde9a11b0 0x559dded46f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:44.392 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.381+0000 7fdd313c4740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559dde9782f0 0x559dded475d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:44.393 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.381+0000 7fdd313c4740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559dde9804d0 0x559dded48120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:44.393 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.381+0000 7fdd313c4740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x559ddecba000 con 0x559dde9a11b0 2024-08-30T09:13:44.393 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.381+0000 7fdd313c4740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x559ddeca9590 con 0x559dde9804d0 2024-08-30T09:13:44.393 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.381+0000 7fdd313c4740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x559ddecad540 con 0x559dde9782f0 2024-08-30T09:13:44.393 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.381+0000 7fdd28916640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559dde9a11b0 0x559dded46f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:13:44.393 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.381+0000 7fdd28916640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559dde9a11b0 0x559dded46f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:39238/0 (socket says 172.21.15.42:39238) 2024-08-30T09:13:44.393 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.381+0000 7fdd28916640 1 -- 172.21.15.42:0/2882152710 learned_addr learned my addr 172.21.15.42:0/2882152710 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:13:44.393 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.381+0000 7fdd237fe640 1 --2- 172.21.15.42:0/2882152710 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559dde9804d0 0x559dded48120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:13:44.393 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.381+0000 7fdd217fa640 1 -- 172.21.15.42:0/3708006518 <== mon.2 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7fdd14004ba0 con 0x7fdd24101830 2024-08-30T09:13:44.393 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.382+0000 7fdd237fe640 1 -- 172.21.15.42:0/2882152710 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559dde9782f0 msgr2=0x559dded475d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-30T09:13:44.394 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.382+0000 7fdd237fe640 1 --2- 172.21.15.42:0/2882152710 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559dde9782f0 0x559dded475d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:44.394 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.382+0000 7fdd237fe640 1 -- 172.21.15.42:0/2882152710 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559dde9a11b0 msgr2=0x559dded46f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:13:44.394 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.382+0000 7fdd237fe640 1 --2- 172.21.15.42:0/2882152710 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559dde9a11b0 0x559dded46f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:44.394 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.382+0000 7fdd237fe640 1 -- 172.21.15.42:0/2882152710 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559ddea0c790 con 0x559dde9804d0 2024-08-30T09:13:44.394 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.382+0000 7fdd28916640 1 --2- 172.21.15.42:0/2882152710 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559dde9a11b0 0x559dded46f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-30T09:13:44.394 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.382+0000 7fdd237fe640 1 --2- 172.21.15.42:0/2882152710 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559dde9804d0 0x559dded48120 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fdd18009c20 tx=0x7fdd18011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a0524500711bb0e server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:13:44.394 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.382+0000 7fdd0a7fc640 1 -- 172.21.15.42:0/2882152710 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd1801e070 con 0x559dde9804d0 2024-08-30T09:13:44.394 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.382+0000 7fdd0a7fc640 1 -- 172.21.15.42:0/2882152710 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd18009dd0 con 0x559dde9804d0 2024-08-30T09:13:44.394 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.382+0000 7fdd0a7fc640 1 -- 172.21.15.42:0/2882152710 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd18018770 con 0x559dde9804d0 2024-08-30T09:13:44.394 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.382+0000 7fdd217fa640 1 --2- 172.21.15.42:0/3708006518 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fdd0c0b4b30 0x7fdd0c0b7040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:44.395 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.383+0000 7fdd28916640 1 --2- 172.21.15.42:0/3708006518 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fdd0c0b4b30 0x7fdd0c0b7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:13:44.395 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.383+0000 7fdd217fa640 1 -- 172.21.15.42:0/3708006518 <== mon.2 v2:172.21.15.167:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7fdd140e5460 con 0x7fdd24101830 2024-08-30T09:13:44.395 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.383+0000 7fdd313c4740 1 -- 172.21.15.42:0/2882152710 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559dde9804d0 msgr2=0x559dded48120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:13:44.395 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.383+0000 7fdd313c4740 1 --2- 172.21.15.42:0/2882152710 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559dde9804d0 0x559dded48120 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fdd18009c20 tx=0x7fdd18011810 comp rx=0 tx=0).stop 2024-08-30T09:13:44.395 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.383+0000 7fdd313c4740 1 -- 172.21.15.42:0/2882152710 shutdown_connections 2024-08-30T09:13:44.395 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.383+0000 7fdd313c4740 1 --2- 172.21.15.42:0/2882152710 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559dde9782f0 0x559dded475d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:44.395 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.383+0000 7fdd313c4740 1 --2- 172.21.15.42:0/2882152710 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559dde9804d0 0x559dded48120 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:44.395 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.383+0000 7fdd313c4740 1 --2- 172.21.15.42:0/2882152710 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559dde9a11b0 0x559dded46f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:44.395 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.383+0000 7fdd313c4740 1 -- 172.21.15.42:0/2882152710 >> 172.21.15.42:0/2882152710 conn(0x559dde995300 msgr2=0x559dded429f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:13:44.395 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.383+0000 7fdd313c4740 1 -- 172.21.15.42:0/2882152710 shutdown_connections 2024-08-30T09:13:44.395 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.383+0000 7fdd28916640 1 --2- 172.21.15.42:0/3708006518 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fdd0c0b4b30 0x7fdd0c0b7040 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fdd1c0094e0 tx=0x7fdd1c031af0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:13:44.396 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.383+0000 7fdd313c4740 1 -- 172.21.15.42:0/2882152710 wait complete. 2024-08-30T09:13:44.396 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.384+0000 7fdd313c4740 1 -- start start 2024-08-30T09:13:44.396 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.384+0000 7fdd313c4740 1 Processor -- start 2024-08-30T09:13:44.396 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.384+0000 7fdd313c4740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559dde9782f0 0x559dded50a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:44.396 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.384+0000 7fdd313c4740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559dde9804d0 0x559dded51100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:44.396 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.384+0000 7fdd313c4740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559dde9a11b0 0x559dded51c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:44.396 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.384+0000 7fdd313c4740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x559ddebe2ab0 con 0x559dde9a11b0 2024-08-30T09:13:44.396 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.384+0000 7fdd313c4740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x559ddec30680 con 0x559dde9804d0 2024-08-30T09:13:44.396 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.384+0000 7fdd313c4740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x559dded48c40 con 0x559dde9782f0 2024-08-30T09:13:44.396 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.384+0000 7fdd313c4740 10 client.0 _ll_register_callbacks cb 0x559dde9b8a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:13:44.397 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.384+0000 7fdd237fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559dde9804d0 0x559dded51100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:13:44.397 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.384+0000 7fdd23fff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559dde9782f0 0x559dded50a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:13:44.397 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.384+0000 7fdd23fff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559dde9782f0 0x559dded50a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:36556/0 (socket says 172.21.15.42:36556) 2024-08-30T09:13:44.397 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.384+0000 7fdd237fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559dde9804d0 0x559dded51100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:57276/0 (socket says 172.21.15.42:57276) 2024-08-30T09:13:44.397 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.384+0000 7fdd23fff640 1 -- 172.21.15.42:0/2168949009 learned_addr learned my addr 172.21.15.42:0/2168949009 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:13:44.397 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.385+0000 7fdd28916640 1 --2- 172.21.15.42:0/2168949009 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559dde9a11b0 0x559dded51c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:13:44.397 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.385+0000 7fdd23fff640 1 -- 172.21.15.42:0/2168949009 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559dde9804d0 msgr2=0x559dded51100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:13:44.397 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.385+0000 7fdd23fff640 1 --2- 172.21.15.42:0/2168949009 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559dde9804d0 0x559dded51100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:44.397 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.385+0000 7fdd23fff640 1 -- 172.21.15.42:0/2168949009 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559dde9a11b0 msgr2=0x559dded51c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:13:44.397 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.385+0000 7fdd23fff640 1 --2- 172.21.15.42:0/2168949009 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559dde9a11b0 0x559dded51c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:44.397 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.385+0000 7fdd23fff640 1 -- 172.21.15.42:0/2168949009 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559ddea85620 con 0x559dde9782f0 2024-08-30T09:13:44.398 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.385+0000 7fdd237fe640 1 --2- 172.21.15.42:0/2168949009 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559dde9804d0 0x559dded51100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-30T09:13:44.398 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.385+0000 7fdd28916640 1 --2- 172.21.15.42:0/2168949009 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559dde9a11b0 0x559dded51c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-30T09:13:44.398 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.385+0000 7fdd23fff640 1 --2- 172.21.15.42:0/2168949009 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559dde9782f0 0x559dded50a40 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fdd140ab800 tx=0x7fdd140ab330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ac17513d2d527058 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:13:44.398 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.385+0000 7fdd037fe640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-08-30T09:13:44.398 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.385+0000 7fdd037fe640 1 -- 172.21.15.42:0/2168949009 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd140b0070 con 0x559dde9782f0 2024-08-30T09:13:44.398 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.385+0000 7fdd313c4740 1 -- 172.21.15.42:0/2168949009 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdd18009dd0 con 0x559dde9782f0 2024-08-30T09:13:44.398 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.385+0000 7fdd037fe640 1 -- 172.21.15.42:0/2168949009 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7fdd140054f0 con 0x559dde9782f0 2024-08-30T09:13:44.398 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.386+0000 7fdd037fe640 1 -- 172.21.15.42:0/2168949009 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fdd140c6050 con 0x559dde9782f0 2024-08-30T09:13:44.398 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.386+0000 7fdd313c4740 10 client.27382 fetch_fsmap learned FSMap version 45 2024-08-30T09:13:44.398 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.386+0000 7fdd313c4740 1 -- 172.21.15.42:0/2168949009 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x559dded5ace0 con 0x559dde9782f0 2024-08-30T09:13:44.398 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.386+0000 7fdd037fe640 1 -- 172.21.15.42:0/2168949009 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd18009dd0 con 0x559dde9782f0 2024-08-30T09:13:44.399 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.386+0000 7fdd037fe640 1 -- 172.21.15.42:0/2168949009 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 45) ==== 8362+0+0 (secure 0 0 0) 0x7fdd140cebe0 con 0x559dde9782f0 2024-08-30T09:13:44.399 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.386+0000 7fdd313c4740 10 client.27382 fetch_fsmap finished waiting for FSMap version 45 2024-08-30T09:13:44.399 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.386+0000 7fdd313c4740 10 client.27382 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-30T09:13:44.399 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.386+0000 7fdd313c4740 20 client.27382 populate_metadata read hostname 'smithi042' 2024-08-30T09:13:44.399 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.386+0000 7fdd313c4740 1 --2- 172.21.15.42:0/2168949009 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x559dde95fd80 0x559dded5e390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:44.399 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.386+0000 7fdd313c4740 4 client.27382 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-30T09:13:44.399 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.386+0000 7fdd313c4740 1 -- 172.21.15.42:0/2168949009 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559ddecbbf50 con 0x559dde95fd80 2024-08-30T09:13:44.399 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.386+0000 7fdd237fe640 1 --2- 172.21.15.42:0/2168949009 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x559dde95fd80 0x559dded5e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:13:44.399 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.387+0000 7fdd037fe640 10 client.27382 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:13:44.399 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.387+0000 7fdd237fe640 1 --2- 172.21.15.42:0/2168949009 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x559dde95fd80 0x559dded5e390 crc :-1 s=READY pgs=269 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed30beb901b391e9 server_cookie=dd8364a4447eb8c2 in_seq=0 out_seq=0 2024-08-30T09:13:44.399 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.387+0000 7fdd037fe640 1 -- 172.21.15.42:0/2168949009 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x559ddecbbf50 con 0x559dde95fd80 2024-08-30T09:13:44.400 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.387+0000 7fdd037fe640 10 client.27382 handle_command_reply: tid=0 2024-08-30T09:13:44.400 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.387+0000 7fdd313c4740 1 client.27382 shutdown 2024-08-30T09:13:44.400 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.388+0000 7fdd313c4740 1 -- 172.21.15.42:0/2168949009 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559dde9782f0 msgr2=0x559dded50a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:13:44.400 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.388+0000 7fdd313c4740 1 --2- 172.21.15.42:0/2168949009 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559dde9782f0 0x559dded50a40 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fdd140ab800 tx=0x7fdd140ab330 comp rx=0 tx=0).stop 2024-08-30T09:13:44.400 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.388+0000 7fdd313c4740 1 -- 172.21.15.42:0/2168949009 shutdown_connections 2024-08-30T09:13:44.400 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.388+0000 7fdd313c4740 1 --2- 172.21.15.42:0/2168949009 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x559dde95fd80 0x559dded5e390 crc :-1 s=READY pgs=269 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:44.400 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.388+0000 7fdd313c4740 1 --2- 172.21.15.42:0/2168949009 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559dde9a11b0 0x559dded51c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:44.400 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.388+0000 7fdd313c4740 1 --2- 172.21.15.42:0/2168949009 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559dde9804d0 0x559dded51100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:44.400 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.388+0000 7fdd313c4740 1 --2- 172.21.15.42:0/2168949009 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559dde9782f0 0x559dded50a40 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:44.400 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.388+0000 7fdd313c4740 1 -- 172.21.15.42:0/2168949009 >> 172.21.15.42:0/2168949009 conn(0x559dde995300 msgr2=0x559dded40c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:13:44.400 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.388+0000 7fdd037fe640 0 client.27382 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:13:44.400 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.388+0000 7fdd313c4740 1 -- 172.21.15.42:0/2168949009 shutdown_connections 2024-08-30T09:13:44.401 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.388+0000 7fdd313c4740 1 -- 172.21.15.42:0/2168949009 wait complete. 2024-08-30T09:13:44.401 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.388+0000 7fdd313c4740 20 client.27382 trim_cache size 0 max 16384 2024-08-30T09:13:44.423 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.420+0000 7fdd313c4740 1 Processor -- start 2024-08-30T09:13:44.428 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.425+0000 7fdd313c4740 1 -- start start 2024-08-30T09:13:44.429 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.425+0000 7fdd313c4740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559dde9782f0 0x559dded42700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:44.429 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.425+0000 7fdd313c4740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559dde9804d0 0x559dded40b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:44.429 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.425+0000 7fdd313c4740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559dde9a11b0 0x559dded410a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:44.429 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.425+0000 7fdd313c4740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fdd241b0ef0 con 0x559dde9a11b0 2024-08-30T09:13:44.429 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.425+0000 7fdd313c4740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fdd241ae760 con 0x559dde9804d0 2024-08-30T09:13:44.429 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.425+0000 7fdd313c4740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fdd241ac5b0 con 0x559dde9782f0 2024-08-30T09:13:44.433 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.430+0000 7fdd237fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559dde9804d0 0x559dded40b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:13:44.434 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.431+0000 7fdd23fff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559dde9782f0 0x559dded42700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:13:44.437 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.432+0000 7fdd237fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559dde9804d0 0x559dded40b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:57278/0 (socket says 172.21.15.42:57278) 2024-08-30T09:13:44.437 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.432+0000 7fdd237fe640 1 -- 172.21.15.42:0/1080671885 learned_addr learned my addr 172.21.15.42:0/1080671885 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:13:44.440 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.435+0000 7fdd28916640 1 --2- 172.21.15.42:0/1080671885 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559dde9a11b0 0x559dded410a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:13:44.440 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.436+0000 7fdd237fe640 1 -- 172.21.15.42:0/1080671885 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559dde9782f0 msgr2=0x559dded42700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:13:44.440 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.436+0000 7fdd237fe640 1 --2- 172.21.15.42:0/1080671885 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559dde9782f0 0x559dded42700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:44.441 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.436+0000 7fdd237fe640 1 -- 172.21.15.42:0/1080671885 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559dde9a11b0 msgr2=0x559dded410a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:13:44.441 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.436+0000 7fdd237fe640 1 --2- 172.21.15.42:0/1080671885 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559dde9a11b0 0x559dded410a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:44.441 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.436+0000 7fdd237fe640 1 -- 172.21.15.42:0/1080671885 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559dde95ffb0 con 0x559dde9804d0 2024-08-30T09:13:44.442 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.439+0000 7fdd237fe640 1 --2- 172.21.15.42:0/1080671885 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559dde9804d0 0x559dded40b60 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fdd18009b40 tx=0x7fdd18009570 comp rx=0 tx=0).ready entity=mon.2 client_cookie=92e99b3c01f8f595 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:13:44.445 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.441+0000 7fdd09ffb640 1 -- 172.21.15.42:0/1080671885 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd1801e040 con 0x559dde9804d0 2024-08-30T09:13:44.446 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.441+0000 7fdd09ffb640 1 -- 172.21.15.42:0/1080671885 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd180035f0 con 0x559dde9804d0 2024-08-30T09:13:44.446 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.443+0000 7fdd313c4740 1 -- 172.21.15.42:0/1080671885 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559dde9804d0 msgr2=0x559dded40b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:13:44.446 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.443+0000 7fdd313c4740 1 --2- 172.21.15.42:0/1080671885 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559dde9804d0 0x559dded40b60 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fdd18009b40 tx=0x7fdd18009570 comp rx=0 tx=0).stop 2024-08-30T09:13:44.447 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.445+0000 7fdd313c4740 1 -- 172.21.15.42:0/1080671885 shutdown_connections 2024-08-30T09:13:44.448 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.445+0000 7fdd313c4740 1 --2- 172.21.15.42:0/1080671885 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559dde9a11b0 0x559dded410a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:44.448 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.445+0000 7fdd313c4740 1 --2- 172.21.15.42:0/1080671885 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559dde9804d0 0x559dded40b60 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:44.448 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.445+0000 7fdd313c4740 1 --2- 172.21.15.42:0/1080671885 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559dde9782f0 0x559dded42700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:44.448 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.445+0000 7fdd313c4740 1 -- 172.21.15.42:0/1080671885 >> 172.21.15.42:0/1080671885 conn(0x559dde995300 msgr2=0x559dded4b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:13:44.458 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.456+0000 7fdd313c4740 1 -- 172.21.15.42:0/1080671885 shutdown_connections 2024-08-30T09:13:44.463 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.460+0000 7fdd313c4740 1 -- 172.21.15.42:0/1080671885 wait complete. 2024-08-30T09:13:44.463 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.460+0000 7fdd313c4740 1 -- start start 2024-08-30T09:13:44.463 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.460+0000 7fdd313c4740 1 Processor -- start 2024-08-30T09:13:44.472 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.468+0000 7fdd313c4740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559dde9782f0 0x559dded53d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:44.472 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.468+0000 7fdd313c4740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559dde9804d0 0x559dded54410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:44.472 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.468+0000 7fdd313c4740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559dde9a11b0 0x559dded54f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:44.472 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.469+0000 7fdd313c4740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fdd241acc20 con 0x559dde9782f0 2024-08-30T09:13:44.472 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.469+0000 7fdd313c4740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fdd241ae090 con 0x559dde9804d0 2024-08-30T09:13:44.472 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.469+0000 7fdd313c4740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fdd241ad2f0 con 0x559dde9a11b0 2024-08-30T09:13:44.472 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.469+0000 7fdd313c4740 10 client.0 _ll_register_callbacks cb 0x7fdd140046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:13:44.473 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.469+0000 7fdd23fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559dde9782f0 0x559dded53d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:13:44.473 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.469+0000 7fdd23fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559dde9782f0 0x559dded53d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:39268/0 (socket says 172.21.15.42:39268) 2024-08-30T09:13:44.473 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.469+0000 7fdd23fff640 1 -- 172.21.15.42:0/3309015358 learned_addr learned my addr 172.21.15.42:0/3309015358 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:13:44.474 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.470+0000 7fdd23fff640 1 -- 172.21.15.42:0/3309015358 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559dde9a11b0 msgr2=0x559dded54f80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-30T09:13:44.474 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.470+0000 7fdd23fff640 1 --2- 172.21.15.42:0/3309015358 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559dde9a11b0 0x559dded54f80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:44.474 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.470+0000 7fdd23fff640 1 -- 172.21.15.42:0/3309015358 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559dde9804d0 msgr2=0x559dded54410 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-30T09:13:44.474 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.470+0000 7fdd23fff640 1 --2- 172.21.15.42:0/3309015358 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559dde9804d0 0x559dded54410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:44.474 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.470+0000 7fdd23fff640 1 -- 172.21.15.42:0/3309015358 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559dded556e0 con 0x559dde9782f0 2024-08-30T09:13:44.478 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.476+0000 7fdd23fff640 1 --2- 172.21.15.42:0/3309015358 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559dde9782f0 0x559dded53d50 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fdd1400c6a0 tx=0x7fdd14005460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af6813ccf639de2 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:13:44.482 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.477+0000 7fdd037fe640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-08-30T09:13:44.484 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.480+0000 7fdd037fe640 1 -- 172.21.15.42:0/3309015358 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd140ca050 con 0x559dde9782f0 2024-08-30T09:13:44.484 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.480+0000 7fdd037fe640 1 -- 172.21.15.42:0/3309015358 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7fdd140aad60 con 0x559dde9782f0 2024-08-30T09:13:44.484 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.480+0000 7fdd313c4740 1 -- 172.21.15.42:0/3309015358 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdd180035f0 con 0x559dde9782f0 2024-08-30T09:13:44.490 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.485+0000 7fdd037fe640 1 -- 172.21.15.42:0/3309015358 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fdd140b0040 con 0x559dde9782f0 2024-08-30T09:13:44.491 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.489+0000 7fdd313c4740 10 client.17610 fetch_fsmap learned FSMap version 45 2024-08-30T09:13:44.491 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.489+0000 7fdd313c4740 1 -- 172.21.15.42:0/3309015358 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x559dded589e0 con 0x559dde9782f0 2024-08-30T09:13:44.493 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.490+0000 7fdd037fe640 1 -- 172.21.15.42:0/3309015358 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd180035f0 con 0x559dde9782f0 2024-08-30T09:13:44.493 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.490+0000 7fdd037fe640 1 -- 172.21.15.42:0/3309015358 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 45) ==== 8362+0+0 (secure 0 0 0) 0x7fdd140aa7d0 con 0x559dde9782f0 2024-08-30T09:13:44.493 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.490+0000 7fdd313c4740 10 client.17610 fetch_fsmap finished waiting for FSMap version 45 2024-08-30T09:13:44.493 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.490+0000 7fdd313c4740 10 client.17610 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-30T09:13:44.493 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.490+0000 7fdd313c4740 20 client.17610 populate_metadata read hostname 'smithi042' 2024-08-30T09:13:44.493 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.490+0000 7fdd313c4740 1 --2- 172.21.15.42:0/3309015358 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x559dde9351c0 0x559ddede4620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:44.493 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.490+0000 7fdd313c4740 4 client.17610 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-30T09:13:44.494 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.490+0000 7fdd313c4740 1 -- 172.21.15.42:0/3309015358 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559ddecb79f0 con 0x559dde9351c0 2024-08-30T09:13:44.494 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.491+0000 7fdd237fe640 1 --2- 172.21.15.42:0/3309015358 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x559dde9351c0 0x559ddede4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:13:44.495 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.493+0000 7fdd037fe640 10 client.17610 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:13:44.497 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.494+0000 7fdd237fe640 1 --2- 172.21.15.42:0/3309015358 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x559dde9351c0 0x559ddede4620 crc :-1 s=READY pgs=270 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=75b9c79f98dc139b server_cookie=f88d1e208f3927fb in_seq=0 out_seq=0 2024-08-30T09:13:44.587 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.583+0000 7fdd037fe640 1 -- 172.21.15.42:0/3309015358 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559ddecb79f0 con 0x559dde9351c0 2024-08-30T09:13:44.588 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.583+0000 7fdd037fe640 10 client.17610 handle_command_reply: tid=0 2024-08-30T09:13:44.588 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.583+0000 7fdd313c4740 1 client.17610 shutdown 2024-08-30T09:13:44.588 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.584+0000 7fdd313c4740 1 -- 172.21.15.42:0/3309015358 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559dde9782f0 msgr2=0x559dded53d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:13:44.588 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.584+0000 7fdd313c4740 1 --2- 172.21.15.42:0/3309015358 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559dde9782f0 0x559dded53d50 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fdd1400c6a0 tx=0x7fdd14005460 comp rx=0 tx=0).stop 2024-08-30T09:13:44.588 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.584+0000 7fdd313c4740 1 -- 172.21.15.42:0/3309015358 shutdown_connections 2024-08-30T09:13:44.588 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.584+0000 7fdd313c4740 1 --2- 172.21.15.42:0/3309015358 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x559dde9351c0 0x559ddede4620 crc :-1 s=READY pgs=270 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:44.588 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.584+0000 7fdd313c4740 1 --2- 172.21.15.42:0/3309015358 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559dde9a11b0 0x559dded54f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:44.588 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.584+0000 7fdd313c4740 1 --2- 172.21.15.42:0/3309015358 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559dde9804d0 0x559dded54410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:44.589 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.584+0000 7fdd313c4740 1 --2- 172.21.15.42:0/3309015358 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559dde9782f0 0x559dded53d50 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:44.589 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.584+0000 7fdd313c4740 1 -- 172.21.15.42:0/3309015358 >> 172.21.15.42:0/3309015358 conn(0x559dde995300 msgr2=0x559dded50580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:13:44.589 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.584+0000 7fdd037fe640 0 client.17610 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:13:44.589 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.584+0000 7fdd313c4740 1 -- 172.21.15.42:0/3309015358 shutdown_connections 2024-08-30T09:13:44.589 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.584+0000 7fdd313c4740 1 -- 172.21.15.42:0/3309015358 wait complete. 2024-08-30T09:13:44.589 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.584+0000 7fdd313c4740 20 client.17610 trim_cache size 0 max 16384 2024-08-30T09:13:44.589 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-08-30T09:13:44.590 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2024-08-30T09:13:44.590 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-08-30T09:13:44.590 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-08-30T09:13:44.590 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.585+0000 7fdd2a379640 1 -- 172.21.15.42:0/3708006518 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fdd0c0b4b30 msgr2=0x7fdd0c0b7040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:13:44.591 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.585+0000 7fdd2a379640 1 --2- 172.21.15.42:0/3708006518 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fdd0c0b4b30 0x7fdd0c0b7040 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fdd1c0094e0 tx=0x7fdd1c031af0 comp rx=0 tx=0).stop 2024-08-30T09:13:44.591 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.585+0000 7fdd2a379640 1 -- 172.21.15.42:0/3708006518 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdd24101830 msgr2=0x7fdd24103fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:13:44.591 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.585+0000 7fdd2a379640 1 --2- 172.21.15.42:0/3708006518 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdd24101830 0x7fdd24103fc0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fdd1400e9f0 tx=0x7fdd1400eeb0 comp rx=0 tx=0).stop 2024-08-30T09:13:44.591 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.585+0000 7fdd2a379640 1 -- 172.21.15.42:0/3708006518 shutdown_connections 2024-08-30T09:13:44.591 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.585+0000 7fdd2a379640 1 --2- 172.21.15.42:0/3708006518 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fdd0c0b4b30 0x7fdd0c0b7040 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:44.591 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.585+0000 7fdd2a379640 1 --2- 172.21.15.42:0/3708006518 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdd24106980 0x7fdd24102880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:44.591 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.585+0000 7fdd2a379640 1 --2- 172.21.15.42:0/3708006518 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdd241092e0 0x7fdd24102dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:44.591 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.585+0000 7fdd2a379640 1 --2- 172.21.15.42:0/3708006518 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdd24101830 0x7fdd24103fc0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:44.591 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.585+0000 7fdd2a379640 1 -- 172.21.15.42:0/3708006518 >> 172.21.15.42:0/3708006518 conn(0x7fdd240fd430 msgr2=0x7fdd24107750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:13:44.591 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.586+0000 7fdd2a379640 1 -- 172.21.15.42:0/3708006518 shutdown_connections 2024-08-30T09:13:44.592 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:44.586+0000 7fdd2a379640 1 -- 172.21.15.42:0/3708006518 wait complete. 2024-08-30T09:13:45.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:13:45 smithi167 ceph-mon[22063]: pgmap v762: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 4.9 MiB/s wr, 561 op/s 2024-08-30T09:13:45.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:13:45 smithi042 ceph-mon[18652]: pgmap v762: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 4.9 MiB/s wr, 561 op/s 2024-08-30T09:13:45.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:13:45 smithi190 ceph-mon[22001]: pgmap v762: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 4.9 MiB/s wr, 561 op/s 2024-08-30T09:13:45.953 INFO:teuthology.orchestra.run.smithi042.stderr:progress: 780.0 s, 508.6 tps, lat 194.965 ms stddev 247.474 2024-08-30T09:13:47.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:13:47 smithi167 ceph-mon[22063]: pgmap v763: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 4.4 MiB/s wr, 527 op/s 2024-08-30T09:13:47.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:13:47 smithi042 ceph-mon[18652]: pgmap v763: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 4.4 MiB/s wr, 527 op/s 2024-08-30T09:13:47.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:13:47 smithi190 ceph-mon[22001]: pgmap v763: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 4.4 MiB/s wr, 527 op/s 2024-08-30T09:13:48.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:13:48 smithi167 ceph-mon[22063]: scrub summary: idle 2024-08-30T09:13:48.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:13:48 smithi167 ceph-mon[22063]: scrub summary: idle 2024-08-30T09:13:48.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:13:48 smithi042 ceph-mon[18652]: scrub summary: idle 2024-08-30T09:13:48.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:13:48 smithi042 ceph-mon[18652]: scrub summary: idle 2024-08-30T09:13:48.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:13:48 smithi190 ceph-mon[22001]: scrub summary: idle 2024-08-30T09:13:48.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:13:48 smithi190 ceph-mon[22001]: scrub summary: idle 2024-08-30T09:13:49.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:13:49 smithi167 ceph-mon[22063]: pgmap v764: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 3.9 MiB/s wr, 486 op/s 2024-08-30T09:13:49.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:13:49 smithi167 ceph-mon[22063]: scrub summary: idle 2024-08-30T09:13:49.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:13:49 smithi167 ceph-mon[22063]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-30T09:13:49.385 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:13:49 smithi042 ceph-mon[18652]: pgmap v764: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 3.9 MiB/s wr, 486 op/s 2024-08-30T09:13:49.385 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:13:49 smithi042 ceph-mon[18652]: scrub summary: idle 2024-08-30T09:13:49.385 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:13:49 smithi042 ceph-mon[18652]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-30T09:13:49.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:13:49 smithi190 ceph-mon[22001]: pgmap v764: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 3.9 MiB/s wr, 486 op/s 2024-08-30T09:13:49.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:13:49 smithi190 ceph-mon[22001]: scrub summary: idle 2024-08-30T09:13:49.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:13:49 smithi190 ceph-mon[22001]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-30T09:13:50.953 INFO:teuthology.orchestra.run.smithi042.stderr:progress: 785.0 s, 478.4 tps, lat 212.405 ms stddev 243.856 2024-08-30T09:13:51.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:13:51 smithi167 ceph-mon[22063]: pgmap v765: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 5.3 MiB/s wr, 643 op/s 2024-08-30T09:13:51.382 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:13:51 smithi042 ceph-mon[18652]: pgmap v765: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 5.3 MiB/s wr, 643 op/s 2024-08-30T09:13:51.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:13:51 smithi190 ceph-mon[22001]: pgmap v765: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 5.3 MiB/s wr, 643 op/s 2024-08-30T09:13:53.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:13:53 smithi167 ceph-mon[22063]: pgmap v766: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 4.3 MiB/s wr, 522 op/s 2024-08-30T09:13:53.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:13:53 smithi042 ceph-mon[18652]: pgmap v766: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 4.3 MiB/s wr, 522 op/s 2024-08-30T09:13:53.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:13:53 smithi190 ceph-mon[22001]: pgmap v766: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 4.3 MiB/s wr, 522 op/s 2024-08-30T09:13:54.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:13:54 smithi167 ceph-mon[22063]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-30T09:13:54.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:13:54 smithi167 ceph-mon[22063]: pgmap v767: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 5.2 MiB/s wr, 634 op/s 2024-08-30T09:13:54.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:13:54 smithi042 ceph-mon[18652]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-30T09:13:54.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:13:54 smithi042 ceph-mon[18652]: pgmap v767: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 5.2 MiB/s wr, 634 op/s 2024-08-30T09:13:54.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:13:54 smithi190 ceph-mon[22001]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-30T09:13:54.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:13:54 smithi190 ceph-mon[22001]: pgmap v767: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 5.2 MiB/s wr, 634 op/s 2024-08-30T09:13:55.953 INFO:teuthology.orchestra.run.smithi042.stderr:progress: 790.0 s, 547.0 tps, lat 178.688 ms stddev 206.914 2024-08-30T09:13:56.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:13:56 smithi042 ceph-mon[18652]: pgmap v768: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 4.4 MiB/s wr, 494 op/s 2024-08-30T09:13:56.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:13:56 smithi190 ceph-mon[22001]: pgmap v768: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 4.4 MiB/s wr, 494 op/s 2024-08-30T09:13:56.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:13:56 smithi167 ceph-mon[22063]: pgmap v768: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 4.4 MiB/s wr, 494 op/s 2024-08-30T09:13:57.751 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.745+0000 7f5c00ee8640 1 -- 172.21.15.42:0/2360271181 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5bf40a5be0 msgr2=0x7f5bf40a5ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:13:57.752 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.745+0000 7f5c00ee8640 1 --2- 172.21.15.42:0/2360271181 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5bf40a5be0 0x7f5bf40a5ff0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f5be8009860 tx=0x7f5be802a340 comp rx=0 tx=0).stop 2024-08-30T09:13:57.752 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.747+0000 7f5c00ee8640 1 -- 172.21.15.42:0/2360271181 shutdown_connections 2024-08-30T09:13:57.752 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.747+0000 7f5c00ee8640 1 --2- 172.21.15.42:0/2360271181 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5bf40a6580 0x7f5bf40aaef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:57.752 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.747+0000 7f5c00ee8640 1 --2- 172.21.15.42:0/2360271181 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5bf40a5be0 0x7f5bf40a5ff0 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:57.752 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.747+0000 7f5c00ee8640 1 --2- 172.21.15.42:0/2360271181 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5bf40a4cf0 0x7f5bf40a5160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:57.752 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.747+0000 7f5c00ee8640 1 -- 172.21.15.42:0/2360271181 >> 172.21.15.42:0/2360271181 conn(0x7f5bf40a0480 msgr2=0x7f5bf40a2930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:13:57.758 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.754+0000 7f5c00ee8640 1 -- 172.21.15.42:0/2360271181 shutdown_connections 2024-08-30T09:13:57.761 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.755+0000 7f5c00ee8640 1 -- 172.21.15.42:0/2360271181 wait complete. 2024-08-30T09:13:57.761 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.755+0000 7f5c00ee8640 1 Processor -- start 2024-08-30T09:13:57.761 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.756+0000 7f5c00ee8640 1 -- start start 2024-08-30T09:13:57.761 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.756+0000 7f5c00ee8640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5bf40a4cf0 0x7f5bf413b420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:57.761 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.756+0000 7f5c00ee8640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5bf40a5be0 0x7f5bf413ba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:57.761 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.756+0000 7f5c00ee8640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5bf40a6580 0x7f5bf413c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:57.761 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.756+0000 7f5c00ee8640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f5bf40ae860 con 0x7f5bf40a6580 2024-08-30T09:13:57.761 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.756+0000 7f5c00ee8640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f5bf40ae6e0 con 0x7f5bf40a4cf0 2024-08-30T09:13:57.761 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.756+0000 7f5c00ee8640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f5bf40ae9e0 con 0x7f5bf40a5be0 2024-08-30T09:13:57.761 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.756+0000 7f5bfb7fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5bf40a4cf0 0x7f5bf413b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:13:57.762 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.756+0000 7f5bfb7fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5bf40a4cf0 0x7f5bf413b420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:53304/0 (socket says 172.21.15.42:53304) 2024-08-30T09:13:57.762 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.756+0000 7f5bfb7fe640 1 -- 172.21.15.42:0/4027601394 learned_addr learned my addr 172.21.15.42:0/4027601394 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:13:57.762 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.757+0000 7f5bfaffd640 1 --2- 172.21.15.42:0/4027601394 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5bf40a5be0 0x7f5bf413ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:13:57.762 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.757+0000 7f5bfbfff640 1 --2- 172.21.15.42:0/4027601394 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5bf40a6580 0x7f5bf413c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:13:57.762 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.759+0000 7f5bfbfff640 1 -- 172.21.15.42:0/4027601394 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5bf40a5be0 msgr2=0x7f5bf413ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:13:57.762 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.759+0000 7f5bfbfff640 1 --2- 172.21.15.42:0/4027601394 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5bf40a5be0 0x7f5bf413ba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:57.762 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.759+0000 7f5bfbfff640 1 -- 172.21.15.42:0/4027601394 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5bf40a4cf0 msgr2=0x7f5bf413b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:13:57.762 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.759+0000 7f5bfbfff640 1 --2- 172.21.15.42:0/4027601394 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5bf40a4cf0 0x7f5bf413b420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:57.762 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.759+0000 7f5bfbfff640 1 -- 172.21.15.42:0/4027601394 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bf413ce60 con 0x7f5bf40a6580 2024-08-30T09:13:57.765 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.759+0000 7f5bfbfff640 1 --2- 172.21.15.42:0/4027601394 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5bf40a6580 0x7f5bf413c640 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f5bfc069430 tx=0x7f5bfc0726a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:13:57.767 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.764+0000 7f5bf8ff9640 1 -- 172.21.15.42:0/4027601394 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bfc07a070 con 0x7f5bf40a6580 2024-08-30T09:13:57.767 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.764+0000 7f5bf8ff9640 1 -- 172.21.15.42:0/4027601394 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5bfc076ae0 con 0x7f5bf40a6580 2024-08-30T09:13:57.767 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.764+0000 7f5bf8ff9640 1 -- 172.21.15.42:0/4027601394 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bfc06b630 con 0x7f5bf40a6580 2024-08-30T09:13:57.767 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.764+0000 7f5c00ee8640 1 -- 172.21.15.42:0/4027601394 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5bf413d150 con 0x7f5bf40a6580 2024-08-30T09:13:57.767 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.764+0000 7f5c00ee8640 1 -- 172.21.15.42:0/4027601394 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5bf400bbb0 con 0x7f5bf40a6580 2024-08-30T09:13:57.773 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.770+0000 7f5bf8ff9640 1 -- 172.21.15.42:0/4027601394 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7f5bfc07d070 con 0x7f5bf40a6580 2024-08-30T09:13:57.779 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.775+0000 7f5c090b1740 1 Processor -- start 2024-08-30T09:13:57.779 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.775+0000 7f5c090b1740 1 -- start start 2024-08-30T09:13:57.779 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.776+0000 7f5c090b1740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5635ca04a1b0 0x5635ca3f0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:57.779 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.776+0000 7f5c090b1740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5635ca0212f0 0x5635ca3f07d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:57.779 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.776+0000 7f5c090b1740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5635ca0294d0 0x5635ca3f1320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:57.780 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.776+0000 7f5c090b1740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x5635ca363000 con 0x5635ca0294d0 2024-08-30T09:13:57.780 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.776+0000 7f5c090b1740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5635ca352590 con 0x5635ca0212f0 2024-08-30T09:13:57.780 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.776+0000 7f5c090b1740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5635ca356540 con 0x5635ca04a1b0 2024-08-30T09:13:57.794 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.780+0000 7f5bfb7fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5635ca0212f0 0x5635ca3f07d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:13:57.794 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.780+0000 7f5bfb7fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5635ca0212f0 0x5635ca3f07d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:53314/0 (socket says 172.21.15.42:53314) 2024-08-30T09:13:57.794 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.781+0000 7f5bfb7fe640 1 -- 172.21.15.42:0/47767001 learned_addr learned my addr 172.21.15.42:0/47767001 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:13:57.794 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.781+0000 7f5bfaffd640 1 --2- 172.21.15.42:0/47767001 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5635ca04a1b0 0x5635ca3f0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:13:57.794 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.784+0000 7f5bf8ff9640 1 --2- 172.21.15.42:0/4027601394 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f5bec0b4d40 0x7f5bec0b7250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:57.794 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.785+0000 7f5bfb7fe640 1 -- 172.21.15.42:0/47767001 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5635ca04a1b0 msgr2=0x5635ca3f0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:13:57.794 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.785+0000 7f5bfb7fe640 1 --2- 172.21.15.42:0/47767001 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5635ca04a1b0 0x5635ca3f0160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:57.794 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.785+0000 7f5bfb7fe640 1 -- 172.21.15.42:0/47767001 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5635ca0294d0 msgr2=0x5635ca3f1320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:13:57.794 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.785+0000 7f5bfb7fe640 1 --2- 172.21.15.42:0/47767001 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5635ca0294d0 0x5635ca3f1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:57.795 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.785+0000 7f5bfb7fe640 1 -- 172.21.15.42:0/47767001 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5635ca0b5790 con 0x5635ca0212f0 2024-08-30T09:13:57.795 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.785+0000 7f5bfb7fe640 1 --2- 172.21.15.42:0/47767001 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5635ca0212f0 0x5635ca3f07d0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f5bf001ad60 tx=0x7f5bf00227c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3ce9d942c5c6d826 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:13:57.795 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.785+0000 7f5bf8ff9640 1 -- 172.21.15.42:0/4027601394 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f5bfc14aa10 con 0x7f5bf40a6580 2024-08-30T09:13:57.795 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.786+0000 7f5bfbfff640 1 --2- 172.21.15.42:0/4027601394 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f5bec0b4d40 0x7f5bec0b7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:13:57.795 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.787+0000 7f5bfbfff640 1 --2- 172.21.15.42:0/4027601394 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f5bec0b4d40 0x7f5bec0b7250 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f5bfc066a10 tx=0x7f5bfc0693c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:13:57.795 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.786+0000 7f5be5ffb640 1 -- 172.21.15.42:0/47767001 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bf002f070 con 0x5635ca0212f0 2024-08-30T09:13:57.795 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.787+0000 7f5be5ffb640 1 -- 172.21.15.42:0/47767001 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5bf0020b10 con 0x5635ca0212f0 2024-08-30T09:13:57.795 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.787+0000 7f5bfaffd640 1 --2- 172.21.15.42:0/47767001 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5635ca04a1b0 0x5635ca3f0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-30T09:13:57.795 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.787+0000 7f5be5ffb640 1 -- 172.21.15.42:0/47767001 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bf002a9c0 con 0x5635ca0212f0 2024-08-30T09:13:57.795 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.788+0000 7f5c090b1740 1 -- 172.21.15.42:0/47767001 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5635ca0212f0 msgr2=0x5635ca3f07d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:13:57.795 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.788+0000 7f5c090b1740 1 --2- 172.21.15.42:0/47767001 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5635ca0212f0 0x5635ca3f07d0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f5bf001ad60 tx=0x7f5bf00227c0 comp rx=0 tx=0).stop 2024-08-30T09:13:57.797 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.794+0000 7f5c090b1740 1 -- 172.21.15.42:0/47767001 shutdown_connections 2024-08-30T09:13:57.797 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.794+0000 7f5c090b1740 1 --2- 172.21.15.42:0/47767001 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5635ca0294d0 0x5635ca3f1320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:57.797 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.794+0000 7f5c090b1740 1 --2- 172.21.15.42:0/47767001 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5635ca0212f0 0x5635ca3f07d0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:57.797 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.794+0000 7f5c090b1740 1 --2- 172.21.15.42:0/47767001 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5635ca04a1b0 0x5635ca3f0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:57.797 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.794+0000 7f5c090b1740 1 -- 172.21.15.42:0/47767001 >> 172.21.15.42:0/47767001 conn(0x5635ca03e300 msgr2=0x5635ca3ebbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:13:57.802 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.796+0000 7f5c090b1740 1 -- 172.21.15.42:0/47767001 shutdown_connections 2024-08-30T09:13:57.809 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.801+0000 7f5c090b1740 1 -- 172.21.15.42:0/47767001 wait complete. 2024-08-30T09:13:57.809 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.801+0000 7f5c090b1740 1 -- start start 2024-08-30T09:13:57.809 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.801+0000 7f5c090b1740 1 Processor -- start 2024-08-30T09:13:57.810 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.804+0000 7f5c090b1740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5635ca0212f0 0x5635ca3f9be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:57.810 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.804+0000 7f5c090b1740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5635ca0294d0 0x5635ca3fa2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:57.810 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.804+0000 7f5c090b1740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5635ca04a1b0 0x5635ca3fae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:57.810 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.804+0000 7f5c090b1740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x5635ca28bab0 con 0x5635ca04a1b0 2024-08-30T09:13:57.810 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.804+0000 7f5c090b1740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5635ca2d9680 con 0x5635ca0212f0 2024-08-30T09:13:57.810 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.804+0000 7f5c090b1740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5635ca3f1e40 con 0x5635ca0294d0 2024-08-30T09:13:57.810 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.804+0000 7f5c090b1740 10 client.0 _ll_register_callbacks cb 0x5635ca061a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:13:57.810 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.804+0000 7f5bfaffd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5635ca0212f0 0x5635ca3f9be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:13:57.810 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.804+0000 7f5bfaffd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5635ca0212f0 0x5635ca3f9be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:53324/0 (socket says 172.21.15.42:53324) 2024-08-30T09:13:57.810 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.804+0000 7f5bfaffd640 1 -- 172.21.15.42:0/3012637950 learned_addr learned my addr 172.21.15.42:0/3012637950 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:13:57.810 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.805+0000 7f5bfb7fe640 1 --2- 172.21.15.42:0/3012637950 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5635ca0294d0 0x5635ca3fa2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:13:57.810 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.805+0000 7f5bfaffd640 1 -- 172.21.15.42:0/3012637950 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5635ca0294d0 msgr2=0x5635ca3fa2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:13:57.810 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.805+0000 7f5bfaffd640 1 --2- 172.21.15.42:0/3012637950 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5635ca0294d0 0x5635ca3fa2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:57.810 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.805+0000 7f5bfaffd640 1 -- 172.21.15.42:0/3012637950 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5635ca04a1b0 msgr2=0x5635ca3fae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:13:57.811 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.805+0000 7f5bfaffd640 1 --2- 172.21.15.42:0/3012637950 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5635ca04a1b0 0x5635ca3fae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:57.811 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.805+0000 7f5bfaffd640 1 -- 172.21.15.42:0/3012637950 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5635ca12e620 con 0x5635ca0212f0 2024-08-30T09:13:57.811 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.805+0000 7f5bfaffd640 1 --2- 172.21.15.42:0/3012637950 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5635ca0212f0 0x5635ca3f9be0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f5be8009780 tx=0x7f5be802cee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3a39c615651101bd server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:13:57.811 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.806+0000 7f5bdeffd640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-30T09:13:57.811 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.806+0000 7f5bdeffd640 1 -- 172.21.15.42:0/3012637950 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5be8042020 con 0x5635ca0212f0 2024-08-30T09:13:57.811 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.806+0000 7f5c090b1740 1 -- 172.21.15.42:0/3012637950 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5bf0020b10 con 0x5635ca0212f0 2024-08-30T09:13:57.811 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.806+0000 7f5bdeffd640 1 -- 172.21.15.42:0/3012637950 <== mon.2 v2:172.21.15.167:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f5be8030550 con 0x5635ca0212f0 2024-08-30T09:13:57.811 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.806+0000 7f5bdeffd640 1 -- 172.21.15.42:0/3012637950 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f5be804d050 con 0x5635ca0212f0 2024-08-30T09:13:57.811 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.807+0000 7f5c090b1740 10 client.27467 fetch_fsmap learned FSMap version 47 2024-08-30T09:13:57.811 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.807+0000 7f5c090b1740 1 -- 172.21.15.42:0/3012637950 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x5635ca403d70 con 0x5635ca0212f0 2024-08-30T09:13:57.814 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.811+0000 7f5bdeffd640 1 -- 172.21.15.42:0/3012637950 <== mon.2 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5bf0020b10 con 0x5635ca0212f0 2024-08-30T09:13:57.814 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.811+0000 7f5bdeffd640 1 -- 172.21.15.42:0/3012637950 <== mon.2 v2:172.21.15.167:3300/0 5 ==== fsmap(e 47) ==== 8370+0+0 (secure 0 0 0) 0x7f5be80039a0 con 0x5635ca0212f0 2024-08-30T09:13:57.814 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.811+0000 7f5c090b1740 10 client.27467 fetch_fsmap finished waiting for FSMap version 47 2024-08-30T09:13:57.814 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.811+0000 7f5c090b1740 10 client.27467 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-30T09:13:57.814 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.811+0000 7f5c090b1740 20 client.27467 populate_metadata read hostname 'smithi042' 2024-08-30T09:13:57.814 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.811+0000 7f5c090b1740 1 --2- 172.21.15.42:0/3012637950 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x5635ca008d80 0x5635ca4079c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:57.814 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.811+0000 7f5c090b1740 4 client.27467 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-30T09:13:57.815 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.811+0000 7f5c090b1740 1 -- 172.21.15.42:0/3012637950 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5635ca364f50 con 0x5635ca008d80 2024-08-30T09:13:57.815 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.811+0000 7f5bfb7fe640 1 --2- 172.21.15.42:0/3012637950 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x5635ca008d80 0x5635ca4079c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:13:57.825 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.820+0000 7f5bfb7fe640 1 --2- 172.21.15.42:0/3012637950 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x5635ca008d80 0x5635ca4079c0 crc :-1 s=READY pgs=271 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f87d4ca9f28cc45 server_cookie=c8d89631637a1d51 in_seq=0 out_seq=0 2024-08-30T09:13:57.825 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.820+0000 7f5bdeffd640 10 client.27467 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:13:57.825 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.821+0000 7f5bdeffd640 1 -- 172.21.15.42:0/3012637950 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5635ca364f50 con 0x5635ca008d80 2024-08-30T09:13:57.825 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.821+0000 7f5bdeffd640 10 client.27467 handle_command_reply: tid=0 2024-08-30T09:13:57.825 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.821+0000 7f5c090b1740 1 client.27467 shutdown 2024-08-30T09:13:57.829 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.826+0000 7f5c090b1740 1 -- 172.21.15.42:0/3012637950 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5635ca0212f0 msgr2=0x5635ca3f9be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:13:57.829 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.826+0000 7f5c090b1740 1 --2- 172.21.15.42:0/3012637950 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5635ca0212f0 0x5635ca3f9be0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f5be8009780 tx=0x7f5be802cee0 comp rx=0 tx=0).stop 2024-08-30T09:13:57.839 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.834+0000 7f5c090b1740 1 -- 172.21.15.42:0/3012637950 shutdown_connections 2024-08-30T09:13:57.839 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.834+0000 7f5c090b1740 1 --2- 172.21.15.42:0/3012637950 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x5635ca008d80 0x5635ca4079c0 crc :-1 s=READY pgs=271 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:57.839 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.834+0000 7f5c090b1740 1 --2- 172.21.15.42:0/3012637950 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5635ca0294d0 0x5635ca3fa2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:57.839 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.834+0000 7f5c090b1740 1 --2- 172.21.15.42:0/3012637950 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5635ca04a1b0 0x5635ca3fae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:57.839 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.834+0000 7f5c090b1740 1 --2- 172.21.15.42:0/3012637950 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5635ca0212f0 0x5635ca3f9be0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:57.839 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.834+0000 7f5c090b1740 1 -- 172.21.15.42:0/3012637950 >> 172.21.15.42:0/3012637950 conn(0x5635ca03e300 msgr2=0x5635ca3ea150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:13:57.839 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.834+0000 7f5bdeffd640 0 client.27467 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:13:57.842 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.838+0000 7f5c090b1740 1 -- 172.21.15.42:0/3012637950 shutdown_connections 2024-08-30T09:13:57.842 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.839+0000 7f5c090b1740 1 -- 172.21.15.42:0/3012637950 wait complete. 2024-08-30T09:13:57.842 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.839+0000 7f5c090b1740 20 client.27467 trim_cache size 0 max 16384 2024-08-30T09:13:57.876 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.870+0000 7f5c090b1740 1 Processor -- start 2024-08-30T09:13:57.876 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.870+0000 7f5c090b1740 1 -- start start 2024-08-30T09:13:57.876 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.870+0000 7f5c090b1740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5635ca0212f0 0x5635ca3f8360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:57.876 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.871+0000 7f5c090b1740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5635ca0294d0 0x5635ca3f3d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:57.876 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.871+0000 7f5c090b1740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5635ca04a1b0 0x5635ca3f4270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:57.876 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.871+0000 7f5bfaffd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5635ca0212f0 0x5635ca3f8360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:13:57.876 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.871+0000 7f5bfaffd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5635ca0212f0 0x5635ca3f8360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:34768/0 (socket says 172.21.15.42:34768) 2024-08-30T09:13:57.876 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.871+0000 7f5bfaffd640 1 -- 172.21.15.42:0/1240165693 learned_addr learned my addr 172.21.15.42:0/1240165693 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:13:57.876 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.871+0000 7f5c090b1740 1 -- 172.21.15.42:0/1240165693 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f5be0012600 con 0x5635ca0294d0 2024-08-30T09:13:57.876 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.871+0000 7f5c090b1740 1 -- 172.21.15.42:0/1240165693 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f5be000fe70 con 0x5635ca04a1b0 2024-08-30T09:13:57.877 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.871+0000 7f5c090b1740 1 -- 172.21.15.42:0/1240165693 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f5be000dcc0 con 0x5635ca0212f0 2024-08-30T09:13:57.877 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.873+0000 7f5bfaffd640 1 --2- 172.21.15.42:0/1240165693 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5635ca04a1b0 0x5635ca3f4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:13:57.877 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.874+0000 7f5bfaffd640 1 -- 172.21.15.42:0/1240165693 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5635ca04a1b0 msgr2=0x5635ca3f4270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:13:57.877 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.874+0000 7f5bfaffd640 1 --2- 172.21.15.42:0/1240165693 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5635ca04a1b0 0x5635ca3f4270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:57.877 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.874+0000 7f5bfaffd640 1 -- 172.21.15.42:0/1240165693 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5635ca0294d0 msgr2=0x5635ca3f3d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:13:57.877 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.874+0000 7f5bfaffd640 1 --2- 172.21.15.42:0/1240165693 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5635ca0294d0 0x5635ca3f3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:57.877 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.874+0000 7f5bfaffd640 1 -- 172.21.15.42:0/1240165693 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5635ca3fa920 con 0x5635ca0212f0 2024-08-30T09:13:57.877 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.874+0000 7f5bfaffd640 1 --2- 172.21.15.42:0/1240165693 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5635ca0212f0 0x5635ca3f8360 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f5be8009250 tx=0x7f5be802a340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=357286cf0e179ef5 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:13:57.881 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.877+0000 7f5be57fa640 1 -- 172.21.15.42:0/1240165693 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5be8042040 con 0x5635ca0212f0 2024-08-30T09:13:57.882 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.877+0000 7f5be57fa640 1 -- 172.21.15.42:0/1240165693 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5be8003f90 con 0x5635ca0212f0 2024-08-30T09:13:57.882 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.877+0000 7f5be57fa640 1 -- 172.21.15.42:0/1240165693 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5be8037040 con 0x5635ca0212f0 2024-08-30T09:13:57.882 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.879+0000 7f5c090b1740 1 -- 172.21.15.42:0/1240165693 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5635ca0212f0 msgr2=0x5635ca3f8360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:13:57.882 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.879+0000 7f5c090b1740 1 --2- 172.21.15.42:0/1240165693 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5635ca0212f0 0x5635ca3f8360 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f5be8009250 tx=0x7f5be802a340 comp rx=0 tx=0).stop 2024-08-30T09:13:57.883 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.881+0000 7f5c090b1740 1 -- 172.21.15.42:0/1240165693 shutdown_connections 2024-08-30T09:13:57.884 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.881+0000 7f5c090b1740 1 --2- 172.21.15.42:0/1240165693 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5635ca04a1b0 0x5635ca3f4270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:57.884 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.881+0000 7f5c090b1740 1 --2- 172.21.15.42:0/1240165693 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5635ca0294d0 0x5635ca3f3d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:57.884 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.881+0000 7f5c090b1740 1 --2- 172.21.15.42:0/1240165693 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5635ca0212f0 0x5635ca3f8360 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:57.884 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.881+0000 7f5c090b1740 1 -- 172.21.15.42:0/1240165693 >> 172.21.15.42:0/1240165693 conn(0x5635ca03e300 msgr2=0x5635ca3f4b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:13:57.889 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.886+0000 7f5c090b1740 1 -- 172.21.15.42:0/1240165693 shutdown_connections 2024-08-30T09:13:57.889 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.886+0000 7f5c090b1740 1 -- 172.21.15.42:0/1240165693 wait complete. 2024-08-30T09:13:57.890 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.887+0000 7f5c090b1740 1 -- start start 2024-08-30T09:13:57.890 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.887+0000 7f5c090b1740 1 Processor -- start 2024-08-30T09:13:57.891 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.887+0000 7f5c090b1740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5635ca0212f0 0x5635ca401270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:57.891 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.887+0000 7f5c090b1740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5635ca0294d0 0x5635ca3fff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:57.891 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.887+0000 7f5c090b1740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5635ca04a1b0 0x5635ca400470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:57.891 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.887+0000 7f5c090b1740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f5be000e330 con 0x5635ca0212f0 2024-08-30T09:13:57.891 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.888+0000 7f5c090b1740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f5be000f7a0 con 0x5635ca0294d0 2024-08-30T09:13:57.891 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.888+0000 7f5c090b1740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f5be000ea00 con 0x5635ca04a1b0 2024-08-30T09:13:57.891 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.888+0000 7f5c090b1740 10 client.0 _ll_register_callbacks cb 0x7f5bfc0645b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:13:57.891 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.888+0000 7f5bfb7fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5635ca0294d0 0x5635ca3fff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:13:57.891 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.888+0000 7f5bfb7fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5635ca0294d0 0x5635ca3fff30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:53348/0 (socket says 172.21.15.42:53348) 2024-08-30T09:13:57.891 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.888+0000 7f5bfb7fe640 1 -- 172.21.15.42:0/432610176 learned_addr learned my addr 172.21.15.42:0/432610176 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:13:57.891 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.888+0000 7f5bfaffd640 1 --2- 172.21.15.42:0/432610176 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5635ca04a1b0 0x5635ca400470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:13:57.892 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.888+0000 7f5bfb7fe640 1 -- 172.21.15.42:0/432610176 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5635ca04a1b0 msgr2=0x5635ca400470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:13:57.892 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.888+0000 7f5bfb7fe640 1 --2- 172.21.15.42:0/432610176 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5635ca04a1b0 0x5635ca400470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:57.892 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.888+0000 7f5bfb7fe640 1 -- 172.21.15.42:0/432610176 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5635ca0212f0 msgr2=0x5635ca401270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:13:57.892 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.888+0000 7f5bfb7fe640 1 --2- 172.21.15.42:0/432610176 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5635ca0212f0 0x5635ca401270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:57.892 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.888+0000 7f5bfb7fe640 1 -- 172.21.15.42:0/432610176 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5635ca4009b0 con 0x5635ca0294d0 2024-08-30T09:13:57.892 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.889+0000 7f5bfb7fe640 1 --2- 172.21.15.42:0/432610176 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5635ca0294d0 0x5635ca3fff30 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f5bf001af50 tx=0x7f5bf001a500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c97b8bd17645fe38 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:13:57.892 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.889+0000 7f5bdeffd640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-30T09:13:57.898 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.895+0000 7f5bdeffd640 1 -- 172.21.15.42:0/432610176 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bf002f040 con 0x5635ca0294d0 2024-08-30T09:13:57.898 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.895+0000 7f5c090b1740 1 -- 172.21.15.42:0/432610176 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5be8003f90 con 0x5635ca0294d0 2024-08-30T09:13:57.898 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.895+0000 7f5bdeffd640 1 -- 172.21.15.42:0/432610176 <== mon.2 v2:172.21.15.167:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f5bf0019040 con 0x5635ca0294d0 2024-08-30T09:13:57.906 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.904+0000 7f5bdeffd640 1 -- 172.21.15.42:0/432610176 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f5bf002a410 con 0x5635ca0294d0 2024-08-30T09:13:57.908 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.905+0000 7f5c090b1740 10 client.27473 fetch_fsmap learned FSMap version 47 2024-08-30T09:13:57.908 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.905+0000 7f5c090b1740 1 -- 172.21.15.42:0/432610176 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x5635ca3ff870 con 0x5635ca0294d0 2024-08-30T09:13:57.909 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.906+0000 7f5bdeffd640 1 -- 172.21.15.42:0/432610176 <== mon.2 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5be8003f90 con 0x5635ca0294d0 2024-08-30T09:13:57.910 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.907+0000 7f5bdeffd640 1 -- 172.21.15.42:0/432610176 <== mon.2 v2:172.21.15.167:3300/0 5 ==== fsmap(e 47) ==== 8370+0+0 (secure 0 0 0) 0x7f5bf003c070 con 0x5635ca0294d0 2024-08-30T09:13:57.913 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.911+0000 7f5c090b1740 10 client.27473 fetch_fsmap finished waiting for FSMap version 47 2024-08-30T09:13:57.913 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.911+0000 7f5c090b1740 10 client.27473 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-30T09:13:57.914 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.911+0000 7f5c090b1740 20 client.27473 populate_metadata read hostname 'smithi042' 2024-08-30T09:13:57.914 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.911+0000 7f5c090b1740 1 --2- 172.21.15.42:0/432610176 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x5635c9fde1c0 0x5635ca4749c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:13:57.914 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.911+0000 7f5c090b1740 4 client.27473 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-30T09:13:57.914 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.911+0000 7f5c090b1740 1 -- 172.21.15.42:0/432610176 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5635ca3609f0 con 0x5635c9fde1c0 2024-08-30T09:13:57.918 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.915+0000 7f5bfaffd640 1 --2- 172.21.15.42:0/432610176 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x5635c9fde1c0 0x5635ca4749c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:13:57.921 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.918+0000 7f5bfaffd640 1 --2- 172.21.15.42:0/432610176 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x5635c9fde1c0 0x5635ca4749c0 crc :-1 s=READY pgs=272 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ecd0bfaf4671baa2 server_cookie=36cccfd4258e8c70 in_seq=0 out_seq=0 2024-08-30T09:13:57.921 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.919+0000 7f5bdeffd640 10 client.27473 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:13:57.938 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.935+0000 7f5bdeffd640 1 -- 172.21.15.42:0/432610176 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5635ca3609f0 con 0x5635c9fde1c0 2024-08-30T09:13:57.938 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.935+0000 7f5bdeffd640 10 client.27473 handle_command_reply: tid=0 2024-08-30T09:13:57.938 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.935+0000 7f5c090b1740 1 client.27473 shutdown 2024-08-30T09:13:57.939 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.935+0000 7f5c090b1740 1 -- 172.21.15.42:0/432610176 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5635ca0294d0 msgr2=0x5635ca3fff30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:13:57.939 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.935+0000 7f5c090b1740 1 --2- 172.21.15.42:0/432610176 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5635ca0294d0 0x5635ca3fff30 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f5bf001af50 tx=0x7f5bf001a500 comp rx=0 tx=0).stop 2024-08-30T09:13:57.939 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.935+0000 7f5c090b1740 1 -- 172.21.15.42:0/432610176 shutdown_connections 2024-08-30T09:13:57.939 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.935+0000 7f5c090b1740 1 --2- 172.21.15.42:0/432610176 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x5635c9fde1c0 0x5635ca4749c0 crc :-1 s=READY pgs=272 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:57.939 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.935+0000 7f5c090b1740 1 --2- 172.21.15.42:0/432610176 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5635ca04a1b0 0x5635ca400470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:57.939 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.935+0000 7f5c090b1740 1 --2- 172.21.15.42:0/432610176 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5635ca0294d0 0x5635ca3fff30 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:57.939 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.935+0000 7f5c090b1740 1 --2- 172.21.15.42:0/432610176 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5635ca0212f0 0x5635ca401270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:57.939 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.935+0000 7f5c090b1740 1 -- 172.21.15.42:0/432610176 >> 172.21.15.42:0/432610176 conn(0x5635ca03e300 msgr2=0x5635ca3f3d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:13:57.939 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.935+0000 7f5bdeffd640 0 client.27473 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:13:57.939 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.935+0000 7f5c090b1740 1 -- 172.21.15.42:0/432610176 shutdown_connections 2024-08-30T09:13:57.939 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.935+0000 7f5c090b1740 1 -- 172.21.15.42:0/432610176 wait complete. 2024-08-30T09:13:57.939 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.936+0000 7f5c090b1740 20 client.27473 trim_cache size 0 max 16384 2024-08-30T09:13:57.940 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-08-30T09:13:57.940 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2024-08-30T09:13:57.940 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-08-30T09:13:57.940 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-08-30T09:13:57.941 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.936+0000 7f5c00ee8640 1 -- 172.21.15.42:0/4027601394 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f5bec0b4d40 msgr2=0x7f5bec0b7250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:13:57.941 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.936+0000 7f5c00ee8640 1 --2- 172.21.15.42:0/4027601394 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f5bec0b4d40 0x7f5bec0b7250 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f5bfc066a10 tx=0x7f5bfc0693c0 comp rx=0 tx=0).stop 2024-08-30T09:13:57.941 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.936+0000 7f5c00ee8640 1 -- 172.21.15.42:0/4027601394 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5bf40a6580 msgr2=0x7f5bf413c640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:13:57.941 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.936+0000 7f5c00ee8640 1 --2- 172.21.15.42:0/4027601394 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5bf40a6580 0x7f5bf413c640 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f5bfc069430 tx=0x7f5bfc0726a0 comp rx=0 tx=0).stop 2024-08-30T09:13:57.941 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.937+0000 7f5c00ee8640 1 -- 172.21.15.42:0/4027601394 shutdown_connections 2024-08-30T09:13:57.941 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.937+0000 7f5c00ee8640 1 --2- 172.21.15.42:0/4027601394 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f5bec0b4d40 0x7f5bec0b7250 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:57.941 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.937+0000 7f5c00ee8640 1 --2- 172.21.15.42:0/4027601394 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5bf40a5be0 0x7f5bf413ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:57.941 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.937+0000 7f5c00ee8640 1 --2- 172.21.15.42:0/4027601394 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5bf40a6580 0x7f5bf413c640 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:57.941 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.937+0000 7f5c00ee8640 1 --2- 172.21.15.42:0/4027601394 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5bf40a4cf0 0x7f5bf413b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:13:57.941 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.937+0000 7f5c00ee8640 1 -- 172.21.15.42:0/4027601394 >> 172.21.15.42:0/4027601394 conn(0x7f5bf40a0480 msgr2=0x7f5bf40a8ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:13:57.941 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.937+0000 7f5c00ee8640 1 -- 172.21.15.42:0/4027601394 shutdown_connections 2024-08-30T09:13:57.941 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:13:57.937+0000 7f5c00ee8640 1 -- 172.21.15.42:0/4027601394 wait complete. 2024-08-30T09:13:58.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:13:58 smithi042 ceph-mon[18652]: pgmap v769: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 4.5 MiB/s wr, 441 op/s 2024-08-30T09:13:58.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:13:58 smithi190 ceph-mon[22001]: pgmap v769: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 4.5 MiB/s wr, 441 op/s 2024-08-30T09:13:58.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:13:58 smithi167 ceph-mon[22063]: pgmap v769: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 4.5 MiB/s wr, 441 op/s 2024-08-30T09:14:00.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:14:00 smithi190 ceph-mon[22001]: pgmap v770: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 5.9 MiB/s wr, 602 op/s 2024-08-30T09:14:00.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:14:00 smithi167 ceph-mon[22063]: pgmap v770: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 5.9 MiB/s wr, 602 op/s 2024-08-30T09:14:00.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:14:00 smithi042 ceph-mon[18652]: pgmap v770: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 5.9 MiB/s wr, 602 op/s 2024-08-30T09:14:00.953 INFO:teuthology.orchestra.run.smithi042.stderr:progress: 795.0 s, 498.2 tps, lat 203.946 ms stddev 229.323 2024-08-30T09:14:02.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:14:02 smithi190 ceph-mon[22001]: pgmap v771: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 4.2 MiB/s wr, 443 op/s 2024-08-30T09:14:02.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:14:02 smithi167 ceph-mon[22063]: pgmap v771: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 4.2 MiB/s wr, 443 op/s 2024-08-30T09:14:02.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:14:02 smithi042 ceph-mon[18652]: pgmap v771: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 4.2 MiB/s wr, 443 op/s 2024-08-30T09:14:04.685 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:14:04 smithi042 ceph-mon[18652]: pgmap v772: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 5.3 MiB/s wr, 565 op/s 2024-08-30T09:14:04.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:14:04 smithi190 ceph-mon[22001]: pgmap v772: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 5.3 MiB/s wr, 565 op/s 2024-08-30T09:14:04.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:14:04 smithi167 ceph-mon[22063]: pgmap v772: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 5.3 MiB/s wr, 565 op/s 2024-08-30T09:14:05.953 INFO:teuthology.orchestra.run.smithi042.stderr:progress: 800.0 s, 473.8 tps, lat 209.408 ms stddev 260.321 2024-08-30T09:14:06.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:14:06 smithi190 ceph-mon[22001]: pgmap v773: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 4.7 MiB/s wr, 484 op/s 2024-08-30T09:14:06.712 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:14:06 smithi042 ceph-mon[18652]: pgmap v773: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 4.7 MiB/s wr, 484 op/s 2024-08-30T09:14:06.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:14:06 smithi167 ceph-mon[22063]: pgmap v773: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 4.7 MiB/s wr, 484 op/s 2024-08-30T09:14:08.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:14:08 smithi042 ceph-mon[18652]: pgmap v774: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 4.2 MiB/s wr, 441 op/s 2024-08-30T09:14:08.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:14:08 smithi190 ceph-mon[22001]: pgmap v774: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 4.2 MiB/s wr, 441 op/s 2024-08-30T09:14:08.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:14:08 smithi167 ceph-mon[22063]: pgmap v774: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 4.2 MiB/s wr, 441 op/s 2024-08-30T09:14:10.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:14:10 smithi190 ceph-mon[22001]: pgmap v775: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 5.1 MiB/s wr, 552 op/s 2024-08-30T09:14:10.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:14:10 smithi167 ceph-mon[22063]: pgmap v775: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 5.1 MiB/s wr, 552 op/s 2024-08-30T09:14:10.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:14:10 smithi042 ceph-mon[18652]: pgmap v775: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 5.1 MiB/s wr, 552 op/s 2024-08-30T09:14:10.953 INFO:teuthology.orchestra.run.smithi042.stderr:progress: 805.0 s, 445.0 tps, lat 218.821 ms stddev 258.609 2024-08-30T09:14:11.112 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.107+0000 7f61e4378640 1 -- 172.21.15.42:0/3047227208 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f61dc101770 msgr2=0x7f61dc101b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:14:11.112 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.107+0000 7f61e4378640 1 --2- 172.21.15.42:0/3047227208 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f61dc101770 0x7f61dc101b80 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f61d4009ca0 tx=0x7f61d402a8f0 comp rx=0 tx=0).stop 2024-08-30T09:14:11.119 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.115+0000 7f61e4378640 1 -- 172.21.15.42:0/3047227208 shutdown_connections 2024-08-30T09:14:11.119 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.115+0000 7f61e4378640 1 --2- 172.21.15.42:0/3047227208 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f61dc102d50 0x7f61dc107800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:11.119 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.115+0000 7f61e4378640 1 --2- 172.21.15.42:0/3047227208 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f61dc1021a0 0x7f61dc102610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:11.119 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.115+0000 7f61e4378640 1 --2- 172.21.15.42:0/3047227208 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f61dc101770 0x7f61dc101b80 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:11.119 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.115+0000 7f61e4378640 1 -- 172.21.15.42:0/3047227208 >> 172.21.15.42:0/3047227208 conn(0x7f61dc0fd430 msgr2=0x7f61dc0ff8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:14:11.122 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.120+0000 7f61e4378640 1 -- 172.21.15.42:0/3047227208 shutdown_connections 2024-08-30T09:14:11.127 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.122+0000 7f61e4378640 1 -- 172.21.15.42:0/3047227208 wait complete. 2024-08-30T09:14:11.127 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.123+0000 7f61e4378640 1 Processor -- start 2024-08-30T09:14:11.127 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.123+0000 7f61e4378640 1 -- start start 2024-08-30T09:14:11.127 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.123+0000 7f61e4378640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f61dc101770 0x7f61dc0720e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:14:11.127 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.123+0000 7f61e4378640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f61dc1021a0 0x7f61dc072750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:14:11.127 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.123+0000 7f61e4378640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f61dc102d50 0x7f61dc073300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:14:11.127 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.123+0000 7f61e4378640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f61dc1157a0 con 0x7f61dc101770 2024-08-30T09:14:11.127 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.123+0000 7f61e4378640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f61dc115620 con 0x7f61dc102d50 2024-08-30T09:14:11.127 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.123+0000 7f61e4378640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f61dc115920 con 0x7f61dc1021a0 2024-08-30T09:14:11.132 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.125+0000 7f61e2114640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f61dc101770 0x7f61dc0720e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:14:11.132 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.125+0000 7f61e2114640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f61dc101770 0x7f61dc0720e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:53348/0 (socket says 172.21.15.42:53348) 2024-08-30T09:14:11.133 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.125+0000 7f61e2114640 1 -- 172.21.15.42:0/456611669 learned_addr learned my addr 172.21.15.42:0/456611669 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:14:11.133 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.127+0000 7f61e2114640 1 -- 172.21.15.42:0/456611669 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f61dc1021a0 msgr2=0x7f61dc072750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:14:11.133 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.127+0000 7f61e2114640 1 --2- 172.21.15.42:0/456611669 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f61dc1021a0 0x7f61dc072750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:11.133 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.127+0000 7f61e2114640 1 -- 172.21.15.42:0/456611669 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f61dc102d50 msgr2=0x7f61dc073300 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-08-30T09:14:11.133 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.127+0000 7f61e2114640 1 --2- 172.21.15.42:0/456611669 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f61dc102d50 0x7f61dc073300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:11.133 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.127+0000 7f61e2114640 1 -- 172.21.15.42:0/456611669 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61dc073b20 con 0x7f61dc101770 2024-08-30T09:14:11.133 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.128+0000 7f61e2114640 1 --2- 172.21.15.42:0/456611669 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f61dc101770 0x7f61dc0720e0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f61d402aee0 tx=0x7f61d402f870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:14:11.133 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.128+0000 7f61d37fe640 1 -- 172.21.15.42:0/456611669 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d402fdf0 con 0x7f61dc101770 2024-08-30T09:14:11.133 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.128+0000 7f61e4378640 1 -- 172.21.15.42:0/456611669 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61dc073db0 con 0x7f61dc101770 2024-08-30T09:14:11.133 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.128+0000 7f61e4378640 1 -- 172.21.15.42:0/456611669 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61dc074550 con 0x7f61dc101770 2024-08-30T09:14:11.133 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.129+0000 7f61d37fe640 1 -- 172.21.15.42:0/456611669 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f61d402d070 con 0x7f61dc101770 2024-08-30T09:14:11.133 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.129+0000 7f61d37fe640 1 -- 172.21.15.42:0/456611669 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d403b660 con 0x7f61dc101770 2024-08-30T09:14:11.134 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.132+0000 7f61eaf22740 1 Processor -- start 2024-08-30T09:14:11.134 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.133+0000 7f61eaf22740 1 -- start start 2024-08-30T09:14:11.136 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.133+0000 7f61eaf22740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558918f621b0 0x558919308180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:14:11.136 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.133+0000 7f61eaf22740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558918f392f0 0x5589193087f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:14:11.136 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.133+0000 7f61eaf22740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558918f414d0 0x558919309340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:14:11.136 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.133+0000 7f61eaf22740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55891927b000 con 0x558918f414d0 2024-08-30T09:14:11.136 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.133+0000 7f61eaf22740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55891926a590 con 0x558918f621b0 2024-08-30T09:14:11.136 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.133+0000 7f61eaf22740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55891926e540 con 0x558918f392f0 2024-08-30T09:14:11.136 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.134+0000 7f61e2915640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558918f621b0 0x558919308180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:14:11.136 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.134+0000 7f61e2915640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558918f621b0 0x558919308180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:55030/0 (socket says 172.21.15.42:55030) 2024-08-30T09:14:11.136 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.134+0000 7f61e2915640 1 -- 172.21.15.42:0/3099957532 learned_addr learned my addr 172.21.15.42:0/3099957532 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:14:11.137 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.135+0000 7f61d37fe640 1 -- 172.21.15.42:0/456611669 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7f61d40305f0 con 0x7f61dc101770 2024-08-30T09:14:11.139 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.137+0000 7f61d37fe640 1 --2- 172.21.15.42:0/456611669 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f61c80b4d40 0x7f61c80b7250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:14:11.139 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.137+0000 7f61e2114640 1 --2- 172.21.15.42:0/3099957532 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558918f392f0 0x5589193087f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:14:11.139 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.137+0000 7f61d37fe640 1 -- 172.21.15.42:0/456611669 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f61d4106590 con 0x7f61dc101770 2024-08-30T09:14:11.140 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.139+0000 7f61e1913640 1 --2- 172.21.15.42:0/3099957532 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558918f414d0 0x558919309340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:14:11.141 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.139+0000 7f61e2114640 1 -- 172.21.15.42:0/3099957532 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558918f621b0 msgr2=0x558919308180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:14:11.141 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.139+0000 7f61e2114640 1 --2- 172.21.15.42:0/3099957532 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558918f621b0 0x558919308180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:11.141 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.139+0000 7f61e2114640 1 -- 172.21.15.42:0/3099957532 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558918f414d0 msgr2=0x558919309340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:14:11.141 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.139+0000 7f61e2114640 1 --2- 172.21.15.42:0/3099957532 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558918f414d0 0x558919309340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:11.142 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.139+0000 7f61e2114640 1 -- 172.21.15.42:0/3099957532 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558918fcd790 con 0x558918f392f0 2024-08-30T09:14:11.142 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.140+0000 7f61e2915640 1 --2- 172.21.15.42:0/456611669 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f61c80b4d40 0x7f61c80b7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:14:11.142 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.140+0000 7f61e2114640 1 --2- 172.21.15.42:0/3099957532 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558918f392f0 0x5589193087f0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f61d4009dc0 tx=0x7f61d4031ac0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7a77b2a627069d7f server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:14:11.142 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.141+0000 7f61d0bfe640 1 -- 172.21.15.42:0/3099957532 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d40cc6e0 con 0x558918f392f0 2024-08-30T09:14:11.142 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.141+0000 7f61d0bfe640 1 -- 172.21.15.42:0/3099957532 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f61d4036030 con 0x558918f392f0 2024-08-30T09:14:11.144 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.142+0000 7f61eaf22740 1 -- 172.21.15.42:0/3099957532 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558918f392f0 msgr2=0x5589193087f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:14:11.144 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.142+0000 7f61eaf22740 1 --2- 172.21.15.42:0/3099957532 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558918f392f0 0x5589193087f0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f61d4009dc0 tx=0x7f61d4031ac0 comp rx=0 tx=0).stop 2024-08-30T09:14:11.146 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.144+0000 7f61eaf22740 1 -- 172.21.15.42:0/3099957532 shutdown_connections 2024-08-30T09:14:11.146 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.144+0000 7f61eaf22740 1 --2- 172.21.15.42:0/3099957532 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558918f392f0 0x5589193087f0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:11.146 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.144+0000 7f61eaf22740 1 --2- 172.21.15.42:0/3099957532 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558918f414d0 0x558919309340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:11.146 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.144+0000 7f61eaf22740 1 --2- 172.21.15.42:0/3099957532 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558918f621b0 0x558919308180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:11.146 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.144+0000 7f61eaf22740 1 -- 172.21.15.42:0/3099957532 >> 172.21.15.42:0/3099957532 conn(0x558918f56300 msgr2=0x558919303c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:14:11.147 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.145+0000 7f61e2915640 1 --2- 172.21.15.42:0/456611669 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f61c80b4d40 0x7f61c80b7250 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f61cc009e80 tx=0x7f61cc004770 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:14:11.147 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.146+0000 7f61eaf22740 1 -- 172.21.15.42:0/3099957532 shutdown_connections 2024-08-30T09:14:11.148 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.146+0000 7f61eaf22740 1 -- 172.21.15.42:0/3099957532 wait complete. 2024-08-30T09:14:11.150 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.147+0000 7f61eaf22740 1 -- start start 2024-08-30T09:14:11.150 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.147+0000 7f61eaf22740 1 Processor -- start 2024-08-30T09:14:11.150 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.148+0000 7f61eaf22740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558918f392f0 0x5589193118d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:14:11.150 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.148+0000 7f61eaf22740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558918f414d0 0x558919311f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:14:11.150 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.148+0000 7f61eaf22740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558918f621b0 0x558919312a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:14:11.150 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.148+0000 7f61eaf22740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x5589191a3ab0 con 0x558918f621b0 2024-08-30T09:14:11.150 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.148+0000 7f61eaf22740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5589191f1680 con 0x558918f414d0 2024-08-30T09:14:11.150 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.148+0000 7f61eaf22740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x558919309c20 con 0x558918f392f0 2024-08-30T09:14:11.150 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.148+0000 7f61eaf22740 10 client.0 _ll_register_callbacks cb 0x558918f79a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:14:11.151 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.148+0000 7f61e2915640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558918f621b0 0x558919312a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:14:11.151 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.148+0000 7f61e2915640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558918f621b0 0x558919312a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:53362/0 (socket says 172.21.15.42:53362) 2024-08-30T09:14:11.151 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.148+0000 7f61e2915640 1 -- 172.21.15.42:0/805611025 learned_addr learned my addr 172.21.15.42:0/805611025 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:14:11.154 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.150+0000 7f61e2114640 1 --2- 172.21.15.42:0/805611025 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558918f392f0 0x5589193118d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:14:11.154 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.150+0000 7f61e1913640 1 --2- 172.21.15.42:0/805611025 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558918f414d0 0x558919311f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:14:11.154 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.151+0000 7f61e2915640 1 -- 172.21.15.42:0/805611025 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558918f392f0 msgr2=0x5589193118d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:14:11.155 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.151+0000 7f61e2915640 1 --2- 172.21.15.42:0/805611025 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558918f392f0 0x5589193118d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:11.155 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.151+0000 7f61e2915640 1 -- 172.21.15.42:0/805611025 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558918f414d0 msgr2=0x558919311f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:14:11.155 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.151+0000 7f61e2915640 1 --2- 172.21.15.42:0/805611025 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558918f414d0 0x558919311f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:11.155 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.151+0000 7f61e2915640 1 -- 172.21.15.42:0/805611025 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558919046620 con 0x558918f621b0 2024-08-30T09:14:11.155 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.152+0000 7f61e2915640 1 --2- 172.21.15.42:0/805611025 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558918f621b0 0x558919312a90 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f61cc006c00 tx=0x7f61cc004700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85d90ec743506f5c server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:14:11.155 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.152+0000 7f61c5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-08-30T09:14:11.155 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.152+0000 7f61c5ffb640 1 -- 172.21.15.42:0/805611025 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61cc015ba0 con 0x558918f621b0 2024-08-30T09:14:11.155 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.152+0000 7f61eaf22740 1 -- 172.21.15.42:0/805611025 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f61d4036030 con 0x558918f621b0 2024-08-30T09:14:11.155 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.152+0000 7f61c5ffb640 1 -- 172.21.15.42:0/805611025 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f61cc015d40 con 0x558918f621b0 2024-08-30T09:14:11.157 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.153+0000 7f61c5ffb640 1 -- 172.21.15.42:0/805611025 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f61cc02e050 con 0x558918f621b0 2024-08-30T09:14:11.157 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.155+0000 7f61eaf22740 10 client.17628 fetch_fsmap learned FSMap version 47 2024-08-30T09:14:11.157 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.155+0000 7f61eaf22740 1 -- 172.21.15.42:0/805611025 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55891931bae0 con 0x558918f621b0 2024-08-30T09:14:11.160 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.157+0000 7f61c5ffb640 1 -- 172.21.15.42:0/805611025 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f61d4036030 con 0x558918f621b0 2024-08-30T09:14:11.160 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.157+0000 7f61c5ffb640 1 -- 172.21.15.42:0/805611025 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 47) ==== 8370+0+0 (secure 0 0 0) 0x7f61cc028350 con 0x558918f621b0 2024-08-30T09:14:11.160 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.158+0000 7f61eaf22740 10 client.17628 fetch_fsmap finished waiting for FSMap version 47 2024-08-30T09:14:11.160 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.158+0000 7f61eaf22740 10 client.17628 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-30T09:14:11.160 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.158+0000 7f61eaf22740 20 client.17628 populate_metadata read hostname 'smithi042' 2024-08-30T09:14:11.160 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.158+0000 7f61eaf22740 1 --2- 172.21.15.42:0/805611025 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x558918f20d80 0x55891931f310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:14:11.160 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.158+0000 7f61eaf22740 4 client.17628 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-30T09:14:11.160 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.158+0000 7f61eaf22740 1 -- 172.21.15.42:0/805611025 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55891927cf50 con 0x558918f20d80 2024-08-30T09:14:11.161 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.158+0000 7f61e2114640 1 --2- 172.21.15.42:0/805611025 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x558918f20d80 0x55891931f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:14:11.161 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.159+0000 7f61c5ffb640 10 client.17628 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:14:11.161 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.159+0000 7f61e2114640 1 --2- 172.21.15.42:0/805611025 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x558918f20d80 0x55891931f310 crc :-1 s=READY pgs=273 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aed87fbcd3f5642c server_cookie=5a465ca819c26568 in_seq=0 out_seq=0 2024-08-30T09:14:11.161 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.160+0000 7f61c5ffb640 1 -- 172.21.15.42:0/805611025 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55891927cf50 con 0x558918f20d80 2024-08-30T09:14:11.162 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.160+0000 7f61c5ffb640 10 client.17628 handle_command_reply: tid=0 2024-08-30T09:14:11.162 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.160+0000 7f61eaf22740 1 client.17628 shutdown 2024-08-30T09:14:11.163 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.161+0000 7f61eaf22740 1 -- 172.21.15.42:0/805611025 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558918f621b0 msgr2=0x558919312a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:14:11.163 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.161+0000 7f61eaf22740 1 --2- 172.21.15.42:0/805611025 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558918f621b0 0x558919312a90 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f61cc006c00 tx=0x7f61cc004700 comp rx=0 tx=0).stop 2024-08-30T09:14:11.163 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.161+0000 7f61eaf22740 1 -- 172.21.15.42:0/805611025 shutdown_connections 2024-08-30T09:14:11.163 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.161+0000 7f61eaf22740 1 --2- 172.21.15.42:0/805611025 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x558918f20d80 0x55891931f310 crc :-1 s=READY pgs=273 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:11.163 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.161+0000 7f61eaf22740 1 --2- 172.21.15.42:0/805611025 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558918f621b0 0x558919312a90 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:11.163 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.161+0000 7f61eaf22740 1 --2- 172.21.15.42:0/805611025 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558918f414d0 0x558919311f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:11.163 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.161+0000 7f61eaf22740 1 --2- 172.21.15.42:0/805611025 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558918f392f0 0x5589193118d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:11.163 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.161+0000 7f61eaf22740 1 -- 172.21.15.42:0/805611025 >> 172.21.15.42:0/805611025 conn(0x558918f56300 msgr2=0x558919304810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:14:11.163 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.161+0000 7f61c5ffb640 0 client.17628 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:14:11.163 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.161+0000 7f61eaf22740 1 -- 172.21.15.42:0/805611025 shutdown_connections 2024-08-30T09:14:11.164 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.161+0000 7f61eaf22740 1 -- 172.21.15.42:0/805611025 wait complete. 2024-08-30T09:14:11.164 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.161+0000 7f61eaf22740 20 client.17628 trim_cache size 0 max 16384 2024-08-30T09:14:11.194 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.191+0000 7f61eaf22740 1 Processor -- start 2024-08-30T09:14:11.194 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.192+0000 7f61eaf22740 1 -- start start 2024-08-30T09:14:11.194 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.192+0000 7f61eaf22740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558918f392f0 0x55891930b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:14:11.194 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.192+0000 7f61eaf22740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558918f414d0 0x558919307d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:14:11.194 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.192+0000 7f61eaf22740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558918f621b0 0x558919303550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:14:11.194 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.192+0000 7f61e2114640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558918f392f0 0x55891930b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:14:11.194 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.192+0000 7f61e2114640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558918f392f0 0x55891930b0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:55042/0 (socket says 172.21.15.42:55042) 2024-08-30T09:14:11.195 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.192+0000 7f61e2114640 1 -- 172.21.15.42:0/3859752733 learned_addr learned my addr 172.21.15.42:0/3859752733 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:14:11.195 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.193+0000 7f61e2915640 1 --2- 172.21.15.42:0/3859752733 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558918f621b0 0x558919303550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:14:11.196 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.193+0000 7f61eaf22740 1 -- 172.21.15.42:0/3859752733 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f61c0012650 con 0x558918f621b0 2024-08-30T09:14:11.196 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.193+0000 7f61eaf22740 1 -- 172.21.15.42:0/3859752733 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f61c000fec0 con 0x558918f392f0 2024-08-30T09:14:11.196 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.193+0000 7f61eaf22740 1 -- 172.21.15.42:0/3859752733 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f61c000dd10 con 0x558918f414d0 2024-08-30T09:14:11.196 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.193+0000 7f61e2915640 1 -- 172.21.15.42:0/3859752733 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558918f414d0 msgr2=0x558919307d70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-30T09:14:11.196 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.193+0000 7f61e2915640 1 --2- 172.21.15.42:0/3859752733 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558918f414d0 0x558919307d70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:11.196 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.193+0000 7f61e2915640 1 -- 172.21.15.42:0/3859752733 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558918f392f0 msgr2=0x55891930b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:14:11.196 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.193+0000 7f61e2915640 1 --2- 172.21.15.42:0/3859752733 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558918f392f0 0x55891930b0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:11.196 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.193+0000 7f61e2915640 1 -- 172.21.15.42:0/3859752733 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558919308bf0 con 0x558918f621b0 2024-08-30T09:14:11.196 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.193+0000 7f61e2915640 1 --2- 172.21.15.42:0/3859752733 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558918f621b0 0x558919303550 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f61cc028910 tx=0x7f61cc0092c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=439a563b84bff3f server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:14:11.196 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.194+0000 7f61c7fff640 1 -- 172.21.15.42:0/3859752733 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61cc038070 con 0x558918f621b0 2024-08-30T09:14:11.196 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.194+0000 7f61c7fff640 1 -- 172.21.15.42:0/3859752733 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f61cc025070 con 0x558918f621b0 2024-08-30T09:14:11.196 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.194+0000 7f61c7fff640 1 -- 172.21.15.42:0/3859752733 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61cc034420 con 0x558918f621b0 2024-08-30T09:14:11.197 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.195+0000 7f61eaf22740 1 -- 172.21.15.42:0/3859752733 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558918f621b0 msgr2=0x558919303550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:14:11.197 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.195+0000 7f61eaf22740 1 --2- 172.21.15.42:0/3859752733 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558918f621b0 0x558919303550 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f61cc028910 tx=0x7f61cc0092c0 comp rx=0 tx=0).stop 2024-08-30T09:14:11.197 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.195+0000 7f61eaf22740 1 -- 172.21.15.42:0/3859752733 shutdown_connections 2024-08-30T09:14:11.197 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.195+0000 7f61eaf22740 1 --2- 172.21.15.42:0/3859752733 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558918f414d0 0x558919307d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:11.197 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.195+0000 7f61eaf22740 1 --2- 172.21.15.42:0/3859752733 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558918f621b0 0x558919303550 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:11.197 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.195+0000 7f61eaf22740 1 --2- 172.21.15.42:0/3859752733 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558918f392f0 0x55891930b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:11.197 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.195+0000 7f61eaf22740 1 -- 172.21.15.42:0/3859752733 >> 172.21.15.42:0/3859752733 conn(0x558918f56300 msgr2=0x558919310ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:14:11.198 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.196+0000 7f61eaf22740 1 -- 172.21.15.42:0/3859752733 shutdown_connections 2024-08-30T09:14:11.198 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.196+0000 7f61eaf22740 1 -- 172.21.15.42:0/3859752733 wait complete. 2024-08-30T09:14:11.198 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.196+0000 7f61eaf22740 1 -- start start 2024-08-30T09:14:11.199 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.197+0000 7f61eaf22740 1 Processor -- start 2024-08-30T09:14:11.199 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.197+0000 7f61eaf22740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558918f392f0 0x558919316cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:14:11.199 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.197+0000 7f61eaf22740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558918f414d0 0x558919319260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:14:11.199 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.198+0000 7f61e2114640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558918f392f0 0x558919316cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:14:11.200 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.198+0000 7f61e2114640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558918f392f0 0x558919316cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:55056/0 (socket says 172.21.15.42:55056) 2024-08-30T09:14:11.200 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.198+0000 7f61e2114640 1 -- 172.21.15.42:0/4134140818 learned_addr learned my addr 172.21.15.42:0/4134140818 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:14:11.201 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.198+0000 7f61e1913640 1 --2- 172.21.15.42:0/4134140818 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558918f414d0 0x558919319260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:14:11.202 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.198+0000 7f61eaf22740 1 --2- 172.21.15.42:0/4134140818 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558918f621b0 0x558919387cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:14:11.202 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.198+0000 7f61eaf22740 1 -- 172.21.15.42:0/4134140818 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f61c000e380 con 0x558918f621b0 2024-08-30T09:14:11.202 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.198+0000 7f61eaf22740 1 -- 172.21.15.42:0/4134140818 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f61c000f7f0 con 0x558918f392f0 2024-08-30T09:14:11.202 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.198+0000 7f61eaf22740 1 -- 172.21.15.42:0/4134140818 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f61c000ea50 con 0x558918f414d0 2024-08-30T09:14:11.202 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.198+0000 7f61e2915640 1 --2- 172.21.15.42:0/4134140818 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558918f621b0 0x558919387cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:14:11.202 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.199+0000 7f61eaf22740 10 client.0 _ll_register_callbacks cb 0x7f61cc016690 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:14:11.202 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.199+0000 7f61e2114640 1 -- 172.21.15.42:0/4134140818 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558918f414d0 msgr2=0x558919319260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:14:11.202 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.199+0000 7f61e2114640 1 --2- 172.21.15.42:0/4134140818 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558918f414d0 0x558919319260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:11.202 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.199+0000 7f61e2114640 1 -- 172.21.15.42:0/4134140818 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558918f621b0 msgr2=0x558919387cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:14:11.202 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.199+0000 7f61e2114640 1 --2- 172.21.15.42:0/4134140818 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558918f621b0 0x558919387cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:11.202 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.199+0000 7f61e2114640 1 -- 172.21.15.42:0/4134140818 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558919388380 con 0x558918f392f0 2024-08-30T09:14:11.202 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.199+0000 7f61e2114640 1 --2- 172.21.15.42:0/4134140818 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558918f392f0 0x558919316cf0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f61d4048040 tx=0x7f61d4009c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d31683e48a037690 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:14:11.203 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.199+0000 7f61c5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-30T09:14:11.203 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.199+0000 7f61c5ffb640 1 -- 172.21.15.42:0/4134140818 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d40df040 con 0x558918f392f0 2024-08-30T09:14:11.203 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.200+0000 7f61eaf22740 1 -- 172.21.15.42:0/4134140818 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f61cc025070 con 0x558918f392f0 2024-08-30T09:14:11.203 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.200+0000 7f61c5ffb640 1 -- 172.21.15.42:0/4134140818 <== mon.2 v2:172.21.15.167:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f61d4106260 con 0x558918f392f0 2024-08-30T09:14:11.203 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.200+0000 7f61c5ffb640 1 -- 172.21.15.42:0/4134140818 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f61d40cd2d0 con 0x558918f392f0 2024-08-30T09:14:11.203 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.200+0000 7f61eaf22740 10 client.27488 fetch_fsmap learned FSMap version 47 2024-08-30T09:14:11.203 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.200+0000 7f61eaf22740 1 -- 172.21.15.42:0/4134140818 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({fsmap=47}) -- 0x558919389530 con 0x558918f392f0 2024-08-30T09:14:11.204 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.200+0000 7f61c5ffb640 1 -- 172.21.15.42:0/4134140818 <== mon.2 v2:172.21.15.167:3300/0 4 ==== fsmap(e 47) ==== 8370+0+0 (secure 0 0 0) 0x7f61d40b5040 con 0x558918f392f0 2024-08-30T09:14:11.204 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.200+0000 7f61eaf22740 10 client.27488 fetch_fsmap finished waiting for FSMap version 47 2024-08-30T09:14:11.204 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.200+0000 7f61eaf22740 10 client.27488 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-30T09:14:11.204 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.200+0000 7f61eaf22740 20 client.27488 populate_metadata read hostname 'smithi042' 2024-08-30T09:14:11.204 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.200+0000 7f61eaf22740 1 --2- 172.21.15.42:0/4134140818 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x558918ef61c0 0x55891938bf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:14:11.204 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.200+0000 7f61eaf22740 4 client.27488 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-30T09:14:11.204 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.200+0000 7f61eaf22740 1 -- 172.21.15.42:0/4134140818 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f61cc025070 con 0x558918ef61c0 2024-08-30T09:14:11.204 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.201+0000 7f61e1913640 1 --2- 172.21.15.42:0/4134140818 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x558918ef61c0 0x55891938bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:14:11.204 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.201+0000 7f61c5ffb640 10 client.27488 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:14:11.205 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.201+0000 7f61e1913640 1 --2- 172.21.15.42:0/4134140818 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x558918ef61c0 0x55891938bf40 crc :-1 s=READY pgs=274 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=396baf63b3e19d1b server_cookie=7dfeea835db2ddff in_seq=0 out_seq=0 2024-08-30T09:14:11.216 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.214+0000 7f61c5ffb640 1 -- 172.21.15.42:0/4134140818 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f61cc025070 con 0x558918ef61c0 2024-08-30T09:14:11.216 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.214+0000 7f61c5ffb640 10 client.27488 handle_command_reply: tid=0 2024-08-30T09:14:11.216 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.214+0000 7f61eaf22740 1 client.27488 shutdown 2024-08-30T09:14:11.216 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.214+0000 7f61eaf22740 1 -- 172.21.15.42:0/4134140818 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558918f392f0 msgr2=0x558919316cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:14:11.216 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.214+0000 7f61eaf22740 1 --2- 172.21.15.42:0/4134140818 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558918f392f0 0x558919316cf0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f61d4048040 tx=0x7f61d4009c20 comp rx=0 tx=0).stop 2024-08-30T09:14:11.217 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.214+0000 7f61eaf22740 1 -- 172.21.15.42:0/4134140818 shutdown_connections 2024-08-30T09:14:11.217 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.214+0000 7f61eaf22740 1 --2- 172.21.15.42:0/4134140818 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x558918ef61c0 0x55891938bf40 crc :-1 s=READY pgs=274 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:11.217 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.214+0000 7f61eaf22740 1 --2- 172.21.15.42:0/4134140818 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558918f414d0 0x558919319260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:11.217 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.214+0000 7f61eaf22740 1 --2- 172.21.15.42:0/4134140818 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558918f621b0 0x558919387cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:11.217 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.214+0000 7f61eaf22740 1 --2- 172.21.15.42:0/4134140818 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558918f392f0 0x558919316cf0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:11.217 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.214+0000 7f61eaf22740 1 -- 172.21.15.42:0/4134140818 >> 172.21.15.42:0/4134140818 conn(0x558918f56300 msgr2=0x558919307d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:14:11.218 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.215+0000 7f61c5ffb640 0 client.27488 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:14:11.218 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.215+0000 7f61eaf22740 1 -- 172.21.15.42:0/4134140818 shutdown_connections 2024-08-30T09:14:11.218 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.215+0000 7f61eaf22740 1 -- 172.21.15.42:0/4134140818 wait complete. 2024-08-30T09:14:11.218 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.215+0000 7f61eaf22740 20 client.27488 trim_cache size 0 max 16384 2024-08-30T09:14:11.218 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-08-30T09:14:11.218 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2024-08-30T09:14:11.218 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-08-30T09:14:11.218 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-08-30T09:14:11.219 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.216+0000 7f61e4378640 1 -- 172.21.15.42:0/456611669 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f61c80b4d40 msgr2=0x7f61c80b7250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:14:11.219 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.216+0000 7f61e4378640 1 --2- 172.21.15.42:0/456611669 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f61c80b4d40 0x7f61c80b7250 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f61cc009e80 tx=0x7f61cc004770 comp rx=0 tx=0).stop 2024-08-30T09:14:11.219 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.216+0000 7f61e4378640 1 -- 172.21.15.42:0/456611669 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f61dc101770 msgr2=0x7f61dc0720e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:14:11.219 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.216+0000 7f61e4378640 1 --2- 172.21.15.42:0/456611669 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f61dc101770 0x7f61dc0720e0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f61d402aee0 tx=0x7f61d402f870 comp rx=0 tx=0).stop 2024-08-30T09:14:11.219 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.216+0000 7f61e4378640 1 -- 172.21.15.42:0/456611669 shutdown_connections 2024-08-30T09:14:11.219 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.216+0000 7f61e4378640 1 --2- 172.21.15.42:0/456611669 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f61c80b4d40 0x7f61c80b7250 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:11.219 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.216+0000 7f61e4378640 1 --2- 172.21.15.42:0/456611669 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f61dc1021a0 0x7f61dc072750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:11.220 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.216+0000 7f61e4378640 1 --2- 172.21.15.42:0/456611669 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f61dc102d50 0x7f61dc073300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:11.220 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.216+0000 7f61e4378640 1 --2- 172.21.15.42:0/456611669 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f61dc101770 0x7f61dc0720e0 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:11.220 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.216+0000 7f61e4378640 1 -- 172.21.15.42:0/456611669 >> 172.21.15.42:0/456611669 conn(0x7f61dc0fd430 msgr2=0x7f61dc1037e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:14:11.220 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.216+0000 7f61e4378640 1 -- 172.21.15.42:0/456611669 shutdown_connections 2024-08-30T09:14:11.220 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:11.217+0000 7f61e4378640 1 -- 172.21.15.42:0/456611669 wait complete. 2024-08-30T09:14:12.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:14:12 smithi190 ceph-mon[22001]: pgmap v776: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 3.8 MiB/s wr, 411 op/s 2024-08-30T09:14:12.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:14:12 smithi167 ceph-mon[22063]: pgmap v776: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 3.8 MiB/s wr, 411 op/s 2024-08-30T09:14:12.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:14:12 smithi042 ceph-mon[18652]: pgmap v776: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 3.8 MiB/s wr, 411 op/s 2024-08-30T09:14:13.425 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-30T09:14:13.840 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T09:14:14.583 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:14:14 smithi042 ceph-mon[18652]: pgmap v777: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 5.2 MiB/s wr, 541 op/s 2024-08-30T09:14:14.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:14:14 smithi190 ceph-mon[22001]: pgmap v777: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 5.2 MiB/s wr, 541 op/s 2024-08-30T09:14:14.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:14:14 smithi167 ceph-mon[22063]: pgmap v777: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 5.2 MiB/s wr, 541 op/s 2024-08-30T09:14:15.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.484+0000 7ff2a9fa3640 1 -- 172.21.15.42:0/316785930 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff2a411c9d0 msgr2=0x7ff2a411eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:14:15.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.484+0000 7ff2a9fa3640 1 --2- 172.21.15.42:0/316785930 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff2a411c9d0 0x7ff2a411eee0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7ff29c009be0 tx=0x7ff29c02a710 comp rx=0 tx=0).stop 2024-08-30T09:14:15.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.484+0000 7ff2a9fa3640 1 -- 172.21.15.42:0/316785930 shutdown_connections 2024-08-30T09:14:15.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.485+0000 7ff2a9fa3640 1 --2- 172.21.15.42:0/316785930 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff2a410a780 0x7ff2a410ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:15.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.485+0000 7ff2a9fa3640 1 --2- 172.21.15.42:0/316785930 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff2a411c9d0 0x7ff2a411eee0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:15.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.485+0000 7ff2a9fa3640 1 --2- 172.21.15.42:0/316785930 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff2a4111ce0 0x7ff2a410a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:15.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.485+0000 7ff2a9fa3640 1 -- 172.21.15.42:0/316785930 >> 172.21.15.42:0/316785930 conn(0x7ff2a4070c60 msgr2=0x7ff2a4071070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:14:15.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.486+0000 7ff2a9fa3640 1 -- 172.21.15.42:0/316785930 shutdown_connections 2024-08-30T09:14:15.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.488+0000 7ff2a9fa3640 1 -- 172.21.15.42:0/316785930 wait complete. 2024-08-30T09:14:15.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.489+0000 7ff2a9fa3640 1 Processor -- start 2024-08-30T09:14:15.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.489+0000 7ff2a9fa3640 1 -- start start 2024-08-30T09:14:15.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.490+0000 7ff2a9fa3640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff2a410a780 0x7ff2a411c140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:14:15.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.490+0000 7ff2a9fa3640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff2a4111ce0 0x7ff2a4118420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:14:15.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.490+0000 7ff2a9fa3640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff2a411c9d0 0x7ff2a4118960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:14:15.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.490+0000 7ff2a9fa3640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7ff2a4114960 con 0x7ff2a410a780 2024-08-30T09:14:15.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.490+0000 7ff2a9fa3640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff2a41147e0 con 0x7ff2a411c9d0 2024-08-30T09:14:15.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.490+0000 7ff2a9fa3640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7ff2a4114ae0 con 0x7ff2a4111ce0 2024-08-30T09:14:15.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.493+0000 7ff2a3fff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff2a411c9d0 0x7ff2a4118960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:14:15.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.494+0000 7ff2a37fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff2a410a780 0x7ff2a411c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:14:15.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.494+0000 7ff2a37fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff2a410a780 0x7ff2a411c140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:53400/0 (socket says 172.21.15.42:53400) 2024-08-30T09:14:15.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.494+0000 7ff2a37fe640 1 -- 172.21.15.42:0/318793998 learned_addr learned my addr 172.21.15.42:0/318793998 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:14:15.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.494+0000 7ff29affd640 1 --2- 172.21.15.42:0/318793998 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff2a4111ce0 0x7ff2a4118420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:14:15.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.495+0000 7ff2a37fe640 1 -- 172.21.15.42:0/318793998 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff2a4111ce0 msgr2=0x7ff2a4118420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:14:15.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.495+0000 7ff2a37fe640 1 --2- 172.21.15.42:0/318793998 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff2a4111ce0 0x7ff2a4118420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:15.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.495+0000 7ff2a37fe640 1 -- 172.21.15.42:0/318793998 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff2a411c9d0 msgr2=0x7ff2a4118960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:14:15.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.495+0000 7ff2a37fe640 1 --2- 172.21.15.42:0/318793998 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff2a411c9d0 0x7ff2a4118960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:15.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.495+0000 7ff2a37fe640 1 -- 172.21.15.42:0/318793998 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2a4119130 con 0x7ff2a410a780 2024-08-30T09:14:15.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.497+0000 7ff2a37fe640 1 --2- 172.21.15.42:0/318793998 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff2a410a780 0x7ff2a411c140 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7ff28c00ca80 tx=0x7ff28c00cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:14:15.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.497+0000 7ff2a17fa640 1 -- 172.21.15.42:0/318793998 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff28c007d30 con 0x7ff2a410a780 2024-08-30T09:14:15.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.497+0000 7ff2a17fa640 1 -- 172.21.15.42:0/318793998 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff28c00f900 con 0x7ff2a410a780 2024-08-30T09:14:15.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.497+0000 7ff2a17fa640 1 -- 172.21.15.42:0/318793998 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff28c018960 con 0x7ff2a410a780 2024-08-30T09:14:15.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.497+0000 7ff2a9fa3640 1 -- 172.21.15.42:0/318793998 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2a41ceb80 con 0x7ff2a410a780 2024-08-30T09:14:15.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.497+0000 7ff2a9fa3640 1 -- 172.21.15.42:0/318793998 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2a41cf070 con 0x7ff2a410a780 2024-08-30T09:14:15.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.504+0000 7ff2b0518740 1 Processor -- start 2024-08-30T09:14:15.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.504+0000 7ff2b0518740 1 -- start start 2024-08-30T09:14:15.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.504+0000 7ff2b0518740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fbd9865070 0x55fbd9c0a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:14:15.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.505+0000 7ff2b0518740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fbd983c1b0 0x55fbd9c0a800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:14:15.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.505+0000 7ff2b0518740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fbd9844390 0x55fbd9c0ad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:14:15.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.505+0000 7ff2b0518740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55fbd9b81630 con 0x55fbd9865070 2024-08-30T09:14:15.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.505+0000 7ff2a3fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fbd9865070 0x55fbd9c0a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:14:15.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.505+0000 7ff2a3fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fbd9865070 0x55fbd9c0a2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:53402/0 (socket says 172.21.15.42:53402) 2024-08-30T09:14:15.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.505+0000 7ff2a3fff640 1 -- 172.21.15.42:0/825530802 learned_addr learned my addr 172.21.15.42:0/825530802 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:14:15.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.505+0000 7ff2b0518740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55fbd9b70bc0 con 0x55fbd983c1b0 2024-08-30T09:14:15.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.505+0000 7ff2b0518740 1 -- 172.21.15.42:0/825530802 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55fbd9b74b70 con 0x55fbd9844390 2024-08-30T09:14:15.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.505+0000 7ff29affd640 1 --2- 172.21.15.42:0/825530802 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fbd9844390 0x55fbd9c0ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:14:15.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.506+0000 7ff29affd640 1 -- 172.21.15.42:0/825530802 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fbd983c1b0 msgr2=0x55fbd9c0a800 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-30T09:14:15.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.506+0000 7ff29affd640 1 --2- 172.21.15.42:0/825530802 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fbd983c1b0 0x55fbd9c0a800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:15.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.506+0000 7ff29affd640 1 -- 172.21.15.42:0/825530802 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fbd9865070 msgr2=0x55fbd9c0a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:14:15.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.506+0000 7ff29affd640 1 --2- 172.21.15.42:0/825530802 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fbd9865070 0x55fbd9c0a2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:15.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.506+0000 7ff29affd640 1 -- 172.21.15.42:0/825530802 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fbd98d0650 con 0x55fbd9844390 2024-08-30T09:14:15.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.506+0000 7ff29affd640 1 --2- 172.21.15.42:0/825530802 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fbd9844390 0x55fbd9c0ad40 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7ff2a406c710 tx=0x7ff2940107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=60ec55a6f426a14a server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:14:15.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.509+0000 7ff299ffb640 1 -- 172.21.15.42:0/825530802 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff294004030 con 0x55fbd9844390 2024-08-30T09:14:15.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.509+0000 7ff299ffb640 1 -- 172.21.15.42:0/825530802 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2940035f0 con 0x55fbd9844390 2024-08-30T09:14:15.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.510+0000 7ff2b0518740 1 -- 172.21.15.42:0/825530802 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fbd9844390 msgr2=0x55fbd9c0ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:14:15.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.510+0000 7ff2b0518740 1 --2- 172.21.15.42:0/825530802 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fbd9844390 0x55fbd9c0ad40 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7ff2a406c710 tx=0x7ff2940107a0 comp rx=0 tx=0).stop 2024-08-30T09:14:15.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.517+0000 7ff2b0518740 1 -- 172.21.15.42:0/825530802 shutdown_connections 2024-08-30T09:14:15.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.517+0000 7ff2b0518740 1 --2- 172.21.15.42:0/825530802 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fbd9844390 0x55fbd9c0ad40 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:15.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.517+0000 7ff2b0518740 1 --2- 172.21.15.42:0/825530802 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fbd983c1b0 0x55fbd9c0a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:15.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.517+0000 7ff2b0518740 1 --2- 172.21.15.42:0/825530802 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fbd9865070 0x55fbd9c0a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:15.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.517+0000 7ff2b0518740 1 -- 172.21.15.42:0/825530802 >> 172.21.15.42:0/825530802 conn(0x55fbd9859460 msgr2=0x55fbd9c081b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:14:15.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.518+0000 7ff2a17fa640 1 -- 172.21.15.42:0/318793998 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7ff28c01d070 con 0x7ff2a410a780 2024-08-30T09:14:15.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.519+0000 7ff2a17fa640 1 --2- 172.21.15.42:0/318793998 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7ff2900b4d40 0x7ff2900b7250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:14:15.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.519+0000 7ff2a17fa640 1 -- 172.21.15.42:0/318793998 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7ff28c0e52d0 con 0x7ff2a410a780 2024-08-30T09:14:15.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.523+0000 7ff2b0518740 1 -- 172.21.15.42:0/825530802 shutdown_connections 2024-08-30T09:14:15.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.523+0000 7ff29affd640 1 --2- 172.21.15.42:0/318793998 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7ff2900b4d40 0x7ff2900b7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:14:15.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.523+0000 7ff2b0518740 1 -- 172.21.15.42:0/825530802 wait complete. 2024-08-30T09:14:15.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.523+0000 7ff2b0518740 1 -- start start 2024-08-30T09:14:15.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.524+0000 7ff2b0518740 1 Processor -- start 2024-08-30T09:14:15.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.524+0000 7ff2b0518740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fbd983c1b0 0x55fbd9c15a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:14:15.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.524+0000 7ff2b0518740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fbd9844390 0x55fbd9c15f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:14:15.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.524+0000 7ff2b0518740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fbd9865070 0x55fbd9c21e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:14:15.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.524+0000 7ff2b0518740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55fbd99d3700 con 0x55fbd9865070 2024-08-30T09:14:15.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.524+0000 7ff2b0518740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55fbd9af7700 con 0x55fbd983c1b0 2024-08-30T09:14:15.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.524+0000 7ff2b0518740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55fbd9c18af0 con 0x55fbd9844390 2024-08-30T09:14:15.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.524+0000 7ff2a3fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fbd9865070 0x55fbd9c21e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:14:15.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.524+0000 7ff2a3fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fbd9865070 0x55fbd9c21e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:53418/0 (socket says 172.21.15.42:53418) 2024-08-30T09:14:15.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.524+0000 7ff2a3fff640 1 -- 172.21.15.42:0/1823444635 learned_addr learned my addr 172.21.15.42:0/1823444635 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:14:15.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.524+0000 7ff2b0518740 10 client.0 _ll_register_callbacks cb 0x55fbd9b26dc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:14:15.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.525+0000 7ff2a3fff640 1 -- 172.21.15.42:0/1823444635 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fbd9844390 msgr2=0x55fbd9c15f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:14:15.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.525+0000 7ff2a3fff640 1 --2- 172.21.15.42:0/1823444635 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fbd9844390 0x55fbd9c15f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:15.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.525+0000 7ff2a3fff640 1 -- 172.21.15.42:0/1823444635 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fbd983c1b0 msgr2=0x55fbd9c15a30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-30T09:14:15.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.525+0000 7ff2a3fff640 1 --2- 172.21.15.42:0/1823444635 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fbd983c1b0 0x55fbd9c15a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:15.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.525+0000 7ff2a3fff640 1 -- 172.21.15.42:0/1823444635 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fbd9b308b0 con 0x55fbd9865070 2024-08-30T09:14:15.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.525+0000 7ff2a3fff640 1 --2- 172.21.15.42:0/1823444635 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fbd9865070 0x55fbd9c21e90 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7ff29c034040 tx=0x7ff29c009870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2eaa02acfc84002c server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:14:15.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.528+0000 7ff282ffd640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-08-30T09:14:15.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.529+0000 7ff282ffd640 1 -- 172.21.15.42:0/1823444635 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff29c036040 con 0x55fbd9865070 2024-08-30T09:14:15.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.529+0000 7ff2b0518740 1 -- 172.21.15.42:0/1823444635 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff2940035f0 con 0x55fbd9865070 2024-08-30T09:14:15.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.530+0000 7ff282ffd640 1 -- 172.21.15.42:0/1823444635 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7ff29c030d00 con 0x55fbd9865070 2024-08-30T09:14:15.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.530+0000 7ff282ffd640 1 -- 172.21.15.42:0/1823444635 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7ff29c031440 con 0x55fbd9865070 2024-08-30T09:14:15.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.530+0000 7ff29affd640 1 --2- 172.21.15.42:0/318793998 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7ff2900b4d40 0x7ff2900b7250 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7ff294003eb0 tx=0x7ff294010fa0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:14:15.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.535+0000 7ff2b0518740 10 client.17646 fetch_fsmap learned FSMap version 47 2024-08-30T09:14:15.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.535+0000 7ff2b0518740 1 -- 172.21.15.42:0/1823444635 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55fbd9c24940 con 0x55fbd9865070 2024-08-30T09:14:15.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.535+0000 7ff282ffd640 1 -- 172.21.15.42:0/1823444635 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff29c031a90 con 0x55fbd9865070 2024-08-30T09:14:15.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.535+0000 7ff282ffd640 1 -- 172.21.15.42:0/1823444635 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 47) ==== 8370+0+0 (secure 0 0 0) 0x7ff29c05e430 con 0x55fbd9865070 2024-08-30T09:14:15.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.535+0000 7ff2b0518740 10 client.17646 fetch_fsmap finished waiting for FSMap version 47 2024-08-30T09:14:15.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.535+0000 7ff2b0518740 10 client.17646 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-30T09:14:15.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.535+0000 7ff2b0518740 20 client.17646 populate_metadata read hostname 'smithi042' 2024-08-30T09:14:15.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.536+0000 7ff2b0518740 1 --2- 172.21.15.42:0/1823444635 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55fbd98239f0 0x55fbd9c28340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:14:15.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.536+0000 7ff2b0518740 4 client.17646 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-30T09:14:15.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.536+0000 7ff2b0518740 1 -- 172.21.15.42:0/1823444635 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7ff2940035f0 con 0x55fbd98239f0 2024-08-30T09:14:15.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.540+0000 7ff2a37fe640 1 --2- 172.21.15.42:0/1823444635 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55fbd98239f0 0x55fbd9c28340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:14:15.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.547+0000 7ff282ffd640 10 client.17646 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:14:15.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.547+0000 7ff2a37fe640 1 --2- 172.21.15.42:0/1823444635 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55fbd98239f0 0x55fbd9c28340 crc :-1 s=READY pgs=275 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b3965e60dbc5b071 server_cookie=97a4d00dac1db524 in_seq=0 out_seq=0 2024-08-30T09:14:15.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.549+0000 7ff282ffd640 1 -- 172.21.15.42:0/1823444635 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7ff2940035f0 con 0x55fbd98239f0 2024-08-30T09:14:15.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.549+0000 7ff282ffd640 10 client.17646 handle_command_reply: tid=0 2024-08-30T09:14:15.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.549+0000 7ff2b0518740 1 client.17646 shutdown 2024-08-30T09:14:15.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.550+0000 7ff2b0518740 1 -- 172.21.15.42:0/1823444635 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fbd9865070 msgr2=0x55fbd9c21e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:14:15.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.550+0000 7ff2b0518740 1 --2- 172.21.15.42:0/1823444635 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fbd9865070 0x55fbd9c21e90 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7ff29c034040 tx=0x7ff29c009870 comp rx=0 tx=0).stop 2024-08-30T09:14:15.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.551+0000 7ff2b0518740 1 -- 172.21.15.42:0/1823444635 shutdown_connections 2024-08-30T09:14:15.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.551+0000 7ff2b0518740 1 --2- 172.21.15.42:0/1823444635 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55fbd98239f0 0x55fbd9c28340 crc :-1 s=READY pgs=275 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:15.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.551+0000 7ff2b0518740 1 --2- 172.21.15.42:0/1823444635 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fbd9844390 0x55fbd9c15f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:15.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.551+0000 7ff2b0518740 1 --2- 172.21.15.42:0/1823444635 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fbd9865070 0x55fbd9c21e90 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:15.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.551+0000 7ff282ffd640 0 client.17646 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:14:15.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.551+0000 7ff2b0518740 1 --2- 172.21.15.42:0/1823444635 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fbd983c1b0 0x55fbd9c15a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:15.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.551+0000 7ff2b0518740 1 -- 172.21.15.42:0/1823444635 >> 172.21.15.42:0/1823444635 conn(0x55fbd9859460 msgr2=0x55fbd9c0b2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:14:15.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.552+0000 7ff2b0518740 1 -- 172.21.15.42:0/1823444635 shutdown_connections 2024-08-30T09:14:15.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.556+0000 7ff2b0518740 1 -- 172.21.15.42:0/1823444635 wait complete. 2024-08-30T09:14:15.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.558+0000 7ff2b0518740 20 client.17646 trim_cache size 0 max 16384 2024-08-30T09:14:15.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.580+0000 7ff2b0518740 1 Processor -- start 2024-08-30T09:14:15.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.580+0000 7ff2b0518740 1 -- start start 2024-08-30T09:14:15.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.581+0000 7ff2b0518740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fbd983c1b0 0x55fbd9c081e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:14:15.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.581+0000 7ff2b0518740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fbd9844390 0x55fbd9c21d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:14:15.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.581+0000 7ff2b0518740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fbd9865070 0x55fbd9c157e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:14:15.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.581+0000 7ff2b0518740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7ff2a40818d0 con 0x55fbd9865070 2024-08-30T09:14:15.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.581+0000 7ff2b0518740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff2a407f140 con 0x55fbd9844390 2024-08-30T09:14:15.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.581+0000 7ff2b0518740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7ff2a407cf90 con 0x55fbd983c1b0 2024-08-30T09:14:15.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.581+0000 7ff2a3fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fbd9865070 0x55fbd9c157e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:14:15.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.581+0000 7ff2a3fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fbd9865070 0x55fbd9c157e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:53434/0 (socket says 172.21.15.42:53434) 2024-08-30T09:14:15.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.581+0000 7ff2a3fff640 1 -- 172.21.15.42:0/1288275293 learned_addr learned my addr 172.21.15.42:0/1288275293 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:14:15.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.581+0000 7ff2a37fe640 1 --2- 172.21.15.42:0/1288275293 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fbd983c1b0 0x55fbd9c081e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:14:15.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.581+0000 7ff29affd640 1 --2- 172.21.15.42:0/1288275293 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fbd9844390 0x55fbd9c21d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:14:15.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.581+0000 7ff2a3fff640 1 -- 172.21.15.42:0/1288275293 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fbd983c1b0 msgr2=0x55fbd9c081e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:14:15.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.581+0000 7ff2a3fff640 1 --2- 172.21.15.42:0/1288275293 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fbd983c1b0 0x55fbd9c081e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:15.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.581+0000 7ff2a3fff640 1 -- 172.21.15.42:0/1288275293 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fbd9844390 msgr2=0x55fbd9c21d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:14:15.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.581+0000 7ff2a3fff640 1 --2- 172.21.15.42:0/1288275293 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fbd9844390 0x55fbd9c21d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:15.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.581+0000 7ff2a3fff640 1 -- 172.21.15.42:0/1288275293 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fbd9c13f30 con 0x55fbd9865070 2024-08-30T09:14:15.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.581+0000 7ff29affd640 1 --2- 172.21.15.42:0/1288275293 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fbd9844390 0x55fbd9c21d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-30T09:14:15.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.581+0000 7ff2a3fff640 1 --2- 172.21.15.42:0/1288275293 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fbd9865070 0x55fbd9c157e0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7ff29c006010 tx=0x7ff29c02ae90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d65f43dab75ba53 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:14:15.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.582+0000 7ff2997fa640 1 -- 172.21.15.42:0/1288275293 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff29c07b070 con 0x55fbd9865070 2024-08-30T09:14:15.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.582+0000 7ff2997fa640 1 -- 172.21.15.42:0/1288275293 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff29c031a90 con 0x55fbd9865070 2024-08-30T09:14:15.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.582+0000 7ff2997fa640 1 -- 172.21.15.42:0/1288275293 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff29c036070 con 0x55fbd9865070 2024-08-30T09:14:15.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.582+0000 7ff2b0518740 1 -- 172.21.15.42:0/1288275293 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fbd9865070 msgr2=0x55fbd9c157e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:14:15.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.582+0000 7ff2b0518740 1 --2- 172.21.15.42:0/1288275293 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fbd9865070 0x55fbd9c157e0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7ff29c006010 tx=0x7ff29c02ae90 comp rx=0 tx=0).stop 2024-08-30T09:14:15.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.582+0000 7ff2b0518740 1 -- 172.21.15.42:0/1288275293 shutdown_connections 2024-08-30T09:14:15.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.582+0000 7ff2b0518740 1 --2- 172.21.15.42:0/1288275293 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fbd9865070 0x55fbd9c157e0 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:15.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.582+0000 7ff2b0518740 1 --2- 172.21.15.42:0/1288275293 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fbd9844390 0x55fbd9c21d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:15.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.582+0000 7ff2b0518740 1 --2- 172.21.15.42:0/1288275293 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fbd983c1b0 0x55fbd9c081e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:15.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.582+0000 7ff2b0518740 1 -- 172.21.15.42:0/1288275293 >> 172.21.15.42:0/1288275293 conn(0x55fbd9859460 msgr2=0x55fbd9c07ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:14:15.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.583+0000 7ff2b0518740 1 -- 172.21.15.42:0/1288275293 shutdown_connections 2024-08-30T09:14:15.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.583+0000 7ff2b0518740 1 -- 172.21.15.42:0/1288275293 wait complete. 2024-08-30T09:14:15.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.583+0000 7ff2b0518740 1 -- start start 2024-08-30T09:14:15.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.583+0000 7ff2b0518740 1 Processor -- start 2024-08-30T09:14:15.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.583+0000 7ff2b0518740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fbd983c1b0 0x55fbd9c19e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:14:15.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.583+0000 7ff2b0518740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fbd9844390 0x55fbd9c1a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:14:15.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.583+0000 7ff2b0518740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fbd9865070 0x55fbd9c1a900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:14:15.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.583+0000 7ff2b0518740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7ff2a407d600 con 0x55fbd9865070 2024-08-30T09:14:15.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.583+0000 7ff2b0518740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff2a407ea70 con 0x55fbd9844390 2024-08-30T09:14:15.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.583+0000 7ff2b0518740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7ff2a407dcd0 con 0x55fbd983c1b0 2024-08-30T09:14:15.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.584+0000 7ff2b0518740 10 client.0 _ll_register_callbacks cb 0x7ff29c034070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:14:15.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.584+0000 7ff2a3fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fbd9865070 0x55fbd9c1a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:14:15.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.584+0000 7ff2a3fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fbd9865070 0x55fbd9c1a900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:53446/0 (socket says 172.21.15.42:53446) 2024-08-30T09:14:15.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.584+0000 7ff2a37fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fbd983c1b0 0x55fbd9c19e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:14:15.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.584+0000 7ff2a3fff640 1 -- 172.21.15.42:0/1463208270 learned_addr learned my addr 172.21.15.42:0/1463208270 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:14:15.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.584+0000 7ff2a3fff640 1 -- 172.21.15.42:0/1463208270 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fbd983c1b0 msgr2=0x55fbd9c19e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:14:15.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.584+0000 7ff2a3fff640 1 --2- 172.21.15.42:0/1463208270 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fbd983c1b0 0x55fbd9c19e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:15.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.584+0000 7ff2a3fff640 1 -- 172.21.15.42:0/1463208270 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fbd9844390 msgr2=0x55fbd9c1a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:14:15.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.584+0000 7ff29affd640 1 --2- 172.21.15.42:0/1463208270 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fbd9844390 0x55fbd9c1a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:14:15.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.584+0000 7ff2a3fff640 1 --2- 172.21.15.42:0/1463208270 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fbd9844390 0x55fbd9c1a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:15.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.584+0000 7ff2a3fff640 1 -- 172.21.15.42:0/1463208270 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fbd9c91df0 con 0x55fbd9865070 2024-08-30T09:14:15.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.584+0000 7ff29affd640 1 --2- 172.21.15.42:0/1463208270 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fbd9844390 0x55fbd9c1a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-30T09:14:15.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.584+0000 7ff2a3fff640 1 --2- 172.21.15.42:0/1463208270 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fbd9865070 0x55fbd9c1a900 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7ff29c079040 tx=0x7ff29c009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17989fd86bea57f0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:14:15.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.584+0000 7ff282ffd640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-08-30T09:14:15.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.584+0000 7ff282ffd640 1 -- 172.21.15.42:0/1463208270 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff29c084060 con 0x55fbd9865070 2024-08-30T09:14:15.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.584+0000 7ff2b0518740 1 -- 172.21.15.42:0/1463208270 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff29c031a90 con 0x55fbd9865070 2024-08-30T09:14:15.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.584+0000 7ff282ffd640 1 -- 172.21.15.42:0/1463208270 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7ff29c05eba0 con 0x55fbd9865070 2024-08-30T09:14:15.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.585+0000 7ff282ffd640 1 -- 172.21.15.42:0/1463208270 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7ff29c08b050 con 0x55fbd9865070 2024-08-30T09:14:15.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.585+0000 7ff2b0518740 10 client.17652 fetch_fsmap learned FSMap version 47 2024-08-30T09:14:15.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.585+0000 7ff2b0518740 1 -- 172.21.15.42:0/1463208270 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55fbd9c92ff0 con 0x55fbd9865070 2024-08-30T09:14:15.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.585+0000 7ff282ffd640 1 -- 172.21.15.42:0/1463208270 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff29c031a90 con 0x55fbd9865070 2024-08-30T09:14:15.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.586+0000 7ff282ffd640 1 -- 172.21.15.42:0/1463208270 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 47) ==== 8370+0+0 (secure 0 0 0) 0x7ff29c069480 con 0x55fbd9865070 2024-08-30T09:14:15.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.586+0000 7ff2b0518740 10 client.17652 fetch_fsmap finished waiting for FSMap version 47 2024-08-30T09:14:15.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.586+0000 7ff2b0518740 10 client.17652 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-30T09:14:15.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.586+0000 7ff2b0518740 20 client.17652 populate_metadata read hostname 'smithi042' 2024-08-30T09:14:15.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.586+0000 7ff2b0518740 1 --2- 172.21.15.42:0/1463208270 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55fbd97f8da0 0x55fbd9c95720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:14:15.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.586+0000 7ff2b0518740 4 client.17652 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-30T09:14:15.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.586+0000 7ff2b0518740 1 -- 172.21.15.42:0/1463208270 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55fbd9b83580 con 0x55fbd97f8da0 2024-08-30T09:14:15.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.586+0000 7ff2a37fe640 1 --2- 172.21.15.42:0/1463208270 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55fbd97f8da0 0x55fbd9c95720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:14:15.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.586+0000 7ff282ffd640 10 client.17652 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:14:15.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.586+0000 7ff2a37fe640 1 --2- 172.21.15.42:0/1463208270 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55fbd97f8da0 0x55fbd9c95720 crc :-1 s=READY pgs=276 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d0c695a5f7fe9aa0 server_cookie=faf501ba6ab8c2f7 in_seq=0 out_seq=0 2024-08-30T09:14:15.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.587+0000 7ff282ffd640 1 -- 172.21.15.42:0/1463208270 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+4634 (crc 0 0 0) 0x55fbd9b83580 con 0x55fbd97f8da0 2024-08-30T09:14:15.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.587+0000 7ff282ffd640 10 client.17652 handle_command_reply: tid=0 2024-08-30T09:14:15.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.587+0000 7ff2b0518740 1 client.17652 shutdown 2024-08-30T09:14:15.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.587+0000 7ff2b0518740 1 -- 172.21.15.42:0/1463208270 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fbd9865070 msgr2=0x55fbd9c1a900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:14:15.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.587+0000 7ff2b0518740 1 --2- 172.21.15.42:0/1463208270 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fbd9865070 0x55fbd9c1a900 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7ff29c079040 tx=0x7ff29c009500 comp rx=0 tx=0).stop 2024-08-30T09:14:15.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.587+0000 7ff2b0518740 1 -- 172.21.15.42:0/1463208270 shutdown_connections 2024-08-30T09:14:15.032 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-08-30T09:14:15.032 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "no active scrubs running", 2024-08-30T09:14:15.032 INFO:teuthology.orchestra.run.smithi042.stdout: "scrubs": {}, 2024-08-30T09:14:15.033 INFO:teuthology.orchestra.run.smithi042.stdout: "scrub_stats": { 2024-08-30T09:14:15.033 INFO:teuthology.orchestra.run.smithi042.stdout: "03521407-a747-49b9-a415-6233551c26fb": { 2024-08-30T09:14:15.033 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:13:39.387817+0000", 2024-08-30T09:14:15.033 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:14:15.033 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:14:15.033 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:14:15.033 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:14:15.033 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:14:15.033 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:14:15.033 INFO:teuthology.orchestra.run.smithi042.stdout: "0cbf734e-f988-4b21-acac-57b55e3bde62": { 2024-08-30T09:14:15.033 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:12:45.306738+0000", 2024-08-30T09:14:15.033 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:14:15.033 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:14:15.034 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:14:15.034 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:14:15.034 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:14:15.034 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:14:15.034 INFO:teuthology.orchestra.run.smithi042.stdout: "5e2ed104-0625-478a-a78b-584e7f4c2817": { 2024-08-30T09:14:15.034 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:00:33.928400+0000", 2024-08-30T09:14:15.034 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:14:15.034 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:14:15.034 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:14:15.034 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:14:15.034 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:14:15.034 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:14:15.034 INFO:teuthology.orchestra.run.smithi042.stdout: "c827ad7a-44ee-4e1d-9bb3-51fb7ab00c73": { 2024-08-30T09:14:15.034 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:02:21.686517+0000", 2024-08-30T09:14:15.034 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:14:15.035 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:14:15.035 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:14:15.035 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:14:15.035 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:14:15.035 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:14:15.035 INFO:teuthology.orchestra.run.smithi042.stdout: "85134e4b-ab4a-4885-b07e-a3ad01eaf13d": { 2024-08-30T09:14:15.035 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:01:27.738462+0000", 2024-08-30T09:14:15.035 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:14:15.035 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:14:15.035 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:14:15.035 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:14:15.035 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:14:15.035 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:14:15.035 INFO:teuthology.orchestra.run.smithi042.stdout: "749f4867-0b70-49e5-af2d-8362a240ebc3": { 2024-08-30T09:14:15.035 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:03:16.386554+0000", 2024-08-30T09:14:15.036 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:14:15.036 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:14:15.036 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:14:15.036 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:14:15.036 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:14:15.036 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:14:15.036 INFO:teuthology.orchestra.run.smithi042.stdout: "88cd7478-2b20-47de-ba51-256479749232": { 2024-08-30T09:14:15.036 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:04:08.922625+0000", 2024-08-30T09:14:15.036 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:14:15.036 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:14:15.036 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:14:15.036 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:14:15.036 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:14:15.037 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:14:15.037 INFO:teuthology.orchestra.run.smithi042.stdout: "347d05ed-4bd3-4590-b099-314f719aab39": { 2024-08-30T09:14:15.037 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:05:00.439721+0000", 2024-08-30T09:14:15.037 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:14:15.037 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:14:15.037 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:14:15.037 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:14:15.037 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:14:15.037 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:14:15.037 INFO:teuthology.orchestra.run.smithi042.stdout: "1af4252a-4a99-47e9-bbf0-becf5e573dd5": { 2024-08-30T09:14:15.037 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:05:48.565881+0000", 2024-08-30T09:14:15.037 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:14:15.037 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:14:15.038 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:14:15.038 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:14:15.038 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:14:15.038 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:14:15.038 INFO:teuthology.orchestra.run.smithi042.stdout: "665ec740-7254-49cb-84ef-deff71c461b0": { 2024-08-30T09:14:15.038 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:06:37.144035+0000", 2024-08-30T09:14:15.038 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:14:15.038 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:14:15.039 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:14:15.039 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:14:15.039 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:14:15.039 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:14:15.039 INFO:teuthology.orchestra.run.smithi042.stdout: "598d8a75-f1cc-4388-80bc-a408bdb4eaac": { 2024-08-30T09:14:15.039 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:07:29.721122+0000", 2024-08-30T09:14:15.039 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:14:15.039 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:14:15.039 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:14:15.039 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:14:15.039 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:14:15.039 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:14:15.039 INFO:teuthology.orchestra.run.smithi042.stdout: "b62c4ffc-9387-4677-b9ba-1e94ba308f17": { 2024-08-30T09:14:15.040 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:11:03.363478+0000", 2024-08-30T09:14:15.040 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:14:15.040 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:14:15.040 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:14:15.040 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:14:15.040 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:14:15.040 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:14:15.040 INFO:teuthology.orchestra.run.smithi042.stdout: "a7210f9a-046b-4188-953c-819b9f1863a9": { 2024-08-30T09:14:15.040 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:10:07.608430+0000", 2024-08-30T09:14:15.040 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:14:15.040 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:14:15.040 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:14:15.040 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:14:15.040 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:14:15.041 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:14:15.041 INFO:teuthology.orchestra.run.smithi042.stdout: "99a29ec1-9455-4ae4-854a-398b32896675": { 2024-08-30T09:14:15.041 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:08:24.923176+0000", 2024-08-30T09:14:15.041 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:14:15.041 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:14:15.041 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:14:15.041 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:14:15.041 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:14:15.041 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:14:15.041 INFO:teuthology.orchestra.run.smithi042.stdout: "ff464e7f-123b-4bc6-bc7e-aced018168de": { 2024-08-30T09:14:15.041 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:11:54.879598+0000", 2024-08-30T09:14:15.041 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:14:15.041 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:14:15.042 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:14:15.042 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:14:15.042 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:14:15.042 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:14:15.042 INFO:teuthology.orchestra.run.smithi042.stdout: "37f2f28f-496f-41a6-b4cf-8846fb3f43ab": { 2024-08-30T09:14:15.042 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:09:17.428286+0000", 2024-08-30T09:14:15.042 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:14:15.042 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:14:15.042 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:14:15.042 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:14:15.042 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:14:15.042 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:14:15.042 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:14:15.042 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-08-30T09:14:15.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.587+0000 7ff2b0518740 1 --2- 172.21.15.42:0/1463208270 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55fbd97f8da0 0x55fbd9c95720 crc :-1 s=READY pgs=276 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:15.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.587+0000 7ff2b0518740 1 --2- 172.21.15.42:0/1463208270 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fbd9865070 0x55fbd9c1a900 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:15.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.587+0000 7ff2b0518740 1 --2- 172.21.15.42:0/1463208270 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fbd9844390 0x55fbd9c1a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:15.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.587+0000 7ff2b0518740 1 --2- 172.21.15.42:0/1463208270 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fbd983c1b0 0x55fbd9c19e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:15.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.587+0000 7ff2b0518740 1 -- 172.21.15.42:0/1463208270 >> 172.21.15.42:0/1463208270 conn(0x55fbd9859460 msgr2=0x55fbd9c157e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:14:15.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.587+0000 7ff282ffd640 0 client.17652 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:14:15.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.587+0000 7ff2b0518740 1 -- 172.21.15.42:0/1463208270 shutdown_connections 2024-08-30T09:14:15.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.587+0000 7ff2b0518740 1 -- 172.21.15.42:0/1463208270 wait complete. 2024-08-30T09:14:15.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.587+0000 7ff2b0518740 20 client.17652 trim_cache size 0 max 16384 2024-08-30T09:14:15.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.588+0000 7ff2a9fa3640 1 -- 172.21.15.42:0/318793998 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7ff2900b4d40 msgr2=0x7ff2900b7250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:14:15.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.588+0000 7ff2a9fa3640 1 --2- 172.21.15.42:0/318793998 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7ff2900b4d40 0x7ff2900b7250 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7ff294003eb0 tx=0x7ff294010fa0 comp rx=0 tx=0).stop 2024-08-30T09:14:15.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.588+0000 7ff2a9fa3640 1 -- 172.21.15.42:0/318793998 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff2a410a780 msgr2=0x7ff2a411c140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:14:15.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.588+0000 7ff2a9fa3640 1 --2- 172.21.15.42:0/318793998 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff2a410a780 0x7ff2a411c140 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7ff28c00ca80 tx=0x7ff28c00cf40 comp rx=0 tx=0).stop 2024-08-30T09:14:15.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.588+0000 7ff2a9fa3640 1 -- 172.21.15.42:0/318793998 shutdown_connections 2024-08-30T09:14:15.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.588+0000 7ff2a9fa3640 1 --2- 172.21.15.42:0/318793998 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7ff2900b4d40 0x7ff2900b7250 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:15.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.588+0000 7ff2a9fa3640 1 --2- 172.21.15.42:0/318793998 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff2a4111ce0 0x7ff2a4118420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:15.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.588+0000 7ff2a9fa3640 1 --2- 172.21.15.42:0/318793998 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff2a411c9d0 0x7ff2a4118960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:15.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.588+0000 7ff2a9fa3640 1 --2- 172.21.15.42:0/318793998 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff2a410a780 0x7ff2a411c140 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:15.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.588+0000 7ff2a9fa3640 1 -- 172.21.15.42:0/318793998 >> 172.21.15.42:0/318793998 conn(0x7ff2a4070c60 msgr2=0x7ff2a410b7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:14:15.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.588+0000 7ff2a9fa3640 1 -- 172.21.15.42:0/318793998 shutdown_connections 2024-08-30T09:14:15.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:14.588+0000 7ff2a9fa3640 1 -- 172.21.15.42:0/318793998 wait complete. 2024-08-30T09:14:15.953 INFO:teuthology.orchestra.run.smithi042.stderr:progress: 810.0 s, 428.2 tps, lat 239.710 ms stddev 286.745 2024-08-30T09:14:16.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:14:16 smithi190 ceph-mon[22001]: pgmap v778: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 3.9 MiB/s wr, 421 op/s 2024-08-30T09:14:16.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:14:16 smithi167 ceph-mon[22063]: pgmap v778: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 3.9 MiB/s wr, 421 op/s 2024-08-30T09:14:16.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:14:16 smithi042 ceph-mon[18652]: pgmap v778: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 3.9 MiB/s wr, 421 op/s 2024-08-30T09:14:17.833 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-08-30T09:14:17.833 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph fs dump --format=json 2024-08-30T09:14:18.216 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T09:14:18.664 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:14:18 smithi042 ceph-mon[18652]: pgmap v779: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 3.7 MiB/s wr, 400 op/s 2024-08-30T09:14:18.686 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:14:18 smithi190 ceph-mon[22001]: pgmap v779: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 3.7 MiB/s wr, 400 op/s 2024-08-30T09:14:18.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:14:18 smithi167 ceph-mon[22063]: pgmap v779: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 3.7 MiB/s wr, 400 op/s 2024-08-30T09:14:18.946 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:18.944+0000 7fcb7bcc3640 1 -- 172.21.15.42:0/4759019 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcb7411c9d0 msgr2=0x7fcb7411eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:14:18.947 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:18.944+0000 7fcb7bcc3640 1 --2- 172.21.15.42:0/4759019 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcb7411c9d0 0x7fcb7411eee0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fcb6c009ca0 tx=0x7fcb6c02a8f0 comp rx=0 tx=0).stop 2024-08-30T09:14:18.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:18.948+0000 7fcb7bcc3640 1 -- 172.21.15.42:0/4759019 shutdown_connections 2024-08-30T09:14:18.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:18.948+0000 7fcb7bcc3640 1 --2- 172.21.15.42:0/4759019 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcb7410a780 0x7fcb7410ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:18.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:18.948+0000 7fcb7bcc3640 1 --2- 172.21.15.42:0/4759019 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcb7411c9d0 0x7fcb7411eee0 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:18.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:18.948+0000 7fcb7bcc3640 1 --2- 172.21.15.42:0/4759019 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcb74111ce0 0x7fcb7410a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:18.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:18.948+0000 7fcb7bcc3640 1 -- 172.21.15.42:0/4759019 >> 172.21.15.42:0/4759019 conn(0x7fcb74070c60 msgr2=0x7fcb74071070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:14:18.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:18.948+0000 7fcb7bcc3640 1 -- 172.21.15.42:0/4759019 shutdown_connections 2024-08-30T09:14:18.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:18.948+0000 7fcb7bcc3640 1 -- 172.21.15.42:0/4759019 wait complete. 2024-08-30T09:14:18.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:18.949+0000 7fcb7bcc3640 1 Processor -- start 2024-08-30T09:14:18.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:18.949+0000 7fcb7bcc3640 1 -- start start 2024-08-30T09:14:18.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:18.949+0000 7fcb7bcc3640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcb7410a780 0x7fcb7411c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:14:18.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:18.949+0000 7fcb7bcc3640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcb74111ce0 0x7fcb74116570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:14:18.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:18.949+0000 7fcb7bcc3640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcb7411c9d0 0x7fcb74116bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:14:18.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:18.949+0000 7fcb7bcc3640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fcb74112a20 con 0x7fcb74111ce0 2024-08-30T09:14:18.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:18.949+0000 7fcb7bcc3640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fcb741128a0 con 0x7fcb7411c9d0 2024-08-30T09:14:18.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:18.949+0000 7fcb7bcc3640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fcb74112ba0 con 0x7fcb7410a780 2024-08-30T09:14:18.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:18.951+0000 7fcb79a5f640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcb7410a780 0x7fcb7411c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:14:18.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:18.951+0000 7fcb79a5f640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcb7410a780 0x7fcb7411c2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:36148/0 (socket says 172.21.15.42:36148) 2024-08-30T09:14:18.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:18.951+0000 7fcb79a5f640 1 -- 172.21.15.42:0/610700085 learned_addr learned my addr 172.21.15.42:0/610700085 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:14:18.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:18.951+0000 7fcb7a260640 1 --2- 172.21.15.42:0/610700085 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcb7411c9d0 0x7fcb74116bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:14:18.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:18.951+0000 7fcb7925e640 1 --2- 172.21.15.42:0/610700085 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcb74111ce0 0x7fcb74116570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:14:18.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:18.952+0000 7fcb7925e640 1 -- 172.21.15.42:0/610700085 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcb7410a780 msgr2=0x7fcb7411c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:14:18.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:18.952+0000 7fcb7925e640 1 --2- 172.21.15.42:0/610700085 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcb7410a780 0x7fcb7411c2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:18.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:18.952+0000 7fcb7925e640 1 -- 172.21.15.42:0/610700085 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcb7411c9d0 msgr2=0x7fcb74116bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:14:18.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:18.952+0000 7fcb7925e640 1 --2- 172.21.15.42:0/610700085 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcb7411c9d0 0x7fcb74116bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:18.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:18.952+0000 7fcb7925e640 1 -- 172.21.15.42:0/610700085 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb741bda90 con 0x7fcb74111ce0 2024-08-30T09:14:18.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:18.952+0000 7fcb7925e640 1 --2- 172.21.15.42:0/610700085 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcb74111ce0 0x7fcb74116570 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fcb7406c480 tx=0x7fcb6400ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:14:18.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:18.956+0000 7fcb6affd640 1 -- 172.21.15.42:0/610700085 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb6400cc40 con 0x7fcb74111ce0 2024-08-30T09:14:18.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:18.956+0000 7fcb7bcc3640 1 -- 172.21.15.42:0/610700085 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb741bdc60 con 0x7fcb74111ce0 2024-08-30T09:14:18.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:18.957+0000 7fcb7bcc3640 1 -- 172.21.15.42:0/610700085 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb741be140 con 0x7fcb74111ce0 2024-08-30T09:14:18.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:18.958+0000 7fcb6affd640 1 -- 172.21.15.42:0/610700085 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb6400cde0 con 0x7fcb74111ce0 2024-08-30T09:14:18.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:18.958+0000 7fcb6affd640 1 -- 172.21.15.42:0/610700085 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb64010430 con 0x7fcb74111ce0 2024-08-30T09:14:18.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:18.962+0000 7fcb7bcc3640 1 -- 172.21.15.42:0/610700085 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb580052c0 con 0x7fcb74111ce0 2024-08-30T09:14:18.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:18.966+0000 7fcb6affd640 1 -- 172.21.15.42:0/610700085 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7fcb640035f0 con 0x7fcb74111ce0 2024-08-30T09:14:18.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:18.967+0000 7fcb6affd640 1 --2- 172.21.15.42:0/610700085 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fcb600b4d40 0x7fcb600b7250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:14:18.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:18.967+0000 7fcb6affd640 1 -- 172.21.15.42:0/610700085 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7fcb640e56c0 con 0x7fcb74111ce0 2024-08-30T09:14:18.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:18.975+0000 7fcb79a5f640 1 --2- 172.21.15.42:0/610700085 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fcb600b4d40 0x7fcb600b7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:14:18.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:18.981+0000 7fcb6affd640 1 -- 172.21.15.42:0/610700085 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7fcb640ac400 con 0x7fcb74111ce0 2024-08-30T09:14:18.988 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:18.986+0000 7fcb79a5f640 1 --2- 172.21.15.42:0/610700085 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fcb600b4d40 0x7fcb600b7250 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fcb7411c000 tx=0x7fcb70005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:14:19.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:19.397+0000 7fcb7bcc3640 1 -- 172.21.15.42:0/610700085 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcb580055b0 con 0x7fcb74111ce0 2024-08-30T09:14:19.401 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:19.399+0000 7fcb6affd640 1 -- 172.21.15.42:0/610700085 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) ==== 94+0+11921 (secure 0 0 0) 0x7fcb640abd00 con 0x7fcb74111ce0 2024-08-30T09:14:19.401 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-08-30T09:14:19.402 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":47,"btime":"2024-08-30T09:13:53:043532+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14703,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/3188400000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":3188400000},{"type":"v1","addr":"172.21.15.42:6835","nonce":3188400000}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14715,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/3356537674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":3356537674},{"type":"v1","addr":"172.21.15.42:6837","nonce":3356537674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14766,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6839/2107785513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6838","nonce":2107785513},{"type":"v1","addr":"172.21.15.42:6839","nonce":2107785513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14778,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6841/2298438674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6840","nonce":2298438674},{"type":"v1","addr":"172.21.15.42:6841","nonce":2298438674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24553,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/543555570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":543555570},{"type":"v1","addr":"172.21.15.190:6833","nonce":543555570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24554,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/3907818941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":3907818941},{"type":"v1","addr":"172.21.15.167:6833","nonce":3907818941}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":47,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-30T08:58:03.055084+0000","modified":"2024-08-30T09:13:52.652648+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24629,"mds_1":24610,"mds_2":24599},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24572":{"gid":24572,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6835/234596845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":234596845},{"type":"v1","addr":"172.21.15.167:6835","nonce":234596845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24592":{"gid":24592,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6835/247905709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":247905709},{"type":"v1","addr":"172.21.15.190:6835","nonce":247905709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24598":{"gid":24598,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6837/863256349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6836","nonce":863256349},{"type":"v1","addr":"172.21.15.190:6837","nonce":863256349}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24599":{"gid":24599,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.167:6837/3011837856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":3011837856},{"type":"v1","addr":"172.21.15.167:6837","nonce":3011837856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24610":{"gid":24610,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.190:6839/719981110","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6838","nonce":719981110},{"type":"v1","addr":"172.21.15.190:6839","nonce":719981110}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24629":{"gid":24629,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.167:6839/3604645207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":3604645207},{"type":"v1","addr":"172.21.15.167:6839","nonce":3604645207}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24629,"qdb_cluster":[24599,24610,24629]},"id":1}]} 2024-08-30T09:14:19.402 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 47 2024-08-30T09:14:19.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:19.403+0000 7fcb68ff9640 1 -- 172.21.15.42:0/610700085 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fcb600b4d40 msgr2=0x7fcb600b7250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:14:19.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:19.404+0000 7fcb68ff9640 1 --2- 172.21.15.42:0/610700085 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fcb600b4d40 0x7fcb600b7250 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fcb7411c000 tx=0x7fcb70005f90 comp rx=0 tx=0).stop 2024-08-30T09:14:19.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:19.404+0000 7fcb68ff9640 1 -- 172.21.15.42:0/610700085 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcb74111ce0 msgr2=0x7fcb74116570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:14:19.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:19.404+0000 7fcb68ff9640 1 --2- 172.21.15.42:0/610700085 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcb74111ce0 0x7fcb74116570 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fcb7406c480 tx=0x7fcb6400ee50 comp rx=0 tx=0).stop 2024-08-30T09:14:19.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:19.404+0000 7fcb68ff9640 1 -- 172.21.15.42:0/610700085 shutdown_connections 2024-08-30T09:14:19.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:19.404+0000 7fcb68ff9640 1 --2- 172.21.15.42:0/610700085 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fcb600b4d40 0x7fcb600b7250 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:19.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:19.404+0000 7fcb68ff9640 1 --2- 172.21.15.42:0/610700085 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcb7411c9d0 0x7fcb74116bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:19.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:19.404+0000 7fcb68ff9640 1 --2- 172.21.15.42:0/610700085 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcb74111ce0 0x7fcb74116570 secure :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fcb7406c480 tx=0x7fcb6400ee50 comp rx=0 tx=0).stop 2024-08-30T09:14:19.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:19.404+0000 7fcb68ff9640 1 --2- 172.21.15.42:0/610700085 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcb7410a780 0x7fcb7411c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:19.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:19.404+0000 7fcb68ff9640 1 -- 172.21.15.42:0/610700085 >> 172.21.15.42:0/610700085 conn(0x7fcb74070c60 msgr2=0x7fcb7410b7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:14:19.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:19.404+0000 7fcb68ff9640 1 -- 172.21.15.42:0/610700085 shutdown_connections 2024-08-30T09:14:19.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:19.404+0000 7fcb68ff9640 1 -- 172.21.15.42:0/610700085 wait complete. 2024-08-30T09:14:19.733 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:14:19 smithi042 ceph-mon[18652]: from='client.? 172.21.15.42:0/610700085' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-30T09:14:19.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:14:19 smithi167 ceph-mon[22063]: from='client.? 172.21.15.42:0/610700085' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-30T09:14:19.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:14:19 smithi190 ceph-mon[22001]: from='client.? 172.21.15.42:0/610700085' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-30T09:14:20.142 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-08-30T09:14:20.578 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T09:14:20.721 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:14:20 smithi042 ceph-mon[18652]: pgmap v780: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 4.9 MiB/s wr, 512 op/s 2024-08-30T09:14:20.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:14:20 smithi167 ceph-mon[22063]: pgmap v780: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 4.9 MiB/s wr, 512 op/s 2024-08-30T09:14:20.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:14:20 smithi190 ceph-mon[22001]: pgmap v780: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 4.9 MiB/s wr, 512 op/s 2024-08-30T09:14:20.968 INFO:teuthology.orchestra.run.smithi042.stderr:progress: 815.0 s, 487.8 tps, lat 208.649 ms stddev 252.545 2024-08-30T09:14:22.578 INFO:teuthology.orchestra.run.smithi042.stdout:[] 2024-08-30T09:14:22.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.248+0000 7f227b2c9640 1 -- 172.21.15.42:0/3746547496 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f227411c9d0 msgr2=0x7f227411eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:14:22.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.248+0000 7f227b2c9640 1 --2- 172.21.15.42:0/3746547496 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f227411c9d0 0x7f227411eee0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f226c009be0 tx=0x7f226c02a710 comp rx=0 tx=0).stop 2024-08-30T09:14:22.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.248+0000 7f227b2c9640 1 -- 172.21.15.42:0/3746547496 shutdown_connections 2024-08-30T09:14:22.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.248+0000 7f227b2c9640 1 --2- 172.21.15.42:0/3746547496 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f227410a780 0x7f227410ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:22.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.249+0000 7f227b2c9640 1 --2- 172.21.15.42:0/3746547496 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f227411c9d0 0x7f227411eee0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:22.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.249+0000 7f227b2c9640 1 --2- 172.21.15.42:0/3746547496 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2274111ce0 0x7f227410a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:22.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.249+0000 7f227b2c9640 1 -- 172.21.15.42:0/3746547496 >> 172.21.15.42:0/3746547496 conn(0x7f2274070c60 msgr2=0x7f2274071070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:14:22.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.253+0000 7f227b2c9640 1 -- 172.21.15.42:0/3746547496 shutdown_connections 2024-08-30T09:14:22.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.253+0000 7f227b2c9640 1 -- 172.21.15.42:0/3746547496 wait complete. 2024-08-30T09:14:22.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.254+0000 7f227b2c9640 1 Processor -- start 2024-08-30T09:14:22.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.254+0000 7f227b2c9640 1 -- start start 2024-08-30T09:14:22.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.254+0000 7f227b2c9640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f227410a780 0x7f22741a9020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:14:22.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.255+0000 7f227b2c9640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2274111ce0 0x7f22741a95c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:14:22.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.255+0000 7f227b2c9640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f227411c9d0 0x7f22741ae860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:14:22.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.255+0000 7f227b2c9640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f2274122690 con 0x7f2274111ce0 2024-08-30T09:14:22.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.255+0000 7f227b2c9640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f2274122510 con 0x7f227411c9d0 2024-08-30T09:14:22.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.255+0000 7f227b2c9640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f2274122810 con 0x7f227410a780 2024-08-30T09:14:22.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.255+0000 7f2279065640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f227410a780 0x7f22741a9020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:14:22.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.255+0000 7f2279065640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f227410a780 0x7f22741a9020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:36164/0 (socket says 172.21.15.42:36164) 2024-08-30T09:14:22.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.255+0000 7f2279065640 1 -- 172.21.15.42:0/1176884291 learned_addr learned my addr 172.21.15.42:0/1176884291 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:14:22.580 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.255+0000 7f2279866640 1 --2- 172.21.15.42:0/1176884291 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f227411c9d0 0x7f22741ae860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:14:22.580 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.255+0000 7f2278864640 1 --2- 172.21.15.42:0/1176884291 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2274111ce0 0x7f22741a95c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:14:22.580 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.255+0000 7f2279065640 1 -- 172.21.15.42:0/1176884291 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f227411c9d0 msgr2=0x7f22741ae860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:14:22.580 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.255+0000 7f2279065640 1 --2- 172.21.15.42:0/1176884291 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f227411c9d0 0x7f22741ae860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:22.580 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.255+0000 7f2279065640 1 -- 172.21.15.42:0/1176884291 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2274111ce0 msgr2=0x7f22741a95c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:14:22.580 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.255+0000 7f2279065640 1 --2- 172.21.15.42:0/1176884291 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2274111ce0 0x7f22741a95c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:22.580 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.256+0000 7f2279065640 1 -- 172.21.15.42:0/1176884291 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22741aef60 con 0x7f227410a780 2024-08-30T09:14:22.580 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.262+0000 7f2279065640 1 --2- 172.21.15.42:0/1176884291 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f227410a780 0x7f22741a9020 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f2270009f20 tx=0x7f227000edd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:14:22.580 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.262+0000 7f226a7fc640 1 -- 172.21.15.42:0/1176884291 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f227000ccc0 con 0x7f227410a780 2024-08-30T09:14:22.580 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.262+0000 7f226a7fc640 1 -- 172.21.15.42:0/1176884291 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f227000ce60 con 0x7f227410a780 2024-08-30T09:14:22.580 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.262+0000 7f226a7fc640 1 -- 172.21.15.42:0/1176884291 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2270032440 con 0x7f227410a780 2024-08-30T09:14:22.580 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.263+0000 7f227b2c9640 1 -- 172.21.15.42:0/1176884291 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22741af250 con 0x7f227410a780 2024-08-30T09:14:22.580 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.263+0000 7f227b2c9640 1 -- 172.21.15.42:0/1176884291 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2274112120 con 0x7f227410a780 2024-08-30T09:14:22.580 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.267+0000 7f228183e740 1 Processor -- start 2024-08-30T09:14:22.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.267+0000 7f228183e740 1 -- start start 2024-08-30T09:14:22.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.267+0000 7f228183e740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559440b91070 0x559440f365f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:14:22.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.267+0000 7f228183e740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559440b681b0 0x559440f36b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:14:22.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.267+0000 7f228183e740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559440b70390 0x559440f37070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:14:22.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.267+0000 7f228183e740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x559440ead680 con 0x559440b91070 2024-08-30T09:14:22.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.267+0000 7f228183e740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x559440e9cc10 con 0x559440b70390 2024-08-30T09:14:22.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.267+0000 7f228183e740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x559440ea0bc0 con 0x559440b681b0 2024-08-30T09:14:22.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.268+0000 7f2279866640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559440b91070 0x559440f365f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:14:22.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.268+0000 7f2279866640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559440b91070 0x559440f365f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60476/0 (socket says 172.21.15.42:60476) 2024-08-30T09:14:22.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.268+0000 7f2279866640 1 -- 172.21.15.42:0/156266720 learned_addr learned my addr 172.21.15.42:0/156266720 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:14:22.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.269+0000 7f2278864640 1 --2- 172.21.15.42:0/156266720 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559440b70390 0x559440f37070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:14:22.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.269+0000 7f2279866640 1 -- 172.21.15.42:0/156266720 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559440b681b0 msgr2=0x559440f36b30 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-30T09:14:22.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.269+0000 7f2279866640 1 --2- 172.21.15.42:0/156266720 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559440b681b0 0x559440f36b30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:22.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.269+0000 7f2279866640 1 -- 172.21.15.42:0/156266720 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559440b70390 msgr2=0x559440f37070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:14:22.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.269+0000 7f2279866640 1 --2- 172.21.15.42:0/156266720 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559440b70390 0x559440f37070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:22.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.269+0000 7f2279866640 1 -- 172.21.15.42:0/156266720 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559440bfc650 con 0x559440b91070 2024-08-30T09:14:22.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.272+0000 7f226a7fc640 1 -- 172.21.15.42:0/1176884291 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7f22700046c0 con 0x7f227410a780 2024-08-30T09:14:22.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.272+0000 7f2279866640 1 --2- 172.21.15.42:0/156266720 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559440b91070 0x559440f365f0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f226c033000 tx=0x7f226c0303a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57e9d46e114d1292 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:14:22.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.273+0000 7f226a7fc640 1 --2- 172.21.15.42:0/1176884291 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f22600b4c60 0x7f22600b7170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:14:22.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.273+0000 7f226a7fc640 1 -- 172.21.15.42:0/1176884291 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f2270107a70 con 0x7f227410a780 2024-08-30T09:14:22.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.274+0000 7f2279065640 1 --2- 172.21.15.42:0/1176884291 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f22600b4c60 0x7f22600b7170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:14:22.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.274+0000 7f2279065640 1 --2- 172.21.15.42:0/1176884291 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f22600b4c60 0x7f22600b7170 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f227000c6a0 tx=0x7f22700b5040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:14:22.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.276+0000 7f225effd640 1 -- 172.21.15.42:0/156266720 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f226c043070 con 0x559440b91070 2024-08-30T09:14:22.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.276+0000 7f225effd640 1 -- 172.21.15.42:0/156266720 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f226c003970 con 0x559440b91070 2024-08-30T09:14:22.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.277+0000 7f228183e740 1 -- 172.21.15.42:0/156266720 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559440b91070 msgr2=0x559440f365f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:14:22.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.277+0000 7f228183e740 1 --2- 172.21.15.42:0/156266720 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559440b91070 0x559440f365f0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f226c033000 tx=0x7f226c0303a0 comp rx=0 tx=0).stop 2024-08-30T09:14:22.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.277+0000 7f228183e740 1 -- 172.21.15.42:0/156266720 shutdown_connections 2024-08-30T09:14:22.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.277+0000 7f228183e740 1 --2- 172.21.15.42:0/156266720 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559440b681b0 0x559440f36b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:22.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.277+0000 7f228183e740 1 --2- 172.21.15.42:0/156266720 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559440b70390 0x559440f37070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:22.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.277+0000 7f228183e740 1 --2- 172.21.15.42:0/156266720 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559440b91070 0x559440f365f0 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:22.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.277+0000 7f228183e740 1 -- 172.21.15.42:0/156266720 >> 172.21.15.42:0/156266720 conn(0x559440b85460 msgr2=0x559440f344e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:14:22.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.278+0000 7f228183e740 1 -- 172.21.15.42:0/156266720 shutdown_connections 2024-08-30T09:14:22.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.278+0000 7f228183e740 1 -- 172.21.15.42:0/156266720 wait complete. 2024-08-30T09:14:22.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.278+0000 7f228183e740 1 -- start start 2024-08-30T09:14:22.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.278+0000 7f228183e740 1 Processor -- start 2024-08-30T09:14:22.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.279+0000 7f228183e740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559440b681b0 0x559440f44310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:14:22.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.279+0000 7f228183e740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559440b70390 0x559440f417e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:14:22.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.279+0000 7f228183e740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559440b4f9f0 0x559440f41e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:14:22.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.279+0000 7f228183e740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x559440cff700 con 0x559440b4f9f0 2024-08-30T09:14:22.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.279+0000 7f228183e740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x559440e23750 con 0x559440b681b0 2024-08-30T09:14:22.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.279+0000 7f228183e740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x559440f44d30 con 0x559440b70390 2024-08-30T09:14:22.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.279+0000 7f228183e740 10 client.0 _ll_register_callbacks cb 0x559440e52e10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:14:22.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.281+0000 7f2279866640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559440b681b0 0x559440f44310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:14:22.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.281+0000 7f2279866640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559440b681b0 0x559440f44310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:54610/0 (socket says 172.21.15.42:54610) 2024-08-30T09:14:22.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.281+0000 7f2279866640 1 -- 172.21.15.42:0/1150910439 learned_addr learned my addr 172.21.15.42:0/1150910439 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:14:22.584 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:14:22 smithi042 ceph-mon[18652]: pgmap v781: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 3.7 MiB/s wr, 406 op/s 2024-08-30T09:14:22.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.281+0000 7f2279866640 1 --2- 172.21.15.42:0/1150910439 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559440b4f9f0 0x559440f41e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:14:22.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.281+0000 7f2279866640 1 -- 172.21.15.42:0/1150910439 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559440b70390 msgr2=0x559440f417e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-30T09:14:22.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.281+0000 7f2279866640 1 --2- 172.21.15.42:0/1150910439 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559440b70390 0x559440f417e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:22.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.281+0000 7f2279866640 1 -- 172.21.15.42:0/1150910439 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559440b4f9f0 msgr2=0x559440f41e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:14:22.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.281+0000 7f2279866640 1 --2- 172.21.15.42:0/1150910439 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559440b4f9f0 0x559440f41e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:22.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.281+0000 7f2279866640 1 -- 172.21.15.42:0/1150910439 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559440e5c900 con 0x559440b681b0 2024-08-30T09:14:22.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.283+0000 7f2279866640 1 --2- 172.21.15.42:0/1150910439 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559440b681b0 0x559440f44310 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f226c0099a0 tx=0x7f226c005cf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9b75332d69137ea8 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:14:22.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.283+0000 7f225cff9640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-30T09:14:22.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.283+0000 7f225cff9640 1 -- 172.21.15.42:0/1150910439 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f226c043030 con 0x559440b681b0 2024-08-30T09:14:22.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.283+0000 7f228183e740 1 -- 172.21.15.42:0/1150910439 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x559440eaf5d0 con 0x559440b681b0 2024-08-30T09:14:22.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.283+0000 7f225cff9640 1 -- 172.21.15.42:0/1150910439 <== mon.2 v2:172.21.15.167:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f226c030af0 con 0x559440b681b0 2024-08-30T09:14:22.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.284+0000 7f225cff9640 1 -- 172.21.15.42:0/1150910439 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f226c057050 con 0x559440b681b0 2024-08-30T09:14:22.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.284+0000 7f228183e740 10 client.27509 fetch_fsmap learned FSMap version 47 2024-08-30T09:14:22.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.284+0000 7f228183e740 1 -- 172.21.15.42:0/1150910439 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x559440f50660 con 0x559440b681b0 2024-08-30T09:14:22.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.285+0000 7f225cff9640 1 -- 172.21.15.42:0/1150910439 <== mon.2 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x559440eaf5d0 con 0x559440b681b0 2024-08-30T09:14:22.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.285+0000 7f225cff9640 1 -- 172.21.15.42:0/1150910439 <== mon.2 v2:172.21.15.167:3300/0 5 ==== fsmap(e 47) ==== 8370+0+0 (secure 0 0 0) 0x7f226c0044c0 con 0x559440b681b0 2024-08-30T09:14:22.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.285+0000 7f228183e740 10 client.27509 fetch_fsmap finished waiting for FSMap version 47 2024-08-30T09:14:22.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.285+0000 7f228183e740 10 client.27509 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-08-30T09:14:22.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.285+0000 7f228183e740 20 client.27509 populate_metadata read hostname 'smithi042' 2024-08-30T09:14:22.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.285+0000 7f228183e740 1 --2- 172.21.15.42:0/1150910439 >> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] conn(0x559440b24da0 0x559440f539c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:14:22.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.285+0000 7f228183e740 4 client.27509 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-30T09:14:22.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.285+0000 7f228183e740 1 -- 172.21.15.42:0/1150910439 --> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559440eab070 con 0x559440b24da0 2024-08-30T09:14:22.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.286+0000 7f2278864640 1 --2- 172.21.15.42:0/1150910439 >> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] conn(0x559440b24da0 0x559440f539c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:14:22.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.287+0000 7f2278864640 1 --2- 172.21.15.42:0/1150910439 >> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] conn(0x559440b24da0 0x559440f539c0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=df8e1716d4e6e773 server_cookie=74ec4cdc37a4cd6b in_seq=0 out_seq=0 2024-08-30T09:14:22.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.287+0000 7f225cff9640 10 client.27509 ms_handle_connect on v2:172.21.15.167:6836/3011837856 2024-08-30T09:14:22.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.288+0000 7f225cff9640 1 -- 172.21.15.42:0/1150910439 <== mds.2 v2:172.21.15.167:6836/3011837856 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x559440eab070 con 0x559440b24da0 2024-08-30T09:14:22.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.288+0000 7f225cff9640 10 client.27509 handle_command_reply: tid=0 2024-08-30T09:14:22.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.288+0000 7f228183e740 1 client.27509 shutdown 2024-08-30T09:14:22.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.289+0000 7f228183e740 1 -- 172.21.15.42:0/1150910439 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559440b681b0 msgr2=0x559440f44310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:14:22.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.289+0000 7f228183e740 1 --2- 172.21.15.42:0/1150910439 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559440b681b0 0x559440f44310 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f226c0099a0 tx=0x7f226c005cf0 comp rx=0 tx=0).stop 2024-08-30T09:14:22.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.289+0000 7f228183e740 1 -- 172.21.15.42:0/1150910439 shutdown_connections 2024-08-30T09:14:22.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.289+0000 7f228183e740 1 --2- 172.21.15.42:0/1150910439 >> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] conn(0x559440b24da0 0x559440f539c0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:22.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.289+0000 7f228183e740 1 --2- 172.21.15.42:0/1150910439 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559440b70390 0x559440f417e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:22.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.289+0000 7f228183e740 1 --2- 172.21.15.42:0/1150910439 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559440b4f9f0 0x559440f41e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:22.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.289+0000 7f228183e740 1 --2- 172.21.15.42:0/1150910439 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559440b681b0 0x559440f44310 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:22.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.289+0000 7f228183e740 1 -- 172.21.15.42:0/1150910439 >> 172.21.15.42:0/1150910439 conn(0x559440b85460 msgr2=0x559440f397d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:14:22.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.289+0000 7f225cff9640 0 client.27509 ms_handle_reset on v2:172.21.15.167:6836/3011837856 2024-08-30T09:14:22.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.289+0000 7f228183e740 1 -- 172.21.15.42:0/1150910439 shutdown_connections 2024-08-30T09:14:22.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.289+0000 7f228183e740 1 -- 172.21.15.42:0/1150910439 wait complete. 2024-08-30T09:14:22.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.289+0000 7f228183e740 20 client.27509 trim_cache size 0 max 16384 2024-08-30T09:14:22.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.314+0000 7f228183e740 1 Processor -- start 2024-08-30T09:14:22.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.315+0000 7f228183e740 1 -- start start 2024-08-30T09:14:22.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.315+0000 7f228183e740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559440b4f9f0 0x559440f3d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:14:22.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.315+0000 7f228183e740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559440b681b0 0x559440f37fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:14:22.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.315+0000 7f228183e740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559440b70390 0x559440f43410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:14:22.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.315+0000 7f228183e740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f22741b9960 con 0x559440b70390 2024-08-30T09:14:22.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.315+0000 7f228183e740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f22741b71d0 con 0x559440b681b0 2024-08-30T09:14:22.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.315+0000 7f228183e740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f22741aa150 con 0x559440b4f9f0 2024-08-30T09:14:22.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.315+0000 7f2279866640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559440b4f9f0 0x559440f3d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:14:22.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.315+0000 7f2279866640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559440b4f9f0 0x559440f3d170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:36194/0 (socket says 172.21.15.42:36194) 2024-08-30T09:14:22.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.315+0000 7f2279866640 1 -- 172.21.15.42:0/3793666431 learned_addr learned my addr 172.21.15.42:0/3793666431 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:14:22.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.315+0000 7f2279866640 1 --2- 172.21.15.42:0/3793666431 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559440b70390 0x559440f43410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:14:22.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.315+0000 7f2278864640 1 --2- 172.21.15.42:0/3793666431 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559440b681b0 0x559440f37fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:14:22.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.316+0000 7f2279866640 1 -- 172.21.15.42:0/3793666431 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559440b681b0 msgr2=0x559440f37fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:14:22.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.316+0000 7f2279866640 1 --2- 172.21.15.42:0/3793666431 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559440b681b0 0x559440f37fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:22.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.316+0000 7f2279866640 1 -- 172.21.15.42:0/3793666431 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559440b70390 msgr2=0x559440f43410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:14:22.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.316+0000 7f2279866640 1 --2- 172.21.15.42:0/3793666431 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559440b70390 0x559440f43410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:22.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.316+0000 7f2279866640 1 -- 172.21.15.42:0/3793666431 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559440e69e80 con 0x559440b4f9f0 2024-08-30T09:14:22.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.316+0000 7f2279866640 1 --2- 172.21.15.42:0/3793666431 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559440b4f9f0 0x559440f3d170 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f226c002230 tx=0x7f226c005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2b2f7d1d48c0b0fa server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:14:22.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.316+0000 7f225e7fc640 1 -- 172.21.15.42:0/3793666431 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f226c043070 con 0x559440b4f9f0 2024-08-30T09:14:22.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.316+0000 7f225e7fc640 1 -- 172.21.15.42:0/3793666431 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f226c03d460 con 0x559440b4f9f0 2024-08-30T09:14:22.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.316+0000 7f225e7fc640 1 -- 172.21.15.42:0/3793666431 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f226c047660 con 0x559440b4f9f0 2024-08-30T09:14:22.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.317+0000 7f228183e740 1 -- 172.21.15.42:0/3793666431 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559440b4f9f0 msgr2=0x559440f3d170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:14:22.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.317+0000 7f228183e740 1 --2- 172.21.15.42:0/3793666431 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559440b4f9f0 0x559440f3d170 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f226c002230 tx=0x7f226c005fd0 comp rx=0 tx=0).stop 2024-08-30T09:14:22.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.317+0000 7f228183e740 1 -- 172.21.15.42:0/3793666431 shutdown_connections 2024-08-30T09:14:22.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.317+0000 7f228183e740 1 --2- 172.21.15.42:0/3793666431 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559440b70390 0x559440f43410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:22.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.317+0000 7f228183e740 1 --2- 172.21.15.42:0/3793666431 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559440b681b0 0x559440f37fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:22.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.317+0000 7f228183e740 1 --2- 172.21.15.42:0/3793666431 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559440b4f9f0 0x559440f3d170 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:22.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.317+0000 7f228183e740 1 -- 172.21.15.42:0/3793666431 >> 172.21.15.42:0/3793666431 conn(0x559440b85460 msgr2=0x559440f42d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:14:22.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.318+0000 7f228183e740 1 -- 172.21.15.42:0/3793666431 shutdown_connections 2024-08-30T09:14:22.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.318+0000 7f228183e740 1 -- 172.21.15.42:0/3793666431 wait complete. 2024-08-30T09:14:22.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.318+0000 7f228183e740 1 -- start start 2024-08-30T09:14:22.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.318+0000 7f228183e740 1 Processor -- start 2024-08-30T09:14:22.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.319+0000 7f228183e740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559440b4f9f0 0x559440f56f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:14:22.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.319+0000 7f228183e740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559440b681b0 0x559440f543d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:14:22.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.319+0000 7f2279866640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559440b4f9f0 0x559440f56f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:14:22.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.319+0000 7f2279866640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559440b4f9f0 0x559440f56f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60512/0 (socket says 172.21.15.42:60512) 2024-08-30T09:14:22.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.319+0000 7f2279866640 1 -- 172.21.15.42:0/1828355153 learned_addr learned my addr 172.21.15.42:0/1828355153 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:14:22.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.319+0000 7f228183e740 1 --2- 172.21.15.42:0/1828355153 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559440b70390 0x559440f54910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:14:22.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.319+0000 7f228183e740 1 -- 172.21.15.42:0/1828355153 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f227411a650 con 0x559440b4f9f0 2024-08-30T09:14:22.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.319+0000 7f228183e740 1 -- 172.21.15.42:0/1828355153 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f22741b6b00 con 0x559440b70390 2024-08-30T09:14:22.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.319+0000 7f228183e740 1 -- 172.21.15.42:0/1828355153 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f22741b5d60 con 0x559440b681b0 2024-08-30T09:14:22.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.319+0000 7f2278864640 1 --2- 172.21.15.42:0/1828355153 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559440b681b0 0x559440f543d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:14:22.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.320+0000 7f2279866640 1 --2- 172.21.15.42:0/1828355153 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559440b70390 0x559440f54910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:14:22.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.320+0000 7f2279866640 1 -- 172.21.15.42:0/1828355153 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559440b681b0 msgr2=0x559440f543d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:14:22.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.320+0000 7f2279866640 1 --2- 172.21.15.42:0/1828355153 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559440b681b0 0x559440f543d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:22.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.320+0000 7f2279866640 1 -- 172.21.15.42:0/1828355153 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559440b70390 msgr2=0x559440f54910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:14:22.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.320+0000 7f2279866640 1 --2- 172.21.15.42:0/1828355153 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559440b70390 0x559440f54910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:22.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.320+0000 7f2279866640 1 -- 172.21.15.42:0/1828355153 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559440f54e50 con 0x559440b4f9f0 2024-08-30T09:14:22.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.320+0000 7f2279866640 1 --2- 172.21.15.42:0/1828355153 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559440b4f9f0 0x559440f56f00 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f226c009d00 tx=0x7f226c000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adcdfed3a6f4a31b server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:14:22.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.320+0000 7f2253fff640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-08-30T09:14:22.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.320+0000 7f228183e740 10 client.0 _ll_register_callbacks cb 0x7f226400c2d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:14:22.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.320+0000 7f2253fff640 1 -- 172.21.15.42:0/1828355153 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f226c043040 con 0x559440b4f9f0 2024-08-30T09:14:22.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.320+0000 7f2253fff640 1 -- 172.21.15.42:0/1828355153 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f226c047400 con 0x559440b4f9f0 2024-08-30T09:14:22.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.320+0000 7f228183e740 1 -- 172.21.15.42:0/1828355153 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x559440ea87b0 con 0x559440b4f9f0 2024-08-30T09:14:22.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.321+0000 7f2253fff640 1 -- 172.21.15.42:0/1828355153 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f226c047600 con 0x559440b4f9f0 2024-08-30T09:14:22.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.321+0000 7f228183e740 10 client.17676 fetch_fsmap learned FSMap version 47 2024-08-30T09:14:22.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.321+0000 7f228183e740 1 -- 172.21.15.42:0/1828355153 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({fsmap=47}) -- 0x559440fbdb30 con 0x559440b4f9f0 2024-08-30T09:14:22.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.321+0000 7f2253fff640 1 -- 172.21.15.42:0/1828355153 <== mon.0 v2:172.21.15.42:3300/0 4 ==== fsmap(e 47) ==== 8370+0+0 (secure 0 0 0) 0x7f226c05a070 con 0x559440b4f9f0 2024-08-30T09:14:22.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.321+0000 7f228183e740 10 client.17676 fetch_fsmap finished waiting for FSMap version 47 2024-08-30T09:14:22.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.321+0000 7f228183e740 10 client.17676 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-08-30T09:14:22.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.321+0000 7f228183e740 20 client.17676 populate_metadata read hostname 'smithi042' 2024-08-30T09:14:22.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.321+0000 7f228183e740 1 --2- 172.21.15.42:0/1828355153 >> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] conn(0x559440fbe090 0x559440fc04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:14:22.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.321+0000 7f228183e740 4 client.17676 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-30T09:14:22.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.321+0000 7f228183e740 1 -- 172.21.15.42:0/1828355153 --> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559440ea87b0 con 0x559440fbe090 2024-08-30T09:14:22.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.321+0000 7f2278864640 1 --2- 172.21.15.42:0/1828355153 >> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] conn(0x559440fbe090 0x559440fc04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:14:22.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.322+0000 7f2253fff640 10 client.17676 ms_handle_connect on v2:172.21.15.167:6836/3011837856 2024-08-30T09:14:22.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.322+0000 7f2278864640 1 --2- 172.21.15.42:0/1828355153 >> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] conn(0x559440fbe090 0x559440fc04b0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=483a346529a424e1 server_cookie=8bb4fe6400810fc1 in_seq=0 out_seq=0 2024-08-30T09:14:22.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.322+0000 7f2253fff640 1 -- 172.21.15.42:0/1828355153 <== mds.2 v2:172.21.15.167:6836/3011837856 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559440ea87b0 con 0x559440fbe090 2024-08-30T09:14:22.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.322+0000 7f2253fff640 10 client.17676 handle_command_reply: tid=0 2024-08-30T09:14:22.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.322+0000 7f228183e740 1 client.17676 shutdown 2024-08-30T09:14:22.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.322+0000 7f228183e740 1 -- 172.21.15.42:0/1828355153 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559440b4f9f0 msgr2=0x559440f56f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:14:22.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.322+0000 7f228183e740 1 --2- 172.21.15.42:0/1828355153 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559440b4f9f0 0x559440f56f00 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f226c009d00 tx=0x7f226c000f40 comp rx=0 tx=0).stop 2024-08-30T09:14:22.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.322+0000 7f228183e740 1 -- 172.21.15.42:0/1828355153 shutdown_connections 2024-08-30T09:14:22.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.322+0000 7f228183e740 1 --2- 172.21.15.42:0/1828355153 >> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] conn(0x559440fbe090 0x559440fc04b0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:22.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.322+0000 7f228183e740 1 --2- 172.21.15.42:0/1828355153 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559440b681b0 0x559440f543d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:22.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.322+0000 7f2253fff640 0 client.17676 ms_handle_reset on v2:172.21.15.167:6836/3011837856 2024-08-30T09:14:22.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.322+0000 7f228183e740 1 --2- 172.21.15.42:0/1828355153 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559440b70390 0x559440f54910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:22.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.322+0000 7f228183e740 1 --2- 172.21.15.42:0/1828355153 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559440b4f9f0 0x559440f56f00 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:22.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.322+0000 7f228183e740 1 -- 172.21.15.42:0/1828355153 >> 172.21.15.42:0/1828355153 conn(0x559440b85460 msgr2=0x559440f37d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:14:22.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.323+0000 7f228183e740 1 -- 172.21.15.42:0/1828355153 shutdown_connections 2024-08-30T09:14:22.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.323+0000 7f228183e740 1 -- 172.21.15.42:0/1828355153 wait complete. 2024-08-30T09:14:22.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.323+0000 7f228183e740 20 client.17676 trim_cache size 0 max 16384 2024-08-30T09:14:22.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.323+0000 7f225ffff640 1 -- 172.21.15.42:0/1176884291 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f22600b4c60 msgr2=0x7f22600b7170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:14:22.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.323+0000 7f225ffff640 1 --2- 172.21.15.42:0/1176884291 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f22600b4c60 0x7f22600b7170 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f227000c6a0 tx=0x7f22700b5040 comp rx=0 tx=0).stop 2024-08-30T09:14:22.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.323+0000 7f225ffff640 1 -- 172.21.15.42:0/1176884291 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f227410a780 msgr2=0x7f22741a9020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:14:22.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.323+0000 7f225ffff640 1 --2- 172.21.15.42:0/1176884291 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f227410a780 0x7f22741a9020 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f2270009f20 tx=0x7f227000edd0 comp rx=0 tx=0).stop 2024-08-30T09:14:22.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.323+0000 7f225ffff640 1 -- 172.21.15.42:0/1176884291 shutdown_connections 2024-08-30T09:14:22.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.323+0000 7f225ffff640 1 --2- 172.21.15.42:0/1176884291 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f22600b4c60 0x7f22600b7170 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:22.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.323+0000 7f225ffff640 1 --2- 172.21.15.42:0/1176884291 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f227411c9d0 0x7f22741ae860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:22.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.323+0000 7f225ffff640 1 --2- 172.21.15.42:0/1176884291 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2274111ce0 0x7f22741a95c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:22.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.323+0000 7f225ffff640 1 --2- 172.21.15.42:0/1176884291 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f227410a780 0x7f22741a9020 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:22.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.323+0000 7f225ffff640 1 -- 172.21.15.42:0/1176884291 >> 172.21.15.42:0/1176884291 conn(0x7f2274070c60 msgr2=0x7f227411e0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:14:22.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.324+0000 7f225ffff640 1 -- 172.21.15.42:0/1176884291 shutdown_connections 2024-08-30T09:14:22.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:22.324+0000 7f225ffff640 1 -- 172.21.15.42:0/1176884291 wait complete. 2024-08-30T09:14:22.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:14:22 smithi167 ceph-mon[22063]: pgmap v781: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 3.7 MiB/s wr, 406 op/s 2024-08-30T09:14:22.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:14:22 smithi190 ceph-mon[22001]: pgmap v781: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 3.7 MiB/s wr, 406 op/s 2024-08-30T09:14:23.409 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-08-30T09:14:23.813 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T09:14:24.400 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.397+0000 7fd821fe5640 1 -- 172.21.15.42:0/3227181433 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd81c10a160 msgr2=0x7fd81c10c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:14:24.400 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.397+0000 7fd821fe5640 1 --2- 172.21.15.42:0/3227181433 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd81c10a160 0x7fd81c10c580 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fd80c009860 tx=0x7fd80c02a360 comp rx=0 tx=0).stop 2024-08-30T09:14:24.405 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.402+0000 7fd821fe5640 1 -- 172.21.15.42:0/3227181433 shutdown_connections 2024-08-30T09:14:24.405 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.402+0000 7fd821fe5640 1 --2- 172.21.15.42:0/3227181433 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd81c10f5d0 0x7fd81c10fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:24.406 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.402+0000 7fd821fe5640 1 --2- 172.21.15.42:0/3227181433 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd81c10cac0 0x7fd81c10eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:24.406 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.402+0000 7fd821fe5640 1 --2- 172.21.15.42:0/3227181433 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd81c10a160 0x7fd81c10c580 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:24.406 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.402+0000 7fd821fe5640 1 -- 172.21.15.42:0/3227181433 >> 172.21.15.42:0/3227181433 conn(0x7fd81c0703e0 msgr2=0x7fd81c072890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:14:24.420 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.415+0000 7fd821fe5640 1 -- 172.21.15.42:0/3227181433 shutdown_connections 2024-08-30T09:14:24.420 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.417+0000 7fd821fe5640 1 -- 172.21.15.42:0/3227181433 wait complete. 2024-08-30T09:14:24.420 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.418+0000 7fd821fe5640 1 Processor -- start 2024-08-30T09:14:24.422 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.418+0000 7fd821fe5640 1 -- start start 2024-08-30T09:14:24.422 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.418+0000 7fd821fe5640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd81c10a160 0x7fd81c1a01a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:14:24.423 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.418+0000 7fd821fe5640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd81c10cac0 0x7fd81c1a0810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:14:24.423 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.418+0000 7fd821fe5640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd81c10f5d0 0x7fd81c1a13c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:14:24.423 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.418+0000 7fd821fe5640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fd81c113550 con 0x7fd81c10cac0 2024-08-30T09:14:24.423 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.418+0000 7fd821fe5640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd81c1133d0 con 0x7fd81c10a160 2024-08-30T09:14:24.423 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.418+0000 7fd821fe5640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fd81c1136d0 con 0x7fd81c10f5d0 2024-08-30T09:14:24.424 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.419+0000 7fd81bfff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd81c10cac0 0x7fd81c1a0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:14:24.424 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.419+0000 7fd81bfff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd81c10cac0 0x7fd81c1a0810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60538/0 (socket says 172.21.15.42:60538) 2024-08-30T09:14:24.424 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.419+0000 7fd81bfff640 1 -- 172.21.15.42:0/2999025708 learned_addr learned my addr 172.21.15.42:0/2999025708 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:14:24.424 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.419+0000 7fd820fe3640 1 --2- 172.21.15.42:0/2999025708 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd81c10a160 0x7fd81c1a01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:14:24.424 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.419+0000 7fd820fe3640 1 -- 172.21.15.42:0/2999025708 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd81c10f5d0 msgr2=0x7fd81c1a13c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:14:24.424 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.419+0000 7fd820fe3640 1 --2- 172.21.15.42:0/2999025708 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd81c10f5d0 0x7fd81c1a13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:24.425 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.420+0000 7fd820fe3640 1 -- 172.21.15.42:0/2999025708 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd81c10cac0 msgr2=0x7fd81c1a0810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:14:24.425 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.420+0000 7fd820fe3640 1 --2- 172.21.15.42:0/2999025708 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd81c10cac0 0x7fd81c1a0810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:24.425 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.420+0000 7fd820fe3640 1 -- 172.21.15.42:0/2999025708 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd81c1a1be0 con 0x7fd81c10a160 2024-08-30T09:14:24.425 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.420+0000 7fd820fe3640 1 --2- 172.21.15.42:0/2999025708 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd81c10a160 0x7fd81c1a01a0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fd80c009780 tx=0x7fd80c033e20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:14:24.426 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.421+0000 7fd819ffb640 1 -- 172.21.15.42:0/2999025708 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd80c03f030 con 0x7fd81c10a160 2024-08-30T09:14:24.426 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.421+0000 7fd821fe5640 1 -- 172.21.15.42:0/2999025708 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd81c1a1e70 con 0x7fd81c10a160 2024-08-30T09:14:24.427 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.421+0000 7fd821fe5640 1 -- 172.21.15.42:0/2999025708 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd81c1a85c0 con 0x7fd81c10a160 2024-08-30T09:14:24.427 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.424+0000 7fd82a28d740 1 Processor -- start 2024-08-30T09:14:24.429 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.425+0000 7fd82a28d740 1 -- start start 2024-08-30T09:14:24.429 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.425+0000 7fd82a28d740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560f1de071b0 0x560f1e1ad160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:14:24.429 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.425+0000 7fd82a28d740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560f1ddde2f0 0x560f1e1ad7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:14:24.429 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.425+0000 7fd82a28d740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560f1dde64d0 0x560f1e1ae320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:14:24.429 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.425+0000 7fd82a28d740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x560f1e120000 con 0x560f1de071b0 2024-08-30T09:14:24.429 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.425+0000 7fd82a28d740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x560f1e10f590 con 0x560f1dde64d0 2024-08-30T09:14:24.430 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.425+0000 7fd82a28d740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x560f1e113540 con 0x560f1ddde2f0 2024-08-30T09:14:24.430 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.425+0000 7fd819ffb640 1 -- 172.21.15.42:0/2999025708 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd80c004020 con 0x7fd81c10a160 2024-08-30T09:14:24.430 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.425+0000 7fd819ffb640 1 -- 172.21.15.42:0/2999025708 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd80c03f030 con 0x7fd81c10a160 2024-08-30T09:14:24.430 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.427+0000 7fd820fe3640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560f1ddde2f0 0x560f1e1ad7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:14:24.430 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.427+0000 7fd820fe3640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560f1ddde2f0 0x560f1e1ad7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:36244/0 (socket says 172.21.15.42:36244) 2024-08-30T09:14:24.430 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.427+0000 7fd820fe3640 1 -- 172.21.15.42:0/491232644 learned_addr learned my addr 172.21.15.42:0/491232644 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:14:24.432 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.427+0000 7fd820fe3640 1 -- 172.21.15.42:0/491232644 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560f1dde64d0 msgr2=0x560f1e1ae320 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-30T09:14:24.432 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.427+0000 7fd820fe3640 1 --2- 172.21.15.42:0/491232644 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560f1dde64d0 0x560f1e1ae320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:24.432 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.427+0000 7fd820fe3640 1 -- 172.21.15.42:0/491232644 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560f1de071b0 msgr2=0x560f1e1ad160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:14:24.432 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.427+0000 7fd820fe3640 1 --2- 172.21.15.42:0/491232644 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560f1de071b0 0x560f1e1ad160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:24.433 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.429+0000 7fd820fe3640 1 -- 172.21.15.42:0/491232644 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f1de72790 con 0x560f1ddde2f0 2024-08-30T09:14:24.440 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.435+0000 7fd819ffb640 1 -- 172.21.15.42:0/2999025708 <== mon.2 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7fd80c042070 con 0x7fd81c10a160 2024-08-30T09:14:24.441 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.436+0000 7fd819ffb640 1 --2- 172.21.15.42:0/2999025708 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fd8040b4f70 0x7fd8040b7480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:14:24.441 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.436+0000 7fd8217e4640 1 --2- 172.21.15.42:0/2999025708 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fd8040b4f70 0x7fd8040b7480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:14:24.441 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.437+0000 7fd8217e4640 1 --2- 172.21.15.42:0/2999025708 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fd8040b4f70 0x7fd8040b7480 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fd814005e30 tx=0x7fd814005da0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:14:24.445 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.442+0000 7fd820fe3640 1 --2- 172.21.15.42:0/491232644 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560f1ddde2f0 0x560f1e1ad7d0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fd80c0328b0 tx=0x7fd80c0328e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2f4cece509765d03 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:14:24.446 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.443+0000 7fd819ffb640 1 -- 172.21.15.42:0/2999025708 <== mon.2 v2:172.21.15.167:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7fd80c040070 con 0x7fd81c10a160 2024-08-30T09:14:24.448 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.446+0000 7fd802ffd640 1 -- 172.21.15.42:0/491232644 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd80c0c9480 con 0x560f1ddde2f0 2024-08-30T09:14:24.448 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.446+0000 7fd802ffd640 1 -- 172.21.15.42:0/491232644 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd80c03ce80 con 0x560f1ddde2f0 2024-08-30T09:14:24.448 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.446+0000 7fd802ffd640 1 -- 172.21.15.42:0/491232644 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd80c0c9620 con 0x560f1ddde2f0 2024-08-30T09:14:24.450 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.449+0000 7fd82a28d740 1 -- 172.21.15.42:0/491232644 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560f1ddde2f0 msgr2=0x560f1e1ad7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:14:24.450 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.449+0000 7fd82a28d740 1 --2- 172.21.15.42:0/491232644 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560f1ddde2f0 0x560f1e1ad7d0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fd80c0328b0 tx=0x7fd80c0328e0 comp rx=0 tx=0).stop 2024-08-30T09:14:24.453 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.451+0000 7fd82a28d740 1 -- 172.21.15.42:0/491232644 shutdown_connections 2024-08-30T09:14:24.453 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.451+0000 7fd82a28d740 1 --2- 172.21.15.42:0/491232644 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560f1ddde2f0 0x560f1e1ad7d0 secure :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fd80c0328b0 tx=0x7fd80c0328e0 comp rx=0 tx=0).stop 2024-08-30T09:14:24.453 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.451+0000 7fd82a28d740 1 --2- 172.21.15.42:0/491232644 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560f1dde64d0 0x560f1e1ae320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:24.453 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.451+0000 7fd82a28d740 1 --2- 172.21.15.42:0/491232644 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560f1de071b0 0x560f1e1ad160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:24.453 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.451+0000 7fd82a28d740 1 -- 172.21.15.42:0/491232644 >> 172.21.15.42:0/491232644 conn(0x560f1ddfb300 msgr2=0x560f1e1a8bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:14:24.463 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.461+0000 7fd82a28d740 1 -- 172.21.15.42:0/491232644 shutdown_connections 2024-08-30T09:14:24.467 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.465+0000 7fd82a28d740 1 -- 172.21.15.42:0/491232644 wait complete. 2024-08-30T09:14:24.468 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.465+0000 7fd82a28d740 1 -- start start 2024-08-30T09:14:24.468 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.465+0000 7fd82a28d740 1 Processor -- start 2024-08-30T09:14:24.472 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.469+0000 7fd82a28d740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560f1dde64d0 0x560f1e1b6920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:14:24.473 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.469+0000 7fd82a28d740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560f1de071b0 0x560f1e1b6fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:14:24.473 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.471+0000 7fd820fe3640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560f1dde64d0 0x560f1e1b6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:14:24.473 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.471+0000 7fd820fe3640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560f1dde64d0 0x560f1e1b6920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:36246/0 (socket says 172.21.15.42:36246) 2024-08-30T09:14:24.474 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.472+0000 7fd82a28d740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560f1ddc5d80 0x560f1e1b7b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:14:24.474 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.472+0000 7fd82a28d740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x560f1e048ab0 con 0x560f1de071b0 2024-08-30T09:14:24.474 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.472+0000 7fd82a28d740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x560f1e096680 con 0x560f1ddc5d80 2024-08-30T09:14:24.474 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.472+0000 7fd82a28d740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x560f1e1aec00 con 0x560f1dde64d0 2024-08-30T09:14:24.475 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.472+0000 7fd8217e4640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560f1ddc5d80 0x560f1e1b7b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:14:24.475 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.473+0000 7fd8217e4640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560f1ddc5d80 0x560f1e1b7b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:54670/0 (socket says 172.21.15.42:54670) 2024-08-30T09:14:24.475 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.473+0000 7fd8217e4640 1 -- 172.21.15.42:0/900802534 learned_addr learned my addr 172.21.15.42:0/900802534 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:14:24.475 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.473+0000 7fd82a28d740 10 client.0 _ll_register_callbacks cb 0x560f1de1ea00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:14:24.475 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.473+0000 7fd8217e4640 1 -- 172.21.15.42:0/900802534 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560f1dde64d0 msgr2=0x560f1e1b6920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:14:24.475 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.473+0000 7fd8217e4640 1 --2- 172.21.15.42:0/900802534 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560f1dde64d0 0x560f1e1b6920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:24.475 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.473+0000 7fd8217e4640 1 -- 172.21.15.42:0/900802534 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560f1de071b0 msgr2=0x560f1e1b6fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:14:24.476 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.473+0000 7fd8217e4640 1 --2- 172.21.15.42:0/900802534 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560f1de071b0 0x560f1e1b6fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:24.476 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.473+0000 7fd8217e4640 1 -- 172.21.15.42:0/900802534 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f1deeb620 con 0x560f1ddc5d80 2024-08-30T09:14:24.478 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.474+0000 7fd820fe3640 1 --2- 172.21.15.42:0/900802534 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560f1dde64d0 0x560f1e1b6920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello state changed while learned_addr, mark_down or replacing must be happened just now 2024-08-30T09:14:24.478 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.476+0000 7fd8217e4640 1 --2- 172.21.15.42:0/900802534 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560f1ddc5d80 0x560f1e1b7b50 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fd814020990 tx=0x7fd814005bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=530592e5d9c9843e server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:14:24.480 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.477+0000 7fd7fbfff640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-30T09:14:24.484 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.481+0000 7fd7fbfff640 1 -- 172.21.15.42:0/900802534 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd81401fe40 con 0x560f1ddc5d80 2024-08-30T09:14:24.484 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.481+0000 7fd7fbfff640 1 -- 172.21.15.42:0/900802534 <== mon.2 v2:172.21.15.167:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7fd81401db20 con 0x560f1ddc5d80 2024-08-30T09:14:24.485 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.483+0000 7fd82a28d740 1 -- 172.21.15.42:0/900802534 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd80c03ce80 con 0x560f1ddc5d80 2024-08-30T09:14:24.488 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.486+0000 7fd7fbfff640 1 -- 172.21.15.42:0/900802534 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7fd81401dd20 con 0x560f1ddc5d80 2024-08-30T09:14:24.489 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.486+0000 7fd82a28d740 10 client.27524 fetch_fsmap learned FSMap version 47 2024-08-30T09:14:24.489 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.487+0000 7fd82a28d740 1 -- 172.21.15.42:0/900802534 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x560f1e1c0ba0 con 0x560f1ddc5d80 2024-08-30T09:14:24.495 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.493+0000 7fd7fbfff640 1 -- 172.21.15.42:0/900802534 <== mon.2 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd814020e50 con 0x560f1ddc5d80 2024-08-30T09:14:24.497 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.494+0000 7fd7fbfff640 1 -- 172.21.15.42:0/900802534 <== mon.2 v2:172.21.15.167:3300/0 5 ==== fsmap(e 47) ==== 8370+0+0 (secure 0 0 0) 0x7fd81402d630 con 0x560f1ddc5d80 2024-08-30T09:14:24.497 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.494+0000 7fd82a28d740 10 client.27524 fetch_fsmap finished waiting for FSMap version 47 2024-08-30T09:14:24.497 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.495+0000 7fd82a28d740 10 client.27524 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-30T09:14:24.497 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.495+0000 7fd82a28d740 20 client.27524 populate_metadata read hostname 'smithi042' 2024-08-30T09:14:24.497 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.495+0000 7fd82a28d740 1 --2- 172.21.15.42:0/900802534 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x560f1dd9b1c0 0x560f1e1c4470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:14:24.497 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.495+0000 7fd82a28d740 4 client.27524 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-30T09:14:24.497 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.495+0000 7fd82a28d740 1 -- 172.21.15.42:0/900802534 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fd80c03ce80 con 0x560f1dd9b1c0 2024-08-30T09:14:24.499 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.497+0000 7fd820fe3640 1 --2- 172.21.15.42:0/900802534 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x560f1dd9b1c0 0x560f1e1c4470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:14:24.508 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.506+0000 7fd820fe3640 1 --2- 172.21.15.42:0/900802534 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x560f1dd9b1c0 0x560f1e1c4470 crc :-1 s=READY pgs=277 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8386e8788df72eca server_cookie=94bb5a37dd55204b in_seq=0 out_seq=0 2024-08-30T09:14:24.509 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.508+0000 7fd7fbfff640 10 client.27524 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:14:24.513 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.511+0000 7fd7fbfff640 1 -- 172.21.15.42:0/900802534 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7fd80c03ce80 con 0x560f1dd9b1c0 2024-08-30T09:14:24.513 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.511+0000 7fd7fbfff640 10 client.27524 handle_command_reply: tid=0 2024-08-30T09:14:24.513 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.511+0000 7fd82a28d740 1 client.27524 shutdown 2024-08-30T09:14:24.526 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.524+0000 7fd82a28d740 1 -- 172.21.15.42:0/900802534 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560f1ddc5d80 msgr2=0x560f1e1b7b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:14:24.526 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.524+0000 7fd82a28d740 1 --2- 172.21.15.42:0/900802534 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560f1ddc5d80 0x560f1e1b7b50 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fd814020990 tx=0x7fd814005bd0 comp rx=0 tx=0).stop 2024-08-30T09:14:24.532 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.530+0000 7fd82a28d740 1 -- 172.21.15.42:0/900802534 shutdown_connections 2024-08-30T09:14:24.532 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.530+0000 7fd82a28d740 1 --2- 172.21.15.42:0/900802534 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x560f1dd9b1c0 0x560f1e1c4470 crc :-1 s=READY pgs=277 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:24.532 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.530+0000 7fd82a28d740 1 --2- 172.21.15.42:0/900802534 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560f1ddc5d80 0x560f1e1b7b50 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:24.532 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.530+0000 7fd82a28d740 1 --2- 172.21.15.42:0/900802534 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560f1de071b0 0x560f1e1b6fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:24.532 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.530+0000 7fd82a28d740 1 --2- 172.21.15.42:0/900802534 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560f1dde64d0 0x560f1e1b6920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:24.532 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.530+0000 7fd82a28d740 1 -- 172.21.15.42:0/900802534 >> 172.21.15.42:0/900802534 conn(0x560f1ddfb300 msgr2=0x560f1e1ae1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:14:24.532 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.530+0000 7fd7fbfff640 0 client.27524 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:14:24.532 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.530+0000 7fd82a28d740 1 -- 172.21.15.42:0/900802534 shutdown_connections 2024-08-30T09:14:24.536 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.535+0000 7fd82a28d740 1 -- 172.21.15.42:0/900802534 wait complete. 2024-08-30T09:14:24.537 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.536+0000 7fd82a28d740 20 client.27524 trim_cache size 0 max 16384 2024-08-30T09:14:24.574 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.572+0000 7fd82a28d740 1 Processor -- start 2024-08-30T09:14:24.575 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.572+0000 7fd82a28d740 1 -- start start 2024-08-30T09:14:24.575 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.573+0000 7fd82a28d740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560f1ddc5d80 0x560f1e1b80b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:14:24.575 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.573+0000 7fd82a28d740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560f1dde64d0 0x560f1e1b1750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:14:24.575 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.573+0000 7fd82a28d740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560f1de071b0 0x560f1e1b1c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:14:24.575 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.573+0000 7fd82a28d740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fd808012480 con 0x560f1dde64d0 2024-08-30T09:14:24.575 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.573+0000 7fd82a28d740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd80800fcf0 con 0x560f1ddc5d80 2024-08-30T09:14:24.575 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.573+0000 7fd82a28d740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fd80800dbf0 con 0x560f1de071b0 2024-08-30T09:14:24.576 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.573+0000 7fd820fe3640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560f1ddc5d80 0x560f1e1b80b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:14:24.576 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.573+0000 7fd820fe3640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560f1ddc5d80 0x560f1e1b80b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:54686/0 (socket says 172.21.15.42:54686) 2024-08-30T09:14:24.576 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.573+0000 7fd820fe3640 1 -- 172.21.15.42:0/2537395923 learned_addr learned my addr 172.21.15.42:0/2537395923 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:14:24.576 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.573+0000 7fd820fe3640 1 -- 172.21.15.42:0/2537395923 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560f1de071b0 msgr2=0x560f1e1b1c90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-30T09:14:24.577 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.573+0000 7fd820fe3640 1 --2- 172.21.15.42:0/2537395923 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560f1de071b0 0x560f1e1b1c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:24.577 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.574+0000 7fd820fe3640 1 -- 172.21.15.42:0/2537395923 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560f1dde64d0 msgr2=0x560f1e1b1750 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-30T09:14:24.577 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.574+0000 7fd820fe3640 1 --2- 172.21.15.42:0/2537395923 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560f1dde64d0 0x560f1e1b1750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:24.577 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.574+0000 7fd820fe3640 1 -- 172.21.15.42:0/2537395923 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f1e1a6f40 con 0x560f1ddc5d80 2024-08-30T09:14:24.577 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.574+0000 7fd820fe3640 1 --2- 172.21.15.42:0/2537395923 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560f1ddc5d80 0x560f1e1b80b0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fd80c0c9ed0 tx=0x7fd80c0328e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c313c6d7027ad7bd server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:14:24.577 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.574+0000 7fd8027fc640 1 -- 172.21.15.42:0/2537395923 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd80c037070 con 0x560f1ddc5d80 2024-08-30T09:14:24.577 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.574+0000 7fd8027fc640 1 -- 172.21.15.42:0/2537395923 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd80c030900 con 0x560f1ddc5d80 2024-08-30T09:14:24.580 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.577+0000 7fd8027fc640 1 -- 172.21.15.42:0/2537395923 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd80c03f040 con 0x560f1ddc5d80 2024-08-30T09:14:24.580 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.578+0000 7fd82a28d740 1 -- 172.21.15.42:0/2537395923 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560f1ddc5d80 msgr2=0x560f1e1b80b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:14:24.580 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.578+0000 7fd82a28d740 1 --2- 172.21.15.42:0/2537395923 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560f1ddc5d80 0x560f1e1b80b0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fd80c0c9ed0 tx=0x7fd80c0328e0 comp rx=0 tx=0).stop 2024-08-30T09:14:24.580 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.579+0000 7fd82a28d740 1 -- 172.21.15.42:0/2537395923 shutdown_connections 2024-08-30T09:14:24.580 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.579+0000 7fd82a28d740 1 --2- 172.21.15.42:0/2537395923 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560f1de071b0 0x560f1e1b1c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:24.580 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.579+0000 7fd82a28d740 1 --2- 172.21.15.42:0/2537395923 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560f1dde64d0 0x560f1e1b1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:24.581 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.579+0000 7fd82a28d740 1 --2- 172.21.15.42:0/2537395923 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560f1ddc5d80 0x560f1e1b80b0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:24.581 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.579+0000 7fd82a28d740 1 -- 172.21.15.42:0/2537395923 >> 172.21.15.42:0/2537395923 conn(0x560f1ddfb300 msgr2=0x560f1e1adfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:14:24.584 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.582+0000 7fd82a28d740 1 -- 172.21.15.42:0/2537395923 shutdown_connections 2024-08-30T09:14:24.585 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.583+0000 7fd82a28d740 1 -- 172.21.15.42:0/2537395923 wait complete. 2024-08-30T09:14:24.585 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.583+0000 7fd82a28d740 1 -- start start 2024-08-30T09:14:24.585 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.583+0000 7fd82a28d740 1 Processor -- start 2024-08-30T09:14:24.586 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.584+0000 7fd82a28d740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560f1ddc5d80 0x560f1e1ba000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:14:24.586 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.584+0000 7fd82a28d740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560f1dde64d0 0x560f1e1ba6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:14:24.586 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.584+0000 7fd82a28d740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560f1de071b0 0x560f1e1be2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:14:24.586 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.584+0000 7fd82a28d740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fd80800e1b0 con 0x560f1dde64d0 2024-08-30T09:14:24.586 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.584+0000 7fd82a28d740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd80800f620 con 0x560f1de071b0 2024-08-30T09:14:24.586 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.584+0000 7fd82a28d740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fd80800e880 con 0x560f1ddc5d80 2024-08-30T09:14:24.587 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.584+0000 7fd82a28d740 10 client.0 _ll_register_callbacks cb 0x7fd81401aef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:14:24.587 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.584+0000 7fd8217e4640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560f1de071b0 0x560f1e1be2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:14:24.587 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.584+0000 7fd8217e4640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560f1de071b0 0x560f1e1be2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:54704/0 (socket says 172.21.15.42:54704) 2024-08-30T09:14:24.587 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.584+0000 7fd8217e4640 1 -- 172.21.15.42:0/3581922747 learned_addr learned my addr 172.21.15.42:0/3581922747 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:14:24.587 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.584+0000 7fd820fe3640 1 --2- 172.21.15.42:0/3581922747 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560f1ddc5d80 0x560f1e1ba000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:14:24.588 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.584+0000 7fd8217e4640 1 -- 172.21.15.42:0/3581922747 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560f1ddc5d80 msgr2=0x560f1e1ba000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:14:24.588 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.585+0000 7fd8217e4640 1 --2- 172.21.15.42:0/3581922747 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560f1ddc5d80 0x560f1e1ba000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:24.588 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.585+0000 7fd8217e4640 1 -- 172.21.15.42:0/3581922747 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560f1dde64d0 msgr2=0x560f1e1ba6c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-30T09:14:24.588 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.585+0000 7fd8217e4640 1 --2- 172.21.15.42:0/3581922747 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560f1dde64d0 0x560f1e1ba6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:24.588 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.585+0000 7fd8217e4640 1 -- 172.21.15.42:0/3581922747 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f1e1bea10 con 0x560f1de071b0 2024-08-30T09:14:24.588 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.585+0000 7fd8217e4640 1 --2- 172.21.15.42:0/3581922747 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560f1de071b0 0x560f1e1be2b0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fd814035410 tx=0x7fd81401dec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7f11ff8970f904d5 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:14:24.588 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.585+0000 7fd7fbfff640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-30T09:14:24.588 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.585+0000 7fd7fbfff640 1 -- 172.21.15.42:0/3581922747 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd814021e60 con 0x560f1de071b0 2024-08-30T09:14:24.588 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.585+0000 7fd82a28d740 1 -- 172.21.15.42:0/3581922747 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd80c030900 con 0x560f1de071b0 2024-08-30T09:14:24.589 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.586+0000 7fd7fbfff640 1 -- 172.21.15.42:0/3581922747 <== mon.2 v2:172.21.15.167:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7fd81402dcc0 con 0x560f1de071b0 2024-08-30T09:14:24.589 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.586+0000 7fd7fbfff640 1 -- 172.21.15.42:0/3581922747 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7fd814054050 con 0x560f1de071b0 2024-08-30T09:14:24.589 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.586+0000 7fd82a28d740 10 client.27533 fetch_fsmap learned FSMap version 47 2024-08-30T09:14:24.589 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.586+0000 7fd82a28d740 1 -- 172.21.15.42:0/3581922747 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x560f1e1bbe80 con 0x560f1de071b0 2024-08-30T09:14:24.592 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.590+0000 7fd7fbfff640 1 -- 172.21.15.42:0/3581922747 <== mon.2 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd80c030900 con 0x560f1de071b0 2024-08-30T09:14:24.592 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.590+0000 7fd7fbfff640 1 -- 172.21.15.42:0/3581922747 <== mon.2 v2:172.21.15.167:3300/0 5 ==== fsmap(e 47) ==== 8370+0+0 (secure 0 0 0) 0x7fd814049430 con 0x560f1de071b0 2024-08-30T09:14:24.592 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.590+0000 7fd82a28d740 10 client.27533 fetch_fsmap finished waiting for FSMap version 47 2024-08-30T09:14:24.592 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.590+0000 7fd82a28d740 10 client.27533 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-30T09:14:24.592 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.590+0000 7fd82a28d740 20 client.27533 populate_metadata read hostname 'smithi042' 2024-08-30T09:14:24.592 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.590+0000 7fd82a28d740 1 --2- 172.21.15.42:0/3581922747 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x560f1e22f560 0x560f1e231980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:14:24.592 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.590+0000 7fd82a28d740 4 client.27533 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-30T09:14:24.592 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.590+0000 7fd82a28d740 1 -- 172.21.15.42:0/3581922747 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560f1e121f50 con 0x560f1e22f560 2024-08-30T09:14:24.596 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.593+0000 7fd820fe3640 1 --2- 172.21.15.42:0/3581922747 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x560f1e22f560 0x560f1e231980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:14:24.602 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.601+0000 7fd7fbfff640 10 client.27533 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:14:24.602 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.601+0000 7fd820fe3640 1 --2- 172.21.15.42:0/3581922747 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x560f1e22f560 0x560f1e231980 crc :-1 s=READY pgs=278 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9b549aedb1befdac server_cookie=f459e6c8a3191c3c in_seq=0 out_seq=0 2024-08-30T09:14:24.623 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:14:24 smithi042 ceph-mon[18652]: pgmap v782: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.7 MiB/s wr, 501 op/s 2024-08-30T09:14:24.770 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.768+0000 7fd7fbfff640 1 -- 172.21.15.42:0/3581922747 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560f1e121f50 con 0x560f1e22f560 2024-08-30T09:14:24.770 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.768+0000 7fd7fbfff640 10 client.27533 handle_command_reply: tid=0 2024-08-30T09:14:24.770 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.768+0000 7fd82a28d740 1 client.27533 shutdown 2024-08-30T09:14:24.770 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.768+0000 7fd82a28d740 1 -- 172.21.15.42:0/3581922747 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560f1de071b0 msgr2=0x560f1e1be2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:14:24.770 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.768+0000 7fd82a28d740 1 --2- 172.21.15.42:0/3581922747 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560f1de071b0 0x560f1e1be2b0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fd814035410 tx=0x7fd81401dec0 comp rx=0 tx=0).stop 2024-08-30T09:14:24.770 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.768+0000 7fd82a28d740 1 -- 172.21.15.42:0/3581922747 shutdown_connections 2024-08-30T09:14:24.771 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.768+0000 7fd82a28d740 1 --2- 172.21.15.42:0/3581922747 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x560f1e22f560 0x560f1e231980 crc :-1 s=READY pgs=278 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:24.771 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.768+0000 7fd82a28d740 1 --2- 172.21.15.42:0/3581922747 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560f1de071b0 0x560f1e1be2b0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:24.771 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.768+0000 7fd82a28d740 1 --2- 172.21.15.42:0/3581922747 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560f1dde64d0 0x560f1e1ba6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:24.771 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.768+0000 7fd82a28d740 1 --2- 172.21.15.42:0/3581922747 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560f1ddc5d80 0x560f1e1ba000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:24.771 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.768+0000 7fd82a28d740 1 -- 172.21.15.42:0/3581922747 >> 172.21.15.42:0/3581922747 conn(0x560f1ddfb300 msgr2=0x560f1e1adfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:14:24.771 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.768+0000 7fd7fbfff640 0 client.27533 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:14:24.771 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.769+0000 7fd82a28d740 1 -- 172.21.15.42:0/3581922747 shutdown_connections 2024-08-30T09:14:24.772 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.769+0000 7fd82a28d740 1 -- 172.21.15.42:0/3581922747 wait complete. 2024-08-30T09:14:24.772 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.769+0000 7fd82a28d740 20 client.27533 trim_cache size 0 max 16384 2024-08-30T09:14:24.772 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-08-30T09:14:24.772 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2024-08-30T09:14:24.772 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-08-30T09:14:24.772 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-08-30T09:14:24.772 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.770+0000 7fd821fe5640 1 -- 172.21.15.42:0/2999025708 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fd8040b4f70 msgr2=0x7fd8040b7480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:14:24.773 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.770+0000 7fd821fe5640 1 --2- 172.21.15.42:0/2999025708 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fd8040b4f70 0x7fd8040b7480 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fd814005e30 tx=0x7fd814005da0 comp rx=0 tx=0).stop 2024-08-30T09:14:24.773 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.770+0000 7fd821fe5640 1 -- 172.21.15.42:0/2999025708 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd81c10a160 msgr2=0x7fd81c1a01a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:14:24.773 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.770+0000 7fd821fe5640 1 --2- 172.21.15.42:0/2999025708 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd81c10a160 0x7fd81c1a01a0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fd80c009780 tx=0x7fd80c033e20 comp rx=0 tx=0).stop 2024-08-30T09:14:24.773 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.772+0000 7fd821fe5640 1 -- 172.21.15.42:0/2999025708 shutdown_connections 2024-08-30T09:14:24.774 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.772+0000 7fd821fe5640 1 --2- 172.21.15.42:0/2999025708 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fd8040b4f70 0x7fd8040b7480 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:24.774 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.772+0000 7fd821fe5640 1 --2- 172.21.15.42:0/2999025708 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd81c10f5d0 0x7fd81c1a13c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:24.774 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.772+0000 7fd821fe5640 1 --2- 172.21.15.42:0/2999025708 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd81c10cac0 0x7fd81c1a0810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:24.774 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.772+0000 7fd821fe5640 1 --2- 172.21.15.42:0/2999025708 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd81c10a160 0x7fd81c1a01a0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:24.774 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.772+0000 7fd821fe5640 1 -- 172.21.15.42:0/2999025708 >> 172.21.15.42:0/2999025708 conn(0x7fd81c0703e0 msgr2=0x7fd81c0707f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:14:24.774 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.772+0000 7fd821fe5640 1 -- 172.21.15.42:0/2999025708 shutdown_connections 2024-08-30T09:14:24.774 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:24.772+0000 7fd821fe5640 1 -- 172.21.15.42:0/2999025708 wait complete. 2024-08-30T09:14:24.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:14:24 smithi167 ceph-mon[22063]: pgmap v782: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.7 MiB/s wr, 501 op/s 2024-08-30T09:14:24.916 INFO:teuthology.orchestra.run.smithi042.stdout:[] 2024-08-30T09:14:24.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.569+0000 7fa524e4c640 1 -- 172.21.15.42:0/2142800536 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa52010b680 msgr2=0x7fa52010ba90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:14:24.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.569+0000 7fa524e4c640 1 --2- 172.21.15.42:0/2142800536 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa52010b680 0x7fa52010ba90 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fa51400fb10 tx=0x7fa51402e5b0 comp rx=0 tx=0).stop 2024-08-30T09:14:24.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.570+0000 7fa524e4c640 1 -- 172.21.15.42:0/2142800536 shutdown_connections 2024-08-30T09:14:24.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.570+0000 7fa524e4c640 1 --2- 172.21.15.42:0/2142800536 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa52010ffb0 0x7fa52007b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:24.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.570+0000 7fa524e4c640 1 --2- 172.21.15.42:0/2142800536 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa52010f660 0x7fa52010fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:24.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.570+0000 7fa524e4c640 1 --2- 172.21.15.42:0/2142800536 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa52010b680 0x7fa52010ba90 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:24.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.570+0000 7fa524e4c640 1 -- 172.21.15.42:0/2142800536 >> 172.21.15.42:0/2142800536 conn(0x7fa52006e790 msgr2=0x7fa52006ebd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:14:24.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.572+0000 7fa524e4c640 1 -- 172.21.15.42:0/2142800536 shutdown_connections 2024-08-30T09:14:24.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.572+0000 7fa524e4c640 1 -- 172.21.15.42:0/2142800536 wait complete. 2024-08-30T09:14:24.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.573+0000 7fa524e4c640 1 Processor -- start 2024-08-30T09:14:24.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.579+0000 7fa524e4c640 1 -- start start 2024-08-30T09:14:24.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.579+0000 7fa524e4c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa52010b680 0x7fa520084fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:14:24.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.579+0000 7fa524e4c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa52010f660 0x7fa52007f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:14:24.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.579+0000 7fa524e4c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa52010ffb0 0x7fa52007f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:14:24.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.579+0000 7fa524e4c640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fa52007d730 con 0x7fa52010b680 2024-08-30T09:14:24.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.579+0000 7fa524e4c640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa52007d5b0 con 0x7fa52010ffb0 2024-08-30T09:14:24.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.579+0000 7fa524e4c640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fa52007d8b0 con 0x7fa52010f660 2024-08-30T09:14:24.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.580+0000 7fa51f7fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa52010b680 0x7fa520084fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:14:24.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.580+0000 7fa51f7fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa52010b680 0x7fa520084fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60584/0 (socket says 172.21.15.42:60584) 2024-08-30T09:14:24.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.580+0000 7fa51f7fe640 1 -- 172.21.15.42:0/3648964086 learned_addr learned my addr 172.21.15.42:0/3648964086 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:14:24.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.580+0000 7fa51f7fe640 1 -- 172.21.15.42:0/3648964086 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa52010f660 msgr2=0x7fa52007f2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:14:24.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.580+0000 7fa51f7fe640 1 --2- 172.21.15.42:0/3648964086 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa52010f660 0x7fa52007f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:24.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.580+0000 7fa51f7fe640 1 -- 172.21.15.42:0/3648964086 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa52010ffb0 msgr2=0x7fa52007f8a0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-08-30T09:14:24.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.580+0000 7fa51f7fe640 1 --2- 172.21.15.42:0/3648964086 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa52010ffb0 0x7fa52007f8a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:24.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.580+0000 7fa51f7fe640 1 -- 172.21.15.42:0/3648964086 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5200800c0 con 0x7fa52010b680 2024-08-30T09:14:24.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.580+0000 7fa51f7fe640 1 --2- 172.21.15.42:0/3648964086 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa52010b680 0x7fa520084fe0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fa514005ab0 tx=0x7fa514005040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:14:24.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.581+0000 7fa51cff9640 1 -- 172.21.15.42:0/3648964086 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa514005640 con 0x7fa52010b680 2024-08-30T09:14:24.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.581+0000 7fa524e4c640 1 -- 172.21.15.42:0/3648964086 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5201cea00 con 0x7fa52010b680 2024-08-30T09:14:24.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.581+0000 7fa524e4c640 1 -- 172.21.15.42:0/3648964086 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5201cef30 con 0x7fa52010b680 2024-08-30T09:14:24.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.584+0000 7fa52c623740 1 Processor -- start 2024-08-30T09:14:24.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.585+0000 7fa51cff9640 1 -- 172.21.15.42:0/3648964086 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa514043070 con 0x7fa52010b680 2024-08-30T09:14:24.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.585+0000 7fa51cff9640 1 -- 172.21.15.42:0/3648964086 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa514004a40 con 0x7fa52010b680 2024-08-30T09:14:24.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.588+0000 7fa52c623740 1 -- start start 2024-08-30T09:14:24.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.593+0000 7fa52c623740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b8c5cf6070 0x55b8c609b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:14:24.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.593+0000 7fa52c623740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b8c5ccd1b0 0x55b8c609b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:14:24.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.593+0000 7fa52c623740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b8c5cd5390 0x55b8c609bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:14:24.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.593+0000 7fa52c623740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55b8c6012680 con 0x55b8c5cd5390 2024-08-30T09:14:24.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.593+0000 7fa52c623740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55b8c6001c10 con 0x55b8c5cf6070 2024-08-30T09:14:24.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.593+0000 7fa52c623740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55b8c6005bc0 con 0x55b8c5ccd1b0 2024-08-30T09:14:24.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.595+0000 7fa51f7fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b8c5ccd1b0 0x55b8c609b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:14:24.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.595+0000 7fa51f7fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b8c5ccd1b0 0x55b8c609b9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:36280/0 (socket says 172.21.15.42:36280) 2024-08-30T09:14:24.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.595+0000 7fa51f7fe640 1 -- 172.21.15.42:0/3764942876 learned_addr learned my addr 172.21.15.42:0/3764942876 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:14:24.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.595+0000 7fa51f7fe640 1 -- 172.21.15.42:0/3764942876 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b8c5cf6070 msgr2=0x55b8c609b4b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-30T09:14:24.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.595+0000 7fa51f7fe640 1 --2- 172.21.15.42:0/3764942876 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b8c5cf6070 0x55b8c609b4b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:24.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.595+0000 7fa51f7fe640 1 -- 172.21.15.42:0/3764942876 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b8c5cd5390 msgr2=0x55b8c609bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:14:24.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.595+0000 7fa51f7fe640 1 --2- 172.21.15.42:0/3764942876 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b8c5cd5390 0x55b8c609bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:24.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.595+0000 7fa51f7fe640 1 -- 172.21.15.42:0/3764942876 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b8c5d61650 con 0x55b8c5ccd1b0 2024-08-30T09:14:24.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.596+0000 7fa51cff9640 1 -- 172.21.15.42:0/3648964086 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7fa514004be0 con 0x7fa52010b680 2024-08-30T09:14:24.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.596+0000 7fa51f7fe640 1 --2- 172.21.15.42:0/3764942876 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b8c5ccd1b0 0x55b8c609b9f0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fa514109d80 tx=0x7fa51410c830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a9ddc77f9734a4c9 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:14:24.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.597+0000 7fa51cff9640 1 --2- 172.21.15.42:0/3648964086 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fa5080b4dd0 0x7fa5080b72e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:14:24.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.598+0000 7fa505ffb640 1 -- 172.21.15.42:0/3764942876 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa51410a440 con 0x55b8c5ccd1b0 2024-08-30T09:14:24.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.598+0000 7fa51ffff640 1 --2- 172.21.15.42:0/3648964086 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fa5080b4dd0 0x7fa5080b72e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:14:24.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.598+0000 7fa505ffb640 1 -- 172.21.15.42:0/3764942876 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa51410aa90 con 0x55b8c5ccd1b0 2024-08-30T09:14:24.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.598+0000 7fa51cff9640 1 -- 172.21.15.42:0/3648964086 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7fa514105e20 con 0x7fa52010b680 2024-08-30T09:14:24.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.599+0000 7fa505ffb640 1 -- 172.21.15.42:0/3764942876 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5141167f0 con 0x55b8c5ccd1b0 2024-08-30T09:14:24.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.599+0000 7fa52c623740 1 -- 172.21.15.42:0/3764942876 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b8c5ccd1b0 msgr2=0x55b8c609b9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:14:24.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.599+0000 7fa52c623740 1 --2- 172.21.15.42:0/3764942876 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b8c5ccd1b0 0x55b8c609b9f0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fa514109d80 tx=0x7fa51410c830 comp rx=0 tx=0).stop 2024-08-30T09:14:24.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.599+0000 7fa51ffff640 1 --2- 172.21.15.42:0/3648964086 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fa5080b4dd0 0x7fa5080b72e0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fa51000a780 tx=0x7fa51000c040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:14:24.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.599+0000 7fa52c623740 1 -- 172.21.15.42:0/3764942876 shutdown_connections 2024-08-30T09:14:24.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.599+0000 7fa52c623740 1 --2- 172.21.15.42:0/3764942876 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b8c5ccd1b0 0x55b8c609b9f0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:24.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.599+0000 7fa52c623740 1 --2- 172.21.15.42:0/3764942876 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b8c5cd5390 0x55b8c609bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:24.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.599+0000 7fa52c623740 1 --2- 172.21.15.42:0/3764942876 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b8c5cf6070 0x55b8c609b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:24.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.599+0000 7fa52c623740 1 -- 172.21.15.42:0/3764942876 >> 172.21.15.42:0/3764942876 conn(0x55b8c5cea460 msgr2=0x55b8c60993a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:14:24.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.599+0000 7fa52c623740 1 -- 172.21.15.42:0/3764942876 shutdown_connections 2024-08-30T09:14:24.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.599+0000 7fa52c623740 1 -- 172.21.15.42:0/3764942876 wait complete. 2024-08-30T09:14:24.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.600+0000 7fa52c623740 1 -- start start 2024-08-30T09:14:24.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.600+0000 7fa52c623740 1 Processor -- start 2024-08-30T09:14:24.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.601+0000 7fa52c623740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b8c5ccd1b0 0x55b8c60a6c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:14:24.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.601+0000 7fa52c623740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b8c5cd5390 0x55b8c60a71b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:14:24.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.601+0000 7fa52c623740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b8c5cf6070 0x55b8c60b2de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:14:24.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.601+0000 7fa52c623740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55b8c5e64700 con 0x55b8c5ccd1b0 2024-08-30T09:14:24.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.601+0000 7fa52c623740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55b8c5f88750 con 0x55b8c5cf6070 2024-08-30T09:14:24.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.601+0000 7fa52c623740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55b8c60a9a50 con 0x55b8c5cd5390 2024-08-30T09:14:24.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.601+0000 7fa52c623740 10 client.0 _ll_register_callbacks cb 0x55b8c5fb7e10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:14:24.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.601+0000 7fa51f7fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b8c5ccd1b0 0x55b8c60a6c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:14:24.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.601+0000 7fa51effd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b8c5cd5390 0x55b8c60a71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:14:24.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.601+0000 7fa51f7fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b8c5ccd1b0 0x55b8c60a6c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60606/0 (socket says 172.21.15.42:60606) 2024-08-30T09:14:24.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.601+0000 7fa51f7fe640 1 -- 172.21.15.42:0/3958786778 learned_addr learned my addr 172.21.15.42:0/3958786778 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:14:24.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.601+0000 7fa51ffff640 1 --2- 172.21.15.42:0/3958786778 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b8c5cf6070 0x55b8c60b2de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:14:24.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.601+0000 7fa51f7fe640 1 -- 172.21.15.42:0/3958786778 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b8c5cd5390 msgr2=0x55b8c60a71b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:14:24.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.601+0000 7fa51f7fe640 1 --2- 172.21.15.42:0/3958786778 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b8c5cd5390 0x55b8c60a71b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:24.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.601+0000 7fa51f7fe640 1 -- 172.21.15.42:0/3958786778 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b8c5cf6070 msgr2=0x55b8c60b2de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:14:24.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.601+0000 7fa51f7fe640 1 --2- 172.21.15.42:0/3958786778 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b8c5cf6070 0x55b8c60b2de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:24.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.601+0000 7fa51f7fe640 1 -- 172.21.15.42:0/3958786778 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b8c5fc1900 con 0x55b8c5ccd1b0 2024-08-30T09:14:24.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.602+0000 7fa51effd640 1 --2- 172.21.15.42:0/3958786778 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b8c5cd5390 0x55b8c60a71b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-30T09:14:24.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.602+0000 7fa51ffff640 1 --2- 172.21.15.42:0/3958786778 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b8c5cf6070 0x55b8c60b2de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-30T09:14:24.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.602+0000 7fa51f7fe640 1 --2- 172.21.15.42:0/3958786778 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b8c5ccd1b0 0x55b8c60a6c70 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fa5140c96d0 tx=0x7fa51410c2e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af020a0aa1bd073d server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:14:24.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.602+0000 7fa4feffd640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-08-30T09:14:24.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.602+0000 7fa4feffd640 1 -- 172.21.15.42:0/3958786778 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5140c8320 con 0x55b8c5ccd1b0 2024-08-30T09:14:24.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.602+0000 7fa4feffd640 1 -- 172.21.15.42:0/3958786778 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7fa5140c8a50 con 0x55b8c5ccd1b0 2024-08-30T09:14:24.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.602+0000 7fa52c623740 1 -- 172.21.15.42:0/3958786778 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa51410aa90 con 0x55b8c5ccd1b0 2024-08-30T09:14:24.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.602+0000 7fa4feffd640 1 -- 172.21.15.42:0/3958786778 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7fa5140cbbd0 con 0x55b8c5ccd1b0 2024-08-30T09:14:24.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.602+0000 7fa52c623740 10 client.17688 fetch_fsmap learned FSMap version 47 2024-08-30T09:14:24.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.602+0000 7fa52c623740 1 -- 172.21.15.42:0/3958786778 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55b8c60b58a0 con 0x55b8c5ccd1b0 2024-08-30T09:14:24.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.602+0000 7fa4feffd640 1 -- 172.21.15.42:0/3958786778 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa51410aa90 con 0x55b8c5ccd1b0 2024-08-30T09:14:24.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.603+0000 7fa4feffd640 1 -- 172.21.15.42:0/3958786778 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 47) ==== 8370+0+0 (secure 0 0 0) 0x7fa514108070 con 0x55b8c5ccd1b0 2024-08-30T09:14:24.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.603+0000 7fa52c623740 10 client.17688 fetch_fsmap finished waiting for FSMap version 47 2024-08-30T09:14:24.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.603+0000 7fa52c623740 10 client.17688 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-08-30T09:14:24.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.603+0000 7fa52c623740 20 client.17688 populate_metadata read hostname 'smithi042' 2024-08-30T09:14:24.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.603+0000 7fa52c623740 1 --2- 172.21.15.42:0/3958786778 >> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] conn(0x55b8c5cb49f0 0x55b8c60b8c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:14:24.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.603+0000 7fa52c623740 4 client.17688 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-30T09:14:24.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.603+0000 7fa52c623740 1 -- 172.21.15.42:0/3958786778 --> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b8c60145d0 con 0x55b8c5cb49f0 2024-08-30T09:14:24.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.603+0000 7fa51effd640 1 --2- 172.21.15.42:0/3958786778 >> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] conn(0x55b8c5cb49f0 0x55b8c60b8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:14:24.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.604+0000 7fa4feffd640 10 client.17688 ms_handle_connect on v2:172.21.15.190:6838/719981110 2024-08-30T09:14:24.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.604+0000 7fa51effd640 1 --2- 172.21.15.42:0/3958786778 >> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] conn(0x55b8c5cb49f0 0x55b8c60b8c00 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c317815cec0112d7 server_cookie=839587b70dae326a in_seq=0 out_seq=0 2024-08-30T09:14:24.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.605+0000 7fa4feffd640 1 -- 172.21.15.42:0/3958786778 <== mds.1 v2:172.21.15.190:6838/719981110 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b8c60145d0 con 0x55b8c5cb49f0 2024-08-30T09:14:24.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.605+0000 7fa4feffd640 10 client.17688 handle_command_reply: tid=0 2024-08-30T09:14:24.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.605+0000 7fa52c623740 1 client.17688 shutdown 2024-08-30T09:14:24.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.605+0000 7fa52c623740 1 -- 172.21.15.42:0/3958786778 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b8c5ccd1b0 msgr2=0x55b8c60a6c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:14:24.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.605+0000 7fa52c623740 1 --2- 172.21.15.42:0/3958786778 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b8c5ccd1b0 0x55b8c60a6c70 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fa5140c96d0 tx=0x7fa51410c2e0 comp rx=0 tx=0).stop 2024-08-30T09:14:24.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.605+0000 7fa52c623740 1 -- 172.21.15.42:0/3958786778 shutdown_connections 2024-08-30T09:14:24.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.605+0000 7fa52c623740 1 --2- 172.21.15.42:0/3958786778 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b8c5cd5390 0x55b8c60a71b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:24.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.605+0000 7fa52c623740 1 --2- 172.21.15.42:0/3958786778 >> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] conn(0x55b8c5cb49f0 0x55b8c60b8c00 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:24.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.605+0000 7fa52c623740 1 --2- 172.21.15.42:0/3958786778 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b8c5cf6070 0x55b8c60b2de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:24.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.605+0000 7fa52c623740 1 --2- 172.21.15.42:0/3958786778 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b8c5ccd1b0 0x55b8c60a6c70 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:24.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.605+0000 7fa4feffd640 0 client.17688 ms_handle_reset on v2:172.21.15.190:6838/719981110 2024-08-30T09:14:24.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.605+0000 7fa52c623740 1 -- 172.21.15.42:0/3958786778 >> 172.21.15.42:0/3958786778 conn(0x55b8c5cea460 msgr2=0x55b8c609e640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:14:24.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.605+0000 7fa52c623740 1 -- 172.21.15.42:0/3958786778 shutdown_connections 2024-08-30T09:14:24.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.606+0000 7fa52c623740 1 -- 172.21.15.42:0/3958786778 wait complete. 2024-08-30T09:14:24.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.606+0000 7fa52c623740 20 client.17688 trim_cache size 0 max 16384 2024-08-30T09:14:24.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.621+0000 7fa52c623740 1 Processor -- start 2024-08-30T09:14:24.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.621+0000 7fa52c623740 1 -- start start 2024-08-30T09:14:24.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.622+0000 7fa52c623740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b8c5ccd1b0 0x55b8c609cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:14:24.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.622+0000 7fa52c623740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b8c5cd5390 0x55b8c60b2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:14:24.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.622+0000 7fa52c623740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b8c5cf6070 0x55b8c60a3e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:14:24.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.622+0000 7fa52c623740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fa50c0124b0 con 0x55b8c5cd5390 2024-08-30T09:14:24.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.622+0000 7fa52c623740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa50c00fd20 con 0x55b8c5ccd1b0 2024-08-30T09:14:24.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.622+0000 7fa52c623740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fa50c00db70 con 0x55b8c5cf6070 2024-08-30T09:14:24.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.623+0000 7fa51effd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b8c5cd5390 0x55b8c60b2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:14:24.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.623+0000 7fa51effd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b8c5cd5390 0x55b8c60b2ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60616/0 (socket says 172.21.15.42:60616) 2024-08-30T09:14:24.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.623+0000 7fa51effd640 1 -- 172.21.15.42:0/1771381122 learned_addr learned my addr 172.21.15.42:0/1771381122 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:14:24.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.624+0000 7fa51effd640 1 -- 172.21.15.42:0/1771381122 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b8c5cf6070 msgr2=0x55b8c60a3e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:14:24.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.624+0000 7fa51effd640 1 --2- 172.21.15.42:0/1771381122 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b8c5cf6070 0x55b8c60a3e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:24.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.624+0000 7fa51effd640 1 -- 172.21.15.42:0/1771381122 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b8c5ccd1b0 msgr2=0x55b8c609cbf0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-30T09:14:24.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.624+0000 7fa51effd640 1 --2- 172.21.15.42:0/1771381122 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b8c5ccd1b0 0x55b8c609cbf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:24.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.624+0000 7fa51effd640 1 -- 172.21.15.42:0/1771381122 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b8c60a7f30 con 0x55b8c5cd5390 2024-08-30T09:14:24.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.624+0000 7fa51effd640 1 --2- 172.21.15.42:0/1771381122 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b8c5cd5390 0x55b8c60b2ce0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fa5180041f0 tx=0x7fa518003a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb8ab2024f31dbd9 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:14:24.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.625+0000 7fa5057fa640 1 -- 172.21.15.42:0/1771381122 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa518008c10 con 0x55b8c5cd5390 2024-08-30T09:14:24.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.625+0000 7fa5057fa640 1 -- 172.21.15.42:0/1771381122 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa518008db0 con 0x55b8c5cd5390 2024-08-30T09:14:24.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.625+0000 7fa5057fa640 1 -- 172.21.15.42:0/1771381122 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa51801e460 con 0x55b8c5cd5390 2024-08-30T09:14:24.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.627+0000 7fa52c623740 1 -- 172.21.15.42:0/1771381122 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b8c5cd5390 msgr2=0x55b8c60b2ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:14:24.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.627+0000 7fa52c623740 1 --2- 172.21.15.42:0/1771381122 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b8c5cd5390 0x55b8c60b2ce0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fa5180041f0 tx=0x7fa518003a70 comp rx=0 tx=0).stop 2024-08-30T09:14:24.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.627+0000 7fa52c623740 1 -- 172.21.15.42:0/1771381122 shutdown_connections 2024-08-30T09:14:24.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.627+0000 7fa52c623740 1 --2- 172.21.15.42:0/1771381122 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b8c5cf6070 0x55b8c60a3e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:24.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.627+0000 7fa52c623740 1 --2- 172.21.15.42:0/1771381122 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b8c5cd5390 0x55b8c60b2ce0 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:24.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.627+0000 7fa52c623740 1 --2- 172.21.15.42:0/1771381122 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b8c5ccd1b0 0x55b8c609cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:24.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.627+0000 7fa52c623740 1 -- 172.21.15.42:0/1771381122 >> 172.21.15.42:0/1771381122 conn(0x55b8c5cea460 msgr2=0x55b8c609c4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:14:24.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.628+0000 7fa52c623740 1 -- 172.21.15.42:0/1771381122 shutdown_connections 2024-08-30T09:14:24.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.628+0000 7fa52c623740 1 -- 172.21.15.42:0/1771381122 wait complete. 2024-08-30T09:14:24.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.628+0000 7fa52c623740 1 -- start start 2024-08-30T09:14:24.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.628+0000 7fa52c623740 1 Processor -- start 2024-08-30T09:14:24.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.629+0000 7fa52c623740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b8c5ccd1b0 0x55b8c60ab4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:14:24.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.629+0000 7fa52c623740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b8c5cd5390 0x55b8c60ab9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:14:24.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.629+0000 7fa52c623740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b8c5cf6070 0x55b8c61224b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:14:24.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.629+0000 7fa52c623740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fa50c00e1e0 con 0x55b8c5cd5390 2024-08-30T09:14:24.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.629+0000 7fa52c623740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa50c00f650 con 0x55b8c5cf6070 2024-08-30T09:14:24.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.629+0000 7fa52c623740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fa50c00e8b0 con 0x55b8c5ccd1b0 2024-08-30T09:14:24.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.629+0000 7fa52c623740 10 client.0 _ll_register_callbacks cb 0x7fa514109c50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:14:24.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.629+0000 7fa51effd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b8c5cd5390 0x55b8c60ab9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:14:24.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.629+0000 7fa51f7fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b8c5ccd1b0 0x55b8c60ab4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:14:24.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.629+0000 7fa51effd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b8c5cd5390 0x55b8c60ab9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60622/0 (socket says 172.21.15.42:60622) 2024-08-30T09:14:24.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.629+0000 7fa51effd640 1 -- 172.21.15.42:0/91564475 learned_addr learned my addr 172.21.15.42:0/91564475 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:14:24.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.629+0000 7fa51ffff640 1 --2- 172.21.15.42:0/91564475 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b8c5cf6070 0x55b8c61224b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:14:24.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.629+0000 7fa51f7fe640 1 -- 172.21.15.42:0/91564475 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b8c5cf6070 msgr2=0x55b8c61224b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:14:24.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.629+0000 7fa51f7fe640 1 --2- 172.21.15.42:0/91564475 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b8c5cf6070 0x55b8c61224b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:24.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.629+0000 7fa51f7fe640 1 -- 172.21.15.42:0/91564475 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b8c5cd5390 msgr2=0x55b8c60ab9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:14:24.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.630+0000 7fa51f7fe640 1 --2- 172.21.15.42:0/91564475 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b8c5cd5390 0x55b8c60ab9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:24.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.630+0000 7fa51f7fe640 1 -- 172.21.15.42:0/91564475 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b8c61229f0 con 0x55b8c5ccd1b0 2024-08-30T09:14:24.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.630+0000 7fa51effd640 1 --2- 172.21.15.42:0/91564475 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b8c5cd5390 0x55b8c60ab9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-30T09:14:24.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.630+0000 7fa51f7fe640 1 --2- 172.21.15.42:0/91564475 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b8c5ccd1b0 0x55b8c60ab4a0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fa51400f020 tx=0x7fa514005eb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8b2782f0716f454d server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:14:24.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.630+0000 7fa4feffd640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-08-30T09:14:24.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.630+0000 7fa4feffd640 1 -- 172.21.15.42:0/91564475 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5140cbdb0 con 0x55b8c5ccd1b0 2024-08-30T09:14:24.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.630+0000 7fa52c623740 1 -- 172.21.15.42:0/91564475 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa518008db0 con 0x55b8c5ccd1b0 2024-08-30T09:14:24.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.630+0000 7fa4feffd640 1 -- 172.21.15.42:0/91564475 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7fa5140c8a50 con 0x55b8c5ccd1b0 2024-08-30T09:14:24.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.630+0000 7fa4feffd640 1 -- 172.21.15.42:0/91564475 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7fa5140cc230 con 0x55b8c5ccd1b0 2024-08-30T09:14:24.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.630+0000 7fa52c623740 10 client.27457 fetch_fsmap learned FSMap version 47 2024-08-30T09:14:24.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.631+0000 7fa52c623740 1 -- 172.21.15.42:0/91564475 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55b8c61229f0 con 0x55b8c5ccd1b0 2024-08-30T09:14:24.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.631+0000 7fa4feffd640 1 -- 172.21.15.42:0/91564475 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa518008db0 con 0x55b8c5ccd1b0 2024-08-30T09:14:24.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.631+0000 7fa4feffd640 1 -- 172.21.15.42:0/91564475 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 47) ==== 8370+0+0 (secure 0 0 0) 0x7fa514108070 con 0x55b8c5ccd1b0 2024-08-30T09:14:24.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.631+0000 7fa52c623740 10 client.27457 fetch_fsmap finished waiting for FSMap version 47 2024-08-30T09:14:24.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.631+0000 7fa52c623740 10 client.27457 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-08-30T09:14:24.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.631+0000 7fa52c623740 20 client.27457 populate_metadata read hostname 'smithi042' 2024-08-30T09:14:24.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.631+0000 7fa52c623740 1 --2- 172.21.15.42:0/91564475 >> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] conn(0x55b8c5c89da0 0x55b8c6126010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:14:24.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.631+0000 7fa52c623740 4 client.27457 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-30T09:14:24.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.631+0000 7fa52c623740 1 -- 172.21.15.42:0/91564475 --> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b8c6010070 con 0x55b8c5c89da0 2024-08-30T09:14:24.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.632+0000 7fa51effd640 1 --2- 172.21.15.42:0/91564475 >> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] conn(0x55b8c5c89da0 0x55b8c6126010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:14:24.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.632+0000 7fa51effd640 1 --2- 172.21.15.42:0/91564475 >> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] conn(0x55b8c5c89da0 0x55b8c6126010 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=bcb048cac037ba6b server_cookie=daafd2542f777abc in_seq=0 out_seq=0 2024-08-30T09:14:24.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.632+0000 7fa4feffd640 10 client.27457 ms_handle_connect on v2:172.21.15.190:6838/719981110 2024-08-30T09:14:24.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.632+0000 7fa4feffd640 1 -- 172.21.15.42:0/91564475 <== mds.1 v2:172.21.15.190:6838/719981110 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b8c6010070 con 0x55b8c5c89da0 2024-08-30T09:14:24.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.632+0000 7fa4feffd640 10 client.27457 handle_command_reply: tid=0 2024-08-30T09:14:24.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.633+0000 7fa52c623740 1 client.27457 shutdown 2024-08-30T09:14:24.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.633+0000 7fa52c623740 1 -- 172.21.15.42:0/91564475 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b8c5ccd1b0 msgr2=0x55b8c60ab4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:14:24.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.633+0000 7fa52c623740 1 --2- 172.21.15.42:0/91564475 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b8c5ccd1b0 0x55b8c60ab4a0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fa51400f020 tx=0x7fa514005eb0 comp rx=0 tx=0).stop 2024-08-30T09:14:24.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.633+0000 7fa52c623740 1 -- 172.21.15.42:0/91564475 shutdown_connections 2024-08-30T09:14:24.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.633+0000 7fa52c623740 1 --2- 172.21.15.42:0/91564475 >> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] conn(0x55b8c5c89da0 0x55b8c6126010 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:24.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.633+0000 7fa52c623740 1 --2- 172.21.15.42:0/91564475 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b8c5cf6070 0x55b8c61224b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:24.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.633+0000 7fa52c623740 1 --2- 172.21.15.42:0/91564475 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b8c5cd5390 0x55b8c60ab9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:24.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.633+0000 7fa4feffd640 0 client.27457 ms_handle_reset on v2:172.21.15.190:6838/719981110 2024-08-30T09:14:24.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.633+0000 7fa52c623740 1 --2- 172.21.15.42:0/91564475 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b8c5ccd1b0 0x55b8c60ab4a0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:24.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.633+0000 7fa52c623740 1 -- 172.21.15.42:0/91564475 >> 172.21.15.42:0/91564475 conn(0x55b8c5cea460 msgr2=0x55b8c609c400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:14:24.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.633+0000 7fa52c623740 1 -- 172.21.15.42:0/91564475 shutdown_connections 2024-08-30T09:14:24.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.633+0000 7fa52c623740 1 -- 172.21.15.42:0/91564475 wait complete. 2024-08-30T09:14:24.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.633+0000 7fa52c623740 20 client.27457 trim_cache size 0 max 16384 2024-08-30T09:14:24.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.633+0000 7fa524e4c640 1 -- 172.21.15.42:0/3648964086 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fa5080b4dd0 msgr2=0x7fa5080b72e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:14:24.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.633+0000 7fa524e4c640 1 --2- 172.21.15.42:0/3648964086 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fa5080b4dd0 0x7fa5080b72e0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fa51000a780 tx=0x7fa51000c040 comp rx=0 tx=0).stop 2024-08-30T09:14:24.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.634+0000 7fa524e4c640 1 -- 172.21.15.42:0/3648964086 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa52010b680 msgr2=0x7fa520084fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:14:24.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.634+0000 7fa524e4c640 1 --2- 172.21.15.42:0/3648964086 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa52010b680 0x7fa520084fe0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fa514005ab0 tx=0x7fa514005040 comp rx=0 tx=0).stop 2024-08-30T09:14:24.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.634+0000 7fa524e4c640 1 -- 172.21.15.42:0/3648964086 shutdown_connections 2024-08-30T09:14:24.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.634+0000 7fa524e4c640 1 --2- 172.21.15.42:0/3648964086 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fa5080b4dd0 0x7fa5080b72e0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:24.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.634+0000 7fa524e4c640 1 --2- 172.21.15.42:0/3648964086 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa52010f660 0x7fa52007f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:24.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.634+0000 7fa524e4c640 1 --2- 172.21.15.42:0/3648964086 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa52010ffb0 0x7fa52007f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:24.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.634+0000 7fa524e4c640 1 --2- 172.21.15.42:0/3648964086 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa52010b680 0x7fa520084fe0 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:24.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.634+0000 7fa524e4c640 1 -- 172.21.15.42:0/3648964086 >> 172.21.15.42:0/3648964086 conn(0x7fa52006e790 msgr2=0x7fa52010d850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:14:24.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.634+0000 7fa524e4c640 1 -- 172.21.15.42:0/3648964086 shutdown_connections 2024-08-30T09:14:24.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:24.634+0000 7fa524e4c640 1 -- 172.21.15.42:0/3648964086 wait complete. 2024-08-30T09:14:24.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:14:24 smithi190 ceph-mon[22001]: pgmap v782: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.7 MiB/s wr, 501 op/s 2024-08-30T09:14:25.953 INFO:teuthology.orchestra.run.smithi042.stderr:progress: 820.0 s, 403.2 tps, lat 239.686 ms stddev 268.291 2024-08-30T09:14:26.529 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-08-30T09:14:26.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:14:26 smithi167 ceph-mon[22063]: pgmap v783: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 3.7 MiB/s wr, 390 op/s 2024-08-30T09:14:26.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:14:26 smithi042 ceph-mon[18652]: pgmap v783: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 3.7 MiB/s wr, 390 op/s 2024-08-30T09:14:26.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:14:26 smithi190 ceph-mon[22001]: pgmap v783: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 3.7 MiB/s wr, 390 op/s 2024-08-30T09:14:26.991 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T09:14:27.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.703+0000 7f5cf0fa8640 1 -- 172.21.15.42:0/2660536984 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5cec11c940 msgr2=0x7f5cec11ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:14:27.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.703+0000 7f5cf0fa8640 1 --2- 172.21.15.42:0/2660536984 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5cec11c940 0x7f5cec11ed60 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f5ce0009860 tx=0x7f5ce002a380 comp rx=0 tx=0).stop 2024-08-30T09:14:27.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.703+0000 7f5cf0fa8640 1 -- 172.21.15.42:0/2660536984 shutdown_connections 2024-08-30T09:14:27.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.703+0000 7f5cf0fa8640 1 --2- 172.21.15.42:0/2660536984 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5cec11c940 0x7f5cec11ed60 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:27.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.703+0000 7f5cf0fa8640 1 --2- 172.21.15.42:0/2660536984 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5cec10d990 0x7f5cec10dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:27.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.703+0000 7f5cf0fa8640 1 --2- 172.21.15.42:0/2660536984 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5cec111a70 0x7f5cec10d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:27.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.703+0000 7f5cf0fa8640 1 -- 172.21.15.42:0/2660536984 >> 172.21.15.42:0/2660536984 conn(0x7f5cec06e790 msgr2=0x7f5cec06ebd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:14:27.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.707+0000 7f5cf0fa8640 1 -- 172.21.15.42:0/2660536984 shutdown_connections 2024-08-30T09:14:27.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.710+0000 7f5cf0fa8640 1 -- 172.21.15.42:0/2660536984 wait complete. 2024-08-30T09:14:27.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.711+0000 7f5cf0fa8640 1 Processor -- start 2024-08-30T09:14:27.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.711+0000 7f5cf0fa8640 1 -- start start 2024-08-30T09:14:27.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.712+0000 7f5cf0fa8640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5cec10d990 0x7f5cec11c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:14:27.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.712+0000 7f5cf0fa8640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5cec111a70 0x7f5cec116560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:14:27.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.712+0000 7f5cf0fa8640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5cec11c940 0x7f5cec116bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:14:27.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.712+0000 7f5ceb7fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5cec10d990 0x7f5cec11c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:14:27.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.712+0000 7f5ceb7fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5cec10d990 0x7f5cec11c2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60644/0 (socket says 172.21.15.42:60644) 2024-08-30T09:14:27.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.712+0000 7f5ceb7fe640 1 -- 172.21.15.42:0/1016812104 learned_addr learned my addr 172.21.15.42:0/1016812104 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:14:27.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.712+0000 7f5cf0fa8640 1 -- 172.21.15.42:0/1016812104 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f5cec112a00 con 0x7f5cec10d990 2024-08-30T09:14:27.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.712+0000 7f5cf0fa8640 1 -- 172.21.15.42:0/1016812104 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f5cec112880 con 0x7f5cec11c940 2024-08-30T09:14:27.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.712+0000 7f5cf0fa8640 1 -- 172.21.15.42:0/1016812104 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f5cec112b80 con 0x7f5cec111a70 2024-08-30T09:14:27.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.712+0000 7f5cebfff640 1 --2- 172.21.15.42:0/1016812104 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5cec11c940 0x7f5cec116bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:14:27.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.717+0000 7f5ceb7fe640 1 -- 172.21.15.42:0/1016812104 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5cec111a70 msgr2=0x7f5cec116560 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-08-30T09:14:27.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.717+0000 7f5ceb7fe640 1 --2- 172.21.15.42:0/1016812104 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5cec111a70 0x7f5cec116560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:27.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.717+0000 7f5ceb7fe640 1 -- 172.21.15.42:0/1016812104 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5cec11c940 msgr2=0x7f5cec116bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:14:27.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.717+0000 7f5ceb7fe640 1 --2- 172.21.15.42:0/1016812104 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5cec11c940 0x7f5cec116bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:27.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.717+0000 7f5ceb7fe640 1 -- 172.21.15.42:0/1016812104 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cec075390 con 0x7f5cec10d990 2024-08-30T09:14:27.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.717+0000 7f5ceb7fe640 1 --2- 172.21.15.42:0/1016812104 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5cec10d990 0x7f5cec11c2a0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f5ce400ebc0 tx=0x7f5ce400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:14:27.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.723+0000 7f5ce8ff9640 1 -- 172.21.15.42:0/1016812104 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ce4003b70 con 0x7f5cec10d990 2024-08-30T09:14:27.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.724+0000 7f5cf0fa8640 1 -- 172.21.15.42:0/1016812104 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5cec075680 con 0x7f5cec10d990 2024-08-30T09:14:27.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.724+0000 7f5cf0fa8640 1 -- 172.21.15.42:0/1016812104 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5cec075bc0 con 0x7f5cec10d990 2024-08-30T09:14:27.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.724+0000 7f5ce8ff9640 1 -- 172.21.15.42:0/1016812104 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ce4003d10 con 0x7f5cec10d990 2024-08-30T09:14:27.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.724+0000 7f5ce8ff9640 1 -- 172.21.15.42:0/1016812104 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ce40198a0 con 0x7f5cec10d990 2024-08-30T09:14:27.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.730+0000 7f5cf877f740 1 Processor -- start 2024-08-30T09:14:27.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.730+0000 7f5cf877f740 1 -- start start 2024-08-30T09:14:27.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.731+0000 7f5cf877f740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b4e0760070 0x55b4e0b054f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:14:27.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.731+0000 7f5cf877f740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b4e07371b0 0x55b4e0b05a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:14:27.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.731+0000 7f5cf877f740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b4e073f390 0x55b4e0b05f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:14:27.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.731+0000 7f5cf877f740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55b4e0a7c680 con 0x55b4e0760070 2024-08-30T09:14:27.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.731+0000 7f5cf877f740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55b4e0a6bc10 con 0x55b4e07371b0 2024-08-30T09:14:27.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.731+0000 7f5cf877f740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55b4e0a6fbc0 con 0x55b4e073f390 2024-08-30T09:14:27.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.734+0000 7f5ceaffd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b4e073f390 0x55b4e0b05f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:14:27.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.737+0000 7f5ceaffd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b4e073f390 0x55b4e0b05f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:36342/0 (socket says 172.21.15.42:36342) 2024-08-30T09:14:27.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.737+0000 7f5ceaffd640 1 -- 172.21.15.42:0/2611105762 learned_addr learned my addr 172.21.15.42:0/2611105762 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:14:27.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.739+0000 7f5ceaffd640 1 -- 172.21.15.42:0/2611105762 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b4e07371b0 msgr2=0x55b4e0b05a30 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-30T09:14:27.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.739+0000 7f5ceaffd640 1 --2- 172.21.15.42:0/2611105762 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b4e07371b0 0x55b4e0b05a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:27.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.739+0000 7f5ceaffd640 1 -- 172.21.15.42:0/2611105762 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b4e0760070 msgr2=0x55b4e0b054f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:14:27.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.739+0000 7f5ceaffd640 1 --2- 172.21.15.42:0/2611105762 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b4e0760070 0x55b4e0b054f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:27.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.739+0000 7f5ceaffd640 1 -- 172.21.15.42:0/2611105762 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b4e07cb650 con 0x55b4e073f390 2024-08-30T09:14:27.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.750+0000 7f5ceaffd640 1 --2- 172.21.15.42:0/2611105762 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b4e073f390 0x55b4e0b05f70 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f5cdc009a30 tx=0x7f5cdc009ef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=857249bd8ec79db4 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:14:27.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.750+0000 7f5ce8ff9640 1 -- 172.21.15.42:0/1016812104 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7f5ce4019a40 con 0x7f5cec10d990 2024-08-30T09:14:27.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.751+0000 7f5ce8ff9640 1 --2- 172.21.15.42:0/1016812104 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f5cd40b4d40 0x7f5cd40b7250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:14:27.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.752+0000 7f5ce8ff9640 1 -- 172.21.15.42:0/1016812104 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f5ce40e5c00 con 0x7f5cec10d990 2024-08-30T09:14:27.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.754+0000 7f5cd1ffb640 1 -- 172.21.15.42:0/2611105762 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cdc00ec30 con 0x55b4e073f390 2024-08-30T09:14:27.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.754+0000 7f5cd1ffb640 1 -- 172.21.15.42:0/2611105762 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5cdc00edd0 con 0x55b4e073f390 2024-08-30T09:14:27.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.754+0000 7f5cd1ffb640 1 -- 172.21.15.42:0/2611105762 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cdc004a00 con 0x55b4e073f390 2024-08-30T09:14:27.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.754+0000 7f5cebfff640 1 --2- 172.21.15.42:0/1016812104 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f5cd40b4d40 0x7f5cd40b7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:14:27.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.755+0000 7f5cf877f740 1 -- 172.21.15.42:0/2611105762 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b4e073f390 msgr2=0x55b4e0b05f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:14:27.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.755+0000 7f5cf877f740 1 --2- 172.21.15.42:0/2611105762 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b4e073f390 0x55b4e0b05f70 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f5cdc009a30 tx=0x7f5cdc009ef0 comp rx=0 tx=0).stop 2024-08-30T09:14:27.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.759+0000 7f5cebfff640 1 --2- 172.21.15.42:0/1016812104 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f5cd40b4d40 0x7f5cd40b7250 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f5ce0009830 tx=0x7f5ce0009390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:14:27.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.759+0000 7f5cf877f740 1 -- 172.21.15.42:0/2611105762 shutdown_connections 2024-08-30T09:14:27.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.759+0000 7f5cf877f740 1 --2- 172.21.15.42:0/2611105762 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b4e073f390 0x55b4e0b05f70 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:27.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.759+0000 7f5cf877f740 1 --2- 172.21.15.42:0/2611105762 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b4e07371b0 0x55b4e0b05a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:27.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.759+0000 7f5cf877f740 1 --2- 172.21.15.42:0/2611105762 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b4e0760070 0x55b4e0b054f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:27.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.759+0000 7f5cf877f740 1 -- 172.21.15.42:0/2611105762 >> 172.21.15.42:0/2611105762 conn(0x55b4e0754460 msgr2=0x55b4e0b033e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:14:27.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.759+0000 7f5cf877f740 1 -- 172.21.15.42:0/2611105762 shutdown_connections 2024-08-30T09:14:27.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.760+0000 7f5cf877f740 1 -- 172.21.15.42:0/2611105762 wait complete. 2024-08-30T09:14:27.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.761+0000 7f5cf877f740 1 -- start start 2024-08-30T09:14:27.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.761+0000 7f5cf877f740 1 Processor -- start 2024-08-30T09:14:27.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.763+0000 7f5cf877f740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b4e07371b0 0x55b4e0b10c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:14:27.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.763+0000 7f5cf877f740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b4e073f390 0x55b4e0b111c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:14:27.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.763+0000 7f5cf877f740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b4e0760070 0x55b4e0b1d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:14:27.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.763+0000 7f5cf877f740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55b4e08ce700 con 0x55b4e0760070 2024-08-30T09:14:27.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.763+0000 7f5cf877f740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55b4e09f2750 con 0x55b4e07371b0 2024-08-30T09:14:27.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.763+0000 7f5cf877f740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55b4e0b13cd0 con 0x55b4e073f390 2024-08-30T09:14:27.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.763+0000 7f5cf877f740 10 client.0 _ll_register_callbacks cb 0x55b4e0a21e10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:14:27.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.764+0000 7f5cebfff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b4e0760070 0x55b4e0b1d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:14:27.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.764+0000 7f5cebfff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b4e0760070 0x55b4e0b1d050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60654/0 (socket says 172.21.15.42:60654) 2024-08-30T09:14:27.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.764+0000 7f5cebfff640 1 -- 172.21.15.42:0/2774721738 learned_addr learned my addr 172.21.15.42:0/2774721738 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:14:27.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.764+0000 7f5ceaffd640 1 --2- 172.21.15.42:0/2774721738 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b4e073f390 0x55b4e0b111c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:14:27.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.764+0000 7f5ceaffd640 1 -- 172.21.15.42:0/2774721738 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b4e07371b0 msgr2=0x55b4e0b10c80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-30T09:14:27.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.764+0000 7f5ceaffd640 1 --2- 172.21.15.42:0/2774721738 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b4e07371b0 0x55b4e0b10c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:27.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.764+0000 7f5ceaffd640 1 -- 172.21.15.42:0/2774721738 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b4e0760070 msgr2=0x55b4e0b1d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:14:27.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.764+0000 7f5ceaffd640 1 --2- 172.21.15.42:0/2774721738 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b4e0760070 0x55b4e0b1d050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:27.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.764+0000 7f5ceaffd640 1 -- 172.21.15.42:0/2774721738 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b4e0a2b900 con 0x55b4e073f390 2024-08-30T09:14:27.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.765+0000 7f5ceaffd640 1 --2- 172.21.15.42:0/2774721738 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b4e073f390 0x55b4e0b111c0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f5cdc005560 tx=0x7f5cdc00ec60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=87d61e58d56e65f5 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:14:27.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.765+0000 7f5ccaffd640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-08-30T09:14:27.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.765+0000 7f5ccaffd640 1 -- 172.21.15.42:0/2774721738 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cdc004450 con 0x55b4e073f390 2024-08-30T09:14:27.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.765+0000 7f5cf877f740 1 -- 172.21.15.42:0/2774721738 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5cdc00edd0 con 0x55b4e073f390 2024-08-30T09:14:27.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.765+0000 7f5ccaffd640 1 -- 172.21.15.42:0/2774721738 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f5cdc0046c0 con 0x55b4e073f390 2024-08-30T09:14:27.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.769+0000 7f5ccaffd640 1 -- 172.21.15.42:0/2774721738 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f5cdc0048c0 con 0x55b4e073f390 2024-08-30T09:14:27.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.769+0000 7f5cf877f740 10 client.27466 fetch_fsmap learned FSMap version 47 2024-08-30T09:14:27.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.769+0000 7f5cf877f740 1 -- 172.21.15.42:0/2774721738 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55b4e0b1faf0 con 0x55b4e073f390 2024-08-30T09:14:27.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.772+0000 7f5ccaffd640 1 -- 172.21.15.42:0/2774721738 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5cdc00edd0 con 0x55b4e073f390 2024-08-30T09:14:27.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.772+0000 7f5ccaffd640 1 -- 172.21.15.42:0/2774721738 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 47) ==== 8370+0+0 (secure 0 0 0) 0x7f5cdc018470 con 0x55b4e073f390 2024-08-30T09:14:27.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.772+0000 7f5cf877f740 10 client.27466 fetch_fsmap finished waiting for FSMap version 47 2024-08-30T09:14:27.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.772+0000 7f5cf877f740 10 client.27466 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-30T09:14:27.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.772+0000 7f5cf877f740 20 client.27466 populate_metadata read hostname 'smithi042' 2024-08-30T09:14:27.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.772+0000 7f5cf877f740 1 --2- 172.21.15.42:0/2774721738 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55b4e071e9f0 0x55b4e0b23350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:14:27.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.772+0000 7f5cf877f740 4 client.27466 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-30T09:14:27.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.772+0000 7f5cf877f740 1 -- 172.21.15.42:0/2774721738 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b4e0a7e5d0 con 0x55b4e071e9f0 2024-08-30T09:14:27.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.776+0000 7f5ceb7fe640 1 --2- 172.21.15.42:0/2774721738 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55b4e071e9f0 0x55b4e0b23350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:14:27.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.778+0000 7f5ceb7fe640 1 --2- 172.21.15.42:0/2774721738 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55b4e071e9f0 0x55b4e0b23350 crc :-1 s=READY pgs=279 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a89cfa249163b632 server_cookie=7d2b1c353de7509a in_seq=0 out_seq=0 2024-08-30T09:14:27.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.780+0000 7f5ccaffd640 10 client.27466 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:14:27.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.784+0000 7f5ccaffd640 1 -- 172.21.15.42:0/2774721738 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b4e0a7e5d0 con 0x55b4e071e9f0 2024-08-30T09:14:27.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.784+0000 7f5ccaffd640 10 client.27466 handle_command_reply: tid=0 2024-08-30T09:14:27.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.784+0000 7f5cf877f740 1 client.27466 shutdown 2024-08-30T09:14:27.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.791+0000 7f5cf877f740 1 -- 172.21.15.42:0/2774721738 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b4e073f390 msgr2=0x55b4e0b111c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:14:27.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.791+0000 7f5cf877f740 1 --2- 172.21.15.42:0/2774721738 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b4e073f390 0x55b4e0b111c0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f5cdc005560 tx=0x7f5cdc00ec60 comp rx=0 tx=0).stop 2024-08-30T09:14:27.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.791+0000 7f5cf877f740 1 -- 172.21.15.42:0/2774721738 shutdown_connections 2024-08-30T09:14:27.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.791+0000 7f5cf877f740 1 --2- 172.21.15.42:0/2774721738 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55b4e071e9f0 0x55b4e0b23350 crc :-1 s=READY pgs=279 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:27.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.791+0000 7f5cf877f740 1 --2- 172.21.15.42:0/2774721738 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b4e073f390 0x55b4e0b111c0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:27.796 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.791+0000 7f5cf877f740 1 --2- 172.21.15.42:0/2774721738 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b4e0760070 0x55b4e0b1d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:27.796 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.791+0000 7f5cf877f740 1 --2- 172.21.15.42:0/2774721738 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b4e07371b0 0x55b4e0b10c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:27.796 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.791+0000 7f5cf877f740 1 -- 172.21.15.42:0/2774721738 >> 172.21.15.42:0/2774721738 conn(0x55b4e0754460 msgr2=0x55b4e0b08680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:14:27.796 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.791+0000 7f5ccaffd640 0 client.27466 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:14:27.796 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.794+0000 7f5cf877f740 1 -- 172.21.15.42:0/2774721738 shutdown_connections 2024-08-30T09:14:27.797 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.795+0000 7f5cf877f740 1 -- 172.21.15.42:0/2774721738 wait complete. 2024-08-30T09:14:27.797 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.795+0000 7f5cf877f740 20 client.27466 trim_cache size 0 max 16384 2024-08-30T09:14:27.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.820+0000 7f5cf877f740 1 Processor -- start 2024-08-30T09:14:27.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.820+0000 7f5cf877f740 1 -- start start 2024-08-30T09:14:27.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.821+0000 7f5cf877f740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b4e07371b0 0x55b4e0b03150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:14:27.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.821+0000 7f5cf877f740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b4e073f390 0x55b4e0b0de70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:14:27.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.821+0000 7f5ceb7fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b4e07371b0 0x55b4e0b03150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:14:27.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.821+0000 7f5ceb7fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b4e07371b0 0x55b4e0b03150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:54776/0 (socket says 172.21.15.42:54776) 2024-08-30T09:14:27.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.821+0000 7f5cf877f740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b4e0760070 0x55b4e0b0e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:14:27.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.821+0000 7f5cebfff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b4e0760070 0x55b4e0b0e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:14:27.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.821+0000 7f5ceaffd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b4e073f390 0x55b4e0b0de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:14:27.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.821+0000 7f5ceaffd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b4e073f390 0x55b4e0b0de70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60668/0 (socket says 172.21.15.42:60668) 2024-08-30T09:14:27.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.821+0000 7f5cebfff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b4e0760070 0x55b4e0b0e3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:36354/0 (socket says 172.21.15.42:36354) 2024-08-30T09:14:27.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.821+0000 7f5ceb7fe640 1 -- 172.21.15.42:0/174618753 learned_addr learned my addr 172.21.15.42:0/174618753 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:14:27.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.821+0000 7f5cf877f740 1 -- 172.21.15.42:0/174618753 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f5cd8012600 con 0x55b4e073f390 2024-08-30T09:14:27.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.821+0000 7f5cf877f740 1 -- 172.21.15.42:0/174618753 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f5cd800fe70 con 0x55b4e07371b0 2024-08-30T09:14:27.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.821+0000 7f5cf877f740 1 -- 172.21.15.42:0/174618753 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f5cd800dcc0 con 0x55b4e0760070 2024-08-30T09:14:27.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.822+0000 7f5ceaffd640 1 -- 172.21.15.42:0/174618753 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b4e0760070 msgr2=0x55b4e0b0e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:14:27.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.822+0000 7f5ceaffd640 1 --2- 172.21.15.42:0/174618753 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b4e0760070 0x55b4e0b0e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:27.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.822+0000 7f5ceaffd640 1 -- 172.21.15.42:0/174618753 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b4e07371b0 msgr2=0x55b4e0b03150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:14:27.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.822+0000 7f5ceaffd640 1 --2- 172.21.15.42:0/174618753 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b4e07371b0 0x55b4e0b03150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:27.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.822+0000 7f5ceaffd640 1 -- 172.21.15.42:0/174618753 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b4e0afd520 con 0x55b4e073f390 2024-08-30T09:14:27.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.823+0000 7f5ceaffd640 1 --2- 172.21.15.42:0/174618753 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b4e073f390 0x55b4e0b0de70 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f5cdc000c00 tx=0x7f5cdc0054a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59082cb7eb40771c server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:14:27.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.824+0000 7f5cd17fa640 1 -- 172.21.15.42:0/174618753 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cdc005850 con 0x55b4e073f390 2024-08-30T09:14:27.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.824+0000 7f5cd17fa640 1 -- 172.21.15.42:0/174618753 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5cdc009600 con 0x55b4e073f390 2024-08-30T09:14:27.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.824+0000 7f5cd17fa640 1 -- 172.21.15.42:0/174618753 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cdc0044d0 con 0x55b4e073f390 2024-08-30T09:14:27.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.825+0000 7f5cf877f740 1 -- 172.21.15.42:0/174618753 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b4e073f390 msgr2=0x55b4e0b0de70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:14:27.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.825+0000 7f5cf877f740 1 --2- 172.21.15.42:0/174618753 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b4e073f390 0x55b4e0b0de70 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f5cdc000c00 tx=0x7f5cdc0054a0 comp rx=0 tx=0).stop 2024-08-30T09:14:27.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.826+0000 7f5cf877f740 1 -- 172.21.15.42:0/174618753 shutdown_connections 2024-08-30T09:14:27.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.826+0000 7f5cf877f740 1 --2- 172.21.15.42:0/174618753 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b4e0760070 0x55b4e0b0e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:27.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.826+0000 7f5cf877f740 1 --2- 172.21.15.42:0/174618753 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b4e073f390 0x55b4e0b0de70 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:27.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.826+0000 7f5cf877f740 1 --2- 172.21.15.42:0/174618753 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b4e07371b0 0x55b4e0b03150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:27.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.826+0000 7f5cf877f740 1 -- 172.21.15.42:0/174618753 >> 172.21.15.42:0/174618753 conn(0x55b4e0754460 msgr2=0x55b4e0b02a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:14:27.829 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.827+0000 7f5cf877f740 1 -- 172.21.15.42:0/174618753 shutdown_connections 2024-08-30T09:14:27.829 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.827+0000 7f5cf877f740 1 -- 172.21.15.42:0/174618753 wait complete. 2024-08-30T09:14:27.829 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.827+0000 7f5cf877f740 1 -- start start 2024-08-30T09:14:27.829 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.828+0000 7f5cf877f740 1 Processor -- start 2024-08-30T09:14:27.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.828+0000 7f5cf877f740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b4e07371b0 0x55b4e0b19890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:14:27.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.829+0000 7f5cf877f740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b4e073f390 0x55b4e0b052b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:14:27.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.829+0000 7f5cf877f740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b4e0760070 0x55b4e0b058e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:14:27.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.829+0000 7f5ceb7fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b4e07371b0 0x55b4e0b19890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:14:27.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.829+0000 7f5ceaffd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b4e073f390 0x55b4e0b052b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:14:27.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.829+0000 7f5ceb7fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b4e07371b0 0x55b4e0b19890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60680/0 (socket says 172.21.15.42:60680) 2024-08-30T09:14:27.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.829+0000 7f5ceaffd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b4e073f390 0x55b4e0b052b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:54792/0 (socket says 172.21.15.42:54792) 2024-08-30T09:14:27.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.829+0000 7f5ceb7fe640 1 -- 172.21.15.42:0/1263209994 learned_addr learned my addr 172.21.15.42:0/1263209994 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:14:27.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.829+0000 7f5cf877f740 1 -- 172.21.15.42:0/1263209994 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f5cd800e330 con 0x55b4e07371b0 2024-08-30T09:14:27.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.829+0000 7f5cf877f740 1 -- 172.21.15.42:0/1263209994 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f5cd800f7a0 con 0x55b4e073f390 2024-08-30T09:14:27.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.829+0000 7f5cebfff640 1 --2- 172.21.15.42:0/1263209994 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b4e0760070 0x55b4e0b058e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:14:27.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.829+0000 7f5cf877f740 1 -- 172.21.15.42:0/1263209994 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f5cd800ea00 con 0x55b4e0760070 2024-08-30T09:14:27.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.829+0000 7f5cf877f740 10 client.0 _ll_register_callbacks cb 0x7f5cdc0053f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:14:27.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.830+0000 7f5cebfff640 1 -- 172.21.15.42:0/1263209994 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b4e073f390 msgr2=0x55b4e0b052b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:14:27.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.830+0000 7f5cebfff640 1 --2- 172.21.15.42:0/1263209994 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b4e073f390 0x55b4e0b052b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:27.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.830+0000 7f5cebfff640 1 -- 172.21.15.42:0/1263209994 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b4e07371b0 msgr2=0x55b4e0b19890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:14:27.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.830+0000 7f5cebfff640 1 --2- 172.21.15.42:0/1263209994 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b4e07371b0 0x55b4e0b19890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:27.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.830+0000 7f5cebfff640 1 -- 172.21.15.42:0/1263209994 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b4e0b06070 con 0x55b4e0760070 2024-08-30T09:14:27.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.831+0000 7f5cebfff640 1 --2- 172.21.15.42:0/1263209994 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b4e0760070 0x55b4e0b058e0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f5ce00095f0 tx=0x7f5ce002a380 comp rx=0 tx=0).ready entity=mon.1 client_cookie=91ca22ed144fc098 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:14:27.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.831+0000 7f5ccaffd640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-08-30T09:14:27.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.831+0000 7f5ccaffd640 1 -- 172.21.15.42:0/1263209994 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ce0050070 con 0x55b4e0760070 2024-08-30T09:14:27.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.831+0000 7f5ccaffd640 1 -- 172.21.15.42:0/1263209994 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f5ce0005240 con 0x55b4e0760070 2024-08-30T09:14:27.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.831+0000 7f5cf877f740 1 -- 172.21.15.42:0/1263209994 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5cdc009600 con 0x55b4e0760070 2024-08-30T09:14:27.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.832+0000 7f5ccaffd640 1 -- 172.21.15.42:0/1263209994 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f5ce0037040 con 0x55b4e0760070 2024-08-30T09:14:27.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.832+0000 7f5cf877f740 10 client.27472 fetch_fsmap learned FSMap version 47 2024-08-30T09:14:27.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.832+0000 7f5cf877f740 1 -- 172.21.15.42:0/1263209994 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55b4e0b8ddb0 con 0x55b4e0760070 2024-08-30T09:14:27.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.832+0000 7f5ccaffd640 1 -- 172.21.15.42:0/1263209994 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5cdc009600 con 0x55b4e0760070 2024-08-30T09:14:27.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.832+0000 7f5ccaffd640 1 -- 172.21.15.42:0/1263209994 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 47) ==== 8370+0+0 (secure 0 0 0) 0x7f5ce004b040 con 0x55b4e0760070 2024-08-30T09:14:27.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.832+0000 7f5cf877f740 10 client.27472 fetch_fsmap finished waiting for FSMap version 47 2024-08-30T09:14:27.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.833+0000 7f5cf877f740 10 client.27472 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-30T09:14:27.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.833+0000 7f5cf877f740 20 client.27472 populate_metadata read hostname 'smithi042' 2024-08-30T09:14:27.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.833+0000 7f5cf877f740 1 --2- 172.21.15.42:0/1263209994 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55b4e06f3da0 0x55b4e0b90500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:14:27.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.833+0000 7f5cf877f740 4 client.27472 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-30T09:14:27.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.833+0000 7f5cf877f740 1 -- 172.21.15.42:0/1263209994 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b4e0a7a070 con 0x55b4e06f3da0 2024-08-30T09:14:27.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.833+0000 7f5ceb7fe640 1 --2- 172.21.15.42:0/1263209994 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55b4e06f3da0 0x55b4e0b90500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:14:27.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.834+0000 7f5ccaffd640 10 client.27472 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:14:27.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.835+0000 7f5ceb7fe640 1 --2- 172.21.15.42:0/1263209994 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55b4e06f3da0 0x55b4e0b90500 crc :-1 s=READY pgs=280 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b4fe386ce85edd5a server_cookie=1b1c7d4a6a59b47f in_seq=0 out_seq=0 2024-08-30T09:14:27.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.835+0000 7f5ccaffd640 1 -- 172.21.15.42:0/1263209994 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b4e0a7a070 con 0x55b4e06f3da0 2024-08-30T09:14:27.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.835+0000 7f5ccaffd640 10 client.27472 handle_command_reply: tid=0 2024-08-30T09:14:27.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.835+0000 7f5cf877f740 1 client.27472 shutdown 2024-08-30T09:14:27.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.836+0000 7f5cf877f740 1 -- 172.21.15.42:0/1263209994 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b4e0760070 msgr2=0x55b4e0b058e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:14:27.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.836+0000 7f5cf877f740 1 --2- 172.21.15.42:0/1263209994 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b4e0760070 0x55b4e0b058e0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f5ce00095f0 tx=0x7f5ce002a380 comp rx=0 tx=0).stop 2024-08-30T09:14:27.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.836+0000 7f5cf877f740 1 -- 172.21.15.42:0/1263209994 shutdown_connections 2024-08-30T09:14:27.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.836+0000 7f5cf877f740 1 --2- 172.21.15.42:0/1263209994 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55b4e06f3da0 0x55b4e0b90500 crc :-1 s=READY pgs=280 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:27.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.836+0000 7f5ccaffd640 0 client.27472 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:14:27.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.836+0000 7f5cf877f740 1 --2- 172.21.15.42:0/1263209994 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b4e0760070 0x55b4e0b058e0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:27.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.836+0000 7f5cf877f740 1 --2- 172.21.15.42:0/1263209994 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b4e073f390 0x55b4e0b052b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:27.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.836+0000 7f5cf877f740 1 --2- 172.21.15.42:0/1263209994 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b4e07371b0 0x55b4e0b19890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:27.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.836+0000 7f5cf877f740 1 -- 172.21.15.42:0/1263209994 >> 172.21.15.42:0/1263209994 conn(0x55b4e0754460 msgr2=0x55b4e0b0f100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:14:27.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.837+0000 7f5cf877f740 1 -- 172.21.15.42:0/1263209994 shutdown_connections 2024-08-30T09:14:27.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.837+0000 7f5cf877f740 1 -- 172.21.15.42:0/1263209994 wait complete. 2024-08-30T09:14:27.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.837+0000 7f5cf877f740 20 client.27472 trim_cache size 0 max 16384 2024-08-30T09:14:27.840 INFO:teuthology.orchestra.run.smithi042.stdout:[] 2024-08-30T09:14:27.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.839+0000 7f5cf0fa8640 1 -- 172.21.15.42:0/1016812104 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f5cd40b4d40 msgr2=0x7f5cd40b7250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:14:27.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.839+0000 7f5cf0fa8640 1 --2- 172.21.15.42:0/1016812104 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f5cd40b4d40 0x7f5cd40b7250 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f5ce0009830 tx=0x7f5ce0009390 comp rx=0 tx=0).stop 2024-08-30T09:14:27.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.839+0000 7f5cf0fa8640 1 -- 172.21.15.42:0/1016812104 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5cec10d990 msgr2=0x7f5cec11c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:14:27.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.839+0000 7f5cf0fa8640 1 --2- 172.21.15.42:0/1016812104 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5cec10d990 0x7f5cec11c2a0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f5ce400ebc0 tx=0x7f5ce400c6a0 comp rx=0 tx=0).stop 2024-08-30T09:14:27.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.839+0000 7f5cf0fa8640 1 -- 172.21.15.42:0/1016812104 shutdown_connections 2024-08-30T09:14:27.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.839+0000 7f5cf0fa8640 1 --2- 172.21.15.42:0/1016812104 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f5cd40b4d40 0x7f5cd40b7250 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:27.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.839+0000 7f5cf0fa8640 1 --2- 172.21.15.42:0/1016812104 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5cec111a70 0x7f5cec116560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:27.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.839+0000 7f5cf0fa8640 1 --2- 172.21.15.42:0/1016812104 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5cec11c940 0x7f5cec116bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:27.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.839+0000 7f5cf0fa8640 1 --2- 172.21.15.42:0/1016812104 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5cec10d990 0x7f5cec11c2a0 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:27.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.839+0000 7f5cf0fa8640 1 -- 172.21.15.42:0/1016812104 >> 172.21.15.42:0/1016812104 conn(0x7f5cec06e790 msgr2=0x7f5cec110720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:14:27.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.839+0000 7f5cf0fa8640 1 -- 172.21.15.42:0/1016812104 shutdown_connections 2024-08-30T09:14:27.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:27.839+0000 7f5cf0fa8640 1 -- 172.21.15.42:0/1016812104 wait complete. 2024-08-30T09:14:28.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:14:28 smithi167 ceph-mon[22063]: pgmap v784: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 3.7 MiB/s wr, 387 op/s 2024-08-30T09:14:28.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:14:28 smithi042 ceph-mon[18652]: pgmap v784: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 3.7 MiB/s wr, 387 op/s 2024-08-30T09:14:28.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:14:28 smithi190 ceph-mon[22001]: pgmap v784: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 3.7 MiB/s wr, 387 op/s 2024-08-30T09:14:29.615 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-08-30T09:14:29.615 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-08-30T09:14:30.122 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T09:14:30.572 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:14:30 smithi042 ceph-mon[18652]: pgmap v785: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 4.5 MiB/s wr, 481 op/s 2024-08-30T09:14:30.572 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:14:30 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-30T09:14:30.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:14:30 smithi167 ceph-mon[22063]: pgmap v785: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 4.5 MiB/s wr, 481 op/s 2024-08-30T09:14:30.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:14:30 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-30T09:14:30.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:14:30 smithi190 ceph-mon[22001]: pgmap v785: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 4.5 MiB/s wr, 481 op/s 2024-08-30T09:14:30.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:14:30 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-30T09:14:30.953 INFO:teuthology.orchestra.run.smithi042.stderr:progress: 825.0 s, 486.4 tps, lat 209.130 ms stddev 259.297 2024-08-30T09:14:31.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.728+0000 7f60205ca640 1 -- 172.21.15.42:0/1269133791 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f601811c9d0 msgr2=0x7f601811eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:14:31.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.728+0000 7f60205ca640 1 --2- 172.21.15.42:0/1269133791 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f601811c9d0 0x7f601811eee0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f6008009860 tx=0x7f600802a360 comp rx=0 tx=0).stop 2024-08-30T09:14:31.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.728+0000 7f60205ca640 1 -- 172.21.15.42:0/1269133791 shutdown_connections 2024-08-30T09:14:31.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.728+0000 7f60205ca640 1 --2- 172.21.15.42:0/1269133791 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f601811c9d0 0x7f601811eee0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:31.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.728+0000 7f60205ca640 1 --2- 172.21.15.42:0/1269133791 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f601810a780 0x7f601810ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:31.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.728+0000 7f60205ca640 1 --2- 172.21.15.42:0/1269133791 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6018111ce0 0x7f601810a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:31.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.728+0000 7f60205ca640 1 -- 172.21.15.42:0/1269133791 >> 172.21.15.42:0/1269133791 conn(0x7f6018070c60 msgr2=0x7f6018071070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:14:31.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.735+0000 7f60205ca640 1 -- 172.21.15.42:0/1269133791 shutdown_connections 2024-08-30T09:14:31.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.735+0000 7f60205ca640 1 -- 172.21.15.42:0/1269133791 wait complete. 2024-08-30T09:14:31.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.736+0000 7f60205ca640 1 Processor -- start 2024-08-30T09:14:31.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.736+0000 7f60205ca640 1 -- start start 2024-08-30T09:14:31.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.737+0000 7f60205ca640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f601810a780 0x7f60181a8fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:14:31.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.737+0000 7f60205ca640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6018111ce0 0x7f60181a9550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:14:31.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.737+0000 7f60205ca640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f601811c9d0 0x7f60181ae7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:14:31.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.737+0000 7f60205ca640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f6018122640 con 0x7f601810a780 2024-08-30T09:14:31.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.737+0000 7f60205ca640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f60181224c0 con 0x7f601811c9d0 2024-08-30T09:14:31.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.737+0000 7f60205ca640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f60181227c0 con 0x7f6018111ce0 2024-08-30T09:14:31.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.743+0000 7f601eb67640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f601811c9d0 0x7f60181ae7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:14:31.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.743+0000 7f601eb67640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f601811c9d0 0x7f60181ae7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:45444/0 (socket says 172.21.15.42:45444) 2024-08-30T09:14:31.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.743+0000 7f601eb67640 1 -- 172.21.15.42:0/503133465 learned_addr learned my addr 172.21.15.42:0/503133465 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:14:31.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.743+0000 7f601db65640 1 --2- 172.21.15.42:0/503133465 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6018111ce0 0x7f60181a9550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:14:31.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.744+0000 7f601eb67640 1 -- 172.21.15.42:0/503133465 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6018111ce0 msgr2=0x7f60181a9550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:14:31.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.744+0000 7f601eb67640 1 --2- 172.21.15.42:0/503133465 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6018111ce0 0x7f60181a9550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:31.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.744+0000 7f601eb67640 1 -- 172.21.15.42:0/503133465 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f601810a780 msgr2=0x7f60181a8fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:14:31.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.744+0000 7f601eb67640 1 --2- 172.21.15.42:0/503133465 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f601810a780 0x7f60181a8fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:31.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.744+0000 7f601eb67640 1 -- 172.21.15.42:0/503133465 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60181aeef0 con 0x7f601811c9d0 2024-08-30T09:14:31.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.744+0000 7f601eb67640 1 --2- 172.21.15.42:0/503133465 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f601811c9d0 0x7f60181ae7f0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f600802a950 tx=0x7f6008032d40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:14:31.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.745+0000 7f600f7fe640 1 -- 172.21.15.42:0/503133465 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6008037040 con 0x7f601811c9d0 2024-08-30T09:14:31.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.745+0000 7f600f7fe640 1 -- 172.21.15.42:0/503133465 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6008041070 con 0x7f601811c9d0 2024-08-30T09:14:31.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.745+0000 7f60205ca640 1 -- 172.21.15.42:0/503133465 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60181af180 con 0x7f601811c9d0 2024-08-30T09:14:31.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.745+0000 7f60205ca640 1 -- 172.21.15.42:0/503133465 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6018112120 con 0x7f601811c9d0 2024-08-30T09:14:31.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.745+0000 7f600f7fe640 1 -- 172.21.15.42:0/503133465 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6008003850 con 0x7f601811c9d0 2024-08-30T09:14:31.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.749+0000 7f600f7fe640 1 -- 172.21.15.42:0/503133465 <== mon.2 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7f6008004050 con 0x7f601811c9d0 2024-08-30T09:14:31.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.750+0000 7f600f7fe640 1 --2- 172.21.15.42:0/503133465 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f60040b4c30 0x7f60040b7140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:14:31.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.751+0000 7f600f7fe640 1 -- 172.21.15.42:0/503133465 <== mon.2 v2:172.21.15.167:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f60081052d0 con 0x7f601811c9d0 2024-08-30T09:14:31.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.752+0000 7f601e366640 1 --2- 172.21.15.42:0/503133465 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f60040b4c30 0x7f60040b7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:14:31.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.767+0000 7f6026b3f740 1 Processor -- start 2024-08-30T09:14:31.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.770+0000 7f6026b3f740 1 -- start start 2024-08-30T09:14:31.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.771+0000 7f6026b3f740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e045f79350 0x55e04631f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:14:31.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.771+0000 7f6026b3f740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e045f50490 0x55e04631fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:14:31.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.771+0000 7f6026b3f740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e045f58670 0x55e046320200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:14:31.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.771+0000 7f6026b3f740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55e0462955f0 con 0x55e045f50490 2024-08-30T09:14:31.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.771+0000 7f6026b3f740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55e046284b80 con 0x55e045f79350 2024-08-30T09:14:31.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.771+0000 7f6026b3f740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55e046288b30 con 0x55e045f58670 2024-08-30T09:14:31.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.772+0000 7f601e366640 1 --2- 172.21.15.42:0/503133465 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f60040b4c30 0x7f60040b7140 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f60140058c0 tx=0x7f6014005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:14:31.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.774+0000 7f601db65640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e045f50490 0x55e04631fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:14:31.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.774+0000 7f601db65640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e045f50490 0x55e04631fcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42926/0 (socket says 172.21.15.42:42926) 2024-08-30T09:14:31.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.774+0000 7f601db65640 1 -- 172.21.15.42:0/3055945200 learned_addr learned my addr 172.21.15.42:0/3055945200 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:14:31.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.774+0000 7f601eb67640 1 --2- 172.21.15.42:0/3055945200 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e045f58670 0x55e046320200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:14:31.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.779+0000 7f601e366640 1 --2- 172.21.15.42:0/3055945200 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e045f79350 0x55e04631f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:14:31.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.779+0000 7f601db65640 1 -- 172.21.15.42:0/3055945200 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e045f58670 msgr2=0x55e046320200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:14:31.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.779+0000 7f601db65640 1 --2- 172.21.15.42:0/3055945200 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e045f58670 0x55e046320200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:31.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.779+0000 7f601db65640 1 -- 172.21.15.42:0/3055945200 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e045f79350 msgr2=0x55e04631f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:14:31.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.779+0000 7f601db65640 1 --2- 172.21.15.42:0/3055945200 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e045f79350 0x55e04631f780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:31.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.779+0000 7f601db65640 1 -- 172.21.15.42:0/3055945200 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e045fe4930 con 0x55e045f50490 2024-08-30T09:14:31.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.788+0000 7f601db65640 1 --2- 172.21.15.42:0/3055945200 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e045f50490 0x55e04631fcc0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f6010009ef0 tx=0x7f60100107e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f559fd8d78c68135 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:14:31.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.790+0000 7f600cff9640 1 -- 172.21.15.42:0/3055945200 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6010004d20 con 0x55e045f50490 2024-08-30T09:14:31.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.790+0000 7f600cff9640 1 -- 172.21.15.42:0/3055945200 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f601000eb10 con 0x55e045f50490 2024-08-30T09:14:31.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.791+0000 7f6026b3f740 1 -- 172.21.15.42:0/3055945200 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e045f50490 msgr2=0x55e04631fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:14:31.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.791+0000 7f6026b3f740 1 --2- 172.21.15.42:0/3055945200 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e045f50490 0x55e04631fcc0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f6010009ef0 tx=0x7f60100107e0 comp rx=0 tx=0).stop 2024-08-30T09:14:31.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.794+0000 7f6026b3f740 1 -- 172.21.15.42:0/3055945200 shutdown_connections 2024-08-30T09:14:31.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.794+0000 7f6026b3f740 1 --2- 172.21.15.42:0/3055945200 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e045f58670 0x55e046320200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:31.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.794+0000 7f6026b3f740 1 --2- 172.21.15.42:0/3055945200 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e045f50490 0x55e04631fcc0 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:31.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.794+0000 7f6026b3f740 1 --2- 172.21.15.42:0/3055945200 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e045f79350 0x55e04631f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:31.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.794+0000 7f6026b3f740 1 -- 172.21.15.42:0/3055945200 >> 172.21.15.42:0/3055945200 conn(0x55e045f6d740 msgr2=0x55e04631d670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:14:31.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.796+0000 7f6026b3f740 1 -- 172.21.15.42:0/3055945200 shutdown_connections 2024-08-30T09:14:31.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.798+0000 7f6026b3f740 1 -- 172.21.15.42:0/3055945200 wait complete. 2024-08-30T09:14:31.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.798+0000 7f6026b3f740 1 -- start start 2024-08-30T09:14:31.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.798+0000 7f6026b3f740 1 Processor -- start 2024-08-30T09:14:31.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.799+0000 7f6026b3f740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e045f50490 0x55e04632ae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:14:31.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.799+0000 7f6026b3f740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e045f58670 0x55e04632b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:14:31.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.799+0000 7f6026b3f740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e045f79350 0x55e046337320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:14:31.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.799+0000 7f6026b3f740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55e0460e7890 con 0x55e045f50490 2024-08-30T09:14:31.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.799+0000 7f6026b3f740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55e04620b790 con 0x55e045f58670 2024-08-30T09:14:31.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.799+0000 7f6026b3f740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55e04632dfb0 con 0x55e045f79350 2024-08-30T09:14:31.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.799+0000 7f6026b3f740 10 client.0 _ll_register_callbacks cb 0x55e045f90ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:14:31.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.800+0000 7f601db65640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e045f58670 0x55e04632b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:14:31.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.800+0000 7f601db65640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e045f58670 0x55e04632b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:45454/0 (socket says 172.21.15.42:45454) 2024-08-30T09:14:31.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.800+0000 7f601db65640 1 -- 172.21.15.42:0/358904651 learned_addr learned my addr 172.21.15.42:0/358904651 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:14:31.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.800+0000 7f601eb67640 1 --2- 172.21.15.42:0/358904651 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e045f79350 0x55e046337320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:14:31.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.801+0000 7f601db65640 1 -- 172.21.15.42:0/358904651 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e045f79350 msgr2=0x55e046337320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:14:31.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.802+0000 7f601db65640 1 --2- 172.21.15.42:0/358904651 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e045f79350 0x55e046337320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:31.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.802+0000 7f601db65640 1 -- 172.21.15.42:0/358904651 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e045f50490 msgr2=0x55e04632ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:14:31.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.802+0000 7f601e366640 1 --2- 172.21.15.42:0/358904651 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e045f50490 0x55e04632ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:14:31.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.802+0000 7f601db65640 1 --2- 172.21.15.42:0/358904651 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e045f50490 0x55e04632ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:31.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.802+0000 7f601db65640 1 -- 172.21.15.42:0/358904651 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e046242f30 con 0x55e045f58670 2024-08-30T09:14:31.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.802+0000 7f601e366640 1 --2- 172.21.15.42:0/358904651 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e045f50490 0x55e04632ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-30T09:14:31.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.805+0000 7f601db65640 1 --2- 172.21.15.42:0/358904651 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e045f58670 0x55e04632b3a0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f601001a040 tx=0x7f601000e890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b681be4925e4ae7 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:14:31.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.805+0000 7f6001ffb640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-30T09:14:31.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.806+0000 7f6001ffb640 1 -- 172.21.15.42:0/358904651 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6010003b70 con 0x55e045f58670 2024-08-30T09:14:31.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.806+0000 7f6026b3f740 1 -- 172.21.15.42:0/358904651 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f601000eb10 con 0x55e045f58670 2024-08-30T09:14:31.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.806+0000 7f6001ffb640 1 -- 172.21.15.42:0/358904651 <== mon.2 v2:172.21.15.167:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f6010003d10 con 0x55e045f58670 2024-08-30T09:14:31.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.810+0000 7f6001ffb640 1 -- 172.21.15.42:0/358904651 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f6010032050 con 0x55e045f58670 2024-08-30T09:14:31.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.814+0000 7f6026b3f740 10 client.27557 fetch_fsmap learned FSMap version 47 2024-08-30T09:14:31.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.814+0000 7f6026b3f740 1 -- 172.21.15.42:0/358904651 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55e046339d70 con 0x55e045f58670 2024-08-30T09:14:31.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.815+0000 7f6001ffb640 1 -- 172.21.15.42:0/358904651 <== mon.2 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f601000eb10 con 0x55e045f58670 2024-08-30T09:14:31.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.816+0000 7f6001ffb640 1 -- 172.21.15.42:0/358904651 <== mon.2 v2:172.21.15.167:3300/0 5 ==== fsmap(e 47) ==== 8370+0+0 (secure 0 0 0) 0x7f6010017030 con 0x55e045f58670 2024-08-30T09:14:31.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.816+0000 7f6026b3f740 10 client.27557 fetch_fsmap finished waiting for FSMap version 47 2024-08-30T09:14:31.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.816+0000 7f6026b3f740 10 client.27557 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-30T09:14:31.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.816+0000 7f6026b3f740 20 client.27557 populate_metadata read hostname 'smithi042' 2024-08-30T09:14:31.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.816+0000 7f6026b3f740 1 --2- 172.21.15.42:0/358904651 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55e045f37cd0 0x55e04633d680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:14:31.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.816+0000 7f6026b3f740 4 client.27557 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-30T09:14:31.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.816+0000 7f6026b3f740 1 -- 172.21.15.42:0/358904651 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e046297540 con 0x55e045f37cd0 2024-08-30T09:14:31.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.817+0000 7f601e366640 1 --2- 172.21.15.42:0/358904651 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55e045f37cd0 0x55e04633d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:14:31.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.818+0000 7f6001ffb640 10 client.27557 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:14:31.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.818+0000 7f601e366640 1 --2- 172.21.15.42:0/358904651 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55e045f37cd0 0x55e04633d680 crc :-1 s=READY pgs=281 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cbe9e0d49fab1c94 server_cookie=e46ddddbc04d3065 in_seq=0 out_seq=0 2024-08-30T09:14:31.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.819+0000 7f6001ffb640 1 -- 172.21.15.42:0/358904651 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55e046297540 con 0x55e045f37cd0 2024-08-30T09:14:31.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.819+0000 7f6001ffb640 10 client.27557 handle_command_reply: tid=0 2024-08-30T09:14:31.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.819+0000 7f6026b3f740 1 client.27557 shutdown 2024-08-30T09:14:31.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.820+0000 7f6026b3f740 1 -- 172.21.15.42:0/358904651 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e045f58670 msgr2=0x55e04632b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:14:31.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.820+0000 7f6026b3f740 1 --2- 172.21.15.42:0/358904651 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e045f58670 0x55e04632b3a0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f601001a040 tx=0x7f601000e890 comp rx=0 tx=0).stop 2024-08-30T09:14:31.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.822+0000 7f6026b3f740 1 -- 172.21.15.42:0/358904651 shutdown_connections 2024-08-30T09:14:31.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.822+0000 7f6026b3f740 1 --2- 172.21.15.42:0/358904651 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55e045f37cd0 0x55e04633d680 crc :-1 s=READY pgs=281 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:31.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.822+0000 7f6026b3f740 1 --2- 172.21.15.42:0/358904651 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e045f79350 0x55e046337320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:31.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.822+0000 7f6026b3f740 1 --2- 172.21.15.42:0/358904651 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e045f58670 0x55e04632b3a0 secure :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f601001a040 tx=0x7f601000e890 comp rx=0 tx=0).stop 2024-08-30T09:14:31.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.822+0000 7f6026b3f740 1 --2- 172.21.15.42:0/358904651 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e045f50490 0x55e04632ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:31.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.822+0000 7f6001ffb640 0 client.27557 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:14:31.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.822+0000 7f6026b3f740 1 -- 172.21.15.42:0/358904651 >> 172.21.15.42:0/358904651 conn(0x55e045f6d740 msgr2=0x55e046322910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:14:31.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.823+0000 7f6026b3f740 1 -- 172.21.15.42:0/358904651 shutdown_connections 2024-08-30T09:14:31.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.823+0000 7f6026b3f740 1 -- 172.21.15.42:0/358904651 wait complete. 2024-08-30T09:14:31.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.823+0000 7f6026b3f740 20 client.27557 trim_cache size 0 max 16384 2024-08-30T09:14:31.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.854+0000 7f6026b3f740 1 Processor -- start 2024-08-30T09:14:31.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.854+0000 7f6026b3f740 1 -- start start 2024-08-30T09:14:31.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.854+0000 7f6026b3f740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e045f50490 0x55e04632ca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:14:31.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.854+0000 7f6026b3f740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e045f79350 0x55e046337220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:14:31.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.854+0000 7f6026b3f740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e045f0d080 0x55e046328050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:14:31.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.854+0000 7f6026b3f740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f60181ca650 con 0x55e045f50490 2024-08-30T09:14:31.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.854+0000 7f6026b3f740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f60181c7ec0 con 0x55e045f0d080 2024-08-30T09:14:31.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.854+0000 7f6026b3f740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f601810b380 con 0x55e045f79350 2024-08-30T09:14:31.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.855+0000 7f601e366640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e045f50490 0x55e04632ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:14:31.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.855+0000 7f601e366640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e045f50490 0x55e04632ca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42938/0 (socket says 172.21.15.42:42938) 2024-08-30T09:14:31.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.855+0000 7f601e366640 1 -- 172.21.15.42:0/4128578174 learned_addr learned my addr 172.21.15.42:0/4128578174 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:14:31.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.855+0000 7f601db65640 1 --2- 172.21.15.42:0/4128578174 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e045f79350 0x55e046337220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:14:31.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.855+0000 7f601eb67640 1 --2- 172.21.15.42:0/4128578174 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e045f0d080 0x55e046328050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:14:31.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.855+0000 7f601e366640 1 -- 172.21.15.42:0/4128578174 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e045f79350 msgr2=0x55e046337220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:14:31.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.855+0000 7f601e366640 1 --2- 172.21.15.42:0/4128578174 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e045f79350 0x55e046337220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:31.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.855+0000 7f601e366640 1 -- 172.21.15.42:0/4128578174 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e045f0d080 msgr2=0x55e046328050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:14:31.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.855+0000 7f601e366640 1 --2- 172.21.15.42:0/4128578174 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e045f0d080 0x55e046328050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:31.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.855+0000 7f601e366640 1 -- 172.21.15.42:0/4128578174 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e046329dc0 con 0x55e045f50490 2024-08-30T09:14:31.811 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.855+0000 7f601db65640 1 --2- 172.21.15.42:0/4128578174 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e045f79350 0x55e046337220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-30T09:14:31.811 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.855+0000 7f601e366640 1 --2- 172.21.15.42:0/4128578174 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e045f50490 0x55e04632ca90 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x55e04633dde0 tx=0x7f6014009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48f240b50d24a4be server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:14:31.811 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.855+0000 7f6003fff640 1 -- 172.21.15.42:0/4128578174 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60140109a0 con 0x55e045f50490 2024-08-30T09:14:31.811 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.856+0000 7f6003fff640 1 -- 172.21.15.42:0/4128578174 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6014005020 con 0x55e045f50490 2024-08-30T09:14:31.811 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.856+0000 7f6003fff640 1 -- 172.21.15.42:0/4128578174 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6014004b20 con 0x55e045f50490 2024-08-30T09:14:31.811 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.856+0000 7f6026b3f740 1 -- 172.21.15.42:0/4128578174 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e045f50490 msgr2=0x55e04632ca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:14:31.811 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.856+0000 7f6026b3f740 1 --2- 172.21.15.42:0/4128578174 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e045f50490 0x55e04632ca90 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x55e04633dde0 tx=0x7f6014009f90 comp rx=0 tx=0).stop 2024-08-30T09:14:31.811 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.857+0000 7f6026b3f740 1 -- 172.21.15.42:0/4128578174 shutdown_connections 2024-08-30T09:14:31.811 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.857+0000 7f6026b3f740 1 --2- 172.21.15.42:0/4128578174 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e045f79350 0x55e046337220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:31.811 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.857+0000 7f6026b3f740 1 --2- 172.21.15.42:0/4128578174 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e045f0d080 0x55e046328050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:31.811 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.857+0000 7f6026b3f740 1 --2- 172.21.15.42:0/4128578174 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e045f50490 0x55e04632ca90 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:31.811 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.857+0000 7f6026b3f740 1 -- 172.21.15.42:0/4128578174 >> 172.21.15.42:0/4128578174 conn(0x55e045f6d740 msgr2=0x55e04632c380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:14:31.811 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.857+0000 7f6026b3f740 1 -- 172.21.15.42:0/4128578174 shutdown_connections 2024-08-30T09:14:31.811 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.857+0000 7f6026b3f740 1 -- 172.21.15.42:0/4128578174 wait complete. 2024-08-30T09:14:31.811 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.857+0000 7f6026b3f740 1 -- start start 2024-08-30T09:14:31.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.857+0000 7f6026b3f740 1 Processor -- start 2024-08-30T09:14:31.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.858+0000 7f6026b3f740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e045f0d080 0x55e04632fb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:14:31.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.858+0000 7f6026b3f740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e045f50490 0x55e0463300e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:14:31.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.858+0000 7f6026b3f740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e045f79350 0x55e0463a1010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:14:31.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.858+0000 7f601e366640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e045f0d080 0x55e04632fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:14:31.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.858+0000 7f6026b3f740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f601811a3b0 con 0x55e045f79350 2024-08-30T09:14:31.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.858+0000 7f601e366640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e045f0d080 0x55e04632fb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:54200/0 (socket says 172.21.15.42:54200) 2024-08-30T09:14:31.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.858+0000 7f601e366640 1 -- 172.21.15.42:0/82833006 learned_addr learned my addr 172.21.15.42:0/82833006 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:14:31.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.858+0000 7f6026b3f740 1 -- 172.21.15.42:0/82833006 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f60181c77f0 con 0x55e045f50490 2024-08-30T09:14:31.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.858+0000 7f6026b3f740 1 -- 172.21.15.42:0/82833006 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f60181c6a70 con 0x55e045f0d080 2024-08-30T09:14:31.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.858+0000 7f601db65640 1 --2- 172.21.15.42:0/82833006 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e045f50490 0x55e0463300e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:14:31.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.858+0000 7f601eb67640 1 --2- 172.21.15.42:0/82833006 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e045f79350 0x55e0463a1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:14:31.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.858+0000 7f6026b3f740 10 client.0 _ll_register_callbacks cb 0x7f6010010f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:14:31.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.858+0000 7f601e366640 1 -- 172.21.15.42:0/82833006 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e045f50490 msgr2=0x55e0463300e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:14:31.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.858+0000 7f601e366640 1 --2- 172.21.15.42:0/82833006 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e045f50490 0x55e0463300e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:31.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.858+0000 7f601e366640 1 -- 172.21.15.42:0/82833006 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e045f79350 msgr2=0x55e0463a1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:14:31.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.858+0000 7f601e366640 1 --2- 172.21.15.42:0/82833006 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e045f79350 0x55e0463a1010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:31.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.858+0000 7f601e366640 1 -- 172.21.15.42:0/82833006 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e0463a1680 con 0x55e045f0d080 2024-08-30T09:14:31.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.858+0000 7f6026b3f740 1 -- 172.21.15.42:0/82833006 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6014005020 con 0x55e045f0d080 2024-08-30T09:14:31.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.859+0000 7f601e366640 1 --2- 172.21.15.42:0/82833006 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e045f0d080 0x55e04632fb00 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f6014004700 tx=0x7f6014005370 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a0a14b24f8161801 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:14:31.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.859+0000 7f6001ffb640 10 client.27490 ms_handle_connect on v2:172.21.15.190:3300/0 2024-08-30T09:14:31.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.859+0000 7f6001ffb640 1 -- 172.21.15.42:0/82833006 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6014032020 con 0x55e045f0d080 2024-08-30T09:14:31.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.859+0000 7f6001ffb640 1 -- 172.21.15.42:0/82833006 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f6014010bb0 con 0x55e045f0d080 2024-08-30T09:14:31.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.859+0000 7f6001ffb640 1 -- 172.21.15.42:0/82833006 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f6014010df0 con 0x55e045f0d080 2024-08-30T09:14:31.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.859+0000 7f6026b3f740 10 client.27490 fetch_fsmap learned FSMap version 47 2024-08-30T09:14:31.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.859+0000 7f6026b3f740 1 -- 172.21.15.42:0/82833006 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({fsmap=47}) -- 0x55e0463a8880 con 0x55e045f0d080 2024-08-30T09:14:31.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.859+0000 7f6001ffb640 1 -- 172.21.15.42:0/82833006 <== mon.1 v2:172.21.15.190:3300/0 4 ==== fsmap(e 47) ==== 8370+0+0 (secure 0 0 0) 0x7f60140048a0 con 0x55e045f0d080 2024-08-30T09:14:31.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.859+0000 7f6026b3f740 10 client.27490 fetch_fsmap finished waiting for FSMap version 47 2024-08-30T09:14:31.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.859+0000 7f6026b3f740 10 client.27490 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-30T09:14:31.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.859+0000 7f6026b3f740 20 client.27490 populate_metadata read hostname 'smithi042' 2024-08-30T09:14:31.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.860+0000 7f6026b3f740 1 --2- 172.21.15.42:0/82833006 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55e0463a8fa0 0x55e0463ab3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:14:31.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.860+0000 7f6026b3f740 4 client.27490 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-08-30T09:14:31.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.860+0000 7f6026b3f740 1 -- 172.21.15.42:0/82833006 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55e046292fe0 con 0x55e0463a8fa0 2024-08-30T09:14:31.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.860+0000 7f601db65640 1 --2- 172.21.15.42:0/82833006 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55e0463a8fa0 0x55e0463ab3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:14:31.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.861+0000 7f6001ffb640 10 client.27490 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:14:31.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.861+0000 7f601db65640 1 --2- 172.21.15.42:0/82833006 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55e0463a8fa0 0x55e0463ab3c0 crc :-1 s=READY pgs=282 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d56e661cb02dc6cd server_cookie=5aa56faa5bf7ea5d in_seq=0 out_seq=0 2024-08-30T09:14:31.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.862+0000 7f6001ffb640 1 -- 172.21.15.42:0/82833006 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55e046292fe0 con 0x55e0463a8fa0 2024-08-30T09:14:31.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.862+0000 7f6001ffb640 10 client.27490 handle_command_reply: tid=0 2024-08-30T09:14:31.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.862+0000 7f6026b3f740 1 client.27490 shutdown 2024-08-30T09:14:31.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.863+0000 7f6026b3f740 1 -- 172.21.15.42:0/82833006 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e045f0d080 msgr2=0x55e04632fb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:14:31.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.863+0000 7f6026b3f740 1 --2- 172.21.15.42:0/82833006 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e045f0d080 0x55e04632fb00 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f6014004700 tx=0x7f6014005370 comp rx=0 tx=0).stop 2024-08-30T09:14:31.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.863+0000 7f6026b3f740 1 -- 172.21.15.42:0/82833006 shutdown_connections 2024-08-30T09:14:31.815 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-08-30T09:14:31.815 INFO:teuthology.orchestra.run.smithi042.stdout: "return_code": 0, 2024-08-30T09:14:31.815 INFO:teuthology.orchestra.run.smithi042.stdout: "scrub_tag": "d325166d-e7e4-4ac2-9eaf-fa602e480cf6", 2024-08-30T09:14:31.815 INFO:teuthology.orchestra.run.smithi042.stdout: "mode": "asynchronous" 2024-08-30T09:14:31.815 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-08-30T09:14:31.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.863+0000 7f6026b3f740 1 --2- 172.21.15.42:0/82833006 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55e0463a8fa0 0x55e0463ab3c0 crc :-1 s=READY pgs=282 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:31.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.863+0000 7f6026b3f740 1 --2- 172.21.15.42:0/82833006 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e045f79350 0x55e0463a1010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:31.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.863+0000 7f6001ffb640 0 client.27490 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:14:31.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.863+0000 7f6026b3f740 1 --2- 172.21.15.42:0/82833006 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e045f50490 0x55e0463300e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:31.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.863+0000 7f6026b3f740 1 --2- 172.21.15.42:0/82833006 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e045f0d080 0x55e04632fb00 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:31.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.863+0000 7f6026b3f740 1 -- 172.21.15.42:0/82833006 >> 172.21.15.42:0/82833006 conn(0x55e045f6d740 msgr2=0x55e04632c310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:14:31.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.864+0000 7f6026b3f740 1 -- 172.21.15.42:0/82833006 shutdown_connections 2024-08-30T09:14:31.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.864+0000 7f6026b3f740 1 -- 172.21.15.42:0/82833006 wait complete. 2024-08-30T09:14:31.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.864+0000 7f6026b3f740 20 client.27490 trim_cache size 0 max 16384 2024-08-30T09:14:31.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.864+0000 7f60205ca640 1 -- 172.21.15.42:0/503133465 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f60040b4c30 msgr2=0x7f60040b7140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:14:31.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.864+0000 7f60205ca640 1 --2- 172.21.15.42:0/503133465 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f60040b4c30 0x7f60040b7140 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f60140058c0 tx=0x7f6014005850 comp rx=0 tx=0).stop 2024-08-30T09:14:31.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.864+0000 7f60205ca640 1 -- 172.21.15.42:0/503133465 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f601811c9d0 msgr2=0x7f60181ae7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:14:31.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.864+0000 7f60205ca640 1 --2- 172.21.15.42:0/503133465 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f601811c9d0 0x7f60181ae7f0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f600802a950 tx=0x7f6008032d40 comp rx=0 tx=0).stop 2024-08-30T09:14:31.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.864+0000 7f60205ca640 1 -- 172.21.15.42:0/503133465 shutdown_connections 2024-08-30T09:14:31.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.864+0000 7f60205ca640 1 --2- 172.21.15.42:0/503133465 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f60040b4c30 0x7f60040b7140 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:31.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.864+0000 7f60205ca640 1 --2- 172.21.15.42:0/503133465 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6018111ce0 0x7f60181a9550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:31.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.864+0000 7f60205ca640 1 --2- 172.21.15.42:0/503133465 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f601811c9d0 0x7f60181ae7f0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:31.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.864+0000 7f60205ca640 1 --2- 172.21.15.42:0/503133465 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f601810a780 0x7f60181a8fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:31.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.864+0000 7f60205ca640 1 -- 172.21.15.42:0/503133465 >> 172.21.15.42:0/503133465 conn(0x7f6018070c60 msgr2=0x7f601811e0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:14:31.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.864+0000 7f60205ca640 1 -- 172.21.15.42:0/503133465 shutdown_connections 2024-08-30T09:14:31.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:30.865+0000 7f60205ca640 1 -- 172.21.15.42:0/503133465 wait complete. 2024-08-30T09:14:31.817 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:14:31 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T09:14:31.818 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:14:31 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-30T09:14:31.818 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:14:31 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T09:14:31.818 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:14:31 smithi042 ceph-mon[18652]: scrub summary: active paths [/] 2024-08-30T09:14:31.818 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:14:31 smithi042 ceph-mon[18652]: scrub summary: idle+waiting paths [/] 2024-08-30T09:14:31.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:14:31 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T09:14:31.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:14:31 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-30T09:14:31.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:14:31 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T09:14:31.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:14:31 smithi167 ceph-mon[22063]: scrub summary: active paths [/] 2024-08-30T09:14:31.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:14:31 smithi167 ceph-mon[22063]: scrub summary: idle+waiting paths [/] 2024-08-30T09:14:31.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:14:31 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T09:14:31.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:14:31 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-30T09:14:31.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:14:31 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T09:14:31.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:14:31 smithi190 ceph-mon[22001]: scrub summary: active paths [/] 2024-08-30T09:14:31.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:14:31 smithi190 ceph-mon[22001]: scrub summary: idle+waiting paths [/] 2024-08-30T09:14:33.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:14:32 smithi167 ceph-mon[22063]: scrub queued for path: / 2024-08-30T09:14:33.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:14:32 smithi167 ceph-mon[22063]: scrub summary: idle+waiting paths [/] 2024-08-30T09:14:33.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:14:32 smithi167 ceph-mon[22063]: scrub summary: active paths [/] 2024-08-30T09:14:33.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:14:32 smithi167 ceph-mon[22063]: scrub summary: active paths [/] 2024-08-30T09:14:33.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:14:32 smithi167 ceph-mon[22063]: scrub summary: idle+waiting paths [/] 2024-08-30T09:14:33.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:14:32 smithi167 ceph-mon[22063]: scrub summary: active paths [/] 2024-08-30T09:14:33.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:14:32 smithi167 ceph-mon[22063]: pgmap v786: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 3.7 MiB/s wr, 399 op/s 2024-08-30T09:14:33.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:14:32 smithi042 ceph-mon[18652]: scrub queued for path: / 2024-08-30T09:14:33.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:14:32 smithi042 ceph-mon[18652]: scrub summary: idle+waiting paths [/] 2024-08-30T09:14:33.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:14:32 smithi042 ceph-mon[18652]: scrub summary: active paths [/] 2024-08-30T09:14:33.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:14:32 smithi042 ceph-mon[18652]: scrub summary: active paths [/] 2024-08-30T09:14:33.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:14:32 smithi042 ceph-mon[18652]: scrub summary: idle+waiting paths [/] 2024-08-30T09:14:33.134 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:14:32 smithi042 ceph-mon[18652]: scrub summary: active paths [/] 2024-08-30T09:14:33.134 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:14:32 smithi042 ceph-mon[18652]: pgmap v786: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 3.7 MiB/s wr, 399 op/s 2024-08-30T09:14:33.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:14:32 smithi190 ceph-mon[22001]: scrub queued for path: / 2024-08-30T09:14:33.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:14:32 smithi190 ceph-mon[22001]: scrub summary: idle+waiting paths [/] 2024-08-30T09:14:33.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:14:32 smithi190 ceph-mon[22001]: scrub summary: active paths [/] 2024-08-30T09:14:33.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:14:32 smithi190 ceph-mon[22001]: scrub summary: active paths [/] 2024-08-30T09:14:33.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:14:32 smithi190 ceph-mon[22001]: scrub summary: idle+waiting paths [/] 2024-08-30T09:14:33.187 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:14:32 smithi190 ceph-mon[22001]: scrub summary: active paths [/] 2024-08-30T09:14:33.187 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:14:32 smithi190 ceph-mon[22001]: pgmap v786: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 3.7 MiB/s wr, 399 op/s 2024-08-30T09:14:34.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:14:33 smithi167 ceph-mon[22063]: scrub summary: idle+waiting paths [/] 2024-08-30T09:14:34.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:14:33 smithi167 ceph-mon[22063]: scrub summary: idle+waiting paths [/] 2024-08-30T09:14:34.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:14:33 smithi042 ceph-mon[18652]: scrub summary: idle+waiting paths [/] 2024-08-30T09:14:34.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:14:33 smithi042 ceph-mon[18652]: scrub summary: idle+waiting paths [/] 2024-08-30T09:14:34.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:14:33 smithi190 ceph-mon[22001]: scrub summary: idle+waiting paths [/] 2024-08-30T09:14:34.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:14:33 smithi190 ceph-mon[22001]: scrub summary: idle+waiting paths [/] 2024-08-30T09:14:34.588 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-30T09:14:34.884 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:14:34 smithi042 ceph-mon[18652]: pgmap v787: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 4.7 MiB/s wr, 554 op/s 2024-08-30T09:14:35.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:14:34 smithi167 ceph-mon[22063]: pgmap v787: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 4.7 MiB/s wr, 554 op/s 2024-08-30T09:14:35.126 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T09:14:35.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:14:34 smithi190 ceph-mon[22001]: pgmap v787: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 4.7 MiB/s wr, 554 op/s 2024-08-30T09:14:35.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.936+0000 7fb19ac47640 1 -- 172.21.15.42:0/3619953508 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb18c09b2b0 msgr2=0x7fb18c0a1ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:14:35.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.936+0000 7fb19ac47640 1 --2- 172.21.15.42:0/3619953508 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb18c09b2b0 0x7fb18c0a1ae0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fb194066220 tx=0x7fb194097e60 comp rx=0 tx=0).stop 2024-08-30T09:14:35.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.937+0000 7fb19ac47640 1 -- 172.21.15.42:0/3619953508 shutdown_connections 2024-08-30T09:14:35.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.937+0000 7fb19ac47640 1 --2- 172.21.15.42:0/3619953508 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb18c09b2b0 0x7fb18c0a1ae0 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:35.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.937+0000 7fb19ac47640 1 --2- 172.21.15.42:0/3619953508 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb18c09a960 0x7fb18c09ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:35.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.937+0000 7fb19ac47640 1 --2- 172.21.15.42:0/3619953508 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb18c095020 0x7fb18c095430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:35.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.937+0000 7fb19ac47640 1 -- 172.21.15.42:0/3619953508 >> 172.21.15.42:0/3619953508 conn(0x7fb18c00cd70 msgr2=0x7fb18c00d180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:14:35.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.937+0000 7fb19ac47640 1 -- 172.21.15.42:0/3619953508 shutdown_connections 2024-08-30T09:14:35.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.937+0000 7fb19ac47640 1 -- 172.21.15.42:0/3619953508 wait complete. 2024-08-30T09:14:35.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.938+0000 7fb19ac47640 1 Processor -- start 2024-08-30T09:14:35.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.938+0000 7fb19ac47640 1 -- start start 2024-08-30T09:14:35.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.938+0000 7fb19ac47640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb18c095020 0x7fb18c0a07b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:14:35.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.938+0000 7fb19ac47640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb18c09a960 0x7fb18c0a0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:14:35.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.939+0000 7fb19ac47640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb18c09b2b0 0x7fb18c0a1230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:14:35.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.939+0000 7fb199c45640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb18c095020 0x7fb18c0a07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:14:35.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.939+0000 7fb19ac47640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fb18c0a3780 con 0x7fb18c09b2b0 2024-08-30T09:14:35.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.939+0000 7fb19ac47640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb18c0a3600 con 0x7fb18c095020 2024-08-30T09:14:35.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.939+0000 7fb19ac47640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fb18c0a3900 con 0x7fb18c09a960 2024-08-30T09:14:35.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.939+0000 7fb199c45640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb18c095020 0x7fb18c0a07b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:45500/0 (socket says 172.21.15.42:45500) 2024-08-30T09:14:35.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.939+0000 7fb199c45640 1 -- 172.21.15.42:0/3122749465 learned_addr learned my addr 172.21.15.42:0/3122749465 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:14:35.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.939+0000 7fb199444640 1 --2- 172.21.15.42:0/3122749465 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb18c09a960 0x7fb18c0a0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:14:35.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.939+0000 7fb199c45640 1 -- 172.21.15.42:0/3122749465 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb18c09a960 msgr2=0x7fb18c0a0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:14:35.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.939+0000 7fb199c45640 1 --2- 172.21.15.42:0/3122749465 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb18c09a960 0x7fb18c0a0cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:35.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.939+0000 7fb199c45640 1 -- 172.21.15.42:0/3122749465 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb18c09b2b0 msgr2=0x7fb18c0a1230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:14:35.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.939+0000 7fb199c45640 1 --2- 172.21.15.42:0/3122749465 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb18c09b2b0 0x7fb18c0a1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:35.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.939+0000 7fb199c45640 1 -- 172.21.15.42:0/3122749465 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb18c147920 con 0x7fb18c095020 2024-08-30T09:14:35.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.939+0000 7fb199c45640 1 --2- 172.21.15.42:0/3122749465 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb18c095020 0x7fb18c0a07b0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fb1900097d0 tx=0x7fb190009c90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:14:35.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.939+0000 7fb18affd640 1 -- 172.21.15.42:0/3122749465 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb19000eab0 con 0x7fb18c095020 2024-08-30T09:14:35.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.940+0000 7fb19ac47640 1 -- 172.21.15.42:0/3122749465 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb18c147c10 con 0x7fb18c095020 2024-08-30T09:14:35.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.940+0000 7fb18affd640 1 -- 172.21.15.42:0/3122749465 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb190019070 con 0x7fb18c095020 2024-08-30T09:14:35.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.940+0000 7fb18affd640 1 -- 172.21.15.42:0/3122749465 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb190013380 con 0x7fb18c095020 2024-08-30T09:14:35.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.940+0000 7fb19ac47640 1 -- 172.21.15.42:0/3122749465 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb18c1481a0 con 0x7fb18c095020 2024-08-30T09:14:35.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.943+0000 7fb1a241e740 1 Processor -- start 2024-08-30T09:14:35.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.944+0000 7fb1a241e740 1 -- start start 2024-08-30T09:14:35.947 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.944+0000 7fb18affd640 1 -- 172.21.15.42:0/3122749465 <== mon.2 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7fb190013520 con 0x7fb18c095020 2024-08-30T09:14:35.947 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.944+0000 7fb1a241e740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d97ad5b070 0x55d97b100460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:14:35.947 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.944+0000 7fb1a241e740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d97ad321b0 0x55d97b1009a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:14:35.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.944+0000 7fb1a241e740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d97ad3a390 0x55d97b100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:14:35.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.944+0000 7fb1a241e740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55d97b077630 con 0x55d97ad5b070 2024-08-30T09:14:35.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.944+0000 7fb1a241e740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55d97b066bc0 con 0x55d97ad3a390 2024-08-30T09:14:35.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.944+0000 7fb1a241e740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55d97b06ab70 con 0x55d97ad321b0 2024-08-30T09:14:35.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.944+0000 7fb199c45640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d97ad321b0 0x55d97b1009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:14:35.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.944+0000 7fb199444640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d97ad3a390 0x55d97b100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:14:35.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.944+0000 7fb199c45640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d97ad321b0 0x55d97b1009a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:54238/0 (socket says 172.21.15.42:54238) 2024-08-30T09:14:35.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.944+0000 7fb199c45640 1 -- 172.21.15.42:0/1798182233 learned_addr learned my addr 172.21.15.42:0/1798182233 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:14:35.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.944+0000 7fb19a446640 1 --2- 172.21.15.42:0/1798182233 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d97ad5b070 0x55d97b100460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:14:35.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.945+0000 7fb199c45640 1 -- 172.21.15.42:0/1798182233 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d97ad3a390 msgr2=0x55d97b100ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:14:35.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.945+0000 7fb199c45640 1 --2- 172.21.15.42:0/1798182233 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d97ad3a390 0x55d97b100ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:35.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.945+0000 7fb199c45640 1 -- 172.21.15.42:0/1798182233 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d97ad5b070 msgr2=0x55d97b100460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:14:35.949 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.945+0000 7fb199c45640 1 --2- 172.21.15.42:0/1798182233 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d97ad5b070 0x55d97b100460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:35.949 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.945+0000 7fb199c45640 1 -- 172.21.15.42:0/1798182233 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d97adc6650 con 0x55d97ad321b0 2024-08-30T09:14:35.949 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.945+0000 7fb199c45640 1 --2- 172.21.15.42:0/1798182233 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d97ad321b0 0x55d97b1009a0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fb1900ed240 tx=0x7fb1900ed700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cdeb4e8fef3bc956 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:14:35.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.945+0000 7fb17ffff640 1 -- 172.21.15.42:0/1798182233 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1900f8070 con 0x55d97ad321b0 2024-08-30T09:14:35.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.945+0000 7fb17ffff640 1 -- 172.21.15.42:0/1798182233 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb1900e5af0 con 0x55d97ad321b0 2024-08-30T09:14:35.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.945+0000 7fb17ffff640 1 -- 172.21.15.42:0/1798182233 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1900f39a0 con 0x55d97ad321b0 2024-08-30T09:14:35.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.946+0000 7fb18affd640 1 --2- 172.21.15.42:0/3122749465 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fb1800b4bb0 0x7fb1800b70c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:14:35.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.947+0000 7fb18affd640 1 -- 172.21.15.42:0/3122749465 <== mon.2 v2:172.21.15.167:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7fb1900e5250 con 0x7fb18c095020 2024-08-30T09:14:35.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.947+0000 7fb1a241e740 1 -- 172.21.15.42:0/1798182233 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d97ad321b0 msgr2=0x55d97b1009a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:14:35.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.947+0000 7fb1a241e740 1 --2- 172.21.15.42:0/1798182233 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d97ad321b0 0x55d97b1009a0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fb1900ed240 tx=0x7fb1900ed700 comp rx=0 tx=0).stop 2024-08-30T09:14:35.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.947+0000 7fb19a446640 1 --2- 172.21.15.42:0/3122749465 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fb1800b4bb0 0x7fb1800b70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:14:35.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.947+0000 7fb1a241e740 1 -- 172.21.15.42:0/1798182233 shutdown_connections 2024-08-30T09:14:35.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.947+0000 7fb1a241e740 1 --2- 172.21.15.42:0/1798182233 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d97ad321b0 0x55d97b1009a0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:35.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.947+0000 7fb1a241e740 1 --2- 172.21.15.42:0/1798182233 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d97ad3a390 0x55d97b100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:35.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.947+0000 7fb1a241e740 1 --2- 172.21.15.42:0/1798182233 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d97ad5b070 0x55d97b100460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:35.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.947+0000 7fb1a241e740 1 -- 172.21.15.42:0/1798182233 >> 172.21.15.42:0/1798182233 conn(0x55d97ad4f460 msgr2=0x55d97b0fe350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:14:35.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.948+0000 7fb1a241e740 1 -- 172.21.15.42:0/1798182233 shutdown_connections 2024-08-30T09:14:35.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.948+0000 7fb1a241e740 1 -- 172.21.15.42:0/1798182233 wait complete. 2024-08-30T09:14:35.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.948+0000 7fb19a446640 1 --2- 172.21.15.42:0/3122749465 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fb1800b4bb0 0x7fb1800b70c0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fb194066a10 tx=0x7fb1940a2040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:14:35.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.949+0000 7fb1a241e740 1 -- start start 2024-08-30T09:14:35.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.949+0000 7fb1a241e740 1 Processor -- start 2024-08-30T09:14:35.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.949+0000 7fb1a241e740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d97ad321b0 0x55d97b10bc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:14:35.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.949+0000 7fb1a241e740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d97ad3a390 0x55d97b10c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:14:35.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.949+0000 7fb1a241e740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d97ad5b070 0x55d97b117fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:14:35.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.949+0000 7fb1a241e740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55d97aec9700 con 0x55d97ad5b070 2024-08-30T09:14:35.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.949+0000 7fb1a241e740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55d97afed700 con 0x55d97ad321b0 2024-08-30T09:14:35.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.949+0000 7fb1a241e740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55d97b10ec40 con 0x55d97ad3a390 2024-08-30T09:14:35.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.949+0000 7fb19a446640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d97ad5b070 0x55d97b117fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:14:35.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.950+0000 7fb19a446640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d97ad5b070 0x55d97b117fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42990/0 (socket says 172.21.15.42:42990) 2024-08-30T09:14:35.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.950+0000 7fb199c45640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d97ad321b0 0x55d97b10bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:14:35.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.950+0000 7fb199444640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d97ad3a390 0x55d97b10c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:14:35.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.950+0000 7fb19a446640 1 -- 172.21.15.42:0/2046283060 learned_addr learned my addr 172.21.15.42:0/2046283060 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:14:35.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.950+0000 7fb1a241e740 10 client.0 _ll_register_callbacks cb 0x55d97b01cdc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:14:35.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.950+0000 7fb199c45640 1 -- 172.21.15.42:0/2046283060 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d97ad3a390 msgr2=0x55d97b10c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:14:35.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.950+0000 7fb199c45640 1 --2- 172.21.15.42:0/2046283060 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d97ad3a390 0x55d97b10c140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:35.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.950+0000 7fb199c45640 1 -- 172.21.15.42:0/2046283060 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d97ad5b070 msgr2=0x55d97b117fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:14:35.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.950+0000 7fb199c45640 1 --2- 172.21.15.42:0/2046283060 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d97ad5b070 0x55d97b117fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:35.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.950+0000 7fb199c45640 1 -- 172.21.15.42:0/2046283060 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d97b0268b0 con 0x55d97ad321b0 2024-08-30T09:14:35.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.950+0000 7fb199c45640 1 --2- 172.21.15.42:0/2046283060 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d97ad321b0 0x55d97b10bc00 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fb1900a8b00 tx=0x7fb1900f3400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5154c6946420be73 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:14:35.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.950+0000 7fb17d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-30T09:14:35.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.950+0000 7fb17d7fa640 1 -- 172.21.15.42:0/2046283060 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1900a7750 con 0x55d97ad321b0 2024-08-30T09:14:35.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.950+0000 7fb17d7fa640 1 -- 172.21.15.42:0/2046283060 <== mon.2 v2:172.21.15.167:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7fb1900e72e0 con 0x55d97ad321b0 2024-08-30T09:14:35.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.951+0000 7fb1a241e740 1 -- 172.21.15.42:0/2046283060 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb1900e5af0 con 0x55d97ad321b0 2024-08-30T09:14:35.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.951+0000 7fb17d7fa640 1 -- 172.21.15.42:0/2046283060 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7fb1900ab000 con 0x55d97ad321b0 2024-08-30T09:14:35.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.951+0000 7fb1a241e740 10 client.27575 fetch_fsmap learned FSMap version 47 2024-08-30T09:14:35.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.951+0000 7fb1a241e740 1 -- 172.21.15.42:0/2046283060 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55d97b11aa20 con 0x55d97ad321b0 2024-08-30T09:14:35.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.951+0000 7fb17d7fa640 1 -- 172.21.15.42:0/2046283060 <== mon.2 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb1900e5af0 con 0x55d97ad321b0 2024-08-30T09:14:35.956 INFO:teuthology.orchestra.run.smithi042.stderr:progress: 830.0 s, 453.0 tps, lat 224.230 ms stddev 267.100 2024-08-30T09:14:35.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.951+0000 7fb17d7fa640 1 -- 172.21.15.42:0/2046283060 <== mon.2 v2:172.21.15.167:3300/0 5 ==== fsmap(e 47) ==== 8370+0+0 (secure 0 0 0) 0x7fb1900a7b20 con 0x55d97ad321b0 2024-08-30T09:14:35.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.951+0000 7fb1a241e740 10 client.27575 fetch_fsmap finished waiting for FSMap version 47 2024-08-30T09:14:35.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.951+0000 7fb1a241e740 10 client.27575 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-30T09:14:35.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.951+0000 7fb1a241e740 20 client.27575 populate_metadata read hostname 'smithi042' 2024-08-30T09:14:35.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.951+0000 7fb1a241e740 1 --2- 172.21.15.42:0/2046283060 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55d97ad199f0 0x55d97b11dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:14:35.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.951+0000 7fb1a241e740 4 client.27575 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-30T09:14:35.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.951+0000 7fb1a241e740 1 -- 172.21.15.42:0/2046283060 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d97b079580 con 0x55d97ad199f0 2024-08-30T09:14:35.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.952+0000 7fb199444640 1 --2- 172.21.15.42:0/2046283060 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55d97ad199f0 0x55d97b11dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:14:35.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.952+0000 7fb17d7fa640 10 client.27575 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:14:35.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.952+0000 7fb199444640 1 --2- 172.21.15.42:0/2046283060 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55d97ad199f0 0x55d97b11dd80 crc :-1 s=READY pgs=283 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b7c6b4a0cde1633d server_cookie=6d269db2cfdde6f3 in_seq=0 out_seq=0 2024-08-30T09:14:35.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.953+0000 7fb17d7fa640 1 -- 172.21.15.42:0/2046283060 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d97b079580 con 0x55d97ad199f0 2024-08-30T09:14:35.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.953+0000 7fb17d7fa640 10 client.27575 handle_command_reply: tid=0 2024-08-30T09:14:35.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.953+0000 7fb1a241e740 1 client.27575 shutdown 2024-08-30T09:14:35.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.953+0000 7fb1a241e740 1 -- 172.21.15.42:0/2046283060 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d97ad321b0 msgr2=0x55d97b10bc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:14:35.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.953+0000 7fb1a241e740 1 --2- 172.21.15.42:0/2046283060 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d97ad321b0 0x55d97b10bc00 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fb1900a8b00 tx=0x7fb1900f3400 comp rx=0 tx=0).stop 2024-08-30T09:14:35.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.953+0000 7fb1a241e740 1 -- 172.21.15.42:0/2046283060 shutdown_connections 2024-08-30T09:14:35.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.953+0000 7fb1a241e740 1 --2- 172.21.15.42:0/2046283060 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55d97ad199f0 0x55d97b11dd80 crc :-1 s=READY pgs=283 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:35.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.953+0000 7fb1a241e740 1 --2- 172.21.15.42:0/2046283060 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d97ad3a390 0x55d97b10c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:35.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.953+0000 7fb1a241e740 1 --2- 172.21.15.42:0/2046283060 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d97ad5b070 0x55d97b117fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:35.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.953+0000 7fb1a241e740 1 --2- 172.21.15.42:0/2046283060 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d97ad321b0 0x55d97b10bc00 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:35.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.953+0000 7fb1a241e740 1 -- 172.21.15.42:0/2046283060 >> 172.21.15.42:0/2046283060 conn(0x55d97ad4f460 msgr2=0x55d97b1035f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:14:35.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.953+0000 7fb17d7fa640 0 client.27575 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:14:35.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.953+0000 7fb1a241e740 1 -- 172.21.15.42:0/2046283060 shutdown_connections 2024-08-30T09:14:35.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.953+0000 7fb1a241e740 1 -- 172.21.15.42:0/2046283060 wait complete. 2024-08-30T09:14:35.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.954+0000 7fb1a241e740 20 client.27575 trim_cache size 0 max 16384 2024-08-30T09:14:35.981 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.980+0000 7fb1a241e740 1 Processor -- start 2024-08-30T09:14:35.981 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.980+0000 7fb1a241e740 1 -- start start 2024-08-30T09:14:35.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.980+0000 7fb1a241e740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d97ad321b0 0x55d97b0fe3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:14:35.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.980+0000 7fb1a241e740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d97ad3a390 0x55d97b10d830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:14:35.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.980+0000 7fb1a241e740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d97ad5b070 0x55d97b117f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:14:35.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.980+0000 7fb1a241e740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fb1780124e0 con 0x55d97ad5b070 2024-08-30T09:14:35.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.980+0000 7fb1a241e740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb17800fd50 con 0x55d97ad321b0 2024-08-30T09:14:35.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.980+0000 7fb1a241e740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fb17800dba0 con 0x55d97ad3a390 2024-08-30T09:14:35.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.980+0000 7fb199c45640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d97ad321b0 0x55d97b0fe3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:14:35.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.980+0000 7fb19a446640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d97ad5b070 0x55d97b117f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:14:35.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.980+0000 7fb199c45640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d97ad321b0 0x55d97b0fe3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:45530/0 (socket says 172.21.15.42:45530) 2024-08-30T09:14:35.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.980+0000 7fb199c45640 1 -- 172.21.15.42:0/1163678370 learned_addr learned my addr 172.21.15.42:0/1163678370 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:14:35.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.980+0000 7fb199444640 1 --2- 172.21.15.42:0/1163678370 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d97ad3a390 0x55d97b10d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:14:35.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.981+0000 7fb199c45640 1 -- 172.21.15.42:0/1163678370 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d97ad3a390 msgr2=0x55d97b10d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:14:35.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.981+0000 7fb199c45640 1 --2- 172.21.15.42:0/1163678370 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d97ad3a390 0x55d97b10d830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:35.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.981+0000 7fb199c45640 1 -- 172.21.15.42:0/1163678370 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d97ad5b070 msgr2=0x55d97b117f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:14:35.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.981+0000 7fb199c45640 1 --2- 172.21.15.42:0/1163678370 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d97ad5b070 0x55d97b117f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:35.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.981+0000 7fb199c45640 1 -- 172.21.15.42:0/1163678370 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d97b10e340 con 0x55d97ad321b0 2024-08-30T09:14:35.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.981+0000 7fb19a446640 1 --2- 172.21.15.42:0/1163678370 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d97ad5b070 0x55d97b117f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-30T09:14:35.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.981+0000 7fb199c45640 1 --2- 172.21.15.42:0/1163678370 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d97ad321b0 0x55d97b0fe3d0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fb1900aa200 tx=0x7fb1900ed700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=83891d3aaf99b4e0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:14:35.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.981+0000 7fb17f7fe640 1 -- 172.21.15.42:0/1163678370 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1900ab1e0 con 0x55d97ad321b0 2024-08-30T09:14:35.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.982+0000 7fb17f7fe640 1 -- 172.21.15.42:0/1163678370 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb1900edc00 con 0x55d97ad321b0 2024-08-30T09:14:35.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.982+0000 7fb17f7fe640 1 -- 172.21.15.42:0/1163678370 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1900ab660 con 0x55d97ad321b0 2024-08-30T09:14:35.987 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.982+0000 7fb1a241e740 1 -- 172.21.15.42:0/1163678370 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d97ad321b0 msgr2=0x55d97b0fe3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:14:35.988 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.982+0000 7fb1a241e740 1 --2- 172.21.15.42:0/1163678370 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d97ad321b0 0x55d97b0fe3d0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fb1900aa200 tx=0x7fb1900ed700 comp rx=0 tx=0).stop 2024-08-30T09:14:35.988 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.983+0000 7fb1a241e740 1 -- 172.21.15.42:0/1163678370 shutdown_connections 2024-08-30T09:14:35.988 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.983+0000 7fb1a241e740 1 --2- 172.21.15.42:0/1163678370 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d97ad3a390 0x55d97b10d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:35.988 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.983+0000 7fb1a241e740 1 --2- 172.21.15.42:0/1163678370 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d97ad5b070 0x55d97b117f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:35.988 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.983+0000 7fb1a241e740 1 --2- 172.21.15.42:0/1163678370 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d97ad321b0 0x55d97b0fe3d0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:35.988 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.983+0000 7fb1a241e740 1 -- 172.21.15.42:0/1163678370 >> 172.21.15.42:0/1163678370 conn(0x55d97ad4f460 msgr2=0x55d97b10d120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:14:35.988 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.983+0000 7fb1a241e740 1 -- 172.21.15.42:0/1163678370 shutdown_connections 2024-08-30T09:14:35.988 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.983+0000 7fb1a241e740 1 -- 172.21.15.42:0/1163678370 wait complete. 2024-08-30T09:14:35.988 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.983+0000 7fb1a241e740 1 -- start start 2024-08-30T09:14:35.988 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.984+0000 7fb1a241e740 1 Processor -- start 2024-08-30T09:14:35.988 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.984+0000 7fb1a241e740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d97ad321b0 0x55d97b11df20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:14:35.988 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.984+0000 7fb1a241e740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d97ad3a390 0x55d97b11e460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:14:35.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.984+0000 7fb1a241e740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d97ad5b070 0x55d97b11e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:14:35.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.984+0000 7fb199444640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d97ad3a390 0x55d97b11e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:14:35.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.984+0000 7fb199444640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d97ad3a390 0x55d97b11e460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:54262/0 (socket says 172.21.15.42:54262) 2024-08-30T09:14:35.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.984+0000 7fb199c45640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d97ad321b0 0x55d97b11df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:14:35.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.984+0000 7fb199c45640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d97ad321b0 0x55d97b11df20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:45538/0 (socket says 172.21.15.42:45538) 2024-08-30T09:14:35.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.984+0000 7fb19a446640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d97ad5b070 0x55d97b11e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:14:35.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.984+0000 7fb19a446640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d97ad5b070 0x55d97b11e9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43014/0 (socket says 172.21.15.42:43014) 2024-08-30T09:14:35.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.984+0000 7fb1a241e740 1 -- 172.21.15.42:0/1485683819 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fb17800e210 con 0x55d97ad5b070 2024-08-30T09:14:35.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.984+0000 7fb1a241e740 1 -- 172.21.15.42:0/1485683819 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb17800f680 con 0x55d97ad321b0 2024-08-30T09:14:35.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.984+0000 7fb1a241e740 1 -- 172.21.15.42:0/1485683819 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fb17800e8e0 con 0x55d97ad3a390 2024-08-30T09:14:35.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.984+0000 7fb199444640 1 -- 172.21.15.42:0/1485683819 learned_addr learned my addr 172.21.15.42:0/1485683819 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:14:35.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.985+0000 7fb1a241e740 10 client.0 _ll_register_callbacks cb 0x7fb1900f6070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:14:35.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.986+0000 7fb199444640 1 -- 172.21.15.42:0/1485683819 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d97ad321b0 msgr2=0x55d97b11df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:14:35.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.986+0000 7fb199444640 1 --2- 172.21.15.42:0/1485683819 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d97ad321b0 0x55d97b11df20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:35.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.986+0000 7fb199444640 1 -- 172.21.15.42:0/1485683819 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d97ad5b070 msgr2=0x55d97b11e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:14:35.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.986+0000 7fb199444640 1 --2- 172.21.15.42:0/1485683819 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d97ad5b070 0x55d97b11e9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:35.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.986+0000 7fb199444640 1 -- 172.21.15.42:0/1485683819 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d97b1874f0 con 0x55d97ad3a390 2024-08-30T09:14:35.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.987+0000 7fb17d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-08-30T09:14:35.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.987+0000 7fb199444640 1 --2- 172.21.15.42:0/1485683819 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d97ad3a390 0x55d97b11e460 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x55d97b06ab70 tx=0x7fb1840128e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=77a13aa1959d5a23 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:14:35.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.987+0000 7fb17d7fa640 1 -- 172.21.15.42:0/1485683819 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb184005020 con 0x55d97ad3a390 2024-08-30T09:14:35.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.987+0000 7fb17d7fa640 1 -- 172.21.15.42:0/1485683819 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7fb1840051c0 con 0x55d97ad3a390 2024-08-30T09:14:35.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.987+0000 7fb1a241e740 1 -- 172.21.15.42:0/1485683819 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb1900edc00 con 0x55d97ad3a390 2024-08-30T09:14:35.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.988+0000 7fb17d7fa640 1 -- 172.21.15.42:0/1485683819 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7fb1840053c0 con 0x55d97ad3a390 2024-08-30T09:14:35.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.988+0000 7fb1a241e740 10 client.27508 fetch_fsmap learned FSMap version 47 2024-08-30T09:14:35.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.988+0000 7fb1a241e740 1 -- 172.21.15.42:0/1485683819 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55d97b188760 con 0x55d97ad3a390 2024-08-30T09:14:35.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.988+0000 7fb17d7fa640 1 -- 172.21.15.42:0/1485683819 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb1900edc00 con 0x55d97ad3a390 2024-08-30T09:14:35.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.988+0000 7fb17d7fa640 1 -- 172.21.15.42:0/1485683819 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 47) ==== 8370+0+0 (secure 0 0 0) 0x7fb18401f5c0 con 0x55d97ad3a390 2024-08-30T09:14:35.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.988+0000 7fb1a241e740 10 client.27508 fetch_fsmap finished waiting for FSMap version 47 2024-08-30T09:14:35.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.988+0000 7fb1a241e740 10 client.27508 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-30T09:14:35.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.988+0000 7fb1a241e740 20 client.27508 populate_metadata read hostname 'smithi042' 2024-08-30T09:14:35.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.988+0000 7fb1a241e740 1 --2- 172.21.15.42:0/1485683819 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55d97aceeda0 0x55d97b18b030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:14:35.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.988+0000 7fb1a241e740 4 client.27508 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-30T09:14:35.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.988+0000 7fb1a241e740 1 -- 172.21.15.42:0/1485683819 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d97b075020 con 0x55d97aceeda0 2024-08-30T09:14:35.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.988+0000 7fb199c45640 1 --2- 172.21.15.42:0/1485683819 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55d97aceeda0 0x55d97b18b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:14:35.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.989+0000 7fb17d7fa640 10 client.27508 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:14:35.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.989+0000 7fb199c45640 1 --2- 172.21.15.42:0/1485683819 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55d97aceeda0 0x55d97b18b030 crc :-1 s=READY pgs=284 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2995072347bba711 server_cookie=283d3db6f8492879 in_seq=0 out_seq=0 2024-08-30T09:14:35.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.989+0000 7fb17d7fa640 1 -- 172.21.15.42:0/1485683819 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+5114 (crc 0 0 0) 0x55d97b075020 con 0x55d97aceeda0 2024-08-30T09:14:35.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.989+0000 7fb17d7fa640 10 client.27508 handle_command_reply: tid=0 2024-08-30T09:14:35.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.989+0000 7fb1a241e740 1 client.27508 shutdown 2024-08-30T09:14:35.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.989+0000 7fb1a241e740 1 -- 172.21.15.42:0/1485683819 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d97ad3a390 msgr2=0x55d97b11e460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:14:35.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.989+0000 7fb1a241e740 1 --2- 172.21.15.42:0/1485683819 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d97ad3a390 0x55d97b11e460 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x55d97b06ab70 tx=0x7fb1840128e0 comp rx=0 tx=0).stop 2024-08-30T09:14:35.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.989+0000 7fb1a241e740 1 -- 172.21.15.42:0/1485683819 shutdown_connections 2024-08-30T09:14:35.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.989+0000 7fb1a241e740 1 --2- 172.21.15.42:0/1485683819 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55d97aceeda0 0x55d97b18b030 crc :-1 s=READY pgs=284 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:35.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.989+0000 7fb1a241e740 1 --2- 172.21.15.42:0/1485683819 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d97ad3a390 0x55d97b11e460 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:35.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.989+0000 7fb1a241e740 1 --2- 172.21.15.42:0/1485683819 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d97ad5b070 0x55d97b11e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:35.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.989+0000 7fb1a241e740 1 --2- 172.21.15.42:0/1485683819 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d97ad321b0 0x55d97b11df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:35.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.990+0000 7fb1a241e740 1 -- 172.21.15.42:0/1485683819 >> 172.21.15.42:0/1485683819 conn(0x55d97ad4f460 msgr2=0x55d97b10a080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:14:35.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.990+0000 7fb17d7fa640 0 client.27508 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:14:35.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.990+0000 7fb1a241e740 1 -- 172.21.15.42:0/1485683819 shutdown_connections 2024-08-30T09:14:35.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.990+0000 7fb1a241e740 1 -- 172.21.15.42:0/1485683819 wait complete. 2024-08-30T09:14:35.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.990+0000 7fb1a241e740 20 client.27508 trim_cache size 0 max 16384 2024-08-30T09:14:35.996 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-08-30T09:14:35.996 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "IDLE (waiting for more scrubs)", 2024-08-30T09:14:35.996 INFO:teuthology.orchestra.run.smithi042.stdout: "scrubs": { 2024-08-30T09:14:35.996 INFO:teuthology.orchestra.run.smithi042.stdout: "d325166d-e7e4-4ac2-9eaf-fa602e480cf6": { 2024-08-30T09:14:35.996 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:14:35.996 INFO:teuthology.orchestra.run.smithi042.stdout: "tag": "d325166d-e7e4-4ac2-9eaf-fa602e480cf6", 2024-08-30T09:14:35.996 INFO:teuthology.orchestra.run.smithi042.stdout: "options": "recursive,force" 2024-08-30T09:14:35.996 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:14:35.996 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:14:35.996 INFO:teuthology.orchestra.run.smithi042.stdout: "scrub_stats": { 2024-08-30T09:14:35.997 INFO:teuthology.orchestra.run.smithi042.stdout: "d325166d-e7e4-4ac2-9eaf-fa602e480cf6": { 2024-08-30T09:14:35.997 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:14:30.861941+0000", 2024-08-30T09:14:35.997 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:14:35.997 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:14:35.997 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:14:35.997 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:14:35.997 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:14:35.997 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:14:35.997 INFO:teuthology.orchestra.run.smithi042.stdout: "03521407-a747-49b9-a415-6233551c26fb": { 2024-08-30T09:14:35.997 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:13:39.387817+0000", 2024-08-30T09:14:35.997 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:14:35.997 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:14:35.997 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:14:35.997 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:14:35.998 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:14:35.998 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:14:35.998 INFO:teuthology.orchestra.run.smithi042.stdout: "0cbf734e-f988-4b21-acac-57b55e3bde62": { 2024-08-30T09:14:35.998 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:12:45.306738+0000", 2024-08-30T09:14:35.998 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:14:35.998 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:14:35.998 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:14:35.998 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:14:35.998 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:14:35.998 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:14:35.998 INFO:teuthology.orchestra.run.smithi042.stdout: "5e2ed104-0625-478a-a78b-584e7f4c2817": { 2024-08-30T09:14:35.998 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:00:33.928400+0000", 2024-08-30T09:14:35.998 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:14:35.998 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:14:35.998 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:14:35.999 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:14:35.999 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:14:35.999 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:14:35.999 INFO:teuthology.orchestra.run.smithi042.stdout: "c827ad7a-44ee-4e1d-9bb3-51fb7ab00c73": { 2024-08-30T09:14:35.999 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:02:21.686517+0000", 2024-08-30T09:14:35.999 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:14:35.999 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:14:35.999 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:14:35.999 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:14:35.999 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:14:35.999 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:14:35.999 INFO:teuthology.orchestra.run.smithi042.stdout: "85134e4b-ab4a-4885-b07e-a3ad01eaf13d": { 2024-08-30T09:14:35.999 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:01:27.738462+0000", 2024-08-30T09:14:35.999 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:14:36.000 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:14:36.000 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:14:36.000 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:14:36.000 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:14:36.000 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:14:36.000 INFO:teuthology.orchestra.run.smithi042.stdout: "749f4867-0b70-49e5-af2d-8362a240ebc3": { 2024-08-30T09:14:36.000 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:03:16.386554+0000", 2024-08-30T09:14:36.000 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:14:36.000 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:14:36.000 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:14:36.000 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:14:36.000 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:14:36.000 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:14:36.000 INFO:teuthology.orchestra.run.smithi042.stdout: "88cd7478-2b20-47de-ba51-256479749232": { 2024-08-30T09:14:36.001 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:04:08.922625+0000", 2024-08-30T09:14:36.001 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:14:36.001 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:14:36.001 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:14:36.001 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:14:36.001 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:14:36.001 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:14:36.001 INFO:teuthology.orchestra.run.smithi042.stdout: "347d05ed-4bd3-4590-b099-314f719aab39": { 2024-08-30T09:14:36.001 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:05:00.439721+0000", 2024-08-30T09:14:36.001 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:14:36.001 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:14:36.001 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:14:36.001 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:14:36.001 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:14:36.002 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:14:36.002 INFO:teuthology.orchestra.run.smithi042.stdout: "1af4252a-4a99-47e9-bbf0-becf5e573dd5": { 2024-08-30T09:14:36.002 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:05:48.565881+0000", 2024-08-30T09:14:36.002 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:14:36.002 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:14:36.002 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:14:36.002 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:14:36.002 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:14:36.002 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:14:36.002 INFO:teuthology.orchestra.run.smithi042.stdout: "665ec740-7254-49cb-84ef-deff71c461b0": { 2024-08-30T09:14:36.002 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:06:37.144035+0000", 2024-08-30T09:14:36.002 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:14:36.002 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:14:36.003 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:14:36.003 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:14:36.003 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:14:36.003 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:14:36.003 INFO:teuthology.orchestra.run.smithi042.stdout: "598d8a75-f1cc-4388-80bc-a408bdb4eaac": { 2024-08-30T09:14:36.003 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:07:29.721122+0000", 2024-08-30T09:14:36.003 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:14:36.003 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:14:36.003 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:14:36.003 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:14:36.003 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:14:36.003 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:14:36.003 INFO:teuthology.orchestra.run.smithi042.stdout: "b62c4ffc-9387-4677-b9ba-1e94ba308f17": { 2024-08-30T09:14:36.004 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:11:03.363478+0000", 2024-08-30T09:14:36.004 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:14:36.004 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:14:36.004 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:14:36.004 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:14:36.004 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:14:36.004 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:14:36.004 INFO:teuthology.orchestra.run.smithi042.stdout: "a7210f9a-046b-4188-953c-819b9f1863a9": { 2024-08-30T09:14:36.004 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:10:07.608430+0000", 2024-08-30T09:14:36.004 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:14:36.004 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:14:36.004 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:14:36.004 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:14:36.004 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:14:36.005 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:14:36.005 INFO:teuthology.orchestra.run.smithi042.stdout: "99a29ec1-9455-4ae4-854a-398b32896675": { 2024-08-30T09:14:36.005 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:08:24.923176+0000", 2024-08-30T09:14:36.005 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:14:36.005 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:14:36.005 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:14:36.005 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:14:36.005 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:14:36.005 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:14:36.005 INFO:teuthology.orchestra.run.smithi042.stdout: "ff464e7f-123b-4bc6-bc7e-aced018168de": { 2024-08-30T09:14:36.005 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:11:54.879598+0000", 2024-08-30T09:14:36.005 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:14:36.005 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:14:36.005 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:14:36.005 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:14:36.006 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:14:36.006 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:14:36.006 INFO:teuthology.orchestra.run.smithi042.stdout: "37f2f28f-496f-41a6-b4cf-8846fb3f43ab": { 2024-08-30T09:14:36.006 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:09:17.428286+0000", 2024-08-30T09:14:36.006 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:14:36.006 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:14:36.006 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:14:36.006 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:14:36.006 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:14:36.006 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:14:36.006 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:14:36.006 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-08-30T09:14:36.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.990+0000 7fb19ac47640 1 -- 172.21.15.42:0/3122749465 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fb1800b4bb0 msgr2=0x7fb1800b70c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:14:36.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.990+0000 7fb19ac47640 1 --2- 172.21.15.42:0/3122749465 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fb1800b4bb0 0x7fb1800b70c0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fb194066a10 tx=0x7fb1940a2040 comp rx=0 tx=0).stop 2024-08-30T09:14:36.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.990+0000 7fb19ac47640 1 -- 172.21.15.42:0/3122749465 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb18c095020 msgr2=0x7fb18c0a07b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:14:36.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.990+0000 7fb19ac47640 1 --2- 172.21.15.42:0/3122749465 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb18c095020 0x7fb18c0a07b0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fb1900097d0 tx=0x7fb190009c90 comp rx=0 tx=0).stop 2024-08-30T09:14:36.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.991+0000 7fb19ac47640 1 -- 172.21.15.42:0/3122749465 shutdown_connections 2024-08-30T09:14:36.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.991+0000 7fb19ac47640 1 --2- 172.21.15.42:0/3122749465 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fb1800b4bb0 0x7fb1800b70c0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:36.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.991+0000 7fb19ac47640 1 --2- 172.21.15.42:0/3122749465 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb18c09a960 0x7fb18c0a0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:36.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.991+0000 7fb19ac47640 1 --2- 172.21.15.42:0/3122749465 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb18c09b2b0 0x7fb18c0a1230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:36.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.991+0000 7fb19ac47640 1 --2- 172.21.15.42:0/3122749465 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb18c095020 0x7fb18c0a07b0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:36.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.991+0000 7fb19ac47640 1 -- 172.21.15.42:0/3122749465 >> 172.21.15.42:0/3122749465 conn(0x7fb18c00cd70 msgr2=0x7fb18c097e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:14:36.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.991+0000 7fb19ac47640 1 -- 172.21.15.42:0/3122749465 shutdown_connections 2024-08-30T09:14:36.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:14:35.991+0000 7fb19ac47640 1 -- 172.21.15.42:0/3122749465 wait complete. 2024-08-30T09:14:37.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:14:36 smithi167 ceph-mon[22063]: pgmap v788: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 3.8 MiB/s wr, 473 op/s 2024-08-30T09:14:37.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:14:36 smithi042 ceph-mon[18652]: pgmap v788: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 3.8 MiB/s wr, 473 op/s 2024-08-30T09:14:37.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:14:36 smithi190 ceph-mon[22001]: pgmap v788: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 3.8 MiB/s wr, 473 op/s 2024-08-30T09:14:37.521 INFO:tasks.cephfs.filesystem:scrub status for tag:d325166d-e7e4-4ac2-9eaf-fa602e480cf6 - {'path': '/', 'tag': 'd325166d-e7e4-4ac2-9eaf-fa602e480cf6', 'options': 'recursive,force'} 2024-08-30T09:14:37.932 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:37.930+0000 7f998c770640 1 -- 172.21.15.42:0/45620551 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f99841068c0 msgr2=0x7f9984108ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:14:37.933 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:37.930+0000 7f998c770640 1 --2- 172.21.15.42:0/45620551 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f99841068c0 0x7f9984108ce0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f9974009900 tx=0x7f997402a410 comp rx=0 tx=0).stop 2024-08-30T09:14:37.933 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:37.930+0000 7f998c770640 1 -- 172.21.15.42:0/45620551 shutdown_connections 2024-08-30T09:14:37.933 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:37.930+0000 7f998c770640 1 --2- 172.21.15.42:0/45620551 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9984109220 0x7f998410b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:37.933 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:37.930+0000 7f998c770640 1 --2- 172.21.15.42:0/45620551 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f99841068c0 0x7f9984108ce0 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:37.933 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:37.930+0000 7f998c770640 1 --2- 172.21.15.42:0/45620551 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9984101830 0x7f9984101c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:37.933 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:37.930+0000 7f998c770640 1 -- 172.21.15.42:0/45620551 >> 172.21.15.42:0/45620551 conn(0x7f99840fd430 msgr2=0x7f99840ff8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:14:37.933 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:37.931+0000 7f998c770640 1 -- 172.21.15.42:0/45620551 shutdown_connections 2024-08-30T09:14:37.933 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:37.931+0000 7f998c770640 1 -- 172.21.15.42:0/45620551 wait complete. 2024-08-30T09:14:37.936 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:37.931+0000 7f998c770640 1 Processor -- start 2024-08-30T09:14:37.936 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:37.931+0000 7f998c770640 1 -- start start 2024-08-30T09:14:37.936 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:37.931+0000 7f998c770640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9984101830 0x7f9984199d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:14:37.937 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:37.931+0000 7f998c770640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f99841068c0 0x7f998419a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:14:37.937 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:37.931+0000 7f998c770640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9984109220 0x7f998419af20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:14:37.937 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:37.931+0000 7f998c770640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f998410f100 con 0x7f99841068c0 2024-08-30T09:14:37.937 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:37.931+0000 7f998c770640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f998410ef80 con 0x7f9984109220 2024-08-30T09:14:37.937 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:37.931+0000 7f998c770640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f998410f280 con 0x7f9984101830 2024-08-30T09:14:37.937 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:37.932+0000 7f9989d0b640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f99841068c0 0x7f998419a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:14:37.937 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:37.932+0000 7f998a50c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9984101830 0x7f9984199d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:14:37.938 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:37.932+0000 7f998a50c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9984101830 0x7f9984199d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:41468/0 (socket says 172.21.15.42:41468) 2024-08-30T09:14:37.938 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:37.932+0000 7f9989d0b640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f99841068c0 0x7f998419a370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:39796/0 (socket says 172.21.15.42:39796) 2024-08-30T09:14:37.938 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:37.932+0000 7f998ad0d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9984109220 0x7f998419af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:14:37.938 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:37.932+0000 7f998a50c640 1 -- 172.21.15.42:0/2822224648 learned_addr learned my addr 172.21.15.42:0/2822224648 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:14:37.938 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:37.932+0000 7f9989d0b640 1 -- 172.21.15.42:0/2822224648 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9984101830 msgr2=0x7f9984199d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:14:37.938 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:37.932+0000 7f9989d0b640 1 --2- 172.21.15.42:0/2822224648 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9984101830 0x7f9984199d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:37.938 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:37.932+0000 7f9989d0b640 1 -- 172.21.15.42:0/2822224648 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9984109220 msgr2=0x7f998419af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:14:37.938 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:37.932+0000 7f9989d0b640 1 --2- 172.21.15.42:0/2822224648 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9984109220 0x7f998419af20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:37.938 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:37.932+0000 7f9989d0b640 1 -- 172.21.15.42:0/2822224648 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f998419b740 con 0x7f99841068c0 2024-08-30T09:14:37.938 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:37.932+0000 7f9989d0b640 1 --2- 172.21.15.42:0/2822224648 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f99841068c0 0x7f998419a370 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f997402aa00 tx=0x7f9974003d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:14:37.938 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:37.932+0000 7f997b7fe640 1 -- 172.21.15.42:0/2822224648 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f997403f070 con 0x7f99841068c0 2024-08-30T09:14:37.938 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:37.932+0000 7f998c770640 1 -- 172.21.15.42:0/2822224648 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f998419b9d0 con 0x7f99841068c0 2024-08-30T09:14:37.938 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:37.932+0000 7f997b7fe640 1 -- 172.21.15.42:0/2822224648 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f997403b460 con 0x7f99841068c0 2024-08-30T09:14:37.939 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:37.932+0000 7f997b7fe640 1 -- 172.21.15.42:0/2822224648 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9974035030 con 0x7f99841068c0 2024-08-30T09:14:37.939 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:37.932+0000 7f998c770640 1 -- 172.21.15.42:0/2822224648 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99841a1f40 con 0x7f99841068c0 2024-08-30T09:14:37.940 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:37.936+0000 7f99936d8740 1 Processor -- start 2024-08-30T09:14:37.940 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:37.936+0000 7f99936d8740 1 -- start start 2024-08-30T09:14:37.940 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:37.937+0000 7f99936d8740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5566eba731b0 0x5566ebe18f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:14:37.941 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:37.937+0000 7f99936d8740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5566eba4a2f0 0x5566ebe195d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:14:37.941 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:37.937+0000 7f99936d8740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5566eba524d0 0x5566ebe1a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:14:37.941 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:37.937+0000 7f99936d8740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x5566ebd8c000 con 0x5566eba4a2f0 2024-08-30T09:14:37.941 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:37.937+0000 7f99936d8740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5566ebd7b590 con 0x5566eba524d0 2024-08-30T09:14:37.941 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:37.937+0000 7f99936d8740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5566ebd7f540 con 0x5566eba731b0 2024-08-30T09:14:37.941 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:37.937+0000 7f998ad0d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5566eba731b0 0x5566ebe18f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:14:37.941 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:37.937+0000 7f998ad0d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5566eba731b0 0x5566ebe18f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:41476/0 (socket says 172.21.15.42:41476) 2024-08-30T09:14:37.941 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:37.937+0000 7f998ad0d640 1 -- 172.21.15.42:0/199630587 learned_addr learned my addr 172.21.15.42:0/199630587 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:14:37.941 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:37.937+0000 7f998a50c640 1 --2- 172.21.15.42:0/199630587 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5566eba4a2f0 0x5566ebe195d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:14:37.942 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:37.937+0000 7f998ad0d640 1 -- 172.21.15.42:0/199630587 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5566eba524d0 msgr2=0x5566ebe1a120 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-30T09:14:37.942 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:37.937+0000 7f998ad0d640 1 --2- 172.21.15.42:0/199630587 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5566eba524d0 0x5566ebe1a120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:37.942 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:37.937+0000 7f998ad0d640 1 -- 172.21.15.42:0/199630587 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5566eba4a2f0 msgr2=0x5566ebe195d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:14:37.942 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:37.937+0000 7f998ad0d640 1 --2- 172.21.15.42:0/199630587 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5566eba4a2f0 0x5566ebe195d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:37.942 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:37.937+0000 7f998ad0d640 1 -- 172.21.15.42:0/199630587 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5566ebade790 con 0x5566eba731b0 2024-08-30T09:14:37.942 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:37.937+0000 7f998a50c640 1 --2- 172.21.15.42:0/199630587 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5566eba4a2f0 0x5566ebe195d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-30T09:14:37.942 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:37.938+0000 7f998ad0d640 1 --2- 172.21.15.42:0/199630587 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5566eba731b0 0x5566ebe18f60 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f997c009f70 tx=0x7f997c010800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ebc2e2d0e06a4762 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:14:37.942 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:37.938+0000 7f9978ff9640 1 -- 172.21.15.42:0/199630587 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f997c016070 con 0x5566eba731b0 2024-08-30T09:14:37.942 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:37.938+0000 7f9978ff9640 1 -- 172.21.15.42:0/199630587 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f997c004d20 con 0x5566eba731b0 2024-08-30T09:14:37.942 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:37.938+0000 7f9978ff9640 1 -- 172.21.15.42:0/199630587 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f997c012790 con 0x5566eba731b0 2024-08-30T09:14:37.942 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:37.938+0000 7f997b7fe640 1 -- 172.21.15.42:0/2822224648 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7f9974042070 con 0x7f99841068c0 2024-08-30T09:14:37.943 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:37.939+0000 7f99936d8740 1 -- 172.21.15.42:0/199630587 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5566eba731b0 msgr2=0x5566ebe18f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:14:37.943 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:37.939+0000 7f99936d8740 1 --2- 172.21.15.42:0/199630587 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5566eba731b0 0x5566ebe18f60 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f997c009f70 tx=0x7f997c010800 comp rx=0 tx=0).stop 2024-08-30T09:14:37.943 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:37.939+0000 7f997b7fe640 1 --2- 172.21.15.42:0/2822224648 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f99700b4b30 0x7f99700b7040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:14:37.943 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:37.940+0000 7f99936d8740 1 -- 172.21.15.42:0/199630587 shutdown_connections 2024-08-30T09:14:37.943 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:37.940+0000 7f99936d8740 1 --2- 172.21.15.42:0/199630587 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5566eba524d0 0x5566ebe1a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:37.944 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:37.940+0000 7f997b7fe640 1 -- 172.21.15.42:0/2822224648 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f9974105d10 con 0x7f99841068c0 2024-08-30T09:14:37.944 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:37.940+0000 7f99936d8740 1 --2- 172.21.15.42:0/199630587 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5566eba4a2f0 0x5566ebe195d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:37.944 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:37.940+0000 7f99936d8740 1 --2- 172.21.15.42:0/199630587 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5566eba731b0 0x5566ebe18f60 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:37.944 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:37.940+0000 7f99936d8740 1 -- 172.21.15.42:0/199630587 >> 172.21.15.42:0/199630587 conn(0x5566eba67300 msgr2=0x5566ebe149f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:14:37.949 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:37.947+0000 7f99936d8740 1 -- 172.21.15.42:0/199630587 shutdown_connections 2024-08-30T09:14:37.954 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:37.952+0000 7f998a50c640 1 --2- 172.21.15.42:0/2822224648 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f99700b4b30 0x7f99700b7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:14:37.955 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:37.953+0000 7f99936d8740 1 -- 172.21.15.42:0/199630587 wait complete. 2024-08-30T09:14:37.955 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:37.953+0000 7f99936d8740 1 -- start start 2024-08-30T09:14:37.955 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:37.953+0000 7f99936d8740 1 Processor -- start 2024-08-30T09:14:37.957 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:37.955+0000 7f99936d8740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5566eba4a2f0 0x5566ebe24f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:14:37.957 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:37.955+0000 7f99936d8740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5566eba524d0 0x5566ebe23cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:14:37.957 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:37.955+0000 7f99936d8740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5566eba731b0 0x5566ebe243d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:14:37.957 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:37.955+0000 7f99936d8740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x5566ebcb4ab0 con 0x5566eba731b0 2024-08-30T09:14:37.957 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:37.955+0000 7f99936d8740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5566ebd02680 con 0x5566eba4a2f0 2024-08-30T09:14:37.957 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:37.955+0000 7f99936d8740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5566ebe1ac40 con 0x5566eba524d0 2024-08-30T09:14:37.957 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:37.955+0000 7f99936d8740 10 client.0 _ll_register_callbacks cb 0x5566eba8aa00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:14:37.957 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:37.955+0000 7f998ad0d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5566eba731b0 0x5566ebe243d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:14:37.957 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:37.955+0000 7f998ad0d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5566eba731b0 0x5566ebe243d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:39816/0 (socket says 172.21.15.42:39816) 2024-08-30T09:14:37.957 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:37.955+0000 7f998ad0d640 1 -- 172.21.15.42:0/1585932944 learned_addr learned my addr 172.21.15.42:0/1585932944 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:14:37.960 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:37.955+0000 7f9989d0b640 1 --2- 172.21.15.42:0/1585932944 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5566eba524d0 0x5566ebe23cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:14:37.960 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:37.958+0000 7f998a50c640 1 --2- 172.21.15.42:0/1585932944 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5566eba4a2f0 0x5566ebe24f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:14:37.961 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:37.958+0000 7f998ad0d640 1 -- 172.21.15.42:0/1585932944 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5566eba524d0 msgr2=0x5566ebe23cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:14:37.961 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:37.958+0000 7f998ad0d640 1 --2- 172.21.15.42:0/1585932944 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5566eba524d0 0x5566ebe23cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:37.961 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:37.958+0000 7f998ad0d640 1 -- 172.21.15.42:0/1585932944 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5566eba4a2f0 msgr2=0x5566ebe24f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:14:37.961 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:37.958+0000 7f998ad0d640 1 --2- 172.21.15.42:0/1585932944 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5566eba4a2f0 0x5566ebe24f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:37.961 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:37.958+0000 7f998ad0d640 1 -- 172.21.15.42:0/1585932944 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5566ebb57620 con 0x5566eba731b0 2024-08-30T09:14:37.961 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:37.959+0000 7f998ad0d640 1 --2- 172.21.15.42:0/1585932944 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5566eba731b0 0x5566ebe243d0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f997c005290 tx=0x7f997c0052c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e01222470ccbf580 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:14:37.962 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:37.960+0000 7f996dffb640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-08-30T09:14:37.962 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:37.960+0000 7f996dffb640 1 -- 172.21.15.42:0/1585932944 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f997c016070 con 0x5566eba731b0 2024-08-30T09:14:37.966 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:37.960+0000 7f99936d8740 1 -- 172.21.15.42:0/1585932944 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f997c004d20 con 0x5566eba731b0 2024-08-30T09:14:37.966 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:37.960+0000 7f996dffb640 1 -- 172.21.15.42:0/1585932944 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f997c003b70 con 0x5566eba731b0 2024-08-30T09:14:37.967 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:37.965+0000 7f996dffb640 1 -- 172.21.15.42:0/1585932944 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f997c02f050 con 0x5566eba731b0 2024-08-30T09:14:37.967 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:37.965+0000 7f998a50c640 1 --2- 172.21.15.42:0/2822224648 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f99700b4b30 0x7f99700b7040 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f9980005fd0 tx=0x7f998000bce0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:14:37.967 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:37.966+0000 7f99936d8740 10 client.17745 fetch_fsmap learned FSMap version 47 2024-08-30T09:14:37.968 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:37.966+0000 7f99936d8740 1 -- 172.21.15.42:0/1585932944 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x5566ebe46450 con 0x5566eba731b0 2024-08-30T09:14:37.969 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:37.967+0000 7f996dffb640 1 -- 172.21.15.42:0/1585932944 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f997c004d20 con 0x5566eba731b0 2024-08-30T09:14:37.969 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:37.967+0000 7f996dffb640 1 -- 172.21.15.42:0/1585932944 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 47) ==== 8370+0+0 (secure 0 0 0) 0x7f997c00f040 con 0x5566eba731b0 2024-08-30T09:14:37.970 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:37.968+0000 7f99936d8740 10 client.17745 fetch_fsmap finished waiting for FSMap version 47 2024-08-30T09:14:37.970 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:37.968+0000 7f99936d8740 10 client.17745 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-30T09:14:37.970 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:37.968+0000 7f99936d8740 20 client.17745 populate_metadata read hostname 'smithi042' 2024-08-30T09:14:37.970 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:37.968+0000 7f99936d8740 1 --2- 172.21.15.42:0/1585932944 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x5566eba31d80 0x5566ebe498c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:14:37.970 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:37.968+0000 7f99936d8740 4 client.17745 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-30T09:14:37.970 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:37.968+0000 7f99936d8740 1 -- 172.21.15.42:0/1585932944 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5566ebd8df50 con 0x5566eba31d80 2024-08-30T09:14:37.974 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:37.970+0000 7f998a50c640 1 --2- 172.21.15.42:0/1585932944 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x5566eba31d80 0x5566ebe498c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:14:37.980 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:37.978+0000 7f998a50c640 1 --2- 172.21.15.42:0/1585932944 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x5566eba31d80 0x5566ebe498c0 crc :-1 s=READY pgs=285 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=48bd61ecefdbe6b8 server_cookie=be509d2143b86097 in_seq=0 out_seq=0 2024-08-30T09:14:37.981 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:37.980+0000 7f996dffb640 10 client.17745 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:14:37.981 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:37.980+0000 7f996dffb640 1 -- 172.21.15.42:0/1585932944 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5566ebd8df50 con 0x5566eba31d80 2024-08-30T09:14:37.982 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:37.980+0000 7f996dffb640 10 client.17745 handle_command_reply: tid=0 2024-08-30T09:14:37.983 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:37.980+0000 7f99936d8740 1 client.17745 shutdown 2024-08-30T09:14:37.993 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:37.991+0000 7f99936d8740 1 -- 172.21.15.42:0/1585932944 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5566eba731b0 msgr2=0x5566ebe243d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:14:37.994 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:37.991+0000 7f99936d8740 1 --2- 172.21.15.42:0/1585932944 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5566eba731b0 0x5566ebe243d0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f997c005290 tx=0x7f997c0052c0 comp rx=0 tx=0).stop 2024-08-30T09:14:37.999 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:37.997+0000 7f99936d8740 1 -- 172.21.15.42:0/1585932944 shutdown_connections 2024-08-30T09:14:37.999 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:37.997+0000 7f99936d8740 1 --2- 172.21.15.42:0/1585932944 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x5566eba31d80 0x5566ebe498c0 crc :-1 s=READY pgs=285 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:37.999 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:37.997+0000 7f99936d8740 1 --2- 172.21.15.42:0/1585932944 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5566eba524d0 0x5566ebe23cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:37.999 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:37.997+0000 7f99936d8740 1 --2- 172.21.15.42:0/1585932944 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5566eba731b0 0x5566ebe243d0 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:37.999 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:37.997+0000 7f99936d8740 1 --2- 172.21.15.42:0/1585932944 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5566eba4a2f0 0x5566ebe24f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:38.000 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:37.997+0000 7f99936d8740 1 -- 172.21.15.42:0/1585932944 >> 172.21.15.42:0/1585932944 conn(0x5566eba67300 msgr2=0x5566ebe149d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:14:38.000 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:37.997+0000 7f996dffb640 0 client.17745 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:14:38.007 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:38.003+0000 7f99936d8740 1 -- 172.21.15.42:0/1585932944 shutdown_connections 2024-08-30T09:14:38.008 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:38.006+0000 7f99936d8740 1 -- 172.21.15.42:0/1585932944 wait complete. 2024-08-30T09:14:38.009 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:38.007+0000 7f99936d8740 20 client.17745 trim_cache size 0 max 16384 2024-08-30T09:14:38.054 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:38.052+0000 7f99936d8740 1 Processor -- start 2024-08-30T09:14:38.054 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:38.052+0000 7f99936d8740 1 -- start start 2024-08-30T09:14:38.055 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:38.052+0000 7f99936d8740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5566eba4a2f0 0x5566ebe12b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:14:38.055 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:38.052+0000 7f99936d8740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5566eba524d0 0x5566ebe130a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:14:38.055 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:38.052+0000 7f99936d8740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5566eba731b0 0x5566ebe20ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:14:38.055 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:38.053+0000 7f99936d8740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f99841a4330 con 0x5566eba4a2f0 2024-08-30T09:14:38.055 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:38.053+0000 7f99936d8740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f9984074dc0 con 0x5566eba731b0 2024-08-30T09:14:38.056 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:38.053+0000 7f99936d8740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f9984072c10 con 0x5566eba524d0 2024-08-30T09:14:38.056 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:38.053+0000 7f9989d0b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5566eba524d0 0x5566ebe130a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:14:38.056 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:38.053+0000 7f9989d0b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5566eba524d0 0x5566ebe130a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:41494/0 (socket says 172.21.15.42:41494) 2024-08-30T09:14:38.056 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:38.053+0000 7f9989d0b640 1 -- 172.21.15.42:0/1075724684 learned_addr learned my addr 172.21.15.42:0/1075724684 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:14:38.057 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:38.054+0000 7f998ad0d640 1 --2- 172.21.15.42:0/1075724684 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5566eba731b0 0x5566ebe20ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:14:38.057 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:38.054+0000 7f998a50c640 1 --2- 172.21.15.42:0/1075724684 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5566eba4a2f0 0x5566ebe12b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:14:38.057 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:38.055+0000 7f9989d0b640 1 -- 172.21.15.42:0/1075724684 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5566eba731b0 msgr2=0x5566ebe20ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:14:38.057 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:38.055+0000 7f9989d0b640 1 --2- 172.21.15.42:0/1075724684 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5566eba731b0 0x5566ebe20ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:38.057 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:38.055+0000 7f9989d0b640 1 -- 172.21.15.42:0/1075724684 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5566eba4a2f0 msgr2=0x5566ebe12b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:14:38.057 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:38.055+0000 7f9989d0b640 1 --2- 172.21.15.42:0/1075724684 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5566eba4a2f0 0x5566ebe12b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:38.057 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:38.055+0000 7f9989d0b640 1 -- 172.21.15.42:0/1075724684 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5566ebe20c70 con 0x5566eba524d0 2024-08-30T09:14:38.057 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:38.055+0000 7f998a50c640 1 --2- 172.21.15.42:0/1075724684 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5566eba4a2f0 0x5566ebe12b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-30T09:14:38.058 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:38.055+0000 7f9989d0b640 1 --2- 172.21.15.42:0/1075724684 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5566eba524d0 0x5566ebe130a0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f99740c84e0 tx=0x7f997402a410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a366bcc84e4f9e9c server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:14:38.058 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:38.055+0000 7f996ffff640 1 -- 172.21.15.42:0/1075724684 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9974049040 con 0x5566eba524d0 2024-08-30T09:14:38.059 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:38.057+0000 7f996ffff640 1 -- 172.21.15.42:0/1075724684 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f99740cc460 con 0x5566eba524d0 2024-08-30T09:14:38.059 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:38.057+0000 7f996ffff640 1 -- 172.21.15.42:0/1075724684 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99740cc6c0 con 0x5566eba524d0 2024-08-30T09:14:38.061 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:38.059+0000 7f99936d8740 1 -- 172.21.15.42:0/1075724684 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5566eba524d0 msgr2=0x5566ebe130a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:14:38.061 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:38.059+0000 7f99936d8740 1 --2- 172.21.15.42:0/1075724684 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5566eba524d0 0x5566ebe130a0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f99740c84e0 tx=0x7f997402a410 comp rx=0 tx=0).stop 2024-08-30T09:14:38.061 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:38.059+0000 7f99936d8740 1 -- 172.21.15.42:0/1075724684 shutdown_connections 2024-08-30T09:14:38.061 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:38.059+0000 7f99936d8740 1 --2- 172.21.15.42:0/1075724684 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5566eba524d0 0x5566ebe130a0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:38.061 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:38.059+0000 7f99936d8740 1 --2- 172.21.15.42:0/1075724684 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5566eba731b0 0x5566ebe20ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:38.061 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:38.059+0000 7f99936d8740 1 --2- 172.21.15.42:0/1075724684 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5566eba4a2f0 0x5566ebe12b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:38.061 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:38.059+0000 7f99936d8740 1 -- 172.21.15.42:0/1075724684 >> 172.21.15.42:0/1075724684 conn(0x5566eba67300 msgr2=0x5566ebe21c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:14:38.061 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:38.059+0000 7f99936d8740 1 -- 172.21.15.42:0/1075724684 shutdown_connections 2024-08-30T09:14:38.062 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:38.059+0000 7f99936d8740 1 -- 172.21.15.42:0/1075724684 wait complete. 2024-08-30T09:14:38.064 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:38.059+0000 7f99936d8740 1 -- start start 2024-08-30T09:14:38.064 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:38.060+0000 7f99936d8740 1 Processor -- start 2024-08-30T09:14:38.064 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:38.060+0000 7f99936d8740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5566eba4a2f0 0x5566ebe30af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:14:38.064 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:38.060+0000 7f99936d8740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5566eba524d0 0x5566ebe311b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:14:38.064 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:38.060+0000 7f99936d8740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5566eba731b0 0x5566ebe31c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:14:38.064 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:38.060+0000 7f99936d8740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f9984073280 con 0x5566eba4a2f0 2024-08-30T09:14:38.064 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:38.060+0000 7f99936d8740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f99840746f0 con 0x5566eba524d0 2024-08-30T09:14:38.064 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:38.060+0000 7f99936d8740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f9984073950 con 0x5566eba731b0 2024-08-30T09:14:38.064 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:38.060+0000 7f99936d8740 10 client.0 _ll_register_callbacks cb 0x7f997c009df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:14:38.064 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:38.060+0000 7f998a50c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5566eba4a2f0 0x5566ebe30af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:14:38.064 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:38.060+0000 7f9989d0b640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5566eba524d0 0x5566ebe311b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:14:38.065 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:38.060+0000 7f9989d0b640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5566eba524d0 0x5566ebe311b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:50514/0 (socket says 172.21.15.42:50514) 2024-08-30T09:14:38.065 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:38.060+0000 7f998a50c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5566eba4a2f0 0x5566ebe30af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:39842/0 (socket says 172.21.15.42:39842) 2024-08-30T09:14:38.065 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:38.060+0000 7f9989d0b640 1 -- 172.21.15.42:0/1913229868 learned_addr learned my addr 172.21.15.42:0/1913229868 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:14:38.065 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:38.060+0000 7f998ad0d640 1 --2- 172.21.15.42:0/1913229868 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5566eba731b0 0x5566ebe31c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:14:38.065 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:38.061+0000 7f998a50c640 1 -- 172.21.15.42:0/1913229868 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5566eba731b0 msgr2=0x5566ebe31c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:14:38.065 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:38.061+0000 7f998a50c640 1 --2- 172.21.15.42:0/1913229868 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5566eba731b0 0x5566ebe31c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:38.065 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:38.061+0000 7f998a50c640 1 -- 172.21.15.42:0/1913229868 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5566eba524d0 msgr2=0x5566ebe311b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:14:38.065 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:38.061+0000 7f998a50c640 1 --2- 172.21.15.42:0/1913229868 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5566eba524d0 0x5566ebe311b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:38.065 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:38.061+0000 7f998a50c640 1 -- 172.21.15.42:0/1913229868 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5566ebe323e0 con 0x5566eba4a2f0 2024-08-30T09:14:38.065 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:38.061+0000 7f9989d0b640 1 --2- 172.21.15.42:0/1913229868 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5566eba524d0 0x5566ebe311b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-30T09:14:38.065 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:38.061+0000 7f998a50c640 1 --2- 172.21.15.42:0/1913229868 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5566eba4a2f0 0x5566ebe30af0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f997c009d30 tx=0x7f99800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f21edadbc6ecdd3 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:14:38.065 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:38.061+0000 7f996dffb640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-08-30T09:14:38.065 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:38.061+0000 7f996dffb640 1 -- 172.21.15.42:0/1913229868 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99800142d0 con 0x5566eba4a2f0 2024-08-30T09:14:38.066 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:38.061+0000 7f99936d8740 1 -- 172.21.15.42:0/1913229868 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f99740cc460 con 0x5566eba4a2f0 2024-08-30T09:14:38.066 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:38.061+0000 7f996dffb640 1 -- 172.21.15.42:0/1913229868 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f9980005020 con 0x5566eba4a2f0 2024-08-30T09:14:38.066 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:38.061+0000 7f996dffb640 1 -- 172.21.15.42:0/1913229868 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f9980036050 con 0x5566eba4a2f0 2024-08-30T09:14:38.066 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:38.061+0000 7f99936d8740 10 client.17751 fetch_fsmap learned FSMap version 47 2024-08-30T09:14:38.066 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:38.061+0000 7f99936d8740 1 -- 172.21.15.42:0/1913229868 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x5566ebe337f0 con 0x5566eba4a2f0 2024-08-30T09:14:38.066 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:38.062+0000 7f996dffb640 1 -- 172.21.15.42:0/1913229868 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f99740cc460 con 0x5566eba4a2f0 2024-08-30T09:14:38.066 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:38.062+0000 7f996dffb640 1 -- 172.21.15.42:0/1913229868 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 47) ==== 8370+0+0 (secure 0 0 0) 0x7f998001f040 con 0x5566eba4a2f0 2024-08-30T09:14:38.066 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:38.062+0000 7f99936d8740 10 client.17751 fetch_fsmap finished waiting for FSMap version 47 2024-08-30T09:14:38.066 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:38.062+0000 7f99936d8740 10 client.17751 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-30T09:14:38.066 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:38.062+0000 7f99936d8740 20 client.17751 populate_metadata read hostname 'smithi042' 2024-08-30T09:14:38.067 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:38.062+0000 7f99936d8740 1 --2- 172.21.15.42:0/1913229868 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x5566eba071c0 0x5566ebeb6700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:14:38.067 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:38.062+0000 7f99936d8740 4 client.17751 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-30T09:14:38.067 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:38.062+0000 7f99936d8740 1 -- 172.21.15.42:0/1913229868 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5566ebd899f0 con 0x5566eba071c0 2024-08-30T09:14:38.067 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:38.062+0000 7f9989d0b640 1 --2- 172.21.15.42:0/1913229868 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x5566eba071c0 0x5566ebeb6700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:14:38.067 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:38.062+0000 7f996dffb640 10 client.17751 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:14:38.067 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:38.062+0000 7f9989d0b640 1 --2- 172.21.15.42:0/1913229868 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x5566eba071c0 0x5566ebeb6700 crc :-1 s=READY pgs=286 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=459870c59fc6f793 server_cookie=f4d7dab4bb53a49b in_seq=0 out_seq=0 2024-08-30T09:14:38.079 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:38.076+0000 7f996dffb640 1 -- 172.21.15.42:0/1913229868 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5566ebd899f0 con 0x5566eba071c0 2024-08-30T09:14:38.079 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:38.076+0000 7f996dffb640 10 client.17751 handle_command_reply: tid=0 2024-08-30T09:14:38.079 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:38.076+0000 7f99936d8740 1 client.17751 shutdown 2024-08-30T09:14:38.079 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:38.077+0000 7f99936d8740 1 -- 172.21.15.42:0/1913229868 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5566eba4a2f0 msgr2=0x5566ebe30af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:14:38.079 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:38.077+0000 7f99936d8740 1 --2- 172.21.15.42:0/1913229868 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5566eba4a2f0 0x5566ebe30af0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f997c009d30 tx=0x7f99800046c0 comp rx=0 tx=0).stop 2024-08-30T09:14:38.079 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:38.077+0000 7f99936d8740 1 -- 172.21.15.42:0/1913229868 shutdown_connections 2024-08-30T09:14:38.079 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:38.077+0000 7f99936d8740 1 --2- 172.21.15.42:0/1913229868 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x5566eba071c0 0x5566ebeb6700 crc :-1 s=READY pgs=286 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:38.079 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:38.077+0000 7f99936d8740 1 --2- 172.21.15.42:0/1913229868 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5566eba731b0 0x5566ebe31c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:38.079 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:38.077+0000 7f99936d8740 1 --2- 172.21.15.42:0/1913229868 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5566eba524d0 0x5566ebe311b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:38.079 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:38.077+0000 7f99936d8740 1 --2- 172.21.15.42:0/1913229868 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5566eba4a2f0 0x5566ebe30af0 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:38.080 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:38.077+0000 7f99936d8740 1 -- 172.21.15.42:0/1913229868 >> 172.21.15.42:0/1913229868 conn(0x5566eba67300 msgr2=0x5566ebe21c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:14:38.080 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:38.077+0000 7f996dffb640 0 client.17751 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:14:38.080 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:38.077+0000 7f99936d8740 1 -- 172.21.15.42:0/1913229868 shutdown_connections 2024-08-30T09:14:38.080 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:38.077+0000 7f99936d8740 1 -- 172.21.15.42:0/1913229868 wait complete. 2024-08-30T09:14:38.080 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:38.077+0000 7f99936d8740 20 client.17751 trim_cache size 0 max 16384 2024-08-30T09:14:38.080 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-08-30T09:14:38.080 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2024-08-30T09:14:38.080 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-08-30T09:14:38.080 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-08-30T09:14:38.081 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:38.078+0000 7f998c770640 1 -- 172.21.15.42:0/2822224648 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f99700b4b30 msgr2=0x7f99700b7040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:14:38.081 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:38.078+0000 7f998c770640 1 --2- 172.21.15.42:0/2822224648 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f99700b4b30 0x7f99700b7040 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f9980005fd0 tx=0x7f998000bce0 comp rx=0 tx=0).stop 2024-08-30T09:14:38.081 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:38.078+0000 7f998c770640 1 -- 172.21.15.42:0/2822224648 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f99841068c0 msgr2=0x7f998419a370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:14:38.081 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:38.078+0000 7f998c770640 1 --2- 172.21.15.42:0/2822224648 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f99841068c0 0x7f998419a370 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f997402aa00 tx=0x7f9974003d50 comp rx=0 tx=0).stop 2024-08-30T09:14:38.081 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:38.078+0000 7f998c770640 1 -- 172.21.15.42:0/2822224648 shutdown_connections 2024-08-30T09:14:38.081 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:38.078+0000 7f998c770640 1 --2- 172.21.15.42:0/2822224648 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f99700b4b30 0x7f99700b7040 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:38.081 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:38.078+0000 7f998c770640 1 --2- 172.21.15.42:0/2822224648 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9984109220 0x7f998419af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:38.081 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:38.078+0000 7f998c770640 1 --2- 172.21.15.42:0/2822224648 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f99841068c0 0x7f998419a370 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:38.082 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:38.078+0000 7f998c770640 1 --2- 172.21.15.42:0/2822224648 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9984101830 0x7f9984199d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:38.082 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:38.078+0000 7f998c770640 1 -- 172.21.15.42:0/2822224648 >> 172.21.15.42:0/2822224648 conn(0x7f99840fd430 msgr2=0x7f9984107770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:14:38.082 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:38.078+0000 7f998c770640 1 -- 172.21.15.42:0/2822224648 shutdown_connections 2024-08-30T09:14:38.082 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:38.078+0000 7f998c770640 1 -- 172.21.15.42:0/2822224648 wait complete. 2024-08-30T09:14:39.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:14:38 smithi167 ceph-mon[22063]: pgmap v789: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 3.6 MiB/s wr, 480 op/s 2024-08-30T09:14:39.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:14:38 smithi167 ceph-mon[22063]: scrub summary: idle 2024-08-30T09:14:39.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:14:38 smithi167 ceph-mon[22063]: scrub summary: idle 2024-08-30T09:14:39.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:14:38 smithi167 ceph-mon[22063]: scrub summary: idle 2024-08-30T09:14:39.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:14:38 smithi042 ceph-mon[18652]: pgmap v789: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 3.6 MiB/s wr, 480 op/s 2024-08-30T09:14:39.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:14:38 smithi042 ceph-mon[18652]: scrub summary: idle 2024-08-30T09:14:39.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:14:38 smithi042 ceph-mon[18652]: scrub summary: idle 2024-08-30T09:14:39.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:14:38 smithi042 ceph-mon[18652]: scrub summary: idle 2024-08-30T09:14:39.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:14:38 smithi190 ceph-mon[22001]: pgmap v789: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 3.6 MiB/s wr, 480 op/s 2024-08-30T09:14:39.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:14:38 smithi190 ceph-mon[22001]: scrub summary: idle 2024-08-30T09:14:39.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:14:38 smithi190 ceph-mon[22001]: scrub summary: idle 2024-08-30T09:14:39.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:14:38 smithi190 ceph-mon[22001]: scrub summary: idle 2024-08-30T09:14:40.953 INFO:teuthology.orchestra.run.smithi042.stderr:progress: 835.0 s, 518.6 tps, lat 190.952 ms stddev 221.394 2024-08-30T09:14:41.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:14:40 smithi167 ceph-mon[22063]: pgmap v790: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 4.9 MiB/s wr, 609 op/s 2024-08-30T09:14:41.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:14:40 smithi042 ceph-mon[18652]: pgmap v790: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 4.9 MiB/s wr, 609 op/s 2024-08-30T09:14:41.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:14:40 smithi190 ceph-mon[22001]: pgmap v790: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 4.9 MiB/s wr, 609 op/s 2024-08-30T09:14:43.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:14:42 smithi167 ceph-mon[22063]: pgmap v791: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 4.3 MiB/s wr, 524 op/s 2024-08-30T09:14:43.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:14:42 smithi042 ceph-mon[18652]: pgmap v791: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 4.3 MiB/s wr, 524 op/s 2024-08-30T09:14:43.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:14:42 smithi190 ceph-mon[22001]: pgmap v791: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 4.3 MiB/s wr, 524 op/s 2024-08-30T09:14:44.988 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:14:44 smithi190 ceph-mon[22001]: pgmap v792: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 4.6 MiB/s wr, 585 op/s 2024-08-30T09:14:45.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:14:44 smithi167 ceph-mon[22063]: pgmap v792: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 4.6 MiB/s wr, 585 op/s 2024-08-30T09:14:45.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:14:44 smithi042 ceph-mon[18652]: pgmap v792: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 4.6 MiB/s wr, 585 op/s 2024-08-30T09:14:45.953 INFO:teuthology.orchestra.run.smithi042.stderr:progress: 840.0 s, 458.0 tps, lat 216.707 ms stddev 276.619 2024-08-30T09:14:47.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:14:46 smithi167 ceph-mon[22063]: pgmap v793: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 4.1 MiB/s wr, 476 op/s 2024-08-30T09:14:47.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:14:46 smithi042 ceph-mon[18652]: pgmap v793: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 4.1 MiB/s wr, 476 op/s 2024-08-30T09:14:47.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:14:46 smithi190 ceph-mon[22001]: pgmap v793: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 4.1 MiB/s wr, 476 op/s 2024-08-30T09:14:49.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:14:48 smithi167 ceph-mon[22063]: pgmap v794: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 4.1 MiB/s wr, 457 op/s 2024-08-30T09:14:49.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:14:48 smithi042 ceph-mon[18652]: pgmap v794: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 4.1 MiB/s wr, 457 op/s 2024-08-30T09:14:49.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:14:48 smithi190 ceph-mon[22001]: pgmap v794: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 4.1 MiB/s wr, 457 op/s 2024-08-30T09:14:50.953 INFO:teuthology.orchestra.run.smithi042.stderr:progress: 845.0 s, 453.4 tps, lat 223.443 ms stddev 283.749 2024-08-30T09:14:51.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:14:50 smithi167 ceph-mon[22063]: pgmap v795: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 5.0 MiB/s wr, 550 op/s 2024-08-30T09:14:51.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:14:50 smithi042 ceph-mon[18652]: pgmap v795: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 5.0 MiB/s wr, 550 op/s 2024-08-30T09:14:51.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:14:50 smithi190 ceph-mon[22001]: pgmap v795: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 5.0 MiB/s wr, 550 op/s 2024-08-30T09:14:51.277 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.275+0000 7f14954ce640 1 -- 172.21.15.42:0/1003745597 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f149010f770 msgr2=0x7f149011a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:14:51.277 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.275+0000 7f14954ce640 1 --2- 172.21.15.42:0/1003745597 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f149010f770 0x7f149011a930 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f1484009860 tx=0x7f148402a360 comp rx=0 tx=0).stop 2024-08-30T09:14:51.285 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.281+0000 7f14954ce640 1 -- 172.21.15.42:0/1003745597 shutdown_connections 2024-08-30T09:14:51.285 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.281+0000 7f14954ce640 1 --2- 172.21.15.42:0/1003745597 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f149010cb50 0x7f149010f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:51.285 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.281+0000 7f14954ce640 1 --2- 172.21.15.42:0/1003745597 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f149010f770 0x7f149011a930 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:51.285 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.281+0000 7f14954ce640 1 --2- 172.21.15.42:0/1003745597 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f149010a160 0x7f149010c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:51.285 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.281+0000 7f14954ce640 1 -- 172.21.15.42:0/1003745597 >> 172.21.15.42:0/1003745597 conn(0x7f14900705b0 msgr2=0x7f1490072a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:14:51.288 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.285+0000 7f14954ce640 1 -- 172.21.15.42:0/1003745597 shutdown_connections 2024-08-30T09:14:51.293 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.289+0000 7f14954ce640 1 -- 172.21.15.42:0/1003745597 wait complete. 2024-08-30T09:14:51.293 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.289+0000 7f14954ce640 1 Processor -- start 2024-08-30T09:14:51.300 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.293+0000 7f14954ce640 1 -- start start 2024-08-30T09:14:51.300 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.293+0000 7f14954ce640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f149010a160 0x7f149011a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:14:51.300 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.293+0000 7f14954ce640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f149010cb50 0x7f1490118bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:14:51.300 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.293+0000 7f14954ce640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f149010f770 0x7f1490119100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:14:51.300 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.293+0000 7f14954ce640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f14901129e0 con 0x7f149010a160 2024-08-30T09:14:51.300 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.293+0000 7f14954ce640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f1490112860 con 0x7f149010f770 2024-08-30T09:14:51.300 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.293+0000 7f14954ce640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f1490112b60 con 0x7f149010cb50 2024-08-30T09:14:51.300 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.295+0000 7f148effd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f149010a160 0x7f149011a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:14:51.300 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.295+0000 7f148effd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f149010a160 0x7f149011a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:46546/0 (socket says 172.21.15.42:46546) 2024-08-30T09:14:51.300 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.295+0000 7f148effd640 1 -- 172.21.15.42:0/3579807485 learned_addr learned my addr 172.21.15.42:0/3579807485 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:14:51.300 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.295+0000 7f148f7fe640 1 --2- 172.21.15.42:0/3579807485 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f149010f770 0x7f1490119100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:14:51.301 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.295+0000 7f148effd640 1 -- 172.21.15.42:0/3579807485 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f149010cb50 msgr2=0x7f1490118bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:14:51.301 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.295+0000 7f148e7fc640 1 --2- 172.21.15.42:0/3579807485 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f149010cb50 0x7f1490118bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:14:51.301 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.296+0000 7f148effd640 1 --2- 172.21.15.42:0/3579807485 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f149010cb50 0x7f1490118bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:51.301 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.296+0000 7f148effd640 1 -- 172.21.15.42:0/3579807485 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f149010f770 msgr2=0x7f1490119100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:14:51.301 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.296+0000 7f148effd640 1 --2- 172.21.15.42:0/3579807485 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f149010f770 0x7f1490119100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:51.301 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.296+0000 7f148effd640 1 -- 172.21.15.42:0/3579807485 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1490119670 con 0x7f149010a160 2024-08-30T09:14:51.301 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.296+0000 7f148e7fc640 1 --2- 172.21.15.42:0/3579807485 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f149010cb50 0x7f1490118bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-30T09:14:51.301 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.296+0000 7f148f7fe640 1 --2- 172.21.15.42:0/3579807485 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f149010f770 0x7f1490119100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-30T09:14:51.304 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.298+0000 7f148effd640 1 --2- 172.21.15.42:0/3579807485 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f149010a160 0x7f149011a300 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f1488009fd0 tx=0x7f148800c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:14:51.304 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.298+0000 7f1477fff640 1 -- 172.21.15.42:0/3579807485 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1488038620 con 0x7f149010a160 2024-08-30T09:14:51.304 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.298+0000 7f14954ce640 1 -- 172.21.15.42:0/3579807485 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1490119960 con 0x7f149010a160 2024-08-30T09:14:51.305 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.298+0000 7f14954ce640 1 -- 172.21.15.42:0/3579807485 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f149011d950 con 0x7f149010a160 2024-08-30T09:14:51.305 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.299+0000 7f1477fff640 1 -- 172.21.15.42:0/3579807485 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1488038c70 con 0x7f149010a160 2024-08-30T09:14:51.305 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.300+0000 7f1477fff640 1 -- 172.21.15.42:0/3579807485 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1488038620 con 0x7f149010a160 2024-08-30T09:14:51.305 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.302+0000 7f149c434740 1 Processor -- start 2024-08-30T09:14:51.305 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.302+0000 7f149c434740 1 -- start start 2024-08-30T09:14:51.305 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.302+0000 7f149c434740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5611af4561b0 0x5611af7fc180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:14:51.305 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.302+0000 7f149c434740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5611af42d2f0 0x5611af7fc7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:14:51.305 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.303+0000 7f149c434740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5611af4354d0 0x5611af7fd340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:14:51.305 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.303+0000 7f149c434740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x5611af76f000 con 0x5611af42d2f0 2024-08-30T09:14:51.305 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.303+0000 7f149c434740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5611af75e590 con 0x5611af4561b0 2024-08-30T09:14:51.305 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.303+0000 7f149c434740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5611af762540 con 0x5611af4354d0 2024-08-30T09:14:51.307 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.305+0000 7f148f7fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5611af4561b0 0x5611af7fc180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:14:51.307 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.305+0000 7f148effd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5611af42d2f0 0x5611af7fc7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:14:51.307 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.305+0000 7f148effd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5611af42d2f0 0x5611af7fc7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:46562/0 (socket says 172.21.15.42:46562) 2024-08-30T09:14:51.307 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.305+0000 7f148effd640 1 -- 172.21.15.42:0/4158969171 learned_addr learned my addr 172.21.15.42:0/4158969171 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:14:51.307 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.305+0000 7f1477fff640 1 -- 172.21.15.42:0/3579807485 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7f14880387c0 con 0x7f149010a160 2024-08-30T09:14:51.308 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.306+0000 7f148effd640 1 -- 172.21.15.42:0/4158969171 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5611af4354d0 msgr2=0x5611af7fd340 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-30T09:14:51.308 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.306+0000 7f148effd640 1 --2- 172.21.15.42:0/4158969171 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5611af4354d0 0x5611af7fd340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:51.308 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.306+0000 7f148effd640 1 -- 172.21.15.42:0/4158969171 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5611af4561b0 msgr2=0x5611af7fc180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:14:51.308 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.306+0000 7f148effd640 1 --2- 172.21.15.42:0/4158969171 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5611af4561b0 0x5611af7fc180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:51.308 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.306+0000 7f148effd640 1 -- 172.21.15.42:0/4158969171 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5611af4c1790 con 0x5611af42d2f0 2024-08-30T09:14:51.309 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.306+0000 7f1477fff640 1 --2- 172.21.15.42:0/3579807485 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f14780b4e00 0x7f14780b7310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:14:51.310 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.307+0000 7f148effd640 1 --2- 172.21.15.42:0/4158969171 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5611af42d2f0 0x5611af7fc7f0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f148810d9d0 tx=0x7f148810de90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e44b9a34adc72cfc server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:14:51.310 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.308+0000 7f14757fa640 1 -- 172.21.15.42:0/4158969171 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14880c9e70 con 0x5611af42d2f0 2024-08-30T09:14:51.310 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.308+0000 7f14757fa640 1 -- 172.21.15.42:0/4158969171 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14880f4070 con 0x5611af42d2f0 2024-08-30T09:14:51.310 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.308+0000 7f1477fff640 1 -- 172.21.15.42:0/3579807485 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f1488107970 con 0x7f149010a160 2024-08-30T09:14:51.312 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.309+0000 7f149c434740 1 -- 172.21.15.42:0/4158969171 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5611af42d2f0 msgr2=0x5611af7fc7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:14:51.313 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.309+0000 7f149c434740 1 --2- 172.21.15.42:0/4158969171 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5611af42d2f0 0x5611af7fc7f0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f148810d9d0 tx=0x7f148810de90 comp rx=0 tx=0).stop 2024-08-30T09:14:51.313 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.310+0000 7f148f7fe640 1 --2- 172.21.15.42:0/3579807485 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f14780b4e00 0x7f14780b7310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:14:51.313 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.310+0000 7f149c434740 1 -- 172.21.15.42:0/4158969171 shutdown_connections 2024-08-30T09:14:51.313 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.310+0000 7f149c434740 1 --2- 172.21.15.42:0/4158969171 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5611af4354d0 0x5611af7fd340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:51.313 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.310+0000 7f149c434740 1 --2- 172.21.15.42:0/4158969171 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5611af42d2f0 0x5611af7fc7f0 secure :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f148810d9d0 tx=0x7f148810de90 comp rx=0 tx=0).stop 2024-08-30T09:14:51.313 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.310+0000 7f149c434740 1 --2- 172.21.15.42:0/4158969171 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5611af4561b0 0x5611af7fc180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:51.313 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.310+0000 7f149c434740 1 -- 172.21.15.42:0/4158969171 >> 172.21.15.42:0/4158969171 conn(0x5611af44a300 msgr2=0x5611af7f7c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:14:51.316 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.314+0000 7f149c434740 1 -- 172.21.15.42:0/4158969171 shutdown_connections 2024-08-30T09:14:51.316 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.314+0000 7f148f7fe640 1 --2- 172.21.15.42:0/3579807485 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f14780b4e00 0x7f14780b7310 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f1484030380 tx=0x7f14840304f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:14:51.317 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.315+0000 7f149c434740 1 -- 172.21.15.42:0/4158969171 wait complete. 2024-08-30T09:14:51.317 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.315+0000 7f149c434740 1 -- start start 2024-08-30T09:14:51.319 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.315+0000 7f149c434740 1 Processor -- start 2024-08-30T09:14:51.319 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.317+0000 7f149c434740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5611af4354d0 0x5611af805d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:14:51.319 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.317+0000 7f149c434740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5611af4561b0 0x5611af8063e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:14:51.319 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.317+0000 7f149c434740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5611af414d80 0x5611af806ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:14:51.319 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.317+0000 7f149c434740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x5611af697ab0 con 0x5611af414d80 2024-08-30T09:14:51.319 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.317+0000 7f149c434740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5611af6e5680 con 0x5611af4561b0 2024-08-30T09:14:51.319 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.317+0000 7f149c434740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5611af7fdeb0 con 0x5611af4354d0 2024-08-30T09:14:51.319 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.317+0000 7f149c434740 10 client.0 _ll_register_callbacks cb 0x5611af46da00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:14:51.320 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.318+0000 7f148e7fc640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5611af4561b0 0x5611af8063e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:14:51.320 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.318+0000 7f148f7fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5611af414d80 0x5611af806ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:14:51.320 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.318+0000 7f148f7fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5611af414d80 0x5611af806ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:46572/0 (socket says 172.21.15.42:46572) 2024-08-30T09:14:51.320 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.318+0000 7f148f7fe640 1 -- 172.21.15.42:0/350958928 learned_addr learned my addr 172.21.15.42:0/350958928 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:14:51.321 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.319+0000 7f148f7fe640 1 -- 172.21.15.42:0/350958928 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5611af4354d0 msgr2=0x5611af805d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:14:51.321 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.319+0000 7f148f7fe640 1 --2- 172.21.15.42:0/350958928 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5611af4354d0 0x5611af805d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:51.321 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.319+0000 7f148f7fe640 1 -- 172.21.15.42:0/350958928 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5611af4561b0 msgr2=0x5611af8063e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:14:51.321 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.319+0000 7f148f7fe640 1 --2- 172.21.15.42:0/350958928 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5611af4561b0 0x5611af8063e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:51.321 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.319+0000 7f148f7fe640 1 -- 172.21.15.42:0/350958928 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5611af53a620 con 0x5611af414d80 2024-08-30T09:14:51.321 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.319+0000 7f148f7fe640 1 --2- 172.21.15.42:0/350958928 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5611af414d80 0x5611af806ee0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f1484034000 tx=0x7f148402a360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=975d0ef55f905a45 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:14:51.322 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.321+0000 7f1472ff5640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-08-30T09:14:51.323 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.321+0000 7f1472ff5640 1 -- 172.21.15.42:0/350958928 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f148404c020 con 0x5611af414d80 2024-08-30T09:14:51.323 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.321+0000 7f149c434740 1 -- 172.21.15.42:0/350958928 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f14880f4070 con 0x5611af414d80 2024-08-30T09:14:51.323 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.321+0000 7f1472ff5640 1 -- 172.21.15.42:0/350958928 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f1484003d70 con 0x5611af414d80 2024-08-30T09:14:51.327 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.325+0000 7f1472ff5640 1 -- 172.21.15.42:0/350958928 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f1484057050 con 0x5611af414d80 2024-08-30T09:14:51.329 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.327+0000 7f149c434740 10 client.17763 fetch_fsmap learned FSMap version 47 2024-08-30T09:14:51.330 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.327+0000 7f149c434740 1 -- 172.21.15.42:0/350958928 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x5611af80fed0 con 0x5611af414d80 2024-08-30T09:14:51.333 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.331+0000 7f1472ff5640 1 -- 172.21.15.42:0/350958928 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14880f4070 con 0x5611af414d80 2024-08-30T09:14:51.333 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.331+0000 7f1472ff5640 1 -- 172.21.15.42:0/350958928 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 47) ==== 8370+0+0 (secure 0 0 0) 0x7f148403d410 con 0x5611af414d80 2024-08-30T09:14:51.336 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.331+0000 7f149c434740 10 client.17763 fetch_fsmap finished waiting for FSMap version 47 2024-08-30T09:14:51.336 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.331+0000 7f149c434740 10 client.17763 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-30T09:14:51.336 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.331+0000 7f149c434740 20 client.17763 populate_metadata read hostname 'smithi042' 2024-08-30T09:14:51.337 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.331+0000 7f149c434740 1 --2- 172.21.15.42:0/350958928 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x5611af3ea1c0 0x5611af8136f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:14:51.337 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.331+0000 7f149c434740 4 client.17763 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-30T09:14:51.337 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.331+0000 7f149c434740 1 -- 172.21.15.42:0/350958928 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5611af770f50 con 0x5611af3ea1c0 2024-08-30T09:14:51.337 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.332+0000 7f148effd640 1 --2- 172.21.15.42:0/350958928 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x5611af3ea1c0 0x5611af8136f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:14:51.338 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.336+0000 7f148effd640 1 --2- 172.21.15.42:0/350958928 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x5611af3ea1c0 0x5611af8136f0 crc :-1 s=READY pgs=287 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=605e9d9dc602d31f server_cookie=1640715ee8a4baf0 in_seq=0 out_seq=0 2024-08-30T09:14:51.338 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.336+0000 7f1472ff5640 10 client.17763 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:14:51.339 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.337+0000 7f1472ff5640 1 -- 172.21.15.42:0/350958928 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5611af770f50 con 0x5611af3ea1c0 2024-08-30T09:14:51.339 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.337+0000 7f1472ff5640 10 client.17763 handle_command_reply: tid=0 2024-08-30T09:14:51.339 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.337+0000 7f149c434740 1 client.17763 shutdown 2024-08-30T09:14:51.342 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.341+0000 7f149c434740 1 -- 172.21.15.42:0/350958928 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5611af414d80 msgr2=0x5611af806ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:14:51.342 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.341+0000 7f149c434740 1 --2- 172.21.15.42:0/350958928 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5611af414d80 0x5611af806ee0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f1484034000 tx=0x7f148402a360 comp rx=0 tx=0).stop 2024-08-30T09:14:51.343 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.341+0000 7f149c434740 1 -- 172.21.15.42:0/350958928 shutdown_connections 2024-08-30T09:14:51.343 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.341+0000 7f149c434740 1 --2- 172.21.15.42:0/350958928 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x5611af3ea1c0 0x5611af8136f0 crc :-1 s=READY pgs=287 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:51.343 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.341+0000 7f149c434740 1 --2- 172.21.15.42:0/350958928 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5611af414d80 0x5611af806ee0 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:51.343 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.341+0000 7f149c434740 1 --2- 172.21.15.42:0/350958928 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5611af4561b0 0x5611af8063e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:51.343 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.341+0000 7f149c434740 1 --2- 172.21.15.42:0/350958928 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5611af4354d0 0x5611af805d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:51.343 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.341+0000 7f149c434740 1 -- 172.21.15.42:0/350958928 >> 172.21.15.42:0/350958928 conn(0x5611af44a300 msgr2=0x5611af7f8810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:14:51.344 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.341+0000 7f1472ff5640 0 client.17763 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:14:51.344 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.341+0000 7f149c434740 1 -- 172.21.15.42:0/350958928 shutdown_connections 2024-08-30T09:14:51.344 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.342+0000 7f149c434740 1 -- 172.21.15.42:0/350958928 wait complete. 2024-08-30T09:14:51.344 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.343+0000 7f149c434740 20 client.17763 trim_cache size 0 max 16384 2024-08-30T09:14:51.382 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.375+0000 7f149c434740 1 Processor -- start 2024-08-30T09:14:51.382 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.375+0000 7f149c434740 1 -- start start 2024-08-30T09:14:51.382 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.375+0000 7f149c434740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5611af414d80 0x5611af7fbd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:14:51.382 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.375+0000 7f149c434740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5611af4354d0 0x5611af804440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:14:51.383 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.376+0000 7f149c434740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5611af4561b0 0x5611af804980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:14:51.383 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.376+0000 7f148effd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5611af414d80 0x5611af7fbd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:14:51.383 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.376+0000 7f148effd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5611af414d80 0x5611af7fbd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:58688/0 (socket says 172.21.15.42:58688) 2024-08-30T09:14:51.383 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.376+0000 7f148effd640 1 -- 172.21.15.42:0/2390274179 learned_addr learned my addr 172.21.15.42:0/2390274179 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:14:51.383 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.376+0000 7f149c434740 1 -- 172.21.15.42:0/2390274179 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f147c012600 con 0x5611af4354d0 2024-08-30T09:14:51.383 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.376+0000 7f149c434740 1 -- 172.21.15.42:0/2390274179 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f147c00fe70 con 0x5611af4561b0 2024-08-30T09:14:51.383 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.376+0000 7f149c434740 1 -- 172.21.15.42:0/2390274179 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f147c00dcc0 con 0x5611af414d80 2024-08-30T09:14:51.383 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.377+0000 7f148f7fe640 1 --2- 172.21.15.42:0/2390274179 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5611af4561b0 0x5611af804980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:14:51.383 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.379+0000 7f148e7fc640 1 --2- 172.21.15.42:0/2390274179 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5611af4354d0 0x5611af804440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:14:51.387 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.381+0000 7f148f7fe640 1 -- 172.21.15.42:0/2390274179 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5611af414d80 msgr2=0x5611af7fbd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:14:51.387 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.381+0000 7f148f7fe640 1 --2- 172.21.15.42:0/2390274179 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5611af414d80 0x5611af7fbd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:51.387 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.381+0000 7f148f7fe640 1 -- 172.21.15.42:0/2390274179 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5611af4354d0 msgr2=0x5611af804440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:14:51.387 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.381+0000 7f148f7fe640 1 --2- 172.21.15.42:0/2390274179 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5611af4354d0 0x5611af804440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:51.387 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.381+0000 7f148f7fe640 1 -- 172.21.15.42:0/2390274179 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5611af80fcf0 con 0x5611af4561b0 2024-08-30T09:14:51.388 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.381+0000 7f148effd640 1 --2- 172.21.15.42:0/2390274179 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5611af414d80 0x5611af7fbd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-30T09:14:51.388 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.381+0000 7f148f7fe640 1 --2- 172.21.15.42:0/2390274179 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5611af4561b0 0x5611af804980 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f14840039c0 tx=0x7f14840096b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=12a339e3e9cebcad server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:14:51.388 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.383+0000 7f1474ff9640 1 -- 172.21.15.42:0/2390274179 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f148404c070 con 0x5611af4561b0 2024-08-30T09:14:51.388 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.386+0000 7f1474ff9640 1 -- 172.21.15.42:0/2390274179 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1484043070 con 0x5611af4561b0 2024-08-30T09:14:51.388 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.386+0000 7f1474ff9640 1 -- 172.21.15.42:0/2390274179 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1484037040 con 0x5611af4561b0 2024-08-30T09:14:51.389 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.388+0000 7f149c434740 1 -- 172.21.15.42:0/2390274179 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5611af4561b0 msgr2=0x5611af804980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:14:51.389 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.388+0000 7f149c434740 1 --2- 172.21.15.42:0/2390274179 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5611af4561b0 0x5611af804980 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f14840039c0 tx=0x7f14840096b0 comp rx=0 tx=0).stop 2024-08-30T09:14:51.392 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.390+0000 7f149c434740 1 -- 172.21.15.42:0/2390274179 shutdown_connections 2024-08-30T09:14:51.392 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.390+0000 7f149c434740 1 --2- 172.21.15.42:0/2390274179 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5611af4561b0 0x5611af804980 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:51.392 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.390+0000 7f149c434740 1 --2- 172.21.15.42:0/2390274179 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5611af4354d0 0x5611af804440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:51.392 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.390+0000 7f149c434740 1 --2- 172.21.15.42:0/2390274179 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5611af414d80 0x5611af7fbd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:51.392 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.390+0000 7f149c434740 1 -- 172.21.15.42:0/2390274179 >> 172.21.15.42:0/2390274179 conn(0x5611af44a300 msgr2=0x5611af804f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:14:51.392 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.391+0000 7f149c434740 1 -- 172.21.15.42:0/2390274179 shutdown_connections 2024-08-30T09:14:51.393 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.391+0000 7f149c434740 1 -- 172.21.15.42:0/2390274179 wait complete. 2024-08-30T09:14:51.393 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.391+0000 7f149c434740 1 -- start start 2024-08-30T09:14:51.393 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.391+0000 7f149c434740 1 Processor -- start 2024-08-30T09:14:51.394 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.392+0000 7f149c434740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5611af414d80 0x5611af808630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:14:51.394 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.392+0000 7f149c434740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5611af4354d0 0x5611af808cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:14:51.394 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.392+0000 7f148effd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5611af414d80 0x5611af808630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:14:51.394 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.392+0000 7f148effd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5611af414d80 0x5611af808630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:40244/0 (socket says 172.21.15.42:40244) 2024-08-30T09:14:51.394 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.392+0000 7f148effd640 1 -- 172.21.15.42:0/194346543 learned_addr learned my addr 172.21.15.42:0/194346543 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:14:51.395 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.392+0000 7f148e7fc640 1 --2- 172.21.15.42:0/194346543 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5611af4354d0 0x5611af808cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:14:51.395 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.393+0000 7f149c434740 1 --2- 172.21.15.42:0/194346543 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5611af4561b0 0x5611af809860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:14:51.396 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.393+0000 7f149c434740 1 -- 172.21.15.42:0/194346543 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f147c00e330 con 0x5611af4561b0 2024-08-30T09:14:51.396 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.393+0000 7f149c434740 1 -- 172.21.15.42:0/194346543 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f147c00f7a0 con 0x5611af414d80 2024-08-30T09:14:51.396 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.393+0000 7f149c434740 1 -- 172.21.15.42:0/194346543 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f147c00ea00 con 0x5611af4354d0 2024-08-30T09:14:51.396 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.393+0000 7f148effd640 1 -- 172.21.15.42:0/194346543 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5611af4354d0 msgr2=0x5611af808cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:14:51.396 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.393+0000 7f148effd640 1 --2- 172.21.15.42:0/194346543 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5611af4354d0 0x5611af808cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:51.396 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.393+0000 7f148effd640 1 -- 172.21.15.42:0/194346543 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5611af4561b0 msgr2=0x5611af809860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:14:51.396 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.393+0000 7f148effd640 1 --2- 172.21.15.42:0/194346543 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5611af4561b0 0x5611af809860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:51.396 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.393+0000 7f148effd640 1 -- 172.21.15.42:0/194346543 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5611af809fc0 con 0x5611af414d80 2024-08-30T09:14:51.397 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.394+0000 7f148effd640 1 --2- 172.21.15.42:0/194346543 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5611af414d80 0x5611af808630 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f148800cfd0 tx=0x7f14880d2820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b2f7076ea4387634 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:14:51.397 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.394+0000 7f1472ff5640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-30T09:14:51.397 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.394+0000 7f1472ff5640 1 -- 172.21.15.42:0/194346543 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14880cd900 con 0x5611af414d80 2024-08-30T09:14:51.397 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.394+0000 7f1472ff5640 1 -- 172.21.15.42:0/194346543 <== mon.2 v2:172.21.15.167:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f148810d220 con 0x5611af414d80 2024-08-30T09:14:51.397 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.394+0000 7f149c434740 10 client.0 _ll_register_callbacks cb 0x7f1488009460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:14:51.397 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.394+0000 7f149c434740 1 -- 172.21.15.42:0/194346543 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1484043070 con 0x5611af414d80 2024-08-30T09:14:51.398 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.395+0000 7f1472ff5640 1 -- 172.21.15.42:0/194346543 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f14880cdd80 con 0x5611af414d80 2024-08-30T09:14:51.398 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.395+0000 7f149c434740 10 client.27611 fetch_fsmap learned FSMap version 47 2024-08-30T09:14:51.399 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.395+0000 7f149c434740 1 -- 172.21.15.42:0/194346543 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({fsmap=47}) -- 0x5611af80d280 con 0x5611af414d80 2024-08-30T09:14:51.399 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.396+0000 7f1472ff5640 1 -- 172.21.15.42:0/194346543 <== mon.2 v2:172.21.15.167:3300/0 4 ==== fsmap(e 47) ==== 8370+0+0 (secure 0 0 0) 0x7f148800b040 con 0x5611af414d80 2024-08-30T09:14:51.399 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.396+0000 7f149c434740 10 client.27611 fetch_fsmap finished waiting for FSMap version 47 2024-08-30T09:14:51.399 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.396+0000 7f149c434740 10 client.27611 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-30T09:14:51.399 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.396+0000 7f149c434740 20 client.27611 populate_metadata read hostname 'smithi042' 2024-08-30T09:14:51.399 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.396+0000 7f149c434740 1 --2- 172.21.15.42:0/194346543 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x5611af87e0f0 0x5611af880510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:14:51.399 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.397+0000 7f149c434740 4 client.27611 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-30T09:14:51.399 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.397+0000 7f149c434740 1 -- 172.21.15.42:0/194346543 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5611af76c9f0 con 0x5611af87e0f0 2024-08-30T09:14:51.399 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.397+0000 7f148e7fc640 1 --2- 172.21.15.42:0/194346543 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x5611af87e0f0 0x5611af880510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:14:51.401 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.400+0000 7f148e7fc640 1 --2- 172.21.15.42:0/194346543 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x5611af87e0f0 0x5611af880510 crc :-1 s=READY pgs=288 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=36daa3db3d0d6b84 server_cookie=54e10726afb663f0 in_seq=0 out_seq=0 2024-08-30T09:14:51.401 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.400+0000 7f1472ff5640 10 client.27611 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:14:51.426 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.424+0000 7f1472ff5640 1 -- 172.21.15.42:0/194346543 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5611af76c9f0 con 0x5611af87e0f0 2024-08-30T09:14:51.426 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.424+0000 7f1472ff5640 10 client.27611 handle_command_reply: tid=0 2024-08-30T09:14:51.426 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.424+0000 7f149c434740 1 client.27611 shutdown 2024-08-30T09:14:51.428 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.425+0000 7f149c434740 1 -- 172.21.15.42:0/194346543 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5611af414d80 msgr2=0x5611af808630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:14:51.428 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.426+0000 7f149c434740 1 --2- 172.21.15.42:0/194346543 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5611af414d80 0x5611af808630 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f148800cfd0 tx=0x7f14880d2820 comp rx=0 tx=0).stop 2024-08-30T09:14:51.428 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.426+0000 7f149c434740 1 -- 172.21.15.42:0/194346543 shutdown_connections 2024-08-30T09:14:51.428 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.426+0000 7f149c434740 1 --2- 172.21.15.42:0/194346543 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x5611af87e0f0 0x5611af880510 crc :-1 s=READY pgs=288 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:51.428 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.426+0000 7f149c434740 1 --2- 172.21.15.42:0/194346543 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5611af4354d0 0x5611af808cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:51.428 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.426+0000 7f149c434740 1 --2- 172.21.15.42:0/194346543 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5611af4561b0 0x5611af809860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:51.428 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.426+0000 7f149c434740 1 --2- 172.21.15.42:0/194346543 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5611af414d80 0x5611af808630 secure :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f148800cfd0 tx=0x7f14880d2820 comp rx=0 tx=0).stop 2024-08-30T09:14:51.428 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.426+0000 7f149c434740 1 -- 172.21.15.42:0/194346543 >> 172.21.15.42:0/194346543 conn(0x5611af44a300 msgr2=0x5611af7fbd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:14:51.428 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.426+0000 7f1472ff5640 0 client.27611 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:14:51.428 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.426+0000 7f149c434740 1 -- 172.21.15.42:0/194346543 shutdown_connections 2024-08-30T09:14:51.429 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.426+0000 7f149c434740 1 -- 172.21.15.42:0/194346543 wait complete. 2024-08-30T09:14:51.429 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.427+0000 7f149c434740 20 client.27611 trim_cache size 0 max 16384 2024-08-30T09:14:51.429 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-08-30T09:14:51.429 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2024-08-30T09:14:51.429 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-08-30T09:14:51.430 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-08-30T09:14:51.431 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.429+0000 7f14954ce640 1 -- 172.21.15.42:0/3579807485 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f14780b4e00 msgr2=0x7f14780b7310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:14:51.431 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.429+0000 7f14954ce640 1 --2- 172.21.15.42:0/3579807485 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f14780b4e00 0x7f14780b7310 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f1484030380 tx=0x7f14840304f0 comp rx=0 tx=0).stop 2024-08-30T09:14:51.431 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.429+0000 7f14954ce640 1 -- 172.21.15.42:0/3579807485 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f149010a160 msgr2=0x7f149011a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:14:51.431 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.429+0000 7f14954ce640 1 --2- 172.21.15.42:0/3579807485 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f149010a160 0x7f149011a300 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f1488009fd0 tx=0x7f148800c800 comp rx=0 tx=0).stop 2024-08-30T09:14:51.431 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.429+0000 7f14954ce640 1 -- 172.21.15.42:0/3579807485 shutdown_connections 2024-08-30T09:14:51.431 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.429+0000 7f14954ce640 1 --2- 172.21.15.42:0/3579807485 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f14780b4e00 0x7f14780b7310 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:51.431 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.429+0000 7f14954ce640 1 --2- 172.21.15.42:0/3579807485 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f149010cb50 0x7f1490118bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:51.432 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.429+0000 7f14954ce640 1 --2- 172.21.15.42:0/3579807485 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f149010f770 0x7f1490119100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:51.432 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.429+0000 7f14954ce640 1 --2- 172.21.15.42:0/3579807485 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f149010a160 0x7f149011a300 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:14:51.432 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.429+0000 7f14954ce640 1 -- 172.21.15.42:0/3579807485 >> 172.21.15.42:0/3579807485 conn(0x7f14900705b0 msgr2=0x7f149010b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:14:51.432 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.429+0000 7f14954ce640 1 -- 172.21.15.42:0/3579807485 shutdown_connections 2024-08-30T09:14:51.432 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:14:51.429+0000 7f14954ce640 1 -- 172.21.15.42:0/3579807485 wait complete. 2024-08-30T09:14:53.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:14:52 smithi167 ceph-mon[22063]: pgmap v796: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 3.7 MiB/s wr, 426 op/s 2024-08-30T09:14:53.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:14:52 smithi042 ceph-mon[18652]: pgmap v796: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 3.7 MiB/s wr, 426 op/s 2024-08-30T09:14:53.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:14:52 smithi190 ceph-mon[22001]: pgmap v796: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 3.7 MiB/s wr, 426 op/s 2024-08-30T09:14:55.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:14:54 smithi167 ceph-mon[22063]: pgmap v797: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 4.3 MiB/s wr, 494 op/s 2024-08-30T09:14:55.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:14:54 smithi042 ceph-mon[18652]: pgmap v797: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 4.3 MiB/s wr, 494 op/s 2024-08-30T09:14:55.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:14:54 smithi190 ceph-mon[22001]: pgmap v797: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 4.3 MiB/s wr, 494 op/s 2024-08-30T09:14:55.953 INFO:teuthology.orchestra.run.smithi042.stderr:progress: 850.0 s, 447.2 tps, lat 217.347 ms stddev 257.491 2024-08-30T09:14:57.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:14:56 smithi167 ceph-mon[22063]: pgmap v798: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 4.3 MiB/s wr, 465 op/s 2024-08-30T09:14:57.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:14:56 smithi042 ceph-mon[18652]: pgmap v798: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 4.3 MiB/s wr, 465 op/s 2024-08-30T09:14:57.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:14:56 smithi190 ceph-mon[22001]: pgmap v798: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 4.3 MiB/s wr, 465 op/s 2024-08-30T09:14:59.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:14:58 smithi167 ceph-mon[22063]: pgmap v799: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 3.7 MiB/s wr, 419 op/s 2024-08-30T09:14:59.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:14:58 smithi042 ceph-mon[18652]: pgmap v799: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 3.7 MiB/s wr, 419 op/s 2024-08-30T09:14:59.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:14:58 smithi190 ceph-mon[22001]: pgmap v799: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 3.7 MiB/s wr, 419 op/s 2024-08-30T09:15:00.953 INFO:teuthology.orchestra.run.smithi042.stderr:progress: 855.0 s, 470.0 tps, lat 212.933 ms stddev 264.706 2024-08-30T09:15:01.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:15:00 smithi167 ceph-mon[22063]: pgmap v800: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 4.5 MiB/s wr, 521 op/s 2024-08-30T09:15:01.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:15:00 smithi042 ceph-mon[18652]: pgmap v800: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 4.5 MiB/s wr, 521 op/s 2024-08-30T09:15:01.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:15:00 smithi190 ceph-mon[22001]: pgmap v800: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 4.5 MiB/s wr, 521 op/s 2024-08-30T09:15:03.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:15:02 smithi167 ceph-mon[22063]: pgmap v801: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 3.7 MiB/s wr, 423 op/s 2024-08-30T09:15:03.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:15:02 smithi042 ceph-mon[18652]: pgmap v801: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 3.7 MiB/s wr, 423 op/s 2024-08-30T09:15:03.188 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:15:02 smithi190 ceph-mon[22001]: pgmap v801: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 3.7 MiB/s wr, 423 op/s 2024-08-30T09:15:04.589 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.585+0000 7f7881884640 1 -- 172.21.15.42:0/571713868 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f787c102d50 msgr2=0x7f787c107800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:15:04.589 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.585+0000 7f7881884640 1 --2- 172.21.15.42:0/571713868 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f787c102d50 0x7f787c107800 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f787401acb0 tx=0x7f787403b920 comp rx=0 tx=0).stop 2024-08-30T09:15:04.589 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.586+0000 7f7881884640 1 -- 172.21.15.42:0/571713868 shutdown_connections 2024-08-30T09:15:04.590 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.586+0000 7f7881884640 1 --2- 172.21.15.42:0/571713868 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f787c102d50 0x7f787c107800 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:04.590 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.586+0000 7f7881884640 1 --2- 172.21.15.42:0/571713868 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f787c1021a0 0x7f787c102610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:04.590 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.586+0000 7f7881884640 1 --2- 172.21.15.42:0/571713868 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f787c101770 0x7f787c101b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:04.590 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.586+0000 7f7881884640 1 -- 172.21.15.42:0/571713868 >> 172.21.15.42:0/571713868 conn(0x7f787c0fd430 msgr2=0x7f787c0ff8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:15:04.596 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.589+0000 7f7881884640 1 -- 172.21.15.42:0/571713868 shutdown_connections 2024-08-30T09:15:04.596 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.590+0000 7f7881884640 1 -- 172.21.15.42:0/571713868 wait complete. 2024-08-30T09:15:04.596 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.591+0000 7f7881884640 1 Processor -- start 2024-08-30T09:15:04.602 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.593+0000 7f7881884640 1 -- start start 2024-08-30T09:15:04.602 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.593+0000 7f7881884640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f787c101770 0x7f787c074030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:04.602 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.593+0000 7f7881884640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f787c1021a0 0x7f787c0745d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:04.602 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.593+0000 7f7881884640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f787c102d50 0x7f787c1a7680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:04.602 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.593+0000 7f7881884640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f787c10af60 con 0x7f787c101770 2024-08-30T09:15:04.602 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.593+0000 7f7881884640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f787c10ade0 con 0x7f787c1021a0 2024-08-30T09:15:04.602 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.593+0000 7f7881884640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f787c10b0e0 con 0x7f787c102d50 2024-08-30T09:15:04.602 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.594+0000 7f787a7fc640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f787c1021a0 0x7f787c0745d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:15:04.602 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.594+0000 7f787a7fc640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f787c1021a0 0x7f787c0745d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:40510/0 (socket says 172.21.15.42:40510) 2024-08-30T09:15:04.602 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.594+0000 7f787a7fc640 1 -- 172.21.15.42:0/1455786413 learned_addr learned my addr 172.21.15.42:0/1455786413 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:15:04.602 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.594+0000 7f787b7fe640 1 --2- 172.21.15.42:0/1455786413 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f787c102d50 0x7f787c1a7680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:15:04.603 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.594+0000 7f787affd640 1 --2- 172.21.15.42:0/1455786413 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f787c101770 0x7f787c074030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:15:04.603 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.595+0000 7f787affd640 1 -- 172.21.15.42:0/1455786413 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f787c102d50 msgr2=0x7f787c1a7680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:15:04.603 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.595+0000 7f787affd640 1 --2- 172.21.15.42:0/1455786413 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f787c102d50 0x7f787c1a7680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:04.603 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.595+0000 7f787affd640 1 -- 172.21.15.42:0/1455786413 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f787c1021a0 msgr2=0x7f787c0745d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:15:04.603 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.595+0000 7f787affd640 1 --2- 172.21.15.42:0/1455786413 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f787c1021a0 0x7f787c0745d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:04.603 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.595+0000 7f787affd640 1 -- 172.21.15.42:0/1455786413 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f787c1a7e00 con 0x7f787c101770 2024-08-30T09:15:04.603 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.595+0000 7f787affd640 1 --2- 172.21.15.42:0/1455786413 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f787c101770 0x7f787c074030 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f786c030a00 tx=0x7f786c030ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:15:04.607 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.596+0000 7f7880882640 1 -- 172.21.15.42:0/1455786413 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f786c00ccc0 con 0x7f787c101770 2024-08-30T09:15:04.607 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.596+0000 7f7881884640 1 -- 172.21.15.42:0/1455786413 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f787c1a80f0 con 0x7f787c101770 2024-08-30T09:15:04.607 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.596+0000 7f7881884640 1 -- 172.21.15.42:0/1455786413 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f787c1a8640 con 0x7f787c101770 2024-08-30T09:15:04.608 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.597+0000 7f7880882640 1 -- 172.21.15.42:0/1455786413 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f786c0328f0 con 0x7f787c101770 2024-08-30T09:15:04.608 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.597+0000 7f7880882640 1 -- 172.21.15.42:0/1455786413 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f786c03a8e0 con 0x7f787c101770 2024-08-30T09:15:04.611 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.608+0000 7f7880882640 1 -- 172.21.15.42:0/1455786413 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7f786c03aa80 con 0x7f787c101770 2024-08-30T09:15:04.611 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.608+0000 7f78887ee740 1 Processor -- start 2024-08-30T09:15:04.611 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.609+0000 7f78887ee740 1 -- start start 2024-08-30T09:15:04.614 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.609+0000 7f78887ee740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55776e7a41b0 0x55776eb49f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:04.614 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.609+0000 7f78887ee740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55776e77b2f0 0x55776eb4a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:04.614 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.609+0000 7f78887ee740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55776e7834d0 0x55776eb4b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:04.614 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.609+0000 7f78887ee740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55776eabd000 con 0x55776e77b2f0 2024-08-30T09:15:04.614 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.609+0000 7f78887ee740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55776eaac590 con 0x55776e7834d0 2024-08-30T09:15:04.614 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.609+0000 7f78887ee740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55776eab0540 con 0x55776e7a41b0 2024-08-30T09:15:04.614 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.610+0000 7f787affd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55776e77b2f0 0x55776eb4a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:15:04.614 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.610+0000 7f787affd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55776e77b2f0 0x55776eb4a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:44952/0 (socket says 172.21.15.42:44952) 2024-08-30T09:15:04.614 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.610+0000 7f7880882640 1 --2- 172.21.15.42:0/1455786413 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f78640b4d00 0x7f78640b7210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:04.614 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.610+0000 7f787affd640 1 -- 172.21.15.42:0/4033250270 learned_addr learned my addr 172.21.15.42:0/4033250270 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:15:04.614 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.610+0000 7f787b7fe640 1 --2- 172.21.15.42:0/4033250270 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55776e7a41b0 0x55776eb49f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:15:04.614 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.610+0000 7f7880882640 1 -- 172.21.15.42:0/1455786413 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f786c1086a0 con 0x7f787c101770 2024-08-30T09:15:04.615 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.610+0000 7f787b7fe640 1 --2- 172.21.15.42:0/1455786413 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f78640b4d00 0x7f78640b7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:15:04.615 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.611+0000 7f787a7fc640 1 --2- 172.21.15.42:0/4033250270 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55776e7834d0 0x55776eb4b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:15:04.615 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.611+0000 7f787b7fe640 1 -- 172.21.15.42:0/4033250270 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55776e7834d0 msgr2=0x55776eb4b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:15:04.615 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.611+0000 7f787b7fe640 1 --2- 172.21.15.42:0/4033250270 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55776e7834d0 0x55776eb4b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:04.615 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.611+0000 7f787b7fe640 1 -- 172.21.15.42:0/4033250270 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55776e77b2f0 msgr2=0x55776eb4a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:15:04.615 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.611+0000 7f787b7fe640 1 --2- 172.21.15.42:0/4033250270 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55776e77b2f0 0x55776eb4a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:04.615 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.611+0000 7f787b7fe640 1 -- 172.21.15.42:0/4033250270 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55776e80f790 con 0x55776e7a41b0 2024-08-30T09:15:04.615 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.611+0000 7f787affd640 1 --2- 172.21.15.42:0/4033250270 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55776e77b2f0 0x55776eb4a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-30T09:15:04.615 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.611+0000 7f787b7fe640 1 --2- 172.21.15.42:0/4033250270 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55776e7a41b0 0x55776eb49f60 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f7874006010 tx=0x7f7874003cc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7fc9e8a5426bbbaa server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:15:04.615 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.611+0000 7f7861ffb640 1 -- 172.21.15.42:0/4033250270 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7874050070 con 0x55776e7a41b0 2024-08-30T09:15:04.615 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.612+0000 7f787b7fe640 1 --2- 172.21.15.42:0/1455786413 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f78640b4d00 0x7f78640b7210 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f7874003fd0 tx=0x7f7874015000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:15:04.615 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.612+0000 7f7861ffb640 1 -- 172.21.15.42:0/4033250270 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7874040e70 con 0x55776e7a41b0 2024-08-30T09:15:04.615 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.612+0000 7f7861ffb640 1 -- 172.21.15.42:0/4033250270 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7874045040 con 0x55776e7a41b0 2024-08-30T09:15:04.616 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.613+0000 7f78887ee740 1 -- 172.21.15.42:0/4033250270 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55776e7a41b0 msgr2=0x55776eb49f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:15:04.617 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.613+0000 7f78887ee740 1 --2- 172.21.15.42:0/4033250270 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55776e7a41b0 0x55776eb49f60 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f7874006010 tx=0x7f7874003cc0 comp rx=0 tx=0).stop 2024-08-30T09:15:04.617 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.613+0000 7f78887ee740 1 -- 172.21.15.42:0/4033250270 shutdown_connections 2024-08-30T09:15:04.617 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.613+0000 7f78887ee740 1 --2- 172.21.15.42:0/4033250270 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55776e7834d0 0x55776eb4b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:04.617 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.613+0000 7f78887ee740 1 --2- 172.21.15.42:0/4033250270 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55776e77b2f0 0x55776eb4a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:04.617 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.613+0000 7f78887ee740 1 --2- 172.21.15.42:0/4033250270 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55776e7a41b0 0x55776eb49f60 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:04.617 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.613+0000 7f78887ee740 1 -- 172.21.15.42:0/4033250270 >> 172.21.15.42:0/4033250270 conn(0x55776e798300 msgr2=0x55776eb459f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:15:04.617 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.614+0000 7f78887ee740 1 -- 172.21.15.42:0/4033250270 shutdown_connections 2024-08-30T09:15:04.618 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.617+0000 7f78887ee740 1 -- 172.21.15.42:0/4033250270 wait complete. 2024-08-30T09:15:04.619 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.617+0000 7f78887ee740 1 -- start start 2024-08-30T09:15:04.619 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.617+0000 7f78887ee740 1 Processor -- start 2024-08-30T09:15:04.624 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.620+0000 7f78887ee740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55776e77b2f0 0x55776eb539a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:04.625 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.620+0000 7f78887ee740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55776e7834d0 0x55776eb54060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:04.625 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.620+0000 7f78887ee740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55776e7a41b0 0x55776eb54bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:04.625 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.620+0000 7f78887ee740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55776e9e5ab0 con 0x55776e77b2f0 2024-08-30T09:15:04.625 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.620+0000 7f78887ee740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55776ea33680 con 0x55776e7834d0 2024-08-30T09:15:04.625 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.620+0000 7f78887ee740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55776eb4bbe0 con 0x55776e7a41b0 2024-08-30T09:15:04.625 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.620+0000 7f78887ee740 10 client.0 _ll_register_callbacks cb 0x55776e7bba00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:15:04.625 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.622+0000 7f787a7fc640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55776e7834d0 0x55776eb54060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:15:04.625 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.622+0000 7f787a7fc640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55776e7834d0 0x55776eb54060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:40532/0 (socket says 172.21.15.42:40532) 2024-08-30T09:15:04.625 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.622+0000 7f787a7fc640 1 -- 172.21.15.42:0/3054741647 learned_addr learned my addr 172.21.15.42:0/3054741647 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:15:04.627 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.623+0000 7f787b7fe640 1 --2- 172.21.15.42:0/3054741647 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55776e7a41b0 0x55776eb54bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:15:04.627 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.624+0000 7f787a7fc640 1 -- 172.21.15.42:0/3054741647 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55776e7a41b0 msgr2=0x55776eb54bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:15:04.627 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.624+0000 7f787a7fc640 1 --2- 172.21.15.42:0/3054741647 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55776e7a41b0 0x55776eb54bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:04.627 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.624+0000 7f787a7fc640 1 -- 172.21.15.42:0/3054741647 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55776e77b2f0 msgr2=0x55776eb539a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:15:04.627 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.624+0000 7f787a7fc640 1 --2- 172.21.15.42:0/3054741647 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55776e77b2f0 0x55776eb539a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:04.627 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.624+0000 7f787a7fc640 1 -- 172.21.15.42:0/3054741647 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55776e888620 con 0x55776e7834d0 2024-08-30T09:15:04.627 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.625+0000 7f787a7fc640 1 --2- 172.21.15.42:0/3054741647 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55776e7834d0 0x55776eb54060 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f7870011260 tx=0x7f7870011720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=26cc3fd4bbb4dc05 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:15:04.628 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.626+0000 7f785affd640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-30T09:15:04.628 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.626+0000 7f785affd640 1 -- 172.21.15.42:0/3054741647 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7870019070 con 0x55776e7834d0 2024-08-30T09:15:04.628 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.626+0000 7f785affd640 1 -- 172.21.15.42:0/3054741647 <== mon.2 v2:172.21.15.167:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f7870015870 con 0x55776e7834d0 2024-08-30T09:15:04.628 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.626+0000 7f78887ee740 1 -- 172.21.15.42:0/3054741647 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7874040e70 con 0x55776e7834d0 2024-08-30T09:15:04.631 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.629+0000 7f785affd640 1 -- 172.21.15.42:0/3054741647 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f7870023050 con 0x55776e7834d0 2024-08-30T09:15:04.631 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.629+0000 7f78887ee740 10 client.27620 fetch_fsmap learned FSMap version 47 2024-08-30T09:15:04.631 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.629+0000 7f78887ee740 1 -- 172.21.15.42:0/3054741647 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55776eb5dc20 con 0x55776e7834d0 2024-08-30T09:15:04.633 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.631+0000 7f785affd640 1 -- 172.21.15.42:0/3054741647 <== mon.2 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7874040e70 con 0x55776e7834d0 2024-08-30T09:15:04.633 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.631+0000 7f785affd640 1 -- 172.21.15.42:0/3054741647 <== mon.2 v2:172.21.15.167:3300/0 5 ==== fsmap(e 47) ==== 8370+0+0 (secure 0 0 0) 0x7f787001ebf0 con 0x55776e7834d0 2024-08-30T09:15:04.633 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.631+0000 7f78887ee740 10 client.27620 fetch_fsmap finished waiting for FSMap version 47 2024-08-30T09:15:04.633 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.631+0000 7f78887ee740 10 client.27620 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-30T09:15:04.633 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.631+0000 7f78887ee740 20 client.27620 populate_metadata read hostname 'smithi042' 2024-08-30T09:15:04.634 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.631+0000 7f78887ee740 1 --2- 172.21.15.42:0/3054741647 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55776e762d80 0x55776eb614c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:04.634 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.632+0000 7f78887ee740 4 client.27620 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-30T09:15:04.634 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.632+0000 7f78887ee740 1 -- 172.21.15.42:0/3054741647 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55776eabef50 con 0x55776e762d80 2024-08-30T09:15:04.634 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.632+0000 7f787affd640 1 --2- 172.21.15.42:0/3054741647 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55776e762d80 0x55776eb614c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:15:04.635 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.633+0000 7f787affd640 1 --2- 172.21.15.42:0/3054741647 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55776e762d80 0x55776eb614c0 crc :-1 s=READY pgs=289 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea7b33ea5331824b server_cookie=cda2046a591b29b4 in_seq=0 out_seq=0 2024-08-30T09:15:04.635 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.633+0000 7f785affd640 10 client.27620 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:15:04.636 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.634+0000 7f785affd640 1 -- 172.21.15.42:0/3054741647 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55776eabef50 con 0x55776e762d80 2024-08-30T09:15:04.636 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.634+0000 7f785affd640 10 client.27620 handle_command_reply: tid=0 2024-08-30T09:15:04.636 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.634+0000 7f78887ee740 1 client.27620 shutdown 2024-08-30T09:15:04.638 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.636+0000 7f78887ee740 1 -- 172.21.15.42:0/3054741647 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55776e7834d0 msgr2=0x55776eb54060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:15:04.638 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.636+0000 7f78887ee740 1 --2- 172.21.15.42:0/3054741647 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55776e7834d0 0x55776eb54060 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f7870011260 tx=0x7f7870011720 comp rx=0 tx=0).stop 2024-08-30T09:15:04.638 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.636+0000 7f78887ee740 1 -- 172.21.15.42:0/3054741647 shutdown_connections 2024-08-30T09:15:04.638 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.636+0000 7f78887ee740 1 --2- 172.21.15.42:0/3054741647 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55776e762d80 0x55776eb614c0 crc :-1 s=READY pgs=289 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:04.638 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.636+0000 7f78887ee740 1 --2- 172.21.15.42:0/3054741647 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55776e7a41b0 0x55776eb54bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:04.639 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.636+0000 7f78887ee740 1 --2- 172.21.15.42:0/3054741647 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55776e7834d0 0x55776eb54060 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:04.639 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.636+0000 7f78887ee740 1 --2- 172.21.15.42:0/3054741647 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55776e77b2f0 0x55776eb539a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:04.639 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.636+0000 7f78887ee740 1 -- 172.21.15.42:0/3054741647 >> 172.21.15.42:0/3054741647 conn(0x55776e798300 msgr2=0x55776eb43c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:15:04.639 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.636+0000 7f785affd640 0 client.27620 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:15:04.639 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.636+0000 7f78887ee740 1 -- 172.21.15.42:0/3054741647 shutdown_connections 2024-08-30T09:15:04.641 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.639+0000 7f78887ee740 1 -- 172.21.15.42:0/3054741647 wait complete. 2024-08-30T09:15:04.641 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.639+0000 7f78887ee740 20 client.27620 trim_cache size 0 max 16384 2024-08-30T09:15:04.661 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.658+0000 7f78887ee740 1 Processor -- start 2024-08-30T09:15:04.661 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.658+0000 7f78887ee740 1 -- start start 2024-08-30T09:15:04.661 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.659+0000 7f78887ee740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55776e77b2f0 0x55776eb540b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:04.661 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.659+0000 7f78887ee740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55776e7834d0 0x55776eb44640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:04.661 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.659+0000 7f78887ee740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55776e7a41b0 0x55776eb44b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:04.661 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.659+0000 7f78887ee740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f78680125b0 con 0x55776e77b2f0 2024-08-30T09:15:04.661 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.659+0000 7f78887ee740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f786800fe20 con 0x55776e7834d0 2024-08-30T09:15:04.661 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.659+0000 7f78887ee740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f786800dc70 con 0x55776e7a41b0 2024-08-30T09:15:04.662 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.659+0000 7f787affd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55776e77b2f0 0x55776eb540b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:15:04.662 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.659+0000 7f787affd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55776e77b2f0 0x55776eb540b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:44966/0 (socket says 172.21.15.42:44966) 2024-08-30T09:15:04.662 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.659+0000 7f787affd640 1 -- 172.21.15.42:0/198347848 learned_addr learned my addr 172.21.15.42:0/198347848 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:15:04.662 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.659+0000 7f787b7fe640 1 --2- 172.21.15.42:0/198347848 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55776e7a41b0 0x55776eb44b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:15:04.662 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.659+0000 7f787a7fc640 1 --2- 172.21.15.42:0/198347848 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55776e7834d0 0x55776eb44640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:15:04.662 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.659+0000 7f787affd640 1 -- 172.21.15.42:0/198347848 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55776e7a41b0 msgr2=0x55776eb44b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:15:04.662 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.659+0000 7f787affd640 1 --2- 172.21.15.42:0/198347848 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55776e7a41b0 0x55776eb44b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:04.662 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.659+0000 7f787affd640 1 -- 172.21.15.42:0/198347848 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55776e7834d0 msgr2=0x55776eb44640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:15:04.662 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.659+0000 7f787affd640 1 --2- 172.21.15.42:0/198347848 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55776e7834d0 0x55776eb44640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:04.662 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.659+0000 7f787affd640 1 -- 172.21.15.42:0/198347848 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55776eb4f040 con 0x55776e77b2f0 2024-08-30T09:15:04.663 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.659+0000 7f787affd640 1 --2- 172.21.15.42:0/198347848 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55776e77b2f0 0x55776eb540b0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f786c005f60 tx=0x7f786c036000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f76f07ffd7341ff1 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:15:04.663 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.660+0000 7f78617fa640 1 -- 172.21.15.42:0/198347848 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f786c105ec0 con 0x55776e77b2f0 2024-08-30T09:15:04.663 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.660+0000 7f78617fa640 1 -- 172.21.15.42:0/198347848 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f786c032aa0 con 0x55776e77b2f0 2024-08-30T09:15:04.663 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.660+0000 7f78617fa640 1 -- 172.21.15.42:0/198347848 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f786c0d3320 con 0x55776e77b2f0 2024-08-30T09:15:04.663 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.661+0000 7f78887ee740 1 -- 172.21.15.42:0/198347848 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55776e77b2f0 msgr2=0x55776eb540b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:15:04.663 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.661+0000 7f78887ee740 1 --2- 172.21.15.42:0/198347848 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55776e77b2f0 0x55776eb540b0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f786c005f60 tx=0x7f786c036000 comp rx=0 tx=0).stop 2024-08-30T09:15:04.664 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.661+0000 7f78887ee740 1 -- 172.21.15.42:0/198347848 shutdown_connections 2024-08-30T09:15:04.664 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.661+0000 7f78887ee740 1 --2- 172.21.15.42:0/198347848 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55776e7a41b0 0x55776eb44b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:04.664 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.661+0000 7f78887ee740 1 --2- 172.21.15.42:0/198347848 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55776e7834d0 0x55776eb44640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:04.664 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.661+0000 7f78887ee740 1 --2- 172.21.15.42:0/198347848 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55776e77b2f0 0x55776eb540b0 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:04.664 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.661+0000 7f78887ee740 1 -- 172.21.15.42:0/198347848 >> 172.21.15.42:0/198347848 conn(0x55776e798300 msgr2=0x55776eb539a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:15:04.664 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.661+0000 7f78887ee740 1 -- 172.21.15.42:0/198347848 shutdown_connections 2024-08-30T09:15:04.664 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.661+0000 7f78887ee740 1 -- 172.21.15.42:0/198347848 wait complete. 2024-08-30T09:15:04.667 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.661+0000 7f78887ee740 1 -- start start 2024-08-30T09:15:04.667 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.662+0000 7f78887ee740 1 Processor -- start 2024-08-30T09:15:04.667 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.662+0000 7f78887ee740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55776e77b2f0 0x55776eb5b920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:04.667 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.662+0000 7f78887ee740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55776e7834d0 0x55776eb5a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:04.668 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.662+0000 7f78887ee740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55776e7a41b0 0x55776eb5acd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:04.668 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.662+0000 7f78887ee740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f786800e2e0 con 0x55776e7a41b0 2024-08-30T09:15:04.668 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.662+0000 7f78887ee740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f786800f750 con 0x55776e7834d0 2024-08-30T09:15:04.668 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.662+0000 7f78887ee740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f786800e9b0 con 0x55776e77b2f0 2024-08-30T09:15:04.668 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.662+0000 7f78887ee740 10 client.0 _ll_register_callbacks cb 0x7f78700095a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:15:04.668 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.662+0000 7f787a7fc640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55776e7834d0 0x55776eb5a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:15:04.668 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.663+0000 7f787a7fc640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55776e7834d0 0x55776eb5a5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:40550/0 (socket says 172.21.15.42:40550) 2024-08-30T09:15:04.668 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.663+0000 7f787a7fc640 1 -- 172.21.15.42:0/3927759265 learned_addr learned my addr 172.21.15.42:0/3927759265 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:15:04.668 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.663+0000 7f787b7fe640 1 --2- 172.21.15.42:0/3927759265 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55776e7a41b0 0x55776eb5acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:15:04.668 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.663+0000 7f787affd640 1 --2- 172.21.15.42:0/3927759265 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55776e77b2f0 0x55776eb5b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:15:04.668 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.663+0000 7f787a7fc640 1 -- 172.21.15.42:0/3927759265 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55776e77b2f0 msgr2=0x55776eb5b920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:15:04.668 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.663+0000 7f787a7fc640 1 --2- 172.21.15.42:0/3927759265 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55776e77b2f0 0x55776eb5b920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:04.668 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.663+0000 7f787a7fc640 1 -- 172.21.15.42:0/3927759265 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55776e7a41b0 msgr2=0x55776eb5acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:15:04.669 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.663+0000 7f787a7fc640 1 --2- 172.21.15.42:0/3927759265 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55776e7a41b0 0x55776eb5acd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:04.669 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.663+0000 7f787a7fc640 1 -- 172.21.15.42:0/3927759265 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55776eb591a0 con 0x55776e7834d0 2024-08-30T09:15:04.669 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.663+0000 7f787a7fc640 1 --2- 172.21.15.42:0/3927759265 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55776e7834d0 0x55776eb5a5e0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f78700145a0 tx=0x7f7870005eb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b7e6b489b71597b8 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:15:04.669 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.663+0000 7f785affd640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-30T09:15:04.669 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.664+0000 7f785affd640 1 -- 172.21.15.42:0/3927759265 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7870019040 con 0x55776e7834d0 2024-08-30T09:15:04.669 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.664+0000 7f785affd640 1 -- 172.21.15.42:0/3927759265 <== mon.2 v2:172.21.15.167:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f7870004450 con 0x55776e7834d0 2024-08-30T09:15:04.669 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.664+0000 7f78887ee740 1 -- 172.21.15.42:0/3927759265 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f786c032aa0 con 0x55776e7834d0 2024-08-30T09:15:04.669 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.664+0000 7f785affd640 1 -- 172.21.15.42:0/3927759265 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f7870038050 con 0x55776e7834d0 2024-08-30T09:15:04.669 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.664+0000 7f78887ee740 10 client.27626 fetch_fsmap learned FSMap version 47 2024-08-30T09:15:04.669 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.664+0000 7f78887ee740 1 -- 172.21.15.42:0/3927759265 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55776ebcbfe0 con 0x55776e7834d0 2024-08-30T09:15:04.669 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.664+0000 7f785affd640 1 -- 172.21.15.42:0/3927759265 <== mon.2 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f786c032aa0 con 0x55776e7834d0 2024-08-30T09:15:04.669 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.664+0000 7f785affd640 1 -- 172.21.15.42:0/3927759265 <== mon.2 v2:172.21.15.167:3300/0 5 ==== fsmap(e 47) ==== 8370+0+0 (secure 0 0 0) 0x7f7870031060 con 0x55776e7834d0 2024-08-30T09:15:04.670 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.665+0000 7f78887ee740 10 client.27626 fetch_fsmap finished waiting for FSMap version 47 2024-08-30T09:15:04.670 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.665+0000 7f78887ee740 10 client.27626 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-30T09:15:04.670 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.665+0000 7f78887ee740 20 client.27626 populate_metadata read hostname 'smithi042' 2024-08-30T09:15:04.670 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.665+0000 7f78887ee740 1 --2- 172.21.15.42:0/3927759265 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55776e7381c0 0x55776ebceab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:04.670 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.665+0000 7f78887ee740 4 client.27626 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-30T09:15:04.670 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.665+0000 7f78887ee740 1 -- 172.21.15.42:0/3927759265 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55776eaba9f0 con 0x55776e7381c0 2024-08-30T09:15:04.670 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.665+0000 7f787affd640 1 --2- 172.21.15.42:0/3927759265 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55776e7381c0 0x55776ebceab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:15:04.670 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.665+0000 7f785affd640 10 client.27626 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:15:04.670 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.665+0000 7f787affd640 1 --2- 172.21.15.42:0/3927759265 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55776e7381c0 0x55776ebceab0 crc :-1 s=READY pgs=290 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1be17d2cf23c9442 server_cookie=4065c326b0486548 in_seq=0 out_seq=0 2024-08-30T09:15:04.682 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.679+0000 7f785affd640 1 -- 172.21.15.42:0/3927759265 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55776eaba9f0 con 0x55776e7381c0 2024-08-30T09:15:04.682 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.679+0000 7f785affd640 10 client.27626 handle_command_reply: tid=0 2024-08-30T09:15:04.682 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.679+0000 7f78887ee740 1 client.27626 shutdown 2024-08-30T09:15:04.682 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.679+0000 7f78887ee740 1 -- 172.21.15.42:0/3927759265 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55776e7834d0 msgr2=0x55776eb5a5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:15:04.682 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.679+0000 7f78887ee740 1 --2- 172.21.15.42:0/3927759265 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55776e7834d0 0x55776eb5a5e0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f78700145a0 tx=0x7f7870005eb0 comp rx=0 tx=0).stop 2024-08-30T09:15:04.682 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.679+0000 7f78887ee740 1 -- 172.21.15.42:0/3927759265 shutdown_connections 2024-08-30T09:15:04.682 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.679+0000 7f78887ee740 1 --2- 172.21.15.42:0/3927759265 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55776e7381c0 0x55776ebceab0 crc :-1 s=READY pgs=290 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:04.682 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.679+0000 7f78887ee740 1 --2- 172.21.15.42:0/3927759265 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55776e7a41b0 0x55776eb5acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:04.682 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.679+0000 7f78887ee740 1 --2- 172.21.15.42:0/3927759265 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55776e7834d0 0x55776eb5a5e0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:04.682 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.679+0000 7f78887ee740 1 --2- 172.21.15.42:0/3927759265 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55776e77b2f0 0x55776eb5b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:04.682 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.679+0000 7f78887ee740 1 -- 172.21.15.42:0/3927759265 >> 172.21.15.42:0/3927759265 conn(0x55776e798300 msgr2=0x55776eb539a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:15:04.682 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.679+0000 7f785affd640 0 client.27626 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:15:04.682 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.679+0000 7f78887ee740 1 -- 172.21.15.42:0/3927759265 shutdown_connections 2024-08-30T09:15:04.682 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.679+0000 7f78887ee740 1 -- 172.21.15.42:0/3927759265 wait complete. 2024-08-30T09:15:04.683 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.680+0000 7f78887ee740 20 client.27626 trim_cache size 0 max 16384 2024-08-30T09:15:04.683 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-08-30T09:15:04.683 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2024-08-30T09:15:04.683 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-08-30T09:15:04.683 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-08-30T09:15:04.684 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.680+0000 7f7881884640 1 -- 172.21.15.42:0/1455786413 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f78640b4d00 msgr2=0x7f78640b7210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:15:04.684 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.680+0000 7f7881884640 1 --2- 172.21.15.42:0/1455786413 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f78640b4d00 0x7f78640b7210 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f7874003fd0 tx=0x7f7874015000 comp rx=0 tx=0).stop 2024-08-30T09:15:04.684 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.680+0000 7f7881884640 1 -- 172.21.15.42:0/1455786413 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f787c101770 msgr2=0x7f787c074030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:15:04.684 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.680+0000 7f7881884640 1 --2- 172.21.15.42:0/1455786413 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f787c101770 0x7f787c074030 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f786c030a00 tx=0x7f786c030ec0 comp rx=0 tx=0).stop 2024-08-30T09:15:04.684 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.680+0000 7f7881884640 1 -- 172.21.15.42:0/1455786413 shutdown_connections 2024-08-30T09:15:04.684 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.680+0000 7f7881884640 1 --2- 172.21.15.42:0/1455786413 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f78640b4d00 0x7f78640b7210 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:04.685 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.680+0000 7f7881884640 1 --2- 172.21.15.42:0/1455786413 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f787c102d50 0x7f787c1a7680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:04.685 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.680+0000 7f7881884640 1 --2- 172.21.15.42:0/1455786413 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f787c1021a0 0x7f787c0745d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:04.685 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.680+0000 7f7881884640 1 --2- 172.21.15.42:0/1455786413 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f787c101770 0x7f787c074030 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:04.685 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.680+0000 7f7881884640 1 -- 172.21.15.42:0/1455786413 >> 172.21.15.42:0/1455786413 conn(0x7f787c0fd430 msgr2=0x7f787c1057d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:15:04.685 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.681+0000 7f7881884640 1 -- 172.21.15.42:0/1455786413 shutdown_connections 2024-08-30T09:15:04.685 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:04.681+0000 7f7881884640 1 -- 172.21.15.42:0/1455786413 wait complete. 2024-08-30T09:15:05.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:15:04 smithi167 ceph-mon[22063]: pgmap v802: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 4.4 MiB/s wr, 482 op/s 2024-08-30T09:15:05.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:15:04 smithi042 ceph-mon[18652]: pgmap v802: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 4.4 MiB/s wr, 482 op/s 2024-08-30T09:15:05.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:15:04 smithi190 ceph-mon[22001]: pgmap v802: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 4.4 MiB/s wr, 482 op/s 2024-08-30T09:15:05.953 INFO:teuthology.orchestra.run.smithi042.stderr:progress: 860.0 s, 478.8 tps, lat 215.817 ms stddev 255.638 2024-08-30T09:15:07.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:15:06 smithi167 ceph-mon[22063]: pgmap v803: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 4.1 MiB/s wr, 454 op/s 2024-08-30T09:15:07.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:15:06 smithi042 ceph-mon[18652]: pgmap v803: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 4.1 MiB/s wr, 454 op/s 2024-08-30T09:15:07.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:15:06 smithi190 ceph-mon[22001]: pgmap v803: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 4.1 MiB/s wr, 454 op/s 2024-08-30T09:15:07.522 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-30T09:15:08.068 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T09:15:08.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.813+0000 7f0243fff640 1 -- 172.21.15.42:0/2092139842 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0244105690 msgr2=0x7f0244101920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:15:08.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.813+0000 7f0243fff640 1 --2- 172.21.15.42:0/2092139842 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0244105690 0x7f0244101920 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f023c013490 tx=0x7f023c032b30 comp rx=0 tx=0).stop 2024-08-30T09:15:08.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.814+0000 7f0243fff640 1 -- 172.21.15.42:0/2092139842 shutdown_connections 2024-08-30T09:15:08.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.814+0000 7f0243fff640 1 --2- 172.21.15.42:0/2092139842 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0244104d40 0x7f0244105150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:08.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.814+0000 7f0243fff640 1 --2- 172.21.15.42:0/2092139842 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0244105690 0x7f0244101920 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:08.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.814+0000 7f0243fff640 1 --2- 172.21.15.42:0/2092139842 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0244102cc0 0x7f02441030d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:08.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.814+0000 7f0243fff640 1 -- 172.21.15.42:0/2092139842 >> 172.21.15.42:0/2092139842 conn(0x7f0244077840 msgr2=0x7f0244077c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:15:08.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.818+0000 7f0243fff640 1 -- 172.21.15.42:0/2092139842 shutdown_connections 2024-08-30T09:15:08.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.819+0000 7f0243fff640 1 -- 172.21.15.42:0/2092139842 wait complete. 2024-08-30T09:15:08.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.820+0000 7f0243fff640 1 Processor -- start 2024-08-30T09:15:08.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.820+0000 7f0243fff640 1 -- start start 2024-08-30T09:15:08.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.820+0000 7f0243fff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0244102cc0 0x7f0244074620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:08.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.820+0000 7f0243fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0244104d40 0x7f0244074ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:08.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.820+0000 7f0243fff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0244105690 0x7f024406e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:08.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.820+0000 7f0243fff640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f024406cd50 con 0x7f0244104d40 2024-08-30T09:15:08.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.820+0000 7f0243fff640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f024406cbd0 con 0x7f0244102cc0 2024-08-30T09:15:08.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.820+0000 7f0243fff640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f024406ced0 con 0x7f0244105690 2024-08-30T09:15:08.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.820+0000 7f0242ffd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0244102cc0 0x7f0244074620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:15:08.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.820+0000 7f0242ffd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0244102cc0 0x7f0244074620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:60314/0 (socket says 172.21.15.42:60314) 2024-08-30T09:15:08.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.821+0000 7f0242ffd640 1 -- 172.21.15.42:0/3941540221 learned_addr learned my addr 172.21.15.42:0/3941540221 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:15:08.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.821+0000 7f02427fc640 1 --2- 172.21.15.42:0/3941540221 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0244104d40 0x7f0244074ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:15:08.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.821+0000 7f02437fe640 1 --2- 172.21.15.42:0/3941540221 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0244105690 0x7f024406e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:15:08.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.821+0000 7f0242ffd640 1 -- 172.21.15.42:0/3941540221 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0244105690 msgr2=0x7f024406e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:15:08.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.821+0000 7f0242ffd640 1 --2- 172.21.15.42:0/3941540221 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0244105690 0x7f024406e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:08.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.821+0000 7f0242ffd640 1 -- 172.21.15.42:0/3941540221 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0244104d40 msgr2=0x7f0244074ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:15:08.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.821+0000 7f0242ffd640 1 --2- 172.21.15.42:0/3941540221 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0244104d40 0x7f0244074ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:08.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.821+0000 7f0242ffd640 1 -- 172.21.15.42:0/3941540221 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f024406f120 con 0x7f0244102cc0 2024-08-30T09:15:08.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.822+0000 7f0242ffd640 1 --2- 172.21.15.42:0/3941540221 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0244102cc0 0x7f0244074620 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f023800b470 tx=0x7f023800b930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:15:08.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.823+0000 7f022bfff640 1 -- 172.21.15.42:0/3941540221 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0238015020 con 0x7f0244102cc0 2024-08-30T09:15:08.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.823+0000 7f022bfff640 1 -- 172.21.15.42:0/3941540221 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0238011d40 con 0x7f0244102cc0 2024-08-30T09:15:08.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.823+0000 7f022bfff640 1 -- 172.21.15.42:0/3941540221 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f023800f340 con 0x7f0244102cc0 2024-08-30T09:15:08.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.823+0000 7f0243fff640 1 -- 172.21.15.42:0/3941540221 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f024406f410 con 0x7f0244102cc0 2024-08-30T09:15:08.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.823+0000 7f0243fff640 1 -- 172.21.15.42:0/3941540221 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02441ad080 con 0x7f0244102cc0 2024-08-30T09:15:08.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.828+0000 7f022bfff640 1 -- 172.21.15.42:0/3941540221 <== mon.2 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7f0238018020 con 0x7f0244102cc0 2024-08-30T09:15:08.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.828+0000 7f024f82c740 1 Processor -- start 2024-08-30T09:15:08.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.828+0000 7f024f82c740 1 -- start start 2024-08-30T09:15:08.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.829+0000 7f024f82c740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b6a6806070 0x55b6a6bab4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:08.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.829+0000 7f024f82c740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b6a67dd1b0 0x55b6a6bab9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:08.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.829+0000 7f024f82c740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b6a67e5390 0x55b6a6babf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:08.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.829+0000 7f024f82c740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55b6a6b22630 con 0x55b6a67e5390 2024-08-30T09:15:08.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.829+0000 7f024f82c740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55b6a6b11bc0 con 0x55b6a67dd1b0 2024-08-30T09:15:08.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.829+0000 7f024f82c740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55b6a6b15b70 con 0x55b6a6806070 2024-08-30T09:15:08.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.829+0000 7f02437fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b6a6806070 0x55b6a6bab4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:15:08.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.829+0000 7f02437fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b6a6806070 0x55b6a6bab4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:40264/0 (socket says 172.21.15.42:40264) 2024-08-30T09:15:08.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.829+0000 7f02437fe640 1 -- 172.21.15.42:0/3679155521 learned_addr learned my addr 172.21.15.42:0/3679155521 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:15:08.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.829+0000 7f02427fc640 1 --2- 172.21.15.42:0/3679155521 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b6a67e5390 0x55b6a6babf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:15:08.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.829+0000 7f022bfff640 1 --2- 172.21.15.42:0/3941540221 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f022c0b4d40 0x7f022c0b7250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:08.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.829+0000 7f0242ffd640 1 --2- 172.21.15.42:0/3679155521 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b6a67dd1b0 0x55b6a6bab9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:15:08.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.830+0000 7f0242ffd640 1 -- 172.21.15.42:0/3679155521 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b6a6806070 msgr2=0x55b6a6bab4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:15:08.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.830+0000 7f0242ffd640 1 --2- 172.21.15.42:0/3679155521 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b6a6806070 0x55b6a6bab4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:08.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.830+0000 7f0242ffd640 1 -- 172.21.15.42:0/3679155521 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b6a67e5390 msgr2=0x55b6a6babf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:15:08.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.830+0000 7f0242ffd640 1 --2- 172.21.15.42:0/3679155521 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b6a67e5390 0x55b6a6babf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:08.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.830+0000 7f0242ffd640 1 -- 172.21.15.42:0/3679155521 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6a6871650 con 0x55b6a67dd1b0 2024-08-30T09:15:08.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.830+0000 7f0242ffd640 1 --2- 172.21.15.42:0/3679155521 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b6a67dd1b0 0x55b6a6bab9e0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f02380e2230 tx=0x7f02380ed550 comp rx=0 tx=0).ready entity=mon.2 client_cookie=faf994afc33ede73 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:15:08.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.830+0000 7f02437fe640 1 --2- 172.21.15.42:0/3941540221 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f022c0b4d40 0x7f022c0b7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:15:08.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.830+0000 7f02297fa640 1 -- 172.21.15.42:0/3679155521 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02380a7810 con 0x55b6a67dd1b0 2024-08-30T09:15:08.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.830+0000 7f02297fa640 1 -- 172.21.15.42:0/3679155521 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f02380ee040 con 0x55b6a67dd1b0 2024-08-30T09:15:08.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.830+0000 7f022bfff640 1 -- 172.21.15.42:0/3941540221 <== mon.2 v2:172.21.15.167:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f02380e4b80 con 0x7f0244102cc0 2024-08-30T09:15:08.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.831+0000 7f02297fa640 1 -- 172.21.15.42:0/3679155521 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02380aa930 con 0x55b6a67dd1b0 2024-08-30T09:15:08.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.831+0000 7f024f82c740 1 -- 172.21.15.42:0/3679155521 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b6a67dd1b0 msgr2=0x55b6a6bab9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:15:08.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.831+0000 7f024f82c740 1 --2- 172.21.15.42:0/3679155521 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b6a67dd1b0 0x55b6a6bab9e0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f02380e2230 tx=0x7f02380ed550 comp rx=0 tx=0).stop 2024-08-30T09:15:08.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.832+0000 7f02437fe640 1 --2- 172.21.15.42:0/3941540221 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f022c0b4d40 0x7f022c0b7250 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f023c013490 tx=0x7f023c032b30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:15:08.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.832+0000 7f024f82c740 1 -- 172.21.15.42:0/3679155521 shutdown_connections 2024-08-30T09:15:08.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.832+0000 7f024f82c740 1 --2- 172.21.15.42:0/3679155521 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b6a67e5390 0x55b6a6babf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:08.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.832+0000 7f024f82c740 1 --2- 172.21.15.42:0/3679155521 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b6a67dd1b0 0x55b6a6bab9e0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:08.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.832+0000 7f024f82c740 1 --2- 172.21.15.42:0/3679155521 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b6a6806070 0x55b6a6bab4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:08.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.832+0000 7f024f82c740 1 -- 172.21.15.42:0/3679155521 >> 172.21.15.42:0/3679155521 conn(0x55b6a67fa460 msgr2=0x55b6a6ba9390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:15:08.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.832+0000 7f024f82c740 1 -- 172.21.15.42:0/3679155521 shutdown_connections 2024-08-30T09:15:08.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.832+0000 7f024f82c740 1 -- 172.21.15.42:0/3679155521 wait complete. 2024-08-30T09:15:08.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.832+0000 7f024f82c740 1 -- start start 2024-08-30T09:15:08.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.832+0000 7f024f82c740 1 Processor -- start 2024-08-30T09:15:08.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.833+0000 7f024f82c740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b6a67dd1b0 0x55b6a6bb6b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:08.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.833+0000 7f024f82c740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b6a67e5390 0x55b6a6bb7060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:08.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.833+0000 7f024f82c740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b6a6806070 0x55b6a6bb75c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:08.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.833+0000 7f024f82c740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55b6a6974700 con 0x55b6a67e5390 2024-08-30T09:15:08.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.833+0000 7f024f82c740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55b6a6a98700 con 0x55b6a6806070 2024-08-30T09:15:08.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.833+0000 7f024f82c740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55b6a6bb9cd0 con 0x55b6a67dd1b0 2024-08-30T09:15:08.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.833+0000 7f024f82c740 10 client.0 _ll_register_callbacks cb 0x55b6a6ac7dc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:15:08.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.833+0000 7f0242ffd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b6a67dd1b0 0x55b6a6bb6b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:15:08.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.833+0000 7f02427fc640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b6a67e5390 0x55b6a6bb7060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:15:08.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.833+0000 7f0242ffd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b6a67dd1b0 0x55b6a6bb6b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:40280/0 (socket says 172.21.15.42:40280) 2024-08-30T09:15:08.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.833+0000 7f0242ffd640 1 -- 172.21.15.42:0/1254182791 learned_addr learned my addr 172.21.15.42:0/1254182791 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:15:08.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.833+0000 7f02437fe640 1 --2- 172.21.15.42:0/1254182791 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b6a6806070 0x55b6a6bb75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:15:08.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.833+0000 7f0242ffd640 1 -- 172.21.15.42:0/1254182791 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b6a6806070 msgr2=0x55b6a6bb75c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:15:08.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.833+0000 7f0242ffd640 1 --2- 172.21.15.42:0/1254182791 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b6a6806070 0x55b6a6bb75c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:08.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.833+0000 7f0242ffd640 1 -- 172.21.15.42:0/1254182791 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b6a67e5390 msgr2=0x55b6a6bb7060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:15:08.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.833+0000 7f0242ffd640 1 --2- 172.21.15.42:0/1254182791 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b6a67e5390 0x55b6a6bb7060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:08.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.833+0000 7f0242ffd640 1 -- 172.21.15.42:0/1254182791 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b6a6ad18b0 con 0x55b6a67dd1b0 2024-08-30T09:15:08.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.834+0000 7f0242ffd640 1 --2- 172.21.15.42:0/1254182791 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b6a67dd1b0 0x55b6a6bb6b20 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f02380aacd0 tx=0x7f02380a7680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=68d3c5fd2b8965e7 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:15:08.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.834+0000 7f0226ff5640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-08-30T09:15:08.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.834+0000 7f0226ff5640 1 -- 172.21.15.42:0/1254182791 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02380aab10 con 0x55b6a67dd1b0 2024-08-30T09:15:08.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.834+0000 7f024f82c740 1 -- 172.21.15.42:0/1254182791 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f02380ee040 con 0x55b6a67dd1b0 2024-08-30T09:15:08.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.834+0000 7f0226ff5640 1 -- 172.21.15.42:0/1254182791 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f02380ede40 con 0x55b6a67dd1b0 2024-08-30T09:15:08.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.834+0000 7f0226ff5640 1 -- 172.21.15.42:0/1254182791 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f02380aad00 con 0x55b6a67dd1b0 2024-08-30T09:15:08.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.834+0000 7f024f82c740 10 client.27565 fetch_fsmap learned FSMap version 48 2024-08-30T09:15:08.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.834+0000 7f024f82c740 1 -- 172.21.15.42:0/1254182791 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55b6a6bc5a30 con 0x55b6a67dd1b0 2024-08-30T09:15:08.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.835+0000 7f0226ff5640 1 -- 172.21.15.42:0/1254182791 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f02380ee040 con 0x55b6a67dd1b0 2024-08-30T09:15:08.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.835+0000 7f0226ff5640 1 -- 172.21.15.42:0/1254182791 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 48) ==== 8366+0+0 (secure 0 0 0) 0x7f02380d0050 con 0x55b6a67dd1b0 2024-08-30T09:15:08.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.835+0000 7f024f82c740 10 client.27565 fetch_fsmap finished waiting for FSMap version 48 2024-08-30T09:15:08.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.835+0000 7f024f82c740 10 client.27565 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-30T09:15:08.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.835+0000 7f024f82c740 20 client.27565 populate_metadata read hostname 'smithi042' 2024-08-30T09:15:08.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.835+0000 7f024f82c740 1 --2- 172.21.15.42:0/1254182791 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55b6a67c49f0 0x55b6a6bc8d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:08.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.835+0000 7f024f82c740 4 client.27565 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-30T09:15:08.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.835+0000 7f024f82c740 1 -- 172.21.15.42:0/1254182791 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b6a6b24580 con 0x55b6a67c49f0 2024-08-30T09:15:08.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.835+0000 7f02427fc640 1 --2- 172.21.15.42:0/1254182791 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55b6a67c49f0 0x55b6a6bc8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:15:08.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.836+0000 7f0226ff5640 10 client.27565 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:15:08.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.836+0000 7f02427fc640 1 --2- 172.21.15.42:0/1254182791 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55b6a67c49f0 0x55b6a6bc8d90 crc :-1 s=READY pgs=291 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e9a4088f79cfa9c6 server_cookie=97fdde21580fc1a2 in_seq=0 out_seq=0 2024-08-30T09:15:08.843 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.837+0000 7f0226ff5640 1 -- 172.21.15.42:0/1254182791 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b6a6b24580 con 0x55b6a67c49f0 2024-08-30T09:15:08.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.837+0000 7f0226ff5640 10 client.27565 handle_command_reply: tid=0 2024-08-30T09:15:08.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.838+0000 7f024f82c740 1 client.27565 shutdown 2024-08-30T09:15:08.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.838+0000 7f024f82c740 1 -- 172.21.15.42:0/1254182791 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b6a67dd1b0 msgr2=0x55b6a6bb6b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:15:08.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.838+0000 7f024f82c740 1 --2- 172.21.15.42:0/1254182791 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b6a67dd1b0 0x55b6a6bb6b20 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f02380aacd0 tx=0x7f02380a7680 comp rx=0 tx=0).stop 2024-08-30T09:15:08.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.839+0000 7f024f82c740 1 -- 172.21.15.42:0/1254182791 shutdown_connections 2024-08-30T09:15:08.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.839+0000 7f024f82c740 1 --2- 172.21.15.42:0/1254182791 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55b6a67c49f0 0x55b6a6bc8d90 crc :-1 s=READY pgs=291 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:08.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.839+0000 7f0226ff5640 0 client.27565 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:15:08.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.839+0000 7f024f82c740 1 --2- 172.21.15.42:0/1254182791 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b6a6806070 0x55b6a6bb75c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:08.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.839+0000 7f024f82c740 1 --2- 172.21.15.42:0/1254182791 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b6a67e5390 0x55b6a6bb7060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:08.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.839+0000 7f024f82c740 1 --2- 172.21.15.42:0/1254182791 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b6a67dd1b0 0x55b6a6bb6b20 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:08.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.839+0000 7f024f82c740 1 -- 172.21.15.42:0/1254182791 >> 172.21.15.42:0/1254182791 conn(0x55b6a67fa460 msgr2=0x55b6a6bae630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:15:08.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.840+0000 7f024f82c740 1 -- 172.21.15.42:0/1254182791 shutdown_connections 2024-08-30T09:15:08.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.840+0000 7f024f82c740 1 -- 172.21.15.42:0/1254182791 wait complete. 2024-08-30T09:15:08.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.840+0000 7f024f82c740 20 client.27565 trim_cache size 0 max 16384 2024-08-30T09:15:08.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.874+0000 7f024f82c740 1 Processor -- start 2024-08-30T09:15:08.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.874+0000 7f024f82c740 1 -- start start 2024-08-30T09:15:08.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.874+0000 7f024f82c740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b6a67c49f0 0x55b6a6bb8a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:08.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.874+0000 7f024f82c740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b6a67dd1b0 0x55b6a6bacbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:08.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.874+0000 7f024f82c740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b6a67e5390 0x55b6a6bb3d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:08.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.874+0000 7f024f82c740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f02300123a0 con 0x55b6a67dd1b0 2024-08-30T09:15:08.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.874+0000 7f024f82c740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f023000fc40 con 0x55b6a67e5390 2024-08-30T09:15:08.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.874+0000 7f024f82c740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f023000da70 con 0x55b6a67c49f0 2024-08-30T09:15:08.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.875+0000 7f0242ffd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b6a67c49f0 0x55b6a6bb8a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:15:08.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.875+0000 7f0242ffd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b6a67c49f0 0x55b6a6bb8a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:40296/0 (socket says 172.21.15.42:40296) 2024-08-30T09:15:08.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.875+0000 7f0242ffd640 1 -- 172.21.15.42:0/89299575 learned_addr learned my addr 172.21.15.42:0/89299575 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:15:08.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.876+0000 7f02427fc640 1 --2- 172.21.15.42:0/89299575 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b6a67dd1b0 0x55b6a6bacbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:15:08.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.876+0000 7f0242ffd640 1 -- 172.21.15.42:0/89299575 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b6a67e5390 msgr2=0x55b6a6bb3d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:15:08.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.876+0000 7f0242ffd640 1 --2- 172.21.15.42:0/89299575 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b6a67e5390 0x55b6a6bb3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:08.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.876+0000 7f0242ffd640 1 -- 172.21.15.42:0/89299575 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b6a67dd1b0 msgr2=0x55b6a6bacbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:15:08.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.876+0000 7f0242ffd640 1 --2- 172.21.15.42:0/89299575 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b6a67dd1b0 0x55b6a6bacbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:08.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.876+0000 7f0242ffd640 1 -- 172.21.15.42:0/89299575 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6a6ba9c80 con 0x55b6a67c49f0 2024-08-30T09:15:08.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.878+0000 7f0242ffd640 1 --2- 172.21.15.42:0/89299575 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b6a67c49f0 0x55b6a6bb8a00 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f02380a8430 tx=0x7f0238009380 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2dcbcb1705d62bec server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:15:08.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.880+0000 7f0228ff9640 1 -- 172.21.15.42:0/89299575 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02380aaf90 con 0x55b6a67c49f0 2024-08-30T09:15:08.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.880+0000 7f0228ff9640 1 -- 172.21.15.42:0/89299575 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f02380ab8c0 con 0x55b6a67c49f0 2024-08-30T09:15:08.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.881+0000 7f024f82c740 1 -- 172.21.15.42:0/89299575 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b6a67c49f0 msgr2=0x55b6a6bb8a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:15:08.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.881+0000 7f024f82c740 1 --2- 172.21.15.42:0/89299575 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b6a67c49f0 0x55b6a6bb8a00 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f02380a8430 tx=0x7f0238009380 comp rx=0 tx=0).stop 2024-08-30T09:15:08.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.883+0000 7f024f82c740 1 -- 172.21.15.42:0/89299575 shutdown_connections 2024-08-30T09:15:08.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.883+0000 7f024f82c740 1 --2- 172.21.15.42:0/89299575 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b6a67e5390 0x55b6a6bb3d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:08.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.883+0000 7f024f82c740 1 --2- 172.21.15.42:0/89299575 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b6a67dd1b0 0x55b6a6bacbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:08.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.883+0000 7f024f82c740 1 --2- 172.21.15.42:0/89299575 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b6a67c49f0 0x55b6a6bb8a00 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:08.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.883+0000 7f024f82c740 1 -- 172.21.15.42:0/89299575 >> 172.21.15.42:0/89299575 conn(0x55b6a67fa460 msgr2=0x55b6a6bac4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:15:08.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.893+0000 7f024f82c740 1 -- 172.21.15.42:0/89299575 shutdown_connections 2024-08-30T09:15:08.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.897+0000 7f024f82c740 1 -- 172.21.15.42:0/89299575 wait complete. 2024-08-30T09:15:08.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.897+0000 7f024f82c740 1 -- start start 2024-08-30T09:15:08.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.897+0000 7f024f82c740 1 Processor -- start 2024-08-30T09:15:08.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.906+0000 7f024f82c740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b6a67c49f0 0x55b6a6bca550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:08.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.906+0000 7f024f82c740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b6a67dd1b0 0x55b6a6bcaa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:08.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.906+0000 7f024f82c740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b6a67e5390 0x55b6a6c32510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:08.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.906+0000 7f024f82c740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f023000e100 con 0x55b6a67c49f0 2024-08-30T09:15:08.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.906+0000 7f024f82c740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f023000f570 con 0x55b6a67e5390 2024-08-30T09:15:08.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.906+0000 7f024f82c740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f023000e7d0 con 0x55b6a67dd1b0 2024-08-30T09:15:08.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.906+0000 7f024f82c740 10 client.0 _ll_register_callbacks cb 0x7f02340097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:15:08.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.911+0000 7f02427fc640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b6a67dd1b0 0x55b6a6bcaa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:15:08.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.911+0000 7f02427fc640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b6a67dd1b0 0x55b6a6bcaa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:40310/0 (socket says 172.21.15.42:40310) 2024-08-30T09:15:08.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.911+0000 7f02427fc640 1 -- 172.21.15.42:0/1205339678 learned_addr learned my addr 172.21.15.42:0/1205339678 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:15:08.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.911+0000 7f0242ffd640 1 --2- 172.21.15.42:0/1205339678 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b6a67c49f0 0x55b6a6bca550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:15:08.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.913+0000 7f0242ffd640 1 -- 172.21.15.42:0/1205339678 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b6a67dd1b0 msgr2=0x55b6a6bcaa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:15:08.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.913+0000 7f0242ffd640 1 --2- 172.21.15.42:0/1205339678 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b6a67dd1b0 0x55b6a6bcaa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:08.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.913+0000 7f0242ffd640 1 -- 172.21.15.42:0/1205339678 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b6a67e5390 msgr2=0x55b6a6c32510 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-30T09:15:08.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.913+0000 7f0242ffd640 1 --2- 172.21.15.42:0/1205339678 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b6a67e5390 0x55b6a6c32510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:08.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.913+0000 7f0242ffd640 1 -- 172.21.15.42:0/1205339678 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b6a6c32aa0 con 0x55b6a67c49f0 2024-08-30T09:15:08.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.914+0000 7f0242ffd640 1 --2- 172.21.15.42:0/1205339678 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b6a67c49f0 0x55b6a6bca550 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f02380a8430 tx=0x7f02380e8070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ec022fff573e1aa server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:15:08.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.914+0000 7f0226ff5640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-08-30T09:15:08.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.916+0000 7f0226ff5640 1 -- 172.21.15.42:0/1205339678 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02380a6eb0 con 0x55b6a67c49f0 2024-08-30T09:15:08.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.916+0000 7f0226ff5640 1 -- 172.21.15.42:0/1205339678 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f02380aa630 con 0x55b6a67c49f0 2024-08-30T09:15:08.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.916+0000 7f024f82c740 1 -- 172.21.15.42:0/1205339678 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f02380ab8c0 con 0x55b6a67c49f0 2024-08-30T09:15:08.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.921+0000 7f0226ff5640 1 -- 172.21.15.42:0/1205339678 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f02380a7080 con 0x55b6a67c49f0 2024-08-30T09:15:08.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.922+0000 7f024f82c740 10 client.17796 fetch_fsmap learned FSMap version 48 2024-08-30T09:15:08.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.922+0000 7f024f82c740 1 -- 172.21.15.42:0/1205339678 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55b6a6c32aa0 con 0x55b6a67c49f0 2024-08-30T09:15:08.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.925+0000 7f0226ff5640 1 -- 172.21.15.42:0/1205339678 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f02380ab8c0 con 0x55b6a67c49f0 2024-08-30T09:15:08.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.925+0000 7f0226ff5640 1 -- 172.21.15.42:0/1205339678 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 48) ==== 8366+0+0 (secure 0 0 0) 0x7f02380d0070 con 0x55b6a67c49f0 2024-08-30T09:15:08.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.926+0000 7f024f82c740 10 client.17796 fetch_fsmap finished waiting for FSMap version 48 2024-08-30T09:15:08.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.926+0000 7f024f82c740 10 client.17796 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-30T09:15:08.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.926+0000 7f024f82c740 20 client.17796 populate_metadata read hostname 'smithi042' 2024-08-30T09:15:08.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.926+0000 7f024f82c740 1 --2- 172.21.15.42:0/1205339678 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55b6a6806070 0x55b6a6c36080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:08.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.926+0000 7f024f82c740 4 client.17796 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-30T09:15:08.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.926+0000 7f024f82c740 1 -- 172.21.15.42:0/1205339678 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b6a6b20020 con 0x55b6a6806070 2024-08-30T09:15:08.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.927+0000 7f02427fc640 1 --2- 172.21.15.42:0/1205339678 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55b6a6806070 0x55b6a6c36080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:15:08.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.942+0000 7f02427fc640 1 --2- 172.21.15.42:0/1205339678 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55b6a6806070 0x55b6a6c36080 crc :-1 s=READY pgs=292 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1c6dc2fa6280eab9 server_cookie=c6bdaa2580e7bbb0 in_seq=0 out_seq=0 2024-08-30T09:15:08.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.942+0000 7f0226ff5640 10 client.17796 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:15:08.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.942+0000 7f0226ff5640 1 -- 172.21.15.42:0/1205339678 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+4918 (crc 0 0 0) 0x55b6a6b20020 con 0x55b6a6806070 2024-08-30T09:15:08.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.942+0000 7f0226ff5640 10 client.17796 handle_command_reply: tid=0 2024-08-30T09:15:08.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.942+0000 7f024f82c740 1 client.17796 shutdown 2024-08-30T09:15:08.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.943+0000 7f024f82c740 1 -- 172.21.15.42:0/1205339678 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b6a67c49f0 msgr2=0x55b6a6bca550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:15:08.949 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.943+0000 7f024f82c740 1 --2- 172.21.15.42:0/1205339678 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b6a67c49f0 0x55b6a6bca550 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f02380a8430 tx=0x7f02380e8070 comp rx=0 tx=0).stop 2024-08-30T09:15:08.949 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.947+0000 7f024f82c740 1 -- 172.21.15.42:0/1205339678 shutdown_connections 2024-08-30T09:15:08.949 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.947+0000 7f024f82c740 1 --2- 172.21.15.42:0/1205339678 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55b6a6806070 0x55b6a6c36080 crc :-1 s=READY pgs=292 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:08.949 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.947+0000 7f024f82c740 1 --2- 172.21.15.42:0/1205339678 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b6a67dd1b0 0x55b6a6bcaa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:08.949 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.947+0000 7f024f82c740 1 --2- 172.21.15.42:0/1205339678 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b6a67e5390 0x55b6a6c32510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:08.949 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.947+0000 7f024f82c740 1 --2- 172.21.15.42:0/1205339678 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b6a67c49f0 0x55b6a6bca550 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:08.949 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.947+0000 7f024f82c740 1 -- 172.21.15.42:0/1205339678 >> 172.21.15.42:0/1205339678 conn(0x55b6a67fa460 msgr2=0x55b6a6bb8760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:15:08.949 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.947+0000 7f0226ff5640 0 client.17796 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:15:08.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.950+0000 7f024f82c740 1 -- 172.21.15.42:0/1205339678 shutdown_connections 2024-08-30T09:15:08.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.951+0000 7f024f82c740 1 -- 172.21.15.42:0/1205339678 wait complete. 2024-08-30T09:15:08.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.952+0000 7f024f82c740 20 client.17796 trim_cache size 0 max 16384 2024-08-30T09:15:08.954 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-08-30T09:15:08.954 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "no active scrubs running", 2024-08-30T09:15:08.954 INFO:teuthology.orchestra.run.smithi042.stdout: "scrubs": {}, 2024-08-30T09:15:08.954 INFO:teuthology.orchestra.run.smithi042.stdout: "scrub_stats": { 2024-08-30T09:15:08.954 INFO:teuthology.orchestra.run.smithi042.stdout: "d325166d-e7e4-4ac2-9eaf-fa602e480cf6": { 2024-08-30T09:15:08.954 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:14:30.861941+0000", 2024-08-30T09:15:08.955 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:15:08.955 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:15:08.955 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:15:08.955 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:15:08.955 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:15:08.955 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:15:08.955 INFO:teuthology.orchestra.run.smithi042.stdout: "03521407-a747-49b9-a415-6233551c26fb": { 2024-08-30T09:15:08.955 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:13:39.387817+0000", 2024-08-30T09:15:08.955 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:15:08.955 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:15:08.955 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:15:08.955 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:15:08.955 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:15:08.955 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:15:08.956 INFO:teuthology.orchestra.run.smithi042.stdout: "0cbf734e-f988-4b21-acac-57b55e3bde62": { 2024-08-30T09:15:08.956 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:12:45.306738+0000", 2024-08-30T09:15:08.956 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:15:08.956 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:15:08.956 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:15:08.956 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:15:08.956 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:15:08.956 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:15:08.956 INFO:teuthology.orchestra.run.smithi042.stdout: "5e2ed104-0625-478a-a78b-584e7f4c2817": { 2024-08-30T09:15:08.956 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:00:33.928400+0000", 2024-08-30T09:15:08.956 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:15:08.956 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:15:08.956 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:15:08.956 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:15:08.957 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:15:08.957 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:15:08.957 INFO:teuthology.orchestra.run.smithi042.stdout: "c827ad7a-44ee-4e1d-9bb3-51fb7ab00c73": { 2024-08-30T09:15:08.957 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:02:21.686517+0000", 2024-08-30T09:15:08.957 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:15:08.957 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:15:08.957 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:15:08.957 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:15:08.957 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:15:08.957 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:15:08.957 INFO:teuthology.orchestra.run.smithi042.stdout: "85134e4b-ab4a-4885-b07e-a3ad01eaf13d": { 2024-08-30T09:15:08.957 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:01:27.738462+0000", 2024-08-30T09:15:08.957 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:15:08.958 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:15:08.958 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:15:08.958 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:15:08.958 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:15:08.958 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:15:08.958 INFO:teuthology.orchestra.run.smithi042.stdout: "749f4867-0b70-49e5-af2d-8362a240ebc3": { 2024-08-30T09:15:08.958 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:03:16.386554+0000", 2024-08-30T09:15:08.958 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:15:08.958 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:15:08.958 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:15:08.958 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:15:08.958 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:15:08.958 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:15:08.958 INFO:teuthology.orchestra.run.smithi042.stdout: "88cd7478-2b20-47de-ba51-256479749232": { 2024-08-30T09:15:08.959 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:04:08.922625+0000", 2024-08-30T09:15:08.959 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:15:08.959 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:15:08.959 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:15:08.959 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:15:08.959 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:15:08.959 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:15:08.959 INFO:teuthology.orchestra.run.smithi042.stdout: "347d05ed-4bd3-4590-b099-314f719aab39": { 2024-08-30T09:15:08.959 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:05:00.439721+0000", 2024-08-30T09:15:08.959 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:15:08.959 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:15:08.959 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:15:08.959 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:15:08.960 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:15:08.960 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:15:08.960 INFO:teuthology.orchestra.run.smithi042.stdout: "1af4252a-4a99-47e9-bbf0-becf5e573dd5": { 2024-08-30T09:15:08.960 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:05:48.565881+0000", 2024-08-30T09:15:08.960 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:15:08.960 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:15:08.960 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:15:08.960 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:15:08.960 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:15:08.960 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:15:08.960 INFO:teuthology.orchestra.run.smithi042.stdout: "665ec740-7254-49cb-84ef-deff71c461b0": { 2024-08-30T09:15:08.960 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:06:37.144035+0000", 2024-08-30T09:15:08.960 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:15:08.961 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:15:08.961 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:15:08.961 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:15:08.961 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:15:08.961 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:15:08.961 INFO:teuthology.orchestra.run.smithi042.stdout: "598d8a75-f1cc-4388-80bc-a408bdb4eaac": { 2024-08-30T09:15:08.961 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:07:29.721122+0000", 2024-08-30T09:15:08.961 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:15:08.961 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:15:08.961 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:15:08.961 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:15:08.961 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:15:08.961 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:15:08.961 INFO:teuthology.orchestra.run.smithi042.stdout: "b62c4ffc-9387-4677-b9ba-1e94ba308f17": { 2024-08-30T09:15:08.961 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:11:03.363478+0000", 2024-08-30T09:15:08.962 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:15:08.962 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:15:08.962 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:15:08.962 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:15:08.962 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:15:08.962 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:15:08.962 INFO:teuthology.orchestra.run.smithi042.stdout: "a7210f9a-046b-4188-953c-819b9f1863a9": { 2024-08-30T09:15:08.962 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:10:07.608430+0000", 2024-08-30T09:15:08.962 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:15:08.962 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:15:08.962 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:15:08.962 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:15:08.962 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:15:08.962 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:15:08.963 INFO:teuthology.orchestra.run.smithi042.stdout: "99a29ec1-9455-4ae4-854a-398b32896675": { 2024-08-30T09:15:08.963 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:08:24.923176+0000", 2024-08-30T09:15:08.963 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:15:08.963 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:15:08.963 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:15:08.963 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:15:08.963 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:15:08.963 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:15:08.963 INFO:teuthology.orchestra.run.smithi042.stdout: "ff464e7f-123b-4bc6-bc7e-aced018168de": { 2024-08-30T09:15:08.963 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:11:54.879598+0000", 2024-08-30T09:15:08.963 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:15:08.963 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:15:08.963 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:15:08.964 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:15:08.964 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:15:08.964 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:15:08.964 INFO:teuthology.orchestra.run.smithi042.stdout: "37f2f28f-496f-41a6-b4cf-8846fb3f43ab": { 2024-08-30T09:15:08.964 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:09:17.428286+0000", 2024-08-30T09:15:08.964 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:15:08.964 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:15:08.964 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:15:08.964 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:15:08.964 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:15:08.964 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:15:08.964 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:15:08.964 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-08-30T09:15:08.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.953+0000 7f0243fff640 1 -- 172.21.15.42:0/3941540221 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f022c0b4d40 msgr2=0x7f022c0b7250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:15:08.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.953+0000 7f0243fff640 1 --2- 172.21.15.42:0/3941540221 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f022c0b4d40 0x7f022c0b7250 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f023c013490 tx=0x7f023c032b30 comp rx=0 tx=0).stop 2024-08-30T09:15:08.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.953+0000 7f0243fff640 1 -- 172.21.15.42:0/3941540221 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0244102cc0 msgr2=0x7f0244074620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:15:08.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.953+0000 7f0243fff640 1 --2- 172.21.15.42:0/3941540221 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0244102cc0 0x7f0244074620 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f023800b470 tx=0x7f023800b930 comp rx=0 tx=0).stop 2024-08-30T09:15:08.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.953+0000 7f0243fff640 1 -- 172.21.15.42:0/3941540221 shutdown_connections 2024-08-30T09:15:08.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.953+0000 7f0243fff640 1 --2- 172.21.15.42:0/3941540221 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f022c0b4d40 0x7f022c0b7250 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:08.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.953+0000 7f0243fff640 1 --2- 172.21.15.42:0/3941540221 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0244105690 0x7f024406e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:08.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.953+0000 7f0243fff640 1 --2- 172.21.15.42:0/3941540221 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0244104d40 0x7f0244074ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:08.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.953+0000 7f0243fff640 1 --2- 172.21.15.42:0/3941540221 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0244102cc0 0x7f0244074620 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:08.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.953+0000 7f0243fff640 1 -- 172.21.15.42:0/3941540221 >> 172.21.15.42:0/3941540221 conn(0x7f0244077840 msgr2=0x7f0244105d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:15:08.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.953+0000 7f0243fff640 1 -- 172.21.15.42:0/3941540221 shutdown_connections 2024-08-30T09:15:08.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:08.954+0000 7f0243fff640 1 -- 172.21.15.42:0/3941540221 wait complete. 2024-08-30T09:15:09.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:15:08 smithi167 ceph-mon[22063]: pgmap v804: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 3.6 MiB/s wr, 415 op/s 2024-08-30T09:15:09.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:15:08 smithi042 ceph-mon[18652]: pgmap v804: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 3.6 MiB/s wr, 415 op/s 2024-08-30T09:15:09.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:15:08 smithi190 ceph-mon[22001]: pgmap v804: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 3.6 MiB/s wr, 415 op/s 2024-08-30T09:15:09.861 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-08-30T09:15:09.862 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph fs dump --format=json 2024-08-30T09:15:10.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:15:09 smithi167 ceph-mon[22063]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-30T09:15:10.109 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:15:09 smithi042 ceph-mon[18652]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-30T09:15:10.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:15:09 smithi190 ceph-mon[22001]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-30T09:15:10.268 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T09:15:10.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:10.910+0000 7f892f5c1640 1 -- 172.21.15.42:0/4153143835 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8928101a80 msgr2=0x7f8928101e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:15:10.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:10.910+0000 7f892f5c1640 1 --2- 172.21.15.42:0/4153143835 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8928101a80 0x7f8928101e90 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f8918009860 tx=0x7f891802a380 comp rx=0 tx=0).stop 2024-08-30T09:15:10.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:10.913+0000 7f892f5c1640 1 -- 172.21.15.42:0/4153143835 shutdown_connections 2024-08-30T09:15:10.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:10.913+0000 7f892f5c1640 1 --2- 172.21.15.42:0/4153143835 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f89281023d0 0x7f8928105f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:10.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:10.913+0000 7f892f5c1640 1 --2- 172.21.15.42:0/4153143835 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8928101a80 0x7f8928101e90 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:10.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:10.913+0000 7f892f5c1640 1 --2- 172.21.15.42:0/4153143835 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8928105190 0x7f89281055a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:10.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:10.913+0000 7f892f5c1640 1 -- 172.21.15.42:0/4153143835 >> 172.21.15.42:0/4153143835 conn(0x7f8928077a00 msgr2=0x7f8928077e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:15:10.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:10.915+0000 7f892f5c1640 1 -- 172.21.15.42:0/4153143835 shutdown_connections 2024-08-30T09:15:10.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:10.915+0000 7f892f5c1640 1 -- 172.21.15.42:0/4153143835 wait complete. 2024-08-30T09:15:10.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:10.916+0000 7f892f5c1640 1 Processor -- start 2024-08-30T09:15:10.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:10.920+0000 7f892f5c1640 1 -- start start 2024-08-30T09:15:10.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:10.920+0000 7f892f5c1640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8928101a80 0x7f89281a06a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:10.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:10.920+0000 7f892f5c1640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f89281023d0 0x7f89281a0c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:10.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:10.921+0000 7f892f5c1640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8928105190 0x7f89281a5ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:10.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:10.921+0000 7f892f5c1640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f8928119e70 con 0x7f8928105190 2024-08-30T09:15:10.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:10.921+0000 7f892f5c1640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f8928119cf0 con 0x7f89281023d0 2024-08-30T09:15:10.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:10.921+0000 7f892f5c1640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f8928119ff0 con 0x7f8928101a80 2024-08-30T09:15:10.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:10.921+0000 7f892db5e640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8928105190 0x7f89281a5ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:15:10.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:10.921+0000 7f892d35d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8928101a80 0x7f89281a06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:15:10.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:10.921+0000 7f892d35d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8928101a80 0x7f89281a06a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:40322/0 (socket says 172.21.15.42:40322) 2024-08-30T09:15:10.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:10.921+0000 7f892d35d640 1 -- 172.21.15.42:0/1122378764 learned_addr learned my addr 172.21.15.42:0/1122378764 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:15:10.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:10.922+0000 7f892d35d640 1 -- 172.21.15.42:0/1122378764 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f89281023d0 msgr2=0x7f89281a0c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:15:10.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:10.922+0000 7f892d35d640 1 --2- 172.21.15.42:0/1122378764 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f89281023d0 0x7f89281a0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:10.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:10.922+0000 7f892d35d640 1 -- 172.21.15.42:0/1122378764 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8928105190 msgr2=0x7f89281a5ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:15:10.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:10.922+0000 7f892d35d640 1 --2- 172.21.15.42:0/1122378764 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8928105190 0x7f89281a5ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:10.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:10.922+0000 7f892d35d640 1 -- 172.21.15.42:0/1122378764 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89281a6610 con 0x7f8928101a80 2024-08-30T09:15:10.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:10.928+0000 7f892d35d640 1 --2- 172.21.15.42:0/1122378764 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8928101a80 0x7f89281a06a0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f892400d950 tx=0x7f892400de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:15:10.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:10.932+0000 7f891e7fc640 1 -- 172.21.15.42:0/1122378764 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8924017070 con 0x7f8928101a80 2024-08-30T09:15:10.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:10.932+0000 7f892f5c1640 1 -- 172.21.15.42:0/1122378764 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89281a6900 con 0x7f8928101a80 2024-08-30T09:15:10.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:10.932+0000 7f892f5c1640 1 -- 172.21.15.42:0/1122378764 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89281ad060 con 0x7f8928101a80 2024-08-30T09:15:10.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:10.932+0000 7f891e7fc640 1 -- 172.21.15.42:0/1122378764 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8924009c30 con 0x7f8928101a80 2024-08-30T09:15:10.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:10.932+0000 7f891e7fc640 1 -- 172.21.15.42:0/1122378764 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f892400f620 con 0x7f8928101a80 2024-08-30T09:15:10.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:10.937+0000 7f892f5c1640 1 -- 172.21.15.42:0/1122378764 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f890c0052c0 con 0x7f8928101a80 2024-08-30T09:15:10.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:10.940+0000 7f891e7fc640 1 -- 172.21.15.42:0/1122378764 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7f8924005030 con 0x7f8928101a80 2024-08-30T09:15:10.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:10.941+0000 7f891e7fc640 1 --2- 172.21.15.42:0/1122378764 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f89140b4d40 0x7f89140b7250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:10.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:10.941+0000 7f891e7fc640 1 -- 172.21.15.42:0/1122378764 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f89240e4d90 con 0x7f8928101a80 2024-08-30T09:15:10.949 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:10.947+0000 7f892cb5c640 1 --2- 172.21.15.42:0/1122378764 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f89140b4d40 0x7f89140b7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:15:10.953 INFO:teuthology.orchestra.run.smithi042.stderr:progress: 865.0 s, 430.2 tps, lat 229.855 ms stddev 308.400 2024-08-30T09:15:10.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:10.955+0000 7f891e7fc640 1 -- 172.21.15.42:0/1122378764 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f89240abbd0 con 0x7f8928101a80 2024-08-30T09:15:10.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:10.957+0000 7f892cb5c640 1 --2- 172.21.15.42:0/1122378764 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f89140b4d40 0x7f89140b7250 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f8918035040 tx=0x7f891800b050 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:15:11.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:15:10 smithi167 ceph-mon[22063]: pgmap v805: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 4.4 MiB/s wr, 511 op/s 2024-08-30T09:15:11.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:15:10 smithi042 ceph-mon[18652]: pgmap v805: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 4.4 MiB/s wr, 511 op/s 2024-08-30T09:15:11.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:15:10 smithi190 ceph-mon[22001]: pgmap v805: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 4.4 MiB/s wr, 511 op/s 2024-08-30T09:15:11.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:11.406+0000 7f892f5c1640 1 -- 172.21.15.42:0/1122378764 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f890c0055b0 con 0x7f8928101a80 2024-08-30T09:15:11.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:11.408+0000 7f891e7fc640 1 -- 172.21.15.42:0/1122378764 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) ==== 94+0+11919 (secure 0 0 0) 0x7f8924009de0 con 0x7f8928101a80 2024-08-30T09:15:11.410 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-08-30T09:15:11.410 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":48,"btime":"2024-08-30T09:15:08:784472+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14703,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/3188400000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":3188400000},{"type":"v1","addr":"172.21.15.42:6835","nonce":3188400000}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14715,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/3356537674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":3356537674},{"type":"v1","addr":"172.21.15.42:6837","nonce":3356537674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14766,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6839/2107785513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6838","nonce":2107785513},{"type":"v1","addr":"172.21.15.42:6839","nonce":2107785513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14778,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6841/2298438674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6840","nonce":2298438674},{"type":"v1","addr":"172.21.15.42:6841","nonce":2298438674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24553,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/543555570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":543555570},{"type":"v1","addr":"172.21.15.190:6833","nonce":543555570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24554,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/3907818941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":3907818941},{"type":"v1","addr":"172.21.15.167:6833","nonce":3907818941}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":48,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-30T08:58:03.055084+0000","modified":"2024-08-30T09:15:08.339777+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24629,"mds_1":24610,"mds_2":24599},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24572":{"gid":24572,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6835/234596845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":234596845},{"type":"v1","addr":"172.21.15.167:6835","nonce":234596845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24592":{"gid":24592,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6835/247905709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":247905709},{"type":"v1","addr":"172.21.15.190:6835","nonce":247905709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24598":{"gid":24598,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6837/863256349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6836","nonce":863256349},{"type":"v1","addr":"172.21.15.190:6837","nonce":863256349}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24599":{"gid":24599,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.167:6837/3011837856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":3011837856},{"type":"v1","addr":"172.21.15.167:6837","nonce":3011837856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24610":{"gid":24610,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.190:6839/719981110","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6838","nonce":719981110},{"type":"v1","addr":"172.21.15.190:6839","nonce":719981110}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24629":{"gid":24629,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.167:6839/3604645207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":3604645207},{"type":"v1","addr":"172.21.15.167:6839","nonce":3604645207}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24629,"qdb_cluster":[24629,24610,24599]},"id":1}]} 2024-08-30T09:15:11.410 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 48 2024-08-30T09:15:11.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:11.416+0000 7f892f5c1640 1 -- 172.21.15.42:0/1122378764 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f89140b4d40 msgr2=0x7f89140b7250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:15:11.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:11.416+0000 7f892f5c1640 1 --2- 172.21.15.42:0/1122378764 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f89140b4d40 0x7f89140b7250 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f8918035040 tx=0x7f891800b050 comp rx=0 tx=0).stop 2024-08-30T09:15:11.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:11.416+0000 7f892f5c1640 1 -- 172.21.15.42:0/1122378764 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8928101a80 msgr2=0x7f89281a06a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:15:11.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:11.416+0000 7f892f5c1640 1 --2- 172.21.15.42:0/1122378764 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8928101a80 0x7f89281a06a0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f892400d950 tx=0x7f892400de10 comp rx=0 tx=0).stop 2024-08-30T09:15:11.425 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:11.423+0000 7f892f5c1640 1 -- 172.21.15.42:0/1122378764 shutdown_connections 2024-08-30T09:15:11.425 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:11.423+0000 7f892f5c1640 1 --2- 172.21.15.42:0/1122378764 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f89140b4d40 0x7f89140b7250 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:11.425 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:11.423+0000 7f892f5c1640 1 --2- 172.21.15.42:0/1122378764 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8928105190 0x7f89281a5ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:11.425 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:11.423+0000 7f892f5c1640 1 --2- 172.21.15.42:0/1122378764 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f89281023d0 0x7f89281a0c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:11.425 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:11.423+0000 7f892f5c1640 1 --2- 172.21.15.42:0/1122378764 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8928101a80 0x7f89281a06a0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:11.425 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:11.423+0000 7f892f5c1640 1 -- 172.21.15.42:0/1122378764 >> 172.21.15.42:0/1122378764 conn(0x7f8928077a00 msgr2=0x7f8928100a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:15:11.425 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:11.423+0000 7f892f5c1640 1 -- 172.21.15.42:0/1122378764 shutdown_connections 2024-08-30T09:15:11.425 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:11.423+0000 7f892f5c1640 1 -- 172.21.15.42:0/1122378764 wait complete. 2024-08-30T09:15:12.121 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:15:11 smithi042 ceph-mon[18652]: from='client.? 172.21.15.42:0/1122378764' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-30T09:15:12.171 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-08-30T09:15:12.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:15:11 smithi190 ceph-mon[22001]: from='client.? 172.21.15.42:0/1122378764' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-30T09:15:12.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:15:11 smithi167 ceph-mon[22063]: from='client.? 172.21.15.42:0/1122378764' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-30T09:15:12.700 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T09:15:13.085 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:15:12 smithi042 ceph-mon[18652]: pgmap v806: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 4.0 MiB/s wr, 430 op/s 2024-08-30T09:15:13.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:15:12 smithi190 ceph-mon[22001]: pgmap v806: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 4.0 MiB/s wr, 430 op/s 2024-08-30T09:15:13.227 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:15:12 smithi167 ceph-mon[22063]: pgmap v806: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 4.0 MiB/s wr, 430 op/s 2024-08-30T09:15:13.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.370+0000 7f842adef640 1 -- 172.21.15.42:0/2679340925 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f841c0956f0 msgr2=0x7f841c095b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:15:13.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.370+0000 7f842adef640 1 --2- 172.21.15.42:0/2679340925 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f841c0956f0 0x7f841c095b00 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f8420009900 tx=0x7f842002a410 comp rx=0 tx=0).stop 2024-08-30T09:15:13.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.370+0000 7f842adef640 1 -- 172.21.15.42:0/2679340925 shutdown_connections 2024-08-30T09:15:13.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.370+0000 7f842adef640 1 --2- 172.21.15.42:0/2679340925 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f841c097700 0x7f841c097b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:13.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.370+0000 7f842adef640 1 --2- 172.21.15.42:0/2679340925 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f841c098050 0x7f841c0a1ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:13.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.370+0000 7f842adef640 1 --2- 172.21.15.42:0/2679340925 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f841c0956f0 0x7f841c095b00 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:13.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.370+0000 7f842adef640 1 -- 172.21.15.42:0/2679340925 >> 172.21.15.42:0/2679340925 conn(0x7f841c00c4d0 msgr2=0x7f841c00c8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:15:13.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.371+0000 7f842adef640 1 -- 172.21.15.42:0/2679340925 shutdown_connections 2024-08-30T09:15:13.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.371+0000 7f842adef640 1 -- 172.21.15.42:0/2679340925 wait complete. 2024-08-30T09:15:13.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.372+0000 7f842adef640 1 Processor -- start 2024-08-30T09:15:13.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.372+0000 7f842adef640 1 -- start start 2024-08-30T09:15:13.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.372+0000 7f842adef640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f841c0956f0 0x7f841c12b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:13.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.372+0000 7f842adef640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f841c097700 0x7f841c12bed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:13.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.372+0000 7f842adef640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f841c098050 0x7f841c131170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:13.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.372+0000 7f8429ded640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f841c0956f0 0x7f841c12b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:15:13.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.372+0000 7f8429ded640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f841c0956f0 0x7f841c12b930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:47698/0 (socket says 172.21.15.42:47698) 2024-08-30T09:15:13.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.373+0000 7f842adef640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f841c0a4f80 con 0x7f841c0956f0 2024-08-30T09:15:13.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.373+0000 7f84295ec640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f841c097700 0x7f841c12bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:15:13.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.374+0000 7f842adef640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f841c0a4e00 con 0x7f841c097700 2024-08-30T09:15:13.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.374+0000 7f84295ec640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f841c097700 0x7f841c12bed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:60376/0 (socket says 172.21.15.42:60376) 2024-08-30T09:15:13.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.374+0000 7f84295ec640 1 -- 172.21.15.42:0/3517894737 learned_addr learned my addr 172.21.15.42:0/3517894737 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:15:13.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.374+0000 7f842adef640 1 -- 172.21.15.42:0/3517894737 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f841c0a5100 con 0x7f841c098050 2024-08-30T09:15:13.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.375+0000 7f842a5ee640 1 --2- 172.21.15.42:0/3517894737 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f841c098050 0x7f841c131170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:15:13.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.375+0000 7f84295ec640 1 -- 172.21.15.42:0/3517894737 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f841c098050 msgr2=0x7f841c131170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:15:13.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.375+0000 7f84295ec640 1 --2- 172.21.15.42:0/3517894737 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f841c098050 0x7f841c131170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:13.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.375+0000 7f84295ec640 1 -- 172.21.15.42:0/3517894737 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f841c0956f0 msgr2=0x7f841c12b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:15:13.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.375+0000 7f84295ec640 1 --2- 172.21.15.42:0/3517894737 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f841c0956f0 0x7f841c12b930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:13.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.375+0000 7f84295ec640 1 -- 172.21.15.42:0/3517894737 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f841c1318a0 con 0x7f841c097700 2024-08-30T09:15:13.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.375+0000 7f84295ec640 1 --2- 172.21.15.42:0/3517894737 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f841c097700 0x7f841c12bed0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f841400d950 tx=0x7f841400de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:15:13.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.376+0000 7f841affd640 1 -- 172.21.15.42:0/3517894737 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8414017070 con 0x7f841c097700 2024-08-30T09:15:13.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.376+0000 7f842adef640 1 -- 172.21.15.42:0/3517894737 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f841c131b90 con 0x7f841c097700 2024-08-30T09:15:13.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.376+0000 7f841affd640 1 -- 172.21.15.42:0/3517894737 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8414009c30 con 0x7f841c097700 2024-08-30T09:15:13.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.376+0000 7f842adef640 1 -- 172.21.15.42:0/3517894737 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f841c094350 con 0x7f841c097700 2024-08-30T09:15:13.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.376+0000 7f841affd640 1 -- 172.21.15.42:0/3517894737 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84140047a0 con 0x7f841c097700 2024-08-30T09:15:13.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.379+0000 7f841affd640 1 -- 172.21.15.42:0/3517894737 <== mon.2 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7f8414004020 con 0x7f841c097700 2024-08-30T09:15:13.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.379+0000 7f84325c6740 1 Processor -- start 2024-08-30T09:15:13.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.380+0000 7f84325c6740 1 -- start start 2024-08-30T09:15:13.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.380+0000 7f841affd640 1 --2- 172.21.15.42:0/3517894737 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f84100b4bb0 0x7f84100b70c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:13.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.380+0000 7f84325c6740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561d9185f070 0x561d91c044b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:13.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.380+0000 7f84325c6740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561d918361b0 0x561d91c049f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:13.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.380+0000 7f84325c6740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561d9183e390 0x561d91c04f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:13.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.380+0000 7f84325c6740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x561d91b7b680 con 0x561d9183e390 2024-08-30T09:15:13.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.380+0000 7f84325c6740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x561d91b6ac10 con 0x561d9185f070 2024-08-30T09:15:13.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.380+0000 7f84325c6740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x561d91b6ebc0 con 0x561d918361b0 2024-08-30T09:15:13.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.380+0000 7f842a5ee640 1 --2- 172.21.15.42:0/3517894737 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f84100b4bb0 0x7f84100b70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:15:13.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.381+0000 7f841affd640 1 -- 172.21.15.42:0/3517894737 <== mon.2 v2:172.21.15.167:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f84140e51a0 con 0x7f841c097700 2024-08-30T09:15:13.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.381+0000 7f8429ded640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561d9185f070 0x561d91c044b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:15:13.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.381+0000 7f842a5ee640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561d9183e390 0x561d91c04f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:15:13.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.381+0000 7f842a5ee640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561d9183e390 0x561d91c04f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:47704/0 (socket says 172.21.15.42:47704) 2024-08-30T09:15:13.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.381+0000 7f84295ec640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561d918361b0 0x561d91c049f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:15:13.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.381+0000 7f842a5ee640 1 -- 172.21.15.42:0/2240823137 learned_addr learned my addr 172.21.15.42:0/2240823137 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:15:13.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.381+0000 7f84295ec640 1 -- 172.21.15.42:0/2240823137 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561d9185f070 msgr2=0x561d91c044b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:15:13.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.381+0000 7f84295ec640 1 --2- 172.21.15.42:0/2240823137 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561d9185f070 0x561d91c044b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:13.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.381+0000 7f84295ec640 1 -- 172.21.15.42:0/2240823137 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561d9183e390 msgr2=0x561d91c04f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:15:13.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.381+0000 7f84295ec640 1 --2- 172.21.15.42:0/2240823137 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561d9183e390 0x561d91c04f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:13.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.381+0000 7f84295ec640 1 -- 172.21.15.42:0/2240823137 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561d918ca650 con 0x561d918361b0 2024-08-30T09:15:13.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.382+0000 7f842a5ee640 1 --2- 172.21.15.42:0/3517894737 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f84100b4bb0 0x7f84100b70c0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f8424066a10 tx=0x7f8424069770 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:15:13.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.382+0000 7f84295ec640 1 --2- 172.21.15.42:0/2240823137 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561d918361b0 0x561d91c049f0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f8414005fd0 tx=0x7f84140aaf50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fc4825dc583e9c27 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:15:13.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.382+0000 7f840ffff640 1 -- 172.21.15.42:0/2240823137 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84140abee0 con 0x561d918361b0 2024-08-30T09:15:13.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.382+0000 7f840ffff640 1 -- 172.21.15.42:0/2240823137 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8414009de0 con 0x561d918361b0 2024-08-30T09:15:13.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.382+0000 7f840ffff640 1 -- 172.21.15.42:0/2240823137 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84140e2aa0 con 0x561d918361b0 2024-08-30T09:15:13.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.383+0000 7f84325c6740 1 -- 172.21.15.42:0/2240823137 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561d918361b0 msgr2=0x561d91c049f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:15:13.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.383+0000 7f84325c6740 1 --2- 172.21.15.42:0/2240823137 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561d918361b0 0x561d91c049f0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f8414005fd0 tx=0x7f84140aaf50 comp rx=0 tx=0).stop 2024-08-30T09:15:13.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.383+0000 7f84325c6740 1 -- 172.21.15.42:0/2240823137 shutdown_connections 2024-08-30T09:15:13.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.383+0000 7f84325c6740 1 --2- 172.21.15.42:0/2240823137 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561d918361b0 0x561d91c049f0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:13.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.384+0000 7f84325c6740 1 --2- 172.21.15.42:0/2240823137 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561d9183e390 0x561d91c04f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:13.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.384+0000 7f84325c6740 1 --2- 172.21.15.42:0/2240823137 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561d9185f070 0x561d91c044b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:13.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.384+0000 7f84325c6740 1 -- 172.21.15.42:0/2240823137 >> 172.21.15.42:0/2240823137 conn(0x561d91853460 msgr2=0x561d91c023a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:15:13.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.386+0000 7f84325c6740 1 -- 172.21.15.42:0/2240823137 shutdown_connections 2024-08-30T09:15:13.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.386+0000 7f84325c6740 1 -- 172.21.15.42:0/2240823137 wait complete. 2024-08-30T09:15:13.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.386+0000 7f84325c6740 1 -- start start 2024-08-30T09:15:13.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.387+0000 7f84325c6740 1 Processor -- start 2024-08-30T09:15:13.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.387+0000 7f84325c6740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561d918361b0 0x561d91c0fc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:13.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.388+0000 7f84325c6740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561d9183e390 0x561d91c101d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:13.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.388+0000 7f84325c6740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561d9185f070 0x561d91c1c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:13.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.388+0000 7f84325c6740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x561d919cd700 con 0x561d918361b0 2024-08-30T09:15:13.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.388+0000 7f84325c6740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x561d91af1750 con 0x561d9183e390 2024-08-30T09:15:13.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.388+0000 7f84325c6740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x561d91c12c90 con 0x561d9185f070 2024-08-30T09:15:13.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.388+0000 7f842a5ee640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561d9185f070 0x561d91c1c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:15:13.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.388+0000 7f842a5ee640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561d9185f070 0x561d91c1c000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:40356/0 (socket says 172.21.15.42:40356) 2024-08-30T09:15:13.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.388+0000 7f842a5ee640 1 -- 172.21.15.42:0/2417492838 learned_addr learned my addr 172.21.15.42:0/2417492838 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:15:13.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.388+0000 7f84325c6740 10 client.0 _ll_register_callbacks cb 0x561d91b20e10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:15:13.397 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.392+0000 7f84295ec640 1 --2- 172.21.15.42:0/2417492838 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561d9183e390 0x561d91c101d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:15:13.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.393+0000 7f842a5ee640 1 -- 172.21.15.42:0/2417492838 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561d9183e390 msgr2=0x561d91c101d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:15:13.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.393+0000 7f842a5ee640 1 --2- 172.21.15.42:0/2417492838 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561d9183e390 0x561d91c101d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:13.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.393+0000 7f842a5ee640 1 -- 172.21.15.42:0/2417492838 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561d918361b0 msgr2=0x561d91c0fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:15:13.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.393+0000 7f842a5ee640 1 --2- 172.21.15.42:0/2417492838 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561d918361b0 0x561d91c0fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:13.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.393+0000 7f842a5ee640 1 -- 172.21.15.42:0/2417492838 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561d91b2a900 con 0x561d9185f070 2024-08-30T09:15:13.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.395+0000 7f842a5ee640 1 --2- 172.21.15.42:0/2417492838 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561d9185f070 0x561d91c1c000 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f8424069c20 tx=0x7f8424069bb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8c2e0b0194f34e44 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:15:13.401 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.397+0000 7f840d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-08-30T09:15:13.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.397+0000 7f840d7fa640 1 -- 172.21.15.42:0/2417492838 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f842408c280 con 0x561d9185f070 2024-08-30T09:15:13.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.397+0000 7f840d7fa640 1 -- 172.21.15.42:0/2417492838 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f842408c420 con 0x561d9185f070 2024-08-30T09:15:13.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.400+0000 7f84325c6740 1 -- 172.21.15.42:0/2417492838 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8414009de0 con 0x561d9185f070 2024-08-30T09:15:13.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.402+0000 7f840d7fa640 1 -- 172.21.15.42:0/2417492838 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f842406c780 con 0x561d9185f070 2024-08-30T09:15:13.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.410+0000 7f84325c6740 10 client.27589 fetch_fsmap learned FSMap version 48 2024-08-30T09:15:13.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.410+0000 7f84325c6740 1 -- 172.21.15.42:0/2417492838 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x561d91c1eaf0 con 0x561d9185f070 2024-08-30T09:15:13.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.413+0000 7f840d7fa640 1 -- 172.21.15.42:0/2417492838 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f842406cdd0 con 0x561d9185f070 2024-08-30T09:15:13.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.413+0000 7f840d7fa640 1 -- 172.21.15.42:0/2417492838 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 48) ==== 8366+0+0 (secure 0 0 0) 0x7f842409c050 con 0x561d9185f070 2024-08-30T09:15:13.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.413+0000 7f84325c6740 10 client.27589 fetch_fsmap finished waiting for FSMap version 48 2024-08-30T09:15:13.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.413+0000 7f84325c6740 10 client.27589 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-08-30T09:15:13.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.413+0000 7f84325c6740 20 client.27589 populate_metadata read hostname 'smithi042' 2024-08-30T09:15:13.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.413+0000 7f84325c6740 1 --2- 172.21.15.42:0/2417492838 >> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] conn(0x561d9181d9f0 0x561d91c22450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:13.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.413+0000 7f84325c6740 4 client.27589 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-30T09:15:13.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.413+0000 7f84325c6740 1 -- 172.21.15.42:0/2417492838 --> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f8414009de0 con 0x561d9181d9f0 2024-08-30T09:15:13.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.414+0000 7f8429ded640 1 --2- 172.21.15.42:0/2417492838 >> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] conn(0x561d9181d9f0 0x561d91c22450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:15:13.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.414+0000 7f840d7fa640 10 client.27589 ms_handle_connect on v2:172.21.15.167:6836/3011837856 2024-08-30T09:15:13.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.415+0000 7f8429ded640 1 --2- 172.21.15.42:0/2417492838 >> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] conn(0x561d9181d9f0 0x561d91c22450 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ae7d822e29e7eb78 server_cookie=f16c1d341c35d6ce in_seq=0 out_seq=0 2024-08-30T09:15:13.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.420+0000 7f840d7fa640 1 -- 172.21.15.42:0/2417492838 <== mds.2 v2:172.21.15.167:6836/3011837856 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f8414009de0 con 0x561d9181d9f0 2024-08-30T09:15:13.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.420+0000 7f840d7fa640 10 client.27589 handle_command_reply: tid=0 2024-08-30T09:15:13.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.420+0000 7f84325c6740 1 client.27589 shutdown 2024-08-30T09:15:13.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.420+0000 7f84325c6740 1 -- 172.21.15.42:0/2417492838 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561d9185f070 msgr2=0x561d91c1c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:15:13.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.420+0000 7f84325c6740 1 --2- 172.21.15.42:0/2417492838 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561d9185f070 0x561d91c1c000 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f8424069c20 tx=0x7f8424069bb0 comp rx=0 tx=0).stop 2024-08-30T09:15:13.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.428+0000 7f84325c6740 1 -- 172.21.15.42:0/2417492838 shutdown_connections 2024-08-30T09:15:13.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.428+0000 7f84325c6740 1 --2- 172.21.15.42:0/2417492838 >> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] conn(0x561d9181d9f0 0x561d91c22450 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:13.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.428+0000 7f84325c6740 1 --2- 172.21.15.42:0/2417492838 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561d9185f070 0x561d91c1c000 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:13.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.428+0000 7f84325c6740 1 --2- 172.21.15.42:0/2417492838 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561d9183e390 0x561d91c101d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:13.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.428+0000 7f84325c6740 1 --2- 172.21.15.42:0/2417492838 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561d918361b0 0x561d91c0fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:13.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.428+0000 7f84325c6740 1 -- 172.21.15.42:0/2417492838 >> 172.21.15.42:0/2417492838 conn(0x561d91853460 msgr2=0x561d91c07640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:15:13.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.428+0000 7f840d7fa640 0 client.27589 ms_handle_reset on v2:172.21.15.167:6836/3011837856 2024-08-30T09:15:13.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.428+0000 7f84325c6740 1 -- 172.21.15.42:0/2417492838 shutdown_connections 2024-08-30T09:15:13.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.429+0000 7f84325c6740 1 -- 172.21.15.42:0/2417492838 wait complete. 2024-08-30T09:15:13.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.431+0000 7f84325c6740 20 client.27589 trim_cache size 0 max 16384 2024-08-30T09:15:13.466 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.464+0000 7f84325c6740 1 Processor -- start 2024-08-30T09:15:13.466 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.464+0000 7f84325c6740 1 -- start start 2024-08-30T09:15:13.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.465+0000 7f84325c6740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561d918361b0 0x561d91c05bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:13.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.465+0000 7f84325c6740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561d9183e390 0x561d91c1bf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:13.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.465+0000 7f84325c6740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561d9185f070 0x561d91c1c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:13.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.465+0000 7f8429ded640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561d918361b0 0x561d91c05bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:15:13.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.465+0000 7f84325c6740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f841c1438f0 con 0x561d9185f070 2024-08-30T09:15:13.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.465+0000 7f84325c6740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f841c141190 con 0x561d9183e390 2024-08-30T09:15:13.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.465+0000 7f84325c6740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f841c13efe0 con 0x561d918361b0 2024-08-30T09:15:13.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.466+0000 7f842a5ee640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561d9185f070 0x561d91c1c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:15:13.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.466+0000 7f842a5ee640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561d9185f070 0x561d91c1c440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:47710/0 (socket says 172.21.15.42:47710) 2024-08-30T09:15:13.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.466+0000 7f842a5ee640 1 -- 172.21.15.42:0/1502015069 learned_addr learned my addr 172.21.15.42:0/1502015069 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:15:13.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.466+0000 7f842a5ee640 1 -- 172.21.15.42:0/1502015069 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561d918361b0 msgr2=0x561d91c05bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:15:13.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.466+0000 7f842a5ee640 1 --2- 172.21.15.42:0/1502015069 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561d918361b0 0x561d91c05bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:13.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.466+0000 7f842a5ee640 1 -- 172.21.15.42:0/1502015069 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561d9183e390 msgr2=0x561d91c1bf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:15:13.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.466+0000 7f842a5ee640 1 --2- 172.21.15.42:0/1502015069 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561d9183e390 0x561d91c1bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:13.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.466+0000 7f842a5ee640 1 -- 172.21.15.42:0/1502015069 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561d91c0e190 con 0x561d9185f070 2024-08-30T09:15:13.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.467+0000 7f842a5ee640 1 --2- 172.21.15.42:0/1502015069 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561d9185f070 0x561d91c1c440 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f8424051ab0 tx=0x7f8424078f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcb4e0a85b429d16 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:15:13.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.469+0000 7f840f7fe640 1 -- 172.21.15.42:0/1502015069 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84240a2070 con 0x561d9185f070 2024-08-30T09:15:13.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.469+0000 7f840f7fe640 1 -- 172.21.15.42:0/1502015069 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f842408c5c0 con 0x561d9185f070 2024-08-30T09:15:13.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.469+0000 7f840f7fe640 1 -- 172.21.15.42:0/1502015069 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f842406ce30 con 0x561d9185f070 2024-08-30T09:15:13.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.474+0000 7f84325c6740 1 -- 172.21.15.42:0/1502015069 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561d9185f070 msgr2=0x561d91c1c440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:15:13.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.474+0000 7f84325c6740 1 --2- 172.21.15.42:0/1502015069 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561d9185f070 0x561d91c1c440 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f8424051ab0 tx=0x7f8424078f90 comp rx=0 tx=0).stop 2024-08-30T09:15:13.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.474+0000 7f84325c6740 1 -- 172.21.15.42:0/1502015069 shutdown_connections 2024-08-30T09:15:13.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.474+0000 7f84325c6740 1 --2- 172.21.15.42:0/1502015069 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561d9185f070 0x561d91c1c440 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:13.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.474+0000 7f84325c6740 1 --2- 172.21.15.42:0/1502015069 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561d9183e390 0x561d91c1bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:13.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.474+0000 7f84325c6740 1 --2- 172.21.15.42:0/1502015069 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561d918361b0 0x561d91c05bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:13.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.474+0000 7f84325c6740 1 -- 172.21.15.42:0/1502015069 >> 172.21.15.42:0/1502015069 conn(0x561d91853460 msgr2=0x561d91c054e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:15:13.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.477+0000 7f84325c6740 1 -- 172.21.15.42:0/1502015069 shutdown_connections 2024-08-30T09:15:13.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.477+0000 7f84325c6740 1 -- 172.21.15.42:0/1502015069 wait complete. 2024-08-30T09:15:13.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.478+0000 7f84325c6740 1 -- start start 2024-08-30T09:15:13.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.478+0000 7f84325c6740 1 Processor -- start 2024-08-30T09:15:13.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.479+0000 7f84325c6740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561d918361b0 0x561d91c14310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:13.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.479+0000 7f84325c6740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561d9183e390 0x561d91c14850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:13.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.479+0000 7f84325c6740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561d9185f070 0x561d91c8be50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:13.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.479+0000 7f84325c6740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f841c13f650 con 0x561d9183e390 2024-08-30T09:15:13.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.479+0000 7f84325c6740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f841c140ac0 con 0x561d9185f070 2024-08-30T09:15:13.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.480+0000 7f84325c6740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f841c13fd20 con 0x561d918361b0 2024-08-30T09:15:13.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.480+0000 7f84325c6740 10 client.0 _ll_register_callbacks cb 0x7f842406a030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:15:13.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.480+0000 7f84295ec640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561d9183e390 0x561d91c14850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:15:13.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.480+0000 7f84295ec640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561d9183e390 0x561d91c14850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:47724/0 (socket says 172.21.15.42:47724) 2024-08-30T09:15:13.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.480+0000 7f84295ec640 1 -- 172.21.15.42:0/3526875586 learned_addr learned my addr 172.21.15.42:0/3526875586 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:15:13.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.481+0000 7f84295ec640 1 -- 172.21.15.42:0/3526875586 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561d918361b0 msgr2=0x561d91c14310 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-30T09:15:13.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.481+0000 7f84295ec640 1 --2- 172.21.15.42:0/3526875586 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561d918361b0 0x561d91c14310 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:13.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.481+0000 7f84295ec640 1 -- 172.21.15.42:0/3526875586 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561d9185f070 msgr2=0x561d91c8be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:15:13.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.481+0000 7f84295ec640 1 --2- 172.21.15.42:0/3526875586 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561d9185f070 0x561d91c8be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:13.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.481+0000 7f84295ec640 1 -- 172.21.15.42:0/3526875586 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561d91c8c390 con 0x561d9183e390 2024-08-30T09:15:13.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.481+0000 7f84295ec640 1 --2- 172.21.15.42:0/3526875586 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561d9183e390 0x561d91c14850 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f8414009610 tx=0x7f8414004560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72dbcbef2ce5fe25 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:15:13.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.482+0000 7f840d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-08-30T09:15:13.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.482+0000 7f840d7fa640 1 -- 172.21.15.42:0/3526875586 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8414009de0 con 0x561d9183e390 2024-08-30T09:15:13.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.482+0000 7f84325c6740 1 -- 172.21.15.42:0/3526875586 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f842408c5c0 con 0x561d9183e390 2024-08-30T09:15:13.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.482+0000 7f840d7fa640 1 -- 172.21.15.42:0/3526875586 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f84140aaa40 con 0x561d9183e390 2024-08-30T09:15:13.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.483+0000 7f840d7fa640 1 -- 172.21.15.42:0/3526875586 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f84140abee0 con 0x561d9183e390 2024-08-30T09:15:13.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.485+0000 7f84325c6740 10 client.17817 fetch_fsmap learned FSMap version 48 2024-08-30T09:15:13.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.485+0000 7f84325c6740 1 -- 172.21.15.42:0/3526875586 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x561d91c8d460 con 0x561d9183e390 2024-08-30T09:15:13.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.489+0000 7f840d7fa640 1 -- 172.21.15.42:0/3526875586 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f842408c5c0 con 0x561d9183e390 2024-08-30T09:15:13.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.489+0000 7f840d7fa640 1 -- 172.21.15.42:0/3526875586 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 48) ==== 8366+0+0 (secure 0 0 0) 0x7f84140d21b0 con 0x561d9183e390 2024-08-30T09:15:13.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.489+0000 7f84325c6740 10 client.17817 fetch_fsmap finished waiting for FSMap version 48 2024-08-30T09:15:13.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.489+0000 7f84325c6740 10 client.17817 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-08-30T09:15:13.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.489+0000 7f84325c6740 20 client.17817 populate_metadata read hostname 'smithi042' 2024-08-30T09:15:13.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.489+0000 7f84325c6740 1 --2- 172.21.15.42:0/3526875586 >> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] conn(0x561d917f2da0 0x561d91c8fc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:13.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.489+0000 7f84325c6740 4 client.17817 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-30T09:15:13.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.489+0000 7f84325c6740 1 -- 172.21.15.42:0/3526875586 --> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561d91b7d5d0 con 0x561d917f2da0 2024-08-30T09:15:13.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.491+0000 7f8429ded640 1 --2- 172.21.15.42:0/3526875586 >> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] conn(0x561d917f2da0 0x561d91c8fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:15:13.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.498+0000 7f8429ded640 1 --2- 172.21.15.42:0/3526875586 >> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] conn(0x561d917f2da0 0x561d91c8fc40 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=11737918a258de22 server_cookie=f2cd6c36cdc53817 in_seq=0 out_seq=0 2024-08-30T09:15:13.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.499+0000 7f840d7fa640 10 client.17817 ms_handle_connect on v2:172.21.15.167:6836/3011837856 2024-08-30T09:15:13.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.501+0000 7f840d7fa640 1 -- 172.21.15.42:0/3526875586 <== mds.2 v2:172.21.15.167:6836/3011837856 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561d91b7d5d0 con 0x561d917f2da0 2024-08-30T09:15:13.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.501+0000 7f840d7fa640 10 client.17817 handle_command_reply: tid=0 2024-08-30T09:15:13.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.501+0000 7f84325c6740 1 client.17817 shutdown 2024-08-30T09:15:13.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.507+0000 7f84325c6740 1 -- 172.21.15.42:0/3526875586 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561d9183e390 msgr2=0x561d91c14850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:15:13.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.507+0000 7f84325c6740 1 --2- 172.21.15.42:0/3526875586 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561d9183e390 0x561d91c14850 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f8414009610 tx=0x7f8414004560 comp rx=0 tx=0).stop 2024-08-30T09:15:13.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.507+0000 7f84325c6740 1 -- 172.21.15.42:0/3526875586 shutdown_connections 2024-08-30T09:15:13.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.507+0000 7f84325c6740 1 --2- 172.21.15.42:0/3526875586 >> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] conn(0x561d917f2da0 0x561d91c8fc40 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:13.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.507+0000 7f84325c6740 1 --2- 172.21.15.42:0/3526875586 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561d9185f070 0x561d91c8be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:13.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.507+0000 7f84325c6740 1 --2- 172.21.15.42:0/3526875586 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561d9183e390 0x561d91c14850 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:13.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.507+0000 7f84325c6740 1 --2- 172.21.15.42:0/3526875586 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561d918361b0 0x561d91c14310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:13.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.507+0000 7f84325c6740 1 -- 172.21.15.42:0/3526875586 >> 172.21.15.42:0/3526875586 conn(0x561d91853460 msgr2=0x561d91c101d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:15:13.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.507+0000 7f840d7fa640 0 client.17817 ms_handle_reset on v2:172.21.15.167:6836/3011837856 2024-08-30T09:15:13.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.507+0000 7f84325c6740 1 -- 172.21.15.42:0/3526875586 shutdown_connections 2024-08-30T09:15:13.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.507+0000 7f84325c6740 1 -- 172.21.15.42:0/3526875586 wait complete. 2024-08-30T09:15:13.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.508+0000 7f84325c6740 20 client.17817 trim_cache size 0 max 16384 2024-08-30T09:15:13.512 INFO:teuthology.orchestra.run.smithi042.stdout:[] 2024-08-30T09:15:13.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.517+0000 7f842adef640 1 -- 172.21.15.42:0/3517894737 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f84100b4bb0 msgr2=0x7f84100b70c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:15:13.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.517+0000 7f842adef640 1 --2- 172.21.15.42:0/3517894737 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f84100b4bb0 0x7f84100b70c0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f8424066a10 tx=0x7f8424069770 comp rx=0 tx=0).stop 2024-08-30T09:15:13.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.517+0000 7f842adef640 1 -- 172.21.15.42:0/3517894737 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f841c097700 msgr2=0x7f841c12bed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:15:13.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.517+0000 7f842adef640 1 --2- 172.21.15.42:0/3517894737 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f841c097700 0x7f841c12bed0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f841400d950 tx=0x7f841400de10 comp rx=0 tx=0).stop 2024-08-30T09:15:13.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.517+0000 7f842adef640 1 -- 172.21.15.42:0/3517894737 shutdown_connections 2024-08-30T09:15:13.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.517+0000 7f842adef640 1 --2- 172.21.15.42:0/3517894737 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f84100b4bb0 0x7f84100b70c0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:13.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.517+0000 7f842adef640 1 --2- 172.21.15.42:0/3517894737 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f841c098050 0x7f841c131170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:13.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.517+0000 7f842adef640 1 --2- 172.21.15.42:0/3517894737 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f841c097700 0x7f841c12bed0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:13.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.517+0000 7f842adef640 1 --2- 172.21.15.42:0/3517894737 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f841c0956f0 0x7f841c12b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:13.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.517+0000 7f842adef640 1 -- 172.21.15.42:0/3517894737 >> 172.21.15.42:0/3517894737 conn(0x7f841c00c4d0 msgr2=0x7f841c099610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:15:13.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.518+0000 7f842adef640 1 -- 172.21.15.42:0/3517894737 shutdown_connections 2024-08-30T09:15:13.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:13.518+0000 7f842adef640 1 -- 172.21.15.42:0/3517894737 wait complete. 2024-08-30T09:15:14.491 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-08-30T09:15:14.993 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T09:15:15.121 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:15:14 smithi042 ceph-mon[18652]: pgmap v807: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 4.1 MiB/s wr, 457 op/s 2024-08-30T09:15:15.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:15:14 smithi190 ceph-mon[22001]: pgmap v807: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 4.1 MiB/s wr, 457 op/s 2024-08-30T09:15:15.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:15:14 smithi167 ceph-mon[22063]: pgmap v807: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 4.1 MiB/s wr, 457 op/s 2024-08-30T09:15:15.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.639+0000 7fe5c0fb8640 1 -- 172.21.15.42:0/692257294 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe5bc102800 msgr2=0x7fe5bc102c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:15:15.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.639+0000 7fe5c0fb8640 1 --2- 172.21.15.42:0/692257294 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe5bc102800 0x7fe5bc102c10 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fe5ac009ca0 tx=0x7fe5ac02ece0 comp rx=0 tx=0).stop 2024-08-30T09:15:15.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.639+0000 7fe5c0fb8640 1 -- 172.21.15.42:0/692257294 shutdown_connections 2024-08-30T09:15:15.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.639+0000 7fe5c0fb8640 1 --2- 172.21.15.42:0/692257294 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe5bc108140 0x7fe5bc108550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:15.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.639+0000 7fe5c0fb8640 1 --2- 172.21.15.42:0/692257294 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe5bc108a90 0x7fe5bc10f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:15.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.639+0000 7fe5c0fb8640 1 --2- 172.21.15.42:0/692257294 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe5bc102800 0x7fe5bc102c10 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:15.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.639+0000 7fe5c0fb8640 1 -- 172.21.15.42:0/692257294 >> 172.21.15.42:0/692257294 conn(0x7fe5bc0794d0 msgr2=0x7fe5bc0798e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:15:15.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.639+0000 7fe5c0fb8640 1 -- 172.21.15.42:0/692257294 shutdown_connections 2024-08-30T09:15:15.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.639+0000 7fe5c0fb8640 1 -- 172.21.15.42:0/692257294 wait complete. 2024-08-30T09:15:15.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.640+0000 7fe5c0fb8640 1 Processor -- start 2024-08-30T09:15:15.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.640+0000 7fe5c0fb8640 1 -- start start 2024-08-30T09:15:15.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.641+0000 7fe5c0fb8640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe5bc102800 0x7fe5bc197fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:15.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.641+0000 7fe5c0fb8640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe5bc108140 0x7fe5bc198570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:15.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.641+0000 7fe5c0fb8640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe5bc108a90 0x7fe5bc19d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:15.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.641+0000 7fe5c0fb8640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fe5bc1115a0 con 0x7fe5bc108140 2024-08-30T09:15:15.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.641+0000 7fe5c0fb8640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe5bc111420 con 0x7fe5bc108a90 2024-08-30T09:15:15.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.641+0000 7fe5c0fb8640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fe5bc111720 con 0x7fe5bc102800 2024-08-30T09:15:15.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.641+0000 7fe5ba59c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe5bc102800 0x7fe5bc197fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:15:15.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.641+0000 7fe5ba59c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe5bc102800 0x7fe5bc197fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:40390/0 (socket says 172.21.15.42:40390) 2024-08-30T09:15:15.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.641+0000 7fe5ba59c640 1 -- 172.21.15.42:0/4073143483 learned_addr learned my addr 172.21.15.42:0/4073143483 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:15:15.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.641+0000 7fe5b9d9b640 1 --2- 172.21.15.42:0/4073143483 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe5bc108140 0x7fe5bc198570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:15:15.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.641+0000 7fe5bad9d640 1 --2- 172.21.15.42:0/4073143483 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe5bc108a90 0x7fe5bc19d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:15:15.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.641+0000 7fe5ba59c640 1 -- 172.21.15.42:0/4073143483 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe5bc108a90 msgr2=0x7fe5bc19d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:15:15.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.641+0000 7fe5ba59c640 1 --2- 172.21.15.42:0/4073143483 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe5bc108a90 0x7fe5bc19d810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:15.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.642+0000 7fe5ba59c640 1 -- 172.21.15.42:0/4073143483 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe5bc108140 msgr2=0x7fe5bc198570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:15:15.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.642+0000 7fe5ba59c640 1 --2- 172.21.15.42:0/4073143483 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe5bc108140 0x7fe5bc198570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:15.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.642+0000 7fe5ba59c640 1 -- 172.21.15.42:0/4073143483 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5bc19df40 con 0x7fe5bc102800 2024-08-30T09:15:15.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.642+0000 7fe5ba59c640 1 --2- 172.21.15.42:0/4073143483 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe5bc102800 0x7fe5bc197fd0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fe5ac02f2d0 tx=0x7fe5ac003ee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:15:15.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.642+0000 7fe5ab7fe640 1 -- 172.21.15.42:0/4073143483 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5ac03c040 con 0x7fe5bc102800 2024-08-30T09:15:15.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.642+0000 7fe5c0fb8640 1 -- 172.21.15.42:0/4073143483 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5bc19e1d0 con 0x7fe5bc102800 2024-08-30T09:15:15.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.642+0000 7fe5ab7fe640 1 -- 172.21.15.42:0/4073143483 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe5ac047070 con 0x7fe5bc102800 2024-08-30T09:15:15.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.642+0000 7fe5ab7fe640 1 -- 172.21.15.42:0/4073143483 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5ac0378e0 con 0x7fe5bc102800 2024-08-30T09:15:15.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.642+0000 7fe5c0fb8640 1 -- 172.21.15.42:0/4073143483 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5bc1a4990 con 0x7fe5bc102800 2024-08-30T09:15:15.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.646+0000 7fe5c752d740 1 Processor -- start 2024-08-30T09:15:15.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.650+0000 7fe5ab7fe640 1 -- 172.21.15.42:0/4073143483 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7fe5ac035ba0 con 0x7fe5bc102800 2024-08-30T09:15:15.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.651+0000 7fe5ab7fe640 1 --2- 172.21.15.42:0/4073143483 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fe5a40b4bb0 0x7fe5a40b70c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:15.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.651+0000 7fe5ab7fe640 1 -- 172.21.15.42:0/4073143483 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7fe5ac0c8950 con 0x7fe5bc102800 2024-08-30T09:15:15.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.654+0000 7fe5c752d740 1 -- start start 2024-08-30T09:15:15.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.654+0000 7fe5c752d740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558ccbc5c070 0x558ccc009b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:15.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.654+0000 7fe5c752d740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558ccbc331b0 0x558ccc00a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:15.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.654+0000 7fe5c752d740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558ccbc3b390 0x558ccc00a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:15.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.654+0000 7fe5c752d740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x558ccbf78680 con 0x558ccbc3b390 2024-08-30T09:15:15.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.654+0000 7fe5c752d740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x558ccbf67c10 con 0x558ccbc331b0 2024-08-30T09:15:15.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.654+0000 7fe5c752d740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x558ccbf6bbc0 con 0x558ccbc5c070 2024-08-30T09:15:15.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.655+0000 7fe5b9d9b640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558ccbc331b0 0x558ccc00a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:15:15.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.655+0000 7fe5b9d9b640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558ccbc331b0 0x558ccc00a0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:60448/0 (socket says 172.21.15.42:60448) 2024-08-30T09:15:15.659 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.655+0000 7fe5b9d9b640 1 -- 172.21.15.42:0/328738476 learned_addr learned my addr 172.21.15.42:0/328738476 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:15:15.659 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.655+0000 7fe5bad9d640 1 --2- 172.21.15.42:0/328738476 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558ccbc3b390 0x558ccc00a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:15:15.659 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.656+0000 7fe5bad9d640 1 --2- 172.21.15.42:0/4073143483 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fe5a40b4bb0 0x7fe5a40b70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:15:15.659 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.656+0000 7fe5b9d9b640 1 -- 172.21.15.42:0/328738476 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558ccbc5c070 msgr2=0x558ccc009b60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-30T09:15:15.659 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.656+0000 7fe5b9d9b640 1 --2- 172.21.15.42:0/328738476 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558ccbc5c070 0x558ccc009b60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:15.659 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.656+0000 7fe5b9d9b640 1 -- 172.21.15.42:0/328738476 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558ccbc3b390 msgr2=0x558ccc00a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:15:15.659 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.656+0000 7fe5b9d9b640 1 --2- 172.21.15.42:0/328738476 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558ccbc3b390 0x558ccc00a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:15.660 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.656+0000 7fe5b9d9b640 1 -- 172.21.15.42:0/328738476 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ccbcc7650 con 0x558ccbc331b0 2024-08-30T09:15:15.660 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.657+0000 7fe5b9d9b640 1 --2- 172.21.15.42:0/328738476 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558ccbc331b0 0x558ccc00a0a0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fe5b000bc50 tx=0x7fe5b000b3f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c71469d6fe12f9d8 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:15:15.661 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.657+0000 7fe5a8ff9640 1 -- 172.21.15.42:0/328738476 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5b0019030 con 0x558ccbc331b0 2024-08-30T09:15:15.661 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.657+0000 7fe5a8ff9640 1 -- 172.21.15.42:0/328738476 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe5b00054d0 con 0x558ccbc331b0 2024-08-30T09:15:15.661 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.659+0000 7fe5c752d740 1 -- 172.21.15.42:0/328738476 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558ccbc331b0 msgr2=0x558ccc00a0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:15:15.661 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.659+0000 7fe5c752d740 1 --2- 172.21.15.42:0/328738476 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558ccbc331b0 0x558ccc00a0a0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fe5b000bc50 tx=0x7fe5b000b3f0 comp rx=0 tx=0).stop 2024-08-30T09:15:15.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.663+0000 7fe5bad9d640 1 --2- 172.21.15.42:0/4073143483 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fe5a40b4bb0 0x7fe5a40b70c0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fe5b40058c0 tx=0x7fe5b4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:15:15.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.668+0000 7fe5c752d740 1 -- 172.21.15.42:0/328738476 shutdown_connections 2024-08-30T09:15:15.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.668+0000 7fe5c752d740 1 --2- 172.21.15.42:0/328738476 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558ccbc3b390 0x558ccc00a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:15.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.668+0000 7fe5c752d740 1 --2- 172.21.15.42:0/328738476 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558ccbc331b0 0x558ccc00a0a0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:15.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.668+0000 7fe5c752d740 1 --2- 172.21.15.42:0/328738476 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558ccbc5c070 0x558ccc009b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:15.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.668+0000 7fe5c752d740 1 -- 172.21.15.42:0/328738476 >> 172.21.15.42:0/328738476 conn(0x558ccbc50460 msgr2=0x558ccc007a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:15:15.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.668+0000 7fe5c752d740 1 -- 172.21.15.42:0/328738476 shutdown_connections 2024-08-30T09:15:15.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.668+0000 7fe5c752d740 1 -- 172.21.15.42:0/328738476 wait complete. 2024-08-30T09:15:15.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.669+0000 7fe5c752d740 1 -- start start 2024-08-30T09:15:15.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.669+0000 7fe5c752d740 1 Processor -- start 2024-08-30T09:15:15.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.669+0000 7fe5c752d740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558ccbc331b0 0x558ccc00de40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:15.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.669+0000 7fe5c752d740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558ccbc3b390 0x558ccc00e380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:15.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.669+0000 7fe5c752d740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558ccbc5c070 0x558ccc016eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:15.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.670+0000 7fe5c752d740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x558ccbdca700 con 0x558ccbc5c070 2024-08-30T09:15:15.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.670+0000 7fe5c752d740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x558ccbeee750 con 0x558ccbc3b390 2024-08-30T09:15:15.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.670+0000 7fe5c752d740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x558ccbffef50 con 0x558ccbc331b0 2024-08-30T09:15:15.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.670+0000 7fe5c752d740 10 client.0 _ll_register_callbacks cb 0x558ccbf1de10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:15:15.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.674+0000 7fe5ba59c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558ccbc331b0 0x558ccc00de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:15:15.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.674+0000 7fe5ba59c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558ccbc331b0 0x558ccc00de40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:40412/0 (socket says 172.21.15.42:40412) 2024-08-30T09:15:15.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.674+0000 7fe5ba59c640 1 -- 172.21.15.42:0/639644299 learned_addr learned my addr 172.21.15.42:0/639644299 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:15:15.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.675+0000 7fe5b9d9b640 1 --2- 172.21.15.42:0/639644299 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558ccbc3b390 0x558ccc00e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:15:15.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.675+0000 7fe5bad9d640 1 --2- 172.21.15.42:0/639644299 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558ccbc5c070 0x558ccc016eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:15:15.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.679+0000 7fe5ba59c640 1 -- 172.21.15.42:0/639644299 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558ccbc3b390 msgr2=0x558ccc00e380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:15:15.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.679+0000 7fe5ba59c640 1 --2- 172.21.15.42:0/639644299 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558ccbc3b390 0x558ccc00e380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:15.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.679+0000 7fe5ba59c640 1 -- 172.21.15.42:0/639644299 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558ccbc5c070 msgr2=0x558ccc016eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:15:15.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.679+0000 7fe5ba59c640 1 --2- 172.21.15.42:0/639644299 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558ccbc5c070 0x558ccc016eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:15.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.679+0000 7fe5ba59c640 1 -- 172.21.15.42:0/639644299 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ccbf27900 con 0x558ccbc331b0 2024-08-30T09:15:15.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.682+0000 7fe5ba59c640 1 --2- 172.21.15.42:0/639644299 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558ccbc331b0 0x558ccc00de40 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fe5ac0c93d0 tx=0x7fe5ac0c6a10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=37c2d0274ed51e6a server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:15:15.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.682+0000 7fe5a1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-08-30T09:15:15.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.686+0000 7fe5a1ffb640 1 -- 172.21.15.42:0/639644299 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5ac0c8b50 con 0x558ccbc331b0 2024-08-30T09:15:15.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.686+0000 7fe5a1ffb640 1 -- 172.21.15.42:0/639644299 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7fe5ac0cbcb0 con 0x558ccbc331b0 2024-08-30T09:15:15.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.686+0000 7fe5c752d740 1 -- 172.21.15.42:0/639644299 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe5b00054d0 con 0x558ccbc331b0 2024-08-30T09:15:15.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.689+0000 7fe5a1ffb640 1 -- 172.21.15.42:0/639644299 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7fe5ac0c8cf0 con 0x558ccbc331b0 2024-08-30T09:15:15.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.689+0000 7fe5c752d740 10 client.27598 fetch_fsmap learned FSMap version 48 2024-08-30T09:15:15.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.689+0000 7fe5c752d740 1 -- 172.21.15.42:0/639644299 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x558ccc02c930 con 0x558ccbc331b0 2024-08-30T09:15:15.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.691+0000 7fe5a1ffb640 1 -- 172.21.15.42:0/639644299 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe5b00054d0 con 0x558ccbc331b0 2024-08-30T09:15:15.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.691+0000 7fe5a1ffb640 1 -- 172.21.15.42:0/639644299 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 48) ==== 8366+0+0 (secure 0 0 0) 0x7fe5ac03f030 con 0x558ccbc331b0 2024-08-30T09:15:15.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.691+0000 7fe5c752d740 10 client.27598 fetch_fsmap finished waiting for FSMap version 48 2024-08-30T09:15:15.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.691+0000 7fe5c752d740 10 client.27598 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-08-30T09:15:15.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.691+0000 7fe5c752d740 20 client.27598 populate_metadata read hostname 'smithi042' 2024-08-30T09:15:15.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.691+0000 7fe5c752d740 1 --2- 172.21.15.42:0/639644299 >> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] conn(0x558ccbc1a9f0 0x558ccc02fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:15.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.691+0000 7fe5c752d740 4 client.27598 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-30T09:15:15.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.691+0000 7fe5c752d740 1 -- 172.21.15.42:0/639644299 --> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558ccbf7a5d0 con 0x558ccbc1a9f0 2024-08-30T09:15:15.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.695+0000 7fe5b9d9b640 1 --2- 172.21.15.42:0/639644299 >> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] conn(0x558ccbc1a9f0 0x558ccc02fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:15:15.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.705+0000 7fe5a1ffb640 10 client.27598 ms_handle_connect on v2:172.21.15.190:6838/719981110 2024-08-30T09:15:15.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.705+0000 7fe5b9d9b640 1 --2- 172.21.15.42:0/639644299 >> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] conn(0x558ccbc1a9f0 0x558ccc02fe10 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ba1e86e0ebdaec24 server_cookie=13959c4eb83bf194 in_seq=0 out_seq=0 2024-08-30T09:15:15.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.706+0000 7fe5a1ffb640 1 -- 172.21.15.42:0/639644299 <== mds.1 v2:172.21.15.190:6838/719981110 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x558ccbf7a5d0 con 0x558ccbc1a9f0 2024-08-30T09:15:15.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.706+0000 7fe5a1ffb640 10 client.27598 handle_command_reply: tid=0 2024-08-30T09:15:15.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.706+0000 7fe5c752d740 1 client.27598 shutdown 2024-08-30T09:15:15.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.706+0000 7fe5c752d740 1 -- 172.21.15.42:0/639644299 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558ccbc331b0 msgr2=0x558ccc00de40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:15:15.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.706+0000 7fe5c752d740 1 --2- 172.21.15.42:0/639644299 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558ccbc331b0 0x558ccc00de40 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fe5ac0c93d0 tx=0x7fe5ac0c6a10 comp rx=0 tx=0).stop 2024-08-30T09:15:15.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.709+0000 7fe5c752d740 1 -- 172.21.15.42:0/639644299 shutdown_connections 2024-08-30T09:15:15.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.709+0000 7fe5c752d740 1 --2- 172.21.15.42:0/639644299 >> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] conn(0x558ccbc1a9f0 0x558ccc02fe10 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:15.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.709+0000 7fe5c752d740 1 --2- 172.21.15.42:0/639644299 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558ccbc5c070 0x558ccc016eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:15.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.709+0000 7fe5c752d740 1 --2- 172.21.15.42:0/639644299 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558ccbc3b390 0x558ccc00e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:15.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.709+0000 7fe5c752d740 1 --2- 172.21.15.42:0/639644299 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558ccbc331b0 0x558ccc00de40 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:15.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.709+0000 7fe5c752d740 1 -- 172.21.15.42:0/639644299 >> 172.21.15.42:0/639644299 conn(0x558ccbc50460 msgr2=0x558ccc00ccf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:15:15.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.709+0000 7fe5a1ffb640 0 client.27598 ms_handle_reset on v2:172.21.15.190:6838/719981110 2024-08-30T09:15:15.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.710+0000 7fe5c752d740 1 -- 172.21.15.42:0/639644299 shutdown_connections 2024-08-30T09:15:15.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.710+0000 7fe5c752d740 1 -- 172.21.15.42:0/639644299 wait complete. 2024-08-30T09:15:15.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.710+0000 7fe5c752d740 20 client.27598 trim_cache size 0 max 16384 2024-08-30T09:15:15.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.746+0000 7fe5c752d740 1 Processor -- start 2024-08-30T09:15:15.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.747+0000 7fe5c752d740 1 -- start start 2024-08-30T09:15:15.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.747+0000 7fe5c752d740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558ccbc331b0 0x558ccc00e110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:15.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.747+0000 7fe5c752d740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558ccbc3b390 0x558ccc003880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:15.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.747+0000 7fe5c752d740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558ccbc5c070 0x558ccc003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:15.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.747+0000 7fe5c752d740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fe5bc1b0e40 con 0x558ccbc5c070 2024-08-30T09:15:15.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.747+0000 7fe5c752d740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe5bc1ae6e0 con 0x558ccbc331b0 2024-08-30T09:15:15.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.747+0000 7fe5ba59c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558ccbc331b0 0x558ccc00e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:15:15.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.747+0000 7fe5c752d740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fe5bc1ac530 con 0x558ccbc3b390 2024-08-30T09:15:15.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.747+0000 7fe5ba59c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558ccbc331b0 0x558ccc00e110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:60474/0 (socket says 172.21.15.42:60474) 2024-08-30T09:15:15.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.747+0000 7fe5ba59c640 1 -- 172.21.15.42:0/2415705490 learned_addr learned my addr 172.21.15.42:0/2415705490 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:15:15.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.747+0000 7fe5b9d9b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558ccbc3b390 0x558ccc003880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:15:15.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.748+0000 7fe5ba59c640 1 -- 172.21.15.42:0/2415705490 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558ccbc3b390 msgr2=0x558ccc003880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:15:15.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.748+0000 7fe5ba59c640 1 --2- 172.21.15.42:0/2415705490 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558ccbc3b390 0x558ccc003880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:15.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.748+0000 7fe5ba59c640 1 -- 172.21.15.42:0/2415705490 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558ccbc5c070 msgr2=0x558ccc003dc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-30T09:15:15.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.748+0000 7fe5ba59c640 1 --2- 172.21.15.42:0/2415705490 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558ccbc5c070 0x558ccc003dc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:15.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.748+0000 7fe5ba59c640 1 -- 172.21.15.42:0/2415705490 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ccc00be50 con 0x558ccbc331b0 2024-08-30T09:15:15.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.749+0000 7fe5ba59c640 1 --2- 172.21.15.42:0/2415705490 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558ccbc331b0 0x558ccc00e110 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fe5ac005e30 tx=0x7fe5ac0d5050 comp rx=0 tx=0).ready entity=mon.2 client_cookie=109db69d02097314 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:15:15.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.750+0000 7fe5a3fff640 1 -- 172.21.15.42:0/2415705490 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5ac0c8ec0 con 0x558ccbc331b0 2024-08-30T09:15:15.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.750+0000 7fe5a3fff640 1 -- 172.21.15.42:0/2415705490 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe5ac04ee70 con 0x558ccbc331b0 2024-08-30T09:15:15.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.751+0000 7fe5a3fff640 1 -- 172.21.15.42:0/2415705490 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5ac0c9060 con 0x558ccbc331b0 2024-08-30T09:15:15.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.753+0000 7fe5c752d740 1 -- 172.21.15.42:0/2415705490 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558ccbc331b0 msgr2=0x558ccc00e110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:15:15.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.753+0000 7fe5c752d740 1 --2- 172.21.15.42:0/2415705490 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558ccbc331b0 0x558ccc00e110 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fe5ac005e30 tx=0x7fe5ac0d5050 comp rx=0 tx=0).stop 2024-08-30T09:15:15.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.754+0000 7fe5c752d740 1 -- 172.21.15.42:0/2415705490 shutdown_connections 2024-08-30T09:15:15.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.754+0000 7fe5c752d740 1 --2- 172.21.15.42:0/2415705490 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558ccbc3b390 0x558ccc003880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:15.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.754+0000 7fe5c752d740 1 --2- 172.21.15.42:0/2415705490 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558ccbc5c070 0x558ccc003dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:15.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.754+0000 7fe5c752d740 1 --2- 172.21.15.42:0/2415705490 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558ccbc331b0 0x558ccc00e110 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:15.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.754+0000 7fe5c752d740 1 -- 172.21.15.42:0/2415705490 >> 172.21.15.42:0/2415705490 conn(0x558ccbc50460 msgr2=0x558ccc00da00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:15:15.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.755+0000 7fe5c752d740 1 -- 172.21.15.42:0/2415705490 shutdown_connections 2024-08-30T09:15:15.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.755+0000 7fe5c752d740 1 -- 172.21.15.42:0/2415705490 wait complete. 2024-08-30T09:15:15.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.756+0000 7fe5c752d740 1 -- start start 2024-08-30T09:15:15.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.756+0000 7fe5c752d740 1 Processor -- start 2024-08-30T09:15:15.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.756+0000 7fe5c752d740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558ccbc331b0 0x558ccc012730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:15.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.756+0000 7fe5c752d740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558ccbc3b390 0x558ccc012d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:15.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.756+0000 7fe5c752d740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558ccbc5c070 0x558ccc092810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:15.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.756+0000 7fe5c752d740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fe5bc1acba0 con 0x558ccbc5c070 2024-08-30T09:15:15.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.756+0000 7fe5c752d740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe5bc1ae010 con 0x558ccbc3b390 2024-08-30T09:15:15.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.756+0000 7fe5c752d740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fe5bc1ad270 con 0x558ccbc331b0 2024-08-30T09:15:15.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.756+0000 7fe5c752d740 10 client.0 _ll_register_callbacks cb 0x7fe5ac0d3f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:15:15.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.757+0000 7fe5ba59c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558ccbc331b0 0x558ccc012730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:15:15.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.757+0000 7fe5ba59c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558ccbc331b0 0x558ccc012730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:40436/0 (socket says 172.21.15.42:40436) 2024-08-30T09:15:15.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.757+0000 7fe5ba59c640 1 -- 172.21.15.42:0/3888069320 learned_addr learned my addr 172.21.15.42:0/3888069320 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:15:15.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.757+0000 7fe5b9d9b640 1 --2- 172.21.15.42:0/3888069320 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558ccbc3b390 0x558ccc012d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:15:15.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.758+0000 7fe5ba59c640 1 -- 172.21.15.42:0/3888069320 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558ccbc3b390 msgr2=0x558ccc012d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:15:15.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.758+0000 7fe5ba59c640 1 --2- 172.21.15.42:0/3888069320 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558ccbc3b390 0x558ccc012d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:15.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.758+0000 7fe5ba59c640 1 -- 172.21.15.42:0/3888069320 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558ccbc5c070 msgr2=0x558ccc092810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:15:15.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.758+0000 7fe5ba59c640 1 --2- 172.21.15.42:0/3888069320 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558ccbc5c070 0x558ccc092810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:15.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.758+0000 7fe5ba59c640 1 -- 172.21.15.42:0/3888069320 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ccc092e80 con 0x558ccbc331b0 2024-08-30T09:15:15.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.758+0000 7fe5ba59c640 1 --2- 172.21.15.42:0/3888069320 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558ccbc331b0 0x558ccc012730 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fe5ac0c93d0 tx=0x7fe5ac0095c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e0d2dc8ce3d672b server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:15:15.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.758+0000 7fe5a1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-08-30T09:15:15.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.758+0000 7fe5a1ffb640 1 -- 172.21.15.42:0/3888069320 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5ac0c9230 con 0x558ccbc331b0 2024-08-30T09:15:15.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.758+0000 7fe5c752d740 1 -- 172.21.15.42:0/3888069320 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe5ac04ee70 con 0x558ccbc331b0 2024-08-30T09:15:15.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.758+0000 7fe5a1ffb640 1 -- 172.21.15.42:0/3888069320 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7fe5ac037a80 con 0x558ccbc331b0 2024-08-30T09:15:15.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.758+0000 7fe5a1ffb640 1 -- 172.21.15.42:0/3888069320 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7fe5ac0c9400 con 0x558ccbc331b0 2024-08-30T09:15:15.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.758+0000 7fe5c752d740 10 client.27604 fetch_fsmap learned FSMap version 48 2024-08-30T09:15:15.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.758+0000 7fe5c752d740 1 -- 172.21.15.42:0/3888069320 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x558ccc092e80 con 0x558ccbc331b0 2024-08-30T09:15:15.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.759+0000 7fe5a1ffb640 1 -- 172.21.15.42:0/3888069320 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe5ac04ee70 con 0x558ccbc331b0 2024-08-30T09:15:15.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.759+0000 7fe5a1ffb640 1 -- 172.21.15.42:0/3888069320 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 48) ==== 8366+0+0 (secure 0 0 0) 0x7fe5ac0c8800 con 0x558ccbc331b0 2024-08-30T09:15:15.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.760+0000 7fe5c752d740 10 client.27604 fetch_fsmap finished waiting for FSMap version 48 2024-08-30T09:15:15.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.760+0000 7fe5c752d740 10 client.27604 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-08-30T09:15:15.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.760+0000 7fe5c752d740 20 client.27604 populate_metadata read hostname 'smithi042' 2024-08-30T09:15:15.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.760+0000 7fe5c752d740 1 --2- 172.21.15.42:0/3888069320 >> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] conn(0x558ccbbefda0 0x558ccc09d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:15.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.760+0000 7fe5c752d740 4 client.27604 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-30T09:15:15.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.760+0000 7fe5c752d740 1 -- 172.21.15.42:0/3888069320 --> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558ccbf76070 con 0x558ccbbefda0 2024-08-30T09:15:15.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.760+0000 7fe5b9d9b640 1 --2- 172.21.15.42:0/3888069320 >> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] conn(0x558ccbbefda0 0x558ccc09d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:15:15.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.761+0000 7fe5a1ffb640 10 client.27604 ms_handle_connect on v2:172.21.15.190:6838/719981110 2024-08-30T09:15:15.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.761+0000 7fe5b9d9b640 1 --2- 172.21.15.42:0/3888069320 >> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] conn(0x558ccbbefda0 0x558ccc09d6e0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=84d709f2f0428d0c server_cookie=385ccb4146d6028c in_seq=0 out_seq=0 2024-08-30T09:15:15.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.761+0000 7fe5a1ffb640 1 -- 172.21.15.42:0/3888069320 <== mds.1 v2:172.21.15.190:6838/719981110 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558ccbf76070 con 0x558ccbbefda0 2024-08-30T09:15:15.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.761+0000 7fe5a1ffb640 10 client.27604 handle_command_reply: tid=0 2024-08-30T09:15:15.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.761+0000 7fe5c752d740 1 client.27604 shutdown 2024-08-30T09:15:15.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.761+0000 7fe5c752d740 1 -- 172.21.15.42:0/3888069320 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558ccbc331b0 msgr2=0x558ccc012730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:15:15.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.761+0000 7fe5c752d740 1 --2- 172.21.15.42:0/3888069320 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558ccbc331b0 0x558ccc012730 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fe5ac0c93d0 tx=0x7fe5ac0095c0 comp rx=0 tx=0).stop 2024-08-30T09:15:15.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.761+0000 7fe5c752d740 1 -- 172.21.15.42:0/3888069320 shutdown_connections 2024-08-30T09:15:15.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.761+0000 7fe5c752d740 1 --2- 172.21.15.42:0/3888069320 >> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] conn(0x558ccbbefda0 0x558ccc09d6e0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:15.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.761+0000 7fe5c752d740 1 --2- 172.21.15.42:0/3888069320 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558ccbc5c070 0x558ccc092810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:15.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.761+0000 7fe5c752d740 1 --2- 172.21.15.42:0/3888069320 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558ccbc3b390 0x558ccc012d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:15.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.761+0000 7fe5c752d740 1 --2- 172.21.15.42:0/3888069320 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558ccbc331b0 0x558ccc012730 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:15.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.761+0000 7fe5c752d740 1 -- 172.21.15.42:0/3888069320 >> 172.21.15.42:0/3888069320 conn(0x558ccbc50460 msgr2=0x558ccc00ab90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:15:15.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.761+0000 7fe5a1ffb640 0 client.27604 ms_handle_reset on v2:172.21.15.190:6838/719981110 2024-08-30T09:15:15.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.762+0000 7fe5c752d740 1 -- 172.21.15.42:0/3888069320 shutdown_connections 2024-08-30T09:15:15.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.762+0000 7fe5c752d740 1 -- 172.21.15.42:0/3888069320 wait complete. 2024-08-30T09:15:15.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.762+0000 7fe5c752d740 20 client.27604 trim_cache size 0 max 16384 2024-08-30T09:15:15.766 INFO:teuthology.orchestra.run.smithi042.stdout:[] 2024-08-30T09:15:15.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.762+0000 7fe5c0fb8640 1 -- 172.21.15.42:0/4073143483 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fe5a40b4bb0 msgr2=0x7fe5a40b70c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:15:15.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.762+0000 7fe5c0fb8640 1 --2- 172.21.15.42:0/4073143483 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fe5a40b4bb0 0x7fe5a40b70c0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fe5b40058c0 tx=0x7fe5b4005f90 comp rx=0 tx=0).stop 2024-08-30T09:15:15.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.762+0000 7fe5c0fb8640 1 -- 172.21.15.42:0/4073143483 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe5bc102800 msgr2=0x7fe5bc197fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:15:15.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.762+0000 7fe5c0fb8640 1 --2- 172.21.15.42:0/4073143483 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe5bc102800 0x7fe5bc197fd0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fe5ac02f2d0 tx=0x7fe5ac003ee0 comp rx=0 tx=0).stop 2024-08-30T09:15:15.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.763+0000 7fe5c0fb8640 1 -- 172.21.15.42:0/4073143483 shutdown_connections 2024-08-30T09:15:15.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.763+0000 7fe5c0fb8640 1 --2- 172.21.15.42:0/4073143483 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fe5a40b4bb0 0x7fe5a40b70c0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:15.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.763+0000 7fe5c0fb8640 1 --2- 172.21.15.42:0/4073143483 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe5bc108a90 0x7fe5bc19d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:15.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.763+0000 7fe5c0fb8640 1 --2- 172.21.15.42:0/4073143483 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe5bc108140 0x7fe5bc198570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:15.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.763+0000 7fe5c0fb8640 1 --2- 172.21.15.42:0/4073143483 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe5bc102800 0x7fe5bc197fd0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:15.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.763+0000 7fe5c0fb8640 1 -- 172.21.15.42:0/4073143483 >> 172.21.15.42:0/4073143483 conn(0x7fe5bc0794d0 msgr2=0x7fe5bc1011f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:15:15.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.763+0000 7fe5c0fb8640 1 -- 172.21.15.42:0/4073143483 shutdown_connections 2024-08-30T09:15:15.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:15.763+0000 7fe5c0fb8640 1 -- 172.21.15.42:0/4073143483 wait complete. 2024-08-30T09:15:15.953 INFO:teuthology.orchestra.run.smithi042.stderr:progress: 870.0 s, 412.0 tps, lat 238.219 ms stddev 289.594 2024-08-30T09:15:17.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:15:16 smithi042 ceph-mon[18652]: pgmap v808: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 3.9 MiB/s wr, 445 op/s 2024-08-30T09:15:17.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:15:16 smithi190 ceph-mon[22001]: pgmap v808: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 3.9 MiB/s wr, 445 op/s 2024-08-30T09:15:17.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:15:16 smithi167 ceph-mon[22063]: pgmap v808: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 3.9 MiB/s wr, 445 op/s 2024-08-30T09:15:17.813 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.811+0000 7fbdba719640 1 -- 172.21.15.42:0/2699950807 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbdb4102b70 msgr2=0x7fbdb4102f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:15:17.813 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.811+0000 7fbdba719640 1 --2- 172.21.15.42:0/2699950807 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbdb4102b70 0x7fbdb4102f80 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fbda8009900 tx=0x7fbda802a410 comp rx=0 tx=0).stop 2024-08-30T09:15:17.814 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.811+0000 7fbdba719640 1 -- 172.21.15.42:0/2699950807 shutdown_connections 2024-08-30T09:15:17.814 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.811+0000 7fbdba719640 1 --2- 172.21.15.42:0/2699950807 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbdb4103510 0x7fbdb4109eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:17.814 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.811+0000 7fbdba719640 1 --2- 172.21.15.42:0/2699950807 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbdb4102b70 0x7fbdb4102f80 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:17.814 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.811+0000 7fbdba719640 1 --2- 172.21.15.42:0/2699950807 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbdb4101af0 0x7fbdb4101f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:17.814 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.811+0000 7fbdba719640 1 -- 172.21.15.42:0/2699950807 >> 172.21.15.42:0/2699950807 conn(0x7fbdb40fd430 msgr2=0x7fbdb40ff8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:15:17.814 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.811+0000 7fbdba719640 1 -- 172.21.15.42:0/2699950807 shutdown_connections 2024-08-30T09:15:17.814 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.811+0000 7fbdba719640 1 -- 172.21.15.42:0/2699950807 wait complete. 2024-08-30T09:15:17.814 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.811+0000 7fbdba719640 1 Processor -- start 2024-08-30T09:15:17.814 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.812+0000 7fbdba719640 1 -- start start 2024-08-30T09:15:17.816 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.812+0000 7fbdba719640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbdb4101af0 0x7fbdb4197ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:17.816 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.812+0000 7fbdba719640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbdb4102b70 0x7fbdb4198140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:17.816 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.812+0000 7fbdba719640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbdb4103510 0x7fbdb4198cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:17.816 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.812+0000 7fbdba719640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fbdb410cee0 con 0x7fbdb4101af0 2024-08-30T09:15:17.816 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.812+0000 7fbdba719640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fbdb410cd60 con 0x7fbdb4102b70 2024-08-30T09:15:17.816 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.812+0000 7fbdba719640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fbdb410d060 con 0x7fbdb4103510 2024-08-30T09:15:17.816 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.812+0000 7fbdb3fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbdb4101af0 0x7fbdb4197ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:15:17.816 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.812+0000 7fbdb3fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbdb4101af0 0x7fbdb4197ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:47782/0 (socket says 172.21.15.42:47782) 2024-08-30T09:15:17.817 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.812+0000 7fbdb3fff640 1 -- 172.21.15.42:0/2506454894 learned_addr learned my addr 172.21.15.42:0/2506454894 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:15:17.817 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.812+0000 7fbdb8cb6640 1 --2- 172.21.15.42:0/2506454894 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbdb4103510 0x7fbdb4198cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:15:17.817 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.812+0000 7fbdb37fe640 1 --2- 172.21.15.42:0/2506454894 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbdb4102b70 0x7fbdb4198140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:15:17.817 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.812+0000 7fbdb3fff640 1 -- 172.21.15.42:0/2506454894 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbdb4103510 msgr2=0x7fbdb4198cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:15:17.817 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.812+0000 7fbdb3fff640 1 --2- 172.21.15.42:0/2506454894 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbdb4103510 0x7fbdb4198cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:17.817 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.812+0000 7fbdb3fff640 1 -- 172.21.15.42:0/2506454894 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbdb4102b70 msgr2=0x7fbdb4198140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:15:17.817 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.812+0000 7fbdb3fff640 1 --2- 172.21.15.42:0/2506454894 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbdb4102b70 0x7fbdb4198140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:17.817 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.812+0000 7fbdb3fff640 1 -- 172.21.15.42:0/2506454894 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdb4199510 con 0x7fbdb4101af0 2024-08-30T09:15:17.817 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.813+0000 7fbdb3fff640 1 --2- 172.21.15.42:0/2506454894 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbdb4101af0 0x7fbdb4197ad0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fbda400edd0 tx=0x7fbda400a720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:15:17.817 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.813+0000 7fbdb17fa640 1 -- 172.21.15.42:0/2506454894 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbda4004220 con 0x7fbdb4101af0 2024-08-30T09:15:17.817 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.813+0000 7fbdb17fa640 1 -- 172.21.15.42:0/2506454894 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbda400ae30 con 0x7fbdb4101af0 2024-08-30T09:15:17.817 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.813+0000 7fbdba719640 1 -- 172.21.15.42:0/2506454894 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbdb4199800 con 0x7fbdb4101af0 2024-08-30T09:15:17.817 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.813+0000 7fbdb17fa640 1 -- 172.21.15.42:0/2506454894 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbda4004d00 con 0x7fbdb4101af0 2024-08-30T09:15:17.817 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.813+0000 7fbdba719640 1 -- 172.21.15.42:0/2506454894 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbdb419fef0 con 0x7fbdb4101af0 2024-08-30T09:15:17.824 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.816+0000 7fbdc1686740 1 Processor -- start 2024-08-30T09:15:17.824 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.816+0000 7fbdc1686740 1 -- start start 2024-08-30T09:15:17.824 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.816+0000 7fbdc1686740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5580694761b0 0x55806981e390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:17.824 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.816+0000 7fbdc1686740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55806944d2f0 0x55806981ea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:17.824 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.816+0000 7fbdc1686740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5580694554d0 0x55806981f550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:17.824 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.816+0000 7fbdc1686740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55806978f000 con 0x5580694554d0 2024-08-30T09:15:17.825 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.816+0000 7fbdc1686740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55806977e590 con 0x5580694761b0 2024-08-30T09:15:17.825 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.816+0000 7fbdc1686740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x558069782540 con 0x55806944d2f0 2024-08-30T09:15:17.825 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.816+0000 7fbdb37fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5580694554d0 0x55806981f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:15:17.825 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.816+0000 7fbdb37fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5580694554d0 0x55806981f550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:47786/0 (socket says 172.21.15.42:47786) 2024-08-30T09:15:17.825 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.816+0000 7fbdb37fe640 1 -- 172.21.15.42:0/1309582510 learned_addr learned my addr 172.21.15.42:0/1309582510 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:15:17.825 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.816+0000 7fbdb17fa640 1 -- 172.21.15.42:0/2506454894 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7fbda4005020 con 0x7fbdb4101af0 2024-08-30T09:15:17.825 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.816+0000 7fbdb8cb6640 1 --2- 172.21.15.42:0/1309582510 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5580694761b0 0x55806981e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:15:17.825 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.816+0000 7fbdb37fe640 1 -- 172.21.15.42:0/1309582510 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55806944d2f0 msgr2=0x55806981ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:15:17.825 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.816+0000 7fbdb37fe640 1 --2- 172.21.15.42:0/1309582510 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55806944d2f0 0x55806981ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:17.825 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.816+0000 7fbdb37fe640 1 -- 172.21.15.42:0/1309582510 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5580694761b0 msgr2=0x55806981e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:15:17.825 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.816+0000 7fbdb37fe640 1 --2- 172.21.15.42:0/1309582510 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5580694761b0 0x55806981e390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:17.825 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.816+0000 7fbdb37fe640 1 -- 172.21.15.42:0/1309582510 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5580694e1790 con 0x5580694554d0 2024-08-30T09:15:17.825 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.816+0000 7fbdb8cb6640 1 --2- 172.21.15.42:0/1309582510 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5580694761b0 0x55806981e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-30T09:15:17.826 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.817+0000 7fbdb37fe640 1 --2- 172.21.15.42:0/1309582510 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5580694554d0 0x55806981f550 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fbda802a930 tx=0x7fbda8031650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49939b7bcd3dbe9 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:15:17.826 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.817+0000 7fbd9a7fc640 1 -- 172.21.15.42:0/1309582510 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbda8035030 con 0x5580694554d0 2024-08-30T09:15:17.826 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.817+0000 7fbdb17fa640 1 --2- 172.21.15.42:0/2506454894 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fbd9c0b4bb0 0x7fbd9c0b70c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:17.826 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.817+0000 7fbd9a7fc640 1 -- 172.21.15.42:0/1309582510 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbda8040070 con 0x5580694554d0 2024-08-30T09:15:17.826 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.817+0000 7fbd9a7fc640 1 -- 172.21.15.42:0/1309582510 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbda80039f0 con 0x5580694554d0 2024-08-30T09:15:17.826 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.817+0000 7fbdb8cb6640 1 --2- 172.21.15.42:0/2506454894 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fbd9c0b4bb0 0x7fbd9c0b70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:15:17.826 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.817+0000 7fbdb17fa640 1 -- 172.21.15.42:0/2506454894 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7fbda40e5200 con 0x7fbdb4101af0 2024-08-30T09:15:17.826 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.818+0000 7fbdc1686740 1 -- 172.21.15.42:0/1309582510 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5580694554d0 msgr2=0x55806981f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:15:17.826 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.818+0000 7fbdb8cb6640 1 --2- 172.21.15.42:0/2506454894 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fbd9c0b4bb0 0x7fbd9c0b70c0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fbdac005fd0 tx=0x7fbdac009600 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:15:17.826 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.818+0000 7fbdc1686740 1 --2- 172.21.15.42:0/1309582510 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5580694554d0 0x55806981f550 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fbda802a930 tx=0x7fbda8031650 comp rx=0 tx=0).stop 2024-08-30T09:15:17.826 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.818+0000 7fbdc1686740 1 -- 172.21.15.42:0/1309582510 shutdown_connections 2024-08-30T09:15:17.826 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.818+0000 7fbdc1686740 1 --2- 172.21.15.42:0/1309582510 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55806944d2f0 0x55806981ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:17.826 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.818+0000 7fbdc1686740 1 --2- 172.21.15.42:0/1309582510 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5580694554d0 0x55806981f550 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:17.826 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.818+0000 7fbdc1686740 1 --2- 172.21.15.42:0/1309582510 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5580694761b0 0x55806981e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:17.827 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.818+0000 7fbdc1686740 1 -- 172.21.15.42:0/1309582510 >> 172.21.15.42:0/1309582510 conn(0x55806946a300 msgr2=0x558069817c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:15:17.827 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.818+0000 7fbdc1686740 1 -- 172.21.15.42:0/1309582510 shutdown_connections 2024-08-30T09:15:17.827 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.818+0000 7fbdc1686740 1 -- 172.21.15.42:0/1309582510 wait complete. 2024-08-30T09:15:17.827 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.818+0000 7fbdc1686740 1 -- start start 2024-08-30T09:15:17.827 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.818+0000 7fbdc1686740 1 Processor -- start 2024-08-30T09:15:17.827 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.819+0000 7fbdc1686740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55806944d2f0 0x558069827e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:17.827 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.819+0000 7fbdc1686740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5580694554d0 0x5580698284f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:17.827 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.819+0000 7fbdc1686740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5580694761b0 0x558069829060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:17.827 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.819+0000 7fbdc1686740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x5580696b7ab0 con 0x55806944d2f0 2024-08-30T09:15:17.827 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.819+0000 7fbdc1686740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x558069705680 con 0x5580694554d0 2024-08-30T09:15:17.827 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.819+0000 7fbdc1686740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x558069820010 con 0x5580694761b0 2024-08-30T09:15:17.827 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.819+0000 7fbdc1686740 10 client.0 _ll_register_callbacks cb 0x55806948da00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:15:17.827 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.819+0000 7fbdb3fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55806944d2f0 0x558069827e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:15:17.828 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.819+0000 7fbdb3fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55806944d2f0 0x558069827e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:47788/0 (socket says 172.21.15.42:47788) 2024-08-30T09:15:17.828 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.819+0000 7fbdb3fff640 1 -- 172.21.15.42:0/4076610474 learned_addr learned my addr 172.21.15.42:0/4076610474 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:15:17.828 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.819+0000 7fbdb8cb6640 1 --2- 172.21.15.42:0/4076610474 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5580694761b0 0x558069829060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:15:17.828 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.819+0000 7fbdb37fe640 1 --2- 172.21.15.42:0/4076610474 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5580694554d0 0x5580698284f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:15:17.828 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.819+0000 7fbdb3fff640 1 -- 172.21.15.42:0/4076610474 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5580694761b0 msgr2=0x558069829060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:15:17.828 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.819+0000 7fbdb3fff640 1 --2- 172.21.15.42:0/4076610474 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5580694761b0 0x558069829060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:17.828 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.819+0000 7fbdb3fff640 1 -- 172.21.15.42:0/4076610474 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5580694554d0 msgr2=0x5580698284f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:15:17.828 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.819+0000 7fbdb3fff640 1 --2- 172.21.15.42:0/4076610474 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5580694554d0 0x5580698284f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:17.828 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.819+0000 7fbdb3fff640 1 -- 172.21.15.42:0/4076610474 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55806955a620 con 0x55806944d2f0 2024-08-30T09:15:17.828 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.819+0000 7fbdb37fe640 1 --2- 172.21.15.42:0/4076610474 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5580694554d0 0x5580698284f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-30T09:15:17.828 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.819+0000 7fbdb3fff640 1 --2- 172.21.15.42:0/4076610474 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55806944d2f0 0x558069827e30 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fbda40e29c0 tx=0x7fbda40ac820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61485815a3fe22bf server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:15:17.828 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.819+0000 7fbd937fe640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-08-30T09:15:17.828 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.819+0000 7fbd937fe640 1 -- 172.21.15.42:0/4076610474 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbda401e040 con 0x55806944d2f0 2024-08-30T09:15:17.829 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.820+0000 7fbd937fe640 1 -- 172.21.15.42:0/4076610474 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7fbda4005310 con 0x55806944d2f0 2024-08-30T09:15:17.829 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.820+0000 7fbdc1686740 1 -- 172.21.15.42:0/4076610474 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbda8040070 con 0x55806944d2f0 2024-08-30T09:15:17.829 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.820+0000 7fbd937fe640 1 -- 172.21.15.42:0/4076610474 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7fbda40a8040 con 0x55806944d2f0 2024-08-30T09:15:17.829 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.820+0000 7fbdc1686740 10 client.17841 fetch_fsmap learned FSMap version 48 2024-08-30T09:15:17.829 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.820+0000 7fbdc1686740 1 -- 172.21.15.42:0/4076610474 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x558069832020 con 0x55806944d2f0 2024-08-30T09:15:17.829 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.820+0000 7fbd937fe640 1 -- 172.21.15.42:0/4076610474 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbda8040070 con 0x55806944d2f0 2024-08-30T09:15:17.829 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.820+0000 7fbd937fe640 1 -- 172.21.15.42:0/4076610474 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 48) ==== 8366+0+0 (secure 0 0 0) 0x7fbda400b040 con 0x55806944d2f0 2024-08-30T09:15:17.829 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.820+0000 7fbdc1686740 10 client.17841 fetch_fsmap finished waiting for FSMap version 48 2024-08-30T09:15:17.829 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.820+0000 7fbdc1686740 10 client.17841 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-30T09:15:17.829 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.820+0000 7fbdc1686740 20 client.17841 populate_metadata read hostname 'smithi042' 2024-08-30T09:15:17.829 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.820+0000 7fbdc1686740 1 --2- 172.21.15.42:0/4076610474 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x558069434d80 0x5580698357f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:17.829 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.820+0000 7fbdc1686740 4 client.17841 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-30T09:15:17.829 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.820+0000 7fbdc1686740 1 -- 172.21.15.42:0/4076610474 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558069790f50 con 0x558069434d80 2024-08-30T09:15:17.829 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.820+0000 7fbdb37fe640 1 --2- 172.21.15.42:0/4076610474 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x558069434d80 0x5580698357f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:15:17.830 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.821+0000 7fbd937fe640 10 client.17841 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:15:17.830 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.821+0000 7fbdb37fe640 1 --2- 172.21.15.42:0/4076610474 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x558069434d80 0x5580698357f0 crc :-1 s=READY pgs=293 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1eab1187676bc8c7 server_cookie=4fa6bb2dd7fbfd99 in_seq=0 out_seq=0 2024-08-30T09:15:17.830 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.821+0000 7fbd937fe640 1 -- 172.21.15.42:0/4076610474 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x558069790f50 con 0x558069434d80 2024-08-30T09:15:17.830 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.821+0000 7fbd937fe640 10 client.17841 handle_command_reply: tid=0 2024-08-30T09:15:17.830 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.821+0000 7fbdc1686740 1 client.17841 shutdown 2024-08-30T09:15:17.830 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.822+0000 7fbdc1686740 1 -- 172.21.15.42:0/4076610474 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55806944d2f0 msgr2=0x558069827e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:15:17.830 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.822+0000 7fbdc1686740 1 --2- 172.21.15.42:0/4076610474 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55806944d2f0 0x558069827e30 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fbda40e29c0 tx=0x7fbda40ac820 comp rx=0 tx=0).stop 2024-08-30T09:15:17.830 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.822+0000 7fbdc1686740 1 -- 172.21.15.42:0/4076610474 shutdown_connections 2024-08-30T09:15:17.830 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.822+0000 7fbdc1686740 1 --2- 172.21.15.42:0/4076610474 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x558069434d80 0x5580698357f0 crc :-1 s=READY pgs=293 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:17.830 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.822+0000 7fbdc1686740 1 --2- 172.21.15.42:0/4076610474 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5580694761b0 0x558069829060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:17.830 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.822+0000 7fbdc1686740 1 --2- 172.21.15.42:0/4076610474 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5580694554d0 0x5580698284f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:17.830 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.822+0000 7fbd937fe640 0 client.17841 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:15:17.830 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.822+0000 7fbdc1686740 1 --2- 172.21.15.42:0/4076610474 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55806944d2f0 0x558069827e30 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:17.830 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.822+0000 7fbdc1686740 1 -- 172.21.15.42:0/4076610474 >> 172.21.15.42:0/4076610474 conn(0x55806946a300 msgr2=0x558069816170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:15:17.831 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.822+0000 7fbdc1686740 1 -- 172.21.15.42:0/4076610474 shutdown_connections 2024-08-30T09:15:17.831 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.822+0000 7fbdc1686740 1 -- 172.21.15.42:0/4076610474 wait complete. 2024-08-30T09:15:17.831 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.822+0000 7fbdc1686740 20 client.17841 trim_cache size 0 max 16384 2024-08-30T09:15:17.839 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.836+0000 7fbdc1686740 1 Processor -- start 2024-08-30T09:15:17.839 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.836+0000 7fbdc1686740 1 -- start start 2024-08-30T09:15:17.840 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.836+0000 7fbdc1686740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55806944d2f0 0x55806981dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:17.840 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.837+0000 7fbdc1686740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5580694554d0 0x55806981f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:17.840 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.837+0000 7fbdc1686740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5580694761b0 0x558069822e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:17.840 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.837+0000 7fbdc1686740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fbdb41acb60 con 0x5580694761b0 2024-08-30T09:15:17.840 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.837+0000 7fbdc1686740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fbdb41aa400 con 0x5580694554d0 2024-08-30T09:15:17.840 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.837+0000 7fbdc1686740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fbdb41a8250 con 0x55806944d2f0 2024-08-30T09:15:17.840 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.837+0000 7fbdb8cb6640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5580694761b0 0x558069822e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:15:17.840 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.837+0000 7fbdb8cb6640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5580694761b0 0x558069822e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:47798/0 (socket says 172.21.15.42:47798) 2024-08-30T09:15:17.840 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.837+0000 7fbdb8cb6640 1 -- 172.21.15.42:0/818577547 learned_addr learned my addr 172.21.15.42:0/818577547 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:15:17.840 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.837+0000 7fbdb3fff640 1 --2- 172.21.15.42:0/818577547 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55806944d2f0 0x55806981dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:15:17.840 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.837+0000 7fbdb37fe640 1 --2- 172.21.15.42:0/818577547 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5580694554d0 0x55806981f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:15:17.840 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.837+0000 7fbdb8cb6640 1 -- 172.21.15.42:0/818577547 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55806944d2f0 msgr2=0x55806981dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:15:17.841 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.837+0000 7fbdb8cb6640 1 --2- 172.21.15.42:0/818577547 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55806944d2f0 0x55806981dfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:17.841 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.837+0000 7fbdb8cb6640 1 -- 172.21.15.42:0/818577547 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5580694554d0 msgr2=0x55806981f1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:15:17.841 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.837+0000 7fbdb8cb6640 1 --2- 172.21.15.42:0/818577547 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5580694554d0 0x55806981f1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:17.841 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.837+0000 7fbdb8cb6640 1 -- 172.21.15.42:0/818577547 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55806981ec10 con 0x5580694761b0 2024-08-30T09:15:17.841 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.837+0000 7fbdb3fff640 1 --2- 172.21.15.42:0/818577547 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55806944d2f0 0x55806981dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-30T09:15:17.841 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.837+0000 7fbdb37fe640 1 --2- 172.21.15.42:0/818577547 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5580694554d0 0x55806981f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-30T09:15:17.841 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.837+0000 7fbdb8cb6640 1 --2- 172.21.15.42:0/818577547 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5580694761b0 0x558069822e00 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fbdac010700 tx=0x7fbdac019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bd8a8c54afeec57 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:15:17.841 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.837+0000 7fbd99ffb640 1 -- 172.21.15.42:0/818577547 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdac004d20 con 0x5580694761b0 2024-08-30T09:15:17.841 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.837+0000 7fbd99ffb640 1 -- 172.21.15.42:0/818577547 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbdac01ca70 con 0x5580694761b0 2024-08-30T09:15:17.841 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.837+0000 7fbd99ffb640 1 -- 172.21.15.42:0/818577547 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdac005780 con 0x5580694761b0 2024-08-30T09:15:17.844 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.838+0000 7fbdc1686740 1 -- 172.21.15.42:0/818577547 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5580694761b0 msgr2=0x558069822e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:15:17.844 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.838+0000 7fbdc1686740 1 --2- 172.21.15.42:0/818577547 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5580694761b0 0x558069822e00 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fbdac010700 tx=0x7fbdac019000 comp rx=0 tx=0).stop 2024-08-30T09:15:17.844 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.838+0000 7fbdc1686740 1 -- 172.21.15.42:0/818577547 shutdown_connections 2024-08-30T09:15:17.844 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.838+0000 7fbdc1686740 1 --2- 172.21.15.42:0/818577547 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5580694761b0 0x558069822e00 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:17.845 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.838+0000 7fbdc1686740 1 --2- 172.21.15.42:0/818577547 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5580694554d0 0x55806981f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:17.845 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.838+0000 7fbdc1686740 1 --2- 172.21.15.42:0/818577547 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55806944d2f0 0x55806981dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:17.845 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.838+0000 7fbdc1686740 1 -- 172.21.15.42:0/818577547 >> 172.21.15.42:0/818577547 conn(0x55806946a300 msgr2=0x558069817550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:15:17.845 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.838+0000 7fbdc1686740 1 -- 172.21.15.42:0/818577547 shutdown_connections 2024-08-30T09:15:17.845 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.838+0000 7fbdc1686740 1 -- 172.21.15.42:0/818577547 wait complete. 2024-08-30T09:15:17.845 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.838+0000 7fbdc1686740 1 -- start start 2024-08-30T09:15:17.845 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.839+0000 7fbdc1686740 1 Processor -- start 2024-08-30T09:15:17.845 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.839+0000 7fbdc1686740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55806944d2f0 0x55806982d000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:17.845 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.839+0000 7fbdc1686740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5580694554d0 0x55806982d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:17.845 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.839+0000 7fbdc1686740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5580694761b0 0x55806982e230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:17.845 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.839+0000 7fbdc1686740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fbdb41a88c0 con 0x5580694761b0 2024-08-30T09:15:17.845 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.839+0000 7fbdc1686740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fbdb41a9d30 con 0x55806944d2f0 2024-08-30T09:15:17.845 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.839+0000 7fbdc1686740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fbdb41a8f90 con 0x5580694554d0 2024-08-30T09:15:17.846 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.839+0000 7fbdc1686740 10 client.0 _ll_register_callbacks cb 0x7fbda40a8910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:15:17.846 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.839+0000 7fbdb8cb6640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5580694761b0 0x55806982e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:15:17.846 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.839+0000 7fbdb3fff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55806944d2f0 0x55806982d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:15:17.846 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.839+0000 7fbdb8cb6640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5580694761b0 0x55806982e230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:47806/0 (socket says 172.21.15.42:47806) 2024-08-30T09:15:17.846 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.839+0000 7fbdb8cb6640 1 -- 172.21.15.42:0/992954570 learned_addr learned my addr 172.21.15.42:0/992954570 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:15:17.846 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.839+0000 7fbdb37fe640 1 --2- 172.21.15.42:0/992954570 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5580694554d0 0x55806982d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:15:17.846 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.839+0000 7fbdb8cb6640 1 -- 172.21.15.42:0/992954570 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5580694554d0 msgr2=0x55806982d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:15:17.846 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.839+0000 7fbdb8cb6640 1 --2- 172.21.15.42:0/992954570 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5580694554d0 0x55806982d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:17.846 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.839+0000 7fbdb8cb6640 1 -- 172.21.15.42:0/992954570 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55806944d2f0 msgr2=0x55806982d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:15:17.846 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.839+0000 7fbdb8cb6640 1 --2- 172.21.15.42:0/992954570 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55806944d2f0 0x55806982d000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:17.846 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.839+0000 7fbdb8cb6640 1 -- 172.21.15.42:0/992954570 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55806982e990 con 0x5580694761b0 2024-08-30T09:15:17.846 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.840+0000 7fbdb8cb6640 1 --2- 172.21.15.42:0/992954570 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5580694761b0 0x55806982e230 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fbdac02a040 tx=0x7fbdac01c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a666bd5cac9bca1d server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:15:17.846 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.840+0000 7fbd937fe640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-08-30T09:15:17.846 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.840+0000 7fbd937fe640 1 -- 172.21.15.42:0/992954570 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdac036020 con 0x5580694761b0 2024-08-30T09:15:17.847 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.840+0000 7fbd937fe640 1 -- 172.21.15.42:0/992954570 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7fbdac005480 con 0x5580694761b0 2024-08-30T09:15:17.847 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.840+0000 7fbdc1686740 1 -- 172.21.15.42:0/992954570 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbdac01ca70 con 0x5580694761b0 2024-08-30T09:15:17.847 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.840+0000 7fbd937fe640 1 -- 172.21.15.42:0/992954570 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7fbdac041050 con 0x5580694761b0 2024-08-30T09:15:17.847 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.840+0000 7fbdc1686740 10 client.17847 fetch_fsmap learned FSMap version 48 2024-08-30T09:15:17.847 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.840+0000 7fbdc1686740 1 -- 172.21.15.42:0/992954570 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55806982fcb0 con 0x5580694761b0 2024-08-30T09:15:17.847 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.840+0000 7fbd937fe640 1 -- 172.21.15.42:0/992954570 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbdac01ca70 con 0x5580694761b0 2024-08-30T09:15:17.847 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.840+0000 7fbd937fe640 1 -- 172.21.15.42:0/992954570 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 48) ==== 8366+0+0 (secure 0 0 0) 0x7fbdac027070 con 0x5580694761b0 2024-08-30T09:15:17.847 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.840+0000 7fbdc1686740 10 client.17847 fetch_fsmap finished waiting for FSMap version 48 2024-08-30T09:15:17.847 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.840+0000 7fbdc1686740 10 client.17847 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-30T09:15:17.847 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.840+0000 7fbdc1686740 20 client.17847 populate_metadata read hostname 'smithi042' 2024-08-30T09:15:17.847 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.840+0000 7fbdc1686740 1 --2- 172.21.15.42:0/992954570 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55806940a1c0 0x5580698a2470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:17.847 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.840+0000 7fbdc1686740 4 client.17847 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-30T09:15:17.847 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.840+0000 7fbdc1686740 1 -- 172.21.15.42:0/992954570 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55806978c9f0 con 0x55806940a1c0 2024-08-30T09:15:17.848 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.841+0000 7fbdb3fff640 1 --2- 172.21.15.42:0/992954570 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55806940a1c0 0x5580698a2470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:15:17.848 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.841+0000 7fbdb3fff640 1 --2- 172.21.15.42:0/992954570 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55806940a1c0 0x5580698a2470 crc :-1 s=READY pgs=294 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=61664bad804d54ce server_cookie=15984402a8d14806 in_seq=0 out_seq=0 2024-08-30T09:15:17.848 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.841+0000 7fbd937fe640 10 client.17847 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:15:17.951 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.949+0000 7fbd937fe640 1 -- 172.21.15.42:0/992954570 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55806978c9f0 con 0x55806940a1c0 2024-08-30T09:15:17.952 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.949+0000 7fbd937fe640 10 client.17847 handle_command_reply: tid=0 2024-08-30T09:15:17.952 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.949+0000 7fbdc1686740 1 client.17847 shutdown 2024-08-30T09:15:17.952 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.949+0000 7fbdc1686740 1 -- 172.21.15.42:0/992954570 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5580694761b0 msgr2=0x55806982e230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:15:17.952 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.949+0000 7fbdc1686740 1 --2- 172.21.15.42:0/992954570 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5580694761b0 0x55806982e230 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fbdac02a040 tx=0x7fbdac01c8e0 comp rx=0 tx=0).stop 2024-08-30T09:15:17.952 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.949+0000 7fbdc1686740 1 -- 172.21.15.42:0/992954570 shutdown_connections 2024-08-30T09:15:17.952 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.949+0000 7fbdc1686740 1 --2- 172.21.15.42:0/992954570 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55806940a1c0 0x5580698a2470 crc :-1 s=READY pgs=294 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:17.952 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.949+0000 7fbdc1686740 1 --2- 172.21.15.42:0/992954570 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5580694554d0 0x55806982d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:17.952 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.949+0000 7fbd937fe640 0 client.17847 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:15:17.952 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.949+0000 7fbdc1686740 1 --2- 172.21.15.42:0/992954570 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5580694761b0 0x55806982e230 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:17.952 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.949+0000 7fbdc1686740 1 --2- 172.21.15.42:0/992954570 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55806944d2f0 0x55806982d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:17.952 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.949+0000 7fbdc1686740 1 -- 172.21.15.42:0/992954570 >> 172.21.15.42:0/992954570 conn(0x55806946a300 msgr2=0x55806981f1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:15:17.953 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.949+0000 7fbdc1686740 1 -- 172.21.15.42:0/992954570 shutdown_connections 2024-08-30T09:15:17.953 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.949+0000 7fbdc1686740 1 -- 172.21.15.42:0/992954570 wait complete. 2024-08-30T09:15:17.953 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.950+0000 7fbdc1686740 20 client.17847 trim_cache size 0 max 16384 2024-08-30T09:15:17.953 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-08-30T09:15:17.953 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2024-08-30T09:15:17.953 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-08-30T09:15:17.953 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-08-30T09:15:17.954 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.950+0000 7fbdba719640 1 -- 172.21.15.42:0/2506454894 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fbd9c0b4bb0 msgr2=0x7fbd9c0b70c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:15:17.954 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.950+0000 7fbdba719640 1 --2- 172.21.15.42:0/2506454894 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fbd9c0b4bb0 0x7fbd9c0b70c0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fbdac005fd0 tx=0x7fbdac009600 comp rx=0 tx=0).stop 2024-08-30T09:15:17.954 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.950+0000 7fbdba719640 1 -- 172.21.15.42:0/2506454894 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbdb4101af0 msgr2=0x7fbdb4197ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:15:17.954 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.950+0000 7fbdba719640 1 --2- 172.21.15.42:0/2506454894 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbdb4101af0 0x7fbdb4197ad0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fbda400edd0 tx=0x7fbda400a720 comp rx=0 tx=0).stop 2024-08-30T09:15:17.954 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.951+0000 7fbdba719640 1 -- 172.21.15.42:0/2506454894 shutdown_connections 2024-08-30T09:15:17.954 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.951+0000 7fbdba719640 1 --2- 172.21.15.42:0/2506454894 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fbd9c0b4bb0 0x7fbd9c0b70c0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:17.954 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.951+0000 7fbdba719640 1 --2- 172.21.15.42:0/2506454894 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbdb4103510 0x7fbdb4198cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:17.954 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.951+0000 7fbdba719640 1 --2- 172.21.15.42:0/2506454894 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbdb4102b70 0x7fbdb4198140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:17.955 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.951+0000 7fbdba719640 1 --2- 172.21.15.42:0/2506454894 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbdb4101af0 0x7fbdb4197ad0 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:17.955 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.951+0000 7fbdba719640 1 -- 172.21.15.42:0/2506454894 >> 172.21.15.42:0/2506454894 conn(0x7fbdb40fd430 msgr2=0x7fbdb4105e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:15:17.955 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.951+0000 7fbdba719640 1 -- 172.21.15.42:0/2506454894 shutdown_connections 2024-08-30T09:15:17.955 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:17.951+0000 7fbdba719640 1 -- 172.21.15.42:0/2506454894 wait complete. 2024-08-30T09:15:19.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:15:18 smithi042 ceph-mon[18652]: pgmap v809: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 3.5 MiB/s wr, 396 op/s 2024-08-30T09:15:19.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:15:18 smithi190 ceph-mon[22001]: pgmap v809: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 3.5 MiB/s wr, 396 op/s 2024-08-30T09:15:19.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:15:18 smithi167 ceph-mon[22063]: pgmap v809: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 3.5 MiB/s wr, 396 op/s 2024-08-30T09:15:20.953 INFO:teuthology.orchestra.run.smithi042.stderr:progress: 875.0 s, 475.4 tps, lat 212.015 ms stddev 245.901 2024-08-30T09:15:21.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:15:20 smithi042 ceph-mon[18652]: pgmap v810: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 4.4 MiB/s wr, 490 op/s 2024-08-30T09:15:21.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:15:20 smithi190 ceph-mon[22001]: pgmap v810: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 4.4 MiB/s wr, 490 op/s 2024-08-30T09:15:21.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:15:20 smithi167 ceph-mon[22063]: pgmap v810: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 4.4 MiB/s wr, 490 op/s 2024-08-30T09:15:21.552 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-08-30T09:15:22.024 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T09:15:22.655 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.653+0000 7f0183257640 1 -- 172.21.15.42:0/345269754 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f017c10ffd0 msgr2=0x7f017c116800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:15:22.655 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.653+0000 7f0183257640 1 --2- 172.21.15.42:0/345269754 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f017c10ffd0 0x7f017c116800 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f0174009ca0 tx=0x7f017402a8f0 comp rx=0 tx=0).stop 2024-08-30T09:15:22.656 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.654+0000 7f0183257640 1 -- 172.21.15.42:0/345269754 shutdown_connections 2024-08-30T09:15:22.656 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.654+0000 7f0183257640 1 --2- 172.21.15.42:0/345269754 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f017c10ffd0 0x7f017c116800 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:22.657 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.654+0000 7f0183257640 1 --2- 172.21.15.42:0/345269754 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f017c10f680 0x7f017c10fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:22.657 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.654+0000 7f0183257640 1 --2- 172.21.15.42:0/345269754 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f017c109d40 0x7f017c10a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:22.657 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.654+0000 7f0183257640 1 -- 172.21.15.42:0/345269754 >> 172.21.15.42:0/345269754 conn(0x7f017c079320 msgr2=0x7f017c107f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:15:22.659 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.655+0000 7f0183257640 1 -- 172.21.15.42:0/345269754 shutdown_connections 2024-08-30T09:15:22.659 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.655+0000 7f0183257640 1 -- 172.21.15.42:0/345269754 wait complete. 2024-08-30T09:15:22.659 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.657+0000 7f0183257640 1 Processor -- start 2024-08-30T09:15:22.659 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.657+0000 7f0183257640 1 -- start start 2024-08-30T09:15:22.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.657+0000 7f0183257640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f017c109d40 0x7f017c074c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:22.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.657+0000 7f0183257640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f017c10f680 0x7f017c06df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:22.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.657+0000 7f0183257640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f017c10ffd0 0x7f017c06e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:22.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.657+0000 7f0183257640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f017c119d50 con 0x7f017c10ffd0 2024-08-30T09:15:22.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.657+0000 7f0183257640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f017c119bd0 con 0x7f017c10f680 2024-08-30T09:15:22.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.657+0000 7f0183257640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f017c119ed0 con 0x7f017c109d40 2024-08-30T09:15:22.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.657+0000 7f0182a56640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f017c10ffd0 0x7f017c06e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:15:22.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.658+0000 7f0182a56640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f017c10ffd0 0x7f017c06e4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:32996/0 (socket says 172.21.15.42:32996) 2024-08-30T09:15:22.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.658+0000 7f0182a56640 1 -- 172.21.15.42:0/438428592 learned_addr learned my addr 172.21.15.42:0/438428592 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:15:22.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.658+0000 7f0182a56640 1 -- 172.21.15.42:0/438428592 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f017c109d40 msgr2=0x7f017c074c40 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-08-30T09:15:22.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.658+0000 7f0182a56640 1 --2- 172.21.15.42:0/438428592 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f017c109d40 0x7f017c074c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:22.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.658+0000 7f0182a56640 1 -- 172.21.15.42:0/438428592 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f017c10f680 msgr2=0x7f017c06df10 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-08-30T09:15:22.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.658+0000 7f0182a56640 1 --2- 172.21.15.42:0/438428592 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f017c10f680 0x7f017c06df10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:22.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.658+0000 7f0182a56640 1 -- 172.21.15.42:0/438428592 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f017c06ecf0 con 0x7f017c10ffd0 2024-08-30T09:15:22.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.659+0000 7f0182a56640 1 --2- 172.21.15.42:0/438428592 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f017c10ffd0 0x7f017c06e4d0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f0174030800 tx=0x7f0174030830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:15:22.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.660+0000 7f01737fe640 1 -- 172.21.15.42:0/438428592 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0174030e20 con 0x7f017c10ffd0 2024-08-30T09:15:22.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.660+0000 7f01737fe640 1 -- 172.21.15.42:0/438428592 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0174046930 con 0x7f017c10ffd0 2024-08-30T09:15:22.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.660+0000 7f01737fe640 1 -- 172.21.15.42:0/438428592 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f017403a990 con 0x7f017c10ffd0 2024-08-30T09:15:22.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.660+0000 7f0183257640 1 -- 172.21.15.42:0/438428592 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f017c073910 con 0x7f017c10ffd0 2024-08-30T09:15:22.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.661+0000 7f0183257640 1 -- 172.21.15.42:0/438428592 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f017c073e80 con 0x7f017c10ffd0 2024-08-30T09:15:22.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.665+0000 7f018aa2e740 1 Processor -- start 2024-08-30T09:15:22.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.670+0000 7f01737fe640 1 -- 172.21.15.42:0/438428592 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7f0174046480 con 0x7f017c10ffd0 2024-08-30T09:15:22.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.671+0000 7f018aa2e740 1 -- start start 2024-08-30T09:15:22.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.671+0000 7f018aa2e740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5606c1ae9070 0x5606c1e8e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:22.674 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.671+0000 7f018aa2e740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5606c1ac01b0 0x5606c1e8e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:22.674 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.671+0000 7f018aa2e740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5606c1ac8390 0x5606c1e8ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:22.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.671+0000 7f018aa2e740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x5606c1e05680 con 0x5606c1ac01b0 2024-08-30T09:15:22.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.671+0000 7f018aa2e740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5606c1df4c10 con 0x5606c1ae9070 2024-08-30T09:15:22.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.671+0000 7f018aa2e740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5606c1df8bc0 con 0x5606c1ac8390 2024-08-30T09:15:22.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.672+0000 7f0181a54640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5606c1ae9070 0x5606c1e8e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:15:22.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.672+0000 7f0181a54640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5606c1ae9070 0x5606c1e8e4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:52542/0 (socket says 172.21.15.42:52542) 2024-08-30T09:15:22.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.672+0000 7f0181a54640 1 -- 172.21.15.42:0/1741271582 learned_addr learned my addr 172.21.15.42:0/1741271582 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:15:22.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.672+0000 7f0181a54640 1 --2- 172.21.15.42:0/1741271582 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5606c1ac8390 0x5606c1e8ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:15:22.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.672+0000 7f0181a54640 1 -- 172.21.15.42:0/1741271582 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5606c1ac8390 msgr2=0x5606c1e8ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:15:22.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.672+0000 7f0181a54640 1 --2- 172.21.15.42:0/1741271582 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5606c1ac8390 0x5606c1e8ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:22.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.672+0000 7f0181a54640 1 -- 172.21.15.42:0/1741271582 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5606c1ac01b0 msgr2=0x5606c1e8e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:15:22.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.672+0000 7f0181a54640 1 --2- 172.21.15.42:0/1741271582 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5606c1ac01b0 0x5606c1e8e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:22.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.672+0000 7f0181a54640 1 -- 172.21.15.42:0/1741271582 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5606c1b54650 con 0x5606c1ae9070 2024-08-30T09:15:22.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.672+0000 7f0181a54640 1 --2- 172.21.15.42:0/1741271582 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5606c1ae9070 0x5606c1e8e4b0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f017800dfc0 tx=0x7f0178006f60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=50a6c794a85807c0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:15:22.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.674+0000 7f01737fe640 1 --2- 172.21.15.42:0/438428592 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f01680b4cc0 0x7f01680b71d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:22.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.675+0000 7f01737fe640 1 -- 172.21.15.42:0/438428592 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f01741069f0 con 0x7f017c10ffd0 2024-08-30T09:15:22.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.676+0000 7f0170d9a640 1 -- 172.21.15.42:0/1741271582 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f017801c070 con 0x5606c1ae9070 2024-08-30T09:15:22.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.676+0000 7f0170d9a640 1 -- 172.21.15.42:0/1741271582 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0178010040 con 0x5606c1ae9070 2024-08-30T09:15:22.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.676+0000 7f0170d9a640 1 -- 172.21.15.42:0/1741271582 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f017800f050 con 0x5606c1ae9070 2024-08-30T09:15:22.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.677+0000 7f018aa2e740 1 -- 172.21.15.42:0/1741271582 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5606c1ae9070 msgr2=0x5606c1e8e4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:15:22.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.677+0000 7f018aa2e740 1 --2- 172.21.15.42:0/1741271582 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5606c1ae9070 0x5606c1e8e4b0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f017800dfc0 tx=0x7f0178006f60 comp rx=0 tx=0).stop 2024-08-30T09:15:22.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.677+0000 7f018aa2e740 1 -- 172.21.15.42:0/1741271582 shutdown_connections 2024-08-30T09:15:22.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.677+0000 7f018aa2e740 1 --2- 172.21.15.42:0/1741271582 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5606c1ac8390 0x5606c1e8ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:22.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.677+0000 7f018aa2e740 1 --2- 172.21.15.42:0/1741271582 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5606c1ac01b0 0x5606c1e8e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:22.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.677+0000 7f018aa2e740 1 --2- 172.21.15.42:0/1741271582 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5606c1ae9070 0x5606c1e8e4b0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:22.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.677+0000 7f018aa2e740 1 -- 172.21.15.42:0/1741271582 >> 172.21.15.42:0/1741271582 conn(0x5606c1add460 msgr2=0x5606c1e8c3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:15:22.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.679+0000 7f0182255640 1 --2- 172.21.15.42:0/438428592 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f01680b4cc0 0x7f01680b71d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:15:22.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.683+0000 7f018aa2e740 1 -- 172.21.15.42:0/1741271582 shutdown_connections 2024-08-30T09:15:22.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.687+0000 7f018aa2e740 1 -- 172.21.15.42:0/1741271582 wait complete. 2024-08-30T09:15:22.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.687+0000 7f018aa2e740 1 -- start start 2024-08-30T09:15:22.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.687+0000 7f018aa2e740 1 Processor -- start 2024-08-30T09:15:22.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.696+0000 7f018aa2e740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5606c1ac01b0 0x5606c1e99fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:22.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.696+0000 7f018aa2e740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5606c1ac8390 0x5606c1e9a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:22.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.696+0000 7f018aa2e740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5606c1ae9070 0x5606c1ea5f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:22.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.696+0000 7f018aa2e740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x5606c1c57700 con 0x5606c1ac8390 2024-08-30T09:15:22.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.696+0000 7f018aa2e740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5606c1d7b750 con 0x5606c1ae9070 2024-08-30T09:15:22.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.696+0000 7f018aa2e740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5606c1e9cc90 con 0x5606c1ac01b0 2024-08-30T09:15:22.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.696+0000 7f018aa2e740 10 client.0 _ll_register_callbacks cb 0x5606c1daae10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:15:22.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.697+0000 7f0182255640 1 --2- 172.21.15.42:0/438428592 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f01680b4cc0 0x7f01680b71d0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x5606c1e8d130 tx=0x7f016c006c20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:15:22.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.701+0000 7f0182255640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5606c1ac01b0 0x5606c1e99fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:15:22.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.701+0000 7f0182255640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5606c1ac01b0 0x5606c1e99fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:34174/0 (socket says 172.21.15.42:34174) 2024-08-30T09:15:22.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.701+0000 7f0182255640 1 -- 172.21.15.42:0/2691519624 learned_addr learned my addr 172.21.15.42:0/2691519624 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:15:22.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.703+0000 7f0181a54640 1 --2- 172.21.15.42:0/2691519624 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5606c1ac8390 0x5606c1e9a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:15:22.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.704+0000 7f0182a56640 1 --2- 172.21.15.42:0/2691519624 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5606c1ae9070 0x5606c1ea5f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:15:22.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.710+0000 7f0182255640 1 -- 172.21.15.42:0/2691519624 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5606c1ae9070 msgr2=0x5606c1ea5f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:15:22.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.710+0000 7f0182255640 1 --2- 172.21.15.42:0/2691519624 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5606c1ae9070 0x5606c1ea5f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:22.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.710+0000 7f0182255640 1 -- 172.21.15.42:0/2691519624 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5606c1ac8390 msgr2=0x5606c1e9a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:15:22.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.710+0000 7f0182255640 1 --2- 172.21.15.42:0/2691519624 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5606c1ac8390 0x5606c1e9a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:22.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.710+0000 7f0182255640 1 -- 172.21.15.42:0/2691519624 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5606c1db4900 con 0x5606c1ac01b0 2024-08-30T09:15:22.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.713+0000 7f0182255640 1 --2- 172.21.15.42:0/2691519624 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5606c1ac01b0 0x5606c1e99fb0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f016c013f50 tx=0x7f016c0047d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a9add165fe6874b9 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:15:22.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.715+0000 7f0165ffb640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-08-30T09:15:22.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.715+0000 7f0165ffb640 1 -- 172.21.15.42:0/2691519624 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f016c01d5f0 con 0x5606c1ac01b0 2024-08-30T09:15:22.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.715+0000 7f018aa2e740 1 -- 172.21.15.42:0/2691519624 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0178010040 con 0x5606c1ac01b0 2024-08-30T09:15:22.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.715+0000 7f0165ffb640 1 -- 172.21.15.42:0/2691519624 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f016c01d790 con 0x5606c1ac01b0 2024-08-30T09:15:22.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.720+0000 7f0165ffb640 1 -- 172.21.15.42:0/2691519624 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f016c01d990 con 0x5606c1ac01b0 2024-08-30T09:15:22.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.724+0000 7f018aa2e740 10 client.27628 fetch_fsmap learned FSMap version 48 2024-08-30T09:15:22.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.724+0000 7f018aa2e740 1 -- 172.21.15.42:0/2691519624 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x5606c1eb9bf0 con 0x5606c1ac01b0 2024-08-30T09:15:22.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.725+0000 7f0165ffb640 1 -- 172.21.15.42:0/2691519624 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0178010040 con 0x5606c1ac01b0 2024-08-30T09:15:22.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.725+0000 7f0165ffb640 1 -- 172.21.15.42:0/2691519624 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 48) ==== 8366+0+0 (secure 0 0 0) 0x7f016c012070 con 0x5606c1ac01b0 2024-08-30T09:15:22.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.725+0000 7f018aa2e740 10 client.27628 fetch_fsmap finished waiting for FSMap version 48 2024-08-30T09:15:22.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.726+0000 7f018aa2e740 10 client.27628 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-30T09:15:22.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.726+0000 7f018aa2e740 20 client.27628 populate_metadata read hostname 'smithi042' 2024-08-30T09:15:22.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.726+0000 7f018aa2e740 1 --2- 172.21.15.42:0/2691519624 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x5606c1aa79f0 0x5606c1ebd640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:22.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.726+0000 7f018aa2e740 4 client.27628 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-30T09:15:22.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.726+0000 7f018aa2e740 1 -- 172.21.15.42:0/2691519624 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5606c1e075d0 con 0x5606c1aa79f0 2024-08-30T09:15:22.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.728+0000 7f0181a54640 1 --2- 172.21.15.42:0/2691519624 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x5606c1aa79f0 0x5606c1ebd640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:15:22.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.736+0000 7f0181a54640 1 --2- 172.21.15.42:0/2691519624 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x5606c1aa79f0 0x5606c1ebd640 crc :-1 s=READY pgs=295 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3d71e3e68a8bd81a server_cookie=aa39da62cf7c43c5 in_seq=0 out_seq=0 2024-08-30T09:15:22.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.737+0000 7f0165ffb640 10 client.27628 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:15:22.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.743+0000 7f0165ffb640 1 -- 172.21.15.42:0/2691519624 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5606c1e075d0 con 0x5606c1aa79f0 2024-08-30T09:15:22.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.743+0000 7f0165ffb640 10 client.27628 handle_command_reply: tid=0 2024-08-30T09:15:22.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.743+0000 7f018aa2e740 1 client.27628 shutdown 2024-08-30T09:15:22.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.746+0000 7f018aa2e740 1 -- 172.21.15.42:0/2691519624 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5606c1ac01b0 msgr2=0x5606c1e99fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:15:22.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.746+0000 7f018aa2e740 1 --2- 172.21.15.42:0/2691519624 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5606c1ac01b0 0x5606c1e99fb0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f016c013f50 tx=0x7f016c0047d0 comp rx=0 tx=0).stop 2024-08-30T09:15:22.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.749+0000 7f018aa2e740 1 -- 172.21.15.42:0/2691519624 shutdown_connections 2024-08-30T09:15:22.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.749+0000 7f018aa2e740 1 --2- 172.21.15.42:0/2691519624 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x5606c1aa79f0 0x5606c1ebd640 crc :-1 s=READY pgs=295 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:22.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.749+0000 7f018aa2e740 1 --2- 172.21.15.42:0/2691519624 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5606c1ae9070 0x5606c1ea5f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:22.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.749+0000 7f018aa2e740 1 --2- 172.21.15.42:0/2691519624 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5606c1ac8390 0x5606c1e9a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:22.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.749+0000 7f018aa2e740 1 --2- 172.21.15.42:0/2691519624 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5606c1ac01b0 0x5606c1e99fb0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:22.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.749+0000 7f018aa2e740 1 -- 172.21.15.42:0/2691519624 >> 172.21.15.42:0/2691519624 conn(0x5606c1add460 msgr2=0x5606c1e91690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:15:22.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.749+0000 7f0165ffb640 0 client.27628 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:15:22.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.752+0000 7f018aa2e740 1 -- 172.21.15.42:0/2691519624 shutdown_connections 2024-08-30T09:15:22.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.756+0000 7f018aa2e740 1 -- 172.21.15.42:0/2691519624 wait complete. 2024-08-30T09:15:22.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.760+0000 7f018aa2e740 20 client.27628 trim_cache size 0 max 16384 2024-08-30T09:15:22.793 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.791+0000 7f018aa2e740 1 Processor -- start 2024-08-30T09:15:22.794 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.792+0000 7f018aa2e740 1 -- start start 2024-08-30T09:15:22.794 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.792+0000 7f018aa2e740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5606c1ac01b0 0x5606c1e945d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:22.794 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.792+0000 7f018aa2e740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5606c1ac8390 0x5606c1e8c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:22.794 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.792+0000 7f018aa2e740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5606c1ae9070 0x5606c1ea5e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:22.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.792+0000 7f018aa2e740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f017c1b9730 con 0x5606c1ac8390 2024-08-30T09:15:22.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.793+0000 7f018aa2e740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f017c1b6fd0 con 0x5606c1ac01b0 2024-08-30T09:15:22.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.793+0000 7f018aa2e740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f017c10a1d0 con 0x5606c1ae9070 2024-08-30T09:15:22.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.793+0000 7f0182255640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5606c1ac01b0 0x5606c1e945d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:15:22.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.793+0000 7f0182255640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5606c1ac01b0 0x5606c1e945d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:52546/0 (socket says 172.21.15.42:52546) 2024-08-30T09:15:22.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.793+0000 7f0182255640 1 -- 172.21.15.42:0/2980817877 learned_addr learned my addr 172.21.15.42:0/2980817877 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:15:22.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.793+0000 7f0182255640 1 -- 172.21.15.42:0/2980817877 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5606c1ae9070 msgr2=0x5606c1ea5e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:15:22.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.793+0000 7f0182255640 1 --2- 172.21.15.42:0/2980817877 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5606c1ae9070 0x5606c1ea5e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:22.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.793+0000 7f0182255640 1 -- 172.21.15.42:0/2980817877 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5606c1ac8390 msgr2=0x5606c1e8c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:15:22.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.793+0000 7f0182255640 1 --2- 172.21.15.42:0/2980817877 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5606c1ac8390 0x5606c1e8c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:22.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.793+0000 7f0182255640 1 -- 172.21.15.42:0/2980817877 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5606c1e90a70 con 0x5606c1ac01b0 2024-08-30T09:15:22.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.797+0000 7f0182255640 1 --2- 172.21.15.42:0/2980817877 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5606c1ac01b0 0x5606c1e945d0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f016c009eb0 tx=0x7f016c025590 comp rx=0 tx=0).ready entity=mon.2 client_cookie=39c2fac8f8b217a6 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:15:22.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.802+0000 7f0167fff640 1 -- 172.21.15.42:0/2980817877 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f016c035070 con 0x5606c1ac01b0 2024-08-30T09:15:22.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.802+0000 7f0167fff640 1 -- 172.21.15.42:0/2980817877 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f016c022070 con 0x5606c1ac01b0 2024-08-30T09:15:22.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.802+0000 7f0167fff640 1 -- 172.21.15.42:0/2980817877 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f016c010360 con 0x5606c1ac01b0 2024-08-30T09:15:22.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.803+0000 7f018aa2e740 1 -- 172.21.15.42:0/2980817877 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5606c1ac01b0 msgr2=0x5606c1e945d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:15:22.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.804+0000 7f018aa2e740 1 --2- 172.21.15.42:0/2980817877 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5606c1ac01b0 0x5606c1e945d0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f016c009eb0 tx=0x7f016c025590 comp rx=0 tx=0).stop 2024-08-30T09:15:22.811 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.809+0000 7f018aa2e740 1 -- 172.21.15.42:0/2980817877 shutdown_connections 2024-08-30T09:15:22.811 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.809+0000 7f018aa2e740 1 --2- 172.21.15.42:0/2980817877 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5606c1ae9070 0x5606c1ea5e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:22.811 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.809+0000 7f018aa2e740 1 --2- 172.21.15.42:0/2980817877 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5606c1ac8390 0x5606c1e8c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:22.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.809+0000 7f018aa2e740 1 --2- 172.21.15.42:0/2980817877 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5606c1ac01b0 0x5606c1e945d0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:22.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.809+0000 7f018aa2e740 1 -- 172.21.15.42:0/2980817877 >> 172.21.15.42:0/2980817877 conn(0x5606c1add460 msgr2=0x5606c1e8bd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:15:22.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.814+0000 7f018aa2e740 1 -- 172.21.15.42:0/2980817877 shutdown_connections 2024-08-30T09:15:22.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.814+0000 7f018aa2e740 1 -- 172.21.15.42:0/2980817877 wait complete. 2024-08-30T09:15:22.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.815+0000 7f018aa2e740 1 -- start start 2024-08-30T09:15:22.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.815+0000 7f018aa2e740 1 Processor -- start 2024-08-30T09:15:22.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.816+0000 7f018aa2e740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5606c1ac01b0 0x5606c1e9e020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:22.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.816+0000 7f018aa2e740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5606c1ac8390 0x5606c1e9e560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:22.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.816+0000 7f018aa2e740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5606c1ae9070 0x5606c1e9eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:22.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.816+0000 7f018aa2e740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f017c083440 con 0x5606c1ac01b0 2024-08-30T09:15:22.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.816+0000 7f0181a54640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5606c1ac8390 0x5606c1e9e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:15:22.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.816+0000 7f0181a54640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5606c1ac8390 0x5606c1e9e560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:34182/0 (socket says 172.21.15.42:34182) 2024-08-30T09:15:22.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.816+0000 7f0181a54640 1 -- 172.21.15.42:0/1668219878 learned_addr learned my addr 172.21.15.42:0/1668219878 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:15:22.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.816+0000 7f018aa2e740 1 -- 172.21.15.42:0/1668219878 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f017c1b6900 con 0x5606c1ae9070 2024-08-30T09:15:22.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.816+0000 7f018aa2e740 1 -- 172.21.15.42:0/1668219878 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f017c083b10 con 0x5606c1ac8390 2024-08-30T09:15:22.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.817+0000 7f0182255640 1 --2- 172.21.15.42:0/1668219878 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5606c1ac01b0 0x5606c1e9e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:15:22.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.817+0000 7f018aa2e740 10 client.0 _ll_register_callbacks cb 0x7f016c009360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:15:22.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.819+0000 7f0182255640 1 -- 172.21.15.42:0/1668219878 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5606c1ac8390 msgr2=0x5606c1e9e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:15:22.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.819+0000 7f0182255640 1 --2- 172.21.15.42:0/1668219878 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5606c1ac8390 0x5606c1e9e560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:22.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.819+0000 7f0182255640 1 -- 172.21.15.42:0/1668219878 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5606c1ae9070 msgr2=0x5606c1e9eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:15:22.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.819+0000 7f0182255640 1 --2- 172.21.15.42:0/1668219878 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5606c1ae9070 0x5606c1e9eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:22.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.819+0000 7f0182255640 1 -- 172.21.15.42:0/1668219878 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5606c1f271a0 con 0x5606c1ac01b0 2024-08-30T09:15:22.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.824+0000 7f0182255640 1 --2- 172.21.15.42:0/1668219878 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5606c1ac01b0 0x5606c1e9e020 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f016c006fd0 tx=0x7f016c010850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42e4a42e561c4a9 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:15:22.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.826+0000 7f0165ffb640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-08-30T09:15:22.829 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.826+0000 7f0165ffb640 1 -- 172.21.15.42:0/1668219878 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f016c035070 con 0x5606c1ac01b0 2024-08-30T09:15:22.829 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.826+0000 7f018aa2e740 1 -- 172.21.15.42:0/1668219878 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f016c022070 con 0x5606c1ac01b0 2024-08-30T09:15:22.829 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.827+0000 7f0165ffb640 1 -- 172.21.15.42:0/1668219878 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f016c01dd40 con 0x5606c1ac01b0 2024-08-30T09:15:22.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.827+0000 7f0165ffb640 1 -- 172.21.15.42:0/1668219878 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f016c043050 con 0x5606c1ac01b0 2024-08-30T09:15:22.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.828+0000 7f018aa2e740 10 client.17856 fetch_fsmap learned FSMap version 48 2024-08-30T09:15:22.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.828+0000 7f018aa2e740 1 -- 172.21.15.42:0/1668219878 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x5606c1f283e0 con 0x5606c1ac01b0 2024-08-30T09:15:22.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.829+0000 7f0165ffb640 1 -- 172.21.15.42:0/1668219878 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f016c022070 con 0x5606c1ac01b0 2024-08-30T09:15:22.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.829+0000 7f0165ffb640 1 -- 172.21.15.42:0/1668219878 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 48) ==== 8366+0+0 (secure 0 0 0) 0x7f016c012040 con 0x5606c1ac01b0 2024-08-30T09:15:22.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.829+0000 7f018aa2e740 10 client.17856 fetch_fsmap finished waiting for FSMap version 48 2024-08-30T09:15:22.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.829+0000 7f018aa2e740 10 client.17856 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-30T09:15:22.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.829+0000 7f018aa2e740 20 client.17856 populate_metadata read hostname 'smithi042' 2024-08-30T09:15:22.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.829+0000 7f018aa2e740 1 --2- 172.21.15.42:0/1668219878 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x5606c1a7cda0 0x5606c1f2ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:22.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.829+0000 7f018aa2e740 4 client.17856 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-30T09:15:22.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.829+0000 7f018aa2e740 1 -- 172.21.15.42:0/1668219878 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5606c1e03070 con 0x5606c1a7cda0 2024-08-30T09:15:22.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.835+0000 7f0181a54640 1 --2- 172.21.15.42:0/1668219878 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x5606c1a7cda0 0x5606c1f2ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:15:22.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.848+0000 7f0181a54640 1 --2- 172.21.15.42:0/1668219878 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x5606c1a7cda0 0x5606c1f2ab00 crc :-1 s=READY pgs=296 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f041bfd698300015 server_cookie=f9ac273d43dba284 in_seq=0 out_seq=0 2024-08-30T09:15:22.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.848+0000 7f0165ffb640 10 client.17856 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:15:22.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.849+0000 7f0165ffb640 1 -- 172.21.15.42:0/1668219878 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5606c1e03070 con 0x5606c1a7cda0 2024-08-30T09:15:22.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.849+0000 7f0165ffb640 10 client.17856 handle_command_reply: tid=0 2024-08-30T09:15:22.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.849+0000 7f018aa2e740 1 client.17856 shutdown 2024-08-30T09:15:22.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.852+0000 7f018aa2e740 1 -- 172.21.15.42:0/1668219878 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5606c1ac01b0 msgr2=0x5606c1e9e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:15:22.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.852+0000 7f018aa2e740 1 --2- 172.21.15.42:0/1668219878 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5606c1ac01b0 0x5606c1e9e020 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f016c006fd0 tx=0x7f016c010850 comp rx=0 tx=0).stop 2024-08-30T09:15:22.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.853+0000 7f018aa2e740 1 -- 172.21.15.42:0/1668219878 shutdown_connections 2024-08-30T09:15:22.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.853+0000 7f018aa2e740 1 --2- 172.21.15.42:0/1668219878 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x5606c1a7cda0 0x5606c1f2ab00 crc :-1 s=READY pgs=296 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:22.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.853+0000 7f018aa2e740 1 --2- 172.21.15.42:0/1668219878 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5606c1ac8390 0x5606c1e9e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:22.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.853+0000 7f018aa2e740 1 --2- 172.21.15.42:0/1668219878 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5606c1ae9070 0x5606c1e9eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:22.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.853+0000 7f018aa2e740 1 --2- 172.21.15.42:0/1668219878 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5606c1ac01b0 0x5606c1e9e020 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:22.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.853+0000 7f018aa2e740 1 -- 172.21.15.42:0/1668219878 >> 172.21.15.42:0/1668219878 conn(0x5606c1add460 msgr2=0x5606c1e9b260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:15:22.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.853+0000 7f0165ffb640 0 client.17856 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:15:22.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.857+0000 7f018aa2e740 1 -- 172.21.15.42:0/1668219878 shutdown_connections 2024-08-30T09:15:22.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.857+0000 7f018aa2e740 1 -- 172.21.15.42:0/1668219878 wait complete. 2024-08-30T09:15:22.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.858+0000 7f018aa2e740 20 client.17856 trim_cache size 0 max 16384 2024-08-30T09:15:22.861 INFO:teuthology.orchestra.run.smithi042.stdout:[] 2024-08-30T09:15:22.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.859+0000 7f0183257640 1 -- 172.21.15.42:0/438428592 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f01680b4cc0 msgr2=0x7f01680b71d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:15:22.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.859+0000 7f0183257640 1 --2- 172.21.15.42:0/438428592 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f01680b4cc0 0x7f01680b71d0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x5606c1e8d130 tx=0x7f016c006c20 comp rx=0 tx=0).stop 2024-08-30T09:15:22.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.860+0000 7f0183257640 1 -- 172.21.15.42:0/438428592 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f017c10ffd0 msgr2=0x7f017c06e4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:15:22.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.860+0000 7f0183257640 1 --2- 172.21.15.42:0/438428592 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f017c10ffd0 0x7f017c06e4d0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f0174030800 tx=0x7f0174030830 comp rx=0 tx=0).stop 2024-08-30T09:15:22.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.860+0000 7f0183257640 1 -- 172.21.15.42:0/438428592 shutdown_connections 2024-08-30T09:15:22.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.860+0000 7f0183257640 1 --2- 172.21.15.42:0/438428592 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f01680b4cc0 0x7f01680b71d0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:22.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.860+0000 7f0183257640 1 --2- 172.21.15.42:0/438428592 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f017c10ffd0 0x7f017c06e4d0 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:22.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.860+0000 7f0183257640 1 --2- 172.21.15.42:0/438428592 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f017c10f680 0x7f017c06df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:22.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.860+0000 7f0183257640 1 --2- 172.21.15.42:0/438428592 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f017c109d40 0x7f017c074c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:22.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.860+0000 7f0183257640 1 -- 172.21.15.42:0/438428592 >> 172.21.15.42:0/438428592 conn(0x7f017c079320 msgr2=0x7f017c079af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:15:22.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.864+0000 7f0183257640 1 -- 172.21.15.42:0/438428592 shutdown_connections 2024-08-30T09:15:22.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:22.864+0000 7f0183257640 1 -- 172.21.15.42:0/438428592 wait complete. 2024-08-30T09:15:23.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:15:22 smithi042 ceph-mon[18652]: pgmap v811: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 3.5 MiB/s wr, 376 op/s 2024-08-30T09:15:23.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:15:22 smithi190 ceph-mon[22001]: pgmap v811: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 3.5 MiB/s wr, 376 op/s 2024-08-30T09:15:23.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:15:22 smithi167 ceph-mon[22063]: pgmap v811: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 3.5 MiB/s wr, 376 op/s 2024-08-30T09:15:24.594 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-08-30T09:15:24.595 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-08-30T09:15:24.885 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:15:24 smithi042 ceph-mon[18652]: pgmap v812: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 4.0 MiB/s wr, 445 op/s 2024-08-30T09:15:25.078 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T09:15:25.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:15:24 smithi190 ceph-mon[22001]: pgmap v812: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 4.0 MiB/s wr, 445 op/s 2024-08-30T09:15:25.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:15:24 smithi167 ceph-mon[22063]: pgmap v812: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 4.0 MiB/s wr, 445 op/s 2024-08-30T09:15:25.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.775+0000 7fcfb6b1c640 1 -- 172.21.15.42:0/1887641871 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcfb0111ce0 msgr2=0x7fcfb010a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:15:25.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.775+0000 7fcfb6b1c640 1 --2- 172.21.15.42:0/1887641871 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcfb0111ce0 0x7fcfb010a160 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fcf98009860 tx=0x7fcf9802a360 comp rx=0 tx=0).stop 2024-08-30T09:15:25.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.778+0000 7fcfb6b1c640 1 -- 172.21.15.42:0/1887641871 shutdown_connections 2024-08-30T09:15:25.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.778+0000 7fcfb6b1c640 1 --2- 172.21.15.42:0/1887641871 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcfb011c9d0 0x7fcfb011eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:25.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.778+0000 7fcfb6b1c640 1 --2- 172.21.15.42:0/1887641871 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcfb010a780 0x7fcfb010ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:25.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.778+0000 7fcfb6b1c640 1 --2- 172.21.15.42:0/1887641871 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcfb0111ce0 0x7fcfb010a160 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:25.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.778+0000 7fcfb6b1c640 1 -- 172.21.15.42:0/1887641871 >> 172.21.15.42:0/1887641871 conn(0x7fcfb0070c60 msgr2=0x7fcfb0071070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:15:25.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.779+0000 7fcfb6b1c640 1 -- 172.21.15.42:0/1887641871 shutdown_connections 2024-08-30T09:15:25.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.783+0000 7fcfb6b1c640 1 -- 172.21.15.42:0/1887641871 wait complete. 2024-08-30T09:15:25.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.784+0000 7fcfb6b1c640 1 Processor -- start 2024-08-30T09:15:25.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.784+0000 7fcfb6b1c640 1 -- start start 2024-08-30T09:15:25.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.784+0000 7fcfb6b1c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcfb010a780 0x7fcfb011c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:25.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.784+0000 7fcfb6b1c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcfb0111ce0 0x7fcfb0116550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:25.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.784+0000 7fcfb6b1c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcfb011c9d0 0x7fcfb0116bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:25.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.784+0000 7fcfb6b1c640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fcfb0112a20 con 0x7fcfb0111ce0 2024-08-30T09:15:25.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.784+0000 7fcfb6b1c640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fcfb01128a0 con 0x7fcfb010a780 2024-08-30T09:15:25.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.784+0000 7fcfb6b1c640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fcfb0112ba0 con 0x7fcfb011c9d0 2024-08-30T09:15:25.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.785+0000 7fcfb50b9640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcfb011c9d0 0x7fcfb0116bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:15:25.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.787+0000 7fcfaffff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcfb0111ce0 0x7fcfb0116550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:15:25.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.787+0000 7fcfaffff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcfb0111ce0 0x7fcfb0116550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33048/0 (socket says 172.21.15.42:33048) 2024-08-30T09:15:25.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.787+0000 7fcfaffff640 1 -- 172.21.15.42:0/810604176 learned_addr learned my addr 172.21.15.42:0/810604176 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:15:25.794 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.789+0000 7fcfb48b8640 1 --2- 172.21.15.42:0/810604176 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcfb010a780 0x7fcfb011c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:15:25.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.793+0000 7fcfaffff640 1 -- 172.21.15.42:0/810604176 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcfb011c9d0 msgr2=0x7fcfb0116bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:15:25.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.793+0000 7fcfaffff640 1 --2- 172.21.15.42:0/810604176 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcfb011c9d0 0x7fcfb0116bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:25.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.793+0000 7fcfaffff640 1 -- 172.21.15.42:0/810604176 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcfb010a780 msgr2=0x7fcfb011c290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:15:25.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.793+0000 7fcfaffff640 1 --2- 172.21.15.42:0/810604176 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcfb010a780 0x7fcfb011c290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:25.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.793+0000 7fcfaffff640 1 -- 172.21.15.42:0/810604176 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfb01bdb70 con 0x7fcfb0111ce0 2024-08-30T09:15:25.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.801+0000 7fcfaffff640 1 --2- 172.21.15.42:0/810604176 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcfb0111ce0 0x7fcfb0116550 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fcfa800ecf0 tx=0x7fcfa800c6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:15:25.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.802+0000 7fcfb50b9640 1 --2- 172.21.15.42:0/810604176 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcfb011c9d0 0x7fcfb0116bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-30T09:15:25.811 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.806+0000 7fcfadffb640 1 -- 172.21.15.42:0/810604176 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfa8003b70 con 0x7fcfb0111ce0 2024-08-30T09:15:25.811 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.806+0000 7fcfadffb640 1 -- 172.21.15.42:0/810604176 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcfa8003d10 con 0x7fcfb0111ce0 2024-08-30T09:15:25.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.806+0000 7fcfadffb640 1 -- 172.21.15.42:0/810604176 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfa800f8a0 con 0x7fcfb0111ce0 2024-08-30T09:15:25.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.811+0000 7fcfb6b1c640 1 -- 172.21.15.42:0/810604176 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcfb01bdd40 con 0x7fcfb0111ce0 2024-08-30T09:15:25.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.811+0000 7fcfb6b1c640 1 -- 172.21.15.42:0/810604176 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcfb01be1a0 con 0x7fcfb0111ce0 2024-08-30T09:15:25.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.814+0000 7fcfbd091740 1 Processor -- start 2024-08-30T09:15:25.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.823+0000 7fcfbd091740 1 -- start start 2024-08-30T09:15:25.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.823+0000 7fcfbd091740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563f3fda1350 0x563f40147780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:25.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.823+0000 7fcfbd091740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563f3fd78490 0x563f40147cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:25.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.823+0000 7fcfbd091740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563f3fd80670 0x563f40148200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:25.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.823+0000 7fcfbd091740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x563f400bd5f0 con 0x563f3fd80670 2024-08-30T09:15:25.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.823+0000 7fcfbd091740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x563f400acb80 con 0x563f3fda1350 2024-08-30T09:15:25.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.823+0000 7fcfbd091740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x563f400b0b30 con 0x563f3fd78490 2024-08-30T09:15:25.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.823+0000 7fcfadffb640 1 -- 172.21.15.42:0/810604176 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7fcfa801e020 con 0x7fcfb0111ce0 2024-08-30T09:15:25.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.823+0000 7fcfb48b8640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563f3fd78490 0x563f40147cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:15:25.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.823+0000 7fcfb48b8640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563f3fd78490 0x563f40147cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:34214/0 (socket says 172.21.15.42:34214) 2024-08-30T09:15:25.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.823+0000 7fcfb48b8640 1 -- 172.21.15.42:0/3713008496 learned_addr learned my addr 172.21.15.42:0/3713008496 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:15:25.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.824+0000 7fcfb48b8640 1 -- 172.21.15.42:0/3713008496 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563f3fda1350 msgr2=0x563f40147780 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-30T09:15:25.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.824+0000 7fcfb48b8640 1 --2- 172.21.15.42:0/3713008496 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563f3fda1350 0x563f40147780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:25.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.824+0000 7fcfb48b8640 1 -- 172.21.15.42:0/3713008496 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563f3fd80670 msgr2=0x563f40148200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:15:25.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.824+0000 7fcfb48b8640 1 --2- 172.21.15.42:0/3713008496 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563f3fd80670 0x563f40148200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:25.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.824+0000 7fcfb48b8640 1 -- 172.21.15.42:0/3713008496 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563f3fe0c930 con 0x563f3fd78490 2024-08-30T09:15:25.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.826+0000 7fcfadffb640 1 --2- 172.21.15.42:0/810604176 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fcf9c0b4d40 0x7fcf9c0b7250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:25.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.826+0000 7fcfadffb640 1 -- 172.21.15.42:0/810604176 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7fcfa80e5490 con 0x7fcfb0111ce0 2024-08-30T09:15:25.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.830+0000 7fcfb50b9640 1 --2- 172.21.15.42:0/810604176 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fcf9c0b4d40 0x7fcf9c0b7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:15:25.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.833+0000 7fcfb48b8640 1 --2- 172.21.15.42:0/3713008496 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563f3fd78490 0x563f40147cc0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fcf98005eb0 tx=0x7fcf98009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4a29c6af50c04e41 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:15:25.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.836+0000 7fcfb50b9640 1 --2- 172.21.15.42:0/810604176 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fcf9c0b4d40 0x7fcf9c0b7250 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fcfa0006fd0 tx=0x7fcfa0009060 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:15:25.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.837+0000 7fcfa6ffd640 1 -- 172.21.15.42:0/3713008496 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf98042020 con 0x563f3fd78490 2024-08-30T09:15:25.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.837+0000 7fcfa6ffd640 1 -- 172.21.15.42:0/3713008496 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcf98038040 con 0x563f3fd78490 2024-08-30T09:15:25.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.838+0000 7fcfbd091740 1 -- 172.21.15.42:0/3713008496 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563f3fd78490 msgr2=0x563f40147cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:15:25.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.838+0000 7fcfbd091740 1 --2- 172.21.15.42:0/3713008496 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563f3fd78490 0x563f40147cc0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fcf98005eb0 tx=0x7fcf98009580 comp rx=0 tx=0).stop 2024-08-30T09:15:25.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.839+0000 7fcfbd091740 1 -- 172.21.15.42:0/3713008496 shutdown_connections 2024-08-30T09:15:25.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.839+0000 7fcfbd091740 1 --2- 172.21.15.42:0/3713008496 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563f3fd78490 0x563f40147cc0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:25.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.839+0000 7fcfbd091740 1 --2- 172.21.15.42:0/3713008496 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563f3fd80670 0x563f40148200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:25.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.840+0000 7fcfbd091740 1 --2- 172.21.15.42:0/3713008496 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563f3fda1350 0x563f40147780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:25.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.840+0000 7fcfbd091740 1 -- 172.21.15.42:0/3713008496 >> 172.21.15.42:0/3713008496 conn(0x563f3fd95740 msgr2=0x563f40145670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:15:25.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.840+0000 7fcfbd091740 1 -- 172.21.15.42:0/3713008496 shutdown_connections 2024-08-30T09:15:25.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.840+0000 7fcfbd091740 1 -- 172.21.15.42:0/3713008496 wait complete. 2024-08-30T09:15:25.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.840+0000 7fcfbd091740 1 -- start start 2024-08-30T09:15:25.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.841+0000 7fcfbd091740 1 Processor -- start 2024-08-30T09:15:25.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.841+0000 7fcfbd091740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563f3fd78490 0x563f40152e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:25.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.841+0000 7fcfbd091740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563f3fd80670 0x563f401533c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:25.846 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.841+0000 7fcfbd091740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563f3fda1350 0x563f4015f290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:25.846 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.841+0000 7fcfbd091740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x563f3ff0f890 con 0x563f3fd78490 2024-08-30T09:15:25.846 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.841+0000 7fcfbd091740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x563f40033790 con 0x563f3fda1350 2024-08-30T09:15:25.846 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.841+0000 7fcfbd091740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x563f40155f50 con 0x563f3fd80670 2024-08-30T09:15:25.846 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.841+0000 7fcfbd091740 10 client.0 _ll_register_callbacks cb 0x563f3fdb8ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:15:25.846 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.842+0000 7fcfaffff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563f3fd80670 0x563f401533c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:15:25.846 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.842+0000 7fcfb50b9640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563f3fda1350 0x563f4015f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:15:25.846 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.842+0000 7fcfb50b9640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563f3fda1350 0x563f4015f290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:52588/0 (socket says 172.21.15.42:52588) 2024-08-30T09:15:25.846 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.842+0000 7fcfb50b9640 1 -- 172.21.15.42:0/4006585745 learned_addr learned my addr 172.21.15.42:0/4006585745 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:15:25.846 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.842+0000 7fcfaffff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563f3fd80670 0x563f401533c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:34228/0 (socket says 172.21.15.42:34228) 2024-08-30T09:15:25.846 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.843+0000 7fcfaffff640 1 -- 172.21.15.42:0/4006585745 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563f3fda1350 msgr2=0x563f4015f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:15:25.846 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.843+0000 7fcfaffff640 1 --2- 172.21.15.42:0/4006585745 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563f3fda1350 0x563f4015f290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:25.847 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.843+0000 7fcfb48b8640 1 --2- 172.21.15.42:0/4006585745 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563f3fd78490 0x563f40152e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:15:25.847 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.843+0000 7fcfaffff640 1 -- 172.21.15.42:0/4006585745 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563f3fd78490 msgr2=0x563f40152e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:15:25.847 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.843+0000 7fcfaffff640 1 --2- 172.21.15.42:0/4006585745 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563f3fd78490 0x563f40152e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:25.847 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.843+0000 7fcfaffff640 1 -- 172.21.15.42:0/4006585745 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563f4006af30 con 0x563f3fd80670 2024-08-30T09:15:25.847 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.843+0000 7fcfb48b8640 1 --2- 172.21.15.42:0/4006585745 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563f3fd78490 0x563f40152e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-30T09:15:25.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.847+0000 7fcfaffff640 1 --2- 172.21.15.42:0/4006585745 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563f3fd80670 0x563f401533c0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fcfa8009330 tx=0x7fcfa8005b10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c3a72a74990f3c5b server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:15:25.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.847+0000 7fcf8ffff640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-08-30T09:15:25.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.848+0000 7fcf8ffff640 1 -- 172.21.15.42:0/4006585745 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfa80b0070 con 0x563f3fd80670 2024-08-30T09:15:25.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.849+0000 7fcfbd091740 1 -- 172.21.15.42:0/4006585745 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcf98038040 con 0x563f3fd80670 2024-08-30T09:15:25.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.849+0000 7fcf8ffff640 1 -- 172.21.15.42:0/4006585745 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7fcfa80a80c0 con 0x563f3fd80670 2024-08-30T09:15:25.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.849+0000 7fcf8ffff640 1 -- 172.21.15.42:0/4006585745 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7fcfa80ac1d0 con 0x563f3fd80670 2024-08-30T09:15:25.854 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.850+0000 7fcfbd091740 10 client.27643 fetch_fsmap learned FSMap version 48 2024-08-30T09:15:25.854 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.850+0000 7fcfbd091740 1 -- 172.21.15.42:0/4006585745 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x563f40161dc0 con 0x563f3fd80670 2024-08-30T09:15:25.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.859+0000 7fcf8ffff640 1 -- 172.21.15.42:0/4006585745 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcf98038040 con 0x563f3fd80670 2024-08-30T09:15:25.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.859+0000 7fcf8ffff640 1 -- 172.21.15.42:0/4006585745 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 48) ==== 8366+0+0 (secure 0 0 0) 0x7fcfa800b070 con 0x563f3fd80670 2024-08-30T09:15:25.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.862+0000 7fcfbd091740 10 client.27643 fetch_fsmap finished waiting for FSMap version 48 2024-08-30T09:15:25.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.862+0000 7fcfbd091740 10 client.27643 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-30T09:15:25.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.862+0000 7fcfbd091740 20 client.27643 populate_metadata read hostname 'smithi042' 2024-08-30T09:15:25.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.862+0000 7fcfbd091740 1 --2- 172.21.15.42:0/4006585745 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x563f3fd5fcd0 0x563f401652c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:25.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.862+0000 7fcfbd091740 4 client.27643 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-30T09:15:25.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.862+0000 7fcfbd091740 1 -- 172.21.15.42:0/4006585745 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563f400bf540 con 0x563f3fd5fcd0 2024-08-30T09:15:25.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.867+0000 7fcfb48b8640 1 --2- 172.21.15.42:0/4006585745 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x563f3fd5fcd0 0x563f401652c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:15:25.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.885+0000 7fcfb48b8640 1 --2- 172.21.15.42:0/4006585745 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x563f3fd5fcd0 0x563f401652c0 crc :-1 s=READY pgs=297 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=79a36cbdc9e46df0 server_cookie=fadaae7dea84bddf in_seq=0 out_seq=0 2024-08-30T09:15:25.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.885+0000 7fcf8ffff640 10 client.27643 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:15:25.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.886+0000 7fcf8ffff640 1 -- 172.21.15.42:0/4006585745 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x563f400bf540 con 0x563f3fd5fcd0 2024-08-30T09:15:25.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.886+0000 7fcf8ffff640 10 client.27643 handle_command_reply: tid=0 2024-08-30T09:15:25.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.887+0000 7fcfbd091740 1 client.27643 shutdown 2024-08-30T09:15:25.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.892+0000 7fcfbd091740 1 -- 172.21.15.42:0/4006585745 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563f3fd80670 msgr2=0x563f401533c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:15:25.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.892+0000 7fcfbd091740 1 --2- 172.21.15.42:0/4006585745 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563f3fd80670 0x563f401533c0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fcfa8009330 tx=0x7fcfa8005b10 comp rx=0 tx=0).stop 2024-08-30T09:15:25.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.892+0000 7fcfbd091740 1 -- 172.21.15.42:0/4006585745 shutdown_connections 2024-08-30T09:15:25.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.892+0000 7fcfbd091740 1 --2- 172.21.15.42:0/4006585745 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x563f3fd5fcd0 0x563f401652c0 crc :-1 s=READY pgs=297 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:25.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.892+0000 7fcfbd091740 1 --2- 172.21.15.42:0/4006585745 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563f3fd80670 0x563f401533c0 secure :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fcfa8009330 tx=0x7fcfa8005b10 comp rx=0 tx=0).stop 2024-08-30T09:15:25.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.892+0000 7fcfbd091740 1 --2- 172.21.15.42:0/4006585745 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563f3fda1350 0x563f4015f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:25.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.892+0000 7fcfbd091740 1 --2- 172.21.15.42:0/4006585745 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563f3fd78490 0x563f40152e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:25.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.892+0000 7fcf8ffff640 0 client.27643 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:15:25.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.892+0000 7fcfbd091740 1 -- 172.21.15.42:0/4006585745 >> 172.21.15.42:0/4006585745 conn(0x563f3fd95740 msgr2=0x563f4014a960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:15:25.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.893+0000 7fcfbd091740 1 -- 172.21.15.42:0/4006585745 shutdown_connections 2024-08-30T09:15:25.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.893+0000 7fcfbd091740 1 -- 172.21.15.42:0/4006585745 wait complete. 2024-08-30T09:15:25.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.893+0000 7fcfbd091740 20 client.27643 trim_cache size 0 max 16384 2024-08-30T09:15:25.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.928+0000 7fcfbd091740 1 Processor -- start 2024-08-30T09:15:25.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.929+0000 7fcfbd091740 1 -- start start 2024-08-30T09:15:25.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.929+0000 7fcfbd091740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563f3fd78490 0x563f40154d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:25.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.929+0000 7fcfbd091740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563f3fda1350 0x563f4015f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:25.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.929+0000 7fcfbd091740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563f3fd35080 0x563f40148dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:25.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.929+0000 7fcfb48b8640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563f3fd78490 0x563f40154d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:15:25.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.929+0000 7fcfb48b8640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563f3fd78490 0x563f40154d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:34230/0 (socket says 172.21.15.42:34230) 2024-08-30T09:15:25.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.929+0000 7fcfb48b8640 1 -- 172.21.15.42:0/3191023850 learned_addr learned my addr 172.21.15.42:0/3191023850 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:15:25.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.930+0000 7fcfbd091740 1 -- 172.21.15.42:0/3191023850 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fcfb01ca960 con 0x563f3fd35080 2024-08-30T09:15:25.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.930+0000 7fcfbd091740 1 -- 172.21.15.42:0/3191023850 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fcfb01c8200 con 0x563f3fda1350 2024-08-30T09:15:25.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.930+0000 7fcfbd091740 1 -- 172.21.15.42:0/3191023850 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fcfb01c6030 con 0x563f3fd78490 2024-08-30T09:15:25.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.934+0000 7fcfb50b9640 1 --2- 172.21.15.42:0/3191023850 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563f3fd35080 0x563f40148dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:15:25.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.934+0000 7fcfb48b8640 1 -- 172.21.15.42:0/3191023850 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563f3fda1350 msgr2=0x563f4015f190 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-30T09:15:25.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.934+0000 7fcfb48b8640 1 --2- 172.21.15.42:0/3191023850 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563f3fda1350 0x563f4015f190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:25.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.934+0000 7fcfb48b8640 1 -- 172.21.15.42:0/3191023850 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563f3fd35080 msgr2=0x563f40148dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:15:25.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.934+0000 7fcfb48b8640 1 --2- 172.21.15.42:0/3191023850 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563f3fd35080 0x563f40148dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:25.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.934+0000 7fcfb48b8640 1 -- 172.21.15.42:0/3191023850 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563f4014dab0 con 0x563f3fd78490 2024-08-30T09:15:25.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.943+0000 7fcfb48b8640 1 --2- 172.21.15.42:0/3191023850 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563f3fd78490 0x563f40154d60 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fcf9802a950 tx=0x7fcf98000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=276e26bec3ab5a4f server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:15:25.947 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.944+0000 7fcfa67fc640 1 -- 172.21.15.42:0/3191023850 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf98042040 con 0x563f3fd78490 2024-08-30T09:15:25.947 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.944+0000 7fcfa67fc640 1 -- 172.21.15.42:0/3191023850 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcf980055e0 con 0x563f3fd78490 2024-08-30T09:15:25.947 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.944+0000 7fcfa67fc640 1 -- 172.21.15.42:0/3191023850 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf98004920 con 0x563f3fd78490 2024-08-30T09:15:25.949 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.946+0000 7fcfbd091740 1 -- 172.21.15.42:0/3191023850 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563f3fd78490 msgr2=0x563f40154d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:15:25.949 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.946+0000 7fcfbd091740 1 --2- 172.21.15.42:0/3191023850 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563f3fd78490 0x563f40154d60 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fcf9802a950 tx=0x7fcf98000bc0 comp rx=0 tx=0).stop 2024-08-30T09:15:25.949 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.946+0000 7fcfbd091740 1 -- 172.21.15.42:0/3191023850 shutdown_connections 2024-08-30T09:15:25.949 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.946+0000 7fcfbd091740 1 --2- 172.21.15.42:0/3191023850 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563f3fd35080 0x563f40148dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:25.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.946+0000 7fcfbd091740 1 --2- 172.21.15.42:0/3191023850 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563f3fda1350 0x563f4015f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:25.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.946+0000 7fcfbd091740 1 --2- 172.21.15.42:0/3191023850 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563f3fd78490 0x563f40154d60 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:25.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.947+0000 7fcfbd091740 1 -- 172.21.15.42:0/3191023850 >> 172.21.15.42:0/3191023850 conn(0x563f3fd95740 msgr2=0x563f401486b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:15:25.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.951+0000 7fcfbd091740 1 -- 172.21.15.42:0/3191023850 shutdown_connections 2024-08-30T09:15:25.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.951+0000 7fcfbd091740 1 -- 172.21.15.42:0/3191023850 wait complete. 2024-08-30T09:15:25.954 INFO:teuthology.orchestra.run.smithi042.stderr:progress: 880.0 s, 538.0 tps, lat 178.778 ms stddev 225.474 2024-08-30T09:15:25.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.951+0000 7fcfbd091740 1 -- start start 2024-08-30T09:15:25.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.952+0000 7fcfbd091740 1 Processor -- start 2024-08-30T09:15:25.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.952+0000 7fcfbd091740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563f3fd35080 0x563f4015bb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:25.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.953+0000 7fcfbd091740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563f3fd78490 0x563f40158030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:25.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.953+0000 7fcfb48b8640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563f3fd35080 0x563f4015bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:15:25.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.953+0000 7fcfbd091740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563f3fda1350 0x563f401586b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:25.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.953+0000 7fcfb48b8640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563f3fd35080 0x563f4015bb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33082/0 (socket says 172.21.15.42:33082) 2024-08-30T09:15:25.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.953+0000 7fcfb48b8640 1 -- 172.21.15.42:0/2194831636 learned_addr learned my addr 172.21.15.42:0/2194831636 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:15:25.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.953+0000 7fcfbd091740 1 -- 172.21.15.42:0/2194831636 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fcfb01c66c0 con 0x563f3fd35080 2024-08-30T09:15:25.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.953+0000 7fcfbd091740 1 -- 172.21.15.42:0/2194831636 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fcfb01c7b30 con 0x563f3fd78490 2024-08-30T09:15:25.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.953+0000 7fcfbd091740 1 -- 172.21.15.42:0/2194831636 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fcfb01c6d90 con 0x563f3fda1350 2024-08-30T09:15:25.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.953+0000 7fcfbd091740 10 client.0 _ll_register_callbacks cb 0x7fcfa8004820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:15:25.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.954+0000 7fcfaffff640 1 --2- 172.21.15.42:0/2194831636 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563f3fd78490 0x563f40158030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:15:25.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.954+0000 7fcfb50b9640 1 --2- 172.21.15.42:0/2194831636 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563f3fda1350 0x563f401586b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:15:25.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.955+0000 7fcfb48b8640 1 -- 172.21.15.42:0/2194831636 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563f3fda1350 msgr2=0x563f401586b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:15:25.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.955+0000 7fcfb48b8640 1 --2- 172.21.15.42:0/2194831636 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563f3fda1350 0x563f401586b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:25.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.955+0000 7fcfb48b8640 1 -- 172.21.15.42:0/2194831636 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563f3fd78490 msgr2=0x563f40158030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:15:25.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.955+0000 7fcfb48b8640 1 --2- 172.21.15.42:0/2194831636 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563f3fd78490 0x563f40158030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:25.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.955+0000 7fcfb48b8640 1 -- 172.21.15.42:0/2194831636 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563f40158e40 con 0x563f3fd35080 2024-08-30T09:15:25.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.961+0000 7fcfb48b8640 1 --2- 172.21.15.42:0/2194831636 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563f3fd35080 0x563f4015bb20 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fcf9802a8d0 tx=0x7fcf98005f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0e4d0d69904d3bb server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:15:25.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.961+0000 7fcf8ffff640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-08-30T09:15:25.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.964+0000 7fcf8ffff640 1 -- 172.21.15.42:0/2194831636 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf98042040 con 0x563f3fd35080 2024-08-30T09:15:25.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.964+0000 7fcf8ffff640 1 -- 172.21.15.42:0/2194831636 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7fcf980306c0 con 0x563f3fd35080 2024-08-30T09:15:25.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.964+0000 7fcfbd091740 1 -- 172.21.15.42:0/2194831636 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcf980055e0 con 0x563f3fd35080 2024-08-30T09:15:25.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.968+0000 7fcf8ffff640 1 -- 172.21.15.42:0/2194831636 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7fcf980308c0 con 0x563f3fd35080 2024-08-30T09:15:25.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.969+0000 7fcfbd091740 10 client.17868 fetch_fsmap learned FSMap version 48 2024-08-30T09:15:25.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.969+0000 7fcfbd091740 1 -- 172.21.15.42:0/2194831636 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x563f401d0d30 con 0x563f3fd35080 2024-08-30T09:15:25.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.973+0000 7fcf8ffff640 1 -- 172.21.15.42:0/2194831636 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcf98054030 con 0x563f3fd35080 2024-08-30T09:15:25.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.973+0000 7fcf8ffff640 1 -- 172.21.15.42:0/2194831636 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 48) ==== 8366+0+0 (secure 0 0 0) 0x7fcf98032720 con 0x563f3fd35080 2024-08-30T09:15:25.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.974+0000 7fcfbd091740 10 client.17868 fetch_fsmap finished waiting for FSMap version 48 2024-08-30T09:15:25.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.974+0000 7fcfbd091740 10 client.17868 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-30T09:15:25.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.974+0000 7fcfbd091740 20 client.17868 populate_metadata read hostname 'smithi042' 2024-08-30T09:15:25.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.974+0000 7fcfbd091740 1 --2- 172.21.15.42:0/2194831636 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x563f401d0fb0 0x563f401d33d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:25.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.974+0000 7fcfbd091740 4 client.17868 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-08-30T09:15:25.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.974+0000 7fcfbd091740 1 -- 172.21.15.42:0/2194831636 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7fcf980055e0 con 0x563f401d0fb0 2024-08-30T09:15:25.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.976+0000 7fcfaffff640 1 --2- 172.21.15.42:0/2194831636 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x563f401d0fb0 0x563f401d33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:15:25.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.980+0000 7fcf8ffff640 10 client.17868 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:15:25.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.980+0000 7fcfaffff640 1 --2- 172.21.15.42:0/2194831636 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x563f401d0fb0 0x563f401d33d0 crc :-1 s=READY pgs=298 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e874b16fe88b5af4 server_cookie=de2ac1b919816080 in_seq=0 out_seq=0 2024-08-30T09:15:25.986 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.984+0000 7fcf8ffff640 1 -- 172.21.15.42:0/2194831636 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7fcf980055e0 con 0x563f401d0fb0 2024-08-30T09:15:25.986 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.984+0000 7fcf8ffff640 10 client.17868 handle_command_reply: tid=0 2024-08-30T09:15:25.986 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.984+0000 7fcfbd091740 1 client.17868 shutdown 2024-08-30T09:15:25.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.988+0000 7fcfbd091740 1 -- 172.21.15.42:0/2194831636 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563f3fd35080 msgr2=0x563f4015bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:15:25.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.988+0000 7fcfbd091740 1 --2- 172.21.15.42:0/2194831636 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563f3fd35080 0x563f4015bb20 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fcf9802a8d0 tx=0x7fcf98005f10 comp rx=0 tx=0).stop 2024-08-30T09:15:25.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.989+0000 7fcfbd091740 1 -- 172.21.15.42:0/2194831636 shutdown_connections 2024-08-30T09:15:25.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.989+0000 7fcfbd091740 1 --2- 172.21.15.42:0/2194831636 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x563f401d0fb0 0x563f401d33d0 crc :-1 s=READY pgs=298 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:25.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.989+0000 7fcfbd091740 1 --2- 172.21.15.42:0/2194831636 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563f3fda1350 0x563f401586b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:25.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.989+0000 7fcfbd091740 1 --2- 172.21.15.42:0/2194831636 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563f3fd78490 0x563f40158030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:25.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.989+0000 7fcfbd091740 1 --2- 172.21.15.42:0/2194831636 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563f3fd35080 0x563f4015bb20 secure :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fcf9802a8d0 tx=0x7fcf98005f10 comp rx=0 tx=0).stop 2024-08-30T09:15:25.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.989+0000 7fcfbd091740 1 -- 172.21.15.42:0/2194831636 >> 172.21.15.42:0/2194831636 conn(0x563f3fd95740 msgr2=0x563f40149070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:15:25.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.989+0000 7fcf8ffff640 0 client.17868 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:15:25.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.990+0000 7fcfbd091740 1 -- 172.21.15.42:0/2194831636 shutdown_connections 2024-08-30T09:15:25.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.990+0000 7fcfbd091740 1 -- 172.21.15.42:0/2194831636 wait complete. 2024-08-30T09:15:25.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.990+0000 7fcfbd091740 20 client.17868 trim_cache size 0 max 16384 2024-08-30T09:15:25.993 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-08-30T09:15:25.993 INFO:teuthology.orchestra.run.smithi042.stdout: "return_code": 0, 2024-08-30T09:15:25.993 INFO:teuthology.orchestra.run.smithi042.stdout: "scrub_tag": "3f8863e7-c1a4-472c-8bd7-690aeeb13b82", 2024-08-30T09:15:25.993 INFO:teuthology.orchestra.run.smithi042.stdout: "mode": "asynchronous" 2024-08-30T09:15:25.993 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-08-30T09:15:25.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.991+0000 7fcfb6b1c640 1 -- 172.21.15.42:0/810604176 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fcf9c0b4d40 msgr2=0x7fcf9c0b7250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:15:25.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.991+0000 7fcfb6b1c640 1 --2- 172.21.15.42:0/810604176 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fcf9c0b4d40 0x7fcf9c0b7250 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fcfa0006fd0 tx=0x7fcfa0009060 comp rx=0 tx=0).stop 2024-08-30T09:15:25.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.991+0000 7fcfb6b1c640 1 -- 172.21.15.42:0/810604176 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcfb0111ce0 msgr2=0x7fcfb0116550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:15:25.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.991+0000 7fcfb6b1c640 1 --2- 172.21.15.42:0/810604176 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcfb0111ce0 0x7fcfb0116550 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fcfa800ecf0 tx=0x7fcfa800c6e0 comp rx=0 tx=0).stop 2024-08-30T09:15:25.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.991+0000 7fcfb6b1c640 1 -- 172.21.15.42:0/810604176 shutdown_connections 2024-08-30T09:15:25.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.991+0000 7fcfb6b1c640 1 --2- 172.21.15.42:0/810604176 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fcf9c0b4d40 0x7fcf9c0b7250 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:25.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.991+0000 7fcfb6b1c640 1 --2- 172.21.15.42:0/810604176 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcfb011c9d0 0x7fcfb0116bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:25.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.991+0000 7fcfb6b1c640 1 --2- 172.21.15.42:0/810604176 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcfb0111ce0 0x7fcfb0116550 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:25.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.991+0000 7fcfb6b1c640 1 --2- 172.21.15.42:0/810604176 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcfb010a780 0x7fcfb011c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:25.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.991+0000 7fcfb6b1c640 1 -- 172.21.15.42:0/810604176 >> 172.21.15.42:0/810604176 conn(0x7fcfb0070c60 msgr2=0x7fcfb010b7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:15:25.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.991+0000 7fcfb6b1c640 1 -- 172.21.15.42:0/810604176 shutdown_connections 2024-08-30T09:15:25.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:25.991+0000 7fcfb6b1c640 1 -- 172.21.15.42:0/810604176 wait complete. 2024-08-30T09:15:26.764 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-30T09:15:27.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:15:26 smithi042 ceph-mon[18652]: pgmap v813: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.2 MiB/s wr, 461 op/s 2024-08-30T09:15:27.134 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:15:26 smithi042 ceph-mon[18652]: scrub queued for path: / 2024-08-30T09:15:27.134 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:15:26 smithi042 ceph-mon[18652]: scrub summary: idle+waiting paths [/] 2024-08-30T09:15:27.134 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:15:26 smithi042 ceph-mon[18652]: scrub summary: active paths [/] 2024-08-30T09:15:27.134 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:15:26 smithi042 ceph-mon[18652]: scrub summary: active paths [/] 2024-08-30T09:15:27.134 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:15:26 smithi042 ceph-mon[18652]: scrub summary: idle+waiting paths [/] 2024-08-30T09:15:27.134 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:15:26 smithi042 ceph-mon[18652]: scrub summary: active paths [/] 2024-08-30T09:15:27.134 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:15:26 smithi042 ceph-mon[18652]: scrub summary: active paths [/] 2024-08-30T09:15:27.134 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:15:26 smithi042 ceph-mon[18652]: scrub summary: idle+waiting paths [/] 2024-08-30T09:15:27.134 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:15:26 smithi042 ceph-mon[18652]: scrub summary: active paths [/] 2024-08-30T09:15:27.134 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:15:26 smithi042 ceph-mon[18652]: scrub summary: idle+waiting paths [/] 2024-08-30T09:15:27.134 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:15:26 smithi042 ceph-mon[18652]: scrub summary: idle+waiting paths [/] 2024-08-30T09:15:27.134 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:15:26 smithi042 ceph-mon[18652]: scrub summary: idle+waiting paths [/] 2024-08-30T09:15:27.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:15:26 smithi190 ceph-mon[22001]: pgmap v813: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.2 MiB/s wr, 461 op/s 2024-08-30T09:15:27.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:15:26 smithi190 ceph-mon[22001]: scrub queued for path: / 2024-08-30T09:15:27.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:15:26 smithi190 ceph-mon[22001]: scrub summary: idle+waiting paths [/] 2024-08-30T09:15:27.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:15:26 smithi190 ceph-mon[22001]: scrub summary: active paths [/] 2024-08-30T09:15:27.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:15:26 smithi190 ceph-mon[22001]: scrub summary: active paths [/] 2024-08-30T09:15:27.187 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:15:26 smithi190 ceph-mon[22001]: scrub summary: idle+waiting paths [/] 2024-08-30T09:15:27.187 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:15:26 smithi190 ceph-mon[22001]: scrub summary: active paths [/] 2024-08-30T09:15:27.187 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:15:26 smithi190 ceph-mon[22001]: scrub summary: active paths [/] 2024-08-30T09:15:27.187 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:15:26 smithi190 ceph-mon[22001]: scrub summary: idle+waiting paths [/] 2024-08-30T09:15:27.187 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:15:26 smithi190 ceph-mon[22001]: scrub summary: active paths [/] 2024-08-30T09:15:27.187 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:15:26 smithi190 ceph-mon[22001]: scrub summary: idle+waiting paths [/] 2024-08-30T09:15:27.187 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:15:26 smithi190 ceph-mon[22001]: scrub summary: idle+waiting paths [/] 2024-08-30T09:15:27.187 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:15:26 smithi190 ceph-mon[22001]: scrub summary: idle+waiting paths [/] 2024-08-30T09:15:27.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:15:26 smithi167 ceph-mon[22063]: pgmap v813: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.2 MiB/s wr, 461 op/s 2024-08-30T09:15:27.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:15:26 smithi167 ceph-mon[22063]: scrub queued for path: / 2024-08-30T09:15:27.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:15:26 smithi167 ceph-mon[22063]: scrub summary: idle+waiting paths [/] 2024-08-30T09:15:27.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:15:26 smithi167 ceph-mon[22063]: scrub summary: active paths [/] 2024-08-30T09:15:27.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:15:26 smithi167 ceph-mon[22063]: scrub summary: active paths [/] 2024-08-30T09:15:27.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:15:26 smithi167 ceph-mon[22063]: scrub summary: idle+waiting paths [/] 2024-08-30T09:15:27.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:15:26 smithi167 ceph-mon[22063]: scrub summary: active paths [/] 2024-08-30T09:15:27.328 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:15:26 smithi167 ceph-mon[22063]: scrub summary: active paths [/] 2024-08-30T09:15:27.328 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:15:26 smithi167 ceph-mon[22063]: scrub summary: idle+waiting paths [/] 2024-08-30T09:15:27.328 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:15:26 smithi167 ceph-mon[22063]: scrub summary: active paths [/] 2024-08-30T09:15:27.328 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:15:26 smithi167 ceph-mon[22063]: scrub summary: idle+waiting paths [/] 2024-08-30T09:15:27.328 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:15:26 smithi167 ceph-mon[22063]: scrub summary: idle+waiting paths [/] 2024-08-30T09:15:27.328 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:15:26 smithi167 ceph-mon[22063]: scrub summary: idle+waiting paths [/] 2024-08-30T09:15:27.351 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T09:15:27.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:27.986+0000 7f6063fff640 1 -- 172.21.15.42:0/2703420104 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6064072890 msgr2=0x7f6064072ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:15:27.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:27.986+0000 7f6063fff640 1 --2- 172.21.15.42:0/2703420104 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6064072890 0x7f6064072ca0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f605c005fd0 tx=0x7f605c02e720 comp rx=0 tx=0).stop 2024-08-30T09:15:27.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:27.987+0000 7f6063fff640 1 -- 172.21.15.42:0/2703420104 shutdown_connections 2024-08-30T09:15:27.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:27.987+0000 7f6063fff640 1 --2- 172.21.15.42:0/2703420104 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f60640731e0 0x7f6064073a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:27.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:27.987+0000 7f6063fff640 1 --2- 172.21.15.42:0/2703420104 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6064072890 0x7f6064072ca0 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:27.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:27.987+0000 7f6063fff640 1 --2- 172.21.15.42:0/2703420104 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f606406cf50 0x7f606406d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:27.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:27.987+0000 7f6063fff640 1 -- 172.21.15.42:0/2703420104 >> 172.21.15.42:0/2703420104 conn(0x7f6064078420 msgr2=0x7f6064078860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:15:27.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:27.994+0000 7f6063fff640 1 -- 172.21.15.42:0/2703420104 shutdown_connections 2024-08-30T09:15:28.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:27.997+0000 7f6063fff640 1 -- 172.21.15.42:0/2703420104 wait complete. 2024-08-30T09:15:28.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:27.998+0000 7f6063fff640 1 Processor -- start 2024-08-30T09:15:28.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:27.998+0000 7f6063fff640 1 -- start start 2024-08-30T09:15:28.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:27.998+0000 7f6063fff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f60640731e0 0x7f60641a6ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:28.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:27.998+0000 7f6063fff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f60641a7530 0x7f60641a0200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:28.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:27.998+0000 7f6063fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f60641a08a0 0x7f60641a5ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:28.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:27.998+0000 7f6063fff640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f6064119d70 con 0x7f60641a08a0 2024-08-30T09:15:28.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:27.999+0000 7f6063fff640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f6064119bf0 con 0x7f60640731e0 2024-08-30T09:15:28.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:27.999+0000 7f6063fff640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f6064119ef0 con 0x7f60641a7530 2024-08-30T09:15:28.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.003+0000 7f60637fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f60641a08a0 0x7f60641a5ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:15:28.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.003+0000 7f60637fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f60641a08a0 0x7f60641a5ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60872/0 (socket says 172.21.15.42:60872) 2024-08-30T09:15:28.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.003+0000 7f60637fe640 1 -- 172.21.15.42:0/2965869584 learned_addr learned my addr 172.21.15.42:0/2965869584 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:15:28.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.003+0000 7f60627fc640 1 --2- 172.21.15.42:0/2965869584 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f60641a7530 0x7f60641a0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:15:28.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.004+0000 7f6062ffd640 1 --2- 172.21.15.42:0/2965869584 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f60640731e0 0x7f60641a6ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:15:28.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.004+0000 7f60637fe640 1 -- 172.21.15.42:0/2965869584 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f60641a7530 msgr2=0x7f60641a0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:15:28.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.004+0000 7f60637fe640 1 --2- 172.21.15.42:0/2965869584 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f60641a7530 0x7f60641a0200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:28.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.004+0000 7f60637fe640 1 -- 172.21.15.42:0/2965869584 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f60640731e0 msgr2=0x7f60641a6ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:15:28.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.004+0000 7f60637fe640 1 --2- 172.21.15.42:0/2965869584 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f60640731e0 0x7f60641a6ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:28.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.004+0000 7f60637fe640 1 -- 172.21.15.42:0/2965869584 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60641a60e0 con 0x7f60641a08a0 2024-08-30T09:15:28.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.005+0000 7f60637fe640 1 --2- 172.21.15.42:0/2965869584 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f60641a08a0 0x7f60641a5ba0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f605400a850 tx=0x7f605400ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:15:28.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.006+0000 7f605bfff640 1 -- 172.21.15.42:0/2965869584 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60540079b0 con 0x7f60641a08a0 2024-08-30T09:15:28.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.006+0000 7f6063fff640 1 -- 172.21.15.42:0/2965869584 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60641a6370 con 0x7f60641a08a0 2024-08-30T09:15:28.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.006+0000 7f6063fff640 1 -- 172.21.15.42:0/2965869584 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60641a6880 con 0x7f60641a08a0 2024-08-30T09:15:28.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.007+0000 7f605bfff640 1 -- 172.21.15.42:0/2965869584 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6054005020 con 0x7f60641a08a0 2024-08-30T09:15:28.012 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.008+0000 7f605bfff640 1 -- 172.21.15.42:0/2965869584 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6054004cd0 con 0x7f60641a08a0 2024-08-30T09:15:28.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.014+0000 7f606f939740 1 Processor -- start 2024-08-30T09:15:28.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.014+0000 7f606f939740 1 -- start start 2024-08-30T09:15:28.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.015+0000 7f606f939740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561d72153070 0x561d724f8460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:28.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.015+0000 7f606f939740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561d7212a1b0 0x561d724f89a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:28.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.015+0000 7f606f939740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561d72132390 0x561d724f8ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:28.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.015+0000 7f606f939740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x561d7246f630 con 0x561d7212a1b0 2024-08-30T09:15:28.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.015+0000 7f606f939740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x561d7245ebc0 con 0x561d72153070 2024-08-30T09:15:28.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.015+0000 7f606f939740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x561d72462b70 con 0x561d72132390 2024-08-30T09:15:28.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.025+0000 7f60627fc640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561d72153070 0x561d724f8460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:15:28.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.025+0000 7f60627fc640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561d72132390 0x561d724f8ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:15:28.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.026+0000 7f60627fc640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561d72153070 0x561d724f8460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:47466/0 (socket says 172.21.15.42:47466) 2024-08-30T09:15:28.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.026+0000 7f60627fc640 1 -- 172.21.15.42:0/3840441608 learned_addr learned my addr 172.21.15.42:0/3840441608 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:15:28.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.026+0000 7f605bfff640 1 -- 172.21.15.42:0/2965869584 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7f60540051d0 con 0x7f60641a08a0 2024-08-30T09:15:28.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.027+0000 7f6062ffd640 1 --2- 172.21.15.42:0/3840441608 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561d7212a1b0 0x561d724f89a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:15:28.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.034+0000 7f605bfff640 1 --2- 172.21.15.42:0/2965869584 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f60500b4b90 0x7f60500b70a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:28.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.035+0000 7f605bfff640 1 -- 172.21.15.42:0/2965869584 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f6054106910 con 0x7f60641a08a0 2024-08-30T09:15:28.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.036+0000 7f60637fe640 1 --2- 172.21.15.42:0/2965869584 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f60500b4b90 0x7f60500b70a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:15:28.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.037+0000 7f60637fe640 1 --2- 172.21.15.42:0/2965869584 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f60500b4b90 0x7f60500b70a0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f60540ccee0 tx=0x7f6054004680 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:15:28.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.038+0000 7f60627fc640 1 -- 172.21.15.42:0/3840441608 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561d72132390 msgr2=0x561d724f8ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:15:28.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.038+0000 7f60627fc640 1 --2- 172.21.15.42:0/3840441608 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561d72132390 0x561d724f8ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:28.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.038+0000 7f60627fc640 1 -- 172.21.15.42:0/3840441608 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561d7212a1b0 msgr2=0x561d724f89a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:15:28.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.038+0000 7f60627fc640 1 --2- 172.21.15.42:0/3840441608 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561d7212a1b0 0x561d724f89a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:28.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.038+0000 7f60627fc640 1 -- 172.21.15.42:0/3840441608 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561d721be650 con 0x561d72153070 2024-08-30T09:15:28.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.040+0000 7f60627fc640 1 --2- 172.21.15.42:0/3840441608 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561d72153070 0x561d724f8460 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f605c00afd0 tx=0x7f605c02ec20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a7ca2279f0151e91 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:15:28.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.041+0000 7f60597fa640 1 -- 172.21.15.42:0/3840441608 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f605c046070 con 0x561d72153070 2024-08-30T09:15:28.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.041+0000 7f60597fa640 1 -- 172.21.15.42:0/3840441608 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f605c03b040 con 0x561d72153070 2024-08-30T09:15:28.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.042+0000 7f606f939740 1 -- 172.21.15.42:0/3840441608 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561d72153070 msgr2=0x561d724f8460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:15:28.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.042+0000 7f606f939740 1 --2- 172.21.15.42:0/3840441608 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561d72153070 0x561d724f8460 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f605c00afd0 tx=0x7f605c02ec20 comp rx=0 tx=0).stop 2024-08-30T09:15:28.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.043+0000 7f606f939740 1 -- 172.21.15.42:0/3840441608 shutdown_connections 2024-08-30T09:15:28.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.043+0000 7f606f939740 1 --2- 172.21.15.42:0/3840441608 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561d72132390 0x561d724f8ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:28.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.043+0000 7f606f939740 1 --2- 172.21.15.42:0/3840441608 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561d7212a1b0 0x561d724f89a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:28.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.043+0000 7f606f939740 1 --2- 172.21.15.42:0/3840441608 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561d72153070 0x561d724f8460 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:28.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.043+0000 7f606f939740 1 -- 172.21.15.42:0/3840441608 >> 172.21.15.42:0/3840441608 conn(0x561d72147460 msgr2=0x561d724f6350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:15:28.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.043+0000 7f606f939740 1 -- 172.21.15.42:0/3840441608 shutdown_connections 2024-08-30T09:15:28.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.043+0000 7f606f939740 1 -- 172.21.15.42:0/3840441608 wait complete. 2024-08-30T09:15:28.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.044+0000 7f606f939740 1 -- start start 2024-08-30T09:15:28.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.044+0000 7f606f939740 1 Processor -- start 2024-08-30T09:15:28.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.045+0000 7f606f939740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561d7212a1b0 0x561d72503b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:28.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.045+0000 7f606f939740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561d72132390 0x561d72504070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:28.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.045+0000 7f606f939740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561d72153070 0x561d7250ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:28.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.045+0000 7f606f939740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x561d722c1700 con 0x561d72132390 2024-08-30T09:15:28.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.045+0000 7f60627fc640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561d7212a1b0 0x561d72503b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:15:28.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.045+0000 7f60627fc640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561d7212a1b0 0x561d72503b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:50772/0 (socket says 172.21.15.42:50772) 2024-08-30T09:15:28.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.045+0000 7f6062ffd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561d72132390 0x561d72504070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:15:28.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.045+0000 7f60627fc640 1 -- 172.21.15.42:0/2684293710 learned_addr learned my addr 172.21.15.42:0/2684293710 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:15:28.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.045+0000 7f606f939740 1 -- 172.21.15.42:0/2684293710 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x561d723e5700 con 0x561d72153070 2024-08-30T09:15:28.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.045+0000 7f606f939740 1 -- 172.21.15.42:0/2684293710 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x561d72506c30 con 0x561d7212a1b0 2024-08-30T09:15:28.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.045+0000 7f606f939740 10 client.0 _ll_register_callbacks cb 0x561d72414dc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:15:28.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.046+0000 7f60627fc640 1 --2- 172.21.15.42:0/2684293710 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561d72153070 0x561d7250ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:15:28.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.046+0000 7f60627fc640 1 -- 172.21.15.42:0/2684293710 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561d72153070 msgr2=0x561d7250ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:15:28.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.046+0000 7f60627fc640 1 --2- 172.21.15.42:0/2684293710 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561d72153070 0x561d7250ffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:28.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.046+0000 7f60627fc640 1 -- 172.21.15.42:0/2684293710 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561d72132390 msgr2=0x561d72504070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:15:28.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.046+0000 7f60627fc640 1 --2- 172.21.15.42:0/2684293710 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561d72132390 0x561d72504070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:28.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.046+0000 7f60627fc640 1 -- 172.21.15.42:0/2684293710 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561d7241e8b0 con 0x561d7212a1b0 2024-08-30T09:15:28.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.050+0000 7f60627fc640 1 --2- 172.21.15.42:0/2684293710 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561d7212a1b0 0x561d72503b30 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f605c02ed10 tx=0x7f605c008540 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b2fb723a6825b931 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:15:28.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.050+0000 7f604a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-08-30T09:15:28.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.053+0000 7f604a7fc640 1 -- 172.21.15.42:0/2684293710 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f605c046040 con 0x561d7212a1b0 2024-08-30T09:15:28.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.053+0000 7f604a7fc640 1 -- 172.21.15.42:0/2684293710 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f605c004450 con 0x561d7212a1b0 2024-08-30T09:15:28.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.053+0000 7f606f939740 1 -- 172.21.15.42:0/2684293710 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f605c03b040 con 0x561d7212a1b0 2024-08-30T09:15:28.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.054+0000 7f604a7fc640 1 -- 172.21.15.42:0/2684293710 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f605c054050 con 0x561d7212a1b0 2024-08-30T09:15:28.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.055+0000 7f606f939740 10 client.27658 fetch_fsmap learned FSMap version 48 2024-08-30T09:15:28.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.055+0000 7f606f939740 1 -- 172.21.15.42:0/2684293710 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x561d725129b0 con 0x561d7212a1b0 2024-08-30T09:15:28.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.056+0000 7f604a7fc640 1 -- 172.21.15.42:0/2684293710 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f605c047040 con 0x561d7212a1b0 2024-08-30T09:15:28.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.056+0000 7f604a7fc640 1 -- 172.21.15.42:0/2684293710 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 48) ==== 8366+0+0 (secure 0 0 0) 0x7f605c040070 con 0x561d7212a1b0 2024-08-30T09:15:28.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.056+0000 7f606f939740 10 client.27658 fetch_fsmap finished waiting for FSMap version 48 2024-08-30T09:15:28.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.056+0000 7f606f939740 10 client.27658 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-30T09:15:28.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.056+0000 7f606f939740 20 client.27658 populate_metadata read hostname 'smithi042' 2024-08-30T09:15:28.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.056+0000 7f606f939740 1 --2- 172.21.15.42:0/2684293710 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x561d721119f0 0x561d72515d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:28.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.056+0000 7f606f939740 4 client.27658 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-30T09:15:28.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.056+0000 7f606f939740 1 -- 172.21.15.42:0/2684293710 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f605c03b040 con 0x561d721119f0 2024-08-30T09:15:28.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.058+0000 7f6062ffd640 1 --2- 172.21.15.42:0/2684293710 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x561d721119f0 0x561d72515d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:15:28.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.062+0000 7f6062ffd640 1 --2- 172.21.15.42:0/2684293710 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x561d721119f0 0x561d72515d10 crc :-1 s=READY pgs=299 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b3abc0f053639c82 server_cookie=637696eaef834d8d in_seq=0 out_seq=0 2024-08-30T09:15:28.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.062+0000 7f604a7fc640 10 client.27658 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:15:28.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.066+0000 7f604a7fc640 1 -- 172.21.15.42:0/2684293710 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f605c03b040 con 0x561d721119f0 2024-08-30T09:15:28.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.066+0000 7f604a7fc640 10 client.27658 handle_command_reply: tid=0 2024-08-30T09:15:28.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.066+0000 7f606f939740 1 client.27658 shutdown 2024-08-30T09:15:28.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.067+0000 7f606f939740 1 -- 172.21.15.42:0/2684293710 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561d7212a1b0 msgr2=0x561d72503b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:15:28.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.067+0000 7f606f939740 1 --2- 172.21.15.42:0/2684293710 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561d7212a1b0 0x561d72503b30 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f605c02ed10 tx=0x7f605c008540 comp rx=0 tx=0).stop 2024-08-30T09:15:28.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.074+0000 7f606f939740 1 -- 172.21.15.42:0/2684293710 shutdown_connections 2024-08-30T09:15:28.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.074+0000 7f606f939740 1 --2- 172.21.15.42:0/2684293710 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x561d721119f0 0x561d72515d10 crc :-1 s=READY pgs=299 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:28.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.074+0000 7f606f939740 1 --2- 172.21.15.42:0/2684293710 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561d72153070 0x561d7250ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:28.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.074+0000 7f604a7fc640 0 client.27658 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:15:28.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.074+0000 7f606f939740 1 --2- 172.21.15.42:0/2684293710 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561d72132390 0x561d72504070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:28.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.074+0000 7f606f939740 1 --2- 172.21.15.42:0/2684293710 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561d7212a1b0 0x561d72503b30 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:28.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.074+0000 7f606f939740 1 -- 172.21.15.42:0/2684293710 >> 172.21.15.42:0/2684293710 conn(0x561d72147460 msgr2=0x561d724fb5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:15:28.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.074+0000 7f606f939740 1 -- 172.21.15.42:0/2684293710 shutdown_connections 2024-08-30T09:15:28.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.075+0000 7f606f939740 1 -- 172.21.15.42:0/2684293710 wait complete. 2024-08-30T09:15:28.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.075+0000 7f606f939740 20 client.27658 trim_cache size 0 max 16384 2024-08-30T09:15:28.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.109+0000 7f606f939740 1 Processor -- start 2024-08-30T09:15:28.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.109+0000 7f606f939740 1 -- start start 2024-08-30T09:15:28.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.110+0000 7f606f939740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561d721119f0 0x561d72503ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:28.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.110+0000 7f606f939740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561d7212a1b0 0x561d7250ff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:28.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.111+0000 7f606f939740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561d72132390 0x561d72500d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:28.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.111+0000 7f60627fc640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561d721119f0 0x561d72503ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:15:28.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.111+0000 7f60627fc640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561d721119f0 0x561d72503ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:47480/0 (socket says 172.21.15.42:47480) 2024-08-30T09:15:28.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.111+0000 7f60627fc640 1 -- 172.21.15.42:0/738380300 learned_addr learned my addr 172.21.15.42:0/738380300 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:15:28.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.111+0000 7f606f939740 1 -- 172.21.15.42:0/738380300 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f6044012610 con 0x561d7212a1b0 2024-08-30T09:15:28.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.111+0000 7f606f939740 1 -- 172.21.15.42:0/738380300 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f604400feb0 con 0x561d721119f0 2024-08-30T09:15:28.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.111+0000 7f606f939740 1 -- 172.21.15.42:0/738380300 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f604400dd00 con 0x561d72132390 2024-08-30T09:15:28.114 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.112+0000 7f60627fc640 1 --2- 172.21.15.42:0/738380300 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561d72132390 0x561d72500d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:15:28.114 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.112+0000 7f60627fc640 1 -- 172.21.15.42:0/738380300 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561d72132390 msgr2=0x561d72500d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:15:28.114 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.112+0000 7f60627fc640 1 --2- 172.21.15.42:0/738380300 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561d72132390 0x561d72500d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:28.114 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.112+0000 7f60627fc640 1 -- 172.21.15.42:0/738380300 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561d7212a1b0 msgr2=0x561d7250ff40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-30T09:15:28.114 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.112+0000 7f60627fc640 1 --2- 172.21.15.42:0/738380300 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561d7212a1b0 0x561d7250ff40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:28.114 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.112+0000 7f60627fc640 1 -- 172.21.15.42:0/738380300 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561d724facf0 con 0x561d721119f0 2024-08-30T09:15:28.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.118+0000 7f60627fc640 1 --2- 172.21.15.42:0/738380300 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561d721119f0 0x561d72503ff0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f605c044040 tx=0x7f605c005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e83f732a9a49ca26 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:15:28.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.118+0000 7f6058ff9640 1 -- 172.21.15.42:0/738380300 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f605c046050 con 0x561d721119f0 2024-08-30T09:15:28.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.118+0000 7f6058ff9640 1 -- 172.21.15.42:0/738380300 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f605c035870 con 0x561d721119f0 2024-08-30T09:15:28.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.119+0000 7f606f939740 1 -- 172.21.15.42:0/738380300 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561d721119f0 msgr2=0x561d72503ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:15:28.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.119+0000 7f606f939740 1 --2- 172.21.15.42:0/738380300 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561d721119f0 0x561d72503ff0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f605c044040 tx=0x7f605c005fd0 comp rx=0 tx=0).stop 2024-08-30T09:15:28.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.121+0000 7f606f939740 1 -- 172.21.15.42:0/738380300 shutdown_connections 2024-08-30T09:15:28.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.121+0000 7f606f939740 1 --2- 172.21.15.42:0/738380300 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561d72132390 0x561d72500d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:28.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.121+0000 7f606f939740 1 --2- 172.21.15.42:0/738380300 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561d7212a1b0 0x561d7250ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:28.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.121+0000 7f606f939740 1 --2- 172.21.15.42:0/738380300 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561d721119f0 0x561d72503ff0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:28.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.121+0000 7f606f939740 1 -- 172.21.15.42:0/738380300 >> 172.21.15.42:0/738380300 conn(0x561d72147460 msgr2=0x561d725038e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:15:28.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.122+0000 7f606f939740 1 -- 172.21.15.42:0/738380300 shutdown_connections 2024-08-30T09:15:28.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.122+0000 7f606f939740 1 -- 172.21.15.42:0/738380300 wait complete. 2024-08-30T09:15:28.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.122+0000 7f606f939740 1 -- start start 2024-08-30T09:15:28.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.122+0000 7f606f939740 1 Processor -- start 2024-08-30T09:15:28.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.123+0000 7f606f939740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561d721119f0 0x561d725151e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:28.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.123+0000 7f606f939740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561d7212a1b0 0x561d725157c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:28.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.123+0000 7f606f939740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561d72132390 0x561d7257f480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:28.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.123+0000 7f606f939740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f604400e370 con 0x561d721119f0 2024-08-30T09:15:28.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.123+0000 7f606f939740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f604400f7e0 con 0x561d72132390 2024-08-30T09:15:28.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.123+0000 7f606f939740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f604400ea40 con 0x561d7212a1b0 2024-08-30T09:15:28.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.123+0000 7f606f939740 10 client.0 _ll_register_callbacks cb 0x7f605c044070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:15:28.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.124+0000 7f6062ffd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561d7212a1b0 0x561d725157c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:15:28.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.124+0000 7f60627fc640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561d72132390 0x561d7257f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:15:28.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.124+0000 7f60627fc640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561d721119f0 0x561d725151e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:15:28.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.124+0000 7f60627fc640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561d721119f0 0x561d725151e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60904/0 (socket says 172.21.15.42:60904) 2024-08-30T09:15:28.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.124+0000 7f60627fc640 1 -- 172.21.15.42:0/2993517711 learned_addr learned my addr 172.21.15.42:0/2993517711 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:15:28.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.127+0000 7f60627fc640 1 -- 172.21.15.42:0/2993517711 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561d7212a1b0 msgr2=0x561d725157c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:15:28.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.127+0000 7f60627fc640 1 --2- 172.21.15.42:0/2993517711 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561d7212a1b0 0x561d725157c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:28.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.127+0000 7f60627fc640 1 -- 172.21.15.42:0/2993517711 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561d72132390 msgr2=0x561d7257f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:15:28.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.127+0000 7f60627fc640 1 --2- 172.21.15.42:0/2993517711 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561d72132390 0x561d7257f480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:28.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.127+0000 7f60627fc640 1 -- 172.21.15.42:0/2993517711 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561d7257f9c0 con 0x561d721119f0 2024-08-30T09:15:28.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.128+0000 7f60627fc640 1 --2- 172.21.15.42:0/2993517711 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561d721119f0 0x561d725151e0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f605c008510 tx=0x7f605c02e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c314d5248d1aa2a5 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:15:28.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.128+0000 7f604a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-08-30T09:15:28.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.128+0000 7f604a7fc640 1 -- 172.21.15.42:0/2993517711 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f605c046040 con 0x561d721119f0 2024-08-30T09:15:28.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.129+0000 7f606f939740 1 -- 172.21.15.42:0/2993517711 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x561d724f68b0 con 0x561d721119f0 2024-08-30T09:15:28.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.129+0000 7f604a7fc640 1 -- 172.21.15.42:0/2993517711 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f605c004450 con 0x561d721119f0 2024-08-30T09:15:28.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.130+0000 7f604a7fc640 1 -- 172.21.15.42:0/2993517711 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f605c068050 con 0x561d721119f0 2024-08-30T09:15:28.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.135+0000 7f606f939740 10 client.17883 fetch_fsmap learned FSMap version 48 2024-08-30T09:15:28.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.135+0000 7f606f939740 1 -- 172.21.15.42:0/2993517711 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x561d72580b80 con 0x561d721119f0 2024-08-30T09:15:28.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.139+0000 7f604a7fc640 1 -- 172.21.15.42:0/2993517711 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x561d724f68b0 con 0x561d721119f0 2024-08-30T09:15:28.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.139+0000 7f604a7fc640 1 -- 172.21.15.42:0/2993517711 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 48) ==== 8366+0+0 (secure 0 0 0) 0x7f605c040040 con 0x561d721119f0 2024-08-30T09:15:28.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.139+0000 7f606f939740 10 client.17883 fetch_fsmap finished waiting for FSMap version 48 2024-08-30T09:15:28.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.139+0000 7f606f939740 10 client.17883 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-30T09:15:28.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.139+0000 7f606f939740 20 client.17883 populate_metadata read hostname 'smithi042' 2024-08-30T09:15:28.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.139+0000 7f606f939740 1 --2- 172.21.15.42:0/2993517711 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x561d72153070 0x561d725831d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:28.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.139+0000 7f606f939740 4 client.17883 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-30T09:15:28.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.139+0000 7f606f939740 1 -- 172.21.15.42:0/2993517711 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "scrub status"}) -- 0x561d72471580 con 0x561d72153070 2024-08-30T09:15:28.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.144+0000 7f6062ffd640 1 --2- 172.21.15.42:0/2993517711 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x561d72153070 0x561d725831d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:15:28.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.148+0000 7f6062ffd640 1 --2- 172.21.15.42:0/2993517711 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x561d72153070 0x561d725831d0 crc :-1 s=READY pgs=300 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=85e170058f7f86f6 server_cookie=b7fd5f2bd6f3d0b in_seq=0 out_seq=0 2024-08-30T09:15:28.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.149+0000 7f604a7fc640 10 client.17883 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:15:28.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.149+0000 7f604a7fc640 1 -- 172.21.15.42:0/2993517711 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+5398 (crc 0 0 0) 0x561d72471580 con 0x561d72153070 2024-08-30T09:15:28.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.149+0000 7f604a7fc640 10 client.17883 handle_command_reply: tid=0 2024-08-30T09:15:28.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.149+0000 7f606f939740 1 client.17883 shutdown 2024-08-30T09:15:28.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.159+0000 7f606f939740 1 -- 172.21.15.42:0/2993517711 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561d721119f0 msgr2=0x561d725151e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:15:28.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.159+0000 7f606f939740 1 --2- 172.21.15.42:0/2993517711 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561d721119f0 0x561d725151e0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f605c008510 tx=0x7f605c02e720 comp rx=0 tx=0).stop 2024-08-30T09:15:28.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.159+0000 7f606f939740 1 -- 172.21.15.42:0/2993517711 shutdown_connections 2024-08-30T09:15:28.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.159+0000 7f606f939740 1 --2- 172.21.15.42:0/2993517711 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x561d72153070 0x561d725831d0 crc :-1 s=READY pgs=300 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:28.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.159+0000 7f606f939740 1 --2- 172.21.15.42:0/2993517711 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561d7212a1b0 0x561d725157c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:28.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.159+0000 7f606f939740 1 --2- 172.21.15.42:0/2993517711 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561d72132390 0x561d7257f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:28.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.159+0000 7f606f939740 1 --2- 172.21.15.42:0/2993517711 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561d721119f0 0x561d725151e0 secure :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f605c008510 tx=0x7f605c02e720 comp rx=0 tx=0).stop 2024-08-30T09:15:28.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.159+0000 7f606f939740 1 -- 172.21.15.42:0/2993517711 >> 172.21.15.42:0/2993517711 conn(0x561d72147460 msgr2=0x561d724f47c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:15:28.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.159+0000 7f604a7fc640 0 client.17883 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:15:28.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.160+0000 7f606f939740 1 -- 172.21.15.42:0/2993517711 shutdown_connections 2024-08-30T09:15:28.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.160+0000 7f606f939740 1 -- 172.21.15.42:0/2993517711 wait complete. 2024-08-30T09:15:28.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.160+0000 7f606f939740 20 client.17883 trim_cache size 0 max 16384 2024-08-30T09:15:28.168 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-08-30T09:15:28.168 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "IDLE (waiting for more scrubs)", 2024-08-30T09:15:28.168 INFO:teuthology.orchestra.run.smithi042.stdout: "scrubs": { 2024-08-30T09:15:28.168 INFO:teuthology.orchestra.run.smithi042.stdout: "3f8863e7-c1a4-472c-8bd7-690aeeb13b82": { 2024-08-30T09:15:28.168 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:15:28.168 INFO:teuthology.orchestra.run.smithi042.stdout: "tag": "3f8863e7-c1a4-472c-8bd7-690aeeb13b82", 2024-08-30T09:15:28.169 INFO:teuthology.orchestra.run.smithi042.stdout: "options": "recursive,force" 2024-08-30T09:15:28.169 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:15:28.169 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:15:28.169 INFO:teuthology.orchestra.run.smithi042.stdout: "scrub_stats": { 2024-08-30T09:15:28.169 INFO:teuthology.orchestra.run.smithi042.stdout: "3f8863e7-c1a4-472c-8bd7-690aeeb13b82": { 2024-08-30T09:15:28.169 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:15:25.981002+0000", 2024-08-30T09:15:28.169 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:15:28.169 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:15:28.169 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:15:28.169 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:15:28.169 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:15:28.169 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:15:28.170 INFO:teuthology.orchestra.run.smithi042.stdout: "d325166d-e7e4-4ac2-9eaf-fa602e480cf6": { 2024-08-30T09:15:28.170 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:14:30.861941+0000", 2024-08-30T09:15:28.170 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:15:28.170 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:15:28.170 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:15:28.170 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:15:28.170 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:15:28.170 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:15:28.170 INFO:teuthology.orchestra.run.smithi042.stdout: "03521407-a747-49b9-a415-6233551c26fb": { 2024-08-30T09:15:28.170 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:13:39.387817+0000", 2024-08-30T09:15:28.170 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:15:28.170 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:15:28.170 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:15:28.170 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:15:28.171 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:15:28.171 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:15:28.171 INFO:teuthology.orchestra.run.smithi042.stdout: "0cbf734e-f988-4b21-acac-57b55e3bde62": { 2024-08-30T09:15:28.171 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:12:45.306738+0000", 2024-08-30T09:15:28.171 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:15:28.171 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:15:28.171 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:15:28.171 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:15:28.171 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:15:28.171 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:15:28.171 INFO:teuthology.orchestra.run.smithi042.stdout: "5e2ed104-0625-478a-a78b-584e7f4c2817": { 2024-08-30T09:15:28.171 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:00:33.928400+0000", 2024-08-30T09:15:28.171 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:15:28.171 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:15:28.172 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:15:28.172 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:15:28.172 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:15:28.172 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:15:28.172 INFO:teuthology.orchestra.run.smithi042.stdout: "c827ad7a-44ee-4e1d-9bb3-51fb7ab00c73": { 2024-08-30T09:15:28.172 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:02:21.686517+0000", 2024-08-30T09:15:28.172 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:15:28.172 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:15:28.172 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:15:28.172 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:15:28.172 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:15:28.172 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:15:28.172 INFO:teuthology.orchestra.run.smithi042.stdout: "85134e4b-ab4a-4885-b07e-a3ad01eaf13d": { 2024-08-30T09:15:28.172 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:01:27.738462+0000", 2024-08-30T09:15:28.173 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:15:28.173 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:15:28.173 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:15:28.173 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:15:28.173 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:15:28.173 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:15:28.173 INFO:teuthology.orchestra.run.smithi042.stdout: "749f4867-0b70-49e5-af2d-8362a240ebc3": { 2024-08-30T09:15:28.173 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:03:16.386554+0000", 2024-08-30T09:15:28.173 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:15:28.173 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:15:28.173 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:15:28.173 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:15:28.173 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:15:28.174 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:15:28.174 INFO:teuthology.orchestra.run.smithi042.stdout: "88cd7478-2b20-47de-ba51-256479749232": { 2024-08-30T09:15:28.174 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:04:08.922625+0000", 2024-08-30T09:15:28.174 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:15:28.174 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:15:28.174 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:15:28.174 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:15:28.174 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:15:28.174 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:15:28.174 INFO:teuthology.orchestra.run.smithi042.stdout: "347d05ed-4bd3-4590-b099-314f719aab39": { 2024-08-30T09:15:28.174 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:05:00.439721+0000", 2024-08-30T09:15:28.174 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:15:28.174 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:15:28.174 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:15:28.175 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:15:28.175 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:15:28.175 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:15:28.175 INFO:teuthology.orchestra.run.smithi042.stdout: "1af4252a-4a99-47e9-bbf0-becf5e573dd5": { 2024-08-30T09:15:28.175 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:05:48.565881+0000", 2024-08-30T09:15:28.175 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:15:28.175 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:15:28.175 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:15:28.175 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:15:28.175 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:15:28.175 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:15:28.175 INFO:teuthology.orchestra.run.smithi042.stdout: "665ec740-7254-49cb-84ef-deff71c461b0": { 2024-08-30T09:15:28.175 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:06:37.144035+0000", 2024-08-30T09:15:28.175 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:15:28.176 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:15:28.176 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:15:28.176 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:15:28.176 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:15:28.176 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:15:28.176 INFO:teuthology.orchestra.run.smithi042.stdout: "598d8a75-f1cc-4388-80bc-a408bdb4eaac": { 2024-08-30T09:15:28.176 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:07:29.721122+0000", 2024-08-30T09:15:28.176 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:15:28.176 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:15:28.176 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:15:28.176 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:15:28.176 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:15:28.176 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:15:28.176 INFO:teuthology.orchestra.run.smithi042.stdout: "b62c4ffc-9387-4677-b9ba-1e94ba308f17": { 2024-08-30T09:15:28.177 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:11:03.363478+0000", 2024-08-30T09:15:28.177 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:15:28.177 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:15:28.177 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:15:28.177 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:15:28.177 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:15:28.177 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:15:28.177 INFO:teuthology.orchestra.run.smithi042.stdout: "a7210f9a-046b-4188-953c-819b9f1863a9": { 2024-08-30T09:15:28.177 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:10:07.608430+0000", 2024-08-30T09:15:28.177 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:15:28.177 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:15:28.177 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:15:28.177 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:15:28.177 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:15:28.178 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:15:28.178 INFO:teuthology.orchestra.run.smithi042.stdout: "99a29ec1-9455-4ae4-854a-398b32896675": { 2024-08-30T09:15:28.178 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:08:24.923176+0000", 2024-08-30T09:15:28.178 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:15:28.178 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:15:28.178 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:15:28.178 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:15:28.178 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:15:28.178 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:15:28.178 INFO:teuthology.orchestra.run.smithi042.stdout: "ff464e7f-123b-4bc6-bc7e-aced018168de": { 2024-08-30T09:15:28.178 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:11:54.879598+0000", 2024-08-30T09:15:28.178 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:15:28.178 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:15:28.179 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:15:28.179 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:15:28.179 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:15:28.179 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:15:28.179 INFO:teuthology.orchestra.run.smithi042.stdout: "37f2f28f-496f-41a6-b4cf-8846fb3f43ab": { 2024-08-30T09:15:28.179 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:09:17.428286+0000", 2024-08-30T09:15:28.179 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:15:28.179 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:15:28.179 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:15:28.179 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:15:28.179 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:15:28.179 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:15:28.179 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:15:28.179 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-08-30T09:15:28.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.169+0000 7f6063fff640 1 -- 172.21.15.42:0/2965869584 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f60500b4b90 msgr2=0x7f60500b70a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:15:28.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.169+0000 7f6063fff640 1 --2- 172.21.15.42:0/2965869584 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f60500b4b90 0x7f60500b70a0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f60540ccee0 tx=0x7f6054004680 comp rx=0 tx=0).stop 2024-08-30T09:15:28.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.169+0000 7f6063fff640 1 -- 172.21.15.42:0/2965869584 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f60641a08a0 msgr2=0x7f60641a5ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:15:28.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.169+0000 7f6063fff640 1 --2- 172.21.15.42:0/2965869584 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f60641a08a0 0x7f60641a5ba0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f605400a850 tx=0x7f605400ad10 comp rx=0 tx=0).stop 2024-08-30T09:15:28.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.169+0000 7f6063fff640 1 -- 172.21.15.42:0/2965869584 shutdown_connections 2024-08-30T09:15:28.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.169+0000 7f6063fff640 1 --2- 172.21.15.42:0/2965869584 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f60500b4b90 0x7f60500b70a0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:28.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.169+0000 7f6063fff640 1 --2- 172.21.15.42:0/2965869584 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f60641a7530 0x7f60641a0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:28.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.169+0000 7f6063fff640 1 --2- 172.21.15.42:0/2965869584 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f60641a08a0 0x7f60641a5ba0 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:28.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.169+0000 7f6063fff640 1 --2- 172.21.15.42:0/2965869584 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f60640731e0 0x7f60641a6ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:28.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.169+0000 7f6063fff640 1 -- 172.21.15.42:0/2965869584 >> 172.21.15.42:0/2965869584 conn(0x7f6064078420 msgr2=0x7f60640706a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:15:28.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.170+0000 7f6063fff640 1 -- 172.21.15.42:0/2965869584 shutdown_connections 2024-08-30T09:15:28.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:15:28.170+0000 7f6063fff640 1 -- 172.21.15.42:0/2965869584 wait complete. 2024-08-30T09:15:28.883 INFO:tasks.cephfs.filesystem:scrub status for tag:3f8863e7-c1a4-472c-8bd7-690aeeb13b82 - {'path': '/', 'tag': '3f8863e7-c1a4-472c-8bd7-690aeeb13b82', 'options': 'recursive,force'} 2024-08-30T09:15:29.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:15:28 smithi042 ceph-mon[18652]: pgmap v814: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 3.7 MiB/s wr, 428 op/s 2024-08-30T09:15:29.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:15:28 smithi190 ceph-mon[22001]: pgmap v814: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 3.7 MiB/s wr, 428 op/s 2024-08-30T09:15:29.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:15:28 smithi167 ceph-mon[22063]: pgmap v814: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 3.7 MiB/s wr, 428 op/s 2024-08-30T09:15:30.953 INFO:teuthology.orchestra.run.smithi042.stderr:progress: 885.0 s, 489.2 tps, lat 212.935 ms stddev 267.005 2024-08-30T09:15:31.156 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.154+0000 7f371db7b640 1 -- 172.21.15.42:0/1876398492 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f371810a160 msgr2=0x7f371810c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:15:31.157 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.154+0000 7f371db7b640 1 --2- 172.21.15.42:0/1876398492 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f371810a160 0x7f371810c580 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f3710009be0 tx=0x7f371002a710 comp rx=0 tx=0).stop 2024-08-30T09:15:31.168 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.162+0000 7f371db7b640 1 -- 172.21.15.42:0/1876398492 shutdown_connections 2024-08-30T09:15:31.168 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.162+0000 7f371db7b640 1 --2- 172.21.15.42:0/1876398492 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f371810cb50 0x7f371810f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:31.168 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.162+0000 7f371db7b640 1 --2- 172.21.15.42:0/1876398492 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f371810f770 0x7f371811a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:31.168 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.162+0000 7f371db7b640 1 --2- 172.21.15.42:0/1876398492 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f371810a160 0x7f371810c580 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:31.168 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.162+0000 7f371db7b640 1 -- 172.21.15.42:0/1876398492 >> 172.21.15.42:0/1876398492 conn(0x7f37180705b0 msgr2=0x7f3718072a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:15:31.175 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.172+0000 7f371db7b640 1 -- 172.21.15.42:0/1876398492 shutdown_connections 2024-08-30T09:15:31.178 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.176+0000 7f371db7b640 1 -- 172.21.15.42:0/1876398492 wait complete. 2024-08-30T09:15:31.178 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.177+0000 7f371db7b640 1 Processor -- start 2024-08-30T09:15:31.185 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.183+0000 7f371db7b640 1 -- start start 2024-08-30T09:15:31.186 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.183+0000 7f371db7b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f371810a160 0x7f37181a8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:31.186 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.184+0000 7f371db7b640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f371810cb50 0x7f37181a9150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:31.186 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.184+0000 7f371db7b640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f371810f770 0x7f37181a9d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:31.186 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.184+0000 7f371db7b640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f371811dea0 con 0x7f371810cb50 2024-08-30T09:15:31.186 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.184+0000 7f371db7b640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f371811dd20 con 0x7f371810f770 2024-08-30T09:15:31.186 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.184+0000 7f371db7b640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f371811e020 con 0x7f371810a160 2024-08-30T09:15:31.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:15:30 smithi190 ceph-mon[22001]: pgmap v815: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 4.8 MiB/s wr, 616 op/s 2024-08-30T09:15:31.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:15:30 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-30T09:15:31.189 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.186+0000 7f37177fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f371810a160 0x7f37181a8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:15:31.189 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.187+0000 7f37177fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f371810a160 0x7f37181a8ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:50800/0 (socket says 172.21.15.42:50800) 2024-08-30T09:15:31.189 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.187+0000 7f37177fe640 1 -- 172.21.15.42:0/1819173151 learned_addr learned my addr 172.21.15.42:0/1819173151 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:15:31.193 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.188+0000 7f37177fe640 1 -- 172.21.15.42:0/1819173151 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f371810f770 msgr2=0x7f37181a9d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:15:31.193 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.188+0000 7f37177fe640 1 --2- 172.21.15.42:0/1819173151 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f371810f770 0x7f37181a9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:31.193 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.188+0000 7f37177fe640 1 -- 172.21.15.42:0/1819173151 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f371810cb50 msgr2=0x7f37181a9150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:15:31.193 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.188+0000 7f37177fe640 1 --2- 172.21.15.42:0/1819173151 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f371810cb50 0x7f37181a9150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:31.193 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.188+0000 7f37177fe640 1 -- 172.21.15.42:0/1819173151 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37181aa520 con 0x7f371810a160 2024-08-30T09:15:31.200 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.196+0000 7f37177fe640 1 --2- 172.21.15.42:0/1819173151 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f371810a160 0x7f37181a8ae0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f3710034040 tx=0x7f3710031ed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:15:31.200 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.198+0000 7f3714ff9640 1 -- 172.21.15.42:0/1819173151 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3710041070 con 0x7f371810a160 2024-08-30T09:15:31.200 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.198+0000 7f3714ff9640 1 -- 172.21.15.42:0/1819173151 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3710030550 con 0x7f371810a160 2024-08-30T09:15:31.200 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.198+0000 7f3714ff9640 1 -- 172.21.15.42:0/1819173151 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3710036040 con 0x7f371810a160 2024-08-30T09:15:31.200 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.198+0000 7f371db7b640 1 -- 172.21.15.42:0/1819173151 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37181aa7b0 con 0x7f371810a160 2024-08-30T09:15:31.200 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.198+0000 7f371db7b640 1 -- 172.21.15.42:0/1819173151 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37181b0d20 con 0x7f371810a160 2024-08-30T09:15:31.212 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.206+0000 7f3724bc2740 1 Processor -- start 2024-08-30T09:15:31.212 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.206+0000 7f3714ff9640 1 -- 172.21.15.42:0/1819173151 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7f3710044070 con 0x7f371810a160 2024-08-30T09:15:31.222 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.219+0000 7f3724bc2740 1 -- start start 2024-08-30T09:15:31.222 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.219+0000 7f3724bc2740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556b198111b0 0x556b19bb6f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:31.222 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.219+0000 7f3724bc2740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556b197e82f0 0x556b19bb75d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:31.222 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.219+0000 7f3724bc2740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556b197f04d0 0x556b19bb8120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:31.222 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.219+0000 7f3724bc2740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x556b19b2a000 con 0x556b198111b0 2024-08-30T09:15:31.222 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.219+0000 7f3724bc2740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x556b19b19590 con 0x556b197e82f0 2024-08-30T09:15:31.222 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.219+0000 7f3724bc2740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x556b19b1d540 con 0x556b197f04d0 2024-08-30T09:15:31.222 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.220+0000 7f37177fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556b197e82f0 0x556b19bb75d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:15:31.222 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.220+0000 7f37177fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556b197e82f0 0x556b19bb75d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:47516/0 (socket says 172.21.15.42:47516) 2024-08-30T09:15:31.222 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.220+0000 7f37177fe640 1 -- 172.21.15.42:0/3821927757 learned_addr learned my addr 172.21.15.42:0/3821927757 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:15:31.225 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.223+0000 7f37177fe640 1 -- 172.21.15.42:0/3821927757 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556b197f04d0 msgr2=0x556b19bb8120 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-30T09:15:31.225 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.223+0000 7f37177fe640 1 --2- 172.21.15.42:0/3821927757 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556b197f04d0 0x556b19bb8120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:31.225 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.223+0000 7f37177fe640 1 -- 172.21.15.42:0/3821927757 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556b198111b0 msgr2=0x556b19bb6f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:15:31.225 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.223+0000 7f37177fe640 1 --2- 172.21.15.42:0/3821927757 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556b198111b0 0x556b19bb6f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:31.225 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.223+0000 7f37177fe640 1 -- 172.21.15.42:0/3821927757 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b1987c790 con 0x556b197e82f0 2024-08-30T09:15:31.225 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.223+0000 7f3714ff9640 1 --2- 172.21.15.42:0/1819173151 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f37000b4d00 0x7f37000b7210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:31.225 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.223+0000 7f37177fe640 1 --2- 172.21.15.42:0/3821927757 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556b197e82f0 0x556b19bb75d0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f3710109f40 tx=0x7f371010e1a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d1bc99a8232a542 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:15:31.226 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.224+0000 7f3714ff9640 1 -- 172.21.15.42:0/1819173151 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f37101063f0 con 0x7f371810a160 2024-08-30T09:15:31.228 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.226+0000 7f36fdffb640 1 -- 172.21.15.42:0/3821927757 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37100c8860 con 0x556b197e82f0 2024-08-30T09:15:31.228 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.226+0000 7f36fdffb640 1 -- 172.21.15.42:0/3821927757 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f37100cc540 con 0x556b197e82f0 2024-08-30T09:15:31.229 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.228+0000 7f3724bc2740 1 -- 172.21.15.42:0/3821927757 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556b197e82f0 msgr2=0x556b19bb75d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:15:31.230 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.228+0000 7f3724bc2740 1 --2- 172.21.15.42:0/3821927757 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556b197e82f0 0x556b19bb75d0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f3710109f40 tx=0x7f371010e1a0 comp rx=0 tx=0).stop 2024-08-30T09:15:31.235 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.233+0000 7f3717fff640 1 --2- 172.21.15.42:0/1819173151 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f37000b4d00 0x7f37000b7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:15:31.235 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.233+0000 7f3724bc2740 1 -- 172.21.15.42:0/3821927757 shutdown_connections 2024-08-30T09:15:31.236 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.233+0000 7f3724bc2740 1 --2- 172.21.15.42:0/3821927757 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556b197f04d0 0x556b19bb8120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:31.236 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.233+0000 7f3724bc2740 1 --2- 172.21.15.42:0/3821927757 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556b197e82f0 0x556b19bb75d0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:31.236 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.233+0000 7f3724bc2740 1 --2- 172.21.15.42:0/3821927757 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556b198111b0 0x556b19bb6f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:31.236 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.233+0000 7f3724bc2740 1 -- 172.21.15.42:0/3821927757 >> 172.21.15.42:0/3821927757 conn(0x556b19805300 msgr2=0x556b19bb29f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:15:31.239 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.238+0000 7f3724bc2740 1 -- 172.21.15.42:0/3821927757 shutdown_connections 2024-08-30T09:15:31.244 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.242+0000 7f3724bc2740 1 -- 172.21.15.42:0/3821927757 wait complete. 2024-08-30T09:15:31.244 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.242+0000 7f3724bc2740 1 -- start start 2024-08-30T09:15:31.244 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.242+0000 7f3724bc2740 1 Processor -- start 2024-08-30T09:15:31.249 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.247+0000 7f3724bc2740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556b197e82f0 0x556b19bd1680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:31.249 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.247+0000 7f3724bc2740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556b197f04d0 0x556b19bd1d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:31.249 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.247+0000 7f3724bc2740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556b198111b0 0x556b19bd28b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:31.249 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.247+0000 7f3724bc2740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x556b19a52ab0 con 0x556b197f04d0 2024-08-30T09:15:31.249 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.247+0000 7f3724bc2740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x556b19aa0680 con 0x556b197e82f0 2024-08-30T09:15:31.250 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.247+0000 7f3724bc2740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x556b19bb8a00 con 0x556b198111b0 2024-08-30T09:15:31.250 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.247+0000 7f3724bc2740 10 client.0 _ll_register_callbacks cb 0x556b19828a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:15:31.252 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.247+0000 7f3717fff640 1 --2- 172.21.15.42:0/1819173151 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f37000b4d00 0x7f37000b7210 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f3708009510 tx=0x7f3708009480 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:15:31.253 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.249+0000 7f37177fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556b197e82f0 0x556b19bd1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:15:31.253 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.249+0000 7f37177fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556b197e82f0 0x556b19bd1680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:47524/0 (socket says 172.21.15.42:47524) 2024-08-30T09:15:31.253 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.249+0000 7f37177fe640 1 -- 172.21.15.42:0/4128466705 learned_addr learned my addr 172.21.15.42:0/4128466705 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:15:31.253 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.250+0000 7f3717fff640 1 --2- 172.21.15.42:0/4128466705 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556b198111b0 0x556b19bd28b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:15:31.253 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.250+0000 7f37177fe640 1 -- 172.21.15.42:0/4128466705 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556b198111b0 msgr2=0x556b19bd28b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:15:31.253 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.250+0000 7f37177fe640 1 --2- 172.21.15.42:0/4128466705 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556b198111b0 0x556b19bd28b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:31.253 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.250+0000 7f37177fe640 1 -- 172.21.15.42:0/4128466705 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556b197f04d0 msgr2=0x556b19bd1d40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-30T09:15:31.253 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.250+0000 7f37177fe640 1 --2- 172.21.15.42:0/4128466705 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556b197f04d0 0x556b19bd1d40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:31.253 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.250+0000 7f37177fe640 1 -- 172.21.15.42:0/4128466705 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b198f5620 con 0x556b197e82f0 2024-08-30T09:15:31.256 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.254+0000 7f37177fe640 1 --2- 172.21.15.42:0/4128466705 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556b197e82f0 0x556b19bd1680 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f37100c9c10 tx=0x7f37100d5ec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=57e446e43844cc4e server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:15:31.262 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.260+0000 7f36f6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-30T09:15:31.262 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.260+0000 7f36f6ffd640 1 -- 172.21.15.42:0/4128466705 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f371004b070 con 0x556b197e82f0 2024-08-30T09:15:31.262 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.260+0000 7f3724bc2740 1 -- 172.21.15.42:0/4128466705 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f37100cc540 con 0x556b197e82f0 2024-08-30T09:15:31.262 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.260+0000 7f36f6ffd640 1 -- 172.21.15.42:0/4128466705 <== mon.2 v2:172.21.15.167:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f3710106120 con 0x556b197e82f0 2024-08-30T09:15:31.265 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.264+0000 7f36f6ffd640 1 -- 172.21.15.42:0/4128466705 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f37100cc1a0 con 0x556b197e82f0 2024-08-30T09:15:31.268 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.266+0000 7f3724bc2740 10 client.27728 fetch_fsmap learned FSMap version 48 2024-08-30T09:15:31.268 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.266+0000 7f3724bc2740 1 -- 172.21.15.42:0/4128466705 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x556b19bdb800 con 0x556b197e82f0 2024-08-30T09:15:31.273 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.271+0000 7f36f6ffd640 1 -- 172.21.15.42:0/4128466705 <== mon.2 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f37100cc540 con 0x556b197e82f0 2024-08-30T09:15:31.273 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.271+0000 7f36f6ffd640 1 -- 172.21.15.42:0/4128466705 <== mon.2 v2:172.21.15.167:3300/0 5 ==== fsmap(e 48) ==== 8366+0+0 (secure 0 0 0) 0x7f37100ccc90 con 0x556b197e82f0 2024-08-30T09:15:31.273 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.271+0000 7f3724bc2740 10 client.27728 fetch_fsmap finished waiting for FSMap version 48 2024-08-30T09:15:31.273 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.271+0000 7f3724bc2740 10 client.27728 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-30T09:15:31.273 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.271+0000 7f3724bc2740 20 client.27728 populate_metadata read hostname 'smithi042' 2024-08-30T09:15:31.273 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.271+0000 7f3724bc2740 1 --2- 172.21.15.42:0/4128466705 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x556b197cfd80 0x556b19bdece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:31.273 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.271+0000 7f3724bc2740 4 client.27728 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-30T09:15:31.273 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.271+0000 7f3724bc2740 1 -- 172.21.15.42:0/4128466705 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556b19b2bf50 con 0x556b197cfd80 2024-08-30T09:15:31.275 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.271+0000 7f3716ffd640 1 --2- 172.21.15.42:0/4128466705 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x556b197cfd80 0x556b19bdece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:15:31.275 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.272+0000 7f3716ffd640 1 --2- 172.21.15.42:0/4128466705 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x556b197cfd80 0x556b19bdece0 crc :-1 s=READY pgs=301 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=126af3df8db631a6 server_cookie=7c80ffb900275e6e in_seq=0 out_seq=0 2024-08-30T09:15:31.275 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.272+0000 7f36f6ffd640 10 client.27728 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:15:31.276 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.273+0000 7f36f6ffd640 1 -- 172.21.15.42:0/4128466705 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x556b19b2bf50 con 0x556b197cfd80 2024-08-30T09:15:31.276 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.273+0000 7f36f6ffd640 10 client.27728 handle_command_reply: tid=0 2024-08-30T09:15:31.276 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.273+0000 7f3724bc2740 1 client.27728 shutdown 2024-08-30T09:15:31.293 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.292+0000 7f3724bc2740 1 -- 172.21.15.42:0/4128466705 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556b197e82f0 msgr2=0x556b19bd1680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:15:31.293 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.292+0000 7f3724bc2740 1 --2- 172.21.15.42:0/4128466705 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556b197e82f0 0x556b19bd1680 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f37100c9c10 tx=0x7f37100d5ec0 comp rx=0 tx=0).stop 2024-08-30T09:15:31.298 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.294+0000 7f3724bc2740 1 -- 172.21.15.42:0/4128466705 shutdown_connections 2024-08-30T09:15:31.298 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.294+0000 7f3724bc2740 1 --2- 172.21.15.42:0/4128466705 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x556b197cfd80 0x556b19bdece0 crc :-1 s=READY pgs=301 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:31.298 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.294+0000 7f3724bc2740 1 --2- 172.21.15.42:0/4128466705 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556b198111b0 0x556b19bd28b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:31.298 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.294+0000 7f3724bc2740 1 --2- 172.21.15.42:0/4128466705 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556b197f04d0 0x556b19bd1d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:31.298 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.294+0000 7f3724bc2740 1 --2- 172.21.15.42:0/4128466705 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556b197e82f0 0x556b19bd1680 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:31.298 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.294+0000 7f3724bc2740 1 -- 172.21.15.42:0/4128466705 >> 172.21.15.42:0/4128466705 conn(0x556b19805300 msgr2=0x556b19bb0bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:15:31.298 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.294+0000 7f36f6ffd640 0 client.27728 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:15:31.302 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.297+0000 7f3724bc2740 1 -- 172.21.15.42:0/4128466705 shutdown_connections 2024-08-30T09:15:31.302 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.297+0000 7f3724bc2740 1 -- 172.21.15.42:0/4128466705 wait complete. 2024-08-30T09:15:31.302 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.298+0000 7f3724bc2740 20 client.27728 trim_cache size 0 max 16384 2024-08-30T09:15:31.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:15:30 smithi167 ceph-mon[22063]: pgmap v815: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 4.8 MiB/s wr, 616 op/s 2024-08-30T09:15:31.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:15:30 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-30T09:15:31.349 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.348+0000 7f3724bc2740 1 Processor -- start 2024-08-30T09:15:31.350 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.348+0000 7f3724bc2740 1 -- start start 2024-08-30T09:15:31.351 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.348+0000 7f3724bc2740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556b19805300 0x556b19bbb4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:31.351 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.348+0000 7f3724bc2740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556b197e82f0 0x556b19bbb9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:31.351 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.349+0000 7f3724bc2740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556b197f04d0 0x556b19bb6b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:31.351 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.349+0000 7f3724bc2740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f37040125d0 con 0x556b19805300 2024-08-30T09:15:31.351 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.349+0000 7f3724bc2740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f370400fe70 con 0x556b197e82f0 2024-08-30T09:15:31.351 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.349+0000 7f3724bc2740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f370400dcc0 con 0x556b197f04d0 2024-08-30T09:15:31.351 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.350+0000 7f3717fff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556b197f04d0 0x556b19bb6b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:15:31.351 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.350+0000 7f3717fff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556b197f04d0 0x556b19bb6b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:50820/0 (socket says 172.21.15.42:50820) 2024-08-30T09:15:31.352 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.350+0000 7f3717fff640 1 -- 172.21.15.42:0/3154465283 learned_addr learned my addr 172.21.15.42:0/3154465283 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:15:31.353 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.351+0000 7f3716ffd640 1 --2- 172.21.15.42:0/3154465283 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556b197e82f0 0x556b19bbb9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:15:31.355 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.352+0000 7f37177fe640 1 --2- 172.21.15.42:0/3154465283 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556b19805300 0x556b19bbb4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:15:31.356 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.352+0000 7f3716ffd640 1 -- 172.21.15.42:0/3154465283 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556b197f04d0 msgr2=0x556b19bb6b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:15:31.356 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.352+0000 7f3716ffd640 1 --2- 172.21.15.42:0/3154465283 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556b197f04d0 0x556b19bb6b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:31.356 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.352+0000 7f3716ffd640 1 -- 172.21.15.42:0/3154465283 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556b19805300 msgr2=0x556b19bbb4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:15:31.356 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.352+0000 7f3716ffd640 1 --2- 172.21.15.42:0/3154465283 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556b19805300 0x556b19bbb4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:31.356 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.352+0000 7f3716ffd640 1 -- 172.21.15.42:0/3154465283 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b19bb8af0 con 0x556b197e82f0 2024-08-30T09:15:31.356 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.353+0000 7f3716ffd640 1 --2- 172.21.15.42:0/3154465283 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556b197e82f0 0x556b19bbb9e0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x556b19bdf3e0 tx=0x7f370c005850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3283a8d5f97506f9 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:15:31.356 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.353+0000 7f36fd7fa640 1 -- 172.21.15.42:0/3154465283 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f370c004020 con 0x556b197e82f0 2024-08-30T09:15:31.356 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.353+0000 7f36fd7fa640 1 -- 172.21.15.42:0/3154465283 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f370c01c030 con 0x556b197e82f0 2024-08-30T09:15:31.356 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.353+0000 7f36fd7fa640 1 -- 172.21.15.42:0/3154465283 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f370c010690 con 0x556b197e82f0 2024-08-30T09:15:31.357 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.354+0000 7f3724bc2740 1 -- 172.21.15.42:0/3154465283 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556b197e82f0 msgr2=0x556b19bbb9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:15:31.357 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.354+0000 7f3724bc2740 1 --2- 172.21.15.42:0/3154465283 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556b197e82f0 0x556b19bbb9e0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x556b19bdf3e0 tx=0x7f370c005850 comp rx=0 tx=0).stop 2024-08-30T09:15:31.357 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.355+0000 7f3724bc2740 1 -- 172.21.15.42:0/3154465283 shutdown_connections 2024-08-30T09:15:31.357 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.355+0000 7f3724bc2740 1 --2- 172.21.15.42:0/3154465283 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556b197f04d0 0x556b19bb6b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:31.357 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.355+0000 7f3724bc2740 1 --2- 172.21.15.42:0/3154465283 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556b197e82f0 0x556b19bbb9e0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:31.357 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.355+0000 7f3724bc2740 1 --2- 172.21.15.42:0/3154465283 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556b19805300 0x556b19bbb4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:31.357 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.355+0000 7f3724bc2740 1 -- 172.21.15.42:0/3154465283 >> 172.21.15.42:0/3154465283 conn(0x556b19bd0450 msgr2=0x556b19bb2330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:15:31.357 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.355+0000 7f3724bc2740 1 -- 172.21.15.42:0/3154465283 shutdown_connections 2024-08-30T09:15:31.358 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.355+0000 7f3724bc2740 1 -- 172.21.15.42:0/3154465283 wait complete. 2024-08-30T09:15:31.358 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.356+0000 7f3724bc2740 1 -- start start 2024-08-30T09:15:31.358 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.356+0000 7f3724bc2740 1 Processor -- start 2024-08-30T09:15:31.359 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.357+0000 7f3724bc2740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556b197e82f0 0x556b19bd4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:31.359 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.357+0000 7f3724bc2740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556b197f04d0 0x556b19bd5260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:31.359 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.357+0000 7f3724bc2740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556b19805300 0x556b19bd8de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:31.359 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.357+0000 7f3724bc2740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f370400e330 con 0x556b197f04d0 2024-08-30T09:15:31.360 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.357+0000 7f3724bc2740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f370400f7a0 con 0x556b19805300 2024-08-30T09:15:31.360 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.357+0000 7f3724bc2740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f370400ea00 con 0x556b197e82f0 2024-08-30T09:15:31.360 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.357+0000 7f3724bc2740 10 client.0 _ll_register_callbacks cb 0x7f3710109d90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:15:31.360 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.357+0000 7f37177fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556b197e82f0 0x556b19bd4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:15:31.360 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.357+0000 7f37177fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556b197e82f0 0x556b19bd4ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:50836/0 (socket says 172.21.15.42:50836) 2024-08-30T09:15:31.360 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.357+0000 7f37177fe640 1 -- 172.21.15.42:0/2630719905 learned_addr learned my addr 172.21.15.42:0/2630719905 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:15:31.360 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.357+0000 7f3717fff640 1 --2- 172.21.15.42:0/2630719905 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556b19805300 0x556b19bd8de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:15:31.361 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.358+0000 7f3716ffd640 1 --2- 172.21.15.42:0/2630719905 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556b197f04d0 0x556b19bd5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:15:31.362 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.358+0000 7f37177fe640 1 -- 172.21.15.42:0/2630719905 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556b19805300 msgr2=0x556b19bd8de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:15:31.362 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.358+0000 7f37177fe640 1 --2- 172.21.15.42:0/2630719905 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556b19805300 0x556b19bd8de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:31.362 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.358+0000 7f37177fe640 1 -- 172.21.15.42:0/2630719905 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556b197f04d0 msgr2=0x556b19bd5260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:15:31.362 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.358+0000 7f37177fe640 1 --2- 172.21.15.42:0/2630719905 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556b197f04d0 0x556b19bd5260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:31.362 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.358+0000 7f37177fe640 1 -- 172.21.15.42:0/2630719905 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b19bd9540 con 0x556b197e82f0 2024-08-30T09:15:31.362 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.358+0000 7f3717fff640 1 --2- 172.21.15.42:0/2630719905 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556b19805300 0x556b19bd8de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-30T09:15:31.362 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.358+0000 7f37177fe640 1 --2- 172.21.15.42:0/2630719905 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556b197e82f0 0x556b19bd4ba0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f371003ca50 tx=0x7f37100cd210 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6142bb464320a03e server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:15:31.362 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.358+0000 7f36f6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-08-30T09:15:31.362 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.358+0000 7f36f6ffd640 1 -- 172.21.15.42:0/2630719905 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f371004b040 con 0x556b197e82f0 2024-08-30T09:15:31.362 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.359+0000 7f3724bc2740 1 -- 172.21.15.42:0/2630719905 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f370c01c030 con 0x556b197e82f0 2024-08-30T09:15:31.362 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.359+0000 7f36f6ffd640 1 -- 172.21.15.42:0/2630719905 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f37100ca980 con 0x556b197e82f0 2024-08-30T09:15:31.363 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.359+0000 7f36f6ffd640 1 -- 172.21.15.42:0/2630719905 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f37100cc380 con 0x556b197e82f0 2024-08-30T09:15:31.363 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.359+0000 7f3724bc2740 10 client.27676 fetch_fsmap learned FSMap version 48 2024-08-30T09:15:31.363 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.359+0000 7f3724bc2740 1 -- 172.21.15.42:0/2630719905 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x556b19bd9540 con 0x556b197e82f0 2024-08-30T09:15:31.363 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.360+0000 7f36f6ffd640 1 -- 172.21.15.42:0/2630719905 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f370c01c030 con 0x556b197e82f0 2024-08-30T09:15:31.363 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.360+0000 7f36f6ffd640 1 -- 172.21.15.42:0/2630719905 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 48) ==== 8366+0+0 (secure 0 0 0) 0x7f37100cbc90 con 0x556b197e82f0 2024-08-30T09:15:31.363 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.360+0000 7f3724bc2740 10 client.27676 fetch_fsmap finished waiting for FSMap version 48 2024-08-30T09:15:31.364 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.360+0000 7f3724bc2740 10 client.27676 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-30T09:15:31.364 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.360+0000 7f3724bc2740 20 client.27676 populate_metadata read hostname 'smithi042' 2024-08-30T09:15:31.364 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.360+0000 7f3724bc2740 1 --2- 172.21.15.42:0/2630719905 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x556b198111b0 0x556b19c4b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:31.364 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.360+0000 7f3724bc2740 4 client.27676 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-30T09:15:31.364 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.360+0000 7f3724bc2740 1 -- 172.21.15.42:0/2630719905 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556b19b279f0 con 0x556b198111b0 2024-08-30T09:15:31.364 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.360+0000 7f3716ffd640 1 --2- 172.21.15.42:0/2630719905 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x556b198111b0 0x556b19c4b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:15:31.364 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.361+0000 7f3716ffd640 1 --2- 172.21.15.42:0/2630719905 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x556b198111b0 0x556b19c4b7e0 crc :-1 s=READY pgs=302 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c9dfeadff010414 server_cookie=9544113e7a825c38 in_seq=0 out_seq=0 2024-08-30T09:15:31.364 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.361+0000 7f36f6ffd640 10 client.27676 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:15:31.381 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.379+0000 7f36f6ffd640 1 -- 172.21.15.42:0/2630719905 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556b19b279f0 con 0x556b198111b0 2024-08-30T09:15:31.381 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.379+0000 7f36f6ffd640 10 client.27676 handle_command_reply: tid=0 2024-08-30T09:15:31.381 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.379+0000 7f3724bc2740 1 client.27676 shutdown 2024-08-30T09:15:31.381 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.380+0000 7f3724bc2740 1 -- 172.21.15.42:0/2630719905 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556b197e82f0 msgr2=0x556b19bd4ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:15:31.382 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.380+0000 7f3724bc2740 1 --2- 172.21.15.42:0/2630719905 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556b197e82f0 0x556b19bd4ba0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f371003ca50 tx=0x7f37100cd210 comp rx=0 tx=0).stop 2024-08-30T09:15:31.382 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.381+0000 7f3724bc2740 1 -- 172.21.15.42:0/2630719905 shutdown_connections 2024-08-30T09:15:31.382 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.381+0000 7f3724bc2740 1 --2- 172.21.15.42:0/2630719905 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x556b198111b0 0x556b19c4b7e0 crc :-1 s=READY pgs=302 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:31.383 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.381+0000 7f3724bc2740 1 --2- 172.21.15.42:0/2630719905 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556b19805300 0x556b19bd8de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:31.383 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.381+0000 7f3724bc2740 1 --2- 172.21.15.42:0/2630719905 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556b197f04d0 0x556b19bd5260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:31.383 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.381+0000 7f3724bc2740 1 --2- 172.21.15.42:0/2630719905 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556b197e82f0 0x556b19bd4ba0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:31.383 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.381+0000 7f3724bc2740 1 -- 172.21.15.42:0/2630719905 >> 172.21.15.42:0/2630719905 conn(0x556b19bd0450 msgr2=0x556b19bb6b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:15:31.383 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.381+0000 7f36f6ffd640 0 client.27676 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:15:31.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:15:30 smithi042 ceph-mon[18652]: pgmap v815: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 4.8 MiB/s wr, 616 op/s 2024-08-30T09:15:31.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:15:30 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-30T09:15:31.384 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.382+0000 7f3724bc2740 1 -- 172.21.15.42:0/2630719905 shutdown_connections 2024-08-30T09:15:31.384 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.382+0000 7f3724bc2740 1 -- 172.21.15.42:0/2630719905 wait complete. 2024-08-30T09:15:31.384 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.382+0000 7f3724bc2740 20 client.27676 trim_cache size 0 max 16384 2024-08-30T09:15:31.385 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-08-30T09:15:31.385 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2024-08-30T09:15:31.385 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-08-30T09:15:31.385 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-08-30T09:15:31.390 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.387+0000 7f371db7b640 1 -- 172.21.15.42:0/1819173151 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f37000b4d00 msgr2=0x7f37000b7210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:15:31.390 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.388+0000 7f371db7b640 1 --2- 172.21.15.42:0/1819173151 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f37000b4d00 0x7f37000b7210 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f3708009510 tx=0x7f3708009480 comp rx=0 tx=0).stop 2024-08-30T09:15:31.391 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.388+0000 7f371db7b640 1 -- 172.21.15.42:0/1819173151 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f371810a160 msgr2=0x7f37181a8ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:15:31.391 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.388+0000 7f371db7b640 1 --2- 172.21.15.42:0/1819173151 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f371810a160 0x7f37181a8ae0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f3710034040 tx=0x7f3710031ed0 comp rx=0 tx=0).stop 2024-08-30T09:15:31.391 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.388+0000 7f371db7b640 1 -- 172.21.15.42:0/1819173151 shutdown_connections 2024-08-30T09:15:31.391 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.388+0000 7f371db7b640 1 --2- 172.21.15.42:0/1819173151 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f37000b4d00 0x7f37000b7210 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:31.391 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.388+0000 7f371db7b640 1 --2- 172.21.15.42:0/1819173151 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f371810f770 0x7f37181a9d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:31.391 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.388+0000 7f371db7b640 1 --2- 172.21.15.42:0/1819173151 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f371810cb50 0x7f37181a9150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:31.391 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.388+0000 7f371db7b640 1 --2- 172.21.15.42:0/1819173151 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f371810a160 0x7f37181a8ae0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:31.391 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.388+0000 7f371db7b640 1 -- 172.21.15.42:0/1819173151 >> 172.21.15.42:0/1819173151 conn(0x7f37180705b0 msgr2=0x7f371810af30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:15:31.391 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.389+0000 7f371db7b640 1 -- 172.21.15.42:0/1819173151 shutdown_connections 2024-08-30T09:15:31.391 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:31.389+0000 7f371db7b640 1 -- 172.21.15.42:0/1819173151 wait complete. 2024-08-30T09:15:32.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:15:31 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T09:15:32.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:15:31 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-30T09:15:32.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:15:31 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T09:15:32.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:15:31 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T09:15:32.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:15:31 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-30T09:15:32.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:15:31 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T09:15:32.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:15:31 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T09:15:32.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:15:31 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-30T09:15:32.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:15:31 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T09:15:33.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:15:32 smithi190 ceph-mon[22001]: pgmap v816: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 3.8 MiB/s wr, 513 op/s 2024-08-30T09:15:33.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:15:32 smithi190 ceph-mon[22001]: scrub summary: idle 2024-08-30T09:15:33.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:15:32 smithi190 ceph-mon[22001]: scrub summary: idle 2024-08-30T09:15:33.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:15:32 smithi167 ceph-mon[22063]: pgmap v816: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 3.8 MiB/s wr, 513 op/s 2024-08-30T09:15:33.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:15:32 smithi167 ceph-mon[22063]: scrub summary: idle 2024-08-30T09:15:33.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:15:32 smithi167 ceph-mon[22063]: scrub summary: idle 2024-08-30T09:15:33.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:15:32 smithi042 ceph-mon[18652]: pgmap v816: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 3.8 MiB/s wr, 513 op/s 2024-08-30T09:15:33.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:15:32 smithi042 ceph-mon[18652]: scrub summary: idle 2024-08-30T09:15:33.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:15:32 smithi042 ceph-mon[18652]: scrub summary: idle 2024-08-30T09:15:34.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:15:33 smithi190 ceph-mon[22001]: scrub summary: idle 2024-08-30T09:15:34.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:15:33 smithi167 ceph-mon[22063]: scrub summary: idle 2024-08-30T09:15:34.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:15:33 smithi042 ceph-mon[18652]: scrub summary: idle 2024-08-30T09:15:35.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:15:34 smithi190 ceph-mon[22001]: pgmap v817: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 4.5 MiB/s wr, 610 op/s 2024-08-30T09:15:35.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:15:34 smithi167 ceph-mon[22063]: pgmap v817: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 4.5 MiB/s wr, 610 op/s 2024-08-30T09:15:35.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:15:34 smithi042 ceph-mon[18652]: pgmap v817: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 4.5 MiB/s wr, 610 op/s 2024-08-30T09:15:35.953 INFO:teuthology.orchestra.run.smithi042.stderr:progress: 890.0 s, 503.4 tps, lat 201.202 ms stddev 233.818 2024-08-30T09:15:37.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:15:36 smithi190 ceph-mon[22001]: pgmap v818: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 4.5 MiB/s wr, 590 op/s 2024-08-30T09:15:37.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:15:36 smithi167 ceph-mon[22063]: pgmap v818: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 4.5 MiB/s wr, 590 op/s 2024-08-30T09:15:37.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:15:36 smithi042 ceph-mon[18652]: pgmap v818: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 4.5 MiB/s wr, 590 op/s 2024-08-30T09:15:39.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:15:38 smithi167 ceph-mon[22063]: pgmap v819: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 4.0 MiB/s wr, 537 op/s 2024-08-30T09:15:39.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:15:38 smithi042 ceph-mon[18652]: pgmap v819: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 4.0 MiB/s wr, 537 op/s 2024-08-30T09:15:39.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:15:38 smithi190 ceph-mon[22001]: pgmap v819: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 4.0 MiB/s wr, 537 op/s 2024-08-30T09:15:40.953 INFO:teuthology.orchestra.run.smithi042.stderr:progress: 895.0 s, 482.0 tps, lat 204.143 ms stddev 248.871 2024-08-30T09:15:41.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:15:40 smithi167 ceph-mon[22063]: pgmap v820: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 5.2 MiB/s wr, 643 op/s 2024-08-30T09:15:41.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:15:40 smithi042 ceph-mon[18652]: pgmap v820: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 5.2 MiB/s wr, 643 op/s 2024-08-30T09:15:41.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:15:40 smithi190 ceph-mon[22001]: pgmap v820: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 5.2 MiB/s wr, 643 op/s 2024-08-30T09:15:43.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:15:42 smithi167 ceph-mon[22063]: pgmap v821: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 3.9 MiB/s wr, 442 op/s 2024-08-30T09:15:43.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:15:42 smithi042 ceph-mon[18652]: pgmap v821: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 3.9 MiB/s wr, 442 op/s 2024-08-30T09:15:43.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:15:42 smithi190 ceph-mon[22001]: pgmap v821: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 3.9 MiB/s wr, 442 op/s 2024-08-30T09:15:44.543 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.537+0000 7f427968d640 1 -- 172.21.15.42:0/2373731057 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4274102d50 msgr2=0x7f4274107800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:15:44.543 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.537+0000 7f427968d640 1 --2- 172.21.15.42:0/2373731057 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4274102d50 0x7f4274107800 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f426c009ca0 tx=0x7f426c02a8f0 comp rx=0 tx=0).stop 2024-08-30T09:15:44.547 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.541+0000 7f427968d640 1 -- 172.21.15.42:0/2373731057 shutdown_connections 2024-08-30T09:15:44.547 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.541+0000 7f427968d640 1 --2- 172.21.15.42:0/2373731057 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f42741021a0 0x7f4274102610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:44.547 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.541+0000 7f427968d640 1 --2- 172.21.15.42:0/2373731057 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4274102d50 0x7f4274107800 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:44.547 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.541+0000 7f427968d640 1 --2- 172.21.15.42:0/2373731057 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4274101770 0x7f4274101b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:44.547 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.541+0000 7f427968d640 1 -- 172.21.15.42:0/2373731057 >> 172.21.15.42:0/2373731057 conn(0x7f42740fd430 msgr2=0x7f42740ff8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:15:44.550 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.545+0000 7f427968d640 1 -- 172.21.15.42:0/2373731057 shutdown_connections 2024-08-30T09:15:44.553 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.548+0000 7f427968d640 1 -- 172.21.15.42:0/2373731057 wait complete. 2024-08-30T09:15:44.553 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.549+0000 7f427968d640 1 Processor -- start 2024-08-30T09:15:44.557 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.553+0000 7f427968d640 1 -- start start 2024-08-30T09:15:44.557 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.553+0000 7f427968d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4274101770 0x7f4274073ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:44.557 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.553+0000 7f427968d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f42741021a0 0x7f42740745d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:44.557 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.553+0000 7f427968d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4274102d50 0x7f4274072bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:44.557 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.553+0000 7f427968d640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f427406d050 con 0x7f42741021a0 2024-08-30T09:15:44.557 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.553+0000 7f427968d640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f427406ced0 con 0x7f4274102d50 2024-08-30T09:15:44.557 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.553+0000 7f427968d640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f427406d1d0 con 0x7f4274101770 2024-08-30T09:15:44.560 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.557+0000 7f42737fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4274102d50 0x7f4274072bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:15:44.561 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.558+0000 7f42727fc640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f42741021a0 0x7f42740745d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:15:44.561 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.558+0000 7f42727fc640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f42741021a0 0x7f42740745d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:57130/0 (socket says 172.21.15.42:57130) 2024-08-30T09:15:44.561 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.558+0000 7f42727fc640 1 -- 172.21.15.42:0/1803037001 learned_addr learned my addr 172.21.15.42:0/1803037001 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:15:44.562 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.558+0000 7f42727fc640 1 -- 172.21.15.42:0/1803037001 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4274101770 msgr2=0x7f4274073ff0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-08-30T09:15:44.562 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.558+0000 7f42727fc640 1 --2- 172.21.15.42:0/1803037001 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4274101770 0x7f4274073ff0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:44.562 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.558+0000 7f42727fc640 1 -- 172.21.15.42:0/1803037001 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4274102d50 msgr2=0x7f4274072bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:15:44.562 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.558+0000 7f42727fc640 1 --2- 172.21.15.42:0/1803037001 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4274102d50 0x7f4274072bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:44.562 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.558+0000 7f42727fc640 1 -- 172.21.15.42:0/1803037001 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4274073120 con 0x7f42741021a0 2024-08-30T09:15:44.562 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.559+0000 7f42727fc640 1 --2- 172.21.15.42:0/1803037001 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f42741021a0 0x7f42740745d0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f426800c970 tx=0x7f426800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:15:44.565 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.559+0000 7f425bfff640 1 -- 172.21.15.42:0/1803037001 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4268007b50 con 0x7f42741021a0 2024-08-30T09:15:44.565 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.560+0000 7f427968d640 1 -- 172.21.15.42:0/1803037001 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4274073410 con 0x7f42741021a0 2024-08-30T09:15:44.565 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.560+0000 7f425bfff640 1 -- 172.21.15.42:0/1803037001 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4268007cf0 con 0x7f42741021a0 2024-08-30T09:15:44.565 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.560+0000 7f425bfff640 1 -- 172.21.15.42:0/1803037001 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4268004e40 con 0x7f42741021a0 2024-08-30T09:15:44.565 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.560+0000 7f427968d640 1 -- 172.21.15.42:0/1803037001 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42740739b0 con 0x7f42741021a0 2024-08-30T09:15:44.569 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.563+0000 7f42805fc740 1 Processor -- start 2024-08-30T09:15:44.570 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.563+0000 7f42805fc740 1 -- start start 2024-08-30T09:15:44.576 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.563+0000 7f42805fc740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555dccf5f1b0 0x555dcd304f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:44.576 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.563+0000 7f42805fc740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555dccf362f0 0x555dcd3055d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:44.576 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.563+0000 7f42805fc740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555dccf3e4d0 0x555dcd306120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:44.576 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.563+0000 7f42805fc740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x555dcd278000 con 0x555dccf362f0 2024-08-30T09:15:44.576 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.563+0000 7f42805fc740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x555dcd267590 con 0x555dccf3e4d0 2024-08-30T09:15:44.577 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.563+0000 7f42805fc740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x555dcd26b540 con 0x555dccf5f1b0 2024-08-30T09:15:44.577 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.567+0000 7f425bfff640 1 -- 172.21.15.42:0/1803037001 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7f4268005020 con 0x7f42741021a0 2024-08-30T09:15:44.577 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.568+0000 7f42737fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555dccf5f1b0 0x555dcd304f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:15:44.577 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.568+0000 7f42737fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555dccf5f1b0 0x555dcd304f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:33418/0 (socket says 172.21.15.42:33418) 2024-08-30T09:15:44.577 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.568+0000 7f42737fe640 1 -- 172.21.15.42:0/1233033498 learned_addr learned my addr 172.21.15.42:0/1233033498 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:15:44.577 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.570+0000 7f42727fc640 1 --2- 172.21.15.42:0/1233033498 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555dccf3e4d0 0x555dcd306120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:15:44.577 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.570+0000 7f4272ffd640 1 --2- 172.21.15.42:0/1233033498 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555dccf362f0 0x555dcd3055d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:15:44.577 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.571+0000 7f425bfff640 1 --2- 172.21.15.42:0/1803037001 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f425c0b4c60 0x7f425c0b7170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:44.577 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.571+0000 7f425bfff640 1 -- 172.21.15.42:0/1803037001 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f42680e51c0 con 0x7f42741021a0 2024-08-30T09:15:44.577 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.572+0000 7f42727fc640 1 -- 172.21.15.42:0/1233033498 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555dccf5f1b0 msgr2=0x555dcd304f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:15:44.577 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.572+0000 7f42727fc640 1 --2- 172.21.15.42:0/1233033498 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555dccf5f1b0 0x555dcd304f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:44.577 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.572+0000 7f42727fc640 1 -- 172.21.15.42:0/1233033498 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555dccf362f0 msgr2=0x555dcd3055d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:15:44.577 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.572+0000 7f42727fc640 1 --2- 172.21.15.42:0/1233033498 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555dccf362f0 0x555dcd3055d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:44.577 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.572+0000 7f42727fc640 1 -- 172.21.15.42:0/1233033498 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555dccfca790 con 0x555dccf3e4d0 2024-08-30T09:15:44.578 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.572+0000 7f42727fc640 1 --2- 172.21.15.42:0/1233033498 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555dccf3e4d0 0x555dcd306120 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f42680e2870 tx=0x7f4268000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=49687cd330155fe3 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:15:44.579 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.576+0000 7f42597fa640 1 -- 172.21.15.42:0/1233033498 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f426801d070 con 0x555dccf3e4d0 2024-08-30T09:15:44.580 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.576+0000 7f42597fa640 1 -- 172.21.15.42:0/1233033498 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f42680046c0 con 0x555dccf3e4d0 2024-08-30T09:15:44.580 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.576+0000 7f42597fa640 1 -- 172.21.15.42:0/1233033498 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42680b4b20 con 0x555dccf3e4d0 2024-08-30T09:15:44.580 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.577+0000 7f42805fc740 1 -- 172.21.15.42:0/1233033498 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555dccf3e4d0 msgr2=0x555dcd306120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:15:44.580 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.577+0000 7f42805fc740 1 --2- 172.21.15.42:0/1233033498 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555dccf3e4d0 0x555dcd306120 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f42680e2870 tx=0x7f4268000bc0 comp rx=0 tx=0).stop 2024-08-30T09:15:44.584 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.578+0000 7f42737fe640 1 --2- 172.21.15.42:0/1803037001 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f425c0b4c60 0x7f425c0b7170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:15:44.589 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.584+0000 7f42805fc740 1 -- 172.21.15.42:0/1233033498 shutdown_connections 2024-08-30T09:15:44.589 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.584+0000 7f42805fc740 1 --2- 172.21.15.42:0/1233033498 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555dccf3e4d0 0x555dcd306120 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:44.589 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.584+0000 7f42805fc740 1 --2- 172.21.15.42:0/1233033498 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555dccf362f0 0x555dcd3055d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:44.589 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.584+0000 7f42805fc740 1 --2- 172.21.15.42:0/1233033498 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555dccf5f1b0 0x555dcd304f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:44.589 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.584+0000 7f42805fc740 1 -- 172.21.15.42:0/1233033498 >> 172.21.15.42:0/1233033498 conn(0x555dccf53300 msgr2=0x555dcd3009f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:15:44.593 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.590+0000 7f42737fe640 1 --2- 172.21.15.42:0/1803037001 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f425c0b4c60 0x7f425c0b7170 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f426c031a40 tx=0x7f426c0095c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:15:44.594 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.591+0000 7f42805fc740 1 -- 172.21.15.42:0/1233033498 shutdown_connections 2024-08-30T09:15:44.600 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.596+0000 7f42805fc740 1 -- 172.21.15.42:0/1233033498 wait complete. 2024-08-30T09:15:44.600 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.596+0000 7f42805fc740 1 -- start start 2024-08-30T09:15:44.602 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.596+0000 7f42805fc740 1 Processor -- start 2024-08-30T09:15:44.602 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.597+0000 7f42805fc740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555dccf362f0 0x555dcd30e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:44.602 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.597+0000 7f42805fc740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555dccf3e4d0 0x555dcd30f070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:44.602 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.597+0000 7f42805fc740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555dccf5f1b0 0x555dcd30fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:44.602 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.597+0000 7f42805fc740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x555dcd1a0ab0 con 0x555dccf3e4d0 2024-08-30T09:15:44.602 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.597+0000 7f42805fc740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x555dcd1ee680 con 0x555dccf5f1b0 2024-08-30T09:15:44.602 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.597+0000 7f42805fc740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x555dcd306b80 con 0x555dccf362f0 2024-08-30T09:15:44.602 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.597+0000 7f42805fc740 10 client.0 _ll_register_callbacks cb 0x555dccf76a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:15:44.602 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.597+0000 7f4272ffd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555dccf362f0 0x555dcd30e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:15:44.602 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.597+0000 7f4272ffd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555dccf362f0 0x555dcd30e9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:33428/0 (socket says 172.21.15.42:33428) 2024-08-30T09:15:44.603 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.597+0000 7f4272ffd640 1 -- 172.21.15.42:0/2147326735 learned_addr learned my addr 172.21.15.42:0/2147326735 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:15:44.603 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.597+0000 7f42737fe640 1 --2- 172.21.15.42:0/2147326735 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555dccf5f1b0 0x555dcd30fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:15:44.603 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.598+0000 7f4272ffd640 1 -- 172.21.15.42:0/2147326735 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555dccf5f1b0 msgr2=0x555dcd30fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:15:44.603 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.598+0000 7f4272ffd640 1 --2- 172.21.15.42:0/2147326735 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555dccf5f1b0 0x555dcd30fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:44.603 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.598+0000 7f4272ffd640 1 -- 172.21.15.42:0/2147326735 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555dccf3e4d0 msgr2=0x555dcd30f070 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-30T09:15:44.603 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.598+0000 7f4272ffd640 1 --2- 172.21.15.42:0/2147326735 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555dccf3e4d0 0x555dcd30f070 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:44.603 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.598+0000 7f4272ffd640 1 -- 172.21.15.42:0/2147326735 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555dcd043620 con 0x555dccf362f0 2024-08-30T09:15:44.603 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.598+0000 7f4272ffd640 1 --2- 172.21.15.42:0/2147326735 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555dccf362f0 0x555dcd30e9b0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f426400d190 tx=0x7f426400d590 comp rx=0 tx=0).ready entity=mon.1 client_cookie=213779e5fd6a93db server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:15:44.605 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.602+0000 7f4256ff5640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-08-30T09:15:44.606 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.602+0000 7f4256ff5640 1 -- 172.21.15.42:0/2147326735 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4264014430 con 0x555dccf362f0 2024-08-30T09:15:44.606 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.603+0000 7f4256ff5640 1 -- 172.21.15.42:0/2147326735 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f426400f040 con 0x555dccf362f0 2024-08-30T09:15:44.608 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.604+0000 7f42805fc740 1 -- 172.21.15.42:0/2147326735 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x555dcd279f50 con 0x555dccf362f0 2024-08-30T09:15:44.608 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.605+0000 7f4256ff5640 1 -- 172.21.15.42:0/2147326735 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f42640145d0 con 0x555dccf362f0 2024-08-30T09:15:44.610 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.606+0000 7f42805fc740 10 client.27682 fetch_fsmap learned FSMap version 48 2024-08-30T09:15:44.610 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.606+0000 7f42805fc740 1 -- 172.21.15.42:0/2147326735 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x555dcd318c30 con 0x555dccf362f0 2024-08-30T09:15:44.616 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.613+0000 7f4256ff5640 1 -- 172.21.15.42:0/2147326735 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x555dcd279f50 con 0x555dccf362f0 2024-08-30T09:15:44.616 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.613+0000 7f4256ff5640 1 -- 172.21.15.42:0/2147326735 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 48) ==== 8366+0+0 (secure 0 0 0) 0x7f4264013070 con 0x555dccf362f0 2024-08-30T09:15:44.616 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.613+0000 7f42805fc740 10 client.27682 fetch_fsmap finished waiting for FSMap version 48 2024-08-30T09:15:44.616 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.613+0000 7f42805fc740 10 client.27682 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-30T09:15:44.616 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.613+0000 7f42805fc740 20 client.27682 populate_metadata read hostname 'smithi042' 2024-08-30T09:15:44.616 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.613+0000 7f42805fc740 1 --2- 172.21.15.42:0/2147326735 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x555dccf1dd80 0x555dcd31c940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:44.616 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.613+0000 7f42805fc740 4 client.27682 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-30T09:15:44.616 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.613+0000 7f42805fc740 1 -- 172.21.15.42:0/2147326735 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555dcd2759f0 con 0x555dccf1dd80 2024-08-30T09:15:44.625 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.618+0000 7f42727fc640 1 --2- 172.21.15.42:0/2147326735 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x555dccf1dd80 0x555dcd31c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:15:44.631 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.627+0000 7f42727fc640 1 --2- 172.21.15.42:0/2147326735 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x555dccf1dd80 0x555dcd31c940 crc :-1 s=READY pgs=303 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=582cacb6594cb14c server_cookie=494236c30f2aef1c in_seq=0 out_seq=0 2024-08-30T09:15:44.631 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.628+0000 7f4256ff5640 10 client.27682 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:15:44.632 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.629+0000 7f4256ff5640 1 -- 172.21.15.42:0/2147326735 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x555dcd2759f0 con 0x555dccf1dd80 2024-08-30T09:15:44.632 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.629+0000 7f4256ff5640 10 client.27682 handle_command_reply: tid=0 2024-08-30T09:15:44.632 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.629+0000 7f42805fc740 1 client.27682 shutdown 2024-08-30T09:15:44.634 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.632+0000 7f42805fc740 1 -- 172.21.15.42:0/2147326735 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555dccf362f0 msgr2=0x555dcd30e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:15:44.635 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.632+0000 7f42805fc740 1 --2- 172.21.15.42:0/2147326735 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555dccf362f0 0x555dcd30e9b0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f426400d190 tx=0x7f426400d590 comp rx=0 tx=0).stop 2024-08-30T09:15:44.640 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.637+0000 7f42805fc740 1 -- 172.21.15.42:0/2147326735 shutdown_connections 2024-08-30T09:15:44.640 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.637+0000 7f42805fc740 1 --2- 172.21.15.42:0/2147326735 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x555dccf1dd80 0x555dcd31c940 crc :-1 s=READY pgs=303 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:44.641 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.637+0000 7f42805fc740 1 --2- 172.21.15.42:0/2147326735 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555dccf5f1b0 0x555dcd30fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:44.641 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.637+0000 7f42805fc740 1 --2- 172.21.15.42:0/2147326735 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555dccf3e4d0 0x555dcd30f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:44.641 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.637+0000 7f42805fc740 1 --2- 172.21.15.42:0/2147326735 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555dccf362f0 0x555dcd30e9b0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:44.641 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.637+0000 7f42805fc740 1 -- 172.21.15.42:0/2147326735 >> 172.21.15.42:0/2147326735 conn(0x555dccf53300 msgr2=0x555dcd2fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:15:44.641 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.637+0000 7f4256ff5640 0 client.27682 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:15:44.645 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.643+0000 7f42805fc740 1 -- 172.21.15.42:0/2147326735 shutdown_connections 2024-08-30T09:15:44.646 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.644+0000 7f42805fc740 1 -- 172.21.15.42:0/2147326735 wait complete. 2024-08-30T09:15:44.647 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.645+0000 7f42805fc740 20 client.27682 trim_cache size 0 max 16384 2024-08-30T09:15:44.678 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.674+0000 7f42805fc740 1 Processor -- start 2024-08-30T09:15:44.678 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.674+0000 7f42805fc740 1 -- start start 2024-08-30T09:15:44.678 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.674+0000 7f42805fc740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555dccf362f0 0x555dcd2feb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:44.678 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.674+0000 7f42805fc740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555dccf3e4d0 0x555dcd2ff0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:44.678 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.674+0000 7f42805fc740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555dccf5f1b0 0x555dcd30cf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:44.678 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.675+0000 7f42805fc740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f42600125e0 con 0x555dccf362f0 2024-08-30T09:15:44.678 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.675+0000 7f42805fc740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f426000fe80 con 0x555dccf3e4d0 2024-08-30T09:15:44.678 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.675+0000 7f42805fc740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f426000dcd0 con 0x555dccf5f1b0 2024-08-30T09:15:44.678 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.675+0000 7f42727fc640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555dccf3e4d0 0x555dcd2ff0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:15:44.678 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.675+0000 7f42727fc640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555dccf3e4d0 0x555dcd2ff0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:60364/0 (socket says 172.21.15.42:60364) 2024-08-30T09:15:44.678 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.675+0000 7f42727fc640 1 -- 172.21.15.42:0/2412921004 learned_addr learned my addr 172.21.15.42:0/2412921004 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:15:44.678 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.675+0000 7f42737fe640 1 --2- 172.21.15.42:0/2412921004 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555dccf5f1b0 0x555dcd30cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:15:44.679 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.675+0000 7f4272ffd640 1 --2- 172.21.15.42:0/2412921004 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555dccf362f0 0x555dcd2feb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:15:44.680 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.675+0000 7f42727fc640 1 -- 172.21.15.42:0/2412921004 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555dccf5f1b0 msgr2=0x555dcd30cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:15:44.680 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.675+0000 7f42727fc640 1 --2- 172.21.15.42:0/2412921004 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555dccf5f1b0 0x555dcd30cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:44.680 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.675+0000 7f42727fc640 1 -- 172.21.15.42:0/2412921004 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555dccf362f0 msgr2=0x555dcd2feb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:15:44.680 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.675+0000 7f42727fc640 1 --2- 172.21.15.42:0/2412921004 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555dccf362f0 0x555dcd2feb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:44.681 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.675+0000 7f42727fc640 1 -- 172.21.15.42:0/2412921004 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555dcd30ff50 con 0x555dccf3e4d0 2024-08-30T09:15:44.681 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.675+0000 7f42727fc640 1 --2- 172.21.15.42:0/2412921004 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555dccf3e4d0 0x555dcd2ff0a0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f42680e54f0 tx=0x7f4268005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=864c266649aba0c8 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:15:44.681 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.676+0000 7f4258ff9640 1 -- 172.21.15.42:0/2412921004 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f426801d040 con 0x555dccf3e4d0 2024-08-30T09:15:44.681 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.676+0000 7f4258ff9640 1 -- 172.21.15.42:0/2412921004 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f42680abf00 con 0x555dccf3e4d0 2024-08-30T09:15:44.681 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.676+0000 7f4258ff9640 1 -- 172.21.15.42:0/2412921004 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42680b48f0 con 0x555dccf3e4d0 2024-08-30T09:15:44.681 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.677+0000 7f42805fc740 1 -- 172.21.15.42:0/2412921004 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555dccf3e4d0 msgr2=0x555dcd2ff0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:15:44.681 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.677+0000 7f42805fc740 1 --2- 172.21.15.42:0/2412921004 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555dccf3e4d0 0x555dcd2ff0a0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f42680e54f0 tx=0x7f4268005740 comp rx=0 tx=0).stop 2024-08-30T09:15:44.681 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.677+0000 7f42805fc740 1 -- 172.21.15.42:0/2412921004 shutdown_connections 2024-08-30T09:15:44.681 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.677+0000 7f42805fc740 1 --2- 172.21.15.42:0/2412921004 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555dccf5f1b0 0x555dcd30cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:44.681 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.677+0000 7f42805fc740 1 --2- 172.21.15.42:0/2412921004 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555dccf3e4d0 0x555dcd2ff0a0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:44.681 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.677+0000 7f42805fc740 1 --2- 172.21.15.42:0/2412921004 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555dccf362f0 0x555dcd2feb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:44.681 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.677+0000 7f42805fc740 1 -- 172.21.15.42:0/2412921004 >> 172.21.15.42:0/2412921004 conn(0x555dccf53300 msgr2=0x555dcd3099d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:15:44.681 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.677+0000 7f42805fc740 1 -- 172.21.15.42:0/2412921004 shutdown_connections 2024-08-30T09:15:44.682 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.677+0000 7f42805fc740 1 -- 172.21.15.42:0/2412921004 wait complete. 2024-08-30T09:15:44.682 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.677+0000 7f42805fc740 1 -- start start 2024-08-30T09:15:44.682 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.678+0000 7f42805fc740 1 Processor -- start 2024-08-30T09:15:44.684 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.678+0000 7f42805fc740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555dccf362f0 0x555dcd311480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:44.685 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.678+0000 7f42805fc740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555dccf3e4d0 0x555dcd311b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:44.685 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.678+0000 7f42805fc740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555dccf5f1b0 0x555dcd3126b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:44.685 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.678+0000 7f42805fc740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f426000e340 con 0x555dccf362f0 2024-08-30T09:15:44.685 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.678+0000 7f42727fc640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555dccf3e4d0 0x555dcd311b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:15:44.685 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.678+0000 7f42727fc640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555dccf3e4d0 0x555dcd311b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:60368/0 (socket says 172.21.15.42:60368) 2024-08-30T09:15:44.685 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.678+0000 7f42727fc640 1 -- 172.21.15.42:0/2988797084 learned_addr learned my addr 172.21.15.42:0/2988797084 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:15:44.685 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.678+0000 7f4272ffd640 1 --2- 172.21.15.42:0/2988797084 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555dccf362f0 0x555dcd311480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:15:44.685 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.678+0000 7f42805fc740 1 -- 172.21.15.42:0/2988797084 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f426000f7b0 con 0x555dccf3e4d0 2024-08-30T09:15:44.685 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.678+0000 7f42737fe640 1 --2- 172.21.15.42:0/2988797084 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555dccf5f1b0 0x555dcd3126b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:15:44.685 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.678+0000 7f42805fc740 1 -- 172.21.15.42:0/2988797084 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f426000ea10 con 0x555dccf5f1b0 2024-08-30T09:15:44.685 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.679+0000 7f42805fc740 10 client.0 _ll_register_callbacks cb 0x7f426400b120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:15:44.686 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.679+0000 7f42727fc640 1 -- 172.21.15.42:0/2988797084 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555dccf5f1b0 msgr2=0x555dcd3126b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:15:44.686 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.679+0000 7f42727fc640 1 --2- 172.21.15.42:0/2988797084 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555dccf5f1b0 0x555dcd3126b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:44.686 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.679+0000 7f42727fc640 1 -- 172.21.15.42:0/2988797084 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555dccf362f0 msgr2=0x555dcd311480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:15:44.686 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.679+0000 7f42727fc640 1 --2- 172.21.15.42:0/2988797084 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555dccf362f0 0x555dcd311480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:44.686 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.679+0000 7f42727fc640 1 -- 172.21.15.42:0/2988797084 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555dcd312e10 con 0x555dccf3e4d0 2024-08-30T09:15:44.686 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.679+0000 7f42727fc640 1 --2- 172.21.15.42:0/2988797084 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555dccf3e4d0 0x555dcd311b40 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f4268007590 tx=0x7f42680ab260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5572cab270e94d58 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:15:44.686 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.679+0000 7f4256ff5640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-30T09:15:44.686 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.679+0000 7f4256ff5640 1 -- 172.21.15.42:0/2988797084 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42680e8930 con 0x555dccf3e4d0 2024-08-30T09:15:44.686 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.680+0000 7f4256ff5640 1 -- 172.21.15.42:0/2988797084 <== mon.2 v2:172.21.15.167:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f42680b48a0 con 0x555dccf3e4d0 2024-08-30T09:15:44.686 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.680+0000 7f42805fc740 1 -- 172.21.15.42:0/2988797084 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f42680abf00 con 0x555dccf3e4d0 2024-08-30T09:15:44.686 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.680+0000 7f4256ff5640 1 -- 172.21.15.42:0/2988797084 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f42680b4b20 con 0x555dccf3e4d0 2024-08-30T09:15:44.686 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.680+0000 7f42805fc740 10 client.27752 fetch_fsmap learned FSMap version 48 2024-08-30T09:15:44.686 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.680+0000 7f42805fc740 1 -- 172.21.15.42:0/2988797084 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x555dcd3160c0 con 0x555dccf3e4d0 2024-08-30T09:15:44.686 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.680+0000 7f4256ff5640 1 -- 172.21.15.42:0/2988797084 <== mon.2 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f42680aaa60 con 0x555dccf3e4d0 2024-08-30T09:15:44.686 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.681+0000 7f4256ff5640 1 -- 172.21.15.42:0/2988797084 <== mon.2 v2:172.21.15.167:3300/0 5 ==== fsmap(e 48) ==== 8366+0+0 (secure 0 0 0) 0x7f42680aa4d0 con 0x555dccf3e4d0 2024-08-30T09:15:44.687 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.681+0000 7f42805fc740 10 client.27752 fetch_fsmap finished waiting for FSMap version 48 2024-08-30T09:15:44.687 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.681+0000 7f42805fc740 10 client.27752 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-30T09:15:44.687 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.681+0000 7f42805fc740 20 client.27752 populate_metadata read hostname 'smithi042' 2024-08-30T09:15:44.687 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.681+0000 7f42805fc740 1 --2- 172.21.15.42:0/2988797084 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x555dccef31c0 0x555dcd316570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:44.687 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.681+0000 7f42805fc740 4 client.27752 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-30T09:15:44.687 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.681+0000 7f42805fc740 1 -- 172.21.15.42:0/2988797084 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f42680abf00 con 0x555dccef31c0 2024-08-30T09:15:44.687 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.681+0000 7f4272ffd640 1 --2- 172.21.15.42:0/2988797084 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x555dccef31c0 0x555dcd316570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:15:44.687 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.682+0000 7f4256ff5640 10 client.27752 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:15:44.687 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.682+0000 7f4272ffd640 1 --2- 172.21.15.42:0/2988797084 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x555dccef31c0 0x555dcd316570 crc :-1 s=READY pgs=304 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=87bbb1ed48500c47 server_cookie=941a91a3e83768de in_seq=0 out_seq=0 2024-08-30T09:15:44.718 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.714+0000 7f4256ff5640 1 -- 172.21.15.42:0/2988797084 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f42680abf00 con 0x555dccef31c0 2024-08-30T09:15:44.718 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.715+0000 7f4256ff5640 10 client.27752 handle_command_reply: tid=0 2024-08-30T09:15:44.718 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.715+0000 7f42805fc740 1 client.27752 shutdown 2024-08-30T09:15:44.722 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.719+0000 7f42805fc740 1 -- 172.21.15.42:0/2988797084 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555dccf3e4d0 msgr2=0x555dcd311b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:15:44.722 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.719+0000 7f42805fc740 1 --2- 172.21.15.42:0/2988797084 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555dccf3e4d0 0x555dcd311b40 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f4268007590 tx=0x7f42680ab260 comp rx=0 tx=0).stop 2024-08-30T09:15:44.724 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.721+0000 7f42805fc740 1 -- 172.21.15.42:0/2988797084 shutdown_connections 2024-08-30T09:15:44.724 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.721+0000 7f42805fc740 1 --2- 172.21.15.42:0/2988797084 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x555dccef31c0 0x555dcd316570 crc :-1 s=READY pgs=304 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:44.724 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.721+0000 7f42805fc740 1 --2- 172.21.15.42:0/2988797084 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555dccf5f1b0 0x555dcd3126b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:44.724 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.721+0000 7f42805fc740 1 --2- 172.21.15.42:0/2988797084 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555dccf3e4d0 0x555dcd311b40 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:44.724 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.721+0000 7f42805fc740 1 --2- 172.21.15.42:0/2988797084 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555dccf362f0 0x555dcd311480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:44.724 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.721+0000 7f42805fc740 1 -- 172.21.15.42:0/2988797084 >> 172.21.15.42:0/2988797084 conn(0x555dccf53300 msgr2=0x555dcd2feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:15:44.727 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.721+0000 7f4256ff5640 0 client.27752 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:15:44.730 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.727+0000 7f42805fc740 1 -- 172.21.15.42:0/2988797084 shutdown_connections 2024-08-30T09:15:44.730 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.727+0000 7f42805fc740 1 -- 172.21.15.42:0/2988797084 wait complete. 2024-08-30T09:15:44.732 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.728+0000 7f42805fc740 20 client.27752 trim_cache size 0 max 16384 2024-08-30T09:15:44.735 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-08-30T09:15:44.735 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2024-08-30T09:15:44.735 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-08-30T09:15:44.736 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-08-30T09:15:44.737 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.734+0000 7f427968d640 1 -- 172.21.15.42:0/1803037001 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f425c0b4c60 msgr2=0x7f425c0b7170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:15:44.737 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.734+0000 7f427968d640 1 --2- 172.21.15.42:0/1803037001 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f425c0b4c60 0x7f425c0b7170 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f426c031a40 tx=0x7f426c0095c0 comp rx=0 tx=0).stop 2024-08-30T09:15:44.737 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.734+0000 7f427968d640 1 -- 172.21.15.42:0/1803037001 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f42741021a0 msgr2=0x7f42740745d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:15:44.738 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.734+0000 7f427968d640 1 --2- 172.21.15.42:0/1803037001 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f42741021a0 0x7f42740745d0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f426800c970 tx=0x7f426800ce30 comp rx=0 tx=0).stop 2024-08-30T09:15:44.742 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.739+0000 7f427968d640 1 -- 172.21.15.42:0/1803037001 shutdown_connections 2024-08-30T09:15:44.742 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.739+0000 7f427968d640 1 --2- 172.21.15.42:0/1803037001 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f425c0b4c60 0x7f425c0b7170 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:44.742 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.739+0000 7f427968d640 1 --2- 172.21.15.42:0/1803037001 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4274102d50 0x7f4274072bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:44.743 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.739+0000 7f427968d640 1 --2- 172.21.15.42:0/1803037001 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f42741021a0 0x7f42740745d0 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:44.743 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.739+0000 7f427968d640 1 --2- 172.21.15.42:0/1803037001 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4274101770 0x7f4274073ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:44.743 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.739+0000 7f427968d640 1 -- 172.21.15.42:0/1803037001 >> 172.21.15.42:0/1803037001 conn(0x7f42740fd430 msgr2=0x7f42741037e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:15:44.746 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.741+0000 7f427968d640 1 -- 172.21.15.42:0/1803037001 shutdown_connections 2024-08-30T09:15:44.747 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:44.743+0000 7f427968d640 1 -- 172.21.15.42:0/1803037001 wait complete. 2024-08-30T09:15:45.238 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:15:44 smithi190 ceph-mon[22001]: pgmap v822: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 5.7 MiB/s wr, 539 op/s 2024-08-30T09:15:45.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:15:44 smithi167 ceph-mon[22063]: pgmap v822: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 5.7 MiB/s wr, 539 op/s 2024-08-30T09:15:45.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:15:44 smithi042 ceph-mon[18652]: pgmap v822: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 5.7 MiB/s wr, 539 op/s 2024-08-30T09:15:45.953 INFO:teuthology.orchestra.run.smithi042.stderr:progress: 900.0 s, 473.2 tps, lat 216.777 ms stddev 254.132 2024-08-30T09:15:46.354 INFO:teuthology.orchestra.run.smithi042.stdout:transaction type: 2024-08-30T09:15:46.354 INFO:teuthology.orchestra.run.smithi042.stdout:scaling factor: 32 2024-08-30T09:15:46.355 INFO:teuthology.orchestra.run.smithi042.stdout:query mode: simple 2024-08-30T09:15:46.355 INFO:teuthology.orchestra.run.smithi042.stdout:number of clients: 100 2024-08-30T09:15:46.355 INFO:teuthology.orchestra.run.smithi042.stdout:number of threads: 4 2024-08-30T09:15:46.355 INFO:teuthology.orchestra.run.smithi042.stdout:duration: 900 s 2024-08-30T09:15:46.355 INFO:teuthology.orchestra.run.smithi042.stdout:number of transactions actually processed: 828566 2024-08-30T09:15:46.355 INFO:teuthology.orchestra.run.smithi042.stdout:latency average = 108.626 ms 2024-08-30T09:15:46.355 INFO:teuthology.orchestra.run.smithi042.stdout:latency stddev = 177.011 ms 2024-08-30T09:15:46.355 INFO:teuthology.orchestra.run.smithi042.stdout:tps = 920.219081 (including connections establishing) 2024-08-30T09:15:46.355 INFO:teuthology.orchestra.run.smithi042.stdout:tps = 920.221955 (excluding connections establishing) 2024-08-30T09:15:46.360 DEBUG:teuthology.orchestra.run.smithi042:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo systemctl stop postgresql' 2024-08-30T09:15:47.228 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:15:46 smithi042 ceph-mon[18652]: pgmap v823: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 5.8 MiB/s wr, 524 op/s 2024-08-30T09:15:47.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:15:46 smithi167 ceph-mon[22063]: pgmap v823: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 5.8 MiB/s wr, 524 op/s 2024-08-30T09:15:47.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:15:46 smithi190 ceph-mon[22001]: pgmap v823: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 5.8 MiB/s wr, 524 op/s 2024-08-30T09:15:49.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:15:48 smithi167 ceph-mon[22063]: pgmap v824: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 4.9 MiB/s wr, 446 op/s 2024-08-30T09:15:49.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:15:48 smithi042 ceph-mon[18652]: pgmap v824: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 4.9 MiB/s wr, 446 op/s 2024-08-30T09:15:49.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:15:48 smithi190 ceph-mon[22001]: pgmap v824: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 4.9 MiB/s wr, 446 op/s 2024-08-30T09:15:50.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:15:49 smithi167 ceph-mon[22063]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-30T09:15:50.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:15:49 smithi042 ceph-mon[18652]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-30T09:15:50.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:15:49 smithi190 ceph-mon[22001]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-30T09:15:51.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:15:50 smithi167 ceph-mon[22063]: pgmap v825: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 7.4 MiB/s wr, 642 op/s 2024-08-30T09:15:51.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:15:50 smithi042 ceph-mon[18652]: pgmap v825: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 7.4 MiB/s wr, 642 op/s 2024-08-30T09:15:51.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:15:50 smithi190 ceph-mon[22001]: pgmap v825: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 7.4 MiB/s wr, 642 op/s 2024-08-30T09:15:53.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:15:53 smithi167 ceph-mon[22063]: pgmap v826: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 6.3 MiB/s wr, 528 op/s 2024-08-30T09:15:53.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:15:52 smithi042 ceph-mon[18652]: pgmap v826: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 6.3 MiB/s wr, 528 op/s 2024-08-30T09:15:53.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:15:53 smithi190 ceph-mon[22001]: pgmap v826: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 6.3 MiB/s wr, 528 op/s 2024-08-30T09:15:55.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:15:55 smithi167 ceph-mon[22063]: pgmap v827: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 7.0 MiB/s wr, 609 op/s 2024-08-30T09:15:55.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:15:55 smithi042 ceph-mon[18652]: pgmap v827: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 7.0 MiB/s wr, 609 op/s 2024-08-30T09:15:55.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:15:55 smithi190 ceph-mon[22001]: pgmap v827: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 7.0 MiB/s wr, 609 op/s 2024-08-30T09:15:57.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:15:57 smithi167 ceph-mon[22063]: pgmap v828: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 6.8 MiB/s wr, 675 op/s 2024-08-30T09:15:57.347 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:15:57 smithi042 ceph-mon[18652]: pgmap v828: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 6.8 MiB/s wr, 675 op/s 2024-08-30T09:15:57.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:15:57 smithi190 ceph-mon[22001]: pgmap v828: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 6.8 MiB/s wr, 675 op/s 2024-08-30T09:15:57.947 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:57.942+0000 7fc971b8b640 1 -- 172.21.15.42:0/3712053226 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc96c10a160 msgr2=0x7fc96c10c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:15:57.947 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:57.942+0000 7fc971b8b640 1 --2- 172.21.15.42:0/3712053226 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc96c10a160 0x7fc96c10c580 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fc95c009900 tx=0x7fc95c02a440 comp rx=0 tx=0).stop 2024-08-30T09:15:57.954 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:57.951+0000 7fc971b8b640 1 -- 172.21.15.42:0/3712053226 shutdown_connections 2024-08-30T09:15:57.954 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:57.951+0000 7fc971b8b640 1 --2- 172.21.15.42:0/3712053226 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc96c10f770 0x7fc96c11a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:57.955 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:57.951+0000 7fc971b8b640 1 --2- 172.21.15.42:0/3712053226 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc96c10cb50 0x7fc96c10f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:57.955 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:57.951+0000 7fc971b8b640 1 --2- 172.21.15.42:0/3712053226 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc96c10a160 0x7fc96c10c580 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:57.955 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:57.951+0000 7fc971b8b640 1 -- 172.21.15.42:0/3712053226 >> 172.21.15.42:0/3712053226 conn(0x7fc96c0705b0 msgr2=0x7fc96c072a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:15:57.958 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:57.953+0000 7fc971b8b640 1 -- 172.21.15.42:0/3712053226 shutdown_connections 2024-08-30T09:15:57.958 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:57.955+0000 7fc971b8b640 1 -- 172.21.15.42:0/3712053226 wait complete. 2024-08-30T09:15:57.958 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:57.955+0000 7fc971b8b640 1 Processor -- start 2024-08-30T09:15:57.962 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:57.957+0000 7fc971b8b640 1 -- start start 2024-08-30T09:15:57.962 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:57.957+0000 7fc971b8b640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc96c10a160 0x7fc96c1a8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:57.962 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:57.957+0000 7fc971b8b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc96c10cb50 0x7fc96c1a9150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:57.963 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:57.957+0000 7fc971b8b640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc96c10f770 0x7fc96c1a9d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:57.963 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:57.957+0000 7fc971b8b640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fc96c11def0 con 0x7fc96c10f770 2024-08-30T09:15:57.963 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:57.957+0000 7fc971b8b640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc96c11dd70 con 0x7fc96c10a160 2024-08-30T09:15:57.963 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:57.957+0000 7fc971b8b640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fc96c11e070 con 0x7fc96c10cb50 2024-08-30T09:15:57.963 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:57.957+0000 7fc96bfff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc96c10f770 0x7fc96c1a9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:15:57.963 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:57.957+0000 7fc96bfff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc96c10f770 0x7fc96c1a9d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:32790/0 (socket says 172.21.15.42:32790) 2024-08-30T09:15:57.963 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:57.957+0000 7fc96bfff640 1 -- 172.21.15.42:0/1175973302 learned_addr learned my addr 172.21.15.42:0/1175973302 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:15:57.963 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:57.958+0000 7fc96affd640 1 --2- 172.21.15.42:0/1175973302 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc96c10cb50 0x7fc96c1a9150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:15:57.963 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:57.958+0000 7fc96bfff640 1 -- 172.21.15.42:0/1175973302 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc96c10cb50 msgr2=0x7fc96c1a9150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:15:57.963 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:57.958+0000 7fc96bfff640 1 --2- 172.21.15.42:0/1175973302 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc96c10cb50 0x7fc96c1a9150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:57.963 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:57.958+0000 7fc96bfff640 1 -- 172.21.15.42:0/1175973302 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc96c10a160 msgr2=0x7fc96c1a8ae0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-08-30T09:15:57.963 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:57.958+0000 7fc96bfff640 1 --2- 172.21.15.42:0/1175973302 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc96c10a160 0x7fc96c1a8ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:57.964 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:57.958+0000 7fc96bfff640 1 -- 172.21.15.42:0/1175973302 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc96c1aa520 con 0x7fc96c10f770 2024-08-30T09:15:57.964 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:57.958+0000 7fc96bfff640 1 --2- 172.21.15.42:0/1175973302 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc96c10f770 0x7fc96c1a9d00 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fc95400e9b0 tx=0x7fc95400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:15:57.964 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:57.960+0000 7fc968ff9640 1 -- 172.21.15.42:0/1175973302 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc95400cd80 con 0x7fc96c10f770 2024-08-30T09:15:57.964 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:57.960+0000 7fc968ff9640 1 -- 172.21.15.42:0/1175973302 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc9540035f0 con 0x7fc96c10f770 2024-08-30T09:15:57.964 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:57.960+0000 7fc968ff9640 1 -- 172.21.15.42:0/1175973302 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc954010690 con 0x7fc96c10f770 2024-08-30T09:15:57.966 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:57.964+0000 7fc971b8b640 1 -- 172.21.15.42:0/1175973302 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc96c1aa810 con 0x7fc96c10f770 2024-08-30T09:15:57.967 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:57.964+0000 7fc971b8b640 1 -- 172.21.15.42:0/1175973302 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc96c112120 con 0x7fc96c10f770 2024-08-30T09:15:57.974 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:57.971+0000 7fc978bd6740 1 Processor -- start 2024-08-30T09:15:57.976 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:57.973+0000 7fc968ff9640 1 -- 172.21.15.42:0/1175973302 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7fc954003a80 con 0x7fc96c10f770 2024-08-30T09:15:57.976 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:57.973+0000 7fc978bd6740 1 -- start start 2024-08-30T09:15:57.976 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:57.973+0000 7fc978bd6740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55aa342401b0 0x55aa345e6180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:57.976 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:57.973+0000 7fc978bd6740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55aa342172f0 0x55aa345e67f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:57.976 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:57.973+0000 7fc978bd6740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55aa3421f4d0 0x55aa345e7340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:57.976 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:57.973+0000 7fc978bd6740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55aa34559000 con 0x55aa342172f0 2024-08-30T09:15:57.976 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:57.973+0000 7fc978bd6740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55aa34548590 con 0x55aa342401b0 2024-08-30T09:15:57.976 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:57.973+0000 7fc978bd6740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55aa3454c540 con 0x55aa3421f4d0 2024-08-30T09:15:57.979 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:57.976+0000 7fc96affd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55aa342401b0 0x55aa345e6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:15:57.979 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:57.976+0000 7fc96affd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55aa342401b0 0x55aa345e6180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:37210/0 (socket says 172.21.15.42:37210) 2024-08-30T09:15:57.979 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:57.976+0000 7fc96affd640 1 -- 172.21.15.42:0/1760909951 learned_addr learned my addr 172.21.15.42:0/1760909951 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:15:57.979 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:57.976+0000 7fc96affd640 1 --2- 172.21.15.42:0/1760909951 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55aa3421f4d0 0x55aa345e7340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:15:57.980 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:57.978+0000 7fc96b7fe640 1 --2- 172.21.15.42:0/1760909951 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55aa342172f0 0x55aa345e67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:15:57.981 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:57.978+0000 7fc968ff9640 1 --2- 172.21.15.42:0/1175973302 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fc9580b4d40 0x7fc9580b7250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:57.982 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:57.978+0000 7fc96affd640 1 -- 172.21.15.42:0/1760909951 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55aa3421f4d0 msgr2=0x55aa345e7340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:15:57.982 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:57.978+0000 7fc96affd640 1 --2- 172.21.15.42:0/1760909951 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55aa3421f4d0 0x55aa345e7340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:57.982 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:57.978+0000 7fc96affd640 1 -- 172.21.15.42:0/1760909951 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55aa342172f0 msgr2=0x55aa345e67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:15:57.982 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:57.978+0000 7fc96affd640 1 --2- 172.21.15.42:0/1760909951 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55aa342172f0 0x55aa345e67f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:57.982 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:57.978+0000 7fc96affd640 1 -- 172.21.15.42:0/1760909951 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aa342ab790 con 0x55aa342401b0 2024-08-30T09:15:57.982 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:57.979+0000 7fc96bfff640 1 --2- 172.21.15.42:0/1175973302 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fc9580b4d40 0x7fc9580b7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:15:57.984 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:57.982+0000 7fc968ff9640 1 -- 172.21.15.42:0/1175973302 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7fc9540e5880 con 0x7fc96c10f770 2024-08-30T09:15:57.987 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:57.983+0000 7fc96affd640 1 --2- 172.21.15.42:0/1760909951 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55aa342401b0 0x55aa345e6180 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fc964009270 tx=0x7fc964009ac0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=74212148859d77d2 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:15:57.991 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:57.989+0000 7fc961ffb640 1 -- 172.21.15.42:0/1760909951 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc964013d00 con 0x55aa342401b0 2024-08-30T09:15:57.992 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:57.989+0000 7fc961ffb640 1 -- 172.21.15.42:0/1760909951 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc964009e10 con 0x55aa342401b0 2024-08-30T09:15:57.992 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:57.989+0000 7fc961ffb640 1 -- 172.21.15.42:0/1760909951 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc964011b80 con 0x55aa342401b0 2024-08-30T09:15:57.997 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:57.991+0000 7fc978bd6740 1 -- 172.21.15.42:0/1760909951 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55aa342401b0 msgr2=0x55aa345e6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:15:57.997 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:57.991+0000 7fc978bd6740 1 --2- 172.21.15.42:0/1760909951 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55aa342401b0 0x55aa345e6180 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fc964009270 tx=0x7fc964009ac0 comp rx=0 tx=0).stop 2024-08-30T09:15:58.001 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:57.998+0000 7fc978bd6740 1 -- 172.21.15.42:0/1760909951 shutdown_connections 2024-08-30T09:15:58.001 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:57.998+0000 7fc978bd6740 1 --2- 172.21.15.42:0/1760909951 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55aa3421f4d0 0x55aa345e7340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:58.002 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:57.998+0000 7fc978bd6740 1 --2- 172.21.15.42:0/1760909951 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55aa342172f0 0x55aa345e67f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:58.002 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:57.998+0000 7fc978bd6740 1 --2- 172.21.15.42:0/1760909951 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55aa342401b0 0x55aa345e6180 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:58.002 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:57.998+0000 7fc978bd6740 1 -- 172.21.15.42:0/1760909951 >> 172.21.15.42:0/1760909951 conn(0x55aa34234300 msgr2=0x55aa345e1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:15:58.006 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:58.003+0000 7fc96bfff640 1 --2- 172.21.15.42:0/1175973302 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fc9580b4d40 0x7fc9580b7250 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fc95400d010 tx=0x7fc954003e50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:15:58.011 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:58.008+0000 7fc978bd6740 1 -- 172.21.15.42:0/1760909951 shutdown_connections 2024-08-30T09:15:58.014 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:58.011+0000 7fc978bd6740 1 -- 172.21.15.42:0/1760909951 wait complete. 2024-08-30T09:15:58.015 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:58.012+0000 7fc978bd6740 1 -- start start 2024-08-30T09:15:58.016 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:58.012+0000 7fc978bd6740 1 Processor -- start 2024-08-30T09:15:58.018 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:58.015+0000 7fc978bd6740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55aa342172f0 0x55aa345efc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:58.018 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:58.015+0000 7fc978bd6740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55aa3421f4d0 0x55aa345f0300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:58.018 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:58.015+0000 7fc978bd6740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55aa342401b0 0x55aa345f0e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:58.019 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:58.015+0000 7fc978bd6740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55aa34481ab0 con 0x55aa3421f4d0 2024-08-30T09:15:58.019 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:58.015+0000 7fc978bd6740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55aa344cf680 con 0x55aa342172f0 2024-08-30T09:15:58.019 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:58.015+0000 7fc978bd6740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55aa345e7e00 con 0x55aa342401b0 2024-08-30T09:15:58.019 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:58.015+0000 7fc978bd6740 10 client.0 _ll_register_callbacks cb 0x55aa34257a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:15:58.019 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:58.016+0000 7fc96b7fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55aa3421f4d0 0x55aa345f0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:15:58.019 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:58.016+0000 7fc96b7fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55aa3421f4d0 0x55aa345f0300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:32816/0 (socket says 172.21.15.42:32816) 2024-08-30T09:15:58.019 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:58.016+0000 7fc96b7fe640 1 -- 172.21.15.42:0/2943307868 learned_addr learned my addr 172.21.15.42:0/2943307868 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:15:58.022 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:58.020+0000 7fc96b7fe640 1 -- 172.21.15.42:0/2943307868 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55aa342401b0 msgr2=0x55aa345f0e70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-30T09:15:58.022 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:58.020+0000 7fc96b7fe640 1 --2- 172.21.15.42:0/2943307868 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55aa342401b0 0x55aa345f0e70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:58.022 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:58.020+0000 7fc96b7fe640 1 -- 172.21.15.42:0/2943307868 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55aa342172f0 msgr2=0x55aa345efc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:15:58.024 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:58.020+0000 7fc96affd640 1 --2- 172.21.15.42:0/2943307868 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55aa342172f0 0x55aa345efc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:15:58.025 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:58.022+0000 7fc96b7fe640 1 --2- 172.21.15.42:0/2943307868 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55aa342172f0 0x55aa345efc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:58.025 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:58.022+0000 7fc96b7fe640 1 -- 172.21.15.42:0/2943307868 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aa34324620 con 0x55aa3421f4d0 2024-08-30T09:15:58.025 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:58.022+0000 7fc96affd640 1 --2- 172.21.15.42:0/2943307868 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55aa342172f0 0x55aa345efc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-30T09:15:58.030 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:58.027+0000 7fc96b7fe640 1 --2- 172.21.15.42:0/2943307868 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55aa3421f4d0 0x55aa345f0300 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fc95c009230 tx=0x7fc95c005270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdf55807f4fb60f2 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:15:58.030 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:58.028+0000 7fc94affd640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-08-30T09:15:58.037 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:58.034+0000 7fc94affd640 1 -- 172.21.15.42:0/2943307868 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc95c040070 con 0x55aa3421f4d0 2024-08-30T09:15:58.037 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:58.034+0000 7fc978bd6740 1 -- 172.21.15.42:0/2943307868 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55aa3455af50 con 0x55aa3421f4d0 2024-08-30T09:15:58.037 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:58.034+0000 7fc94affd640 1 -- 172.21.15.42:0/2943307868 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7fc95c0048c0 con 0x55aa3421f4d0 2024-08-30T09:15:58.043 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:58.040+0000 7fc94affd640 1 -- 172.21.15.42:0/2943307868 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7fc95c04a050 con 0x55aa3421f4d0 2024-08-30T09:15:58.043 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:58.040+0000 7fc978bd6740 10 client.17916 fetch_fsmap learned FSMap version 49 2024-08-30T09:15:58.043 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:58.040+0000 7fc978bd6740 1 -- 172.21.15.42:0/2943307868 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x55aa345f9e30 con 0x55aa3421f4d0 2024-08-30T09:15:58.051 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:58.048+0000 7fc94affd640 1 -- 172.21.15.42:0/2943307868 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55aa3455af50 con 0x55aa3421f4d0 2024-08-30T09:15:58.051 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:58.048+0000 7fc94affd640 1 -- 172.21.15.42:0/2943307868 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 49) ==== 8362+0+0 (secure 0 0 0) 0x7fc95c03a040 con 0x55aa3421f4d0 2024-08-30T09:15:58.051 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:58.048+0000 7fc978bd6740 10 client.17916 fetch_fsmap finished waiting for FSMap version 49 2024-08-30T09:15:58.051 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:58.048+0000 7fc978bd6740 10 client.17916 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-30T09:15:58.051 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:58.048+0000 7fc978bd6740 20 client.17916 populate_metadata read hostname 'smithi042' 2024-08-30T09:15:58.051 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:58.048+0000 7fc978bd6740 1 --2- 172.21.15.42:0/2943307868 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55aa341fed80 0x55aa345fdad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:58.051 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:58.048+0000 7fc978bd6740 4 client.17916 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-30T09:15:58.052 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:58.048+0000 7fc978bd6740 1 -- 172.21.15.42:0/2943307868 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55aa345569f0 con 0x55aa341fed80 2024-08-30T09:15:58.057 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:58.055+0000 7fc96affd640 1 --2- 172.21.15.42:0/2943307868 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55aa341fed80 0x55aa345fdad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:15:58.066 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:58.063+0000 7fc96affd640 1 --2- 172.21.15.42:0/2943307868 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55aa341fed80 0x55aa345fdad0 crc :-1 s=READY pgs=305 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f6d019b3555217b1 server_cookie=ff72fb712deb9b79 in_seq=0 out_seq=0 2024-08-30T09:15:58.066 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:58.064+0000 7fc94affd640 10 client.17916 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:15:58.071 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:58.068+0000 7fc94affd640 1 -- 172.21.15.42:0/2943307868 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55aa345569f0 con 0x55aa341fed80 2024-08-30T09:15:58.071 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:58.068+0000 7fc94affd640 10 client.17916 handle_command_reply: tid=0 2024-08-30T09:15:58.071 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:58.068+0000 7fc978bd6740 1 client.17916 shutdown 2024-08-30T09:15:58.082 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:58.079+0000 7fc978bd6740 1 -- 172.21.15.42:0/2943307868 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55aa3421f4d0 msgr2=0x55aa345f0300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:15:58.083 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:58.079+0000 7fc978bd6740 1 --2- 172.21.15.42:0/2943307868 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55aa3421f4d0 0x55aa345f0300 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fc95c009230 tx=0x7fc95c005270 comp rx=0 tx=0).stop 2024-08-30T09:15:58.086 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:58.083+0000 7fc978bd6740 1 -- 172.21.15.42:0/2943307868 shutdown_connections 2024-08-30T09:15:58.086 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:58.083+0000 7fc978bd6740 1 --2- 172.21.15.42:0/2943307868 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55aa341fed80 0x55aa345fdad0 crc :-1 s=READY pgs=305 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:58.086 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:58.083+0000 7fc978bd6740 1 --2- 172.21.15.42:0/2943307868 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55aa342401b0 0x55aa345f0e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:58.086 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:58.083+0000 7fc978bd6740 1 --2- 172.21.15.42:0/2943307868 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55aa3421f4d0 0x55aa345f0300 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:58.086 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:58.083+0000 7fc978bd6740 1 --2- 172.21.15.42:0/2943307868 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55aa342172f0 0x55aa345efc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:58.086 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:58.083+0000 7fc978bd6740 1 -- 172.21.15.42:0/2943307868 >> 172.21.15.42:0/2943307868 conn(0x55aa34234300 msgr2=0x55aa345e0170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:15:58.086 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:58.083+0000 7fc94affd640 0 client.17916 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:15:58.096 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:58.093+0000 7fc978bd6740 1 -- 172.21.15.42:0/2943307868 shutdown_connections 2024-08-30T09:15:58.096 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:58.094+0000 7fc978bd6740 1 -- 172.21.15.42:0/2943307868 wait complete. 2024-08-30T09:15:58.097 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:58.094+0000 7fc978bd6740 20 client.17916 trim_cache size 0 max 16384 2024-08-30T09:15:58.128 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:58.125+0000 7fc978bd6740 1 Processor -- start 2024-08-30T09:15:58.128 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:58.125+0000 7fc978bd6740 1 -- start start 2024-08-30T09:15:58.131 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:58.127+0000 7fc978bd6740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55aa342172f0 0x55aa345dc400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:58.131 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:58.128+0000 7fc978bd6740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55aa3421f4d0 0x55aa345eee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:58.131 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:58.128+0000 7fc978bd6740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55aa342401b0 0x55aa345eabf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:58.131 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:58.128+0000 7fc96affd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55aa342172f0 0x55aa345dc400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:15:58.131 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:58.128+0000 7fc96affd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55aa342172f0 0x55aa345dc400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:37226/0 (socket says 172.21.15.42:37226) 2024-08-30T09:15:58.131 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:58.128+0000 7fc96b7fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55aa3421f4d0 0x55aa345eee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:15:58.131 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:58.128+0000 7fc96b7fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55aa3421f4d0 0x55aa345eee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:38762/0 (socket says 172.21.15.42:38762) 2024-08-30T09:15:58.132 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:58.128+0000 7fc96affd640 1 -- 172.21.15.42:0/3697591199 learned_addr learned my addr 172.21.15.42:0/3697591199 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:15:58.132 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:58.128+0000 7fc978bd6740 1 -- 172.21.15.42:0/3697591199 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fc96c1c6390 con 0x55aa342401b0 2024-08-30T09:15:58.132 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:58.128+0000 7fc978bd6740 1 -- 172.21.15.42:0/3697591199 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc96c1c3c60 con 0x55aa342172f0 2024-08-30T09:15:58.132 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:58.128+0000 7fc978bd6740 1 -- 172.21.15.42:0/3697591199 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fc96c11a8b0 con 0x55aa3421f4d0 2024-08-30T09:15:58.135 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:58.132+0000 7fc96affd640 1 --2- 172.21.15.42:0/3697591199 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55aa342401b0 0x55aa345eabf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:15:58.135 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:58.132+0000 7fc96affd640 1 -- 172.21.15.42:0/3697591199 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55aa3421f4d0 msgr2=0x55aa345eee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:15:58.135 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:58.132+0000 7fc96affd640 1 --2- 172.21.15.42:0/3697591199 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55aa3421f4d0 0x55aa345eee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:58.136 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:58.132+0000 7fc96affd640 1 -- 172.21.15.42:0/3697591199 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55aa342401b0 msgr2=0x55aa345eabf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:15:58.136 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:58.132+0000 7fc96affd640 1 --2- 172.21.15.42:0/3697591199 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55aa342401b0 0x55aa345eabf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:58.136 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:58.132+0000 7fc96affd640 1 -- 172.21.15.42:0/3697591199 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aa345ede50 con 0x55aa342172f0 2024-08-30T09:15:58.138 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:58.136+0000 7fc96affd640 1 --2- 172.21.15.42:0/3697591199 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55aa342172f0 0x55aa345dc400 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fc964000f80 tx=0x7fc964005bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d4dc5fbf4cb616b0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:15:58.141 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:58.137+0000 7fc9617fa640 1 -- 172.21.15.42:0/3697591199 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc964009b30 con 0x55aa342172f0 2024-08-30T09:15:58.141 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:58.137+0000 7fc9617fa640 1 -- 172.21.15.42:0/3697591199 <== mon.2 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc964003e20 con 0x55aa342172f0 2024-08-30T09:15:58.141 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:58.137+0000 7fc9617fa640 1 -- 172.21.15.42:0/3697591199 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc964005450 con 0x55aa342172f0 2024-08-30T09:15:58.144 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:58.140+0000 7fc978bd6740 1 -- 172.21.15.42:0/3697591199 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55aa342172f0 msgr2=0x55aa345dc400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:15:58.144 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:58.140+0000 7fc978bd6740 1 --2- 172.21.15.42:0/3697591199 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55aa342172f0 0x55aa345dc400 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fc964000f80 tx=0x7fc964005bd0 comp rx=0 tx=0).stop 2024-08-30T09:15:58.148 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:58.145+0000 7fc978bd6740 1 -- 172.21.15.42:0/3697591199 shutdown_connections 2024-08-30T09:15:58.148 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:58.145+0000 7fc978bd6740 1 --2- 172.21.15.42:0/3697591199 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55aa3421f4d0 0x55aa345eee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:58.148 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:58.145+0000 7fc978bd6740 1 --2- 172.21.15.42:0/3697591199 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55aa342401b0 0x55aa345eabf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:58.148 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:58.145+0000 7fc978bd6740 1 --2- 172.21.15.42:0/3697591199 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55aa342172f0 0x55aa345dc400 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:58.148 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:58.145+0000 7fc978bd6740 1 -- 172.21.15.42:0/3697591199 >> 172.21.15.42:0/3697591199 conn(0x55aa34234300 msgr2=0x55aa345e5e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:15:58.152 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:58.149+0000 7fc978bd6740 1 -- 172.21.15.42:0/3697591199 shutdown_connections 2024-08-30T09:15:58.152 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:58.149+0000 7fc978bd6740 1 -- 172.21.15.42:0/3697591199 wait complete. 2024-08-30T09:15:58.152 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:58.149+0000 7fc978bd6740 1 -- start start 2024-08-30T09:15:58.153 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:58.150+0000 7fc978bd6740 1 Processor -- start 2024-08-30T09:15:58.153 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:58.150+0000 7fc978bd6740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55aa342172f0 0x55aa345f4b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:58.154 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:58.150+0000 7fc978bd6740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55aa3421f4d0 0x55aa345f51e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:58.154 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:58.150+0000 7fc978bd6740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55aa342401b0 0x55aa345f5d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:58.154 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:58.150+0000 7fc96affd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55aa342172f0 0x55aa345f4b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:15:58.154 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:58.150+0000 7fc96b7fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55aa3421f4d0 0x55aa345f51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:15:58.154 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:58.150+0000 7fc96affd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55aa342172f0 0x55aa345f4b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:37230/0 (socket says 172.21.15.42:37230) 2024-08-30T09:15:58.154 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:58.150+0000 7fc96b7fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55aa3421f4d0 0x55aa345f51e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:38770/0 (socket says 172.21.15.42:38770) 2024-08-30T09:15:58.154 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:58.150+0000 7fc96affd640 1 -- 172.21.15.42:0/2670380742 learned_addr learned my addr 172.21.15.42:0/2670380742 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:15:58.154 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:58.150+0000 7fc978bd6740 1 -- 172.21.15.42:0/2670380742 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fc96c1c2120 con 0x55aa342401b0 2024-08-30T09:15:58.154 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:58.150+0000 7fc978bd6740 1 -- 172.21.15.42:0/2670380742 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc96c1c3590 con 0x55aa342172f0 2024-08-30T09:15:58.154 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:58.150+0000 7fc978bd6740 1 -- 172.21.15.42:0/2670380742 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fc96c1c27f0 con 0x55aa3421f4d0 2024-08-30T09:15:58.154 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:58.150+0000 7fc978bd6740 10 client.0 _ll_register_callbacks cb 0x7fc954005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:15:58.156 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:58.152+0000 7fc96affd640 1 --2- 172.21.15.42:0/2670380742 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55aa342401b0 0x55aa345f5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:15:58.156 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:58.152+0000 7fc96affd640 1 -- 172.21.15.42:0/2670380742 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55aa3421f4d0 msgr2=0x55aa345f51e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:15:58.157 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:58.152+0000 7fc96affd640 1 --2- 172.21.15.42:0/2670380742 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55aa3421f4d0 0x55aa345f51e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:58.157 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:58.152+0000 7fc96affd640 1 -- 172.21.15.42:0/2670380742 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55aa342401b0 msgr2=0x55aa345f5d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:15:58.157 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:58.152+0000 7fc96affd640 1 --2- 172.21.15.42:0/2670380742 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55aa342401b0 0x55aa345f5d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:15:58.157 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:58.152+0000 7fc96affd640 1 -- 172.21.15.42:0/2670380742 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aa345f6460 con 0x55aa342172f0 2024-08-30T09:15:58.157 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:58.152+0000 7fc96b7fe640 1 --2- 172.21.15.42:0/2670380742 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55aa3421f4d0 0x55aa345f51e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello state changed while learned_addr, mark_down or replacing must be happened just now 2024-08-30T09:15:58.158 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:58.154+0000 7fc96affd640 1 --2- 172.21.15.42:0/2670380742 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55aa342172f0 0x55aa345f4b20 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fc964009990 tx=0x7fc9640098e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=60eec19cb218b869 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:15:58.159 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:58.154+0000 7fc94affd640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-30T09:15:58.162 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:58.157+0000 7fc94affd640 1 -- 172.21.15.42:0/2670380742 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc964003b70 con 0x55aa342172f0 2024-08-30T09:15:58.162 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:58.157+0000 7fc978bd6740 1 -- 172.21.15.42:0/2670380742 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc964003e20 con 0x55aa342172f0 2024-08-30T09:15:58.162 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:58.157+0000 7fc94affd640 1 -- 172.21.15.42:0/2670380742 <== mon.2 v2:172.21.15.167:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7fc9640051d0 con 0x55aa342172f0 2024-08-30T09:15:58.165 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:58.160+0000 7fc94affd640 1 -- 172.21.15.42:0/2670380742 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7fc964039050 con 0x55aa342172f0 2024-08-30T09:15:58.165 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:58.161+0000 7fc978bd6740 10 client.27764 fetch_fsmap learned FSMap version 49 2024-08-30T09:15:58.165 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:58.161+0000 7fc978bd6740 1 -- 172.21.15.42:0/2670380742 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x55aa345f7780 con 0x55aa342172f0 2024-08-30T09:15:58.169 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:58.164+0000 7fc94affd640 1 -- 172.21.15.42:0/2670380742 <== mon.2 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc964003e20 con 0x55aa342172f0 2024-08-30T09:15:58.169 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:58.164+0000 7fc94affd640 1 -- 172.21.15.42:0/2670380742 <== mon.2 v2:172.21.15.167:3300/0 5 ==== fsmap(e 49) ==== 8362+0+0 (secure 0 0 0) 0x7fc9640117c0 con 0x55aa342172f0 2024-08-30T09:15:58.169 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:58.164+0000 7fc978bd6740 10 client.27764 fetch_fsmap finished waiting for FSMap version 49 2024-08-30T09:15:58.170 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:58.164+0000 7fc978bd6740 10 client.27764 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-30T09:15:58.170 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:58.165+0000 7fc978bd6740 20 client.27764 populate_metadata read hostname 'smithi042' 2024-08-30T09:15:58.170 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:58.165+0000 7fc978bd6740 1 --2- 172.21.15.42:0/2670380742 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55aa341d41c0 0x55aa3466a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:15:58.170 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:58.165+0000 7fc978bd6740 4 client.27764 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-30T09:15:58.170 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:58.165+0000 7fc978bd6740 1 -- 172.21.15.42:0/2670380742 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55aa34554130 con 0x55aa341d41c0 2024-08-30T09:15:58.170 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:58.166+0000 7fc96b7fe640 1 --2- 172.21.15.42:0/2670380742 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55aa341d41c0 0x55aa3466a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:15:58.181 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:58.177+0000 7fc96b7fe640 1 --2- 172.21.15.42:0/2670380742 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55aa341d41c0 0x55aa3466a5f0 crc :-1 s=READY pgs=306 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3009acb4d664a40 server_cookie=6c80e7cd53e2e172 in_seq=0 out_seq=0 2024-08-30T09:15:58.185 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:15:58.179+0000 7fc94affd640 10 client.27764 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:15:58.884 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-30T09:15:59.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:15:59 smithi167 ceph-mon[22063]: pgmap v829: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 6.0 MiB/s wr, 595 op/s 2024-08-30T09:15:59.391 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:15:59 smithi042 ceph-mon[18652]: pgmap v829: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 6.0 MiB/s wr, 595 op/s 2024-08-30T09:15:59.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:15:59 smithi190 ceph-mon[22001]: pgmap v829: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 6.0 MiB/s wr, 595 op/s 2024-08-30T09:15:59.554 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T09:16:00.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.216+0000 7f0daa6be640 1 -- 172.21.15.42:0/3311196797 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0da410f870 msgr2=0x7f0da410fc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:16:00.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.216+0000 7f0daa6be640 1 --2- 172.21.15.42:0/3311196797 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0da410f870 0x7f0da410fc80 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f0d94009860 tx=0x7f0d9402a340 comp rx=0 tx=0).stop 2024-08-30T09:16:00.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.216+0000 7f0daa6be640 1 -- 172.21.15.42:0/3311196797 shutdown_connections 2024-08-30T09:16:00.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.216+0000 7f0daa6be640 1 --2- 172.21.15.42:0/3311196797 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0da410f870 0x7f0da410fc80 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:00.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.216+0000 7f0daa6be640 1 --2- 172.21.15.42:0/3311196797 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0da41101c0 0x7f0da41169f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:00.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.216+0000 7f0daa6be640 1 --2- 172.21.15.42:0/3311196797 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0da4109f30 0x7f0da410a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:00.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.216+0000 7f0daa6be640 1 -- 172.21.15.42:0/3311196797 >> 172.21.15.42:0/3311196797 conn(0x7f0da4077f10 msgr2=0x7f0da4107f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:16:00.221 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.216+0000 7f0daa6be640 1 -- 172.21.15.42:0/3311196797 shutdown_connections 2024-08-30T09:16:00.221 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.216+0000 7f0daa6be640 1 -- 172.21.15.42:0/3311196797 wait complete. 2024-08-30T09:16:00.221 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.217+0000 7f0daa6be640 1 Processor -- start 2024-08-30T09:16:00.221 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.217+0000 7f0daa6be640 1 -- start start 2024-08-30T09:16:00.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.217+0000 7f0daa6be640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0da4109f30 0x7f0da41a0870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:16:00.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.218+0000 7f0daa6be640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0da410f870 0x7f0da41a0e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:16:00.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.218+0000 7f0daa6be640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0da41101c0 0x7f0da41a60b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:16:00.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.218+0000 7f0daa6be640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f0da4119e80 con 0x7f0da41101c0 2024-08-30T09:16:00.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.218+0000 7f0daa6be640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f0da4119d00 con 0x7f0da410f870 2024-08-30T09:16:00.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.218+0000 7f0daa6be640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f0da411a000 con 0x7f0da4109f30 2024-08-30T09:16:00.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.218+0000 7f0da3fff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0da4109f30 0x7f0da41a0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:16:00.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.218+0000 7f0da37fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0da410f870 0x7f0da41a0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:16:00.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.218+0000 7f0da3fff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0da4109f30 0x7f0da41a0870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.42:38796/0 (socket says 172.21.15.42:38796) 2024-08-30T09:16:00.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.218+0000 7f0da3fff640 1 -- 172.21.15.42:0/2822367327 learned_addr learned my addr 172.21.15.42:0/2822367327 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:16:00.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.218+0000 7f0da8c5b640 1 --2- 172.21.15.42:0/2822367327 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0da41101c0 0x7f0da41a60b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:16:00.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.218+0000 7f0da3fff640 1 -- 172.21.15.42:0/2822367327 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0da410f870 msgr2=0x7f0da41a0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:16:00.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.218+0000 7f0da3fff640 1 --2- 172.21.15.42:0/2822367327 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0da410f870 0x7f0da41a0e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:00.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.218+0000 7f0da3fff640 1 -- 172.21.15.42:0/2822367327 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0da41101c0 msgr2=0x7f0da41a60b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:16:00.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.218+0000 7f0da3fff640 1 --2- 172.21.15.42:0/2822367327 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0da41101c0 0x7f0da41a60b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:00.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.218+0000 7f0da3fff640 1 -- 172.21.15.42:0/2822367327 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0da41a67e0 con 0x7f0da4109f30 2024-08-30T09:16:00.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.218+0000 7f0da8c5b640 1 --2- 172.21.15.42:0/2822367327 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0da41101c0 0x7f0da41a60b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-30T09:16:00.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.219+0000 7f0da3fff640 1 --2- 172.21.15.42:0/2822367327 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0da4109f30 0x7f0da41a0870 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f0d8c00d770 tx=0x7f0d8c00dc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:16:00.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.219+0000 7f0da17fa640 1 -- 172.21.15.42:0/2822367327 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d8c017070 con 0x7f0da4109f30 2024-08-30T09:16:00.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.219+0000 7f0daa6be640 1 -- 172.21.15.42:0/2822367327 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0da41a6ad0 con 0x7f0da4109f30 2024-08-30T09:16:00.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.219+0000 7f0daa6be640 1 -- 172.21.15.42:0/2822367327 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0da41ad230 con 0x7f0da4109f30 2024-08-30T09:16:00.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.219+0000 7f0da17fa640 1 -- 172.21.15.42:0/2822367327 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d8c0054d0 con 0x7f0da4109f30 2024-08-30T09:16:00.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.219+0000 7f0da17fa640 1 -- 172.21.15.42:0/2822367327 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d8c004d50 con 0x7f0da4109f30 2024-08-30T09:16:00.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.223+0000 7f0da17fa640 1 -- 172.21.15.42:0/2822367327 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7f0d8c009c30 con 0x7f0da4109f30 2024-08-30T09:16:00.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.224+0000 7f0db0c33740 1 Processor -- start 2024-08-30T09:16:00.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.225+0000 7f0db0c33740 1 -- start start 2024-08-30T09:16:00.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.225+0000 7f0db0c33740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5608dae43070 0x5608db1e82c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:16:00.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.225+0000 7f0db0c33740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5608dae1a1b0 0x5608db1e8800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:16:00.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.225+0000 7f0db0c33740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5608dae22390 0x5608db1e8d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:16:00.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.225+0000 7f0db0c33740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x5608db15f630 con 0x5608dae22390 2024-08-30T09:16:00.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.225+0000 7f0db0c33740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5608db14ebc0 con 0x5608dae1a1b0 2024-08-30T09:16:00.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.225+0000 7f0db0c33740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5608db152b70 con 0x5608dae43070 2024-08-30T09:16:00.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.225+0000 7f0da3fff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5608dae1a1b0 0x5608db1e8800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:16:00.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.225+0000 7f0da3fff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5608dae1a1b0 0x5608db1e8800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:37246/0 (socket says 172.21.15.42:37246) 2024-08-30T09:16:00.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.225+0000 7f0da3fff640 1 -- 172.21.15.42:0/3806598893 learned_addr learned my addr 172.21.15.42:0/3806598893 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:16:00.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.225+0000 7f0da8c5b640 1 --2- 172.21.15.42:0/3806598893 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5608dae43070 0x5608db1e82c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:16:00.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.225+0000 7f0da37fe640 1 --2- 172.21.15.42:0/3806598893 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5608dae22390 0x5608db1e8d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:16:00.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.226+0000 7f0da8c5b640 1 -- 172.21.15.42:0/3806598893 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5608dae1a1b0 msgr2=0x5608db1e8800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:16:00.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.226+0000 7f0da8c5b640 1 --2- 172.21.15.42:0/3806598893 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5608dae1a1b0 0x5608db1e8800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:00.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.226+0000 7f0da8c5b640 1 -- 172.21.15.42:0/3806598893 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5608dae22390 msgr2=0x5608db1e8d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:16:00.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.226+0000 7f0da8c5b640 1 --2- 172.21.15.42:0/3806598893 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5608dae22390 0x5608db1e8d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:00.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.226+0000 7f0da8c5b640 1 -- 172.21.15.42:0/3806598893 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5608daeae650 con 0x5608dae43070 2024-08-30T09:16:00.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.226+0000 7f0da8c5b640 1 --2- 172.21.15.42:0/3806598893 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5608dae43070 0x5608db1e82c0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f0d9c009f90 tx=0x7f0d9c010860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a48cbaf7945d6805 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:16:00.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.226+0000 7f0d9a7fc640 1 -- 172.21.15.42:0/3806598893 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d9c011700 con 0x5608dae43070 2024-08-30T09:16:00.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.226+0000 7f0d9a7fc640 1 -- 172.21.15.42:0/3806598893 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d9c011d50 con 0x5608dae43070 2024-08-30T09:16:00.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.226+0000 7f0d9a7fc640 1 -- 172.21.15.42:0/3806598893 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d9c01ad50 con 0x5608dae43070 2024-08-30T09:16:00.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.227+0000 7f0da17fa640 1 --2- 172.21.15.42:0/2822367327 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f0d900b4bb0 0x7f0d900b70c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:16:00.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.227+0000 7f0da17fa640 1 -- 172.21.15.42:0/2822367327 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f0d8c0e4cb0 con 0x7f0da4109f30 2024-08-30T09:16:00.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.228+0000 7f0db0c33740 1 -- 172.21.15.42:0/3806598893 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5608dae43070 msgr2=0x5608db1e82c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:16:00.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.228+0000 7f0db0c33740 1 --2- 172.21.15.42:0/3806598893 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5608dae43070 0x5608db1e82c0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f0d9c009f90 tx=0x7f0d9c010860 comp rx=0 tx=0).stop 2024-08-30T09:16:00.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.228+0000 7f0db0c33740 1 -- 172.21.15.42:0/3806598893 shutdown_connections 2024-08-30T09:16:00.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.228+0000 7f0db0c33740 1 --2- 172.21.15.42:0/3806598893 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5608dae22390 0x5608db1e8d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:00.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.228+0000 7f0db0c33740 1 --2- 172.21.15.42:0/3806598893 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5608dae1a1b0 0x5608db1e8800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:00.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.228+0000 7f0db0c33740 1 --2- 172.21.15.42:0/3806598893 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5608dae43070 0x5608db1e82c0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:00.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.228+0000 7f0db0c33740 1 -- 172.21.15.42:0/3806598893 >> 172.21.15.42:0/3806598893 conn(0x5608dae37460 msgr2=0x5608db1e61b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:16:00.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.228+0000 7f0db0c33740 1 -- 172.21.15.42:0/3806598893 shutdown_connections 2024-08-30T09:16:00.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.229+0000 7f0da3fff640 1 --2- 172.21.15.42:0/2822367327 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f0d900b4bb0 0x7f0d900b70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:16:00.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.229+0000 7f0db0c33740 1 -- 172.21.15.42:0/3806598893 wait complete. 2024-08-30T09:16:00.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.230+0000 7f0db0c33740 1 -- start start 2024-08-30T09:16:00.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.230+0000 7f0da3fff640 1 --2- 172.21.15.42:0/2822367327 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f0d900b4bb0 0x7f0d900b70c0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f0d8c009610 tx=0x7f0d8c0b4040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:16:00.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.230+0000 7f0db0c33740 1 Processor -- start 2024-08-30T09:16:00.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.231+0000 7f0db0c33740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5608dae1a1b0 0x5608db1f39c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:16:00.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.231+0000 7f0db0c33740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5608dae22390 0x5608db1f3f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:16:00.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.232+0000 7f0db0c33740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5608dae43070 0x5608db1ffe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:16:00.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.232+0000 7f0db0c33740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x5608dafb1700 con 0x5608dae22390 2024-08-30T09:16:00.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.232+0000 7f0db0c33740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5608db0d5700 con 0x5608dae1a1b0 2024-08-30T09:16:00.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.232+0000 7f0db0c33740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5608db1f6aa0 con 0x5608dae43070 2024-08-30T09:16:00.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.232+0000 7f0da37fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5608dae22390 0x5608db1f3f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:16:00.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.232+0000 7f0da37fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5608dae22390 0x5608db1f3f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:32862/0 (socket says 172.21.15.42:32862) 2024-08-30T09:16:00.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.232+0000 7f0da37fe640 1 -- 172.21.15.42:0/61767197 learned_addr learned my addr 172.21.15.42:0/61767197 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:16:00.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.233+0000 7f0db0c33740 10 client.0 _ll_register_callbacks cb 0x5608db104dc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:16:00.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.233+0000 7f0da8c5b640 1 --2- 172.21.15.42:0/61767197 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5608dae1a1b0 0x5608db1f39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:16:00.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.233+0000 7f0da37fe640 1 -- 172.21.15.42:0/61767197 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5608dae43070 msgr2=0x5608db1ffe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:16:00.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.233+0000 7f0da8c5b640 1 --2- 172.21.15.42:0/61767197 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5608dae43070 0x5608db1ffe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:16:00.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.233+0000 7f0da37fe640 1 --2- 172.21.15.42:0/61767197 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5608dae43070 0x5608db1ffe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:00.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.233+0000 7f0da37fe640 1 -- 172.21.15.42:0/61767197 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5608dae1a1b0 msgr2=0x5608db1f39c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:16:00.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.233+0000 7f0da37fe640 1 --2- 172.21.15.42:0/61767197 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5608dae1a1b0 0x5608db1f39c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:00.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.233+0000 7f0da37fe640 1 -- 172.21.15.42:0/61767197 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5608db10e8b0 con 0x5608dae22390 2024-08-30T09:16:00.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.234+0000 7f0da8c5b640 1 --2- 172.21.15.42:0/61767197 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5608dae1a1b0 0x5608db1f39c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-30T09:16:00.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.234+0000 7f0da37fe640 1 --2- 172.21.15.42:0/61767197 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5608dae22390 0x5608db1f3f00 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f0d940303f0 tx=0x7f0d94004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47820470b3dc29f7 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:16:00.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.234+0000 7f0d837fe640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-08-30T09:16:00.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.234+0000 7f0d837fe640 1 -- 172.21.15.42:0/61767197 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d94041070 con 0x5608dae22390 2024-08-30T09:16:00.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.234+0000 7f0d837fe640 1 -- 172.21.15.42:0/61767197 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f0d94030450 con 0x5608dae22390 2024-08-30T09:16:00.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.235+0000 7f0db0c33740 1 -- 172.21.15.42:0/61767197 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d9c011d50 con 0x5608dae22390 2024-08-30T09:16:00.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.235+0000 7f0d837fe640 1 -- 172.21.15.42:0/61767197 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f0d9404b050 con 0x5608dae22390 2024-08-30T09:16:00.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.235+0000 7f0db0c33740 10 client.17934 fetch_fsmap learned FSMap version 49 2024-08-30T09:16:00.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.235+0000 7f0db0c33740 1 -- 172.21.15.42:0/61767197 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x5608db202850 con 0x5608dae22390 2024-08-30T09:16:00.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.236+0000 7f0d837fe640 1 -- 172.21.15.42:0/61767197 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d94005cf0 con 0x5608dae22390 2024-08-30T09:16:00.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.236+0000 7f0d837fe640 1 -- 172.21.15.42:0/61767197 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 49) ==== 8362+0+0 (secure 0 0 0) 0x7f0d94032440 con 0x5608dae22390 2024-08-30T09:16:00.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.236+0000 7f0db0c33740 10 client.17934 fetch_fsmap finished waiting for FSMap version 49 2024-08-30T09:16:00.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.236+0000 7f0db0c33740 10 client.17934 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-30T09:16:00.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.236+0000 7f0db0c33740 20 client.17934 populate_metadata read hostname 'smithi042' 2024-08-30T09:16:00.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.236+0000 7f0db0c33740 1 --2- 172.21.15.42:0/61767197 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x5608dae019f0 0x5608db205e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:16:00.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.236+0000 7f0db0c33740 4 client.17934 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-30T09:16:00.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.236+0000 7f0db0c33740 1 -- 172.21.15.42:0/61767197 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f0d9c011d50 con 0x5608dae019f0 2024-08-30T09:16:00.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.237+0000 7f0da8c5b640 1 --2- 172.21.15.42:0/61767197 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x5608dae019f0 0x5608db205e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:16:00.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.239+0000 7f0da8c5b640 1 --2- 172.21.15.42:0/61767197 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x5608dae019f0 0x5608db205e80 crc :-1 s=READY pgs=307 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ff09a4cca8226200 server_cookie=aa98f96397d8bd71 in_seq=0 out_seq=0 2024-08-30T09:16:00.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.239+0000 7f0d837fe640 10 client.17934 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:16:00.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.240+0000 7f0d837fe640 1 -- 172.21.15.42:0/61767197 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f0d9c011d50 con 0x5608dae019f0 2024-08-30T09:16:00.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.240+0000 7f0d837fe640 10 client.17934 handle_command_reply: tid=0 2024-08-30T09:16:00.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.240+0000 7f0db0c33740 1 client.17934 shutdown 2024-08-30T09:16:00.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.240+0000 7f0db0c33740 1 -- 172.21.15.42:0/61767197 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5608dae22390 msgr2=0x5608db1f3f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:16:00.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.240+0000 7f0db0c33740 1 --2- 172.21.15.42:0/61767197 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5608dae22390 0x5608db1f3f00 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f0d940303f0 tx=0x7f0d94004270 comp rx=0 tx=0).stop 2024-08-30T09:16:00.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.240+0000 7f0db0c33740 1 -- 172.21.15.42:0/61767197 shutdown_connections 2024-08-30T09:16:00.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.240+0000 7f0db0c33740 1 --2- 172.21.15.42:0/61767197 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x5608dae019f0 0x5608db205e80 crc :-1 s=READY pgs=307 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:00.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.241+0000 7f0db0c33740 1 --2- 172.21.15.42:0/61767197 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5608dae43070 0x5608db1ffe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:00.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.241+0000 7f0d837fe640 0 client.17934 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:16:00.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.241+0000 7f0db0c33740 1 --2- 172.21.15.42:0/61767197 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5608dae22390 0x5608db1f3f00 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:00.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.241+0000 7f0db0c33740 1 --2- 172.21.15.42:0/61767197 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5608dae1a1b0 0x5608db1f39c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:00.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.241+0000 7f0db0c33740 1 -- 172.21.15.42:0/61767197 >> 172.21.15.42:0/61767197 conn(0x5608dae37460 msgr2=0x5608db1eb450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:16:00.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.241+0000 7f0db0c33740 1 -- 172.21.15.42:0/61767197 shutdown_connections 2024-08-30T09:16:00.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.241+0000 7f0db0c33740 1 -- 172.21.15.42:0/61767197 wait complete. 2024-08-30T09:16:00.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.241+0000 7f0db0c33740 20 client.17934 trim_cache size 0 max 16384 2024-08-30T09:16:00.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.278+0000 7f0db0c33740 1 Processor -- start 2024-08-30T09:16:00.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.278+0000 7f0db0c33740 1 -- start start 2024-08-30T09:16:00.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.279+0000 7f0db0c33740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5608dae019f0 0x5608db1ebad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:16:00.282 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:16:00.279+0000 7fc94affd640 1 -- 172.21.15.42:0/2670380742 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55aa34554130 con 0x55aa341d41c0 2024-08-30T09:16:00.282 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:16:00.279+0000 7fc94affd640 10 client.27764 handle_command_reply: tid=0 2024-08-30T09:16:00.282 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:16:00.279+0000 7fc978bd6740 1 client.27764 shutdown 2024-08-30T09:16:00.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.279+0000 7f0db0c33740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5608dae1a1b0 0x5608db1f3930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:16:00.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.279+0000 7f0da8c5b640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5608dae019f0 0x5608db1ebad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:16:00.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.279+0000 7f0da8c5b640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5608dae019f0 0x5608db1ebad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:32864/0 (socket says 172.21.15.42:32864) 2024-08-30T09:16:00.283 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-08-30T09:16:00.283 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2024-08-30T09:16:00.283 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-08-30T09:16:00.283 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-08-30T09:16:00.284 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:16:00.279+0000 7fc978bd6740 1 -- 172.21.15.42:0/2670380742 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55aa342172f0 msgr2=0x55aa345f4b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:16:00.284 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:16:00.279+0000 7fc978bd6740 1 --2- 172.21.15.42:0/2670380742 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55aa342172f0 0x55aa345f4b20 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fc964009990 tx=0x7fc9640098e0 comp rx=0 tx=0).stop 2024-08-30T09:16:00.284 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:16:00.279+0000 7fc978bd6740 1 -- 172.21.15.42:0/2670380742 shutdown_connections 2024-08-30T09:16:00.284 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:16:00.279+0000 7fc978bd6740 1 --2- 172.21.15.42:0/2670380742 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55aa341d41c0 0x55aa3466a5f0 crc :-1 s=READY pgs=306 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:00.284 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:16:00.279+0000 7fc978bd6740 1 --2- 172.21.15.42:0/2670380742 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55aa3421f4d0 0x55aa345f51e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:00.284 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:16:00.279+0000 7fc978bd6740 1 --2- 172.21.15.42:0/2670380742 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55aa342401b0 0x55aa345f5d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:00.284 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:16:00.279+0000 7fc978bd6740 1 --2- 172.21.15.42:0/2670380742 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55aa342172f0 0x55aa345f4b20 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:00.284 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:16:00.279+0000 7fc978bd6740 1 -- 172.21.15.42:0/2670380742 >> 172.21.15.42:0/2670380742 conn(0x55aa34234300 msgr2=0x55aa345e5d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:16:00.284 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:16:00.279+0000 7fc94affd640 0 client.27764 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:16:00.284 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:16:00.280+0000 7fc978bd6740 1 -- 172.21.15.42:0/2670380742 shutdown_connections 2024-08-30T09:16:00.285 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:16:00.280+0000 7fc978bd6740 1 -- 172.21.15.42:0/2670380742 wait complete. 2024-08-30T09:16:00.285 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:16:00.280+0000 7fc978bd6740 20 client.27764 trim_cache size 0 max 16384 2024-08-30T09:16:00.285 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.279+0000 7f0da8c5b640 1 -- 172.21.15.42:0/26290533 learned_addr learned my addr 172.21.15.42:0/26290533 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:16:00.285 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.279+0000 7f0da37fe640 1 --2- 172.21.15.42:0/26290533 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5608dae1a1b0 0x5608db1f3930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:16:00.285 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.279+0000 7f0db0c33740 1 --2- 172.21.15.42:0/26290533 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5608dae22390 0x5608db1f0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:16:00.285 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.279+0000 7f0db0c33740 1 -- 172.21.15.42:0/26290533 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f0d84012520 con 0x5608dae019f0 2024-08-30T09:16:00.285 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.279+0000 7f0db0c33740 1 -- 172.21.15.42:0/26290533 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f0d8400fdf0 con 0x5608dae22390 2024-08-30T09:16:00.285 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.279+0000 7f0db0c33740 1 -- 172.21.15.42:0/26290533 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f0d8400dc40 con 0x5608dae1a1b0 2024-08-30T09:16:00.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.280+0000 7f0da8c5b640 1 --2- 172.21.15.42:0/26290533 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5608dae22390 0x5608db1f0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:16:00.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.280+0000 7f0da8c5b640 1 -- 172.21.15.42:0/26290533 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5608dae1a1b0 msgr2=0x5608db1f3930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:16:00.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.280+0000 7f0da8c5b640 1 --2- 172.21.15.42:0/26290533 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5608dae1a1b0 0x5608db1f3930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:00.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.280+0000 7f0da8c5b640 1 -- 172.21.15.42:0/26290533 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5608dae22390 msgr2=0x5608db1f0bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:16:00.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.280+0000 7f0da8c5b640 1 --2- 172.21.15.42:0/26290533 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5608dae22390 0x5608db1f0bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:00.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.280+0000 7f0da8c5b640 1 -- 172.21.15.42:0/26290533 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5608db1f2920 con 0x5608dae019f0 2024-08-30T09:16:00.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.280+0000 7f0da8c5b640 1 --2- 172.21.15.42:0/26290533 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5608dae019f0 0x5608db1ebad0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f0d9c01a8c0 tx=0x7f0d9c013040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90aac9dcb6ac8f2c server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:16:00.286 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:16:00.281+0000 7fc971b8b640 1 -- 172.21.15.42:0/1175973302 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fc9580b4d40 msgr2=0x7fc9580b7250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:16:00.287 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:16:00.281+0000 7fc971b8b640 1 --2- 172.21.15.42:0/1175973302 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fc9580b4d40 0x7fc9580b7250 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fc95400d010 tx=0x7fc954003e50 comp rx=0 tx=0).stop 2024-08-30T09:16:00.287 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:16:00.281+0000 7fc971b8b640 1 -- 172.21.15.42:0/1175973302 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc96c10f770 msgr2=0x7fc96c1a9d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:16:00.287 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:16:00.281+0000 7fc971b8b640 1 --2- 172.21.15.42:0/1175973302 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc96c10f770 0x7fc96c1a9d00 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fc95400e9b0 tx=0x7fc95400ee70 comp rx=0 tx=0).stop 2024-08-30T09:16:00.287 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:16:00.281+0000 7fc971b8b640 1 -- 172.21.15.42:0/1175973302 shutdown_connections 2024-08-30T09:16:00.287 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:16:00.281+0000 7fc971b8b640 1 --2- 172.21.15.42:0/1175973302 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fc9580b4d40 0x7fc9580b7250 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:00.287 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:16:00.281+0000 7fc971b8b640 1 --2- 172.21.15.42:0/1175973302 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc96c10cb50 0x7fc96c1a9150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:00.287 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:16:00.281+0000 7fc971b8b640 1 --2- 172.21.15.42:0/1175973302 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc96c10f770 0x7fc96c1a9d00 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:00.287 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:16:00.281+0000 7fc971b8b640 1 --2- 172.21.15.42:0/1175973302 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc96c10a160 0x7fc96c1a8ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:00.287 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:16:00.281+0000 7fc971b8b640 1 -- 172.21.15.42:0/1175973302 >> 172.21.15.42:0/1175973302 conn(0x7fc96c0705b0 msgr2=0x7fc96c10af30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:16:00.287 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:16:00.281+0000 7fc971b8b640 1 -- 172.21.15.42:0/1175973302 shutdown_connections 2024-08-30T09:16:00.287 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-08-30T09:16:00.281+0000 7fc971b8b640 1 -- 172.21.15.42:0/1175973302 wait complete. 2024-08-30T09:16:00.289 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.281+0000 7f0d99ffb640 1 -- 172.21.15.42:0/26290533 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d9c004d00 con 0x5608dae019f0 2024-08-30T09:16:00.289 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.281+0000 7f0d99ffb640 1 -- 172.21.15.42:0/26290533 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d9c01ae40 con 0x5608dae019f0 2024-08-30T09:16:00.290 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.281+0000 7f0d99ffb640 1 -- 172.21.15.42:0/26290533 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d9c011c50 con 0x5608dae019f0 2024-08-30T09:16:00.290 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.282+0000 7f0db0c33740 1 -- 172.21.15.42:0/26290533 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5608dae019f0 msgr2=0x5608db1ebad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:16:00.290 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.283+0000 7f0db0c33740 1 --2- 172.21.15.42:0/26290533 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5608dae019f0 0x5608db1ebad0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f0d9c01a8c0 tx=0x7f0d9c013040 comp rx=0 tx=0).stop 2024-08-30T09:16:00.290 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.283+0000 7f0db0c33740 1 -- 172.21.15.42:0/26290533 shutdown_connections 2024-08-30T09:16:00.290 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.283+0000 7f0db0c33740 1 --2- 172.21.15.42:0/26290533 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5608dae1a1b0 0x5608db1f3930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:00.290 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.283+0000 7f0db0c33740 1 --2- 172.21.15.42:0/26290533 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5608dae22390 0x5608db1f0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:00.290 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.283+0000 7f0db0c33740 1 --2- 172.21.15.42:0/26290533 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5608dae019f0 0x5608db1ebad0 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:00.290 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.283+0000 7f0db0c33740 1 -- 172.21.15.42:0/26290533 >> 172.21.15.42:0/26290533 conn(0x5608dae37460 msgr2=0x5608db1eb3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:16:00.290 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.283+0000 7f0db0c33740 1 -- 172.21.15.42:0/26290533 shutdown_connections 2024-08-30T09:16:00.290 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.283+0000 7f0db0c33740 1 -- 172.21.15.42:0/26290533 wait complete. 2024-08-30T09:16:00.290 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.284+0000 7f0db0c33740 1 -- start start 2024-08-30T09:16:00.290 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.285+0000 7f0db0c33740 1 Processor -- start 2024-08-30T09:16:00.290 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.285+0000 7f0db0c33740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5608dae019f0 0x5608db1fc3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:16:00.290 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.286+0000 7f0db0c33740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5608dae1a1b0 0x5608db1f8890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:16:00.290 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.286+0000 7f0da8c5b640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5608dae019f0 0x5608db1fc3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:16:00.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.286+0000 7f0da8c5b640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5608dae019f0 0x5608db1fc3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.42:37282/0 (socket says 172.21.15.42:37282) 2024-08-30T09:16:00.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.286+0000 7f0da8c5b640 1 -- 172.21.15.42:0/219238615 learned_addr learned my addr 172.21.15.42:0/219238615 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:16:00.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.286+0000 7f0db0c33740 1 --2- 172.21.15.42:0/219238615 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5608dae22390 0x5608db1f8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:16:00.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.286+0000 7f0da37fe640 1 --2- 172.21.15.42:0/219238615 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5608dae1a1b0 0x5608db1f8890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:16:00.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.286+0000 7f0db0c33740 1 -- 172.21.15.42:0/219238615 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f0d8400e2b0 con 0x5608dae22390 2024-08-30T09:16:00.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.286+0000 7f0db0c33740 1 -- 172.21.15.42:0/219238615 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f0d8400f720 con 0x5608dae019f0 2024-08-30T09:16:00.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.287+0000 7f0db0c33740 1 -- 172.21.15.42:0/219238615 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f0d8400e980 con 0x5608dae1a1b0 2024-08-30T09:16:00.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.287+0000 7f0da8c5b640 1 --2- 172.21.15.42:0/219238615 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5608dae22390 0x5608db1f8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:16:00.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.287+0000 7f0da8c5b640 1 -- 172.21.15.42:0/219238615 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5608dae1a1b0 msgr2=0x5608db1f8890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:16:00.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.287+0000 7f0da8c5b640 1 --2- 172.21.15.42:0/219238615 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5608dae1a1b0 0x5608db1f8890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:00.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.287+0000 7f0da8c5b640 1 -- 172.21.15.42:0/219238615 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5608dae22390 msgr2=0x5608db1f8dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:16:00.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.287+0000 7f0da8c5b640 1 --2- 172.21.15.42:0/219238615 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5608dae22390 0x5608db1f8dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:00.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.287+0000 7f0da8c5b640 1 -- 172.21.15.42:0/219238615 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5608db1f93c0 con 0x5608dae019f0 2024-08-30T09:16:00.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.287+0000 7f0db0c33740 10 client.0 _ll_register_callbacks cb 0x7f0d9c0093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:16:00.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.287+0000 7f0db0c33740 1 -- 172.21.15.42:0/219238615 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5608db161580 con 0x5608dae019f0 2024-08-30T09:16:00.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.288+0000 7f0da8c5b640 1 --2- 172.21.15.42:0/219238615 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5608dae019f0 0x5608db1fc3d0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f0d9c009a40 tx=0x7f0d9c010fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c0f936a6cf04560f server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:16:00.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.288+0000 7f0d837fe640 10 client.27782 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-30T09:16:00.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.288+0000 7f0d837fe640 1 -- 172.21.15.42:0/219238615 <== mon.2 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d9c004b00 con 0x5608dae019f0 2024-08-30T09:16:00.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.288+0000 7f0d837fe640 1 -- 172.21.15.42:0/219238615 <== mon.2 v2:172.21.15.167:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f0d9c004ca0 con 0x5608dae019f0 2024-08-30T09:16:00.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.288+0000 7f0d837fe640 1 -- 172.21.15.42:0/219238615 <== mon.2 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f0d9c0118d0 con 0x5608dae019f0 2024-08-30T09:16:00.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.288+0000 7f0db0c33740 10 client.27782 fetch_fsmap learned FSMap version 49 2024-08-30T09:16:00.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.288+0000 7f0db0c33740 1 -- 172.21.15.42:0/219238615 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({fsmap=49}) -- 0x5608db271280 con 0x5608dae019f0 2024-08-30T09:16:00.294 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.289+0000 7f0d837fe640 1 -- 172.21.15.42:0/219238615 <== mon.2 v2:172.21.15.167:3300/0 4 ==== fsmap(e 49) ==== 8362+0+0 (secure 0 0 0) 0x7f0d9c03e020 con 0x5608dae019f0 2024-08-30T09:16:00.294 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.289+0000 7f0db0c33740 10 client.27782 fetch_fsmap finished waiting for FSMap version 49 2024-08-30T09:16:00.294 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.289+0000 7f0db0c33740 10 client.27782 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-30T09:16:00.294 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.289+0000 7f0db0c33740 20 client.27782 populate_metadata read hostname 'smithi042' 2024-08-30T09:16:00.294 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.289+0000 7f0db0c33740 1 --2- 172.21.15.42:0/219238615 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x5608dae43070 0x5608db2737a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:16:00.294 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.289+0000 7f0db0c33740 4 client.27782 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-30T09:16:00.294 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.289+0000 7f0db0c33740 1 -- 172.21.15.42:0/219238615 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5608db15d020 con 0x5608dae43070 2024-08-30T09:16:00.294 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.289+0000 7f0da37fe640 1 --2- 172.21.15.42:0/219238615 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x5608dae43070 0x5608db2737a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:16:00.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.290+0000 7f0d837fe640 10 client.27782 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:16:00.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.290+0000 7f0da37fe640 1 --2- 172.21.15.42:0/219238615 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x5608dae43070 0x5608db2737a0 crc :-1 s=READY pgs=308 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c5b43fb862b8ee78 server_cookie=dcb09ff62cecd39e in_seq=0 out_seq=0 2024-08-30T09:16:00.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.291+0000 7f0d837fe640 1 -- 172.21.15.42:0/219238615 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+5202 (crc 0 0 0) 0x5608db15d020 con 0x5608dae43070 2024-08-30T09:16:00.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.291+0000 7f0d837fe640 10 client.27782 handle_command_reply: tid=0 2024-08-30T09:16:00.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.291+0000 7f0db0c33740 1 client.27782 shutdown 2024-08-30T09:16:00.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.291+0000 7f0db0c33740 1 -- 172.21.15.42:0/219238615 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5608dae019f0 msgr2=0x5608db1fc3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:16:00.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.291+0000 7f0db0c33740 1 --2- 172.21.15.42:0/219238615 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5608dae019f0 0x5608db1fc3d0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f0d9c009a40 tx=0x7f0d9c010fd0 comp rx=0 tx=0).stop 2024-08-30T09:16:00.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.291+0000 7f0db0c33740 1 -- 172.21.15.42:0/219238615 shutdown_connections 2024-08-30T09:16:00.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.291+0000 7f0db0c33740 1 --2- 172.21.15.42:0/219238615 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x5608dae43070 0x5608db2737a0 crc :-1 s=READY pgs=308 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:00.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.291+0000 7f0db0c33740 1 --2- 172.21.15.42:0/219238615 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5608dae1a1b0 0x5608db1f8890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:00.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.291+0000 7f0db0c33740 1 --2- 172.21.15.42:0/219238615 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5608dae22390 0x5608db1f8dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:00.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.291+0000 7f0db0c33740 1 --2- 172.21.15.42:0/219238615 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5608dae019f0 0x5608db1fc3d0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:00.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.291+0000 7f0db0c33740 1 -- 172.21.15.42:0/219238615 >> 172.21.15.42:0/219238615 conn(0x5608dae37460 msgr2=0x5608db1e92f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:16:00.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.291+0000 7f0d837fe640 0 client.27782 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:16:00.297 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.292+0000 7f0db0c33740 1 -- 172.21.15.42:0/219238615 shutdown_connections 2024-08-30T09:16:00.297 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.292+0000 7f0db0c33740 1 -- 172.21.15.42:0/219238615 wait complete. 2024-08-30T09:16:00.297 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.292+0000 7f0db0c33740 20 client.27782 trim_cache size 0 max 16384 2024-08-30T09:16:00.297 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-08-30T09:16:00.297 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "no active scrubs running", 2024-08-30T09:16:00.297 INFO:teuthology.orchestra.run.smithi042.stdout: "scrubs": {}, 2024-08-30T09:16:00.297 INFO:teuthology.orchestra.run.smithi042.stdout: "scrub_stats": { 2024-08-30T09:16:00.297 INFO:teuthology.orchestra.run.smithi042.stdout: "3f8863e7-c1a4-472c-8bd7-690aeeb13b82": { 2024-08-30T09:16:00.298 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:15:25.981002+0000", 2024-08-30T09:16:00.298 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:16:00.298 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:16:00.298 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:16:00.298 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:16:00.298 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:16:00.298 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:00.298 INFO:teuthology.orchestra.run.smithi042.stdout: "d325166d-e7e4-4ac2-9eaf-fa602e480cf6": { 2024-08-30T09:16:00.298 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:14:30.861941+0000", 2024-08-30T09:16:00.298 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:16:00.298 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:16:00.298 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:16:00.298 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:16:00.298 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:16:00.298 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:00.299 INFO:teuthology.orchestra.run.smithi042.stdout: "03521407-a747-49b9-a415-6233551c26fb": { 2024-08-30T09:16:00.299 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:13:39.387817+0000", 2024-08-30T09:16:00.299 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:16:00.299 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:16:00.299 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:16:00.299 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:16:00.299 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:16:00.299 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:00.299 INFO:teuthology.orchestra.run.smithi042.stdout: "0cbf734e-f988-4b21-acac-57b55e3bde62": { 2024-08-30T09:16:00.299 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:12:45.306738+0000", 2024-08-30T09:16:00.299 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:16:00.299 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:16:00.299 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:16:00.300 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:16:00.300 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:16:00.300 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:00.300 INFO:teuthology.orchestra.run.smithi042.stdout: "5e2ed104-0625-478a-a78b-584e7f4c2817": { 2024-08-30T09:16:00.300 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:00:33.928400+0000", 2024-08-30T09:16:00.300 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:16:00.300 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:16:00.300 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:16:00.300 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:16:00.300 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:16:00.300 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:00.300 INFO:teuthology.orchestra.run.smithi042.stdout: "c827ad7a-44ee-4e1d-9bb3-51fb7ab00c73": { 2024-08-30T09:16:00.300 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:02:21.686517+0000", 2024-08-30T09:16:00.300 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:16:00.301 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:16:00.301 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:16:00.301 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:16:00.301 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:16:00.301 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:00.301 INFO:teuthology.orchestra.run.smithi042.stdout: "85134e4b-ab4a-4885-b07e-a3ad01eaf13d": { 2024-08-30T09:16:00.301 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:01:27.738462+0000", 2024-08-30T09:16:00.301 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:16:00.301 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:16:00.301 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:16:00.301 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:16:00.301 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:16:00.301 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:00.301 INFO:teuthology.orchestra.run.smithi042.stdout: "749f4867-0b70-49e5-af2d-8362a240ebc3": { 2024-08-30T09:16:00.301 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:03:16.386554+0000", 2024-08-30T09:16:00.302 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:16:00.302 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:16:00.302 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:16:00.302 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:16:00.302 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:16:00.302 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:00.302 INFO:teuthology.orchestra.run.smithi042.stdout: "88cd7478-2b20-47de-ba51-256479749232": { 2024-08-30T09:16:00.302 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:04:08.922625+0000", 2024-08-30T09:16:00.302 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:16:00.302 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:16:00.302 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:16:00.302 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:16:00.302 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:16:00.302 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:00.303 INFO:teuthology.orchestra.run.smithi042.stdout: "347d05ed-4bd3-4590-b099-314f719aab39": { 2024-08-30T09:16:00.303 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:05:00.439721+0000", 2024-08-30T09:16:00.303 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:16:00.303 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:16:00.303 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:16:00.303 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:16:00.303 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:16:00.303 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:00.303 INFO:teuthology.orchestra.run.smithi042.stdout: "1af4252a-4a99-47e9-bbf0-becf5e573dd5": { 2024-08-30T09:16:00.303 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:05:48.565881+0000", 2024-08-30T09:16:00.303 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:16:00.303 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:16:00.303 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:16:00.303 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:16:00.304 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:16:00.304 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:00.304 INFO:teuthology.orchestra.run.smithi042.stdout: "665ec740-7254-49cb-84ef-deff71c461b0": { 2024-08-30T09:16:00.304 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:06:37.144035+0000", 2024-08-30T09:16:00.304 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:16:00.304 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:16:00.304 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:16:00.304 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:16:00.304 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:16:00.304 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:00.304 INFO:teuthology.orchestra.run.smithi042.stdout: "598d8a75-f1cc-4388-80bc-a408bdb4eaac": { 2024-08-30T09:16:00.304 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:07:29.721122+0000", 2024-08-30T09:16:00.305 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:16:00.305 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:16:00.305 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:16:00.305 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:16:00.305 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:16:00.305 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:00.305 INFO:teuthology.orchestra.run.smithi042.stdout: "b62c4ffc-9387-4677-b9ba-1e94ba308f17": { 2024-08-30T09:16:00.305 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:11:03.363478+0000", 2024-08-30T09:16:00.305 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:16:00.305 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:16:00.305 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:16:00.305 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:16:00.306 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:16:00.306 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:00.306 INFO:teuthology.orchestra.run.smithi042.stdout: "a7210f9a-046b-4188-953c-819b9f1863a9": { 2024-08-30T09:16:00.306 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:10:07.608430+0000", 2024-08-30T09:16:00.306 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:16:00.306 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:16:00.306 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:16:00.306 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:16:00.306 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:16:00.306 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:00.306 INFO:teuthology.orchestra.run.smithi042.stdout: "99a29ec1-9455-4ae4-854a-398b32896675": { 2024-08-30T09:16:00.306 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:08:24.923176+0000", 2024-08-30T09:16:00.306 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:16:00.306 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:16:00.307 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:16:00.307 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:16:00.307 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:16:00.307 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:00.307 INFO:teuthology.orchestra.run.smithi042.stdout: "ff464e7f-123b-4bc6-bc7e-aced018168de": { 2024-08-30T09:16:00.307 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:11:54.879598+0000", 2024-08-30T09:16:00.307 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:16:00.307 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:16:00.307 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:16:00.307 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:16:00.307 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:16:00.307 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:00.307 INFO:teuthology.orchestra.run.smithi042.stdout: "37f2f28f-496f-41a6-b4cf-8846fb3f43ab": { 2024-08-30T09:16:00.307 INFO:teuthology.orchestra.run.smithi042.stdout: "start_time": "2024-08-30T09:09:17.428286+0000", 2024-08-30T09:16:00.307 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-08-30T09:16:00.308 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_started": 0, 2024-08-30T09:16:00.308 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_passed": 0, 2024-08-30T09:16:00.308 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_failed": 0, 2024-08-30T09:16:00.308 INFO:teuthology.orchestra.run.smithi042.stdout: "uninline_skipped": 0 2024-08-30T09:16:00.308 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:00.308 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:00.308 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-08-30T09:16:00.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.294+0000 7f0daa6be640 1 -- 172.21.15.42:0/2822367327 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f0d900b4bb0 msgr2=0x7f0d900b70c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:16:00.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.294+0000 7f0daa6be640 1 --2- 172.21.15.42:0/2822367327 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f0d900b4bb0 0x7f0d900b70c0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f0d8c009610 tx=0x7f0d8c0b4040 comp rx=0 tx=0).stop 2024-08-30T09:16:00.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.294+0000 7f0daa6be640 1 -- 172.21.15.42:0/2822367327 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0da4109f30 msgr2=0x7f0da41a0870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:16:00.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.294+0000 7f0daa6be640 1 --2- 172.21.15.42:0/2822367327 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0da4109f30 0x7f0da41a0870 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f0d8c00d770 tx=0x7f0d8c00dc30 comp rx=0 tx=0).stop 2024-08-30T09:16:00.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.294+0000 7f0daa6be640 1 -- 172.21.15.42:0/2822367327 shutdown_connections 2024-08-30T09:16:00.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.294+0000 7f0daa6be640 1 --2- 172.21.15.42:0/2822367327 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f0d900b4bb0 0x7f0d900b70c0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:00.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.294+0000 7f0daa6be640 1 --2- 172.21.15.42:0/2822367327 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0da41101c0 0x7f0da41a60b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:00.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.294+0000 7f0daa6be640 1 --2- 172.21.15.42:0/2822367327 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0da410f870 0x7f0da41a0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:00.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.294+0000 7f0daa6be640 1 --2- 172.21.15.42:0/2822367327 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0da4109f30 0x7f0da41a0870 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:00.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.294+0000 7f0daa6be640 1 -- 172.21.15.42:0/2822367327 >> 172.21.15.42:0/2822367327 conn(0x7f0da4077f10 msgr2=0x7f0da410db90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:16:00.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.295+0000 7f0daa6be640 1 -- 172.21.15.42:0/2822367327 shutdown_connections 2024-08-30T09:16:00.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:00.295+0000 7f0daa6be640 1 -- 172.21.15.42:0/2822367327 wait complete. 2024-08-30T09:16:00.733 DEBUG:teuthology.orchestra.run.smithi042:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -lZaR /tmp/cephfs/postgres/' 2024-08-30T09:16:00.802 INFO:teuthology.orchestra.run.smithi042.stdout:/tmp/cephfs/postgres/: 2024-08-30T09:16:00.802 INFO:teuthology.orchestra.run.smithi042.stdout:total 0 2024-08-30T09:16:00.802 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Aug 30 09:00 . 2024-08-30T09:16:00.802 INFO:teuthology.orchestra.run.smithi042.stdout:drwxrwxrwt. 3 root root system_u:object_r:postgresql_db_t:s0 1 Aug 30 09:00 .. 2024-08-30T09:16:00.802 INFO:teuthology.orchestra.run.smithi042.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Aug 30 09:16 data 2024-08-30T09:16:00.802 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-08-30T09:16:00.802 INFO:teuthology.orchestra.run.smithi042.stdout:/tmp/cephfs/postgres/data: 2024-08-30T09:16:00.803 INFO:teuthology.orchestra.run.smithi042.stdout:total 36 2024-08-30T09:16:00.803 INFO:teuthology.orchestra.run.smithi042.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Aug 30 09:16 . 2024-08-30T09:16:00.803 INFO:teuthology.orchestra.run.smithi042.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Aug 30 09:00 .. 2024-08-30T09:16:00.803 INFO:teuthology.orchestra.run.smithi042.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Aug 30 09:00 base 2024-08-30T09:16:00.803 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 30 Aug 30 09:00 current_logfiles 2024-08-30T09:16:00.803 INFO:teuthology.orchestra.run.smithi042.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 58 Aug 30 09:00 global 2024-08-30T09:16:00.803 INFO:teuthology.orchestra.run.smithi042.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Aug 30 09:00 log 2024-08-30T09:16:00.803 INFO:teuthology.orchestra.run.smithi042.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 pg_commit_ts 2024-08-30T09:16:00.803 INFO:teuthology.orchestra.run.smithi042.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 pg_dynshmem 2024-08-30T09:16:00.803 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 4516 Aug 30 09:00 pg_hba.conf 2024-08-30T09:16:00.803 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1636 Aug 30 09:00 pg_ident.conf 2024-08-30T09:16:00.803 INFO:teuthology.orchestra.run.smithi042.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Aug 30 09:16 pg_logical 2024-08-30T09:16:00.803 INFO:teuthology.orchestra.run.smithi042.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Aug 30 09:00 pg_multixact 2024-08-30T09:16:00.804 INFO:teuthology.orchestra.run.smithi042.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 pg_notify 2024-08-30T09:16:00.804 INFO:teuthology.orchestra.run.smithi042.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 pg_replslot 2024-08-30T09:16:00.804 INFO:teuthology.orchestra.run.smithi042.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 pg_serial 2024-08-30T09:16:00.804 INFO:teuthology.orchestra.run.smithi042.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 pg_snapshots 2024-08-30T09:16:00.804 INFO:teuthology.orchestra.run.smithi042.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Aug 30 09:16 pg_stat 2024-08-30T09:16:00.804 INFO:teuthology.orchestra.run.smithi042.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:16 pg_stat_tmp 2024-08-30T09:16:00.804 INFO:teuthology.orchestra.run.smithi042.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Aug 30 09:15 pg_subtrans 2024-08-30T09:16:00.804 INFO:teuthology.orchestra.run.smithi042.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 pg_tblspc 2024-08-30T09:16:00.804 INFO:teuthology.orchestra.run.smithi042.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 pg_twophase 2024-08-30T09:16:00.804 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Aug 30 09:00 PG_VERSION 2024-08-30T09:16:00.804 INFO:teuthology.orchestra.run.smithi042.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 60 Aug 30 09:16 pg_wal 2024-08-30T09:16:00.804 INFO:teuthology.orchestra.run.smithi042.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Aug 30 09:00 pg_xact 2024-08-30T09:16:00.804 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 88 Aug 30 09:00 postgresql.auto.conf 2024-08-30T09:16:00.805 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 28074 Aug 30 09:00 postgresql.conf 2024-08-30T09:16:00.805 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 51 Aug 30 09:00 postmaster.opts 2024-08-30T09:16:00.805 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-08-30T09:16:00.805 INFO:teuthology.orchestra.run.smithi042.stdout:/tmp/cephfs/postgres/data/base: 2024-08-30T09:16:00.805 INFO:teuthology.orchestra.run.smithi042.stdout:total 0 2024-08-30T09:16:00.805 INFO:teuthology.orchestra.run.smithi042.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Aug 30 09:00 . 2024-08-30T09:16:00.805 INFO:teuthology.orchestra.run.smithi042.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Aug 30 09:16 .. 2024-08-30T09:16:00.805 INFO:teuthology.orchestra.run.smithi042.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Aug 30 09:00 1 2024-08-30T09:16:00.805 INFO:teuthology.orchestra.run.smithi042.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Aug 30 09:00 13493 2024-08-30T09:16:00.805 INFO:teuthology.orchestra.run.smithi042.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 305 Aug 30 09:07 13494 2024-08-30T09:16:00.806 INFO:teuthology.orchestra.run.smithi042.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 pgsql_tmp 2024-08-30T09:16:00.806 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-08-30T09:16:00.806 INFO:teuthology.orchestra.run.smithi042.stdout:/tmp/cephfs/postgres/data/base/1: 2024-08-30T09:16:00.806 INFO:teuthology.orchestra.run.smithi042.stdout:total 7777 2024-08-30T09:16:00.806 INFO:teuthology.orchestra.run.smithi042.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Aug 30 09:00 . 2024-08-30T09:16:00.806 INFO:teuthology.orchestra.run.smithi042.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Aug 30 09:00 .. 2024-08-30T09:16:00.806 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 112 2024-08-30T09:16:00.806 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 113 2024-08-30T09:16:00.806 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Aug 30 09:00 1247 2024-08-30T09:16:00.806 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 1247_fsm 2024-08-30T09:16:00.806 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 1247_vm 2024-08-30T09:16:00.806 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 Aug 30 09:00 1249 2024-08-30T09:16:00.806 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 1249_fsm 2024-08-30T09:16:00.807 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 1249_vm 2024-08-30T09:16:00.807 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 Aug 30 09:00 1255 2024-08-30T09:16:00.807 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 1255_fsm 2024-08-30T09:16:00.807 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 1255_vm 2024-08-30T09:16:00.807 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Aug 30 09:00 1259 2024-08-30T09:16:00.807 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 1259_fsm 2024-08-30T09:16:00.807 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 1259_vm 2024-08-30T09:16:00.807 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 Aug 30 09:00 13344 2024-08-30T09:16:00.807 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 13344_fsm 2024-08-30T09:16:00.807 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 13344_vm 2024-08-30T09:16:00.807 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 13346 2024-08-30T09:16:00.807 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 13348 2024-08-30T09:16:00.808 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 13349 2024-08-30T09:16:00.808 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 13349_fsm 2024-08-30T09:16:00.808 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 13349_vm 2024-08-30T09:16:00.808 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 13351 2024-08-30T09:16:00.808 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 13353 2024-08-30T09:16:00.808 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 13354 2024-08-30T09:16:00.808 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 13354_fsm 2024-08-30T09:16:00.808 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 13354_vm 2024-08-30T09:16:00.808 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 13356 2024-08-30T09:16:00.808 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 13358 2024-08-30T09:16:00.808 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 13359 2024-08-30T09:16:00.809 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 13359_fsm 2024-08-30T09:16:00.809 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 13359_vm 2024-08-30T09:16:00.809 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 13361 2024-08-30T09:16:00.809 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 13363 2024-08-30T09:16:00.809 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 1417 2024-08-30T09:16:00.809 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 1418 2024-08-30T09:16:00.809 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 174 2024-08-30T09:16:00.809 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 175 2024-08-30T09:16:00.809 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2187 2024-08-30T09:16:00.809 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 2224 2024-08-30T09:16:00.809 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 2328 2024-08-30T09:16:00.809 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 2336 2024-08-30T09:16:00.809 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2337 2024-08-30T09:16:00.809 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2579 2024-08-30T09:16:00.810 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2600 2024-08-30T09:16:00.810 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 2600_fsm 2024-08-30T09:16:00.810 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2600_vm 2024-08-30T09:16:00.810 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2601 2024-08-30T09:16:00.810 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 2601_fsm 2024-08-30T09:16:00.810 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2601_vm 2024-08-30T09:16:00.810 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Aug 30 09:00 2602 2024-08-30T09:16:00.810 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 2602_fsm 2024-08-30T09:16:00.810 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2602_vm 2024-08-30T09:16:00.810 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 30 09:00 2603 2024-08-30T09:16:00.810 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 2603_fsm 2024-08-30T09:16:00.810 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2603_vm 2024-08-30T09:16:00.810 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 2604 2024-08-30T09:16:00.811 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2605 2024-08-30T09:16:00.811 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 2605_fsm 2024-08-30T09:16:00.811 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2605_vm 2024-08-30T09:16:00.811 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2606 2024-08-30T09:16:00.811 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 2606_fsm 2024-08-30T09:16:00.811 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2606_vm 2024-08-30T09:16:00.811 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2607 2024-08-30T09:16:00.811 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 2607_fsm 2024-08-30T09:16:00.811 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2607_vm 2024-08-30T09:16:00.811 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 Aug 30 09:00 2608 2024-08-30T09:16:00.811 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 2608_fsm 2024-08-30T09:16:00.811 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2608_vm 2024-08-30T09:16:00.811 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 Aug 30 09:00 2609 2024-08-30T09:16:00.812 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 2609_fsm 2024-08-30T09:16:00.812 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2609_vm 2024-08-30T09:16:00.812 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 30 09:00 2610 2024-08-30T09:16:00.812 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 2610_fsm 2024-08-30T09:16:00.812 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2610_vm 2024-08-30T09:16:00.812 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 2611 2024-08-30T09:16:00.812 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2612 2024-08-30T09:16:00.812 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 2612_fsm 2024-08-30T09:16:00.812 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2612_vm 2024-08-30T09:16:00.812 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 2613 2024-08-30T09:16:00.812 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2615 2024-08-30T09:16:00.812 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 2615_fsm 2024-08-30T09:16:00.812 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2615_vm 2024-08-30T09:16:00.813 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2616 2024-08-30T09:16:00.813 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 2616_fsm 2024-08-30T09:16:00.813 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2616_vm 2024-08-30T09:16:00.813 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 Aug 30 09:00 2617 2024-08-30T09:16:00.813 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 2617_fsm 2024-08-30T09:16:00.813 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2617_vm 2024-08-30T09:16:00.813 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Aug 30 09:00 2618 2024-08-30T09:16:00.813 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 2618_fsm 2024-08-30T09:16:00.813 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2618_vm 2024-08-30T09:16:00.813 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 147456 Aug 30 09:00 2619 2024-08-30T09:16:00.813 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 2619_fsm 2024-08-30T09:16:00.813 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2619_vm 2024-08-30T09:16:00.813 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 2620 2024-08-30T09:16:00.813 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2650 2024-08-30T09:16:00.814 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2651 2024-08-30T09:16:00.814 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2652 2024-08-30T09:16:00.814 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Aug 30 09:00 2653 2024-08-30T09:16:00.814 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Aug 30 09:00 2654 2024-08-30T09:16:00.814 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 30 09:00 2655 2024-08-30T09:16:00.814 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2656 2024-08-30T09:16:00.814 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2657 2024-08-30T09:16:00.814 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Aug 30 09:00 2658 2024-08-30T09:16:00.814 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Aug 30 09:00 2659 2024-08-30T09:16:00.814 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2660 2024-08-30T09:16:00.815 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2661 2024-08-30T09:16:00.815 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 30 09:00 2662 2024-08-30T09:16:00.815 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Aug 30 09:00 2663 2024-08-30T09:16:00.815 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2664 2024-08-30T09:16:00.815 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2665 2024-08-30T09:16:00.815 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2666 2024-08-30T09:16:00.815 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2667 2024-08-30T09:16:00.815 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2668 2024-08-30T09:16:00.815 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2669 2024-08-30T09:16:00.815 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2670 2024-08-30T09:16:00.815 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 Aug 30 09:00 2673 2024-08-30T09:16:00.815 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 Aug 30 09:00 2674 2024-08-30T09:16:00.815 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 Aug 30 09:00 2675 2024-08-30T09:16:00.816 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2678 2024-08-30T09:16:00.816 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2679 2024-08-30T09:16:00.816 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2680 2024-08-30T09:16:00.816 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2681 2024-08-30T09:16:00.816 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2682 2024-08-30T09:16:00.816 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2683 2024-08-30T09:16:00.816 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2684 2024-08-30T09:16:00.816 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2685 2024-08-30T09:16:00.816 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2686 2024-08-30T09:16:00.816 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2687 2024-08-30T09:16:00.816 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Aug 30 09:00 2688 2024-08-30T09:16:00.817 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Aug 30 09:00 2689 2024-08-30T09:16:00.817 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Aug 30 09:00 2690 2024-08-30T09:16:00.817 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 Aug 30 09:00 2691 2024-08-30T09:16:00.817 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2692 2024-08-30T09:16:00.817 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2693 2024-08-30T09:16:00.817 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2696 2024-08-30T09:16:00.817 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2699 2024-08-30T09:16:00.817 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2701 2024-08-30T09:16:00.817 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2702 2024-08-30T09:16:00.817 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 30 09:00 2703 2024-08-30T09:16:00.817 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Aug 30 09:00 2704 2024-08-30T09:16:00.817 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2753 2024-08-30T09:16:00.817 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 2753_fsm 2024-08-30T09:16:00.817 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2753_vm 2024-08-30T09:16:00.817 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2754 2024-08-30T09:16:00.818 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2755 2024-08-30T09:16:00.818 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 30 09:00 2756 2024-08-30T09:16:00.818 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 30 09:00 2757 2024-08-30T09:16:00.818 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 2830 2024-08-30T09:16:00.818 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2831 2024-08-30T09:16:00.818 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 2832 2024-08-30T09:16:00.818 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2833 2024-08-30T09:16:00.818 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 2834 2024-08-30T09:16:00.818 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2835 2024-08-30T09:16:00.818 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 2836 2024-08-30T09:16:00.818 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2837 2024-08-30T09:16:00.818 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 Aug 30 09:00 2838 2024-08-30T09:16:00.818 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 2838_fsm 2024-08-30T09:16:00.818 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2838_vm 2024-08-30T09:16:00.819 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2839 2024-08-30T09:16:00.819 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 2840 2024-08-30T09:16:00.819 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 2840_fsm 2024-08-30T09:16:00.819 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2840_vm 2024-08-30T09:16:00.819 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2841 2024-08-30T09:16:00.819 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 2995 2024-08-30T09:16:00.819 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2996 2024-08-30T09:16:00.819 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3079 2024-08-30T09:16:00.819 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 3079_fsm 2024-08-30T09:16:00.819 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3079_vm 2024-08-30T09:16:00.819 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 3080 2024-08-30T09:16:00.819 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 3081 2024-08-30T09:16:00.819 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Aug 30 09:00 3085 2024-08-30T09:16:00.819 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 3118 2024-08-30T09:16:00.820 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3119 2024-08-30T09:16:00.820 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Aug 30 09:00 3164 2024-08-30T09:16:00.820 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 3256 2024-08-30T09:16:00.820 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3257 2024-08-30T09:16:00.820 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3258 2024-08-30T09:16:00.820 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 3350 2024-08-30T09:16:00.820 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3351 2024-08-30T09:16:00.820 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3379 2024-08-30T09:16:00.820 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3380 2024-08-30T09:16:00.820 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 3381 2024-08-30T09:16:00.820 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 3394 2024-08-30T09:16:00.820 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 3394_fsm 2024-08-30T09:16:00.820 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3394_vm 2024-08-30T09:16:00.820 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 3395 2024-08-30T09:16:00.820 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 3429 2024-08-30T09:16:00.821 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 3430 2024-08-30T09:16:00.821 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3431 2024-08-30T09:16:00.821 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3433 2024-08-30T09:16:00.821 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 3439 2024-08-30T09:16:00.821 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3440 2024-08-30T09:16:00.821 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Aug 30 09:00 3455 2024-08-30T09:16:00.821 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 Aug 30 09:00 3456 2024-08-30T09:16:00.821 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 3456_fsm 2024-08-30T09:16:00.821 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3456_vm 2024-08-30T09:16:00.821 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 3466 2024-08-30T09:16:00.821 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3467 2024-08-30T09:16:00.821 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3468 2024-08-30T09:16:00.821 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 3501 2024-08-30T09:16:00.821 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3502 2024-08-30T09:16:00.822 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3503 2024-08-30T09:16:00.822 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3534 2024-08-30T09:16:00.822 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3541 2024-08-30T09:16:00.822 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 3541_fsm 2024-08-30T09:16:00.822 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3541_vm 2024-08-30T09:16:00.822 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 3542 2024-08-30T09:16:00.822 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3574 2024-08-30T09:16:00.822 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3575 2024-08-30T09:16:00.822 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 3576 2024-08-30T09:16:00.822 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 3596 2024-08-30T09:16:00.822 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3597 2024-08-30T09:16:00.822 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 3598 2024-08-30T09:16:00.822 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3599 2024-08-30T09:16:00.822 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3600 2024-08-30T09:16:00.823 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 3600_fsm 2024-08-30T09:16:00.823 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3600_vm 2024-08-30T09:16:00.823 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3601 2024-08-30T09:16:00.823 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 3601_fsm 2024-08-30T09:16:00.823 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3601_vm 2024-08-30T09:16:00.823 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3602 2024-08-30T09:16:00.823 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 3602_fsm 2024-08-30T09:16:00.823 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3602_vm 2024-08-30T09:16:00.824 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 3603 2024-08-30T09:16:00.824 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 3603_fsm 2024-08-30T09:16:00.824 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3603_vm 2024-08-30T09:16:00.824 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 3604 2024-08-30T09:16:00.824 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 3605 2024-08-30T09:16:00.824 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 3606 2024-08-30T09:16:00.824 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 3607 2024-08-30T09:16:00.824 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 3608 2024-08-30T09:16:00.824 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 30 09:00 3609 2024-08-30T09:16:00.824 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 3712 2024-08-30T09:16:00.824 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3764 2024-08-30T09:16:00.824 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 3764_fsm 2024-08-30T09:16:00.824 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3764_vm 2024-08-30T09:16:00.824 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 3766 2024-08-30T09:16:00.825 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 3767 2024-08-30T09:16:00.825 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3997 2024-08-30T09:16:00.825 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 4143 2024-08-30T09:16:00.825 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 4144 2024-08-30T09:16:00.825 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 4145 2024-08-30T09:16:00.825 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 4146 2024-08-30T09:16:00.825 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 4147 2024-08-30T09:16:00.825 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 4148 2024-08-30T09:16:00.825 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 4149 2024-08-30T09:16:00.825 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 4150 2024-08-30T09:16:00.825 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 4151 2024-08-30T09:16:00.825 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 4152 2024-08-30T09:16:00.825 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 4153 2024-08-30T09:16:00.825 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 4154 2024-08-30T09:16:00.825 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 4155 2024-08-30T09:16:00.826 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 4156 2024-08-30T09:16:00.826 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 4157 2024-08-30T09:16:00.826 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 4158 2024-08-30T09:16:00.826 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 4159 2024-08-30T09:16:00.826 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 4160 2024-08-30T09:16:00.826 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 4161 2024-08-30T09:16:00.826 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 4162 2024-08-30T09:16:00.827 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 4163 2024-08-30T09:16:00.827 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 4164 2024-08-30T09:16:00.827 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 4165 2024-08-30T09:16:00.827 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 4166 2024-08-30T09:16:00.827 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 4167 2024-08-30T09:16:00.827 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 4168 2024-08-30T09:16:00.827 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 4169 2024-08-30T09:16:00.827 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 4170 2024-08-30T09:16:00.827 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 4171 2024-08-30T09:16:00.828 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 4172 2024-08-30T09:16:00.828 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 4173 2024-08-30T09:16:00.828 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 4174 2024-08-30T09:16:00.828 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 5002 2024-08-30T09:16:00.828 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 548 2024-08-30T09:16:00.828 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 549 2024-08-30T09:16:00.828 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 6102 2024-08-30T09:16:00.828 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 6104 2024-08-30T09:16:00.828 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 6106 2024-08-30T09:16:00.828 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 6110 2024-08-30T09:16:00.828 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 6111 2024-08-30T09:16:00.828 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 6112 2024-08-30T09:16:00.828 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 6113 2024-08-30T09:16:00.828 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 6117 2024-08-30T09:16:00.829 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 826 2024-08-30T09:16:00.829 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 827 2024-08-30T09:16:00.829 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 828 2024-08-30T09:16:00.829 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Aug 30 09:00 pg_filenode.map 2024-08-30T09:16:00.829 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Aug 30 09:00 PG_VERSION 2024-08-30T09:16:00.829 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-08-30T09:16:00.829 INFO:teuthology.orchestra.run.smithi042.stdout:/tmp/cephfs/postgres/data/base/13493: 2024-08-30T09:16:00.829 INFO:teuthology.orchestra.run.smithi042.stdout:total 7777 2024-08-30T09:16:00.829 INFO:teuthology.orchestra.run.smithi042.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Aug 30 09:00 . 2024-08-30T09:16:00.829 INFO:teuthology.orchestra.run.smithi042.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Aug 30 09:00 .. 2024-08-30T09:16:00.829 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 112 2024-08-30T09:16:00.829 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 113 2024-08-30T09:16:00.829 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Aug 30 09:00 1247 2024-08-30T09:16:00.830 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 1247_fsm 2024-08-30T09:16:00.830 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 1247_vm 2024-08-30T09:16:00.830 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 Aug 30 09:00 1249 2024-08-30T09:16:00.830 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 1249_fsm 2024-08-30T09:16:00.830 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 1249_vm 2024-08-30T09:16:00.830 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 Aug 30 09:00 1255 2024-08-30T09:16:00.830 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 1255_fsm 2024-08-30T09:16:00.830 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 1255_vm 2024-08-30T09:16:00.830 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Aug 30 09:00 1259 2024-08-30T09:16:00.830 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 1259_fsm 2024-08-30T09:16:00.830 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 1259_vm 2024-08-30T09:16:00.830 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 Aug 30 09:00 13344 2024-08-30T09:16:00.830 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 13344_fsm 2024-08-30T09:16:00.831 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 13344_vm 2024-08-30T09:16:00.831 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 13346 2024-08-30T09:16:00.831 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 13348 2024-08-30T09:16:00.831 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 13349 2024-08-30T09:16:00.831 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 13349_fsm 2024-08-30T09:16:00.831 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 13349_vm 2024-08-30T09:16:00.831 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 13351 2024-08-30T09:16:00.831 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 13353 2024-08-30T09:16:00.831 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 13354 2024-08-30T09:16:00.831 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 13354_fsm 2024-08-30T09:16:00.831 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 13354_vm 2024-08-30T09:16:00.831 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 13356 2024-08-30T09:16:00.831 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 13358 2024-08-30T09:16:00.831 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 13359 2024-08-30T09:16:00.831 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 13359_fsm 2024-08-30T09:16:00.832 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 13359_vm 2024-08-30T09:16:00.832 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 13361 2024-08-30T09:16:00.832 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 13363 2024-08-30T09:16:00.832 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 1417 2024-08-30T09:16:00.832 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 1418 2024-08-30T09:16:00.832 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 174 2024-08-30T09:16:00.832 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 175 2024-08-30T09:16:00.832 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2187 2024-08-30T09:16:00.832 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 2224 2024-08-30T09:16:00.832 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 2328 2024-08-30T09:16:00.832 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 2336 2024-08-30T09:16:00.832 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2337 2024-08-30T09:16:00.832 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2579 2024-08-30T09:16:00.832 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2600 2024-08-30T09:16:00.833 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 2600_fsm 2024-08-30T09:16:00.833 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2600_vm 2024-08-30T09:16:00.833 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2601 2024-08-30T09:16:00.833 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 2601_fsm 2024-08-30T09:16:00.833 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2601_vm 2024-08-30T09:16:00.833 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Aug 30 09:00 2602 2024-08-30T09:16:00.833 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 2602_fsm 2024-08-30T09:16:00.833 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2602_vm 2024-08-30T09:16:00.833 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 30 09:00 2603 2024-08-30T09:16:00.833 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 2603_fsm 2024-08-30T09:16:00.833 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2603_vm 2024-08-30T09:16:00.833 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 2604 2024-08-30T09:16:00.833 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2605 2024-08-30T09:16:00.834 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 2605_fsm 2024-08-30T09:16:00.834 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2605_vm 2024-08-30T09:16:00.834 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2606 2024-08-30T09:16:00.834 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 2606_fsm 2024-08-30T09:16:00.834 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2606_vm 2024-08-30T09:16:00.834 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2607 2024-08-30T09:16:00.834 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 2607_fsm 2024-08-30T09:16:00.834 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2607_vm 2024-08-30T09:16:00.834 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 Aug 30 09:00 2608 2024-08-30T09:16:00.834 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 2608_fsm 2024-08-30T09:16:00.834 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2608_vm 2024-08-30T09:16:00.834 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 Aug 30 09:00 2609 2024-08-30T09:16:00.835 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 2609_fsm 2024-08-30T09:16:00.835 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2609_vm 2024-08-30T09:16:00.835 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 30 09:00 2610 2024-08-30T09:16:00.835 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 2610_fsm 2024-08-30T09:16:00.835 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2610_vm 2024-08-30T09:16:00.835 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 2611 2024-08-30T09:16:00.835 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2612 2024-08-30T09:16:00.835 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 2612_fsm 2024-08-30T09:16:00.835 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2612_vm 2024-08-30T09:16:00.835 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 2613 2024-08-30T09:16:00.835 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2615 2024-08-30T09:16:00.835 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 2615_fsm 2024-08-30T09:16:00.835 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2615_vm 2024-08-30T09:16:00.835 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2616 2024-08-30T09:16:00.836 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 2616_fsm 2024-08-30T09:16:00.836 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2616_vm 2024-08-30T09:16:00.836 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 Aug 30 09:00 2617 2024-08-30T09:16:00.836 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 2617_fsm 2024-08-30T09:16:00.836 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2617_vm 2024-08-30T09:16:00.836 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Aug 30 09:00 2618 2024-08-30T09:16:00.836 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 2618_fsm 2024-08-30T09:16:00.836 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2618_vm 2024-08-30T09:16:00.836 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 147456 Aug 30 09:00 2619 2024-08-30T09:16:00.836 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 2619_fsm 2024-08-30T09:16:00.836 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2619_vm 2024-08-30T09:16:00.836 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 2620 2024-08-30T09:16:00.836 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2650 2024-08-30T09:16:00.837 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2651 2024-08-30T09:16:00.837 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2652 2024-08-30T09:16:00.837 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Aug 30 09:00 2653 2024-08-30T09:16:00.837 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Aug 30 09:00 2654 2024-08-30T09:16:00.837 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 30 09:00 2655 2024-08-30T09:16:00.837 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2656 2024-08-30T09:16:00.837 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2657 2024-08-30T09:16:00.837 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Aug 30 09:00 2658 2024-08-30T09:16:00.837 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Aug 30 09:00 2659 2024-08-30T09:16:00.838 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2660 2024-08-30T09:16:00.838 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2661 2024-08-30T09:16:00.838 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 30 09:00 2662 2024-08-30T09:16:00.838 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Aug 30 09:00 2663 2024-08-30T09:16:00.838 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2664 2024-08-30T09:16:00.838 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2665 2024-08-30T09:16:00.838 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2666 2024-08-30T09:16:00.838 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2667 2024-08-30T09:16:00.838 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2668 2024-08-30T09:16:00.838 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2669 2024-08-30T09:16:00.838 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2670 2024-08-30T09:16:00.838 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 Aug 30 09:00 2673 2024-08-30T09:16:00.838 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 Aug 30 09:00 2674 2024-08-30T09:16:00.838 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 Aug 30 09:00 2675 2024-08-30T09:16:00.838 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2678 2024-08-30T09:16:00.839 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2679 2024-08-30T09:16:00.839 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2680 2024-08-30T09:16:00.839 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2681 2024-08-30T09:16:00.839 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2682 2024-08-30T09:16:00.839 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2683 2024-08-30T09:16:00.839 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2684 2024-08-30T09:16:00.839 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2685 2024-08-30T09:16:00.839 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2686 2024-08-30T09:16:00.839 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2687 2024-08-30T09:16:00.839 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Aug 30 09:00 2688 2024-08-30T09:16:00.839 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Aug 30 09:00 2689 2024-08-30T09:16:00.839 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Aug 30 09:00 2690 2024-08-30T09:16:00.839 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 Aug 30 09:00 2691 2024-08-30T09:16:00.839 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2692 2024-08-30T09:16:00.839 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2693 2024-08-30T09:16:00.840 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2696 2024-08-30T09:16:00.840 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2699 2024-08-30T09:16:00.840 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2701 2024-08-30T09:16:00.840 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2702 2024-08-30T09:16:00.840 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 30 09:00 2703 2024-08-30T09:16:00.840 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Aug 30 09:00 2704 2024-08-30T09:16:00.840 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2753 2024-08-30T09:16:00.840 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 2753_fsm 2024-08-30T09:16:00.840 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2753_vm 2024-08-30T09:16:00.840 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2754 2024-08-30T09:16:00.840 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2755 2024-08-30T09:16:00.840 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 30 09:00 2756 2024-08-30T09:16:00.841 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 30 09:00 2757 2024-08-30T09:16:00.841 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 2830 2024-08-30T09:16:00.841 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2831 2024-08-30T09:16:00.841 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 2832 2024-08-30T09:16:00.841 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2833 2024-08-30T09:16:00.841 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 2834 2024-08-30T09:16:00.841 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2835 2024-08-30T09:16:00.841 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 2836 2024-08-30T09:16:00.841 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2837 2024-08-30T09:16:00.841 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 Aug 30 09:00 2838 2024-08-30T09:16:00.841 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 2838_fsm 2024-08-30T09:16:00.841 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2838_vm 2024-08-30T09:16:00.841 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2839 2024-08-30T09:16:00.841 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 2840 2024-08-30T09:16:00.842 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 2840_fsm 2024-08-30T09:16:00.842 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2840_vm 2024-08-30T09:16:00.842 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2841 2024-08-30T09:16:00.842 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 2995 2024-08-30T09:16:00.842 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2996 2024-08-30T09:16:00.842 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3079 2024-08-30T09:16:00.842 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 3079_fsm 2024-08-30T09:16:00.842 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3079_vm 2024-08-30T09:16:00.842 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 3080 2024-08-30T09:16:00.842 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 3081 2024-08-30T09:16:00.842 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Aug 30 09:00 3085 2024-08-30T09:16:00.842 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 3118 2024-08-30T09:16:00.842 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3119 2024-08-30T09:16:00.842 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Aug 30 09:00 3164 2024-08-30T09:16:00.842 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 3256 2024-08-30T09:16:00.843 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3257 2024-08-30T09:16:00.843 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3258 2024-08-30T09:16:00.843 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 3350 2024-08-30T09:16:00.843 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3351 2024-08-30T09:16:00.843 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3379 2024-08-30T09:16:00.843 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3380 2024-08-30T09:16:00.843 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 3381 2024-08-30T09:16:00.843 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 3394 2024-08-30T09:16:00.843 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 3394_fsm 2024-08-30T09:16:00.843 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3394_vm 2024-08-30T09:16:00.843 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 3395 2024-08-30T09:16:00.843 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 3429 2024-08-30T09:16:00.843 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 3430 2024-08-30T09:16:00.843 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3431 2024-08-30T09:16:00.844 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3433 2024-08-30T09:16:00.844 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 3439 2024-08-30T09:16:00.844 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3440 2024-08-30T09:16:00.844 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Aug 30 09:00 3455 2024-08-30T09:16:00.844 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 Aug 30 09:00 3456 2024-08-30T09:16:00.844 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 3456_fsm 2024-08-30T09:16:00.844 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3456_vm 2024-08-30T09:16:00.844 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 3466 2024-08-30T09:16:00.844 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3467 2024-08-30T09:16:00.844 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3468 2024-08-30T09:16:00.844 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 3501 2024-08-30T09:16:00.844 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3502 2024-08-30T09:16:00.845 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3503 2024-08-30T09:16:00.845 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3534 2024-08-30T09:16:00.845 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3541 2024-08-30T09:16:00.845 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 3541_fsm 2024-08-30T09:16:00.845 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3541_vm 2024-08-30T09:16:00.845 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 3542 2024-08-30T09:16:00.845 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3574 2024-08-30T09:16:00.845 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3575 2024-08-30T09:16:00.846 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 3576 2024-08-30T09:16:00.846 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 3596 2024-08-30T09:16:00.846 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3597 2024-08-30T09:16:00.846 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 3598 2024-08-30T09:16:00.846 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3599 2024-08-30T09:16:00.846 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3600 2024-08-30T09:16:00.846 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 3600_fsm 2024-08-30T09:16:00.846 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3600_vm 2024-08-30T09:16:00.846 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3601 2024-08-30T09:16:00.846 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 3601_fsm 2024-08-30T09:16:00.846 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3601_vm 2024-08-30T09:16:00.846 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3602 2024-08-30T09:16:00.846 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 3602_fsm 2024-08-30T09:16:00.846 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3602_vm 2024-08-30T09:16:00.847 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 3603 2024-08-30T09:16:00.847 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 3603_fsm 2024-08-30T09:16:00.847 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3603_vm 2024-08-30T09:16:00.847 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 3604 2024-08-30T09:16:00.847 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 3605 2024-08-30T09:16:00.847 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 3606 2024-08-30T09:16:00.847 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 3607 2024-08-30T09:16:00.847 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 3608 2024-08-30T09:16:00.847 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 30 09:00 3609 2024-08-30T09:16:00.847 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 3712 2024-08-30T09:16:00.847 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3764 2024-08-30T09:16:00.847 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 3764_fsm 2024-08-30T09:16:00.847 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3764_vm 2024-08-30T09:16:00.847 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 3766 2024-08-30T09:16:00.847 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 3767 2024-08-30T09:16:00.848 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3997 2024-08-30T09:16:00.848 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 4143 2024-08-30T09:16:00.848 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 4144 2024-08-30T09:16:00.848 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 4145 2024-08-30T09:16:00.848 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 4146 2024-08-30T09:16:00.848 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 4147 2024-08-30T09:16:00.848 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 4148 2024-08-30T09:16:00.848 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 4149 2024-08-30T09:16:00.848 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 4150 2024-08-30T09:16:00.848 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 4151 2024-08-30T09:16:00.848 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 4152 2024-08-30T09:16:00.848 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 4153 2024-08-30T09:16:00.848 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 4154 2024-08-30T09:16:00.848 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 4155 2024-08-30T09:16:00.848 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 4156 2024-08-30T09:16:00.849 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 4157 2024-08-30T09:16:00.849 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 4158 2024-08-30T09:16:00.849 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 4159 2024-08-30T09:16:00.849 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 4160 2024-08-30T09:16:00.849 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 4161 2024-08-30T09:16:00.849 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 4162 2024-08-30T09:16:00.849 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 4163 2024-08-30T09:16:00.849 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 4164 2024-08-30T09:16:00.849 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 4165 2024-08-30T09:16:00.849 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 4166 2024-08-30T09:16:00.849 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 4167 2024-08-30T09:16:00.849 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 4168 2024-08-30T09:16:00.849 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 4169 2024-08-30T09:16:00.850 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 4170 2024-08-30T09:16:00.850 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 4171 2024-08-30T09:16:00.850 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 4172 2024-08-30T09:16:00.852 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 4173 2024-08-30T09:16:00.852 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 4174 2024-08-30T09:16:00.852 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 5002 2024-08-30T09:16:00.852 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 548 2024-08-30T09:16:00.852 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 549 2024-08-30T09:16:00.853 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 6102 2024-08-30T09:16:00.853 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 6104 2024-08-30T09:16:00.853 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 6106 2024-08-30T09:16:00.853 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 6110 2024-08-30T09:16:00.853 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 6111 2024-08-30T09:16:00.853 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 6112 2024-08-30T09:16:00.853 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 6113 2024-08-30T09:16:00.853 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 6117 2024-08-30T09:16:00.853 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 826 2024-08-30T09:16:00.853 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 827 2024-08-30T09:16:00.853 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 828 2024-08-30T09:16:00.853 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Aug 30 09:00 pg_filenode.map 2024-08-30T09:16:00.853 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Aug 30 09:00 PG_VERSION 2024-08-30T09:16:00.854 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-08-30T09:16:00.854 INFO:teuthology.orchestra.run.smithi042.stdout:/tmp/cephfs/postgres/data/base/13494: 2024-08-30T09:16:00.854 INFO:teuthology.orchestra.run.smithi042.stdout:total 552310 2024-08-30T09:16:00.854 INFO:teuthology.orchestra.run.smithi042.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 305 Aug 30 09:07 . 2024-08-30T09:16:00.854 INFO:teuthology.orchestra.run.smithi042.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Aug 30 09:00 .. 2024-08-30T09:16:00.854 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 112 2024-08-30T09:16:00.854 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 113 2024-08-30T09:16:00.854 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Aug 30 09:00 1247 2024-08-30T09:16:00.854 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 1247_fsm 2024-08-30T09:16:00.854 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 1247_vm 2024-08-30T09:16:00.854 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 Aug 30 09:00 1249 2024-08-30T09:16:00.854 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 1249_fsm 2024-08-30T09:16:00.854 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 1249_vm 2024-08-30T09:16:00.854 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 Aug 30 09:00 1255 2024-08-30T09:16:00.854 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 1255_fsm 2024-08-30T09:16:00.855 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 1255_vm 2024-08-30T09:16:00.855 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Aug 30 09:15 1259 2024-08-30T09:16:00.855 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 1259_fsm 2024-08-30T09:16:00.855 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 1259_vm 2024-08-30T09:16:00.855 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 Aug 30 09:00 13344 2024-08-30T09:16:00.855 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 13344_fsm 2024-08-30T09:16:00.855 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 13344_vm 2024-08-30T09:16:00.855 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 13346 2024-08-30T09:16:00.855 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 13348 2024-08-30T09:16:00.855 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 13349 2024-08-30T09:16:00.856 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 13349_fsm 2024-08-30T09:16:00.856 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 13349_vm 2024-08-30T09:16:00.856 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 13351 2024-08-30T09:16:00.856 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 13353 2024-08-30T09:16:00.856 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 13354 2024-08-30T09:16:00.856 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 13354_fsm 2024-08-30T09:16:00.856 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 13354_vm 2024-08-30T09:16:00.856 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 13356 2024-08-30T09:16:00.856 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 13358 2024-08-30T09:16:00.856 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 13359 2024-08-30T09:16:00.856 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 13359_fsm 2024-08-30T09:16:00.856 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 13359_vm 2024-08-30T09:16:00.856 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 13361 2024-08-30T09:16:00.856 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 13363 2024-08-30T09:16:00.857 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 1417 2024-08-30T09:16:00.857 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 1418 2024-08-30T09:16:00.857 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 440320000 Aug 30 09:15 16396 2024-08-30T09:16:00.857 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 131072 Aug 30 09:15 16396_fsm 2024-08-30T09:16:00.857 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:09 16396_vm 2024-08-30T09:16:00.857 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 425984 Aug 30 09:15 16397 2024-08-30T09:16:00.857 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:15 16397_fsm 2024-08-30T09:16:00.857 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:15 16397_vm 2024-08-30T09:16:00.857 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 974848 Aug 30 09:15 16399 2024-08-30T09:16:00.857 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:15 16399_fsm 2024-08-30T09:16:00.857 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:15 16399_vm 2024-08-30T09:16:00.857 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:15 16400 2024-08-30T09:16:00.857 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Aug 30 09:15 16402 2024-08-30T09:16:00.857 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 71901184 Aug 30 09:15 16404 2024-08-30T09:16:00.857 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 43458560 Aug 30 09:15 16406 2024-08-30T09:16:00.858 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 30 09:15 16406_fsm 2024-08-30T09:16:00.858 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:10 16406_vm 2024-08-30T09:16:00.858 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 174 2024-08-30T09:16:00.858 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 175 2024-08-30T09:16:00.858 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2187 2024-08-30T09:16:00.858 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 2224 2024-08-30T09:16:00.858 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 2328 2024-08-30T09:16:00.858 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 2336 2024-08-30T09:16:00.858 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2337 2024-08-30T09:16:00.858 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2579 2024-08-30T09:16:00.858 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2600 2024-08-30T09:16:00.858 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 2600_fsm 2024-08-30T09:16:00.858 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2600_vm 2024-08-30T09:16:00.858 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2601 2024-08-30T09:16:00.859 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 2601_fsm 2024-08-30T09:16:00.859 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2601_vm 2024-08-30T09:16:00.859 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Aug 30 09:00 2602 2024-08-30T09:16:00.859 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 2602_fsm 2024-08-30T09:16:00.859 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2602_vm 2024-08-30T09:16:00.859 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 30 09:00 2603 2024-08-30T09:16:00.859 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 2603_fsm 2024-08-30T09:16:00.859 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2603_vm 2024-08-30T09:16:00.859 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 2604 2024-08-30T09:16:00.859 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2605 2024-08-30T09:16:00.859 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 2605_fsm 2024-08-30T09:16:00.860 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2605_vm 2024-08-30T09:16:00.860 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2606 2024-08-30T09:16:00.860 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 2606_fsm 2024-08-30T09:16:00.860 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2606_vm 2024-08-30T09:16:00.860 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2607 2024-08-30T09:16:00.860 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 2607_fsm 2024-08-30T09:16:00.860 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2607_vm 2024-08-30T09:16:00.860 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 Aug 30 09:00 2608 2024-08-30T09:16:00.860 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 2608_fsm 2024-08-30T09:16:00.860 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2608_vm 2024-08-30T09:16:00.860 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 Aug 30 09:00 2609 2024-08-30T09:16:00.860 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 2609_fsm 2024-08-30T09:16:00.861 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2609_vm 2024-08-30T09:16:00.861 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 30 09:00 2610 2024-08-30T09:16:00.861 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 2610_fsm 2024-08-30T09:16:00.861 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2610_vm 2024-08-30T09:16:00.861 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 2611 2024-08-30T09:16:00.861 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2612 2024-08-30T09:16:00.861 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 2612_fsm 2024-08-30T09:16:00.861 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2612_vm 2024-08-30T09:16:00.861 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 2613 2024-08-30T09:16:00.861 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2615 2024-08-30T09:16:00.861 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 2615_fsm 2024-08-30T09:16:00.861 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2615_vm 2024-08-30T09:16:00.861 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2616 2024-08-30T09:16:00.861 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 2616_fsm 2024-08-30T09:16:00.861 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2616_vm 2024-08-30T09:16:00.862 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 Aug 30 09:00 2617 2024-08-30T09:16:00.862 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 2617_fsm 2024-08-30T09:16:00.862 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2617_vm 2024-08-30T09:16:00.862 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Aug 30 09:00 2618 2024-08-30T09:16:00.862 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 2618_fsm 2024-08-30T09:16:00.862 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2618_vm 2024-08-30T09:16:00.862 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 180224 Aug 30 09:15 2619 2024-08-30T09:16:00.862 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:15 2619_fsm 2024-08-30T09:16:00.862 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:09 2619_vm 2024-08-30T09:16:00.862 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 2620 2024-08-30T09:16:00.862 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2650 2024-08-30T09:16:00.862 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2651 2024-08-30T09:16:00.862 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2652 2024-08-30T09:16:00.862 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Aug 30 09:00 2653 2024-08-30T09:16:00.863 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Aug 30 09:00 2654 2024-08-30T09:16:00.863 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 30 09:00 2655 2024-08-30T09:16:00.863 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2656 2024-08-30T09:16:00.863 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2657 2024-08-30T09:16:00.863 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Aug 30 09:00 2658 2024-08-30T09:16:00.863 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Aug 30 09:00 2659 2024-08-30T09:16:00.863 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2660 2024-08-30T09:16:00.863 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2661 2024-08-30T09:16:00.863 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 30 09:00 2662 2024-08-30T09:16:00.863 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Aug 30 09:00 2663 2024-08-30T09:16:00.863 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2664 2024-08-30T09:16:00.863 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2665 2024-08-30T09:16:00.863 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2666 2024-08-30T09:16:00.863 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2667 2024-08-30T09:16:00.864 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2668 2024-08-30T09:16:00.864 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2669 2024-08-30T09:16:00.864 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2670 2024-08-30T09:16:00.864 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 Aug 30 09:00 2673 2024-08-30T09:16:00.864 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 Aug 30 09:00 2674 2024-08-30T09:16:00.864 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 Aug 30 09:00 2675 2024-08-30T09:16:00.864 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2678 2024-08-30T09:16:00.864 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2679 2024-08-30T09:16:00.864 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2680 2024-08-30T09:16:00.864 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2681 2024-08-30T09:16:00.864 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2682 2024-08-30T09:16:00.865 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2683 2024-08-30T09:16:00.865 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2684 2024-08-30T09:16:00.865 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2685 2024-08-30T09:16:00.865 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2686 2024-08-30T09:16:00.865 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2687 2024-08-30T09:16:00.865 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Aug 30 09:00 2688 2024-08-30T09:16:00.865 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Aug 30 09:00 2689 2024-08-30T09:16:00.865 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Aug 30 09:00 2690 2024-08-30T09:16:00.865 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 Aug 30 09:00 2691 2024-08-30T09:16:00.865 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2692 2024-08-30T09:16:00.865 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2693 2024-08-30T09:16:00.866 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 30 09:15 2696 2024-08-30T09:16:00.866 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2699 2024-08-30T09:16:00.866 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2701 2024-08-30T09:16:00.866 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2702 2024-08-30T09:16:00.866 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 30 09:00 2703 2024-08-30T09:16:00.866 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Aug 30 09:00 2704 2024-08-30T09:16:00.866 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2753 2024-08-30T09:16:00.866 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 2753_fsm 2024-08-30T09:16:00.866 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2753_vm 2024-08-30T09:16:00.866 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2754 2024-08-30T09:16:00.866 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2755 2024-08-30T09:16:00.866 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 30 09:00 2756 2024-08-30T09:16:00.866 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 30 09:00 2757 2024-08-30T09:16:00.867 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 2830 2024-08-30T09:16:00.867 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2831 2024-08-30T09:16:00.867 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 2832 2024-08-30T09:16:00.867 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2833 2024-08-30T09:16:00.867 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 2834 2024-08-30T09:16:00.867 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2835 2024-08-30T09:16:00.867 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 2836 2024-08-30T09:16:00.867 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2837 2024-08-30T09:16:00.867 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 Aug 30 09:00 2838 2024-08-30T09:16:00.867 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 2838_fsm 2024-08-30T09:16:00.867 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2838_vm 2024-08-30T09:16:00.867 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2839 2024-08-30T09:16:00.867 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 2840 2024-08-30T09:16:00.867 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 2840_fsm 2024-08-30T09:16:00.867 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2840_vm 2024-08-30T09:16:00.868 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2841 2024-08-30T09:16:00.868 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 2995 2024-08-30T09:16:00.868 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2996 2024-08-30T09:16:00.868 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3079 2024-08-30T09:16:00.868 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 3079_fsm 2024-08-30T09:16:00.868 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3079_vm 2024-08-30T09:16:00.868 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 3080 2024-08-30T09:16:00.868 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 3081 2024-08-30T09:16:00.868 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Aug 30 09:00 3085 2024-08-30T09:16:00.868 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 3118 2024-08-30T09:16:00.868 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3119 2024-08-30T09:16:00.868 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Aug 30 09:00 3164 2024-08-30T09:16:00.868 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 3256 2024-08-30T09:16:00.868 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3257 2024-08-30T09:16:00.868 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3258 2024-08-30T09:16:00.869 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 3350 2024-08-30T09:16:00.869 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3351 2024-08-30T09:16:00.869 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3379 2024-08-30T09:16:00.869 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3380 2024-08-30T09:16:00.869 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 3381 2024-08-30T09:16:00.869 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 3394 2024-08-30T09:16:00.869 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 3394_fsm 2024-08-30T09:16:00.869 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3394_vm 2024-08-30T09:16:00.870 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 3395 2024-08-30T09:16:00.870 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 3429 2024-08-30T09:16:00.870 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 3430 2024-08-30T09:16:00.870 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3431 2024-08-30T09:16:00.870 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3433 2024-08-30T09:16:00.870 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 3439 2024-08-30T09:16:00.870 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3440 2024-08-30T09:16:00.870 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Aug 30 09:00 3455 2024-08-30T09:16:00.870 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 Aug 30 09:00 3456 2024-08-30T09:16:00.870 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 3456_fsm 2024-08-30T09:16:00.870 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3456_vm 2024-08-30T09:16:00.870 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 3466 2024-08-30T09:16:00.870 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3467 2024-08-30T09:16:00.870 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3468 2024-08-30T09:16:00.870 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 3501 2024-08-30T09:16:00.871 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3502 2024-08-30T09:16:00.871 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3503 2024-08-30T09:16:00.871 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3534 2024-08-30T09:16:00.871 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3541 2024-08-30T09:16:00.871 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 3541_fsm 2024-08-30T09:16:00.871 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3541_vm 2024-08-30T09:16:00.871 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 3542 2024-08-30T09:16:00.871 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3574 2024-08-30T09:16:00.871 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3575 2024-08-30T09:16:00.871 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 3576 2024-08-30T09:16:00.871 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 3596 2024-08-30T09:16:00.871 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3597 2024-08-30T09:16:00.871 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 3598 2024-08-30T09:16:00.871 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3599 2024-08-30T09:16:00.871 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3600 2024-08-30T09:16:00.872 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 3600_fsm 2024-08-30T09:16:00.872 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3600_vm 2024-08-30T09:16:00.872 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3601 2024-08-30T09:16:00.872 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 3601_fsm 2024-08-30T09:16:00.872 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3601_vm 2024-08-30T09:16:00.872 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3602 2024-08-30T09:16:00.872 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 3602_fsm 2024-08-30T09:16:00.872 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3602_vm 2024-08-30T09:16:00.872 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 3603 2024-08-30T09:16:00.872 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 3603_fsm 2024-08-30T09:16:00.872 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3603_vm 2024-08-30T09:16:00.872 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 3604 2024-08-30T09:16:00.872 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 3605 2024-08-30T09:16:00.872 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 3606 2024-08-30T09:16:00.873 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 3607 2024-08-30T09:16:00.873 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 3608 2024-08-30T09:16:00.873 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 30 09:00 3609 2024-08-30T09:16:00.873 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 3712 2024-08-30T09:16:00.873 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3764 2024-08-30T09:16:00.873 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 3764_fsm 2024-08-30T09:16:00.873 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3764_vm 2024-08-30T09:16:00.873 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 3766 2024-08-30T09:16:00.873 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 3767 2024-08-30T09:16:00.873 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3997 2024-08-30T09:16:00.873 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 4143 2024-08-30T09:16:00.873 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 4144 2024-08-30T09:16:00.873 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 4145 2024-08-30T09:16:00.873 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 4146 2024-08-30T09:16:00.873 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 4147 2024-08-30T09:16:00.874 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 4148 2024-08-30T09:16:00.874 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 4149 2024-08-30T09:16:00.874 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 4150 2024-08-30T09:16:00.874 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 4151 2024-08-30T09:16:00.874 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 4152 2024-08-30T09:16:00.874 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 4153 2024-08-30T09:16:00.874 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 4154 2024-08-30T09:16:00.874 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 4155 2024-08-30T09:16:00.874 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 4156 2024-08-30T09:16:00.874 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 4157 2024-08-30T09:16:00.874 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 4158 2024-08-30T09:16:00.874 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 4159 2024-08-30T09:16:00.874 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 4160 2024-08-30T09:16:00.874 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 4161 2024-08-30T09:16:00.875 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 4162 2024-08-30T09:16:00.875 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 4163 2024-08-30T09:16:00.875 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 4164 2024-08-30T09:16:00.875 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 4165 2024-08-30T09:16:00.875 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 4166 2024-08-30T09:16:00.875 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 4167 2024-08-30T09:16:00.875 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 4168 2024-08-30T09:16:00.875 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 4169 2024-08-30T09:16:00.875 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 4170 2024-08-30T09:16:00.875 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 4171 2024-08-30T09:16:00.875 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 4172 2024-08-30T09:16:00.875 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 4173 2024-08-30T09:16:00.875 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 4174 2024-08-30T09:16:00.875 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 5002 2024-08-30T09:16:00.875 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 548 2024-08-30T09:16:00.876 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 549 2024-08-30T09:16:00.876 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 6102 2024-08-30T09:16:00.876 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 6104 2024-08-30T09:16:00.876 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 6106 2024-08-30T09:16:00.876 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 6110 2024-08-30T09:16:00.876 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 6111 2024-08-30T09:16:00.876 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 6112 2024-08-30T09:16:00.876 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 6113 2024-08-30T09:16:00.876 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 6117 2024-08-30T09:16:00.876 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 826 2024-08-30T09:16:00.876 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 827 2024-08-30T09:16:00.876 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 828 2024-08-30T09:16:00.876 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Aug 30 09:00 pg_filenode.map 2024-08-30T09:16:00.876 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 151596 Aug 30 09:00 pg_internal.init 2024-08-30T09:16:00.877 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Aug 30 09:00 PG_VERSION 2024-08-30T09:16:00.877 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-08-30T09:16:00.877 INFO:teuthology.orchestra.run.smithi042.stdout:/tmp/cephfs/postgres/data/base/pgsql_tmp: 2024-08-30T09:16:00.877 INFO:teuthology.orchestra.run.smithi042.stdout:total 0 2024-08-30T09:16:00.877 INFO:teuthology.orchestra.run.smithi042.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 . 2024-08-30T09:16:00.877 INFO:teuthology.orchestra.run.smithi042.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Aug 30 09:00 .. 2024-08-30T09:16:00.877 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-08-30T09:16:00.877 INFO:teuthology.orchestra.run.smithi042.stdout:/tmp/cephfs/postgres/data/global: 2024-08-30T09:16:00.877 INFO:teuthology.orchestra.run.smithi042.stdout:total 560 2024-08-30T09:16:00.877 INFO:teuthology.orchestra.run.smithi042.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 58 Aug 30 09:00 . 2024-08-30T09:16:00.877 INFO:teuthology.orchestra.run.smithi042.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Aug 30 09:16 .. 2024-08-30T09:16:00.877 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 1213 2024-08-30T09:16:00.877 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 1213_fsm 2024-08-30T09:16:00.877 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 1213_vm 2024-08-30T09:16:00.877 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 1214 2024-08-30T09:16:00.878 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 1214_fsm 2024-08-30T09:16:00.878 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 1214_vm 2024-08-30T09:16:00.878 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 1232 2024-08-30T09:16:00.878 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 1233 2024-08-30T09:16:00.878 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 1260 2024-08-30T09:16:00.878 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 1260_fsm 2024-08-30T09:16:00.878 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 1260_vm 2024-08-30T09:16:00.878 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 1261 2024-08-30T09:16:00.878 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 1261_fsm 2024-08-30T09:16:00.878 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 1261_vm 2024-08-30T09:16:00.878 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 1262 2024-08-30T09:16:00.879 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 1262_fsm 2024-08-30T09:16:00.879 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 1262_vm 2024-08-30T09:16:00.879 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2396 2024-08-30T09:16:00.879 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 30 09:00 2396_fsm 2024-08-30T09:16:00.879 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2396_vm 2024-08-30T09:16:00.879 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2397 2024-08-30T09:16:00.879 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2671 2024-08-30T09:16:00.879 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2672 2024-08-30T09:16:00.879 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2676 2024-08-30T09:16:00.879 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2677 2024-08-30T09:16:00.879 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2694 2024-08-30T09:16:00.879 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2695 2024-08-30T09:16:00.879 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2697 2024-08-30T09:16:00.879 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 30 09:00 2698 2024-08-30T09:16:00.880 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 2846 2024-08-30T09:16:00.880 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2847 2024-08-30T09:16:00.880 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 2964 2024-08-30T09:16:00.880 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2965 2024-08-30T09:16:00.880 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 2966 2024-08-30T09:16:00.880 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 2967 2024-08-30T09:16:00.880 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 3592 2024-08-30T09:16:00.880 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 3593 2024-08-30T09:16:00.880 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 4060 2024-08-30T09:16:00.880 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 4061 2024-08-30T09:16:00.880 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 4175 2024-08-30T09:16:00.880 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 4176 2024-08-30T09:16:00.880 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 4177 2024-08-30T09:16:00.880 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 4178 2024-08-30T09:16:00.881 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 4181 2024-08-30T09:16:00.881 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 4182 2024-08-30T09:16:00.881 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 4183 2024-08-30T09:16:00.881 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 4184 2024-08-30T09:16:00.881 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 4185 2024-08-30T09:16:00.881 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 4186 2024-08-30T09:16:00.881 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 6000 2024-08-30T09:16:00.881 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 6001 2024-08-30T09:16:00.881 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 6002 2024-08-30T09:16:00.881 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 6100 2024-08-30T09:16:00.881 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 6114 2024-08-30T09:16:00.881 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 6115 2024-08-30T09:16:00.881 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:16 pg_control 2024-08-30T09:16:00.881 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Aug 30 09:00 pg_filenode.map 2024-08-30T09:16:00.881 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 23144 Aug 30 09:00 pg_internal.init 2024-08-30T09:16:00.882 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-08-30T09:16:00.882 INFO:teuthology.orchestra.run.smithi042.stdout:/tmp/cephfs/postgres/data/log: 2024-08-30T09:16:00.882 INFO:teuthology.orchestra.run.smithi042.stdout:total 2 2024-08-30T09:16:00.882 INFO:teuthology.orchestra.run.smithi042.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Aug 30 09:00 . 2024-08-30T09:16:00.882 INFO:teuthology.orchestra.run.smithi042.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Aug 30 09:16 .. 2024-08-30T09:16:00.882 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1222 Aug 30 09:16 postgresql-Fri.log 2024-08-30T09:16:00.882 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-08-30T09:16:00.882 INFO:teuthology.orchestra.run.smithi042.stdout:/tmp/cephfs/postgres/data/pg_commit_ts: 2024-08-30T09:16:00.882 INFO:teuthology.orchestra.run.smithi042.stdout:total 0 2024-08-30T09:16:00.882 INFO:teuthology.orchestra.run.smithi042.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 . 2024-08-30T09:16:00.882 INFO:teuthology.orchestra.run.smithi042.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Aug 30 09:16 .. 2024-08-30T09:16:00.882 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-08-30T09:16:00.882 INFO:teuthology.orchestra.run.smithi042.stdout:/tmp/cephfs/postgres/data/pg_dynshmem: 2024-08-30T09:16:00.882 INFO:teuthology.orchestra.run.smithi042.stdout:total 0 2024-08-30T09:16:00.882 INFO:teuthology.orchestra.run.smithi042.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 . 2024-08-30T09:16:00.883 INFO:teuthology.orchestra.run.smithi042.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Aug 30 09:16 .. 2024-08-30T09:16:00.883 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-08-30T09:16:00.883 INFO:teuthology.orchestra.run.smithi042.stdout:/tmp/cephfs/postgres/data/pg_logical: 2024-08-30T09:16:00.883 INFO:teuthology.orchestra.run.smithi042.stdout:total 1 2024-08-30T09:16:00.883 INFO:teuthology.orchestra.run.smithi042.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Aug 30 09:16 . 2024-08-30T09:16:00.883 INFO:teuthology.orchestra.run.smithi042.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Aug 30 09:16 .. 2024-08-30T09:16:00.883 INFO:teuthology.orchestra.run.smithi042.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 mappings 2024-08-30T09:16:00.883 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8 Aug 30 09:16 replorigin_checkpoint 2024-08-30T09:16:00.883 INFO:teuthology.orchestra.run.smithi042.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 snapshots 2024-08-30T09:16:00.883 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-08-30T09:16:00.883 INFO:teuthology.orchestra.run.smithi042.stdout:/tmp/cephfs/postgres/data/pg_logical/mappings: 2024-08-30T09:16:00.883 INFO:teuthology.orchestra.run.smithi042.stdout:total 0 2024-08-30T09:16:00.883 INFO:teuthology.orchestra.run.smithi042.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 . 2024-08-30T09:16:00.883 INFO:teuthology.orchestra.run.smithi042.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Aug 30 09:16 .. 2024-08-30T09:16:00.883 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-08-30T09:16:00.884 INFO:teuthology.orchestra.run.smithi042.stdout:/tmp/cephfs/postgres/data/pg_logical/snapshots: 2024-08-30T09:16:00.884 INFO:teuthology.orchestra.run.smithi042.stdout:total 0 2024-08-30T09:16:00.884 INFO:teuthology.orchestra.run.smithi042.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 . 2024-08-30T09:16:00.884 INFO:teuthology.orchestra.run.smithi042.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Aug 30 09:16 .. 2024-08-30T09:16:00.884 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-08-30T09:16:00.884 INFO:teuthology.orchestra.run.smithi042.stdout:/tmp/cephfs/postgres/data/pg_multixact: 2024-08-30T09:16:00.884 INFO:teuthology.orchestra.run.smithi042.stdout:total 0 2024-08-30T09:16:00.884 INFO:teuthology.orchestra.run.smithi042.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Aug 30 09:00 . 2024-08-30T09:16:00.884 INFO:teuthology.orchestra.run.smithi042.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Aug 30 09:16 .. 2024-08-30T09:16:00.884 INFO:teuthology.orchestra.run.smithi042.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Aug 30 09:00 members 2024-08-30T09:16:00.884 INFO:teuthology.orchestra.run.smithi042.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Aug 30 09:00 offsets 2024-08-30T09:16:00.884 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-08-30T09:16:00.884 INFO:teuthology.orchestra.run.smithi042.stdout:/tmp/cephfs/postgres/data/pg_multixact/members: 2024-08-30T09:16:00.884 INFO:teuthology.orchestra.run.smithi042.stdout:total 8 2024-08-30T09:16:00.885 INFO:teuthology.orchestra.run.smithi042.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Aug 30 09:00 . 2024-08-30T09:16:00.885 INFO:teuthology.orchestra.run.smithi042.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Aug 30 09:00 .. 2024-08-30T09:16:00.885 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:00 0000 2024-08-30T09:16:00.885 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-08-30T09:16:00.885 INFO:teuthology.orchestra.run.smithi042.stdout:/tmp/cephfs/postgres/data/pg_multixact/offsets: 2024-08-30T09:16:00.885 INFO:teuthology.orchestra.run.smithi042.stdout:total 8 2024-08-30T09:16:00.885 INFO:teuthology.orchestra.run.smithi042.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Aug 30 09:00 . 2024-08-30T09:16:00.885 INFO:teuthology.orchestra.run.smithi042.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Aug 30 09:00 .. 2024-08-30T09:16:00.885 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 30 09:05 0000 2024-08-30T09:16:00.885 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-08-30T09:16:00.885 INFO:teuthology.orchestra.run.smithi042.stdout:/tmp/cephfs/postgres/data/pg_notify: 2024-08-30T09:16:00.885 INFO:teuthology.orchestra.run.smithi042.stdout:total 0 2024-08-30T09:16:00.885 INFO:teuthology.orchestra.run.smithi042.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 . 2024-08-30T09:16:00.885 INFO:teuthology.orchestra.run.smithi042.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Aug 30 09:16 .. 2024-08-30T09:16:00.885 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-08-30T09:16:00.886 INFO:teuthology.orchestra.run.smithi042.stdout:/tmp/cephfs/postgres/data/pg_replslot: 2024-08-30T09:16:00.886 INFO:teuthology.orchestra.run.smithi042.stdout:total 0 2024-08-30T09:16:00.886 INFO:teuthology.orchestra.run.smithi042.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 . 2024-08-30T09:16:00.886 INFO:teuthology.orchestra.run.smithi042.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Aug 30 09:16 .. 2024-08-30T09:16:00.886 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-08-30T09:16:00.886 INFO:teuthology.orchestra.run.smithi042.stdout:/tmp/cephfs/postgres/data/pg_serial: 2024-08-30T09:16:00.886 INFO:teuthology.orchestra.run.smithi042.stdout:total 0 2024-08-30T09:16:00.886 INFO:teuthology.orchestra.run.smithi042.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 . 2024-08-30T09:16:00.886 INFO:teuthology.orchestra.run.smithi042.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Aug 30 09:16 .. 2024-08-30T09:16:00.886 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-08-30T09:16:00.886 INFO:teuthology.orchestra.run.smithi042.stdout:/tmp/cephfs/postgres/data/pg_snapshots: 2024-08-30T09:16:00.886 INFO:teuthology.orchestra.run.smithi042.stdout:total 0 2024-08-30T09:16:00.887 INFO:teuthology.orchestra.run.smithi042.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 . 2024-08-30T09:16:00.887 INFO:teuthology.orchestra.run.smithi042.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Aug 30 09:16 .. 2024-08-30T09:16:00.887 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-08-30T09:16:00.887 INFO:teuthology.orchestra.run.smithi042.stdout:/tmp/cephfs/postgres/data/pg_stat: 2024-08-30T09:16:00.887 INFO:teuthology.orchestra.run.smithi042.stdout:total 15 2024-08-30T09:16:00.887 INFO:teuthology.orchestra.run.smithi042.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Aug 30 09:16 . 2024-08-30T09:16:00.888 INFO:teuthology.orchestra.run.smithi042.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Aug 30 09:16 .. 2024-08-30T09:16:00.888 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1952 Aug 30 09:16 db_0.stat 2024-08-30T09:16:00.888 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 10979 Aug 30 09:16 db_13494.stat 2024-08-30T09:16:00.888 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1151 Aug 30 09:16 global.stat 2024-08-30T09:16:00.888 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-08-30T09:16:00.888 INFO:teuthology.orchestra.run.smithi042.stdout:/tmp/cephfs/postgres/data/pg_stat_tmp: 2024-08-30T09:16:00.888 INFO:teuthology.orchestra.run.smithi042.stdout:total 0 2024-08-30T09:16:00.888 INFO:teuthology.orchestra.run.smithi042.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:16 . 2024-08-30T09:16:00.888 INFO:teuthology.orchestra.run.smithi042.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Aug 30 09:16 .. 2024-08-30T09:16:00.889 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-08-30T09:16:00.889 INFO:teuthology.orchestra.run.smithi042.stdout:/tmp/cephfs/postgres/data/pg_subtrans: 2024-08-30T09:16:00.889 INFO:teuthology.orchestra.run.smithi042.stdout:total 168 2024-08-30T09:16:00.889 INFO:teuthology.orchestra.run.smithi042.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Aug 30 09:15 . 2024-08-30T09:16:00.889 INFO:teuthology.orchestra.run.smithi042.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Aug 30 09:16 .. 2024-08-30T09:16:00.889 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 172032 Aug 30 09:15 000C 2024-08-30T09:16:00.889 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-08-30T09:16:00.889 INFO:teuthology.orchestra.run.smithi042.stdout:/tmp/cephfs/postgres/data/pg_tblspc: 2024-08-30T09:16:00.889 INFO:teuthology.orchestra.run.smithi042.stdout:total 0 2024-08-30T09:16:00.889 INFO:teuthology.orchestra.run.smithi042.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 . 2024-08-30T09:16:00.889 INFO:teuthology.orchestra.run.smithi042.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Aug 30 09:16 .. 2024-08-30T09:16:00.889 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-08-30T09:16:00.889 INFO:teuthology.orchestra.run.smithi042.stdout:/tmp/cephfs/postgres/data/pg_twophase: 2024-08-30T09:16:00.890 INFO:teuthology.orchestra.run.smithi042.stdout:total 0 2024-08-30T09:16:00.890 INFO:teuthology.orchestra.run.smithi042.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 . 2024-08-30T09:16:00.890 INFO:teuthology.orchestra.run.smithi042.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Aug 30 09:16 .. 2024-08-30T09:16:00.890 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-08-30T09:16:00.890 INFO:teuthology.orchestra.run.smithi042.stdout:/tmp/cephfs/postgres/data/pg_wal: 2024-08-30T09:16:00.890 INFO:teuthology.orchestra.run.smithi042.stdout:total 966656 2024-08-30T09:16:00.890 INFO:teuthology.orchestra.run.smithi042.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 60 Aug 30 09:16 . 2024-08-30T09:16:00.890 INFO:teuthology.orchestra.run.smithi042.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Aug 30 09:16 .. 2024-08-30T09:16:00.890 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 30 09:16 000000010000000000000065 2024-08-30T09:16:00.890 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 30 09:05 000000010000000000000066 2024-08-30T09:16:00.890 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 30 09:07 000000010000000000000067 2024-08-30T09:16:00.890 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 30 09:06 000000010000000000000068 2024-08-30T09:16:00.890 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 30 09:05 000000010000000000000069 2024-08-30T09:16:00.890 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 30 09:08 00000001000000000000006A 2024-08-30T09:16:00.891 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 30 09:06 00000001000000000000006B 2024-08-30T09:16:00.891 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 30 09:07 00000001000000000000006C 2024-08-30T09:16:00.891 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 30 09:06 00000001000000000000006D 2024-08-30T09:16:00.891 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 30 09:06 00000001000000000000006E 2024-08-30T09:16:00.891 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 30 09:08 00000001000000000000006F 2024-08-30T09:16:00.891 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 30 09:09 000000010000000000000070 2024-08-30T09:16:00.891 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 30 09:06 000000010000000000000071 2024-08-30T09:16:00.891 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 30 09:06 000000010000000000000072 2024-08-30T09:16:00.891 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 30 09:08 000000010000000000000073 2024-08-30T09:16:00.891 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 30 09:07 000000010000000000000074 2024-08-30T09:16:00.891 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 30 09:07 000000010000000000000075 2024-08-30T09:16:00.891 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 30 09:05 000000010000000000000076 2024-08-30T09:16:00.891 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 30 09:05 000000010000000000000077 2024-08-30T09:16:00.891 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 30 09:06 000000010000000000000078 2024-08-30T09:16:00.891 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 30 09:06 000000010000000000000079 2024-08-30T09:16:00.892 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 30 09:09 00000001000000000000007A 2024-08-30T09:16:00.892 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 30 09:05 00000001000000000000007B 2024-08-30T09:16:00.892 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 30 09:06 00000001000000000000007C 2024-08-30T09:16:00.892 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 30 09:06 00000001000000000000007D 2024-08-30T09:16:00.892 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 30 09:05 00000001000000000000007E 2024-08-30T09:16:00.892 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 30 09:07 00000001000000000000007F 2024-08-30T09:16:00.892 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 30 09:10 000000010000000000000080 2024-08-30T09:16:00.892 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 30 09:06 000000010000000000000081 2024-08-30T09:16:00.892 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 30 09:06 000000010000000000000082 2024-08-30T09:16:00.892 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 30 09:05 000000010000000000000083 2024-08-30T09:16:00.892 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 30 09:11 000000010000000000000084 2024-08-30T09:16:00.892 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 30 09:12 000000010000000000000085 2024-08-30T09:16:00.892 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 30 09:11 000000010000000000000086 2024-08-30T09:16:00.893 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 30 09:12 000000010000000000000087 2024-08-30T09:16:00.893 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 30 09:12 000000010000000000000088 2024-08-30T09:16:00.893 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 30 09:15 000000010000000000000089 2024-08-30T09:16:00.893 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 30 09:11 00000001000000000000008A 2024-08-30T09:16:00.893 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 30 09:11 00000001000000000000008B 2024-08-30T09:16:00.893 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 30 09:14 00000001000000000000008C 2024-08-30T09:16:00.893 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 30 09:13 00000001000000000000008D 2024-08-30T09:16:00.893 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 30 09:11 00000001000000000000008E 2024-08-30T09:16:00.893 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 30 09:14 00000001000000000000008F 2024-08-30T09:16:00.893 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 30 09:11 000000010000000000000090 2024-08-30T09:16:00.893 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 30 09:12 000000010000000000000091 2024-08-30T09:16:00.893 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 30 09:10 000000010000000000000092 2024-08-30T09:16:00.893 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 30 09:12 000000010000000000000093 2024-08-30T09:16:00.893 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 30 09:11 000000010000000000000094 2024-08-30T09:16:00.894 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 30 09:12 000000010000000000000095 2024-08-30T09:16:00.894 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 30 09:13 000000010000000000000096 2024-08-30T09:16:00.894 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 30 09:11 000000010000000000000097 2024-08-30T09:16:00.894 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 30 09:10 000000010000000000000098 2024-08-30T09:16:00.894 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 30 09:11 000000010000000000000099 2024-08-30T09:16:00.894 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 30 09:11 00000001000000000000009A 2024-08-30T09:16:00.894 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 30 09:14 00000001000000000000009B 2024-08-30T09:16:00.894 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 30 09:11 00000001000000000000009C 2024-08-30T09:16:00.894 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 30 09:10 00000001000000000000009D 2024-08-30T09:16:00.894 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 30 09:10 00000001000000000000009E 2024-08-30T09:16:00.894 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 30 09:13 00000001000000000000009F 2024-08-30T09:16:00.894 INFO:teuthology.orchestra.run.smithi042.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 archive_status 2024-08-30T09:16:00.894 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-08-30T09:16:00.895 INFO:teuthology.orchestra.run.smithi042.stdout:/tmp/cephfs/postgres/data/pg_wal/archive_status: 2024-08-30T09:16:00.895 INFO:teuthology.orchestra.run.smithi042.stdout:total 0 2024-08-30T09:16:00.895 INFO:teuthology.orchestra.run.smithi042.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 30 09:00 . 2024-08-30T09:16:00.895 INFO:teuthology.orchestra.run.smithi042.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 60 Aug 30 09:16 .. 2024-08-30T09:16:00.895 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-08-30T09:16:00.895 INFO:teuthology.orchestra.run.smithi042.stdout:/tmp/cephfs/postgres/data/pg_xact: 2024-08-30T09:16:00.895 INFO:teuthology.orchestra.run.smithi042.stdout:total 208 2024-08-30T09:16:00.895 INFO:teuthology.orchestra.run.smithi042.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Aug 30 09:00 . 2024-08-30T09:16:00.895 INFO:teuthology.orchestra.run.smithi042.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Aug 30 09:16 .. 2024-08-30T09:16:00.895 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 212992 Aug 30 09:15 0000 2024-08-30T09:16:00.895 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-08-30T09:16:00.906 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi042.front.sepia.ceph.com 2024-08-30T09:16:00.914 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-08-30T09:16:00.924 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-08-30T09:16:01.197 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-08-30T09:16:01.197 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph fs dump --format=json 2024-08-30T09:16:01.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:16:01 smithi167 ceph-mon[22063]: pgmap v830: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 7.6 MiB/s wr, 789 op/s 2024-08-30T09:16:01.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:16:01 smithi042 ceph-mon[18652]: pgmap v830: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 7.6 MiB/s wr, 789 op/s 2024-08-30T09:16:01.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:16:01 smithi190 ceph-mon[22001]: pgmap v830: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 7.6 MiB/s wr, 789 op/s 2024-08-30T09:16:01.463 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T09:16:01.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:01.962+0000 7f6267386640 1 -- 172.21.15.42:0/3656572839 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f62601023f0 msgr2=0x7f6260102800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:16:01.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:01.962+0000 7f6267386640 1 --2- 172.21.15.42:0/3656572839 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f62601023f0 0x7f6260102800 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f625c009900 tx=0x7f625c02a480 comp rx=0 tx=0).stop 2024-08-30T09:16:01.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:01.962+0000 7f6267386640 1 -- 172.21.15.42:0/3656572839 shutdown_connections 2024-08-30T09:16:01.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:01.962+0000 7f6267386640 1 --2- 172.21.15.42:0/3656572839 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f626010ffa0 0x7f62601123c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:01.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:01.962+0000 7f6267386640 1 --2- 172.21.15.42:0/3656572839 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6260102d40 0x7f626010fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:01.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:01.962+0000 7f6267386640 1 --2- 172.21.15.42:0/3656572839 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f62601023f0 0x7f6260102800 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:01.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:01.962+0000 7f6267386640 1 -- 172.21.15.42:0/3656572839 >> 172.21.15.42:0/3656572839 conn(0x7f6260079510 msgr2=0x7f6260079920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:16:01.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:01.963+0000 7f6267386640 1 -- 172.21.15.42:0/3656572839 shutdown_connections 2024-08-30T09:16:01.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:01.963+0000 7f6267386640 1 -- 172.21.15.42:0/3656572839 wait complete. 2024-08-30T09:16:01.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:01.963+0000 7f6267386640 1 Processor -- start 2024-08-30T09:16:01.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:01.963+0000 7f6267386640 1 -- start start 2024-08-30T09:16:01.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:01.964+0000 7f6267386640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f62601023f0 0x7f626019c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:16:01.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:01.964+0000 7f6267386640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6260102d40 0x7f626019caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:16:01.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:01.964+0000 7f6267386640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f626010ffa0 0x7f62601a1d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:16:01.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:01.964+0000 7f6267386640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f6260115a90 con 0x7f6260102d40 2024-08-30T09:16:01.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:01.964+0000 7f6267386640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f6260115910 con 0x7f626010ffa0 2024-08-30T09:16:01.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:01.964+0000 7f6267386640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f6260115c10 con 0x7f62601023f0 2024-08-30T09:16:01.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:01.964+0000 7f6264921640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6260102d40 0x7f626019caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:16:01.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:01.964+0000 7f6264921640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6260102d40 0x7f626019caa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:32896/0 (socket says 172.21.15.42:32896) 2024-08-30T09:16:01.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:01.964+0000 7f6264921640 1 -- 172.21.15.42:0/2260263486 learned_addr learned my addr 172.21.15.42:0/2260263486 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:16:01.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:01.964+0000 7f6265122640 1 --2- 172.21.15.42:0/2260263486 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f62601023f0 0x7f626019c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:16:01.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:01.964+0000 7f6264921640 1 -- 172.21.15.42:0/2260263486 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f62601023f0 msgr2=0x7f626019c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:16:01.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:01.964+0000 7f6264921640 1 --2- 172.21.15.42:0/2260263486 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f62601023f0 0x7f626019c500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:01.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:01.964+0000 7f6264921640 1 -- 172.21.15.42:0/2260263486 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f626010ffa0 msgr2=0x7f62601a1d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:16:01.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:01.964+0000 7f6264921640 1 --2- 172.21.15.42:0/2260263486 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f626010ffa0 0x7f62601a1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:01.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:01.964+0000 7f6264921640 1 -- 172.21.15.42:0/2260263486 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62601a2470 con 0x7f6260102d40 2024-08-30T09:16:01.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:01.964+0000 7f6264921640 1 --2- 172.21.15.42:0/2260263486 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6260102d40 0x7f626019caa0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f625000d950 tx=0x7f625000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:16:01.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:01.964+0000 7f62567fc640 1 -- 172.21.15.42:0/2260263486 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6250017070 con 0x7f6260102d40 2024-08-30T09:16:01.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:01.964+0000 7f62567fc640 1 -- 172.21.15.42:0/2260263486 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6250009c30 con 0x7f6260102d40 2024-08-30T09:16:01.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:01.964+0000 7f6267386640 1 -- 172.21.15.42:0/2260263486 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62601a2760 con 0x7f6260102d40 2024-08-30T09:16:01.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:01.964+0000 7f62567fc640 1 -- 172.21.15.42:0/2260263486 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f625000f620 con 0x7f6260102d40 2024-08-30T09:16:01.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:01.964+0000 7f6267386640 1 -- 172.21.15.42:0/2260263486 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62601a8ec0 con 0x7f6260102d40 2024-08-30T09:16:01.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:01.965+0000 7f6267386640 1 -- 172.21.15.42:0/2260263486 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f626006c000 con 0x7f6260102d40 2024-08-30T09:16:01.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:01.967+0000 7f62567fc640 1 -- 172.21.15.42:0/2260263486 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7f6250005030 con 0x7f6260102d40 2024-08-30T09:16:01.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:01.968+0000 7f62567fc640 1 --2- 172.21.15.42:0/2260263486 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f624c0b4b30 0x7f624c0b7040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:16:01.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:01.968+0000 7f6265122640 1 --2- 172.21.15.42:0/2260263486 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f624c0b4b30 0x7f624c0b7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:16:01.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:01.968+0000 7f62567fc640 1 -- 172.21.15.42:0/2260263486 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f62500e51c0 con 0x7f6260102d40 2024-08-30T09:16:01.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:01.968+0000 7f6265122640 1 --2- 172.21.15.42:0/2260263486 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f624c0b4b30 0x7f624c0b7040 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f625c031440 tx=0x7f625c005d50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:16:01.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:01.971+0000 7f62567fc640 1 -- 172.21.15.42:0/2260263486 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f62500abf60 con 0x7f6260102d40 2024-08-30T09:16:02.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:02.195+0000 7f6267386640 1 -- 172.21.15.42:0/2260263486 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6260102800 con 0x7f6260102d40 2024-08-30T09:16:02.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:02.195+0000 7f62567fc640 1 -- 172.21.15.42:0/2260263486 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) ==== 94+0+11918 (secure 0 0 0) 0x7f6250009de0 con 0x7f6260102d40 2024-08-30T09:16:02.198 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-08-30T09:16:02.199 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":49,"btime":"2024-08-30T09:15:48:971024+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14703,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/3188400000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":3188400000},{"type":"v1","addr":"172.21.15.42:6835","nonce":3188400000}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14715,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/3356537674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":3356537674},{"type":"v1","addr":"172.21.15.42:6837","nonce":3356537674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14766,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6839/2107785513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6838","nonce":2107785513},{"type":"v1","addr":"172.21.15.42:6839","nonce":2107785513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14778,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6841/2298438674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6840","nonce":2298438674},{"type":"v1","addr":"172.21.15.42:6841","nonce":2298438674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24553,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/543555570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":543555570},{"type":"v1","addr":"172.21.15.190:6833","nonce":543555570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24554,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/3907818941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":3907818941},{"type":"v1","addr":"172.21.15.167:6833","nonce":3907818941}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":49,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-30T08:58:03.055084+0000","modified":"2024-08-30T09:15:48.340515+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24629,"mds_1":24610,"mds_2":24599},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24572":{"gid":24572,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6835/234596845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":234596845},{"type":"v1","addr":"172.21.15.167:6835","nonce":234596845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24592":{"gid":24592,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6835/247905709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":247905709},{"type":"v1","addr":"172.21.15.190:6835","nonce":247905709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24598":{"gid":24598,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6837/863256349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6836","nonce":863256349},{"type":"v1","addr":"172.21.15.190:6837","nonce":863256349}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24599":{"gid":24599,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.167:6837/3011837856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":3011837856},{"type":"v1","addr":"172.21.15.167:6837","nonce":3011837856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24610":{"gid":24610,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.190:6839/719981110","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6838","nonce":719981110},{"type":"v1","addr":"172.21.15.190:6839","nonce":719981110}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24629":{"gid":24629,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.167:6839/3604645207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":3604645207},{"type":"v1","addr":"172.21.15.167:6839","nonce":3604645207}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24629,"qdb_cluster":[24599,24610,24629]},"id":1}]} 2024-08-30T09:16:02.199 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 49 2024-08-30T09:16:02.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:02.197+0000 7f6267386640 1 -- 172.21.15.42:0/2260263486 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f624c0b4b30 msgr2=0x7f624c0b7040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:16:02.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:02.197+0000 7f6267386640 1 --2- 172.21.15.42:0/2260263486 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f624c0b4b30 0x7f624c0b7040 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f625c031440 tx=0x7f625c005d50 comp rx=0 tx=0).stop 2024-08-30T09:16:02.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:02.197+0000 7f6267386640 1 -- 172.21.15.42:0/2260263486 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6260102d40 msgr2=0x7f626019caa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:16:02.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:02.197+0000 7f6267386640 1 --2- 172.21.15.42:0/2260263486 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6260102d40 0x7f626019caa0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f625000d950 tx=0x7f625000de10 comp rx=0 tx=0).stop 2024-08-30T09:16:02.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:02.197+0000 7f6267386640 1 -- 172.21.15.42:0/2260263486 shutdown_connections 2024-08-30T09:16:02.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:02.197+0000 7f6267386640 1 --2- 172.21.15.42:0/2260263486 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f624c0b4b30 0x7f624c0b7040 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:02.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:02.197+0000 7f6267386640 1 --2- 172.21.15.42:0/2260263486 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f626010ffa0 0x7f62601a1d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:02.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:02.197+0000 7f6267386640 1 --2- 172.21.15.42:0/2260263486 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6260102d40 0x7f626019caa0 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:02.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:02.197+0000 7f6267386640 1 --2- 172.21.15.42:0/2260263486 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f62601023f0 0x7f626019c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:02.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:02.197+0000 7f6267386640 1 -- 172.21.15.42:0/2260263486 >> 172.21.15.42:0/2260263486 conn(0x7f6260079510 msgr2=0x7f6260109bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:16:02.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:02.198+0000 7f6267386640 1 -- 172.21.15.42:0/2260263486 shutdown_connections 2024-08-30T09:16:02.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:02.198+0000 7f6267386640 1 -- 172.21.15.42:0/2260263486 wait complete. 2024-08-30T09:16:02.865 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-08-30T09:16:03.120 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T09:16:03.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:16:03 smithi042 ceph-mon[18652]: pgmap v831: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 5.0 MiB/s wr, 580 op/s 2024-08-30T09:16:03.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:16:03 smithi042 ceph-mon[18652]: from='client.? 172.21.15.42:0/2260263486' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-30T09:16:03.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:16:03 smithi167 ceph-mon[22063]: pgmap v831: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 5.0 MiB/s wr, 580 op/s 2024-08-30T09:16:03.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:16:03 smithi167 ceph-mon[22063]: from='client.? 172.21.15.42:0/2260263486' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-30T09:16:03.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:16:03 smithi190 ceph-mon[22001]: pgmap v831: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 5.0 MiB/s wr, 580 op/s 2024-08-30T09:16:03.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:16:03 smithi190 ceph-mon[22001]: from='client.? 172.21.15.42:0/2260263486' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-30T09:16:03.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.616+0000 7f9c3c9cb640 1 -- 172.21.15.42:0/3437744436 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9c38102e70 msgr2=0x7f9c38103280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:16:03.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.616+0000 7f9c3c9cb640 1 --2- 172.21.15.42:0/3437744436 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9c38102e70 0x7f9c38103280 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f9c28009900 tx=0x7f9c2802a410 comp rx=0 tx=0).stop 2024-08-30T09:16:03.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.616+0000 7f9c3c9cb640 1 -- 172.21.15.42:0/3437744436 shutdown_connections 2024-08-30T09:16:03.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.616+0000 7f9c3c9cb640 1 --2- 172.21.15.42:0/3437744436 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9c381058b0 0x7f9c3810f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:03.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.616+0000 7f9c3c9cb640 1 --2- 172.21.15.42:0/3437744436 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9c38104f60 0x7f9c38105370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:03.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.616+0000 7f9c3c9cb640 1 --2- 172.21.15.42:0/3437744436 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9c38102e70 0x7f9c38103280 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:03.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.616+0000 7f9c3c9cb640 1 -- 172.21.15.42:0/3437744436 >> 172.21.15.42:0/3437744436 conn(0x7f9c38079d30 msgr2=0x7f9c3807a140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:16:03.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.616+0000 7f9c3c9cb640 1 -- 172.21.15.42:0/3437744436 shutdown_connections 2024-08-30T09:16:03.621 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.616+0000 7f9c3c9cb640 1 -- 172.21.15.42:0/3437744436 wait complete. 2024-08-30T09:16:03.621 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.617+0000 7f9c3c9cb640 1 Processor -- start 2024-08-30T09:16:03.621 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.617+0000 7f9c3c9cb640 1 -- start start 2024-08-30T09:16:03.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.617+0000 7f9c3c9cb640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9c38102e70 0x7f9c3819a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:16:03.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.617+0000 7f9c3c9cb640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9c38104f60 0x7f9c3819a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:16:03.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.617+0000 7f9c3c9cb640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9c381058b0 0x7f9c3819fb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:16:03.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.617+0000 7f9c3c9cb640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f9c381138d0 con 0x7f9c381058b0 2024-08-30T09:16:03.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.617+0000 7f9c3c9cb640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f9c38113750 con 0x7f9c38104f60 2024-08-30T09:16:03.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.617+0000 7f9c3c9cb640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f9c38113a50 con 0x7f9c38102e70 2024-08-30T09:16:03.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.617+0000 7f9c36d9d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9c381058b0 0x7f9c3819fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:16:03.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.618+0000 7f9c36d9d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9c381058b0 0x7f9c3819fb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:32914/0 (socket says 172.21.15.42:32914) 2024-08-30T09:16:03.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.618+0000 7f9c36d9d640 1 -- 172.21.15.42:0/2406682600 learned_addr learned my addr 172.21.15.42:0/2406682600 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:16:03.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.618+0000 7f9c2ffff640 1 --2- 172.21.15.42:0/2406682600 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9c38104f60 0x7f9c3819a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:16:03.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.618+0000 7f9c3659c640 1 --2- 172.21.15.42:0/2406682600 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9c38102e70 0x7f9c3819a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:16:03.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.618+0000 7f9c36d9d640 1 -- 172.21.15.42:0/2406682600 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9c38102e70 msgr2=0x7f9c3819a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:16:03.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.618+0000 7f9c36d9d640 1 --2- 172.21.15.42:0/2406682600 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9c38102e70 0x7f9c3819a2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:03.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.618+0000 7f9c36d9d640 1 -- 172.21.15.42:0/2406682600 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9c38104f60 msgr2=0x7f9c3819a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:16:03.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.618+0000 7f9c36d9d640 1 --2- 172.21.15.42:0/2406682600 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9c38104f60 0x7f9c3819a860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:03.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.618+0000 7f9c36d9d640 1 -- 172.21.15.42:0/2406682600 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c381a0230 con 0x7f9c381058b0 2024-08-30T09:16:03.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.618+0000 7f9c36d9d640 1 --2- 172.21.15.42:0/2406682600 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9c381058b0 0x7f9c3819fb00 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f9c3000cad0 tx=0x7f9c3000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:16:03.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.618+0000 7f9c2f7fe640 1 -- 172.21.15.42:0/2406682600 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c3000ae20 con 0x7f9c381058b0 2024-08-30T09:16:03.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.618+0000 7f9c3c9cb640 1 -- 172.21.15.42:0/2406682600 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c381a0520 con 0x7f9c381058b0 2024-08-30T09:16:03.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.618+0000 7f9c2f7fe640 1 -- 172.21.15.42:0/2406682600 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9c30005020 con 0x7f9c381058b0 2024-08-30T09:16:03.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.618+0000 7f9c2f7fe640 1 -- 172.21.15.42:0/2406682600 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c3000f6b0 con 0x7f9c381058b0 2024-08-30T09:16:03.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.618+0000 7f9c3c9cb640 1 -- 172.21.15.42:0/2406682600 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c381a6c80 con 0x7f9c381058b0 2024-08-30T09:16:03.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.621+0000 7f9c2f7fe640 1 -- 172.21.15.42:0/2406682600 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7f9c300051d0 con 0x7f9c381058b0 2024-08-30T09:16:03.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.621+0000 7f9c42f40740 1 Processor -- start 2024-08-30T09:16:03.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.621+0000 7f9c42f40740 1 -- start start 2024-08-30T09:16:03.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.621+0000 7f9c42f40740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560e3f6bf070 0x560e3fa64310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:16:03.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.622+0000 7f9c42f40740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560e3f6961b0 0x560e3fa64850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:16:03.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.622+0000 7f9c42f40740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560e3f69e390 0x560e3fa64d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:16:03.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.622+0000 7f9c42f40740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x560e3f9db680 con 0x560e3f6961b0 2024-08-30T09:16:03.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.622+0000 7f9c42f40740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x560e3f9cac10 con 0x560e3f69e390 2024-08-30T09:16:03.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.622+0000 7f9c42f40740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x560e3f9cebc0 con 0x560e3f6bf070 2024-08-30T09:16:03.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.622+0000 7f9c3659c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560e3f6961b0 0x560e3fa64850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:16:03.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.622+0000 7f9c3659c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560e3f6961b0 0x560e3fa64850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:32920/0 (socket says 172.21.15.42:32920) 2024-08-30T09:16:03.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.622+0000 7f9c3659c640 1 -- 172.21.15.42:0/3841212399 learned_addr learned my addr 172.21.15.42:0/3841212399 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:16:03.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.622+0000 7f9c2ffff640 1 --2- 172.21.15.42:0/3841212399 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560e3f6bf070 0x560e3fa64310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:16:03.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.622+0000 7f9c2f7fe640 1 --2- 172.21.15.42:0/2406682600 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f9c240b4b30 0x7f9c240b7040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:16:03.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.622+0000 7f9c2ffff640 1 --2- 172.21.15.42:0/3841212399 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560e3f69e390 0x560e3fa64d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:16:03.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.622+0000 7f9c3659c640 1 -- 172.21.15.42:0/3841212399 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560e3f6bf070 msgr2=0x560e3fa64310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:16:03.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.622+0000 7f9c3659c640 1 --2- 172.21.15.42:0/3841212399 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560e3f6bf070 0x560e3fa64310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:03.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.622+0000 7f9c3659c640 1 -- 172.21.15.42:0/3841212399 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560e3f69e390 msgr2=0x560e3fa64d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:16:03.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.622+0000 7f9c3659c640 1 --2- 172.21.15.42:0/3841212399 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560e3f69e390 0x560e3fa64d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:03.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.622+0000 7f9c3659c640 1 -- 172.21.15.42:0/3841212399 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e3f72a650 con 0x560e3f6961b0 2024-08-30T09:16:03.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.622+0000 7f9c2ffff640 1 --2- 172.21.15.42:0/3841212399 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560e3f69e390 0x560e3fa64d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-30T09:16:03.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.622+0000 7f9c36d9d640 1 --2- 172.21.15.42:0/2406682600 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f9c240b4b30 0x7f9c240b7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:16:03.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.622+0000 7f9c2f7fe640 1 -- 172.21.15.42:0/2406682600 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f9c300e5440 con 0x7f9c381058b0 2024-08-30T09:16:03.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.622+0000 7f9c3659c640 1 --2- 172.21.15.42:0/3841212399 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560e3f6961b0 0x560e3fa64850 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f9c28009900 tx=0x7f9c28032e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6fef1e81b7f1215 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:16:03.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.623+0000 7f9c2cff9640 1 -- 172.21.15.42:0/3841212399 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c28040070 con 0x560e3f6961b0 2024-08-30T09:16:03.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.623+0000 7f9c2cff9640 1 -- 172.21.15.42:0/3841212399 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9c280316a0 con 0x560e3f6961b0 2024-08-30T09:16:03.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.623+0000 7f9c2cff9640 1 -- 172.21.15.42:0/3841212399 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c28035040 con 0x560e3f6961b0 2024-08-30T09:16:03.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.624+0000 7f9c42f40740 1 -- 172.21.15.42:0/3841212399 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560e3f6961b0 msgr2=0x560e3fa64850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:16:03.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.624+0000 7f9c36d9d640 1 --2- 172.21.15.42:0/2406682600 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f9c240b4b30 0x7f9c240b7040 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f9c30000f80 tx=0x7f9c30070590 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:16:03.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.624+0000 7f9c42f40740 1 --2- 172.21.15.42:0/3841212399 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560e3f6961b0 0x560e3fa64850 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f9c28009900 tx=0x7f9c28032e60 comp rx=0 tx=0).stop 2024-08-30T09:16:03.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.624+0000 7f9c42f40740 1 -- 172.21.15.42:0/3841212399 shutdown_connections 2024-08-30T09:16:03.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.624+0000 7f9c42f40740 1 --2- 172.21.15.42:0/3841212399 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560e3f69e390 0x560e3fa64d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:03.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.624+0000 7f9c42f40740 1 --2- 172.21.15.42:0/3841212399 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560e3f6961b0 0x560e3fa64850 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:03.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.624+0000 7f9c42f40740 1 --2- 172.21.15.42:0/3841212399 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560e3f6bf070 0x560e3fa64310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:03.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.624+0000 7f9c42f40740 1 -- 172.21.15.42:0/3841212399 >> 172.21.15.42:0/3841212399 conn(0x560e3f6b3460 msgr2=0x560e3fa62200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:16:03.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.624+0000 7f9c42f40740 1 -- 172.21.15.42:0/3841212399 shutdown_connections 2024-08-30T09:16:03.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.624+0000 7f9c42f40740 1 -- 172.21.15.42:0/3841212399 wait complete. 2024-08-30T09:16:03.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.624+0000 7f9c42f40740 1 -- start start 2024-08-30T09:16:03.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.624+0000 7f9c42f40740 1 Processor -- start 2024-08-30T09:16:03.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.625+0000 7f9c42f40740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560e3f6961b0 0x560e3fa6fa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:16:03.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.625+0000 7f9c42f40740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560e3f69e390 0x560e3fa6ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:16:03.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.625+0000 7f9c42f40740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560e3f6bf070 0x560e3fa7bde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:16:03.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.625+0000 7f9c42f40740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x560e3f82d700 con 0x560e3f69e390 2024-08-30T09:16:03.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.625+0000 7f9c42f40740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x560e3f951750 con 0x560e3f6bf070 2024-08-30T09:16:03.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.625+0000 7f9c42f40740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x560e3fa72a90 con 0x560e3f6961b0 2024-08-30T09:16:03.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.625+0000 7f9c42f40740 10 client.0 _ll_register_callbacks cb 0x560e3f980e10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:16:03.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.625+0000 7f9c3659c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560e3f69e390 0x560e3fa6ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:16:03.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.625+0000 7f9c3659c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560e3f69e390 0x560e3fa6ffb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:32936/0 (socket says 172.21.15.42:32936) 2024-08-30T09:16:03.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.625+0000 7f9c3659c640 1 -- 172.21.15.42:0/1297778673 learned_addr learned my addr 172.21.15.42:0/1297778673 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:16:03.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.625+0000 7f9c2ffff640 1 --2- 172.21.15.42:0/1297778673 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560e3f6bf070 0x560e3fa7bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:16:03.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.625+0000 7f9c2ffff640 1 --2- 172.21.15.42:0/1297778673 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560e3f6961b0 0x560e3fa6fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:16:03.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.625+0000 7f9c3659c640 1 -- 172.21.15.42:0/1297778673 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560e3f6961b0 msgr2=0x560e3fa6fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:16:03.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.625+0000 7f9c3659c640 1 --2- 172.21.15.42:0/1297778673 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560e3f6961b0 0x560e3fa6fa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:03.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.625+0000 7f9c3659c640 1 -- 172.21.15.42:0/1297778673 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560e3f6bf070 msgr2=0x560e3fa7bde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:16:03.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.625+0000 7f9c3659c640 1 --2- 172.21.15.42:0/1297778673 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560e3f6bf070 0x560e3fa7bde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:03.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.625+0000 7f9c3659c640 1 -- 172.21.15.42:0/1297778673 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e3f98a900 con 0x560e3f69e390 2024-08-30T09:16:03.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.625+0000 7f9c2ffff640 1 --2- 172.21.15.42:0/1297778673 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560e3f6961b0 0x560e3fa6fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-30T09:16:03.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.625+0000 7f9c3659c640 1 --2- 172.21.15.42:0/1297778673 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560e3f69e390 0x560e3fa6ffb0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f9c2802a930 tx=0x7f9c280052a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97d21b7ce6cf5068 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:16:03.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.625+0000 7f9c1dffb640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-08-30T09:16:03.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.625+0000 7f9c1dffb640 1 -- 172.21.15.42:0/1297778673 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c28035070 con 0x560e3f69e390 2024-08-30T09:16:03.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.626+0000 7f9c1dffb640 1 -- 172.21.15.42:0/1297778673 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f9c28004020 con 0x560e3f69e390 2024-08-30T09:16:03.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.626+0000 7f9c42f40740 1 -- 172.21.15.42:0/1297778673 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9c280316a0 con 0x560e3f69e390 2024-08-30T09:16:03.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.626+0000 7f9c1dffb640 1 -- 172.21.15.42:0/1297778673 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f9c28054050 con 0x560e3f69e390 2024-08-30T09:16:03.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.626+0000 7f9c42f40740 10 client.17958 fetch_fsmap learned FSMap version 49 2024-08-30T09:16:03.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.626+0000 7f9c42f40740 1 -- 172.21.15.42:0/1297778673 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x560e3fa7e7d0 con 0x560e3f69e390 2024-08-30T09:16:03.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.626+0000 7f9c1dffb640 1 -- 172.21.15.42:0/1297778673 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9c280316a0 con 0x560e3f69e390 2024-08-30T09:16:03.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.626+0000 7f9c1dffb640 1 -- 172.21.15.42:0/1297778673 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 49) ==== 8362+0+0 (secure 0 0 0) 0x7f9c28004750 con 0x560e3f69e390 2024-08-30T09:16:03.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.626+0000 7f9c42f40740 10 client.17958 fetch_fsmap finished waiting for FSMap version 49 2024-08-30T09:16:03.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.626+0000 7f9c42f40740 10 client.17958 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-08-30T09:16:03.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.626+0000 7f9c42f40740 20 client.17958 populate_metadata read hostname 'smithi042' 2024-08-30T09:16:03.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.626+0000 7f9c42f40740 1 --2- 172.21.15.42:0/1297778673 >> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] conn(0x560e3f67d9f0 0x560e3fa81b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:16:03.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.626+0000 7f9c42f40740 4 client.17958 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-30T09:16:03.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.626+0000 7f9c42f40740 1 -- 172.21.15.42:0/1297778673 --> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560e3f9dd5d0 con 0x560e3f67d9f0 2024-08-30T09:16:03.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.627+0000 7f9c2ffff640 1 --2- 172.21.15.42:0/1297778673 >> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] conn(0x560e3f67d9f0 0x560e3fa81b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:16:03.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.627+0000 7f9c1dffb640 10 client.17958 ms_handle_connect on v2:172.21.15.167:6836/3011837856 2024-08-30T09:16:03.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.627+0000 7f9c2ffff640 1 --2- 172.21.15.42:0/1297778673 >> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] conn(0x560e3f67d9f0 0x560e3fa81b30 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=25f8679d96d43646 server_cookie=1b8f6006d595b4f2 in_seq=0 out_seq=0 2024-08-30T09:16:03.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.628+0000 7f9c1dffb640 1 -- 172.21.15.42:0/1297778673 <== mds.2 v2:172.21.15.167:6836/3011837856 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x560e3f9dd5d0 con 0x560e3f67d9f0 2024-08-30T09:16:03.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.628+0000 7f9c1dffb640 10 client.17958 handle_command_reply: tid=0 2024-08-30T09:16:03.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.628+0000 7f9c42f40740 1 client.17958 shutdown 2024-08-30T09:16:03.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.628+0000 7f9c42f40740 1 -- 172.21.15.42:0/1297778673 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560e3f69e390 msgr2=0x560e3fa6ffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:16:03.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.628+0000 7f9c42f40740 1 --2- 172.21.15.42:0/1297778673 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560e3f69e390 0x560e3fa6ffb0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f9c2802a930 tx=0x7f9c280052a0 comp rx=0 tx=0).stop 2024-08-30T09:16:03.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.628+0000 7f9c42f40740 1 -- 172.21.15.42:0/1297778673 shutdown_connections 2024-08-30T09:16:03.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.628+0000 7f9c42f40740 1 --2- 172.21.15.42:0/1297778673 >> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] conn(0x560e3f67d9f0 0x560e3fa81b30 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:03.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.628+0000 7f9c1dffb640 0 client.17958 ms_handle_reset on v2:172.21.15.167:6836/3011837856 2024-08-30T09:16:03.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.628+0000 7f9c42f40740 1 --2- 172.21.15.42:0/1297778673 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560e3f6bf070 0x560e3fa7bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:03.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.628+0000 7f9c42f40740 1 --2- 172.21.15.42:0/1297778673 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560e3f69e390 0x560e3fa6ffb0 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:03.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.628+0000 7f9c42f40740 1 --2- 172.21.15.42:0/1297778673 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560e3f6961b0 0x560e3fa6fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:03.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.628+0000 7f9c42f40740 1 -- 172.21.15.42:0/1297778673 >> 172.21.15.42:0/1297778673 conn(0x560e3f6b3460 msgr2=0x560e3fa674a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:16:03.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.628+0000 7f9c42f40740 1 -- 172.21.15.42:0/1297778673 shutdown_connections 2024-08-30T09:16:03.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.628+0000 7f9c42f40740 1 -- 172.21.15.42:0/1297778673 wait complete. 2024-08-30T09:16:03.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.628+0000 7f9c42f40740 20 client.17958 trim_cache size 0 max 16384 2024-08-30T09:16:03.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.641+0000 7f9c42f40740 1 Processor -- start 2024-08-30T09:16:03.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.641+0000 7f9c42f40740 1 -- start start 2024-08-30T09:16:03.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.642+0000 7f9c42f40740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560e3f67d9f0 0x560e3fa67b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:16:03.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.642+0000 7f9c42f40740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560e3f6961b0 0x560e3fa7bd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:16:03.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.642+0000 7f9c42f40740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560e3f69e390 0x560e3fa61b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:16:03.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.642+0000 7f9c42f40740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f9c381b3210 con 0x560e3f69e390 2024-08-30T09:16:03.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.642+0000 7f9c42f40740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f9c381b0ae0 con 0x560e3f6961b0 2024-08-30T09:16:03.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.642+0000 7f9c42f40740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f9c381ae930 con 0x560e3f67d9f0 2024-08-30T09:16:03.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.642+0000 7f9c2ffff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560e3f69e390 0x560e3fa61b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:16:03.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.642+0000 7f9c2ffff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560e3f69e390 0x560e3fa61b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:32946/0 (socket says 172.21.15.42:32946) 2024-08-30T09:16:03.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.642+0000 7f9c2ffff640 1 -- 172.21.15.42:0/3900637200 learned_addr learned my addr 172.21.15.42:0/3900637200 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:16:03.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.642+0000 7f9c2ffff640 1 --2- 172.21.15.42:0/3900637200 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560e3f67d9f0 0x560e3fa67b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:16:03.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.642+0000 7f9c3659c640 1 --2- 172.21.15.42:0/3900637200 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560e3f6961b0 0x560e3fa7bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:16:03.647 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.642+0000 7f9c2ffff640 1 -- 172.21.15.42:0/3900637200 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560e3f67d9f0 msgr2=0x560e3fa67b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:16:03.647 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.642+0000 7f9c2ffff640 1 --2- 172.21.15.42:0/3900637200 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560e3f67d9f0 0x560e3fa67b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:03.647 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.642+0000 7f9c2ffff640 1 -- 172.21.15.42:0/3900637200 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560e3f6961b0 msgr2=0x560e3fa7bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:16:03.647 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.642+0000 7f9c2ffff640 1 --2- 172.21.15.42:0/3900637200 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560e3f6961b0 0x560e3fa7bd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:03.647 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.642+0000 7f9c2ffff640 1 -- 172.21.15.42:0/3900637200 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e3fa72580 con 0x560e3f69e390 2024-08-30T09:16:03.647 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.642+0000 7f9c3659c640 1 --2- 172.21.15.42:0/3900637200 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560e3f6961b0 0x560e3fa7bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-30T09:16:03.647 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.642+0000 7f9c2ffff640 1 --2- 172.21.15.42:0/3900637200 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560e3f69e390 0x560e3fa61b20 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f9c38101a70 tx=0x7f9c200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff47c12c829b745a server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:16:03.647 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.643+0000 7f9c1ffff640 1 -- 172.21.15.42:0/3900637200 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c20013cc0 con 0x560e3f69e390 2024-08-30T09:16:03.647 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.643+0000 7f9c1ffff640 1 -- 172.21.15.42:0/3900637200 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9c20005020 con 0x560e3f69e390 2024-08-30T09:16:03.647 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.643+0000 7f9c1ffff640 1 -- 172.21.15.42:0/3900637200 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c20013e60 con 0x560e3f69e390 2024-08-30T09:16:03.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.643+0000 7f9c42f40740 1 -- 172.21.15.42:0/3900637200 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560e3f69e390 msgr2=0x560e3fa61b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:16:03.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.643+0000 7f9c42f40740 1 --2- 172.21.15.42:0/3900637200 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560e3f69e390 0x560e3fa61b20 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f9c38101a70 tx=0x7f9c200046c0 comp rx=0 tx=0).stop 2024-08-30T09:16:03.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.643+0000 7f9c42f40740 1 -- 172.21.15.42:0/3900637200 shutdown_connections 2024-08-30T09:16:03.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.643+0000 7f9c42f40740 1 --2- 172.21.15.42:0/3900637200 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560e3f69e390 0x560e3fa61b20 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:03.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.643+0000 7f9c42f40740 1 --2- 172.21.15.42:0/3900637200 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560e3f6961b0 0x560e3fa7bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:03.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.643+0000 7f9c42f40740 1 --2- 172.21.15.42:0/3900637200 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560e3f67d9f0 0x560e3fa67b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:03.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.643+0000 7f9c42f40740 1 -- 172.21.15.42:0/3900637200 >> 172.21.15.42:0/3900637200 conn(0x560e3f6b3460 msgr2=0x560e3fa67410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:16:03.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.643+0000 7f9c42f40740 1 -- 172.21.15.42:0/3900637200 shutdown_connections 2024-08-30T09:16:03.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.643+0000 7f9c42f40740 1 -- 172.21.15.42:0/3900637200 wait complete. 2024-08-30T09:16:03.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.644+0000 7f9c42f40740 1 -- start start 2024-08-30T09:16:03.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.644+0000 7f9c42f40740 1 Processor -- start 2024-08-30T09:16:03.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.644+0000 7f9c42f40740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560e3f67d9f0 0x560e3fa82c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:16:03.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.644+0000 7f9c42f40740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560e3f6961b0 0x560e3fa83190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:16:03.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.644+0000 7f9c42f40740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560e3f69e390 0x560e3fa836d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:16:03.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.644+0000 7f9c42f40740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f9c381aefa0 con 0x560e3f69e390 2024-08-30T09:16:03.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.644+0000 7f9c42f40740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f9c381b0410 con 0x560e3f6961b0 2024-08-30T09:16:03.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.644+0000 7f9c42f40740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f9c381af670 con 0x560e3f67d9f0 2024-08-30T09:16:03.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.644+0000 7f9c42f40740 10 client.0 _ll_register_callbacks cb 0x7f9c28032db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:16:03.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.644+0000 7f9c2ffff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560e3f69e390 0x560e3fa836d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:16:03.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.644+0000 7f9c2ffff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560e3f69e390 0x560e3fa836d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:32948/0 (socket says 172.21.15.42:32948) 2024-08-30T09:16:03.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.644+0000 7f9c2ffff640 1 -- 172.21.15.42:0/2062172111 learned_addr learned my addr 172.21.15.42:0/2062172111 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:16:03.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.644+0000 7f9c3659c640 1 --2- 172.21.15.42:0/2062172111 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560e3f6961b0 0x560e3fa83190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:16:03.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.645+0000 7f9c2ffff640 1 --2- 172.21.15.42:0/2062172111 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560e3f67d9f0 0x560e3fa82c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:16:03.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.645+0000 7f9c2ffff640 1 -- 172.21.15.42:0/2062172111 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560e3f67d9f0 msgr2=0x560e3fa82c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:16:03.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.645+0000 7f9c2ffff640 1 --2- 172.21.15.42:0/2062172111 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560e3f67d9f0 0x560e3fa82c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:03.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.645+0000 7f9c2ffff640 1 -- 172.21.15.42:0/2062172111 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560e3f6961b0 msgr2=0x560e3fa83190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:16:03.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.645+0000 7f9c2ffff640 1 --2- 172.21.15.42:0/2062172111 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560e3f6961b0 0x560e3fa83190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:03.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.645+0000 7f9c2ffff640 1 -- 172.21.15.42:0/2062172111 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e3faeb3b0 con 0x560e3f69e390 2024-08-30T09:16:03.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.645+0000 7f9c2ffff640 1 --2- 172.21.15.42:0/2062172111 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560e3f69e390 0x560e3fa836d0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x560e3fa652b0 tx=0x7f9c20015040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da0371b1483021ed server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:16:03.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.645+0000 7f9c1dffb640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-08-30T09:16:03.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.645+0000 7f9c1dffb640 1 -- 172.21.15.42:0/2062172111 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c200044e0 con 0x560e3f69e390 2024-08-30T09:16:03.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.645+0000 7f9c1dffb640 1 -- 172.21.15.42:0/2062172111 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f9c200040f0 con 0x560e3f69e390 2024-08-30T09:16:03.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.645+0000 7f9c42f40740 1 -- 172.21.15.42:0/2062172111 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9c20005020 con 0x560e3f69e390 2024-08-30T09:16:03.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.645+0000 7f9c1dffb640 1 -- 172.21.15.42:0/2062172111 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f9c200042f0 con 0x560e3f69e390 2024-08-30T09:16:03.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.645+0000 7f9c42f40740 10 client.17964 fetch_fsmap learned FSMap version 49 2024-08-30T09:16:03.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.645+0000 7f9c42f40740 1 -- 172.21.15.42:0/2062172111 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x560e3faec630 con 0x560e3f69e390 2024-08-30T09:16:03.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.645+0000 7f9c1dffb640 1 -- 172.21.15.42:0/2062172111 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9c20005020 con 0x560e3f69e390 2024-08-30T09:16:03.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.646+0000 7f9c1dffb640 1 -- 172.21.15.42:0/2062172111 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 49) ==== 8362+0+0 (secure 0 0 0) 0x7f9c20034040 con 0x560e3f69e390 2024-08-30T09:16:03.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.646+0000 7f9c42f40740 10 client.17964 fetch_fsmap finished waiting for FSMap version 49 2024-08-30T09:16:03.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.646+0000 7f9c42f40740 10 client.17964 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-08-30T09:16:03.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.646+0000 7f9c42f40740 20 client.17964 populate_metadata read hostname 'smithi042' 2024-08-30T09:16:03.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.646+0000 7f9c42f40740 1 --2- 172.21.15.42:0/2062172111 >> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] conn(0x560e3f6bf070 0x560e3faeece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:16:03.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.646+0000 7f9c42f40740 4 client.17964 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-30T09:16:03.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.646+0000 7f9c42f40740 1 -- 172.21.15.42:0/2062172111 --> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560e3f9d9070 con 0x560e3f6bf070 2024-08-30T09:16:03.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.646+0000 7f9c3659c640 1 --2- 172.21.15.42:0/2062172111 >> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] conn(0x560e3f6bf070 0x560e3faeece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:16:03.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.646+0000 7f9c3659c640 1 --2- 172.21.15.42:0/2062172111 >> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] conn(0x560e3f6bf070 0x560e3faeece0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1bd200c00116d5b0 server_cookie=fb3d048c58fb09f5 in_seq=0 out_seq=0 2024-08-30T09:16:03.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.646+0000 7f9c1dffb640 10 client.17964 ms_handle_connect on v2:172.21.15.167:6836/3011837856 2024-08-30T09:16:03.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.647+0000 7f9c1dffb640 1 -- 172.21.15.42:0/2062172111 <== mds.2 v2:172.21.15.167:6836/3011837856 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560e3f9d9070 con 0x560e3f6bf070 2024-08-30T09:16:03.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.647+0000 7f9c1dffb640 10 client.17964 handle_command_reply: tid=0 2024-08-30T09:16:03.655 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.647+0000 7f9c42f40740 1 client.17964 shutdown 2024-08-30T09:16:03.655 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.647+0000 7f9c42f40740 1 -- 172.21.15.42:0/2062172111 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560e3f69e390 msgr2=0x560e3fa836d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:16:03.655 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.647+0000 7f9c42f40740 1 --2- 172.21.15.42:0/2062172111 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560e3f69e390 0x560e3fa836d0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x560e3fa652b0 tx=0x7f9c20015040 comp rx=0 tx=0).stop 2024-08-30T09:16:03.655 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.647+0000 7f9c42f40740 1 -- 172.21.15.42:0/2062172111 shutdown_connections 2024-08-30T09:16:03.655 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.647+0000 7f9c42f40740 1 --2- 172.21.15.42:0/2062172111 >> [v2:172.21.15.167:6836/3011837856,v1:172.21.15.167:6837/3011837856] conn(0x560e3f6bf070 0x560e3faeece0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:03.655 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.647+0000 7f9c42f40740 1 --2- 172.21.15.42:0/2062172111 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560e3f69e390 0x560e3fa836d0 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:03.655 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.647+0000 7f9c42f40740 1 --2- 172.21.15.42:0/2062172111 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560e3f6961b0 0x560e3fa83190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:03.655 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.647+0000 7f9c42f40740 1 --2- 172.21.15.42:0/2062172111 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560e3f67d9f0 0x560e3fa82c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:03.655 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.647+0000 7f9c42f40740 1 -- 172.21.15.42:0/2062172111 >> 172.21.15.42:0/2062172111 conn(0x560e3f6b3460 msgr2=0x560e3fa6def0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:16:03.655 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.647+0000 7f9c1dffb640 0 client.17964 ms_handle_reset on v2:172.21.15.167:6836/3011837856 2024-08-30T09:16:03.655 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.647+0000 7f9c42f40740 1 -- 172.21.15.42:0/2062172111 shutdown_connections 2024-08-30T09:16:03.655 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.647+0000 7f9c42f40740 1 -- 172.21.15.42:0/2062172111 wait complete. 2024-08-30T09:16:03.655 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.647+0000 7f9c42f40740 20 client.17964 trim_cache size 0 max 16384 2024-08-30T09:16:03.656 INFO:teuthology.orchestra.run.smithi042.stdout:[] 2024-08-30T09:16:03.656 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.647+0000 7f9c3c9cb640 1 -- 172.21.15.42:0/2406682600 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f9c240b4b30 msgr2=0x7f9c240b7040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:16:03.656 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.647+0000 7f9c3c9cb640 1 --2- 172.21.15.42:0/2406682600 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f9c240b4b30 0x7f9c240b7040 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f9c30000f80 tx=0x7f9c30070590 comp rx=0 tx=0).stop 2024-08-30T09:16:03.657 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.647+0000 7f9c3c9cb640 1 -- 172.21.15.42:0/2406682600 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9c381058b0 msgr2=0x7f9c3819fb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:16:03.657 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.647+0000 7f9c3c9cb640 1 --2- 172.21.15.42:0/2406682600 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9c381058b0 0x7f9c3819fb00 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f9c3000cad0 tx=0x7f9c3000cf90 comp rx=0 tx=0).stop 2024-08-30T09:16:03.657 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.648+0000 7f9c3c9cb640 1 -- 172.21.15.42:0/2406682600 shutdown_connections 2024-08-30T09:16:03.657 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.648+0000 7f9c3c9cb640 1 --2- 172.21.15.42:0/2406682600 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f9c240b4b30 0x7f9c240b7040 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:03.657 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.648+0000 7f9c3c9cb640 1 --2- 172.21.15.42:0/2406682600 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9c381058b0 0x7f9c3819fb00 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:03.657 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.648+0000 7f9c3c9cb640 1 --2- 172.21.15.42:0/2406682600 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9c38104f60 0x7f9c3819a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:03.657 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.648+0000 7f9c3c9cb640 1 --2- 172.21.15.42:0/2406682600 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9c38102e70 0x7f9c3819a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:03.657 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.648+0000 7f9c3c9cb640 1 -- 172.21.15.42:0/2406682600 >> 172.21.15.42:0/2406682600 conn(0x7f9c38079d30 msgr2=0x7f9c38106db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:16:03.657 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.648+0000 7f9c3c9cb640 1 -- 172.21.15.42:0/2406682600 shutdown_connections 2024-08-30T09:16:03.657 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:03.648+0000 7f9c3c9cb640 1 -- 172.21.15.42:0/2406682600 wait complete. 2024-08-30T09:16:04.452 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-08-30T09:16:04.705 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T09:16:05.107 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:16:05 smithi042 ceph-mon[18652]: pgmap v832: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 4.9 MiB/s wr, 571 op/s 2024-08-30T09:16:05.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:16:05 smithi167 ceph-mon[22063]: pgmap v832: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 4.9 MiB/s wr, 571 op/s 2024-08-30T09:16:05.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.350+0000 7f9fda6df640 1 -- 172.21.15.42:0/3583018026 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9fd41081f0 msgr2=0x7f9fd4108600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:16:05.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.350+0000 7f9fda6df640 1 --2- 172.21.15.42:0/3583018026 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9fd41081f0 0x7f9fd4108600 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f9fc8009900 tx=0x7f9fc802a410 comp rx=0 tx=0).stop 2024-08-30T09:16:05.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.350+0000 7f9fda6df640 1 -- 172.21.15.42:0/3583018026 shutdown_connections 2024-08-30T09:16:05.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.350+0000 7f9fda6df640 1 --2- 172.21.15.42:0/3583018026 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9fd4108b40 0x7f9fd410f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:05.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.350+0000 7f9fda6df640 1 --2- 172.21.15.42:0/3583018026 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9fd41081f0 0x7f9fd4108600 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:05.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.350+0000 7f9fda6df640 1 --2- 172.21.15.42:0/3583018026 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9fd41028b0 0x7f9fd4102cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:05.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.350+0000 7f9fda6df640 1 -- 172.21.15.42:0/3583018026 >> 172.21.15.42:0/3583018026 conn(0x7f9fd4079470 msgr2=0x7f9fd4079880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:16:05.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.350+0000 7f9fda6df640 1 -- 172.21.15.42:0/3583018026 shutdown_connections 2024-08-30T09:16:05.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.350+0000 7f9fda6df640 1 -- 172.21.15.42:0/3583018026 wait complete. 2024-08-30T09:16:05.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.351+0000 7f9fda6df640 1 Processor -- start 2024-08-30T09:16:05.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.351+0000 7f9fda6df640 1 -- start start 2024-08-30T09:16:05.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.351+0000 7f9fda6df640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9fd41028b0 0x7f9fd4199250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:16:05.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.351+0000 7f9fda6df640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9fd41081f0 0x7f9fd41997f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:16:05.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.351+0000 7f9fda6df640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9fd4108b40 0x7f9fd419ea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:16:05.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.351+0000 7f9fda6df640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f9fd41127c0 con 0x7f9fd41081f0 2024-08-30T09:16:05.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.351+0000 7f9fda6df640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f9fd4112640 con 0x7f9fd4108b40 2024-08-30T09:16:05.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.351+0000 7f9fda6df640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f9fd4112940 con 0x7f9fd41028b0 2024-08-30T09:16:05.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.351+0000 7f9fd37fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9fd41081f0 0x7f9fd41997f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:16:05.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.351+0000 7f9fd37fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9fd41081f0 0x7f9fd41997f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:32968/0 (socket says 172.21.15.42:32968) 2024-08-30T09:16:05.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.351+0000 7f9fd37fe640 1 -- 172.21.15.42:0/4078239113 learned_addr learned my addr 172.21.15.42:0/4078239113 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:16:05.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.352+0000 7f9fd8c7c640 1 --2- 172.21.15.42:0/4078239113 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9fd4108b40 0x7f9fd419ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:16:05.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.352+0000 7f9fd3fff640 1 --2- 172.21.15.42:0/4078239113 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9fd41028b0 0x7f9fd4199250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:16:05.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.352+0000 7f9fd37fe640 1 -- 172.21.15.42:0/4078239113 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9fd41028b0 msgr2=0x7f9fd4199250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:16:05.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.352+0000 7f9fd37fe640 1 --2- 172.21.15.42:0/4078239113 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9fd41028b0 0x7f9fd4199250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:05.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.352+0000 7f9fd37fe640 1 -- 172.21.15.42:0/4078239113 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9fd4108b40 msgr2=0x7f9fd419ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:16:05.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.352+0000 7f9fd37fe640 1 --2- 172.21.15.42:0/4078239113 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9fd4108b40 0x7f9fd419ea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:05.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.352+0000 7f9fd37fe640 1 -- 172.21.15.42:0/4078239113 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fd419f100 con 0x7f9fd41081f0 2024-08-30T09:16:05.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.352+0000 7f9fd37fe640 1 --2- 172.21.15.42:0/4078239113 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9fd41081f0 0x7f9fd41997f0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f9fc802a930 tx=0x7f9fc8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:16:05.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.352+0000 7f9fd17fa640 1 -- 172.21.15.42:0/4078239113 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fc803f030 con 0x7f9fd41081f0 2024-08-30T09:16:05.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.352+0000 7f9fda6df640 1 -- 172.21.15.42:0/4078239113 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fd419f390 con 0x7f9fd41081f0 2024-08-30T09:16:05.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.352+0000 7f9fd17fa640 1 -- 172.21.15.42:0/4078239113 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9fc8003cb0 con 0x7f9fd41081f0 2024-08-30T09:16:05.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.352+0000 7f9fd17fa640 1 -- 172.21.15.42:0/4078239113 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fc8035070 con 0x7f9fd41081f0 2024-08-30T09:16:05.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.352+0000 7f9fda6df640 1 -- 172.21.15.42:0/4078239113 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fd419f870 con 0x7f9fd41081f0 2024-08-30T09:16:05.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.355+0000 7f9fe0c54740 1 Processor -- start 2024-08-30T09:16:05.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.355+0000 7f9fe0c54740 1 -- start start 2024-08-30T09:16:05.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.355+0000 7f9fd17fa640 1 -- 172.21.15.42:0/4078239113 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7f9fc8042030 con 0x7f9fd41081f0 2024-08-30T09:16:05.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.355+0000 7f9fe0c54740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ddbc712070 0x55ddbcab9df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:16:05.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.355+0000 7f9fe0c54740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ddbc6e91b0 0x55ddbcab7310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:16:05.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.355+0000 7f9fe0c54740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ddbc6f1390 0x55ddbcab79a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:16:05.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.355+0000 7f9fe0c54740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55ddbca2e680 con 0x55ddbc6e91b0 2024-08-30T09:16:05.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.355+0000 7f9fe0c54740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55ddbca1dc10 con 0x55ddbc712070 2024-08-30T09:16:05.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.355+0000 7f9fe0c54740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55ddbca21bc0 con 0x55ddbc6f1390 2024-08-30T09:16:05.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.355+0000 7f9fd3fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ddbc6e91b0 0x55ddbcab7310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:16:05.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.355+0000 7f9fd3fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ddbc6e91b0 0x55ddbcab7310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:32978/0 (socket says 172.21.15.42:32978) 2024-08-30T09:16:05.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.356+0000 7f9fd3fff640 1 -- 172.21.15.42:0/704983612 learned_addr learned my addr 172.21.15.42:0/704983612 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:16:05.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.356+0000 7f9fd8c7c640 1 --2- 172.21.15.42:0/704983612 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ddbc712070 0x55ddbcab9df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:16:05.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.356+0000 7f9fd37fe640 1 --2- 172.21.15.42:0/704983612 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ddbc6f1390 0x55ddbcab79a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:16:05.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.356+0000 7f9fd3fff640 1 -- 172.21.15.42:0/704983612 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ddbc6f1390 msgr2=0x55ddbcab79a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:16:05.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.356+0000 7f9fd3fff640 1 --2- 172.21.15.42:0/704983612 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ddbc6f1390 0x55ddbcab79a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:05.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.356+0000 7f9fd3fff640 1 -- 172.21.15.42:0/704983612 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ddbc712070 msgr2=0x55ddbcab9df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:16:05.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.356+0000 7f9fd3fff640 1 --2- 172.21.15.42:0/704983612 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ddbc712070 0x55ddbcab9df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:05.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.356+0000 7f9fd3fff640 1 -- 172.21.15.42:0/704983612 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ddbc77d650 con 0x55ddbc6e91b0 2024-08-30T09:16:05.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.356+0000 7f9fd37fe640 1 --2- 172.21.15.42:0/704983612 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ddbc6f1390 0x55ddbcab79a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-30T09:16:05.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.356+0000 7f9fd17fa640 1 --2- 172.21.15.42:0/4078239113 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f9fbc0b4b60 0x7f9fbc0b7070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:16:05.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.356+0000 7f9fd3fff640 1 --2- 172.21.15.42:0/704983612 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ddbc6e91b0 0x55ddbcab7310 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f9fc400cbe0 tx=0x7f9fc400e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d12ddc5eef2c97c server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:16:05.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.356+0000 7f9fd8c7c640 1 --2- 172.21.15.42:0/4078239113 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f9fbc0b4b60 0x7f9fbc0b7070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:16:05.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.356+0000 7f9fd17fa640 1 -- 172.21.15.42:0/4078239113 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f9fc8105ba0 con 0x7f9fd41081f0 2024-08-30T09:16:05.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.356+0000 7f9fba7fc640 1 -- 172.21.15.42:0/704983612 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fc4010630 con 0x55ddbc6e91b0 2024-08-30T09:16:05.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.356+0000 7f9fba7fc640 1 -- 172.21.15.42:0/704983612 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9fc4010c80 con 0x55ddbc6e91b0 2024-08-30T09:16:05.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.356+0000 7f9fba7fc640 1 -- 172.21.15.42:0/704983612 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fc401ab70 con 0x55ddbc6e91b0 2024-08-30T09:16:05.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.357+0000 7f9fe0c54740 1 -- 172.21.15.42:0/704983612 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ddbc6e91b0 msgr2=0x55ddbcab7310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:16:05.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.357+0000 7f9fd8c7c640 1 --2- 172.21.15.42:0/4078239113 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f9fbc0b4b60 0x7f9fbc0b7070 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f9fcc007ed0 tx=0x7f9fcc007680 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:16:05.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.357+0000 7f9fe0c54740 1 --2- 172.21.15.42:0/704983612 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ddbc6e91b0 0x55ddbcab7310 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f9fc400cbe0 tx=0x7f9fc400e720 comp rx=0 tx=0).stop 2024-08-30T09:16:05.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.357+0000 7f9fe0c54740 1 -- 172.21.15.42:0/704983612 shutdown_connections 2024-08-30T09:16:05.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.357+0000 7f9fe0c54740 1 --2- 172.21.15.42:0/704983612 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ddbc6f1390 0x55ddbcab79a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:05.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.357+0000 7f9fe0c54740 1 --2- 172.21.15.42:0/704983612 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ddbc6e91b0 0x55ddbcab7310 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:05.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.357+0000 7f9fe0c54740 1 --2- 172.21.15.42:0/704983612 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ddbc712070 0x55ddbcab9df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:05.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.358+0000 7f9fe0c54740 1 -- 172.21.15.42:0/704983612 >> 172.21.15.42:0/704983612 conn(0x55ddbc706460 msgr2=0x55ddbcab5200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:16:05.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.358+0000 7f9fe0c54740 1 -- 172.21.15.42:0/704983612 shutdown_connections 2024-08-30T09:16:05.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.358+0000 7f9fe0c54740 1 -- 172.21.15.42:0/704983612 wait complete. 2024-08-30T09:16:05.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.358+0000 7f9fe0c54740 1 -- start start 2024-08-30T09:16:05.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.358+0000 7f9fe0c54740 1 Processor -- start 2024-08-30T09:16:05.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.359+0000 7f9fe0c54740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ddbc6e91b0 0x55ddbcac6ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:16:05.368 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.359+0000 7f9fe0c54740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ddbc6f1390 0x55ddbcac7530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:16:05.368 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.359+0000 7f9fe0c54740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ddbc712070 0x55ddbcad3230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:16:05.368 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.359+0000 7f9fe0c54740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55ddbc880700 con 0x55ddbc712070 2024-08-30T09:16:05.368 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.359+0000 7f9fe0c54740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55ddbc9a4750 con 0x55ddbc6e91b0 2024-08-30T09:16:05.368 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.359+0000 7f9fe0c54740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55ddbcac9e40 con 0x55ddbc6f1390 2024-08-30T09:16:05.368 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.359+0000 7f9fe0c54740 10 client.0 _ll_register_callbacks cb 0x55ddbc9d3e10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:16:05.368 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.359+0000 7f9fd8c7c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ddbc712070 0x55ddbcad3230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:16:05.368 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.359+0000 7f9fd8c7c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ddbc712070 0x55ddbcad3230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:32994/0 (socket says 172.21.15.42:32994) 2024-08-30T09:16:05.368 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.359+0000 7f9fd8c7c640 1 -- 172.21.15.42:0/1094039163 learned_addr learned my addr 172.21.15.42:0/1094039163 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:16:05.368 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.359+0000 7f9fd3fff640 1 --2- 172.21.15.42:0/1094039163 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ddbc6e91b0 0x55ddbcac6ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:16:05.368 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.359+0000 7f9fd37fe640 1 --2- 172.21.15.42:0/1094039163 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ddbc6f1390 0x55ddbcac7530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:16:05.368 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.359+0000 7f9fd8c7c640 1 -- 172.21.15.42:0/1094039163 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ddbc6f1390 msgr2=0x55ddbcac7530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:16:05.368 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.359+0000 7f9fd8c7c640 1 --2- 172.21.15.42:0/1094039163 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ddbc6f1390 0x55ddbcac7530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:05.368 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.359+0000 7f9fd8c7c640 1 -- 172.21.15.42:0/1094039163 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ddbc6e91b0 msgr2=0x55ddbcac6ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:16:05.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.359+0000 7f9fd8c7c640 1 --2- 172.21.15.42:0/1094039163 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ddbc6e91b0 0x55ddbcac6ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:05.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.359+0000 7f9fd8c7c640 1 -- 172.21.15.42:0/1094039163 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ddbc9dd900 con 0x55ddbc712070 2024-08-30T09:16:05.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.359+0000 7f9fd8c7c640 1 --2- 172.21.15.42:0/1094039163 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ddbc712070 0x55ddbcad3230 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f9fcc00edc0 tx=0x7f9fcc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb4f60444337aea0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:16:05.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.359+0000 7f9fb37fe640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-08-30T09:16:05.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.360+0000 7f9fb37fe640 1 -- 172.21.15.42:0/1094039163 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fcc01cd30 con 0x55ddbc712070 2024-08-30T09:16:05.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.360+0000 7f9fb37fe640 1 -- 172.21.15.42:0/1094039163 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f9fcc01d410 con 0x55ddbc712070 2024-08-30T09:16:05.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.360+0000 7f9fe0c54740 1 -- 172.21.15.42:0/1094039163 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9fc4010c80 con 0x55ddbc712070 2024-08-30T09:16:05.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.360+0000 7f9fb37fe640 1 -- 172.21.15.42:0/1094039163 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f9fcc01d610 con 0x55ddbc712070 2024-08-30T09:16:05.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.360+0000 7f9fe0c54740 10 client.17976 fetch_fsmap learned FSMap version 49 2024-08-30T09:16:05.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.360+0000 7f9fe0c54740 1 -- 172.21.15.42:0/1094039163 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x55ddbcad5c50 con 0x55ddbc712070 2024-08-30T09:16:05.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.360+0000 7f9fb37fe640 1 -- 172.21.15.42:0/1094039163 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9fc4010c80 con 0x55ddbc712070 2024-08-30T09:16:05.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.360+0000 7f9fb37fe640 1 -- 172.21.15.42:0/1094039163 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 49) ==== 8362+0+0 (secure 0 0 0) 0x7f9fcc0288c0 con 0x55ddbc712070 2024-08-30T09:16:05.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.360+0000 7f9fe0c54740 10 client.17976 fetch_fsmap finished waiting for FSMap version 49 2024-08-30T09:16:05.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.360+0000 7f9fe0c54740 10 client.17976 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-08-30T09:16:05.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.360+0000 7f9fe0c54740 20 client.17976 populate_metadata read hostname 'smithi042' 2024-08-30T09:16:05.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.360+0000 7f9fe0c54740 1 --2- 172.21.15.42:0/1094039163 >> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] conn(0x55ddbc6d09f0 0x55ddbcad91f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:16:05.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.360+0000 7f9fe0c54740 4 client.17976 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-30T09:16:05.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.360+0000 7f9fe0c54740 1 -- 172.21.15.42:0/1094039163 --> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ddbca305d0 con 0x55ddbc6d09f0 2024-08-30T09:16:05.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.361+0000 7f9fd3fff640 1 --2- 172.21.15.42:0/1094039163 >> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] conn(0x55ddbc6d09f0 0x55ddbcad91f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:16:05.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.361+0000 7f9fd3fff640 1 --2- 172.21.15.42:0/1094039163 >> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] conn(0x55ddbc6d09f0 0x55ddbcad91f0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a3378aee1a28a768 server_cookie=c658f29159d9acdb in_seq=0 out_seq=0 2024-08-30T09:16:05.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.361+0000 7f9fb37fe640 10 client.17976 ms_handle_connect on v2:172.21.15.190:6838/719981110 2024-08-30T09:16:05.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.362+0000 7f9fb37fe640 1 -- 172.21.15.42:0/1094039163 <== mds.1 v2:172.21.15.190:6838/719981110 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55ddbca305d0 con 0x55ddbc6d09f0 2024-08-30T09:16:05.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.362+0000 7f9fb37fe640 10 client.17976 handle_command_reply: tid=0 2024-08-30T09:16:05.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.362+0000 7f9fe0c54740 1 client.17976 shutdown 2024-08-30T09:16:05.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.362+0000 7f9fe0c54740 1 -- 172.21.15.42:0/1094039163 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ddbc712070 msgr2=0x55ddbcad3230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:16:05.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.362+0000 7f9fe0c54740 1 --2- 172.21.15.42:0/1094039163 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ddbc712070 0x55ddbcad3230 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f9fcc00edc0 tx=0x7f9fcc004a40 comp rx=0 tx=0).stop 2024-08-30T09:16:05.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.362+0000 7f9fe0c54740 1 -- 172.21.15.42:0/1094039163 shutdown_connections 2024-08-30T09:16:05.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.362+0000 7f9fe0c54740 1 --2- 172.21.15.42:0/1094039163 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ddbc6f1390 0x55ddbcac7530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:05.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.362+0000 7f9fe0c54740 1 --2- 172.21.15.42:0/1094039163 >> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] conn(0x55ddbc6d09f0 0x55ddbcad91f0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:05.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.362+0000 7f9fe0c54740 1 --2- 172.21.15.42:0/1094039163 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ddbc712070 0x55ddbcad3230 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:05.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.362+0000 7f9fe0c54740 1 --2- 172.21.15.42:0/1094039163 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ddbc6e91b0 0x55ddbcac6ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:05.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.362+0000 7f9fe0c54740 1 -- 172.21.15.42:0/1094039163 >> 172.21.15.42:0/1094039163 conn(0x55ddbc706460 msgr2=0x55ddbcab8d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:16:05.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.362+0000 7f9fb37fe640 0 client.17976 ms_handle_reset on v2:172.21.15.190:6838/719981110 2024-08-30T09:16:05.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.363+0000 7f9fe0c54740 1 -- 172.21.15.42:0/1094039163 shutdown_connections 2024-08-30T09:16:05.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.363+0000 7f9fe0c54740 1 -- 172.21.15.42:0/1094039163 wait complete. 2024-08-30T09:16:05.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.363+0000 7f9fe0c54740 20 client.17976 trim_cache size 0 max 16384 2024-08-30T09:16:05.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.376+0000 7f9fe0c54740 1 Processor -- start 2024-08-30T09:16:05.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.376+0000 7f9fe0c54740 1 -- start start 2024-08-30T09:16:05.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.376+0000 7f9fe0c54740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ddbc6e91b0 0x55ddbcabd4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:16:05.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.376+0000 7f9fe0c54740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ddbc6f1390 0x55ddbcad3130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:16:05.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.377+0000 7f9fe0c54740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ddbc712070 0x55ddbcad3670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:16:05.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.377+0000 7f9fe0c54740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f9fd41b0f20 con 0x55ddbc6e91b0 2024-08-30T09:16:05.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.377+0000 7f9fe0c54740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f9fd41ae7f0 con 0x55ddbc6f1390 2024-08-30T09:16:05.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.377+0000 7f9fe0c54740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f9fd41ac640 con 0x55ddbc712070 2024-08-30T09:16:05.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.377+0000 7f9fd3fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ddbc6e91b0 0x55ddbcabd4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:16:05.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.377+0000 7f9fd3fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ddbc6e91b0 0x55ddbcabd4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:32998/0 (socket says 172.21.15.42:32998) 2024-08-30T09:16:05.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.377+0000 7f9fd3fff640 1 -- 172.21.15.42:0/2068804623 learned_addr learned my addr 172.21.15.42:0/2068804623 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:16:05.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.377+0000 7f9fd37fe640 1 --2- 172.21.15.42:0/2068804623 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ddbc6f1390 0x55ddbcad3130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:16:05.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.377+0000 7f9fd8c7c640 1 --2- 172.21.15.42:0/2068804623 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ddbc712070 0x55ddbcad3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:16:05.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.377+0000 7f9fd3fff640 1 -- 172.21.15.42:0/2068804623 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ddbc712070 msgr2=0x55ddbcad3670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:16:05.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.377+0000 7f9fd3fff640 1 --2- 172.21.15.42:0/2068804623 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ddbc712070 0x55ddbcad3670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:05.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.377+0000 7f9fd3fff640 1 -- 172.21.15.42:0/2068804623 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ddbc6f1390 msgr2=0x55ddbcad3130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:16:05.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.377+0000 7f9fd3fff640 1 --2- 172.21.15.42:0/2068804623 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ddbc6f1390 0x55ddbcad3130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:05.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.377+0000 7f9fd3fff640 1 -- 172.21.15.42:0/2068804623 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ddbcac54f0 con 0x55ddbc6e91b0 2024-08-30T09:16:05.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.377+0000 7f9fd3fff640 1 --2- 172.21.15.42:0/2068804623 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ddbc6e91b0 0x55ddbcabd4a0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f9fc4005fd0 tx=0x7f9fc401a5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d1f85062c303de9 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:16:05.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.377+0000 7f9fb9ffb640 1 -- 172.21.15.42:0/2068804623 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fc4005240 con 0x55ddbc6e91b0 2024-08-30T09:16:05.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.377+0000 7f9fb9ffb640 1 -- 172.21.15.42:0/2068804623 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9fc401ac70 con 0x55ddbc6e91b0 2024-08-30T09:16:05.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.377+0000 7f9fb9ffb640 1 -- 172.21.15.42:0/2068804623 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fc40048c0 con 0x55ddbc6e91b0 2024-08-30T09:16:05.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.378+0000 7f9fe0c54740 1 -- 172.21.15.42:0/2068804623 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ddbc6e91b0 msgr2=0x55ddbcabd4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:16:05.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.378+0000 7f9fe0c54740 1 --2- 172.21.15.42:0/2068804623 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ddbc6e91b0 0x55ddbcabd4a0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f9fc4005fd0 tx=0x7f9fc401a5a0 comp rx=0 tx=0).stop 2024-08-30T09:16:05.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.378+0000 7f9fe0c54740 1 -- 172.21.15.42:0/2068804623 shutdown_connections 2024-08-30T09:16:05.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.378+0000 7f9fe0c54740 1 --2- 172.21.15.42:0/2068804623 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ddbc712070 0x55ddbcad3670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:05.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.378+0000 7f9fe0c54740 1 --2- 172.21.15.42:0/2068804623 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ddbc6f1390 0x55ddbcad3130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:05.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.378+0000 7f9fe0c54740 1 --2- 172.21.15.42:0/2068804623 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ddbc6e91b0 0x55ddbcabd4a0 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:05.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.378+0000 7f9fe0c54740 1 -- 172.21.15.42:0/2068804623 >> 172.21.15.42:0/2068804623 conn(0x55ddbc706460 msgr2=0x55ddbcac5870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:16:05.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.378+0000 7f9fe0c54740 1 -- 172.21.15.42:0/2068804623 shutdown_connections 2024-08-30T09:16:05.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.378+0000 7f9fe0c54740 1 -- 172.21.15.42:0/2068804623 wait complete. 2024-08-30T09:16:05.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.379+0000 7f9fe0c54740 1 -- start start 2024-08-30T09:16:05.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.379+0000 7f9fe0c54740 1 Processor -- start 2024-08-30T09:16:05.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.379+0000 7f9fe0c54740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ddbc6e91b0 0x55ddbcacf190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:16:05.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.379+0000 7f9fe0c54740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ddbc6f1390 0x55ddbcacf710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:16:05.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.379+0000 7f9fe0c54740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ddbc712070 0x55ddbcacc160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:16:05.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.379+0000 7f9fe0c54740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f9fd41accb0 con 0x55ddbc712070 2024-08-30T09:16:05.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.379+0000 7f9fe0c54740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f9fd41ae120 con 0x55ddbc6f1390 2024-08-30T09:16:05.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.379+0000 7f9fe0c54740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f9fd41ad380 con 0x55ddbc6e91b0 2024-08-30T09:16:05.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.379+0000 7f9fe0c54740 10 client.0 _ll_register_callbacks cb 0x7f9fcc007b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:16:05.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.380+0000 7f9fd8c7c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ddbc712070 0x55ddbcacc160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:16:05.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.380+0000 7f9fd8c7c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ddbc712070 0x55ddbcacc160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33006/0 (socket says 172.21.15.42:33006) 2024-08-30T09:16:05.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.380+0000 7f9fd8c7c640 1 -- 172.21.15.42:0/877957265 learned_addr learned my addr 172.21.15.42:0/877957265 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:16:05.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.380+0000 7f9fd3fff640 1 --2- 172.21.15.42:0/877957265 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ddbc6e91b0 0x55ddbcacf190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:16:05.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.380+0000 7f9fd37fe640 1 --2- 172.21.15.42:0/877957265 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ddbc6f1390 0x55ddbcacf710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:16:05.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.380+0000 7f9fd8c7c640 1 -- 172.21.15.42:0/877957265 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ddbc6e91b0 msgr2=0x55ddbcacf190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:16:05.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.380+0000 7f9fd8c7c640 1 --2- 172.21.15.42:0/877957265 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ddbc6e91b0 0x55ddbcacf190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:05.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.380+0000 7f9fd8c7c640 1 -- 172.21.15.42:0/877957265 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ddbc6f1390 msgr2=0x55ddbcacf710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:16:05.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.380+0000 7f9fd8c7c640 1 --2- 172.21.15.42:0/877957265 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ddbc6f1390 0x55ddbcacf710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:05.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.380+0000 7f9fd8c7c640 1 -- 172.21.15.42:0/877957265 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ddbcacc6a0 con 0x55ddbc712070 2024-08-30T09:16:05.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.380+0000 7f9fd37fe640 1 --2- 172.21.15.42:0/877957265 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ddbc6f1390 0x55ddbcacf710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-30T09:16:05.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.380+0000 7f9fd8c7c640 1 --2- 172.21.15.42:0/877957265 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ddbc712070 0x55ddbcacc160 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f9fcc028d60 tx=0x7f9fcc004d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6b75a8e445a335e server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:16:05.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.380+0000 7f9fb37fe640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-08-30T09:16:05.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.380+0000 7f9fb37fe640 1 -- 172.21.15.42:0/877957265 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fcc03c020 con 0x55ddbc712070 2024-08-30T09:16:05.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.380+0000 7f9fb37fe640 1 -- 172.21.15.42:0/877957265 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f9fcc028690 con 0x55ddbc712070 2024-08-30T09:16:05.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.380+0000 7f9fe0c54740 1 -- 172.21.15.42:0/877957265 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9fc401ac70 con 0x55ddbc712070 2024-08-30T09:16:05.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.381+0000 7f9fb37fe640 1 -- 172.21.15.42:0/877957265 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f9fcc047050 con 0x55ddbc712070 2024-08-30T09:16:05.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.381+0000 7f9fe0c54740 10 client.17982 fetch_fsmap learned FSMap version 49 2024-08-30T09:16:05.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.381+0000 7f9fe0c54740 1 -- 172.21.15.42:0/877957265 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x55ddbcb442e0 con 0x55ddbc712070 2024-08-30T09:16:05.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.381+0000 7f9fb37fe640 1 -- 172.21.15.42:0/877957265 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9fc401ac70 con 0x55ddbc712070 2024-08-30T09:16:05.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.381+0000 7f9fb37fe640 1 -- 172.21.15.42:0/877957265 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 49) ==== 8362+0+0 (secure 0 0 0) 0x7f9fcc0288d0 con 0x55ddbc712070 2024-08-30T09:16:05.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.381+0000 7f9fe0c54740 10 client.17982 fetch_fsmap finished waiting for FSMap version 49 2024-08-30T09:16:05.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.381+0000 7f9fe0c54740 10 client.17982 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-08-30T09:16:05.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.381+0000 7f9fe0c54740 20 client.17982 populate_metadata read hostname 'smithi042' 2024-08-30T09:16:05.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.381+0000 7f9fe0c54740 1 --2- 172.21.15.42:0/877957265 >> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] conn(0x55ddbc6a5da0 0x55ddbcb46a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:16:05.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.381+0000 7f9fe0c54740 4 client.17982 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-30T09:16:05.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.381+0000 7f9fe0c54740 1 -- 172.21.15.42:0/877957265 --> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ddbca2c070 con 0x55ddbc6a5da0 2024-08-30T09:16:05.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.381+0000 7f9fd3fff640 1 --2- 172.21.15.42:0/877957265 >> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] conn(0x55ddbc6a5da0 0x55ddbcb46a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:16:05.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.382+0000 7f9fd3fff640 1 --2- 172.21.15.42:0/877957265 >> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] conn(0x55ddbc6a5da0 0x55ddbcb46a30 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=bc00e8b6cf5bb7ef server_cookie=a37bbe15140c65f4 in_seq=0 out_seq=0 2024-08-30T09:16:05.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.382+0000 7f9fb37fe640 10 client.17982 ms_handle_connect on v2:172.21.15.190:6838/719981110 2024-08-30T09:16:05.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.382+0000 7f9fb37fe640 1 -- 172.21.15.42:0/877957265 <== mds.1 v2:172.21.15.190:6838/719981110 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ddbca2c070 con 0x55ddbc6a5da0 2024-08-30T09:16:05.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.382+0000 7f9fb37fe640 10 client.17982 handle_command_reply: tid=0 2024-08-30T09:16:05.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.382+0000 7f9fe0c54740 1 client.17982 shutdown 2024-08-30T09:16:05.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.382+0000 7f9fe0c54740 1 -- 172.21.15.42:0/877957265 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ddbc712070 msgr2=0x55ddbcacc160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:16:05.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.382+0000 7f9fe0c54740 1 --2- 172.21.15.42:0/877957265 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ddbc712070 0x55ddbcacc160 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f9fcc028d60 tx=0x7f9fcc004d60 comp rx=0 tx=0).stop 2024-08-30T09:16:05.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.382+0000 7f9fe0c54740 1 -- 172.21.15.42:0/877957265 shutdown_connections 2024-08-30T09:16:05.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.382+0000 7f9fe0c54740 1 --2- 172.21.15.42:0/877957265 >> [v2:172.21.15.190:6838/719981110,v1:172.21.15.190:6839/719981110] conn(0x55ddbc6a5da0 0x55ddbcb46a30 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:05.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.382+0000 7f9fe0c54740 1 --2- 172.21.15.42:0/877957265 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ddbc712070 0x55ddbcacc160 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:05.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.382+0000 7f9fe0c54740 1 --2- 172.21.15.42:0/877957265 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ddbc6f1390 0x55ddbcacf710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:05.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.382+0000 7f9fb37fe640 0 client.17982 ms_handle_reset on v2:172.21.15.190:6838/719981110 2024-08-30T09:16:05.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.382+0000 7f9fe0c54740 1 --2- 172.21.15.42:0/877957265 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ddbc6e91b0 0x55ddbcacf190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:05.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.382+0000 7f9fe0c54740 1 -- 172.21.15.42:0/877957265 >> 172.21.15.42:0/877957265 conn(0x55ddbc706460 msgr2=0x55ddbcab8320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:16:05.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.382+0000 7f9fe0c54740 1 -- 172.21.15.42:0/877957265 shutdown_connections 2024-08-30T09:16:05.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.383+0000 7f9fe0c54740 1 -- 172.21.15.42:0/877957265 wait complete. 2024-08-30T09:16:05.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.383+0000 7f9fe0c54740 20 client.17982 trim_cache size 0 max 16384 2024-08-30T09:16:05.390 INFO:teuthology.orchestra.run.smithi042.stdout:[] 2024-08-30T09:16:05.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.383+0000 7f9fda6df640 1 -- 172.21.15.42:0/4078239113 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f9fbc0b4b60 msgr2=0x7f9fbc0b7070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:16:05.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.383+0000 7f9fda6df640 1 --2- 172.21.15.42:0/4078239113 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f9fbc0b4b60 0x7f9fbc0b7070 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f9fcc007ed0 tx=0x7f9fcc007680 comp rx=0 tx=0).stop 2024-08-30T09:16:05.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.383+0000 7f9fda6df640 1 -- 172.21.15.42:0/4078239113 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9fd41081f0 msgr2=0x7f9fd41997f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:16:05.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.383+0000 7f9fda6df640 1 --2- 172.21.15.42:0/4078239113 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9fd41081f0 0x7f9fd41997f0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f9fc802a930 tx=0x7f9fc8004270 comp rx=0 tx=0).stop 2024-08-30T09:16:05.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.383+0000 7f9fda6df640 1 -- 172.21.15.42:0/4078239113 shutdown_connections 2024-08-30T09:16:05.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.383+0000 7f9fda6df640 1 --2- 172.21.15.42:0/4078239113 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f9fbc0b4b60 0x7f9fbc0b7070 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:05.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.383+0000 7f9fda6df640 1 --2- 172.21.15.42:0/4078239113 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9fd4108b40 0x7f9fd419ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:05.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.383+0000 7f9fda6df640 1 --2- 172.21.15.42:0/4078239113 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9fd41081f0 0x7f9fd41997f0 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:05.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.383+0000 7f9fda6df640 1 --2- 172.21.15.42:0/4078239113 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9fd41028b0 0x7f9fd4199250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:05.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.383+0000 7f9fda6df640 1 -- 172.21.15.42:0/4078239113 >> 172.21.15.42:0/4078239113 conn(0x7f9fd4079470 msgr2=0x7f9fd41062e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:16:05.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.383+0000 7f9fda6df640 1 -- 172.21.15.42:0/4078239113 shutdown_connections 2024-08-30T09:16:05.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:05.383+0000 7f9fda6df640 1 -- 172.21.15.42:0/4078239113 wait complete. 2024-08-30T09:16:05.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:16:05 smithi190 ceph-mon[22001]: pgmap v832: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 4.9 MiB/s wr, 571 op/s 2024-08-30T09:16:06.129 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-08-30T09:16:06.382 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T09:16:06.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.886+0000 7fd11aeff640 1 -- 172.21.15.42:0/1741821675 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd114102820 msgr2=0x7fd114102c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:16:06.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.886+0000 7fd11aeff640 1 --2- 172.21.15.42:0/1741821675 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd114102820 0x7fd114102c30 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fd108009900 tx=0x7fd10802a410 comp rx=0 tx=0).stop 2024-08-30T09:16:06.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.886+0000 7fd11aeff640 1 -- 172.21.15.42:0/1741821675 shutdown_connections 2024-08-30T09:16:06.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.886+0000 7fd11aeff640 1 --2- 172.21.15.42:0/1741821675 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd114108ab0 0x7fd11410f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:06.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.886+0000 7fd11aeff640 1 --2- 172.21.15.42:0/1741821675 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd114108160 0x7fd114108570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:06.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.886+0000 7fd11aeff640 1 --2- 172.21.15.42:0/1741821675 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd114102820 0x7fd114102c30 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:06.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.887+0000 7fd11aeff640 1 -- 172.21.15.42:0/1741821675 >> 172.21.15.42:0/1741821675 conn(0x7fd1140794e0 msgr2=0x7fd1140798f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:16:06.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.887+0000 7fd11aeff640 1 -- 172.21.15.42:0/1741821675 shutdown_connections 2024-08-30T09:16:06.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.887+0000 7fd11aeff640 1 -- 172.21.15.42:0/1741821675 wait complete. 2024-08-30T09:16:06.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.887+0000 7fd11aeff640 1 Processor -- start 2024-08-30T09:16:06.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.887+0000 7fd11aeff640 1 -- start start 2024-08-30T09:16:06.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.888+0000 7fd11aeff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd114102820 0x7fd1141991a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:16:06.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.888+0000 7fd11aeff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd114108160 0x7fd114199740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:16:06.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.888+0000 7fd11aeff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd114108ab0 0x7fd11419e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:16:06.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.888+0000 7fd11aeff640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fd114112790 con 0x7fd114108ab0 2024-08-30T09:16:06.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.888+0000 7fd11aeff640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd114112610 con 0x7fd114108160 2024-08-30T09:16:06.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.888+0000 7fd11aeff640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fd114112910 con 0x7fd114102820 2024-08-30T09:16:06.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.888+0000 7fd11949c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd114108ab0 0x7fd11419e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:16:06.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.888+0000 7fd11949c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd114108ab0 0x7fd11419e9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33022/0 (socket says 172.21.15.42:33022) 2024-08-30T09:16:06.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.888+0000 7fd11949c640 1 -- 172.21.15.42:0/2982026364 learned_addr learned my addr 172.21.15.42:0/2982026364 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:16:06.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.888+0000 7fd113fff640 1 --2- 172.21.15.42:0/2982026364 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd114108160 0x7fd114199740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:16:06.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.888+0000 7fd118c9b640 1 --2- 172.21.15.42:0/2982026364 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd114102820 0x7fd1141991a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:16:06.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.888+0000 7fd11949c640 1 -- 172.21.15.42:0/2982026364 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd114102820 msgr2=0x7fd1141991a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:16:06.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.888+0000 7fd11949c640 1 --2- 172.21.15.42:0/2982026364 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd114102820 0x7fd1141991a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:06.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.888+0000 7fd11949c640 1 -- 172.21.15.42:0/2982026364 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd114108160 msgr2=0x7fd114199740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:16:06.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.888+0000 7fd11949c640 1 --2- 172.21.15.42:0/2982026364 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd114108160 0x7fd114199740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:06.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.888+0000 7fd11949c640 1 -- 172.21.15.42:0/2982026364 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd11419f110 con 0x7fd114108ab0 2024-08-30T09:16:06.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.888+0000 7fd11949c640 1 --2- 172.21.15.42:0/2982026364 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd114108ab0 0x7fd11419e9e0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fd10c00b970 tx=0x7fd10c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:16:06.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.888+0000 7fd111ffb640 1 -- 172.21.15.42:0/2982026364 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd10c019070 con 0x7fd114108ab0 2024-08-30T09:16:06.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.889+0000 7fd111ffb640 1 -- 172.21.15.42:0/2982026364 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd10c014440 con 0x7fd114108ab0 2024-08-30T09:16:06.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.889+0000 7fd11aeff640 1 -- 172.21.15.42:0/2982026364 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd11419f400 con 0x7fd114108ab0 2024-08-30T09:16:06.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.889+0000 7fd111ffb640 1 -- 172.21.15.42:0/2982026364 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd10c0075d0 con 0x7fd114108ab0 2024-08-30T09:16:06.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.889+0000 7fd11aeff640 1 -- 172.21.15.42:0/2982026364 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1141a5be0 con 0x7fd114108ab0 2024-08-30T09:16:06.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.891+0000 7fd111ffb640 1 -- 172.21.15.42:0/2982026364 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7fd10c01c070 con 0x7fd114108ab0 2024-08-30T09:16:06.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.891+0000 7fd121474740 1 Processor -- start 2024-08-30T09:16:06.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.891+0000 7fd121474740 1 -- start start 2024-08-30T09:16:06.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.892+0000 7fd121474740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55740edcb070 0x55740f17c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:16:06.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.892+0000 7fd121474740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55740eda21b0 0x55740f17c930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:16:06.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.892+0000 7fd121474740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55740edaa390 0x55740f17b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:16:06.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.892+0000 7fd121474740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55740f0e7680 con 0x55740edaa390 2024-08-30T09:16:06.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.892+0000 7fd121474740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55740f0d6c10 con 0x55740eda21b0 2024-08-30T09:16:06.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.892+0000 7fd121474740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55740f0dabc0 con 0x55740edcb070 2024-08-30T09:16:06.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.892+0000 7fd113fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55740edaa390 0x55740f17b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:16:06.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.892+0000 7fd113fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55740edaa390 0x55740f17b0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33024/0 (socket says 172.21.15.42:33024) 2024-08-30T09:16:06.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.892+0000 7fd113fff640 1 -- 172.21.15.42:0/3729999911 learned_addr learned my addr 172.21.15.42:0/3729999911 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:16:06.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.892+0000 7fd111ffb640 1 --2- 172.21.15.42:0/2982026364 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fd1040b4bb0 0x7fd1040b70c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:16:06.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.892+0000 7fd118c9b640 1 --2- 172.21.15.42:0/3729999911 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55740eda21b0 0x55740f17c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:16:06.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.892+0000 7fd113fff640 1 --2- 172.21.15.42:0/3729999911 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55740edcb070 0x55740f17c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:16:06.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.892+0000 7fd11949c640 1 --2- 172.21.15.42:0/2982026364 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fd1040b4bb0 0x7fd1040b70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:16:06.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.892+0000 7fd111ffb640 1 -- 172.21.15.42:0/2982026364 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7fd10c0e4930 con 0x7fd114108ab0 2024-08-30T09:16:06.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.892+0000 7fd113fff640 1 -- 172.21.15.42:0/3729999911 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55740edcb070 msgr2=0x55740f17c3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:16:06.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.892+0000 7fd113fff640 1 --2- 172.21.15.42:0/3729999911 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55740edcb070 0x55740f17c3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:06.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.892+0000 7fd113fff640 1 -- 172.21.15.42:0/3729999911 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55740eda21b0 msgr2=0x55740f17c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:16:06.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.892+0000 7fd113fff640 1 --2- 172.21.15.42:0/3729999911 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55740eda21b0 0x55740f17c930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:06.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.892+0000 7fd113fff640 1 -- 172.21.15.42:0/3729999911 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55740ee36650 con 0x55740edaa390 2024-08-30T09:16:06.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.893+0000 7fd113fff640 1 --2- 172.21.15.42:0/3729999911 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55740edaa390 0x55740f17b0d0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fd100012770 tx=0x7fd100004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15a69ede85761fad server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:16:06.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.893+0000 7fd11949c640 1 --2- 172.21.15.42:0/2982026364 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fd1040b4bb0 0x7fd1040b70c0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fd10c000f80 tx=0x7fd10c004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:16:06.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.893+0000 7fd0feffd640 1 -- 172.21.15.42:0/3729999911 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd100012cf0 con 0x55740edaa390 2024-08-30T09:16:06.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.893+0000 7fd0feffd640 1 -- 172.21.15.42:0/3729999911 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd100010770 con 0x55740edaa390 2024-08-30T09:16:06.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.893+0000 7fd0feffd640 1 -- 172.21.15.42:0/3729999911 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1000044e0 con 0x55740edaa390 2024-08-30T09:16:06.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.894+0000 7fd121474740 1 -- 172.21.15.42:0/3729999911 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55740edaa390 msgr2=0x55740f17b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:16:06.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.894+0000 7fd121474740 1 --2- 172.21.15.42:0/3729999911 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55740edaa390 0x55740f17b0d0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fd100012770 tx=0x7fd100004730 comp rx=0 tx=0).stop 2024-08-30T09:16:06.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.894+0000 7fd121474740 1 -- 172.21.15.42:0/3729999911 shutdown_connections 2024-08-30T09:16:06.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.894+0000 7fd121474740 1 --2- 172.21.15.42:0/3729999911 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55740edaa390 0x55740f17b0d0 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:06.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.894+0000 7fd121474740 1 --2- 172.21.15.42:0/3729999911 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55740eda21b0 0x55740f17c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:06.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.894+0000 7fd121474740 1 --2- 172.21.15.42:0/3729999911 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55740edcb070 0x55740f17c3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:06.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.894+0000 7fd121474740 1 -- 172.21.15.42:0/3729999911 >> 172.21.15.42:0/3729999911 conn(0x55740edbf460 msgr2=0x55740f16db70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:16:06.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.894+0000 7fd121474740 1 -- 172.21.15.42:0/3729999911 shutdown_connections 2024-08-30T09:16:06.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.894+0000 7fd121474740 1 -- 172.21.15.42:0/3729999911 wait complete. 2024-08-30T09:16:06.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.894+0000 7fd121474740 1 -- start start 2024-08-30T09:16:06.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.894+0000 7fd121474740 1 Processor -- start 2024-08-30T09:16:06.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.895+0000 7fd121474740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55740eda21b0 0x55740f180710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:16:06.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.895+0000 7fd121474740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55740edaa390 0x55740f180cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:16:06.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.895+0000 7fd121474740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55740edcb070 0x55740f185d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:16:06.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.895+0000 7fd121474740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55740ef39700 con 0x55740edaa390 2024-08-30T09:16:06.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.895+0000 7fd121474740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55740f05d750 con 0x55740eda21b0 2024-08-30T09:16:06.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.895+0000 7fd121474740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55740f17bba0 con 0x55740edcb070 2024-08-30T09:16:06.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.895+0000 7fd121474740 10 client.0 _ll_register_callbacks cb 0x55740f08ce10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:16:06.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.895+0000 7fd118c9b640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55740edaa390 0x55740f180cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:16:06.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.895+0000 7fd118c9b640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55740edaa390 0x55740f180cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33028/0 (socket says 172.21.15.42:33028) 2024-08-30T09:16:06.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.895+0000 7fd118c9b640 1 -- 172.21.15.42:0/964037862 learned_addr learned my addr 172.21.15.42:0/964037862 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:16:06.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.895+0000 7fd113fff640 1 --2- 172.21.15.42:0/964037862 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55740edcb070 0x55740f185d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:16:06.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.895+0000 7fd113fff640 1 --2- 172.21.15.42:0/964037862 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55740eda21b0 0x55740f180710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:16:06.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.895+0000 7fd118c9b640 1 -- 172.21.15.42:0/964037862 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55740edcb070 msgr2=0x55740f185d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:16:06.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.895+0000 7fd118c9b640 1 --2- 172.21.15.42:0/964037862 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55740edcb070 0x55740f185d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:06.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.895+0000 7fd118c9b640 1 -- 172.21.15.42:0/964037862 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55740eda21b0 msgr2=0x55740f180710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:16:06.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.895+0000 7fd118c9b640 1 --2- 172.21.15.42:0/964037862 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55740eda21b0 0x55740f180710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:06.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.895+0000 7fd118c9b640 1 -- 172.21.15.42:0/964037862 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55740f096900 con 0x55740edaa390 2024-08-30T09:16:06.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.896+0000 7fd118c9b640 1 --2- 172.21.15.42:0/964037862 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55740edaa390 0x55740f180cf0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fd108009900 tx=0x7fd1080316d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee292cd569dbfca5 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:16:06.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.896+0000 7fd0f3fff640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-08-30T09:16:06.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.896+0000 7fd0f3fff640 1 -- 172.21.15.42:0/964037862 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd108035040 con 0x55740edaa390 2024-08-30T09:16:06.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.896+0000 7fd0f3fff640 1 -- 172.21.15.42:0/964037862 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7fd108004040 con 0x55740edaa390 2024-08-30T09:16:06.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.896+0000 7fd121474740 1 -- 172.21.15.42:0/964037862 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd100010770 con 0x55740edaa390 2024-08-30T09:16:06.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.896+0000 7fd0f3fff640 1 -- 172.21.15.42:0/964037862 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7fd10804f050 con 0x55740edaa390 2024-08-30T09:16:06.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.896+0000 7fd121474740 10 client.17994 fetch_fsmap learned FSMap version 50 2024-08-30T09:16:06.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.896+0000 7fd121474740 1 -- 172.21.15.42:0/964037862 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x55740f18ec60 con 0x55740edaa390 2024-08-30T09:16:06.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.896+0000 7fd0f3fff640 1 -- 172.21.15.42:0/964037862 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd100010770 con 0x55740edaa390 2024-08-30T09:16:06.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.896+0000 7fd0f3fff640 1 -- 172.21.15.42:0/964037862 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 50) ==== 8366+0+0 (secure 0 0 0) 0x7fd108004870 con 0x55740edaa390 2024-08-30T09:16:06.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.896+0000 7fd121474740 10 client.17994 fetch_fsmap finished waiting for FSMap version 50 2024-08-30T09:16:06.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.896+0000 7fd121474740 10 client.17994 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-30T09:16:06.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.897+0000 7fd121474740 20 client.17994 populate_metadata read hostname 'smithi042' 2024-08-30T09:16:06.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.897+0000 7fd121474740 1 --2- 172.21.15.42:0/964037862 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55740ed899f0 0x55740f1921a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:16:06.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.897+0000 7fd121474740 4 client.17994 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-30T09:16:06.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.897+0000 7fd121474740 1 -- 172.21.15.42:0/964037862 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55740f0e95d0 con 0x55740ed899f0 2024-08-30T09:16:06.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.897+0000 7fd113fff640 1 --2- 172.21.15.42:0/964037862 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55740ed899f0 0x55740f1921a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:16:06.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.897+0000 7fd0f3fff640 10 client.17994 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:16:06.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.897+0000 7fd113fff640 1 --2- 172.21.15.42:0/964037862 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55740ed899f0 0x55740f1921a0 crc :-1 s=READY pgs=309 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c8289a64a2c16c5a server_cookie=50be3cb3300e7e29 in_seq=0 out_seq=0 2024-08-30T09:16:06.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.898+0000 7fd0f3fff640 1 -- 172.21.15.42:0/964037862 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55740f0e95d0 con 0x55740ed899f0 2024-08-30T09:16:06.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.898+0000 7fd0f3fff640 10 client.17994 handle_command_reply: tid=0 2024-08-30T09:16:06.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.898+0000 7fd121474740 1 client.17994 shutdown 2024-08-30T09:16:06.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.899+0000 7fd121474740 1 -- 172.21.15.42:0/964037862 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55740edaa390 msgr2=0x55740f180cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:16:06.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.899+0000 7fd121474740 1 --2- 172.21.15.42:0/964037862 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55740edaa390 0x55740f180cf0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fd108009900 tx=0x7fd1080316d0 comp rx=0 tx=0).stop 2024-08-30T09:16:06.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.899+0000 7fd121474740 1 -- 172.21.15.42:0/964037862 shutdown_connections 2024-08-30T09:16:06.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.899+0000 7fd121474740 1 --2- 172.21.15.42:0/964037862 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55740ed899f0 0x55740f1921a0 crc :-1 s=READY pgs=309 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:06.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.899+0000 7fd121474740 1 --2- 172.21.15.42:0/964037862 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55740edcb070 0x55740f185d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:06.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.899+0000 7fd121474740 1 --2- 172.21.15.42:0/964037862 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55740edaa390 0x55740f180cf0 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:06.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.899+0000 7fd121474740 1 --2- 172.21.15.42:0/964037862 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55740eda21b0 0x55740f180710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:06.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.899+0000 7fd121474740 1 -- 172.21.15.42:0/964037862 >> 172.21.15.42:0/964037862 conn(0x55740edbf460 msgr2=0x55740f16db40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:16:06.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.899+0000 7fd0f3fff640 0 client.17994 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:16:06.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.899+0000 7fd121474740 1 -- 172.21.15.42:0/964037862 shutdown_connections 2024-08-30T09:16:06.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.899+0000 7fd121474740 1 -- 172.21.15.42:0/964037862 wait complete. 2024-08-30T09:16:06.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.899+0000 7fd121474740 20 client.17994 trim_cache size 0 max 16384 2024-08-30T09:16:06.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.912+0000 7fd121474740 1 Processor -- start 2024-08-30T09:16:06.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.912+0000 7fd121474740 1 -- start start 2024-08-30T09:16:06.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.912+0000 7fd121474740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55740eda21b0 0x55740f1790a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:16:06.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.912+0000 7fd121474740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55740edaa390 0x55740f16d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:16:06.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.912+0000 7fd121474740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55740edcb070 0x55740f16dd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:16:06.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.912+0000 7fd121474740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fd1141b0fa0 con 0x55740edcb070 2024-08-30T09:16:06.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.912+0000 7fd121474740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd1141ae840 con 0x55740edaa390 2024-08-30T09:16:06.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.912+0000 7fd121474740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fd1141ac690 con 0x55740eda21b0 2024-08-30T09:16:06.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.912+0000 7fd113fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55740edcb070 0x55740f16dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:16:06.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.913+0000 7fd113fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55740edcb070 0x55740f16dd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33044/0 (socket says 172.21.15.42:33044) 2024-08-30T09:16:06.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.913+0000 7fd113fff640 1 -- 172.21.15.42:0/2175535023 learned_addr learned my addr 172.21.15.42:0/2175535023 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:16:06.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.913+0000 7fd113fff640 1 --2- 172.21.15.42:0/2175535023 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55740eda21b0 0x55740f1790a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:16:06.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.913+0000 7fd118c9b640 1 --2- 172.21.15.42:0/2175535023 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55740edaa390 0x55740f16d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:16:06.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.913+0000 7fd113fff640 1 -- 172.21.15.42:0/2175535023 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55740eda21b0 msgr2=0x55740f1790a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:16:06.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.913+0000 7fd113fff640 1 --2- 172.21.15.42:0/2175535023 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55740eda21b0 0x55740f1790a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:06.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.913+0000 7fd113fff640 1 -- 172.21.15.42:0/2175535023 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55740edaa390 msgr2=0x55740f16d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:16:06.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.913+0000 7fd113fff640 1 --2- 172.21.15.42:0/2175535023 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55740edaa390 0x55740f16d7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:06.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.913+0000 7fd113fff640 1 -- 172.21.15.42:0/2175535023 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55740f1817d0 con 0x55740edcb070 2024-08-30T09:16:06.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.913+0000 7fd118c9b640 1 --2- 172.21.15.42:0/2175535023 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55740edaa390 0x55740f16d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-30T09:16:06.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.913+0000 7fd113fff640 1 --2- 172.21.15.42:0/2175535023 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55740edcb070 0x55740f16dd10 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fd10000bfd0 tx=0x7fd10000b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c78313323eb97f97 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:16:06.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.913+0000 7fd0fe7fc640 1 -- 172.21.15.42:0/2175535023 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd100023070 con 0x55740edcb070 2024-08-30T09:16:06.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.913+0000 7fd0fe7fc640 1 -- 172.21.15.42:0/2175535023 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd100016040 con 0x55740edcb070 2024-08-30T09:16:06.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.913+0000 7fd0fe7fc640 1 -- 172.21.15.42:0/2175535023 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd100004730 con 0x55740edcb070 2024-08-30T09:16:06.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.914+0000 7fd121474740 1 -- 172.21.15.42:0/2175535023 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55740edcb070 msgr2=0x55740f16dd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:16:06.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.914+0000 7fd121474740 1 --2- 172.21.15.42:0/2175535023 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55740edcb070 0x55740f16dd10 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fd10000bfd0 tx=0x7fd10000b390 comp rx=0 tx=0).stop 2024-08-30T09:16:06.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.914+0000 7fd121474740 1 -- 172.21.15.42:0/2175535023 shutdown_connections 2024-08-30T09:16:06.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.914+0000 7fd121474740 1 --2- 172.21.15.42:0/2175535023 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55740edcb070 0x55740f16dd10 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:06.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.914+0000 7fd121474740 1 --2- 172.21.15.42:0/2175535023 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55740edaa390 0x55740f16d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:06.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.914+0000 7fd121474740 1 --2- 172.21.15.42:0/2175535023 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55740eda21b0 0x55740f1790a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:06.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.914+0000 7fd121474740 1 -- 172.21.15.42:0/2175535023 >> 172.21.15.42:0/2175535023 conn(0x55740edbf460 msgr2=0x55740f179bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:16:06.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.914+0000 7fd121474740 1 -- 172.21.15.42:0/2175535023 shutdown_connections 2024-08-30T09:16:06.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.914+0000 7fd121474740 1 -- 172.21.15.42:0/2175535023 wait complete. 2024-08-30T09:16:06.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.914+0000 7fd121474740 1 -- start start 2024-08-30T09:16:06.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.914+0000 7fd121474740 1 Processor -- start 2024-08-30T09:16:06.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.915+0000 7fd121474740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55740eda21b0 0x55740f186d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:16:06.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.915+0000 7fd121474740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55740edaa390 0x55740f187280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:16:06.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.915+0000 7fd121474740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55740edcb070 0x55740f1877c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:16:06.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.915+0000 7fd121474740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fd1141acd00 con 0x55740edaa390 2024-08-30T09:16:06.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.915+0000 7fd121474740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd1141ae170 con 0x55740eda21b0 2024-08-30T09:16:06.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.915+0000 7fd121474740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fd1141ad3d0 con 0x55740edcb070 2024-08-30T09:16:06.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.915+0000 7fd121474740 10 client.0 _ll_register_callbacks cb 0x7fd108005e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-30T09:16:06.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.915+0000 7fd118c9b640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55740edaa390 0x55740f187280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:16:06.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.915+0000 7fd118c9b640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55740edaa390 0x55740f187280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33058/0 (socket says 172.21.15.42:33058) 2024-08-30T09:16:06.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.915+0000 7fd118c9b640 1 -- 172.21.15.42:0/3101913005 learned_addr learned my addr 172.21.15.42:0/3101913005 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:16:06.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.915+0000 7fd113fff640 1 --2- 172.21.15.42:0/3101913005 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55740edcb070 0x55740f1877c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:16:06.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.915+0000 7fd118c9b640 1 -- 172.21.15.42:0/3101913005 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55740edcb070 msgr2=0x55740f1877c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:16:06.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.915+0000 7fd118c9b640 1 --2- 172.21.15.42:0/3101913005 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55740edcb070 0x55740f1877c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:06.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.915+0000 7fd118c9b640 1 -- 172.21.15.42:0/3101913005 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55740eda21b0 msgr2=0x55740f186d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-30T09:16:06.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.915+0000 7fd118c9b640 1 --2- 172.21.15.42:0/3101913005 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55740eda21b0 0x55740f186d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:06.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.915+0000 7fd118c9b640 1 -- 172.21.15.42:0/3101913005 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55740f18be90 con 0x55740edaa390 2024-08-30T09:16:06.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.915+0000 7fd113fff640 1 --2- 172.21.15.42:0/3101913005 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55740edcb070 0x55740f1877c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-30T09:16:06.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.916+0000 7fd118c9b640 1 --2- 172.21.15.42:0/3101913005 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55740edaa390 0x55740f187280 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7fd10802a930 tx=0x7fd10802ab80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95a54be3d7a000f3 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:16:06.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.916+0000 7fd0f3fff640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-08-30T09:16:06.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.916+0000 7fd0f3fff640 1 -- 172.21.15.42:0/3101913005 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd108058060 con 0x55740edaa390 2024-08-30T09:16:06.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.916+0000 7fd0f3fff640 1 -- 172.21.15.42:0/3101913005 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7fd1080046c0 con 0x55740edaa390 2024-08-30T09:16:06.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.916+0000 7fd121474740 1 -- 172.21.15.42:0/3101913005 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd100016040 con 0x55740edaa390 2024-08-30T09:16:06.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.916+0000 7fd0f3fff640 1 -- 172.21.15.42:0/3101913005 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7fd108035070 con 0x55740edaa390 2024-08-30T09:16:06.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.916+0000 7fd121474740 10 client.18000 fetch_fsmap learned FSMap version 50 2024-08-30T09:16:06.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.916+0000 7fd121474740 1 -- 172.21.15.42:0/3101913005 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x55740f1fce50 con 0x55740edaa390 2024-08-30T09:16:06.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.916+0000 7fd0f3fff640 1 -- 172.21.15.42:0/3101913005 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd100016040 con 0x55740edaa390 2024-08-30T09:16:06.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.916+0000 7fd0f3fff640 1 -- 172.21.15.42:0/3101913005 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 50) ==== 8366+0+0 (secure 0 0 0) 0x7fd108046590 con 0x55740edaa390 2024-08-30T09:16:06.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.916+0000 7fd121474740 10 client.18000 fetch_fsmap finished waiting for FSMap version 50 2024-08-30T09:16:06.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.916+0000 7fd121474740 10 client.18000 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-30T09:16:06.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.916+0000 7fd121474740 20 client.18000 populate_metadata read hostname 'smithi042' 2024-08-30T09:16:06.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.917+0000 7fd121474740 1 --2- 172.21.15.42:0/3101913005 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55740ed5eda0 0x55740f1ff640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:16:06.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.917+0000 7fd121474740 4 client.18000 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-30T09:16:06.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.917+0000 7fd121474740 1 -- 172.21.15.42:0/3101913005 --> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55740f0e5070 con 0x55740ed5eda0 2024-08-30T09:16:06.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.917+0000 7fd113fff640 1 --2- 172.21.15.42:0/3101913005 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55740ed5eda0 0x55740f1ff640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:16:06.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.917+0000 7fd113fff640 1 --2- 172.21.15.42:0/3101913005 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55740ed5eda0 0x55740f1ff640 crc :-1 s=READY pgs=310 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a215d6d4074148bb server_cookie=97850b1561239bae in_seq=0 out_seq=0 2024-08-30T09:16:06.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.917+0000 7fd0f3fff640 10 client.18000 ms_handle_connect on v2:172.21.15.167:6838/3604645207 2024-08-30T09:16:06.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.918+0000 7fd0f3fff640 1 -- 172.21.15.42:0/3101913005 <== mds.0 v2:172.21.15.167:6838/3604645207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55740f0e5070 con 0x55740ed5eda0 2024-08-30T09:16:06.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.918+0000 7fd0f3fff640 10 client.18000 handle_command_reply: tid=0 2024-08-30T09:16:06.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.918+0000 7fd121474740 1 client.18000 shutdown 2024-08-30T09:16:06.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.918+0000 7fd121474740 1 -- 172.21.15.42:0/3101913005 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55740edaa390 msgr2=0x55740f187280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:16:06.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.918+0000 7fd121474740 1 --2- 172.21.15.42:0/3101913005 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55740edaa390 0x55740f187280 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7fd10802a930 tx=0x7fd10802ab80 comp rx=0 tx=0).stop 2024-08-30T09:16:06.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.918+0000 7fd121474740 1 -- 172.21.15.42:0/3101913005 shutdown_connections 2024-08-30T09:16:06.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.918+0000 7fd121474740 1 --2- 172.21.15.42:0/3101913005 >> [v2:172.21.15.167:6838/3604645207,v1:172.21.15.167:6839/3604645207] conn(0x55740ed5eda0 0x55740f1ff640 crc :-1 s=READY pgs=310 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:06.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.918+0000 7fd121474740 1 --2- 172.21.15.42:0/3101913005 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55740edcb070 0x55740f1877c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:06.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.918+0000 7fd121474740 1 --2- 172.21.15.42:0/3101913005 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55740edaa390 0x55740f187280 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:06.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.918+0000 7fd121474740 1 --2- 172.21.15.42:0/3101913005 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55740eda21b0 0x55740f186d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:06.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.918+0000 7fd121474740 1 -- 172.21.15.42:0/3101913005 >> 172.21.15.42:0/3101913005 conn(0x55740edbf460 msgr2=0x55740f17ed50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:16:06.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.918+0000 7fd0f3fff640 0 client.18000 ms_handle_reset on v2:172.21.15.167:6838/3604645207 2024-08-30T09:16:06.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.918+0000 7fd121474740 1 -- 172.21.15.42:0/3101913005 shutdown_connections 2024-08-30T09:16:06.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.918+0000 7fd121474740 1 -- 172.21.15.42:0/3101913005 wait complete. 2024-08-30T09:16:06.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.918+0000 7fd121474740 20 client.18000 trim_cache size 0 max 16384 2024-08-30T09:16:06.926 INFO:teuthology.orchestra.run.smithi042.stdout:[] 2024-08-30T09:16:06.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.919+0000 7fd11aeff640 1 -- 172.21.15.42:0/2982026364 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fd1040b4bb0 msgr2=0x7fd1040b70c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:16:06.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.919+0000 7fd11aeff640 1 --2- 172.21.15.42:0/2982026364 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fd1040b4bb0 0x7fd1040b70c0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fd10c000f80 tx=0x7fd10c004a40 comp rx=0 tx=0).stop 2024-08-30T09:16:06.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.919+0000 7fd11aeff640 1 -- 172.21.15.42:0/2982026364 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd114108ab0 msgr2=0x7fd11419e9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:16:06.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.919+0000 7fd11aeff640 1 --2- 172.21.15.42:0/2982026364 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd114108ab0 0x7fd11419e9e0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fd10c00b970 tx=0x7fd10c00be30 comp rx=0 tx=0).stop 2024-08-30T09:16:06.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.919+0000 7fd11aeff640 1 -- 172.21.15.42:0/2982026364 shutdown_connections 2024-08-30T09:16:06.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.919+0000 7fd11aeff640 1 --2- 172.21.15.42:0/2982026364 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fd1040b4bb0 0x7fd1040b70c0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:06.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.919+0000 7fd11aeff640 1 --2- 172.21.15.42:0/2982026364 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd114108ab0 0x7fd11419e9e0 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:06.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.919+0000 7fd11aeff640 1 --2- 172.21.15.42:0/2982026364 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd114108160 0x7fd114199740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:06.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.919+0000 7fd11aeff640 1 --2- 172.21.15.42:0/2982026364 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd114102820 0x7fd1141991a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:06.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.919+0000 7fd11aeff640 1 -- 172.21.15.42:0/2982026364 >> 172.21.15.42:0/2982026364 conn(0x7fd1140794e0 msgr2=0x7fd1141011c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:16:06.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.919+0000 7fd11aeff640 1 -- 172.21.15.42:0/2982026364 shutdown_connections 2024-08-30T09:16:06.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:06.919+0000 7fd11aeff640 1 -- 172.21.15.42:0/2982026364 wait complete. 2024-08-30T09:16:07.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:16:07 smithi042 ceph-mon[18652]: pgmap v833: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 4.3 MiB/s wr, 512 op/s 2024-08-30T09:16:07.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:16:07 smithi042 ceph-mon[18652]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-30T09:16:07.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:16:07 smithi167 ceph-mon[22063]: pgmap v833: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 4.3 MiB/s wr, 512 op/s 2024-08-30T09:16:07.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:16:07 smithi167 ceph-mon[22063]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-30T09:16:07.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:16:07 smithi190 ceph-mon[22001]: pgmap v833: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 4.3 MiB/s wr, 512 op/s 2024-08-30T09:16:07.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:16:07 smithi190 ceph-mon[22001]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-30T09:16:07.592 ERROR:tasks.fwd_scrub.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_d19e6118f7015f0047264006cb0c873528eb3ae6/qa/tasks/fwd_scrub.py", line 35, in _run self.do_scrub() File "/home/teuthworker/src/git.ceph.com_ceph-c_d19e6118f7015f0047264006cb0c873528eb3ae6/qa/tasks/fwd_scrub.py", line 49, in do_scrub self.sleep_unless_stopped(self.sleep_between_iterations) File "/home/teuthworker/src/git.ceph.com_ceph-c_d19e6118f7015f0047264006cb0c873528eb3ae6/qa/tasks/thrasher.py", line 50, in sleep_unless_stopped raise self.Stopped() tasks.thrasher.ThrasherGreenlet.Stopped 2024-08-30T09:16:07.593 INFO:tasks.fwd_scrub:done joining 2024-08-30T09:16:07.593 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-08-30T09:16:07.605 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-30T09:16:07.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:07.722+0000 7f0663c07640 1 -- 172.21.15.42:0/1934691567 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f065c102310 msgr2=0x7f065c102780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:16:07.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:07.722+0000 7f0663c07640 1 --2- 172.21.15.42:0/1934691567 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f065c102310 0x7f065c102780 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f0658009900 tx=0x7f065802a410 comp rx=0 tx=0).stop 2024-08-30T09:16:07.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:07.722+0000 7f0663c07640 1 -- 172.21.15.42:0/1934691567 shutdown_connections 2024-08-30T09:16:07.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:07.722+0000 7f0663c07640 1 --2- 172.21.15.42:0/1934691567 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f065c103390 0x7f065c1037a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:07.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:07.722+0000 7f0663c07640 1 --2- 172.21.15.42:0/1934691567 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f065c103d30 0x7f065c10a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:07.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:07.722+0000 7f0663c07640 1 --2- 172.21.15.42:0/1934691567 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f065c102310 0x7f065c102780 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:07.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:07.722+0000 7f0663c07640 1 -- 172.21.15.42:0/1934691567 >> 172.21.15.42:0/1934691567 conn(0x7f065c0fdc50 msgr2=0x7f065c100100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:16:07.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:07.722+0000 7f0663c07640 1 -- 172.21.15.42:0/1934691567 shutdown_connections 2024-08-30T09:16:07.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:07.722+0000 7f0663c07640 1 -- 172.21.15.42:0/1934691567 wait complete. 2024-08-30T09:16:07.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:07.722+0000 7f0663c07640 1 Processor -- start 2024-08-30T09:16:07.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:07.722+0000 7f0663c07640 1 -- start start 2024-08-30T09:16:07.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:07.723+0000 7f0663c07640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f065c102310 0x7f065c198c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:16:07.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:07.723+0000 7f0663c07640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f065c103390 0x7f065c1992a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:16:07.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:07.723+0000 7f0663c07640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f065c103d30 0x7f065c199e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:16:07.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:07.723+0000 7f0663c07640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f065c10dfb0 con 0x7f065c103390 2024-08-30T09:16:07.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:07.723+0000 7f0663c07640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f065c10de30 con 0x7f065c102310 2024-08-30T09:16:07.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:07.723+0000 7f0663c07640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f065c10e130 con 0x7f065c103d30 2024-08-30T09:16:07.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:07.723+0000 7f06611a2640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f065c103390 0x7f065c1992a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:16:07.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:07.723+0000 7f06611a2640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f065c103390 0x7f065c1992a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33068/0 (socket says 172.21.15.42:33068) 2024-08-30T09:16:07.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:07.723+0000 7f06611a2640 1 -- 172.21.15.42:0/2738031979 learned_addr learned my addr 172.21.15.42:0/2738031979 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:16:07.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:07.723+0000 7f06619a3640 1 --2- 172.21.15.42:0/2738031979 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f065c102310 0x7f065c198c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:16:07.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:07.723+0000 7f06621a4640 1 --2- 172.21.15.42:0/2738031979 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f065c103d30 0x7f065c199e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:16:07.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:07.723+0000 7f06611a2640 1 -- 172.21.15.42:0/2738031979 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f065c103d30 msgr2=0x7f065c199e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:16:07.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:07.723+0000 7f06611a2640 1 --2- 172.21.15.42:0/2738031979 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f065c103d30 0x7f065c199e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:07.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:07.723+0000 7f06611a2640 1 -- 172.21.15.42:0/2738031979 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f065c102310 msgr2=0x7f065c198c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:16:07.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:07.723+0000 7f06611a2640 1 --2- 172.21.15.42:0/2738031979 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f065c102310 0x7f065c198c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:07.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:07.723+0000 7f06611a2640 1 -- 172.21.15.42:0/2738031979 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f065c19a5e0 con 0x7f065c103390 2024-08-30T09:16:07.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:07.723+0000 7f06611a2640 1 --2- 172.21.15.42:0/2738031979 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f065c103390 0x7f065c1992a0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f064c00b4a0 tx=0x7f064c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:16:07.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:07.723+0000 7f0652ffd640 1 -- 172.21.15.42:0/2738031979 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f064c014070 con 0x7f065c103390 2024-08-30T09:16:07.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:07.723+0000 7f0652ffd640 1 -- 172.21.15.42:0/2738031979 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f064c010ac0 con 0x7f065c103390 2024-08-30T09:16:07.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:07.723+0000 7f0663c07640 1 -- 172.21.15.42:0/2738031979 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f065c19a8d0 con 0x7f065c103390 2024-08-30T09:16:07.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:07.723+0000 7f0652ffd640 1 -- 172.21.15.42:0/2738031979 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f064c017c30 con 0x7f065c103390 2024-08-30T09:16:07.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:07.723+0000 7f0663c07640 1 -- 172.21.15.42:0/2738031979 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f065c1a0ef0 con 0x7f065c103390 2024-08-30T09:16:07.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:07.724+0000 7f0663c07640 1 -- 172.21.15.42:0/2738031979 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f065c1094e0 con 0x7f065c103390 2024-08-30T09:16:07.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:07.727+0000 7f0652ffd640 1 -- 172.21.15.42:0/2738031979 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7f064c017070 con 0x7f065c103390 2024-08-30T09:16:07.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:07.728+0000 7f0652ffd640 1 --2- 172.21.15.42:0/2738031979 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f06480b4b30 0x7f06480b7040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:16:07.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:07.728+0000 7f0652ffd640 1 -- 172.21.15.42:0/2738031979 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f064c0e4c60 con 0x7f065c103390 2024-08-30T09:16:07.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:07.728+0000 7f06619a3640 1 --2- 172.21.15.42:0/2738031979 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f06480b4b30 0x7f06480b7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:16:07.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:07.728+0000 7f06619a3640 1 --2- 172.21.15.42:0/2738031979 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f06480b4b30 0x7f06480b7040 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f0658031440 tx=0x7f06580314a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:16:07.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:07.731+0000 7f0652ffd640 1 -- 172.21.15.42:0/2738031979 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f064c0ab9a0 con 0x7f065c103390 2024-08-30T09:16:07.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:07.918+0000 7f0663c07640 1 -- 172.21.15.42:0/2738031979 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f065c02e600 con 0x7f065c103390 2024-08-30T09:16:07.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:07.919+0000 7f0652ffd640 1 -- 172.21.15.42:0/2738031979 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+24867 (secure 0 0 0) 0x7f064c0ab2a0 con 0x7f065c103390 2024-08-30T09:16:07.923 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-08-30T09:16:07.923 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":83,"fsid":"b0895c1a-66ac-11ef-bcd4-c7b262605968","created":"2024-08-30T08:49:44.858265+0000","modified":"2024-08-30T08:59:46.645920+0000","last_up_change":"2024-08-30T08:57:12.367791+0000","last_in_change":"2024-08-30T08:56:50.274678+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-30T08:53:27.297022+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-30T08:57:58.489964+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-30T08:58:01.064783+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-08-30T08:58:07.416534+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"029d62d7-a6c6-4905-8c76-6dffa9d57b92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":2617890537},{"type":"v1","addr":"172.21.15.42:6803","nonce":2617890537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":2617890537},{"type":"v1","addr":"172.21.15.42:6805","nonce":2617890537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2617890537},{"type":"v1","addr":"172.21.15.42:6809","nonce":2617890537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":2617890537},{"type":"v1","addr":"172.21.15.42:6807","nonce":2617890537}]},"public_addr":"172.21.15.42:6803/2617890537","cluster_addr":"172.21.15.42:6805/2617890537","heartbeat_back_addr":"172.21.15.42:6809/2617890537","heartbeat_front_addr":"172.21.15.42:6807/2617890537","state":["exists","up"]},{"osd":1,"uuid":"3a78384b-0207-41df-9384-22665007d5b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":2791960082},{"type":"v1","addr":"172.21.15.167:6801","nonce":2791960082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":2791960082},{"type":"v1","addr":"172.21.15.167:6803","nonce":2791960082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":2791960082},{"type":"v1","addr":"172.21.15.167:6807","nonce":2791960082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":2791960082},{"type":"v1","addr":"172.21.15.167:6805","nonce":2791960082}]},"public_addr":"172.21.15.167:6801/2791960082","cluster_addr":"172.21.15.167:6803/2791960082","heartbeat_back_addr":"172.21.15.167:6807/2791960082","heartbeat_front_addr":"172.21.15.167:6805/2791960082","state":["exists","up"]},{"osd":2,"uuid":"16001809-c27c-4f52-948a-9998261acb4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":1101326404},{"type":"v1","addr":"172.21.15.190:6801","nonce":1101326404}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":1101326404},{"type":"v1","addr":"172.21.15.190:6803","nonce":1101326404}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":1101326404},{"type":"v1","addr":"172.21.15.190:6807","nonce":1101326404}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":1101326404},{"type":"v1","addr":"172.21.15.190:6805","nonce":1101326404}]},"public_addr":"172.21.15.190:6801/1101326404","cluster_addr":"172.21.15.190:6803/1101326404","heartbeat_back_addr":"172.21.15.190:6807/1101326404","heartbeat_front_addr":"172.21.15.190:6805/1101326404","state":["exists","up"]},{"osd":3,"uuid":"87bb3e36-9e9e-4b19-9d8c-0cdf6e089d3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":1135599916},{"type":"v1","addr":"172.21.15.42:6811","nonce":1135599916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":1135599916},{"type":"v1","addr":"172.21.15.42:6813","nonce":1135599916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":1135599916},{"type":"v1","addr":"172.21.15.42:6817","nonce":1135599916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":1135599916},{"type":"v1","addr":"172.21.15.42:6815","nonce":1135599916}]},"public_addr":"172.21.15.42:6811/1135599916","cluster_addr":"172.21.15.42:6813/1135599916","heartbeat_back_addr":"172.21.15.42:6817/1135599916","heartbeat_front_addr":"172.21.15.42:6815/1135599916","state":["exists","up"]},{"osd":4,"uuid":"3ca1aae3-c972-4fb9-ac86-236279c30d64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":2927511419},{"type":"v1","addr":"172.21.15.167:6809","nonce":2927511419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":2927511419},{"type":"v1","addr":"172.21.15.167:6811","nonce":2927511419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":2927511419},{"type":"v1","addr":"172.21.15.167:6815","nonce":2927511419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":2927511419},{"type":"v1","addr":"172.21.15.167:6813","nonce":2927511419}]},"public_addr":"172.21.15.167:6809/2927511419","cluster_addr":"172.21.15.167:6811/2927511419","heartbeat_back_addr":"172.21.15.167:6815/2927511419","heartbeat_front_addr":"172.21.15.167:6813/2927511419","state":["exists","up"]},{"osd":5,"uuid":"5d07f54b-0d07-4efe-9bcf-390755ce0258","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":2557145333},{"type":"v1","addr":"172.21.15.190:6809","nonce":2557145333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":2557145333},{"type":"v1","addr":"172.21.15.190:6811","nonce":2557145333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":2557145333},{"type":"v1","addr":"172.21.15.190:6815","nonce":2557145333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":2557145333},{"type":"v1","addr":"172.21.15.190:6813","nonce":2557145333}]},"public_addr":"172.21.15.190:6809/2557145333","cluster_addr":"172.21.15.190:6811/2557145333","heartbeat_back_addr":"172.21.15.190:6815/2557145333","heartbeat_front_addr":"172.21.15.190:6813/2557145333","state":["exists","up"]},{"osd":6,"uuid":"96ad64af-44ff-4197-97d4-156f0cb88e81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":3213526100},{"type":"v1","addr":"172.21.15.42:6819","nonce":3213526100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":3213526100},{"type":"v1","addr":"172.21.15.42:6821","nonce":3213526100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":3213526100},{"type":"v1","addr":"172.21.15.42:6825","nonce":3213526100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":3213526100},{"type":"v1","addr":"172.21.15.42:6823","nonce":3213526100}]},"public_addr":"172.21.15.42:6819/3213526100","cluster_addr":"172.21.15.42:6821/3213526100","heartbeat_back_addr":"172.21.15.42:6825/3213526100","heartbeat_front_addr":"172.21.15.42:6823/3213526100","state":["exists","up"]},{"osd":7,"uuid":"337a2112-eaf7-4e6b-86be-4e40d6ddf835","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":2962799814},{"type":"v1","addr":"172.21.15.167:6817","nonce":2962799814}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":2962799814},{"type":"v1","addr":"172.21.15.167:6819","nonce":2962799814}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":2962799814},{"type":"v1","addr":"172.21.15.167:6823","nonce":2962799814}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":2962799814},{"type":"v1","addr":"172.21.15.167:6821","nonce":2962799814}]},"public_addr":"172.21.15.167:6817/2962799814","cluster_addr":"172.21.15.167:6819/2962799814","heartbeat_back_addr":"172.21.15.167:6823/2962799814","heartbeat_front_addr":"172.21.15.167:6821/2962799814","state":["exists","up"]},{"osd":8,"uuid":"219a7224-4331-464b-ac86-c51054d51d91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":3242198843},{"type":"v1","addr":"172.21.15.190:6817","nonce":3242198843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":3242198843},{"type":"v1","addr":"172.21.15.190:6819","nonce":3242198843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":3242198843},{"type":"v1","addr":"172.21.15.190:6823","nonce":3242198843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":3242198843},{"type":"v1","addr":"172.21.15.190:6821","nonce":3242198843}]},"public_addr":"172.21.15.190:6817/3242198843","cluster_addr":"172.21.15.190:6819/3242198843","heartbeat_back_addr":"172.21.15.190:6823/3242198843","heartbeat_front_addr":"172.21.15.190:6821/3242198843","state":["exists","up"]},{"osd":9,"uuid":"2f2f5177-a009-40d9-991c-76aec8a5cba6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":1123536325},{"type":"v1","addr":"172.21.15.42:6827","nonce":1123536325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":1123536325},{"type":"v1","addr":"172.21.15.42:6829","nonce":1123536325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":1123536325},{"type":"v1","addr":"172.21.15.42:6833","nonce":1123536325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":1123536325},{"type":"v1","addr":"172.21.15.42:6831","nonce":1123536325}]},"public_addr":"172.21.15.42:6827/1123536325","cluster_addr":"172.21.15.42:6829/1123536325","heartbeat_back_addr":"172.21.15.42:6833/1123536325","heartbeat_front_addr":"172.21.15.42:6831/1123536325","state":["exists","up"]},{"osd":10,"uuid":"96ef5058-2518-4862-a165-17666b33a3e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":3451198319},{"type":"v1","addr":"172.21.15.167:6825","nonce":3451198319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":3451198319},{"type":"v1","addr":"172.21.15.167:6827","nonce":3451198319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":3451198319},{"type":"v1","addr":"172.21.15.167:6831","nonce":3451198319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":3451198319},{"type":"v1","addr":"172.21.15.167:6829","nonce":3451198319}]},"public_addr":"172.21.15.167:6825/3451198319","cluster_addr":"172.21.15.167:6827/3451198319","heartbeat_back_addr":"172.21.15.167:6831/3451198319","heartbeat_front_addr":"172.21.15.167:6829/3451198319","state":["exists","up"]},{"osd":11,"uuid":"a4eb9bd3-3faa-4c81-ba6d-c2084b72c9ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":380064457},{"type":"v1","addr":"172.21.15.190:6825","nonce":380064457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":380064457},{"type":"v1","addr":"172.21.15.190:6827","nonce":380064457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":380064457},{"type":"v1","addr":"172.21.15.190:6831","nonce":380064457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":380064457},{"type":"v1","addr":"172.21.15.190:6829","nonce":380064457}]},"public_addr":"172.21.15.190:6825/380064457","cluster_addr":"172.21.15.190:6827/380064457","heartbeat_back_addr":"172.21.15.190:6831/380064457","heartbeat_front_addr":"172.21.15.190:6829/380064457","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-30T08:52:36.511448+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-30T08:53:00.027277+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-30T08:53:23.770245+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-30T08:53:49.322226+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-30T08:54:14.350234+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-30T08:54:39.491793+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-30T08:55:04.870596+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-30T08:55:30.314150+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-30T08:55:54.483843+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-30T08:56:20.586115+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-30T08:56:45.666783+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-30T08:57:10.718387+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"2.1d","mappings":[{"from":2,"to":1}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"4.9","mappings":[{"from":5,"to":8}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.42:0/2691233260":"2024-08-31T08:50:44.821070+0000","172.21.15.42:0/1518053351":"2024-08-31T08:50:44.821070+0000","172.21.15.42:0/352001020":"2024-08-31T08:50:44.821070+0000","172.21.15.42:6801/1733768634":"2024-08-31T08:50:44.821070+0000","172.21.15.42:0/1000597888":"2024-08-31T08:50:18.561851+0000","172.21.15.42:6800/1733768634":"2024-08-31T08:50:44.821070+0000","172.21.15.42:6801/3538882081":"2024-08-31T08:50:18.561851+0000","172.21.15.42:6800/3538882081":"2024-08-31T08:50:18.561851+0000","172.21.15.42:0/145366508":"2024-08-31T08:50:18.561851+0000","172.21.15.42:0/3346355213":"2024-08-31T08:50:18.561851+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-30T09:16:07.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:07.922+0000 7f0663c07640 1 -- 172.21.15.42:0/2738031979 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f06480b4b30 msgr2=0x7f06480b7040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:16:07.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:07.922+0000 7f0663c07640 1 --2- 172.21.15.42:0/2738031979 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f06480b4b30 0x7f06480b7040 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f0658031440 tx=0x7f06580314a0 comp rx=0 tx=0).stop 2024-08-30T09:16:07.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:07.922+0000 7f0663c07640 1 -- 172.21.15.42:0/2738031979 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f065c103390 msgr2=0x7f065c1992a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:16:07.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:07.922+0000 7f0663c07640 1 --2- 172.21.15.42:0/2738031979 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f065c103390 0x7f065c1992a0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f064c00b4a0 tx=0x7f064c00b960 comp rx=0 tx=0).stop 2024-08-30T09:16:07.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:07.922+0000 7f0663c07640 1 -- 172.21.15.42:0/2738031979 shutdown_connections 2024-08-30T09:16:07.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:07.922+0000 7f0663c07640 1 --2- 172.21.15.42:0/2738031979 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f06480b4b30 0x7f06480b7040 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:07.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:07.922+0000 7f0663c07640 1 --2- 172.21.15.42:0/2738031979 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f065c103d30 0x7f065c199e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:07.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:07.922+0000 7f0663c07640 1 --2- 172.21.15.42:0/2738031979 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f065c103390 0x7f065c1992a0 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:07.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:07.922+0000 7f0663c07640 1 --2- 172.21.15.42:0/2738031979 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f065c102310 0x7f065c198c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:07.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:07.922+0000 7f0663c07640 1 -- 172.21.15.42:0/2738031979 >> 172.21.15.42:0/2738031979 conn(0x7f065c0fdc50 msgr2=0x7f065c1066b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:16:07.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:07.922+0000 7f0663c07640 1 -- 172.21.15.42:0/2738031979 shutdown_connections 2024-08-30T09:16:07.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:07.922+0000 7f0663c07640 1 -- 172.21.15.42:0/2738031979 wait complete. 2024-08-30T09:16:07.937 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-08-30T08:53:27.297022+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-08-30T08:57:58.489964+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '73', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-08-30T08:58:01.064783+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '73', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-08-30T08:58:07.416534+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-08-30T09:16:07.937 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-08-30T09:16:08.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:08.049+0000 7fe60fa0a640 1 -- 172.21.15.42:0/3622362244 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe6081091f0 msgr2=0x7fe60810b690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:16:08.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:08.049+0000 7fe60fa0a640 1 --2- 172.21.15.42:0/3622362244 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe6081091f0 0x7fe60810b690 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7fe600009ca0 tx=0x7fe60002a8c0 comp rx=0 tx=0).stop 2024-08-30T09:16:08.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:08.049+0000 7fe60fa0a640 1 -- 172.21.15.42:0/3622362244 shutdown_connections 2024-08-30T09:16:08.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:08.049+0000 7fe60fa0a640 1 --2- 172.21.15.42:0/3622362244 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe608106890 0x7fe608108cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:08.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:08.049+0000 7fe60fa0a640 1 --2- 172.21.15.42:0/3622362244 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe6081091f0 0x7fe60810b690 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:08.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:08.049+0000 7fe60fa0a640 1 --2- 172.21.15.42:0/3622362244 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe608101800 0x7fe608101c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:08.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:08.049+0000 7fe60fa0a640 1 -- 172.21.15.42:0/3622362244 >> 172.21.15.42:0/3622362244 conn(0x7fe6080fd400 msgr2=0x7fe6080ff8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:16:08.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:08.050+0000 7fe60fa0a640 1 -- 172.21.15.42:0/3622362244 shutdown_connections 2024-08-30T09:16:08.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:08.050+0000 7fe60fa0a640 1 -- 172.21.15.42:0/3622362244 wait complete. 2024-08-30T09:16:08.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:08.050+0000 7fe60fa0a640 1 Processor -- start 2024-08-30T09:16:08.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:08.050+0000 7fe60fa0a640 1 -- start start 2024-08-30T09:16:08.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:08.050+0000 7fe60fa0a640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe608101800 0x7fe608103ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:16:08.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:08.050+0000 7fe60fa0a640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe608106890 0x7fe608102790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:16:08.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:08.050+0000 7fe60fa0a640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe6081091f0 0x7fe608102cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:16:08.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:08.050+0000 7fe60fa0a640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fe60810f0f0 con 0x7fe608101800 2024-08-30T09:16:08.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:08.050+0000 7fe60fa0a640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe60810ef70 con 0x7fe6081091f0 2024-08-30T09:16:08.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:08.050+0000 7fe60fa0a640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fe60810f270 con 0x7fe608106890 2024-08-30T09:16:08.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:08.051+0000 7fe60d7a6640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe608101800 0x7fe608103ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:16:08.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:08.051+0000 7fe60d7a6640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe608101800 0x7fe608103ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34352/0 (socket says 172.21.15.42:34352) 2024-08-30T09:16:08.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:08.051+0000 7fe60d7a6640 1 -- 172.21.15.42:0/3044014488 learned_addr learned my addr 172.21.15.42:0/3044014488 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:16:08.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:08.051+0000 7fe60cfa5640 1 --2- 172.21.15.42:0/3044014488 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe608106890 0x7fe608102790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:16:08.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:08.051+0000 7fe60dfa7640 1 --2- 172.21.15.42:0/3044014488 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe6081091f0 0x7fe608102cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:16:08.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:08.051+0000 7fe60d7a6640 1 -- 172.21.15.42:0/3044014488 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe608106890 msgr2=0x7fe608102790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:16:08.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:08.051+0000 7fe60d7a6640 1 --2- 172.21.15.42:0/3044014488 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe608106890 0x7fe608102790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:08.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:08.051+0000 7fe60d7a6640 1 -- 172.21.15.42:0/3044014488 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe6081091f0 msgr2=0x7fe608102cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:16:08.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:08.051+0000 7fe60d7a6640 1 --2- 172.21.15.42:0/3044014488 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe6081091f0 0x7fe608102cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:08.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:08.051+0000 7fe60d7a6640 1 -- 172.21.15.42:0/3044014488 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe608103400 con 0x7fe608101800 2024-08-30T09:16:08.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:08.051+0000 7fe60cfa5640 1 --2- 172.21.15.42:0/3044014488 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe608106890 0x7fe608102790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-30T09:16:08.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:08.051+0000 7fe60d7a6640 1 --2- 172.21.15.42:0/3044014488 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe608101800 0x7fe608103ed0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7fe60400e9f0 tx=0x7fe60400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:16:08.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:08.051+0000 7fe5fe7fc640 1 -- 172.21.15.42:0/3044014488 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe60400cd80 con 0x7fe608101800 2024-08-30T09:16:08.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:08.051+0000 7fe5fe7fc640 1 -- 172.21.15.42:0/3044014488 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe6040108e0 con 0x7fe608101800 2024-08-30T09:16:08.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:08.051+0000 7fe5fe7fc640 1 -- 172.21.15.42:0/3044014488 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe604018980 con 0x7fe608101800 2024-08-30T09:16:08.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:08.051+0000 7fe60fa0a640 1 -- 172.21.15.42:0/3044014488 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6081a3cb0 con 0x7fe608101800 2024-08-30T09:16:08.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:08.051+0000 7fe60fa0a640 1 -- 172.21.15.42:0/3044014488 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6081a41a0 con 0x7fe608101800 2024-08-30T09:16:08.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:08.052+0000 7fe60fa0a640 1 -- 172.21.15.42:0/3044014488 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6080fef20 con 0x7fe608101800 2024-08-30T09:16:08.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:08.054+0000 7fe5fe7fc640 1 -- 172.21.15.42:0/3044014488 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7fe604010430 con 0x7fe608101800 2024-08-30T09:16:08.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:08.055+0000 7fe5fe7fc640 1 --2- 172.21.15.42:0/3044014488 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fe5f40b4bb0 0x7fe5f40b70c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:16:08.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:08.055+0000 7fe60cfa5640 1 --2- 172.21.15.42:0/3044014488 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fe5f40b4bb0 0x7fe5f40b70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:16:08.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:08.055+0000 7fe5fe7fc640 1 -- 172.21.15.42:0/3044014488 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7fe6040e5d90 con 0x7fe608101800 2024-08-30T09:16:08.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:08.056+0000 7fe60cfa5640 1 --2- 172.21.15.42:0/3044014488 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fe5f40b4bb0 0x7fe5f40b70c0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fe5f8009c20 tx=0x7fe5f8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:16:08.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:08.058+0000 7fe5fe7fc640 1 -- 172.21.15.42:0/3044014488 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7fe6040acb30 con 0x7fe608101800 2024-08-30T09:16:08.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:08.244+0000 7fe60fa0a640 1 -- 172.21.15.42:0/3044014488 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fe60802e680 con 0x7fe608101800 2024-08-30T09:16:08.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:08.245+0000 7fe5fe7fc640 1 -- 172.21.15.42:0/3044014488 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v83) ==== 93+0+10 (secure 0 0 0) 0x7fe6040ac430 con 0x7fe608101800 2024-08-30T09:16:08.248 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 1 2024-08-30T09:16:08.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:08.247+0000 7fe60fa0a640 1 -- 172.21.15.42:0/3044014488 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fe5f40b4bb0 msgr2=0x7fe5f40b70c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:16:08.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:08.247+0000 7fe60fa0a640 1 --2- 172.21.15.42:0/3044014488 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fe5f40b4bb0 0x7fe5f40b70c0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fe5f8009c20 tx=0x7fe5f8005f90 comp rx=0 tx=0).stop 2024-08-30T09:16:08.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:08.247+0000 7fe60fa0a640 1 -- 172.21.15.42:0/3044014488 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe608101800 msgr2=0x7fe608103ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:16:08.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:08.247+0000 7fe60fa0a640 1 --2- 172.21.15.42:0/3044014488 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe608101800 0x7fe608103ed0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7fe60400e9f0 tx=0x7fe60400eeb0 comp rx=0 tx=0).stop 2024-08-30T09:16:08.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:08.247+0000 7fe60fa0a640 1 -- 172.21.15.42:0/3044014488 shutdown_connections 2024-08-30T09:16:08.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:08.247+0000 7fe60fa0a640 1 --2- 172.21.15.42:0/3044014488 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fe5f40b4bb0 0x7fe5f40b70c0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:08.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:08.247+0000 7fe60fa0a640 1 --2- 172.21.15.42:0/3044014488 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe608106890 0x7fe608102790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:08.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:08.247+0000 7fe60fa0a640 1 --2- 172.21.15.42:0/3044014488 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe6081091f0 0x7fe608102cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:08.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:08.247+0000 7fe60fa0a640 1 --2- 172.21.15.42:0/3044014488 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe608101800 0x7fe608103ed0 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:08.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:08.247+0000 7fe60fa0a640 1 -- 172.21.15.42:0/3044014488 >> 172.21.15.42:0/3044014488 conn(0x7fe6080fd400 msgr2=0x7fe608107660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:16:08.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:08.248+0000 7fe60fa0a640 1 -- 172.21.15.42:0/3044014488 shutdown_connections 2024-08-30T09:16:08.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:08.248+0000 7fe60fa0a640 1 -- 172.21.15.42:0/3044014488 wait complete. 2024-08-30T09:16:08.261 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:16:08 smithi042 ceph-mon[18652]: from='client.? 172.21.15.42:0/2738031979' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-30T09:16:08.262 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-08-30T09:16:08.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:16:08 smithi167 ceph-mon[22063]: from='client.? 172.21.15.42:0/2738031979' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-30T09:16:08.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:08.372+0000 7f0891351640 1 -- 172.21.15.42:0/49864366 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f088c102990 msgr2=0x7f088c104e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:16:08.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:08.372+0000 7f0891351640 1 --2- 172.21.15.42:0/49864366 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f088c102990 0x7f088c104e10 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f087c009900 tx=0x7f087c02a410 comp rx=0 tx=0).stop 2024-08-30T09:16:08.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:08.372+0000 7f0891351640 1 -- 172.21.15.42:0/49864366 shutdown_connections 2024-08-30T09:16:08.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:08.372+0000 7f0891351640 1 --2- 172.21.15.42:0/49864366 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f088c105450 0x7f088c107930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:08.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:08.372+0000 7f0891351640 1 --2- 172.21.15.42:0/49864366 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f088c108070 0x7f088c10a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:08.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:08.372+0000 7f0891351640 1 --2- 172.21.15.42:0/49864366 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f088c102990 0x7f088c104e10 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:08.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:08.372+0000 7f0891351640 1 -- 172.21.15.42:0/49864366 >> 172.21.15.42:0/49864366 conn(0x7f088c0fd400 msgr2=0x7f088c0ff8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:16:08.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:08.372+0000 7f0891351640 1 -- 172.21.15.42:0/49864366 shutdown_connections 2024-08-30T09:16:08.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:08.372+0000 7f0891351640 1 -- 172.21.15.42:0/49864366 wait complete. 2024-08-30T09:16:08.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:08.373+0000 7f0891351640 1 Processor -- start 2024-08-30T09:16:08.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:08.373+0000 7f0891351640 1 -- start start 2024-08-30T09:16:08.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:08.373+0000 7f0891351640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f088c102990 0x7f088c101f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:16:08.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:08.373+0000 7f0891351640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f088c105450 0x7f088c19eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:16:08.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:08.373+0000 7f0891351640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f088c108070 0x7f088c1a1070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:16:08.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:08.373+0000 7f0891351640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f088c10e050 con 0x7f088c108070 2024-08-30T09:16:08.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:08.373+0000 7f0891351640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f088c10ded0 con 0x7f088c105450 2024-08-30T09:16:08.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:08.373+0000 7f0891351640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f088c10e1d0 con 0x7f088c102990 2024-08-30T09:16:08.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:08.374+0000 7f088b7fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f088c108070 0x7f088c1a1070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:16:08.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:08.374+0000 7f088b7fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f088c108070 0x7f088c1a1070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34370/0 (socket says 172.21.15.42:34370) 2024-08-30T09:16:08.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:08.374+0000 7f088b7fe640 1 -- 172.21.15.42:0/3300753592 learned_addr learned my addr 172.21.15.42:0/3300753592 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:16:08.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:08.374+0000 7f088a7fc640 1 --2- 172.21.15.42:0/3300753592 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f088c105450 0x7f088c19eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:16:08.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:08.374+0000 7f088affd640 1 --2- 172.21.15.42:0/3300753592 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f088c102990 0x7f088c101f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:16:08.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:08.374+0000 7f088b7fe640 1 -- 172.21.15.42:0/3300753592 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f088c102990 msgr2=0x7f088c101f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:16:08.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:08.374+0000 7f088b7fe640 1 --2- 172.21.15.42:0/3300753592 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f088c102990 0x7f088c101f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:08.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:08.374+0000 7f088b7fe640 1 -- 172.21.15.42:0/3300753592 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f088c105450 msgr2=0x7f088c19eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:16:08.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:08.374+0000 7f088b7fe640 1 --2- 172.21.15.42:0/3300753592 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f088c105450 0x7f088c19eae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:08.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:08.374+0000 7f088b7fe640 1 -- 172.21.15.42:0/3300753592 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f088c1a1890 con 0x7f088c108070 2024-08-30T09:16:08.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:08.374+0000 7f088b7fe640 1 --2- 172.21.15.42:0/3300753592 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f088c108070 0x7f088c1a1070 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f088400b970 tx=0x7f088400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:16:08.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:08.374+0000 7f0873fff640 1 -- 172.21.15.42:0/3300753592 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0884019070 con 0x7f088c108070 2024-08-30T09:16:08.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:08.374+0000 7f0891351640 1 -- 172.21.15.42:0/3300753592 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f088c1a1b80 con 0x7f088c108070 2024-08-30T09:16:08.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:08.375+0000 7f0873fff640 1 -- 172.21.15.42:0/3300753592 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0884014440 con 0x7f088c108070 2024-08-30T09:16:08.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:08.375+0000 7f0873fff640 1 -- 172.21.15.42:0/3300753592 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08840075d0 con 0x7f088c108070 2024-08-30T09:16:08.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:08.375+0000 7f0891351640 1 -- 172.21.15.42:0/3300753592 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f088c1a20d0 con 0x7f088c108070 2024-08-30T09:16:08.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:08.376+0000 7f0891351640 1 -- 172.21.15.42:0/3300753592 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f088c0fef20 con 0x7f088c108070 2024-08-30T09:16:08.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:08.378+0000 7f0873fff640 1 -- 172.21.15.42:0/3300753592 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7f088401c070 con 0x7f088c108070 2024-08-30T09:16:08.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:08.379+0000 7f0873fff640 1 --2- 172.21.15.42:0/3300753592 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f08740b4b30 0x7f08740b7040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:16:08.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:08.379+0000 7f0873fff640 1 -- 172.21.15.42:0/3300753592 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f08840e4d80 con 0x7f088c108070 2024-08-30T09:16:08.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:08.379+0000 7f088affd640 1 --2- 172.21.15.42:0/3300753592 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f08740b4b30 0x7f08740b7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:16:08.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:08.380+0000 7f088affd640 1 --2- 172.21.15.42:0/3300753592 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f08740b4b30 0x7f08740b7040 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f087c000c00 tx=0x7f087c031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:16:08.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:08.383+0000 7f0873fff640 1 -- 172.21.15.42:0/3300753592 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f08840ab4d0 con 0x7f088c108070 2024-08-30T09:16:08.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:16:08 smithi190 ceph-mon[22001]: from='client.? 172.21.15.42:0/2738031979' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-30T09:16:08.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:08.574+0000 7f0891351640 1 -- 172.21.15.42:0/3300753592 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f088c02e5c0 con 0x7f088c108070 2024-08-30T09:16:08.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:08.575+0000 7f0873fff640 1 -- 172.21.15.42:0/3300753592 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v83) ==== 104+0+11 (secure 0 0 0) 0x7f08840afc90 con 0x7f088c108070 2024-08-30T09:16:08.578 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 32 2024-08-30T09:16:08.580 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:08.577+0000 7f0891351640 1 -- 172.21.15.42:0/3300753592 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f08740b4b30 msgr2=0x7f08740b7040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:16:08.580 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:08.577+0000 7f0891351640 1 --2- 172.21.15.42:0/3300753592 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f08740b4b30 0x7f08740b7040 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f087c000c00 tx=0x7f087c031400 comp rx=0 tx=0).stop 2024-08-30T09:16:08.580 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:08.577+0000 7f0891351640 1 -- 172.21.15.42:0/3300753592 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f088c108070 msgr2=0x7f088c1a1070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:16:08.580 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:08.577+0000 7f0891351640 1 --2- 172.21.15.42:0/3300753592 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f088c108070 0x7f088c1a1070 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f088400b970 tx=0x7f088400bd70 comp rx=0 tx=0).stop 2024-08-30T09:16:08.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:08.577+0000 7f0891351640 1 -- 172.21.15.42:0/3300753592 shutdown_connections 2024-08-30T09:16:08.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:08.577+0000 7f0891351640 1 --2- 172.21.15.42:0/3300753592 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f08740b4b30 0x7f08740b7040 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:08.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:08.577+0000 7f0891351640 1 --2- 172.21.15.42:0/3300753592 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f088c108070 0x7f088c1a1070 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:08.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:08.577+0000 7f0891351640 1 --2- 172.21.15.42:0/3300753592 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f088c105450 0x7f088c19eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:08.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:08.577+0000 7f0891351640 1 --2- 172.21.15.42:0/3300753592 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f088c102990 0x7f088c101f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:08.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:08.577+0000 7f0891351640 1 -- 172.21.15.42:0/3300753592 >> 172.21.15.42:0/3300753592 conn(0x7f088c0fd400 msgr2=0x7f088c108fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:16:08.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:08.577+0000 7f0891351640 1 -- 172.21.15.42:0/3300753592 shutdown_connections 2024-08-30T09:16:08.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:08.577+0000 7f0891351640 1 -- 172.21.15.42:0/3300753592 wait complete. 2024-08-30T09:16:08.592 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-08-30T09:16:08.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:08.703+0000 7f102df39640 1 -- 172.21.15.42:0/1455800226 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1028109180 msgr2=0x7f102810b690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:16:08.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:08.703+0000 7f102df39640 1 --2- 172.21.15.42:0/1455800226 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1028109180 0x7f102810b690 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f1020009c90 tx=0x7f102002a860 comp rx=0 tx=0).stop 2024-08-30T09:16:08.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:08.703+0000 7f102df39640 1 -- 172.21.15.42:0/1455800226 shutdown_connections 2024-08-30T09:16:08.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:08.703+0000 7f102df39640 1 --2- 172.21.15.42:0/1455800226 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1028109180 0x7f102810b690 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:08.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:08.703+0000 7f102df39640 1 --2- 172.21.15.42:0/1455800226 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f10281065c0 0x7f1028108a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:08.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:08.703+0000 7f102df39640 1 --2- 172.21.15.42:0/1455800226 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f102806c190 0x7f1028106080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:08.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:08.703+0000 7f102df39640 1 -- 172.21.15.42:0/1455800226 >> 172.21.15.42:0/1455800226 conn(0x7f10280fdc80 msgr2=0x7f1028100130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:16:08.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:08.703+0000 7f102df39640 1 -- 172.21.15.42:0/1455800226 shutdown_connections 2024-08-30T09:16:08.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:08.703+0000 7f102df39640 1 -- 172.21.15.42:0/1455800226 wait complete. 2024-08-30T09:16:08.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:08.704+0000 7f102df39640 1 Processor -- start 2024-08-30T09:16:08.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:08.704+0000 7f102df39640 1 -- start start 2024-08-30T09:16:08.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:08.704+0000 7f102df39640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f102806c190 0x7f1028103650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:16:08.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:08.704+0000 7f102df39640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f10281065c0 0x7f1028101f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:16:08.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:08.704+0000 7f102df39640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1028109180 0x7f10281024c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:16:08.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:08.704+0000 7f102df39640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f102810f0a0 con 0x7f102806c190 2024-08-30T09:16:08.710 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:08.704+0000 7f102df39640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f102810ef20 con 0x7f10281065c0 2024-08-30T09:16:08.710 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:08.704+0000 7f102df39640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f102810f220 con 0x7f1028109180 2024-08-30T09:16:08.710 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:08.705+0000 7f10277fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f102806c190 0x7f1028103650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:16:08.710 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:08.705+0000 7f10277fe640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f102806c190 0x7f1028103650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34398/0 (socket says 172.21.15.42:34398) 2024-08-30T09:16:08.710 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:08.705+0000 7f10277fe640 1 -- 172.21.15.42:0/2662574099 learned_addr learned my addr 172.21.15.42:0/2662574099 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:16:08.710 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:08.705+0000 7f10277fe640 1 -- 172.21.15.42:0/2662574099 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1028109180 msgr2=0x7f10281024c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:16:08.710 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:08.705+0000 7f10277fe640 1 --2- 172.21.15.42:0/2662574099 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1028109180 0x7f10281024c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:08.710 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:08.705+0000 7f101effd640 1 --2- 172.21.15.42:0/2662574099 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f10281065c0 0x7f1028101f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:16:08.710 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:08.705+0000 7f10277fe640 1 -- 172.21.15.42:0/2662574099 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f10281065c0 msgr2=0x7f1028101f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:16:08.710 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:08.705+0000 7f10277fe640 1 --2- 172.21.15.42:0/2662574099 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f10281065c0 0x7f1028101f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:08.710 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:08.705+0000 7f10277fe640 1 -- 172.21.15.42:0/2662574099 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1028102b60 con 0x7f102806c190 2024-08-30T09:16:08.710 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:08.705+0000 7f10277fe640 1 --2- 172.21.15.42:0/2662574099 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f102806c190 0x7f1028103650 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f101800d950 tx=0x7f101800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:16:08.710 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:08.705+0000 7f10257fa640 1 -- 172.21.15.42:0/2662574099 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1018017070 con 0x7f102806c190 2024-08-30T09:16:08.710 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:08.705+0000 7f10257fa640 1 -- 172.21.15.42:0/2662574099 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1018009c30 con 0x7f102806c190 2024-08-30T09:16:08.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:08.705+0000 7f10257fa640 1 -- 172.21.15.42:0/2662574099 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f101800f620 con 0x7f102806c190 2024-08-30T09:16:08.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:08.705+0000 7f102df39640 1 -- 172.21.15.42:0/2662574099 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10281a3d90 con 0x7f102806c190 2024-08-30T09:16:08.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:08.705+0000 7f102df39640 1 -- 172.21.15.42:0/2662574099 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10281a41a0 con 0x7f102806c190 2024-08-30T09:16:08.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:08.706+0000 7f102df39640 1 -- 172.21.15.42:0/2662574099 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10280ff7a0 con 0x7f102806c190 2024-08-30T09:16:08.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:08.708+0000 7f10257fa640 1 -- 172.21.15.42:0/2662574099 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7f1018010460 con 0x7f102806c190 2024-08-30T09:16:08.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:08.709+0000 7f10257fa640 1 --2- 172.21.15.42:0/2662574099 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f10140b4b30 0x7f10140b7040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:16:08.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:08.709+0000 7f101effd640 1 --2- 172.21.15.42:0/2662574099 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f10140b4b30 0x7f10140b7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:16:08.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:08.709+0000 7f10257fa640 1 -- 172.21.15.42:0/2662574099 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f10180e5210 con 0x7f102806c190 2024-08-30T09:16:08.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:08.709+0000 7f101effd640 1 --2- 172.21.15.42:0/2662574099 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f10140b4b30 0x7f10140b7040 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f1010005fd0 tx=0x7f1010005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:16:08.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:08.712+0000 7f10257fa640 1 -- 172.21.15.42:0/2662574099 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f10180abf50 con 0x7f102806c190 2024-08-30T09:16:08.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:08.898+0000 7f102df39640 1 -- 172.21.15.42:0/2662574099 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f102802e5e0 con 0x7f102806c190 2024-08-30T09:16:08.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:08.898+0000 7f10257fa640 1 -- 172.21.15.42:0/2662574099 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v83) ==== 100+0+11 (secure 0 0 0) 0x7f1018009de0 con 0x7f102806c190 2024-08-30T09:16:08.901 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 64 2024-08-30T09:16:08.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:08.900+0000 7f102df39640 1 -- 172.21.15.42:0/2662574099 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f10140b4b30 msgr2=0x7f10140b7040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:16:08.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:08.900+0000 7f102df39640 1 --2- 172.21.15.42:0/2662574099 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f10140b4b30 0x7f10140b7040 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f1010005fd0 tx=0x7f1010005850 comp rx=0 tx=0).stop 2024-08-30T09:16:08.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:08.900+0000 7f102df39640 1 -- 172.21.15.42:0/2662574099 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f102806c190 msgr2=0x7f1028103650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:16:08.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:08.900+0000 7f102df39640 1 --2- 172.21.15.42:0/2662574099 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f102806c190 0x7f1028103650 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f101800d950 tx=0x7f101800de10 comp rx=0 tx=0).stop 2024-08-30T09:16:08.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:08.900+0000 7f102df39640 1 -- 172.21.15.42:0/2662574099 shutdown_connections 2024-08-30T09:16:08.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:08.900+0000 7f102df39640 1 --2- 172.21.15.42:0/2662574099 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f10140b4b30 0x7f10140b7040 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:08.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:08.900+0000 7f102df39640 1 --2- 172.21.15.42:0/2662574099 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1028109180 0x7f10281024c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:08.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:08.900+0000 7f102df39640 1 --2- 172.21.15.42:0/2662574099 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f10281065c0 0x7f1028101f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:08.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:08.900+0000 7f102df39640 1 --2- 172.21.15.42:0/2662574099 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f102806c190 0x7f1028103650 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:08.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:08.900+0000 7f102df39640 1 -- 172.21.15.42:0/2662574099 >> 172.21.15.42:0/2662574099 conn(0x7f10280fdc80 msgr2=0x7f1028104b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:16:08.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:08.900+0000 7f102df39640 1 -- 172.21.15.42:0/2662574099 shutdown_connections 2024-08-30T09:16:08.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:08.900+0000 7f102df39640 1 -- 172.21.15.42:0/2662574099 wait complete. 2024-08-30T09:16:08.914 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-08-30T09:16:09.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:09.025+0000 7f6d90e53640 1 -- 172.21.15.42:0/2635612290 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6d8c102320 msgr2=0x7f6d8c102790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:16:09.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:09.025+0000 7f6d90e53640 1 --2- 172.21.15.42:0/2635612290 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6d8c102320 0x7f6d8c102790 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f6d7c009ca0 tx=0x7f6d7c02a8c0 comp rx=0 tx=0).stop 2024-08-30T09:16:09.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:09.025+0000 7f6d90e53640 1 -- 172.21.15.42:0/2635612290 shutdown_connections 2024-08-30T09:16:09.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:09.025+0000 7f6d90e53640 1 --2- 172.21.15.42:0/2635612290 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6d8c103d40 0x7f6d8c10a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:09.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:09.025+0000 7f6d90e53640 1 --2- 172.21.15.42:0/2635612290 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6d8c1033a0 0x7f6d8c1037b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:09.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:09.025+0000 7f6d90e53640 1 --2- 172.21.15.42:0/2635612290 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6d8c102320 0x7f6d8c102790 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:09.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:09.025+0000 7f6d90e53640 1 -- 172.21.15.42:0/2635612290 >> 172.21.15.42:0/2635612290 conn(0x7f6d8c0fdc80 msgr2=0x7f6d8c100110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:16:09.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:09.025+0000 7f6d90e53640 1 -- 172.21.15.42:0/2635612290 shutdown_connections 2024-08-30T09:16:09.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:09.025+0000 7f6d90e53640 1 -- 172.21.15.42:0/2635612290 wait complete. 2024-08-30T09:16:09.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:09.026+0000 7f6d90e53640 1 Processor -- start 2024-08-30T09:16:09.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:09.026+0000 7f6d90e53640 1 -- start start 2024-08-30T09:16:09.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:09.026+0000 7f6d90e53640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6d8c102320 0x7f6d8c198b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:16:09.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:09.026+0000 7f6d90e53640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6d8c1033a0 0x7f6d8c1991d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:16:09.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:09.026+0000 7f6d90e53640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6d8c103d40 0x7f6d8c199d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:16:09.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:09.026+0000 7f6d90e53640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f6d8c10dfa0 con 0x7f6d8c102320 2024-08-30T09:16:09.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:09.026+0000 7f6d90e53640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f6d8c10de20 con 0x7f6d8c1033a0 2024-08-30T09:16:09.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:09.026+0000 7f6d90e53640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f6d8c10e120 con 0x7f6d8c103d40 2024-08-30T09:16:09.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:09.026+0000 7f6d8a59c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6d8c102320 0x7f6d8c198b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:16:09.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:09.026+0000 7f6d8a59c640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6d8c102320 0x7f6d8c198b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34416/0 (socket says 172.21.15.42:34416) 2024-08-30T09:16:09.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:09.026+0000 7f6d8a59c640 1 -- 172.21.15.42:0/70882895 learned_addr learned my addr 172.21.15.42:0/70882895 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:16:09.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:09.026+0000 7f6d8ad9d640 1 --2- 172.21.15.42:0/70882895 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6d8c103d40 0x7f6d8c199d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:16:09.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:09.026+0000 7f6d89d9b640 1 --2- 172.21.15.42:0/70882895 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6d8c1033a0 0x7f6d8c1991d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:16:09.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:09.026+0000 7f6d8a59c640 1 -- 172.21.15.42:0/70882895 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6d8c103d40 msgr2=0x7f6d8c199d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:16:09.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:09.026+0000 7f6d8a59c640 1 --2- 172.21.15.42:0/70882895 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6d8c103d40 0x7f6d8c199d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:09.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:09.026+0000 7f6d8a59c640 1 -- 172.21.15.42:0/70882895 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6d8c1033a0 msgr2=0x7f6d8c1991d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:16:09.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:09.026+0000 7f6d8a59c640 1 --2- 172.21.15.42:0/70882895 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6d8c1033a0 0x7f6d8c1991d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:09.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:09.026+0000 7f6d8a59c640 1 -- 172.21.15.42:0/70882895 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d8c19a5a0 con 0x7f6d8c102320 2024-08-30T09:16:09.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:09.026+0000 7f6d89d9b640 1 --2- 172.21.15.42:0/70882895 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6d8c1033a0 0x7f6d8c1991d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-30T09:16:09.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:09.026+0000 7f6d8a59c640 1 --2- 172.21.15.42:0/70882895 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6d8c102320 0x7f6d8c198b60 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f6d7c02ade0 tx=0x7f6d7c003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:16:09.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:09.027+0000 7f6d7b7fe640 1 -- 172.21.15.42:0/70882895 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d7c0317c0 con 0x7f6d8c102320 2024-08-30T09:16:09.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:09.027+0000 7f6d90e53640 1 -- 172.21.15.42:0/70882895 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d8c19a830 con 0x7f6d8c102320 2024-08-30T09:16:09.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:09.027+0000 7f6d7b7fe640 1 -- 172.21.15.42:0/70882895 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d7c0044f0 con 0x7f6d8c102320 2024-08-30T09:16:09.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:09.027+0000 7f6d7b7fe640 1 -- 172.21.15.42:0/70882895 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d7c02fd40 con 0x7f6d8c102320 2024-08-30T09:16:09.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:09.027+0000 7f6d90e53640 1 -- 172.21.15.42:0/70882895 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d8c1a0f80 con 0x7f6d8c102320 2024-08-30T09:16:09.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:09.028+0000 7f6d90e53640 1 -- 172.21.15.42:0/70882895 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d8c1094f0 con 0x7f6d8c102320 2024-08-30T09:16:09.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:09.029+0000 7f6d7b7fe640 1 -- 172.21.15.42:0/70882895 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7f6d7c004040 con 0x7f6d8c102320 2024-08-30T09:16:09.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:09.030+0000 7f6d7b7fe640 1 --2- 172.21.15.42:0/70882895 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f6d740b4b30 0x7f6d740b7040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:16:09.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:09.030+0000 7f6d89d9b640 1 --2- 172.21.15.42:0/70882895 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f6d740b4b30 0x7f6d740b7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:16:09.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:09.030+0000 7f6d7b7fe640 1 -- 172.21.15.42:0/70882895 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f6d7c106af0 con 0x7f6d8c102320 2024-08-30T09:16:09.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:09.030+0000 7f6d89d9b640 1 --2- 172.21.15.42:0/70882895 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f6d740b4b30 0x7f6d740b7040 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f6d800058c0 tx=0x7f6d80005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:16:09.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:09.033+0000 7f6d7b7fe640 1 -- 172.21.15.42:0/70882895 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f6d7c0cd890 con 0x7f6d8c102320 2024-08-30T09:16:09.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:09.222+0000 7f6d90e53640 1 -- 172.21.15.42:0/70882895 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f6d8c063d60 con 0x7f6d8c102320 2024-08-30T09:16:09.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:09.222+0000 7f6d7b7fe640 1 -- 172.21.15.42:0/70882895 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v83) ==== 103+0+11 (secure 0 0 0) 0x7f6d7c0cd190 con 0x7f6d8c102320 2024-08-30T09:16:09.225 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 32 2024-08-30T09:16:09.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:09.225+0000 7f6d90e53640 1 -- 172.21.15.42:0/70882895 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f6d740b4b30 msgr2=0x7f6d740b7040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:16:09.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:09.225+0000 7f6d90e53640 1 --2- 172.21.15.42:0/70882895 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f6d740b4b30 0x7f6d740b7040 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f6d800058c0 tx=0x7f6d80005850 comp rx=0 tx=0).stop 2024-08-30T09:16:09.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:09.226+0000 7f6d90e53640 1 -- 172.21.15.42:0/70882895 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6d8c102320 msgr2=0x7f6d8c198b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:16:09.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:09.226+0000 7f6d90e53640 1 --2- 172.21.15.42:0/70882895 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6d8c102320 0x7f6d8c198b60 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f6d7c02ade0 tx=0x7f6d7c003ba0 comp rx=0 tx=0).stop 2024-08-30T09:16:09.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:09.226+0000 7f6d90e53640 1 -- 172.21.15.42:0/70882895 shutdown_connections 2024-08-30T09:16:09.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:09.226+0000 7f6d90e53640 1 --2- 172.21.15.42:0/70882895 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f6d740b4b30 0x7f6d740b7040 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:09.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:09.226+0000 7f6d90e53640 1 --2- 172.21.15.42:0/70882895 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6d8c103d40 0x7f6d8c199d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:09.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:09.226+0000 7f6d90e53640 1 --2- 172.21.15.42:0/70882895 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6d8c1033a0 0x7f6d8c1991d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:09.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:09.226+0000 7f6d90e53640 1 --2- 172.21.15.42:0/70882895 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6d8c102320 0x7f6d8c198b60 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:09.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:09.226+0000 7f6d90e53640 1 -- 172.21.15.42:0/70882895 >> 172.21.15.42:0/70882895 conn(0x7f6d8c0fdc80 msgr2=0x7f6d8c1066c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:16:09.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:09.226+0000 7f6d90e53640 1 -- 172.21.15.42:0/70882895 shutdown_connections 2024-08-30T09:16:09.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:09.226+0000 7f6d90e53640 1 -- 172.21.15.42:0/70882895 wait complete. 2024-08-30T09:16:09.238 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:16:09 smithi042 ceph-mon[18652]: pgmap v834: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 2.6 MiB/s wr, 324 op/s 2024-08-30T09:16:09.238 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:16:09 smithi042 ceph-mon[18652]: from='client.? 172.21.15.42:0/3044014488' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-08-30T09:16:09.238 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:16:09 smithi042 ceph-mon[18652]: from='client.? 172.21.15.42:0/3300753592' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} : dispatch 2024-08-30T09:16:09.238 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:16:09 smithi042 ceph-mon[18652]: from='client.? 172.21.15.42:0/2662574099' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} : dispatch 2024-08-30T09:16:09.239 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-08-30T09:16:09.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:16:09 smithi167 ceph-mon[22063]: pgmap v834: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 2.6 MiB/s wr, 324 op/s 2024-08-30T09:16:09.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:16:09 smithi167 ceph-mon[22063]: from='client.? 172.21.15.42:0/3044014488' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-08-30T09:16:09.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:16:09 smithi167 ceph-mon[22063]: from='client.? 172.21.15.42:0/3300753592' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} : dispatch 2024-08-30T09:16:09.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:16:09 smithi167 ceph-mon[22063]: from='client.? 172.21.15.42:0/2662574099' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} : dispatch 2024-08-30T09:16:09.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:09.347+0000 7f864eb0b640 1 -- 172.21.15.42:0/2506816766 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f86481065a0 msgr2=0x7f8648108a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:16:09.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:09.347+0000 7f864eb0b640 1 --2- 172.21.15.42:0/2506816766 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f86481065a0 0x7f8648108a20 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f863c009900 tx=0x7f863c02a410 comp rx=0 tx=0).stop 2024-08-30T09:16:09.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:09.347+0000 7f864eb0b640 1 -- 172.21.15.42:0/2506816766 shutdown_connections 2024-08-30T09:16:09.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:09.347+0000 7f864eb0b640 1 --2- 172.21.15.42:0/2506816766 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8648109160 0x7f864810b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:09.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:09.347+0000 7f864eb0b640 1 --2- 172.21.15.42:0/2506816766 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f86481065a0 0x7f8648108a20 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:09.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:09.347+0000 7f864eb0b640 1 --2- 172.21.15.42:0/2506816766 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f864806c200 0x7f8648106060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:09.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:09.347+0000 7f864eb0b640 1 -- 172.21.15.42:0/2506816766 >> 172.21.15.42:0/2506816766 conn(0x7f86480fdc80 msgr2=0x7f8648100110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:16:09.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:09.347+0000 7f864eb0b640 1 -- 172.21.15.42:0/2506816766 shutdown_connections 2024-08-30T09:16:09.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:09.347+0000 7f864eb0b640 1 -- 172.21.15.42:0/2506816766 wait complete. 2024-08-30T09:16:09.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:09.348+0000 7f864eb0b640 1 Processor -- start 2024-08-30T09:16:09.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:09.348+0000 7f864eb0b640 1 -- start start 2024-08-30T09:16:09.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:09.348+0000 7f864eb0b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f864806c200 0x7f8648199cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:16:09.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:09.348+0000 7f864eb0b640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f86481065a0 0x7f864819a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:16:09.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:09.348+0000 7f864eb0b640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8648109160 0x7f864819aed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:16:09.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:09.348+0000 7f864eb0b640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f864810f080 con 0x7f8648109160 2024-08-30T09:16:09.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:09.348+0000 7f864eb0b640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f864810ef00 con 0x7f86481065a0 2024-08-30T09:16:09.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:09.348+0000 7f864eb0b640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f864810f200 con 0x7f864806c200 2024-08-30T09:16:09.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:09.348+0000 7f864d0a8640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8648109160 0x7f864819aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:16:09.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:09.348+0000 7f864d0a8640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8648109160 0x7f864819aed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34436/0 (socket says 172.21.15.42:34436) 2024-08-30T09:16:09.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:09.348+0000 7f864d0a8640 1 -- 172.21.15.42:0/1912388417 learned_addr learned my addr 172.21.15.42:0/1912388417 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:16:09.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:09.348+0000 7f864c8a7640 1 --2- 172.21.15.42:0/1912388417 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f864806c200 0x7f8648199cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:16:09.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:09.348+0000 7f8647fff640 1 --2- 172.21.15.42:0/1912388417 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f86481065a0 0x7f864819a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:16:09.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:09.348+0000 7f864d0a8640 1 -- 172.21.15.42:0/1912388417 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f864806c200 msgr2=0x7f8648199cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:16:09.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:09.348+0000 7f864d0a8640 1 --2- 172.21.15.42:0/1912388417 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f864806c200 0x7f8648199cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:09.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:09.348+0000 7f864d0a8640 1 -- 172.21.15.42:0/1912388417 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f86481065a0 msgr2=0x7f864819a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:16:09.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:09.348+0000 7f864d0a8640 1 --2- 172.21.15.42:0/1912388417 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f86481065a0 0x7f864819a320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:09.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:09.348+0000 7f864d0a8640 1 -- 172.21.15.42:0/1912388417 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f864819b6f0 con 0x7f8648109160 2024-08-30T09:16:09.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:09.349+0000 7f864d0a8640 1 --2- 172.21.15.42:0/1912388417 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8648109160 0x7f864819aed0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f864000eaa0 tx=0x7f864000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:16:09.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:09.349+0000 7f8645ffb640 1 -- 172.21.15.42:0/1912388417 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f864000cdb0 con 0x7f8648109160 2024-08-30T09:16:09.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:09.349+0000 7f8645ffb640 1 -- 172.21.15.42:0/1912388417 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8640005020 con 0x7f8648109160 2024-08-30T09:16:09.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:09.349+0000 7f864eb0b640 1 -- 172.21.15.42:0/1912388417 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f864819b9e0 con 0x7f8648109160 2024-08-30T09:16:09.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:09.349+0000 7f8645ffb640 1 -- 172.21.15.42:0/1912388417 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86400116d0 con 0x7f8648109160 2024-08-30T09:16:09.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:09.349+0000 7f864eb0b640 1 -- 172.21.15.42:0/1912388417 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86481a20d0 con 0x7f8648109160 2024-08-30T09:16:09.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:09.350+0000 7f864eb0b640 1 -- 172.21.15.42:0/1912388417 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8648106a10 con 0x7f8648109160 2024-08-30T09:16:09.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:09.351+0000 7f8645ffb640 1 -- 172.21.15.42:0/1912388417 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7f86400051d0 con 0x7f8648109160 2024-08-30T09:16:09.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:09.352+0000 7f8645ffb640 1 --2- 172.21.15.42:0/1912388417 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f86300b4b30 0x7f86300b7040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:16:09.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:09.352+0000 7f864c8a7640 1 --2- 172.21.15.42:0/1912388417 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f86300b4b30 0x7f86300b7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:16:09.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:09.352+0000 7f8645ffb640 1 -- 172.21.15.42:0/1912388417 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f86400e57f0 con 0x7f8648109160 2024-08-30T09:16:09.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:09.352+0000 7f864c8a7640 1 --2- 172.21.15.42:0/1912388417 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f86300b4b30 0x7f86300b7040 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f8638005fd0 tx=0x7f8638005e20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:16:09.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:09.356+0000 7f8645ffb640 1 -- 172.21.15.42:0/1912388417 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f8640016030 con 0x7f8648109160 2024-08-30T09:16:09.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:16:09 smithi190 ceph-mon[22001]: pgmap v834: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 2.6 MiB/s wr, 324 op/s 2024-08-30T09:16:09.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:16:09 smithi190 ceph-mon[22001]: from='client.? 172.21.15.42:0/3044014488' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-08-30T09:16:09.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:16:09 smithi190 ceph-mon[22001]: from='client.? 172.21.15.42:0/3300753592' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} : dispatch 2024-08-30T09:16:09.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:16:09 smithi190 ceph-mon[22001]: from='client.? 172.21.15.42:0/2662574099' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} : dispatch 2024-08-30T09:16:09.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:09.593+0000 7f864eb0b640 1 -- 172.21.15.42:0/1912388417 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7f8648063d60 con 0x7f8648109160 2024-08-30T09:16:09.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:09.601+0000 7f8645ffb640 1 -- 172.21.15.42:0/1912388417 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v18) ==== 81+0+825792 (secure 0 0 0) 0x7f86400ac530 con 0x7f8648109160 2024-08-30T09:16:09.605 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-08-30T09:16:09.605 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-08-30T09:16:09.605 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 18, 2024-08-30T09:16:09.605 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:09.605 INFO:teuthology.orchestra.run.smithi042.stdout: "active_gid": 14150, 2024-08-30T09:16:09.605 INFO:teuthology.orchestra.run.smithi042.stdout: "active_name": "x", 2024-08-30T09:16:09.605 INFO:teuthology.orchestra.run.smithi042.stdout: "active_addrs": { 2024-08-30T09:16:09.605 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-08-30T09:16:09.605 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-08-30T09:16:09.606 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-08-30T09:16:09.606 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6800", 2024-08-30T09:16:09.606 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3005140902 2024-08-30T09:16:09.606 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.606 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-08-30T09:16:09.606 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-08-30T09:16:09.606 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6801", 2024-08-30T09:16:09.606 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3005140902 2024-08-30T09:16:09.606 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:09.606 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-08-30T09:16:09.606 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.606 INFO:teuthology.orchestra.run.smithi042.stdout: "active_addr": "172.21.15.42:6801/3005140902", 2024-08-30T09:16:09.606 INFO:teuthology.orchestra.run.smithi042.stdout: "active_change": "2024-08-30T08:50:44.821228+0000", 2024-08-30T09:16:09.606 INFO:teuthology.orchestra.run.smithi042.stdout: "active_mgr_features": 4540701547738038271, 2024-08-30T09:16:09.607 INFO:teuthology.orchestra.run.smithi042.stdout: "available": true, 2024-08-30T09:16:09.607 INFO:teuthology.orchestra.run.smithi042.stdout: "standbys": [ 2024-08-30T09:16:09.607 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-08-30T09:16:09.607 INFO:teuthology.orchestra.run.smithi042.stdout: "gid": 24121, 2024-08-30T09:16:09.607 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "z", 2024-08-30T09:16:09.607 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr_features": 4540701547738038271, 2024-08-30T09:16:09.607 INFO:teuthology.orchestra.run.smithi042.stdout: "available_modules": [ 2024-08-30T09:16:09.607 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-08-30T09:16:09.607 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "alerts", 2024-08-30T09:16:09.607 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-08-30T09:16:09.607 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-08-30T09:16:09.607 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-08-30T09:16:09.607 INFO:teuthology.orchestra.run.smithi042.stdout: "interval": { 2024-08-30T09:16:09.607 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "interval", 2024-08-30T09:16:09.607 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-08-30T09:16:09.607 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.608 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:09.608 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "60", 2024-08-30T09:16:09.608 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.608 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.608 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.608 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "How frequently to reexamine health status", 2024-08-30T09:16:09.608 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.608 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.608 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.608 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.608 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-08-30T09:16:09.608 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-08-30T09:16:09.608 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:09.609 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.609 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:09.609 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:09.609 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.609 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.609 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:09.609 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:09.609 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:09.609 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:09.609 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:09.609 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:09.610 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:09.610 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:09.610 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.610 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.610 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.610 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.610 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.610 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-08-30T09:16:09.610 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-08-30T09:16:09.610 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:09.610 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.610 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:09.610 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:09.611 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.611 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.611 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.611 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.611 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.611 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.611 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.611 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.611 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-08-30T09:16:09.611 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-08-30T09:16:09.611 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:09.611 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.611 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:09.611 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-08-30T09:16:09.612 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.612 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.612 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:09.612 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:09.612 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:09.612 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:09.612 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:09.612 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:09.612 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:09.612 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:09.612 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.612 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.612 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.612 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.612 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.613 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-08-30T09:16:09.613 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-08-30T09:16:09.613 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:09.613 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.613 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:09.613 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:09.613 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.613 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.613 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.613 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.613 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.613 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.613 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.613 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.613 INFO:teuthology.orchestra.run.smithi042.stdout: "smtp_destination": { 2024-08-30T09:16:09.614 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "smtp_destination", 2024-08-30T09:16:09.614 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:09.614 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.614 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:09.614 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:09.614 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.614 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.614 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.614 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Email address to send alerts to", 2024-08-30T09:16:09.614 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.614 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.614 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.614 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.614 INFO:teuthology.orchestra.run.smithi042.stdout: "smtp_from_name": { 2024-08-30T09:16:09.614 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "smtp_from_name", 2024-08-30T09:16:09.615 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:09.615 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.615 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:09.615 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "Ceph", 2024-08-30T09:16:09.615 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.615 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.615 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.615 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Email From: name", 2024-08-30T09:16:09.615 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.615 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.615 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.615 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.615 INFO:teuthology.orchestra.run.smithi042.stdout: "smtp_host": { 2024-08-30T09:16:09.615 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "smtp_host", 2024-08-30T09:16:09.615 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:09.616 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.616 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:09.616 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:09.616 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.616 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.616 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.616 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "SMTP server", 2024-08-30T09:16:09.616 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.616 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.616 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.616 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.616 INFO:teuthology.orchestra.run.smithi042.stdout: "smtp_password": { 2024-08-30T09:16:09.616 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "smtp_password", 2024-08-30T09:16:09.616 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:09.616 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.617 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:09.617 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:09.617 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.617 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.617 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.617 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Password to authenticate with", 2024-08-30T09:16:09.617 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.617 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.617 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.617 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.617 INFO:teuthology.orchestra.run.smithi042.stdout: "smtp_port": { 2024-08-30T09:16:09.617 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "smtp_port", 2024-08-30T09:16:09.617 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:09.617 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.617 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:09.617 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "465", 2024-08-30T09:16:09.618 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.618 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.618 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.618 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "SMTP port", 2024-08-30T09:16:09.618 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.618 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.618 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.618 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.618 INFO:teuthology.orchestra.run.smithi042.stdout: "smtp_sender": { 2024-08-30T09:16:09.618 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "smtp_sender", 2024-08-30T09:16:09.618 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:09.618 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.618 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:09.618 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:09.618 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.619 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.619 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.619 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "SMTP envelope sender", 2024-08-30T09:16:09.619 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.619 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.619 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.619 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.619 INFO:teuthology.orchestra.run.smithi042.stdout: "smtp_ssl": { 2024-08-30T09:16:09.619 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "smtp_ssl", 2024-08-30T09:16:09.619 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:09.619 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.619 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:09.619 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-08-30T09:16:09.619 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.619 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.620 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.620 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Use SSL to connect to SMTP server", 2024-08-30T09:16:09.620 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.620 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.620 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.620 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.620 INFO:teuthology.orchestra.run.smithi042.stdout: "smtp_user": { 2024-08-30T09:16:09.620 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "smtp_user", 2024-08-30T09:16:09.620 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:09.620 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.620 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:09.620 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:09.620 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.620 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.620 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.621 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "User to authenticate as", 2024-08-30T09:16:09.621 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.621 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.621 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.621 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.621 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-08-30T09:16:09.621 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-08-30T09:16:09.621 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:09.621 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-08-30T09:16:09.621 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:09.621 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-08-30T09:16:09.621 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.621 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.621 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.621 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.622 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.622 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.622 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.622 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:09.622 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:09.622 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.622 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-08-30T09:16:09.622 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "balancer", 2024-08-30T09:16:09.622 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-08-30T09:16:09.622 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-08-30T09:16:09.622 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-08-30T09:16:09.622 INFO:teuthology.orchestra.run.smithi042.stdout: "active": { 2024-08-30T09:16:09.623 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "active", 2024-08-30T09:16:09.623 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:09.623 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.623 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:09.623 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-08-30T09:16:09.623 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.623 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.623 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.623 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "automatically balance PGs across cluster", 2024-08-30T09:16:09.623 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.623 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.623 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.623 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.624 INFO:teuthology.orchestra.run.smithi042.stdout: "begin_time": { 2024-08-30T09:16:09.624 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "begin_time", 2024-08-30T09:16:09.624 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:09.624 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.624 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:09.624 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0000", 2024-08-30T09:16:09.624 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.624 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.624 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.624 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "beginning time of day to automatically balance", 2024-08-30T09:16:09.624 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-08-30T09:16:09.624 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.624 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.624 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.625 INFO:teuthology.orchestra.run.smithi042.stdout: "begin_weekday": { 2024-08-30T09:16:09.625 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "begin_weekday", 2024-08-30T09:16:09.625 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "uint", 2024-08-30T09:16:09.625 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.625 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:09.625 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-08-30T09:16:09.625 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "0", 2024-08-30T09:16:09.625 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "6", 2024-08-30T09:16:09.625 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.625 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-08-30T09:16:09.625 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-08-30T09:16:09.625 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.625 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.625 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.625 INFO:teuthology.orchestra.run.smithi042.stdout: "crush_compat_max_iterations": { 2024-08-30T09:16:09.626 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "crush_compat_max_iterations", 2024-08-30T09:16:09.626 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "uint", 2024-08-30T09:16:09.626 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.626 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:09.626 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "25", 2024-08-30T09:16:09.626 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "1", 2024-08-30T09:16:09.626 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "250", 2024-08-30T09:16:09.626 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.626 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-08-30T09:16:09.626 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.626 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.626 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.626 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.626 INFO:teuthology.orchestra.run.smithi042.stdout: "crush_compat_metrics": { 2024-08-30T09:16:09.626 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "crush_compat_metrics", 2024-08-30T09:16:09.627 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:09.627 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.627 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:09.627 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "pgs,objects,bytes", 2024-08-30T09:16:09.627 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.627 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.627 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.627 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-08-30T09:16:09.627 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-08-30T09:16:09.627 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.627 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.627 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.627 INFO:teuthology.orchestra.run.smithi042.stdout: "crush_compat_step": { 2024-08-30T09:16:09.627 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "crush_compat_step", 2024-08-30T09:16:09.627 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "float", 2024-08-30T09:16:09.628 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.628 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:09.628 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0.5", 2024-08-30T09:16:09.628 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "0.001", 2024-08-30T09:16:09.628 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "0.999", 2024-08-30T09:16:09.628 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.628 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "aggressiveness of optimization", 2024-08-30T09:16:09.628 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-08-30T09:16:09.628 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.628 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.628 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.628 INFO:teuthology.orchestra.run.smithi042.stdout: "end_time": { 2024-08-30T09:16:09.628 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "end_time", 2024-08-30T09:16:09.628 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:09.628 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.629 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:09.629 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "2359", 2024-08-30T09:16:09.629 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.629 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.629 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.629 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "ending time of day to automatically balance", 2024-08-30T09:16:09.629 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-08-30T09:16:09.629 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.629 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.629 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.629 INFO:teuthology.orchestra.run.smithi042.stdout: "end_weekday": { 2024-08-30T09:16:09.629 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "end_weekday", 2024-08-30T09:16:09.629 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "uint", 2024-08-30T09:16:09.629 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.629 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:09.630 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-08-30T09:16:09.630 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "0", 2024-08-30T09:16:09.630 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "6", 2024-08-30T09:16:09.630 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.630 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-08-30T09:16:09.630 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-08-30T09:16:09.630 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.630 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.630 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.630 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-08-30T09:16:09.630 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-08-30T09:16:09.630 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:09.630 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.630 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:09.630 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:09.630 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.631 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.631 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:09.631 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:09.631 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:09.631 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:09.639 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:09.639 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:09.639 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:09.639 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:09.639 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.639 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.640 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.640 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.640 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.640 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-08-30T09:16:09.640 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-08-30T09:16:09.640 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:09.640 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.640 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:09.640 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:09.640 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.640 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.640 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.640 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.641 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.641 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.641 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.641 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.641 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-08-30T09:16:09.641 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-08-30T09:16:09.641 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:09.641 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.641 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:09.641 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-08-30T09:16:09.641 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.641 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.641 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:09.642 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:09.642 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:09.642 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:09.642 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:09.642 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:09.642 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:09.642 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:09.642 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.642 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.642 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.642 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.642 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.642 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-08-30T09:16:09.642 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-08-30T09:16:09.642 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:09.643 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.643 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:09.643 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:09.643 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.643 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.643 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.643 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.643 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.643 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.643 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.643 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.643 INFO:teuthology.orchestra.run.smithi042.stdout: "min_score": { 2024-08-30T09:16:09.643 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "min_score", 2024-08-30T09:16:09.643 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "float", 2024-08-30T09:16:09.643 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.643 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:09.644 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-08-30T09:16:09.644 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.644 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.644 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.644 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-08-30T09:16:09.644 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.644 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.644 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.644 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.644 INFO:teuthology.orchestra.run.smithi042.stdout: "mode": { 2024-08-30T09:16:09.644 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "mode", 2024-08-30T09:16:09.644 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:09.644 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.644 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:09.644 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "upmap", 2024-08-30T09:16:09.645 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.645 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.645 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:09.645 INFO:teuthology.orchestra.run.smithi042.stdout: "crush-compat", 2024-08-30T09:16:09.645 INFO:teuthology.orchestra.run.smithi042.stdout: "none", 2024-08-30T09:16:09.645 INFO:teuthology.orchestra.run.smithi042.stdout: "read", 2024-08-30T09:16:09.645 INFO:teuthology.orchestra.run.smithi042.stdout: "upmap", 2024-08-30T09:16:09.645 INFO:teuthology.orchestra.run.smithi042.stdout: "upmap-read" 2024-08-30T09:16:09.645 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:09.645 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Balancer mode", 2024-08-30T09:16:09.645 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.645 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.645 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.646 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.646 INFO:teuthology.orchestra.run.smithi042.stdout: "pool_ids": { 2024-08-30T09:16:09.646 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "pool_ids", 2024-08-30T09:16:09.646 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:09.646 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.646 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:09.646 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:09.646 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.646 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.646 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.646 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-08-30T09:16:09.646 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.646 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.646 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.646 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.647 INFO:teuthology.orchestra.run.smithi042.stdout: "sleep_interval": { 2024-08-30T09:16:09.647 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sleep_interval", 2024-08-30T09:16:09.647 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-08-30T09:16:09.647 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.647 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:09.647 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "60", 2024-08-30T09:16:09.647 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.647 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.647 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.647 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-08-30T09:16:09.647 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.647 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.647 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.647 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.647 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-08-30T09:16:09.648 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-08-30T09:16:09.648 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:09.648 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-08-30T09:16:09.648 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:09.648 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-08-30T09:16:09.648 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.648 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.648 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.648 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.648 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.648 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.648 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.648 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.648 INFO:teuthology.orchestra.run.smithi042.stdout: "upmap_max_deviation": { 2024-08-30T09:16:09.648 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "upmap_max_deviation", 2024-08-30T09:16:09.649 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:09.649 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.649 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:09.649 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "5", 2024-08-30T09:16:09.649 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "1", 2024-08-30T09:16:09.649 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.649 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.649 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "deviation below which no optimization is attempted", 2024-08-30T09:16:09.649 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-08-30T09:16:09.649 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.649 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.649 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.649 INFO:teuthology.orchestra.run.smithi042.stdout: "upmap_max_optimizations": { 2024-08-30T09:16:09.649 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "upmap_max_optimizations", 2024-08-30T09:16:09.649 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "uint", 2024-08-30T09:16:09.650 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.650 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:09.650 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "10", 2024-08-30T09:16:09.650 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.650 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.650 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.650 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-08-30T09:16:09.650 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.650 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.650 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.650 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:09.650 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:09.650 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.650 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-08-30T09:16:09.650 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "cephadm", 2024-08-30T09:16:09.651 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-08-30T09:16:09.651 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-08-30T09:16:09.651 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-08-30T09:16:09.651 INFO:teuthology.orchestra.run.smithi042.stdout: "agent_down_multiplier": { 2024-08-30T09:16:09.651 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "agent_down_multiplier", 2024-08-30T09:16:09.651 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "float", 2024-08-30T09:16:09.651 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.651 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:09.651 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "3.0", 2024-08-30T09:16:09.651 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.651 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.651 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.651 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-08-30T09:16:09.651 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.652 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.652 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.652 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.652 INFO:teuthology.orchestra.run.smithi042.stdout: "agent_refresh_rate": { 2024-08-30T09:16:09.652 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "agent_refresh_rate", 2024-08-30T09:16:09.652 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-08-30T09:16:09.652 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.652 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:09.652 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "20", 2024-08-30T09:16:09.652 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.652 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.652 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.652 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-08-30T09:16:09.653 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.653 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.653 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.653 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.653 INFO:teuthology.orchestra.run.smithi042.stdout: "agent_starting_port": { 2024-08-30T09:16:09.653 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "agent_starting_port", 2024-08-30T09:16:09.653 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:09.653 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.653 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:09.653 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "4721", 2024-08-30T09:16:09.653 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.653 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.653 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.653 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-08-30T09:16:09.653 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.654 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.654 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.654 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.654 INFO:teuthology.orchestra.run.smithi042.stdout: "allow_ptrace": { 2024-08-30T09:16:09.654 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "allow_ptrace", 2024-08-30T09:16:09.654 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:09.654 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.654 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:09.654 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:09.654 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.654 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.654 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.654 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-08-30T09:16:09.654 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-08-30T09:16:09.655 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.655 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.655 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.655 INFO:teuthology.orchestra.run.smithi042.stdout: "autotune_interval": { 2024-08-30T09:16:09.655 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "autotune_interval", 2024-08-30T09:16:09.655 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-08-30T09:16:09.655 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.655 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:09.655 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "600", 2024-08-30T09:16:09.655 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.655 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.655 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.655 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "how frequently to autotune daemon memory", 2024-08-30T09:16:09.655 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.656 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.656 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.656 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.656 INFO:teuthology.orchestra.run.smithi042.stdout: "autotune_memory_target_ratio": { 2024-08-30T09:16:09.656 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "autotune_memory_target_ratio", 2024-08-30T09:16:09.656 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "float", 2024-08-30T09:16:09.656 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.656 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:09.656 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0.7", 2024-08-30T09:16:09.656 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.656 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.656 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.656 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-08-30T09:16:09.657 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.657 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.657 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.657 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.657 INFO:teuthology.orchestra.run.smithi042.stdout: "cephadm_log_destination": { 2024-08-30T09:16:09.657 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "cephadm_log_destination", 2024-08-30T09:16:09.657 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:09.657 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.657 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:09.657 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:09.657 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.657 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.657 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:09.657 INFO:teuthology.orchestra.run.smithi042.stdout: "file", 2024-08-30T09:16:09.658 INFO:teuthology.orchestra.run.smithi042.stdout: "file,syslog", 2024-08-30T09:16:09.658 INFO:teuthology.orchestra.run.smithi042.stdout: "syslog" 2024-08-30T09:16:09.658 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:09.658 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-08-30T09:16:09.658 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.658 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.658 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.658 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.658 INFO:teuthology.orchestra.run.smithi042.stdout: "cgroups_split": { 2024-08-30T09:16:09.658 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "cgroups_split", 2024-08-30T09:16:09.658 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:09.658 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.658 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:09.658 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-08-30T09:16:09.658 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.658 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.659 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.659 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-08-30T09:16:09.659 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.659 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.659 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.659 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.659 INFO:teuthology.orchestra.run.smithi042.stdout: "config_checks_enabled": { 2024-08-30T09:16:09.659 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "config_checks_enabled", 2024-08-30T09:16:09.659 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:09.659 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.659 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:09.659 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:09.659 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.659 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.659 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.660 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-08-30T09:16:09.660 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.660 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.660 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.660 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.660 INFO:teuthology.orchestra.run.smithi042.stdout: "config_dashboard": { 2024-08-30T09:16:09.660 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "config_dashboard", 2024-08-30T09:16:09.660 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:09.660 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.660 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:09.660 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-08-30T09:16:09.660 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.660 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.660 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.661 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-08-30T09:16:09.661 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.661 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.661 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.661 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.661 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_alertmanager": { 2024-08-30T09:16:09.661 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_alertmanager", 2024-08-30T09:16:09.661 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:09.661 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.661 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:09.661 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.27.0", 2024-08-30T09:16:09.661 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.661 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.661 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.662 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Prometheus container image", 2024-08-30T09:16:09.662 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.662 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.662 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.662 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.662 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_base": { 2024-08-30T09:16:09.662 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_base", 2024-08-30T09:16:09.662 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:09.662 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.662 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:09.662 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/ceph/ceph", 2024-08-30T09:16:09.662 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.662 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.662 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.662 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Container image name, without the tag", 2024-08-30T09:16:09.662 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.663 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.663 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.663 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.663 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_elasticsearch": { 2024-08-30T09:16:09.663 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_elasticsearch", 2024-08-30T09:16:09.663 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:09.663 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.663 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:09.663 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-08-30T09:16:09.663 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.663 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.663 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.663 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "elasticsearch container image", 2024-08-30T09:16:09.663 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.664 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.664 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.664 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.664 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_grafana": { 2024-08-30T09:16:09.664 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_grafana", 2024-08-30T09:16:09.664 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:09.664 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.664 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:09.664 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/ceph/grafana:10.4.0", 2024-08-30T09:16:09.664 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.664 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.664 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.664 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Prometheus container image", 2024-08-30T09:16:09.664 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.665 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.665 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.665 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.665 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_haproxy": { 2024-08-30T09:16:09.665 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_haproxy", 2024-08-30T09:16:09.665 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:09.665 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.665 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:09.665 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-08-30T09:16:09.665 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.665 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.665 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.665 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "HAproxy container image", 2024-08-30T09:16:09.666 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.666 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.666 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.666 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.666 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_jaeger_agent": { 2024-08-30T09:16:09.666 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_jaeger_agent", 2024-08-30T09:16:09.666 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:09.666 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.666 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:09.666 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-08-30T09:16:09.666 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.666 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.666 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.666 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Jaeger agent container image", 2024-08-30T09:16:09.666 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.666 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.667 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.667 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.667 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_jaeger_collector": { 2024-08-30T09:16:09.667 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_jaeger_collector", 2024-08-30T09:16:09.667 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:09.667 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.667 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:09.667 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-08-30T09:16:09.667 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.667 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.667 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.667 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Jaeger collector container image", 2024-08-30T09:16:09.667 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.667 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.667 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.668 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.668 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_jaeger_query": { 2024-08-30T09:16:09.668 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_jaeger_query", 2024-08-30T09:16:09.668 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:09.668 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.668 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:09.668 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-08-30T09:16:09.668 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.668 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.668 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.668 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Jaeger query container image", 2024-08-30T09:16:09.668 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.668 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.669 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.669 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.669 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_keepalived": { 2024-08-30T09:16:09.669 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_keepalived", 2024-08-30T09:16:09.669 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:09.669 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.669 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:09.669 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-08-30T09:16:09.669 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.669 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.669 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.669 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Keepalived container image", 2024-08-30T09:16:09.669 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.669 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.669 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.670 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.670 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_loki": { 2024-08-30T09:16:09.670 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_loki", 2024-08-30T09:16:09.670 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:09.670 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.670 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:09.670 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-08-30T09:16:09.670 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.670 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.670 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.670 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Loki container image", 2024-08-30T09:16:09.670 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.670 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.670 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.670 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.670 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_nginx": { 2024-08-30T09:16:09.671 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_nginx", 2024-08-30T09:16:09.671 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:09.671 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.671 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:09.671 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/ceph/nginx:1.26.1", 2024-08-30T09:16:09.671 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.671 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.671 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.671 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Nginx container image", 2024-08-30T09:16:09.671 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.671 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.671 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.671 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.671 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_node_exporter": { 2024-08-30T09:16:09.671 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_node_exporter", 2024-08-30T09:16:09.672 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:09.672 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.672 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:09.672 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.7.0", 2024-08-30T09:16:09.672 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.672 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.672 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.672 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Prometheus container image", 2024-08-30T09:16:09.672 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.672 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.672 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.672 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.672 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_nvmeof": { 2024-08-30T09:16:09.672 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_nvmeof", 2024-08-30T09:16:09.672 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:09.673 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.673 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:09.673 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.17", 2024-08-30T09:16:09.673 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.673 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.673 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.673 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Nvme-of container image", 2024-08-30T09:16:09.673 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.673 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.673 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.673 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.673 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_oauth2_proxy": { 2024-08-30T09:16:09.673 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_oauth2_proxy", 2024-08-30T09:16:09.673 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:09.673 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.674 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:09.674 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/oauth2-proxy/oauth2-proxy:v7.6.0", 2024-08-30T09:16:09.674 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.674 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.674 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.674 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "oauth2-proxy container image", 2024-08-30T09:16:09.674 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.674 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.674 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.674 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.674 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_prometheus": { 2024-08-30T09:16:09.674 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_prometheus", 2024-08-30T09:16:09.674 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:09.674 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.675 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:09.675 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/prometheus/prometheus:v2.51.0", 2024-08-30T09:16:09.675 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.675 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.675 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.675 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Prometheus container image", 2024-08-30T09:16:09.675 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.675 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.675 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.675 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.675 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_promtail": { 2024-08-30T09:16:09.675 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_promtail", 2024-08-30T09:16:09.675 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:09.676 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.676 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:09.676 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-08-30T09:16:09.676 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.676 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.676 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.676 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Promtail container image", 2024-08-30T09:16:09.676 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.676 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.676 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.676 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.676 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_samba": { 2024-08-30T09:16:09.676 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_samba", 2024-08-30T09:16:09.676 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:09.676 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.677 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:09.677 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-08-30T09:16:09.677 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.677 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.677 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.677 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Samba/SMB container image", 2024-08-30T09:16:09.677 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.677 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.677 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.677 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.677 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_snmp_gateway": { 2024-08-30T09:16:09.677 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_snmp_gateway", 2024-08-30T09:16:09.677 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:09.677 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.677 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:09.678 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-08-30T09:16:09.678 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.678 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.678 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.678 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "SNMP Gateway container image", 2024-08-30T09:16:09.678 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.678 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.678 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.678 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.678 INFO:teuthology.orchestra.run.smithi042.stdout: "container_init": { 2024-08-30T09:16:09.678 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_init", 2024-08-30T09:16:09.678 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:09.678 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.679 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:09.679 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-08-30T09:16:09.679 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.679 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.679 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.679 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Run podman/docker with `--init`", 2024-08-30T09:16:09.679 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.679 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.679 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.679 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.679 INFO:teuthology.orchestra.run.smithi042.stdout: "daemon_cache_timeout": { 2024-08-30T09:16:09.679 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "daemon_cache_timeout", 2024-08-30T09:16:09.679 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-08-30T09:16:09.679 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.679 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:09.680 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "600", 2024-08-30T09:16:09.680 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.680 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.680 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.680 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-08-30T09:16:09.680 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.680 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.680 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.680 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.680 INFO:teuthology.orchestra.run.smithi042.stdout: "default_cephadm_command_timeout": { 2024-08-30T09:16:09.680 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "default_cephadm_command_timeout", 2024-08-30T09:16:09.680 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:09.680 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.680 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:09.680 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "900", 2024-08-30T09:16:09.681 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.681 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.681 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.681 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-08-30T09:16:09.681 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.681 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.681 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.681 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.681 INFO:teuthology.orchestra.run.smithi042.stdout: "default_registry": { 2024-08-30T09:16:09.681 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "default_registry", 2024-08-30T09:16:09.681 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:09.681 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.681 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:09.681 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "docker.io", 2024-08-30T09:16:09.681 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.682 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.682 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.682 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-08-30T09:16:09.682 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.682 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.682 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.682 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.682 INFO:teuthology.orchestra.run.smithi042.stdout: "device_cache_timeout": { 2024-08-30T09:16:09.682 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "device_cache_timeout", 2024-08-30T09:16:09.682 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-08-30T09:16:09.682 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.682 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:09.682 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "1800", 2024-08-30T09:16:09.682 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.682 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.682 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.683 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "seconds to cache device inventory", 2024-08-30T09:16:09.683 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.683 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.683 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.683 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.683 INFO:teuthology.orchestra.run.smithi042.stdout: "device_enhanced_scan": { 2024-08-30T09:16:09.683 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "device_enhanced_scan", 2024-08-30T09:16:09.683 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:09.683 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.683 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:09.683 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:09.683 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.683 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.683 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.684 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Use libstoragemgmt during device scans", 2024-08-30T09:16:09.684 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.684 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.684 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.684 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.684 INFO:teuthology.orchestra.run.smithi042.stdout: "facts_cache_timeout": { 2024-08-30T09:16:09.684 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "facts_cache_timeout", 2024-08-30T09:16:09.684 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-08-30T09:16:09.684 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.684 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:09.684 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "60", 2024-08-30T09:16:09.684 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.684 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.685 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.685 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "seconds to cache host facts data", 2024-08-30T09:16:09.685 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.685 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.685 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.685 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.685 INFO:teuthology.orchestra.run.smithi042.stdout: "grafana_dashboards_path": { 2024-08-30T09:16:09.685 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "grafana_dashboards_path", 2024-08-30T09:16:09.685 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:09.685 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.685 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:09.685 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-08-30T09:16:09.685 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.685 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.686 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.686 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-08-30T09:16:09.686 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.686 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.686 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.686 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.686 INFO:teuthology.orchestra.run.smithi042.stdout: "host_check_interval": { 2024-08-30T09:16:09.686 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "host_check_interval", 2024-08-30T09:16:09.686 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-08-30T09:16:09.686 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.686 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:09.686 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "600", 2024-08-30T09:16:09.686 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.686 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.686 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.687 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "how frequently to perform a host check", 2024-08-30T09:16:09.687 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.687 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.687 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.687 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.687 INFO:teuthology.orchestra.run.smithi042.stdout: "hw_monitoring": { 2024-08-30T09:16:09.687 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "hw_monitoring", 2024-08-30T09:16:09.687 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:09.687 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.687 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:09.687 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:09.687 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.687 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.687 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.688 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-08-30T09:16:09.688 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.688 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.688 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.688 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.688 INFO:teuthology.orchestra.run.smithi042.stdout: "inventory_list_all": { 2024-08-30T09:16:09.688 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "inventory_list_all", 2024-08-30T09:16:09.688 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:09.688 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.688 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:09.688 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:09.688 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.688 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.688 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.689 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-08-30T09:16:09.689 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.689 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.689 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.689 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.689 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-08-30T09:16:09.689 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-08-30T09:16:09.689 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:09.689 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.689 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:09.689 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:09.689 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.689 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.689 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:09.689 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:09.690 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:09.690 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:09.690 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:09.690 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:09.690 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:09.690 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:09.690 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.690 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.690 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.690 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.690 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.690 INFO:teuthology.orchestra.run.smithi042.stdout: "log_refresh_metadata": { 2024-08-30T09:16:09.690 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_refresh_metadata", 2024-08-30T09:16:09.690 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:09.690 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.690 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:09.691 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:09.691 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.691 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.691 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.691 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-08-30T09:16:09.691 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.691 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.691 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.691 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.691 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-08-30T09:16:09.691 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-08-30T09:16:09.691 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:09.691 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.691 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:09.691 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-08-30T09:16:09.692 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.692 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.692 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.692 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-08-30T09:16:09.692 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.692 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.692 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.692 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.692 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-08-30T09:16:09.692 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-08-30T09:16:09.692 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:09.692 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.692 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:09.692 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-08-30T09:16:09.692 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.692 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.693 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:09.693 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:09.693 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:09.693 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:09.693 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:09.693 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:09.693 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:09.693 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:09.693 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.693 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.693 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.693 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.693 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.693 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-08-30T09:16:09.693 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-08-30T09:16:09.694 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:09.694 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.694 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:09.694 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:09.694 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.694 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.694 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.694 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.694 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.694 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.694 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.694 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.694 INFO:teuthology.orchestra.run.smithi042.stdout: "manage_etc_ceph_ceph_conf": { 2024-08-30T09:16:09.694 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-08-30T09:16:09.695 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:09.695 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.695 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:09.695 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:09.695 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.695 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.695 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.695 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-08-30T09:16:09.695 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.695 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.695 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.695 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.695 INFO:teuthology.orchestra.run.smithi042.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-08-30T09:16:09.695 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-08-30T09:16:09.696 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:09.696 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.696 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:09.696 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "*", 2024-08-30T09:16:09.696 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.696 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.696 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.696 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-08-30T09:16:09.696 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.696 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.696 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.696 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.696 INFO:teuthology.orchestra.run.smithi042.stdout: "max_count_per_host": { 2024-08-30T09:16:09.696 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "max_count_per_host", 2024-08-30T09:16:09.696 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:09.697 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.697 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:09.697 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "10", 2024-08-30T09:16:09.697 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.697 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.697 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.697 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "max number of daemons per service per host", 2024-08-30T09:16:09.697 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.697 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.697 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.697 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.697 INFO:teuthology.orchestra.run.smithi042.stdout: "max_osd_draining_count": { 2024-08-30T09:16:09.697 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "max_osd_draining_count", 2024-08-30T09:16:09.697 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:09.697 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.698 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:09.698 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "10", 2024-08-30T09:16:09.698 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.698 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.698 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.698 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-08-30T09:16:09.698 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.698 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.698 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.698 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.698 INFO:teuthology.orchestra.run.smithi042.stdout: "migration_current": { 2024-08-30T09:16:09.698 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "migration_current", 2024-08-30T09:16:09.698 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:09.698 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.698 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:09.699 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:09.699 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.699 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.699 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.699 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "internal - do not modify", 2024-08-30T09:16:09.699 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.699 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.699 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.699 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.699 INFO:teuthology.orchestra.run.smithi042.stdout: "mode": { 2024-08-30T09:16:09.699 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "mode", 2024-08-30T09:16:09.699 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:09.699 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.699 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:09.700 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "root", 2024-08-30T09:16:09.700 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.700 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.700 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:09.700 INFO:teuthology.orchestra.run.smithi042.stdout: "cephadm-package", 2024-08-30T09:16:09.700 INFO:teuthology.orchestra.run.smithi042.stdout: "root" 2024-08-30T09:16:09.700 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:09.700 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "mode for remote execution of cephadm", 2024-08-30T09:16:09.700 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.700 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.700 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.700 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.700 INFO:teuthology.orchestra.run.smithi042.stdout: "oob_default_addr": { 2024-08-30T09:16:09.700 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "oob_default_addr", 2024-08-30T09:16:09.701 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:09.701 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.701 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:09.701 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "169.254.1.1", 2024-08-30T09:16:09.701 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.701 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.701 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.701 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Default address for RedFish API (oob management).", 2024-08-30T09:16:09.701 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.701 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.701 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.701 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.701 INFO:teuthology.orchestra.run.smithi042.stdout: "prometheus_alerts_path": { 2024-08-30T09:16:09.701 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "prometheus_alerts_path", 2024-08-30T09:16:09.701 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:09.702 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.702 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:09.702 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-08-30T09:16:09.702 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.702 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.702 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.702 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-08-30T09:16:09.702 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.702 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.702 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.702 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.702 INFO:teuthology.orchestra.run.smithi042.stdout: "registry_insecure": { 2024-08-30T09:16:09.702 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "registry_insecure", 2024-08-30T09:16:09.702 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:09.702 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.702 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:09.703 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:09.703 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.703 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.703 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.703 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-08-30T09:16:09.703 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.703 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.703 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.703 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.703 INFO:teuthology.orchestra.run.smithi042.stdout: "registry_password": { 2024-08-30T09:16:09.703 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "registry_password", 2024-08-30T09:16:09.703 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:09.703 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.703 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:09.703 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:09.704 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.704 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.704 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.704 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-08-30T09:16:09.704 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.704 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.704 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.704 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.704 INFO:teuthology.orchestra.run.smithi042.stdout: "registry_url": { 2024-08-30T09:16:09.704 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "registry_url", 2024-08-30T09:16:09.704 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:09.704 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.704 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:09.704 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:09.704 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.705 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.705 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.705 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-08-30T09:16:09.705 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.705 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.705 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.705 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.705 INFO:teuthology.orchestra.run.smithi042.stdout: "registry_username": { 2024-08-30T09:16:09.705 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "registry_username", 2024-08-30T09:16:09.705 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:09.705 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.705 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:09.705 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:09.705 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.705 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.706 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.706 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-08-30T09:16:09.706 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.706 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.706 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.706 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.706 INFO:teuthology.orchestra.run.smithi042.stdout: "secure_monitoring_stack": { 2024-08-30T09:16:09.706 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "secure_monitoring_stack", 2024-08-30T09:16:09.706 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:09.706 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.706 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:09.706 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:09.706 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.706 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.706 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.707 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-08-30T09:16:09.707 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.707 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.707 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.707 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.707 INFO:teuthology.orchestra.run.smithi042.stdout: "service_discovery_port": { 2024-08-30T09:16:09.707 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "service_discovery_port", 2024-08-30T09:16:09.707 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:09.707 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.707 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:09.707 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "8765", 2024-08-30T09:16:09.707 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.707 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.707 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.707 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "cephadm service discovery port", 2024-08-30T09:16:09.708 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.708 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.708 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.708 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.708 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-08-30T09:16:09.708 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-08-30T09:16:09.708 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:09.708 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-08-30T09:16:09.708 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:09.708 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-08-30T09:16:09.708 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.708 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.708 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.708 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.709 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.709 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.709 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.709 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.709 INFO:teuthology.orchestra.run.smithi042.stdout: "ssh_config_file": { 2024-08-30T09:16:09.709 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ssh_config_file", 2024-08-30T09:16:09.709 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:09.709 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.709 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:09.709 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:09.710 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.710 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.710 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.710 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-08-30T09:16:09.710 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.710 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.710 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.710 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.710 INFO:teuthology.orchestra.run.smithi042.stdout: "use_agent": { 2024-08-30T09:16:09.710 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "use_agent", 2024-08-30T09:16:09.710 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:09.710 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.710 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:09.710 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:09.710 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.710 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.711 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.711 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-08-30T09:16:09.711 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.711 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.711 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.711 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.711 INFO:teuthology.orchestra.run.smithi042.stdout: "use_repo_digest": { 2024-08-30T09:16:09.711 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "use_repo_digest", 2024-08-30T09:16:09.711 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:09.711 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.711 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:09.711 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-08-30T09:16:09.711 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.712 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.712 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.712 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-08-30T09:16:09.712 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.712 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.712 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.712 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.712 INFO:teuthology.orchestra.run.smithi042.stdout: "warn_on_failed_host_check": { 2024-08-30T09:16:09.712 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "warn_on_failed_host_check", 2024-08-30T09:16:09.712 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:09.712 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.712 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:09.712 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-08-30T09:16:09.712 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.712 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.713 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.713 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "raise a health warning if the host check fails", 2024-08-30T09:16:09.713 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.713 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.713 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.713 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.713 INFO:teuthology.orchestra.run.smithi042.stdout: "warn_on_stray_daemons": { 2024-08-30T09:16:09.713 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "warn_on_stray_daemons", 2024-08-30T09:16:09.713 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:09.713 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.713 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:09.713 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-08-30T09:16:09.713 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.713 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.713 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.714 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-08-30T09:16:09.714 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.714 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.714 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.714 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.714 INFO:teuthology.orchestra.run.smithi042.stdout: "warn_on_stray_hosts": { 2024-08-30T09:16:09.714 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "warn_on_stray_hosts", 2024-08-30T09:16:09.714 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:09.714 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.714 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:09.714 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-08-30T09:16:09.714 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.714 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.714 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.714 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-08-30T09:16:09.715 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.715 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.715 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.715 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:09.715 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:09.715 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.715 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-08-30T09:16:09.715 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "crash", 2024-08-30T09:16:09.715 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-08-30T09:16:09.715 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-08-30T09:16:09.715 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-08-30T09:16:09.715 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-08-30T09:16:09.715 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-08-30T09:16:09.715 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:09.715 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.716 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:09.716 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:09.716 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.716 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.716 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:09.716 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:09.716 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:09.716 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:09.716 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:09.716 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:09.716 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:09.716 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:09.716 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.716 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.716 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.717 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.717 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.717 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-08-30T09:16:09.717 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-08-30T09:16:09.717 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:09.717 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.717 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:09.717 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:09.717 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.717 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.717 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.717 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.717 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.717 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.717 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.717 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.718 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-08-30T09:16:09.718 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-08-30T09:16:09.718 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:09.718 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.718 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:09.718 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-08-30T09:16:09.718 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.718 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.718 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:09.718 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:09.718 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:09.718 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:09.718 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:09.718 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:09.719 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:09.719 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:09.719 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.719 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.719 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.719 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.719 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.719 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-08-30T09:16:09.719 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-08-30T09:16:09.719 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:09.719 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.719 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:09.719 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:09.719 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.719 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.720 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.720 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.720 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.720 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.720 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.720 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.720 INFO:teuthology.orchestra.run.smithi042.stdout: "retain_interval": { 2024-08-30T09:16:09.720 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "retain_interval", 2024-08-30T09:16:09.720 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-08-30T09:16:09.720 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.720 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:09.720 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "31536000", 2024-08-30T09:16:09.720 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.720 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.720 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.720 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "how long to retain crashes before pruning them", 2024-08-30T09:16:09.721 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.721 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.721 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.721 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.721 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-08-30T09:16:09.721 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-08-30T09:16:09.721 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:09.721 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-08-30T09:16:09.721 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:09.721 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-08-30T09:16:09.721 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.721 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.721 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.721 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.721 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.722 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.722 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.722 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.722 INFO:teuthology.orchestra.run.smithi042.stdout: "warn_recent_interval": { 2024-08-30T09:16:09.722 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "warn_recent_interval", 2024-08-30T09:16:09.722 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-08-30T09:16:09.722 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.722 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:09.722 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "1209600", 2024-08-30T09:16:09.722 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.722 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.722 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.722 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "time interval in which to warn about recent crashes", 2024-08-30T09:16:09.722 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.722 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.723 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.723 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:09.723 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:09.723 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.723 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-08-30T09:16:09.723 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "dashboard", 2024-08-30T09:16:09.723 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-08-30T09:16:09.723 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-08-30T09:16:09.723 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-08-30T09:16:09.723 INFO:teuthology.orchestra.run.smithi042.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-08-30T09:16:09.723 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-08-30T09:16:09.723 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:09.723 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.723 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:09.724 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "10", 2024-08-30T09:16:09.724 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.724 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.724 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.724 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.724 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.724 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.724 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.724 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.724 INFO:teuthology.orchestra.run.smithi042.stdout: "ALERTMANAGER_API_HOST": { 2024-08-30T09:16:09.724 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ALERTMANAGER_API_HOST", 2024-08-30T09:16:09.724 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:09.724 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.725 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:09.725 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:09.725 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.725 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.725 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.725 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.725 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.725 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.725 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.725 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.725 INFO:teuthology.orchestra.run.smithi042.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-08-30T09:16:09.725 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-08-30T09:16:09.725 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:09.725 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.726 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:09.726 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-08-30T09:16:09.726 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.726 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.726 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.726 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.726 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.726 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.726 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.726 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.726 INFO:teuthology.orchestra.run.smithi042.stdout: "AUDIT_API_ENABLED": { 2024-08-30T09:16:09.726 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "AUDIT_API_ENABLED", 2024-08-30T09:16:09.726 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:09.727 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.727 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:09.727 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:09.727 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.727 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.727 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.727 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.727 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.727 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.727 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.727 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.727 INFO:teuthology.orchestra.run.smithi042.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-08-30T09:16:09.727 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-08-30T09:16:09.728 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:09.728 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.728 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:09.728 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-08-30T09:16:09.728 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.728 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.728 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.728 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.728 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.728 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.728 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.728 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.728 INFO:teuthology.orchestra.run.smithi042.stdout: "ENABLE_BROWSABLE_API": { 2024-08-30T09:16:09.729 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ENABLE_BROWSABLE_API", 2024-08-30T09:16:09.729 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:09.729 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.729 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:09.729 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-08-30T09:16:09.729 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.729 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.729 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.729 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.729 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.729 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.729 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.729 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.729 INFO:teuthology.orchestra.run.smithi042.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-08-30T09:16:09.730 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-08-30T09:16:09.730 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:09.730 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.730 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:09.730 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-08-30T09:16:09.730 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.730 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.730 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.730 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.730 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.730 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.730 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.730 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.730 INFO:teuthology.orchestra.run.smithi042.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-08-30T09:16:09.730 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-08-30T09:16:09.731 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:09.731 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.731 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:09.731 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-08-30T09:16:09.731 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.731 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.731 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.731 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.731 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.731 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.731 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.731 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.731 INFO:teuthology.orchestra.run.smithi042.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-08-30T09:16:09.731 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-08-30T09:16:09.731 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:09.732 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.732 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:09.732 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-08-30T09:16:09.732 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.732 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.732 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.732 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.732 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.732 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.732 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.732 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.732 INFO:teuthology.orchestra.run.smithi042.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-08-30T09:16:09.732 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-08-30T09:16:09.732 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:09.732 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.733 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:09.733 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-08-30T09:16:09.733 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.733 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.733 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.733 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.733 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.733 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.733 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.733 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.733 INFO:teuthology.orchestra.run.smithi042.stdout: "FEATURE_TOGGLE_NFS": { 2024-08-30T09:16:09.733 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-08-30T09:16:09.733 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:09.733 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.734 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:09.734 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-08-30T09:16:09.734 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.734 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.734 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.734 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.734 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.734 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.734 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.734 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.734 INFO:teuthology.orchestra.run.smithi042.stdout: "FEATURE_TOGGLE_RBD": { 2024-08-30T09:16:09.734 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-08-30T09:16:09.734 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:09.734 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.735 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:09.735 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-08-30T09:16:09.735 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.735 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.735 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.735 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.735 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.735 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.735 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.735 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.735 INFO:teuthology.orchestra.run.smithi042.stdout: "FEATURE_TOGGLE_RGW": { 2024-08-30T09:16:09.735 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-08-30T09:16:09.735 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:09.735 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.736 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:09.736 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-08-30T09:16:09.736 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.736 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.736 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.736 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.736 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.736 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.736 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.736 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.736 INFO:teuthology.orchestra.run.smithi042.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-08-30T09:16:09.736 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-08-30T09:16:09.736 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:09.736 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.737 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:09.737 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:09.737 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.737 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.737 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.737 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.737 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.737 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.737 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.737 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.737 INFO:teuthology.orchestra.run.smithi042.stdout: "GRAFANA_API_PASSWORD": { 2024-08-30T09:16:09.737 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "GRAFANA_API_PASSWORD", 2024-08-30T09:16:09.737 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:09.737 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.738 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:09.738 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "admin", 2024-08-30T09:16:09.738 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.738 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.738 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.738 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.738 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.738 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.738 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.738 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.738 INFO:teuthology.orchestra.run.smithi042.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-08-30T09:16:09.738 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-08-30T09:16:09.738 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:09.738 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.738 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:09.739 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-08-30T09:16:09.739 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.739 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.739 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.739 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.739 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.739 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.739 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.739 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.739 INFO:teuthology.orchestra.run.smithi042.stdout: "GRAFANA_API_URL": { 2024-08-30T09:16:09.739 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "GRAFANA_API_URL", 2024-08-30T09:16:09.739 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:09.739 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.739 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:09.740 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:09.740 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.740 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.740 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.740 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.740 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.740 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.740 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.740 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.740 INFO:teuthology.orchestra.run.smithi042.stdout: "GRAFANA_API_USERNAME": { 2024-08-30T09:16:09.740 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "GRAFANA_API_USERNAME", 2024-08-30T09:16:09.740 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:09.740 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.740 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:09.741 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "admin", 2024-08-30T09:16:09.741 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.741 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.741 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.741 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.741 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.741 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.741 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.741 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.741 INFO:teuthology.orchestra.run.smithi042.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-08-30T09:16:09.741 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-08-30T09:16:09.741 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:09.741 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.741 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:09.742 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:09.742 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.742 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.742 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.742 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.742 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.742 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.742 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.742 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.742 INFO:teuthology.orchestra.run.smithi042.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-08-30T09:16:09.742 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-08-30T09:16:09.742 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:09.742 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.742 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:09.742 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:09.743 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.743 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.743 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.743 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.743 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.743 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.743 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.743 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.743 INFO:teuthology.orchestra.run.smithi042.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-08-30T09:16:09.743 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-08-30T09:16:09.743 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:09.743 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.743 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:09.743 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-08-30T09:16:09.743 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.744 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.744 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.744 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.744 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.744 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.744 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.744 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.744 INFO:teuthology.orchestra.run.smithi042.stdout: "ISSUE_TRACKER_API_KEY": { 2024-08-30T09:16:09.744 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-08-30T09:16:09.744 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:09.744 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.744 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:09.744 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:09.744 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.745 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.745 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.745 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.745 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.745 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.745 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.745 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.745 INFO:teuthology.orchestra.run.smithi042.stdout: "MANAGED_BY_CLUSTERS": { 2024-08-30T09:16:09.745 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-08-30T09:16:09.745 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:09.745 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.745 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:09.745 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "[]", 2024-08-30T09:16:09.745 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.745 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.746 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.746 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.746 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.746 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.746 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.746 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.746 INFO:teuthology.orchestra.run.smithi042.stdout: "MULTICLUSTER_CONFIG": { 2024-08-30T09:16:09.746 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "MULTICLUSTER_CONFIG", 2024-08-30T09:16:09.746 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:09.746 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.746 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:09.746 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "{}", 2024-08-30T09:16:09.746 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.746 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.747 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.747 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.747 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.747 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.747 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.747 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.747 INFO:teuthology.orchestra.run.smithi042.stdout: "PROMETHEUS_API_HOST": { 2024-08-30T09:16:09.747 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PROMETHEUS_API_HOST", 2024-08-30T09:16:09.747 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:09.747 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.747 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:09.747 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:09.747 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.747 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.747 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.748 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.748 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.748 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.748 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.748 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.748 INFO:teuthology.orchestra.run.smithi042.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-08-30T09:16:09.748 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-08-30T09:16:09.748 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:09.748 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.748 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:09.748 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-08-30T09:16:09.748 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.748 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.748 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.748 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.749 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.749 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.749 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.749 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.749 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-08-30T09:16:09.749 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-08-30T09:16:09.749 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:09.749 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.749 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:09.749 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:09.749 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.749 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.749 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.749 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.750 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.750 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.750 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.750 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.750 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-08-30T09:16:09.750 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-08-30T09:16:09.750 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:09.750 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.750 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:09.750 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:09.750 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.750 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.750 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.750 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.750 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.751 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.751 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.751 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.751 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-08-30T09:16:09.751 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-08-30T09:16:09.751 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:09.751 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.751 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:09.751 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-08-30T09:16:09.751 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.751 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.751 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.751 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.752 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.752 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.752 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.752 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.752 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-08-30T09:16:09.752 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-08-30T09:16:09.752 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:09.752 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.752 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:09.752 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-08-30T09:16:09.752 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.752 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.752 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.752 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.753 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.753 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.753 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.753 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.753 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-08-30T09:16:09.753 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-08-30T09:16:09.753 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:09.753 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.753 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:09.753 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:09.753 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.753 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.753 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.753 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.753 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.754 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.754 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.754 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.754 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-08-30T09:16:09.754 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-08-30T09:16:09.754 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:09.754 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.754 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:09.754 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:09.754 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.754 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.754 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.754 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.754 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.754 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.755 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.755 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.755 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-08-30T09:16:09.755 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-08-30T09:16:09.755 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:09.755 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.755 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:09.755 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:09.755 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.755 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.755 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.755 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.755 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.755 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.755 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.756 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.756 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_ENABLED": { 2024-08-30T09:16:09.756 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_ENABLED", 2024-08-30T09:16:09.756 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:09.756 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.756 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:09.756 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-08-30T09:16:09.756 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.756 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.756 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.756 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.756 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.756 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.756 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.757 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.757 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-08-30T09:16:09.757 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-08-30T09:16:09.757 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:09.757 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.757 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:09.757 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-08-30T09:16:09.757 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.757 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.757 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.757 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.757 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.757 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.757 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.758 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.758 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-08-30T09:16:09.758 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-08-30T09:16:09.758 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:09.758 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.758 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:09.758 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "10", 2024-08-30T09:16:09.758 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.758 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.758 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.758 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.758 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.758 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.758 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.758 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.759 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-08-30T09:16:09.759 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-08-30T09:16:09.759 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:09.759 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.759 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:09.759 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "8", 2024-08-30T09:16:09.759 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.759 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.759 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.759 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.759 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.759 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.759 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.759 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.759 INFO:teuthology.orchestra.run.smithi042.stdout: "REST_REQUESTS_TIMEOUT": { 2024-08-30T09:16:09.759 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-08-30T09:16:09.760 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:09.760 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.760 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:09.760 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "45", 2024-08-30T09:16:09.760 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.760 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.760 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.760 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.760 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.760 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.760 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.760 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.760 INFO:teuthology.orchestra.run.smithi042.stdout: "RGW_API_ACCESS_KEY": { 2024-08-30T09:16:09.760 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "RGW_API_ACCESS_KEY", 2024-08-30T09:16:09.761 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:09.761 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.761 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:09.761 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:09.761 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.761 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.761 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.761 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.761 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.761 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.761 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.761 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.761 INFO:teuthology.orchestra.run.smithi042.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-08-30T09:16:09.761 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-08-30T09:16:09.761 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:09.762 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.762 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:09.762 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "admin", 2024-08-30T09:16:09.762 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.762 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.762 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.762 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.762 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.762 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.762 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.762 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.762 INFO:teuthology.orchestra.run.smithi042.stdout: "RGW_API_SECRET_KEY": { 2024-08-30T09:16:09.762 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "RGW_API_SECRET_KEY", 2024-08-30T09:16:09.762 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:09.762 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.763 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:09.763 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:09.763 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.763 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.763 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.763 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.763 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.763 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.763 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.763 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.763 INFO:teuthology.orchestra.run.smithi042.stdout: "RGW_API_SSL_VERIFY": { 2024-08-30T09:16:09.763 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "RGW_API_SSL_VERIFY", 2024-08-30T09:16:09.763 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:09.763 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.764 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:09.764 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-08-30T09:16:09.764 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.764 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.764 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.764 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.764 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.764 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.764 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.764 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.764 INFO:teuthology.orchestra.run.smithi042.stdout: "UNSAFE_TLS_v1_2": { 2024-08-30T09:16:09.764 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "UNSAFE_TLS_v1_2", 2024-08-30T09:16:09.764 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:09.764 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.764 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:09.765 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:09.765 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.765 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.765 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.765 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.765 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.765 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.765 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.765 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.765 INFO:teuthology.orchestra.run.smithi042.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-08-30T09:16:09.765 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-08-30T09:16:09.765 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:09.765 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.765 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:09.766 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-08-30T09:16:09.766 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.766 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.766 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.766 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.766 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.766 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.766 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.766 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.766 INFO:teuthology.orchestra.run.smithi042.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-08-30T09:16:09.766 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-08-30T09:16:09.766 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:09.766 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.766 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:09.766 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "10", 2024-08-30T09:16:09.767 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.767 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.767 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.767 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.767 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.767 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.767 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.767 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.767 INFO:teuthology.orchestra.run.smithi042.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-08-30T09:16:09.767 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-08-30T09:16:09.767 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:09.767 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.767 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:09.767 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "5", 2024-08-30T09:16:09.767 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.768 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.768 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.768 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.768 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.768 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.768 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.768 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.768 INFO:teuthology.orchestra.run.smithi042.stdout: "cross_origin_url": { 2024-08-30T09:16:09.768 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "cross_origin_url", 2024-08-30T09:16:09.768 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:09.768 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.768 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:09.768 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:09.768 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.768 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.769 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.769 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.769 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.769 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.769 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.769 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.769 INFO:teuthology.orchestra.run.smithi042.stdout: "crt_file": { 2024-08-30T09:16:09.769 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "crt_file", 2024-08-30T09:16:09.769 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:09.769 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.769 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:09.769 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:09.769 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.769 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.770 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.770 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.770 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.770 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.770 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.770 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.770 INFO:teuthology.orchestra.run.smithi042.stdout: "debug": { 2024-08-30T09:16:09.770 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "debug", 2024-08-30T09:16:09.770 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:09.770 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.770 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:09.770 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:09.770 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.770 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.771 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.771 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Enable/disable debug options", 2024-08-30T09:16:09.771 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.771 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.771 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.771 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.771 INFO:teuthology.orchestra.run.smithi042.stdout: "jwt_token_ttl": { 2024-08-30T09:16:09.771 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "jwt_token_ttl", 2024-08-30T09:16:09.771 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:09.771 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.771 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:09.771 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "28800", 2024-08-30T09:16:09.771 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.771 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.772 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.772 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.772 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.772 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.772 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.772 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.772 INFO:teuthology.orchestra.run.smithi042.stdout: "key_file": { 2024-08-30T09:16:09.772 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "key_file", 2024-08-30T09:16:09.772 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:09.772 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.772 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:09.772 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:09.772 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.772 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.772 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.773 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.773 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.773 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.773 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.773 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.773 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-08-30T09:16:09.773 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-08-30T09:16:09.773 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:09.773 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.773 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:09.773 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:09.773 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.773 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.773 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:09.773 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:09.773 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:09.774 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:09.774 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:09.774 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:09.774 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:09.774 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:09.774 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.774 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.774 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.774 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.774 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.774 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-08-30T09:16:09.774 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-08-30T09:16:09.774 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:09.774 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.775 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:09.775 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:09.775 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.775 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.775 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.775 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.775 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.775 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.775 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.775 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.775 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-08-30T09:16:09.775 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-08-30T09:16:09.775 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:09.775 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.776 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:09.776 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-08-30T09:16:09.776 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.776 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.776 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:09.776 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:09.776 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:09.776 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:09.776 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:09.776 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:09.776 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:09.776 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:09.776 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.776 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.777 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.777 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.777 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.777 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-08-30T09:16:09.777 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-08-30T09:16:09.777 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:09.777 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.777 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:09.777 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:09.777 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.777 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.777 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.777 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.777 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.777 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.778 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.778 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.778 INFO:teuthology.orchestra.run.smithi042.stdout: "motd": { 2024-08-30T09:16:09.778 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "motd", 2024-08-30T09:16:09.778 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:09.778 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.778 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:09.778 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:09.778 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.778 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.778 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.778 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "The message of the day", 2024-08-30T09:16:09.778 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.778 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.778 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.779 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.779 INFO:teuthology.orchestra.run.smithi042.stdout: "redirect_resolve_ip_addr": { 2024-08-30T09:16:09.779 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "redirect_resolve_ip_addr", 2024-08-30T09:16:09.779 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:09.779 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.779 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:09.779 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:09.779 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.779 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.779 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.779 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.779 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.779 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.779 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.780 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.780 INFO:teuthology.orchestra.run.smithi042.stdout: "server_addr": { 2024-08-30T09:16:09.780 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "server_addr", 2024-08-30T09:16:09.780 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:09.780 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.780 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:09.780 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "::", 2024-08-30T09:16:09.780 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.780 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.780 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.780 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.780 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.780 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.781 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.781 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.781 INFO:teuthology.orchestra.run.smithi042.stdout: "server_port": { 2024-08-30T09:16:09.781 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "server_port", 2024-08-30T09:16:09.781 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:09.781 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.781 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:09.781 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "8080", 2024-08-30T09:16:09.781 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.781 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.781 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.781 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.781 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.781 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.782 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.782 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.782 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-08-30T09:16:09.782 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-08-30T09:16:09.782 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:09.782 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-08-30T09:16:09.782 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:09.782 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-08-30T09:16:09.782 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.782 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.782 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.782 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.782 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.782 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.782 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.782 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.783 INFO:teuthology.orchestra.run.smithi042.stdout: "ssl": { 2024-08-30T09:16:09.783 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ssl", 2024-08-30T09:16:09.783 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:09.783 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.783 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:09.783 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-08-30T09:16:09.783 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.783 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.783 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.783 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.783 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.783 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.783 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.783 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.783 INFO:teuthology.orchestra.run.smithi042.stdout: "ssl_server_port": { 2024-08-30T09:16:09.784 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ssl_server_port", 2024-08-30T09:16:09.784 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:09.784 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.784 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:09.784 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "8443", 2024-08-30T09:16:09.784 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.784 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.784 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.784 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.784 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.784 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.784 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.784 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.785 INFO:teuthology.orchestra.run.smithi042.stdout: "standby_behaviour": { 2024-08-30T09:16:09.785 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "standby_behaviour", 2024-08-30T09:16:09.785 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:09.785 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.785 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:09.785 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "redirect", 2024-08-30T09:16:09.785 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.785 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.785 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:09.785 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:09.785 INFO:teuthology.orchestra.run.smithi042.stdout: "redirect" 2024-08-30T09:16:09.785 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:09.785 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.786 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.786 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.786 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.786 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.786 INFO:teuthology.orchestra.run.smithi042.stdout: "standby_error_status_code": { 2024-08-30T09:16:09.786 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "standby_error_status_code", 2024-08-30T09:16:09.786 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:09.786 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.786 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:09.786 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "500", 2024-08-30T09:16:09.786 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "400", 2024-08-30T09:16:09.786 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "599", 2024-08-30T09:16:09.786 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.786 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.787 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.787 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.787 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.787 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.787 INFO:teuthology.orchestra.run.smithi042.stdout: "url_prefix": { 2024-08-30T09:16:09.787 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "url_prefix", 2024-08-30T09:16:09.787 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:09.787 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.787 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:09.787 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:09.787 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.787 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.787 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.787 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.787 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.787 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.788 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.788 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:09.788 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:09.788 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.788 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-08-30T09:16:09.788 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "devicehealth", 2024-08-30T09:16:09.788 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-08-30T09:16:09.788 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-08-30T09:16:09.788 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-08-30T09:16:09.788 INFO:teuthology.orchestra.run.smithi042.stdout: "enable_monitoring": { 2024-08-30T09:16:09.788 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "enable_monitoring", 2024-08-30T09:16:09.788 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:09.788 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.788 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:09.789 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-08-30T09:16:09.789 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.789 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.789 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.789 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "monitor device health metrics", 2024-08-30T09:16:09.789 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.789 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.789 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.789 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.789 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-08-30T09:16:09.789 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-08-30T09:16:09.789 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:09.789 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.789 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:09.790 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:09.790 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.790 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.790 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:09.790 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:09.790 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:09.790 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:09.790 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:09.790 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:09.790 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:09.790 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:09.790 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.790 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.790 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.790 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.790 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.791 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-08-30T09:16:09.791 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-08-30T09:16:09.791 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:09.791 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.791 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:09.791 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:09.791 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.791 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.791 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.791 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.791 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.791 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.791 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.791 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.791 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-08-30T09:16:09.792 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-08-30T09:16:09.792 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:09.792 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.792 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:09.792 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-08-30T09:16:09.792 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.792 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.792 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:09.792 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:09.792 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:09.792 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:09.792 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:09.792 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:09.792 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:09.792 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:09.793 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.793 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.793 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.793 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.793 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.793 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-08-30T09:16:09.793 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-08-30T09:16:09.793 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:09.793 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.793 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:09.793 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:09.793 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.793 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.793 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.793 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.794 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.794 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.794 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.794 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.794 INFO:teuthology.orchestra.run.smithi042.stdout: "mark_out_threshold": { 2024-08-30T09:16:09.794 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "mark_out_threshold", 2024-08-30T09:16:09.794 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-08-30T09:16:09.794 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.794 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:09.794 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "2419200", 2024-08-30T09:16:09.794 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.794 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.794 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.794 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-08-30T09:16:09.794 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.795 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.795 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.795 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.795 INFO:teuthology.orchestra.run.smithi042.stdout: "pool_name": { 2024-08-30T09:16:09.795 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "pool_name", 2024-08-30T09:16:09.795 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:09.795 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.795 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:09.795 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "device_health_metrics", 2024-08-30T09:16:09.795 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.795 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.795 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.795 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "name of pool in which to store device health metrics", 2024-08-30T09:16:09.795 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.796 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.796 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.796 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.796 INFO:teuthology.orchestra.run.smithi042.stdout: "retention_period": { 2024-08-30T09:16:09.796 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "retention_period", 2024-08-30T09:16:09.796 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-08-30T09:16:09.796 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.796 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:09.796 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "15552000", 2024-08-30T09:16:09.796 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.796 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.796 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.796 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "how long to retain device health metrics", 2024-08-30T09:16:09.796 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.797 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.797 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.797 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.797 INFO:teuthology.orchestra.run.smithi042.stdout: "scrape_frequency": { 2024-08-30T09:16:09.797 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "scrape_frequency", 2024-08-30T09:16:09.797 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-08-30T09:16:09.797 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.797 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:09.797 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "86400", 2024-08-30T09:16:09.797 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.797 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.797 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.797 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "how frequently to scrape device health metrics", 2024-08-30T09:16:09.797 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.798 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.798 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.798 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.798 INFO:teuthology.orchestra.run.smithi042.stdout: "self_heal": { 2024-08-30T09:16:09.798 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "self_heal", 2024-08-30T09:16:09.798 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:09.798 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.798 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:09.798 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-08-30T09:16:09.798 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.798 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.798 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.798 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-08-30T09:16:09.799 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.799 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.799 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.799 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.799 INFO:teuthology.orchestra.run.smithi042.stdout: "sleep_interval": { 2024-08-30T09:16:09.799 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sleep_interval", 2024-08-30T09:16:09.799 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-08-30T09:16:09.799 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.799 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:09.799 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "600", 2024-08-30T09:16:09.799 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.799 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.799 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.799 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "how frequently to wake up and check device health", 2024-08-30T09:16:09.800 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.800 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.800 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.800 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.800 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-08-30T09:16:09.800 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-08-30T09:16:09.800 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:09.800 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-08-30T09:16:09.800 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:09.800 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-08-30T09:16:09.800 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.800 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.801 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.801 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.801 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.801 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.801 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.801 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.801 INFO:teuthology.orchestra.run.smithi042.stdout: "warn_threshold": { 2024-08-30T09:16:09.801 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "warn_threshold", 2024-08-30T09:16:09.801 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-08-30T09:16:09.801 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.801 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:09.801 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "7257600", 2024-08-30T09:16:09.801 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.801 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.802 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.802 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-08-30T09:16:09.802 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.802 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.802 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.802 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:09.802 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:09.802 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.802 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-08-30T09:16:09.802 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "diskprediction_local", 2024-08-30T09:16:09.802 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-08-30T09:16:09.802 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-08-30T09:16:09.802 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-08-30T09:16:09.802 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-08-30T09:16:09.803 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-08-30T09:16:09.803 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:09.803 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.803 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:09.803 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:09.803 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.803 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.803 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:09.803 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:09.803 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:09.803 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:09.803 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:09.803 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:09.803 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:09.803 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:09.803 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.804 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.804 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.804 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.804 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.804 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-08-30T09:16:09.804 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-08-30T09:16:09.804 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:09.804 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.804 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:09.804 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:09.804 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.804 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.804 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.805 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.805 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.805 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.805 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.805 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.805 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-08-30T09:16:09.805 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-08-30T09:16:09.805 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:09.805 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.805 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:09.805 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-08-30T09:16:09.805 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.805 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.805 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:09.805 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:09.806 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:09.806 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:09.806 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:09.806 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:09.806 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:09.806 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:09.806 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.806 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.806 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.806 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.806 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.806 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-08-30T09:16:09.806 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-08-30T09:16:09.806 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:09.806 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.807 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:09.807 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:09.807 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.807 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.807 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.807 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.807 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.807 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.807 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.807 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.807 INFO:teuthology.orchestra.run.smithi042.stdout: "predict_interval": { 2024-08-30T09:16:09.807 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "predict_interval", 2024-08-30T09:16:09.807 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:09.807 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.808 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:09.808 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "86400", 2024-08-30T09:16:09.808 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.808 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.808 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.808 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.808 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.808 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.808 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.808 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.808 INFO:teuthology.orchestra.run.smithi042.stdout: "predictor_model": { 2024-08-30T09:16:09.808 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "predictor_model", 2024-08-30T09:16:09.808 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:09.808 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.808 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:09.809 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "prophetstor", 2024-08-30T09:16:09.809 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.809 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.809 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.809 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.809 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.809 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.809 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.809 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.809 INFO:teuthology.orchestra.run.smithi042.stdout: "sleep_interval": { 2024-08-30T09:16:09.810 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sleep_interval", 2024-08-30T09:16:09.810 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:09.810 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.810 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:09.810 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "600", 2024-08-30T09:16:09.810 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.810 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.810 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.810 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.810 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.810 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.810 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.810 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.810 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-08-30T09:16:09.810 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-08-30T09:16:09.811 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:09.811 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-08-30T09:16:09.811 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:09.811 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-08-30T09:16:09.811 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.811 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.811 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.811 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.811 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.811 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.811 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.811 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:09.811 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:09.811 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.811 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-08-30T09:16:09.812 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "influx", 2024-08-30T09:16:09.812 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": false, 2024-08-30T09:16:09.812 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "influxdb python module not found", 2024-08-30T09:16:09.812 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-08-30T09:16:09.812 INFO:teuthology.orchestra.run.smithi042.stdout: "batch_size": { 2024-08-30T09:16:09.812 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "batch_size", 2024-08-30T09:16:09.812 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:09.812 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.812 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:09.812 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "5000", 2024-08-30T09:16:09.812 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.812 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.812 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.813 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-08-30T09:16:09.813 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.813 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.813 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.813 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.813 INFO:teuthology.orchestra.run.smithi042.stdout: "database": { 2024-08-30T09:16:09.813 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "database", 2024-08-30T09:16:09.813 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:09.813 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.813 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:09.813 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "ceph", 2024-08-30T09:16:09.813 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.813 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.813 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.813 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-08-30T09:16:09.814 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.814 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.814 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.814 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.814 INFO:teuthology.orchestra.run.smithi042.stdout: "hostname": { 2024-08-30T09:16:09.814 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "hostname", 2024-08-30T09:16:09.814 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:09.814 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.814 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:09.814 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:09.814 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.814 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.814 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.814 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "InfluxDB server hostname", 2024-08-30T09:16:09.815 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.815 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.815 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.815 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.815 INFO:teuthology.orchestra.run.smithi042.stdout: "interval": { 2024-08-30T09:16:09.815 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "interval", 2024-08-30T09:16:09.815 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-08-30T09:16:09.815 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.815 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:09.815 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "30", 2024-08-30T09:16:09.815 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "5", 2024-08-30T09:16:09.815 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.815 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.815 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-08-30T09:16:09.815 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.816 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.816 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.816 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.816 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-08-30T09:16:09.816 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-08-30T09:16:09.816 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:09.816 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.816 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:09.816 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:09.816 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.816 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.816 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:09.816 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:09.816 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:09.817 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:09.817 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:09.817 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:09.817 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:09.817 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:09.817 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.817 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.817 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.817 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.817 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.817 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-08-30T09:16:09.817 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-08-30T09:16:09.817 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:09.817 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.817 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:09.818 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:09.818 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.818 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.818 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.818 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.818 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.818 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.818 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.818 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.818 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-08-30T09:16:09.818 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-08-30T09:16:09.818 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:09.818 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.818 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:09.818 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-08-30T09:16:09.819 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.819 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.819 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:09.819 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:09.819 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:09.819 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:09.819 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:09.819 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:09.819 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:09.819 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:09.819 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.819 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.819 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.819 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.819 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.820 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-08-30T09:16:09.820 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-08-30T09:16:09.820 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:09.820 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.820 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:09.820 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:09.820 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.820 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.820 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.820 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.820 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.820 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.820 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.820 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.820 INFO:teuthology.orchestra.run.smithi042.stdout: "password": { 2024-08-30T09:16:09.821 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "password", 2024-08-30T09:16:09.821 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:09.821 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.821 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:09.821 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:09.821 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.821 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.821 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.821 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "password of InfluxDB server user", 2024-08-30T09:16:09.821 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.821 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.821 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.821 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.821 INFO:teuthology.orchestra.run.smithi042.stdout: "port": { 2024-08-30T09:16:09.821 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "port", 2024-08-30T09:16:09.822 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:09.822 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.822 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:09.822 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "8086", 2024-08-30T09:16:09.822 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.822 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.822 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.822 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "InfluxDB server port", 2024-08-30T09:16:09.822 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.822 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.822 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.822 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.822 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-08-30T09:16:09.822 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-08-30T09:16:09.822 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:09.823 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-08-30T09:16:09.823 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:09.823 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-08-30T09:16:09.823 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.823 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.823 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.823 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.823 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.823 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.823 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.823 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.823 INFO:teuthology.orchestra.run.smithi042.stdout: "ssl": { 2024-08-30T09:16:09.823 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ssl", 2024-08-30T09:16:09.823 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:09.824 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.824 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:09.824 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "false", 2024-08-30T09:16:09.824 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.824 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.824 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.824 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-08-30T09:16:09.824 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.824 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.824 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.824 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.824 INFO:teuthology.orchestra.run.smithi042.stdout: "threads": { 2024-08-30T09:16:09.824 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "threads", 2024-08-30T09:16:09.824 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:09.825 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.825 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:09.825 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "5", 2024-08-30T09:16:09.825 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "1", 2024-08-30T09:16:09.825 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "32", 2024-08-30T09:16:09.825 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.825 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-08-30T09:16:09.825 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.825 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.825 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.825 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.825 INFO:teuthology.orchestra.run.smithi042.stdout: "username": { 2024-08-30T09:16:09.825 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "username", 2024-08-30T09:16:09.825 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:09.825 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.826 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:09.826 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:09.826 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.826 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.826 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.826 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "username of InfluxDB server user", 2024-08-30T09:16:09.826 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.826 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.826 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.826 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.826 INFO:teuthology.orchestra.run.smithi042.stdout: "verify_ssl": { 2024-08-30T09:16:09.826 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "verify_ssl", 2024-08-30T09:16:09.826 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:09.826 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.826 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:09.827 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "true", 2024-08-30T09:16:09.827 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.827 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.827 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.827 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-08-30T09:16:09.827 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.827 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.827 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.827 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:09.827 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:09.827 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.827 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-08-30T09:16:09.827 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "insights", 2024-08-30T09:16:09.827 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-08-30T09:16:09.828 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-08-30T09:16:09.828 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-08-30T09:16:09.828 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-08-30T09:16:09.828 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-08-30T09:16:09.828 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:09.828 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.828 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:09.828 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:09.828 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.828 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.828 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:09.828 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:09.828 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:09.828 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:09.829 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:09.829 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:09.829 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:09.829 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:09.829 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.829 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.829 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.829 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.829 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.829 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-08-30T09:16:09.829 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-08-30T09:16:09.829 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:09.829 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.830 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:09.830 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:09.830 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.830 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.830 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.830 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.830 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.830 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.830 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.830 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.830 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-08-30T09:16:09.830 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-08-30T09:16:09.830 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:09.830 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.830 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:09.831 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-08-30T09:16:09.831 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.831 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.831 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:09.831 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:09.831 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:09.831 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:09.831 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:09.831 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:09.831 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:09.831 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:09.831 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.831 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.831 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.831 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.832 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.832 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-08-30T09:16:09.832 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-08-30T09:16:09.832 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:09.832 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.832 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:09.832 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:09.832 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.832 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.832 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.832 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.832 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.832 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.832 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.833 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.833 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-08-30T09:16:09.833 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-08-30T09:16:09.833 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:09.833 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-08-30T09:16:09.833 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:09.833 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-08-30T09:16:09.833 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.833 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.833 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.833 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.833 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.833 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.834 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.834 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:09.834 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:09.834 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.834 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-08-30T09:16:09.834 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "iostat", 2024-08-30T09:16:09.834 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-08-30T09:16:09.834 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-08-30T09:16:09.834 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-08-30T09:16:09.834 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-08-30T09:16:09.834 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-08-30T09:16:09.834 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:09.834 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.834 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:09.835 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:09.835 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.835 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.835 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:09.835 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:09.835 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:09.835 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:09.835 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:09.835 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:09.835 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:09.835 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:09.835 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.835 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.835 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.836 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.836 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.836 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-08-30T09:16:09.836 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-08-30T09:16:09.836 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:09.836 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.836 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:09.836 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:09.836 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.836 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.836 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.836 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.836 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.836 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.836 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.837 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.837 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-08-30T09:16:09.837 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-08-30T09:16:09.837 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:09.837 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.837 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:09.837 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-08-30T09:16:09.837 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.837 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.837 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:09.837 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:09.837 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:09.837 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:09.837 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:09.838 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:09.838 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:09.838 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:09.838 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.838 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.838 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.838 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.838 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.838 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-08-30T09:16:09.838 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-08-30T09:16:09.838 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:09.838 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.838 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:09.838 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:09.838 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.838 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.839 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.839 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.839 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.839 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.839 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.839 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.839 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-08-30T09:16:09.839 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-08-30T09:16:09.839 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:09.839 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-08-30T09:16:09.839 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:09.839 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-08-30T09:16:09.839 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.839 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.840 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.840 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.840 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.840 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.840 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.840 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:09.840 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:09.840 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.840 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-08-30T09:16:09.840 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "k8sevents", 2024-08-30T09:16:09.840 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-08-30T09:16:09.840 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-08-30T09:16:09.840 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-08-30T09:16:09.840 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph_event_retention_days": { 2024-08-30T09:16:09.841 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ceph_event_retention_days", 2024-08-30T09:16:09.841 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:09.841 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.841 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:09.841 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "7", 2024-08-30T09:16:09.841 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.841 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.841 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.841 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Days to hold ceph event information within local cache", 2024-08-30T09:16:09.841 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.841 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.841 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.841 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.842 INFO:teuthology.orchestra.run.smithi042.stdout: "config_check_secs": { 2024-08-30T09:16:09.842 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "config_check_secs", 2024-08-30T09:16:09.842 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:09.842 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.842 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:09.842 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "10", 2024-08-30T09:16:09.842 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "10", 2024-08-30T09:16:09.842 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.842 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.842 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-08-30T09:16:09.842 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.842 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.842 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.842 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.843 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-08-30T09:16:09.843 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-08-30T09:16:09.843 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:09.843 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.843 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:09.843 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:09.843 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.843 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.843 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:09.843 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:09.843 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:09.843 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:09.843 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:09.843 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:09.843 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:09.844 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:09.844 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.844 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.844 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.844 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.844 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.844 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-08-30T09:16:09.844 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-08-30T09:16:09.844 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:09.844 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.844 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:09.844 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:09.844 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.844 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.844 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.845 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.845 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.845 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.845 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.845 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.845 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-08-30T09:16:09.845 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-08-30T09:16:09.845 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:09.845 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.845 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:09.845 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-08-30T09:16:09.845 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.845 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.845 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:09.846 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:09.846 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:09.846 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:09.846 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:09.846 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:09.846 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:09.846 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:09.846 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.846 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.846 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.846 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.846 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.846 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-08-30T09:16:09.846 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-08-30T09:16:09.847 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:09.847 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.847 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:09.847 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:09.847 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.847 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.847 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.847 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.847 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.847 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.847 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.847 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.847 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-08-30T09:16:09.848 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-08-30T09:16:09.848 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:09.848 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-08-30T09:16:09.848 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:09.848 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-08-30T09:16:09.848 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.848 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.848 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.848 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.848 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.848 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.848 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.848 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:09.848 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:09.848 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.848 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-08-30T09:16:09.849 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "localpool", 2024-08-30T09:16:09.849 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-08-30T09:16:09.849 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-08-30T09:16:09.849 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-08-30T09:16:09.849 INFO:teuthology.orchestra.run.smithi042.stdout: "failure_domain": { 2024-08-30T09:16:09.849 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "failure_domain", 2024-08-30T09:16:09.849 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:09.849 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.849 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:09.849 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "host", 2024-08-30T09:16:09.849 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.849 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.849 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.849 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "failure domain for any created local pool", 2024-08-30T09:16:09.850 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-08-30T09:16:09.850 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.850 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.850 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.850 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-08-30T09:16:09.850 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-08-30T09:16:09.850 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:09.850 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.850 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:09.850 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:09.850 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.850 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.850 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:09.850 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:09.850 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:09.851 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:09.851 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:09.851 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:09.851 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:09.851 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:09.851 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.851 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.851 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.851 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.851 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.851 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-08-30T09:16:09.851 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-08-30T09:16:09.851 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:09.852 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.852 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:09.852 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:09.852 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.852 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.852 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.852 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.852 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.852 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.852 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.852 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.852 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-08-30T09:16:09.852 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-08-30T09:16:09.852 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:09.853 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.853 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:09.853 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-08-30T09:16:09.853 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.853 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.853 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:09.853 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:09.853 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:09.853 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:09.853 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:09.853 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:09.853 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:09.853 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:09.853 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.853 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.854 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.854 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.854 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.854 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-08-30T09:16:09.854 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-08-30T09:16:09.854 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:09.854 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.854 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:09.854 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:09.854 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.854 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.854 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.854 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.854 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.854 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.855 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.855 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.855 INFO:teuthology.orchestra.run.smithi042.stdout: "min_size": { 2024-08-30T09:16:09.855 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "min_size", 2024-08-30T09:16:09.855 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:09.855 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.855 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:09.855 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:09.855 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.855 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.855 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.855 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "default min_size for any created local pool", 2024-08-30T09:16:09.856 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-08-30T09:16:09.856 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.856 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.856 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.856 INFO:teuthology.orchestra.run.smithi042.stdout: "num_rep": { 2024-08-30T09:16:09.856 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "num_rep", 2024-08-30T09:16:09.856 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:09.856 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.856 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:09.856 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "3", 2024-08-30T09:16:09.856 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.856 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.857 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.857 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "default replica count for any created local pool", 2024-08-30T09:16:09.857 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.857 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.857 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.857 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.857 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num": { 2024-08-30T09:16:09.857 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "pg_num", 2024-08-30T09:16:09.857 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:09.857 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.857 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:09.857 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "128", 2024-08-30T09:16:09.857 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.858 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.858 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.858 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "default pg_num for any created local pool", 2024-08-30T09:16:09.858 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.858 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.858 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.858 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.858 INFO:teuthology.orchestra.run.smithi042.stdout: "prefix": { 2024-08-30T09:16:09.858 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "prefix", 2024-08-30T09:16:09.858 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:09.858 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.858 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:09.858 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:09.858 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.859 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.859 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.859 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "name prefix for any created local pool", 2024-08-30T09:16:09.859 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.859 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.859 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.859 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.859 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-08-30T09:16:09.859 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-08-30T09:16:09.859 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:09.859 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-08-30T09:16:09.859 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:09.859 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-08-30T09:16:09.860 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.860 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.860 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.860 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.860 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.860 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.860 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.860 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.860 INFO:teuthology.orchestra.run.smithi042.stdout: "subtree": { 2024-08-30T09:16:09.860 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "subtree", 2024-08-30T09:16:09.860 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:09.860 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.860 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:09.860 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "rack", 2024-08-30T09:16:09.860 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.861 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.861 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.861 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "CRUSH level for which to create a local pool", 2024-08-30T09:16:09.861 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-08-30T09:16:09.861 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.861 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.861 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:09.861 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:09.861 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.861 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-08-30T09:16:09.861 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "mds_autoscaler", 2024-08-30T09:16:09.861 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-08-30T09:16:09.862 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-08-30T09:16:09.862 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-08-30T09:16:09.862 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-08-30T09:16:09.862 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-08-30T09:16:09.862 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:09.862 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.862 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:09.862 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:09.862 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.862 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.862 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:09.862 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:09.862 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:09.862 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:09.862 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:09.863 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:09.863 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:09.863 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:09.863 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.863 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.863 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.863 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.863 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.863 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-08-30T09:16:09.863 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-08-30T09:16:09.863 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:09.863 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.863 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:09.863 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:09.863 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.864 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.864 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.864 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.864 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.864 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.864 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.864 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.864 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-08-30T09:16:09.864 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-08-30T09:16:09.864 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:09.864 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.864 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:09.864 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-08-30T09:16:09.864 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.864 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.865 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:09.865 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:09.865 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:09.865 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:09.865 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:09.865 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:09.865 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:09.865 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:09.865 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.865 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.865 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.865 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.865 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.865 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-08-30T09:16:09.866 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-08-30T09:16:09.866 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:09.866 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.866 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:09.866 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:09.866 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.866 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.866 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.866 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.866 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.866 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.866 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.866 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.866 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-08-30T09:16:09.867 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-08-30T09:16:09.867 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:09.867 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-08-30T09:16:09.867 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:09.867 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-08-30T09:16:09.867 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.867 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.867 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.867 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.867 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.867 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.867 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.867 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:09.867 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:09.867 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.868 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-08-30T09:16:09.868 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "mirroring", 2024-08-30T09:16:09.868 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-08-30T09:16:09.868 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-08-30T09:16:09.868 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-08-30T09:16:09.868 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-08-30T09:16:09.868 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-08-30T09:16:09.868 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:09.868 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.868 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:09.868 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:09.868 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.868 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.868 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:09.868 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:09.869 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:09.869 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:09.869 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:09.869 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:09.869 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:09.869 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:09.869 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.869 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.869 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.869 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.869 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.869 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-08-30T09:16:09.869 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-08-30T09:16:09.869 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:09.870 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.870 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:09.870 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:09.870 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.870 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.870 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.870 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.870 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.870 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.870 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.870 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.870 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-08-30T09:16:09.870 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-08-30T09:16:09.870 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:09.870 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.871 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:09.871 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-08-30T09:16:09.871 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.871 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.871 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:09.871 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:09.871 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:09.871 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:09.871 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:09.871 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:09.871 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:09.871 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:09.871 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.871 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.871 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.872 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.872 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.872 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-08-30T09:16:09.872 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-08-30T09:16:09.872 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:09.872 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.872 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:09.872 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:09.872 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.872 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.872 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.872 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.872 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.872 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.873 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.873 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.873 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-08-30T09:16:09.873 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-08-30T09:16:09.873 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:09.873 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-08-30T09:16:09.873 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:09.873 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-08-30T09:16:09.873 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.873 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.873 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.873 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.873 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.873 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.873 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.874 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:09.874 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:09.874 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.874 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-08-30T09:16:09.874 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "nfs", 2024-08-30T09:16:09.874 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-08-30T09:16:09.874 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-08-30T09:16:09.874 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-08-30T09:16:09.874 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-08-30T09:16:09.874 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-08-30T09:16:09.874 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:09.874 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.874 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:09.874 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:09.874 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.874 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.875 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:09.875 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:09.875 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:09.875 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:09.875 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:09.875 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:09.875 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:09.875 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:09.875 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.875 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.875 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.875 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.875 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.875 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-08-30T09:16:09.876 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-08-30T09:16:09.876 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:09.876 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.876 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:09.876 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:09.876 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.876 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.876 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.876 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.876 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.876 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.876 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.876 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.876 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-08-30T09:16:09.877 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-08-30T09:16:09.877 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:09.877 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.877 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:09.877 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-08-30T09:16:09.877 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.877 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.877 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:09.877 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:09.877 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:09.877 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:09.877 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:09.877 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:09.877 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:09.877 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:09.878 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.878 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.878 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.878 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.878 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.878 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-08-30T09:16:09.878 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-08-30T09:16:09.878 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:09.878 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.878 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:09.878 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:09.878 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.878 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.878 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.878 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.879 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.879 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.879 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.879 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.879 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-08-30T09:16:09.879 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-08-30T09:16:09.879 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:09.879 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-08-30T09:16:09.879 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:09.879 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-08-30T09:16:09.879 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.879 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.879 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.879 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.879 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.879 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.880 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.880 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:09.880 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:09.880 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.880 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-08-30T09:16:09.880 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "orchestrator", 2024-08-30T09:16:09.880 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-08-30T09:16:09.880 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-08-30T09:16:09.880 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-08-30T09:16:09.880 INFO:teuthology.orchestra.run.smithi042.stdout: "fail_fs": { 2024-08-30T09:16:09.880 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "fail_fs", 2024-08-30T09:16:09.880 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:09.880 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.880 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:09.881 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:09.881 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.881 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.881 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.881 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-08-30T09:16:09.881 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.881 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.881 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.881 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.881 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-08-30T09:16:09.881 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-08-30T09:16:09.881 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:09.881 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.882 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:09.882 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:09.882 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.882 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.882 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:09.882 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:09.882 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:09.882 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:09.882 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:09.882 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:09.882 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:09.882 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:09.882 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.882 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.883 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.883 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.883 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.883 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-08-30T09:16:09.883 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-08-30T09:16:09.883 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:09.883 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.883 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:09.883 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:09.883 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.883 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.883 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.883 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.883 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.883 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.884 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.884 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.884 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-08-30T09:16:09.884 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-08-30T09:16:09.884 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:09.884 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.884 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:09.884 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-08-30T09:16:09.884 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.884 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.884 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:09.884 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:09.884 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:09.884 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:09.885 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:09.885 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:09.885 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:09.885 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:09.885 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.885 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.885 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.885 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.885 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.885 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-08-30T09:16:09.885 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-08-30T09:16:09.885 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:09.885 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.885 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:09.886 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:09.886 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.886 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.886 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.886 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.886 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.886 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.886 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.886 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.886 INFO:teuthology.orchestra.run.smithi042.stdout: "orchestrator": { 2024-08-30T09:16:09.886 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "orchestrator", 2024-08-30T09:16:09.886 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:09.886 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.886 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:09.886 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:09.887 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.887 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.887 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:09.887 INFO:teuthology.orchestra.run.smithi042.stdout: "cephadm", 2024-08-30T09:16:09.887 INFO:teuthology.orchestra.run.smithi042.stdout: "rook", 2024-08-30T09:16:09.887 INFO:teuthology.orchestra.run.smithi042.stdout: "test_orchestrator" 2024-08-30T09:16:09.887 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:09.887 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Orchestrator backend", 2024-08-30T09:16:09.887 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.887 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.887 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.887 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.887 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-08-30T09:16:09.887 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-08-30T09:16:09.887 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:09.888 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-08-30T09:16:09.888 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:09.888 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-08-30T09:16:09.888 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.888 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.888 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.888 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.888 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.888 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.888 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.888 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:09.888 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:09.888 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.888 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-08-30T09:16:09.888 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "osd_perf_query", 2024-08-30T09:16:09.889 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-08-30T09:16:09.889 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-08-30T09:16:09.889 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-08-30T09:16:09.889 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-08-30T09:16:09.889 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-08-30T09:16:09.889 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:09.889 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.889 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:09.889 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:09.889 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.889 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.889 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:09.889 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:09.889 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:09.890 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:09.890 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:09.890 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:09.890 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:09.890 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:09.890 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.890 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.890 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.890 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.890 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.890 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-08-30T09:16:09.890 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-08-30T09:16:09.890 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:09.890 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.890 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:09.891 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:09.891 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.891 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.891 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.891 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.891 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.891 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.891 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.891 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.891 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-08-30T09:16:09.891 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-08-30T09:16:09.891 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:09.891 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.891 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:09.891 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-08-30T09:16:09.892 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.892 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.892 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:09.892 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:09.892 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:09.892 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:09.892 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:09.892 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:09.892 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:09.892 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:09.892 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.892 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.892 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.892 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.892 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.893 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-08-30T09:16:09.893 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-08-30T09:16:09.893 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:09.893 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.893 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:09.893 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:09.893 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.893 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.893 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.893 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.893 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.893 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.893 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.893 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.894 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-08-30T09:16:09.894 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-08-30T09:16:09.894 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:09.894 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-08-30T09:16:09.894 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:09.894 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-08-30T09:16:09.894 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.894 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.894 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.894 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.894 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.894 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.894 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.894 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:09.894 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:09.895 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.895 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-08-30T09:16:09.895 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "osd_support", 2024-08-30T09:16:09.895 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-08-30T09:16:09.895 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-08-30T09:16:09.895 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-08-30T09:16:09.895 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-08-30T09:16:09.895 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-08-30T09:16:09.895 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:09.895 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.895 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:09.895 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:09.895 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.895 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.896 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:09.896 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:09.896 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:09.896 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:09.896 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:09.896 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:09.896 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:09.896 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:09.896 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.896 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.896 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.896 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.896 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.896 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-08-30T09:16:09.896 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-08-30T09:16:09.897 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:09.897 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.897 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:09.897 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:09.897 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.897 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.897 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.897 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.897 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.897 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.897 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.897 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.897 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-08-30T09:16:09.897 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-08-30T09:16:09.898 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:09.898 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.898 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:09.898 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-08-30T09:16:09.898 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.898 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.898 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:09.898 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:09.898 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:09.898 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:09.898 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:09.898 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:09.898 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:09.898 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:09.898 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.898 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.899 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.899 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.899 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.899 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-08-30T09:16:09.899 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-08-30T09:16:09.899 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:09.899 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.899 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:09.899 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:09.899 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.899 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.899 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.899 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.899 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.900 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.900 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.900 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.900 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-08-30T09:16:09.900 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-08-30T09:16:09.900 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:09.900 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-08-30T09:16:09.900 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:09.900 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-08-30T09:16:09.900 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.900 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.900 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.900 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.900 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.900 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.901 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.901 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:09.901 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:09.901 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.901 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-08-30T09:16:09.901 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "pg_autoscaler", 2024-08-30T09:16:09.901 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-08-30T09:16:09.901 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-08-30T09:16:09.901 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-08-30T09:16:09.901 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-08-30T09:16:09.901 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-08-30T09:16:09.901 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:09.901 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.901 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:09.902 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:09.902 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.902 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.902 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:09.902 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:09.902 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:09.902 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:09.902 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:09.902 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:09.902 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:09.902 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:09.902 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.902 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.902 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.902 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.903 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.903 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-08-30T09:16:09.903 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-08-30T09:16:09.903 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:09.903 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.903 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:09.903 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:09.903 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.903 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.903 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.903 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.903 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.903 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.903 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.903 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.903 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-08-30T09:16:09.904 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-08-30T09:16:09.904 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:09.904 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.904 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:09.904 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-08-30T09:16:09.904 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.904 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.904 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:09.904 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:09.904 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:09.904 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:09.904 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:09.904 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:09.904 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:09.905 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:09.905 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.905 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.905 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.905 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.905 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.905 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-08-30T09:16:09.905 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-08-30T09:16:09.905 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:09.905 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.905 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:09.905 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:09.905 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.905 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.905 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.906 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.906 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.906 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.906 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.906 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.906 INFO:teuthology.orchestra.run.smithi042.stdout: "sleep_interval": { 2024-08-30T09:16:09.906 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sleep_interval", 2024-08-30T09:16:09.906 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-08-30T09:16:09.906 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.906 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:09.906 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "60", 2024-08-30T09:16:09.906 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.906 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.906 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.906 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.907 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.907 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.907 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.907 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.907 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-08-30T09:16:09.907 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-08-30T09:16:09.907 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:09.907 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-08-30T09:16:09.907 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:09.907 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-08-30T09:16:09.907 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.907 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.907 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.907 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.907 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.908 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.908 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.908 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.908 INFO:teuthology.orchestra.run.smithi042.stdout: "threshold": { 2024-08-30T09:16:09.908 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "threshold", 2024-08-30T09:16:09.908 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "float", 2024-08-30T09:16:09.908 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.908 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:09.908 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "3.0", 2024-08-30T09:16:09.908 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "1.0", 2024-08-30T09:16:09.908 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.908 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.908 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "scaling threshold", 2024-08-30T09:16:09.908 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-08-30T09:16:09.909 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.909 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.909 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:09.909 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:09.909 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.909 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-08-30T09:16:09.909 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "progress", 2024-08-30T09:16:09.909 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-08-30T09:16:09.909 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-08-30T09:16:09.909 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-08-30T09:16:09.909 INFO:teuthology.orchestra.run.smithi042.stdout: "allow_pg_recovery_event": { 2024-08-30T09:16:09.909 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "allow_pg_recovery_event", 2024-08-30T09:16:09.909 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:09.909 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.910 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:09.910 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:09.910 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.910 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.910 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.910 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "allow the module to show pg recovery progress", 2024-08-30T09:16:09.910 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.910 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.910 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.910 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.911 INFO:teuthology.orchestra.run.smithi042.stdout: "enabled": { 2024-08-30T09:16:09.911 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "enabled", 2024-08-30T09:16:09.911 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:09.911 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.911 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:09.911 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-08-30T09:16:09.911 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.911 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.911 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.911 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.911 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.911 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.911 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.911 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.912 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-08-30T09:16:09.912 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-08-30T09:16:09.912 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:09.912 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.912 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:09.912 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:09.912 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.912 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.912 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:09.912 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:09.912 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:09.913 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:09.913 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:09.913 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:09.913 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:09.913 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:09.913 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.913 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.913 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.913 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.913 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.913 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-08-30T09:16:09.913 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-08-30T09:16:09.913 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:09.913 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.914 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:09.914 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:09.914 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.914 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.914 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.914 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.914 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.914 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.914 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.914 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.914 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-08-30T09:16:09.914 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-08-30T09:16:09.914 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:09.914 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.914 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:09.914 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-08-30T09:16:09.915 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.915 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.915 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:09.915 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:09.915 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:09.915 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:09.915 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:09.915 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:09.915 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:09.915 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:09.915 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.915 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.915 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.915 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.915 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.916 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-08-30T09:16:09.916 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-08-30T09:16:09.916 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:09.916 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.916 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:09.916 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:09.916 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.916 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.916 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.916 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.916 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.916 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.916 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.916 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.917 INFO:teuthology.orchestra.run.smithi042.stdout: "max_completed_events": { 2024-08-30T09:16:09.917 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "max_completed_events", 2024-08-30T09:16:09.917 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:09.917 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.917 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:09.917 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "50", 2024-08-30T09:16:09.917 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.917 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.917 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.917 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "number of past completed events to remember", 2024-08-30T09:16:09.917 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.917 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.917 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.917 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.917 INFO:teuthology.orchestra.run.smithi042.stdout: "sleep_interval": { 2024-08-30T09:16:09.918 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sleep_interval", 2024-08-30T09:16:09.918 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-08-30T09:16:09.918 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.918 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:09.918 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "5", 2024-08-30T09:16:09.918 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.918 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.918 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.918 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "how long the module is going to sleep", 2024-08-30T09:16:09.918 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.918 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.918 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.918 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.918 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-08-30T09:16:09.919 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-08-30T09:16:09.919 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:09.919 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-08-30T09:16:09.919 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:09.919 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-08-30T09:16:09.919 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.919 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.919 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.919 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.919 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.919 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.919 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.919 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:09.920 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:09.920 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.920 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-08-30T09:16:09.920 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "prometheus", 2024-08-30T09:16:09.920 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-08-30T09:16:09.920 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-08-30T09:16:09.920 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-08-30T09:16:09.920 INFO:teuthology.orchestra.run.smithi042.stdout: "cache": { 2024-08-30T09:16:09.920 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "cache", 2024-08-30T09:16:09.920 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:09.920 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.920 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:09.920 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-08-30T09:16:09.920 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.921 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.921 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.921 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.921 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.921 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.921 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.921 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.921 INFO:teuthology.orchestra.run.smithi042.stdout: "exclude_perf_counters": { 2024-08-30T09:16:09.921 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "exclude_perf_counters", 2024-08-30T09:16:09.921 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:09.921 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.922 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:09.922 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-08-30T09:16:09.922 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.922 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.922 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.922 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-08-30T09:16:09.922 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-08-30T09:16:09.922 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.922 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.922 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.922 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-08-30T09:16:09.922 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-08-30T09:16:09.922 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:09.922 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.923 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:09.923 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:09.923 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.923 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.923 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:09.923 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:09.923 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:09.923 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:09.923 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:09.923 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:09.923 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:09.923 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:09.923 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.924 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.924 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.924 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.924 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.924 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-08-30T09:16:09.924 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-08-30T09:16:09.924 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:09.924 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.924 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:09.924 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:09.924 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.924 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.924 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.924 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.925 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.925 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.925 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.925 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.925 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-08-30T09:16:09.925 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-08-30T09:16:09.925 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:09.925 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.925 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:09.925 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-08-30T09:16:09.925 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.925 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.925 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:09.925 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:09.926 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:09.926 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:09.926 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:09.926 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:09.926 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:09.926 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:09.926 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.926 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.926 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.926 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.926 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.926 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-08-30T09:16:09.926 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-08-30T09:16:09.926 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:09.926 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.927 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:09.927 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:09.927 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.927 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.927 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.927 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.927 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.927 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.927 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.927 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.927 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_stats_pools": { 2024-08-30T09:16:09.927 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rbd_stats_pools", 2024-08-30T09:16:09.927 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:09.927 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.927 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:09.928 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:09.928 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.928 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.928 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.928 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.928 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.928 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.928 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.928 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.928 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_stats_pools_refresh_interval": { 2024-08-30T09:16:09.928 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-08-30T09:16:09.928 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:09.928 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.928 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:09.929 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "300", 2024-08-30T09:16:09.929 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.929 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.929 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.929 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.929 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.929 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.929 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.929 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.929 INFO:teuthology.orchestra.run.smithi042.stdout: "scrape_interval": { 2024-08-30T09:16:09.929 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "scrape_interval", 2024-08-30T09:16:09.929 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "float", 2024-08-30T09:16:09.929 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.929 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:09.930 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "15.0", 2024-08-30T09:16:09.930 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.930 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.930 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.930 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.930 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.930 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.930 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.930 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.930 INFO:teuthology.orchestra.run.smithi042.stdout: "server_addr": { 2024-08-30T09:16:09.930 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "server_addr", 2024-08-30T09:16:09.930 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:09.930 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.930 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:09.930 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "::", 2024-08-30T09:16:09.931 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.931 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.931 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.931 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-08-30T09:16:09.931 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.931 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.931 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.931 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.931 INFO:teuthology.orchestra.run.smithi042.stdout: "server_port": { 2024-08-30T09:16:09.931 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "server_port", 2024-08-30T09:16:09.931 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:09.931 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.931 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:09.931 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "9283", 2024-08-30T09:16:09.931 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.931 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.932 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.932 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-08-30T09:16:09.932 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.932 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.932 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.932 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.932 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-08-30T09:16:09.932 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-08-30T09:16:09.932 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:09.932 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-08-30T09:16:09.932 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:09.932 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-08-30T09:16:09.932 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.933 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.933 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.933 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.933 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.933 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.933 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.933 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.933 INFO:teuthology.orchestra.run.smithi042.stdout: "stale_cache_strategy": { 2024-08-30T09:16:09.933 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "stale_cache_strategy", 2024-08-30T09:16:09.933 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:09.933 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.933 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:09.933 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "log", 2024-08-30T09:16:09.933 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.933 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.934 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.934 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.934 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.934 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.934 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.934 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.934 INFO:teuthology.orchestra.run.smithi042.stdout: "standby_behaviour": { 2024-08-30T09:16:09.934 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "standby_behaviour", 2024-08-30T09:16:09.934 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:09.934 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.934 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:09.934 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "default", 2024-08-30T09:16:09.934 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.935 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.935 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:09.935 INFO:teuthology.orchestra.run.smithi042.stdout: "default", 2024-08-30T09:16:09.935 INFO:teuthology.orchestra.run.smithi042.stdout: "error" 2024-08-30T09:16:09.935 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:09.935 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.935 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.935 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.935 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.935 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.935 INFO:teuthology.orchestra.run.smithi042.stdout: "standby_error_status_code": { 2024-08-30T09:16:09.935 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "standby_error_status_code", 2024-08-30T09:16:09.935 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:09.935 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.935 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:09.936 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "500", 2024-08-30T09:16:09.936 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "400", 2024-08-30T09:16:09.936 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "599", 2024-08-30T09:16:09.936 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.936 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.936 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.936 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.936 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.936 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:09.936 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:09.936 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.936 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-08-30T09:16:09.936 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rbd_support", 2024-08-30T09:16:09.937 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-08-30T09:16:09.937 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-08-30T09:16:09.937 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-08-30T09:16:09.937 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-08-30T09:16:09.937 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-08-30T09:16:09.937 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:09.937 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.937 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:09.937 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:09.937 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.937 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.937 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:09.937 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:09.937 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:09.938 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:09.938 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:09.938 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:09.938 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:09.938 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:09.938 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.938 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.938 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.938 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.938 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.938 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-08-30T09:16:09.938 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-08-30T09:16:09.938 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:09.938 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.939 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:09.939 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:09.939 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.939 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.939 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.939 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.939 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.939 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.939 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.939 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.939 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-08-30T09:16:09.939 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-08-30T09:16:09.939 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:09.939 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.939 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:09.940 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-08-30T09:16:09.940 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.940 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.940 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:09.940 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:09.940 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:09.940 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:09.940 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:09.940 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:09.940 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:09.940 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:09.940 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.940 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.940 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.940 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.941 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.941 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-08-30T09:16:09.941 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-08-30T09:16:09.941 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:09.941 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.941 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:09.941 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:09.941 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.941 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.941 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.941 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.941 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.941 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.941 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.941 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.942 INFO:teuthology.orchestra.run.smithi042.stdout: "max_concurrent_snap_create": { 2024-08-30T09:16:09.942 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "max_concurrent_snap_create", 2024-08-30T09:16:09.942 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:09.942 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.942 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:09.942 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "10", 2024-08-30T09:16:09.942 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.942 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.942 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.942 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.942 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.942 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.942 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.942 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.943 INFO:teuthology.orchestra.run.smithi042.stdout: "mirror_snapshot_schedule": { 2024-08-30T09:16:09.943 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "mirror_snapshot_schedule", 2024-08-30T09:16:09.943 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:09.943 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.943 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:09.943 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:09.943 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.943 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.943 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.943 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.943 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.943 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.943 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.943 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.943 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-08-30T09:16:09.943 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-08-30T09:16:09.944 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:09.944 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-08-30T09:16:09.944 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:09.944 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-08-30T09:16:09.944 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.944 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.944 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.944 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.944 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.944 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.944 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.944 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.944 INFO:teuthology.orchestra.run.smithi042.stdout: "trash_purge_schedule": { 2024-08-30T09:16:09.944 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "trash_purge_schedule", 2024-08-30T09:16:09.944 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:09.945 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.945 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:09.945 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:09.945 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.945 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.945 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.945 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.945 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.945 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.945 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.945 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:09.945 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:09.945 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.945 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-08-30T09:16:09.946 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "restful", 2024-08-30T09:16:09.946 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-08-30T09:16:09.946 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-08-30T09:16:09.946 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-08-30T09:16:09.946 INFO:teuthology.orchestra.run.smithi042.stdout: "enable_auth": { 2024-08-30T09:16:09.946 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "enable_auth", 2024-08-30T09:16:09.946 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:09.946 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.946 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:09.946 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-08-30T09:16:09.946 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.946 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.946 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.946 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.946 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.947 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.947 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.947 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.947 INFO:teuthology.orchestra.run.smithi042.stdout: "key_file": { 2024-08-30T09:16:09.947 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "key_file", 2024-08-30T09:16:09.947 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:09.947 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.947 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:09.947 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:09.947 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.947 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.947 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.947 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.948 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.948 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.948 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.948 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.948 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-08-30T09:16:09.948 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-08-30T09:16:09.948 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:09.948 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.948 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:09.948 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:09.948 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.948 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.948 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:09.949 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:09.949 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:09.949 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:09.949 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:09.949 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:09.949 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:09.949 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:09.949 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.949 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.949 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.949 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.949 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.949 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-08-30T09:16:09.949 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-08-30T09:16:09.949 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:09.950 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.950 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:09.950 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:09.950 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.950 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.950 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.950 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.950 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.950 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.950 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.950 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.950 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-08-30T09:16:09.950 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-08-30T09:16:09.951 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:09.951 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.951 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:09.951 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-08-30T09:16:09.951 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.951 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.951 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:09.951 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:09.951 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:09.951 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:09.951 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:09.951 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:09.951 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:09.951 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:09.952 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.952 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.952 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.952 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.952 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.952 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-08-30T09:16:09.952 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-08-30T09:16:09.952 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:09.952 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.952 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:09.952 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:09.952 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.953 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.953 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.953 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.953 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.953 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.953 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.953 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.953 INFO:teuthology.orchestra.run.smithi042.stdout: "max_requests": { 2024-08-30T09:16:09.953 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "max_requests", 2024-08-30T09:16:09.953 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:09.953 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.953 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:09.953 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "500", 2024-08-30T09:16:09.954 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.954 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.954 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.954 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.", 2024-08-30T09:16:09.954 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.954 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.954 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.954 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.954 INFO:teuthology.orchestra.run.smithi042.stdout: "server_addr": { 2024-08-30T09:16:09.954 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "server_addr", 2024-08-30T09:16:09.954 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:09.954 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.954 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:09.955 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:09.955 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.955 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.955 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.955 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.955 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.955 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.955 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.955 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.955 INFO:teuthology.orchestra.run.smithi042.stdout: "server_port": { 2024-08-30T09:16:09.955 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "server_port", 2024-08-30T09:16:09.955 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:09.955 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.955 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:09.955 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:09.956 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.956 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.956 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.956 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.956 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.956 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.956 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.956 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.956 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-08-30T09:16:09.956 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-08-30T09:16:09.956 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:09.956 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-08-30T09:16:09.956 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:09.956 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-08-30T09:16:09.957 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.957 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.957 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.957 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.957 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.957 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.957 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.957 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:09.957 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:09.957 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.957 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-08-30T09:16:09.957 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rgw", 2024-08-30T09:16:09.957 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-08-30T09:16:09.957 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-08-30T09:16:09.958 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-08-30T09:16:09.958 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-08-30T09:16:09.958 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-08-30T09:16:09.958 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:09.958 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.958 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:09.958 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:09.958 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.958 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.958 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:09.958 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:09.958 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:09.958 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:09.958 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:09.958 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:09.959 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:09.959 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:09.959 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.959 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.959 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.959 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.959 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.959 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-08-30T09:16:09.959 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-08-30T09:16:09.959 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:09.959 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.959 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:09.959 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:09.959 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.959 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.960 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.960 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.960 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.960 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.960 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.960 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.960 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-08-30T09:16:09.960 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-08-30T09:16:09.960 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:09.960 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.960 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:09.960 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-08-30T09:16:09.960 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.960 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.960 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:09.961 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:09.961 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:09.961 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:09.961 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:09.961 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:09.961 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:09.961 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:09.961 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.961 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.961 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.961 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.961 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.961 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-08-30T09:16:09.962 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-08-30T09:16:09.962 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:09.962 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.962 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:09.962 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:09.962 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.962 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.962 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.962 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.962 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.962 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.962 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.962 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.962 INFO:teuthology.orchestra.run.smithi042.stdout: "secondary_zone_period_retry_limit": { 2024-08-30T09:16:09.962 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "secondary_zone_period_retry_limit", 2024-08-30T09:16:09.963 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:09.963 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.963 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:09.963 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "5", 2024-08-30T09:16:09.963 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.963 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.963 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.963 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-08-30T09:16:09.963 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.963 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.963 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.963 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.963 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-08-30T09:16:09.963 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-08-30T09:16:09.963 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:09.964 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-08-30T09:16:09.964 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:09.964 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-08-30T09:16:09.964 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.964 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.964 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.964 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.964 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.964 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.964 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.964 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:09.964 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:09.964 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.965 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-08-30T09:16:09.965 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rook", 2024-08-30T09:16:09.965 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-08-30T09:16:09.965 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-08-30T09:16:09.965 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-08-30T09:16:09.965 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-08-30T09:16:09.965 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-08-30T09:16:09.965 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:09.965 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.965 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:09.965 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:09.965 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.965 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.965 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:09.965 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:09.966 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:09.966 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:09.966 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:09.966 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:09.966 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:09.966 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:09.966 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.966 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.966 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.966 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.966 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.966 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-08-30T09:16:09.966 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-08-30T09:16:09.967 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:09.967 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.967 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:09.967 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:09.967 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.967 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.967 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.967 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.967 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.967 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.967 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.967 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.967 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-08-30T09:16:09.967 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-08-30T09:16:09.967 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:09.968 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.968 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:09.968 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-08-30T09:16:09.968 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.968 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.968 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:09.968 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:09.968 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:09.968 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:09.968 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:09.968 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:09.968 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:09.968 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:09.968 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.968 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.969 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.969 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.969 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.969 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-08-30T09:16:09.969 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-08-30T09:16:09.969 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:09.969 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.969 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:09.969 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:09.969 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.969 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.969 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.969 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.969 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.970 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.970 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.970 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.970 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-08-30T09:16:09.970 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-08-30T09:16:09.970 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:09.970 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-08-30T09:16:09.970 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:09.970 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-08-30T09:16:09.970 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.970 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.970 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.970 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.970 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.970 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.971 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.971 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.971 INFO:teuthology.orchestra.run.smithi042.stdout: "storage_class": { 2024-08-30T09:16:09.971 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "storage_class", 2024-08-30T09:16:09.971 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:09.971 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.971 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:09.971 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "local", 2024-08-30T09:16:09.971 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.971 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.971 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.971 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-08-30T09:16:09.971 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.971 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.972 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.972 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:09.972 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:09.972 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.972 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-08-30T09:16:09.972 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "selftest", 2024-08-30T09:16:09.972 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-08-30T09:16:09.972 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-08-30T09:16:09.972 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-08-30T09:16:09.972 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-08-30T09:16:09.972 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-08-30T09:16:09.972 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:09.972 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.973 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:09.973 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:09.973 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.973 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.973 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:09.973 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:09.973 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:09.973 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:09.973 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:09.973 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:09.973 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:09.973 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:09.973 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.973 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.974 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.974 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.974 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.974 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-08-30T09:16:09.974 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-08-30T09:16:09.974 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:09.974 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.974 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:09.974 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:09.974 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.974 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.974 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.974 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.974 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.974 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.975 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.975 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.975 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-08-30T09:16:09.975 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-08-30T09:16:09.975 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:09.975 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.975 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:09.975 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-08-30T09:16:09.975 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.975 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.975 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:09.975 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:09.975 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:09.976 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:09.976 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:09.976 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:09.976 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:09.976 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:09.976 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.976 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.976 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.976 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.976 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.976 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-08-30T09:16:09.976 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-08-30T09:16:09.977 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:09.977 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.977 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:09.977 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:09.977 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.977 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.977 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.977 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.977 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.977 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.978 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.978 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.978 INFO:teuthology.orchestra.run.smithi042.stdout: "roption1": { 2024-08-30T09:16:09.978 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "roption1", 2024-08-30T09:16:09.978 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:09.978 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.978 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:09.978 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:09.978 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.978 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.978 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.978 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.978 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.979 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.979 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.979 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.979 INFO:teuthology.orchestra.run.smithi042.stdout: "roption2": { 2024-08-30T09:16:09.979 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "roption2", 2024-08-30T09:16:09.979 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:09.979 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.979 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:09.979 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "xyz", 2024-08-30T09:16:09.979 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.979 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.980 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.980 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.980 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.980 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.980 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.980 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.980 INFO:teuthology.orchestra.run.smithi042.stdout: "rwoption1": { 2024-08-30T09:16:09.980 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rwoption1", 2024-08-30T09:16:09.980 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:09.980 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.980 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:09.980 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:09.980 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.980 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.981 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.981 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.981 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.981 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.981 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.981 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.981 INFO:teuthology.orchestra.run.smithi042.stdout: "rwoption2": { 2024-08-30T09:16:09.981 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rwoption2", 2024-08-30T09:16:09.981 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:09.981 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.981 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:09.981 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:09.981 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.982 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.982 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.982 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.982 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.982 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.982 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.982 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.982 INFO:teuthology.orchestra.run.smithi042.stdout: "rwoption3": { 2024-08-30T09:16:09.982 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rwoption3", 2024-08-30T09:16:09.982 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "float", 2024-08-30T09:16:09.982 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.982 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:09.982 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:09.983 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.983 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.983 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.983 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.983 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.983 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.983 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.983 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.983 INFO:teuthology.orchestra.run.smithi042.stdout: "rwoption4": { 2024-08-30T09:16:09.983 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rwoption4", 2024-08-30T09:16:09.983 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:09.983 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.983 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:09.983 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:09.984 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.984 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.984 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.984 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.984 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.984 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.984 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.984 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.984 INFO:teuthology.orchestra.run.smithi042.stdout: "rwoption5": { 2024-08-30T09:16:09.984 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rwoption5", 2024-08-30T09:16:09.984 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:09.984 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.984 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:09.985 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:09.985 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.985 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.985 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.985 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.985 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.985 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.985 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.985 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.985 INFO:teuthology.orchestra.run.smithi042.stdout: "rwoption6": { 2024-08-30T09:16:09.985 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rwoption6", 2024-08-30T09:16:09.985 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:09.985 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.986 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:09.986 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-08-30T09:16:09.986 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.986 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.986 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.986 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.986 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.986 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.986 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.986 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.986 INFO:teuthology.orchestra.run.smithi042.stdout: "rwoption7": { 2024-08-30T09:16:09.987 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rwoption7", 2024-08-30T09:16:09.987 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:09.987 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.987 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:09.987 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:09.987 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "1", 2024-08-30T09:16:09.987 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "42", 2024-08-30T09:16:09.987 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.987 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.987 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.987 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.988 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.988 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.988 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-08-30T09:16:09.988 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-08-30T09:16:09.988 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:09.988 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-08-30T09:16:09.988 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:09.988 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-08-30T09:16:09.988 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.988 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.988 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.988 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.988 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.989 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.989 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.989 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.989 INFO:teuthology.orchestra.run.smithi042.stdout: "testkey": { 2024-08-30T09:16:09.989 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "testkey", 2024-08-30T09:16:09.989 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:09.989 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.989 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:09.989 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:09.989 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.989 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.989 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.989 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.990 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.990 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.990 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.990 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.990 INFO:teuthology.orchestra.run.smithi042.stdout: "testlkey": { 2024-08-30T09:16:09.990 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "testlkey", 2024-08-30T09:16:09.990 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:09.990 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.990 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:09.990 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:09.990 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.990 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.990 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.991 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.991 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.991 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.991 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.991 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.991 INFO:teuthology.orchestra.run.smithi042.stdout: "testnewline": { 2024-08-30T09:16:09.991 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "testnewline", 2024-08-30T09:16:09.991 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:09.991 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.991 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:09.991 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:09.991 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.992 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.992 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.992 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.992 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.992 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.992 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.992 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:09.992 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:09.992 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.992 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-08-30T09:16:09.992 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "smb", 2024-08-30T09:16:09.992 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-08-30T09:16:09.992 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-08-30T09:16:09.993 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-08-30T09:16:09.993 INFO:teuthology.orchestra.run.smithi042.stdout: "internal_store_backend": { 2024-08-30T09:16:09.993 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "internal_store_backend", 2024-08-30T09:16:09.993 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:09.993 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-08-30T09:16:09.993 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:09.993 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:09.993 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.993 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.993 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.993 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "set internal store backend. for develoment and testing only", 2024-08-30T09:16:09.993 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.993 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.994 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.994 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.994 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-08-30T09:16:09.994 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-08-30T09:16:09.994 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:09.994 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.994 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:09.994 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:09.994 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.994 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.994 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:09.994 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:09.994 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:09.994 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:09.995 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:09.995 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:09.995 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:09.995 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:09.995 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.995 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.995 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.995 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.995 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.995 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-08-30T09:16:09.995 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-08-30T09:16:09.995 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:09.995 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.995 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:09.995 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:09.996 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.996 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.996 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.996 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.996 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.996 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.996 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.996 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.996 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-08-30T09:16:09.996 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-08-30T09:16:09.996 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:09.996 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.996 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:09.997 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-08-30T09:16:09.997 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.997 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.997 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:09.997 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:09.997 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:09.997 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:09.997 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:09.997 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:09.997 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:09.997 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:09.997 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.997 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.997 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.998 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.998 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.998 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-08-30T09:16:09.998 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-08-30T09:16:09.998 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:09.998 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:09.998 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:09.998 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:09.998 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.998 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.998 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.998 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.998 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.999 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:09.999 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:09.999 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:09.999 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-08-30T09:16:09.999 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-08-30T09:16:09.999 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:09.999 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-08-30T09:16:09.999 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:09.999 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-08-30T09:16:09.999 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:09.999 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:09.999 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:09.999 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:09.999 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:09.999 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.000 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.000 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.000 INFO:teuthology.orchestra.run.smithi042.stdout: "update_orchestration": { 2024-08-30T09:16:10.000 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "update_orchestration", 2024-08-30T09:16:10.000 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.000 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.000 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.000 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-08-30T09:16:10.000 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.000 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.000 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.000 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-08-30T09:16:10.000 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.000 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.001 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.001 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:10.001 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:10.001 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.001 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-08-30T09:16:10.001 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "snap_schedule", 2024-08-30T09:16:10.001 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-08-30T09:16:10.001 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-08-30T09:16:10.001 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-08-30T09:16:10.001 INFO:teuthology.orchestra.run.smithi042.stdout: "allow_m_granularity": { 2024-08-30T09:16:10.001 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "allow_m_granularity", 2024-08-30T09:16:10.001 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.002 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.002 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.002 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:10.002 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.002 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.002 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.002 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "allow minute scheduled snapshots", 2024-08-30T09:16:10.002 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.002 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.002 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.002 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.002 INFO:teuthology.orchestra.run.smithi042.stdout: "dump_on_update": { 2024-08-30T09:16:10.002 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "dump_on_update", 2024-08-30T09:16:10.003 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.003 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.003 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.003 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:10.003 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.003 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.003 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.003 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "dump database to debug log on update", 2024-08-30T09:16:10.003 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.003 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.003 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.003 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.003 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-08-30T09:16:10.003 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-08-30T09:16:10.003 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.004 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.004 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.004 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:10.004 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.004 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.004 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:10.004 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:10.004 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:10.004 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:10.004 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:10.004 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:10.004 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:10.004 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:10.004 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.005 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.005 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.005 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.005 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.005 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-08-30T09:16:10.005 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-08-30T09:16:10.005 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.005 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.005 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.005 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:10.005 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.005 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.005 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.006 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.006 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.006 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.006 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.006 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.006 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-08-30T09:16:10.006 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-08-30T09:16:10.006 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.006 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.006 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.006 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-08-30T09:16:10.006 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.006 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.006 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:10.007 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:10.007 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:10.007 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:10.007 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:10.007 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:10.007 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:10.007 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:10.007 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.007 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.007 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.007 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.007 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.008 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-08-30T09:16:10.008 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-08-30T09:16:10.008 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.008 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.008 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.008 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:10.008 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.008 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.008 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.008 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.008 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.008 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.008 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.008 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.009 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-08-30T09:16:10.009 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-08-30T09:16:10.009 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:10.009 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-08-30T09:16:10.009 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.009 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-08-30T09:16:10.009 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.009 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.009 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.009 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.009 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.009 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.009 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.010 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:10.010 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:10.010 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.010 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-08-30T09:16:10.010 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "stats", 2024-08-30T09:16:10.010 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-08-30T09:16:10.010 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-08-30T09:16:10.010 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-08-30T09:16:10.010 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-08-30T09:16:10.010 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-08-30T09:16:10.010 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.010 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.010 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.011 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:10.011 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.011 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.011 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:10.011 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:10.011 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:10.011 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:10.011 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:10.011 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:10.011 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:10.011 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:10.011 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.011 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.011 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.011 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.012 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.012 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-08-30T09:16:10.012 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-08-30T09:16:10.012 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.012 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.012 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.012 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:10.012 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.012 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.012 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.012 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.012 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.012 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.013 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.013 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.013 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-08-30T09:16:10.013 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-08-30T09:16:10.013 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.013 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.013 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.013 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-08-30T09:16:10.013 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.013 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.013 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:10.013 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:10.014 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:10.014 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:10.014 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:10.014 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:10.014 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:10.014 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:10.014 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.014 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.014 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.014 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.014 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.014 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-08-30T09:16:10.014 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-08-30T09:16:10.015 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.015 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.015 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.015 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:10.015 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.015 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.015 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.015 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.015 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.015 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.015 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.015 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.015 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-08-30T09:16:10.015 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-08-30T09:16:10.016 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:10.016 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-08-30T09:16:10.016 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.016 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-08-30T09:16:10.016 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.016 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.016 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.016 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.019 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.019 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.020 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.020 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:10.020 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:10.020 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.020 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-08-30T09:16:10.020 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "status", 2024-08-30T09:16:10.020 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-08-30T09:16:10.021 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-08-30T09:16:10.021 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-08-30T09:16:10.021 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-08-30T09:16:10.021 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-08-30T09:16:10.021 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.021 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.022 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.022 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:10.022 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.022 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.022 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:10.022 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:10.022 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:10.022 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:10.023 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:10.023 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:10.023 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:10.023 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:10.023 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.023 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.023 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.024 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.024 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.024 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-08-30T09:16:10.024 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-08-30T09:16:10.024 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.024 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.024 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.025 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:10.025 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.025 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.025 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.025 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.025 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.025 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.025 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.025 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.025 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-08-30T09:16:10.025 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-08-30T09:16:10.025 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.026 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.026 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.026 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-08-30T09:16:10.026 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.026 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.026 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:10.026 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:10.026 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:10.026 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:10.026 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:10.026 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:10.026 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:10.026 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:10.026 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.027 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.027 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.027 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.027 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.027 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-08-30T09:16:10.027 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-08-30T09:16:10.027 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.027 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.027 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.027 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:10.027 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.027 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.027 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.027 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.028 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.028 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.028 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.028 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.028 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-08-30T09:16:10.028 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-08-30T09:16:10.028 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:10.028 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-08-30T09:16:10.028 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.028 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-08-30T09:16:10.028 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.029 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.029 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.029 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.029 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.029 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.029 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.029 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:10.029 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:10.029 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.029 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-08-30T09:16:10.029 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "telegraf", 2024-08-30T09:16:10.029 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-08-30T09:16:10.029 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-08-30T09:16:10.030 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-08-30T09:16:10.030 INFO:teuthology.orchestra.run.smithi042.stdout: "address": { 2024-08-30T09:16:10.030 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "address", 2024-08-30T09:16:10.030 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.030 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.030 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.030 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-08-30T09:16:10.030 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.030 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.030 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.031 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.031 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.031 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.031 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.031 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.031 INFO:teuthology.orchestra.run.smithi042.stdout: "interval": { 2024-08-30T09:16:10.031 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "interval", 2024-08-30T09:16:10.031 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-08-30T09:16:10.031 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.031 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.031 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "15", 2024-08-30T09:16:10.032 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.032 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.032 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.032 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.032 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.032 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.032 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.032 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.032 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-08-30T09:16:10.032 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-08-30T09:16:10.032 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.032 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.032 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.032 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:10.033 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.033 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.033 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:10.033 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:10.033 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:10.033 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:10.033 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:10.033 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:10.033 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:10.033 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:10.033 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.033 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.033 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.033 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.034 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.034 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-08-30T09:16:10.034 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-08-30T09:16:10.034 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.034 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.034 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.034 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:10.034 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.034 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.034 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.034 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.034 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.034 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.035 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.035 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.035 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-08-30T09:16:10.035 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-08-30T09:16:10.035 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.035 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.035 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.035 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-08-30T09:16:10.035 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.035 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.035 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:10.035 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:10.035 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:10.035 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:10.035 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:10.036 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:10.036 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:10.036 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:10.036 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.036 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.036 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.036 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.036 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.036 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-08-30T09:16:10.036 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-08-30T09:16:10.036 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.036 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.037 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.037 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:10.037 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.037 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.037 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.037 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.037 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.037 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.037 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.037 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.037 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-08-30T09:16:10.037 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-08-30T09:16:10.037 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:10.038 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-08-30T09:16:10.038 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.038 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-08-30T09:16:10.038 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.038 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.038 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.038 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.038 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.038 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.038 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.038 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:10.038 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:10.038 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.038 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-08-30T09:16:10.039 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "telemetry", 2024-08-30T09:16:10.039 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-08-30T09:16:10.039 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-08-30T09:16:10.039 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-08-30T09:16:10.039 INFO:teuthology.orchestra.run.smithi042.stdout: "channel_basic": { 2024-08-30T09:16:10.039 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "channel_basic", 2024-08-30T09:16:10.039 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.039 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.039 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.039 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-08-30T09:16:10.039 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.039 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.039 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.039 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Share basic cluster information (size, version)", 2024-08-30T09:16:10.040 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.040 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.040 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.040 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.040 INFO:teuthology.orchestra.run.smithi042.stdout: "channel_crash": { 2024-08-30T09:16:10.040 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "channel_crash", 2024-08-30T09:16:10.040 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.040 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.040 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.040 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-08-30T09:16:10.040 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.040 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.040 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.040 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-08-30T09:16:10.041 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.041 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.041 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.041 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.041 INFO:teuthology.orchestra.run.smithi042.stdout: "channel_device": { 2024-08-30T09:16:10.041 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "channel_device", 2024-08-30T09:16:10.041 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.041 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.041 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.041 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-08-30T09:16:10.041 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.041 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.041 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.042 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-08-30T09:16:10.042 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.042 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.042 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.042 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.042 INFO:teuthology.orchestra.run.smithi042.stdout: "channel_ident": { 2024-08-30T09:16:10.042 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "channel_ident", 2024-08-30T09:16:10.042 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.042 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.042 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.042 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:10.042 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.043 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.043 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.043 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-08-30T09:16:10.043 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.043 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.043 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.043 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.043 INFO:teuthology.orchestra.run.smithi042.stdout: "channel_perf": { 2024-08-30T09:16:10.043 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "channel_perf", 2024-08-30T09:16:10.043 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.043 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.043 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.043 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:10.043 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.044 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.044 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.044 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Share various performance metrics of a cluster", 2024-08-30T09:16:10.044 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.044 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.044 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.044 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.044 INFO:teuthology.orchestra.run.smithi042.stdout: "contact": { 2024-08-30T09:16:10.044 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "contact", 2024-08-30T09:16:10.044 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.044 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.044 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.044 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:10.044 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.045 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.045 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.045 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.045 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.045 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.045 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.045 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.045 INFO:teuthology.orchestra.run.smithi042.stdout: "description": { 2024-08-30T09:16:10.045 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "description", 2024-08-30T09:16:10.045 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.045 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.045 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.045 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:10.045 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.045 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.046 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.046 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.046 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.046 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.046 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.046 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.046 INFO:teuthology.orchestra.run.smithi042.stdout: "device_url": { 2024-08-30T09:16:10.046 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "device_url", 2024-08-30T09:16:10.046 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.046 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.046 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.046 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-08-30T09:16:10.046 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.046 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.046 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.047 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.047 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.047 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.047 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.047 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.047 INFO:teuthology.orchestra.run.smithi042.stdout: "enabled": { 2024-08-30T09:16:10.047 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "enabled", 2024-08-30T09:16:10.047 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.047 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.047 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.047 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:10.047 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.048 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.048 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.048 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.048 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.048 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.048 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.048 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.048 INFO:teuthology.orchestra.run.smithi042.stdout: "interval": { 2024-08-30T09:16:10.048 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "interval", 2024-08-30T09:16:10.048 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:10.048 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.048 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.048 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "24", 2024-08-30T09:16:10.049 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "8", 2024-08-30T09:16:10.049 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.049 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.049 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.049 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.049 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.049 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.049 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.049 INFO:teuthology.orchestra.run.smithi042.stdout: "last_opt_revision": { 2024-08-30T09:16:10.049 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "last_opt_revision", 2024-08-30T09:16:10.049 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:10.049 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.049 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.050 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "1", 2024-08-30T09:16:10.050 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.050 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.050 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.050 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.050 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.050 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.050 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.050 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.050 INFO:teuthology.orchestra.run.smithi042.stdout: "leaderboard": { 2024-08-30T09:16:10.050 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "leaderboard", 2024-08-30T09:16:10.050 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.050 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.050 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.050 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:10.051 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.051 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.051 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.051 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.051 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.051 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.051 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.051 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.051 INFO:teuthology.orchestra.run.smithi042.stdout: "leaderboard_description": { 2024-08-30T09:16:10.051 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "leaderboard_description", 2024-08-30T09:16:10.051 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.051 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.051 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.052 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:10.052 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.052 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.052 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.052 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.052 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.052 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.052 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.052 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.052 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-08-30T09:16:10.052 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-08-30T09:16:10.052 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.052 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.052 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.053 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:10.053 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.053 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.053 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:10.053 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:10.053 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:10.053 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:10.053 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:10.053 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:10.053 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:10.053 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:10.053 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.053 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.054 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.054 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.054 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.054 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-08-30T09:16:10.054 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-08-30T09:16:10.054 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.054 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.054 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.054 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:10.054 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.054 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.054 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.054 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.054 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.055 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.055 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.055 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.055 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-08-30T09:16:10.055 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-08-30T09:16:10.055 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.055 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.055 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.055 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-08-30T09:16:10.055 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.055 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.055 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:10.055 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:10.056 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:10.056 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:10.056 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:10.056 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:10.056 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:10.056 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:10.056 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.056 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.056 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.056 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.056 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.056 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-08-30T09:16:10.056 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-08-30T09:16:10.056 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.057 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.057 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.057 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:10.057 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.057 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.057 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.057 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.057 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.057 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.057 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.057 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.057 INFO:teuthology.orchestra.run.smithi042.stdout: "organization": { 2024-08-30T09:16:10.057 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "organization", 2024-08-30T09:16:10.058 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.058 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.058 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.058 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:10.058 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.058 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.058 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.058 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.058 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.058 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.058 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.058 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.058 INFO:teuthology.orchestra.run.smithi042.stdout: "proxy": { 2024-08-30T09:16:10.059 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "proxy", 2024-08-30T09:16:10.059 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.059 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.059 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.059 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:10.059 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.059 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.059 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.059 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.059 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.059 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.059 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.059 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.059 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-08-30T09:16:10.059 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-08-30T09:16:10.060 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:10.060 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-08-30T09:16:10.060 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.060 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-08-30T09:16:10.060 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.060 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.060 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.060 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.060 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.060 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.060 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.060 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.060 INFO:teuthology.orchestra.run.smithi042.stdout: "url": { 2024-08-30T09:16:10.060 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "url", 2024-08-30T09:16:10.061 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.061 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.061 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.061 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-08-30T09:16:10.061 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.061 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.061 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.061 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.061 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.061 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.061 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.061 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:10.061 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:10.061 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.062 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-08-30T09:16:10.062 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "test_orchestrator", 2024-08-30T09:16:10.062 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-08-30T09:16:10.062 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-08-30T09:16:10.062 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-08-30T09:16:10.062 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-08-30T09:16:10.062 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-08-30T09:16:10.062 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.062 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.062 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.062 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:10.062 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.062 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.062 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:10.063 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:10.063 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:10.063 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:10.063 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:10.063 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:10.063 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:10.063 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:10.063 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.063 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.063 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.063 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.063 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.063 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-08-30T09:16:10.063 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-08-30T09:16:10.064 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.064 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.064 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.064 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:10.064 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.064 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.064 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.064 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.064 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.064 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.064 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.064 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.064 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-08-30T09:16:10.064 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-08-30T09:16:10.065 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.065 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.065 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.065 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-08-30T09:16:10.065 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.065 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.065 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:10.065 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:10.065 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:10.065 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:10.065 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:10.065 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:10.065 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:10.066 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:10.066 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.066 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.066 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.066 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.066 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.066 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-08-30T09:16:10.066 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-08-30T09:16:10.066 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.066 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.066 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.066 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:10.066 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.066 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.067 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.067 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.067 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.067 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.067 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.067 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.067 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-08-30T09:16:10.067 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-08-30T09:16:10.067 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:10.067 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-08-30T09:16:10.067 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.067 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-08-30T09:16:10.067 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.067 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.068 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.068 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.068 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.068 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.068 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.068 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:10.068 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:10.068 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.068 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-08-30T09:16:10.068 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "volumes", 2024-08-30T09:16:10.068 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-08-30T09:16:10.068 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-08-30T09:16:10.068 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-08-30T09:16:10.068 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-08-30T09:16:10.069 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-08-30T09:16:10.069 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.069 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.069 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.069 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:10.069 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.069 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.069 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:10.069 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:10.069 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:10.069 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:10.069 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:10.069 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:10.069 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:10.070 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:10.070 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.070 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.070 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.070 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.070 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.070 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-08-30T09:16:10.070 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-08-30T09:16:10.070 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.070 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.070 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.070 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:10.070 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.070 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.071 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.071 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.071 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.071 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.071 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.071 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.071 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-08-30T09:16:10.071 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-08-30T09:16:10.071 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.071 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.071 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.071 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-08-30T09:16:10.071 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.071 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.072 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:10.072 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:10.072 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:10.072 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:10.072 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:10.072 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:10.072 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:10.072 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:10.072 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.072 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.072 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.072 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.072 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.073 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-08-30T09:16:10.073 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-08-30T09:16:10.073 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.073 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.073 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.073 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:10.073 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.073 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.073 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.073 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.073 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.073 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.073 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.073 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.074 INFO:teuthology.orchestra.run.smithi042.stdout: "max_concurrent_clones": { 2024-08-30T09:16:10.074 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "max_concurrent_clones", 2024-08-30T09:16:10.074 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:10.074 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.074 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.074 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "4", 2024-08-30T09:16:10.074 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.074 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.074 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.074 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Number of asynchronous cloner threads", 2024-08-30T09:16:10.074 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.074 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.074 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.074 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.074 INFO:teuthology.orchestra.run.smithi042.stdout: "periodic_async_work": { 2024-08-30T09:16:10.075 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "periodic_async_work", 2024-08-30T09:16:10.075 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.075 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.075 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.075 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:10.075 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.075 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.075 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.075 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Periodically check for async work", 2024-08-30T09:16:10.075 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.075 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.075 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.075 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.075 INFO:teuthology.orchestra.run.smithi042.stdout: "snapshot_clone_delay": { 2024-08-30T09:16:10.076 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "snapshot_clone_delay", 2024-08-30T09:16:10.076 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:10.076 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.076 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.076 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-08-30T09:16:10.076 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.076 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.076 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.076 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-08-30T09:16:10.076 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.076 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.076 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.076 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.077 INFO:teuthology.orchestra.run.smithi042.stdout: "snapshot_clone_no_wait": { 2024-08-30T09:16:10.077 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "snapshot_clone_no_wait", 2024-08-30T09:16:10.077 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.077 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.077 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.077 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-08-30T09:16:10.077 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.077 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.077 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.077 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-08-30T09:16:10.077 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.077 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.077 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.078 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.078 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-08-30T09:16:10.078 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-08-30T09:16:10.078 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:10.078 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-08-30T09:16:10.078 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.078 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-08-30T09:16:10.078 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.078 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.078 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.078 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.078 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.078 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.078 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.078 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:10.079 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:10.079 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.079 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-08-30T09:16:10.079 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "zabbix", 2024-08-30T09:16:10.079 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-08-30T09:16:10.079 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-08-30T09:16:10.079 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-08-30T09:16:10.079 INFO:teuthology.orchestra.run.smithi042.stdout: "discovery_interval": { 2024-08-30T09:16:10.079 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "discovery_interval", 2024-08-30T09:16:10.079 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "uint", 2024-08-30T09:16:10.079 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.079 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.079 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "100", 2024-08-30T09:16:10.080 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.080 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.080 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.080 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.080 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.080 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.080 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.080 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.080 INFO:teuthology.orchestra.run.smithi042.stdout: "identifier": { 2024-08-30T09:16:10.080 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "identifier", 2024-08-30T09:16:10.080 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.080 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.080 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.080 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:10.080 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.081 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.081 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.081 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.081 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.081 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.081 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.081 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.081 INFO:teuthology.orchestra.run.smithi042.stdout: "interval": { 2024-08-30T09:16:10.081 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "interval", 2024-08-30T09:16:10.081 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-08-30T09:16:10.081 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.081 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.081 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "60", 2024-08-30T09:16:10.082 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.082 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.082 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.082 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.082 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.082 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.082 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.082 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.082 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-08-30T09:16:10.082 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-08-30T09:16:10.082 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.082 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.082 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.082 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:10.083 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.083 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.083 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:10.083 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:10.083 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:10.083 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:10.083 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:10.083 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:10.083 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:10.083 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:10.083 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.083 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.083 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.084 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.084 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.084 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-08-30T09:16:10.084 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-08-30T09:16:10.084 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.084 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.084 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.084 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:10.084 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.084 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.084 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.084 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.084 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.084 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.085 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.085 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.085 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-08-30T09:16:10.085 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-08-30T09:16:10.085 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.085 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.085 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.085 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-08-30T09:16:10.085 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.085 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.085 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:10.085 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:10.086 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:10.086 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:10.086 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:10.086 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:10.086 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:10.086 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:10.086 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.086 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.086 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.086 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.086 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.086 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-08-30T09:16:10.086 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-08-30T09:16:10.086 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.086 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.087 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.087 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:10.087 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.087 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.087 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.087 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.087 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.087 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.087 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.087 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.087 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-08-30T09:16:10.087 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-08-30T09:16:10.087 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:10.088 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-08-30T09:16:10.088 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.088 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-08-30T09:16:10.088 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.088 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.088 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.088 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.088 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.088 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.088 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.088 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.088 INFO:teuthology.orchestra.run.smithi042.stdout: "zabbix_host": { 2024-08-30T09:16:10.088 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "zabbix_host", 2024-08-30T09:16:10.089 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.089 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.089 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.089 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:10.089 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.089 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.089 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.089 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.089 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.089 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.089 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.089 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.089 INFO:teuthology.orchestra.run.smithi042.stdout: "zabbix_port": { 2024-08-30T09:16:10.089 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "zabbix_port", 2024-08-30T09:16:10.089 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:10.090 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.090 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.090 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "10051", 2024-08-30T09:16:10.090 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.090 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.090 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.090 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.090 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.090 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.090 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.090 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.090 INFO:teuthology.orchestra.run.smithi042.stdout: "zabbix_sender": { 2024-08-30T09:16:10.090 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "zabbix_sender", 2024-08-30T09:16:10.090 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.090 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.091 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.091 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-08-30T09:16:10.091 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.091 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.091 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.091 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.091 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.091 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.091 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.091 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:10.091 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:10.091 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:10.091 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-08-30T09:16:10.091 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.091 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-08-30T09:16:10.092 INFO:teuthology.orchestra.run.smithi042.stdout: "gid": 24134, 2024-08-30T09:16:10.092 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "y", 2024-08-30T09:16:10.092 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr_features": 4540701547738038271, 2024-08-30T09:16:10.092 INFO:teuthology.orchestra.run.smithi042.stdout: "available_modules": [ 2024-08-30T09:16:10.092 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-08-30T09:16:10.092 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "alerts", 2024-08-30T09:16:10.092 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-08-30T09:16:10.092 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-08-30T09:16:10.092 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-08-30T09:16:10.092 INFO:teuthology.orchestra.run.smithi042.stdout: "interval": { 2024-08-30T09:16:10.092 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "interval", 2024-08-30T09:16:10.092 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-08-30T09:16:10.092 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.092 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.093 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "60", 2024-08-30T09:16:10.093 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.093 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.093 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.093 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "How frequently to reexamine health status", 2024-08-30T09:16:10.093 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.093 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.093 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.093 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.093 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-08-30T09:16:10.093 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-08-30T09:16:10.093 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.093 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.093 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.093 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:10.094 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.094 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.094 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:10.094 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:10.094 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:10.094 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:10.094 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:10.094 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:10.094 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:10.094 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:10.094 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.094 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.094 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.094 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.094 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.095 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-08-30T09:16:10.095 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-08-30T09:16:10.095 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.095 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.095 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.095 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:10.095 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.095 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.095 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.095 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.095 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.095 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.095 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.095 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.096 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-08-30T09:16:10.096 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-08-30T09:16:10.096 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.096 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.096 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.096 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-08-30T09:16:10.096 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.096 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.096 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:10.096 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:10.096 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:10.096 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:10.096 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:10.096 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:10.097 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:10.097 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:10.097 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.097 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.097 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.097 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.097 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.097 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-08-30T09:16:10.097 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-08-30T09:16:10.097 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.097 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.097 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.097 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:10.098 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.098 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.098 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.098 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.098 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.098 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.098 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.098 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.098 INFO:teuthology.orchestra.run.smithi042.stdout: "smtp_destination": { 2024-08-30T09:16:10.098 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "smtp_destination", 2024-08-30T09:16:10.098 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.098 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.098 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.098 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:10.098 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.099 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.099 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.099 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Email address to send alerts to", 2024-08-30T09:16:10.099 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.099 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.099 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.099 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.099 INFO:teuthology.orchestra.run.smithi042.stdout: "smtp_from_name": { 2024-08-30T09:16:10.099 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "smtp_from_name", 2024-08-30T09:16:10.099 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.099 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.099 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.099 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "Ceph", 2024-08-30T09:16:10.099 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.099 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.100 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.100 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Email From: name", 2024-08-30T09:16:10.100 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.100 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.100 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.100 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.100 INFO:teuthology.orchestra.run.smithi042.stdout: "smtp_host": { 2024-08-30T09:16:10.100 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "smtp_host", 2024-08-30T09:16:10.100 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.100 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.100 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.100 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:10.100 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.100 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.101 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.101 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "SMTP server", 2024-08-30T09:16:10.101 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.101 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.101 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.101 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.101 INFO:teuthology.orchestra.run.smithi042.stdout: "smtp_password": { 2024-08-30T09:16:10.101 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "smtp_password", 2024-08-30T09:16:10.101 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.101 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.101 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.101 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:10.101 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.101 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.101 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.102 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Password to authenticate with", 2024-08-30T09:16:10.102 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.102 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.102 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.102 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.102 INFO:teuthology.orchestra.run.smithi042.stdout: "smtp_port": { 2024-08-30T09:16:10.102 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "smtp_port", 2024-08-30T09:16:10.102 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:10.102 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.102 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.102 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "465", 2024-08-30T09:16:10.103 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.103 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.103 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.103 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "SMTP port", 2024-08-30T09:16:10.103 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.103 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.104 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.104 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.104 INFO:teuthology.orchestra.run.smithi042.stdout: "smtp_sender": { 2024-08-30T09:16:10.104 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "smtp_sender", 2024-08-30T09:16:10.104 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.104 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.105 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.105 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:10.105 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.105 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.105 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.106 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "SMTP envelope sender", 2024-08-30T09:16:10.106 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.106 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.106 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.106 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.106 INFO:teuthology.orchestra.run.smithi042.stdout: "smtp_ssl": { 2024-08-30T09:16:10.107 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "smtp_ssl", 2024-08-30T09:16:10.107 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.107 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.107 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.107 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-08-30T09:16:10.107 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.107 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.107 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.107 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Use SSL to connect to SMTP server", 2024-08-30T09:16:10.107 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.107 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.107 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.108 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.108 INFO:teuthology.orchestra.run.smithi042.stdout: "smtp_user": { 2024-08-30T09:16:10.108 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "smtp_user", 2024-08-30T09:16:10.108 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.108 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.108 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.108 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:10.108 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.108 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.108 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.108 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "User to authenticate as", 2024-08-30T09:16:10.108 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.109 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.109 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.109 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.109 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-08-30T09:16:10.109 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-08-30T09:16:10.109 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:10.109 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-08-30T09:16:10.109 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.109 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-08-30T09:16:10.109 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.109 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.109 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.109 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.109 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.110 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.110 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.110 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:10.110 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:10.110 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.110 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-08-30T09:16:10.111 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "balancer", 2024-08-30T09:16:10.111 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-08-30T09:16:10.111 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-08-30T09:16:10.111 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-08-30T09:16:10.111 INFO:teuthology.orchestra.run.smithi042.stdout: "active": { 2024-08-30T09:16:10.111 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "active", 2024-08-30T09:16:10.112 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.112 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.112 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.112 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-08-30T09:16:10.112 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.112 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.113 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.113 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "automatically balance PGs across cluster", 2024-08-30T09:16:10.113 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.113 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.113 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.114 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.114 INFO:teuthology.orchestra.run.smithi042.stdout: "begin_time": { 2024-08-30T09:16:10.114 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "begin_time", 2024-08-30T09:16:10.114 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.114 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.114 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.114 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0000", 2024-08-30T09:16:10.114 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.115 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.115 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.115 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "beginning time of day to automatically balance", 2024-08-30T09:16:10.115 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-08-30T09:16:10.115 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.115 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.115 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.115 INFO:teuthology.orchestra.run.smithi042.stdout: "begin_weekday": { 2024-08-30T09:16:10.115 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "begin_weekday", 2024-08-30T09:16:10.115 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "uint", 2024-08-30T09:16:10.115 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.115 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.116 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-08-30T09:16:10.116 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "0", 2024-08-30T09:16:10.116 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "6", 2024-08-30T09:16:10.116 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.116 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-08-30T09:16:10.116 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-08-30T09:16:10.116 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.116 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.116 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.116 INFO:teuthology.orchestra.run.smithi042.stdout: "crush_compat_max_iterations": { 2024-08-30T09:16:10.116 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "crush_compat_max_iterations", 2024-08-30T09:16:10.116 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "uint", 2024-08-30T09:16:10.116 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.117 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.117 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "25", 2024-08-30T09:16:10.117 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "1", 2024-08-30T09:16:10.117 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "250", 2024-08-30T09:16:10.117 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.117 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-08-30T09:16:10.117 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.117 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.117 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.117 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.117 INFO:teuthology.orchestra.run.smithi042.stdout: "crush_compat_metrics": { 2024-08-30T09:16:10.117 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "crush_compat_metrics", 2024-08-30T09:16:10.117 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.118 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.118 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.118 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "pgs,objects,bytes", 2024-08-30T09:16:10.118 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.118 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.118 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.118 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-08-30T09:16:10.118 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-08-30T09:16:10.118 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.118 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.118 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.118 INFO:teuthology.orchestra.run.smithi042.stdout: "crush_compat_step": { 2024-08-30T09:16:10.118 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "crush_compat_step", 2024-08-30T09:16:10.119 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "float", 2024-08-30T09:16:10.119 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.119 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.119 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0.5", 2024-08-30T09:16:10.119 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "0.001", 2024-08-30T09:16:10.119 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "0.999", 2024-08-30T09:16:10.119 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.119 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "aggressiveness of optimization", 2024-08-30T09:16:10.119 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-08-30T09:16:10.119 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.119 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.119 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.119 INFO:teuthology.orchestra.run.smithi042.stdout: "end_time": { 2024-08-30T09:16:10.119 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "end_time", 2024-08-30T09:16:10.120 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.120 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.120 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.120 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "2359", 2024-08-30T09:16:10.120 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.120 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.120 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.120 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "ending time of day to automatically balance", 2024-08-30T09:16:10.120 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-08-30T09:16:10.120 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.120 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.120 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.120 INFO:teuthology.orchestra.run.smithi042.stdout: "end_weekday": { 2024-08-30T09:16:10.121 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "end_weekday", 2024-08-30T09:16:10.121 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "uint", 2024-08-30T09:16:10.121 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.121 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.121 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-08-30T09:16:10.121 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "0", 2024-08-30T09:16:10.121 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "6", 2024-08-30T09:16:10.121 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.121 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-08-30T09:16:10.121 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-08-30T09:16:10.121 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.121 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.122 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.122 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-08-30T09:16:10.122 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-08-30T09:16:10.122 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.122 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.122 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.122 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:10.122 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.122 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.122 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:10.122 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:10.122 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:10.122 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:10.123 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:10.123 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:10.123 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:10.123 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:10.123 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.123 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.123 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.123 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.123 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.123 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-08-30T09:16:10.123 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-08-30T09:16:10.123 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.123 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.123 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.124 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:10.124 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.124 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.124 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.124 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.124 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.124 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.124 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.124 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.124 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-08-30T09:16:10.124 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-08-30T09:16:10.124 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.125 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.125 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.125 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-08-30T09:16:10.125 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.125 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.125 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:10.125 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:10.125 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:10.125 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:10.125 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:10.125 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:10.125 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:10.125 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:10.126 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.126 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.126 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.126 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.126 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.126 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-08-30T09:16:10.126 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-08-30T09:16:10.126 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.126 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.126 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.126 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:10.126 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.126 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.127 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.127 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.127 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.127 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.127 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.127 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.127 INFO:teuthology.orchestra.run.smithi042.stdout: "min_score": { 2024-08-30T09:16:10.127 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "min_score", 2024-08-30T09:16:10.127 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "float", 2024-08-30T09:16:10.127 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.127 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.127 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-08-30T09:16:10.127 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.127 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.127 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.128 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-08-30T09:16:10.128 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.128 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.128 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.128 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.128 INFO:teuthology.orchestra.run.smithi042.stdout: "mode": { 2024-08-30T09:16:10.128 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "mode", 2024-08-30T09:16:10.128 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.128 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.128 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.128 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "upmap", 2024-08-30T09:16:10.128 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.129 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.129 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:10.129 INFO:teuthology.orchestra.run.smithi042.stdout: "crush-compat", 2024-08-30T09:16:10.129 INFO:teuthology.orchestra.run.smithi042.stdout: "none", 2024-08-30T09:16:10.129 INFO:teuthology.orchestra.run.smithi042.stdout: "read", 2024-08-30T09:16:10.129 INFO:teuthology.orchestra.run.smithi042.stdout: "upmap", 2024-08-30T09:16:10.129 INFO:teuthology.orchestra.run.smithi042.stdout: "upmap-read" 2024-08-30T09:16:10.129 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:10.129 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Balancer mode", 2024-08-30T09:16:10.129 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.129 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.129 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.129 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.130 INFO:teuthology.orchestra.run.smithi042.stdout: "pool_ids": { 2024-08-30T09:16:10.130 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "pool_ids", 2024-08-30T09:16:10.130 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.130 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.130 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.130 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:10.130 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.130 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.130 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.130 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-08-30T09:16:10.130 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.130 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.130 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.130 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.130 INFO:teuthology.orchestra.run.smithi042.stdout: "sleep_interval": { 2024-08-30T09:16:10.131 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sleep_interval", 2024-08-30T09:16:10.131 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-08-30T09:16:10.131 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.131 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.131 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "60", 2024-08-30T09:16:10.131 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.131 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.131 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.131 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-08-30T09:16:10.131 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.131 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.131 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.131 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.131 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-08-30T09:16:10.131 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-08-30T09:16:10.132 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:10.132 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-08-30T09:16:10.132 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.132 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-08-30T09:16:10.132 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.132 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.132 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.132 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.132 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.132 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.132 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.132 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.132 INFO:teuthology.orchestra.run.smithi042.stdout: "upmap_max_deviation": { 2024-08-30T09:16:10.132 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "upmap_max_deviation", 2024-08-30T09:16:10.133 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:10.133 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.133 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.133 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "5", 2024-08-30T09:16:10.133 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "1", 2024-08-30T09:16:10.133 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.133 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.133 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "deviation below which no optimization is attempted", 2024-08-30T09:16:10.133 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-08-30T09:16:10.133 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.133 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.133 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.133 INFO:teuthology.orchestra.run.smithi042.stdout: "upmap_max_optimizations": { 2024-08-30T09:16:10.133 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "upmap_max_optimizations", 2024-08-30T09:16:10.134 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "uint", 2024-08-30T09:16:10.134 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.134 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.134 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "10", 2024-08-30T09:16:10.134 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.134 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.134 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.134 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-08-30T09:16:10.134 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.134 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.134 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.134 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:10.134 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:10.134 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.134 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-08-30T09:16:10.135 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "cephadm", 2024-08-30T09:16:10.135 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-08-30T09:16:10.135 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-08-30T09:16:10.135 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-08-30T09:16:10.135 INFO:teuthology.orchestra.run.smithi042.stdout: "agent_down_multiplier": { 2024-08-30T09:16:10.135 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "agent_down_multiplier", 2024-08-30T09:16:10.135 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "float", 2024-08-30T09:16:10.135 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.135 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.135 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "3.0", 2024-08-30T09:16:10.135 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.135 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.135 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.135 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-08-30T09:16:10.136 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.136 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.136 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.136 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.136 INFO:teuthology.orchestra.run.smithi042.stdout: "agent_refresh_rate": { 2024-08-30T09:16:10.136 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "agent_refresh_rate", 2024-08-30T09:16:10.136 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-08-30T09:16:10.136 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.136 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.136 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "20", 2024-08-30T09:16:10.136 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.136 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.136 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.136 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-08-30T09:16:10.137 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.137 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.137 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.137 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.137 INFO:teuthology.orchestra.run.smithi042.stdout: "agent_starting_port": { 2024-08-30T09:16:10.137 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "agent_starting_port", 2024-08-30T09:16:10.137 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:10.137 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.137 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.137 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "4721", 2024-08-30T09:16:10.137 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.137 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.137 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.138 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-08-30T09:16:10.138 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.138 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.138 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.138 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.138 INFO:teuthology.orchestra.run.smithi042.stdout: "allow_ptrace": { 2024-08-30T09:16:10.138 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "allow_ptrace", 2024-08-30T09:16:10.138 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.138 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.138 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.138 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:10.138 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.138 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.138 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.139 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-08-30T09:16:10.139 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-08-30T09:16:10.139 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.139 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.139 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.139 INFO:teuthology.orchestra.run.smithi042.stdout: "autotune_interval": { 2024-08-30T09:16:10.139 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "autotune_interval", 2024-08-30T09:16:10.139 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-08-30T09:16:10.139 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.139 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.139 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "600", 2024-08-30T09:16:10.139 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.139 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.139 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.140 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "how frequently to autotune daemon memory", 2024-08-30T09:16:10.140 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.140 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.140 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.140 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.140 INFO:teuthology.orchestra.run.smithi042.stdout: "autotune_memory_target_ratio": { 2024-08-30T09:16:10.140 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "autotune_memory_target_ratio", 2024-08-30T09:16:10.140 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "float", 2024-08-30T09:16:10.140 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.140 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.140 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0.7", 2024-08-30T09:16:10.140 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.140 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.140 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.141 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-08-30T09:16:10.141 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.141 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.141 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.141 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.141 INFO:teuthology.orchestra.run.smithi042.stdout: "cephadm_log_destination": { 2024-08-30T09:16:10.141 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "cephadm_log_destination", 2024-08-30T09:16:10.141 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.141 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.141 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.141 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:10.141 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.141 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.141 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:10.142 INFO:teuthology.orchestra.run.smithi042.stdout: "file", 2024-08-30T09:16:10.142 INFO:teuthology.orchestra.run.smithi042.stdout: "file,syslog", 2024-08-30T09:16:10.142 INFO:teuthology.orchestra.run.smithi042.stdout: "syslog" 2024-08-30T09:16:10.142 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:10.142 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-08-30T09:16:10.142 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.142 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.142 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.142 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.142 INFO:teuthology.orchestra.run.smithi042.stdout: "cgroups_split": { 2024-08-30T09:16:10.142 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "cgroups_split", 2024-08-30T09:16:10.142 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.142 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.142 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.143 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-08-30T09:16:10.143 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.143 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.143 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.143 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-08-30T09:16:10.143 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.143 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.143 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.143 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.143 INFO:teuthology.orchestra.run.smithi042.stdout: "config_checks_enabled": { 2024-08-30T09:16:10.143 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "config_checks_enabled", 2024-08-30T09:16:10.143 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.143 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.143 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.144 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:10.144 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.144 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.144 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.144 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-08-30T09:16:10.144 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.144 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.144 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.144 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.144 INFO:teuthology.orchestra.run.smithi042.stdout: "config_dashboard": { 2024-08-30T09:16:10.144 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "config_dashboard", 2024-08-30T09:16:10.144 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.144 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.145 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.145 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-08-30T09:16:10.145 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.145 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.145 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.145 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-08-30T09:16:10.145 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.145 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.145 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.145 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.145 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_alertmanager": { 2024-08-30T09:16:10.145 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_alertmanager", 2024-08-30T09:16:10.145 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.145 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.145 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.146 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.27.0", 2024-08-30T09:16:10.146 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.146 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.146 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.146 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Prometheus container image", 2024-08-30T09:16:10.146 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.146 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.146 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.146 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.146 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_base": { 2024-08-30T09:16:10.146 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_base", 2024-08-30T09:16:10.146 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.146 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.146 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.146 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/ceph/ceph", 2024-08-30T09:16:10.147 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.147 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.147 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.147 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Container image name, without the tag", 2024-08-30T09:16:10.147 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.147 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.147 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.147 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.147 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_elasticsearch": { 2024-08-30T09:16:10.147 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_elasticsearch", 2024-08-30T09:16:10.147 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.147 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.147 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.148 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-08-30T09:16:10.148 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.148 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.148 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.148 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "elasticsearch container image", 2024-08-30T09:16:10.148 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.148 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.148 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.148 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.148 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_grafana": { 2024-08-30T09:16:10.148 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_grafana", 2024-08-30T09:16:10.148 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.148 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.149 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.149 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/ceph/grafana:10.4.0", 2024-08-30T09:16:10.149 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.149 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.149 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.149 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Prometheus container image", 2024-08-30T09:16:10.149 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.149 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.149 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.149 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.149 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_haproxy": { 2024-08-30T09:16:10.149 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_haproxy", 2024-08-30T09:16:10.149 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.149 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.150 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.150 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-08-30T09:16:10.150 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.150 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.150 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.150 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "HAproxy container image", 2024-08-30T09:16:10.150 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.150 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.150 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.150 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.150 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_jaeger_agent": { 2024-08-30T09:16:10.150 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_jaeger_agent", 2024-08-30T09:16:10.150 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.150 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.150 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.151 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-08-30T09:16:10.151 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.151 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.151 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.151 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Jaeger agent container image", 2024-08-30T09:16:10.151 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.151 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.151 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.151 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.151 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_jaeger_collector": { 2024-08-30T09:16:10.151 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_jaeger_collector", 2024-08-30T09:16:10.151 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.151 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.152 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.152 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-08-30T09:16:10.152 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.152 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.152 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.152 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Jaeger collector container image", 2024-08-30T09:16:10.152 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.152 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.152 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.152 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.152 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_jaeger_query": { 2024-08-30T09:16:10.152 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_jaeger_query", 2024-08-30T09:16:10.152 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.153 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.153 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.153 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-08-30T09:16:10.153 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.153 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.153 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.153 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Jaeger query container image", 2024-08-30T09:16:10.153 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.153 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.153 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.153 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.153 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_keepalived": { 2024-08-30T09:16:10.153 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_keepalived", 2024-08-30T09:16:10.153 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.154 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.154 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.154 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-08-30T09:16:10.154 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.154 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.154 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.154 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Keepalived container image", 2024-08-30T09:16:10.154 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.154 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.154 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.154 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.154 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_loki": { 2024-08-30T09:16:10.154 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_loki", 2024-08-30T09:16:10.154 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.155 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.155 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.155 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-08-30T09:16:10.155 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.155 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.155 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.155 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Loki container image", 2024-08-30T09:16:10.155 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.155 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.155 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.155 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.155 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_nginx": { 2024-08-30T09:16:10.155 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_nginx", 2024-08-30T09:16:10.155 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.155 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.156 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.156 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/ceph/nginx:1.26.1", 2024-08-30T09:16:10.156 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.156 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.156 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.156 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Nginx container image", 2024-08-30T09:16:10.156 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.156 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.156 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.156 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.156 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_node_exporter": { 2024-08-30T09:16:10.156 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_node_exporter", 2024-08-30T09:16:10.157 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.157 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.157 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.157 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.7.0", 2024-08-30T09:16:10.157 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.157 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.157 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.157 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Prometheus container image", 2024-08-30T09:16:10.157 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.157 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.157 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.157 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.157 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_nvmeof": { 2024-08-30T09:16:10.157 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_nvmeof", 2024-08-30T09:16:10.158 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.158 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.158 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.158 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.17", 2024-08-30T09:16:10.158 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.158 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.158 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.158 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Nvme-of container image", 2024-08-30T09:16:10.158 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.158 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.158 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.158 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.158 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_oauth2_proxy": { 2024-08-30T09:16:10.158 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_oauth2_proxy", 2024-08-30T09:16:10.159 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.159 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.159 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.159 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/oauth2-proxy/oauth2-proxy:v7.6.0", 2024-08-30T09:16:10.159 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.159 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.159 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.159 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "oauth2-proxy container image", 2024-08-30T09:16:10.159 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.159 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.159 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.159 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.159 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_prometheus": { 2024-08-30T09:16:10.159 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_prometheus", 2024-08-30T09:16:10.160 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.160 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.160 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.160 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/prometheus/prometheus:v2.51.0", 2024-08-30T09:16:10.160 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.160 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.160 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.160 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Prometheus container image", 2024-08-30T09:16:10.160 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.160 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.160 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.160 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.160 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_promtail": { 2024-08-30T09:16:10.160 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_promtail", 2024-08-30T09:16:10.160 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.161 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.161 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.161 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-08-30T09:16:10.161 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.161 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.161 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.161 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Promtail container image", 2024-08-30T09:16:10.161 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.161 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.161 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.161 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.161 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_samba": { 2024-08-30T09:16:10.161 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_samba", 2024-08-30T09:16:10.162 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.162 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.162 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.162 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-08-30T09:16:10.162 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.162 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.162 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.162 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Samba/SMB container image", 2024-08-30T09:16:10.162 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.162 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.162 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.162 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.162 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_snmp_gateway": { 2024-08-30T09:16:10.163 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_snmp_gateway", 2024-08-30T09:16:10.163 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.163 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.163 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.163 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-08-30T09:16:10.163 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.163 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.163 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.163 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "SNMP Gateway container image", 2024-08-30T09:16:10.163 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.163 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.163 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.163 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.163 INFO:teuthology.orchestra.run.smithi042.stdout: "container_init": { 2024-08-30T09:16:10.164 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_init", 2024-08-30T09:16:10.164 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.164 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.164 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.164 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-08-30T09:16:10.164 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.164 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.164 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.164 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Run podman/docker with `--init`", 2024-08-30T09:16:10.164 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.164 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.164 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.164 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.164 INFO:teuthology.orchestra.run.smithi042.stdout: "daemon_cache_timeout": { 2024-08-30T09:16:10.165 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "daemon_cache_timeout", 2024-08-30T09:16:10.165 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-08-30T09:16:10.165 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.165 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.165 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "600", 2024-08-30T09:16:10.165 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.165 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.165 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.165 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-08-30T09:16:10.165 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.165 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.165 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.165 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.166 INFO:teuthology.orchestra.run.smithi042.stdout: "default_cephadm_command_timeout": { 2024-08-30T09:16:10.166 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "default_cephadm_command_timeout", 2024-08-30T09:16:10.166 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:10.166 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.166 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.166 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "900", 2024-08-30T09:16:10.166 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.166 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.166 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.166 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-08-30T09:16:10.166 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.166 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.166 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.167 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.167 INFO:teuthology.orchestra.run.smithi042.stdout: "default_registry": { 2024-08-30T09:16:10.167 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "default_registry", 2024-08-30T09:16:10.167 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.167 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.167 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.167 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "docker.io", 2024-08-30T09:16:10.167 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.167 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.167 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.167 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-08-30T09:16:10.167 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.168 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.168 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.168 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.168 INFO:teuthology.orchestra.run.smithi042.stdout: "device_cache_timeout": { 2024-08-30T09:16:10.168 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "device_cache_timeout", 2024-08-30T09:16:10.168 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-08-30T09:16:10.168 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.168 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.168 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "1800", 2024-08-30T09:16:10.168 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.168 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.168 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.168 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "seconds to cache device inventory", 2024-08-30T09:16:10.168 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.169 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.169 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.169 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.169 INFO:teuthology.orchestra.run.smithi042.stdout: "device_enhanced_scan": { 2024-08-30T09:16:10.169 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "device_enhanced_scan", 2024-08-30T09:16:10.169 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.169 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.169 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.169 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:10.169 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.169 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.169 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.169 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Use libstoragemgmt during device scans", 2024-08-30T09:16:10.169 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.170 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.170 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.170 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.170 INFO:teuthology.orchestra.run.smithi042.stdout: "facts_cache_timeout": { 2024-08-30T09:16:10.170 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "facts_cache_timeout", 2024-08-30T09:16:10.170 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-08-30T09:16:10.170 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.170 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.170 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "60", 2024-08-30T09:16:10.170 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.170 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.170 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.170 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "seconds to cache host facts data", 2024-08-30T09:16:10.171 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.171 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.171 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.171 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.171 INFO:teuthology.orchestra.run.smithi042.stdout: "grafana_dashboards_path": { 2024-08-30T09:16:10.171 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "grafana_dashboards_path", 2024-08-30T09:16:10.171 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.171 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.171 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.171 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-08-30T09:16:10.171 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.171 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.171 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.172 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-08-30T09:16:10.172 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.172 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.172 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.172 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.172 INFO:teuthology.orchestra.run.smithi042.stdout: "host_check_interval": { 2024-08-30T09:16:10.172 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "host_check_interval", 2024-08-30T09:16:10.172 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-08-30T09:16:10.172 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.172 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.172 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "600", 2024-08-30T09:16:10.172 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.172 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.173 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.173 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "how frequently to perform a host check", 2024-08-30T09:16:10.173 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.173 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.173 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.173 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.173 INFO:teuthology.orchestra.run.smithi042.stdout: "hw_monitoring": { 2024-08-30T09:16:10.173 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "hw_monitoring", 2024-08-30T09:16:10.173 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.173 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.173 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.173 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:10.173 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.173 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.174 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.174 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-08-30T09:16:10.174 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.174 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.174 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.174 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.174 INFO:teuthology.orchestra.run.smithi042.stdout: "inventory_list_all": { 2024-08-30T09:16:10.174 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "inventory_list_all", 2024-08-30T09:16:10.174 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.174 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.174 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.174 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:10.174 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.174 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.174 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.175 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-08-30T09:16:10.175 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.175 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.175 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.175 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.175 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-08-30T09:16:10.175 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-08-30T09:16:10.175 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.175 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.175 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.175 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:10.175 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.175 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.175 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:10.176 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:10.176 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:10.176 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:10.176 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:10.176 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:10.176 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:10.176 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:10.176 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.176 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.176 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.176 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.176 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.176 INFO:teuthology.orchestra.run.smithi042.stdout: "log_refresh_metadata": { 2024-08-30T09:16:10.177 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_refresh_metadata", 2024-08-30T09:16:10.177 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.177 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.177 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.177 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:10.177 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.177 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.177 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.177 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-08-30T09:16:10.177 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.177 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.177 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.177 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.178 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-08-30T09:16:10.178 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-08-30T09:16:10.178 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.178 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.178 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.178 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-08-30T09:16:10.178 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.178 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.178 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.178 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-08-30T09:16:10.178 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.178 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.178 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.178 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.179 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-08-30T09:16:10.179 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-08-30T09:16:10.179 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.179 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.179 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.179 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-08-30T09:16:10.179 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.179 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.179 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:10.179 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:10.179 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:10.179 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:10.179 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:10.179 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:10.180 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:10.180 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:10.180 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.180 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.180 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.180 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.180 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.180 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-08-30T09:16:10.180 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-08-30T09:16:10.180 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.180 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.180 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.180 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:10.180 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.181 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.181 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.181 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.181 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.181 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.181 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.181 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.181 INFO:teuthology.orchestra.run.smithi042.stdout: "manage_etc_ceph_ceph_conf": { 2024-08-30T09:16:10.181 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-08-30T09:16:10.181 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.181 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.181 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.181 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:10.181 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.182 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.182 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.182 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-08-30T09:16:10.182 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.182 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.182 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.182 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.182 INFO:teuthology.orchestra.run.smithi042.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-08-30T09:16:10.182 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-08-30T09:16:10.182 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.182 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.183 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.183 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "*", 2024-08-30T09:16:10.183 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.183 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.183 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.183 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-08-30T09:16:10.183 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.183 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.183 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.183 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.183 INFO:teuthology.orchestra.run.smithi042.stdout: "max_count_per_host": { 2024-08-30T09:16:10.183 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "max_count_per_host", 2024-08-30T09:16:10.183 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:10.183 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.184 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.184 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "10", 2024-08-30T09:16:10.184 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.184 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.184 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.184 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "max number of daemons per service per host", 2024-08-30T09:16:10.184 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.184 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.184 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.184 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.184 INFO:teuthology.orchestra.run.smithi042.stdout: "max_osd_draining_count": { 2024-08-30T09:16:10.184 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "max_osd_draining_count", 2024-08-30T09:16:10.184 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:10.184 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.185 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.185 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "10", 2024-08-30T09:16:10.185 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.185 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.185 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.185 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-08-30T09:16:10.185 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.185 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.185 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.185 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.185 INFO:teuthology.orchestra.run.smithi042.stdout: "migration_current": { 2024-08-30T09:16:10.185 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "migration_current", 2024-08-30T09:16:10.185 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:10.186 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.186 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.186 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:10.186 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.186 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.186 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.186 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "internal - do not modify", 2024-08-30T09:16:10.186 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.186 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.186 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.186 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.186 INFO:teuthology.orchestra.run.smithi042.stdout: "mode": { 2024-08-30T09:16:10.186 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "mode", 2024-08-30T09:16:10.187 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.187 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.187 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.187 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "root", 2024-08-30T09:16:10.187 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.187 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.187 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:10.187 INFO:teuthology.orchestra.run.smithi042.stdout: "cephadm-package", 2024-08-30T09:16:10.187 INFO:teuthology.orchestra.run.smithi042.stdout: "root" 2024-08-30T09:16:10.187 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:10.187 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "mode for remote execution of cephadm", 2024-08-30T09:16:10.187 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.187 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.187 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.188 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.188 INFO:teuthology.orchestra.run.smithi042.stdout: "oob_default_addr": { 2024-08-30T09:16:10.188 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "oob_default_addr", 2024-08-30T09:16:10.188 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.188 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.188 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.188 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "169.254.1.1", 2024-08-30T09:16:10.188 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.188 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.188 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.188 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Default address for RedFish API (oob management).", 2024-08-30T09:16:10.188 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.189 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.189 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.189 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.189 INFO:teuthology.orchestra.run.smithi042.stdout: "prometheus_alerts_path": { 2024-08-30T09:16:10.189 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "prometheus_alerts_path", 2024-08-30T09:16:10.189 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.189 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.189 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.189 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-08-30T09:16:10.189 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.189 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.189 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.189 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-08-30T09:16:10.189 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.189 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.190 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.190 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.190 INFO:teuthology.orchestra.run.smithi042.stdout: "registry_insecure": { 2024-08-30T09:16:10.190 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "registry_insecure", 2024-08-30T09:16:10.190 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.190 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.190 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.190 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:10.190 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.190 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.190 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.190 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-08-30T09:16:10.190 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.190 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.191 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.191 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.191 INFO:teuthology.orchestra.run.smithi042.stdout: "registry_password": { 2024-08-30T09:16:10.191 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "registry_password", 2024-08-30T09:16:10.191 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.191 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.191 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.191 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:10.191 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.191 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.191 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.191 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-08-30T09:16:10.191 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.191 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.191 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.192 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.192 INFO:teuthology.orchestra.run.smithi042.stdout: "registry_url": { 2024-08-30T09:16:10.192 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "registry_url", 2024-08-30T09:16:10.192 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.192 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.192 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.192 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:10.192 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.192 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.192 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.192 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-08-30T09:16:10.192 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.193 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.193 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.193 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.193 INFO:teuthology.orchestra.run.smithi042.stdout: "registry_username": { 2024-08-30T09:16:10.193 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "registry_username", 2024-08-30T09:16:10.193 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.193 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.193 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.193 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:10.193 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.193 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.193 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.193 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-08-30T09:16:10.193 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.194 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.194 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.194 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.194 INFO:teuthology.orchestra.run.smithi042.stdout: "secure_monitoring_stack": { 2024-08-30T09:16:10.194 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "secure_monitoring_stack", 2024-08-30T09:16:10.194 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.194 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.194 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.194 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:10.194 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.194 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.194 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.194 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-08-30T09:16:10.195 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.195 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.195 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.195 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.195 INFO:teuthology.orchestra.run.smithi042.stdout: "service_discovery_port": { 2024-08-30T09:16:10.195 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "service_discovery_port", 2024-08-30T09:16:10.195 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:10.195 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.195 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.195 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "8765", 2024-08-30T09:16:10.195 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.195 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.195 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.195 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "cephadm service discovery port", 2024-08-30T09:16:10.196 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.196 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.196 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.196 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.196 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-08-30T09:16:10.196 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-08-30T09:16:10.196 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:10.196 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-08-30T09:16:10.196 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.196 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-08-30T09:16:10.196 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.196 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.196 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.196 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.197 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.197 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.197 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.197 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.197 INFO:teuthology.orchestra.run.smithi042.stdout: "ssh_config_file": { 2024-08-30T09:16:10.197 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ssh_config_file", 2024-08-30T09:16:10.197 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.197 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.197 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.197 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:10.197 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.197 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.197 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.197 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-08-30T09:16:10.198 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.198 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.198 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.198 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.198 INFO:teuthology.orchestra.run.smithi042.stdout: "use_agent": { 2024-08-30T09:16:10.198 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "use_agent", 2024-08-30T09:16:10.198 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.198 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.198 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.198 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:10.198 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.198 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.198 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.198 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-08-30T09:16:10.199 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.199 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.199 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.199 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.199 INFO:teuthology.orchestra.run.smithi042.stdout: "use_repo_digest": { 2024-08-30T09:16:10.199 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "use_repo_digest", 2024-08-30T09:16:10.199 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.199 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.199 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.199 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-08-30T09:16:10.199 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.199 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.199 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.200 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-08-30T09:16:10.200 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.200 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.200 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.200 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.200 INFO:teuthology.orchestra.run.smithi042.stdout: "warn_on_failed_host_check": { 2024-08-30T09:16:10.200 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "warn_on_failed_host_check", 2024-08-30T09:16:10.200 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.200 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.200 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.200 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-08-30T09:16:10.200 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.200 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.200 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.201 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "raise a health warning if the host check fails", 2024-08-30T09:16:10.201 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.201 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.201 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.201 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.201 INFO:teuthology.orchestra.run.smithi042.stdout: "warn_on_stray_daemons": { 2024-08-30T09:16:10.201 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "warn_on_stray_daemons", 2024-08-30T09:16:10.201 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.201 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.201 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.201 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-08-30T09:16:10.201 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.201 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.201 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.202 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-08-30T09:16:10.202 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.202 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.202 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.202 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.202 INFO:teuthology.orchestra.run.smithi042.stdout: "warn_on_stray_hosts": { 2024-08-30T09:16:10.202 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "warn_on_stray_hosts", 2024-08-30T09:16:10.202 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.202 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.202 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.202 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-08-30T09:16:10.202 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.202 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.202 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.202 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-08-30T09:16:10.203 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.203 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.203 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.203 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:10.203 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:10.203 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.203 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-08-30T09:16:10.203 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "crash", 2024-08-30T09:16:10.203 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-08-30T09:16:10.203 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-08-30T09:16:10.203 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-08-30T09:16:10.203 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-08-30T09:16:10.203 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-08-30T09:16:10.204 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.204 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.204 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.204 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:10.204 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.204 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.204 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:10.204 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:10.204 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:10.204 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:10.204 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:10.204 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:10.204 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:10.204 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:10.205 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.205 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.205 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.205 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.205 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.205 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-08-30T09:16:10.205 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-08-30T09:16:10.205 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.205 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.205 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.205 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:10.205 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.205 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.205 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.205 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.206 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.206 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.206 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.206 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.206 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-08-30T09:16:10.206 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-08-30T09:16:10.206 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.206 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.206 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.206 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-08-30T09:16:10.206 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.206 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.206 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:10.206 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:10.206 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:10.207 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:10.207 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:10.207 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:10.207 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:10.207 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:10.207 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.207 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.207 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.207 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.207 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.207 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-08-30T09:16:10.207 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-08-30T09:16:10.207 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.207 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.208 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.208 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:10.208 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.208 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.208 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.208 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.208 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.208 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.208 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.208 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.208 INFO:teuthology.orchestra.run.smithi042.stdout: "retain_interval": { 2024-08-30T09:16:10.208 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "retain_interval", 2024-08-30T09:16:10.208 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-08-30T09:16:10.209 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.209 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.209 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "31536000", 2024-08-30T09:16:10.209 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.209 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.209 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.209 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "how long to retain crashes before pruning them", 2024-08-30T09:16:10.209 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.209 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.209 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.209 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.209 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-08-30T09:16:10.209 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-08-30T09:16:10.210 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:10.210 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-08-30T09:16:10.210 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.210 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-08-30T09:16:10.210 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.210 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.210 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.210 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.210 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.210 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.210 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.210 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.210 INFO:teuthology.orchestra.run.smithi042.stdout: "warn_recent_interval": { 2024-08-30T09:16:10.211 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "warn_recent_interval", 2024-08-30T09:16:10.211 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-08-30T09:16:10.211 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.211 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.211 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "1209600", 2024-08-30T09:16:10.211 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.211 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.211 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.211 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "time interval in which to warn about recent crashes", 2024-08-30T09:16:10.211 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.211 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.211 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.211 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:10.211 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:10.212 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.212 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-08-30T09:16:10.212 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "dashboard", 2024-08-30T09:16:10.212 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-08-30T09:16:10.212 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-08-30T09:16:10.212 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-08-30T09:16:10.212 INFO:teuthology.orchestra.run.smithi042.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-08-30T09:16:10.212 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-08-30T09:16:10.212 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:10.212 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.212 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.212 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "10", 2024-08-30T09:16:10.212 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.212 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.213 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.213 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.213 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.213 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.213 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.213 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.213 INFO:teuthology.orchestra.run.smithi042.stdout: "ALERTMANAGER_API_HOST": { 2024-08-30T09:16:10.213 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ALERTMANAGER_API_HOST", 2024-08-30T09:16:10.213 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.213 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.213 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.213 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:10.213 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.213 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.214 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.214 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.214 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.214 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.214 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.214 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.214 INFO:teuthology.orchestra.run.smithi042.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-08-30T09:16:10.214 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-08-30T09:16:10.214 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.214 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.214 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.214 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-08-30T09:16:10.214 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.214 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.215 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.215 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.215 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.215 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.215 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.215 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.215 INFO:teuthology.orchestra.run.smithi042.stdout: "AUDIT_API_ENABLED": { 2024-08-30T09:16:10.215 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "AUDIT_API_ENABLED", 2024-08-30T09:16:10.215 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.215 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.215 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.215 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:10.215 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.215 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.216 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.216 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.216 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.216 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.216 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.216 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.216 INFO:teuthology.orchestra.run.smithi042.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-08-30T09:16:10.216 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-08-30T09:16:10.216 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.216 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.216 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.216 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-08-30T09:16:10.216 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.216 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.217 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.217 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.217 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.217 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.217 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.217 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.217 INFO:teuthology.orchestra.run.smithi042.stdout: "ENABLE_BROWSABLE_API": { 2024-08-30T09:16:10.217 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ENABLE_BROWSABLE_API", 2024-08-30T09:16:10.217 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.217 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.217 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.217 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-08-30T09:16:10.217 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.217 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.218 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.218 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.218 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.218 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.218 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.218 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.218 INFO:teuthology.orchestra.run.smithi042.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-08-30T09:16:10.218 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-08-30T09:16:10.218 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.218 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.218 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.218 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-08-30T09:16:10.218 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.219 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.219 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.219 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.219 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.219 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.219 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.219 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.219 INFO:teuthology.orchestra.run.smithi042.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-08-30T09:16:10.219 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-08-30T09:16:10.219 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.219 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.219 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.219 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-08-30T09:16:10.219 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.219 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.220 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.220 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.220 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.220 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.220 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.220 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.220 INFO:teuthology.orchestra.run.smithi042.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-08-30T09:16:10.220 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-08-30T09:16:10.220 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.220 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.220 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.220 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-08-30T09:16:10.220 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.221 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.221 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.221 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.221 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.221 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.221 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.221 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.221 INFO:teuthology.orchestra.run.smithi042.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-08-30T09:16:10.221 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-08-30T09:16:10.221 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.221 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.221 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.221 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-08-30T09:16:10.221 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.222 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.222 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.222 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.222 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.222 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.222 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.222 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.222 INFO:teuthology.orchestra.run.smithi042.stdout: "FEATURE_TOGGLE_NFS": { 2024-08-30T09:16:10.222 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-08-30T09:16:10.222 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.222 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.222 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.222 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-08-30T09:16:10.222 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.223 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.223 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.223 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.223 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.223 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.223 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.223 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.223 INFO:teuthology.orchestra.run.smithi042.stdout: "FEATURE_TOGGLE_RBD": { 2024-08-30T09:16:10.223 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-08-30T09:16:10.223 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.223 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.223 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.223 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-08-30T09:16:10.223 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.224 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.224 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.224 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.224 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.224 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.224 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.224 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.224 INFO:teuthology.orchestra.run.smithi042.stdout: "FEATURE_TOGGLE_RGW": { 2024-08-30T09:16:10.224 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-08-30T09:16:10.224 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.224 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.224 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.224 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-08-30T09:16:10.224 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.225 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.225 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.225 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.225 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.225 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.225 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.225 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.225 INFO:teuthology.orchestra.run.smithi042.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-08-30T09:16:10.225 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-08-30T09:16:10.225 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.225 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.225 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.225 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:10.225 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.225 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.226 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.226 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.226 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.226 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.226 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.226 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.226 INFO:teuthology.orchestra.run.smithi042.stdout: "GRAFANA_API_PASSWORD": { 2024-08-30T09:16:10.226 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "GRAFANA_API_PASSWORD", 2024-08-30T09:16:10.226 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.226 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.226 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.226 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "admin", 2024-08-30T09:16:10.226 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.226 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.226 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.227 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.227 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.227 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.227 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.227 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.227 INFO:teuthology.orchestra.run.smithi042.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-08-30T09:16:10.227 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-08-30T09:16:10.227 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.227 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.227 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.227 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-08-30T09:16:10.227 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.227 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.227 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.227 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.228 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.228 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.228 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.228 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.228 INFO:teuthology.orchestra.run.smithi042.stdout: "GRAFANA_API_URL": { 2024-08-30T09:16:10.228 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "GRAFANA_API_URL", 2024-08-30T09:16:10.228 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.228 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.228 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.228 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:10.228 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.228 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.228 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.229 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.229 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.229 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.229 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.229 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.229 INFO:teuthology.orchestra.run.smithi042.stdout: "GRAFANA_API_USERNAME": { 2024-08-30T09:16:10.229 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "GRAFANA_API_USERNAME", 2024-08-30T09:16:10.229 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.229 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.229 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.229 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "admin", 2024-08-30T09:16:10.229 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.229 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.229 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.230 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.230 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.230 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.230 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.230 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.230 INFO:teuthology.orchestra.run.smithi042.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-08-30T09:16:10.230 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-08-30T09:16:10.230 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.230 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.230 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.230 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:10.230 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.230 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.230 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.231 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.231 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.231 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.231 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.231 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.231 INFO:teuthology.orchestra.run.smithi042.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-08-30T09:16:10.231 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-08-30T09:16:10.231 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.231 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.231 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.231 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:10.231 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.231 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.231 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.232 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.232 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.232 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.232 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.232 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.232 INFO:teuthology.orchestra.run.smithi042.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-08-30T09:16:10.232 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-08-30T09:16:10.232 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.232 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.232 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.232 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-08-30T09:16:10.232 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.232 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.232 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.233 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.233 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.233 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.233 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.233 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.233 INFO:teuthology.orchestra.run.smithi042.stdout: "ISSUE_TRACKER_API_KEY": { 2024-08-30T09:16:10.233 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-08-30T09:16:10.233 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.233 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.233 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.233 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:10.233 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.233 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.234 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.234 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.234 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.234 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.234 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.234 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.234 INFO:teuthology.orchestra.run.smithi042.stdout: "MANAGED_BY_CLUSTERS": { 2024-08-30T09:16:10.234 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-08-30T09:16:10.234 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.234 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.234 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.234 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "[]", 2024-08-30T09:16:10.234 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.234 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.235 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.235 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.235 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.235 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.235 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.235 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.235 INFO:teuthology.orchestra.run.smithi042.stdout: "MULTICLUSTER_CONFIG": { 2024-08-30T09:16:10.235 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "MULTICLUSTER_CONFIG", 2024-08-30T09:16:10.235 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.235 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.235 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.235 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "{}", 2024-08-30T09:16:10.235 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.235 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.236 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.236 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.236 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.236 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.236 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.236 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.236 INFO:teuthology.orchestra.run.smithi042.stdout: "PROMETHEUS_API_HOST": { 2024-08-30T09:16:10.236 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PROMETHEUS_API_HOST", 2024-08-30T09:16:10.236 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.236 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.236 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.236 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:10.236 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.236 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.237 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.237 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.237 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.237 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.237 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.237 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.237 INFO:teuthology.orchestra.run.smithi042.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-08-30T09:16:10.237 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-08-30T09:16:10.237 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.237 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.237 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.237 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-08-30T09:16:10.238 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.238 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.238 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.238 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.238 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.238 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.238 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.238 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.238 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-08-30T09:16:10.238 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-08-30T09:16:10.238 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.238 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.239 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.239 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:10.239 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.239 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.239 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.239 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.239 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.239 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.239 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.239 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.239 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-08-30T09:16:10.239 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-08-30T09:16:10.239 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.239 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.239 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.240 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:10.240 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.240 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.240 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.240 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.240 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.240 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.240 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.240 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.240 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-08-30T09:16:10.240 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-08-30T09:16:10.240 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.240 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.240 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.241 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-08-30T09:16:10.241 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.241 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.241 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.241 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.241 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.241 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.241 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.241 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.241 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-08-30T09:16:10.241 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-08-30T09:16:10.241 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.241 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.241 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.242 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-08-30T09:16:10.242 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.242 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.242 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.242 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.242 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.242 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.242 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.242 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.242 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-08-30T09:16:10.242 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-08-30T09:16:10.242 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.242 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.242 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.243 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:10.243 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.243 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.243 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.243 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.243 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.243 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.243 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.243 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.243 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-08-30T09:16:10.243 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-08-30T09:16:10.243 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.243 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.244 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.244 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:10.244 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.244 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.244 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.244 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.244 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.244 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.244 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.244 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.244 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-08-30T09:16:10.244 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-08-30T09:16:10.245 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.245 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.245 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.245 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:10.245 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.245 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.245 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.245 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.245 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.245 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.245 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.245 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.245 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_ENABLED": { 2024-08-30T09:16:10.245 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_ENABLED", 2024-08-30T09:16:10.245 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.246 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.246 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.246 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-08-30T09:16:10.246 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.246 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.246 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.246 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.246 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.246 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.246 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.246 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.246 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-08-30T09:16:10.246 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-08-30T09:16:10.246 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.247 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.247 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.247 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-08-30T09:16:10.247 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.247 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.247 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.247 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.247 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.247 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.247 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.247 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.247 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-08-30T09:16:10.248 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-08-30T09:16:10.248 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:10.248 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.248 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.248 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "10", 2024-08-30T09:16:10.248 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.248 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.248 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.248 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.248 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.248 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.248 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.248 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.249 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-08-30T09:16:10.249 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-08-30T09:16:10.249 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:10.249 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.249 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.249 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "8", 2024-08-30T09:16:10.249 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.249 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.249 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.249 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.249 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.249 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.250 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.250 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.250 INFO:teuthology.orchestra.run.smithi042.stdout: "REST_REQUESTS_TIMEOUT": { 2024-08-30T09:16:10.250 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-08-30T09:16:10.250 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:10.250 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.250 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.250 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "45", 2024-08-30T09:16:10.250 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.250 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.250 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.250 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.250 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.250 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.251 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.251 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.251 INFO:teuthology.orchestra.run.smithi042.stdout: "RGW_API_ACCESS_KEY": { 2024-08-30T09:16:10.251 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "RGW_API_ACCESS_KEY", 2024-08-30T09:16:10.251 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.251 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.251 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.251 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:10.251 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.251 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.251 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.251 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.251 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.251 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.252 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.252 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.252 INFO:teuthology.orchestra.run.smithi042.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-08-30T09:16:10.252 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-08-30T09:16:10.252 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.252 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.252 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.252 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "admin", 2024-08-30T09:16:10.252 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.252 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.252 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.252 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.252 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.253 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.253 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.253 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.253 INFO:teuthology.orchestra.run.smithi042.stdout: "RGW_API_SECRET_KEY": { 2024-08-30T09:16:10.253 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "RGW_API_SECRET_KEY", 2024-08-30T09:16:10.253 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.253 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.253 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.253 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:10.253 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.253 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.253 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.253 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.253 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.253 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.254 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.254 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.254 INFO:teuthology.orchestra.run.smithi042.stdout: "RGW_API_SSL_VERIFY": { 2024-08-30T09:16:10.254 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "RGW_API_SSL_VERIFY", 2024-08-30T09:16:10.254 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.254 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.254 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.254 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-08-30T09:16:10.254 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.254 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.254 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.254 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.254 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.254 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.255 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.255 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.255 INFO:teuthology.orchestra.run.smithi042.stdout: "UNSAFE_TLS_v1_2": { 2024-08-30T09:16:10.255 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "UNSAFE_TLS_v1_2", 2024-08-30T09:16:10.255 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.255 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.255 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.255 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:10.255 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.255 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.255 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.255 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.255 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.255 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.255 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.256 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.256 INFO:teuthology.orchestra.run.smithi042.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-08-30T09:16:10.256 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-08-30T09:16:10.256 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:10.256 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.256 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.256 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-08-30T09:16:10.256 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.256 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.256 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.256 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.256 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.256 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.256 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.257 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.257 INFO:teuthology.orchestra.run.smithi042.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-08-30T09:16:10.257 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-08-30T09:16:10.257 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:10.257 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.257 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.257 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "10", 2024-08-30T09:16:10.257 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.257 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.257 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.257 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.257 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.257 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.257 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.258 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.258 INFO:teuthology.orchestra.run.smithi042.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-08-30T09:16:10.258 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-08-30T09:16:10.258 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:10.258 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.258 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.258 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "5", 2024-08-30T09:16:10.258 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.258 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.258 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.258 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.258 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.258 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.259 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.259 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.259 INFO:teuthology.orchestra.run.smithi042.stdout: "cross_origin_url": { 2024-08-30T09:16:10.259 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "cross_origin_url", 2024-08-30T09:16:10.259 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.259 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.259 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.259 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:10.259 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.259 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.259 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.259 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.259 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.260 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.260 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.260 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.260 INFO:teuthology.orchestra.run.smithi042.stdout: "crt_file": { 2024-08-30T09:16:10.260 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "crt_file", 2024-08-30T09:16:10.260 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.260 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.260 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.260 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:10.260 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.260 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.260 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.260 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.260 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.260 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.261 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.261 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.261 INFO:teuthology.orchestra.run.smithi042.stdout: "debug": { 2024-08-30T09:16:10.261 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "debug", 2024-08-30T09:16:10.261 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.261 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.261 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.261 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:10.261 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.261 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.261 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.261 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Enable/disable debug options", 2024-08-30T09:16:10.261 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.261 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.262 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.262 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.262 INFO:teuthology.orchestra.run.smithi042.stdout: "jwt_token_ttl": { 2024-08-30T09:16:10.262 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "jwt_token_ttl", 2024-08-30T09:16:10.262 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:10.262 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.262 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.262 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "28800", 2024-08-30T09:16:10.262 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.262 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.262 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.262 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.262 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.262 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.263 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.263 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.263 INFO:teuthology.orchestra.run.smithi042.stdout: "key_file": { 2024-08-30T09:16:10.263 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "key_file", 2024-08-30T09:16:10.263 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.263 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.263 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.263 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:10.263 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.263 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.263 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.263 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.263 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.263 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.264 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.264 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.264 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-08-30T09:16:10.264 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-08-30T09:16:10.264 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.264 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.264 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.264 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:10.264 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.264 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.264 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:10.264 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:10.264 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:10.264 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:10.265 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:10.265 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:10.265 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:10.265 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:10.265 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.265 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.265 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.265 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.265 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.265 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-08-30T09:16:10.265 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-08-30T09:16:10.265 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.265 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.265 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.266 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:10.266 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.266 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.266 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.266 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.266 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.266 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.266 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.266 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.266 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-08-30T09:16:10.266 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-08-30T09:16:10.266 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.266 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.266 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.267 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-08-30T09:16:10.267 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.267 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.267 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:10.267 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:10.267 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:10.267 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:10.267 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:10.267 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:10.267 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:10.267 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:10.267 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.267 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.268 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.268 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.268 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.268 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-08-30T09:16:10.268 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-08-30T09:16:10.268 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.268 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.268 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.268 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:10.268 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.268 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.268 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.268 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.268 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.269 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.269 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.269 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.269 INFO:teuthology.orchestra.run.smithi042.stdout: "motd": { 2024-08-30T09:16:10.269 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "motd", 2024-08-30T09:16:10.269 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.269 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.269 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.269 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:10.269 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.269 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.269 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.269 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "The message of the day", 2024-08-30T09:16:10.269 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.269 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.270 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.270 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.270 INFO:teuthology.orchestra.run.smithi042.stdout: "redirect_resolve_ip_addr": { 2024-08-30T09:16:10.270 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "redirect_resolve_ip_addr", 2024-08-30T09:16:10.270 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.270 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.270 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.270 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:10.270 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.270 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.270 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.270 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.270 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.270 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.270 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.271 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.271 INFO:teuthology.orchestra.run.smithi042.stdout: "server_addr": { 2024-08-30T09:16:10.271 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "server_addr", 2024-08-30T09:16:10.271 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.271 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.271 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.271 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "::", 2024-08-30T09:16:10.271 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.271 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.271 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.271 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.271 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.271 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.272 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.272 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.272 INFO:teuthology.orchestra.run.smithi042.stdout: "server_port": { 2024-08-30T09:16:10.272 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "server_port", 2024-08-30T09:16:10.272 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:10.272 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.272 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.272 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "8080", 2024-08-30T09:16:10.272 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.272 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.272 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.272 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.272 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.272 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.273 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.273 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.273 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-08-30T09:16:10.273 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-08-30T09:16:10.273 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:10.273 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-08-30T09:16:10.273 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.273 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-08-30T09:16:10.273 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.273 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.273 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.273 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.273 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.273 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.274 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.274 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.274 INFO:teuthology.orchestra.run.smithi042.stdout: "ssl": { 2024-08-30T09:16:10.274 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ssl", 2024-08-30T09:16:10.274 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.274 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.274 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.274 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-08-30T09:16:10.274 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.274 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.274 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.274 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.274 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.274 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.275 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.275 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.275 INFO:teuthology.orchestra.run.smithi042.stdout: "ssl_server_port": { 2024-08-30T09:16:10.275 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ssl_server_port", 2024-08-30T09:16:10.275 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:10.275 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.275 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.275 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "8443", 2024-08-30T09:16:10.275 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.275 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.275 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.275 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.275 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.275 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.275 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.276 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.276 INFO:teuthology.orchestra.run.smithi042.stdout: "standby_behaviour": { 2024-08-30T09:16:10.276 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "standby_behaviour", 2024-08-30T09:16:10.276 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.276 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.276 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.276 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "redirect", 2024-08-30T09:16:10.276 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.276 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.276 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:10.276 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:10.276 INFO:teuthology.orchestra.run.smithi042.stdout: "redirect" 2024-08-30T09:16:10.277 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:10.277 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.277 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.277 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.277 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.277 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.277 INFO:teuthology.orchestra.run.smithi042.stdout: "standby_error_status_code": { 2024-08-30T09:16:10.277 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "standby_error_status_code", 2024-08-30T09:16:10.277 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:10.277 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.277 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.277 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "500", 2024-08-30T09:16:10.277 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "400", 2024-08-30T09:16:10.277 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "599", 2024-08-30T09:16:10.277 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.278 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.278 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.278 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.278 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.278 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.278 INFO:teuthology.orchestra.run.smithi042.stdout: "url_prefix": { 2024-08-30T09:16:10.278 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "url_prefix", 2024-08-30T09:16:10.278 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.278 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.278 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.278 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:10.278 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.278 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.279 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.279 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.279 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.279 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.279 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.279 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:10.279 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:10.279 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.279 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-08-30T09:16:10.279 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "devicehealth", 2024-08-30T09:16:10.279 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-08-30T09:16:10.279 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-08-30T09:16:10.279 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-08-30T09:16:10.280 INFO:teuthology.orchestra.run.smithi042.stdout: "enable_monitoring": { 2024-08-30T09:16:10.280 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "enable_monitoring", 2024-08-30T09:16:10.280 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.280 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.280 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.280 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-08-30T09:16:10.280 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.280 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.280 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.280 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "monitor device health metrics", 2024-08-30T09:16:10.280 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.280 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.280 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.280 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.281 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-08-30T09:16:10.281 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-08-30T09:16:10.281 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.281 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.281 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.281 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:10.281 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.281 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.281 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:10.281 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:10.281 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:10.281 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:10.281 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:10.282 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:10.282 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:10.282 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:10.282 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.282 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.282 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.282 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.282 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.282 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-08-30T09:16:10.282 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-08-30T09:16:10.282 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.282 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.282 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.283 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:10.283 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.283 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.283 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.283 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.283 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.283 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.283 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.283 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.283 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-08-30T09:16:10.283 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-08-30T09:16:10.283 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.283 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.284 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.284 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-08-30T09:16:10.284 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.284 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.284 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:10.284 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:10.284 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:10.284 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:10.284 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:10.284 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:10.285 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:10.285 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:10.285 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.285 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.285 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.285 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.285 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.285 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-08-30T09:16:10.285 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-08-30T09:16:10.285 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.285 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.285 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.286 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:10.286 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.286 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.286 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.286 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.286 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.286 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.286 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.286 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.286 INFO:teuthology.orchestra.run.smithi042.stdout: "mark_out_threshold": { 2024-08-30T09:16:10.286 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "mark_out_threshold", 2024-08-30T09:16:10.286 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-08-30T09:16:10.286 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.287 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.287 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "2419200", 2024-08-30T09:16:10.287 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.287 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.287 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.287 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-08-30T09:16:10.287 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.287 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.287 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.287 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.287 INFO:teuthology.orchestra.run.smithi042.stdout: "pool_name": { 2024-08-30T09:16:10.287 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "pool_name", 2024-08-30T09:16:10.287 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.287 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.288 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.288 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "device_health_metrics", 2024-08-30T09:16:10.288 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.288 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.288 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.288 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "name of pool in which to store device health metrics", 2024-08-30T09:16:10.288 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.288 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.288 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.288 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.288 INFO:teuthology.orchestra.run.smithi042.stdout: "retention_period": { 2024-08-30T09:16:10.288 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "retention_period", 2024-08-30T09:16:10.289 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-08-30T09:16:10.289 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.289 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.289 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "15552000", 2024-08-30T09:16:10.289 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.289 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.289 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.289 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "how long to retain device health metrics", 2024-08-30T09:16:10.289 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.289 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.289 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.289 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.289 INFO:teuthology.orchestra.run.smithi042.stdout: "scrape_frequency": { 2024-08-30T09:16:10.289 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "scrape_frequency", 2024-08-30T09:16:10.290 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-08-30T09:16:10.290 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.290 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.290 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "86400", 2024-08-30T09:16:10.290 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.290 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.290 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.290 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "how frequently to scrape device health metrics", 2024-08-30T09:16:10.290 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.290 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.290 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.290 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.290 INFO:teuthology.orchestra.run.smithi042.stdout: "self_heal": { 2024-08-30T09:16:10.290 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "self_heal", 2024-08-30T09:16:10.291 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.291 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.291 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.291 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-08-30T09:16:10.291 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.291 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.291 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.291 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-08-30T09:16:10.291 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.291 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.291 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.291 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.291 INFO:teuthology.orchestra.run.smithi042.stdout: "sleep_interval": { 2024-08-30T09:16:10.291 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sleep_interval", 2024-08-30T09:16:10.291 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-08-30T09:16:10.292 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.292 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.292 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "600", 2024-08-30T09:16:10.292 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.292 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.292 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.292 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "how frequently to wake up and check device health", 2024-08-30T09:16:10.292 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.292 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.292 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.292 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.292 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-08-30T09:16:10.292 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-08-30T09:16:10.292 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:10.293 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-08-30T09:16:10.293 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.293 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-08-30T09:16:10.293 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.293 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.293 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.293 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.293 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.293 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.293 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.293 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.293 INFO:teuthology.orchestra.run.smithi042.stdout: "warn_threshold": { 2024-08-30T09:16:10.293 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "warn_threshold", 2024-08-30T09:16:10.293 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-08-30T09:16:10.294 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.294 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.294 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "7257600", 2024-08-30T09:16:10.294 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.294 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.294 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.294 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-08-30T09:16:10.294 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.294 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.294 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.294 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:10.294 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:10.294 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.294 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-08-30T09:16:10.295 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "diskprediction_local", 2024-08-30T09:16:10.295 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-08-30T09:16:10.295 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-08-30T09:16:10.295 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-08-30T09:16:10.295 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-08-30T09:16:10.295 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-08-30T09:16:10.295 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.295 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.295 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.295 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:10.295 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.295 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.295 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:10.296 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:10.296 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:10.296 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:10.296 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:10.296 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:10.296 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:10.296 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:10.296 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.296 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.296 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.296 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.296 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.296 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-08-30T09:16:10.297 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-08-30T09:16:10.297 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.297 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.297 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.297 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:10.297 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.297 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.297 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.297 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.297 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.297 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.297 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.297 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.297 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-08-30T09:16:10.298 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-08-30T09:16:10.298 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.298 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.298 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.298 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-08-30T09:16:10.298 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.298 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.298 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:10.298 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:10.298 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:10.298 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:10.298 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:10.298 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:10.299 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:10.299 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:10.299 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.299 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.299 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.299 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.299 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.299 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-08-30T09:16:10.299 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-08-30T09:16:10.299 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.299 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.299 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.300 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:10.300 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.300 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.300 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.300 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.300 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.300 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.300 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.300 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.300 INFO:teuthology.orchestra.run.smithi042.stdout: "predict_interval": { 2024-08-30T09:16:10.300 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "predict_interval", 2024-08-30T09:16:10.300 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.300 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.301 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.301 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "86400", 2024-08-30T09:16:10.301 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.301 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.301 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.301 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.301 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.301 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.301 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.301 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.301 INFO:teuthology.orchestra.run.smithi042.stdout: "predictor_model": { 2024-08-30T09:16:10.301 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "predictor_model", 2024-08-30T09:16:10.302 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.302 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.302 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.302 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "prophetstor", 2024-08-30T09:16:10.302 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.302 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.302 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.302 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.302 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.302 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.302 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.302 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.302 INFO:teuthology.orchestra.run.smithi042.stdout: "sleep_interval": { 2024-08-30T09:16:10.303 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sleep_interval", 2024-08-30T09:16:10.303 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.303 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.303 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.303 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "600", 2024-08-30T09:16:10.303 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.303 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.303 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.303 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.303 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.303 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.303 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.303 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.303 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-08-30T09:16:10.304 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-08-30T09:16:10.304 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:10.304 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-08-30T09:16:10.304 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.304 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-08-30T09:16:10.304 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.304 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.304 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.304 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.304 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.304 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.304 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.304 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:10.305 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:10.305 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.305 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-08-30T09:16:10.305 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "influx", 2024-08-30T09:16:10.305 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": false, 2024-08-30T09:16:10.305 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "influxdb python module not found", 2024-08-30T09:16:10.305 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-08-30T09:16:10.305 INFO:teuthology.orchestra.run.smithi042.stdout: "batch_size": { 2024-08-30T09:16:10.305 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "batch_size", 2024-08-30T09:16:10.305 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:10.305 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.305 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.305 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "5000", 2024-08-30T09:16:10.306 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.306 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.306 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.306 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-08-30T09:16:10.306 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.306 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.306 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.306 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.306 INFO:teuthology.orchestra.run.smithi042.stdout: "database": { 2024-08-30T09:16:10.306 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "database", 2024-08-30T09:16:10.306 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.306 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.306 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.307 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "ceph", 2024-08-30T09:16:10.307 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.307 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.307 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.307 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-08-30T09:16:10.307 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.307 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.307 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.307 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.307 INFO:teuthology.orchestra.run.smithi042.stdout: "hostname": { 2024-08-30T09:16:10.307 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "hostname", 2024-08-30T09:16:10.307 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.307 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.308 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.308 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:10.308 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.308 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.308 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.308 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "InfluxDB server hostname", 2024-08-30T09:16:10.308 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.308 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.308 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.308 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.308 INFO:teuthology.orchestra.run.smithi042.stdout: "interval": { 2024-08-30T09:16:10.308 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "interval", 2024-08-30T09:16:10.309 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-08-30T09:16:10.309 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.309 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.309 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "30", 2024-08-30T09:16:10.309 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "5", 2024-08-30T09:16:10.309 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.309 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.309 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-08-30T09:16:10.309 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.309 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.309 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.309 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.309 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-08-30T09:16:10.310 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-08-30T09:16:10.310 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.310 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.310 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.310 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:10.310 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.310 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.310 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:10.310 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:10.310 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:10.310 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:10.310 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:10.310 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:10.311 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:10.311 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:10.311 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.311 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.311 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.311 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.311 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.311 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-08-30T09:16:10.311 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-08-30T09:16:10.311 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.311 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.311 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.311 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:10.312 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.312 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.312 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.312 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.312 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.312 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.312 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.312 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.312 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-08-30T09:16:10.312 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-08-30T09:16:10.312 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.312 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.312 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.312 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-08-30T09:16:10.313 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.313 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.313 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:10.313 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:10.313 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:10.313 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:10.313 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:10.313 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:10.313 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:10.313 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:10.313 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.313 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.313 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.314 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.314 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.314 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-08-30T09:16:10.314 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-08-30T09:16:10.314 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.314 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.314 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.314 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:10.314 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.314 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.314 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.314 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.314 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.314 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.315 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.315 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.315 INFO:teuthology.orchestra.run.smithi042.stdout: "password": { 2024-08-30T09:16:10.315 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "password", 2024-08-30T09:16:10.315 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.315 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.315 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.315 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:10.315 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.315 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.315 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.315 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "password of InfluxDB server user", 2024-08-30T09:16:10.315 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.316 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.316 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.316 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.316 INFO:teuthology.orchestra.run.smithi042.stdout: "port": { 2024-08-30T09:16:10.316 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "port", 2024-08-30T09:16:10.316 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:10.316 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.316 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.316 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "8086", 2024-08-30T09:16:10.316 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.316 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.316 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.316 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "InfluxDB server port", 2024-08-30T09:16:10.317 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.317 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.317 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.317 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.317 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-08-30T09:16:10.317 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-08-30T09:16:10.317 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:10.317 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-08-30T09:16:10.317 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.317 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-08-30T09:16:10.317 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.317 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.317 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.318 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.318 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.318 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.318 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.318 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.318 INFO:teuthology.orchestra.run.smithi042.stdout: "ssl": { 2024-08-30T09:16:10.318 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ssl", 2024-08-30T09:16:10.318 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.318 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.318 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.318 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "false", 2024-08-30T09:16:10.318 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.318 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.319 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.319 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-08-30T09:16:10.319 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.319 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.319 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.319 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.319 INFO:teuthology.orchestra.run.smithi042.stdout: "threads": { 2024-08-30T09:16:10.319 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "threads", 2024-08-30T09:16:10.319 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:10.319 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.319 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.319 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "5", 2024-08-30T09:16:10.319 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "1", 2024-08-30T09:16:10.320 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "32", 2024-08-30T09:16:10.320 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.320 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-08-30T09:16:10.320 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.320 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.320 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.320 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.320 INFO:teuthology.orchestra.run.smithi042.stdout: "username": { 2024-08-30T09:16:10.320 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "username", 2024-08-30T09:16:10.320 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.320 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.320 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.320 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:10.321 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.321 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.321 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.321 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "username of InfluxDB server user", 2024-08-30T09:16:10.321 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.321 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.321 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.321 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.321 INFO:teuthology.orchestra.run.smithi042.stdout: "verify_ssl": { 2024-08-30T09:16:10.321 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "verify_ssl", 2024-08-30T09:16:10.321 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.321 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.321 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.322 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "true", 2024-08-30T09:16:10.322 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.322 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.322 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.322 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-08-30T09:16:10.322 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.322 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.322 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.322 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:10.322 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:10.322 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.322 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-08-30T09:16:10.323 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "insights", 2024-08-30T09:16:10.323 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-08-30T09:16:10.323 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-08-30T09:16:10.323 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-08-30T09:16:10.323 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-08-30T09:16:10.323 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-08-30T09:16:10.323 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.323 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.323 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.323 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:10.323 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.323 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.323 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:10.324 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:10.324 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:10.324 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:10.324 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:10.324 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:10.324 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:10.324 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:10.324 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.324 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.324 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.324 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.324 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.324 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-08-30T09:16:10.324 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-08-30T09:16:10.325 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.325 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.325 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.325 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:10.325 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.325 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.325 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.325 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.325 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.325 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.325 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.325 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.325 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-08-30T09:16:10.325 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-08-30T09:16:10.326 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.326 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.326 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.326 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-08-30T09:16:10.326 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.326 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.326 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:10.326 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:10.326 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:10.326 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:10.326 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:10.326 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:10.326 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:10.326 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:10.327 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.327 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.327 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.327 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.327 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.327 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-08-30T09:16:10.327 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-08-30T09:16:10.327 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.327 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.327 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.327 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:10.327 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.327 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.328 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.328 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.328 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.328 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.328 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.328 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.328 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-08-30T09:16:10.328 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-08-30T09:16:10.328 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:10.328 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-08-30T09:16:10.328 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.328 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-08-30T09:16:10.328 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.329 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.329 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.329 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.329 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.329 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.329 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.329 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:10.329 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:10.329 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.329 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-08-30T09:16:10.329 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "iostat", 2024-08-30T09:16:10.329 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-08-30T09:16:10.329 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-08-30T09:16:10.330 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-08-30T09:16:10.330 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-08-30T09:16:10.330 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-08-30T09:16:10.330 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.330 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.330 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.330 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:10.330 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.330 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.330 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:10.330 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:10.330 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:10.330 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:10.330 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:10.331 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:10.331 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:10.331 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:10.331 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.331 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.331 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.331 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.331 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.331 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-08-30T09:16:10.331 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-08-30T09:16:10.331 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.331 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.331 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.331 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:10.331 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.332 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.332 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.332 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.332 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.332 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.332 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.332 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.332 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-08-30T09:16:10.332 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-08-30T09:16:10.332 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.332 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.332 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.332 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-08-30T09:16:10.332 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.333 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.333 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:10.333 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:10.333 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:10.333 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:10.333 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:10.333 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:10.333 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:10.333 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:10.333 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.333 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.333 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.333 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.333 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.334 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-08-30T09:16:10.334 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-08-30T09:16:10.334 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.334 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.334 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.334 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:10.334 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.334 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.334 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.334 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.334 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.334 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.334 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.334 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.334 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-08-30T09:16:10.335 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-08-30T09:16:10.335 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:10.335 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-08-30T09:16:10.335 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.335 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-08-30T09:16:10.335 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.335 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.335 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.335 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.335 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.335 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.335 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.335 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:10.336 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:10.336 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.336 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-08-30T09:16:10.336 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "k8sevents", 2024-08-30T09:16:10.336 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-08-30T09:16:10.336 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-08-30T09:16:10.336 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-08-30T09:16:10.336 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph_event_retention_days": { 2024-08-30T09:16:10.336 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ceph_event_retention_days", 2024-08-30T09:16:10.336 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:10.336 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.336 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.336 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "7", 2024-08-30T09:16:10.337 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.337 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.337 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.337 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Days to hold ceph event information within local cache", 2024-08-30T09:16:10.337 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.337 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.337 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.337 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.337 INFO:teuthology.orchestra.run.smithi042.stdout: "config_check_secs": { 2024-08-30T09:16:10.337 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "config_check_secs", 2024-08-30T09:16:10.337 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:10.337 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.338 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.338 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "10", 2024-08-30T09:16:10.338 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "10", 2024-08-30T09:16:10.338 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.338 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.338 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-08-30T09:16:10.338 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.338 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.338 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.338 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.338 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-08-30T09:16:10.338 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-08-30T09:16:10.338 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.338 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.339 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.339 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:10.339 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.339 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.339 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:10.339 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:10.339 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:10.339 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:10.339 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:10.339 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:10.339 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:10.339 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:10.339 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.339 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.340 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.340 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.340 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.340 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-08-30T09:16:10.340 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-08-30T09:16:10.340 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.340 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.340 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.340 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:10.340 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.340 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.340 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.340 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.341 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.341 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.341 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.341 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.341 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-08-30T09:16:10.341 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-08-30T09:16:10.341 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.341 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.342 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.342 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-08-30T09:16:10.342 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.342 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.342 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:10.342 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:10.342 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:10.342 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:10.342 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:10.342 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:10.342 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:10.342 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:10.342 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.343 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.343 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.343 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.343 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.343 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-08-30T09:16:10.343 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-08-30T09:16:10.343 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.343 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.343 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.343 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:10.343 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.343 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.343 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.344 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.344 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.344 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.344 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.344 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.344 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-08-30T09:16:10.344 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-08-30T09:16:10.344 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:10.344 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-08-30T09:16:10.344 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.344 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-08-30T09:16:10.344 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.344 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.344 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.345 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.345 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.345 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.345 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.345 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:10.345 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:10.345 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.345 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-08-30T09:16:10.345 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "localpool", 2024-08-30T09:16:10.345 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-08-30T09:16:10.345 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-08-30T09:16:10.345 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-08-30T09:16:10.345 INFO:teuthology.orchestra.run.smithi042.stdout: "failure_domain": { 2024-08-30T09:16:10.345 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "failure_domain", 2024-08-30T09:16:10.346 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.346 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.346 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.346 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "host", 2024-08-30T09:16:10.346 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.346 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.346 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.346 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "failure domain for any created local pool", 2024-08-30T09:16:10.346 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-08-30T09:16:10.346 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.346 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.346 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.346 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-08-30T09:16:10.346 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-08-30T09:16:10.346 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.347 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.347 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.347 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:10.347 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.347 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.347 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:10.347 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:10.347 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:10.347 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:10.347 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:10.347 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:10.347 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:10.347 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:10.347 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.348 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.348 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.348 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.348 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.348 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-08-30T09:16:10.348 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-08-30T09:16:10.348 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.348 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.348 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.348 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:10.348 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.348 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.348 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.349 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.349 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.349 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.349 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.349 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.349 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-08-30T09:16:10.349 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-08-30T09:16:10.349 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.349 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.349 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.349 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-08-30T09:16:10.349 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.349 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.349 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:10.350 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:10.350 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:10.350 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:10.350 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:10.350 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:10.350 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:10.350 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:10.350 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.350 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.350 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.350 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.350 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.350 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-08-30T09:16:10.350 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-08-30T09:16:10.351 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.351 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.351 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.351 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:10.351 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.351 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.351 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.351 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.351 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.351 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.351 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.351 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.351 INFO:teuthology.orchestra.run.smithi042.stdout: "min_size": { 2024-08-30T09:16:10.351 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "min_size", 2024-08-30T09:16:10.351 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:10.352 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.352 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.352 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:10.352 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.352 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.352 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.352 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "default min_size for any created local pool", 2024-08-30T09:16:10.352 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-08-30T09:16:10.352 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.352 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.352 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.352 INFO:teuthology.orchestra.run.smithi042.stdout: "num_rep": { 2024-08-30T09:16:10.352 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "num_rep", 2024-08-30T09:16:10.353 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:10.353 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.353 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.353 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "3", 2024-08-30T09:16:10.353 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.353 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.353 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.353 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "default replica count for any created local pool", 2024-08-30T09:16:10.353 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.353 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.353 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.353 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.353 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num": { 2024-08-30T09:16:10.353 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "pg_num", 2024-08-30T09:16:10.354 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:10.354 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.354 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.354 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "128", 2024-08-30T09:16:10.354 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.354 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.354 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.354 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "default pg_num for any created local pool", 2024-08-30T09:16:10.354 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.354 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.354 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.354 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.354 INFO:teuthology.orchestra.run.smithi042.stdout: "prefix": { 2024-08-30T09:16:10.354 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "prefix", 2024-08-30T09:16:10.354 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.355 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.355 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.355 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:10.355 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.355 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.355 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.355 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "name prefix for any created local pool", 2024-08-30T09:16:10.355 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.355 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.355 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.355 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.355 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-08-30T09:16:10.355 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-08-30T09:16:10.356 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:10.356 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-08-30T09:16:10.356 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.356 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-08-30T09:16:10.356 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.356 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.356 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.356 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.356 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.356 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.356 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.356 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.356 INFO:teuthology.orchestra.run.smithi042.stdout: "subtree": { 2024-08-30T09:16:10.356 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "subtree", 2024-08-30T09:16:10.357 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.357 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.357 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.357 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "rack", 2024-08-30T09:16:10.357 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.357 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.357 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.357 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "CRUSH level for which to create a local pool", 2024-08-30T09:16:10.357 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-08-30T09:16:10.357 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.357 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.357 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:10.357 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:10.358 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.358 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-08-30T09:16:10.358 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "mds_autoscaler", 2024-08-30T09:16:10.358 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-08-30T09:16:10.358 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-08-30T09:16:10.358 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-08-30T09:16:10.358 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-08-30T09:16:10.358 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-08-30T09:16:10.358 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.358 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.358 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.358 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:10.358 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.358 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.359 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:10.359 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:10.359 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:10.359 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:10.359 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:10.359 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:10.359 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:10.359 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:10.359 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.359 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.359 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.359 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.359 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.360 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-08-30T09:16:10.360 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-08-30T09:16:10.360 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.360 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.360 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.360 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:10.360 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.360 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.360 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.360 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.360 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.360 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.360 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.360 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.361 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-08-30T09:16:10.361 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-08-30T09:16:10.361 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.361 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.361 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.361 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-08-30T09:16:10.361 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.361 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.361 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:10.361 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:10.361 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:10.361 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:10.361 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:10.361 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:10.362 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:10.362 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:10.362 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.362 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.362 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.362 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.362 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.362 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-08-30T09:16:10.362 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-08-30T09:16:10.362 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.362 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.362 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.362 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:10.362 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.363 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.363 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.363 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.363 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.363 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.363 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.363 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.363 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-08-30T09:16:10.363 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-08-30T09:16:10.363 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:10.363 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-08-30T09:16:10.363 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.363 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-08-30T09:16:10.363 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.363 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.364 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.364 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.364 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.364 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.364 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.364 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:10.364 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:10.364 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.364 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-08-30T09:16:10.364 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "mirroring", 2024-08-30T09:16:10.364 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-08-30T09:16:10.364 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-08-30T09:16:10.365 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-08-30T09:16:10.365 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-08-30T09:16:10.365 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-08-30T09:16:10.365 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.365 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.365 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.365 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:10.365 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.365 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.365 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:10.365 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:10.365 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:10.365 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:10.365 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:10.366 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:10.366 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:10.366 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:10.366 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.366 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.366 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.366 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.366 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.366 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-08-30T09:16:10.366 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-08-30T09:16:10.366 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.366 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.366 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.366 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:10.366 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.367 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.367 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.367 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.367 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.367 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.367 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.367 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.367 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-08-30T09:16:10.367 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-08-30T09:16:10.367 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.367 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.367 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.367 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-08-30T09:16:10.367 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.368 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.368 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:10.368 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:10.368 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:10.368 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:10.368 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:10.368 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:10.368 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:10.368 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:10.368 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.368 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.368 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.368 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.369 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.369 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-08-30T09:16:10.369 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-08-30T09:16:10.369 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.369 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.369 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.369 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:10.369 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.369 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.369 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.369 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.369 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.369 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.369 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.370 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.370 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-08-30T09:16:10.370 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-08-30T09:16:10.370 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:10.370 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-08-30T09:16:10.370 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.370 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-08-30T09:16:10.370 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.370 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.370 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.370 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.370 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.370 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.370 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.370 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:10.371 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:10.371 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.371 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-08-30T09:16:10.371 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "nfs", 2024-08-30T09:16:10.371 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-08-30T09:16:10.371 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-08-30T09:16:10.371 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-08-30T09:16:10.371 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-08-30T09:16:10.371 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-08-30T09:16:10.371 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.371 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.371 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.372 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:10.372 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.372 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.372 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:10.372 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:10.372 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:10.372 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:10.372 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:10.372 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:10.372 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:10.372 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:10.372 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.372 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.373 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.373 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.373 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.373 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-08-30T09:16:10.373 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-08-30T09:16:10.373 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.373 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.373 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.373 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:10.373 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.373 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.373 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.373 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.373 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.374 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.374 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.374 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.374 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-08-30T09:16:10.374 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-08-30T09:16:10.374 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.374 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.374 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.374 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-08-30T09:16:10.374 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.374 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.374 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:10.374 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:10.374 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:10.375 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:10.375 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:10.375 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:10.375 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:10.375 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:10.375 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.375 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.375 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.375 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.375 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.375 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-08-30T09:16:10.375 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-08-30T09:16:10.375 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.375 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.376 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.376 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:10.376 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.376 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.376 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.376 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.376 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.376 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.376 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.376 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.376 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-08-30T09:16:10.376 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-08-30T09:16:10.376 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:10.376 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-08-30T09:16:10.377 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.377 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-08-30T09:16:10.377 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.377 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.377 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.377 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.377 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.377 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.377 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.377 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:10.377 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:10.377 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.377 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-08-30T09:16:10.378 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "orchestrator", 2024-08-30T09:16:10.378 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-08-30T09:16:10.378 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-08-30T09:16:10.378 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-08-30T09:16:10.378 INFO:teuthology.orchestra.run.smithi042.stdout: "fail_fs": { 2024-08-30T09:16:10.378 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "fail_fs", 2024-08-30T09:16:10.378 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.378 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.378 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.378 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:10.378 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.378 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.378 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.378 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-08-30T09:16:10.379 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.379 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.379 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.379 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.379 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-08-30T09:16:10.379 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-08-30T09:16:10.379 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.379 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.379 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.379 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:10.379 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.379 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.379 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:10.379 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:10.379 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:10.380 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:10.380 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:10.380 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:10.380 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:10.380 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:10.380 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.380 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.380 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.380 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.380 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.380 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-08-30T09:16:10.380 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-08-30T09:16:10.380 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.380 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.381 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.381 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:10.381 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.381 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.381 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.381 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.381 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.381 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.381 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.381 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.381 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-08-30T09:16:10.381 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-08-30T09:16:10.381 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.382 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.382 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.382 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-08-30T09:16:10.382 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.382 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.382 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:10.382 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:10.382 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:10.382 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:10.382 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:10.382 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:10.382 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:10.382 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:10.383 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.383 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.383 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.383 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.383 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.383 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-08-30T09:16:10.383 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-08-30T09:16:10.383 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.383 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.383 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.383 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:10.383 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.383 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.383 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.384 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.384 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.384 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.384 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.384 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.384 INFO:teuthology.orchestra.run.smithi042.stdout: "orchestrator": { 2024-08-30T09:16:10.384 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "orchestrator", 2024-08-30T09:16:10.384 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.384 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.384 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.384 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:10.384 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.384 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.384 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:10.385 INFO:teuthology.orchestra.run.smithi042.stdout: "cephadm", 2024-08-30T09:16:10.385 INFO:teuthology.orchestra.run.smithi042.stdout: "rook", 2024-08-30T09:16:10.385 INFO:teuthology.orchestra.run.smithi042.stdout: "test_orchestrator" 2024-08-30T09:16:10.385 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:10.385 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Orchestrator backend", 2024-08-30T09:16:10.385 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.385 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.385 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.385 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.385 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-08-30T09:16:10.385 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-08-30T09:16:10.385 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:10.385 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-08-30T09:16:10.386 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.386 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-08-30T09:16:10.386 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.386 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.386 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.386 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.386 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.386 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.386 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.386 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:10.386 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:10.386 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.387 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-08-30T09:16:10.387 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "osd_perf_query", 2024-08-30T09:16:10.387 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-08-30T09:16:10.387 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-08-30T09:16:10.387 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-08-30T09:16:10.387 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-08-30T09:16:10.387 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-08-30T09:16:10.387 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.387 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.387 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.387 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:10.387 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.387 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.387 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:10.387 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:10.388 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:10.388 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:10.388 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:10.388 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:10.388 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:10.388 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:10.388 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.388 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.388 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.388 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.388 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.388 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-08-30T09:16:10.388 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-08-30T09:16:10.389 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.389 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.389 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.389 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:10.389 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.389 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.389 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.389 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.389 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.389 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.389 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.389 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.390 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-08-30T09:16:10.390 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-08-30T09:16:10.390 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.390 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.390 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.390 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-08-30T09:16:10.390 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.390 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.390 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:10.390 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:10.390 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:10.390 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:10.390 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:10.390 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:10.390 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:10.391 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:10.391 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.391 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.391 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.391 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.391 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.391 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-08-30T09:16:10.391 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-08-30T09:16:10.391 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.391 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.391 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.391 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:10.391 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.391 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.391 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.392 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.392 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.392 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.392 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.392 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.392 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-08-30T09:16:10.392 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-08-30T09:16:10.392 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:10.392 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-08-30T09:16:10.392 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.392 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-08-30T09:16:10.392 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.392 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.393 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.393 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.393 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.393 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.393 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.393 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:10.393 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:10.393 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.393 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-08-30T09:16:10.393 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "osd_support", 2024-08-30T09:16:10.393 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-08-30T09:16:10.393 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-08-30T09:16:10.393 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-08-30T09:16:10.394 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-08-30T09:16:10.394 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-08-30T09:16:10.394 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.394 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.394 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.394 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:10.394 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.394 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.394 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:10.394 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:10.394 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:10.394 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:10.394 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:10.394 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:10.395 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:10.395 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:10.395 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.395 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.395 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.395 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.395 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.395 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-08-30T09:16:10.395 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-08-30T09:16:10.395 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.395 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.395 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.395 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:10.396 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.396 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.396 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.396 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.396 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.396 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.396 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.396 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.396 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-08-30T09:16:10.396 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-08-30T09:16:10.396 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.396 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.396 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.396 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-08-30T09:16:10.396 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.397 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.397 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:10.397 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:10.397 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:10.397 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:10.397 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:10.397 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:10.397 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:10.397 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:10.397 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.397 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.397 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.397 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.398 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.398 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-08-30T09:16:10.398 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-08-30T09:16:10.398 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.398 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.398 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.398 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:10.398 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.398 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.398 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.398 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.398 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.398 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.398 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.398 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.399 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-08-30T09:16:10.399 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-08-30T09:16:10.399 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:10.399 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-08-30T09:16:10.399 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.399 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-08-30T09:16:10.399 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.399 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.399 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.399 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.399 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.399 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.399 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.399 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:10.400 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:10.400 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.400 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-08-30T09:16:10.400 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "pg_autoscaler", 2024-08-30T09:16:10.400 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-08-30T09:16:10.400 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-08-30T09:16:10.400 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-08-30T09:16:10.400 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-08-30T09:16:10.400 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-08-30T09:16:10.400 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.400 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.400 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.400 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:10.400 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.401 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.401 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:10.401 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:10.401 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:10.401 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:10.401 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:10.401 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:10.401 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:10.401 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:10.401 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.401 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.401 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.401 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.401 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.402 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-08-30T09:16:10.402 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-08-30T09:16:10.402 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.402 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.402 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.402 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:10.402 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.402 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.402 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.402 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.402 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.402 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.402 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.402 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.403 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-08-30T09:16:10.403 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-08-30T09:16:10.403 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.403 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.403 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.403 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-08-30T09:16:10.403 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.403 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.403 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:10.403 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:10.403 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:10.403 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:10.403 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:10.403 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:10.403 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:10.404 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:10.404 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.404 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.404 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.404 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.404 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.404 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-08-30T09:16:10.404 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-08-30T09:16:10.404 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.404 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.404 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.404 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:10.404 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.404 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.405 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.405 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.405 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.405 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.405 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.405 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.405 INFO:teuthology.orchestra.run.smithi042.stdout: "sleep_interval": { 2024-08-30T09:16:10.405 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sleep_interval", 2024-08-30T09:16:10.405 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-08-30T09:16:10.405 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.405 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.405 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "60", 2024-08-30T09:16:10.405 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.405 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.405 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.406 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.406 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.406 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.406 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.406 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.406 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-08-30T09:16:10.406 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-08-30T09:16:10.406 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:10.406 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-08-30T09:16:10.406 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.406 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-08-30T09:16:10.406 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.406 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.406 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.406 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.407 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.407 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.407 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.407 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.407 INFO:teuthology.orchestra.run.smithi042.stdout: "threshold": { 2024-08-30T09:16:10.407 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "threshold", 2024-08-30T09:16:10.407 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "float", 2024-08-30T09:16:10.407 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.407 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.407 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "3.0", 2024-08-30T09:16:10.407 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "1.0", 2024-08-30T09:16:10.407 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.407 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.408 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "scaling threshold", 2024-08-30T09:16:10.408 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-08-30T09:16:10.408 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.408 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.408 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:10.408 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:10.408 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.408 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-08-30T09:16:10.408 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "progress", 2024-08-30T09:16:10.408 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-08-30T09:16:10.408 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-08-30T09:16:10.408 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-08-30T09:16:10.408 INFO:teuthology.orchestra.run.smithi042.stdout: "allow_pg_recovery_event": { 2024-08-30T09:16:10.408 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "allow_pg_recovery_event", 2024-08-30T09:16:10.409 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.409 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.409 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.409 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:10.409 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.409 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.409 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.409 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "allow the module to show pg recovery progress", 2024-08-30T09:16:10.409 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.409 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.409 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.409 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.409 INFO:teuthology.orchestra.run.smithi042.stdout: "enabled": { 2024-08-30T09:16:10.410 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "enabled", 2024-08-30T09:16:10.410 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.410 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.410 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.410 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-08-30T09:16:10.410 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.410 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.410 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.410 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.410 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.410 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.410 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.410 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.410 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-08-30T09:16:10.410 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-08-30T09:16:10.411 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.411 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.411 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.411 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:10.411 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.411 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.411 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:10.411 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:10.411 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:10.411 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:10.411 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:10.411 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:10.411 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:10.411 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:10.412 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.412 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.412 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.412 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.412 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.412 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-08-30T09:16:10.412 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-08-30T09:16:10.412 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.412 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.412 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.412 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:10.412 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.412 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.412 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.413 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.413 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.413 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.413 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.413 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.413 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-08-30T09:16:10.413 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-08-30T09:16:10.413 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.413 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.413 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.413 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-08-30T09:16:10.413 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.413 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.413 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:10.414 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:10.414 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:10.414 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:10.414 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:10.414 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:10.414 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:10.414 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:10.414 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.414 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.414 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.414 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.414 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.414 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-08-30T09:16:10.414 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-08-30T09:16:10.414 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.415 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.415 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.415 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:10.415 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.415 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.415 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.415 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.415 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.415 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.415 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.415 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.415 INFO:teuthology.orchestra.run.smithi042.stdout: "max_completed_events": { 2024-08-30T09:16:10.415 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "max_completed_events", 2024-08-30T09:16:10.415 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:10.416 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.416 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.416 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "50", 2024-08-30T09:16:10.416 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.416 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.416 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.416 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "number of past completed events to remember", 2024-08-30T09:16:10.416 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.416 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.416 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.416 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.416 INFO:teuthology.orchestra.run.smithi042.stdout: "sleep_interval": { 2024-08-30T09:16:10.416 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sleep_interval", 2024-08-30T09:16:10.416 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-08-30T09:16:10.416 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.417 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.417 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "5", 2024-08-30T09:16:10.417 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.417 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.417 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.417 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "how long the module is going to sleep", 2024-08-30T09:16:10.417 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.417 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.417 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.417 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.417 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-08-30T09:16:10.417 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-08-30T09:16:10.417 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:10.417 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-08-30T09:16:10.418 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.418 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-08-30T09:16:10.418 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.418 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.418 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.418 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.418 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.418 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.418 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.418 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:10.418 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:10.418 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.418 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-08-30T09:16:10.418 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "prometheus", 2024-08-30T09:16:10.418 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-08-30T09:16:10.419 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-08-30T09:16:10.419 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-08-30T09:16:10.419 INFO:teuthology.orchestra.run.smithi042.stdout: "cache": { 2024-08-30T09:16:10.419 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "cache", 2024-08-30T09:16:10.419 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.419 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.419 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.419 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-08-30T09:16:10.419 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.419 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.419 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.419 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.419 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.419 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.419 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.420 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.420 INFO:teuthology.orchestra.run.smithi042.stdout: "exclude_perf_counters": { 2024-08-30T09:16:10.420 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "exclude_perf_counters", 2024-08-30T09:16:10.420 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.420 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.420 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.420 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-08-30T09:16:10.420 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.420 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.420 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.420 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-08-30T09:16:10.420 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-08-30T09:16:10.420 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.420 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.421 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.421 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-08-30T09:16:10.421 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-08-30T09:16:10.421 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.421 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.421 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.421 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:10.421 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.421 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.421 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:10.421 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:10.421 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:10.421 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:10.422 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:10.422 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:10.422 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:10.422 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:10.422 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.422 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.422 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.422 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.422 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.422 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-08-30T09:16:10.422 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-08-30T09:16:10.422 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.422 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.423 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.423 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:10.423 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.423 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.423 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.423 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.423 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.423 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.423 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.423 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.423 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-08-30T09:16:10.423 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-08-30T09:16:10.423 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.424 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.424 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.424 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-08-30T09:16:10.424 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.424 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.424 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:10.424 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:10.424 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:10.424 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:10.424 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:10.424 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:10.424 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:10.424 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:10.424 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.425 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.425 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.425 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.425 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.425 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-08-30T09:16:10.425 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-08-30T09:16:10.425 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.425 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.425 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.425 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:10.425 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.425 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.425 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.425 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.426 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.426 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.426 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.426 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.426 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_stats_pools": { 2024-08-30T09:16:10.426 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rbd_stats_pools", 2024-08-30T09:16:10.426 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.426 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.426 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.426 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:10.426 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.426 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.426 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.426 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.427 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.427 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.427 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.427 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.427 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_stats_pools_refresh_interval": { 2024-08-30T09:16:10.427 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-08-30T09:16:10.427 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:10.427 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.427 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.427 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "300", 2024-08-30T09:16:10.427 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.427 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.427 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.427 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.428 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.428 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.428 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.428 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.428 INFO:teuthology.orchestra.run.smithi042.stdout: "scrape_interval": { 2024-08-30T09:16:10.428 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "scrape_interval", 2024-08-30T09:16:10.428 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "float", 2024-08-30T09:16:10.428 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.428 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.428 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "15.0", 2024-08-30T09:16:10.428 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.428 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.428 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.428 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.429 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.429 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.429 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.429 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.429 INFO:teuthology.orchestra.run.smithi042.stdout: "server_addr": { 2024-08-30T09:16:10.429 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "server_addr", 2024-08-30T09:16:10.429 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.429 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.429 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.429 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "::", 2024-08-30T09:16:10.429 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.429 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.429 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.430 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-08-30T09:16:10.430 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.430 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.430 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.430 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.430 INFO:teuthology.orchestra.run.smithi042.stdout: "server_port": { 2024-08-30T09:16:10.430 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "server_port", 2024-08-30T09:16:10.430 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:10.430 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.430 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.430 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "9283", 2024-08-30T09:16:10.430 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.430 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.431 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.431 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-08-30T09:16:10.431 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.431 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.431 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.431 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.431 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-08-30T09:16:10.431 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-08-30T09:16:10.431 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:10.431 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-08-30T09:16:10.431 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.431 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-08-30T09:16:10.431 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.431 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.431 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.432 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.432 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.432 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.432 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.432 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.432 INFO:teuthology.orchestra.run.smithi042.stdout: "stale_cache_strategy": { 2024-08-30T09:16:10.432 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "stale_cache_strategy", 2024-08-30T09:16:10.432 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.432 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.432 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.432 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "log", 2024-08-30T09:16:10.432 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.432 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.432 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.433 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.433 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.433 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.433 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.433 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.433 INFO:teuthology.orchestra.run.smithi042.stdout: "standby_behaviour": { 2024-08-30T09:16:10.433 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "standby_behaviour", 2024-08-30T09:16:10.433 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.433 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.433 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.433 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "default", 2024-08-30T09:16:10.433 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.433 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.433 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:10.433 INFO:teuthology.orchestra.run.smithi042.stdout: "default", 2024-08-30T09:16:10.434 INFO:teuthology.orchestra.run.smithi042.stdout: "error" 2024-08-30T09:16:10.434 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:10.434 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.434 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.434 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.434 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.434 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.434 INFO:teuthology.orchestra.run.smithi042.stdout: "standby_error_status_code": { 2024-08-30T09:16:10.434 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "standby_error_status_code", 2024-08-30T09:16:10.434 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:10.434 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.434 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.434 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "500", 2024-08-30T09:16:10.434 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "400", 2024-08-30T09:16:10.434 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "599", 2024-08-30T09:16:10.435 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.435 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.435 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.435 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.435 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.435 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:10.435 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:10.435 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.435 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-08-30T09:16:10.435 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rbd_support", 2024-08-30T09:16:10.435 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-08-30T09:16:10.435 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-08-30T09:16:10.435 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-08-30T09:16:10.436 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-08-30T09:16:10.436 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-08-30T09:16:10.436 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.436 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.436 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.436 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:10.436 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.436 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.436 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:10.436 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:10.436 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:10.436 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:10.436 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:10.436 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:10.437 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:10.437 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:10.437 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.437 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.437 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.437 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.437 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.437 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-08-30T09:16:10.437 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-08-30T09:16:10.437 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.437 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.437 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.437 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:10.437 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.438 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.438 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.438 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.438 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.438 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.438 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.438 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.438 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-08-30T09:16:10.438 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-08-30T09:16:10.438 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.438 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.438 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.438 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-08-30T09:16:10.438 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.439 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.439 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:10.439 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:10.439 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:10.439 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:10.439 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:10.439 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:10.439 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:10.439 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:10.439 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.439 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.439 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.439 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.439 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.439 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-08-30T09:16:10.440 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-08-30T09:16:10.440 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.440 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.440 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.440 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:10.440 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.440 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.440 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.440 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.440 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.440 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.440 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.440 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.440 INFO:teuthology.orchestra.run.smithi042.stdout: "max_concurrent_snap_create": { 2024-08-30T09:16:10.441 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "max_concurrent_snap_create", 2024-08-30T09:16:10.441 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:10.441 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.441 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.441 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "10", 2024-08-30T09:16:10.441 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.441 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.441 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.441 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.441 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.441 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.441 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.441 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.442 INFO:teuthology.orchestra.run.smithi042.stdout: "mirror_snapshot_schedule": { 2024-08-30T09:16:10.442 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "mirror_snapshot_schedule", 2024-08-30T09:16:10.442 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.442 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.442 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.442 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:10.442 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.442 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.442 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.442 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.442 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.442 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.442 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.442 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.443 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-08-30T09:16:10.443 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-08-30T09:16:10.443 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:10.443 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-08-30T09:16:10.443 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.443 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-08-30T09:16:10.443 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.443 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.443 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.443 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.443 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.443 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.443 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.443 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.444 INFO:teuthology.orchestra.run.smithi042.stdout: "trash_purge_schedule": { 2024-08-30T09:16:10.444 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "trash_purge_schedule", 2024-08-30T09:16:10.444 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.444 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.444 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.444 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:10.444 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.444 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.444 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.444 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.444 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.444 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.444 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.444 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:10.445 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:10.445 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.445 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-08-30T09:16:10.445 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "restful", 2024-08-30T09:16:10.445 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-08-30T09:16:10.445 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-08-30T09:16:10.445 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-08-30T09:16:10.445 INFO:teuthology.orchestra.run.smithi042.stdout: "enable_auth": { 2024-08-30T09:16:10.445 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "enable_auth", 2024-08-30T09:16:10.445 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.445 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.445 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.445 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-08-30T09:16:10.445 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.446 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.446 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.446 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.446 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.446 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.446 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.446 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.446 INFO:teuthology.orchestra.run.smithi042.stdout: "key_file": { 2024-08-30T09:16:10.446 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "key_file", 2024-08-30T09:16:10.446 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.446 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.446 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.446 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:10.446 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.446 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.447 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.447 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.447 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.447 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.447 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.447 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.447 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-08-30T09:16:10.447 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-08-30T09:16:10.447 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.447 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.447 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.447 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:10.447 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.447 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.448 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:10.448 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:10.448 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:10.448 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:10.448 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:10.448 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:10.448 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:10.448 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:10.448 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.448 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.448 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.448 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.448 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.448 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-08-30T09:16:10.449 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-08-30T09:16:10.449 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.449 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.449 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.449 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:10.449 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.449 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.449 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.449 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.449 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.449 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.449 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.449 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.450 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-08-30T09:16:10.450 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-08-30T09:16:10.450 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.450 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.450 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.450 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-08-30T09:16:10.450 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.450 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.450 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:10.450 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:10.450 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:10.450 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:10.450 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:10.450 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:10.451 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:10.451 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:10.451 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.451 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.451 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.451 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.451 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.451 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-08-30T09:16:10.451 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-08-30T09:16:10.451 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.451 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.451 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.451 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:10.451 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.451 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.452 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.452 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.452 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.452 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.452 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.452 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.452 INFO:teuthology.orchestra.run.smithi042.stdout: "max_requests": { 2024-08-30T09:16:10.452 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "max_requests", 2024-08-30T09:16:10.452 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:10.452 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.452 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.452 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "500", 2024-08-30T09:16:10.452 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.452 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.453 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.453 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.", 2024-08-30T09:16:10.453 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.453 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.453 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.453 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.453 INFO:teuthology.orchestra.run.smithi042.stdout: "server_addr": { 2024-08-30T09:16:10.453 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "server_addr", 2024-08-30T09:16:10.453 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.453 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.453 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.453 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:10.453 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.454 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.454 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.454 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.454 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.454 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.454 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.454 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.454 INFO:teuthology.orchestra.run.smithi042.stdout: "server_port": { 2024-08-30T09:16:10.454 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "server_port", 2024-08-30T09:16:10.454 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.454 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.454 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.454 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:10.454 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.454 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.455 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.455 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.455 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.455 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.455 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.455 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.455 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-08-30T09:16:10.455 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-08-30T09:16:10.455 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:10.455 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-08-30T09:16:10.455 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.455 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-08-30T09:16:10.455 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.455 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.456 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.456 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.456 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.456 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.456 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.456 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:10.456 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:10.456 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.456 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-08-30T09:16:10.456 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rgw", 2024-08-30T09:16:10.456 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-08-30T09:16:10.456 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-08-30T09:16:10.456 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-08-30T09:16:10.456 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-08-30T09:16:10.456 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-08-30T09:16:10.457 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.457 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.457 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.457 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:10.457 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.457 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.457 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:10.457 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:10.457 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:10.457 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:10.457 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:10.457 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:10.457 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:10.457 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:10.458 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.458 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.458 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.458 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.458 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.458 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-08-30T09:16:10.458 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-08-30T09:16:10.458 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.458 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.458 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.458 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:10.458 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.458 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.458 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.459 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.459 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.459 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.459 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.459 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.459 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-08-30T09:16:10.459 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-08-30T09:16:10.459 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.459 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.459 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.459 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-08-30T09:16:10.459 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.459 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.460 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:10.460 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:10.460 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:10.460 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:10.460 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:10.460 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:10.460 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:10.460 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:10.460 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.460 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.460 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.460 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.460 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.460 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-08-30T09:16:10.461 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-08-30T09:16:10.461 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.461 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.461 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.461 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:10.461 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.461 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.461 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.461 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.461 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.461 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.461 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.461 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.461 INFO:teuthology.orchestra.run.smithi042.stdout: "secondary_zone_period_retry_limit": { 2024-08-30T09:16:10.462 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "secondary_zone_period_retry_limit", 2024-08-30T09:16:10.462 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:10.462 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.462 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.462 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "5", 2024-08-30T09:16:10.462 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.462 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.462 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.462 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-08-30T09:16:10.462 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.462 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.462 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.462 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.463 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-08-30T09:16:10.463 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-08-30T09:16:10.463 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:10.463 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-08-30T09:16:10.463 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.463 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-08-30T09:16:10.463 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.463 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.463 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.463 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.463 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.463 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.463 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.463 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:10.464 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:10.464 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.464 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-08-30T09:16:10.464 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rook", 2024-08-30T09:16:10.464 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-08-30T09:16:10.464 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-08-30T09:16:10.464 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-08-30T09:16:10.464 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-08-30T09:16:10.464 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-08-30T09:16:10.464 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.464 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.464 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.465 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:10.465 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.465 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.465 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:10.465 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:10.465 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:10.465 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:10.465 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:10.465 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:10.465 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:10.465 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:10.465 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.465 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.465 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.466 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.466 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.466 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-08-30T09:16:10.466 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-08-30T09:16:10.466 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.466 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.466 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.466 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:10.466 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.466 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.466 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.466 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.466 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.466 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.467 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.467 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.467 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-08-30T09:16:10.467 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-08-30T09:16:10.467 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.467 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.467 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.467 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-08-30T09:16:10.467 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.467 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.467 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:10.467 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:10.467 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:10.467 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:10.468 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:10.468 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:10.468 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:10.468 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:10.468 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.468 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.468 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.468 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.468 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.468 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-08-30T09:16:10.468 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-08-30T09:16:10.468 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.468 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.469 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.469 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:10.469 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.469 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.469 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.469 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.469 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.469 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.469 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.469 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.469 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-08-30T09:16:10.469 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-08-30T09:16:10.470 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:10.470 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-08-30T09:16:10.470 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.470 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-08-30T09:16:10.470 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.470 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.470 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.470 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.470 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.470 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.470 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.470 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.470 INFO:teuthology.orchestra.run.smithi042.stdout: "storage_class": { 2024-08-30T09:16:10.470 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "storage_class", 2024-08-30T09:16:10.470 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.471 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.471 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.471 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "local", 2024-08-30T09:16:10.471 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.471 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.471 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.471 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-08-30T09:16:10.471 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.471 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.471 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.471 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:10.471 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:10.471 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.471 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-08-30T09:16:10.471 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "selftest", 2024-08-30T09:16:10.472 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-08-30T09:16:10.472 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-08-30T09:16:10.472 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-08-30T09:16:10.472 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-08-30T09:16:10.472 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-08-30T09:16:10.472 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.472 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.472 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.472 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:10.472 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.472 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.472 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:10.472 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:10.472 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:10.473 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:10.473 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:10.473 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:10.473 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:10.473 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:10.473 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.473 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.473 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.473 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.473 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.473 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-08-30T09:16:10.473 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-08-30T09:16:10.473 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.473 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.474 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.474 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:10.474 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.474 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.474 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.474 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.474 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.474 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.474 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.474 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.474 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-08-30T09:16:10.474 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-08-30T09:16:10.474 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.474 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.475 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.475 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-08-30T09:16:10.475 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.475 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.475 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:10.475 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:10.475 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:10.475 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:10.475 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:10.475 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:10.475 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:10.475 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:10.475 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.475 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.475 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.476 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.476 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.476 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-08-30T09:16:10.476 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-08-30T09:16:10.476 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.476 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.476 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.476 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:10.476 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.476 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.476 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.476 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.476 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.476 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.477 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.477 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.477 INFO:teuthology.orchestra.run.smithi042.stdout: "roption1": { 2024-08-30T09:16:10.477 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "roption1", 2024-08-30T09:16:10.477 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.477 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.477 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.477 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:10.477 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.477 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.477 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.477 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.477 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.478 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.478 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.478 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.478 INFO:teuthology.orchestra.run.smithi042.stdout: "roption2": { 2024-08-30T09:16:10.478 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "roption2", 2024-08-30T09:16:10.478 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.478 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.478 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.478 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "xyz", 2024-08-30T09:16:10.478 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.478 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.478 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.478 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.478 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.479 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.479 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.479 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.479 INFO:teuthology.orchestra.run.smithi042.stdout: "rwoption1": { 2024-08-30T09:16:10.479 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rwoption1", 2024-08-30T09:16:10.479 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.479 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.479 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.479 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:10.479 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.479 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.479 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.479 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.480 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.480 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.480 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.480 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.480 INFO:teuthology.orchestra.run.smithi042.stdout: "rwoption2": { 2024-08-30T09:16:10.480 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rwoption2", 2024-08-30T09:16:10.480 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:10.480 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.480 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.480 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:10.480 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.480 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.480 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.481 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.481 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.481 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.481 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.481 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.481 INFO:teuthology.orchestra.run.smithi042.stdout: "rwoption3": { 2024-08-30T09:16:10.481 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rwoption3", 2024-08-30T09:16:10.481 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "float", 2024-08-30T09:16:10.481 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.481 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.481 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:10.481 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.481 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.481 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.481 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.482 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.482 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.482 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.482 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.482 INFO:teuthology.orchestra.run.smithi042.stdout: "rwoption4": { 2024-08-30T09:16:10.482 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rwoption4", 2024-08-30T09:16:10.482 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.482 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.482 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.482 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:10.482 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.482 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.482 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.482 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.483 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.483 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.483 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.483 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.483 INFO:teuthology.orchestra.run.smithi042.stdout: "rwoption5": { 2024-08-30T09:16:10.483 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rwoption5", 2024-08-30T09:16:10.483 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.483 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.483 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.483 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:10.483 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.483 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.483 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.483 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.484 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.484 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.484 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.484 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.484 INFO:teuthology.orchestra.run.smithi042.stdout: "rwoption6": { 2024-08-30T09:16:10.484 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rwoption6", 2024-08-30T09:16:10.484 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.484 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.484 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.484 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-08-30T09:16:10.484 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.484 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.484 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.484 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.485 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.485 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.485 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.485 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.485 INFO:teuthology.orchestra.run.smithi042.stdout: "rwoption7": { 2024-08-30T09:16:10.485 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rwoption7", 2024-08-30T09:16:10.485 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:10.485 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.485 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.485 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:10.485 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "1", 2024-08-30T09:16:10.485 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "42", 2024-08-30T09:16:10.485 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.485 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.485 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.486 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.486 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.486 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.486 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-08-30T09:16:10.486 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-08-30T09:16:10.486 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:10.486 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-08-30T09:16:10.486 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.486 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-08-30T09:16:10.486 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.486 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.486 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.486 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.487 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.487 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.487 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.487 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.487 INFO:teuthology.orchestra.run.smithi042.stdout: "testkey": { 2024-08-30T09:16:10.487 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "testkey", 2024-08-30T09:16:10.487 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.487 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.487 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.487 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:10.487 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.487 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.487 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.487 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.488 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.488 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.488 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.488 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.488 INFO:teuthology.orchestra.run.smithi042.stdout: "testlkey": { 2024-08-30T09:16:10.488 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "testlkey", 2024-08-30T09:16:10.488 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.488 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.488 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.488 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:10.488 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.488 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.488 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.488 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.489 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.489 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.489 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.489 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.489 INFO:teuthology.orchestra.run.smithi042.stdout: "testnewline": { 2024-08-30T09:16:10.489 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "testnewline", 2024-08-30T09:16:10.489 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.489 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.489 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.489 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:10.489 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.489 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.489 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.489 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.490 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.490 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.490 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.490 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:10.490 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:10.490 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.490 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-08-30T09:16:10.490 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "smb", 2024-08-30T09:16:10.490 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-08-30T09:16:10.490 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-08-30T09:16:10.490 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-08-30T09:16:10.490 INFO:teuthology.orchestra.run.smithi042.stdout: "internal_store_backend": { 2024-08-30T09:16:10.491 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "internal_store_backend", 2024-08-30T09:16:10.491 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.491 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-08-30T09:16:10.491 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.491 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:10.491 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.491 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.491 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.491 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "set internal store backend. for develoment and testing only", 2024-08-30T09:16:10.491 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.491 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.491 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.491 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.491 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-08-30T09:16:10.491 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-08-30T09:16:10.492 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.492 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.492 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.492 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:10.492 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.492 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.492 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:10.492 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:10.492 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:10.492 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:10.492 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:10.492 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:10.492 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:10.492 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:10.493 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.493 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.493 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.493 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.493 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.493 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-08-30T09:16:10.493 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-08-30T09:16:10.493 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.493 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.493 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.493 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:10.493 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.493 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.493 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.494 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.494 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.494 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.494 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.494 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.494 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-08-30T09:16:10.494 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-08-30T09:16:10.494 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.494 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.494 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.494 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-08-30T09:16:10.494 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.494 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.494 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:10.494 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:10.495 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:10.495 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:10.495 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:10.495 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:10.495 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:10.495 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:10.495 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.495 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.495 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.495 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.495 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.495 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-08-30T09:16:10.495 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-08-30T09:16:10.495 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.495 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.496 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.496 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:10.496 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.496 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.496 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.496 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.496 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.496 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.496 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.496 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.496 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-08-30T09:16:10.496 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-08-30T09:16:10.496 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:10.496 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-08-30T09:16:10.497 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.497 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-08-30T09:16:10.497 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.497 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.497 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.497 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.497 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.497 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.497 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.497 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.497 INFO:teuthology.orchestra.run.smithi042.stdout: "update_orchestration": { 2024-08-30T09:16:10.497 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "update_orchestration", 2024-08-30T09:16:10.497 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.497 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.498 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.498 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-08-30T09:16:10.498 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.498 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.498 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.498 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-08-30T09:16:10.498 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.498 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.498 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.498 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:10.498 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:10.498 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.498 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-08-30T09:16:10.498 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "snap_schedule", 2024-08-30T09:16:10.499 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-08-30T09:16:10.499 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-08-30T09:16:10.499 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-08-30T09:16:10.499 INFO:teuthology.orchestra.run.smithi042.stdout: "allow_m_granularity": { 2024-08-30T09:16:10.499 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "allow_m_granularity", 2024-08-30T09:16:10.499 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.499 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.499 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.499 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:10.499 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.499 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.499 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.499 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "allow minute scheduled snapshots", 2024-08-30T09:16:10.499 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.499 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.500 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.500 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.500 INFO:teuthology.orchestra.run.smithi042.stdout: "dump_on_update": { 2024-08-30T09:16:10.500 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "dump_on_update", 2024-08-30T09:16:10.500 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.500 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.500 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.500 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:10.500 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.500 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.500 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.500 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "dump database to debug log on update", 2024-08-30T09:16:10.500 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.500 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.501 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.501 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.501 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-08-30T09:16:10.501 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-08-30T09:16:10.501 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.501 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.501 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.501 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:10.501 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.501 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.501 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:10.501 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:10.501 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:10.502 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:10.502 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:10.502 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:10.502 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:10.502 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:10.502 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.502 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.502 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.502 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.502 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.502 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-08-30T09:16:10.502 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-08-30T09:16:10.502 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.502 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.502 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.503 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:10.503 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.503 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.503 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.503 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.503 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.503 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.503 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.503 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.503 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-08-30T09:16:10.503 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-08-30T09:16:10.503 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.503 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.503 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.503 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-08-30T09:16:10.504 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.504 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.504 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:10.504 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:10.504 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:10.504 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:10.504 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:10.504 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:10.504 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:10.504 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:10.504 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.504 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.504 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.504 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.504 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.505 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-08-30T09:16:10.505 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-08-30T09:16:10.505 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.505 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.505 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.505 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:10.505 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.505 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.505 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.505 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.505 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.505 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.505 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.505 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.505 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-08-30T09:16:10.506 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-08-30T09:16:10.506 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:10.506 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-08-30T09:16:10.506 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.506 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-08-30T09:16:10.506 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.506 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.506 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.506 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.506 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.506 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.506 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.506 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:10.507 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:10.507 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.507 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-08-30T09:16:10.507 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "stats", 2024-08-30T09:16:10.507 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-08-30T09:16:10.507 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-08-30T09:16:10.507 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-08-30T09:16:10.507 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-08-30T09:16:10.507 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-08-30T09:16:10.507 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.507 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.507 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.507 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:10.507 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.507 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.508 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:10.508 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:10.508 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:10.508 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:10.508 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:10.508 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:10.508 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:10.508 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:10.508 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.508 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.508 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.508 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.508 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.509 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-08-30T09:16:10.509 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-08-30T09:16:10.509 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.509 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.509 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.509 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:10.509 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.509 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.509 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.509 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.509 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.509 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.509 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.510 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.510 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-08-30T09:16:10.510 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-08-30T09:16:10.510 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.510 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.510 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.510 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-08-30T09:16:10.510 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.510 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.510 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:10.510 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:10.510 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:10.510 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:10.510 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:10.511 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:10.511 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:10.511 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:10.511 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.511 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.511 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.511 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.511 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.511 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-08-30T09:16:10.511 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-08-30T09:16:10.511 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.511 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.511 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.512 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:10.512 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.512 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.512 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.512 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.512 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.512 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.512 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.512 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.512 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-08-30T09:16:10.512 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-08-30T09:16:10.512 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:10.512 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-08-30T09:16:10.513 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.513 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-08-30T09:16:10.513 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.513 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.513 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.513 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.513 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.513 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.513 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.513 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:10.513 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:10.513 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.513 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-08-30T09:16:10.513 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "status", 2024-08-30T09:16:10.514 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-08-30T09:16:10.514 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-08-30T09:16:10.514 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-08-30T09:16:10.514 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-08-30T09:16:10.514 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-08-30T09:16:10.514 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.514 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.514 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.514 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:10.514 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.514 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.514 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:10.514 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:10.514 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:10.514 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:10.514 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:10.515 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:10.515 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:10.515 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:10.515 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.515 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.515 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.515 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.515 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.515 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-08-30T09:16:10.515 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-08-30T09:16:10.515 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.515 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.515 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.515 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:10.516 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.516 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.516 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.516 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.516 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.516 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.516 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.516 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.516 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-08-30T09:16:10.516 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-08-30T09:16:10.516 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.516 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.516 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.516 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-08-30T09:16:10.517 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.517 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.517 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:10.517 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:10.517 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:10.517 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:10.517 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:10.517 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:10.517 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:10.517 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:10.517 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.517 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.517 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.517 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.518 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.518 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-08-30T09:16:10.518 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-08-30T09:16:10.518 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.518 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.518 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.518 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:10.518 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.518 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.518 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.518 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.518 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.518 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.518 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.518 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.519 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-08-30T09:16:10.519 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-08-30T09:16:10.519 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:10.519 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-08-30T09:16:10.519 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.519 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-08-30T09:16:10.519 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.519 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.519 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.519 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.519 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.519 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.519 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.519 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:10.519 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:10.520 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.520 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-08-30T09:16:10.520 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "telegraf", 2024-08-30T09:16:10.520 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-08-30T09:16:10.520 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-08-30T09:16:10.520 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-08-30T09:16:10.520 INFO:teuthology.orchestra.run.smithi042.stdout: "address": { 2024-08-30T09:16:10.520 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "address", 2024-08-30T09:16:10.520 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.520 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.520 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.520 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-08-30T09:16:10.520 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.520 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.521 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.521 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.521 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.521 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.521 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.521 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.521 INFO:teuthology.orchestra.run.smithi042.stdout: "interval": { 2024-08-30T09:16:10.521 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "interval", 2024-08-30T09:16:10.521 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-08-30T09:16:10.521 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.521 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.521 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "15", 2024-08-30T09:16:10.521 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.522 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.522 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.522 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.522 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.522 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.522 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.522 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.522 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-08-30T09:16:10.522 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-08-30T09:16:10.522 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.522 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.522 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.522 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:10.522 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.523 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.523 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:10.523 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:10.523 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:10.523 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:10.523 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:10.523 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:10.523 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:10.523 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:10.523 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.523 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.523 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.523 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.524 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.524 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-08-30T09:16:10.524 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-08-30T09:16:10.524 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.524 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.524 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.524 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:10.524 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.524 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.524 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.524 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.524 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.524 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.525 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.525 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.525 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-08-30T09:16:10.525 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-08-30T09:16:10.525 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.525 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.525 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.525 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-08-30T09:16:10.525 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.525 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.525 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:10.525 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:10.525 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:10.525 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:10.526 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:10.526 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:10.526 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:10.526 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:10.526 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.526 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.526 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.526 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.526 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.526 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-08-30T09:16:10.526 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-08-30T09:16:10.526 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.526 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.526 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.526 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:10.527 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.527 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.527 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.527 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.527 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.527 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.527 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.527 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.527 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-08-30T09:16:10.527 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-08-30T09:16:10.527 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:10.527 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-08-30T09:16:10.527 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.527 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-08-30T09:16:10.527 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.528 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.528 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.528 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.528 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.528 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.528 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.528 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:10.528 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:10.528 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.528 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-08-30T09:16:10.528 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "telemetry", 2024-08-30T09:16:10.528 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-08-30T09:16:10.528 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-08-30T09:16:10.528 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-08-30T09:16:10.529 INFO:teuthology.orchestra.run.smithi042.stdout: "channel_basic": { 2024-08-30T09:16:10.529 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "channel_basic", 2024-08-30T09:16:10.529 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.529 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.529 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.529 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-08-30T09:16:10.529 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.529 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.529 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.529 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Share basic cluster information (size, version)", 2024-08-30T09:16:10.529 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.529 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.529 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.530 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.530 INFO:teuthology.orchestra.run.smithi042.stdout: "channel_crash": { 2024-08-30T09:16:10.530 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "channel_crash", 2024-08-30T09:16:10.530 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.530 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.530 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.530 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-08-30T09:16:10.530 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.530 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.530 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.530 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-08-30T09:16:10.530 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.530 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.530 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.530 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.531 INFO:teuthology.orchestra.run.smithi042.stdout: "channel_device": { 2024-08-30T09:16:10.531 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "channel_device", 2024-08-30T09:16:10.531 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.531 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.531 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.531 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-08-30T09:16:10.531 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.531 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.531 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.531 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-08-30T09:16:10.531 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.531 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.532 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.532 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.532 INFO:teuthology.orchestra.run.smithi042.stdout: "channel_ident": { 2024-08-30T09:16:10.532 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "channel_ident", 2024-08-30T09:16:10.532 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.532 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.532 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.532 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:10.533 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.533 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.533 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.533 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-08-30T09:16:10.533 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.533 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.533 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.533 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.533 INFO:teuthology.orchestra.run.smithi042.stdout: "channel_perf": { 2024-08-30T09:16:10.533 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "channel_perf", 2024-08-30T09:16:10.533 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.533 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.533 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.534 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:10.534 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.534 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.534 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.534 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Share various performance metrics of a cluster", 2024-08-30T09:16:10.534 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.534 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.534 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.534 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.534 INFO:teuthology.orchestra.run.smithi042.stdout: "contact": { 2024-08-30T09:16:10.534 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "contact", 2024-08-30T09:16:10.534 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.534 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.534 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.535 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:10.535 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.535 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.535 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.535 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.535 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.535 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.536 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.536 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.536 INFO:teuthology.orchestra.run.smithi042.stdout: "description": { 2024-08-30T09:16:10.536 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "description", 2024-08-30T09:16:10.536 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.536 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.537 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.537 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:10.537 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.537 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.537 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.537 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.537 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.537 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.537 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.537 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.538 INFO:teuthology.orchestra.run.smithi042.stdout: "device_url": { 2024-08-30T09:16:10.538 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "device_url", 2024-08-30T09:16:10.538 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.538 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.538 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.538 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-08-30T09:16:10.538 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.538 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.538 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.539 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.539 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.539 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.539 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.539 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.539 INFO:teuthology.orchestra.run.smithi042.stdout: "enabled": { 2024-08-30T09:16:10.539 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "enabled", 2024-08-30T09:16:10.539 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.540 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.540 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.540 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:10.540 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.540 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.540 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.540 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.540 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.540 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.540 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.540 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.541 INFO:teuthology.orchestra.run.smithi042.stdout: "interval": { 2024-08-30T09:16:10.541 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "interval", 2024-08-30T09:16:10.541 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:10.541 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.541 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.541 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "24", 2024-08-30T09:16:10.541 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "8", 2024-08-30T09:16:10.541 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.542 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.542 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.542 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.542 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.542 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.542 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.542 INFO:teuthology.orchestra.run.smithi042.stdout: "last_opt_revision": { 2024-08-30T09:16:10.542 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "last_opt_revision", 2024-08-30T09:16:10.542 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:10.542 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.542 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.542 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "1", 2024-08-30T09:16:10.542 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.543 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.543 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.543 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.543 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.543 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.543 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.543 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.543 INFO:teuthology.orchestra.run.smithi042.stdout: "leaderboard": { 2024-08-30T09:16:10.543 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "leaderboard", 2024-08-30T09:16:10.543 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.543 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.543 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.543 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:10.543 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.544 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.544 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.544 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.544 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.544 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.544 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.544 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.544 INFO:teuthology.orchestra.run.smithi042.stdout: "leaderboard_description": { 2024-08-30T09:16:10.544 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "leaderboard_description", 2024-08-30T09:16:10.544 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.544 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.544 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.544 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:10.545 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.545 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.545 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.545 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.545 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.545 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.545 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.545 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.545 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-08-30T09:16:10.545 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-08-30T09:16:10.545 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.545 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.545 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.546 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:10.546 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.546 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.546 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:10.546 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:10.546 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:10.546 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:10.546 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:10.546 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:10.546 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:10.546 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:10.546 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.546 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.546 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.547 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.547 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.547 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-08-30T09:16:10.547 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-08-30T09:16:10.547 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.547 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.547 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.547 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:10.547 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.547 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.547 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.547 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.547 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.547 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.548 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.548 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.548 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-08-30T09:16:10.548 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-08-30T09:16:10.548 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.548 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.548 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.548 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-08-30T09:16:10.548 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.548 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.548 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:10.548 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:10.548 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:10.548 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:10.549 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:10.549 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:10.549 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:10.549 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:10.549 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.549 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.549 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.549 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.549 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.549 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-08-30T09:16:10.549 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-08-30T09:16:10.549 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.549 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.550 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.550 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:10.550 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.550 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.550 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.550 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.550 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.550 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.550 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.550 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.550 INFO:teuthology.orchestra.run.smithi042.stdout: "organization": { 2024-08-30T09:16:10.550 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "organization", 2024-08-30T09:16:10.550 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.550 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.551 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.551 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:10.551 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.551 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.551 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.551 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.551 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.551 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.551 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.551 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.551 INFO:teuthology.orchestra.run.smithi042.stdout: "proxy": { 2024-08-30T09:16:10.551 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "proxy", 2024-08-30T09:16:10.551 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.551 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.552 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.552 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:10.552 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.552 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.552 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.552 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.552 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.552 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.552 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.552 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.552 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-08-30T09:16:10.552 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-08-30T09:16:10.552 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:10.552 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-08-30T09:16:10.553 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.553 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-08-30T09:16:10.553 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.553 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.553 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.553 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.553 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.553 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.553 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.553 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.553 INFO:teuthology.orchestra.run.smithi042.stdout: "url": { 2024-08-30T09:16:10.553 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "url", 2024-08-30T09:16:10.553 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.554 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.554 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.554 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-08-30T09:16:10.554 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.554 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.554 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.554 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.554 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.554 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.554 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.554 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:10.554 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:10.554 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.554 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-08-30T09:16:10.554 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "test_orchestrator", 2024-08-30T09:16:10.555 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-08-30T09:16:10.555 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-08-30T09:16:10.555 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-08-30T09:16:10.555 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-08-30T09:16:10.555 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-08-30T09:16:10.555 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.555 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.555 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.555 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:10.555 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.555 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.555 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:10.555 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:10.556 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:10.556 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:10.556 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:10.556 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:10.556 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:10.556 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:10.556 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.556 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.556 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.556 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.556 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.556 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-08-30T09:16:10.556 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-08-30T09:16:10.556 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.557 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.557 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.557 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:10.557 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.557 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.557 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.557 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.557 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.557 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.557 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.557 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.557 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-08-30T09:16:10.557 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-08-30T09:16:10.557 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.558 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.558 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.558 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-08-30T09:16:10.558 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.558 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.558 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:10.558 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:10.558 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:10.558 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:10.558 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:10.558 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:10.558 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:10.558 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:10.558 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.558 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.559 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.559 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.559 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.559 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-08-30T09:16:10.559 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-08-30T09:16:10.559 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.559 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.559 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.559 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:10.559 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.559 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.559 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.559 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.560 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.560 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.560 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.560 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.560 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-08-30T09:16:10.560 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-08-30T09:16:10.560 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:10.560 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-08-30T09:16:10.560 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.560 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-08-30T09:16:10.560 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.560 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.560 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.560 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.561 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.561 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.561 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.561 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:10.561 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:10.561 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.561 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-08-30T09:16:10.561 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "volumes", 2024-08-30T09:16:10.561 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-08-30T09:16:10.561 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-08-30T09:16:10.561 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-08-30T09:16:10.561 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-08-30T09:16:10.561 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-08-30T09:16:10.561 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.562 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.562 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.562 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:10.562 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.562 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.562 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:10.562 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:10.562 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:10.562 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:10.562 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:10.562 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:10.562 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:10.562 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:10.562 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.563 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.563 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.563 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.563 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.563 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-08-30T09:16:10.563 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-08-30T09:16:10.563 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.563 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.563 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.563 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:10.563 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.563 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.563 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.563 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.563 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.564 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.564 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.564 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.564 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-08-30T09:16:10.564 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-08-30T09:16:10.564 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.564 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.564 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.564 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-08-30T09:16:10.564 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.564 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.564 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:10.564 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:10.564 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:10.564 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:10.565 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:10.565 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:10.565 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:10.565 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:10.565 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.565 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.565 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.565 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.565 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.565 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-08-30T09:16:10.565 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-08-30T09:16:10.565 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.565 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.566 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.566 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:10.566 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.566 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.566 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.566 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.566 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.566 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.566 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.566 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.566 INFO:teuthology.orchestra.run.smithi042.stdout: "max_concurrent_clones": { 2024-08-30T09:16:10.566 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "max_concurrent_clones", 2024-08-30T09:16:10.566 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:10.566 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.566 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.567 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "4", 2024-08-30T09:16:10.567 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.567 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.567 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.567 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Number of asynchronous cloner threads", 2024-08-30T09:16:10.567 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.567 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.567 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.567 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.567 INFO:teuthology.orchestra.run.smithi042.stdout: "periodic_async_work": { 2024-08-30T09:16:10.567 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "periodic_async_work", 2024-08-30T09:16:10.567 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.567 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.567 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.568 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:10.568 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.568 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.568 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.568 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Periodically check for async work", 2024-08-30T09:16:10.568 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.568 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.568 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.568 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.568 INFO:teuthology.orchestra.run.smithi042.stdout: "snapshot_clone_delay": { 2024-08-30T09:16:10.568 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "snapshot_clone_delay", 2024-08-30T09:16:10.568 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:10.568 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.568 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.569 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-08-30T09:16:10.569 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.569 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.569 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.569 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-08-30T09:16:10.569 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.569 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.569 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.569 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.569 INFO:teuthology.orchestra.run.smithi042.stdout: "snapshot_clone_no_wait": { 2024-08-30T09:16:10.569 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "snapshot_clone_no_wait", 2024-08-30T09:16:10.569 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.569 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.570 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.570 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-08-30T09:16:10.570 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.570 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.570 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.570 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-08-30T09:16:10.570 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.570 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.570 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.570 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.570 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-08-30T09:16:10.570 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-08-30T09:16:10.570 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:10.570 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-08-30T09:16:10.570 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.571 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-08-30T09:16:10.571 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.571 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.571 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.571 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.571 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.571 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.571 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.571 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:10.571 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:10.571 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.571 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-08-30T09:16:10.571 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "zabbix", 2024-08-30T09:16:10.571 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-08-30T09:16:10.571 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-08-30T09:16:10.572 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-08-30T09:16:10.572 INFO:teuthology.orchestra.run.smithi042.stdout: "discovery_interval": { 2024-08-30T09:16:10.572 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "discovery_interval", 2024-08-30T09:16:10.572 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "uint", 2024-08-30T09:16:10.572 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.572 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.572 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "100", 2024-08-30T09:16:10.572 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.572 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.572 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.572 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.572 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.572 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.572 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.572 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.573 INFO:teuthology.orchestra.run.smithi042.stdout: "identifier": { 2024-08-30T09:16:10.573 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "identifier", 2024-08-30T09:16:10.573 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.573 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.573 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.573 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:10.573 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.573 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.573 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.573 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.573 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.573 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.573 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.573 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.574 INFO:teuthology.orchestra.run.smithi042.stdout: "interval": { 2024-08-30T09:16:10.574 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "interval", 2024-08-30T09:16:10.574 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-08-30T09:16:10.574 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.574 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.574 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "60", 2024-08-30T09:16:10.574 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.574 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.574 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.574 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.574 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.574 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.574 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.574 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.574 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-08-30T09:16:10.575 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-08-30T09:16:10.575 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.575 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.575 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.575 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:10.575 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.575 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.575 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:10.575 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:10.575 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:10.575 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:10.575 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:10.575 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:10.575 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:10.576 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:10.576 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.576 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.576 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.576 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.576 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.576 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-08-30T09:16:10.576 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-08-30T09:16:10.576 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.576 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.576 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.576 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:10.576 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.576 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.577 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.577 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.577 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.577 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.577 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.577 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.577 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-08-30T09:16:10.577 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-08-30T09:16:10.577 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.577 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.577 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.577 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-08-30T09:16:10.577 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.578 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.578 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:10.578 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:10.578 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:10.578 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:10.578 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:10.578 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:10.578 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:10.578 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:10.578 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.578 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.578 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.578 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.579 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.579 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-08-30T09:16:10.579 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-08-30T09:16:10.579 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.579 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.579 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.579 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:10.579 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.579 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.579 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.579 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.579 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.579 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.579 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.580 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.580 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-08-30T09:16:10.580 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-08-30T09:16:10.580 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:10.580 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-08-30T09:16:10.580 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.580 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-08-30T09:16:10.580 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.580 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.580 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.580 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.580 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.580 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.580 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.581 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.581 INFO:teuthology.orchestra.run.smithi042.stdout: "zabbix_host": { 2024-08-30T09:16:10.581 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "zabbix_host", 2024-08-30T09:16:10.581 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.581 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.581 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.581 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:10.581 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.581 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.581 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.581 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.581 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.581 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.581 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.582 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.582 INFO:teuthology.orchestra.run.smithi042.stdout: "zabbix_port": { 2024-08-30T09:16:10.582 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "zabbix_port", 2024-08-30T09:16:10.582 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:10.582 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.582 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.582 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "10051", 2024-08-30T09:16:10.582 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.582 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.582 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.582 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.582 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.582 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.582 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.583 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.583 INFO:teuthology.orchestra.run.smithi042.stdout: "zabbix_sender": { 2024-08-30T09:16:10.583 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "zabbix_sender", 2024-08-30T09:16:10.583 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.583 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.583 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.583 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-08-30T09:16:10.583 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.583 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.583 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.583 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.583 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.583 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.583 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.583 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:10.584 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:10.584 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:10.584 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-08-30T09:16:10.584 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:10.584 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:10.584 INFO:teuthology.orchestra.run.smithi042.stdout: "modules": [ 2024-08-30T09:16:10.584 INFO:teuthology.orchestra.run.smithi042.stdout: "cephadm", 2024-08-30T09:16:10.584 INFO:teuthology.orchestra.run.smithi042.stdout: "dashboard", 2024-08-30T09:16:10.584 INFO:teuthology.orchestra.run.smithi042.stdout: "iostat", 2024-08-30T09:16:10.584 INFO:teuthology.orchestra.run.smithi042.stdout: "nfs", 2024-08-30T09:16:10.584 INFO:teuthology.orchestra.run.smithi042.stdout: "restful" 2024-08-30T09:16:10.584 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:10.584 INFO:teuthology.orchestra.run.smithi042.stdout: "available_modules": [ 2024-08-30T09:16:10.585 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-08-30T09:16:10.585 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "alerts", 2024-08-30T09:16:10.585 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-08-30T09:16:10.585 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-08-30T09:16:10.585 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-08-30T09:16:10.585 INFO:teuthology.orchestra.run.smithi042.stdout: "interval": { 2024-08-30T09:16:10.585 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "interval", 2024-08-30T09:16:10.585 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-08-30T09:16:10.585 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.585 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.585 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "60", 2024-08-30T09:16:10.585 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.585 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.585 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.586 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "How frequently to reexamine health status", 2024-08-30T09:16:10.586 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.586 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.586 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.586 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.586 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-08-30T09:16:10.586 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-08-30T09:16:10.586 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.586 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.586 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.586 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:10.586 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.587 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.587 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:10.587 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:10.587 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:10.587 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:10.587 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:10.587 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:10.587 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:10.587 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:10.587 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.587 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.587 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.587 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.588 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.588 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-08-30T09:16:10.588 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-08-30T09:16:10.588 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.588 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.588 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.588 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:10.588 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.588 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.588 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.588 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.588 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.588 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.588 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.589 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.589 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-08-30T09:16:10.589 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-08-30T09:16:10.589 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.589 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.589 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.589 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-08-30T09:16:10.589 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.589 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.589 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:10.589 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:10.589 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:10.589 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:10.589 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:10.590 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:10.590 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:10.590 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:10.590 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.590 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.590 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.590 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.590 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.590 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-08-30T09:16:10.590 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-08-30T09:16:10.590 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.590 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.590 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.590 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:10.591 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.591 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.591 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.591 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.591 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.591 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.591 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.591 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.591 INFO:teuthology.orchestra.run.smithi042.stdout: "smtp_destination": { 2024-08-30T09:16:10.591 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "smtp_destination", 2024-08-30T09:16:10.591 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.591 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.591 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.591 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:10.591 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.592 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.592 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.592 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Email address to send alerts to", 2024-08-30T09:16:10.592 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.592 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.592 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.592 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.592 INFO:teuthology.orchestra.run.smithi042.stdout: "smtp_from_name": { 2024-08-30T09:16:10.592 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "smtp_from_name", 2024-08-30T09:16:10.592 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.592 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.592 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.592 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "Ceph", 2024-08-30T09:16:10.592 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.592 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.593 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.593 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Email From: name", 2024-08-30T09:16:10.593 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.593 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.593 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.593 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.593 INFO:teuthology.orchestra.run.smithi042.stdout: "smtp_host": { 2024-08-30T09:16:10.593 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "smtp_host", 2024-08-30T09:16:10.593 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.593 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.593 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.593 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:10.593 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.594 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.594 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.594 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "SMTP server", 2024-08-30T09:16:10.594 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.594 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.594 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.595 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.595 INFO:teuthology.orchestra.run.smithi042.stdout: "smtp_password": { 2024-08-30T09:16:10.595 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "smtp_password", 2024-08-30T09:16:10.595 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.595 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.595 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.595 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:10.595 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.595 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.595 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.595 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Password to authenticate with", 2024-08-30T09:16:10.595 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.595 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.595 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.595 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.596 INFO:teuthology.orchestra.run.smithi042.stdout: "smtp_port": { 2024-08-30T09:16:10.596 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "smtp_port", 2024-08-30T09:16:10.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:09.607+0000 7f864eb0b640 1 -- 172.21.15.42:0/1912388417 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f86300b4b30 msgr2=0x7f86300b7040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:16:10.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:09.607+0000 7f864eb0b640 1 --2- 172.21.15.42:0/1912388417 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f86300b4b30 0x7f86300b7040 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f8638005fd0 tx=0x7f8638005e20 comp rx=0 tx=0).stop 2024-08-30T09:16:10.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:09.607+0000 7f864eb0b640 1 -- 172.21.15.42:0/1912388417 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8648109160 msgr2=0x7f864819aed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:16:10.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:09.607+0000 7f864eb0b640 1 --2- 172.21.15.42:0/1912388417 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8648109160 0x7f864819aed0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f864000eaa0 tx=0x7f864000ef60 comp rx=0 tx=0).stop 2024-08-30T09:16:10.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:09.607+0000 7f864eb0b640 1 -- 172.21.15.42:0/1912388417 shutdown_connections 2024-08-30T09:16:10.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:09.607+0000 7f864eb0b640 1 --2- 172.21.15.42:0/1912388417 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f86300b4b30 0x7f86300b7040 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:10.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:09.607+0000 7f864eb0b640 1 --2- 172.21.15.42:0/1912388417 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8648109160 0x7f864819aed0 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:10.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:09.607+0000 7f864eb0b640 1 --2- 172.21.15.42:0/1912388417 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f86481065a0 0x7f864819a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:10.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:09.607+0000 7f864eb0b640 1 --2- 172.21.15.42:0/1912388417 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f864806c200 0x7f8648199cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:10.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:09.607+0000 7f864eb0b640 1 -- 172.21.15.42:0/1912388417 >> 172.21.15.42:0/1912388417 conn(0x7f86480fdc80 msgr2=0x7f8648104ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:16:10.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:09.607+0000 7f864eb0b640 1 -- 172.21.15.42:0/1912388417 shutdown_connections 2024-08-30T09:16:10.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:09.607+0000 7f864eb0b640 1 -- 172.21.15.42:0/1912388417 wait complete. 2024-08-30T09:16:10.599 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:10.599 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.599 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.600 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "465", 2024-08-30T09:16:10.600 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.600 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.600 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.600 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "SMTP port", 2024-08-30T09:16:10.600 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.600 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.600 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.600 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.600 INFO:teuthology.orchestra.run.smithi042.stdout: "smtp_sender": { 2024-08-30T09:16:10.600 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "smtp_sender", 2024-08-30T09:16:10.600 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.600 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.600 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.601 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:10.601 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.601 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.601 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.601 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "SMTP envelope sender", 2024-08-30T09:16:10.601 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.601 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.601 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.601 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.601 INFO:teuthology.orchestra.run.smithi042.stdout: "smtp_ssl": { 2024-08-30T09:16:10.601 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "smtp_ssl", 2024-08-30T09:16:10.601 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.601 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.601 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.601 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-08-30T09:16:10.602 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.602 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.602 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.602 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Use SSL to connect to SMTP server", 2024-08-30T09:16:10.602 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.602 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.602 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.602 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.602 INFO:teuthology.orchestra.run.smithi042.stdout: "smtp_user": { 2024-08-30T09:16:10.602 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "smtp_user", 2024-08-30T09:16:10.602 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.602 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.602 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.602 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:10.603 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.603 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.603 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.603 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "User to authenticate as", 2024-08-30T09:16:10.603 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.603 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.603 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.603 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.603 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-08-30T09:16:10.603 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-08-30T09:16:10.603 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:10.603 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-08-30T09:16:10.603 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.603 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-08-30T09:16:10.604 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.604 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.604 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.604 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.604 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.604 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.604 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.604 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:10.604 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:10.604 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.604 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-08-30T09:16:10.604 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "balancer", 2024-08-30T09:16:10.604 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-08-30T09:16:10.604 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-08-30T09:16:10.605 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-08-30T09:16:10.605 INFO:teuthology.orchestra.run.smithi042.stdout: "active": { 2024-08-30T09:16:10.605 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "active", 2024-08-30T09:16:10.605 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.605 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.605 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.605 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-08-30T09:16:10.605 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.605 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.605 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.605 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "automatically balance PGs across cluster", 2024-08-30T09:16:10.605 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.605 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.605 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.606 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.606 INFO:teuthology.orchestra.run.smithi042.stdout: "begin_time": { 2024-08-30T09:16:10.606 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "begin_time", 2024-08-30T09:16:10.606 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.606 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.606 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.606 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0000", 2024-08-30T09:16:10.606 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.606 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.606 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.606 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "beginning time of day to automatically balance", 2024-08-30T09:16:10.606 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-08-30T09:16:10.606 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.606 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.606 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.607 INFO:teuthology.orchestra.run.smithi042.stdout: "begin_weekday": { 2024-08-30T09:16:10.607 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "begin_weekday", 2024-08-30T09:16:10.607 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "uint", 2024-08-30T09:16:10.607 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.607 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.607 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-08-30T09:16:10.607 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "0", 2024-08-30T09:16:10.607 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "6", 2024-08-30T09:16:10.607 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.607 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-08-30T09:16:10.607 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-08-30T09:16:10.607 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.607 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.607 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.608 INFO:teuthology.orchestra.run.smithi042.stdout: "crush_compat_max_iterations": { 2024-08-30T09:16:10.608 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "crush_compat_max_iterations", 2024-08-30T09:16:10.608 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "uint", 2024-08-30T09:16:10.608 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.608 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.608 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "25", 2024-08-30T09:16:10.608 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "1", 2024-08-30T09:16:10.608 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "250", 2024-08-30T09:16:10.608 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.608 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-08-30T09:16:10.608 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.608 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.608 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.608 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.609 INFO:teuthology.orchestra.run.smithi042.stdout: "crush_compat_metrics": { 2024-08-30T09:16:10.609 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "crush_compat_metrics", 2024-08-30T09:16:10.609 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.609 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.609 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.609 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "pgs,objects,bytes", 2024-08-30T09:16:10.609 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.609 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.609 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.609 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-08-30T09:16:10.609 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-08-30T09:16:10.609 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.610 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.610 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.610 INFO:teuthology.orchestra.run.smithi042.stdout: "crush_compat_step": { 2024-08-30T09:16:10.610 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "crush_compat_step", 2024-08-30T09:16:10.610 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "float", 2024-08-30T09:16:10.610 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.610 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.610 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0.5", 2024-08-30T09:16:10.610 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "0.001", 2024-08-30T09:16:10.610 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "0.999", 2024-08-30T09:16:10.610 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.610 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "aggressiveness of optimization", 2024-08-30T09:16:10.610 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-08-30T09:16:10.611 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.611 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.611 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.611 INFO:teuthology.orchestra.run.smithi042.stdout: "end_time": { 2024-08-30T09:16:10.611 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "end_time", 2024-08-30T09:16:10.611 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.611 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.611 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.611 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "2359", 2024-08-30T09:16:10.611 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.611 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.611 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.611 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "ending time of day to automatically balance", 2024-08-30T09:16:10.611 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-08-30T09:16:10.612 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.612 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.612 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.612 INFO:teuthology.orchestra.run.smithi042.stdout: "end_weekday": { 2024-08-30T09:16:10.612 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "end_weekday", 2024-08-30T09:16:10.612 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "uint", 2024-08-30T09:16:10.612 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.612 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.612 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-08-30T09:16:10.612 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "0", 2024-08-30T09:16:10.612 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "6", 2024-08-30T09:16:10.612 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.612 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-08-30T09:16:10.613 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-08-30T09:16:10.613 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.613 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.613 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.613 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-08-30T09:16:10.613 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-08-30T09:16:10.613 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.613 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.613 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.613 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:10.613 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.613 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.613 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:10.613 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:10.614 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:10.614 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:10.614 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:10.614 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:10.614 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:10.614 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:10.614 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.614 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.614 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.614 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.614 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.614 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-08-30T09:16:10.614 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-08-30T09:16:10.614 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.614 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.615 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.615 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:10.615 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.615 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.615 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.615 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.615 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.615 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.615 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.615 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.615 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-08-30T09:16:10.615 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-08-30T09:16:10.615 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.615 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.616 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.616 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-08-30T09:16:10.616 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.616 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.616 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:10.616 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:10.616 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:10.616 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:10.616 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:10.616 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:10.616 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:10.616 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:10.616 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.616 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.617 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.617 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.617 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.617 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-08-30T09:16:10.617 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-08-30T09:16:10.617 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.617 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.617 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.617 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:10.617 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.617 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.617 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.617 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.617 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.617 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.618 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.618 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.618 INFO:teuthology.orchestra.run.smithi042.stdout: "min_score": { 2024-08-30T09:16:10.618 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "min_score", 2024-08-30T09:16:10.618 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "float", 2024-08-30T09:16:10.618 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.618 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.618 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-08-30T09:16:10.618 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.618 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.618 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.618 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-08-30T09:16:10.618 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.618 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.619 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.619 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.619 INFO:teuthology.orchestra.run.smithi042.stdout: "mode": { 2024-08-30T09:16:10.619 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "mode", 2024-08-30T09:16:10.619 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.619 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.619 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.619 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "upmap", 2024-08-30T09:16:10.619 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.619 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.619 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:10.619 INFO:teuthology.orchestra.run.smithi042.stdout: "crush-compat", 2024-08-30T09:16:10.619 INFO:teuthology.orchestra.run.smithi042.stdout: "none", 2024-08-30T09:16:10.619 INFO:teuthology.orchestra.run.smithi042.stdout: "read", 2024-08-30T09:16:10.619 INFO:teuthology.orchestra.run.smithi042.stdout: "upmap", 2024-08-30T09:16:10.620 INFO:teuthology.orchestra.run.smithi042.stdout: "upmap-read" 2024-08-30T09:16:10.620 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:10.620 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Balancer mode", 2024-08-30T09:16:10.620 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.620 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.620 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.620 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.620 INFO:teuthology.orchestra.run.smithi042.stdout: "pool_ids": { 2024-08-30T09:16:10.620 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "pool_ids", 2024-08-30T09:16:10.620 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.620 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.620 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.620 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:10.620 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.621 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.621 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.621 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-08-30T09:16:10.621 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.621 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.621 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.621 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.621 INFO:teuthology.orchestra.run.smithi042.stdout: "sleep_interval": { 2024-08-30T09:16:10.621 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sleep_interval", 2024-08-30T09:16:10.621 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-08-30T09:16:10.621 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.621 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.621 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "60", 2024-08-30T09:16:10.621 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.622 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.622 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.622 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-08-30T09:16:10.622 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.622 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.622 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.622 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.622 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-08-30T09:16:10.622 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-08-30T09:16:10.622 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:10.622 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-08-30T09:16:10.622 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.622 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-08-30T09:16:10.622 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.622 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.623 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.623 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.623 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.623 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.623 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.623 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.623 INFO:teuthology.orchestra.run.smithi042.stdout: "upmap_max_deviation": { 2024-08-30T09:16:10.623 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "upmap_max_deviation", 2024-08-30T09:16:10.623 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:10.623 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.623 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.623 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "5", 2024-08-30T09:16:10.623 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "1", 2024-08-30T09:16:10.623 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.624 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.624 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "deviation below which no optimization is attempted", 2024-08-30T09:16:10.624 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-08-30T09:16:10.624 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.624 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.624 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.624 INFO:teuthology.orchestra.run.smithi042.stdout: "upmap_max_optimizations": { 2024-08-30T09:16:10.624 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "upmap_max_optimizations", 2024-08-30T09:16:10.624 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "uint", 2024-08-30T09:16:10.624 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.624 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.624 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "10", 2024-08-30T09:16:10.624 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.625 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.625 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.625 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-08-30T09:16:10.625 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.625 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.625 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.625 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:10.625 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:10.625 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.625 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-08-30T09:16:10.625 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "cephadm", 2024-08-30T09:16:10.625 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-08-30T09:16:10.625 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-08-30T09:16:10.625 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-08-30T09:16:10.626 INFO:teuthology.orchestra.run.smithi042.stdout: "agent_down_multiplier": { 2024-08-30T09:16:10.626 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "agent_down_multiplier", 2024-08-30T09:16:10.626 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "float", 2024-08-30T09:16:10.626 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.626 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.626 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "3.0", 2024-08-30T09:16:10.626 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.626 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.626 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.626 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-08-30T09:16:10.626 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.626 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.627 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.627 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.627 INFO:teuthology.orchestra.run.smithi042.stdout: "agent_refresh_rate": { 2024-08-30T09:16:10.627 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "agent_refresh_rate", 2024-08-30T09:16:10.627 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-08-30T09:16:10.627 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.627 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.627 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "20", 2024-08-30T09:16:10.627 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.627 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.627 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.627 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-08-30T09:16:10.627 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.627 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.628 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.628 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.628 INFO:teuthology.orchestra.run.smithi042.stdout: "agent_starting_port": { 2024-08-30T09:16:10.628 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "agent_starting_port", 2024-08-30T09:16:10.628 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:10.628 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.628 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.628 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "4721", 2024-08-30T09:16:10.628 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.628 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.628 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.628 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-08-30T09:16:10.629 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.629 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.629 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.629 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.629 INFO:teuthology.orchestra.run.smithi042.stdout: "allow_ptrace": { 2024-08-30T09:16:10.629 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "allow_ptrace", 2024-08-30T09:16:10.629 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.629 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.629 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.629 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:10.629 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.629 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.629 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.629 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-08-30T09:16:10.630 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-08-30T09:16:10.630 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.630 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.630 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.630 INFO:teuthology.orchestra.run.smithi042.stdout: "autotune_interval": { 2024-08-30T09:16:10.630 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "autotune_interval", 2024-08-30T09:16:10.630 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-08-30T09:16:10.630 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.630 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.630 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "600", 2024-08-30T09:16:10.630 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.630 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.630 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.630 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "how frequently to autotune daemon memory", 2024-08-30T09:16:10.630 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.631 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.631 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.631 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.631 INFO:teuthology.orchestra.run.smithi042.stdout: "autotune_memory_target_ratio": { 2024-08-30T09:16:10.631 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "autotune_memory_target_ratio", 2024-08-30T09:16:10.631 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "float", 2024-08-30T09:16:10.631 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.631 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.631 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0.7", 2024-08-30T09:16:10.631 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.631 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.631 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.631 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-08-30T09:16:10.631 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.632 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.632 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.632 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.632 INFO:teuthology.orchestra.run.smithi042.stdout: "cephadm_log_destination": { 2024-08-30T09:16:10.632 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "cephadm_log_destination", 2024-08-30T09:16:10.632 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.632 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.632 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.632 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:10.632 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.632 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.632 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:10.632 INFO:teuthology.orchestra.run.smithi042.stdout: "file", 2024-08-30T09:16:10.632 INFO:teuthology.orchestra.run.smithi042.stdout: "file,syslog", 2024-08-30T09:16:10.633 INFO:teuthology.orchestra.run.smithi042.stdout: "syslog" 2024-08-30T09:16:10.633 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:10.633 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-08-30T09:16:10.633 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.633 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.633 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.633 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.633 INFO:teuthology.orchestra.run.smithi042.stdout: "cgroups_split": { 2024-08-30T09:16:10.633 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "cgroups_split", 2024-08-30T09:16:10.633 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.633 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.633 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.634 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-08-30T09:16:10.634 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.634 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.634 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.634 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-08-30T09:16:10.634 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.634 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.634 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.634 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.634 INFO:teuthology.orchestra.run.smithi042.stdout: "config_checks_enabled": { 2024-08-30T09:16:10.634 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "config_checks_enabled", 2024-08-30T09:16:10.634 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.634 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.634 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.634 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:10.635 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.635 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.635 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.635 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-08-30T09:16:10.635 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.635 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.635 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.635 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.635 INFO:teuthology.orchestra.run.smithi042.stdout: "config_dashboard": { 2024-08-30T09:16:10.635 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "config_dashboard", 2024-08-30T09:16:10.635 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.635 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.635 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.635 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-08-30T09:16:10.635 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.636 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.636 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.636 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-08-30T09:16:10.636 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.636 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.636 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.636 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.636 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_alertmanager": { 2024-08-30T09:16:10.636 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_alertmanager", 2024-08-30T09:16:10.636 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.636 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.636 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.636 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.27.0", 2024-08-30T09:16:10.637 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.637 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.637 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.637 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Prometheus container image", 2024-08-30T09:16:10.637 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.637 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.637 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.637 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.637 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_base": { 2024-08-30T09:16:10.637 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_base", 2024-08-30T09:16:10.637 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.637 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.637 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.638 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/ceph/ceph", 2024-08-30T09:16:10.638 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.638 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.638 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.638 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Container image name, without the tag", 2024-08-30T09:16:10.638 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.638 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.638 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.638 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.638 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_elasticsearch": { 2024-08-30T09:16:10.638 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_elasticsearch", 2024-08-30T09:16:10.638 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.638 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.638 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.639 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-08-30T09:16:10.639 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.639 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.639 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.639 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "elasticsearch container image", 2024-08-30T09:16:10.639 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.639 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.639 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.639 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.639 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_grafana": { 2024-08-30T09:16:10.639 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_grafana", 2024-08-30T09:16:10.639 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.639 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.639 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.640 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/ceph/grafana:10.4.0", 2024-08-30T09:16:10.640 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.640 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.640 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.640 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Prometheus container image", 2024-08-30T09:16:10.640 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.640 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.640 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.640 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.640 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_haproxy": { 2024-08-30T09:16:10.640 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_haproxy", 2024-08-30T09:16:10.640 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.640 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.641 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.641 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-08-30T09:16:10.641 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.641 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.641 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.641 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "HAproxy container image", 2024-08-30T09:16:10.641 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.641 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.641 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.641 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.641 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_jaeger_agent": { 2024-08-30T09:16:10.641 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_jaeger_agent", 2024-08-30T09:16:10.641 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.641 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.641 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.642 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-08-30T09:16:10.642 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.642 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.642 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.642 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Jaeger agent container image", 2024-08-30T09:16:10.642 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.642 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.642 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.642 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.642 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_jaeger_collector": { 2024-08-30T09:16:10.642 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_jaeger_collector", 2024-08-30T09:16:10.642 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.642 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.642 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.643 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-08-30T09:16:10.643 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.643 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.643 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.643 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Jaeger collector container image", 2024-08-30T09:16:10.643 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.643 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.643 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.643 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.643 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_jaeger_query": { 2024-08-30T09:16:10.643 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_jaeger_query", 2024-08-30T09:16:10.643 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.643 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.643 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.643 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-08-30T09:16:10.644 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.644 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.644 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.644 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Jaeger query container image", 2024-08-30T09:16:10.644 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.644 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.644 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.644 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.644 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_keepalived": { 2024-08-30T09:16:10.644 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_keepalived", 2024-08-30T09:16:10.644 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.644 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.644 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.644 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-08-30T09:16:10.645 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.645 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.645 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.645 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Keepalived container image", 2024-08-30T09:16:10.645 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.645 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.645 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.645 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.645 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_loki": { 2024-08-30T09:16:10.645 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_loki", 2024-08-30T09:16:10.645 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.645 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.645 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.646 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-08-30T09:16:10.646 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.646 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.646 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.646 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Loki container image", 2024-08-30T09:16:10.646 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.646 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.646 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.646 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.646 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_nginx": { 2024-08-30T09:16:10.646 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_nginx", 2024-08-30T09:16:10.646 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.646 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.646 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.647 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/ceph/nginx:1.26.1", 2024-08-30T09:16:10.647 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.647 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.647 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.647 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Nginx container image", 2024-08-30T09:16:10.647 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.647 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.647 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.647 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.647 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_node_exporter": { 2024-08-30T09:16:10.647 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_node_exporter", 2024-08-30T09:16:10.647 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.647 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.647 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.648 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.7.0", 2024-08-30T09:16:10.648 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.648 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.648 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.648 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Prometheus container image", 2024-08-30T09:16:10.648 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.648 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.648 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.648 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.648 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_nvmeof": { 2024-08-30T09:16:10.648 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_nvmeof", 2024-08-30T09:16:10.648 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.648 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.648 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.648 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.17", 2024-08-30T09:16:10.649 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.649 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.649 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.649 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Nvme-of container image", 2024-08-30T09:16:10.649 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.649 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.649 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.649 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.649 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_oauth2_proxy": { 2024-08-30T09:16:10.649 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_oauth2_proxy", 2024-08-30T09:16:10.649 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.649 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.649 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.650 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/oauth2-proxy/oauth2-proxy:v7.6.0", 2024-08-30T09:16:10.650 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.650 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.650 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.650 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "oauth2-proxy container image", 2024-08-30T09:16:10.650 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.650 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.650 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.650 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.650 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_prometheus": { 2024-08-30T09:16:10.650 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_prometheus", 2024-08-30T09:16:10.650 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.650 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.651 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.651 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/prometheus/prometheus:v2.51.0", 2024-08-30T09:16:10.651 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.651 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.651 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.651 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Prometheus container image", 2024-08-30T09:16:10.651 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.651 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.651 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.651 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.651 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_promtail": { 2024-08-30T09:16:10.651 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_promtail", 2024-08-30T09:16:10.651 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.652 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.652 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.652 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-08-30T09:16:10.652 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.652 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.652 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.652 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Promtail container image", 2024-08-30T09:16:10.652 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.652 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.652 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.652 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.652 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_samba": { 2024-08-30T09:16:10.652 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_samba", 2024-08-30T09:16:10.652 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.653 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.653 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.653 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-08-30T09:16:10.653 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.653 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.653 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.653 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Samba/SMB container image", 2024-08-30T09:16:10.653 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.653 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.653 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.653 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.653 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_snmp_gateway": { 2024-08-30T09:16:10.653 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_snmp_gateway", 2024-08-30T09:16:10.653 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.653 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.654 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.654 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-08-30T09:16:10.654 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.654 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.654 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.654 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "SNMP Gateway container image", 2024-08-30T09:16:10.654 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.654 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.654 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.654 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.654 INFO:teuthology.orchestra.run.smithi042.stdout: "container_init": { 2024-08-30T09:16:10.654 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_init", 2024-08-30T09:16:10.654 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.654 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.655 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.655 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-08-30T09:16:10.655 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.655 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.655 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.655 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Run podman/docker with `--init`", 2024-08-30T09:16:10.655 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.655 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.655 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.655 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.655 INFO:teuthology.orchestra.run.smithi042.stdout: "daemon_cache_timeout": { 2024-08-30T09:16:10.655 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "daemon_cache_timeout", 2024-08-30T09:16:10.655 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-08-30T09:16:10.656 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.656 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.656 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "600", 2024-08-30T09:16:10.656 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.656 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.656 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.656 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-08-30T09:16:10.656 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.656 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.656 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.656 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.656 INFO:teuthology.orchestra.run.smithi042.stdout: "default_cephadm_command_timeout": { 2024-08-30T09:16:10.656 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "default_cephadm_command_timeout", 2024-08-30T09:16:10.657 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:10.657 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.657 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.657 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "900", 2024-08-30T09:16:10.657 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.657 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.657 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.657 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-08-30T09:16:10.657 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.657 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.657 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.657 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.657 INFO:teuthology.orchestra.run.smithi042.stdout: "default_registry": { 2024-08-30T09:16:10.657 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "default_registry", 2024-08-30T09:16:10.657 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.658 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.658 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.658 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "docker.io", 2024-08-30T09:16:10.658 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.658 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.658 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.658 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-08-30T09:16:10.658 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.658 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.658 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.658 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.658 INFO:teuthology.orchestra.run.smithi042.stdout: "device_cache_timeout": { 2024-08-30T09:16:10.658 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "device_cache_timeout", 2024-08-30T09:16:10.659 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-08-30T09:16:10.659 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.659 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.659 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "1800", 2024-08-30T09:16:10.659 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.659 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.659 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.659 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "seconds to cache device inventory", 2024-08-30T09:16:10.659 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.659 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.659 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.659 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.659 INFO:teuthology.orchestra.run.smithi042.stdout: "device_enhanced_scan": { 2024-08-30T09:16:10.660 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "device_enhanced_scan", 2024-08-30T09:16:10.660 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.660 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.660 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.660 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:10.660 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.660 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.660 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.660 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Use libstoragemgmt during device scans", 2024-08-30T09:16:10.660 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.660 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.660 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.660 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.661 INFO:teuthology.orchestra.run.smithi042.stdout: "facts_cache_timeout": { 2024-08-30T09:16:10.661 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "facts_cache_timeout", 2024-08-30T09:16:10.661 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-08-30T09:16:10.661 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.661 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.661 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "60", 2024-08-30T09:16:10.661 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.661 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.661 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.661 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "seconds to cache host facts data", 2024-08-30T09:16:10.661 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.661 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.661 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.662 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.662 INFO:teuthology.orchestra.run.smithi042.stdout: "grafana_dashboards_path": { 2024-08-30T09:16:10.662 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "grafana_dashboards_path", 2024-08-30T09:16:10.662 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.662 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.662 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.662 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-08-30T09:16:10.662 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.662 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.662 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.662 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-08-30T09:16:10.662 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.662 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.662 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.662 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.663 INFO:teuthology.orchestra.run.smithi042.stdout: "host_check_interval": { 2024-08-30T09:16:10.663 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "host_check_interval", 2024-08-30T09:16:10.663 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-08-30T09:16:10.663 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.663 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.663 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "600", 2024-08-30T09:16:10.663 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.663 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.663 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.663 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "how frequently to perform a host check", 2024-08-30T09:16:10.663 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.663 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.663 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.663 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.664 INFO:teuthology.orchestra.run.smithi042.stdout: "hw_monitoring": { 2024-08-30T09:16:10.664 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "hw_monitoring", 2024-08-30T09:16:10.664 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.664 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.664 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.664 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:10.664 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.664 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.664 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.664 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-08-30T09:16:10.664 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.664 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.664 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.664 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.665 INFO:teuthology.orchestra.run.smithi042.stdout: "inventory_list_all": { 2024-08-30T09:16:10.665 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "inventory_list_all", 2024-08-30T09:16:10.665 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.665 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.665 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.665 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:10.665 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.665 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.665 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.665 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-08-30T09:16:10.665 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.665 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.666 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.666 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.666 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-08-30T09:16:10.666 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-08-30T09:16:10.666 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.666 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.666 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.666 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:10.666 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.666 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.666 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:10.666 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:10.666 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:10.666 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:10.666 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:10.667 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:10.667 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:10.667 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:10.667 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.667 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.667 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.667 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.667 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.667 INFO:teuthology.orchestra.run.smithi042.stdout: "log_refresh_metadata": { 2024-08-30T09:16:10.667 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_refresh_metadata", 2024-08-30T09:16:10.667 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.667 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.667 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.667 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:10.667 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.668 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.668 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.668 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-08-30T09:16:10.668 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.668 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.668 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.668 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.668 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-08-30T09:16:10.668 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-08-30T09:16:10.668 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.668 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.668 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.669 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-08-30T09:16:10.669 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.669 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.669 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.669 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-08-30T09:16:10.669 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.669 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.669 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.669 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.669 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-08-30T09:16:10.669 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-08-30T09:16:10.669 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.669 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.670 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.670 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-08-30T09:16:10.670 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.670 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.670 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:10.670 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:10.670 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:10.670 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:10.670 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:10.670 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:10.670 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:10.670 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:10.670 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.670 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.671 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.671 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.671 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.671 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-08-30T09:16:10.671 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-08-30T09:16:10.671 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.671 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.671 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.671 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:10.671 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.671 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.671 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.671 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.671 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.671 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.672 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.672 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.672 INFO:teuthology.orchestra.run.smithi042.stdout: "manage_etc_ceph_ceph_conf": { 2024-08-30T09:16:10.672 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-08-30T09:16:10.672 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.672 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.672 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.672 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:10.672 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.672 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.672 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.672 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-08-30T09:16:10.672 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.672 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.672 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.673 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.673 INFO:teuthology.orchestra.run.smithi042.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-08-30T09:16:10.673 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-08-30T09:16:10.673 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.673 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.673 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.673 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "*", 2024-08-30T09:16:10.673 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.673 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.673 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.673 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-08-30T09:16:10.673 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.673 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.673 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.674 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.674 INFO:teuthology.orchestra.run.smithi042.stdout: "max_count_per_host": { 2024-08-30T09:16:10.674 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "max_count_per_host", 2024-08-30T09:16:10.674 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:10.674 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.674 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.674 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "10", 2024-08-30T09:16:10.674 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.674 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.674 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.674 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "max number of daemons per service per host", 2024-08-30T09:16:10.674 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.674 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.674 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.674 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.675 INFO:teuthology.orchestra.run.smithi042.stdout: "max_osd_draining_count": { 2024-08-30T09:16:10.675 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "max_osd_draining_count", 2024-08-30T09:16:10.675 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:10.675 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.675 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.675 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "10", 2024-08-30T09:16:10.675 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.675 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.675 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.675 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-08-30T09:16:10.675 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.675 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.675 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.675 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.675 INFO:teuthology.orchestra.run.smithi042.stdout: "migration_current": { 2024-08-30T09:16:10.676 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "migration_current", 2024-08-30T09:16:10.676 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:10.676 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.676 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.676 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:10.676 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.676 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.676 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.676 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "internal - do not modify", 2024-08-30T09:16:10.676 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.676 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.676 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.676 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.676 INFO:teuthology.orchestra.run.smithi042.stdout: "mode": { 2024-08-30T09:16:10.677 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "mode", 2024-08-30T09:16:10.677 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.677 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.677 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.677 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "root", 2024-08-30T09:16:10.677 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.677 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.677 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:10.677 INFO:teuthology.orchestra.run.smithi042.stdout: "cephadm-package", 2024-08-30T09:16:10.677 INFO:teuthology.orchestra.run.smithi042.stdout: "root" 2024-08-30T09:16:10.677 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:10.677 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "mode for remote execution of cephadm", 2024-08-30T09:16:10.677 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.678 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.678 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.678 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.678 INFO:teuthology.orchestra.run.smithi042.stdout: "oob_default_addr": { 2024-08-30T09:16:10.678 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "oob_default_addr", 2024-08-30T09:16:10.678 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.678 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.678 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.678 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "169.254.1.1", 2024-08-30T09:16:10.678 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.678 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.678 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.678 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Default address for RedFish API (oob management).", 2024-08-30T09:16:10.678 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.679 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.679 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.679 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.679 INFO:teuthology.orchestra.run.smithi042.stdout: "prometheus_alerts_path": { 2024-08-30T09:16:10.679 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "prometheus_alerts_path", 2024-08-30T09:16:10.679 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.679 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.679 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.679 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-08-30T09:16:10.679 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.679 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.679 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.679 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-08-30T09:16:10.679 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.680 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.680 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.680 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.680 INFO:teuthology.orchestra.run.smithi042.stdout: "registry_insecure": { 2024-08-30T09:16:10.680 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "registry_insecure", 2024-08-30T09:16:10.680 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.680 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.680 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.680 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:10.680 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.680 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.680 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.680 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-08-30T09:16:10.680 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.681 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.681 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.681 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.681 INFO:teuthology.orchestra.run.smithi042.stdout: "registry_password": { 2024-08-30T09:16:10.681 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "registry_password", 2024-08-30T09:16:10.681 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.681 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.681 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.681 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:10.681 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.681 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.681 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.681 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-08-30T09:16:10.681 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.682 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.682 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.682 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.682 INFO:teuthology.orchestra.run.smithi042.stdout: "registry_url": { 2024-08-30T09:16:10.682 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "registry_url", 2024-08-30T09:16:10.682 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.682 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.682 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.682 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:10.682 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.682 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.682 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.682 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-08-30T09:16:10.682 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.682 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.683 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.683 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.683 INFO:teuthology.orchestra.run.smithi042.stdout: "registry_username": { 2024-08-30T09:16:10.683 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "registry_username", 2024-08-30T09:16:10.683 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.683 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.683 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.683 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:10.683 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.683 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.683 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.683 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-08-30T09:16:10.683 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.683 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.683 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.684 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.684 INFO:teuthology.orchestra.run.smithi042.stdout: "secure_monitoring_stack": { 2024-08-30T09:16:10.684 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "secure_monitoring_stack", 2024-08-30T09:16:10.684 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.684 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.684 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.684 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:10.684 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.684 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.684 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.684 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-08-30T09:16:10.684 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.684 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.684 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.685 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.685 INFO:teuthology.orchestra.run.smithi042.stdout: "service_discovery_port": { 2024-08-30T09:16:10.685 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "service_discovery_port", 2024-08-30T09:16:10.685 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:10.685 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.685 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.685 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "8765", 2024-08-30T09:16:10.685 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.685 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.685 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.685 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "cephadm service discovery port", 2024-08-30T09:16:10.685 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.685 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.685 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.686 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.686 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-08-30T09:16:10.686 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-08-30T09:16:10.686 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:10.686 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-08-30T09:16:10.686 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.686 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-08-30T09:16:10.686 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.686 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.686 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.686 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.686 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.686 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.686 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.687 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.687 INFO:teuthology.orchestra.run.smithi042.stdout: "ssh_config_file": { 2024-08-30T09:16:10.687 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ssh_config_file", 2024-08-30T09:16:10.687 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.687 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.687 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.687 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:10.687 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.687 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.687 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.687 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-08-30T09:16:10.687 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.688 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.688 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.688 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.688 INFO:teuthology.orchestra.run.smithi042.stdout: "use_agent": { 2024-08-30T09:16:10.688 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "use_agent", 2024-08-30T09:16:10.688 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.688 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.688 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.688 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:10.688 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.688 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.688 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.688 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-08-30T09:16:10.688 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.689 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.689 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.689 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.689 INFO:teuthology.orchestra.run.smithi042.stdout: "use_repo_digest": { 2024-08-30T09:16:10.689 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "use_repo_digest", 2024-08-30T09:16:10.689 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.689 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.689 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.689 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-08-30T09:16:10.689 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.689 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.689 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.689 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-08-30T09:16:10.689 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.690 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.690 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.690 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.690 INFO:teuthology.orchestra.run.smithi042.stdout: "warn_on_failed_host_check": { 2024-08-30T09:16:10.690 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "warn_on_failed_host_check", 2024-08-30T09:16:10.690 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.690 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.690 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.690 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-08-30T09:16:10.690 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.690 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.690 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.690 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "raise a health warning if the host check fails", 2024-08-30T09:16:10.690 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.691 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.691 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.691 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.691 INFO:teuthology.orchestra.run.smithi042.stdout: "warn_on_stray_daemons": { 2024-08-30T09:16:10.691 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "warn_on_stray_daemons", 2024-08-30T09:16:10.691 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.691 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.691 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.691 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-08-30T09:16:10.691 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.691 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.691 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.691 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-08-30T09:16:10.691 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.691 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.692 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.692 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.692 INFO:teuthology.orchestra.run.smithi042.stdout: "warn_on_stray_hosts": { 2024-08-30T09:16:10.692 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "warn_on_stray_hosts", 2024-08-30T09:16:10.692 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.692 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.692 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.692 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-08-30T09:16:10.692 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.692 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.692 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.692 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-08-30T09:16:10.692 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.692 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.693 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.693 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:10.693 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:10.693 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.693 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-08-30T09:16:10.693 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "crash", 2024-08-30T09:16:10.693 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-08-30T09:16:10.693 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-08-30T09:16:10.693 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-08-30T09:16:10.693 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-08-30T09:16:10.693 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-08-30T09:16:10.693 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.693 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.694 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.694 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:10.694 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.694 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.694 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:10.694 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:10.694 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:10.694 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:10.694 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:10.694 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:10.694 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:10.694 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:10.694 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.694 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.694 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.695 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.695 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.695 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-08-30T09:16:10.695 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-08-30T09:16:10.695 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.695 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.695 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.695 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:10.695 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.695 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.695 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.695 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.695 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.695 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.695 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.696 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.696 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-08-30T09:16:10.696 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-08-30T09:16:10.696 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.696 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.696 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.696 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-08-30T09:16:10.696 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.696 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.696 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:10.696 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:10.696 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:10.696 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:10.696 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:10.697 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:10.697 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:10.697 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:10.697 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.697 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.697 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.697 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.697 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.697 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-08-30T09:16:10.697 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-08-30T09:16:10.697 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.697 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.697 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.697 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:10.698 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.698 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.698 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.698 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.698 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.698 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.698 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.698 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.698 INFO:teuthology.orchestra.run.smithi042.stdout: "retain_interval": { 2024-08-30T09:16:10.698 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "retain_interval", 2024-08-30T09:16:10.698 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-08-30T09:16:10.698 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.698 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.698 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "31536000", 2024-08-30T09:16:10.699 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.699 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.699 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.699 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "how long to retain crashes before pruning them", 2024-08-30T09:16:10.699 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.699 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.699 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.699 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.699 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-08-30T09:16:10.699 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-08-30T09:16:10.699 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:10.699 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-08-30T09:16:10.699 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.699 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-08-30T09:16:10.699 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.700 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.700 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.700 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.700 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.700 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.700 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.700 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.700 INFO:teuthology.orchestra.run.smithi042.stdout: "warn_recent_interval": { 2024-08-30T09:16:10.700 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "warn_recent_interval", 2024-08-30T09:16:10.700 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-08-30T09:16:10.700 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.700 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.700 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "1209600", 2024-08-30T09:16:10.700 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.701 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.701 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.701 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "time interval in which to warn about recent crashes", 2024-08-30T09:16:10.701 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.701 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.701 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.701 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:10.701 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:10.701 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.701 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-08-30T09:16:10.701 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "dashboard", 2024-08-30T09:16:10.701 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-08-30T09:16:10.701 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-08-30T09:16:10.701 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-08-30T09:16:10.701 INFO:teuthology.orchestra.run.smithi042.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-08-30T09:16:10.702 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-08-30T09:16:10.702 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:10.702 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.702 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.702 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "10", 2024-08-30T09:16:10.702 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.702 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.702 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.702 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.702 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.702 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.702 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.702 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.702 INFO:teuthology.orchestra.run.smithi042.stdout: "ALERTMANAGER_API_HOST": { 2024-08-30T09:16:10.703 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ALERTMANAGER_API_HOST", 2024-08-30T09:16:10.703 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.703 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.703 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.703 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:10.703 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.703 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.703 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.703 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.703 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.703 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.703 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.703 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.703 INFO:teuthology.orchestra.run.smithi042.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-08-30T09:16:10.704 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-08-30T09:16:10.704 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.704 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.704 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.704 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-08-30T09:16:10.704 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.704 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.704 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.704 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.704 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.704 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.704 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.704 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.704 INFO:teuthology.orchestra.run.smithi042.stdout: "AUDIT_API_ENABLED": { 2024-08-30T09:16:10.704 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "AUDIT_API_ENABLED", 2024-08-30T09:16:10.705 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.705 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.705 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.705 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:10.705 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.705 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.705 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.705 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.705 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.705 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.705 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.705 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.705 INFO:teuthology.orchestra.run.smithi042.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-08-30T09:16:10.705 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-08-30T09:16:10.706 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.706 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.706 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.706 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-08-30T09:16:10.706 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.706 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.706 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.706 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.706 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.706 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.706 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.706 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.706 INFO:teuthology.orchestra.run.smithi042.stdout: "ENABLE_BROWSABLE_API": { 2024-08-30T09:16:10.706 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ENABLE_BROWSABLE_API", 2024-08-30T09:16:10.706 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.707 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.707 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.707 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-08-30T09:16:10.707 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.707 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.707 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.707 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.707 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.707 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.707 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.707 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.707 INFO:teuthology.orchestra.run.smithi042.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-08-30T09:16:10.707 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-08-30T09:16:10.707 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.707 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.708 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.708 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-08-30T09:16:10.708 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.708 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.708 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.708 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.708 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.708 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.708 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.708 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.708 INFO:teuthology.orchestra.run.smithi042.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-08-30T09:16:10.708 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-08-30T09:16:10.708 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.708 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.709 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.709 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-08-30T09:16:10.709 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.709 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.709 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.709 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.709 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.709 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.709 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.709 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.709 INFO:teuthology.orchestra.run.smithi042.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-08-30T09:16:10.709 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-08-30T09:16:10.709 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.710 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.710 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.710 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-08-30T09:16:10.710 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.710 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.710 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.710 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.710 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.710 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.710 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.710 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.710 INFO:teuthology.orchestra.run.smithi042.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-08-30T09:16:10.710 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-08-30T09:16:10.711 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.711 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.711 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.711 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-08-30T09:16:10.711 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.711 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.711 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.711 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.711 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.711 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.711 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.711 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.712 INFO:teuthology.orchestra.run.smithi042.stdout: "FEATURE_TOGGLE_NFS": { 2024-08-30T09:16:10.712 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-08-30T09:16:10.712 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.712 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.712 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.712 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-08-30T09:16:10.712 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.712 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.712 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.712 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.712 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.712 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.712 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.713 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.713 INFO:teuthology.orchestra.run.smithi042.stdout: "FEATURE_TOGGLE_RBD": { 2024-08-30T09:16:10.713 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-08-30T09:16:10.713 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.713 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.713 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.713 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-08-30T09:16:10.713 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.713 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.713 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.713 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.713 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.714 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.714 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.714 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.714 INFO:teuthology.orchestra.run.smithi042.stdout: "FEATURE_TOGGLE_RGW": { 2024-08-30T09:16:10.714 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-08-30T09:16:10.714 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.714 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.714 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.714 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-08-30T09:16:10.714 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.714 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.714 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.714 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.714 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.715 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.715 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.715 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.715 INFO:teuthology.orchestra.run.smithi042.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-08-30T09:16:10.715 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-08-30T09:16:10.715 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.715 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.715 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.715 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:10.715 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.715 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.715 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.715 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.715 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.715 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.716 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.716 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.716 INFO:teuthology.orchestra.run.smithi042.stdout: "GRAFANA_API_PASSWORD": { 2024-08-30T09:16:10.716 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "GRAFANA_API_PASSWORD", 2024-08-30T09:16:10.716 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.716 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.716 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.716 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "admin", 2024-08-30T09:16:10.716 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.716 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.716 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.716 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.716 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.716 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.716 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.717 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.717 INFO:teuthology.orchestra.run.smithi042.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-08-30T09:16:10.717 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-08-30T09:16:10.717 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.717 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.717 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.717 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-08-30T09:16:10.717 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.717 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.717 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.717 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.717 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.717 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.717 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.717 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.718 INFO:teuthology.orchestra.run.smithi042.stdout: "GRAFANA_API_URL": { 2024-08-30T09:16:10.718 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "GRAFANA_API_URL", 2024-08-30T09:16:10.718 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.718 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.718 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.718 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:10.718 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.718 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.718 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.718 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.718 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.718 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.718 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.718 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.718 INFO:teuthology.orchestra.run.smithi042.stdout: "GRAFANA_API_USERNAME": { 2024-08-30T09:16:10.719 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "GRAFANA_API_USERNAME", 2024-08-30T09:16:10.719 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.719 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.719 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.719 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "admin", 2024-08-30T09:16:10.719 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.719 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.719 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.719 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.719 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.719 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.719 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.720 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.720 INFO:teuthology.orchestra.run.smithi042.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-08-30T09:16:10.720 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-08-30T09:16:10.720 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.720 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.720 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.720 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:10.720 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.720 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.720 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.720 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.720 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.720 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.720 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.720 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.721 INFO:teuthology.orchestra.run.smithi042.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-08-30T09:16:10.721 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-08-30T09:16:10.721 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.721 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.721 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.721 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:10.721 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.721 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.721 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.721 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.721 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.721 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.721 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.722 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.722 INFO:teuthology.orchestra.run.smithi042.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-08-30T09:16:10.722 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-08-30T09:16:10.722 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.722 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.722 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.722 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-08-30T09:16:10.722 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.722 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.722 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.722 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.722 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.722 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.722 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.723 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.723 INFO:teuthology.orchestra.run.smithi042.stdout: "ISSUE_TRACKER_API_KEY": { 2024-08-30T09:16:10.723 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-08-30T09:16:10.723 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.723 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.723 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.723 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:10.723 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.723 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.723 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.723 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.723 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.723 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.723 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.724 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.724 INFO:teuthology.orchestra.run.smithi042.stdout: "MANAGED_BY_CLUSTERS": { 2024-08-30T09:16:10.724 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-08-30T09:16:10.724 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.724 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.724 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.724 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "[]", 2024-08-30T09:16:10.724 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.724 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.724 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.724 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.724 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.724 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.724 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.725 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.725 INFO:teuthology.orchestra.run.smithi042.stdout: "MULTICLUSTER_CONFIG": { 2024-08-30T09:16:10.725 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "MULTICLUSTER_CONFIG", 2024-08-30T09:16:10.725 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.725 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.725 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.725 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "{}", 2024-08-30T09:16:10.725 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.725 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.725 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.725 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.725 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.725 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.725 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.726 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.726 INFO:teuthology.orchestra.run.smithi042.stdout: "PROMETHEUS_API_HOST": { 2024-08-30T09:16:10.726 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PROMETHEUS_API_HOST", 2024-08-30T09:16:10.726 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.726 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.726 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.726 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:10.726 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.726 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.726 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.726 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.726 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.726 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.726 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.726 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.727 INFO:teuthology.orchestra.run.smithi042.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-08-30T09:16:10.727 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-08-30T09:16:10.727 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.727 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.727 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.727 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-08-30T09:16:10.727 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.727 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.727 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.727 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.727 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.727 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.727 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.728 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.728 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-08-30T09:16:10.728 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-08-30T09:16:10.728 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.728 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.728 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.728 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:10.728 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.728 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.728 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.728 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.728 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.728 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.729 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.729 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.729 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-08-30T09:16:10.729 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-08-30T09:16:10.729 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.729 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.729 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.729 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:10.729 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.729 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.729 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.729 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.729 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.730 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.730 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.730 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.730 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-08-30T09:16:10.730 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-08-30T09:16:10.730 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.730 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.730 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.730 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-08-30T09:16:10.730 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.730 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.730 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.730 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.730 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.731 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.731 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.731 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.731 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-08-30T09:16:10.731 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-08-30T09:16:10.731 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.731 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.731 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.731 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-08-30T09:16:10.731 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.731 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.731 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.731 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.731 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.731 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.732 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.732 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.732 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-08-30T09:16:10.732 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-08-30T09:16:10.732 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.732 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.732 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.732 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:10.732 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.732 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.732 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.732 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.732 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.732 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.733 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.733 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.733 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-08-30T09:16:10.733 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-08-30T09:16:10.733 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.733 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.733 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.733 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:10.733 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.733 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.733 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.733 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.733 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.733 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.734 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.734 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.734 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-08-30T09:16:10.734 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-08-30T09:16:10.734 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.734 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.734 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.734 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:10.734 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.734 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.734 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.734 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.734 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.734 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.734 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.735 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.735 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_ENABLED": { 2024-08-30T09:16:10.735 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_ENABLED", 2024-08-30T09:16:10.735 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.735 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.735 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.735 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-08-30T09:16:10.735 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.735 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.735 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.735 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.735 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.735 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.735 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.735 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.736 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-08-30T09:16:10.736 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-08-30T09:16:10.736 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.736 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.736 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.736 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-08-30T09:16:10.736 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.736 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.736 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.736 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.736 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.736 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.736 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.736 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.737 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-08-30T09:16:10.737 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-08-30T09:16:10.737 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:10.737 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.737 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.737 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "10", 2024-08-30T09:16:10.737 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.737 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.737 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.737 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.737 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.737 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.737 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.738 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.738 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-08-30T09:16:10.738 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-08-30T09:16:10.738 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:10.738 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.738 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.738 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "8", 2024-08-30T09:16:10.738 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.738 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.738 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.738 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.738 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.738 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.738 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.739 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.739 INFO:teuthology.orchestra.run.smithi042.stdout: "REST_REQUESTS_TIMEOUT": { 2024-08-30T09:16:10.739 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-08-30T09:16:10.739 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:10.739 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.739 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.739 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "45", 2024-08-30T09:16:10.739 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.739 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.739 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.739 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.739 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.739 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.740 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.740 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.740 INFO:teuthology.orchestra.run.smithi042.stdout: "RGW_API_ACCESS_KEY": { 2024-08-30T09:16:10.740 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "RGW_API_ACCESS_KEY", 2024-08-30T09:16:10.740 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.740 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.740 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.740 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:10.740 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.740 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.740 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.740 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.740 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.740 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.741 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.741 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.741 INFO:teuthology.orchestra.run.smithi042.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-08-30T09:16:10.741 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-08-30T09:16:10.741 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.741 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.741 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.741 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "admin", 2024-08-30T09:16:10.741 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.741 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.741 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.741 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.741 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.741 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.742 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.742 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.742 INFO:teuthology.orchestra.run.smithi042.stdout: "RGW_API_SECRET_KEY": { 2024-08-30T09:16:10.742 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "RGW_API_SECRET_KEY", 2024-08-30T09:16:10.742 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.742 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.742 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.742 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:10.742 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.742 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.742 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.742 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.742 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.742 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.742 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.743 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.743 INFO:teuthology.orchestra.run.smithi042.stdout: "RGW_API_SSL_VERIFY": { 2024-08-30T09:16:10.743 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "RGW_API_SSL_VERIFY", 2024-08-30T09:16:10.743 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.743 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.743 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.743 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-08-30T09:16:10.743 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.743 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.743 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.743 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.743 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.744 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.744 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.744 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.744 INFO:teuthology.orchestra.run.smithi042.stdout: "UNSAFE_TLS_v1_2": { 2024-08-30T09:16:10.744 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "UNSAFE_TLS_v1_2", 2024-08-30T09:16:10.744 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.744 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.744 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.744 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:10.744 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.744 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.744 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.744 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.744 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.745 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.745 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.745 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.745 INFO:teuthology.orchestra.run.smithi042.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-08-30T09:16:10.745 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-08-30T09:16:10.745 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:10.745 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.745 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.745 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-08-30T09:16:10.745 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.745 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.745 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.745 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.746 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.746 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.746 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.746 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.746 INFO:teuthology.orchestra.run.smithi042.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-08-30T09:16:10.746 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-08-30T09:16:10.746 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:10.746 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.746 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.746 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "10", 2024-08-30T09:16:10.746 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.746 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.746 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.746 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.747 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.747 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.747 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.747 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.747 INFO:teuthology.orchestra.run.smithi042.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-08-30T09:16:10.747 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-08-30T09:16:10.747 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:10.747 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.747 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.747 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "5", 2024-08-30T09:16:10.747 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.747 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.747 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.747 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.747 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.748 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.748 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.748 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.748 INFO:teuthology.orchestra.run.smithi042.stdout: "cross_origin_url": { 2024-08-30T09:16:10.748 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "cross_origin_url", 2024-08-30T09:16:10.748 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.748 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.748 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.748 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:10.748 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.748 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.748 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.748 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.748 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.748 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.749 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.749 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.749 INFO:teuthology.orchestra.run.smithi042.stdout: "crt_file": { 2024-08-30T09:16:10.749 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "crt_file", 2024-08-30T09:16:10.749 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.749 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.749 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.749 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:10.749 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.749 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.749 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.749 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.749 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.750 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.750 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.750 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.750 INFO:teuthology.orchestra.run.smithi042.stdout: "debug": { 2024-08-30T09:16:10.750 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "debug", 2024-08-30T09:16:10.750 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.750 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.750 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.750 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:10.750 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.750 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.750 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.750 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Enable/disable debug options", 2024-08-30T09:16:10.750 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.750 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.751 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.751 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.751 INFO:teuthology.orchestra.run.smithi042.stdout: "jwt_token_ttl": { 2024-08-30T09:16:10.751 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "jwt_token_ttl", 2024-08-30T09:16:10.751 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:10.751 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.751 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.751 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "28800", 2024-08-30T09:16:10.751 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.751 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.751 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.751 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.751 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.751 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.752 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.752 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.752 INFO:teuthology.orchestra.run.smithi042.stdout: "key_file": { 2024-08-30T09:16:10.752 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "key_file", 2024-08-30T09:16:10.752 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.752 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.752 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.752 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:10.752 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.752 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.752 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.752 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.752 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.752 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.752 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.753 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.753 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-08-30T09:16:10.753 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-08-30T09:16:10.753 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.753 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.753 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.753 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:10.753 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.753 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.753 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:10.753 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:10.753 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:10.753 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:10.753 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:10.754 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:10.754 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:10.754 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:10.754 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.754 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.754 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.754 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.754 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.754 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-08-30T09:16:10.754 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-08-30T09:16:10.754 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.754 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.754 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.754 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:10.755 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.755 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.755 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.755 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.755 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.755 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.755 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.755 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.755 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-08-30T09:16:10.755 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-08-30T09:16:10.755 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.755 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.755 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.756 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-08-30T09:16:10.756 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.756 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.756 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:10.756 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:10.756 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:10.756 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:10.756 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:10.756 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:10.756 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:10.756 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:10.756 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.757 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.757 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.757 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.757 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.757 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-08-30T09:16:10.757 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-08-30T09:16:10.757 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.757 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.757 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.757 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:10.757 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.757 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.757 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.758 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.758 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.758 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.758 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.758 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.758 INFO:teuthology.orchestra.run.smithi042.stdout: "motd": { 2024-08-30T09:16:10.758 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "motd", 2024-08-30T09:16:10.758 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.758 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.758 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.758 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:10.758 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.758 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.758 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.759 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "The message of the day", 2024-08-30T09:16:10.759 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.759 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.759 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.759 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.759 INFO:teuthology.orchestra.run.smithi042.stdout: "redirect_resolve_ip_addr": { 2024-08-30T09:16:10.759 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "redirect_resolve_ip_addr", 2024-08-30T09:16:10.759 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.759 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.759 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.759 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:10.759 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.759 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.759 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.760 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.760 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.760 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.760 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.760 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.760 INFO:teuthology.orchestra.run.smithi042.stdout: "server_addr": { 2024-08-30T09:16:10.760 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "server_addr", 2024-08-30T09:16:10.760 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.760 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.760 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.760 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "::", 2024-08-30T09:16:10.760 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.760 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.761 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.761 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.761 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.761 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.761 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.761 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.761 INFO:teuthology.orchestra.run.smithi042.stdout: "server_port": { 2024-08-30T09:16:10.761 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "server_port", 2024-08-30T09:16:10.761 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:10.761 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.761 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.761 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "8080", 2024-08-30T09:16:10.762 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.762 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.762 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.762 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.762 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.762 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.762 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.762 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.762 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-08-30T09:16:10.762 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-08-30T09:16:10.762 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:10.762 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-08-30T09:16:10.762 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.762 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-08-30T09:16:10.763 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.763 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.763 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.763 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.763 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.763 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.763 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.763 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.763 INFO:teuthology.orchestra.run.smithi042.stdout: "ssl": { 2024-08-30T09:16:10.763 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ssl", 2024-08-30T09:16:10.763 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.763 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.763 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.763 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-08-30T09:16:10.764 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.764 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.764 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.764 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.764 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.764 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.764 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.764 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.764 INFO:teuthology.orchestra.run.smithi042.stdout: "ssl_server_port": { 2024-08-30T09:16:10.764 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ssl_server_port", 2024-08-30T09:16:10.764 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:10.764 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.764 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.764 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "8443", 2024-08-30T09:16:10.765 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.765 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.765 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.765 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.765 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.765 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.765 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.765 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.765 INFO:teuthology.orchestra.run.smithi042.stdout: "standby_behaviour": { 2024-08-30T09:16:10.765 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "standby_behaviour", 2024-08-30T09:16:10.765 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.765 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.765 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.765 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "redirect", 2024-08-30T09:16:10.766 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.766 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.766 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:10.766 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:10.766 INFO:teuthology.orchestra.run.smithi042.stdout: "redirect" 2024-08-30T09:16:10.766 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:10.766 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.766 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.766 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.766 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.766 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.766 INFO:teuthology.orchestra.run.smithi042.stdout: "standby_error_status_code": { 2024-08-30T09:16:10.766 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "standby_error_status_code", 2024-08-30T09:16:10.766 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:10.767 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.767 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.767 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "500", 2024-08-30T09:16:10.767 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "400", 2024-08-30T09:16:10.767 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "599", 2024-08-30T09:16:10.767 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.767 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.767 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.767 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.767 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.767 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.767 INFO:teuthology.orchestra.run.smithi042.stdout: "url_prefix": { 2024-08-30T09:16:10.767 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "url_prefix", 2024-08-30T09:16:10.767 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.767 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.768 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.768 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:10.768 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.768 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.768 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.768 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.768 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.768 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.768 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.768 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:10.768 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:10.768 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.768 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-08-30T09:16:10.769 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "devicehealth", 2024-08-30T09:16:10.769 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-08-30T09:16:10.769 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-08-30T09:16:10.769 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-08-30T09:16:10.769 INFO:teuthology.orchestra.run.smithi042.stdout: "enable_monitoring": { 2024-08-30T09:16:10.769 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "enable_monitoring", 2024-08-30T09:16:10.769 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.769 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.769 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.769 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-08-30T09:16:10.769 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.769 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.769 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.770 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "monitor device health metrics", 2024-08-30T09:16:10.770 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.770 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.770 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.770 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.770 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-08-30T09:16:10.770 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-08-30T09:16:10.770 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.770 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.770 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.770 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:10.770 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.770 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.771 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:10.771 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:10.771 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:10.771 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:10.771 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:10.771 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:10.771 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:10.771 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:10.771 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.771 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.771 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.771 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.771 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.771 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-08-30T09:16:10.772 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-08-30T09:16:10.772 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.772 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.772 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.772 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:10.772 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.772 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.772 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.772 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.772 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.772 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.772 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.772 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.772 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-08-30T09:16:10.773 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-08-30T09:16:10.773 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.773 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.773 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.773 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-08-30T09:16:10.773 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.773 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.773 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:10.773 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:10.773 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:10.773 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:10.773 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:10.773 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:10.773 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:10.774 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:10.774 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.774 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.774 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.774 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.774 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.774 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-08-30T09:16:10.774 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-08-30T09:16:10.774 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.774 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.774 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.774 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:10.774 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.774 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.775 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.775 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.775 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.775 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.775 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.775 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.775 INFO:teuthology.orchestra.run.smithi042.stdout: "mark_out_threshold": { 2024-08-30T09:16:10.775 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "mark_out_threshold", 2024-08-30T09:16:10.775 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-08-30T09:16:10.775 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.775 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.775 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "2419200", 2024-08-30T09:16:10.775 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.775 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.775 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.776 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-08-30T09:16:10.776 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.776 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.776 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.776 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.776 INFO:teuthology.orchestra.run.smithi042.stdout: "pool_name": { 2024-08-30T09:16:10.776 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "pool_name", 2024-08-30T09:16:10.776 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.776 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.776 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.776 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "device_health_metrics", 2024-08-30T09:16:10.776 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.776 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.776 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.777 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "name of pool in which to store device health metrics", 2024-08-30T09:16:10.777 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.777 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.777 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.777 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.777 INFO:teuthology.orchestra.run.smithi042.stdout: "retention_period": { 2024-08-30T09:16:10.777 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "retention_period", 2024-08-30T09:16:10.777 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-08-30T09:16:10.777 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.777 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.777 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "15552000", 2024-08-30T09:16:10.777 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.777 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.777 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.778 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "how long to retain device health metrics", 2024-08-30T09:16:10.778 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.778 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.778 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.778 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.778 INFO:teuthology.orchestra.run.smithi042.stdout: "scrape_frequency": { 2024-08-30T09:16:10.778 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "scrape_frequency", 2024-08-30T09:16:10.778 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-08-30T09:16:10.778 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.778 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.778 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "86400", 2024-08-30T09:16:10.778 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.778 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.779 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.779 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "how frequently to scrape device health metrics", 2024-08-30T09:16:10.779 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.779 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.779 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.779 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.779 INFO:teuthology.orchestra.run.smithi042.stdout: "self_heal": { 2024-08-30T09:16:10.779 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "self_heal", 2024-08-30T09:16:10.779 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.779 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.779 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.779 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-08-30T09:16:10.779 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.779 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.780 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.780 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-08-30T09:16:10.780 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.780 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.780 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.780 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.780 INFO:teuthology.orchestra.run.smithi042.stdout: "sleep_interval": { 2024-08-30T09:16:10.780 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sleep_interval", 2024-08-30T09:16:10.780 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-08-30T09:16:10.780 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.780 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.780 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "600", 2024-08-30T09:16:10.780 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.780 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.781 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.781 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "how frequently to wake up and check device health", 2024-08-30T09:16:10.781 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.781 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.781 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.781 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.781 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-08-30T09:16:10.781 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-08-30T09:16:10.781 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:10.781 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-08-30T09:16:10.781 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.781 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-08-30T09:16:10.781 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.781 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.782 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.782 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.782 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.782 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.782 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.782 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.782 INFO:teuthology.orchestra.run.smithi042.stdout: "warn_threshold": { 2024-08-30T09:16:10.782 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "warn_threshold", 2024-08-30T09:16:10.782 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-08-30T09:16:10.782 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.782 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.782 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "7257600", 2024-08-30T09:16:10.782 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.782 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.782 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.783 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-08-30T09:16:10.783 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.783 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.783 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.783 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:10.783 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:10.783 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.783 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-08-30T09:16:10.783 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "diskprediction_local", 2024-08-30T09:16:10.783 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-08-30T09:16:10.783 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-08-30T09:16:10.783 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-08-30T09:16:10.783 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-08-30T09:16:10.783 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-08-30T09:16:10.783 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.784 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.784 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.784 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:10.784 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.784 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.784 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:10.784 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:10.784 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:10.784 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:10.784 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:10.784 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:10.784 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:10.784 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:10.785 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.785 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.785 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.785 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.785 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.785 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-08-30T09:16:10.785 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-08-30T09:16:10.785 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.785 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.785 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.785 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:10.785 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.785 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.785 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.786 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.786 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.786 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.786 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.786 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.786 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-08-30T09:16:10.786 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-08-30T09:16:10.786 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.786 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.786 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.786 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-08-30T09:16:10.786 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.786 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.786 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:10.786 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:10.786 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:10.787 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:10.787 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:10.787 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:10.787 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:10.787 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:10.787 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.787 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.787 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.787 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.787 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.787 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-08-30T09:16:10.787 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-08-30T09:16:10.788 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.788 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.788 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.788 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:10.788 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.788 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.788 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.788 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.788 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.788 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.788 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.788 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.788 INFO:teuthology.orchestra.run.smithi042.stdout: "predict_interval": { 2024-08-30T09:16:10.788 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "predict_interval", 2024-08-30T09:16:10.789 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.789 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.789 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.789 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "86400", 2024-08-30T09:16:10.789 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.789 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.789 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.789 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.789 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.789 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.789 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.789 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.789 INFO:teuthology.orchestra.run.smithi042.stdout: "predictor_model": { 2024-08-30T09:16:10.789 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "predictor_model", 2024-08-30T09:16:10.790 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.790 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.790 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.790 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "prophetstor", 2024-08-30T09:16:10.790 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.790 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.790 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.790 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.790 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.790 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.790 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.790 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.790 INFO:teuthology.orchestra.run.smithi042.stdout: "sleep_interval": { 2024-08-30T09:16:10.790 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sleep_interval", 2024-08-30T09:16:10.791 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.791 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.791 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.791 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "600", 2024-08-30T09:16:10.791 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.791 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.791 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.791 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.791 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.791 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.791 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.791 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.791 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-08-30T09:16:10.791 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-08-30T09:16:10.791 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:10.792 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-08-30T09:16:10.792 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.792 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-08-30T09:16:10.792 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.792 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.792 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.792 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.792 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.792 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.792 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.792 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:10.792 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:10.793 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.793 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-08-30T09:16:10.793 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "influx", 2024-08-30T09:16:10.793 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": false, 2024-08-30T09:16:10.793 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "influxdb python module not found", 2024-08-30T09:16:10.793 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-08-30T09:16:10.793 INFO:teuthology.orchestra.run.smithi042.stdout: "batch_size": { 2024-08-30T09:16:10.793 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "batch_size", 2024-08-30T09:16:10.793 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:10.793 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.793 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.793 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "5000", 2024-08-30T09:16:10.793 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.793 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.793 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.794 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-08-30T09:16:10.794 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.794 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.794 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.794 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.794 INFO:teuthology.orchestra.run.smithi042.stdout: "database": { 2024-08-30T09:16:10.794 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "database", 2024-08-30T09:16:10.794 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.794 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.794 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.794 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "ceph", 2024-08-30T09:16:10.794 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.794 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.794 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.795 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-08-30T09:16:10.795 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.795 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.795 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.795 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.795 INFO:teuthology.orchestra.run.smithi042.stdout: "hostname": { 2024-08-30T09:16:10.795 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "hostname", 2024-08-30T09:16:10.795 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.795 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.795 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.795 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:10.795 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.795 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.795 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.795 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "InfluxDB server hostname", 2024-08-30T09:16:10.796 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.796 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.796 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.796 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.796 INFO:teuthology.orchestra.run.smithi042.stdout: "interval": { 2024-08-30T09:16:10.796 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "interval", 2024-08-30T09:16:10.796 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-08-30T09:16:10.796 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.796 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.796 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "30", 2024-08-30T09:16:10.796 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "5", 2024-08-30T09:16:10.796 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.796 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.796 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-08-30T09:16:10.797 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.797 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.797 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.797 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.797 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-08-30T09:16:10.797 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-08-30T09:16:10.797 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.797 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.797 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.797 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:10.797 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.797 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.797 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:10.797 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:10.797 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:10.798 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:10.798 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:10.798 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:10.798 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:10.798 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:10.798 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.798 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.798 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.798 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.798 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.798 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-08-30T09:16:10.798 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-08-30T09:16:10.798 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.798 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.799 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.799 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:10.799 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.799 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.799 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.799 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.799 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.799 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.799 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.799 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.799 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-08-30T09:16:10.799 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-08-30T09:16:10.799 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.799 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.799 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.800 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-08-30T09:16:10.800 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.800 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.800 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:10.800 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:10.800 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:10.800 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:10.800 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:10.800 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:10.800 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:10.800 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:10.800 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.800 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.800 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.801 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.801 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.801 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-08-30T09:16:10.801 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-08-30T09:16:10.801 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.801 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.801 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.801 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:10.801 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.801 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.801 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.801 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.801 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.801 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.801 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.802 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.802 INFO:teuthology.orchestra.run.smithi042.stdout: "password": { 2024-08-30T09:16:10.802 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "password", 2024-08-30T09:16:10.802 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.802 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.802 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.802 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:10.802 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.802 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.802 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.802 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "password of InfluxDB server user", 2024-08-30T09:16:10.802 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.802 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.802 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.803 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.803 INFO:teuthology.orchestra.run.smithi042.stdout: "port": { 2024-08-30T09:16:10.803 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "port", 2024-08-30T09:16:10.803 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:10.803 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.803 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.803 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "8086", 2024-08-30T09:16:10.803 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.803 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.803 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.803 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "InfluxDB server port", 2024-08-30T09:16:10.803 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.803 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.803 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.803 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.804 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-08-30T09:16:10.804 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-08-30T09:16:10.804 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:10.804 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-08-30T09:16:10.804 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.804 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-08-30T09:16:10.804 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.804 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.804 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.804 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.804 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.804 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.804 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.804 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.805 INFO:teuthology.orchestra.run.smithi042.stdout: "ssl": { 2024-08-30T09:16:10.805 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ssl", 2024-08-30T09:16:10.805 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.805 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.805 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.805 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "false", 2024-08-30T09:16:10.805 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.805 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.805 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.805 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-08-30T09:16:10.805 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.805 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.805 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.805 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.806 INFO:teuthology.orchestra.run.smithi042.stdout: "threads": { 2024-08-30T09:16:10.806 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "threads", 2024-08-30T09:16:10.806 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:10.806 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.806 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.806 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "5", 2024-08-30T09:16:10.806 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "1", 2024-08-30T09:16:10.806 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "32", 2024-08-30T09:16:10.806 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.806 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-08-30T09:16:10.806 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.806 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.806 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.806 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.806 INFO:teuthology.orchestra.run.smithi042.stdout: "username": { 2024-08-30T09:16:10.807 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "username", 2024-08-30T09:16:10.807 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.807 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.807 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.807 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:10.807 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.807 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.807 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.807 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "username of InfluxDB server user", 2024-08-30T09:16:10.807 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.807 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.807 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.807 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.807 INFO:teuthology.orchestra.run.smithi042.stdout: "verify_ssl": { 2024-08-30T09:16:10.807 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "verify_ssl", 2024-08-30T09:16:10.808 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.808 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.808 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.808 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "true", 2024-08-30T09:16:10.808 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.808 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.808 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.808 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-08-30T09:16:10.808 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.808 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.808 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.808 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:10.808 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:10.808 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.809 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-08-30T09:16:10.809 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "insights", 2024-08-30T09:16:10.809 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-08-30T09:16:10.809 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-08-30T09:16:10.809 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-08-30T09:16:10.809 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-08-30T09:16:10.809 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-08-30T09:16:10.809 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.809 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.809 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.809 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:10.809 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.809 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.809 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:10.809 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:10.810 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:10.810 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:10.810 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:10.810 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:10.810 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:10.810 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:10.810 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.810 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.810 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.810 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.810 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.810 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-08-30T09:16:10.810 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-08-30T09:16:10.810 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.811 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.811 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.811 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:10.811 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.811 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.811 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.811 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.811 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.811 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.811 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.811 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.811 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-08-30T09:16:10.811 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-08-30T09:16:10.811 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.811 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.812 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.812 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-08-30T09:16:10.812 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.812 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.812 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:10.812 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:10.812 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:10.812 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:10.812 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:10.812 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:10.812 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:10.812 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:10.812 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.812 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.813 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.813 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.813 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.813 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-08-30T09:16:10.813 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-08-30T09:16:10.813 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.813 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.813 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.813 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:10.813 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.813 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.813 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.813 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.814 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.814 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.814 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.814 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.814 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-08-30T09:16:10.814 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-08-30T09:16:10.814 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:10.814 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-08-30T09:16:10.814 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.814 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-08-30T09:16:10.814 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.814 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.814 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.814 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.814 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.815 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.815 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.815 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:10.815 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:10.815 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.815 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-08-30T09:16:10.815 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "iostat", 2024-08-30T09:16:10.815 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-08-30T09:16:10.815 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-08-30T09:16:10.815 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-08-30T09:16:10.815 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-08-30T09:16:10.815 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-08-30T09:16:10.815 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.815 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.815 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.816 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:10.816 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.816 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.816 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:10.816 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:10.816 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:10.816 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:10.816 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:10.816 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:10.816 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:10.816 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:10.816 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.816 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.816 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.817 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.817 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.817 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-08-30T09:16:10.817 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-08-30T09:16:10.817 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.817 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.817 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.817 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:10.817 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.817 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.817 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.817 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.817 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.817 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.817 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.818 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.818 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-08-30T09:16:10.818 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-08-30T09:16:10.818 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.818 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.818 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.818 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-08-30T09:16:10.818 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.818 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.818 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:10.818 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:10.818 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:10.818 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:10.818 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:10.818 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:10.819 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:10.819 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:10.819 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.819 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.819 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.819 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.819 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.819 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-08-30T09:16:10.819 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-08-30T09:16:10.819 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.819 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.819 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.819 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:10.819 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.819 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.820 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.820 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.820 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.820 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.820 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.820 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.820 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-08-30T09:16:10.820 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-08-30T09:16:10.820 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:10.820 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-08-30T09:16:10.820 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.820 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-08-30T09:16:10.820 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.820 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.821 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.821 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.821 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.821 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.821 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.821 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:10.821 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:10.821 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.821 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-08-30T09:16:10.821 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "k8sevents", 2024-08-30T09:16:10.821 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-08-30T09:16:10.821 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-08-30T09:16:10.821 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-08-30T09:16:10.821 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph_event_retention_days": { 2024-08-30T09:16:10.822 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ceph_event_retention_days", 2024-08-30T09:16:10.822 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:10.822 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.822 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.822 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "7", 2024-08-30T09:16:10.822 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.822 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.822 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.822 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Days to hold ceph event information within local cache", 2024-08-30T09:16:10.822 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.822 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.822 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.822 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.822 INFO:teuthology.orchestra.run.smithi042.stdout: "config_check_secs": { 2024-08-30T09:16:10.823 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "config_check_secs", 2024-08-30T09:16:10.823 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:10.823 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.823 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.823 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "10", 2024-08-30T09:16:10.823 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "10", 2024-08-30T09:16:10.823 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.823 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.823 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-08-30T09:16:10.823 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.823 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.823 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.823 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.823 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-08-30T09:16:10.823 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-08-30T09:16:10.824 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.824 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.824 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.824 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:10.824 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.824 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.824 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:10.824 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:10.824 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:10.824 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:10.824 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:10.824 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:10.824 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:10.824 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:10.825 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.825 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.825 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.825 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.825 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.825 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-08-30T09:16:10.825 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-08-30T09:16:10.825 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.825 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.825 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.825 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:10.825 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.825 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.826 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.826 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.826 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.826 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.826 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.826 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.826 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-08-30T09:16:10.826 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-08-30T09:16:10.826 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.826 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.827 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.827 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-08-30T09:16:10.827 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.827 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.827 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:10.827 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:10.828 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:10.828 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:10.828 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:10.828 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:10.828 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:10.828 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:10.829 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.829 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.829 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.829 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.829 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.829 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-08-30T09:16:10.829 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-08-30T09:16:10.829 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.830 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.830 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.830 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:10.830 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.830 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.830 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.831 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.831 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.831 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.831 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.831 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.831 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-08-30T09:16:10.831 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-08-30T09:16:10.831 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:10.832 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-08-30T09:16:10.832 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.832 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-08-30T09:16:10.832 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.832 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.832 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.832 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.833 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.833 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.833 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.833 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:10.833 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:10.833 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.834 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-08-30T09:16:10.834 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "localpool", 2024-08-30T09:16:10.834 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-08-30T09:16:10.834 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-08-30T09:16:10.834 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-08-30T09:16:10.835 INFO:teuthology.orchestra.run.smithi042.stdout: "failure_domain": { 2024-08-30T09:16:10.835 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "failure_domain", 2024-08-30T09:16:10.835 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.835 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.835 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.835 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "host", 2024-08-30T09:16:10.836 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.836 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.836 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.836 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "failure domain for any created local pool", 2024-08-30T09:16:10.836 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-08-30T09:16:10.836 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.837 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.837 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.837 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-08-30T09:16:10.837 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-08-30T09:16:10.837 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.837 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.838 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.838 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:10.838 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.838 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.838 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:10.838 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:10.839 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:10.839 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:10.839 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:10.839 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:10.839 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:10.839 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:10.840 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.840 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.840 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.840 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.840 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.840 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-08-30T09:16:10.841 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-08-30T09:16:10.841 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.841 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.841 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.841 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:10.841 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.842 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.842 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.842 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.842 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.842 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.842 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.843 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.843 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-08-30T09:16:10.843 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-08-30T09:16:10.843 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.843 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.843 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.844 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-08-30T09:16:10.844 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.844 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.844 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:10.844 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:10.844 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:10.845 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:10.845 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:10.845 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:10.845 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:10.845 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:10.846 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.846 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.846 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.846 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.846 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.846 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-08-30T09:16:10.847 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-08-30T09:16:10.847 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.847 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.847 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.847 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:10.847 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.848 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.848 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.848 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.848 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.848 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.848 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.849 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.849 INFO:teuthology.orchestra.run.smithi042.stdout: "min_size": { 2024-08-30T09:16:10.849 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "min_size", 2024-08-30T09:16:10.849 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:10.849 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.850 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.850 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:10.850 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.850 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.850 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.850 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "default min_size for any created local pool", 2024-08-30T09:16:10.851 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-08-30T09:16:10.851 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.851 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.851 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.851 INFO:teuthology.orchestra.run.smithi042.stdout: "num_rep": { 2024-08-30T09:16:10.851 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "num_rep", 2024-08-30T09:16:10.852 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:10.852 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.852 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.852 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "3", 2024-08-30T09:16:10.852 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.853 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.853 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.853 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "default replica count for any created local pool", 2024-08-30T09:16:10.853 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.853 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.853 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.854 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.854 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num": { 2024-08-30T09:16:10.854 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "pg_num", 2024-08-30T09:16:10.854 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:10.854 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.854 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.854 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "128", 2024-08-30T09:16:10.855 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.855 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.855 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.855 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "default pg_num for any created local pool", 2024-08-30T09:16:10.855 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.855 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.856 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.856 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.856 INFO:teuthology.orchestra.run.smithi042.stdout: "prefix": { 2024-08-30T09:16:10.856 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "prefix", 2024-08-30T09:16:10.856 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.857 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.857 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.857 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:10.857 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.857 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.857 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.858 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "name prefix for any created local pool", 2024-08-30T09:16:10.858 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.858 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.858 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.858 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.859 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-08-30T09:16:10.859 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-08-30T09:16:10.859 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:10.859 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-08-30T09:16:10.859 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.859 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-08-30T09:16:10.860 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.860 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.860 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.860 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.860 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.861 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.861 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.861 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.861 INFO:teuthology.orchestra.run.smithi042.stdout: "subtree": { 2024-08-30T09:16:10.861 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "subtree", 2024-08-30T09:16:10.861 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.862 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.862 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.862 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "rack", 2024-08-30T09:16:10.862 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.862 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.862 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.863 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "CRUSH level for which to create a local pool", 2024-08-30T09:16:10.863 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-08-30T09:16:10.863 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.863 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.863 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:10.863 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:10.864 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.864 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-08-30T09:16:10.864 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "mds_autoscaler", 2024-08-30T09:16:10.864 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-08-30T09:16:10.864 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-08-30T09:16:10.864 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-08-30T09:16:10.865 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-08-30T09:16:10.865 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-08-30T09:16:10.865 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.865 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.865 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.865 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:10.866 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.866 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.866 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:10.866 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:10.866 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:10.866 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:10.867 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:10.867 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:10.867 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:10.867 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:10.867 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.867 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.867 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.868 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.868 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.868 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-08-30T09:16:10.868 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-08-30T09:16:10.868 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.868 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.869 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.869 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:10.869 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.869 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.869 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.870 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.870 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.870 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.870 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.870 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.870 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-08-30T09:16:10.871 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-08-30T09:16:10.871 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.871 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.871 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.871 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-08-30T09:16:10.871 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.872 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.872 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:10.872 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:10.872 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:10.872 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:10.872 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:10.872 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:10.873 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:10.873 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:10.873 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.873 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.873 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.873 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.874 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.874 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-08-30T09:16:10.874 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-08-30T09:16:10.874 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.874 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.874 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.875 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:10.875 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.875 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.875 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.875 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.875 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.875 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.876 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.876 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.876 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-08-30T09:16:10.876 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-08-30T09:16:10.876 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:10.876 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-08-30T09:16:10.877 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.877 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-08-30T09:16:10.877 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.877 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.877 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.877 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.878 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.878 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.878 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.878 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:10.878 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:10.879 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.879 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-08-30T09:16:10.879 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "mirroring", 2024-08-30T09:16:10.879 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-08-30T09:16:10.879 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-08-30T09:16:10.879 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-08-30T09:16:10.880 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-08-30T09:16:10.880 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-08-30T09:16:10.880 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.880 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.880 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.880 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:10.880 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.881 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.881 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:10.881 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:10.881 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:10.881 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:10.881 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:10.882 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:10.882 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:10.882 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:10.882 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.882 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.882 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.883 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.883 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.883 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-08-30T09:16:10.883 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-08-30T09:16:10.883 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.883 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.884 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.884 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:10.884 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.884 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.884 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.884 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.884 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.885 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.885 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.885 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.885 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-08-30T09:16:10.885 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-08-30T09:16:10.885 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.885 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.885 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.886 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-08-30T09:16:10.886 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.886 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.886 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:10.886 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:10.886 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:10.886 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:10.886 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:10.887 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:10.887 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:10.887 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:10.887 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.887 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.887 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.887 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.887 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.887 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-08-30T09:16:10.888 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-08-30T09:16:10.888 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.888 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.888 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.888 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:10.888 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.888 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.888 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.889 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.889 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.889 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.889 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.889 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.889 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-08-30T09:16:10.889 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-08-30T09:16:10.889 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:10.890 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-08-30T09:16:10.890 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.890 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-08-30T09:16:10.890 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.890 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.890 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.891 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.891 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.891 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.891 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.891 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:10.891 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:10.892 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.892 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-08-30T09:16:10.892 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "nfs", 2024-08-30T09:16:10.892 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-08-30T09:16:10.892 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-08-30T09:16:10.892 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-08-30T09:16:10.893 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-08-30T09:16:10.893 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-08-30T09:16:10.893 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.893 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.893 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.893 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:10.894 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.894 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.894 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:10.894 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:10.894 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:10.894 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:10.894 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:10.895 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:10.895 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:10.895 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:10.895 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.895 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.895 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.896 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.896 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.896 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-08-30T09:16:10.896 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-08-30T09:16:10.896 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.896 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.896 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.896 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:10.896 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.896 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.896 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.896 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.896 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.897 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.897 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.897 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.897 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-08-30T09:16:10.897 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-08-30T09:16:10.897 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.898 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.898 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.898 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-08-30T09:16:10.898 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.898 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.898 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:10.898 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:10.899 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:10.899 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:10.899 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:10.899 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:10.899 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:10.899 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:10.900 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.900 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.900 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.900 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.900 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.900 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-08-30T09:16:10.901 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-08-30T09:16:10.901 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.901 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.901 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.901 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:10.901 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.901 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.901 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.902 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.902 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.902 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.902 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.902 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.902 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-08-30T09:16:10.902 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-08-30T09:16:10.903 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:10.903 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-08-30T09:16:10.903 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.903 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-08-30T09:16:10.903 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.903 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.903 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.903 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.904 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.904 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.904 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.904 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:10.904 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:10.904 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.904 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-08-30T09:16:10.905 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "orchestrator", 2024-08-30T09:16:10.905 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-08-30T09:16:10.905 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-08-30T09:16:10.905 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-08-30T09:16:10.905 INFO:teuthology.orchestra.run.smithi042.stdout: "fail_fs": { 2024-08-30T09:16:10.905 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "fail_fs", 2024-08-30T09:16:10.906 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.906 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.906 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.906 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:10.906 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.906 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.906 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.907 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-08-30T09:16:10.907 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.907 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.907 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.907 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.907 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-08-30T09:16:10.907 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-08-30T09:16:10.908 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.908 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.908 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.908 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:10.908 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.908 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.908 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:10.909 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:10.909 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:10.909 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:10.909 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:10.909 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:10.909 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:10.909 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:10.910 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.910 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.910 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.910 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.910 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.910 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-08-30T09:16:10.910 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-08-30T09:16:10.911 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.911 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.911 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.911 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:10.911 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.911 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.911 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.911 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.911 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.911 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.911 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.911 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.911 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-08-30T09:16:10.912 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-08-30T09:16:10.912 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.912 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.912 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.912 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-08-30T09:16:10.912 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.912 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.912 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:10.912 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:10.912 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:10.912 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:10.912 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:10.912 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:10.913 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:10.913 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:10.913 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.913 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.913 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.913 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.913 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.913 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-08-30T09:16:10.913 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-08-30T09:16:10.913 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.913 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.913 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.914 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:10.914 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.914 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.914 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.914 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.914 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.914 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.914 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.914 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.914 INFO:teuthology.orchestra.run.smithi042.stdout: "orchestrator": { 2024-08-30T09:16:10.914 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "orchestrator", 2024-08-30T09:16:10.914 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.914 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.914 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.915 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:10.915 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.915 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.915 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:10.915 INFO:teuthology.orchestra.run.smithi042.stdout: "cephadm", 2024-08-30T09:16:10.915 INFO:teuthology.orchestra.run.smithi042.stdout: "rook", 2024-08-30T09:16:10.915 INFO:teuthology.orchestra.run.smithi042.stdout: "test_orchestrator" 2024-08-30T09:16:10.915 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:10.915 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Orchestrator backend", 2024-08-30T09:16:10.915 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.915 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.915 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.915 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.915 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-08-30T09:16:10.915 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-08-30T09:16:10.916 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:10.916 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-08-30T09:16:10.916 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.916 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-08-30T09:16:10.916 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.916 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.916 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.916 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.916 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.916 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.916 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.916 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:10.916 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:10.916 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.917 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-08-30T09:16:10.917 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "osd_perf_query", 2024-08-30T09:16:10.917 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-08-30T09:16:10.917 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-08-30T09:16:10.917 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-08-30T09:16:10.917 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-08-30T09:16:10.917 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-08-30T09:16:10.917 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.917 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.917 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.917 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:10.917 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.917 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.917 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:10.918 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:10.918 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:10.918 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:10.918 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:10.918 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:10.918 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:10.918 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:10.918 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.918 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.918 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.918 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.918 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.918 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-08-30T09:16:10.918 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-08-30T09:16:10.918 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.919 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.919 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.919 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:10.919 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.919 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.919 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.919 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.919 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.919 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.919 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.919 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.919 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-08-30T09:16:10.920 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-08-30T09:16:10.920 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.920 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.920 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.920 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-08-30T09:16:10.920 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.920 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.920 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:10.920 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:10.920 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:10.920 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:10.920 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:10.920 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:10.921 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:10.921 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:10.921 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.921 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.921 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.921 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.921 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.921 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-08-30T09:16:10.921 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-08-30T09:16:10.921 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.921 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.921 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.921 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:10.921 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.922 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.922 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.922 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.922 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.922 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.922 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.922 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.922 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-08-30T09:16:10.922 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-08-30T09:16:10.922 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:10.922 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-08-30T09:16:10.922 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.922 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-08-30T09:16:10.923 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.923 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.923 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.923 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.923 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.923 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.923 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.923 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:10.923 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:10.923 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.923 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-08-30T09:16:10.923 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "osd_support", 2024-08-30T09:16:10.923 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-08-30T09:16:10.923 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-08-30T09:16:10.923 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-08-30T09:16:10.924 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-08-30T09:16:10.924 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-08-30T09:16:10.924 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.924 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.924 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.924 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:10.924 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.924 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.924 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:10.924 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:10.924 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:10.924 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:10.924 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:10.924 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:10.925 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:10.925 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:10.925 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.925 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.925 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.925 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.925 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.925 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-08-30T09:16:10.925 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-08-30T09:16:10.925 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.925 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.925 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.925 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:10.925 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.926 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.926 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.926 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.926 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.926 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.926 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.926 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.926 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-08-30T09:16:10.926 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-08-30T09:16:10.926 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.926 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.926 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.926 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-08-30T09:16:10.926 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.926 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.927 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:10.927 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:10.927 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:10.927 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:10.927 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:10.927 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:10.927 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:10.927 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:10.927 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.927 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.927 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.927 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.927 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.927 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-08-30T09:16:10.927 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-08-30T09:16:10.928 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.928 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.928 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.928 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:10.928 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.928 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.928 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.928 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.928 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.928 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.928 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.928 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.928 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-08-30T09:16:10.928 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-08-30T09:16:10.929 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:10.929 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-08-30T09:16:10.929 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.929 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-08-30T09:16:10.929 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.929 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.929 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.929 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.929 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.929 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.929 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.929 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:10.929 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:10.930 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.930 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-08-30T09:16:10.930 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "pg_autoscaler", 2024-08-30T09:16:10.930 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-08-30T09:16:10.930 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-08-30T09:16:10.930 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-08-30T09:16:10.930 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-08-30T09:16:10.930 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-08-30T09:16:10.930 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.930 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.930 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.930 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:10.930 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.930 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.930 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:10.931 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:10.931 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:10.931 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:10.931 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:10.931 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:10.931 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:10.931 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:10.931 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.931 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.931 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.931 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.931 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.931 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-08-30T09:16:10.931 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-08-30T09:16:10.932 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.932 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.932 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.932 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:10.932 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.932 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.932 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.932 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.932 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.932 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.932 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.932 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.932 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-08-30T09:16:10.932 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-08-30T09:16:10.932 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.933 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.933 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.933 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-08-30T09:16:10.933 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.933 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.933 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:10.933 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:10.933 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:10.933 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:10.933 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:10.933 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:10.933 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:10.933 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:10.933 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.934 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.934 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.934 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.934 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.934 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-08-30T09:16:10.934 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-08-30T09:16:10.934 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.934 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.934 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.934 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:10.934 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.934 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.934 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.934 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.935 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.935 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.935 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.935 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.935 INFO:teuthology.orchestra.run.smithi042.stdout: "sleep_interval": { 2024-08-30T09:16:10.935 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sleep_interval", 2024-08-30T09:16:10.935 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-08-30T09:16:10.935 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.935 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.935 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "60", 2024-08-30T09:16:10.935 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.935 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.935 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.935 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.935 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.936 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.936 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.936 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.936 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-08-30T09:16:10.936 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-08-30T09:16:10.936 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:10.936 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-08-30T09:16:10.936 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.936 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-08-30T09:16:10.936 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.936 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.936 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.936 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.936 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.937 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.937 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.937 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.937 INFO:teuthology.orchestra.run.smithi042.stdout: "threshold": { 2024-08-30T09:16:10.937 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "threshold", 2024-08-30T09:16:10.937 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "float", 2024-08-30T09:16:10.937 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.937 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.937 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "3.0", 2024-08-30T09:16:10.937 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "1.0", 2024-08-30T09:16:10.937 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.937 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.937 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "scaling threshold", 2024-08-30T09:16:10.937 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-08-30T09:16:10.938 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.938 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.938 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:10.938 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:10.938 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.938 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-08-30T09:16:10.938 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "progress", 2024-08-30T09:16:10.938 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-08-30T09:16:10.938 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-08-30T09:16:10.938 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-08-30T09:16:10.938 INFO:teuthology.orchestra.run.smithi042.stdout: "allow_pg_recovery_event": { 2024-08-30T09:16:10.938 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "allow_pg_recovery_event", 2024-08-30T09:16:10.938 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.938 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.939 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.939 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:10.939 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.939 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.939 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.939 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "allow the module to show pg recovery progress", 2024-08-30T09:16:10.939 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.939 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.939 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.939 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.939 INFO:teuthology.orchestra.run.smithi042.stdout: "enabled": { 2024-08-30T09:16:10.939 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "enabled", 2024-08-30T09:16:10.939 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.939 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.939 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.940 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-08-30T09:16:10.940 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.940 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.940 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.940 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.940 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.940 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.940 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.940 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.940 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-08-30T09:16:10.940 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-08-30T09:16:10.940 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.940 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.940 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.941 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:10.941 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.941 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.941 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:10.941 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:10.941 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:10.941 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:10.941 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:10.941 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:10.941 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:10.941 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:10.941 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.941 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.942 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.942 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.942 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.942 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-08-30T09:16:10.942 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-08-30T09:16:10.942 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.942 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.942 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.942 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:10.942 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.942 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.942 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.942 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.942 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.942 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.943 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.943 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.943 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-08-30T09:16:10.943 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-08-30T09:16:10.943 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.943 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.943 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.943 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-08-30T09:16:10.943 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.943 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.943 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:10.943 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:10.943 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:10.943 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:10.943 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:10.944 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:10.944 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:10.944 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:10.944 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.944 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.944 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.944 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.944 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.944 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-08-30T09:16:10.944 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-08-30T09:16:10.944 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.944 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.944 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.944 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:10.945 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.945 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.945 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.945 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.945 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.945 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.945 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.945 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.945 INFO:teuthology.orchestra.run.smithi042.stdout: "max_completed_events": { 2024-08-30T09:16:10.945 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "max_completed_events", 2024-08-30T09:16:10.945 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:10.945 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.945 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.945 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "50", 2024-08-30T09:16:10.946 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.946 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.946 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.946 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "number of past completed events to remember", 2024-08-30T09:16:10.946 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.946 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.946 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.946 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.946 INFO:teuthology.orchestra.run.smithi042.stdout: "sleep_interval": { 2024-08-30T09:16:10.946 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sleep_interval", 2024-08-30T09:16:10.946 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-08-30T09:16:10.946 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.946 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.946 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "5", 2024-08-30T09:16:10.946 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.947 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.947 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.947 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "how long the module is going to sleep", 2024-08-30T09:16:10.947 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.947 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.947 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.947 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.947 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-08-30T09:16:10.947 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-08-30T09:16:10.947 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:10.947 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-08-30T09:16:10.947 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.947 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-08-30T09:16:10.947 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.948 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.948 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.948 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.948 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.948 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.948 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.948 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:10.948 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:10.948 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.948 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-08-30T09:16:10.948 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "prometheus", 2024-08-30T09:16:10.948 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-08-30T09:16:10.948 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-08-30T09:16:10.948 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-08-30T09:16:10.948 INFO:teuthology.orchestra.run.smithi042.stdout: "cache": { 2024-08-30T09:16:10.949 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "cache", 2024-08-30T09:16:10.949 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.949 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.949 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.949 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-08-30T09:16:10.949 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.949 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.949 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.949 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.949 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.949 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.949 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.949 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.949 INFO:teuthology.orchestra.run.smithi042.stdout: "exclude_perf_counters": { 2024-08-30T09:16:10.950 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "exclude_perf_counters", 2024-08-30T09:16:10.950 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.950 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.950 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.950 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-08-30T09:16:10.950 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.950 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.950 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.950 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-08-30T09:16:10.950 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-08-30T09:16:10.950 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.950 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.950 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.951 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-08-30T09:16:10.951 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-08-30T09:16:10.951 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.951 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.951 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.951 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:10.951 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.951 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.951 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:10.951 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:10.951 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:10.951 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:10.951 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:10.951 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:10.951 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:10.952 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:10.952 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.952 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.952 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.952 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.952 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.952 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-08-30T09:16:10.952 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-08-30T09:16:10.952 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.952 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.952 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.952 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:10.952 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.952 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.953 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.953 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.953 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.953 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.953 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.953 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.953 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-08-30T09:16:10.953 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-08-30T09:16:10.953 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.953 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.953 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.953 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-08-30T09:16:10.953 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.953 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.954 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:10.954 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:10.954 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:10.954 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:10.954 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:10.954 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:10.954 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:10.954 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:10.954 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.954 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.954 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.954 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.954 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.954 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-08-30T09:16:10.954 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-08-30T09:16:10.955 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.955 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.955 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.955 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:10.955 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.955 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.955 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.955 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.955 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.955 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.955 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.955 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.955 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_stats_pools": { 2024-08-30T09:16:10.956 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rbd_stats_pools", 2024-08-30T09:16:10.956 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.956 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.956 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.956 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:10.956 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.956 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.956 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.956 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.956 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.956 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.956 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.956 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.956 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_stats_pools_refresh_interval": { 2024-08-30T09:16:10.957 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-08-30T09:16:10.957 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:10.957 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.957 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.957 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "300", 2024-08-30T09:16:10.957 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.957 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.957 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.957 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.957 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.957 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.957 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.957 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.957 INFO:teuthology.orchestra.run.smithi042.stdout: "scrape_interval": { 2024-08-30T09:16:10.958 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "scrape_interval", 2024-08-30T09:16:10.958 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "float", 2024-08-30T09:16:10.958 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.958 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.958 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "15.0", 2024-08-30T09:16:10.958 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.958 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.958 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.958 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.958 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.958 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.958 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.958 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.958 INFO:teuthology.orchestra.run.smithi042.stdout: "server_addr": { 2024-08-30T09:16:10.958 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "server_addr", 2024-08-30T09:16:10.959 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.959 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.959 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.959 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "::", 2024-08-30T09:16:10.959 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.959 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.959 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.959 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-08-30T09:16:10.959 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.959 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.959 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.959 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.959 INFO:teuthology.orchestra.run.smithi042.stdout: "server_port": { 2024-08-30T09:16:10.959 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "server_port", 2024-08-30T09:16:10.960 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:10.960 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.960 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.960 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "9283", 2024-08-30T09:16:10.960 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.960 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.960 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.960 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-08-30T09:16:10.960 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.960 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.960 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.960 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.960 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-08-30T09:16:10.960 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-08-30T09:16:10.961 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:10.961 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-08-30T09:16:10.961 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.961 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-08-30T09:16:10.961 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.961 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.961 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.961 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.961 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.961 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.961 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.961 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.961 INFO:teuthology.orchestra.run.smithi042.stdout: "stale_cache_strategy": { 2024-08-30T09:16:10.962 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "stale_cache_strategy", 2024-08-30T09:16:10.962 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.962 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.962 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.962 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "log", 2024-08-30T09:16:10.962 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.962 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.962 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.962 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.962 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.962 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.962 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.962 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.962 INFO:teuthology.orchestra.run.smithi042.stdout: "standby_behaviour": { 2024-08-30T09:16:10.963 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "standby_behaviour", 2024-08-30T09:16:10.963 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.963 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.963 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.963 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "default", 2024-08-30T09:16:10.963 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.963 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.963 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:10.963 INFO:teuthology.orchestra.run.smithi042.stdout: "default", 2024-08-30T09:16:10.963 INFO:teuthology.orchestra.run.smithi042.stdout: "error" 2024-08-30T09:16:10.963 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:10.963 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.963 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.963 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.964 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.964 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.964 INFO:teuthology.orchestra.run.smithi042.stdout: "standby_error_status_code": { 2024-08-30T09:16:10.964 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "standby_error_status_code", 2024-08-30T09:16:10.964 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:10.964 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.964 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.964 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "500", 2024-08-30T09:16:10.964 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "400", 2024-08-30T09:16:10.964 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "599", 2024-08-30T09:16:10.964 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.964 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.964 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.964 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.964 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.965 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:10.965 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:10.965 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.965 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-08-30T09:16:10.965 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rbd_support", 2024-08-30T09:16:10.965 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-08-30T09:16:10.965 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-08-30T09:16:10.965 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-08-30T09:16:10.965 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-08-30T09:16:10.965 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-08-30T09:16:10.965 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.965 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.965 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.965 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:10.965 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.966 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.966 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:10.966 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:10.966 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:10.966 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:10.966 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:10.966 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:10.966 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:10.966 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:10.966 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.966 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.966 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.966 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.966 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.967 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-08-30T09:16:10.967 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-08-30T09:16:10.967 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.967 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.967 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.967 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:10.967 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.967 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.967 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.967 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.967 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.967 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.967 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.967 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.968 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-08-30T09:16:10.968 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-08-30T09:16:10.968 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.968 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.968 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.968 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-08-30T09:16:10.968 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.968 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.968 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:10.968 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:10.968 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:10.968 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:10.968 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:10.968 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:10.968 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:10.969 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:10.969 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.969 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.969 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.969 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.969 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.969 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-08-30T09:16:10.969 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-08-30T09:16:10.969 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.969 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.969 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.969 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:10.970 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.970 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.970 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.970 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.970 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.970 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.970 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.970 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.970 INFO:teuthology.orchestra.run.smithi042.stdout: "max_concurrent_snap_create": { 2024-08-30T09:16:10.970 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "max_concurrent_snap_create", 2024-08-30T09:16:10.970 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:10.970 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.970 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.970 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "10", 2024-08-30T09:16:10.970 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.971 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.971 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.971 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.971 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.971 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.971 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.971 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.971 INFO:teuthology.orchestra.run.smithi042.stdout: "mirror_snapshot_schedule": { 2024-08-30T09:16:10.971 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "mirror_snapshot_schedule", 2024-08-30T09:16:10.971 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.971 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.971 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.971 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:10.972 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.972 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.972 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.972 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.972 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.972 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.972 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.972 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.972 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-08-30T09:16:10.972 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-08-30T09:16:10.972 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:10.972 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-08-30T09:16:10.972 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.972 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-08-30T09:16:10.972 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.973 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.973 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.973 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.973 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.973 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.973 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.973 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.973 INFO:teuthology.orchestra.run.smithi042.stdout: "trash_purge_schedule": { 2024-08-30T09:16:10.973 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "trash_purge_schedule", 2024-08-30T09:16:10.973 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.973 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.973 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.973 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:10.973 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.974 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.974 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.974 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.974 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.974 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.974 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.974 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:10.974 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:10.974 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.974 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-08-30T09:16:10.974 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "restful", 2024-08-30T09:16:10.974 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-08-30T09:16:10.974 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-08-30T09:16:10.974 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-08-30T09:16:10.974 INFO:teuthology.orchestra.run.smithi042.stdout: "enable_auth": { 2024-08-30T09:16:10.975 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "enable_auth", 2024-08-30T09:16:10.975 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.975 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.975 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.975 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-08-30T09:16:10.975 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.975 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.975 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.975 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.975 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.975 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.975 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.975 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.975 INFO:teuthology.orchestra.run.smithi042.stdout: "key_file": { 2024-08-30T09:16:10.975 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "key_file", 2024-08-30T09:16:10.976 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.976 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.976 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.976 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:10.976 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.976 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.976 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.976 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.976 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.976 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.976 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.976 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.976 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-08-30T09:16:10.976 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-08-30T09:16:10.977 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.977 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.977 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.977 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:10.977 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.977 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.977 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:10.977 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:10.977 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:10.977 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:10.977 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:10.977 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:10.977 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:10.977 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:10.977 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.978 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.978 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.978 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.978 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.978 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-08-30T09:16:10.978 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-08-30T09:16:10.978 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.978 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.978 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.978 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:10.978 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.978 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.978 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.978 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.979 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.979 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.979 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.979 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.979 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-08-30T09:16:10.979 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-08-30T09:16:10.979 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.979 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.979 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.979 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-08-30T09:16:10.979 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.979 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.979 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:10.979 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:10.980 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:10.980 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:10.980 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:10.980 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:10.980 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:10.980 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:10.980 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.980 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.980 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.980 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.980 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.980 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-08-30T09:16:10.980 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-08-30T09:16:10.981 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.981 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.981 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.981 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:10.981 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.981 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.981 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.981 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.981 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.981 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.981 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.981 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.981 INFO:teuthology.orchestra.run.smithi042.stdout: "max_requests": { 2024-08-30T09:16:10.981 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "max_requests", 2024-08-30T09:16:10.981 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:10.981 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.982 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.982 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "500", 2024-08-30T09:16:10.982 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.982 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.982 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.982 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.", 2024-08-30T09:16:10.982 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.982 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.982 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.982 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.982 INFO:teuthology.orchestra.run.smithi042.stdout: "server_addr": { 2024-08-30T09:16:10.982 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "server_addr", 2024-08-30T09:16:10.982 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.983 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.983 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.983 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:10.983 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.983 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.983 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.983 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.983 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.983 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.983 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.983 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.983 INFO:teuthology.orchestra.run.smithi042.stdout: "server_port": { 2024-08-30T09:16:10.983 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "server_port", 2024-08-30T09:16:10.983 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.983 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.984 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.984 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:10.984 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.984 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.984 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.984 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.984 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.984 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.984 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.984 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.984 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-08-30T09:16:10.984 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-08-30T09:16:10.984 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:10.984 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-08-30T09:16:10.984 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.985 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-08-30T09:16:10.985 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.985 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.985 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.985 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.985 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.985 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.985 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.985 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:10.985 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:10.985 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.985 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-08-30T09:16:10.985 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rgw", 2024-08-30T09:16:10.985 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-08-30T09:16:10.986 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-08-30T09:16:10.986 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-08-30T09:16:10.986 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-08-30T09:16:10.986 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-08-30T09:16:10.986 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.986 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.986 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.986 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:10.986 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.986 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.986 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:10.986 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:10.986 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:10.986 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:10.986 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:10.987 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:10.987 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:10.987 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:10.987 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.987 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.987 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.987 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.987 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.987 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-08-30T09:16:10.987 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-08-30T09:16:10.987 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.987 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.987 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.987 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:10.988 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.988 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.988 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.988 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.988 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.988 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.988 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.988 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.988 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-08-30T09:16:10.988 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-08-30T09:16:10.988 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.988 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.989 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.989 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-08-30T09:16:10.989 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.989 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.989 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:10.989 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:10.989 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:10.989 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:10.989 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:10.989 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:10.989 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:10.989 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:10.989 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.989 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.989 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.990 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.990 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.990 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-08-30T09:16:10.990 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-08-30T09:16:10.990 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.990 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.990 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.990 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:10.990 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.990 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.990 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.990 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.990 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.990 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.991 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.991 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.991 INFO:teuthology.orchestra.run.smithi042.stdout: "secondary_zone_period_retry_limit": { 2024-08-30T09:16:10.991 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "secondary_zone_period_retry_limit", 2024-08-30T09:16:10.991 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:10.991 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.991 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.991 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "5", 2024-08-30T09:16:10.991 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.991 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.991 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.991 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-08-30T09:16:10.991 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.991 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.991 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.992 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.992 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-08-30T09:16:10.992 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-08-30T09:16:10.992 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:10.992 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-08-30T09:16:10.992 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.992 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-08-30T09:16:10.992 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.992 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.992 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.992 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.992 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.992 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.992 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.993 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:10.993 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:10.993 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.993 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-08-30T09:16:10.993 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rook", 2024-08-30T09:16:10.993 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-08-30T09:16:10.993 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-08-30T09:16:10.993 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-08-30T09:16:10.993 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-08-30T09:16:10.993 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-08-30T09:16:10.993 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.993 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.993 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.993 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:10.994 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.994 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.994 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:10.994 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:10.994 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:10.994 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:10.994 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:10.994 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:10.994 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:10.994 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:10.994 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.994 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.994 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.994 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.994 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.995 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-08-30T09:16:10.995 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-08-30T09:16:10.995 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.995 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.995 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.995 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:10.995 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.995 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.995 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.995 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.995 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.995 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.995 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.995 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.995 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-08-30T09:16:10.996 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-08-30T09:16:10.996 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.996 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.996 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.996 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-08-30T09:16:10.996 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.996 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.996 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:10.996 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:10.996 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:10.996 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:10.996 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:10.996 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:10.997 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:10.997 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:10.997 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.997 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.997 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.997 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.997 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.997 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-08-30T09:16:10.997 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-08-30T09:16:10.997 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:10.997 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.997 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.997 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:10.997 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.997 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.998 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.998 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.998 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.998 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.998 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.998 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.998 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-08-30T09:16:10.998 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-08-30T09:16:10.998 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:10.998 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-08-30T09:16:10.998 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:10.998 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-08-30T09:16:10.998 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.998 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.999 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:10.999 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:10.999 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:10.999 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:10.999 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:10.999 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:10.999 INFO:teuthology.orchestra.run.smithi042.stdout: "storage_class": { 2024-08-30T09:16:10.999 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "storage_class", 2024-08-30T09:16:10.999 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:10.999 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:10.999 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:10.999 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "local", 2024-08-30T09:16:10.999 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:10.999 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:10.999 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:11.000 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-08-30T09:16:11.000 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:11.000 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:11.000 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:11.000 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:11.000 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:11.000 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:11.000 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-08-30T09:16:11.000 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "selftest", 2024-08-30T09:16:11.000 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-08-30T09:16:11.000 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-08-30T09:16:11.000 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-08-30T09:16:11.000 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-08-30T09:16:11.000 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-08-30T09:16:11.000 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:11.001 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:11.001 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:11.001 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:11.001 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:11.001 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:11.001 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:11.001 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:11.001 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:11.001 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:11.001 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:11.001 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:11.001 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:11.001 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:11.002 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:11.002 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:11.002 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:11.002 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:11.002 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:11.002 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-08-30T09:16:11.002 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-08-30T09:16:11.002 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:11.002 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:11.002 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:11.002 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:11.002 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:11.002 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:11.002 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:11.002 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:11.003 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:11.003 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:11.003 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:11.003 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:11.003 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-08-30T09:16:11.003 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-08-30T09:16:11.003 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:11.003 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:11.003 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:11.003 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-08-30T09:16:11.003 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:11.003 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:11.003 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:11.003 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:11.004 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:11.004 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:11.004 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:11.004 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:11.004 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:11.004 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:11.004 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:11.004 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:11.004 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:11.004 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:11.004 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:11.004 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-08-30T09:16:11.004 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-08-30T09:16:11.004 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:11.004 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:11.005 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:11.005 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:11.005 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:11.005 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:11.005 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:11.005 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:11.005 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:11.005 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:11.005 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:11.005 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:11.005 INFO:teuthology.orchestra.run.smithi042.stdout: "roption1": { 2024-08-30T09:16:11.005 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "roption1", 2024-08-30T09:16:11.005 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:11.005 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:11.005 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:11.006 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:11.006 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:11.006 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:11.006 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:11.006 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:11.006 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:11.006 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:11.006 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:11.006 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:11.006 INFO:teuthology.orchestra.run.smithi042.stdout: "roption2": { 2024-08-30T09:16:11.006 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "roption2", 2024-08-30T09:16:11.006 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:11.006 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:11.006 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:11.006 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "xyz", 2024-08-30T09:16:11.007 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:11.007 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:11.007 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:11.007 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:11.007 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:11.007 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:11.007 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:11.007 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:11.007 INFO:teuthology.orchestra.run.smithi042.stdout: "rwoption1": { 2024-08-30T09:16:11.007 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rwoption1", 2024-08-30T09:16:11.007 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:11.007 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:11.007 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:11.007 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:11.007 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:11.008 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:11.008 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:11.008 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:11.008 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:11.008 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:11.008 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:11.008 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:11.008 INFO:teuthology.orchestra.run.smithi042.stdout: "rwoption2": { 2024-08-30T09:16:11.008 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rwoption2", 2024-08-30T09:16:11.008 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:11.008 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:11.008 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:11.008 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:11.008 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:11.008 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:11.009 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:11.009 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:11.009 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:11.009 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:11.009 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:11.009 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:11.009 INFO:teuthology.orchestra.run.smithi042.stdout: "rwoption3": { 2024-08-30T09:16:11.009 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rwoption3", 2024-08-30T09:16:11.009 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "float", 2024-08-30T09:16:11.009 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:11.009 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:11.009 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:11.009 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:11.009 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:11.010 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:11.010 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:11.010 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:11.010 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:11.010 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:11.010 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:11.010 INFO:teuthology.orchestra.run.smithi042.stdout: "rwoption4": { 2024-08-30T09:16:11.010 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rwoption4", 2024-08-30T09:16:11.010 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:11.010 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:11.010 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:11.010 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:11.010 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:11.010 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:11.011 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:11.011 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:11.011 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:11.011 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:11.011 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:11.011 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:11.011 INFO:teuthology.orchestra.run.smithi042.stdout: "rwoption5": { 2024-08-30T09:16:11.011 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rwoption5", 2024-08-30T09:16:11.011 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:11.011 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:11.011 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:11.011 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:11.011 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:11.011 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:11.011 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:11.012 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:11.012 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:11.012 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:11.012 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:11.012 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:11.012 INFO:teuthology.orchestra.run.smithi042.stdout: "rwoption6": { 2024-08-30T09:16:11.012 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rwoption6", 2024-08-30T09:16:11.012 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:11.012 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:11.012 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:11.012 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-08-30T09:16:11.012 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:11.012 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:11.012 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:11.013 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:11.013 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:11.013 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:11.013 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:11.013 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:11.013 INFO:teuthology.orchestra.run.smithi042.stdout: "rwoption7": { 2024-08-30T09:16:11.013 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rwoption7", 2024-08-30T09:16:11.013 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:11.013 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:11.013 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:11.013 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:11.013 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "1", 2024-08-30T09:16:11.013 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "42", 2024-08-30T09:16:11.013 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:11.014 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:11.014 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:11.014 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:11.014 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:11.014 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:11.014 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-08-30T09:16:11.014 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-08-30T09:16:11.014 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:11.014 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-08-30T09:16:11.014 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:11.014 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-08-30T09:16:11.014 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:11.014 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:11.014 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:11.015 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:11.015 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:11.015 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:11.015 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:11.015 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:11.015 INFO:teuthology.orchestra.run.smithi042.stdout: "testkey": { 2024-08-30T09:16:11.015 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "testkey", 2024-08-30T09:16:11.015 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:11.015 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:11.015 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:11.015 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:11.015 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:11.015 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:11.015 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:11.015 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:11.015 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:11.016 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:11.016 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:11.016 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:11.016 INFO:teuthology.orchestra.run.smithi042.stdout: "testlkey": { 2024-08-30T09:16:11.016 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "testlkey", 2024-08-30T09:16:11.016 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:11.016 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:11.016 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:11.016 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:11.016 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:11.016 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:11.016 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:11.016 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:11.016 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:11.017 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:11.017 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:11.017 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:11.017 INFO:teuthology.orchestra.run.smithi042.stdout: "testnewline": { 2024-08-30T09:16:11.017 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "testnewline", 2024-08-30T09:16:11.017 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:11.017 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:11.017 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:11.017 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:11.017 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:11.017 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:11.017 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:11.017 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:11.017 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:11.018 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:11.018 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:11.018 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:11.018 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:11.018 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:11.018 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-08-30T09:16:11.018 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "smb", 2024-08-30T09:16:11.018 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-08-30T09:16:11.018 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-08-30T09:16:11.018 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-08-30T09:16:11.018 INFO:teuthology.orchestra.run.smithi042.stdout: "internal_store_backend": { 2024-08-30T09:16:11.018 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "internal_store_backend", 2024-08-30T09:16:11.018 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:11.018 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-08-30T09:16:11.018 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:11.019 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:11.019 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:11.019 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:11.019 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:11.019 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "set internal store backend. for develoment and testing only", 2024-08-30T09:16:11.019 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:11.019 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:11.019 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:11.019 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:11.019 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-08-30T09:16:11.019 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-08-30T09:16:11.019 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:11.019 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:11.019 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:11.019 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:11.020 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:11.020 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:11.020 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:11.020 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:11.020 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:11.020 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:11.020 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:11.020 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:11.020 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:11.020 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:11.020 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:11.020 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:11.020 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:11.021 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:11.021 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:11.021 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-08-30T09:16:11.021 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-08-30T09:16:11.021 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:11.021 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:11.021 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:11.021 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:11.021 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:11.021 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:11.021 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:11.021 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:11.021 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:11.021 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:11.022 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:11.022 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:11.022 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-08-30T09:16:11.022 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-08-30T09:16:11.022 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:11.022 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:11.022 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:11.022 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-08-30T09:16:11.022 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:11.022 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:11.022 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:11.022 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:11.022 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:11.022 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:11.022 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:11.023 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:11.023 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:11.023 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:11.023 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:11.023 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:11.023 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:11.023 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:11.023 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:11.023 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-08-30T09:16:11.023 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-08-30T09:16:11.023 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:11.023 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:11.023 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:11.023 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:11.023 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:11.024 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:11.024 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:11.024 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:11.024 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:11.024 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:11.024 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:11.024 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:11.024 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-08-30T09:16:11.024 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-08-30T09:16:11.024 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:11.024 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-08-30T09:16:11.024 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:11.024 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-08-30T09:16:11.024 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:11.024 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:11.025 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:11.025 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:11.025 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:11.025 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:11.025 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:11.025 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:11.025 INFO:teuthology.orchestra.run.smithi042.stdout: "update_orchestration": { 2024-08-30T09:16:11.025 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "update_orchestration", 2024-08-30T09:16:11.025 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:11.025 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:11.025 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:11.025 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-08-30T09:16:11.025 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:11.025 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:11.026 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:11.026 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-08-30T09:16:11.026 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:11.026 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:11.026 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:11.026 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:11.026 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:11.026 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:11.026 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-08-30T09:16:11.026 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "snap_schedule", 2024-08-30T09:16:11.026 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-08-30T09:16:11.026 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-08-30T09:16:11.026 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-08-30T09:16:11.026 INFO:teuthology.orchestra.run.smithi042.stdout: "allow_m_granularity": { 2024-08-30T09:16:11.027 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "allow_m_granularity", 2024-08-30T09:16:11.027 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:11.027 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:11.027 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:11.027 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:11.027 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:11.027 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:11.027 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:11.027 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "allow minute scheduled snapshots", 2024-08-30T09:16:11.027 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:11.027 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:11.027 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:11.027 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:11.027 INFO:teuthology.orchestra.run.smithi042.stdout: "dump_on_update": { 2024-08-30T09:16:11.027 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "dump_on_update", 2024-08-30T09:16:11.028 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:11.028 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:11.028 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:11.028 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:11.028 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:11.028 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:11.028 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:11.028 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "dump database to debug log on update", 2024-08-30T09:16:11.028 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:11.028 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:11.028 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:11.028 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:11.028 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-08-30T09:16:11.028 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-08-30T09:16:11.029 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:11.029 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:11.029 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:11.029 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:11.029 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:11.029 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:11.029 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:11.029 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:11.029 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:11.029 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:11.029 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:11.029 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:11.029 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:11.029 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:11.030 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:11.030 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:11.030 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:11.030 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:11.030 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:11.030 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-08-30T09:16:11.030 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-08-30T09:16:11.030 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:11.030 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:11.030 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:11.030 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:11.030 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:11.030 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:11.030 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:11.031 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:11.031 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:11.031 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:11.031 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:11.031 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:11.031 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-08-30T09:16:11.031 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-08-30T09:16:11.031 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:11.031 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:11.031 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:11.031 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-08-30T09:16:11.031 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:11.031 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:11.031 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:11.032 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:11.032 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:11.032 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:11.032 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:11.032 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:11.032 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:11.032 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:11.032 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:11.032 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:11.032 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:11.032 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:11.032 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:11.032 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-08-30T09:16:11.032 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-08-30T09:16:11.033 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:11.033 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:11.033 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:11.033 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:11.033 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:11.033 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:11.033 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:11.033 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:11.033 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:11.033 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:11.033 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:11.033 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:11.033 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-08-30T09:16:11.033 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-08-30T09:16:11.034 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:11.034 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-08-30T09:16:11.034 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:11.034 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-08-30T09:16:11.034 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:11.034 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:11.034 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:11.034 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:11.034 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:11.034 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:11.034 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:11.034 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:11.034 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:11.034 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:11.034 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-08-30T09:16:11.035 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "stats", 2024-08-30T09:16:11.035 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-08-30T09:16:11.035 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-08-30T09:16:11.035 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-08-30T09:16:11.035 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-08-30T09:16:11.035 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-08-30T09:16:11.035 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:11.035 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:11.035 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:11.035 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:11.035 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:11.035 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:11.035 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:11.035 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:11.035 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:11.036 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:11.036 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:11.036 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:11.036 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:11.036 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:11.036 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:11.036 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:11.036 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:11.036 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:11.036 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:11.036 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-08-30T09:16:11.036 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-08-30T09:16:11.036 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:11.036 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:11.037 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:11.037 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:11.037 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:11.037 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:11.037 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:11.037 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:11.037 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:11.037 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:11.037 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:11.037 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:11.037 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-08-30T09:16:11.037 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-08-30T09:16:11.037 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:11.038 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:11.038 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:11.038 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-08-30T09:16:11.038 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:11.038 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:11.038 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:11.038 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:11.038 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:11.038 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:11.038 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:11.038 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:11.038 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:11.038 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:11.038 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:11.039 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:11.039 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:11.039 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:11.039 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:11.039 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-08-30T09:16:11.039 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-08-30T09:16:11.039 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:11.039 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:11.039 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:11.039 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:11.039 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:11.039 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:11.039 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:11.039 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:11.039 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:11.040 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:11.040 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:11.040 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:11.040 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-08-30T09:16:11.040 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-08-30T09:16:11.040 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:11.040 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-08-30T09:16:11.040 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:11.040 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-08-30T09:16:11.040 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:11.040 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:11.040 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:11.040 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:11.040 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:11.040 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:11.041 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:11.041 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:11.041 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:11.041 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:11.041 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-08-30T09:16:11.041 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "status", 2024-08-30T09:16:11.041 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-08-30T09:16:11.041 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-08-30T09:16:11.041 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-08-30T09:16:11.041 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-08-30T09:16:11.041 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-08-30T09:16:11.041 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:11.041 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:11.042 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:11.042 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:11.042 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:11.042 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:11.042 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:11.042 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:11.042 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:11.042 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:11.042 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:11.042 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:11.042 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:11.042 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:11.042 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:11.042 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:11.043 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:11.043 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:11.043 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:11.043 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-08-30T09:16:11.043 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-08-30T09:16:11.043 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:11.043 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:11.043 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:11.043 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:11.043 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:11.043 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:11.043 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:11.043 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:11.043 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:11.043 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:11.043 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:11.044 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:11.044 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-08-30T09:16:11.044 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-08-30T09:16:11.044 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:11.044 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:11.044 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:11.044 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-08-30T09:16:11.044 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:11.044 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:11.044 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:11.044 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:11.044 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:11.044 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:11.044 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:11.045 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:11.045 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:11.045 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:11.045 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:11.045 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:11.045 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:11.045 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:11.045 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:11.045 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-08-30T09:16:11.045 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-08-30T09:16:11.045 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:11.045 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:11.045 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:11.045 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:11.046 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:11.046 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:11.046 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:11.046 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:11.046 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:11.046 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:11.046 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:11.046 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:11.046 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-08-30T09:16:11.046 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-08-30T09:16:11.046 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:11.046 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-08-30T09:16:11.046 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:11.046 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-08-30T09:16:11.047 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:11.047 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:11.047 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:11.047 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:11.047 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:11.047 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:11.047 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:11.047 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:11.047 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:11.047 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:11.047 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-08-30T09:16:11.047 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "telegraf", 2024-08-30T09:16:11.047 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-08-30T09:16:11.048 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-08-30T09:16:11.048 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-08-30T09:16:11.048 INFO:teuthology.orchestra.run.smithi042.stdout: "address": { 2024-08-30T09:16:11.048 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "address", 2024-08-30T09:16:11.048 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:11.048 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:11.048 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:11.048 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-08-30T09:16:11.048 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:11.048 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:11.048 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:11.048 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:11.048 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:11.048 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:11.048 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:11.049 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:11.049 INFO:teuthology.orchestra.run.smithi042.stdout: "interval": { 2024-08-30T09:16:11.049 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "interval", 2024-08-30T09:16:11.049 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-08-30T09:16:11.049 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:11.049 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:11.049 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "15", 2024-08-30T09:16:11.049 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:11.049 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:11.049 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:11.049 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:11.049 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:11.049 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:11.050 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:11.050 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:11.050 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-08-30T09:16:11.050 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-08-30T09:16:11.050 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:11.050 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:11.050 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:11.050 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:11.050 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:11.050 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:11.050 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:11.050 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:11.050 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:11.051 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:11.051 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:11.051 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:11.051 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:11.051 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:11.051 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:11.051 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:11.051 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:11.051 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:11.051 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:11.051 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-08-30T09:16:11.051 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-08-30T09:16:11.051 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:11.052 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:11.052 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:11.052 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:11.052 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:11.052 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:11.052 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:11.052 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:11.052 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:11.052 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:11.052 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:11.052 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:11.052 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-08-30T09:16:11.052 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-08-30T09:16:11.052 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:11.053 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:11.053 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:11.053 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-08-30T09:16:11.053 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:11.053 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:11.053 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:11.053 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:11.053 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:11.053 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:11.053 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:11.053 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:11.053 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:11.053 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:11.053 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:11.054 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:11.054 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:11.054 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:11.054 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:11.054 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-08-30T09:16:11.054 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-08-30T09:16:11.054 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:11.054 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:11.054 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:11.054 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:11.054 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:11.054 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:11.054 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:11.054 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:11.054 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:11.054 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:11.055 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:11.055 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:11.055 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-08-30T09:16:11.055 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-08-30T09:16:11.055 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:11.055 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-08-30T09:16:11.055 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:11.055 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-08-30T09:16:11.055 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:11.055 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:11.055 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:11.055 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:11.055 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:11.055 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:11.056 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:11.056 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:11.056 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:11.056 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:11.056 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-08-30T09:16:11.056 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "telemetry", 2024-08-30T09:16:11.056 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-08-30T09:16:11.056 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-08-30T09:16:11.056 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-08-30T09:16:11.056 INFO:teuthology.orchestra.run.smithi042.stdout: "channel_basic": { 2024-08-30T09:16:11.056 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "channel_basic", 2024-08-30T09:16:11.056 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:11.056 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:11.056 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:11.057 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-08-30T09:16:11.057 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:11.057 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:11.057 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:11.057 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Share basic cluster information (size, version)", 2024-08-30T09:16:11.057 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:11.057 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:11.057 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:11.057 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:11.057 INFO:teuthology.orchestra.run.smithi042.stdout: "channel_crash": { 2024-08-30T09:16:11.057 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "channel_crash", 2024-08-30T09:16:11.057 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:11.057 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:11.057 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:11.058 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-08-30T09:16:11.058 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:11.058 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:11.058 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:11.058 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-08-30T09:16:11.058 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:11.058 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:11.058 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:11.058 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:11.058 INFO:teuthology.orchestra.run.smithi042.stdout: "channel_device": { 2024-08-30T09:16:11.058 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "channel_device", 2024-08-30T09:16:11.058 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:11.058 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:11.058 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:11.058 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-08-30T09:16:11.059 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:11.059 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:11.059 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:11.059 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-08-30T09:16:11.059 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:11.059 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:11.059 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:11.059 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:11.059 INFO:teuthology.orchestra.run.smithi042.stdout: "channel_ident": { 2024-08-30T09:16:11.059 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "channel_ident", 2024-08-30T09:16:11.059 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:11.059 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:11.059 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:11.060 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:11.060 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:11.060 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:11.060 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:11.060 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-08-30T09:16:11.060 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:11.060 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:11.060 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:11.060 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:11.060 INFO:teuthology.orchestra.run.smithi042.stdout: "channel_perf": { 2024-08-30T09:16:11.060 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "channel_perf", 2024-08-30T09:16:11.060 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:11.060 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:11.060 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:11.060 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:11.061 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:11.061 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:11.061 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:11.061 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Share various performance metrics of a cluster", 2024-08-30T09:16:11.061 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:11.061 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:11.061 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:11.061 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:11.061 INFO:teuthology.orchestra.run.smithi042.stdout: "contact": { 2024-08-30T09:16:11.061 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "contact", 2024-08-30T09:16:11.061 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:11.061 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:11.061 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:11.062 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:11.062 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:11.062 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:11.062 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:11.062 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:11.062 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:11.062 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:11.062 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:11.062 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:11.062 INFO:teuthology.orchestra.run.smithi042.stdout: "description": { 2024-08-30T09:16:11.062 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "description", 2024-08-30T09:16:11.062 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:11.062 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:11.062 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:11.063 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:11.063 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:11.063 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:11.063 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:11.063 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:11.063 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:11.063 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:11.063 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:11.063 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:11.063 INFO:teuthology.orchestra.run.smithi042.stdout: "device_url": { 2024-08-30T09:16:11.063 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "device_url", 2024-08-30T09:16:11.063 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:11.063 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:11.063 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:11.064 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-08-30T09:16:11.064 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:11.064 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:11.064 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:11.064 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:11.064 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:11.064 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:11.064 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:11.064 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:11.064 INFO:teuthology.orchestra.run.smithi042.stdout: "enabled": { 2024-08-30T09:16:11.064 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "enabled", 2024-08-30T09:16:11.064 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:11.064 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:11.065 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:11.065 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:11.065 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:11.065 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:11.065 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:11.065 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:11.065 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:11.065 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:11.065 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:11.065 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:11.065 INFO:teuthology.orchestra.run.smithi042.stdout: "interval": { 2024-08-30T09:16:11.065 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "interval", 2024-08-30T09:16:11.065 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:11.065 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:11.065 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:11.066 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "24", 2024-08-30T09:16:11.066 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "8", 2024-08-30T09:16:11.066 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:11.066 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:11.066 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:11.066 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:11.066 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:11.066 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:11.066 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:11.066 INFO:teuthology.orchestra.run.smithi042.stdout: "last_opt_revision": { 2024-08-30T09:16:11.066 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "last_opt_revision", 2024-08-30T09:16:11.066 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:11.066 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:11.066 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:11.066 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "1", 2024-08-30T09:16:11.067 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:11.067 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:11.067 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:11.067 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:11.067 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:11.067 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:11.067 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:11.067 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:11.067 INFO:teuthology.orchestra.run.smithi042.stdout: "leaderboard": { 2024-08-30T09:16:11.067 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "leaderboard", 2024-08-30T09:16:11.067 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:11.067 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:11.067 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:11.067 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:11.068 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:11.068 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:11.068 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:11.068 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:11.068 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:11.068 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:11.068 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:11.068 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:11.068 INFO:teuthology.orchestra.run.smithi042.stdout: "leaderboard_description": { 2024-08-30T09:16:11.068 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "leaderboard_description", 2024-08-30T09:16:11.068 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:11.068 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:11.068 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:11.068 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:11.068 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:11.069 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:11.069 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:11.069 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:11.069 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:11.069 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:11.069 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:11.069 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:11.069 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-08-30T09:16:11.069 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-08-30T09:16:11.069 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:11.069 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:11.069 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:11.069 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:11.069 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:11.070 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:11.070 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:11.070 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:11.070 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:11.070 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:11.070 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:11.070 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:11.070 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:11.070 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:11.070 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:11.070 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:11.070 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:11.070 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:11.070 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:11.071 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-08-30T09:16:11.071 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-08-30T09:16:11.071 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:11.071 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:11.071 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:11.071 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:11.071 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:11.071 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:11.071 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:11.071 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:11.071 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:11.071 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:11.071 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:11.071 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:11.072 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-08-30T09:16:11.072 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-08-30T09:16:11.072 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:11.072 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:11.072 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:11.072 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-08-30T09:16:11.072 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:11.072 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:11.072 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:11.072 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:11.072 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:11.072 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:11.072 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:11.072 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:11.073 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:11.073 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:11.073 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:11.073 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:11.073 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:11.073 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:11.073 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:11.073 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-08-30T09:16:11.073 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-08-30T09:16:11.073 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:11.073 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:11.073 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:11.073 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:11.073 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:11.073 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:11.074 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:11.074 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:11.074 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:11.074 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:11.074 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:11.074 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:11.074 INFO:teuthology.orchestra.run.smithi042.stdout: "organization": { 2024-08-30T09:16:11.074 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "organization", 2024-08-30T09:16:11.074 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:11.074 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:11.074 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:11.074 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:11.074 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:11.074 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:11.074 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:11.075 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:11.075 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:11.075 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:11.075 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:11.075 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:11.075 INFO:teuthology.orchestra.run.smithi042.stdout: "proxy": { 2024-08-30T09:16:11.075 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "proxy", 2024-08-30T09:16:11.075 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:11.075 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:11.075 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:11.075 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:11.075 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:11.075 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:11.075 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:11.075 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:11.076 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:11.076 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:11.076 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:11.076 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:11.076 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-08-30T09:16:11.076 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-08-30T09:16:11.076 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:11.076 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-08-30T09:16:11.076 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:11.076 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-08-30T09:16:11.076 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:11.076 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:11.076 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:11.076 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:11.076 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:11.077 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:11.077 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:11.077 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:11.077 INFO:teuthology.orchestra.run.smithi042.stdout: "url": { 2024-08-30T09:16:11.077 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "url", 2024-08-30T09:16:11.077 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:11.077 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:11.077 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:11.077 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-08-30T09:16:11.077 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:11.077 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:11.077 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:11.077 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:11.077 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:11.078 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:11.078 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:11.078 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:11.078 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:11.078 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:11.078 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-08-30T09:16:11.078 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "test_orchestrator", 2024-08-30T09:16:11.078 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-08-30T09:16:11.078 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-08-30T09:16:11.078 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-08-30T09:16:11.078 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-08-30T09:16:11.078 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-08-30T09:16:11.078 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:11.078 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:11.078 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:11.079 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:11.079 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:11.079 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:11.079 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:11.079 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:11.079 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:11.079 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:11.079 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:11.079 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:11.079 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:11.079 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:11.079 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:11.079 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:11.080 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:11.080 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:11.080 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:11.080 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-08-30T09:16:11.080 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-08-30T09:16:11.080 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:11.080 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:11.080 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:11.080 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:11.080 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:11.080 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:11.080 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:11.080 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:11.080 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:11.081 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:11.081 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:11.081 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:11.081 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-08-30T09:16:11.081 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-08-30T09:16:11.081 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:11.081 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:11.081 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:11.081 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-08-30T09:16:11.081 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:11.081 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:11.081 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:11.081 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:11.081 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:11.081 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:11.081 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:11.082 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:11.082 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:11.082 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:11.082 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:11.082 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:11.082 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:11.082 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:11.082 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:11.082 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-08-30T09:16:11.082 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-08-30T09:16:11.082 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:11.082 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:11.082 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:11.082 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:11.083 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:11.083 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:11.083 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:11.083 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:11.083 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:11.083 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:11.083 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:11.083 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:11.083 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-08-30T09:16:11.083 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-08-30T09:16:11.083 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:11.083 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-08-30T09:16:11.083 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:11.083 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-08-30T09:16:11.084 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:11.084 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:11.084 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:11.084 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:11.084 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:11.084 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:11.084 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:11.084 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:11.084 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:11.084 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:11.084 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-08-30T09:16:11.084 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "volumes", 2024-08-30T09:16:11.084 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-08-30T09:16:11.084 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-08-30T09:16:11.085 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-08-30T09:16:11.085 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-08-30T09:16:11.085 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-08-30T09:16:11.085 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:11.085 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:11.085 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:11.085 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:11.085 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:11.085 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:11.085 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:11.085 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:11.085 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:11.085 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:11.085 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:11.085 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:11.086 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:11.086 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:11.086 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:11.086 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:11.086 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:11.086 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:11.086 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:11.086 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-08-30T09:16:11.086 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-08-30T09:16:11.086 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:11.086 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:11.086 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:11.086 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:11.086 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:11.087 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:11.087 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:11.087 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:11.087 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:11.087 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:11.087 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:11.087 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:11.087 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-08-30T09:16:11.087 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-08-30T09:16:11.087 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:11.087 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:11.087 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:11.087 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-08-30T09:16:11.087 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:11.087 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:11.088 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:11.088 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:11.088 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:11.088 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:11.088 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:11.088 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:11.088 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:11.088 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:11.088 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:11.088 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:11.088 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:11.088 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:11.088 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:11.089 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-08-30T09:16:11.089 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-08-30T09:16:11.089 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:11.089 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:11.089 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:11.089 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:11.089 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:11.089 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:11.089 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:11.089 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:11.089 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:11.089 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:11.089 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:11.089 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:11.089 INFO:teuthology.orchestra.run.smithi042.stdout: "max_concurrent_clones": { 2024-08-30T09:16:11.090 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "max_concurrent_clones", 2024-08-30T09:16:11.090 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:11.090 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:11.090 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:11.090 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "4", 2024-08-30T09:16:11.090 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:11.090 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:11.090 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:11.090 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Number of asynchronous cloner threads", 2024-08-30T09:16:11.090 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:11.090 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:11.090 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:11.090 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:11.090 INFO:teuthology.orchestra.run.smithi042.stdout: "periodic_async_work": { 2024-08-30T09:16:11.091 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "periodic_async_work", 2024-08-30T09:16:11.091 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:11.091 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:11.091 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:11.091 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:11.091 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:11.091 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:11.091 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:11.091 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Periodically check for async work", 2024-08-30T09:16:11.091 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:11.091 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:11.091 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:11.091 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:11.091 INFO:teuthology.orchestra.run.smithi042.stdout: "snapshot_clone_delay": { 2024-08-30T09:16:11.091 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "snapshot_clone_delay", 2024-08-30T09:16:11.092 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:11.092 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:11.092 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:11.092 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-08-30T09:16:11.092 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:11.092 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:11.092 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:11.092 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-08-30T09:16:11.092 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:11.092 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:11.092 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:11.092 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:11.092 INFO:teuthology.orchestra.run.smithi042.stdout: "snapshot_clone_no_wait": { 2024-08-30T09:16:11.092 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "snapshot_clone_no_wait", 2024-08-30T09:16:11.093 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:11.093 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:11.093 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:11.093 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-08-30T09:16:11.093 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:11.093 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:11.093 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:11.093 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-08-30T09:16:11.093 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:11.093 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:11.093 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:11.093 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:11.093 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-08-30T09:16:11.093 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-08-30T09:16:11.094 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:11.094 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-08-30T09:16:11.094 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:11.094 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-08-30T09:16:11.094 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:11.094 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:11.094 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:11.094 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:11.094 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:11.094 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:11.094 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:11.094 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:11.094 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:11.094 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:11.094 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-08-30T09:16:11.095 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "zabbix", 2024-08-30T09:16:11.095 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-08-30T09:16:11.095 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-08-30T09:16:11.095 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-08-30T09:16:11.095 INFO:teuthology.orchestra.run.smithi042.stdout: "discovery_interval": { 2024-08-30T09:16:11.095 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "discovery_interval", 2024-08-30T09:16:11.095 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "uint", 2024-08-30T09:16:11.095 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:11.095 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:11.095 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "100", 2024-08-30T09:16:11.095 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:11.095 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:11.095 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:11.095 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:11.095 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:11.096 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:11.096 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:11.096 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:11.096 INFO:teuthology.orchestra.run.smithi042.stdout: "identifier": { 2024-08-30T09:16:11.096 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "identifier", 2024-08-30T09:16:11.096 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:11.096 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:11.096 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:11.096 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:11.096 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:11.096 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:11.096 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:11.096 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:11.097 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:11.097 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:11.097 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:11.097 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:11.097 INFO:teuthology.orchestra.run.smithi042.stdout: "interval": { 2024-08-30T09:16:11.097 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "interval", 2024-08-30T09:16:11.097 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-08-30T09:16:11.097 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:11.097 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:11.097 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "60", 2024-08-30T09:16:11.097 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:11.097 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:11.097 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:11.097 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:11.097 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:11.098 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:11.098 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:11.098 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:11.098 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-08-30T09:16:11.098 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-08-30T09:16:11.098 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:11.098 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:11.098 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:11.098 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:11.098 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:11.098 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:11.098 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:11.098 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:11.098 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:11.098 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:11.099 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:11.099 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:11.099 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:11.099 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:11.099 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:11.099 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:11.099 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:11.099 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:11.099 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:11.099 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-08-30T09:16:11.099 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-08-30T09:16:11.099 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:11.099 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:11.099 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:11.100 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:11.100 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:11.100 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:11.100 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:11.100 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:11.100 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:11.100 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:11.100 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:11.100 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:11.100 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-08-30T09:16:11.100 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-08-30T09:16:11.100 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:11.100 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:11.100 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:11.100 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-08-30T09:16:11.101 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:11.101 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:11.101 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-08-30T09:16:11.101 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-08-30T09:16:11.101 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-08-30T09:16:11.101 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-08-30T09:16:11.101 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-08-30T09:16:11.101 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-08-30T09:16:11.101 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-08-30T09:16:11.101 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:11.101 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:11.101 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:11.101 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:11.101 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:11.101 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:11.102 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-08-30T09:16:11.102 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-08-30T09:16:11.102 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-08-30T09:16:11.102 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:11.102 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:11.102 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-08-30T09:16:11.102 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:11.102 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:11.102 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:11.102 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:11.102 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:11.102 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:11.102 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:11.102 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:11.102 INFO:teuthology.orchestra.run.smithi042.stdout: "sqlite3_killpoint": { 2024-08-30T09:16:11.103 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sqlite3_killpoint", 2024-08-30T09:16:11.103 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:11.103 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "dev", 2024-08-30T09:16:11.103 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-08-30T09:16:11.103 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-08-30T09:16:11.103 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:11.103 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:11.103 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:11.103 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:11.103 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:11.103 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:11.103 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:11.103 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:11.103 INFO:teuthology.orchestra.run.smithi042.stdout: "zabbix_host": { 2024-08-30T09:16:11.104 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "zabbix_host", 2024-08-30T09:16:11.104 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:11.104 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:11.104 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:11.104 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-08-30T09:16:11.104 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:11.104 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:11.104 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:11.104 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:11.104 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:11.104 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:11.104 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:11.104 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:11.104 INFO:teuthology.orchestra.run.smithi042.stdout: "zabbix_port": { 2024-08-30T09:16:11.104 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "zabbix_port", 2024-08-30T09:16:11.105 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-08-30T09:16:11.105 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:11.105 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:11.105 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "10051", 2024-08-30T09:16:11.105 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:11.105 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:11.105 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:11.105 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:11.105 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:11.105 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:11.105 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:11.105 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:11.105 INFO:teuthology.orchestra.run.smithi042.stdout: "zabbix_sender": { 2024-08-30T09:16:11.105 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "zabbix_sender", 2024-08-30T09:16:11.105 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-08-30T09:16:11.106 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-08-30T09:16:11.106 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-08-30T09:16:11.106 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-08-30T09:16:11.106 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-08-30T09:16:11.106 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-08-30T09:16:11.106 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-08-30T09:16:11.106 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-08-30T09:16:11.106 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-08-30T09:16:11.106 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-08-30T09:16:11.106 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-08-30T09:16:11.106 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:11.106 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:11.106 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:11.106 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:11.106 INFO:teuthology.orchestra.run.smithi042.stdout: "services": { 2024-08-30T09:16:11.107 INFO:teuthology.orchestra.run.smithi042.stdout: "dashboard": "https://172.21.15.42:8443/" 2024-08-30T09:16:11.107 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:11.107 INFO:teuthology.orchestra.run.smithi042.stdout: "always_on_modules": { 2024-08-30T09:16:11.107 INFO:teuthology.orchestra.run.smithi042.stdout: "octopus": [ 2024-08-30T09:16:11.107 INFO:teuthology.orchestra.run.smithi042.stdout: "balancer", 2024-08-30T09:16:11.107 INFO:teuthology.orchestra.run.smithi042.stdout: "crash", 2024-08-30T09:16:11.107 INFO:teuthology.orchestra.run.smithi042.stdout: "devicehealth", 2024-08-30T09:16:11.107 INFO:teuthology.orchestra.run.smithi042.stdout: "orchestrator", 2024-08-30T09:16:11.107 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_autoscaler", 2024-08-30T09:16:11.107 INFO:teuthology.orchestra.run.smithi042.stdout: "progress", 2024-08-30T09:16:11.107 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_support", 2024-08-30T09:16:11.107 INFO:teuthology.orchestra.run.smithi042.stdout: "status", 2024-08-30T09:16:11.107 INFO:teuthology.orchestra.run.smithi042.stdout: "telemetry", 2024-08-30T09:16:11.107 INFO:teuthology.orchestra.run.smithi042.stdout: "volumes" 2024-08-30T09:16:11.107 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:11.108 INFO:teuthology.orchestra.run.smithi042.stdout: "pacific": [ 2024-08-30T09:16:11.108 INFO:teuthology.orchestra.run.smithi042.stdout: "balancer", 2024-08-30T09:16:11.108 INFO:teuthology.orchestra.run.smithi042.stdout: "crash", 2024-08-30T09:16:11.108 INFO:teuthology.orchestra.run.smithi042.stdout: "devicehealth", 2024-08-30T09:16:11.108 INFO:teuthology.orchestra.run.smithi042.stdout: "orchestrator", 2024-08-30T09:16:11.108 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_autoscaler", 2024-08-30T09:16:11.108 INFO:teuthology.orchestra.run.smithi042.stdout: "progress", 2024-08-30T09:16:11.108 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_support", 2024-08-30T09:16:11.108 INFO:teuthology.orchestra.run.smithi042.stdout: "status", 2024-08-30T09:16:11.108 INFO:teuthology.orchestra.run.smithi042.stdout: "telemetry", 2024-08-30T09:16:11.108 INFO:teuthology.orchestra.run.smithi042.stdout: "volumes" 2024-08-30T09:16:11.108 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:11.108 INFO:teuthology.orchestra.run.smithi042.stdout: "quincy": [ 2024-08-30T09:16:11.108 INFO:teuthology.orchestra.run.smithi042.stdout: "balancer", 2024-08-30T09:16:11.108 INFO:teuthology.orchestra.run.smithi042.stdout: "crash", 2024-08-30T09:16:11.109 INFO:teuthology.orchestra.run.smithi042.stdout: "devicehealth", 2024-08-30T09:16:11.109 INFO:teuthology.orchestra.run.smithi042.stdout: "orchestrator", 2024-08-30T09:16:11.109 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_autoscaler", 2024-08-30T09:16:11.109 INFO:teuthology.orchestra.run.smithi042.stdout: "progress", 2024-08-30T09:16:11.109 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_support", 2024-08-30T09:16:11.109 INFO:teuthology.orchestra.run.smithi042.stdout: "status", 2024-08-30T09:16:11.109 INFO:teuthology.orchestra.run.smithi042.stdout: "telemetry", 2024-08-30T09:16:11.109 INFO:teuthology.orchestra.run.smithi042.stdout: "volumes" 2024-08-30T09:16:11.109 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:11.109 INFO:teuthology.orchestra.run.smithi042.stdout: "reef": [ 2024-08-30T09:16:11.109 INFO:teuthology.orchestra.run.smithi042.stdout: "balancer", 2024-08-30T09:16:11.109 INFO:teuthology.orchestra.run.smithi042.stdout: "crash", 2024-08-30T09:16:11.109 INFO:teuthology.orchestra.run.smithi042.stdout: "devicehealth", 2024-08-30T09:16:11.109 INFO:teuthology.orchestra.run.smithi042.stdout: "orchestrator", 2024-08-30T09:16:11.109 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_autoscaler", 2024-08-30T09:16:11.110 INFO:teuthology.orchestra.run.smithi042.stdout: "progress", 2024-08-30T09:16:11.110 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_support", 2024-08-30T09:16:11.110 INFO:teuthology.orchestra.run.smithi042.stdout: "status", 2024-08-30T09:16:11.110 INFO:teuthology.orchestra.run.smithi042.stdout: "telemetry", 2024-08-30T09:16:11.110 INFO:teuthology.orchestra.run.smithi042.stdout: "volumes" 2024-08-30T09:16:11.110 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-08-30T09:16:11.110 INFO:teuthology.orchestra.run.smithi042.stdout: "squid": [ 2024-08-30T09:16:11.110 INFO:teuthology.orchestra.run.smithi042.stdout: "balancer", 2024-08-30T09:16:11.110 INFO:teuthology.orchestra.run.smithi042.stdout: "crash", 2024-08-30T09:16:11.110 INFO:teuthology.orchestra.run.smithi042.stdout: "devicehealth", 2024-08-30T09:16:11.110 INFO:teuthology.orchestra.run.smithi042.stdout: "orchestrator", 2024-08-30T09:16:11.110 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_autoscaler", 2024-08-30T09:16:11.110 INFO:teuthology.orchestra.run.smithi042.stdout: "progress", 2024-08-30T09:16:11.110 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_support", 2024-08-30T09:16:11.110 INFO:teuthology.orchestra.run.smithi042.stdout: "status", 2024-08-30T09:16:11.111 INFO:teuthology.orchestra.run.smithi042.stdout: "telemetry", 2024-08-30T09:16:11.111 INFO:teuthology.orchestra.run.smithi042.stdout: "volumes" 2024-08-30T09:16:11.111 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-08-30T09:16:11.111 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:11.111 INFO:teuthology.orchestra.run.smithi042.stdout: "last_failure_osd_epoch": 3, 2024-08-30T09:16:11.111 INFO:teuthology.orchestra.run.smithi042.stdout: "active_clients": [ 2024-08-30T09:16:11.111 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-08-30T09:16:11.111 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "devicehealth", 2024-08-30T09:16:11.111 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-08-30T09:16:11.111 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-08-30T09:16:11.111 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-08-30T09:16:11.111 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:0", 2024-08-30T09:16:11.111 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3049932087 2024-08-30T09:16:11.111 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:11.111 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-08-30T09:16:11.112 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:11.112 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-08-30T09:16:11.112 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "libcephsqlite", 2024-08-30T09:16:11.112 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-08-30T09:16:11.112 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-08-30T09:16:11.112 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-08-30T09:16:11.112 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:0", 2024-08-30T09:16:11.112 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 569652528 2024-08-30T09:16:11.112 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:11.112 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-08-30T09:16:11.112 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:11.112 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-08-30T09:16:11.112 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rbd_support", 2024-08-30T09:16:11.112 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-08-30T09:16:11.113 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-08-30T09:16:11.113 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-08-30T09:16:11.113 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:0", 2024-08-30T09:16:11.113 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1784618714 2024-08-30T09:16:11.113 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:11.113 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-08-30T09:16:11.113 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:11.113 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-08-30T09:16:11.113 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "volumes", 2024-08-30T09:16:11.113 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-08-30T09:16:11.113 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-08-30T09:16:11.113 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-08-30T09:16:11.113 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:0", 2024-08-30T09:16:11.113 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1749217175 2024-08-30T09:16:11.113 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:11.114 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-08-30T09:16:11.114 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:11.114 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-08-30T09:16:11.114 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-08-30T09:16:11.171 DEBUG:tasks.check_counter:Getting stats from a 2024-08-30T09:16:11.171 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph daemon mds.a perf dump 2024-08-30T09:16:11.171 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:16:10 smithi042 ceph-mon[18652]: from='client.? 172.21.15.42:0/70882895' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} : dispatch 2024-08-30T09:16:11.171 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:16:10 smithi042 ceph-mon[18652]: from='client.? 172.21.15.42:0/1912388417' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json-pretty"} : dispatch 2024-08-30T09:16:11.172 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:16:10 smithi190 ceph-mon[22001]: from='client.? 172.21.15.42:0/70882895' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} : dispatch 2024-08-30T09:16:11.172 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:16:10 smithi190 ceph-mon[22001]: from='client.? 172.21.15.42:0/1912388417' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json-pretty"} : dispatch 2024-08-30T09:16:11.172 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:16:10 smithi167 ceph-mon[22063]: from='client.? 172.21.15.42:0/70882895' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} : dispatch 2024-08-30T09:16:11.172 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:16:10 smithi167 ceph-mon[22063]: from='client.? 172.21.15.42:0/1912388417' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json-pretty"} : dispatch 2024-08-30T09:16:11.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:16:11 smithi167 ceph-mon[22063]: pgmap v835: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 2.5 MiB/s wr, 311 op/s 2024-08-30T09:16:11.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:16:11 smithi042 ceph-mon[18652]: pgmap v835: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 2.5 MiB/s wr, 311 op/s 2024-08-30T09:16:11.432 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T09:16:11.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:16:11 smithi190 ceph-mon[22001]: pgmap v835: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 2.5 MiB/s wr, 311 op/s 2024-08-30T09:16:12.627 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-08-30T09:16:12.627 INFO:teuthology.orchestra.run.smithi042.stdout: "AsyncMessenger::Worker-0": { 2024-08-30T09:16:12.627 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_messages": 4, 2024-08-30T09:16:12.627 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages": 4, 2024-08-30T09:16:12.628 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_bytes": 2944, 2024-08-30T09:16:12.628 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_bytes": 448, 2024-08-30T09:16:12.628 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_created_connections": 4, 2024-08-30T09:16:12.628 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_active_connections": 0, 2024-08-30T09:16:12.628 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_total_time": 0.003165338, 2024-08-30T09:16:12.628 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_send_time": 0.000046610, 2024-08-30T09:16:12.628 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_recv_time": 0.002190832, 2024-08-30T09:16:12.628 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-08-30T09:16:12.628 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages_queue_lat": { 2024-08-30T09:16:12.628 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 3, 2024-08-30T09:16:12.628 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.003921611, 2024-08-30T09:16:12.628 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.001307203 2024-08-30T09:16:12.629 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:12.629 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_handle_ack_lat": { 2024-08-30T09:16:12.629 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-08-30T09:16:12.629 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-08-30T09:16:12.629 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-08-30T09:16:12.629 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:12.629 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-08-30T09:16:12.629 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_encrypted_bytes": 448 2024-08-30T09:16:12.629 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:12.629 INFO:teuthology.orchestra.run.smithi042.stdout: "AsyncMessenger::Worker-1": { 2024-08-30T09:16:12.629 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_messages": 297, 2024-08-30T09:16:12.629 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages": 294, 2024-08-30T09:16:12.629 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_bytes": 545008, 2024-08-30T09:16:12.630 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_bytes": 181520, 2024-08-30T09:16:12.630 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_created_connections": 2, 2024-08-30T09:16:12.630 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_active_connections": 1, 2024-08-30T09:16:12.630 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_total_time": 0.050274243, 2024-08-30T09:16:12.630 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_send_time": 0.023054313, 2024-08-30T09:16:12.630 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_recv_time": 0.032148734, 2024-08-30T09:16:12.630 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-08-30T09:16:12.630 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages_queue_lat": { 2024-08-30T09:16:12.630 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 293, 2024-08-30T09:16:12.630 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.020254049, 2024-08-30T09:16:12.630 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000069126 2024-08-30T09:16:12.630 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:12.630 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_handle_ack_lat": { 2024-08-30T09:16:12.631 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-08-30T09:16:12.631 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-08-30T09:16:12.631 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-08-30T09:16:12.631 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:12.631 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_encrypted_bytes": 545008, 2024-08-30T09:16:12.631 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_encrypted_bytes": 181520 2024-08-30T09:16:12.631 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:12.631 INFO:teuthology.orchestra.run.smithi042.stdout: "AsyncMessenger::Worker-2": { 2024-08-30T09:16:12.631 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_messages": 2, 2024-08-30T09:16:12.631 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages": 225, 2024-08-30T09:16:12.631 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_bytes": 320, 2024-08-30T09:16:12.631 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_bytes": 199728, 2024-08-30T09:16:12.631 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_created_connections": 4, 2024-08-30T09:16:12.632 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_active_connections": 2, 2024-08-30T09:16:12.632 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_total_time": 0.021964872, 2024-08-30T09:16:12.632 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_send_time": 0.016451622, 2024-08-30T09:16:12.632 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_recv_time": 0.002300750, 2024-08-30T09:16:12.632 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-08-30T09:16:12.632 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages_queue_lat": { 2024-08-30T09:16:12.632 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 223, 2024-08-30T09:16:12.632 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.017394345, 2024-08-30T09:16:12.632 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000078001 2024-08-30T09:16:12.632 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:12.633 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_handle_ack_lat": { 2024-08-30T09:16:12.633 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-08-30T09:16:12.633 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-08-30T09:16:12.633 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-08-30T09:16:12.633 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:12.633 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_encrypted_bytes": 320, 2024-08-30T09:16:12.633 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_encrypted_bytes": 199728 2024-08-30T09:16:12.633 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:12.633 INFO:teuthology.orchestra.run.smithi042.stdout: "cct": { 2024-08-30T09:16:12.633 INFO:teuthology.orchestra.run.smithi042.stdout: "total_workers": 0, 2024-08-30T09:16:12.633 INFO:teuthology.orchestra.run.smithi042.stdout: "unhealthy_workers": 0 2024-08-30T09:16:12.633 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:12.633 INFO:teuthology.orchestra.run.smithi042.stdout: "mempool": { 2024-08-30T09:16:12.634 INFO:teuthology.orchestra.run.smithi042.stdout: "bloom_filter_bytes": 0, 2024-08-30T09:16:12.634 INFO:teuthology.orchestra.run.smithi042.stdout: "bloom_filter_items": 0, 2024-08-30T09:16:12.634 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_alloc_bytes": 0, 2024-08-30T09:16:12.634 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_alloc_items": 0, 2024-08-30T09:16:12.634 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_data_bytes": 0, 2024-08-30T09:16:12.634 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_data_items": 0, 2024-08-30T09:16:12.634 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_onode_bytes": 0, 2024-08-30T09:16:12.634 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_onode_items": 0, 2024-08-30T09:16:12.634 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_meta_bytes": 0, 2024-08-30T09:16:12.634 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_meta_items": 0, 2024-08-30T09:16:12.634 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_other_bytes": 0, 2024-08-30T09:16:12.634 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_other_items": 0, 2024-08-30T09:16:12.634 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_buffer_bytes": 0, 2024-08-30T09:16:12.634 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_buffer_items": 0, 2024-08-30T09:16:12.635 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_extent_bytes": 0, 2024-08-30T09:16:12.635 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_extent_items": 0, 2024-08-30T09:16:12.635 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_blob_bytes": 0, 2024-08-30T09:16:12.635 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_blob_items": 0, 2024-08-30T09:16:12.635 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_shared_blob_bytes": 0, 2024-08-30T09:16:12.635 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_shared_blob_items": 0, 2024-08-30T09:16:12.635 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_inline_bl_bytes": 0, 2024-08-30T09:16:12.635 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_inline_bl_items": 0, 2024-08-30T09:16:12.635 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_fsck_bytes": 0, 2024-08-30T09:16:12.635 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_fsck_items": 0, 2024-08-30T09:16:12.635 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_txc_bytes": 0, 2024-08-30T09:16:12.635 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_txc_items": 0, 2024-08-30T09:16:12.635 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_deferred_bytes": 0, 2024-08-30T09:16:12.636 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_deferred_items": 0, 2024-08-30T09:16:12.636 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_bytes": 0, 2024-08-30T09:16:12.636 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_items": 0, 2024-08-30T09:16:12.636 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_bytes": 0, 2024-08-30T09:16:12.636 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_items": 0, 2024-08-30T09:16:12.636 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_reader_bytes": 0, 2024-08-30T09:16:12.636 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_reader_items": 0, 2024-08-30T09:16:12.636 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_writer_bytes": 0, 2024-08-30T09:16:12.636 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_writer_items": 0, 2024-08-30T09:16:12.636 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_anon_bytes": 148744, 2024-08-30T09:16:12.636 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_anon_items": 30, 2024-08-30T09:16:12.636 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_meta_bytes": 0, 2024-08-30T09:16:12.637 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_meta_items": 0, 2024-08-30T09:16:12.637 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_bytes": 0, 2024-08-30T09:16:12.637 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_items": 0, 2024-08-30T09:16:12.637 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mapbl_bytes": 0, 2024-08-30T09:16:12.637 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mapbl_items": 0, 2024-08-30T09:16:12.637 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pglog_bytes": 0, 2024-08-30T09:16:12.637 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pglog_items": 0, 2024-08-30T09:16:12.637 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_bytes": 0, 2024-08-30T09:16:12.637 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_items": 0, 2024-08-30T09:16:12.637 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_mapping_bytes": 0, 2024-08-30T09:16:12.637 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_mapping_items": 0, 2024-08-30T09:16:12.637 INFO:teuthology.orchestra.run.smithi042.stdout: "pgmap_bytes": 0, 2024-08-30T09:16:12.637 INFO:teuthology.orchestra.run.smithi042.stdout: "pgmap_items": 0, 2024-08-30T09:16:12.638 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_co_bytes": 928, 2024-08-30T09:16:12.638 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_co_items": 2, 2024-08-30T09:16:12.638 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_1_bytes": 0, 2024-08-30T09:16:12.638 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_1_items": 0, 2024-08-30T09:16:12.638 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_2_bytes": 0, 2024-08-30T09:16:12.638 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_2_items": 0 2024-08-30T09:16:12.638 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:12.638 INFO:teuthology.orchestra.run.smithi042.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-08-30T09:16:12.638 INFO:teuthology.orchestra.run.smithi042.stdout: "val": 0, 2024-08-30T09:16:12.638 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 104857600, 2024-08-30T09:16:12.638 INFO:teuthology.orchestra.run.smithi042.stdout: "get_started": 0, 2024-08-30T09:16:12.638 INFO:teuthology.orchestra.run.smithi042.stdout: "get": 299, 2024-08-30T09:16:12.638 INFO:teuthology.orchestra.run.smithi042.stdout: "get_sum": 503025, 2024-08-30T09:16:12.638 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_fail": 0, 2024-08-30T09:16:12.639 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_success": 299, 2024-08-30T09:16:12.639 INFO:teuthology.orchestra.run.smithi042.stdout: "take": 0, 2024-08-30T09:16:12.639 INFO:teuthology.orchestra.run.smithi042.stdout: "take_sum": 0, 2024-08-30T09:16:12.639 INFO:teuthology.orchestra.run.smithi042.stdout: "put": 299, 2024-08-30T09:16:12.639 INFO:teuthology.orchestra.run.smithi042.stdout: "put_sum": 503025, 2024-08-30T09:16:12.639 INFO:teuthology.orchestra.run.smithi042.stdout: "wait": { 2024-08-30T09:16:12.639 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-08-30T09:16:12.639 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-08-30T09:16:12.639 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-08-30T09:16:12.639 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:12.639 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:12.639 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-08-30T09:16:13.281 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:16:13 smithi042 ceph-mon[18652]: pgmap v836: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 686 KiB/s wr, 96 op/s 2024-08-30T09:16:13.309 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.a 2024-08-30T09:16:13.309 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.a 2024-08-30T09:16:13.310 DEBUG:tasks.check_counter:Getting stats from d 2024-08-30T09:16:13.310 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph daemon mds.d perf dump 2024-08-30T09:16:13.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:16:13 smithi167 ceph-mon[22063]: pgmap v836: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 686 KiB/s wr, 96 op/s 2024-08-30T09:16:13.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:16:13 smithi190 ceph-mon[22001]: pgmap v836: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 686 KiB/s wr, 96 op/s 2024-08-30T09:16:13.563 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T09:16:14.102 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-08-30T09:16:14.102 INFO:teuthology.orchestra.run.smithi042.stdout: "AsyncMessenger::Worker-0": { 2024-08-30T09:16:14.102 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_messages": 303, 2024-08-30T09:16:14.102 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages": 300, 2024-08-30T09:16:14.102 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_bytes": 546704, 2024-08-30T09:16:14.102 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_bytes": 184880, 2024-08-30T09:16:14.102 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_created_connections": 4, 2024-08-30T09:16:14.102 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_active_connections": 0, 2024-08-30T09:16:14.103 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_total_time": 0.056898952, 2024-08-30T09:16:14.103 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_send_time": 0.026670313, 2024-08-30T09:16:14.103 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_recv_time": 0.034252817, 2024-08-30T09:16:14.103 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-08-30T09:16:14.103 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages_queue_lat": { 2024-08-30T09:16:14.103 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 299, 2024-08-30T09:16:14.103 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.015671208, 2024-08-30T09:16:14.103 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000052412 2024-08-30T09:16:14.103 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:14.103 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_handle_ack_lat": { 2024-08-30T09:16:14.103 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-08-30T09:16:14.103 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-08-30T09:16:14.103 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-08-30T09:16:14.104 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:14.104 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_encrypted_bytes": 546704, 2024-08-30T09:16:14.104 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_encrypted_bytes": 184880 2024-08-30T09:16:14.104 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:14.104 INFO:teuthology.orchestra.run.smithi042.stdout: "AsyncMessenger::Worker-1": { 2024-08-30T09:16:14.104 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_messages": 2, 2024-08-30T09:16:14.104 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages": 229, 2024-08-30T09:16:14.104 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_bytes": 320, 2024-08-30T09:16:14.104 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_bytes": 200432, 2024-08-30T09:16:14.104 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_created_connections": 4, 2024-08-30T09:16:14.104 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_active_connections": 2, 2024-08-30T09:16:14.104 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_total_time": 0.022764417, 2024-08-30T09:16:14.104 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_send_time": 0.017897460, 2024-08-30T09:16:14.104 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_recv_time": 0.002005670, 2024-08-30T09:16:14.105 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-08-30T09:16:14.105 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages_queue_lat": { 2024-08-30T09:16:14.105 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 227, 2024-08-30T09:16:14.105 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.025439920, 2024-08-30T09:16:14.105 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000112070 2024-08-30T09:16:14.105 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:14.105 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_handle_ack_lat": { 2024-08-30T09:16:14.105 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-08-30T09:16:14.105 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-08-30T09:16:14.105 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-08-30T09:16:14.105 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:14.105 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_encrypted_bytes": 320, 2024-08-30T09:16:14.105 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_encrypted_bytes": 200432 2024-08-30T09:16:14.105 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:14.106 INFO:teuthology.orchestra.run.smithi042.stdout: "AsyncMessenger::Worker-2": { 2024-08-30T09:16:14.106 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_messages": 4, 2024-08-30T09:16:14.106 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages": 4, 2024-08-30T09:16:14.106 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_bytes": 2944, 2024-08-30T09:16:14.106 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_bytes": 448, 2024-08-30T09:16:14.106 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_created_connections": 2, 2024-08-30T09:16:14.106 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_active_connections": 1, 2024-08-30T09:16:14.106 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_total_time": 0.002170753, 2024-08-30T09:16:14.106 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_send_time": 0.000037435, 2024-08-30T09:16:14.106 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_recv_time": 0.002137340, 2024-08-30T09:16:14.106 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-08-30T09:16:14.106 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages_queue_lat": { 2024-08-30T09:16:14.106 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 3, 2024-08-30T09:16:14.106 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.003774350, 2024-08-30T09:16:14.107 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.001258116 2024-08-30T09:16:14.107 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:14.107 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_handle_ack_lat": { 2024-08-30T09:16:14.107 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-08-30T09:16:14.107 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-08-30T09:16:14.107 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-08-30T09:16:14.107 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:14.107 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-08-30T09:16:14.107 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_encrypted_bytes": 448 2024-08-30T09:16:14.107 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:14.107 INFO:teuthology.orchestra.run.smithi042.stdout: "cct": { 2024-08-30T09:16:14.107 INFO:teuthology.orchestra.run.smithi042.stdout: "total_workers": 0, 2024-08-30T09:16:14.108 INFO:teuthology.orchestra.run.smithi042.stdout: "unhealthy_workers": 0 2024-08-30T09:16:14.108 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:14.108 INFO:teuthology.orchestra.run.smithi042.stdout: "mempool": { 2024-08-30T09:16:14.108 INFO:teuthology.orchestra.run.smithi042.stdout: "bloom_filter_bytes": 0, 2024-08-30T09:16:14.108 INFO:teuthology.orchestra.run.smithi042.stdout: "bloom_filter_items": 0, 2024-08-30T09:16:14.108 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_alloc_bytes": 0, 2024-08-30T09:16:14.108 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_alloc_items": 0, 2024-08-30T09:16:14.108 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_data_bytes": 0, 2024-08-30T09:16:14.108 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_data_items": 0, 2024-08-30T09:16:14.108 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_onode_bytes": 0, 2024-08-30T09:16:14.108 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_onode_items": 0, 2024-08-30T09:16:14.108 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_meta_bytes": 0, 2024-08-30T09:16:14.108 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_meta_items": 0, 2024-08-30T09:16:14.109 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_other_bytes": 0, 2024-08-30T09:16:14.109 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_other_items": 0, 2024-08-30T09:16:14.109 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_buffer_bytes": 0, 2024-08-30T09:16:14.109 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_buffer_items": 0, 2024-08-30T09:16:14.109 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_extent_bytes": 0, 2024-08-30T09:16:14.109 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_extent_items": 0, 2024-08-30T09:16:14.109 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_blob_bytes": 0, 2024-08-30T09:16:14.109 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_blob_items": 0, 2024-08-30T09:16:14.109 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_shared_blob_bytes": 0, 2024-08-30T09:16:14.109 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_shared_blob_items": 0, 2024-08-30T09:16:14.109 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_inline_bl_bytes": 0, 2024-08-30T09:16:14.109 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_inline_bl_items": 0, 2024-08-30T09:16:14.109 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_fsck_bytes": 0, 2024-08-30T09:16:14.110 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_fsck_items": 0, 2024-08-30T09:16:14.110 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_txc_bytes": 0, 2024-08-30T09:16:14.110 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_txc_items": 0, 2024-08-30T09:16:14.110 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_deferred_bytes": 0, 2024-08-30T09:16:14.110 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_deferred_items": 0, 2024-08-30T09:16:14.110 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_bytes": 0, 2024-08-30T09:16:14.110 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_items": 0, 2024-08-30T09:16:14.110 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_bytes": 0, 2024-08-30T09:16:14.110 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_items": 0, 2024-08-30T09:16:14.110 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_reader_bytes": 0, 2024-08-30T09:16:14.110 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_reader_items": 0, 2024-08-30T09:16:14.110 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_writer_bytes": 0, 2024-08-30T09:16:14.110 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_writer_items": 0, 2024-08-30T09:16:14.110 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_anon_bytes": 148721, 2024-08-30T09:16:14.111 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_anon_items": 30, 2024-08-30T09:16:14.111 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_meta_bytes": 0, 2024-08-30T09:16:14.111 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_meta_items": 0, 2024-08-30T09:16:14.111 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_bytes": 0, 2024-08-30T09:16:14.111 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_items": 0, 2024-08-30T09:16:14.111 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mapbl_bytes": 0, 2024-08-30T09:16:14.111 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mapbl_items": 0, 2024-08-30T09:16:14.111 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pglog_bytes": 0, 2024-08-30T09:16:14.111 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pglog_items": 0, 2024-08-30T09:16:14.111 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_bytes": 0, 2024-08-30T09:16:14.111 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_items": 0, 2024-08-30T09:16:14.111 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_mapping_bytes": 0, 2024-08-30T09:16:14.111 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_mapping_items": 0, 2024-08-30T09:16:14.111 INFO:teuthology.orchestra.run.smithi042.stdout: "pgmap_bytes": 0, 2024-08-30T09:16:14.112 INFO:teuthology.orchestra.run.smithi042.stdout: "pgmap_items": 0, 2024-08-30T09:16:14.112 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_co_bytes": 928, 2024-08-30T09:16:14.112 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_co_items": 2, 2024-08-30T09:16:14.112 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_1_bytes": 0, 2024-08-30T09:16:14.112 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_1_items": 0, 2024-08-30T09:16:14.112 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_2_bytes": 0, 2024-08-30T09:16:14.112 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_2_items": 0 2024-08-30T09:16:14.112 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:14.112 INFO:teuthology.orchestra.run.smithi042.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-08-30T09:16:14.112 INFO:teuthology.orchestra.run.smithi042.stdout: "val": 0, 2024-08-30T09:16:14.112 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 104857600, 2024-08-30T09:16:14.112 INFO:teuthology.orchestra.run.smithi042.stdout: "get_started": 0, 2024-08-30T09:16:14.112 INFO:teuthology.orchestra.run.smithi042.stdout: "get": 305, 2024-08-30T09:16:14.112 INFO:teuthology.orchestra.run.smithi042.stdout: "get_sum": 503869, 2024-08-30T09:16:14.113 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_fail": 0, 2024-08-30T09:16:14.113 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_success": 305, 2024-08-30T09:16:14.113 INFO:teuthology.orchestra.run.smithi042.stdout: "take": 0, 2024-08-30T09:16:14.113 INFO:teuthology.orchestra.run.smithi042.stdout: "take_sum": 0, 2024-08-30T09:16:14.113 INFO:teuthology.orchestra.run.smithi042.stdout: "put": 305, 2024-08-30T09:16:14.113 INFO:teuthology.orchestra.run.smithi042.stdout: "put_sum": 503869, 2024-08-30T09:16:14.113 INFO:teuthology.orchestra.run.smithi042.stdout: "wait": { 2024-08-30T09:16:14.113 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-08-30T09:16:14.113 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-08-30T09:16:14.113 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-08-30T09:16:14.113 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:14.113 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:14.113 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-08-30T09:16:14.823 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-08-30T09:16:14.824 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-08-30T09:16:14.824 DEBUG:tasks.check_counter:Getting stats from g 2024-08-30T09:16:14.824 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph daemon mds.g perf dump 2024-08-30T09:16:15.081 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T09:16:15.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:16:15 smithi167 ceph-mon[22063]: pgmap v837: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 533 KiB/s wr, 78 op/s 2024-08-30T09:16:15.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:16:15 smithi167 ceph-mon[22063]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-30T09:16:15.343 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:16:15 smithi042 ceph-mon[18652]: pgmap v837: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 533 KiB/s wr, 78 op/s 2024-08-30T09:16:15.343 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:16:15 smithi042 ceph-mon[18652]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-30T09:16:15.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:16:15 smithi190 ceph-mon[22001]: pgmap v837: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 533 KiB/s wr, 78 op/s 2024-08-30T09:16:15.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:16:15 smithi190 ceph-mon[22001]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-30T09:16:15.696 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-08-30T09:16:15.696 INFO:teuthology.orchestra.run.smithi042.stdout: "AsyncMessenger::Worker-0": { 2024-08-30T09:16:15.696 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_messages": 0, 2024-08-30T09:16:15.696 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages": 2, 2024-08-30T09:16:15.697 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_bytes": 0, 2024-08-30T09:16:15.697 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_bytes": 0, 2024-08-30T09:16:15.697 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_created_connections": 4, 2024-08-30T09:16:15.697 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_active_connections": 0, 2024-08-30T09:16:15.697 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_total_time": 0.001223493, 2024-08-30T09:16:15.697 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_send_time": 0.000000000, 2024-08-30T09:16:15.697 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_recv_time": 0.000242598, 2024-08-30T09:16:15.697 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-08-30T09:16:15.697 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages_queue_lat": { 2024-08-30T09:16:15.697 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-08-30T09:16:15.697 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-08-30T09:16:15.697 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-08-30T09:16:15.698 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:15.698 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_handle_ack_lat": { 2024-08-30T09:16:15.698 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-08-30T09:16:15.698 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-08-30T09:16:15.698 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-08-30T09:16:15.698 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:15.698 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_encrypted_bytes": 0, 2024-08-30T09:16:15.698 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_encrypted_bytes": 0 2024-08-30T09:16:15.698 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:15.698 INFO:teuthology.orchestra.run.smithi042.stdout: "AsyncMessenger::Worker-1": { 2024-08-30T09:16:15.698 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_messages": 2, 2024-08-30T09:16:15.698 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages": 226, 2024-08-30T09:16:15.698 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_bytes": 320, 2024-08-30T09:16:15.699 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_bytes": 199904, 2024-08-30T09:16:15.699 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_created_connections": 4, 2024-08-30T09:16:15.699 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_active_connections": 2, 2024-08-30T09:16:15.699 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_total_time": 0.022610676, 2024-08-30T09:16:15.699 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_send_time": 0.017104460, 2024-08-30T09:16:15.699 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_recv_time": 0.002441656, 2024-08-30T09:16:15.699 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-08-30T09:16:15.699 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages_queue_lat": { 2024-08-30T09:16:15.699 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 224, 2024-08-30T09:16:15.699 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.014820190, 2024-08-30T09:16:15.699 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000066161 2024-08-30T09:16:15.699 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:15.699 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_handle_ack_lat": { 2024-08-30T09:16:15.699 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-08-30T09:16:15.700 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-08-30T09:16:15.700 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-08-30T09:16:15.700 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:15.700 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_encrypted_bytes": 320, 2024-08-30T09:16:15.700 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_encrypted_bytes": 199904 2024-08-30T09:16:15.700 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:15.700 INFO:teuthology.orchestra.run.smithi042.stdout: "AsyncMessenger::Worker-2": { 2024-08-30T09:16:15.700 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_messages": 304, 2024-08-30T09:16:15.700 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages": 299, 2024-08-30T09:16:15.700 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_bytes": 548832, 2024-08-30T09:16:15.700 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_bytes": 183408, 2024-08-30T09:16:15.700 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_created_connections": 2, 2024-08-30T09:16:15.700 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_active_connections": 1, 2024-08-30T09:16:15.701 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_total_time": 0.058192355, 2024-08-30T09:16:15.701 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_send_time": 0.025279877, 2024-08-30T09:16:15.701 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_recv_time": 0.038780043, 2024-08-30T09:16:15.701 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-08-30T09:16:15.701 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages_queue_lat": { 2024-08-30T09:16:15.701 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 299, 2024-08-30T09:16:15.701 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.030608102, 2024-08-30T09:16:15.701 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000102368 2024-08-30T09:16:15.701 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:15.701 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_handle_ack_lat": { 2024-08-30T09:16:15.701 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-08-30T09:16:15.701 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-08-30T09:16:15.701 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-08-30T09:16:15.701 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:15.702 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_encrypted_bytes": 548832, 2024-08-30T09:16:15.702 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_encrypted_bytes": 183408 2024-08-30T09:16:15.702 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:15.702 INFO:teuthology.orchestra.run.smithi042.stdout: "cct": { 2024-08-30T09:16:15.702 INFO:teuthology.orchestra.run.smithi042.stdout: "total_workers": 0, 2024-08-30T09:16:15.702 INFO:teuthology.orchestra.run.smithi042.stdout: "unhealthy_workers": 0 2024-08-30T09:16:15.702 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:15.702 INFO:teuthology.orchestra.run.smithi042.stdout: "mempool": { 2024-08-30T09:16:15.702 INFO:teuthology.orchestra.run.smithi042.stdout: "bloom_filter_bytes": 0, 2024-08-30T09:16:15.702 INFO:teuthology.orchestra.run.smithi042.stdout: "bloom_filter_items": 0, 2024-08-30T09:16:15.702 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_alloc_bytes": 0, 2024-08-30T09:16:15.702 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_alloc_items": 0, 2024-08-30T09:16:15.702 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_data_bytes": 0, 2024-08-30T09:16:15.702 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_data_items": 0, 2024-08-30T09:16:15.702 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_onode_bytes": 0, 2024-08-30T09:16:15.703 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_onode_items": 0, 2024-08-30T09:16:15.703 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_meta_bytes": 0, 2024-08-30T09:16:15.703 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_meta_items": 0, 2024-08-30T09:16:15.703 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_other_bytes": 0, 2024-08-30T09:16:15.703 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_other_items": 0, 2024-08-30T09:16:15.703 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_buffer_bytes": 0, 2024-08-30T09:16:15.703 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_buffer_items": 0, 2024-08-30T09:16:15.703 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_extent_bytes": 0, 2024-08-30T09:16:15.703 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_extent_items": 0, 2024-08-30T09:16:15.703 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_blob_bytes": 0, 2024-08-30T09:16:15.703 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_blob_items": 0, 2024-08-30T09:16:15.703 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_shared_blob_bytes": 0, 2024-08-30T09:16:15.703 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_shared_blob_items": 0, 2024-08-30T09:16:15.703 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_inline_bl_bytes": 0, 2024-08-30T09:16:15.703 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_inline_bl_items": 0, 2024-08-30T09:16:15.704 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_fsck_bytes": 0, 2024-08-30T09:16:15.704 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_fsck_items": 0, 2024-08-30T09:16:15.704 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_txc_bytes": 0, 2024-08-30T09:16:15.704 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_txc_items": 0, 2024-08-30T09:16:15.704 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_deferred_bytes": 0, 2024-08-30T09:16:15.704 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_deferred_items": 0, 2024-08-30T09:16:15.704 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_bytes": 0, 2024-08-30T09:16:15.704 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_items": 0, 2024-08-30T09:16:15.704 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_bytes": 0, 2024-08-30T09:16:15.704 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_items": 0, 2024-08-30T09:16:15.704 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_reader_bytes": 0, 2024-08-30T09:16:15.704 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_reader_items": 0, 2024-08-30T09:16:15.704 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_writer_bytes": 0, 2024-08-30T09:16:15.705 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_writer_items": 0, 2024-08-30T09:16:15.705 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_anon_bytes": 148704, 2024-08-30T09:16:15.705 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_anon_items": 29, 2024-08-30T09:16:15.705 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_meta_bytes": 0, 2024-08-30T09:16:15.705 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_meta_items": 0, 2024-08-30T09:16:15.705 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_bytes": 0, 2024-08-30T09:16:15.705 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_items": 0, 2024-08-30T09:16:15.705 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mapbl_bytes": 0, 2024-08-30T09:16:15.705 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mapbl_items": 0, 2024-08-30T09:16:15.705 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pglog_bytes": 0, 2024-08-30T09:16:15.705 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pglog_items": 0, 2024-08-30T09:16:15.705 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_bytes": 0, 2024-08-30T09:16:15.705 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_items": 0, 2024-08-30T09:16:15.706 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_mapping_bytes": 0, 2024-08-30T09:16:15.706 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_mapping_items": 0, 2024-08-30T09:16:15.706 INFO:teuthology.orchestra.run.smithi042.stdout: "pgmap_bytes": 0, 2024-08-30T09:16:15.706 INFO:teuthology.orchestra.run.smithi042.stdout: "pgmap_items": 0, 2024-08-30T09:16:15.706 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_co_bytes": 928, 2024-08-30T09:16:15.706 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_co_items": 2, 2024-08-30T09:16:15.706 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_1_bytes": 0, 2024-08-30T09:16:15.706 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_1_items": 0, 2024-08-30T09:16:15.706 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_2_bytes": 0, 2024-08-30T09:16:15.706 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_2_items": 0 2024-08-30T09:16:15.706 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:15.706 INFO:teuthology.orchestra.run.smithi042.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-08-30T09:16:15.706 INFO:teuthology.orchestra.run.smithi042.stdout: "val": 0, 2024-08-30T09:16:15.706 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 104857600, 2024-08-30T09:16:15.707 INFO:teuthology.orchestra.run.smithi042.stdout: "get_started": 0, 2024-08-30T09:16:15.707 INFO:teuthology.orchestra.run.smithi042.stdout: "get": 302, 2024-08-30T09:16:15.707 INFO:teuthology.orchestra.run.smithi042.stdout: "get_sum": 503479, 2024-08-30T09:16:15.707 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_fail": 0, 2024-08-30T09:16:15.707 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_success": 302, 2024-08-30T09:16:15.707 INFO:teuthology.orchestra.run.smithi042.stdout: "take": 0, 2024-08-30T09:16:15.707 INFO:teuthology.orchestra.run.smithi042.stdout: "take_sum": 0, 2024-08-30T09:16:15.707 INFO:teuthology.orchestra.run.smithi042.stdout: "put": 302, 2024-08-30T09:16:15.707 INFO:teuthology.orchestra.run.smithi042.stdout: "put_sum": 503479, 2024-08-30T09:16:15.707 INFO:teuthology.orchestra.run.smithi042.stdout: "wait": { 2024-08-30T09:16:15.707 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-08-30T09:16:15.707 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-08-30T09:16:15.707 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-08-30T09:16:15.708 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:15.708 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:15.708 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-08-30T09:16:16.998 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.g 2024-08-30T09:16:16.998 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.g 2024-08-30T09:16:16.998 DEBUG:tasks.check_counter:Getting stats from j 2024-08-30T09:16:16.998 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph daemon mds.j perf dump 2024-08-30T09:16:17.264 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T09:16:17.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:16:17 smithi167 ceph-mon[22063]: pgmap v838: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 292 KiB/s wr, 50 op/s 2024-08-30T09:16:17.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:16:17 smithi042 ceph-mon[18652]: pgmap v838: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 292 KiB/s wr, 50 op/s 2024-08-30T09:16:17.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:16:17 smithi190 ceph-mon[22001]: pgmap v838: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 292 KiB/s wr, 50 op/s 2024-08-30T09:16:17.807 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-08-30T09:16:17.807 INFO:teuthology.orchestra.run.smithi042.stdout: "AsyncMessenger::Worker-0": { 2024-08-30T09:16:17.807 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_messages": 305, 2024-08-30T09:16:17.807 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages": 302, 2024-08-30T09:16:17.808 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_bytes": 547248, 2024-08-30T09:16:17.808 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_bytes": 186160, 2024-08-30T09:16:17.808 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_created_connections": 4, 2024-08-30T09:16:17.808 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_active_connections": 0, 2024-08-30T09:16:17.808 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_total_time": 0.055738752, 2024-08-30T09:16:17.808 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_send_time": 0.025926080, 2024-08-30T09:16:17.808 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_recv_time": 0.034540944, 2024-08-30T09:16:17.808 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-08-30T09:16:17.808 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages_queue_lat": { 2024-08-30T09:16:17.808 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 301, 2024-08-30T09:16:17.808 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.022893603, 2024-08-30T09:16:17.808 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000076058 2024-08-30T09:16:17.808 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:17.809 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_handle_ack_lat": { 2024-08-30T09:16:17.809 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-08-30T09:16:17.809 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-08-30T09:16:17.809 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-08-30T09:16:17.809 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:17.809 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_encrypted_bytes": 547248, 2024-08-30T09:16:17.809 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_encrypted_bytes": 186160 2024-08-30T09:16:17.809 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:17.809 INFO:teuthology.orchestra.run.smithi042.stdout: "AsyncMessenger::Worker-1": { 2024-08-30T09:16:17.809 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_messages": 2, 2024-08-30T09:16:17.809 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages": 231, 2024-08-30T09:16:17.809 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_bytes": 320, 2024-08-30T09:16:17.809 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_bytes": 200784, 2024-08-30T09:16:17.810 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_created_connections": 4, 2024-08-30T09:16:17.810 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_active_connections": 2, 2024-08-30T09:16:17.810 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_total_time": 0.023322932, 2024-08-30T09:16:17.810 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_send_time": 0.018080005, 2024-08-30T09:16:17.810 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_recv_time": 0.002653705, 2024-08-30T09:16:17.810 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-08-30T09:16:17.810 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages_queue_lat": { 2024-08-30T09:16:17.810 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 229, 2024-08-30T09:16:17.810 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.019065666, 2024-08-30T09:16:17.810 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000083256 2024-08-30T09:16:17.810 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:17.810 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_handle_ack_lat": { 2024-08-30T09:16:17.810 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-08-30T09:16:17.810 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-08-30T09:16:17.811 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-08-30T09:16:17.811 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:17.811 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_encrypted_bytes": 320, 2024-08-30T09:16:17.811 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_encrypted_bytes": 200784 2024-08-30T09:16:17.811 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:17.811 INFO:teuthology.orchestra.run.smithi042.stdout: "AsyncMessenger::Worker-2": { 2024-08-30T09:16:17.811 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_messages": 4, 2024-08-30T09:16:17.811 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages": 4, 2024-08-30T09:16:17.811 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_bytes": 2944, 2024-08-30T09:16:17.811 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_bytes": 448, 2024-08-30T09:16:17.811 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_created_connections": 2, 2024-08-30T09:16:17.811 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_active_connections": 1, 2024-08-30T09:16:17.811 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_total_time": 0.002305798, 2024-08-30T09:16:17.812 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_send_time": 0.000037395, 2024-08-30T09:16:17.812 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_recv_time": 0.002247419, 2024-08-30T09:16:17.812 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-08-30T09:16:17.812 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages_queue_lat": { 2024-08-30T09:16:17.812 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 3, 2024-08-30T09:16:17.812 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.003925886, 2024-08-30T09:16:17.812 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.001308628 2024-08-30T09:16:17.812 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:17.812 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_handle_ack_lat": { 2024-08-30T09:16:17.813 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-08-30T09:16:17.813 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-08-30T09:16:17.813 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-08-30T09:16:17.813 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:17.813 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-08-30T09:16:17.813 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_encrypted_bytes": 448 2024-08-30T09:16:17.813 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:17.813 INFO:teuthology.orchestra.run.smithi042.stdout: "cct": { 2024-08-30T09:16:17.813 INFO:teuthology.orchestra.run.smithi042.stdout: "total_workers": 0, 2024-08-30T09:16:17.813 INFO:teuthology.orchestra.run.smithi042.stdout: "unhealthy_workers": 0 2024-08-30T09:16:17.813 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:17.813 INFO:teuthology.orchestra.run.smithi042.stdout: "mempool": { 2024-08-30T09:16:17.813 INFO:teuthology.orchestra.run.smithi042.stdout: "bloom_filter_bytes": 0, 2024-08-30T09:16:17.814 INFO:teuthology.orchestra.run.smithi042.stdout: "bloom_filter_items": 0, 2024-08-30T09:16:17.814 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_alloc_bytes": 0, 2024-08-30T09:16:17.814 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_alloc_items": 0, 2024-08-30T09:16:17.814 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_data_bytes": 0, 2024-08-30T09:16:17.814 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_data_items": 0, 2024-08-30T09:16:17.814 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_onode_bytes": 0, 2024-08-30T09:16:17.814 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_onode_items": 0, 2024-08-30T09:16:17.814 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_meta_bytes": 0, 2024-08-30T09:16:17.814 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_meta_items": 0, 2024-08-30T09:16:17.814 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_other_bytes": 0, 2024-08-30T09:16:17.814 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_other_items": 0, 2024-08-30T09:16:17.814 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_buffer_bytes": 0, 2024-08-30T09:16:17.814 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_buffer_items": 0, 2024-08-30T09:16:17.815 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_extent_bytes": 0, 2024-08-30T09:16:17.815 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_extent_items": 0, 2024-08-30T09:16:17.815 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_blob_bytes": 0, 2024-08-30T09:16:17.815 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_blob_items": 0, 2024-08-30T09:16:17.815 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_shared_blob_bytes": 0, 2024-08-30T09:16:17.815 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_shared_blob_items": 0, 2024-08-30T09:16:17.815 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_inline_bl_bytes": 0, 2024-08-30T09:16:17.815 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_inline_bl_items": 0, 2024-08-30T09:16:17.815 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_fsck_bytes": 0, 2024-08-30T09:16:17.815 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_fsck_items": 0, 2024-08-30T09:16:17.815 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_txc_bytes": 0, 2024-08-30T09:16:17.815 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_txc_items": 0, 2024-08-30T09:16:17.815 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_deferred_bytes": 0, 2024-08-30T09:16:17.815 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_deferred_items": 0, 2024-08-30T09:16:17.816 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_bytes": 0, 2024-08-30T09:16:17.816 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_items": 0, 2024-08-30T09:16:17.816 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_bytes": 0, 2024-08-30T09:16:17.816 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_items": 0, 2024-08-30T09:16:17.816 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_reader_bytes": 0, 2024-08-30T09:16:17.816 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_reader_items": 0, 2024-08-30T09:16:17.816 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_writer_bytes": 0, 2024-08-30T09:16:17.816 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_writer_items": 0, 2024-08-30T09:16:17.816 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_anon_bytes": 148744, 2024-08-30T09:16:17.816 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_anon_items": 30, 2024-08-30T09:16:17.816 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_meta_bytes": 0, 2024-08-30T09:16:17.816 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_meta_items": 0, 2024-08-30T09:16:17.816 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_bytes": 0, 2024-08-30T09:16:17.817 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_items": 0, 2024-08-30T09:16:17.817 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mapbl_bytes": 0, 2024-08-30T09:16:17.817 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mapbl_items": 0, 2024-08-30T09:16:17.817 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pglog_bytes": 0, 2024-08-30T09:16:17.817 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pglog_items": 0, 2024-08-30T09:16:17.817 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_bytes": 0, 2024-08-30T09:16:17.817 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_items": 0, 2024-08-30T09:16:17.817 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_mapping_bytes": 0, 2024-08-30T09:16:17.817 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_mapping_items": 0, 2024-08-30T09:16:17.817 INFO:teuthology.orchestra.run.smithi042.stdout: "pgmap_bytes": 0, 2024-08-30T09:16:17.817 INFO:teuthology.orchestra.run.smithi042.stdout: "pgmap_items": 0, 2024-08-30T09:16:17.817 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_co_bytes": 928, 2024-08-30T09:16:17.817 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_co_items": 2, 2024-08-30T09:16:17.818 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_1_bytes": 0, 2024-08-30T09:16:17.818 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_1_items": 0, 2024-08-30T09:16:17.818 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_2_bytes": 0, 2024-08-30T09:16:17.818 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_2_items": 0 2024-08-30T09:16:17.818 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-08-30T09:16:17.818 INFO:teuthology.orchestra.run.smithi042.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-08-30T09:16:17.818 INFO:teuthology.orchestra.run.smithi042.stdout: "val": 0, 2024-08-30T09:16:17.818 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 104857600, 2024-08-30T09:16:17.818 INFO:teuthology.orchestra.run.smithi042.stdout: "get_started": 0, 2024-08-30T09:16:17.818 INFO:teuthology.orchestra.run.smithi042.stdout: "get": 307, 2024-08-30T09:16:17.818 INFO:teuthology.orchestra.run.smithi042.stdout: "get_sum": 504129, 2024-08-30T09:16:17.818 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_fail": 0, 2024-08-30T09:16:17.818 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_success": 307, 2024-08-30T09:16:17.819 INFO:teuthology.orchestra.run.smithi042.stdout: "take": 0, 2024-08-30T09:16:17.819 INFO:teuthology.orchestra.run.smithi042.stdout: "take_sum": 0, 2024-08-30T09:16:17.819 INFO:teuthology.orchestra.run.smithi042.stdout: "put": 307, 2024-08-30T09:16:17.819 INFO:teuthology.orchestra.run.smithi042.stdout: "put_sum": 504129, 2024-08-30T09:16:17.819 INFO:teuthology.orchestra.run.smithi042.stdout: "wait": { 2024-08-30T09:16:17.819 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-08-30T09:16:17.819 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-08-30T09:16:17.819 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-08-30T09:16:17.819 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:17.819 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-08-30T09:16:17.819 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-08-30T09:16:18.617 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-08-30T09:16:18.618 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-08-30T09:16:18.618 DEBUG:tasks.check_counter:Getting stats from b 2024-08-30T09:16:18.618 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph daemon mds.b perf dump 2024-08-30T09:16:18.994 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.b/config 2024-08-30T09:16:19.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:16:19 smithi167 ceph-mon[22063]: pgmap v839: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 5.7 KiB/s wr, 10 op/s 2024-08-30T09:16:19.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:16:19 smithi042 ceph-mon[18652]: pgmap v839: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 5.7 KiB/s wr, 10 op/s 2024-08-30T09:16:19.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:16:19 smithi190 ceph-mon[22001]: pgmap v839: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 5.7 KiB/s wr, 10 op/s 2024-08-30T09:16:19.789 INFO:teuthology.orchestra.run.smithi167.stdout:{ 2024-08-30T09:16:19.789 INFO:teuthology.orchestra.run.smithi167.stdout: "AsyncMessenger::Worker-0": { 2024-08-30T09:16:19.789 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_messages": 6992, 2024-08-30T09:16:19.789 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages": 8386, 2024-08-30T09:16:19.789 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_bytes": 3261850, 2024-08-30T09:16:19.789 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_bytes": 16116713, 2024-08-30T09:16:19.789 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_created_connections": 290, 2024-08-30T09:16:19.789 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_active_connections": 7, 2024-08-30T09:16:19.790 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_total_time": 0.744539836, 2024-08-30T09:16:19.790 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_send_time": 0.298603143, 2024-08-30T09:16:19.790 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_recv_time": 0.322657833, 2024-08-30T09:16:19.790 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_fast_dispatch_time": 0.098557478, 2024-08-30T09:16:19.790 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages_queue_lat": { 2024-08-30T09:16:19.790 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 8387, 2024-08-30T09:16:19.790 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 1.298378261, 2024-08-30T09:16:19.790 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000154808 2024-08-30T09:16:19.790 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-30T09:16:19.790 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_handle_ack_lat": { 2024-08-30T09:16:19.790 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 6886, 2024-08-30T09:16:19.790 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.001529877, 2024-08-30T09:16:19.790 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000222 2024-08-30T09:16:19.790 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-30T09:16:19.791 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_encrypted_bytes": 0, 2024-08-30T09:16:19.791 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_encrypted_bytes": 0 2024-08-30T09:16:19.791 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-30T09:16:19.791 INFO:teuthology.orchestra.run.smithi167.stdout: "AsyncMessenger::Worker-1": { 2024-08-30T09:16:19.791 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_messages": 8789, 2024-08-30T09:16:19.791 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages": 8833, 2024-08-30T09:16:19.791 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_bytes": 3939676, 2024-08-30T09:16:19.791 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_bytes": 8086205, 2024-08-30T09:16:19.791 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_created_connections": 8, 2024-08-30T09:16:19.791 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_active_connections": 6, 2024-08-30T09:16:19.791 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_total_time": 0.731405857, 2024-08-30T09:16:19.791 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_send_time": 0.353254895, 2024-08-30T09:16:19.791 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_recv_time": 0.315770306, 2024-08-30T09:16:19.792 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_fast_dispatch_time": 0.104175015, 2024-08-30T09:16:19.792 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages_queue_lat": { 2024-08-30T09:16:19.792 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 8833, 2024-08-30T09:16:19.792 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 3.929667431, 2024-08-30T09:16:19.792 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000444884 2024-08-30T09:16:19.792 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-30T09:16:19.792 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_handle_ack_lat": { 2024-08-30T09:16:19.792 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 8400, 2024-08-30T09:16:19.792 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.001695758, 2024-08-30T09:16:19.792 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000201 2024-08-30T09:16:19.792 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-30T09:16:19.792 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_encrypted_bytes": 710976, 2024-08-30T09:16:19.792 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_encrypted_bytes": 216288 2024-08-30T09:16:19.793 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-30T09:16:19.793 INFO:teuthology.orchestra.run.smithi167.stdout: "AsyncMessenger::Worker-2": { 2024-08-30T09:16:19.793 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_messages": 10890, 2024-08-30T09:16:19.793 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages": 13687, 2024-08-30T09:16:19.793 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_bytes": 4597874, 2024-08-30T09:16:19.793 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_bytes": 8086710, 2024-08-30T09:16:19.793 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_created_connections": 18, 2024-08-30T09:16:19.793 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_active_connections": 7, 2024-08-30T09:16:19.793 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_total_time": 1.095046859, 2024-08-30T09:16:19.793 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_send_time": 0.530705117, 2024-08-30T09:16:19.793 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_recv_time": 0.460807384, 2024-08-30T09:16:19.793 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_fast_dispatch_time": 0.161370868, 2024-08-30T09:16:19.793 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages_queue_lat": { 2024-08-30T09:16:19.794 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 13682, 2024-08-30T09:16:19.794 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 2.683353110, 2024-08-30T09:16:19.794 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000196122 2024-08-30T09:16:19.794 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-30T09:16:19.794 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_handle_ack_lat": { 2024-08-30T09:16:19.794 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 13918, 2024-08-30T09:16:19.794 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.002594644, 2024-08-30T09:16:19.794 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000186 2024-08-30T09:16:19.794 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-30T09:16:19.794 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_encrypted_bytes": 2514896, 2024-08-30T09:16:19.794 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_encrypted_bytes": 5320880 2024-08-30T09:16:19.794 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-30T09:16:19.794 INFO:teuthology.orchestra.run.smithi167.stdout: "cct": { 2024-08-30T09:16:19.795 INFO:teuthology.orchestra.run.smithi167.stdout: "total_workers": 1, 2024-08-30T09:16:19.795 INFO:teuthology.orchestra.run.smithi167.stdout: "unhealthy_workers": 0 2024-08-30T09:16:19.795 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-30T09:16:19.795 INFO:teuthology.orchestra.run.smithi167.stdout: "finisher-MDSRank": { 2024-08-30T09:16:19.795 INFO:teuthology.orchestra.run.smithi167.stdout: "queue_len": 0, 2024-08-30T09:16:19.795 INFO:teuthology.orchestra.run.smithi167.stdout: "complete_latency": { 2024-08-30T09:16:19.795 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 7249, 2024-08-30T09:16:19.795 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 6.643078667, 2024-08-30T09:16:19.795 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000916413 2024-08-30T09:16:19.795 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-08-30T09:16:19.795 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-30T09:16:19.795 INFO:teuthology.orchestra.run.smithi167.stdout: "finisher-PurgeQueue": { 2024-08-30T09:16:19.795 INFO:teuthology.orchestra.run.smithi167.stdout: "queue_len": 0, 2024-08-30T09:16:19.796 INFO:teuthology.orchestra.run.smithi167.stdout: "complete_latency": { 2024-08-30T09:16:19.796 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 395, 2024-08-30T09:16:19.796 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.018295073, 2024-08-30T09:16:19.796 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000046316 2024-08-30T09:16:19.796 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-08-30T09:16:19.796 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-30T09:16:19.796 INFO:teuthology.orchestra.run.smithi167.stdout: "mds": { 2024-08-30T09:16:19.796 INFO:teuthology.orchestra.run.smithi167.stdout: "request": 2473, 2024-08-30T09:16:19.796 INFO:teuthology.orchestra.run.smithi167.stdout: "reply": 2337, 2024-08-30T09:16:19.796 INFO:teuthology.orchestra.run.smithi167.stdout: "reply_latency": { 2024-08-30T09:16:19.796 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 2337, 2024-08-30T09:16:19.796 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 11.982168048, 2024-08-30T09:16:19.796 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.005127157 2024-08-30T09:16:19.797 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-30T09:16:19.797 INFO:teuthology.orchestra.run.smithi167.stdout: "slow_reply": 0, 2024-08-30T09:16:19.797 INFO:teuthology.orchestra.run.smithi167.stdout: "forward": 136, 2024-08-30T09:16:19.797 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_fetch_complete": 17, 2024-08-30T09:16:19.797 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_fetch_keys": 4, 2024-08-30T09:16:19.797 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_commit": 798, 2024-08-30T09:16:19.797 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_split": 3, 2024-08-30T09:16:19.797 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_merge": 0, 2024-08-30T09:16:19.797 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes": 274, 2024-08-30T09:16:19.797 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes_top": 7, 2024-08-30T09:16:19.797 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes_bottom": 249, 2024-08-30T09:16:19.797 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes_pin_tail": 18, 2024-08-30T09:16:19.797 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes_pinned": 264, 2024-08-30T09:16:19.797 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes_expired": 1982, 2024-08-30T09:16:19.798 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes_with_caps": 102, 2024-08-30T09:16:19.798 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": 102, 2024-08-30T09:16:19.798 INFO:teuthology.orchestra.run.smithi167.stdout: "subtrees": 9, 2024-08-30T09:16:19.798 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse": 8665, 2024-08-30T09:16:19.798 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse_hit": 7957, 2024-08-30T09:16:19.798 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse_forward": 135, 2024-08-30T09:16:19.798 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse_discover": 67, 2024-08-30T09:16:19.798 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse_dir_fetch": 1, 2024-08-30T09:16:19.798 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse_remote_ino": 0, 2024-08-30T09:16:19.798 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse_lock": 38, 2024-08-30T09:16:19.798 INFO:teuthology.orchestra.run.smithi167.stdout: "load_cent": 2773, 2024-08-30T09:16:19.798 INFO:teuthology.orchestra.run.smithi167.stdout: "q": 0, 2024-08-30T09:16:19.798 INFO:teuthology.orchestra.run.smithi167.stdout: "exported": 33, 2024-08-30T09:16:19.799 INFO:teuthology.orchestra.run.smithi167.stdout: "exported_inodes": 1739, 2024-08-30T09:16:19.799 INFO:teuthology.orchestra.run.smithi167.stdout: "imported": 13, 2024-08-30T09:16:19.799 INFO:teuthology.orchestra.run.smithi167.stdout: "imported_inodes": 933, 2024-08-30T09:16:19.799 INFO:teuthology.orchestra.run.smithi167.stdout: "openino_dir_fetch": 0, 2024-08-30T09:16:19.799 INFO:teuthology.orchestra.run.smithi167.stdout: "openino_backtrace_fetch": 5625, 2024-08-30T09:16:19.799 INFO:teuthology.orchestra.run.smithi167.stdout: "openino_peer_discover": 0, 2024-08-30T09:16:19.799 INFO:teuthology.orchestra.run.smithi167.stdout: "root_rfiles": 1020, 2024-08-30T09:16:19.799 INFO:teuthology.orchestra.run.smithi167.stdout: "root_rbytes": 1572347444, 2024-08-30T09:16:19.799 INFO:teuthology.orchestra.run.smithi167.stdout: "root_rsnaps": 0, 2024-08-30T09:16:19.799 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_backtrace_fetch": 5625, 2024-08-30T09:16:19.799 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_set_tag": 0, 2024-08-30T09:16:19.799 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_backtrace_repaired": 0, 2024-08-30T09:16:19.799 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_inotable_repaired": 0, 2024-08-30T09:16:19.799 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_dir_inodes": 531, 2024-08-30T09:16:19.800 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_dir_base_inodes": 18, 2024-08-30T09:16:19.800 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_dirfrag_rstats": 513, 2024-08-30T09:16:19.800 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_file_inodes": 5094, 2024-08-30T09:16:19.800 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_inode_file_caps": 11, 2024-08-30T09:16:19.800 INFO:teuthology.orchestra.run.smithi167.stdout: "ceph_cap_op_revoke": 1383, 2024-08-30T09:16:19.800 INFO:teuthology.orchestra.run.smithi167.stdout: "ceph_cap_op_grant": 1919, 2024-08-30T09:16:19.800 INFO:teuthology.orchestra.run.smithi167.stdout: "ceph_cap_op_trunc": 0, 2024-08-30T09:16:19.800 INFO:teuthology.orchestra.run.smithi167.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-08-30T09:16:19.800 INFO:teuthology.orchestra.run.smithi167.stdout: "ceph_cap_op_flush_ack": 0, 2024-08-30T09:16:19.800 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_client_caps": 2794, 2024-08-30T09:16:19.800 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_client_caps_dirty": 1172, 2024-08-30T09:16:19.800 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_client_cap_release": 79, 2024-08-30T09:16:19.800 INFO:teuthology.orchestra.run.smithi167.stdout: "process_request_cap_release": 363 2024-08-30T09:16:19.801 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-30T09:16:19.801 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_cache": { 2024-08-30T09:16:19.801 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_update": 0, 2024-08-30T09:16:19.801 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_update_receipt": 0, 2024-08-30T09:16:19.801 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_try_discover": 0, 2024-08-30T09:16:19.801 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_send_discover": 81, 2024-08-30T09:16:19.801 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_handle_discover": 42, 2024-08-30T09:16:19.801 INFO:teuthology.orchestra.run.smithi167.stdout: "num_strays": 0, 2024-08-30T09:16:19.801 INFO:teuthology.orchestra.run.smithi167.stdout: "num_strays_delayed": 0, 2024-08-30T09:16:19.801 INFO:teuthology.orchestra.run.smithi167.stdout: "num_strays_enqueuing": 0, 2024-08-30T09:16:19.801 INFO:teuthology.orchestra.run.smithi167.stdout: "strays_created": 129, 2024-08-30T09:16:19.801 INFO:teuthology.orchestra.run.smithi167.stdout: "strays_enqueued": 111, 2024-08-30T09:16:19.801 INFO:teuthology.orchestra.run.smithi167.stdout: "strays_reintegrated": 18, 2024-08-30T09:16:19.802 INFO:teuthology.orchestra.run.smithi167.stdout: "strays_migrated": 0, 2024-08-30T09:16:19.802 INFO:teuthology.orchestra.run.smithi167.stdout: "num_recovering_processing": 0, 2024-08-30T09:16:19.802 INFO:teuthology.orchestra.run.smithi167.stdout: "num_recovering_enqueued": 0, 2024-08-30T09:16:19.802 INFO:teuthology.orchestra.run.smithi167.stdout: "num_recovering_prioritized": 0, 2024-08-30T09:16:19.802 INFO:teuthology.orchestra.run.smithi167.stdout: "recovery_started": 0, 2024-08-30T09:16:19.802 INFO:teuthology.orchestra.run.smithi167.stdout: "recovery_completed": 0, 2024-08-30T09:16:19.802 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_quiesce_path": 0, 2024-08-30T09:16:19.802 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_quiesce_inode": 0, 2024-08-30T09:16:19.802 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_enqueue_scrub": 18, 2024-08-30T09:16:19.802 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_exportdir": 33, 2024-08-30T09:16:19.802 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_flush": 0, 2024-08-30T09:16:19.802 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_fragmentdir": 3, 2024-08-30T09:16:19.802 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_fragstats": 0, 2024-08-30T09:16:19.802 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_inodestats": 0, 2024-08-30T09:16:19.803 INFO:teuthology.orchestra.run.smithi167.stdout: "uninline_started": 0, 2024-08-30T09:16:19.803 INFO:teuthology.orchestra.run.smithi167.stdout: "uninline_succeeded": 0, 2024-08-30T09:16:19.803 INFO:teuthology.orchestra.run.smithi167.stdout: "uninline_write_failed": 0 2024-08-30T09:16:19.803 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-30T09:16:19.803 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_log": { 2024-08-30T09:16:19.803 INFO:teuthology.orchestra.run.smithi167.stdout: "evlrg": 0, 2024-08-30T09:16:19.803 INFO:teuthology.orchestra.run.smithi167.stdout: "evadd": 4282, 2024-08-30T09:16:19.803 INFO:teuthology.orchestra.run.smithi167.stdout: "evex": 4240, 2024-08-30T09:16:19.803 INFO:teuthology.orchestra.run.smithi167.stdout: "evtrm": 4240, 2024-08-30T09:16:19.803 INFO:teuthology.orchestra.run.smithi167.stdout: "ev": 42, 2024-08-30T09:16:19.803 INFO:teuthology.orchestra.run.smithi167.stdout: "evexg": 0, 2024-08-30T09:16:19.803 INFO:teuthology.orchestra.run.smithi167.stdout: "evexd": 23, 2024-08-30T09:16:19.803 INFO:teuthology.orchestra.run.smithi167.stdout: "segadd": 74, 2024-08-30T09:16:19.803 INFO:teuthology.orchestra.run.smithi167.stdout: "segex": 73, 2024-08-30T09:16:19.804 INFO:teuthology.orchestra.run.smithi167.stdout: "segtrm": 73, 2024-08-30T09:16:19.804 INFO:teuthology.orchestra.run.smithi167.stdout: "seg": 1, 2024-08-30T09:16:19.804 INFO:teuthology.orchestra.run.smithi167.stdout: "segmjr": 1, 2024-08-30T09:16:19.804 INFO:teuthology.orchestra.run.smithi167.stdout: "segexg": 0, 2024-08-30T09:16:19.804 INFO:teuthology.orchestra.run.smithi167.stdout: "segexd": 1, 2024-08-30T09:16:19.804 INFO:teuthology.orchestra.run.smithi167.stdout: "expos": 16269097, 2024-08-30T09:16:19.804 INFO:teuthology.orchestra.run.smithi167.stdout: "wrpos": 16428219, 2024-08-30T09:16:19.804 INFO:teuthology.orchestra.run.smithi167.stdout: "rdpos": 0, 2024-08-30T09:16:19.804 INFO:teuthology.orchestra.run.smithi167.stdout: "jlat": { 2024-08-30T09:16:19.804 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 2011, 2024-08-30T09:16:19.804 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 33.688654911, 2024-08-30T09:16:19.804 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.016752190 2024-08-30T09:16:19.804 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-30T09:16:19.805 INFO:teuthology.orchestra.run.smithi167.stdout: "replayed": 0 2024-08-30T09:16:19.805 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-30T09:16:19.805 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_mem": { 2024-08-30T09:16:19.805 INFO:teuthology.orchestra.run.smithi167.stdout: "ino": 131, 2024-08-30T09:16:19.805 INFO:teuthology.orchestra.run.smithi167.stdout: "ino+": 2091, 2024-08-30T09:16:19.805 INFO:teuthology.orchestra.run.smithi167.stdout: "ino-": 1960, 2024-08-30T09:16:19.805 INFO:teuthology.orchestra.run.smithi167.stdout: "dir": 63, 2024-08-30T09:16:19.805 INFO:teuthology.orchestra.run.smithi167.stdout: "dir+": 95, 2024-08-30T09:16:19.805 INFO:teuthology.orchestra.run.smithi167.stdout: "dir-": 32, 2024-08-30T09:16:19.805 INFO:teuthology.orchestra.run.smithi167.stdout: "dn": 274, 2024-08-30T09:16:19.805 INFO:teuthology.orchestra.run.smithi167.stdout: "dn+": 2647, 2024-08-30T09:16:19.805 INFO:teuthology.orchestra.run.smithi167.stdout: "dn-": 2373, 2024-08-30T09:16:19.805 INFO:teuthology.orchestra.run.smithi167.stdout: "cap": 102, 2024-08-30T09:16:19.805 INFO:teuthology.orchestra.run.smithi167.stdout: "cap+": 1336, 2024-08-30T09:16:19.806 INFO:teuthology.orchestra.run.smithi167.stdout: "cap-": 1234, 2024-08-30T09:16:19.806 INFO:teuthology.orchestra.run.smithi167.stdout: "rss": 139756, 2024-08-30T09:16:19.806 INFO:teuthology.orchestra.run.smithi167.stdout: "heap": 375275520 2024-08-30T09:16:19.806 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-30T09:16:19.806 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_server": { 2024-08-30T09:16:19.806 INFO:teuthology.orchestra.run.smithi167.stdout: "dispatch_client_request": 2756, 2024-08-30T09:16:19.806 INFO:teuthology.orchestra.run.smithi167.stdout: "dispatch_server_request": 254, 2024-08-30T09:16:19.806 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_client_request": 2473, 2024-08-30T09:16:19.806 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_client_session": 235, 2024-08-30T09:16:19.806 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_peer_request": 412, 2024-08-30T09:16:19.806 INFO:teuthology.orchestra.run.smithi167.stdout: "req_create_latency": { 2024-08-30T09:16:19.806 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 1094, 2024-08-30T09:16:19.806 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 2.348220612, 2024-08-30T09:16:19.807 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.002146453 2024-08-30T09:16:19.807 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-30T09:16:19.807 INFO:teuthology.orchestra.run.smithi167.stdout: "req_getattr_latency": { 2024-08-30T09:16:19.807 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 365, 2024-08-30T09:16:19.807 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 8.314570705, 2024-08-30T09:16:19.807 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.022779645 2024-08-30T09:16:19.807 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-30T09:16:19.807 INFO:teuthology.orchestra.run.smithi167.stdout: "req_getfilelock_latency": { 2024-08-30T09:16:19.807 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-30T09:16:19.807 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-30T09:16:19.807 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-30T09:16:19.807 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-30T09:16:19.807 INFO:teuthology.orchestra.run.smithi167.stdout: "req_link_latency": { 2024-08-30T09:16:19.807 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 18, 2024-08-30T09:16:19.808 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.031049605, 2024-08-30T09:16:19.808 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.001724978 2024-08-30T09:16:19.808 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-30T09:16:19.808 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lookup_latency": { 2024-08-30T09:16:19.808 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 161, 2024-08-30T09:16:19.808 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.339516113, 2024-08-30T09:16:19.808 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.002108795 2024-08-30T09:16:19.808 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-30T09:16:19.808 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lookuphash_latency": { 2024-08-30T09:16:19.808 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-30T09:16:19.808 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-30T09:16:19.808 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-30T09:16:19.808 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-30T09:16:19.809 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lookupino_latency": { 2024-08-30T09:16:19.809 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 2, 2024-08-30T09:16:19.809 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000270335, 2024-08-30T09:16:19.809 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000135167 2024-08-30T09:16:19.809 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-30T09:16:19.809 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lookupname_latency": { 2024-08-30T09:16:19.809 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-30T09:16:19.809 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-30T09:16:19.809 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-30T09:16:19.809 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-30T09:16:19.809 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lookupparent_latency": { 2024-08-30T09:16:19.809 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-30T09:16:19.809 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-30T09:16:19.810 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-30T09:16:19.810 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-30T09:16:19.810 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lookupsnap_latency": { 2024-08-30T09:16:19.810 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-30T09:16:19.810 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-30T09:16:19.810 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-30T09:16:19.811 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-30T09:16:19.811 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lssnap_latency": { 2024-08-30T09:16:19.811 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-30T09:16:19.811 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-30T09:16:19.811 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-30T09:16:19.811 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-30T09:16:19.811 INFO:teuthology.orchestra.run.smithi167.stdout: "req_mkdir_latency": { 2024-08-30T09:16:19.811 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 37, 2024-08-30T09:16:19.811 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.131320355, 2024-08-30T09:16:19.811 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.003549198 2024-08-30T09:16:19.811 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-30T09:16:19.811 INFO:teuthology.orchestra.run.smithi167.stdout: "req_mknod_latency": { 2024-08-30T09:16:19.811 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-30T09:16:19.812 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-30T09:16:19.812 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-30T09:16:19.812 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-30T09:16:19.812 INFO:teuthology.orchestra.run.smithi167.stdout: "req_mksnap_latency": { 2024-08-30T09:16:19.812 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-30T09:16:19.812 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-30T09:16:19.812 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-30T09:16:19.812 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-30T09:16:19.812 INFO:teuthology.orchestra.run.smithi167.stdout: "req_open_latency": { 2024-08-30T09:16:19.812 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 440, 2024-08-30T09:16:19.812 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.187139781, 2024-08-30T09:16:19.812 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000425317 2024-08-30T09:16:19.813 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-30T09:16:19.813 INFO:teuthology.orchestra.run.smithi167.stdout: "req_readdir_latency": { 2024-08-30T09:16:19.813 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 49, 2024-08-30T09:16:19.813 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.093074786, 2024-08-30T09:16:19.813 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.001899485 2024-08-30T09:16:19.813 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-30T09:16:19.813 INFO:teuthology.orchestra.run.smithi167.stdout: "req_rename_latency": { 2024-08-30T09:16:19.813 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 127, 2024-08-30T09:16:19.813 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.402640587, 2024-08-30T09:16:19.813 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.003170398 2024-08-30T09:16:19.813 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-30T09:16:19.813 INFO:teuthology.orchestra.run.smithi167.stdout: "req_renamesnap_latency": { 2024-08-30T09:16:19.813 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-30T09:16:19.814 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-30T09:16:19.814 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-30T09:16:19.814 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-30T09:16:19.814 INFO:teuthology.orchestra.run.smithi167.stdout: "req_snapdiff_latency": { 2024-08-30T09:16:19.814 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-30T09:16:19.814 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-30T09:16:19.814 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-30T09:16:19.814 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-30T09:16:19.814 INFO:teuthology.orchestra.run.smithi167.stdout: "req_rmdir_latency": { 2024-08-30T09:16:19.814 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 2, 2024-08-30T09:16:19.814 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.012290561, 2024-08-30T09:16:19.814 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.006145280 2024-08-30T09:16:19.814 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-30T09:16:19.814 INFO:teuthology.orchestra.run.smithi167.stdout: "req_rmsnap_latency": { 2024-08-30T09:16:19.815 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-30T09:16:19.815 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-30T09:16:19.815 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-30T09:16:19.815 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-30T09:16:19.815 INFO:teuthology.orchestra.run.smithi167.stdout: "req_rmxattr_latency": { 2024-08-30T09:16:19.815 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-30T09:16:19.815 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-30T09:16:19.815 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-30T09:16:19.815 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-30T09:16:19.815 INFO:teuthology.orchestra.run.smithi167.stdout: "req_setattr_latency": { 2024-08-30T09:16:19.815 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 2, 2024-08-30T09:16:19.815 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.001492719, 2024-08-30T09:16:19.815 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000746359 2024-08-30T09:16:19.816 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-30T09:16:19.816 INFO:teuthology.orchestra.run.smithi167.stdout: "req_setdirlayout_latency": { 2024-08-30T09:16:19.816 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-30T09:16:19.816 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-30T09:16:19.816 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-30T09:16:19.816 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-30T09:16:19.816 INFO:teuthology.orchestra.run.smithi167.stdout: "req_setfilelock_latency": { 2024-08-30T09:16:19.816 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-30T09:16:19.816 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-30T09:16:19.816 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-30T09:16:19.816 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-30T09:16:19.816 INFO:teuthology.orchestra.run.smithi167.stdout: "req_setlayout_latency": { 2024-08-30T09:16:19.816 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-30T09:16:19.817 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-30T09:16:19.817 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-30T09:16:19.817 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-30T09:16:19.817 INFO:teuthology.orchestra.run.smithi167.stdout: "req_setxattr_latency": { 2024-08-30T09:16:19.817 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 8, 2024-08-30T09:16:19.817 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.024548823, 2024-08-30T09:16:19.817 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.003068602 2024-08-30T09:16:19.817 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-30T09:16:19.817 INFO:teuthology.orchestra.run.smithi167.stdout: "req_symlink_latency": { 2024-08-30T09:16:19.817 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-30T09:16:19.817 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-30T09:16:19.817 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-30T09:16:19.817 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-30T09:16:19.817 INFO:teuthology.orchestra.run.smithi167.stdout: "req_unlink_latency": { 2024-08-30T09:16:19.818 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 32, 2024-08-30T09:16:19.818 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.096033066, 2024-08-30T09:16:19.818 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.003001033 2024-08-30T09:16:19.818 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-30T09:16:19.818 INFO:teuthology.orchestra.run.smithi167.stdout: "cap_revoke_eviction": 0, 2024-08-30T09:16:19.818 INFO:teuthology.orchestra.run.smithi167.stdout: "cap_acquisition_throttle": 0, 2024-08-30T09:16:19.818 INFO:teuthology.orchestra.run.smithi167.stdout: "req_getvxattr_latency": { 2024-08-30T09:16:19.818 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-30T09:16:19.818 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-30T09:16:19.818 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-30T09:16:19.818 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-08-30T09:16:19.818 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-30T09:16:19.818 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_sessions": { 2024-08-30T09:16:19.819 INFO:teuthology.orchestra.run.smithi167.stdout: "session_count": 1, 2024-08-30T09:16:19.819 INFO:teuthology.orchestra.run.smithi167.stdout: "session_add": 3, 2024-08-30T09:16:19.819 INFO:teuthology.orchestra.run.smithi167.stdout: "session_remove": 2, 2024-08-30T09:16:19.819 INFO:teuthology.orchestra.run.smithi167.stdout: "sessions_open": 1, 2024-08-30T09:16:19.819 INFO:teuthology.orchestra.run.smithi167.stdout: "sessions_stale": 0, 2024-08-30T09:16:19.819 INFO:teuthology.orchestra.run.smithi167.stdout: "total_load": 98, 2024-08-30T09:16:19.819 INFO:teuthology.orchestra.run.smithi167.stdout: "average_load": 98, 2024-08-30T09:16:19.819 INFO:teuthology.orchestra.run.smithi167.stdout: "avg_session_uptime": 954, 2024-08-30T09:16:19.819 INFO:teuthology.orchestra.run.smithi167.stdout: "mdthresh_evicted": 0 2024-08-30T09:16:19.819 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-30T09:16:19.819 INFO:teuthology.orchestra.run.smithi167.stdout: "mempool": { 2024-08-30T09:16:19.819 INFO:teuthology.orchestra.run.smithi167.stdout: "bloom_filter_bytes": 0, 2024-08-30T09:16:19.819 INFO:teuthology.orchestra.run.smithi167.stdout: "bloom_filter_items": 0, 2024-08-30T09:16:19.819 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_alloc_bytes": 0, 2024-08-30T09:16:19.820 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_alloc_items": 0, 2024-08-30T09:16:19.820 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_data_bytes": 0, 2024-08-30T09:16:19.820 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_data_items": 0, 2024-08-30T09:16:19.820 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_onode_bytes": 0, 2024-08-30T09:16:19.820 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_onode_items": 0, 2024-08-30T09:16:19.820 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_meta_bytes": 0, 2024-08-30T09:16:19.820 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_meta_items": 0, 2024-08-30T09:16:19.820 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_other_bytes": 0, 2024-08-30T09:16:19.820 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_other_items": 0, 2024-08-30T09:16:19.820 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_buffer_bytes": 0, 2024-08-30T09:16:19.820 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_buffer_items": 0, 2024-08-30T09:16:19.820 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_extent_bytes": 0, 2024-08-30T09:16:19.820 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_extent_items": 0, 2024-08-30T09:16:19.821 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_blob_bytes": 0, 2024-08-30T09:16:19.821 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_blob_items": 0, 2024-08-30T09:16:19.821 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_shared_blob_bytes": 0, 2024-08-30T09:16:19.821 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_shared_blob_items": 0, 2024-08-30T09:16:19.821 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_inline_bl_bytes": 0, 2024-08-30T09:16:19.821 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_inline_bl_items": 0, 2024-08-30T09:16:19.821 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_fsck_bytes": 0, 2024-08-30T09:16:19.821 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_fsck_items": 0, 2024-08-30T09:16:19.821 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_txc_bytes": 0, 2024-08-30T09:16:19.821 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_txc_items": 0, 2024-08-30T09:16:19.821 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_writing_deferred_bytes": 0, 2024-08-30T09:16:19.821 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_writing_deferred_items": 0, 2024-08-30T09:16:19.821 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_writing_bytes": 0, 2024-08-30T09:16:19.821 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_writing_items": 0, 2024-08-30T09:16:19.822 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_bytes": 0, 2024-08-30T09:16:19.822 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_items": 0, 2024-08-30T09:16:19.822 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_file_reader_bytes": 0, 2024-08-30T09:16:19.822 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_file_reader_items": 0, 2024-08-30T09:16:19.822 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_file_writer_bytes": 0, 2024-08-30T09:16:19.822 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_file_writer_items": 0, 2024-08-30T09:16:19.822 INFO:teuthology.orchestra.run.smithi167.stdout: "buffer_anon_bytes": 343111, 2024-08-30T09:16:19.822 INFO:teuthology.orchestra.run.smithi167.stdout: "buffer_anon_items": 220, 2024-08-30T09:16:19.822 INFO:teuthology.orchestra.run.smithi167.stdout: "buffer_meta_bytes": 88, 2024-08-30T09:16:19.822 INFO:teuthology.orchestra.run.smithi167.stdout: "buffer_meta_items": 1, 2024-08-30T09:16:19.822 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_bytes": 0, 2024-08-30T09:16:19.822 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_items": 0, 2024-08-30T09:16:19.822 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_mapbl_bytes": 0, 2024-08-30T09:16:19.823 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_mapbl_items": 0, 2024-08-30T09:16:19.823 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_pglog_bytes": 0, 2024-08-30T09:16:19.823 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_pglog_items": 0, 2024-08-30T09:16:19.823 INFO:teuthology.orchestra.run.smithi167.stdout: "osdmap_bytes": 8856, 2024-08-30T09:16:19.823 INFO:teuthology.orchestra.run.smithi167.stdout: "osdmap_items": 166, 2024-08-30T09:16:19.823 INFO:teuthology.orchestra.run.smithi167.stdout: "osdmap_mapping_bytes": 0, 2024-08-30T09:16:19.823 INFO:teuthology.orchestra.run.smithi167.stdout: "osdmap_mapping_items": 0, 2024-08-30T09:16:19.823 INFO:teuthology.orchestra.run.smithi167.stdout: "pgmap_bytes": 0, 2024-08-30T09:16:19.823 INFO:teuthology.orchestra.run.smithi167.stdout: "pgmap_items": 0, 2024-08-30T09:16:19.823 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_co_bytes": 639659, 2024-08-30T09:16:19.823 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_co_items": 15894, 2024-08-30T09:16:19.823 INFO:teuthology.orchestra.run.smithi167.stdout: "unittest_1_bytes": 0, 2024-08-30T09:16:19.823 INFO:teuthology.orchestra.run.smithi167.stdout: "unittest_1_items": 0, 2024-08-30T09:16:19.823 INFO:teuthology.orchestra.run.smithi167.stdout: "unittest_2_bytes": 0, 2024-08-30T09:16:19.824 INFO:teuthology.orchestra.run.smithi167.stdout: "unittest_2_items": 0 2024-08-30T09:16:19.824 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-30T09:16:19.824 INFO:teuthology.orchestra.run.smithi167.stdout: "objecter": { 2024-08-30T09:16:19.824 INFO:teuthology.orchestra.run.smithi167.stdout: "op_active": 0, 2024-08-30T09:16:19.824 INFO:teuthology.orchestra.run.smithi167.stdout: "op_laggy": 0, 2024-08-30T09:16:19.824 INFO:teuthology.orchestra.run.smithi167.stdout: "op_send": 12022, 2024-08-30T09:16:19.824 INFO:teuthology.orchestra.run.smithi167.stdout: "op_send_bytes": 15172255, 2024-08-30T09:16:19.824 INFO:teuthology.orchestra.run.smithi167.stdout: "op_resend": 32, 2024-08-30T09:16:19.824 INFO:teuthology.orchestra.run.smithi167.stdout: "op_reply": 12022, 2024-08-30T09:16:19.824 INFO:teuthology.orchestra.run.smithi167.stdout: "op_latency": { 2024-08-30T09:16:19.824 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 12022, 2024-08-30T09:16:19.824 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 429.097547396, 2024-08-30T09:16:19.824 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.035692692 2024-08-30T09:16:19.825 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-30T09:16:19.825 INFO:teuthology.orchestra.run.smithi167.stdout: "op_inflight": 0, 2024-08-30T09:16:19.825 INFO:teuthology.orchestra.run.smithi167.stdout: "oplen_avg": { 2024-08-30T09:16:19.825 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 12022, 2024-08-30T09:16:19.825 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 17007 2024-08-30T09:16:19.825 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-30T09:16:19.825 INFO:teuthology.orchestra.run.smithi167.stdout: "op": 12022, 2024-08-30T09:16:19.825 INFO:teuthology.orchestra.run.smithi167.stdout: "op_r": 5726, 2024-08-30T09:16:19.825 INFO:teuthology.orchestra.run.smithi167.stdout: "op_w": 6296, 2024-08-30T09:16:19.825 INFO:teuthology.orchestra.run.smithi167.stdout: "op_rmw": 0, 2024-08-30T09:16:19.825 INFO:teuthology.orchestra.run.smithi167.stdout: "op_pg": 0, 2024-08-30T09:16:19.825 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_stat": 729, 2024-08-30T09:16:19.825 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_create": 2158, 2024-08-30T09:16:19.826 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_read": 62, 2024-08-30T09:16:19.826 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_write": 2055, 2024-08-30T09:16:19.826 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_writefull": 281, 2024-08-30T09:16:19.826 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_writesame": 0, 2024-08-30T09:16:19.826 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_append": 0, 2024-08-30T09:16:19.826 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_zero": 0, 2024-08-30T09:16:19.826 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_truncate": 3, 2024-08-30T09:16:19.826 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_delete": 750, 2024-08-30T09:16:19.826 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_mapext": 0, 2024-08-30T09:16:19.826 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_sparse_read": 0, 2024-08-30T09:16:19.826 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_clonerange": 0, 2024-08-30T09:16:19.826 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_getxattr": 5660, 2024-08-30T09:16:19.826 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_setxattr": 3270, 2024-08-30T09:16:19.826 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_cmpxattr": 0, 2024-08-30T09:16:19.826 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_rmxattr": 0, 2024-08-30T09:16:19.827 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_resetxattrs": 0, 2024-08-30T09:16:19.827 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_call": 0, 2024-08-30T09:16:19.827 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_watch": 0, 2024-08-30T09:16:19.827 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_notify": 0, 2024-08-30T09:16:19.827 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_src_cmpxattr": 0, 2024-08-30T09:16:19.827 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_pgls": 0, 2024-08-30T09:16:19.827 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_pgls_filter": 0, 2024-08-30T09:16:19.827 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_other": 0, 2024-08-30T09:16:19.827 INFO:teuthology.orchestra.run.smithi167.stdout: "linger_active": 0, 2024-08-30T09:16:19.827 INFO:teuthology.orchestra.run.smithi167.stdout: "linger_send": 0, 2024-08-30T09:16:19.827 INFO:teuthology.orchestra.run.smithi167.stdout: "linger_resend": 0, 2024-08-30T09:16:19.827 INFO:teuthology.orchestra.run.smithi167.stdout: "linger_ping": 0, 2024-08-30T09:16:19.827 INFO:teuthology.orchestra.run.smithi167.stdout: "poolop_active": 0, 2024-08-30T09:16:19.827 INFO:teuthology.orchestra.run.smithi167.stdout: "poolop_send": 0, 2024-08-30T09:16:19.828 INFO:teuthology.orchestra.run.smithi167.stdout: "poolop_resend": 0, 2024-08-30T09:16:19.828 INFO:teuthology.orchestra.run.smithi167.stdout: "poolstat_active": 0, 2024-08-30T09:16:19.828 INFO:teuthology.orchestra.run.smithi167.stdout: "poolstat_send": 0, 2024-08-30T09:16:19.828 INFO:teuthology.orchestra.run.smithi167.stdout: "poolstat_resend": 0, 2024-08-30T09:16:19.828 INFO:teuthology.orchestra.run.smithi167.stdout: "statfs_active": 0, 2024-08-30T09:16:19.828 INFO:teuthology.orchestra.run.smithi167.stdout: "statfs_send": 0, 2024-08-30T09:16:19.828 INFO:teuthology.orchestra.run.smithi167.stdout: "statfs_resend": 0, 2024-08-30T09:16:19.828 INFO:teuthology.orchestra.run.smithi167.stdout: "command_active": 0, 2024-08-30T09:16:19.828 INFO:teuthology.orchestra.run.smithi167.stdout: "command_send": 0, 2024-08-30T09:16:19.828 INFO:teuthology.orchestra.run.smithi167.stdout: "command_resend": 0, 2024-08-30T09:16:19.828 INFO:teuthology.orchestra.run.smithi167.stdout: "map_epoch": 83, 2024-08-30T09:16:19.828 INFO:teuthology.orchestra.run.smithi167.stdout: "map_full": 0, 2024-08-30T09:16:19.828 INFO:teuthology.orchestra.run.smithi167.stdout: "map_inc": 10, 2024-08-30T09:16:19.829 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_sessions": 12, 2024-08-30T09:16:19.829 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_session_open": 12, 2024-08-30T09:16:19.829 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_session_close": 0, 2024-08-30T09:16:19.829 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_laggy": 0, 2024-08-30T09:16:19.829 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_wr": 1870, 2024-08-30T09:16:19.829 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_rd": 42, 2024-08-30T09:16:19.829 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_del": 127 2024-08-30T09:16:19.829 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-30T09:16:19.829 INFO:teuthology.orchestra.run.smithi167.stdout: "oft": { 2024-08-30T09:16:19.829 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_total_objs": 1, 2024-08-30T09:16:19.829 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_total_kv_pairs": 0, 2024-08-30T09:16:19.829 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_total_updates": 0, 2024-08-30T09:16:19.829 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_total_removes": 0 2024-08-30T09:16:19.829 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-30T09:16:19.830 INFO:teuthology.orchestra.run.smithi167.stdout: "purge_queue": { 2024-08-30T09:16:19.830 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_executing_ops": 0, 2024-08-30T09:16:19.830 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_executing_ops_high_water": 24, 2024-08-30T09:16:19.830 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_executing": 0, 2024-08-30T09:16:19.830 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_executing_high_water": 8, 2024-08-30T09:16:19.830 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_executed_ops": 234, 2024-08-30T09:16:19.830 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_executed": 111, 2024-08-30T09:16:19.830 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_item_in_journal": 0 2024-08-30T09:16:19.830 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-30T09:16:19.830 INFO:teuthology.orchestra.run.smithi167.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-08-30T09:16:19.830 INFO:teuthology.orchestra.run.smithi167.stdout: "val": 0, 2024-08-30T09:16:19.830 INFO:teuthology.orchestra.run.smithi167.stdout: "max": 104857600, 2024-08-30T09:16:19.830 INFO:teuthology.orchestra.run.smithi167.stdout: "get_started": 0, 2024-08-30T09:16:19.830 INFO:teuthology.orchestra.run.smithi167.stdout: "get": 26667, 2024-08-30T09:16:19.831 INFO:teuthology.orchestra.run.smithi167.stdout: "get_sum": 9046482, 2024-08-30T09:16:19.831 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_fail": 0, 2024-08-30T09:16:19.831 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_success": 26667, 2024-08-30T09:16:19.831 INFO:teuthology.orchestra.run.smithi167.stdout: "take": 0, 2024-08-30T09:16:19.831 INFO:teuthology.orchestra.run.smithi167.stdout: "take_sum": 0, 2024-08-30T09:16:19.831 INFO:teuthology.orchestra.run.smithi167.stdout: "put": 26667, 2024-08-30T09:16:19.831 INFO:teuthology.orchestra.run.smithi167.stdout: "put_sum": 9046482, 2024-08-30T09:16:19.831 INFO:teuthology.orchestra.run.smithi167.stdout: "wait": { 2024-08-30T09:16:19.831 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-30T09:16:19.831 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-30T09:16:19.831 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-30T09:16:19.831 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-08-30T09:16:19.831 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-30T09:16:19.831 INFO:teuthology.orchestra.run.smithi167.stdout: "throttle-objecter_bytes": { 2024-08-30T09:16:19.832 INFO:teuthology.orchestra.run.smithi167.stdout: "val": 0, 2024-08-30T09:16:19.832 INFO:teuthology.orchestra.run.smithi167.stdout: "max": 104857600, 2024-08-30T09:16:19.832 INFO:teuthology.orchestra.run.smithi167.stdout: "get_started": 0, 2024-08-30T09:16:19.832 INFO:teuthology.orchestra.run.smithi167.stdout: "get": 0, 2024-08-30T09:16:19.832 INFO:teuthology.orchestra.run.smithi167.stdout: "get_sum": 0, 2024-08-30T09:16:19.832 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_fail": 0, 2024-08-30T09:16:19.832 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_success": 0, 2024-08-30T09:16:19.832 INFO:teuthology.orchestra.run.smithi167.stdout: "take": 12022, 2024-08-30T09:16:19.832 INFO:teuthology.orchestra.run.smithi167.stdout: "take_sum": 15184050, 2024-08-30T09:16:19.832 INFO:teuthology.orchestra.run.smithi167.stdout: "put": 12022, 2024-08-30T09:16:19.832 INFO:teuthology.orchestra.run.smithi167.stdout: "put_sum": 15184050, 2024-08-30T09:16:19.832 INFO:teuthology.orchestra.run.smithi167.stdout: "wait": { 2024-08-30T09:16:19.832 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-30T09:16:19.833 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-30T09:16:19.833 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-30T09:16:19.833 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-08-30T09:16:19.833 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-30T09:16:19.833 INFO:teuthology.orchestra.run.smithi167.stdout: "throttle-objecter_ops": { 2024-08-30T09:16:19.833 INFO:teuthology.orchestra.run.smithi167.stdout: "val": 0, 2024-08-30T09:16:19.833 INFO:teuthology.orchestra.run.smithi167.stdout: "max": 1024, 2024-08-30T09:16:19.833 INFO:teuthology.orchestra.run.smithi167.stdout: "get_started": 0, 2024-08-30T09:16:19.833 INFO:teuthology.orchestra.run.smithi167.stdout: "get": 0, 2024-08-30T09:16:19.833 INFO:teuthology.orchestra.run.smithi167.stdout: "get_sum": 0, 2024-08-30T09:16:19.833 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_fail": 0, 2024-08-30T09:16:19.833 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_success": 0, 2024-08-30T09:16:19.833 INFO:teuthology.orchestra.run.smithi167.stdout: "take": 12022, 2024-08-30T09:16:19.834 INFO:teuthology.orchestra.run.smithi167.stdout: "take_sum": 12022, 2024-08-30T09:16:19.834 INFO:teuthology.orchestra.run.smithi167.stdout: "put": 12022, 2024-08-30T09:16:19.834 INFO:teuthology.orchestra.run.smithi167.stdout: "put_sum": 12022, 2024-08-30T09:16:19.834 INFO:teuthology.orchestra.run.smithi167.stdout: "wait": { 2024-08-30T09:16:19.834 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-30T09:16:19.834 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-30T09:16:19.834 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-30T09:16:19.834 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-08-30T09:16:19.834 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-30T09:16:19.834 INFO:teuthology.orchestra.run.smithi167.stdout: "throttle-write_buf_throttle": { 2024-08-30T09:16:19.834 INFO:teuthology.orchestra.run.smithi167.stdout: "val": 0, 2024-08-30T09:16:19.834 INFO:teuthology.orchestra.run.smithi167.stdout: "max": 3758096384, 2024-08-30T09:16:19.834 INFO:teuthology.orchestra.run.smithi167.stdout: "get_started": 0, 2024-08-30T09:16:19.834 INFO:teuthology.orchestra.run.smithi167.stdout: "get": 111, 2024-08-30T09:16:19.835 INFO:teuthology.orchestra.run.smithi167.stdout: "get_sum": 12083, 2024-08-30T09:16:19.835 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_fail": 0, 2024-08-30T09:16:19.835 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_success": 111, 2024-08-30T09:16:19.835 INFO:teuthology.orchestra.run.smithi167.stdout: "take": 0, 2024-08-30T09:16:19.835 INFO:teuthology.orchestra.run.smithi167.stdout: "take_sum": 0, 2024-08-30T09:16:19.835 INFO:teuthology.orchestra.run.smithi167.stdout: "put": 44, 2024-08-30T09:16:19.835 INFO:teuthology.orchestra.run.smithi167.stdout: "put_sum": 12083, 2024-08-30T09:16:19.835 INFO:teuthology.orchestra.run.smithi167.stdout: "wait": { 2024-08-30T09:16:19.835 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-30T09:16:19.835 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-30T09:16:19.835 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-30T09:16:19.835 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-08-30T09:16:19.835 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-30T09:16:19.836 INFO:teuthology.orchestra.run.smithi167.stdout: "throttle-write_buf_throttle-0x555a32cce4e0": { 2024-08-30T09:16:19.836 INFO:teuthology.orchestra.run.smithi167.stdout: "val": 0, 2024-08-30T09:16:19.836 INFO:teuthology.orchestra.run.smithi167.stdout: "max": 3758096384, 2024-08-30T09:16:19.836 INFO:teuthology.orchestra.run.smithi167.stdout: "get_started": 0, 2024-08-30T09:16:19.836 INFO:teuthology.orchestra.run.smithi167.stdout: "get": 4282, 2024-08-30T09:16:19.836 INFO:teuthology.orchestra.run.smithi167.stdout: "get_sum": 12233915, 2024-08-30T09:16:19.836 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_fail": 0, 2024-08-30T09:16:19.836 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_success": 4282, 2024-08-30T09:16:19.836 INFO:teuthology.orchestra.run.smithi167.stdout: "take": 0, 2024-08-30T09:16:19.836 INFO:teuthology.orchestra.run.smithi167.stdout: "take_sum": 0, 2024-08-30T09:16:19.836 INFO:teuthology.orchestra.run.smithi167.stdout: "put": 2011, 2024-08-30T09:16:19.836 INFO:teuthology.orchestra.run.smithi167.stdout: "put_sum": 12233915, 2024-08-30T09:16:19.836 INFO:teuthology.orchestra.run.smithi167.stdout: "wait": { 2024-08-30T09:16:19.837 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-30T09:16:19.837 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-30T09:16:19.837 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-30T09:16:19.837 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-08-30T09:16:19.837 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-30T09:16:19.837 INFO:teuthology.orchestra.run.smithi167.stdout: "trackedop": { 2024-08-30T09:16:19.837 INFO:teuthology.orchestra.run.smithi167.stdout: "slow_ops_count": 0 2024-08-30T09:16:19.837 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-08-30T09:16:19.837 INFO:teuthology.orchestra.run.smithi167.stdout:} 2024-08-30T09:16:20.938 INFO:tasks.check_counter:Daemon mds.b mds.exported=33 2024-08-30T09:16:20.938 INFO:tasks.check_counter:Daemon mds.b mds.imported=13 2024-08-30T09:16:20.938 DEBUG:tasks.check_counter:Getting stats from e 2024-08-30T09:16:20.938 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph daemon mds.e perf dump 2024-08-30T09:16:21.196 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.b/config 2024-08-30T09:16:21.207 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:16:21 smithi167 ceph-mon[22063]: pgmap v840: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 12 op/s 2024-08-30T09:16:21.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:16:21 smithi042 ceph-mon[18652]: pgmap v840: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 12 op/s 2024-08-30T09:16:21.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:16:21 smithi190 ceph-mon[22001]: pgmap v840: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 12 op/s 2024-08-30T09:16:21.823 INFO:teuthology.orchestra.run.smithi167.stdout:{ 2024-08-30T09:16:21.824 INFO:teuthology.orchestra.run.smithi167.stdout: "AsyncMessenger::Worker-0": { 2024-08-30T09:16:21.824 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_messages": 1264, 2024-08-30T09:16:21.824 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages": 1266, 2024-08-30T09:16:21.824 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_bytes": 3136589, 2024-08-30T09:16:21.824 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_bytes": 402015, 2024-08-30T09:16:21.824 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_created_connections": 7, 2024-08-30T09:16:21.824 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_active_connections": 2, 2024-08-30T09:16:21.824 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_total_time": 0.132387045, 2024-08-30T09:16:21.824 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_send_time": 0.058663917, 2024-08-30T09:16:21.825 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_recv_time": 0.029436368, 2024-08-30T09:16:21.825 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_fast_dispatch_time": 0.034104556, 2024-08-30T09:16:21.825 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages_queue_lat": { 2024-08-30T09:16:21.825 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 1264, 2024-08-30T09:16:21.825 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.061936867, 2024-08-30T09:16:21.825 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000049000 2024-08-30T09:16:21.825 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-30T09:16:21.825 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_handle_ack_lat": { 2024-08-30T09:16:21.825 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-30T09:16:21.825 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-30T09:16:21.825 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-30T09:16:21.825 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-30T09:16:21.826 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_encrypted_bytes": 0, 2024-08-30T09:16:21.826 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_encrypted_bytes": 0 2024-08-30T09:16:21.826 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-30T09:16:21.826 INFO:teuthology.orchestra.run.smithi167.stdout: "AsyncMessenger::Worker-1": { 2024-08-30T09:16:21.826 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_messages": 4, 2024-08-30T09:16:21.826 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages": 232, 2024-08-30T09:16:21.826 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_bytes": 1422, 2024-08-30T09:16:21.826 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_bytes": 566626, 2024-08-30T09:16:21.826 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_created_connections": 8, 2024-08-30T09:16:21.826 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_active_connections": 4, 2024-08-30T09:16:21.826 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_total_time": 0.017513350, 2024-08-30T09:16:21.826 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_send_time": 0.012721444, 2024-08-30T09:16:21.827 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_recv_time": 0.002749761, 2024-08-30T09:16:21.827 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_fast_dispatch_time": 0.000047782, 2024-08-30T09:16:21.827 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages_queue_lat": { 2024-08-30T09:16:21.827 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 230, 2024-08-30T09:16:21.827 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.016095787, 2024-08-30T09:16:21.827 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000069981 2024-08-30T09:16:21.827 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-30T09:16:21.827 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_handle_ack_lat": { 2024-08-30T09:16:21.827 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-30T09:16:21.827 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-30T09:16:21.827 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-30T09:16:21.827 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-30T09:16:21.827 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_encrypted_bytes": 320, 2024-08-30T09:16:21.828 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_encrypted_bytes": 565984 2024-08-30T09:16:21.828 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-30T09:16:21.828 INFO:teuthology.orchestra.run.smithi167.stdout: "AsyncMessenger::Worker-2": { 2024-08-30T09:16:21.828 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_messages": 2330, 2024-08-30T09:16:21.828 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages": 2300, 2024-08-30T09:16:21.828 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_bytes": 1256864, 2024-08-30T09:16:21.828 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_bytes": 820294, 2024-08-30T09:16:21.828 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_created_connections": 4, 2024-08-30T09:16:21.828 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_active_connections": 3, 2024-08-30T09:16:21.828 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_total_time": 0.278288635, 2024-08-30T09:16:21.828 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_send_time": 0.107506211, 2024-08-30T09:16:21.828 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_recv_time": 0.084028259, 2024-08-30T09:16:21.828 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_fast_dispatch_time": 0.076808052, 2024-08-30T09:16:21.829 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages_queue_lat": { 2024-08-30T09:16:21.829 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 2300, 2024-08-30T09:16:21.829 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.141661065, 2024-08-30T09:16:21.829 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000061591 2024-08-30T09:16:21.829 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-30T09:16:21.829 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_handle_ack_lat": { 2024-08-30T09:16:21.829 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-30T09:16:21.829 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-30T09:16:21.829 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-30T09:16:21.829 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-30T09:16:21.829 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_encrypted_bytes": 674240, 2024-08-30T09:16:21.829 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_encrypted_bytes": 187792 2024-08-30T09:16:21.829 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-30T09:16:21.830 INFO:teuthology.orchestra.run.smithi167.stdout: "cct": { 2024-08-30T09:16:21.830 INFO:teuthology.orchestra.run.smithi167.stdout: "total_workers": 1, 2024-08-30T09:16:21.830 INFO:teuthology.orchestra.run.smithi167.stdout: "unhealthy_workers": 0 2024-08-30T09:16:21.830 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-30T09:16:21.830 INFO:teuthology.orchestra.run.smithi167.stdout: "finisher-MDSRank": { 2024-08-30T09:16:21.830 INFO:teuthology.orchestra.run.smithi167.stdout: "queue_len": 0, 2024-08-30T09:16:21.830 INFO:teuthology.orchestra.run.smithi167.stdout: "complete_latency": { 2024-08-30T09:16:21.830 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 5512, 2024-08-30T09:16:21.830 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.173421686, 2024-08-30T09:16:21.830 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000031462 2024-08-30T09:16:21.830 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-08-30T09:16:21.830 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-30T09:16:21.830 INFO:teuthology.orchestra.run.smithi167.stdout: "finisher-PurgeQueue": { 2024-08-30T09:16:21.830 INFO:teuthology.orchestra.run.smithi167.stdout: "queue_len": 0, 2024-08-30T09:16:21.831 INFO:teuthology.orchestra.run.smithi167.stdout: "complete_latency": { 2024-08-30T09:16:21.831 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-30T09:16:21.831 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-30T09:16:21.831 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-30T09:16:21.831 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-08-30T09:16:21.831 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-30T09:16:21.831 INFO:teuthology.orchestra.run.smithi167.stdout: "mds": { 2024-08-30T09:16:21.831 INFO:teuthology.orchestra.run.smithi167.stdout: "request": 0, 2024-08-30T09:16:21.831 INFO:teuthology.orchestra.run.smithi167.stdout: "reply": 0, 2024-08-30T09:16:21.831 INFO:teuthology.orchestra.run.smithi167.stdout: "reply_latency": { 2024-08-30T09:16:21.831 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-30T09:16:21.831 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-30T09:16:21.831 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-30T09:16:21.832 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-30T09:16:21.832 INFO:teuthology.orchestra.run.smithi167.stdout: "slow_reply": 0, 2024-08-30T09:16:21.832 INFO:teuthology.orchestra.run.smithi167.stdout: "forward": 0, 2024-08-30T09:16:21.832 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_fetch_complete": 0, 2024-08-30T09:16:21.832 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_fetch_keys": 0, 2024-08-30T09:16:21.832 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_commit": 0, 2024-08-30T09:16:21.832 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_split": 0, 2024-08-30T09:16:21.832 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_merge": 0, 2024-08-30T09:16:21.832 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes": 593, 2024-08-30T09:16:21.832 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes_top": 62, 2024-08-30T09:16:21.832 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes_bottom": 531, 2024-08-30T09:16:21.832 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes_pin_tail": 0, 2024-08-30T09:16:21.832 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes_pinned": 504, 2024-08-30T09:16:21.832 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes_expired": 0, 2024-08-30T09:16:21.833 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes_with_caps": 0, 2024-08-30T09:16:21.833 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": 0, 2024-08-30T09:16:21.833 INFO:teuthology.orchestra.run.smithi167.stdout: "subtrees": 14, 2024-08-30T09:16:21.833 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse": 0, 2024-08-30T09:16:21.833 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse_hit": 0, 2024-08-30T09:16:21.833 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse_forward": 0, 2024-08-30T09:16:21.833 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse_discover": 0, 2024-08-30T09:16:21.833 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse_dir_fetch": 0, 2024-08-30T09:16:21.833 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse_remote_ino": 0, 2024-08-30T09:16:21.833 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse_lock": 0, 2024-08-30T09:16:21.833 INFO:teuthology.orchestra.run.smithi167.stdout: "load_cent": 0, 2024-08-30T09:16:21.833 INFO:teuthology.orchestra.run.smithi167.stdout: "q": 0, 2024-08-30T09:16:21.833 INFO:teuthology.orchestra.run.smithi167.stdout: "exported": 0, 2024-08-30T09:16:21.834 INFO:teuthology.orchestra.run.smithi167.stdout: "exported_inodes": 0, 2024-08-30T09:16:21.834 INFO:teuthology.orchestra.run.smithi167.stdout: "imported": 0, 2024-08-30T09:16:21.834 INFO:teuthology.orchestra.run.smithi167.stdout: "imported_inodes": 0, 2024-08-30T09:16:21.834 INFO:teuthology.orchestra.run.smithi167.stdout: "openino_dir_fetch": 0, 2024-08-30T09:16:21.834 INFO:teuthology.orchestra.run.smithi167.stdout: "openino_backtrace_fetch": 0, 2024-08-30T09:16:21.834 INFO:teuthology.orchestra.run.smithi167.stdout: "openino_peer_discover": 0, 2024-08-30T09:16:21.834 INFO:teuthology.orchestra.run.smithi167.stdout: "root_rfiles": 0, 2024-08-30T09:16:21.834 INFO:teuthology.orchestra.run.smithi167.stdout: "root_rbytes": 0, 2024-08-30T09:16:21.834 INFO:teuthology.orchestra.run.smithi167.stdout: "root_rsnaps": 0, 2024-08-30T09:16:21.834 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_backtrace_fetch": 0, 2024-08-30T09:16:21.834 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_set_tag": 0, 2024-08-30T09:16:21.834 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_backtrace_repaired": 0, 2024-08-30T09:16:21.834 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_inotable_repaired": 0, 2024-08-30T09:16:21.835 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_dir_inodes": 0, 2024-08-30T09:16:21.835 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_dir_base_inodes": 0, 2024-08-30T09:16:21.835 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_dirfrag_rstats": 0, 2024-08-30T09:16:21.835 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_file_inodes": 0, 2024-08-30T09:16:21.835 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_inode_file_caps": 0, 2024-08-30T09:16:21.835 INFO:teuthology.orchestra.run.smithi167.stdout: "ceph_cap_op_revoke": 0, 2024-08-30T09:16:21.835 INFO:teuthology.orchestra.run.smithi167.stdout: "ceph_cap_op_grant": 0, 2024-08-30T09:16:21.835 INFO:teuthology.orchestra.run.smithi167.stdout: "ceph_cap_op_trunc": 0, 2024-08-30T09:16:21.835 INFO:teuthology.orchestra.run.smithi167.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-08-30T09:16:21.835 INFO:teuthology.orchestra.run.smithi167.stdout: "ceph_cap_op_flush_ack": 0, 2024-08-30T09:16:21.835 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_client_caps": 0, 2024-08-30T09:16:21.835 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_client_caps_dirty": 0, 2024-08-30T09:16:21.836 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_client_cap_release": 0, 2024-08-30T09:16:21.836 INFO:teuthology.orchestra.run.smithi167.stdout: "process_request_cap_release": 0 2024-08-30T09:16:21.836 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-30T09:16:21.836 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_cache": { 2024-08-30T09:16:21.836 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_update": 0, 2024-08-30T09:16:21.836 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_update_receipt": 0, 2024-08-30T09:16:21.836 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_try_discover": 0, 2024-08-30T09:16:21.836 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_send_discover": 0, 2024-08-30T09:16:21.836 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_handle_discover": 0, 2024-08-30T09:16:21.836 INFO:teuthology.orchestra.run.smithi167.stdout: "num_strays": 0, 2024-08-30T09:16:21.836 INFO:teuthology.orchestra.run.smithi167.stdout: "num_strays_delayed": 0, 2024-08-30T09:16:21.836 INFO:teuthology.orchestra.run.smithi167.stdout: "num_strays_enqueuing": 0, 2024-08-30T09:16:21.836 INFO:teuthology.orchestra.run.smithi167.stdout: "strays_created": 0, 2024-08-30T09:16:21.836 INFO:teuthology.orchestra.run.smithi167.stdout: "strays_enqueued": 0, 2024-08-30T09:16:21.837 INFO:teuthology.orchestra.run.smithi167.stdout: "strays_reintegrated": 0, 2024-08-30T09:16:21.837 INFO:teuthology.orchestra.run.smithi167.stdout: "strays_migrated": 0, 2024-08-30T09:16:21.837 INFO:teuthology.orchestra.run.smithi167.stdout: "num_recovering_processing": 0, 2024-08-30T09:16:21.837 INFO:teuthology.orchestra.run.smithi167.stdout: "num_recovering_enqueued": 0, 2024-08-30T09:16:21.837 INFO:teuthology.orchestra.run.smithi167.stdout: "num_recovering_prioritized": 0, 2024-08-30T09:16:21.837 INFO:teuthology.orchestra.run.smithi167.stdout: "recovery_started": 0, 2024-08-30T09:16:21.837 INFO:teuthology.orchestra.run.smithi167.stdout: "recovery_completed": 0, 2024-08-30T09:16:21.837 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_quiesce_path": 0, 2024-08-30T09:16:21.837 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_quiesce_inode": 0, 2024-08-30T09:16:21.837 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_enqueue_scrub": 0, 2024-08-30T09:16:21.837 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_exportdir": 0, 2024-08-30T09:16:21.837 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_flush": 0, 2024-08-30T09:16:21.838 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_fragmentdir": 0, 2024-08-30T09:16:21.838 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_fragstats": 0, 2024-08-30T09:16:21.838 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_inodestats": 0, 2024-08-30T09:16:21.838 INFO:teuthology.orchestra.run.smithi167.stdout: "uninline_started": 0, 2024-08-30T09:16:21.838 INFO:teuthology.orchestra.run.smithi167.stdout: "uninline_succeeded": 0, 2024-08-30T09:16:21.838 INFO:teuthology.orchestra.run.smithi167.stdout: "uninline_write_failed": 0 2024-08-30T09:16:21.838 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-30T09:16:21.838 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_log": { 2024-08-30T09:16:21.838 INFO:teuthology.orchestra.run.smithi167.stdout: "evlrg": 0, 2024-08-30T09:16:21.838 INFO:teuthology.orchestra.run.smithi167.stdout: "evadd": 0, 2024-08-30T09:16:21.838 INFO:teuthology.orchestra.run.smithi167.stdout: "evex": 0, 2024-08-30T09:16:21.838 INFO:teuthology.orchestra.run.smithi167.stdout: "evtrm": 0, 2024-08-30T09:16:21.839 INFO:teuthology.orchestra.run.smithi167.stdout: "ev": 2094, 2024-08-30T09:16:21.839 INFO:teuthology.orchestra.run.smithi167.stdout: "evexg": 0, 2024-08-30T09:16:21.839 INFO:teuthology.orchestra.run.smithi167.stdout: "evexd": 0, 2024-08-30T09:16:21.839 INFO:teuthology.orchestra.run.smithi167.stdout: "segadd": 0, 2024-08-30T09:16:21.839 INFO:teuthology.orchestra.run.smithi167.stdout: "segex": 0, 2024-08-30T09:16:21.839 INFO:teuthology.orchestra.run.smithi167.stdout: "segtrm": 0, 2024-08-30T09:16:21.839 INFO:teuthology.orchestra.run.smithi167.stdout: "seg": 4, 2024-08-30T09:16:21.839 INFO:teuthology.orchestra.run.smithi167.stdout: "segmjr": 2, 2024-08-30T09:16:21.839 INFO:teuthology.orchestra.run.smithi167.stdout: "segexg": 0, 2024-08-30T09:16:21.839 INFO:teuthology.orchestra.run.smithi167.stdout: "segexd": 0, 2024-08-30T09:16:21.839 INFO:teuthology.orchestra.run.smithi167.stdout: "expos": 4194304, 2024-08-30T09:16:21.839 INFO:teuthology.orchestra.run.smithi167.stdout: "wrpos": 7003258, 2024-08-30T09:16:21.839 INFO:teuthology.orchestra.run.smithi167.stdout: "rdpos": 6996504, 2024-08-30T09:16:21.839 INFO:teuthology.orchestra.run.smithi167.stdout: "jlat": { 2024-08-30T09:16:21.839 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-30T09:16:21.840 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-30T09:16:21.840 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-30T09:16:21.840 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-30T09:16:21.840 INFO:teuthology.orchestra.run.smithi167.stdout: "replayed": 2094 2024-08-30T09:16:21.840 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-30T09:16:21.840 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_mem": { 2024-08-30T09:16:21.840 INFO:teuthology.orchestra.run.smithi167.stdout: "ino": 466, 2024-08-30T09:16:21.840 INFO:teuthology.orchestra.run.smithi167.stdout: "ino+": 664, 2024-08-30T09:16:21.840 INFO:teuthology.orchestra.run.smithi167.stdout: "ino-": 198, 2024-08-30T09:16:21.840 INFO:teuthology.orchestra.run.smithi167.stdout: "dir": 24, 2024-08-30T09:16:21.840 INFO:teuthology.orchestra.run.smithi167.stdout: "dir+": 29, 2024-08-30T09:16:21.840 INFO:teuthology.orchestra.run.smithi167.stdout: "dir-": 5, 2024-08-30T09:16:21.840 INFO:teuthology.orchestra.run.smithi167.stdout: "dn": 593, 2024-08-30T09:16:21.840 INFO:teuthology.orchestra.run.smithi167.stdout: "dn+": 685, 2024-08-30T09:16:21.841 INFO:teuthology.orchestra.run.smithi167.stdout: "dn-": 92, 2024-08-30T09:16:21.841 INFO:teuthology.orchestra.run.smithi167.stdout: "cap": 0, 2024-08-30T09:16:21.841 INFO:teuthology.orchestra.run.smithi167.stdout: "cap+": 0, 2024-08-30T09:16:21.841 INFO:teuthology.orchestra.run.smithi167.stdout: "cap-": 0, 2024-08-30T09:16:21.841 INFO:teuthology.orchestra.run.smithi167.stdout: "rss": 120500, 2024-08-30T09:16:21.841 INFO:teuthology.orchestra.run.smithi167.stdout: "heap": 319631360 2024-08-30T09:16:21.841 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-30T09:16:21.841 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_server": { 2024-08-30T09:16:21.841 INFO:teuthology.orchestra.run.smithi167.stdout: "dispatch_client_request": 0, 2024-08-30T09:16:21.841 INFO:teuthology.orchestra.run.smithi167.stdout: "dispatch_server_request": 0, 2024-08-30T09:16:21.841 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_client_request": 0, 2024-08-30T09:16:21.841 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_client_session": 0, 2024-08-30T09:16:21.841 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_peer_request": 0, 2024-08-30T09:16:21.841 INFO:teuthology.orchestra.run.smithi167.stdout: "req_create_latency": { 2024-08-30T09:16:21.842 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-30T09:16:21.842 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-30T09:16:21.842 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-30T09:16:21.842 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-30T09:16:21.842 INFO:teuthology.orchestra.run.smithi167.stdout: "req_getattr_latency": { 2024-08-30T09:16:21.842 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-30T09:16:21.842 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-30T09:16:21.842 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-30T09:16:21.842 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-30T09:16:21.842 INFO:teuthology.orchestra.run.smithi167.stdout: "req_getfilelock_latency": { 2024-08-30T09:16:21.842 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-30T09:16:21.842 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-30T09:16:21.842 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-30T09:16:21.842 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-30T09:16:21.842 INFO:teuthology.orchestra.run.smithi167.stdout: "req_link_latency": { 2024-08-30T09:16:21.843 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-30T09:16:21.843 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-30T09:16:21.843 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-30T09:16:21.843 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-30T09:16:21.843 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lookup_latency": { 2024-08-30T09:16:21.843 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-30T09:16:21.843 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-30T09:16:21.843 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-30T09:16:21.843 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-30T09:16:21.843 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lookuphash_latency": { 2024-08-30T09:16:21.843 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-30T09:16:21.843 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-30T09:16:21.843 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-30T09:16:21.843 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-30T09:16:21.844 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lookupino_latency": { 2024-08-30T09:16:21.844 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-30T09:16:21.844 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-30T09:16:21.844 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-30T09:16:21.844 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-30T09:16:21.844 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lookupname_latency": { 2024-08-30T09:16:21.844 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-30T09:16:21.844 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-30T09:16:21.844 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-30T09:16:21.844 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-30T09:16:21.844 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lookupparent_latency": { 2024-08-30T09:16:21.844 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-30T09:16:21.844 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-30T09:16:21.845 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-30T09:16:21.845 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-30T09:16:21.845 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lookupsnap_latency": { 2024-08-30T09:16:21.845 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-30T09:16:21.845 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-30T09:16:21.845 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-30T09:16:21.845 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-30T09:16:21.845 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lssnap_latency": { 2024-08-30T09:16:21.845 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-30T09:16:21.846 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-30T09:16:21.846 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-30T09:16:21.846 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-30T09:16:21.846 INFO:teuthology.orchestra.run.smithi167.stdout: "req_mkdir_latency": { 2024-08-30T09:16:21.846 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-30T09:16:21.846 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-30T09:16:21.846 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-30T09:16:21.847 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-30T09:16:21.847 INFO:teuthology.orchestra.run.smithi167.stdout: "req_mknod_latency": { 2024-08-30T09:16:21.847 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-30T09:16:21.847 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-30T09:16:21.847 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-30T09:16:21.847 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-30T09:16:21.847 INFO:teuthology.orchestra.run.smithi167.stdout: "req_mksnap_latency": { 2024-08-30T09:16:21.847 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-30T09:16:21.847 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-30T09:16:21.847 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-30T09:16:21.847 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-30T09:16:21.847 INFO:teuthology.orchestra.run.smithi167.stdout: "req_open_latency": { 2024-08-30T09:16:21.847 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-30T09:16:21.847 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-30T09:16:21.848 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-30T09:16:21.848 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-30T09:16:21.848 INFO:teuthology.orchestra.run.smithi167.stdout: "req_readdir_latency": { 2024-08-30T09:16:21.848 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-30T09:16:21.848 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-30T09:16:21.848 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-30T09:16:21.848 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-30T09:16:21.848 INFO:teuthology.orchestra.run.smithi167.stdout: "req_rename_latency": { 2024-08-30T09:16:21.848 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-30T09:16:21.848 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-30T09:16:21.848 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-30T09:16:21.848 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-30T09:16:21.849 INFO:teuthology.orchestra.run.smithi167.stdout: "req_renamesnap_latency": { 2024-08-30T09:16:21.849 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-30T09:16:21.849 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-30T09:16:21.849 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-30T09:16:21.849 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-30T09:16:21.849 INFO:teuthology.orchestra.run.smithi167.stdout: "req_snapdiff_latency": { 2024-08-30T09:16:21.849 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-30T09:16:21.849 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-30T09:16:21.849 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-30T09:16:21.849 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-30T09:16:21.849 INFO:teuthology.orchestra.run.smithi167.stdout: "req_rmdir_latency": { 2024-08-30T09:16:21.849 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-30T09:16:21.849 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-30T09:16:21.850 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-30T09:16:21.850 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-30T09:16:21.850 INFO:teuthology.orchestra.run.smithi167.stdout: "req_rmsnap_latency": { 2024-08-30T09:16:21.850 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-30T09:16:21.850 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-30T09:16:21.850 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-30T09:16:21.850 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-30T09:16:21.850 INFO:teuthology.orchestra.run.smithi167.stdout: "req_rmxattr_latency": { 2024-08-30T09:16:21.850 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-30T09:16:21.850 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-30T09:16:21.850 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-30T09:16:21.850 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-30T09:16:21.851 INFO:teuthology.orchestra.run.smithi167.stdout: "req_setattr_latency": { 2024-08-30T09:16:21.851 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-30T09:16:21.851 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-30T09:16:21.851 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-30T09:16:21.851 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-30T09:16:21.851 INFO:teuthology.orchestra.run.smithi167.stdout: "req_setdirlayout_latency": { 2024-08-30T09:16:21.851 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-30T09:16:21.851 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-30T09:16:21.851 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-30T09:16:21.851 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-30T09:16:21.851 INFO:teuthology.orchestra.run.smithi167.stdout: "req_setfilelock_latency": { 2024-08-30T09:16:21.851 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-30T09:16:21.851 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-30T09:16:21.852 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-30T09:16:21.852 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-30T09:16:21.852 INFO:teuthology.orchestra.run.smithi167.stdout: "req_setlayout_latency": { 2024-08-30T09:16:21.852 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-30T09:16:21.852 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-30T09:16:21.852 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-30T09:16:21.852 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-30T09:16:21.852 INFO:teuthology.orchestra.run.smithi167.stdout: "req_setxattr_latency": { 2024-08-30T09:16:21.852 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-30T09:16:21.852 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-30T09:16:21.852 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-30T09:16:21.852 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-30T09:16:21.852 INFO:teuthology.orchestra.run.smithi167.stdout: "req_symlink_latency": { 2024-08-30T09:16:21.853 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-30T09:16:21.853 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-30T09:16:21.853 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-30T09:16:21.853 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-30T09:16:21.853 INFO:teuthology.orchestra.run.smithi167.stdout: "req_unlink_latency": { 2024-08-30T09:16:21.853 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-30T09:16:21.853 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-30T09:16:21.853 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-30T09:16:21.853 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-30T09:16:21.853 INFO:teuthology.orchestra.run.smithi167.stdout: "cap_revoke_eviction": 0, 2024-08-30T09:16:21.853 INFO:teuthology.orchestra.run.smithi167.stdout: "cap_acquisition_throttle": 0, 2024-08-30T09:16:21.853 INFO:teuthology.orchestra.run.smithi167.stdout: "req_getvxattr_latency": { 2024-08-30T09:16:21.853 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-30T09:16:21.854 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-30T09:16:21.854 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-30T09:16:21.854 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-08-30T09:16:21.854 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-30T09:16:21.854 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_sessions": { 2024-08-30T09:16:21.854 INFO:teuthology.orchestra.run.smithi167.stdout: "session_count": 1, 2024-08-30T09:16:21.854 INFO:teuthology.orchestra.run.smithi167.stdout: "session_add": 2, 2024-08-30T09:16:21.854 INFO:teuthology.orchestra.run.smithi167.stdout: "session_remove": 1, 2024-08-30T09:16:21.854 INFO:teuthology.orchestra.run.smithi167.stdout: "sessions_open": 1, 2024-08-30T09:16:21.854 INFO:teuthology.orchestra.run.smithi167.stdout: "sessions_stale": 0, 2024-08-30T09:16:21.854 INFO:teuthology.orchestra.run.smithi167.stdout: "total_load": 0, 2024-08-30T09:16:21.854 INFO:teuthology.orchestra.run.smithi167.stdout: "average_load": 0, 2024-08-30T09:16:21.855 INFO:teuthology.orchestra.run.smithi167.stdout: "avg_session_uptime": 1984, 2024-08-30T09:16:21.855 INFO:teuthology.orchestra.run.smithi167.stdout: "mdthresh_evicted": 0 2024-08-30T09:16:21.855 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-30T09:16:21.855 INFO:teuthology.orchestra.run.smithi167.stdout: "mempool": { 2024-08-30T09:16:21.855 INFO:teuthology.orchestra.run.smithi167.stdout: "bloom_filter_bytes": 0, 2024-08-30T09:16:21.855 INFO:teuthology.orchestra.run.smithi167.stdout: "bloom_filter_items": 0, 2024-08-30T09:16:21.855 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_alloc_bytes": 0, 2024-08-30T09:16:21.855 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_alloc_items": 0, 2024-08-30T09:16:21.855 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_data_bytes": 0, 2024-08-30T09:16:21.855 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_data_items": 0, 2024-08-30T09:16:21.855 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_onode_bytes": 0, 2024-08-30T09:16:21.855 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_onode_items": 0, 2024-08-30T09:16:21.856 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_meta_bytes": 0, 2024-08-30T09:16:21.856 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_meta_items": 0, 2024-08-30T09:16:21.856 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_other_bytes": 0, 2024-08-30T09:16:21.856 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_other_items": 0, 2024-08-30T09:16:21.856 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_buffer_bytes": 0, 2024-08-30T09:16:21.856 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_buffer_items": 0, 2024-08-30T09:16:21.856 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_extent_bytes": 0, 2024-08-30T09:16:21.856 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_extent_items": 0, 2024-08-30T09:16:21.856 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_blob_bytes": 0, 2024-08-30T09:16:21.856 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_blob_items": 0, 2024-08-30T09:16:21.856 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_shared_blob_bytes": 0, 2024-08-30T09:16:21.856 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_shared_blob_items": 0, 2024-08-30T09:16:21.857 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_inline_bl_bytes": 0, 2024-08-30T09:16:21.857 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_inline_bl_items": 0, 2024-08-30T09:16:21.857 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_fsck_bytes": 0, 2024-08-30T09:16:21.857 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_fsck_items": 0, 2024-08-30T09:16:21.857 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_txc_bytes": 0, 2024-08-30T09:16:21.857 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_txc_items": 0, 2024-08-30T09:16:21.857 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_writing_deferred_bytes": 0, 2024-08-30T09:16:21.857 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_writing_deferred_items": 0, 2024-08-30T09:16:21.857 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_writing_bytes": 0, 2024-08-30T09:16:21.857 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_writing_items": 0, 2024-08-30T09:16:21.857 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_bytes": 0, 2024-08-30T09:16:21.857 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_items": 0, 2024-08-30T09:16:21.857 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_file_reader_bytes": 0, 2024-08-30T09:16:21.858 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_file_reader_items": 0, 2024-08-30T09:16:21.858 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_file_writer_bytes": 0, 2024-08-30T09:16:21.858 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_file_writer_items": 0, 2024-08-30T09:16:21.858 INFO:teuthology.orchestra.run.smithi167.stdout: "buffer_anon_bytes": 215159, 2024-08-30T09:16:21.858 INFO:teuthology.orchestra.run.smithi167.stdout: "buffer_anon_items": 513, 2024-08-30T09:16:21.858 INFO:teuthology.orchestra.run.smithi167.stdout: "buffer_meta_bytes": 0, 2024-08-30T09:16:21.858 INFO:teuthology.orchestra.run.smithi167.stdout: "buffer_meta_items": 0, 2024-08-30T09:16:21.858 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_bytes": 0, 2024-08-30T09:16:21.858 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_items": 0, 2024-08-30T09:16:21.858 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_mapbl_bytes": 0, 2024-08-30T09:16:21.858 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_mapbl_items": 0, 2024-08-30T09:16:21.858 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_pglog_bytes": 0, 2024-08-30T09:16:21.858 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_pglog_items": 0, 2024-08-30T09:16:21.859 INFO:teuthology.orchestra.run.smithi167.stdout: "osdmap_bytes": 8856, 2024-08-30T09:16:21.859 INFO:teuthology.orchestra.run.smithi167.stdout: "osdmap_items": 166, 2024-08-30T09:16:21.859 INFO:teuthology.orchestra.run.smithi167.stdout: "osdmap_mapping_bytes": 0, 2024-08-30T09:16:21.859 INFO:teuthology.orchestra.run.smithi167.stdout: "osdmap_mapping_items": 0, 2024-08-30T09:16:21.859 INFO:teuthology.orchestra.run.smithi167.stdout: "pgmap_bytes": 0, 2024-08-30T09:16:21.859 INFO:teuthology.orchestra.run.smithi167.stdout: "pgmap_items": 0, 2024-08-30T09:16:21.859 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_co_bytes": 1512599, 2024-08-30T09:16:21.859 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_co_items": 12893, 2024-08-30T09:16:21.859 INFO:teuthology.orchestra.run.smithi167.stdout: "unittest_1_bytes": 0, 2024-08-30T09:16:21.859 INFO:teuthology.orchestra.run.smithi167.stdout: "unittest_1_items": 0, 2024-08-30T09:16:21.859 INFO:teuthology.orchestra.run.smithi167.stdout: "unittest_2_bytes": 0, 2024-08-30T09:16:21.859 INFO:teuthology.orchestra.run.smithi167.stdout: "unittest_2_items": 0 2024-08-30T09:16:21.859 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-30T09:16:21.860 INFO:teuthology.orchestra.run.smithi167.stdout: "objecter": { 2024-08-30T09:16:21.860 INFO:teuthology.orchestra.run.smithi167.stdout: "op_active": 0, 2024-08-30T09:16:21.860 INFO:teuthology.orchestra.run.smithi167.stdout: "op_laggy": 0, 2024-08-30T09:16:21.860 INFO:teuthology.orchestra.run.smithi167.stdout: "op_send": 3255, 2024-08-30T09:16:21.860 INFO:teuthology.orchestra.run.smithi167.stdout: "op_send_bytes": 21, 2024-08-30T09:16:21.860 INFO:teuthology.orchestra.run.smithi167.stdout: "op_resend": 3, 2024-08-30T09:16:21.860 INFO:teuthology.orchestra.run.smithi167.stdout: "op_reply": 3255, 2024-08-30T09:16:21.860 INFO:teuthology.orchestra.run.smithi167.stdout: "op_latency": { 2024-08-30T09:16:21.860 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 3255, 2024-08-30T09:16:21.860 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 15.658731457, 2024-08-30T09:16:21.860 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.004810670 2024-08-30T09:16:21.860 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-30T09:16:21.860 INFO:teuthology.orchestra.run.smithi167.stdout: "op_inflight": 0, 2024-08-30T09:16:21.861 INFO:teuthology.orchestra.run.smithi167.stdout: "oplen_avg": { 2024-08-30T09:16:21.861 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 3255, 2024-08-30T09:16:21.861 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 3256 2024-08-30T09:16:21.861 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-30T09:16:21.861 INFO:teuthology.orchestra.run.smithi167.stdout: "op": 3255, 2024-08-30T09:16:21.861 INFO:teuthology.orchestra.run.smithi167.stdout: "op_r": 3255, 2024-08-30T09:16:21.861 INFO:teuthology.orchestra.run.smithi167.stdout: "op_w": 0, 2024-08-30T09:16:21.861 INFO:teuthology.orchestra.run.smithi167.stdout: "op_rmw": 0, 2024-08-30T09:16:21.861 INFO:teuthology.orchestra.run.smithi167.stdout: "op_pg": 0, 2024-08-30T09:16:21.861 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_stat": 1989, 2024-08-30T09:16:21.861 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_create": 0, 2024-08-30T09:16:21.861 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_read": 1264, 2024-08-30T09:16:21.861 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_write": 0, 2024-08-30T09:16:21.861 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_writefull": 0, 2024-08-30T09:16:21.862 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_writesame": 0, 2024-08-30T09:16:21.862 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_append": 0, 2024-08-30T09:16:21.862 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_zero": 0, 2024-08-30T09:16:21.862 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_truncate": 0, 2024-08-30T09:16:21.862 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_delete": 0, 2024-08-30T09:16:21.862 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_mapext": 0, 2024-08-30T09:16:21.862 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_sparse_read": 0, 2024-08-30T09:16:21.862 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_clonerange": 0, 2024-08-30T09:16:21.862 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_getxattr": 1, 2024-08-30T09:16:21.862 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_setxattr": 0, 2024-08-30T09:16:21.862 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_cmpxattr": 0, 2024-08-30T09:16:21.862 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_rmxattr": 0, 2024-08-30T09:16:21.862 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_resetxattrs": 0, 2024-08-30T09:16:21.862 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_call": 0, 2024-08-30T09:16:21.863 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_watch": 0, 2024-08-30T09:16:21.863 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_notify": 0, 2024-08-30T09:16:21.863 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_src_cmpxattr": 0, 2024-08-30T09:16:21.863 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_pgls": 0, 2024-08-30T09:16:21.863 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_pgls_filter": 0, 2024-08-30T09:16:21.863 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_other": 0, 2024-08-30T09:16:21.863 INFO:teuthology.orchestra.run.smithi167.stdout: "linger_active": 0, 2024-08-30T09:16:21.863 INFO:teuthology.orchestra.run.smithi167.stdout: "linger_send": 0, 2024-08-30T09:16:21.863 INFO:teuthology.orchestra.run.smithi167.stdout: "linger_resend": 0, 2024-08-30T09:16:21.863 INFO:teuthology.orchestra.run.smithi167.stdout: "linger_ping": 0, 2024-08-30T09:16:21.863 INFO:teuthology.orchestra.run.smithi167.stdout: "poolop_active": 0, 2024-08-30T09:16:21.863 INFO:teuthology.orchestra.run.smithi167.stdout: "poolop_send": 0, 2024-08-30T09:16:21.863 INFO:teuthology.orchestra.run.smithi167.stdout: "poolop_resend": 0, 2024-08-30T09:16:21.863 INFO:teuthology.orchestra.run.smithi167.stdout: "poolstat_active": 0, 2024-08-30T09:16:21.863 INFO:teuthology.orchestra.run.smithi167.stdout: "poolstat_send": 0, 2024-08-30T09:16:21.864 INFO:teuthology.orchestra.run.smithi167.stdout: "poolstat_resend": 0, 2024-08-30T09:16:21.864 INFO:teuthology.orchestra.run.smithi167.stdout: "statfs_active": 0, 2024-08-30T09:16:21.864 INFO:teuthology.orchestra.run.smithi167.stdout: "statfs_send": 0, 2024-08-30T09:16:21.864 INFO:teuthology.orchestra.run.smithi167.stdout: "statfs_resend": 0, 2024-08-30T09:16:21.864 INFO:teuthology.orchestra.run.smithi167.stdout: "command_active": 0, 2024-08-30T09:16:21.864 INFO:teuthology.orchestra.run.smithi167.stdout: "command_send": 0, 2024-08-30T09:16:21.864 INFO:teuthology.orchestra.run.smithi167.stdout: "command_resend": 0, 2024-08-30T09:16:21.864 INFO:teuthology.orchestra.run.smithi167.stdout: "map_epoch": 83, 2024-08-30T09:16:21.864 INFO:teuthology.orchestra.run.smithi167.stdout: "map_full": 0, 2024-08-30T09:16:21.864 INFO:teuthology.orchestra.run.smithi167.stdout: "map_inc": 2, 2024-08-30T09:16:21.864 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_sessions": 6, 2024-08-30T09:16:21.864 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_session_open": 9, 2024-08-30T09:16:21.865 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_session_close": 3, 2024-08-30T09:16:21.865 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_laggy": 0, 2024-08-30T09:16:21.865 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_wr": 0, 2024-08-30T09:16:21.865 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_rd": 2, 2024-08-30T09:16:21.865 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_del": 0 2024-08-30T09:16:21.865 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-30T09:16:21.865 INFO:teuthology.orchestra.run.smithi167.stdout: "oft": { 2024-08-30T09:16:21.865 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_total_objs": 0, 2024-08-30T09:16:21.865 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_total_kv_pairs": 0, 2024-08-30T09:16:21.865 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_total_updates": 0, 2024-08-30T09:16:21.865 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_total_removes": 0 2024-08-30T09:16:21.865 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-30T09:16:21.865 INFO:teuthology.orchestra.run.smithi167.stdout: "purge_queue": { 2024-08-30T09:16:21.866 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_executing_ops": 0, 2024-08-30T09:16:21.866 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_executing_ops_high_water": 0, 2024-08-30T09:16:21.866 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_executing": 0, 2024-08-30T09:16:21.866 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_executing_high_water": 0, 2024-08-30T09:16:21.866 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_executed_ops": 0, 2024-08-30T09:16:21.866 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_executed": 0, 2024-08-30T09:16:21.866 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_item_in_journal": 0 2024-08-30T09:16:21.866 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-30T09:16:21.866 INFO:teuthology.orchestra.run.smithi167.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-08-30T09:16:21.866 INFO:teuthology.orchestra.run.smithi167.stdout: "val": 0, 2024-08-30T09:16:21.866 INFO:teuthology.orchestra.run.smithi167.stdout: "max": 104857600, 2024-08-30T09:16:21.866 INFO:teuthology.orchestra.run.smithi167.stdout: "get_started": 0, 2024-08-30T09:16:21.866 INFO:teuthology.orchestra.run.smithi167.stdout: "get": 3594, 2024-08-30T09:16:21.866 INFO:teuthology.orchestra.run.smithi167.stdout: "get_sum": 4051158, 2024-08-30T09:16:21.867 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_fail": 0, 2024-08-30T09:16:21.867 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_success": 3594, 2024-08-30T09:16:21.867 INFO:teuthology.orchestra.run.smithi167.stdout: "take": 0, 2024-08-30T09:16:21.867 INFO:teuthology.orchestra.run.smithi167.stdout: "take_sum": 0, 2024-08-30T09:16:21.867 INFO:teuthology.orchestra.run.smithi167.stdout: "put": 3594, 2024-08-30T09:16:21.867 INFO:teuthology.orchestra.run.smithi167.stdout: "put_sum": 4051158, 2024-08-30T09:16:21.867 INFO:teuthology.orchestra.run.smithi167.stdout: "wait": { 2024-08-30T09:16:21.867 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-30T09:16:21.867 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-30T09:16:21.867 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-30T09:16:21.867 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-08-30T09:16:21.867 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-30T09:16:21.867 INFO:teuthology.orchestra.run.smithi167.stdout: "throttle-objecter_bytes": { 2024-08-30T09:16:21.867 INFO:teuthology.orchestra.run.smithi167.stdout: "val": 0, 2024-08-30T09:16:21.868 INFO:teuthology.orchestra.run.smithi167.stdout: "max": 104857600, 2024-08-30T09:16:21.868 INFO:teuthology.orchestra.run.smithi167.stdout: "get_started": 0, 2024-08-30T09:16:21.868 INFO:teuthology.orchestra.run.smithi167.stdout: "get": 0, 2024-08-30T09:16:21.868 INFO:teuthology.orchestra.run.smithi167.stdout: "get_sum": 0, 2024-08-30T09:16:21.868 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_fail": 0, 2024-08-30T09:16:21.868 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_success": 0, 2024-08-30T09:16:21.868 INFO:teuthology.orchestra.run.smithi167.stdout: "take": 3255, 2024-08-30T09:16:21.868 INFO:teuthology.orchestra.run.smithi167.stdout: "take_sum": 2808959, 2024-08-30T09:16:21.868 INFO:teuthology.orchestra.run.smithi167.stdout: "put": 3255, 2024-08-30T09:16:21.868 INFO:teuthology.orchestra.run.smithi167.stdout: "put_sum": 2808959, 2024-08-30T09:16:21.868 INFO:teuthology.orchestra.run.smithi167.stdout: "wait": { 2024-08-30T09:16:21.868 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-30T09:16:21.868 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-30T09:16:21.868 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-30T09:16:21.869 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-08-30T09:16:21.869 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-30T09:16:21.869 INFO:teuthology.orchestra.run.smithi167.stdout: "throttle-objecter_ops": { 2024-08-30T09:16:21.869 INFO:teuthology.orchestra.run.smithi167.stdout: "val": 0, 2024-08-30T09:16:21.869 INFO:teuthology.orchestra.run.smithi167.stdout: "max": 1024, 2024-08-30T09:16:21.869 INFO:teuthology.orchestra.run.smithi167.stdout: "get_started": 0, 2024-08-30T09:16:21.869 INFO:teuthology.orchestra.run.smithi167.stdout: "get": 0, 2024-08-30T09:16:21.869 INFO:teuthology.orchestra.run.smithi167.stdout: "get_sum": 0, 2024-08-30T09:16:21.869 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_fail": 0, 2024-08-30T09:16:21.869 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_success": 0, 2024-08-30T09:16:21.869 INFO:teuthology.orchestra.run.smithi167.stdout: "take": 3255, 2024-08-30T09:16:21.869 INFO:teuthology.orchestra.run.smithi167.stdout: "take_sum": 3255, 2024-08-30T09:16:21.869 INFO:teuthology.orchestra.run.smithi167.stdout: "put": 3255, 2024-08-30T09:16:21.870 INFO:teuthology.orchestra.run.smithi167.stdout: "put_sum": 3255, 2024-08-30T09:16:21.870 INFO:teuthology.orchestra.run.smithi167.stdout: "wait": { 2024-08-30T09:16:21.870 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-30T09:16:21.870 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-30T09:16:21.870 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-30T09:16:21.870 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-08-30T09:16:21.870 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-30T09:16:21.870 INFO:teuthology.orchestra.run.smithi167.stdout: "throttle-write_buf_throttle": { 2024-08-30T09:16:21.870 INFO:teuthology.orchestra.run.smithi167.stdout: "val": 0, 2024-08-30T09:16:21.870 INFO:teuthology.orchestra.run.smithi167.stdout: "max": 3758096384, 2024-08-30T09:16:21.870 INFO:teuthology.orchestra.run.smithi167.stdout: "get_started": 0, 2024-08-30T09:16:21.870 INFO:teuthology.orchestra.run.smithi167.stdout: "get": 0, 2024-08-30T09:16:21.870 INFO:teuthology.orchestra.run.smithi167.stdout: "get_sum": 0, 2024-08-30T09:16:21.871 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_fail": 0, 2024-08-30T09:16:21.871 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_success": 0, 2024-08-30T09:16:21.871 INFO:teuthology.orchestra.run.smithi167.stdout: "take": 0, 2024-08-30T09:16:21.871 INFO:teuthology.orchestra.run.smithi167.stdout: "take_sum": 0, 2024-08-30T09:16:21.871 INFO:teuthology.orchestra.run.smithi167.stdout: "put": 0, 2024-08-30T09:16:21.871 INFO:teuthology.orchestra.run.smithi167.stdout: "put_sum": 0, 2024-08-30T09:16:21.871 INFO:teuthology.orchestra.run.smithi167.stdout: "wait": { 2024-08-30T09:16:21.871 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-30T09:16:21.871 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-30T09:16:21.871 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-30T09:16:21.871 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-08-30T09:16:21.871 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-30T09:16:21.871 INFO:teuthology.orchestra.run.smithi167.stdout: "throttle-write_buf_throttle-0x5633e59028f0": { 2024-08-30T09:16:21.871 INFO:teuthology.orchestra.run.smithi167.stdout: "val": 0, 2024-08-30T09:16:21.872 INFO:teuthology.orchestra.run.smithi167.stdout: "max": 3758096384, 2024-08-30T09:16:21.872 INFO:teuthology.orchestra.run.smithi167.stdout: "get_started": 0, 2024-08-30T09:16:21.872 INFO:teuthology.orchestra.run.smithi167.stdout: "get": 0, 2024-08-30T09:16:21.872 INFO:teuthology.orchestra.run.smithi167.stdout: "get_sum": 0, 2024-08-30T09:16:21.872 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_fail": 0, 2024-08-30T09:16:21.872 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_success": 0, 2024-08-30T09:16:21.872 INFO:teuthology.orchestra.run.smithi167.stdout: "take": 0, 2024-08-30T09:16:21.872 INFO:teuthology.orchestra.run.smithi167.stdout: "take_sum": 0, 2024-08-30T09:16:21.872 INFO:teuthology.orchestra.run.smithi167.stdout: "put": 0, 2024-08-30T09:16:21.872 INFO:teuthology.orchestra.run.smithi167.stdout: "put_sum": 0, 2024-08-30T09:16:21.872 INFO:teuthology.orchestra.run.smithi167.stdout: "wait": { 2024-08-30T09:16:21.872 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-30T09:16:21.872 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-30T09:16:21.872 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-30T09:16:21.873 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-08-30T09:16:21.873 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-30T09:16:21.873 INFO:teuthology.orchestra.run.smithi167.stdout: "trackedop": { 2024-08-30T09:16:21.873 INFO:teuthology.orchestra.run.smithi167.stdout: "slow_ops_count": 0 2024-08-30T09:16:21.873 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-08-30T09:16:21.873 INFO:teuthology.orchestra.run.smithi167.stdout:} 2024-08-30T09:16:22.692 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-08-30T09:16:22.692 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-08-30T09:16:22.692 DEBUG:tasks.check_counter:Getting stats from h 2024-08-30T09:16:22.692 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph daemon mds.h perf dump 2024-08-30T09:16:22.953 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.b/config 2024-08-30T09:16:23.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:16:23 smithi167 ceph-mon[22063]: pgmap v841: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 7.7 KiB/s wr, 7 op/s 2024-08-30T09:16:23.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:16:23 smithi042 ceph-mon[18652]: pgmap v841: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 7.7 KiB/s wr, 7 op/s 2024-08-30T09:16:23.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:16:23 smithi190 ceph-mon[22001]: pgmap v841: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 7.7 KiB/s wr, 7 op/s 2024-08-30T09:16:23.594 INFO:teuthology.orchestra.run.smithi167.stdout:{ 2024-08-30T09:16:23.594 INFO:teuthology.orchestra.run.smithi167.stdout: "AsyncMessenger::Worker-0": { 2024-08-30T09:16:23.595 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_messages": 8785, 2024-08-30T09:16:23.595 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages": 10188, 2024-08-30T09:16:23.595 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_bytes": 4394457, 2024-08-30T09:16:23.595 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_bytes": 7781618, 2024-08-30T09:16:23.595 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_created_connections": 48, 2024-08-30T09:16:23.595 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_active_connections": 5, 2024-08-30T09:16:23.595 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_total_time": 0.881366296, 2024-08-30T09:16:23.595 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_send_time": 0.448484720, 2024-08-30T09:16:23.595 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_recv_time": 0.347502985, 2024-08-30T09:16:23.595 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_fast_dispatch_time": 0.103881142, 2024-08-30T09:16:23.595 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages_queue_lat": { 2024-08-30T09:16:23.595 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 10185, 2024-08-30T09:16:23.595 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 1.807091293, 2024-08-30T09:16:23.596 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000177426 2024-08-30T09:16:23.596 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-30T09:16:23.596 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_handle_ack_lat": { 2024-08-30T09:16:23.596 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 6718, 2024-08-30T09:16:23.596 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.001655149, 2024-08-30T09:16:23.596 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000246 2024-08-30T09:16:23.596 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-30T09:16:23.596 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_encrypted_bytes": 2138400, 2024-08-30T09:16:23.596 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_encrypted_bytes": 3039584 2024-08-30T09:16:23.596 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-30T09:16:23.596 INFO:teuthology.orchestra.run.smithi167.stdout: "AsyncMessenger::Worker-1": { 2024-08-30T09:16:23.596 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_messages": 3586, 2024-08-30T09:16:23.596 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages": 3654, 2024-08-30T09:16:23.597 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_bytes": 2035770, 2024-08-30T09:16:23.597 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_bytes": 1860941, 2024-08-30T09:16:23.597 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_created_connections": 11, 2024-08-30T09:16:23.597 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_active_connections": 6, 2024-08-30T09:16:23.597 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_total_time": 0.291028494, 2024-08-30T09:16:23.597 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_send_time": 0.127208873, 2024-08-30T09:16:23.597 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_recv_time": 0.080275866, 2024-08-30T09:16:23.597 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_fast_dispatch_time": 0.066174775, 2024-08-30T09:16:23.597 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages_queue_lat": { 2024-08-30T09:16:23.597 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 3653, 2024-08-30T09:16:23.597 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.399575612, 2024-08-30T09:16:23.597 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000109382 2024-08-30T09:16:23.597 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-30T09:16:23.598 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_handle_ack_lat": { 2024-08-30T09:16:23.598 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 781, 2024-08-30T09:16:23.598 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000188430, 2024-08-30T09:16:23.598 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000241 2024-08-30T09:16:23.598 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-30T09:16:23.598 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_encrypted_bytes": 3104, 2024-08-30T09:16:23.598 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_encrypted_bytes": 480624 2024-08-30T09:16:23.598 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-30T09:16:23.598 INFO:teuthology.orchestra.run.smithi167.stdout: "AsyncMessenger::Worker-2": { 2024-08-30T09:16:23.598 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_messages": 6421, 2024-08-30T09:16:23.598 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages": 6348, 2024-08-30T09:16:23.598 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_bytes": 2835903, 2024-08-30T09:16:23.598 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_bytes": 2220444, 2024-08-30T09:16:23.599 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_created_connections": 8, 2024-08-30T09:16:23.599 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_active_connections": 5, 2024-08-30T09:16:23.599 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_total_time": 0.554623159, 2024-08-30T09:16:23.599 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_send_time": 0.279445420, 2024-08-30T09:16:23.599 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_recv_time": 0.241305492, 2024-08-30T09:16:23.599 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_fast_dispatch_time": 0.050468233, 2024-08-30T09:16:23.599 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages_queue_lat": { 2024-08-30T09:16:23.599 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 6348, 2024-08-30T09:16:23.599 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.522415354, 2024-08-30T09:16:23.599 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000082296 2024-08-30T09:16:23.599 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-30T09:16:23.599 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_handle_ack_lat": { 2024-08-30T09:16:23.599 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 8242, 2024-08-30T09:16:23.600 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.002144858, 2024-08-30T09:16:23.600 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000260 2024-08-30T09:16:23.600 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-30T09:16:23.600 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_encrypted_bytes": 0, 2024-08-30T09:16:23.600 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_encrypted_bytes": 0 2024-08-30T09:16:23.600 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-30T09:16:23.600 INFO:teuthology.orchestra.run.smithi167.stdout: "cct": { 2024-08-30T09:16:23.600 INFO:teuthology.orchestra.run.smithi167.stdout: "total_workers": 1, 2024-08-30T09:16:23.600 INFO:teuthology.orchestra.run.smithi167.stdout: "unhealthy_workers": 0 2024-08-30T09:16:23.600 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-30T09:16:23.600 INFO:teuthology.orchestra.run.smithi167.stdout: "finisher-MDSRank": { 2024-08-30T09:16:23.600 INFO:teuthology.orchestra.run.smithi167.stdout: "queue_len": 0, 2024-08-30T09:16:23.601 INFO:teuthology.orchestra.run.smithi167.stdout: "complete_latency": { 2024-08-30T09:16:23.601 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 5307, 2024-08-30T09:16:23.601 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 6.310275028, 2024-08-30T09:16:23.601 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.001189047 2024-08-30T09:16:23.601 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-08-30T09:16:23.601 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-30T09:16:23.601 INFO:teuthology.orchestra.run.smithi167.stdout: "finisher-PurgeQueue": { 2024-08-30T09:16:23.601 INFO:teuthology.orchestra.run.smithi167.stdout: "queue_len": 0, 2024-08-30T09:16:23.601 INFO:teuthology.orchestra.run.smithi167.stdout: "complete_latency": { 2024-08-30T09:16:23.601 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 637, 2024-08-30T09:16:23.601 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.029277434, 2024-08-30T09:16:23.601 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000045961 2024-08-30T09:16:23.601 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-08-30T09:16:23.602 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-30T09:16:23.602 INFO:teuthology.orchestra.run.smithi167.stdout: "mds": { 2024-08-30T09:16:23.602 INFO:teuthology.orchestra.run.smithi167.stdout: "request": 1256, 2024-08-30T09:16:23.602 INFO:teuthology.orchestra.run.smithi167.stdout: "reply": 1255, 2024-08-30T09:16:23.602 INFO:teuthology.orchestra.run.smithi167.stdout: "reply_latency": { 2024-08-30T09:16:23.602 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 1255, 2024-08-30T09:16:23.602 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 48.816068644, 2024-08-30T09:16:23.602 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.038897265 2024-08-30T09:16:23.602 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-30T09:16:23.602 INFO:teuthology.orchestra.run.smithi167.stdout: "slow_reply": 0, 2024-08-30T09:16:23.602 INFO:teuthology.orchestra.run.smithi167.stdout: "forward": 1, 2024-08-30T09:16:23.602 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_fetch_complete": 2, 2024-08-30T09:16:23.602 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_fetch_keys": 0, 2024-08-30T09:16:23.603 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_commit": 11, 2024-08-30T09:16:23.603 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_split": 0, 2024-08-30T09:16:23.603 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_merge": 0, 2024-08-30T09:16:23.603 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes": 479, 2024-08-30T09:16:23.603 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes_top": 1, 2024-08-30T09:16:23.603 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes_bottom": 455, 2024-08-30T09:16:23.603 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes_pin_tail": 23, 2024-08-30T09:16:23.603 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes_pinned": 477, 2024-08-30T09:16:23.603 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes_expired": 96, 2024-08-30T09:16:23.603 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes_with_caps": 459, 2024-08-30T09:16:23.603 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": 459, 2024-08-30T09:16:23.603 INFO:teuthology.orchestra.run.smithi167.stdout: "subtrees": 15, 2024-08-30T09:16:23.603 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse": 10103, 2024-08-30T09:16:23.603 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse_hit": 9932, 2024-08-30T09:16:23.604 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse_forward": 0, 2024-08-30T09:16:23.604 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse_discover": 10, 2024-08-30T09:16:23.604 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse_dir_fetch": 0, 2024-08-30T09:16:23.604 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse_remote_ino": 0, 2024-08-30T09:16:23.604 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse_lock": 35, 2024-08-30T09:16:23.604 INFO:teuthology.orchestra.run.smithi167.stdout: "load_cent": 9371, 2024-08-30T09:16:23.604 INFO:teuthology.orchestra.run.smithi167.stdout: "q": 0, 2024-08-30T09:16:23.604 INFO:teuthology.orchestra.run.smithi167.stdout: "exported": 4, 2024-08-30T09:16:23.604 INFO:teuthology.orchestra.run.smithi167.stdout: "exported_inodes": 87, 2024-08-30T09:16:23.604 INFO:teuthology.orchestra.run.smithi167.stdout: "imported": 16, 2024-08-30T09:16:23.604 INFO:teuthology.orchestra.run.smithi167.stdout: "imported_inodes": 547, 2024-08-30T09:16:23.604 INFO:teuthology.orchestra.run.smithi167.stdout: "openino_dir_fetch": 0, 2024-08-30T09:16:23.604 INFO:teuthology.orchestra.run.smithi167.stdout: "openino_backtrace_fetch": 7676, 2024-08-30T09:16:23.605 INFO:teuthology.orchestra.run.smithi167.stdout: "openino_peer_discover": 1, 2024-08-30T09:16:23.605 INFO:teuthology.orchestra.run.smithi167.stdout: "root_rfiles": 1023, 2024-08-30T09:16:23.605 INFO:teuthology.orchestra.run.smithi167.stdout: "root_rbytes": 1605055526, 2024-08-30T09:16:23.605 INFO:teuthology.orchestra.run.smithi167.stdout: "root_rsnaps": 0, 2024-08-30T09:16:23.605 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_backtrace_fetch": 7676, 2024-08-30T09:16:23.605 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_set_tag": 0, 2024-08-30T09:16:23.605 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_backtrace_repaired": 0, 2024-08-30T09:16:23.605 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_inotable_repaired": 0, 2024-08-30T09:16:23.605 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_dir_inodes": 0, 2024-08-30T09:16:23.605 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_dir_base_inodes": 0, 2024-08-30T09:16:23.605 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_dirfrag_rstats": 0, 2024-08-30T09:16:23.605 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_file_inodes": 7676, 2024-08-30T09:16:23.605 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_inode_file_caps": 87, 2024-08-30T09:16:23.606 INFO:teuthology.orchestra.run.smithi167.stdout: "ceph_cap_op_revoke": 1904, 2024-08-30T09:16:23.606 INFO:teuthology.orchestra.run.smithi167.stdout: "ceph_cap_op_grant": 960, 2024-08-30T09:16:23.606 INFO:teuthology.orchestra.run.smithi167.stdout: "ceph_cap_op_trunc": 1, 2024-08-30T09:16:23.606 INFO:teuthology.orchestra.run.smithi167.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-08-30T09:16:23.606 INFO:teuthology.orchestra.run.smithi167.stdout: "ceph_cap_op_flush_ack": 0, 2024-08-30T09:16:23.606 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_client_caps": 2501, 2024-08-30T09:16:23.606 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_client_caps_dirty": 424, 2024-08-30T09:16:23.606 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_client_cap_release": 63, 2024-08-30T09:16:23.606 INFO:teuthology.orchestra.run.smithi167.stdout: "process_request_cap_release": 461 2024-08-30T09:16:23.606 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-30T09:16:23.606 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_cache": { 2024-08-30T09:16:23.606 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_update": 0, 2024-08-30T09:16:23.606 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_update_receipt": 0, 2024-08-30T09:16:23.607 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_try_discover": 0, 2024-08-30T09:16:23.607 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_send_discover": 11, 2024-08-30T09:16:23.607 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_handle_discover": 95, 2024-08-30T09:16:23.607 INFO:teuthology.orchestra.run.smithi167.stdout: "num_strays": 0, 2024-08-30T09:16:23.607 INFO:teuthology.orchestra.run.smithi167.stdout: "num_strays_delayed": 0, 2024-08-30T09:16:23.607 INFO:teuthology.orchestra.run.smithi167.stdout: "num_strays_enqueuing": 0, 2024-08-30T09:16:23.607 INFO:teuthology.orchestra.run.smithi167.stdout: "strays_created": 120, 2024-08-30T09:16:23.607 INFO:teuthology.orchestra.run.smithi167.stdout: "strays_enqueued": 120, 2024-08-30T09:16:23.607 INFO:teuthology.orchestra.run.smithi167.stdout: "strays_reintegrated": 0, 2024-08-30T09:16:23.607 INFO:teuthology.orchestra.run.smithi167.stdout: "strays_migrated": 0, 2024-08-30T09:16:23.607 INFO:teuthology.orchestra.run.smithi167.stdout: "num_recovering_processing": 0, 2024-08-30T09:16:23.607 INFO:teuthology.orchestra.run.smithi167.stdout: "num_recovering_enqueued": 0, 2024-08-30T09:16:23.607 INFO:teuthology.orchestra.run.smithi167.stdout: "num_recovering_prioritized": 0, 2024-08-30T09:16:23.607 INFO:teuthology.orchestra.run.smithi167.stdout: "recovery_started": 0, 2024-08-30T09:16:23.608 INFO:teuthology.orchestra.run.smithi167.stdout: "recovery_completed": 0, 2024-08-30T09:16:23.608 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_quiesce_path": 0, 2024-08-30T09:16:23.608 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_quiesce_inode": 0, 2024-08-30T09:16:23.608 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_enqueue_scrub": 0, 2024-08-30T09:16:23.608 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_exportdir": 4, 2024-08-30T09:16:23.608 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_flush": 0, 2024-08-30T09:16:23.608 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_fragmentdir": 0, 2024-08-30T09:16:23.608 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_fragstats": 0, 2024-08-30T09:16:23.608 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_inodestats": 0, 2024-08-30T09:16:23.608 INFO:teuthology.orchestra.run.smithi167.stdout: "uninline_started": 0, 2024-08-30T09:16:23.608 INFO:teuthology.orchestra.run.smithi167.stdout: "uninline_succeeded": 0, 2024-08-30T09:16:23.608 INFO:teuthology.orchestra.run.smithi167.stdout: "uninline_write_failed": 0 2024-08-30T09:16:23.609 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-30T09:16:23.609 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_log": { 2024-08-30T09:16:23.609 INFO:teuthology.orchestra.run.smithi167.stdout: "evlrg": 0, 2024-08-30T09:16:23.609 INFO:teuthology.orchestra.run.smithi167.stdout: "evadd": 2094, 2024-08-30T09:16:23.609 INFO:teuthology.orchestra.run.smithi167.stdout: "evex": 0, 2024-08-30T09:16:23.609 INFO:teuthology.orchestra.run.smithi167.stdout: "evtrm": 0, 2024-08-30T09:16:23.609 INFO:teuthology.orchestra.run.smithi167.stdout: "ev": 2094, 2024-08-30T09:16:23.609 INFO:teuthology.orchestra.run.smithi167.stdout: "evexg": 0, 2024-08-30T09:16:23.609 INFO:teuthology.orchestra.run.smithi167.stdout: "evexd": 0, 2024-08-30T09:16:23.609 INFO:teuthology.orchestra.run.smithi167.stdout: "segadd": 4, 2024-08-30T09:16:23.609 INFO:teuthology.orchestra.run.smithi167.stdout: "segex": 0, 2024-08-30T09:16:23.609 INFO:teuthology.orchestra.run.smithi167.stdout: "segtrm": 0, 2024-08-30T09:16:23.610 INFO:teuthology.orchestra.run.smithi167.stdout: "seg": 4, 2024-08-30T09:16:23.610 INFO:teuthology.orchestra.run.smithi167.stdout: "segmjr": 2, 2024-08-30T09:16:23.610 INFO:teuthology.orchestra.run.smithi167.stdout: "segexg": 0, 2024-08-30T09:16:23.610 INFO:teuthology.orchestra.run.smithi167.stdout: "segexd": 0, 2024-08-30T09:16:23.610 INFO:teuthology.orchestra.run.smithi167.stdout: "expos": 4194304, 2024-08-30T09:16:23.610 INFO:teuthology.orchestra.run.smithi167.stdout: "wrpos": 7003258, 2024-08-30T09:16:23.610 INFO:teuthology.orchestra.run.smithi167.stdout: "rdpos": 0, 2024-08-30T09:16:23.610 INFO:teuthology.orchestra.run.smithi167.stdout: "jlat": { 2024-08-30T09:16:23.610 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 1073, 2024-08-30T09:16:23.610 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 16.781501617, 2024-08-30T09:16:23.611 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.015639796 2024-08-30T09:16:23.611 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-30T09:16:23.611 INFO:teuthology.orchestra.run.smithi167.stdout: "replayed": 0 2024-08-30T09:16:23.611 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-30T09:16:23.611 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_mem": { 2024-08-30T09:16:23.611 INFO:teuthology.orchestra.run.smithi167.stdout: "ino": 481, 2024-08-30T09:16:23.611 INFO:teuthology.orchestra.run.smithi167.stdout: "ino+": 681, 2024-08-30T09:16:23.611 INFO:teuthology.orchestra.run.smithi167.stdout: "ino-": 200, 2024-08-30T09:16:23.611 INFO:teuthology.orchestra.run.smithi167.stdout: "dir": 34, 2024-08-30T09:16:23.611 INFO:teuthology.orchestra.run.smithi167.stdout: "dir+": 37, 2024-08-30T09:16:23.611 INFO:teuthology.orchestra.run.smithi167.stdout: "dir-": 3, 2024-08-30T09:16:23.611 INFO:teuthology.orchestra.run.smithi167.stdout: "dn": 482, 2024-08-30T09:16:23.611 INFO:teuthology.orchestra.run.smithi167.stdout: "dn+": 702, 2024-08-30T09:16:23.612 INFO:teuthology.orchestra.run.smithi167.stdout: "dn-": 220, 2024-08-30T09:16:23.612 INFO:teuthology.orchestra.run.smithi167.stdout: "cap": 459, 2024-08-30T09:16:23.612 INFO:teuthology.orchestra.run.smithi167.stdout: "cap+": 1025, 2024-08-30T09:16:23.612 INFO:teuthology.orchestra.run.smithi167.stdout: "cap-": 566, 2024-08-30T09:16:23.612 INFO:teuthology.orchestra.run.smithi167.stdout: "rss": 128532, 2024-08-30T09:16:23.612 INFO:teuthology.orchestra.run.smithi167.stdout: "heap": 335425536 2024-08-30T09:16:23.612 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-30T09:16:23.612 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_server": { 2024-08-30T09:16:23.612 INFO:teuthology.orchestra.run.smithi167.stdout: "dispatch_client_request": 2105, 2024-08-30T09:16:23.612 INFO:teuthology.orchestra.run.smithi167.stdout: "dispatch_server_request": 34, 2024-08-30T09:16:23.612 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_client_request": 1256, 2024-08-30T09:16:23.612 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_client_session": 19, 2024-08-30T09:16:23.612 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_peer_request": 319, 2024-08-30T09:16:23.612 INFO:teuthology.orchestra.run.smithi167.stdout: "req_create_latency": { 2024-08-30T09:16:23.613 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 114, 2024-08-30T09:16:23.613 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.218657025, 2024-08-30T09:16:23.613 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.001918044 2024-08-30T09:16:23.613 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-30T09:16:23.613 INFO:teuthology.orchestra.run.smithi167.stdout: "req_getattr_latency": { 2024-08-30T09:16:23.613 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 666, 2024-08-30T09:16:23.613 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 12.541530184, 2024-08-30T09:16:23.613 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.018831126 2024-08-30T09:16:23.613 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-30T09:16:23.613 INFO:teuthology.orchestra.run.smithi167.stdout: "req_getfilelock_latency": { 2024-08-30T09:16:23.614 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-30T09:16:23.614 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-30T09:16:23.614 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-30T09:16:23.614 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-30T09:16:23.614 INFO:teuthology.orchestra.run.smithi167.stdout: "req_link_latency": { 2024-08-30T09:16:23.614 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-30T09:16:23.614 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-30T09:16:23.614 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-30T09:16:23.614 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-30T09:16:23.614 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lookup_latency": { 2024-08-30T09:16:23.614 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 287, 2024-08-30T09:16:23.614 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 34.853730655, 2024-08-30T09:16:23.614 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.121441570 2024-08-30T09:16:23.615 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-30T09:16:23.615 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lookuphash_latency": { 2024-08-30T09:16:23.615 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-30T09:16:23.615 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-30T09:16:23.615 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-30T09:16:23.615 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-30T09:16:23.615 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lookupino_latency": { 2024-08-30T09:16:23.615 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-30T09:16:23.615 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-30T09:16:23.615 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-30T09:16:23.615 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-30T09:16:23.615 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lookupname_latency": { 2024-08-30T09:16:23.615 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-30T09:16:23.616 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-30T09:16:23.616 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-30T09:16:23.616 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-30T09:16:23.616 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lookupparent_latency": { 2024-08-30T09:16:23.616 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-30T09:16:23.616 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-30T09:16:23.616 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-30T09:16:23.616 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-30T09:16:23.616 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lookupsnap_latency": { 2024-08-30T09:16:23.616 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-30T09:16:23.616 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-30T09:16:23.617 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-30T09:16:23.617 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-30T09:16:23.617 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lssnap_latency": { 2024-08-30T09:16:23.617 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-30T09:16:23.617 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-30T09:16:23.617 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-30T09:16:23.617 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-30T09:16:23.617 INFO:teuthology.orchestra.run.smithi167.stdout: "req_mkdir_latency": { 2024-08-30T09:16:23.617 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-30T09:16:23.618 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-30T09:16:23.618 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-30T09:16:23.618 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-30T09:16:23.618 INFO:teuthology.orchestra.run.smithi167.stdout: "req_mknod_latency": { 2024-08-30T09:16:23.618 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-30T09:16:23.618 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-30T09:16:23.618 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-30T09:16:23.618 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-30T09:16:23.618 INFO:teuthology.orchestra.run.smithi167.stdout: "req_mksnap_latency": { 2024-08-30T09:16:23.618 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-30T09:16:23.618 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-30T09:16:23.618 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-30T09:16:23.618 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-30T09:16:23.618 INFO:teuthology.orchestra.run.smithi167.stdout: "req_open_latency": { 2024-08-30T09:16:23.619 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 55, 2024-08-30T09:16:23.619 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.043837722, 2024-08-30T09:16:23.619 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000797049 2024-08-30T09:16:23.619 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-30T09:16:23.619 INFO:teuthology.orchestra.run.smithi167.stdout: "req_readdir_latency": { 2024-08-30T09:16:23.619 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 12, 2024-08-30T09:16:23.619 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.010240648, 2024-08-30T09:16:23.619 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000853387 2024-08-30T09:16:23.619 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-30T09:16:23.619 INFO:teuthology.orchestra.run.smithi167.stdout: "req_rename_latency": { 2024-08-30T09:16:23.619 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 114, 2024-08-30T09:16:23.619 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 1.129474043, 2024-08-30T09:16:23.619 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.009907667 2024-08-30T09:16:23.620 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-30T09:16:23.620 INFO:teuthology.orchestra.run.smithi167.stdout: "req_renamesnap_latency": { 2024-08-30T09:16:23.620 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-30T09:16:23.620 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-30T09:16:23.620 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-30T09:16:23.620 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-30T09:16:23.620 INFO:teuthology.orchestra.run.smithi167.stdout: "req_snapdiff_latency": { 2024-08-30T09:16:23.620 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-30T09:16:23.620 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-30T09:16:23.620 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-30T09:16:23.620 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-30T09:16:23.620 INFO:teuthology.orchestra.run.smithi167.stdout: "req_rmdir_latency": { 2024-08-30T09:16:23.620 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-30T09:16:23.621 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-30T09:16:23.621 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-30T09:16:23.621 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-30T09:16:23.621 INFO:teuthology.orchestra.run.smithi167.stdout: "req_rmsnap_latency": { 2024-08-30T09:16:23.621 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-30T09:16:23.621 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-30T09:16:23.621 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-30T09:16:23.621 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-30T09:16:23.621 INFO:teuthology.orchestra.run.smithi167.stdout: "req_rmxattr_latency": { 2024-08-30T09:16:23.621 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-30T09:16:23.621 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-30T09:16:23.621 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-30T09:16:23.621 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-30T09:16:23.621 INFO:teuthology.orchestra.run.smithi167.stdout: "req_setattr_latency": { 2024-08-30T09:16:23.621 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 1, 2024-08-30T09:16:23.622 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.006692513, 2024-08-30T09:16:23.622 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.006692513 2024-08-30T09:16:23.622 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-30T09:16:23.622 INFO:teuthology.orchestra.run.smithi167.stdout: "req_setdirlayout_latency": { 2024-08-30T09:16:23.622 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-30T09:16:23.622 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-30T09:16:23.622 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-30T09:16:23.622 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-30T09:16:23.622 INFO:teuthology.orchestra.run.smithi167.stdout: "req_setfilelock_latency": { 2024-08-30T09:16:23.622 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-30T09:16:23.622 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-30T09:16:23.622 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-30T09:16:23.623 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-30T09:16:23.623 INFO:teuthology.orchestra.run.smithi167.stdout: "req_setlayout_latency": { 2024-08-30T09:16:23.623 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-30T09:16:23.623 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-30T09:16:23.623 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-30T09:16:23.623 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-30T09:16:23.623 INFO:teuthology.orchestra.run.smithi167.stdout: "req_setxattr_latency": { 2024-08-30T09:16:23.623 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-30T09:16:23.623 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-30T09:16:23.623 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-30T09:16:23.623 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-30T09:16:23.623 INFO:teuthology.orchestra.run.smithi167.stdout: "req_symlink_latency": { 2024-08-30T09:16:23.623 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-30T09:16:23.624 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-30T09:16:23.624 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-30T09:16:23.624 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-30T09:16:23.624 INFO:teuthology.orchestra.run.smithi167.stdout: "req_unlink_latency": { 2024-08-30T09:16:23.624 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 6, 2024-08-30T09:16:23.624 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.011905854, 2024-08-30T09:16:23.624 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.001984309 2024-08-30T09:16:23.624 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-30T09:16:23.624 INFO:teuthology.orchestra.run.smithi167.stdout: "cap_revoke_eviction": 0, 2024-08-30T09:16:23.624 INFO:teuthology.orchestra.run.smithi167.stdout: "cap_acquisition_throttle": 0, 2024-08-30T09:16:23.624 INFO:teuthology.orchestra.run.smithi167.stdout: "req_getvxattr_latency": { 2024-08-30T09:16:23.624 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-30T09:16:23.625 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-30T09:16:23.625 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-30T09:16:23.625 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-08-30T09:16:23.625 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-30T09:16:23.625 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_sessions": { 2024-08-30T09:16:23.625 INFO:teuthology.orchestra.run.smithi167.stdout: "session_count": 1, 2024-08-30T09:16:23.625 INFO:teuthology.orchestra.run.smithi167.stdout: "session_add": 2, 2024-08-30T09:16:23.625 INFO:teuthology.orchestra.run.smithi167.stdout: "session_remove": 1, 2024-08-30T09:16:23.625 INFO:teuthology.orchestra.run.smithi167.stdout: "sessions_open": 1, 2024-08-30T09:16:23.625 INFO:teuthology.orchestra.run.smithi167.stdout: "sessions_stale": 0, 2024-08-30T09:16:23.625 INFO:teuthology.orchestra.run.smithi167.stdout: "total_load": 139, 2024-08-30T09:16:23.625 INFO:teuthology.orchestra.run.smithi167.stdout: "average_load": 139, 2024-08-30T09:16:23.625 INFO:teuthology.orchestra.run.smithi167.stdout: "avg_session_uptime": 947, 2024-08-30T09:16:23.625 INFO:teuthology.orchestra.run.smithi167.stdout: "mdthresh_evicted": 0 2024-08-30T09:16:23.625 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-30T09:16:23.626 INFO:teuthology.orchestra.run.smithi167.stdout: "mempool": { 2024-08-30T09:16:23.626 INFO:teuthology.orchestra.run.smithi167.stdout: "bloom_filter_bytes": 0, 2024-08-30T09:16:23.626 INFO:teuthology.orchestra.run.smithi167.stdout: "bloom_filter_items": 0, 2024-08-30T09:16:23.626 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_alloc_bytes": 0, 2024-08-30T09:16:23.626 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_alloc_items": 0, 2024-08-30T09:16:23.626 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_data_bytes": 0, 2024-08-30T09:16:23.626 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_data_items": 0, 2024-08-30T09:16:23.626 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_onode_bytes": 0, 2024-08-30T09:16:23.626 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_onode_items": 0, 2024-08-30T09:16:23.626 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_meta_bytes": 0, 2024-08-30T09:16:23.626 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_meta_items": 0, 2024-08-30T09:16:23.626 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_other_bytes": 0, 2024-08-30T09:16:23.626 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_other_items": 0, 2024-08-30T09:16:23.627 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_buffer_bytes": 0, 2024-08-30T09:16:23.627 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_buffer_items": 0, 2024-08-30T09:16:23.627 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_extent_bytes": 0, 2024-08-30T09:16:23.627 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_extent_items": 0, 2024-08-30T09:16:23.627 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_blob_bytes": 0, 2024-08-30T09:16:23.627 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_blob_items": 0, 2024-08-30T09:16:23.627 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_shared_blob_bytes": 0, 2024-08-30T09:16:23.627 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_shared_blob_items": 0, 2024-08-30T09:16:23.627 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_inline_bl_bytes": 0, 2024-08-30T09:16:23.627 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_inline_bl_items": 0, 2024-08-30T09:16:23.627 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_fsck_bytes": 0, 2024-08-30T09:16:23.627 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_fsck_items": 0, 2024-08-30T09:16:23.627 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_txc_bytes": 0, 2024-08-30T09:16:23.627 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_txc_items": 0, 2024-08-30T09:16:23.628 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_writing_deferred_bytes": 0, 2024-08-30T09:16:23.628 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_writing_deferred_items": 0, 2024-08-30T09:16:23.628 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_writing_bytes": 0, 2024-08-30T09:16:23.628 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_writing_items": 0, 2024-08-30T09:16:23.628 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_bytes": 0, 2024-08-30T09:16:23.628 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_items": 0, 2024-08-30T09:16:23.628 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_file_reader_bytes": 0, 2024-08-30T09:16:23.628 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_file_reader_items": 0, 2024-08-30T09:16:23.628 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_file_writer_bytes": 0, 2024-08-30T09:16:23.628 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_file_writer_items": 0, 2024-08-30T09:16:23.628 INFO:teuthology.orchestra.run.smithi167.stdout: "buffer_anon_bytes": 301063, 2024-08-30T09:16:23.628 INFO:teuthology.orchestra.run.smithi167.stdout: "buffer_anon_items": 572, 2024-08-30T09:16:23.628 INFO:teuthology.orchestra.run.smithi167.stdout: "buffer_meta_bytes": 0, 2024-08-30T09:16:23.628 INFO:teuthology.orchestra.run.smithi167.stdout: "buffer_meta_items": 0, 2024-08-30T09:16:23.629 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_bytes": 0, 2024-08-30T09:16:23.629 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_items": 0, 2024-08-30T09:16:23.629 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_mapbl_bytes": 0, 2024-08-30T09:16:23.629 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_mapbl_items": 0, 2024-08-30T09:16:23.629 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_pglog_bytes": 0, 2024-08-30T09:16:23.629 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_pglog_items": 0, 2024-08-30T09:16:23.629 INFO:teuthology.orchestra.run.smithi167.stdout: "osdmap_bytes": 8856, 2024-08-30T09:16:23.629 INFO:teuthology.orchestra.run.smithi167.stdout: "osdmap_items": 166, 2024-08-30T09:16:23.629 INFO:teuthology.orchestra.run.smithi167.stdout: "osdmap_mapping_bytes": 0, 2024-08-30T09:16:23.629 INFO:teuthology.orchestra.run.smithi167.stdout: "osdmap_mapping_items": 0, 2024-08-30T09:16:23.629 INFO:teuthology.orchestra.run.smithi167.stdout: "pgmap_bytes": 0, 2024-08-30T09:16:23.629 INFO:teuthology.orchestra.run.smithi167.stdout: "pgmap_items": 0, 2024-08-30T09:16:23.629 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_co_bytes": 1680481, 2024-08-30T09:16:23.630 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_co_items": 19450, 2024-08-30T09:16:23.630 INFO:teuthology.orchestra.run.smithi167.stdout: "unittest_1_bytes": 0, 2024-08-30T09:16:23.630 INFO:teuthology.orchestra.run.smithi167.stdout: "unittest_1_items": 0, 2024-08-30T09:16:23.630 INFO:teuthology.orchestra.run.smithi167.stdout: "unittest_2_bytes": 0, 2024-08-30T09:16:23.630 INFO:teuthology.orchestra.run.smithi167.stdout: "unittest_2_items": 0 2024-08-30T09:16:23.630 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-30T09:16:23.630 INFO:teuthology.orchestra.run.smithi167.stdout: "objecter": { 2024-08-30T09:16:23.630 INFO:teuthology.orchestra.run.smithi167.stdout: "op_active": 0, 2024-08-30T09:16:23.630 INFO:teuthology.orchestra.run.smithi167.stdout: "op_laggy": 0, 2024-08-30T09:16:23.630 INFO:teuthology.orchestra.run.smithi167.stdout: "op_send": 9457, 2024-08-30T09:16:23.630 INFO:teuthology.orchestra.run.smithi167.stdout: "op_send_bytes": 2955717, 2024-08-30T09:16:23.630 INFO:teuthology.orchestra.run.smithi167.stdout: "op_resend": 33, 2024-08-30T09:16:23.630 INFO:teuthology.orchestra.run.smithi167.stdout: "op_reply": 9457, 2024-08-30T09:16:23.631 INFO:teuthology.orchestra.run.smithi167.stdout: "op_latency": { 2024-08-30T09:16:23.631 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 9457, 2024-08-30T09:16:23.631 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 121.646913228, 2024-08-30T09:16:23.631 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.012863160 2024-08-30T09:16:23.631 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-30T09:16:23.631 INFO:teuthology.orchestra.run.smithi167.stdout: "op_inflight": 0, 2024-08-30T09:16:23.631 INFO:teuthology.orchestra.run.smithi167.stdout: "oplen_avg": { 2024-08-30T09:16:23.631 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 9457, 2024-08-30T09:16:23.631 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 9635 2024-08-30T09:16:23.631 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-30T09:16:23.631 INFO:teuthology.orchestra.run.smithi167.stdout: "op": 9457, 2024-08-30T09:16:23.631 INFO:teuthology.orchestra.run.smithi167.stdout: "op_r": 7768, 2024-08-30T09:16:23.631 INFO:teuthology.orchestra.run.smithi167.stdout: "op_w": 1689, 2024-08-30T09:16:23.631 INFO:teuthology.orchestra.run.smithi167.stdout: "op_rmw": 0, 2024-08-30T09:16:23.632 INFO:teuthology.orchestra.run.smithi167.stdout: "op_pg": 0, 2024-08-30T09:16:23.632 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_stat": 0, 2024-08-30T09:16:23.632 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_create": 10, 2024-08-30T09:16:23.632 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_read": 90, 2024-08-30T09:16:23.632 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_write": 1163, 2024-08-30T09:16:23.632 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_writefull": 123, 2024-08-30T09:16:23.632 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_writesame": 0, 2024-08-30T09:16:23.632 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_append": 0, 2024-08-30T09:16:23.632 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_zero": 0, 2024-08-30T09:16:23.632 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_truncate": 0, 2024-08-30T09:16:23.632 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_delete": 252, 2024-08-30T09:16:23.632 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_mapext": 0, 2024-08-30T09:16:23.632 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_sparse_read": 0, 2024-08-30T09:16:23.633 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_clonerange": 0, 2024-08-30T09:16:23.633 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_getxattr": 7677, 2024-08-30T09:16:23.633 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_setxattr": 20, 2024-08-30T09:16:23.633 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_cmpxattr": 0, 2024-08-30T09:16:23.633 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_rmxattr": 0, 2024-08-30T09:16:23.633 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_resetxattrs": 0, 2024-08-30T09:16:23.633 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_call": 0, 2024-08-30T09:16:23.633 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_watch": 0, 2024-08-30T09:16:23.633 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_notify": 0, 2024-08-30T09:16:23.633 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_src_cmpxattr": 0, 2024-08-30T09:16:23.633 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_pgls": 0, 2024-08-30T09:16:23.633 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_pgls_filter": 0, 2024-08-30T09:16:23.633 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_other": 1, 2024-08-30T09:16:23.633 INFO:teuthology.orchestra.run.smithi167.stdout: "linger_active": 0, 2024-08-30T09:16:23.634 INFO:teuthology.orchestra.run.smithi167.stdout: "linger_send": 0, 2024-08-30T09:16:23.634 INFO:teuthology.orchestra.run.smithi167.stdout: "linger_resend": 0, 2024-08-30T09:16:23.634 INFO:teuthology.orchestra.run.smithi167.stdout: "linger_ping": 0, 2024-08-30T09:16:23.634 INFO:teuthology.orchestra.run.smithi167.stdout: "poolop_active": 0, 2024-08-30T09:16:23.634 INFO:teuthology.orchestra.run.smithi167.stdout: "poolop_send": 0, 2024-08-30T09:16:23.634 INFO:teuthology.orchestra.run.smithi167.stdout: "poolop_resend": 0, 2024-08-30T09:16:23.634 INFO:teuthology.orchestra.run.smithi167.stdout: "poolstat_active": 0, 2024-08-30T09:16:23.634 INFO:teuthology.orchestra.run.smithi167.stdout: "poolstat_send": 0, 2024-08-30T09:16:23.634 INFO:teuthology.orchestra.run.smithi167.stdout: "poolstat_resend": 0, 2024-08-30T09:16:23.634 INFO:teuthology.orchestra.run.smithi167.stdout: "statfs_active": 0, 2024-08-30T09:16:23.634 INFO:teuthology.orchestra.run.smithi167.stdout: "statfs_send": 0, 2024-08-30T09:16:23.634 INFO:teuthology.orchestra.run.smithi167.stdout: "statfs_resend": 0, 2024-08-30T09:16:23.634 INFO:teuthology.orchestra.run.smithi167.stdout: "command_active": 0, 2024-08-30T09:16:23.634 INFO:teuthology.orchestra.run.smithi167.stdout: "command_send": 0, 2024-08-30T09:16:23.635 INFO:teuthology.orchestra.run.smithi167.stdout: "command_resend": 0, 2024-08-30T09:16:23.635 INFO:teuthology.orchestra.run.smithi167.stdout: "map_epoch": 83, 2024-08-30T09:16:23.635 INFO:teuthology.orchestra.run.smithi167.stdout: "map_full": 0, 2024-08-30T09:16:23.635 INFO:teuthology.orchestra.run.smithi167.stdout: "map_inc": 3, 2024-08-30T09:16:23.635 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_sessions": 12, 2024-08-30T09:16:23.635 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_session_open": 12, 2024-08-30T09:16:23.635 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_session_close": 0, 2024-08-30T09:16:23.635 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_laggy": 0, 2024-08-30T09:16:23.635 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_wr": 220, 2024-08-30T09:16:23.635 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_rd": 4, 2024-08-30T09:16:23.635 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_del": 75 2024-08-30T09:16:23.635 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-30T09:16:23.635 INFO:teuthology.orchestra.run.smithi167.stdout: "oft": { 2024-08-30T09:16:23.635 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_total_objs": 1, 2024-08-30T09:16:23.636 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_total_kv_pairs": 0, 2024-08-30T09:16:23.636 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_total_updates": 0, 2024-08-30T09:16:23.636 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_total_removes": 0 2024-08-30T09:16:23.636 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-30T09:16:23.636 INFO:teuthology.orchestra.run.smithi167.stdout: "purge_queue": { 2024-08-30T09:16:23.636 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_executing_ops": 0, 2024-08-30T09:16:23.636 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_executing_ops_high_water": 12, 2024-08-30T09:16:23.636 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_executing": 0, 2024-08-30T09:16:23.636 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_executing_high_water": 6, 2024-08-30T09:16:23.636 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_executed_ops": 240, 2024-08-30T09:16:23.636 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_executed": 120, 2024-08-30T09:16:23.636 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_item_in_journal": 0 2024-08-30T09:16:23.636 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-30T09:16:23.636 INFO:teuthology.orchestra.run.smithi167.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-08-30T09:16:23.637 INFO:teuthology.orchestra.run.smithi167.stdout: "val": 0, 2024-08-30T09:16:23.637 INFO:teuthology.orchestra.run.smithi167.stdout: "max": 104857600, 2024-08-30T09:16:23.637 INFO:teuthology.orchestra.run.smithi167.stdout: "get_started": 0, 2024-08-30T09:16:23.637 INFO:teuthology.orchestra.run.smithi167.stdout: "get": 18788, 2024-08-30T09:16:23.637 INFO:teuthology.orchestra.run.smithi167.stdout: "get_sum": 7364618, 2024-08-30T09:16:23.637 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_fail": 0, 2024-08-30T09:16:23.637 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_success": 18788, 2024-08-30T09:16:23.637 INFO:teuthology.orchestra.run.smithi167.stdout: "take": 0, 2024-08-30T09:16:23.637 INFO:teuthology.orchestra.run.smithi167.stdout: "take_sum": 0, 2024-08-30T09:16:23.637 INFO:teuthology.orchestra.run.smithi167.stdout: "put": 18788, 2024-08-30T09:16:23.637 INFO:teuthology.orchestra.run.smithi167.stdout: "put_sum": 7364618, 2024-08-30T09:16:23.637 INFO:teuthology.orchestra.run.smithi167.stdout: "wait": { 2024-08-30T09:16:23.637 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-30T09:16:23.637 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-30T09:16:23.637 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-30T09:16:23.638 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-08-30T09:16:23.638 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-30T09:16:23.638 INFO:teuthology.orchestra.run.smithi167.stdout: "throttle-objecter_bytes": { 2024-08-30T09:16:23.638 INFO:teuthology.orchestra.run.smithi167.stdout: "val": 0, 2024-08-30T09:16:23.638 INFO:teuthology.orchestra.run.smithi167.stdout: "max": 104857600, 2024-08-30T09:16:23.638 INFO:teuthology.orchestra.run.smithi167.stdout: "get_started": 0, 2024-08-30T09:16:23.638 INFO:teuthology.orchestra.run.smithi167.stdout: "get": 0, 2024-08-30T09:16:23.638 INFO:teuthology.orchestra.run.smithi167.stdout: "get_sum": 0, 2024-08-30T09:16:23.638 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_fail": 0, 2024-08-30T09:16:23.638 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_success": 0, 2024-08-30T09:16:23.638 INFO:teuthology.orchestra.run.smithi167.stdout: "take": 9457, 2024-08-30T09:16:23.638 INFO:teuthology.orchestra.run.smithi167.stdout: "take_sum": 2968765, 2024-08-30T09:16:23.638 INFO:teuthology.orchestra.run.smithi167.stdout: "put": 9457, 2024-08-30T09:16:23.639 INFO:teuthology.orchestra.run.smithi167.stdout: "put_sum": 2968765, 2024-08-30T09:16:23.639 INFO:teuthology.orchestra.run.smithi167.stdout: "wait": { 2024-08-30T09:16:23.639 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-30T09:16:23.639 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-30T09:16:23.639 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-30T09:16:23.639 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-08-30T09:16:23.639 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-30T09:16:23.639 INFO:teuthology.orchestra.run.smithi167.stdout: "throttle-objecter_ops": { 2024-08-30T09:16:23.639 INFO:teuthology.orchestra.run.smithi167.stdout: "val": 0, 2024-08-30T09:16:23.639 INFO:teuthology.orchestra.run.smithi167.stdout: "max": 1024, 2024-08-30T09:16:23.639 INFO:teuthology.orchestra.run.smithi167.stdout: "get_started": 0, 2024-08-30T09:16:23.639 INFO:teuthology.orchestra.run.smithi167.stdout: "get": 0, 2024-08-30T09:16:23.639 INFO:teuthology.orchestra.run.smithi167.stdout: "get_sum": 0, 2024-08-30T09:16:23.640 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_fail": 0, 2024-08-30T09:16:23.640 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_success": 0, 2024-08-30T09:16:23.640 INFO:teuthology.orchestra.run.smithi167.stdout: "take": 9457, 2024-08-30T09:16:23.640 INFO:teuthology.orchestra.run.smithi167.stdout: "take_sum": 9457, 2024-08-30T09:16:23.640 INFO:teuthology.orchestra.run.smithi167.stdout: "put": 9457, 2024-08-30T09:16:23.640 INFO:teuthology.orchestra.run.smithi167.stdout: "put_sum": 9457, 2024-08-30T09:16:23.640 INFO:teuthology.orchestra.run.smithi167.stdout: "wait": { 2024-08-30T09:16:23.640 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-30T09:16:23.640 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-30T09:16:23.640 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-30T09:16:23.640 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-08-30T09:16:23.640 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-30T09:16:23.641 INFO:teuthology.orchestra.run.smithi167.stdout: "throttle-write_buf_throttle": { 2024-08-30T09:16:23.641 INFO:teuthology.orchestra.run.smithi167.stdout: "val": 0, 2024-08-30T09:16:23.641 INFO:teuthology.orchestra.run.smithi167.stdout: "max": 3758096384, 2024-08-30T09:16:23.641 INFO:teuthology.orchestra.run.smithi167.stdout: "get_started": 0, 2024-08-30T09:16:23.641 INFO:teuthology.orchestra.run.smithi167.stdout: "get": 120, 2024-08-30T09:16:23.641 INFO:teuthology.orchestra.run.smithi167.stdout: "get_sum": 13080, 2024-08-30T09:16:23.641 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_fail": 0, 2024-08-30T09:16:23.641 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_success": 120, 2024-08-30T09:16:23.641 INFO:teuthology.orchestra.run.smithi167.stdout: "take": 0, 2024-08-30T09:16:23.641 INFO:teuthology.orchestra.run.smithi167.stdout: "take_sum": 0, 2024-08-30T09:16:23.641 INFO:teuthology.orchestra.run.smithi167.stdout: "put": 90, 2024-08-30T09:16:23.641 INFO:teuthology.orchestra.run.smithi167.stdout: "put_sum": 13080, 2024-08-30T09:16:23.641 INFO:teuthology.orchestra.run.smithi167.stdout: "wait": { 2024-08-30T09:16:23.641 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-30T09:16:23.641 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-30T09:16:23.642 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-30T09:16:23.642 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-08-30T09:16:23.642 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-30T09:16:23.642 INFO:teuthology.orchestra.run.smithi167.stdout: "throttle-write_buf_throttle-0x564dd82548f0": { 2024-08-30T09:16:23.642 INFO:teuthology.orchestra.run.smithi167.stdout: "val": 0, 2024-08-30T09:16:23.642 INFO:teuthology.orchestra.run.smithi167.stdout: "max": 3758096384, 2024-08-30T09:16:23.642 INFO:teuthology.orchestra.run.smithi167.stdout: "get_started": 0, 2024-08-30T09:16:23.642 INFO:teuthology.orchestra.run.smithi167.stdout: "get": 2094, 2024-08-30T09:16:23.642 INFO:teuthology.orchestra.run.smithi167.stdout: "get_sum": 2808954, 2024-08-30T09:16:23.642 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_fail": 0, 2024-08-30T09:16:23.642 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_success": 2094, 2024-08-30T09:16:23.642 INFO:teuthology.orchestra.run.smithi167.stdout: "take": 0, 2024-08-30T09:16:23.642 INFO:teuthology.orchestra.run.smithi167.stdout: "take_sum": 0, 2024-08-30T09:16:23.642 INFO:teuthology.orchestra.run.smithi167.stdout: "put": 1073, 2024-08-30T09:16:23.643 INFO:teuthology.orchestra.run.smithi167.stdout: "put_sum": 2808954, 2024-08-30T09:16:23.643 INFO:teuthology.orchestra.run.smithi167.stdout: "wait": { 2024-08-30T09:16:23.643 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-30T09:16:23.643 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-30T09:16:23.643 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-30T09:16:23.643 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-08-30T09:16:23.643 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-30T09:16:23.643 INFO:teuthology.orchestra.run.smithi167.stdout: "trackedop": { 2024-08-30T09:16:23.643 INFO:teuthology.orchestra.run.smithi167.stdout: "slow_ops_count": 0 2024-08-30T09:16:23.643 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-08-30T09:16:23.643 INFO:teuthology.orchestra.run.smithi167.stdout:} 2024-08-30T09:16:24.394 INFO:tasks.check_counter:Daemon mds.h mds.exported=4 2024-08-30T09:16:24.394 INFO:tasks.check_counter:Daemon mds.h mds.imported=16 2024-08-30T09:16:24.394 DEBUG:tasks.check_counter:Getting stats from k 2024-08-30T09:16:24.394 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph daemon mds.k perf dump 2024-08-30T09:16:24.644 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.b/config 2024-08-30T09:16:25.193 INFO:teuthology.orchestra.run.smithi167.stdout:{ 2024-08-30T09:16:25.193 INFO:teuthology.orchestra.run.smithi167.stdout: "AsyncMessenger::Worker-0": { 2024-08-30T09:16:25.193 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_messages": 0, 2024-08-30T09:16:25.193 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages": 2, 2024-08-30T09:16:25.193 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_bytes": 0, 2024-08-30T09:16:25.194 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_bytes": 0, 2024-08-30T09:16:25.194 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_created_connections": 4, 2024-08-30T09:16:25.194 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_active_connections": 0, 2024-08-30T09:16:25.194 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_total_time": 0.001108811, 2024-08-30T09:16:25.194 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_send_time": 0.000000000, 2024-08-30T09:16:25.194 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_recv_time": 0.000256166, 2024-08-30T09:16:25.194 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-08-30T09:16:25.194 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages_queue_lat": { 2024-08-30T09:16:25.194 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-30T09:16:25.194 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-30T09:16:25.194 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-30T09:16:25.194 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-30T09:16:25.194 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_handle_ack_lat": { 2024-08-30T09:16:25.195 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-30T09:16:25.195 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-30T09:16:25.195 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-30T09:16:25.195 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-30T09:16:25.195 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_encrypted_bytes": 0, 2024-08-30T09:16:25.195 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_encrypted_bytes": 0 2024-08-30T09:16:25.195 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-30T09:16:25.195 INFO:teuthology.orchestra.run.smithi167.stdout: "AsyncMessenger::Worker-1": { 2024-08-30T09:16:25.195 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_messages": 312, 2024-08-30T09:16:25.195 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages": 307, 2024-08-30T09:16:25.195 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_bytes": 551008, 2024-08-30T09:16:25.195 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_bytes": 188528, 2024-08-30T09:16:25.196 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_created_connections": 2, 2024-08-30T09:16:25.196 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_active_connections": 1, 2024-08-30T09:16:25.196 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_total_time": 0.048609399, 2024-08-30T09:16:25.196 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_send_time": 0.022856860, 2024-08-30T09:16:25.196 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_recv_time": 0.032734980, 2024-08-30T09:16:25.196 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-08-30T09:16:25.196 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages_queue_lat": { 2024-08-30T09:16:25.196 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 307, 2024-08-30T09:16:25.196 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.025062503, 2024-08-30T09:16:25.196 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000081636 2024-08-30T09:16:25.196 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-30T09:16:25.196 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_handle_ack_lat": { 2024-08-30T09:16:25.196 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-30T09:16:25.197 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-30T09:16:25.197 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-30T09:16:25.197 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-30T09:16:25.197 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_encrypted_bytes": 551008, 2024-08-30T09:16:25.197 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_encrypted_bytes": 188528 2024-08-30T09:16:25.197 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-30T09:16:25.197 INFO:teuthology.orchestra.run.smithi167.stdout: "AsyncMessenger::Worker-2": { 2024-08-30T09:16:25.197 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_messages": 2, 2024-08-30T09:16:25.197 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages": 233, 2024-08-30T09:16:25.197 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_bytes": 320, 2024-08-30T09:16:25.197 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_bytes": 201136, 2024-08-30T09:16:25.197 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_created_connections": 4, 2024-08-30T09:16:25.198 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_active_connections": 2, 2024-08-30T09:16:25.198 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_total_time": 0.016526074, 2024-08-30T09:16:25.198 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_send_time": 0.012839394, 2024-08-30T09:16:25.198 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_recv_time": 0.001442842, 2024-08-30T09:16:25.198 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-08-30T09:16:25.198 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages_queue_lat": { 2024-08-30T09:16:25.198 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 231, 2024-08-30T09:16:25.198 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.022939328, 2024-08-30T09:16:25.198 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000099304 2024-08-30T09:16:25.198 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-30T09:16:25.198 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_handle_ack_lat": { 2024-08-30T09:16:25.198 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-30T09:16:25.198 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-30T09:16:25.199 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-30T09:16:25.199 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-30T09:16:25.199 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_encrypted_bytes": 320, 2024-08-30T09:16:25.199 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_encrypted_bytes": 201136 2024-08-30T09:16:25.199 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-30T09:16:25.199 INFO:teuthology.orchestra.run.smithi167.stdout: "cct": { 2024-08-30T09:16:25.199 INFO:teuthology.orchestra.run.smithi167.stdout: "total_workers": 0, 2024-08-30T09:16:25.199 INFO:teuthology.orchestra.run.smithi167.stdout: "unhealthy_workers": 0 2024-08-30T09:16:25.199 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-30T09:16:25.199 INFO:teuthology.orchestra.run.smithi167.stdout: "mempool": { 2024-08-30T09:16:25.199 INFO:teuthology.orchestra.run.smithi167.stdout: "bloom_filter_bytes": 0, 2024-08-30T09:16:25.199 INFO:teuthology.orchestra.run.smithi167.stdout: "bloom_filter_items": 0, 2024-08-30T09:16:25.199 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_alloc_bytes": 0, 2024-08-30T09:16:25.199 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_alloc_items": 0, 2024-08-30T09:16:25.200 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_data_bytes": 0, 2024-08-30T09:16:25.200 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_data_items": 0, 2024-08-30T09:16:25.200 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_onode_bytes": 0, 2024-08-30T09:16:25.200 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_onode_items": 0, 2024-08-30T09:16:25.200 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_meta_bytes": 0, 2024-08-30T09:16:25.200 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_meta_items": 0, 2024-08-30T09:16:25.200 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_other_bytes": 0, 2024-08-30T09:16:25.200 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_other_items": 0, 2024-08-30T09:16:25.200 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_buffer_bytes": 0, 2024-08-30T09:16:25.200 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_buffer_items": 0, 2024-08-30T09:16:25.200 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_extent_bytes": 0, 2024-08-30T09:16:25.200 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_extent_items": 0, 2024-08-30T09:16:25.201 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_blob_bytes": 0, 2024-08-30T09:16:25.201 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_blob_items": 0, 2024-08-30T09:16:25.201 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_shared_blob_bytes": 0, 2024-08-30T09:16:25.201 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_shared_blob_items": 0, 2024-08-30T09:16:25.201 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_inline_bl_bytes": 0, 2024-08-30T09:16:25.201 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_inline_bl_items": 0, 2024-08-30T09:16:25.201 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_fsck_bytes": 0, 2024-08-30T09:16:25.201 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_fsck_items": 0, 2024-08-30T09:16:25.201 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_txc_bytes": 0, 2024-08-30T09:16:25.201 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_txc_items": 0, 2024-08-30T09:16:25.201 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_writing_deferred_bytes": 0, 2024-08-30T09:16:25.201 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_writing_deferred_items": 0, 2024-08-30T09:16:25.202 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_writing_bytes": 0, 2024-08-30T09:16:25.202 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_writing_items": 0, 2024-08-30T09:16:25.202 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_bytes": 0, 2024-08-30T09:16:25.202 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_items": 0, 2024-08-30T09:16:25.202 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_file_reader_bytes": 0, 2024-08-30T09:16:25.202 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_file_reader_items": 0, 2024-08-30T09:16:25.202 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_file_writer_bytes": 0, 2024-08-30T09:16:25.202 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_file_writer_items": 0, 2024-08-30T09:16:25.202 INFO:teuthology.orchestra.run.smithi167.stdout: "buffer_anon_bytes": 148704, 2024-08-30T09:16:25.202 INFO:teuthology.orchestra.run.smithi167.stdout: "buffer_anon_items": 29, 2024-08-30T09:16:25.202 INFO:teuthology.orchestra.run.smithi167.stdout: "buffer_meta_bytes": 0, 2024-08-30T09:16:25.202 INFO:teuthology.orchestra.run.smithi167.stdout: "buffer_meta_items": 0, 2024-08-30T09:16:25.202 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_bytes": 0, 2024-08-30T09:16:25.202 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_items": 0, 2024-08-30T09:16:25.203 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_mapbl_bytes": 0, 2024-08-30T09:16:25.203 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_mapbl_items": 0, 2024-08-30T09:16:25.203 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_pglog_bytes": 0, 2024-08-30T09:16:25.203 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_pglog_items": 0, 2024-08-30T09:16:25.203 INFO:teuthology.orchestra.run.smithi167.stdout: "osdmap_bytes": 0, 2024-08-30T09:16:25.203 INFO:teuthology.orchestra.run.smithi167.stdout: "osdmap_items": 0, 2024-08-30T09:16:25.203 INFO:teuthology.orchestra.run.smithi167.stdout: "osdmap_mapping_bytes": 0, 2024-08-30T09:16:25.203 INFO:teuthology.orchestra.run.smithi167.stdout: "osdmap_mapping_items": 0, 2024-08-30T09:16:25.203 INFO:teuthology.orchestra.run.smithi167.stdout: "pgmap_bytes": 0, 2024-08-30T09:16:25.203 INFO:teuthology.orchestra.run.smithi167.stdout: "pgmap_items": 0, 2024-08-30T09:16:25.203 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_co_bytes": 928, 2024-08-30T09:16:25.203 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_co_items": 2, 2024-08-30T09:16:25.204 INFO:teuthology.orchestra.run.smithi167.stdout: "unittest_1_bytes": 0, 2024-08-30T09:16:25.204 INFO:teuthology.orchestra.run.smithi167.stdout: "unittest_1_items": 0, 2024-08-30T09:16:25.204 INFO:teuthology.orchestra.run.smithi167.stdout: "unittest_2_bytes": 0, 2024-08-30T09:16:25.204 INFO:teuthology.orchestra.run.smithi167.stdout: "unittest_2_items": 0 2024-08-30T09:16:25.204 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-30T09:16:25.204 INFO:teuthology.orchestra.run.smithi167.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-08-30T09:16:25.204 INFO:teuthology.orchestra.run.smithi167.stdout: "val": 0, 2024-08-30T09:16:25.204 INFO:teuthology.orchestra.run.smithi167.stdout: "max": 104857600, 2024-08-30T09:16:25.204 INFO:teuthology.orchestra.run.smithi167.stdout: "get_started": 0, 2024-08-30T09:16:25.204 INFO:teuthology.orchestra.run.smithi167.stdout: "get": 310, 2024-08-30T09:16:25.204 INFO:teuthology.orchestra.run.smithi167.stdout: "get_sum": 504519, 2024-08-30T09:16:25.204 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_fail": 0, 2024-08-30T09:16:25.204 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_success": 310, 2024-08-30T09:16:25.204 INFO:teuthology.orchestra.run.smithi167.stdout: "take": 0, 2024-08-30T09:16:25.205 INFO:teuthology.orchestra.run.smithi167.stdout: "take_sum": 0, 2024-08-30T09:16:25.205 INFO:teuthology.orchestra.run.smithi167.stdout: "put": 310, 2024-08-30T09:16:25.205 INFO:teuthology.orchestra.run.smithi167.stdout: "put_sum": 504519, 2024-08-30T09:16:25.205 INFO:teuthology.orchestra.run.smithi167.stdout: "wait": { 2024-08-30T09:16:25.205 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-30T09:16:25.205 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-30T09:16:25.205 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-30T09:16:25.205 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-08-30T09:16:25.205 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-08-30T09:16:25.205 INFO:teuthology.orchestra.run.smithi167.stdout:} 2024-08-30T09:16:25.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:16:25 smithi167 ceph-mon[22063]: pgmap v842: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.7 KiB/s wr, 8 op/s 2024-08-30T09:16:25.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:16:25 smithi042 ceph-mon[18652]: pgmap v842: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.7 KiB/s wr, 8 op/s 2024-08-30T09:16:25.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:16:25 smithi190 ceph-mon[22001]: pgmap v842: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.7 KiB/s wr, 8 op/s 2024-08-30T09:16:25.988 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.k 2024-08-30T09:16:25.988 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.k 2024-08-30T09:16:25.988 DEBUG:tasks.check_counter:Getting stats from c 2024-08-30T09:16:25.988 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph daemon mds.c perf dump 2024-08-30T09:16:26.286 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.c/config 2024-08-30T09:16:27.128 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-08-30T09:16:27.128 INFO:teuthology.orchestra.run.smithi190.stdout: "AsyncMessenger::Worker-0": { 2024-08-30T09:16:27.128 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_messages": 1503, 2024-08-30T09:16:27.128 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_messages": 1473, 2024-08-30T09:16:27.128 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_bytes": 5249002, 2024-08-30T09:16:27.128 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_bytes": 557154, 2024-08-30T09:16:27.128 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_created_connections": 5, 2024-08-30T09:16:27.128 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_active_connections": 1, 2024-08-30T09:16:27.128 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_total_time": 0.193110963, 2024-08-30T09:16:27.128 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_send_time": 0.081838787, 2024-08-30T09:16:27.129 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_recv_time": 0.066058256, 2024-08-30T09:16:27.129 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_fast_dispatch_time": 0.040904585, 2024-08-30T09:16:27.129 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_messages_queue_lat": { 2024-08-30T09:16:27.129 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 1473, 2024-08-30T09:16:27.129 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.100085043, 2024-08-30T09:16:27.129 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000067946 2024-08-30T09:16:27.129 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-30T09:16:27.129 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_handle_ack_lat": { 2024-08-30T09:16:27.129 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-30T09:16:27.129 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-30T09:16:27.129 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-30T09:16:27.129 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-30T09:16:27.130 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_encrypted_bytes": 674400, 2024-08-30T09:16:27.130 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_encrypted_bytes": 187952 2024-08-30T09:16:27.130 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-30T09:16:27.130 INFO:teuthology.orchestra.run.smithi190.stdout: "AsyncMessenger::Worker-1": { 2024-08-30T09:16:27.130 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_messages": 1688, 2024-08-30T09:16:27.130 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_messages": 1916, 2024-08-30T09:16:27.130 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_bytes": 4270101, 2024-08-30T09:16:27.130 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_bytes": 1014100, 2024-08-30T09:16:27.130 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_created_connections": 5, 2024-08-30T09:16:27.130 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_active_connections": 4, 2024-08-30T09:16:27.130 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_total_time": 0.208856540, 2024-08-30T09:16:27.130 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_send_time": 0.074088071, 2024-08-30T09:16:27.130 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_recv_time": 0.046282008, 2024-08-30T09:16:27.131 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_fast_dispatch_time": 0.076163618, 2024-08-30T09:16:27.131 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_messages_queue_lat": { 2024-08-30T09:16:27.131 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 1914, 2024-08-30T09:16:27.131 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.114848181, 2024-08-30T09:16:27.131 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000060004 2024-08-30T09:16:27.131 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-30T09:16:27.131 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_handle_ack_lat": { 2024-08-30T09:16:27.131 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-30T09:16:27.131 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-30T09:16:27.131 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-30T09:16:27.131 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-30T09:16:27.131 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_encrypted_bytes": 160, 2024-08-30T09:16:27.131 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_encrypted_bytes": 477568 2024-08-30T09:16:27.131 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-30T09:16:27.132 INFO:teuthology.orchestra.run.smithi190.stdout: "AsyncMessenger::Worker-2": { 2024-08-30T09:16:27.132 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_messages": 607, 2024-08-30T09:16:27.132 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_messages": 609, 2024-08-30T09:16:27.132 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_bytes": 4349476, 2024-08-30T09:16:27.132 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_bytes": 193033, 2024-08-30T09:16:27.132 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_created_connections": 5, 2024-08-30T09:16:27.132 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_active_connections": 3, 2024-08-30T09:16:27.132 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_total_time": 0.076429077, 2024-08-30T09:16:27.132 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_send_time": 0.034416649, 2024-08-30T09:16:27.132 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_recv_time": 0.017620332, 2024-08-30T09:16:27.132 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_fast_dispatch_time": 0.019978474, 2024-08-30T09:16:27.132 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_messages_queue_lat": { 2024-08-30T09:16:27.133 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 607, 2024-08-30T09:16:27.133 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.026669122, 2024-08-30T09:16:27.133 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000043935 2024-08-30T09:16:27.133 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-30T09:16:27.133 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_handle_ack_lat": { 2024-08-30T09:16:27.133 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-30T09:16:27.133 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-30T09:16:27.133 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-30T09:16:27.133 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-30T09:16:27.133 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_encrypted_bytes": 0, 2024-08-30T09:16:27.133 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_encrypted_bytes": 0 2024-08-30T09:16:27.133 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-30T09:16:27.133 INFO:teuthology.orchestra.run.smithi190.stdout: "cct": { 2024-08-30T09:16:27.134 INFO:teuthology.orchestra.run.smithi190.stdout: "total_workers": 1, 2024-08-30T09:16:27.134 INFO:teuthology.orchestra.run.smithi190.stdout: "unhealthy_workers": 0 2024-08-30T09:16:27.134 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-30T09:16:27.134 INFO:teuthology.orchestra.run.smithi190.stdout: "finisher-MDSRank": { 2024-08-30T09:16:27.134 INFO:teuthology.orchestra.run.smithi190.stdout: "queue_len": 0, 2024-08-30T09:16:27.134 INFO:teuthology.orchestra.run.smithi190.stdout: "complete_latency": { 2024-08-30T09:16:27.134 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 5836, 2024-08-30T09:16:27.134 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.190792846, 2024-08-30T09:16:27.134 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000032692 2024-08-30T09:16:27.134 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-08-30T09:16:27.134 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-30T09:16:27.134 INFO:teuthology.orchestra.run.smithi190.stdout: "finisher-PurgeQueue": { 2024-08-30T09:16:27.134 INFO:teuthology.orchestra.run.smithi190.stdout: "queue_len": 0, 2024-08-30T09:16:27.135 INFO:teuthology.orchestra.run.smithi190.stdout: "complete_latency": { 2024-08-30T09:16:27.135 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-30T09:16:27.135 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-30T09:16:27.135 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-30T09:16:27.135 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-08-30T09:16:27.135 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-30T09:16:27.135 INFO:teuthology.orchestra.run.smithi190.stdout: "mds": { 2024-08-30T09:16:27.135 INFO:teuthology.orchestra.run.smithi190.stdout: "request": 0, 2024-08-30T09:16:27.135 INFO:teuthology.orchestra.run.smithi190.stdout: "reply": 0, 2024-08-30T09:16:27.135 INFO:teuthology.orchestra.run.smithi190.stdout: "reply_latency": { 2024-08-30T09:16:27.135 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-30T09:16:27.135 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-30T09:16:27.135 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-30T09:16:27.135 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-30T09:16:27.136 INFO:teuthology.orchestra.run.smithi190.stdout: "slow_reply": 0, 2024-08-30T09:16:27.136 INFO:teuthology.orchestra.run.smithi190.stdout: "forward": 0, 2024-08-30T09:16:27.136 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_fetch_complete": 0, 2024-08-30T09:16:27.136 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_fetch_keys": 0, 2024-08-30T09:16:27.136 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_commit": 0, 2024-08-30T09:16:27.136 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_split": 0, 2024-08-30T09:16:27.136 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_merge": 0, 2024-08-30T09:16:27.136 INFO:teuthology.orchestra.run.smithi190.stdout: "inodes": 405, 2024-08-30T09:16:27.136 INFO:teuthology.orchestra.run.smithi190.stdout: "inodes_top": 116, 2024-08-30T09:16:27.136 INFO:teuthology.orchestra.run.smithi190.stdout: "inodes_bottom": 289, 2024-08-30T09:16:27.136 INFO:teuthology.orchestra.run.smithi190.stdout: "inodes_pin_tail": 0, 2024-08-30T09:16:27.136 INFO:teuthology.orchestra.run.smithi190.stdout: "inodes_pinned": 239, 2024-08-30T09:16:27.136 INFO:teuthology.orchestra.run.smithi190.stdout: "inodes_expired": 0, 2024-08-30T09:16:27.137 INFO:teuthology.orchestra.run.smithi190.stdout: "inodes_with_caps": 0, 2024-08-30T09:16:27.137 INFO:teuthology.orchestra.run.smithi190.stdout: "caps": 0, 2024-08-30T09:16:27.137 INFO:teuthology.orchestra.run.smithi190.stdout: "subtrees": 5, 2024-08-30T09:16:27.137 INFO:teuthology.orchestra.run.smithi190.stdout: "traverse": 0, 2024-08-30T09:16:27.137 INFO:teuthology.orchestra.run.smithi190.stdout: "traverse_hit": 0, 2024-08-30T09:16:27.137 INFO:teuthology.orchestra.run.smithi190.stdout: "traverse_forward": 0, 2024-08-30T09:16:27.137 INFO:teuthology.orchestra.run.smithi190.stdout: "traverse_discover": 0, 2024-08-30T09:16:27.137 INFO:teuthology.orchestra.run.smithi190.stdout: "traverse_dir_fetch": 0, 2024-08-30T09:16:27.137 INFO:teuthology.orchestra.run.smithi190.stdout: "traverse_remote_ino": 0, 2024-08-30T09:16:27.137 INFO:teuthology.orchestra.run.smithi190.stdout: "traverse_lock": 0, 2024-08-30T09:16:27.137 INFO:teuthology.orchestra.run.smithi190.stdout: "load_cent": 0, 2024-08-30T09:16:27.137 INFO:teuthology.orchestra.run.smithi190.stdout: "q": 0, 2024-08-30T09:16:27.137 INFO:teuthology.orchestra.run.smithi190.stdout: "exported": 0, 2024-08-30T09:16:27.138 INFO:teuthology.orchestra.run.smithi190.stdout: "exported_inodes": 0, 2024-08-30T09:16:27.138 INFO:teuthology.orchestra.run.smithi190.stdout: "imported": 0, 2024-08-30T09:16:27.138 INFO:teuthology.orchestra.run.smithi190.stdout: "imported_inodes": 0, 2024-08-30T09:16:27.138 INFO:teuthology.orchestra.run.smithi190.stdout: "openino_dir_fetch": 0, 2024-08-30T09:16:27.138 INFO:teuthology.orchestra.run.smithi190.stdout: "openino_backtrace_fetch": 0, 2024-08-30T09:16:27.138 INFO:teuthology.orchestra.run.smithi190.stdout: "openino_peer_discover": 0, 2024-08-30T09:16:27.138 INFO:teuthology.orchestra.run.smithi190.stdout: "root_rfiles": 1020, 2024-08-30T09:16:27.138 INFO:teuthology.orchestra.run.smithi190.stdout: "root_rbytes": 1572347444, 2024-08-30T09:16:27.138 INFO:teuthology.orchestra.run.smithi190.stdout: "root_rsnaps": 0, 2024-08-30T09:16:27.138 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_backtrace_fetch": 0, 2024-08-30T09:16:27.138 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_set_tag": 0, 2024-08-30T09:16:27.138 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_backtrace_repaired": 0, 2024-08-30T09:16:27.138 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_inotable_repaired": 0, 2024-08-30T09:16:27.138 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_dir_inodes": 0, 2024-08-30T09:16:27.139 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_dir_base_inodes": 0, 2024-08-30T09:16:27.139 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_dirfrag_rstats": 0, 2024-08-30T09:16:27.139 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_file_inodes": 0, 2024-08-30T09:16:27.139 INFO:teuthology.orchestra.run.smithi190.stdout: "handle_inode_file_caps": 0, 2024-08-30T09:16:27.139 INFO:teuthology.orchestra.run.smithi190.stdout: "ceph_cap_op_revoke": 0, 2024-08-30T09:16:27.139 INFO:teuthology.orchestra.run.smithi190.stdout: "ceph_cap_op_grant": 0, 2024-08-30T09:16:27.139 INFO:teuthology.orchestra.run.smithi190.stdout: "ceph_cap_op_trunc": 0, 2024-08-30T09:16:27.139 INFO:teuthology.orchestra.run.smithi190.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-08-30T09:16:27.139 INFO:teuthology.orchestra.run.smithi190.stdout: "ceph_cap_op_flush_ack": 0, 2024-08-30T09:16:27.139 INFO:teuthology.orchestra.run.smithi190.stdout: "handle_client_caps": 0, 2024-08-30T09:16:27.139 INFO:teuthology.orchestra.run.smithi190.stdout: "handle_client_caps_dirty": 0, 2024-08-30T09:16:27.139 INFO:teuthology.orchestra.run.smithi190.stdout: "handle_client_cap_release": 0, 2024-08-30T09:16:27.139 INFO:teuthology.orchestra.run.smithi190.stdout: "process_request_cap_release": 0 2024-08-30T09:16:27.140 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-30T09:16:27.140 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_cache": { 2024-08-30T09:16:27.140 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_update": 0, 2024-08-30T09:16:27.140 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_update_receipt": 0, 2024-08-30T09:16:27.140 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_try_discover": 0, 2024-08-30T09:16:27.140 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_send_discover": 0, 2024-08-30T09:16:27.140 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_handle_discover": 0, 2024-08-30T09:16:27.140 INFO:teuthology.orchestra.run.smithi190.stdout: "num_strays": 0, 2024-08-30T09:16:27.140 INFO:teuthology.orchestra.run.smithi190.stdout: "num_strays_delayed": 0, 2024-08-30T09:16:27.140 INFO:teuthology.orchestra.run.smithi190.stdout: "num_strays_enqueuing": 0, 2024-08-30T09:16:27.140 INFO:teuthology.orchestra.run.smithi190.stdout: "strays_created": 0, 2024-08-30T09:16:27.140 INFO:teuthology.orchestra.run.smithi190.stdout: "strays_enqueued": 0, 2024-08-30T09:16:27.140 INFO:teuthology.orchestra.run.smithi190.stdout: "strays_reintegrated": 0, 2024-08-30T09:16:27.141 INFO:teuthology.orchestra.run.smithi190.stdout: "strays_migrated": 0, 2024-08-30T09:16:27.141 INFO:teuthology.orchestra.run.smithi190.stdout: "num_recovering_processing": 0, 2024-08-30T09:16:27.141 INFO:teuthology.orchestra.run.smithi190.stdout: "num_recovering_enqueued": 0, 2024-08-30T09:16:27.141 INFO:teuthology.orchestra.run.smithi190.stdout: "num_recovering_prioritized": 0, 2024-08-30T09:16:27.141 INFO:teuthology.orchestra.run.smithi190.stdout: "recovery_started": 0, 2024-08-30T09:16:27.141 INFO:teuthology.orchestra.run.smithi190.stdout: "recovery_completed": 0, 2024-08-30T09:16:27.141 INFO:teuthology.orchestra.run.smithi190.stdout: "ireq_quiesce_path": 0, 2024-08-30T09:16:27.141 INFO:teuthology.orchestra.run.smithi190.stdout: "ireq_quiesce_inode": 0, 2024-08-30T09:16:27.141 INFO:teuthology.orchestra.run.smithi190.stdout: "ireq_enqueue_scrub": 0, 2024-08-30T09:16:27.141 INFO:teuthology.orchestra.run.smithi190.stdout: "ireq_exportdir": 0, 2024-08-30T09:16:27.141 INFO:teuthology.orchestra.run.smithi190.stdout: "ireq_flush": 0, 2024-08-30T09:16:27.142 INFO:teuthology.orchestra.run.smithi190.stdout: "ireq_fragmentdir": 0, 2024-08-30T09:16:27.142 INFO:teuthology.orchestra.run.smithi190.stdout: "ireq_fragstats": 0, 2024-08-30T09:16:27.142 INFO:teuthology.orchestra.run.smithi190.stdout: "ireq_inodestats": 0, 2024-08-30T09:16:27.142 INFO:teuthology.orchestra.run.smithi190.stdout: "uninline_started": 0, 2024-08-30T09:16:27.142 INFO:teuthology.orchestra.run.smithi190.stdout: "uninline_succeeded": 0, 2024-08-30T09:16:27.142 INFO:teuthology.orchestra.run.smithi190.stdout: "uninline_write_failed": 0 2024-08-30T09:16:27.143 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-30T09:16:27.144 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_log": { 2024-08-30T09:16:27.144 INFO:teuthology.orchestra.run.smithi190.stdout: "evlrg": 0, 2024-08-30T09:16:27.144 INFO:teuthology.orchestra.run.smithi190.stdout: "evadd": 0, 2024-08-30T09:16:27.144 INFO:teuthology.orchestra.run.smithi190.stdout: "evex": 0, 2024-08-30T09:16:27.144 INFO:teuthology.orchestra.run.smithi190.stdout: "evtrm": 0, 2024-08-30T09:16:27.144 INFO:teuthology.orchestra.run.smithi190.stdout: "ev": 4282, 2024-08-30T09:16:27.144 INFO:teuthology.orchestra.run.smithi190.stdout: "evexg": 0, 2024-08-30T09:16:27.144 INFO:teuthology.orchestra.run.smithi190.stdout: "evexd": 0, 2024-08-30T09:16:27.144 INFO:teuthology.orchestra.run.smithi190.stdout: "segadd": 0, 2024-08-30T09:16:27.144 INFO:teuthology.orchestra.run.smithi190.stdout: "segex": 0, 2024-08-30T09:16:27.144 INFO:teuthology.orchestra.run.smithi190.stdout: "segtrm": 0, 2024-08-30T09:16:27.144 INFO:teuthology.orchestra.run.smithi190.stdout: "seg": 2, 2024-08-30T09:16:27.145 INFO:teuthology.orchestra.run.smithi190.stdout: "segmjr": 71, 2024-08-30T09:16:27.145 INFO:teuthology.orchestra.run.smithi190.stdout: "segexg": 0, 2024-08-30T09:16:27.145 INFO:teuthology.orchestra.run.smithi190.stdout: "segexd": 0, 2024-08-30T09:16:27.145 INFO:teuthology.orchestra.run.smithi190.stdout: "expos": 16269097, 2024-08-30T09:16:27.145 INFO:teuthology.orchestra.run.smithi190.stdout: "wrpos": 16428219, 2024-08-30T09:16:27.145 INFO:teuthology.orchestra.run.smithi190.stdout: "rdpos": 16426754, 2024-08-30T09:16:27.145 INFO:teuthology.orchestra.run.smithi190.stdout: "jlat": { 2024-08-30T09:16:27.145 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-30T09:16:27.145 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-30T09:16:27.145 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-30T09:16:27.145 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-30T09:16:27.145 INFO:teuthology.orchestra.run.smithi190.stdout: "replayed": 4282 2024-08-30T09:16:27.145 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-30T09:16:27.146 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_mem": { 2024-08-30T09:16:27.146 INFO:teuthology.orchestra.run.smithi190.stdout: "ino": 120, 2024-08-30T09:16:27.146 INFO:teuthology.orchestra.run.smithi190.stdout: "ino+": 1894, 2024-08-30T09:16:27.146 INFO:teuthology.orchestra.run.smithi190.stdout: "ino-": 1774, 2024-08-30T09:16:27.146 INFO:teuthology.orchestra.run.smithi190.stdout: "dir": 41, 2024-08-30T09:16:27.146 INFO:teuthology.orchestra.run.smithi190.stdout: "dir+": 101, 2024-08-30T09:16:27.146 INFO:teuthology.orchestra.run.smithi190.stdout: "dir-": 60, 2024-08-30T09:16:27.146 INFO:teuthology.orchestra.run.smithi190.stdout: "dn": 405, 2024-08-30T09:16:27.146 INFO:teuthology.orchestra.run.smithi190.stdout: "dn+": 2116, 2024-08-30T09:16:27.146 INFO:teuthology.orchestra.run.smithi190.stdout: "dn-": 1711, 2024-08-30T09:16:27.146 INFO:teuthology.orchestra.run.smithi190.stdout: "cap": 0, 2024-08-30T09:16:27.146 INFO:teuthology.orchestra.run.smithi190.stdout: "cap+": 0, 2024-08-30T09:16:27.146 INFO:teuthology.orchestra.run.smithi190.stdout: "cap-": 0, 2024-08-30T09:16:27.147 INFO:teuthology.orchestra.run.smithi190.stdout: "rss": 120764, 2024-08-30T09:16:27.147 INFO:teuthology.orchestra.run.smithi190.stdout: "heap": 323014656 2024-08-30T09:16:27.147 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-30T09:16:27.147 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_server": { 2024-08-30T09:16:27.147 INFO:teuthology.orchestra.run.smithi190.stdout: "dispatch_client_request": 0, 2024-08-30T09:16:27.147 INFO:teuthology.orchestra.run.smithi190.stdout: "dispatch_server_request": 0, 2024-08-30T09:16:27.147 INFO:teuthology.orchestra.run.smithi190.stdout: "handle_client_request": 0, 2024-08-30T09:16:27.147 INFO:teuthology.orchestra.run.smithi190.stdout: "handle_client_session": 0, 2024-08-30T09:16:27.147 INFO:teuthology.orchestra.run.smithi190.stdout: "handle_peer_request": 0, 2024-08-30T09:16:27.147 INFO:teuthology.orchestra.run.smithi190.stdout: "req_create_latency": { 2024-08-30T09:16:27.147 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-30T09:16:27.147 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-30T09:16:27.147 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-30T09:16:27.147 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-30T09:16:27.148 INFO:teuthology.orchestra.run.smithi190.stdout: "req_getattr_latency": { 2024-08-30T09:16:27.148 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-30T09:16:27.148 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-30T09:16:27.148 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-30T09:16:27.148 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-30T09:16:27.148 INFO:teuthology.orchestra.run.smithi190.stdout: "req_getfilelock_latency": { 2024-08-30T09:16:27.148 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-30T09:16:27.148 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-30T09:16:27.148 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-30T09:16:27.148 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-30T09:16:27.148 INFO:teuthology.orchestra.run.smithi190.stdout: "req_link_latency": { 2024-08-30T09:16:27.148 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-30T09:16:27.148 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-30T09:16:27.149 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-30T09:16:27.149 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-30T09:16:27.149 INFO:teuthology.orchestra.run.smithi190.stdout: "req_lookup_latency": { 2024-08-30T09:16:27.149 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-30T09:16:27.149 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-30T09:16:27.149 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-30T09:16:27.149 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-30T09:16:27.149 INFO:teuthology.orchestra.run.smithi190.stdout: "req_lookuphash_latency": { 2024-08-30T09:16:27.149 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-30T09:16:27.149 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-30T09:16:27.149 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-30T09:16:27.149 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-30T09:16:27.149 INFO:teuthology.orchestra.run.smithi190.stdout: "req_lookupino_latency": { 2024-08-30T09:16:27.150 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-30T09:16:27.150 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-30T09:16:27.150 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-30T09:16:27.150 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-30T09:16:27.150 INFO:teuthology.orchestra.run.smithi190.stdout: "req_lookupname_latency": { 2024-08-30T09:16:27.150 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-30T09:16:27.150 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-30T09:16:27.150 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-30T09:16:27.150 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-30T09:16:27.150 INFO:teuthology.orchestra.run.smithi190.stdout: "req_lookupparent_latency": { 2024-08-30T09:16:27.150 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-30T09:16:27.150 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-30T09:16:27.150 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-30T09:16:27.151 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-30T09:16:27.151 INFO:teuthology.orchestra.run.smithi190.stdout: "req_lookupsnap_latency": { 2024-08-30T09:16:27.151 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-30T09:16:27.151 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-30T09:16:27.151 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-30T09:16:27.151 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-30T09:16:27.151 INFO:teuthology.orchestra.run.smithi190.stdout: "req_lssnap_latency": { 2024-08-30T09:16:27.152 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-30T09:16:27.152 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-30T09:16:27.152 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-30T09:16:27.152 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-30T09:16:27.152 INFO:teuthology.orchestra.run.smithi190.stdout: "req_mkdir_latency": { 2024-08-30T09:16:27.152 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-30T09:16:27.152 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-30T09:16:27.152 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-30T09:16:27.152 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-30T09:16:27.152 INFO:teuthology.orchestra.run.smithi190.stdout: "req_mknod_latency": { 2024-08-30T09:16:27.152 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-30T09:16:27.152 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-30T09:16:27.153 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-30T09:16:27.153 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-30T09:16:27.153 INFO:teuthology.orchestra.run.smithi190.stdout: "req_mksnap_latency": { 2024-08-30T09:16:27.153 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-30T09:16:27.153 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-30T09:16:27.153 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-30T09:16:27.153 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-30T09:16:27.153 INFO:teuthology.orchestra.run.smithi190.stdout: "req_open_latency": { 2024-08-30T09:16:27.153 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-30T09:16:27.153 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-30T09:16:27.153 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-30T09:16:27.153 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-30T09:16:27.153 INFO:teuthology.orchestra.run.smithi190.stdout: "req_readdir_latency": { 2024-08-30T09:16:27.154 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-30T09:16:27.154 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-30T09:16:27.154 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-30T09:16:27.154 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-30T09:16:27.154 INFO:teuthology.orchestra.run.smithi190.stdout: "req_rename_latency": { 2024-08-30T09:16:27.154 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-30T09:16:27.154 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-30T09:16:27.154 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-30T09:16:27.154 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-30T09:16:27.154 INFO:teuthology.orchestra.run.smithi190.stdout: "req_renamesnap_latency": { 2024-08-30T09:16:27.154 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-30T09:16:27.154 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-30T09:16:27.154 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-30T09:16:27.155 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-30T09:16:27.155 INFO:teuthology.orchestra.run.smithi190.stdout: "req_snapdiff_latency": { 2024-08-30T09:16:27.155 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-30T09:16:27.155 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-30T09:16:27.155 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-30T09:16:27.155 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-30T09:16:27.155 INFO:teuthology.orchestra.run.smithi190.stdout: "req_rmdir_latency": { 2024-08-30T09:16:27.155 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-30T09:16:27.155 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-30T09:16:27.155 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-30T09:16:27.155 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-30T09:16:27.155 INFO:teuthology.orchestra.run.smithi190.stdout: "req_rmsnap_latency": { 2024-08-30T09:16:27.155 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-30T09:16:27.156 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-30T09:16:27.156 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-30T09:16:27.156 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-30T09:16:27.156 INFO:teuthology.orchestra.run.smithi190.stdout: "req_rmxattr_latency": { 2024-08-30T09:16:27.156 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-30T09:16:27.156 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-30T09:16:27.156 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-30T09:16:27.156 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-30T09:16:27.156 INFO:teuthology.orchestra.run.smithi190.stdout: "req_setattr_latency": { 2024-08-30T09:16:27.156 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-30T09:16:27.156 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-30T09:16:27.156 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-30T09:16:27.156 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-30T09:16:27.156 INFO:teuthology.orchestra.run.smithi190.stdout: "req_setdirlayout_latency": { 2024-08-30T09:16:27.157 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-30T09:16:27.157 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-30T09:16:27.157 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-30T09:16:27.157 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-30T09:16:27.157 INFO:teuthology.orchestra.run.smithi190.stdout: "req_setfilelock_latency": { 2024-08-30T09:16:27.157 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-30T09:16:27.157 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-30T09:16:27.157 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-30T09:16:27.157 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-30T09:16:27.157 INFO:teuthology.orchestra.run.smithi190.stdout: "req_setlayout_latency": { 2024-08-30T09:16:27.157 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-30T09:16:27.157 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-30T09:16:27.157 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-30T09:16:27.158 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-30T09:16:27.158 INFO:teuthology.orchestra.run.smithi190.stdout: "req_setxattr_latency": { 2024-08-30T09:16:27.158 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-30T09:16:27.158 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-30T09:16:27.158 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-30T09:16:27.158 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-30T09:16:27.158 INFO:teuthology.orchestra.run.smithi190.stdout: "req_symlink_latency": { 2024-08-30T09:16:27.158 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-30T09:16:27.158 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-30T09:16:27.158 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-30T09:16:27.158 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-30T09:16:27.158 INFO:teuthology.orchestra.run.smithi190.stdout: "req_unlink_latency": { 2024-08-30T09:16:27.158 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-30T09:16:27.159 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-30T09:16:27.159 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-30T09:16:27.159 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-30T09:16:27.159 INFO:teuthology.orchestra.run.smithi190.stdout: "cap_revoke_eviction": 0, 2024-08-30T09:16:27.159 INFO:teuthology.orchestra.run.smithi190.stdout: "cap_acquisition_throttle": 0, 2024-08-30T09:16:27.159 INFO:teuthology.orchestra.run.smithi190.stdout: "req_getvxattr_latency": { 2024-08-30T09:16:27.159 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-30T09:16:27.159 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-30T09:16:27.159 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-30T09:16:27.159 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-08-30T09:16:27.159 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-30T09:16:27.159 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_sessions": { 2024-08-30T09:16:27.159 INFO:teuthology.orchestra.run.smithi190.stdout: "session_count": 1, 2024-08-30T09:16:27.160 INFO:teuthology.orchestra.run.smithi190.stdout: "session_add": 3, 2024-08-30T09:16:27.160 INFO:teuthology.orchestra.run.smithi190.stdout: "session_remove": 2, 2024-08-30T09:16:27.160 INFO:teuthology.orchestra.run.smithi190.stdout: "sessions_open": 1, 2024-08-30T09:16:27.160 INFO:teuthology.orchestra.run.smithi190.stdout: "sessions_stale": 0, 2024-08-30T09:16:27.160 INFO:teuthology.orchestra.run.smithi190.stdout: "total_load": 0, 2024-08-30T09:16:27.160 INFO:teuthology.orchestra.run.smithi190.stdout: "average_load": 0, 2024-08-30T09:16:27.160 INFO:teuthology.orchestra.run.smithi190.stdout: "avg_session_uptime": 1995, 2024-08-30T09:16:27.160 INFO:teuthology.orchestra.run.smithi190.stdout: "mdthresh_evicted": 0 2024-08-30T09:16:27.160 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-30T09:16:27.160 INFO:teuthology.orchestra.run.smithi190.stdout: "mempool": { 2024-08-30T09:16:27.160 INFO:teuthology.orchestra.run.smithi190.stdout: "bloom_filter_bytes": 0, 2024-08-30T09:16:27.160 INFO:teuthology.orchestra.run.smithi190.stdout: "bloom_filter_items": 0, 2024-08-30T09:16:27.160 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_alloc_bytes": 0, 2024-08-30T09:16:27.160 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_alloc_items": 0, 2024-08-30T09:16:27.161 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_data_bytes": 0, 2024-08-30T09:16:27.161 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_data_items": 0, 2024-08-30T09:16:27.161 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_onode_bytes": 0, 2024-08-30T09:16:27.161 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_onode_items": 0, 2024-08-30T09:16:27.161 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_meta_bytes": 0, 2024-08-30T09:16:27.161 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_meta_items": 0, 2024-08-30T09:16:27.161 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_other_bytes": 0, 2024-08-30T09:16:27.161 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_other_items": 0, 2024-08-30T09:16:27.161 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_buffer_bytes": 0, 2024-08-30T09:16:27.161 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_buffer_items": 0, 2024-08-30T09:16:27.161 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_extent_bytes": 0, 2024-08-30T09:16:27.161 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_extent_items": 0, 2024-08-30T09:16:27.161 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_blob_bytes": 0, 2024-08-30T09:16:27.162 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_blob_items": 0, 2024-08-30T09:16:27.162 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_shared_blob_bytes": 0, 2024-08-30T09:16:27.162 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_shared_blob_items": 0, 2024-08-30T09:16:27.162 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_inline_bl_bytes": 0, 2024-08-30T09:16:27.162 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_inline_bl_items": 0, 2024-08-30T09:16:27.162 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_fsck_bytes": 0, 2024-08-30T09:16:27.162 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_fsck_items": 0, 2024-08-30T09:16:27.162 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_txc_bytes": 0, 2024-08-30T09:16:27.162 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_txc_items": 0, 2024-08-30T09:16:27.162 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_writing_deferred_bytes": 0, 2024-08-30T09:16:27.162 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_writing_deferred_items": 0, 2024-08-30T09:16:27.162 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_writing_bytes": 0, 2024-08-30T09:16:27.162 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_writing_items": 0, 2024-08-30T09:16:27.162 INFO:teuthology.orchestra.run.smithi190.stdout: "bluefs_bytes": 0, 2024-08-30T09:16:27.162 INFO:teuthology.orchestra.run.smithi190.stdout: "bluefs_items": 0, 2024-08-30T09:16:27.163 INFO:teuthology.orchestra.run.smithi190.stdout: "bluefs_file_reader_bytes": 0, 2024-08-30T09:16:27.163 INFO:teuthology.orchestra.run.smithi190.stdout: "bluefs_file_reader_items": 0, 2024-08-30T09:16:27.163 INFO:teuthology.orchestra.run.smithi190.stdout: "bluefs_file_writer_bytes": 0, 2024-08-30T09:16:27.163 INFO:teuthology.orchestra.run.smithi190.stdout: "bluefs_file_writer_items": 0, 2024-08-30T09:16:27.163 INFO:teuthology.orchestra.run.smithi190.stdout: "buffer_anon_bytes": 193864, 2024-08-30T09:16:27.163 INFO:teuthology.orchestra.run.smithi190.stdout: "buffer_anon_items": 153, 2024-08-30T09:16:27.163 INFO:teuthology.orchestra.run.smithi190.stdout: "buffer_meta_bytes": 0, 2024-08-30T09:16:27.163 INFO:teuthology.orchestra.run.smithi190.stdout: "buffer_meta_items": 0, 2024-08-30T09:16:27.163 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_bytes": 0, 2024-08-30T09:16:27.163 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_items": 0, 2024-08-30T09:16:27.163 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_mapbl_bytes": 0, 2024-08-30T09:16:27.163 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_mapbl_items": 0, 2024-08-30T09:16:27.163 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_pglog_bytes": 0, 2024-08-30T09:16:27.163 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_pglog_items": 0, 2024-08-30T09:16:27.163 INFO:teuthology.orchestra.run.smithi190.stdout: "osdmap_bytes": 8856, 2024-08-30T09:16:27.163 INFO:teuthology.orchestra.run.smithi190.stdout: "osdmap_items": 166, 2024-08-30T09:16:27.164 INFO:teuthology.orchestra.run.smithi190.stdout: "osdmap_mapping_bytes": 0, 2024-08-30T09:16:27.164 INFO:teuthology.orchestra.run.smithi190.stdout: "osdmap_mapping_items": 0, 2024-08-30T09:16:27.164 INFO:teuthology.orchestra.run.smithi190.stdout: "pgmap_bytes": 0, 2024-08-30T09:16:27.164 INFO:teuthology.orchestra.run.smithi190.stdout: "pgmap_items": 0, 2024-08-30T09:16:27.164 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_co_bytes": 618299, 2024-08-30T09:16:27.164 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_co_items": 14260, 2024-08-30T09:16:27.164 INFO:teuthology.orchestra.run.smithi190.stdout: "unittest_1_bytes": 0, 2024-08-30T09:16:27.164 INFO:teuthology.orchestra.run.smithi190.stdout: "unittest_1_items": 0, 2024-08-30T09:16:27.164 INFO:teuthology.orchestra.run.smithi190.stdout: "unittest_2_bytes": 0, 2024-08-30T09:16:27.164 INFO:teuthology.orchestra.run.smithi190.stdout: "unittest_2_items": 0 2024-08-30T09:16:27.164 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-30T09:16:27.164 INFO:teuthology.orchestra.run.smithi190.stdout: "objecter": { 2024-08-30T09:16:27.164 INFO:teuthology.orchestra.run.smithi190.stdout: "op_active": 0, 2024-08-30T09:16:27.164 INFO:teuthology.orchestra.run.smithi190.stdout: "op_laggy": 0, 2024-08-30T09:16:27.165 INFO:teuthology.orchestra.run.smithi190.stdout: "op_send": 3455, 2024-08-30T09:16:27.165 INFO:teuthology.orchestra.run.smithi190.stdout: "op_send_bytes": 26, 2024-08-30T09:16:27.165 INFO:teuthology.orchestra.run.smithi190.stdout: "op_resend": 4, 2024-08-30T09:16:27.165 INFO:teuthology.orchestra.run.smithi190.stdout: "op_reply": 3455, 2024-08-30T09:16:27.165 INFO:teuthology.orchestra.run.smithi190.stdout: "op_latency": { 2024-08-30T09:16:27.165 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 3455, 2024-08-30T09:16:27.165 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 34.374026434, 2024-08-30T09:16:27.165 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.009949066 2024-08-30T09:16:27.165 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-30T09:16:27.165 INFO:teuthology.orchestra.run.smithi190.stdout: "op_inflight": 0, 2024-08-30T09:16:27.165 INFO:teuthology.orchestra.run.smithi190.stdout: "oplen_avg": { 2024-08-30T09:16:27.165 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 3455, 2024-08-30T09:16:27.165 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 3456 2024-08-30T09:16:27.166 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-30T09:16:27.166 INFO:teuthology.orchestra.run.smithi190.stdout: "op": 3455, 2024-08-30T09:16:27.166 INFO:teuthology.orchestra.run.smithi190.stdout: "op_r": 3455, 2024-08-30T09:16:27.166 INFO:teuthology.orchestra.run.smithi190.stdout: "op_w": 0, 2024-08-30T09:16:27.166 INFO:teuthology.orchestra.run.smithi190.stdout: "op_rmw": 0, 2024-08-30T09:16:27.166 INFO:teuthology.orchestra.run.smithi190.stdout: "op_pg": 0, 2024-08-30T09:16:27.166 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_stat": 2050, 2024-08-30T09:16:27.166 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_create": 0, 2024-08-30T09:16:27.166 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_read": 1402, 2024-08-30T09:16:27.166 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_write": 0, 2024-08-30T09:16:27.166 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_writefull": 0, 2024-08-30T09:16:27.166 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_writesame": 0, 2024-08-30T09:16:27.166 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_append": 0, 2024-08-30T09:16:27.166 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_zero": 0, 2024-08-30T09:16:27.167 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_truncate": 0, 2024-08-30T09:16:27.167 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_delete": 0, 2024-08-30T09:16:27.167 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_mapext": 0, 2024-08-30T09:16:27.167 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_sparse_read": 0, 2024-08-30T09:16:27.167 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_clonerange": 0, 2024-08-30T09:16:27.167 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_getxattr": 2, 2024-08-30T09:16:27.167 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_setxattr": 0, 2024-08-30T09:16:27.167 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_cmpxattr": 0, 2024-08-30T09:16:27.167 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_rmxattr": 0, 2024-08-30T09:16:27.167 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_resetxattrs": 0, 2024-08-30T09:16:27.167 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_call": 0, 2024-08-30T09:16:27.167 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_watch": 0, 2024-08-30T09:16:27.167 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_notify": 0, 2024-08-30T09:16:27.167 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_src_cmpxattr": 0, 2024-08-30T09:16:27.167 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_pgls": 0, 2024-08-30T09:16:27.168 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_pgls_filter": 0, 2024-08-30T09:16:27.168 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_other": 0, 2024-08-30T09:16:27.168 INFO:teuthology.orchestra.run.smithi190.stdout: "linger_active": 0, 2024-08-30T09:16:27.168 INFO:teuthology.orchestra.run.smithi190.stdout: "linger_send": 0, 2024-08-30T09:16:27.168 INFO:teuthology.orchestra.run.smithi190.stdout: "linger_resend": 0, 2024-08-30T09:16:27.168 INFO:teuthology.orchestra.run.smithi190.stdout: "linger_ping": 0, 2024-08-30T09:16:27.168 INFO:teuthology.orchestra.run.smithi190.stdout: "poolop_active": 0, 2024-08-30T09:16:27.168 INFO:teuthology.orchestra.run.smithi190.stdout: "poolop_send": 0, 2024-08-30T09:16:27.168 INFO:teuthology.orchestra.run.smithi190.stdout: "poolop_resend": 0, 2024-08-30T09:16:27.168 INFO:teuthology.orchestra.run.smithi190.stdout: "poolstat_active": 0, 2024-08-30T09:16:27.168 INFO:teuthology.orchestra.run.smithi190.stdout: "poolstat_send": 0, 2024-08-30T09:16:27.168 INFO:teuthology.orchestra.run.smithi190.stdout: "poolstat_resend": 0, 2024-08-30T09:16:27.168 INFO:teuthology.orchestra.run.smithi190.stdout: "statfs_active": 0, 2024-08-30T09:16:27.168 INFO:teuthology.orchestra.run.smithi190.stdout: "statfs_send": 0, 2024-08-30T09:16:27.169 INFO:teuthology.orchestra.run.smithi190.stdout: "statfs_resend": 0, 2024-08-30T09:16:27.169 INFO:teuthology.orchestra.run.smithi190.stdout: "command_active": 0, 2024-08-30T09:16:27.169 INFO:teuthology.orchestra.run.smithi190.stdout: "command_send": 0, 2024-08-30T09:16:27.169 INFO:teuthology.orchestra.run.smithi190.stdout: "command_resend": 0, 2024-08-30T09:16:27.169 INFO:teuthology.orchestra.run.smithi190.stdout: "map_epoch": 83, 2024-08-30T09:16:27.169 INFO:teuthology.orchestra.run.smithi190.stdout: "map_full": 0, 2024-08-30T09:16:27.169 INFO:teuthology.orchestra.run.smithi190.stdout: "map_inc": 2, 2024-08-30T09:16:27.169 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_sessions": 5, 2024-08-30T09:16:27.169 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_session_open": 6, 2024-08-30T09:16:27.169 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_session_close": 1, 2024-08-30T09:16:27.169 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_laggy": 0, 2024-08-30T09:16:27.169 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_wr": 0, 2024-08-30T09:16:27.169 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_rd": 2, 2024-08-30T09:16:27.169 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_del": 0 2024-08-30T09:16:27.170 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-30T09:16:27.170 INFO:teuthology.orchestra.run.smithi190.stdout: "oft": { 2024-08-30T09:16:27.170 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_total_objs": 0, 2024-08-30T09:16:27.170 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_total_kv_pairs": 0, 2024-08-30T09:16:27.170 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_total_updates": 0, 2024-08-30T09:16:27.170 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_total_removes": 0 2024-08-30T09:16:27.170 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-30T09:16:27.170 INFO:teuthology.orchestra.run.smithi190.stdout: "purge_queue": { 2024-08-30T09:16:27.170 INFO:teuthology.orchestra.run.smithi190.stdout: "pq_executing_ops": 0, 2024-08-30T09:16:27.170 INFO:teuthology.orchestra.run.smithi190.stdout: "pq_executing_ops_high_water": 0, 2024-08-30T09:16:27.170 INFO:teuthology.orchestra.run.smithi190.stdout: "pq_executing": 0, 2024-08-30T09:16:27.170 INFO:teuthology.orchestra.run.smithi190.stdout: "pq_executing_high_water": 0, 2024-08-30T09:16:27.170 INFO:teuthology.orchestra.run.smithi190.stdout: "pq_executed_ops": 0, 2024-08-30T09:16:27.170 INFO:teuthology.orchestra.run.smithi190.stdout: "pq_executed": 0, 2024-08-30T09:16:27.170 INFO:teuthology.orchestra.run.smithi190.stdout: "pq_item_in_journal": 0 2024-08-30T09:16:27.170 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-30T09:16:27.171 INFO:teuthology.orchestra.run.smithi190.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-08-30T09:16:27.171 INFO:teuthology.orchestra.run.smithi190.stdout: "val": 0, 2024-08-30T09:16:27.171 INFO:teuthology.orchestra.run.smithi190.stdout: "max": 104857600, 2024-08-30T09:16:27.171 INFO:teuthology.orchestra.run.smithi190.stdout: "get_started": 0, 2024-08-30T09:16:27.171 INFO:teuthology.orchestra.run.smithi190.stdout: "get": 3794, 2024-08-30T09:16:27.171 INFO:teuthology.orchestra.run.smithi190.stdout: "get_sum": 13506859, 2024-08-30T09:16:27.171 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_fail": 0, 2024-08-30T09:16:27.171 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_success": 3794, 2024-08-30T09:16:27.171 INFO:teuthology.orchestra.run.smithi190.stdout: "take": 0, 2024-08-30T09:16:27.171 INFO:teuthology.orchestra.run.smithi190.stdout: "take_sum": 0, 2024-08-30T09:16:27.171 INFO:teuthology.orchestra.run.smithi190.stdout: "put": 3794, 2024-08-30T09:16:27.171 INFO:teuthology.orchestra.run.smithi190.stdout: "put_sum": 13506859, 2024-08-30T09:16:27.171 INFO:teuthology.orchestra.run.smithi190.stdout: "wait": { 2024-08-30T09:16:27.171 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-30T09:16:27.171 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-30T09:16:27.172 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-30T09:16:27.172 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-08-30T09:16:27.172 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-30T09:16:27.172 INFO:teuthology.orchestra.run.smithi190.stdout: "throttle-objecter_bytes": { 2024-08-30T09:16:27.172 INFO:teuthology.orchestra.run.smithi190.stdout: "val": 0, 2024-08-30T09:16:27.172 INFO:teuthology.orchestra.run.smithi190.stdout: "max": 104857600, 2024-08-30T09:16:27.172 INFO:teuthology.orchestra.run.smithi190.stdout: "get_started": 0, 2024-08-30T09:16:27.172 INFO:teuthology.orchestra.run.smithi190.stdout: "get": 0, 2024-08-30T09:16:27.172 INFO:teuthology.orchestra.run.smithi190.stdout: "get_sum": 0, 2024-08-30T09:16:27.172 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_fail": 0, 2024-08-30T09:16:27.172 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_success": 0, 2024-08-30T09:16:27.172 INFO:teuthology.orchestra.run.smithi190.stdout: "take": 3455, 2024-08-30T09:16:27.172 INFO:teuthology.orchestra.run.smithi190.stdout: "take_sum": 12233925, 2024-08-30T09:16:27.173 INFO:teuthology.orchestra.run.smithi190.stdout: "put": 3455, 2024-08-30T09:16:27.173 INFO:teuthology.orchestra.run.smithi190.stdout: "put_sum": 12233925, 2024-08-30T09:16:27.173 INFO:teuthology.orchestra.run.smithi190.stdout: "wait": { 2024-08-30T09:16:27.173 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-30T09:16:27.173 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-30T09:16:27.173 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-30T09:16:27.173 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-08-30T09:16:27.173 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-30T09:16:27.173 INFO:teuthology.orchestra.run.smithi190.stdout: "throttle-objecter_ops": { 2024-08-30T09:16:27.173 INFO:teuthology.orchestra.run.smithi190.stdout: "val": 0, 2024-08-30T09:16:27.173 INFO:teuthology.orchestra.run.smithi190.stdout: "max": 1024, 2024-08-30T09:16:27.173 INFO:teuthology.orchestra.run.smithi190.stdout: "get_started": 0, 2024-08-30T09:16:27.173 INFO:teuthology.orchestra.run.smithi190.stdout: "get": 0, 2024-08-30T09:16:27.174 INFO:teuthology.orchestra.run.smithi190.stdout: "get_sum": 0, 2024-08-30T09:16:27.174 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_fail": 0, 2024-08-30T09:16:27.174 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_success": 0, 2024-08-30T09:16:27.174 INFO:teuthology.orchestra.run.smithi190.stdout: "take": 3455, 2024-08-30T09:16:27.174 INFO:teuthology.orchestra.run.smithi190.stdout: "take_sum": 3455, 2024-08-30T09:16:27.174 INFO:teuthology.orchestra.run.smithi190.stdout: "put": 3455, 2024-08-30T09:16:27.174 INFO:teuthology.orchestra.run.smithi190.stdout: "put_sum": 3455, 2024-08-30T09:16:27.174 INFO:teuthology.orchestra.run.smithi190.stdout: "wait": { 2024-08-30T09:16:27.174 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-30T09:16:27.174 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-30T09:16:27.174 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-30T09:16:27.174 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-08-30T09:16:27.174 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-30T09:16:27.174 INFO:teuthology.orchestra.run.smithi190.stdout: "throttle-write_buf_throttle": { 2024-08-30T09:16:27.174 INFO:teuthology.orchestra.run.smithi190.stdout: "val": 0, 2024-08-30T09:16:27.175 INFO:teuthology.orchestra.run.smithi190.stdout: "max": 3758096384, 2024-08-30T09:16:27.175 INFO:teuthology.orchestra.run.smithi190.stdout: "get_started": 0, 2024-08-30T09:16:27.175 INFO:teuthology.orchestra.run.smithi190.stdout: "get": 0, 2024-08-30T09:16:27.175 INFO:teuthology.orchestra.run.smithi190.stdout: "get_sum": 0, 2024-08-30T09:16:27.175 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_fail": 0, 2024-08-30T09:16:27.175 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_success": 0, 2024-08-30T09:16:27.175 INFO:teuthology.orchestra.run.smithi190.stdout: "take": 0, 2024-08-30T09:16:27.175 INFO:teuthology.orchestra.run.smithi190.stdout: "take_sum": 0, 2024-08-30T09:16:27.175 INFO:teuthology.orchestra.run.smithi190.stdout: "put": 0, 2024-08-30T09:16:27.175 INFO:teuthology.orchestra.run.smithi190.stdout: "put_sum": 0, 2024-08-30T09:16:27.175 INFO:teuthology.orchestra.run.smithi190.stdout: "wait": { 2024-08-30T09:16:27.175 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-30T09:16:27.175 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-30T09:16:27.175 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-30T09:16:27.175 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-08-30T09:16:27.175 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-30T09:16:27.176 INFO:teuthology.orchestra.run.smithi190.stdout: "throttle-write_buf_throttle-0x55f4ef4f6750": { 2024-08-30T09:16:27.176 INFO:teuthology.orchestra.run.smithi190.stdout: "val": 0, 2024-08-30T09:16:27.176 INFO:teuthology.orchestra.run.smithi190.stdout: "max": 3758096384, 2024-08-30T09:16:27.176 INFO:teuthology.orchestra.run.smithi190.stdout: "get_started": 0, 2024-08-30T09:16:27.176 INFO:teuthology.orchestra.run.smithi190.stdout: "get": 0, 2024-08-30T09:16:27.176 INFO:teuthology.orchestra.run.smithi190.stdout: "get_sum": 0, 2024-08-30T09:16:27.176 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_fail": 0, 2024-08-30T09:16:27.176 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_success": 0, 2024-08-30T09:16:27.176 INFO:teuthology.orchestra.run.smithi190.stdout: "take": 0, 2024-08-30T09:16:27.176 INFO:teuthology.orchestra.run.smithi190.stdout: "take_sum": 0, 2024-08-30T09:16:27.176 INFO:teuthology.orchestra.run.smithi190.stdout: "put": 0, 2024-08-30T09:16:27.176 INFO:teuthology.orchestra.run.smithi190.stdout: "put_sum": 0, 2024-08-30T09:16:27.176 INFO:teuthology.orchestra.run.smithi190.stdout: "wait": { 2024-08-30T09:16:27.176 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-30T09:16:27.177 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-30T09:16:27.177 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-30T09:16:27.177 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-08-30T09:16:27.177 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-30T09:16:27.177 INFO:teuthology.orchestra.run.smithi190.stdout: "trackedop": { 2024-08-30T09:16:27.177 INFO:teuthology.orchestra.run.smithi190.stdout: "slow_ops_count": 0 2024-08-30T09:16:27.177 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-08-30T09:16:27.177 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-08-30T09:16:27.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:16:27 smithi042 ceph-mon[18652]: pgmap v843: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 7.7 KiB/s wr, 8 op/s 2024-08-30T09:16:27.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:16:27 smithi190 ceph-mon[22001]: pgmap v843: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 7.7 KiB/s wr, 8 op/s 2024-08-30T09:16:27.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:16:27 smithi167 ceph-mon[22063]: pgmap v843: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 7.7 KiB/s wr, 8 op/s 2024-08-30T09:16:28.519 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-08-30T09:16:28.519 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2024-08-30T09:16:28.519 DEBUG:tasks.check_counter:Getting stats from f 2024-08-30T09:16:28.519 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph daemon mds.f perf dump 2024-08-30T09:16:28.777 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.c/config 2024-08-30T09:16:29.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:16:29 smithi190 ceph-mon[22001]: pgmap v844: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 5.5 KiB/s wr, 6 op/s 2024-08-30T09:16:29.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:16:29 smithi042 ceph-mon[18652]: pgmap v844: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 5.5 KiB/s wr, 6 op/s 2024-08-30T09:16:29.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:16:29 smithi167 ceph-mon[22063]: pgmap v844: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 5.5 KiB/s wr, 6 op/s 2024-08-30T09:16:29.650 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-08-30T09:16:29.650 INFO:teuthology.orchestra.run.smithi190.stdout: "AsyncMessenger::Worker-0": { 2024-08-30T09:16:29.650 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_messages": 1374, 2024-08-30T09:16:29.650 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_messages": 1346, 2024-08-30T09:16:29.650 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_bytes": 1019664, 2024-08-30T09:16:29.651 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_bytes": 517432, 2024-08-30T09:16:29.651 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_created_connections": 5, 2024-08-30T09:16:29.651 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_active_connections": 1, 2024-08-30T09:16:29.651 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_total_time": 0.183211261, 2024-08-30T09:16:29.651 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_send_time": 0.077919791, 2024-08-30T09:16:29.651 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_recv_time": 0.061107496, 2024-08-30T09:16:29.651 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_fast_dispatch_time": 0.040551184, 2024-08-30T09:16:29.651 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_messages_queue_lat": { 2024-08-30T09:16:29.651 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 1345, 2024-08-30T09:16:29.651 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.112210231, 2024-08-30T09:16:29.651 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000083427 2024-08-30T09:16:29.651 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-30T09:16:29.651 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_handle_ack_lat": { 2024-08-30T09:16:29.651 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-30T09:16:29.652 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-30T09:16:29.652 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-30T09:16:29.652 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-30T09:16:29.652 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_encrypted_bytes": 671568, 2024-08-30T09:16:29.652 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_encrypted_bytes": 187984 2024-08-30T09:16:29.652 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-30T09:16:29.652 INFO:teuthology.orchestra.run.smithi190.stdout: "AsyncMessenger::Worker-1": { 2024-08-30T09:16:29.652 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_messages": 8, 2024-08-30T09:16:29.652 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_messages": 235, 2024-08-30T09:16:29.652 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_bytes": 3820, 2024-08-30T09:16:29.652 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_bytes": 570586, 2024-08-30T09:16:29.652 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_created_connections": 6, 2024-08-30T09:16:29.653 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_active_connections": 3, 2024-08-30T09:16:29.653 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_total_time": 0.018222748, 2024-08-30T09:16:29.653 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_send_time": 0.012615467, 2024-08-30T09:16:29.653 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_recv_time": 0.003670123, 2024-08-30T09:16:29.653 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_fast_dispatch_time": 0.000079609, 2024-08-30T09:16:29.653 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_messages_queue_lat": { 2024-08-30T09:16:29.653 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 234, 2024-08-30T09:16:29.653 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.025567381, 2024-08-30T09:16:29.653 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000109262 2024-08-30T09:16:29.653 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-30T09:16:29.653 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_handle_ack_lat": { 2024-08-30T09:16:29.653 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-30T09:16:29.653 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-30T09:16:29.654 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-30T09:16:29.654 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-30T09:16:29.654 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_encrypted_bytes": 3264, 2024-08-30T09:16:29.654 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_encrypted_bytes": 569888 2024-08-30T09:16:29.654 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-30T09:16:29.654 INFO:teuthology.orchestra.run.smithi190.stdout: "AsyncMessenger::Worker-2": { 2024-08-30T09:16:29.654 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_messages": 2252, 2024-08-30T09:16:29.654 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_messages": 2254, 2024-08-30T09:16:29.654 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_bytes": 5036763, 2024-08-30T09:16:29.654 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_bytes": 716143, 2024-08-30T09:16:29.654 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_created_connections": 5, 2024-08-30T09:16:29.654 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_active_connections": 3, 2024-08-30T09:16:29.654 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_total_time": 0.215481694, 2024-08-30T09:16:29.655 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_send_time": 0.076185457, 2024-08-30T09:16:29.655 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_recv_time": 0.049697959, 2024-08-30T09:16:29.655 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_fast_dispatch_time": 0.077292282, 2024-08-30T09:16:29.655 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_messages_queue_lat": { 2024-08-30T09:16:29.655 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 2252, 2024-08-30T09:16:29.655 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.113290049, 2024-08-30T09:16:29.655 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000050306 2024-08-30T09:16:29.655 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-30T09:16:29.655 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_handle_ack_lat": { 2024-08-30T09:16:29.655 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-30T09:16:29.655 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-30T09:16:29.655 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-30T09:16:29.656 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-30T09:16:29.656 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_encrypted_bytes": 0, 2024-08-30T09:16:29.656 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_encrypted_bytes": 0 2024-08-30T09:16:29.656 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-30T09:16:29.656 INFO:teuthology.orchestra.run.smithi190.stdout: "cct": { 2024-08-30T09:16:29.656 INFO:teuthology.orchestra.run.smithi190.stdout: "total_workers": 1, 2024-08-30T09:16:29.656 INFO:teuthology.orchestra.run.smithi190.stdout: "unhealthy_workers": 0 2024-08-30T09:16:29.656 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-30T09:16:29.656 INFO:teuthology.orchestra.run.smithi190.stdout: "finisher-MDSRank": { 2024-08-30T09:16:29.656 INFO:teuthology.orchestra.run.smithi190.stdout: "queue_len": 0, 2024-08-30T09:16:29.656 INFO:teuthology.orchestra.run.smithi190.stdout: "complete_latency": { 2024-08-30T09:16:29.656 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 5491, 2024-08-30T09:16:29.656 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.181992570, 2024-08-30T09:16:29.657 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000033143 2024-08-30T09:16:29.657 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-08-30T09:16:29.657 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-30T09:16:29.657 INFO:teuthology.orchestra.run.smithi190.stdout: "finisher-PurgeQueue": { 2024-08-30T09:16:29.657 INFO:teuthology.orchestra.run.smithi190.stdout: "queue_len": 0, 2024-08-30T09:16:29.657 INFO:teuthology.orchestra.run.smithi190.stdout: "complete_latency": { 2024-08-30T09:16:29.657 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-30T09:16:29.657 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-30T09:16:29.657 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-30T09:16:29.657 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-08-30T09:16:29.657 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-30T09:16:29.657 INFO:teuthology.orchestra.run.smithi190.stdout: "mds": { 2024-08-30T09:16:29.657 INFO:teuthology.orchestra.run.smithi190.stdout: "request": 0, 2024-08-30T09:16:29.658 INFO:teuthology.orchestra.run.smithi190.stdout: "reply": 0, 2024-08-30T09:16:29.658 INFO:teuthology.orchestra.run.smithi190.stdout: "reply_latency": { 2024-08-30T09:16:29.658 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-30T09:16:29.658 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-30T09:16:29.658 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-30T09:16:29.658 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-30T09:16:29.658 INFO:teuthology.orchestra.run.smithi190.stdout: "slow_reply": 0, 2024-08-30T09:16:29.658 INFO:teuthology.orchestra.run.smithi190.stdout: "forward": 0, 2024-08-30T09:16:29.658 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_fetch_complete": 0, 2024-08-30T09:16:29.658 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_fetch_keys": 0, 2024-08-30T09:16:29.658 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_commit": 0, 2024-08-30T09:16:29.658 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_split": 0, 2024-08-30T09:16:29.658 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_merge": 0, 2024-08-30T09:16:29.658 INFO:teuthology.orchestra.run.smithi190.stdout: "inodes": 662, 2024-08-30T09:16:29.659 INFO:teuthology.orchestra.run.smithi190.stdout: "inodes_top": 337, 2024-08-30T09:16:29.659 INFO:teuthology.orchestra.run.smithi190.stdout: "inodes_bottom": 325, 2024-08-30T09:16:29.659 INFO:teuthology.orchestra.run.smithi190.stdout: "inodes_pin_tail": 0, 2024-08-30T09:16:29.659 INFO:teuthology.orchestra.run.smithi190.stdout: "inodes_pinned": 180, 2024-08-30T09:16:29.659 INFO:teuthology.orchestra.run.smithi190.stdout: "inodes_expired": 0, 2024-08-30T09:16:29.659 INFO:teuthology.orchestra.run.smithi190.stdout: "inodes_with_caps": 0, 2024-08-30T09:16:29.659 INFO:teuthology.orchestra.run.smithi190.stdout: "caps": 0, 2024-08-30T09:16:29.659 INFO:teuthology.orchestra.run.smithi190.stdout: "subtrees": 13, 2024-08-30T09:16:29.659 INFO:teuthology.orchestra.run.smithi190.stdout: "traverse": 0, 2024-08-30T09:16:29.659 INFO:teuthology.orchestra.run.smithi190.stdout: "traverse_hit": 0, 2024-08-30T09:16:29.659 INFO:teuthology.orchestra.run.smithi190.stdout: "traverse_forward": 0, 2024-08-30T09:16:29.659 INFO:teuthology.orchestra.run.smithi190.stdout: "traverse_discover": 0, 2024-08-30T09:16:29.660 INFO:teuthology.orchestra.run.smithi190.stdout: "traverse_dir_fetch": 0, 2024-08-30T09:16:29.660 INFO:teuthology.orchestra.run.smithi190.stdout: "traverse_remote_ino": 0, 2024-08-30T09:16:29.660 INFO:teuthology.orchestra.run.smithi190.stdout: "traverse_lock": 0, 2024-08-30T09:16:29.660 INFO:teuthology.orchestra.run.smithi190.stdout: "load_cent": 0, 2024-08-30T09:16:29.660 INFO:teuthology.orchestra.run.smithi190.stdout: "q": 0, 2024-08-30T09:16:29.660 INFO:teuthology.orchestra.run.smithi190.stdout: "exported": 0, 2024-08-30T09:16:29.660 INFO:teuthology.orchestra.run.smithi190.stdout: "exported_inodes": 0, 2024-08-30T09:16:29.660 INFO:teuthology.orchestra.run.smithi190.stdout: "imported": 0, 2024-08-30T09:16:29.660 INFO:teuthology.orchestra.run.smithi190.stdout: "imported_inodes": 0, 2024-08-30T09:16:29.660 INFO:teuthology.orchestra.run.smithi190.stdout: "openino_dir_fetch": 0, 2024-08-30T09:16:29.660 INFO:teuthology.orchestra.run.smithi190.stdout: "openino_backtrace_fetch": 0, 2024-08-30T09:16:29.660 INFO:teuthology.orchestra.run.smithi190.stdout: "openino_peer_discover": 0, 2024-08-30T09:16:29.660 INFO:teuthology.orchestra.run.smithi190.stdout: "root_rfiles": 0, 2024-08-30T09:16:29.660 INFO:teuthology.orchestra.run.smithi190.stdout: "root_rbytes": 0, 2024-08-30T09:16:29.660 INFO:teuthology.orchestra.run.smithi190.stdout: "root_rsnaps": 0, 2024-08-30T09:16:29.661 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_backtrace_fetch": 0, 2024-08-30T09:16:29.661 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_set_tag": 0, 2024-08-30T09:16:29.661 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_backtrace_repaired": 0, 2024-08-30T09:16:29.661 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_inotable_repaired": 0, 2024-08-30T09:16:29.661 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_dir_inodes": 0, 2024-08-30T09:16:29.661 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_dir_base_inodes": 0, 2024-08-30T09:16:29.661 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_dirfrag_rstats": 0, 2024-08-30T09:16:29.661 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_file_inodes": 0, 2024-08-30T09:16:29.661 INFO:teuthology.orchestra.run.smithi190.stdout: "handle_inode_file_caps": 0, 2024-08-30T09:16:29.661 INFO:teuthology.orchestra.run.smithi190.stdout: "ceph_cap_op_revoke": 0, 2024-08-30T09:16:29.661 INFO:teuthology.orchestra.run.smithi190.stdout: "ceph_cap_op_grant": 0, 2024-08-30T09:16:29.661 INFO:teuthology.orchestra.run.smithi190.stdout: "ceph_cap_op_trunc": 0, 2024-08-30T09:16:29.661 INFO:teuthology.orchestra.run.smithi190.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-08-30T09:16:29.661 INFO:teuthology.orchestra.run.smithi190.stdout: "ceph_cap_op_flush_ack": 0, 2024-08-30T09:16:29.662 INFO:teuthology.orchestra.run.smithi190.stdout: "handle_client_caps": 0, 2024-08-30T09:16:29.662 INFO:teuthology.orchestra.run.smithi190.stdout: "handle_client_caps_dirty": 0, 2024-08-30T09:16:29.662 INFO:teuthology.orchestra.run.smithi190.stdout: "handle_client_cap_release": 0, 2024-08-30T09:16:29.662 INFO:teuthology.orchestra.run.smithi190.stdout: "process_request_cap_release": 0 2024-08-30T09:16:29.662 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-30T09:16:29.662 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_cache": { 2024-08-30T09:16:29.662 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_update": 0, 2024-08-30T09:16:29.662 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_update_receipt": 0, 2024-08-30T09:16:29.662 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_try_discover": 0, 2024-08-30T09:16:29.662 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_send_discover": 0, 2024-08-30T09:16:29.662 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_handle_discover": 0, 2024-08-30T09:16:29.662 INFO:teuthology.orchestra.run.smithi190.stdout: "num_strays": 0, 2024-08-30T09:16:29.662 INFO:teuthology.orchestra.run.smithi190.stdout: "num_strays_delayed": 0, 2024-08-30T09:16:29.662 INFO:teuthology.orchestra.run.smithi190.stdout: "num_strays_enqueuing": 0, 2024-08-30T09:16:29.663 INFO:teuthology.orchestra.run.smithi190.stdout: "strays_created": 0, 2024-08-30T09:16:29.663 INFO:teuthology.orchestra.run.smithi190.stdout: "strays_enqueued": 0, 2024-08-30T09:16:29.663 INFO:teuthology.orchestra.run.smithi190.stdout: "strays_reintegrated": 0, 2024-08-30T09:16:29.663 INFO:teuthology.orchestra.run.smithi190.stdout: "strays_migrated": 0, 2024-08-30T09:16:29.663 INFO:teuthology.orchestra.run.smithi190.stdout: "num_recovering_processing": 0, 2024-08-30T09:16:29.663 INFO:teuthology.orchestra.run.smithi190.stdout: "num_recovering_enqueued": 0, 2024-08-30T09:16:29.663 INFO:teuthology.orchestra.run.smithi190.stdout: "num_recovering_prioritized": 0, 2024-08-30T09:16:29.663 INFO:teuthology.orchestra.run.smithi190.stdout: "recovery_started": 0, 2024-08-30T09:16:29.663 INFO:teuthology.orchestra.run.smithi190.stdout: "recovery_completed": 0, 2024-08-30T09:16:29.663 INFO:teuthology.orchestra.run.smithi190.stdout: "ireq_quiesce_path": 0, 2024-08-30T09:16:29.663 INFO:teuthology.orchestra.run.smithi190.stdout: "ireq_quiesce_inode": 0, 2024-08-30T09:16:29.663 INFO:teuthology.orchestra.run.smithi190.stdout: "ireq_enqueue_scrub": 0, 2024-08-30T09:16:29.663 INFO:teuthology.orchestra.run.smithi190.stdout: "ireq_exportdir": 0, 2024-08-30T09:16:29.663 INFO:teuthology.orchestra.run.smithi190.stdout: "ireq_flush": 0, 2024-08-30T09:16:29.663 INFO:teuthology.orchestra.run.smithi190.stdout: "ireq_fragmentdir": 0, 2024-08-30T09:16:29.664 INFO:teuthology.orchestra.run.smithi190.stdout: "ireq_fragstats": 0, 2024-08-30T09:16:29.664 INFO:teuthology.orchestra.run.smithi190.stdout: "ireq_inodestats": 0, 2024-08-30T09:16:29.664 INFO:teuthology.orchestra.run.smithi190.stdout: "uninline_started": 0, 2024-08-30T09:16:29.664 INFO:teuthology.orchestra.run.smithi190.stdout: "uninline_succeeded": 0, 2024-08-30T09:16:29.664 INFO:teuthology.orchestra.run.smithi190.stdout: "uninline_write_failed": 0 2024-08-30T09:16:29.664 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-30T09:16:29.664 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_log": { 2024-08-30T09:16:29.664 INFO:teuthology.orchestra.run.smithi190.stdout: "evlrg": 0, 2024-08-30T09:16:29.664 INFO:teuthology.orchestra.run.smithi190.stdout: "evadd": 0, 2024-08-30T09:16:29.664 INFO:teuthology.orchestra.run.smithi190.stdout: "evex": 0, 2024-08-30T09:16:29.664 INFO:teuthology.orchestra.run.smithi190.stdout: "evtrm": 0, 2024-08-30T09:16:29.664 INFO:teuthology.orchestra.run.smithi190.stdout: "ev": 2458, 2024-08-30T09:16:29.665 INFO:teuthology.orchestra.run.smithi190.stdout: "evexg": 0, 2024-08-30T09:16:29.665 INFO:teuthology.orchestra.run.smithi190.stdout: "evexd": 0, 2024-08-30T09:16:29.665 INFO:teuthology.orchestra.run.smithi190.stdout: "segadd": 0, 2024-08-30T09:16:29.665 INFO:teuthology.orchestra.run.smithi190.stdout: "segex": 0, 2024-08-30T09:16:29.665 INFO:teuthology.orchestra.run.smithi190.stdout: "segtrm": 0, 2024-08-30T09:16:29.665 INFO:teuthology.orchestra.run.smithi190.stdout: "seg": 5, 2024-08-30T09:16:29.665 INFO:teuthology.orchestra.run.smithi190.stdout: "segmjr": 2, 2024-08-30T09:16:29.665 INFO:teuthology.orchestra.run.smithi190.stdout: "segexg": 0, 2024-08-30T09:16:29.665 INFO:teuthology.orchestra.run.smithi190.stdout: "segexd": 0, 2024-08-30T09:16:29.665 INFO:teuthology.orchestra.run.smithi190.stdout: "expos": 4194304, 2024-08-30T09:16:29.665 INFO:teuthology.orchestra.run.smithi190.stdout: "wrpos": 8659614, 2024-08-30T09:16:29.665 INFO:teuthology.orchestra.run.smithi190.stdout: "rdpos": 8652865, 2024-08-30T09:16:29.665 INFO:teuthology.orchestra.run.smithi190.stdout: "jlat": { 2024-08-30T09:16:29.666 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-30T09:16:29.666 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-30T09:16:29.666 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-30T09:16:29.666 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-30T09:16:29.666 INFO:teuthology.orchestra.run.smithi190.stdout: "replayed": 2458 2024-08-30T09:16:29.666 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-30T09:16:29.666 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_mem": { 2024-08-30T09:16:29.666 INFO:teuthology.orchestra.run.smithi190.stdout: "ino": 507, 2024-08-30T09:16:29.666 INFO:teuthology.orchestra.run.smithi190.stdout: "ino+": 1304, 2024-08-30T09:16:29.666 INFO:teuthology.orchestra.run.smithi190.stdout: "ino-": 797, 2024-08-30T09:16:29.666 INFO:teuthology.orchestra.run.smithi190.stdout: "dir": 38, 2024-08-30T09:16:29.666 INFO:teuthology.orchestra.run.smithi190.stdout: "dir+": 81, 2024-08-30T09:16:29.666 INFO:teuthology.orchestra.run.smithi190.stdout: "dir-": 43, 2024-08-30T09:16:29.666 INFO:teuthology.orchestra.run.smithi190.stdout: "dn": 662, 2024-08-30T09:16:29.667 INFO:teuthology.orchestra.run.smithi190.stdout: "dn+": 1618, 2024-08-30T09:16:29.667 INFO:teuthology.orchestra.run.smithi190.stdout: "dn-": 956, 2024-08-30T09:16:29.667 INFO:teuthology.orchestra.run.smithi190.stdout: "cap": 0, 2024-08-30T09:16:29.667 INFO:teuthology.orchestra.run.smithi190.stdout: "cap+": 0, 2024-08-30T09:16:29.667 INFO:teuthology.orchestra.run.smithi190.stdout: "cap-": 0, 2024-08-30T09:16:29.667 INFO:teuthology.orchestra.run.smithi190.stdout: "rss": 121820, 2024-08-30T09:16:29.667 INFO:teuthology.orchestra.run.smithi190.stdout: "heap": 321789952 2024-08-30T09:16:29.667 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-30T09:16:29.667 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_server": { 2024-08-30T09:16:29.667 INFO:teuthology.orchestra.run.smithi190.stdout: "dispatch_client_request": 0, 2024-08-30T09:16:29.667 INFO:teuthology.orchestra.run.smithi190.stdout: "dispatch_server_request": 0, 2024-08-30T09:16:29.667 INFO:teuthology.orchestra.run.smithi190.stdout: "handle_client_request": 0, 2024-08-30T09:16:29.667 INFO:teuthology.orchestra.run.smithi190.stdout: "handle_client_session": 0, 2024-08-30T09:16:29.667 INFO:teuthology.orchestra.run.smithi190.stdout: "handle_peer_request": 0, 2024-08-30T09:16:29.668 INFO:teuthology.orchestra.run.smithi190.stdout: "req_create_latency": { 2024-08-30T09:16:29.668 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-30T09:16:29.668 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-30T09:16:29.668 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-30T09:16:29.668 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-30T09:16:29.668 INFO:teuthology.orchestra.run.smithi190.stdout: "req_getattr_latency": { 2024-08-30T09:16:29.668 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-30T09:16:29.668 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-30T09:16:29.668 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-30T09:16:29.668 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-30T09:16:29.668 INFO:teuthology.orchestra.run.smithi190.stdout: "req_getfilelock_latency": { 2024-08-30T09:16:29.668 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-30T09:16:29.668 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-30T09:16:29.669 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-30T09:16:29.669 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-30T09:16:29.669 INFO:teuthology.orchestra.run.smithi190.stdout: "req_link_latency": { 2024-08-30T09:16:29.669 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-30T09:16:29.669 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-30T09:16:29.669 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-30T09:16:29.669 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-30T09:16:29.669 INFO:teuthology.orchestra.run.smithi190.stdout: "req_lookup_latency": { 2024-08-30T09:16:29.669 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-30T09:16:29.670 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-30T09:16:29.670 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-30T09:16:29.670 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-30T09:16:29.670 INFO:teuthology.orchestra.run.smithi190.stdout: "req_lookuphash_latency": { 2024-08-30T09:16:29.670 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-30T09:16:29.670 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-30T09:16:29.670 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-30T09:16:29.670 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-30T09:16:29.670 INFO:teuthology.orchestra.run.smithi190.stdout: "req_lookupino_latency": { 2024-08-30T09:16:29.670 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-30T09:16:29.670 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-30T09:16:29.670 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-30T09:16:29.670 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-30T09:16:29.670 INFO:teuthology.orchestra.run.smithi190.stdout: "req_lookupname_latency": { 2024-08-30T09:16:29.671 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-30T09:16:29.671 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-30T09:16:29.671 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-30T09:16:29.671 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-30T09:16:29.671 INFO:teuthology.orchestra.run.smithi190.stdout: "req_lookupparent_latency": { 2024-08-30T09:16:29.671 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-30T09:16:29.671 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-30T09:16:29.671 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-30T09:16:29.671 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-30T09:16:29.671 INFO:teuthology.orchestra.run.smithi190.stdout: "req_lookupsnap_latency": { 2024-08-30T09:16:29.671 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-30T09:16:29.671 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-30T09:16:29.671 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-30T09:16:29.671 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-30T09:16:29.671 INFO:teuthology.orchestra.run.smithi190.stdout: "req_lssnap_latency": { 2024-08-30T09:16:29.672 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-30T09:16:29.672 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-30T09:16:29.672 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-30T09:16:29.672 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-30T09:16:29.672 INFO:teuthology.orchestra.run.smithi190.stdout: "req_mkdir_latency": { 2024-08-30T09:16:29.672 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-30T09:16:29.672 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-30T09:16:29.673 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-30T09:16:29.673 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-30T09:16:29.673 INFO:teuthology.orchestra.run.smithi190.stdout: "req_mknod_latency": { 2024-08-30T09:16:29.673 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-30T09:16:29.673 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-30T09:16:29.673 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-30T09:16:29.673 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-30T09:16:29.673 INFO:teuthology.orchestra.run.smithi190.stdout: "req_mksnap_latency": { 2024-08-30T09:16:29.673 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-30T09:16:29.673 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-30T09:16:29.673 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-30T09:16:29.673 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-30T09:16:29.673 INFO:teuthology.orchestra.run.smithi190.stdout: "req_open_latency": { 2024-08-30T09:16:29.674 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-30T09:16:29.674 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-30T09:16:29.674 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-30T09:16:29.674 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-30T09:16:29.674 INFO:teuthology.orchestra.run.smithi190.stdout: "req_readdir_latency": { 2024-08-30T09:16:29.674 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-30T09:16:29.674 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-30T09:16:29.674 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-30T09:16:29.674 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-30T09:16:29.674 INFO:teuthology.orchestra.run.smithi190.stdout: "req_rename_latency": { 2024-08-30T09:16:29.674 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-30T09:16:29.674 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-30T09:16:29.674 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-30T09:16:29.674 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-30T09:16:29.675 INFO:teuthology.orchestra.run.smithi190.stdout: "req_renamesnap_latency": { 2024-08-30T09:16:29.675 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-30T09:16:29.675 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-30T09:16:29.675 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-30T09:16:29.675 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-30T09:16:29.675 INFO:teuthology.orchestra.run.smithi190.stdout: "req_snapdiff_latency": { 2024-08-30T09:16:29.675 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-30T09:16:29.675 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-30T09:16:29.675 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-30T09:16:29.675 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-30T09:16:29.675 INFO:teuthology.orchestra.run.smithi190.stdout: "req_rmdir_latency": { 2024-08-30T09:16:29.675 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-30T09:16:29.675 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-30T09:16:29.676 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-30T09:16:29.676 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-30T09:16:29.676 INFO:teuthology.orchestra.run.smithi190.stdout: "req_rmsnap_latency": { 2024-08-30T09:16:29.676 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-30T09:16:29.676 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-30T09:16:29.676 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-30T09:16:29.676 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-30T09:16:29.676 INFO:teuthology.orchestra.run.smithi190.stdout: "req_rmxattr_latency": { 2024-08-30T09:16:29.676 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-30T09:16:29.676 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-30T09:16:29.676 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-30T09:16:29.676 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-30T09:16:29.676 INFO:teuthology.orchestra.run.smithi190.stdout: "req_setattr_latency": { 2024-08-30T09:16:29.677 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-30T09:16:29.677 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-30T09:16:29.677 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-30T09:16:29.677 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-30T09:16:29.677 INFO:teuthology.orchestra.run.smithi190.stdout: "req_setdirlayout_latency": { 2024-08-30T09:16:29.677 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-30T09:16:29.677 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-30T09:16:29.677 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-30T09:16:29.677 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-30T09:16:29.677 INFO:teuthology.orchestra.run.smithi190.stdout: "req_setfilelock_latency": { 2024-08-30T09:16:29.677 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-30T09:16:29.677 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-30T09:16:29.677 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-30T09:16:29.677 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-30T09:16:29.678 INFO:teuthology.orchestra.run.smithi190.stdout: "req_setlayout_latency": { 2024-08-30T09:16:29.678 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-30T09:16:29.678 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-30T09:16:29.678 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-30T09:16:29.678 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-30T09:16:29.678 INFO:teuthology.orchestra.run.smithi190.stdout: "req_setxattr_latency": { 2024-08-30T09:16:29.678 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-30T09:16:29.678 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-30T09:16:29.678 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-30T09:16:29.678 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-30T09:16:29.678 INFO:teuthology.orchestra.run.smithi190.stdout: "req_symlink_latency": { 2024-08-30T09:16:29.678 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-30T09:16:29.678 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-30T09:16:29.679 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-30T09:16:29.679 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-30T09:16:29.679 INFO:teuthology.orchestra.run.smithi190.stdout: "req_unlink_latency": { 2024-08-30T09:16:29.679 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-30T09:16:29.679 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-30T09:16:29.679 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-30T09:16:29.679 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-30T09:16:29.679 INFO:teuthology.orchestra.run.smithi190.stdout: "cap_revoke_eviction": 0, 2024-08-30T09:16:29.679 INFO:teuthology.orchestra.run.smithi190.stdout: "cap_acquisition_throttle": 0, 2024-08-30T09:16:29.679 INFO:teuthology.orchestra.run.smithi190.stdout: "req_getvxattr_latency": { 2024-08-30T09:16:29.679 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-30T09:16:29.679 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-30T09:16:29.679 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-30T09:16:29.680 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-08-30T09:16:29.680 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-30T09:16:29.680 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_sessions": { 2024-08-30T09:16:29.680 INFO:teuthology.orchestra.run.smithi190.stdout: "session_count": 1, 2024-08-30T09:16:29.680 INFO:teuthology.orchestra.run.smithi190.stdout: "session_add": 2, 2024-08-30T09:16:29.680 INFO:teuthology.orchestra.run.smithi190.stdout: "session_remove": 1, 2024-08-30T09:16:29.680 INFO:teuthology.orchestra.run.smithi190.stdout: "sessions_open": 1, 2024-08-30T09:16:29.680 INFO:teuthology.orchestra.run.smithi190.stdout: "sessions_stale": 0, 2024-08-30T09:16:29.680 INFO:teuthology.orchestra.run.smithi190.stdout: "total_load": 0, 2024-08-30T09:16:29.680 INFO:teuthology.orchestra.run.smithi190.stdout: "average_load": 0, 2024-08-30T09:16:29.680 INFO:teuthology.orchestra.run.smithi190.stdout: "avg_session_uptime": 1998, 2024-08-30T09:16:29.680 INFO:teuthology.orchestra.run.smithi190.stdout: "mdthresh_evicted": 0 2024-08-30T09:16:29.680 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-30T09:16:29.680 INFO:teuthology.orchestra.run.smithi190.stdout: "mempool": { 2024-08-30T09:16:29.681 INFO:teuthology.orchestra.run.smithi190.stdout: "bloom_filter_bytes": 0, 2024-08-30T09:16:29.681 INFO:teuthology.orchestra.run.smithi190.stdout: "bloom_filter_items": 0, 2024-08-30T09:16:29.681 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_alloc_bytes": 0, 2024-08-30T09:16:29.681 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_alloc_items": 0, 2024-08-30T09:16:29.681 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_data_bytes": 0, 2024-08-30T09:16:29.681 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_data_items": 0, 2024-08-30T09:16:29.681 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_onode_bytes": 0, 2024-08-30T09:16:29.681 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_onode_items": 0, 2024-08-30T09:16:29.681 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_meta_bytes": 0, 2024-08-30T09:16:29.681 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_meta_items": 0, 2024-08-30T09:16:29.681 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_other_bytes": 0, 2024-08-30T09:16:29.681 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_other_items": 0, 2024-08-30T09:16:29.681 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_buffer_bytes": 0, 2024-08-30T09:16:29.681 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_buffer_items": 0, 2024-08-30T09:16:29.682 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_extent_bytes": 0, 2024-08-30T09:16:29.682 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_extent_items": 0, 2024-08-30T09:16:29.682 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_blob_bytes": 0, 2024-08-30T09:16:29.682 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_blob_items": 0, 2024-08-30T09:16:29.682 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_shared_blob_bytes": 0, 2024-08-30T09:16:29.682 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_shared_blob_items": 0, 2024-08-30T09:16:29.682 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_inline_bl_bytes": 0, 2024-08-30T09:16:29.682 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_inline_bl_items": 0, 2024-08-30T09:16:29.682 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_fsck_bytes": 0, 2024-08-30T09:16:29.682 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_fsck_items": 0, 2024-08-30T09:16:29.682 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_txc_bytes": 0, 2024-08-30T09:16:29.682 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_txc_items": 0, 2024-08-30T09:16:29.682 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_writing_deferred_bytes": 0, 2024-08-30T09:16:29.682 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_writing_deferred_items": 0, 2024-08-30T09:16:29.682 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_writing_bytes": 0, 2024-08-30T09:16:29.683 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_writing_items": 0, 2024-08-30T09:16:29.683 INFO:teuthology.orchestra.run.smithi190.stdout: "bluefs_bytes": 0, 2024-08-30T09:16:29.683 INFO:teuthology.orchestra.run.smithi190.stdout: "bluefs_items": 0, 2024-08-30T09:16:29.683 INFO:teuthology.orchestra.run.smithi190.stdout: "bluefs_file_reader_bytes": 0, 2024-08-30T09:16:29.683 INFO:teuthology.orchestra.run.smithi190.stdout: "bluefs_file_reader_items": 0, 2024-08-30T09:16:29.683 INFO:teuthology.orchestra.run.smithi190.stdout: "bluefs_file_writer_bytes": 0, 2024-08-30T09:16:29.683 INFO:teuthology.orchestra.run.smithi190.stdout: "bluefs_file_writer_items": 0, 2024-08-30T09:16:29.683 INFO:teuthology.orchestra.run.smithi190.stdout: "buffer_anon_bytes": 200439, 2024-08-30T09:16:29.683 INFO:teuthology.orchestra.run.smithi190.stdout: "buffer_anon_items": 544, 2024-08-30T09:16:29.683 INFO:teuthology.orchestra.run.smithi190.stdout: "buffer_meta_bytes": 0, 2024-08-30T09:16:29.683 INFO:teuthology.orchestra.run.smithi190.stdout: "buffer_meta_items": 0, 2024-08-30T09:16:29.683 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_bytes": 0, 2024-08-30T09:16:29.683 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_items": 0, 2024-08-30T09:16:29.683 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_mapbl_bytes": 0, 2024-08-30T09:16:29.684 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_mapbl_items": 0, 2024-08-30T09:16:29.684 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_pglog_bytes": 0, 2024-08-30T09:16:29.684 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_pglog_items": 0, 2024-08-30T09:16:29.684 INFO:teuthology.orchestra.run.smithi190.stdout: "osdmap_bytes": 8856, 2024-08-30T09:16:29.684 INFO:teuthology.orchestra.run.smithi190.stdout: "osdmap_items": 166, 2024-08-30T09:16:29.684 INFO:teuthology.orchestra.run.smithi190.stdout: "osdmap_mapping_bytes": 0, 2024-08-30T09:16:29.684 INFO:teuthology.orchestra.run.smithi190.stdout: "osdmap_mapping_items": 0, 2024-08-30T09:16:29.684 INFO:teuthology.orchestra.run.smithi190.stdout: "pgmap_bytes": 0, 2024-08-30T09:16:29.684 INFO:teuthology.orchestra.run.smithi190.stdout: "pgmap_items": 0, 2024-08-30T09:16:29.684 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_co_bytes": 1665005, 2024-08-30T09:16:29.684 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_co_items": 12992, 2024-08-30T09:16:29.684 INFO:teuthology.orchestra.run.smithi190.stdout: "unittest_1_bytes": 0, 2024-08-30T09:16:29.684 INFO:teuthology.orchestra.run.smithi190.stdout: "unittest_1_items": 0, 2024-08-30T09:16:29.684 INFO:teuthology.orchestra.run.smithi190.stdout: "unittest_2_bytes": 0, 2024-08-30T09:16:29.685 INFO:teuthology.orchestra.run.smithi190.stdout: "unittest_2_items": 0 2024-08-30T09:16:29.685 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-30T09:16:29.685 INFO:teuthology.orchestra.run.smithi190.stdout: "objecter": { 2024-08-30T09:16:29.685 INFO:teuthology.orchestra.run.smithi190.stdout: "op_active": 0, 2024-08-30T09:16:29.685 INFO:teuthology.orchestra.run.smithi190.stdout: "op_laggy": 0, 2024-08-30T09:16:29.685 INFO:teuthology.orchestra.run.smithi190.stdout: "op_send": 3290, 2024-08-30T09:16:29.685 INFO:teuthology.orchestra.run.smithi190.stdout: "op_send_bytes": 21, 2024-08-30T09:16:29.685 INFO:teuthology.orchestra.run.smithi190.stdout: "op_resend": 3, 2024-08-30T09:16:29.685 INFO:teuthology.orchestra.run.smithi190.stdout: "op_reply": 3290, 2024-08-30T09:16:29.685 INFO:teuthology.orchestra.run.smithi190.stdout: "op_latency": { 2024-08-30T09:16:29.685 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 3290, 2024-08-30T09:16:29.685 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 41.922816632, 2024-08-30T09:16:29.685 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.012742497 2024-08-30T09:16:29.686 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-30T09:16:29.686 INFO:teuthology.orchestra.run.smithi190.stdout: "op_inflight": 0, 2024-08-30T09:16:29.686 INFO:teuthology.orchestra.run.smithi190.stdout: "oplen_avg": { 2024-08-30T09:16:29.686 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 3290, 2024-08-30T09:16:29.686 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 3291 2024-08-30T09:16:29.686 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-30T09:16:29.686 INFO:teuthology.orchestra.run.smithi190.stdout: "op": 3290, 2024-08-30T09:16:29.686 INFO:teuthology.orchestra.run.smithi190.stdout: "op_r": 3290, 2024-08-30T09:16:29.686 INFO:teuthology.orchestra.run.smithi190.stdout: "op_w": 0, 2024-08-30T09:16:29.686 INFO:teuthology.orchestra.run.smithi190.stdout: "op_rmw": 0, 2024-08-30T09:16:29.686 INFO:teuthology.orchestra.run.smithi190.stdout: "op_pg": 0, 2024-08-30T09:16:29.686 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_stat": 1987, 2024-08-30T09:16:29.686 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_create": 0, 2024-08-30T09:16:29.686 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_read": 1301, 2024-08-30T09:16:29.686 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_write": 0, 2024-08-30T09:16:29.687 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_writefull": 0, 2024-08-30T09:16:29.687 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_writesame": 0, 2024-08-30T09:16:29.687 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_append": 0, 2024-08-30T09:16:29.687 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_zero": 0, 2024-08-30T09:16:29.687 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_truncate": 0, 2024-08-30T09:16:29.687 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_delete": 0, 2024-08-30T09:16:29.687 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_mapext": 0, 2024-08-30T09:16:29.687 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_sparse_read": 0, 2024-08-30T09:16:29.687 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_clonerange": 0, 2024-08-30T09:16:29.687 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_getxattr": 1, 2024-08-30T09:16:29.687 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_setxattr": 0, 2024-08-30T09:16:29.687 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_cmpxattr": 0, 2024-08-30T09:16:29.687 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_rmxattr": 0, 2024-08-30T09:16:29.687 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_resetxattrs": 0, 2024-08-30T09:16:29.688 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_call": 0, 2024-08-30T09:16:29.688 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_watch": 0, 2024-08-30T09:16:29.688 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_notify": 0, 2024-08-30T09:16:29.688 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_src_cmpxattr": 0, 2024-08-30T09:16:29.688 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_pgls": 0, 2024-08-30T09:16:29.688 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_pgls_filter": 0, 2024-08-30T09:16:29.688 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_other": 0, 2024-08-30T09:16:29.688 INFO:teuthology.orchestra.run.smithi190.stdout: "linger_active": 0, 2024-08-30T09:16:29.688 INFO:teuthology.orchestra.run.smithi190.stdout: "linger_send": 0, 2024-08-30T09:16:29.688 INFO:teuthology.orchestra.run.smithi190.stdout: "linger_resend": 0, 2024-08-30T09:16:29.688 INFO:teuthology.orchestra.run.smithi190.stdout: "linger_ping": 0, 2024-08-30T09:16:29.688 INFO:teuthology.orchestra.run.smithi190.stdout: "poolop_active": 0, 2024-08-30T09:16:29.688 INFO:teuthology.orchestra.run.smithi190.stdout: "poolop_send": 0, 2024-08-30T09:16:29.689 INFO:teuthology.orchestra.run.smithi190.stdout: "poolop_resend": 0, 2024-08-30T09:16:29.689 INFO:teuthology.orchestra.run.smithi190.stdout: "poolstat_active": 0, 2024-08-30T09:16:29.689 INFO:teuthology.orchestra.run.smithi190.stdout: "poolstat_send": 0, 2024-08-30T09:16:29.689 INFO:teuthology.orchestra.run.smithi190.stdout: "poolstat_resend": 0, 2024-08-30T09:16:29.689 INFO:teuthology.orchestra.run.smithi190.stdout: "statfs_active": 0, 2024-08-30T09:16:29.689 INFO:teuthology.orchestra.run.smithi190.stdout: "statfs_send": 0, 2024-08-30T09:16:29.689 INFO:teuthology.orchestra.run.smithi190.stdout: "statfs_resend": 0, 2024-08-30T09:16:29.689 INFO:teuthology.orchestra.run.smithi190.stdout: "command_active": 0, 2024-08-30T09:16:29.689 INFO:teuthology.orchestra.run.smithi190.stdout: "command_send": 0, 2024-08-30T09:16:29.689 INFO:teuthology.orchestra.run.smithi190.stdout: "command_resend": 0, 2024-08-30T09:16:29.689 INFO:teuthology.orchestra.run.smithi190.stdout: "map_epoch": 83, 2024-08-30T09:16:29.689 INFO:teuthology.orchestra.run.smithi190.stdout: "map_full": 0, 2024-08-30T09:16:29.689 INFO:teuthology.orchestra.run.smithi190.stdout: "map_inc": 2, 2024-08-30T09:16:29.690 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_sessions": 4, 2024-08-30T09:16:29.690 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_session_open": 6, 2024-08-30T09:16:29.690 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_session_close": 2, 2024-08-30T09:16:29.690 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_laggy": 0, 2024-08-30T09:16:29.690 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_wr": 0, 2024-08-30T09:16:29.690 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_rd": 2, 2024-08-30T09:16:29.690 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_del": 0 2024-08-30T09:16:29.690 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-30T09:16:29.690 INFO:teuthology.orchestra.run.smithi190.stdout: "oft": { 2024-08-30T09:16:29.690 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_total_objs": 0, 2024-08-30T09:16:29.690 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_total_kv_pairs": 0, 2024-08-30T09:16:29.690 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_total_updates": 0, 2024-08-30T09:16:29.690 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_total_removes": 0 2024-08-30T09:16:29.690 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-30T09:16:29.691 INFO:teuthology.orchestra.run.smithi190.stdout: "purge_queue": { 2024-08-30T09:16:29.691 INFO:teuthology.orchestra.run.smithi190.stdout: "pq_executing_ops": 0, 2024-08-30T09:16:29.691 INFO:teuthology.orchestra.run.smithi190.stdout: "pq_executing_ops_high_water": 0, 2024-08-30T09:16:29.691 INFO:teuthology.orchestra.run.smithi190.stdout: "pq_executing": 0, 2024-08-30T09:16:29.691 INFO:teuthology.orchestra.run.smithi190.stdout: "pq_executing_high_water": 0, 2024-08-30T09:16:29.691 INFO:teuthology.orchestra.run.smithi190.stdout: "pq_executed_ops": 0, 2024-08-30T09:16:29.691 INFO:teuthology.orchestra.run.smithi190.stdout: "pq_executed": 0, 2024-08-30T09:16:29.691 INFO:teuthology.orchestra.run.smithi190.stdout: "pq_item_in_journal": 0 2024-08-30T09:16:29.691 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-30T09:16:29.691 INFO:teuthology.orchestra.run.smithi190.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-08-30T09:16:29.691 INFO:teuthology.orchestra.run.smithi190.stdout: "val": 0, 2024-08-30T09:16:29.691 INFO:teuthology.orchestra.run.smithi190.stdout: "max": 104857600, 2024-08-30T09:16:29.691 INFO:teuthology.orchestra.run.smithi190.stdout: "get_started": 0, 2024-08-30T09:16:29.691 INFO:teuthology.orchestra.run.smithi190.stdout: "get": 3630, 2024-08-30T09:16:29.691 INFO:teuthology.orchestra.run.smithi190.stdout: "get_sum": 5713238, 2024-08-30T09:16:29.692 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_fail": 0, 2024-08-30T09:16:29.692 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_success": 3630, 2024-08-30T09:16:29.692 INFO:teuthology.orchestra.run.smithi190.stdout: "take": 0, 2024-08-30T09:16:29.692 INFO:teuthology.orchestra.run.smithi190.stdout: "take_sum": 0, 2024-08-30T09:16:29.692 INFO:teuthology.orchestra.run.smithi190.stdout: "put": 3630, 2024-08-30T09:16:29.692 INFO:teuthology.orchestra.run.smithi190.stdout: "put_sum": 5713238, 2024-08-30T09:16:29.692 INFO:teuthology.orchestra.run.smithi190.stdout: "wait": { 2024-08-30T09:16:29.692 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-30T09:16:29.692 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-30T09:16:29.692 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-30T09:16:29.692 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-08-30T09:16:29.692 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-30T09:16:29.692 INFO:teuthology.orchestra.run.smithi190.stdout: "throttle-objecter_bytes": { 2024-08-30T09:16:29.692 INFO:teuthology.orchestra.run.smithi190.stdout: "val": 0, 2024-08-30T09:16:29.693 INFO:teuthology.orchestra.run.smithi190.stdout: "max": 104857600, 2024-08-30T09:16:29.693 INFO:teuthology.orchestra.run.smithi190.stdout: "get_started": 0, 2024-08-30T09:16:29.693 INFO:teuthology.orchestra.run.smithi190.stdout: "get": 0, 2024-08-30T09:16:29.693 INFO:teuthology.orchestra.run.smithi190.stdout: "get_sum": 0, 2024-08-30T09:16:29.693 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_fail": 0, 2024-08-30T09:16:29.693 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_success": 0, 2024-08-30T09:16:29.693 INFO:teuthology.orchestra.run.smithi190.stdout: "take": 3290, 2024-08-30T09:16:29.693 INFO:teuthology.orchestra.run.smithi190.stdout: "take_sum": 4465315, 2024-08-30T09:16:29.693 INFO:teuthology.orchestra.run.smithi190.stdout: "put": 3290, 2024-08-30T09:16:29.693 INFO:teuthology.orchestra.run.smithi190.stdout: "put_sum": 4465315, 2024-08-30T09:16:29.693 INFO:teuthology.orchestra.run.smithi190.stdout: "wait": { 2024-08-30T09:16:29.693 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-30T09:16:29.693 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-30T09:16:29.693 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-30T09:16:29.694 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-08-30T09:16:29.694 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-30T09:16:29.694 INFO:teuthology.orchestra.run.smithi190.stdout: "throttle-objecter_ops": { 2024-08-30T09:16:29.694 INFO:teuthology.orchestra.run.smithi190.stdout: "val": 0, 2024-08-30T09:16:29.694 INFO:teuthology.orchestra.run.smithi190.stdout: "max": 1024, 2024-08-30T09:16:29.694 INFO:teuthology.orchestra.run.smithi190.stdout: "get_started": 0, 2024-08-30T09:16:29.694 INFO:teuthology.orchestra.run.smithi190.stdout: "get": 0, 2024-08-30T09:16:29.694 INFO:teuthology.orchestra.run.smithi190.stdout: "get_sum": 0, 2024-08-30T09:16:29.694 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_fail": 0, 2024-08-30T09:16:29.694 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_success": 0, 2024-08-30T09:16:29.694 INFO:teuthology.orchestra.run.smithi190.stdout: "take": 3290, 2024-08-30T09:16:29.694 INFO:teuthology.orchestra.run.smithi190.stdout: "take_sum": 3290, 2024-08-30T09:16:29.694 INFO:teuthology.orchestra.run.smithi190.stdout: "put": 3290, 2024-08-30T09:16:29.694 INFO:teuthology.orchestra.run.smithi190.stdout: "put_sum": 3290, 2024-08-30T09:16:29.695 INFO:teuthology.orchestra.run.smithi190.stdout: "wait": { 2024-08-30T09:16:29.695 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-30T09:16:29.695 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-30T09:16:29.695 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-30T09:16:29.695 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-08-30T09:16:29.695 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-30T09:16:29.695 INFO:teuthology.orchestra.run.smithi190.stdout: "throttle-write_buf_throttle": { 2024-08-30T09:16:29.695 INFO:teuthology.orchestra.run.smithi190.stdout: "val": 0, 2024-08-30T09:16:29.695 INFO:teuthology.orchestra.run.smithi190.stdout: "max": 3758096384, 2024-08-30T09:16:29.695 INFO:teuthology.orchestra.run.smithi190.stdout: "get_started": 0, 2024-08-30T09:16:29.695 INFO:teuthology.orchestra.run.smithi190.stdout: "get": 0, 2024-08-30T09:16:29.695 INFO:teuthology.orchestra.run.smithi190.stdout: "get_sum": 0, 2024-08-30T09:16:29.695 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_fail": 0, 2024-08-30T09:16:29.695 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_success": 0, 2024-08-30T09:16:29.696 INFO:teuthology.orchestra.run.smithi190.stdout: "take": 0, 2024-08-30T09:16:29.696 INFO:teuthology.orchestra.run.smithi190.stdout: "take_sum": 0, 2024-08-30T09:16:29.696 INFO:teuthology.orchestra.run.smithi190.stdout: "put": 0, 2024-08-30T09:16:29.696 INFO:teuthology.orchestra.run.smithi190.stdout: "put_sum": 0, 2024-08-30T09:16:29.696 INFO:teuthology.orchestra.run.smithi190.stdout: "wait": { 2024-08-30T09:16:29.696 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-30T09:16:29.696 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-30T09:16:29.696 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-30T09:16:29.696 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-08-30T09:16:29.696 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-30T09:16:29.696 INFO:teuthology.orchestra.run.smithi190.stdout: "throttle-write_buf_throttle-0x557a6a670c30": { 2024-08-30T09:16:29.696 INFO:teuthology.orchestra.run.smithi190.stdout: "val": 0, 2024-08-30T09:16:29.696 INFO:teuthology.orchestra.run.smithi190.stdout: "max": 3758096384, 2024-08-30T09:16:29.697 INFO:teuthology.orchestra.run.smithi190.stdout: "get_started": 0, 2024-08-30T09:16:29.697 INFO:teuthology.orchestra.run.smithi190.stdout: "get": 0, 2024-08-30T09:16:29.697 INFO:teuthology.orchestra.run.smithi190.stdout: "get_sum": 0, 2024-08-30T09:16:29.697 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_fail": 0, 2024-08-30T09:16:29.697 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_success": 0, 2024-08-30T09:16:29.697 INFO:teuthology.orchestra.run.smithi190.stdout: "take": 0, 2024-08-30T09:16:29.697 INFO:teuthology.orchestra.run.smithi190.stdout: "take_sum": 0, 2024-08-30T09:16:29.697 INFO:teuthology.orchestra.run.smithi190.stdout: "put": 0, 2024-08-30T09:16:29.697 INFO:teuthology.orchestra.run.smithi190.stdout: "put_sum": 0, 2024-08-30T09:16:29.697 INFO:teuthology.orchestra.run.smithi190.stdout: "wait": { 2024-08-30T09:16:29.697 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-30T09:16:29.697 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-30T09:16:29.697 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-30T09:16:29.698 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-08-30T09:16:29.698 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-30T09:16:29.698 INFO:teuthology.orchestra.run.smithi190.stdout: "trackedop": { 2024-08-30T09:16:29.698 INFO:teuthology.orchestra.run.smithi190.stdout: "slow_ops_count": 0 2024-08-30T09:16:29.698 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-08-30T09:16:29.698 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-08-30T09:16:30.647 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-08-30T09:16:30.647 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-08-30T09:16:30.647 DEBUG:tasks.check_counter:Getting stats from i 2024-08-30T09:16:30.648 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph daemon mds.i perf dump 2024-08-30T09:16:30.894 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.c/config 2024-08-30T09:16:31.331 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:16:31 smithi190 ceph-mon[22001]: pgmap v845: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.7 KiB/s wr, 9 op/s 2024-08-30T09:16:31.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:16:31 smithi042 ceph-mon[18652]: pgmap v845: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.7 KiB/s wr, 9 op/s 2024-08-30T09:16:31.465 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-08-30T09:16:31.466 INFO:teuthology.orchestra.run.smithi190.stdout: "AsyncMessenger::Worker-0": { 2024-08-30T09:16:31.466 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_messages": 7055, 2024-08-30T09:16:31.466 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_messages": 5663, 2024-08-30T09:16:31.466 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_bytes": 3296810, 2024-08-30T09:16:31.466 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_bytes": 7247154, 2024-08-30T09:16:31.466 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_created_connections": 45, 2024-08-30T09:16:31.466 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_active_connections": 4, 2024-08-30T09:16:31.466 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_total_time": 0.568019671, 2024-08-30T09:16:31.466 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_send_time": 0.232680300, 2024-08-30T09:16:31.466 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_recv_time": 0.292559349, 2024-08-30T09:16:31.466 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_fast_dispatch_time": 0.086194435, 2024-08-30T09:16:31.467 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_messages_queue_lat": { 2024-08-30T09:16:31.467 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 5661, 2024-08-30T09:16:31.467 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.516823103, 2024-08-30T09:16:31.467 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000091295 2024-08-30T09:16:31.467 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-30T09:16:31.467 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_handle_ack_lat": { 2024-08-30T09:16:31.467 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 6756, 2024-08-30T09:16:31.467 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.001454392, 2024-08-30T09:16:31.467 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000215 2024-08-30T09:16:31.467 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-30T09:16:31.467 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_encrypted_bytes": 0, 2024-08-30T09:16:31.467 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_encrypted_bytes": 0 2024-08-30T09:16:31.467 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-30T09:16:31.467 INFO:teuthology.orchestra.run.smithi190.stdout: "AsyncMessenger::Worker-1": { 2024-08-30T09:16:31.468 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_messages": 6449, 2024-08-30T09:16:31.468 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_messages": 8467, 2024-08-30T09:16:31.468 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_bytes": 2440312, 2024-08-30T09:16:31.468 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_bytes": 4755820, 2024-08-30T09:16:31.468 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_created_connections": 14, 2024-08-30T09:16:31.468 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_active_connections": 6, 2024-08-30T09:16:31.468 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_total_time": 0.696339439, 2024-08-30T09:16:31.468 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_send_time": 0.335272673, 2024-08-30T09:16:31.468 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_recv_time": 0.342746211, 2024-08-30T09:16:31.468 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_fast_dispatch_time": 0.084749933, 2024-08-30T09:16:31.468 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_messages_queue_lat": { 2024-08-30T09:16:31.468 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 8466, 2024-08-30T09:16:31.468 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.893940749, 2024-08-30T09:16:31.469 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000105591 2024-08-30T09:16:31.469 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-30T09:16:31.469 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_handle_ack_lat": { 2024-08-30T09:16:31.469 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 9537, 2024-08-30T09:16:31.469 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.002174927, 2024-08-30T09:16:31.469 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000228 2024-08-30T09:16:31.469 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-30T09:16:31.469 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_encrypted_bytes": 1863616, 2024-08-30T09:16:31.469 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_encrypted_bytes": 4331488 2024-08-30T09:16:31.469 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-30T09:16:31.469 INFO:teuthology.orchestra.run.smithi190.stdout: "AsyncMessenger::Worker-2": { 2024-08-30T09:16:31.469 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_messages": 3304, 2024-08-30T09:16:31.469 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_messages": 3435, 2024-08-30T09:16:31.470 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_bytes": 2109299, 2024-08-30T09:16:31.470 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_bytes": 1413513, 2024-08-30T09:16:31.470 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_created_connections": 9, 2024-08-30T09:16:31.470 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_active_connections": 6, 2024-08-30T09:16:31.470 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_total_time": 0.388758552, 2024-08-30T09:16:31.470 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_send_time": 0.143870911, 2024-08-30T09:16:31.470 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_recv_time": 0.099676205, 2024-08-30T09:16:31.470 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_fast_dispatch_time": 0.129460362, 2024-08-30T09:16:31.470 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_messages_queue_lat": { 2024-08-30T09:16:31.470 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 3434, 2024-08-30T09:16:31.470 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.472348984, 2024-08-30T09:16:31.470 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000137550 2024-08-30T09:16:31.470 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-30T09:16:31.470 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_handle_ack_lat": { 2024-08-30T09:16:31.471 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 741, 2024-08-30T09:16:31.471 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000172286, 2024-08-30T09:16:31.471 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000232 2024-08-30T09:16:31.471 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-30T09:16:31.471 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_encrypted_bytes": 696848, 2024-08-30T09:16:31.471 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_encrypted_bytes": 213776 2024-08-30T09:16:31.471 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-30T09:16:31.471 INFO:teuthology.orchestra.run.smithi190.stdout: "cct": { 2024-08-30T09:16:31.471 INFO:teuthology.orchestra.run.smithi190.stdout: "total_workers": 1, 2024-08-30T09:16:31.471 INFO:teuthology.orchestra.run.smithi190.stdout: "unhealthy_workers": 0 2024-08-30T09:16:31.471 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-30T09:16:31.471 INFO:teuthology.orchestra.run.smithi190.stdout: "finisher-MDSRank": { 2024-08-30T09:16:31.471 INFO:teuthology.orchestra.run.smithi190.stdout: "queue_len": 0, 2024-08-30T09:16:31.471 INFO:teuthology.orchestra.run.smithi190.stdout: "complete_latency": { 2024-08-30T09:16:31.471 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 4952, 2024-08-30T09:16:31.472 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 4.826806916, 2024-08-30T09:16:31.472 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000974718 2024-08-30T09:16:31.472 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-08-30T09:16:31.472 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-30T09:16:31.472 INFO:teuthology.orchestra.run.smithi190.stdout: "finisher-PurgeQueue": { 2024-08-30T09:16:31.472 INFO:teuthology.orchestra.run.smithi190.stdout: "queue_len": 0, 2024-08-30T09:16:31.472 INFO:teuthology.orchestra.run.smithi190.stdout: "complete_latency": { 2024-08-30T09:16:31.472 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 79, 2024-08-30T09:16:31.472 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.003498968, 2024-08-30T09:16:31.472 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000044290 2024-08-30T09:16:31.472 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-08-30T09:16:31.472 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-30T09:16:31.472 INFO:teuthology.orchestra.run.smithi190.stdout: "mds": { 2024-08-30T09:16:31.473 INFO:teuthology.orchestra.run.smithi190.stdout: "request": 1921, 2024-08-30T09:16:31.473 INFO:teuthology.orchestra.run.smithi190.stdout: "reply": 1908, 2024-08-30T09:16:31.473 INFO:teuthology.orchestra.run.smithi190.stdout: "reply_latency": { 2024-08-30T09:16:31.473 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 1908, 2024-08-30T09:16:31.473 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 107.193930684, 2024-08-30T09:16:31.473 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.056181305 2024-08-30T09:16:31.473 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-30T09:16:31.473 INFO:teuthology.orchestra.run.smithi190.stdout: "slow_reply": 0, 2024-08-30T09:16:31.473 INFO:teuthology.orchestra.run.smithi190.stdout: "forward": 13, 2024-08-30T09:16:31.473 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_fetch_complete": 0, 2024-08-30T09:16:31.473 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_fetch_keys": 0, 2024-08-30T09:16:31.473 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_commit": 11, 2024-08-30T09:16:31.473 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_split": 0, 2024-08-30T09:16:31.473 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_merge": 0, 2024-08-30T09:16:31.474 INFO:teuthology.orchestra.run.smithi190.stdout: "inodes": 700, 2024-08-30T09:16:31.474 INFO:teuthology.orchestra.run.smithi190.stdout: "inodes_top": 41, 2024-08-30T09:16:31.474 INFO:teuthology.orchestra.run.smithi190.stdout: "inodes_bottom": 18, 2024-08-30T09:16:31.474 INFO:teuthology.orchestra.run.smithi190.stdout: "inodes_pin_tail": 641, 2024-08-30T09:16:31.474 INFO:teuthology.orchestra.run.smithi190.stdout: "inodes_pinned": 641, 2024-08-30T09:16:31.474 INFO:teuthology.orchestra.run.smithi190.stdout: "inodes_expired": 895, 2024-08-30T09:16:31.474 INFO:teuthology.orchestra.run.smithi190.stdout: "inodes_with_caps": 500, 2024-08-30T09:16:31.474 INFO:teuthology.orchestra.run.smithi190.stdout: "caps": 500, 2024-08-30T09:16:31.474 INFO:teuthology.orchestra.run.smithi190.stdout: "subtrees": 14, 2024-08-30T09:16:31.474 INFO:teuthology.orchestra.run.smithi190.stdout: "traverse": 8625, 2024-08-30T09:16:31.474 INFO:teuthology.orchestra.run.smithi190.stdout: "traverse_hit": 7805, 2024-08-30T09:16:31.474 INFO:teuthology.orchestra.run.smithi190.stdout: "traverse_forward": 6, 2024-08-30T09:16:31.474 INFO:teuthology.orchestra.run.smithi190.stdout: "traverse_discover": 11, 2024-08-30T09:16:31.474 INFO:teuthology.orchestra.run.smithi190.stdout: "traverse_dir_fetch": 0, 2024-08-30T09:16:31.475 INFO:teuthology.orchestra.run.smithi190.stdout: "traverse_remote_ino": 0, 2024-08-30T09:16:31.475 INFO:teuthology.orchestra.run.smithi190.stdout: "traverse_lock": 227, 2024-08-30T09:16:31.475 INFO:teuthology.orchestra.run.smithi190.stdout: "load_cent": 3270, 2024-08-30T09:16:31.475 INFO:teuthology.orchestra.run.smithi190.stdout: "q": 2, 2024-08-30T09:16:31.475 INFO:teuthology.orchestra.run.smithi190.stdout: "exported": 13, 2024-08-30T09:16:31.475 INFO:teuthology.orchestra.run.smithi190.stdout: "exported_inodes": 928, 2024-08-30T09:16:31.475 INFO:teuthology.orchestra.run.smithi190.stdout: "imported": 21, 2024-08-30T09:16:31.475 INFO:teuthology.orchestra.run.smithi190.stdout: "imported_inodes": 1274, 2024-08-30T09:16:31.475 INFO:teuthology.orchestra.run.smithi190.stdout: "openino_dir_fetch": 0, 2024-08-30T09:16:31.475 INFO:teuthology.orchestra.run.smithi190.stdout: "openino_backtrace_fetch": 4634, 2024-08-30T09:16:31.475 INFO:teuthology.orchestra.run.smithi190.stdout: "openino_peer_discover": 0, 2024-08-30T09:16:31.475 INFO:teuthology.orchestra.run.smithi190.stdout: "root_rfiles": 1023, 2024-08-30T09:16:31.475 INFO:teuthology.orchestra.run.smithi190.stdout: "root_rbytes": 1605055526, 2024-08-30T09:16:31.476 INFO:teuthology.orchestra.run.smithi190.stdout: "root_rsnaps": 0, 2024-08-30T09:16:31.476 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_backtrace_fetch": 4634, 2024-08-30T09:16:31.476 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_set_tag": 0, 2024-08-30T09:16:31.476 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_backtrace_repaired": 0, 2024-08-30T09:16:31.476 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_inotable_repaired": 0, 2024-08-30T09:16:31.476 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_dir_inodes": 89, 2024-08-30T09:16:31.476 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_dir_base_inodes": 0, 2024-08-30T09:16:31.476 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_dirfrag_rstats": 89, 2024-08-30T09:16:31.476 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_file_inodes": 4545, 2024-08-30T09:16:31.476 INFO:teuthology.orchestra.run.smithi190.stdout: "handle_inode_file_caps": 0, 2024-08-30T09:16:31.476 INFO:teuthology.orchestra.run.smithi190.stdout: "ceph_cap_op_revoke": 1994, 2024-08-30T09:16:31.476 INFO:teuthology.orchestra.run.smithi190.stdout: "ceph_cap_op_grant": 1531, 2024-08-30T09:16:31.476 INFO:teuthology.orchestra.run.smithi190.stdout: "ceph_cap_op_trunc": 0, 2024-08-30T09:16:31.476 INFO:teuthology.orchestra.run.smithi190.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-08-30T09:16:31.477 INFO:teuthology.orchestra.run.smithi190.stdout: "ceph_cap_op_flush_ack": 0, 2024-08-30T09:16:31.477 INFO:teuthology.orchestra.run.smithi190.stdout: "handle_client_caps": 2856, 2024-08-30T09:16:31.477 INFO:teuthology.orchestra.run.smithi190.stdout: "handle_client_caps_dirty": 657, 2024-08-30T09:16:31.477 INFO:teuthology.orchestra.run.smithi190.stdout: "handle_client_cap_release": 68, 2024-08-30T09:16:31.477 INFO:teuthology.orchestra.run.smithi190.stdout: "process_request_cap_release": 677 2024-08-30T09:16:31.477 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-30T09:16:31.477 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_cache": { 2024-08-30T09:16:31.477 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_update": 0, 2024-08-30T09:16:31.477 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_update_receipt": 0, 2024-08-30T09:16:31.477 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_try_discover": 0, 2024-08-30T09:16:31.477 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_send_discover": 54, 2024-08-30T09:16:31.477 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_handle_discover": 9, 2024-08-30T09:16:31.477 INFO:teuthology.orchestra.run.smithi190.stdout: "num_strays": 0, 2024-08-30T09:16:31.478 INFO:teuthology.orchestra.run.smithi190.stdout: "num_strays_delayed": 0, 2024-08-30T09:16:31.478 INFO:teuthology.orchestra.run.smithi190.stdout: "num_strays_enqueuing": 0, 2024-08-30T09:16:31.478 INFO:teuthology.orchestra.run.smithi190.stdout: "strays_created": 156, 2024-08-30T09:16:31.478 INFO:teuthology.orchestra.run.smithi190.stdout: "strays_enqueued": 15, 2024-08-30T09:16:31.478 INFO:teuthology.orchestra.run.smithi190.stdout: "strays_reintegrated": 141, 2024-08-30T09:16:31.478 INFO:teuthology.orchestra.run.smithi190.stdout: "strays_migrated": 0, 2024-08-30T09:16:31.478 INFO:teuthology.orchestra.run.smithi190.stdout: "num_recovering_processing": 0, 2024-08-30T09:16:31.478 INFO:teuthology.orchestra.run.smithi190.stdout: "num_recovering_enqueued": 0, 2024-08-30T09:16:31.478 INFO:teuthology.orchestra.run.smithi190.stdout: "num_recovering_prioritized": 0, 2024-08-30T09:16:31.478 INFO:teuthology.orchestra.run.smithi190.stdout: "recovery_started": 0, 2024-08-30T09:16:31.478 INFO:teuthology.orchestra.run.smithi190.stdout: "recovery_completed": 0, 2024-08-30T09:16:31.478 INFO:teuthology.orchestra.run.smithi190.stdout: "ireq_quiesce_path": 0, 2024-08-30T09:16:31.478 INFO:teuthology.orchestra.run.smithi190.stdout: "ireq_quiesce_inode": 0, 2024-08-30T09:16:31.478 INFO:teuthology.orchestra.run.smithi190.stdout: "ireq_enqueue_scrub": 0, 2024-08-30T09:16:31.478 INFO:teuthology.orchestra.run.smithi190.stdout: "ireq_exportdir": 13, 2024-08-30T09:16:31.479 INFO:teuthology.orchestra.run.smithi190.stdout: "ireq_flush": 0, 2024-08-30T09:16:31.479 INFO:teuthology.orchestra.run.smithi190.stdout: "ireq_fragmentdir": 0, 2024-08-30T09:16:31.479 INFO:teuthology.orchestra.run.smithi190.stdout: "ireq_fragstats": 0, 2024-08-30T09:16:31.479 INFO:teuthology.orchestra.run.smithi190.stdout: "ireq_inodestats": 0, 2024-08-30T09:16:31.479 INFO:teuthology.orchestra.run.smithi190.stdout: "uninline_started": 0, 2024-08-30T09:16:31.479 INFO:teuthology.orchestra.run.smithi190.stdout: "uninline_succeeded": 0, 2024-08-30T09:16:31.479 INFO:teuthology.orchestra.run.smithi190.stdout: "uninline_write_failed": 0 2024-08-30T09:16:31.479 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-30T09:16:31.479 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_log": { 2024-08-30T09:16:31.479 INFO:teuthology.orchestra.run.smithi190.stdout: "evlrg": 0, 2024-08-30T09:16:31.479 INFO:teuthology.orchestra.run.smithi190.stdout: "evadd": 2458, 2024-08-30T09:16:31.479 INFO:teuthology.orchestra.run.smithi190.stdout: "evex": 0, 2024-08-30T09:16:31.479 INFO:teuthology.orchestra.run.smithi190.stdout: "evtrm": 0, 2024-08-30T09:16:31.479 INFO:teuthology.orchestra.run.smithi190.stdout: "ev": 2458, 2024-08-30T09:16:31.480 INFO:teuthology.orchestra.run.smithi190.stdout: "evexg": 0, 2024-08-30T09:16:31.480 INFO:teuthology.orchestra.run.smithi190.stdout: "evexd": 0, 2024-08-30T09:16:31.480 INFO:teuthology.orchestra.run.smithi190.stdout: "segadd": 5, 2024-08-30T09:16:31.480 INFO:teuthology.orchestra.run.smithi190.stdout: "segex": 0, 2024-08-30T09:16:31.480 INFO:teuthology.orchestra.run.smithi190.stdout: "segtrm": 0, 2024-08-30T09:16:31.480 INFO:teuthology.orchestra.run.smithi190.stdout: "seg": 5, 2024-08-30T09:16:31.480 INFO:teuthology.orchestra.run.smithi190.stdout: "segmjr": 2, 2024-08-30T09:16:31.480 INFO:teuthology.orchestra.run.smithi190.stdout: "segexg": 0, 2024-08-30T09:16:31.480 INFO:teuthology.orchestra.run.smithi190.stdout: "segexd": 0, 2024-08-30T09:16:31.480 INFO:teuthology.orchestra.run.smithi190.stdout: "expos": 4194304, 2024-08-30T09:16:31.480 INFO:teuthology.orchestra.run.smithi190.stdout: "wrpos": 8659614, 2024-08-30T09:16:31.480 INFO:teuthology.orchestra.run.smithi190.stdout: "rdpos": 0, 2024-08-30T09:16:31.481 INFO:teuthology.orchestra.run.smithi190.stdout: "jlat": { 2024-08-30T09:16:31.481 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 1548, 2024-08-30T09:16:31.481 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 37.809508866, 2024-08-30T09:16:31.481 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.024424747 2024-08-30T09:16:31.481 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-30T09:16:31.481 INFO:teuthology.orchestra.run.smithi190.stdout: "replayed": 0 2024-08-30T09:16:31.481 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-30T09:16:31.481 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_mem": { 2024-08-30T09:16:31.481 INFO:teuthology.orchestra.run.smithi190.stdout: "ino": 579, 2024-08-30T09:16:31.481 INFO:teuthology.orchestra.run.smithi190.stdout: "ino+": 1310, 2024-08-30T09:16:31.481 INFO:teuthology.orchestra.run.smithi190.stdout: "ino-": 731, 2024-08-30T09:16:31.481 INFO:teuthology.orchestra.run.smithi190.stdout: "dir": 51, 2024-08-30T09:16:31.481 INFO:teuthology.orchestra.run.smithi190.stdout: "dir+": 81, 2024-08-30T09:16:31.481 INFO:teuthology.orchestra.run.smithi190.stdout: "dir-": 30, 2024-08-30T09:16:31.482 INFO:teuthology.orchestra.run.smithi190.stdout: "dn": 702, 2024-08-30T09:16:31.482 INFO:teuthology.orchestra.run.smithi190.stdout: "dn+": 1626, 2024-08-30T09:16:31.482 INFO:teuthology.orchestra.run.smithi190.stdout: "dn-": 924, 2024-08-30T09:16:31.482 INFO:teuthology.orchestra.run.smithi190.stdout: "cap": 500, 2024-08-30T09:16:31.482 INFO:teuthology.orchestra.run.smithi190.stdout: "cap+": 935, 2024-08-30T09:16:31.482 INFO:teuthology.orchestra.run.smithi190.stdout: "cap-": 435, 2024-08-30T09:16:31.482 INFO:teuthology.orchestra.run.smithi190.stdout: "rss": 128748, 2024-08-30T09:16:31.482 INFO:teuthology.orchestra.run.smithi190.stdout: "heap": 335425536 2024-08-30T09:16:31.482 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-30T09:16:31.482 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_server": { 2024-08-30T09:16:31.482 INFO:teuthology.orchestra.run.smithi190.stdout: "dispatch_client_request": 2648, 2024-08-30T09:16:31.482 INFO:teuthology.orchestra.run.smithi190.stdout: "dispatch_server_request": 5, 2024-08-30T09:16:31.482 INFO:teuthology.orchestra.run.smithi190.stdout: "handle_client_request": 1921, 2024-08-30T09:16:31.482 INFO:teuthology.orchestra.run.smithi190.stdout: "handle_client_session": 313, 2024-08-30T09:16:31.483 INFO:teuthology.orchestra.run.smithi190.stdout: "handle_peer_request": 57, 2024-08-30T09:16:31.483 INFO:teuthology.orchestra.run.smithi190.stdout: "req_create_latency": { 2024-08-30T09:16:31.483 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 58, 2024-08-30T09:16:31.483 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.254680036, 2024-08-30T09:16:31.483 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.004391035 2024-08-30T09:16:31.483 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-30T09:16:31.483 INFO:teuthology.orchestra.run.smithi190.stdout: "req_getattr_latency": { 2024-08-30T09:16:31.483 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 641, 2024-08-30T09:16:31.483 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 28.413487856, 2024-08-30T09:16:31.483 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.044326814 2024-08-30T09:16:31.483 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-30T09:16:31.483 INFO:teuthology.orchestra.run.smithi190.stdout: "req_getfilelock_latency": { 2024-08-30T09:16:31.483 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-30T09:16:31.483 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-30T09:16:31.484 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-30T09:16:31.484 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-30T09:16:31.484 INFO:teuthology.orchestra.run.smithi190.stdout: "req_link_latency": { 2024-08-30T09:16:31.484 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 141, 2024-08-30T09:16:31.484 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.325822810, 2024-08-30T09:16:31.484 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.002310800 2024-08-30T09:16:31.484 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-30T09:16:31.484 INFO:teuthology.orchestra.run.smithi190.stdout: "req_lookup_latency": { 2024-08-30T09:16:31.484 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 634, 2024-08-30T09:16:31.484 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 76.606155200, 2024-08-30T09:16:31.484 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.120829897 2024-08-30T09:16:31.484 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-30T09:16:31.484 INFO:teuthology.orchestra.run.smithi190.stdout: "req_lookuphash_latency": { 2024-08-30T09:16:31.484 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-30T09:16:31.485 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-30T09:16:31.485 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-30T09:16:31.485 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-30T09:16:31.485 INFO:teuthology.orchestra.run.smithi190.stdout: "req_lookupino_latency": { 2024-08-30T09:16:31.485 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-30T09:16:31.485 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-30T09:16:31.485 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-30T09:16:31.485 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-30T09:16:31.485 INFO:teuthology.orchestra.run.smithi190.stdout: "req_lookupname_latency": { 2024-08-30T09:16:31.485 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-30T09:16:31.485 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-30T09:16:31.485 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-30T09:16:31.485 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-30T09:16:31.486 INFO:teuthology.orchestra.run.smithi190.stdout: "req_lookupparent_latency": { 2024-08-30T09:16:31.486 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-30T09:16:31.486 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-30T09:16:31.486 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-30T09:16:31.486 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-30T09:16:31.486 INFO:teuthology.orchestra.run.smithi190.stdout: "req_lookupsnap_latency": { 2024-08-30T09:16:31.486 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-30T09:16:31.487 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-30T09:16:31.487 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-30T09:16:31.487 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-30T09:16:31.487 INFO:teuthology.orchestra.run.smithi190.stdout: "req_lssnap_latency": { 2024-08-30T09:16:31.487 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-30T09:16:31.487 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-30T09:16:31.487 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-30T09:16:31.487 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-30T09:16:31.487 INFO:teuthology.orchestra.run.smithi190.stdout: "req_mkdir_latency": { 2024-08-30T09:16:31.487 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-30T09:16:31.487 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-30T09:16:31.488 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-30T09:16:31.488 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-30T09:16:31.488 INFO:teuthology.orchestra.run.smithi190.stdout: "req_mknod_latency": { 2024-08-30T09:16:31.488 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-30T09:16:31.488 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-30T09:16:31.488 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-30T09:16:31.488 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-30T09:16:31.488 INFO:teuthology.orchestra.run.smithi190.stdout: "req_mksnap_latency": { 2024-08-30T09:16:31.488 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-30T09:16:31.488 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-30T09:16:31.488 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-30T09:16:31.488 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-30T09:16:31.488 INFO:teuthology.orchestra.run.smithi190.stdout: "req_open_latency": { 2024-08-30T09:16:31.489 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 115, 2024-08-30T09:16:31.489 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.148163009, 2024-08-30T09:16:31.489 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.001288373 2024-08-30T09:16:31.489 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-30T09:16:31.489 INFO:teuthology.orchestra.run.smithi190.stdout: "req_readdir_latency": { 2024-08-30T09:16:31.489 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 22, 2024-08-30T09:16:31.489 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.024688973, 2024-08-30T09:16:31.489 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.001122226 2024-08-30T09:16:31.489 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-30T09:16:31.489 INFO:teuthology.orchestra.run.smithi190.stdout: "req_rename_latency": { 2024-08-30T09:16:31.489 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 143, 2024-08-30T09:16:31.489 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 1.192517857, 2024-08-30T09:16:31.489 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.008339285 2024-08-30T09:16:31.490 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-30T09:16:31.490 INFO:teuthology.orchestra.run.smithi190.stdout: "req_renamesnap_latency": { 2024-08-30T09:16:31.490 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-30T09:16:31.490 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-30T09:16:31.490 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-30T09:16:31.490 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-30T09:16:31.490 INFO:teuthology.orchestra.run.smithi190.stdout: "req_snapdiff_latency": { 2024-08-30T09:16:31.490 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-30T09:16:31.490 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-30T09:16:31.490 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-30T09:16:31.490 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-30T09:16:31.490 INFO:teuthology.orchestra.run.smithi190.stdout: "req_rmdir_latency": { 2024-08-30T09:16:31.490 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-30T09:16:31.491 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-30T09:16:31.491 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-30T09:16:31.491 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-30T09:16:31.491 INFO:teuthology.orchestra.run.smithi190.stdout: "req_rmsnap_latency": { 2024-08-30T09:16:31.491 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-30T09:16:31.491 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-30T09:16:31.491 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-30T09:16:31.491 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-30T09:16:31.491 INFO:teuthology.orchestra.run.smithi190.stdout: "req_rmxattr_latency": { 2024-08-30T09:16:31.491 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-30T09:16:31.491 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-30T09:16:31.491 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-30T09:16:31.491 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-30T09:16:31.492 INFO:teuthology.orchestra.run.smithi190.stdout: "req_setattr_latency": { 2024-08-30T09:16:31.492 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-30T09:16:31.492 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-30T09:16:31.492 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-30T09:16:31.492 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-30T09:16:31.492 INFO:teuthology.orchestra.run.smithi190.stdout: "req_setdirlayout_latency": { 2024-08-30T09:16:31.492 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-30T09:16:31.492 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-30T09:16:31.492 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-30T09:16:31.492 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-30T09:16:31.492 INFO:teuthology.orchestra.run.smithi190.stdout: "req_setfilelock_latency": { 2024-08-30T09:16:31.492 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-30T09:16:31.492 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-30T09:16:31.493 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-30T09:16:31.493 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-30T09:16:31.493 INFO:teuthology.orchestra.run.smithi190.stdout: "req_setlayout_latency": { 2024-08-30T09:16:31.493 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-30T09:16:31.493 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-30T09:16:31.493 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-30T09:16:31.493 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-30T09:16:31.493 INFO:teuthology.orchestra.run.smithi190.stdout: "req_setxattr_latency": { 2024-08-30T09:16:31.493 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-30T09:16:31.493 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-30T09:16:31.493 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-30T09:16:31.494 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-30T09:16:31.494 INFO:teuthology.orchestra.run.smithi190.stdout: "req_symlink_latency": { 2024-08-30T09:16:31.494 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-30T09:16:31.494 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-30T09:16:31.494 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-30T09:16:31.494 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-30T09:16:31.494 INFO:teuthology.orchestra.run.smithi190.stdout: "req_unlink_latency": { 2024-08-30T09:16:31.494 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 154, 2024-08-30T09:16:31.494 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.228414943, 2024-08-30T09:16:31.494 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.001483213 2024-08-30T09:16:31.494 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-30T09:16:31.494 INFO:teuthology.orchestra.run.smithi190.stdout: "cap_revoke_eviction": 0, 2024-08-30T09:16:31.494 INFO:teuthology.orchestra.run.smithi190.stdout: "cap_acquisition_throttle": 0, 2024-08-30T09:16:31.495 INFO:teuthology.orchestra.run.smithi190.stdout: "req_getvxattr_latency": { 2024-08-30T09:16:31.495 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-30T09:16:31.495 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-30T09:16:31.495 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-30T09:16:31.495 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-08-30T09:16:31.495 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-30T09:16:31.495 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_sessions": { 2024-08-30T09:16:31.495 INFO:teuthology.orchestra.run.smithi190.stdout: "session_count": 1, 2024-08-30T09:16:31.495 INFO:teuthology.orchestra.run.smithi190.stdout: "session_add": 2, 2024-08-30T09:16:31.495 INFO:teuthology.orchestra.run.smithi190.stdout: "session_remove": 1, 2024-08-30T09:16:31.495 INFO:teuthology.orchestra.run.smithi190.stdout: "sessions_open": 1, 2024-08-30T09:16:31.495 INFO:teuthology.orchestra.run.smithi190.stdout: "sessions_stale": 0, 2024-08-30T09:16:31.496 INFO:teuthology.orchestra.run.smithi190.stdout: "total_load": 284, 2024-08-30T09:16:31.496 INFO:teuthology.orchestra.run.smithi190.stdout: "average_load": 284, 2024-08-30T09:16:31.496 INFO:teuthology.orchestra.run.smithi190.stdout: "avg_session_uptime": 2000, 2024-08-30T09:16:31.496 INFO:teuthology.orchestra.run.smithi190.stdout: "mdthresh_evicted": 0 2024-08-30T09:16:31.496 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-30T09:16:31.496 INFO:teuthology.orchestra.run.smithi190.stdout: "mempool": { 2024-08-30T09:16:31.496 INFO:teuthology.orchestra.run.smithi190.stdout: "bloom_filter_bytes": 0, 2024-08-30T09:16:31.496 INFO:teuthology.orchestra.run.smithi190.stdout: "bloom_filter_items": 0, 2024-08-30T09:16:31.496 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_alloc_bytes": 0, 2024-08-30T09:16:31.496 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_alloc_items": 0, 2024-08-30T09:16:31.496 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_data_bytes": 0, 2024-08-30T09:16:31.496 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_data_items": 0, 2024-08-30T09:16:31.496 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_onode_bytes": 0, 2024-08-30T09:16:31.497 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_onode_items": 0, 2024-08-30T09:16:31.497 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_meta_bytes": 0, 2024-08-30T09:16:31.497 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_meta_items": 0, 2024-08-30T09:16:31.497 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_other_bytes": 0, 2024-08-30T09:16:31.497 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_other_items": 0, 2024-08-30T09:16:31.497 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_buffer_bytes": 0, 2024-08-30T09:16:31.497 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_buffer_items": 0, 2024-08-30T09:16:31.497 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_extent_bytes": 0, 2024-08-30T09:16:31.497 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_extent_items": 0, 2024-08-30T09:16:31.497 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_blob_bytes": 0, 2024-08-30T09:16:31.497 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_blob_items": 0, 2024-08-30T09:16:31.497 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_shared_blob_bytes": 0, 2024-08-30T09:16:31.497 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_shared_blob_items": 0, 2024-08-30T09:16:31.498 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_inline_bl_bytes": 0, 2024-08-30T09:16:31.498 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_inline_bl_items": 0, 2024-08-30T09:16:31.498 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_fsck_bytes": 0, 2024-08-30T09:16:31.498 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_fsck_items": 0, 2024-08-30T09:16:31.498 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_txc_bytes": 0, 2024-08-30T09:16:31.498 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_txc_items": 0, 2024-08-30T09:16:31.498 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_writing_deferred_bytes": 0, 2024-08-30T09:16:31.498 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_writing_deferred_items": 0, 2024-08-30T09:16:31.498 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_writing_bytes": 0, 2024-08-30T09:16:31.498 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_writing_items": 0, 2024-08-30T09:16:31.498 INFO:teuthology.orchestra.run.smithi190.stdout: "bluefs_bytes": 0, 2024-08-30T09:16:31.498 INFO:teuthology.orchestra.run.smithi190.stdout: "bluefs_items": 0, 2024-08-30T09:16:31.499 INFO:teuthology.orchestra.run.smithi190.stdout: "bluefs_file_reader_bytes": 0, 2024-08-30T09:16:31.499 INFO:teuthology.orchestra.run.smithi190.stdout: "bluefs_file_reader_items": 0, 2024-08-30T09:16:31.499 INFO:teuthology.orchestra.run.smithi190.stdout: "bluefs_file_writer_bytes": 0, 2024-08-30T09:16:31.499 INFO:teuthology.orchestra.run.smithi190.stdout: "bluefs_file_writer_items": 0, 2024-08-30T09:16:31.499 INFO:teuthology.orchestra.run.smithi190.stdout: "buffer_anon_bytes": 320812, 2024-08-30T09:16:31.499 INFO:teuthology.orchestra.run.smithi190.stdout: "buffer_anon_items": 680, 2024-08-30T09:16:31.499 INFO:teuthology.orchestra.run.smithi190.stdout: "buffer_meta_bytes": 0, 2024-08-30T09:16:31.499 INFO:teuthology.orchestra.run.smithi190.stdout: "buffer_meta_items": 0, 2024-08-30T09:16:31.499 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_bytes": 0, 2024-08-30T09:16:31.499 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_items": 0, 2024-08-30T09:16:31.499 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_mapbl_bytes": 0, 2024-08-30T09:16:31.499 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_mapbl_items": 0, 2024-08-30T09:16:31.500 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_pglog_bytes": 0, 2024-08-30T09:16:31.500 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_pglog_items": 0, 2024-08-30T09:16:31.500 INFO:teuthology.orchestra.run.smithi190.stdout: "osdmap_bytes": 8856, 2024-08-30T09:16:31.500 INFO:teuthology.orchestra.run.smithi190.stdout: "osdmap_items": 166, 2024-08-30T09:16:31.500 INFO:teuthology.orchestra.run.smithi190.stdout: "osdmap_mapping_bytes": 0, 2024-08-30T09:16:31.500 INFO:teuthology.orchestra.run.smithi190.stdout: "osdmap_mapping_items": 0, 2024-08-30T09:16:31.500 INFO:teuthology.orchestra.run.smithi190.stdout: "pgmap_bytes": 0, 2024-08-30T09:16:31.500 INFO:teuthology.orchestra.run.smithi190.stdout: "pgmap_items": 0, 2024-08-30T09:16:31.500 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_co_bytes": 2087363, 2024-08-30T09:16:31.500 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_co_items": 28054, 2024-08-30T09:16:31.500 INFO:teuthology.orchestra.run.smithi190.stdout: "unittest_1_bytes": 0, 2024-08-30T09:16:31.500 INFO:teuthology.orchestra.run.smithi190.stdout: "unittest_1_items": 0, 2024-08-30T09:16:31.500 INFO:teuthology.orchestra.run.smithi190.stdout: "unittest_2_bytes": 0, 2024-08-30T09:16:31.500 INFO:teuthology.orchestra.run.smithi190.stdout: "unittest_2_items": 0 2024-08-30T09:16:31.501 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-30T09:16:31.501 INFO:teuthology.orchestra.run.smithi190.stdout: "objecter": { 2024-08-30T09:16:31.501 INFO:teuthology.orchestra.run.smithi190.stdout: "op_active": 0, 2024-08-30T09:16:31.501 INFO:teuthology.orchestra.run.smithi190.stdout: "op_laggy": 0, 2024-08-30T09:16:31.501 INFO:teuthology.orchestra.run.smithi190.stdout: "op_send": 6535, 2024-08-30T09:16:31.501 INFO:teuthology.orchestra.run.smithi190.stdout: "op_send_bytes": 4574589, 2024-08-30T09:16:31.501 INFO:teuthology.orchestra.run.smithi190.stdout: "op_resend": 28, 2024-08-30T09:16:31.501 INFO:teuthology.orchestra.run.smithi190.stdout: "op_reply": 6535, 2024-08-30T09:16:31.501 INFO:teuthology.orchestra.run.smithi190.stdout: "op_latency": { 2024-08-30T09:16:31.501 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 6535, 2024-08-30T09:16:31.501 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 75.230846395, 2024-08-30T09:16:31.501 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.011511988 2024-08-30T09:16:31.502 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-30T09:16:31.502 INFO:teuthology.orchestra.run.smithi190.stdout: "op_inflight": 0, 2024-08-30T09:16:31.502 INFO:teuthology.orchestra.run.smithi190.stdout: "oplen_avg": { 2024-08-30T09:16:31.502 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 6535, 2024-08-30T09:16:31.502 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 6767 2024-08-30T09:16:31.502 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-30T09:16:31.502 INFO:teuthology.orchestra.run.smithi190.stdout: "op": 6535, 2024-08-30T09:16:31.502 INFO:teuthology.orchestra.run.smithi190.stdout: "op_r": 4642, 2024-08-30T09:16:31.502 INFO:teuthology.orchestra.run.smithi190.stdout: "op_w": 1893, 2024-08-30T09:16:31.502 INFO:teuthology.orchestra.run.smithi190.stdout: "op_rmw": 0, 2024-08-30T09:16:31.502 INFO:teuthology.orchestra.run.smithi190.stdout: "op_pg": 0, 2024-08-30T09:16:31.502 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_stat": 0, 2024-08-30T09:16:31.502 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_create": 10, 2024-08-30T09:16:31.502 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_read": 8, 2024-08-30T09:16:31.502 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_write": 1556, 2024-08-30T09:16:31.503 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_writefull": 72, 2024-08-30T09:16:31.503 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_writesame": 0, 2024-08-30T09:16:31.503 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_append": 0, 2024-08-30T09:16:31.503 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_zero": 0, 2024-08-30T09:16:31.503 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_truncate": 0, 2024-08-30T09:16:31.503 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_delete": 49, 2024-08-30T09:16:31.503 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_mapext": 0, 2024-08-30T09:16:31.503 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_sparse_read": 0, 2024-08-30T09:16:31.503 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_clonerange": 0, 2024-08-30T09:16:31.503 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_getxattr": 4634, 2024-08-30T09:16:31.503 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_setxattr": 20, 2024-08-30T09:16:31.503 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_cmpxattr": 0, 2024-08-30T09:16:31.503 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_rmxattr": 0, 2024-08-30T09:16:31.503 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_resetxattrs": 0, 2024-08-30T09:16:31.503 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_call": 0, 2024-08-30T09:16:31.504 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_watch": 0, 2024-08-30T09:16:31.504 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_notify": 0, 2024-08-30T09:16:31.504 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_src_cmpxattr": 0, 2024-08-30T09:16:31.504 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_pgls": 0, 2024-08-30T09:16:31.504 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_pgls_filter": 0, 2024-08-30T09:16:31.504 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_other": 0, 2024-08-30T09:16:31.504 INFO:teuthology.orchestra.run.smithi190.stdout: "linger_active": 0, 2024-08-30T09:16:31.504 INFO:teuthology.orchestra.run.smithi190.stdout: "linger_send": 0, 2024-08-30T09:16:31.504 INFO:teuthology.orchestra.run.smithi190.stdout: "linger_resend": 0, 2024-08-30T09:16:31.504 INFO:teuthology.orchestra.run.smithi190.stdout: "linger_ping": 0, 2024-08-30T09:16:31.504 INFO:teuthology.orchestra.run.smithi190.stdout: "poolop_active": 0, 2024-08-30T09:16:31.504 INFO:teuthology.orchestra.run.smithi190.stdout: "poolop_send": 0, 2024-08-30T09:16:31.504 INFO:teuthology.orchestra.run.smithi190.stdout: "poolop_resend": 0, 2024-08-30T09:16:31.505 INFO:teuthology.orchestra.run.smithi190.stdout: "poolstat_active": 0, 2024-08-30T09:16:31.505 INFO:teuthology.orchestra.run.smithi190.stdout: "poolstat_send": 0, 2024-08-30T09:16:31.505 INFO:teuthology.orchestra.run.smithi190.stdout: "poolstat_resend": 0, 2024-08-30T09:16:31.505 INFO:teuthology.orchestra.run.smithi190.stdout: "statfs_active": 0, 2024-08-30T09:16:31.505 INFO:teuthology.orchestra.run.smithi190.stdout: "statfs_send": 0, 2024-08-30T09:16:31.505 INFO:teuthology.orchestra.run.smithi190.stdout: "statfs_resend": 0, 2024-08-30T09:16:31.505 INFO:teuthology.orchestra.run.smithi190.stdout: "command_active": 0, 2024-08-30T09:16:31.505 INFO:teuthology.orchestra.run.smithi190.stdout: "command_send": 0, 2024-08-30T09:16:31.505 INFO:teuthology.orchestra.run.smithi190.stdout: "command_resend": 0, 2024-08-30T09:16:31.505 INFO:teuthology.orchestra.run.smithi190.stdout: "map_epoch": 83, 2024-08-30T09:16:31.505 INFO:teuthology.orchestra.run.smithi190.stdout: "map_full": 0, 2024-08-30T09:16:31.505 INFO:teuthology.orchestra.run.smithi190.stdout: "map_inc": 4, 2024-08-30T09:16:31.505 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_sessions": 12, 2024-08-30T09:16:31.505 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_session_open": 12, 2024-08-30T09:16:31.506 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_session_close": 0, 2024-08-30T09:16:31.506 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_laggy": 0, 2024-08-30T09:16:31.506 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_wr": 354, 2024-08-30T09:16:31.506 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_rd": 0, 2024-08-30T09:16:31.506 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_del": 64 2024-08-30T09:16:31.506 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-30T09:16:31.506 INFO:teuthology.orchestra.run.smithi190.stdout: "oft": { 2024-08-30T09:16:31.506 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_total_objs": 1, 2024-08-30T09:16:31.506 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_total_kv_pairs": 0, 2024-08-30T09:16:31.506 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_total_updates": 0, 2024-08-30T09:16:31.506 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_total_removes": 0 2024-08-30T09:16:31.506 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-30T09:16:31.506 INFO:teuthology.orchestra.run.smithi190.stdout: "purge_queue": { 2024-08-30T09:16:31.506 INFO:teuthology.orchestra.run.smithi190.stdout: "pq_executing_ops": 0, 2024-08-30T09:16:31.506 INFO:teuthology.orchestra.run.smithi190.stdout: "pq_executing_ops_high_water": 16, 2024-08-30T09:16:31.507 INFO:teuthology.orchestra.run.smithi190.stdout: "pq_executing": 0, 2024-08-30T09:16:31.507 INFO:teuthology.orchestra.run.smithi190.stdout: "pq_executing_high_water": 8, 2024-08-30T09:16:31.507 INFO:teuthology.orchestra.run.smithi190.stdout: "pq_executed_ops": 36, 2024-08-30T09:16:31.507 INFO:teuthology.orchestra.run.smithi190.stdout: "pq_executed": 15, 2024-08-30T09:16:31.507 INFO:teuthology.orchestra.run.smithi190.stdout: "pq_item_in_journal": 0 2024-08-30T09:16:31.507 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-30T09:16:31.507 INFO:teuthology.orchestra.run.smithi190.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-08-30T09:16:31.507 INFO:teuthology.orchestra.run.smithi190.stdout: "val": 0, 2024-08-30T09:16:31.507 INFO:teuthology.orchestra.run.smithi190.stdout: "max": 104857600, 2024-08-30T09:16:31.507 INFO:teuthology.orchestra.run.smithi190.stdout: "get_started": 0, 2024-08-30T09:16:31.507 INFO:teuthology.orchestra.run.smithi190.stdout: "get": 16804, 2024-08-30T09:16:31.507 INFO:teuthology.orchestra.run.smithi190.stdout: "get_sum": 6061119, 2024-08-30T09:16:31.507 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_fail": 0, 2024-08-30T09:16:31.508 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_success": 16804, 2024-08-30T09:16:31.508 INFO:teuthology.orchestra.run.smithi190.stdout: "take": 0, 2024-08-30T09:16:31.508 INFO:teuthology.orchestra.run.smithi190.stdout: "take_sum": 0, 2024-08-30T09:16:31.508 INFO:teuthology.orchestra.run.smithi190.stdout: "put": 16804, 2024-08-30T09:16:31.508 INFO:teuthology.orchestra.run.smithi190.stdout: "put_sum": 6061119, 2024-08-30T09:16:31.508 INFO:teuthology.orchestra.run.smithi190.stdout: "wait": { 2024-08-30T09:16:31.508 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-30T09:16:31.508 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-30T09:16:31.508 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-30T09:16:31.508 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-08-30T09:16:31.508 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-30T09:16:31.508 INFO:teuthology.orchestra.run.smithi190.stdout: "throttle-objecter_bytes": { 2024-08-30T09:16:31.508 INFO:teuthology.orchestra.run.smithi190.stdout: "val": 0, 2024-08-30T09:16:31.508 INFO:teuthology.orchestra.run.smithi190.stdout: "max": 104857600, 2024-08-30T09:16:31.509 INFO:teuthology.orchestra.run.smithi190.stdout: "get_started": 0, 2024-08-30T09:16:31.509 INFO:teuthology.orchestra.run.smithi190.stdout: "get": 0, 2024-08-30T09:16:31.509 INFO:teuthology.orchestra.run.smithi190.stdout: "get_sum": 0, 2024-08-30T09:16:31.509 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_fail": 0, 2024-08-30T09:16:31.509 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_success": 0, 2024-08-30T09:16:31.509 INFO:teuthology.orchestra.run.smithi190.stdout: "take": 6535, 2024-08-30T09:16:31.509 INFO:teuthology.orchestra.run.smithi190.stdout: "take_sum": 4576224, 2024-08-30T09:16:31.509 INFO:teuthology.orchestra.run.smithi190.stdout: "put": 6535, 2024-08-30T09:16:31.509 INFO:teuthology.orchestra.run.smithi190.stdout: "put_sum": 4576224, 2024-08-30T09:16:31.509 INFO:teuthology.orchestra.run.smithi190.stdout: "wait": { 2024-08-30T09:16:31.509 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-30T09:16:31.509 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-30T09:16:31.509 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-30T09:16:31.510 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-08-30T09:16:31.510 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-30T09:16:31.510 INFO:teuthology.orchestra.run.smithi190.stdout: "throttle-objecter_ops": { 2024-08-30T09:16:31.510 INFO:teuthology.orchestra.run.smithi190.stdout: "val": 0, 2024-08-30T09:16:31.510 INFO:teuthology.orchestra.run.smithi190.stdout: "max": 1024, 2024-08-30T09:16:31.510 INFO:teuthology.orchestra.run.smithi190.stdout: "get_started": 0, 2024-08-30T09:16:31.510 INFO:teuthology.orchestra.run.smithi190.stdout: "get": 0, 2024-08-30T09:16:31.510 INFO:teuthology.orchestra.run.smithi190.stdout: "get_sum": 0, 2024-08-30T09:16:31.510 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_fail": 0, 2024-08-30T09:16:31.510 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_success": 0, 2024-08-30T09:16:31.510 INFO:teuthology.orchestra.run.smithi190.stdout: "take": 6535, 2024-08-30T09:16:31.510 INFO:teuthology.orchestra.run.smithi190.stdout: "take_sum": 6535, 2024-08-30T09:16:31.510 INFO:teuthology.orchestra.run.smithi190.stdout: "put": 6535, 2024-08-30T09:16:31.511 INFO:teuthology.orchestra.run.smithi190.stdout: "put_sum": 6535, 2024-08-30T09:16:31.511 INFO:teuthology.orchestra.run.smithi190.stdout: "wait": { 2024-08-30T09:16:31.511 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-30T09:16:31.511 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-30T09:16:31.511 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-30T09:16:31.511 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-08-30T09:16:31.511 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-30T09:16:31.511 INFO:teuthology.orchestra.run.smithi190.stdout: "throttle-write_buf_throttle": { 2024-08-30T09:16:31.511 INFO:teuthology.orchestra.run.smithi190.stdout: "val": 0, 2024-08-30T09:16:31.511 INFO:teuthology.orchestra.run.smithi190.stdout: "max": 3758096384, 2024-08-30T09:16:31.511 INFO:teuthology.orchestra.run.smithi190.stdout: "get_started": 0, 2024-08-30T09:16:31.511 INFO:teuthology.orchestra.run.smithi190.stdout: "get": 15, 2024-08-30T09:16:31.511 INFO:teuthology.orchestra.run.smithi190.stdout: "get_sum": 1635, 2024-08-30T09:16:31.511 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_fail": 0, 2024-08-30T09:16:31.512 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_success": 15, 2024-08-30T09:16:31.512 INFO:teuthology.orchestra.run.smithi190.stdout: "take": 0, 2024-08-30T09:16:31.512 INFO:teuthology.orchestra.run.smithi190.stdout: "take_sum": 0, 2024-08-30T09:16:31.512 INFO:teuthology.orchestra.run.smithi190.stdout: "put": 8, 2024-08-30T09:16:31.512 INFO:teuthology.orchestra.run.smithi190.stdout: "put_sum": 1635, 2024-08-30T09:16:31.512 INFO:teuthology.orchestra.run.smithi190.stdout: "wait": { 2024-08-30T09:16:31.512 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-30T09:16:31.512 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-30T09:16:31.512 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-30T09:16:31.512 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-08-30T09:16:31.512 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-30T09:16:31.512 INFO:teuthology.orchestra.run.smithi190.stdout: "throttle-write_buf_throttle-0x55f3ce06c8f0": { 2024-08-30T09:16:31.512 INFO:teuthology.orchestra.run.smithi190.stdout: "val": 0, 2024-08-30T09:16:31.513 INFO:teuthology.orchestra.run.smithi190.stdout: "max": 3758096384, 2024-08-30T09:16:31.513 INFO:teuthology.orchestra.run.smithi190.stdout: "get_started": 0, 2024-08-30T09:16:31.513 INFO:teuthology.orchestra.run.smithi190.stdout: "get": 2458, 2024-08-30T09:16:31.513 INFO:teuthology.orchestra.run.smithi190.stdout: "get_sum": 4465310, 2024-08-30T09:16:31.513 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_fail": 0, 2024-08-30T09:16:31.513 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_success": 2458, 2024-08-30T09:16:31.513 INFO:teuthology.orchestra.run.smithi190.stdout: "take": 0, 2024-08-30T09:16:31.513 INFO:teuthology.orchestra.run.smithi190.stdout: "take_sum": 0, 2024-08-30T09:16:31.513 INFO:teuthology.orchestra.run.smithi190.stdout: "put": 1548, 2024-08-30T09:16:31.513 INFO:teuthology.orchestra.run.smithi190.stdout: "put_sum": 4465310, 2024-08-30T09:16:31.513 INFO:teuthology.orchestra.run.smithi190.stdout: "wait": { 2024-08-30T09:16:31.513 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-30T09:16:31.513 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-30T09:16:31.513 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-30T09:16:31.514 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-08-30T09:16:31.514 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-30T09:16:31.514 INFO:teuthology.orchestra.run.smithi190.stdout: "trackedop": { 2024-08-30T09:16:31.514 INFO:teuthology.orchestra.run.smithi190.stdout: "slow_ops_count": 0 2024-08-30T09:16:31.514 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-08-30T09:16:31.514 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-08-30T09:16:31.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:16:31 smithi167 ceph-mon[22063]: pgmap v845: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.7 KiB/s wr, 9 op/s 2024-08-30T09:16:32.324 INFO:tasks.check_counter:Daemon mds.i mds.exported=13 2024-08-30T09:16:32.325 INFO:tasks.check_counter:Daemon mds.i mds.imported=21 2024-08-30T09:16:32.325 DEBUG:tasks.check_counter:Getting stats from l 2024-08-30T09:16:32.325 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph daemon mds.l perf dump 2024-08-30T09:16:32.350 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:16:32 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-30T09:16:32.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:16:32 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-30T09:16:32.564 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:16:32 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-30T09:16:32.581 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.c/config 2024-08-30T09:16:33.162 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-08-30T09:16:33.162 INFO:teuthology.orchestra.run.smithi190.stdout: "AsyncMessenger::Worker-0": { 2024-08-30T09:16:33.162 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_messages": 0, 2024-08-30T09:16:33.162 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_messages": 2, 2024-08-30T09:16:33.162 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_bytes": 0, 2024-08-30T09:16:33.162 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_bytes": 0, 2024-08-30T09:16:33.162 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_created_connections": 4, 2024-08-30T09:16:33.162 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_active_connections": 0, 2024-08-30T09:16:33.162 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_total_time": 0.001144755, 2024-08-30T09:16:33.162 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_send_time": 0.000000000, 2024-08-30T09:16:33.163 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_recv_time": 0.000287954, 2024-08-30T09:16:33.163 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-08-30T09:16:33.163 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_messages_queue_lat": { 2024-08-30T09:16:33.163 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-30T09:16:33.163 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-30T09:16:33.163 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-30T09:16:33.163 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-30T09:16:33.163 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_handle_ack_lat": { 2024-08-30T09:16:33.163 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-30T09:16:33.163 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-30T09:16:33.163 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-30T09:16:33.163 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-30T09:16:33.163 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_encrypted_bytes": 0, 2024-08-30T09:16:33.164 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_encrypted_bytes": 0 2024-08-30T09:16:33.164 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-30T09:16:33.164 INFO:teuthology.orchestra.run.smithi190.stdout: "AsyncMessenger::Worker-1": { 2024-08-30T09:16:33.164 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_messages": 311, 2024-08-30T09:16:33.164 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_messages": 308, 2024-08-30T09:16:33.164 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_bytes": 548880, 2024-08-30T09:16:33.164 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_bytes": 190000, 2024-08-30T09:16:33.164 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_created_connections": 2, 2024-08-30T09:16:33.164 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_active_connections": 1, 2024-08-30T09:16:33.164 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_total_time": 0.045985580, 2024-08-30T09:16:33.164 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_send_time": 0.022588045, 2024-08-30T09:16:33.164 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_recv_time": 0.029827426, 2024-08-30T09:16:33.164 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-08-30T09:16:33.164 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_messages_queue_lat": { 2024-08-30T09:16:33.165 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 307, 2024-08-30T09:16:33.165 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.017281019, 2024-08-30T09:16:33.165 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000056289 2024-08-30T09:16:33.165 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-30T09:16:33.165 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_handle_ack_lat": { 2024-08-30T09:16:33.165 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-30T09:16:33.165 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-30T09:16:33.165 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-30T09:16:33.165 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-30T09:16:33.165 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_encrypted_bytes": 548880, 2024-08-30T09:16:33.165 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_encrypted_bytes": 190000 2024-08-30T09:16:33.165 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-30T09:16:33.165 INFO:teuthology.orchestra.run.smithi190.stdout: "AsyncMessenger::Worker-2": { 2024-08-30T09:16:33.166 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_messages": 6, 2024-08-30T09:16:33.166 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_messages": 237, 2024-08-30T09:16:33.166 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_bytes": 3264, 2024-08-30T09:16:33.166 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_bytes": 201936, 2024-08-30T09:16:33.166 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_created_connections": 4, 2024-08-30T09:16:33.166 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_active_connections": 2, 2024-08-30T09:16:33.166 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_total_time": 0.017726331, 2024-08-30T09:16:33.166 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_send_time": 0.012477736, 2024-08-30T09:16:33.166 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_recv_time": 0.002983270, 2024-08-30T09:16:33.166 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-08-30T09:16:33.166 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_messages_queue_lat": { 2024-08-30T09:16:33.166 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 236, 2024-08-30T09:16:33.166 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.013739025, 2024-08-30T09:16:33.167 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000058216 2024-08-30T09:16:33.167 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-30T09:16:33.167 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_handle_ack_lat": { 2024-08-30T09:16:33.167 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-30T09:16:33.167 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-30T09:16:33.167 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-30T09:16:33.167 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-30T09:16:33.167 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_encrypted_bytes": 3264, 2024-08-30T09:16:33.167 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_encrypted_bytes": 201936 2024-08-30T09:16:33.167 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-30T09:16:33.167 INFO:teuthology.orchestra.run.smithi190.stdout: "cct": { 2024-08-30T09:16:33.167 INFO:teuthology.orchestra.run.smithi190.stdout: "total_workers": 0, 2024-08-30T09:16:33.167 INFO:teuthology.orchestra.run.smithi190.stdout: "unhealthy_workers": 0 2024-08-30T09:16:33.168 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-30T09:16:33.168 INFO:teuthology.orchestra.run.smithi190.stdout: "mempool": { 2024-08-30T09:16:33.168 INFO:teuthology.orchestra.run.smithi190.stdout: "bloom_filter_bytes": 0, 2024-08-30T09:16:33.168 INFO:teuthology.orchestra.run.smithi190.stdout: "bloom_filter_items": 0, 2024-08-30T09:16:33.168 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_alloc_bytes": 0, 2024-08-30T09:16:33.168 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_alloc_items": 0, 2024-08-30T09:16:33.168 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_data_bytes": 0, 2024-08-30T09:16:33.168 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_data_items": 0, 2024-08-30T09:16:33.168 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_onode_bytes": 0, 2024-08-30T09:16:33.168 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_onode_items": 0, 2024-08-30T09:16:33.168 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_meta_bytes": 0, 2024-08-30T09:16:33.168 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_meta_items": 0, 2024-08-30T09:16:33.168 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_other_bytes": 0, 2024-08-30T09:16:33.169 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_other_items": 0, 2024-08-30T09:16:33.169 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_buffer_bytes": 0, 2024-08-30T09:16:33.169 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_buffer_items": 0, 2024-08-30T09:16:33.169 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_extent_bytes": 0, 2024-08-30T09:16:33.169 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_extent_items": 0, 2024-08-30T09:16:33.169 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_blob_bytes": 0, 2024-08-30T09:16:33.169 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_blob_items": 0, 2024-08-30T09:16:33.169 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_shared_blob_bytes": 0, 2024-08-30T09:16:33.169 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_shared_blob_items": 0, 2024-08-30T09:16:33.169 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_inline_bl_bytes": 0, 2024-08-30T09:16:33.169 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_inline_bl_items": 0, 2024-08-30T09:16:33.169 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_fsck_bytes": 0, 2024-08-30T09:16:33.169 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_fsck_items": 0, 2024-08-30T09:16:33.170 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_txc_bytes": 0, 2024-08-30T09:16:33.170 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_txc_items": 0, 2024-08-30T09:16:33.170 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_writing_deferred_bytes": 0, 2024-08-30T09:16:33.170 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_writing_deferred_items": 0, 2024-08-30T09:16:33.170 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_writing_bytes": 0, 2024-08-30T09:16:33.170 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_writing_items": 0, 2024-08-30T09:16:33.170 INFO:teuthology.orchestra.run.smithi190.stdout: "bluefs_bytes": 0, 2024-08-30T09:16:33.170 INFO:teuthology.orchestra.run.smithi190.stdout: "bluefs_items": 0, 2024-08-30T09:16:33.170 INFO:teuthology.orchestra.run.smithi190.stdout: "bluefs_file_reader_bytes": 0, 2024-08-30T09:16:33.170 INFO:teuthology.orchestra.run.smithi190.stdout: "bluefs_file_reader_items": 0, 2024-08-30T09:16:33.170 INFO:teuthology.orchestra.run.smithi190.stdout: "bluefs_file_writer_bytes": 0, 2024-08-30T09:16:33.170 INFO:teuthology.orchestra.run.smithi190.stdout: "bluefs_file_writer_items": 0, 2024-08-30T09:16:33.170 INFO:teuthology.orchestra.run.smithi190.stdout: "buffer_anon_bytes": 148744, 2024-08-30T09:16:33.170 INFO:teuthology.orchestra.run.smithi190.stdout: "buffer_anon_items": 30, 2024-08-30T09:16:33.171 INFO:teuthology.orchestra.run.smithi190.stdout: "buffer_meta_bytes": 0, 2024-08-30T09:16:33.171 INFO:teuthology.orchestra.run.smithi190.stdout: "buffer_meta_items": 0, 2024-08-30T09:16:33.171 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_bytes": 0, 2024-08-30T09:16:33.171 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_items": 0, 2024-08-30T09:16:33.171 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_mapbl_bytes": 0, 2024-08-30T09:16:33.171 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_mapbl_items": 0, 2024-08-30T09:16:33.171 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_pglog_bytes": 0, 2024-08-30T09:16:33.171 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_pglog_items": 0, 2024-08-30T09:16:33.171 INFO:teuthology.orchestra.run.smithi190.stdout: "osdmap_bytes": 0, 2024-08-30T09:16:33.171 INFO:teuthology.orchestra.run.smithi190.stdout: "osdmap_items": 0, 2024-08-30T09:16:33.171 INFO:teuthology.orchestra.run.smithi190.stdout: "osdmap_mapping_bytes": 0, 2024-08-30T09:16:33.171 INFO:teuthology.orchestra.run.smithi190.stdout: "osdmap_mapping_items": 0, 2024-08-30T09:16:33.171 INFO:teuthology.orchestra.run.smithi190.stdout: "pgmap_bytes": 0, 2024-08-30T09:16:33.171 INFO:teuthology.orchestra.run.smithi190.stdout: "pgmap_items": 0, 2024-08-30T09:16:33.171 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_co_bytes": 928, 2024-08-30T09:16:33.172 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_co_items": 2, 2024-08-30T09:16:33.172 INFO:teuthology.orchestra.run.smithi190.stdout: "unittest_1_bytes": 0, 2024-08-30T09:16:33.172 INFO:teuthology.orchestra.run.smithi190.stdout: "unittest_1_items": 0, 2024-08-30T09:16:33.172 INFO:teuthology.orchestra.run.smithi190.stdout: "unittest_2_bytes": 0, 2024-08-30T09:16:33.172 INFO:teuthology.orchestra.run.smithi190.stdout: "unittest_2_items": 0 2024-08-30T09:16:33.172 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-30T09:16:33.172 INFO:teuthology.orchestra.run.smithi190.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-08-30T09:16:33.172 INFO:teuthology.orchestra.run.smithi190.stdout: "val": 0, 2024-08-30T09:16:33.172 INFO:teuthology.orchestra.run.smithi190.stdout: "max": 104857600, 2024-08-30T09:16:33.172 INFO:teuthology.orchestra.run.smithi190.stdout: "get_started": 0, 2024-08-30T09:16:33.172 INFO:teuthology.orchestra.run.smithi190.stdout: "get": 313, 2024-08-30T09:16:33.172 INFO:teuthology.orchestra.run.smithi190.stdout: "get_sum": 504909, 2024-08-30T09:16:33.172 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_fail": 0, 2024-08-30T09:16:33.173 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_success": 313, 2024-08-30T09:16:33.173 INFO:teuthology.orchestra.run.smithi190.stdout: "take": 0, 2024-08-30T09:16:33.173 INFO:teuthology.orchestra.run.smithi190.stdout: "take_sum": 0, 2024-08-30T09:16:33.173 INFO:teuthology.orchestra.run.smithi190.stdout: "put": 313, 2024-08-30T09:16:33.173 INFO:teuthology.orchestra.run.smithi190.stdout: "put_sum": 504909, 2024-08-30T09:16:33.173 INFO:teuthology.orchestra.run.smithi190.stdout: "wait": { 2024-08-30T09:16:33.173 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-30T09:16:33.173 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-30T09:16:33.173 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-30T09:16:33.173 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-08-30T09:16:33.173 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-08-30T09:16:33.173 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-08-30T09:16:33.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:16:33 smithi042 ceph-mon[18652]: pgmap v846: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 3.9 KiB/s wr, 6 op/s 2024-08-30T09:16:33.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:16:33 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T09:16:33.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:16:33 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-30T09:16:33.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:16:33 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T09:16:33.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:16:33 smithi190 ceph-mon[22001]: pgmap v846: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 3.9 KiB/s wr, 6 op/s 2024-08-30T09:16:33.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:16:33 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T09:16:33.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:16:33 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-30T09:16:33.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:16:33 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T09:16:33.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:16:33 smithi167 ceph-mon[22063]: pgmap v846: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 3.9 KiB/s wr, 6 op/s 2024-08-30T09:16:33.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:16:33 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-30T09:16:33.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:16:33 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-30T09:16:33.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:16:33 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' 2024-08-30T09:16:33.870 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.l 2024-08-30T09:16:33.870 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.l 2024-08-30T09:16:33.870 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-08-30T09:16:33.880 INFO:tasks.kclient:Unmounting kernel clients... 2024-08-30T09:16:33.880 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-08-30T09:16:33.880 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/proc/self/mounts of=/dev/stdout 2024-08-30T09:16:33.912 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-08-30T09:16:33.912 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/proc/self/mounts of=/dev/stdout 2024-08-30T09:16:33.972 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph osd blocklist ls 2024-08-30T09:16:34.278 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T09:16:34.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:34.761+0000 7fbf4567b640 1 -- 172.21.15.42:0/330706941 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbf40102880 msgr2=0x7fbf40102c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:16:34.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:34.761+0000 7fbf4567b640 1 --2- 172.21.15.42:0/330706941 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbf40102880 0x7fbf40102c90 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fbf30009900 tx=0x7fbf3002a410 comp rx=0 tx=0).stop 2024-08-30T09:16:34.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:34.762+0000 7fbf4567b640 1 -- 172.21.15.42:0/330706941 shutdown_connections 2024-08-30T09:16:34.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:34.762+0000 7fbf4567b640 1 --2- 172.21.15.42:0/330706941 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbf40108ac0 0x7fbf4010f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:34.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:34.762+0000 7fbf4567b640 1 --2- 172.21.15.42:0/330706941 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbf40108170 0x7fbf40108580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:34.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:34.762+0000 7fbf4567b640 1 --2- 172.21.15.42:0/330706941 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbf40102880 0x7fbf40102c90 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:34.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:34.762+0000 7fbf4567b640 1 -- 172.21.15.42:0/330706941 >> 172.21.15.42:0/330706941 conn(0x7fbf40079530 msgr2=0x7fbf40079940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:16:34.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:34.762+0000 7fbf4567b640 1 -- 172.21.15.42:0/330706941 shutdown_connections 2024-08-30T09:16:34.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:34.762+0000 7fbf4567b640 1 -- 172.21.15.42:0/330706941 wait complete. 2024-08-30T09:16:34.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:34.762+0000 7fbf4567b640 1 Processor -- start 2024-08-30T09:16:34.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:34.763+0000 7fbf4567b640 1 -- start start 2024-08-30T09:16:34.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:34.763+0000 7fbf4567b640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbf40102880 0x7fbf4019b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:16:34.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:34.763+0000 7fbf4567b640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbf40108170 0x7fbf4019b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:16:34.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:34.763+0000 7fbf4567b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbf40108ac0 0x7fbf4019bb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:16:34.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:34.763+0000 7fbf4567b640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fbf40112770 con 0x7fbf40102880 2024-08-30T09:16:34.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:34.763+0000 7fbf4567b640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fbf401125f0 con 0x7fbf40108170 2024-08-30T09:16:34.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:34.763+0000 7fbf4567b640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fbf401128f0 con 0x7fbf40108ac0 2024-08-30T09:16:34.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:34.763+0000 7fbf3effd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbf40102880 0x7fbf4019b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:16:34.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:34.763+0000 7fbf3effd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbf40102880 0x7fbf4019b0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:47226/0 (socket says 172.21.15.42:47226) 2024-08-30T09:16:34.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:34.763+0000 7fbf3effd640 1 -- 172.21.15.42:0/2518583668 learned_addr learned my addr 172.21.15.42:0/2518583668 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:16:34.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:34.763+0000 7fbf37fff640 1 --2- 172.21.15.42:0/2518583668 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbf40108170 0x7fbf4019b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:16:34.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:34.763+0000 7fbf3f7fe640 1 --2- 172.21.15.42:0/2518583668 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbf40108ac0 0x7fbf4019bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:16:34.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:34.763+0000 7fbf3effd640 1 -- 172.21.15.42:0/2518583668 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbf40108ac0 msgr2=0x7fbf4019bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:16:34.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:34.763+0000 7fbf3effd640 1 --2- 172.21.15.42:0/2518583668 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbf40108ac0 0x7fbf4019bb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:34.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:34.763+0000 7fbf3effd640 1 -- 172.21.15.42:0/2518583668 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbf40108170 msgr2=0x7fbf4019b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:16:34.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:34.763+0000 7fbf3effd640 1 --2- 172.21.15.42:0/2518583668 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbf40108170 0x7fbf4019b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:34.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:34.764+0000 7fbf3effd640 1 -- 172.21.15.42:0/2518583668 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf401a6350 con 0x7fbf40102880 2024-08-30T09:16:34.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:34.764+0000 7fbf3effd640 1 --2- 172.21.15.42:0/2518583668 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbf40102880 0x7fbf4019b0c0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fbf30005940 tx=0x7fbf300042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:16:34.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:34.764+0000 7fbf3cff9640 1 -- 172.21.15.42:0/2518583668 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf3003f030 con 0x7fbf40102880 2024-08-30T09:16:34.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:34.764+0000 7fbf4567b640 1 -- 172.21.15.42:0/2518583668 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf401a65e0 con 0x7fbf40102880 2024-08-30T09:16:34.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:34.764+0000 7fbf3cff9640 1 -- 172.21.15.42:0/2518583668 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf30003cb0 con 0x7fbf40102880 2024-08-30T09:16:34.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:34.764+0000 7fbf3cff9640 1 -- 172.21.15.42:0/2518583668 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf30035070 con 0x7fbf40102880 2024-08-30T09:16:34.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:34.764+0000 7fbf4567b640 1 -- 172.21.15.42:0/2518583668 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf401a6b10 con 0x7fbf40102880 2024-08-30T09:16:34.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:34.765+0000 7fbf4567b640 1 -- 172.21.15.42:0/2518583668 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf40102c90 con 0x7fbf40102880 2024-08-30T09:16:34.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:34.767+0000 7fbf3cff9640 1 -- 172.21.15.42:0/2518583668 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7fbf30042030 con 0x7fbf40102880 2024-08-30T09:16:34.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:34.768+0000 7fbf3cff9640 1 --2- 172.21.15.42:0/2518583668 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fbf2c0b4b60 0x7fbf2c0b7070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:16:34.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:34.768+0000 7fbf37fff640 1 --2- 172.21.15.42:0/2518583668 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fbf2c0b4b60 0x7fbf2c0b7070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:16:34.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:34.768+0000 7fbf3cff9640 1 -- 172.21.15.42:0/2518583668 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7fbf30105c90 con 0x7fbf40102880 2024-08-30T09:16:34.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:34.768+0000 7fbf37fff640 1 --2- 172.21.15.42:0/2518583668 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fbf2c0b4b60 0x7fbf2c0b7070 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fbf40079fd0 tx=0x7fbf2800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:16:34.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:34.771+0000 7fbf3cff9640 1 -- 172.21.15.42:0/2518583668 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7fbf300cc9d0 con 0x7fbf40102880 2024-08-30T09:16:34.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:34.975+0000 7fbf4567b640 1 -- 172.21.15.42:0/2518583668 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fbf4006b9d0 con 0x7fbf40102880 2024-08-30T09:16:34.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:34.976+0000 7fbf3cff9640 1 -- 172.21.15.42:0/2518583668 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v83) ==== 81+0+590 (secure 0 0 0) 0x7fbf300cc2d0 con 0x7fbf40102880 2024-08-30T09:16:34.978 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/2691233260 2024-08-31T08:50:44.821070+0000 2024-08-30T09:16:34.978 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/1518053351 2024-08-31T08:50:44.821070+0000 2024-08-30T09:16:34.978 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/352001020 2024-08-31T08:50:44.821070+0000 2024-08-30T09:16:34.978 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:6801/1733768634 2024-08-31T08:50:44.821070+0000 2024-08-30T09:16:34.978 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/1000597888 2024-08-31T08:50:18.561851+0000 2024-08-30T09:16:34.978 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:6800/1733768634 2024-08-31T08:50:44.821070+0000 2024-08-30T09:16:34.978 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:6801/3538882081 2024-08-31T08:50:18.561851+0000 2024-08-30T09:16:34.978 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:6800/3538882081 2024-08-31T08:50:18.561851+0000 2024-08-30T09:16:34.979 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/145366508 2024-08-31T08:50:18.561851+0000 2024-08-30T09:16:34.979 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/3346355213 2024-08-31T08:50:18.561851+0000 2024-08-30T09:16:34.979 INFO:teuthology.orchestra.run.smithi042.stderr:listed 10 entries 2024-08-30T09:16:34.981 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:34.978+0000 7fbf4567b640 1 -- 172.21.15.42:0/2518583668 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fbf2c0b4b60 msgr2=0x7fbf2c0b7070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:16:34.981 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:34.978+0000 7fbf4567b640 1 --2- 172.21.15.42:0/2518583668 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fbf2c0b4b60 0x7fbf2c0b7070 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fbf40079fd0 tx=0x7fbf2800b390 comp rx=0 tx=0).stop 2024-08-30T09:16:34.981 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:34.978+0000 7fbf4567b640 1 -- 172.21.15.42:0/2518583668 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbf40102880 msgr2=0x7fbf4019b0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:16:34.981 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:34.978+0000 7fbf4567b640 1 --2- 172.21.15.42:0/2518583668 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbf40102880 0x7fbf4019b0c0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fbf30005940 tx=0x7fbf300042f0 comp rx=0 tx=0).stop 2024-08-30T09:16:34.981 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:34.979+0000 7fbf4567b640 1 -- 172.21.15.42:0/2518583668 shutdown_connections 2024-08-30T09:16:34.981 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:34.979+0000 7fbf4567b640 1 --2- 172.21.15.42:0/2518583668 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fbf2c0b4b60 0x7fbf2c0b7070 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:34.981 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:34.979+0000 7fbf4567b640 1 --2- 172.21.15.42:0/2518583668 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbf40108ac0 0x7fbf4019bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:34.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:34.979+0000 7fbf4567b640 1 --2- 172.21.15.42:0/2518583668 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbf40108170 0x7fbf4019b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:34.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:34.979+0000 7fbf4567b640 1 --2- 172.21.15.42:0/2518583668 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbf40102880 0x7fbf4019b0c0 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:34.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:34.979+0000 7fbf4567b640 1 -- 172.21.15.42:0/2518583668 >> 172.21.15.42:0/2518583668 conn(0x7fbf40079530 msgr2=0x7fbf40106270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:16:34.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:34.979+0000 7fbf4567b640 1 -- 172.21.15.42:0/2518583668 shutdown_connections 2024-08-30T09:16:34.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:34.979+0000 7fbf4567b640 1 -- 172.21.15.42:0/2518583668 wait complete. 2024-08-30T09:16:35.349 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:16:35 smithi042 ceph-mon[18652]: pgmap v847: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 4.0 KiB/s wr, 7 op/s 2024-08-30T09:16:35.349 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:16:35 smithi042 ceph-mon[18652]: from='client.? 172.21.15.42:0/2518583668' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-08-30T09:16:35.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:16:35 smithi190 ceph-mon[22001]: pgmap v847: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 4.0 KiB/s wr, 7 op/s 2024-08-30T09:16:35.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:16:35 smithi190 ceph-mon[22001]: from='client.? 172.21.15.42:0/2518583668' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-08-30T09:16:35.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:16:35 smithi167 ceph-mon[22063]: pgmap v847: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 4.0 KiB/s wr, 7 op/s 2024-08-30T09:16:35.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:16:35 smithi167 ceph-mon[22063]: from='client.? 172.21.15.42:0/2518583668' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-08-30T09:16:35.686 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-08-30T09:16:35.686 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-30T09:16:35.686 DEBUG:teuthology.orchestra.run.smithi042:> sudo umount /tmp/cephfs 2024-08-30T09:16:35.842 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi042.front.sepia.ceph.com 2024-08-30T09:16:35.842 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-30T09:16:35.843 DEBUG:teuthology.orchestra.run.smithi042:> (cd /home/ubuntu/cephtest && exec rmdir -- /tmp/cephfs) 2024-08-30T09:16:35.880 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-30T09:16:35.880 DEBUG:teuthology.orchestra.run.smithi042:> ip netns list 2024-08-30T09:16:35.941 INFO:teuthology.orchestra.run.smithi042.stdout:ceph-ns--tmp-cephfs (id: 0) 2024-08-30T09:16:35.942 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-30T09:16:35.942 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip netns delete ceph-ns--tmp-cephfs 2024-08-30T09:16:36.009 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-30T09:16:36.009 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip link delete ceph-brx 2024-08-30T09:16:36.115 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-08-30T09:16:36.126 INFO:tasks.cephadm:Teardown begin 2024-08-30T09:16:36.126 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-08-30T09:16:36.126 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph fs dump --format=json 2024-08-30T09:16:36.383 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T09:16:36.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:16:36 smithi042 ceph-mon[18652]: pgmap v848: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 4.6 KiB/s wr, 7 op/s 2024-08-30T09:16:36.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:16:36 smithi190 ceph-mon[22001]: pgmap v848: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 4.6 KiB/s wr, 7 op/s 2024-08-30T09:16:36.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:16:36 smithi167 ceph-mon[22063]: pgmap v848: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 4.6 KiB/s wr, 7 op/s 2024-08-30T09:16:36.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:36.912+0000 7f7d39564640 1 -- 172.21.15.42:0/2960314535 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7d34103900 msgr2=0x7f7d34103d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:16:36.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:36.912+0000 7f7d39564640 1 --2- 172.21.15.42:0/2960314535 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7d34103900 0x7f7d34103d10 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f7d24009900 tx=0x7f7d2402a410 comp rx=0 tx=0).stop 2024-08-30T09:16:36.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:36.912+0000 7f7d39564640 1 -- 172.21.15.42:0/2960314535 shutdown_connections 2024-08-30T09:16:36.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:36.912+0000 7f7d39564640 1 --2- 172.21.15.42:0/2960314535 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7d34109240 0x7f7d34109650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:36.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:36.912+0000 7f7d39564640 1 --2- 172.21.15.42:0/2960314535 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7d34109b90 0x7f7d341103c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:36.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:36.912+0000 7f7d39564640 1 --2- 172.21.15.42:0/2960314535 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7d34103900 0x7f7d34103d10 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:36.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:36.912+0000 7f7d39564640 1 -- 172.21.15.42:0/2960314535 >> 172.21.15.42:0/2960314535 conn(0x7f7d3407b5f0 msgr2=0x7f7d3407ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:16:36.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:36.912+0000 7f7d39564640 1 -- 172.21.15.42:0/2960314535 shutdown_connections 2024-08-30T09:16:36.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:36.912+0000 7f7d39564640 1 -- 172.21.15.42:0/2960314535 wait complete. 2024-08-30T09:16:36.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:36.913+0000 7f7d39564640 1 Processor -- start 2024-08-30T09:16:36.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:36.913+0000 7f7d39564640 1 -- start start 2024-08-30T09:16:36.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:36.913+0000 7f7d39564640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7d34103900 0x7f7d3419a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:16:36.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:36.914+0000 7f7d39564640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7d34109240 0x7f7d3419a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:16:36.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:36.914+0000 7f7d39564640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7d34109b90 0x7f7d3419fb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:16:36.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:36.914+0000 7f7d39564640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f7d341138d0 con 0x7f7d34109240 2024-08-30T09:16:36.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:36.914+0000 7f7d39564640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f7d34113750 con 0x7f7d34109b90 2024-08-30T09:16:36.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:36.914+0000 7f7d39564640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f7d34113a50 con 0x7f7d34103900 2024-08-30T09:16:36.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:36.914+0000 7f7d327fc640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7d34109240 0x7f7d3419a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:16:36.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:36.914+0000 7f7d327fc640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7d34109240 0x7f7d3419a860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:47238/0 (socket says 172.21.15.42:47238) 2024-08-30T09:16:36.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:36.914+0000 7f7d327fc640 1 -- 172.21.15.42:0/1326083726 learned_addr learned my addr 172.21.15.42:0/1326083726 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:16:36.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:36.914+0000 7f7d32ffd640 1 --2- 172.21.15.42:0/1326083726 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7d34103900 0x7f7d3419a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:16:36.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:36.914+0000 7f7d337fe640 1 --2- 172.21.15.42:0/1326083726 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7d34109b90 0x7f7d3419fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:16:36.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:36.914+0000 7f7d327fc640 1 -- 172.21.15.42:0/1326083726 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7d34103900 msgr2=0x7f7d3419a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:16:36.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:36.914+0000 7f7d327fc640 1 --2- 172.21.15.42:0/1326083726 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7d34103900 0x7f7d3419a2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:36.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:36.914+0000 7f7d327fc640 1 -- 172.21.15.42:0/1326083726 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7d34109b90 msgr2=0x7f7d3419fb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:16:36.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:36.914+0000 7f7d327fc640 1 --2- 172.21.15.42:0/1326083726 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7d34109b90 0x7f7d3419fb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:36.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:36.914+0000 7f7d327fc640 1 -- 172.21.15.42:0/1326083726 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d341a0230 con 0x7f7d34109240 2024-08-30T09:16:36.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:36.914+0000 7f7d327fc640 1 --2- 172.21.15.42:0/1326083726 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7d34109240 0x7f7d3419a860 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f7d2800d750 tx=0x7f7d2800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:16:36.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:36.914+0000 7f7d1bfff640 1 -- 172.21.15.42:0/1326083726 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d28017070 con 0x7f7d34109240 2024-08-30T09:16:36.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:36.914+0000 7f7d1bfff640 1 -- 172.21.15.42:0/1326083726 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d280054d0 con 0x7f7d34109240 2024-08-30T09:16:36.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:36.914+0000 7f7d39564640 1 -- 172.21.15.42:0/1326083726 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d341a0520 con 0x7f7d34109240 2024-08-30T09:16:36.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:36.915+0000 7f7d1bfff640 1 -- 172.21.15.42:0/1326083726 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d28004c30 con 0x7f7d34109240 2024-08-30T09:16:36.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:36.915+0000 7f7d39564640 1 -- 172.21.15.42:0/1326083726 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d341a6c80 con 0x7f7d34109240 2024-08-30T09:16:36.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:36.915+0000 7f7d39564640 1 -- 172.21.15.42:0/1326083726 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d34103d10 con 0x7f7d34109240 2024-08-30T09:16:36.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:36.917+0000 7f7d1bfff640 1 -- 172.21.15.42:0/1326083726 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7f7d28004dd0 con 0x7f7d34109240 2024-08-30T09:16:36.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:36.918+0000 7f7d1bfff640 1 --2- 172.21.15.42:0/1326083726 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f7d1c0b4b30 0x7f7d1c0b7040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:16:36.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:36.918+0000 7f7d32ffd640 1 --2- 172.21.15.42:0/1326083726 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f7d1c0b4b30 0x7f7d1c0b7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:16:36.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:36.918+0000 7f7d1bfff640 1 -- 172.21.15.42:0/1326083726 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f7d280e5390 con 0x7f7d34109240 2024-08-30T09:16:36.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:36.919+0000 7f7d32ffd640 1 --2- 172.21.15.42:0/1326083726 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f7d1c0b4b30 0x7f7d1c0b7040 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f7d24031440 tx=0x7f7d24005d50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:16:36.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:36.922+0000 7f7d1bfff640 1 -- 172.21.15.42:0/1326083726 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f7d280ac130 con 0x7f7d34109240 2024-08-30T09:16:37.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:37.147+0000 7f7d39564640 1 -- 172.21.15.42:0/1326083726 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7d3419b3f0 con 0x7f7d34109240 2024-08-30T09:16:37.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:37.147+0000 7f7d1bfff640 1 -- 172.21.15.42:0/1326083726 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) ==== 94+0+11920 (secure 0 0 0) 0x7f7d280aba30 con 0x7f7d34109240 2024-08-30T09:16:37.150 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-08-30T09:16:37.150 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":51,"btime":"2024-08-30T09:16:14:060252+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14703,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/3188400000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":3188400000},{"type":"v1","addr":"172.21.15.42:6835","nonce":3188400000}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14715,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/3356537674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":3356537674},{"type":"v1","addr":"172.21.15.42:6837","nonce":3356537674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14766,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6839/2107785513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6838","nonce":2107785513},{"type":"v1","addr":"172.21.15.42:6839","nonce":2107785513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14778,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6841/2298438674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6840","nonce":2298438674},{"type":"v1","addr":"172.21.15.42:6841","nonce":2298438674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24553,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/543555570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":543555570},{"type":"v1","addr":"172.21.15.190:6833","nonce":543555570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24554,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/3907818941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":3907818941},{"type":"v1","addr":"172.21.15.167:6833","nonce":3907818941}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":51,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-30T08:58:03.055084+0000","modified":"2024-08-30T09:16:13.340860+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24629,"mds_1":24610,"mds_2":24599},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24572":{"gid":24572,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6835/234596845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":234596845},{"type":"v1","addr":"172.21.15.167:6835","nonce":234596845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24592":{"gid":24592,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6835/247905709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":247905709},{"type":"v1","addr":"172.21.15.190:6835","nonce":247905709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24598":{"gid":24598,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6837/863256349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6836","nonce":863256349},{"type":"v1","addr":"172.21.15.190:6837","nonce":863256349}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24599":{"gid":24599,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.167:6837/3011837856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":3011837856},{"type":"v1","addr":"172.21.15.167:6837","nonce":3011837856}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24610":{"gid":24610,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.190:6839/719981110","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6838","nonce":719981110},{"type":"v1","addr":"172.21.15.190:6839","nonce":719981110}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24629":{"gid":24629,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.167:6839/3604645207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":3604645207},{"type":"v1","addr":"172.21.15.167:6839","nonce":3604645207}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24629,"qdb_cluster":[24599,24610,24629]},"id":1}]} 2024-08-30T09:16:37.150 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 51 2024-08-30T09:16:37.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:37.150+0000 7f7d39564640 1 -- 172.21.15.42:0/1326083726 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f7d1c0b4b30 msgr2=0x7f7d1c0b7040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:16:37.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:37.150+0000 7f7d39564640 1 --2- 172.21.15.42:0/1326083726 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f7d1c0b4b30 0x7f7d1c0b7040 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f7d24031440 tx=0x7f7d24005d50 comp rx=0 tx=0).stop 2024-08-30T09:16:37.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:37.150+0000 7f7d39564640 1 -- 172.21.15.42:0/1326083726 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7d34109240 msgr2=0x7f7d3419a860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:16:37.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:37.150+0000 7f7d39564640 1 --2- 172.21.15.42:0/1326083726 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7d34109240 0x7f7d3419a860 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f7d2800d750 tx=0x7f7d2800dc10 comp rx=0 tx=0).stop 2024-08-30T09:16:37.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:37.150+0000 7f7d39564640 1 -- 172.21.15.42:0/1326083726 shutdown_connections 2024-08-30T09:16:37.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:37.150+0000 7f7d39564640 1 --2- 172.21.15.42:0/1326083726 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f7d1c0b4b30 0x7f7d1c0b7040 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:37.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:37.150+0000 7f7d39564640 1 --2- 172.21.15.42:0/1326083726 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7d34109b90 0x7f7d3419fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:37.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:37.150+0000 7f7d39564640 1 --2- 172.21.15.42:0/1326083726 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7d34109240 0x7f7d3419a860 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:37.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:37.150+0000 7f7d39564640 1 --2- 172.21.15.42:0/1326083726 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7d34103900 0x7f7d3419a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:37.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:37.150+0000 7f7d39564640 1 -- 172.21.15.42:0/1326083726 >> 172.21.15.42:0/1326083726 conn(0x7f7d3407b5f0 msgr2=0x7f7d341072d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:16:37.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:37.150+0000 7f7d39564640 1 -- 172.21.15.42:0/1326083726 shutdown_connections 2024-08-30T09:16:37.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:37.150+0000 7f7d39564640 1 -- 172.21.15.42:0/1326083726 wait complete. 2024-08-30T09:16:37.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:16:37 smithi190 ceph-mon[22001]: from='client.? 172.21.15.42:0/1326083726' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-30T09:16:37.526 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:16:37 smithi042 ceph-mon[18652]: from='client.? 172.21.15.42:0/1326083726' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-30T09:16:37.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:16:37 smithi167 ceph-mon[22063]: from='client.? 172.21.15.42:0/1326083726' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-30T09:16:37.867 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph fs dump --format=json 2024-08-30T09:16:38.125 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T09:16:38.413 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:16:38 smithi042 ceph-mon[18652]: pgmap v849: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 4.6 KiB/s wr, 6 op/s 2024-08-30T09:16:38.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:16:38 smithi190 ceph-mon[22001]: pgmap v849: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 4.6 KiB/s wr, 6 op/s 2024-08-30T09:16:38.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:16:38 smithi167 ceph-mon[22063]: pgmap v849: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 4.6 KiB/s wr, 6 op/s 2024-08-30T09:16:38.650 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:38.647+0000 7fc5e9436640 1 -- 172.21.15.42:0/664481704 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc5e4109bb0 msgr2=0x7fc5e41103e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:16:38.650 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:38.647+0000 7fc5e9436640 1 --2- 172.21.15.42:0/664481704 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc5e4109bb0 0x7fc5e41103e0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7fc5dc009900 tx=0x7fc5dc02a410 comp rx=0 tx=0).stop 2024-08-30T09:16:38.650 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:38.647+0000 7fc5e9436640 1 -- 172.21.15.42:0/664481704 shutdown_connections 2024-08-30T09:16:38.650 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:38.647+0000 7fc5e9436640 1 --2- 172.21.15.42:0/664481704 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc5e4109260 0x7fc5e4109670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:38.650 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:38.647+0000 7fc5e9436640 1 --2- 172.21.15.42:0/664481704 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc5e4109bb0 0x7fc5e41103e0 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:38.650 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:38.647+0000 7fc5e9436640 1 --2- 172.21.15.42:0/664481704 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc5e4103920 0x7fc5e4103d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:38.650 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:38.647+0000 7fc5e9436640 1 -- 172.21.15.42:0/664481704 >> 172.21.15.42:0/664481704 conn(0x7fc5e40ff3b0 msgr2=0x7fc5e4101800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:16:38.650 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:38.647+0000 7fc5e9436640 1 -- 172.21.15.42:0/664481704 shutdown_connections 2024-08-30T09:16:38.650 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:38.647+0000 7fc5e9436640 1 -- 172.21.15.42:0/664481704 wait complete. 2024-08-30T09:16:38.650 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:38.648+0000 7fc5e9436640 1 Processor -- start 2024-08-30T09:16:38.650 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:38.648+0000 7fc5e9436640 1 -- start start 2024-08-30T09:16:38.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:38.648+0000 7fc5e9436640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc5e4103920 0x7fc5e419a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:16:38.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:38.648+0000 7fc5e9436640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc5e4109260 0x7fc5e419a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:16:38.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:38.648+0000 7fc5e9436640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc5e4109bb0 0x7fc5e419fac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:16:38.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:38.648+0000 7fc5e9436640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fc5e4113870 con 0x7fc5e4109260 2024-08-30T09:16:38.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:38.648+0000 7fc5e9436640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc5e41136f0 con 0x7fc5e4103920 2024-08-30T09:16:38.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:38.648+0000 7fc5e9436640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fc5e41139f0 con 0x7fc5e4109bb0 2024-08-30T09:16:38.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:38.648+0000 7fc5e27fc640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc5e4109260 0x7fc5e419a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:16:38.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:38.648+0000 7fc5e27fc640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc5e4109260 0x7fc5e419a820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:32990/0 (socket says 172.21.15.42:32990) 2024-08-30T09:16:38.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:38.648+0000 7fc5e27fc640 1 -- 172.21.15.42:0/618516904 learned_addr learned my addr 172.21.15.42:0/618516904 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:16:38.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:38.648+0000 7fc5e2ffd640 1 --2- 172.21.15.42:0/618516904 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc5e4103920 0x7fc5e419a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:16:38.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:38.649+0000 7fc5e37fe640 1 --2- 172.21.15.42:0/618516904 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc5e4109bb0 0x7fc5e419fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:16:38.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:38.649+0000 7fc5e27fc640 1 -- 172.21.15.42:0/618516904 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc5e4109bb0 msgr2=0x7fc5e419fac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:16:38.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:38.649+0000 7fc5e27fc640 1 --2- 172.21.15.42:0/618516904 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc5e4109bb0 0x7fc5e419fac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:38.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:38.649+0000 7fc5e27fc640 1 -- 172.21.15.42:0/618516904 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc5e4103920 msgr2=0x7fc5e419a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:16:38.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:38.649+0000 7fc5e27fc640 1 --2- 172.21.15.42:0/618516904 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc5e4103920 0x7fc5e419a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:38.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:38.649+0000 7fc5e27fc640 1 -- 172.21.15.42:0/618516904 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5e41a01f0 con 0x7fc5e4109260 2024-08-30T09:16:38.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:38.649+0000 7fc5e27fc640 1 --2- 172.21.15.42:0/618516904 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc5e4109260 0x7fc5e419a820 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7fc5cc00d750 tx=0x7fc5cc00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:16:38.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:38.649+0000 7fc5dbfff640 1 -- 172.21.15.42:0/618516904 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5cc017070 con 0x7fc5e4109260 2024-08-30T09:16:38.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:38.649+0000 7fc5dbfff640 1 -- 172.21.15.42:0/618516904 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc5cc0054d0 con 0x7fc5e4109260 2024-08-30T09:16:38.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:38.649+0000 7fc5dbfff640 1 -- 172.21.15.42:0/618516904 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5cc004c30 con 0x7fc5e4109260 2024-08-30T09:16:38.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:38.649+0000 7fc5e9436640 1 -- 172.21.15.42:0/618516904 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5e41a04e0 con 0x7fc5e4109260 2024-08-30T09:16:38.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:38.649+0000 7fc5e9436640 1 -- 172.21.15.42:0/618516904 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5e41a6c40 con 0x7fc5e4109260 2024-08-30T09:16:38.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:38.650+0000 7fc5e9436640 1 -- 172.21.15.42:0/618516904 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5e4104ba0 con 0x7fc5e4109260 2024-08-30T09:16:38.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:38.652+0000 7fc5dbfff640 1 -- 172.21.15.42:0/618516904 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7fc5cc004dd0 con 0x7fc5e4109260 2024-08-30T09:16:38.655 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:38.653+0000 7fc5dbfff640 1 --2- 172.21.15.42:0/618516904 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fc5d00b4b30 0x7fc5d00b7040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:16:38.655 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:38.653+0000 7fc5e2ffd640 1 --2- 172.21.15.42:0/618516904 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fc5d00b4b30 0x7fc5d00b7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:16:38.655 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:38.653+0000 7fc5dbfff640 1 -- 172.21.15.42:0/618516904 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7fc5cc0e5260 con 0x7fc5e4109260 2024-08-30T09:16:38.655 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:38.653+0000 7fc5e2ffd640 1 --2- 172.21.15.42:0/618516904 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fc5d00b4b30 0x7fc5d00b7040 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fc5d4005fd0 tx=0x7fc5d4009680 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:16:38.657 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:38.656+0000 7fc5dbfff640 1 -- 172.21.15.42:0/618516904 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7fc5cc0ac000 con 0x7fc5e4109260 2024-08-30T09:16:38.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:38.879+0000 7fc5e9436640 1 -- 172.21.15.42:0/618516904 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc5e410b9d0 con 0x7fc5e4109260 2024-08-30T09:16:38.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:38.880+0000 7fc5dbfff640 1 -- 172.21.15.42:0/618516904 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) ==== 94+0+11920 (secure 0 0 0) 0x7fc5cc0ab900 con 0x7fc5e4109260 2024-08-30T09:16:38.882 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-08-30T09:16:38.882 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":51,"btime":"2024-08-30T09:16:14:060252+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14703,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/3188400000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":3188400000},{"type":"v1","addr":"172.21.15.42:6835","nonce":3188400000}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14715,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/3356537674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":3356537674},{"type":"v1","addr":"172.21.15.42:6837","nonce":3356537674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14766,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6839/2107785513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6838","nonce":2107785513},{"type":"v1","addr":"172.21.15.42:6839","nonce":2107785513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14778,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6841/2298438674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6840","nonce":2298438674},{"type":"v1","addr":"172.21.15.42:6841","nonce":2298438674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24553,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/543555570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":543555570},{"type":"v1","addr":"172.21.15.190:6833","nonce":543555570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24554,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/3907818941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":3907818941},{"type":"v1","addr":"172.21.15.167:6833","nonce":3907818941}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":51,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-30T08:58:03.055084+0000","modified":"2024-08-30T09:16:13.340860+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24629,"mds_1":24610,"mds_2":24599},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24572":{"gid":24572,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6835/234596845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":234596845},{"type":"v1","addr":"172.21.15.167:6835","nonce":234596845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24592":{"gid":24592,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6835/247905709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":247905709},{"type":"v1","addr":"172.21.15.190:6835","nonce":247905709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24598":{"gid":24598,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6837/863256349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6836","nonce":863256349},{"type":"v1","addr":"172.21.15.190:6837","nonce":863256349}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24599":{"gid":24599,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.167:6837/3011837856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":3011837856},{"type":"v1","addr":"172.21.15.167:6837","nonce":3011837856}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24610":{"gid":24610,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.190:6839/719981110","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6838","nonce":719981110},{"type":"v1","addr":"172.21.15.190:6839","nonce":719981110}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24629":{"gid":24629,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.167:6839/3604645207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":3604645207},{"type":"v1","addr":"172.21.15.167:6839","nonce":3604645207}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24629,"qdb_cluster":[24599,24610,24629]},"id":1}]} 2024-08-30T09:16:38.883 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 51 2024-08-30T09:16:38.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:38.882+0000 7fc5e9436640 1 -- 172.21.15.42:0/618516904 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fc5d00b4b30 msgr2=0x7fc5d00b7040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:16:38.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:38.882+0000 7fc5e9436640 1 --2- 172.21.15.42:0/618516904 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fc5d00b4b30 0x7fc5d00b7040 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fc5d4005fd0 tx=0x7fc5d4009680 comp rx=0 tx=0).stop 2024-08-30T09:16:38.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:38.882+0000 7fc5e9436640 1 -- 172.21.15.42:0/618516904 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc5e4109260 msgr2=0x7fc5e419a820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:16:38.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:38.882+0000 7fc5e9436640 1 --2- 172.21.15.42:0/618516904 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc5e4109260 0x7fc5e419a820 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7fc5cc00d750 tx=0x7fc5cc00dc10 comp rx=0 tx=0).stop 2024-08-30T09:16:38.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:38.882+0000 7fc5e9436640 1 -- 172.21.15.42:0/618516904 shutdown_connections 2024-08-30T09:16:38.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:38.882+0000 7fc5e9436640 1 --2- 172.21.15.42:0/618516904 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fc5d00b4b30 0x7fc5d00b7040 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:38.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:38.882+0000 7fc5e9436640 1 --2- 172.21.15.42:0/618516904 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc5e4109bb0 0x7fc5e419fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:38.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:38.882+0000 7fc5e9436640 1 --2- 172.21.15.42:0/618516904 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc5e4109260 0x7fc5e419a820 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:38.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:38.882+0000 7fc5e9436640 1 --2- 172.21.15.42:0/618516904 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc5e4103920 0x7fc5e419a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:38.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:38.882+0000 7fc5e9436640 1 -- 172.21.15.42:0/618516904 >> 172.21.15.42:0/618516904 conn(0x7fc5e40ff3b0 msgr2=0x7fc5e4108890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:16:38.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:38.882+0000 7fc5e9436640 1 -- 172.21.15.42:0/618516904 shutdown_connections 2024-08-30T09:16:38.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:38.882+0000 7fc5e9436640 1 -- 172.21.15.42:0/618516904 wait complete. 2024-08-30T09:16:39.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:16:39 smithi190 ceph-mon[22001]: from='client.? 172.21.15.42:0/618516904' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-30T09:16:39.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:16:39 smithi167 ceph-mon[22063]: from='client.? 172.21.15.42:0/618516904' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-30T09:16:39.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:16:39 smithi042 ceph-mon[18652]: from='client.? 172.21.15.42:0/618516904' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-30T09:16:39.781 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph osd dump --format=json 2024-08-30T09:16:40.042 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T09:16:40.432 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:16:40 smithi190 ceph-mon[22001]: pgmap v850: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 22 KiB/s wr, 10 op/s 2024-08-30T09:16:40.576 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:16:40 smithi042 ceph-mon[18652]: pgmap v850: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 22 KiB/s wr, 10 op/s 2024-08-30T09:16:40.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:16:40 smithi167 ceph-mon[22063]: pgmap v850: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 22 KiB/s wr, 10 op/s 2024-08-30T09:16:40.668 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:40.666+0000 7f5886cc7640 1 -- 172.21.15.42:0/1501047635 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5880108a90 msgr2=0x7f588010f2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:16:40.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:40.666+0000 7f5886cc7640 1 --2- 172.21.15.42:0/1501047635 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5880108a90 0x7f588010f2c0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f5870009900 tx=0x7f587002a410 comp rx=0 tx=0).stop 2024-08-30T09:16:40.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:40.666+0000 7f5886cc7640 1 -- 172.21.15.42:0/1501047635 shutdown_connections 2024-08-30T09:16:40.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:40.666+0000 7f5886cc7640 1 --2- 172.21.15.42:0/1501047635 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5880108140 0x7f5880108550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:40.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:40.666+0000 7f5886cc7640 1 --2- 172.21.15.42:0/1501047635 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5880108a90 0x7f588010f2c0 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:40.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:40.666+0000 7f5886cc7640 1 --2- 172.21.15.42:0/1501047635 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5880102800 0x7f5880102c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:40.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:40.666+0000 7f5886cc7640 1 -- 172.21.15.42:0/1501047635 >> 172.21.15.42:0/1501047635 conn(0x7f58800794d0 msgr2=0x7f58800798e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:16:40.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:40.666+0000 7f5886cc7640 1 -- 172.21.15.42:0/1501047635 shutdown_connections 2024-08-30T09:16:40.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:40.666+0000 7f5886cc7640 1 -- 172.21.15.42:0/1501047635 wait complete. 2024-08-30T09:16:40.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:40.667+0000 7f5886cc7640 1 Processor -- start 2024-08-30T09:16:40.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:40.667+0000 7f5886cc7640 1 -- start start 2024-08-30T09:16:40.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:40.668+0000 7f5886cc7640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5880102800 0x7f588019a190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:16:40.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:40.668+0000 7f5886cc7640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5880108140 0x7f588019a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:16:40.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:40.668+0000 7f5886cc7640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5880108a90 0x7f588019f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:16:40.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:40.668+0000 7f5886cc7640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f5880112710 con 0x7f5880108140 2024-08-30T09:16:40.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:40.668+0000 7f5886cc7640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f5880112590 con 0x7f5880102800 2024-08-30T09:16:40.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:40.668+0000 7f5886cc7640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f5880112890 con 0x7f5880108a90 2024-08-30T09:16:40.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:40.668+0000 7f587ffff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5880108140 0x7f588019a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:16:40.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:40.668+0000 7f587ffff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5880108140 0x7f588019a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33008/0 (socket says 172.21.15.42:33008) 2024-08-30T09:16:40.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:40.668+0000 7f587ffff640 1 -- 172.21.15.42:0/2927142607 learned_addr learned my addr 172.21.15.42:0/2927142607 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:16:40.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:40.668+0000 7f5884a63640 1 --2- 172.21.15.42:0/2927142607 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5880102800 0x7f588019a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:16:40.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:40.668+0000 7f5885264640 1 --2- 172.21.15.42:0/2927142607 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5880108a90 0x7f588019f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:16:40.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:40.668+0000 7f587ffff640 1 -- 172.21.15.42:0/2927142607 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5880108a90 msgr2=0x7f588019f9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:16:40.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:40.668+0000 7f587ffff640 1 --2- 172.21.15.42:0/2927142607 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5880108a90 0x7f588019f9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:40.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:40.668+0000 7f587ffff640 1 -- 172.21.15.42:0/2927142607 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5880102800 msgr2=0x7f588019a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:16:40.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:40.668+0000 7f587ffff640 1 --2- 172.21.15.42:0/2927142607 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5880102800 0x7f588019a190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:40.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:40.668+0000 7f587ffff640 1 -- 172.21.15.42:0/2927142607 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58801a0100 con 0x7f5880108140 2024-08-30T09:16:40.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:40.668+0000 7f5885264640 1 --2- 172.21.15.42:0/2927142607 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5880108a90 0x7f588019f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-30T09:16:40.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:40.668+0000 7f587ffff640 1 --2- 172.21.15.42:0/2927142607 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5880108140 0x7f588019a730 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f586800d740 tx=0x7f586800dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:16:40.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:40.669+0000 7f587dffb640 1 -- 172.21.15.42:0/2927142607 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5868017070 con 0x7f5880108140 2024-08-30T09:16:40.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:40.669+0000 7f587dffb640 1 -- 172.21.15.42:0/2927142607 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f586800de50 con 0x7f5880108140 2024-08-30T09:16:40.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:40.669+0000 7f5886cc7640 1 -- 172.21.15.42:0/2927142607 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58801a03f0 con 0x7f5880108140 2024-08-30T09:16:40.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:40.669+0000 7f587dffb640 1 -- 172.21.15.42:0/2927142607 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5868004ce0 con 0x7f5880108140 2024-08-30T09:16:40.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:40.669+0000 7f5886cc7640 1 -- 172.21.15.42:0/2927142607 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58801a6b50 con 0x7f5880108140 2024-08-30T09:16:40.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:40.670+0000 7f5886cc7640 1 -- 172.21.15.42:0/2927142607 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5880102c10 con 0x7f5880108140 2024-08-30T09:16:40.674 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:40.671+0000 7f587dffb640 1 -- 172.21.15.42:0/2927142607 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7f5868004e80 con 0x7f5880108140 2024-08-30T09:16:40.674 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:40.672+0000 7f587dffb640 1 --2- 172.21.15.42:0/2927142607 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f586c0b4b30 0x7f586c0b7040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:16:40.674 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:40.672+0000 7f587dffb640 1 -- 172.21.15.42:0/2927142607 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f58680e5260 con 0x7f5880108140 2024-08-30T09:16:40.674 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:40.672+0000 7f5884a63640 1 --2- 172.21.15.42:0/2927142607 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f586c0b4b30 0x7f586c0b7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:16:40.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:40.673+0000 7f5884a63640 1 --2- 172.21.15.42:0/2927142607 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f586c0b4b30 0x7f586c0b7040 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f5878005c40 tx=0x7f587800a590 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:16:40.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:40.675+0000 7f587dffb640 1 -- 172.21.15.42:0/2927142607 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f58680ac000 con 0x7f5880108140 2024-08-30T09:16:40.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:40.863+0000 7f5886cc7640 1 -- 172.21.15.42:0/2927142607 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f58801a6d80 con 0x7f5880108140 2024-08-30T09:16:40.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:40.864+0000 7f587dffb640 1 -- 172.21.15.42:0/2927142607 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+24867 (secure 0 0 0) 0x7f58680ab900 con 0x7f5880108140 2024-08-30T09:16:40.867 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-08-30T09:16:40.867 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":83,"fsid":"b0895c1a-66ac-11ef-bcd4-c7b262605968","created":"2024-08-30T08:49:44.858265+0000","modified":"2024-08-30T08:59:46.645920+0000","last_up_change":"2024-08-30T08:57:12.367791+0000","last_in_change":"2024-08-30T08:56:50.274678+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-30T08:53:27.297022+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-30T08:57:58.489964+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-30T08:58:01.064783+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-08-30T08:58:07.416534+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"029d62d7-a6c6-4905-8c76-6dffa9d57b92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":2617890537},{"type":"v1","addr":"172.21.15.42:6803","nonce":2617890537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":2617890537},{"type":"v1","addr":"172.21.15.42:6805","nonce":2617890537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2617890537},{"type":"v1","addr":"172.21.15.42:6809","nonce":2617890537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":2617890537},{"type":"v1","addr":"172.21.15.42:6807","nonce":2617890537}]},"public_addr":"172.21.15.42:6803/2617890537","cluster_addr":"172.21.15.42:6805/2617890537","heartbeat_back_addr":"172.21.15.42:6809/2617890537","heartbeat_front_addr":"172.21.15.42:6807/2617890537","state":["exists","up"]},{"osd":1,"uuid":"3a78384b-0207-41df-9384-22665007d5b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":2791960082},{"type":"v1","addr":"172.21.15.167:6801","nonce":2791960082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":2791960082},{"type":"v1","addr":"172.21.15.167:6803","nonce":2791960082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":2791960082},{"type":"v1","addr":"172.21.15.167:6807","nonce":2791960082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":2791960082},{"type":"v1","addr":"172.21.15.167:6805","nonce":2791960082}]},"public_addr":"172.21.15.167:6801/2791960082","cluster_addr":"172.21.15.167:6803/2791960082","heartbeat_back_addr":"172.21.15.167:6807/2791960082","heartbeat_front_addr":"172.21.15.167:6805/2791960082","state":["exists","up"]},{"osd":2,"uuid":"16001809-c27c-4f52-948a-9998261acb4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":1101326404},{"type":"v1","addr":"172.21.15.190:6801","nonce":1101326404}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":1101326404},{"type":"v1","addr":"172.21.15.190:6803","nonce":1101326404}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":1101326404},{"type":"v1","addr":"172.21.15.190:6807","nonce":1101326404}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":1101326404},{"type":"v1","addr":"172.21.15.190:6805","nonce":1101326404}]},"public_addr":"172.21.15.190:6801/1101326404","cluster_addr":"172.21.15.190:6803/1101326404","heartbeat_back_addr":"172.21.15.190:6807/1101326404","heartbeat_front_addr":"172.21.15.190:6805/1101326404","state":["exists","up"]},{"osd":3,"uuid":"87bb3e36-9e9e-4b19-9d8c-0cdf6e089d3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":1135599916},{"type":"v1","addr":"172.21.15.42:6811","nonce":1135599916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":1135599916},{"type":"v1","addr":"172.21.15.42:6813","nonce":1135599916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":1135599916},{"type":"v1","addr":"172.21.15.42:6817","nonce":1135599916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":1135599916},{"type":"v1","addr":"172.21.15.42:6815","nonce":1135599916}]},"public_addr":"172.21.15.42:6811/1135599916","cluster_addr":"172.21.15.42:6813/1135599916","heartbeat_back_addr":"172.21.15.42:6817/1135599916","heartbeat_front_addr":"172.21.15.42:6815/1135599916","state":["exists","up"]},{"osd":4,"uuid":"3ca1aae3-c972-4fb9-ac86-236279c30d64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":2927511419},{"type":"v1","addr":"172.21.15.167:6809","nonce":2927511419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":2927511419},{"type":"v1","addr":"172.21.15.167:6811","nonce":2927511419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":2927511419},{"type":"v1","addr":"172.21.15.167:6815","nonce":2927511419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":2927511419},{"type":"v1","addr":"172.21.15.167:6813","nonce":2927511419}]},"public_addr":"172.21.15.167:6809/2927511419","cluster_addr":"172.21.15.167:6811/2927511419","heartbeat_back_addr":"172.21.15.167:6815/2927511419","heartbeat_front_addr":"172.21.15.167:6813/2927511419","state":["exists","up"]},{"osd":5,"uuid":"5d07f54b-0d07-4efe-9bcf-390755ce0258","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":2557145333},{"type":"v1","addr":"172.21.15.190:6809","nonce":2557145333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":2557145333},{"type":"v1","addr":"172.21.15.190:6811","nonce":2557145333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":2557145333},{"type":"v1","addr":"172.21.15.190:6815","nonce":2557145333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":2557145333},{"type":"v1","addr":"172.21.15.190:6813","nonce":2557145333}]},"public_addr":"172.21.15.190:6809/2557145333","cluster_addr":"172.21.15.190:6811/2557145333","heartbeat_back_addr":"172.21.15.190:6815/2557145333","heartbeat_front_addr":"172.21.15.190:6813/2557145333","state":["exists","up"]},{"osd":6,"uuid":"96ad64af-44ff-4197-97d4-156f0cb88e81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":3213526100},{"type":"v1","addr":"172.21.15.42:6819","nonce":3213526100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":3213526100},{"type":"v1","addr":"172.21.15.42:6821","nonce":3213526100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":3213526100},{"type":"v1","addr":"172.21.15.42:6825","nonce":3213526100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":3213526100},{"type":"v1","addr":"172.21.15.42:6823","nonce":3213526100}]},"public_addr":"172.21.15.42:6819/3213526100","cluster_addr":"172.21.15.42:6821/3213526100","heartbeat_back_addr":"172.21.15.42:6825/3213526100","heartbeat_front_addr":"172.21.15.42:6823/3213526100","state":["exists","up"]},{"osd":7,"uuid":"337a2112-eaf7-4e6b-86be-4e40d6ddf835","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":2962799814},{"type":"v1","addr":"172.21.15.167:6817","nonce":2962799814}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":2962799814},{"type":"v1","addr":"172.21.15.167:6819","nonce":2962799814}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":2962799814},{"type":"v1","addr":"172.21.15.167:6823","nonce":2962799814}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":2962799814},{"type":"v1","addr":"172.21.15.167:6821","nonce":2962799814}]},"public_addr":"172.21.15.167:6817/2962799814","cluster_addr":"172.21.15.167:6819/2962799814","heartbeat_back_addr":"172.21.15.167:6823/2962799814","heartbeat_front_addr":"172.21.15.167:6821/2962799814","state":["exists","up"]},{"osd":8,"uuid":"219a7224-4331-464b-ac86-c51054d51d91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":3242198843},{"type":"v1","addr":"172.21.15.190:6817","nonce":3242198843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":3242198843},{"type":"v1","addr":"172.21.15.190:6819","nonce":3242198843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":3242198843},{"type":"v1","addr":"172.21.15.190:6823","nonce":3242198843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":3242198843},{"type":"v1","addr":"172.21.15.190:6821","nonce":3242198843}]},"public_addr":"172.21.15.190:6817/3242198843","cluster_addr":"172.21.15.190:6819/3242198843","heartbeat_back_addr":"172.21.15.190:6823/3242198843","heartbeat_front_addr":"172.21.15.190:6821/3242198843","state":["exists","up"]},{"osd":9,"uuid":"2f2f5177-a009-40d9-991c-76aec8a5cba6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":1123536325},{"type":"v1","addr":"172.21.15.42:6827","nonce":1123536325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":1123536325},{"type":"v1","addr":"172.21.15.42:6829","nonce":1123536325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":1123536325},{"type":"v1","addr":"172.21.15.42:6833","nonce":1123536325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":1123536325},{"type":"v1","addr":"172.21.15.42:6831","nonce":1123536325}]},"public_addr":"172.21.15.42:6827/1123536325","cluster_addr":"172.21.15.42:6829/1123536325","heartbeat_back_addr":"172.21.15.42:6833/1123536325","heartbeat_front_addr":"172.21.15.42:6831/1123536325","state":["exists","up"]},{"osd":10,"uuid":"96ef5058-2518-4862-a165-17666b33a3e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":3451198319},{"type":"v1","addr":"172.21.15.167:6825","nonce":3451198319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":3451198319},{"type":"v1","addr":"172.21.15.167:6827","nonce":3451198319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":3451198319},{"type":"v1","addr":"172.21.15.167:6831","nonce":3451198319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":3451198319},{"type":"v1","addr":"172.21.15.167:6829","nonce":3451198319}]},"public_addr":"172.21.15.167:6825/3451198319","cluster_addr":"172.21.15.167:6827/3451198319","heartbeat_back_addr":"172.21.15.167:6831/3451198319","heartbeat_front_addr":"172.21.15.167:6829/3451198319","state":["exists","up"]},{"osd":11,"uuid":"a4eb9bd3-3faa-4c81-ba6d-c2084b72c9ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":380064457},{"type":"v1","addr":"172.21.15.190:6825","nonce":380064457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":380064457},{"type":"v1","addr":"172.21.15.190:6827","nonce":380064457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":380064457},{"type":"v1","addr":"172.21.15.190:6831","nonce":380064457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":380064457},{"type":"v1","addr":"172.21.15.190:6829","nonce":380064457}]},"public_addr":"172.21.15.190:6825/380064457","cluster_addr":"172.21.15.190:6827/380064457","heartbeat_back_addr":"172.21.15.190:6831/380064457","heartbeat_front_addr":"172.21.15.190:6829/380064457","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-30T08:52:36.511448+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-30T08:53:00.027277+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-30T08:53:23.770245+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-30T08:53:49.322226+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-30T08:54:14.350234+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-30T08:54:39.491793+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-30T08:55:04.870596+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-30T08:55:30.314150+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-30T08:55:54.483843+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-30T08:56:20.586115+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-30T08:56:45.666783+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-30T08:57:10.718387+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"2.1d","mappings":[{"from":2,"to":1}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"4.9","mappings":[{"from":5,"to":8}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.42:0/2691233260":"2024-08-31T08:50:44.821070+0000","172.21.15.42:0/1518053351":"2024-08-31T08:50:44.821070+0000","172.21.15.42:0/352001020":"2024-08-31T08:50:44.821070+0000","172.21.15.42:6801/1733768634":"2024-08-31T08:50:44.821070+0000","172.21.15.42:0/1000597888":"2024-08-31T08:50:18.561851+0000","172.21.15.42:6800/1733768634":"2024-08-31T08:50:44.821070+0000","172.21.15.42:6801/3538882081":"2024-08-31T08:50:18.561851+0000","172.21.15.42:6800/3538882081":"2024-08-31T08:50:18.561851+0000","172.21.15.42:0/145366508":"2024-08-31T08:50:18.561851+0000","172.21.15.42:0/3346355213":"2024-08-31T08:50:18.561851+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-30T09:16:40.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:40.867+0000 7f5886cc7640 1 -- 172.21.15.42:0/2927142607 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f586c0b4b30 msgr2=0x7f586c0b7040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:16:40.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:40.867+0000 7f5886cc7640 1 --2- 172.21.15.42:0/2927142607 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f586c0b4b30 0x7f586c0b7040 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f5878005c40 tx=0x7f587800a590 comp rx=0 tx=0).stop 2024-08-30T09:16:40.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:40.867+0000 7f5886cc7640 1 -- 172.21.15.42:0/2927142607 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5880108140 msgr2=0x7f588019a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:16:40.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:40.867+0000 7f5886cc7640 1 --2- 172.21.15.42:0/2927142607 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5880108140 0x7f588019a730 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f586800d740 tx=0x7f586800dc00 comp rx=0 tx=0).stop 2024-08-30T09:16:40.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:40.867+0000 7f5886cc7640 1 -- 172.21.15.42:0/2927142607 shutdown_connections 2024-08-30T09:16:40.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:40.867+0000 7f5886cc7640 1 --2- 172.21.15.42:0/2927142607 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f586c0b4b30 0x7f586c0b7040 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:40.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:40.867+0000 7f5886cc7640 1 --2- 172.21.15.42:0/2927142607 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5880108a90 0x7f588019f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:40.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:40.867+0000 7f5886cc7640 1 --2- 172.21.15.42:0/2927142607 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5880108140 0x7f588019a730 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:40.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:40.867+0000 7f5886cc7640 1 --2- 172.21.15.42:0/2927142607 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5880102800 0x7f588019a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:40.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:40.867+0000 7f5886cc7640 1 -- 172.21.15.42:0/2927142607 >> 172.21.15.42:0/2927142607 conn(0x7f58800794d0 msgr2=0x7f5880103ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:16:40.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:40.867+0000 7f5886cc7640 1 -- 172.21.15.42:0/2927142607 shutdown_connections 2024-08-30T09:16:40.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:40.867+0000 7f5886cc7640 1 -- 172.21.15.42:0/2927142607 wait complete. 2024-08-30T09:16:41.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:16:41 smithi190 ceph-mon[22001]: from='client.? 172.21.15.42:0/2927142607' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-30T09:16:41.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:16:41 smithi167 ceph-mon[22063]: from='client.? 172.21.15.42:0/2927142607' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-30T09:16:41.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:16:41 smithi042 ceph-mon[18652]: from='client.? 172.21.15.42:0/2927142607' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-30T09:16:41.706 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph fs fail cephfs 2024-08-30T09:16:41.963 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T09:16:42.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:16:42 smithi042 ceph-mon[18652]: pgmap v851: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 7 op/s 2024-08-30T09:16:42.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:16:42 smithi190 ceph-mon[22001]: pgmap v851: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 7 op/s 2024-08-30T09:16:42.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:42.477+0000 7f393aab3640 1 -- 172.21.15.42:0/2374764157 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3934109c50 msgr2=0x7f3934110480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:16:42.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:42.477+0000 7f393aab3640 1 --2- 172.21.15.42:0/2374764157 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3934109c50 0x7f3934110480 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f392c009ca0 tx=0x7f392c02a8c0 comp rx=0 tx=0).stop 2024-08-30T09:16:42.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:42.478+0000 7f393aab3640 1 -- 172.21.15.42:0/2374764157 shutdown_connections 2024-08-30T09:16:42.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:42.478+0000 7f393aab3640 1 --2- 172.21.15.42:0/2374764157 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3934109c50 0x7f3934110480 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:42.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:42.478+0000 7f393aab3640 1 --2- 172.21.15.42:0/2374764157 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3934109300 0x7f3934109710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:42.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:42.478+0000 7f393aab3640 1 --2- 172.21.15.42:0/2374764157 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3934101e20 0x7f3934102230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:42.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:42.478+0000 7f393aab3640 1 -- 172.21.15.42:0/2374764157 >> 172.21.15.42:0/2374764157 conn(0x7f3934079c60 msgr2=0x7f393407a070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:16:42.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:42.478+0000 7f393aab3640 1 -- 172.21.15.42:0/2374764157 shutdown_connections 2024-08-30T09:16:42.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:42.478+0000 7f393aab3640 1 -- 172.21.15.42:0/2374764157 wait complete. 2024-08-30T09:16:42.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:42.478+0000 7f393aab3640 1 Processor -- start 2024-08-30T09:16:42.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:42.478+0000 7f393aab3640 1 -- start start 2024-08-30T09:16:42.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:42.479+0000 7f393aab3640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3934101e20 0x7f393419a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:16:42.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:42.479+0000 7f393aab3640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3934109300 0x7f393419a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:16:42.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:42.479+0000 7f393aab3640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3934109c50 0x7f393419fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:16:42.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:42.479+0000 7f393aab3640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f3934113910 con 0x7f3934109300 2024-08-30T09:16:42.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:42.479+0000 7f393aab3640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f3934113790 con 0x7f3934109c50 2024-08-30T09:16:42.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:42.479+0000 7f393aab3640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f3934113a90 con 0x7f3934101e20 2024-08-30T09:16:42.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:42.479+0000 7f3933fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3934109300 0x7f393419a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:16:42.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:42.479+0000 7f3933fff640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3934109300 0x7f393419a7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33032/0 (socket says 172.21.15.42:33032) 2024-08-30T09:16:42.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:42.479+0000 7f3933fff640 1 -- 172.21.15.42:0/1498866746 learned_addr learned my addr 172.21.15.42:0/1498866746 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:16:42.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:42.479+0000 7f3939050640 1 --2- 172.21.15.42:0/1498866746 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3934109c50 0x7f393419fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:16:42.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:42.479+0000 7f393884f640 1 --2- 172.21.15.42:0/1498866746 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3934101e20 0x7f393419a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:16:42.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:42.479+0000 7f3933fff640 1 -- 172.21.15.42:0/1498866746 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3934101e20 msgr2=0x7f393419a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:16:42.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:42.479+0000 7f3933fff640 1 --2- 172.21.15.42:0/1498866746 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3934101e20 0x7f393419a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:42.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:42.479+0000 7f3933fff640 1 -- 172.21.15.42:0/1498866746 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3934109c50 msgr2=0x7f393419fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:16:42.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:42.479+0000 7f3933fff640 1 --2- 172.21.15.42:0/1498866746 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3934109c50 0x7f393419fa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:42.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:42.479+0000 7f3933fff640 1 -- 172.21.15.42:0/1498866746 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39341a01b0 con 0x7f3934109300 2024-08-30T09:16:42.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:42.479+0000 7f3939050640 1 --2- 172.21.15.42:0/1498866746 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3934109c50 0x7f393419fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-30T09:16:42.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:42.479+0000 7f393884f640 1 --2- 172.21.15.42:0/1498866746 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3934101e20 0x7f393419a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-30T09:16:42.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:42.479+0000 7f3933fff640 1 --2- 172.21.15.42:0/1498866746 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3934109300 0x7f393419a7e0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f392000d740 tx=0x7f392000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:16:42.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:42.479+0000 7f3931ffb640 1 -- 172.21.15.42:0/1498866746 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3920017070 con 0x7f3934109300 2024-08-30T09:16:42.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:42.480+0000 7f393aab3640 1 -- 172.21.15.42:0/1498866746 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39341a04a0 con 0x7f3934109300 2024-08-30T09:16:42.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:42.480+0000 7f3931ffb640 1 -- 172.21.15.42:0/1498866746 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f392000de50 con 0x7f3934109300 2024-08-30T09:16:42.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:42.480+0000 7f3931ffb640 1 -- 172.21.15.42:0/1498866746 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f392000f620 con 0x7f3934109300 2024-08-30T09:16:42.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:42.480+0000 7f393aab3640 1 -- 172.21.15.42:0/1498866746 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39341a6c00 con 0x7f3934109300 2024-08-30T09:16:42.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:42.481+0000 7f393aab3640 1 -- 172.21.15.42:0/1498866746 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3934102230 con 0x7f3934109300 2024-08-30T09:16:42.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:42.482+0000 7f3931ffb640 1 -- 172.21.15.42:0/1498866746 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7f392001a070 con 0x7f3934109300 2024-08-30T09:16:42.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:42.483+0000 7f3931ffb640 1 --2- 172.21.15.42:0/1498866746 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f39240b4b30 0x7f39240b7040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:16:42.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:42.483+0000 7f393884f640 1 --2- 172.21.15.42:0/1498866746 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f39240b4b30 0x7f39240b7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:16:42.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:42.483+0000 7f3931ffb640 1 -- 172.21.15.42:0/1498866746 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9150+0+0 (secure 0 0 0) 0x7f39200e4fc0 con 0x7f3934109300 2024-08-30T09:16:42.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:42.484+0000 7f393884f640 1 --2- 172.21.15.42:0/1498866746 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f39240b4b30 0x7f39240b7040 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f39280058c0 tx=0x7f3928005ec0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:16:42.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:42.486+0000 7f3931ffb640 1 -- 172.21.15.42:0/1498866746 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f39200abd00 con 0x7f3934109300 2024-08-30T09:16:42.577 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:16:42 smithi167 ceph-mon[22063]: pgmap v851: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 7 op/s 2024-08-30T09:16:42.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:42.711+0000 7f393aab3640 1 -- 172.21.15.42:0/1498866746 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f393419b370 con 0x7f3934109300 2024-08-30T09:16:42.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:42.714+0000 7f3931ffb640 1 -- 172.21.15.42:0/1498866746 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v52) ==== 161+0+0 (secure 0 0 0) 0x7f39200b4240 con 0x7f3934109300 2024-08-30T09:16:42.716 INFO:teuthology.orchestra.run.smithi042.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-08-30T09:16:42.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:42.718+0000 7f393aab3640 1 -- 172.21.15.42:0/1498866746 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f39240b4b30 msgr2=0x7f39240b7040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:16:42.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:42.718+0000 7f393aab3640 1 --2- 172.21.15.42:0/1498866746 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f39240b4b30 0x7f39240b7040 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f39280058c0 tx=0x7f3928005ec0 comp rx=0 tx=0).stop 2024-08-30T09:16:42.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:42.718+0000 7f393aab3640 1 -- 172.21.15.42:0/1498866746 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3934109300 msgr2=0x7f393419a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:16:42.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:42.718+0000 7f393aab3640 1 --2- 172.21.15.42:0/1498866746 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3934109300 0x7f393419a7e0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f392000d740 tx=0x7f392000dc00 comp rx=0 tx=0).stop 2024-08-30T09:16:42.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:42.718+0000 7f393aab3640 1 -- 172.21.15.42:0/1498866746 shutdown_connections 2024-08-30T09:16:42.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:42.718+0000 7f393aab3640 1 --2- 172.21.15.42:0/1498866746 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f39240b4b30 0x7f39240b7040 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:42.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:42.718+0000 7f393aab3640 1 --2- 172.21.15.42:0/1498866746 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3934109c50 0x7f393419fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:42.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:42.718+0000 7f393aab3640 1 --2- 172.21.15.42:0/1498866746 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3934109300 0x7f393419a7e0 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:42.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:42.718+0000 7f393aab3640 1 --2- 172.21.15.42:0/1498866746 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3934101e20 0x7f393419a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:42.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:42.718+0000 7f393aab3640 1 -- 172.21.15.42:0/1498866746 >> 172.21.15.42:0/1498866746 conn(0x7f3934079c60 msgr2=0x7f39341072f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:16:42.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:42.719+0000 7f393aab3640 1 -- 172.21.15.42:0/1498866746 shutdown_connections 2024-08-30T09:16:42.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:42.719+0000 7f393aab3640 1 -- 172.21.15.42:0/1498866746 wait complete. 2024-08-30T09:16:43.077 INFO:journalctl@ceph.mds.e.smithi167.stdout:Aug 30 09:16:42 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mds-e[51862]: ignoring --setuser ceph since I am not root 2024-08-30T09:16:43.078 INFO:journalctl@ceph.mds.e.smithi167.stdout:Aug 30 09:16:42 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mds-e[51862]: ignoring --setgroup ceph since I am not root 2024-08-30T09:16:43.078 INFO:journalctl@ceph.mds.e.smithi167.stdout:Aug 30 09:16:42 smithi167 ceph-mds[51866]: -- 172.21.15.167:0/2622346458 <== mon.0 v2:172.21.15.42:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56130e8b9860 con 0x561312a5f180 2024-08-30T09:16:43.078 INFO:journalctl@ceph.mds.e.smithi167.stdout:Aug 30 09:16:42 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mds-e[51862]: starting mds.e at 2024-08-30T09:16:43.078 INFO:journalctl@ceph.mds.b.smithi167.stdout:Aug 30 09:16:42 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mds-b[52473]: ignoring --setuser ceph since I am not root 2024-08-30T09:16:43.078 INFO:journalctl@ceph.mds.b.smithi167.stdout:Aug 30 09:16:42 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mds-b[52473]: ignoring --setgroup ceph since I am not root 2024-08-30T09:16:43.078 INFO:journalctl@ceph.mds.b.smithi167.stdout:Aug 30 09:16:42 smithi167 ceph-mds[52477]: -- 172.21.15.167:0/3200984194 <== mon.0 v2:172.21.15.42:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ffedebdc20 con 0x55fff2063600 2024-08-30T09:16:43.078 INFO:journalctl@ceph.mds.b.smithi167.stdout:Aug 30 09:16:42 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mds-b[52473]: starting mds.b at 2024-08-30T09:16:43.078 INFO:journalctl@ceph.mds.h.smithi167.stdout:Aug 30 09:16:42 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mds-h[52167]: ignoring --setuser ceph since I am not root 2024-08-30T09:16:43.079 INFO:journalctl@ceph.mds.h.smithi167.stdout:Aug 30 09:16:42 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mds-h[52167]: ignoring --setgroup ceph since I am not root 2024-08-30T09:16:43.079 INFO:journalctl@ceph.mds.h.smithi167.stdout:Aug 30 09:16:42 smithi167 ceph-mds[52171]: -- 172.21.15.167:0/782304475 <== mon.0 v2:172.21.15.42:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563f588c1860 con 0x563f5ca67180 2024-08-30T09:16:43.079 INFO:journalctl@ceph.mds.h.smithi167.stdout:Aug 30 09:16:42 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mds-h[52167]: starting mds.h at 2024-08-30T09:16:43.079 INFO:journalctl@ceph.mds.h.smithi167.stdout:Aug 30 09:16:42 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mds-h[52167]: 2024-08-30T09:16:42.820+0000 7f1d9e518640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6835/0: (98) Address already in use 2024-08-30T09:16:43.100 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:16:42 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mon-a[18648]: 2024-08-30T09:16:42.712+0000 7fdb19116640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-08-30T09:16:43.178 INFO:journalctl@ceph.mds.f.smithi190.stdout:Aug 30 09:16:42 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mds-f[52114]: ignoring --setuser ceph since I am not root 2024-08-30T09:16:43.178 INFO:journalctl@ceph.mds.f.smithi190.stdout:Aug 30 09:16:42 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mds-f[52114]: ignoring --setgroup ceph since I am not root 2024-08-30T09:16:43.179 INFO:journalctl@ceph.mds.f.smithi190.stdout:Aug 30 09:16:42 smithi190 ceph-mds[52118]: -- 172.21.15.190:0/380219381 <== mon.0 v2:172.21.15.42:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e24cd57860 con 0x55e250efda80 2024-08-30T09:16:43.179 INFO:journalctl@ceph.mds.f.smithi190.stdout:Aug 30 09:16:42 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mds-f[52114]: starting mds.f at 2024-08-30T09:16:43.179 INFO:journalctl@ceph.mds.i.smithi190.stdout:Aug 30 09:16:42 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mds-i[52728]: ignoring --setuser ceph since I am not root 2024-08-30T09:16:43.179 INFO:journalctl@ceph.mds.i.smithi190.stdout:Aug 30 09:16:42 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mds-i[52728]: ignoring --setgroup ceph since I am not root 2024-08-30T09:16:43.179 INFO:journalctl@ceph.mds.i.smithi190.stdout:Aug 30 09:16:42 smithi190 ceph-mds[52732]: -- 172.21.15.190:0/1037395732 <== mon.2 v2:172.21.15.167:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561d5b1af860 con 0x561d5f355180 2024-08-30T09:16:43.179 INFO:journalctl@ceph.mds.i.smithi190.stdout:Aug 30 09:16:42 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mds-i[52728]: starting mds.i at 2024-08-30T09:16:43.179 INFO:journalctl@ceph.mds.c.smithi190.stdout:Aug 30 09:16:42 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mds-c[52413]: ignoring --setuser ceph since I am not root 2024-08-30T09:16:43.180 INFO:journalctl@ceph.mds.c.smithi190.stdout:Aug 30 09:16:42 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mds-c[52413]: ignoring --setgroup ceph since I am not root 2024-08-30T09:16:43.180 INFO:journalctl@ceph.mds.c.smithi190.stdout:Aug 30 09:16:42 smithi190 ceph-mds[52417]: -- 172.21.15.190:0/2059479122 <== mon.0 v2:172.21.15.42:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ae7d4d9860 con 0x55ae8167f180 2024-08-30T09:16:43.180 INFO:journalctl@ceph.mds.c.smithi190.stdout:Aug 30 09:16:42 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mds-c[52413]: starting mds.c at 2024-08-30T09:16:43.365 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:16:43 smithi167 ceph-mon[22063]: from='client.? 172.21.15.42:0/1498866746' entity='client.admin' cmd={"prefix": "fs fail", "fs_name": "cephfs"} : dispatch 2024-08-30T09:16:43.365 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:16:43 smithi167 ceph-mon[22063]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-08-30T09:16:43.365 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:16:43 smithi167 ceph-mon[22063]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-08-30T09:16:43.365 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:16:43 smithi167 ceph-mon[22063]: from='client.? 172.21.15.42:0/1498866746' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-08-30T09:16:43.365 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:16:43 smithi167 ceph-mon[22063]: osdmap e84: 12 total, 12 up, 12 in 2024-08-30T09:16:43.365 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:16:43 smithi167 ceph-mon[22063]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-08-30T09:16:43.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:16:43 smithi042 ceph-mon[18652]: from='client.? 172.21.15.42:0/1498866746' entity='client.admin' cmd={"prefix": "fs fail", "fs_name": "cephfs"} : dispatch 2024-08-30T09:16:43.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:16:43 smithi042 ceph-mon[18652]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-08-30T09:16:43.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:16:43 smithi042 ceph-mon[18652]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-08-30T09:16:43.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:16:43 smithi042 ceph-mon[18652]: from='client.? 172.21.15.42:0/1498866746' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-08-30T09:16:43.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:16:43 smithi042 ceph-mon[18652]: osdmap e84: 12 total, 12 up, 12 in 2024-08-30T09:16:43.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:16:43 smithi042 ceph-mon[18652]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-08-30T09:16:43.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:16:43 smithi190 ceph-mon[22001]: from='client.? 172.21.15.42:0/1498866746' entity='client.admin' cmd={"prefix": "fs fail", "fs_name": "cephfs"} : dispatch 2024-08-30T09:16:43.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:16:43 smithi190 ceph-mon[22001]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-08-30T09:16:43.437 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:16:43 smithi190 ceph-mon[22001]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-08-30T09:16:43.437 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:16:43 smithi190 ceph-mon[22001]: from='client.? 172.21.15.42:0/1498866746' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-08-30T09:16:43.437 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:16:43 smithi190 ceph-mon[22001]: osdmap e84: 12 total, 12 up, 12 in 2024-08-30T09:16:43.437 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:16:43 smithi190 ceph-mon[22001]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-08-30T09:16:43.541 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph fs rm cephfs --yes-i-really-mean-it 2024-08-30T09:16:43.799 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T09:16:44.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:44.291+0000 7f1d5f3fc640 1 -- 172.21.15.42:0/2335311429 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1d58102800 msgr2=0x7f1d58102c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:16:44.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:44.291+0000 7f1d5f3fc640 1 --2- 172.21.15.42:0/2335311429 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1d58102800 0x7f1d58102c10 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f1d54009900 tx=0x7f1d5402a410 comp rx=0 tx=0).stop 2024-08-30T09:16:44.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:44.291+0000 7f1d5f3fc640 1 -- 172.21.15.42:0/2335311429 shutdown_connections 2024-08-30T09:16:44.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:44.291+0000 7f1d5f3fc640 1 --2- 172.21.15.42:0/2335311429 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1d58108a90 0x7f1d5810f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:44.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:44.291+0000 7f1d5f3fc640 1 --2- 172.21.15.42:0/2335311429 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1d58108140 0x7f1d58108550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:44.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:44.291+0000 7f1d5f3fc640 1 --2- 172.21.15.42:0/2335311429 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1d58102800 0x7f1d58102c10 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:44.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:44.291+0000 7f1d5f3fc640 1 -- 172.21.15.42:0/2335311429 >> 172.21.15.42:0/2335311429 conn(0x7f1d580794d0 msgr2=0x7f1d580798e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:16:44.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:44.291+0000 7f1d5f3fc640 1 -- 172.21.15.42:0/2335311429 shutdown_connections 2024-08-30T09:16:44.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:44.291+0000 7f1d5f3fc640 1 -- 172.21.15.42:0/2335311429 wait complete. 2024-08-30T09:16:44.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:44.292+0000 7f1d5f3fc640 1 Processor -- start 2024-08-30T09:16:44.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:44.292+0000 7f1d5f3fc640 1 -- start start 2024-08-30T09:16:44.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:44.292+0000 7f1d5f3fc640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1d58102800 0x7f1d581991a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:16:44.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:44.292+0000 7f1d5f3fc640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1d58108140 0x7f1d58199740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:16:44.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:44.292+0000 7f1d5f3fc640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1d58108a90 0x7f1d5819e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:16:44.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:44.292+0000 7f1d5f3fc640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f1d581126f0 con 0x7f1d58108a90 2024-08-30T09:16:44.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:44.292+0000 7f1d5f3fc640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f1d58112570 con 0x7f1d58102800 2024-08-30T09:16:44.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:44.292+0000 7f1d5f3fc640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f1d58112870 con 0x7f1d58108140 2024-08-30T09:16:44.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:44.292+0000 7f1d5d999640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1d58108a90 0x7f1d5819e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:16:44.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:44.292+0000 7f1d5d198640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1d58102800 0x7f1d581991a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:16:44.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:44.292+0000 7f1d5d999640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1d58108a90 0x7f1d5819e9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33046/0 (socket says 172.21.15.42:33046) 2024-08-30T09:16:44.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:44.292+0000 7f1d5d999640 1 -- 172.21.15.42:0/3105371323 learned_addr learned my addr 172.21.15.42:0/3105371323 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:16:44.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:44.293+0000 7f1d5c997640 1 --2- 172.21.15.42:0/3105371323 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1d58108140 0x7f1d58199740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:16:44.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:44.293+0000 7f1d5d999640 1 -- 172.21.15.42:0/3105371323 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1d58108140 msgr2=0x7f1d58199740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:16:44.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:44.293+0000 7f1d5d999640 1 --2- 172.21.15.42:0/3105371323 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1d58108140 0x7f1d58199740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:44.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:44.293+0000 7f1d5d999640 1 -- 172.21.15.42:0/3105371323 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1d58102800 msgr2=0x7f1d581991a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:16:44.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:44.293+0000 7f1d5d999640 1 --2- 172.21.15.42:0/3105371323 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1d58102800 0x7f1d581991a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:44.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:44.293+0000 7f1d5d999640 1 -- 172.21.15.42:0/3105371323 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d5819f050 con 0x7f1d58108a90 2024-08-30T09:16:44.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:44.293+0000 7f1d5d198640 1 --2- 172.21.15.42:0/3105371323 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1d58102800 0x7f1d581991a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-30T09:16:44.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:44.293+0000 7f1d5c997640 1 --2- 172.21.15.42:0/3105371323 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1d58108140 0x7f1d58199740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-30T09:16:44.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:44.293+0000 7f1d5d999640 1 --2- 172.21.15.42:0/3105371323 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1d58108a90 0x7f1d5819e9e0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f1d5000cce0 tx=0x7f1d5000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:16:44.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:44.293+0000 7f1d4e7fc640 1 -- 172.21.15.42:0/3105371323 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d5000f450 con 0x7f1d58108a90 2024-08-30T09:16:44.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:44.293+0000 7f1d4e7fc640 1 -- 172.21.15.42:0/3105371323 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d5000faa0 con 0x7f1d58108a90 2024-08-30T09:16:44.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:44.293+0000 7f1d5f3fc640 1 -- 172.21.15.42:0/3105371323 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d5819f340 con 0x7f1d58108a90 2024-08-30T09:16:44.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:44.293+0000 7f1d4e7fc640 1 -- 172.21.15.42:0/3105371323 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d50018b70 con 0x7f1d58108a90 2024-08-30T09:16:44.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:44.293+0000 7f1d5f3fc640 1 -- 172.21.15.42:0/3105371323 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d581a5b60 con 0x7f1d58108a90 2024-08-30T09:16:44.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:44.294+0000 7f1d5f3fc640 1 -- 172.21.15.42:0/3105371323 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d58102c10 con 0x7f1d58108a90 2024-08-30T09:16:44.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:44.296+0000 7f1d4e7fc640 1 -- 172.21.15.42:0/3105371323 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7f1d50018d10 con 0x7f1d58108a90 2024-08-30T09:16:44.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:44.296+0000 7f1d4e7fc640 1 --2- 172.21.15.42:0/3105371323 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f1d440b4b60 0x7f1d440b7070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:16:44.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:44.297+0000 7f1d5d198640 1 --2- 172.21.15.42:0/3105371323 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f1d440b4b60 0x7f1d440b7070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:16:44.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:44.297+0000 7f1d4e7fc640 1 -- 172.21.15.42:0/3105371323 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9408+0+0 (secure 0 0 0) 0x7f1d500e5a70 con 0x7f1d58108a90 2024-08-30T09:16:44.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:44.297+0000 7f1d5d198640 1 --2- 172.21.15.42:0/3105371323 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f1d440b4b60 0x7f1d440b7070 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f1d54031440 tx=0x7f1d54032ed0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:16:44.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:44.300+0000 7f1d4e7fc640 1 -- 172.21.15.42:0/3105371323 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f1d50014030 con 0x7f1d58108a90 2024-08-30T09:16:44.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:44.525+0000 7f1d5f3fc640 1 -- 172.21.15.42:0/3105371323 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f1d5810a860 con 0x7f1d58108a90 2024-08-30T09:16:44.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:16:44 smithi042 ceph-mon[18652]: pgmap v853: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 21 KiB/s wr, 8 op/s 2024-08-30T09:16:44.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:16:44 smithi042 ceph-mon[18652]: mds.? [v2:172.21.15.190:6836/2329704506,v1:172.21.15.190:6838/2329704506] up:boot 2024-08-30T09:16:44.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:16:44 smithi042 ceph-mon[18652]: mds.? [v2:172.21.15.190:6835/4293082271,v1:172.21.15.190:6839/4293082271] up:boot 2024-08-30T09:16:44.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:16:44 smithi042 ceph-mon[18652]: mds.? [v2:172.21.15.190:6834/3791509344,v1:172.21.15.190:6837/3791509344] up:boot 2024-08-30T09:16:44.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:16:44 smithi042 ceph-mon[18652]: mds.? [v2:172.21.15.167:6836/662534802,v1:172.21.15.167:6838/662534802] up:boot 2024-08-30T09:16:44.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:16:44 smithi042 ceph-mon[18652]: mds.? [v2:172.21.15.167:6834/1702704416,v1:172.21.15.167:6837/1702704416] up:boot 2024-08-30T09:16:44.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:16:44 smithi042 ceph-mon[18652]: mds.? [v2:172.21.15.167:6835/3953078704,v1:172.21.15.167:6839/3953078704] up:boot 2024-08-30T09:16:44.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:16:44 smithi042 ceph-mon[18652]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-08-30T09:16:44.884 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:16:44 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "f"} : dispatch 2024-08-30T09:16:44.884 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:16:44 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "e"} : dispatch 2024-08-30T09:16:44.884 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:16:44 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2024-08-30T09:16:44.884 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:16:44 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-08-30T09:16:44.884 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:16:44 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2024-08-30T09:16:44.884 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:16:44 smithi042 ceph-mon[18652]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-08-30T09:16:44.884 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:16:44 smithi042 ceph-mon[18652]: from='client.? 172.21.15.42:0/3105371323' entity='client.admin' cmd={"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} : dispatch 2024-08-30T09:16:45.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:16:44 smithi167 ceph-mon[22063]: pgmap v853: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 21 KiB/s wr, 8 op/s 2024-08-30T09:16:45.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:16:44 smithi167 ceph-mon[22063]: mds.? [v2:172.21.15.190:6836/2329704506,v1:172.21.15.190:6838/2329704506] up:boot 2024-08-30T09:16:45.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:16:44 smithi167 ceph-mon[22063]: mds.? [v2:172.21.15.190:6835/4293082271,v1:172.21.15.190:6839/4293082271] up:boot 2024-08-30T09:16:45.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:16:44 smithi167 ceph-mon[22063]: mds.? [v2:172.21.15.190:6834/3791509344,v1:172.21.15.190:6837/3791509344] up:boot 2024-08-30T09:16:45.078 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:16:44 smithi167 ceph-mon[22063]: mds.? [v2:172.21.15.167:6836/662534802,v1:172.21.15.167:6838/662534802] up:boot 2024-08-30T09:16:45.078 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:16:44 smithi167 ceph-mon[22063]: mds.? [v2:172.21.15.167:6834/1702704416,v1:172.21.15.167:6837/1702704416] up:boot 2024-08-30T09:16:45.078 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:16:44 smithi167 ceph-mon[22063]: mds.? [v2:172.21.15.167:6835/3953078704,v1:172.21.15.167:6839/3953078704] up:boot 2024-08-30T09:16:45.078 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:16:44 smithi167 ceph-mon[22063]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-08-30T09:16:45.078 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:16:44 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "f"} : dispatch 2024-08-30T09:16:45.078 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:16:44 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "e"} : dispatch 2024-08-30T09:16:45.078 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:16:44 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2024-08-30T09:16:45.078 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:16:44 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-08-30T09:16:45.078 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:16:44 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2024-08-30T09:16:45.078 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:16:44 smithi167 ceph-mon[22063]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-08-30T09:16:45.078 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:16:44 smithi167 ceph-mon[22063]: from='client.? 172.21.15.42:0/3105371323' entity='client.admin' cmd={"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} : dispatch 2024-08-30T09:16:45.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:45.178+0000 7f1d4e7fc640 1 -- 172.21.15.42:0/3105371323 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v54) ==== 104+0+0 (secure 0 0 0) 0x7f1d500ac7a0 con 0x7f1d58108a90 2024-08-30T09:16:45.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:45.181+0000 7f1d5f3fc640 1 -- 172.21.15.42:0/3105371323 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f1d440b4b60 msgr2=0x7f1d440b7070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:16:45.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:45.181+0000 7f1d5f3fc640 1 --2- 172.21.15.42:0/3105371323 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f1d440b4b60 0x7f1d440b7070 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f1d54031440 tx=0x7f1d54032ed0 comp rx=0 tx=0).stop 2024-08-30T09:16:45.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:45.181+0000 7f1d5f3fc640 1 -- 172.21.15.42:0/3105371323 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1d58108a90 msgr2=0x7f1d5819e9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:16:45.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:45.181+0000 7f1d5f3fc640 1 --2- 172.21.15.42:0/3105371323 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1d58108a90 0x7f1d5819e9e0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f1d5000cce0 tx=0x7f1d5000a620 comp rx=0 tx=0).stop 2024-08-30T09:16:45.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:45.181+0000 7f1d5f3fc640 1 -- 172.21.15.42:0/3105371323 shutdown_connections 2024-08-30T09:16:45.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:45.181+0000 7f1d5f3fc640 1 --2- 172.21.15.42:0/3105371323 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f1d440b4b60 0x7f1d440b7070 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:45.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:45.181+0000 7f1d5f3fc640 1 --2- 172.21.15.42:0/3105371323 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1d58108140 0x7f1d58199740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:45.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:45.181+0000 7f1d5f3fc640 1 --2- 172.21.15.42:0/3105371323 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1d58108a90 0x7f1d5819e9e0 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:45.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:45.181+0000 7f1d5f3fc640 1 --2- 172.21.15.42:0/3105371323 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1d58102800 0x7f1d581991a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:45.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:45.181+0000 7f1d5f3fc640 1 -- 172.21.15.42:0/3105371323 >> 172.21.15.42:0/3105371323 conn(0x7f1d580794d0 msgr2=0x7f1d58101250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:16:45.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:45.181+0000 7f1d5f3fc640 1 -- 172.21.15.42:0/3105371323 shutdown_connections 2024-08-30T09:16:45.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:45.181+0000 7f1d5f3fc640 1 -- 172.21.15.42:0/3105371323 wait complete. 2024-08-30T09:16:45.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:16:44 smithi190 ceph-mon[22001]: pgmap v853: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 21 KiB/s wr, 8 op/s 2024-08-30T09:16:45.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:16:44 smithi190 ceph-mon[22001]: mds.? [v2:172.21.15.190:6836/2329704506,v1:172.21.15.190:6838/2329704506] up:boot 2024-08-30T09:16:45.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:16:44 smithi190 ceph-mon[22001]: mds.? [v2:172.21.15.190:6835/4293082271,v1:172.21.15.190:6839/4293082271] up:boot 2024-08-30T09:16:45.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:16:44 smithi190 ceph-mon[22001]: mds.? [v2:172.21.15.190:6834/3791509344,v1:172.21.15.190:6837/3791509344] up:boot 2024-08-30T09:16:45.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:16:44 smithi190 ceph-mon[22001]: mds.? [v2:172.21.15.167:6836/662534802,v1:172.21.15.167:6838/662534802] up:boot 2024-08-30T09:16:45.187 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:16:44 smithi190 ceph-mon[22001]: mds.? [v2:172.21.15.167:6834/1702704416,v1:172.21.15.167:6837/1702704416] up:boot 2024-08-30T09:16:45.187 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:16:44 smithi190 ceph-mon[22001]: mds.? [v2:172.21.15.167:6835/3953078704,v1:172.21.15.167:6839/3953078704] up:boot 2024-08-30T09:16:45.187 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:16:44 smithi190 ceph-mon[22001]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-08-30T09:16:45.187 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:16:44 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "f"} : dispatch 2024-08-30T09:16:45.187 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:16:44 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "e"} : dispatch 2024-08-30T09:16:45.187 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:16:44 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2024-08-30T09:16:45.187 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:16:44 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-08-30T09:16:45.187 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:16:44 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2024-08-30T09:16:45.187 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:16:44 smithi190 ceph-mon[22001]: from='mgr.14150 172.21.15.42:0/1634153318' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-08-30T09:16:45.187 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:16:44 smithi190 ceph-mon[22001]: from='client.? 172.21.15.42:0/3105371323' entity='client.admin' cmd={"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} : dispatch 2024-08-30T09:16:45.979 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:16:45 smithi042 ceph-mon[18652]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-08-30T09:16:45.979 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:16:45 smithi042 ceph-mon[18652]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-08-30T09:16:45.979 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:16:45 smithi042 ceph-mon[18652]: Cluster is now healthy 2024-08-30T09:16:45.979 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:16:45 smithi042 ceph-mon[18652]: from='client.? 172.21.15.42:0/3105371323' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-08-30T09:16:45.979 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:16:45 smithi042 ceph-mon[18652]: fsmap 12 up:standby 2024-08-30T09:16:45.980 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-08-30T09:16:46.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:16:45 smithi167 ceph-mon[22063]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-08-30T09:16:46.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:16:45 smithi167 ceph-mon[22063]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-08-30T09:16:46.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:16:45 smithi167 ceph-mon[22063]: Cluster is now healthy 2024-08-30T09:16:46.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:16:45 smithi167 ceph-mon[22063]: from='client.? 172.21.15.42:0/3105371323' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-08-30T09:16:46.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:16:45 smithi167 ceph-mon[22063]: fsmap 12 up:standby 2024-08-30T09:16:46.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:16:45 smithi190 ceph-mon[22001]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-08-30T09:16:46.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:16:45 smithi190 ceph-mon[22001]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-08-30T09:16:46.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:16:45 smithi190 ceph-mon[22001]: Cluster is now healthy 2024-08-30T09:16:46.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:16:45 smithi190 ceph-mon[22001]: from='client.? 172.21.15.42:0/3105371323' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-08-30T09:16:46.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:16:45 smithi190 ceph-mon[22001]: fsmap 12 up:standby 2024-08-30T09:16:46.246 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T09:16:46.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:16:46 smithi042 ceph-mon[18652]: pgmap v854: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 21 KiB/s wr, 8 op/s 2024-08-30T09:16:46.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:46.778+0000 7f220178e640 1 -- 172.21.15.42:0/2986020452 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f21fc108150 msgr2=0x7f21fc108560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:16:46.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:46.778+0000 7f220178e640 1 --2- 172.21.15.42:0/2986020452 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f21fc108150 0x7f21fc108560 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f21f0009900 tx=0x7f21f002a410 comp rx=0 tx=0).stop 2024-08-30T09:16:46.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:46.778+0000 7f220178e640 1 -- 172.21.15.42:0/2986020452 shutdown_connections 2024-08-30T09:16:46.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:46.778+0000 7f220178e640 1 --2- 172.21.15.42:0/2986020452 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f21fc108aa0 0x7f21fc10f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:46.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:46.778+0000 7f220178e640 1 --2- 172.21.15.42:0/2986020452 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f21fc108150 0x7f21fc108560 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:46.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:46.778+0000 7f220178e640 1 --2- 172.21.15.42:0/2986020452 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f21fc102810 0x7f21fc102c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:46.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:46.778+0000 7f220178e640 1 -- 172.21.15.42:0/2986020452 >> 172.21.15.42:0/2986020452 conn(0x7f21fc0794d0 msgr2=0x7f21fc0798e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:16:46.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:46.778+0000 7f220178e640 1 -- 172.21.15.42:0/2986020452 shutdown_connections 2024-08-30T09:16:46.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:46.778+0000 7f220178e640 1 -- 172.21.15.42:0/2986020452 wait complete. 2024-08-30T09:16:46.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:46.778+0000 7f220178e640 1 Processor -- start 2024-08-30T09:16:46.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:46.779+0000 7f220178e640 1 -- start start 2024-08-30T09:16:46.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:46.779+0000 7f220178e640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f21fc102810 0x7f21fc1991c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:16:46.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:46.779+0000 7f220178e640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f21fc108150 0x7f21fc199760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:16:46.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:46.779+0000 7f220178e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f21fc108aa0 0x7f21fc19ea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:16:46.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:46.779+0000 7f220178e640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f21fc1126f0 con 0x7f21fc102810 2024-08-30T09:16:46.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:46.779+0000 7f220178e640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f21fc112570 con 0x7f21fc108150 2024-08-30T09:16:46.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:46.779+0000 7f220178e640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f21fc112870 con 0x7f21fc108aa0 2024-08-30T09:16:46.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:46.779+0000 7f21faffd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f21fc102810 0x7f21fc1991c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:16:46.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:46.779+0000 7f21faffd640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f21fc102810 0x7f21fc1991c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33066/0 (socket says 172.21.15.42:33066) 2024-08-30T09:16:46.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:46.779+0000 7f21faffd640 1 -- 172.21.15.42:0/2875279143 learned_addr learned my addr 172.21.15.42:0/2875279143 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:16:46.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:46.779+0000 7f21fb7fe640 1 --2- 172.21.15.42:0/2875279143 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f21fc108aa0 0x7f21fc19ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:16:46.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:46.779+0000 7f21fa7fc640 1 --2- 172.21.15.42:0/2875279143 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f21fc108150 0x7f21fc199760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:16:46.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:46.780+0000 7f21faffd640 1 -- 172.21.15.42:0/2875279143 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f21fc108aa0 msgr2=0x7f21fc19ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:16:46.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:46.780+0000 7f21faffd640 1 --2- 172.21.15.42:0/2875279143 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f21fc108aa0 0x7f21fc19ea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:46.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:46.780+0000 7f21faffd640 1 -- 172.21.15.42:0/2875279143 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f21fc108150 msgr2=0x7f21fc199760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:16:46.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:46.780+0000 7f21faffd640 1 --2- 172.21.15.42:0/2875279143 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f21fc108150 0x7f21fc199760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:46.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:46.780+0000 7f21faffd640 1 -- 172.21.15.42:0/2875279143 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21fc19f130 con 0x7f21fc102810 2024-08-30T09:16:46.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:46.780+0000 7f21fa7fc640 1 --2- 172.21.15.42:0/2875279143 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f21fc108150 0x7f21fc199760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-30T09:16:46.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:46.780+0000 7f21faffd640 1 --2- 172.21.15.42:0/2875279143 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f21fc102810 0x7f21fc1991c0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f21ec00e9f0 tx=0x7f21ec00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:16:46.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:46.780+0000 7f21e3fff640 1 -- 172.21.15.42:0/2875279143 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21ec00cd80 con 0x7f21fc102810 2024-08-30T09:16:46.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:46.780+0000 7f21e3fff640 1 -- 172.21.15.42:0/2875279143 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f21ec0054d0 con 0x7f21fc102810 2024-08-30T09:16:46.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:46.780+0000 7f21e3fff640 1 -- 172.21.15.42:0/2875279143 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21ec010690 con 0x7f21fc102810 2024-08-30T09:16:46.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:46.780+0000 7f220178e640 1 -- 172.21.15.42:0/2875279143 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21fc19f420 con 0x7f21fc102810 2024-08-30T09:16:46.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:46.780+0000 7f220178e640 1 -- 172.21.15.42:0/2875279143 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21fc1a5b80 con 0x7f21fc102810 2024-08-30T09:16:46.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:46.781+0000 7f220178e640 1 -- 172.21.15.42:0/2875279143 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21fc102c20 con 0x7f21fc102810 2024-08-30T09:16:46.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:46.783+0000 7f21e3fff640 1 -- 172.21.15.42:0/2875279143 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7f21ec004e30 con 0x7f21fc102810 2024-08-30T09:16:46.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:46.783+0000 7f21e3fff640 1 --2- 172.21.15.42:0/2875279143 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f21e40b4b30 0x7f21e40b7040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:16:46.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:46.784+0000 7f21fa7fc640 1 --2- 172.21.15.42:0/2875279143 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f21e40b4b30 0x7f21e40b7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:16:46.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:46.784+0000 7f21e3fff640 1 -- 172.21.15.42:0/2875279143 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9408+0+0 (secure 0 0 0) 0x7f21ec0e5c80 con 0x7f21fc102810 2024-08-30T09:16:46.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:46.784+0000 7f21fa7fc640 1 --2- 172.21.15.42:0/2875279143 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f21e40b4b30 0x7f21e40b7040 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f21f0000c00 tx=0x7f21f0032cc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:16:46.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:46.787+0000 7f21e3fff640 1 -- 172.21.15.42:0/2875279143 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f21ec056130 con 0x7f21fc102810 2024-08-30T09:16:46.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:16:46 smithi190 ceph-mon[22001]: pgmap v854: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 21 KiB/s wr, 8 op/s 2024-08-30T09:16:46.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:46.977+0000 7f220178e640 1 -- 172.21.15.42:0/2875279143 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f21fc19a2f0 con 0x7f21fc102810 2024-08-30T09:16:47.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:16:46 smithi167 ceph-mon[22063]: pgmap v854: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 21 KiB/s wr, 8 op/s 2024-08-30T09:16:47.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:47.725+0000 7f21e3fff640 1 -- 172.21.15.42:0/2875279143 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v85) ==== 181+0+0 (secure 0 0 0) 0x7f21fc19a2f0 con 0x7f21fc102810 2024-08-30T09:16:47.728 INFO:teuthology.orchestra.run.smithi042.stderr:pool 'cephfs_metadata' removed 2024-08-30T09:16:47.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:47.729+0000 7f220178e640 1 -- 172.21.15.42:0/2875279143 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f21e40b4b30 msgr2=0x7f21e40b7040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:16:47.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:47.729+0000 7f220178e640 1 --2- 172.21.15.42:0/2875279143 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f21e40b4b30 0x7f21e40b7040 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f21f0000c00 tx=0x7f21f0032cc0 comp rx=0 tx=0).stop 2024-08-30T09:16:47.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:47.729+0000 7f220178e640 1 -- 172.21.15.42:0/2875279143 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f21fc102810 msgr2=0x7f21fc1991c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:16:47.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:47.729+0000 7f220178e640 1 --2- 172.21.15.42:0/2875279143 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f21fc102810 0x7f21fc1991c0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f21ec00e9f0 tx=0x7f21ec00edf0 comp rx=0 tx=0).stop 2024-08-30T09:16:47.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:47.729+0000 7f220178e640 1 -- 172.21.15.42:0/2875279143 shutdown_connections 2024-08-30T09:16:47.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:47.729+0000 7f220178e640 1 --2- 172.21.15.42:0/2875279143 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f21e40b4b30 0x7f21e40b7040 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:47.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:47.729+0000 7f220178e640 1 --2- 172.21.15.42:0/2875279143 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f21fc108aa0 0x7f21fc19ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:47.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:47.729+0000 7f220178e640 1 --2- 172.21.15.42:0/2875279143 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f21fc108150 0x7f21fc199760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:47.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:47.729+0000 7f220178e640 1 --2- 172.21.15.42:0/2875279143 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f21fc102810 0x7f21fc1991c0 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:47.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:47.729+0000 7f220178e640 1 -- 172.21.15.42:0/2875279143 >> 172.21.15.42:0/2875279143 conn(0x7f21fc0794d0 msgr2=0x7f21fc104280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:16:47.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:47.730+0000 7f220178e640 1 -- 172.21.15.42:0/2875279143 shutdown_connections 2024-08-30T09:16:47.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:47.730+0000 7f220178e640 1 -- 172.21.15.42:0/2875279143 wait complete. 2024-08-30T09:16:47.834 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:16:47 smithi042 ceph-mon[18652]: from='client.? 172.21.15.42:0/2875279143' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} : dispatch 2024-08-30T09:16:48.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:16:47 smithi167 ceph-mon[22063]: from='client.? 172.21.15.42:0/2875279143' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} : dispatch 2024-08-30T09:16:48.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:16:47 smithi190 ceph-mon[22001]: from='client.? 172.21.15.42:0/2875279143' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} : dispatch 2024-08-30T09:16:48.458 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-08-30T09:16:48.712 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T09:16:48.984 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:16:48 smithi042 ceph-mon[18652]: pgmap v855: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 21 KiB/s wr, 7 op/s 2024-08-30T09:16:48.984 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:16:48 smithi042 ceph-mon[18652]: from='client.? 172.21.15.42:0/2875279143' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-08-30T09:16:48.985 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:16:48 smithi042 ceph-mon[18652]: osdmap e85: 12 total, 12 up, 12 in 2024-08-30T09:16:49.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:16:48 smithi167 ceph-mon[22063]: pgmap v855: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 21 KiB/s wr, 7 op/s 2024-08-30T09:16:49.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:16:48 smithi167 ceph-mon[22063]: from='client.? 172.21.15.42:0/2875279143' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-08-30T09:16:49.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:16:48 smithi167 ceph-mon[22063]: osdmap e85: 12 total, 12 up, 12 in 2024-08-30T09:16:49.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:16:48 smithi190 ceph-mon[22001]: pgmap v855: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 21 KiB/s wr, 7 op/s 2024-08-30T09:16:49.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:16:48 smithi190 ceph-mon[22001]: from='client.? 172.21.15.42:0/2875279143' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-08-30T09:16:49.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:16:48 smithi190 ceph-mon[22001]: osdmap e85: 12 total, 12 up, 12 in 2024-08-30T09:16:49.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:49.204+0000 7fee83c7d640 1 -- 172.21.15.42:0/457124826 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fee7c108a90 msgr2=0x7fee7c10f2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:16:49.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:49.204+0000 7fee83c7d640 1 --2- 172.21.15.42:0/457124826 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fee7c108a90 0x7fee7c10f2c0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7fee74009ca0 tx=0x7fee7402a8c0 comp rx=0 tx=0).stop 2024-08-30T09:16:49.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:49.204+0000 7fee83c7d640 1 -- 172.21.15.42:0/457124826 shutdown_connections 2024-08-30T09:16:49.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:49.204+0000 7fee83c7d640 1 --2- 172.21.15.42:0/457124826 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fee7c108140 0x7fee7c108550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:49.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:49.204+0000 7fee83c7d640 1 --2- 172.21.15.42:0/457124826 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fee7c108a90 0x7fee7c10f2c0 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:49.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:49.204+0000 7fee83c7d640 1 --2- 172.21.15.42:0/457124826 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fee7c102800 0x7fee7c102c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:49.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:49.204+0000 7fee83c7d640 1 -- 172.21.15.42:0/457124826 >> 172.21.15.42:0/457124826 conn(0x7fee7c0794d0 msgr2=0x7fee7c0798e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:16:49.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:49.204+0000 7fee83c7d640 1 -- 172.21.15.42:0/457124826 shutdown_connections 2024-08-30T09:16:49.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:49.204+0000 7fee83c7d640 1 -- 172.21.15.42:0/457124826 wait complete. 2024-08-30T09:16:49.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:49.205+0000 7fee83c7d640 1 Processor -- start 2024-08-30T09:16:49.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:49.205+0000 7fee83c7d640 1 -- start start 2024-08-30T09:16:49.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:49.205+0000 7fee83c7d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fee7c102800 0x7fee7c199160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:16:49.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:49.205+0000 7fee83c7d640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fee7c108140 0x7fee7c199700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:16:49.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:49.205+0000 7fee83c7d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fee7c108a90 0x7fee7c19e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:16:49.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:49.205+0000 7fee83c7d640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fee7c112710 con 0x7fee7c108140 2024-08-30T09:16:49.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:49.205+0000 7fee83c7d640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fee7c112590 con 0x7fee7c102800 2024-08-30T09:16:49.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:49.205+0000 7fee83c7d640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fee7c112890 con 0x7fee7c108a90 2024-08-30T09:16:49.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:49.205+0000 7fee81218640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fee7c108140 0x7fee7c199700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:16:49.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:49.206+0000 7fee81218640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fee7c108140 0x7fee7c199700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:48600/0 (socket says 172.21.15.42:48600) 2024-08-30T09:16:49.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:49.206+0000 7fee81218640 1 -- 172.21.15.42:0/1310414528 learned_addr learned my addr 172.21.15.42:0/1310414528 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:16:49.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:49.206+0000 7fee81a19640 1 --2- 172.21.15.42:0/1310414528 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fee7c102800 0x7fee7c199160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:16:49.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:49.206+0000 7fee8221a640 1 --2- 172.21.15.42:0/1310414528 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fee7c108a90 0x7fee7c19e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:16:49.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:49.206+0000 7fee81218640 1 -- 172.21.15.42:0/1310414528 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fee7c108a90 msgr2=0x7fee7c19e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:16:49.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:49.206+0000 7fee81218640 1 --2- 172.21.15.42:0/1310414528 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fee7c108a90 0x7fee7c19e9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:49.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:49.206+0000 7fee81218640 1 -- 172.21.15.42:0/1310414528 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fee7c102800 msgr2=0x7fee7c199160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:16:49.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:49.206+0000 7fee81218640 1 --2- 172.21.15.42:0/1310414528 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fee7c102800 0x7fee7c199160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:49.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:49.206+0000 7fee81218640 1 -- 172.21.15.42:0/1310414528 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee7c19f010 con 0x7fee7c108140 2024-08-30T09:16:49.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:49.206+0000 7fee8221a640 1 --2- 172.21.15.42:0/1310414528 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fee7c108a90 0x7fee7c19e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-30T09:16:49.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:49.206+0000 7fee81a19640 1 --2- 172.21.15.42:0/1310414528 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fee7c102800 0x7fee7c199160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-30T09:16:49.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:49.206+0000 7fee81218640 1 --2- 172.21.15.42:0/1310414528 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fee7c108140 0x7fee7c199700 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7fee6c00d740 tx=0x7fee6c00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:16:49.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:49.206+0000 7fee72ffd640 1 -- 172.21.15.42:0/1310414528 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee6c017070 con 0x7fee7c108140 2024-08-30T09:16:49.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:49.206+0000 7fee72ffd640 1 -- 172.21.15.42:0/1310414528 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee6c00de50 con 0x7fee7c108140 2024-08-30T09:16:49.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:49.206+0000 7fee83c7d640 1 -- 172.21.15.42:0/1310414528 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee7c19f300 con 0x7fee7c108140 2024-08-30T09:16:49.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:49.206+0000 7fee72ffd640 1 -- 172.21.15.42:0/1310414528 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee6c004ce0 con 0x7fee7c108140 2024-08-30T09:16:49.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:49.206+0000 7fee83c7d640 1 -- 172.21.15.42:0/1310414528 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee7c1a5b20 con 0x7fee7c108140 2024-08-30T09:16:49.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:49.208+0000 7fee83c7d640 1 -- 172.21.15.42:0/1310414528 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee7c102c10 con 0x7fee7c108140 2024-08-30T09:16:49.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:49.209+0000 7fee72ffd640 1 -- 172.21.15.42:0/1310414528 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7fee6c004e80 con 0x7fee7c108140 2024-08-30T09:16:49.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:49.210+0000 7fee72ffd640 1 --2- 172.21.15.42:0/1310414528 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fee680b4b60 0x7fee680b7070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:16:49.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:49.210+0000 7fee81a19640 1 --2- 172.21.15.42:0/1310414528 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fee680b4b60 0x7fee680b7070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:16:49.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:49.210+0000 7fee72ffd640 1 -- 172.21.15.42:0/1310414528 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8854+0+0 (secure 0 0 0) 0x7fee6c0e5220 con 0x7fee7c108140 2024-08-30T09:16:49.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:49.210+0000 7fee81a19640 1 --2- 172.21.15.42:0/1310414528 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fee680b4b60 0x7fee680b7070 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fee7800abe0 tx=0x7fee7800a5c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:16:49.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:49.213+0000 7fee72ffd640 1 -- 172.21.15.42:0/1310414528 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7fee6c0ac080 con 0x7fee7c108140 2024-08-30T09:16:49.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:49.408+0000 7fee83c7d640 1 -- 172.21.15.42:0/1310414528 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fee7c10a880 con 0x7fee7c108140 2024-08-30T09:16:49.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:49.733+0000 7fee72ffd640 1 -- 172.21.15.42:0/1310414528 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v86) ==== 169+0+0 (secure 0 0 0) 0x7fee6c0ab980 con 0x7fee7c108140 2024-08-30T09:16:49.735 INFO:teuthology.orchestra.run.smithi042.stderr:pool 'cephfs_data' removed 2024-08-30T09:16:49.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:49.736+0000 7fee83c7d640 1 -- 172.21.15.42:0/1310414528 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fee680b4b60 msgr2=0x7fee680b7070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:16:49.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:49.736+0000 7fee83c7d640 1 --2- 172.21.15.42:0/1310414528 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fee680b4b60 0x7fee680b7070 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fee7800abe0 tx=0x7fee7800a5c0 comp rx=0 tx=0).stop 2024-08-30T09:16:49.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:49.736+0000 7fee83c7d640 1 -- 172.21.15.42:0/1310414528 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fee7c108140 msgr2=0x7fee7c199700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:16:49.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:49.736+0000 7fee83c7d640 1 --2- 172.21.15.42:0/1310414528 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fee7c108140 0x7fee7c199700 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7fee6c00d740 tx=0x7fee6c00dc00 comp rx=0 tx=0).stop 2024-08-30T09:16:49.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:49.736+0000 7fee83c7d640 1 -- 172.21.15.42:0/1310414528 shutdown_connections 2024-08-30T09:16:49.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:49.736+0000 7fee83c7d640 1 --2- 172.21.15.42:0/1310414528 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7fee680b4b60 0x7fee680b7070 secure :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fee7800abe0 tx=0x7fee7800a5c0 comp rx=0 tx=0).stop 2024-08-30T09:16:49.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:49.736+0000 7fee83c7d640 1 --2- 172.21.15.42:0/1310414528 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fee7c108a90 0x7fee7c19e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:49.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:49.736+0000 7fee83c7d640 1 --2- 172.21.15.42:0/1310414528 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fee7c108140 0x7fee7c199700 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:49.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:49.736+0000 7fee83c7d640 1 --2- 172.21.15.42:0/1310414528 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fee7c102800 0x7fee7c199160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:49.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:49.736+0000 7fee83c7d640 1 -- 172.21.15.42:0/1310414528 >> 172.21.15.42:0/1310414528 conn(0x7fee7c0794d0 msgr2=0x7fee7c105040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:16:49.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:49.738+0000 7fee83c7d640 1 -- 172.21.15.42:0/1310414528 shutdown_connections 2024-08-30T09:16:49.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:49.738+0000 7fee83c7d640 1 -- 172.21.15.42:0/1310414528 wait complete. 2024-08-30T09:16:50.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:16:49 smithi167 ceph-mon[22063]: from='client.? 172.21.15.42:0/1310414528' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} : dispatch 2024-08-30T09:16:50.133 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:16:49 smithi042 ceph-mon[18652]: from='client.? 172.21.15.42:0/1310414528' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} : dispatch 2024-08-30T09:16:50.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:16:49 smithi190 ceph-mon[22001]: from='client.? 172.21.15.42:0/1310414528' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} : dispatch 2024-08-30T09:16:50.669 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 -- ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-08-30T09:16:50.930 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/mon.a/config 2024-08-30T09:16:51.051 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:16:50 smithi042 ceph-mon[18652]: pgmap v857: 97 pgs: 97 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail 2024-08-30T09:16:51.051 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:16:50 smithi042 ceph-mon[18652]: from='client.? 172.21.15.42:0/1310414528' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-08-30T09:16:51.051 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:16:50 smithi042 ceph-mon[18652]: osdmap e86: 12 total, 12 up, 12 in 2024-08-30T09:16:51.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:16:50 smithi167 ceph-mon[22063]: pgmap v857: 97 pgs: 97 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail 2024-08-30T09:16:51.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:16:50 smithi167 ceph-mon[22063]: from='client.? 172.21.15.42:0/1310414528' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-08-30T09:16:51.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:16:50 smithi167 ceph-mon[22063]: osdmap e86: 12 total, 12 up, 12 in 2024-08-30T09:16:51.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:16:50 smithi190 ceph-mon[22001]: pgmap v857: 97 pgs: 97 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail 2024-08-30T09:16:51.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:16:50 smithi190 ceph-mon[22001]: from='client.? 172.21.15.42:0/1310414528' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-08-30T09:16:51.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:16:50 smithi190 ceph-mon[22001]: osdmap e86: 12 total, 12 up, 12 in 2024-08-30T09:16:51.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:51.399+0000 7f30c53eb640 1 -- 172.21.15.42:0/2409886076 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f30c01050a0 msgr2=0x7f30c010d5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:16:51.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:51.399+0000 7f30c53eb640 1 --2- 172.21.15.42:0/2409886076 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f30c01050a0 0x7f30c010d5f0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f30b4009900 tx=0x7f30b402a410 comp rx=0 tx=0).stop 2024-08-30T09:16:51.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:51.399+0000 7f30c53eb640 1 -- 172.21.15.42:0/2409886076 shutdown_connections 2024-08-30T09:16:51.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:51.399+0000 7f30c53eb640 1 --2- 172.21.15.42:0/2409886076 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f30c010dbb0 0x7f30c01100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:51.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:51.399+0000 7f30c53eb640 1 --2- 172.21.15.42:0/2409886076 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f30c01050a0 0x7f30c010d5f0 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:51.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:51.399+0000 7f30c53eb640 1 --2- 172.21.15.42:0/2409886076 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f30c0104750 0x7f30c0104b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:51.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:51.399+0000 7f30c53eb640 1 -- 172.21.15.42:0/2409886076 >> 172.21.15.42:0/2409886076 conn(0x7f30c00794d0 msgr2=0x7f30c00798e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:16:51.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:51.399+0000 7f30c53eb640 1 -- 172.21.15.42:0/2409886076 shutdown_connections 2024-08-30T09:16:51.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:51.399+0000 7f30c53eb640 1 -- 172.21.15.42:0/2409886076 wait complete. 2024-08-30T09:16:51.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:51.400+0000 7f30c53eb640 1 Processor -- start 2024-08-30T09:16:51.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:51.400+0000 7f30c53eb640 1 -- start start 2024-08-30T09:16:51.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:51.400+0000 7f30c53eb640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f30c0104750 0x7f30c0102490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:16:51.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:51.400+0000 7f30c53eb640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f30c01050a0 0x7f30c01029d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:16:51.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:51.400+0000 7f30c53eb640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f30c010dbb0 0x7f30c0102f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:16:51.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:51.400+0000 7f30c53eb640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f30c01138f0 con 0x7f30c01050a0 2024-08-30T09:16:51.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:51.400+0000 7f30c53eb640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f30c0113770 con 0x7f30c010dbb0 2024-08-30T09:16:51.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:51.400+0000 7f30c53eb640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f30c0113a70 con 0x7f30c0104750 2024-08-30T09:16:51.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:51.400+0000 7f30be7fc640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f30c01050a0 0x7f30c01029d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:16:51.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:51.400+0000 7f30be7fc640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f30c01050a0 0x7f30c01029d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:48606/0 (socket says 172.21.15.42:48606) 2024-08-30T09:16:51.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:51.400+0000 7f30be7fc640 1 -- 172.21.15.42:0/1567301342 learned_addr learned my addr 172.21.15.42:0/1567301342 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-08-30T09:16:51.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:51.400+0000 7f30beffd640 1 --2- 172.21.15.42:0/1567301342 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f30c0104750 0x7f30c0102490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:16:51.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:51.401+0000 7f30bf7fe640 1 --2- 172.21.15.42:0/1567301342 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f30c010dbb0 0x7f30c0102f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:16:51.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:51.401+0000 7f30be7fc640 1 -- 172.21.15.42:0/1567301342 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f30c0104750 msgr2=0x7f30c0102490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:16:51.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:51.401+0000 7f30be7fc640 1 --2- 172.21.15.42:0/1567301342 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f30c0104750 0x7f30c0102490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:51.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:51.401+0000 7f30be7fc640 1 -- 172.21.15.42:0/1567301342 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f30c010dbb0 msgr2=0x7f30c0102f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:16:51.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:51.401+0000 7f30be7fc640 1 --2- 172.21.15.42:0/1567301342 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f30c010dbb0 0x7f30c0102f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:51.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:51.401+0000 7f30be7fc640 1 -- 172.21.15.42:0/1567301342 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30c01a8640 con 0x7f30c01050a0 2024-08-30T09:16:51.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:51.401+0000 7f30be7fc640 1 --2- 172.21.15.42:0/1567301342 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f30c01050a0 0x7f30c01029d0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f30b402a930 tx=0x7f30b4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:16:51.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:51.401+0000 7f30a7fff640 1 -- 172.21.15.42:0/1567301342 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30b403f030 con 0x7f30c01050a0 2024-08-30T09:16:51.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:51.401+0000 7f30a7fff640 1 -- 172.21.15.42:0/1567301342 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f30b4003aa0 con 0x7f30c01050a0 2024-08-30T09:16:51.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:51.401+0000 7f30c53eb640 1 -- 172.21.15.42:0/1567301342 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30c01a88d0 con 0x7f30c01050a0 2024-08-30T09:16:51.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:51.401+0000 7f30a7fff640 1 -- 172.21.15.42:0/1567301342 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30b4035070 con 0x7f30c01050a0 2024-08-30T09:16:51.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:51.401+0000 7f30c53eb640 1 -- 172.21.15.42:0/1567301342 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30c01a8e30 con 0x7f30c01050a0 2024-08-30T09:16:51.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:51.402+0000 7f30c53eb640 1 -- 172.21.15.42:0/1567301342 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30c0104be0 con 0x7f30c01050a0 2024-08-30T09:16:51.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:51.405+0000 7f30a7fff640 1 -- 172.21.15.42:0/1567301342 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) ==== 152577+0+0 (secure 0 0 0) 0x7f30b4042030 con 0x7f30c01050a0 2024-08-30T09:16:51.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:51.405+0000 7f30a7fff640 1 --2- 172.21.15.42:0/1567301342 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f30a80b4b30 0x7f30a80b7040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-30T09:16:51.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:51.405+0000 7f30beffd640 1 --2- 172.21.15.42:0/1567301342 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f30a80b4b30 0x7f30a80b7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-30T09:16:51.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:51.405+0000 7f30a7fff640 1 -- 172.21.15.42:0/1567301342 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8324+0+0 (secure 0 0 0) 0x7f30b4105dc0 con 0x7f30c01050a0 2024-08-30T09:16:51.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:51.406+0000 7f30beffd640 1 --2- 172.21.15.42:0/1567301342 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f30a80b4b30 0x7f30a80b7040 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f30c0103ee0 tx=0x7f30b0009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-30T09:16:51.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:51.408+0000 7f30a7fff640 1 -- 172.21.15.42:0/1567301342 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f30b40cce30 con 0x7f30c01050a0 2024-08-30T09:16:51.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:51.597+0000 7f30c53eb640 1 -- 172.21.15.42:0/1567301342 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) -- 0x7f30c002e430 con 0x7f30c01050a0 2024-08-30T09:16:51.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:51.740+0000 7f30a7fff640 1 -- 172.21.15.42:0/1567301342 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v87) ==== 178+0+0 (secure 0 0 0) 0x7f30b40cc730 con 0x7f30c01050a0 2024-08-30T09:16:51.743 INFO:teuthology.orchestra.run.smithi042.stderr:pool 'cephfs_data_ec' removed 2024-08-30T09:16:51.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:51.744+0000 7f30c53eb640 1 -- 172.21.15.42:0/1567301342 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f30a80b4b30 msgr2=0x7f30a80b7040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:16:51.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:51.744+0000 7f30c53eb640 1 --2- 172.21.15.42:0/1567301342 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f30a80b4b30 0x7f30a80b7040 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f30c0103ee0 tx=0x7f30b0009110 comp rx=0 tx=0).stop 2024-08-30T09:16:51.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:51.744+0000 7f30c53eb640 1 -- 172.21.15.42:0/1567301342 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f30c01050a0 msgr2=0x7f30c01029d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-30T09:16:51.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:51.744+0000 7f30c53eb640 1 --2- 172.21.15.42:0/1567301342 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f30c01050a0 0x7f30c01029d0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f30b402a930 tx=0x7f30b4004270 comp rx=0 tx=0).stop 2024-08-30T09:16:51.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:51.744+0000 7f30c53eb640 1 -- 172.21.15.42:0/1567301342 shutdown_connections 2024-08-30T09:16:51.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:51.744+0000 7f30c53eb640 1 --2- 172.21.15.42:0/1567301342 >> [v2:172.21.15.42:6800/3005140902,v1:172.21.15.42:6801/3005140902] conn(0x7f30a80b4b30 0x7f30a80b7040 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:51.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:51.744+0000 7f30c53eb640 1 --2- 172.21.15.42:0/1567301342 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f30c010dbb0 0x7f30c0102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:51.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:51.744+0000 7f30c53eb640 1 --2- 172.21.15.42:0/1567301342 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f30c01050a0 0x7f30c01029d0 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:51.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:51.744+0000 7f30c53eb640 1 --2- 172.21.15.42:0/1567301342 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f30c0104750 0x7f30c0102490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-30T09:16:51.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:51.744+0000 7f30c53eb640 1 -- 172.21.15.42:0/1567301342 >> 172.21.15.42:0/1567301342 conn(0x7f30c00794d0 msgr2=0x7f30c0100690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-30T09:16:51.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:51.744+0000 7f30c53eb640 1 -- 172.21.15.42:0/1567301342 shutdown_connections 2024-08-30T09:16:51.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-08-30T09:16:51.744+0000 7f30c53eb640 1 -- 172.21.15.42:0/1567301342 wait complete. 2024-08-30T09:16:51.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:16:51 smithi042 ceph-mon[18652]: from='client.? 172.21.15.42:0/1567301342' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} : dispatch 2024-08-30T09:16:52.077 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:16:51 smithi167 ceph-mon[22063]: from='client.? 172.21.15.42:0/1567301342' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} : dispatch 2024-08-30T09:16:52.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:16:51 smithi190 ceph-mon[22001]: from='client.? 172.21.15.42:0/1567301342' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} : dispatch 2024-08-30T09:16:52.490 DEBUG:teuthology.orchestra.run.smithi042:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-08-30T09:16:52.535 DEBUG:teuthology.orchestra.run.smithi167:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-08-30T09:16:52.717 DEBUG:teuthology.orchestra.run.smithi190:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-08-30T09:16:52.769 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-08-30T09:16:52.769 DEBUG:teuthology.orchestra.run.smithi042:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-08-30T09:16:52.805 DEBUG:teuthology.orchestra.run.smithi167:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-08-30T09:16:52.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:16:52 smithi167 ceph-mon[22063]: pgmap v859: 33 pgs: 33 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail 2024-08-30T09:16:52.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:16:52 smithi167 ceph-mon[22063]: from='client.? 172.21.15.42:0/1567301342' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-08-30T09:16:52.827 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:16:52 smithi167 ceph-mon[22063]: osdmap e87: 12 total, 12 up, 12 in 2024-08-30T09:16:52.830 DEBUG:teuthology.orchestra.run.smithi190:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-08-30T09:16:52.865 INFO:tasks.cephadm:Stopping all daemons... 2024-08-30T09:16:52.866 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-08-30T09:16:52.866 DEBUG:teuthology.orchestra.run.smithi042:> sudo systemctl stop ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968@mon.a 2024-08-30T09:16:53.134 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:16:52 smithi042 ceph-mon[18652]: pgmap v859: 33 pgs: 33 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail 2024-08-30T09:16:53.134 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:16:52 smithi042 ceph-mon[18652]: from='client.? 172.21.15.42:0/1567301342' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-08-30T09:16:53.134 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:16:52 smithi042 ceph-mon[18652]: osdmap e87: 12 total, 12 up, 12 in 2024-08-30T09:16:53.134 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:16:52 smithi042 systemd[1]: Stopping Ceph mon.a for b0895c1a-66ac-11ef-bcd4-c7b262605968... 2024-08-30T09:16:53.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:16:52 smithi190 ceph-mon[22001]: pgmap v859: 33 pgs: 33 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail 2024-08-30T09:16:53.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:16:52 smithi190 ceph-mon[22001]: from='client.? 172.21.15.42:0/1567301342' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-08-30T09:16:53.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:16:52 smithi190 ceph-mon[22001]: osdmap e87: 12 total, 12 up, 12 in 2024-08-30T09:16:53.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:16:53 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mon-a[18648]: 2024-08-30T09:16:53.364+0000 7fdb1e921640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-08-30T09:16:53.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:16:53 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mon-a[18648]: 2024-08-30T09:16:53.365+0000 7fdb1e921640 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-08-30T09:16:53.633 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:16:53 smithi042 podman[100435]: 2024-08-30 09:16:53.589564664 +0000 UTC m=+0.363544590 container died e677a01ba2f722b1e6e32fc4353612a39c416728efa70ee5abac8539d253b076 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6, name=ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mon-a, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.37.1, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-08-30T09:16:56.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:16:56 smithi042 podman[100435]: 2024-08-30 09:16:56.148584409 +0000 UTC m=+2.922564312 container cleanup e677a01ba2f722b1e6e32fc4353612a39c416728efa70ee5abac8539d253b076 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6, name=ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mon-a, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.1, ceph=True) 2024-08-30T09:16:56.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Aug 30 09:16:56 smithi042 bash[100435]: ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mon-a 2024-08-30T09:16:56.590 DEBUG:teuthology.orchestra.run.smithi042:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968@mon.a.service' 2024-08-30T09:16:56.658 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-30T09:16:56.658 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-08-30T09:16:56.658 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-08-30T09:16:56.659 DEBUG:teuthology.orchestra.run.smithi167:> sudo systemctl stop ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968@mon.b 2024-08-30T09:16:57.029 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:16:56 smithi167 systemd[1]: Stopping Ceph mon.b for b0895c1a-66ac-11ef-bcd4-c7b262605968... 2024-08-30T09:16:57.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:16:57 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mon-b[22059]: 2024-08-30T09:16:57.027+0000 7fda5f271640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-08-30T09:16:57.327 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:16:57 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mon-b[22059]: 2024-08-30T09:16:57.027+0000 7fda5f271640 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-08-30T09:16:57.328 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:16:57 smithi167 podman[57643]: 2024-08-30 09:16:57.098728127 +0000 UTC m=+0.191355622 container died 124e20337385547f2cef1d71179df9116b9312accfbded0e24f9a41b248792ab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mon-b, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.1) 2024-08-30T09:16:57.789 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:16:57 smithi167 podman[57643]: 2024-08-30 09:16:57.494559153 +0000 UTC m=+0.587186647 container cleanup 124e20337385547f2cef1d71179df9116b9312accfbded0e24f9a41b248792ab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mon-b, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.1, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-08-30T09:16:57.789 INFO:journalctl@ceph.mon.b.smithi167.stdout:Aug 30 09:16:57 smithi167 bash[57643]: ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mon-b 2024-08-30T09:16:57.910 DEBUG:teuthology.orchestra.run.smithi167:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968@mon.b.service' 2024-08-30T09:16:57.965 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-30T09:16:57.965 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-08-30T09:16:57.965 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-08-30T09:16:57.965 DEBUG:teuthology.orchestra.run.smithi190:> sudo systemctl stop ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968@mon.c 2024-08-30T09:16:58.436 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:16:58 smithi190 systemd[1]: Stopping Ceph mon.c for b0895c1a-66ac-11ef-bcd4-c7b262605968... 2024-08-30T09:16:58.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:16:58 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mon-c[21983]: 2024-08-30T09:16:58.575+0000 7f597bde2640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-08-30T09:16:58.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:16:58 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mon-c[21983]: 2024-08-30T09:16:58.575+0000 7f597bde2640 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-08-30T09:16:58.936 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:16:58 smithi190 podman[58268]: 2024-08-30 09:16:58.677501008 +0000 UTC m=+0.224166224 container died a523197617ba300e0d1dc684b83cabc78c4c056ba02591ffda49b8e1e7bd9cf9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mon-c, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.37.1, CEPH_POINT_RELEASE=) 2024-08-30T09:16:59.353 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:16:59 smithi190 podman[58268]: 2024-08-30 09:16:59.059111949 +0000 UTC m=+0.605777152 container cleanup a523197617ba300e0d1dc684b83cabc78c4c056ba02591ffda49b8e1e7bd9cf9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mon-c, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, io.buildah.version=1.37.1, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, GIT_BRANCH=HEAD) 2024-08-30T09:16:59.353 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 30 09:16:59 smithi190 bash[58268]: ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mon-c 2024-08-30T09:16:59.473 DEBUG:teuthology.orchestra.run.smithi190:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968@mon.c.service' 2024-08-30T09:16:59.528 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-30T09:16:59.529 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-08-30T09:16:59.529 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-08-30T09:16:59.529 DEBUG:teuthology.orchestra.run.smithi042:> sudo systemctl stop ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968@mgr.x 2024-08-30T09:16:59.839 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Aug 30 09:16:59 smithi042 systemd[1]: Stopping Ceph mgr.x for b0895c1a-66ac-11ef-bcd4-c7b262605968... 2024-08-30T09:17:00.133 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Aug 30 09:16:59 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mgr-x[18896]: 2024-08-30T09:16:59.836+0000 7f1b71b95640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-30T09:17:00.133 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Aug 30 09:16:59 smithi042 podman[100550]: 2024-08-30 09:16:59.868247343 +0000 UTC m=+0.145419557 container died c24e99374a5dc1ba7b14271d729edf828792ec9b703b90e1f5e4cc17b0a95294 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6, name=ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mgr-x, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, ceph=True, io.buildah.version=1.37.1, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-08-30T09:17:00.609 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Aug 30 09:17:00 smithi042 podman[100550]: 2024-08-30 09:17:00.322041149 +0000 UTC m=+0.599213362 container cleanup c24e99374a5dc1ba7b14271d729edf828792ec9b703b90e1f5e4cc17b0a95294 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6, name=ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mgr-x, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.1, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-08-30T09:17:00.609 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Aug 30 09:17:00 smithi042 bash[100550]: ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mgr-x 2024-08-30T09:17:00.758 DEBUG:teuthology.orchestra.run.smithi042:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968@mgr.x.service' 2024-08-30T09:17:00.793 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-30T09:17:00.793 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-08-30T09:17:00.794 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-08-30T09:17:00.794 DEBUG:teuthology.orchestra.run.smithi167:> sudo systemctl stop ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968@mgr.y 2024-08-30T09:17:01.077 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Aug 30 09:17:00 smithi167 systemd[1]: Stopping Ceph mgr.y for b0895c1a-66ac-11ef-bcd4-c7b262605968... 2024-08-30T09:17:01.427 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Aug 30 09:17:01 smithi167 podman[57765]: 2024-08-30 09:17:01.137357437 +0000 UTC m=+0.154626259 container died f497860332d64d70cbaf577853b0328a857a63b912d97ba81cd114c3767d6165 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mgr-y, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.1, GIT_CLEAN=True) 2024-08-30T09:17:01.753 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Aug 30 09:17:01 smithi167 podman[57765]: 2024-08-30 09:17:01.525310721 +0000 UTC m=+0.542579532 container cleanup f497860332d64d70cbaf577853b0328a857a63b912d97ba81cd114c3767d6165 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mgr-y, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, ceph=True, io.buildah.version=1.37.1, CEPH_POINT_RELEASE=) 2024-08-30T09:17:01.753 INFO:journalctl@ceph.mgr.y.smithi167.stdout:Aug 30 09:17:01 smithi167 bash[57765]: ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mgr-y 2024-08-30T09:17:01.882 DEBUG:teuthology.orchestra.run.smithi167:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968@mgr.y.service' 2024-08-30T09:17:01.918 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-30T09:17:01.918 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-08-30T09:17:01.919 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-08-30T09:17:01.919 DEBUG:teuthology.orchestra.run.smithi190:> sudo systemctl stop ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968@mgr.z 2024-08-30T09:17:02.252 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Aug 30 09:17:01 smithi190 systemd[1]: Stopping Ceph mgr.z for b0895c1a-66ac-11ef-bcd4-c7b262605968... 2024-08-30T09:17:02.549 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Aug 30 09:17:02 smithi190 podman[58387]: 2024-08-30 09:17:02.251291989 +0000 UTC m=+0.141586957 container died 4d9568cd493dbda9f9982cb47ed695eb535f5617f393d6f0f99b31bb7a1a3458 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mgr-z, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.37.1, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux ) 2024-08-30T09:17:02.926 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Aug 30 09:17:02 smithi190 podman[58387]: 2024-08-30 09:17:02.647889345 +0000 UTC m=+0.538184312 container cleanup 4d9568cd493dbda9f9982cb47ed695eb535f5617f393d6f0f99b31bb7a1a3458 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mgr-z, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, io.buildah.version=1.37.1, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-08-30T09:17:02.926 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Aug 30 09:17:02 smithi190 bash[58387]: ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mgr-z 2024-08-30T09:17:03.060 DEBUG:teuthology.orchestra.run.smithi190:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968@mgr.z.service' 2024-08-30T09:17:03.094 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-30T09:17:03.095 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-08-30T09:17:03.095 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-08-30T09:17:03.095 DEBUG:teuthology.orchestra.run.smithi042:> sudo systemctl stop ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968@osd.0 2024-08-30T09:17:03.383 INFO:journalctl@ceph.osd.0.smithi042.stdout:Aug 30 09:17:03 smithi042 systemd[1]: Stopping Ceph osd.0 for b0895c1a-66ac-11ef-bcd4-c7b262605968... 2024-08-30T09:17:03.883 INFO:journalctl@ceph.osd.0.smithi042.stdout:Aug 30 09:17:03 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-0[35040]: 2024-08-30T09:17:03.426+0000 7f46f8f1f640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-30T09:17:03.883 INFO:journalctl@ceph.osd.0.smithi042.stdout:Aug 30 09:17:03 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-0[35040]: 2024-08-30T09:17:03.427+0000 7f46f8f1f640 -1 osd.0 87 *** Got signal Terminated *** 2024-08-30T09:17:03.883 INFO:journalctl@ceph.osd.0.smithi042.stdout:Aug 30 09:17:03 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-0[35040]: 2024-08-30T09:17:03.427+0000 7f46f8f1f640 -1 osd.0 87 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-08-30T09:17:09.383 INFO:journalctl@ceph.osd.0.smithi042.stdout:Aug 30 09:17:08 smithi042 podman[100670]: 2024-08-30 09:17:08.92931715 +0000 UTC m=+5.634450605 container died f5c7b845b9a794f842a728232dcd5f775349830fa1aec3bb782f8bf1ddec024a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.37.1, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-08-30T09:17:09.772 INFO:journalctl@ceph.osd.0.smithi042.stdout:Aug 30 09:17:09 smithi042 podman[100670]: 2024-08-30 09:17:09.569577038 +0000 UTC m=+6.274710495 container cleanup f5c7b845b9a794f842a728232dcd5f775349830fa1aec3bb782f8bf1ddec024a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-0, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, io.buildah.version=1.37.1, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image) 2024-08-30T09:17:09.772 INFO:journalctl@ceph.osd.0.smithi042.stdout:Aug 30 09:17:09 smithi042 bash[100670]: ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-0 2024-08-30T09:17:10.133 INFO:journalctl@ceph.osd.0.smithi042.stdout:Aug 30 09:17:09 smithi042 podman[100685]: 2024-08-30 09:17:09.771123903 +0000 UTC m=+0.757105707 container remove f5c7b845b9a794f842a728232dcd5f775349830fa1aec3bb782f8bf1ddec024a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-0, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.37.1, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-08-30T09:17:10.443 INFO:journalctl@ceph.osd.0.smithi042.stdout:Aug 30 09:17:10 smithi042 podman[100774]: 2024-08-30 09:17:10.08211621 +0000 UTC m=+0.016162319 image pull 79f4cde5e36911a5dca995eeb98098c2c8aae81dd52a088ef2c867bc3363bad0 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61 2024-08-30T09:17:10.443 INFO:journalctl@ceph.osd.0.smithi042.stdout:Aug 30 09:17:10 smithi042 podman[100774]: 2024-08-30 09:17:10.188897165 +0000 UTC m=+0.122943271 container create ffde415ed53957351dbddb85cff177a80b341abb97cc6f3820b6d3bda4c1ca51 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-0-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.1, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-08-30T09:17:10.706 INFO:journalctl@ceph.osd.0.smithi042.stdout:Aug 30 09:17:10 smithi042 podman[100774]: 2024-08-30 09:17:10.441502787 +0000 UTC m=+0.375548896 container init ffde415ed53957351dbddb85cff177a80b341abb97cc6f3820b6d3bda4c1ca51 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-0-deactivate, org.label-schema.license=GPLv2, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, CEPH_POINT_RELEASE=, io.buildah.version=1.37.1, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image) 2024-08-30T09:17:10.706 INFO:journalctl@ceph.osd.0.smithi042.stdout:Aug 30 09:17:10 smithi042 podman[100774]: 2024-08-30 09:17:10.445795242 +0000 UTC m=+0.379841351 container start ffde415ed53957351dbddb85cff177a80b341abb97cc6f3820b6d3bda4c1ca51 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-0-deactivate, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.37.1, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, ceph=True) 2024-08-30T09:17:10.706 INFO:journalctl@ceph.osd.0.smithi042.stdout:Aug 30 09:17:10 smithi042 podman[100774]: 2024-08-30 09:17:10.558689141 +0000 UTC m=+0.492735267 container attach ffde415ed53957351dbddb85cff177a80b341abb97cc6f3820b6d3bda4c1ca51 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-0-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, io.buildah.version=1.37.1, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611) 2024-08-30T09:17:11.063 INFO:journalctl@ceph.osd.0.smithi042.stdout:Aug 30 09:17:10 smithi042 podman[100774]: 2024-08-30 09:17:10.705110885 +0000 UTC m=+0.639157000 container died ffde415ed53957351dbddb85cff177a80b341abb97cc6f3820b6d3bda4c1ca51 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-0-deactivate, GIT_CLEAN=True, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.1, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, ceph=True) 2024-08-30T09:17:11.369 DEBUG:teuthology.orchestra.run.smithi042:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968@osd.0.service' 2024-08-30T09:17:11.410 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-30T09:17:11.410 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-08-30T09:17:11.410 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-08-30T09:17:11.410 DEBUG:teuthology.orchestra.run.smithi167:> sudo systemctl stop ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968@osd.1 2024-08-30T09:17:11.748 INFO:journalctl@ceph.osd.1.smithi167.stdout:Aug 30 09:17:11 smithi167 systemd[1]: Stopping Ceph osd.1 for b0895c1a-66ac-11ef-bcd4-c7b262605968... 2024-08-30T09:17:12.077 INFO:journalctl@ceph.osd.1.smithi167.stdout:Aug 30 09:17:11 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-1[26506]: 2024-08-30T09:17:11.746+0000 7f9a6f711640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-30T09:17:12.077 INFO:journalctl@ceph.osd.1.smithi167.stdout:Aug 30 09:17:11 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-1[26506]: 2024-08-30T09:17:11.746+0000 7f9a6f711640 -1 osd.1 87 *** Got signal Terminated *** 2024-08-30T09:17:12.077 INFO:journalctl@ceph.osd.1.smithi167.stdout:Aug 30 09:17:11 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-1[26506]: 2024-08-30T09:17:11.746+0000 7f9a6f711640 -1 osd.1 87 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-08-30T09:17:17.708 INFO:journalctl@ceph.osd.1.smithi167.stdout:Aug 30 09:17:17 smithi167 podman[57891]: 2024-08-30 09:17:17.392487659 +0000 UTC m=+5.782615551 container died 9f014f5eae102c62075f3b971c59ed8f916d478ec15508b4bc580800f6045d13 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-1, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.1, org.label-schema.license=GPLv2) 2024-08-30T09:17:18.067 INFO:journalctl@ceph.osd.1.smithi167.stdout:Aug 30 09:17:17 smithi167 podman[57891]: 2024-08-30 09:17:17.830723382 +0000 UTC m=+6.220851273 container cleanup 9f014f5eae102c62075f3b971c59ed8f916d478ec15508b4bc580800f6045d13 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-1, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, io.buildah.version=1.37.1, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-08-30T09:17:18.067 INFO:journalctl@ceph.osd.1.smithi167.stdout:Aug 30 09:17:17 smithi167 bash[57891]: ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-1 2024-08-30T09:17:18.327 INFO:journalctl@ceph.osd.1.smithi167.stdout:Aug 30 09:17:18 smithi167 podman[57906]: 2024-08-30 09:17:18.065676576 +0000 UTC m=+0.674224617 container remove 9f014f5eae102c62075f3b971c59ed8f916d478ec15508b4bc580800f6045d13 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-1, io.buildah.version=1.37.1, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-08-30T09:17:18.763 INFO:journalctl@ceph.osd.1.smithi167.stdout:Aug 30 09:17:18 smithi167 podman[58004]: 2024-08-30 09:17:18.508542571 +0000 UTC m=+0.109349713 container create 70b123b9cb2b5667f2e1f888ad9e654bb41e3122f97ee36a6d8ae94d1b2f501f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-1-deactivate, ceph=True, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, io.buildah.version=1.37.1, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-08-30T09:17:18.763 INFO:journalctl@ceph.osd.1.smithi167.stdout:Aug 30 09:17:18 smithi167 podman[58004]: 2024-08-30 09:17:18.415404459 +0000 UTC m=+0.016211622 image pull 79f4cde5e36911a5dca995eeb98098c2c8aae81dd52a088ef2c867bc3363bad0 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61 2024-08-30T09:17:19.077 INFO:journalctl@ceph.osd.1.smithi167.stdout:Aug 30 09:17:18 smithi167 podman[58004]: 2024-08-30 09:17:18.761488544 +0000 UTC m=+0.362295695 container init 70b123b9cb2b5667f2e1f888ad9e654bb41e3122f97ee36a6d8ae94d1b2f501f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-1-deactivate, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, io.buildah.version=1.37.1, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-08-30T09:17:19.077 INFO:journalctl@ceph.osd.1.smithi167.stdout:Aug 30 09:17:18 smithi167 podman[58004]: 2024-08-30 09:17:18.765060419 +0000 UTC m=+0.365867562 container start 70b123b9cb2b5667f2e1f888ad9e654bb41e3122f97ee36a6d8ae94d1b2f501f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-1-deactivate, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.37.1, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-08-30T09:17:19.077 INFO:journalctl@ceph.osd.1.smithi167.stdout:Aug 30 09:17:18 smithi167 podman[58004]: 2024-08-30 09:17:18.878632076 +0000 UTC m=+0.479439222 container attach 70b123b9cb2b5667f2e1f888ad9e654bb41e3122f97ee36a6d8ae94d1b2f501f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-1-deactivate, org.label-schema.build-date=20240731, ceph=True, io.buildah.version=1.37.1, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-08-30T09:17:19.077 INFO:journalctl@ceph.osd.1.smithi167.stdout:Aug 30 09:17:19 smithi167 podman[58004]: 2024-08-30 09:17:19.075985834 +0000 UTC m=+0.676792980 container died 70b123b9cb2b5667f2e1f888ad9e654bb41e3122f97ee36a6d8ae94d1b2f501f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-1-deactivate, ceph=True, io.buildah.version=1.37.1, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-08-30T09:17:19.739 DEBUG:teuthology.orchestra.run.smithi167:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968@osd.1.service' 2024-08-30T09:17:19.780 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-30T09:17:19.780 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-08-30T09:17:19.780 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-08-30T09:17:19.780 DEBUG:teuthology.orchestra.run.smithi190:> sudo systemctl stop ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968@osd.2 2024-08-30T09:17:20.112 INFO:journalctl@ceph.osd.2.smithi190.stdout:Aug 30 09:17:19 smithi190 systemd[1]: Stopping Ceph osd.2 for b0895c1a-66ac-11ef-bcd4-c7b262605968... 2024-08-30T09:17:20.380 INFO:journalctl@ceph.osd.2.smithi190.stdout:Aug 30 09:17:20 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-2[26986]: 2024-08-30T09:17:20.109+0000 7f4be5bac640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-30T09:17:20.380 INFO:journalctl@ceph.osd.2.smithi190.stdout:Aug 30 09:17:20 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-2[26986]: 2024-08-30T09:17:20.109+0000 7f4be5bac640 -1 osd.2 87 *** Got signal Terminated *** 2024-08-30T09:17:20.380 INFO:journalctl@ceph.osd.2.smithi190.stdout:Aug 30 09:17:20 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-2[26986]: 2024-08-30T09:17:20.109+0000 7f4be5bac640 -1 osd.2 87 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-08-30T09:17:25.936 INFO:journalctl@ceph.osd.2.smithi190.stdout:Aug 30 09:17:25 smithi190 podman[58511]: 2024-08-30 09:17:25.63669638 +0000 UTC m=+5.649921648 container died ad614b27c306cc3ed8fd34982960d86480dbb92a88c7bdf584d52819ec8cc27f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-2, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.37.1, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731) 2024-08-30T09:17:26.297 INFO:journalctl@ceph.osd.2.smithi190.stdout:Aug 30 09:17:26 smithi190 podman[58511]: 2024-08-30 09:17:26.044106761 +0000 UTC m=+6.057332026 container cleanup ad614b27c306cc3ed8fd34982960d86480dbb92a88c7bdf584d52819ec8cc27f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-2, io.buildah.version=1.37.1, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, ceph=True) 2024-08-30T09:17:26.297 INFO:journalctl@ceph.osd.2.smithi190.stdout:Aug 30 09:17:26 smithi190 bash[58511]: ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-2 2024-08-30T09:17:26.603 INFO:journalctl@ceph.osd.2.smithi190.stdout:Aug 30 09:17:26 smithi190 podman[58538]: 2024-08-30 09:17:26.295685016 +0000 UTC m=+0.659296070 container remove ad614b27c306cc3ed8fd34982960d86480dbb92a88c7bdf584d52819ec8cc27f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.1, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image) 2024-08-30T09:17:26.857 INFO:journalctl@ceph.osd.2.smithi190.stdout:Aug 30 09:17:26 smithi190 podman[58630]: 2024-08-30 09:17:26.505245433 +0000 UTC m=+0.015825918 image pull 79f4cde5e36911a5dca995eeb98098c2c8aae81dd52a088ef2c867bc3363bad0 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61 2024-08-30T09:17:26.858 INFO:journalctl@ceph.osd.2.smithi190.stdout:Aug 30 09:17:26 smithi190 podman[58630]: 2024-08-30 09:17:26.638256195 +0000 UTC m=+0.148836669 container create d1639930a71fdbec27274dddd48e374808a50690eacf485a485133e041e931a2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-2-deactivate, GIT_BRANCH=HEAD, io.buildah.version=1.37.1, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-08-30T09:17:27.134 INFO:journalctl@ceph.osd.2.smithi190.stdout:Aug 30 09:17:27 smithi190 podman[58630]: 2024-08-30 09:17:27.024334475 +0000 UTC m=+0.534914948 container init d1639930a71fdbec27274dddd48e374808a50690eacf485a485133e041e931a2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-2-deactivate, org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.1, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731) 2024-08-30T09:17:27.134 INFO:journalctl@ceph.osd.2.smithi190.stdout:Aug 30 09:17:27 smithi190 podman[58630]: 2024-08-30 09:17:27.027882023 +0000 UTC m=+0.538462496 container start d1639930a71fdbec27274dddd48e374808a50690eacf485a485133e041e931a2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-2-deactivate, org.label-schema.build-date=20240731, ceph=True, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.37.1, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-08-30T09:17:27.134 INFO:journalctl@ceph.osd.2.smithi190.stdout:Aug 30 09:17:27 smithi190 podman[58630]: 2024-08-30 09:17:27.132843901 +0000 UTC m=+0.643424419 container attach d1639930a71fdbec27274dddd48e374808a50690eacf485a485133e041e931a2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-2-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.1, ceph=True, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731) 2024-08-30T09:17:27.436 INFO:journalctl@ceph.osd.2.smithi190.stdout:Aug 30 09:17:27 smithi190 podman[58630]: 2024-08-30 09:17:27.332487755 +0000 UTC m=+0.843068229 container died d1639930a71fdbec27274dddd48e374808a50690eacf485a485133e041e931a2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-2-deactivate, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.37.1, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux ) 2024-08-30T09:17:27.938 DEBUG:teuthology.orchestra.run.smithi190:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968@osd.2.service' 2024-08-30T09:17:27.979 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-30T09:17:27.980 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-08-30T09:17:27.980 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-08-30T09:17:27.980 DEBUG:teuthology.orchestra.run.smithi042:> sudo systemctl stop ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968@osd.3 2024-08-30T09:17:28.294 INFO:journalctl@ceph.osd.3.smithi042.stdout:Aug 30 09:17:28 smithi042 systemd[1]: Stopping Ceph osd.3 for b0895c1a-66ac-11ef-bcd4-c7b262605968... 2024-08-30T09:17:28.633 INFO:journalctl@ceph.osd.3.smithi042.stdout:Aug 30 09:17:28 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-3[41913]: 2024-08-30T09:17:28.291+0000 7f80163c8640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-30T09:17:28.633 INFO:journalctl@ceph.osd.3.smithi042.stdout:Aug 30 09:17:28 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-3[41913]: 2024-08-30T09:17:28.291+0000 7f80163c8640 -1 osd.3 87 *** Got signal Terminated *** 2024-08-30T09:17:28.633 INFO:journalctl@ceph.osd.3.smithi042.stdout:Aug 30 09:17:28 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-3[41913]: 2024-08-30T09:17:28.291+0000 7f80163c8640 -1 osd.3 87 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-08-30T09:17:30.077 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:17:29 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:17:29.615+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:03.239730+0000 front 2024-08-30T09:17:03.239661+0000 (oldest deadline 2024-08-30T09:17:29.139608+0000) 2024-08-30T09:17:30.186 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:17:29 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:17:29.875+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:04.091032+0000 front 2024-08-30T09:17:04.090868+0000 (oldest deadline 2024-08-30T09:17:29.391066+0000) 2024-08-30T09:17:30.883 INFO:journalctl@ceph.osd.6.smithi042.stdout:Aug 30 09:17:30 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-6[48371]: 2024-08-30T09:17:30.505+0000 7fcbc47c8640 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:07.508340+0000 front 2024-08-30T09:17:07.508372+0000 (oldest deadline 2024-08-30T09:17:29.808488+0000) 2024-08-30T09:17:31.077 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:17:30 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:17:30.579+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:03.239730+0000 front 2024-08-30T09:17:03.239661+0000 (oldest deadline 2024-08-30T09:17:29.139608+0000) 2024-08-30T09:17:31.186 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:17:30 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:17:30.871+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:04.091032+0000 front 2024-08-30T09:17:04.090868+0000 (oldest deadline 2024-08-30T09:17:29.391066+0000) 2024-08-30T09:17:31.883 INFO:journalctl@ceph.osd.3.smithi042.stdout:Aug 30 09:17:31 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-3[41913]: 2024-08-30T09:17:31.436+0000 7f800e9ab640 -1 osd.3 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:06.003532+0000 front 2024-08-30T09:17:06.003572+0000 (oldest deadline 2024-08-30T09:17:31.303773+0000) 2024-08-30T09:17:31.883 INFO:journalctl@ceph.osd.6.smithi042.stdout:Aug 30 09:17:31 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-6[48371]: 2024-08-30T09:17:31.537+0000 7fcbc47c8640 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:07.508340+0000 front 2024-08-30T09:17:07.508372+0000 (oldest deadline 2024-08-30T09:17:29.808488+0000) 2024-08-30T09:17:31.884 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:17:31 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9[55148]: 2024-08-30T09:17:31.411+0000 7f7587f3d640 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:07.800304+0000 front 2024-08-30T09:17:07.800409+0000 (oldest deadline 2024-08-30T09:17:30.700443+0000) 2024-08-30T09:17:32.077 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:17:31 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:17:31.629+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:03.239730+0000 front 2024-08-30T09:17:03.239661+0000 (oldest deadline 2024-08-30T09:17:29.139608+0000) 2024-08-30T09:17:32.186 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:17:31 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:17:31.903+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:04.091032+0000 front 2024-08-30T09:17:04.090868+0000 (oldest deadline 2024-08-30T09:17:29.391066+0000) 2024-08-30T09:17:32.186 INFO:journalctl@ceph.osd.5.smithi190.stdout:Aug 30 09:17:31 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-5[33866]: 2024-08-30T09:17:31.798+0000 7ffa64ccc640 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:06.221012+0000 front 2024-08-30T09:17:06.220784+0000 (oldest deadline 2024-08-30T09:17:31.521037+0000) 2024-08-30T09:17:32.883 INFO:journalctl@ceph.osd.3.smithi042.stdout:Aug 30 09:17:32 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-3[41913]: 2024-08-30T09:17:32.449+0000 7f800e9ab640 -1 osd.3 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:06.003532+0000 front 2024-08-30T09:17:06.003572+0000 (oldest deadline 2024-08-30T09:17:31.303773+0000) 2024-08-30T09:17:32.883 INFO:journalctl@ceph.osd.6.smithi042.stdout:Aug 30 09:17:32 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-6[48371]: 2024-08-30T09:17:32.502+0000 7fcbc47c8640 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:07.508340+0000 front 2024-08-30T09:17:07.508372+0000 (oldest deadline 2024-08-30T09:17:29.808488+0000) 2024-08-30T09:17:32.883 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:17:32 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9[55148]: 2024-08-30T09:17:32.453+0000 7f7587f3d640 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:07.800304+0000 front 2024-08-30T09:17:07.800409+0000 (oldest deadline 2024-08-30T09:17:30.700443+0000) 2024-08-30T09:17:33.077 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:17:32 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:17:32.643+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:03.239730+0000 front 2024-08-30T09:17:03.239661+0000 (oldest deadline 2024-08-30T09:17:29.139608+0000) 2024-08-30T09:17:33.186 INFO:journalctl@ceph.osd.5.smithi190.stdout:Aug 30 09:17:32 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-5[33866]: 2024-08-30T09:17:32.785+0000 7ffa64ccc640 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:06.221012+0000 front 2024-08-30T09:17:06.220784+0000 (oldest deadline 2024-08-30T09:17:31.521037+0000) 2024-08-30T09:17:33.186 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:17:32 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:17:32.914+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:04.091032+0000 front 2024-08-30T09:17:04.090868+0000 (oldest deadline 2024-08-30T09:17:29.391066+0000) 2024-08-30T09:17:33.856 INFO:journalctl@ceph.osd.6.smithi042.stdout:Aug 30 09:17:33 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-6[48371]: 2024-08-30T09:17:33.523+0000 7fcbc47c8640 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:07.508340+0000 front 2024-08-30T09:17:07.508372+0000 (oldest deadline 2024-08-30T09:17:29.808488+0000) 2024-08-30T09:17:33.857 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:17:33 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9[55148]: 2024-08-30T09:17:33.450+0000 7f7587f3d640 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:07.800304+0000 front 2024-08-30T09:17:07.800409+0000 (oldest deadline 2024-08-30T09:17:30.700443+0000) 2024-08-30T09:17:34.077 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:17:33 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:17:33.661+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:03.239730+0000 front 2024-08-30T09:17:03.239661+0000 (oldest deadline 2024-08-30T09:17:29.139608+0000) 2024-08-30T09:17:34.132 INFO:journalctl@ceph.osd.3.smithi042.stdout:Aug 30 09:17:33 smithi042 podman[100903]: 2024-08-30 09:17:33.854991995 +0000 UTC m=+5.677481885 container died 91558ede3e8855383c16c91cf1165e50f7716b425984a7a39a58df27face39de (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-3, io.buildah.version=1.37.1, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0) 2024-08-30T09:17:34.186 INFO:journalctl@ceph.osd.5.smithi190.stdout:Aug 30 09:17:33 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-5[33866]: 2024-08-30T09:17:33.828+0000 7ffa64ccc640 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:06.221012+0000 front 2024-08-30T09:17:06.220784+0000 (oldest deadline 2024-08-30T09:17:31.521037+0000) 2024-08-30T09:17:34.186 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:17:33 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:17:33.919+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:04.091032+0000 front 2024-08-30T09:17:04.090868+0000 (oldest deadline 2024-08-30T09:17:29.391066+0000) 2024-08-30T09:17:34.467 INFO:journalctl@ceph.osd.3.smithi042.stdout:Aug 30 09:17:34 smithi042 podman[100903]: 2024-08-30 09:17:34.342637713 +0000 UTC m=+6.165127633 container cleanup 91558ede3e8855383c16c91cf1165e50f7716b425984a7a39a58df27face39de (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.37.1, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-08-30T09:17:34.467 INFO:journalctl@ceph.osd.3.smithi042.stdout:Aug 30 09:17:34 smithi042 bash[100903]: ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-3 2024-08-30T09:17:34.883 INFO:journalctl@ceph.osd.6.smithi042.stdout:Aug 30 09:17:34 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-6[48371]: 2024-08-30T09:17:34.527+0000 7fcbc47c8640 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:07.508340+0000 front 2024-08-30T09:17:07.508372+0000 (oldest deadline 2024-08-30T09:17:29.808488+0000) 2024-08-30T09:17:34.883 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:17:34 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9[55148]: 2024-08-30T09:17:34.465+0000 7f7587f3d640 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:07.800304+0000 front 2024-08-30T09:17:07.800409+0000 (oldest deadline 2024-08-30T09:17:30.700443+0000) 2024-08-30T09:17:34.884 INFO:journalctl@ceph.osd.3.smithi042.stdout:Aug 30 09:17:34 smithi042 podman[100918]: 2024-08-30 09:17:34.627920055 +0000 UTC m=+0.768812788 container remove 91558ede3e8855383c16c91cf1165e50f7716b425984a7a39a58df27face39de (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-3, org.label-schema.schema-version=1.0, io.buildah.version=1.37.1, org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731) 2024-08-30T09:17:35.077 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:17:34 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:17:34.619+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:03.239730+0000 front 2024-08-30T09:17:03.239661+0000 (oldest deadline 2024-08-30T09:17:29.139608+0000) 2024-08-30T09:17:35.186 INFO:journalctl@ceph.osd.5.smithi190.stdout:Aug 30 09:17:34 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-5[33866]: 2024-08-30T09:17:34.870+0000 7ffa64ccc640 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:06.221012+0000 front 2024-08-30T09:17:06.220784+0000 (oldest deadline 2024-08-30T09:17:31.521037+0000) 2024-08-30T09:17:35.186 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:17:34 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:17:34.949+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:04.091032+0000 front 2024-08-30T09:17:04.090868+0000 (oldest deadline 2024-08-30T09:17:29.391066+0000) 2024-08-30T09:17:35.200 INFO:journalctl@ceph.osd.3.smithi042.stdout:Aug 30 09:17:34 smithi042 podman[101009]: 2024-08-30 09:17:34.843666136 +0000 UTC m=+0.016074004 image pull 79f4cde5e36911a5dca995eeb98098c2c8aae81dd52a088ef2c867bc3363bad0 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61 2024-08-30T09:17:35.200 INFO:journalctl@ceph.osd.3.smithi042.stdout:Aug 30 09:17:34 smithi042 podman[101009]: 2024-08-30 09:17:34.96217763 +0000 UTC m=+0.134585539 container create a8e8f670ca8a69dc77d794945f40a254ab8c35d7e32c23a751aadfa9ffca3489 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-3-deactivate, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, io.buildah.version=1.37.1) 2024-08-30T09:17:35.494 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:17:35 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9[55148]: 2024-08-30T09:17:35.415+0000 7f7587f3d640 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:07.800304+0000 front 2024-08-30T09:17:07.800409+0000 (oldest deadline 2024-08-30T09:17:30.700443+0000) 2024-08-30T09:17:35.494 INFO:journalctl@ceph.osd.6.smithi042.stdout:Aug 30 09:17:35 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-6[48371]: 2024-08-30T09:17:35.492+0000 7fcbc47c8640 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:07.508340+0000 front 2024-08-30T09:17:07.508372+0000 (oldest deadline 2024-08-30T09:17:29.808488+0000) 2024-08-30T09:17:35.494 INFO:journalctl@ceph.osd.3.smithi042.stdout:Aug 30 09:17:35 smithi042 podman[101009]: 2024-08-30 09:17:35.198076242 +0000 UTC m=+0.370484110 container init a8e8f670ca8a69dc77d794945f40a254ab8c35d7e32c23a751aadfa9ffca3489 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-3-deactivate, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.37.1, GIT_CLEAN=True) 2024-08-30T09:17:35.495 INFO:journalctl@ceph.osd.3.smithi042.stdout:Aug 30 09:17:35 smithi042 podman[101009]: 2024-08-30 09:17:35.201591783 +0000 UTC m=+0.373999651 container start a8e8f670ca8a69dc77d794945f40a254ab8c35d7e32c23a751aadfa9ffca3489 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-3-deactivate, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, io.buildah.version=1.37.1, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, maintainer=Guillaume Abrioux ) 2024-08-30T09:17:35.495 INFO:journalctl@ceph.osd.3.smithi042.stdout:Aug 30 09:17:35 smithi042 podman[101009]: 2024-08-30 09:17:35.281629272 +0000 UTC m=+0.454037139 container attach a8e8f670ca8a69dc77d794945f40a254ab8c35d7e32c23a751aadfa9ffca3489 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-3-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.37.1) 2024-08-30T09:17:35.495 INFO:journalctl@ceph.osd.3.smithi042.stdout:Aug 30 09:17:35 smithi042 podman[101009]: 2024-08-30 09:17:35.391146684 +0000 UTC m=+0.563554551 container died a8e8f670ca8a69dc77d794945f40a254ab8c35d7e32c23a751aadfa9ffca3489 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-3-deactivate, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.1) 2024-08-30T09:17:35.827 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:17:35 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:17:35.570+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:03.239730+0000 front 2024-08-30T09:17:03.239661+0000 (oldest deadline 2024-08-30T09:17:29.139608+0000) 2024-08-30T09:17:36.186 INFO:journalctl@ceph.osd.5.smithi190.stdout:Aug 30 09:17:35 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-5[33866]: 2024-08-30T09:17:35.821+0000 7ffa64ccc640 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:06.221012+0000 front 2024-08-30T09:17:06.220784+0000 (oldest deadline 2024-08-30T09:17:31.521037+0000) 2024-08-30T09:17:36.186 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:17:35 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:17:35.909+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:04.091032+0000 front 2024-08-30T09:17:04.090868+0000 (oldest deadline 2024-08-30T09:17:29.391066+0000) 2024-08-30T09:17:36.216 DEBUG:teuthology.orchestra.run.smithi042:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968@osd.3.service' 2024-08-30T09:17:36.263 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-30T09:17:36.263 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-08-30T09:17:36.263 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-08-30T09:17:36.263 DEBUG:teuthology.orchestra.run.smithi167:> sudo systemctl stop ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968@osd.4 2024-08-30T09:17:36.577 INFO:journalctl@ceph.osd.4.smithi167.stdout:Aug 30 09:17:36 smithi167 systemd[1]: Stopping Ceph osd.4 for b0895c1a-66ac-11ef-bcd4-c7b262605968... 2024-08-30T09:17:36.883 INFO:journalctl@ceph.osd.6.smithi042.stdout:Aug 30 09:17:36 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-6[48371]: 2024-08-30T09:17:36.446+0000 7fcbc47c8640 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:07.508340+0000 front 2024-08-30T09:17:07.508372+0000 (oldest deadline 2024-08-30T09:17:29.808488+0000) 2024-08-30T09:17:36.883 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:17:36 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9[55148]: 2024-08-30T09:17:36.452+0000 7f7587f3d640 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:07.800304+0000 front 2024-08-30T09:17:07.800409+0000 (oldest deadline 2024-08-30T09:17:30.700443+0000) 2024-08-30T09:17:37.077 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:17:36 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:17:36.609+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:03.239730+0000 front 2024-08-30T09:17:03.239661+0000 (oldest deadline 2024-08-30T09:17:29.139608+0000) 2024-08-30T09:17:37.077 INFO:journalctl@ceph.osd.4.smithi167.stdout:Aug 30 09:17:36 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-4[33404]: 2024-08-30T09:17:36.576+0000 7fd0a0ccd640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-30T09:17:37.077 INFO:journalctl@ceph.osd.4.smithi167.stdout:Aug 30 09:17:36 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-4[33404]: 2024-08-30T09:17:36.576+0000 7fd0a0ccd640 -1 osd.4 87 *** Got signal Terminated *** 2024-08-30T09:17:37.078 INFO:journalctl@ceph.osd.4.smithi167.stdout:Aug 30 09:17:36 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-4[33404]: 2024-08-30T09:17:36.576+0000 7fd0a0ccd640 -1 osd.4 87 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-08-30T09:17:37.186 INFO:journalctl@ceph.osd.5.smithi190.stdout:Aug 30 09:17:36 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-5[33866]: 2024-08-30T09:17:36.824+0000 7ffa64ccc640 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:06.221012+0000 front 2024-08-30T09:17:06.220784+0000 (oldest deadline 2024-08-30T09:17:31.521037+0000) 2024-08-30T09:17:37.186 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:17:36 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:17:36.904+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:04.091032+0000 front 2024-08-30T09:17:04.090868+0000 (oldest deadline 2024-08-30T09:17:29.391066+0000) 2024-08-30T09:17:37.883 INFO:journalctl@ceph.osd.6.smithi042.stdout:Aug 30 09:17:37 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-6[48371]: 2024-08-30T09:17:37.442+0000 7fcbc47c8640 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:07.508340+0000 front 2024-08-30T09:17:07.508372+0000 (oldest deadline 2024-08-30T09:17:29.808488+0000) 2024-08-30T09:17:37.883 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:17:37 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9[55148]: 2024-08-30T09:17:37.488+0000 7f7587f3d640 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:07.800304+0000 front 2024-08-30T09:17:07.800409+0000 (oldest deadline 2024-08-30T09:17:30.700443+0000) 2024-08-30T09:17:38.077 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:17:37 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:17:37.652+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:03.239730+0000 front 2024-08-30T09:17:03.239661+0000 (oldest deadline 2024-08-30T09:17:29.139608+0000) 2024-08-30T09:17:38.186 INFO:journalctl@ceph.osd.5.smithi190.stdout:Aug 30 09:17:37 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-5[33866]: 2024-08-30T09:17:37.783+0000 7ffa64ccc640 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:06.221012+0000 front 2024-08-30T09:17:06.220784+0000 (oldest deadline 2024-08-30T09:17:31.521037+0000) 2024-08-30T09:17:38.186 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:17:37 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:17:37.897+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:04.091032+0000 front 2024-08-30T09:17:04.090868+0000 (oldest deadline 2024-08-30T09:17:29.391066+0000) 2024-08-30T09:17:38.186 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:17:37 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:17:37.897+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:12.193073+0000 front 2024-08-30T09:17:12.192820+0000 (oldest deadline 2024-08-30T09:17:37.492761+0000) 2024-08-30T09:17:38.634 INFO:journalctl@ceph.osd.7.smithi167.stdout:Aug 30 09:17:38 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-7[40306]: 2024-08-30T09:17:38.353+0000 7f33e554b640 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:15.442207+0000 front 2024-08-30T09:17:15.442257+0000 (oldest deadline 2024-08-30T09:17:37.642922+0000) 2024-08-30T09:17:38.883 INFO:journalctl@ceph.osd.6.smithi042.stdout:Aug 30 09:17:38 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-6[48371]: 2024-08-30T09:17:38.453+0000 7fcbc47c8640 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:07.508340+0000 front 2024-08-30T09:17:07.508372+0000 (oldest deadline 2024-08-30T09:17:29.808488+0000) 2024-08-30T09:17:38.883 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:17:38 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9[55148]: 2024-08-30T09:17:38.448+0000 7f7587f3d640 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:07.800304+0000 front 2024-08-30T09:17:07.800409+0000 (oldest deadline 2024-08-30T09:17:30.700443+0000) 2024-08-30T09:17:38.936 INFO:journalctl@ceph.osd.5.smithi190.stdout:Aug 30 09:17:38 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-5[33866]: 2024-08-30T09:17:38.798+0000 7ffa64ccc640 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:06.221012+0000 front 2024-08-30T09:17:06.220784+0000 (oldest deadline 2024-08-30T09:17:31.521037+0000) 2024-08-30T09:17:38.936 INFO:journalctl@ceph.osd.5.smithi190.stdout:Aug 30 09:17:38 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-5[33866]: 2024-08-30T09:17:38.798+0000 7ffa64ccc640 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:14.923100+0000 front 2024-08-30T09:17:14.922995+0000 (oldest deadline 2024-08-30T09:17:37.822873+0000) 2024-08-30T09:17:38.937 INFO:journalctl@ceph.osd.8.smithi190.stdout:Aug 30 09:17:38 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-8[40364]: 2024-08-30T09:17:38.665+0000 7fa492b20640 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:14.793297+0000 front 2024-08-30T09:17:14.793353+0000 (oldest deadline 2024-08-30T09:17:38.293306+0000) 2024-08-30T09:17:38.937 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:17:38 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:17:38.850+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:04.091032+0000 front 2024-08-30T09:17:04.090868+0000 (oldest deadline 2024-08-30T09:17:29.391066+0000) 2024-08-30T09:17:38.937 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:17:38 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:17:38.850+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:12.193073+0000 front 2024-08-30T09:17:12.192820+0000 (oldest deadline 2024-08-30T09:17:37.492761+0000) 2024-08-30T09:17:39.077 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:17:38 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:17:38.632+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:03.239730+0000 front 2024-08-30T09:17:03.239661+0000 (oldest deadline 2024-08-30T09:17:29.139608+0000) 2024-08-30T09:17:39.077 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:17:38 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:17:38.632+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:12.040451+0000 front 2024-08-30T09:17:12.040473+0000 (oldest deadline 2024-08-30T09:17:37.841115+0000) 2024-08-30T09:17:39.683 INFO:journalctl@ceph.osd.7.smithi167.stdout:Aug 30 09:17:39 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-7[40306]: 2024-08-30T09:17:39.382+0000 7f33e554b640 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:15.442207+0000 front 2024-08-30T09:17:15.442257+0000 (oldest deadline 2024-08-30T09:17:37.642922+0000) 2024-08-30T09:17:39.883 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:17:39 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9[55148]: 2024-08-30T09:17:39.438+0000 7f7587f3d640 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:07.800304+0000 front 2024-08-30T09:17:07.800409+0000 (oldest deadline 2024-08-30T09:17:30.700443+0000) 2024-08-30T09:17:39.883 INFO:journalctl@ceph.osd.6.smithi042.stdout:Aug 30 09:17:39 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-6[48371]: 2024-08-30T09:17:39.458+0000 7fcbc47c8640 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:07.508340+0000 front 2024-08-30T09:17:07.508372+0000 (oldest deadline 2024-08-30T09:17:29.808488+0000) 2024-08-30T09:17:39.883 INFO:journalctl@ceph.osd.6.smithi042.stdout:Aug 30 09:17:39 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-6[48371]: 2024-08-30T09:17:39.458+0000 7fcbc47c8640 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:15.610149+0000 front 2024-08-30T09:17:15.610331+0000 (oldest deadline 2024-08-30T09:17:39.110116+0000) 2024-08-30T09:17:39.936 INFO:journalctl@ceph.osd.8.smithi190.stdout:Aug 30 09:17:39 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-8[40364]: 2024-08-30T09:17:39.665+0000 7fa492b20640 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:14.793297+0000 front 2024-08-30T09:17:14.793353+0000 (oldest deadline 2024-08-30T09:17:38.293306+0000) 2024-08-30T09:17:39.937 INFO:journalctl@ceph.osd.5.smithi190.stdout:Aug 30 09:17:39 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-5[33866]: 2024-08-30T09:17:39.771+0000 7ffa64ccc640 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:06.221012+0000 front 2024-08-30T09:17:06.220784+0000 (oldest deadline 2024-08-30T09:17:31.521037+0000) 2024-08-30T09:17:39.937 INFO:journalctl@ceph.osd.5.smithi190.stdout:Aug 30 09:17:39 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-5[33866]: 2024-08-30T09:17:39.771+0000 7ffa64ccc640 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:14.923100+0000 front 2024-08-30T09:17:14.922995+0000 (oldest deadline 2024-08-30T09:17:37.822873+0000) 2024-08-30T09:17:39.937 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:17:39 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:17:39.856+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:04.091032+0000 front 2024-08-30T09:17:04.090868+0000 (oldest deadline 2024-08-30T09:17:29.391066+0000) 2024-08-30T09:17:39.937 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:17:39 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:17:39.856+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:12.193073+0000 front 2024-08-30T09:17:12.192820+0000 (oldest deadline 2024-08-30T09:17:37.492761+0000) 2024-08-30T09:17:40.077 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:17:39 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:17:39.680+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:03.239730+0000 front 2024-08-30T09:17:03.239661+0000 (oldest deadline 2024-08-30T09:17:29.139608+0000) 2024-08-30T09:17:40.077 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:17:39 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:17:39.680+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:12.040451+0000 front 2024-08-30T09:17:12.040473+0000 (oldest deadline 2024-08-30T09:17:37.841115+0000) 2024-08-30T09:17:40.715 INFO:journalctl@ceph.osd.7.smithi167.stdout:Aug 30 09:17:40 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-7[40306]: 2024-08-30T09:17:40.364+0000 7f33e554b640 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:15.442207+0000 front 2024-08-30T09:17:15.442257+0000 (oldest deadline 2024-08-30T09:17:37.642922+0000) 2024-08-30T09:17:40.883 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:17:40 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9[55148]: 2024-08-30T09:17:40.465+0000 7f7587f3d640 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:07.800304+0000 front 2024-08-30T09:17:07.800409+0000 (oldest deadline 2024-08-30T09:17:30.700443+0000) 2024-08-30T09:17:40.883 INFO:journalctl@ceph.osd.6.smithi042.stdout:Aug 30 09:17:40 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-6[48371]: 2024-08-30T09:17:40.449+0000 7fcbc47c8640 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:07.508340+0000 front 2024-08-30T09:17:07.508372+0000 (oldest deadline 2024-08-30T09:17:29.808488+0000) 2024-08-30T09:17:40.883 INFO:journalctl@ceph.osd.6.smithi042.stdout:Aug 30 09:17:40 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-6[48371]: 2024-08-30T09:17:40.449+0000 7fcbc47c8640 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:15.610149+0000 front 2024-08-30T09:17:15.610331+0000 (oldest deadline 2024-08-30T09:17:39.110116+0000) 2024-08-30T09:17:40.936 INFO:journalctl@ceph.osd.5.smithi190.stdout:Aug 30 09:17:40 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-5[33866]: 2024-08-30T09:17:40.746+0000 7ffa64ccc640 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:06.221012+0000 front 2024-08-30T09:17:06.220784+0000 (oldest deadline 2024-08-30T09:17:31.521037+0000) 2024-08-30T09:17:40.936 INFO:journalctl@ceph.osd.5.smithi190.stdout:Aug 30 09:17:40 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-5[33866]: 2024-08-30T09:17:40.746+0000 7ffa64ccc640 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:14.923100+0000 front 2024-08-30T09:17:14.922995+0000 (oldest deadline 2024-08-30T09:17:37.822873+0000) 2024-08-30T09:17:40.937 INFO:journalctl@ceph.osd.8.smithi190.stdout:Aug 30 09:17:40 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-8[40364]: 2024-08-30T09:17:40.665+0000 7fa492b20640 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:14.793297+0000 front 2024-08-30T09:17:14.793353+0000 (oldest deadline 2024-08-30T09:17:38.293306+0000) 2024-08-30T09:17:40.937 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:17:40 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:17:40.869+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:04.091032+0000 front 2024-08-30T09:17:04.090868+0000 (oldest deadline 2024-08-30T09:17:29.391066+0000) 2024-08-30T09:17:40.937 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:17:40 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:17:40.869+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:12.193073+0000 front 2024-08-30T09:17:12.192820+0000 (oldest deadline 2024-08-30T09:17:37.492761+0000) 2024-08-30T09:17:41.077 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:17:40 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:17:40.713+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:03.239730+0000 front 2024-08-30T09:17:03.239661+0000 (oldest deadline 2024-08-30T09:17:29.139608+0000) 2024-08-30T09:17:41.077 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:17:40 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:17:40.713+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:12.040451+0000 front 2024-08-30T09:17:12.040473+0000 (oldest deadline 2024-08-30T09:17:37.841115+0000) 2024-08-30T09:17:41.696 INFO:journalctl@ceph.osd.7.smithi167.stdout:Aug 30 09:17:41 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-7[40306]: 2024-08-30T09:17:41.380+0000 7f33e554b640 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:15.442207+0000 front 2024-08-30T09:17:15.442257+0000 (oldest deadline 2024-08-30T09:17:37.642922+0000) 2024-08-30T09:17:41.883 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:17:41 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9[55148]: 2024-08-30T09:17:41.463+0000 7f7587f3d640 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:07.800304+0000 front 2024-08-30T09:17:07.800409+0000 (oldest deadline 2024-08-30T09:17:30.700443+0000) 2024-08-30T09:17:41.883 INFO:journalctl@ceph.osd.6.smithi042.stdout:Aug 30 09:17:41 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-6[48371]: 2024-08-30T09:17:41.414+0000 7fcbc47c8640 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:07.508340+0000 front 2024-08-30T09:17:07.508372+0000 (oldest deadline 2024-08-30T09:17:29.808488+0000) 2024-08-30T09:17:41.883 INFO:journalctl@ceph.osd.6.smithi042.stdout:Aug 30 09:17:41 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-6[48371]: 2024-08-30T09:17:41.414+0000 7fcbc47c8640 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:15.610149+0000 front 2024-08-30T09:17:15.610331+0000 (oldest deadline 2024-08-30T09:17:39.110116+0000) 2024-08-30T09:17:41.936 INFO:journalctl@ceph.osd.5.smithi190.stdout:Aug 30 09:17:41 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-5[33866]: 2024-08-30T09:17:41.745+0000 7ffa64ccc640 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:06.221012+0000 front 2024-08-30T09:17:06.220784+0000 (oldest deadline 2024-08-30T09:17:31.521037+0000) 2024-08-30T09:17:41.936 INFO:journalctl@ceph.osd.5.smithi190.stdout:Aug 30 09:17:41 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-5[33866]: 2024-08-30T09:17:41.745+0000 7ffa64ccc640 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:14.923100+0000 front 2024-08-30T09:17:14.922995+0000 (oldest deadline 2024-08-30T09:17:37.822873+0000) 2024-08-30T09:17:41.936 INFO:journalctl@ceph.osd.8.smithi190.stdout:Aug 30 09:17:41 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-8[40364]: 2024-08-30T09:17:41.652+0000 7fa492b20640 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:14.793297+0000 front 2024-08-30T09:17:14.793353+0000 (oldest deadline 2024-08-30T09:17:38.293306+0000) 2024-08-30T09:17:41.937 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:17:41 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:17:41.841+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:04.091032+0000 front 2024-08-30T09:17:04.090868+0000 (oldest deadline 2024-08-30T09:17:29.391066+0000) 2024-08-30T09:17:41.937 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:17:41 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:17:41.841+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:12.193073+0000 front 2024-08-30T09:17:12.192820+0000 (oldest deadline 2024-08-30T09:17:37.492761+0000) 2024-08-30T09:17:42.077 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:17:41 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:17:41.694+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:03.239730+0000 front 2024-08-30T09:17:03.239661+0000 (oldest deadline 2024-08-30T09:17:29.139608+0000) 2024-08-30T09:17:42.077 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:17:41 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:17:41.694+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:12.040451+0000 front 2024-08-30T09:17:12.040473+0000 (oldest deadline 2024-08-30T09:17:37.841115+0000) 2024-08-30T09:17:42.577 INFO:journalctl@ceph.osd.7.smithi167.stdout:Aug 30 09:17:42 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-7[40306]: 2024-08-30T09:17:42.384+0000 7f33e554b640 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:15.442207+0000 front 2024-08-30T09:17:15.442257+0000 (oldest deadline 2024-08-30T09:17:37.642922+0000) 2024-08-30T09:17:42.577 INFO:journalctl@ceph.osd.4.smithi167.stdout:Aug 30 09:17:42 smithi167 podman[58107]: 2024-08-30 09:17:42.265299285 +0000 UTC m=+5.810631900 container died 69bb5c8c83a599c0472afcfbfafdfe0efdc71c5ba51755dc8583fc620e4ba0c1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-4, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, ceph=True, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, io.buildah.version=1.37.1, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=) 2024-08-30T09:17:42.883 INFO:journalctl@ceph.osd.6.smithi042.stdout:Aug 30 09:17:42 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-6[48371]: 2024-08-30T09:17:42.386+0000 7fcbc47c8640 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:07.508340+0000 front 2024-08-30T09:17:07.508372+0000 (oldest deadline 2024-08-30T09:17:29.808488+0000) 2024-08-30T09:17:42.883 INFO:journalctl@ceph.osd.6.smithi042.stdout:Aug 30 09:17:42 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-6[48371]: 2024-08-30T09:17:42.386+0000 7fcbc47c8640 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:15.610149+0000 front 2024-08-30T09:17:15.610331+0000 (oldest deadline 2024-08-30T09:17:39.110116+0000) 2024-08-30T09:17:42.883 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:17:42 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9[55148]: 2024-08-30T09:17:42.476+0000 7f7587f3d640 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:07.800304+0000 front 2024-08-30T09:17:07.800409+0000 (oldest deadline 2024-08-30T09:17:30.700443+0000) 2024-08-30T09:17:42.883 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:17:42 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9[55148]: 2024-08-30T09:17:42.476+0000 7f7587f3d640 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:16.601811+0000 front 2024-08-30T09:17:16.601614+0000 (oldest deadline 2024-08-30T09:17:41.901651+0000) 2024-08-30T09:17:42.885 INFO:journalctl@ceph.osd.8.smithi190.stdout:Aug 30 09:17:42 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-8[40364]: 2024-08-30T09:17:42.620+0000 7fa492b20640 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:14.793297+0000 front 2024-08-30T09:17:14.793353+0000 (oldest deadline 2024-08-30T09:17:38.293306+0000) 2024-08-30T09:17:42.885 INFO:journalctl@ceph.osd.5.smithi190.stdout:Aug 30 09:17:42 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-5[33866]: 2024-08-30T09:17:42.736+0000 7ffa64ccc640 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:06.221012+0000 front 2024-08-30T09:17:06.220784+0000 (oldest deadline 2024-08-30T09:17:31.521037+0000) 2024-08-30T09:17:42.885 INFO:journalctl@ceph.osd.5.smithi190.stdout:Aug 30 09:17:42 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-5[33866]: 2024-08-30T09:17:42.736+0000 7ffa64ccc640 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:14.923100+0000 front 2024-08-30T09:17:14.922995+0000 (oldest deadline 2024-08-30T09:17:37.822873+0000) 2024-08-30T09:17:42.947 INFO:journalctl@ceph.osd.4.smithi167.stdout:Aug 30 09:17:42 smithi167 podman[58107]: 2024-08-30 09:17:42.685751345 +0000 UTC m=+6.231083969 container cleanup 69bb5c8c83a599c0472afcfbfafdfe0efdc71c5ba51755dc8583fc620e4ba0c1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-4, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.37.1, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image) 2024-08-30T09:17:42.947 INFO:journalctl@ceph.osd.4.smithi167.stdout:Aug 30 09:17:42 smithi167 bash[58107]: ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-4 2024-08-30T09:17:42.947 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:17:42 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:17:42.736+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:03.239730+0000 front 2024-08-30T09:17:03.239661+0000 (oldest deadline 2024-08-30T09:17:29.139608+0000) 2024-08-30T09:17:42.947 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:17:42 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:17:42.736+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:12.040451+0000 front 2024-08-30T09:17:12.040473+0000 (oldest deadline 2024-08-30T09:17:37.841115+0000) 2024-08-30T09:17:43.186 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:17:42 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:17:42.883+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:04.091032+0000 front 2024-08-30T09:17:04.090868+0000 (oldest deadline 2024-08-30T09:17:29.391066+0000) 2024-08-30T09:17:43.186 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:17:42 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:17:42.883+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:12.193073+0000 front 2024-08-30T09:17:12.192820+0000 (oldest deadline 2024-08-30T09:17:37.492761+0000) 2024-08-30T09:17:43.327 INFO:journalctl@ceph.osd.4.smithi167.stdout:Aug 30 09:17:42 smithi167 podman[58123]: 2024-08-30 09:17:42.945500701 +0000 UTC m=+0.667119427 container remove 69bb5c8c83a599c0472afcfbfafdfe0efdc71c5ba51755dc8583fc620e4ba0c1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-4, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.37.1, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-08-30T09:17:43.675 INFO:journalctl@ceph.osd.7.smithi167.stdout:Aug 30 09:17:43 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-7[40306]: 2024-08-30T09:17:43.426+0000 7f33e554b640 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:15.442207+0000 front 2024-08-30T09:17:15.442257+0000 (oldest deadline 2024-08-30T09:17:37.642922+0000) 2024-08-30T09:17:43.675 INFO:journalctl@ceph.osd.4.smithi167.stdout:Aug 30 09:17:43 smithi167 podman[58210]: 2024-08-30 09:17:43.254330999 +0000 UTC m=+0.015830236 image pull 79f4cde5e36911a5dca995eeb98098c2c8aae81dd52a088ef2c867bc3363bad0 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61 2024-08-30T09:17:43.676 INFO:journalctl@ceph.osd.4.smithi167.stdout:Aug 30 09:17:43 smithi167 podman[58210]: 2024-08-30 09:17:43.363558512 +0000 UTC m=+0.125057814 container create e561ac5eb3cf669d1459b8dc57b071b4c81ee0ac96a48a85be27bae57cef1337 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-4-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.vendor=CentOS, io.buildah.version=1.37.1, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-08-30T09:17:43.676 INFO:journalctl@ceph.osd.4.smithi167.stdout:Aug 30 09:17:43 smithi167 podman[58210]: 2024-08-30 09:17:43.591018641 +0000 UTC m=+0.352517868 container init e561ac5eb3cf669d1459b8dc57b071b4c81ee0ac96a48a85be27bae57cef1337 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-4-deactivate, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.37.1, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True) 2024-08-30T09:17:43.676 INFO:journalctl@ceph.osd.4.smithi167.stdout:Aug 30 09:17:43 smithi167 podman[58210]: 2024-08-30 09:17:43.594472828 +0000 UTC m=+0.355972058 container start e561ac5eb3cf669d1459b8dc57b071b4c81ee0ac96a48a85be27bae57cef1337 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-4-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, io.buildah.version=1.37.1, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=) 2024-08-30T09:17:43.676 INFO:journalctl@ceph.osd.4.smithi167.stdout:Aug 30 09:17:43 smithi167 podman[58210]: 2024-08-30 09:17:43.674316385 +0000 UTC m=+0.435815613 container attach e561ac5eb3cf669d1459b8dc57b071b4c81ee0ac96a48a85be27bae57cef1337 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-4-deactivate, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.1, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=) 2024-08-30T09:17:43.883 INFO:journalctl@ceph.osd.6.smithi042.stdout:Aug 30 09:17:43 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-6[48371]: 2024-08-30T09:17:43.431+0000 7fcbc47c8640 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:07.508340+0000 front 2024-08-30T09:17:07.508372+0000 (oldest deadline 2024-08-30T09:17:29.808488+0000) 2024-08-30T09:17:43.883 INFO:journalctl@ceph.osd.6.smithi042.stdout:Aug 30 09:17:43 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-6[48371]: 2024-08-30T09:17:43.431+0000 7fcbc47c8640 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:15.610149+0000 front 2024-08-30T09:17:15.610331+0000 (oldest deadline 2024-08-30T09:17:39.110116+0000) 2024-08-30T09:17:43.883 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:17:43 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9[55148]: 2024-08-30T09:17:43.453+0000 7f7587f3d640 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:07.800304+0000 front 2024-08-30T09:17:07.800409+0000 (oldest deadline 2024-08-30T09:17:30.700443+0000) 2024-08-30T09:17:43.883 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:17:43 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9[55148]: 2024-08-30T09:17:43.453+0000 7f7587f3d640 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:16.601811+0000 front 2024-08-30T09:17:16.601614+0000 (oldest deadline 2024-08-30T09:17:41.901651+0000) 2024-08-30T09:17:43.936 INFO:journalctl@ceph.osd.5.smithi190.stdout:Aug 30 09:17:43 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-5[33866]: 2024-08-30T09:17:43.700+0000 7ffa64ccc640 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:06.221012+0000 front 2024-08-30T09:17:06.220784+0000 (oldest deadline 2024-08-30T09:17:31.521037+0000) 2024-08-30T09:17:43.936 INFO:journalctl@ceph.osd.5.smithi190.stdout:Aug 30 09:17:43 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-5[33866]: 2024-08-30T09:17:43.700+0000 7ffa64ccc640 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:14.923100+0000 front 2024-08-30T09:17:14.922995+0000 (oldest deadline 2024-08-30T09:17:37.822873+0000) 2024-08-30T09:17:43.936 INFO:journalctl@ceph.osd.8.smithi190.stdout:Aug 30 09:17:43 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-8[40364]: 2024-08-30T09:17:43.666+0000 7fa492b20640 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:14.793297+0000 front 2024-08-30T09:17:14.793353+0000 (oldest deadline 2024-08-30T09:17:38.293306+0000) 2024-08-30T09:17:43.937 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:17:43 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:17:43.909+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:04.091032+0000 front 2024-08-30T09:17:04.090868+0000 (oldest deadline 2024-08-30T09:17:29.391066+0000) 2024-08-30T09:17:43.937 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:17:43 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:17:43.909+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:12.193073+0000 front 2024-08-30T09:17:12.192820+0000 (oldest deadline 2024-08-30T09:17:37.492761+0000) 2024-08-30T09:17:44.077 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:17:43 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:17:43.716+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:03.239730+0000 front 2024-08-30T09:17:03.239661+0000 (oldest deadline 2024-08-30T09:17:29.139608+0000) 2024-08-30T09:17:44.077 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:17:43 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:17:43.716+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:12.040451+0000 front 2024-08-30T09:17:12.040473+0000 (oldest deadline 2024-08-30T09:17:37.841115+0000) 2024-08-30T09:17:44.077 INFO:journalctl@ceph.osd.4.smithi167.stdout:Aug 30 09:17:43 smithi167 podman[58210]: 2024-08-30 09:17:43.794013044 +0000 UTC m=+0.555512280 container died e561ac5eb3cf669d1459b8dc57b071b4c81ee0ac96a48a85be27bae57cef1337 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-4-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, io.buildah.version=1.37.1, ceph=True) 2024-08-30T09:17:44.398 DEBUG:teuthology.orchestra.run.smithi167:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968@osd.4.service' 2024-08-30T09:17:44.437 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-30T09:17:44.438 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-08-30T09:17:44.438 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-08-30T09:17:44.438 DEBUG:teuthology.orchestra.run.smithi190:> sudo systemctl stop ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968@osd.5 2024-08-30T09:17:44.759 INFO:journalctl@ceph.osd.8.smithi190.stdout:Aug 30 09:17:44 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-8[40364]: 2024-08-30T09:17:44.656+0000 7fa492b20640 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:14.793297+0000 front 2024-08-30T09:17:14.793353+0000 (oldest deadline 2024-08-30T09:17:38.293306+0000) 2024-08-30T09:17:44.759 INFO:journalctl@ceph.osd.5.smithi190.stdout:Aug 30 09:17:44 smithi190 systemd[1]: Stopping Ceph osd.5 for b0895c1a-66ac-11ef-bcd4-c7b262605968... 2024-08-30T09:17:44.759 INFO:journalctl@ceph.osd.5.smithi190.stdout:Aug 30 09:17:44 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-5[33866]: 2024-08-30T09:17:44.702+0000 7ffa64ccc640 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:06.221012+0000 front 2024-08-30T09:17:06.220784+0000 (oldest deadline 2024-08-30T09:17:31.521037+0000) 2024-08-30T09:17:44.759 INFO:journalctl@ceph.osd.5.smithi190.stdout:Aug 30 09:17:44 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-5[33866]: 2024-08-30T09:17:44.702+0000 7ffa64ccc640 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:14.923100+0000 front 2024-08-30T09:17:14.922995+0000 (oldest deadline 2024-08-30T09:17:37.822873+0000) 2024-08-30T09:17:44.759 INFO:journalctl@ceph.osd.5.smithi190.stdout:Aug 30 09:17:44 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-5[33866]: 2024-08-30T09:17:44.756+0000 7ffa6c6e9640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-30T09:17:44.759 INFO:journalctl@ceph.osd.5.smithi190.stdout:Aug 30 09:17:44 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-5[33866]: 2024-08-30T09:17:44.756+0000 7ffa6c6e9640 -1 osd.5 87 *** Got signal Terminated *** 2024-08-30T09:17:44.759 INFO:journalctl@ceph.osd.5.smithi190.stdout:Aug 30 09:17:44 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-5[33866]: 2024-08-30T09:17:44.756+0000 7ffa6c6e9640 -1 osd.5 87 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-08-30T09:17:44.762 INFO:journalctl@ceph.osd.7.smithi167.stdout:Aug 30 09:17:44 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-7[40306]: 2024-08-30T09:17:44.437+0000 7f33e554b640 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:15.442207+0000 front 2024-08-30T09:17:15.442257+0000 (oldest deadline 2024-08-30T09:17:37.642922+0000) 2024-08-30T09:17:44.883 INFO:journalctl@ceph.osd.6.smithi042.stdout:Aug 30 09:17:44 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-6[48371]: 2024-08-30T09:17:44.457+0000 7fcbc47c8640 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:07.508340+0000 front 2024-08-30T09:17:07.508372+0000 (oldest deadline 2024-08-30T09:17:29.808488+0000) 2024-08-30T09:17:44.883 INFO:journalctl@ceph.osd.6.smithi042.stdout:Aug 30 09:17:44 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-6[48371]: 2024-08-30T09:17:44.457+0000 7fcbc47c8640 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:15.610149+0000 front 2024-08-30T09:17:15.610331+0000 (oldest deadline 2024-08-30T09:17:39.110116+0000) 2024-08-30T09:17:44.883 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:17:44 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9[55148]: 2024-08-30T09:17:44.465+0000 7f7587f3d640 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:07.800304+0000 front 2024-08-30T09:17:07.800409+0000 (oldest deadline 2024-08-30T09:17:30.700443+0000) 2024-08-30T09:17:44.883 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:17:44 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9[55148]: 2024-08-30T09:17:44.465+0000 7f7587f3d640 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:16.601811+0000 front 2024-08-30T09:17:16.601614+0000 (oldest deadline 2024-08-30T09:17:41.901651+0000) 2024-08-30T09:17:45.077 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:17:44 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:17:44.760+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:03.239730+0000 front 2024-08-30T09:17:03.239661+0000 (oldest deadline 2024-08-30T09:17:29.139608+0000) 2024-08-30T09:17:45.077 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:17:44 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:17:44.760+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:12.040451+0000 front 2024-08-30T09:17:12.040473+0000 (oldest deadline 2024-08-30T09:17:37.841115+0000) 2024-08-30T09:17:45.186 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:17:44 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:17:44.895+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:04.091032+0000 front 2024-08-30T09:17:04.090868+0000 (oldest deadline 2024-08-30T09:17:29.391066+0000) 2024-08-30T09:17:45.186 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:17:44 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:17:44.895+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:12.193073+0000 front 2024-08-30T09:17:12.192820+0000 (oldest deadline 2024-08-30T09:17:37.492761+0000) 2024-08-30T09:17:45.768 INFO:journalctl@ceph.osd.7.smithi167.stdout:Aug 30 09:17:45 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-7[40306]: 2024-08-30T09:17:45.431+0000 7f33e554b640 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:15.442207+0000 front 2024-08-30T09:17:15.442257+0000 (oldest deadline 2024-08-30T09:17:37.642922+0000) 2024-08-30T09:17:45.883 INFO:journalctl@ceph.osd.6.smithi042.stdout:Aug 30 09:17:45 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-6[48371]: 2024-08-30T09:17:45.416+0000 7fcbc47c8640 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:07.508340+0000 front 2024-08-30T09:17:07.508372+0000 (oldest deadline 2024-08-30T09:17:29.808488+0000) 2024-08-30T09:17:45.883 INFO:journalctl@ceph.osd.6.smithi042.stdout:Aug 30 09:17:45 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-6[48371]: 2024-08-30T09:17:45.416+0000 7fcbc47c8640 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:15.610149+0000 front 2024-08-30T09:17:15.610331+0000 (oldest deadline 2024-08-30T09:17:39.110116+0000) 2024-08-30T09:17:45.884 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:17:45 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9[55148]: 2024-08-30T09:17:45.479+0000 7f7587f3d640 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:07.800304+0000 front 2024-08-30T09:17:07.800409+0000 (oldest deadline 2024-08-30T09:17:30.700443+0000) 2024-08-30T09:17:45.884 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:17:45 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9[55148]: 2024-08-30T09:17:45.479+0000 7f7587f3d640 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:16.601811+0000 front 2024-08-30T09:17:16.601614+0000 (oldest deadline 2024-08-30T09:17:41.901651+0000) 2024-08-30T09:17:45.936 INFO:journalctl@ceph.osd.5.smithi190.stdout:Aug 30 09:17:45 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-5[33866]: 2024-08-30T09:17:45.669+0000 7ffa64ccc640 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:06.221012+0000 front 2024-08-30T09:17:06.220784+0000 (oldest deadline 2024-08-30T09:17:31.521037+0000) 2024-08-30T09:17:45.936 INFO:journalctl@ceph.osd.5.smithi190.stdout:Aug 30 09:17:45 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-5[33866]: 2024-08-30T09:17:45.669+0000 7ffa64ccc640 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:14.923100+0000 front 2024-08-30T09:17:14.922995+0000 (oldest deadline 2024-08-30T09:17:37.822873+0000) 2024-08-30T09:17:45.937 INFO:journalctl@ceph.osd.8.smithi190.stdout:Aug 30 09:17:45 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-8[40364]: 2024-08-30T09:17:45.607+0000 7fa492b20640 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:14.793297+0000 front 2024-08-30T09:17:14.793353+0000 (oldest deadline 2024-08-30T09:17:38.293306+0000) 2024-08-30T09:17:45.937 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:17:45 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:17:45.851+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:04.091032+0000 front 2024-08-30T09:17:04.090868+0000 (oldest deadline 2024-08-30T09:17:29.391066+0000) 2024-08-30T09:17:45.937 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:17:45 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:17:45.851+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:12.193073+0000 front 2024-08-30T09:17:12.192820+0000 (oldest deadline 2024-08-30T09:17:37.492761+0000) 2024-08-30T09:17:46.077 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:17:45 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:17:45.765+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:03.239730+0000 front 2024-08-30T09:17:03.239661+0000 (oldest deadline 2024-08-30T09:17:29.139608+0000) 2024-08-30T09:17:46.077 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:17:45 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:17:45.765+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:12.040451+0000 front 2024-08-30T09:17:12.040473+0000 (oldest deadline 2024-08-30T09:17:37.841115+0000) 2024-08-30T09:17:46.790 INFO:journalctl@ceph.osd.7.smithi167.stdout:Aug 30 09:17:46 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-7[40306]: 2024-08-30T09:17:46.440+0000 7f33e554b640 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:15.442207+0000 front 2024-08-30T09:17:15.442257+0000 (oldest deadline 2024-08-30T09:17:37.642922+0000) 2024-08-30T09:17:46.883 INFO:journalctl@ceph.osd.6.smithi042.stdout:Aug 30 09:17:46 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-6[48371]: 2024-08-30T09:17:46.413+0000 7fcbc47c8640 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:07.508340+0000 front 2024-08-30T09:17:07.508372+0000 (oldest deadline 2024-08-30T09:17:29.808488+0000) 2024-08-30T09:17:46.883 INFO:journalctl@ceph.osd.6.smithi042.stdout:Aug 30 09:17:46 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-6[48371]: 2024-08-30T09:17:46.413+0000 7fcbc47c8640 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:15.610149+0000 front 2024-08-30T09:17:15.610331+0000 (oldest deadline 2024-08-30T09:17:39.110116+0000) 2024-08-30T09:17:46.883 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:17:46 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9[55148]: 2024-08-30T09:17:46.442+0000 7f7587f3d640 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:07.800304+0000 front 2024-08-30T09:17:07.800409+0000 (oldest deadline 2024-08-30T09:17:30.700443+0000) 2024-08-30T09:17:46.883 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:17:46 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9[55148]: 2024-08-30T09:17:46.442+0000 7f7587f3d640 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:16.601811+0000 front 2024-08-30T09:17:16.601614+0000 (oldest deadline 2024-08-30T09:17:41.901651+0000) 2024-08-30T09:17:46.936 INFO:journalctl@ceph.osd.8.smithi190.stdout:Aug 30 09:17:46 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-8[40364]: 2024-08-30T09:17:46.586+0000 7fa492b20640 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:14.793297+0000 front 2024-08-30T09:17:14.793353+0000 (oldest deadline 2024-08-30T09:17:38.293306+0000) 2024-08-30T09:17:46.936 INFO:journalctl@ceph.osd.5.smithi190.stdout:Aug 30 09:17:46 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-5[33866]: 2024-08-30T09:17:46.689+0000 7ffa64ccc640 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:06.221012+0000 front 2024-08-30T09:17:06.220784+0000 (oldest deadline 2024-08-30T09:17:31.521037+0000) 2024-08-30T09:17:46.936 INFO:journalctl@ceph.osd.5.smithi190.stdout:Aug 30 09:17:46 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-5[33866]: 2024-08-30T09:17:46.689+0000 7ffa64ccc640 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:14.923100+0000 front 2024-08-30T09:17:14.922995+0000 (oldest deadline 2024-08-30T09:17:37.822873+0000) 2024-08-30T09:17:46.937 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:17:46 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:17:46.826+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:04.091032+0000 front 2024-08-30T09:17:04.090868+0000 (oldest deadline 2024-08-30T09:17:29.391066+0000) 2024-08-30T09:17:46.937 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:17:46 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:17:46.826+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:12.193073+0000 front 2024-08-30T09:17:12.192820+0000 (oldest deadline 2024-08-30T09:17:37.492761+0000) 2024-08-30T09:17:46.937 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:17:46 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:17:46.826+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-30T09:17:24.494221+0000 front 2024-08-30T09:17:24.494455+0000 (oldest deadline 2024-08-30T09:17:46.194420+0000) 2024-08-30T09:17:47.077 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:17:46 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:17:46.787+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:03.239730+0000 front 2024-08-30T09:17:03.239661+0000 (oldest deadline 2024-08-30T09:17:29.139608+0000) 2024-08-30T09:17:47.077 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:17:46 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:17:46.787+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:12.040451+0000 front 2024-08-30T09:17:12.040473+0000 (oldest deadline 2024-08-30T09:17:37.841115+0000) 2024-08-30T09:17:47.633 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:17:47 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9[55148]: 2024-08-30T09:17:47.429+0000 7f7587f3d640 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:07.800304+0000 front 2024-08-30T09:17:07.800409+0000 (oldest deadline 2024-08-30T09:17:30.700443+0000) 2024-08-30T09:17:47.633 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:17:47 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9[55148]: 2024-08-30T09:17:47.429+0000 7f7587f3d640 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:16.601811+0000 front 2024-08-30T09:17:16.601614+0000 (oldest deadline 2024-08-30T09:17:41.901651+0000) 2024-08-30T09:17:47.633 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:17:47 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9[55148]: 2024-08-30T09:17:47.429+0000 7f7587f3d640 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-30T09:17:21.902428+0000 front 2024-08-30T09:17:21.902400+0000 (oldest deadline 2024-08-30T09:17:47.202527+0000) 2024-08-30T09:17:47.634 INFO:journalctl@ceph.osd.6.smithi042.stdout:Aug 30 09:17:47 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-6[48371]: 2024-08-30T09:17:47.380+0000 7fcbc47c8640 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:07.508340+0000 front 2024-08-30T09:17:07.508372+0000 (oldest deadline 2024-08-30T09:17:29.808488+0000) 2024-08-30T09:17:47.634 INFO:journalctl@ceph.osd.6.smithi042.stdout:Aug 30 09:17:47 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-6[48371]: 2024-08-30T09:17:47.380+0000 7fcbc47c8640 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:15.610149+0000 front 2024-08-30T09:17:15.610331+0000 (oldest deadline 2024-08-30T09:17:39.110116+0000) 2024-08-30T09:17:47.634 INFO:journalctl@ceph.osd.6.smithi042.stdout:Aug 30 09:17:47 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-6[48371]: 2024-08-30T09:17:47.380+0000 7fcbc47c8640 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-30T09:17:25.011349+0000 front 2024-08-30T09:17:25.011311+0000 (oldest deadline 2024-08-30T09:17:47.311396+0000) 2024-08-30T09:17:47.795 INFO:journalctl@ceph.osd.7.smithi167.stdout:Aug 30 09:17:47 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-7[40306]: 2024-08-30T09:17:47.445+0000 7f33e554b640 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:15.442207+0000 front 2024-08-30T09:17:15.442257+0000 (oldest deadline 2024-08-30T09:17:37.642922+0000) 2024-08-30T09:17:47.876 INFO:journalctl@ceph.osd.8.smithi190.stdout:Aug 30 09:17:47 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-8[40364]: 2024-08-30T09:17:47.580+0000 7fa492b20640 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:14.793297+0000 front 2024-08-30T09:17:14.793353+0000 (oldest deadline 2024-08-30T09:17:38.293306+0000) 2024-08-30T09:17:47.877 INFO:journalctl@ceph.osd.5.smithi190.stdout:Aug 30 09:17:47 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-5[33866]: 2024-08-30T09:17:47.723+0000 7ffa64ccc640 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:06.221012+0000 front 2024-08-30T09:17:06.220784+0000 (oldest deadline 2024-08-30T09:17:31.521037+0000) 2024-08-30T09:17:47.877 INFO:journalctl@ceph.osd.5.smithi190.stdout:Aug 30 09:17:47 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-5[33866]: 2024-08-30T09:17:47.723+0000 7ffa64ccc640 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:14.923100+0000 front 2024-08-30T09:17:14.922995+0000 (oldest deadline 2024-08-30T09:17:37.822873+0000) 2024-08-30T09:17:47.877 INFO:journalctl@ceph.osd.5.smithi190.stdout:Aug 30 09:17:47 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-5[33866]: 2024-08-30T09:17:47.723+0000 7ffa64ccc640 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-30T09:17:23.723877+0000 front 2024-08-30T09:17:23.723947+0000 (oldest deadline 2024-08-30T09:17:47.224031+0000) 2024-08-30T09:17:48.077 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:17:47 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:17:47.792+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:03.239730+0000 front 2024-08-30T09:17:03.239661+0000 (oldest deadline 2024-08-30T09:17:29.139608+0000) 2024-08-30T09:17:48.077 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:17:47 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:17:47.792+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:12.040451+0000 front 2024-08-30T09:17:12.040473+0000 (oldest deadline 2024-08-30T09:17:37.841115+0000) 2024-08-30T09:17:48.186 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:17:47 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:17:47.874+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:04.091032+0000 front 2024-08-30T09:17:04.090868+0000 (oldest deadline 2024-08-30T09:17:29.391066+0000) 2024-08-30T09:17:48.186 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:17:47 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:17:47.874+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:12.193073+0000 front 2024-08-30T09:17:12.192820+0000 (oldest deadline 2024-08-30T09:17:37.492761+0000) 2024-08-30T09:17:48.187 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:17:47 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:17:47.874+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-30T09:17:24.494221+0000 front 2024-08-30T09:17:24.494455+0000 (oldest deadline 2024-08-30T09:17:46.194420+0000) 2024-08-30T09:17:48.577 INFO:journalctl@ceph.osd.7.smithi167.stdout:Aug 30 09:17:48 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-7[40306]: 2024-08-30T09:17:48.495+0000 7f33e554b640 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:15.442207+0000 front 2024-08-30T09:17:15.442257+0000 (oldest deadline 2024-08-30T09:17:37.642922+0000) 2024-08-30T09:17:48.577 INFO:journalctl@ceph.osd.7.smithi167.stdout:Aug 30 09:17:48 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-7[40306]: 2024-08-30T09:17:48.495+0000 7f33e554b640 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-30T09:17:21.644664+0000 front 2024-08-30T09:17:21.644755+0000 (oldest deadline 2024-08-30T09:17:47.544689+0000) 2024-08-30T09:17:48.883 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:17:48 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9[55148]: 2024-08-30T09:17:48.412+0000 7f7587f3d640 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:07.800304+0000 front 2024-08-30T09:17:07.800409+0000 (oldest deadline 2024-08-30T09:17:30.700443+0000) 2024-08-30T09:17:48.883 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:17:48 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9[55148]: 2024-08-30T09:17:48.412+0000 7f7587f3d640 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:16.601811+0000 front 2024-08-30T09:17:16.601614+0000 (oldest deadline 2024-08-30T09:17:41.901651+0000) 2024-08-30T09:17:48.883 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:17:48 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9[55148]: 2024-08-30T09:17:48.412+0000 7f7587f3d640 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-30T09:17:21.902428+0000 front 2024-08-30T09:17:21.902400+0000 (oldest deadline 2024-08-30T09:17:47.202527+0000) 2024-08-30T09:17:48.883 INFO:journalctl@ceph.osd.6.smithi042.stdout:Aug 30 09:17:48 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-6[48371]: 2024-08-30T09:17:48.427+0000 7fcbc47c8640 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:07.508340+0000 front 2024-08-30T09:17:07.508372+0000 (oldest deadline 2024-08-30T09:17:29.808488+0000) 2024-08-30T09:17:48.883 INFO:journalctl@ceph.osd.6.smithi042.stdout:Aug 30 09:17:48 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-6[48371]: 2024-08-30T09:17:48.427+0000 7fcbc47c8640 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:15.610149+0000 front 2024-08-30T09:17:15.610331+0000 (oldest deadline 2024-08-30T09:17:39.110116+0000) 2024-08-30T09:17:48.883 INFO:journalctl@ceph.osd.6.smithi042.stdout:Aug 30 09:17:48 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-6[48371]: 2024-08-30T09:17:48.427+0000 7fcbc47c8640 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-30T09:17:25.011349+0000 front 2024-08-30T09:17:25.011311+0000 (oldest deadline 2024-08-30T09:17:47.311396+0000) 2024-08-30T09:17:48.936 INFO:journalctl@ceph.osd.8.smithi190.stdout:Aug 30 09:17:48 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-8[40364]: 2024-08-30T09:17:48.616+0000 7fa492b20640 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:14.793297+0000 front 2024-08-30T09:17:14.793353+0000 (oldest deadline 2024-08-30T09:17:38.293306+0000) 2024-08-30T09:17:48.936 INFO:journalctl@ceph.osd.5.smithi190.stdout:Aug 30 09:17:48 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-5[33866]: 2024-08-30T09:17:48.749+0000 7ffa64ccc640 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:06.221012+0000 front 2024-08-30T09:17:06.220784+0000 (oldest deadline 2024-08-30T09:17:31.521037+0000) 2024-08-30T09:17:48.937 INFO:journalctl@ceph.osd.5.smithi190.stdout:Aug 30 09:17:48 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-5[33866]: 2024-08-30T09:17:48.749+0000 7ffa64ccc640 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:14.923100+0000 front 2024-08-30T09:17:14.922995+0000 (oldest deadline 2024-08-30T09:17:37.822873+0000) 2024-08-30T09:17:48.937 INFO:journalctl@ceph.osd.5.smithi190.stdout:Aug 30 09:17:48 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-5[33866]: 2024-08-30T09:17:48.749+0000 7ffa64ccc640 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-30T09:17:23.723877+0000 front 2024-08-30T09:17:23.723947+0000 (oldest deadline 2024-08-30T09:17:47.224031+0000) 2024-08-30T09:17:48.937 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:17:48 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:17:48.837+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:04.091032+0000 front 2024-08-30T09:17:04.090868+0000 (oldest deadline 2024-08-30T09:17:29.391066+0000) 2024-08-30T09:17:48.937 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:17:48 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:17:48.837+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:12.193073+0000 front 2024-08-30T09:17:12.192820+0000 (oldest deadline 2024-08-30T09:17:37.492761+0000) 2024-08-30T09:17:48.937 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:17:48 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:17:48.837+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-30T09:17:24.494221+0000 front 2024-08-30T09:17:24.494455+0000 (oldest deadline 2024-08-30T09:17:46.194420+0000) 2024-08-30T09:17:49.077 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:17:48 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:17:48.812+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:03.239730+0000 front 2024-08-30T09:17:03.239661+0000 (oldest deadline 2024-08-30T09:17:29.139608+0000) 2024-08-30T09:17:49.077 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:17:48 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:17:48.812+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:12.040451+0000 front 2024-08-30T09:17:12.040473+0000 (oldest deadline 2024-08-30T09:17:37.841115+0000) 2024-08-30T09:17:49.077 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:17:48 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:17:48.812+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-30T09:17:22.442437+0000 front 2024-08-30T09:17:22.442414+0000 (oldest deadline 2024-08-30T09:17:48.243149+0000) 2024-08-30T09:17:49.806 INFO:journalctl@ceph.osd.7.smithi167.stdout:Aug 30 09:17:49 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-7[40306]: 2024-08-30T09:17:49.504+0000 7f33e554b640 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:15.442207+0000 front 2024-08-30T09:17:15.442257+0000 (oldest deadline 2024-08-30T09:17:37.642922+0000) 2024-08-30T09:17:49.806 INFO:journalctl@ceph.osd.7.smithi167.stdout:Aug 30 09:17:49 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-7[40306]: 2024-08-30T09:17:49.504+0000 7f33e554b640 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-30T09:17:21.644664+0000 front 2024-08-30T09:17:21.644755+0000 (oldest deadline 2024-08-30T09:17:47.544689+0000) 2024-08-30T09:17:49.865 INFO:journalctl@ceph.osd.8.smithi190.stdout:Aug 30 09:17:49 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-8[40364]: 2024-08-30T09:17:49.570+0000 7fa492b20640 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:14.793297+0000 front 2024-08-30T09:17:14.793353+0000 (oldest deadline 2024-08-30T09:17:38.293306+0000) 2024-08-30T09:17:49.865 INFO:journalctl@ceph.osd.8.smithi190.stdout:Aug 30 09:17:49 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-8[40364]: 2024-08-30T09:17:49.570+0000 7fa492b20640 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-30T09:17:24.695219+0000 front 2024-08-30T09:17:24.695243+0000 (oldest deadline 2024-08-30T09:17:48.795417+0000) 2024-08-30T09:17:49.865 INFO:journalctl@ceph.osd.5.smithi190.stdout:Aug 30 09:17:49 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-5[33866]: 2024-08-30T09:17:49.731+0000 7ffa64ccc640 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:06.221012+0000 front 2024-08-30T09:17:06.220784+0000 (oldest deadline 2024-08-30T09:17:31.521037+0000) 2024-08-30T09:17:49.865 INFO:journalctl@ceph.osd.5.smithi190.stdout:Aug 30 09:17:49 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-5[33866]: 2024-08-30T09:17:49.731+0000 7ffa64ccc640 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:14.923100+0000 front 2024-08-30T09:17:14.922995+0000 (oldest deadline 2024-08-30T09:17:37.822873+0000) 2024-08-30T09:17:49.866 INFO:journalctl@ceph.osd.5.smithi190.stdout:Aug 30 09:17:49 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-5[33866]: 2024-08-30T09:17:49.731+0000 7ffa64ccc640 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-30T09:17:23.723877+0000 front 2024-08-30T09:17:23.723947+0000 (oldest deadline 2024-08-30T09:17:47.224031+0000) 2024-08-30T09:17:49.883 INFO:journalctl@ceph.osd.6.smithi042.stdout:Aug 30 09:17:49 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-6[48371]: 2024-08-30T09:17:49.468+0000 7fcbc47c8640 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:07.508340+0000 front 2024-08-30T09:17:07.508372+0000 (oldest deadline 2024-08-30T09:17:29.808488+0000) 2024-08-30T09:17:49.883 INFO:journalctl@ceph.osd.6.smithi042.stdout:Aug 30 09:17:49 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-6[48371]: 2024-08-30T09:17:49.468+0000 7fcbc47c8640 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:15.610149+0000 front 2024-08-30T09:17:15.610331+0000 (oldest deadline 2024-08-30T09:17:39.110116+0000) 2024-08-30T09:17:49.883 INFO:journalctl@ceph.osd.6.smithi042.stdout:Aug 30 09:17:49 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-6[48371]: 2024-08-30T09:17:49.468+0000 7fcbc47c8640 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-30T09:17:25.011349+0000 front 2024-08-30T09:17:25.011311+0000 (oldest deadline 2024-08-30T09:17:47.311396+0000) 2024-08-30T09:17:49.883 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:17:49 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9[55148]: 2024-08-30T09:17:49.430+0000 7f7587f3d640 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:07.800304+0000 front 2024-08-30T09:17:07.800409+0000 (oldest deadline 2024-08-30T09:17:30.700443+0000) 2024-08-30T09:17:49.883 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:17:49 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9[55148]: 2024-08-30T09:17:49.430+0000 7f7587f3d640 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:16.601811+0000 front 2024-08-30T09:17:16.601614+0000 (oldest deadline 2024-08-30T09:17:41.901651+0000) 2024-08-30T09:17:49.883 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:17:49 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9[55148]: 2024-08-30T09:17:49.430+0000 7f7587f3d640 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-30T09:17:21.902428+0000 front 2024-08-30T09:17:21.902400+0000 (oldest deadline 2024-08-30T09:17:47.202527+0000) 2024-08-30T09:17:50.077 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:17:49 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:17:49.803+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:03.239730+0000 front 2024-08-30T09:17:03.239661+0000 (oldest deadline 2024-08-30T09:17:29.139608+0000) 2024-08-30T09:17:50.077 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:17:49 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:17:49.803+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:12.040451+0000 front 2024-08-30T09:17:12.040473+0000 (oldest deadline 2024-08-30T09:17:37.841115+0000) 2024-08-30T09:17:50.077 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:17:49 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:17:49.803+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-30T09:17:22.442437+0000 front 2024-08-30T09:17:22.442414+0000 (oldest deadline 2024-08-30T09:17:48.243149+0000) 2024-08-30T09:17:50.186 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:17:49 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:17:49.863+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:04.091032+0000 front 2024-08-30T09:17:04.090868+0000 (oldest deadline 2024-08-30T09:17:29.391066+0000) 2024-08-30T09:17:50.187 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:17:49 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:17:49.863+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:12.193073+0000 front 2024-08-30T09:17:12.192820+0000 (oldest deadline 2024-08-30T09:17:37.492761+0000) 2024-08-30T09:17:50.187 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:17:49 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:17:49.863+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-30T09:17:24.494221+0000 front 2024-08-30T09:17:24.494455+0000 (oldest deadline 2024-08-30T09:17:46.194420+0000) 2024-08-30T09:17:50.463 INFO:journalctl@ceph.osd.5.smithi190.stdout:Aug 30 09:17:50 smithi190 podman[58732]: 2024-08-30 09:17:50.400062688 +0000 UTC m=+5.768994100 container died a3343420eb389be15325bab0fbc93ddca97f13c85d36b68a84d8b1b7a2241493 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.37.1, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-08-30T09:17:50.744 INFO:journalctl@ceph.osd.8.smithi190.stdout:Aug 30 09:17:50 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-8[40364]: 2024-08-30T09:17:50.617+0000 7fa492b20640 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:14.793297+0000 front 2024-08-30T09:17:14.793353+0000 (oldest deadline 2024-08-30T09:17:38.293306+0000) 2024-08-30T09:17:50.744 INFO:journalctl@ceph.osd.8.smithi190.stdout:Aug 30 09:17:50 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-8[40364]: 2024-08-30T09:17:50.617+0000 7fa492b20640 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-30T09:17:24.695219+0000 front 2024-08-30T09:17:24.695243+0000 (oldest deadline 2024-08-30T09:17:48.795417+0000) 2024-08-30T09:17:50.827 INFO:journalctl@ceph.osd.7.smithi167.stdout:Aug 30 09:17:50 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-7[40306]: 2024-08-30T09:17:50.498+0000 7f33e554b640 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:15.442207+0000 front 2024-08-30T09:17:15.442257+0000 (oldest deadline 2024-08-30T09:17:37.642922+0000) 2024-08-30T09:17:50.827 INFO:journalctl@ceph.osd.7.smithi167.stdout:Aug 30 09:17:50 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-7[40306]: 2024-08-30T09:17:50.498+0000 7f33e554b640 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-30T09:17:21.644664+0000 front 2024-08-30T09:17:21.644755+0000 (oldest deadline 2024-08-30T09:17:47.544689+0000) 2024-08-30T09:17:50.883 INFO:journalctl@ceph.osd.6.smithi042.stdout:Aug 30 09:17:50 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-6[48371]: 2024-08-30T09:17:50.437+0000 7fcbc47c8640 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:07.508340+0000 front 2024-08-30T09:17:07.508372+0000 (oldest deadline 2024-08-30T09:17:29.808488+0000) 2024-08-30T09:17:50.883 INFO:journalctl@ceph.osd.6.smithi042.stdout:Aug 30 09:17:50 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-6[48371]: 2024-08-30T09:17:50.437+0000 7fcbc47c8640 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:15.610149+0000 front 2024-08-30T09:17:15.610331+0000 (oldest deadline 2024-08-30T09:17:39.110116+0000) 2024-08-30T09:17:50.883 INFO:journalctl@ceph.osd.6.smithi042.stdout:Aug 30 09:17:50 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-6[48371]: 2024-08-30T09:17:50.437+0000 7fcbc47c8640 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-30T09:17:25.011349+0000 front 2024-08-30T09:17:25.011311+0000 (oldest deadline 2024-08-30T09:17:47.311396+0000) 2024-08-30T09:17:50.883 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:17:50 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9[55148]: 2024-08-30T09:17:50.411+0000 7f7587f3d640 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:07.800304+0000 front 2024-08-30T09:17:07.800409+0000 (oldest deadline 2024-08-30T09:17:30.700443+0000) 2024-08-30T09:17:50.883 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:17:50 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9[55148]: 2024-08-30T09:17:50.411+0000 7f7587f3d640 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:16.601811+0000 front 2024-08-30T09:17:16.601614+0000 (oldest deadline 2024-08-30T09:17:41.901651+0000) 2024-08-30T09:17:50.884 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:17:50 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9[55148]: 2024-08-30T09:17:50.411+0000 7f7587f3d640 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-30T09:17:21.902428+0000 front 2024-08-30T09:17:21.902400+0000 (oldest deadline 2024-08-30T09:17:47.202527+0000) 2024-08-30T09:17:51.043 INFO:journalctl@ceph.osd.5.smithi190.stdout:Aug 30 09:17:50 smithi190 podman[58732]: 2024-08-30 09:17:50.824176155 +0000 UTC m=+6.193107571 container cleanup a3343420eb389be15325bab0fbc93ddca97f13c85d36b68a84d8b1b7a2241493 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-5, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.1, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611) 2024-08-30T09:17:51.044 INFO:journalctl@ceph.osd.5.smithi190.stdout:Aug 30 09:17:50 smithi190 bash[58732]: ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-5 2024-08-30T09:17:51.044 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:17:50 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:17:50.845+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:04.091032+0000 front 2024-08-30T09:17:04.090868+0000 (oldest deadline 2024-08-30T09:17:29.391066+0000) 2024-08-30T09:17:51.044 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:17:50 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:17:50.845+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:12.193073+0000 front 2024-08-30T09:17:12.192820+0000 (oldest deadline 2024-08-30T09:17:37.492761+0000) 2024-08-30T09:17:51.044 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:17:50 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:17:50.845+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-30T09:17:24.494221+0000 front 2024-08-30T09:17:24.494455+0000 (oldest deadline 2024-08-30T09:17:46.194420+0000) 2024-08-30T09:17:51.327 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:17:50 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:17:50.835+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:03.239730+0000 front 2024-08-30T09:17:03.239661+0000 (oldest deadline 2024-08-30T09:17:29.139608+0000) 2024-08-30T09:17:51.327 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:17:50 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:17:50.835+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:12.040451+0000 front 2024-08-30T09:17:12.040473+0000 (oldest deadline 2024-08-30T09:17:37.841115+0000) 2024-08-30T09:17:51.327 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:17:50 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:17:50.835+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-30T09:17:22.442437+0000 front 2024-08-30T09:17:22.442414+0000 (oldest deadline 2024-08-30T09:17:48.243149+0000) 2024-08-30T09:17:51.436 INFO:journalctl@ceph.osd.5.smithi190.stdout:Aug 30 09:17:51 smithi190 podman[58748]: 2024-08-30 09:17:51.042116234 +0000 UTC m=+0.636319284 container remove a3343420eb389be15325bab0fbc93ddca97f13c85d36b68a84d8b1b7a2241493 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-5, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.1, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-08-30T09:17:51.789 INFO:journalctl@ceph.osd.8.smithi190.stdout:Aug 30 09:17:51 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-8[40364]: 2024-08-30T09:17:51.646+0000 7fa492b20640 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:14.793297+0000 front 2024-08-30T09:17:14.793353+0000 (oldest deadline 2024-08-30T09:17:38.293306+0000) 2024-08-30T09:17:51.789 INFO:journalctl@ceph.osd.8.smithi190.stdout:Aug 30 09:17:51 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-8[40364]: 2024-08-30T09:17:51.646+0000 7fa492b20640 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-30T09:17:24.695219+0000 front 2024-08-30T09:17:24.695243+0000 (oldest deadline 2024-08-30T09:17:48.795417+0000) 2024-08-30T09:17:51.789 INFO:journalctl@ceph.osd.5.smithi190.stdout:Aug 30 09:17:51 smithi190 podman[58844]: 2024-08-30 09:17:51.411408372 +0000 UTC m=+0.019324039 image pull 79f4cde5e36911a5dca995eeb98098c2c8aae81dd52a088ef2c867bc3363bad0 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61 2024-08-30T09:17:51.789 INFO:journalctl@ceph.osd.5.smithi190.stdout:Aug 30 09:17:51 smithi190 podman[58844]: 2024-08-30 09:17:51.534840591 +0000 UTC m=+0.142756252 container create c264a2033c34f47f5fc94102e67e93fc5de9ded981e96570bf6bdc99385d1f5b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-5-deactivate, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, io.buildah.version=1.37.1) 2024-08-30T09:17:51.827 INFO:journalctl@ceph.osd.7.smithi167.stdout:Aug 30 09:17:51 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-7[40306]: 2024-08-30T09:17:51.523+0000 7f33e554b640 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:15.442207+0000 front 2024-08-30T09:17:15.442257+0000 (oldest deadline 2024-08-30T09:17:37.642922+0000) 2024-08-30T09:17:51.827 INFO:journalctl@ceph.osd.7.smithi167.stdout:Aug 30 09:17:51 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-7[40306]: 2024-08-30T09:17:51.523+0000 7f33e554b640 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-30T09:17:21.644664+0000 front 2024-08-30T09:17:21.644755+0000 (oldest deadline 2024-08-30T09:17:47.544689+0000) 2024-08-30T09:17:51.883 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:17:51 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9[55148]: 2024-08-30T09:17:51.439+0000 7f7587f3d640 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:07.800304+0000 front 2024-08-30T09:17:07.800409+0000 (oldest deadline 2024-08-30T09:17:30.700443+0000) 2024-08-30T09:17:51.883 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:17:51 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9[55148]: 2024-08-30T09:17:51.439+0000 7f7587f3d640 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:16.601811+0000 front 2024-08-30T09:17:16.601614+0000 (oldest deadline 2024-08-30T09:17:41.901651+0000) 2024-08-30T09:17:51.883 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:17:51 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9[55148]: 2024-08-30T09:17:51.439+0000 7f7587f3d640 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-30T09:17:21.902428+0000 front 2024-08-30T09:17:21.902400+0000 (oldest deadline 2024-08-30T09:17:47.202527+0000) 2024-08-30T09:17:51.883 INFO:journalctl@ceph.osd.6.smithi042.stdout:Aug 30 09:17:51 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-6[48371]: 2024-08-30T09:17:51.394+0000 7fcbc47c8640 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:07.508340+0000 front 2024-08-30T09:17:07.508372+0000 (oldest deadline 2024-08-30T09:17:29.808488+0000) 2024-08-30T09:17:51.883 INFO:journalctl@ceph.osd.6.smithi042.stdout:Aug 30 09:17:51 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-6[48371]: 2024-08-30T09:17:51.394+0000 7fcbc47c8640 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:15.610149+0000 front 2024-08-30T09:17:15.610331+0000 (oldest deadline 2024-08-30T09:17:39.110116+0000) 2024-08-30T09:17:51.883 INFO:journalctl@ceph.osd.6.smithi042.stdout:Aug 30 09:17:51 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-6[48371]: 2024-08-30T09:17:51.394+0000 7fcbc47c8640 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-30T09:17:25.011349+0000 front 2024-08-30T09:17:25.011311+0000 (oldest deadline 2024-08-30T09:17:47.311396+0000) 2024-08-30T09:17:52.186 INFO:journalctl@ceph.osd.5.smithi190.stdout:Aug 30 09:17:51 smithi190 podman[58844]: 2024-08-30 09:17:51.787414157 +0000 UTC m=+0.395329824 container init c264a2033c34f47f5fc94102e67e93fc5de9ded981e96570bf6bdc99385d1f5b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-5-deactivate, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.1, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True) 2024-08-30T09:17:52.187 INFO:journalctl@ceph.osd.5.smithi190.stdout:Aug 30 09:17:51 smithi190 podman[58844]: 2024-08-30 09:17:51.790758905 +0000 UTC m=+0.398674566 container start c264a2033c34f47f5fc94102e67e93fc5de9ded981e96570bf6bdc99385d1f5b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-5-deactivate, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.1, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-08-30T09:17:52.187 INFO:journalctl@ceph.osd.5.smithi190.stdout:Aug 30 09:17:51 smithi190 podman[58844]: 2024-08-30 09:17:51.903990703 +0000 UTC m=+0.511906372 container attach c264a2033c34f47f5fc94102e67e93fc5de9ded981e96570bf6bdc99385d1f5b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-5-deactivate, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.37.1) 2024-08-30T09:17:52.187 INFO:journalctl@ceph.osd.5.smithi190.stdout:Aug 30 09:17:51 smithi190 podman[58844]: 2024-08-30 09:17:51.978058838 +0000 UTC m=+0.585974499 container died c264a2033c34f47f5fc94102e67e93fc5de9ded981e96570bf6bdc99385d1f5b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-5-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.1, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2) 2024-08-30T09:17:52.187 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:17:51 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:17:51.809+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:04.091032+0000 front 2024-08-30T09:17:04.090868+0000 (oldest deadline 2024-08-30T09:17:29.391066+0000) 2024-08-30T09:17:52.187 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:17:51 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:17:51.809+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:12.193073+0000 front 2024-08-30T09:17:12.192820+0000 (oldest deadline 2024-08-30T09:17:37.492761+0000) 2024-08-30T09:17:52.187 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:17:51 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:17:51.809+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-30T09:17:24.494221+0000 front 2024-08-30T09:17:24.494455+0000 (oldest deadline 2024-08-30T09:17:46.194420+0000) 2024-08-30T09:17:52.327 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:17:51 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:17:51.843+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:03.239730+0000 front 2024-08-30T09:17:03.239661+0000 (oldest deadline 2024-08-30T09:17:29.139608+0000) 2024-08-30T09:17:52.327 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:17:51 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:17:51.843+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:12.040451+0000 front 2024-08-30T09:17:12.040473+0000 (oldest deadline 2024-08-30T09:17:37.841115+0000) 2024-08-30T09:17:52.327 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:17:51 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:17:51.843+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-30T09:17:22.442437+0000 front 2024-08-30T09:17:22.442414+0000 (oldest deadline 2024-08-30T09:17:48.243149+0000) 2024-08-30T09:17:52.699 DEBUG:teuthology.orchestra.run.smithi190:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968@osd.5.service' 2024-08-30T09:17:52.738 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-30T09:17:52.739 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-08-30T09:17:52.739 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-08-30T09:17:52.739 DEBUG:teuthology.orchestra.run.smithi042:> sudo systemctl stop ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968@osd.6 2024-08-30T09:17:52.766 INFO:journalctl@ceph.osd.6.smithi042.stdout:Aug 30 09:17:52 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-6[48371]: 2024-08-30T09:17:52.380+0000 7fcbc47c8640 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:07.508340+0000 front 2024-08-30T09:17:07.508372+0000 (oldest deadline 2024-08-30T09:17:29.808488+0000) 2024-08-30T09:17:52.766 INFO:journalctl@ceph.osd.6.smithi042.stdout:Aug 30 09:17:52 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-6[48371]: 2024-08-30T09:17:52.380+0000 7fcbc47c8640 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:15.610149+0000 front 2024-08-30T09:17:15.610331+0000 (oldest deadline 2024-08-30T09:17:39.110116+0000) 2024-08-30T09:17:52.766 INFO:journalctl@ceph.osd.6.smithi042.stdout:Aug 30 09:17:52 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-6[48371]: 2024-08-30T09:17:52.380+0000 7fcbc47c8640 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-30T09:17:25.011349+0000 front 2024-08-30T09:17:25.011311+0000 (oldest deadline 2024-08-30T09:17:47.311396+0000) 2024-08-30T09:17:52.767 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:17:52 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9[55148]: 2024-08-30T09:17:52.417+0000 7f7587f3d640 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:07.800304+0000 front 2024-08-30T09:17:07.800409+0000 (oldest deadline 2024-08-30T09:17:30.700443+0000) 2024-08-30T09:17:52.767 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:17:52 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9[55148]: 2024-08-30T09:17:52.417+0000 7f7587f3d640 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:16.601811+0000 front 2024-08-30T09:17:16.601614+0000 (oldest deadline 2024-08-30T09:17:41.901651+0000) 2024-08-30T09:17:52.767 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:17:52 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9[55148]: 2024-08-30T09:17:52.417+0000 7f7587f3d640 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-30T09:17:21.902428+0000 front 2024-08-30T09:17:21.902400+0000 (oldest deadline 2024-08-30T09:17:47.202527+0000) 2024-08-30T09:17:52.827 INFO:journalctl@ceph.osd.7.smithi167.stdout:Aug 30 09:17:52 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-7[40306]: 2024-08-30T09:17:52.547+0000 7f33e554b640 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:15.442207+0000 front 2024-08-30T09:17:15.442257+0000 (oldest deadline 2024-08-30T09:17:37.642922+0000) 2024-08-30T09:17:52.827 INFO:journalctl@ceph.osd.7.smithi167.stdout:Aug 30 09:17:52 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-7[40306]: 2024-08-30T09:17:52.547+0000 7f33e554b640 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-30T09:17:21.644664+0000 front 2024-08-30T09:17:21.644755+0000 (oldest deadline 2024-08-30T09:17:47.544689+0000) 2024-08-30T09:17:52.936 INFO:journalctl@ceph.osd.8.smithi190.stdout:Aug 30 09:17:52 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-8[40364]: 2024-08-30T09:17:52.644+0000 7fa492b20640 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:14.793297+0000 front 2024-08-30T09:17:14.793353+0000 (oldest deadline 2024-08-30T09:17:38.293306+0000) 2024-08-30T09:17:52.936 INFO:journalctl@ceph.osd.8.smithi190.stdout:Aug 30 09:17:52 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-8[40364]: 2024-08-30T09:17:52.644+0000 7fa492b20640 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-30T09:17:24.695219+0000 front 2024-08-30T09:17:24.695243+0000 (oldest deadline 2024-08-30T09:17:48.795417+0000) 2024-08-30T09:17:52.936 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:17:52 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:17:52.821+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:04.091032+0000 front 2024-08-30T09:17:04.090868+0000 (oldest deadline 2024-08-30T09:17:29.391066+0000) 2024-08-30T09:17:52.936 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:17:52 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:17:52.821+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:12.193073+0000 front 2024-08-30T09:17:12.192820+0000 (oldest deadline 2024-08-30T09:17:37.492761+0000) 2024-08-30T09:17:52.937 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:17:52 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:17:52.821+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-30T09:17:24.494221+0000 front 2024-08-30T09:17:24.494455+0000 (oldest deadline 2024-08-30T09:17:46.194420+0000) 2024-08-30T09:17:53.049 INFO:journalctl@ceph.osd.6.smithi042.stdout:Aug 30 09:17:52 smithi042 systemd[1]: Stopping Ceph osd.6 for b0895c1a-66ac-11ef-bcd4-c7b262605968... 2024-08-30T09:17:53.327 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:17:52 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:17:52.865+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:03.239730+0000 front 2024-08-30T09:17:03.239661+0000 (oldest deadline 2024-08-30T09:17:29.139608+0000) 2024-08-30T09:17:53.327 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:17:52 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:17:52.865+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:12.040451+0000 front 2024-08-30T09:17:12.040473+0000 (oldest deadline 2024-08-30T09:17:37.841115+0000) 2024-08-30T09:17:53.327 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:17:52 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:17:52.865+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-30T09:17:22.442437+0000 front 2024-08-30T09:17:22.442414+0000 (oldest deadline 2024-08-30T09:17:48.243149+0000) 2024-08-30T09:17:53.335 INFO:journalctl@ceph.osd.6.smithi042.stdout:Aug 30 09:17:53 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-6[48371]: 2024-08-30T09:17:53.046+0000 7fcbc81dd640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-30T09:17:53.335 INFO:journalctl@ceph.osd.6.smithi042.stdout:Aug 30 09:17:53 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-6[48371]: 2024-08-30T09:17:53.046+0000 7fcbc81dd640 -1 osd.6 87 *** Got signal Terminated *** 2024-08-30T09:17:53.335 INFO:journalctl@ceph.osd.6.smithi042.stdout:Aug 30 09:17:53 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-6[48371]: 2024-08-30T09:17:53.046+0000 7fcbc81dd640 -1 osd.6 87 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-08-30T09:17:53.335 INFO:journalctl@ceph.osd.6.smithi042.stdout:Aug 30 09:17:53 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-6[48371]: 2024-08-30T09:17:53.331+0000 7fcbc47c8640 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:07.508340+0000 front 2024-08-30T09:17:07.508372+0000 (oldest deadline 2024-08-30T09:17:29.808488+0000) 2024-08-30T09:17:53.335 INFO:journalctl@ceph.osd.6.smithi042.stdout:Aug 30 09:17:53 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-6[48371]: 2024-08-30T09:17:53.331+0000 7fcbc47c8640 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:15.610149+0000 front 2024-08-30T09:17:15.610331+0000 (oldest deadline 2024-08-30T09:17:39.110116+0000) 2024-08-30T09:17:53.633 INFO:journalctl@ceph.osd.6.smithi042.stdout:Aug 30 09:17:53 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-6[48371]: 2024-08-30T09:17:53.331+0000 7fcbc47c8640 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-30T09:17:25.011349+0000 front 2024-08-30T09:17:25.011311+0000 (oldest deadline 2024-08-30T09:17:47.311396+0000) 2024-08-30T09:17:53.633 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:17:53 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9[55148]: 2024-08-30T09:17:53.381+0000 7f7587f3d640 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:07.800304+0000 front 2024-08-30T09:17:07.800409+0000 (oldest deadline 2024-08-30T09:17:30.700443+0000) 2024-08-30T09:17:53.633 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:17:53 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9[55148]: 2024-08-30T09:17:53.381+0000 7f7587f3d640 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:16.601811+0000 front 2024-08-30T09:17:16.601614+0000 (oldest deadline 2024-08-30T09:17:41.901651+0000) 2024-08-30T09:17:53.633 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:17:53 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9[55148]: 2024-08-30T09:17:53.381+0000 7f7587f3d640 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-30T09:17:21.902428+0000 front 2024-08-30T09:17:21.902400+0000 (oldest deadline 2024-08-30T09:17:47.202527+0000) 2024-08-30T09:17:53.827 INFO:journalctl@ceph.osd.7.smithi167.stdout:Aug 30 09:17:53 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-7[40306]: 2024-08-30T09:17:53.551+0000 7f33e554b640 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:15.442207+0000 front 2024-08-30T09:17:15.442257+0000 (oldest deadline 2024-08-30T09:17:37.642922+0000) 2024-08-30T09:17:53.827 INFO:journalctl@ceph.osd.7.smithi167.stdout:Aug 30 09:17:53 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-7[40306]: 2024-08-30T09:17:53.551+0000 7f33e554b640 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-30T09:17:21.644664+0000 front 2024-08-30T09:17:21.644755+0000 (oldest deadline 2024-08-30T09:17:47.544689+0000) 2024-08-30T09:17:53.936 INFO:journalctl@ceph.osd.8.smithi190.stdout:Aug 30 09:17:53 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-8[40364]: 2024-08-30T09:17:53.679+0000 7fa492b20640 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:14.793297+0000 front 2024-08-30T09:17:14.793353+0000 (oldest deadline 2024-08-30T09:17:38.293306+0000) 2024-08-30T09:17:53.936 INFO:journalctl@ceph.osd.8.smithi190.stdout:Aug 30 09:17:53 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-8[40364]: 2024-08-30T09:17:53.679+0000 7fa492b20640 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-30T09:17:24.695219+0000 front 2024-08-30T09:17:24.695243+0000 (oldest deadline 2024-08-30T09:17:48.795417+0000) 2024-08-30T09:17:53.936 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:17:53 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:17:53.790+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:04.091032+0000 front 2024-08-30T09:17:04.090868+0000 (oldest deadline 2024-08-30T09:17:29.391066+0000) 2024-08-30T09:17:53.937 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:17:53 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:17:53.790+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:12.193073+0000 front 2024-08-30T09:17:12.192820+0000 (oldest deadline 2024-08-30T09:17:37.492761+0000) 2024-08-30T09:17:53.937 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:17:53 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:17:53.790+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-30T09:17:24.494221+0000 front 2024-08-30T09:17:24.494455+0000 (oldest deadline 2024-08-30T09:17:46.194420+0000) 2024-08-30T09:17:54.327 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:17:53 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:17:53.875+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:03.239730+0000 front 2024-08-30T09:17:03.239661+0000 (oldest deadline 2024-08-30T09:17:29.139608+0000) 2024-08-30T09:17:54.327 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:17:53 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:17:53.875+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:12.040451+0000 front 2024-08-30T09:17:12.040473+0000 (oldest deadline 2024-08-30T09:17:37.841115+0000) 2024-08-30T09:17:54.327 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:17:53 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:17:53.875+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-30T09:17:22.442437+0000 front 2024-08-30T09:17:22.442414+0000 (oldest deadline 2024-08-30T09:17:48.243149+0000) 2024-08-30T09:17:54.633 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:17:54 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9[55148]: 2024-08-30T09:17:54.361+0000 7f7587f3d640 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:07.800304+0000 front 2024-08-30T09:17:07.800409+0000 (oldest deadline 2024-08-30T09:17:30.700443+0000) 2024-08-30T09:17:54.633 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:17:54 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9[55148]: 2024-08-30T09:17:54.361+0000 7f7587f3d640 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:16.601811+0000 front 2024-08-30T09:17:16.601614+0000 (oldest deadline 2024-08-30T09:17:41.901651+0000) 2024-08-30T09:17:54.633 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:17:54 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9[55148]: 2024-08-30T09:17:54.361+0000 7f7587f3d640 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-30T09:17:21.902428+0000 front 2024-08-30T09:17:21.902400+0000 (oldest deadline 2024-08-30T09:17:47.202527+0000) 2024-08-30T09:17:54.634 INFO:journalctl@ceph.osd.6.smithi042.stdout:Aug 30 09:17:54 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-6[48371]: 2024-08-30T09:17:54.329+0000 7fcbc47c8640 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:07.508340+0000 front 2024-08-30T09:17:07.508372+0000 (oldest deadline 2024-08-30T09:17:29.808488+0000) 2024-08-30T09:17:54.634 INFO:journalctl@ceph.osd.6.smithi042.stdout:Aug 30 09:17:54 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-6[48371]: 2024-08-30T09:17:54.329+0000 7fcbc47c8640 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:15.610149+0000 front 2024-08-30T09:17:15.610331+0000 (oldest deadline 2024-08-30T09:17:39.110116+0000) 2024-08-30T09:17:54.634 INFO:journalctl@ceph.osd.6.smithi042.stdout:Aug 30 09:17:54 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-6[48371]: 2024-08-30T09:17:54.329+0000 7fcbc47c8640 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-30T09:17:25.011349+0000 front 2024-08-30T09:17:25.011311+0000 (oldest deadline 2024-08-30T09:17:47.311396+0000) 2024-08-30T09:17:54.827 INFO:journalctl@ceph.osd.7.smithi167.stdout:Aug 30 09:17:54 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-7[40306]: 2024-08-30T09:17:54.523+0000 7f33e554b640 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:15.442207+0000 front 2024-08-30T09:17:15.442257+0000 (oldest deadline 2024-08-30T09:17:37.642922+0000) 2024-08-30T09:17:54.827 INFO:journalctl@ceph.osd.7.smithi167.stdout:Aug 30 09:17:54 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-7[40306]: 2024-08-30T09:17:54.523+0000 7f33e554b640 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-30T09:17:21.644664+0000 front 2024-08-30T09:17:21.644755+0000 (oldest deadline 2024-08-30T09:17:47.544689+0000) 2024-08-30T09:17:55.186 INFO:journalctl@ceph.osd.8.smithi190.stdout:Aug 30 09:17:54 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-8[40364]: 2024-08-30T09:17:54.699+0000 7fa492b20640 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:14.793297+0000 front 2024-08-30T09:17:14.793353+0000 (oldest deadline 2024-08-30T09:17:38.293306+0000) 2024-08-30T09:17:55.186 INFO:journalctl@ceph.osd.8.smithi190.stdout:Aug 30 09:17:54 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-8[40364]: 2024-08-30T09:17:54.699+0000 7fa492b20640 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-30T09:17:24.695219+0000 front 2024-08-30T09:17:24.695243+0000 (oldest deadline 2024-08-30T09:17:48.795417+0000) 2024-08-30T09:17:55.186 INFO:journalctl@ceph.osd.8.smithi190.stdout:Aug 30 09:17:54 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-8[40364]: 2024-08-30T09:17:54.699+0000 7fa492b20640 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-08-30T09:17:28.796078+0000 front 2024-08-30T09:17:28.796109+0000 (oldest deadline 2024-08-30T09:17:54.696291+0000) 2024-08-30T09:17:55.187 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:17:54 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:17:54.831+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:04.091032+0000 front 2024-08-30T09:17:04.090868+0000 (oldest deadline 2024-08-30T09:17:29.391066+0000) 2024-08-30T09:17:55.187 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:17:54 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:17:54.831+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:12.193073+0000 front 2024-08-30T09:17:12.192820+0000 (oldest deadline 2024-08-30T09:17:37.492761+0000) 2024-08-30T09:17:55.187 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:17:54 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:17:54.831+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-30T09:17:24.494221+0000 front 2024-08-30T09:17:24.494455+0000 (oldest deadline 2024-08-30T09:17:46.194420+0000) 2024-08-30T09:17:55.327 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:17:54 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:17:54.918+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:03.239730+0000 front 2024-08-30T09:17:03.239661+0000 (oldest deadline 2024-08-30T09:17:29.139608+0000) 2024-08-30T09:17:55.327 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:17:54 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:17:54.918+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:12.040451+0000 front 2024-08-30T09:17:12.040473+0000 (oldest deadline 2024-08-30T09:17:37.841115+0000) 2024-08-30T09:17:55.327 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:17:54 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:17:54.918+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-30T09:17:22.442437+0000 front 2024-08-30T09:17:22.442414+0000 (oldest deadline 2024-08-30T09:17:48.243149+0000) 2024-08-30T09:17:55.633 INFO:journalctl@ceph.osd.6.smithi042.stdout:Aug 30 09:17:55 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-6[48371]: 2024-08-30T09:17:55.349+0000 7fcbc47c8640 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:07.508340+0000 front 2024-08-30T09:17:07.508372+0000 (oldest deadline 2024-08-30T09:17:29.808488+0000) 2024-08-30T09:17:55.633 INFO:journalctl@ceph.osd.6.smithi042.stdout:Aug 30 09:17:55 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-6[48371]: 2024-08-30T09:17:55.349+0000 7fcbc47c8640 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:15.610149+0000 front 2024-08-30T09:17:15.610331+0000 (oldest deadline 2024-08-30T09:17:39.110116+0000) 2024-08-30T09:17:55.633 INFO:journalctl@ceph.osd.6.smithi042.stdout:Aug 30 09:17:55 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-6[48371]: 2024-08-30T09:17:55.349+0000 7fcbc47c8640 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-30T09:17:25.011349+0000 front 2024-08-30T09:17:25.011311+0000 (oldest deadline 2024-08-30T09:17:47.311396+0000) 2024-08-30T09:17:55.633 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:17:55 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9[55148]: 2024-08-30T09:17:55.366+0000 7f7587f3d640 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:07.800304+0000 front 2024-08-30T09:17:07.800409+0000 (oldest deadline 2024-08-30T09:17:30.700443+0000) 2024-08-30T09:17:55.634 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:17:55 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9[55148]: 2024-08-30T09:17:55.366+0000 7f7587f3d640 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:16.601811+0000 front 2024-08-30T09:17:16.601614+0000 (oldest deadline 2024-08-30T09:17:41.901651+0000) 2024-08-30T09:17:55.634 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:17:55 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9[55148]: 2024-08-30T09:17:55.366+0000 7f7587f3d640 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-30T09:17:21.902428+0000 front 2024-08-30T09:17:21.902400+0000 (oldest deadline 2024-08-30T09:17:47.202527+0000) 2024-08-30T09:17:55.827 INFO:journalctl@ceph.osd.7.smithi167.stdout:Aug 30 09:17:55 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-7[40306]: 2024-08-30T09:17:55.533+0000 7f33e554b640 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:15.442207+0000 front 2024-08-30T09:17:15.442257+0000 (oldest deadline 2024-08-30T09:17:37.642922+0000) 2024-08-30T09:17:55.827 INFO:journalctl@ceph.osd.7.smithi167.stdout:Aug 30 09:17:55 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-7[40306]: 2024-08-30T09:17:55.533+0000 7f33e554b640 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-30T09:17:21.644664+0000 front 2024-08-30T09:17:21.644755+0000 (oldest deadline 2024-08-30T09:17:47.544689+0000) 2024-08-30T09:17:56.186 INFO:journalctl@ceph.osd.8.smithi190.stdout:Aug 30 09:17:55 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-8[40364]: 2024-08-30T09:17:55.699+0000 7fa492b20640 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:14.793297+0000 front 2024-08-30T09:17:14.793353+0000 (oldest deadline 2024-08-30T09:17:38.293306+0000) 2024-08-30T09:17:56.186 INFO:journalctl@ceph.osd.8.smithi190.stdout:Aug 30 09:17:55 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-8[40364]: 2024-08-30T09:17:55.699+0000 7fa492b20640 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-30T09:17:24.695219+0000 front 2024-08-30T09:17:24.695243+0000 (oldest deadline 2024-08-30T09:17:48.795417+0000) 2024-08-30T09:17:56.187 INFO:journalctl@ceph.osd.8.smithi190.stdout:Aug 30 09:17:55 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-8[40364]: 2024-08-30T09:17:55.699+0000 7fa492b20640 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-08-30T09:17:28.796078+0000 front 2024-08-30T09:17:28.796109+0000 (oldest deadline 2024-08-30T09:17:54.696291+0000) 2024-08-30T09:17:56.187 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:17:55 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:17:55.825+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:04.091032+0000 front 2024-08-30T09:17:04.090868+0000 (oldest deadline 2024-08-30T09:17:29.391066+0000) 2024-08-30T09:17:56.187 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:17:55 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:17:55.825+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:12.193073+0000 front 2024-08-30T09:17:12.192820+0000 (oldest deadline 2024-08-30T09:17:37.492761+0000) 2024-08-30T09:17:56.187 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:17:55 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:17:55.825+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-30T09:17:24.494221+0000 front 2024-08-30T09:17:24.494455+0000 (oldest deadline 2024-08-30T09:17:46.194420+0000) 2024-08-30T09:17:56.327 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:17:55 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:17:55.888+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:03.239730+0000 front 2024-08-30T09:17:03.239661+0000 (oldest deadline 2024-08-30T09:17:29.139608+0000) 2024-08-30T09:17:56.327 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:17:55 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:17:55.888+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:12.040451+0000 front 2024-08-30T09:17:12.040473+0000 (oldest deadline 2024-08-30T09:17:37.841115+0000) 2024-08-30T09:17:56.327 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:17:55 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:17:55.888+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-30T09:17:22.442437+0000 front 2024-08-30T09:17:22.442414+0000 (oldest deadline 2024-08-30T09:17:48.243149+0000) 2024-08-30T09:17:56.327 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:17:55 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:17:55.888+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-08-30T09:17:32.944470+0000 front 2024-08-30T09:17:32.944528+0000 (oldest deadline 2024-08-30T09:17:55.244278+0000) 2024-08-30T09:17:56.827 INFO:journalctl@ceph.osd.7.smithi167.stdout:Aug 30 09:17:56 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-7[40306]: 2024-08-30T09:17:56.563+0000 7f33e554b640 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:15.442207+0000 front 2024-08-30T09:17:15.442257+0000 (oldest deadline 2024-08-30T09:17:37.642922+0000) 2024-08-30T09:17:56.827 INFO:journalctl@ceph.osd.7.smithi167.stdout:Aug 30 09:17:56 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-7[40306]: 2024-08-30T09:17:56.563+0000 7f33e554b640 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-30T09:17:21.644664+0000 front 2024-08-30T09:17:21.644755+0000 (oldest deadline 2024-08-30T09:17:47.544689+0000) 2024-08-30T09:17:56.883 INFO:journalctl@ceph.osd.6.smithi042.stdout:Aug 30 09:17:56 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-6[48371]: 2024-08-30T09:17:56.397+0000 7fcbc47c8640 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:07.508340+0000 front 2024-08-30T09:17:07.508372+0000 (oldest deadline 2024-08-30T09:17:29.808488+0000) 2024-08-30T09:17:56.883 INFO:journalctl@ceph.osd.6.smithi042.stdout:Aug 30 09:17:56 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-6[48371]: 2024-08-30T09:17:56.397+0000 7fcbc47c8640 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:15.610149+0000 front 2024-08-30T09:17:15.610331+0000 (oldest deadline 2024-08-30T09:17:39.110116+0000) 2024-08-30T09:17:56.883 INFO:journalctl@ceph.osd.6.smithi042.stdout:Aug 30 09:17:56 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-6[48371]: 2024-08-30T09:17:56.397+0000 7fcbc47c8640 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-30T09:17:25.011349+0000 front 2024-08-30T09:17:25.011311+0000 (oldest deadline 2024-08-30T09:17:47.311396+0000) 2024-08-30T09:17:56.883 INFO:journalctl@ceph.osd.6.smithi042.stdout:Aug 30 09:17:56 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-6[48371]: 2024-08-30T09:17:56.397+0000 7fcbc47c8640 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-08-30T09:17:31.313370+0000 front 2024-08-30T09:17:31.313342+0000 (oldest deadline 2024-08-30T09:17:56.013584+0000) 2024-08-30T09:17:56.883 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:17:56 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9[55148]: 2024-08-30T09:17:56.413+0000 7f7587f3d640 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:07.800304+0000 front 2024-08-30T09:17:07.800409+0000 (oldest deadline 2024-08-30T09:17:30.700443+0000) 2024-08-30T09:17:56.883 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:17:56 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9[55148]: 2024-08-30T09:17:56.413+0000 7f7587f3d640 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:16.601811+0000 front 2024-08-30T09:17:16.601614+0000 (oldest deadline 2024-08-30T09:17:41.901651+0000) 2024-08-30T09:17:56.884 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:17:56 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9[55148]: 2024-08-30T09:17:56.413+0000 7f7587f3d640 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-30T09:17:21.902428+0000 front 2024-08-30T09:17:21.902400+0000 (oldest deadline 2024-08-30T09:17:47.202527+0000) 2024-08-30T09:17:56.884 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:17:56 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9[55148]: 2024-08-30T09:17:56.413+0000 7f7587f3d640 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-08-30T09:17:31.704449+0000 front 2024-08-30T09:17:31.704477+0000 (oldest deadline 2024-08-30T09:17:56.404636+0000) 2024-08-30T09:17:56.936 INFO:journalctl@ceph.osd.8.smithi190.stdout:Aug 30 09:17:56 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-8[40364]: 2024-08-30T09:17:56.674+0000 7fa492b20640 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:14.793297+0000 front 2024-08-30T09:17:14.793353+0000 (oldest deadline 2024-08-30T09:17:38.293306+0000) 2024-08-30T09:17:56.936 INFO:journalctl@ceph.osd.8.smithi190.stdout:Aug 30 09:17:56 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-8[40364]: 2024-08-30T09:17:56.674+0000 7fa492b20640 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-30T09:17:24.695219+0000 front 2024-08-30T09:17:24.695243+0000 (oldest deadline 2024-08-30T09:17:48.795417+0000) 2024-08-30T09:17:56.936 INFO:journalctl@ceph.osd.8.smithi190.stdout:Aug 30 09:17:56 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-8[40364]: 2024-08-30T09:17:56.674+0000 7fa492b20640 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-08-30T09:17:28.796078+0000 front 2024-08-30T09:17:28.796109+0000 (oldest deadline 2024-08-30T09:17:54.696291+0000) 2024-08-30T09:17:56.936 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:17:56 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:17:56.823+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:04.091032+0000 front 2024-08-30T09:17:04.090868+0000 (oldest deadline 2024-08-30T09:17:29.391066+0000) 2024-08-30T09:17:56.937 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:17:56 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:17:56.823+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:12.193073+0000 front 2024-08-30T09:17:12.192820+0000 (oldest deadline 2024-08-30T09:17:37.492761+0000) 2024-08-30T09:17:56.937 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:17:56 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:17:56.823+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-30T09:17:24.494221+0000 front 2024-08-30T09:17:24.494455+0000 (oldest deadline 2024-08-30T09:17:46.194420+0000) 2024-08-30T09:17:57.327 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:17:56 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:17:56.877+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:03.239730+0000 front 2024-08-30T09:17:03.239661+0000 (oldest deadline 2024-08-30T09:17:29.139608+0000) 2024-08-30T09:17:57.327 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:17:56 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:17:56.877+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:12.040451+0000 front 2024-08-30T09:17:12.040473+0000 (oldest deadline 2024-08-30T09:17:37.841115+0000) 2024-08-30T09:17:57.327 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:17:56 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:17:56.877+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-30T09:17:22.442437+0000 front 2024-08-30T09:17:22.442414+0000 (oldest deadline 2024-08-30T09:17:48.243149+0000) 2024-08-30T09:17:57.327 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:17:56 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:17:56.877+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-08-30T09:17:32.944470+0000 front 2024-08-30T09:17:32.944528+0000 (oldest deadline 2024-08-30T09:17:55.244278+0000) 2024-08-30T09:17:57.633 INFO:journalctl@ceph.osd.6.smithi042.stdout:Aug 30 09:17:57 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-6[48371]: 2024-08-30T09:17:57.350+0000 7fcbc47c8640 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:07.508340+0000 front 2024-08-30T09:17:07.508372+0000 (oldest deadline 2024-08-30T09:17:29.808488+0000) 2024-08-30T09:17:57.633 INFO:journalctl@ceph.osd.6.smithi042.stdout:Aug 30 09:17:57 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-6[48371]: 2024-08-30T09:17:57.350+0000 7fcbc47c8640 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:15.610149+0000 front 2024-08-30T09:17:15.610331+0000 (oldest deadline 2024-08-30T09:17:39.110116+0000) 2024-08-30T09:17:57.633 INFO:journalctl@ceph.osd.6.smithi042.stdout:Aug 30 09:17:57 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-6[48371]: 2024-08-30T09:17:57.350+0000 7fcbc47c8640 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-30T09:17:25.011349+0000 front 2024-08-30T09:17:25.011311+0000 (oldest deadline 2024-08-30T09:17:47.311396+0000) 2024-08-30T09:17:57.633 INFO:journalctl@ceph.osd.6.smithi042.stdout:Aug 30 09:17:57 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-6[48371]: 2024-08-30T09:17:57.350+0000 7fcbc47c8640 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-08-30T09:17:31.313370+0000 front 2024-08-30T09:17:31.313342+0000 (oldest deadline 2024-08-30T09:17:56.013584+0000) 2024-08-30T09:17:57.633 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:17:57 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9[55148]: 2024-08-30T09:17:57.382+0000 7f7587f3d640 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:07.800304+0000 front 2024-08-30T09:17:07.800409+0000 (oldest deadline 2024-08-30T09:17:30.700443+0000) 2024-08-30T09:17:57.633 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:17:57 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9[55148]: 2024-08-30T09:17:57.382+0000 7f7587f3d640 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:16.601811+0000 front 2024-08-30T09:17:16.601614+0000 (oldest deadline 2024-08-30T09:17:41.901651+0000) 2024-08-30T09:17:57.634 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:17:57 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9[55148]: 2024-08-30T09:17:57.382+0000 7f7587f3d640 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-30T09:17:21.902428+0000 front 2024-08-30T09:17:21.902400+0000 (oldest deadline 2024-08-30T09:17:47.202527+0000) 2024-08-30T09:17:57.634 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:17:57 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9[55148]: 2024-08-30T09:17:57.382+0000 7f7587f3d640 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-08-30T09:17:31.704449+0000 front 2024-08-30T09:17:31.704477+0000 (oldest deadline 2024-08-30T09:17:56.404636+0000) 2024-08-30T09:17:57.898 INFO:journalctl@ceph.osd.7.smithi167.stdout:Aug 30 09:17:57 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-7[40306]: 2024-08-30T09:17:57.611+0000 7f33e554b640 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:15.442207+0000 front 2024-08-30T09:17:15.442257+0000 (oldest deadline 2024-08-30T09:17:37.642922+0000) 2024-08-30T09:17:57.898 INFO:journalctl@ceph.osd.7.smithi167.stdout:Aug 30 09:17:57 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-7[40306]: 2024-08-30T09:17:57.611+0000 7f33e554b640 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-30T09:17:21.644664+0000 front 2024-08-30T09:17:21.644755+0000 (oldest deadline 2024-08-30T09:17:47.544689+0000) 2024-08-30T09:17:57.898 INFO:journalctl@ceph.osd.7.smithi167.stdout:Aug 30 09:17:57 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-7[40306]: 2024-08-30T09:17:57.611+0000 7f33e554b640 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-08-30T09:17:32.547612+0000 front 2024-08-30T09:17:32.547709+0000 (oldest deadline 2024-08-30T09:17:56.647661+0000) 2024-08-30T09:17:57.936 INFO:journalctl@ceph.osd.8.smithi190.stdout:Aug 30 09:17:57 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-8[40364]: 2024-08-30T09:17:57.677+0000 7fa492b20640 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:14.793297+0000 front 2024-08-30T09:17:14.793353+0000 (oldest deadline 2024-08-30T09:17:38.293306+0000) 2024-08-30T09:17:57.936 INFO:journalctl@ceph.osd.8.smithi190.stdout:Aug 30 09:17:57 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-8[40364]: 2024-08-30T09:17:57.677+0000 7fa492b20640 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-30T09:17:24.695219+0000 front 2024-08-30T09:17:24.695243+0000 (oldest deadline 2024-08-30T09:17:48.795417+0000) 2024-08-30T09:17:57.936 INFO:journalctl@ceph.osd.8.smithi190.stdout:Aug 30 09:17:57 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-8[40364]: 2024-08-30T09:17:57.677+0000 7fa492b20640 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-08-30T09:17:28.796078+0000 front 2024-08-30T09:17:28.796109+0000 (oldest deadline 2024-08-30T09:17:54.696291+0000) 2024-08-30T09:17:57.936 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:17:57 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:17:57.836+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:04.091032+0000 front 2024-08-30T09:17:04.090868+0000 (oldest deadline 2024-08-30T09:17:29.391066+0000) 2024-08-30T09:17:57.937 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:17:57 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:17:57.836+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:12.193073+0000 front 2024-08-30T09:17:12.192820+0000 (oldest deadline 2024-08-30T09:17:37.492761+0000) 2024-08-30T09:17:57.937 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:17:57 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:17:57.836+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-30T09:17:24.494221+0000 front 2024-08-30T09:17:24.494455+0000 (oldest deadline 2024-08-30T09:17:46.194420+0000) 2024-08-30T09:17:58.327 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:17:57 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:17:57.896+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:03.239730+0000 front 2024-08-30T09:17:03.239661+0000 (oldest deadline 2024-08-30T09:17:29.139608+0000) 2024-08-30T09:17:58.327 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:17:57 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:17:57.896+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:12.040451+0000 front 2024-08-30T09:17:12.040473+0000 (oldest deadline 2024-08-30T09:17:37.841115+0000) 2024-08-30T09:17:58.327 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:17:57 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:17:57.896+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-30T09:17:22.442437+0000 front 2024-08-30T09:17:22.442414+0000 (oldest deadline 2024-08-30T09:17:48.243149+0000) 2024-08-30T09:17:58.327 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:17:57 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:17:57.896+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-08-30T09:17:32.944470+0000 front 2024-08-30T09:17:32.944528+0000 (oldest deadline 2024-08-30T09:17:55.244278+0000) 2024-08-30T09:17:58.666 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:17:58 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9[55148]: 2024-08-30T09:17:58.412+0000 7f7587f3d640 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:07.800304+0000 front 2024-08-30T09:17:07.800409+0000 (oldest deadline 2024-08-30T09:17:30.700443+0000) 2024-08-30T09:17:58.667 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:17:58 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9[55148]: 2024-08-30T09:17:58.412+0000 7f7587f3d640 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:16.601811+0000 front 2024-08-30T09:17:16.601614+0000 (oldest deadline 2024-08-30T09:17:41.901651+0000) 2024-08-30T09:17:58.667 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:17:58 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9[55148]: 2024-08-30T09:17:58.412+0000 7f7587f3d640 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-30T09:17:21.902428+0000 front 2024-08-30T09:17:21.902400+0000 (oldest deadline 2024-08-30T09:17:47.202527+0000) 2024-08-30T09:17:58.667 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:17:58 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9[55148]: 2024-08-30T09:17:58.412+0000 7f7587f3d640 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-08-30T09:17:31.704449+0000 front 2024-08-30T09:17:31.704477+0000 (oldest deadline 2024-08-30T09:17:56.404636+0000) 2024-08-30T09:17:58.910 INFO:journalctl@ceph.osd.7.smithi167.stdout:Aug 30 09:17:58 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-7[40306]: 2024-08-30T09:17:58.641+0000 7f33e554b640 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:15.442207+0000 front 2024-08-30T09:17:15.442257+0000 (oldest deadline 2024-08-30T09:17:37.642922+0000) 2024-08-30T09:17:58.910 INFO:journalctl@ceph.osd.7.smithi167.stdout:Aug 30 09:17:58 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-7[40306]: 2024-08-30T09:17:58.641+0000 7f33e554b640 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-30T09:17:21.644664+0000 front 2024-08-30T09:17:21.644755+0000 (oldest deadline 2024-08-30T09:17:47.544689+0000) 2024-08-30T09:17:58.910 INFO:journalctl@ceph.osd.7.smithi167.stdout:Aug 30 09:17:58 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-7[40306]: 2024-08-30T09:17:58.641+0000 7f33e554b640 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-08-30T09:17:32.547612+0000 front 2024-08-30T09:17:32.547709+0000 (oldest deadline 2024-08-30T09:17:56.647661+0000) 2024-08-30T09:17:58.936 INFO:journalctl@ceph.osd.8.smithi190.stdout:Aug 30 09:17:58 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-8[40364]: 2024-08-30T09:17:58.678+0000 7fa492b20640 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:14.793297+0000 front 2024-08-30T09:17:14.793353+0000 (oldest deadline 2024-08-30T09:17:38.293306+0000) 2024-08-30T09:17:58.936 INFO:journalctl@ceph.osd.8.smithi190.stdout:Aug 30 09:17:58 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-8[40364]: 2024-08-30T09:17:58.678+0000 7fa492b20640 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-30T09:17:24.695219+0000 front 2024-08-30T09:17:24.695243+0000 (oldest deadline 2024-08-30T09:17:48.795417+0000) 2024-08-30T09:17:58.936 INFO:journalctl@ceph.osd.8.smithi190.stdout:Aug 30 09:17:58 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-8[40364]: 2024-08-30T09:17:58.678+0000 7fa492b20640 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-08-30T09:17:28.796078+0000 front 2024-08-30T09:17:28.796109+0000 (oldest deadline 2024-08-30T09:17:54.696291+0000) 2024-08-30T09:17:58.937 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:17:58 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:17:58.802+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:04.091032+0000 front 2024-08-30T09:17:04.090868+0000 (oldest deadline 2024-08-30T09:17:29.391066+0000) 2024-08-30T09:17:58.937 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:17:58 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:17:58.802+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:12.193073+0000 front 2024-08-30T09:17:12.192820+0000 (oldest deadline 2024-08-30T09:17:37.492761+0000) 2024-08-30T09:17:58.937 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:17:58 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:17:58.802+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-30T09:17:24.494221+0000 front 2024-08-30T09:17:24.494455+0000 (oldest deadline 2024-08-30T09:17:46.194420+0000) 2024-08-30T09:17:58.958 INFO:journalctl@ceph.osd.6.smithi042.stdout:Aug 30 09:17:58 smithi042 podman[101113]: 2024-08-30 09:17:58.664792104 +0000 UTC m=+5.726313644 container died 838c2fa5133aed5a17a4a846f37d6a83c943f16824a632410027e6d4902e8463 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-6, io.buildah.version=1.37.1, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-08-30T09:17:59.327 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:17:58 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:17:58.908+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:03.239730+0000 front 2024-08-30T09:17:03.239661+0000 (oldest deadline 2024-08-30T09:17:29.139608+0000) 2024-08-30T09:17:59.327 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:17:58 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:17:58.908+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:12.040451+0000 front 2024-08-30T09:17:12.040473+0000 (oldest deadline 2024-08-30T09:17:37.841115+0000) 2024-08-30T09:17:59.327 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:17:58 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:17:58.908+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-30T09:17:22.442437+0000 front 2024-08-30T09:17:22.442414+0000 (oldest deadline 2024-08-30T09:17:48.243149+0000) 2024-08-30T09:17:59.327 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:17:58 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:17:58.908+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-08-30T09:17:32.944470+0000 front 2024-08-30T09:17:32.944528+0000 (oldest deadline 2024-08-30T09:17:55.244278+0000) 2024-08-30T09:17:59.375 INFO:journalctl@ceph.osd.6.smithi042.stdout:Aug 30 09:17:59 smithi042 podman[101113]: 2024-08-30 09:17:59.139739528 +0000 UTC m=+6.201261067 container cleanup 838c2fa5133aed5a17a4a846f37d6a83c943f16824a632410027e6d4902e8463 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-6, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.37.1, org.label-schema.schema-version=1.0) 2024-08-30T09:17:59.375 INFO:journalctl@ceph.osd.6.smithi042.stdout:Aug 30 09:17:59 smithi042 bash[101113]: ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-6 2024-08-30T09:17:59.633 INFO:journalctl@ceph.osd.6.smithi042.stdout:Aug 30 09:17:59 smithi042 podman[101139]: 2024-08-30 09:17:59.40354926 +0000 UTC m=+0.736081346 container remove 838c2fa5133aed5a17a4a846f37d6a83c943f16824a632410027e6d4902e8463 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-6, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.build-date=20240731, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.1, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0) 2024-08-30T09:17:59.633 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:17:59 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9[55148]: 2024-08-30T09:17:59.372+0000 7f7587f3d640 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:07.800304+0000 front 2024-08-30T09:17:07.800409+0000 (oldest deadline 2024-08-30T09:17:30.700443+0000) 2024-08-30T09:17:59.633 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:17:59 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9[55148]: 2024-08-30T09:17:59.372+0000 7f7587f3d640 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:16.601811+0000 front 2024-08-30T09:17:16.601614+0000 (oldest deadline 2024-08-30T09:17:41.901651+0000) 2024-08-30T09:17:59.633 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:17:59 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9[55148]: 2024-08-30T09:17:59.372+0000 7f7587f3d640 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-30T09:17:21.902428+0000 front 2024-08-30T09:17:21.902400+0000 (oldest deadline 2024-08-30T09:17:47.202527+0000) 2024-08-30T09:17:59.633 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:17:59 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9[55148]: 2024-08-30T09:17:59.372+0000 7f7587f3d640 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-08-30T09:17:31.704449+0000 front 2024-08-30T09:17:31.704477+0000 (oldest deadline 2024-08-30T09:17:56.404636+0000) 2024-08-30T09:18:00.077 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:17:59 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:17:59.919+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:03.239730+0000 front 2024-08-30T09:17:03.239661+0000 (oldest deadline 2024-08-30T09:17:29.139608+0000) 2024-08-30T09:18:00.077 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:17:59 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:17:59.919+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:12.040451+0000 front 2024-08-30T09:17:12.040473+0000 (oldest deadline 2024-08-30T09:17:37.841115+0000) 2024-08-30T09:18:00.077 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:17:59 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:17:59.919+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-30T09:17:22.442437+0000 front 2024-08-30T09:17:22.442414+0000 (oldest deadline 2024-08-30T09:17:48.243149+0000) 2024-08-30T09:18:00.077 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:17:59 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:17:59.919+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-08-30T09:17:32.944470+0000 front 2024-08-30T09:17:32.944528+0000 (oldest deadline 2024-08-30T09:17:55.244278+0000) 2024-08-30T09:18:00.077 INFO:journalctl@ceph.osd.7.smithi167.stdout:Aug 30 09:17:59 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-7[40306]: 2024-08-30T09:17:59.677+0000 7f33e554b640 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:15.442207+0000 front 2024-08-30T09:17:15.442257+0000 (oldest deadline 2024-08-30T09:17:37.642922+0000) 2024-08-30T09:18:00.077 INFO:journalctl@ceph.osd.7.smithi167.stdout:Aug 30 09:17:59 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-7[40306]: 2024-08-30T09:17:59.677+0000 7f33e554b640 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-30T09:17:21.644664+0000 front 2024-08-30T09:17:21.644755+0000 (oldest deadline 2024-08-30T09:17:47.544689+0000) 2024-08-30T09:18:00.077 INFO:journalctl@ceph.osd.7.smithi167.stdout:Aug 30 09:17:59 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-7[40306]: 2024-08-30T09:17:59.677+0000 7f33e554b640 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-08-30T09:17:32.547612+0000 front 2024-08-30T09:17:32.547709+0000 (oldest deadline 2024-08-30T09:17:56.647661+0000) 2024-08-30T09:18:00.129 INFO:journalctl@ceph.osd.6.smithi042.stdout:Aug 30 09:17:59 smithi042 podman[101228]: 2024-08-30 09:17:59.61893052 +0000 UTC m=+0.015997513 image pull 79f4cde5e36911a5dca995eeb98098c2c8aae81dd52a088ef2c867bc3363bad0 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61 2024-08-30T09:18:00.129 INFO:journalctl@ceph.osd.6.smithi042.stdout:Aug 30 09:17:59 smithi042 podman[101228]: 2024-08-30 09:17:59.725536808 +0000 UTC m=+0.122603787 container create d5fc9d52035912415e4586f1c01722cf19156eb37caa7368b3e971907ba65922 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-6-deactivate, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.37.1, CEPH_POINT_RELEASE=, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-08-30T09:18:00.129 INFO:journalctl@ceph.osd.6.smithi042.stdout:Aug 30 09:17:59 smithi042 podman[101228]: 2024-08-30 09:17:59.936443918 +0000 UTC m=+0.333510897 container init d5fc9d52035912415e4586f1c01722cf19156eb37caa7368b3e971907ba65922 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-6-deactivate, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.1, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611) 2024-08-30T09:18:00.129 INFO:journalctl@ceph.osd.6.smithi042.stdout:Aug 30 09:17:59 smithi042 podman[101228]: 2024-08-30 09:17:59.939666365 +0000 UTC m=+0.336733356 container start d5fc9d52035912415e4586f1c01722cf19156eb37caa7368b3e971907ba65922 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-6-deactivate, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.1, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, maintainer=Guillaume Abrioux ) 2024-08-30T09:18:00.186 INFO:journalctl@ceph.osd.8.smithi190.stdout:Aug 30 09:17:59 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-8[40364]: 2024-08-30T09:17:59.692+0000 7fa492b20640 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:14.793297+0000 front 2024-08-30T09:17:14.793353+0000 (oldest deadline 2024-08-30T09:17:38.293306+0000) 2024-08-30T09:18:00.186 INFO:journalctl@ceph.osd.8.smithi190.stdout:Aug 30 09:17:59 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-8[40364]: 2024-08-30T09:17:59.692+0000 7fa492b20640 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-30T09:17:24.695219+0000 front 2024-08-30T09:17:24.695243+0000 (oldest deadline 2024-08-30T09:17:48.795417+0000) 2024-08-30T09:18:00.186 INFO:journalctl@ceph.osd.8.smithi190.stdout:Aug 30 09:17:59 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-8[40364]: 2024-08-30T09:17:59.692+0000 7fa492b20640 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-08-30T09:17:28.796078+0000 front 2024-08-30T09:17:28.796109+0000 (oldest deadline 2024-08-30T09:17:54.696291+0000) 2024-08-30T09:18:00.186 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:17:59 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:17:59.792+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:04.091032+0000 front 2024-08-30T09:17:04.090868+0000 (oldest deadline 2024-08-30T09:17:29.391066+0000) 2024-08-30T09:18:00.186 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:17:59 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:17:59.792+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:12.193073+0000 front 2024-08-30T09:17:12.192820+0000 (oldest deadline 2024-08-30T09:17:37.492761+0000) 2024-08-30T09:18:00.186 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:17:59 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:17:59.792+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-30T09:17:24.494221+0000 front 2024-08-30T09:17:24.494455+0000 (oldest deadline 2024-08-30T09:17:46.194420+0000) 2024-08-30T09:18:00.383 INFO:journalctl@ceph.osd.6.smithi042.stdout:Aug 30 09:18:00 smithi042 podman[101228]: 2024-08-30 09:18:00.136272615 +0000 UTC m=+0.533339609 container attach d5fc9d52035912415e4586f1c01722cf19156eb37caa7368b3e971907ba65922 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-6-deactivate, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.1, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-08-30T09:18:00.383 INFO:journalctl@ceph.osd.6.smithi042.stdout:Aug 30 09:18:00 smithi042 podman[101228]: 2024-08-30 09:18:00.136931461 +0000 UTC m=+0.533998446 container died d5fc9d52035912415e4586f1c01722cf19156eb37caa7368b3e971907ba65922 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-6-deactivate, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.37.1, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image) 2024-08-30T09:18:00.800 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:18:00 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9[55148]: 2024-08-30T09:18:00.394+0000 7f7587f3d640 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:07.800304+0000 front 2024-08-30T09:17:07.800409+0000 (oldest deadline 2024-08-30T09:17:30.700443+0000) 2024-08-30T09:18:00.800 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:18:00 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9[55148]: 2024-08-30T09:18:00.394+0000 7f7587f3d640 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:16.601811+0000 front 2024-08-30T09:17:16.601614+0000 (oldest deadline 2024-08-30T09:17:41.901651+0000) 2024-08-30T09:18:00.800 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:18:00 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9[55148]: 2024-08-30T09:18:00.394+0000 7f7587f3d640 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-30T09:17:21.902428+0000 front 2024-08-30T09:17:21.902400+0000 (oldest deadline 2024-08-30T09:17:47.202527+0000) 2024-08-30T09:18:00.800 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:18:00 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9[55148]: 2024-08-30T09:18:00.394+0000 7f7587f3d640 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-08-30T09:17:31.704449+0000 front 2024-08-30T09:17:31.704477+0000 (oldest deadline 2024-08-30T09:17:56.404636+0000) 2024-08-30T09:18:00.830 DEBUG:teuthology.orchestra.run.smithi042:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968@osd.6.service' 2024-08-30T09:18:00.915 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-30T09:18:00.915 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-08-30T09:18:00.915 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-08-30T09:18:00.916 DEBUG:teuthology.orchestra.run.smithi167:> sudo systemctl stop ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968@osd.7 2024-08-30T09:18:01.077 INFO:journalctl@ceph.osd.7.smithi167.stdout:Aug 30 09:18:00 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-7[40306]: 2024-08-30T09:18:00.727+0000 7f33e554b640 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:15.442207+0000 front 2024-08-30T09:17:15.442257+0000 (oldest deadline 2024-08-30T09:17:37.642922+0000) 2024-08-30T09:18:01.077 INFO:journalctl@ceph.osd.7.smithi167.stdout:Aug 30 09:18:00 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-7[40306]: 2024-08-30T09:18:00.727+0000 7f33e554b640 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-30T09:17:21.644664+0000 front 2024-08-30T09:17:21.644755+0000 (oldest deadline 2024-08-30T09:17:47.544689+0000) 2024-08-30T09:18:01.077 INFO:journalctl@ceph.osd.7.smithi167.stdout:Aug 30 09:18:00 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-7[40306]: 2024-08-30T09:18:00.727+0000 7f33e554b640 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-08-30T09:17:32.547612+0000 front 2024-08-30T09:17:32.547709+0000 (oldest deadline 2024-08-30T09:17:56.647661+0000) 2024-08-30T09:18:01.077 INFO:journalctl@ceph.osd.7.smithi167.stdout:Aug 30 09:18:00 smithi167 systemd[1]: Stopping Ceph osd.7 for b0895c1a-66ac-11ef-bcd4-c7b262605968... 2024-08-30T09:18:01.078 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:18:00 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:18:00.957+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:03.239730+0000 front 2024-08-30T09:17:03.239661+0000 (oldest deadline 2024-08-30T09:17:29.139608+0000) 2024-08-30T09:18:01.078 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:18:00 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:18:00.957+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:12.040451+0000 front 2024-08-30T09:17:12.040473+0000 (oldest deadline 2024-08-30T09:17:37.841115+0000) 2024-08-30T09:18:01.078 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:18:00 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:18:00.957+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-30T09:17:22.442437+0000 front 2024-08-30T09:17:22.442414+0000 (oldest deadline 2024-08-30T09:17:48.243149+0000) 2024-08-30T09:18:01.078 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:18:00 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:18:00.957+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-08-30T09:17:32.944470+0000 front 2024-08-30T09:17:32.944528+0000 (oldest deadline 2024-08-30T09:17:55.244278+0000) 2024-08-30T09:18:01.186 INFO:journalctl@ceph.osd.8.smithi190.stdout:Aug 30 09:18:00 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-8[40364]: 2024-08-30T09:18:00.742+0000 7fa492b20640 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:14.793297+0000 front 2024-08-30T09:17:14.793353+0000 (oldest deadline 2024-08-30T09:17:38.293306+0000) 2024-08-30T09:18:01.186 INFO:journalctl@ceph.osd.8.smithi190.stdout:Aug 30 09:18:00 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-8[40364]: 2024-08-30T09:18:00.742+0000 7fa492b20640 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-30T09:17:24.695219+0000 front 2024-08-30T09:17:24.695243+0000 (oldest deadline 2024-08-30T09:17:48.795417+0000) 2024-08-30T09:18:01.186 INFO:journalctl@ceph.osd.8.smithi190.stdout:Aug 30 09:18:00 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-8[40364]: 2024-08-30T09:18:00.742+0000 7fa492b20640 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-08-30T09:17:28.796078+0000 front 2024-08-30T09:17:28.796109+0000 (oldest deadline 2024-08-30T09:17:54.696291+0000) 2024-08-30T09:18:01.186 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:00 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:00.793+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:04.091032+0000 front 2024-08-30T09:17:04.090868+0000 (oldest deadline 2024-08-30T09:17:29.391066+0000) 2024-08-30T09:18:01.187 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:00 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:00.793+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:12.193073+0000 front 2024-08-30T09:17:12.192820+0000 (oldest deadline 2024-08-30T09:17:37.492761+0000) 2024-08-30T09:18:01.187 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:00 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:00.793+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-30T09:17:24.494221+0000 front 2024-08-30T09:17:24.494455+0000 (oldest deadline 2024-08-30T09:17:46.194420+0000) 2024-08-30T09:18:01.577 INFO:journalctl@ceph.osd.7.smithi167.stdout:Aug 30 09:18:01 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-7[40306]: 2024-08-30T09:18:01.247+0000 7f33e8f60640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-30T09:18:01.577 INFO:journalctl@ceph.osd.7.smithi167.stdout:Aug 30 09:18:01 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-7[40306]: 2024-08-30T09:18:01.247+0000 7f33e8f60640 -1 osd.7 87 *** Got signal Terminated *** 2024-08-30T09:18:01.577 INFO:journalctl@ceph.osd.7.smithi167.stdout:Aug 30 09:18:01 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-7[40306]: 2024-08-30T09:18:01.247+0000 7f33e8f60640 -1 osd.7 87 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-08-30T09:18:01.633 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:18:01 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9[55148]: 2024-08-30T09:18:01.353+0000 7f7587f3d640 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:07.800304+0000 front 2024-08-30T09:17:07.800409+0000 (oldest deadline 2024-08-30T09:17:30.700443+0000) 2024-08-30T09:18:01.633 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:18:01 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9[55148]: 2024-08-30T09:18:01.353+0000 7f7587f3d640 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:16.601811+0000 front 2024-08-30T09:17:16.601614+0000 (oldest deadline 2024-08-30T09:17:41.901651+0000) 2024-08-30T09:18:01.633 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:18:01 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9[55148]: 2024-08-30T09:18:01.353+0000 7f7587f3d640 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-30T09:17:21.902428+0000 front 2024-08-30T09:17:21.902400+0000 (oldest deadline 2024-08-30T09:17:47.202527+0000) 2024-08-30T09:18:01.633 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:18:01 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9[55148]: 2024-08-30T09:18:01.353+0000 7f7587f3d640 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-08-30T09:17:31.704449+0000 front 2024-08-30T09:17:31.704477+0000 (oldest deadline 2024-08-30T09:17:56.404636+0000) 2024-08-30T09:18:01.936 INFO:journalctl@ceph.osd.8.smithi190.stdout:Aug 30 09:18:01 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-8[40364]: 2024-08-30T09:18:01.706+0000 7fa492b20640 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:14.793297+0000 front 2024-08-30T09:17:14.793353+0000 (oldest deadline 2024-08-30T09:17:38.293306+0000) 2024-08-30T09:18:01.936 INFO:journalctl@ceph.osd.8.smithi190.stdout:Aug 30 09:18:01 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-8[40364]: 2024-08-30T09:18:01.706+0000 7fa492b20640 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-30T09:17:24.695219+0000 front 2024-08-30T09:17:24.695243+0000 (oldest deadline 2024-08-30T09:17:48.795417+0000) 2024-08-30T09:18:01.936 INFO:journalctl@ceph.osd.8.smithi190.stdout:Aug 30 09:18:01 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-8[40364]: 2024-08-30T09:18:01.706+0000 7fa492b20640 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-08-30T09:17:28.796078+0000 front 2024-08-30T09:17:28.796109+0000 (oldest deadline 2024-08-30T09:17:54.696291+0000) 2024-08-30T09:18:01.937 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:01 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:01.766+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:04.091032+0000 front 2024-08-30T09:17:04.090868+0000 (oldest deadline 2024-08-30T09:17:29.391066+0000) 2024-08-30T09:18:01.937 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:01 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:01.766+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:12.193073+0000 front 2024-08-30T09:17:12.192820+0000 (oldest deadline 2024-08-30T09:17:37.492761+0000) 2024-08-30T09:18:01.937 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:01 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:01.766+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-30T09:17:24.494221+0000 front 2024-08-30T09:17:24.494455+0000 (oldest deadline 2024-08-30T09:17:46.194420+0000) 2024-08-30T09:18:02.077 INFO:journalctl@ceph.osd.7.smithi167.stdout:Aug 30 09:18:01 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-7[40306]: 2024-08-30T09:18:01.733+0000 7f33e554b640 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:15.442207+0000 front 2024-08-30T09:17:15.442257+0000 (oldest deadline 2024-08-30T09:17:37.642922+0000) 2024-08-30T09:18:02.077 INFO:journalctl@ceph.osd.7.smithi167.stdout:Aug 30 09:18:01 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-7[40306]: 2024-08-30T09:18:01.733+0000 7f33e554b640 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-30T09:17:21.644664+0000 front 2024-08-30T09:17:21.644755+0000 (oldest deadline 2024-08-30T09:17:47.544689+0000) 2024-08-30T09:18:02.077 INFO:journalctl@ceph.osd.7.smithi167.stdout:Aug 30 09:18:01 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-7[40306]: 2024-08-30T09:18:01.733+0000 7f33e554b640 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-08-30T09:17:32.547612+0000 front 2024-08-30T09:17:32.547709+0000 (oldest deadline 2024-08-30T09:17:56.647661+0000) 2024-08-30T09:18:02.077 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:18:01 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:18:01.973+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:03.239730+0000 front 2024-08-30T09:17:03.239661+0000 (oldest deadline 2024-08-30T09:17:29.139608+0000) 2024-08-30T09:18:02.077 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:18:01 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:18:01.973+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:12.040451+0000 front 2024-08-30T09:17:12.040473+0000 (oldest deadline 2024-08-30T09:17:37.841115+0000) 2024-08-30T09:18:02.078 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:18:01 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:18:01.973+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-30T09:17:22.442437+0000 front 2024-08-30T09:17:22.442414+0000 (oldest deadline 2024-08-30T09:17:48.243149+0000) 2024-08-30T09:18:02.078 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:18:01 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:18:01.973+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-08-30T09:17:32.944470+0000 front 2024-08-30T09:17:32.944528+0000 (oldest deadline 2024-08-30T09:17:55.244278+0000) 2024-08-30T09:18:02.633 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:18:02 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9[55148]: 2024-08-30T09:18:02.379+0000 7f7587f3d640 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:07.800304+0000 front 2024-08-30T09:17:07.800409+0000 (oldest deadline 2024-08-30T09:17:30.700443+0000) 2024-08-30T09:18:02.633 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:18:02 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9[55148]: 2024-08-30T09:18:02.379+0000 7f7587f3d640 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:16.601811+0000 front 2024-08-30T09:17:16.601614+0000 (oldest deadline 2024-08-30T09:17:41.901651+0000) 2024-08-30T09:18:02.633 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:18:02 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9[55148]: 2024-08-30T09:18:02.379+0000 7f7587f3d640 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-30T09:17:21.902428+0000 front 2024-08-30T09:17:21.902400+0000 (oldest deadline 2024-08-30T09:17:47.202527+0000) 2024-08-30T09:18:02.633 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:18:02 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9[55148]: 2024-08-30T09:18:02.379+0000 7f7587f3d640 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-08-30T09:17:31.704449+0000 front 2024-08-30T09:17:31.704477+0000 (oldest deadline 2024-08-30T09:17:56.404636+0000) 2024-08-30T09:18:03.010 INFO:journalctl@ceph.osd.7.smithi167.stdout:Aug 30 09:18:02 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-7[40306]: 2024-08-30T09:18:02.728+0000 7f33e554b640 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:15.442207+0000 front 2024-08-30T09:17:15.442257+0000 (oldest deadline 2024-08-30T09:17:37.642922+0000) 2024-08-30T09:18:03.010 INFO:journalctl@ceph.osd.7.smithi167.stdout:Aug 30 09:18:02 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-7[40306]: 2024-08-30T09:18:02.728+0000 7f33e554b640 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-30T09:17:21.644664+0000 front 2024-08-30T09:17:21.644755+0000 (oldest deadline 2024-08-30T09:17:47.544689+0000) 2024-08-30T09:18:03.010 INFO:journalctl@ceph.osd.7.smithi167.stdout:Aug 30 09:18:02 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-7[40306]: 2024-08-30T09:18:02.728+0000 7f33e554b640 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-08-30T09:17:32.547612+0000 front 2024-08-30T09:17:32.547709+0000 (oldest deadline 2024-08-30T09:17:56.647661+0000) 2024-08-30T09:18:03.186 INFO:journalctl@ceph.osd.8.smithi190.stdout:Aug 30 09:18:02 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-8[40364]: 2024-08-30T09:18:02.709+0000 7fa492b20640 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:14.793297+0000 front 2024-08-30T09:17:14.793353+0000 (oldest deadline 2024-08-30T09:17:38.293306+0000) 2024-08-30T09:18:03.186 INFO:journalctl@ceph.osd.8.smithi190.stdout:Aug 30 09:18:02 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-8[40364]: 2024-08-30T09:18:02.709+0000 7fa492b20640 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-30T09:17:24.695219+0000 front 2024-08-30T09:17:24.695243+0000 (oldest deadline 2024-08-30T09:17:48.795417+0000) 2024-08-30T09:18:03.186 INFO:journalctl@ceph.osd.8.smithi190.stdout:Aug 30 09:18:02 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-8[40364]: 2024-08-30T09:18:02.709+0000 7fa492b20640 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-08-30T09:17:28.796078+0000 front 2024-08-30T09:17:28.796109+0000 (oldest deadline 2024-08-30T09:17:54.696291+0000) 2024-08-30T09:18:03.186 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:02 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:02.805+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:04.091032+0000 front 2024-08-30T09:17:04.090868+0000 (oldest deadline 2024-08-30T09:17:29.391066+0000) 2024-08-30T09:18:03.187 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:02 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:02.805+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:12.193073+0000 front 2024-08-30T09:17:12.192820+0000 (oldest deadline 2024-08-30T09:17:37.492761+0000) 2024-08-30T09:18:03.187 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:02 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:02.805+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-30T09:17:24.494221+0000 front 2024-08-30T09:17:24.494455+0000 (oldest deadline 2024-08-30T09:17:46.194420+0000) 2024-08-30T09:18:03.327 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:18:03 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:18:03.008+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:03.239730+0000 front 2024-08-30T09:17:03.239661+0000 (oldest deadline 2024-08-30T09:17:29.139608+0000) 2024-08-30T09:18:03.327 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:18:03 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:18:03.008+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:12.040451+0000 front 2024-08-30T09:17:12.040473+0000 (oldest deadline 2024-08-30T09:17:37.841115+0000) 2024-08-30T09:18:03.327 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:18:03 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:18:03.008+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-30T09:17:22.442437+0000 front 2024-08-30T09:17:22.442414+0000 (oldest deadline 2024-08-30T09:17:48.243149+0000) 2024-08-30T09:18:03.328 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:18:03 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:18:03.008+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-08-30T09:17:32.944470+0000 front 2024-08-30T09:17:32.944528+0000 (oldest deadline 2024-08-30T09:17:55.244278+0000) 2024-08-30T09:18:03.633 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:18:03 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9[55148]: 2024-08-30T09:18:03.346+0000 7f7587f3d640 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:07.800304+0000 front 2024-08-30T09:17:07.800409+0000 (oldest deadline 2024-08-30T09:17:30.700443+0000) 2024-08-30T09:18:03.633 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:18:03 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9[55148]: 2024-08-30T09:18:03.346+0000 7f7587f3d640 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:16.601811+0000 front 2024-08-30T09:17:16.601614+0000 (oldest deadline 2024-08-30T09:17:41.901651+0000) 2024-08-30T09:18:03.633 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:18:03 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9[55148]: 2024-08-30T09:18:03.346+0000 7f7587f3d640 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-30T09:17:21.902428+0000 front 2024-08-30T09:17:21.902400+0000 (oldest deadline 2024-08-30T09:17:47.202527+0000) 2024-08-30T09:18:03.633 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:18:03 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9[55148]: 2024-08-30T09:18:03.346+0000 7f7587f3d640 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-08-30T09:17:31.704449+0000 front 2024-08-30T09:17:31.704477+0000 (oldest deadline 2024-08-30T09:17:56.404636+0000) 2024-08-30T09:18:04.039 INFO:journalctl@ceph.osd.7.smithi167.stdout:Aug 30 09:18:03 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-7[40306]: 2024-08-30T09:18:03.704+0000 7f33e554b640 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:15.442207+0000 front 2024-08-30T09:17:15.442257+0000 (oldest deadline 2024-08-30T09:17:37.642922+0000) 2024-08-30T09:18:04.039 INFO:journalctl@ceph.osd.7.smithi167.stdout:Aug 30 09:18:03 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-7[40306]: 2024-08-30T09:18:03.704+0000 7f33e554b640 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-30T09:17:21.644664+0000 front 2024-08-30T09:17:21.644755+0000 (oldest deadline 2024-08-30T09:17:47.544689+0000) 2024-08-30T09:18:04.039 INFO:journalctl@ceph.osd.7.smithi167.stdout:Aug 30 09:18:03 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-7[40306]: 2024-08-30T09:18:03.704+0000 7f33e554b640 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-08-30T09:17:32.547612+0000 front 2024-08-30T09:17:32.547709+0000 (oldest deadline 2024-08-30T09:17:56.647661+0000) 2024-08-30T09:18:04.186 INFO:journalctl@ceph.osd.8.smithi190.stdout:Aug 30 09:18:03 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-8[40364]: 2024-08-30T09:18:03.689+0000 7fa492b20640 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:14.793297+0000 front 2024-08-30T09:17:14.793353+0000 (oldest deadline 2024-08-30T09:17:38.293306+0000) 2024-08-30T09:18:04.186 INFO:journalctl@ceph.osd.8.smithi190.stdout:Aug 30 09:18:03 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-8[40364]: 2024-08-30T09:18:03.689+0000 7fa492b20640 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-30T09:17:24.695219+0000 front 2024-08-30T09:17:24.695243+0000 (oldest deadline 2024-08-30T09:17:48.795417+0000) 2024-08-30T09:18:04.186 INFO:journalctl@ceph.osd.8.smithi190.stdout:Aug 30 09:18:03 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-8[40364]: 2024-08-30T09:18:03.689+0000 7fa492b20640 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-08-30T09:17:28.796078+0000 front 2024-08-30T09:17:28.796109+0000 (oldest deadline 2024-08-30T09:17:54.696291+0000) 2024-08-30T09:18:04.186 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:03 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:03.788+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:04.091032+0000 front 2024-08-30T09:17:04.090868+0000 (oldest deadline 2024-08-30T09:17:29.391066+0000) 2024-08-30T09:18:04.186 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:03 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:03.788+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:12.193073+0000 front 2024-08-30T09:17:12.192820+0000 (oldest deadline 2024-08-30T09:17:37.492761+0000) 2024-08-30T09:18:04.186 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:03 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:03.788+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-30T09:17:24.494221+0000 front 2024-08-30T09:17:24.494455+0000 (oldest deadline 2024-08-30T09:17:46.194420+0000) 2024-08-30T09:18:04.327 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:18:04 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:18:04.037+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:03.239730+0000 front 2024-08-30T09:17:03.239661+0000 (oldest deadline 2024-08-30T09:17:29.139608+0000) 2024-08-30T09:18:04.327 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:18:04 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:18:04.037+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:12.040451+0000 front 2024-08-30T09:17:12.040473+0000 (oldest deadline 2024-08-30T09:17:37.841115+0000) 2024-08-30T09:18:04.327 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:18:04 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:18:04.037+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-30T09:17:22.442437+0000 front 2024-08-30T09:17:22.442414+0000 (oldest deadline 2024-08-30T09:17:48.243149+0000) 2024-08-30T09:18:04.327 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:18:04 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:18:04.037+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-08-30T09:17:32.944470+0000 front 2024-08-30T09:17:32.944528+0000 (oldest deadline 2024-08-30T09:17:55.244278+0000) 2024-08-30T09:18:04.327 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:18:04 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:18:04.037+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.167:6812 osd.4 since back 2024-08-30T09:17:38.546318+0000 front 2024-08-30T09:17:38.546353+0000 (oldest deadline 2024-08-30T09:18:03.246447+0000) 2024-08-30T09:18:04.633 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:18:04 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9[55148]: 2024-08-30T09:18:04.372+0000 7f7587f3d640 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:07.800304+0000 front 2024-08-30T09:17:07.800409+0000 (oldest deadline 2024-08-30T09:17:30.700443+0000) 2024-08-30T09:18:04.633 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:18:04 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9[55148]: 2024-08-30T09:18:04.372+0000 7f7587f3d640 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:16.601811+0000 front 2024-08-30T09:17:16.601614+0000 (oldest deadline 2024-08-30T09:17:41.901651+0000) 2024-08-30T09:18:04.633 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:18:04 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9[55148]: 2024-08-30T09:18:04.372+0000 7f7587f3d640 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-30T09:17:21.902428+0000 front 2024-08-30T09:17:21.902400+0000 (oldest deadline 2024-08-30T09:17:47.202527+0000) 2024-08-30T09:18:04.633 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:18:04 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9[55148]: 2024-08-30T09:18:04.372+0000 7f7587f3d640 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-08-30T09:17:31.704449+0000 front 2024-08-30T09:17:31.704477+0000 (oldest deadline 2024-08-30T09:17:56.404636+0000) 2024-08-30T09:18:05.045 INFO:journalctl@ceph.osd.7.smithi167.stdout:Aug 30 09:18:04 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-7[40306]: 2024-08-30T09:18:04.667+0000 7f33e554b640 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:15.442207+0000 front 2024-08-30T09:17:15.442257+0000 (oldest deadline 2024-08-30T09:17:37.642922+0000) 2024-08-30T09:18:05.046 INFO:journalctl@ceph.osd.7.smithi167.stdout:Aug 30 09:18:04 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-7[40306]: 2024-08-30T09:18:04.667+0000 7f33e554b640 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-30T09:17:21.644664+0000 front 2024-08-30T09:17:21.644755+0000 (oldest deadline 2024-08-30T09:17:47.544689+0000) 2024-08-30T09:18:05.046 INFO:journalctl@ceph.osd.7.smithi167.stdout:Aug 30 09:18:04 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-7[40306]: 2024-08-30T09:18:04.667+0000 7f33e554b640 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-08-30T09:17:32.547612+0000 front 2024-08-30T09:17:32.547709+0000 (oldest deadline 2024-08-30T09:17:56.647661+0000) 2024-08-30T09:18:05.186 INFO:journalctl@ceph.osd.8.smithi190.stdout:Aug 30 09:18:04 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-8[40364]: 2024-08-30T09:18:04.702+0000 7fa492b20640 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:14.793297+0000 front 2024-08-30T09:17:14.793353+0000 (oldest deadline 2024-08-30T09:17:38.293306+0000) 2024-08-30T09:18:05.186 INFO:journalctl@ceph.osd.8.smithi190.stdout:Aug 30 09:18:04 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-8[40364]: 2024-08-30T09:18:04.702+0000 7fa492b20640 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-30T09:17:24.695219+0000 front 2024-08-30T09:17:24.695243+0000 (oldest deadline 2024-08-30T09:17:48.795417+0000) 2024-08-30T09:18:05.186 INFO:journalctl@ceph.osd.8.smithi190.stdout:Aug 30 09:18:04 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-8[40364]: 2024-08-30T09:18:04.702+0000 7fa492b20640 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-08-30T09:17:28.796078+0000 front 2024-08-30T09:17:28.796109+0000 (oldest deadline 2024-08-30T09:17:54.696291+0000) 2024-08-30T09:18:05.187 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:04 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:04.754+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:04.091032+0000 front 2024-08-30T09:17:04.090868+0000 (oldest deadline 2024-08-30T09:17:29.391066+0000) 2024-08-30T09:18:05.187 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:04 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:04.754+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:12.193073+0000 front 2024-08-30T09:17:12.192820+0000 (oldest deadline 2024-08-30T09:17:37.492761+0000) 2024-08-30T09:18:05.187 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:04 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:04.754+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-30T09:17:24.494221+0000 front 2024-08-30T09:17:24.494455+0000 (oldest deadline 2024-08-30T09:17:46.194420+0000) 2024-08-30T09:18:05.187 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:04 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:04.754+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.167:6812 osd.4 since back 2024-08-30T09:17:38.897956+0000 front 2024-08-30T09:17:38.897901+0000 (oldest deadline 2024-08-30T09:18:04.698743+0000) 2024-08-30T09:18:05.327 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:18:05 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:18:05.043+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:03.239730+0000 front 2024-08-30T09:17:03.239661+0000 (oldest deadline 2024-08-30T09:17:29.139608+0000) 2024-08-30T09:18:05.327 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:18:05 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:18:05.043+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:12.040451+0000 front 2024-08-30T09:17:12.040473+0000 (oldest deadline 2024-08-30T09:17:37.841115+0000) 2024-08-30T09:18:05.327 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:18:05 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:18:05.043+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-30T09:17:22.442437+0000 front 2024-08-30T09:17:22.442414+0000 (oldest deadline 2024-08-30T09:17:48.243149+0000) 2024-08-30T09:18:05.327 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:18:05 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:18:05.043+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-08-30T09:17:32.944470+0000 front 2024-08-30T09:17:32.944528+0000 (oldest deadline 2024-08-30T09:17:55.244278+0000) 2024-08-30T09:18:05.327 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:18:05 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:18:05.043+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.167:6812 osd.4 since back 2024-08-30T09:17:38.546318+0000 front 2024-08-30T09:17:38.546353+0000 (oldest deadline 2024-08-30T09:18:03.246447+0000) 2024-08-30T09:18:05.633 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:18:05 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9[55148]: 2024-08-30T09:18:05.376+0000 7f7587f3d640 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:07.800304+0000 front 2024-08-30T09:17:07.800409+0000 (oldest deadline 2024-08-30T09:17:30.700443+0000) 2024-08-30T09:18:05.633 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:18:05 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9[55148]: 2024-08-30T09:18:05.376+0000 7f7587f3d640 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:16.601811+0000 front 2024-08-30T09:17:16.601614+0000 (oldest deadline 2024-08-30T09:17:41.901651+0000) 2024-08-30T09:18:05.633 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:18:05 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9[55148]: 2024-08-30T09:18:05.376+0000 7f7587f3d640 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-30T09:17:21.902428+0000 front 2024-08-30T09:17:21.902400+0000 (oldest deadline 2024-08-30T09:17:47.202527+0000) 2024-08-30T09:18:05.633 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:18:05 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9[55148]: 2024-08-30T09:18:05.376+0000 7f7587f3d640 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-08-30T09:17:31.704449+0000 front 2024-08-30T09:17:31.704477+0000 (oldest deadline 2024-08-30T09:17:56.404636+0000) 2024-08-30T09:18:05.936 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:05 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:05.742+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:04.091032+0000 front 2024-08-30T09:17:04.090868+0000 (oldest deadline 2024-08-30T09:17:29.391066+0000) 2024-08-30T09:18:05.936 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:05 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:05.742+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:12.193073+0000 front 2024-08-30T09:17:12.192820+0000 (oldest deadline 2024-08-30T09:17:37.492761+0000) 2024-08-30T09:18:05.936 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:05 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:05.742+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-30T09:17:24.494221+0000 front 2024-08-30T09:17:24.494455+0000 (oldest deadline 2024-08-30T09:17:46.194420+0000) 2024-08-30T09:18:05.936 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:05 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:05.742+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.167:6812 osd.4 since back 2024-08-30T09:17:38.897956+0000 front 2024-08-30T09:17:38.897901+0000 (oldest deadline 2024-08-30T09:18:04.698743+0000) 2024-08-30T09:18:05.937 INFO:journalctl@ceph.osd.8.smithi190.stdout:Aug 30 09:18:05 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-8[40364]: 2024-08-30T09:18:05.655+0000 7fa492b20640 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:14.793297+0000 front 2024-08-30T09:17:14.793353+0000 (oldest deadline 2024-08-30T09:17:38.293306+0000) 2024-08-30T09:18:05.937 INFO:journalctl@ceph.osd.8.smithi190.stdout:Aug 30 09:18:05 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-8[40364]: 2024-08-30T09:18:05.655+0000 7fa492b20640 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-30T09:17:24.695219+0000 front 2024-08-30T09:17:24.695243+0000 (oldest deadline 2024-08-30T09:17:48.795417+0000) 2024-08-30T09:18:05.937 INFO:journalctl@ceph.osd.8.smithi190.stdout:Aug 30 09:18:05 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-8[40364]: 2024-08-30T09:18:05.655+0000 7fa492b20640 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-08-30T09:17:28.796078+0000 front 2024-08-30T09:17:28.796109+0000 (oldest deadline 2024-08-30T09:17:54.696291+0000) 2024-08-30T09:18:05.937 INFO:journalctl@ceph.osd.8.smithi190.stdout:Aug 30 09:18:05 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-8[40364]: 2024-08-30T09:18:05.655+0000 7fa492b20640 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.167:6812 osd.4 since back 2024-08-30T09:17:38.797502+0000 front 2024-08-30T09:17:38.797429+0000 (oldest deadline 2024-08-30T09:18:05.198235+0000) 2024-08-30T09:18:06.014 INFO:journalctl@ceph.osd.7.smithi167.stdout:Aug 30 09:18:05 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-7[40306]: 2024-08-30T09:18:05.643+0000 7f33e554b640 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:15.442207+0000 front 2024-08-30T09:17:15.442257+0000 (oldest deadline 2024-08-30T09:17:37.642922+0000) 2024-08-30T09:18:06.014 INFO:journalctl@ceph.osd.7.smithi167.stdout:Aug 30 09:18:05 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-7[40306]: 2024-08-30T09:18:05.643+0000 7f33e554b640 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-30T09:17:21.644664+0000 front 2024-08-30T09:17:21.644755+0000 (oldest deadline 2024-08-30T09:17:47.544689+0000) 2024-08-30T09:18:06.015 INFO:journalctl@ceph.osd.7.smithi167.stdout:Aug 30 09:18:05 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-7[40306]: 2024-08-30T09:18:05.643+0000 7f33e554b640 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-08-30T09:17:32.547612+0000 front 2024-08-30T09:17:32.547709+0000 (oldest deadline 2024-08-30T09:17:56.647661+0000) 2024-08-30T09:18:06.327 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:18:06 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:18:06.012+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:03.239730+0000 front 2024-08-30T09:17:03.239661+0000 (oldest deadline 2024-08-30T09:17:29.139608+0000) 2024-08-30T09:18:06.327 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:18:06 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:18:06.012+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:12.040451+0000 front 2024-08-30T09:17:12.040473+0000 (oldest deadline 2024-08-30T09:17:37.841115+0000) 2024-08-30T09:18:06.327 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:18:06 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:18:06.012+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-30T09:17:22.442437+0000 front 2024-08-30T09:17:22.442414+0000 (oldest deadline 2024-08-30T09:17:48.243149+0000) 2024-08-30T09:18:06.327 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:18:06 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:18:06.012+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-08-30T09:17:32.944470+0000 front 2024-08-30T09:17:32.944528+0000 (oldest deadline 2024-08-30T09:17:55.244278+0000) 2024-08-30T09:18:06.327 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:18:06 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:18:06.012+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.167:6812 osd.4 since back 2024-08-30T09:17:38.546318+0000 front 2024-08-30T09:17:38.546353+0000 (oldest deadline 2024-08-30T09:18:03.246447+0000) 2024-08-30T09:18:06.633 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:18:06 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9[55148]: 2024-08-30T09:18:06.371+0000 7f7587f3d640 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:07.800304+0000 front 2024-08-30T09:17:07.800409+0000 (oldest deadline 2024-08-30T09:17:30.700443+0000) 2024-08-30T09:18:06.633 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:18:06 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9[55148]: 2024-08-30T09:18:06.371+0000 7f7587f3d640 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:16.601811+0000 front 2024-08-30T09:17:16.601614+0000 (oldest deadline 2024-08-30T09:17:41.901651+0000) 2024-08-30T09:18:06.633 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:18:06 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9[55148]: 2024-08-30T09:18:06.371+0000 7f7587f3d640 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-30T09:17:21.902428+0000 front 2024-08-30T09:17:21.902400+0000 (oldest deadline 2024-08-30T09:17:47.202527+0000) 2024-08-30T09:18:06.633 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:18:06 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9[55148]: 2024-08-30T09:18:06.371+0000 7f7587f3d640 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-08-30T09:17:31.704449+0000 front 2024-08-30T09:17:31.704477+0000 (oldest deadline 2024-08-30T09:17:56.404636+0000) 2024-08-30T09:18:06.633 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:18:06 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9[55148]: 2024-08-30T09:18:06.371+0000 7f7587f3d640 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.167:6812 osd.4 since back 2024-08-30T09:17:41.105874+0000 front 2024-08-30T09:17:41.106081+0000 (oldest deadline 2024-08-30T09:18:05.706638+0000) 2024-08-30T09:18:06.936 INFO:journalctl@ceph.osd.8.smithi190.stdout:Aug 30 09:18:06 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-8[40364]: 2024-08-30T09:18:06.634+0000 7fa492b20640 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:14.793297+0000 front 2024-08-30T09:17:14.793353+0000 (oldest deadline 2024-08-30T09:17:38.293306+0000) 2024-08-30T09:18:06.936 INFO:journalctl@ceph.osd.8.smithi190.stdout:Aug 30 09:18:06 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-8[40364]: 2024-08-30T09:18:06.634+0000 7fa492b20640 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-30T09:17:24.695219+0000 front 2024-08-30T09:17:24.695243+0000 (oldest deadline 2024-08-30T09:17:48.795417+0000) 2024-08-30T09:18:06.936 INFO:journalctl@ceph.osd.8.smithi190.stdout:Aug 30 09:18:06 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-8[40364]: 2024-08-30T09:18:06.634+0000 7fa492b20640 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-08-30T09:17:28.796078+0000 front 2024-08-30T09:17:28.796109+0000 (oldest deadline 2024-08-30T09:17:54.696291+0000) 2024-08-30T09:18:06.936 INFO:journalctl@ceph.osd.8.smithi190.stdout:Aug 30 09:18:06 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-8[40364]: 2024-08-30T09:18:06.634+0000 7fa492b20640 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.167:6812 osd.4 since back 2024-08-30T09:17:38.797502+0000 front 2024-08-30T09:17:38.797429+0000 (oldest deadline 2024-08-30T09:18:05.198235+0000) 2024-08-30T09:18:06.937 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:06 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:06.699+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:04.091032+0000 front 2024-08-30T09:17:04.090868+0000 (oldest deadline 2024-08-30T09:17:29.391066+0000) 2024-08-30T09:18:06.937 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:06 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:06.699+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:12.193073+0000 front 2024-08-30T09:17:12.192820+0000 (oldest deadline 2024-08-30T09:17:37.492761+0000) 2024-08-30T09:18:06.937 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:06 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:06.699+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-30T09:17:24.494221+0000 front 2024-08-30T09:17:24.494455+0000 (oldest deadline 2024-08-30T09:17:46.194420+0000) 2024-08-30T09:18:06.937 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:06 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:06.699+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.167:6812 osd.4 since back 2024-08-30T09:17:38.897956+0000 front 2024-08-30T09:17:38.897901+0000 (oldest deadline 2024-08-30T09:18:04.698743+0000) 2024-08-30T09:18:07.266 INFO:journalctl@ceph.osd.7.smithi167.stdout:Aug 30 09:18:06 smithi167 podman[58326]: 2024-08-30 09:18:06.953854124 +0000 UTC m=+5.846245956 container died 2d8d08314073b2842276edb8f8a344994f19ff53c0a995db18c2a0bd32b14a83 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-7, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.37.1, maintainer=Guillaume Abrioux , RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image) 2024-08-30T09:18:07.266 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:18:07 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:18:06.999+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:03.239730+0000 front 2024-08-30T09:17:03.239661+0000 (oldest deadline 2024-08-30T09:17:29.139608+0000) 2024-08-30T09:18:07.266 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:18:07 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:18:06.999+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:12.040451+0000 front 2024-08-30T09:17:12.040473+0000 (oldest deadline 2024-08-30T09:17:37.841115+0000) 2024-08-30T09:18:07.266 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:18:07 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:18:06.999+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-30T09:17:22.442437+0000 front 2024-08-30T09:17:22.442414+0000 (oldest deadline 2024-08-30T09:17:48.243149+0000) 2024-08-30T09:18:07.267 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:18:07 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:18:06.999+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-08-30T09:17:32.944470+0000 front 2024-08-30T09:17:32.944528+0000 (oldest deadline 2024-08-30T09:17:55.244278+0000) 2024-08-30T09:18:07.267 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:18:07 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:18:06.999+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.167:6812 osd.4 since back 2024-08-30T09:17:38.546318+0000 front 2024-08-30T09:17:38.546353+0000 (oldest deadline 2024-08-30T09:18:03.246447+0000) 2024-08-30T09:18:07.577 INFO:journalctl@ceph.osd.7.smithi167.stdout:Aug 30 09:18:07 smithi167 podman[58326]: 2024-08-30 09:18:07.347503738 +0000 UTC m=+6.239895558 container cleanup 2d8d08314073b2842276edb8f8a344994f19ff53c0a995db18c2a0bd32b14a83 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-7, ceph=True, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, CEPH_POINT_RELEASE=, io.buildah.version=1.37.1, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD) 2024-08-30T09:18:07.577 INFO:journalctl@ceph.osd.7.smithi167.stdout:Aug 30 09:18:07 smithi167 bash[58326]: ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-7 2024-08-30T09:18:07.633 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:18:07 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9[55148]: 2024-08-30T09:18:07.333+0000 7f7587f3d640 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:07.800304+0000 front 2024-08-30T09:17:07.800409+0000 (oldest deadline 2024-08-30T09:17:30.700443+0000) 2024-08-30T09:18:07.633 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:18:07 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9[55148]: 2024-08-30T09:18:07.333+0000 7f7587f3d640 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:16.601811+0000 front 2024-08-30T09:17:16.601614+0000 (oldest deadline 2024-08-30T09:17:41.901651+0000) 2024-08-30T09:18:07.633 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:18:07 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9[55148]: 2024-08-30T09:18:07.333+0000 7f7587f3d640 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-30T09:17:21.902428+0000 front 2024-08-30T09:17:21.902400+0000 (oldest deadline 2024-08-30T09:17:47.202527+0000) 2024-08-30T09:18:07.633 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:18:07 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9[55148]: 2024-08-30T09:18:07.333+0000 7f7587f3d640 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-08-30T09:17:31.704449+0000 front 2024-08-30T09:17:31.704477+0000 (oldest deadline 2024-08-30T09:17:56.404636+0000) 2024-08-30T09:18:07.633 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:18:07 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9[55148]: 2024-08-30T09:18:07.333+0000 7f7587f3d640 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.167:6812 osd.4 since back 2024-08-30T09:17:41.105874+0000 front 2024-08-30T09:17:41.106081+0000 (oldest deadline 2024-08-30T09:18:05.706638+0000) 2024-08-30T09:18:07.931 INFO:journalctl@ceph.osd.7.smithi167.stdout:Aug 30 09:18:07 smithi167 podman[58342]: 2024-08-30 09:18:07.623813593 +0000 UTC m=+0.663790001 container remove 2d8d08314073b2842276edb8f8a344994f19ff53c0a995db18c2a0bd32b14a83 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-7, io.buildah.version=1.37.1, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-08-30T09:18:07.936 INFO:journalctl@ceph.osd.8.smithi190.stdout:Aug 30 09:18:07 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-8[40364]: 2024-08-30T09:18:07.633+0000 7fa492b20640 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:14.793297+0000 front 2024-08-30T09:17:14.793353+0000 (oldest deadline 2024-08-30T09:17:38.293306+0000) 2024-08-30T09:18:07.936 INFO:journalctl@ceph.osd.8.smithi190.stdout:Aug 30 09:18:07 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-8[40364]: 2024-08-30T09:18:07.633+0000 7fa492b20640 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-30T09:17:24.695219+0000 front 2024-08-30T09:17:24.695243+0000 (oldest deadline 2024-08-30T09:17:48.795417+0000) 2024-08-30T09:18:07.936 INFO:journalctl@ceph.osd.8.smithi190.stdout:Aug 30 09:18:07 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-8[40364]: 2024-08-30T09:18:07.633+0000 7fa492b20640 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-08-30T09:17:28.796078+0000 front 2024-08-30T09:17:28.796109+0000 (oldest deadline 2024-08-30T09:17:54.696291+0000) 2024-08-30T09:18:07.936 INFO:journalctl@ceph.osd.8.smithi190.stdout:Aug 30 09:18:07 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-8[40364]: 2024-08-30T09:18:07.633+0000 7fa492b20640 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.167:6812 osd.4 since back 2024-08-30T09:17:38.797502+0000 front 2024-08-30T09:17:38.797429+0000 (oldest deadline 2024-08-30T09:18:05.198235+0000) 2024-08-30T09:18:07.936 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:07 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:07.723+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:04.091032+0000 front 2024-08-30T09:17:04.090868+0000 (oldest deadline 2024-08-30T09:17:29.391066+0000) 2024-08-30T09:18:07.937 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:07 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:07.723+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:12.193073+0000 front 2024-08-30T09:17:12.192820+0000 (oldest deadline 2024-08-30T09:17:37.492761+0000) 2024-08-30T09:18:07.937 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:07 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:07.723+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-30T09:17:24.494221+0000 front 2024-08-30T09:17:24.494455+0000 (oldest deadline 2024-08-30T09:17:46.194420+0000) 2024-08-30T09:18:07.937 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:07 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:07.723+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.167:6812 osd.4 since back 2024-08-30T09:17:38.897956+0000 front 2024-08-30T09:17:38.897901+0000 (oldest deadline 2024-08-30T09:18:04.698743+0000) 2024-08-30T09:18:08.229 INFO:journalctl@ceph.osd.7.smithi167.stdout:Aug 30 09:18:07 smithi167 podman[58431]: 2024-08-30 09:18:07.832108055 +0000 UTC m=+0.015723478 image pull 79f4cde5e36911a5dca995eeb98098c2c8aae81dd52a088ef2c867bc3363bad0 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61 2024-08-30T09:18:08.229 INFO:journalctl@ceph.osd.7.smithi167.stdout:Aug 30 09:18:07 smithi167 podman[58431]: 2024-08-30 09:18:07.966458367 +0000 UTC m=+0.150073827 container create f09ca3c7495d86a80851bee62a2d02f4b31f5adf8cf140305cadf9b22f696c16 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-7-deactivate, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.37.1, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0) 2024-08-30T09:18:08.229 INFO:journalctl@ceph.osd.7.smithi167.stdout:Aug 30 09:18:08 smithi167 podman[58431]: 2024-08-30 09:18:08.227651274 +0000 UTC m=+0.411266703 container init f09ca3c7495d86a80851bee62a2d02f4b31f5adf8cf140305cadf9b22f696c16 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-7-deactivate, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, io.buildah.version=1.37.1, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True) 2024-08-30T09:18:08.229 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:18:07 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:18:07.965+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:03.239730+0000 front 2024-08-30T09:17:03.239661+0000 (oldest deadline 2024-08-30T09:17:29.139608+0000) 2024-08-30T09:18:08.229 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:18:07 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:18:07.965+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:12.040451+0000 front 2024-08-30T09:17:12.040473+0000 (oldest deadline 2024-08-30T09:17:37.841115+0000) 2024-08-30T09:18:08.229 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:18:07 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:18:07.965+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-30T09:17:22.442437+0000 front 2024-08-30T09:17:22.442414+0000 (oldest deadline 2024-08-30T09:17:48.243149+0000) 2024-08-30T09:18:08.230 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:18:07 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:18:07.965+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-08-30T09:17:32.944470+0000 front 2024-08-30T09:17:32.944528+0000 (oldest deadline 2024-08-30T09:17:55.244278+0000) 2024-08-30T09:18:08.230 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:18:07 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:18:07.965+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.167:6812 osd.4 since back 2024-08-30T09:17:38.546318+0000 front 2024-08-30T09:17:38.546353+0000 (oldest deadline 2024-08-30T09:18:03.246447+0000) 2024-08-30T09:18:08.577 INFO:journalctl@ceph.osd.7.smithi167.stdout:Aug 30 09:18:08 smithi167 podman[58431]: 2024-08-30 09:18:08.230970251 +0000 UTC m=+0.414585673 container start f09ca3c7495d86a80851bee62a2d02f4b31f5adf8cf140305cadf9b22f696c16 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-7-deactivate, org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.1, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, ceph=True) 2024-08-30T09:18:08.577 INFO:journalctl@ceph.osd.7.smithi167.stdout:Aug 30 09:18:08 smithi167 podman[58431]: 2024-08-30 09:18:08.302737838 +0000 UTC m=+0.486353267 container attach f09ca3c7495d86a80851bee62a2d02f4b31f5adf8cf140305cadf9b22f696c16 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-7-deactivate, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.1, org.label-schema.build-date=20240731, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux ) 2024-08-30T09:18:08.577 INFO:journalctl@ceph.osd.7.smithi167.stdout:Aug 30 09:18:08 smithi167 podman[58431]: 2024-08-30 09:18:08.421767984 +0000 UTC m=+0.605383406 container died f09ca3c7495d86a80851bee62a2d02f4b31f5adf8cf140305cadf9b22f696c16 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-7-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, GIT_BRANCH=HEAD, io.buildah.version=1.37.1, GIT_CLEAN=True) 2024-08-30T09:18:08.633 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:18:08 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9[55148]: 2024-08-30T09:18:08.313+0000 7f7587f3d640 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:07.800304+0000 front 2024-08-30T09:17:07.800409+0000 (oldest deadline 2024-08-30T09:17:30.700443+0000) 2024-08-30T09:18:08.633 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:18:08 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9[55148]: 2024-08-30T09:18:08.313+0000 7f7587f3d640 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:16.601811+0000 front 2024-08-30T09:17:16.601614+0000 (oldest deadline 2024-08-30T09:17:41.901651+0000) 2024-08-30T09:18:08.633 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:18:08 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9[55148]: 2024-08-30T09:18:08.313+0000 7f7587f3d640 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-30T09:17:21.902428+0000 front 2024-08-30T09:17:21.902400+0000 (oldest deadline 2024-08-30T09:17:47.202527+0000) 2024-08-30T09:18:08.633 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:18:08 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9[55148]: 2024-08-30T09:18:08.313+0000 7f7587f3d640 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-08-30T09:17:31.704449+0000 front 2024-08-30T09:17:31.704477+0000 (oldest deadline 2024-08-30T09:17:56.404636+0000) 2024-08-30T09:18:08.633 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:18:08 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9[55148]: 2024-08-30T09:18:08.313+0000 7f7587f3d640 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.167:6812 osd.4 since back 2024-08-30T09:17:41.105874+0000 front 2024-08-30T09:17:41.106081+0000 (oldest deadline 2024-08-30T09:18:05.706638+0000) 2024-08-30T09:18:08.936 INFO:journalctl@ceph.osd.8.smithi190.stdout:Aug 30 09:18:08 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-8[40364]: 2024-08-30T09:18:08.662+0000 7fa492b20640 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:14.793297+0000 front 2024-08-30T09:17:14.793353+0000 (oldest deadline 2024-08-30T09:17:38.293306+0000) 2024-08-30T09:18:08.936 INFO:journalctl@ceph.osd.8.smithi190.stdout:Aug 30 09:18:08 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-8[40364]: 2024-08-30T09:18:08.662+0000 7fa492b20640 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-30T09:17:24.695219+0000 front 2024-08-30T09:17:24.695243+0000 (oldest deadline 2024-08-30T09:17:48.795417+0000) 2024-08-30T09:18:08.936 INFO:journalctl@ceph.osd.8.smithi190.stdout:Aug 30 09:18:08 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-8[40364]: 2024-08-30T09:18:08.662+0000 7fa492b20640 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-08-30T09:17:28.796078+0000 front 2024-08-30T09:17:28.796109+0000 (oldest deadline 2024-08-30T09:17:54.696291+0000) 2024-08-30T09:18:08.936 INFO:journalctl@ceph.osd.8.smithi190.stdout:Aug 30 09:18:08 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-8[40364]: 2024-08-30T09:18:08.662+0000 7fa492b20640 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.167:6812 osd.4 since back 2024-08-30T09:17:38.797502+0000 front 2024-08-30T09:17:38.797429+0000 (oldest deadline 2024-08-30T09:18:05.198235+0000) 2024-08-30T09:18:08.937 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:08 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:08.687+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:04.091032+0000 front 2024-08-30T09:17:04.090868+0000 (oldest deadline 2024-08-30T09:17:29.391066+0000) 2024-08-30T09:18:08.937 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:08 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:08.687+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:12.193073+0000 front 2024-08-30T09:17:12.192820+0000 (oldest deadline 2024-08-30T09:17:37.492761+0000) 2024-08-30T09:18:08.937 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:08 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:08.687+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-30T09:17:24.494221+0000 front 2024-08-30T09:17:24.494455+0000 (oldest deadline 2024-08-30T09:17:46.194420+0000) 2024-08-30T09:18:08.937 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:08 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:08.687+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.167:6812 osd.4 since back 2024-08-30T09:17:38.897956+0000 front 2024-08-30T09:17:38.897901+0000 (oldest deadline 2024-08-30T09:18:04.698743+0000) 2024-08-30T09:18:09.024 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:18:08 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:18:08.956+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:03.239730+0000 front 2024-08-30T09:17:03.239661+0000 (oldest deadline 2024-08-30T09:17:29.139608+0000) 2024-08-30T09:18:09.024 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:18:08 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:18:08.956+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:12.040451+0000 front 2024-08-30T09:17:12.040473+0000 (oldest deadline 2024-08-30T09:17:37.841115+0000) 2024-08-30T09:18:09.024 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:18:08 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:18:08.956+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-30T09:17:22.442437+0000 front 2024-08-30T09:17:22.442414+0000 (oldest deadline 2024-08-30T09:17:48.243149+0000) 2024-08-30T09:18:09.024 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:18:08 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:18:08.956+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-08-30T09:17:32.944470+0000 front 2024-08-30T09:17:32.944528+0000 (oldest deadline 2024-08-30T09:17:55.244278+0000) 2024-08-30T09:18:09.025 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:18:08 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:18:08.956+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.167:6812 osd.4 since back 2024-08-30T09:17:38.546318+0000 front 2024-08-30T09:17:38.546353+0000 (oldest deadline 2024-08-30T09:18:03.246447+0000) 2024-08-30T09:18:09.039 DEBUG:teuthology.orchestra.run.smithi167:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968@osd.7.service' 2024-08-30T09:18:09.078 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-30T09:18:09.078 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-08-30T09:18:09.078 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-08-30T09:18:09.078 DEBUG:teuthology.orchestra.run.smithi190:> sudo systemctl stop ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968@osd.8 2024-08-30T09:18:09.387 INFO:journalctl@ceph.osd.8.smithi190.stdout:Aug 30 09:18:09 smithi190 systemd[1]: Stopping Ceph osd.8 for b0895c1a-66ac-11ef-bcd4-c7b262605968... 2024-08-30T09:18:09.633 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:18:09 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9[55148]: 2024-08-30T09:18:09.326+0000 7f7587f3d640 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:07.800304+0000 front 2024-08-30T09:17:07.800409+0000 (oldest deadline 2024-08-30T09:17:30.700443+0000) 2024-08-30T09:18:09.633 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:18:09 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9[55148]: 2024-08-30T09:18:09.326+0000 7f7587f3d640 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:16.601811+0000 front 2024-08-30T09:17:16.601614+0000 (oldest deadline 2024-08-30T09:17:41.901651+0000) 2024-08-30T09:18:09.633 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:18:09 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9[55148]: 2024-08-30T09:18:09.326+0000 7f7587f3d640 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-30T09:17:21.902428+0000 front 2024-08-30T09:17:21.902400+0000 (oldest deadline 2024-08-30T09:17:47.202527+0000) 2024-08-30T09:18:09.633 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:18:09 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9[55148]: 2024-08-30T09:18:09.326+0000 7f7587f3d640 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-08-30T09:17:31.704449+0000 front 2024-08-30T09:17:31.704477+0000 (oldest deadline 2024-08-30T09:17:56.404636+0000) 2024-08-30T09:18:09.633 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:18:09 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9[55148]: 2024-08-30T09:18:09.326+0000 7f7587f3d640 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.167:6812 osd.4 since back 2024-08-30T09:17:41.105874+0000 front 2024-08-30T09:17:41.106081+0000 (oldest deadline 2024-08-30T09:18:05.706638+0000) 2024-08-30T09:18:09.661 INFO:journalctl@ceph.osd.8.smithi190.stdout:Aug 30 09:18:09 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-8[40364]: 2024-08-30T09:18:09.385+0000 7fa496535640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-30T09:18:09.661 INFO:journalctl@ceph.osd.8.smithi190.stdout:Aug 30 09:18:09 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-8[40364]: 2024-08-30T09:18:09.385+0000 7fa496535640 -1 osd.8 87 *** Got signal Terminated *** 2024-08-30T09:18:09.661 INFO:journalctl@ceph.osd.8.smithi190.stdout:Aug 30 09:18:09 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-8[40364]: 2024-08-30T09:18:09.385+0000 7fa496535640 -1 osd.8 87 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-08-30T09:18:09.661 INFO:journalctl@ceph.osd.8.smithi190.stdout:Aug 30 09:18:09 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-8[40364]: 2024-08-30T09:18:09.659+0000 7fa492b20640 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:14.793297+0000 front 2024-08-30T09:17:14.793353+0000 (oldest deadline 2024-08-30T09:17:38.293306+0000) 2024-08-30T09:18:09.661 INFO:journalctl@ceph.osd.8.smithi190.stdout:Aug 30 09:18:09 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-8[40364]: 2024-08-30T09:18:09.659+0000 7fa492b20640 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-30T09:17:24.695219+0000 front 2024-08-30T09:17:24.695243+0000 (oldest deadline 2024-08-30T09:17:48.795417+0000) 2024-08-30T09:18:09.661 INFO:journalctl@ceph.osd.8.smithi190.stdout:Aug 30 09:18:09 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-8[40364]: 2024-08-30T09:18:09.659+0000 7fa492b20640 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-08-30T09:17:28.796078+0000 front 2024-08-30T09:17:28.796109+0000 (oldest deadline 2024-08-30T09:17:54.696291+0000) 2024-08-30T09:18:09.661 INFO:journalctl@ceph.osd.8.smithi190.stdout:Aug 30 09:18:09 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-8[40364]: 2024-08-30T09:18:09.659+0000 7fa492b20640 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.167:6812 osd.4 since back 2024-08-30T09:17:38.797502+0000 front 2024-08-30T09:17:38.797429+0000 (oldest deadline 2024-08-30T09:18:05.198235+0000) 2024-08-30T09:18:09.936 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:09 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:09.661+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:04.091032+0000 front 2024-08-30T09:17:04.090868+0000 (oldest deadline 2024-08-30T09:17:29.391066+0000) 2024-08-30T09:18:09.936 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:09 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:09.661+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:12.193073+0000 front 2024-08-30T09:17:12.192820+0000 (oldest deadline 2024-08-30T09:17:37.492761+0000) 2024-08-30T09:18:09.936 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:09 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:09.661+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-30T09:17:24.494221+0000 front 2024-08-30T09:17:24.494455+0000 (oldest deadline 2024-08-30T09:17:46.194420+0000) 2024-08-30T09:18:09.936 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:09 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:09.661+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.167:6812 osd.4 since back 2024-08-30T09:17:38.897956+0000 front 2024-08-30T09:17:38.897901+0000 (oldest deadline 2024-08-30T09:18:04.698743+0000) 2024-08-30T09:18:10.243 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:18:09 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:18:09.931+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:03.239730+0000 front 2024-08-30T09:17:03.239661+0000 (oldest deadline 2024-08-30T09:17:29.139608+0000) 2024-08-30T09:18:10.243 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:18:09 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:18:09.931+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:12.040451+0000 front 2024-08-30T09:17:12.040473+0000 (oldest deadline 2024-08-30T09:17:37.841115+0000) 2024-08-30T09:18:10.243 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:18:09 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:18:09.931+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-30T09:17:22.442437+0000 front 2024-08-30T09:17:22.442414+0000 (oldest deadline 2024-08-30T09:17:48.243149+0000) 2024-08-30T09:18:10.243 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:18:09 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:18:09.931+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-08-30T09:17:32.944470+0000 front 2024-08-30T09:17:32.944528+0000 (oldest deadline 2024-08-30T09:17:55.244278+0000) 2024-08-30T09:18:10.243 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:18:09 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:18:09.931+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.167:6812 osd.4 since back 2024-08-30T09:17:38.546318+0000 front 2024-08-30T09:17:38.546353+0000 (oldest deadline 2024-08-30T09:18:03.246447+0000) 2024-08-30T09:18:10.633 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:18:10 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9[55148]: 2024-08-30T09:18:10.352+0000 7f7587f3d640 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:07.800304+0000 front 2024-08-30T09:17:07.800409+0000 (oldest deadline 2024-08-30T09:17:30.700443+0000) 2024-08-30T09:18:10.633 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:18:10 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9[55148]: 2024-08-30T09:18:10.352+0000 7f7587f3d640 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:16.601811+0000 front 2024-08-30T09:17:16.601614+0000 (oldest deadline 2024-08-30T09:17:41.901651+0000) 2024-08-30T09:18:10.633 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:18:10 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9[55148]: 2024-08-30T09:18:10.352+0000 7f7587f3d640 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-30T09:17:21.902428+0000 front 2024-08-30T09:17:21.902400+0000 (oldest deadline 2024-08-30T09:17:47.202527+0000) 2024-08-30T09:18:10.633 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:18:10 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9[55148]: 2024-08-30T09:18:10.352+0000 7f7587f3d640 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-08-30T09:17:31.704449+0000 front 2024-08-30T09:17:31.704477+0000 (oldest deadline 2024-08-30T09:17:56.404636+0000) 2024-08-30T09:18:10.633 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:18:10 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9[55148]: 2024-08-30T09:18:10.352+0000 7f7587f3d640 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.167:6812 osd.4 since back 2024-08-30T09:17:41.105874+0000 front 2024-08-30T09:17:41.106081+0000 (oldest deadline 2024-08-30T09:18:05.706638+0000) 2024-08-30T09:18:10.936 INFO:journalctl@ceph.osd.8.smithi190.stdout:Aug 30 09:18:10 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-8[40364]: 2024-08-30T09:18:10.696+0000 7fa492b20640 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:14.793297+0000 front 2024-08-30T09:17:14.793353+0000 (oldest deadline 2024-08-30T09:17:38.293306+0000) 2024-08-30T09:18:10.936 INFO:journalctl@ceph.osd.8.smithi190.stdout:Aug 30 09:18:10 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-8[40364]: 2024-08-30T09:18:10.696+0000 7fa492b20640 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-30T09:17:24.695219+0000 front 2024-08-30T09:17:24.695243+0000 (oldest deadline 2024-08-30T09:17:48.795417+0000) 2024-08-30T09:18:10.936 INFO:journalctl@ceph.osd.8.smithi190.stdout:Aug 30 09:18:10 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-8[40364]: 2024-08-30T09:18:10.696+0000 7fa492b20640 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-08-30T09:17:28.796078+0000 front 2024-08-30T09:17:28.796109+0000 (oldest deadline 2024-08-30T09:17:54.696291+0000) 2024-08-30T09:18:10.937 INFO:journalctl@ceph.osd.8.smithi190.stdout:Aug 30 09:18:10 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-8[40364]: 2024-08-30T09:18:10.696+0000 7fa492b20640 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.167:6812 osd.4 since back 2024-08-30T09:17:38.797502+0000 front 2024-08-30T09:17:38.797429+0000 (oldest deadline 2024-08-30T09:18:05.198235+0000) 2024-08-30T09:18:10.937 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:10 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:10.667+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:04.091032+0000 front 2024-08-30T09:17:04.090868+0000 (oldest deadline 2024-08-30T09:17:29.391066+0000) 2024-08-30T09:18:10.937 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:10 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:10.667+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:12.193073+0000 front 2024-08-30T09:17:12.192820+0000 (oldest deadline 2024-08-30T09:17:37.492761+0000) 2024-08-30T09:18:10.937 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:10 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:10.667+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-30T09:17:24.494221+0000 front 2024-08-30T09:17:24.494455+0000 (oldest deadline 2024-08-30T09:17:46.194420+0000) 2024-08-30T09:18:10.937 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:10 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:10.667+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.167:6812 osd.4 since back 2024-08-30T09:17:38.897956+0000 front 2024-08-30T09:17:38.897901+0000 (oldest deadline 2024-08-30T09:18:04.698743+0000) 2024-08-30T09:18:11.327 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:18:10 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:18:10.905+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:03.239730+0000 front 2024-08-30T09:17:03.239661+0000 (oldest deadline 2024-08-30T09:17:29.139608+0000) 2024-08-30T09:18:11.327 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:18:10 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:18:10.905+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:12.040451+0000 front 2024-08-30T09:17:12.040473+0000 (oldest deadline 2024-08-30T09:17:37.841115+0000) 2024-08-30T09:18:11.327 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:18:10 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:18:10.905+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-30T09:17:22.442437+0000 front 2024-08-30T09:17:22.442414+0000 (oldest deadline 2024-08-30T09:17:48.243149+0000) 2024-08-30T09:18:11.327 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:18:10 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:18:10.905+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-08-30T09:17:32.944470+0000 front 2024-08-30T09:17:32.944528+0000 (oldest deadline 2024-08-30T09:17:55.244278+0000) 2024-08-30T09:18:11.328 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:18:10 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:18:10.905+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.167:6812 osd.4 since back 2024-08-30T09:17:38.546318+0000 front 2024-08-30T09:17:38.546353+0000 (oldest deadline 2024-08-30T09:18:03.246447+0000) 2024-08-30T09:18:11.633 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:18:11 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9[55148]: 2024-08-30T09:18:11.359+0000 7f7587f3d640 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:07.800304+0000 front 2024-08-30T09:17:07.800409+0000 (oldest deadline 2024-08-30T09:17:30.700443+0000) 2024-08-30T09:18:11.633 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:18:11 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9[55148]: 2024-08-30T09:18:11.359+0000 7f7587f3d640 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:16.601811+0000 front 2024-08-30T09:17:16.601614+0000 (oldest deadline 2024-08-30T09:17:41.901651+0000) 2024-08-30T09:18:11.633 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:18:11 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9[55148]: 2024-08-30T09:18:11.359+0000 7f7587f3d640 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-30T09:17:21.902428+0000 front 2024-08-30T09:17:21.902400+0000 (oldest deadline 2024-08-30T09:17:47.202527+0000) 2024-08-30T09:18:11.633 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:18:11 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9[55148]: 2024-08-30T09:18:11.359+0000 7f7587f3d640 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-08-30T09:17:31.704449+0000 front 2024-08-30T09:17:31.704477+0000 (oldest deadline 2024-08-30T09:17:56.404636+0000) 2024-08-30T09:18:11.633 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:18:11 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9[55148]: 2024-08-30T09:18:11.359+0000 7f7587f3d640 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.167:6812 osd.4 since back 2024-08-30T09:17:41.105874+0000 front 2024-08-30T09:17:41.106081+0000 (oldest deadline 2024-08-30T09:18:05.706638+0000) 2024-08-30T09:18:11.633 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:18:11 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9[55148]: 2024-08-30T09:18:11.359+0000 7f7587f3d640 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-08-30T09:17:45.707247+0000 front 2024-08-30T09:17:45.707326+0000 (oldest deadline 2024-08-30T09:18:11.007036+0000) 2024-08-30T09:18:11.936 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:11 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:11.706+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:04.091032+0000 front 2024-08-30T09:17:04.090868+0000 (oldest deadline 2024-08-30T09:17:29.391066+0000) 2024-08-30T09:18:11.936 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:11 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:11.706+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:12.193073+0000 front 2024-08-30T09:17:12.192820+0000 (oldest deadline 2024-08-30T09:17:37.492761+0000) 2024-08-30T09:18:11.936 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:11 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:11.706+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-30T09:17:24.494221+0000 front 2024-08-30T09:17:24.494455+0000 (oldest deadline 2024-08-30T09:17:46.194420+0000) 2024-08-30T09:18:11.936 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:11 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:11.706+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.167:6812 osd.4 since back 2024-08-30T09:17:38.897956+0000 front 2024-08-30T09:17:38.897901+0000 (oldest deadline 2024-08-30T09:18:04.698743+0000) 2024-08-30T09:18:11.936 INFO:journalctl@ceph.osd.8.smithi190.stdout:Aug 30 09:18:11 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-8[40364]: 2024-08-30T09:18:11.684+0000 7fa492b20640 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:14.793297+0000 front 2024-08-30T09:17:14.793353+0000 (oldest deadline 2024-08-30T09:17:38.293306+0000) 2024-08-30T09:18:11.937 INFO:journalctl@ceph.osd.8.smithi190.stdout:Aug 30 09:18:11 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-8[40364]: 2024-08-30T09:18:11.684+0000 7fa492b20640 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-30T09:17:24.695219+0000 front 2024-08-30T09:17:24.695243+0000 (oldest deadline 2024-08-30T09:17:48.795417+0000) 2024-08-30T09:18:11.937 INFO:journalctl@ceph.osd.8.smithi190.stdout:Aug 30 09:18:11 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-8[40364]: 2024-08-30T09:18:11.684+0000 7fa492b20640 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-08-30T09:17:28.796078+0000 front 2024-08-30T09:17:28.796109+0000 (oldest deadline 2024-08-30T09:17:54.696291+0000) 2024-08-30T09:18:11.937 INFO:journalctl@ceph.osd.8.smithi190.stdout:Aug 30 09:18:11 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-8[40364]: 2024-08-30T09:18:11.684+0000 7fa492b20640 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.167:6812 osd.4 since back 2024-08-30T09:17:38.797502+0000 front 2024-08-30T09:17:38.797429+0000 (oldest deadline 2024-08-30T09:18:05.198235+0000) 2024-08-30T09:18:11.937 INFO:journalctl@ceph.osd.8.smithi190.stdout:Aug 30 09:18:11 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-8[40364]: 2024-08-30T09:18:11.684+0000 7fa492b20640 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-08-30T09:17:45.198651+0000 front 2024-08-30T09:17:45.198623+0000 (oldest deadline 2024-08-30T09:18:11.098717+0000) 2024-08-30T09:18:12.327 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:18:11 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:18:11.908+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:03.239730+0000 front 2024-08-30T09:17:03.239661+0000 (oldest deadline 2024-08-30T09:17:29.139608+0000) 2024-08-30T09:18:12.327 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:18:11 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:18:11.908+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:12.040451+0000 front 2024-08-30T09:17:12.040473+0000 (oldest deadline 2024-08-30T09:17:37.841115+0000) 2024-08-30T09:18:12.327 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:18:11 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:18:11.908+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-30T09:17:22.442437+0000 front 2024-08-30T09:17:22.442414+0000 (oldest deadline 2024-08-30T09:17:48.243149+0000) 2024-08-30T09:18:12.327 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:18:11 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:18:11.908+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-08-30T09:17:32.944470+0000 front 2024-08-30T09:17:32.944528+0000 (oldest deadline 2024-08-30T09:17:55.244278+0000) 2024-08-30T09:18:12.327 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:18:11 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:18:11.908+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.167:6812 osd.4 since back 2024-08-30T09:17:38.546318+0000 front 2024-08-30T09:17:38.546353+0000 (oldest deadline 2024-08-30T09:18:03.246447+0000) 2024-08-30T09:18:12.633 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:18:12 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9[55148]: 2024-08-30T09:18:12.372+0000 7f7587f3d640 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:07.800304+0000 front 2024-08-30T09:17:07.800409+0000 (oldest deadline 2024-08-30T09:17:30.700443+0000) 2024-08-30T09:18:12.633 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:18:12 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9[55148]: 2024-08-30T09:18:12.372+0000 7f7587f3d640 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:16.601811+0000 front 2024-08-30T09:17:16.601614+0000 (oldest deadline 2024-08-30T09:17:41.901651+0000) 2024-08-30T09:18:12.633 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:18:12 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9[55148]: 2024-08-30T09:18:12.372+0000 7f7587f3d640 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-30T09:17:21.902428+0000 front 2024-08-30T09:17:21.902400+0000 (oldest deadline 2024-08-30T09:17:47.202527+0000) 2024-08-30T09:18:12.633 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:18:12 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9[55148]: 2024-08-30T09:18:12.372+0000 7f7587f3d640 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-08-30T09:17:31.704449+0000 front 2024-08-30T09:17:31.704477+0000 (oldest deadline 2024-08-30T09:17:56.404636+0000) 2024-08-30T09:18:12.633 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:18:12 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9[55148]: 2024-08-30T09:18:12.372+0000 7f7587f3d640 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.167:6812 osd.4 since back 2024-08-30T09:17:41.105874+0000 front 2024-08-30T09:17:41.106081+0000 (oldest deadline 2024-08-30T09:18:05.706638+0000) 2024-08-30T09:18:12.633 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:18:12 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9[55148]: 2024-08-30T09:18:12.372+0000 7f7587f3d640 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-08-30T09:17:45.707247+0000 front 2024-08-30T09:17:45.707326+0000 (oldest deadline 2024-08-30T09:18:11.007036+0000) 2024-08-30T09:18:13.186 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:12 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:12.734+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:04.091032+0000 front 2024-08-30T09:17:04.090868+0000 (oldest deadline 2024-08-30T09:17:29.391066+0000) 2024-08-30T09:18:13.186 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:12 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:12.734+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:12.193073+0000 front 2024-08-30T09:17:12.192820+0000 (oldest deadline 2024-08-30T09:17:37.492761+0000) 2024-08-30T09:18:13.186 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:12 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:12.734+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-30T09:17:24.494221+0000 front 2024-08-30T09:17:24.494455+0000 (oldest deadline 2024-08-30T09:17:46.194420+0000) 2024-08-30T09:18:13.186 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:12 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:12.734+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.167:6812 osd.4 since back 2024-08-30T09:17:38.897956+0000 front 2024-08-30T09:17:38.897901+0000 (oldest deadline 2024-08-30T09:18:04.698743+0000) 2024-08-30T09:18:13.187 INFO:journalctl@ceph.osd.8.smithi190.stdout:Aug 30 09:18:12 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-8[40364]: 2024-08-30T09:18:12.691+0000 7fa492b20640 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:14.793297+0000 front 2024-08-30T09:17:14.793353+0000 (oldest deadline 2024-08-30T09:17:38.293306+0000) 2024-08-30T09:18:13.187 INFO:journalctl@ceph.osd.8.smithi190.stdout:Aug 30 09:18:12 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-8[40364]: 2024-08-30T09:18:12.691+0000 7fa492b20640 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-30T09:17:24.695219+0000 front 2024-08-30T09:17:24.695243+0000 (oldest deadline 2024-08-30T09:17:48.795417+0000) 2024-08-30T09:18:13.187 INFO:journalctl@ceph.osd.8.smithi190.stdout:Aug 30 09:18:12 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-8[40364]: 2024-08-30T09:18:12.691+0000 7fa492b20640 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-08-30T09:17:28.796078+0000 front 2024-08-30T09:17:28.796109+0000 (oldest deadline 2024-08-30T09:17:54.696291+0000) 2024-08-30T09:18:13.187 INFO:journalctl@ceph.osd.8.smithi190.stdout:Aug 30 09:18:12 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-8[40364]: 2024-08-30T09:18:12.691+0000 7fa492b20640 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.167:6812 osd.4 since back 2024-08-30T09:17:38.797502+0000 front 2024-08-30T09:17:38.797429+0000 (oldest deadline 2024-08-30T09:18:05.198235+0000) 2024-08-30T09:18:13.187 INFO:journalctl@ceph.osd.8.smithi190.stdout:Aug 30 09:18:12 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-8[40364]: 2024-08-30T09:18:12.691+0000 7fa492b20640 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-08-30T09:17:45.198651+0000 front 2024-08-30T09:17:45.198623+0000 (oldest deadline 2024-08-30T09:18:11.098717+0000) 2024-08-30T09:18:13.327 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:18:12 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:18:12.866+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:03.239730+0000 front 2024-08-30T09:17:03.239661+0000 (oldest deadline 2024-08-30T09:17:29.139608+0000) 2024-08-30T09:18:13.327 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:18:12 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:18:12.866+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:12.040451+0000 front 2024-08-30T09:17:12.040473+0000 (oldest deadline 2024-08-30T09:17:37.841115+0000) 2024-08-30T09:18:13.327 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:18:12 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:18:12.866+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-30T09:17:22.442437+0000 front 2024-08-30T09:17:22.442414+0000 (oldest deadline 2024-08-30T09:17:48.243149+0000) 2024-08-30T09:18:13.327 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:18:12 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:18:12.866+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-08-30T09:17:32.944470+0000 front 2024-08-30T09:17:32.944528+0000 (oldest deadline 2024-08-30T09:17:55.244278+0000) 2024-08-30T09:18:13.327 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:18:12 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:18:12.866+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.167:6812 osd.4 since back 2024-08-30T09:17:38.546318+0000 front 2024-08-30T09:17:38.546353+0000 (oldest deadline 2024-08-30T09:18:03.246447+0000) 2024-08-30T09:18:13.327 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:18:12 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:18:12.866+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-08-30T09:17:49.648107+0000 front 2024-08-30T09:17:49.648282+0000 (oldest deadline 2024-08-30T09:18:12.547941+0000) 2024-08-30T09:18:13.883 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:18:13 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9[55148]: 2024-08-30T09:18:13.382+0000 7f7587f3d640 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:07.800304+0000 front 2024-08-30T09:17:07.800409+0000 (oldest deadline 2024-08-30T09:17:30.700443+0000) 2024-08-30T09:18:13.883 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:18:13 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9[55148]: 2024-08-30T09:18:13.382+0000 7f7587f3d640 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:16.601811+0000 front 2024-08-30T09:17:16.601614+0000 (oldest deadline 2024-08-30T09:17:41.901651+0000) 2024-08-30T09:18:13.883 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:18:13 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9[55148]: 2024-08-30T09:18:13.382+0000 7f7587f3d640 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-30T09:17:21.902428+0000 front 2024-08-30T09:17:21.902400+0000 (oldest deadline 2024-08-30T09:17:47.202527+0000) 2024-08-30T09:18:13.883 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:18:13 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9[55148]: 2024-08-30T09:18:13.382+0000 7f7587f3d640 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-08-30T09:17:31.704449+0000 front 2024-08-30T09:17:31.704477+0000 (oldest deadline 2024-08-30T09:17:56.404636+0000) 2024-08-30T09:18:13.883 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:18:13 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9[55148]: 2024-08-30T09:18:13.382+0000 7f7587f3d640 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.167:6812 osd.4 since back 2024-08-30T09:17:41.105874+0000 front 2024-08-30T09:17:41.106081+0000 (oldest deadline 2024-08-30T09:18:05.706638+0000) 2024-08-30T09:18:13.883 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:18:13 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9[55148]: 2024-08-30T09:18:13.382+0000 7f7587f3d640 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-08-30T09:17:45.707247+0000 front 2024-08-30T09:17:45.707326+0000 (oldest deadline 2024-08-30T09:18:11.007036+0000) 2024-08-30T09:18:13.936 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:13 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:13.689+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:04.091032+0000 front 2024-08-30T09:17:04.090868+0000 (oldest deadline 2024-08-30T09:17:29.391066+0000) 2024-08-30T09:18:13.936 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:13 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:13.689+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:12.193073+0000 front 2024-08-30T09:17:12.192820+0000 (oldest deadline 2024-08-30T09:17:37.492761+0000) 2024-08-30T09:18:13.936 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:13 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:13.689+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-30T09:17:24.494221+0000 front 2024-08-30T09:17:24.494455+0000 (oldest deadline 2024-08-30T09:17:46.194420+0000) 2024-08-30T09:18:13.936 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:13 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:13.689+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.167:6812 osd.4 since back 2024-08-30T09:17:38.897956+0000 front 2024-08-30T09:17:38.897901+0000 (oldest deadline 2024-08-30T09:18:04.698743+0000) 2024-08-30T09:18:13.937 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:13 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:13.689+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-08-30T09:17:46.399666+0000 front 2024-08-30T09:17:46.399767+0000 (oldest deadline 2024-08-30T09:18:12.800338+0000) 2024-08-30T09:18:13.937 INFO:journalctl@ceph.osd.8.smithi190.stdout:Aug 30 09:18:13 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-8[40364]: 2024-08-30T09:18:13.676+0000 7fa492b20640 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:14.793297+0000 front 2024-08-30T09:17:14.793353+0000 (oldest deadline 2024-08-30T09:17:38.293306+0000) 2024-08-30T09:18:13.937 INFO:journalctl@ceph.osd.8.smithi190.stdout:Aug 30 09:18:13 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-8[40364]: 2024-08-30T09:18:13.676+0000 7fa492b20640 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-30T09:17:24.695219+0000 front 2024-08-30T09:17:24.695243+0000 (oldest deadline 2024-08-30T09:17:48.795417+0000) 2024-08-30T09:18:13.937 INFO:journalctl@ceph.osd.8.smithi190.stdout:Aug 30 09:18:13 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-8[40364]: 2024-08-30T09:18:13.676+0000 7fa492b20640 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-08-30T09:17:28.796078+0000 front 2024-08-30T09:17:28.796109+0000 (oldest deadline 2024-08-30T09:17:54.696291+0000) 2024-08-30T09:18:13.937 INFO:journalctl@ceph.osd.8.smithi190.stdout:Aug 30 09:18:13 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-8[40364]: 2024-08-30T09:18:13.676+0000 7fa492b20640 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.167:6812 osd.4 since back 2024-08-30T09:17:38.797502+0000 front 2024-08-30T09:17:38.797429+0000 (oldest deadline 2024-08-30T09:18:05.198235+0000) 2024-08-30T09:18:13.937 INFO:journalctl@ceph.osd.8.smithi190.stdout:Aug 30 09:18:13 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-8[40364]: 2024-08-30T09:18:13.676+0000 7fa492b20640 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-08-30T09:17:45.198651+0000 front 2024-08-30T09:17:45.198623+0000 (oldest deadline 2024-08-30T09:18:11.098717+0000) 2024-08-30T09:18:14.327 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:18:13 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:18:13.827+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:03.239730+0000 front 2024-08-30T09:17:03.239661+0000 (oldest deadline 2024-08-30T09:17:29.139608+0000) 2024-08-30T09:18:14.327 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:18:13 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:18:13.827+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:12.040451+0000 front 2024-08-30T09:17:12.040473+0000 (oldest deadline 2024-08-30T09:17:37.841115+0000) 2024-08-30T09:18:14.327 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:18:13 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:18:13.827+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-30T09:17:22.442437+0000 front 2024-08-30T09:17:22.442414+0000 (oldest deadline 2024-08-30T09:17:48.243149+0000) 2024-08-30T09:18:14.328 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:18:13 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:18:13.827+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-08-30T09:17:32.944470+0000 front 2024-08-30T09:17:32.944528+0000 (oldest deadline 2024-08-30T09:17:55.244278+0000) 2024-08-30T09:18:14.328 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:18:13 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:18:13.827+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.167:6812 osd.4 since back 2024-08-30T09:17:38.546318+0000 front 2024-08-30T09:17:38.546353+0000 (oldest deadline 2024-08-30T09:18:03.246447+0000) 2024-08-30T09:18:14.328 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:18:13 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:18:13.827+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-08-30T09:17:49.648107+0000 front 2024-08-30T09:17:49.648282+0000 (oldest deadline 2024-08-30T09:18:12.547941+0000) 2024-08-30T09:18:14.633 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:18:14 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9[55148]: 2024-08-30T09:18:14.338+0000 7f7587f3d640 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:07.800304+0000 front 2024-08-30T09:17:07.800409+0000 (oldest deadline 2024-08-30T09:17:30.700443+0000) 2024-08-30T09:18:14.633 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:18:14 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9[55148]: 2024-08-30T09:18:14.338+0000 7f7587f3d640 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:16.601811+0000 front 2024-08-30T09:17:16.601614+0000 (oldest deadline 2024-08-30T09:17:41.901651+0000) 2024-08-30T09:18:14.633 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:18:14 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9[55148]: 2024-08-30T09:18:14.338+0000 7f7587f3d640 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-30T09:17:21.902428+0000 front 2024-08-30T09:17:21.902400+0000 (oldest deadline 2024-08-30T09:17:47.202527+0000) 2024-08-30T09:18:14.633 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:18:14 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9[55148]: 2024-08-30T09:18:14.338+0000 7f7587f3d640 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-08-30T09:17:31.704449+0000 front 2024-08-30T09:17:31.704477+0000 (oldest deadline 2024-08-30T09:17:56.404636+0000) 2024-08-30T09:18:14.634 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:18:14 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9[55148]: 2024-08-30T09:18:14.338+0000 7f7587f3d640 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.167:6812 osd.4 since back 2024-08-30T09:17:41.105874+0000 front 2024-08-30T09:17:41.106081+0000 (oldest deadline 2024-08-30T09:18:05.706638+0000) 2024-08-30T09:18:14.634 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:18:14 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9[55148]: 2024-08-30T09:18:14.338+0000 7f7587f3d640 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-08-30T09:17:45.707247+0000 front 2024-08-30T09:17:45.707326+0000 (oldest deadline 2024-08-30T09:18:11.007036+0000) 2024-08-30T09:18:14.936 INFO:journalctl@ceph.osd.8.smithi190.stdout:Aug 30 09:18:14 smithi190 podman[58948]: 2024-08-30 09:18:14.900458328 +0000 UTC m=+5.632345022 container died b5580defe98874a3dbb442039fed0f70e2b24e7221dea443dd08dfdf936934a9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-8, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.37.1) 2024-08-30T09:18:14.936 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:14 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:14.668+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:04.091032+0000 front 2024-08-30T09:17:04.090868+0000 (oldest deadline 2024-08-30T09:17:29.391066+0000) 2024-08-30T09:18:14.936 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:14 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:14.668+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:12.193073+0000 front 2024-08-30T09:17:12.192820+0000 (oldest deadline 2024-08-30T09:17:37.492761+0000) 2024-08-30T09:18:14.937 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:14 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:14.668+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-30T09:17:24.494221+0000 front 2024-08-30T09:17:24.494455+0000 (oldest deadline 2024-08-30T09:17:46.194420+0000) 2024-08-30T09:18:14.937 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:14 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:14.668+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.167:6812 osd.4 since back 2024-08-30T09:17:38.897956+0000 front 2024-08-30T09:17:38.897901+0000 (oldest deadline 2024-08-30T09:18:04.698743+0000) 2024-08-30T09:18:14.937 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:14 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:14.668+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-08-30T09:17:46.399666+0000 front 2024-08-30T09:17:46.399767+0000 (oldest deadline 2024-08-30T09:18:12.800338+0000) 2024-08-30T09:18:15.327 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:18:14 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:18:14.834+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:03.239730+0000 front 2024-08-30T09:17:03.239661+0000 (oldest deadline 2024-08-30T09:17:29.139608+0000) 2024-08-30T09:18:15.327 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:18:14 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:18:14.834+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:12.040451+0000 front 2024-08-30T09:17:12.040473+0000 (oldest deadline 2024-08-30T09:17:37.841115+0000) 2024-08-30T09:18:15.328 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:18:14 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:18:14.834+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-30T09:17:22.442437+0000 front 2024-08-30T09:17:22.442414+0000 (oldest deadline 2024-08-30T09:17:48.243149+0000) 2024-08-30T09:18:15.328 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:18:14 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:18:14.834+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-08-30T09:17:32.944470+0000 front 2024-08-30T09:17:32.944528+0000 (oldest deadline 2024-08-30T09:17:55.244278+0000) 2024-08-30T09:18:15.328 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:18:14 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:18:14.834+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.167:6812 osd.4 since back 2024-08-30T09:17:38.546318+0000 front 2024-08-30T09:17:38.546353+0000 (oldest deadline 2024-08-30T09:18:03.246447+0000) 2024-08-30T09:18:15.328 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:18:14 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:18:14.834+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-08-30T09:17:49.648107+0000 front 2024-08-30T09:17:49.648282+0000 (oldest deadline 2024-08-30T09:18:12.547941+0000) 2024-08-30T09:18:15.588 INFO:journalctl@ceph.osd.8.smithi190.stdout:Aug 30 09:18:15 smithi190 podman[58948]: 2024-08-30 09:18:15.309832982 +0000 UTC m=+6.041719678 container cleanup b5580defe98874a3dbb442039fed0f70e2b24e7221dea443dd08dfdf936934a9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-8, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.37.1, GIT_CLEAN=True) 2024-08-30T09:18:15.588 INFO:journalctl@ceph.osd.8.smithi190.stdout:Aug 30 09:18:15 smithi190 bash[58948]: ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-8 2024-08-30T09:18:15.633 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:18:15 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9[55148]: 2024-08-30T09:18:15.340+0000 7f7587f3d640 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:07.800304+0000 front 2024-08-30T09:17:07.800409+0000 (oldest deadline 2024-08-30T09:17:30.700443+0000) 2024-08-30T09:18:15.633 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:18:15 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9[55148]: 2024-08-30T09:18:15.340+0000 7f7587f3d640 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:16.601811+0000 front 2024-08-30T09:17:16.601614+0000 (oldest deadline 2024-08-30T09:17:41.901651+0000) 2024-08-30T09:18:15.633 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:18:15 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9[55148]: 2024-08-30T09:18:15.340+0000 7f7587f3d640 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-30T09:17:21.902428+0000 front 2024-08-30T09:17:21.902400+0000 (oldest deadline 2024-08-30T09:17:47.202527+0000) 2024-08-30T09:18:15.633 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:18:15 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9[55148]: 2024-08-30T09:18:15.340+0000 7f7587f3d640 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-08-30T09:17:31.704449+0000 front 2024-08-30T09:17:31.704477+0000 (oldest deadline 2024-08-30T09:17:56.404636+0000) 2024-08-30T09:18:15.633 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:18:15 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9[55148]: 2024-08-30T09:18:15.340+0000 7f7587f3d640 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.167:6812 osd.4 since back 2024-08-30T09:17:41.105874+0000 front 2024-08-30T09:17:41.106081+0000 (oldest deadline 2024-08-30T09:18:05.706638+0000) 2024-08-30T09:18:15.633 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:18:15 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9[55148]: 2024-08-30T09:18:15.340+0000 7f7587f3d640 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-08-30T09:17:45.707247+0000 front 2024-08-30T09:17:45.707326+0000 (oldest deadline 2024-08-30T09:18:11.007036+0000) 2024-08-30T09:18:15.936 INFO:journalctl@ceph.osd.8.smithi190.stdout:Aug 30 09:18:15 smithi190 podman[58963]: 2024-08-30 09:18:15.5864769 +0000 UTC m=+0.686743391 container remove b5580defe98874a3dbb442039fed0f70e2b24e7221dea443dd08dfdf936934a9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-8, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.37.1, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-08-30T09:18:15.936 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:15 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:15.661+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:04.091032+0000 front 2024-08-30T09:17:04.090868+0000 (oldest deadline 2024-08-30T09:17:29.391066+0000) 2024-08-30T09:18:15.936 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:15 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:15.661+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:12.193073+0000 front 2024-08-30T09:17:12.192820+0000 (oldest deadline 2024-08-30T09:17:37.492761+0000) 2024-08-30T09:18:15.937 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:15 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:15.661+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-30T09:17:24.494221+0000 front 2024-08-30T09:17:24.494455+0000 (oldest deadline 2024-08-30T09:17:46.194420+0000) 2024-08-30T09:18:15.937 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:15 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:15.661+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.167:6812 osd.4 since back 2024-08-30T09:17:38.897956+0000 front 2024-08-30T09:17:38.897901+0000 (oldest deadline 2024-08-30T09:18:04.698743+0000) 2024-08-30T09:18:15.937 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:15 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:15.661+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-08-30T09:17:46.399666+0000 front 2024-08-30T09:17:46.399767+0000 (oldest deadline 2024-08-30T09:18:12.800338+0000) 2024-08-30T09:18:16.232 INFO:journalctl@ceph.osd.8.smithi190.stdout:Aug 30 09:18:15 smithi190 podman[59052]: 2024-08-30 09:18:15.876615833 +0000 UTC m=+0.015437125 image pull 79f4cde5e36911a5dca995eeb98098c2c8aae81dd52a088ef2c867bc3363bad0 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61 2024-08-30T09:18:16.232 INFO:journalctl@ceph.osd.8.smithi190.stdout:Aug 30 09:18:15 smithi190 podman[59052]: 2024-08-30 09:18:15.987558852 +0000 UTC m=+0.126380179 container create 01b6173518f6f88817270bf89c70adbf0e798bd4619d99741ce8a6ba29ac3d46 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-8-deactivate, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, io.buildah.version=1.37.1, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-08-30T09:18:16.327 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:18:15 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:18:15.867+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:03.239730+0000 front 2024-08-30T09:17:03.239661+0000 (oldest deadline 2024-08-30T09:17:29.139608+0000) 2024-08-30T09:18:16.327 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:18:15 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:18:15.867+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:12.040451+0000 front 2024-08-30T09:17:12.040473+0000 (oldest deadline 2024-08-30T09:17:37.841115+0000) 2024-08-30T09:18:16.327 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:18:15 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:18:15.867+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-30T09:17:22.442437+0000 front 2024-08-30T09:17:22.442414+0000 (oldest deadline 2024-08-30T09:17:48.243149+0000) 2024-08-30T09:18:16.327 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:18:15 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:18:15.867+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-08-30T09:17:32.944470+0000 front 2024-08-30T09:17:32.944528+0000 (oldest deadline 2024-08-30T09:17:55.244278+0000) 2024-08-30T09:18:16.327 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:18:15 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:18:15.867+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.167:6812 osd.4 since back 2024-08-30T09:17:38.546318+0000 front 2024-08-30T09:17:38.546353+0000 (oldest deadline 2024-08-30T09:18:03.246447+0000) 2024-08-30T09:18:16.328 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:18:15 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:18:15.867+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-08-30T09:17:49.648107+0000 front 2024-08-30T09:17:49.648282+0000 (oldest deadline 2024-08-30T09:18:12.547941+0000) 2024-08-30T09:18:16.626 INFO:journalctl@ceph.osd.8.smithi190.stdout:Aug 30 09:18:16 smithi190 podman[59052]: 2024-08-30 09:18:16.231340379 +0000 UTC m=+0.370161664 container init 01b6173518f6f88817270bf89c70adbf0e798bd4619d99741ce8a6ba29ac3d46 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-8-deactivate, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.1, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=) 2024-08-30T09:18:16.626 INFO:journalctl@ceph.osd.8.smithi190.stdout:Aug 30 09:18:16 smithi190 podman[59052]: 2024-08-30 09:18:16.234655153 +0000 UTC m=+0.373476439 container start 01b6173518f6f88817270bf89c70adbf0e798bd4619d99741ce8a6ba29ac3d46 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-8-deactivate, io.buildah.version=1.37.1, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux ) 2024-08-30T09:18:16.626 INFO:journalctl@ceph.osd.8.smithi190.stdout:Aug 30 09:18:16 smithi190 podman[59052]: 2024-08-30 09:18:16.314740658 +0000 UTC m=+0.453561962 container attach 01b6173518f6f88817270bf89c70adbf0e798bd4619d99741ce8a6ba29ac3d46 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-8-deactivate, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240731, io.buildah.version=1.37.1, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0) 2024-08-30T09:18:16.626 INFO:journalctl@ceph.osd.8.smithi190.stdout:Aug 30 09:18:16 smithi190 podman[59075]: 2024-08-30 09:18:16.455045904 +0000 UTC m=+0.016286397 container died 01b6173518f6f88817270bf89c70adbf0e798bd4619d99741ce8a6ba29ac3d46 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-8-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.1, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.build-date=20240731) 2024-08-30T09:18:16.633 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:18:16 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9[55148]: 2024-08-30T09:18:16.295+0000 7f7587f3d640 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:07.800304+0000 front 2024-08-30T09:17:07.800409+0000 (oldest deadline 2024-08-30T09:17:30.700443+0000) 2024-08-30T09:18:16.633 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:18:16 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9[55148]: 2024-08-30T09:18:16.295+0000 7f7587f3d640 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:16.601811+0000 front 2024-08-30T09:17:16.601614+0000 (oldest deadline 2024-08-30T09:17:41.901651+0000) 2024-08-30T09:18:16.633 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:18:16 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9[55148]: 2024-08-30T09:18:16.295+0000 7f7587f3d640 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-30T09:17:21.902428+0000 front 2024-08-30T09:17:21.902400+0000 (oldest deadline 2024-08-30T09:17:47.202527+0000) 2024-08-30T09:18:16.633 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:18:16 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9[55148]: 2024-08-30T09:18:16.295+0000 7f7587f3d640 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-08-30T09:17:31.704449+0000 front 2024-08-30T09:17:31.704477+0000 (oldest deadline 2024-08-30T09:17:56.404636+0000) 2024-08-30T09:18:16.633 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:18:16 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9[55148]: 2024-08-30T09:18:16.295+0000 7f7587f3d640 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.167:6812 osd.4 since back 2024-08-30T09:17:41.105874+0000 front 2024-08-30T09:17:41.106081+0000 (oldest deadline 2024-08-30T09:18:05.706638+0000) 2024-08-30T09:18:16.633 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:18:16 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9[55148]: 2024-08-30T09:18:16.295+0000 7f7587f3d640 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-08-30T09:17:45.707247+0000 front 2024-08-30T09:17:45.707326+0000 (oldest deadline 2024-08-30T09:18:11.007036+0000) 2024-08-30T09:18:16.936 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:16 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:16.622+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:04.091032+0000 front 2024-08-30T09:17:04.090868+0000 (oldest deadline 2024-08-30T09:17:29.391066+0000) 2024-08-30T09:18:16.936 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:16 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:16.622+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:12.193073+0000 front 2024-08-30T09:17:12.192820+0000 (oldest deadline 2024-08-30T09:17:37.492761+0000) 2024-08-30T09:18:16.936 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:16 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:16.622+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-30T09:17:24.494221+0000 front 2024-08-30T09:17:24.494455+0000 (oldest deadline 2024-08-30T09:17:46.194420+0000) 2024-08-30T09:18:16.936 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:16 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:16.622+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.167:6812 osd.4 since back 2024-08-30T09:17:38.897956+0000 front 2024-08-30T09:17:38.897901+0000 (oldest deadline 2024-08-30T09:18:04.698743+0000) 2024-08-30T09:18:16.937 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:16 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:16.622+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-08-30T09:17:46.399666+0000 front 2024-08-30T09:17:46.399767+0000 (oldest deadline 2024-08-30T09:18:12.800338+0000) 2024-08-30T09:18:17.077 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:18:16 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:18:16.821+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:03.239730+0000 front 2024-08-30T09:17:03.239661+0000 (oldest deadline 2024-08-30T09:17:29.139608+0000) 2024-08-30T09:18:17.077 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:18:16 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:18:16.821+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:12.040451+0000 front 2024-08-30T09:17:12.040473+0000 (oldest deadline 2024-08-30T09:17:37.841115+0000) 2024-08-30T09:18:17.077 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:18:16 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:18:16.821+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-30T09:17:22.442437+0000 front 2024-08-30T09:17:22.442414+0000 (oldest deadline 2024-08-30T09:17:48.243149+0000) 2024-08-30T09:18:17.077 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:18:16 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:18:16.821+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-08-30T09:17:32.944470+0000 front 2024-08-30T09:17:32.944528+0000 (oldest deadline 2024-08-30T09:17:55.244278+0000) 2024-08-30T09:18:17.077 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:18:16 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:18:16.821+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.167:6812 osd.4 since back 2024-08-30T09:17:38.546318+0000 front 2024-08-30T09:17:38.546353+0000 (oldest deadline 2024-08-30T09:18:03.246447+0000) 2024-08-30T09:18:17.077 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:18:16 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:18:16.821+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-08-30T09:17:49.648107+0000 front 2024-08-30T09:17:49.648282+0000 (oldest deadline 2024-08-30T09:18:12.547941+0000) 2024-08-30T09:18:17.099 DEBUG:teuthology.orchestra.run.smithi190:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968@osd.8.service' 2024-08-30T09:18:17.140 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-30T09:18:17.140 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-08-30T09:18:17.140 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-08-30T09:18:17.140 DEBUG:teuthology.orchestra.run.smithi042:> sudo systemctl stop ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968@osd.9 2024-08-30T09:18:17.437 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:18:17 smithi042 systemd[1]: Stopping Ceph osd.9 for b0895c1a-66ac-11ef-bcd4-c7b262605968... 2024-08-30T09:18:17.437 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:18:17 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9[55148]: 2024-08-30T09:18:17.308+0000 7f7587f3d640 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:07.800304+0000 front 2024-08-30T09:17:07.800409+0000 (oldest deadline 2024-08-30T09:17:30.700443+0000) 2024-08-30T09:18:17.437 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:18:17 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9[55148]: 2024-08-30T09:18:17.308+0000 7f7587f3d640 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:16.601811+0000 front 2024-08-30T09:17:16.601614+0000 (oldest deadline 2024-08-30T09:17:41.901651+0000) 2024-08-30T09:18:17.437 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:18:17 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9[55148]: 2024-08-30T09:18:17.308+0000 7f7587f3d640 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-30T09:17:21.902428+0000 front 2024-08-30T09:17:21.902400+0000 (oldest deadline 2024-08-30T09:17:47.202527+0000) 2024-08-30T09:18:17.437 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:18:17 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9[55148]: 2024-08-30T09:18:17.308+0000 7f7587f3d640 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-08-30T09:17:31.704449+0000 front 2024-08-30T09:17:31.704477+0000 (oldest deadline 2024-08-30T09:17:56.404636+0000) 2024-08-30T09:18:17.437 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:18:17 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9[55148]: 2024-08-30T09:18:17.308+0000 7f7587f3d640 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.167:6812 osd.4 since back 2024-08-30T09:17:41.105874+0000 front 2024-08-30T09:17:41.106081+0000 (oldest deadline 2024-08-30T09:18:05.706638+0000) 2024-08-30T09:18:17.437 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:18:17 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9[55148]: 2024-08-30T09:18:17.308+0000 7f7587f3d640 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-08-30T09:17:45.707247+0000 front 2024-08-30T09:17:45.707326+0000 (oldest deadline 2024-08-30T09:18:11.007036+0000) 2024-08-30T09:18:17.438 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:18:17 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9[55148]: 2024-08-30T09:18:17.433+0000 7f758f95a640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-30T09:18:17.438 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:18:17 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9[55148]: 2024-08-30T09:18:17.433+0000 7f758f95a640 -1 osd.9 87 *** Got signal Terminated *** 2024-08-30T09:18:17.438 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:18:17 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9[55148]: 2024-08-30T09:18:17.433+0000 7f758f95a640 -1 osd.9 87 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-08-30T09:18:17.686 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:17 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:17.590+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:04.091032+0000 front 2024-08-30T09:17:04.090868+0000 (oldest deadline 2024-08-30T09:17:29.391066+0000) 2024-08-30T09:18:17.686 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:17 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:17.590+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:12.193073+0000 front 2024-08-30T09:17:12.192820+0000 (oldest deadline 2024-08-30T09:17:37.492761+0000) 2024-08-30T09:18:17.686 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:17 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:17.590+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-30T09:17:24.494221+0000 front 2024-08-30T09:17:24.494455+0000 (oldest deadline 2024-08-30T09:17:46.194420+0000) 2024-08-30T09:18:17.686 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:17 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:17.590+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.167:6812 osd.4 since back 2024-08-30T09:17:38.897956+0000 front 2024-08-30T09:17:38.897901+0000 (oldest deadline 2024-08-30T09:18:04.698743+0000) 2024-08-30T09:18:17.687 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:17 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:17.590+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-08-30T09:17:46.399666+0000 front 2024-08-30T09:17:46.399767+0000 (oldest deadline 2024-08-30T09:18:12.800338+0000) 2024-08-30T09:18:18.327 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:18:17 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:18:17.826+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:03.239730+0000 front 2024-08-30T09:17:03.239661+0000 (oldest deadline 2024-08-30T09:17:29.139608+0000) 2024-08-30T09:18:18.327 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:18:17 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:18:17.826+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:12.040451+0000 front 2024-08-30T09:17:12.040473+0000 (oldest deadline 2024-08-30T09:17:37.841115+0000) 2024-08-30T09:18:18.327 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:18:17 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:18:17.826+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-30T09:17:22.442437+0000 front 2024-08-30T09:17:22.442414+0000 (oldest deadline 2024-08-30T09:17:48.243149+0000) 2024-08-30T09:18:18.327 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:18:17 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:18:17.826+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-08-30T09:17:32.944470+0000 front 2024-08-30T09:17:32.944528+0000 (oldest deadline 2024-08-30T09:17:55.244278+0000) 2024-08-30T09:18:18.327 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:18:17 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:18:17.826+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.167:6812 osd.4 since back 2024-08-30T09:17:38.546318+0000 front 2024-08-30T09:17:38.546353+0000 (oldest deadline 2024-08-30T09:18:03.246447+0000) 2024-08-30T09:18:18.327 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:18:17 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:18:17.826+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-08-30T09:17:49.648107+0000 front 2024-08-30T09:17:49.648282+0000 (oldest deadline 2024-08-30T09:18:12.547941+0000) 2024-08-30T09:18:18.633 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:18:18 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9[55148]: 2024-08-30T09:18:18.317+0000 7f7587f3d640 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:07.800304+0000 front 2024-08-30T09:17:07.800409+0000 (oldest deadline 2024-08-30T09:17:30.700443+0000) 2024-08-30T09:18:18.633 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:18:18 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9[55148]: 2024-08-30T09:18:18.317+0000 7f7587f3d640 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:16.601811+0000 front 2024-08-30T09:17:16.601614+0000 (oldest deadline 2024-08-30T09:17:41.901651+0000) 2024-08-30T09:18:18.633 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:18:18 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9[55148]: 2024-08-30T09:18:18.317+0000 7f7587f3d640 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-30T09:17:21.902428+0000 front 2024-08-30T09:17:21.902400+0000 (oldest deadline 2024-08-30T09:17:47.202527+0000) 2024-08-30T09:18:18.633 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:18:18 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9[55148]: 2024-08-30T09:18:18.317+0000 7f7587f3d640 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-08-30T09:17:31.704449+0000 front 2024-08-30T09:17:31.704477+0000 (oldest deadline 2024-08-30T09:17:56.404636+0000) 2024-08-30T09:18:18.633 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:18:18 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9[55148]: 2024-08-30T09:18:18.317+0000 7f7587f3d640 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.167:6812 osd.4 since back 2024-08-30T09:17:41.105874+0000 front 2024-08-30T09:17:41.106081+0000 (oldest deadline 2024-08-30T09:18:05.706638+0000) 2024-08-30T09:18:18.633 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:18:18 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9[55148]: 2024-08-30T09:18:18.317+0000 7f7587f3d640 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-08-30T09:17:45.707247+0000 front 2024-08-30T09:17:45.707326+0000 (oldest deadline 2024-08-30T09:18:11.007036+0000) 2024-08-30T09:18:18.936 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:18 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:18.602+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:04.091032+0000 front 2024-08-30T09:17:04.090868+0000 (oldest deadline 2024-08-30T09:17:29.391066+0000) 2024-08-30T09:18:18.936 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:18 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:18.602+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:12.193073+0000 front 2024-08-30T09:17:12.192820+0000 (oldest deadline 2024-08-30T09:17:37.492761+0000) 2024-08-30T09:18:18.936 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:18 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:18.602+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-30T09:17:24.494221+0000 front 2024-08-30T09:17:24.494455+0000 (oldest deadline 2024-08-30T09:17:46.194420+0000) 2024-08-30T09:18:18.936 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:18 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:18.602+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.167:6812 osd.4 since back 2024-08-30T09:17:38.897956+0000 front 2024-08-30T09:17:38.897901+0000 (oldest deadline 2024-08-30T09:18:04.698743+0000) 2024-08-30T09:18:18.937 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:18 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:18.602+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-08-30T09:17:46.399666+0000 front 2024-08-30T09:17:46.399767+0000 (oldest deadline 2024-08-30T09:18:12.800338+0000) 2024-08-30T09:18:19.327 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:18:18 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:18:18.831+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:03.239730+0000 front 2024-08-30T09:17:03.239661+0000 (oldest deadline 2024-08-30T09:17:29.139608+0000) 2024-08-30T09:18:19.327 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:18:18 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:18:18.831+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:12.040451+0000 front 2024-08-30T09:17:12.040473+0000 (oldest deadline 2024-08-30T09:17:37.841115+0000) 2024-08-30T09:18:19.327 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:18:18 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:18:18.831+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-30T09:17:22.442437+0000 front 2024-08-30T09:17:22.442414+0000 (oldest deadline 2024-08-30T09:17:48.243149+0000) 2024-08-30T09:18:19.328 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:18:18 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:18:18.831+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-08-30T09:17:32.944470+0000 front 2024-08-30T09:17:32.944528+0000 (oldest deadline 2024-08-30T09:17:55.244278+0000) 2024-08-30T09:18:19.328 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:18:18 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:18:18.831+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.167:6812 osd.4 since back 2024-08-30T09:17:38.546318+0000 front 2024-08-30T09:17:38.546353+0000 (oldest deadline 2024-08-30T09:18:03.246447+0000) 2024-08-30T09:18:19.328 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:18:18 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:18:18.831+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-08-30T09:17:49.648107+0000 front 2024-08-30T09:17:49.648282+0000 (oldest deadline 2024-08-30T09:18:12.547941+0000) 2024-08-30T09:18:19.633 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:18:19 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9[55148]: 2024-08-30T09:18:19.349+0000 7f7587f3d640 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:07.800304+0000 front 2024-08-30T09:17:07.800409+0000 (oldest deadline 2024-08-30T09:17:30.700443+0000) 2024-08-30T09:18:19.633 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:18:19 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9[55148]: 2024-08-30T09:18:19.349+0000 7f7587f3d640 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:16.601811+0000 front 2024-08-30T09:17:16.601614+0000 (oldest deadline 2024-08-30T09:17:41.901651+0000) 2024-08-30T09:18:19.633 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:18:19 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9[55148]: 2024-08-30T09:18:19.349+0000 7f7587f3d640 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-30T09:17:21.902428+0000 front 2024-08-30T09:17:21.902400+0000 (oldest deadline 2024-08-30T09:17:47.202527+0000) 2024-08-30T09:18:19.633 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:18:19 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9[55148]: 2024-08-30T09:18:19.349+0000 7f7587f3d640 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-08-30T09:17:31.704449+0000 front 2024-08-30T09:17:31.704477+0000 (oldest deadline 2024-08-30T09:17:56.404636+0000) 2024-08-30T09:18:19.633 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:18:19 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9[55148]: 2024-08-30T09:18:19.349+0000 7f7587f3d640 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.167:6812 osd.4 since back 2024-08-30T09:17:41.105874+0000 front 2024-08-30T09:17:41.106081+0000 (oldest deadline 2024-08-30T09:18:05.706638+0000) 2024-08-30T09:18:19.633 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:18:19 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9[55148]: 2024-08-30T09:18:19.349+0000 7f7587f3d640 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-08-30T09:17:45.707247+0000 front 2024-08-30T09:17:45.707326+0000 (oldest deadline 2024-08-30T09:18:11.007036+0000) 2024-08-30T09:18:19.936 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:19 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:19.642+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:04.091032+0000 front 2024-08-30T09:17:04.090868+0000 (oldest deadline 2024-08-30T09:17:29.391066+0000) 2024-08-30T09:18:19.936 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:19 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:19.642+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:12.193073+0000 front 2024-08-30T09:17:12.192820+0000 (oldest deadline 2024-08-30T09:17:37.492761+0000) 2024-08-30T09:18:19.936 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:19 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:19.642+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-30T09:17:24.494221+0000 front 2024-08-30T09:17:24.494455+0000 (oldest deadline 2024-08-30T09:17:46.194420+0000) 2024-08-30T09:18:19.936 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:19 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:19.642+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.167:6812 osd.4 since back 2024-08-30T09:17:38.897956+0000 front 2024-08-30T09:17:38.897901+0000 (oldest deadline 2024-08-30T09:18:04.698743+0000) 2024-08-30T09:18:19.936 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:19 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:19.642+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-08-30T09:17:46.399666+0000 front 2024-08-30T09:17:46.399767+0000 (oldest deadline 2024-08-30T09:18:12.800338+0000) 2024-08-30T09:18:20.077 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:18:19 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:18:19.847+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:03.239730+0000 front 2024-08-30T09:17:03.239661+0000 (oldest deadline 2024-08-30T09:17:29.139608+0000) 2024-08-30T09:18:20.077 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:18:19 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:18:19.847+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:12.040451+0000 front 2024-08-30T09:17:12.040473+0000 (oldest deadline 2024-08-30T09:17:37.841115+0000) 2024-08-30T09:18:20.077 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:18:19 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:18:19.847+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-30T09:17:22.442437+0000 front 2024-08-30T09:17:22.442414+0000 (oldest deadline 2024-08-30T09:17:48.243149+0000) 2024-08-30T09:18:20.077 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:18:19 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:18:19.847+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-08-30T09:17:32.944470+0000 front 2024-08-30T09:17:32.944528+0000 (oldest deadline 2024-08-30T09:17:55.244278+0000) 2024-08-30T09:18:20.077 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:18:19 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:18:19.847+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.167:6812 osd.4 since back 2024-08-30T09:17:38.546318+0000 front 2024-08-30T09:17:38.546353+0000 (oldest deadline 2024-08-30T09:18:03.246447+0000) 2024-08-30T09:18:20.078 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:18:19 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:18:19.847+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-08-30T09:17:49.648107+0000 front 2024-08-30T09:17:49.648282+0000 (oldest deadline 2024-08-30T09:18:12.547941+0000) 2024-08-30T09:18:20.633 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:18:20 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9[55148]: 2024-08-30T09:18:20.306+0000 7f7587f3d640 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:07.800304+0000 front 2024-08-30T09:17:07.800409+0000 (oldest deadline 2024-08-30T09:17:30.700443+0000) 2024-08-30T09:18:20.633 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:18:20 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9[55148]: 2024-08-30T09:18:20.306+0000 7f7587f3d640 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:16.601811+0000 front 2024-08-30T09:17:16.601614+0000 (oldest deadline 2024-08-30T09:17:41.901651+0000) 2024-08-30T09:18:20.633 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:18:20 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9[55148]: 2024-08-30T09:18:20.306+0000 7f7587f3d640 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-30T09:17:21.902428+0000 front 2024-08-30T09:17:21.902400+0000 (oldest deadline 2024-08-30T09:17:47.202527+0000) 2024-08-30T09:18:20.633 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:18:20 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9[55148]: 2024-08-30T09:18:20.306+0000 7f7587f3d640 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-08-30T09:17:31.704449+0000 front 2024-08-30T09:17:31.704477+0000 (oldest deadline 2024-08-30T09:17:56.404636+0000) 2024-08-30T09:18:20.633 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:18:20 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9[55148]: 2024-08-30T09:18:20.306+0000 7f7587f3d640 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.167:6812 osd.4 since back 2024-08-30T09:17:41.105874+0000 front 2024-08-30T09:17:41.106081+0000 (oldest deadline 2024-08-30T09:18:05.706638+0000) 2024-08-30T09:18:20.633 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:18:20 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9[55148]: 2024-08-30T09:18:20.306+0000 7f7587f3d640 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-08-30T09:17:45.707247+0000 front 2024-08-30T09:17:45.707326+0000 (oldest deadline 2024-08-30T09:18:11.007036+0000) 2024-08-30T09:18:21.186 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:20 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:20.691+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:04.091032+0000 front 2024-08-30T09:17:04.090868+0000 (oldest deadline 2024-08-30T09:17:29.391066+0000) 2024-08-30T09:18:21.186 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:20 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:20.691+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:12.193073+0000 front 2024-08-30T09:17:12.192820+0000 (oldest deadline 2024-08-30T09:17:37.492761+0000) 2024-08-30T09:18:21.186 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:20 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:20.691+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-30T09:17:24.494221+0000 front 2024-08-30T09:17:24.494455+0000 (oldest deadline 2024-08-30T09:17:46.194420+0000) 2024-08-30T09:18:21.186 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:20 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:20.691+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.167:6812 osd.4 since back 2024-08-30T09:17:38.897956+0000 front 2024-08-30T09:17:38.897901+0000 (oldest deadline 2024-08-30T09:18:04.698743+0000) 2024-08-30T09:18:21.187 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:20 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:20.691+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-08-30T09:17:46.399666+0000 front 2024-08-30T09:17:46.399767+0000 (oldest deadline 2024-08-30T09:18:12.800338+0000) 2024-08-30T09:18:21.187 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:20 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:20.691+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.42:6822 osd.6 since back 2024-08-30T09:17:56.801860+0000 front 2024-08-30T09:17:56.801820+0000 (oldest deadline 2024-08-30T09:18:19.701830+0000) 2024-08-30T09:18:21.327 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:18:20 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:18:20.898+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:03.239730+0000 front 2024-08-30T09:17:03.239661+0000 (oldest deadline 2024-08-30T09:17:29.139608+0000) 2024-08-30T09:18:21.327 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:18:20 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:18:20.898+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:12.040451+0000 front 2024-08-30T09:17:12.040473+0000 (oldest deadline 2024-08-30T09:17:37.841115+0000) 2024-08-30T09:18:21.327 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:18:20 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:18:20.898+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-30T09:17:22.442437+0000 front 2024-08-30T09:17:22.442414+0000 (oldest deadline 2024-08-30T09:17:48.243149+0000) 2024-08-30T09:18:21.327 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:18:20 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:18:20.898+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-08-30T09:17:32.944470+0000 front 2024-08-30T09:17:32.944528+0000 (oldest deadline 2024-08-30T09:17:55.244278+0000) 2024-08-30T09:18:21.327 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:18:20 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:18:20.898+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.167:6812 osd.4 since back 2024-08-30T09:17:38.546318+0000 front 2024-08-30T09:17:38.546353+0000 (oldest deadline 2024-08-30T09:18:03.246447+0000) 2024-08-30T09:18:21.327 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:18:20 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:18:20.898+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-08-30T09:17:49.648107+0000 front 2024-08-30T09:17:49.648282+0000 (oldest deadline 2024-08-30T09:18:12.547941+0000) 2024-08-30T09:18:21.328 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:18:20 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:18:20.898+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.42:6822 osd.6 since back 2024-08-30T09:17:56.049071+0000 front 2024-08-30T09:17:56.049221+0000 (oldest deadline 2024-08-30T09:18:20.148952+0000) 2024-08-30T09:18:21.633 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:18:21 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9[55148]: 2024-08-30T09:18:21.327+0000 7f7587f3d640 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:07.800304+0000 front 2024-08-30T09:17:07.800409+0000 (oldest deadline 2024-08-30T09:17:30.700443+0000) 2024-08-30T09:18:21.633 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:18:21 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9[55148]: 2024-08-30T09:18:21.327+0000 7f7587f3d640 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:16.601811+0000 front 2024-08-30T09:17:16.601614+0000 (oldest deadline 2024-08-30T09:17:41.901651+0000) 2024-08-30T09:18:21.633 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:18:21 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9[55148]: 2024-08-30T09:18:21.327+0000 7f7587f3d640 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-30T09:17:21.902428+0000 front 2024-08-30T09:17:21.902400+0000 (oldest deadline 2024-08-30T09:17:47.202527+0000) 2024-08-30T09:18:21.633 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:18:21 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9[55148]: 2024-08-30T09:18:21.327+0000 7f7587f3d640 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-08-30T09:17:31.704449+0000 front 2024-08-30T09:17:31.704477+0000 (oldest deadline 2024-08-30T09:17:56.404636+0000) 2024-08-30T09:18:21.633 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:18:21 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9[55148]: 2024-08-30T09:18:21.327+0000 7f7587f3d640 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.167:6812 osd.4 since back 2024-08-30T09:17:41.105874+0000 front 2024-08-30T09:17:41.106081+0000 (oldest deadline 2024-08-30T09:18:05.706638+0000) 2024-08-30T09:18:21.633 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:18:21 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9[55148]: 2024-08-30T09:18:21.327+0000 7f7587f3d640 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-08-30T09:17:45.707247+0000 front 2024-08-30T09:17:45.707326+0000 (oldest deadline 2024-08-30T09:18:11.007036+0000) 2024-08-30T09:18:22.186 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:21 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:21.739+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:04.091032+0000 front 2024-08-30T09:17:04.090868+0000 (oldest deadline 2024-08-30T09:17:29.391066+0000) 2024-08-30T09:18:22.186 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:21 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:21.739+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:12.193073+0000 front 2024-08-30T09:17:12.192820+0000 (oldest deadline 2024-08-30T09:17:37.492761+0000) 2024-08-30T09:18:22.186 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:21 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:21.739+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-30T09:17:24.494221+0000 front 2024-08-30T09:17:24.494455+0000 (oldest deadline 2024-08-30T09:17:46.194420+0000) 2024-08-30T09:18:22.186 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:21 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:21.739+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.167:6812 osd.4 since back 2024-08-30T09:17:38.897956+0000 front 2024-08-30T09:17:38.897901+0000 (oldest deadline 2024-08-30T09:18:04.698743+0000) 2024-08-30T09:18:22.186 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:21 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:21.739+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-08-30T09:17:46.399666+0000 front 2024-08-30T09:17:46.399767+0000 (oldest deadline 2024-08-30T09:18:12.800338+0000) 2024-08-30T09:18:22.187 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:21 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:21.739+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.42:6822 osd.6 since back 2024-08-30T09:17:56.801860+0000 front 2024-08-30T09:17:56.801820+0000 (oldest deadline 2024-08-30T09:18:19.701830+0000) 2024-08-30T09:18:22.327 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:18:21 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:18:21.861+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:03.239730+0000 front 2024-08-30T09:17:03.239661+0000 (oldest deadline 2024-08-30T09:17:29.139608+0000) 2024-08-30T09:18:22.327 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:18:21 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:18:21.861+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:12.040451+0000 front 2024-08-30T09:17:12.040473+0000 (oldest deadline 2024-08-30T09:17:37.841115+0000) 2024-08-30T09:18:22.327 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:18:21 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:18:21.861+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-30T09:17:22.442437+0000 front 2024-08-30T09:17:22.442414+0000 (oldest deadline 2024-08-30T09:17:48.243149+0000) 2024-08-30T09:18:22.327 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:18:21 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:18:21.861+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-08-30T09:17:32.944470+0000 front 2024-08-30T09:17:32.944528+0000 (oldest deadline 2024-08-30T09:17:55.244278+0000) 2024-08-30T09:18:22.327 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:18:21 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:18:21.861+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.167:6812 osd.4 since back 2024-08-30T09:17:38.546318+0000 front 2024-08-30T09:17:38.546353+0000 (oldest deadline 2024-08-30T09:18:03.246447+0000) 2024-08-30T09:18:22.327 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:18:21 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:18:21.861+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-08-30T09:17:49.648107+0000 front 2024-08-30T09:17:49.648282+0000 (oldest deadline 2024-08-30T09:18:12.547941+0000) 2024-08-30T09:18:22.328 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:18:21 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:18:21.861+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.42:6822 osd.6 since back 2024-08-30T09:17:56.049071+0000 front 2024-08-30T09:17:56.049221+0000 (oldest deadline 2024-08-30T09:18:20.148952+0000) 2024-08-30T09:18:22.633 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:18:22 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9[55148]: 2024-08-30T09:18:22.339+0000 7f7587f3d640 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:07.800304+0000 front 2024-08-30T09:17:07.800409+0000 (oldest deadline 2024-08-30T09:17:30.700443+0000) 2024-08-30T09:18:22.633 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:18:22 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9[55148]: 2024-08-30T09:18:22.339+0000 7f7587f3d640 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:16.601811+0000 front 2024-08-30T09:17:16.601614+0000 (oldest deadline 2024-08-30T09:17:41.901651+0000) 2024-08-30T09:18:22.633 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:18:22 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9[55148]: 2024-08-30T09:18:22.339+0000 7f7587f3d640 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-30T09:17:21.902428+0000 front 2024-08-30T09:17:21.902400+0000 (oldest deadline 2024-08-30T09:17:47.202527+0000) 2024-08-30T09:18:22.633 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:18:22 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9[55148]: 2024-08-30T09:18:22.339+0000 7f7587f3d640 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-08-30T09:17:31.704449+0000 front 2024-08-30T09:17:31.704477+0000 (oldest deadline 2024-08-30T09:17:56.404636+0000) 2024-08-30T09:18:22.633 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:18:22 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9[55148]: 2024-08-30T09:18:22.339+0000 7f7587f3d640 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.167:6812 osd.4 since back 2024-08-30T09:17:41.105874+0000 front 2024-08-30T09:17:41.106081+0000 (oldest deadline 2024-08-30T09:18:05.706638+0000) 2024-08-30T09:18:22.633 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:18:22 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9[55148]: 2024-08-30T09:18:22.339+0000 7f7587f3d640 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-08-30T09:17:45.707247+0000 front 2024-08-30T09:17:45.707326+0000 (oldest deadline 2024-08-30T09:18:11.007036+0000) 2024-08-30T09:18:23.186 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:22 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:22.717+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:04.091032+0000 front 2024-08-30T09:17:04.090868+0000 (oldest deadline 2024-08-30T09:17:29.391066+0000) 2024-08-30T09:18:23.187 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:22 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:22.717+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:12.193073+0000 front 2024-08-30T09:17:12.192820+0000 (oldest deadline 2024-08-30T09:17:37.492761+0000) 2024-08-30T09:18:23.187 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:22 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:22.717+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-30T09:17:24.494221+0000 front 2024-08-30T09:17:24.494455+0000 (oldest deadline 2024-08-30T09:17:46.194420+0000) 2024-08-30T09:18:23.187 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:22 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:22.717+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.167:6812 osd.4 since back 2024-08-30T09:17:38.897956+0000 front 2024-08-30T09:17:38.897901+0000 (oldest deadline 2024-08-30T09:18:04.698743+0000) 2024-08-30T09:18:23.187 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:22 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:22.717+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-08-30T09:17:46.399666+0000 front 2024-08-30T09:17:46.399767+0000 (oldest deadline 2024-08-30T09:18:12.800338+0000) 2024-08-30T09:18:23.187 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:22 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:22.717+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.42:6822 osd.6 since back 2024-08-30T09:17:56.801860+0000 front 2024-08-30T09:17:56.801820+0000 (oldest deadline 2024-08-30T09:18:19.701830+0000) 2024-08-30T09:18:23.254 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:18:22 smithi042 podman[101329]: 2024-08-30 09:18:22.969012927 +0000 UTC m=+5.635009813 container died 440b157a7c9014b568735867b112a70677218f8a91960faaf3b248d611e5b1ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9, org.label-schema.license=GPLv2, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.1, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-08-30T09:18:23.327 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:18:22 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:18:22.852+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:03.239730+0000 front 2024-08-30T09:17:03.239661+0000 (oldest deadline 2024-08-30T09:17:29.139608+0000) 2024-08-30T09:18:23.327 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:18:22 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:18:22.852+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:12.040451+0000 front 2024-08-30T09:17:12.040473+0000 (oldest deadline 2024-08-30T09:17:37.841115+0000) 2024-08-30T09:18:23.327 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:18:22 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:18:22.852+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-30T09:17:22.442437+0000 front 2024-08-30T09:17:22.442414+0000 (oldest deadline 2024-08-30T09:17:48.243149+0000) 2024-08-30T09:18:23.327 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:18:22 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:18:22.852+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-08-30T09:17:32.944470+0000 front 2024-08-30T09:17:32.944528+0000 (oldest deadline 2024-08-30T09:17:55.244278+0000) 2024-08-30T09:18:23.327 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:18:22 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:18:22.852+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.167:6812 osd.4 since back 2024-08-30T09:17:38.546318+0000 front 2024-08-30T09:17:38.546353+0000 (oldest deadline 2024-08-30T09:18:03.246447+0000) 2024-08-30T09:18:23.327 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:18:22 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:18:22.852+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-08-30T09:17:49.648107+0000 front 2024-08-30T09:17:49.648282+0000 (oldest deadline 2024-08-30T09:18:12.547941+0000) 2024-08-30T09:18:23.327 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:18:22 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:18:22.852+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.42:6822 osd.6 since back 2024-08-30T09:17:56.049071+0000 front 2024-08-30T09:17:56.049221+0000 (oldest deadline 2024-08-30T09:18:20.148952+0000) 2024-08-30T09:18:23.596 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:18:23 smithi042 podman[101329]: 2024-08-30 09:18:23.401736469 +0000 UTC m=+6.067733339 container cleanup 440b157a7c9014b568735867b112a70677218f8a91960faaf3b248d611e5b1ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9, io.buildah.version=1.37.1, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-08-30T09:18:23.597 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:18:23 smithi042 bash[101329]: ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9 2024-08-30T09:18:23.883 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:18:23 smithi042 podman[101345]: 2024-08-30 09:18:23.594893899 +0000 UTC m=+0.628243510 container remove 440b157a7c9014b568735867b112a70677218f8a91960faaf3b248d611e5b1ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, io.buildah.version=1.37.1, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-08-30T09:18:24.186 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:23 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:23.699+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:04.091032+0000 front 2024-08-30T09:17:04.090868+0000 (oldest deadline 2024-08-30T09:17:29.391066+0000) 2024-08-30T09:18:24.187 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:23 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:23.699+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:12.193073+0000 front 2024-08-30T09:17:12.192820+0000 (oldest deadline 2024-08-30T09:17:37.492761+0000) 2024-08-30T09:18:24.187 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:23 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:23.699+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-30T09:17:24.494221+0000 front 2024-08-30T09:17:24.494455+0000 (oldest deadline 2024-08-30T09:17:46.194420+0000) 2024-08-30T09:18:24.187 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:23 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:23.699+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.167:6812 osd.4 since back 2024-08-30T09:17:38.897956+0000 front 2024-08-30T09:17:38.897901+0000 (oldest deadline 2024-08-30T09:18:04.698743+0000) 2024-08-30T09:18:24.187 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:23 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:23.699+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-08-30T09:17:46.399666+0000 front 2024-08-30T09:17:46.399767+0000 (oldest deadline 2024-08-30T09:18:12.800338+0000) 2024-08-30T09:18:24.187 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:23 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:23.699+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.42:6822 osd.6 since back 2024-08-30T09:17:56.801860+0000 front 2024-08-30T09:17:56.801820+0000 (oldest deadline 2024-08-30T09:18:19.701830+0000) 2024-08-30T09:18:24.327 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:18:23 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:18:23.851+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:03.239730+0000 front 2024-08-30T09:17:03.239661+0000 (oldest deadline 2024-08-30T09:17:29.139608+0000) 2024-08-30T09:18:24.327 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:18:23 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:18:23.851+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:12.040451+0000 front 2024-08-30T09:17:12.040473+0000 (oldest deadline 2024-08-30T09:17:37.841115+0000) 2024-08-30T09:18:24.327 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:18:23 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:18:23.851+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-30T09:17:22.442437+0000 front 2024-08-30T09:17:22.442414+0000 (oldest deadline 2024-08-30T09:17:48.243149+0000) 2024-08-30T09:18:24.327 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:18:23 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:18:23.851+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-08-30T09:17:32.944470+0000 front 2024-08-30T09:17:32.944528+0000 (oldest deadline 2024-08-30T09:17:55.244278+0000) 2024-08-30T09:18:24.327 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:18:23 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:18:23.851+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.167:6812 osd.4 since back 2024-08-30T09:17:38.546318+0000 front 2024-08-30T09:17:38.546353+0000 (oldest deadline 2024-08-30T09:18:03.246447+0000) 2024-08-30T09:18:24.328 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:18:23 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:18:23.851+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-08-30T09:17:49.648107+0000 front 2024-08-30T09:17:49.648282+0000 (oldest deadline 2024-08-30T09:18:12.547941+0000) 2024-08-30T09:18:24.328 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:18:23 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:18:23.851+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.42:6822 osd.6 since back 2024-08-30T09:17:56.049071+0000 front 2024-08-30T09:17:56.049221+0000 (oldest deadline 2024-08-30T09:18:20.148952+0000) 2024-08-30T09:18:24.349 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:18:24 smithi042 podman[101445]: 2024-08-30 09:18:24.020684834 +0000 UTC m=+0.101678991 container create 95f7d7f1b42051b5ee4b509df40f47081c4f102caa93dd8428646a2fa35fdfb9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9-deactivate, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.1, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.build-date=20240731) 2024-08-30T09:18:24.349 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:18:24 smithi042 podman[101445]: 2024-08-30 09:18:23.935109712 +0000 UTC m=+0.016103878 image pull 79f4cde5e36911a5dca995eeb98098c2c8aae81dd52a088ef2c867bc3363bad0 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61 2024-08-30T09:18:24.350 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:18:24 smithi042 podman[101445]: 2024-08-30 09:18:24.247995203 +0000 UTC m=+0.328989359 container init 95f7d7f1b42051b5ee4b509df40f47081c4f102caa93dd8428646a2fa35fdfb9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9-deactivate, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.build-date=20240731, io.buildah.version=1.37.1, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-08-30T09:18:24.350 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:18:24 smithi042 podman[101445]: 2024-08-30 09:18:24.251241482 +0000 UTC m=+0.332235647 container start 95f7d7f1b42051b5ee4b509df40f47081c4f102caa93dd8428646a2fa35fdfb9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9-deactivate, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.37.1, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=) 2024-08-30T09:18:24.633 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:18:24 smithi042 podman[101445]: 2024-08-30 09:18:24.34810286 +0000 UTC m=+0.429097034 container attach 95f7d7f1b42051b5ee4b509df40f47081c4f102caa93dd8428646a2fa35fdfb9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9-deactivate, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.1, org.label-schema.build-date=20240731) 2024-08-30T09:18:24.633 INFO:journalctl@ceph.osd.9.smithi042.stdout:Aug 30 09:18:24 smithi042 podman[101445]: 2024-08-30 09:18:24.443960863 +0000 UTC m=+0.524955028 container died 95f7d7f1b42051b5ee4b509df40f47081c4f102caa93dd8428646a2fa35fdfb9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-9-deactivate, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.37.1, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-08-30T09:18:25.057 DEBUG:teuthology.orchestra.run.smithi042:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968@osd.9.service' 2024-08-30T09:18:25.099 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-30T09:18:25.099 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-08-30T09:18:25.099 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-08-30T09:18:25.099 DEBUG:teuthology.orchestra.run.smithi167:> sudo systemctl stop ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968@osd.10 2024-08-30T09:18:25.146 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:18:24 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:18:24.878+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:03.239730+0000 front 2024-08-30T09:17:03.239661+0000 (oldest deadline 2024-08-30T09:17:29.139608+0000) 2024-08-30T09:18:25.147 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:18:24 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:18:24.878+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:12.040451+0000 front 2024-08-30T09:17:12.040473+0000 (oldest deadline 2024-08-30T09:17:37.841115+0000) 2024-08-30T09:18:25.147 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:18:24 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:18:24.878+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-30T09:17:22.442437+0000 front 2024-08-30T09:17:22.442414+0000 (oldest deadline 2024-08-30T09:17:48.243149+0000) 2024-08-30T09:18:25.147 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:18:24 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:18:24.878+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-08-30T09:17:32.944470+0000 front 2024-08-30T09:17:32.944528+0000 (oldest deadline 2024-08-30T09:17:55.244278+0000) 2024-08-30T09:18:25.147 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:18:24 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:18:24.878+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.167:6812 osd.4 since back 2024-08-30T09:17:38.546318+0000 front 2024-08-30T09:17:38.546353+0000 (oldest deadline 2024-08-30T09:18:03.246447+0000) 2024-08-30T09:18:25.147 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:18:24 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:18:24.878+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-08-30T09:17:49.648107+0000 front 2024-08-30T09:17:49.648282+0000 (oldest deadline 2024-08-30T09:18:12.547941+0000) 2024-08-30T09:18:25.147 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:18:24 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:18:24.878+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.42:6822 osd.6 since back 2024-08-30T09:17:56.049071+0000 front 2024-08-30T09:17:56.049221+0000 (oldest deadline 2024-08-30T09:18:20.148952+0000) 2024-08-30T09:18:25.186 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:24 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:24.734+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:04.091032+0000 front 2024-08-30T09:17:04.090868+0000 (oldest deadline 2024-08-30T09:17:29.391066+0000) 2024-08-30T09:18:25.186 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:24 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:24.734+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:12.193073+0000 front 2024-08-30T09:17:12.192820+0000 (oldest deadline 2024-08-30T09:17:37.492761+0000) 2024-08-30T09:18:25.186 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:24 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:24.734+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-30T09:17:24.494221+0000 front 2024-08-30T09:17:24.494455+0000 (oldest deadline 2024-08-30T09:17:46.194420+0000) 2024-08-30T09:18:25.187 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:24 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:24.734+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.167:6812 osd.4 since back 2024-08-30T09:17:38.897956+0000 front 2024-08-30T09:17:38.897901+0000 (oldest deadline 2024-08-30T09:18:04.698743+0000) 2024-08-30T09:18:25.187 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:24 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:24.734+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-08-30T09:17:46.399666+0000 front 2024-08-30T09:17:46.399767+0000 (oldest deadline 2024-08-30T09:18:12.800338+0000) 2024-08-30T09:18:25.187 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:24 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:24.734+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.42:6822 osd.6 since back 2024-08-30T09:17:56.801860+0000 front 2024-08-30T09:17:56.801820+0000 (oldest deadline 2024-08-30T09:18:19.701830+0000) 2024-08-30T09:18:25.427 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:18:25 smithi167 systemd[1]: Stopping Ceph osd.10 for b0895c1a-66ac-11ef-bcd4-c7b262605968... 2024-08-30T09:18:25.827 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:18:25 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:18:25.424+0000 7f2d0463e640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-30T09:18:25.827 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:18:25 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:18:25.424+0000 7f2d0463e640 -1 osd.10 87 *** Got signal Terminated *** 2024-08-30T09:18:25.827 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:18:25 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:18:25.424+0000 7f2d0463e640 -1 osd.10 87 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-08-30T09:18:26.186 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:25 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:25.723+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:04.091032+0000 front 2024-08-30T09:17:04.090868+0000 (oldest deadline 2024-08-30T09:17:29.391066+0000) 2024-08-30T09:18:26.186 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:25 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:25.723+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:12.193073+0000 front 2024-08-30T09:17:12.192820+0000 (oldest deadline 2024-08-30T09:17:37.492761+0000) 2024-08-30T09:18:26.186 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:25 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:25.723+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-30T09:17:24.494221+0000 front 2024-08-30T09:17:24.494455+0000 (oldest deadline 2024-08-30T09:17:46.194420+0000) 2024-08-30T09:18:26.187 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:25 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:25.723+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.167:6812 osd.4 since back 2024-08-30T09:17:38.897956+0000 front 2024-08-30T09:17:38.897901+0000 (oldest deadline 2024-08-30T09:18:04.698743+0000) 2024-08-30T09:18:26.187 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:25 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:25.723+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-08-30T09:17:46.399666+0000 front 2024-08-30T09:17:46.399767+0000 (oldest deadline 2024-08-30T09:18:12.800338+0000) 2024-08-30T09:18:26.187 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:25 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:25.723+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.42:6822 osd.6 since back 2024-08-30T09:17:56.801860+0000 front 2024-08-30T09:17:56.801820+0000 (oldest deadline 2024-08-30T09:18:19.701830+0000) 2024-08-30T09:18:26.327 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:18:25 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:18:25.833+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:03.239730+0000 front 2024-08-30T09:17:03.239661+0000 (oldest deadline 2024-08-30T09:17:29.139608+0000) 2024-08-30T09:18:26.327 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:18:25 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:18:25.833+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:12.040451+0000 front 2024-08-30T09:17:12.040473+0000 (oldest deadline 2024-08-30T09:17:37.841115+0000) 2024-08-30T09:18:26.327 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:18:25 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:18:25.833+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-30T09:17:22.442437+0000 front 2024-08-30T09:17:22.442414+0000 (oldest deadline 2024-08-30T09:17:48.243149+0000) 2024-08-30T09:18:26.327 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:18:25 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:18:25.833+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-08-30T09:17:32.944470+0000 front 2024-08-30T09:17:32.944528+0000 (oldest deadline 2024-08-30T09:17:55.244278+0000) 2024-08-30T09:18:26.327 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:18:25 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:18:25.833+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.167:6812 osd.4 since back 2024-08-30T09:17:38.546318+0000 front 2024-08-30T09:17:38.546353+0000 (oldest deadline 2024-08-30T09:18:03.246447+0000) 2024-08-30T09:18:26.327 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:18:25 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:18:25.833+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-08-30T09:17:49.648107+0000 front 2024-08-30T09:17:49.648282+0000 (oldest deadline 2024-08-30T09:18:12.547941+0000) 2024-08-30T09:18:26.327 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:18:25 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:18:25.833+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.42:6822 osd.6 since back 2024-08-30T09:17:56.049071+0000 front 2024-08-30T09:17:56.049221+0000 (oldest deadline 2024-08-30T09:18:20.148952+0000) 2024-08-30T09:18:27.186 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:26 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:26.709+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:04.091032+0000 front 2024-08-30T09:17:04.090868+0000 (oldest deadline 2024-08-30T09:17:29.391066+0000) 2024-08-30T09:18:27.186 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:26 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:26.709+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:12.193073+0000 front 2024-08-30T09:17:12.192820+0000 (oldest deadline 2024-08-30T09:17:37.492761+0000) 2024-08-30T09:18:27.186 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:26 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:26.709+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-30T09:17:24.494221+0000 front 2024-08-30T09:17:24.494455+0000 (oldest deadline 2024-08-30T09:17:46.194420+0000) 2024-08-30T09:18:27.187 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:26 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:26.709+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.167:6812 osd.4 since back 2024-08-30T09:17:38.897956+0000 front 2024-08-30T09:17:38.897901+0000 (oldest deadline 2024-08-30T09:18:04.698743+0000) 2024-08-30T09:18:27.187 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:26 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:26.709+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-08-30T09:17:46.399666+0000 front 2024-08-30T09:17:46.399767+0000 (oldest deadline 2024-08-30T09:18:12.800338+0000) 2024-08-30T09:18:27.187 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:26 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:26.709+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.42:6822 osd.6 since back 2024-08-30T09:17:56.801860+0000 front 2024-08-30T09:17:56.801820+0000 (oldest deadline 2024-08-30T09:18:19.701830+0000) 2024-08-30T09:18:27.327 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:18:26 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:18:26.842+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:03.239730+0000 front 2024-08-30T09:17:03.239661+0000 (oldest deadline 2024-08-30T09:17:29.139608+0000) 2024-08-30T09:18:27.327 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:18:26 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:18:26.842+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:12.040451+0000 front 2024-08-30T09:17:12.040473+0000 (oldest deadline 2024-08-30T09:17:37.841115+0000) 2024-08-30T09:18:27.327 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:18:26 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:18:26.842+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-30T09:17:22.442437+0000 front 2024-08-30T09:17:22.442414+0000 (oldest deadline 2024-08-30T09:17:48.243149+0000) 2024-08-30T09:18:27.327 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:18:26 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:18:26.842+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-08-30T09:17:32.944470+0000 front 2024-08-30T09:17:32.944528+0000 (oldest deadline 2024-08-30T09:17:55.244278+0000) 2024-08-30T09:18:27.328 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:18:26 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:18:26.842+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.167:6812 osd.4 since back 2024-08-30T09:17:38.546318+0000 front 2024-08-30T09:17:38.546353+0000 (oldest deadline 2024-08-30T09:18:03.246447+0000) 2024-08-30T09:18:27.328 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:18:26 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:18:26.842+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-08-30T09:17:49.648107+0000 front 2024-08-30T09:17:49.648282+0000 (oldest deadline 2024-08-30T09:18:12.547941+0000) 2024-08-30T09:18:27.328 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:18:26 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:18:26.842+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.42:6822 osd.6 since back 2024-08-30T09:17:56.049071+0000 front 2024-08-30T09:17:56.049221+0000 (oldest deadline 2024-08-30T09:18:20.148952+0000) 2024-08-30T09:18:28.186 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:27 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:27.738+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:04.091032+0000 front 2024-08-30T09:17:04.090868+0000 (oldest deadline 2024-08-30T09:17:29.391066+0000) 2024-08-30T09:18:28.186 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:27 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:27.738+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:12.193073+0000 front 2024-08-30T09:17:12.192820+0000 (oldest deadline 2024-08-30T09:17:37.492761+0000) 2024-08-30T09:18:28.187 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:27 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:27.738+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-30T09:17:24.494221+0000 front 2024-08-30T09:17:24.494455+0000 (oldest deadline 2024-08-30T09:17:46.194420+0000) 2024-08-30T09:18:28.187 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:27 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:27.738+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.167:6812 osd.4 since back 2024-08-30T09:17:38.897956+0000 front 2024-08-30T09:17:38.897901+0000 (oldest deadline 2024-08-30T09:18:04.698743+0000) 2024-08-30T09:18:28.187 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:27 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:27.738+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-08-30T09:17:46.399666+0000 front 2024-08-30T09:17:46.399767+0000 (oldest deadline 2024-08-30T09:18:12.800338+0000) 2024-08-30T09:18:28.187 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:27 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:27.738+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.42:6822 osd.6 since back 2024-08-30T09:17:56.801860+0000 front 2024-08-30T09:17:56.801820+0000 (oldest deadline 2024-08-30T09:18:19.701830+0000) 2024-08-30T09:18:28.327 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:18:27 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:18:27.827+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:03.239730+0000 front 2024-08-30T09:17:03.239661+0000 (oldest deadline 2024-08-30T09:17:29.139608+0000) 2024-08-30T09:18:28.327 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:18:27 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:18:27.827+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:12.040451+0000 front 2024-08-30T09:17:12.040473+0000 (oldest deadline 2024-08-30T09:17:37.841115+0000) 2024-08-30T09:18:28.327 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:18:27 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:18:27.827+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-30T09:17:22.442437+0000 front 2024-08-30T09:17:22.442414+0000 (oldest deadline 2024-08-30T09:17:48.243149+0000) 2024-08-30T09:18:28.327 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:18:27 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:18:27.827+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-08-30T09:17:32.944470+0000 front 2024-08-30T09:17:32.944528+0000 (oldest deadline 2024-08-30T09:17:55.244278+0000) 2024-08-30T09:18:28.327 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:18:27 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:18:27.827+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.167:6812 osd.4 since back 2024-08-30T09:17:38.546318+0000 front 2024-08-30T09:17:38.546353+0000 (oldest deadline 2024-08-30T09:18:03.246447+0000) 2024-08-30T09:18:28.328 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:18:27 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:18:27.827+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-08-30T09:17:49.648107+0000 front 2024-08-30T09:17:49.648282+0000 (oldest deadline 2024-08-30T09:18:12.547941+0000) 2024-08-30T09:18:28.328 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:18:27 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:18:27.827+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.42:6822 osd.6 since back 2024-08-30T09:17:56.049071+0000 front 2024-08-30T09:17:56.049221+0000 (oldest deadline 2024-08-30T09:18:20.148952+0000) 2024-08-30T09:18:29.077 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:18:28 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:18:28.786+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:03.239730+0000 front 2024-08-30T09:17:03.239661+0000 (oldest deadline 2024-08-30T09:17:29.139608+0000) 2024-08-30T09:18:29.077 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:18:28 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:18:28.786+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:12.040451+0000 front 2024-08-30T09:17:12.040473+0000 (oldest deadline 2024-08-30T09:17:37.841115+0000) 2024-08-30T09:18:29.077 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:18:28 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:18:28.786+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-30T09:17:22.442437+0000 front 2024-08-30T09:17:22.442414+0000 (oldest deadline 2024-08-30T09:17:48.243149+0000) 2024-08-30T09:18:29.077 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:18:28 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:18:28.786+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-08-30T09:17:32.944470+0000 front 2024-08-30T09:17:32.944528+0000 (oldest deadline 2024-08-30T09:17:55.244278+0000) 2024-08-30T09:18:29.077 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:18:28 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:18:28.786+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.167:6812 osd.4 since back 2024-08-30T09:17:38.546318+0000 front 2024-08-30T09:17:38.546353+0000 (oldest deadline 2024-08-30T09:18:03.246447+0000) 2024-08-30T09:18:29.077 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:18:28 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:18:28.786+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-08-30T09:17:49.648107+0000 front 2024-08-30T09:17:49.648282+0000 (oldest deadline 2024-08-30T09:18:12.547941+0000) 2024-08-30T09:18:29.078 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:18:28 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:18:28.786+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.42:6822 osd.6 since back 2024-08-30T09:17:56.049071+0000 front 2024-08-30T09:17:56.049221+0000 (oldest deadline 2024-08-30T09:18:20.148952+0000) 2024-08-30T09:18:29.186 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:28 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:28.712+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:04.091032+0000 front 2024-08-30T09:17:04.090868+0000 (oldest deadline 2024-08-30T09:17:29.391066+0000) 2024-08-30T09:18:29.186 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:28 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:28.712+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:12.193073+0000 front 2024-08-30T09:17:12.192820+0000 (oldest deadline 2024-08-30T09:17:37.492761+0000) 2024-08-30T09:18:29.186 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:28 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:28.712+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-30T09:17:24.494221+0000 front 2024-08-30T09:17:24.494455+0000 (oldest deadline 2024-08-30T09:17:46.194420+0000) 2024-08-30T09:18:29.186 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:28 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:28.712+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.167:6812 osd.4 since back 2024-08-30T09:17:38.897956+0000 front 2024-08-30T09:17:38.897901+0000 (oldest deadline 2024-08-30T09:18:04.698743+0000) 2024-08-30T09:18:29.186 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:28 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:28.712+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-08-30T09:17:46.399666+0000 front 2024-08-30T09:17:46.399767+0000 (oldest deadline 2024-08-30T09:18:12.800338+0000) 2024-08-30T09:18:29.186 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:28 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:28.712+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.42:6822 osd.6 since back 2024-08-30T09:17:56.801860+0000 front 2024-08-30T09:17:56.801820+0000 (oldest deadline 2024-08-30T09:18:19.701830+0000) 2024-08-30T09:18:30.077 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:18:29 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:18:29.784+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:03.239730+0000 front 2024-08-30T09:17:03.239661+0000 (oldest deadline 2024-08-30T09:17:29.139608+0000) 2024-08-30T09:18:30.077 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:18:29 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:18:29.784+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:12.040451+0000 front 2024-08-30T09:17:12.040473+0000 (oldest deadline 2024-08-30T09:17:37.841115+0000) 2024-08-30T09:18:30.077 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:18:29 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:18:29.784+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-30T09:17:22.442437+0000 front 2024-08-30T09:17:22.442414+0000 (oldest deadline 2024-08-30T09:17:48.243149+0000) 2024-08-30T09:18:30.077 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:18:29 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:18:29.784+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-08-30T09:17:32.944470+0000 front 2024-08-30T09:17:32.944528+0000 (oldest deadline 2024-08-30T09:17:55.244278+0000) 2024-08-30T09:18:30.077 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:18:29 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:18:29.784+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.167:6812 osd.4 since back 2024-08-30T09:17:38.546318+0000 front 2024-08-30T09:17:38.546353+0000 (oldest deadline 2024-08-30T09:18:03.246447+0000) 2024-08-30T09:18:30.078 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:18:29 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:18:29.784+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-08-30T09:17:49.648107+0000 front 2024-08-30T09:17:49.648282+0000 (oldest deadline 2024-08-30T09:18:12.547941+0000) 2024-08-30T09:18:30.078 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:18:29 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10[47281]: 2024-08-30T09:18:29.784+0000 7f2d00c29640 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.42:6822 osd.6 since back 2024-08-30T09:17:56.049071+0000 front 2024-08-30T09:17:56.049221+0000 (oldest deadline 2024-08-30T09:18:20.148952+0000) 2024-08-30T09:18:30.186 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:29 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:29.714+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:04.091032+0000 front 2024-08-30T09:17:04.090868+0000 (oldest deadline 2024-08-30T09:17:29.391066+0000) 2024-08-30T09:18:30.186 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:29 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:29.714+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:12.193073+0000 front 2024-08-30T09:17:12.192820+0000 (oldest deadline 2024-08-30T09:17:37.492761+0000) 2024-08-30T09:18:30.186 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:29 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:29.714+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-30T09:17:24.494221+0000 front 2024-08-30T09:17:24.494455+0000 (oldest deadline 2024-08-30T09:17:46.194420+0000) 2024-08-30T09:18:30.186 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:29 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:29.714+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.167:6812 osd.4 since back 2024-08-30T09:17:38.897956+0000 front 2024-08-30T09:17:38.897901+0000 (oldest deadline 2024-08-30T09:18:04.698743+0000) 2024-08-30T09:18:30.186 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:29 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:29.714+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-08-30T09:17:46.399666+0000 front 2024-08-30T09:17:46.399767+0000 (oldest deadline 2024-08-30T09:18:12.800338+0000) 2024-08-30T09:18:30.187 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:29 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:29.714+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.42:6822 osd.6 since back 2024-08-30T09:17:56.801860+0000 front 2024-08-30T09:17:56.801820+0000 (oldest deadline 2024-08-30T09:18:19.701830+0000) 2024-08-30T09:18:30.936 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:30 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:30.683+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:04.091032+0000 front 2024-08-30T09:17:04.090868+0000 (oldest deadline 2024-08-30T09:17:29.391066+0000) 2024-08-30T09:18:30.936 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:30 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:30.683+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:12.193073+0000 front 2024-08-30T09:17:12.192820+0000 (oldest deadline 2024-08-30T09:17:37.492761+0000) 2024-08-30T09:18:30.936 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:30 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:30.683+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-30T09:17:24.494221+0000 front 2024-08-30T09:17:24.494455+0000 (oldest deadline 2024-08-30T09:17:46.194420+0000) 2024-08-30T09:18:30.936 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:30 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:30.683+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.167:6812 osd.4 since back 2024-08-30T09:17:38.897956+0000 front 2024-08-30T09:17:38.897901+0000 (oldest deadline 2024-08-30T09:18:04.698743+0000) 2024-08-30T09:18:30.937 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:30 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:30.683+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-08-30T09:17:46.399666+0000 front 2024-08-30T09:17:46.399767+0000 (oldest deadline 2024-08-30T09:18:12.800338+0000) 2024-08-30T09:18:30.937 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:30 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:30.683+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.42:6822 osd.6 since back 2024-08-30T09:17:56.801860+0000 front 2024-08-30T09:17:56.801820+0000 (oldest deadline 2024-08-30T09:18:19.701830+0000) 2024-08-30T09:18:31.302 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:18:31 smithi167 podman[58547]: 2024-08-30 09:18:31.032808385 +0000 UTC m=+5.740972868 container died 192dcc3ee59029e6f4af31d29d822254d29d015d3b9438a3d945f414b391e82e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.1, maintainer=Guillaume Abrioux ) 2024-08-30T09:18:31.577 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:18:31 smithi167 podman[58547]: 2024-08-30 09:18:31.391828137 +0000 UTC m=+6.099992609 container cleanup 192dcc3ee59029e6f4af31d29d822254d29d015d3b9438a3d945f414b391e82e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.37.1, org.label-schema.schema-version=1.0) 2024-08-30T09:18:31.577 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:18:31 smithi167 bash[58547]: ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10 2024-08-30T09:18:31.936 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:31 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:31.661+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:04.091032+0000 front 2024-08-30T09:17:04.090868+0000 (oldest deadline 2024-08-30T09:17:29.391066+0000) 2024-08-30T09:18:31.936 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:31 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:31.661+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:12.193073+0000 front 2024-08-30T09:17:12.192820+0000 (oldest deadline 2024-08-30T09:17:37.492761+0000) 2024-08-30T09:18:31.936 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:31 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:31.661+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-30T09:17:24.494221+0000 front 2024-08-30T09:17:24.494455+0000 (oldest deadline 2024-08-30T09:17:46.194420+0000) 2024-08-30T09:18:31.936 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:31 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:31.661+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.167:6812 osd.4 since back 2024-08-30T09:17:38.897956+0000 front 2024-08-30T09:17:38.897901+0000 (oldest deadline 2024-08-30T09:18:04.698743+0000) 2024-08-30T09:18:31.936 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:31 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:31.661+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-08-30T09:17:46.399666+0000 front 2024-08-30T09:17:46.399767+0000 (oldest deadline 2024-08-30T09:18:12.800338+0000) 2024-08-30T09:18:31.937 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:31 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:31.661+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.42:6822 osd.6 since back 2024-08-30T09:17:56.801860+0000 front 2024-08-30T09:17:56.801820+0000 (oldest deadline 2024-08-30T09:18:19.701830+0000) 2024-08-30T09:18:31.937 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:31 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:31.661+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.167:6820 osd.7 since back 2024-08-30T09:18:01.903438+0000 front 2024-08-30T09:18:01.903422+0000 (oldest deadline 2024-08-30T09:18:31.303901+0000) 2024-08-30T09:18:31.997 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:18:31 smithi167 podman[58564]: 2024-08-30 09:18:31.684856296 +0000 UTC m=+0.648142714 container remove 192dcc3ee59029e6f4af31d29d822254d29d015d3b9438a3d945f414b391e82e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.1, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, ceph=True) 2024-08-30T09:18:32.306 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:18:31 smithi167 podman[58652]: 2024-08-30 09:18:31.898868422 +0000 UTC m=+0.015720050 image pull 79f4cde5e36911a5dca995eeb98098c2c8aae81dd52a088ef2c867bc3363bad0 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61 2024-08-30T09:18:32.306 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:18:32 smithi167 podman[58652]: 2024-08-30 09:18:32.002263632 +0000 UTC m=+0.119115256 container create f4486af56e717ae57343b29d3f85fffe75b2908847cc4e2cb4e32ae946f481d6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10-deactivate, GIT_BRANCH=HEAD, io.buildah.version=1.37.1, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-08-30T09:18:32.306 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:18:32 smithi167 podman[58652]: 2024-08-30 09:18:32.213027156 +0000 UTC m=+0.329878780 container init f4486af56e717ae57343b29d3f85fffe75b2908847cc4e2cb4e32ae946f481d6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10-deactivate, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, io.buildah.version=1.37.1, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611) 2024-08-30T09:18:32.306 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:18:32 smithi167 podman[58652]: 2024-08-30 09:18:32.216420111 +0000 UTC m=+0.333271735 container start f4486af56e717ae57343b29d3f85fffe75b2908847cc4e2cb4e32ae946f481d6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10-deactivate, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.1) 2024-08-30T09:18:32.306 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:18:32 smithi167 podman[58652]: 2024-08-30 09:18:32.304707646 +0000 UTC m=+0.421559272 container attach f4486af56e717ae57343b29d3f85fffe75b2908847cc4e2cb4e32ae946f481d6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10-deactivate, org.label-schema.license=GPLv2, io.buildah.version=1.37.1, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS) 2024-08-30T09:18:32.577 INFO:journalctl@ceph.osd.10.smithi167.stdout:Aug 30 09:18:32 smithi167 podman[58652]: 2024-08-30 09:18:32.408196824 +0000 UTC m=+0.525048448 container died f4486af56e717ae57343b29d3f85fffe75b2908847cc4e2cb4e32ae946f481d6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-10-deactivate, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, io.buildah.version=1.37.1, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-08-30T09:18:32.936 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:32 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:32.664+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:04.091032+0000 front 2024-08-30T09:17:04.090868+0000 (oldest deadline 2024-08-30T09:17:29.391066+0000) 2024-08-30T09:18:32.936 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:32 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:32.664+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:12.193073+0000 front 2024-08-30T09:17:12.192820+0000 (oldest deadline 2024-08-30T09:17:37.492761+0000) 2024-08-30T09:18:32.936 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:32 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:32.664+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-30T09:17:24.494221+0000 front 2024-08-30T09:17:24.494455+0000 (oldest deadline 2024-08-30T09:17:46.194420+0000) 2024-08-30T09:18:32.936 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:32 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:32.664+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.167:6812 osd.4 since back 2024-08-30T09:17:38.897956+0000 front 2024-08-30T09:17:38.897901+0000 (oldest deadline 2024-08-30T09:18:04.698743+0000) 2024-08-30T09:18:32.937 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:32 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:32.664+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-08-30T09:17:46.399666+0000 front 2024-08-30T09:17:46.399767+0000 (oldest deadline 2024-08-30T09:18:12.800338+0000) 2024-08-30T09:18:32.937 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:32 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:32.664+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.42:6822 osd.6 since back 2024-08-30T09:17:56.801860+0000 front 2024-08-30T09:17:56.801820+0000 (oldest deadline 2024-08-30T09:18:19.701830+0000) 2024-08-30T09:18:32.937 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:32 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:32.664+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.167:6820 osd.7 since back 2024-08-30T09:18:01.903438+0000 front 2024-08-30T09:18:01.903422+0000 (oldest deadline 2024-08-30T09:18:31.303901+0000) 2024-08-30T09:18:32.972 DEBUG:teuthology.orchestra.run.smithi167:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968@osd.10.service' 2024-08-30T09:18:33.009 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-30T09:18:33.009 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-08-30T09:18:33.010 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-08-30T09:18:33.010 DEBUG:teuthology.orchestra.run.smithi190:> sudo systemctl stop ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968@osd.11 2024-08-30T09:18:33.330 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:33 smithi190 systemd[1]: Stopping Ceph osd.11 for b0895c1a-66ac-11ef-bcd4-c7b262605968... 2024-08-30T09:18:33.682 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:33 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:33.328+0000 7f0273efc640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-30T09:18:33.682 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:33 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:33.328+0000 7f0273efc640 -1 osd.11 87 *** Got signal Terminated *** 2024-08-30T09:18:33.682 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:33 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:33.328+0000 7f0273efc640 -1 osd.11 87 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-08-30T09:18:33.936 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:33 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:33.680+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:04.091032+0000 front 2024-08-30T09:17:04.090868+0000 (oldest deadline 2024-08-30T09:17:29.391066+0000) 2024-08-30T09:18:33.936 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:33 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:33.680+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:12.193073+0000 front 2024-08-30T09:17:12.192820+0000 (oldest deadline 2024-08-30T09:17:37.492761+0000) 2024-08-30T09:18:33.937 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:33 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:33.680+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-30T09:17:24.494221+0000 front 2024-08-30T09:17:24.494455+0000 (oldest deadline 2024-08-30T09:17:46.194420+0000) 2024-08-30T09:18:33.937 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:33 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:33.680+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.167:6812 osd.4 since back 2024-08-30T09:17:38.897956+0000 front 2024-08-30T09:17:38.897901+0000 (oldest deadline 2024-08-30T09:18:04.698743+0000) 2024-08-30T09:18:33.937 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:33 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:33.680+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-08-30T09:17:46.399666+0000 front 2024-08-30T09:17:46.399767+0000 (oldest deadline 2024-08-30T09:18:12.800338+0000) 2024-08-30T09:18:33.937 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:33 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:33.680+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.42:6822 osd.6 since back 2024-08-30T09:17:56.801860+0000 front 2024-08-30T09:17:56.801820+0000 (oldest deadline 2024-08-30T09:18:19.701830+0000) 2024-08-30T09:18:33.937 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:33 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:33.680+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.167:6820 osd.7 since back 2024-08-30T09:18:01.903438+0000 front 2024-08-30T09:18:01.903422+0000 (oldest deadline 2024-08-30T09:18:31.303901+0000) 2024-08-30T09:18:34.936 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:34 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:34.643+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:04.091032+0000 front 2024-08-30T09:17:04.090868+0000 (oldest deadline 2024-08-30T09:17:29.391066+0000) 2024-08-30T09:18:34.936 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:34 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:34.643+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:12.193073+0000 front 2024-08-30T09:17:12.192820+0000 (oldest deadline 2024-08-30T09:17:37.492761+0000) 2024-08-30T09:18:34.936 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:34 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:34.643+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-30T09:17:24.494221+0000 front 2024-08-30T09:17:24.494455+0000 (oldest deadline 2024-08-30T09:17:46.194420+0000) 2024-08-30T09:18:34.936 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:34 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:34.643+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.167:6812 osd.4 since back 2024-08-30T09:17:38.897956+0000 front 2024-08-30T09:17:38.897901+0000 (oldest deadline 2024-08-30T09:18:04.698743+0000) 2024-08-30T09:18:34.937 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:34 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:34.643+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-08-30T09:17:46.399666+0000 front 2024-08-30T09:17:46.399767+0000 (oldest deadline 2024-08-30T09:18:12.800338+0000) 2024-08-30T09:18:34.937 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:34 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:34.643+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.42:6822 osd.6 since back 2024-08-30T09:17:56.801860+0000 front 2024-08-30T09:17:56.801820+0000 (oldest deadline 2024-08-30T09:18:19.701830+0000) 2024-08-30T09:18:34.937 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:34 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:34.643+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.167:6820 osd.7 since back 2024-08-30T09:18:01.903438+0000 front 2024-08-30T09:18:01.903422+0000 (oldest deadline 2024-08-30T09:18:31.303901+0000) 2024-08-30T09:18:35.936 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:35 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:35.662+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:04.091032+0000 front 2024-08-30T09:17:04.090868+0000 (oldest deadline 2024-08-30T09:17:29.391066+0000) 2024-08-30T09:18:35.937 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:35 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:35.662+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:12.193073+0000 front 2024-08-30T09:17:12.192820+0000 (oldest deadline 2024-08-30T09:17:37.492761+0000) 2024-08-30T09:18:35.937 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:35 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:35.662+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-30T09:17:24.494221+0000 front 2024-08-30T09:17:24.494455+0000 (oldest deadline 2024-08-30T09:17:46.194420+0000) 2024-08-30T09:18:35.937 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:35 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:35.662+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.167:6812 osd.4 since back 2024-08-30T09:17:38.897956+0000 front 2024-08-30T09:17:38.897901+0000 (oldest deadline 2024-08-30T09:18:04.698743+0000) 2024-08-30T09:18:35.937 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:35 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:35.662+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-08-30T09:17:46.399666+0000 front 2024-08-30T09:17:46.399767+0000 (oldest deadline 2024-08-30T09:18:12.800338+0000) 2024-08-30T09:18:35.937 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:35 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:35.662+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.42:6822 osd.6 since back 2024-08-30T09:17:56.801860+0000 front 2024-08-30T09:17:56.801820+0000 (oldest deadline 2024-08-30T09:18:19.701830+0000) 2024-08-30T09:18:35.937 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:35 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:35.662+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.167:6820 osd.7 since back 2024-08-30T09:18:01.903438+0000 front 2024-08-30T09:18:01.903422+0000 (oldest deadline 2024-08-30T09:18:31.303901+0000) 2024-08-30T09:18:35.937 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:35 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:35.662+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.190:6820 osd.8 since back 2024-08-30T09:18:13.604861+0000 front 2024-08-30T09:18:13.604826+0000 (oldest deadline 2024-08-30T09:18:35.304970+0000) 2024-08-30T09:18:36.936 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:36 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:36.620+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:04.091032+0000 front 2024-08-30T09:17:04.090868+0000 (oldest deadline 2024-08-30T09:17:29.391066+0000) 2024-08-30T09:18:36.936 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:36 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:36.620+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:12.193073+0000 front 2024-08-30T09:17:12.192820+0000 (oldest deadline 2024-08-30T09:17:37.492761+0000) 2024-08-30T09:18:36.937 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:36 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:36.620+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-30T09:17:24.494221+0000 front 2024-08-30T09:17:24.494455+0000 (oldest deadline 2024-08-30T09:17:46.194420+0000) 2024-08-30T09:18:36.937 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:36 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:36.620+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.167:6812 osd.4 since back 2024-08-30T09:17:38.897956+0000 front 2024-08-30T09:17:38.897901+0000 (oldest deadline 2024-08-30T09:18:04.698743+0000) 2024-08-30T09:18:36.937 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:36 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:36.620+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-08-30T09:17:46.399666+0000 front 2024-08-30T09:17:46.399767+0000 (oldest deadline 2024-08-30T09:18:12.800338+0000) 2024-08-30T09:18:36.937 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:36 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:36.620+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.42:6822 osd.6 since back 2024-08-30T09:17:56.801860+0000 front 2024-08-30T09:17:56.801820+0000 (oldest deadline 2024-08-30T09:18:19.701830+0000) 2024-08-30T09:18:36.937 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:36 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:36.620+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.167:6820 osd.7 since back 2024-08-30T09:18:01.903438+0000 front 2024-08-30T09:18:01.903422+0000 (oldest deadline 2024-08-30T09:18:31.303901+0000) 2024-08-30T09:18:36.937 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:36 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:36.620+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.190:6820 osd.8 since back 2024-08-30T09:18:13.604861+0000 front 2024-08-30T09:18:13.604826+0000 (oldest deadline 2024-08-30T09:18:35.304970+0000) 2024-08-30T09:18:37.936 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:37 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:37.656+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-08-30T09:17:04.091032+0000 front 2024-08-30T09:17:04.090868+0000 (oldest deadline 2024-08-30T09:17:29.391066+0000) 2024-08-30T09:18:37.937 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:37 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:37.656+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.167:6804 osd.1 since back 2024-08-30T09:17:12.193073+0000 front 2024-08-30T09:17:12.192820+0000 (oldest deadline 2024-08-30T09:17:37.492761+0000) 2024-08-30T09:18:37.937 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:37 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:37.656+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-30T09:17:24.494221+0000 front 2024-08-30T09:17:24.494455+0000 (oldest deadline 2024-08-30T09:17:46.194420+0000) 2024-08-30T09:18:37.937 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:37 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:37.656+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.167:6812 osd.4 since back 2024-08-30T09:17:38.897956+0000 front 2024-08-30T09:17:38.897901+0000 (oldest deadline 2024-08-30T09:18:04.698743+0000) 2024-08-30T09:18:37.937 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:37 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:37.656+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-08-30T09:17:46.399666+0000 front 2024-08-30T09:17:46.399767+0000 (oldest deadline 2024-08-30T09:18:12.800338+0000) 2024-08-30T09:18:37.937 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:37 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:37.656+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.42:6822 osd.6 since back 2024-08-30T09:17:56.801860+0000 front 2024-08-30T09:17:56.801820+0000 (oldest deadline 2024-08-30T09:18:19.701830+0000) 2024-08-30T09:18:37.937 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:37 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:37.656+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.167:6820 osd.7 since back 2024-08-30T09:18:01.903438+0000 front 2024-08-30T09:18:01.903422+0000 (oldest deadline 2024-08-30T09:18:31.303901+0000) 2024-08-30T09:18:37.937 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:37 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11[47071]: 2024-08-30T09:18:37.656+0000 7f02704e7640 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.190:6820 osd.8 since back 2024-08-30T09:18:13.604861+0000 front 2024-08-30T09:18:13.604826+0000 (oldest deadline 2024-08-30T09:18:35.304970+0000) 2024-08-30T09:18:39.186 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:38 smithi190 podman[59165]: 2024-08-30 09:18:38.929688707 +0000 UTC m=+5.716856094 container died d39072212a6f1a74f5e5f5c2b2213c8252a97f94e360b3cee574b103ac41c14b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11, maintainer=Guillaume Abrioux , RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, io.buildah.version=1.37.1, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-08-30T09:18:39.580 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:39 smithi190 podman[59165]: 2024-08-30 09:18:39.27827081 +0000 UTC m=+6.065438188 container cleanup d39072212a6f1a74f5e5f5c2b2213c8252a97f94e360b3cee574b103ac41c14b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.37.1) 2024-08-30T09:18:39.581 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:39 smithi190 bash[59165]: ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11 2024-08-30T09:18:39.933 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:39 smithi190 podman[59181]: 2024-08-30 09:18:39.579393601 +0000 UTC m=+0.644507532 container remove d39072212a6f1a74f5e5f5c2b2213c8252a97f94e360b3cee574b103ac41c14b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.37.1, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611) 2024-08-30T09:18:40.186 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:39 smithi190 podman[59269]: 2024-08-30 09:18:39.834497401 +0000 UTC m=+0.015607964 image pull 79f4cde5e36911a5dca995eeb98098c2c8aae81dd52a088ef2c867bc3363bad0 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61 2024-08-30T09:18:40.186 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:39 smithi190 podman[59269]: 2024-08-30 09:18:39.946918891 +0000 UTC m=+0.128029446 container create aa5fc8cc384ddf0d2a6d286346a2e8fa1bd067eb767c65f8dc678081b8ba485d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11-deactivate, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.1, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-08-30T09:18:40.187 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:40 smithi190 podman[59269]: 2024-08-30 09:18:40.174013048 +0000 UTC m=+0.355123600 container init aa5fc8cc384ddf0d2a6d286346a2e8fa1bd067eb767c65f8dc678081b8ba485d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11-deactivate, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.1, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-08-30T09:18:40.187 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:40 smithi190 podman[59269]: 2024-08-30 09:18:40.177371379 +0000 UTC m=+0.358481937 container start aa5fc8cc384ddf0d2a6d286346a2e8fa1bd067eb767c65f8dc678081b8ba485d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11-deactivate, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.37.1) 2024-08-30T09:18:40.661 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:40 smithi190 podman[59269]: 2024-08-30 09:18:40.274105775 +0000 UTC m=+0.455216342 container attach aa5fc8cc384ddf0d2a6d286346a2e8fa1bd067eb767c65f8dc678081b8ba485d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11-deactivate, org.label-schema.build-date=20240731, ceph=True, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, CEPH_POINT_RELEASE=, io.buildah.version=1.37.1, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-08-30T09:18:40.661 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:40 smithi190 podman[59269]: 2024-08-30 09:18:40.375079151 +0000 UTC m=+0.556189712 container died aa5fc8cc384ddf0d2a6d286346a2e8fa1bd067eb767c65f8dc678081b8ba485d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11-deactivate, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, io.buildah.version=1.37.1, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image) 2024-08-30T09:18:40.912 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 30 09:18:40 smithi190 podman[59292]: 2024-08-30 09:18:40.894097932 +0000 UTC m=+0.512289952 container remove aa5fc8cc384ddf0d2a6d286346a2e8fa1bd067eb767c65f8dc678081b8ba485d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-osd-11-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, GIT_CLEAN=True, io.buildah.version=1.37.1, ceph=True) 2024-08-30T09:18:40.916 DEBUG:teuthology.orchestra.run.smithi190:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968@osd.11.service' 2024-08-30T09:18:40.954 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-30T09:18:40.954 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-08-30T09:18:40.954 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-08-30T09:18:40.955 DEBUG:teuthology.orchestra.run.smithi042:> sudo systemctl stop ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968@mds.a 2024-08-30T09:18:41.273 INFO:journalctl@ceph.mds.a.smithi042.stdout:Aug 30 09:18:41 smithi042 systemd[1]: Stopping Ceph mds.a for b0895c1a-66ac-11ef-bcd4-c7b262605968... 2024-08-30T09:18:41.633 INFO:journalctl@ceph.mds.a.smithi042.stdout:Aug 30 09:18:41 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mds-a[62223]: 2024-08-30T09:18:41.270+0000 7fd7194d1640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-30T09:18:41.633 INFO:journalctl@ceph.mds.a.smithi042.stdout:Aug 30 09:18:41 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mds-a[62223]: 2024-08-30T09:18:41.270+0000 7fd7194d1640 -1 mds.a *** got signal Terminated *** 2024-08-30T09:18:45.633 INFO:journalctl@ceph.mds.a.smithi042.stdout:Aug 30 09:18:45 smithi042 podman[101549]: 2024-08-30 09:18:45.299633408 +0000 UTC m=+4.138373062 container died f14af3600daf0bd4031f8f92306a9a6dcc2aafa7bd51531d585bfe8d7a5e708e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mds-a, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, io.buildah.version=1.37.1) 2024-08-30T09:18:45.914 INFO:journalctl@ceph.mds.a.smithi042.stdout:Aug 30 09:18:45 smithi042 podman[101549]: 2024-08-30 09:18:45.662138166 +0000 UTC m=+4.500877819 container cleanup f14af3600daf0bd4031f8f92306a9a6dcc2aafa7bd51531d585bfe8d7a5e708e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mds-a, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.1, org.label-schema.build-date=20240731) 2024-08-30T09:18:45.915 INFO:journalctl@ceph.mds.a.smithi042.stdout:Aug 30 09:18:45 smithi042 bash[101549]: ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mds-a 2024-08-30T09:18:46.050 DEBUG:teuthology.orchestra.run.smithi042:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968@mds.a.service' 2024-08-30T09:18:46.085 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-30T09:18:46.085 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-08-30T09:18:46.085 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-08-30T09:18:46.085 DEBUG:teuthology.orchestra.run.smithi042:> sudo systemctl stop ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968@mds.d 2024-08-30T09:18:46.426 INFO:journalctl@ceph.mds.d.smithi042.stdout:Aug 30 09:18:46 smithi042 systemd[1]: Stopping Ceph mds.d for b0895c1a-66ac-11ef-bcd4-c7b262605968... 2024-08-30T09:18:46.883 INFO:journalctl@ceph.mds.d.smithi042.stdout:Aug 30 09:18:46 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mds-d[61037]: 2024-08-30T09:18:46.424+0000 7fbff0623640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-30T09:18:46.883 INFO:journalctl@ceph.mds.d.smithi042.stdout:Aug 30 09:18:46 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mds-d[61037]: 2024-08-30T09:18:46.424+0000 7fbff0623640 -1 mds.d *** got signal Terminated *** 2024-08-30T09:18:50.633 INFO:journalctl@ceph.mds.d.smithi042.stdout:Aug 30 09:18:50 smithi042 podman[101670]: 2024-08-30 09:18:50.353907188 +0000 UTC m=+4.051086268 container died e53c84315db0acd7dedbe9f8e1880bdc7c4793da98a260cd440334b9b46f4d78 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mds-d, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.1, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-08-30T09:18:51.185 INFO:journalctl@ceph.mds.d.smithi042.stdout:Aug 30 09:18:50 smithi042 podman[101670]: 2024-08-30 09:18:50.966106351 +0000 UTC m=+4.663285421 container cleanup e53c84315db0acd7dedbe9f8e1880bdc7c4793da98a260cd440334b9b46f4d78 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mds-d, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.1, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611) 2024-08-30T09:18:51.185 INFO:journalctl@ceph.mds.d.smithi042.stdout:Aug 30 09:18:50 smithi042 bash[101670]: ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mds-d 2024-08-30T09:18:51.418 DEBUG:teuthology.orchestra.run.smithi042:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968@mds.d.service' 2024-08-30T09:18:51.438 INFO:journalctl@ceph.mds.d.smithi042.stdout:Aug 30 09:18:51 smithi042 podman[101683]: 2024-08-30 09:18:51.183942404 +0000 UTC m=+0.830799552 container remove e53c84315db0acd7dedbe9f8e1880bdc7c4793da98a260cd440334b9b46f4d78 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mds-d, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, io.buildah.version=1.37.1, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2) 2024-08-30T09:18:51.438 INFO:journalctl@ceph.mds.d.smithi042.stdout:Aug 30 09:18:51 smithi042 systemd[1]: ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968@mds.d.service: Deactivated successfully. 2024-08-30T09:18:51.438 INFO:journalctl@ceph.mds.d.smithi042.stdout:Aug 30 09:18:51 smithi042 systemd[1]: Stopped Ceph mds.d for b0895c1a-66ac-11ef-bcd4-c7b262605968. 2024-08-30T09:18:51.439 INFO:journalctl@ceph.mds.d.smithi042.stdout:Aug 30 09:18:51 smithi042 systemd[1]: ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968@mds.d.service: Consumed 1.123s CPU time. 2024-08-30T09:18:51.450 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-30T09:18:51.450 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-08-30T09:18:51.450 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-08-30T09:18:51.450 DEBUG:teuthology.orchestra.run.smithi042:> sudo systemctl stop ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968@mds.g 2024-08-30T09:18:51.779 INFO:journalctl@ceph.mds.g.smithi042.stdout:Aug 30 09:18:51 smithi042 systemd[1]: Stopping Ceph mds.g for b0895c1a-66ac-11ef-bcd4-c7b262605968... 2024-08-30T09:18:52.133 INFO:journalctl@ceph.mds.g.smithi042.stdout:Aug 30 09:18:51 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mds-g[61825]: 2024-08-30T09:18:51.777+0000 7fc9ce24b640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-30T09:18:52.133 INFO:journalctl@ceph.mds.g.smithi042.stdout:Aug 30 09:18:51 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mds-g[61825]: 2024-08-30T09:18:51.777+0000 7fc9ce24b640 -1 mds.g *** got signal Terminated *** 2024-08-30T09:18:54.697 INFO:journalctl@ceph.mds.g.smithi042.stdout:Aug 30 09:18:54 smithi042 podman[101792]: 2024-08-30 09:18:54.432060608 +0000 UTC m=+2.756585532 container died 83975ccb3d1e3282f93ea03c7651343123e1348dcbd40668c9e49fb7cafa336f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mds-g, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, CEPH_POINT_RELEASE=, io.buildah.version=1.37.1, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-08-30T09:18:55.064 INFO:journalctl@ceph.mds.g.smithi042.stdout:Aug 30 09:18:54 smithi042 podman[101792]: 2024-08-30 09:18:54.786545874 +0000 UTC m=+3.111070797 container cleanup 83975ccb3d1e3282f93ea03c7651343123e1348dcbd40668c9e49fb7cafa336f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mds-g, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.37.1, GIT_BRANCH=HEAD) 2024-08-30T09:18:55.064 INFO:journalctl@ceph.mds.g.smithi042.stdout:Aug 30 09:18:54 smithi042 bash[101792]: ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mds-g 2024-08-30T09:18:55.193 DEBUG:teuthology.orchestra.run.smithi042:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968@mds.g.service' 2024-08-30T09:18:55.225 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-30T09:18:55.225 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-08-30T09:18:55.225 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-08-30T09:18:55.225 DEBUG:teuthology.orchestra.run.smithi042:> sudo systemctl stop ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968@mds.j 2024-08-30T09:18:55.383 INFO:journalctl@ceph.mds.j.smithi042.stdout:Aug 30 09:18:55 smithi042 systemd[1]: Stopping Ceph mds.j for b0895c1a-66ac-11ef-bcd4-c7b262605968... 2024-08-30T09:18:55.883 INFO:journalctl@ceph.mds.j.smithi042.stdout:Aug 30 09:18:55 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mds-j[60644]: 2024-08-30T09:18:55.556+0000 7f391117a640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-30T09:18:55.883 INFO:journalctl@ceph.mds.j.smithi042.stdout:Aug 30 09:18:55 smithi042 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mds-j[60644]: 2024-08-30T09:18:55.556+0000 7f391117a640 -1 mds.j *** got signal Terminated *** 2024-08-30T09:18:58.883 INFO:journalctl@ceph.mds.j.smithi042.stdout:Aug 30 09:18:58 smithi042 podman[101912]: 2024-08-30 09:18:58.631286169 +0000 UTC m=+3.181442427 container died f4e7191e96589b434ceb97793e9c3562cd8c70c439b0ac6d141aac6a78631e0e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mds-j, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.37.1, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux ) 2024-08-30T09:18:59.294 INFO:journalctl@ceph.mds.j.smithi042.stdout:Aug 30 09:18:59 smithi042 podman[101912]: 2024-08-30 09:18:59.007825747 +0000 UTC m=+3.557981994 container cleanup f4e7191e96589b434ceb97793e9c3562cd8c70c439b0ac6d141aac6a78631e0e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mds-j, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.37.1, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-08-30T09:18:59.294 INFO:journalctl@ceph.mds.j.smithi042.stdout:Aug 30 09:18:59 smithi042 bash[101912]: ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mds-j 2024-08-30T09:18:59.414 DEBUG:teuthology.orchestra.run.smithi042:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968@mds.j.service' 2024-08-30T09:18:59.448 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-30T09:18:59.448 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-08-30T09:18:59.448 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-08-30T09:18:59.448 DEBUG:teuthology.orchestra.run.smithi167:> sudo systemctl stop ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968@mds.b 2024-08-30T09:18:59.748 INFO:journalctl@ceph.mds.b.smithi167.stdout:Aug 30 09:18:59 smithi167 systemd[1]: Stopping Ceph mds.b for b0895c1a-66ac-11ef-bcd4-c7b262605968... 2024-08-30T09:19:00.001 INFO:journalctl@ceph.mds.b.smithi167.stdout:Aug 30 09:18:59 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mds-b[52473]: 2024-08-30T09:18:59.744+0000 7fec02836640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-30T09:19:00.002 INFO:journalctl@ceph.mds.b.smithi167.stdout:Aug 30 09:18:59 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mds-b[52473]: 2024-08-30T09:18:59.744+0000 7fec02836640 -1 mds.b *** got signal Terminated *** 2024-08-30T09:19:03.171 INFO:journalctl@ceph.mds.b.smithi167.stdout:Aug 30 09:19:02 smithi167 podman[58754]: 2024-08-30 09:19:02.868801516 +0000 UTC m=+3.226645096 container died 7ec7569892e37e2a5c54a19a462030d6d5928b79ad6180962113e3ef08db023c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mds-b, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.37.1, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux ) 2024-08-30T09:19:03.430 INFO:journalctl@ceph.mds.b.smithi167.stdout:Aug 30 09:19:03 smithi167 podman[58754]: 2024-08-30 09:19:03.244018858 +0000 UTC m=+3.601862438 container cleanup 7ec7569892e37e2a5c54a19a462030d6d5928b79ad6180962113e3ef08db023c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mds-b, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.1, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True) 2024-08-30T09:19:03.430 INFO:journalctl@ceph.mds.b.smithi167.stdout:Aug 30 09:19:03 smithi167 bash[58754]: ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mds-b 2024-08-30T09:19:03.672 DEBUG:teuthology.orchestra.run.smithi167:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968@mds.b.service' 2024-08-30T09:19:03.695 INFO:journalctl@ceph.mds.b.smithi167.stdout:Aug 30 09:19:03 smithi167 podman[58778]: 2024-08-30 09:19:03.428752255 +0000 UTC m=+0.555289337 container remove 7ec7569892e37e2a5c54a19a462030d6d5928b79ad6180962113e3ef08db023c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mds-b, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, io.buildah.version=1.37.1, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-08-30T09:19:03.695 INFO:journalctl@ceph.mds.b.smithi167.stdout:Aug 30 09:19:03 smithi167 systemd[1]: ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968@mds.b.service: Deactivated successfully. 2024-08-30T09:19:03.695 INFO:journalctl@ceph.mds.b.smithi167.stdout:Aug 30 09:19:03 smithi167 systemd[1]: Stopped Ceph mds.b for b0895c1a-66ac-11ef-bcd4-c7b262605968. 2024-08-30T09:19:03.695 INFO:journalctl@ceph.mds.b.smithi167.stdout:Aug 30 09:19:03 smithi167 systemd[1]: ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968@mds.b.service: Consumed 21.755s CPU time. 2024-08-30T09:19:03.708 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-30T09:19:03.708 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-08-30T09:19:03.708 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-08-30T09:19:03.709 DEBUG:teuthology.orchestra.run.smithi167:> sudo systemctl stop ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968@mds.e 2024-08-30T09:19:04.025 INFO:journalctl@ceph.mds.e.smithi167.stdout:Aug 30 09:19:03 smithi167 systemd[1]: Stopping Ceph mds.e for b0895c1a-66ac-11ef-bcd4-c7b262605968... 2024-08-30T09:19:04.327 INFO:journalctl@ceph.mds.e.smithi167.stdout:Aug 30 09:19:04 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mds-e[51862]: 2024-08-30T09:19:04.023+0000 7f9988a40640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-30T09:19:04.327 INFO:journalctl@ceph.mds.e.smithi167.stdout:Aug 30 09:19:04 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mds-e[51862]: 2024-08-30T09:19:04.023+0000 7f9988a40640 -1 mds.e *** got signal Terminated *** 2024-08-30T09:19:07.136 INFO:journalctl@ceph.mds.e.smithi167.stdout:Aug 30 09:19:06 smithi167 podman[58886]: 2024-08-30 09:19:06.847935151 +0000 UTC m=+2.925528114 container died 7bbee5d4b3ee459b66da5d0b33556e2b04008534654bb68a95be91e272a1c26e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mds-e, io.buildah.version=1.37.1, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731) 2024-08-30T09:19:07.492 INFO:journalctl@ceph.mds.e.smithi167.stdout:Aug 30 09:19:07 smithi167 podman[58886]: 2024-08-30 09:19:07.231665125 +0000 UTC m=+3.309258090 container cleanup 7bbee5d4b3ee459b66da5d0b33556e2b04008534654bb68a95be91e272a1c26e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mds-e, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.1, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-08-30T09:19:07.492 INFO:journalctl@ceph.mds.e.smithi167.stdout:Aug 30 09:19:07 smithi167 bash[58886]: ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mds-e 2024-08-30T09:19:07.666 DEBUG:teuthology.orchestra.run.smithi167:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968@mds.e.service' 2024-08-30T09:19:07.700 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-30T09:19:07.701 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-08-30T09:19:07.701 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-08-30T09:19:07.701 DEBUG:teuthology.orchestra.run.smithi167:> sudo systemctl stop ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968@mds.h 2024-08-30T09:19:08.020 INFO:journalctl@ceph.mds.h.smithi167.stdout:Aug 30 09:19:07 smithi167 systemd[1]: Stopping Ceph mds.h for b0895c1a-66ac-11ef-bcd4-c7b262605968... 2024-08-30T09:19:08.327 INFO:journalctl@ceph.mds.h.smithi167.stdout:Aug 30 09:19:08 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mds-h[52167]: 2024-08-30T09:19:08.018+0000 7f1d9c514640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-30T09:19:08.327 INFO:journalctl@ceph.mds.h.smithi167.stdout:Aug 30 09:19:08 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mds-h[52167]: 2024-08-30T09:19:08.018+0000 7f1d9c514640 -1 mds.h *** got signal Terminated *** 2024-08-30T09:19:11.145 INFO:journalctl@ceph.mds.h.smithi167.stdout:Aug 30 09:19:10 smithi167 podman[59007]: 2024-08-30 09:19:10.848838305 +0000 UTC m=+2.934111731 container died ed5d7ebb56704ae437e9c0753b464521c7c13d7634a7b62b914162a51742a356 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mds-h, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.1, org.label-schema.vendor=CentOS) 2024-08-30T09:19:11.454 INFO:journalctl@ceph.mds.h.smithi167.stdout:Aug 30 09:19:11 smithi167 podman[59007]: 2024-08-30 09:19:11.243737118 +0000 UTC m=+3.329010545 container cleanup ed5d7ebb56704ae437e9c0753b464521c7c13d7634a7b62b914162a51742a356 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mds-h, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.1, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731) 2024-08-30T09:19:11.454 INFO:journalctl@ceph.mds.h.smithi167.stdout:Aug 30 09:19:11 smithi167 bash[59007]: ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mds-h 2024-08-30T09:19:11.603 DEBUG:teuthology.orchestra.run.smithi167:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968@mds.h.service' 2024-08-30T09:19:11.636 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-30T09:19:11.636 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-08-30T09:19:11.637 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-08-30T09:19:11.637 DEBUG:teuthology.orchestra.run.smithi167:> sudo systemctl stop ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968@mds.k 2024-08-30T09:19:12.077 INFO:journalctl@ceph.mds.k.smithi167.stdout:Aug 30 09:19:11 smithi167 systemd[1]: Stopping Ceph mds.k for b0895c1a-66ac-11ef-bcd4-c7b262605968... 2024-08-30T09:19:12.077 INFO:journalctl@ceph.mds.k.smithi167.stdout:Aug 30 09:19:11 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mds-k[51549]: 2024-08-30T09:19:11.939+0000 7f7ab72b9640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-30T09:19:12.077 INFO:journalctl@ceph.mds.k.smithi167.stdout:Aug 30 09:19:11 smithi167 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mds-k[51549]: 2024-08-30T09:19:11.939+0000 7f7ab72b9640 -1 mds.k *** got signal Terminated *** 2024-08-30T09:19:15.616 INFO:journalctl@ceph.mds.k.smithi167.stdout:Aug 30 09:19:15 smithi167 podman[59126]: 2024-08-30 09:19:15.34641714 +0000 UTC m=+3.500103463 container died d26c861d47520fb985d93dfb27fb8a31c62acf498712be89381d91e40abb777a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mds-k, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, io.buildah.version=1.37.1) 2024-08-30T09:19:15.992 INFO:journalctl@ceph.mds.k.smithi167.stdout:Aug 30 09:19:15 smithi167 podman[59126]: 2024-08-30 09:19:15.706109006 +0000 UTC m=+3.859795322 container cleanup d26c861d47520fb985d93dfb27fb8a31c62acf498712be89381d91e40abb777a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mds-k, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.1, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-08-30T09:19:15.992 INFO:journalctl@ceph.mds.k.smithi167.stdout:Aug 30 09:19:15 smithi167 bash[59126]: ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mds-k 2024-08-30T09:19:16.112 DEBUG:teuthology.orchestra.run.smithi167:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968@mds.k.service' 2024-08-30T09:19:16.144 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-30T09:19:16.144 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-08-30T09:19:16.144 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-08-30T09:19:16.145 DEBUG:teuthology.orchestra.run.smithi190:> sudo systemctl stop ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968@mds.c 2024-08-30T09:19:16.436 INFO:journalctl@ceph.mds.c.smithi190.stdout:Aug 30 09:19:16 smithi190 systemd[1]: Stopping Ceph mds.c for b0895c1a-66ac-11ef-bcd4-c7b262605968... 2024-08-30T09:19:16.800 INFO:journalctl@ceph.mds.c.smithi190.stdout:Aug 30 09:19:16 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mds-c[52413]: 2024-08-30T09:19:16.443+0000 7f01baf10640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-30T09:19:16.800 INFO:journalctl@ceph.mds.c.smithi190.stdout:Aug 30 09:19:16 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mds-c[52413]: 2024-08-30T09:19:16.443+0000 7f01baf10640 -1 mds.c *** got signal Terminated *** 2024-08-30T09:19:19.170 INFO:journalctl@ceph.mds.c.smithi190.stdout:Aug 30 09:19:18 smithi190 podman[59373]: 2024-08-30 09:19:18.838464743 +0000 UTC m=+2.497747241 container died 0526a2005b1e060a9394f473a6a6b82e598280c98eee9e48acbaaf1d75b28120 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mds-c, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, ceph=True, io.buildah.version=1.37.1, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-08-30T09:19:19.436 INFO:journalctl@ceph.mds.c.smithi190.stdout:Aug 30 09:19:19 smithi190 podman[59373]: 2024-08-30 09:19:19.25988264 +0000 UTC m=+2.919165180 container cleanup 0526a2005b1e060a9394f473a6a6b82e598280c98eee9e48acbaaf1d75b28120 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mds-c, io.buildah.version=1.37.1, GIT_BRANCH=HEAD, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731) 2024-08-30T09:19:19.436 INFO:journalctl@ceph.mds.c.smithi190.stdout:Aug 30 09:19:19 smithi190 bash[59373]: ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mds-c 2024-08-30T09:19:19.732 INFO:journalctl@ceph.mds.c.smithi190.stdout:Aug 30 09:19:19 smithi190 podman[59395]: 2024-08-30 09:19:19.478244369 +0000 UTC m=+0.637063720 container remove 0526a2005b1e060a9394f473a6a6b82e598280c98eee9e48acbaaf1d75b28120 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mds-c, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.1, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-08-30T09:19:19.737 DEBUG:teuthology.orchestra.run.smithi190:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968@mds.c.service' 2024-08-30T09:19:19.772 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-30T09:19:19.772 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-08-30T09:19:19.772 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-08-30T09:19:19.772 DEBUG:teuthology.orchestra.run.smithi190:> sudo systemctl stop ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968@mds.f 2024-08-30T09:19:20.099 INFO:journalctl@ceph.mds.f.smithi190.stdout:Aug 30 09:19:19 smithi190 systemd[1]: Stopping Ceph mds.f for b0895c1a-66ac-11ef-bcd4-c7b262605968... 2024-08-30T09:19:20.436 INFO:journalctl@ceph.mds.f.smithi190.stdout:Aug 30 09:19:20 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mds-f[52114]: 2024-08-30T09:19:20.096+0000 7f0fe7767640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-30T09:19:20.436 INFO:journalctl@ceph.mds.f.smithi190.stdout:Aug 30 09:19:20 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mds-f[52114]: 2024-08-30T09:19:20.096+0000 7f0fe7767640 -1 mds.f *** got signal Terminated *** 2024-08-30T09:19:23.108 INFO:journalctl@ceph.mds.f.smithi190.stdout:Aug 30 09:19:22 smithi190 podman[59508]: 2024-08-30 09:19:22.82551363 +0000 UTC m=+2.838686976 container died 5f43b96cf342144dd1d4da00172a5619c632f8d5e68b671b9d2cc22c21645ff3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mds-f, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, io.buildah.version=1.37.1, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, ceph=True) 2024-08-30T09:19:23.436 INFO:journalctl@ceph.mds.f.smithi190.stdout:Aug 30 09:19:23 smithi190 podman[59508]: 2024-08-30 09:19:23.222053751 +0000 UTC m=+3.235227104 container cleanup 5f43b96cf342144dd1d4da00172a5619c632f8d5e68b671b9d2cc22c21645ff3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mds-f, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.1, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True) 2024-08-30T09:19:23.436 INFO:journalctl@ceph.mds.f.smithi190.stdout:Aug 30 09:19:23 smithi190 bash[59508]: ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mds-f 2024-08-30T09:19:23.600 DEBUG:teuthology.orchestra.run.smithi190:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968@mds.f.service' 2024-08-30T09:19:23.634 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-30T09:19:23.634 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-08-30T09:19:23.634 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-08-30T09:19:23.634 DEBUG:teuthology.orchestra.run.smithi190:> sudo systemctl stop ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968@mds.i 2024-08-30T09:19:24.186 INFO:journalctl@ceph.mds.i.smithi190.stdout:Aug 30 09:19:23 smithi190 systemd[1]: Stopping Ceph mds.i for b0895c1a-66ac-11ef-bcd4-c7b262605968... 2024-08-30T09:19:24.186 INFO:journalctl@ceph.mds.i.smithi190.stdout:Aug 30 09:19:23 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mds-i[52728]: 2024-08-30T09:19:23.959+0000 7f923107b640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-30T09:19:24.186 INFO:journalctl@ceph.mds.i.smithi190.stdout:Aug 30 09:19:23 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mds-i[52728]: 2024-08-30T09:19:23.959+0000 7f923107b640 -1 mds.i *** got signal Terminated *** 2024-08-30T09:19:27.103 INFO:journalctl@ceph.mds.i.smithi190.stdout:Aug 30 09:19:26 smithi190 podman[59632]: 2024-08-30 09:19:26.827411237 +0000 UTC m=+2.971245980 container died 0987ddbec8abf03313f1a21b63efee1dc3876668467712f47854ea96231ded68 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mds-i, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.build-date=20240731, ceph=True, io.buildah.version=1.37.1) 2024-08-30T09:19:27.436 INFO:journalctl@ceph.mds.i.smithi190.stdout:Aug 30 09:19:27 smithi190 podman[59632]: 2024-08-30 09:19:27.192369059 +0000 UTC m=+3.336203802 container cleanup 0987ddbec8abf03313f1a21b63efee1dc3876668467712f47854ea96231ded68 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mds-i, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_CLEAN=True, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, io.buildah.version=1.37.1, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-08-30T09:19:27.436 INFO:journalctl@ceph.mds.i.smithi190.stdout:Aug 30 09:19:27 smithi190 bash[59632]: ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mds-i 2024-08-30T09:19:27.631 DEBUG:teuthology.orchestra.run.smithi190:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968@mds.i.service' 2024-08-30T09:19:27.664 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-30T09:19:27.664 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-08-30T09:19:27.664 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-08-30T09:19:27.664 DEBUG:teuthology.orchestra.run.smithi190:> sudo systemctl stop ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968@mds.l 2024-08-30T09:19:28.186 INFO:journalctl@ceph.mds.l.smithi190.stdout:Aug 30 09:19:27 smithi190 systemd[1]: Stopping Ceph mds.l for b0895c1a-66ac-11ef-bcd4-c7b262605968... 2024-08-30T09:19:28.186 INFO:journalctl@ceph.mds.l.smithi190.stdout:Aug 30 09:19:27 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mds-l[51811]: 2024-08-30T09:19:27.969+0000 7f3c2af5e640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-30T09:19:28.186 INFO:journalctl@ceph.mds.l.smithi190.stdout:Aug 30 09:19:27 smithi190 ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mds-l[51811]: 2024-08-30T09:19:27.970+0000 7f3c2af5e640 -1 mds.l *** got signal Terminated *** 2024-08-30T09:19:32.272 INFO:journalctl@ceph.mds.l.smithi190.stdout:Aug 30 09:19:31 smithi190 podman[59753]: 2024-08-30 09:19:31.984392635 +0000 UTC m=+4.099139496 container died b0102237447b71169a91a36b5e2376e684526e91f1661337e627ef20df84d0c2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mds-l, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, io.buildah.version=1.37.1, org.label-schema.build-date=20240731) 2024-08-30T09:19:32.589 INFO:journalctl@ceph.mds.l.smithi190.stdout:Aug 30 09:19:32 smithi190 podman[59753]: 2024-08-30 09:19:32.370721472 +0000 UTC m=+4.485468336 container cleanup b0102237447b71169a91a36b5e2376e684526e91f1661337e627ef20df84d0c2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mds-l, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.37.1, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux ) 2024-08-30T09:19:32.590 INFO:journalctl@ceph.mds.l.smithi190.stdout:Aug 30 09:19:32 smithi190 bash[59753]: ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968-mds-l 2024-08-30T09:19:32.721 DEBUG:teuthology.orchestra.run.smithi190:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b0895c1a-66ac-11ef-bcd4-c7b262605968@mds.l.service' 2024-08-30T09:19:32.752 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-30T09:19:32.753 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-08-30T09:19:32.753 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 --force --keep-logs 2024-08-30T09:19:32.962 INFO:teuthology.orchestra.run.smithi042.stdout:Deleting cluster with fsid: b0895c1a-66ac-11ef-bcd4-c7b262605968 2024-08-30T09:19:35.482 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 --force --keep-logs 2024-08-30T09:19:35.684 INFO:teuthology.orchestra.run.smithi167.stdout:Deleting cluster with fsid: b0895c1a-66ac-11ef-bcd4-c7b262605968 2024-08-30T09:19:38.259 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 --force --keep-logs 2024-08-30T09:19:38.458 INFO:teuthology.orchestra.run.smithi190.stdout:Deleting cluster with fsid: b0895c1a-66ac-11ef-bcd4-c7b262605968 2024-08-30T09:19:40.874 DEBUG:teuthology.orchestra.run.smithi042:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-08-30T09:19:40.907 DEBUG:teuthology.orchestra.run.smithi167:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-08-30T09:19:40.941 DEBUG:teuthology.orchestra.run.smithi190:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-08-30T09:19:40.966 INFO:tasks.cephadm:Archiving crash dumps... 2024-08-30T09:19:40.968 DEBUG:teuthology.misc:Transferring archived files from smithi042:/var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/crash to /home/teuthworker/archive/mchangir-2024-08-30_08:30:37-fs-wip-mchangir-testing-20240828.085847-main-debug-distro-default-smithi/7881205/remote/smithi042/crash 2024-08-30T09:19:40.969 DEBUG:teuthology.orchestra.run.smithi042:> sudo tar c -f - -C /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/crash -- . 2024-08-30T09:19:41.029 INFO:teuthology.orchestra.run.smithi042.stderr:tar: /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/crash: Cannot open: No such file or directory 2024-08-30T09:19:41.029 INFO:teuthology.orchestra.run.smithi042.stderr:tar: Error is not recoverable: exiting now 2024-08-30T09:19:41.031 DEBUG:teuthology.misc:Transferring archived files from smithi167:/var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/crash to /home/teuthworker/archive/mchangir-2024-08-30_08:30:37-fs-wip-mchangir-testing-20240828.085847-main-debug-distro-default-smithi/7881205/remote/smithi167/crash 2024-08-30T09:19:41.032 DEBUG:teuthology.orchestra.run.smithi167:> sudo tar c -f - -C /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/crash -- . 2024-08-30T09:19:41.078 INFO:teuthology.orchestra.run.smithi167.stderr:tar: /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/crash: Cannot open: No such file or directory 2024-08-30T09:19:41.079 INFO:teuthology.orchestra.run.smithi167.stderr:tar: Error is not recoverable: exiting now 2024-08-30T09:19:41.081 DEBUG:teuthology.misc:Transferring archived files from smithi190:/var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/crash to /home/teuthworker/archive/mchangir-2024-08-30_08:30:37-fs-wip-mchangir-testing-20240828.085847-main-debug-distro-default-smithi/7881205/remote/smithi190/crash 2024-08-30T09:19:41.082 DEBUG:teuthology.orchestra.run.smithi190:> sudo tar c -f - -C /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/crash -- . 2024-08-30T09:19:41.114 INFO:teuthology.orchestra.run.smithi190.stderr:tar: /var/lib/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/crash: Cannot open: No such file or directory 2024-08-30T09:19:41.114 INFO:teuthology.orchestra.run.smithi190.stderr:tar: Error is not recoverable: exiting now 2024-08-30T09:19:41.115 INFO:tasks.cephadm:Checking cluster log for badness... 2024-08-30T09:19:41.115 DEBUG:teuthology.orchestra.run.smithi042:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v 'fs.*is degraded' | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v 'insufficient standby MDS daemons available' | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-08-30T09:19:41.227 INFO:teuthology.orchestra.run.smithi042.stdout:2024-08-30T09:03:16.440611+0000 mds.b (mds.0) 19 : cluster [WRN] Scrub error on inode 0x1000000000f (/volumes/qa/sv_1/d3168875-d2e5-410d-8754-72d29955777b/postgres/data/pg_subtrans) see mds.b log and `damage ls` output for details 2024-08-30T09:19:41.227 WARNING:tasks.cephadm:Found errors (ERR|WRN|SEC) in cluster log 2024-08-30T09:19:41.227 DEBUG:teuthology.orchestra.run.smithi042:> sudo egrep '\[SEC\]' /var/log/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v 'fs.*is degraded' | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v 'insufficient standby MDS daemons available' | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-08-30T09:19:41.264 DEBUG:teuthology.orchestra.run.smithi042:> sudo egrep '\[ERR\]' /var/log/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v 'fs.*is degraded' | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v 'insufficient standby MDS daemons available' | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-08-30T09:19:41.339 DEBUG:teuthology.orchestra.run.smithi042:> sudo egrep '\[WRN\]' /var/log/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v 'fs.*is degraded' | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v 'insufficient standby MDS daemons available' | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-08-30T09:19:41.414 INFO:teuthology.orchestra.run.smithi042.stdout:2024-08-30T09:03:16.440611+0000 mds.b (mds.0) 19 : cluster [WRN] Scrub error on inode 0x1000000000f (/volumes/qa/sv_1/d3168875-d2e5-410d-8754-72d29955777b/postgres/data/pg_subtrans) see mds.b log and `damage ls` output for details 2024-08-30T09:19:41.414 INFO:tasks.cephadm:Compressing logs... 2024-08-30T09:19:41.415 DEBUG:teuthology.orchestra.run.smithi042:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-08-30T09:19:41.457 DEBUG:teuthology.orchestra.run.smithi167:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-08-30T09:19:41.460 DEBUG:teuthology.orchestra.run.smithi190:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-08-30T09:19:41.497 INFO:teuthology.orchestra.run.smithi190.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-08-30T09:19:41.497 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/ceph-volume.log 2024-08-30T09:19:41.504 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/ceph.audit.log 2024-08-30T09:19:41.504 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/ceph-mds.i.log 2024-08-30T09:19:41.504 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/ceph-mon.c.log 2024-08-30T09:19:41.505 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/ceph.audit.log: /var/log/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/ceph.cephadm.log 2024-08-30T09:19:41.505 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/ceph-osd.11.log 2024-08-30T09:19:41.505 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/ceph-osd.2.log 2024-08-30T09:19:41.505 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/ceph-osd.5.log 2024-08-30T09:19:41.505 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/ceph.log 2024-08-30T09:19:41.505 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/ceph-mds.l.log 2024-08-30T09:19:41.505 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/ceph-osd.8.log 2024-08-30T09:19:41.505 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/ceph-mds.l.log: gzip -5 --verbose -- /var/log/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/ceph-mgr.z.log 2024-08-30T09:19:41.505 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/ceph-mds.c.log 2024-08-30T09:19:41.505 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/ceph-mgr.z.log: gzip -5 --verbose -- /var/log/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/ceph-mds.f.log 2024-08-30T09:19:41.505 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-08-30T09:19:41.506 INFO:teuthology.orchestra.run.smithi167.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-08-30T09:19:41.506 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/ceph-osd.10.log 2024-08-30T09:19:41.506 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/ceph-osd.1.log 2024-08-30T09:19:41.506 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/ceph-osd.7.log 2024-08-30T09:19:41.506 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/ceph-volume.log 2024-08-30T09:19:41.506 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/ceph-mds.b.log 2024-08-30T09:19:41.506 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/ceph-mds.k.log 2024-08-30T09:19:41.506 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/ceph-osd.4.log 2024-08-30T09:19:41.506 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/ceph-mds.k.log: gzip -5 --verbose -- /var/log/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/ceph.audit.log 2024-08-30T09:19:41.506 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/ceph.cephadm.log 2024-08-30T09:19:41.507 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/ceph-mgr.y.log 2024-08-30T09:19:41.507 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/ceph.log 2024-08-30T09:19:41.507 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/ceph-mds.h.log 2024-08-30T09:19:41.507 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/ceph-mon.b.log 2024-08-30T09:19:41.507 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/ceph-mds.e.log 2024-08-30T09:19:41.507 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-08-30T09:19:41.511 INFO:teuthology.orchestra.run.smithi042.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.log: No such file or directory 2024-08-30T09:19:41.512 INFO:teuthology.orchestra.run.smithi042.stderr: 2024-08-30T09:19:41.512 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/ceph-osd.0.log 2024-08-30T09:19:41.513 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/ceph-osd.9.log 2024-08-30T09:19:41.513 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.log: /var/log/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/ceph-volume.log 2024-08-30T09:19:41.513 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/ceph.audit.log 2024-08-30T09:19:41.513 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/ceph-mds.a.log 2024-08-30T09:19:41.516 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/ceph.cephadm.log 2024-08-30T09:19:41.516 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/ceph-mon.a.log 2024-08-30T09:19:41.516 INFO:teuthology.orchestra.run.smithi042.stderr: 92.5% -- replaced with /var/log/ceph/ceph-client.admin.log.gz 2024-08-30T09:19:41.516 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/ceph-mds.g.log 2024-08-30T09:19:41.516 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/ceph-osd.6.log 2024-08-30T09:19:41.516 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/ceph.log 2024-08-30T09:19:41.516 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5/var/log/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/ceph-osd.6.log: --verbose -- /var/log/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/ceph-mds.d.log 2024-08-30T09:19:41.516 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/ceph-mgr.x.log 2024-08-30T09:19:41.516 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/ceph-mds.j.log 2024-08-30T09:19:41.516 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/ceph-osd.3.log 2024-08-30T09:19:41.516 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-08-30T09:19:41.517 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/ceph-osd.3.log: /var/log/ceph/cephadm.log: 87.1% -- replaced with /var/log/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/ceph.log.gz 2024-08-30T09:19:41.548 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/ceph-mds.f.log: /var/log/ceph/cephadm.log: 82.5% -- replaced with /var/log/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/ceph.cephadm.log.gz 2024-08-30T09:19:41.634 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/ceph-mds.e.log: /var/log/ceph/cephadm.log: 92.9% -- replaced with /var/log/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/ceph-mgr.y.log.gz 2024-08-30T09:19:41.656 INFO:teuthology.orchestra.run.smithi190.stderr: 92.9% -- replaced with /var/log/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/ceph-mgr.z.log.gz 2024-08-30T09:19:41.671 INFO:teuthology.orchestra.run.smithi042.stderr: 95.0% -- replaced with /var/log/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/ceph-mds.a.log.gz 2024-08-30T09:19:41.677 INFO:teuthology.orchestra.run.smithi190.stderr: 92.2% -- replaced with /var/log/ceph/cephadm.log.gz 2024-08-30T09:19:41.682 INFO:teuthology.orchestra.run.smithi042.stderr: 82.6% -- replaced with /var/log/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/ceph.cephadm.log.gz 2024-08-30T09:19:41.683 INFO:teuthology.orchestra.run.smithi042.stderr: 95.0% -- replaced with /var/log/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/ceph-mds.d.log.gz 2024-08-30T09:19:41.693 INFO:teuthology.orchestra.run.smithi190.stderr: 94.9% -- replaced with /var/log/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/ceph-mds.l.log.gz 2024-08-30T09:19:41.732 INFO:teuthology.orchestra.run.smithi042.stderr: 94.2% -- replaced with /var/log/ceph/cephadm.log.gz 2024-08-30T09:19:41.745 INFO:teuthology.orchestra.run.smithi167.stderr: 94.9% -- replaced with /var/log/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/ceph-mds.k.log.gz 2024-08-30T09:19:41.755 INFO:teuthology.orchestra.run.smithi042.stderr: 90.1% -- replaced with /var/log/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/ceph.audit.log.gz 2024-08-30T09:19:41.812 INFO:teuthology.orchestra.run.smithi042.stderr: 90.9% -- replaced with /var/log/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/ceph-volume.log.gz 2024-08-30T09:19:41.821 INFO:teuthology.orchestra.run.smithi190.stderr: 87.2% -- replaced with /var/log/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/ceph.log.gz 2024-08-30T09:19:41.823 INFO:teuthology.orchestra.run.smithi167.stderr: 82.5% -- replaced with /var/log/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/ceph.cephadm.log.gz 2024-08-30T09:19:41.828 INFO:teuthology.orchestra.run.smithi167.stderr: 91.9% -- replaced with /var/log/ceph/cephadm.log.gz 2024-08-30T09:19:41.833 INFO:teuthology.orchestra.run.smithi190.stderr: 90.3% -- replaced with /var/log/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/ceph.audit.log.gz 2024-08-30T09:19:41.937 INFO:teuthology.orchestra.run.smithi190.stderr: 91.0% -- replaced with /var/log/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/ceph-volume.log.gz 2024-08-30T09:19:41.943 INFO:teuthology.orchestra.run.smithi042.stderr: 95.1% -- replaced with /var/log/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/ceph-mds.g.log.gz 2024-08-30T09:19:41.968 INFO:teuthology.orchestra.run.smithi167.stderr: 90.3% -- replaced with /var/log/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/ceph.audit.log.gz 2024-08-30T09:19:41.980 INFO:teuthology.orchestra.run.smithi167.stderr: 91.0% -- replaced with /var/log/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/ceph-volume.log.gz 2024-08-30T09:19:41.985 INFO:teuthology.orchestra.run.smithi167.stderr: 87.2% -- replaced with /var/log/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/ceph.log.gz 2024-08-30T09:19:42.931 INFO:teuthology.orchestra.run.smithi042.stderr: 95.1% -- replaced with /var/log/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/ceph-mds.j.log.gz 2024-08-30T09:19:43.306 INFO:teuthology.orchestra.run.smithi190.stderr: 93.8% -- replaced with /var/log/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/ceph-mds.f.log.gz 2024-08-30T09:19:43.917 INFO:teuthology.orchestra.run.smithi167.stderr: 91.5% -- replaced with /var/log/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/ceph-mon.b.log.gz 2024-08-30T09:19:44.457 INFO:teuthology.orchestra.run.smithi190.stderr: 91.5% -- replaced with /var/log/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/ceph-mon.c.log.gz 2024-08-30T09:19:44.891 INFO:teuthology.orchestra.run.smithi042.stderr: 90.1% -- replaced with /var/log/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/ceph-mgr.x.log.gz 2024-08-30T09:19:46.559 INFO:teuthology.orchestra.run.smithi190.stderr: 93.8% -- replaced with /var/log/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/ceph-mds.c.log.gz 2024-08-30T09:19:48.857 INFO:teuthology.orchestra.run.smithi167.stderr: 93.2% -- replaced with /var/log/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/ceph-mds.e.log.gz 2024-08-30T09:20:06.780 INFO:teuthology.orchestra.run.smithi190.stderr: 93.6% -- replaced with /var/log/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/ceph-mds.i.log.gz 2024-08-30T09:20:07.137 INFO:teuthology.orchestra.run.smithi042.stderr: 90.2% -- replaced with /var/log/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/ceph-mon.a.log.gz 2024-08-30T09:20:08.678 INFO:teuthology.orchestra.run.smithi167.stderr: 93.7% -- replaced with /var/log/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/ceph-mds.b.log.gz 2024-08-30T09:20:33.957 INFO:teuthology.orchestra.run.smithi167.stderr: 93.7% -- replaced with /var/log/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/ceph-mds.h.log.gz 2024-08-30T09:22:18.244 INFO:teuthology.orchestra.run.smithi167.stderr: 90.4% -- replaced with /var/log/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/ceph-osd.1.log.gz 2024-08-30T09:23:15.090 INFO:teuthology.orchestra.run.smithi167.stderr: 90.5% -- replaced with /var/log/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/ceph-osd.10.log.gz 2024-08-30T09:23:53.727 INFO:teuthology.orchestra.run.smithi190.stderr: 90.9% -- replaced with /var/log/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/ceph-osd.2.log.gz 2024-08-30T09:24:42.014 INFO:teuthology.orchestra.run.smithi167.stderr: 90.5% -- replaced with /var/log/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/ceph-osd.7.log.gz 2024-08-30T09:25:14.687 INFO:teuthology.orchestra.run.smithi167.stderr: 90.8% -- replaced with /var/log/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/ceph-osd.4.log.gz 2024-08-30T09:25:14.689 INFO:teuthology.orchestra.run.smithi167.stderr: 2024-08-30T09:25:14.689 INFO:teuthology.orchestra.run.smithi167.stderr:real 5m33.214s 2024-08-30T09:25:14.689 INFO:teuthology.orchestra.run.smithi167.stderr:user 5m30.281s 2024-08-30T09:25:14.689 INFO:teuthology.orchestra.run.smithi167.stderr:sys 0m11.977s 2024-08-30T09:26:51.423 INFO:teuthology.orchestra.run.smithi190.stderr: 90.6% -- replaced with /var/log/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/ceph-osd.8.log.gz 2024-08-30T09:27:44.433 INFO:teuthology.orchestra.run.smithi190.stderr: 90.6% -- replaced with /var/log/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/ceph-osd.11.log.gz 2024-08-30T09:28:31.974 INFO:teuthology.orchestra.run.smithi190.stderr: 90.5% -- replaced with /var/log/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/ceph-osd.5.log.gz 2024-08-30T09:28:31.975 INFO:teuthology.orchestra.run.smithi190.stderr: 2024-08-30T09:28:31.976 INFO:teuthology.orchestra.run.smithi190.stderr:real 8m50.501s 2024-08-30T09:28:31.976 INFO:teuthology.orchestra.run.smithi190.stderr:user 6m36.703s 2024-08-30T09:28:31.976 INFO:teuthology.orchestra.run.smithi190.stderr:sys 0m15.358s 2024-08-30T09:32:31.727 INFO:teuthology.orchestra.run.smithi042.stderr: 90.6% -- replaced with /var/log/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/ceph-osd.6.log.gz 2024-08-30T09:32:57.116 INFO:teuthology.orchestra.run.smithi042.stderr: 90.6% -- replaced with /var/log/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/ceph-osd.3.log.gz 2024-08-30T09:35:29.163 INFO:teuthology.orchestra.run.smithi042.stderr: 90.6% -- replaced with /var/log/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/ceph-osd.0.log.gz 2024-08-30T09:36:27.218 INFO:teuthology.orchestra.run.smithi042.stderr: 90.7% -- replaced with /var/log/ceph/b0895c1a-66ac-11ef-bcd4-c7b262605968/ceph-osd.9.log.gz 2024-08-30T09:36:27.220 INFO:teuthology.orchestra.run.smithi042.stderr: 2024-08-30T09:36:27.220 INFO:teuthology.orchestra.run.smithi042.stderr:real 16m45.751s 2024-08-30T09:36:27.220 INFO:teuthology.orchestra.run.smithi042.stderr:user 7m37.968s 2024-08-30T09:36:27.220 INFO:teuthology.orchestra.run.smithi042.stderr:sys 0m18.224s 2024-08-30T09:36:27.221 INFO:tasks.cephadm:Archiving logs... 2024-08-30T09:36:27.221 DEBUG:teuthology.misc:Transferring archived files from smithi042:/var/log/ceph to /home/teuthworker/archive/mchangir-2024-08-30_08:30:37-fs-wip-mchangir-testing-20240828.085847-main-debug-distro-default-smithi/7881205/remote/smithi042/log 2024-08-30T09:36:27.222 DEBUG:teuthology.orchestra.run.smithi042:> sudo tar c -f - -C /var/log/ceph -- . 2024-08-30T09:37:44.583 DEBUG:teuthology.misc:Transferring archived files from smithi167:/var/log/ceph to /home/teuthworker/archive/mchangir-2024-08-30_08:30:37-fs-wip-mchangir-testing-20240828.085847-main-debug-distro-default-smithi/7881205/remote/smithi167/log 2024-08-30T09:37:44.584 DEBUG:teuthology.orchestra.run.smithi167:> sudo tar c -f - -C /var/log/ceph -- . 2024-08-30T09:38:42.826 DEBUG:teuthology.misc:Transferring archived files from smithi190:/var/log/ceph to /home/teuthworker/archive/mchangir-2024-08-30_08:30:37-fs-wip-mchangir-testing-20240828.085847-main-debug-distro-default-smithi/7881205/remote/smithi190/log 2024-08-30T09:38:42.829 DEBUG:teuthology.orchestra.run.smithi190:> sudo tar c -f - -C /var/log/ceph -- . 2024-08-30T09:39:53.945 INFO:tasks.cephadm:Removing cluster... 2024-08-30T09:39:53.946 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 --force 2024-08-30T09:39:54.340 INFO:teuthology.orchestra.run.smithi042.stdout:Deleting cluster with fsid: b0895c1a-66ac-11ef-bcd4-c7b262605968 2024-08-30T09:39:55.129 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 --force 2024-08-30T09:39:55.335 INFO:teuthology.orchestra.run.smithi167.stdout:Deleting cluster with fsid: b0895c1a-66ac-11ef-bcd4-c7b262605968 2024-08-30T09:39:56.047 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid b0895c1a-66ac-11ef-bcd4-c7b262605968 --force 2024-08-30T09:39:56.265 INFO:teuthology.orchestra.run.smithi190.stdout:Deleting cluster with fsid: b0895c1a-66ac-11ef-bcd4-c7b262605968 2024-08-30T09:39:57.063 INFO:tasks.cephadm:Removing cephadm ... 2024-08-30T09:39:57.063 DEBUG:teuthology.orchestra.run.smithi042:> rm -rf /home/ubuntu/cephtest/cephadm 2024-08-30T09:39:57.082 DEBUG:teuthology.orchestra.run.smithi167:> rm -rf /home/ubuntu/cephtest/cephadm 2024-08-30T09:39:57.102 DEBUG:teuthology.orchestra.run.smithi190:> rm -rf /home/ubuntu/cephtest/cephadm 2024-08-30T09:39:57.126 INFO:tasks.cephadm:Teardown complete 2024-08-30T09:39:57.126 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-08-30T09:39:57.137 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-08-30T09:39:57.137 DEBUG:teuthology.orchestra.run.smithi042:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-08-30T09:39:57.140 DEBUG:teuthology.orchestra.run.smithi167:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-08-30T09:39:57.144 DEBUG:teuthology.orchestra.run.smithi190:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-08-30T09:39:57.514 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-08-30T09:39:57.524 INFO:teuthology.task.clock:Checking final clock skew... 2024-08-30T09:39:57.524 DEBUG:teuthology.orchestra.run.smithi042:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-08-30T09:39:57.527 DEBUG:teuthology.orchestra.run.smithi167:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-08-30T09:39:57.530 DEBUG:teuthology.orchestra.run.smithi190:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-08-30T09:39:57.542 INFO:teuthology.orchestra.run.smithi042.stderr:bash: line 1: ntpq: command not found 2024-08-30T09:39:57.544 INFO:teuthology.orchestra.run.smithi190.stderr:bash: line 1: ntpq: command not found 2024-08-30T09:39:57.544 INFO:teuthology.orchestra.run.smithi167.stderr:bash: line 1: ntpq: command not found 2024-08-30T09:39:57.561 INFO:teuthology.orchestra.run.smithi190.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-08-30T09:39:57.561 INFO:teuthology.orchestra.run.smithi190.stdout:=============================================================================== 2024-08-30T09:39:57.562 INFO:teuthology.orchestra.run.smithi190.stdout:^* hv01.front.sepia.ceph.com 3 6 377 16 -180us[ -160us] +/- 54ms 2024-08-30T09:39:57.562 INFO:teuthology.orchestra.run.smithi190.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 18 -797us[ -778us] +/- 61ms 2024-08-30T09:39:57.562 INFO:teuthology.orchestra.run.smithi190.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 19 +1516us[+1535us] +/- 89ms 2024-08-30T09:39:57.562 INFO:teuthology.orchestra.run.smithi190.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-08-30T09:39:57.567 INFO:teuthology.orchestra.run.smithi167.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-08-30T09:39:57.567 INFO:teuthology.orchestra.run.smithi167.stdout:=============================================================================== 2024-08-30T09:39:57.567 INFO:teuthology.orchestra.run.smithi167.stdout:^* hv01.front.sepia.ceph.com 3 6 377 16 -184us[ -182us] +/- 54ms 2024-08-30T09:39:57.567 INFO:teuthology.orchestra.run.smithi167.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 18 -801us[ -799us] +/- 61ms 2024-08-30T09:39:57.567 INFO:teuthology.orchestra.run.smithi167.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 17 +1519us[+1521us] +/- 89ms 2024-08-30T09:39:57.567 INFO:teuthology.orchestra.run.smithi167.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-08-30T09:39:57.569 INFO:teuthology.orchestra.run.smithi042.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-08-30T09:39:57.569 INFO:teuthology.orchestra.run.smithi042.stdout:=============================================================================== 2024-08-30T09:39:57.569 INFO:teuthology.orchestra.run.smithi042.stdout:^* hv01.front.sepia.ceph.com 3 6 377 18 -141us[ -141us] +/- 54ms 2024-08-30T09:39:57.569 INFO:teuthology.orchestra.run.smithi042.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 15 -752us[ -752us] +/- 61ms 2024-08-30T09:39:57.569 INFO:teuthology.orchestra.run.smithi042.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 19 +1560us[+1560us] +/- 89ms 2024-08-30T09:39:57.569 INFO:teuthology.orchestra.run.smithi042.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-08-30T09:39:57.570 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-08-30T09:39:57.579 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-08-30T09:39:57.580 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-08-30T09:39:57.589 DEBUG:teuthology.orchestra.run.smithi042:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-08-30T09:39:57.722 DEBUG:teuthology.orchestra.run.smithi167:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-08-30T09:39:57.801 DEBUG:teuthology.orchestra.run.smithi190:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-08-30T09:39:57.870 DEBUG:teuthology.orchestra.run.smithi042:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-08-30T09:39:57.900 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-30T09:39:57.901 DEBUG:teuthology.orchestra.run.smithi167:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-08-30T09:39:57.930 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-30T09:39:57.930 DEBUG:teuthology.orchestra.run.smithi190:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-08-30T09:39:57.957 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-30T09:39:57.958 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-08-30T09:39:57.969 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-08-30T09:39:57.979 INFO:teuthology.task.internal:Duration was 3318.496354 seconds 2024-08-30T09:39:57.979 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-08-30T09:39:57.988 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-08-30T09:39:57.989 DEBUG:teuthology.orchestra.run.smithi042:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-08-30T09:39:57.991 DEBUG:teuthology.orchestra.run.smithi167:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-08-30T09:39:57.994 DEBUG:teuthology.orchestra.run.smithi190:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-08-30T09:39:58.055 INFO:teuthology.orchestra.run.smithi167.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-08-30T09:39:58.062 INFO:teuthology.orchestra.run.smithi042.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-08-30T09:39:58.070 INFO:teuthology.orchestra.run.smithi190.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-08-30T09:39:58.436 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-08-30T09:39:58.436 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi042.front.sepia.ceph.com 2024-08-30T09:39:58.437 DEBUG:teuthology.orchestra.run.smithi042:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-08-30T09:39:58.459 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi167.front.sepia.ceph.com 2024-08-30T09:39:58.460 DEBUG:teuthology.orchestra.run.smithi167:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-08-30T09:39:58.492 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi190.front.sepia.ceph.com 2024-08-30T09:39:58.492 DEBUG:teuthology.orchestra.run.smithi190:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-08-30T09:39:58.525 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-08-30T09:39:58.525 DEBUG:teuthology.orchestra.run.smithi042:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-08-30T09:39:58.528 DEBUG:teuthology.orchestra.run.smithi167:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-08-30T09:39:58.534 DEBUG:teuthology.orchestra.run.smithi190:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-08-30T09:39:58.596 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-08-30T09:39:58.597 DEBUG:teuthology.orchestra.run.smithi042:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-08-30T09:39:58.599 DEBUG:teuthology.orchestra.run.smithi167:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-08-30T09:39:58.604 DEBUG:teuthology.orchestra.run.smithi190:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-08-30T09:39:59.741 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-08-30T09:39:59.754 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-08-30T09:39:59.754 DEBUG:teuthology.orchestra.run.smithi042:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-08-30T09:39:59.793 DEBUG:teuthology.orchestra.run.smithi167:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-08-30T09:39:59.832 DEBUG:teuthology.orchestra.run.smithi190:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-08-30T09:39:59.882 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-08-30T09:39:59.894 DEBUG:teuthology.orchestra.run.smithi042:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-08-30T09:39:59.896 DEBUG:teuthology.orchestra.run.smithi167:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-08-30T09:39:59.899 DEBUG:teuthology.orchestra.run.smithi190:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-08-30T09:39:59.928 INFO:teuthology.orchestra.run.smithi042.stdout:kernel.core_pattern = core 2024-08-30T09:39:59.934 INFO:teuthology.orchestra.run.smithi190.stdout:kernel.core_pattern = core 2024-08-30T09:39:59.935 INFO:teuthology.orchestra.run.smithi167.stdout:kernel.core_pattern = core 2024-08-30T09:39:59.973 DEBUG:teuthology.orchestra.run.smithi042:> test -e /home/ubuntu/cephtest/archive/coredump 2024-08-30T09:40:00.004 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-30T09:40:00.004 DEBUG:teuthology.orchestra.run.smithi167:> test -e /home/ubuntu/cephtest/archive/coredump 2024-08-30T09:40:00.021 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-30T09:40:00.021 DEBUG:teuthology.orchestra.run.smithi190:> test -e /home/ubuntu/cephtest/archive/coredump 2024-08-30T09:40:00.037 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-30T09:40:00.037 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-08-30T09:40:00.049 INFO:teuthology.task.internal:Transferring archived files... 2024-08-30T09:40:00.050 DEBUG:teuthology.misc:Transferring archived files from smithi042:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/mchangir-2024-08-30_08:30:37-fs-wip-mchangir-testing-20240828.085847-main-debug-distro-default-smithi/7881205/remote/smithi042 2024-08-30T09:40:00.050 DEBUG:teuthology.orchestra.run.smithi042:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-08-30T09:40:00.108 DEBUG:teuthology.misc:Transferring archived files from smithi167:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/mchangir-2024-08-30_08:30:37-fs-wip-mchangir-testing-20240828.085847-main-debug-distro-default-smithi/7881205/remote/smithi167 2024-08-30T09:40:00.108 DEBUG:teuthology.orchestra.run.smithi167:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-08-30T09:40:00.145 DEBUG:teuthology.misc:Transferring archived files from smithi190:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/mchangir-2024-08-30_08:30:37-fs-wip-mchangir-testing-20240828.085847-main-debug-distro-default-smithi/7881205/remote/smithi190 2024-08-30T09:40:00.145 DEBUG:teuthology.orchestra.run.smithi190:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-08-30T09:40:00.181 INFO:teuthology.task.internal:Removing archive directory... 2024-08-30T09:40:00.182 DEBUG:teuthology.orchestra.run.smithi042:> rm -rf -- /home/ubuntu/cephtest/archive 2024-08-30T09:40:00.185 DEBUG:teuthology.orchestra.run.smithi167:> rm -rf -- /home/ubuntu/cephtest/archive 2024-08-30T09:40:00.187 DEBUG:teuthology.orchestra.run.smithi190:> rm -rf -- /home/ubuntu/cephtest/archive 2024-08-30T09:40:00.233 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-08-30T09:40:00.246 INFO:teuthology.task.internal:Not uploading archives. 2024-08-30T09:40:00.246 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-08-30T09:40:00.256 INFO:teuthology.task.internal:Tidying up after the test... 2024-08-30T09:40:00.256 DEBUG:teuthology.orchestra.run.smithi042:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-08-30T09:40:00.259 DEBUG:teuthology.orchestra.run.smithi167:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-08-30T09:40:00.261 DEBUG:teuthology.orchestra.run.smithi190:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-08-30T09:40:00.271 INFO:teuthology.orchestra.run.smithi042.stdout: 262217 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Aug 30 09:40 /home/ubuntu/cephtest 2024-08-30T09:40:00.274 INFO:teuthology.orchestra.run.smithi167.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Aug 30 09:40 /home/ubuntu/cephtest 2024-08-30T09:40:00.289 INFO:teuthology.orchestra.run.smithi190.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Aug 30 09:40 /home/ubuntu/cephtest 2024-08-30T09:40:00.300 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-08-30T09:40:00.312 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-08-30T09:40:00.339 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/secure wsync/no} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-quiesce/no 6-workunit/postgres}} duration: 3318.496353626251 failure_reason: '"2024-08-30T09:03:16.440611+0000 mds.b (mds.0) 19 : cluster [WRN] Scrub error on inode 0x1000000000f (/volumes/qa/sv_1/d3168875-d2e5-410d-8754-72d29955777b/postgres/data/pg_subtrans) see mds.b log and `damage ls` output for details" in cluster log' flavor: default owner: scheduled_mchangir@teuthology success: false 2024-08-30T09:40:00.339 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-08-30T09:40:00.441 INFO:teuthology.run:FAIL